2021-02-10T11:01:18.471 INFO:root:teuthology version: 1.1.0-14fad317 2021-02-10T11:01:18.472 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name khiremat-2021-02-10_10:13:45-fs-wip-khiremat-mgr-authid-bp-7-nautilus-distro-basic-gibba --description fs/basic_functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{rhel_7} tasks/volumes} --owner scheduled_khiremat@teuthology --archive /home/teuthworker/archive/khiremat-2021-02-10_10:13:45-fs-wip-khiremat-mgr-authid-bp-7-nautilus-distro-basic-gibba/5872719 -- /home/teuthworker/archive/khiremat-2021-02-10_10:13:45-fs-wip-khiremat-mgr-authid-bp-7-nautilus-distro-basic-gibba/5872719/orig.config.yaml 2021-02-10T11:01:18.530 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2021-02-10T11:01:18.597 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/khiremat-2021-02-10_10:13:45-fs-wip-khiremat-mgr-authid-bp-7-nautilus-distro-basic-gibba/5872719 branch: wip-khiremat-mgr-authid-bp-7-nautilus description: fs/basic_functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{rhel_7} tasks/volumes} email: khiremat@redhat.com first_in_suite: false job_id: '5872719' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: gibba name: khiremat-2021-02-10_10:13:45-fs-wip-khiremat-mgr-authid-bp-7-nautilus-distro-basic-gibba nuke-on-error: true openstack: - volumes: count: 4 size: 30 - machine: disk: 200 os_type: rhel os_version: '7.9' overrides: admin_socket: branch: wip-khiremat-mgr-authid-bp-7-nautilus ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' mds: debug mds: 20 debug ms: 1 mds bal frag: true mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 mgr: debug client: 10 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(TOO_FEW_PGS\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 962fac13b1a781b8c7d25c1eed50d93813b8f5f0 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: sha1: 962fac13b1a781b8c7d25c1eed50d93813b8f5f0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-khiremat-mgr-authid-bp-7-nautilus sha1: 962fac13b1a781b8c7d25c1eed50d93813b8f5f0 owner: scheduled_khiremat@teuthology priority: 50 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.1 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 - client.2 - client.3 sha1: 962fac13b1a781b8c7d25c1eed50d93813b8f5f0 sleep_before_teardown: 0 suite: fs suite_branch: wip-khiremat-mgr-authid-bp-7-nautilus suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_962fac13b1a781b8c7d25c1eed50d93813b8f5f0/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 962fac13b1a781b8c7d25c1eed50d93813b8f5f0 targets: gibba001.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCUYH0PUZVukSVO9/5yIUBH59HFWSUrEpb7DzfyPkTBTyxU56GwMHZKC4xI9KebrEXDXCAOhT1M3+BEsG1iWZTEV7tCwjBSsRImzspodarlIUJ1lIFayszBAjqS5eAT/G92CIHrkMNY1dmYR24X7hYo+VD2aHNE01Z7ZVbY6WLybagqGUACa1PLmmAd7kW+Aal1a/f70ZDUf6xTuaOZjBgY7ajWFFpwZZvDu3uzCkiadVn6ACugS9unhoOa62BwXIv0i9KfKkQOzWiBZwkX5MDK8ldtDzWiX+sYvVj3N9J0rbyOFDv4E78o9Zwb30S4e7/w/Az6rHrGbqJwYDIcyDdb gibba008.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5wOFVXcmGwExgtBNqJLFpE6v+rZ2LpWLmVDEnU+vqKAWY2NJAQB8234cc6aGJ8HvkxePvVeKnU39ho+M2ssdf9tOogE3vT6CgDEiqluaFc9ctL1eF+Sl1yLBqqSyiu3FAANHtFCHZ+RWfVk4A4a/bNqNXD2K/KbJuhgyNhj+EVew+MakE1iH0bpexJk3lQjc0Tq2VPyC4xoJ1WHxS7zR9y883SaXfEd/CZlbLCSn4t7RRXsoQ6ovsyx+oIaVMMTIQVsZ4ZVIee/2wMOmWrozI+S/15y9iXm9vC6404aP3MnnBSbKW/l0M4ZnyNV6lDkmgc8kQ/FL/r2SkPfnEWUZl tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell rpm: - python3-cephfs extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - ceph: null - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes teuthology_branch: master timestamp: 2021-02-10_10:13:45 tube: gibba user: khiremat verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.gibba.29737 2021-02-10T11:01:18.597 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_962fac13b1a781b8c7d25c1eed50d93813b8f5f0/qa; will attempt to use it 2021-02-10T11:01:18.606 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_962fac13b1a781b8c7d25c1eed50d93813b8f5f0/qa/tasks 2021-02-10T11:01:18.606 INFO:teuthology.run_tasks:Running task internal.check_packages... 2021-02-10T11:01:18.608 INFO:teuthology.task.internal:Checking packages... 2021-02-10T11:01:18.666 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'basic' and ceph hash '962fac13b1a781b8c7d25c1eed50d93813b8f5f0' 2021-02-10T11:01:18.666 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2021-02-10T11:01:18.666 INFO:teuthology.packaging:ref: None 2021-02-10T11:01:18.666 INFO:teuthology.packaging:tag: None 2021-02-10T11:01:18.666 INFO:teuthology.packaging:branch: wip-khiremat-mgr-authid-bp-7-nautilus 2021-02-10T11:01:18.666 INFO:teuthology.packaging:sha1: 962fac13b1a781b8c7d25c1eed50d93813b8f5f0 2021-02-10T11:01:18.666 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F7%2Fx86_64&ref=wip-khiremat-mgr-authid-bp-7-nautilus 2021-02-10T11:01:18.840 INFO:teuthology.task.internal:Found packages for ceph version 14.2.16-280.g962fac13 2021-02-10T11:01:18.840 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2021-02-10T11:01:18.851 INFO:teuthology.task.internal:no buildpackages task found 2021-02-10T11:01:18.851 INFO:teuthology.run_tasks:Running task internal.save_config... 2021-02-10T11:01:18.902 INFO:teuthology.task.internal:Saving configuration 2021-02-10T11:01:18.918 INFO:teuthology.run_tasks:Running task internal.check_lock... 2021-02-10T11:01:18.928 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-10T11:01:18.947 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-02-10 10:56:34.847857', 'description': '/home/teuthworker/archive/khiremat-2021-02-10_10:13:45-fs-wip-khiremat-mgr-authid-bp-7-nautilus-distro-basic-gibba/5872719', 'locked_by': 'scheduled_khiremat@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKaEpzbIWxno5kWcHghDfj+g2Gj4rNqMhr/iFqJvbSFCh9O4G+k6WZhmdf2gsthU4u0+L16DD2eN3lnkpQwpz6Y=', 'os_version': '7.9', 'machine_type': 'gibba', 'vm_host': None, 'os_type': 'rhel', 'arch': 'x86_64', 'mac_address': None, 'name': 'gibba001.front.sepia.ceph.com'} 2021-02-10T11:01:18.965 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-02-10 10:56:34.847875', 'description': '/home/teuthworker/archive/khiremat-2021-02-10_10:13:45-fs-wip-khiremat-mgr-authid-bp-7-nautilus-distro-basic-gibba/5872719', 'locked_by': 'scheduled_khiremat@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBOkB2GJOWl7AJst38YmblgdF/TuZefQKQSj+GLBQ3gthm2sbwCGQ2R0YapQHCabwQaHKd0Zd48yWroCaxH9bdI=', 'os_version': '7.9', 'machine_type': 'gibba', 'vm_host': None, 'os_type': 'rhel', 'arch': 'x86_64', 'mac_address': None, 'name': 'gibba008.front.sepia.ceph.com'} 2021-02-10T11:01:18.966 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2021-02-10T11:01:18.978 INFO:teuthology.task.internal:roles: ubuntu@gibba001.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.1'] 2021-02-10T11:01:18.978 INFO:teuthology.task.internal:roles: ubuntu@gibba008.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.2', 'client.3'] 2021-02-10T11:01:18.978 INFO:teuthology.run_tasks:Running task console_log... 2021-02-10T11:01:19.044 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9c4a517620>, signals=[15]) 2021-02-10T11:01:19.044 INFO:teuthology.run_tasks:Running task internal.connect... 2021-02-10T11:01:19.055 INFO:teuthology.task.internal:Opening connections... 2021-02-10T11:01:19.055 DEBUG:teuthology.task.internal:connecting to ubuntu@gibba001.front.sepia.ceph.com 2021-02-10T11:01:19.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-10T11:01:19.131 DEBUG:teuthology.task.internal:connecting to ubuntu@gibba008.front.sepia.ceph.com 2021-02-10T11:01:19.132 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-10T11:01:19.197 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2021-02-10T11:01:19.208 DEBUG:teuthology.orchestra.run.gibba001:> uname -m 2021-02-10T11:01:19.586 INFO:teuthology.orchestra.run.gibba001.stdout:x86_64 2021-02-10T11:01:19.586 DEBUG:teuthology.orchestra.run.gibba001:> cat /etc/os-release 2021-02-10T11:01:19.921 INFO:teuthology.orchestra.run.gibba001.stdout:NAME="Red Hat Enterprise Linux Server" 2021-02-10T11:01:19.922 INFO:teuthology.orchestra.run.gibba001.stdout:VERSION="7.9 (Maipo)" 2021-02-10T11:01:19.922 INFO:teuthology.orchestra.run.gibba001.stdout:ID="rhel" 2021-02-10T11:01:19.922 INFO:teuthology.orchestra.run.gibba001.stdout:ID_LIKE="fedora" 2021-02-10T11:01:19.922 INFO:teuthology.orchestra.run.gibba001.stdout:VARIANT="Server" 2021-02-10T11:01:19.922 INFO:teuthology.orchestra.run.gibba001.stdout:VARIANT_ID="server" 2021-02-10T11:01:19.922 INFO:teuthology.orchestra.run.gibba001.stdout:VERSION_ID="7.9" 2021-02-10T11:01:19.922 INFO:teuthology.orchestra.run.gibba001.stdout:PRETTY_NAME="Employee SKU" 2021-02-10T11:01:19.922 INFO:teuthology.orchestra.run.gibba001.stdout:ANSI_COLOR="0;31" 2021-02-10T11:01:19.922 INFO:teuthology.orchestra.run.gibba001.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:7.9:GA:server" 2021-02-10T11:01:19.922 INFO:teuthology.orchestra.run.gibba001.stdout:HOME_URL="https://www.redhat.com/" 2021-02-10T11:01:19.923 INFO:teuthology.orchestra.run.gibba001.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-02-10T11:01:19.923 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:01:19.923 INFO:teuthology.orchestra.run.gibba001.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7" 2021-02-10T11:01:19.923 INFO:teuthology.orchestra.run.gibba001.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=7.9 2021-02-10T11:01:19.923 INFO:teuthology.orchestra.run.gibba001.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-02-10T11:01:19.923 INFO:teuthology.orchestra.run.gibba001.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7.9" 2021-02-10T11:01:19.924 INFO:teuthology.lock.ops:Updating gibba001.front.sepia.ceph.com on lock server 2021-02-10T11:01:19.939 DEBUG:teuthology.orchestra.run.gibba008:> uname -m 2021-02-10T11:01:20.289 INFO:teuthology.orchestra.run.gibba008.stdout:x86_64 2021-02-10T11:01:20.289 DEBUG:teuthology.orchestra.run.gibba008:> cat /etc/os-release 2021-02-10T11:01:20.623 INFO:teuthology.orchestra.run.gibba008.stdout:NAME="Red Hat Enterprise Linux Server" 2021-02-10T11:01:20.623 INFO:teuthology.orchestra.run.gibba008.stdout:VERSION="7.9 (Maipo)" 2021-02-10T11:01:20.624 INFO:teuthology.orchestra.run.gibba008.stdout:ID="rhel" 2021-02-10T11:01:20.624 INFO:teuthology.orchestra.run.gibba008.stdout:ID_LIKE="fedora" 2021-02-10T11:01:20.624 INFO:teuthology.orchestra.run.gibba008.stdout:VARIANT="Server" 2021-02-10T11:01:20.624 INFO:teuthology.orchestra.run.gibba008.stdout:VARIANT_ID="server" 2021-02-10T11:01:20.624 INFO:teuthology.orchestra.run.gibba008.stdout:VERSION_ID="7.9" 2021-02-10T11:01:20.624 INFO:teuthology.orchestra.run.gibba008.stdout:PRETTY_NAME="Employee SKU" 2021-02-10T11:01:20.624 INFO:teuthology.orchestra.run.gibba008.stdout:ANSI_COLOR="0;31" 2021-02-10T11:01:20.624 INFO:teuthology.orchestra.run.gibba008.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:7.9:GA:server" 2021-02-10T11:01:20.624 INFO:teuthology.orchestra.run.gibba008.stdout:HOME_URL="https://www.redhat.com/" 2021-02-10T11:01:20.624 INFO:teuthology.orchestra.run.gibba008.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-02-10T11:01:20.624 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:01:20.624 INFO:teuthology.orchestra.run.gibba008.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7" 2021-02-10T11:01:20.625 INFO:teuthology.orchestra.run.gibba008.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=7.9 2021-02-10T11:01:20.625 INFO:teuthology.orchestra.run.gibba008.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-02-10T11:01:20.625 INFO:teuthology.orchestra.run.gibba008.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7.9" 2021-02-10T11:01:20.625 INFO:teuthology.lock.ops:Updating gibba008.front.sepia.ceph.com on lock server 2021-02-10T11:01:20.640 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2021-02-10T11:01:20.654 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2021-02-10T11:01:20.664 INFO:teuthology.task.internal:Checking for old test directory... 2021-02-10T11:01:20.664 DEBUG:teuthology.orchestra.run.gibba001:> test '!' -e /home/ubuntu/cephtest 2021-02-10T11:01:20.669 DEBUG:teuthology.orchestra.run.gibba008:> test '!' -e /home/ubuntu/cephtest 2021-02-10T11:01:21.002 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2021-02-10T11:01:21.013 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2021-02-10T11:01:21.013 DEBUG:teuthology.orchestra.run.gibba001:> test -z $(ls -A /var/lib/ceph) 2021-02-10T11:01:21.016 DEBUG:teuthology.orchestra.run.gibba008:> test -z $(ls -A /var/lib/ceph) 2021-02-10T11:01:21.378 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2021-02-10T11:01:21.424 INFO:teuthology.run_tasks:Running task kernel... 2021-02-10T11:01:21.444 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.y': {'kdb': True, 'sha1': 'distro'}, 'mds.a': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}, 'osd.3': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'mds.b': {'kdb': True, 'sha1': 'distro'}, 'osd.4': {'kdb': True, 'sha1': 'distro'}, 'osd.5': {'kdb': True, 'sha1': 'distro'}, 'osd.6': {'kdb': True, 'sha1': 'distro'}, 'osd.7': {'kdb': True, 'sha1': 'distro'}, 'client.2': {'kdb': True, 'sha1': 'distro'}, 'client.3': {'kdb': True, 'sha1': 'distro'}} 2021-02-10T11:01:21.444 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2021-02-10T11:01:21.444 DEBUG:teuthology.orchestra.run.gibba001:> uname -r 2021-02-10T11:01:21.779 INFO:teuthology.orchestra.run.gibba001.stdout:3.10.0-1160.6.1.el7.x86_64 2021-02-10T11:01:21.779 INFO:teuthology.task.kernel:Running kernel on gibba001: 3.10.0-1160.6.1.el7.x86_64 2021-02-10T11:01:21.779 DEBUG:teuthology.orchestra.run.gibba001:> sudo yum install -y kernel 2021-02-10T11:01:23.044 INFO:teuthology.orchestra.run.gibba001.stdout:Loaded plugins: fastestmirror, langpacks, priorities, product-id, search- 2021-02-10T11:01:23.044 INFO:teuthology.orchestra.run.gibba001.stdout: : disabled-repos, subscription-manager 2021-02-10T11:01:23.198 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:01:23.198 INFO:teuthology.orchestra.run.gibba001.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2021-02-10T11:01:23.198 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:01:23.226 INFO:teuthology.orchestra.run.gibba001.stdout:Determining fastest mirrors 2021-02-10T11:01:23.310 INFO:teuthology.orchestra.run.gibba001.stdout: * epel: download-cc-rdu01.fedoraproject.org 2021-02-10T11:01:24.250 INFO:teuthology.orchestra.run.gibba001.stdout:Nothing to do 2021-02-10T11:01:24.264 DEBUG:teuthology.orchestra.run.gibba001:> echo no | sudo yum reinstall kernel || true 2021-02-10T11:01:24.715 INFO:teuthology.orchestra.run.gibba001.stdout:Loaded plugins: fastestmirror, langpacks, priorities, product-id, search- 2021-02-10T11:01:24.715 INFO:teuthology.orchestra.run.gibba001.stdout: : disabled-repos, subscription-manager 2021-02-10T11:01:24.768 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:01:24.768 INFO:teuthology.orchestra.run.gibba001.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2021-02-10T11:01:24.768 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:01:24.828 INFO:teuthology.orchestra.run.gibba001.stderr:Skipping the running kernel: kernel-3.10.0-1160.6.1.el7.x86_64 2021-02-10T11:01:24.850 INFO:teuthology.orchestra.run.gibba001.stdout:Loading mirror speeds from cached hostfile 2021-02-10T11:01:24.850 INFO:teuthology.orchestra.run.gibba001.stdout: * epel: download-cc-rdu01.fedoraproject.org 2021-02-10T11:01:24.853 INFO:teuthology.orchestra.run.gibba001.stdout:Installed package kernel-3.10.0-1160.el7.x86_64 (from anaconda) not available. 2021-02-10T11:01:24.853 INFO:teuthology.orchestra.run.gibba001.stderr:Error: Nothing to do 2021-02-10T11:01:24.865 DEBUG:teuthology.orchestra.run.gibba001:> rpm -q kernel --last | head -n 1 2021-02-10T11:01:25.320 INFO:teuthology.orchestra.run.gibba001.stdout:kernel-3.10.0-1160.6.1.el7.x86_64 Wed 11 Nov 2020 03:14:13 PM UTC 2021-02-10T11:01:25.321 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1160.6.1.el7.x86_64 2021-02-10T11:01:25.321 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2021-02-10T11:01:25.321 DEBUG:teuthology.orchestra.run.gibba008:> uname -r 2021-02-10T11:01:25.654 INFO:teuthology.orchestra.run.gibba008.stdout:3.10.0-1160.6.1.el7.x86_64 2021-02-10T11:01:25.655 INFO:teuthology.task.kernel:Running kernel on gibba008: 3.10.0-1160.6.1.el7.x86_64 2021-02-10T11:01:25.655 DEBUG:teuthology.orchestra.run.gibba008:> sudo yum install -y kernel 2021-02-10T11:01:26.864 INFO:teuthology.orchestra.run.gibba008.stdout:Loaded plugins: fastestmirror, langpacks, priorities, product-id, search- 2021-02-10T11:01:26.948 INFO:teuthology.orchestra.run.gibba008.stdout: : disabled-repos, subscription-manager 2021-02-10T11:01:27.069 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:01:27.069 INFO:teuthology.orchestra.run.gibba008.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2021-02-10T11:01:27.069 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:01:27.103 INFO:teuthology.orchestra.run.gibba008.stdout:Determining fastest mirrors 2021-02-10T11:01:27.192 INFO:teuthology.orchestra.run.gibba008.stdout: * epel: download-cc-rdu01.fedoraproject.org 2021-02-10T11:01:28.123 INFO:teuthology.orchestra.run.gibba008.stdout:Nothing to do 2021-02-10T11:01:28.136 DEBUG:teuthology.orchestra.run.gibba008:> echo no | sudo yum reinstall kernel || true 2021-02-10T11:01:28.589 INFO:teuthology.orchestra.run.gibba008.stdout:Loaded plugins: fastestmirror, langpacks, priorities, product-id, search- 2021-02-10T11:01:28.589 INFO:teuthology.orchestra.run.gibba008.stdout: : disabled-repos, subscription-manager 2021-02-10T11:01:28.645 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:01:28.646 INFO:teuthology.orchestra.run.gibba008.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2021-02-10T11:01:28.646 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:01:28.704 INFO:teuthology.orchestra.run.gibba008.stderr:Skipping the running kernel: kernel-3.10.0-1160.6.1.el7.x86_64 2021-02-10T11:01:28.720 INFO:teuthology.orchestra.run.gibba008.stdout:Loading mirror speeds from cached hostfile 2021-02-10T11:01:28.720 INFO:teuthology.orchestra.run.gibba008.stdout: * epel: download-cc-rdu01.fedoraproject.org 2021-02-10T11:01:28.723 INFO:teuthology.orchestra.run.gibba008.stdout:Installed package kernel-3.10.0-1160.el7.x86_64 (from anaconda) not available. 2021-02-10T11:01:28.724 INFO:teuthology.orchestra.run.gibba008.stderr:Error: Nothing to do 2021-02-10T11:01:28.735 DEBUG:teuthology.orchestra.run.gibba008:> rpm -q kernel --last | head -n 1 2021-02-10T11:01:29.182 INFO:teuthology.orchestra.run.gibba008.stdout:kernel-3.10.0-1160.6.1.el7.x86_64 Wed 11 Nov 2020 03:14:13 PM UTC 2021-02-10T11:01:29.183 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1160.6.1.el7.x86_64 2021-02-10T11:01:29.183 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2021-02-10T11:01:29.183 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2021-02-10T11:01:29.184 DEBUG:teuthology.orchestra.run.gibba001:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-02-10T11:01:29.555 INFO:teuthology.orchestra.run.gibba001.stdout:ttyS1 2021-02-10T11:01:29.568 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2021-02-10T11:01:29.568 DEBUG:teuthology.orchestra.run.gibba008:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-02-10T11:01:29.916 INFO:teuthology.orchestra.run.gibba008.stdout:ttyS1 2021-02-10T11:01:29.931 INFO:teuthology.run_tasks:Running task internal.base... 2021-02-10T11:01:29.939 INFO:teuthology.task.internal:Creating test directory... 2021-02-10T11:01:29.939 DEBUG:teuthology.orchestra.run.gibba001:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-02-10T11:01:29.947 DEBUG:teuthology.orchestra.run.gibba008:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-02-10T11:01:30.283 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2021-02-10T11:01:30.290 INFO:teuthology.run_tasks:Running task internal.archive... 2021-02-10T11:01:30.329 INFO:teuthology.task.internal:Creating archive directory... 2021-02-10T11:01:30.330 DEBUG:teuthology.orchestra.run.gibba001:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-02-10T11:01:30.336 DEBUG:teuthology.orchestra.run.gibba008:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-02-10T11:01:30.683 INFO:teuthology.run_tasks:Running task internal.coredump... 2021-02-10T11:01:30.690 INFO:teuthology.task.internal:Enabling coredump saving... 2021-02-10T11:01:30.691 DEBUG:teuthology.orchestra.run.gibba001:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-02-10T11:01:30.694 DEBUG:teuthology.orchestra.run.gibba008:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-02-10T11:01:31.035 INFO:teuthology.orchestra.run.gibba001.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-02-10T11:01:31.037 INFO:teuthology.orchestra.run.gibba008.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-02-10T11:01:31.045 INFO:teuthology.orchestra.run.gibba001.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-02-10T11:01:31.047 INFO:teuthology.orchestra.run.gibba008.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-02-10T11:01:31.048 INFO:teuthology.run_tasks:Running task internal.sudo... 2021-02-10T11:01:31.056 INFO:teuthology.task.internal:Configuring sudo... 2021-02-10T11:01:31.056 DEBUG:teuthology.orchestra.run.gibba001:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-02-10T11:01:31.060 DEBUG:teuthology.orchestra.run.gibba008:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-02-10T11:01:31.403 INFO:teuthology.run_tasks:Running task internal.syslog... 2021-02-10T11:01:31.414 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2021-02-10T11:01:31.414 DEBUG:teuthology.orchestra.run.gibba001:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-02-10T11:01:31.417 DEBUG:teuthology.orchestra.run.gibba008:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-02-10T11:01:31.752 DEBUG:teuthology.orchestra.run.gibba001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-02-10T11:01:32.106 DEBUG:teuthology.orchestra.run.gibba001:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2021-02-10T11:01:32.451 DEBUG:teuthology.orchestra.run.gibba001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-02-10T11:01:32.805 DEBUG:teuthology.orchestra.run.gibba001:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2021-02-10T11:01:33.151 DEBUG:teuthology.orchestra.run.gibba001:> set -ex 2021-02-10T11:01:33.151 DEBUG:teuthology.orchestra.run.gibba001:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-02-10T11:01:33.494 DEBUG:teuthology.orchestra.run.gibba008:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-02-10T11:01:33.851 DEBUG:teuthology.orchestra.run.gibba008:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2021-02-10T11:01:34.197 DEBUG:teuthology.orchestra.run.gibba008:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-02-10T11:01:34.548 DEBUG:teuthology.orchestra.run.gibba008:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2021-02-10T11:01:34.894 DEBUG:teuthology.orchestra.run.gibba008:> set -ex 2021-02-10T11:01:34.894 DEBUG:teuthology.orchestra.run.gibba008:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-02-10T11:01:35.236 DEBUG:teuthology.orchestra.run.gibba001:> sudo service rsyslog restart 2021-02-10T11:01:35.239 DEBUG:teuthology.orchestra.run.gibba008:> sudo service rsyslog restart 2021-02-10T11:01:35.601 INFO:teuthology.orchestra.run.gibba001.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2021-02-10T11:01:35.612 INFO:teuthology.orchestra.run.gibba008.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2021-02-10T11:01:35.647 INFO:teuthology.run_tasks:Running task internal.timer... 2021-02-10T11:01:35.656 INFO:teuthology.task.internal:Starting timer... 2021-02-10T11:01:35.656 INFO:teuthology.run_tasks:Running task pcp... 2021-02-10T11:01:35.680 INFO:teuthology.run_tasks:Running task selinux... 2021-02-10T11:01:35.692 DEBUG:teuthology.orchestra.run.gibba001:> sudo service auditd rotate 2021-02-10T11:01:36.066 INFO:teuthology.orchestra.run.gibba001.stdout:Rotating logs: [ OK ] 2021-02-10T11:01:36.067 DEBUG:teuthology.orchestra.run.gibba008:> sudo service auditd rotate 2021-02-10T11:01:36.439 INFO:teuthology.orchestra.run.gibba008.stdout:Rotating logs: [ OK ] 2021-02-10T11:01:36.440 DEBUG:teuthology.task.selinux:Getting current SELinux state 2021-02-10T11:01:36.440 DEBUG:teuthology.orchestra.run.gibba001:> /usr/sbin/getenforce 2021-02-10T11:01:36.783 INFO:teuthology.orchestra.run.gibba001.stdout:Permissive 2021-02-10T11:01:36.783 DEBUG:teuthology.orchestra.run.gibba008:> /usr/sbin/getenforce 2021-02-10T11:01:37.127 INFO:teuthology.orchestra.run.gibba008.stdout:Permissive 2021-02-10T11:01:37.128 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@gibba001.front.sepia.ceph.com': 'permissive', 'ubuntu@gibba008.front.sepia.ceph.com': 'permissive'} 2021-02-10T11:01:37.128 DEBUG:teuthology.orchestra.run.gibba001:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\)' 2021-02-10T11:01:37.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:01:37.474 DEBUG:teuthology.orchestra.run.gibba008:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\)' 2021-02-10T11:01:37.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:01:37.820 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2021-02-10T11:01:37.820 DEBUG:teuthology.orchestra.run.gibba001:> sudo /usr/sbin/setenforce permissive 2021-02-10T11:01:38.164 DEBUG:teuthology.orchestra.run.gibba008:> sudo /usr/sbin/setenforce permissive 2021-02-10T11:01:38.513 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2021-02-10T11:01:38.525 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated or references a specific commit; assuming it is current 2021-02-10T11:01:38.525 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to origin/master 2021-02-10T11:01:38.738 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2021-02-10T11:01:38.742 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit gibba001.front.sepia.ceph.com,gibba008.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2021-02-10T11:01:39.507 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2021-02-10T11:01:41.494 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2021-02-10T11:01:44.533 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2021-02-10T11:01:44.584 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:44 +0000 (0:00:00.162) 0:00:00.162 **** 2021-02-10T11:01:44.603 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:44 +0000 (0:00:00.019) 0:00:00.182 **** 2021-02-10T11:01:47.384 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [gibba008.front.sepia.ceph.com] 2021-02-10T11:01:47.453 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:47 +0000 (0:00:02.850) 0:00:03.032 **** 2021-02-10T11:01:48.041 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2021-02-10T11:01:48.043 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2021-02-10T11:01:48.075 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:48 +0000 (0:00:00.621) 0:00:03.653 **** 2021-02-10T11:01:48.325 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [gibba001.front.sepia.ceph.com] 2021-02-10T11:01:48.399 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:48 +0000 (0:00:00.324) 0:00:03.977 **** 2021-02-10T11:01:48.760 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2021-02-10T11:01:48.761 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-02-10T11:01:48.798 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:48 +0000 (0:00:00.399) 0:00:04.377 **** 2021-02-10T11:01:48.825 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2021-02-10T11:01:48.827 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:01:48.860 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:48 +0000 (0:00:00.061) 0:00:04.439 **** 2021-02-10T11:01:48.866 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2021-02-10T11:01:48.866 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2021-02-10T11:01:48.903 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:48 +0000 (0:00:00.043) 0:00:04.482 **** 2021-02-10T11:01:49.406 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2021-02-10T11:01:49.407 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-02-10T11:01:49.438 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:49 +0000 (0:00:00.534) 0:00:05.017 **** 2021-02-10T11:01:49.444 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2021-02-10T11:01:49.444 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-10T11:01:49.479 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:49 +0000 (0:00:00.041) 0:00:05.058 **** 2021-02-10T11:01:49.485 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2021-02-10T11:01:49.486 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:01:49.524 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:49 +0000 (0:00:00.044) 0:00:05.103 **** 2021-02-10T11:01:49.969 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2021-02-10T11:01:49.970 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-10T11:01:50.001 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:49 +0000 (0:00:00.477) 0:00:05.580 **** 2021-02-10T11:01:51.809 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2021-02-10T11:01:51.810 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/cephlab_sudo", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "state": "file", "uid": 0} 2021-02-10T11:01:51.844 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:51 +0000 (0:00:01.842) 0:00:07.422 **** 2021-02-10T11:01:51.913 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/cephlab_sudo", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "state": "file", "uid": 0} 2021-02-10T11:01:51.947 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:51 +0000 (0:00:00.102) 0:00:07.525 **** 2021-02-10T11:01:52.460 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2021-02-10T11:01:52.461 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4 2021-02-10T11:01:52.462 INFO:teuthology.task.ansible.out:r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-02-10T11:01:52.494 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4 2021-02-10T11:01:52.495 INFO:teuthology.task.ansible.out:r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-02-10T11:01:52.601 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2021-02-10T11:01:52.609 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2021-02-10T11:01:52.755 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:52 +0000 (0:00:00.807) 0:00:08.333 **** 2021-02-10T11:01:52.875 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:52 +0000 (0:00:00.033) 0:00:08.367 **** 2021-02-10T11:01:53.879 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [gibba008.front.sepia.ceph.com] 2021-02-10T11:01:53.953 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2021-02-10T11:01:53.955 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:53 +0000 (0:00:01.165) 0:00:09.532 **** 2021-02-10T11:01:54.644 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2021-02-10T11:01:54.645 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.306020", "end": "2021-02-10 11:01:54.616117", "rc": 0, "start": "2021-02-10 11:01:54.310097", "stderr": "", "stderr_lines": [], "stdout": "df8946f08e870e2d2565b79099289ca48bd825f3", "stdout_lines": ["df8946f08e870e2d2565b79099289ca48bd825f3"]} 2021-02-10T11:01:54.676 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:54 +0000 (0:00:00.722) 0:00:10.255 **** 2021-02-10T11:01:54.841 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [gibba001.front.sepia.ceph.com] 2021-02-10T11:01:54.919 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:54 +0000 (0:00:00.242) 0:00:10.497 **** 2021-02-10T11:01:55.173 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.009211", "end": "2021-02-10 11:01:55.154391", "failed_when_result": false, "rc": 0, "start": "2021-02-10 11:01:55.145180", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2021-02-10T11:01:55.213 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:55 +0000 (0:00:00.294) 0:00:10.791 **** 2021-02-10T11:01:55.241 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2021-02-10T11:01:55.243 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:01:55.288 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:55 +0000 (0:00:00.075) 0:00:10.867 **** 2021-02-10T11:01:55.318 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2021-02-10T11:01:55.319 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.194587", "end": "2021-02-10 11:01:55.282052", "rc": 0, "start": "2021-02-10 11:01:55.087465", "stderr": "", "stderr_lines": [], "stdout": "df8946f08e870e2d2565b79099289ca48bd825f3", "stdout_lines": ["df8946f08e870e2d2565b79099289ca48bd825f3"]} TASK [users : Merge extra_admin_users into managed_admin_users] **************** skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:01:55.358 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:55 +0000 (0:00:00.070) 0:00:10.937 **** 2021-02-10T11:01:55.400 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:55 +0000 (0:00:00.041) 0:00:10.978 **** 2021-02-10T11:01:55.429 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2021-02-10T11:01:55.431 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:01:55.469 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:55 +0000 (0:00:00.070) 0:00:11.048 **** 2021-02-10T11:01:55.497 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2021-02-10T11:01:55.498 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:01:55.541 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:55 +0000 (0:00:00.071) 0:00:11.120 **** 2021-02-10T11:01:55.579 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2021-02-10T11:01:55.581 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:01:55.620 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:55 +0000 (0:00:00.078) 0:00:11.199 **** 2021-02-10T11:01:55.811 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2021-02-10T11:01:55.811 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.006755", "end": "2021-02-10 11:01:55.794089", "failed_when_result": false, "rc": 0, "start": "2021-02-10 11:01:55.787334", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2021-02-10T11:01:55.850 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:55 +0000 (0:00:00.229) 0:00:11.429 **** 2021-02-10T11:01:55.878 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2021-02-10T11:01:55.880 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:01:55.919 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:55 +0000 (0:00:00.068) 0:00:11.497 **** 2021-02-10T11:01:55.951 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2021-02-10T11:01:55.952 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:01:55.988 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:55 +0000 (0:00:00.069) 0:00:11.567 **** 2021-02-10T11:01:56.018 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2021-02-10T11:01:56.019 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:01:56.064 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:56 +0000 (0:00:00.075) 0:00:11.642 **** 2021-02-10T11:01:56.093 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2021-02-10T11:01:56.094 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:01:56.126 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:56 +0000 (0:00:00.061) 0:00:11.704 **** 2021-02-10T11:01:56.134 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* ok: [gibba008.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-10T11:01:56.167 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:56 +0000 (0:00:00.041) 0:00:11.746 **** 2021-02-10T11:01:56.174 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2021-02-10T11:01:56.175 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:01:56.213 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:56 +0000 (0:00:00.045) 0:00:11.792 **** 2021-02-10T11:01:56.664 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2021-02-10T11:01:56.666 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-10T11:01:56.698 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:01:56 +0000 (0:00:00.484) 0:00:12.276 **** 2021-02-10T11:01:56.704 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2021-02-10T11:01:56.705 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-02-10T11:01:57.113 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-02-10T11:01:57.178 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-02-10T11:01:57.543 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-02-10T11:01:57.619 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-02-10T11:01:57.983 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-02-10T11:01:58.057 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-02-10T11:01:58.420 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-02-10T11:01:58.493 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-02-10T11:01:58.857 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-02-10T11:01:58.931 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-02-10T11:01:58.933 INFO:teuthology.task.ansible.out: 2021-02-10T11:01:59.305 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-02-10T11:01:59.368 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-02-10T11:01:59.742 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-02-10T11:01:59.805 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-02-10T11:02:00.179 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-02-10T11:02:00.242 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-02-10T11:02:00.617 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-02-10T11:02:00.681 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-02-10T11:02:01.057 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-02-10T11:02:01.120 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-02-10T11:02:01.493 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-02-10T11:02:01.556 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-02-10T11:02:01.930 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-02-10T11:02:02.003 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-02-10T11:02:02.367 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-02-10T11:02:02.441 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-02-10T11:02:02.803 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-02-10T11:02:02.877 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-02-10T11:02:03.243 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-02-10T11:02:03.317 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-02-10T11:02:03.691 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-02-10T11:02:03.754 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-02-10T11:02:04.128 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-02-10T11:02:04.191 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-02-10T11:02:04.565 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-02-10T11:02:04.629 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-02-10T11:02:05.005 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-02-10T11:02:05.069 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-02-10T11:02:05.443 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-02-10T11:02:05.496 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-02-10T11:02:05.880 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-02-10T11:02:05.933 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-02-10T11:02:06.318 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-02-10T11:02:06.370 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-02-10T11:02:06.765 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-02-10T11:02:06.818 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-02-10T11:02:07.201 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-02-10T11:02:07.254 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-02-10T11:02:07.637 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-02-10T11:02:07.693 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-02-10T11:02:08.076 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-02-10T11:02:08.129 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-02-10T11:02:08.512 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-02-10T11:02:08.565 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-02-10T11:02:08.948 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-02-10T11:02:09.001 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-02-10T11:02:09.386 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-02-10T11:02:09.429 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-02-10T11:02:09.826 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-02-10T11:02:09.869 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-02-10T11:02:10.265 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-02-10T11:02:10.308 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-02-10T11:02:10.701 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-02-10T11:02:10.743 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-02-10T11:02:11.147 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-02-10T11:02:11.179 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-02-10T11:02:11.583 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-02-10T11:02:11.615 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-02-10T11:02:12.013 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-02-10T11:02:12.056 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-02-10T11:02:12.453 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-02-10T11:02:12.506 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-02-10T11:02:12.896 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-02-10T11:02:12.940 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-02-10T11:02:13.332 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-02-10T11:02:13.374 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-02-10T11:02:13.769 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-02-10T11:02:13.822 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-02-10T11:02:14.205 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-02-10T11:02:14.258 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-02-10T11:02:14.643 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-02-10T11:02:14.697 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-02-10T11:02:15.081 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-02-10T11:02:15.134 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-02-10T11:02:15.522 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-02-10T11:02:15.575 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-02-10T11:02:15.960 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-02-10T11:02:16.013 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-02-10T11:02:16.406 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-02-10T11:02:16.449 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-02-10T11:02:16.842 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-02-10T11:02:16.884 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-02-10T11:02:17.278 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-02-10T11:02:17.320 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-02-10T11:02:17.721 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-02-10T11:02:17.753 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-02-10T11:02:18.155 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-02-10T11:02:18.198 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-02-10T11:02:18.595 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-02-10T11:02:18.637 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-02-10T11:02:19.031 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-02-10T11:02:19.074 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-02-10T11:02:19.479 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-02-10T11:02:19.630 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-02-10T11:02:19.919 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-02-10T11:02:19.961 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-02-10T11:02:20.355 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-02-10T11:02:20.398 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-02-10T11:02:20.792 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-02-10T11:02:20.845 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-02-10T11:02:21.244 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-02-10T11:02:21.286 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-02-10T11:02:21.701 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-02-10T11:02:21.744 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-02-10T11:02:22.170 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-02-10T11:02:22.202 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-02-10T11:02:22.617 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-02-10T11:02:22.681 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-02-10T11:02:23.060 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-02-10T11:02:23.125 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-02-10T11:02:23.497 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-02-10T11:02:23.582 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-02-10T11:02:23.935 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-02-10T11:02:24.018 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-02-10T11:02:24.372 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-02-10T11:02:24.456 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-02-10T11:02:24.820 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-02-10T11:02:24.893 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-02-10T11:02:25.256 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-02-10T11:02:25.330 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-02-10T11:02:25.693 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-02-10T11:02:25.767 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-02-10T11:02:26.129 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-02-10T11:02:26.213 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-02-10T11:02:26.575 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-02-10T11:02:26.649 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-02-10T11:02:27.012 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-02-10T11:02:27.086 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-02-10T11:02:27.449 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-02-10T11:02:27.522 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-02-10T11:02:27.897 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-02-10T11:02:27.961 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-02-10T11:02:28.336 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-02-10T11:02:28.399 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-02-10T11:02:28.773 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-02-10T11:02:28.836 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-02-10T11:02:29.210 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-02-10T11:02:29.274 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-02-10T11:02:29.651 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-02-10T11:02:29.714 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-02-10T11:02:30.093 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-02-10T11:02:30.167 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-02-10T11:02:30.535 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-02-10T11:02:30.619 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-02-10T11:02:30.973 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-02-10T11:02:31.058 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-02-10T11:02:31.424 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-02-10T11:02:31.488 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-02-10T11:02:31.869 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-02-10T11:02:31.937 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-02-10T11:02:32.305 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-02-10T11:02:32.369 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-02-10T11:02:32.745 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-02-10T11:02:32.809 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-02-10T11:02:33.196 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-02-10T11:02:33.260 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-02-10T11:02:33.635 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-02-10T11:02:33.708 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-02-10T11:02:34.076 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-02-10T11:02:34.142 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-02-10T11:02:34.511 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-02-10T11:02:34.585 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-02-10T11:02:34.954 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-02-10T11:02:35.019 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-02-10T11:02:35.415 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-02-10T11:02:35.469 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-02-10T11:02:35.857 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-02-10T11:02:35.910 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-02-10T11:02:36.296 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-02-10T11:02:36.348 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-02-10T11:02:36.731 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-02-10T11:02:36.794 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-02-10T11:02:37.169 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-02-10T11:02:37.232 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-02-10T11:02:37.611 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-02-10T11:02:37.664 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-02-10T11:02:38.052 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-02-10T11:02:38.116 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-02-10T11:02:38.483 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-02-10T11:02:38.556 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-02-10T11:02:38.926 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-02-10T11:02:39.001 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-02-10T11:02:39.368 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-02-10T11:02:39.445 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-02-10T11:02:39.830 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-02-10T11:02:39.883 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-02-10T11:02:40.286 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-02-10T11:02:40.330 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-02-10T11:02:40.728 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-02-10T11:02:40.762 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-02-10T11:02:41.168 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-02-10T11:02:41.211 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-02-10T11:02:41.613 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-02-10T11:02:41.647 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-02-10T11:02:42.063 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-02-10T11:02:42.096 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-02-10T11:02:42.528 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-02-10T11:02:42.540 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-02-10T11:02:42.969 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-02-10T11:02:42.981 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-02-10T11:02:43.413 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-02-10T11:02:43.414 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-02-10T11:02:43.858 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-02-10T11:02:43.859 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-02-10T11:02:44.299 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-02-10T11:02:44.300 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-02-10T11:02:44.732 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-02-10T11:02:44.733 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-02-10T11:02:45.170 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-02-10T11:02:45.171 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-02-10T11:02:45.608 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-02-10T11:02:45.609 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-02-10T11:02:46.049 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-02-10T11:02:46.050 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-02-10T11:02:46.490 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-02-10T11:02:46.491 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-02-10T11:02:46.491 INFO:teuthology.task.ansible.out: 2021-02-10T11:02:46.927 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-02-10T11:02:46.928 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-02-10T11:02:47.384 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-02-10T11:02:47.418 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-02-10T11:02:47.845 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-02-10T11:02:47.898 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-02-10T11:02:48.301 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-02-10T11:02:48.354 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-02-10T11:02:48.748 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-02-10T11:02:48.823 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-02-10T11:02:49.201 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-02-10T11:02:49.276 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-02-10T11:02:49.644 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-02-10T11:02:49.719 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-02-10T11:02:50.088 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-02-10T11:02:50.153 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-02-10T11:02:50.543 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-02-10T11:02:50.596 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-02-10T11:02:50.974 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-02-10T11:02:50.975 INFO:teuthology.task.ansible.out: 2021-02-10T11:02:51.039 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-02-10T11:02:51.409 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-02-10T11:02:51.473 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-02-10T11:02:51.849 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-02-10T11:02:51.913 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-02-10T11:02:52.290 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-02-10T11:02:52.364 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-02-10T11:02:52.735 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-02-10T11:02:52.799 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-02-10T11:02:53.177 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-02-10T11:02:53.251 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-02-10T11:02:53.619 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-02-10T11:02:53.683 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-02-10T11:02:54.068 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-02-10T11:02:54.132 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-02-10T11:02:54.133 INFO:teuthology.task.ansible.out: 2021-02-10T11:02:54.506 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-02-10T11:02:54.573 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-02-10T11:02:54.951 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-02-10T11:02:54.952 INFO:teuthology.task.ansible.out: 2021-02-10T11:02:55.015 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-02-10T11:02:55.016 INFO:teuthology.task.ansible.out: 2021-02-10T11:02:55.383 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-02-10T11:02:55.446 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-02-10T11:02:55.825 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-02-10T11:02:55.890 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-02-10T11:02:56.278 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-02-10T11:02:56.769 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2021-02-10T11:02:57.117 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2021-02-10T11:02:57.568 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "name": "adking", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2021-02-10T11:02:57.903 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "name": "adking", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2021-02-10T11:02:58.334 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "name": "amathuri", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2021-02-10T11:02:58.732 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "name": "amathuri", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2021-02-10T11:02:59.118 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10106, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10105} 2021-02-10T11:02:59.251 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:02:59 +0000 (0:01:02.553) 0:01:14.829 **** 2021-02-10T11:02:59.278 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2021-02-10T11:02:59.314 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:02:59 +0000 (0:00:00.063) 0:01:14.893 **** 2021-02-10T11:02:59.363 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2021-02-10T11:02:59.365 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora r 2021-02-10T11:02:59.367 INFO:teuthology.task.ansible.out:T6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c 2021-02-10T11:02:59.368 INFO:teuthology.task.ansible.out:63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich 2021-02-10T11:02:59.370 INFO:teuthology.task.ansible.out:", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiR 2021-02-10T11:02:59.371 INFO:teuthology.task.ansible.out:enVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumr 2021-02-10T11:02:59.373 INFO:teuthology.task.ansible.out:ao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69m 2021-02-10T11:02:59.374 INFO:teuthology.task.ansible.out:h95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5 2021-02-10T11:02:59.374 INFO:teuthology.task.ansible.out:d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d84 2021-02-10T11:02:59.375 INFO:teuthology.task.ansible.out:1a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan 2021-02-10T11:02:59.377 INFO:teuthology.task.ansible.out:@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOen 2021-02-10T11:02:59.378 INFO:teuthology.task.ansible.out:VV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9g 2021-02-10T11:02:59.379 INFO:teuthology.task.ansible.out:I46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": " 2021-02-10T11:02:59.381 INFO:teuthology.task.ansible.out:dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NIN 2021-02-10T11:02:59.382 INFO:teuthology.task.ansible.out:j0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa33958 2021-02-10T11:02:59.382 INFO:teuthology.task.ansible.out:62e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}]}, "changed": false} 2021-02-10T11:02:59.429 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:02:59 +0000 (0:00:00.114) 0:01:15.007 **** 2021-02-10T11:02:59.551 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2021-02-10T11:02:59.552 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10106, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10105} 2021-02-10T11:02:59.666 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:02:59 +0000 (0:00:00.237) 0:01:15.245 **** 2021-02-10T11:02:59.695 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2021-02-10T11:02:59.729 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:02:59 +0000 (0:00:00.062) 0:01:15.308 **** 2021-02-10T11:02:59.790 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2021-02-10T11:02:59.792 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora r 2021-02-10T11:02:59.793 INFO:teuthology.task.ansible.out:T6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c 2021-02-10T11:02:59.795 INFO:teuthology.task.ansible.out:63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich 2021-02-10T11:02:59.796 INFO:teuthology.task.ansible.out:", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiR 2021-02-10T11:02:59.798 INFO:teuthology.task.ansible.out:enVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumr 2021-02-10T11:02:59.799 INFO:teuthology.task.ansible.out:ao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69m 2021-02-10T11:02:59.801 INFO:teuthology.task.ansible.out:h95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5 2021-02-10T11:02:59.801 INFO:teuthology.task.ansible.out:d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d84 2021-02-10T11:02:59.802 INFO:teuthology.task.ansible.out:1a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan 2021-02-10T11:02:59.804 INFO:teuthology.task.ansible.out:@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOen 2021-02-10T11:02:59.805 INFO:teuthology.task.ansible.out:VV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9g 2021-02-10T11:02:59.806 INFO:teuthology.task.ansible.out:I46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": " 2021-02-10T11:02:59.808 INFO:teuthology.task.ansible.out:dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NIN 2021-02-10T11:02:59.809 INFO:teuthology.task.ansible.out:j0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa33958 2021-02-10T11:02:59.810 INFO:teuthology.task.ansible.out:62e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}]}, "changed": false} 2021-02-10T11:02:59.850 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:02:59 +0000 (0:00:00.120) 0:01:15.429 **** 2021-02-10T11:03:00.829 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2021-02-10T11:03:00.831 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com -> localhost] => {"after": "df8946f08e870e2d2565b79099289ca48bd825f3", "attempts": 1, "before": "df8946f08e870e2d2565b79099289ca48bd825f3", "changed": false, "remote_url_changed": false} 2021-02-10T11:03:00.874 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:03:00 +0000 (0:00:01.024) 0:01:16.453 **** 2021-02-10T11:03:00.883 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com -> localhost] => {"after": "df8946f08e870e2d2565b79099289ca48bd825f3", "attempts": 1, "before": "df8946f08e870e2d2565b79099289ca48bd825f3", "changed": false, "remote_url_changed": false} 2021-02-10T11:03:00.930 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:03:00 +0000 (0:00:00.055) 0:01:16.508 **** 2021-02-10T11:03:01.401 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-10T11:03:01.402 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-02-10T11:03:01.423 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-02-10T11:03:01.846 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-02-10T11:03:01.869 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-02-10T11:03:02.309 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-02-10T11:03:02.321 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-02-10T11:03:02.750 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZt 2021-02-10T11:03:02.752 INFO:teuthology.task.ansible.out:IEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-02-10T11:03:02.772 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZt 2021-02-10T11:03:02.773 INFO:teuthology.task.ansible.out:IEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-02-10T11:03:03.198 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-02-10T11:03:03.219 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-02-10T11:03:03.644 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o 2021-02-10T11:03:03.645 INFO:teuthology.task.ansible.out:8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-02-10T11:03:03.688 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o 2021-02-10T11:03:03.689 INFO:teuthology.task.ansible.out:8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-02-10T11:03:04.094 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-02-10T11:03:04.148 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-02-10T11:03:04.558 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-02-10T11:03:04.600 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-02-10T11:03:04.998 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-02-10T11:03:05.009 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:03:05.062 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-02-10T11:03:05.073 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:03:05.438 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMorb 2021-02-10T11:03:05.439 INFO:teuthology.task.ansible.out:nuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-02-10T11:03:05.511 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMorb 2021-02-10T11:03:05.513 INFO:teuthology.task.ansible.out:nuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-02-10T11:03:05.887 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemers 2021-02-10T11:03:05.888 INFO:teuthology.task.ansible.out:on", "validate_certs": true} skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:03:05.899 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:03:05.952 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemers 2021-02-10T11:03:05.953 INFO:teuthology.task.ansible.out:on", "validate_certs": true} 2021-02-10T11:03:05.964 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:03:05.974 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:03:06.341 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-02-10T11:03:06.415 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-02-10T11:03:06.791 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtSo 2021-02-10T11:03:06.792 INFO:teuthology.task.ansible.out:AWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-02-10T11:03:06.854 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtSo 2021-02-10T11:03:06.856 INFO:teuthology.task.ansible.out:AWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-02-10T11:03:07.235 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-02-10T11:03:07.298 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-02-10T11:03:07.682 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2021-02-10T11:03:07.744 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2021-02-10T11:03:08.122 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-02-10T11:03:08.197 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-02-10T11:03:08.565 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-02-10T11:03:08.640 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-02-10T11:03:09.032 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-02-10T11:03:09.086 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-02-10T11:03:09.480 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9C 2021-02-10T11:03:09.482 INFO:teuthology.task.ansible.out:kxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-02-10T11:03:09.536 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9C 2021-02-10T11:03:09.537 INFO:teuthology.task.ansible.out:kxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-02-10T11:03:09.928 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBYB 2021-02-10T11:03:09.930 INFO:teuthology.task.ansible.out:0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-02-10T11:03:09.992 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBYB 2021-02-10T11:03:09.994 INFO:teuthology.task.ansible.out:0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-02-10T11:03:10.384 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-02-10T11:03:10.449 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-02-10T11:03:10.825 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-02-10T11:03:10.898 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-02-10T11:03:11.277 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46T 2021-02-10T11:03:11.279 INFO:teuthology.task.ansible.out:RJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-02-10T11:03:11.343 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46T 2021-02-10T11:03:11.344 INFO:teuthology.task.ansible.out:RJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-02-10T11:03:11.718 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-02-10T11:03:11.781 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-02-10T11:03:12.166 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-02-10T11:03:12.232 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-02-10T11:03:12.606 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-02-10T11:03:12.670 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-02-10T11:03:13.047 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-02-10T11:03:13.120 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-02-10T11:03:13.485 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-02-10T11:03:13.559 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-02-10T11:03:13.934 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQvU 2021-02-10T11:03:13.936 INFO:teuthology.task.ansible.out:xUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-02-10T11:03:14.007 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQvU 2021-02-10T11:03:14.009 INFO:teuthology.task.ansible.out:xUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-02-10T11:03:14.378 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-02-10T11:03:14.454 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-02-10T11:03:14.835 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-02-10T11:03:14.910 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-02-10T11:03:15.287 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-02-10T11:03:15.351 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-02-10T11:03:15.737 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-02-10T11:03:15.790 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-02-10T11:03:16.174 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2021-02-10T11:03:16.248 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2021-02-10T11:03:16.249 INFO:teuthology.task.ansible.out: 2021-02-10T11:03:16.614 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-02-10T11:03:16.698 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-02-10T11:03:17.062 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-02-10T11:03:17.146 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-02-10T11:03:17.503 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-02-10T11:03:17.587 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-02-10T11:03:17.953 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-02-10T11:03:18.027 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-02-10T11:03:18.402 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1O 2021-02-10T11:03:18.404 INFO:teuthology.task.ansible.out:MJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-02-10T11:03:18.476 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1O 2021-02-10T11:03:18.478 INFO:teuthology.task.ansible.out:MJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-02-10T11:03:18.851 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyKz 2021-02-10T11:03:18.852 INFO:teuthology.task.ansible.out:2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-02-10T11:03:18.924 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyKz 2021-02-10T11:03:18.926 INFO:teuthology.task.ansible.out:2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-02-10T11:03:19.289 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-02-10T11:03:19.362 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-02-10T11:03:19.737 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-02-10T11:03:19.811 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-02-10T11:03:20.177 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-02-10T11:03:20.241 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-02-10T11:03:20.627 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nul 2021-02-10T11:03:20.629 INFO:teuthology.task.ansible.out:l, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-02-10T11:03:20.691 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nul 2021-02-10T11:03:20.692 INFO:teuthology.task.ansible.out:l, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-02-10T11:03:21.065 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjEf 2021-02-10T11:03:21.067 INFO:teuthology.task.ansible.out:dVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+dT 2021-02-10T11:03:21.068 INFO:teuthology.task.ansible.out:zNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-02-10T11:03:21.139 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjEf 2021-02-10T11:03:21.141 INFO:teuthology.task.ansible.out:dVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+dT 2021-02-10T11:03:21.142 INFO:teuthology.task.ansible.out:zNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-02-10T11:03:21.505 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-02-10T11:03:21.590 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-02-10T11:03:21.954 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-02-10T11:03:22.028 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-02-10T11:03:22.393 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-02-10T11:03:22.395 INFO:teuthology.task.ansible.out: 2021-02-10T11:03:22.478 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-02-10T11:03:22.855 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-02-10T11:03:22.919 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-02-10T11:03:23.296 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-02-10T11:03:23.297 INFO:teuthology.task.ansible.out: 2021-02-10T11:03:23.370 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-02-10T11:03:23.371 INFO:teuthology.task.ansible.out: 2021-02-10T11:03:23.750 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home/ 2021-02-10T11:03:23.751 INFO:teuthology.task.ansible.out:nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-02-10T11:03:23.816 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home/ 2021-02-10T11:03:23.817 INFO:teuthology.task.ansible.out:nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-02-10T11:03:24.193 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-02-10T11:03:24.256 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-02-10T11:03:24.634 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-02-10T11:03:24.707 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-02-10T11:03:25.070 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "g 2021-02-10T11:03:25.071 INFO:teuthology.task.ansible.out:abrioux", "validate_certs": true} 2021-02-10T11:03:25.153 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "g 2021-02-10T11:03:25.155 INFO:teuthology.task.ansible.out:abrioux", "validate_certs": true} 2021-02-10T11:03:25.517 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-02-10T11:03:25.590 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-02-10T11:03:25.955 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-02-10T11:03:26.039 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-02-10T11:03:26.404 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kkk 2021-02-10T11:03:26.405 INFO:teuthology.task.ansible.out:e6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-02-10T11:03:26.488 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kkk 2021-02-10T11:03:26.489 INFO:teuthology.task.ansible.out:e6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-02-10T11:03:26.842 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-02-10T11:03:26.926 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-02-10T11:03:27.288 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-02-10T11:03:27.373 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-02-10T11:03:27.749 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-02-10T11:03:27.822 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-02-10T11:03:28.178 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-02-10T11:03:28.179 INFO:teuthology.task.ansible.out: 2021-02-10T11:03:28.263 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-02-10T11:03:28.628 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-02-10T11:03:28.704 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-02-10T11:03:29.079 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-02-10T11:03:29.144 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-02-10T11:03:29.565 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-02-10T11:03:29.644 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-02-10T11:03:30.041 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-02-10T11:03:30.095 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-02-10T11:03:30.529 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-02-10T11:03:30.552 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-02-10T11:03:30.997 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-02-10T11:03:30.997 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-02-10T11:03:31.437 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-02-10T11:03:31.448 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-02-10T11:03:31.879 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-02-10T11:03:31.880 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-02-10T11:03:32.322 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-02-10T11:03:32.333 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-02-10T11:03:32.765 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-02-10T11:03:32.777 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-02-10T11:03:33.215 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-02-10T11:03:33.217 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-02-10T11:03:33.659 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-02-10T11:03:33.671 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-02-10T11:03:34.100 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-02-10T11:03:34.121 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-02-10T11:03:34.543 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-02-10T11:03:34.565 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-02-10T11:03:34.981 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-02-10T11:03:35.003 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-02-10T11:03:35.430 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-02-10T11:03:35.441 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-02-10T11:03:35.871 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1l 2021-02-10T11:03:35.872 INFO:teuthology.task.ansible.out:XTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-02-10T11:03:35.882 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1l 2021-02-10T11:03:35.883 INFO:teuthology.task.ansible.out:XTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-02-10T11:03:36.309 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-02-10T11:03:36.321 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-02-10T11:03:36.751 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-02-10T11:03:36.773 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-02-10T11:03:37.192 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-02-10T11:03:37.214 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-02-10T11:03:37.642 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-02-10T11:03:37.653 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-02-10T11:03:38.083 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.localdo 2021-02-10T11:03:38.084 INFO:teuthology.task.ansible.out:main", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-02-10T11:03:38.094 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.localdo 2021-02-10T11:03:38.096 INFO:teuthology.task.ansible.out:main", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-02-10T11:03:38.530 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq2 2021-02-10T11:03:38.532 INFO:teuthology.task.ansible.out:z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-02-10T11:03:38.542 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq2 2021-02-10T11:03:38.543 INFO:teuthology.task.ansible.out:z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-02-10T11:03:38.980 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-02-10T11:03:38.981 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-02-10T11:03:39.429 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-02-10T11:03:39.430 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-02-10T11:03:39.867 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-02-10T11:03:39.868 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-02-10T11:03:40.317 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-02-10T11:03:40.318 INFO:teuthology.task.ansible.out: 2021-02-10T11:03:40.319 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-02-10T11:03:40.774 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-02-10T11:03:40.785 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-02-10T11:03:41.225 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-02-10T11:03:41.226 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-02-10T11:03:41.668 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-02-10T11:03:41.669 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-02-10T11:03:42.108 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loca 2021-02-10T11:03:42.109 INFO:teuthology.task.ansible.out:l", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-02-10T11:03:42.119 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loca 2021-02-10T11:03:42.120 INFO:teuthology.task.ansible.out:l", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-02-10T11:03:42.543 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-02-10T11:03:42.565 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-02-10T11:03:42.992 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-02-10T11:03:43.014 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-02-10T11:03:43.432 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-02-10T11:03:43.443 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-02-10T11:03:43.874 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-02-10T11:03:43.885 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-02-10T11:03:44.312 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-02-10T11:03:44.333 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-02-10T11:03:44.753 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-02-10T11:03:44.775 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-02-10T11:03:45.202 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", " 2021-02-10T11:03:45.203 INFO:teuthology.task.ansible.out:manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-02-10T11:03:45.213 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", " 2021-02-10T11:03:45.215 INFO:teuthology.task.ansible.out:manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-02-10T11:03:45.655 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-02-10T11:03:45.667 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-02-10T11:03:46.109 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-02-10T11:03:46.109 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-02-10T11:03:46.549 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-02-10T11:03:46.561 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-02-10T11:03:47.000 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-02-10T11:03:47.002 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-02-10T11:03:47.442 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-02-10T11:03:47.443 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-02-10T11:03:47.883 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-02-10T11:03:47.894 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-02-10T11:03:48.337 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-02-10T11:03:48.349 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-02-10T11:03:48.780 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-02-10T11:03:48.802 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-02-10T11:03:49.239 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-02-10T11:03:49.250 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-02-10T11:03:49.676 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-02-10T11:03:49.687 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-02-10T11:03:50.127 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-02-10T11:03:50.128 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-02-10T11:03:50.563 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-02-10T11:03:50.575 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-02-10T11:03:51.016 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-02-10T11:03:51.017 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-02-10T11:03:51.455 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-02-10T11:03:51.466 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-02-10T11:03:51.906 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-02-10T11:03:51.907 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-02-10T11:03:52.343 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-02-10T11:03:52.344 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-02-10T11:03:52.783 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-02-10T11:03:52.794 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-02-10T11:03:53.236 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-02-10T11:03:53.237 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-02-10T11:03:53.682 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjLd 2021-02-10T11:03:53.684 INFO:teuthology.task.ansible.out:+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVs 2021-02-10T11:03:53.685 INFO:teuthology.task.ansible.out:gx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjLd+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-02-10T11:03:54.124 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-02-10T11:03:54.135 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-02-10T11:03:54.588 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-02-10T11:03:54.589 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-02-10T11:03:55.050 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-02-10T11:03:55.052 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-02-10T11:03:55.502 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-02-10T11:03:55.504 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-02-10T11:03:55.947 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-02-10T11:03:55.948 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-02-10T11:03:56.387 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-02-10T11:03:56.388 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-02-10T11:03:56.827 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-02-10T11:03:56.827 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-02-10T11:03:57.264 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-02-10T11:03:57.265 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-02-10T11:03:57.702 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-02-10T11:03:57.703 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-02-10T11:03:58.152 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-02-10T11:03:58.152 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-02-10T11:03:58.592 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-02-10T11:03:58.593 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-02-10T11:03:59.041 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-02-10T11:03:59.052 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-02-10T11:03:59.487 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-02-10T11:03:59.499 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-02-10T11:03:59.923 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-02-10T11:03:59.935 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-02-10T11:04:00.371 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-02-10T11:04:00.382 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-02-10T11:04:00.809 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-02-10T11:04:00.830 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-02-10T11:04:01.255 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com", "key_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-02-10T11:04:01.267 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com", "key_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-02-10T11:04:01.706 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-02-10T11:04:01.754 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-02-10T11:04:01.829 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:01 +0000 (0:01:00.899) 0:02:17.408 **** 2021-02-10T11:04:01.861 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:01 +0000 (0:00:00.031) 0:02:17.440 **** 2021-02-10T11:04:01.881 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-02-10T11:04:01.882 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:01.883 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:01.894 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:01.905 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:01.905 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:01.907 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:01.917 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:01.917 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:01.928 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:01.929 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:01.930 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:01.940 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:01.951 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:01.963 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:01.964 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:01.974 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:01.986 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:02.372 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-02-10T11:04:02.394 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:02.394 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:02.426 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-02-10T11:04:02.437 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:02.448 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:02.842 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-02-10T11:04:02.885 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-02-10T11:04:03.292 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-02-10T11:04:03.293 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.304 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.314 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.325 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.326 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-02-10T11:04:03.327 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.337 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.338 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.339 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.349 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.350 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.351 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.361 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.362 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.363 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.373 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.373 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.375 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.385 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.385 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.396 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.397 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.398 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.408 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.409 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.420 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.420 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.422 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.425 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.434 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.434 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.436 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.446 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.446 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.447 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.457 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.458 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.459 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.470 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.471 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.481 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.482 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.483 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.494 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.494 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.505 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.506 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.507 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.518 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.518 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.529 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.530 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.531 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.541 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.542 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.553 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.554 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.555 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.565 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.566 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.567 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.577 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.578 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.589 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.589 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.591 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.601 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.601 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.603 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.613 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.613 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.615 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.625 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.626 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.637 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.638 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.639 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.650 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.651 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.653 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.664 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.665 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.666 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.677 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.677 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.679 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.689 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.689 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.691 INFO:teuthology.task.ansible.out: 2021-02-10T11:04:03.700 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.701 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.702 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.712 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.713 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.714 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.725 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.725 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.727 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.737 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.737 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.738 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.740 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.750 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.751 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.753 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.763 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.763 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.765 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.775 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.775 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.776 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.787 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.787 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.788 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.799 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.799 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.801 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.811 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.811 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.813 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.822 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.823 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.824 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.835 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.835 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.836 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.837 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.847 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.847 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.860 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.861 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.873 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.873 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.875 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.886 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.887 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.888 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.889 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.899 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.901 INFO:teuthology.task.ansible.out: skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.911 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.912 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.913 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.924 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.925 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.926 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.936 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.937 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.938 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.938 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.949 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.949 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.951 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.961 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.962 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.963 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.973 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.974 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.975 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.985 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.987 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.988 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.998 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:03.998 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.000 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.010 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.010 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.011 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.021 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.023 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.033 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.033 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.035 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.045 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.045 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.046 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.056 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.057 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.058 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.068 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.069 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.070 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.080 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.080 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.082 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.092 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.092 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.093 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.103 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.104 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.105 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.115 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.116 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.117 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.127 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.127 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.128 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.138 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.139 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.150 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.150 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.151 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.162 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.162 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.164 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.174 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.175 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.177 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.188 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.188 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.189 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.200 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.200 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.211 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.212 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.213 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.224 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.226 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.227 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.237 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.238 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.239 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.250 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.251 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.252 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.263 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.264 INFO:teuthology.task.ansible.out: skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "skip_reason": "Conditional result was False"} skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.275 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.276 INFO:teuthology.task.ansible.out: skipping: [gibba001.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.296 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.297 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.298 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.330 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:04 +0000 (0:00:02.468) 0:02:19.909 **** 2021-02-10T11:04:04.337 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.388 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:04 +0000 (0:00:00.057) 0:02:19.966 **** 2021-02-10T11:04:04.394 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2021-02-10T11:04:04.395 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.429 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:04 +0000 (0:00:00.040) 0:02:20.007 **** 2021-02-10T11:04:04.437 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:04.471 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:04 +0000 (0:00:00.042) 0:02:20.049 **** 2021-02-10T11:04:04.905 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-10T11:04:04.907 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-02-10T11:04:04.938 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-02-10T11:04:05.343 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-02-10T11:04:05.376 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-02-10T11:04:05.772 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-02-10T11:04:05.814 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-02-10T11:04:06.211 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-02-10T11:04:06.243 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-02-10T11:04:06.647 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-02-10T11:04:06.680 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-02-10T11:04:07.077 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-02-10T11:04:07.108 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-02-10T11:04:07.517 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-02-10T11:04:07.549 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-02-10T11:04:07.955 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-02-10T11:04:07.987 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-02-10T11:04:08.385 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-02-10T11:04:08.418 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-02-10T11:04:08.825 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-02-10T11:04:08.846 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-02-10T11:04:09.260 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-02-10T11:04:09.282 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-02-10T11:04:09.696 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-02-10T11:04:09.717 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-02-10T11:04:10.133 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-02-10T11:04:10.155 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-02-10T11:04:10.570 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-02-10T11:04:10.581 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-02-10T11:04:10.996 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-02-10T11:04:11.018 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-02-10T11:04:11.434 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-02-10T11:04:11.457 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-02-10T11:04:11.872 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-02-10T11:04:11.893 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-02-10T11:04:12.309 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-02-10T11:04:12.330 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-02-10T11:04:12.735 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-02-10T11:04:12.767 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-02-10T11:04:13.172 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-02-10T11:04:13.214 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-02-10T11:04:13.607 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-02-10T11:04:13.649 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-02-10T11:04:14.042 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2021-02-10T11:04:14.295 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2021-02-10T11:04:14.481 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-02-10T11:04:14.575 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-02-10T11:04:14.920 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-02-10T11:04:15.004 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-02-10T11:04:15.356 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-02-10T11:04:15.451 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-02-10T11:04:15.784 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-02-10T11:04:15.878 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-02-10T11:04:16.221 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-02-10T11:04:16.316 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-02-10T11:04:16.658 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-02-10T11:04:16.754 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-02-10T11:04:17.088 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-02-10T11:04:17.193 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-02-10T11:04:17.525 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-02-10T11:04:17.619 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-02-10T11:04:17.962 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-02-10T11:04:18.066 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-02-10T11:04:18.394 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-02-10T11:04:18.499 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-02-10T11:04:18.822 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-02-10T11:04:18.937 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-02-10T11:04:19.276 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-02-10T11:04:19.381 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-02-10T11:04:19.732 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-02-10T11:04:19.847 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-02-10T11:04:20.197 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-02-10T11:04:20.292 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-02-10T11:04:20.640 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-02-10T11:04:20.734 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-02-10T11:04:21.419 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "dorinda", "name": "dorinda", "remove": false, "state": "absent"} 2021-02-10T11:04:21.451 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "dorinda", "name": "dorinda", "remove": false, "state": "absent"} 2021-02-10T11:04:22.110 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "zyan", "name": "zyan", "remove": false, "state": "absent"} 2021-02-10T11:04:22.194 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "zyan", "name": "zyan", "remove": false, "state": "absent"} 2021-02-10T11:04:22.549 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-02-10T11:04:22.633 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-02-10T11:04:22.989 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-02-10T11:04:23.063 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-02-10T11:04:23.421 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-02-10T11:04:23.506 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-02-10T11:04:23.861 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-02-10T11:04:23.946 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-02-10T11:04:24.295 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-02-10T11:04:24.380 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-02-10T11:04:24.725 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-02-10T11:04:24.811 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-02-10T11:04:25.167 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-02-10T11:04:25.251 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-02-10T11:04:25.595 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-02-10T11:04:25.678 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-02-10T11:04:26.033 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-02-10T11:04:26.035 INFO:teuthology.task.ansible.out: 2021-02-10T11:04:26.120 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-02-10T11:04:26.481 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-02-10T11:04:26.543 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:26 +0000 (0:00:22.071) 0:02:42.121 **** 2021-02-10T11:04:26.560 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-02-10T11:04:26.615 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:26 +0000 (0:00:00.072) 0:02:42.194 **** 2021-02-10T11:04:29.454 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-02-10T11:04:29.456 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "checksum": "5ebc9fcf8859a9c716ab10089a7bec7e44d29368", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "ee06ed012f51c2e3764171d4b9180349", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1612955066.5673385-91968877814410/source", "state": "file", "uid": 0} 2021-02-10T11:04:29.490 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:29 +0000 (0:00:02.874) 0:02:45.069 **** 2021-02-10T11:04:29.499 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "checksum": "5ebc9fcf8859a9c716ab10089a7bec7e44d29368", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "ee06ed012f51c2e3764171d4b9180349", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1612955066.6434767-195508434073914/source", "state": "file", "uid": 0} 2021-02-10T11:04:29.540 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:29 +0000 (0:00:00.049) 0:02:45.119 **** 2021-02-10T11:04:29.547 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-02-10T11:04:29.549 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => { "msg": "Host gibba008.front.sepia.ceph.com is running RedHat 7.9 (Maipo)" } 2021-02-10T11:04:29.587 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:29 +0000 (0:00:00.046) 0:02:45.166 **** 2021-02-10T11:04:29.593 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => { "msg": "Host gibba001.front.sepia.ceph.com is running RedHat 7.9 (Maipo)" } 2021-02-10T11:04:29.630 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:29 +0000 (0:00:00.043) 0:02:45.209 **** 2021-02-10T11:04:29.639 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-02-10T11:04:29.640 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/redhat_7.yml) => {"ansible_facts": {"rhsm_repos": ["rhel-7-server-rpms", "rhel-7-server-optional-rpms", "rhel-7-server-extras-rpms", "rhel-ha-for-rhel-7-server-rpms"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/redhat_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/redhat_7.yml"} 2021-02-10T11:04:29.677 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:29 +0000 (0:00:00.046) 0:02:45.256 **** 2021-02-10T11:04:29.685 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/redhat_7.yml) => {"ansible_facts": {"rhsm_repos": ["rhel-7-server-rpms", "rhel-7-server-optional-rpms", "rhel-7-server-extras-rpms", "rhel-ha-for-rhel-7-server-rpms"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/redhat_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/redhat_7.yml"} 2021-02-10T11:04:29.723 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:29 +0000 (0:00:00.045) 0:02:45.302 **** 2021-02-10T11:04:29.729 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2021-02-10T11:04:29.730 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:29.764 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:29 +0000 (0:00:00.040) 0:02:45.343 **** 2021-02-10T11:04:29.770 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:29.807 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:29 +0000 (0:00:00.042) 0:02:45.386 **** 2021-02-10T11:04:30.298 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2021-02-10T11:04:30.298 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.074938", "end": "2021-02-10 11:04:30.276180", "rc": 0, "start": "2021-02-10 11:04:30.201242", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-02-10T11:04:30.338 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:30 +0000 (0:00:00.531) 0:02:45.917 **** 2021-02-10T11:04:30.345 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.070444", "end": "2021-02-10 11:04:30.303765", "rc": 0, "start": "2021-02-10 11:04:30.233321", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-02-10T11:04:30.383 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:30 +0000 (0:00:00.044) 0:02:45.961 **** 2021-02-10T11:04:30.390 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-02-10T11:04:30.391 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:30.439 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:30 +0000 (0:00:00.056) 0:02:46.018 **** 2021-02-10T11:04:30.446 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:30.482 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:30 +0000 (0:00:00.043) 0:02:46.061 **** 2021-02-10T11:04:30.489 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2021-02-10T11:04:30.490 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:30.527 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:30 +0000 (0:00:00.045) 0:02:46.106 **** 2021-02-10T11:04:30.534 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:30.568 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:30 +0000 (0:00:00.040) 0:02:46.147 **** 2021-02-10T11:04:30.574 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-02-10T11:04:30.576 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:30.610 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:30 +0000 (0:00:00.042) 0:02:46.189 **** 2021-02-10T11:04:30.616 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:30.650 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:30 +0000 (0:00:00.040) 0:02:46.229 **** 2021-02-10T11:04:30.656 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2021-02-10T11:04:30.658 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:30.691 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:30 +0000 (0:00:00.041) 0:02:46.270 **** 2021-02-10T11:04:30.698 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:30.733 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:30 +0000 (0:00:00.041) 0:02:46.311 **** 2021-02-10T11:04:30.739 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2021-02-10T11:04:30.741 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:30.775 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:30 +0000 (0:00:00.042) 0:02:46.353 **** 2021-02-10T11:04:30.781 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:30.817 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:30 +0000 (0:00:00.042) 0:02:46.396 **** 2021-02-10T11:04:30.831 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2021-02-10T11:04:30.832 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"ansible_facts": {"entitlements_path": "/etc/ansible/secrets/entitlements.yml"}, "changed": false} 2021-02-10T11:04:30.866 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:30 +0000 (0:00:00.049) 0:02:46.445 **** 2021-02-10T11:04:30.874 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"ansible_facts": {"entitlements_path": "/etc/ansible/secrets/entitlements.yml"}, "changed": false} 2021-02-10T11:04:30.908 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:30 +0000 (0:00:00.041) 0:02:46.486 **** 2021-02-10T11:04:30.915 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-02-10T11:04:30.915 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-10T11:04:30.916 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-10T11:04:30.951 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:30 +0000 (0:00:00.043) 0:02:46.530 **** 2021-02-10T11:04:30.957 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-10T11:04:30.958 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-10T11:04:30.992 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:30 +0000 (0:00:00.040) 0:02:46.571 **** 2021-02-10T11:04:30.999 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2021-02-10T11:04:31.001 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"ansible_facts": {"have_entitlements": true}, "changed": false} 2021-02-10T11:04:31.037 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:31 +0000 (0:00:00.044) 0:02:46.616 **** 2021-02-10T11:04:31.043 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"ansible_facts": {"have_entitlements": true}, "changed": false} 2021-02-10T11:04:31.085 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:31 +0000 (0:00:00.047) 0:02:46.663 **** 2021-02-10T11:04:32.716 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-02-10T11:04:32.718 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["katello-ca-consumer-satellite.front.sepia.ceph.com-1.0-3.noarch providing /root/.ansible/tmp/ansible-moduletmp-1612955071.97-e9OZ8D/katello-ca-consumer-latest.noarchMqJxQ8.rpm is already installed"]} 2021-02-10T11:04:32.750 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:32 +0000 (0:00:01.665) 0:02:48.329 **** 2021-02-10T11:04:32.905 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["katello-ca-consumer-satellite.front.sepia.ceph.com-1.0-3.noarch providing /root/.ansible/tmp/ansible-moduletmp-1612955071.99-t4cExk/katello-ca-consumer-latest.noarchGxXut7.rpm is already installed"]} 2021-02-10T11:04:32.942 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:32 +0000 (0:00:00.191) 0:02:48.520 **** 2021-02-10T11:04:33.572 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-02-10T11:04:33.574 INFO:teuthology.task.ansible.out:fatal: [gibba008.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} ...ignoring 2021-02-10T11:04:33.606 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:33 +0000 (0:00:00.664) 0:02:49.185 **** 2021-02-10T11:04:33.644 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-02-10T11:04:33.645 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"ansible_facts": {"rhsm_registered": false}, "changed": false} 2021-02-10T11:04:33.677 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:33 +0000 (0:00:00.070) 0:02:49.255 **** 2021-02-10T11:04:33.789 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-02-10T11:04:33.790 INFO:teuthology.task.ansible.out:fatal: [gibba001.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} ...ignoring 2021-02-10T11:04:33.824 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:33 +0000 (0:00:00.146) 0:02:49.402 **** 2021-02-10T11:04:33.863 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-02-10T11:04:33.864 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"ansible_facts": {"rhsm_registered": false}, "changed": false} 2021-02-10T11:04:33.895 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:33 +0000 (0:00:00.071) 0:02:49.474 **** 2021-02-10T11:04:38.912 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-02-10T11:04:38.913 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"attempts": 1, "censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2021-02-10T11:04:38.947 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:38 +0000 (0:00:05.051) 0:02:54.525 **** 2021-02-10T11:04:38.985 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-02-10T11:04:38.986 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"ansible_facts": {"rhsm_registered": true}, "changed": false} 2021-02-10T11:04:39.020 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:39 +0000 (0:00:00.073) 0:02:54.599 **** 2021-02-10T11:04:39.692 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-02-10T11:04:39.694 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "cmd": "subscription-manager release --list | grep -E '[0-9]'", "delta": "0:00:00.214923", "end": "2021-02-10 11:04:39.671578", "failed_when_result": false, "rc": 0, "start": "2021-02-10 11:04:39.456655", "stderr": "", "stderr_lines": [], "stdout": "7.4\n7.5\n7.6\n7.7\n7.8\n7.9\n7Server\n8\n8.0\n8.1\n8.2\n8.3", "stdout_lines": ["7.4", "7.5", "7.6", "7.7", "7.8", "7.9", "7Server", "8", "8.0", "8.1", "8.2", "8.3"]} 2021-02-10T11:04:39.728 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:39 +0000 (0:00:00.707) 0:02:55.306 **** 2021-02-10T11:04:39.755 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2021-02-10T11:04:39.756 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:39.789 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:39 +0000 (0:00:00.061) 0:02:55.368 **** 2021-02-10T11:04:39.828 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2021-02-10T11:04:39.829 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:39.862 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:39 +0000 (0:00:00.072) 0:02:55.440 **** 2021-02-10T11:04:39.891 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-02-10T11:04:39.893 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-10T11:04:39.925 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:39 +0000 (0:00:00.063) 0:02:55.504 **** 2021-02-10T11:04:39.962 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-02-10T11:04:39.964 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:40.000 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:39 +0000 (0:00:00.074) 0:02:55.579 **** 2021-02-10T11:04:41.776 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2021-02-10T11:04:41.777 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "cmd": "subscription-manager repos --list | grep -B4 'Enabled:.*1' | grep 'Repo ID:' | sed -e 's/Repo ID:\\s*\\(.*\\)/\\1/' | sort", "delta": "0:00:01.318234", "end": "2021-02-10 11:04:41.752461", "rc": 0, "start": "2021-02-10 11:04:40.434227", "stderr": "", "stderr_lines": [], "stdout": "rhel-7-server-rpms", "stdout_lines": ["rhel-7-server-rpms"]} 2021-02-10T11:04:41.813 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:41 +0000 (0:00:01.812) 0:02:57.392 **** 2021-02-10T11:04:41.851 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-02-10T11:04:41.852 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"ansible_facts": {"repo_list": ["rhel-7-server-rpms"]}, "changed": false} 2021-02-10T11:04:41.884 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:41 +0000 (0:00:00.070) 0:02:57.462 **** 2021-02-10T11:04:41.924 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:41.960 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:41 +0000 (0:00:00.076) 0:02:57.539 **** 2021-02-10T11:04:41.968 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-02-10T11:04:41.968 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"attempts": 1, "censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2021-02-10T11:04:42.007 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:42 +0000 (0:00:00.047) 0:02:57.586 **** 2021-02-10T11:04:42.014 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-02-10T11:04:42.016 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"ansible_facts": {"replace_repos": true}, "changed": false} 2021-02-10T11:04:42.052 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:42 +0000 (0:00:00.044) 0:02:57.630 **** 2021-02-10T11:04:42.059 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-02-10T11:04:42.059 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"ansible_facts": {"rhsm_registered": true}, "changed": false} 2021-02-10T11:04:42.095 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:42 +0000 (0:00:00.042) 0:02:57.674 **** 2021-02-10T11:04:42.102 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2021-02-10T11:04:42.103 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"ansible_facts": {"replace_repos": true}, "changed": false} 2021-02-10T11:04:42.139 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:42 +0000 (0:00:00.043) 0:02:57.717 **** 2021-02-10T11:04:42.783 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-02-10T11:04:42.784 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "cmd": "subscription-manager release --list | grep -E '[0-9]'", "delta": "0:00:00.228791", "end": "2021-02-10 11:04:42.759171", "failed_when_result": false, "rc": 0, "start": "2021-02-10 11:04:42.530380", "stderr": "", "stderr_lines": [], "stdout": "7.4\n7.5\n7.6\n7.7\n7.8\n7.9\n7Server\n8\n8.0\n8.1\n8.2\n8.3", "stdout_lines": ["7.4", "7.5", "7.6", "7.7", "7.8", "7.9", "7Server", "8", "8.0", "8.1", "8.2", "8.3"]} 2021-02-10T11:04:42.817 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:42 +0000 (0:00:00.678) 0:02:58.396 **** 2021-02-10T11:04:42.854 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2021-02-10T11:04:42.855 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:42.893 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:42 +0000 (0:00:00.076) 0:02:58.472 **** 2021-02-10T11:04:42.932 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2021-02-10T11:04:42.933 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:42.965 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:42 +0000 (0:00:00.072) 0:02:58.544 **** 2021-02-10T11:04:42.993 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-02-10T11:04:42.995 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-10T11:04:43.027 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:43 +0000 (0:00:00.061) 0:02:58.606 **** 2021-02-10T11:04:43.065 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-02-10T11:04:43.066 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:43.098 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:43 +0000 (0:00:00.070) 0:02:58.676 **** 2021-02-10T11:04:44.333 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-02-10T11:04:44.334 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2021-02-10T11:04:44.372 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:44 +0000 (0:00:01.274) 0:02:59.951 **** 2021-02-10T11:04:44.916 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2021-02-10T11:04:44.917 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "cmd": "subscription-manager repos --list | grep -B4 'Enabled:.*1' | grep 'Repo ID:' | sed -e 's/Repo ID:\\s*\\(.*\\)/\\1/' | sort", "delta": "0:00:01.339944", "end": "2021-02-10 11:04:44.902097", "rc": 0, "start": "2021-02-10 11:04:43.562153", "stderr": "", "stderr_lines": [], "stdout": "rhel-7-server-rpms", "stdout_lines": ["rhel-7-server-rpms"]} 2021-02-10T11:04:44.963 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:44 +0000 (0:00:00.591) 0:03:00.542 **** 2021-02-10T11:04:45.003 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-02-10T11:04:45.142 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"ansible_facts": {"repo_list": ["rhel-7-server-rpms"]}, "changed": false} Wednesday 10 February 2021 11:04:45 +0000 (0:00:00.075) 0:03:00.618 **** TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Wednesday 10 February 2021 11:04:45 +0000 (0:00:00.071) 0:03:00.689 **** 2021-02-10T11:04:45.148 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-02-10T11:04:45.149 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"ansible_facts": {"replace_repos": true}, "changed": false} 2021-02-10T11:04:45.184 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:45 +0000 (0:00:00.073) 0:03:00.763 **** 2021-02-10T11:04:45.232 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2021-02-10T11:04:45.233 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"ansible_facts": {"replace_repos": true}, "changed": false} 2021-02-10T11:04:45.265 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:45 +0000 (0:00:00.080) 0:03:00.844 **** 2021-02-10T11:04:46.853 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2021-02-10T11:04:46.854 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "cmd": ["subscription-manager", "repos", "--enable=rhel-7-server-rpms", "--enable=rhel-7-server-optional-rpms", "--enable=rhel-7-server-extras-rpms", "--enable=rhel-ha-for-rhel-7-server-rpms"], "delta": "0:00:02.018811", "end": "2021-02-10 11:04:46.832943", "rc": 0, "start": "2021-02-10 11:04:44.814132", "stderr": "", "stderr_lines": [], "stdout": "Repository 'rhel-7-server-rpms' is enabled for this system.\nRepository 'rhel-7-server-optional-rpms' is enabled for this system.\nRepository 'rhel-ha-for-rhel-7-server-rpms' is enabled for this system.\nRepository 'rhel-7-server-extras-rpms' is enabled for this system.", "stdout_lines": ["Repository 'rhel-7-server-rpms' is enabled for this system.", "Repository 'rhel-7-server-optional-rpms' is enabled for this system.", "Repository 'rhel-ha-for-rhel-7-server-rpms' is enabled for this system.", "Repository 'rhel-7-server-extras-rpms' is enabled for this system."]} 2021-02-10T11:04:46.889 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:46 +0000 (0:00:01.623) 0:03:02.467 **** 2021-02-10T11:04:47.356 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-02-10T11:04:47.357 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "path": "/etc/yum.repos.d/cd.repo", "state": "absent"} 2021-02-10T11:04:47.389 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:47 +0000 (0:00:00.500) 0:03:02.968 **** 2021-02-10T11:04:47.405 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-02-10T11:04:47.407 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2021-02-10T11:04:47.440 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:47 +0000 (0:00:00.050) 0:03:03.019 **** 2021-02-10T11:04:47.842 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2021-02-10T11:04:47.844 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-10T11:04:47.880 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:47 +0000 (0:00:00.440) 0:03:03.459 **** 2021-02-10T11:04:49.822 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-02-10T11:04:49.824 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2021-02-10T11:04:49.951 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2021-02-10T11:04:49.952 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "cmd": ["subscription-manager", "repos", "--enable=rhel-7-server-rpms", "--enable=rhel-7-server-optional-rpms", "--enable=rhel-7-server-extras-rpms", "--enable=rhel-ha-for-rhel-7-server-rpms"], "delta": "0:00:02.060990", "end": "2021-02-10 11:04:49.937653", "rc": 0, "start": "2021-02-10 11:04:47.876663", "stderr": "", "stderr_lines": [], "stdout": "Repository 'rhel-7-server-rpms' is enabled for this system.\nRepository 'rhel-7-server-optional-rpms' is enabled for this system.\nRepository 'rhel-ha-for-rhel-7-server-rpms' is enabled for this system.\nRepository 'rhel-7-server-extras-rpms' is enabled for this system.", "stdout_lines": ["Repository 'rhel-7-server-rpms' is enabled for this system.", "Repository 'rhel-7-server-optional-rpms' is enabled for this system.", "Repository 'rhel-ha-for-rhel-7-server-rpms' is enabled for this system.", "Repository 'rhel-7-server-extras-rpms' is enabled for this system."]} 2021-02-10T11:04:49.985 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:49 +0000 (0:00:02.105) 0:03:05.564 **** 2021-02-10T11:04:50.469 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-02-10T11:04:50.470 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "path": "/etc/yum.repos.d/cd.repo", "state": "absent"} 2021-02-10T11:04:50.510 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:50 +0000 (0:00:00.524) 0:03:06.088 **** 2021-02-10T11:04:50.976 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2021-02-10T11:04:50.978 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-10T11:04:51.019 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:51 +0000 (0:00:00.509) 0:03:06.598 **** 2021-02-10T11:04:51.787 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-02-10T11:04:51.789 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2021-02-10T11:04:51.823 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:51 +0000 (0:00:00.804) 0:03:07.402 **** 2021-02-10T11:04:52.969 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2021-02-10T11:04:53.751 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-02-10T11:04:53.752 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "state": "file", "uid": 0} 2021-02-10T11:04:54.885 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-02-10T11:04:54.886 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2021-02-10T11:04:54.930 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:54 +0000 (0:00:03.107) 0:03:10.509 **** 2021-02-10T11:04:55.670 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-02-10T11:04:55.671 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "state": "file", "uid": 0} 2021-02-10T11:04:55.708 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:55 +0000 (0:00:00.777) 0:03:11.287 **** 2021-02-10T11:04:55.747 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-02-10T11:04:55.748 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:55.782 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:55 +0000 (0:00:00.073) 0:03:11.361 **** 2021-02-10T11:04:55.809 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-02-10T11:04:55.811 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:55.845 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:55 +0000 (0:00:00.063) 0:03:11.424 **** 2021-02-10T11:04:55.873 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2021-02-10T11:04:55.874 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:55.906 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:55 +0000 (0:00:00.061) 0:03:11.485 **** 2021-02-10T11:04:55.933 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-02-10T11:04:55.935 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:55.966 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:55 +0000 (0:00:00.060) 0:03:11.545 **** 2021-02-10T11:04:56.004 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-02-10T11:04:56.005 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "skip_reason": "Conditional result was False"} 2021-02-10T11:04:56.016 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "skip_reason": "Conditional result was False"} 2021-02-10T11:04:56.017 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "skip_reason": "Conditional result was False"} 2021-02-10T11:04:56.058 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:56 +0000 (0:00:00.091) 0:03:11.637 **** 2021-02-10T11:04:56.087 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-02-10T11:04:56.088 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:56.120 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:56 +0000 (0:00:00.061) 0:03:11.698 **** 2021-02-10T11:04:56.146 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2021-02-10T11:04:56.148 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:56.179 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:56 +0000 (0:00:00.059) 0:03:11.758 **** 2021-02-10T11:04:56.206 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2021-02-10T11:04:56.208 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:56.239 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:56 +0000 (0:00:00.060) 0:03:11.818 **** 2021-02-10T11:04:56.268 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2021-02-10T11:04:56.269 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:56.301 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:56 +0000 (0:00:00.061) 0:03:11.880 **** 2021-02-10T11:04:56.329 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2021-02-10T11:04:56.331 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:56.362 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:56 +0000 (0:00:00.060) 0:03:11.941 **** 2021-02-10T11:04:56.879 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-02-10T11:04:56.880 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "state": "file", "uid": 0} 2021-02-10T11:04:57.514 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-02-10T11:04:57.516 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.704147", "end": "2021-02-10 11:04:57.500643", "rc": 0, "start": "2021-02-10 11:04:56.796496", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-10T11:04:57.561 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:57 +0000 (0:00:01.198) 0:03:13.139 **** 2021-02-10T11:04:58.399 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-02-10T11:04:58.401 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-50.el7.x86_64 providing krb5-workstation is already installed"]} 2021-02-10T11:04:58.435 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:58 +0000 (0:00:00.874) 0:03:14.014 **** 2021-02-10T11:04:58.474 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-02-10T11:04:58.476 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:58.511 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:58 +0000 (0:00:00.075) 0:03:14.089 **** 2021-02-10T11:04:58.537 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2021-02-10T11:04:58.539 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:58.572 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:58 +0000 (0:00:00.061) 0:03:14.151 **** 2021-02-10T11:04:58.601 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-02-10T11:04:58.603 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:58.657 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:58 +0000 (0:00:00.085) 0:03:14.236 **** 2021-02-10T11:04:58.686 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-02-10T11:04:58.687 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:58.722 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:58 +0000 (0:00:00.064) 0:03:14.300 **** 2021-02-10T11:04:58.813 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-02-10T11:04:58.813 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "state": "file", "uid": 0} 2021-02-10T11:04:58.856 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:58 +0000 (0:00:00.133) 0:03:14.434 **** 2021-02-10T11:04:58.884 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-02-10T11:04:58.885 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:58.919 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:58 +0000 (0:00:00.062) 0:03:14.497 **** 2021-02-10T11:04:58.946 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-02-10T11:04:58.948 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:58.983 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:58 +0000 (0:00:00.064) 0:03:14.562 **** 2021-02-10T11:04:59.012 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2021-02-10T11:04:59.014 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:59.051 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:59 +0000 (0:00:00.068) 0:03:14.630 **** 2021-02-10T11:04:59.078 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-02-10T11:04:59.079 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:59.114 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:59 +0000 (0:00:00.062) 0:03:14.693 **** 2021-02-10T11:04:59.153 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-02-10T11:04:59.154 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "skip_reason": "Conditional result was False"} 2021-02-10T11:04:59.155 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "skip_reason": "Conditional result was False"} 2021-02-10T11:04:59.166 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "skip_reason": "Conditional result was False"} 2021-02-10T11:04:59.167 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "skip_reason": "Conditional result was False"} 2021-02-10T11:04:59.203 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:59 +0000 (0:00:00.088) 0:03:14.782 **** 2021-02-10T11:04:59.230 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:59.264 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:59 +0000 (0:00:00.061) 0:03:14.843 **** 2021-02-10T11:04:59.292 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2021-02-10T11:04:59.293 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:59.326 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:59 +0000 (0:00:00.061) 0:03:14.905 **** 2021-02-10T11:04:59.353 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2021-02-10T11:04:59.355 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:59.386 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:59 +0000 (0:00:00.060) 0:03:14.965 **** 2021-02-10T11:04:59.414 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2021-02-10T11:04:59.416 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:59.449 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:59 +0000 (0:00:00.062) 0:03:15.027 **** 2021-02-10T11:04:59.477 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2021-02-10T11:04:59.478 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:04:59.514 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:04:59 +0000 (0:00:00.065) 0:03:15.093 **** 2021-02-10T11:05:00.636 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2021-02-10T11:05:00.638 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2021-02-10T11:05:00.671 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:00 +0000 (0:00:01.157) 0:03:16.250 **** 2021-02-10T11:05:01.514 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-02-10T11:05:01.516 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:01.567459", "end": "2021-02-10 11:05:01.500859", "rc": 0, "start": "2021-02-10 11:04:59.933400", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-10T11:05:01.550 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:01 +0000 (0:00:00.879) 0:03:17.129 **** 2021-02-10T11:05:02.369 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-02-10T11:05:02.371 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-50.el7.x86_64 providing krb5-workstation is already installed"]} 2021-02-10T11:05:02.402 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:02 +0000 (0:00:00.852) 0:03:17.981 **** 2021-02-10T11:05:02.428 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-02-10T11:05:02.430 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:05:02.462 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:02 +0000 (0:00:00.059) 0:03:18.040 **** 2021-02-10T11:05:02.489 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2021-02-10T11:05:02.490 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:05:02.521 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:02 +0000 (0:00:00.059) 0:03:18.100 **** 2021-02-10T11:05:02.547 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-02-10T11:05:02.549 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:05:02.580 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:02 +0000 (0:00:00.058) 0:03:18.158 **** 2021-02-10T11:05:02.606 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-02-10T11:05:02.607 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:05:02.640 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:02 +0000 (0:00:00.060) 0:03:18.219 **** 2021-02-10T11:05:04.551 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2021-02-10T11:05:04.553 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2021-02-10T11:05:04.587 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:04 +0000 (0:00:01.947) 0:03:20.166 **** 2021-02-10T11:05:28.523 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing smartmontools are up to date", ""]} 2021-02-10T11:05:28.567 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:28 +0000 (0:00:23.979) 0:03:44.146 **** 2021-02-10T11:05:30.588 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-02-10T11:05:30.589 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2021-02-10T11:05:32.071 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2021-02-10T11:05:32.072 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing smartmontools are up to date", ""]} 2021-02-10T11:05:32.109 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:32 +0000 (0:00:03.541) 0:03:47.687 **** 2021-02-10T11:05:32.507 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-02-10T11:05:32.508 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2021-02-10T11:05:34.092 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2021-02-10T11:05:34.417 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2021-02-10T11:05:34.457 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:34 +0000 (0:00:02.348) 0:03:50.036 **** 2021-02-10T11:05:34.909 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-02-10T11:05:34.910 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-02-10T11:05:34.943 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:34 +0000 (0:00:00.485) 0:03:50.522 **** 2021-02-10T11:05:35.997 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-02-10T11:05:35.998 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2021-02-10T11:05:37.881 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2021-02-10T11:05:37.930 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:37 +0000 (0:00:02.986) 0:03:53.509 **** 2021-02-10T11:05:38.029 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-02-10T11:05:38.030 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": true, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "md5sum": "14041bf2666d8f8d277567958c7e9f69", "mode": "0755", "owner": "root", "secontext": "system_u:object_r:bin_t:s0", "size": 13355, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1612955134.979827-233195598704630/source", "state": "file", "uid": 0} 2021-02-10T11:05:38.378 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-02-10T11:05:38.379 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-02-10T11:05:38.412 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:38 +0000 (0:00:00.482) 0:03:53.991 **** 2021-02-10T11:05:39.917 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-02-10T11:05:39.918 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2021-02-10T11:05:41.151 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": true, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "md5sum": "14041bf2666d8f8d277567958c7e9f69", "mode": "0755", "owner": "root", "secontext": "system_u:object_r:bin_t:s0", "size": 13355, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1612955138.4396098-126658643892865/source", "state": "file", "uid": 0} 2021-02-10T11:05:41.810 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2021-02-10T11:05:41.846 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:41 +0000 (0:00:03.433) 0:03:57.425 **** 2021-02-10T11:05:41.874 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-02-10T11:05:41.875 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:05:41.908 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:41 +0000 (0:00:00.061) 0:03:57.487 **** 2021-02-10T11:05:41.948 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2021-02-10T11:05:41.950 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2021-02-10T11:05:41.981 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:41 +0000 (0:00:00.072) 0:03:57.559 **** 2021-02-10T11:05:42.434 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2021-02-10T11:05:42.436 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2021-02-10T11:05:42.473 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:42 +0000 (0:00:00.491) 0:03:58.051 **** 2021-02-10T11:05:42.709 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** skipping: [gibba001.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-10T11:05:42.884 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-10T11:05:42.921 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:42 +0000 (0:00:00.448) 0:03:58.500 **** 2021-02-10T11:05:43.080 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-02-10T11:05:43.082 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2021-02-10T11:05:46.039 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2021-02-10T11:05:46.075 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:46 +0000 (0:00:03.154) 0:04:01.654 **** 2021-02-10T11:05:46.103 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-02-10T11:05:46.105 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:05:46.143 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:46 +0000 (0:00:00.067) 0:04:01.721 **** 2021-02-10T11:05:46.183 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** ok: [gibba008.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2021-02-10T11:05:46.226 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:46 +0000 (0:00:00.083) 0:04:01.805 **** 2021-02-10T11:05:46.680 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2021-02-10T11:05:46.681 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2021-02-10T11:05:46.718 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:46 +0000 (0:00:00.491) 0:04:02.297 **** 2021-02-10T11:05:46.779 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-02-10T11:05:46.781 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-10T11:05:46.791 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-10T11:05:46.812 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-10T11:05:46.814 INFO:teuthology.task.ansible.out: 2021-02-10T11:05:46.823 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-10T11:05:46.824 INFO:teuthology.task.ansible.out: 2021-02-10T11:05:46.858 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:46 +0000 (0:00:00.140) 0:04:02.437 **** 2021-02-10T11:05:46.860 INFO:teuthology.task.ansible.out: 2021-02-10T11:05:47.469 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-02-10T11:05:47.471 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing nagios-common are up to date", "All packages providing nrpe are up to date", "All packages providing nagios-plugins are up to date", "All packages providing nagios-plugins-load are up to date", ""]} 2021-02-10T11:05:47.503 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:47 +0000 (0:00:00.644) 0:04:03.082 **** 2021-02-10T11:05:47.541 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2021-02-10T11:05:47.543 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:05:47.574 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:47 +0000 (0:00:00.071) 0:04:03.153 **** 2021-02-10T11:05:49.535 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2021-02-10T11:05:49.537 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2021-02-10T11:05:49.587 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:49 +0000 (0:00:02.012) 0:04:05.165 **** 2021-02-10T11:05:51.452 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-02-10T11:05:51.453 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing nagios-common are up to date", "All packages providing nrpe are up to date", "All packages providing nagios-plugins are up to date", "All packages providing nagios-plugins-load are up to date", ""]} 2021-02-10T11:05:51.501 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:51 +0000 (0:00:01.913) 0:04:07.079 **** 2021-02-10T11:05:51.519 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-02-10T11:05:51.521 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2021-02-10T11:05:51.554 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:51 +0000 (0:00:00.053) 0:04:07.133 **** 2021-02-10T11:05:51.561 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2021-02-10T11:05:51.563 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:05:51.599 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:51 +0000 (0:00:00.045) 0:04:07.178 **** 2021-02-10T11:05:51.607 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-02-10T11:05:51.608 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2021-02-10T11:05:51.666 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:51 +0000 (0:00:00.066) 0:04:07.245 **** 2021-02-10T11:05:51.668 INFO:teuthology.task.ansible.out: 2021-02-10T11:05:52.121 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-02-10T11:05:52.121 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-10T11:05:52.159 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:52 +0000 (0:00:00.493) 0:04:07.738 **** 2021-02-10T11:05:52.651 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2021-02-10T11:05:52.652 INFO:teuthology.task.ansible.out:fatal: [gibba001.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2021-02-10T11:05:52.686 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:52 +0000 (0:00:00.526) 0:04:08.264 **** 2021-02-10T11:05:52.713 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-02-10T11:05:52.714 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:05:52.746 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:52 +0000 (0:00:00.060) 0:04:08.324 **** 2021-02-10T11:05:53.548 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2021-02-10T11:05:53.550 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2021-02-10T11:05:53.582 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:53 +0000 (0:00:00.836) 0:04:09.161 **** 2021-02-10T11:05:54.673 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2021-02-10T11:05:54.674 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "checksum": "aa9150d4582f919d7a8504c01c8d86f05e3eeb5d", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1184, "state": "file", "uid": 0} 2021-02-10T11:05:54.714 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:54 +0000 (0:00:01.131) 0:04:10.293 **** 2021-02-10T11:05:55.512 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-02-10T11:05:55.514 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2021-02-10T11:05:55.546 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:55 +0000 (0:00:00.831) 0:04:11.125 **** 2021-02-10T11:05:55.584 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-02-10T11:05:55.585 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} 2021-02-10T11:05:55.595 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2021-02-10T11:05:55.630 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:55 +0000 (0:00:00.084) 0:04:11.209 **** 2021-02-10T11:05:55.773 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-02-10T11:05:55.774 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Wed 2021-02-10 11:00:55 UTC", "ActiveEnterTimestampMonotonic": "19584463", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target nss-lookup.target system.slice network-online.target network.target systemd-journald.socket time-sync.target var-run.mount local-fs.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Wed 2021-02-10 11:00:55 UTC", "AssertTimestampMonotonic": "19576120", "Before": "multi-user.target getty@tty1.service xdm.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Wed 2021-02-10 11:00:55 UTC", "ConditionTimestampMonotonic": "19576118", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1197", "ExecMainStartTimestamp": "Wed 2021-02-10 11:00:55 UTC", "ExecMainStartTimestampMonotonic": "19584407", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /e 2021-02-10T11:05:55.775 INFO:teuthology.task.ansible.out:tc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_errors=no ; start_time=[Wed 2021-02-10 11:00:55 UTC] ; stop_time=[n/a] ; pid=1197 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Wed 2021-02-10 11:00:55 UTC", "InactiveExitTimestampMonotonic": "19584463", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127560", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127560", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "1197", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "system.slice basic.target", "Restart": "no", "RestartUSec": "100ms", "Re 2021-02-10T11:05:55.777 INFO:teuthology.task.ansible.out:sult": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Wed 2021-02-10 11:00:55 UTC", "WatchdogTimestampMonotonic": "19584440", "WatchdogUSec": "0"}} 2021-02-10T11:05:55.808 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:55 +0000 (0:00:00.177) 0:04:11.387 **** 2021-02-10T11:05:56.086 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-02-10T11:05:56.088 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-10T11:05:56.122 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:56 +0000 (0:00:00.314) 0:04:11.701 **** 2021-02-10T11:05:56.257 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-02-10T11:05:56.258 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001328", "end": "2021-02-10 11:05:56.237973", "rc": 0, "start": "2021-02-10 11:05:56.236645", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2021-02-10T11:05:56.291 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:56 +0000 (0:00:00.168) 0:04:11.870 **** 2021-02-10T11:05:56.611 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2021-02-10T11:05:56.613 INFO:teuthology.task.ansible.out:fatal: [gibba008.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2021-02-10T11:05:56.647 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:56 +0000 (0:00:00.356) 0:04:12.226 **** 2021-02-10T11:05:56.675 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-02-10T11:05:56.676 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:05:56.710 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:56 +0000 (0:00:00.062) 0:04:12.289 **** 2021-02-10T11:05:57.120 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-02-10T11:05:57.121 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-34.el7.x86_64 providing policycoreutils-python is already installed"]} 2021-02-10T11:05:57.156 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:57 +0000 (0:00:00.446) 0:04:12.735 **** 2021-02-10T11:05:58.638 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2021-02-10T11:05:58.640 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "checksum": "aa9150d4582f919d7a8504c01c8d86f05e3eeb5d", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1184, "state": "file", "uid": 0} 2021-02-10T11:05:58.679 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:58 +0000 (0:00:01.522) 0:04:14.257 **** 2021-02-10T11:05:59.196 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-02-10T11:05:59.196 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Wed 2021-02-10 11:00:51 UTC", "ActiveEnterTimestampMonotonic": "19855128", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "time-sync.target network-online.target network.target nss-lookup.target var-run.mount systemd-journald.socket local-fs.target basic.target system.slice", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Wed 2021-02-10 11:00:51 UTC", "AssertTimestampMonotonic": "19811920", "Before": "xdm.service getty@tty1.service multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Wed 2021-02-10 11:00:51 UTC", "ConditionTimestampMonotonic": "19811917", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1199", "ExecMainStartTimestamp": "Wed 2021-02-10 11:00:51 UTC", "ExecMainStartTimestampMonotonic": "19854728", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /e 2021-02-10T11:05:59.198 INFO:teuthology.task.ansible.out:tc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_errors=no ; start_time=[Wed 2021-02-10 11:00:51 UTC] ; stop_time=[n/a] ; pid=1199 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Wed 2021-02-10 11:00:51 UTC", "InactiveExitTimestampMonotonic": "19855128", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127560", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127560", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "1199", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target system.slice", "Restart": "no", "RestartUSec": "100ms", "Re 2021-02-10T11:05:59.199 INFO:teuthology.task.ansible.out:sult": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Wed 2021-02-10 11:00:51 UTC", "WatchdogTimestampMonotonic": "19855104", "WatchdogUSec": "0"}} 2021-02-10T11:05:59.230 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:59 +0000 (0:00:00.551) 0:04:14.809 **** 2021-02-10T11:05:59.686 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-02-10T11:05:59.687 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001427", "end": "2021-02-10 11:05:59.663675", "rc": 0, "start": "2021-02-10 11:05:59.662248", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2021-02-10T11:05:59.734 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:05:59 +0000 (0:00:00.503) 0:04:15.312 **** 2021-02-10T11:06:00.556 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-02-10T11:06:00.557 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-34.el7.x86_64 providing policycoreutils-python is already installed"]} 2021-02-10T11:06:00.592 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:00 +0000 (0:00:00.858) 0:04:16.171 **** 2021-02-10T11:06:01.706 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2021-02-10T11:06:01.708 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing selinux-policy-targeted are up to date", ""]} 2021-02-10T11:06:01.748 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:01 +0000 (0:00:01.155) 0:04:17.327 **** 2021-02-10T11:06:03.279 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2021-02-10T11:06:03.281 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2021-02-10T11:06:03.315 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:03 +0000 (0:00:01.566) 0:04:18.894 **** 2021-02-10T11:06:05.408 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2021-02-10T11:06:05.410 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing selinux-policy-targeted are up to date", ""]} 2021-02-10T11:06:05.444 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:05 +0000 (0:00:02.129) 0:04:21.023 **** 2021-02-10T11:06:06.285 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2021-02-10T11:06:06.287 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2021-02-10T11:06:06.319 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:06 +0000 (0:00:00.874) 0:04:21.897 **** 2021-02-10T11:06:09.706 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-02-10T11:06:09.707 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:05.936446", "end": "2021-02-10 11:06:09.690958", "failed_when_result": false, "rc": 0, "start": "2021-02-10 11:06:03.754512", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2021-02-10T11:06:09.742 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:09 +0000 (0:00:03.423) 0:04:25.321 **** 2021-02-10T11:06:11.678 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2021-02-10T11:06:11.679 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2021-02-10T11:06:11.711 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:11 +0000 (0:00:01.968) 0:04:27.290 **** 2021-02-10T11:06:12.180 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-02-10T11:06:12.181 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.028314", "end": "2021-02-10 11:06:12.165169", "rc": 0, "start": "2021-02-10 11:06:12.136855", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2021-02-10T11:06:12.216 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:12 +0000 (0:00:00.504) 0:04:27.794 **** 2021-02-10T11:06:12.391 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-02-10T11:06:12.393 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:05.626708", "end": "2021-02-10 11:06:12.377095", "failed_when_result": false, "rc": 0, "start": "2021-02-10 11:06:06.750387", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2021-02-10T11:06:12.427 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:12 +0000 (0:00:00.211) 0:04:28.006 **** 2021-02-10T11:06:12.707 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2021-02-10T11:06:12.709 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.045365", "end": "2021-02-10 11:06:12.690235", "rc": 0, "start": "2021-02-10 11:06:12.644870", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-10T11:06:12.757 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:12 +0000 (0:00:00.329) 0:04:28.336 **** 2021-02-10T11:06:14.360 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2021-02-10T11:06:14.361 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2021-02-10T11:06:14.397 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:14 +0000 (0:00:01.639) 0:04:29.976 **** 2021-02-10T11:06:14.853 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-02-10T11:06:14.855 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.012454", "end": "2021-02-10 11:06:14.839412", "rc": 0, "start": "2021-02-10 11:06:14.826958", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2021-02-10T11:06:14.886 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:14 +0000 (0:00:00.489) 0:04:30.465 **** 2021-02-10T11:06:15.339 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2021-02-10T11:06:15.341 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.006174", "end": "2021-02-10 11:06:15.323789", "rc": 0, "start": "2021-02-10 11:06:15.317615", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-10T11:06:15.373 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:15 +0000 (0:00:00.486) 0:04:30.952 **** 2021-02-10T11:06:18.821 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:05.610935", "end": "2021-02-10 11:06:18.794380", "rc": 0, "start": "2021-02-10 11:06:13.183445", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-10T11:06:18.858 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:18 +0000 (0:00:03.485) 0:04:34.437 **** 2021-02-10T11:06:19.331 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2021-02-10T11:06:19.332 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2021-02-10T11:06:19.364 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:19 +0000 (0:00:00.505) 0:04:34.943 **** 2021-02-10T11:06:19.393 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2021-02-10T11:06:19.394 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2021-02-10T11:06:19.427 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:19 +0000 (0:00:00.063) 0:04:35.006 **** 2021-02-10T11:06:19.475 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-02-10T11:06:19.477 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-02-10T11:06:19.510 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:19 +0000 (0:00:00.082) 0:04:35.089 **** 2021-02-10T11:06:19.558 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-02-10T11:06:19.559 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/redhat_7.yml) => {"ansible_facts": {"common_yum_repos": {"lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/rhel7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}, "rhel-7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-rhel7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "RHEL 7 Local fastcgi Repo"}}, "epel_packages": ["cryptopp-devel", "cryptopp", "dbench", "fcgi", "fuse-sshfs", "perl-JSON-XS", "leveldb", "xmlstarlet"], "nfs_service": "nfs-server", "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "xfsprogs-devel", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "lvm2", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "ant", "lsof", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN", "podman", "podman-docker", "python3"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/redhat_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/redhat_7.yml"} 2021-02-10T11:06:19.592 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:19 +0000 (0:00:00.082) 0:04:35.171 **** 2021-02-10T11:06:19.629 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-02-10T11:06:19.632 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-02-10T11:06:19.666 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:19 +0000 (0:00:00.073) 0:04:35.245 **** 2021-02-10T11:06:20.119 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-02-10T11:06:20.121 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2021-02-10T11:06:20.153 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:20 +0000 (0:00:00.487) 0:04:35.732 **** 2021-02-10T11:06:20.601 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-02-10T11:06:20.602 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2021-02-10T11:06:20.635 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:20 +0000 (0:00:00.481) 0:04:36.213 **** 2021-02-10T11:06:20.662 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-02-10T11:06:20.663 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:06:20.698 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:20 +0000 (0:00:00.062) 0:04:36.276 **** 2021-02-10T11:06:20.725 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-02-10T11:06:20.726 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:06:20.772 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:20 +0000 (0:00:00.074) 0:04:36.351 **** 2021-02-10T11:06:21.239 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-02-10T11:06:21.241 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2021-02-10T11:06:21.276 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:21 +0000 (0:00:00.503) 0:04:36.854 **** 2021-02-10T11:06:21.366 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-02-10T11:06:21.367 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:05.539797", "end": "2021-02-10 11:06:21.346513", "rc": 0, "start": "2021-02-10 11:06:15.806716", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-10T11:06:21.403 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:21 +0000 (0:00:00.126) 0:04:36.981 **** 2021-02-10T11:06:21.867 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2021-02-10T11:06:21.868 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2021-02-10T11:06:21.904 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:21 +0000 (0:00:00.501) 0:04:37.483 **** 2021-02-10T11:06:21.944 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2021-02-10T11:06:21.945 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2021-02-10T11:06:21.983 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:21 +0000 (0:00:00.078) 0:04:37.562 **** 2021-02-10T11:06:22.032 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-02-10T11:06:22.033 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-02-10T11:06:22.070 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:22 +0000 (0:00:00.086) 0:04:37.648 **** 2021-02-10T11:06:22.119 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-02-10T11:06:22.120 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/redhat_7.yml) => {"ansible_facts": {"common_yum_repos": {"lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/rhel7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}, "rhel-7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-rhel7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "RHEL 7 Local fastcgi Repo"}}, "epel_packages": ["cryptopp-devel", "cryptopp", "dbench", "fcgi", "fuse-sshfs", "perl-JSON-XS", "leveldb", "xmlstarlet"], "nfs_service": "nfs-server", "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "xfsprogs-devel", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "lvm2", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "ant", "lsof", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN", "podman", "podman-docker", "python3"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/redhat_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/redhat_7.yml"} 2021-02-10T11:06:22.156 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:22 +0000 (0:00:00.086) 0:04:37.735 **** 2021-02-10T11:06:22.194 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-02-10T11:06:22.195 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-02-10T11:06:22.227 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:22 +0000 (0:00:00.071) 0:04:37.806 **** 2021-02-10T11:06:22.672 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-02-10T11:06:22.673 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2021-02-10T11:06:22.704 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:22 +0000 (0:00:00.476) 0:04:38.283 **** 2021-02-10T11:06:23.149 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-02-10T11:06:23.151 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2021-02-10T11:06:23.184 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:23 +0000 (0:00:00.479) 0:04:38.762 **** 2021-02-10T11:06:23.211 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-02-10T11:06:23.212 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:06:23.249 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:23 +0000 (0:00:00.065) 0:04:38.827 **** 2021-02-10T11:06:23.267 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-02-10T11:06:23.269 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2021-02-10T11:06:23.303 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:23 +0000 (0:00:00.054) 0:04:38.882 **** 2021-02-10T11:06:23.310 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-02-10T11:06:23.311 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:06:23.345 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:23 +0000 (0:00:00.041) 0:04:38.923 **** 2021-02-10T11:06:23.351 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-02-10T11:06:23.351 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "gibba001"}, "changed": false} 2021-02-10T11:06:23.387 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:23 +0000 (0:00:00.042) 0:04:38.965 **** 2021-02-10T11:06:23.808 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-02-10T11:06:23.810 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2021-02-10T11:06:23.848 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:23 +0000 (0:00:00.461) 0:04:39.427 **** 2021-02-10T11:06:24.472 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2021-02-10T11:06:24.473 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "gibba001", "ansible_hostname": "gibba001", "ansible_nodename": "gibba001"}, "changed": false, "name": "gibba001"} 2021-02-10T11:06:24.505 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:24 +0000 (0:00:00.657) 0:04:40.084 **** 2021-02-10T11:06:25.841 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-02-10T11:06:25.842 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2021-02-10T11:06:25.878 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:25 +0000 (0:00:01.372) 0:04:41.457 **** 2021-02-10T11:06:25.915 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* ok: [gibba008.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "gibba008"}, "changed": false} 2021-02-10T11:06:25.949 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:25 +0000 (0:00:00.070) 0:04:41.527 **** 2021-02-10T11:06:26.415 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2021-02-10T11:06:26.416 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2021-02-10T11:06:26.449 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:26 +0000 (0:00:00.500) 0:04:42.027 **** 2021-02-10T11:06:26.509 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2021-02-10T11:06:26.510 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "gibba008", "ansible_hostname": "gibba008", "ansible_nodename": "gibba008"}, "changed": false, "name": "gibba008"} 2021-02-10T11:06:26.546 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:26 +0000 (0:00:00.097) 0:04:42.125 **** 2021-02-10T11:06:28.350 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2021-02-10T11:06:28.352 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2021-02-10T11:06:28.385 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:28 +0000 (0:00:01.838) 0:04:43.964 **** 2021-02-10T11:06:28.446 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2021-02-10T11:06:28.447 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2021-02-10T11:06:28.490 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:28 +0000 (0:00:00.105) 0:04:44.069 **** 2021-02-10T11:06:29.138 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-02-10T11:06:29.140 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-02-10T11:06:29.174 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:29 +0000 (0:00:00.683) 0:04:44.753 **** 2021-02-10T11:06:30.431 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2021-02-10T11:06:30.432 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2021-02-10T11:06:30.468 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:30 +0000 (0:00:01.293) 0:04:46.046 **** 2021-02-10T11:06:31.140 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-02-10T11:06:31.142 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2021-02-10T11:06:31.175 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:31 +0000 (0:00:00.707) 0:04:46.754 **** 2021-02-10T11:06:31.237 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-02-10T11:06:31.238 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-02-10T11:06:31.274 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:31 +0000 (0:00:00.098) 0:04:46.853 **** 2021-02-10T11:06:31.624 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-02-10T11:06:31.626 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2021-02-10T11:06:31.660 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:31 +0000 (0:00:00.385) 0:04:47.239 **** 2021-02-10T11:06:32.107 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-02-10T11:06:32.108 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "kvm", "state": "present", "system": false} 2021-02-10T11:06:32.141 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:32 +0000 (0:00:00.481) 0:04:47.720 **** 2021-02-10T11:06:32.605 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-02-10T11:06:32.607 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-02-10T11:06:32.638 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:32 +0000 (0:00:00.497) 0:04:48.217 **** 2021-02-10T11:06:33.198 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-02-10T11:06:33.199 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2021-02-10T11:06:33.237 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:33 +0000 (0:00:00.598) 0:04:48.815 **** 2021-02-10T11:06:33.696 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-02-10T11:06:33.697 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2021-02-10T11:06:33.729 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:33 +0000 (0:00:00.491) 0:04:49.307 **** 2021-02-10T11:06:34.175 INFO:teuthology.task.ansible.out: 2021-02-10T11:06:34.176 INFO:teuthology.task.ansible.out:TASK [testnode : Ensure the group kvm exists.] ********************************* ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "kvm", "state": "present", "system": false} 2021-02-10T11:06:34.209 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:34 +0000 (0:00:00.480) 0:04:49.788 **** 2021-02-10T11:06:34.573 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2021-02-10T11:06:34.609 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:34 +0000 (0:00:00.399) 0:04:50.187 **** 2021-02-10T11:06:34.668 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-02-10T11:06:34.670 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-02-10T11:06:34.704 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:34 +0000 (0:00:00.095) 0:04:50.283 **** 2021-02-10T11:06:36.537 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-02-10T11:06:36.538 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2021-02-10T11:06:36.574 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:36 +0000 (0:00:01.869) 0:04:52.153 **** 2021-02-10T11:06:36.634 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-02-10T11:06:36.634 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2021-02-10T11:06:36.672 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:36 +0000 (0:00:00.098) 0:04:52.251 **** 2021-02-10T11:06:38.604 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-02-10T11:06:38.606 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2021-02-10T11:06:38.638 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:38 +0000 (0:00:01.966) 0:04:54.217 **** 2021-02-10T11:06:38.666 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-02-10T11:06:38.667 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'. If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:01.650420", "end": "2021-02-10 11:06:38.650758", "rc": 0, "start": "2021-02-10 11:06:37.000338", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-10T11:06:38.701 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:38 +0000 (0:00:00.062) 0:04:54.279 **** 2021-02-10T11:06:38.728 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-02-10T11:06:38.730 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:06:38.762 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:38 +0000 (0:00:00.061) 0:04:54.341 **** 2021-02-10T11:06:40.649 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-02-10T11:06:40.649 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:01.561438", "end": "2021-02-10 11:06:40.628327", "rc": 0, "start": "2021-02-10 11:06:39.066889", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-10T11:06:40.685 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:40 +0000 (0:00:01.922) 0:04:56.264 **** 2021-02-10T11:06:40.724 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-02-10T11:06:40.726 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-02-10T11:06:40.727 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'key': 'rhel-7-fcgi-ceph', 'value': {'name': 'RHEL 7 Local fastcgi Repo', 'baseurl': 'http://gitbuilder.ceph.com/mod_fastcgi-rpm-rhel7-x86_64-basic/ref/master/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "e6aba9c9ab05e27255ed430f065502af284a869b", "dest": "/etc/yum.repos.d/rhel-7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "rhel-7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-rhel7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "RHEL 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/rhel-7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 245, "state": "file", "uid": 0} 2021-02-10T11:06:40.764 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:40 +0000 (0:00:00.078) 0:04:56.342 **** 2021-02-10T11:06:42.618 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "e3b44e00721d55ceb436a2b7c0ec58cd5bc39dc9", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 207, "state": "file", "uid": 0} 2021-02-10T11:06:42.654 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:42 +0000 (0:00:01.890) 0:04:58.233 **** 2021-02-10T11:06:42.682 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-02-10T11:06:42.715 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:42 +0000 (0:00:00.060) 0:04:58.293 **** 2021-02-10T11:06:42.725 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-02-10T11:06:42.726 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'key': 'rhel-7-fcgi-ceph', 'value': {'name': 'RHEL 7 Local fastcgi Repo', 'baseurl': 'http://gitbuilder.ceph.com/mod_fastcgi-rpm-rhel7-x86_64-basic/ref/master/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "e6aba9c9ab05e27255ed430f065502af284a869b", "dest": "/etc/yum.repos.d/rhel-7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "rhel-7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-rhel7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "RHEL 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/rhel-7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 245, "state": "file", "uid": 0} 2021-02-10T11:06:42.748 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-02-10T11:06:42.781 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:42 +0000 (0:00:00.066) 0:04:58.359 **** 2021-02-10T11:06:42.808 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2021-02-10T11:06:42.809 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:06:42.841 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:42 +0000 (0:00:00.060) 0:04:58.420 **** 2021-02-10T11:06:42.891 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-02-10T11:06:42.892 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-02-10T11:06:42.927 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:42 +0000 (0:00:00.085) 0:04:58.506 **** 2021-02-10T11:06:42.954 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2021-02-10T11:06:42.955 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:06:42.987 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:42 +0000 (0:00:00.060) 0:04:58.566 **** 2021-02-10T11:06:43.015 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-10T11:06:43.017 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:06:43.052 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:43 +0000 (0:00:00.065) 0:04:58.631 **** 2021-02-10T11:06:43.885 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-02-10T11:06:43.886 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-54.el7_8.noarch providing yum-utils is already installed"]} 2021-02-10T11:06:43.921 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:43 +0000 (0:00:00.868) 0:04:59.499 **** 2021-02-10T11:06:44.633 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-02-10T11:06:44.634 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "e3b44e00721d55ceb436a2b7c0ec58cd5bc39dc9", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 207, "state": "file", "uid": 0} 2021-02-10T11:06:44.672 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:44 +0000 (0:00:00.751) 0:05:00.250 **** 2021-02-10T11:06:44.700 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-02-10T11:06:44.745 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:44 +0000 (0:00:00.072) 0:05:00.323 **** 2021-02-10T11:06:44.783 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-02-10T11:06:44.817 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:44 +0000 (0:00:00.072) 0:05:00.396 **** 2021-02-10T11:06:44.845 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2021-02-10T11:06:44.846 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:06:44.883 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:44 +0000 (0:00:00.065) 0:05:00.461 **** 2021-02-10T11:06:44.931 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-02-10T11:06:44.932 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2021-02-10T11:06:44.953 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-02-10T11:06:44.987 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:44 +0000 (0:00:00.104) 0:05:00.566 **** 2021-02-10T11:06:44.993 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-02-10T11:06:44.994 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.609819", "end": "2021-02-10 11:06:44.967849", "rc": 0, "start": "2021-02-10 11:06:44.358030", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities, product-id, subscription-\n : manager\nLoading mirror speeds from cached hostfile\n * epel: download-cc-rdu01.fedoraproject.org\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities, product-id, subscription-", " : manager", "Loading mirror speeds from cached hostfile", " * epel: download-cc-rdu01.fedoraproject.org", "No unfinished transactions left."]} 2021-02-10T11:06:45.031 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:45 +0000 (0:00:00.043) 0:05:00.609 **** 2021-02-10T11:06:45.039 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2021-02-10T11:06:45.040 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:06:45.077 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:45 +0000 (0:00:00.046) 0:05:00.655 **** 2021-02-10T11:06:45.104 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-10T11:06:45.106 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:06:45.141 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:45 +0000 (0:00:00.064) 0:05:00.720 **** 2021-02-10T11:06:45.495 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-02-10T11:06:45.495 INFO:teuthology.task.ansible.out:fatal: [gibba001.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.018052", "end": "2021-02-10 11:06:45.480614", "msg": "non-zero return code", "rc": 1, "start": "2021-02-10 11:06:45.462562", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2021-02-10T11:06:45.532 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:45 +0000 (0:00:00.390) 0:05:01.110 **** 2021-02-10T11:06:45.561 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-02-10T11:06:45.562 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:06:45.596 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:45 +0000 (0:00:00.064) 0:05:01.175 **** 2021-02-10T11:06:45.958 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-02-10T11:06:45.960 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-54.el7_8.noarch providing yum-utils is already installed"]} 2021-02-10T11:06:45.994 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:45 +0000 (0:00:00.397) 0:05:01.572 **** 2021-02-10T11:06:47.060 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-02-10T11:06:47.061 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.596847", "end": "2021-02-10 11:06:47.040913", "rc": 0, "start": "2021-02-10 11:06:46.444066", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities, product-id, subscription-\n : manager\nLoading mirror speeds from cached hostfile\n * epel: download-cc-rdu01.fedoraproject.org\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities, product-id, subscription-", " : manager", "Loading mirror speeds from cached hostfile", " * epel: download-cc-rdu01.fedoraproject.org", "No unfinished transactions left."]} 2021-02-10T11:06:47.093 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:47 +0000 (0:00:01.099) 0:05:02.672 **** 2021-02-10T11:06:47.563 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-02-10T11:06:47.564 INFO:teuthology.task.ansible.out:fatal: [gibba008.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.018067", "end": "2021-02-10 11:06:47.541482", "msg": "non-zero return code", "rc": 1, "start": "2021-02-10 11:06:47.523415", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2021-02-10T11:06:47.610 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:47 +0000 (0:00:00.516) 0:05:03.188 **** 2021-02-10T11:06:47.650 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-02-10T11:06:47.651 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:06:47.687 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:47 +0000 (0:00:00.077) 0:05:03.266 **** 2021-02-10T11:06:55.165 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-02-10T11:06:55.167 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities, product-id, search-\n : disabled-repos, subscription-manager\nResolving Dependencies\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-175.el7_9.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-175.el7_9.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-175.el7_9.1.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7 will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-175.el7_9.1 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-175.el7_9.1 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-175.el7_9.1 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n librados2 x86_64 1:10.2.5-4.el7 @rhel-7-server-rpms 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 @rhel-7-server-rpms 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7 @rhel-7- 2021-02-10T11:06:55.168 INFO:teuthology.task.ansible.out:fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img x86_64 10:1.5.3-175.el7_9.1 @rhel-7-server-rpms 2.3 M\n qemu-kvm x86_64 10:1.5.3-175.el7_9.1 @rhel-7-server-rpms 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-175.el7_9.1 @rhel-7-server-rpms 755 k\n\nTransaction Summary\n================================================================================\nRemove 3 Packages (+3 Dependent packages)\n\nInstalled size: 22 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-175.el7_9.1.x86_64 1/6 \n Erasing : 10:qemu-img-1.5.3-175.el7_9.1.x86_64 2/6 \n Erasing : 10:qemu-kvm-common-1.5.3-175.el7_9.1.x86_64 3/6 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/6 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/6 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.x86_64 6/6 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\nLoaded plugins: fastestmirror, langpacks, priorities, product-id, subscription-\n : manager\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/6 \n Verifying : 10:qemu-kvm-1.5.3-175.el7_9.1.x86_64 2/6 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.x86_64 3/6 \n Verifying : 10:qemu-kvm-common-1.5.3-175.el7_9.1.x86_64 4/6 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 5/6 \n Verifying : 10:qemu-img-1.5.3-175.el7_9.1.x86_64 6/6 \n\nRemoved:\n librados2.x86_64 1:10.2.5-4.el7 librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7 \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-175.el7_9.1 2021-02-10T11:06:55.170 INFO:teuthology.task.ansible.out:\n qemu-kvm.x86_64 10:1.5.3-175.el7_9.1 \n qemu-kvm-common.x86_64 10:1.5.3-175.el7_9.1 \n\nComplete!\n"]} 2021-02-10T11:06:55.204 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:55 +0000 (0:00:07.516) 0:05:10.783 **** 2021-02-10T11:06:57.040 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities, product-id, search-\n : disabled-repos, subscription-manager\nResolving Dependencies\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-175.el7_9.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-175.el7_9.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-175.el7_9.1.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7 will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-175.el7_9.1 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-175.el7_9.1 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-175.el7_9.1 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n librados2 x86_64 1:10.2.5-4.el7 @rhel-7-server-rpms 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 @rhel-7-server-rpms 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7 @rhel-7- 2021-02-10T11:06:57.042 INFO:teuthology.task.ansible.out:fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img x86_64 10:1.5.3-175.el7_9.1 @rhel-7-server-rpms 2.3 M\n qemu-kvm x86_64 10:1.5.3-175.el7_9.1 @rhel-7-server-rpms 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-175.el7_9.1 @rhel-7-server-rpms 755 k\n\nTransaction Summary\n================================================================================\nRemove 3 Packages (+3 Dependent packages)\n\nInstalled size: 22 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-175.el7_9.1.x86_64 1/6 \n Erasing : 10:qemu-img-1.5.3-175.el7_9.1.x86_64 2/6 \n Erasing : 10:qemu-kvm-common-1.5.3-175.el7_9.1.x86_64 3/6 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/6 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/6 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.x86_64 6/6 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\nLoaded plugins: fastestmirror, langpacks, priorities, product-id, subscription-\n : manager\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/6 \n Verifying : 10:qemu-kvm-1.5.3-175.el7_9.1.x86_64 2/6 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.x86_64 3/6 \n Verifying : 10:qemu-kvm-common-1.5.3-175.el7_9.1.x86_64 4/6 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 5/6 \n Verifying : 10:qemu-img-1.5.3-175.el7_9.1.x86_64 6/6 \n\nRemoved:\n librados2.x86_64 1:10.2.5-4.el7 librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7 \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-175.el7_9.1 2021-02-10T11:06:57.043 INFO:teuthology.task.ansible.out:\n qemu-kvm.x86_64 10:1.5.3-175.el7_9.1 \n qemu-kvm-common.x86_64 10:1.5.3-175.el7_9.1 \n\nComplete!\n"]} 2021-02-10T11:06:57.081 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:57 +0000 (0:00:01.876) 0:05:12.659 **** 2021-02-10T11:06:59.864 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-02-10T11:06:59.864 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "leveldb", "xmlstarlet", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities, product-id, search-\n : disabled-repos, subscription-manager\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-28.el7 will be erased\n---> Package leveldb.x86_64 0:1.12.0-11.el7 will be erased\n---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-28.el7 @rhel-7-server-rpms 24 k\n leveldb x86_64 1.12.0-11.el7 @epel 439 k\n xmlstarlet x86_64 1.6.1-1.el7 @epel 192 k\n\nTransaction Summary\n================================================================================\nRemove 3 Packages\n\nInstalled size: 655 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : xmlstarlet-1.6.1-1.el7.x86_64 1/3 \n Erasing : boost-random-1.53.0-28.el7.x86_64 2/3 \n Erasing : leveldb-1.12.0-11.el7.x86_64 2021-02-10T11:06:59.865 INFO:teuthology.task.ansible.out: 3/3 \nLoaded plugins: fastestmirror, langpacks, priorities, product-id, subscription-\n : manager\n Verifying : leveldb-1.12.0-11.el7.x86_64 1/3 \n Verifying : boost-random-1.53.0-28.el7.x86_64 2/3 \n Verifying : xmlstarlet-1.6.1-1.el7.x86_64 3/3 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-28.el7 leveldb.x86_64 0:1.12.0-11.el7 \n xmlstarlet.x86_64 0:1.6.1-1.el7 \n\nComplete!\n"]} 2021-02-10T11:06:59.899 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:06:59 +0000 (0:00:02.817) 0:05:15.477 **** 2021-02-10T11:07:01.640 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "leveldb", "xmlstarlet", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities, product-id, search-\n : disabled-repos, subscription-manager\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-28.el7 will be erased\n---> Package leveldb.x86_64 0:1.12.0-11.el7 will be erased\n---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-28.el7 @rhel-7-server-rpms 24 k\n leveldb x86_64 1.12.0-11.el7 @epel 439 k\n xmlstarlet x86_64 1.6.1-1.el7 @epel 192 k\n\nTransaction Summary\n================================================================================\nRemove 3 Packages\n\nInstalled size: 655 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : xmlstarlet-1.6.1-1.el7.x86_64 1/3 \n Erasing : boost-random-1.53.0-28.el7.x86_64 2/3 \n Erasing : leveldb-1.12.0-11.el7.x86_64 2021-02-10T11:07:01.641 INFO:teuthology.task.ansible.out: 3/3 \nLoaded plugins: fastestmirror, langpacks, priorities, product-id, subscription-\n : manager\n Verifying : leveldb-1.12.0-11.el7.x86_64 1/3 \n Verifying : boost-random-1.53.0-28.el7.x86_64 2/3 \n Verifying : xmlstarlet-1.6.1-1.el7.x86_64 3/3 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-28.el7 leveldb.x86_64 0:1.12.0-11.el7 \n xmlstarlet.x86_64 0:1.6.1-1.el7 \n\nComplete!\n"]} 2021-02-10T11:07:01.678 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:01 +0000 (0:00:01.779) 0:05:17.257 **** 2021-02-10T11:07:16.233 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-02-10T11:07:16.234 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": ["qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7"]}, "msg": "", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-54.el7_8.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-54.el7_8.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-19.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-120.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-23.el7_8.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-44.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7_2.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-28.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-22.el7.x86_64 providing xfsprogs is already installed", "xfsprogs-devel-4.5.0-22.el7.x86_64 providing xfsprogs-devel is already installed", "gdisk-0.8.10-3.el7.x86_64 providing gdisk is already installed", "parted-3.1-32.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "7:lvm2-2.02.187-6.el7.x86_64 providing lvm2 is already installed", "1:openssl-1.0.2k-19.el7.x86_64 providing openssl is already installed", "libuuid-2.23.2-65.el7.x86_64 providing libuuid is already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is already installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.15.0-11.el7.x86_64 providing valgrind is already installed", "python-nose-1 2021-02-10T11:07:16.235 INFO:teuthology.task.ansible.out:.3.7-1.el7.noarch providing python-nose is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "lsof-4.87-6.el7.x86_64 providing lsof is already installed", "iozone-3.424-2_ceph.el7.x86_64 providing iozone is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-3.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-65.el7.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-15.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "blktrace-1.0.5-9.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-97.el7_9.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-97.el7_9.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-97.el7_9.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-97.el7_9.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-openjdk-devel-1.6.0.41-1.13.13.1.el7_3.x86_64 providing java-1.6.0-openjdk-devel is already installed", "junit-4.11-8.el7.noarch providing junit4 is already installed", "1:nfs-utils-1.3.0-0.68.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-d 2021-02-10T11:07:16.237 INFO:teuthology.task.ansible.out:evel is already installed", "python-devel-2.7.5-90.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-4.el7_8.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-297.el7.noarch providing perl-CPAN is already installed", "podman-1.6.4-26.el7_9.x86_64 providing podman is already installed", "podman-docker-1.6.4-26.el7_9.noarch providing podman-docker is already installed", "python3-3.6.8-18.el7.x86_64 providing python3 is already installed", "Loaded plugins: fastestmirror, langpacks, priorities, product-id, search-\n : disabled-repos, subscription-manager\nLoading mirror speeds from cached hostfile\n * epel: download-cc-rdu01.fedoraproject.org\nResolving Dependencies\n--> Running transaction check\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7 will be installed\n---> Package qemu-kvm.x86_64 10:1.5.3-175.el7_9.3 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-175.el7_9.3 for package: 10:qemu-kvm-1.5.3-175.el7_9.3.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-175.el7_9.3 for package: 10:qemu-kvm-1.5.3-175.el7_9.3.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-175.el7_9.3.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-175.el7_9.3.x86_64\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-175.el7_9.3 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-175.el7_9.3 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-28.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n P 2021-02-10T11:07:16.238 INFO:teuthology.task.ansible.out:ackage Arch Version Repository Size\n================================================================================\nInstalling:\n mod_fastcgi x86_64 2.4.7-1.ceph.el7 rhel-7-fcgi-ceph 70 k\n qemu-kvm x86_64 10:1.5.3-175.el7_9.3 rhel-7-server-rpms 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-28.el7 rhel-7-server-rpms 39 k\n librados2 x86_64 1:10.2.5-4.el7 rhel-7-server-rpms 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 rhel-7-server-rpms 2.4 M\n qemu-img x86_64 10:1.5.3-175.el7_9.3 rhel-7-server-rpms 704 k\n qemu-kvm-common x86_64 10:1.5.3-175.el7_9.3 rhel-7-server-rpms 440 k\n\nTransaction Summary\n================================================================================\nInstall 2 Packages (+5 Dependent packages)\n\nTotal download size: 7.3 M\nInstalled size: 22 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 31 MB/s | 7.3 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-28.el7.x86_64 1/7 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/7 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/7 \n Installing : 10:qemu-img-1.5.3-175.el7_9.3.x86_64 4/7 \n Installing : 10:qemu-kvm-common-1.5.3-175.el7_9.3.x86_64 5/7 \n Installing : 10:qemu-kvm-1.5.3-175.el7_9.3.x86_64 6/7 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.x86_64 7/7 \nLoaded plugins: fastestmirror, langpacks, priorities, product-id, subscription-\n : manager\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 2021-02-10T11:07:16.239 INFO:teuthology.task.ansible.out: 1/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.x86_64 2/7 \n Verifying : 10:qemu-img-1.5.3-175.el7_9.3.x86_64 3/7 \n Verifying : boost-random-1.53.0-28.el7.x86_64 4/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 5/7 \n Verifying : 10:qemu-kvm-1.5.3-175.el7_9.3.x86_64 6/7 \n Verifying : 10:qemu-kvm-common-1.5.3-175.el7_9.3.x86_64 7/7 \n\nInstalled:\n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7 qemu-kvm.x86_64 10:1.5.3-175.el7_9.3 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-28.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n qemu-img.x86_64 10:1.5.3-175.el7_9.3 \n qemu-kvm-common.x86_64 10:1.5.3-175.el7_9.3 \n\nComplete!\n"]} 2021-02-10T11:07:16.271 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:16 +0000 (0:00:14.592) 0:05:31.850 **** 2021-02-10T11:07:17.531 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": ["qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7"]}, "msg": "", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-54.el7_8.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-54.el7_8.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-19.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-120.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-23.el7_8.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-44.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7_2.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-28.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-22.el7.x86_64 providing xfsprogs is already installed", "xfsprogs-devel-4.5.0-22.el7.x86_64 providing xfsprogs-devel is already installed", "gdisk-0.8.10-3.el7.x86_64 providing gdisk is already installed", "parted-3.1-32.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "7:lvm2-2.02.187-6.el7.x86_64 providing lvm2 is already installed", "1:openssl-1.0.2k-19.el7.x86_64 providing openssl is already installed", "libuuid-2.23.2-65.el7.x86_64 providing libuuid is already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is already installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.15.0-11.el7.x86_64 providing valgrind is already installed", "python-nose-1 2021-02-10T11:07:17.532 INFO:teuthology.task.ansible.out:.3.7-1.el7.noarch providing python-nose is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "lsof-4.87-6.el7.x86_64 providing lsof is already installed", "iozone-3.424-2_ceph.el7.x86_64 providing iozone is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-3.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-65.el7.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-15.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "blktrace-1.0.5-9.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-97.el7_9.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-97.el7_9.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-97.el7_9.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-97.el7_9.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-openjdk-devel-1.6.0.41-1.13.13.1.el7_3.x86_64 providing java-1.6.0-openjdk-devel is already installed", "junit-4.11-8.el7.noarch providing junit4 is already installed", "1:nfs-utils-1.3.0-0.68.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-d 2021-02-10T11:07:17.534 INFO:teuthology.task.ansible.out:evel is already installed", "python-devel-2.7.5-90.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-4.el7_8.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-297.el7.noarch providing perl-CPAN is already installed", "podman-1.6.4-26.el7_9.x86_64 providing podman is already installed", "podman-docker-1.6.4-26.el7_9.noarch providing podman-docker is already installed", "python3-3.6.8-18.el7.x86_64 providing python3 is already installed", "Loaded plugins: fastestmirror, langpacks, priorities, product-id, search-\n : disabled-repos, subscription-manager\nLoading mirror speeds from cached hostfile\n * epel: download-cc-rdu01.fedoraproject.org\nResolving Dependencies\n--> Running transaction check\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7 will be installed\n---> Package qemu-kvm.x86_64 10:1.5.3-175.el7_9.3 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-175.el7_9.3 for package: 10:qemu-kvm-1.5.3-175.el7_9.3.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-175.el7_9.3 for package: 10:qemu-kvm-1.5.3-175.el7_9.3.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-175.el7_9.3.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-175.el7_9.3.x86_64\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-175.el7_9.3 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-175.el7_9.3 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-28.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n P 2021-02-10T11:07:17.535 INFO:teuthology.task.ansible.out:ackage Arch Version Repository Size\n================================================================================\nInstalling:\n mod_fastcgi x86_64 2.4.7-1.ceph.el7 rhel-7-fcgi-ceph 70 k\n qemu-kvm x86_64 10:1.5.3-175.el7_9.3 rhel-7-server-rpms 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-28.el7 rhel-7-server-rpms 39 k\n librados2 x86_64 1:10.2.5-4.el7 rhel-7-server-rpms 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 rhel-7-server-rpms 2.4 M\n qemu-img x86_64 10:1.5.3-175.el7_9.3 rhel-7-server-rpms 704 k\n qemu-kvm-common x86_64 10:1.5.3-175.el7_9.3 rhel-7-server-rpms 440 k\n\nTransaction Summary\n================================================================================\nInstall 2 Packages (+5 Dependent packages)\n\nTotal download size: 7.3 M\nInstalled size: 22 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 32 MB/s | 7.3 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-28.el7.x86_64 1/7 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/7 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/7 \n Installing : 10:qemu-img-1.5.3-175.el7_9.3.x86_64 4/7 \n Installing : 10:qemu-kvm-common-1.5.3-175.el7_9.3.x86_64 5/7 \n Installing : 10:qemu-kvm-1.5.3-175.el7_9.3.x86_64 6/7 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.x86_64 7/7 \nLoaded plugins: fastestmirror, langpacks, priorities, product-id, subscription-\n : man 2021-02-10T11:07:17.536 INFO:teuthology.task.ansible.out:ager\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.x86_64 2/7 \n Verifying : 10:qemu-img-1.5.3-175.el7_9.3.x86_64 3/7 \n Verifying : boost-random-1.53.0-28.el7.x86_64 4/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 5/7 \n Verifying : 10:qemu-kvm-1.5.3-175.el7_9.3.x86_64 6/7 \n Verifying : 10:qemu-kvm-common-1.5.3-175.el7_9.3.x86_64 7/7 \n\nInstalled:\n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7 qemu-kvm.x86_64 10:1.5.3-175.el7_9.3 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-28.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n qemu-img.x86_64 10:1.5.3-175.el7_9.3 \n qemu-kvm-common.x86_64 10:1.5.3-175.el7_9.3 \n\nComplete!\n"]} 2021-02-10T11:07:17.572 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:17 +0000 (0:00:01.300) 0:05:33.150 **** 2021-02-10T11:07:21.053 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-02-10T11:07:21.055 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": ["leveldb", "xmlstarlet"]}, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed", "Loaded plugins: fastestmirror, langpacks, priorities, product-id, search-\n : disabled-repos, subscription-manager\nLoading mirror speeds from cached hostfile\n * epel: download-cc-rdu01.fedoraproject.org\nResolving Dependencies\n--> Running transaction check\n---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed\n---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n leveldb x86_64 1.12.0-11.el7 epel 161 k\n xmlstarlet x86_64 1.6.1-1.el7 epel 62 k\n\nTransaction Summary\n================================================================================\nInstall 2 Packages\n\nTotal download size: 223 k\nInstalled size: 631 k\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 3.1 MB/s | 223 kB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : xmlstarlet-1.6.1-1.el7.x86_64 2021-02-10T11:07:21.056 INFO:teuthology.task.ansible.out: 1/2 \n Installing : leveldb-1.12.0-11.el7.x86_64 2/2 \nLoaded plugins: fastestmirror, langpacks, priorities, product-id, subscription-\n : manager\n Verifying : leveldb-1.12.0-11.el7.x86_64 1/2 \n Verifying : xmlstarlet-1.6.1-1.el7.x86_64 2/2 \n\nInstalled:\n leveldb.x86_64 0:1.12.0-11.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 \n\nComplete!\n"]} 2021-02-10T11:07:21.097 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:21 +0000 (0:00:03.525) 0:05:36.676 **** 2021-02-10T11:07:21.973 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-02-10T11:07:21.975 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} 2021-02-10T11:07:22.006 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:22 +0000 (0:00:00.909) 0:05:37.585 **** 2021-02-10T11:07:22.034 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:22.067 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:22 +0000 (0:00:00.060) 0:05:37.646 **** 2021-02-10T11:07:22.086 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-02-10T11:07:22.087 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": ["leveldb", "xmlstarlet"]}, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed", "Loaded plugins: fastestmirror, langpacks, priorities, product-id, search-\n : disabled-repos, subscription-manager\nLoading mirror speeds from cached hostfile\n * epel: download-cc-rdu01.fedoraproject.org\nResolving Dependencies\n--> Running transaction check\n---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed\n---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n leveldb x86_64 1.12.0-11.el7 epel 161 k\n xmlstarlet x86_64 1.6.1-1.el7 epel 62 k\n\nTransaction Summary\n================================================================================\nInstall 2 Packages\n\nTotal download size: 223 k\nInstalled size: 631 k\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 5.7 MB/s | 223 kB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : xmlstarlet-1.6.1-1.el7.x86_64 2021-02-10T11:07:22.089 INFO:teuthology.task.ansible.out: 1/2 \n Installing : leveldb-1.12.0-11.el7.x86_64 2/2 \nLoaded plugins: fastestmirror, langpacks, priorities, product-id, subscription-\n : manager\n Verifying : leveldb-1.12.0-11.el7.x86_64 1/2 \n Verifying : xmlstarlet-1.6.1-1.el7.x86_64 2/2 \n\nInstalled:\n leveldb.x86_64 0:1.12.0-11.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 \n\nComplete!\n"]} 2021-02-10T11:07:22.125 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:22 +0000 (0:00:00.057) 0:05:37.704 **** 2021-02-10T11:07:22.582 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2021-02-10T11:07:22.584 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "basic.target dbus.service polkit.service system.slice", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "network-pre.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "iptables.service shutdown.target ip6tables.service ebtables.service ipset.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "firewalld.service", "Ign 2021-02-10T11:07:22.586 INFO:teuthology.task.ansible.out:oreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127560", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127560", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "system.slice basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", 2021-02-10T11:07:22.587 INFO:teuthology.task.ansible.out:"TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-02-10T11:07:22.620 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:22 +0000 (0:00:00.495) 0:05:38.199 **** 2021-02-10T11:07:23.008 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-02-10T11:07:23.010 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} 2021-02-10T11:07:23.061 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:23 +0000 (0:00:00.440) 0:05:38.640 **** 2021-02-10T11:07:23.101 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:23.147 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:23 +0000 (0:00:00.085) 0:05:38.726 **** 2021-02-10T11:07:23.156 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-02-10T11:07:23.156 INFO:teuthology.task.ansible.out:fatal: [gibba001.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2021-02-10T11:07:23.194 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:23 +0000 (0:00:00.047) 0:05:38.773 **** 2021-02-10T11:07:23.649 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2021-02-10T11:07:23.651 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "system.slice polkit.service dbus.service basic.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target network-pre.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "iptables.service ipset.service ebtables.service ip6tables.service shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "firewalld.service", "Ign 2021-02-10T11:07:23.652 INFO:teuthology.task.ansible.out:oreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127560", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127560", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "system.slice basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", 2021-02-10T11:07:23.653 INFO:teuthology.task.ansible.out:"TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-02-10T11:07:23.691 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:23 +0000 (0:00:00.497) 0:05:39.270 **** 2021-02-10T11:07:24.142 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-02-10T11:07:24.144 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2021-02-10T11:07:24.178 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:24 +0000 (0:00:00.486) 0:05:39.756 **** 2021-02-10T11:07:24.196 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-02-10T11:07:24.201 INFO:teuthology.task.ansible.out:fatal: [gibba008.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2021-02-10T11:07:24.235 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:24 +0000 (0:00:00.057) 0:05:39.814 **** 2021-02-10T11:07:24.241 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2021-02-10T11:07:24.243 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:24.282 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:24 +0000 (0:00:00.046) 0:05:39.860 **** 2021-02-10T11:07:24.310 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-02-10T11:07:24.312 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:24.345 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:24 +0000 (0:00:00.063) 0:05:39.924 **** 2021-02-10T11:07:24.383 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-02-10T11:07:24.385 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:24.418 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:24 +0000 (0:00:00.073) 0:05:39.997 **** 2021-02-10T11:07:24.446 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2021-02-10T11:07:24.448 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:24.483 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:24 +0000 (0:00:00.064) 0:05:40.062 **** 2021-02-10T11:07:24.523 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-02-10T11:07:24.524 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:24.561 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:24 +0000 (0:00:00.077) 0:05:40.139 **** 2021-02-10T11:07:24.589 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-02-10T11:07:24.591 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:24.627 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:24 +0000 (0:00:00.066) 0:05:40.206 **** 2021-02-10T11:07:24.668 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-02-10T11:07:24.702 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:24 +0000 (0:00:00.074) 0:05:40.281 **** 2021-02-10T11:07:24.732 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-02-10T11:07:24.732 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2021-02-10T11:07:24.734 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:24.766 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:24 +0000 (0:00:00.064) 0:05:40.345 **** 2021-02-10T11:07:24.768 INFO:teuthology.task.ansible.out: 2021-02-10T11:07:24.801 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:24 +0000 (0:00:00.034) 0:05:40.380 **** 2021-02-10T11:07:24.807 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2021-02-10T11:07:24.809 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:24.851 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:24 +0000 (0:00:00.049) 0:05:40.429 **** 2021-02-10T11:07:24.858 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2021-02-10T11:07:24.859 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:24.900 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:24 +0000 (0:00:00.049) 0:05:40.479 **** 2021-02-10T11:07:24.908 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-02-10T11:07:24.908 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-02-10T11:07:24.910 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2021-02-10T11:07:24.949 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:24 +0000 (0:00:00.048) 0:05:40.528 **** 2021-02-10T11:07:24.958 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-02-10T11:07:24.960 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:24.998 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:24 +0000 (0:00:00.049) 0:05:40.577 **** 2021-02-10T11:07:25.007 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-02-10T11:07:25.009 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:25.046 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:25 +0000 (0:00:00.047) 0:05:40.625 **** 2021-02-10T11:07:25.053 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-02-10T11:07:25.054 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:25.088 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:25 +0000 (0:00:00.042) 0:05:40.667 **** 2021-02-10T11:07:25.095 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-02-10T11:07:25.130 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:25 +0000 (0:00:00.041) 0:05:40.709 **** 2021-02-10T11:07:25.138 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2021-02-10T11:07:25.138 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:25.185 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:25 +0000 (0:00:00.055) 0:05:40.764 **** 2021-02-10T11:07:25.192 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2021-02-10T11:07:25.192 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:25.232 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:25 +0000 (0:00:00.046) 0:05:40.811 **** 2021-02-10T11:07:25.239 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-02-10T11:07:25.239 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:25.274 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:25 +0000 (0:00:00.041) 0:05:40.853 **** 2021-02-10T11:07:25.280 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2021-02-10T11:07:25.282 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:25.316 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:25 +0000 (0:00:00.041) 0:05:40.894 **** 2021-02-10T11:07:25.322 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-02-10T11:07:25.324 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:25.361 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:25 +0000 (0:00:00.045) 0:05:40.940 **** 2021-02-10T11:07:25.362 INFO:teuthology.task.ansible.out: 2021-02-10T11:07:25.369 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2021-02-10T11:07:25.370 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:25.408 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:25 +0000 (0:00:00.047) 0:05:40.987 **** 2021-02-10T11:07:25.415 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-02-10T11:07:25.457 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:25 +0000 (0:00:00.048) 0:05:41.036 **** 2021-02-10T11:07:25.465 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-02-10T11:07:25.465 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:25.510 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:25 +0000 (0:00:00.052) 0:05:41.088 **** 2021-02-10T11:07:25.520 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-02-10T11:07:25.520 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:25.558 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:25 +0000 (0:00:00.048) 0:05:41.136 **** 2021-02-10T11:07:25.564 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-02-10T11:07:25.565 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:25.599 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:25 +0000 (0:00:00.041) 0:05:41.178 **** 2021-02-10T11:07:25.605 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2021-02-10T11:07:25.607 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:25.641 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:25 +0000 (0:00:00.041) 0:05:41.220 **** 2021-02-10T11:07:25.651 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-02-10T11:07:25.652 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:25.692 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:25 +0000 (0:00:00.050) 0:05:41.271 **** 2021-02-10T11:07:25.699 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-02-10T11:07:25.701 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [gibba008.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2021-02-10T11:07:25.736 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:25 +0000 (0:00:00.043) 0:05:41.314 **** 2021-02-10T11:07:25.743 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-02-10T11:07:25.745 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:25.785 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:25 +0000 (0:00:00.049) 0:05:41.364 **** 2021-02-10T11:07:25.792 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-02-10T11:07:25.794 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:25.828 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:25 +0000 (0:00:00.042) 0:05:41.407 **** 2021-02-10T11:07:25.834 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-02-10T11:07:25.835 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:25.870 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:25 +0000 (0:00:00.041) 0:05:41.449 **** 2021-02-10T11:07:25.879 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-02-10T11:07:25.915 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:25 +0000 (0:00:00.045) 0:05:41.494 **** 2021-02-10T11:07:25.922 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2021-02-10T11:07:25.924 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:25.971 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:25 +0000 (0:00:00.055) 0:05:41.550 **** 2021-02-10T11:07:25.978 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2021-02-10T11:07:25.979 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.014 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.043) 0:05:41.593 **** 2021-02-10T11:07:26.021 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2021-02-10T11:07:26.023 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.057 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.042) 0:05:41.635 **** 2021-02-10T11:07:26.063 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2021-02-10T11:07:26.065 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.098 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.041) 0:05:41.677 **** 2021-02-10T11:07:26.104 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-02-10T11:07:26.106 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.139 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.041) 0:05:41.718 **** 2021-02-10T11:07:26.146 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2021-02-10T11:07:26.147 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.180 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.041) 0:05:41.759 **** 2021-02-10T11:07:26.189 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-02-10T11:07:26.190 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.224 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.043) 0:05:41.803 **** 2021-02-10T11:07:26.231 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.266 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.041) 0:05:41.845 **** 2021-02-10T11:07:26.273 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2021-02-10T11:07:26.274 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.308 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.042) 0:05:41.887 **** 2021-02-10T11:07:26.315 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.349 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.040) 0:05:41.928 **** 2021-02-10T11:07:26.355 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-02-10T11:07:26.357 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.392 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.042) 0:05:41.971 **** 2021-02-10T11:07:26.401 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-02-10T11:07:26.402 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.439 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.047) 0:05:42.018 **** 2021-02-10T11:07:26.446 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-02-10T11:07:26.448 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.483 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.043) 0:05:42.062 **** 2021-02-10T11:07:26.490 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-02-10T11:07:26.492 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.530 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.046) 0:05:42.108 **** 2021-02-10T11:07:26.536 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-02-10T11:07:26.538 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.570 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.040) 0:05:42.149 **** 2021-02-10T11:07:26.578 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-02-10T11:07:26.580 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.613 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.042) 0:05:42.192 **** 2021-02-10T11:07:26.620 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-02-10T11:07:26.621 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.660 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.046) 0:05:42.239 **** 2021-02-10T11:07:26.669 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2021-02-10T11:07:26.670 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.715 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.054) 0:05:42.294 **** 2021-02-10T11:07:26.723 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-02-10T11:07:26.724 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.767 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.052) 0:05:42.346 **** 2021-02-10T11:07:26.774 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2021-02-10T11:07:26.775 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.809 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.042) 0:05:42.388 **** 2021-02-10T11:07:26.816 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-02-10T11:07:26.818 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.858 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.049) 0:05:42.437 **** 2021-02-10T11:07:26.866 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-02-10T11:07:26.867 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.903 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.044) 0:05:42.481 **** 2021-02-10T11:07:26.909 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-02-10T11:07:26.910 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.945 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.042) 0:05:42.524 **** 2021-02-10T11:07:26.952 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-02-10T11:07:26.954 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:26.988 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:26 +0000 (0:00:00.042) 0:05:42.567 **** 2021-02-10T11:07:26.994 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-02-10T11:07:26.996 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:27.034 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:27 +0000 (0:00:00.045) 0:05:42.612 **** 2021-02-10T11:07:27.041 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2021-02-10T11:07:27.042 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:27.083 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:27 +0000 (0:00:00.049) 0:05:42.662 **** 2021-02-10T11:07:27.090 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-02-10T11:07:27.092 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:27.128 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:27 +0000 (0:00:00.044) 0:05:42.706 **** 2021-02-10T11:07:27.136 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-02-10T11:07:27.138 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:27.173 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:27 +0000 (0:00:00.045) 0:05:42.751 **** 2021-02-10T11:07:27.180 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-10T11:07:27.183 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:27.219 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:27 +0000 (0:00:00.046) 0:05:42.798 **** 2021-02-10T11:07:27.231 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:27.274 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:27 +0000 (0:00:00.055) 0:05:42.853 **** 2021-02-10T11:07:27.281 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-02-10T11:07:27.283 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:27.319 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:27 +0000 (0:00:00.044) 0:05:42.897 **** 2021-02-10T11:07:27.326 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-02-10T11:07:27.328 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:27.368 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:27 +0000 (0:00:00.048) 0:05:42.946 **** 2021-02-10T11:07:27.376 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-02-10T11:07:27.378 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:27.419 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:27 +0000 (0:00:00.051) 0:05:42.998 **** 2021-02-10T11:07:27.427 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-02-10T11:07:27.427 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:27.477 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:27 +0000 (0:00:00.058) 0:05:43.056 **** 2021-02-10T11:07:27.485 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-02-10T11:07:27.486 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:27.521 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:27 +0000 (0:00:00.044) 0:05:43.100 **** 2021-02-10T11:07:27.529 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-02-10T11:07:27.530 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:27.582 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:27 +0000 (0:00:00.059) 0:05:43.160 **** 2021-02-10T11:07:27.588 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-02-10T11:07:27.590 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:27.644 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:27 +0000 (0:00:00.062) 0:05:43.223 **** 2021-02-10T11:07:27.653 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-02-10T11:07:27.653 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:27.694 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:27 +0000 (0:00:00.049) 0:05:43.273 **** 2021-02-10T11:07:27.702 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-10T11:07:27.703 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:27.751 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:27 +0000 (0:00:00.056) 0:05:43.329 **** 2021-02-10T11:07:27.758 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-02-10T11:07:27.759 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:27.801 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:27 +0000 (0:00:00.050) 0:05:43.380 **** 2021-02-10T11:07:27.810 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-02-10T11:07:27.811 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:27.853 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:27 +0000 (0:00:00.051) 0:05:43.431 **** 2021-02-10T11:07:27.861 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-02-10T11:07:27.862 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:27.898 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:27 +0000 (0:00:00.044) 0:05:43.476 **** 2021-02-10T11:07:27.905 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-02-10T11:07:27.906 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:27.942 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:27 +0000 (0:00:00.043) 0:05:43.520 **** 2021-02-10T11:07:27.950 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-02-10T11:07:27.951 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:27.991 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:27 +0000 (0:00:00.049) 0:05:43.569 **** 2021-02-10T11:07:28.020 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-10T11:07:28.021 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:28.056 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:28 +0000 (0:00:00.064) 0:05:43.634 **** 2021-02-10T11:07:28.095 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-02-10T11:07:28.096 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:28.132 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:28 +0000 (0:00:00.076) 0:05:43.710 **** 2021-02-10T11:07:28.159 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:28.196 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:28 +0000 (0:00:00.064) 0:05:43.775 **** 2021-02-10T11:07:28.226 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-02-10T11:07:28.231 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:28.262 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:28 +0000 (0:00:00.065) 0:05:43.841 **** 2021-02-10T11:07:28.290 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-02-10T11:07:28.292 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:28.333 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:28 +0000 (0:00:00.070) 0:05:43.912 **** 2021-02-10T11:07:28.375 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-10T11:07:28.376 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:28.409 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:28 +0000 (0:00:00.076) 0:05:43.988 **** 2021-02-10T11:07:28.437 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-02-10T11:07:28.438 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:28.474 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:28 +0000 (0:00:00.064) 0:05:44.052 **** 2021-02-10T11:07:28.513 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:28.549 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:28 +0000 (0:00:00.074) 0:05:44.127 **** 2021-02-10T11:07:29.897 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-02-10T11:07:29.898 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "secontext": "system_u:object_r:exports_t:s0", "size": 578, "state": "file", "uid": 0} 2021-02-10T11:07:29.951 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:29 +0000 (0:00:01.402) 0:05:45.530 **** 2021-02-10T11:07:30.487 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-02-10T11:07:30.488 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nfs-server", "status": {"ActiveEnterTimestamp": "Wed 2021-02-10 11:00:55 UTC", "ActiveEnterTimestampMonotonic": "19989434", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "rpc-statd.service -.mount rpc-gssd.service rpcbind.socket nfs-mountd.service local-fs.target network-online.target nfs-idmapd.service gssproxy.service systemd-journald.socket nfs-config.service system.slice tmp.mount proc-fs-nfsd.mount", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Wed 2021-02-10 11:00:55 UTC", "AssertTimestampMonotonic": "19833268", "Before": "rpc-statd-notify.service", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BoundBy": "nfs-idmapd.service nfs-mountd.service", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Wed 2021-02-10 11:00:55 UTC", "ConditionTimestampMonotonic": "19833268", "ControlGroup": "/system.slice/nfs-server.service", "ControlPID": "0", "DefaultDependencies": "no", "Delegate": "no", "Description": "NFS server and services", "DevicePolicy": "auto", "DropInPaths": "/run/systemd/generator/nfs-server.service.d/order-with-mounts.conf", "EnvironmentFile": "/run/sysconfig/nfs-utils (ignore_errors=yes)", "ExecMainCode": "1", "ExecMainExitTimestamp": "Wed 2021-02-10 11:00:55 UTC", "ExecMainExitTimestampMonotonic": "19978283", "ExecMainPID": "1294", "ExecMainStartTimestamp": "Wed 2021-02-10 11:00:55 UTC", "ExecMainStartTimestampMonotonic": "19877510", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/exportfs ; argv[]=/usr/sbin/exportfs -r ; ignore_errors=yes 2021-02-10T11:07:30.489 INFO:teuthology.task.ansible.out: ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/rpc.nfsd ; argv[]=/usr/sbin/rpc.nfsd $RPCNFSDARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPost": "{ path=/bin/sh ; argv[]=/bin/sh -c if systemctl -q is-active gssproxy; then systemctl reload gssproxy ; fi ; ignore_errors=yes ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPre": "{ path=/usr/sbin/exportfs ; argv[]=/usr/sbin/exportfs -r ; ignore_errors=yes ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/rpc.nfsd ; argv[]=/usr/sbin/rpc.nfsd 0 ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/usr/sbin/exportfs ; argv[]=/usr/sbin/exportfs -f ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nfs-server.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nfs-server.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Wed 2021-02-10 11:00:55 UTC", "InactiveExitTimestampMonotonic": "19834328", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127560", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127560", "LimitSTACK": "18446744073709551615", "LoadState": "lo 2021-02-10T11:07:30.490 INFO:teuthology.task.ansible.out:aded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nfs-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "yes", "Requires": "-.mount nfs-mountd.service network.target system.slice proc-fs-nfsd.mount", "RequiresMountsFor": "/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "exited", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "0", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "oneshot", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "network-online.target rpc-statd-notify.service auth-rpcgss-module.service nfs-config.service rpc-statd.service nfs-idmapd.service rpcbind.socket", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-02-10T11:07:30.529 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:30 +0000 (0:00:00.577) 0:05:46.107 **** 2021-02-10T11:07:30.560 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-02-10T11:07:30.561 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:30.596 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:30 +0000 (0:00:00.067) 0:05:46.175 **** 2021-02-10T11:07:30.605 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "secontext": "system_u:object_r:exports_t:s0", "size": 578, "state": "file", "uid": 0} 2021-02-10T11:07:30.641 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:30 +0000 (0:00:00.044) 0:05:46.219 **** 2021-02-10T11:07:30.651 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-10T11:07:30.652 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:30.689 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:30 +0000 (0:00:00.048) 0:05:46.268 **** 2021-02-10T11:07:30.728 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-02-10T11:07:30.730 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:30.764 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:30 +0000 (0:00:00.074) 0:05:46.342 **** 2021-02-10T11:07:30.803 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-02-10T11:07:30.805 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:30.837 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:30 +0000 (0:00:00.073) 0:05:46.415 **** 2021-02-10T11:07:30.875 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-10T11:07:30.877 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:30.908 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:30 +0000 (0:00:00.071) 0:05:46.487 **** 2021-02-10T11:07:30.936 INFO:teuthology.task.ansible.out: 2021-02-10T11:07:30.938 INFO:teuthology.task.ansible.out:TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:30.989 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:30 +0000 (0:00:00.081) 0:05:46.568 **** 2021-02-10T11:07:31.021 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-02-10T11:07:31.023 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:31.063 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:31 +0000 (0:00:00.073) 0:05:46.642 **** 2021-02-10T11:07:31.093 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2021-02-10T11:07:31.094 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:31.149 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:31 +0000 (0:00:00.085) 0:05:46.728 **** 2021-02-10T11:07:31.159 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-02-10T11:07:31.161 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nfs-server", "status": {"ActiveEnterTimestamp": "Wed 2021-02-10 11:00:51 UTC", "ActiveEnterTimestampMonotonic": "20092639", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "nfs-idmapd.service nfs-config.service rpcbind.socket systemd-journald.socket gssproxy.service network-online.target system.slice local-fs.target rpc-statd.service proc-fs-nfsd.mount rpc-gssd.service nfs-mountd.service -.mount tmp.mount", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Wed 2021-02-10 11:00:51 UTC", "AssertTimestampMonotonic": "19922320", "Before": "rpc-statd-notify.service", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BoundBy": "nfs-idmapd.service nfs-mountd.service", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Wed 2021-02-10 11:00:51 UTC", "ConditionTimestampMonotonic": "19922319", "ControlGroup": "/system.slice/nfs-server.service", "ControlPID": "0", "DefaultDependencies": "no", "Delegate": "no", "Description": "NFS server and services", "DevicePolicy": "auto", "DropInPaths": "/run/systemd/generator/nfs-server.service.d/order-with-mounts.conf", "EnvironmentFile": "/run/sysconfig/nfs-utils (ignore_errors=yes)", "ExecMainCode": "1", "ExecMainExitTimestamp": "Wed 2021-02-10 11:00:51 UTC", "ExecMainExitTimestampMonotonic": "20077423", "ExecMainPID": "1217", "ExecMainStartTimestamp": "Wed 2021-02-10 11:00:51 UTC", "ExecMainStartTimestampMonotonic": "19942327", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/exportfs ; argv[]=/usr/sbin/exportfs -r ; ignore_errors=yes 2021-02-10T11:07:31.162 INFO:teuthology.task.ansible.out: ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/rpc.nfsd ; argv[]=/usr/sbin/rpc.nfsd $RPCNFSDARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPost": "{ path=/bin/sh ; argv[]=/bin/sh -c if systemctl -q is-active gssproxy; then systemctl reload gssproxy ; fi ; ignore_errors=yes ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPre": "{ path=/usr/sbin/exportfs ; argv[]=/usr/sbin/exportfs -r ; ignore_errors=yes ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/rpc.nfsd ; argv[]=/usr/sbin/rpc.nfsd 0 ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/usr/sbin/exportfs ; argv[]=/usr/sbin/exportfs -f ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nfs-server.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nfs-server.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Wed 2021-02-10 11:00:51 UTC", "InactiveExitTimestampMonotonic": "19923706", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127560", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127560", "LimitSTACK": "18446744073709551615", "LoadState": "lo 2021-02-10T11:07:31.164 INFO:teuthology.task.ansible.out:aded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nfs-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "yes", "Requires": "network.target proc-fs-nfsd.mount nfs-mountd.service system.slice -.mount", "RequiresMountsFor": "/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "exited", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "0", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "oneshot", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "rpc-statd-notify.service rpc-statd.service auth-rpcgss-module.service nfs-idmapd.service nfs-config.service rpcbind.socket network-online.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-02-10T11:07:31.211 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:31 +0000 (0:00:00.061) 0:05:46.789 **** 2021-02-10T11:07:31.219 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-02-10T11:07:31.219 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:31.269 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:31 +0000 (0:00:00.058) 0:05:46.847 **** 2021-02-10T11:07:31.277 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-02-10T11:07:31.279 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:31.319 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:31 +0000 (0:00:00.049) 0:05:46.897 **** 2021-02-10T11:07:31.326 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-02-10T11:07:31.328 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:31.365 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:31 +0000 (0:00:00.046) 0:05:46.944 **** 2021-02-10T11:07:31.372 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-10T11:07:31.374 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:31.412 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:31 +0000 (0:00:00.046) 0:05:46.991 **** 2021-02-10T11:07:31.426 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-02-10T11:07:31.427 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:31.467 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:31 +0000 (0:00:00.054) 0:05:47.045 **** 2021-02-10T11:07:31.474 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-02-10T11:07:31.476 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:31.519 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:31 +0000 (0:00:00.052) 0:05:47.097 **** 2021-02-10T11:07:31.528 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-02-10T11:07:31.529 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:31.585 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:31 +0000 (0:00:00.066) 0:05:47.164 **** 2021-02-10T11:07:31.599 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-02-10T11:07:31.600 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:31.639 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:31 +0000 (0:00:00.053) 0:05:47.217 **** 2021-02-10T11:07:31.647 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-02-10T11:07:31.649 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:31.684 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:31 +0000 (0:00:00.045) 0:05:47.263 **** 2021-02-10T11:07:31.690 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-10T11:07:31.692 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:31.727 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:31 +0000 (0:00:00.043) 0:05:47.306 **** 2021-02-10T11:07:31.735 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-02-10T11:07:31.736 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:31.775 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:31 +0000 (0:00:00.047) 0:05:47.353 **** 2021-02-10T11:07:31.782 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-02-10T11:07:31.783 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:31.818 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:31 +0000 (0:00:00.042) 0:05:47.396 **** 2021-02-10T11:07:31.825 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:31.881 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:31 +0000 (0:00:00.063) 0:05:47.460 **** 2021-02-10T11:07:31.888 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-02-10T11:07:31.890 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:31.924 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:31 +0000 (0:00:00.042) 0:05:47.503 **** 2021-02-10T11:07:31.930 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2021-02-10T11:07:31.931 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:31.965 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:31 +0000 (0:00:00.041) 0:05:47.544 **** 2021-02-10T11:07:31.972 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2021-02-10T11:07:31.974 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:32.019 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:32 +0000 (0:00:00.053) 0:05:47.598 **** 2021-02-10T11:07:32.026 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:32.065 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:32 +0000 (0:00:00.045) 0:05:47.643 **** 2021-02-10T11:07:32.073 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-02-10T11:07:32.075 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:32.121 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:32 +0000 (0:00:00.055) 0:05:47.699 **** 2021-02-10T11:07:32.129 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-02-10T11:07:32.129 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:32.165 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:32 +0000 (0:00:00.044) 0:05:47.744 **** 2021-02-10T11:07:32.532 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Wednesday 10 February 2021 11:07:32 +0000 (0:00:00.054) 0:05:47.799 **** TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Wednesday 10 February 2021 11:07:32 +0000 (0:00:00.055) 0:05:47.855 **** TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Wednesday 10 February 2021 11:07:32 +0000 (0:00:00.066) 0:05:47.921 **** TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Wednesday 10 February 2021 11:07:32 +0000 (0:00:00.056) 0:05:47.978 **** TASK [testnode : Upload weekly kernel-clean crontab.] ************************** skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Wednesday 10 February 2021 11:07:32 +0000 (0:00:00.050) 0:05:48.029 **** TASK [testnode : set_fact] ***************************************************** ok: [gibba001.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} Wednesday 10 February 2021 11:07:32 +0000 (0:00:00.054) 0:05:48.084 **** TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:32.559 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:32 +0000 (0:00:00.054) 0:05:48.138 **** 2021-02-10T11:07:32.567 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-02-10T11:07:32.568 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-02-10T11:07:32.580 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-02-10T11:07:32.594 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-02-10T11:07:32.595 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:32.642 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:32 +0000 (0:00:00.082) 0:05:48.220 **** 2021-02-10T11:07:32.652 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-02-10T11:07:32.654 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-02-10T11:07:32.656 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-02-10T11:07:32.657 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-02-10T11:07:32.712 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:32 +0000 (0:00:00.070) 0:05:48.290 **** 2021-02-10T11:07:32.725 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-02-10T11:07:32.727 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:32.776 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:32 +0000 (0:00:00.064) 0:05:48.355 **** 2021-02-10T11:07:32.820 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2021-02-10T11:07:32.820 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:32.820 INFO:teuthology.task.ansible.out: 2021-02-10T11:07:32.883 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:32 +0000 (0:00:00.106) 0:05:48.462 **** 2021-02-10T11:07:32.951 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:32.985 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:32 +0000 (0:00:00.102) 0:05:48.564 **** 2021-02-10T11:07:33.014 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:33.063 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:33 +0000 (0:00:00.078) 0:05:48.642 **** 2021-02-10T11:07:33.074 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-02-10T11:07:33.075 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.008688", "end": "2021-02-10 11:07:32.994904", "rc": 0, "start": "2021-02-10 11:07:32.986216", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/virtualenv/bin/teuthology-lock"]} 2021-02-10T11:07:33.122 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:33 +0000 (0:00:00.058) 0:05:48.701 **** 2021-02-10T11:07:33.131 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2021-02-10T11:07:33.132 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:33.175 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:33 +0000 (0:00:00.051) 0:05:48.752 **** 2021-02-10T11:07:33.180 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2021-02-10T11:07:33.182 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:33.223 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:33 +0000 (0:00:00.049) 0:05:48.802 **** 2021-02-10T11:07:33.230 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-02-10T11:07:33.232 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:33.274 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:33 +0000 (0:00:00.050) 0:05:48.852 **** 2021-02-10T11:07:33.285 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:33.341 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:33 +0000 (0:00:00.067) 0:05:48.920 **** 2021-02-10T11:07:33.349 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-10T11:07:33.350 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-02-10T11:07:33.389 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:33 +0000 (0:00:00.047) 0:05:48.967 **** 2021-02-10T11:07:33.397 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:33.432 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:33 +0000 (0:00:00.043) 0:05:49.011 **** 2021-02-10T11:07:33.453 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-02-10T11:07:33.454 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-02-10T11:07:33.464 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:33.465 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** ok: [gibba008.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-02-10T11:07:33.518 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:33 +0000 (0:00:00.085) 0:05:49.097 **** 2021-02-10T11:07:33.525 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-02-10T11:07:33.525 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-02-10T11:07:33.527 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [gibba008.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-02-10T11:07:33.527 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-02-10T11:07:33.568 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:33 +0000 (0:00:00.049) 0:05:49.146 **** 2021-02-10T11:07:33.811 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-02-10T11:07:33.811 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.004468", "end": "2021-02-10 11:07:33.787928", "rc": 0, "start": "2021-02-10 11:07:33.783460", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/virtualenv/bin/teuthology-lock"]} 2021-02-10T11:07:33.867 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:33 +0000 (0:00:00.300) 0:05:49.446 **** 2021-02-10T11:07:33.909 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:33.957 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:33 +0000 (0:00:00.090) 0:05:49.536 **** 2021-02-10T11:07:33.987 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-02-10T11:07:33.988 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:34.028 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:34 +0000 (0:00:00.070) 0:05:49.607 **** 2021-02-10T11:07:34.067 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-02-10T11:07:34.068 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:34.103 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:34 +0000 (0:00:00.075) 0:05:49.682 **** 2021-02-10T11:07:34.131 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-02-10T11:07:34.132 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:34.166 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:34 +0000 (0:00:00.062) 0:05:49.745 **** 2021-02-10T11:07:34.353 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-02-10T11:07:34.354 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-3.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.170-6.el7.x86_64 providing device-mapper is already installed"]} 2021-02-10T11:07:34.392 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:34 +0000 (0:00:00.226) 0:05:49.971 **** 2021-02-10T11:07:34.445 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-02-10T11:07:34.446 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item={'block_used': 13742196, 'uuid': 'aeb0602f-c6fc-4ca1-8920-0340080ffa5a', 'size_total': 984369086464, 'block_total': 240324484, 'mount': '/', 'block_available': 226582288, 'size_available': 928081051648, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,seclabel,relatime,data=ordered', 'device': '/dev/sda1', 'inode_used': 90165, 'block_size': 4096, 'inode_available': 60964811}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226582288, "block_size": 4096, "block_total": 240324484, "block_used": 13742196, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60964811, "inode_total": 61054976, "inode_used": 90165, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 928081051648, "size_total": 984369086464, "uuid": "aeb0602f-c6fc-4ca1-8920-0340080ffa5a"}} 2021-02-10T11:07:34.486 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:34 +0000 (0:00:00.093) 0:05:50.065 **** 2021-02-10T11:07:34.938 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-02-10T11:07:34.939 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.004523", "end": "2021-02-10 11:07:34.921901", "rc": 0, "start": "2021-02-10 11:07:34.917378", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-10T11:07:34.974 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:34 +0000 (0:00:00.487) 0:05:50.552 **** 2021-02-10T11:07:35.027 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** skipping: [gibba001.front.sepia.ceph.com] => (item={'block_used': 13742196, 'uuid': 'aeb0602f-c6fc-4ca1-8920-0340080ffa5a', 'size_total': 984369086464, 'block_total': 240324484, 'mount': '/', 'block_available': 226582288, 'size_available': 928081051648, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,seclabel,relatime,data=ordered', 'device': '/dev/sda1', 'inode_used': 90165, 'block_size': 4096, 'inode_available': 60964811}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226582288, "block_size": 4096, "block_total": 240324484, "block_used": 13742196, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60964811, "inode_total": 61054976, "inode_used": 90165, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 928081051648, "size_total": 984369086464, "uuid": "aeb0602f-c6fc-4ca1-8920-0340080ffa5a"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:35.086 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:35 +0000 (0:00:00.111) 0:05:50.664 **** 2021-02-10T11:07:35.100 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-02-10T11:07:35.101 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-3.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.170-6.el7.x86_64 providing device-mapper is already installed"]} 2021-02-10T11:07:35.146 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:35 +0000 (0:00:00.060) 0:05:50.725 **** 2021-02-10T11:07:35.154 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-02-10T11:07:35.195 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:35 +0000 (0:00:00.048) 0:05:50.774 **** 2021-02-10T11:07:35.212 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-02-10T11:07:35.214 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item={'block_used': 13742196, 'uuid': 'aeb0602f-c6fc-4ca1-8920-0340080ffa5a', 'size_total': 984369086464, 'block_total': 240324484, 'mount': '/', 'block_available': 226582288, 'size_available': 928081051648, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,seclabel,relatime,data=ordered', 'device': '/dev/sda1', 'inode_used': 90165, 'block_size': 4096, 'inode_available': 60964811}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226582288, "block_size": 4096, "block_total": 240324484, "block_used": 13742196, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60964811, "inode_total": 61054976, "inode_used": 90165, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 928081051648, "size_total": 984369086464, "uuid": "aeb0602f-c6fc-4ca1-8920-0340080ffa5a"}} 2021-02-10T11:07:35.251 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:35 +0000 (0:00:00.055) 0:05:50.830 **** 2021-02-10T11:07:35.259 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-02-10T11:07:35.259 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I3K2IYF6XF', 'wwn-0x50000399ab801df6'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'aeb0602f-c6fc-4ca1-8920-0340080ffa5a', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I3K2IYF6XF-part1', 'wwn-0x50000399ab801df6-part1'], 'uuids': ['aeb0602f-c6fc-4ca1-8920-0340080ffa5a']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab801df6', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I3K2IYF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I3K2IYF6XF", "wwn-0x50000399ab801df6"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I3K2IYF6XF-part1", "wwn-0x50000399ab801df6-part1"], "labels": [], "masters": [], "uuids": ["aeb0602f-c6fc-4ca1-8920-0340080ffa5a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "aeb0602f-c6fc-4ca1-8920-0340080ffa5a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I3K2IYF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab801df6"}}, "skip_reason": "Conditiona 2021-02-10T11:07:35.261 INFO:teuthology.task.ansible.out:l result was False"} 2021-02-10T11:07:35.702 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-02-10T11:07:35.703 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.004672", "end": "2021-02-10 11:07:35.684706", "rc": 0, "start": "2021-02-10 11:07:35.680034", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-10T11:07:35.759 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:35 +0000 (0:00:00.507) 0:05:51.338 **** 2021-02-10T11:07:35.797 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-02-10T11:07:35.798 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'block_used': 13742196, 'uuid': 'aeb0602f-c6fc-4ca1-8920-0340080ffa5a', 'size_total': 984369086464, 'block_total': 240324484, 'mount': '/', 'block_available': 226582288, 'size_available': 928081051648, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,seclabel,relatime,data=ordered', 'device': '/dev/sda1', 'inode_used': 90165, 'block_size': 4096, 'inode_available': 60964811}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226582288, "block_size": 4096, "block_total": 240324484, "block_used": 13742196, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60964811, "inode_total": 61054976, "inode_used": 90165, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 928081051648, "size_total": 984369086464, "uuid": "aeb0602f-c6fc-4ca1-8920-0340080ffa5a"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:35.831 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:35 +0000 (0:00:00.072) 0:05:51.410 **** 2021-02-10T11:07:35.858 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-02-10T11:07:35.891 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:35 +0000 (0:00:00.059) 0:05:51.470 **** 2021-02-10T11:07:35.939 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-02-10T11:07:35.939 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2L5F6XF', 'wwn-0x50000399ab9813f7'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'aeb0602f-c6fc-4ca1-8920-0340080ffa5a', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2L5F6XF-part1', 'wwn-0x50000399ab9813f7-part1'], 'uuids': ['aeb0602f-c6fc-4ca1-8920-0340080ffa5a']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab9813f7', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I6K2L5F6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2L5F6XF", "wwn-0x50000399ab9813f7"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2L5F6XF-part1", "wwn-0x50000399ab9813f7-part1"], "labels": [], "masters": [], "uuids": ["aeb0602f-c6fc-4ca1-8920-0340080ffa5a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "aeb0602f-c6fc-4ca1-8920-0340080ffa5a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I6K2L5F6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab9813f7"}}, "skip_reason": "Conditiona 2021-02-10T11:07:35.941 INFO:teuthology.task.ansible.out:l result was False"} 2021-02-10T11:07:37.181 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-KxvWIs-hGfb-GI2q-YLry-eNcw-gd04-PxqcZ7', 'nvme-INTEL_SSDPEL1K375GA_PHKM9200008P375A', 'nvme-nvme.8086-50484b4d393230303030385033373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': ['vg_nvme-lv_1', 'vg_nvme-lv_2', 'vg_nvme-lv_3', 'vg_nvme-lv_4', 'vg_nvme-lv_5'], 'partitions': {}, 'model': 'INTEL SSDPEL1K375GA', 'serial': 'PHKM9200008P375A', 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.498136", "end": "2021-02-10 11:07:37.156971", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_1", "vg_nvme-lv_2", "vg_nvme-lv_3", "vg_nvme-lv_4", "vg_nvme-lv_5"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-KxvWIs-hGfb-GI2q-YLry-eNcw-gd04-PxqcZ7", "nvme-INTEL_SSDPEL1K375GA_PHKM9200008P375A", "nvme-nvme.8086-50484b4d393230303030385033373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "serial": "PHKM9200008P375A", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:35.658835", "stderr": "", "stderr_lines": [], "stdout": "Creating new 2021-02-10T11:07:37.182 INFO:teuthology.task.ansible.out:GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-10T11:07:37.865 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-je0Evm-WDG9-qGMt-DQzL-aFCD-RIs0-VeJryu', 'nvme-INTEL_SSDPEL1K375GA_PHKM913400DK375A', 'nvme-nvme.8086-50484b4d393133343030444b33373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': ['vg_nvme-lv_1', 'vg_nvme-lv_2', 'vg_nvme-lv_3', 'vg_nvme-lv_4', 'vg_nvme-lv_5'], 'partitions': {}, 'model': 'INTEL SSDPEL1K375GA', 'serial': 'PHKM913400DK375A', 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.484298", "end": "2021-02-10 11:07:37.840581", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_1", "vg_nvme-lv_2", "vg_nvme-lv_3", "vg_nvme-lv_4", "vg_nvme-lv_5"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-je0Evm-WDG9-qGMt-DQzL-aFCD-RIs0-VeJryu", "nvme-INTEL_SSDPEL1K375GA_PHKM913400DK375A", "nvme-nvme.8086-50484b4d393133343030444b33373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "serial": "PHKM913400DK375A", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:36.356283", "stderr": "", "stderr_lines": [], "stdout": "Creating new 2021-02-10T11:07:37.866 INFO:teuthology.task.ansible.out:GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-10T11:07:38.682 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-zCbw8Fs3eOyN3PJm2QSdgq7M8dZH3rtvvIRJOrLN8WgRNY1Po8ZwTtzblyZ8FZgO'], 'uuids': ['d5150095-672a-4809-89cd-9ae4404dbf5b']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHKM9200008P375A', 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.069259", "end": "2021-02-10 11:07:38.663217", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-zCbw8Fs3eOyN3PJm2QSdgq7M8dZH3rtvvIRJOrLN8WgRNY1Po8ZwTtzblyZ8FZgO"], "labels": [], "masters": [], "uuids": ["d5150095-672a-4809-89cd-9ae4404dbf5b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "serial": "PHKM9200008P375A", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:37.593958", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-10T11:07:39.362 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-M4g29rtq99lTYc0npgJTACHCTU5jX4DP60AojITqRhkfWnhwISsbYTm3OeGCUwYF'], 'uuids': ['124587a6-4f6e-41fe-832c-be488a1b8942']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHKM913400DK375A', 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.060227", "end": "2021-02-10 11:07:39.340520", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-M4g29rtq99lTYc0npgJTACHCTU5jX4DP60AojITqRhkfWnhwISsbYTm3OeGCUwYF"], "labels": [], "masters": [], "uuids": ["124587a6-4f6e-41fe-832c-be488a1b8942"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "serial": "PHKM913400DK375A", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:38.280293", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-10T11:07:40.166 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-zCbw8Fs3eOyN3PJm2QSdgq7M8dZH3rtvsDT1Mc2ssyBbx6hHqz4Kj726HY153QD3'], 'uuids': ['ed01cbe7-4608-48df-819e-158f56059727']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHKM9200008P375A', 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.048686", "end": "2021-02-10 11:07:40.146240", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-zCbw8Fs3eOyN3PJm2QSdgq7M8dZH3rtvsDT1Mc2ssyBbx6hHqz4Kj726HY153QD3"], "labels": [], "masters": [], "uuids": ["ed01cbe7-4608-48df-819e-158f56059727"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "serial": "PHKM9200008P375A", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:39.097554", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-10T11:07:40.846 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-M4g29rtq99lTYc0npgJTACHCTU5jX4DPqjzMoDTsDlt20juxsCG6Cd4GYo1tWSeU'], 'uuids': ['bd3da756-a618-4464-846a-a13d63c23c0a']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHKM913400DK375A', 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.049356", "end": "2021-02-10 11:07:40.823686", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-M4g29rtq99lTYc0npgJTACHCTU5jX4DPqjzMoDTsDlt20juxsCG6Cd4GYo1tWSeU"], "labels": [], "masters": [], "uuids": ["bd3da756-a618-4464-846a-a13d63c23c0a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "serial": "PHKM913400DK375A", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:39.774330", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-10T11:07:41.658 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-zCbw8Fs3eOyN3PJm2QSdgq7M8dZH3rtv832jW7Ht489UH93E5WFQ4EBFxpkx3hKs'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHKM9200008P375A', 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.045239", "end": "2021-02-10 11:07:41.629691", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-zCbw8Fs3eOyN3PJm2QSdgq7M8dZH3rtv832jW7Ht489UH93E5WFQ4EBFxpkx3hKs"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "serial": "PHKM9200008P375A", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:40.584452", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-10T11:07:42.327 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-M4g29rtq99lTYc0npgJTACHCTU5jX4DP33GXabxdp1xF0hjs3NtL6opU5DbW5sSD'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHKM913400DK375A', 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.053442", "end": "2021-02-10 11:07:42.308745", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-M4g29rtq99lTYc0npgJTACHCTU5jX4DP33GXabxdp1xF0hjs3NtL6opU5DbW5sSD"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "serial": "PHKM913400DK375A", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:41.255303", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-10T11:07:43.148 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-zCbw8Fs3eOyN3PJm2QSdgq7M8dZH3rtvzP8qNWladoGcjdt9FoJjy0hBqDEdX9nI'], 'uuids': ['073982bb-400e-4542-ba55-30a381c6878b']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHKM9200008P375A', 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.059727", "end": "2021-02-10 11:07:43.129050", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-zCbw8Fs3eOyN3PJm2QSdgq7M8dZH3rtvzP8qNWladoGcjdt9FoJjy0hBqDEdX9nI"], "labels": [], "masters": [], "uuids": ["073982bb-400e-4542-ba55-30a381c6878b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "serial": "PHKM9200008P375A", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:42.069323", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-10T11:07:43.866 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-M4g29rtq99lTYc0npgJTACHCTU5jX4DPmoXsePlypHqTfbK08WPNTY0hrrbESOfl'], 'uuids': ['73dea440-7c76-48d6-bdda-3f3b779328a8']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHKM913400DK375A', 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.095609", "end": "2021-02-10 11:07:43.844114", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-M4g29rtq99lTYc0npgJTACHCTU5jX4DPmoXsePlypHqTfbK08WPNTY0hrrbESOfl"], "labels": [], "masters": [], "uuids": ["73dea440-7c76-48d6-bdda-3f3b779328a8"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "serial": "PHKM913400DK375A", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:42.748505", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-10T11:07:44.623 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-zCbw8Fs3eOyN3PJm2QSdgq7M8dZH3rtvdEhFys0H80kU3T1yG6I5dJl2fVw3QkGL'], 'uuids': ['bbc668d3-795b-48a2-be49-d4b5b704ecd0']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHKM9200008P375A', 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.042574", "end": "2021-02-10 11:07:44.605282", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-zCbw8Fs3eOyN3PJm2QSdgq7M8dZH3rtvdEhFys0H80kU3T1yG6I5dJl2fVw3QkGL"], "labels": [], "masters": [], "uuids": ["bbc668d3-795b-48a2-be49-d4b5b704ecd0"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "serial": "PHKM9200008P375A", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:43.562708", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-10T11:07:44.663 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:44 +0000 (0:00:08.771) 0:06:00.242 **** 2021-02-10T11:07:44.701 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-02-10T11:07:44.701 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I3K2IYF6XF', 'wwn-0x50000399ab801df6'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'aeb0602f-c6fc-4ca1-8920-0340080ffa5a', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I3K2IYF6XF-part1', 'wwn-0x50000399ab801df6-part1'], 'uuids': ['aeb0602f-c6fc-4ca1-8920-0340080ffa5a']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab801df6', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I3K2IYF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I3K2IYF6XF", "wwn-0x50000399ab801df6"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I3K2IYF6XF-part1", "wwn-0x50000399ab801df6-part1"], "labels": [], "masters": [], "uuids": ["aeb0602f-c6fc-4ca1-8920-0340080ffa5a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "aeb0602f-c6fc-4ca1-8920-0340080ffa5a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I3K2IYF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab801df6"}}, "skip_reason": "Conditiona 2021-02-10T11:07:44.703 INFO:teuthology.task.ansible.out:l result was False"} 2021-02-10T11:07:45.162 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-KxvWIs-hGfb-GI2q-YLry-eNcw-gd04-PxqcZ7', 'nvme-INTEL_SSDPEL1K375GA_PHKM9200008P375A', 'nvme-nvme.8086-50484b4d393230303030385033373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': ['vg_nvme-lv_1', 'vg_nvme-lv_2', 'vg_nvme-lv_3', 'vg_nvme-lv_4', 'vg_nvme-lv_5'], 'partitions': {}, 'model': 'INTEL SSDPEL1K375GA', 'serial': 'PHKM9200008P375A', 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.026276", "end": "2021-02-10 11:07:45.145792", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_1", "vg_nvme-lv_2", "vg_nvme-lv_3", "vg_nvme-lv_4", "vg_nvme-lv_5"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-KxvWIs-hGfb-GI2q-YLry-eNcw-gd04-PxqcZ7", "nvme-INTEL_SSDPEL1K375GA_PHKM9200008P375A", "nvme-nvme.8086-50484b4d393230303030385033373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "serial": "PHKM9200008P375A", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:45.119516", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) 2021-02-10T11:07:45.164 INFO:teuthology.task.ansible.out:copied, 0.0241353 s, 4.8 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0241353 s, 4.8 GB/s"], "stdout": "", "stdout_lines": []} 2021-02-10T11:07:45.351 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-02-10T11:07:45.353 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-M4g29rtq99lTYc0npgJTACHCTU5jX4DPRePhzColdjlN6yvGVUPoCBXJ3jPmvTQP'], 'uuids': ['c7adf339-8611-4850-8370-3bb7d86d45bc']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHKM913400DK375A', 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.049811", "end": "2021-02-10 11:07:45.326196", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-M4g29rtq99lTYc0npgJTACHCTU5jX4DPRePhzColdjlN6yvGVUPoCBXJ3jPmvTQP"], "labels": [], "masters": [], "uuids": ["c7adf339-8611-4850-8370-3bb7d86d45bc"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "serial": "PHKM913400DK375A", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:44.276385", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-10T11:07:45.394 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:45 +0000 (0:00:00.730) 0:06:00.972 **** 2021-02-10T11:07:45.432 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-02-10T11:07:45.433 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2L5F6XF', 'wwn-0x50000399ab9813f7'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'aeb0602f-c6fc-4ca1-8920-0340080ffa5a', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2L5F6XF-part1', 'wwn-0x50000399ab9813f7-part1'], 'uuids': ['aeb0602f-c6fc-4ca1-8920-0340080ffa5a']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab9813f7', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I6K2L5F6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2L5F6XF", "wwn-0x50000399ab9813f7"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2L5F6XF-part1", "wwn-0x50000399ab9813f7-part1"], "labels": [], "masters": [], "uuids": ["aeb0602f-c6fc-4ca1-8920-0340080ffa5a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "aeb0602f-c6fc-4ca1-8920-0340080ffa5a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I6K2L5F6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab9813f7"}}, "skip_reason": "Conditiona 2021-02-10T11:07:45.435 INFO:teuthology.task.ansible.out:l result was False"} 2021-02-10T11:07:45.675 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-zCbw8Fs3eOyN3PJm2QSdgq7M8dZH3rtvvIRJOrLN8WgRNY1Po8ZwTtzblyZ8FZgO'], 'uuids': ['d5150095-672a-4809-89cd-9ae4404dbf5b']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHKM9200008P375A', 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.081023", "end": "2021-02-10 11:07:45.657423", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-zCbw8Fs3eOyN3PJm2QSdgq7M8dZH3rtvvIRJOrLN8WgRNY1Po8ZwTtzblyZ8FZgO"], "labels": [], "masters": [], "uuids": ["d5150095-672a-4809-89cd-9ae4404dbf5b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "serial": "PHKM9200008P375A", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:45.576400", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0788598 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0788598 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-02-10T11:07:45.897 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-je0Evm-WDG9-qGMt-DQzL-aFCD-RIs0-VeJryu', 'nvme-INTEL_SSDPEL1K375GA_PHKM913400DK375A', 'nvme-nvme.8086-50484b4d393133343030444b33373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': ['vg_nvme-lv_1', 'vg_nvme-lv_2', 'vg_nvme-lv_3', 'vg_nvme-lv_4', 'vg_nvme-lv_5'], 'partitions': {}, 'model': 'INTEL SSDPEL1K375GA', 'serial': 'PHKM913400DK375A', 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.026148", "end": "2021-02-10 11:07:45.872850", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_1", "vg_nvme-lv_2", "vg_nvme-lv_3", "vg_nvme-lv_4", "vg_nvme-lv_5"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-je0Evm-WDG9-qGMt-DQzL-aFCD-RIs0-VeJryu", "nvme-INTEL_SSDPEL1K375GA_PHKM913400DK375A", "nvme-nvme.8086-50484b4d393133343030444b33373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "serial": "PHKM913400DK375A", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:45.846702", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) 2021-02-10T11:07:45.898 INFO:teuthology.task.ansible.out:copied, 0.0240899 s, 4.8 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0240899 s, 4.8 GB/s"], "stdout": "", "stdout_lines": []} 2021-02-10T11:07:46.191 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-zCbw8Fs3eOyN3PJm2QSdgq7M8dZH3rtvsDT1Mc2ssyBbx6hHqz4Kj726HY153QD3'], 'uuids': ['ed01cbe7-4608-48df-819e-158f56059727']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHKM9200008P375A', 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.080995", "end": "2021-02-10 11:07:46.167754", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-zCbw8Fs3eOyN3PJm2QSdgq7M8dZH3rtvsDT1Mc2ssyBbx6hHqz4Kj726HY153QD3"], "labels": [], "masters": [], "uuids": ["ed01cbe7-4608-48df-819e-158f56059727"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "serial": "PHKM9200008P375A", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:46.086759", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0789193 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0789193 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-02-10T11:07:46.413 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-M4g29rtq99lTYc0npgJTACHCTU5jX4DP60AojITqRhkfWnhwISsbYTm3OeGCUwYF'], 'uuids': ['124587a6-4f6e-41fe-832c-be488a1b8942']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHKM913400DK375A', 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.080907", "end": "2021-02-10 11:07:46.389406", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-M4g29rtq99lTYc0npgJTACHCTU5jX4DP60AojITqRhkfWnhwISsbYTm3OeGCUwYF"], "labels": [], "masters": [], "uuids": ["124587a6-4f6e-41fe-832c-be488a1b8942"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "serial": "PHKM913400DK375A", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:46.308499", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0788569 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0788569 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-02-10T11:07:46.708 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-zCbw8Fs3eOyN3PJm2QSdgq7M8dZH3rtv832jW7Ht489UH93E5WFQ4EBFxpkx3hKs'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHKM9200008P375A', 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.084240", "end": "2021-02-10 11:07:46.683798", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-zCbw8Fs3eOyN3PJm2QSdgq7M8dZH3rtv832jW7Ht489UH93E5WFQ4EBFxpkx3hKs"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "serial": "PHKM9200008P375A", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:46.599558", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0821103 s, 1.4 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0821103 s, 1.4 GB/s"], "stdout": "", "stdout_lines": []} 2021-02-10T11:07:46.930 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-M4g29rtq99lTYc0npgJTACHCTU5jX4DPqjzMoDTsDlt20juxsCG6Cd4GYo1tWSeU'], 'uuids': ['bd3da756-a618-4464-846a-a13d63c23c0a']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHKM913400DK375A', 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.083144", "end": "2021-02-10 11:07:46.912863", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-M4g29rtq99lTYc0npgJTACHCTU5jX4DPqjzMoDTsDlt20juxsCG6Cd4GYo1tWSeU"], "labels": [], "masters": [], "uuids": ["bd3da756-a618-4464-846a-a13d63c23c0a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "serial": "PHKM913400DK375A", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:46.829719", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0811356 s, 1.4 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0811356 s, 1.4 GB/s"], "stdout": "", "stdout_lines": []} 2021-02-10T11:07:47.224 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-zCbw8Fs3eOyN3PJm2QSdgq7M8dZH3rtvzP8qNWladoGcjdt9FoJjy0hBqDEdX9nI'], 'uuids': ['073982bb-400e-4542-ba55-30a381c6878b']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHKM9200008P375A', 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.081407", "end": "2021-02-10 11:07:47.201709", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-zCbw8Fs3eOyN3PJm2QSdgq7M8dZH3rtvzP8qNWladoGcjdt9FoJjy0hBqDEdX9nI"], "labels": [], "masters": [], "uuids": ["073982bb-400e-4542-ba55-30a381c6878b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "serial": "PHKM9200008P375A", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:47.120302", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0793423 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0793423 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-02-10T11:07:47.444 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-M4g29rtq99lTYc0npgJTACHCTU5jX4DP33GXabxdp1xF0hjs3NtL6opU5DbW5sSD'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHKM913400DK375A', 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.080397", "end": "2021-02-10 11:07:47.426809", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-M4g29rtq99lTYc0npgJTACHCTU5jX4DP33GXabxdp1xF0hjs3NtL6opU5DbW5sSD"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "serial": "PHKM913400DK375A", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:47.346412", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0783503 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0783503 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-02-10T11:07:47.747 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-zCbw8Fs3eOyN3PJm2QSdgq7M8dZH3rtvdEhFys0H80kU3T1yG6I5dJl2fVw3QkGL'], 'uuids': ['bbc668d3-795b-48a2-be49-d4b5b704ecd0']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHKM9200008P375A', 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.081875", "end": "2021-02-10 11:07:47.723427", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-zCbw8Fs3eOyN3PJm2QSdgq7M8dZH3rtvdEhFys0H80kU3T1yG6I5dJl2fVw3QkGL"], "labels": [], "masters": [], "uuids": ["bbc668d3-795b-48a2-be49-d4b5b704ecd0"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "serial": "PHKM9200008P375A", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:47.641552", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.079799 s, 1.4 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.079799 s, 1.4 GB/s"], "stdout": "", "stdout_lines": []} 2021-02-10T11:07:47.789 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:47 +0000 (0:00:02.395) 0:06:03.368 **** 2021-02-10T11:07:47.957 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-M4g29rtq99lTYc0npgJTACHCTU5jX4DPmoXsePlypHqTfbK08WPNTY0hrrbESOfl'], 'uuids': ['73dea440-7c76-48d6-bdda-3f3b779328a8']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHKM913400DK375A', 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.080985", "end": "2021-02-10 11:07:47.938648", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-M4g29rtq99lTYc0npgJTACHCTU5jX4DPmoXsePlypHqTfbK08WPNTY0hrrbESOfl"], "labels": [], "masters": [], "uuids": ["73dea440-7c76-48d6-bdda-3f3b779328a8"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "serial": "PHKM913400DK375A", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:47.857663", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0788759 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0788759 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-02-10T11:07:48.319 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2021-02-10T11:07:48.322 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.073288", "end": "2021-02-10 11:07:48.292507", "rc": 0, "start": "2021-02-10 11:07:48.219219", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-10T11:07:48.355 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:48 +0000 (0:00:00.565) 0:06:03.934 **** 2021-02-10T11:07:48.482 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-02-10T11:07:48.483 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-M4g29rtq99lTYc0npgJTACHCTU5jX4DPRePhzColdjlN6yvGVUPoCBXJ3jPmvTQP'], 'uuids': ['c7adf339-8611-4850-8370-3bb7d86d45bc']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHKM913400DK375A', 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.080450", "end": "2021-02-10 11:07:48.464168", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-M4g29rtq99lTYc0npgJTACHCTU5jX4DPRePhzColdjlN6yvGVUPoCBXJ3jPmvTQP"], "labels": [], "masters": [], "uuids": ["c7adf339-8611-4850-8370-3bb7d86d45bc"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "serial": "PHKM913400DK375A", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-10 11:07:48.383718", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0784281 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0784281 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-02-10T11:07:48.530 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:48 +0000 (0:00:00.174) 0:06:04.109 **** 2021-02-10T11:07:48.855 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-02-10T11:07:48.856 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.035509", "end": "2021-02-10 11:07:48.834178", "rc": 0, "start": "2021-02-10 11:07:48.798669", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-10T11:07:48.890 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:48 +0000 (0:00:00.359) 0:06:04.469 **** 2021-02-10T11:07:48.917 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-02-10T11:07:48.953 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:48 +0000 (0:00:00.062) 0:06:04.532 **** 2021-02-10T11:07:48.980 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-02-10T11:07:48.981 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:49.017 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:49 +0000 (0:00:00.063) 0:06:04.595 **** 2021-02-10T11:07:49.045 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-02-10T11:07:49.046 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:49.078 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:49 +0000 (0:00:00.061) 0:06:04.657 **** 2021-02-10T11:07:49.087 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2021-02-10T11:07:49.088 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.088132", "end": "2021-02-10 11:07:49.051824", "rc": 0, "start": "2021-02-10 11:07:48.963692", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-10T11:07:49.123 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:49 +0000 (0:00:00.044) 0:06:04.702 **** 2021-02-10T11:07:49.129 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-02-10T11:07:49.130 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:49.175 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:49 +0000 (0:00:00.052) 0:06:04.754 **** 2021-02-10T11:07:49.203 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-02-10T11:07:49.204 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:49.236 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:49 +0000 (0:00:00.061) 0:06:04.815 **** 2021-02-10T11:07:49.590 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.024413", "end": "2021-02-10 11:07:49.573007", "rc": 0, "start": "2021-02-10 11:07:49.548594", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-10T11:07:49.627 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:49 +0000 (0:00:00.390) 0:06:05.206 **** 2021-02-10T11:07:49.665 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-02-10T11:07:49.709 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:49 +0000 (0:00:00.081) 0:06:05.288 **** 2021-02-10T11:07:49.718 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-02-10T11:07:49.719 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-10T11:07:49.755 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:49 +0000 (0:00:00.045) 0:06:05.334 **** 2021-02-10T11:07:49.761 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-02-10T11:07:49.763 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:49.800 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:49 +0000 (0:00:00.045) 0:06:05.379 **** 2021-02-10T11:07:49.809 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-02-10T11:07:49.809 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'block_used': 13742196, 'uuid': 'aeb0602f-c6fc-4ca1-8920-0340080ffa5a', 'size_total': 984369086464, 'block_total': 240324484, 'mount': '/', 'block_available': 226582288, 'size_available': 928081051648, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,seclabel,relatime,data=ordered', 'device': '/dev/sda1', 'inode_used': 90165, 'block_size': 4096, 'inode_available': 60964811}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226582288, "block_size": 4096, "block_total": 240324484, "block_used": 13742196, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60964811, "inode_total": 61054976, "inode_used": 90165, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 928081051648, "size_total": 984369086464, "uuid": "aeb0602f-c6fc-4ca1-8920-0340080ffa5a"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:49.855 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:49 +0000 (0:00:00.055) 0:06:05.434 **** 2021-02-10T11:07:49.866 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-02-10T11:07:49.867 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:49.904 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:49 +0000 (0:00:00.049) 0:06:05.483 **** 2021-02-10T11:07:49.911 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-02-10T11:07:49.912 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:49.949 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:49 +0000 (0:00:00.044) 0:06:05.528 **** 2021-02-10T11:07:49.957 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-02-10T11:07:49.958 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:49.993 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:49 +0000 (0:00:00.043) 0:06:05.572 **** 2021-02-10T11:07:49.999 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-10T11:07:50.001 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:50.034 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:50 +0000 (0:00:00.040) 0:06:05.612 **** 2021-02-10T11:07:50.040 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-02-10T11:07:50.040 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:50.076 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:50 +0000 (0:00:00.041) 0:06:05.654 **** 2021-02-10T11:07:50.082 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-10T11:07:50.083 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:50.119 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:50 +0000 (0:00:00.042) 0:06:05.697 **** 2021-02-10T11:07:50.156 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-02-10T11:07:50.158 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:50.190 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:50 +0000 (0:00:00.071) 0:06:05.769 **** 2021-02-10T11:07:50.229 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-02-10T11:07:50.230 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:50.276 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:50 +0000 (0:00:00.085) 0:06:05.855 **** 2021-02-10T11:07:50.316 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-02-10T11:07:50.317 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:50.357 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:50 +0000 (0:00:00.081) 0:06:05.936 **** 2021-02-10T11:07:50.524 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-02-10T11:07:50.524 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-10T11:07:50.571 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:50 +0000 (0:00:00.213) 0:06:06.149 **** 2021-02-10T11:07:50.621 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-02-10T11:07:50.623 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'block_used': 13742196, 'uuid': 'aeb0602f-c6fc-4ca1-8920-0340080ffa5a', 'size_total': 984369086464, 'block_total': 240324484, 'mount': '/', 'block_available': 226582288, 'size_available': 928081051648, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,seclabel,relatime,data=ordered', 'device': '/dev/sda1', 'inode_used': 90165, 'block_size': 4096, 'inode_available': 60964811}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226582288, "block_size": 4096, "block_total": 240324484, "block_used": 13742196, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60964811, "inode_total": 61054976, "inode_used": 90165, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 928081051648, "size_total": 984369086464, "uuid": "aeb0602f-c6fc-4ca1-8920-0340080ffa5a"}, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:50.658 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:50 +0000 (0:00:00.087) 0:06:06.237 **** 2021-02-10T11:07:50.700 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-02-10T11:07:50.701 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:50.736 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:50 +0000 (0:00:00.077) 0:06:06.314 **** 2021-02-10T11:07:50.762 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-10T11:07:50.764 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:50.796 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:50 +0000 (0:00:00.060) 0:06:06.375 **** 2021-02-10T11:07:50.824 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-10T11:07:50.826 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:50.861 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:50 +0000 (0:00:00.064) 0:06:06.440 **** 2021-02-10T11:07:50.899 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-02-10T11:07:50.901 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:50.934 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:50 +0000 (0:00:00.072) 0:06:06.513 **** 2021-02-10T11:07:50.963 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-02-10T11:07:50.964 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:51.003 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:51 +0000 (0:00:00.068) 0:06:06.581 **** 2021-02-10T11:07:51.040 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-02-10T11:07:51.042 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:51.073 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:51 +0000 (0:00:00.070) 0:06:06.652 **** 2021-02-10T11:07:51.258 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2021-02-10T11:07:51.259 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-02-10T11:07:51.294 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:51 +0000 (0:00:00.220) 0:06:06.872 **** 2021-02-10T11:07:51.674 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-02-10T11:07:51.708 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:51 +0000 (0:00:00.414) 0:06:07.287 **** 2021-02-10T11:07:52.135 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-02-10T11:07:52.137 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-10T11:07:52.262 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-10T11:07:52.670 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-10T11:07:52.800 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-10T11:07:53.191 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-10T11:07:53.369 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-10T11:07:53.749 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-10T11:07:53.886 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-10T11:07:54.306 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-10T11:07:54.342 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:54 +0000 (0:00:02.633) 0:06:09.920 **** 2021-02-10T11:07:54.412 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-10T11:07:54.457 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:54 +0000 (0:00:00.115) 0:06:10.036 **** 2021-02-10T11:07:54.800 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-02-10T11:07:54.802 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-02-10T11:07:54.848 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:54 +0000 (0:00:00.390) 0:06:10.426 **** 2021-02-10T11:07:54.953 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-02-10T11:07:55.005 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:55 +0000 (0:00:00.158) 0:06:10.584 **** 2021-02-10T11:07:55.362 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-02-10T11:07:55.364 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-10T11:07:55.516 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-10T11:07:55.816 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-10T11:07:55.966 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-10T11:07:56.287 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-10T11:07:56.405 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-10T11:07:56.726 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-10T11:07:56.748 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:56.789 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:56 +0000 (0:00:01.784) 0:06:12.368 **** 2021-02-10T11:07:56.818 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-02-10T11:07:56.819 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:56.860 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:56 +0000 (0:00:00.070) 0:06:12.439 **** 2021-02-10T11:07:56.870 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-02-10T11:07:56.870 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-10T11:07:56.872 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:56.909 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:56 +0000 (0:00:00.049) 0:06:12.488 **** 2021-02-10T11:07:56.917 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-02-10T11:07:56.919 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:56.955 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:56 +0000 (0:00:00.045) 0:06:12.533 **** 2021-02-10T11:07:56.963 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-02-10T11:07:56.965 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:56.999 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:56 +0000 (0:00:00.043) 0:06:12.577 **** 2021-02-10T11:07:57.027 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-02-10T11:07:57.028 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:57.073 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:57 +0000 (0:00:00.074) 0:06:12.652 **** 2021-02-10T11:07:57.412 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-02-10T11:07:57.447 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:57 +0000 (0:00:00.374) 0:06:13.026 **** 2021-02-10T11:07:57.486 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-02-10T11:07:57.487 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:57.525 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:57 +0000 (0:00:00.078) 0:06:13.104 **** 2021-02-10T11:07:57.544 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-02-10T11:07:57.545 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-02-10T11:07:57.586 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:57 +0000 (0:00:00.061) 0:06:13.165 **** 2021-02-10T11:07:57.615 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:57.654 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:57 +0000 (0:00:00.067) 0:06:13.233 **** 2021-02-10T11:07:58.330 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [gibba008.front.sepia.ceph.com] => {"changed": true} changed: [gibba001.front.sepia.ceph.com] => {"changed": true} 2021-02-10T11:07:58.365 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:58 +0000 (0:00:00.711) 0:06:13.944 **** 2021-02-10T11:07:58.402 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:58 +0000 (0:00:00.036) 0:06:13.980 **** 2021-02-10T11:07:59.179 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2021-02-10T11:07:59.181 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-02-10T11:07:59.219 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:59 +0000 (0:00:00.816) 0:06:14.797 **** 2021-02-10T11:07:59.266 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:59 +0000 (0:00:00.047) 0:06:14.845 **** 2021-02-10T11:07:59.276 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** 2021-02-10T11:07:59.276 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:59.322 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:59 +0000 (0:00:00.056) 0:06:14.901 **** 2021-02-10T11:07:59.332 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:59.370 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:59 +0000 (0:00:00.047) 0:06:14.949 **** 2021-02-10T11:07:59.377 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2021-02-10T11:07:59.378 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:59.413 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:59 +0000 (0:00:00.042) 0:06:14.992 **** 2021-02-10T11:07:59.420 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:59.456 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:59 +0000 (0:00:00.043) 0:06:15.035 **** 2021-02-10T11:07:59.464 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-02-10T11:07:59.465 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:59.501 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:59 +0000 (0:00:00.044) 0:06:15.079 **** 2021-02-10T11:07:59.508 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:59.545 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:59 +0000 (0:00:00.044) 0:06:15.124 **** 2021-02-10T11:07:59.551 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-02-10T11:07:59.553 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:59.585 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:59 +0000 (0:00:00.040) 0:06:15.164 **** 2021-02-10T11:07:59.592 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:59.627 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:59 +0000 (0:00:00.042) 0:06:15.206 **** 2021-02-10T11:07:59.629 INFO:teuthology.task.ansible.out: 2021-02-10T11:07:59.635 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-02-10T11:07:59.636 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:59.637 INFO:teuthology.task.ansible.out: 2021-02-10T11:07:59.673 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:59 +0000 (0:00:00.045) 0:06:15.252 **** 2021-02-10T11:07:59.674 INFO:teuthology.task.ansible.out: 2021-02-10T11:07:59.681 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:59.717 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:59 +0000 (0:00:00.043) 0:06:15.296 **** 2021-02-10T11:07:59.725 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-02-10T11:07:59.726 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:59.761 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:59 +0000 (0:00:00.043) 0:06:15.340 **** 2021-02-10T11:07:59.769 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:59.808 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:59 +0000 (0:00:00.046) 0:06:15.386 **** 2021-02-10T11:07:59.814 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-02-10T11:07:59.815 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:59.851 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:59 +0000 (0:00:00.042) 0:06:15.429 **** 2021-02-10T11:07:59.858 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:59.897 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:59 +0000 (0:00:00.046) 0:06:15.476 **** 2021-02-10T11:07:59.905 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-02-10T11:07:59.910 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:59.941 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:59 +0000 (0:00:00.044) 0:06:15.520 **** 2021-02-10T11:07:59.948 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:07:59.982 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:07:59 +0000 (0:00:00.041) 0:06:15.561 **** 2021-02-10T11:07:59.989 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2021-02-10T11:07:59.990 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:00.023 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:00 +0000 (0:00:00.040) 0:06:15.602 **** 2021-02-10T11:08:00.030 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:00.065 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:00 +0000 (0:00:00.041) 0:06:15.644 **** 2021-02-10T11:08:00.072 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-02-10T11:08:00.073 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:00.107 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:00 +0000 (0:00:00.041) 0:06:15.686 **** 2021-02-10T11:08:00.115 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:00.151 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:00 +0000 (0:00:00.043) 0:06:15.729 **** 2021-02-10T11:08:00.159 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2021-02-10T11:08:00.160 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:00.200 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:00 +0000 (0:00:00.049) 0:06:15.779 **** 2021-02-10T11:08:00.207 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:00.245 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:00 +0000 (0:00:00.044) 0:06:15.824 **** 2021-02-10T11:08:01.055 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2021-02-10T11:08:01.057 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-29.el7_8.2.x86_64 providing ntp is already installed"]} 2021-02-10T11:08:01.090 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:01 +0000 (0:00:00.844) 0:06:16.669 **** 2021-02-10T11:08:01.097 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-29.el7_8.2.x86_64 providing ntp is already installed"]} 2021-02-10T11:08:01.134 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:01 +0000 (0:00:00.043) 0:06:16.712 **** 2021-02-10T11:08:01.140 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-02-10T11:08:01.142 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:01.178 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:01 +0000 (0:00:00.044) 0:06:16.757 **** 2021-02-10T11:08:01.185 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:01.224 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:01 +0000 (0:00:00.046) 0:06:16.803 **** 2021-02-10T11:08:01.232 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2021-02-10T11:08:01.234 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:01.274 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:01 +0000 (0:00:00.049) 0:06:16.852 **** 2021-02-10T11:08:01.284 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:01.320 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:01 +0000 (0:00:00.046) 0:06:16.899 **** 2021-02-10T11:08:03.213 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-02-10T11:08:03.215 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2021-02-10T11:08:03.250 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:03 +0000 (0:00:01.930) 0:06:18.829 **** 2021-02-10T11:08:03.268 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2021-02-10T11:08:03.302 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:03 +0000 (0:00:00.051) 0:06:18.880 **** 2021-02-10T11:08:03.308 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-02-10T11:08:03.310 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:03.363 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:03 +0000 (0:00:00.061) 0:06:18.941 **** 2021-02-10T11:08:03.372 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:03.410 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:03 +0000 (0:00:00.047) 0:06:18.989 **** 2021-02-10T11:08:03.905 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-02-10T11:08:03.906 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "tmp.mount ntpdate.service systemd-journald.socket system.slice basic.target syslog.target sntp.service -.mount", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "chronyd.service multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "Limi 2021-02-10T11:08:03.907 INFO:teuthology.task.ansible.out:tAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127560", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127560", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "system.slice basic.target -.mount", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "Timeou 2021-02-10T11:08:03.909 INFO:teuthology.task.ansible.out:tStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-02-10T11:08:03.953 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:03 +0000 (0:00:00.543) 0:06:19.532 **** 2021-02-10T11:08:03.955 INFO:teuthology.task.ansible.out: 2021-02-10T11:08:03.965 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "sntp.service systemd-journald.socket basic.target ntpdate.service tmp.mount -.mount system.slice syslog.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "chronyd.service shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "Limi 2021-02-10T11:08:03.966 INFO:teuthology.task.ansible.out:tAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127560", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127560", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "system.slice -.mount basic.target", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "Timeou 2021-02-10T11:08:03.968 INFO:teuthology.task.ansible.out:tStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-02-10T11:08:04.004 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:04 +0000 (0:00:00.050) 0:06:19.583 **** 2021-02-10T11:08:04.013 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-10T11:08:04.014 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-02-10T11:08:04.064 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:04 +0000 (0:00:00.060) 0:06:19.643 **** 2021-02-10T11:08:04.072 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-02-10T11:08:04.111 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:04 +0000 (0:00:00.046) 0:06:19.689 **** 2021-02-10T11:08:04.117 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-10T11:08:04.118 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:04.154 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:04 +0000 (0:00:00.043) 0:06:19.733 **** 2021-02-10T11:08:04.161 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:04.199 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:04 +0000 (0:00:00.044) 0:06:19.777 **** 2021-02-10T11:08:05.311 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-02-10T11:08:05.313 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-14.el7.noarch providing python-pip is already installed"]} 2021-02-10T11:08:05.367 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:05 +0000 (0:00:01.168) 0:06:20.946 **** 2021-02-10T11:08:05.375 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-14.el7.noarch providing python-pip is already installed"]} 2021-02-10T11:08:05.413 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:05 +0000 (0:00:00.045) 0:06:20.992 **** 2021-02-10T11:08:05.420 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-02-10T11:08:05.421 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:05.457 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:05 +0000 (0:00:00.043) 0:06:21.036 **** 2021-02-10T11:08:05.464 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:05.502 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:05 +0000 (0:00:00.044) 0:06:21.081 **** 2021-02-10T11:08:05.509 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-02-10T11:08:05.510 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:05.546 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:05 +0000 (0:00:00.044) 0:06:21.125 **** 2021-02-10T11:08:05.555 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:05.593 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:05 +0000 (0:00:00.046) 0:06:21.171 **** 2021-02-10T11:08:06.010 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-02-10T11:08:06.013 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2021-02-10T11:08:06.053 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:06 +0000 (0:00:00.460) 0:06:21.631 **** 2021-02-10T11:08:06.065 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2021-02-10T11:08:06.104 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:06 +0000 (0:00:00.051) 0:06:21.683 **** 2021-02-10T11:08:07.980 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-02-10T11:08:07.981 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2021-02-10T11:08:08.017 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:08 +0000 (0:00:01.912) 0:06:23.596 **** 2021-02-10T11:08:08.047 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2021-02-10T11:08:08.088 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:08 +0000 (0:00:00.070) 0:06:23.666 **** 2021-02-10T11:08:09.251 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2021-02-10T11:08:09.253 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 21.0.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 21.0.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-02-10T11:08:09.286 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:09 +0000 (0:00:01.198) 0:06:24.865 **** 2021-02-10T11:08:09.313 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-02-10T11:08:09.314 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:09.347 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:09 +0000 (0:00:00.060) 0:06:24.926 **** 2021-02-10T11:08:09.374 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-02-10T11:08:09.376 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp1s0f0"}, "changed": false} 2021-02-10T11:08:09.424 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:09 +0000 (0:00:00.077) 0:06:25.003 **** 2021-02-10T11:08:09.507 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2021-02-10T11:08:09.508 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 21.0.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 21.0.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-02-10T11:08:09.542 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:09 +0000 (0:00:00.117) 0:06:25.121 **** 2021-02-10T11:08:09.569 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-02-10T11:08:09.571 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:09.609 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:09 +0000 (0:00:00.066) 0:06:25.188 **** 2021-02-10T11:08:09.638 INFO:teuthology.task.ansible.out: 2021-02-10T11:08:09.640 INFO:teuthology.task.ansible.out:TASK [testnode : Set interface] ************************************************ ok: [gibba001.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp1s0f0"}, "changed": false} 2021-02-10T11:08:09.683 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:09 +0000 (0:00:00.073) 0:06:25.261 **** 2021-02-10T11:08:09.875 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-02-10T11:08:09.876 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-02-10T11:08:09.918 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:09 +0000 (0:00:00.234) 0:06:25.496 **** 2021-02-10T11:08:09.946 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2021-02-10T11:08:09.948 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:09.988 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:09 +0000 (0:00:00.070) 0:06:25.567 **** 2021-02-10T11:08:10.017 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-02-10T11:08:10.018 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-02-10T11:08:10.053 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:10 +0000 (0:00:00.065) 0:06:25.632 **** 2021-02-10T11:08:10.082 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-02-10T11:08:10.084 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:10.115 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:10 +0000 (0:00:00.061) 0:06:25.694 **** 2021-02-10T11:08:10.132 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** ok: [gibba001.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-02-10T11:08:10.167 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:10 +0000 (0:00:00.051) 0:06:25.745 **** 2021-02-10T11:08:10.195 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2021-02-10T11:08:10.196 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:10.229 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:10 +0000 (0:00:00.061) 0:06:25.807 **** 2021-02-10T11:08:10.267 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-02-10T11:08:10.269 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-02-10T11:08:10.311 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:10 +0000 (0:00:00.082) 0:06:25.890 **** 2021-02-10T11:08:10.340 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-02-10T11:08:10.342 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:10.381 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:10 +0000 (0:00:00.069) 0:06:25.959 **** 2021-02-10T11:08:10.569 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2021-02-10T11:08:10.571 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-02-10T11:08:10.605 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:10 +0000 (0:00:00.224) 0:06:26.184 **** 2021-02-10T11:08:10.847 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-02-10T11:08:10.881 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:10 +0000 (0:00:00.275) 0:06:26.460 **** 2021-02-10T11:08:11.048 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-02-10T11:08:11.050 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-02-10T11:08:11.088 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:11 +0000 (0:00:00.205) 0:06:26.666 **** 2021-02-10T11:08:11.114 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-02-10T11:08:11.116 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.152 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:11 +0000 (0:00:00.064) 0:06:26.730 **** 2021-02-10T11:08:11.214 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-02-10T11:08:11.216 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [gibba008.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [gibba008.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.226 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item=enp1s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.228 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item=enp1s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.265 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:11 +0000 (0:00:00.113) 0:06:26.844 **** 2021-02-10T11:08:11.321 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-02-10T11:08:11.323 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba008.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.334 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* changed: [gibba001.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-02-10T11:08:11.370 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:11 +0000 (0:00:00.104) 0:06:26.949 **** 2021-02-10T11:08:11.379 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-02-10T11:08:11.381 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba008.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.383 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.427 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:11 +0000 (0:00:00.057) 0:06:27.006 **** 2021-02-10T11:08:11.435 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-02-10T11:08:11.436 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.472 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:11 +0000 (0:00:00.044) 0:06:27.051 **** 2021-02-10T11:08:11.479 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-10T11:08:11.481 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.519 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:11 +0000 (0:00:00.046) 0:06:27.097 **** 2021-02-10T11:08:11.534 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [gibba001.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.540 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.541 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item=enp1s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False"} skipping: [gibba001.front.sepia.ceph.com] => (item=enp1s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.577 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:11 +0000 (0:00:00.058) 0:06:27.156 **** 2021-02-10T11:08:11.586 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-02-10T11:08:11.588 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.632 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:11 +0000 (0:00:00.051) 0:06:27.208 **** 2021-02-10T11:08:11.637 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-02-10T11:08:11.638 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.639 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.640 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.662 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.663 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.664 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-02-10T11:08:11.666 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.699 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:11 +0000 (0:00:00.069) 0:06:27.278 **** 2021-02-10T11:08:11.733 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:11 +0000 (0:00:00.034) 0:06:27.312 **** 2021-02-10T11:08:11.740 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-10T11:08:11.741 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.781 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:11 +0000 (0:00:00.047) 0:06:27.359 **** 2021-02-10T11:08:11.789 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-02-10T11:08:11.790 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.825 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:11 +0000 (0:00:00.045) 0:06:27.404 **** 2021-02-10T11:08:11.832 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-02-10T11:08:11.833 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.870 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:11 +0000 (0:00:00.044) 0:06:27.448 **** 2021-02-10T11:08:11.877 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-02-10T11:08:11.879 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.917 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:11 +0000 (0:00:00.047) 0:06:27.496 **** 2021-02-10T11:08:11.923 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-02-10T11:08:11.925 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:11.964 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:11 +0000 (0:00:00.047) 0:06:27.543 **** 2021-02-10T11:08:11.972 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-02-10T11:08:11.973 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:12.014 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:12 +0000 (0:00:00.050) 0:06:27.593 **** 2021-02-10T11:08:12.021 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-02-10T11:08:12.023 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:12.065 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:12 +0000 (0:00:00.050) 0:06:27.643 **** 2021-02-10T11:08:12.072 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2021-02-10T11:08:12.073 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:12.112 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:12 +0000 (0:00:00.047) 0:06:27.691 **** 2021-02-10T11:08:12.120 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:12.159 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:12 +0000 (0:00:00.047) 0:06:27.738 **** 2021-02-10T11:08:12.166 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-02-10T11:08:12.168 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:12.203 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:12 +0000 (0:00:00.043) 0:06:27.782 **** 2021-02-10T11:08:12.209 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-02-10T11:08:12.211 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:12.253 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:12 +0000 (0:00:00.049) 0:06:27.831 **** 2021-02-10T11:08:12.282 INFO:teuthology.task.ansible.out: 2021-02-10T11:08:12.284 INFO:teuthology.task.ansible.out:TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:12.331 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:12 +0000 (0:00:00.078) 0:06:27.910 **** 2021-02-10T11:08:12.546 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Wednesday 10 February 2021 11:08:12 +0000 (0:00:00.074) 0:06:27.984 **** 2021-02-10T11:08:12.680 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-02-10T11:08:12.681 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.004829", "end": "2021-02-10 11:08:12.650150", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-02-10 11:08:12.645321", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-10T11:08:12.869 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.004944", "end": "2021-02-10 11:08:12.838934", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-02-10 11:08:12.833990", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-10T11:08:13.110 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.002297", "end": "2021-02-10 11:08:13.084511", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-02-10 11:08:13.082214", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-10T11:08:13.298 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.002296", "end": "2021-02-10 11:08:13.271682", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-02-10 11:08:13.269386", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-10T11:08:13.541 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.002168", "end": "2021-02-10 11:08:13.513421", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-02-10 11:08:13.511253", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2021-02-10T11:08:13.729 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.002198", "end": "2021-02-10 11:08:13.701329", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-02-10 11:08:13.699131", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2021-02-10T11:08:13.991 INFO:teuthology.task.ansible.out:ok: [gibba008.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.018447", "end": "2021-02-10 11:08:13.959289", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-02-10 11:08:13.940842", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-10T11:08:14.033 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:14 +0000 (0:00:01.627) 0:06:29.612 **** 2021-02-10T11:08:14.063 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-02-10T11:08:14.098 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:14 +0000 (0:00:00.064) 0:06:29.677 **** 2021-02-10T11:08:14.165 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-02-10T11:08:14.166 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.002553", "end": "2021-02-10 11:08:14.133894", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-02-10 11:08:14.131341", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-10T11:08:14.219 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:14 +0000 (0:00:00.120) 0:06:29.798 **** 2021-02-10T11:08:14.258 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-02-10T11:08:14.295 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:14 +0000 (0:00:00.075) 0:06:29.874 **** 2021-02-10T11:08:14.296 INFO:teuthology.task.ansible.out: 2021-02-10T11:08:14.549 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-02-10T11:08:14.551 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2021-02-10T11:08:14.748 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2021-02-10T11:08:14.849 INFO:teuthology.task.ansible.out: PLAY [testnodes,senta,vossi,jenkins_slaves] ************************************ 2021-02-10T11:08:14.881 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:14 +0000 (0:00:00.585) 0:06:30.460 **** 2021-02-10T11:08:14.914 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:14 +0000 (0:00:00.033) 0:06:30.493 **** 2021-02-10T11:08:15.993 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [gibba001.front.sepia.ceph.com] 2021-02-10T11:08:16.072 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:16 +0000 (0:00:01.156) 0:06:31.650 **** 2021-02-10T11:08:16.099 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-10T11:08:16.100 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:16.136 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:16 +0000 (0:00:00.063) 0:06:31.714 **** 2021-02-10T11:08:16.176 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-10T11:08:16.177 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "yum"}, "changed": false} 2021-02-10T11:08:16.241 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:16 +0000 (0:00:00.105) 0:06:31.820 **** 2021-02-10T11:08:16.294 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2021-02-10T11:08:16.295 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/yum_systems.yml) => {"ansible_facts": {"container_packages": ["podman", "podman-docker"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/yum_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/yum_systems.yml"} 2021-02-10T11:08:16.332 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:16 +0000 (0:00:00.091) 0:06:31.911 **** 2021-02-10T11:08:17.049 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [gibba008.front.sepia.ceph.com] 2021-02-10T11:08:17.132 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:17 +0000 (0:00:00.799) 0:06:32.710 **** 2021-02-10T11:08:17.161 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-10T11:08:17.162 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:17.215 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:17 +0000 (0:00:00.083) 0:06:32.794 **** 2021-02-10T11:08:17.254 INFO:teuthology.task.ansible.out: 2021-02-10T11:08:17.256 INFO:teuthology.task.ansible.out:TASK [container-host : set_fact] *********************************************** ok: [gibba008.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "yum"}, "changed": false} 2021-02-10T11:08:17.294 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:17 +0000 (0:00:00.078) 0:06:32.873 **** 2021-02-10T11:08:17.343 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** ok: [gibba008.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/yum_systems.yml) => {"ansible_facts": {"container_packages": ["podman", "podman-docker"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/yum_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/yum_systems.yml"} 2021-02-10T11:08:17.395 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:17 +0000 (0:00:00.101) 0:06:32.974 **** 2021-02-10T11:08:27.045 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2021-02-10T11:08:27.045 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["podman", "1.6.4-27.el7_9.x86_64 from rhel-7-server-extras-rpms"], ["podman-docker", "1.6.4-27.el7_9.noarch from rhel-7-server-extras-rpms"]]}, "msg": "", "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities, product-id, search-\n : disabled-repos, subscription-manager\nLoading mirror speeds from cached hostfile\n * epel: download-cc-rdu01.fedoraproject.org\nResolving Dependencies\n--> Running transaction check\n---> Package podman.x86_64 0:1.6.4-26.el7_9 will be updated\n---> Package podman.x86_64 0:1.6.4-27.el7_9 will be an update\n---> Package podman-docker.noarch 0:1.6.4-26.el7_9 will be updated\n---> Package podman-docker.noarch 0:1.6.4-27.el7_9 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n podman x86_64 1.6.4-27.el7_9 rhel-7-server-extras-rpms 13 M\n podman-docker noarch 1.6.4-27.el7_9 rhel-7-server-extras-rpms 31 k\n\nTransaction Summary\n================================================================================\nUpgrade 2 Packages\n\nTotal download size: 13 M\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n--------------------------------------------------------------------------------\nTotal 71 MB/s | 13 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : podman-1.6.4-27.el7_9.x86_64 1/4 \n Updating : podman-docker-1.6.4-27.el7_9.noarch 2/4 \n Cleanup : podman-docke 2021-02-10T11:08:27.046 INFO:teuthology.task.ansible.out:r-1.6.4-26.el7_9.noarch 3/4 \n Cleanup : podman-1.6.4-26.el7_9.x86_64 4/4 \nLoaded plugins: fastestmirror, langpacks, priorities, product-id, subscription-\n : manager\n Verifying : podman-docker-1.6.4-27.el7_9.noarch 1/4 \n Verifying : podman-1.6.4-27.el7_9.x86_64 2/4 \n Verifying : podman-docker-1.6.4-26.el7_9.noarch 3/4 \n Verifying : podman-1.6.4-26.el7_9.x86_64 4/4 \n\nUpdated:\n podman.x86_64 0:1.6.4-27.el7_9 podman-docker.noarch 0:1.6.4-27.el7_9 \n\nComplete!\n"]} 2021-02-10T11:08:27.083 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:27 +0000 (0:00:09.688) 0:06:42.662 **** 2021-02-10T11:08:27.121 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-10T11:08:27.123 INFO:teuthology.task.ansible.out:ok: [gibba001.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/containers/registries.conf"}, "changed": false} 2021-02-10T11:08:27.157 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:27 +0000 (0:00:00.073) 0:06:42.736 **** 2021-02-10T11:08:27.184 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-10T11:08:27.186 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:27.220 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:27 +0000 (0:00:00.062) 0:06:42.798 **** 2021-02-10T11:08:27.698 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-02-10T11:08:27.700 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "secontext": "unconfined_u:object_r:cert_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-02-10T11:08:27.971 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2021-02-10T11:08:27.972 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["podman", "1.6.4-27.el7_9.x86_64 from rhel-7-server-extras-rpms"], ["podman-docker", "1.6.4-27.el7_9.noarch from rhel-7-server-extras-rpms"]]}, "msg": "", "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities, product-id, search-\n : disabled-repos, subscription-manager\nLoading mirror speeds from cached hostfile\n * epel: download-cc-rdu01.fedoraproject.org\nResolving Dependencies\n--> Running transaction check\n---> Package podman.x86_64 0:1.6.4-26.el7_9 will be updated\n---> Package podman.x86_64 0:1.6.4-27.el7_9 will be an update\n---> Package podman-docker.noarch 0:1.6.4-26.el7_9 will be updated\n---> Package podman-docker.noarch 0:1.6.4-27.el7_9 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n podman x86_64 1.6.4-27.el7_9 rhel-7-server-extras-rpms 13 M\n podman-docker noarch 1.6.4-27.el7_9 rhel-7-server-extras-rpms 31 k\n\nTransaction Summary\n================================================================================\nUpgrade 2 Packages\n\nTotal download size: 13 M\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n--------------------------------------------------------------------------------\nTotal 81 MB/s | 13 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : podman-1.6.4-27.el7_9.x86_64 1/4 \n Updating : podman-docker-1.6.4-27.el7_9.noarch 2/4 \n Cleanup : podman-docke 2021-02-10T11:08:27.974 INFO:teuthology.task.ansible.out:r-1.6.4-26.el7_9.noarch 3/4 \n Cleanup : podman-1.6.4-26.el7_9.x86_64 4/4 \nLoaded plugins: fastestmirror, langpacks, priorities, product-id, subscription-\n : manager\n Verifying : podman-docker-1.6.4-27.el7_9.noarch 1/4 \n Verifying : podman-1.6.4-27.el7_9.x86_64 2/4 \n Verifying : podman-docker-1.6.4-26.el7_9.noarch 3/4 \n Verifying : podman-1.6.4-26.el7_9.x86_64 4/4 \n\nUpdated:\n podman.x86_64 0:1.6.4-27.el7_9 podman-docker.noarch 0:1.6.4-27.el7_9 \n\nComplete!\n"]} 2021-02-10T11:08:28.008 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:28 +0000 (0:00:00.788) 0:06:43.587 **** 2021-02-10T11:08:28.047 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** ok: [gibba008.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/containers/registries.conf"}, "changed": false} 2021-02-10T11:08:28.086 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:28 +0000 (0:00:00.077) 0:06:43.665 **** 2021-02-10T11:08:28.112 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-10T11:08:28.114 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:28.147 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:28 +0000 (0:00:00.061) 0:06:43.726 **** 2021-02-10T11:08:28.155 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-02-10T11:08:28.155 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-02-10T11:08:28.199 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:28 +0000 (0:00:00.051) 0:06:43.778 **** 2021-02-10T11:08:28.626 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "secontext": "unconfined_u:object_r:cert_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-02-10T11:08:29.069 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-02-10T11:08:29.117 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:29 +0000 (0:00:00.918) 0:06:44.696 **** 2021-02-10T11:08:30.815 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2021-02-10T11:08:30.816 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:cert_t:s0", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1612955308.2398782-64273827327700/source", "state": "file", "uid": 0} 2021-02-10T11:08:31.747 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:cert_t:s0", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1612955309.1576362-105907173514931/source", "state": "file", "uid": 0} 2021-02-10T11:08:33.398 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1612955310.8291578-134492322014504/source", "state": "file", "uid": 0} 2021-02-10T11:08:33.435 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:33 +0000 (0:00:04.317) 0:06:49.013 **** 2021-02-10T11:08:34.326 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1612955311.7528372-167939007149970/source", "state": "file", "uid": 0} 2021-02-10T11:08:34.365 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:34 +0000 (0:00:00.930) 0:06:49.944 **** 2021-02-10T11:08:35.980 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-02-10T11:08:36.066 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "cmd": ["/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "You are using pip version 8.1.2, however version 21.0.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 21.0.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-4LNZ05-build\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nInstalling collected packages: toml, docopt, typing, registries-conf-ctl\n Running setup.py install for docopt: started\n Running setup.py install for docopt: finished with status 'done'\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-4LNZ05-build", "Co 2021-02-10T11:08:36.067 INFO:teuthology.task.ansible.out:llecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Installing collected packages: toml, docopt, typing, registries-conf-ctl", " Running setup.py install for docopt: started", " Running setup.py install for docopt: finished with status 'done'", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} Wednesday 10 February 2021 11:08:36 +0000 (0:00:01.652) 0:06:51.596 **** TASK [container-host : Check for docker's daemon.json] ************************* skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:36.095 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:36 +0000 (0:00:00.076) 0:06:51.673 **** 2021-02-10T11:08:36.132 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/containers/registries.conf if necessary] **** 2021-02-10T11:08:36.133 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:36.170 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:36 +0000 (0:00:00.075) 0:06:51.749 **** 2021-02-10T11:08:36.702 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2021-02-10T11:08:36.703 INFO:teuthology.task.ansible.out:changed: [gibba001.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.084784", "end": "2021-02-10 11:08:36.687838", "rc": 0, "start": "2021-02-10 11:08:36.603054", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-10T11:08:36.739 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:36 +0000 (0:00:00.568) 0:06:52.318 **** 2021-02-10T11:08:36.781 INFO:teuthology.task.ansible.out: TASK [container-host : Restart docker service] ********************************* 2021-02-10T11:08:36.782 INFO:teuthology.task.ansible.out:skipping: [gibba001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:36.923 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-02-10T11:08:36.925 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "cmd": ["/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "You are using pip version 8.1.2, however version 21.0.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 21.0.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-d1zD9x-build\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nInstalling collected packages: toml, docopt, typing, registries-conf-ctl\n Running setup.py install for docopt: started\n Running setup.py install for docopt: finished with status 'done'\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-d1zD9x-build", "Co 2021-02-10T11:08:36.926 INFO:teuthology.task.ansible.out:llecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Installing collected packages: toml, docopt, typing, registries-conf-ctl", " Running setup.py install for docopt: started", " Running setup.py install for docopt: finished with status 'done'", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} 2021-02-10T11:08:36.960 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:36 +0000 (0:00:00.221) 0:06:52.539 **** 2021-02-10T11:08:36.987 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-02-10T11:08:36.988 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:37.024 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:37 +0000 (0:00:00.063) 0:06:52.603 **** 2021-02-10T11:08:37.052 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/containers/registries.conf if necessary] **** 2021-02-10T11:08:37.053 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:37.090 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:37 +0000 (0:00:00.066) 0:06:52.669 **** 2021-02-10T11:08:37.605 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2021-02-10T11:08:37.607 INFO:teuthology.task.ansible.out:changed: [gibba008.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.058206", "end": "2021-02-10 11:08:37.585057", "rc": 0, "start": "2021-02-10 11:08:37.526851", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-10T11:08:37.642 INFO:teuthology.task.ansible.out:Wednesday 10 February 2021 11:08:37 +0000 (0:00:00.551) 0:06:53.220 **** 2021-02-10T11:08:37.689 INFO:teuthology.task.ansible.out: TASK [container-host : Restart docker service] ********************************* 2021-02-10T11:08:37.691 INFO:teuthology.task.ansible.out:skipping: [gibba008.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-10T11:08:37.796 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2021-02-10T11:08:37.800 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2021-02-10T11:08:37.803 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched 2021-02-10T11:08:37.805 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* gibba001.front.sepia.ceph.com : ok=141 changed=45 unreachable=0 failed=0 skipped=169 rescued=0 ignored=4 gibba008.front.sepia.ceph.com : ok=141 changed=45 unreachable=0 failed=0 skipped=169 rescued=0 ignored=4 Wednesday 10 February 2021 11:08:37 +0000 (0:00:00.162) 0:06:53.382 **** =============================================================================== users : Create all admin users with sudo access. ----------------------- 62.55s users : Update authorized_keys using the keys repo --------------------- 60.90s common : Make sure smartmontools is installed -------------------------- 23.98s users : Remove revoked users ------------------------------------------- 22.07s testnode : Install packages -------------------------------------------- 14.59s container-host : Install container packages ----------------------------- 9.69s testnode : Zap all non-root disks --------------------------------------- 8.77s testnode : Ensure ceph packages are not present. ------------------------ 7.52s common : Register with subscription-manager. ---------------------------- 5.05s container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert --- 4.32s testnode : Install epel packages ---------------------------------------- 3.53s common : nrpe - Load SELinux policy package ----------------------------- 3.49s common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/. --- 3.43s common : nrpe - Remove SELinux policy package --------------------------- 3.42s users : Write /keys-repo-sha1 sentinel file ----------------------------- 2.87s Gathering Facts --------------------------------------------------------- 2.85s testnode : Ensure ceph dependency packages are not present. ------------- 2.82s testnode : Create logical volume(s) ------------------------------------- 2.63s users : Update autho 2021-02-10T11:08:37.806 INFO:teuthology.task.ansible.out:rized_keys for each user with literal keys ---------- 2.47s testnode : Blow away lingering OSD data and FSIDs ----------------------- 2.40s 2021-02-10T11:08:38.141 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@gibba001.front.sepia.ceph.com'), Remote(name='ubuntu@gibba008.front.sepia.ceph.com')] 2021-02-10T11:08:38.142 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-10T11:08:38.143 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-10T11:08:38.210 DEBUG:teuthology.orchestra.run.gibba001:> true 2021-02-10T11:08:38.706 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-10T11:08:38.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-10T11:08:38.823 DEBUG:teuthology.orchestra.run.gibba008:> true 2021-02-10T11:08:39.325 INFO:teuthology.run_tasks:Running task clock... 2021-02-10T11:08:39.340 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2021-02-10T11:08:39.340 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-02-10T11:08:39.341 DEBUG:teuthology.orchestra.run.gibba001:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-02-10T11:08:39.694 INFO:teuthology.orchestra.run.gibba001.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2021-02-10T11:08:46.733 INFO:teuthology.orchestra.run.gibba001.stdout:ntpd: time slew +0.000640s 2021-02-10T11:08:46.744 INFO:teuthology.orchestra.run.gibba001.stderr:Failed to start ntp.service: Unit not found. 2021-02-10T11:08:46.783 INFO:teuthology.orchestra.run.gibba001.stdout: remote refid st t when poll reach delay offset jitter 2021-02-10T11:08:46.783 INFO:teuthology.orchestra.run.gibba001.stdout:============================================================================== 2021-02-10T11:08:46.783 INFO:teuthology.orchestra.run.gibba001.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-10T11:08:46.783 INFO:teuthology.orchestra.run.gibba001.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-10T11:08:46.783 INFO:teuthology.orchestra.run.gibba001.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-10T11:08:46.783 INFO:teuthology.orchestra.run.gibba001.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-10T11:08:46.783 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-02-10T11:08:46.784 DEBUG:teuthology.orchestra.run.gibba008:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-02-10T11:08:47.136 INFO:teuthology.orchestra.run.gibba008.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2021-02-10T11:08:54.174 INFO:teuthology.orchestra.run.gibba008.stdout:ntpd: time slew -0.000393s 2021-02-10T11:08:54.187 INFO:teuthology.orchestra.run.gibba008.stderr:Failed to start ntp.service: Unit not found. 2021-02-10T11:08:54.225 INFO:teuthology.orchestra.run.gibba008.stdout: remote refid st t when poll reach delay offset jitter 2021-02-10T11:08:54.225 INFO:teuthology.orchestra.run.gibba008.stdout:============================================================================== 2021-02-10T11:08:54.225 INFO:teuthology.orchestra.run.gibba008.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-10T11:08:54.226 INFO:teuthology.orchestra.run.gibba008.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-10T11:08:54.226 INFO:teuthology.orchestra.run.gibba008.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-10T11:08:54.226 INFO:teuthology.orchestra.run.gibba008.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-10T11:08:54.226 INFO:teuthology.run_tasks:Running task install... 2021-02-10T11:08:54.237 DEBUG:teuthology.task.install:project ceph 2021-02-10T11:08:54.238 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': '962fac13b1a781b8c7d25c1eed50d93813b8f5f0'}} 2021-02-10T11:08:54.238 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel']}, 'sha1': '962fac13b1a781b8c7d25c1eed50d93813b8f5f0'} 2021-02-10T11:08:54.238 INFO:teuthology.task.install:Using flavor: basic 2021-02-10T11:08:54.248 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python-ceph', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-cloud', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-ssh', 'ceph-fuse', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python-ceph', 'rbd-fuse']} 2021-02-10T11:08:54.248 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']} 2021-02-10T11:08:54.248 DEBUG:teuthology.orchestra.run.gibba001:> sudo lsb_release -is 2021-02-10T11:08:54.605 INFO:teuthology.orchestra.run.gibba001.stdout:RedHatEnterpriseServer 2021-02-10T11:08:54.606 DEBUG:teuthology.misc:System to be installed: RedHatEnterpriseServer 2021-02-10T11:08:54.606 DEBUG:teuthology.orchestra.run.gibba008:> sudo lsb_release -is 2021-02-10T11:08:54.607 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel']}, 'extras': None, 'flavor': 'basic', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '962fac13b1a781b8c7d25c1eed50d93813b8f5f0', 'tag': None, 'wait_for_package': False} 2021-02-10T11:08:54.607 DEBUG:teuthology.orchestra.run.gibba001:> sudo lsb_release -is 2021-02-10T11:08:54.961 INFO:teuthology.orchestra.run.gibba001.stdout:RedHatEnterpriseServer 2021-02-10T11:08:54.962 DEBUG:teuthology.misc:System to be installed: RedHatEnterpriseServer 2021-02-10T11:08:54.962 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F7%2Fx86_64&sha1=962fac13b1a781b8c7d25c1eed50d93813b8f5f0 2021-02-10T11:08:54.971 INFO:teuthology.orchestra.run.gibba008.stdout:RedHatEnterpriseServer 2021-02-10T11:08:54.971 DEBUG:teuthology.misc:System to be installed: RedHatEnterpriseServer 2021-02-10T11:08:54.971 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel']}, 'extras': None, 'flavor': 'basic', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '962fac13b1a781b8c7d25c1eed50d93813b8f5f0', 'tag': None, 'wait_for_package': False} 2021-02-10T11:08:54.972 DEBUG:teuthology.orchestra.run.gibba008:> sudo lsb_release -is 2021-02-10T11:08:55.152 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-khiremat-mgr-authid-bp-7-nautilus/962fac13b1a781b8c7d25c1eed50d93813b8f5f0/centos/7/flavors/default/ 2021-02-10T11:08:55.152 INFO:teuthology.task.install.rpm:Package version is 14.2.16-280.g962fac13 2021-02-10T11:08:55.294 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-khiremat-mgr-authid-bp-7-nautilus/962fac13b1a781b8c7d25c1eed50d93813b8f5f0/centos/7/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-khiremat-mgr-authid-bp-7-nautilus/962fac13b1a781b8c7d25c1eed50d93813b8f5f0/centos/7/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-khiremat-mgr-authid-bp-7-nautilus/962fac13b1a781b8c7d25c1eed50d93813b8f5f0/centos/7/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2021-02-10T11:08:55.294 DEBUG:teuthology.orchestra.run.gibba001:> set -ex 2021-02-10T11:08:55.294 DEBUG:teuthology.orchestra.run.gibba001:> sudo dd of=/etc/yum.repos.d/ceph.repo 2021-02-10T11:08:55.327 INFO:teuthology.orchestra.run.gibba008.stdout:RedHatEnterpriseServer 2021-02-10T11:08:55.327 DEBUG:teuthology.misc:System to be installed: RedHatEnterpriseServer 2021-02-10T11:08:55.328 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F7%2Fx86_64&sha1=962fac13b1a781b8c7d25c1eed50d93813b8f5f0 2021-02-10T11:08:55.491 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-khiremat-mgr-authid-bp-7-nautilus/962fac13b1a781b8c7d25c1eed50d93813b8f5f0/centos/7/flavors/default/ 2021-02-10T11:08:55.492 INFO:teuthology.task.install.rpm:Package version is 14.2.16-280.g962fac13 2021-02-10T11:08:55.616 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-khiremat-mgr-authid-bp-7-nautilus/962fac13b1a781b8c7d25c1eed50d93813b8f5f0/centos/7/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-khiremat-mgr-authid-bp-7-nautilus/962fac13b1a781b8c7d25c1eed50d93813b8f5f0/centos/7/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-khiremat-mgr-authid-bp-7-nautilus/962fac13b1a781b8c7d25c1eed50d93813b8f5f0/centos/7/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2021-02-10T11:08:55.616 DEBUG:teuthology.orchestra.run.gibba008:> set -ex 2021-02-10T11:08:55.616 DEBUG:teuthology.orchestra.run.gibba008:> sudo dd of=/etc/yum.repos.d/ceph.repo 2021-02-10T11:08:55.638 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-cloud, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-ssh, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, python-ceph, rbd-fuse, python3-cephfs, bison, flex, elfutils-libelf-devel, openssl-devel, bison, flex, elfutils-libelf-devel, openssl-devel, bison, flex, elfutils-libelf-devel, openssl-devel on remote rpm x86_64 2021-02-10T11:08:55.639 DEBUG:teuthology.orchestra.run.gibba001:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/962fac13b1a781b8c7d25c1eed50d93813b8f5f0/;g' /etc/yum.repos.d/ceph.repo ; fi 2021-02-10T11:08:55.961 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-cloud, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-ssh, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, python-ceph, rbd-fuse, python3-cephfs, bison, flex, elfutils-libelf-devel, openssl-devel, bison, flex, elfutils-libelf-devel, openssl-devel, bison, flex, elfutils-libelf-devel, openssl-devel, bison, flex, elfutils-libelf-devel, openssl-devel on remote rpm x86_64 2021-02-10T11:08:55.961 DEBUG:teuthology.orchestra.run.gibba008:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/962fac13b1a781b8c7d25c1eed50d93813b8f5f0/;g' /etc/yum.repos.d/ceph.repo ; fi 2021-02-10T11:08:55.982 DEBUG:teuthology.orchestra.run.gibba001:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2021-02-10T11:08:56.303 DEBUG:teuthology.orchestra.run.gibba008:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2021-02-10T11:08:56.334 DEBUG:teuthology.orchestra.run.gibba001:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2021-02-10T11:08:56.654 DEBUG:teuthology.orchestra.run.gibba008:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2021-02-10T11:08:56.676 INFO:teuthology.orchestra.run.gibba001.stdout:check_obsoletes = 1 2021-02-10T11:08:56.677 DEBUG:teuthology.orchestra.run.gibba001:> sudo yum clean all 2021-02-10T11:08:56.995 INFO:teuthology.orchestra.run.gibba008.stdout:check_obsoletes = 1 2021-02-10T11:08:56.996 DEBUG:teuthology.orchestra.run.gibba008:> sudo yum clean all 2021-02-10T11:08:57.125 INFO:teuthology.orchestra.run.gibba001.stdout:Loaded plugins: fastestmirror, langpacks, priorities, product-id, search- 2021-02-10T11:08:57.125 INFO:teuthology.orchestra.run.gibba001.stdout: : disabled-repos, subscription-manager 2021-02-10T11:08:57.433 INFO:teuthology.orchestra.run.gibba001.stdout:Cleaning repos: ceph ceph-noarch ceph-source epel lab-extras rhel-7-fcgi-ceph 2021-02-10T11:08:57.433 INFO:teuthology.orchestra.run.gibba001.stdout: : rhel-7-server-extras-rpms rhel-7-server-optional-rpms 2021-02-10T11:08:57.434 INFO:teuthology.orchestra.run.gibba001.stdout: : rhel-7-server-rpms rhel-ha-for-rhel-7-server-rpms 2021-02-10T11:08:57.444 INFO:teuthology.orchestra.run.gibba008.stdout:Loaded plugins: fastestmirror, langpacks, priorities, product-id, search- 2021-02-10T11:08:57.444 INFO:teuthology.orchestra.run.gibba008.stdout: : disabled-repos, subscription-manager 2021-02-10T11:08:57.600 INFO:teuthology.orchestra.run.gibba001.stdout:Cleaning up list of fastest mirrors 2021-02-10T11:08:57.613 DEBUG:teuthology.orchestra.run.gibba001:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-cloud ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-ssh ceph-fuse libcephfs2 libcephfs-devel librados2 librbd1 python-ceph rbd-fuse python3-cephfs bison flex elfutils-libelf-devel openssl-devel bison flex elfutils-libelf-devel openssl-devel bison flex elfutils-libelf-devel openssl-devel bison flex elfutils-libelf-devel openssl-devel 2021-02-10T11:08:57.717 INFO:teuthology.orchestra.run.gibba008.stdout:Cleaning repos: ceph ceph-noarch ceph-source epel lab-extras rhel-7-fcgi-ceph 2021-02-10T11:08:57.717 INFO:teuthology.orchestra.run.gibba008.stdout: : rhel-7-server-extras-rpms rhel-7-server-optional-rpms 2021-02-10T11:08:57.717 INFO:teuthology.orchestra.run.gibba008.stdout: : rhel-7-server-rpms rhel-ha-for-rhel-7-server-rpms 2021-02-10T11:08:57.872 INFO:teuthology.orchestra.run.gibba008.stdout:Cleaning up list of fastest mirrors 2021-02-10T11:08:57.885 DEBUG:teuthology.orchestra.run.gibba008:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-cloud ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-ssh ceph-fuse libcephfs2 libcephfs-devel librados2 librbd1 python-ceph rbd-fuse python3-cephfs bison flex elfutils-libelf-devel openssl-devel bison flex elfutils-libelf-devel openssl-devel bison flex elfutils-libelf-devel openssl-devel bison flex elfutils-libelf-devel openssl-devel 2021-02-10T11:08:58.062 INFO:teuthology.orchestra.run.gibba001.stdout:Loaded plugins: fastestmirror, langpacks, priorities, product-id, search- 2021-02-10T11:08:58.062 INFO:teuthology.orchestra.run.gibba001.stdout: : disabled-repos, subscription-manager 2021-02-10T11:08:58.332 INFO:teuthology.orchestra.run.gibba001.stdout:Determining fastest mirrors 2021-02-10T11:08:58.336 INFO:teuthology.orchestra.run.gibba008.stdout:Loaded plugins: fastestmirror, langpacks, priorities, product-id, search- 2021-02-10T11:08:58.336 INFO:teuthology.orchestra.run.gibba008.stdout: : disabled-repos, subscription-manager 2021-02-10T11:08:58.404 INFO:teuthology.orchestra.run.gibba001.stdout: * epel: download-cc-rdu01.fedoraproject.org 2021-02-10T11:08:58.613 INFO:teuthology.orchestra.run.gibba008.stdout:Determining fastest mirrors 2021-02-10T11:08:58.686 INFO:teuthology.orchestra.run.gibba008.stdout: * epel: download-cc-rdu01.fedoraproject.org 2021-02-10T11:09:26.756 INFO:teuthology.orchestra.run.gibba008.stdout:29 packages excluded due to repository priority protections 2021-02-10T11:09:27.963 INFO:teuthology.orchestra.run.gibba001.stdout:29 packages excluded due to repository priority protections 2021-02-10T11:09:43.610 INFO:teuthology.orchestra.run.gibba008.stdout:Resolving Dependencies 2021-02-10T11:09:43.611 INFO:teuthology.orchestra.run.gibba008.stdout:--> Running transaction check 2021-02-10T11:09:43.611 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package bison.x86_64 0:3.0.4-2.el7 will be installed 2021-02-10T11:09:43.616 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package ceph.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.617 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: ceph-mon = 2:14.2.16-280.g962fac13.el7 for package: 2:ceph-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.618 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: ceph-osd = 2:14.2.16-280.g962fac13.el7 for package: 2:ceph-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.619 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: ceph-mds = 2:14.2.16-280.g962fac13.el7 for package: 2:ceph-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.620 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package ceph-fuse.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.627 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package ceph-mgr.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.629 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: ceph-base = 2:14.2.16-280.g962fac13.el7 for package: 2:ceph-mgr-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.631 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-requests for package: 2:ceph-mgr-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.633 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.634 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.635 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.636 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-bcrypt for package: 2:ceph-mgr-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.637 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package ceph-mgr-dashboard.noarch 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.637 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: ceph-grafana-dashboards = 2:14.2.16-280.g962fac13.el7 for package: 2:ceph-mgr-dashboard-14.2.16-280.g962fac13.el7.noarch 2021-02-10T11:09:43.638 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-routes for package: 2:ceph-mgr-dashboard-14.2.16-280.g962fac13.el7.noarch 2021-02-10T11:09:43.638 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package ceph-mgr-diskprediction-cloud.noarch 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.639 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package ceph-mgr-diskprediction-local.noarch 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.639 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: scipy for package: 2:ceph-mgr-diskprediction-local-14.2.16-280.g962fac13.el7.noarch 2021-02-10T11:09:43.641 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package ceph-mgr-rook.noarch 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.641 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-kubernetes for package: 2:ceph-mgr-rook-14.2.16-280.g962fac13.el7.noarch 2021-02-10T11:09:43.641 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package ceph-mgr-ssh.noarch 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.642 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-remoto for package: 2:ceph-mgr-ssh-14.2.16-280.g962fac13.el7.noarch 2021-02-10T11:09:43.642 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package ceph-radosgw.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.646 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: ceph-selinux = 2:14.2.16-280.g962fac13.el7 for package: 2:ceph-radosgw-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.646 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: librgw2 = 2:14.2.16-280.g962fac13.el7 for package: 2:ceph-radosgw-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.647 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: librabbitmq.so.4()(64bit) for package: 2:ceph-radosgw-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.649 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: liboath.so.0()(64bit) for package: 2:ceph-radosgw-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.650 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package ceph-test.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.653 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: ceph-common = 2:14.2.16-280.g962fac13.el7 for package: 2:ceph-test-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.654 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: jq for package: 2:ceph-test-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.655 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: socat for package: 2:ceph-test-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.656 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-test-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.657 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package elfutils-libelf-devel.x86_64 0:0.176-5.el7 will be installed 2021-02-10T11:09:43.658 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: pkgconfig(zlib) for package: elfutils-libelf-devel-0.176-5.el7.x86_64 2021-02-10T11:09:43.662 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package flex.x86_64 0:2.5.37-6.el7 will be installed 2021-02-10T11:09:43.663 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package libcephfs-devel.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.663 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: librados-devel = 2:14.2.16-280.g962fac13.el7 for package: 2:libcephfs-devel-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.664 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package libcephfs2.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.665 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2021-02-10T11:09:43.665 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package librados2.x86_64 2:14.2.16-280.g962fac13.el7 will be an update 2021-02-10T11:09:43.667 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:librados2-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.667 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2021-02-10T11:09:43.668 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package librbd1.x86_64 2:14.2.16-280.g962fac13.el7 will be an update 2021-02-10T11:09:43.668 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package openssl-devel.x86_64 1:1.0.2k-21.el7_9 will be installed 2021-02-10T11:09:43.669 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: openssl-libs(x86-64) = 1:1.0.2k-21.el7_9 for package: 1:openssl-devel-1.0.2k-21.el7_9.x86_64 2021-02-10T11:09:43.673 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: krb5-devel(x86-64) for package: 1:openssl-devel-1.0.2k-21.el7_9.x86_64 2021-02-10T11:09:43.678 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-ceph-compat.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.679 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-rados = 2:14.2.16-280.g962fac13.el7 for package: 2:python-ceph-compat-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.679 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-cephfs = 2:14.2.16-280.g962fac13.el7 for package: 2:python-ceph-compat-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.680 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-rgw = 2:14.2.16-280.g962fac13.el7 for package: 2:python-ceph-compat-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.681 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-rbd = 2:14.2.16-280.g962fac13.el7 for package: 2:python-ceph-compat-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.682 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python3-cephfs.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.683 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python3-ceph-argparse = 2:14.2.16-280.g962fac13.el7 for package: 2:python3-cephfs-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.684 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python3-rados = 2:14.2.16-280.g962fac13.el7 for package: 2:python3-cephfs-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.685 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package rbd-fuse.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.686 INFO:teuthology.orchestra.run.gibba008.stdout:--> Running transaction check 2021-02-10T11:09:43.686 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package ceph-base.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.690 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package ceph-common.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.692 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-ceph-argparse = 2:14.2.16-280.g962fac13.el7 for package: 2:ceph-common-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.693 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.693 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: librdkafka.so.1()(64bit) for package: 2:ceph-common-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.694 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.696 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:43.696 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package ceph-grafana-dashboards.noarch 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.696 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package ceph-mds.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.696 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package ceph-mon.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.697 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package ceph-osd.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.699 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package ceph-selinux.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.701 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package jq.x86_64 0:1.6-2.el7 will be installed 2021-02-10T11:09:43.701 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: libonig.so.5()(64bit) for package: jq-1.6-2.el7.x86_64 2021-02-10T11:09:43.701 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package krb5-devel.x86_64 0:1.15.1-50.el7 will be installed 2021-02-10T11:09:43.703 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: libverto-devel for package: krb5-devel-1.15.1-50.el7.x86_64 2021-02-10T11:09:43.705 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: libselinux-devel for package: krb5-devel-1.15.1-50.el7.x86_64 2021-02-10T11:09:43.711 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: libcom_err-devel for package: krb5-devel-1.15.1-50.el7.x86_64 2021-02-10T11:09:43.721 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: keyutils-libs-devel for package: krb5-devel-1.15.1-50.el7.x86_64 2021-02-10T11:09:43.723 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package liboath.x86_64 0:2.6.2-1.el7 will be installed 2021-02-10T11:09:43.723 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package librabbitmq.x86_64 0:0.8.0-3.el7 will be installed 2021-02-10T11:09:43.723 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package librados-devel.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.724 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package libradosstriper1.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.724 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package librgw2.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.725 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2021-02-10T11:09:43.726 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2021-02-10T11:09:43.726 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2021-02-10T11:09:43.727 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package openssl-libs.x86_64 1:1.0.2k-19.el7 will be updated 2021-02-10T11:09:43.728 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: openssl-libs(x86-64) = 1:1.0.2k-19.el7 for package: 1:openssl-1.0.2k-19.el7.x86_64 2021-02-10T11:09:43.728 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package openssl-libs.x86_64 1:1.0.2k-21.el7_9 will be an update 2021-02-10T11:09:43.729 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-cephfs.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.729 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2021-02-10T11:09:43.739 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2021-02-10T11:09:43.739 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2021-02-10T11:09:43.740 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2021-02-10T11:09:43.741 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2021-02-10T11:09:43.742 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2021-02-10T11:09:43.743 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2021-02-10T11:09:43.743 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-rados.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.744 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-rbd.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.744 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-requests.noarch 0:2.6.0-10.el7 will be installed 2021-02-10T11:09:43.744 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-10.el7.noarch 2021-02-10T11:09:43.746 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-rgw.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.746 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-routes.noarch 0:1.13-2.el7 will be installed 2021-02-10T11:09:43.746 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-repoze-lru for package: python-routes-1.13-2.el7.noarch 2021-02-10T11:09:43.747 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2021-02-10T11:09:43.747 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python2-bcrypt.x86_64 0:3.1.6-2.el7 will be installed 2021-02-10T11:09:43.747 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python2-six for package: python2-bcrypt-3.1.6-2.el7.x86_64 2021-02-10T11:09:43.748 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python2-kubernetes.noarch 1:11.0.0-6.el7 will be installed 2021-02-10T11:09:43.749 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-certifi for package: 1:python2-kubernetes-11.0.0-6.el7.noarch 2021-02-10T11:09:43.749 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-google-auth for package: 1:python2-kubernetes-11.0.0-6.el7.noarch 2021-02-10T11:09:43.750 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-requests-oauthlib for package: 1:python2-kubernetes-11.0.0-6.el7.noarch 2021-02-10T11:09:43.751 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-websocket-client for package: 1:python2-kubernetes-11.0.0-6.el7.noarch 2021-02-10T11:09:43.753 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python2-remoto.noarch 0:1.0.0-1.el7 will be installed 2021-02-10T11:09:43.754 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python2-execnet >= 1.2.0 for package: python2-remoto-1.0.0-1.el7.noarch 2021-02-10T11:09:43.755 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python3-ceph-argparse.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.755 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python3-rados.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.755 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package scipy.x86_64 0:0.12.1-6.el7 will be installed 2021-02-10T11:09:43.757 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: f2py for package: scipy-0.12.1-6.el7.x86_64 2021-02-10T11:09:43.758 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: libumfpack.so.5()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2021-02-10T11:09:43.760 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: libamd.so.2()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2021-02-10T11:09:43.760 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2021-02-10T11:09:43.761 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package zlib-devel.x86_64 0:1.2.7-19.el7_9 will be installed 2021-02-10T11:09:43.761 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: zlib = 1.2.7-19.el7_9 for package: zlib-devel-1.2.7-19.el7_9.x86_64 2021-02-10T11:09:43.764 INFO:teuthology.orchestra.run.gibba008.stdout:--> Running transaction check 2021-02-10T11:09:43.764 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package keyutils-libs-devel.x86_64 0:1.5.8-3.el7 will be installed 2021-02-10T11:09:43.764 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2021-02-10T11:09:43.765 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package libcom_err-devel.x86_64 0:1.42.9-19.el7 will be installed 2021-02-10T11:09:43.765 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package librdkafka.x86_64 0:0.11.5-1.el7 will be installed 2021-02-10T11:09:43.766 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package libselinux-devel.x86_64 0:2.5-15.el7 will be installed 2021-02-10T11:09:43.766 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: libsepol-devel(x86-64) >= 2.5-10 for package: libselinux-devel-2.5-15.el7.x86_64 2021-02-10T11:09:43.768 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: pkgconfig(libsepol) for package: libselinux-devel-2.5-15.el7.x86_64 2021-02-10T11:09:43.768 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: pkgconfig(libpcre) for package: libselinux-devel-2.5-15.el7.x86_64 2021-02-10T11:09:43.772 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package libverto-devel.x86_64 0:0.2.5-4.el7 will be installed 2021-02-10T11:09:43.773 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package numpy-f2py.x86_64 1:1.7.1-13.el7 will be installed 2021-02-10T11:09:43.773 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package oniguruma.x86_64 0:6.8.2-1.el7 will be installed 2021-02-10T11:09:43.773 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package openssl.x86_64 1:1.0.2k-19.el7 will be updated 2021-02-10T11:09:43.774 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package openssl.x86_64 1:1.0.2k-21.el7_9 will be an update 2021-02-10T11:09:43.775 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-ceph-argparse.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:43.775 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-execnet.noarch 0:1.7.1-3.el7 will be installed 2021-02-10T11:09:43.775 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-apipkg for package: python-execnet-1.7.1-3.el7.noarch 2021-02-10T11:09:43.776 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2021-02-10T11:09:43.776 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-markupsafe for package: python-mako-0.8.1-2.el7.noarch 2021-02-10T11:09:43.777 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2021-02-10T11:09:43.778 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2021-02-10T11:09:43.778 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-repoze-lru.noarch 0:0.4-3.el7 will be installed 2021-02-10T11:09:43.778 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2021-02-10T11:09:43.778 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2021-02-10T11:09:43.778 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2021-02-10T11:09:43.778 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2021-02-10T11:09:43.778 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 will be installed 2021-02-10T11:09:43.779 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2021-02-10T11:09:43.779 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python2-certifi.noarch 0:2018.10.15-5.el7 will be installed 2021-02-10T11:09:43.779 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python2-google-auth.noarch 1:1.1.1-5.el7 will be installed 2021-02-10T11:09:43.779 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-cachetools for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2021-02-10T11:09:43.780 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python2-pyasn1-modules for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2021-02-10T11:09:43.780 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python2-rsa for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2021-02-10T11:09:43.781 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python2-requests-oauthlib.noarch 0:0.8.0-5.el7 will be installed 2021-02-10T11:09:43.781 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python2-oauthlib for package: python2-requests-oauthlib-0.8.0-5.el7.noarch 2021-02-10T11:09:43.782 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python2-six.noarch 0:1.9.0-0.el7 will be installed 2021-02-10T11:09:43.782 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package suitesparse.x86_64 0:4.0.2-10.el7 will be installed 2021-02-10T11:09:43.782 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: libtbbmalloc.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2021-02-10T11:09:43.784 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: libtbb.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2021-02-10T11:09:43.784 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2021-02-10T11:09:43.785 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package zlib.x86_64 0:1.2.7-18.el7 will be updated 2021-02-10T11:09:43.786 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package zlib.x86_64 0:1.2.7-19.el7_9 will be an update 2021-02-10T11:09:43.786 INFO:teuthology.orchestra.run.gibba008.stdout:--> Running transaction check 2021-02-10T11:09:43.786 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package libsepol-devel.x86_64 0:2.5-10.el7 will be installed 2021-02-10T11:09:43.787 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package pcre-devel.x86_64 0:8.32-17.el7 will be installed 2021-02-10T11:09:43.787 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2021-02-10T11:09:43.787 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2021-02-10T11:09:43.788 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-cachetools.noarch 0:1.0.3-1.el7 will be installed 2021-02-10T11:09:43.788 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2021-02-10T11:09:43.788 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python2-apipkg.noarch 0:1.5-5.el7 will be installed 2021-02-10T11:09:43.788 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python2-oauthlib.noarch 0:2.0.1-8.el7 will be installed 2021-02-10T11:09:43.788 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-jwcrypto for package: python2-oauthlib-2.0.1-8.el7.noarch 2021-02-10T11:09:43.789 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python2-pyasn1-modules.noarch 0:0.1.9-7.el7 will be installed 2021-02-10T11:09:43.790 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python2-rsa.noarch 0:3.4.2-2.el7 will be installed 2021-02-10T11:09:43.790 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package tbb.x86_64 0:4.1-9.20130314.el7 will be installed 2021-02-10T11:09:43.790 INFO:teuthology.orchestra.run.gibba008.stdout:--> Running transaction check 2021-02-10T11:09:43.790 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-jwcrypto.noarch 0:0.4.2-1.el7 will be installed 2021-02-10T11:09:43.791 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2021-02-10T11:09:43.791 INFO:teuthology.orchestra.run.gibba008.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2021-02-10T11:09:43.791 INFO:teuthology.orchestra.run.gibba008.stdout:--> Running transaction check 2021-02-10T11:09:43.791 INFO:teuthology.orchestra.run.gibba008.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2021-02-10T11:09:44.351 INFO:teuthology.orchestra.run.gibba008.stdout:--> Finished Dependency Resolution 2021-02-10T11:09:44.407 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:09:44.408 INFO:teuthology.orchestra.run.gibba008.stdout:Dependencies Resolved 2021-02-10T11:09:44.417 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:09:44.417 INFO:teuthology.orchestra.run.gibba008.stdout:================================================================================ 2021-02-10T11:09:44.417 INFO:teuthology.orchestra.run.gibba008.stdout: Package Arch Version Repository Size 2021-02-10T11:09:44.417 INFO:teuthology.orchestra.run.gibba008.stdout:================================================================================ 2021-02-10T11:09:44.417 INFO:teuthology.orchestra.run.gibba008.stdout:Installing: 2021-02-10T11:09:44.417 INFO:teuthology.orchestra.run.gibba008.stdout: bison x86_64 3.0.4-2.el7 rhel-7-server-rpms 674 k 2021-02-10T11:09:44.417 INFO:teuthology.orchestra.run.gibba008.stdout: ceph x86_64 2:14.2.16-280.g962fac13.el7 ceph 2.0 k 2021-02-10T11:09:44.417 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-fuse x86_64 2:14.2.16-280.g962fac13.el7 ceph 571 k 2021-02-10T11:09:44.417 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-mgr x86_64 2:14.2.16-280.g962fac13.el7 ceph 1.8 M 2021-02-10T11:09:44.418 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-mgr-dashboard noarch 2:14.2.16-280.g962fac13.el7 ceph-noarch 4.0 M 2021-02-10T11:09:44.418 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-mgr-diskprediction-cloud 2021-02-10T11:09:44.418 INFO:teuthology.orchestra.run.gibba008.stdout: noarch 2:14.2.16-280.g962fac13.el7 ceph-noarch 83 k 2021-02-10T11:09:44.418 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-mgr-diskprediction-local 2021-02-10T11:09:44.418 INFO:teuthology.orchestra.run.gibba008.stdout: noarch 2:14.2.16-280.g962fac13.el7 ceph-noarch 1.1 M 2021-02-10T11:09:44.418 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-mgr-rook noarch 2:14.2.16-280.g962fac13.el7 ceph-noarch 21 k 2021-02-10T11:09:44.418 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-mgr-ssh noarch 2:14.2.16-280.g962fac13.el7 ceph-noarch 21 k 2021-02-10T11:09:44.418 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-radosgw x86_64 2:14.2.16-280.g962fac13.el7 ceph 5.8 M 2021-02-10T11:09:44.418 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-test x86_64 2:14.2.16-280.g962fac13.el7 ceph 34 M 2021-02-10T11:09:44.418 INFO:teuthology.orchestra.run.gibba008.stdout: elfutils-libelf-devel 2021-02-10T11:09:44.418 INFO:teuthology.orchestra.run.gibba008.stdout: x86_64 0.176-5.el7 rhel-7-server-rpms 40 k 2021-02-10T11:09:44.418 INFO:teuthology.orchestra.run.gibba008.stdout: flex x86_64 2.5.37-6.el7 rhel-7-server-rpms 293 k 2021-02-10T11:09:44.419 INFO:teuthology.orchestra.run.gibba008.stdout: libcephfs-devel x86_64 2:14.2.16-280.g962fac13.el7 ceph 17 k 2021-02-10T11:09:44.419 INFO:teuthology.orchestra.run.gibba008.stdout: libcephfs2 x86_64 2:14.2.16-280.g962fac13.el7 ceph 500 k 2021-02-10T11:09:44.419 INFO:teuthology.orchestra.run.gibba008.stdout: openssl-devel x86_64 1:1.0.2k-21.el7_9 rhel-7-server-rpms 1.5 M 2021-02-10T11:09:44.419 INFO:teuthology.orchestra.run.gibba008.stdout: python-ceph-compat x86_64 2:14.2.16-280.g962fac13.el7 ceph 2.2 k 2021-02-10T11:09:44.419 INFO:teuthology.orchestra.run.gibba008.stdout: python3-cephfs x86_64 2:14.2.16-280.g962fac13.el7 ceph 151 k 2021-02-10T11:09:44.419 INFO:teuthology.orchestra.run.gibba008.stdout: rbd-fuse x86_64 2:14.2.16-280.g962fac13.el7 ceph 69 k 2021-02-10T11:09:44.419 INFO:teuthology.orchestra.run.gibba008.stdout:Updating: 2021-02-10T11:09:44.419 INFO:teuthology.orchestra.run.gibba008.stdout: librados2 x86_64 2:14.2.16-280.g962fac13.el7 ceph 3.4 M 2021-02-10T11:09:44.419 INFO:teuthology.orchestra.run.gibba008.stdout: librbd1 x86_64 2:14.2.16-280.g962fac13.el7 ceph 1.7 M 2021-02-10T11:09:44.419 INFO:teuthology.orchestra.run.gibba008.stdout:Installing for dependencies: 2021-02-10T11:09:44.419 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-base x86_64 2:14.2.16-280.g962fac13.el7 ceph 5.4 M 2021-02-10T11:09:44.419 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-common x86_64 2:14.2.16-280.g962fac13.el7 ceph 19 M 2021-02-10T11:09:44.420 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-grafana-dashboards 2021-02-10T11:09:44.420 INFO:teuthology.orchestra.run.gibba008.stdout: noarch 2:14.2.16-280.g962fac13.el7 ceph-noarch 19 k 2021-02-10T11:09:44.420 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-mds x86_64 2:14.2.16-280.g962fac13.el7 ceph 1.9 M 2021-02-10T11:09:44.420 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-mon x86_64 2:14.2.16-280.g962fac13.el7 ceph 4.3 M 2021-02-10T11:09:44.420 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-osd x86_64 2:14.2.16-280.g962fac13.el7 ceph 16 M 2021-02-10T11:09:44.420 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-selinux x86_64 2:14.2.16-280.g962fac13.el7 ceph 20 k 2021-02-10T11:09:44.420 INFO:teuthology.orchestra.run.gibba008.stdout: jq x86_64 1.6-2.el7 epel 167 k 2021-02-10T11:09:44.420 INFO:teuthology.orchestra.run.gibba008.stdout: keyutils-libs-devel 2021-02-10T11:09:44.420 INFO:teuthology.orchestra.run.gibba008.stdout: x86_64 1.5.8-3.el7 rhel-7-server-rpms 37 k 2021-02-10T11:09:44.420 INFO:teuthology.orchestra.run.gibba008.stdout: krb5-devel x86_64 1.15.1-50.el7 rhel-7-server-rpms 273 k 2021-02-10T11:09:44.420 INFO:teuthology.orchestra.run.gibba008.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2021-02-10T11:09:44.420 INFO:teuthology.orchestra.run.gibba008.stdout: libcom_err-devel x86_64 1.42.9-19.el7 rhel-7-server-rpms 32 k 2021-02-10T11:09:44.421 INFO:teuthology.orchestra.run.gibba008.stdout: liboath x86_64 2.6.2-1.el7 epel 51 k 2021-02-10T11:09:44.421 INFO:teuthology.orchestra.run.gibba008.stdout: librabbitmq x86_64 0.8.0-3.el7 rhel-7-server-rpms 37 k 2021-02-10T11:09:44.421 INFO:teuthology.orchestra.run.gibba008.stdout: librados-devel x86_64 2:14.2.16-280.g962fac13.el7 ceph 86 k 2021-02-10T11:09:44.421 INFO:teuthology.orchestra.run.gibba008.stdout: libradosstriper1 x86_64 2:14.2.16-280.g962fac13.el7 ceph 364 k 2021-02-10T11:09:44.421 INFO:teuthology.orchestra.run.gibba008.stdout: librdkafka x86_64 0.11.5-1.el7 epel 368 k 2021-02-10T11:09:44.421 INFO:teuthology.orchestra.run.gibba008.stdout: librgw2 x86_64 2:14.2.16-280.g962fac13.el7 ceph 5.1 M 2021-02-10T11:09:44.421 INFO:teuthology.orchestra.run.gibba008.stdout: libselinux-devel x86_64 2.5-15.el7 rhel-7-server-rpms 187 k 2021-02-10T11:09:44.421 INFO:teuthology.orchestra.run.gibba008.stdout: libsepol-devel x86_64 2.5-10.el7 rhel-7-server-rpms 77 k 2021-02-10T11:09:44.421 INFO:teuthology.orchestra.run.gibba008.stdout: libverto-devel x86_64 0.2.5-4.el7 rhel-7-server-rpms 12 k 2021-02-10T11:09:44.421 INFO:teuthology.orchestra.run.gibba008.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2021-02-10T11:09:44.421 INFO:teuthology.orchestra.run.gibba008.stdout: numpy-f2py x86_64 1:1.7.1-13.el7 rhel-7-server-rpms 206 k 2021-02-10T11:09:44.421 INFO:teuthology.orchestra.run.gibba008.stdout: oniguruma x86_64 6.8.2-1.el7 epel 181 k 2021-02-10T11:09:44.422 INFO:teuthology.orchestra.run.gibba008.stdout: pcre-devel x86_64 8.32-17.el7 rhel-7-server-rpms 480 k 2021-02-10T11:09:44.422 INFO:teuthology.orchestra.run.gibba008.stdout: python-beaker noarch 1.5.4-10.el7 rhel-7-server-rpms 80 k 2021-02-10T11:09:44.422 INFO:teuthology.orchestra.run.gibba008.stdout: python-cachetools noarch 1.0.3-1.el7 epel 22 k 2021-02-10T11:09:44.422 INFO:teuthology.orchestra.run.gibba008.stdout: python-ceph-argparse 2021-02-10T11:09:44.422 INFO:teuthology.orchestra.run.gibba008.stdout: x86_64 2:14.2.16-280.g962fac13.el7 ceph 35 k 2021-02-10T11:09:44.422 INFO:teuthology.orchestra.run.gibba008.stdout: python-cephfs x86_64 2:14.2.16-280.g962fac13.el7 ceph 113 k 2021-02-10T11:09:44.422 INFO:teuthology.orchestra.run.gibba008.stdout: python-cherrypy noarch 3.2.2-4.el7 rhel-7-server-rpms 422 k 2021-02-10T11:09:44.422 INFO:teuthology.orchestra.run.gibba008.stdout: python-execnet noarch 1.7.1-3.el7 epel 260 k 2021-02-10T11:09:44.422 INFO:teuthology.orchestra.run.gibba008.stdout: python-jwcrypto noarch 0.4.2-1.el7 rhel-7-server-rpms 57 k 2021-02-10T11:09:44.422 INFO:teuthology.orchestra.run.gibba008.stdout: python-mako noarch 0.8.1-2.el7 rhel-7-server-rpms 307 k 2021-02-10T11:09:44.422 INFO:teuthology.orchestra.run.gibba008.stdout: python-markupsafe x86_64 0.11-10.el7 rhel-7-server-rpms 25 k 2021-02-10T11:09:44.422 INFO:teuthology.orchestra.run.gibba008.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 2021-02-10T11:09:44.422 INFO:teuthology.orchestra.run.gibba008.stdout: rhel-7-server-rpms 866 k 2021-02-10T11:09:44.423 INFO:teuthology.orchestra.run.gibba008.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2021-02-10T11:09:44.423 INFO:teuthology.orchestra.run.gibba008.stdout: python-prettytable noarch 0.7.2-3.el7 rhel-7-server-rpms 37 k 2021-02-10T11:09:44.423 INFO:teuthology.orchestra.run.gibba008.stdout: python-rados x86_64 2:14.2.16-280.g962fac13.el7 ceph 199 k 2021-02-10T11:09:44.423 INFO:teuthology.orchestra.run.gibba008.stdout: python-rbd x86_64 2:14.2.16-280.g962fac13.el7 ceph 194 k 2021-02-10T11:09:44.423 INFO:teuthology.orchestra.run.gibba008.stdout: python-repoze-lru noarch 0.4-3.el7 epel 13 k 2021-02-10T11:09:44.423 INFO:teuthology.orchestra.run.gibba008.stdout: python-requests noarch 2.6.0-10.el7 rhel-7-server-rpms 95 k 2021-02-10T11:09:44.423 INFO:teuthology.orchestra.run.gibba008.stdout: python-rgw x86_64 2:14.2.16-280.g962fac13.el7 ceph 77 k 2021-02-10T11:09:44.423 INFO:teuthology.orchestra.run.gibba008.stdout: python-routes noarch 1.13-2.el7 epel 640 k 2021-02-10T11:09:44.423 INFO:teuthology.orchestra.run.gibba008.stdout: python-simplegeneric 2021-02-10T11:09:44.423 INFO:teuthology.orchestra.run.gibba008.stdout: noarch 0.8-7.el7 epel 12 k 2021-02-10T11:09:44.423 INFO:teuthology.orchestra.run.gibba008.stdout: python-singledispatch 2021-02-10T11:09:44.423 INFO:teuthology.orchestra.run.gibba008.stdout: noarch 3.4.0.2-2.el7 epel 18 k 2021-02-10T11:09:44.423 INFO:teuthology.orchestra.run.gibba008.stdout: python-tempita noarch 0.5.1-6.el7 rhel-7-server-rpms 33 k 2021-02-10T11:09:44.424 INFO:teuthology.orchestra.run.gibba008.stdout: python-urllib3 noarch 1.10.2-7.el7 rhel-7-server-rpms 103 k 2021-02-10T11:09:44.424 INFO:teuthology.orchestra.run.gibba008.stdout: python-webob noarch 1.2.3-7.el7 rhel-7-server-optional-rpms 2021-02-10T11:09:44.424 INFO:teuthology.orchestra.run.gibba008.stdout: 202 k 2021-02-10T11:09:44.424 INFO:teuthology.orchestra.run.gibba008.stdout: python-websocket-client 2021-02-10T11:09:44.424 INFO:teuthology.orchestra.run.gibba008.stdout: noarch 0.56.0-3.git3c25814.el7 rhel-7-server-extras-rpms 2021-02-10T11:09:44.424 INFO:teuthology.orchestra.run.gibba008.stdout: 66 k 2021-02-10T11:09:44.424 INFO:teuthology.orchestra.run.gibba008.stdout: python-webtest noarch 1.3.4-6.el7 rhel-7-server-optional-rpms 2021-02-10T11:09:44.424 INFO:teuthology.orchestra.run.gibba008.stdout: 102 k 2021-02-10T11:09:44.424 INFO:teuthology.orchestra.run.gibba008.stdout: python-werkzeug noarch 0.9.1-2.el7 rhel-7-server-extras-rpms 2021-02-10T11:09:44.424 INFO:teuthology.orchestra.run.gibba008.stdout: 562 k 2021-02-10T11:09:44.424 INFO:teuthology.orchestra.run.gibba008.stdout: python2-apipkg noarch 1.5-5.el7 epel 17 k 2021-02-10T11:09:44.424 INFO:teuthology.orchestra.run.gibba008.stdout: python2-bcrypt x86_64 3.1.6-2.el7 epel 39 k 2021-02-10T11:09:44.425 INFO:teuthology.orchestra.run.gibba008.stdout: python2-certifi noarch 2018.10.15-5.el7 epel 12 k 2021-02-10T11:09:44.425 INFO:teuthology.orchestra.run.gibba008.stdout: python2-google-auth 2021-02-10T11:09:44.425 INFO:teuthology.orchestra.run.gibba008.stdout: noarch 1:1.1.1-5.el7 epel 76 k 2021-02-10T11:09:44.425 INFO:teuthology.orchestra.run.gibba008.stdout: python2-kubernetes noarch 1:11.0.0-6.el7 epel 1.1 M 2021-02-10T11:09:44.425 INFO:teuthology.orchestra.run.gibba008.stdout: python2-oauthlib noarch 2.0.1-8.el7 rhel-7-server-rpms 146 k 2021-02-10T11:09:44.425 INFO:teuthology.orchestra.run.gibba008.stdout: python2-pyasn1-modules 2021-02-10T11:09:44.425 INFO:teuthology.orchestra.run.gibba008.stdout: noarch 0.1.9-7.el7 rhel-7-server-optional-rpms 2021-02-10T11:09:44.425 INFO:teuthology.orchestra.run.gibba008.stdout: 59 k 2021-02-10T11:09:44.425 INFO:teuthology.orchestra.run.gibba008.stdout: python2-remoto noarch 1.0.0-1.el7 epel 42 k 2021-02-10T11:09:44.425 INFO:teuthology.orchestra.run.gibba008.stdout: python2-requests-oauthlib 2021-02-10T11:09:44.425 INFO:teuthology.orchestra.run.gibba008.stdout: noarch 0.8.0-5.el7 rhel-7-server-rpms 35 k 2021-02-10T11:09:44.425 INFO:teuthology.orchestra.run.gibba008.stdout: python2-rsa noarch 3.4.2-2.el7 epel 68 k 2021-02-10T11:09:44.425 INFO:teuthology.orchestra.run.gibba008.stdout: python2-six noarch 1.9.0-0.el7 epel 2.9 k 2021-02-10T11:09:44.426 INFO:teuthology.orchestra.run.gibba008.stdout: python3-ceph-argparse 2021-02-10T11:09:44.426 INFO:teuthology.orchestra.run.gibba008.stdout: x86_64 2:14.2.16-280.g962fac13.el7 ceph 35 k 2021-02-10T11:09:44.426 INFO:teuthology.orchestra.run.gibba008.stdout: python3-rados x86_64 2:14.2.16-280.g962fac13.el7 ceph 317 k 2021-02-10T11:09:44.426 INFO:teuthology.orchestra.run.gibba008.stdout: scipy x86_64 0.12.1-6.el7 rhel-7-server-rpms 9.3 M 2021-02-10T11:09:44.426 INFO:teuthology.orchestra.run.gibba008.stdout: socat x86_64 1.7.3.2-2.el7 rhel-7-server-rpms 290 k 2021-02-10T11:09:44.426 INFO:teuthology.orchestra.run.gibba008.stdout: suitesparse x86_64 4.0.2-10.el7 rhel-7-server-rpms 928 k 2021-02-10T11:09:44.426 INFO:teuthology.orchestra.run.gibba008.stdout: tbb x86_64 4.1-9.20130314.el7 rhel-7-server-rpms 124 k 2021-02-10T11:09:44.426 INFO:teuthology.orchestra.run.gibba008.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2021-02-10T11:09:44.426 INFO:teuthology.orchestra.run.gibba008.stdout: zlib-devel x86_64 1.2.7-19.el7_9 rhel-7-server-rpms 50 k 2021-02-10T11:09:44.426 INFO:teuthology.orchestra.run.gibba008.stdout:Updating for dependencies: 2021-02-10T11:09:44.426 INFO:teuthology.orchestra.run.gibba008.stdout: openssl x86_64 1:1.0.2k-21.el7_9 rhel-7-server-rpms 493 k 2021-02-10T11:09:44.426 INFO:teuthology.orchestra.run.gibba008.stdout: openssl-libs x86_64 1:1.0.2k-21.el7_9 rhel-7-server-rpms 1.2 M 2021-02-10T11:09:44.426 INFO:teuthology.orchestra.run.gibba008.stdout: zlib x86_64 1.2.7-19.el7_9 rhel-7-server-rpms 90 k 2021-02-10T11:09:44.427 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:09:44.427 INFO:teuthology.orchestra.run.gibba008.stdout:Transaction Summary 2021-02-10T11:09:44.427 INFO:teuthology.orchestra.run.gibba008.stdout:================================================================================ 2021-02-10T11:09:44.427 INFO:teuthology.orchestra.run.gibba008.stdout:Install 19 Packages (+70 Dependent packages) 2021-02-10T11:09:44.427 INFO:teuthology.orchestra.run.gibba008.stdout:Upgrade 2 Packages (+ 3 Dependent packages) 2021-02-10T11:09:44.427 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:09:44.427 INFO:teuthology.orchestra.run.gibba008.stdout:Total download size: 130 M 2021-02-10T11:09:44.427 INFO:teuthology.orchestra.run.gibba008.stdout:Downloading packages: 2021-02-10T11:09:44.427 INFO:teuthology.orchestra.run.gibba008.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2021-02-10T11:09:44.837 INFO:teuthology.orchestra.run.gibba001.stdout:Resolving Dependencies 2021-02-10T11:09:44.837 INFO:teuthology.orchestra.run.gibba001.stdout:--> Running transaction check 2021-02-10T11:09:44.838 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package bison.x86_64 0:3.0.4-2.el7 will be installed 2021-02-10T11:09:44.844 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package ceph.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.845 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: ceph-mon = 2:14.2.16-280.g962fac13.el7 for package: 2:ceph-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.846 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: ceph-osd = 2:14.2.16-280.g962fac13.el7 for package: 2:ceph-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.847 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: ceph-mds = 2:14.2.16-280.g962fac13.el7 for package: 2:ceph-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.848 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package ceph-fuse.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.855 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package ceph-mgr.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.857 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: ceph-base = 2:14.2.16-280.g962fac13.el7 for package: 2:ceph-mgr-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.858 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-requests for package: 2:ceph-mgr-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.861 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.862 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.863 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.864 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-bcrypt for package: 2:ceph-mgr-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.864 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package ceph-mgr-dashboard.noarch 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.865 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: ceph-grafana-dashboards = 2:14.2.16-280.g962fac13.el7 for package: 2:ceph-mgr-dashboard-14.2.16-280.g962fac13.el7.noarch 2021-02-10T11:09:44.866 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-routes for package: 2:ceph-mgr-dashboard-14.2.16-280.g962fac13.el7.noarch 2021-02-10T11:09:44.866 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package ceph-mgr-diskprediction-cloud.noarch 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.866 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package ceph-mgr-diskprediction-local.noarch 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.867 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: scipy for package: 2:ceph-mgr-diskprediction-local-14.2.16-280.g962fac13.el7.noarch 2021-02-10T11:09:44.868 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package ceph-mgr-rook.noarch 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.868 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-kubernetes for package: 2:ceph-mgr-rook-14.2.16-280.g962fac13.el7.noarch 2021-02-10T11:09:44.869 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package ceph-mgr-ssh.noarch 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.869 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-remoto for package: 2:ceph-mgr-ssh-14.2.16-280.g962fac13.el7.noarch 2021-02-10T11:09:44.870 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package ceph-radosgw.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.873 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: ceph-selinux = 2:14.2.16-280.g962fac13.el7 for package: 2:ceph-radosgw-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.874 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: librgw2 = 2:14.2.16-280.g962fac13.el7 for package: 2:ceph-radosgw-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.875 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: librabbitmq.so.4()(64bit) for package: 2:ceph-radosgw-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.877 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: liboath.so.0()(64bit) for package: 2:ceph-radosgw-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.878 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package ceph-test.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.881 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: ceph-common = 2:14.2.16-280.g962fac13.el7 for package: 2:ceph-test-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.882 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: jq for package: 2:ceph-test-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.883 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: socat for package: 2:ceph-test-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.884 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-test-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.885 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package elfutils-libelf-devel.x86_64 0:0.176-5.el7 will be installed 2021-02-10T11:09:44.885 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: pkgconfig(zlib) for package: elfutils-libelf-devel-0.176-5.el7.x86_64 2021-02-10T11:09:44.890 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package flex.x86_64 0:2.5.37-6.el7 will be installed 2021-02-10T11:09:44.891 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package libcephfs-devel.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.891 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: librados-devel = 2:14.2.16-280.g962fac13.el7 for package: 2:libcephfs-devel-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.892 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package libcephfs2.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.892 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2021-02-10T11:09:44.893 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package librados2.x86_64 2:14.2.16-280.g962fac13.el7 will be an update 2021-02-10T11:09:44.894 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:librados2-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.895 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2021-02-10T11:09:44.896 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package librbd1.x86_64 2:14.2.16-280.g962fac13.el7 will be an update 2021-02-10T11:09:44.897 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package openssl-devel.x86_64 1:1.0.2k-21.el7_9 will be installed 2021-02-10T11:09:44.897 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: openssl-libs(x86-64) = 1:1.0.2k-21.el7_9 for package: 1:openssl-devel-1.0.2k-21.el7_9.x86_64 2021-02-10T11:09:44.901 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: krb5-devel(x86-64) for package: 1:openssl-devel-1.0.2k-21.el7_9.x86_64 2021-02-10T11:09:44.907 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-ceph-compat.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.907 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-rados = 2:14.2.16-280.g962fac13.el7 for package: 2:python-ceph-compat-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.907 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-cephfs = 2:14.2.16-280.g962fac13.el7 for package: 2:python-ceph-compat-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.908 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-rgw = 2:14.2.16-280.g962fac13.el7 for package: 2:python-ceph-compat-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.909 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-rbd = 2:14.2.16-280.g962fac13.el7 for package: 2:python-ceph-compat-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.909 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python3-cephfs.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.911 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python3-ceph-argparse = 2:14.2.16-280.g962fac13.el7 for package: 2:python3-cephfs-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.912 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python3-rados = 2:14.2.16-280.g962fac13.el7 for package: 2:python3-cephfs-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.912 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package rbd-fuse.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.913 INFO:teuthology.orchestra.run.gibba001.stdout:--> Running transaction check 2021-02-10T11:09:44.913 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package ceph-base.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.917 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package ceph-common.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.919 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-ceph-argparse = 2:14.2.16-280.g962fac13.el7 for package: 2:ceph-common-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.920 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.921 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: librdkafka.so.1()(64bit) for package: 2:ceph-common-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.922 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.923 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-14.2.16-280.g962fac13.el7.x86_64 2021-02-10T11:09:44.923 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package ceph-grafana-dashboards.noarch 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.923 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package ceph-mds.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.923 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package ceph-mon.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.924 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package ceph-osd.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.926 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package ceph-selinux.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.928 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package jq.x86_64 0:1.6-2.el7 will be installed 2021-02-10T11:09:44.928 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: libonig.so.5()(64bit) for package: jq-1.6-2.el7.x86_64 2021-02-10T11:09:44.928 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package krb5-devel.x86_64 0:1.15.1-50.el7 will be installed 2021-02-10T11:09:44.931 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: libverto-devel for package: krb5-devel-1.15.1-50.el7.x86_64 2021-02-10T11:09:44.933 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: libselinux-devel for package: krb5-devel-1.15.1-50.el7.x86_64 2021-02-10T11:09:44.938 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: libcom_err-devel for package: krb5-devel-1.15.1-50.el7.x86_64 2021-02-10T11:09:44.948 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: keyutils-libs-devel for package: krb5-devel-1.15.1-50.el7.x86_64 2021-02-10T11:09:44.950 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package liboath.x86_64 0:2.6.2-1.el7 will be installed 2021-02-10T11:09:44.950 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package librabbitmq.x86_64 0:0.8.0-3.el7 will be installed 2021-02-10T11:09:44.950 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package librados-devel.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.951 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package libradosstriper1.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.951 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package librgw2.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.952 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2021-02-10T11:09:44.952 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2021-02-10T11:09:44.953 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2021-02-10T11:09:44.953 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package openssl-libs.x86_64 1:1.0.2k-19.el7 will be updated 2021-02-10T11:09:44.954 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: openssl-libs(x86-64) = 1:1.0.2k-19.el7 for package: 1:openssl-1.0.2k-19.el7.x86_64 2021-02-10T11:09:44.955 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package openssl-libs.x86_64 1:1.0.2k-21.el7_9 will be an update 2021-02-10T11:09:44.956 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-cephfs.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.956 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2021-02-10T11:09:44.957 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2021-02-10T11:09:44.957 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2021-02-10T11:09:44.958 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2021-02-10T11:09:44.958 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2021-02-10T11:09:44.959 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2021-02-10T11:09:44.961 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2021-02-10T11:09:44.961 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-rados.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.962 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-rbd.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.962 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-requests.noarch 0:2.6.0-10.el7 will be installed 2021-02-10T11:09:44.962 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-10.el7.noarch 2021-02-10T11:09:44.963 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-rgw.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.964 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-routes.noarch 0:1.13-2.el7 will be installed 2021-02-10T11:09:44.964 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-repoze-lru for package: python-routes-1.13-2.el7.noarch 2021-02-10T11:09:44.964 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2021-02-10T11:09:44.964 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python2-bcrypt.x86_64 0:3.1.6-2.el7 will be installed 2021-02-10T11:09:44.965 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python2-six for package: python2-bcrypt-3.1.6-2.el7.x86_64 2021-02-10T11:09:44.966 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python2-kubernetes.noarch 1:11.0.0-6.el7 will be installed 2021-02-10T11:09:44.966 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-certifi for package: 1:python2-kubernetes-11.0.0-6.el7.noarch 2021-02-10T11:09:44.967 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-google-auth for package: 1:python2-kubernetes-11.0.0-6.el7.noarch 2021-02-10T11:09:44.967 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-requests-oauthlib for package: 1:python2-kubernetes-11.0.0-6.el7.noarch 2021-02-10T11:09:44.968 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-websocket-client for package: 1:python2-kubernetes-11.0.0-6.el7.noarch 2021-02-10T11:09:44.971 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python2-remoto.noarch 0:1.0.0-1.el7 will be installed 2021-02-10T11:09:44.971 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python2-execnet >= 1.2.0 for package: python2-remoto-1.0.0-1.el7.noarch 2021-02-10T11:09:44.972 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python3-ceph-argparse.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.972 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python3-rados.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.972 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package scipy.x86_64 0:0.12.1-6.el7 will be installed 2021-02-10T11:09:44.974 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: f2py for package: scipy-0.12.1-6.el7.x86_64 2021-02-10T11:09:44.975 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: libumfpack.so.5()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2021-02-10T11:09:44.977 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: libamd.so.2()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2021-02-10T11:09:44.977 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2021-02-10T11:09:44.978 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package zlib-devel.x86_64 0:1.2.7-19.el7_9 will be installed 2021-02-10T11:09:44.978 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: zlib = 1.2.7-19.el7_9 for package: zlib-devel-1.2.7-19.el7_9.x86_64 2021-02-10T11:09:44.981 INFO:teuthology.orchestra.run.gibba001.stdout:--> Running transaction check 2021-02-10T11:09:44.981 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package keyutils-libs-devel.x86_64 0:1.5.8-3.el7 will be installed 2021-02-10T11:09:44.981 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2021-02-10T11:09:44.982 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package libcom_err-devel.x86_64 0:1.42.9-19.el7 will be installed 2021-02-10T11:09:44.982 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package librdkafka.x86_64 0:0.11.5-1.el7 will be installed 2021-02-10T11:09:44.983 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package libselinux-devel.x86_64 0:2.5-15.el7 will be installed 2021-02-10T11:09:44.983 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: libsepol-devel(x86-64) >= 2.5-10 for package: libselinux-devel-2.5-15.el7.x86_64 2021-02-10T11:09:44.985 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: pkgconfig(libsepol) for package: libselinux-devel-2.5-15.el7.x86_64 2021-02-10T11:09:44.985 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: pkgconfig(libpcre) for package: libselinux-devel-2.5-15.el7.x86_64 2021-02-10T11:09:44.989 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package libverto-devel.x86_64 0:0.2.5-4.el7 will be installed 2021-02-10T11:09:44.990 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package numpy-f2py.x86_64 1:1.7.1-13.el7 will be installed 2021-02-10T11:09:44.990 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package oniguruma.x86_64 0:6.8.2-1.el7 will be installed 2021-02-10T11:09:44.991 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package openssl.x86_64 1:1.0.2k-19.el7 will be updated 2021-02-10T11:09:44.991 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package openssl.x86_64 1:1.0.2k-21.el7_9 will be an update 2021-02-10T11:09:44.992 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-ceph-argparse.x86_64 2:14.2.16-280.g962fac13.el7 will be installed 2021-02-10T11:09:44.992 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-execnet.noarch 0:1.7.1-3.el7 will be installed 2021-02-10T11:09:44.992 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-apipkg for package: python-execnet-1.7.1-3.el7.noarch 2021-02-10T11:09:44.993 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2021-02-10T11:09:44.993 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-markupsafe for package: python-mako-0.8.1-2.el7.noarch 2021-02-10T11:09:44.994 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2021-02-10T11:09:44.995 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2021-02-10T11:09:44.995 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-repoze-lru.noarch 0:0.4-3.el7 will be installed 2021-02-10T11:09:44.995 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2021-02-10T11:09:44.995 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2021-02-10T11:09:44.995 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2021-02-10T11:09:44.996 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2021-02-10T11:09:44.996 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 will be installed 2021-02-10T11:09:44.996 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2021-02-10T11:09:44.996 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python2-certifi.noarch 0:2018.10.15-5.el7 will be installed 2021-02-10T11:09:44.996 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python2-google-auth.noarch 1:1.1.1-5.el7 will be installed 2021-02-10T11:09:44.996 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-cachetools for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2021-02-10T11:09:44.996 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python2-pyasn1-modules for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2021-02-10T11:09:44.997 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python2-rsa for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2021-02-10T11:09:44.998 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python2-requests-oauthlib.noarch 0:0.8.0-5.el7 will be installed 2021-02-10T11:09:44.998 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python2-oauthlib for package: python2-requests-oauthlib-0.8.0-5.el7.noarch 2021-02-10T11:09:44.999 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python2-six.noarch 0:1.9.0-0.el7 will be installed 2021-02-10T11:09:44.999 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package suitesparse.x86_64 0:4.0.2-10.el7 will be installed 2021-02-10T11:09:44.999 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: libtbbmalloc.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2021-02-10T11:09:45.001 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: libtbb.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2021-02-10T11:09:45.001 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2021-02-10T11:09:45.001 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package zlib.x86_64 0:1.2.7-18.el7 will be updated 2021-02-10T11:09:45.003 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package zlib.x86_64 0:1.2.7-19.el7_9 will be an update 2021-02-10T11:09:45.003 INFO:teuthology.orchestra.run.gibba001.stdout:--> Running transaction check 2021-02-10T11:09:45.003 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package libsepol-devel.x86_64 0:2.5-10.el7 will be installed 2021-02-10T11:09:45.004 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package pcre-devel.x86_64 0:8.32-17.el7 will be installed 2021-02-10T11:09:45.004 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2021-02-10T11:09:45.004 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2021-02-10T11:09:45.005 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-cachetools.noarch 0:1.0.3-1.el7 will be installed 2021-02-10T11:09:45.005 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2021-02-10T11:09:45.005 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python2-apipkg.noarch 0:1.5-5.el7 will be installed 2021-02-10T11:09:45.005 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python2-oauthlib.noarch 0:2.0.1-8.el7 will be installed 2021-02-10T11:09:45.005 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-jwcrypto for package: python2-oauthlib-2.0.1-8.el7.noarch 2021-02-10T11:09:45.006 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python2-pyasn1-modules.noarch 0:0.1.9-7.el7 will be installed 2021-02-10T11:09:45.006 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python2-rsa.noarch 0:3.4.2-2.el7 will be installed 2021-02-10T11:09:45.007 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package tbb.x86_64 0:4.1-9.20130314.el7 will be installed 2021-02-10T11:09:45.007 INFO:teuthology.orchestra.run.gibba001.stdout:--> Running transaction check 2021-02-10T11:09:45.007 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-jwcrypto.noarch 0:0.4.2-1.el7 will be installed 2021-02-10T11:09:45.007 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2021-02-10T11:09:45.007 INFO:teuthology.orchestra.run.gibba001.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2021-02-10T11:09:45.008 INFO:teuthology.orchestra.run.gibba001.stdout:--> Running transaction check 2021-02-10T11:09:45.008 INFO:teuthology.orchestra.run.gibba001.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2021-02-10T11:09:45.497 INFO:teuthology.orchestra.run.gibba001.stdout:--> Finished Dependency Resolution 2021-02-10T11:09:45.553 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:09:45.553 INFO:teuthology.orchestra.run.gibba001.stdout:Dependencies Resolved 2021-02-10T11:09:45.562 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:09:45.562 INFO:teuthology.orchestra.run.gibba001.stdout:================================================================================ 2021-02-10T11:09:45.562 INFO:teuthology.orchestra.run.gibba001.stdout: Package Arch Version Repository Size 2021-02-10T11:09:45.562 INFO:teuthology.orchestra.run.gibba001.stdout:================================================================================ 2021-02-10T11:09:45.562 INFO:teuthology.orchestra.run.gibba001.stdout:Installing: 2021-02-10T11:09:45.562 INFO:teuthology.orchestra.run.gibba001.stdout: bison x86_64 3.0.4-2.el7 rhel-7-server-rpms 674 k 2021-02-10T11:09:45.562 INFO:teuthology.orchestra.run.gibba001.stdout: ceph x86_64 2:14.2.16-280.g962fac13.el7 ceph 2.0 k 2021-02-10T11:09:45.562 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-fuse x86_64 2:14.2.16-280.g962fac13.el7 ceph 571 k 2021-02-10T11:09:45.563 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-mgr x86_64 2:14.2.16-280.g962fac13.el7 ceph 1.8 M 2021-02-10T11:09:45.563 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-mgr-dashboard noarch 2:14.2.16-280.g962fac13.el7 ceph-noarch 4.0 M 2021-02-10T11:09:45.563 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-mgr-diskprediction-cloud 2021-02-10T11:09:45.563 INFO:teuthology.orchestra.run.gibba001.stdout: noarch 2:14.2.16-280.g962fac13.el7 ceph-noarch 83 k 2021-02-10T11:09:45.563 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-mgr-diskprediction-local 2021-02-10T11:09:45.563 INFO:teuthology.orchestra.run.gibba001.stdout: noarch 2:14.2.16-280.g962fac13.el7 ceph-noarch 1.1 M 2021-02-10T11:09:45.563 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-mgr-rook noarch 2:14.2.16-280.g962fac13.el7 ceph-noarch 21 k 2021-02-10T11:09:45.563 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-mgr-ssh noarch 2:14.2.16-280.g962fac13.el7 ceph-noarch 21 k 2021-02-10T11:09:45.563 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-radosgw x86_64 2:14.2.16-280.g962fac13.el7 ceph 5.8 M 2021-02-10T11:09:45.563 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-test x86_64 2:14.2.16-280.g962fac13.el7 ceph 34 M 2021-02-10T11:09:45.563 INFO:teuthology.orchestra.run.gibba001.stdout: elfutils-libelf-devel 2021-02-10T11:09:45.563 INFO:teuthology.orchestra.run.gibba001.stdout: x86_64 0.176-5.el7 rhel-7-server-rpms 40 k 2021-02-10T11:09:45.563 INFO:teuthology.orchestra.run.gibba001.stdout: flex x86_64 2.5.37-6.el7 rhel-7-server-rpms 293 k 2021-02-10T11:09:45.564 INFO:teuthology.orchestra.run.gibba001.stdout: libcephfs-devel x86_64 2:14.2.16-280.g962fac13.el7 ceph 17 k 2021-02-10T11:09:45.564 INFO:teuthology.orchestra.run.gibba001.stdout: libcephfs2 x86_64 2:14.2.16-280.g962fac13.el7 ceph 500 k 2021-02-10T11:09:45.564 INFO:teuthology.orchestra.run.gibba001.stdout: openssl-devel x86_64 1:1.0.2k-21.el7_9 rhel-7-server-rpms 1.5 M 2021-02-10T11:09:45.564 INFO:teuthology.orchestra.run.gibba001.stdout: python-ceph-compat x86_64 2:14.2.16-280.g962fac13.el7 ceph 2.2 k 2021-02-10T11:09:45.564 INFO:teuthology.orchestra.run.gibba001.stdout: python3-cephfs x86_64 2:14.2.16-280.g962fac13.el7 ceph 151 k 2021-02-10T11:09:45.564 INFO:teuthology.orchestra.run.gibba001.stdout: rbd-fuse x86_64 2:14.2.16-280.g962fac13.el7 ceph 69 k 2021-02-10T11:09:45.564 INFO:teuthology.orchestra.run.gibba001.stdout:Updating: 2021-02-10T11:09:45.564 INFO:teuthology.orchestra.run.gibba001.stdout: librados2 x86_64 2:14.2.16-280.g962fac13.el7 ceph 3.4 M 2021-02-10T11:09:45.564 INFO:teuthology.orchestra.run.gibba001.stdout: librbd1 x86_64 2:14.2.16-280.g962fac13.el7 ceph 1.7 M 2021-02-10T11:09:45.564 INFO:teuthology.orchestra.run.gibba001.stdout:Installing for dependencies: 2021-02-10T11:09:45.564 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-base x86_64 2:14.2.16-280.g962fac13.el7 ceph 5.4 M 2021-02-10T11:09:45.564 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-common x86_64 2:14.2.16-280.g962fac13.el7 ceph 19 M 2021-02-10T11:09:45.564 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-grafana-dashboards 2021-02-10T11:09:45.565 INFO:teuthology.orchestra.run.gibba001.stdout: noarch 2:14.2.16-280.g962fac13.el7 ceph-noarch 19 k 2021-02-10T11:09:45.565 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-mds x86_64 2:14.2.16-280.g962fac13.el7 ceph 1.9 M 2021-02-10T11:09:45.565 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-mon x86_64 2:14.2.16-280.g962fac13.el7 ceph 4.3 M 2021-02-10T11:09:45.565 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-osd x86_64 2:14.2.16-280.g962fac13.el7 ceph 16 M 2021-02-10T11:09:45.565 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-selinux x86_64 2:14.2.16-280.g962fac13.el7 ceph 20 k 2021-02-10T11:09:45.565 INFO:teuthology.orchestra.run.gibba001.stdout: jq x86_64 1.6-2.el7 epel 167 k 2021-02-10T11:09:45.565 INFO:teuthology.orchestra.run.gibba001.stdout: keyutils-libs-devel 2021-02-10T11:09:45.565 INFO:teuthology.orchestra.run.gibba001.stdout: x86_64 1.5.8-3.el7 rhel-7-server-rpms 37 k 2021-02-10T11:09:45.565 INFO:teuthology.orchestra.run.gibba001.stdout: krb5-devel x86_64 1.15.1-50.el7 rhel-7-server-rpms 273 k 2021-02-10T11:09:45.565 INFO:teuthology.orchestra.run.gibba001.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2021-02-10T11:09:45.565 INFO:teuthology.orchestra.run.gibba001.stdout: libcom_err-devel x86_64 1.42.9-19.el7 rhel-7-server-rpms 32 k 2021-02-10T11:09:45.565 INFO:teuthology.orchestra.run.gibba001.stdout: liboath x86_64 2.6.2-1.el7 epel 51 k 2021-02-10T11:09:45.565 INFO:teuthology.orchestra.run.gibba001.stdout: librabbitmq x86_64 0.8.0-3.el7 rhel-7-server-rpms 37 k 2021-02-10T11:09:45.566 INFO:teuthology.orchestra.run.gibba001.stdout: librados-devel x86_64 2:14.2.16-280.g962fac13.el7 ceph 86 k 2021-02-10T11:09:45.566 INFO:teuthology.orchestra.run.gibba001.stdout: libradosstriper1 x86_64 2:14.2.16-280.g962fac13.el7 ceph 364 k 2021-02-10T11:09:45.566 INFO:teuthology.orchestra.run.gibba001.stdout: librdkafka x86_64 0.11.5-1.el7 epel 368 k 2021-02-10T11:09:45.566 INFO:teuthology.orchestra.run.gibba001.stdout: librgw2 x86_64 2:14.2.16-280.g962fac13.el7 ceph 5.1 M 2021-02-10T11:09:45.566 INFO:teuthology.orchestra.run.gibba001.stdout: libselinux-devel x86_64 2.5-15.el7 rhel-7-server-rpms 187 k 2021-02-10T11:09:45.566 INFO:teuthology.orchestra.run.gibba001.stdout: libsepol-devel x86_64 2.5-10.el7 rhel-7-server-rpms 77 k 2021-02-10T11:09:45.566 INFO:teuthology.orchestra.run.gibba001.stdout: libverto-devel x86_64 0.2.5-4.el7 rhel-7-server-rpms 12 k 2021-02-10T11:09:45.566 INFO:teuthology.orchestra.run.gibba001.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2021-02-10T11:09:45.566 INFO:teuthology.orchestra.run.gibba001.stdout: numpy-f2py x86_64 1:1.7.1-13.el7 rhel-7-server-rpms 206 k 2021-02-10T11:09:45.566 INFO:teuthology.orchestra.run.gibba001.stdout: oniguruma x86_64 6.8.2-1.el7 epel 181 k 2021-02-10T11:09:45.566 INFO:teuthology.orchestra.run.gibba001.stdout: pcre-devel x86_64 8.32-17.el7 rhel-7-server-rpms 480 k 2021-02-10T11:09:45.566 INFO:teuthology.orchestra.run.gibba001.stdout: python-beaker noarch 1.5.4-10.el7 rhel-7-server-rpms 80 k 2021-02-10T11:09:45.566 INFO:teuthology.orchestra.run.gibba001.stdout: python-cachetools noarch 1.0.3-1.el7 epel 22 k 2021-02-10T11:09:45.567 INFO:teuthology.orchestra.run.gibba001.stdout: python-ceph-argparse 2021-02-10T11:09:45.567 INFO:teuthology.orchestra.run.gibba001.stdout: x86_64 2:14.2.16-280.g962fac13.el7 ceph 35 k 2021-02-10T11:09:45.567 INFO:teuthology.orchestra.run.gibba001.stdout: python-cephfs x86_64 2:14.2.16-280.g962fac13.el7 ceph 113 k 2021-02-10T11:09:45.567 INFO:teuthology.orchestra.run.gibba001.stdout: python-cherrypy noarch 3.2.2-4.el7 rhel-7-server-rpms 422 k 2021-02-10T11:09:45.567 INFO:teuthology.orchestra.run.gibba001.stdout: python-execnet noarch 1.7.1-3.el7 epel 260 k 2021-02-10T11:09:45.567 INFO:teuthology.orchestra.run.gibba001.stdout: python-jwcrypto noarch 0.4.2-1.el7 rhel-7-server-rpms 57 k 2021-02-10T11:09:45.567 INFO:teuthology.orchestra.run.gibba001.stdout: python-mako noarch 0.8.1-2.el7 rhel-7-server-rpms 307 k 2021-02-10T11:09:45.567 INFO:teuthology.orchestra.run.gibba001.stdout: python-markupsafe x86_64 0.11-10.el7 rhel-7-server-rpms 25 k 2021-02-10T11:09:45.567 INFO:teuthology.orchestra.run.gibba001.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 2021-02-10T11:09:45.567 INFO:teuthology.orchestra.run.gibba001.stdout: rhel-7-server-rpms 866 k 2021-02-10T11:09:45.567 INFO:teuthology.orchestra.run.gibba001.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2021-02-10T11:09:45.567 INFO:teuthology.orchestra.run.gibba001.stdout: python-prettytable noarch 0.7.2-3.el7 rhel-7-server-rpms 37 k 2021-02-10T11:09:45.567 INFO:teuthology.orchestra.run.gibba001.stdout: python-rados x86_64 2:14.2.16-280.g962fac13.el7 ceph 199 k 2021-02-10T11:09:45.567 INFO:teuthology.orchestra.run.gibba001.stdout: python-rbd x86_64 2:14.2.16-280.g962fac13.el7 ceph 194 k 2021-02-10T11:09:45.568 INFO:teuthology.orchestra.run.gibba001.stdout: python-repoze-lru noarch 0.4-3.el7 epel 13 k 2021-02-10T11:09:45.568 INFO:teuthology.orchestra.run.gibba001.stdout: python-requests noarch 2.6.0-10.el7 rhel-7-server-rpms 95 k 2021-02-10T11:09:45.568 INFO:teuthology.orchestra.run.gibba001.stdout: python-rgw x86_64 2:14.2.16-280.g962fac13.el7 ceph 77 k 2021-02-10T11:09:45.568 INFO:teuthology.orchestra.run.gibba001.stdout: python-routes noarch 1.13-2.el7 epel 640 k 2021-02-10T11:09:45.568 INFO:teuthology.orchestra.run.gibba001.stdout: python-simplegeneric 2021-02-10T11:09:45.568 INFO:teuthology.orchestra.run.gibba001.stdout: noarch 0.8-7.el7 epel 12 k 2021-02-10T11:09:45.568 INFO:teuthology.orchestra.run.gibba001.stdout: python-singledispatch 2021-02-10T11:09:45.568 INFO:teuthology.orchestra.run.gibba001.stdout: noarch 3.4.0.2-2.el7 epel 18 k 2021-02-10T11:09:45.568 INFO:teuthology.orchestra.run.gibba001.stdout: python-tempita noarch 0.5.1-6.el7 rhel-7-server-rpms 33 k 2021-02-10T11:09:45.568 INFO:teuthology.orchestra.run.gibba001.stdout: python-urllib3 noarch 1.10.2-7.el7 rhel-7-server-rpms 103 k 2021-02-10T11:09:45.568 INFO:teuthology.orchestra.run.gibba001.stdout: python-webob noarch 1.2.3-7.el7 rhel-7-server-optional-rpms 2021-02-10T11:09:45.568 INFO:teuthology.orchestra.run.gibba001.stdout: 202 k 2021-02-10T11:09:45.568 INFO:teuthology.orchestra.run.gibba001.stdout: python-websocket-client 2021-02-10T11:09:45.569 INFO:teuthology.orchestra.run.gibba001.stdout: noarch 0.56.0-3.git3c25814.el7 rhel-7-server-extras-rpms 2021-02-10T11:09:45.569 INFO:teuthology.orchestra.run.gibba001.stdout: 66 k 2021-02-10T11:09:45.569 INFO:teuthology.orchestra.run.gibba001.stdout: python-webtest noarch 1.3.4-6.el7 rhel-7-server-optional-rpms 2021-02-10T11:09:45.569 INFO:teuthology.orchestra.run.gibba001.stdout: 102 k 2021-02-10T11:09:45.569 INFO:teuthology.orchestra.run.gibba001.stdout: python-werkzeug noarch 0.9.1-2.el7 rhel-7-server-extras-rpms 2021-02-10T11:09:45.569 INFO:teuthology.orchestra.run.gibba001.stdout: 562 k 2021-02-10T11:09:45.569 INFO:teuthology.orchestra.run.gibba001.stdout: python2-apipkg noarch 1.5-5.el7 epel 17 k 2021-02-10T11:09:45.569 INFO:teuthology.orchestra.run.gibba001.stdout: python2-bcrypt x86_64 3.1.6-2.el7 epel 39 k 2021-02-10T11:09:45.569 INFO:teuthology.orchestra.run.gibba001.stdout: python2-certifi noarch 2018.10.15-5.el7 epel 12 k 2021-02-10T11:09:45.569 INFO:teuthology.orchestra.run.gibba001.stdout: python2-google-auth 2021-02-10T11:09:45.569 INFO:teuthology.orchestra.run.gibba001.stdout: noarch 1:1.1.1-5.el7 epel 76 k 2021-02-10T11:09:45.569 INFO:teuthology.orchestra.run.gibba001.stdout: python2-kubernetes noarch 1:11.0.0-6.el7 epel 1.1 M 2021-02-10T11:09:45.569 INFO:teuthology.orchestra.run.gibba001.stdout: python2-oauthlib noarch 2.0.1-8.el7 rhel-7-server-rpms 146 k 2021-02-10T11:09:45.570 INFO:teuthology.orchestra.run.gibba001.stdout: python2-pyasn1-modules 2021-02-10T11:09:45.570 INFO:teuthology.orchestra.run.gibba001.stdout: noarch 0.1.9-7.el7 rhel-7-server-optional-rpms 2021-02-10T11:09:45.570 INFO:teuthology.orchestra.run.gibba001.stdout: 59 k 2021-02-10T11:09:45.570 INFO:teuthology.orchestra.run.gibba001.stdout: python2-remoto noarch 1.0.0-1.el7 epel 42 k 2021-02-10T11:09:45.570 INFO:teuthology.orchestra.run.gibba001.stdout: python2-requests-oauthlib 2021-02-10T11:09:45.570 INFO:teuthology.orchestra.run.gibba001.stdout: noarch 0.8.0-5.el7 rhel-7-server-rpms 35 k 2021-02-10T11:09:45.570 INFO:teuthology.orchestra.run.gibba001.stdout: python2-rsa noarch 3.4.2-2.el7 epel 68 k 2021-02-10T11:09:45.570 INFO:teuthology.orchestra.run.gibba001.stdout: python2-six noarch 1.9.0-0.el7 epel 2.9 k 2021-02-10T11:09:45.570 INFO:teuthology.orchestra.run.gibba001.stdout: python3-ceph-argparse 2021-02-10T11:09:45.570 INFO:teuthology.orchestra.run.gibba001.stdout: x86_64 2:14.2.16-280.g962fac13.el7 ceph 35 k 2021-02-10T11:09:45.570 INFO:teuthology.orchestra.run.gibba001.stdout: python3-rados x86_64 2:14.2.16-280.g962fac13.el7 ceph 317 k 2021-02-10T11:09:45.570 INFO:teuthology.orchestra.run.gibba001.stdout: scipy x86_64 0.12.1-6.el7 rhel-7-server-rpms 9.3 M 2021-02-10T11:09:45.570 INFO:teuthology.orchestra.run.gibba001.stdout: socat x86_64 1.7.3.2-2.el7 rhel-7-server-rpms 290 k 2021-02-10T11:09:45.571 INFO:teuthology.orchestra.run.gibba001.stdout: suitesparse x86_64 4.0.2-10.el7 rhel-7-server-rpms 928 k 2021-02-10T11:09:45.571 INFO:teuthology.orchestra.run.gibba001.stdout: tbb x86_64 4.1-9.20130314.el7 rhel-7-server-rpms 124 k 2021-02-10T11:09:45.571 INFO:teuthology.orchestra.run.gibba001.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2021-02-10T11:09:45.571 INFO:teuthology.orchestra.run.gibba001.stdout: zlib-devel x86_64 1.2.7-19.el7_9 rhel-7-server-rpms 50 k 2021-02-10T11:09:45.571 INFO:teuthology.orchestra.run.gibba001.stdout:Updating for dependencies: 2021-02-10T11:09:45.571 INFO:teuthology.orchestra.run.gibba001.stdout: openssl x86_64 1:1.0.2k-21.el7_9 rhel-7-server-rpms 493 k 2021-02-10T11:09:45.571 INFO:teuthology.orchestra.run.gibba001.stdout: openssl-libs x86_64 1:1.0.2k-21.el7_9 rhel-7-server-rpms 1.2 M 2021-02-10T11:09:45.571 INFO:teuthology.orchestra.run.gibba001.stdout: zlib x86_64 1.2.7-19.el7_9 rhel-7-server-rpms 90 k 2021-02-10T11:09:45.571 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:09:45.571 INFO:teuthology.orchestra.run.gibba001.stdout:Transaction Summary 2021-02-10T11:09:45.571 INFO:teuthology.orchestra.run.gibba001.stdout:================================================================================ 2021-02-10T11:09:45.571 INFO:teuthology.orchestra.run.gibba001.stdout:Install 19 Packages (+70 Dependent packages) 2021-02-10T11:09:45.571 INFO:teuthology.orchestra.run.gibba001.stdout:Upgrade 2 Packages (+ 3 Dependent packages) 2021-02-10T11:09:45.572 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:09:45.572 INFO:teuthology.orchestra.run.gibba001.stdout:Total download size: 130 M 2021-02-10T11:09:45.572 INFO:teuthology.orchestra.run.gibba001.stdout:Downloading packages: 2021-02-10T11:09:45.572 INFO:teuthology.orchestra.run.gibba001.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2021-02-10T11:09:54.043 INFO:teuthology.orchestra.run.gibba001.stdout:-------------------------------------------------------------------------------- 2021-02-10T11:09:54.043 INFO:teuthology.orchestra.run.gibba001.stdout:Total 15 MB/s | 130 MB 00:08 2021-02-10T11:09:54.079 INFO:teuthology.orchestra.run.gibba001.stdout:Running transaction check 2021-02-10T11:09:54.210 INFO:teuthology.orchestra.run.gibba001.stdout:Running transaction test 2021-02-10T11:09:54.294 INFO:teuthology.orchestra.run.gibba001.stdout:Transaction test succeeded 2021-02-10T11:09:54.294 INFO:teuthology.orchestra.run.gibba001.stdout:Running transaction 2021-02-10T11:09:54.671 INFO:teuthology.orchestra.run.gibba008.stdout:-------------------------------------------------------------------------------- 2021-02-10T11:09:54.671 INFO:teuthology.orchestra.run.gibba008.stdout:Total 13 MB/s | 130 MB 00:10 2021-02-10T11:09:54.706 INFO:teuthology.orchestra.run.gibba008.stdout:Running transaction check 2021-02-10T11:09:54.836 INFO:teuthology.orchestra.run.gibba008.stdout:Running transaction test 2021-02-10T11:09:55.096 INFO:teuthology.orchestra.run.gibba001.stdout: Updating : zlib-1.2.7-19.el7_9.x86_64 1/99 2021-02-10T11:09:55.304 INFO:teuthology.orchestra.run.gibba008.stdout:Transaction test succeeded 2021-02-10T11:09:55.304 INFO:teuthology.orchestra.run.gibba008.stdout:Running transaction 2021-02-10T11:09:55.522 INFO:teuthology.orchestra.run.gibba001.stdout: Updating : 1:openssl-libs-1.0.2k-21.el7_9.x86_64 2/99 2021-02-10T11:09:55.872 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : liboath-2.6.2-1.el7.x86_64 3/99 2021-02-10T11:09:56.027 INFO:teuthology.orchestra.run.gibba008.stdout: Updating : zlib-1.2.7-19.el7_9.x86_64 1/99 2021-02-10T11:09:56.167 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : librabbitmq-0.8.0-3.el7.x86_64 4/99 2021-02-10T11:09:56.346 INFO:teuthology.orchestra.run.gibba008.stdout: Updating : 1:openssl-libs-1.0.2k-21.el7_9.x86_64 2/99 2021-02-10T11:09:56.523 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : zlib-devel-1.2.7-19.el7_9.x86_64 5/99 2021-02-10T11:09:56.614 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : liboath-2.6.2-1.el7.x86_64 3/99 2021-02-10T11:09:56.751 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python2-six-1.9.0-0.el7.noarch 6/99 2021-02-10T11:09:56.907 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : librabbitmq-0.8.0-3.el7.x86_64 4/99 2021-02-10T11:09:57.130 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:python-ceph-argparse-14.2.16-280.g962fac13.el7.x86_64 7/99 2021-02-10T11:09:57.162 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : zlib-devel-1.2.7-19.el7_9.x86_64 5/99 2021-02-10T11:09:57.359 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python2-six-1.9.0-0.el7.noarch 6/99 2021-02-10T11:09:57.522 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 8/99 2021-02-10T11:09:57.670 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:python-ceph-argparse-14.2.16-280.g962fac13.el7.x86_64 7/99 2021-02-10T11:09:57.939 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 9/99 2021-02-10T11:09:57.987 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 8/99 2021-02-10T11:09:58.299 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 9/99 2021-02-10T11:09:58.341 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python-webob-1.2.3-7.el7.noarch 10/99 2021-02-10T11:09:58.581 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python-webob-1.2.3-7.el7.noarch 10/99 2021-02-10T11:09:58.644 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 11/99 2021-02-10T11:09:58.868 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 11/99 2021-02-10T11:09:58.997 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python-requests-2.6.0-10.el7.noarch 12/99 2021-02-10T11:09:59.138 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python-requests-2.6.0-10.el7.noarch 12/99 2021-02-10T11:09:59.302 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 13/99 2021-02-10T11:09:59.442 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 13/99 2021-02-10T11:09:59.701 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python2-bcrypt-3.1.6-2.el7.x86_64 14/99 2021-02-10T11:09:59.722 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python2-bcrypt-3.1.6-2.el7.x86_64 14/99 2021-02-10T11:10:00.041 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 15/99 2021-02-10T11:10:00.101 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 15/99 2021-02-10T11:10:00.322 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : librdkafka-0.11.5-1.el7.x86_64 16/99 2021-02-10T11:10:00.550 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : librdkafka-0.11.5-1.el7.x86_64 16/99 2021-02-10T11:10:00.670 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : oniguruma-6.8.2-1.el7.x86_64 17/99 2021-02-10T11:10:00.927 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : jq-1.6-2.el7.x86_64 18/99 2021-02-10T11:10:00.930 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : oniguruma-6.8.2-1.el7.x86_64 17/99 2021-02-10T11:10:01.286 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : tbb-4.1-9.20130314.el7.x86_64 19/99 2021-02-10T11:10:01.303 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : jq-1.6-2.el7.x86_64 18/99 2021-02-10T11:10:01.610 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : suitesparse-4.0.2-10.el7.x86_64 20/99 2021-02-10T11:10:01.722 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : tbb-4.1-9.20130314.el7.x86_64 19/99 2021-02-10T11:10:01.991 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 21/99 2021-02-10T11:10:02.147 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : suitesparse-4.0.2-10.el7.x86_64 20/99 2021-02-10T11:10:02.309 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 22/99 2021-02-10T11:10:02.548 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 21/99 2021-02-10T11:10:02.583 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 23/99 2021-02-10T11:10:02.814 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python-cachetools-1.0.3-1.el7.noarch 24/99 2021-02-10T11:10:02.912 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 22/99 2021-02-10T11:10:03.056 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:ceph-grafana-dashboards-14.2.16-280.g962fac13.el7.noar 25/99 2021-02-10T11:10:03.185 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 23/99 2021-02-10T11:10:03.327 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 26/99 2021-02-10T11:10:03.501 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python-cachetools-1.0.3-1.el7.noarch 24/99 2021-02-10T11:10:03.573 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 27/99 2021-02-10T11:10:03.834 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:ceph-grafana-dashboards-14.2.16-280.g962fac13.el7.noar 25/99 2021-02-10T11:10:03.856 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : libcom_err-devel-1.42.9-19.el7.x86_64 28/99 2021-02-10T11:10:04.059 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:python3-ceph-argparse-14.2.16-280.g962fac13.el7.x86_64 29/99 2021-02-10T11:10:04.202 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 26/99 2021-02-10T11:10:04.415 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python2-certifi-2018.10.15-5.el7.noarch 30/99 2021-02-10T11:10:04.561 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 27/99 2021-02-10T11:10:04.698 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : pcre-devel-8.32-17.el7.x86_64 31/99 2021-02-10T11:10:04.887 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : libcom_err-devel-1.42.9-19.el7.x86_64 28/99 2021-02-10T11:10:04.968 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 32/99 2021-02-10T11:10:05.188 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:python3-ceph-argparse-14.2.16-280.g962fac13.el7.x86_64 29/99 2021-02-10T11:10:05.195 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python2-pyasn1-modules-0.1.9-7.el7.noarch 33/99 2021-02-10T11:10:05.437 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python-jwcrypto-0.4.2-1.el7.noarch 34/99 2021-02-10T11:10:05.542 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python2-certifi-2018.10.15-5.el7.noarch 30/99 2021-02-10T11:10:05.708 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python2-oauthlib-2.0.1-8.el7.noarch 35/99 2021-02-10T11:10:05.936 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : pcre-devel-8.32-17.el7.x86_64 31/99 2021-02-10T11:10:05.941 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python2-requests-oauthlib-0.8.0-5.el7.noarch 36/99 2021-02-10T11:10:06.205 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 32/99 2021-02-10T11:10:06.272 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python2-rsa-3.4.2-2.el7.noarch 37/99 2021-02-10T11:10:06.515 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python2-pyasn1-modules-0.1.9-7.el7.noarch 33/99 2021-02-10T11:10:06.892 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python-jwcrypto-0.4.2-1.el7.noarch 34/99 2021-02-10T11:10:07.179 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 1:python2-google-auth-1.1.1-5.el7.noarch 38/99 2021-02-10T11:10:07.246 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python2-oauthlib-2.0.1-8.el7.noarch 35/99 2021-02-10T11:10:07.473 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 1:python2-kubernetes-11.0.0-6.el7.noarch 39/99 2021-02-10T11:10:07.570 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python2-requests-oauthlib-0.8.0-5.el7.noarch 36/99 2021-02-10T11:10:07.684 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 40/99 2021-02-10T11:10:07.898 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : keyutils-libs-devel-1.5.8-3.el7.x86_64 41/99 2021-02-10T11:10:07.960 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python2-rsa-3.4.2-2.el7.noarch 37/99 2021-02-10T11:10:08.163 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 42/99 2021-02-10T11:10:08.481 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 43/99 2021-02-10T11:10:08.903 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 1:python2-google-auth-1.1.1-5.el7.noarch 38/99 2021-02-10T11:10:09.050 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 44/99 2021-02-10T11:10:09.227 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 1:python2-kubernetes-11.0.0-6.el7.noarch 39/99 2021-02-10T11:10:09.384 INFO:teuthology.orchestra.run.gibba008.stdout: Updating : 2:librados2-14.2.16-280.g962fac13.el7.x86_64 45/99 2021-02-10T11:10:09.538 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 40/99 2021-02-10T11:10:09.643 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:libcephfs2-14.2.16-280.g962fac13.el7.x86_64 46/99 2021-02-10T11:10:09.832 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : keyutils-libs-devel-1.5.8-3.el7.x86_64 41/99 2021-02-10T11:10:10.012 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:python-rados-14.2.16-280.g962fac13.el7.x86_64 47/99 2021-02-10T11:10:10.130 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 42/99 2021-02-10T11:10:10.541 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 43/99 2021-02-10T11:10:10.588 INFO:teuthology.orchestra.run.gibba008.stdout: Updating : 2:librbd1-14.2.16-280.g962fac13.el7.x86_64 48/99 2021-02-10T11:10:10.836 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:librgw2-14.2.16-280.g962fac13.el7.x86_64 49/99 2021-02-10T11:10:11.122 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:python-rgw-14.2.16-280.g962fac13.el7.x86_64 50/99 2021-02-10T11:10:11.157 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 44/99 2021-02-10T11:10:11.391 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:python-rbd-14.2.16-280.g962fac13.el7.x86_64 51/99 2021-02-10T11:10:11.559 INFO:teuthology.orchestra.run.gibba001.stdout: Updating : 2:librados2-14.2.16-280.g962fac13.el7.x86_64 45/99 2021-02-10T11:10:11.707 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:python-cephfs-14.2.16-280.g962fac13.el7.x86_64 52/99 2021-02-10T11:10:11.918 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:libcephfs2-14.2.16-280.g962fac13.el7.x86_64 46/99 2021-02-10T11:10:12.338 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:python-rados-14.2.16-280.g962fac13.el7.x86_64 47/99 2021-02-10T11:10:12.989 INFO:teuthology.orchestra.run.gibba001.stdout: Updating : 2:librbd1-14.2.16-280.g962fac13.el7.x86_64 48/99 2021-02-10T11:10:13.304 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:librgw2-14.2.16-280.g962fac13.el7.x86_64 49/99 2021-02-10T11:10:13.514 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:libradosstriper1-14.2.16-280.g962fac13.el7.x86_64 53/99 2021-02-10T11:10:13.597 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:python-rgw-14.2.16-280.g962fac13.el7.x86_64 50/99 2021-02-10T11:10:13.917 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:python-rbd-14.2.16-280.g962fac13.el7.x86_64 51/99 2021-02-10T11:10:14.249 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:ceph-common-14.2.16-280.g962fac13.el7.x86_64 54/99 2021-02-10T11:10:14.333 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:python-cephfs-14.2.16-280.g962fac13.el7.x86_64 52/99 2021-02-10T11:10:14.681 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:ceph-base-14.2.16-280.g962fac13.el7.x86_64 55/99 2021-02-10T11:10:16.223 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:libradosstriper1-14.2.16-280.g962fac13.el7.x86_64 53/99 2021-02-10T11:10:17.065 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:ceph-common-14.2.16-280.g962fac13.el7.x86_64 54/99 2021-02-10T11:10:17.512 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:ceph-base-14.2.16-280.g962fac13.el7.x86_64 55/99 2021-02-10T11:10:21.978 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:ceph-selinux-14.2.16-280.g962fac13.el7.x86_64 56/99 2021-02-10T11:10:22.413 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:ceph-osd-14.2.16-280.g962fac13.el7.x86_64 57/99 2021-02-10T11:10:22.984 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:ceph-mds-14.2.16-280.g962fac13.el7.x86_64 58/99 2021-02-10T11:10:23.270 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:ceph-mon-14.2.16-280.g962fac13.el7.x86_64 59/99 2021-02-10T11:10:23.561 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:librados-devel-14.2.16-280.g962fac13.el7.x86_64 60/99 2021-02-10T11:10:23.792 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:python3-rados-14.2.16-280.g962fac13.el7.x86_64 61/99 2021-02-10T11:10:24.120 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python-repoze-lru-0.4-3.el7.noarch 62/99 2021-02-10T11:10:24.870 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:ceph-selinux-14.2.16-280.g962fac13.el7.x86_64 56/99 2021-02-10T11:10:25.264 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python-routes-1.13-2.el7.noarch 63/99 2021-02-10T11:10:25.389 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:ceph-osd-14.2.16-280.g962fac13.el7.x86_64 57/99 2021-02-10T11:10:25.560 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 64/99 2021-02-10T11:10:25.884 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python-mako-0.8.1-2.el7.noarch 65/99 2021-02-10T11:10:26.353 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 66/99 2021-02-10T11:10:26.584 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:ceph-mds-14.2.16-280.g962fac13.el7.x86_64 58/99 2021-02-10T11:10:26.735 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:ceph-mgr-14.2.16-280.g962fac13.el7.x86_64 67/99 2021-02-10T11:10:26.912 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:ceph-mon-14.2.16-280.g962fac13.el7.x86_64 59/99 2021-02-10T11:10:27.066 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python2-apipkg-1.5-5.el7.noarch 68/99 2021-02-10T11:10:27.245 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:librados-devel-14.2.16-280.g962fac13.el7.x86_64 60/99 2021-02-10T11:10:27.413 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python-execnet-1.7.1-3.el7.noarch 69/99 2021-02-10T11:10:27.509 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:python3-rados-14.2.16-280.g962fac13.el7.x86_64 61/99 2021-02-10T11:10:27.729 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : python2-remoto-1.0.0-1.el7.noarch 70/99 2021-02-10T11:10:27.897 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python-repoze-lru-0.4-3.el7.noarch 62/99 2021-02-10T11:10:27.956 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : libverto-devel-0.2.5-4.el7.x86_64 71/99 2021-02-10T11:10:28.240 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python-routes-1.13-2.el7.noarch 63/99 2021-02-10T11:10:28.249 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : libsepol-devel-2.5-10.el7.x86_64 72/99 2021-02-10T11:10:28.580 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : libselinux-devel-2.5-15.el7.x86_64 73/99 2021-02-10T11:10:28.635 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 64/99 2021-02-10T11:10:28.886 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : krb5-devel-1.15.1-50.el7.x86_64 74/99 2021-02-10T11:10:29.051 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python-mako-0.8.1-2.el7.noarch 65/99 2021-02-10T11:10:29.651 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 66/99 2021-02-10T11:10:30.103 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:ceph-mgr-14.2.16-280.g962fac13.el7.x86_64 67/99 2021-02-10T11:10:30.144 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 1:numpy-f2py-1.7.1-13.el7.x86_64 75/99 2021-02-10T11:10:30.496 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python2-apipkg-1.5-5.el7.noarch 68/99 2021-02-10T11:10:30.792 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : scipy-0.12.1-6.el7.x86_64 76/99 2021-02-10T11:10:31.014 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python-execnet-1.7.1-3.el7.noarch 69/99 2021-02-10T11:10:31.311 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:ceph-mgr-diskprediction-local-14.2.16-280.g962fac13.el 77/99 2021-02-10T11:10:31.412 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : python2-remoto-1.0.0-1.el7.noarch 70/99 2021-02-10T11:10:31.643 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 1:openssl-devel-1.0.2k-21.el7_9.x86_64 78/99 2021-02-10T11:10:31.688 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : libverto-devel-0.2.5-4.el7.x86_64 71/99 2021-02-10T11:10:31.901 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:ceph-mgr-ssh-14.2.16-280.g962fac13.el7.noarch 79/99 2021-02-10T11:10:32.051 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : libsepol-devel-2.5-10.el7.x86_64 72/99 2021-02-10T11:10:32.199 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:ceph-mgr-diskprediction-cloud-14.2.16-280.g962fac13.el 80/99 2021-02-10T11:10:32.411 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:ceph-14.2.16-280.g962fac13.el7.x86_64 81/99 2021-02-10T11:10:32.437 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : libselinux-devel-2.5-15.el7.x86_64 73/99 2021-02-10T11:10:32.855 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : krb5-devel-1.15.1-50.el7.x86_64 74/99 2021-02-10T11:10:33.174 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:ceph-mgr-rook-14.2.16-280.g962fac13.el7.noarch 82/99 2021-02-10T11:10:33.568 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:ceph-mgr-dashboard-14.2.16-280.g962fac13.el7.noarch 83/99 2021-02-10T11:10:33.843 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:python3-cephfs-14.2.16-280.g962fac13.el7.x86_64 84/99 2021-02-10T11:10:34.169 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 1:numpy-f2py-1.7.1-13.el7.x86_64 75/99 2021-02-10T11:10:34.504 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:libcephfs-devel-14.2.16-280.g962fac13.el7.x86_64 85/99 2021-02-10T11:10:34.835 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : scipy-0.12.1-6.el7.x86_64 76/99 2021-02-10T11:10:35.392 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:ceph-mgr-diskprediction-local-14.2.16-280.g962fac13.el 77/99 2021-02-10T11:10:35.870 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 1:openssl-devel-1.0.2k-21.el7_9.x86_64 78/99 2021-02-10T11:10:36.170 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:ceph-mgr-ssh-14.2.16-280.g962fac13.el7.noarch 79/99 2021-02-10T11:10:36.501 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:ceph-mgr-diskprediction-cloud-14.2.16-280.g962fac13.el 80/99 2021-02-10T11:10:36.738 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:ceph-14.2.16-280.g962fac13.el7.x86_64 81/99 2021-02-10T11:10:37.377 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:ceph-radosgw-14.2.16-280.g962fac13.el7.x86_64 86/99 2021-02-10T11:10:37.560 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:ceph-mgr-rook-14.2.16-280.g962fac13.el7.noarch 82/99 2021-02-10T11:10:37.633 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:ceph-test-14.2.16-280.g962fac13.el7.x86_64 87/99 2021-02-10T11:10:37.816 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:python-ceph-compat-14.2.16-280.g962fac13.el7.x86_64 88/99 2021-02-10T11:10:37.988 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:ceph-mgr-dashboard-14.2.16-280.g962fac13.el7.noarch 83/99 2021-02-10T11:10:38.060 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:rbd-fuse-14.2.16-280.g962fac13.el7.x86_64 89/99 2021-02-10T11:10:38.269 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:python3-cephfs-14.2.16-280.g962fac13.el7.x86_64 84/99 2021-02-10T11:10:38.307 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : 2:ceph-fuse-14.2.16-280.g962fac13.el7.x86_64 90/99 2021-02-10T11:10:38.624 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : elfutils-libelf-devel-0.176-5.el7.x86_64 91/99 2021-02-10T11:10:38.973 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:libcephfs-devel-14.2.16-280.g962fac13.el7.x86_64 85/99 2021-02-10T11:10:39.091 INFO:teuthology.orchestra.run.gibba008.stdout: Updating : 1:openssl-1.0.2k-21.el7_9.x86_64 92/99 2021-02-10T11:10:39.427 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : bison-3.0.4-2.el7.x86_64 93/99 2021-02-10T11:10:39.670 INFO:teuthology.orchestra.run.gibba008.stdout: Installing : flex-2.5.37-6.el7.x86_64 94/99 2021-02-10T11:10:40.233 INFO:teuthology.orchestra.run.gibba008.stdout: Cleanup : 1:openssl-1.0.2k-19.el7.x86_64 95/99 2021-02-10T11:10:40.517 INFO:teuthology.orchestra.run.gibba008.stdout: Cleanup : 1:openssl-libs-1.0.2k-19.el7.x86_64 96/99 2021-02-10T11:10:40.799 INFO:teuthology.orchestra.run.gibba008.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 97/99 2021-02-10T11:10:41.050 INFO:teuthology.orchestra.run.gibba008.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 98/99 2021-02-10T11:10:41.456 INFO:teuthology.orchestra.run.gibba008.stdout: Cleanup : zlib-1.2.7-18.el7.x86_64 99/99 2021-02-10T11:10:41.456 INFO:teuthology.orchestra.run.gibba008.stdout:Loaded plugins: fastestmirror, langpacks, priorities, product-id, subscription- 2021-02-10T11:10:41.457 INFO:teuthology.orchestra.run.gibba008.stdout: : manager 2021-02-10T11:10:41.790 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:ceph-radosgw-14.2.16-280.g962fac13.el7.x86_64 86/99 2021-02-10T11:10:42.109 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:libradosstriper1-14.2.16-280.g962fac13.el7.x86_64 1/99 2021-02-10T11:10:42.141 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:ceph-test-14.2.16-280.g962fac13.el7.x86_64 87/99 2021-02-10T11:10:42.201 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:ceph-selinux-14.2.16-280.g962fac13.el7.x86_64 2/99 2021-02-10T11:10:42.293 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:python-rgw-14.2.16-280.g962fac13.el7.x86_64 3/99 2021-02-10T11:10:42.385 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : suitesparse-4.0.2-10.el7.x86_64 4/99 2021-02-10T11:10:42.438 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:python-ceph-compat-14.2.16-280.g962fac13.el7.x86_64 88/99 2021-02-10T11:10:42.469 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : jq-1.6-2.el7.x86_64 5/99 2021-02-10T11:10:42.569 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python-routes-1.13-2.el7.noarch 6/99 2021-02-10T11:10:42.695 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 1:numpy-f2py-1.7.1-13.el7.x86_64 7/99 2021-02-10T11:10:42.747 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:rbd-fuse-14.2.16-280.g962fac13.el7.x86_64 89/99 2021-02-10T11:10:42.804 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 8/99 2021-02-10T11:10:42.921 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : libsepol-devel-2.5-10.el7.x86_64 9/99 2021-02-10T11:10:43.038 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 10/99 2021-02-10T11:10:43.045 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : 2:ceph-fuse-14.2.16-280.g962fac13.el7.x86_64 90/99 2021-02-10T11:10:43.147 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : libverto-devel-0.2.5-4.el7.x86_64 11/99 2021-02-10T11:10:43.272 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:libcephfs2-14.2.16-280.g962fac13.el7.x86_64 12/99 2021-02-10T11:10:43.381 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:python-rbd-14.2.16-280.g962fac13.el7.x86_64 13/99 2021-02-10T11:10:43.383 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : elfutils-libelf-devel-0.176-5.el7.x86_64 91/99 2021-02-10T11:10:43.498 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 1:openssl-1.0.2k-21.el7_9.x86_64 14/99 2021-02-10T11:10:43.607 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python2-apipkg-1.5-5.el7.noarch 15/99 2021-02-10T11:10:43.716 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 16/99 2021-02-10T11:10:43.808 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 17/99 2021-02-10T11:10:43.812 INFO:teuthology.orchestra.run.gibba001.stdout: Updating : 1:openssl-1.0.2k-21.el7_9.x86_64 92/99 2021-02-10T11:10:43.925 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python-repoze-lru-0.4-3.el7.noarch 18/99 2021-02-10T11:10:44.017 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 1:openssl-libs-1.0.2k-21.el7_9.x86_64 19/99 2021-02-10T11:10:44.118 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python-requests-2.6.0-10.el7.noarch 20/99 2021-02-10T11:10:44.210 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : bison-3.0.4-2.el7.x86_64 93/99 2021-02-10T11:10:44.211 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 21/99 2021-02-10T11:10:44.302 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 22/99 2021-02-10T11:10:44.403 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 1:python2-kubernetes-11.0.0-6.el7.noarch 23/99 2021-02-10T11:10:44.514 INFO:teuthology.orchestra.run.gibba001.stdout: Installing : flex-2.5.37-6.el7.x86_64 94/99 2021-02-10T11:10:44.536 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 24/99 2021-02-10T11:10:44.628 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:ceph-osd-14.2.16-280.g962fac13.el7.x86_64 25/99 2021-02-10T11:10:44.729 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 26/99 2021-02-10T11:10:44.832 INFO:teuthology.orchestra.run.gibba001.stdout: Cleanup : 1:openssl-1.0.2k-19.el7.x86_64 95/99 2021-02-10T11:10:45.225 INFO:teuthology.orchestra.run.gibba001.stdout: Cleanup : 1:openssl-libs-1.0.2k-19.el7.x86_64 96/99 2021-02-10T11:10:45.541 INFO:teuthology.orchestra.run.gibba001.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 97/99 2021-02-10T11:10:45.708 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python2-remoto-1.0.0-1.el7.noarch 27/99 2021-02-10T11:10:45.808 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 28/99 2021-02-10T11:10:45.908 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : keyutils-libs-devel-1.5.8-3.el7.x86_64 29/99 2021-02-10T11:10:46.034 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:ceph-mgr-diskprediction-cloud-14.2.16-280.g962fac13.el 30/99 2021-02-10T11:10:46.118 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:ceph-fuse-14.2.16-280.g962fac13.el7.x86_64 31/99 2021-02-10T11:10:46.218 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:ceph-14.2.16-280.g962fac13.el7.x86_64 32/99 2021-02-10T11:10:46.294 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 33/99 2021-02-10T11:10:46.394 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python2-requests-oauthlib-0.8.0-5.el7.noarch 34/99 2021-02-10T11:10:46.424 INFO:teuthology.orchestra.run.gibba001.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 98/99 2021-02-10T11:10:46.494 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:ceph-mds-14.2.16-280.g962fac13.el7.x86_64 35/99 2021-02-10T11:10:46.587 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 36/99 2021-02-10T11:10:46.704 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python2-rsa-3.4.2-2.el7.noarch 37/99 2021-02-10T11:10:46.812 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:ceph-mgr-ssh-14.2.16-280.g962fac13.el7.noarch 38/99 2021-02-10T11:10:46.869 INFO:teuthology.orchestra.run.gibba001.stdout: Cleanup : zlib-1.2.7-18.el7.x86_64 99/99 2021-02-10T11:10:46.869 INFO:teuthology.orchestra.run.gibba001.stdout:Loaded plugins: fastestmirror, langpacks, priorities, product-id, subscription- 2021-02-10T11:10:46.869 INFO:teuthology.orchestra.run.gibba001.stdout: : manager 2021-02-10T11:10:46.929 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python-jwcrypto-0.4.2-1.el7.noarch 39/99 2021-02-10T11:10:47.046 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 40/99 2021-02-10T11:10:47.156 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : elfutils-libelf-devel-0.176-5.el7.x86_64 41/99 2021-02-10T11:10:47.289 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python2-pyasn1-modules-0.1.9-7.el7.noarch 42/99 2021-02-10T11:10:47.398 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 43/99 2021-02-10T11:10:47.507 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : librabbitmq-0.8.0-3.el7.x86_64 44/99 2021-02-10T11:10:47.603 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:libradosstriper1-14.2.16-280.g962fac13.el7.x86_64 1/99 2021-02-10T11:10:47.616 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:ceph-base-14.2.16-280.g962fac13.el7.x86_64 45/99 2021-02-10T11:10:47.708 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : scipy-0.12.1-6.el7.x86_64 46/99 2021-02-10T11:10:47.736 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:ceph-selinux-14.2.16-280.g962fac13.el7.x86_64 2/99 2021-02-10T11:10:47.800 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:python-ceph-argparse-14.2.16-280.g962fac13.el7.x86_64 47/99 2021-02-10T11:10:47.837 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:python-rgw-14.2.16-280.g962fac13.el7.x86_64 3/99 2021-02-10T11:10:47.909 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : libselinux-devel-2.5-15.el7.x86_64 48/99 2021-02-10T11:10:47.979 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : suitesparse-4.0.2-10.el7.x86_64 4/99 2021-02-10T11:10:48.001 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : zlib-1.2.7-19.el7_9.x86_64 49/99 2021-02-10T11:10:48.102 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python2-oauthlib-2.0.1-8.el7.noarch 50/99 2021-02-10T11:10:48.113 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : jq-1.6-2.el7.x86_64 5/99 2021-02-10T11:10:48.194 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : pcre-devel-8.32-17.el7.x86_64 51/99 2021-02-10T11:10:48.221 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python-routes-1.13-2.el7.noarch 6/99 2021-02-10T11:10:48.294 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:ceph-radosgw-14.2.16-280.g962fac13.el7.x86_64 52/99 2021-02-10T11:10:48.356 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 1:numpy-f2py-1.7.1-13.el7.x86_64 7/99 2021-02-10T11:10:48.369 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:python-cephfs-14.2.16-280.g962fac13.el7.x86_64 53/99 2021-02-10T11:10:48.445 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:libcephfs-devel-14.2.16-280.g962fac13.el7.x86_64 54/99 2021-02-10T11:10:48.508 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 8/99 2021-02-10T11:10:48.596 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:python-rados-14.2.16-280.g962fac13.el7.x86_64 55/99 2021-02-10T11:10:48.627 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : libsepol-devel-2.5-10.el7.x86_64 9/99 2021-02-10T11:10:48.713 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:librados2-14.2.16-280.g962fac13.el7.x86_64 56/99 2021-02-10T11:10:48.765 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 10/99 2021-02-10T11:10:48.830 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : librdkafka-0.11.5-1.el7.x86_64 57/99 2021-02-10T11:10:48.907 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : libverto-devel-0.2.5-4.el7.x86_64 11/99 2021-02-10T11:10:48.947 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : flex-2.5.37-6.el7.x86_64 58/99 2021-02-10T11:10:49.016 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:libcephfs2-14.2.16-280.g962fac13.el7.x86_64 12/99 2021-02-10T11:10:49.057 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python2-certifi-2018.10.15-5.el7.noarch 59/99 2021-02-10T11:10:49.150 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:python-rbd-14.2.16-280.g962fac13.el7.x86_64 13/99 2021-02-10T11:10:49.193 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:python3-ceph-argparse-14.2.16-280.g962fac13.el7.x86_64 60/99 2021-02-10T11:10:49.301 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 1:openssl-1.0.2k-21.el7_9.x86_64 14/99 2021-02-10T11:10:49.307 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:ceph-mgr-14.2.16-280.g962fac13.el7.x86_64 61/99 2021-02-10T11:10:49.409 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python2-apipkg-1.5-5.el7.noarch 15/99 2021-02-10T11:10:49.424 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:librgw2-14.2.16-280.g962fac13.el7.x86_64 62/99 2021-02-10T11:10:49.525 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : krb5-devel-1.15.1-50.el7.x86_64 63/99 2021-02-10T11:10:49.551 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 16/99 2021-02-10T11:10:49.617 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:ceph-test-14.2.16-280.g962fac13.el7.x86_64 64/99 2021-02-10T11:10:49.685 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 17/99 2021-02-10T11:10:49.709 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 65/99 2021-02-10T11:10:49.777 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python-repoze-lru-0.4-3.el7.noarch 18/99 2021-02-10T11:10:49.826 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : libcom_err-devel-1.42.9-19.el7.x86_64 66/99 2021-02-10T11:10:49.902 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 1:openssl-libs-1.0.2k-21.el7_9.x86_64 19/99 2021-02-10T11:10:49.927 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:rbd-fuse-14.2.16-280.g962fac13.el7.x86_64 67/99 2021-02-10T11:10:50.004 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python-requests-2.6.0-10.el7.noarch 20/99 2021-02-10T11:10:50.027 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:librbd1-14.2.16-280.g962fac13.el7.x86_64 68/99 2021-02-10T11:10:50.120 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : liboath-2.6.2-1.el7.x86_64 69/99 2021-02-10T11:10:50.146 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 21/99 2021-02-10T11:10:50.212 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python2-six-1.9.0-0.el7.noarch 70/99 2021-02-10T11:10:50.288 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 22/99 2021-02-10T11:10:50.304 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:ceph-mgr-diskprediction-local-14.2.16-280.g962fac13.el 71/99 2021-02-10T11:10:50.430 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 1:python2-kubernetes-11.0.0-6.el7.noarch 23/99 2021-02-10T11:10:50.438 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python2-bcrypt-3.1.6-2.el7.x86_64 72/99 2021-02-10T11:10:50.530 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 24/99 2021-02-10T11:10:50.563 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 73/99 2021-02-10T11:10:50.664 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:ceph-osd-14.2.16-280.g962fac13.el7.x86_64 25/99 2021-02-10T11:10:50.680 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:ceph-mgr-rook-14.2.16-280.g962fac13.el7.noarch 74/99 2021-02-10T11:10:50.765 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 26/99 2021-02-10T11:10:50.789 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:librados-devel-14.2.16-280.g962fac13.el7.x86_64 75/99 2021-02-10T11:10:50.898 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 76/99 2021-02-10T11:10:51.041 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:ceph-mgr-dashboard-14.2.16-280.g962fac13.el7.noarch 77/99 2021-02-10T11:10:51.157 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:python3-rados-14.2.16-280.g962fac13.el7.x86_64 78/99 2021-02-10T11:10:51.258 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 79/99 2021-02-10T11:10:51.275 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python2-remoto-1.0.0-1.el7.noarch 27/99 2021-02-10T11:10:51.350 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:python-ceph-compat-14.2.16-280.g962fac13.el7.x86_64 80/99 2021-02-10T11:10:51.426 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 28/99 2021-02-10T11:10:51.442 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:ceph-common-14.2.16-280.g962fac13.el7.x86_64 81/99 2021-02-10T11:10:51.543 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:python3-cephfs-14.2.16-280.g962fac13.el7.x86_64 82/99 2021-02-10T11:10:51.560 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : keyutils-libs-devel-1.5.8-3.el7.x86_64 29/99 2021-02-10T11:10:51.652 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:ceph-mgr-diskprediction-cloud-14.2.16-280.g962fac13.el 30/99 2021-02-10T11:10:51.668 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:ceph-grafana-dashboards-14.2.16-280.g962fac13.el7.noar 83/99 2021-02-10T11:10:51.761 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:ceph-fuse-14.2.16-280.g962fac13.el7.x86_64 31/99 2021-02-10T11:10:51.769 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python-execnet-1.7.1-3.el7.noarch 84/99 2021-02-10T11:10:51.853 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:ceph-14.2.16-280.g962fac13.el7.x86_64 32/99 2021-02-10T11:10:51.861 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 85/99 2021-02-10T11:10:51.961 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 2:ceph-mon-14.2.16-280.g962fac13.el7.x86_64 86/99 2021-02-10T11:10:51.962 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 33/99 2021-02-10T11:10:52.053 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : bison-3.0.4-2.el7.x86_64 87/99 2021-02-10T11:10:52.071 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python2-requests-oauthlib-0.8.0-5.el7.noarch 34/99 2021-02-10T11:10:52.154 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 1:openssl-devel-1.0.2k-21.el7_9.x86_64 88/99 2021-02-10T11:10:52.213 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:ceph-mds-14.2.16-280.g962fac13.el7.x86_64 35/99 2021-02-10T11:10:52.288 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python-cachetools-1.0.3-1.el7.noarch 89/99 2021-02-10T11:10:52.347 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 36/99 2021-02-10T11:10:52.396 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 90/99 2021-02-10T11:10:52.447 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python2-rsa-3.4.2-2.el7.noarch 37/99 2021-02-10T11:10:52.514 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : zlib-devel-1.2.7-19.el7_9.x86_64 91/99 2021-02-10T11:10:52.581 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:ceph-mgr-ssh-14.2.16-280.g962fac13.el7.noarch 38/99 2021-02-10T11:10:52.622 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 1:python2-google-auth-1.1.1-5.el7.noarch 92/99 2021-02-10T11:10:52.731 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : tbb-4.1-9.20130314.el7.x86_64 93/99 2021-02-10T11:10:52.731 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python-jwcrypto-0.4.2-1.el7.noarch 39/99 2021-02-10T11:10:52.839 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : oniguruma-6.8.2-1.el7.x86_64 94/99 2021-02-10T11:10:52.840 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 95/99 2021-02-10T11:10:52.840 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : zlib-1.2.7-18.el7.x86_64 96/99 2021-02-10T11:10:52.841 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 1:openssl-libs-1.0.2k-19.el7.x86_64 97/99 2021-02-10T11:10:52.841 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 1:openssl-1.0.2k-19.el7.x86_64 98/99 2021-02-10T11:10:52.871 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 40/99 2021-02-10T11:10:52.991 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : elfutils-libelf-devel-0.176-5.el7.x86_64 41/99 2021-02-10T11:10:53.092 INFO:teuthology.orchestra.run.gibba008.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 99/99 2021-02-10T11:10:53.092 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:10:53.092 INFO:teuthology.orchestra.run.gibba008.stdout:Installed: 2021-02-10T11:10:53.092 INFO:teuthology.orchestra.run.gibba008.stdout: bison.x86_64 0:3.0.4-2.el7 2021-02-10T11:10:53.092 INFO:teuthology.orchestra.run.gibba008.stdout: ceph.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.092 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-fuse.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.092 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-mgr.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.092 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-mgr-dashboard.noarch 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.092 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-mgr-diskprediction-cloud.noarch 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.092 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-mgr-diskprediction-local.noarch 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.093 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-mgr-rook.noarch 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.093 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-mgr-ssh.noarch 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.093 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-radosgw.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.093 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-test.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.093 INFO:teuthology.orchestra.run.gibba008.stdout: elfutils-libelf-devel.x86_64 0:0.176-5.el7 2021-02-10T11:10:53.093 INFO:teuthology.orchestra.run.gibba008.stdout: flex.x86_64 0:2.5.37-6.el7 2021-02-10T11:10:53.093 INFO:teuthology.orchestra.run.gibba008.stdout: libcephfs-devel.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.093 INFO:teuthology.orchestra.run.gibba008.stdout: libcephfs2.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.094 INFO:teuthology.orchestra.run.gibba008.stdout: openssl-devel.x86_64 1:1.0.2k-21.el7_9 2021-02-10T11:10:53.094 INFO:teuthology.orchestra.run.gibba008.stdout: python-ceph-compat.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.094 INFO:teuthology.orchestra.run.gibba008.stdout: python3-cephfs.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.094 INFO:teuthology.orchestra.run.gibba008.stdout: rbd-fuse.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.094 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:10:53.094 INFO:teuthology.orchestra.run.gibba008.stdout:Dependency Installed: 2021-02-10T11:10:53.094 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-base.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.094 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-common.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.094 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-grafana-dashboards.noarch 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.094 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-mds.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.094 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-mon.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.094 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-osd.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.095 INFO:teuthology.orchestra.run.gibba008.stdout: ceph-selinux.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.095 INFO:teuthology.orchestra.run.gibba008.stdout: jq.x86_64 0:1.6-2.el7 2021-02-10T11:10:53.095 INFO:teuthology.orchestra.run.gibba008.stdout: keyutils-libs-devel.x86_64 0:1.5.8-3.el7 2021-02-10T11:10:53.095 INFO:teuthology.orchestra.run.gibba008.stdout: krb5-devel.x86_64 0:1.15.1-50.el7 2021-02-10T11:10:53.095 INFO:teuthology.orchestra.run.gibba008.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2021-02-10T11:10:53.095 INFO:teuthology.orchestra.run.gibba008.stdout: libcom_err-devel.x86_64 0:1.42.9-19.el7 2021-02-10T11:10:53.095 INFO:teuthology.orchestra.run.gibba008.stdout: liboath.x86_64 0:2.6.2-1.el7 2021-02-10T11:10:53.095 INFO:teuthology.orchestra.run.gibba008.stdout: librabbitmq.x86_64 0:0.8.0-3.el7 2021-02-10T11:10:53.095 INFO:teuthology.orchestra.run.gibba008.stdout: librados-devel.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.095 INFO:teuthology.orchestra.run.gibba008.stdout: libradosstriper1.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.095 INFO:teuthology.orchestra.run.gibba008.stdout: librdkafka.x86_64 0:0.11.5-1.el7 2021-02-10T11:10:53.095 INFO:teuthology.orchestra.run.gibba008.stdout: librgw2.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.095 INFO:teuthology.orchestra.run.gibba008.stdout: libselinux-devel.x86_64 0:2.5-15.el7 2021-02-10T11:10:53.096 INFO:teuthology.orchestra.run.gibba008.stdout: libsepol-devel.x86_64 0:2.5-10.el7 2021-02-10T11:10:53.096 INFO:teuthology.orchestra.run.gibba008.stdout: libverto-devel.x86_64 0:0.2.5-4.el7 2021-02-10T11:10:53.096 INFO:teuthology.orchestra.run.gibba008.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2021-02-10T11:10:53.096 INFO:teuthology.orchestra.run.gibba008.stdout: numpy-f2py.x86_64 1:1.7.1-13.el7 2021-02-10T11:10:53.096 INFO:teuthology.orchestra.run.gibba008.stdout: oniguruma.x86_64 0:6.8.2-1.el7 2021-02-10T11:10:53.096 INFO:teuthology.orchestra.run.gibba008.stdout: pcre-devel.x86_64 0:8.32-17.el7 2021-02-10T11:10:53.096 INFO:teuthology.orchestra.run.gibba008.stdout: python-beaker.noarch 0:1.5.4-10.el7 2021-02-10T11:10:53.096 INFO:teuthology.orchestra.run.gibba008.stdout: python-cachetools.noarch 0:1.0.3-1.el7 2021-02-10T11:10:53.096 INFO:teuthology.orchestra.run.gibba008.stdout: python-ceph-argparse.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.096 INFO:teuthology.orchestra.run.gibba008.stdout: python-cephfs.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.096 INFO:teuthology.orchestra.run.gibba008.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2021-02-10T11:10:53.096 INFO:teuthology.orchestra.run.gibba008.stdout: python-execnet.noarch 0:1.7.1-3.el7 2021-02-10T11:10:53.097 INFO:teuthology.orchestra.run.gibba008.stdout: python-jwcrypto.noarch 0:0.4.2-1.el7 2021-02-10T11:10:53.097 INFO:teuthology.orchestra.run.gibba008.stdout: python-mako.noarch 0:0.8.1-2.el7 2021-02-10T11:10:53.097 INFO:teuthology.orchestra.run.gibba008.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2021-02-10T11:10:53.097 INFO:teuthology.orchestra.run.gibba008.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2021-02-10T11:10:53.097 INFO:teuthology.orchestra.run.gibba008.stdout: python-pecan.noarch 0:0.4.5-2.el7 2021-02-10T11:10:53.097 INFO:teuthology.orchestra.run.gibba008.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2021-02-10T11:10:53.097 INFO:teuthology.orchestra.run.gibba008.stdout: python-rados.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.097 INFO:teuthology.orchestra.run.gibba008.stdout: python-rbd.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.097 INFO:teuthology.orchestra.run.gibba008.stdout: python-repoze-lru.noarch 0:0.4-3.el7 2021-02-10T11:10:53.097 INFO:teuthology.orchestra.run.gibba008.stdout: python-requests.noarch 0:2.6.0-10.el7 2021-02-10T11:10:53.098 INFO:teuthology.orchestra.run.gibba008.stdout: python-rgw.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.098 INFO:teuthology.orchestra.run.gibba008.stdout: python-routes.noarch 0:1.13-2.el7 2021-02-10T11:10:53.098 INFO:teuthology.orchestra.run.gibba008.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2021-02-10T11:10:53.098 INFO:teuthology.orchestra.run.gibba008.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2021-02-10T11:10:53.098 INFO:teuthology.orchestra.run.gibba008.stdout: python-tempita.noarch 0:0.5.1-6.el7 2021-02-10T11:10:53.098 INFO:teuthology.orchestra.run.gibba008.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2021-02-10T11:10:53.098 INFO:teuthology.orchestra.run.gibba008.stdout: python-webob.noarch 0:1.2.3-7.el7 2021-02-10T11:10:53.099 INFO:teuthology.orchestra.run.gibba008.stdout: python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 2021-02-10T11:10:53.099 INFO:teuthology.orchestra.run.gibba008.stdout: python-webtest.noarch 0:1.3.4-6.el7 2021-02-10T11:10:53.099 INFO:teuthology.orchestra.run.gibba008.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2021-02-10T11:10:53.099 INFO:teuthology.orchestra.run.gibba008.stdout: python2-apipkg.noarch 0:1.5-5.el7 2021-02-10T11:10:53.099 INFO:teuthology.orchestra.run.gibba008.stdout: python2-bcrypt.x86_64 0:3.1.6-2.el7 2021-02-10T11:10:53.099 INFO:teuthology.orchestra.run.gibba008.stdout: python2-certifi.noarch 0:2018.10.15-5.el7 2021-02-10T11:10:53.099 INFO:teuthology.orchestra.run.gibba008.stdout: python2-google-auth.noarch 1:1.1.1-5.el7 2021-02-10T11:10:53.099 INFO:teuthology.orchestra.run.gibba008.stdout: python2-kubernetes.noarch 1:11.0.0-6.el7 2021-02-10T11:10:53.099 INFO:teuthology.orchestra.run.gibba008.stdout: python2-oauthlib.noarch 0:2.0.1-8.el7 2021-02-10T11:10:53.100 INFO:teuthology.orchestra.run.gibba008.stdout: python2-pyasn1-modules.noarch 0:0.1.9-7.el7 2021-02-10T11:10:53.100 INFO:teuthology.orchestra.run.gibba008.stdout: python2-remoto.noarch 0:1.0.0-1.el7 2021-02-10T11:10:53.100 INFO:teuthology.orchestra.run.gibba008.stdout: python2-requests-oauthlib.noarch 0:0.8.0-5.el7 2021-02-10T11:10:53.100 INFO:teuthology.orchestra.run.gibba008.stdout: python2-rsa.noarch 0:3.4.2-2.el7 2021-02-10T11:10:53.100 INFO:teuthology.orchestra.run.gibba008.stdout: python2-six.noarch 0:1.9.0-0.el7 2021-02-10T11:10:53.100 INFO:teuthology.orchestra.run.gibba008.stdout: python3-ceph-argparse.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.100 INFO:teuthology.orchestra.run.gibba008.stdout: python3-rados.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.100 INFO:teuthology.orchestra.run.gibba008.stdout: scipy.x86_64 0:0.12.1-6.el7 2021-02-10T11:10:53.100 INFO:teuthology.orchestra.run.gibba008.stdout: socat.x86_64 0:1.7.3.2-2.el7 2021-02-10T11:10:53.100 INFO:teuthology.orchestra.run.gibba008.stdout: suitesparse.x86_64 0:4.0.2-10.el7 2021-02-10T11:10:53.100 INFO:teuthology.orchestra.run.gibba008.stdout: tbb.x86_64 0:4.1-9.20130314.el7 2021-02-10T11:10:53.100 INFO:teuthology.orchestra.run.gibba008.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2021-02-10T11:10:53.100 INFO:teuthology.orchestra.run.gibba008.stdout: zlib-devel.x86_64 0:1.2.7-19.el7_9 2021-02-10T11:10:53.101 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:10:53.101 INFO:teuthology.orchestra.run.gibba008.stdout:Updated: 2021-02-10T11:10:53.101 INFO:teuthology.orchestra.run.gibba008.stdout: librados2.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.101 INFO:teuthology.orchestra.run.gibba008.stdout: librbd1.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:10:53.101 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:10:53.101 INFO:teuthology.orchestra.run.gibba008.stdout:Dependency Updated: 2021-02-10T11:10:53.101 INFO:teuthology.orchestra.run.gibba008.stdout: openssl.x86_64 1:1.0.2k-21.el7_9 openssl-libs.x86_64 1:1.0.2k-21.el7_9 2021-02-10T11:10:53.101 INFO:teuthology.orchestra.run.gibba008.stdout: zlib.x86_64 0:1.2.7-19.el7_9 2021-02-10T11:10:53.101 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:10:53.101 INFO:teuthology.orchestra.run.gibba008.stdout:Complete! 2021-02-10T11:10:53.133 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python2-pyasn1-modules-0.1.9-7.el7.noarch 42/99 2021-02-10T11:10:53.220 DEBUG:teuthology.parallel:result is None 2021-02-10T11:10:53.269 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 43/99 2021-02-10T11:10:53.384 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : librabbitmq-0.8.0-3.el7.x86_64 44/99 2021-02-10T11:10:53.526 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:ceph-base-14.2.16-280.g962fac13.el7.x86_64 45/99 2021-02-10T11:10:53.668 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : scipy-0.12.1-6.el7.x86_64 46/99 2021-02-10T11:10:53.760 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:python-ceph-argparse-14.2.16-280.g962fac13.el7.x86_64 47/99 2021-02-10T11:10:53.869 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : libselinux-devel-2.5-15.el7.x86_64 48/99 2021-02-10T11:10:53.961 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : zlib-1.2.7-19.el7_9.x86_64 49/99 2021-02-10T11:10:54.095 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python2-oauthlib-2.0.1-8.el7.noarch 50/99 2021-02-10T11:10:54.196 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : pcre-devel-8.32-17.el7.x86_64 51/99 2021-02-10T11:10:54.338 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:ceph-radosgw-14.2.16-280.g962fac13.el7.x86_64 52/99 2021-02-10T11:10:54.464 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:python-cephfs-14.2.16-280.g962fac13.el7.x86_64 53/99 2021-02-10T11:10:54.564 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:libcephfs-devel-14.2.16-280.g962fac13.el7.x86_64 54/99 2021-02-10T11:10:54.723 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:python-rados-14.2.16-280.g962fac13.el7.x86_64 55/99 2021-02-10T11:10:54.873 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:librados2-14.2.16-280.g962fac13.el7.x86_64 56/99 2021-02-10T11:10:55.015 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : librdkafka-0.11.5-1.el7.x86_64 57/99 2021-02-10T11:10:55.156 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : flex-2.5.37-6.el7.x86_64 58/99 2021-02-10T11:10:55.266 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python2-certifi-2018.10.15-5.el7.noarch 59/99 2021-02-10T11:10:55.408 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:python3-ceph-argparse-14.2.16-280.g962fac13.el7.x86_64 60/99 2021-02-10T11:10:55.545 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:ceph-mgr-14.2.16-280.g962fac13.el7.x86_64 61/99 2021-02-10T11:10:55.659 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:librgw2-14.2.16-280.g962fac13.el7.x86_64 62/99 2021-02-10T11:10:55.801 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : krb5-devel-1.15.1-50.el7.x86_64 63/99 2021-02-10T11:10:55.893 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:ceph-test-14.2.16-280.g962fac13.el7.x86_64 64/99 2021-02-10T11:10:56.411 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 65/99 2021-02-10T11:10:56.544 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : libcom_err-devel-1.42.9-19.el7.x86_64 66/99 2021-02-10T11:10:56.637 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:rbd-fuse-14.2.16-280.g962fac13.el7.x86_64 67/99 2021-02-10T11:10:56.779 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:librbd1-14.2.16-280.g962fac13.el7.x86_64 68/99 2021-02-10T11:10:56.906 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : liboath-2.6.2-1.el7.x86_64 69/99 2021-02-10T11:10:57.014 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python2-six-1.9.0-0.el7.noarch 70/99 2021-02-10T11:10:57.156 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:ceph-mgr-diskprediction-local-14.2.16-280.g962fac13.el 71/99 2021-02-10T11:10:57.256 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python2-bcrypt-3.1.6-2.el7.x86_64 72/99 2021-02-10T11:10:57.415 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 73/99 2021-02-10T11:10:57.574 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:ceph-mgr-rook-14.2.16-280.g962fac13.el7.noarch 74/99 2021-02-10T11:10:57.725 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:librados-devel-14.2.16-280.g962fac13.el7.x86_64 75/99 2021-02-10T11:10:57.861 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 76/99 2021-02-10T11:10:57.976 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:ceph-mgr-dashboard-14.2.16-280.g962fac13.el7.noarch 77/99 2021-02-10T11:10:58.126 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:python3-rados-14.2.16-280.g962fac13.el7.x86_64 78/99 2021-02-10T11:10:58.226 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 79/99 2021-02-10T11:10:58.352 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:python-ceph-compat-14.2.16-280.g962fac13.el7.x86_64 80/99 2021-02-10T11:10:58.482 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:ceph-common-14.2.16-280.g962fac13.el7.x86_64 81/99 2021-02-10T11:10:58.570 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:python3-cephfs-14.2.16-280.g962fac13.el7.x86_64 82/99 2021-02-10T11:10:58.703 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:ceph-grafana-dashboards-14.2.16-280.g962fac13.el7.noar 83/99 2021-02-10T11:10:58.804 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python-execnet-1.7.1-3.el7.noarch 84/99 2021-02-10T11:10:58.938 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 85/99 2021-02-10T11:10:59.080 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 2:ceph-mon-14.2.16-280.g962fac13.el7.x86_64 86/99 2021-02-10T11:10:59.180 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : bison-3.0.4-2.el7.x86_64 87/99 2021-02-10T11:10:59.315 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 1:openssl-devel-1.0.2k-21.el7_9.x86_64 88/99 2021-02-10T11:10:59.465 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python-cachetools-1.0.3-1.el7.noarch 89/99 2021-02-10T11:10:59.574 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 90/99 2021-02-10T11:10:59.724 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : zlib-devel-1.2.7-19.el7_9.x86_64 91/99 2021-02-10T11:10:59.866 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 1:python2-google-auth-1.1.1-5.el7.noarch 92/99 2021-02-10T11:10:59.975 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : tbb-4.1-9.20130314.el7.x86_64 93/99 2021-02-10T11:11:00.117 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : oniguruma-6.8.2-1.el7.x86_64 94/99 2021-02-10T11:11:00.117 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 95/99 2021-02-10T11:11:00.118 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : zlib-1.2.7-18.el7.x86_64 96/99 2021-02-10T11:11:00.118 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 1:openssl-libs-1.0.2k-19.el7.x86_64 97/99 2021-02-10T11:11:00.118 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 1:openssl-1.0.2k-19.el7.x86_64 98/99 2021-02-10T11:11:00.427 INFO:teuthology.orchestra.run.gibba001.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 99/99 2021-02-10T11:11:00.427 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:00.427 INFO:teuthology.orchestra.run.gibba001.stdout:Installed: 2021-02-10T11:11:00.427 INFO:teuthology.orchestra.run.gibba001.stdout: bison.x86_64 0:3.0.4-2.el7 2021-02-10T11:11:00.427 INFO:teuthology.orchestra.run.gibba001.stdout: ceph.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.427 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-fuse.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.427 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-mgr.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.427 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-mgr-dashboard.noarch 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.427 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-mgr-diskprediction-cloud.noarch 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.427 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-mgr-diskprediction-local.noarch 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.428 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-mgr-rook.noarch 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.428 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-mgr-ssh.noarch 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.428 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-radosgw.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.428 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-test.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.428 INFO:teuthology.orchestra.run.gibba001.stdout: elfutils-libelf-devel.x86_64 0:0.176-5.el7 2021-02-10T11:11:00.428 INFO:teuthology.orchestra.run.gibba001.stdout: flex.x86_64 0:2.5.37-6.el7 2021-02-10T11:11:00.428 INFO:teuthology.orchestra.run.gibba001.stdout: libcephfs-devel.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.428 INFO:teuthology.orchestra.run.gibba001.stdout: libcephfs2.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.428 INFO:teuthology.orchestra.run.gibba001.stdout: openssl-devel.x86_64 1:1.0.2k-21.el7_9 2021-02-10T11:11:00.428 INFO:teuthology.orchestra.run.gibba001.stdout: python-ceph-compat.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.428 INFO:teuthology.orchestra.run.gibba001.stdout: python3-cephfs.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.428 INFO:teuthology.orchestra.run.gibba001.stdout: rbd-fuse.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.428 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:00.429 INFO:teuthology.orchestra.run.gibba001.stdout:Dependency Installed: 2021-02-10T11:11:00.429 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-base.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.429 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-common.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.429 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-grafana-dashboards.noarch 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.429 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-mds.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.429 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-mon.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.429 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-osd.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.429 INFO:teuthology.orchestra.run.gibba001.stdout: ceph-selinux.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.429 INFO:teuthology.orchestra.run.gibba001.stdout: jq.x86_64 0:1.6-2.el7 2021-02-10T11:11:00.429 INFO:teuthology.orchestra.run.gibba001.stdout: keyutils-libs-devel.x86_64 0:1.5.8-3.el7 2021-02-10T11:11:00.429 INFO:teuthology.orchestra.run.gibba001.stdout: krb5-devel.x86_64 0:1.15.1-50.el7 2021-02-10T11:11:00.429 INFO:teuthology.orchestra.run.gibba001.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2021-02-10T11:11:00.430 INFO:teuthology.orchestra.run.gibba001.stdout: libcom_err-devel.x86_64 0:1.42.9-19.el7 2021-02-10T11:11:00.430 INFO:teuthology.orchestra.run.gibba001.stdout: liboath.x86_64 0:2.6.2-1.el7 2021-02-10T11:11:00.430 INFO:teuthology.orchestra.run.gibba001.stdout: librabbitmq.x86_64 0:0.8.0-3.el7 2021-02-10T11:11:00.430 INFO:teuthology.orchestra.run.gibba001.stdout: librados-devel.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.430 INFO:teuthology.orchestra.run.gibba001.stdout: libradosstriper1.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.430 INFO:teuthology.orchestra.run.gibba001.stdout: librdkafka.x86_64 0:0.11.5-1.el7 2021-02-10T11:11:00.430 INFO:teuthology.orchestra.run.gibba001.stdout: librgw2.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.430 INFO:teuthology.orchestra.run.gibba001.stdout: libselinux-devel.x86_64 0:2.5-15.el7 2021-02-10T11:11:00.430 INFO:teuthology.orchestra.run.gibba001.stdout: libsepol-devel.x86_64 0:2.5-10.el7 2021-02-10T11:11:00.430 INFO:teuthology.orchestra.run.gibba001.stdout: libverto-devel.x86_64 0:0.2.5-4.el7 2021-02-10T11:11:00.430 INFO:teuthology.orchestra.run.gibba001.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2021-02-10T11:11:00.430 INFO:teuthology.orchestra.run.gibba001.stdout: numpy-f2py.x86_64 1:1.7.1-13.el7 2021-02-10T11:11:00.430 INFO:teuthology.orchestra.run.gibba001.stdout: oniguruma.x86_64 0:6.8.2-1.el7 2021-02-10T11:11:00.431 INFO:teuthology.orchestra.run.gibba001.stdout: pcre-devel.x86_64 0:8.32-17.el7 2021-02-10T11:11:00.431 INFO:teuthology.orchestra.run.gibba001.stdout: python-beaker.noarch 0:1.5.4-10.el7 2021-02-10T11:11:00.431 INFO:teuthology.orchestra.run.gibba001.stdout: python-cachetools.noarch 0:1.0.3-1.el7 2021-02-10T11:11:00.431 INFO:teuthology.orchestra.run.gibba001.stdout: python-ceph-argparse.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.431 INFO:teuthology.orchestra.run.gibba001.stdout: python-cephfs.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.431 INFO:teuthology.orchestra.run.gibba001.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2021-02-10T11:11:00.431 INFO:teuthology.orchestra.run.gibba001.stdout: python-execnet.noarch 0:1.7.1-3.el7 2021-02-10T11:11:00.431 INFO:teuthology.orchestra.run.gibba001.stdout: python-jwcrypto.noarch 0:0.4.2-1.el7 2021-02-10T11:11:00.431 INFO:teuthology.orchestra.run.gibba001.stdout: python-mako.noarch 0:0.8.1-2.el7 2021-02-10T11:11:00.431 INFO:teuthology.orchestra.run.gibba001.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2021-02-10T11:11:00.431 INFO:teuthology.orchestra.run.gibba001.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2021-02-10T11:11:00.431 INFO:teuthology.orchestra.run.gibba001.stdout: python-pecan.noarch 0:0.4.5-2.el7 2021-02-10T11:11:00.431 INFO:teuthology.orchestra.run.gibba001.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2021-02-10T11:11:00.432 INFO:teuthology.orchestra.run.gibba001.stdout: python-rados.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.432 INFO:teuthology.orchestra.run.gibba001.stdout: python-rbd.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.432 INFO:teuthology.orchestra.run.gibba001.stdout: python-repoze-lru.noarch 0:0.4-3.el7 2021-02-10T11:11:00.432 INFO:teuthology.orchestra.run.gibba001.stdout: python-requests.noarch 0:2.6.0-10.el7 2021-02-10T11:11:00.432 INFO:teuthology.orchestra.run.gibba001.stdout: python-rgw.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.432 INFO:teuthology.orchestra.run.gibba001.stdout: python-routes.noarch 0:1.13-2.el7 2021-02-10T11:11:00.432 INFO:teuthology.orchestra.run.gibba001.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2021-02-10T11:11:00.432 INFO:teuthology.orchestra.run.gibba001.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2021-02-10T11:11:00.432 INFO:teuthology.orchestra.run.gibba001.stdout: python-tempita.noarch 0:0.5.1-6.el7 2021-02-10T11:11:00.432 INFO:teuthology.orchestra.run.gibba001.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2021-02-10T11:11:00.432 INFO:teuthology.orchestra.run.gibba001.stdout: python-webob.noarch 0:1.2.3-7.el7 2021-02-10T11:11:00.432 INFO:teuthology.orchestra.run.gibba001.stdout: python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 2021-02-10T11:11:00.433 INFO:teuthology.orchestra.run.gibba001.stdout: python-webtest.noarch 0:1.3.4-6.el7 2021-02-10T11:11:00.433 INFO:teuthology.orchestra.run.gibba001.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2021-02-10T11:11:00.433 INFO:teuthology.orchestra.run.gibba001.stdout: python2-apipkg.noarch 0:1.5-5.el7 2021-02-10T11:11:00.433 INFO:teuthology.orchestra.run.gibba001.stdout: python2-bcrypt.x86_64 0:3.1.6-2.el7 2021-02-10T11:11:00.433 INFO:teuthology.orchestra.run.gibba001.stdout: python2-certifi.noarch 0:2018.10.15-5.el7 2021-02-10T11:11:00.433 INFO:teuthology.orchestra.run.gibba001.stdout: python2-google-auth.noarch 1:1.1.1-5.el7 2021-02-10T11:11:00.433 INFO:teuthology.orchestra.run.gibba001.stdout: python2-kubernetes.noarch 1:11.0.0-6.el7 2021-02-10T11:11:00.433 INFO:teuthology.orchestra.run.gibba001.stdout: python2-oauthlib.noarch 0:2.0.1-8.el7 2021-02-10T11:11:00.433 INFO:teuthology.orchestra.run.gibba001.stdout: python2-pyasn1-modules.noarch 0:0.1.9-7.el7 2021-02-10T11:11:00.433 INFO:teuthology.orchestra.run.gibba001.stdout: python2-remoto.noarch 0:1.0.0-1.el7 2021-02-10T11:11:00.433 INFO:teuthology.orchestra.run.gibba001.stdout: python2-requests-oauthlib.noarch 0:0.8.0-5.el7 2021-02-10T11:11:00.433 INFO:teuthology.orchestra.run.gibba001.stdout: python2-rsa.noarch 0:3.4.2-2.el7 2021-02-10T11:11:00.433 INFO:teuthology.orchestra.run.gibba001.stdout: python2-six.noarch 0:1.9.0-0.el7 2021-02-10T11:11:00.434 INFO:teuthology.orchestra.run.gibba001.stdout: python3-ceph-argparse.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.434 INFO:teuthology.orchestra.run.gibba001.stdout: python3-rados.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.434 INFO:teuthology.orchestra.run.gibba001.stdout: scipy.x86_64 0:0.12.1-6.el7 2021-02-10T11:11:00.434 INFO:teuthology.orchestra.run.gibba001.stdout: socat.x86_64 0:1.7.3.2-2.el7 2021-02-10T11:11:00.434 INFO:teuthology.orchestra.run.gibba001.stdout: suitesparse.x86_64 0:4.0.2-10.el7 2021-02-10T11:11:00.434 INFO:teuthology.orchestra.run.gibba001.stdout: tbb.x86_64 0:4.1-9.20130314.el7 2021-02-10T11:11:00.434 INFO:teuthology.orchestra.run.gibba001.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2021-02-10T11:11:00.434 INFO:teuthology.orchestra.run.gibba001.stdout: zlib-devel.x86_64 0:1.2.7-19.el7_9 2021-02-10T11:11:00.434 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:00.434 INFO:teuthology.orchestra.run.gibba001.stdout:Updated: 2021-02-10T11:11:00.434 INFO:teuthology.orchestra.run.gibba001.stdout: librados2.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.434 INFO:teuthology.orchestra.run.gibba001.stdout: librbd1.x86_64 2:14.2.16-280.g962fac13.el7 2021-02-10T11:11:00.434 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:00.435 INFO:teuthology.orchestra.run.gibba001.stdout:Dependency Updated: 2021-02-10T11:11:00.435 INFO:teuthology.orchestra.run.gibba001.stdout: openssl.x86_64 1:1.0.2k-21.el7_9 openssl-libs.x86_64 1:1.0.2k-21.el7_9 2021-02-10T11:11:00.435 INFO:teuthology.orchestra.run.gibba001.stdout: zlib.x86_64 0:1.2.7-19.el7_9 2021-02-10T11:11:00.435 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:00.435 INFO:teuthology.orchestra.run.gibba001.stdout:Complete! 2021-02-10T11:11:00.555 DEBUG:teuthology.parallel:result is None 2021-02-10T11:11:00.556 DEBUG:teuthology.orchestra.run.gibba001:> sudo lsb_release -is 2021-02-10T11:11:00.928 INFO:teuthology.orchestra.run.gibba001.stdout:RedHatEnterpriseServer 2021-02-10T11:11:00.929 DEBUG:teuthology.misc:System to be installed: RedHatEnterpriseServer 2021-02-10T11:11:00.930 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F7%2Fx86_64&sha1=962fac13b1a781b8c7d25c1eed50d93813b8f5f0 2021-02-10T11:11:01.097 DEBUG:teuthology.orchestra.run.gibba001:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2021-02-10T11:11:01.450 INFO:teuthology.orchestra.run.gibba001.stdout:14.2.16-280.g962fac13.el7 2021-02-10T11:11:01.451 INFO:teuthology.packaging:The installed version of ceph is 14.2.16-280.g962fac13.el7 2021-02-10T11:11:01.451 INFO:teuthology.task.install:The correct ceph version 14.2.16-280.g962fac13 is installed. 2021-02-10T11:11:01.451 DEBUG:teuthology.orchestra.run.gibba008:> sudo lsb_release -is 2021-02-10T11:11:01.823 INFO:teuthology.orchestra.run.gibba008.stdout:RedHatEnterpriseServer 2021-02-10T11:11:01.824 DEBUG:teuthology.misc:System to be installed: RedHatEnterpriseServer 2021-02-10T11:11:01.824 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F7%2Fx86_64&sha1=962fac13b1a781b8c7d25c1eed50d93813b8f5f0 2021-02-10T11:11:01.988 DEBUG:teuthology.orchestra.run.gibba008:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2021-02-10T11:11:02.340 INFO:teuthology.orchestra.run.gibba008.stdout:14.2.16-280.g962fac13.el7 2021-02-10T11:11:02.341 INFO:teuthology.packaging:The installed version of ceph is 14.2.16-280.g962fac13.el7 2021-02-10T11:11:02.341 INFO:teuthology.task.install:The correct ceph version 14.2.16-280.g962fac13 is installed. 2021-02-10T11:11:02.341 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2021-02-10T11:11:02.342 DEBUG:teuthology.orchestra.run.gibba001:> set -ex 2021-02-10T11:11:02.343 DEBUG:teuthology.orchestra.run.gibba001:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-02-10T11:11:02.687 DEBUG:teuthology.orchestra.run.gibba008:> set -ex 2021-02-10T11:11:02.687 DEBUG:teuthology.orchestra.run.gibba008:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-02-10T11:11:03.032 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2021-02-10T11:11:03.033 DEBUG:teuthology.orchestra.run.gibba001:> set -ex 2021-02-10T11:11:03.033 DEBUG:teuthology.orchestra.run.gibba001:> sudo dd of=/usr/bin/daemon-helper 2021-02-10T11:11:03.377 DEBUG:teuthology.orchestra.run.gibba001:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-02-10T11:11:03.719 DEBUG:teuthology.orchestra.run.gibba008:> set -ex 2021-02-10T11:11:03.720 DEBUG:teuthology.orchestra.run.gibba008:> sudo dd of=/usr/bin/daemon-helper 2021-02-10T11:11:04.063 DEBUG:teuthology.orchestra.run.gibba008:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-02-10T11:11:04.405 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2021-02-10T11:11:04.406 DEBUG:teuthology.orchestra.run.gibba001:> set -ex 2021-02-10T11:11:04.406 DEBUG:teuthology.orchestra.run.gibba001:> sudo dd of=/usr/bin/adjust-ulimits 2021-02-10T11:11:04.749 DEBUG:teuthology.orchestra.run.gibba001:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-02-10T11:11:05.092 DEBUG:teuthology.orchestra.run.gibba008:> set -ex 2021-02-10T11:11:05.493 DEBUG:teuthology.orchestra.run.gibba008:> sudo dd of=/usr/bin/adjust-ulimits 2021-02-10T11:11:05.835 DEBUG:teuthology.orchestra.run.gibba008:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-02-10T11:11:06.178 INFO:teuthology.run_tasks:Running task ceph... 2021-02-10T11:11:06.246 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2021-02-10T11:11:06.246 DEBUG:teuthology.orchestra.run.gibba001:> sudo chmod 777 /var/log/ceph 2021-02-10T11:11:06.254 DEBUG:teuthology.orchestra.run.gibba008:> sudo chmod 777 /var/log/ceph 2021-02-10T11:11:06.596 INFO:tasks.ceph:Disabling ceph logrotate... 2021-02-10T11:11:06.597 DEBUG:teuthology.orchestra.run.gibba001:> sudo rm -f -- /etc/logrotate.d/ceph 2021-02-10T11:11:06.600 DEBUG:teuthology.orchestra.run.gibba008:> sudo rm -f -- /etc/logrotate.d/ceph 2021-02-10T11:11:06.941 INFO:tasks.ceph:Creating extra log directories... 2021-02-10T11:11:07.159 DEBUG:teuthology.orchestra.run.gibba001:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-02-10T11:11:07.162 DEBUG:teuthology.orchestra.run.gibba008:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-02-10T11:11:07.507 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2021-02-10T11:11:07.508 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2021-02-10T11:11:07.509 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2021-02-10T11:11:07.509 DEBUG:teuthology.orchestra.run.gibba001:> set -ex 2021-02-10T11:11:07.509 DEBUG:teuthology.orchestra.run.gibba001:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2021-02-10T11:11:07.841 DEBUG:teuthology.orchestra.run.gibba001:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2021-02-10T11:11:08.237 DEBUG:teuthology.orchestra.run.gibba001:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2021-02-10T11:11:08.581 DEBUG:teuthology.orchestra.run.gibba008:> set -ex 2021-02-10T11:11:08.765 DEBUG:teuthology.orchestra.run.gibba008:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2021-02-10T11:11:09.101 DEBUG:teuthology.orchestra.run.gibba008:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2021-02-10T11:11:09.493 DEBUG:teuthology.orchestra.run.gibba008:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2021-02-10T11:11:09.837 INFO:tasks.ceph:Creating ceph cluster ceph... 2021-02-10T11:11:09.837 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': ''}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180}, 'mgr': {'debug client': 10, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(TOO_FEW_PGS\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2021-02-10T11:11:09.837 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/khiremat-2021-02-10_10:13:45-fs-wip-khiremat-mgr-authid-bp-7-nautilus-distro-basic-gibba/5872719', 'branch': 'wip-khiremat-mgr-authid-bp-7-nautilus', 'description': 'fs/basic_functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{rhel_7} tasks/volumes}', 'email': 'khiremat@redhat.com', 'first_in_suite': False, 'job_id': '5872719', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'gibba', 'name': 'khiremat-2021-02-10_10:13:45-fs-wip-khiremat-mgr-authid-bp-7-nautilus-distro-basic-gibba', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 30}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '7.9', 'overrides': {'admin_socket': {'branch': 'wip-khiremat-mgr-authid-bp-7-nautilus'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': ''}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180}, 'mgr': {'debug client': 10, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(TOO_FEW_PGS\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '962fac13b1a781b8c7d25c1eed50d93813b8f5f0'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'sha1': '962fac13b1a781b8c7d25c1eed50d93813b8f5f0'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-khiremat-mgr-authid-bp-7-nautilus', 'sha1': '962fac13b1a781b8c7d25c1eed50d93813b8f5f0'}}, 'owner': 'scheduled_khiremat@teuthology', 'priority': 50, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.1'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.2', 'client.3']], 'sha1': '962fac13b1a781b8c7d25c1eed50d93813b8f5f0', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-khiremat-mgr-authid-bp-7-nautilus', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_962fac13b1a781b8c7d25c1eed50d93813b8f5f0/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '962fac13b1a781b8c7d25c1eed50d93813b8f5f0', 'targets': {'gibba001.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCUYH0PUZVukSVO9/5yIUBH59HFWSUrEpb7DzfyPkTBTyxU56GwMHZKC4xI9KebrEXDXCAOhT1M3+BEsG1iWZTEV7tCwjBSsRImzspodarlIUJ1lIFayszBAjqS5eAT/G92CIHrkMNY1dmYR24X7hYo+VD2aHNE01Z7ZVbY6WLybagqGUACa1PLmmAd7kW+Aal1a/f70ZDUf6xTuaOZjBgY7ajWFFpwZZvDu3uzCkiadVn6ACugS9unhoOa62BwXIv0i9KfKkQOzWiBZwkX5MDK8ldtDzWiX+sYvVj3N9J0rbyOFDv4E78o9Zwb30S4e7/w/Az6rHrGbqJwYDIcyDdb', 'gibba008.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5wOFVXcmGwExgtBNqJLFpE6v+rZ2LpWLmVDEnU+vqKAWY2NJAQB8234cc6aGJ8HvkxePvVeKnU39ho+M2ssdf9tOogE3vT6CgDEiqluaFc9ctL1eF+Sl1yLBqqSyiu3FAANHtFCHZ+RWfVk4A4a/bNqNXD2K/KbJuhgyNhj+EVew+MakE1iH0bpexJk3lQjc0Tq2VPyC4xoJ1WHxS7zR9y883SaXfEd/CZlbLCSn4t7RRXsoQ6ovsyx+oIaVMMTIQVsZ4ZVIee/2wMOmWrozI+S/15y9iXm9vC6404aP3MnnBSbKW/l0M4ZnyNV6lDkmgc8kQ/FL/r2SkPfnEWUZl'}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel']}, 'sha1': '962fac13b1a781b8c7d25c1eed50d93813b8f5f0'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes']}}], 'teuthology_branch': 'master', 'timestamp': '2021-02-10_10:13:45', 'tube': 'gibba', 'user': 'khiremat', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.gibba.29737'} 2021-02-10T11:11:09.837 DEBUG:teuthology.orchestra.run.gibba001:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2021-02-10T11:11:09.841 DEBUG:teuthology.orchestra.run.gibba008:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2021-02-10T11:11:10.177 DEBUG:teuthology.orchestra.run.gibba001:> sudo install -d -m0777 -- /var/run/ceph 2021-02-10T11:11:10.184 DEBUG:teuthology.orchestra.run.gibba008:> sudo install -d -m0777 -- /var/run/ceph 2021-02-10T11:11:10.550 DEBUG:teuthology.orchestra.run.gibba001:> set -ex 2021-02-10T11:11:10.550 DEBUG:teuthology.orchestra.run.gibba001:> dd if=/scratch_devs of=/dev/stdout 2021-02-10T11:11:10.884 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-02-10T11:11:10.885 DEBUG:teuthology.orchestra.run.gibba001:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-02-10T11:11:11.223 INFO:teuthology.orchestra.run.gibba001.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-0’ 2021-02-10T11:11:11.223 INFO:teuthology.orchestra.run.gibba001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-10T11:11:11.223 INFO:teuthology.orchestra.run.gibba001.stdout:Device: 5h/5d Inode: 109434 Links: 1 2021-02-10T11:11:11.223 INFO:teuthology.orchestra.run.gibba001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-10T11:11:11.223 INFO:teuthology.orchestra.run.gibba001.stdout:Context: system_u:object_r:device_t:s0 2021-02-10T11:11:11.223 INFO:teuthology.orchestra.run.gibba001.stdout:Access: 2021-02-10 11:10:39.210336389 +0000 2021-02-10T11:11:11.223 INFO:teuthology.orchestra.run.gibba001.stdout:Modify: 2021-02-10 11:07:52.114551613 +0000 2021-02-10T11:11:11.223 INFO:teuthology.orchestra.run.gibba001.stdout:Change: 2021-02-10 11:07:52.114551613 +0000 2021-02-10T11:11:11.224 INFO:teuthology.orchestra.run.gibba001.stdout: Birth: - 2021-02-10T11:11:11.232 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:11:11.233 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:11:11.233 INFO:teuthology.orchestra.run.gibba001.stderr:512 bytes (512 B) copied, 0.000197285 s, 2.6 MB/s 2021-02-10T11:11:11.235 DEBUG:teuthology.orchestra.run.gibba001:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-02-10T11:11:11.568 INFO:teuthology.orchestra.run.gibba001.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-1’ 2021-02-10T11:11:11.568 INFO:teuthology.orchestra.run.gibba001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-10T11:11:11.568 INFO:teuthology.orchestra.run.gibba001.stdout:Device: 5h/5d Inode: 109460 Links: 1 2021-02-10T11:11:11.568 INFO:teuthology.orchestra.run.gibba001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-10T11:11:11.569 INFO:teuthology.orchestra.run.gibba001.stdout:Context: system_u:object_r:device_t:s0 2021-02-10T11:11:11.569 INFO:teuthology.orchestra.run.gibba001.stdout:Access: 2021-02-10 11:10:39.210336389 +0000 2021-02-10T11:11:11.569 INFO:teuthology.orchestra.run.gibba001.stdout:Modify: 2021-02-10 11:07:52.643557673 +0000 2021-02-10T11:11:11.569 INFO:teuthology.orchestra.run.gibba001.stdout:Change: 2021-02-10 11:07:52.643557673 +0000 2021-02-10T11:11:11.569 INFO:teuthology.orchestra.run.gibba001.stdout: Birth: - 2021-02-10T11:11:11.577 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:11:11.577 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:11:11.578 INFO:teuthology.orchestra.run.gibba001.stderr:512 bytes (512 B) copied, 0.000209707 s, 2.4 MB/s 2021-02-10T11:11:11.579 DEBUG:teuthology.orchestra.run.gibba001:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-02-10T11:11:11.919 INFO:teuthology.orchestra.run.gibba001.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2021-02-10T11:11:11.919 INFO:teuthology.orchestra.run.gibba001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-10T11:11:11.919 INFO:teuthology.orchestra.run.gibba001.stdout:Device: 5h/5d Inode: 108388 Links: 1 2021-02-10T11:11:11.919 INFO:teuthology.orchestra.run.gibba001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-10T11:11:11.919 INFO:teuthology.orchestra.run.gibba001.stdout:Context: system_u:object_r:device_t:s0 2021-02-10T11:11:11.919 INFO:teuthology.orchestra.run.gibba001.stdout:Access: 2021-02-10 11:10:39.210336389 +0000 2021-02-10T11:11:11.920 INFO:teuthology.orchestra.run.gibba001.stdout:Modify: 2021-02-10 11:07:53.168563688 +0000 2021-02-10T11:11:11.920 INFO:teuthology.orchestra.run.gibba001.stdout:Change: 2021-02-10 11:07:53.168563688 +0000 2021-02-10T11:11:11.920 INFO:teuthology.orchestra.run.gibba001.stdout: Birth: - 2021-02-10T11:11:11.929 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:11:11.929 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:11:11.929 INFO:teuthology.orchestra.run.gibba001.stderr:512 bytes (512 B) copied, 0.000186124 s, 2.8 MB/s 2021-02-10T11:11:11.931 DEBUG:teuthology.orchestra.run.gibba001:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-02-10T11:11:12.267 INFO:teuthology.orchestra.run.gibba001.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-3’ 2021-02-10T11:11:12.268 INFO:teuthology.orchestra.run.gibba001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-10T11:11:12.268 INFO:teuthology.orchestra.run.gibba001.stdout:Device: 5h/5d Inode: 111486 Links: 1 2021-02-10T11:11:12.268 INFO:teuthology.orchestra.run.gibba001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-10T11:11:12.268 INFO:teuthology.orchestra.run.gibba001.stdout:Context: system_u:object_r:device_t:s0 2021-02-10T11:11:12.268 INFO:teuthology.orchestra.run.gibba001.stdout:Access: 2021-02-10 11:10:39.210336389 +0000 2021-02-10T11:11:12.268 INFO:teuthology.orchestra.run.gibba001.stdout:Modify: 2021-02-10 11:07:53.723570046 +0000 2021-02-10T11:11:12.269 INFO:teuthology.orchestra.run.gibba001.stdout:Change: 2021-02-10 11:07:53.723570046 +0000 2021-02-10T11:11:12.269 INFO:teuthology.orchestra.run.gibba001.stdout: Birth: - 2021-02-10T11:11:12.277 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:11:12.277 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:11:12.277 INFO:teuthology.orchestra.run.gibba001.stderr:512 bytes (512 B) copied, 0.00017278 s, 3.0 MB/s 2021-02-10T11:11:12.279 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-02-10T11:11:12.279 DEBUG:teuthology.orchestra.run.gibba008:> set -ex 2021-02-10T11:11:12.280 DEBUG:teuthology.orchestra.run.gibba008:> dd if=/scratch_devs of=/dev/stdout 2021-02-10T11:11:12.614 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-02-10T11:11:12.614 DEBUG:teuthology.orchestra.run.gibba008:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-02-10T11:11:12.948 INFO:teuthology.orchestra.run.gibba008.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-0’ 2021-02-10T11:11:12.948 INFO:teuthology.orchestra.run.gibba008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-10T11:11:12.948 INFO:teuthology.orchestra.run.gibba008.stdout:Device: 5h/5d Inode: 82821 Links: 1 2021-02-10T11:11:12.948 INFO:teuthology.orchestra.run.gibba008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-10T11:11:12.948 INFO:teuthology.orchestra.run.gibba008.stdout:Context: system_u:object_r:device_t:s0 2021-02-10T11:11:12.948 INFO:teuthology.orchestra.run.gibba008.stdout:Access: 2021-02-10 11:10:34.698607540 +0000 2021-02-10T11:11:12.948 INFO:teuthology.orchestra.run.gibba008.stdout:Modify: 2021-02-10 11:07:52.239299748 +0000 2021-02-10T11:11:12.948 INFO:teuthology.orchestra.run.gibba008.stdout:Change: 2021-02-10 11:07:52.239299748 +0000 2021-02-10T11:11:12.949 INFO:teuthology.orchestra.run.gibba008.stdout: Birth: - 2021-02-10T11:11:12.958 INFO:teuthology.orchestra.run.gibba008.stderr:1+0 records in 2021-02-10T11:11:12.958 INFO:teuthology.orchestra.run.gibba008.stderr:1+0 records out 2021-02-10T11:11:12.958 INFO:teuthology.orchestra.run.gibba008.stderr:512 bytes (512 B) copied, 0.000204937 s, 2.5 MB/s 2021-02-10T11:11:12.960 DEBUG:teuthology.orchestra.run.gibba008:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-02-10T11:11:13.293 INFO:teuthology.orchestra.run.gibba008.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-1’ 2021-02-10T11:11:13.293 INFO:teuthology.orchestra.run.gibba008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-10T11:11:13.293 INFO:teuthology.orchestra.run.gibba008.stdout:Device: 5h/5d Inode: 91405 Links: 1 2021-02-10T11:11:13.293 INFO:teuthology.orchestra.run.gibba008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-10T11:11:13.293 INFO:teuthology.orchestra.run.gibba008.stdout:Context: system_u:object_r:device_t:s0 2021-02-10T11:11:13.293 INFO:teuthology.orchestra.run.gibba008.stdout:Access: 2021-02-10 11:10:34.698607540 +0000 2021-02-10T11:11:13.293 INFO:teuthology.orchestra.run.gibba008.stdout:Modify: 2021-02-10 11:07:52.778286205 +0000 2021-02-10T11:11:13.294 INFO:teuthology.orchestra.run.gibba008.stdout:Change: 2021-02-10 11:07:52.778286205 +0000 2021-02-10T11:11:13.294 INFO:teuthology.orchestra.run.gibba008.stdout: Birth: - 2021-02-10T11:11:13.302 INFO:teuthology.orchestra.run.gibba008.stderr:1+0 records in 2021-02-10T11:11:13.303 INFO:teuthology.orchestra.run.gibba008.stderr:1+0 records out 2021-02-10T11:11:13.303 INFO:teuthology.orchestra.run.gibba008.stderr:512 bytes (512 B) copied, 0.000248009 s, 2.1 MB/s 2021-02-10T11:11:13.305 DEBUG:teuthology.orchestra.run.gibba008:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-02-10T11:11:13.638 INFO:teuthology.orchestra.run.gibba008.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2021-02-10T11:11:13.638 INFO:teuthology.orchestra.run.gibba008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-10T11:11:13.638 INFO:teuthology.orchestra.run.gibba008.stdout:Device: 5h/5d Inode: 62887 Links: 1 2021-02-10T11:11:13.639 INFO:teuthology.orchestra.run.gibba008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-10T11:11:13.639 INFO:teuthology.orchestra.run.gibba008.stdout:Context: system_u:object_r:device_t:s0 2021-02-10T11:11:13.639 INFO:teuthology.orchestra.run.gibba008.stdout:Access: 2021-02-10 11:10:34.698607540 +0000 2021-02-10T11:11:13.639 INFO:teuthology.orchestra.run.gibba008.stdout:Modify: 2021-02-10 11:07:53.350271834 +0000 2021-02-10T11:11:13.639 INFO:teuthology.orchestra.run.gibba008.stdout:Change: 2021-02-10 11:07:53.350271834 +0000 2021-02-10T11:11:13.639 INFO:teuthology.orchestra.run.gibba008.stdout: Birth: - 2021-02-10T11:11:13.647 INFO:teuthology.orchestra.run.gibba008.stderr:1+0 records in 2021-02-10T11:11:13.647 INFO:teuthology.orchestra.run.gibba008.stderr:1+0 records out 2021-02-10T11:11:13.647 INFO:teuthology.orchestra.run.gibba008.stderr:512 bytes (512 B) copied, 0.000169877 s, 3.0 MB/s 2021-02-10T11:11:13.650 DEBUG:teuthology.orchestra.run.gibba008:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-02-10T11:11:13.983 INFO:teuthology.orchestra.run.gibba008.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-3’ 2021-02-10T11:11:13.983 INFO:teuthology.orchestra.run.gibba008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-10T11:11:13.983 INFO:teuthology.orchestra.run.gibba008.stdout:Device: 5h/5d Inode: 95430 Links: 1 2021-02-10T11:11:13.983 INFO:teuthology.orchestra.run.gibba008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-10T11:11:13.983 INFO:teuthology.orchestra.run.gibba008.stdout:Context: system_u:object_r:device_t:s0 2021-02-10T11:11:13.983 INFO:teuthology.orchestra.run.gibba008.stdout:Access: 2021-02-10 11:10:34.698607540 +0000 2021-02-10T11:11:13.983 INFO:teuthology.orchestra.run.gibba008.stdout:Modify: 2021-02-10 11:07:53.863258945 +0000 2021-02-10T11:11:13.983 INFO:teuthology.orchestra.run.gibba008.stdout:Change: 2021-02-10 11:07:53.863258945 +0000 2021-02-10T11:11:13.983 INFO:teuthology.orchestra.run.gibba008.stdout: Birth: - 2021-02-10T11:11:13.992 INFO:teuthology.orchestra.run.gibba008.stderr:1+0 records in 2021-02-10T11:11:13.992 INFO:teuthology.orchestra.run.gibba008.stderr:1+0 records out 2021-02-10T11:11:13.992 INFO:teuthology.orchestra.run.gibba008.stderr:512 bytes (512 B) copied, 0.000188125 s, 2.7 MB/s 2021-02-10T11:11:13.994 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-02-10T11:11:13.994 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@gibba001.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@gibba008.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2021-02-10T11:11:13.994 INFO:tasks.ceph:Generating config... 2021-02-10T11:11:13.996 INFO:tasks.ceph:[client] client mount timeout = 600 2021-02-10T11:11:13.996 INFO:tasks.ceph:[client] debug client = 20 2021-02-10T11:11:13.996 INFO:tasks.ceph:[client] debug ms = 1 2021-02-10T11:11:13.996 INFO:tasks.ceph:[client] pid file = 2021-02-10T11:11:13.997 INFO:tasks.ceph:[mds] debug mds = 20 2021-02-10T11:11:13.997 INFO:tasks.ceph:[mds] debug ms = 1 2021-02-10T11:11:13.997 INFO:tasks.ceph:[mds] mds bal frag = True 2021-02-10T11:11:13.997 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2021-02-10T11:11:13.997 INFO:tasks.ceph:[mds] mds bal merge size = 5 2021-02-10T11:11:13.997 INFO:tasks.ceph:[mds] mds bal split bits = 3 2021-02-10T11:11:13.997 INFO:tasks.ceph:[mds] mds bal split size = 100 2021-02-10T11:11:13.997 INFO:tasks.ceph:[mds] mds debug frag = True 2021-02-10T11:11:13.997 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2021-02-10T11:11:13.997 INFO:tasks.ceph:[mds] mds op complaint time = 180 2021-02-10T11:11:13.997 INFO:tasks.ceph:[mds] mds verify scatter = True 2021-02-10T11:11:13.997 INFO:tasks.ceph:[mds] osd op complaint time = 180 2021-02-10T11:11:13.997 INFO:tasks.ceph:[mgr] debug client = 10 2021-02-10T11:11:13.997 INFO:tasks.ceph:[mgr] debug mgr = 20 2021-02-10T11:11:13.998 INFO:tasks.ceph:[mgr] debug ms = 1 2021-02-10T11:11:13.998 INFO:tasks.ceph:[mon] debug mon = 20 2021-02-10T11:11:13.998 INFO:tasks.ceph:[mon] debug ms = 1 2021-02-10T11:11:13.998 INFO:tasks.ceph:[mon] debug paxos = 20 2021-02-10T11:11:13.998 INFO:tasks.ceph:[mon] mon op complaint time = 120 2021-02-10T11:11:13.998 INFO:tasks.ceph:[osd] bdev async discard = True 2021-02-10T11:11:13.998 INFO:tasks.ceph:[osd] bdev enable discard = True 2021-02-10T11:11:13.998 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2021-02-10T11:11:13.998 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2021-02-10T11:11:13.998 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2021-02-10T11:11:13.998 INFO:tasks.ceph:[osd] debug bluefs = 20 2021-02-10T11:11:13.998 INFO:tasks.ceph:[osd] debug bluestore = 20 2021-02-10T11:11:13.998 INFO:tasks.ceph:[osd] debug filestore = 20 2021-02-10T11:11:13.998 INFO:tasks.ceph:[osd] debug journal = 20 2021-02-10T11:11:13.999 INFO:tasks.ceph:[osd] debug ms = 20 2021-02-10T11:11:13.999 INFO:tasks.ceph:[osd] debug osd = 25 2021-02-10T11:11:13.999 INFO:tasks.ceph:[osd] debug rocksdb = 10 2021-02-10T11:11:13.999 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2021-02-10T11:11:13.999 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2021-02-10T11:11:13.999 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2021-02-10T11:11:13.999 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2021-02-10T11:11:13.999 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2021-02-10T11:11:13.999 INFO:tasks.ceph:[osd] osd op complaint time = 180 2021-02-10T11:11:13.999 INFO:tasks.ceph:Setting up mon.a... 2021-02-10T11:11:13.999 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2021-02-10T11:11:14.392 INFO:teuthology.orchestra.run.gibba001.stdout:creating /etc/ceph/ceph.keyring 2021-02-10T11:11:14.394 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2021-02-10T11:11:14.762 DEBUG:teuthology.orchestra.run.gibba001:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-02-10T11:11:15.104 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.2.101'), ('mon.b', '172.21.2.108'), ('mon.c', '[v2:172.21.2.108:3301,v1:172.21.2.108:6790]')] 2021-02-10T11:11:15.105 DEBUG:teuthology.orchestra.run.gibba001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.2.101 --add b 172.21.2.108 --addv c '[v2:172.21.2.108:3301,v1:172.21.2.108:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2021-02-10T11:11:15.470 INFO:teuthology.orchestra.run.gibba001.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2021-02-10T11:11:15.470 INFO:teuthology.orchestra.run.gibba001.stdout:monmaptool: generated fsid a6663ba9-2653-4542-8b6a-9323b4e7b545 2021-02-10T11:11:15.470 INFO:teuthology.orchestra.run.gibba001.stdout:epoch 0 2021-02-10T11:11:15.470 INFO:teuthology.orchestra.run.gibba001.stdout:fsid a6663ba9-2653-4542-8b6a-9323b4e7b545 2021-02-10T11:11:15.470 INFO:teuthology.orchestra.run.gibba001.stdout:last_changed 2021-02-10 11:11:15.467857 2021-02-10T11:11:15.470 INFO:teuthology.orchestra.run.gibba001.stdout:created 2021-02-10 11:11:15.467857 2021-02-10T11:11:15.470 INFO:teuthology.orchestra.run.gibba001.stdout:min_mon_release 0 (unknown) 2021-02-10T11:11:15.470 INFO:teuthology.orchestra.run.gibba001.stdout:0: [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] mon.a 2021-02-10T11:11:15.470 INFO:teuthology.orchestra.run.gibba001.stdout:1: [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] mon.b 2021-02-10T11:11:15.471 INFO:teuthology.orchestra.run.gibba001.stdout:2: [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] mon.c 2021-02-10T11:11:15.471 INFO:teuthology.orchestra.run.gibba001.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2021-02-10T11:11:15.472 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID a6663ba9-2653-4542-8b6a-9323b4e7b545... 2021-02-10T11:11:15.473 DEBUG:teuthology.orchestra.run.gibba001:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-02-10T11:11:15.476 DEBUG:teuthology.orchestra.run.gibba008:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-02-10T11:11:15.834 INFO:teuthology.orchestra.run.gibba001.stdout:[global] 2021-02-10T11:11:15.834 INFO:teuthology.orchestra.run.gibba001.stdout: chdir = "" 2021-02-10T11:11:15.834 INFO:teuthology.orchestra.run.gibba001.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-02-10T11:11:15.834 INFO:teuthology.orchestra.run.gibba001.stdout: auth supported = cephx 2021-02-10T11:11:15.834 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.835 INFO:teuthology.orchestra.run.gibba001.stdout: filestore xattr use omap = true 2021-02-10T11:11:15.835 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.835 INFO:teuthology.orchestra.run.gibba001.stdout: mon clock drift allowed = 1.000 2021-02-10T11:11:15.835 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.835 INFO:teuthology.orchestra.run.gibba001.stdout: osd crush chooseleaf type = 0 2021-02-10T11:11:15.835 INFO:teuthology.orchestra.run.gibba001.stdout: auth debug = true 2021-02-10T11:11:15.836 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.836 INFO:teuthology.orchestra.run.gibba001.stdout: ms die on old message = true 2021-02-10T11:11:15.836 INFO:teuthology.orchestra.run.gibba001.stdout: ms die on bug = true 2021-02-10T11:11:15.836 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.836 INFO:teuthology.orchestra.run.gibba001.stdout: mon max pg per osd = 10000 # >= luminous 2021-02-10T11:11:15.836 INFO:teuthology.orchestra.run.gibba001.stdout: mon pg warn max object skew = 0 2021-02-10T11:11:15.836 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.836 INFO:teuthology.orchestra.run.gibba001.stdout: osd pool default size = 2 2021-02-10T11:11:15.836 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.836 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd allow primary affinity = true 2021-02-10T11:11:15.837 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd allow pg remap = true 2021-02-10T11:11:15.837 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on legacy crush tunables = false 2021-02-10T11:11:15.837 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on crush straw calc version zero = false 2021-02-10T11:11:15.837 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on no sortbitwise = false 2021-02-10T11:11:15.837 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on osd down out interval zero = false 2021-02-10T11:11:15.837 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on too few osds = false 2021-02-10T11:11:15.837 INFO:teuthology.orchestra.run.gibba001.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-02-10T11:11:15.837 INFO:teuthology.orchestra.run.gibba001.stdout: mon_warn_on_pool_no_redundancy = false 2021-02-10T11:11:15.837 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.837 INFO:teuthology.orchestra.run.gibba001.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-02-10T11:11:15.837 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.837 INFO:teuthology.orchestra.run.gibba001.stdout: osd default data pool replay window = 5 2021-02-10T11:11:15.837 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.838 INFO:teuthology.orchestra.run.gibba001.stdout: mon allow pool delete = true 2021-02-10T11:11:15.838 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.838 INFO:teuthology.orchestra.run.gibba001.stdout: mon cluster log file level = debug 2021-02-10T11:11:15.838 INFO:teuthology.orchestra.run.gibba001.stdout: debug asserts on shutdown = true 2021-02-10T11:11:15.838 INFO:teuthology.orchestra.run.gibba001.stdout: mon health detail to clog = false 2021-02-10T11:11:15.838 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.838 INFO:teuthology.orchestra.run.gibba001.stdout: # we see this fail in qa on *nautilus*; bump up retries 2021-02-10T11:11:15.838 INFO:teuthology.orchestra.run.gibba001.stdout: mon_client_directed_command_retry = 4 2021-02-10T11:11:15.838 INFO:teuthology.orchestra.run.gibba001.stdout: mon host = "172.21.2.101,172.21.2.108,[v2:172.21.2.108:3301,v1:172.21.2.108:6790]" 2021-02-10T11:11:15.838 INFO:teuthology.orchestra.run.gibba001.stdout: fsid = a6663ba9-2653-4542-8b6a-9323b4e7b545 2021-02-10T11:11:15.838 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.838 INFO:teuthology.orchestra.run.gibba001.stdout:[osd] 2021-02-10T11:11:15.838 INFO:teuthology.orchestra.run.gibba001.stdout: osd journal size = 100 2021-02-10T11:11:15.839 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.839 INFO:teuthology.orchestra.run.gibba001.stdout: osd scrub load threshold = 5.0 2021-02-10T11:11:15.839 INFO:teuthology.orchestra.run.gibba001.stdout: osd scrub max interval = 600 2021-02-10T11:11:15.839 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.839 INFO:teuthology.orchestra.run.gibba001.stdout: osd recover clone overlap = true 2021-02-10T11:11:15.839 INFO:teuthology.orchestra.run.gibba001.stdout: osd recovery max chunk = 1048576 2021-02-10T11:11:15.839 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.839 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug shutdown = true 2021-02-10T11:11:15.839 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug op order = true 2021-02-10T11:11:15.839 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug verify stray on activate = true 2021-02-10T11:11:15.839 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.839 INFO:teuthology.orchestra.run.gibba001.stdout: osd open classes on start = true 2021-02-10T11:11:15.839 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug pg log writeout = true 2021-02-10T11:11:15.839 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.840 INFO:teuthology.orchestra.run.gibba001.stdout: osd deep scrub update digest min age = 30 2021-02-10T11:11:15.840 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.840 INFO:teuthology.orchestra.run.gibba001.stdout: osd map max advance = 10 2021-02-10T11:11:15.840 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.840 INFO:teuthology.orchestra.run.gibba001.stdout: journal zero on create = true 2021-02-10T11:11:15.840 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.840 INFO:teuthology.orchestra.run.gibba001.stdout: filestore ondisk finisher threads = 3 2021-02-10T11:11:15.840 INFO:teuthology.orchestra.run.gibba001.stdout: filestore apply finisher threads = 3 2021-02-10T11:11:15.840 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.840 INFO:teuthology.orchestra.run.gibba001.stdout: bdev debug aio = true 2021-02-10T11:11:15.840 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug misdirected ops = true 2021-02-10T11:11:15.840 INFO:teuthology.orchestra.run.gibba001.stdout: bdev async discard = True 2021-02-10T11:11:15.840 INFO:teuthology.orchestra.run.gibba001.stdout: bdev enable discard = True 2021-02-10T11:11:15.841 INFO:teuthology.orchestra.run.gibba001.stdout: bluestore allocator = bitmap 2021-02-10T11:11:15.841 INFO:teuthology.orchestra.run.gibba001.stdout: bluestore block size = 96636764160 2021-02-10T11:11:15.841 INFO:teuthology.orchestra.run.gibba001.stdout: bluestore fsck on mount = True 2021-02-10T11:11:15.841 INFO:teuthology.orchestra.run.gibba001.stdout: debug bluefs = 20 2021-02-10T11:11:15.841 INFO:teuthology.orchestra.run.gibba001.stdout: debug bluestore = 20 2021-02-10T11:11:15.841 INFO:teuthology.orchestra.run.gibba001.stdout: debug filestore = 20 2021-02-10T11:11:15.841 INFO:teuthology.orchestra.run.gibba001.stdout: debug journal = 20 2021-02-10T11:11:15.841 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 20 2021-02-10T11:11:15.841 INFO:teuthology.orchestra.run.gibba001.stdout: debug osd = 25 2021-02-10T11:11:15.841 INFO:teuthology.orchestra.run.gibba001.stdout: debug rocksdb = 10 2021-02-10T11:11:15.841 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd backfillfull_ratio = 0.85 2021-02-10T11:11:15.841 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd full ratio = 0.9 2021-02-10T11:11:15.841 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd nearfull ratio = 0.8 2021-02-10T11:11:15.842 INFO:teuthology.orchestra.run.gibba001.stdout: osd failsafe full ratio = 0.95 2021-02-10T11:11:15.842 INFO:teuthology.orchestra.run.gibba001.stdout: osd objectstore = bluestore 2021-02-10T11:11:15.842 INFO:teuthology.orchestra.run.gibba001.stdout: osd op complaint time = 180 2021-02-10T11:11:15.842 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.842 INFO:teuthology.orchestra.run.gibba001.stdout:[mgr] 2021-02-10T11:11:15.842 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 1 2021-02-10T11:11:15.842 INFO:teuthology.orchestra.run.gibba001.stdout: debug mgr = 20 2021-02-10T11:11:15.842 INFO:teuthology.orchestra.run.gibba001.stdout: debug mon = 20 2021-02-10T11:11:15.842 INFO:teuthology.orchestra.run.gibba001.stdout: debug auth = 20 2021-02-10T11:11:15.842 INFO:teuthology.orchestra.run.gibba001.stdout: mon reweight min pgs per osd = 4 2021-02-10T11:11:15.842 INFO:teuthology.orchestra.run.gibba001.stdout: mon reweight min bytes per osd = 10 2021-02-10T11:11:15.842 INFO:teuthology.orchestra.run.gibba001.stdout: debug client = 10 2021-02-10T11:11:15.843 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.843 INFO:teuthology.orchestra.run.gibba001.stdout:[mon] 2021-02-10T11:11:15.843 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 1 2021-02-10T11:11:15.843 INFO:teuthology.orchestra.run.gibba001.stdout: debug mon = 20 2021-02-10T11:11:15.843 INFO:teuthology.orchestra.run.gibba001.stdout: debug paxos = 20 2021-02-10T11:11:15.843 INFO:teuthology.orchestra.run.gibba001.stdout: debug auth = 20 2021-02-10T11:11:15.843 INFO:teuthology.orchestra.run.gibba001.stdout: mon data avail warn = 5 2021-02-10T11:11:15.843 INFO:teuthology.orchestra.run.gibba001.stdout: mon mgr mkfs grace = 240 2021-02-10T11:11:15.843 INFO:teuthology.orchestra.run.gibba001.stdout: mon reweight min pgs per osd = 4 2021-02-10T11:11:15.843 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd reporter subtree level = osd 2021-02-10T11:11:15.843 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd prime pg temp = true 2021-02-10T11:11:15.843 INFO:teuthology.orchestra.run.gibba001.stdout: mon reweight min bytes per osd = 10 2021-02-10T11:11:15.843 INFO:teuthology.orchestra.run.gibba001.stdout: mon op complaint time = 120 2021-02-10T11:11:15.843 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:11:15.844 INFO:teuthology.orchestra.run.gibba001.stdout:[client] 2021-02-10T11:11:15.844 INFO:teuthology.orchestra.run.gibba001.stdout: rgw cache enabled = true 2021-02-10T11:11:15.844 INFO:teuthology.orchestra.run.gibba001.stdout: rgw enable ops log = true 2021-02-10T11:11:15.844 INFO:teuthology.orchestra.run.gibba001.stdout: rgw enable usage log = true 2021-02-10T11:11:15.844 INFO:teuthology.orchestra.run.gibba001.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-02-10T11:11:15.844 INFO:teuthology.orchestra.run.gibba001.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-02-10T11:11:15.844 INFO:teuthology.orchestra.run.gibba001.stdout: client mount timeout = 600 2021-02-10T11:11:15.844 INFO:teuthology.orchestra.run.gibba001.stdout: debug client = 20 2021-02-10T11:11:15.844 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 1 2021-02-10T11:11:15.844 INFO:teuthology.orchestra.run.gibba001.stdout: pid file = "" 2021-02-10T11:11:15.844 INFO:teuthology.orchestra.run.gibba001.stdout:[mon.a] 2021-02-10T11:11:15.844 INFO:teuthology.orchestra.run.gibba001.stdout:[mon.b] 2021-02-10T11:11:15.844 INFO:teuthology.orchestra.run.gibba001.stdout:[mon.c] 2021-02-10T11:11:15.845 INFO:teuthology.orchestra.run.gibba001.stdout:[mds.a] 2021-02-10T11:11:15.845 INFO:teuthology.orchestra.run.gibba001.stdout:[mds.b] 2021-02-10T11:11:15.845 INFO:teuthology.orchestra.run.gibba001.stdout:[mds] 2021-02-10T11:11:15.845 INFO:teuthology.orchestra.run.gibba001.stdout: debug mds = 20 2021-02-10T11:11:15.845 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 1 2021-02-10T11:11:15.845 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal frag = True 2021-02-10T11:11:15.845 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal fragment size max = 10000 2021-02-10T11:11:15.845 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal merge size = 5 2021-02-10T11:11:15.845 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal split bits = 3 2021-02-10T11:11:15.845 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal split size = 100 2021-02-10T11:11:15.845 INFO:teuthology.orchestra.run.gibba001.stdout: mds debug frag = True 2021-02-10T11:11:15.845 INFO:teuthology.orchestra.run.gibba001.stdout: mds debug scatterstat = True 2021-02-10T11:11:15.845 INFO:teuthology.orchestra.run.gibba001.stdout: mds op complaint time = 180 2021-02-10T11:11:15.845 INFO:teuthology.orchestra.run.gibba001.stdout: mds verify scatter = True 2021-02-10T11:11:15.846 INFO:teuthology.orchestra.run.gibba001.stdout: osd op complaint time = 180 2021-02-10T11:11:15.846 INFO:teuthology.orchestra.run.gibba008.stdout:[global] 2021-02-10T11:11:15.847 INFO:teuthology.orchestra.run.gibba008.stdout: chdir = "" 2021-02-10T11:11:15.847 INFO:teuthology.orchestra.run.gibba008.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-02-10T11:11:15.847 INFO:teuthology.orchestra.run.gibba008.stdout: auth supported = cephx 2021-02-10T11:11:15.847 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.847 INFO:teuthology.orchestra.run.gibba008.stdout: filestore xattr use omap = true 2021-02-10T11:11:15.847 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.847 INFO:teuthology.orchestra.run.gibba008.stdout: mon clock drift allowed = 1.000 2021-02-10T11:11:15.847 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.847 INFO:teuthology.orchestra.run.gibba008.stdout: osd crush chooseleaf type = 0 2021-02-10T11:11:15.847 INFO:teuthology.orchestra.run.gibba008.stdout: auth debug = true 2021-02-10T11:11:15.847 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.847 INFO:teuthology.orchestra.run.gibba008.stdout: ms die on old message = true 2021-02-10T11:11:15.848 INFO:teuthology.orchestra.run.gibba008.stdout: ms die on bug = true 2021-02-10T11:11:15.848 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.848 INFO:teuthology.orchestra.run.gibba008.stdout: mon max pg per osd = 10000 # >= luminous 2021-02-10T11:11:15.848 INFO:teuthology.orchestra.run.gibba008.stdout: mon pg warn max object skew = 0 2021-02-10T11:11:15.848 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.848 INFO:teuthology.orchestra.run.gibba008.stdout: osd pool default size = 2 2021-02-10T11:11:15.848 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.848 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd allow primary affinity = true 2021-02-10T11:11:15.848 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd allow pg remap = true 2021-02-10T11:11:15.848 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on legacy crush tunables = false 2021-02-10T11:11:15.848 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on crush straw calc version zero = false 2021-02-10T11:11:15.848 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on no sortbitwise = false 2021-02-10T11:11:15.848 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on osd down out interval zero = false 2021-02-10T11:11:15.848 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on too few osds = false 2021-02-10T11:11:15.849 INFO:teuthology.orchestra.run.gibba008.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-02-10T11:11:15.849 INFO:teuthology.orchestra.run.gibba008.stdout: mon_warn_on_pool_no_redundancy = false 2021-02-10T11:11:15.849 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.849 INFO:teuthology.orchestra.run.gibba008.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-02-10T11:11:15.849 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.849 INFO:teuthology.orchestra.run.gibba008.stdout: osd default data pool replay window = 5 2021-02-10T11:11:15.849 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.849 INFO:teuthology.orchestra.run.gibba008.stdout: mon allow pool delete = true 2021-02-10T11:11:15.849 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.849 INFO:teuthology.orchestra.run.gibba008.stdout: mon cluster log file level = debug 2021-02-10T11:11:15.849 INFO:teuthology.orchestra.run.gibba008.stdout: debug asserts on shutdown = true 2021-02-10T11:11:15.849 INFO:teuthology.orchestra.run.gibba008.stdout: mon health detail to clog = false 2021-02-10T11:11:15.849 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.850 INFO:teuthology.orchestra.run.gibba008.stdout: # we see this fail in qa on *nautilus*; bump up retries 2021-02-10T11:11:15.850 INFO:teuthology.orchestra.run.gibba008.stdout: mon_client_directed_command_retry = 4 2021-02-10T11:11:15.850 INFO:teuthology.orchestra.run.gibba008.stdout: mon host = "172.21.2.101,172.21.2.108,[v2:172.21.2.108:3301,v1:172.21.2.108:6790]" 2021-02-10T11:11:15.850 INFO:teuthology.orchestra.run.gibba008.stdout: fsid = a6663ba9-2653-4542-8b6a-9323b4e7b545 2021-02-10T11:11:15.850 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.850 INFO:teuthology.orchestra.run.gibba008.stdout:[osd] 2021-02-10T11:11:15.850 INFO:teuthology.orchestra.run.gibba008.stdout: osd journal size = 100 2021-02-10T11:11:15.850 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.850 INFO:teuthology.orchestra.run.gibba008.stdout: osd scrub load threshold = 5.0 2021-02-10T11:11:15.850 INFO:teuthology.orchestra.run.gibba008.stdout: osd scrub max interval = 600 2021-02-10T11:11:15.850 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.851 INFO:teuthology.orchestra.run.gibba008.stdout: osd recover clone overlap = true 2021-02-10T11:11:15.851 INFO:teuthology.orchestra.run.gibba008.stdout: osd recovery max chunk = 1048576 2021-02-10T11:11:15.851 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.851 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug shutdown = true 2021-02-10T11:11:15.851 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug op order = true 2021-02-10T11:11:15.851 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug verify stray on activate = true 2021-02-10T11:11:15.851 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.851 INFO:teuthology.orchestra.run.gibba008.stdout: osd open classes on start = true 2021-02-10T11:11:15.851 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug pg log writeout = true 2021-02-10T11:11:15.851 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.851 INFO:teuthology.orchestra.run.gibba008.stdout: osd deep scrub update digest min age = 30 2021-02-10T11:11:15.851 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.852 INFO:teuthology.orchestra.run.gibba008.stdout: osd map max advance = 10 2021-02-10T11:11:15.852 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.852 INFO:teuthology.orchestra.run.gibba008.stdout: journal zero on create = true 2021-02-10T11:11:15.852 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.852 INFO:teuthology.orchestra.run.gibba008.stdout: filestore ondisk finisher threads = 3 2021-02-10T11:11:15.852 INFO:teuthology.orchestra.run.gibba008.stdout: filestore apply finisher threads = 3 2021-02-10T11:11:15.852 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.852 INFO:teuthology.orchestra.run.gibba008.stdout: bdev debug aio = true 2021-02-10T11:11:15.852 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug misdirected ops = true 2021-02-10T11:11:15.852 INFO:teuthology.orchestra.run.gibba008.stdout: bdev async discard = True 2021-02-10T11:11:15.852 INFO:teuthology.orchestra.run.gibba008.stdout: bdev enable discard = True 2021-02-10T11:11:15.853 INFO:teuthology.orchestra.run.gibba008.stdout: bluestore allocator = bitmap 2021-02-10T11:11:15.853 INFO:teuthology.orchestra.run.gibba008.stdout: bluestore block size = 96636764160 2021-02-10T11:11:15.853 INFO:teuthology.orchestra.run.gibba008.stdout: bluestore fsck on mount = True 2021-02-10T11:11:15.853 INFO:teuthology.orchestra.run.gibba008.stdout: debug bluefs = 20 2021-02-10T11:11:15.853 INFO:teuthology.orchestra.run.gibba008.stdout: debug bluestore = 20 2021-02-10T11:11:15.853 INFO:teuthology.orchestra.run.gibba008.stdout: debug filestore = 20 2021-02-10T11:11:15.853 INFO:teuthology.orchestra.run.gibba008.stdout: debug journal = 20 2021-02-10T11:11:15.853 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 20 2021-02-10T11:11:15.853 INFO:teuthology.orchestra.run.gibba008.stdout: debug osd = 25 2021-02-10T11:11:15.853 INFO:teuthology.orchestra.run.gibba008.stdout: debug rocksdb = 10 2021-02-10T11:11:15.853 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd backfillfull_ratio = 0.85 2021-02-10T11:11:15.853 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd full ratio = 0.9 2021-02-10T11:11:15.853 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd nearfull ratio = 0.8 2021-02-10T11:11:15.854 INFO:teuthology.orchestra.run.gibba008.stdout: osd failsafe full ratio = 0.95 2021-02-10T11:11:15.854 INFO:teuthology.orchestra.run.gibba008.stdout: osd objectstore = bluestore 2021-02-10T11:11:15.854 INFO:teuthology.orchestra.run.gibba008.stdout: osd op complaint time = 180 2021-02-10T11:11:15.854 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.854 INFO:teuthology.orchestra.run.gibba008.stdout:[mgr] 2021-02-10T11:11:15.854 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 1 2021-02-10T11:11:15.854 INFO:teuthology.orchestra.run.gibba008.stdout: debug mgr = 20 2021-02-10T11:11:15.854 INFO:teuthology.orchestra.run.gibba008.stdout: debug mon = 20 2021-02-10T11:11:15.854 INFO:teuthology.orchestra.run.gibba008.stdout: debug auth = 20 2021-02-10T11:11:15.854 INFO:teuthology.orchestra.run.gibba008.stdout: mon reweight min pgs per osd = 4 2021-02-10T11:11:15.854 INFO:teuthology.orchestra.run.gibba008.stdout: mon reweight min bytes per osd = 10 2021-02-10T11:11:15.854 INFO:teuthology.orchestra.run.gibba008.stdout: debug client = 10 2021-02-10T11:11:15.855 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.855 INFO:teuthology.orchestra.run.gibba008.stdout:[mon] 2021-02-10T11:11:15.855 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 1 2021-02-10T11:11:15.855 INFO:teuthology.orchestra.run.gibba008.stdout: debug mon = 20 2021-02-10T11:11:15.855 INFO:teuthology.orchestra.run.gibba008.stdout: debug paxos = 20 2021-02-10T11:11:15.855 INFO:teuthology.orchestra.run.gibba008.stdout: debug auth = 20 2021-02-10T11:11:15.855 INFO:teuthology.orchestra.run.gibba008.stdout: mon data avail warn = 5 2021-02-10T11:11:15.855 INFO:teuthology.orchestra.run.gibba008.stdout: mon mgr mkfs grace = 240 2021-02-10T11:11:15.855 INFO:teuthology.orchestra.run.gibba008.stdout: mon reweight min pgs per osd = 4 2021-02-10T11:11:15.855 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd reporter subtree level = osd 2021-02-10T11:11:15.855 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd prime pg temp = true 2021-02-10T11:11:15.855 INFO:teuthology.orchestra.run.gibba008.stdout: mon reweight min bytes per osd = 10 2021-02-10T11:11:15.856 INFO:teuthology.orchestra.run.gibba008.stdout: mon op complaint time = 120 2021-02-10T11:11:15.856 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:11:15.856 INFO:teuthology.orchestra.run.gibba008.stdout:[client] 2021-02-10T11:11:15.856 INFO:teuthology.orchestra.run.gibba008.stdout: rgw cache enabled = true 2021-02-10T11:11:15.856 INFO:teuthology.orchestra.run.gibba008.stdout: rgw enable ops log = true 2021-02-10T11:11:15.856 INFO:teuthology.orchestra.run.gibba008.stdout: rgw enable usage log = true 2021-02-10T11:11:15.856 INFO:teuthology.orchestra.run.gibba008.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-02-10T11:11:15.856 INFO:teuthology.orchestra.run.gibba008.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-02-10T11:11:15.856 INFO:teuthology.orchestra.run.gibba008.stdout: client mount timeout = 600 2021-02-10T11:11:15.856 INFO:teuthology.orchestra.run.gibba008.stdout: debug client = 20 2021-02-10T11:11:15.856 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 1 2021-02-10T11:11:15.857 INFO:teuthology.orchestra.run.gibba008.stdout: pid file = "" 2021-02-10T11:11:15.857 INFO:teuthology.orchestra.run.gibba008.stdout:[mon.a] 2021-02-10T11:11:15.857 INFO:teuthology.orchestra.run.gibba008.stdout:[mon.b] 2021-02-10T11:11:15.857 INFO:teuthology.orchestra.run.gibba008.stdout:[mon.c] 2021-02-10T11:11:15.857 INFO:teuthology.orchestra.run.gibba008.stdout:[mds.a] 2021-02-10T11:11:15.857 INFO:teuthology.orchestra.run.gibba008.stdout:[mds.b] 2021-02-10T11:11:15.857 INFO:teuthology.orchestra.run.gibba008.stdout:[mds] 2021-02-10T11:11:15.857 INFO:teuthology.orchestra.run.gibba008.stdout: debug mds = 20 2021-02-10T11:11:15.857 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 1 2021-02-10T11:11:15.857 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal frag = True 2021-02-10T11:11:15.857 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal fragment size max = 10000 2021-02-10T11:11:15.858 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal merge size = 5 2021-02-10T11:11:15.858 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal split bits = 3 2021-02-10T11:11:15.858 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal split size = 100 2021-02-10T11:11:15.858 INFO:teuthology.orchestra.run.gibba008.stdout: mds debug frag = True 2021-02-10T11:11:15.858 INFO:teuthology.orchestra.run.gibba008.stdout: mds debug scatterstat = True 2021-02-10T11:11:15.858 INFO:teuthology.orchestra.run.gibba008.stdout: mds op complaint time = 180 2021-02-10T11:11:15.858 INFO:teuthology.orchestra.run.gibba008.stdout: mds verify scatter = True 2021-02-10T11:11:15.858 INFO:teuthology.orchestra.run.gibba008.stdout: osd op complaint time = 180 2021-02-10T11:11:15.859 INFO:tasks.ceph:Creating admin key on mon.a... 2021-02-10T11:11:15.859 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2021-02-10T11:11:16.237 INFO:tasks.ceph:Copying monmap to all nodes... 2021-02-10T11:11:16.577 DEBUG:teuthology.orchestra.remote:gibba001:/etc/ceph/ceph.keyring is 206B 2021-02-10T11:11:17.254 DEBUG:teuthology.orchestra.remote:gibba001:/home/ubuntu/cephtest/ceph.monmap is 373B 2021-02-10T11:11:17.593 INFO:tasks.ceph:Sending monmap to node ubuntu@gibba001.front.sepia.ceph.com 2021-02-10T11:11:17.593 DEBUG:teuthology.orchestra.run.gibba001:> set -ex 2021-02-10T11:11:17.593 DEBUG:teuthology.orchestra.run.gibba001:> sudo dd of=/etc/ceph/ceph.keyring 2021-02-10T11:11:17.593 DEBUG:teuthology.orchestra.run.gibba001:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-02-10T11:11:17.945 DEBUG:teuthology.orchestra.run.gibba001:> set -ex 2021-02-10T11:11:17.945 DEBUG:teuthology.orchestra.run.gibba001:> dd of=/home/ubuntu/cephtest/ceph.monmap 2021-02-10T11:11:18.280 INFO:tasks.ceph:Sending monmap to node ubuntu@gibba008.front.sepia.ceph.com 2021-02-10T11:11:18.280 DEBUG:teuthology.orchestra.run.gibba008:> set -ex 2021-02-10T11:11:18.280 DEBUG:teuthology.orchestra.run.gibba008:> sudo dd of=/etc/ceph/ceph.keyring 2021-02-10T11:11:18.280 DEBUG:teuthology.orchestra.run.gibba008:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-02-10T11:11:18.633 DEBUG:teuthology.orchestra.run.gibba008:> set -ex 2021-02-10T11:11:18.634 DEBUG:teuthology.orchestra.run.gibba008:> dd of=/home/ubuntu/cephtest/ceph.monmap 2021-02-10T11:11:18.967 INFO:tasks.ceph:Setting up mon nodes... 2021-02-10T11:11:18.967 INFO:tasks.ceph:Setting up mgr nodes... 2021-02-10T11:11:18.967 DEBUG:teuthology.orchestra.run.gibba001:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2021-02-10T11:11:19.344 INFO:teuthology.orchestra.run.gibba001.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2021-02-10T11:11:19.346 DEBUG:teuthology.orchestra.run.gibba008:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2021-02-10T11:11:19.750 INFO:teuthology.orchestra.run.gibba008.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2021-02-10T11:11:19.753 INFO:tasks.ceph:Setting up mds nodes... 2021-02-10T11:11:19.753 DEBUG:teuthology.orchestra.run.gibba001:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2021-02-10T11:11:20.130 INFO:teuthology.orchestra.run.gibba001.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2021-02-10T11:11:20.132 DEBUG:teuthology.orchestra.run.gibba001:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2021-02-10T11:11:20.475 DEBUG:teuthology.orchestra.run.gibba008:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2021-02-10T11:11:20.851 INFO:teuthology.orchestra.run.gibba008.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2021-02-10T11:11:20.853 DEBUG:teuthology.orchestra.run.gibba008:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2021-02-10T11:11:21.196 INFO:tasks.ceph_client:Setting up client nodes... 2021-02-10T11:11:21.197 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2021-02-10T11:11:21.565 INFO:teuthology.orchestra.run.gibba001.stdout:creating /etc/ceph/ceph.client.0.keyring 2021-02-10T11:11:21.577 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2021-02-10T11:11:21.942 INFO:teuthology.orchestra.run.gibba001.stdout:creating /etc/ceph/ceph.client.1.keyring 2021-02-10T11:11:21.954 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2021-02-10T11:11:22.323 INFO:teuthology.orchestra.run.gibba008.stdout:creating /etc/ceph/ceph.client.2.keyring 2021-02-10T11:11:22.335 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2021-02-10T11:11:22.701 INFO:teuthology.orchestra.run.gibba008.stdout:creating /etc/ceph/ceph.client.3.keyring 2021-02-10T11:11:22.713 INFO:tasks.ceph:Running mkfs on osd nodes... 2021-02-10T11:11:22.713 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@gibba001.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@gibba008.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2021-02-10T11:11:22.714 DEBUG:teuthology.orchestra.run.gibba001:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2021-02-10T11:11:23.058 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-02-10T11:11:23.058 INFO:tasks.ceph:role: osd.0 2021-02-10T11:11:23.058 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@gibba001.front.sepia.ceph.com 2021-02-10T11:11:23.059 DEBUG:teuthology.orchestra.run.gibba001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2021-02-10T11:11:23.439 INFO:teuthology.orchestra.run.gibba001.stdout:Discarding blocks...Done. 2021-02-10T11:11:23.439 INFO:teuthology.orchestra.run.gibba001.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5505024 blks 2021-02-10T11:11:23.439 INFO:teuthology.orchestra.run.gibba001.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-10T11:11:23.439 INFO:teuthology.orchestra.run.gibba001.stdout: = crc=1 finobt=0, sparse=0 2021-02-10T11:11:23.439 INFO:teuthology.orchestra.run.gibba001.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-02-10T11:11:23.439 INFO:teuthology.orchestra.run.gibba001.stdout: = sunit=0 swidth=0 blks 2021-02-10T11:11:23.439 INFO:teuthology.orchestra.run.gibba001.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-10T11:11:23.439 INFO:teuthology.orchestra.run.gibba001.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-02-10T11:11:23.439 INFO:teuthology.orchestra.run.gibba001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-10T11:11:23.439 INFO:teuthology.orchestra.run.gibba001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-10T11:11:23.440 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@gibba001.front.sepia.ceph.com -o noatime 2021-02-10T11:11:23.440 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2021-02-10T11:11:23.799 DEBUG:teuthology.orchestra.run.gibba001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2021-02-10T11:11:24.146 DEBUG:teuthology.orchestra.run.gibba001:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2021-02-10T11:11:24.489 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-02-10T11:11:24.489 INFO:tasks.ceph:role: osd.1 2021-02-10T11:11:24.489 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@gibba001.front.sepia.ceph.com 2021-02-10T11:11:24.490 DEBUG:teuthology.orchestra.run.gibba001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2021-02-10T11:11:24.872 INFO:teuthology.orchestra.run.gibba001.stdout:Discarding blocks...Done. 2021-02-10T11:11:24.873 INFO:teuthology.orchestra.run.gibba001.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5505024 blks 2021-02-10T11:11:24.873 INFO:teuthology.orchestra.run.gibba001.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-10T11:11:24.873 INFO:teuthology.orchestra.run.gibba001.stdout: = crc=1 finobt=0, sparse=0 2021-02-10T11:11:24.873 INFO:teuthology.orchestra.run.gibba001.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-02-10T11:11:24.873 INFO:teuthology.orchestra.run.gibba001.stdout: = sunit=0 swidth=0 blks 2021-02-10T11:11:24.873 INFO:teuthology.orchestra.run.gibba001.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-10T11:11:24.874 INFO:teuthology.orchestra.run.gibba001.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-02-10T11:11:24.874 INFO:teuthology.orchestra.run.gibba001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-10T11:11:24.874 INFO:teuthology.orchestra.run.gibba001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-10T11:11:24.876 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@gibba001.front.sepia.ceph.com -o noatime 2021-02-10T11:11:24.876 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2021-02-10T11:11:25.232 DEBUG:teuthology.orchestra.run.gibba001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2021-02-10T11:11:25.577 DEBUG:teuthology.orchestra.run.gibba001:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2021-02-10T11:11:25.919 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-02-10T11:11:25.919 INFO:tasks.ceph:role: osd.2 2021-02-10T11:11:25.919 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@gibba001.front.sepia.ceph.com 2021-02-10T11:11:25.919 DEBUG:teuthology.orchestra.run.gibba001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2021-02-10T11:11:26.302 INFO:teuthology.orchestra.run.gibba001.stdout:Discarding blocks...Done. 2021-02-10T11:11:26.303 INFO:teuthology.orchestra.run.gibba001.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5505024 blks 2021-02-10T11:11:26.303 INFO:teuthology.orchestra.run.gibba001.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-10T11:11:26.303 INFO:teuthology.orchestra.run.gibba001.stdout: = crc=1 finobt=0, sparse=0 2021-02-10T11:11:26.303 INFO:teuthology.orchestra.run.gibba001.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-02-10T11:11:26.303 INFO:teuthology.orchestra.run.gibba001.stdout: = sunit=0 swidth=0 blks 2021-02-10T11:11:26.303 INFO:teuthology.orchestra.run.gibba001.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-10T11:11:26.303 INFO:teuthology.orchestra.run.gibba001.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-02-10T11:11:26.303 INFO:teuthology.orchestra.run.gibba001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-10T11:11:26.303 INFO:teuthology.orchestra.run.gibba001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-10T11:11:26.304 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@gibba001.front.sepia.ceph.com -o noatime 2021-02-10T11:11:26.304 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2021-02-10T11:11:26.661 DEBUG:teuthology.orchestra.run.gibba001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2021-02-10T11:11:27.005 DEBUG:teuthology.orchestra.run.gibba001:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2021-02-10T11:11:27.347 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-02-10T11:11:27.399 INFO:tasks.ceph:role: osd.3 2021-02-10T11:11:27.399 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@gibba001.front.sepia.ceph.com 2021-02-10T11:11:27.399 DEBUG:teuthology.orchestra.run.gibba001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2021-02-10T11:11:27.785 INFO:teuthology.orchestra.run.gibba001.stdout:Discarding blocks...Done. 2021-02-10T11:11:27.785 INFO:teuthology.orchestra.run.gibba001.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5505024 blks 2021-02-10T11:11:27.785 INFO:teuthology.orchestra.run.gibba001.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-10T11:11:27.786 INFO:teuthology.orchestra.run.gibba001.stdout: = crc=1 finobt=0, sparse=0 2021-02-10T11:11:27.786 INFO:teuthology.orchestra.run.gibba001.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-02-10T11:11:27.786 INFO:teuthology.orchestra.run.gibba001.stdout: = sunit=0 swidth=0 blks 2021-02-10T11:11:27.786 INFO:teuthology.orchestra.run.gibba001.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-10T11:11:27.786 INFO:teuthology.orchestra.run.gibba001.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-02-10T11:11:27.786 INFO:teuthology.orchestra.run.gibba001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-10T11:11:27.786 INFO:teuthology.orchestra.run.gibba001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-10T11:11:27.787 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@gibba001.front.sepia.ceph.com -o noatime 2021-02-10T11:11:27.787 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2021-02-10T11:11:28.144 DEBUG:teuthology.orchestra.run.gibba001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2021-02-10T11:11:28.490 DEBUG:teuthology.orchestra.run.gibba001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-02-10T11:11:28.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:28.857 7fb02ae55a80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2021-02-10T11:11:28.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:28.857 7fb02ae55a80 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2021-02-10T11:11:28.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:28.857 7fb02ae55a80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2021-02-10T11:11:28.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:28.857 7fb02ae55a80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2021-02-10T11:11:28.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:28.857 7fb02ae55a80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2021-02-10T11:11:28.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:28.858 7fb02ae55a80 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2021-02-10T11:11:31.152 DEBUG:teuthology.orchestra.run.gibba001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2021-02-10T11:11:31.495 DEBUG:teuthology.orchestra.run.gibba001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-02-10T11:11:31.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:31.878 7f153ff1ea80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2021-02-10T11:11:31.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:31.879 7f153ff1ea80 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2021-02-10T11:11:31.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:31.879 7f153ff1ea80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2021-02-10T11:11:31.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:31.879 7f153ff1ea80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2021-02-10T11:11:31.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:31.879 7f153ff1ea80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2021-02-10T11:11:31.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:31.880 7f153ff1ea80 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2021-02-10T11:11:34.172 DEBUG:teuthology.orchestra.run.gibba001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2021-02-10T11:11:34.516 DEBUG:teuthology.orchestra.run.gibba001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-02-10T11:11:34.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:34.884 7f976ed19a80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2021-02-10T11:11:34.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:34.884 7f976ed19a80 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2021-02-10T11:11:34.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:34.884 7f976ed19a80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2021-02-10T11:11:34.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:34.884 7f976ed19a80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2021-02-10T11:11:34.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:34.884 7f976ed19a80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2021-02-10T11:11:34.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:34.885 7f976ed19a80 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2021-02-10T11:11:37.181 DEBUG:teuthology.orchestra.run.gibba001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2021-02-10T11:11:37.525 DEBUG:teuthology.orchestra.run.gibba001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-02-10T11:11:37.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:37.890 7fe7f36cba80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2021-02-10T11:11:37.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:37.890 7fe7f36cba80 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2021-02-10T11:11:37.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:37.890 7fe7f36cba80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2021-02-10T11:11:37.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:37.890 7fe7f36cba80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2021-02-10T11:11:37.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:37.890 7fe7f36cba80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2021-02-10T11:11:37.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:11:37.891 7fe7f36cba80 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2021-02-10T11:11:39.838 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:11:39.842 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:11:40.190 DEBUG:teuthology.orchestra.run.gibba001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2021-02-10T11:11:40.534 DEBUG:teuthology.orchestra.run.gibba008:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2021-02-10T11:11:40.878 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-02-10T11:11:40.878 INFO:tasks.ceph:role: osd.4 2021-02-10T11:11:40.878 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@gibba008.front.sepia.ceph.com 2021-02-10T11:11:40.878 DEBUG:teuthology.orchestra.run.gibba008:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2021-02-10T11:11:41.268 INFO:teuthology.orchestra.run.gibba008.stdout:Discarding blocks...Done. 2021-02-10T11:11:41.269 INFO:teuthology.orchestra.run.gibba008.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5505024 blks 2021-02-10T11:11:41.269 INFO:teuthology.orchestra.run.gibba008.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-10T11:11:41.269 INFO:teuthology.orchestra.run.gibba008.stdout: = crc=1 finobt=0, sparse=0 2021-02-10T11:11:41.269 INFO:teuthology.orchestra.run.gibba008.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-02-10T11:11:41.270 INFO:teuthology.orchestra.run.gibba008.stdout: = sunit=0 swidth=0 blks 2021-02-10T11:11:41.270 INFO:teuthology.orchestra.run.gibba008.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-10T11:11:41.270 INFO:teuthology.orchestra.run.gibba008.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-02-10T11:11:41.270 INFO:teuthology.orchestra.run.gibba008.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-10T11:11:41.270 INFO:teuthology.orchestra.run.gibba008.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-10T11:11:41.272 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@gibba008.front.sepia.ceph.com -o noatime 2021-02-10T11:11:41.272 DEBUG:teuthology.orchestra.run.gibba008:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2021-02-10T11:11:41.631 DEBUG:teuthology.orchestra.run.gibba008:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2021-02-10T11:11:41.978 DEBUG:teuthology.orchestra.run.gibba008:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2021-02-10T11:11:42.321 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-02-10T11:11:42.322 INFO:tasks.ceph:role: osd.5 2021-02-10T11:11:42.322 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@gibba008.front.sepia.ceph.com 2021-02-10T11:11:42.322 DEBUG:teuthology.orchestra.run.gibba008:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2021-02-10T11:11:42.716 INFO:teuthology.orchestra.run.gibba008.stdout:Discarding blocks...Done. 2021-02-10T11:11:42.716 INFO:teuthology.orchestra.run.gibba008.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5505024 blks 2021-02-10T11:11:42.716 INFO:teuthology.orchestra.run.gibba008.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-10T11:11:42.716 INFO:teuthology.orchestra.run.gibba008.stdout: = crc=1 finobt=0, sparse=0 2021-02-10T11:11:42.716 INFO:teuthology.orchestra.run.gibba008.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-02-10T11:11:42.716 INFO:teuthology.orchestra.run.gibba008.stdout: = sunit=0 swidth=0 blks 2021-02-10T11:11:42.716 INFO:teuthology.orchestra.run.gibba008.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-10T11:11:42.716 INFO:teuthology.orchestra.run.gibba008.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-02-10T11:11:42.716 INFO:teuthology.orchestra.run.gibba008.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-10T11:11:42.716 INFO:teuthology.orchestra.run.gibba008.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-10T11:11:42.717 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@gibba008.front.sepia.ceph.com -o noatime 2021-02-10T11:11:42.717 DEBUG:teuthology.orchestra.run.gibba008:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2021-02-10T11:11:43.074 DEBUG:teuthology.orchestra.run.gibba008:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2021-02-10T11:11:43.419 DEBUG:teuthology.orchestra.run.gibba008:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2021-02-10T11:11:43.762 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-02-10T11:11:43.762 INFO:tasks.ceph:role: osd.6 2021-02-10T11:11:43.762 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@gibba008.front.sepia.ceph.com 2021-02-10T11:11:43.762 DEBUG:teuthology.orchestra.run.gibba008:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2021-02-10T11:11:44.154 INFO:teuthology.orchestra.run.gibba008.stdout:Discarding blocks...Done. 2021-02-10T11:11:44.155 INFO:teuthology.orchestra.run.gibba008.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5505024 blks 2021-02-10T11:11:44.155 INFO:teuthology.orchestra.run.gibba008.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-10T11:11:44.155 INFO:teuthology.orchestra.run.gibba008.stdout: = crc=1 finobt=0, sparse=0 2021-02-10T11:11:44.155 INFO:teuthology.orchestra.run.gibba008.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-02-10T11:11:44.155 INFO:teuthology.orchestra.run.gibba008.stdout: = sunit=0 swidth=0 blks 2021-02-10T11:11:44.155 INFO:teuthology.orchestra.run.gibba008.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-10T11:11:44.155 INFO:teuthology.orchestra.run.gibba008.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-02-10T11:11:44.155 INFO:teuthology.orchestra.run.gibba008.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-10T11:11:44.155 INFO:teuthology.orchestra.run.gibba008.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-10T11:11:44.156 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@gibba008.front.sepia.ceph.com -o noatime 2021-02-10T11:11:44.156 DEBUG:teuthology.orchestra.run.gibba008:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2021-02-10T11:11:44.512 DEBUG:teuthology.orchestra.run.gibba008:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2021-02-10T11:11:44.856 DEBUG:teuthology.orchestra.run.gibba008:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2021-02-10T11:11:45.200 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-02-10T11:11:45.200 INFO:tasks.ceph:role: osd.7 2021-02-10T11:11:45.200 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@gibba008.front.sepia.ceph.com 2021-02-10T11:11:45.200 DEBUG:teuthology.orchestra.run.gibba008:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2021-02-10T11:11:45.588 INFO:teuthology.orchestra.run.gibba008.stdout:Discarding blocks...Done. 2021-02-10T11:11:45.588 INFO:teuthology.orchestra.run.gibba008.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5505024 blks 2021-02-10T11:11:45.588 INFO:teuthology.orchestra.run.gibba008.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-10T11:11:45.588 INFO:teuthology.orchestra.run.gibba008.stdout: = crc=1 finobt=0, sparse=0 2021-02-10T11:11:45.588 INFO:teuthology.orchestra.run.gibba008.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-02-10T11:11:45.589 INFO:teuthology.orchestra.run.gibba008.stdout: = sunit=0 swidth=0 blks 2021-02-10T11:11:45.589 INFO:teuthology.orchestra.run.gibba008.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-10T11:11:45.589 INFO:teuthology.orchestra.run.gibba008.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-02-10T11:11:45.589 INFO:teuthology.orchestra.run.gibba008.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-10T11:11:45.589 INFO:teuthology.orchestra.run.gibba008.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-10T11:11:45.591 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@gibba008.front.sepia.ceph.com -o noatime 2021-02-10T11:11:45.591 DEBUG:teuthology.orchestra.run.gibba008:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2021-02-10T11:11:45.948 DEBUG:teuthology.orchestra.run.gibba008:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2021-02-10T11:11:46.293 DEBUG:teuthology.orchestra.run.gibba008:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-02-10T11:11:46.663 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:46.664 7fce20c7ca80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2021-02-10T11:11:46.663 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:46.664 7fce20c7ca80 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2021-02-10T11:11:46.663 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:46.664 7fce20c7ca80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2021-02-10T11:11:46.663 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:46.664 7fce20c7ca80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2021-02-10T11:11:46.663 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:46.664 7fce20c7ca80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2021-02-10T11:11:46.664 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:46.666 7fce20c7ca80 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2021-02-10T11:11:48.958 DEBUG:teuthology.orchestra.run.gibba008:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2021-02-10T11:11:49.309 DEBUG:teuthology.orchestra.run.gibba008:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-02-10T11:11:49.678 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:49.680 7fb286375a80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2021-02-10T11:11:49.679 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:49.680 7fb286375a80 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2021-02-10T11:11:49.679 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:49.680 7fb286375a80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2021-02-10T11:11:49.679 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:49.680 7fb286375a80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2021-02-10T11:11:49.679 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:49.680 7fb286375a80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2021-02-10T11:11:49.679 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:49.681 7fb286375a80 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2021-02-10T11:11:51.970 DEBUG:teuthology.orchestra.run.gibba008:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2021-02-10T11:11:52.312 DEBUG:teuthology.orchestra.run.gibba008:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-02-10T11:11:52.681 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:52.683 7f01632f0a80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2021-02-10T11:11:52.681 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:52.683 7f01632f0a80 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2021-02-10T11:11:52.681 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:52.683 7f01632f0a80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2021-02-10T11:11:52.682 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:52.683 7f01632f0a80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2021-02-10T11:11:52.682 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:52.683 7f01632f0a80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2021-02-10T11:11:52.682 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:52.684 7f01632f0a80 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2021-02-10T11:11:54.972 DEBUG:teuthology.orchestra.run.gibba008:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2021-02-10T11:11:55.317 DEBUG:teuthology.orchestra.run.gibba008:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-02-10T11:11:55.686 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:55.688 7f729dd5fa80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2021-02-10T11:11:55.686 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:55.688 7f729dd5fa80 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2021-02-10T11:11:55.687 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:55.688 7f729dd5fa80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2021-02-10T11:11:55.687 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:55.688 7f729dd5fa80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2021-02-10T11:11:55.687 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:55.688 7f729dd5fa80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2021-02-10T11:11:55.687 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:11:55.689 7f729dd5fa80 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2021-02-10T11:11:57.981 DEBUG:teuthology.orchestra.run.gibba008:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2021-02-10T11:11:58.324 INFO:tasks.ceph:Reading keys from all nodes... 2021-02-10T11:11:58.325 DEBUG:teuthology.orchestra.run.gibba001:> mktemp 2021-02-10T11:11:58.660 INFO:teuthology.orchestra.run.gibba001.stdout:/tmp/tmp.RY3fBuMLIo 2021-02-10T11:11:58.660 DEBUG:teuthology.orchestra.run.gibba001:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.RY3fBuMLIo 2021-02-10T11:11:59.004 DEBUG:teuthology.orchestra.run.gibba001:> sudo chmod 0666 /tmp/tmp.RY3fBuMLIo 2021-02-10T11:11:59.688 DEBUG:teuthology.orchestra.remote:gibba001:/tmp/tmp.RY3fBuMLIo is 56B 2021-02-10T11:12:00.026 DEBUG:teuthology.orchestra.run.gibba001:> rm -fr /tmp/tmp.RY3fBuMLIo 2021-02-10T11:12:00.361 DEBUG:teuthology.orchestra.run.gibba001:> mktemp 2021-02-10T11:12:00.694 INFO:teuthology.orchestra.run.gibba001.stdout:/tmp/tmp.LlcotbfTbD 2021-02-10T11:12:00.695 DEBUG:teuthology.orchestra.run.gibba001:> sudo cp /var/lib/ceph/mds/ceph-a/keyring /tmp/tmp.LlcotbfTbD 2021-02-10T11:12:01.038 DEBUG:teuthology.orchestra.run.gibba001:> sudo chmod 0666 /tmp/tmp.LlcotbfTbD 2021-02-10T11:12:01.718 DEBUG:teuthology.orchestra.remote:gibba001:/tmp/tmp.LlcotbfTbD is 56B 2021-02-10T11:12:02.057 DEBUG:teuthology.orchestra.run.gibba001:> rm -fr /tmp/tmp.LlcotbfTbD 2021-02-10T11:12:02.391 DEBUG:teuthology.orchestra.run.gibba001:> mktemp 2021-02-10T11:12:02.726 INFO:teuthology.orchestra.run.gibba001.stdout:/tmp/tmp.s3U66vO27D 2021-02-10T11:12:02.726 DEBUG:teuthology.orchestra.run.gibba001:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.s3U66vO27D 2021-02-10T11:12:03.070 DEBUG:teuthology.orchestra.run.gibba001:> sudo chmod 0666 /tmp/tmp.s3U66vO27D 2021-02-10T11:12:03.750 DEBUG:teuthology.orchestra.remote:gibba001:/tmp/tmp.s3U66vO27D is 56B 2021-02-10T11:12:04.091 DEBUG:teuthology.orchestra.run.gibba001:> rm -fr /tmp/tmp.s3U66vO27D 2021-02-10T11:12:04.428 DEBUG:teuthology.orchestra.run.gibba001:> mktemp 2021-02-10T11:12:04.763 INFO:teuthology.orchestra.run.gibba001.stdout:/tmp/tmp.V2PIyJyMLa 2021-02-10T11:12:04.763 DEBUG:teuthology.orchestra.run.gibba001:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.V2PIyJyMLa 2021-02-10T11:12:05.106 DEBUG:teuthology.orchestra.run.gibba001:> sudo chmod 0666 /tmp/tmp.V2PIyJyMLa 2021-02-10T11:12:05.785 DEBUG:teuthology.orchestra.remote:gibba001:/tmp/tmp.V2PIyJyMLa is 56B 2021-02-10T11:12:06.125 DEBUG:teuthology.orchestra.run.gibba001:> rm -fr /tmp/tmp.V2PIyJyMLa 2021-02-10T11:12:06.458 DEBUG:teuthology.orchestra.run.gibba001:> mktemp 2021-02-10T11:12:06.791 INFO:teuthology.orchestra.run.gibba001.stdout:/tmp/tmp.0hdYjMbutg 2021-02-10T11:12:06.792 DEBUG:teuthology.orchestra.run.gibba001:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.0hdYjMbutg 2021-02-10T11:12:07.134 DEBUG:teuthology.orchestra.run.gibba001:> sudo chmod 0666 /tmp/tmp.0hdYjMbutg 2021-02-10T11:12:07.816 DEBUG:teuthology.orchestra.remote:gibba001:/tmp/tmp.0hdYjMbutg is 56B 2021-02-10T11:12:08.156 DEBUG:teuthology.orchestra.run.gibba001:> rm -fr /tmp/tmp.0hdYjMbutg 2021-02-10T11:12:08.490 DEBUG:teuthology.orchestra.run.gibba001:> mktemp 2021-02-10T11:12:09.466 INFO:teuthology.orchestra.run.gibba001.stdout:/tmp/tmp.nU7v4r8Njr 2021-02-10T11:12:09.466 DEBUG:teuthology.orchestra.run.gibba001:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.nU7v4r8Njr 2021-02-10T11:12:09.809 DEBUG:teuthology.orchestra.run.gibba001:> sudo chmod 0666 /tmp/tmp.nU7v4r8Njr 2021-02-10T11:12:10.232 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:12:10.236 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:12:10.491 DEBUG:teuthology.orchestra.remote:gibba001:/tmp/tmp.nU7v4r8Njr is 56B 2021-02-10T11:12:10.831 DEBUG:teuthology.orchestra.run.gibba001:> rm -fr /tmp/tmp.nU7v4r8Njr 2021-02-10T11:12:11.165 DEBUG:teuthology.orchestra.run.gibba008:> mktemp 2021-02-10T11:12:11.501 INFO:teuthology.orchestra.run.gibba008.stdout:/tmp/tmp.yyskjJaQ9Z 2021-02-10T11:12:11.501 DEBUG:teuthology.orchestra.run.gibba008:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.yyskjJaQ9Z 2021-02-10T11:12:11.847 DEBUG:teuthology.orchestra.run.gibba008:> sudo chmod 0666 /tmp/tmp.yyskjJaQ9Z 2021-02-10T11:12:12.528 DEBUG:teuthology.orchestra.remote:gibba008:/tmp/tmp.yyskjJaQ9Z is 56B 2021-02-10T11:12:12.867 DEBUG:teuthology.orchestra.run.gibba008:> rm -fr /tmp/tmp.yyskjJaQ9Z 2021-02-10T11:12:13.201 DEBUG:teuthology.orchestra.run.gibba008:> mktemp 2021-02-10T11:12:13.537 INFO:teuthology.orchestra.run.gibba008.stdout:/tmp/tmp.KP6KIANV7P 2021-02-10T11:12:13.537 DEBUG:teuthology.orchestra.run.gibba008:> sudo cp /var/lib/ceph/mds/ceph-b/keyring /tmp/tmp.KP6KIANV7P 2021-02-10T11:12:13.882 DEBUG:teuthology.orchestra.run.gibba008:> sudo chmod 0666 /tmp/tmp.KP6KIANV7P 2021-02-10T11:12:14.594 DEBUG:teuthology.orchestra.remote:gibba008:/tmp/tmp.KP6KIANV7P is 56B 2021-02-10T11:12:14.949 DEBUG:teuthology.orchestra.run.gibba008:> rm -fr /tmp/tmp.KP6KIANV7P 2021-02-10T11:12:15.288 DEBUG:teuthology.orchestra.run.gibba008:> mktemp 2021-02-10T11:12:15.629 INFO:teuthology.orchestra.run.gibba008.stdout:/tmp/tmp.yaoCv7kdgd 2021-02-10T11:12:15.629 DEBUG:teuthology.orchestra.run.gibba008:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.yaoCv7kdgd 2021-02-10T11:12:15.975 DEBUG:teuthology.orchestra.run.gibba008:> sudo chmod 0666 /tmp/tmp.yaoCv7kdgd 2021-02-10T11:12:16.657 DEBUG:teuthology.orchestra.remote:gibba008:/tmp/tmp.yaoCv7kdgd is 56B 2021-02-10T11:12:16.996 DEBUG:teuthology.orchestra.run.gibba008:> rm -fr /tmp/tmp.yaoCv7kdgd 2021-02-10T11:12:17.330 DEBUG:teuthology.orchestra.run.gibba008:> mktemp 2021-02-10T11:12:17.664 INFO:teuthology.orchestra.run.gibba008.stdout:/tmp/tmp.HgiifEq7Kj 2021-02-10T11:12:17.664 DEBUG:teuthology.orchestra.run.gibba008:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.HgiifEq7Kj 2021-02-10T11:12:18.008 DEBUG:teuthology.orchestra.run.gibba008:> sudo chmod 0666 /tmp/tmp.HgiifEq7Kj 2021-02-10T11:12:18.688 DEBUG:teuthology.orchestra.remote:gibba008:/tmp/tmp.HgiifEq7Kj is 56B 2021-02-10T11:12:19.028 DEBUG:teuthology.orchestra.run.gibba008:> rm -fr /tmp/tmp.HgiifEq7Kj 2021-02-10T11:12:19.361 DEBUG:teuthology.orchestra.run.gibba008:> mktemp 2021-02-10T11:12:19.695 INFO:teuthology.orchestra.run.gibba008.stdout:/tmp/tmp.FML4Vt1vHJ 2021-02-10T11:12:19.696 DEBUG:teuthology.orchestra.run.gibba008:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.FML4Vt1vHJ 2021-02-10T11:12:20.040 DEBUG:teuthology.orchestra.run.gibba008:> sudo chmod 0666 /tmp/tmp.FML4Vt1vHJ 2021-02-10T11:12:20.718 DEBUG:teuthology.orchestra.remote:gibba008:/tmp/tmp.FML4Vt1vHJ is 56B 2021-02-10T11:12:21.059 DEBUG:teuthology.orchestra.run.gibba008:> rm -fr /tmp/tmp.FML4Vt1vHJ 2021-02-10T11:12:21.393 DEBUG:teuthology.orchestra.run.gibba008:> mktemp 2021-02-10T11:12:21.726 INFO:teuthology.orchestra.run.gibba008.stdout:/tmp/tmp.03yjtr0nKH 2021-02-10T11:12:21.726 DEBUG:teuthology.orchestra.run.gibba008:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.03yjtr0nKH 2021-02-10T11:12:22.069 DEBUG:teuthology.orchestra.run.gibba008:> sudo chmod 0666 /tmp/tmp.03yjtr0nKH 2021-02-10T11:12:22.751 DEBUG:teuthology.orchestra.remote:gibba008:/tmp/tmp.03yjtr0nKH is 56B 2021-02-10T11:12:23.091 DEBUG:teuthology.orchestra.run.gibba008:> rm -fr /tmp/tmp.03yjtr0nKH 2021-02-10T11:12:23.764 DEBUG:teuthology.orchestra.remote:gibba001:/etc/ceph/ceph.client.0.keyring is 59B 2021-02-10T11:12:24.442 DEBUG:teuthology.orchestra.remote:gibba001:/etc/ceph/ceph.client.1.keyring is 59B 2021-02-10T11:12:25.122 DEBUG:teuthology.orchestra.remote:gibba008:/etc/ceph/ceph.client.2.keyring is 59B 2021-02-10T11:12:25.799 DEBUG:teuthology.orchestra.remote:gibba008:/etc/ceph/ceph.client.3.keyring is 59B 2021-02-10T11:12:26.138 INFO:tasks.ceph:Adding keys to all mons... 2021-02-10T11:12:26.138 DEBUG:teuthology.orchestra.run.gibba001:> sudo tee -a /etc/ceph/ceph.keyring 2021-02-10T11:12:26.142 DEBUG:teuthology.orchestra.run.gibba008:> sudo tee -a /etc/ceph/ceph.keyring 2021-02-10T11:12:26.481 INFO:teuthology.orchestra.run.gibba001.stdout:[mgr.y] 2021-02-10T11:12:26.481 INFO:teuthology.orchestra.run.gibba001.stdout: key = AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg== 2021-02-10T11:12:26.481 INFO:teuthology.orchestra.run.gibba001.stdout:[mds.a] 2021-02-10T11:12:26.481 INFO:teuthology.orchestra.run.gibba001.stdout: key = AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA== 2021-02-10T11:12:26.481 INFO:teuthology.orchestra.run.gibba001.stdout:[osd.0] 2021-02-10T11:12:26.482 INFO:teuthology.orchestra.run.gibba001.stdout: key = AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw== 2021-02-10T11:12:26.482 INFO:teuthology.orchestra.run.gibba001.stdout:[osd.1] 2021-02-10T11:12:26.482 INFO:teuthology.orchestra.run.gibba001.stdout: key = AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ== 2021-02-10T11:12:26.482 INFO:teuthology.orchestra.run.gibba001.stdout:[osd.2] 2021-02-10T11:12:26.482 INFO:teuthology.orchestra.run.gibba001.stdout: key = AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w== 2021-02-10T11:12:26.482 INFO:teuthology.orchestra.run.gibba001.stdout:[osd.3] 2021-02-10T11:12:26.482 INFO:teuthology.orchestra.run.gibba001.stdout: key = AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA== 2021-02-10T11:12:26.482 INFO:teuthology.orchestra.run.gibba001.stdout:[mgr.x] 2021-02-10T11:12:26.482 INFO:teuthology.orchestra.run.gibba001.stdout: key = AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA== 2021-02-10T11:12:26.482 INFO:teuthology.orchestra.run.gibba001.stdout:[mds.b] 2021-02-10T11:12:26.482 INFO:teuthology.orchestra.run.gibba001.stdout: key = AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ== 2021-02-10T11:12:26.482 INFO:teuthology.orchestra.run.gibba001.stdout:[osd.4] 2021-02-10T11:12:26.482 INFO:teuthology.orchestra.run.gibba001.stdout: key = AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA== 2021-02-10T11:12:26.483 INFO:teuthology.orchestra.run.gibba001.stdout:[osd.5] 2021-02-10T11:12:26.483 INFO:teuthology.orchestra.run.gibba001.stdout: key = AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw== 2021-02-10T11:12:26.483 INFO:teuthology.orchestra.run.gibba001.stdout:[osd.6] 2021-02-10T11:12:26.483 INFO:teuthology.orchestra.run.gibba001.stdout: key = AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ== 2021-02-10T11:12:26.483 INFO:teuthology.orchestra.run.gibba001.stdout:[osd.7] 2021-02-10T11:12:26.483 INFO:teuthology.orchestra.run.gibba001.stdout: key = AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw== 2021-02-10T11:12:26.483 INFO:teuthology.orchestra.run.gibba001.stdout:[client.0] 2021-02-10T11:12:26.483 INFO:teuthology.orchestra.run.gibba001.stdout: key = AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg== 2021-02-10T11:12:26.483 INFO:teuthology.orchestra.run.gibba001.stdout:[client.1] 2021-02-10T11:12:26.483 INFO:teuthology.orchestra.run.gibba001.stdout: key = AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg== 2021-02-10T11:12:26.483 INFO:teuthology.orchestra.run.gibba001.stdout:[client.2] 2021-02-10T11:12:26.483 INFO:teuthology.orchestra.run.gibba001.stdout: key = AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA== 2021-02-10T11:12:26.483 INFO:teuthology.orchestra.run.gibba001.stdout:[client.3] 2021-02-10T11:12:26.483 INFO:teuthology.orchestra.run.gibba001.stdout: key = AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg== 2021-02-10T11:12:26.485 INFO:teuthology.orchestra.run.gibba008.stdout:[mgr.y] 2021-02-10T11:12:26.485 INFO:teuthology.orchestra.run.gibba008.stdout: key = AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg== 2021-02-10T11:12:26.485 INFO:teuthology.orchestra.run.gibba008.stdout:[mds.a] 2021-02-10T11:12:26.485 INFO:teuthology.orchestra.run.gibba008.stdout: key = AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA== 2021-02-10T11:12:26.485 INFO:teuthology.orchestra.run.gibba008.stdout:[osd.0] 2021-02-10T11:12:26.485 INFO:teuthology.orchestra.run.gibba008.stdout: key = AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw== 2021-02-10T11:12:26.485 INFO:teuthology.orchestra.run.gibba008.stdout:[osd.1] 2021-02-10T11:12:26.485 INFO:teuthology.orchestra.run.gibba008.stdout: key = AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ== 2021-02-10T11:12:26.485 INFO:teuthology.orchestra.run.gibba008.stdout:[osd.2] 2021-02-10T11:12:26.485 INFO:teuthology.orchestra.run.gibba008.stdout: key = AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w== 2021-02-10T11:12:26.486 INFO:teuthology.orchestra.run.gibba008.stdout:[osd.3] 2021-02-10T11:12:26.486 INFO:teuthology.orchestra.run.gibba008.stdout: key = AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA== 2021-02-10T11:12:26.486 INFO:teuthology.orchestra.run.gibba008.stdout:[mgr.x] 2021-02-10T11:12:26.486 INFO:teuthology.orchestra.run.gibba008.stdout: key = AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA== 2021-02-10T11:12:26.486 INFO:teuthology.orchestra.run.gibba008.stdout:[mds.b] 2021-02-10T11:12:26.486 INFO:teuthology.orchestra.run.gibba008.stdout: key = AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ== 2021-02-10T11:12:26.486 INFO:teuthology.orchestra.run.gibba008.stdout:[osd.4] 2021-02-10T11:12:26.486 INFO:teuthology.orchestra.run.gibba008.stdout: key = AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA== 2021-02-10T11:12:26.486 INFO:teuthology.orchestra.run.gibba008.stdout:[osd.5] 2021-02-10T11:12:26.486 INFO:teuthology.orchestra.run.gibba008.stdout: key = AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw== 2021-02-10T11:12:26.486 INFO:teuthology.orchestra.run.gibba008.stdout:[osd.6] 2021-02-10T11:12:26.486 INFO:teuthology.orchestra.run.gibba008.stdout: key = AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ== 2021-02-10T11:12:26.486 INFO:teuthology.orchestra.run.gibba008.stdout:[osd.7] 2021-02-10T11:12:26.486 INFO:teuthology.orchestra.run.gibba008.stdout: key = AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw== 2021-02-10T11:12:26.487 INFO:teuthology.orchestra.run.gibba008.stdout:[client.0] 2021-02-10T11:12:26.487 INFO:teuthology.orchestra.run.gibba008.stdout: key = AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg== 2021-02-10T11:12:26.487 INFO:teuthology.orchestra.run.gibba008.stdout:[client.1] 2021-02-10T11:12:26.487 INFO:teuthology.orchestra.run.gibba008.stdout: key = AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg== 2021-02-10T11:12:26.487 INFO:teuthology.orchestra.run.gibba008.stdout:[client.2] 2021-02-10T11:12:26.487 INFO:teuthology.orchestra.run.gibba008.stdout: key = AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA== 2021-02-10T11:12:26.487 INFO:teuthology.orchestra.run.gibba008.stdout:[client.3] 2021-02-10T11:12:26.487 INFO:teuthology.orchestra.run.gibba008.stdout: key = AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg== 2021-02-10T11:12:26.488 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-02-10T11:12:26.491 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-02-10T11:12:26.866 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-02-10T11:12:26.869 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-02-10T11:12:27.246 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-10T11:12:27.249 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-10T11:12:27.623 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-10T11:12:27.626 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-10T11:12:28.001 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-10T11:12:28.004 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-10T11:12:28.378 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-10T11:12:28.381 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-10T11:12:28.755 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-02-10T11:12:28.758 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-02-10T11:12:29.132 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-02-10T11:12:29.135 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-02-10T11:12:29.511 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-10T11:12:29.514 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-10T11:12:29.887 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-10T11:12:29.890 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-10T11:12:30.265 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-10T11:12:30.268 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-10T11:12:30.642 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-10T11:12:30.645 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-10T11:12:31.018 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-10T11:12:31.021 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-10T11:12:31.397 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-10T11:12:31.400 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-10T11:12:31.777 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-10T11:12:31.780 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-10T11:12:32.157 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-10T11:12:32.161 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-10T11:12:32.537 INFO:tasks.ceph:Running mkfs on mon nodes... 2021-02-10T11:12:32.590 DEBUG:teuthology.orchestra.run.gibba001:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2021-02-10T11:12:32.937 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2021-02-10T11:12:33.317 DEBUG:teuthology.orchestra.run.gibba001:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2021-02-10T11:12:33.661 DEBUG:teuthology.orchestra.run.gibba008:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2021-02-10T11:12:34.004 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2021-02-10T11:12:34.386 DEBUG:teuthology.orchestra.run.gibba008:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2021-02-10T11:12:34.730 DEBUG:teuthology.orchestra.run.gibba008:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2021-02-10T11:12:35.075 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2021-02-10T11:12:35.456 DEBUG:teuthology.orchestra.run.gibba008:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2021-02-10T11:12:35.801 DEBUG:teuthology.orchestra.run.gibba001:> rm -- /home/ubuntu/cephtest/ceph.monmap 2021-02-10T11:12:35.804 DEBUG:teuthology.orchestra.run.gibba008:> rm -- /home/ubuntu/cephtest/ceph.monmap 2021-02-10T11:12:36.142 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2021-02-10T11:12:36.143 INFO:tasks.ceph.mon.a:Restarting daemon 2021-02-10T11:12:36.143 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2021-02-10T11:12:36.152 INFO:tasks.ceph.mon.a:Started 2021-02-10T11:12:36.153 INFO:tasks.ceph.mon.b:Restarting daemon 2021-02-10T11:12:36.153 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2021-02-10T11:12:36.178 INFO:tasks.ceph.mon.b:Started 2021-02-10T11:12:36.178 INFO:tasks.ceph.mon.c:Restarting daemon 2021-02-10T11:12:36.178 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2021-02-10T11:12:36.198 INFO:tasks.ceph.mon.c:Started 2021-02-10T11:12:36.198 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2021-02-10T11:12:36.198 INFO:tasks.ceph.mgr.y:Restarting daemon 2021-02-10T11:12:36.198 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2021-02-10T11:12:36.206 INFO:tasks.ceph.mgr.y:Started 2021-02-10T11:12:36.206 INFO:tasks.ceph.mgr.x:Restarting daemon 2021-02-10T11:12:36.207 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2021-02-10T11:12:36.226 INFO:tasks.ceph.mgr.x:Started 2021-02-10T11:12:36.227 INFO:tasks.ceph:Setting crush tunables to default 2021-02-10T11:12:36.227 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --cluster ceph osd crush tunables default 2021-02-10T11:12:36.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.555 7f8d7882d700 1 Processor -- start 2021-02-10T11:12:36.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.555 7f8d7882d700 1 -- start start 2021-02-10T11:12:36.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.556 7f8d7882d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8d740cff00 0x7f8d741b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:36.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.556 7f8d7882d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d740f0b20 con 0x7f8d740d1bf0 2021-02-10T11:12:36.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.556 7f8d7882d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d741b3ff0 con 0x7f8d740d17f0 2021-02-10T11:12:36.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.556 7f8d7882d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d740e5eb0 con 0x7f8d740cff00 2021-02-10T11:12:36.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.556 7f8d72e1d700 1 -- >> v1:172.21.2.101:6789/0 conn(0x7f8d740d1bf0 legacy=0x7f8d740ef620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.2.101:6789/0 2021-02-10T11:12:36.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.556 7f8d7261c700 1 -- >> v1:172.21.2.108:6789/0 conn(0x7f8d740d17f0 legacy=0x7f8d741b31e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.2.108:6789/0 2021-02-10T11:12:36.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.557 7f8d71e1b700 1 -- >> v2:172.21.2.108:3300/0 conn(0x7f8d740cff00 msgr2=0x7f8d741b39d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.2.108:3300/0 2021-02-10T11:12:36.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.557 7f8d71e1b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8d740cff00 0x7f8d741b39d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2021-02-10T11:12:36.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.756 7f8d72e1d700 1 -- >> v1:172.21.2.101:6789/0 conn(0x7f8d740d1bf0 legacy=0x7f8d740ef620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.2.101:6789/0 2021-02-10T11:12:36.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.757 7f8d71e1b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8d740cff00 0x7f8d741b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:36.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.757 7f8d7261c700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f8d740d17f0 0x7f8d741b31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:37342/0 (socket says 172.21.2.101:37342) 2021-02-10T11:12:36.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.757 7f8d7261c700 1 -- 172.21.2.101:0/1146074431 learned_addr learned my addr 172.21.2.101:0/1146074431 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:12:36.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.757 7f8d71e1b700 1 -- 172.21.2.101:0/1146074431 >> v2:172.21.2.108:3300/0 conn(0x7f8d740cff00 msgr2=0x7f8d741b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2021-02-10T11:12:36.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.757 7f8d71e1b700 1 -- 172.21.2.101:0/1146074431 >> v2:172.21.2.108:3300/0 conn(0x7f8d740cff00 msgr2=0x7f8d741b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-02-10T11:12:36.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.757 7f8d71e1b700 1 --2- 172.21.2.101:0/1146074431 >> v2:172.21.2.108:3300/0 conn(0x7f8d740cff00 0x7f8d741b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-02-10T11:12:36.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.757 7f8d71e1b700 1 --2- 172.21.2.101:0/1146074431 >> v2:172.21.2.108:3300/0 conn(0x7f8d740cff00 0x7f8d741b39d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2021-02-10T11:12:36.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.797 7f8d7161a700 1 -- 172.21.2.101:0/1146074431 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2635909117 0 0) 0x7f8d5c000c60 con 0x7f8d740d17f0 2021-02-10T11:12:36.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.797 7f8d7161a700 1 -- 172.21.2.101:0/1146074431 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d60002fe0 con 0x7f8d740d17f0 2021-02-10T11:12:36.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.799 7f8d7161a700 1 -- 172.21.2.101:0/1146074431 <== mon.1 v1:172.21.2.108:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4222555022 0 0) 0x7f8d5c003590 con 0x7f8d740d17f0 2021-02-10T11:12:36.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.799 7f8d7161a700 1 -- 172.21.2.101:0/1146074431 >> v2:172.21.2.108:3300/0 conn(0x7f8d740cff00 msgr2=0x7f8d741b39d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2021-02-10T11:12:36.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.799 7f8d7161a700 1 --2- 172.21.2.101:0/1146074431 >> v2:172.21.2.108:3300/0 conn(0x7f8d740cff00 0x7f8d741b39d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:36.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.799 7f8d7161a700 1 -- 172.21.2.101:0/1146074431 >> v1:172.21.2.101:6789/0 conn(0x7f8d740d1bf0 legacy=0x7f8d740ef620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2021-02-10T11:12:36.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.799 7f8d7161a700 1 -- 172.21.2.101:0/1146074431 --> v1:172.21.2.108:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d740f20b0 con 0x7f8d740d17f0 2021-02-10T11:12:36.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.799 7f8d7161a700 1 -- 172.21.2.101:0/1146074431 <== mon.1 v1:172.21.2.108:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f8d5c003a10 con 0x7f8d740d17f0 2021-02-10T11:12:36.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.799 7f8d7161a700 1 -- 172.21.2.101:0/1146074431 >> v1:172.21.2.108:6789/0 conn(0x7f8d740d17f0 legacy=0x7f8d741b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:36.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.799 7f8d7161a700 1 --2- 172.21.2.101:0/1146074431 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d60003a70 0x7f8d60005f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:36.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.799 7f8d7161a700 1 --2- 172.21.2.101:0/1146074431 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d60006780 0x7f8d60008ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:36.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.799 7f8d7161a700 1 --2- 172.21.2.101:0/1146074431 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d600095a0 0x7f8d6000bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:36.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.799 7f8d7161a700 1 -- 172.21.2.101:0/1146074431 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d60050320 con 0x7f8d60006780 2021-02-10T11:12:36.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.799 7f8d7161a700 1 -- 172.21.2.101:0/1146074431 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d60050540 con 0x7f8d600095a0 2021-02-10T11:12:36.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.799 7f8d7161a700 1 -- 172.21.2.101:0/1146074431 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d60050760 con 0x7f8d60003a70 2021-02-10T11:12:36.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.799 7f8d71e1b700 1 --2- 172.21.2.101:0/1146074431 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d60006780 0x7f8d60008ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:36.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.799 7f8d7261c700 1 --2- 172.21.2.101:0/1146074431 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d60003a70 0x7f8d60005f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:36.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.799 7f8d72e1d700 1 --2- 172.21.2.101:0/1146074431 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d600095a0 0x7f8d6000bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:36.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.800 7f8d71e1b700 1 -- 172.21.2.101:0/1146074431 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d60003a70 msgr2=0x7f8d60005f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:12:36.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.800 7f8d71e1b700 1 --2- 172.21.2.101:0/1146074431 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d60003a70 0x7f8d60005f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:36.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.800 7f8d71e1b700 1 -- 172.21.2.101:0/1146074431 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d600095a0 msgr2=0x7f8d6000bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:12:36.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.800 7f8d71e1b700 1 --2- 172.21.2.101:0/1146074431 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d600095a0 0x7f8d6000bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:36.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.800 7f8d71e1b700 1 -- 172.21.2.101:0/1146074431 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d60050a00 con 0x7f8d60006780 2021-02-10T11:12:36.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.800 7f8d72e1d700 1 -- 172.21.2.101:0/1146074431 reap_dead start 2021-02-10T11:12:36.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.800 7f8d71e1b700 1 --2- 172.21.2.101:0/1146074431 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d60006780 0x7f8d60008ca0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f8d640051e0 tx=0x7f8d640068b0).ready entity=mon.0 client_cookie=1ae65b922772cd18 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:36.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.801 7f8d7161a700 1 -- 172.21.2.101:0/1146074431 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d6400b070 con 0x7f8d60006780 2021-02-10T11:12:36.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.801 7f8d7161a700 1 -- 172.21.2.101:0/1146074431 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6400b3f0 con 0x7f8d60006780 2021-02-10T11:12:36.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.801 7f8d7161a700 1 -- 172.21.2.101:0/1146074431 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d6400b8d0 con 0x7f8d60006780 2021-02-10T11:12:36.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.801 7f8d7882d700 1 -- 172.21.2.101:0/1146074431 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d60006780 msgr2=0x7f8d60008ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:36.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.801 7f8d7882d700 1 --2- 172.21.2.101:0/1146074431 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d60006780 0x7f8d60008ca0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f8d640051e0 tx=0x7f8d640068b0).stop 2021-02-10T11:12:36.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.801 7f8d7882d700 1 -- 172.21.2.101:0/1146074431 shutdown_connections 2021-02-10T11:12:36.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.801 7f8d7882d700 1 --2- 172.21.2.101:0/1146074431 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d60006780 0x7f8d60008ca0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:36.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.801 7f8d7882d700 1 -- 172.21.2.101:0/1146074431 >> 172.21.2.101:0/1146074431 conn(0x7f8d740fde40 msgr2=0x7f8d740ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:12:36.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.801 7f8d7882d700 1 -- 172.21.2.101:0/1146074431 shutdown_connections 2021-02-10T11:12:36.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.801 7f8d7882d700 1 -- 172.21.2.101:0/1146074431 wait complete. 2021-02-10T11:12:36.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.817 7f8d7882d700 1 Processor -- start 2021-02-10T11:12:36.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.817 7f8d7882d700 1 -- start start 2021-02-10T11:12:36.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.818 7f8d7882d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d7419ff20 0x7f8d74090820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:36.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.818 7f8d7882d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d741a0320 0x7f8d74090e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:36.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.818 7f8d7882d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d740b6860 0x7f8d740b6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:36.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.818 7f8d7882d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d740916f0 con 0x7f8d740b6860 2021-02-10T11:12:36.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.818 7f8d7882d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d74091460 con 0x7f8d741a0320 2021-02-10T11:12:36.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.818 7f8d7882d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d74108090 con 0x7f8d7419ff20 2021-02-10T11:12:36.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.818 7f8d7261c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d7419ff20 0x7f8d74090820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:36.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.818 7f8d7261c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d7419ff20 0x7f8d74090820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:36198/0 (socket says 172.21.2.101:36198) 2021-02-10T11:12:36.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.818 7f8d7261c700 1 -- 172.21.2.101:0/2017859514 learned_addr learned my addr 172.21.2.101:0/2017859514 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:12:36.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.818 7f8d71e1b700 1 --2- 172.21.2.101:0/2017859514 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d741a0320 0x7f8d74090e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:36.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.818 7f8d72e1d700 1 --2- 172.21.2.101:0/2017859514 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d740b6860 0x7f8d740b6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:36.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.818 7f8d7261c700 1 -- 172.21.2.101:0/2017859514 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d741a0320 msgr2=0x7f8d74090e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:36.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.818 7f8d7261c700 1 --2- 172.21.2.101:0/2017859514 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d741a0320 0x7f8d74090e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:36.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.818 7f8d7261c700 1 -- 172.21.2.101:0/2017859514 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d740b6860 msgr2=0x7f8d740b6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:36.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.818 7f8d7261c700 1 --2- 172.21.2.101:0/2017859514 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d740b6860 0x7f8d740b6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:36.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.818 7f8d7261c700 1 -- 172.21.2.101:0/2017859514 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d74107c20 con 0x7f8d7419ff20 2021-02-10T11:12:36.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.818 7f8d7261c700 1 --2- 172.21.2.101:0/2017859514 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d7419ff20 0x7f8d74090820 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f8d5c00afc0 tx=0x7f8d5c008fb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:36.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.819 7f8d6affd700 1 -- 172.21.2.101:0/2017859514 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d5c006880 con 0x7f8d7419ff20 2021-02-10T11:12:36.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.819 7f8d6affd700 1 -- 172.21.2.101:0/2017859514 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5c006c00 con 0x7f8d7419ff20 2021-02-10T11:12:36.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.819 7f8d6affd700 1 -- 172.21.2.101:0/2017859514 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d5c0080d0 con 0x7f8d7419ff20 2021-02-10T11:12:36.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.819 7f8d7882d700 1 -- 172.21.2.101:0/2017859514 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d74107c20 con 0x7f8d7419ff20 2021-02-10T11:12:36.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.819 7f8d7882d700 1 -- 172.21.2.101:0/2017859514 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d74107c20 con 0x7f8d7419ff20 2021-02-10T11:12:36.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.819 7f8d6affd700 1 -- 172.21.2.101:0/2017859514 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 1) v1 ==== 190+0+0 (secure 0 0 0) 0x7f8d5c008030 con 0x7f8d7419ff20 2021-02-10T11:12:36.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.819 7f8d6affd700 1 -- 172.21.2.101:0/2017859514 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f8d5c008030 con 0x7f8d7419ff20 2021-02-10T11:12:36.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.857 7f8d7882d700 1 -- 172.21.2.101:0/2017859514 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d741037e0 con 0x7f8d7419ff20 2021-02-10T11:12:36.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.859 7f8d6affd700 1 -- 172.21.2.101:0/2017859514 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+66016 (secure 0 0 0) 0x7f8d5c000c10 con 0x7f8d7419ff20 2021-02-10T11:12:36.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.930 7f8d7882d700 1 -- 172.21.2.101:0/2017859514 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"profile": "default", "prefix": "osd crush tunables"} v 0) v1 -- 0x7f8d740f0b20 con 0x7f8d7419ff20 2021-02-10T11:12:36.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.934 7f8d6affd700 1 -- 172.21.2.101:0/2017859514 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"profile": "default", "prefix": "osd crush tunables"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8d5c000c10 con 0x7f8d7419ff20 2021-02-10T11:12:36.941 INFO:teuthology.orchestra.run.gibba001.stderr:adjusted tunables profile to default 2021-02-10T11:12:36.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.939 7f8d7882d700 1 -- 172.21.2.101:0/2017859514 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d7419ff20 msgr2=0x7f8d74090820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:36.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.939 7f8d7882d700 1 --2- 172.21.2.101:0/2017859514 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d7419ff20 0x7f8d74090820 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f8d5c00afc0 tx=0x7f8d5c008fb0).stop 2021-02-10T11:12:36.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.939 7f8d7882d700 1 -- 172.21.2.101:0/2017859514 shutdown_connections 2021-02-10T11:12:36.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.939 7f8d7882d700 1 --2- 172.21.2.101:0/2017859514 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d740b6860 0x7f8d740b6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:36.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.939 7f8d7882d700 1 --2- 172.21.2.101:0/2017859514 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d741a0320 0x7f8d74090e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:36.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.939 7f8d7882d700 1 --2- 172.21.2.101:0/2017859514 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d7419ff20 0x7f8d74090820 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:36.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.939 7f8d7882d700 1 -- 172.21.2.101:0/2017859514 >> 172.21.2.101:0/2017859514 conn(0x7f8d740b98a0 msgr2=0x7f8d7409c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:12:36.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.939 7f8d7882d700 1 -- 172.21.2.101:0/2017859514 shutdown_connections 2021-02-10T11:12:36.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:12:36.939 7f8d7882d700 1 -- 172.21.2.101:0/2017859514 wait complete. 2021-02-10T11:12:36.951 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2021-02-10T11:12:36.951 DEBUG:teuthology.orchestra.run.gibba001:> mktemp 2021-02-10T11:12:37.288 INFO:teuthology.orchestra.run.gibba001.stdout:/tmp/tmp.3IX3LuyUki 2021-02-10T11:12:37.288 DEBUG:teuthology.orchestra.run.gibba001:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.3IX3LuyUki 2021-02-10T11:12:37.636 DEBUG:teuthology.orchestra.run.gibba001:> sudo chmod 0666 /tmp/tmp.3IX3LuyUki 2021-02-10T11:12:38.320 DEBUG:teuthology.orchestra.remote:gibba001:/tmp/tmp.3IX3LuyUki is 37B 2021-02-10T11:12:38.662 DEBUG:teuthology.orchestra.run.gibba001:> rm -fr /tmp/tmp.3IX3LuyUki 2021-02-10T11:12:38.997 DEBUG:teuthology.orchestra.run.gibba001:> mktemp 2021-02-10T11:12:39.330 INFO:teuthology.orchestra.run.gibba001.stdout:/tmp/tmp.inGq9Nntlg 2021-02-10T11:12:39.331 DEBUG:teuthology.orchestra.run.gibba001:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.inGq9Nntlg 2021-02-10T11:12:39.674 DEBUG:teuthology.orchestra.run.gibba001:> sudo chmod 0666 /tmp/tmp.inGq9Nntlg 2021-02-10T11:12:40.352 DEBUG:teuthology.orchestra.remote:gibba001:/tmp/tmp.inGq9Nntlg is 37B 2021-02-10T11:12:40.602 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:12:40.605 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:12:40.694 DEBUG:teuthology.orchestra.run.gibba001:> rm -fr /tmp/tmp.inGq9Nntlg 2021-02-10T11:12:40.937 DEBUG:teuthology.orchestra.run.gibba001:> mktemp 2021-02-10T11:12:41.271 INFO:teuthology.orchestra.run.gibba001.stdout:/tmp/tmp.7Aru1SsUOI 2021-02-10T11:12:41.271 DEBUG:teuthology.orchestra.run.gibba001:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.7Aru1SsUOI 2021-02-10T11:12:41.614 DEBUG:teuthology.orchestra.run.gibba001:> sudo chmod 0666 /tmp/tmp.7Aru1SsUOI 2021-02-10T11:12:42.293 DEBUG:teuthology.orchestra.remote:gibba001:/tmp/tmp.7Aru1SsUOI is 37B 2021-02-10T11:12:42.633 DEBUG:teuthology.orchestra.run.gibba001:> rm -fr /tmp/tmp.7Aru1SsUOI 2021-02-10T11:12:42.967 DEBUG:teuthology.orchestra.run.gibba001:> mktemp 2021-02-10T11:12:43.302 INFO:teuthology.orchestra.run.gibba001.stdout:/tmp/tmp.jJpQg2YxcX 2021-02-10T11:12:43.302 DEBUG:teuthology.orchestra.run.gibba001:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.jJpQg2YxcX 2021-02-10T11:12:43.645 DEBUG:teuthology.orchestra.run.gibba001:> sudo chmod 0666 /tmp/tmp.jJpQg2YxcX 2021-02-10T11:12:44.325 DEBUG:teuthology.orchestra.remote:gibba001:/tmp/tmp.jJpQg2YxcX is 37B 2021-02-10T11:12:44.667 DEBUG:teuthology.orchestra.run.gibba001:> rm -fr /tmp/tmp.jJpQg2YxcX 2021-02-10T11:12:45.002 DEBUG:teuthology.orchestra.run.gibba008:> mktemp 2021-02-10T11:12:45.338 INFO:teuthology.orchestra.run.gibba008.stdout:/tmp/tmp.f6ETetcsfd 2021-02-10T11:12:45.338 DEBUG:teuthology.orchestra.run.gibba008:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.f6ETetcsfd 2021-02-10T11:12:45.682 DEBUG:teuthology.orchestra.run.gibba008:> sudo chmod 0666 /tmp/tmp.f6ETetcsfd 2021-02-10T11:12:46.361 DEBUG:teuthology.orchestra.remote:gibba008:/tmp/tmp.f6ETetcsfd is 37B 2021-02-10T11:12:46.700 DEBUG:teuthology.orchestra.run.gibba008:> rm -fr /tmp/tmp.f6ETetcsfd 2021-02-10T11:12:47.034 DEBUG:teuthology.orchestra.run.gibba008:> mktemp 2021-02-10T11:12:47.367 INFO:teuthology.orchestra.run.gibba008.stdout:/tmp/tmp.xuWNIRaXFc 2021-02-10T11:12:47.367 DEBUG:teuthology.orchestra.run.gibba008:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.xuWNIRaXFc 2021-02-10T11:12:47.712 DEBUG:teuthology.orchestra.run.gibba008:> sudo chmod 0666 /tmp/tmp.xuWNIRaXFc 2021-02-10T11:12:48.393 DEBUG:teuthology.orchestra.remote:gibba008:/tmp/tmp.xuWNIRaXFc is 37B 2021-02-10T11:12:48.733 DEBUG:teuthology.orchestra.run.gibba008:> rm -fr /tmp/tmp.xuWNIRaXFc 2021-02-10T11:12:49.068 DEBUG:teuthology.orchestra.run.gibba008:> mktemp 2021-02-10T11:12:49.402 INFO:teuthology.orchestra.run.gibba008.stdout:/tmp/tmp.jALsi6iEos 2021-02-10T11:12:49.402 DEBUG:teuthology.orchestra.run.gibba008:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.jALsi6iEos 2021-02-10T11:12:49.746 DEBUG:teuthology.orchestra.run.gibba008:> sudo chmod 0666 /tmp/tmp.jALsi6iEos 2021-02-10T11:12:50.426 DEBUG:teuthology.orchestra.remote:gibba008:/tmp/tmp.jALsi6iEos is 37B 2021-02-10T11:12:50.766 DEBUG:teuthology.orchestra.run.gibba008:> rm -fr /tmp/tmp.jALsi6iEos 2021-02-10T11:12:51.100 DEBUG:teuthology.orchestra.run.gibba008:> mktemp 2021-02-10T11:12:51.434 INFO:teuthology.orchestra.run.gibba008.stdout:/tmp/tmp.aMCpwORyYX 2021-02-10T11:12:51.435 DEBUG:teuthology.orchestra.run.gibba008:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.aMCpwORyYX 2021-02-10T11:12:51.783 DEBUG:teuthology.orchestra.run.gibba008:> sudo chmod 0666 /tmp/tmp.aMCpwORyYX 2021-02-10T11:12:52.858 DEBUG:teuthology.orchestra.remote:gibba008:/tmp/tmp.aMCpwORyYX is 37B 2021-02-10T11:12:53.199 DEBUG:teuthology.orchestra.run.gibba008:> rm -fr /tmp/tmp.aMCpwORyYX 2021-02-10T11:12:53.533 DEBUG:teuthology.orchestra.run.gibba008:> sudo ceph --cluster ceph osd new d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5 0 2021-02-10T11:12:53.959 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.944 7f4ce8358700 1 Processor -- start 2021-02-10T11:12:53.959 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.944 7f4ce8358700 1 -- start start 2021-02-10T11:12:53.959 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.945 7f4ce8358700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4ce00f0630 0x7f4ce00f10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:53.959 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.945 7f4ce8358700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce00f0b20 con 0x7f4ce00d17f0 2021-02-10T11:12:53.959 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.945 7f4ce8358700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce00d1bf0 con 0x7f4ce00cff00 2021-02-10T11:12:53.959 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.945 7f4ce8358700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce00e5eb0 con 0x7f4ce00f0630 2021-02-10T11:12:53.959 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.945 7f4ce5973700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f4ce00cff00 0x7f4ce01b39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.108:33314/0 (socket says 172.21.2.108:33314) 2021-02-10T11:12:53.959 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.945 7f4ce5973700 1 -- 172.21.2.108:0/1955524008 learned_addr learned my addr 172.21.2.108:0/1955524008 (peer_addr_for_me v1:172.21.2.108:0/0) 2021-02-10T11:12:53.959 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.945 7f4ce6975700 1 --2- 172.21.2.108:0/1955524008 >> v2:172.21.2.108:3300/0 conn(0x7f4ce00f0630 0x7f4ce00f10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:53.959 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.946 7f4ce6975700 1 -- 172.21.2.108:0/1955524008 >> v1:172.21.2.108:6789/0 conn(0x7f4ce00cff00 legacy=0x7f4ce01b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:53.960 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.946 7f4ce6975700 1 -- 172.21.2.108:0/1955524008 >> v1:172.21.2.101:6789/0 conn(0x7f4ce00d17f0 legacy=0x7f4ce01b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:12:53.960 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.946 7f4ce5172700 1 -- 172.21.2.108:0/1955524008 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2292476601 0 0) 0x7f4cd4001180 con 0x7f4ce00cff00 2021-02-10T11:12:53.960 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.946 7f4ce6975700 1 -- 172.21.2.108:0/1955524008 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce00d1e10 con 0x7f4ce00f0630 2021-02-10T11:12:53.960 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.946 7f4ce6975700 1 --2- 172.21.2.108:0/1955524008 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ce00f0630 0x7f4ce00f10a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f4cd8006e30 tx=0x7f4cd8009de0).ready entity=mon.1 client_cookie=b2b2aa4514c94b6f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:53.960 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.946 7f4ce5172700 1 -- 172.21.2.108:0/1955524008 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4cd800b300 con 0x7f4ce00f0630 2021-02-10T11:12:53.960 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.946 7f4ce5172700 1 -- 172.21.2.108:0/1955524008 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd800b580 con 0x7f4ce00f0630 2021-02-10T11:12:53.960 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.946 7f4ce5172700 1 -- 172.21.2.108:0/1955524008 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4cd800baa0 con 0x7f4ce00f0630 2021-02-10T11:12:53.960 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.946 7f4ce8358700 1 -- 172.21.2.108:0/1955524008 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ce00f0630 msgr2=0x7f4ce00f10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:53.960 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.946 7f4ce8358700 1 --2- 172.21.2.108:0/1955524008 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ce00f0630 0x7f4ce00f10a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f4cd8006e30 tx=0x7f4cd8009de0).stop 2021-02-10T11:12:53.960 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.947 7f4ce8358700 1 -- 172.21.2.108:0/1955524008 shutdown_connections 2021-02-10T11:12:53.960 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.947 7f4ce8358700 1 --2- 172.21.2.108:0/1955524008 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ce00f0630 0x7f4ce00f10a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:53.960 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.947 7f4ce8358700 1 -- 172.21.2.108:0/1955524008 >> 172.21.2.108:0/1955524008 conn(0x7f4ce00fde40 msgr2=0x7f4ce00ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:12:53.960 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.947 7f4ce8358700 1 -- 172.21.2.108:0/1955524008 shutdown_connections 2021-02-10T11:12:53.961 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.947 7f4ce8358700 1 -- 172.21.2.108:0/1955524008 wait complete. 2021-02-10T11:12:53.962 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.961 7f4ce8358700 1 Processor -- start 2021-02-10T11:12:53.962 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.962 7f4ce8358700 1 -- start start 2021-02-10T11:12:53.962 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.962 7f4ce8358700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ce01960f0 0x7f4ce019b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:53.962 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.962 7f4ce8358700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ce018ecc0 0x7f4ce0195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:53.962 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.962 7f4ce8358700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ce019c280 0x7f4ce019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:53.962 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.962 7f4ce8358700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce018f0c0 con 0x7f4ce019c280 2021-02-10T11:12:53.963 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.962 7f4ce8358700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce0195d60 con 0x7f4ce018ecc0 2021-02-10T11:12:53.963 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.962 7f4ce8358700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce019cca0 con 0x7f4ce01960f0 2021-02-10T11:12:53.963 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.962 7f4ce6174700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ce01960f0 0x7f4ce019b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:53.963 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.962 7f4ce6174700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ce01960f0 0x7f4ce019b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.108:44296/0 (socket says 172.21.2.108:44296) 2021-02-10T11:12:53.963 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.962 7f4ce6174700 1 -- 172.21.2.108:0/3465506227 learned_addr learned my addr 172.21.2.108:0/3465506227 (peer_addr_for_me v2:172.21.2.108:0/0) 2021-02-10T11:12:53.963 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.962 7f4ce5973700 1 --2- 172.21.2.108:0/3465506227 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ce018ecc0 0x7f4ce0195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:53.963 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.963 7f4ce6174700 1 -- 172.21.2.108:0/3465506227 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ce018ecc0 msgr2=0x7f4ce0195740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:53.963 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.963 7f4ce6174700 1 --2- 172.21.2.108:0/3465506227 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ce018ecc0 0x7f4ce0195740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:53.963 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.963 7f4ce6174700 1 -- 172.21.2.108:0/3465506227 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ce019c280 msgr2=0x7f4ce019c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:53.963 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.963 7f4ce6174700 1 --2- 172.21.2.108:0/3465506227 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ce019c280 0x7f4ce019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:12:53.963 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.963 7f4ce6174700 1 -- 172.21.2.108:0/3465506227 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce019cec0 con 0x7f4ce01960f0 2021-02-10T11:12:53.963 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.963 7f4ce6174700 1 --2- 172.21.2.108:0/3465506227 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ce01960f0 0x7f4ce019b6a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f4cdc00b5b0 tx=0x7f4cdc00b950).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:53.963 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.963 7f4cd2ffd700 1 -- 172.21.2.108:0/3465506227 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4cdc00bd40 con 0x7f4ce01960f0 2021-02-10T11:12:53.964 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.963 7f4cd2ffd700 1 -- 172.21.2.108:0/3465506227 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cdc006240 con 0x7f4ce01960f0 2021-02-10T11:12:53.964 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.963 7f4cd2ffd700 1 -- 172.21.2.108:0/3465506227 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4cdc005720 con 0x7f4ce01960f0 2021-02-10T11:12:53.964 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.963 7f4ce8358700 1 -- 172.21.2.108:0/3465506227 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce019cec0 con 0x7f4ce01960f0 2021-02-10T11:12:53.964 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.963 7f4ce8358700 1 -- 172.21.2.108:0/3465506227 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce019cec0 con 0x7f4ce01960f0 2021-02-10T11:12:53.964 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.964 7f4cd2ffd700 1 -- 172.21.2.108:0/3465506227 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4cdc0054a0 con 0x7f4ce01960f0 2021-02-10T11:12:53.964 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.964 7f4cd2ffd700 1 --2- 172.21.2.108:0/3465506227 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cc80186c0 0x7f4cc801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:53.964 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.964 7f4cd2ffd700 1 -- 172.21.2.108:0/3465506227 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7f4cdc0312b0 con 0x7f4ce01960f0 2021-02-10T11:12:53.964 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.964 7f4ce5973700 1 --2- 172.21.2.108:0/3465506227 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cc80186c0 0x7f4cc801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:53.964 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.965 7f4ce5973700 1 --2- 172.21.2.108:0/3465506227 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cc80186c0 0x7f4cc801abc0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:53.992 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.995 7f4ce8358700 1 -- 172.21.2.108:0/3465506227 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce019bd50 con 0x7f4ce01960f0 2021-02-10T11:12:53.995 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:53.998 7f4cd2ffd700 1 -- 172.21.2.108:0/3465506227 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4cdc030d30 con 0x7f4ce01960f0 2021-02-10T11:12:54.130 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.132 7f4ce8358700 1 -- 172.21.2.108:0/3465506227 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5", "id": 0} v 0) v1 -- 0x7f4ce019bd50 con 0x7f4ce01960f0 2021-02-10T11:12:54.135 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.138 7f4cd2ffd700 1 -- 172.21.2.108:0/3465506227 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4cdc00bd40 con 0x7f4ce01960f0 2021-02-10T11:12:54.137 INFO:teuthology.orchestra.run.gibba008.stdout:0 2021-02-10T11:12:54.139 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.141 7f4ce8358700 1 -- 172.21.2.108:0/3465506227 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cc80186c0 msgr2=0x7f4cc801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:54.139 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.141 7f4ce8358700 1 --2- 172.21.2.108:0/3465506227 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cc80186c0 0x7f4cc801abc0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:54.139 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.141 7f4ce8358700 1 -- 172.21.2.108:0/3465506227 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ce01960f0 msgr2=0x7f4ce019b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:54.139 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.141 7f4ce8358700 1 --2- 172.21.2.108:0/3465506227 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ce01960f0 0x7f4ce019b6a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f4cdc00b5b0 tx=0x7f4cdc00b950).stop 2021-02-10T11:12:54.139 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.141 7f4ce8358700 1 -- 172.21.2.108:0/3465506227 shutdown_connections 2021-02-10T11:12:54.139 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.141 7f4ce8358700 1 --2- 172.21.2.108:0/3465506227 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ce019c280 0x7f4ce019c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:12:54.139 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.141 7f4ce8358700 1 --2- 172.21.2.108:0/3465506227 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ce018ecc0 0x7f4ce0195740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:54.139 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.141 7f4ce8358700 1 --2- 172.21.2.108:0/3465506227 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cc80186c0 0x7f4cc801abc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:54.139 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.141 7f4ce8358700 1 --2- 172.21.2.108:0/3465506227 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ce01960f0 0x7f4ce019b6a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:54.139 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.141 7f4ce8358700 1 -- 172.21.2.108:0/3465506227 >> 172.21.2.108:0/3465506227 conn(0x7f4ce019f7e0 msgr2=0x7f4ce018c240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:12:54.140 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.141 7f4ce8358700 1 -- 172.21.2.108:0/3465506227 shutdown_connections 2021-02-10T11:12:54.140 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.142 7f4ce8358700 1 -- 172.21.2.108:0/3465506227 wait complete. 2021-02-10T11:12:54.146 DEBUG:teuthology.orchestra.run.gibba008:> sudo ceph --cluster ceph osd new ddc04ce3-b313-4707-8270-585391e38ecc 1 2021-02-10T11:12:54.572 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.558 7f956c5c3700 1 Processor -- start 2021-02-10T11:12:54.723 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.558 7f956c5c3700 1 -- start start 2021-02-10T11:12:54.724 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.559 7f956c5c3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95640f0630 0x7f95640f10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:54.724 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.559 7f956c5c3700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95640f0b20 con 0x7f95640d17f0 2021-02-10T11:12:54.724 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.559 7f956c5c3700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95640d1bf0 con 0x7f95640cff00 2021-02-10T11:12:54.724 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.559 7f956c5c3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95640e5eb0 con 0x7f95640f0630 2021-02-10T11:12:54.724 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.559 7f956abe0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95640f0630 0x7f95640f10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:54.724 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.559 7f9569bde700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f95640cff00 0x7f95641b39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.108:33326/0 (socket says 172.21.2.108:33326) 2021-02-10T11:12:54.724 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.559 7f9569bde700 1 -- 172.21.2.108:0/859630667 learned_addr learned my addr 172.21.2.108:0/859630667 (peer_addr_for_me v1:172.21.2.108:0/0) 2021-02-10T11:12:54.724 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.559 7f956a3df700 1 --1- 172.21.2.108:0/859630667 >> v1:172.21.2.101:6789/0 conn(0x7f95640d17f0 0x7f95641b31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.108:55186/0 (socket says 172.21.2.108:55186) 2021-02-10T11:12:54.724 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.560 7f956abe0700 1 -- 172.21.2.108:0/859630667 >> v1:172.21.2.108:6789/0 conn(0x7f95640cff00 legacy=0x7f95641b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:54.724 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.560 7f956abe0700 1 -- 172.21.2.108:0/859630667 >> v1:172.21.2.101:6789/0 conn(0x7f95640d17f0 legacy=0x7f95641b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:54.725 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.560 7f95693dd700 1 -- 172.21.2.108:0/859630667 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 401883346 0 0) 0x7f9558001100 con 0x7f95640cff00 2021-02-10T11:12:54.725 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.560 7f956abe0700 1 -- 172.21.2.108:0/859630667 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95640d1e40 con 0x7f95640f0630 2021-02-10T11:12:54.725 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.560 7f956abe0700 1 --2- 172.21.2.108:0/859630667 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95640f0630 0x7f95640f10a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f9554006ab0 tx=0x7f9554009ac0).ready entity=mon.2 client_cookie=fc28d6dc49c4c607 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:54.725 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.560 7f95693dd700 1 -- 172.21.2.108:0/859630667 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f95540066d0 con 0x7f95640f0630 2021-02-10T11:12:54.725 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.560 7f95693dd700 1 -- 172.21.2.108:0/859630667 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95540022e0 con 0x7f95640f0630 2021-02-10T11:12:54.725 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.560 7f95693dd700 1 -- 172.21.2.108:0/859630667 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f95540027c0 con 0x7f95640f0630 2021-02-10T11:12:54.725 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.560 7f956c5c3700 1 -- 172.21.2.108:0/859630667 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95640f0630 msgr2=0x7f95640f10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:54.725 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.560 7f956c5c3700 1 --2- 172.21.2.108:0/859630667 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95640f0630 0x7f95640f10a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f9554006ab0 tx=0x7f9554009ac0).stop 2021-02-10T11:12:54.725 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.561 7f956c5c3700 1 -- 172.21.2.108:0/859630667 shutdown_connections 2021-02-10T11:12:54.725 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.561 7f956c5c3700 1 --2- 172.21.2.108:0/859630667 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95640f0630 0x7f95640f10a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:54.725 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.561 7f956c5c3700 1 -- 172.21.2.108:0/859630667 >> 172.21.2.108:0/859630667 conn(0x7f95640fde40 msgr2=0x7f95640ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:12:54.725 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.561 7f956c5c3700 1 -- 172.21.2.108:0/859630667 shutdown_connections 2021-02-10T11:12:54.726 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.561 7f956c5c3700 1 -- 172.21.2.108:0/859630667 wait complete. 2021-02-10T11:12:54.728 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.575 7f956c5c3700 1 Processor -- start 2021-02-10T11:12:54.728 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.576 7f956c5c3700 1 -- start start 2021-02-10T11:12:54.728 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.576 7f956c5c3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9564097020 0x7f95641a5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:54.728 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.576 7f956c5c3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95640a3570 0x7f95640a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:54.728 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.576 7f956c5c3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f95640a5800 0x7f95640a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:54.728 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.576 7f956c5c3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95640a3f90 con 0x7f95640a3570 2021-02-10T11:12:54.728 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.576 7f956c5c3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9564097420 con 0x7f95640a5800 2021-02-10T11:12:54.728 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.576 7f956c5c3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95640a6220 con 0x7f9564097020 2021-02-10T11:12:54.728 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.576 7f956a3df700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9564097020 0x7f95641a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:54.728 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.576 7f956a3df700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9564097020 0x7f95641a5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.108:44310/0 (socket says 172.21.2.108:44310) 2021-02-10T11:12:54.728 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.576 7f956a3df700 1 -- 172.21.2.108:0/2758221837 learned_addr learned my addr 172.21.2.108:0/2758221837 (peer_addr_for_me v2:172.21.2.108:0/0) 2021-02-10T11:12:54.728 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.576 7f9569bde700 1 --2- 172.21.2.108:0/2758221837 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95640a3570 0x7f95640a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:54.729 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.576 7f956abe0700 1 --2- 172.21.2.108:0/2758221837 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f95640a5800 0x7f95640a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:54.729 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.577 7f956a3df700 1 -- 172.21.2.108:0/2758221837 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f95640a5800 msgr2=0x7f95640a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:54.729 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.577 7f956a3df700 1 --2- 172.21.2.108:0/2758221837 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f95640a5800 0x7f95640a5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:54.729 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.577 7f956a3df700 1 -- 172.21.2.108:0/2758221837 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95640a3570 msgr2=0x7f95640a3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:54.729 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.577 7f956a3df700 1 --2- 172.21.2.108:0/2758221837 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95640a3570 0x7f95640a3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:54.729 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.577 7f956a3df700 1 -- 172.21.2.108:0/2758221837 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95641a5630 con 0x7f9564097020 2021-02-10T11:12:54.729 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.577 7f956a3df700 1 --2- 172.21.2.108:0/2758221837 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9564097020 0x7f95641a5010 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f955c00a950 tx=0x7f955c00acf0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:54.729 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.577 7f9562ffd700 1 -- 172.21.2.108:0/2758221837 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f955c0011a0 con 0x7f9564097020 2021-02-10T11:12:54.729 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.577 7f9562ffd700 1 -- 172.21.2.108:0/2758221837 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955c007160 con 0x7f9564097020 2021-02-10T11:12:54.729 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.577 7f9562ffd700 1 -- 172.21.2.108:0/2758221837 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f955c0046a0 con 0x7f9564097020 2021-02-10T11:12:54.729 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.577 7f956c5c3700 1 -- 172.21.2.108:0/2758221837 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95641a5630 con 0x7f9564097020 2021-02-10T11:12:54.729 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.577 7f956c5c3700 1 -- 172.21.2.108:0/2758221837 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95641a5630 con 0x7f9564097020 2021-02-10T11:12:54.729 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.578 7f9562ffd700 1 -- 172.21.2.108:0/2758221837 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f955c007340 con 0x7f9564097020 2021-02-10T11:12:54.729 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.578 7f9562ffd700 1 --2- 172.21.2.108:0/2758221837 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95440186c0 0x7f954401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:54.730 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.578 7f9562ffd700 1 -- 172.21.2.108:0/2758221837 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 857+0+0 (secure 0 0 0) 0x7f955c0316c0 con 0x7f9564097020 2021-02-10T11:12:54.730 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.578 7f9569bde700 1 --2- 172.21.2.108:0/2758221837 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95440186c0 0x7f954401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:54.730 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.579 7f9569bde700 1 --2- 172.21.2.108:0/2758221837 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95440186c0 0x7f954401abc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:54.730 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.609 7f956c5c3700 1 -- 172.21.2.108:0/2758221837 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95640a3040 con 0x7f9564097020 2021-02-10T11:12:54.730 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.612 7f9562ffd700 1 -- 172.21.2.108:0/2758221837 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f955c02cf80 con 0x7f9564097020 2021-02-10T11:12:54.743 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.746 7f956c5c3700 1 -- 172.21.2.108:0/2758221837 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ddc04ce3-b313-4707-8270-585391e38ecc", "id": 1} v 0) v1 -- 0x7f95640a3040 con 0x7f9564097020 2021-02-10T11:12:54.749 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.751 7f9562ffd700 1 -- 172.21.2.108:0/2758221837 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ddc04ce3-b313-4707-8270-585391e38ecc", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f955c02cf80 con 0x7f9564097020 2021-02-10T11:12:54.751 INFO:teuthology.orchestra.run.gibba008.stdout:1 2021-02-10T11:12:54.753 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.755 7f956c5c3700 1 -- 172.21.2.108:0/2758221837 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95440186c0 msgr2=0x7f954401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:54.753 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.755 7f956c5c3700 1 --2- 172.21.2.108:0/2758221837 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95440186c0 0x7f954401abc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:54.753 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.755 7f956c5c3700 1 -- 172.21.2.108:0/2758221837 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9564097020 msgr2=0x7f95641a5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:54.753 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.755 7f956c5c3700 1 --2- 172.21.2.108:0/2758221837 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9564097020 0x7f95641a5010 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f955c00a950 tx=0x7f955c00acf0).stop 2021-02-10T11:12:54.753 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.755 7f956c5c3700 1 -- 172.21.2.108:0/2758221837 shutdown_connections 2021-02-10T11:12:54.753 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.755 7f956c5c3700 1 --2- 172.21.2.108:0/2758221837 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f95640a5800 0x7f95640a5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:54.753 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.755 7f956c5c3700 1 --2- 172.21.2.108:0/2758221837 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95640a3570 0x7f95640a3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:54.753 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.755 7f956c5c3700 1 --2- 172.21.2.108:0/2758221837 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95440186c0 0x7f954401abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:54.753 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.755 7f956c5c3700 1 --2- 172.21.2.108:0/2758221837 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9564097020 0x7f95641a5010 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:54.754 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.755 7f956c5c3700 1 -- 172.21.2.108:0/2758221837 >> 172.21.2.108:0/2758221837 conn(0x7f95641a37d0 msgr2=0x7f95641b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:12:54.754 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.755 7f956c5c3700 1 -- 172.21.2.108:0/2758221837 shutdown_connections 2021-02-10T11:12:54.754 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:54.755 7f956c5c3700 1 -- 172.21.2.108:0/2758221837 wait complete. 2021-02-10T11:12:54.761 DEBUG:teuthology.orchestra.run.gibba008:> sudo ceph --cluster ceph osd new e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3 2 2021-02-10T11:12:55.188 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.173 7fc9e4dbe700 1 Processor -- start 2021-02-10T11:12:55.188 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.173 7fc9e4dbe700 1 -- start start 2021-02-10T11:12:55.188 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.174 7fc9e4dbe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e00d17f0 0x7fc9e01b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:55.188 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.174 7fc9e4dbe700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e00f0b20 con 0x7fc9e00d1bf0 2021-02-10T11:12:55.189 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.174 7fc9e4dbe700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e01b3ff0 con 0x7fc9e00cff00 2021-02-10T11:12:55.189 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.174 7fc9e4dbe700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e00e5eb0 con 0x7fc9e00d17f0 2021-02-10T11:12:55.189 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.174 7fc9deb13700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e00d17f0 0x7fc9e01b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:55.189 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.174 7fc9deb13700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e00d17f0 0x7fc9e01b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.108:44318/0 (socket says 172.21.2.108:44318) 2021-02-10T11:12:55.189 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.174 7fc9deb13700 1 -- 172.21.2.108:0/2127251863 learned_addr learned my addr 172.21.2.108:0/2127251863 (peer_addr_for_me v2:172.21.2.108:0/0) 2021-02-10T11:12:55.189 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.174 7fc9deb13700 1 -- 172.21.2.108:0/2127251863 >> v1:172.21.2.108:6789/0 conn(0x7fc9e00cff00 legacy=0x7fc9e00ef620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:55.189 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.174 7fc9deb13700 1 -- 172.21.2.108:0/2127251863 >> v1:172.21.2.101:6789/0 conn(0x7fc9e00d1bf0 legacy=0x7fc9e01b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:12:55.189 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.174 7fc9deb13700 1 -- 172.21.2.108:0/2127251863 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e00f20b0 con 0x7fc9e00d17f0 2021-02-10T11:12:55.189 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.175 7fc9deb13700 1 --2- 172.21.2.108:0/2127251863 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e00d17f0 0x7fc9e01b31e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fc9d0006700 tx=0x7fc9d0009b00).ready entity=mon.2 client_cookie=b15300e7bb77be25 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:55.189 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.175 7fc9ddb11700 1 -- 172.21.2.108:0/2127251863 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc9d000c0c0 con 0x7fc9e00d17f0 2021-02-10T11:12:55.189 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.175 7fc9ddb11700 1 -- 172.21.2.108:0/2127251863 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d000c400 con 0x7fc9e00d17f0 2021-02-10T11:12:55.189 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.175 7fc9ddb11700 1 -- 172.21.2.108:0/2127251863 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc9d000c860 con 0x7fc9e00d17f0 2021-02-10T11:12:55.189 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.175 7fc9e4dbe700 1 -- 172.21.2.108:0/2127251863 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e00d17f0 msgr2=0x7fc9e01b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:55.190 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.175 7fc9e4dbe700 1 --2- 172.21.2.108:0/2127251863 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e00d17f0 0x7fc9e01b31e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fc9d0006700 tx=0x7fc9d0009b00).stop 2021-02-10T11:12:55.190 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.176 7fc9e4dbe700 1 -- 172.21.2.108:0/2127251863 shutdown_connections 2021-02-10T11:12:55.190 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.176 7fc9e4dbe700 1 --2- 172.21.2.108:0/2127251863 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e00d17f0 0x7fc9e01b31e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:55.190 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.176 7fc9e4dbe700 1 -- 172.21.2.108:0/2127251863 >> 172.21.2.108:0/2127251863 conn(0x7fc9e00fde40 msgr2=0x7fc9e00ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:12:55.190 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.176 7fc9e4dbe700 1 -- 172.21.2.108:0/2127251863 shutdown_connections 2021-02-10T11:12:55.190 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.176 7fc9e4dbe700 1 -- 172.21.2.108:0/2127251863 wait complete. 2021-02-10T11:12:55.192 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.191 7fc9e4dbe700 1 Processor -- start 2021-02-10T11:12:55.192 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.191 7fc9e4dbe700 1 -- start start 2021-02-10T11:12:55.192 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.192 7fc9e4dbe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e00b8440 0x7fc9e010aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:55.192 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.192 7fc9e4dbe700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9e00b8840 0x7fc9e010b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:55.192 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.192 7fc9e4dbe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9e010b6e0 0x7fc9e00c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:55.192 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.192 7fc9e4dbe700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e008fb50 con 0x7fc9e010b6e0 2021-02-10T11:12:55.192 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.192 7fc9e4dbe700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e00c1c10 con 0x7fc9e00b8840 2021-02-10T11:12:55.192 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.192 7fc9e4dbe700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e00c1e30 con 0x7fc9e00b8440 2021-02-10T11:12:55.193 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.192 7fc9deb13700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e00b8440 0x7fc9e010aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:55.193 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.192 7fc9deb13700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e00b8440 0x7fc9e010aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.108:44324/0 (socket says 172.21.2.108:44324) 2021-02-10T11:12:55.193 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.192 7fc9deb13700 1 -- 172.21.2.108:0/2239530009 learned_addr learned my addr 172.21.2.108:0/2239530009 (peer_addr_for_me v2:172.21.2.108:0/0) 2021-02-10T11:12:55.193 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.192 7fc9de312700 1 --2- 172.21.2.108:0/2239530009 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9e00b8840 0x7fc9e010b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:55.193 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.192 7fc9df314700 1 --2- 172.21.2.108:0/2239530009 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9e010b6e0 0x7fc9e00c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:55.193 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.192 7fc9deb13700 1 -- 172.21.2.108:0/2239530009 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9e00b8840 msgr2=0x7fc9e010b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:55.193 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.192 7fc9deb13700 1 --2- 172.21.2.108:0/2239530009 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9e00b8840 0x7fc9e010b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:55.193 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.192 7fc9deb13700 1 -- 172.21.2.108:0/2239530009 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9e010b6e0 msgr2=0x7fc9e00c25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:55.193 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.192 7fc9deb13700 1 --2- 172.21.2.108:0/2239530009 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9e010b6e0 0x7fc9e00c25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:55.193 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.192 7fc9deb13700 1 -- 172.21.2.108:0/2239530009 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e00c2bc0 con 0x7fc9e00b8440 2021-02-10T11:12:55.193 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.192 7fc9df314700 1 --2- 172.21.2.108:0/2239530009 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9e010b6e0 0x7fc9e00c25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:12:55.193 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.192 7fc9deb13700 1 --2- 172.21.2.108:0/2239530009 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e00b8440 0x7fc9e010aaa0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fc9d000b660 tx=0x7fc9d000ba00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:55.193 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.192 7fc9cf7fe700 1 -- 172.21.2.108:0/2239530009 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc9d0009440 con 0x7fc9e00b8440 2021-02-10T11:12:55.194 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.193 7fc9cf7fe700 1 -- 172.21.2.108:0/2239530009 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d0006390 con 0x7fc9e00b8440 2021-02-10T11:12:55.194 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.193 7fc9e4dbe700 1 -- 172.21.2.108:0/2239530009 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e00c2bc0 con 0x7fc9e00b8440 2021-02-10T11:12:55.194 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.193 7fc9cf7fe700 1 -- 172.21.2.108:0/2239530009 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc9d0002030 con 0x7fc9e00b8440 2021-02-10T11:12:55.194 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.193 7fc9e4dbe700 1 -- 172.21.2.108:0/2239530009 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e00c2bc0 con 0x7fc9e00b8440 2021-02-10T11:12:55.194 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.193 7fc9cf7fe700 1 -- 172.21.2.108:0/2239530009 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc9d00093e0 con 0x7fc9e00b8440 2021-02-10T11:12:55.194 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.193 7fc9cf7fe700 1 --2- 172.21.2.108:0/2239530009 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9c4018730 0x7fc9c401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:55.194 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.193 7fc9cf7fe700 1 -- 172.21.2.108:0/2239530009 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 960+0+0 (secure 0 0 0) 0x7fc9d0031450 con 0x7fc9e00b8440 2021-02-10T11:12:55.194 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.193 7fc9de312700 1 --2- 172.21.2.108:0/2239530009 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9c4018730 0x7fc9c401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:55.194 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.194 7fc9de312700 1 --2- 172.21.2.108:0/2239530009 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9c4018730 0x7fc9c401ac10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:55.221 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.224 7fc9e4dbe700 1 -- 172.21.2.108:0/2239530009 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e010a080 con 0x7fc9e00b8440 2021-02-10T11:12:55.224 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.226 7fc9cf7fe700 1 -- 172.21.2.108:0/2239530009 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc9d0006370 con 0x7fc9e00b8440 2021-02-10T11:12:55.359 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.362 7fc9e4dbe700 1 -- 172.21.2.108:0/2239530009 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3", "id": 2} v 0) v1 -- 0x7fc9e010a080 con 0x7fc9e00b8440 2021-02-10T11:12:55.364 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.366 7fc9cf7fe700 1 -- 172.21.2.108:0/2239530009 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc9d000bdf0 con 0x7fc9e00b8440 2021-02-10T11:12:55.366 INFO:teuthology.orchestra.run.gibba008.stdout:2 2021-02-10T11:12:55.368 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.370 7fc9e4dbe700 1 -- 172.21.2.108:0/2239530009 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9c4018730 msgr2=0x7fc9c401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:55.368 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.370 7fc9e4dbe700 1 --2- 172.21.2.108:0/2239530009 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9c4018730 0x7fc9c401ac10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:55.368 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.370 7fc9e4dbe700 1 -- 172.21.2.108:0/2239530009 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e00b8440 msgr2=0x7fc9e010aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:55.369 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.370 7fc9e4dbe700 1 --2- 172.21.2.108:0/2239530009 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e00b8440 0x7fc9e010aaa0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fc9d000b660 tx=0x7fc9d000ba00).stop 2021-02-10T11:12:55.369 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.370 7fc9e4dbe700 1 -- 172.21.2.108:0/2239530009 shutdown_connections 2021-02-10T11:12:55.369 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.370 7fc9e4dbe700 1 --2- 172.21.2.108:0/2239530009 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9e010b6e0 0x7fc9e00c25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:55.369 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.370 7fc9e4dbe700 1 --2- 172.21.2.108:0/2239530009 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9e00b8840 0x7fc9e010b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:55.369 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.370 7fc9e4dbe700 1 --2- 172.21.2.108:0/2239530009 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9c4018730 0x7fc9c401ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:55.369 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.370 7fc9e4dbe700 1 --2- 172.21.2.108:0/2239530009 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e00b8440 0x7fc9e010aaa0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:55.369 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.370 7fc9e4dbe700 1 -- 172.21.2.108:0/2239530009 >> 172.21.2.108:0/2239530009 conn(0x7fc9e00a1d50 msgr2=0x7fc9e00a0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:12:55.369 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.371 7fc9e4dbe700 1 -- 172.21.2.108:0/2239530009 shutdown_connections 2021-02-10T11:12:55.369 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.371 7fc9e4dbe700 1 -- 172.21.2.108:0/2239530009 wait complete. 2021-02-10T11:12:55.377 DEBUG:teuthology.orchestra.run.gibba008:> sudo ceph --cluster ceph osd new 0d9ef9b1-98aa-4d37-aa16-6305c7d46d52 3 2021-02-10T11:12:55.803 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.789 7fba0a3fd700 1 Processor -- start 2021-02-10T11:12:55.803 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.789 7fba0a3fd700 1 -- start start 2021-02-10T11:12:55.803 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.789 7fba0a3fd700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fba040d17f0 0x7fba041b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:55.803 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.789 7fba0a3fd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba040cff00 0x7fba040ef620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:55.803 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.789 7fba0a3fd700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba040f0b20 con 0x7fba040d1bf0 2021-02-10T11:12:55.803 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.789 7fba0a3fd700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba041b3ff0 con 0x7fba040d17f0 2021-02-10T11:12:55.803 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.789 7fba0a3fd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba040f20b0 con 0x7fba040cff00 2021-02-10T11:12:55.804 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.789 7fba08a1a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba040cff00 0x7fba040ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:55.804 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.789 7fba08a1a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba040cff00 0x7fba040ef620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.108:44328/0 (socket says 172.21.2.108:44328) 2021-02-10T11:12:55.804 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.789 7fba08a1a700 1 -- 172.21.2.108:0/863205638 learned_addr learned my addr 172.21.2.108:0/863205638 (peer_addr_for_me v2:172.21.2.108:0/0) 2021-02-10T11:12:55.804 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.789 7fba03fff700 1 --2- 172.21.2.108:0/863205638 >> v2:172.21.2.108:3300/0 conn(0x7fba040d17f0 0x7fba041b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:55.804 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.789 7fba08a1a700 1 -- 172.21.2.108:0/863205638 >> v2:172.21.2.108:3300/0 conn(0x7fba040d17f0 msgr2=0x7fba041b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:12:55.804 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.789 7fba08a1a700 1 --2- 172.21.2.108:0/863205638 >> v2:172.21.2.108:3300/0 conn(0x7fba040d17f0 0x7fba041b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:55.804 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.789 7fba08a1a700 1 -- 172.21.2.108:0/863205638 >> v1:172.21.2.108:6789/0 conn(0x7fba040d1bf0 legacy=0x7fba041b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:12:55.804 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.789 7fba08a1a700 1 -- 172.21.2.108:0/863205638 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba040f22d0 con 0x7fba040cff00 2021-02-10T11:12:55.804 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.790 7fba08a1a700 1 --2- 172.21.2.108:0/863205638 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba040cff00 0x7fba040ef620 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fb9fc006f00 tx=0x7fb9fc009e90).ready entity=mon.2 client_cookie=3ff43ea7f8aad976 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:55.804 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.790 7fba02ffd700 1 -- 172.21.2.108:0/863205638 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb9fc00b300 con 0x7fba040cff00 2021-02-10T11:12:55.804 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.790 7fba02ffd700 1 -- 172.21.2.108:0/863205638 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9fc00b640 con 0x7fba040cff00 2021-02-10T11:12:55.804 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.790 7fba02ffd700 1 -- 172.21.2.108:0/863205638 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb9fc00baa0 con 0x7fba040cff00 2021-02-10T11:12:55.805 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.790 7fba0a3fd700 1 -- 172.21.2.108:0/863205638 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba040cff00 msgr2=0x7fba040ef620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:55.805 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.790 7fba0a3fd700 1 --2- 172.21.2.108:0/863205638 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba040cff00 0x7fba040ef620 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fb9fc006f00 tx=0x7fb9fc009e90).stop 2021-02-10T11:12:55.805 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.791 7fba0a3fd700 1 -- 172.21.2.108:0/863205638 shutdown_connections 2021-02-10T11:12:55.805 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.791 7fba0a3fd700 1 --2- 172.21.2.108:0/863205638 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba040cff00 0x7fba040ef620 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:55.805 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.791 7fba0a3fd700 1 --2- 172.21.2.108:0/863205638 >> v2:172.21.2.108:3300/0 conn(0x7fba040d17f0 0x7fba041b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:55.805 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.791 7fba0a3fd700 1 -- 172.21.2.108:0/863205638 >> 172.21.2.108:0/863205638 conn(0x7fba040fde40 msgr2=0x7fba040ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:12:55.805 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.791 7fba0a3fd700 1 -- 172.21.2.108:0/863205638 shutdown_connections 2021-02-10T11:12:55.805 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.791 7fba0a3fd700 1 -- 172.21.2.108:0/863205638 wait complete. 2021-02-10T11:12:55.805 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.805 7fba0a3fd700 1 Processor -- start 2021-02-10T11:12:55.805 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.806 7fba0a3fd700 1 -- start start 2021-02-10T11:12:55.807 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.806 7fba0a3fd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba0409a4a0 0x7fba040a7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:55.807 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.806 7fba0a3fd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba040998b0 0x7fba04104580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:55.807 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.806 7fba0a3fd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba04104ba0 0x7fba0418bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:55.807 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.806 7fba0a3fd700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba041081c0 con 0x7fba040998b0 2021-02-10T11:12:55.807 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.806 7fba0a3fd700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0418b310 con 0x7fba04104ba0 2021-02-10T11:12:55.807 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.806 7fba0a3fd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba040a82c0 con 0x7fba0409a4a0 2021-02-10T11:12:55.807 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.806 7fba03fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba0409a4a0 0x7fba040a7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:55.808 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.806 7fba03fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba0409a4a0 0x7fba040a7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.108:44332/0 (socket says 172.21.2.108:44332) 2021-02-10T11:12:55.808 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.806 7fba03fff700 1 -- 172.21.2.108:0/856180323 learned_addr learned my addr 172.21.2.108:0/856180323 (peer_addr_for_me v2:172.21.2.108:0/0) 2021-02-10T11:12:55.808 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.806 7fba08a1a700 1 --2- 172.21.2.108:0/856180323 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba04104ba0 0x7fba0418bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:55.808 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.806 7fba03fff700 1 -- 172.21.2.108:0/856180323 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba04104ba0 msgr2=0x7fba0418bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:55.808 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.806 7fba03fff700 1 --2- 172.21.2.108:0/856180323 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba04104ba0 0x7fba0418bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:55.808 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.806 7fba03fff700 1 -- 172.21.2.108:0/856180323 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba040998b0 msgr2=0x7fba04104580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:12:55.808 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.806 7fba03fff700 1 --2- 172.21.2.108:0/856180323 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba040998b0 0x7fba04104580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:12:55.808 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.806 7fba03fff700 1 -- 172.21.2.108:0/856180323 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba04104fa0 con 0x7fba0409a4a0 2021-02-10T11:12:55.808 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.806 7fba03fff700 1 --2- 172.21.2.108:0/856180323 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba0409a4a0 0x7fba040a7ca0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fb9f4006f60 tx=0x7fb9f400b970).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:55.808 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.807 7fba00ff9700 1 -- 172.21.2.108:0/856180323 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb9f400bd60 con 0x7fba0409a4a0 2021-02-10T11:12:55.808 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.807 7fba00ff9700 1 -- 172.21.2.108:0/856180323 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f4000c90 con 0x7fba0409a4a0 2021-02-10T11:12:55.808 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.807 7fba00ff9700 1 -- 172.21.2.108:0/856180323 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb9f40022b0 con 0x7fba0409a4a0 2021-02-10T11:12:55.808 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.807 7fba0a3fd700 1 -- 172.21.2.108:0/856180323 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba04104fa0 con 0x7fba0409a4a0 2021-02-10T11:12:55.809 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.807 7fba0a3fd700 1 -- 172.21.2.108:0/856180323 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba04104fa0 con 0x7fba0409a4a0 2021-02-10T11:12:55.809 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.807 7fba00ff9700 1 -- 172.21.2.108:0/856180323 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb9f4006b00 con 0x7fba0409a4a0 2021-02-10T11:12:55.809 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.807 7fba00ff9700 1 --2- 172.21.2.108:0/856180323 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9e00186d0 0x7fb9e001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:55.809 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.807 7fba00ff9700 1 -- 172.21.2.108:0/856180323 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1063+0+0 (secure 0 0 0) 0x7fb9f40315f0 con 0x7fba0409a4a0 2021-02-10T11:12:55.809 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.808 7fba037fe700 1 --2- 172.21.2.108:0/856180323 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9e00186d0 0x7fb9e001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:55.809 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.809 7fba037fe700 1 --2- 172.21.2.108:0/856180323 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9e00186d0 0x7fb9e001abd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:55.836 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.839 7fba0a3fd700 1 -- 172.21.2.108:0/856180323 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba040f0b20 con 0x7fba0409a4a0 2021-02-10T11:12:55.839 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.842 7fba00ff9700 1 -- 172.21.2.108:0/856180323 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb9f4000c90 con 0x7fba0409a4a0 2021-02-10T11:12:55.974 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.976 7fba0a3fd700 1 -- 172.21.2.108:0/856180323 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0d9ef9b1-98aa-4d37-aa16-6305c7d46d52", "id": 3} v 0) v1 -- 0x7fba040f0b20 con 0x7fba0409a4a0 2021-02-10T11:12:56.057 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.982 7fba00ff9700 1 -- 172.21.2.108:0/856180323 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0d9ef9b1-98aa-4d37-aa16-6305c7d46d52", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb9f400bd60 con 0x7fba0409a4a0 2021-02-10T11:12:56.057 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.986 7fba0a3fd700 1 -- 172.21.2.108:0/856180323 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9e00186d0 msgr2=0x7fb9e001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:56.057 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.986 7fba0a3fd700 1 --2- 172.21.2.108:0/856180323 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9e00186d0 0x7fb9e001abd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:56.057 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.986 7fba0a3fd700 1 -- 172.21.2.108:0/856180323 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba0409a4a0 msgr2=0x7fba040a7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:56.057 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.986 7fba0a3fd700 1 --2- 172.21.2.108:0/856180323 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba0409a4a0 0x7fba040a7ca0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fb9f4006f60 tx=0x7fb9f400b970).stop 2021-02-10T11:12:56.057 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.986 7fba0a3fd700 1 -- 172.21.2.108:0/856180323 shutdown_connections 2021-02-10T11:12:56.058 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.986 7fba0a3fd700 1 --2- 172.21.2.108:0/856180323 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba04104ba0 0x7fba0418bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:56.058 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.986 7fba0a3fd700 1 --2- 172.21.2.108:0/856180323 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba040998b0 0x7fba04104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:12:56.058 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.986 7fba0a3fd700 1 --2- 172.21.2.108:0/856180323 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9e00186d0 0x7fb9e001abd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:56.058 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.986 7fba0a3fd700 1 --2- 172.21.2.108:0/856180323 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba0409a4a0 0x7fba040a7ca0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:56.058 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.986 7fba0a3fd700 1 -- 172.21.2.108:0/856180323 >> 172.21.2.108:0/856180323 conn(0x7fba0419c4e0 msgr2=0x7fba04199310 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:12:56.058 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.986 7fba0a3fd700 1 -- 172.21.2.108:0/856180323 shutdown_connections 2021-02-10T11:12:56.058 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:55.986 7fba0a3fd700 1 -- 172.21.2.108:0/856180323 wait complete. 2021-02-10T11:12:56.058 INFO:teuthology.orchestra.run.gibba008.stdout:3 2021-02-10T11:12:56.058 DEBUG:teuthology.orchestra.run.gibba008:> sudo ceph --cluster ceph osd new 36742c21-bfeb-4e11-bce9-8e745378b605 4 2021-02-10T11:12:56.486 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.472 7f6265d3c700 1 Processor -- start 2021-02-10T11:12:56.486 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.472 7f6265d3c700 1 -- start start 2021-02-10T11:12:56.486 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.472 7f6265d3c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f62600d17f0 0x7f62601b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:56.486 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.472 7f6265d3c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f62600cff00 0x7f62600ef620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:56.486 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.472 7f6265d3c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62600f0b20 con 0x7f62600d1bf0 2021-02-10T11:12:56.487 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.472 7f6265d3c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62601b3ff0 con 0x7f62600cff00 2021-02-10T11:12:56.487 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.472 7f6265d3c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62600f20b0 con 0x7f62600d17f0 2021-02-10T11:12:56.487 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.473 7f625f7fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f62600d17f0 0x7f62601b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:56.487 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.473 7f625effd700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f62600d1bf0 0x7f62601b3800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.108:33362/0 (socket says 172.21.2.108:33362) 2021-02-10T11:12:56.487 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.473 7f625effd700 1 -- 172.21.2.108:0/3388732941 learned_addr learned my addr 172.21.2.108:0/3388732941 (peer_addr_for_me v1:172.21.2.108:0/0) 2021-02-10T11:12:56.487 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.473 7f625f7fe700 1 -- 172.21.2.108:0/3388732941 >> v2:172.21.2.101:3300/0 conn(0x7f62600cff00 msgr2=0x7f62600ef620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:12:56.487 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.473 7f625f7fe700 1 --2- 172.21.2.108:0/3388732941 >> v2:172.21.2.101:3300/0 conn(0x7f62600cff00 0x7f62600ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:12:56.487 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.473 7f625f7fe700 1 -- 172.21.2.108:0/3388732941 >> v1:172.21.2.108:6789/0 conn(0x7f62600d1bf0 legacy=0x7f62601b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:56.487 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.473 7f625f7fe700 1 -- 172.21.2.108:0/3388732941 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62600f22d0 con 0x7f62600d17f0 2021-02-10T11:12:56.487 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.473 7f625f7fe700 1 --2- 172.21.2.108:0/3388732941 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62600d17f0 0x7f62601b31e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f6250006650 tx=0x7f6250009a80).ready entity=mon.1 client_cookie=e528b47e1224639e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:56.487 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.474 7f625e7fc700 1 -- 172.21.2.108:0/3388732941 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f625000c0c0 con 0x7f62600d17f0 2021-02-10T11:12:56.487 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.474 7f625e7fc700 1 -- 172.21.2.108:0/3388732941 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625000c340 con 0x7f62600d17f0 2021-02-10T11:12:56.487 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.474 7f625e7fc700 1 -- 172.21.2.108:0/3388732941 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f625000c860 con 0x7f62600d17f0 2021-02-10T11:12:56.488 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.474 7f6265d3c700 1 -- 172.21.2.108:0/3388732941 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62600d17f0 msgr2=0x7f62601b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:56.488 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.474 7f6265d3c700 1 --2- 172.21.2.108:0/3388732941 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62600d17f0 0x7f62601b31e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f6250006650 tx=0x7f6250009a80).stop 2021-02-10T11:12:56.488 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.474 7f6265d3c700 1 -- 172.21.2.108:0/3388732941 shutdown_connections 2021-02-10T11:12:56.488 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.474 7f6265d3c700 1 --2- 172.21.2.108:0/3388732941 >> v2:172.21.2.101:3300/0 conn(0x7f62600cff00 0x7f62600ef620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:12:56.488 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.474 7f6265d3c700 1 --2- 172.21.2.108:0/3388732941 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62600d17f0 0x7f62601b31e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:56.488 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.474 7f6265d3c700 1 -- 172.21.2.108:0/3388732941 >> 172.21.2.108:0/3388732941 conn(0x7f62600fde40 msgr2=0x7f62600ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:12:56.488 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.475 7f6265d3c700 1 -- 172.21.2.108:0/3388732941 shutdown_connections 2021-02-10T11:12:56.488 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.475 7f6265d3c700 1 -- 172.21.2.108:0/3388732941 wait complete. 2021-02-10T11:12:56.490 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.489 7f6265d3c700 1 Processor -- start 2021-02-10T11:12:56.490 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.489 7f6265d3c700 1 -- start start 2021-02-10T11:12:56.490 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.490 7f6265d3c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62600ae220 0x7f626008e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:56.490 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.490 7f6265d3c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f626008e900 0x7f626008ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:56.490 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.490 7f6265d3c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f626008f320 0x7f626008c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:56.490 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.490 7f6265d3c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62601b4070 con 0x7f62600ae220 2021-02-10T11:12:56.490 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.490 7f6265d3c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62601b4290 con 0x7f626008e900 2021-02-10T11:12:56.490 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.490 7f6265d3c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626008ca30 con 0x7f626008f320 2021-02-10T11:12:56.490 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.490 7f625effd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f626008e900 0x7f626008ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:56.490 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.490 7f625ffff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f626008f320 0x7f626008c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:56.491 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.490 7f625effd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f626008e900 0x7f626008ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.108:50268/0 (socket says 172.21.2.108:50268) 2021-02-10T11:12:56.491 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.490 7f625effd700 1 -- 172.21.2.108:0/3100420546 learned_addr learned my addr 172.21.2.108:0/3100420546 (peer_addr_for_me v2:172.21.2.108:0/0) 2021-02-10T11:12:56.491 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.490 7f625f7fe700 1 --2- 172.21.2.108:0/3100420546 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62600ae220 0x7f626008e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:56.491 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.491 7f625effd700 1 -- 172.21.2.108:0/3100420546 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f626008f320 msgr2=0x7f626008c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:56.491 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.491 7f625effd700 1 --2- 172.21.2.108:0/3100420546 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f626008f320 0x7f626008c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:56.491 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.491 7f625effd700 1 -- 172.21.2.108:0/3100420546 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62600ae220 msgr2=0x7f626008e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:56.491 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.491 7f625effd700 1 --2- 172.21.2.108:0/3100420546 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62600ae220 0x7f626008e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:56.491 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.491 7f625effd700 1 -- 172.21.2.108:0/3100420546 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626008cc50 con 0x7f626008e900 2021-02-10T11:12:56.491 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.491 7f625f7fe700 1 --2- 172.21.2.108:0/3100420546 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62600ae220 0x7f626008e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:12:56.491 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.491 7f625effd700 1 --2- 172.21.2.108:0/3100420546 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f626008e900 0x7f626008ed00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f6248001c60 tx=0x7f624800a8d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:56.491 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.491 7f6257fff700 1 -- 172.21.2.108:0/3100420546 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f624800acc0 con 0x7f626008e900 2021-02-10T11:12:56.491 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.491 7f6257fff700 1 -- 172.21.2.108:0/3100420546 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6248004140 con 0x7f626008e900 2021-02-10T11:12:56.491 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.491 7f6257fff700 1 -- 172.21.2.108:0/3100420546 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f62480045d0 con 0x7f626008e900 2021-02-10T11:12:56.491 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.491 7f6265d3c700 1 -- 172.21.2.108:0/3100420546 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f626008cc50 con 0x7f626008e900 2021-02-10T11:12:56.492 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.491 7f6265d3c700 1 -- 172.21.2.108:0/3100420546 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f626008cc50 con 0x7f626008e900 2021-02-10T11:12:56.492 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.492 7f6257fff700 1 -- 172.21.2.108:0/3100420546 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6248000ce0 con 0x7f626008e900 2021-02-10T11:12:56.492 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.492 7f6257fff700 1 --2- 172.21.2.108:0/3100420546 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f62440186c0 0x7f624401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:56.492 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.492 7f6257fff700 1 -- 172.21.2.108:0/3100420546 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1166+0+0 (secure 0 0 0) 0x7f62480316a0 con 0x7f626008e900 2021-02-10T11:12:56.492 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.492 7f625f7fe700 1 --2- 172.21.2.108:0/3100420546 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f62440186c0 0x7f624401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:56.492 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.494 7f625f7fe700 1 --2- 172.21.2.108:0/3100420546 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f62440186c0 0x7f624401abc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:56.520 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.522 7f6265d3c700 1 -- 172.21.2.108:0/3100420546 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6260103730 con 0x7f626008e900 2021-02-10T11:12:56.523 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.525 7f6257fff700 1 -- 172.21.2.108:0/3100420546 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f62480009b0 con 0x7f626008e900 2021-02-10T11:12:56.658 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.660 7f6265d3c700 1 -- 172.21.2.108:0/3100420546 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "osd new", "uuid": "36742c21-bfeb-4e11-bce9-8e745378b605", "id": 4} v 0) v1 -- 0x7f62600f0b20 con 0x7f626008e900 2021-02-10T11:12:56.662 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.665 7f6257fff700 1 -- 172.21.2.108:0/3100420546 <== mon.1 v2:172.21.2.108:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "36742c21-bfeb-4e11-bce9-8e745378b605", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6248000ce0 con 0x7f626008e900 2021-02-10T11:12:56.665 INFO:teuthology.orchestra.run.gibba008.stdout:4 2021-02-10T11:12:56.666 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.669 7f6265d3c700 1 -- 172.21.2.108:0/3100420546 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f62440186c0 msgr2=0x7f624401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:56.667 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.669 7f6265d3c700 1 --2- 172.21.2.108:0/3100420546 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f62440186c0 0x7f624401abc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:56.667 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.669 7f6265d3c700 1 -- 172.21.2.108:0/3100420546 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f626008e900 msgr2=0x7f626008ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:56.667 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.669 7f6265d3c700 1 --2- 172.21.2.108:0/3100420546 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f626008e900 0x7f626008ed00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f6248001c60 tx=0x7f624800a8d0).stop 2021-02-10T11:12:56.667 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.669 7f6265d3c700 1 -- 172.21.2.108:0/3100420546 shutdown_connections 2021-02-10T11:12:56.667 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.669 7f6265d3c700 1 --2- 172.21.2.108:0/3100420546 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f62440186c0 0x7f624401abc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:56.667 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.669 7f6265d3c700 1 --2- 172.21.2.108:0/3100420546 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f626008f320 0x7f626008c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:56.667 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.669 7f6265d3c700 1 --2- 172.21.2.108:0/3100420546 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f626008e900 0x7f626008ed00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:56.667 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.669 7f6265d3c700 1 --2- 172.21.2.108:0/3100420546 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62600ae220 0x7f626008e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:56.667 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.669 7f6265d3c700 1 -- 172.21.2.108:0/3100420546 >> 172.21.2.108:0/3100420546 conn(0x7f626019fd30 msgr2=0x7f62601b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:12:56.668 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.669 7f6265d3c700 1 -- 172.21.2.108:0/3100420546 shutdown_connections 2021-02-10T11:12:56.668 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:56.670 7f6265d3c700 1 -- 172.21.2.108:0/3100420546 wait complete. 2021-02-10T11:12:56.675 DEBUG:teuthology.orchestra.run.gibba008:> sudo ceph --cluster ceph osd new 88e694e8-fe98-4d1b-88c8-8c9907ece60b 5 2021-02-10T11:12:57.102 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.088 7f4503bb6700 1 Processor -- start 2021-02-10T11:12:57.102 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.088 7f4503bb6700 1 -- start start 2021-02-10T11:12:57.102 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.088 7f4503bb6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f44fc0d17f0 0x7f44fc1b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:57.102 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.088 7f4503bb6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc0cff00 0x7f44fc0ef620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:57.102 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.088 7f4503bb6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44fc0f0b20 con 0x7f44fc0d1bf0 2021-02-10T11:12:57.103 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.088 7f4503bb6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44fc1b3ff0 con 0x7f44fc0d17f0 2021-02-10T11:12:57.103 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.088 7f4503bb6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc0f20b0 con 0x7f44fc0cff00 2021-02-10T11:12:57.103 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.089 7f45021d3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc0cff00 0x7f44fc0ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:57.103 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.089 7f45021d3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc0cff00 0x7f44fc0ef620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.108:44350/0 (socket says 172.21.2.108:44350) 2021-02-10T11:12:57.103 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.089 7f45021d3700 1 -- 172.21.2.108:0/3675344724 learned_addr learned my addr 172.21.2.108:0/3675344724 (peer_addr_for_me v2:172.21.2.108:0/0) 2021-02-10T11:12:57.103 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.089 7f45021d3700 1 -- 172.21.2.108:0/3675344724 >> v2:172.21.2.101:3300/0 conn(0x7f44fc0d17f0 msgr2=0x7f44fc1b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:12:57.103 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.089 7f45021d3700 1 --2- 172.21.2.108:0/3675344724 >> v2:172.21.2.101:3300/0 conn(0x7f44fc0d17f0 0x7f44fc1b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:12:57.103 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.089 7f45021d3700 1 -- 172.21.2.108:0/3675344724 >> v1:172.21.2.108:6789/0 conn(0x7f44fc0d1bf0 legacy=0x7f44fc1b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:57.103 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.089 7f45021d3700 1 -- 172.21.2.108:0/3675344724 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc0f22d0 con 0x7f44fc0cff00 2021-02-10T11:12:57.103 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.089 7f45021d3700 1 --2- 172.21.2.108:0/3675344724 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc0cff00 0x7f44fc0ef620 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f44f4006f00 tx=0x7f44f4009f10).ready entity=mon.2 client_cookie=38099fb11f8078f1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:57.103 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.089 7f45009d0700 1 -- 172.21.2.108:0/3675344724 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f44f400b3b0 con 0x7f44fc0cff00 2021-02-10T11:12:57.103 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.089 7f45009d0700 1 -- 172.21.2.108:0/3675344724 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f400b730 con 0x7f44fc0cff00 2021-02-10T11:12:57.103 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.089 7f45009d0700 1 -- 172.21.2.108:0/3675344724 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f44f400bc10 con 0x7f44fc0cff00 2021-02-10T11:12:57.104 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.089 7f4503bb6700 1 -- 172.21.2.108:0/3675344724 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc0cff00 msgr2=0x7f44fc0ef620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:57.104 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.089 7f4503bb6700 1 --2- 172.21.2.108:0/3675344724 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc0cff00 0x7f44fc0ef620 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f44f4006f00 tx=0x7f44f4009f10).stop 2021-02-10T11:12:57.104 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.090 7f4503bb6700 1 -- 172.21.2.108:0/3675344724 shutdown_connections 2021-02-10T11:12:57.104 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.090 7f4503bb6700 1 --2- 172.21.2.108:0/3675344724 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc0cff00 0x7f44fc0ef620 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:57.104 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.090 7f4503bb6700 1 --2- 172.21.2.108:0/3675344724 >> v2:172.21.2.101:3300/0 conn(0x7f44fc0d17f0 0x7f44fc1b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:12:57.104 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.090 7f4503bb6700 1 -- 172.21.2.108:0/3675344724 >> 172.21.2.108:0/3675344724 conn(0x7f44fc0fde40 msgr2=0x7f44fc0ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:12:57.104 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.090 7f4503bb6700 1 -- 172.21.2.108:0/3675344724 shutdown_connections 2021-02-10T11:12:57.104 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.090 7f4503bb6700 1 -- 172.21.2.108:0/3675344724 wait complete. 2021-02-10T11:12:57.105 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.105 7f4503bb6700 1 Processor -- start 2021-02-10T11:12:57.106 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.105 7f4503bb6700 1 -- start start 2021-02-10T11:12:57.106 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.105 7f4503bb6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc09a370 0x7f44fc096570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:57.106 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.105 7f4503bb6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44fc096b90 0x7f44fc097640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:57.106 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.105 7f4503bb6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44fc1b4050 0x7f44fc097c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:57.106 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.105 7f4503bb6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc0970c0 con 0x7f44fc096b90 2021-02-10T11:12:57.106 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.105 7f4503bb6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc09a770 con 0x7f44fc1b4050 2021-02-10T11:12:57.106 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.105 7f4503bb6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc1b1bb0 con 0x7f44fc09a370 2021-02-10T11:12:57.106 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.105 7f45021d3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44fc1b4050 0x7f44fc097c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:57.106 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.106 7f45019d2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc09a370 0x7f44fc096570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:57.106 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.106 7f45021d3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44fc1b4050 0x7f44fc097c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.108:50280/0 (socket says 172.21.2.108:50280) 2021-02-10T11:12:57.106 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.106 7f45021d3700 1 -- 172.21.2.108:0/1765668652 learned_addr learned my addr 172.21.2.108:0/1765668652 (peer_addr_for_me v2:172.21.2.108:0/0) 2021-02-10T11:12:57.106 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.106 7f45019d2700 1 -- 172.21.2.108:0/1765668652 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44fc1b4050 msgr2=0x7f44fc097c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:57.107 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.106 7f45019d2700 1 --2- 172.21.2.108:0/1765668652 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44fc1b4050 0x7f44fc097c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:57.107 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.106 7f45019d2700 1 -- 172.21.2.108:0/1765668652 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44fc096b90 msgr2=0x7f44fc097640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:57.107 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.106 7f45019d2700 1 --2- 172.21.2.108:0/1765668652 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44fc096b90 0x7f44fc097640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:12:57.107 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.106 7f45019d2700 1 -- 172.21.2.108:0/1765668652 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc098280 con 0x7f44fc09a370 2021-02-10T11:12:57.107 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.106 7f45019d2700 1 --2- 172.21.2.108:0/1765668652 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc09a370 0x7f44fc096570 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f44f800b510 tx=0x7f44f800b8b0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:57.107 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.106 7f44f27fc700 1 -- 172.21.2.108:0/1765668652 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f44f8006350 con 0x7f44fc09a370 2021-02-10T11:12:57.107 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.106 7f44f27fc700 1 -- 172.21.2.108:0/1765668652 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f800bca0 con 0x7f44fc09a370 2021-02-10T11:12:57.107 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.106 7f44f27fc700 1 -- 172.21.2.108:0/1765668652 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f44f80022b0 con 0x7f44fc09a370 2021-02-10T11:12:57.107 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.106 7f4503bb6700 1 -- 172.21.2.108:0/1765668652 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44fc098280 con 0x7f44fc09a370 2021-02-10T11:12:57.107 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.107 7f4503bb6700 1 -- 172.21.2.108:0/1765668652 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44fc098280 con 0x7f44fc09a370 2021-02-10T11:12:57.107 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.107 7f44f27fc700 1 -- 172.21.2.108:0/1765668652 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f44f8000c90 con 0x7f44fc09a370 2021-02-10T11:12:57.107 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.107 7f44f27fc700 1 --2- 172.21.2.108:0/1765668652 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44e40186c0 0x7f44e401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:57.107 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.107 7f44f27fc700 1 -- 172.21.2.108:0/1765668652 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1269+0+0 (secure 0 0 0) 0x7f44f80314e0 con 0x7f44fc09a370 2021-02-10T11:12:57.107 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.108 7f45011d1700 1 --2- 172.21.2.108:0/1765668652 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44e40186c0 0x7f44e401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:57.108 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.108 7f45011d1700 1 --2- 172.21.2.108:0/1765668652 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44e40186c0 0x7f44e401abc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:57.136 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.138 7f4503bb6700 1 -- 172.21.2.108:0/1765668652 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44fc0f0b20 con 0x7f44fc09a370 2021-02-10T11:12:57.138 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.141 7f44f27fc700 1 -- 172.21.2.108:0/1765668652 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f44f8006350 con 0x7f44fc09a370 2021-02-10T11:12:57.275 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.277 7f4503bb6700 1 -- 172.21.2.108:0/1765668652 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "osd new", "uuid": "88e694e8-fe98-4d1b-88c8-8c9907ece60b", "id": 5} v 0) v1 -- 0x7f44fc0f0b20 con 0x7f44fc09a370 2021-02-10T11:12:57.278 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.281 7f44f27fc700 1 -- 172.21.2.108:0/1765668652 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "88e694e8-fe98-4d1b-88c8-8c9907ece60b", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f44f8000c90 con 0x7f44fc09a370 2021-02-10T11:12:57.282 INFO:teuthology.orchestra.run.gibba008.stdout:5 2021-02-10T11:12:57.284 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.286 7f4503bb6700 1 -- 172.21.2.108:0/1765668652 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44e40186c0 msgr2=0x7f44e401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:57.284 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.286 7f4503bb6700 1 --2- 172.21.2.108:0/1765668652 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44e40186c0 0x7f44e401abc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:57.284 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.286 7f4503bb6700 1 -- 172.21.2.108:0/1765668652 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc09a370 msgr2=0x7f44fc096570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:57.284 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.286 7f4503bb6700 1 --2- 172.21.2.108:0/1765668652 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc09a370 0x7f44fc096570 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f44f800b510 tx=0x7f44f800b8b0).stop 2021-02-10T11:12:57.284 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.286 7f4503bb6700 1 -- 172.21.2.108:0/1765668652 shutdown_connections 2021-02-10T11:12:57.284 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.286 7f4503bb6700 1 --2- 172.21.2.108:0/1765668652 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44fc1b4050 0x7f44fc097c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:57.284 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.286 7f4503bb6700 1 --2- 172.21.2.108:0/1765668652 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44fc096b90 0x7f44fc097640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:12:57.284 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.286 7f4503bb6700 1 --2- 172.21.2.108:0/1765668652 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44e40186c0 0x7f44e401abc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:57.284 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.286 7f4503bb6700 1 --2- 172.21.2.108:0/1765668652 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc09a370 0x7f44fc096570 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:57.285 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.286 7f4503bb6700 1 -- 172.21.2.108:0/1765668652 >> 172.21.2.108:0/1765668652 conn(0x7f44fc105300 msgr2=0x7f44fc18e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:12:57.285 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.287 7f4503bb6700 1 -- 172.21.2.108:0/1765668652 shutdown_connections 2021-02-10T11:12:57.285 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.287 7f4503bb6700 1 -- 172.21.2.108:0/1765668652 wait complete. 2021-02-10T11:12:57.291 DEBUG:teuthology.orchestra.run.gibba008:> sudo ceph --cluster ceph osd new 87a13c6c-cf85-4aaf-916b-e73913484a49 6 2021-02-10T11:12:57.718 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.704 7f216846f700 1 Processor -- start 2021-02-10T11:12:57.719 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.704 7f216846f700 1 -- start start 2021-02-10T11:12:57.719 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.705 7f216846f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21600f0630 0x7f21600f10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:57.719 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.705 7f216846f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21600f0b20 con 0x7f21600d17f0 2021-02-10T11:12:57.719 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.705 7f216846f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21600d1bf0 con 0x7f21600cff00 2021-02-10T11:12:57.719 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.705 7f216846f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21600e5eb0 con 0x7f21600f0630 2021-02-10T11:12:57.719 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.705 7f2165a8a700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f21600cff00 0x7f21601b39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.108:33384/0 (socket says 172.21.2.108:33384) 2021-02-10T11:12:57.719 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.705 7f2165a8a700 1 -- 172.21.2.108:0/191784743 learned_addr learned my addr 172.21.2.108:0/191784743 (peer_addr_for_me v1:172.21.2.108:0/0) 2021-02-10T11:12:57.719 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.705 7f2166a8c700 1 --2- 172.21.2.108:0/191784743 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21600f0630 0x7f21600f10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:57.719 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.705 7f2166a8c700 1 -- 172.21.2.108:0/191784743 >> v1:172.21.2.108:6789/0 conn(0x7f21600cff00 legacy=0x7f21601b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:57.719 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.705 7f2166a8c700 1 -- 172.21.2.108:0/191784743 >> v1:172.21.2.101:6789/0 conn(0x7f21600d17f0 legacy=0x7f21601b31e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:12:57.720 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.705 7f2166a8c700 1 -- 172.21.2.108:0/191784743 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21600d1e40 con 0x7f21600f0630 2021-02-10T11:12:57.720 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.706 7f2166a8c700 1 --2- 172.21.2.108:0/191784743 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21600f0630 0x7f21600f10a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f2158006e30 tx=0x7f2158009de0).ready entity=mon.2 client_cookie=159a60c8be272cea server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:57.720 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.706 7f2165289700 1 -- 172.21.2.108:0/191784743 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f215800b300 con 0x7f21600f0630 2021-02-10T11:12:57.720 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.706 7f2165289700 1 -- 172.21.2.108:0/191784743 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215800b680 con 0x7f21600f0630 2021-02-10T11:12:57.720 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.706 7f2165289700 1 -- 172.21.2.108:0/191784743 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f215800bba0 con 0x7f21600f0630 2021-02-10T11:12:57.720 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.706 7f216846f700 1 -- 172.21.2.108:0/191784743 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21600f0630 msgr2=0x7f21600f10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:57.720 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.706 7f216846f700 1 --2- 172.21.2.108:0/191784743 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21600f0630 0x7f21600f10a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f2158006e30 tx=0x7f2158009de0).stop 2021-02-10T11:12:57.720 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.707 7f216846f700 1 -- 172.21.2.108:0/191784743 shutdown_connections 2021-02-10T11:12:57.720 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.707 7f216846f700 1 --2- 172.21.2.108:0/191784743 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21600f0630 0x7f21600f10a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:57.720 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.707 7f216846f700 1 -- 172.21.2.108:0/191784743 >> 172.21.2.108:0/191784743 conn(0x7f21600fde40 msgr2=0x7f21600ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:12:57.720 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.707 7f216846f700 1 -- 172.21.2.108:0/191784743 shutdown_connections 2021-02-10T11:12:57.720 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.707 7f216846f700 1 -- 172.21.2.108:0/191784743 wait complete. 2021-02-10T11:12:57.720 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.721 7f216846f700 1 Processor -- start 2021-02-10T11:12:57.720 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.721 7f216846f700 1 -- start start 2021-02-10T11:12:57.721 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.722 7f216846f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2160091360 0x7f21600bfb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:57.721 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.722 7f216846f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21600c0140 0x7f21600c0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:57.721 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.722 7f216846f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21601b1190 0x7f21601b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:57.721 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.722 7f216846f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21601b2860 con 0x7f21601b1190 2021-02-10T11:12:57.721 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.722 7f216846f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21600c0b60 con 0x7f2160091360 2021-02-10T11:12:57.722 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.722 7f216846f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21600c0d40 con 0x7f21600c0140 2021-02-10T11:12:57.722 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.722 7f216628b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2160091360 0x7f21600bfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:57.722 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.722 7f2165a8a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21600c0140 0x7f21600c0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:57.725 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.722 7f216628b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2160091360 0x7f21600bfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.108:50294/0 (socket says 172.21.2.108:50294) 2021-02-10T11:12:57.725 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.722 7f2165a8a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21600c0140 0x7f21600c0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.108:44372/0 (socket says 172.21.2.108:44372) 2021-02-10T11:12:57.725 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.722 7f216628b700 1 -- 172.21.2.108:0/2036098252 learned_addr learned my addr 172.21.2.108:0/2036098252 (peer_addr_for_me v2:172.21.2.108:0/0) 2021-02-10T11:12:57.725 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.722 7f2165a8a700 1 -- 172.21.2.108:0/2036098252 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2160091360 msgr2=0x7f21600bfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:57.725 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.722 7f2165a8a700 1 --2- 172.21.2.108:0/2036098252 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2160091360 0x7f21600bfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:57.725 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.722 7f2165a8a700 1 -- 172.21.2.108:0/2036098252 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21601b1190 msgr2=0x7f21601b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:57.725 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.722 7f2165a8a700 1 --2- 172.21.2.108:0/2036098252 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21601b1190 0x7f21601b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:12:57.725 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.722 7f2165a8a700 1 -- 172.21.2.108:0/2036098252 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21601b1bb0 con 0x7f21600c0140 2021-02-10T11:12:57.725 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.722 7f216628b700 1 --2- 172.21.2.108:0/2036098252 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2160091360 0x7f21600bfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:12:57.726 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.722 7f2165a8a700 1 --2- 172.21.2.108:0/2036098252 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21600c0140 0x7f21600c0540 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f215c001e30 tx=0x7f215c00a8a0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:57.726 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.723 7f2152ffd700 1 -- 172.21.2.108:0/2036098252 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f215c0070d0 con 0x7f21600c0140 2021-02-10T11:12:57.726 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.723 7f2152ffd700 1 -- 172.21.2.108:0/2036098252 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215c00ac90 con 0x7f21600c0140 2021-02-10T11:12:57.726 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.723 7f216846f700 1 -- 172.21.2.108:0/2036098252 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21601b1bb0 con 0x7f21600c0140 2021-02-10T11:12:57.726 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.723 7f2152ffd700 1 -- 172.21.2.108:0/2036098252 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f215c004620 con 0x7f21600c0140 2021-02-10T11:12:57.726 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.723 7f216846f700 1 -- 172.21.2.108:0/2036098252 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21601b1bb0 con 0x7f21600c0140 2021-02-10T11:12:57.726 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.723 7f2152ffd700 1 -- 172.21.2.108:0/2036098252 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f215c000f90 con 0x7f21600c0140 2021-02-10T11:12:57.726 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.723 7f2152ffd700 1 --2- 172.21.2.108:0/2036098252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2148018730 0x7f214801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:57.727 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.723 7f2152ffd700 1 -- 172.21.2.108:0/2036098252 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1372+0+0 (secure 0 0 0) 0x7f215c031670 con 0x7f21600c0140 2021-02-10T11:12:57.727 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.724 7f216628b700 1 --2- 172.21.2.108:0/2036098252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2148018730 0x7f214801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:57.727 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.724 7f216628b700 1 --2- 172.21.2.108:0/2036098252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2148018730 0x7f214801ac10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:57.752 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.755 7f216846f700 1 -- 172.21.2.108:0/2036098252 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f216010a010 con 0x7f21600c0140 2021-02-10T11:12:57.758 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.757 7f2152ffd700 1 -- 172.21.2.108:0/2036098252 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f215c000940 con 0x7f21600c0140 2021-02-10T11:12:57.891 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.893 7f216846f700 1 -- 172.21.2.108:0/2036098252 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "osd new", "uuid": "87a13c6c-cf85-4aaf-916b-e73913484a49", "id": 6} v 0) v1 -- 0x7f21600e60a0 con 0x7f21600c0140 2021-02-10T11:12:57.896 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.898 7f2152ffd700 1 -- 172.21.2.108:0/2036098252 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "87a13c6c-cf85-4aaf-916b-e73913484a49", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f215c000f90 con 0x7f21600c0140 2021-02-10T11:12:57.898 INFO:teuthology.orchestra.run.gibba008.stdout:6 2021-02-10T11:12:57.900 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.902 7f216846f700 1 -- 172.21.2.108:0/2036098252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2148018730 msgr2=0x7f214801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:57.900 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.902 7f216846f700 1 --2- 172.21.2.108:0/2036098252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2148018730 0x7f214801ac10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:57.901 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.902 7f216846f700 1 -- 172.21.2.108:0/2036098252 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21600c0140 msgr2=0x7f21600c0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:57.901 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.902 7f216846f700 1 --2- 172.21.2.108:0/2036098252 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21600c0140 0x7f21600c0540 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f215c001e30 tx=0x7f215c00a8a0).stop 2021-02-10T11:12:57.901 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.902 7f216846f700 1 -- 172.21.2.108:0/2036098252 shutdown_connections 2021-02-10T11:12:57.901 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.902 7f216846f700 1 --2- 172.21.2.108:0/2036098252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2148018730 0x7f214801ac10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:57.901 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.902 7f216846f700 1 --2- 172.21.2.108:0/2036098252 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21600c0140 0x7f21600c0540 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:57.901 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.902 7f216846f700 1 --2- 172.21.2.108:0/2036098252 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21601b1190 0x7f21601b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:12:57.901 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.902 7f216846f700 1 --2- 172.21.2.108:0/2036098252 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2160091360 0x7f21600bfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:57.902 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.902 7f216846f700 1 -- 172.21.2.108:0/2036098252 >> 172.21.2.108:0/2036098252 conn(0x7f21600ae1b0 msgr2=0x7f216010d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:12:57.902 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.903 7f216846f700 1 -- 172.21.2.108:0/2036098252 shutdown_connections 2021-02-10T11:12:57.902 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:57.903 7f216846f700 1 -- 172.21.2.108:0/2036098252 wait complete. 2021-02-10T11:12:57.909 DEBUG:teuthology.orchestra.run.gibba008:> sudo ceph --cluster ceph osd new 7b90823f-854c-4592-a95b-a7fa98f93254 7 2021-02-10T11:12:58.336 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.322 7f2efa1cf700 1 Processor -- start 2021-02-10T11:12:58.336 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.322 7f2efa1cf700 1 -- start start 2021-02-10T11:12:58.336 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.322 7f2efa1cf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2ef40f0630 0x7f2ef40f10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:58.336 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.322 7f2efa1cf700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef40f0b20 con 0x7f2ef40d17f0 2021-02-10T11:12:58.336 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.322 7f2efa1cf700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef40d1bf0 con 0x7f2ef40cff00 2021-02-10T11:12:58.336 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.322 7f2efa1cf700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef40e5eb0 con 0x7f2ef40f0630 2021-02-10T11:12:58.337 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.322 7f2ef3fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2ef40f0630 0x7f2ef40f10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:58.337 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.322 7f2ef2ffd700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f2ef40cff00 0x7f2ef41b39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.108:33398/0 (socket says 172.21.2.108:33398) 2021-02-10T11:12:58.337 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.322 7f2ef2ffd700 1 -- 172.21.2.108:0/1733772250 learned_addr learned my addr 172.21.2.108:0/1733772250 (peer_addr_for_me v1:172.21.2.108:0/0) 2021-02-10T11:12:58.337 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.323 7f2ef27fc700 1 -- 172.21.2.108:0/1733772250 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3738630841 0 0) 0x7f2edc0010b0 con 0x7f2ef40cff00 2021-02-10T11:12:58.337 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.323 7f2ef3fff700 1 -- 172.21.2.108:0/1733772250 >> v1:172.21.2.108:6789/0 conn(0x7f2ef40cff00 legacy=0x7f2ef41b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:58.337 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.323 7f2ef3fff700 1 -- 172.21.2.108:0/1733772250 >> v1:172.21.2.101:6789/0 conn(0x7f2ef40d17f0 legacy=0x7f2ef41b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:58.337 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.323 7f2ef3fff700 1 -- 172.21.2.108:0/1733772250 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef40d1e10 con 0x7f2ef40f0630 2021-02-10T11:12:58.337 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.323 7f2ef3fff700 1 --2- 172.21.2.108:0/1733772250 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ef40f0630 0x7f2ef40f10a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f2eec006e50 tx=0x7f2eec009e70).ready entity=mon.0 client_cookie=c64aa9cf867da05e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:58.337 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.323 7f2ef27fc700 1 -- 172.21.2.108:0/1733772250 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2eec00b510 con 0x7f2ef40f0630 2021-02-10T11:12:58.337 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.323 7f2ef27fc700 1 -- 172.21.2.108:0/1733772250 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eec006790 con 0x7f2ef40f0630 2021-02-10T11:12:58.337 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.323 7f2ef27fc700 1 -- 172.21.2.108:0/1733772250 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2eec00b940 con 0x7f2ef40f0630 2021-02-10T11:12:58.337 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.323 7f2efa1cf700 1 -- 172.21.2.108:0/1733772250 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ef40f0630 msgr2=0x7f2ef40f10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:58.337 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.323 7f2efa1cf700 1 --2- 172.21.2.108:0/1733772250 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ef40f0630 0x7f2ef40f10a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f2eec006e50 tx=0x7f2eec009e70).stop 2021-02-10T11:12:58.338 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.324 7f2efa1cf700 1 -- 172.21.2.108:0/1733772250 shutdown_connections 2021-02-10T11:12:58.338 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.324 7f2efa1cf700 1 --2- 172.21.2.108:0/1733772250 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ef40f0630 0x7f2ef40f10a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:58.338 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.324 7f2efa1cf700 1 -- 172.21.2.108:0/1733772250 >> 172.21.2.108:0/1733772250 conn(0x7f2ef40fde40 msgr2=0x7f2ef40ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:12:58.338 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.324 7f2efa1cf700 1 -- 172.21.2.108:0/1733772250 shutdown_connections 2021-02-10T11:12:58.338 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.324 7f2efa1cf700 1 -- 172.21.2.108:0/1733772250 wait complete. 2021-02-10T11:12:58.339 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.339 7f2efa1cf700 1 Processor -- start 2021-02-10T11:12:58.339 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.339 7f2efa1cf700 1 -- start start 2021-02-10T11:12:58.339 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.339 7f2efa1cf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ef41ae510 0x7f2ef4097900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:58.339 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.339 7f2efa1cf700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ef40a57e0 0x7f2ef4097f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:58.339 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.339 7f2efa1cf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ef408ce00 0x7f2ef40b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:58.339 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.339 7f2efa1cf700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef408c880 con 0x7f2ef408ce00 2021-02-10T11:12:58.339 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.339 7f2efa1cf700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef4098540 con 0x7f2ef40a57e0 2021-02-10T11:12:58.339 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.339 7f2efa1cf700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef40b3470 con 0x7f2ef41ae510 2021-02-10T11:12:58.339 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.340 7f2ef2ffd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ef40a57e0 0x7f2ef4097f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:58.339 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.340 7f2ef37fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ef41ae510 0x7f2ef4097900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:58.340 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.340 7f2ef2ffd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ef40a57e0 0x7f2ef4097f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.108:50306/0 (socket says 172.21.2.108:50306) 2021-02-10T11:12:58.340 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.340 7f2ef2ffd700 1 -- 172.21.2.108:0/1613165040 learned_addr learned my addr 172.21.2.108:0/1613165040 (peer_addr_for_me v2:172.21.2.108:0/0) 2021-02-10T11:12:58.340 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.340 7f2ef37fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ef41ae510 0x7f2ef4097900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.108:44382/0 (socket says 172.21.2.108:44382) 2021-02-10T11:12:58.340 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.340 7f2ef3fff700 1 --2- 172.21.2.108:0/1613165040 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ef408ce00 0x7f2ef40b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:58.341 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.340 7f2ef2ffd700 1 -- 172.21.2.108:0/1613165040 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ef41ae510 msgr2=0x7f2ef4097900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:58.341 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.340 7f2ef2ffd700 1 --2- 172.21.2.108:0/1613165040 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ef41ae510 0x7f2ef4097900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:58.341 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.340 7f2ef2ffd700 1 -- 172.21.2.108:0/1613165040 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ef408ce00 msgr2=0x7f2ef40b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:58.341 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.340 7f2ef2ffd700 1 --2- 172.21.2.108:0/1613165040 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ef408ce00 0x7f2ef40b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:58.341 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.340 7f2ef2ffd700 1 -- 172.21.2.108:0/1613165040 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef40b3690 con 0x7f2ef40a57e0 2021-02-10T11:12:58.342 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.340 7f2ef3fff700 1 --2- 172.21.2.108:0/1613165040 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ef408ce00 0x7f2ef40b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:12:58.342 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.340 7f2ef37fe700 1 --2- 172.21.2.108:0/1613165040 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ef41ae510 0x7f2ef4097900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:12:58.342 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.340 7f2ef2ffd700 1 --2- 172.21.2.108:0/1613165040 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ef40a57e0 0x7f2ef4097f20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f2edc001c10 tx=0x7f2edc00a8f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:58.342 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.340 7f2eebfff700 1 -- 172.21.2.108:0/1613165040 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2edc0009b0 con 0x7f2ef40a57e0 2021-02-10T11:12:58.342 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.340 7f2eebfff700 1 -- 172.21.2.108:0/1613165040 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2edc0012c0 con 0x7f2ef40a57e0 2021-02-10T11:12:58.342 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.340 7f2eebfff700 1 -- 172.21.2.108:0/1613165040 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2edc004350 con 0x7f2ef40a57e0 2021-02-10T11:12:58.342 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.340 7f2efa1cf700 1 -- 172.21.2.108:0/1613165040 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef40b3690 con 0x7f2ef40a57e0 2021-02-10T11:12:58.342 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.341 7f2efa1cf700 1 -- 172.21.2.108:0/1613165040 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef40b3690 con 0x7f2ef40a57e0 2021-02-10T11:12:58.342 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.341 7f2eebfff700 1 -- 172.21.2.108:0/1613165040 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2edc007fd0 con 0x7f2ef40a57e0 2021-02-10T11:12:58.342 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.341 7f2eebfff700 1 --2- 172.21.2.108:0/1613165040 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2ed80186c0 0x7f2ed801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:58.342 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.341 7f2eebfff700 1 -- 172.21.2.108:0/1613165040 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1475+0+0 (secure 0 0 0) 0x7f2edc0313a0 con 0x7f2ef40a57e0 2021-02-10T11:12:58.342 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.342 7f2ef37fe700 1 --2- 172.21.2.108:0/1613165040 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2ed80186c0 0x7f2ed801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:58.342 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.342 7f2ef37fe700 1 --2- 172.21.2.108:0/1613165040 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2ed80186c0 0x7f2ed801abc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:58.369 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.372 7f2efa1cf700 1 -- 172.21.2.108:0/1613165040 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef408d200 con 0x7f2ef40a57e0 2021-02-10T11:12:58.372 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.374 7f2eebfff700 1 -- 172.21.2.108:0/1613165040 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2edc000dd0 con 0x7f2ef40a57e0 2021-02-10T11:12:58.510 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.512 7f2efa1cf700 1 -- 172.21.2.108:0/1613165040 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7b90823f-854c-4592-a95b-a7fa98f93254", "id": 7} v 0) v1 -- 0x7f2ef408d200 con 0x7f2ef40a57e0 2021-02-10T11:12:58.514 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.517 7f2eebfff700 1 -- 172.21.2.108:0/1613165040 <== mon.1 v2:172.21.2.108:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7b90823f-854c-4592-a95b-a7fa98f93254", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2edc01f4b0 con 0x7f2ef40a57e0 2021-02-10T11:12:58.517 INFO:teuthology.orchestra.run.gibba008.stdout:7 2021-02-10T11:12:58.518 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.521 7f2efa1cf700 1 -- 172.21.2.108:0/1613165040 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2ed80186c0 msgr2=0x7f2ed801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:58.518 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.521 7f2efa1cf700 1 --2- 172.21.2.108:0/1613165040 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2ed80186c0 0x7f2ed801abc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:58.518 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.521 7f2efa1cf700 1 -- 172.21.2.108:0/1613165040 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ef40a57e0 msgr2=0x7f2ef4097f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:58.519 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.521 7f2efa1cf700 1 --2- 172.21.2.108:0/1613165040 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ef40a57e0 0x7f2ef4097f20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f2edc001c10 tx=0x7f2edc00a8f0).stop 2021-02-10T11:12:58.519 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.521 7f2efa1cf700 1 -- 172.21.2.108:0/1613165040 shutdown_connections 2021-02-10T11:12:58.519 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.521 7f2efa1cf700 1 --2- 172.21.2.108:0/1613165040 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ef408ce00 0x7f2ef40b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:58.519 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.521 7f2efa1cf700 1 --2- 172.21.2.108:0/1613165040 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ef40a57e0 0x7f2ef4097f20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:58.519 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.521 7f2efa1cf700 1 --2- 172.21.2.108:0/1613165040 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2ed80186c0 0x7f2ed801abc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:58.519 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.521 7f2efa1cf700 1 --2- 172.21.2.108:0/1613165040 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ef41ae510 0x7f2ef4097900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:58.520 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.521 7f2efa1cf700 1 -- 172.21.2.108:0/1613165040 >> 172.21.2.108:0/1613165040 conn(0x7f2ef40974c0 msgr2=0x7f2ef41aeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:12:58.520 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.521 7f2efa1cf700 1 -- 172.21.2.108:0/1613165040 shutdown_connections 2021-02-10T11:12:58.520 INFO:teuthology.orchestra.run.gibba008.stderr:2021-02-10 11:12:58.521 7f2efa1cf700 1 -- 172.21.2.108:0/1613165040 wait complete. 2021-02-10T11:12:58.527 INFO:tasks.ceph.osd.0:Restarting daemon 2021-02-10T11:12:58.527 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2021-02-10T11:12:58.538 INFO:tasks.ceph.osd.0:Started 2021-02-10T11:12:58.538 INFO:tasks.ceph.osd.1:Restarting daemon 2021-02-10T11:12:58.538 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2021-02-10T11:12:58.549 INFO:tasks.ceph.osd.1:Started 2021-02-10T11:12:58.549 INFO:tasks.ceph.osd.2:Restarting daemon 2021-02-10T11:12:58.550 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2021-02-10T11:12:58.557 INFO:tasks.ceph.osd.2:Started 2021-02-10T11:12:58.557 INFO:tasks.ceph.osd.3:Restarting daemon 2021-02-10T11:12:58.557 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2021-02-10T11:12:58.570 INFO:tasks.ceph.osd.3:Started 2021-02-10T11:12:58.571 INFO:tasks.ceph.osd.4:Restarting daemon 2021-02-10T11:12:58.571 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2021-02-10T11:12:58.587 INFO:tasks.ceph.osd.4:Started 2021-02-10T11:12:58.587 INFO:tasks.ceph.osd.5:Restarting daemon 2021-02-10T11:12:58.587 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2021-02-10T11:12:58.603 INFO:tasks.ceph.osd.5:Started 2021-02-10T11:12:58.603 INFO:tasks.ceph.osd.6:Restarting daemon 2021-02-10T11:12:58.603 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2021-02-10T11:12:58.619 INFO:tasks.ceph.osd.6:Started 2021-02-10T11:12:58.619 INFO:tasks.ceph.osd.7:Restarting daemon 2021-02-10T11:12:58.620 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2021-02-10T11:12:58.634 INFO:tasks.ceph.osd.7:Started 2021-02-10T11:12:58.635 INFO:tasks.ceph:Waiting for OSDs to come up 2021-02-10T11:12:59.442 DEBUG:teuthology.orchestra.run.gibba001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2021-02-10T11:12:59.460 INFO:tasks.ceph.osd.0.gibba001.stderr:2021-02-10 11:12:59.455 7f1387b44a80 -1 Falling back to public interface 2021-02-10T11:12:59.460 INFO:tasks.ceph.osd.1.gibba001.stderr:2021-02-10 11:12:59.455 7f3b2a814a80 -1 Falling back to public interface 2021-02-10T11:12:59.460 INFO:tasks.ceph.osd.1.gibba001.stderr:2021-02-10 11:12:59.455 7f3b23937700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6802/0: (98) Address already in use 2021-02-10T11:12:59.460 INFO:tasks.ceph.osd.2.gibba001.stderr:2021-02-10 11:12:59.456 7f8a1f5f1a80 -1 Falling back to public interface 2021-02-10T11:12:59.462 INFO:tasks.ceph.osd.3.gibba001.stderr:2021-02-10 11:12:59.458 7f32488a7a80 -1 Falling back to public interface 2021-02-10T11:12:59.505 INFO:tasks.ceph.osd.6.gibba008.stderr:2021-02-10 11:12:59.508 7f5a2c6f3a80 -1 Falling back to public interface 2021-02-10T11:12:59.509 INFO:tasks.ceph.osd.5.gibba008.stderr:2021-02-10 11:12:59.511 7f8f87b10a80 -1 Falling back to public interface 2021-02-10T11:12:59.518 INFO:tasks.ceph.osd.4.gibba008.stderr:2021-02-10 11:12:59.520 7fac8cfbfa80 -1 Falling back to public interface 2021-02-10T11:12:59.518 INFO:tasks.ceph.osd.7.gibba008.stderr:2021-02-10 11:12:59.520 7f5889f23a80 -1 Falling back to public interface 2021-02-10T11:12:59.863 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.842 7f53ad98d700 1 Processor -- start 2021-02-10T11:12:59.864 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.842 7f53ad98d700 1 -- start start 2021-02-10T11:12:59.864 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53ad98d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f53a807e6a0 0x7f53a80d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:59.864 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53ad98d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a807ed80 con 0x7f53a80e5470 2021-02-10T11:12:59.864 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53ad98d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a807eaa0 con 0x7f53a80e5070 2021-02-10T11:12:59.864 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53ad98d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53a81b4080 con 0x7f53a807e6a0 2021-02-10T11:12:59.864 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a6d73700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f53a80e5470 0x7f53a807f300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44012/0 (socket says 172.21.2.101:44012) 2021-02-10T11:12:59.864 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a6d73700 1 -- 172.21.2.101:0/1786276060 learned_addr learned my addr 172.21.2.101:0/1786276060 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:12:59.864 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a7d75700 1 --2- 172.21.2.101:0/1786276060 >> v2:172.21.2.108:3300/0 conn(0x7f53a807e6a0 0x7f53a80d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:59.864 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a6572700 1 -- 172.21.2.101:0/1786276060 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2230174124 0 0) 0x7f5398001180 con 0x7f53a80e5470 2021-02-10T11:12:59.864 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a6572700 1 -- 172.21.2.101:0/1786276060 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5394002fe0 con 0x7f53a80e5470 2021-02-10T11:12:59.865 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a6572700 1 -- 172.21.2.101:0/1786276060 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 770508507 0 0) 0x7f5398001180 con 0x7f53a80e5470 2021-02-10T11:12:59.865 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a6572700 1 -- 172.21.2.101:0/1786276060 >> v2:172.21.2.108:3300/0 conn(0x7f53a807e6a0 msgr2=0x7f53a80d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:12:59.865 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a6572700 1 --2- 172.21.2.101:0/1786276060 >> v2:172.21.2.108:3300/0 conn(0x7f53a807e6a0 0x7f53a80d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:59.865 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a6572700 1 -- 172.21.2.101:0/1786276060 >> v1:172.21.2.108:6789/0 conn(0x7f53a80e5070 legacy=0x7f53a81b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:59.865 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a6572700 1 -- 172.21.2.101:0/1786276060 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a80d1db0 con 0x7f53a80e5470 2021-02-10T11:12:59.865 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a6572700 1 -- 172.21.2.101:0/1786276060 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f5398001680 con 0x7f53a80e5470 2021-02-10T11:12:59.865 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a6572700 1 -- 172.21.2.101:0/1786276060 >> v1:172.21.2.101:6789/0 conn(0x7f53a80e5470 legacy=0x7f53a807f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:59.865 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a6572700 1 --2- 172.21.2.101:0/1786276060 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5394003aa0 0x7f5394005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:59.865 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a6572700 1 --2- 172.21.2.101:0/1786276060 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5394006770 0x7f5394008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:59.865 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a6572700 1 --2- 172.21.2.101:0/1786276060 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5394009590 0x7f539400bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:59.865 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a6572700 1 -- 172.21.2.101:0/1786276060 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539400c300 con 0x7f5394009590 2021-02-10T11:12:59.865 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a6572700 1 -- 172.21.2.101:0/1786276060 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539400c520 con 0x7f5394006770 2021-02-10T11:12:59.865 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a6572700 1 -- 172.21.2.101:0/1786276060 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5394050750 con 0x7f5394003aa0 2021-02-10T11:12:59.866 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a7d75700 1 --2- 172.21.2.101:0/1786276060 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5394009590 0x7f539400bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:59.866 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a6d73700 1 --2- 172.21.2.101:0/1786276060 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5394006770 0x7f5394008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:59.866 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a7574700 1 --2- 172.21.2.101:0/1786276060 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5394003aa0 0x7f5394005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:59.866 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a7d75700 1 -- 172.21.2.101:0/1786276060 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5394003aa0 msgr2=0x7f5394005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:12:59.866 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a7d75700 1 --2- 172.21.2.101:0/1786276060 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5394003aa0 0x7f5394005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:59.866 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a7d75700 1 -- 172.21.2.101:0/1786276060 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5394006770 msgr2=0x7f5394008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:12:59.866 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a7d75700 1 --2- 172.21.2.101:0/1786276060 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5394006770 0x7f5394008c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:59.866 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.843 7f53a7d75700 1 -- 172.21.2.101:0/1786276060 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53940509b0 con 0x7f5394009590 2021-02-10T11:12:59.866 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.844 7f53a7d75700 1 -- 172.21.2.101:0/1786276060 reap_dead start 2021-02-10T11:12:59.866 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.844 7f53a7d75700 1 --2- 172.21.2.101:0/1786276060 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5394009590 0x7f539400bab0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f53a0006a40 tx=0x7f53a000c0c0).ready entity=mon.0 client_cookie=8c3b907a04288ca7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:59.866 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.844 7f53a6572700 1 -- 172.21.2.101:0/1786276060 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f53a0006ab0 con 0x7f5394009590 2021-02-10T11:12:59.867 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.844 7f53a6572700 1 -- 172.21.2.101:0/1786276060 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a0007070 con 0x7f5394009590 2021-02-10T11:12:59.867 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.844 7f53a6572700 1 -- 172.21.2.101:0/1786276060 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f53a00074c0 con 0x7f5394009590 2021-02-10T11:12:59.867 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.844 7f53ad98d700 1 -- 172.21.2.101:0/1786276060 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5394009590 msgr2=0x7f539400bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:59.867 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.844 7f53ad98d700 1 --2- 172.21.2.101:0/1786276060 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5394009590 0x7f539400bab0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f53a0006a40 tx=0x7f53a000c0c0).stop 2021-02-10T11:12:59.867 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.844 7f53ad98d700 1 -- 172.21.2.101:0/1786276060 shutdown_connections 2021-02-10T11:12:59.867 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.844 7f53ad98d700 1 --2- 172.21.2.101:0/1786276060 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5394009590 0x7f539400bab0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:59.867 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.844 7f53ad98d700 1 -- 172.21.2.101:0/1786276060 >> 172.21.2.101:0/1786276060 conn(0x7f53a80ce690 msgr2=0x7f53a80e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:12:59.867 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.844 7f53ad98d700 1 -- 172.21.2.101:0/1786276060 shutdown_connections 2021-02-10T11:12:59.867 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.844 7f53ad98d700 1 -- 172.21.2.101:0/1786276060 wait complete. 2021-02-10T11:12:59.867 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.859 7f53ad98d700 1 Processor -- start 2021-02-10T11:12:59.867 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.859 7f53ad98d700 1 -- start start 2021-02-10T11:12:59.870 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f53ad98d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53a81999f0 0x7f53a808cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:59.870 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f53ad98d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53a80b1290 0x7f53a808d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:59.870 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f53ad98d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53a808d840 0x7f53a808aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:59.870 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f53ad98d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a808dc40 con 0x7f53a80b1290 2021-02-10T11:12:59.871 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f53ad98d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a808de60 con 0x7f53a808d840 2021-02-10T11:12:59.871 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f53ad98d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a808b510 con 0x7f53a81999f0 2021-02-10T11:12:59.871 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f53a6d73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53a80b1290 0x7f53a808d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:59.871 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f53a6d73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53a80b1290 0x7f53a808d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43530/0 (socket says 172.21.2.101:43530) 2021-02-10T11:12:59.871 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f53a6d73700 1 -- 172.21.2.101:0/3342208800 learned_addr learned my addr 172.21.2.101:0/3342208800 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:12:59.871 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f53a7d75700 1 --2- 172.21.2.101:0/3342208800 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53a808d840 0x7f53a808aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:59.871 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f53a7574700 1 --2- 172.21.2.101:0/3342208800 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53a81999f0 0x7f53a808cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:59.871 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f53a6d73700 1 -- 172.21.2.101:0/3342208800 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53a81999f0 msgr2=0x7f53a808cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:59.871 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f53a6d73700 1 --2- 172.21.2.101:0/3342208800 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53a81999f0 0x7f53a808cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:59.872 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f53a6d73700 1 -- 172.21.2.101:0/3342208800 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53a808d840 msgr2=0x7f53a808aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:12:59.872 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f53a6d73700 1 --2- 172.21.2.101:0/3342208800 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53a808d840 0x7f53a808aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:12:59.872 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f53a6d73700 1 -- 172.21.2.101:0/3342208800 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a808b730 con 0x7f53a80b1290 2021-02-10T11:12:59.872 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f53a7574700 1 --2- 172.21.2.101:0/3342208800 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53a81999f0 0x7f53a808cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:12:59.872 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f53a6d73700 1 --2- 172.21.2.101:0/3342208800 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53a80b1290 0x7f53a808d220 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f5398007660 tx=0x7f539800a8f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:59.872 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f539ffff700 1 -- 172.21.2.101:0/3342208800 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f539800ace0 con 0x7f53a80b1290 2021-02-10T11:12:59.872 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f539ffff700 1 -- 172.21.2.101:0/3342208800 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5398001130 con 0x7f53a80b1290 2021-02-10T11:12:59.872 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f53ad98d700 1 -- 172.21.2.101:0/3342208800 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a808b730 con 0x7f53a80b1290 2021-02-10T11:12:59.872 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f539ffff700 1 -- 172.21.2.101:0/3342208800 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5398004590 con 0x7f53a80b1290 2021-02-10T11:12:59.872 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f53ad98d700 1 -- 172.21.2.101:0/3342208800 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a808b730 con 0x7f53a80b1290 2021-02-10T11:12:59.872 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.860 7f539ffff700 1 -- 172.21.2.101:0/3342208800 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f53980043a0 con 0x7f53a80b1290 2021-02-10T11:12:59.872 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.861 7f539ffff700 1 --2- 172.21.2.101:0/3342208800 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f538c0186c0 0x7f538c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:12:59.873 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.861 7f539ffff700 1 -- 172.21.2.101:0/3342208800 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1578+0+0 (secure 0 0 0) 0x7f5398031800 con 0x7f53a80b1290 2021-02-10T11:12:59.873 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.861 7f53a7574700 1 --2- 172.21.2.101:0/3342208800 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f538c0186c0 0x7f538c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:12:59.873 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.861 7f53a7574700 1 --2- 172.21.2.101:0/3342208800 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f538c0186c0 0x7f538c01abc0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:12:59.896 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.892 7f53ad98d700 1 -- 172.21.2.101:0/3342208800 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a808c6d0 con 0x7f53a80b1290 2021-02-10T11:12:59.899 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:12:59.894 7f539ffff700 1 -- 172.21.2.101:0/3342208800 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f53980010d0 con 0x7f53a80b1290 2021-02-10T11:13:00.028 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:00.024 7f53ad98d700 1 -- 172.21.2.101:0/3342208800 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53a808c6d0 con 0x7f53a80b1290 2021-02-10T11:13:00.028 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:00.024 7f539ffff700 1 -- 172.21.2.101:0/3342208800 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+5464 (secure 0 0 0) 0x7f53980043a0 con 0x7f53a80b1290 2021-02-10T11:13:00.029 INFO:teuthology.misc.health.gibba001.stdout: 2021-02-10T11:13:00.029 INFO:teuthology.misc.health.gibba001.stdout:{"epoch":10,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:12:58.508168","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:13:00.031 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:00.026 7f53ad98d700 1 -- 172.21.2.101:0/3342208800 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f538c0186c0 msgr2=0x7f538c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:00.031 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:00.026 7f53ad98d700 1 --2- 172.21.2.101:0/3342208800 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f538c0186c0 0x7f538c01abc0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:00.031 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:00.026 7f53ad98d700 1 -- 172.21.2.101:0/3342208800 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53a80b1290 msgr2=0x7f53a808d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:00.031 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:00.026 7f53ad98d700 1 --2- 172.21.2.101:0/3342208800 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53a80b1290 0x7f53a808d220 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f5398007660 tx=0x7f539800a8f0).stop 2021-02-10T11:13:00.031 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:00.027 7f53ad98d700 1 -- 172.21.2.101:0/3342208800 shutdown_connections 2021-02-10T11:13:00.031 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:00.027 7f53ad98d700 1 --2- 172.21.2.101:0/3342208800 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53a808d840 0x7f53a808aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:00.031 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:00.027 7f53ad98d700 1 --2- 172.21.2.101:0/3342208800 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53a80b1290 0x7f53a808d220 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:00.031 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:00.027 7f53ad98d700 1 --2- 172.21.2.101:0/3342208800 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f538c0186c0 0x7f538c01abc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:00.032 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:00.027 7f53ad98d700 1 --2- 172.21.2.101:0/3342208800 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53a81999f0 0x7f53a808cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:00.032 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:00.027 7f53ad98d700 1 -- 172.21.2.101:0/3342208800 >> 172.21.2.101:0/3342208800 conn(0x7f53a81a31a0 msgr2=0x7f53a818c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:00.032 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:00.027 7f53ad98d700 1 -- 172.21.2.101:0/3342208800 shutdown_connections 2021-02-10T11:13:00.032 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:00.027 7f53ad98d700 1 -- 172.21.2.101:0/3342208800 wait complete. 2021-02-10T11:13:00.039 DEBUG:teuthology.misc:0 of 8 OSDs are up 2021-02-10T11:13:00.494 INFO:tasks.ceph.osd.1.gibba001.stderr:2021-02-10 11:13:00.490 7f3b2a814a80 -1 osd.1 0 log_to_monitors {default=true} 2021-02-10T11:13:00.498 INFO:tasks.ceph.osd.0.gibba001.stderr:2021-02-10 11:13:00.495 7f1387b44a80 -1 osd.0 0 log_to_monitors {default=true} 2021-02-10T11:13:00.499 INFO:tasks.ceph.osd.2.gibba001.stderr:2021-02-10 11:13:00.495 7f8a1f5f1a80 -1 osd.2 0 log_to_monitors {default=true} 2021-02-10T11:13:00.507 INFO:tasks.ceph.osd.3.gibba001.stderr:2021-02-10 11:13:00.503 7f32488a7a80 -1 osd.3 0 log_to_monitors {default=true} 2021-02-10T11:13:00.539 INFO:tasks.ceph.osd.6.gibba008.stderr:2021-02-10 11:13:00.542 7f5a2c6f3a80 -1 osd.6 0 log_to_monitors {default=true} 2021-02-10T11:13:00.549 INFO:tasks.ceph.osd.5.gibba008.stderr:2021-02-10 11:13:00.551 7f8f87b10a80 -1 osd.5 0 log_to_monitors {default=true} 2021-02-10T11:13:00.566 INFO:tasks.ceph.osd.4.gibba008.stderr:2021-02-10 11:13:00.569 7fac8cfbfa80 -1 osd.4 0 log_to_monitors {default=true} 2021-02-10T11:13:00.566 INFO:tasks.ceph.osd.7.gibba008.stderr:2021-02-10 11:13:00.569 7f5889f23a80 -1 osd.7 0 log_to_monitors {default=true} 2021-02-10T11:13:02.217 INFO:tasks.ceph.osd.0.gibba001.stderr:2021-02-10 11:13:02.213 7f136e1bd700 -1 osd.0 0 waiting for initial osdmap 2021-02-10T11:13:02.218 INFO:tasks.ceph.osd.3.gibba001.stderr:2021-02-10 11:13:02.213 7f322ef20700 -1 osd.3 0 waiting for initial osdmap 2021-02-10T11:13:02.218 INFO:tasks.ceph.osd.1.gibba001.stderr:2021-02-10 11:13:02.213 7f3b10e8d700 -1 osd.1 0 waiting for initial osdmap 2021-02-10T11:13:02.218 INFO:tasks.ceph.osd.2.gibba001.stderr:2021-02-10 11:13:02.213 7f8a05c6a700 -1 osd.2 0 waiting for initial osdmap 2021-02-10T11:13:02.219 INFO:tasks.ceph.osd.5.gibba008.stderr:2021-02-10 11:13:02.221 7f8f6e189700 -1 osd.5 0 waiting for initial osdmap 2021-02-10T11:13:02.219 INFO:tasks.ceph.osd.6.gibba008.stderr:2021-02-10 11:13:02.221 7f5a12d6c700 -1 osd.6 0 waiting for initial osdmap 2021-02-10T11:13:02.219 INFO:tasks.ceph.osd.4.gibba008.stderr:2021-02-10 11:13:02.221 7fac73638700 -1 osd.4 0 waiting for initial osdmap 2021-02-10T11:13:02.219 INFO:tasks.ceph.osd.7.gibba008.stderr:2021-02-10 11:13:02.221 7f587059c700 -1 osd.7 0 waiting for initial osdmap 2021-02-10T11:13:02.222 INFO:tasks.ceph.osd.5.gibba008.stderr:2021-02-10 11:13:02.224 7f8f7a3a7700 -1 osd.5 12 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-02-10T11:13:02.222 INFO:tasks.ceph.osd.6.gibba008.stderr:2021-02-10 11:13:02.224 7f5a1ef8a700 -1 osd.6 12 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-02-10T11:13:02.222 INFO:tasks.ceph.osd.2.gibba001.stderr:2021-02-10 11:13:02.218 7f8a11e88700 -1 osd.2 12 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-02-10T11:13:02.223 INFO:tasks.ceph.osd.3.gibba001.stderr:2021-02-10 11:13:02.218 7f323b13e700 -1 osd.3 12 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-02-10T11:13:02.223 INFO:tasks.ceph.osd.0.gibba001.stderr:2021-02-10 11:13:02.219 7f137a3db700 -1 osd.0 12 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-02-10T11:13:02.223 INFO:tasks.ceph.osd.7.gibba008.stderr:2021-02-10 11:13:02.225 7f587c7ba700 -1 osd.7 12 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-02-10T11:13:02.223 INFO:tasks.ceph.osd.4.gibba008.stderr:2021-02-10 11:13:02.225 7fac7f856700 -1 osd.4 12 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-02-10T11:13:02.223 INFO:tasks.ceph.osd.1.gibba001.stderr:2021-02-10 11:13:02.219 7f3b1d0ab700 -1 osd.1 12 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-02-10T11:13:06.847 DEBUG:teuthology.orchestra.run.gibba001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2021-02-10T11:13:07.263 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.242 7f1077d76700 1 Processor -- start 2021-02-10T11:13:07.264 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.242 7f1077d76700 1 -- start start 2021-02-10T11:13:07.264 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.243 7f1077d76700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f10700e5470 0x7f107007e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:07.264 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.243 7f1077d76700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f107007f300 0x7f107007f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:07.264 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.243 7f1077d76700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f107007ed80 con 0x7f10700e5070 2021-02-10T11:13:07.264 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.243 7f1077d76700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10701b4010 con 0x7f107007f300 2021-02-10T11:13:07.264 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.243 7f1077d76700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10700d1790 con 0x7f10700e5470 2021-02-10T11:13:07.264 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.243 7f1075b92700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f10700e5070 0x7f10701b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44250/0 (socket says 172.21.2.101:44250) 2021-02-10T11:13:07.264 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.243 7f1075b92700 1 -- 172.21.2.101:0/172190912 learned_addr learned my addr 172.21.2.101:0/172190912 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:07.264 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.243 7f1076393700 1 --2- 172.21.2.101:0/172190912 >> v2:172.21.2.101:3300/0 conn(0x7f107007f300 0x7f107007f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:07.264 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.243 7f1074b90700 1 -- 172.21.2.101:0/172190912 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899793436 0 0) 0x7f1064001180 con 0x7f10700e5070 2021-02-10T11:13:07.264 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.243 7f1074b90700 1 -- 172.21.2.101:0/172190912 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f105c002fe0 con 0x7f10700e5070 2021-02-10T11:13:07.265 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.244 7f1076393700 1 -- 172.21.2.101:0/172190912 >> v2:172.21.2.108:3300/0 conn(0x7f10700e5470 msgr2=0x7f107007e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:07.265 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.244 7f1076393700 1 --2- 172.21.2.101:0/172190912 >> v2:172.21.2.108:3300/0 conn(0x7f10700e5470 0x7f107007e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:07.265 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.244 7f1076393700 1 -- 172.21.2.101:0/172190912 >> v1:172.21.2.101:6789/0 conn(0x7f10700e5070 legacy=0x7f10701b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:07.265 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.244 7f1076393700 1 -- 172.21.2.101:0/172190912 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10700d19b0 con 0x7f107007f300 2021-02-10T11:13:07.265 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.244 7f1076393700 1 --2- 172.21.2.101:0/172190912 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f107007f300 0x7f107007f700 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f1068006e30 tx=0x7f1068009e60).ready entity=mon.0 client_cookie=90957381c551496e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:07.265 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.244 7f1074b90700 1 -- 172.21.2.101:0/172190912 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1068006a50 con 0x7f107007f300 2021-02-10T11:13:07.265 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.244 7f1074b90700 1 -- 172.21.2.101:0/172190912 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106800b300 con 0x7f107007f300 2021-02-10T11:13:07.265 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.244 7f1074b90700 1 -- 172.21.2.101:0/172190912 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f106800b820 con 0x7f107007f300 2021-02-10T11:13:07.265 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.244 7f1077d76700 1 -- 172.21.2.101:0/172190912 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f107007f300 msgr2=0x7f107007f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:07.265 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.244 7f1077d76700 1 --2- 172.21.2.101:0/172190912 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f107007f300 0x7f107007f700 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f1068006e30 tx=0x7f1068009e60).stop 2021-02-10T11:13:07.265 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.245 7f1077d76700 1 -- 172.21.2.101:0/172190912 shutdown_connections 2021-02-10T11:13:07.265 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.245 7f1077d76700 1 --2- 172.21.2.101:0/172190912 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f107007f300 0x7f107007f700 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:07.265 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.245 7f1077d76700 1 --2- 172.21.2.101:0/172190912 >> v2:172.21.2.108:3300/0 conn(0x7f10700e5470 0x7f107007e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:07.266 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.245 7f1077d76700 1 -- 172.21.2.101:0/172190912 >> 172.21.2.101:0/172190912 conn(0x7f10700ce690 msgr2=0x7f10700e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:07.266 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.245 7f1077d76700 1 -- 172.21.2.101:0/172190912 shutdown_connections 2021-02-10T11:13:07.266 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.245 7f1077d76700 1 -- 172.21.2.101:0/172190912 wait complete. 2021-02-10T11:13:07.266 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.260 7f1077d76700 1 Processor -- start 2021-02-10T11:13:07.266 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.260 7f1077d76700 1 -- start start 2021-02-10T11:13:07.268 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.260 7f1077d76700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f107019d8e0 0x7f107019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:07.268 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.260 7f1077d76700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f107019d410 0x7f1070198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:07.268 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.260 7f1077d76700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1070199340 0x7f107008a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:07.268 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.260 7f1077d76700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10701998a0 con 0x7f107019d8e0 2021-02-10T11:13:07.268 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.260 7f1077d76700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107008aa80 con 0x7f107019d410 2021-02-10T11:13:07.268 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.260 7f1077d76700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107008ac60 con 0x7f1070199340 2021-02-10T11:13:07.268 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.260 7f1075b92700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f107019d8e0 0x7f107019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:07.268 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.260 7f1075b92700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f107019d8e0 0x7f107019cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43762/0 (socket says 172.21.2.101:43762) 2021-02-10T11:13:07.268 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.260 7f1075b92700 1 -- 172.21.2.101:0/2114763290 learned_addr learned my addr 172.21.2.101:0/2114763290 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:07.268 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.261 7f1075b92700 1 -- 172.21.2.101:0/2114763290 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1070199340 msgr2=0x7f107008a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:07.268 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.261 7f1075b92700 1 --2- 172.21.2.101:0/2114763290 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1070199340 0x7f107008a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:07.268 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.261 7f1075b92700 1 -- 172.21.2.101:0/2114763290 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f107019d410 msgr2=0x7f1070198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:07.269 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.261 7f1075b92700 1 --2- 172.21.2.101:0/2114763290 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f107019d410 0x7f1070198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:07.269 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.261 7f1075b92700 1 -- 172.21.2.101:0/2114763290 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107008ae80 con 0x7f107019d8e0 2021-02-10T11:13:07.269 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.261 7f1075b92700 1 --2- 172.21.2.101:0/2114763290 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f107019d8e0 0x7f107019cdf0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f106400a5d0 tx=0x7f106400a970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:07.269 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.261 7f10627fc700 1 -- 172.21.2.101:0/2114763290 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1064000ce0 con 0x7f107019d8e0 2021-02-10T11:13:07.269 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.261 7f10627fc700 1 -- 172.21.2.101:0/2114763290 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10640043a0 con 0x7f107019d8e0 2021-02-10T11:13:07.269 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.261 7f10627fc700 1 -- 172.21.2.101:0/2114763290 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f10640047f0 con 0x7f107019d8e0 2021-02-10T11:13:07.269 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.261 7f1077d76700 1 -- 172.21.2.101:0/2114763290 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f107008ae80 con 0x7f107019d8e0 2021-02-10T11:13:07.269 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.261 7f1077d76700 1 -- 172.21.2.101:0/2114763290 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f107008ae80 con 0x7f107019d8e0 2021-02-10T11:13:07.269 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.262 7f10627fc700 1 -- 172.21.2.101:0/2114763290 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1064004790 con 0x7f107019d8e0 2021-02-10T11:13:07.269 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.262 7f10627fc700 1 --2- 172.21.2.101:0/2114763290 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10580186c0 0x7f105801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:07.269 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.262 7f10627fc700 1 -- 172.21.2.101:0/2114763290 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4400+0+0 (secure 0 0 0) 0x7f10640320a0 con 0x7f107019d8e0 2021-02-10T11:13:07.269 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.262 7f1075391700 1 --2- 172.21.2.101:0/2114763290 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10580186c0 0x7f105801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:07.269 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.262 7f1075391700 1 --2- 172.21.2.101:0/2114763290 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10580186c0 0x7f105801abc0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:07.296 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.292 7f1077d76700 1 -- 172.21.2.101:0/2114763290 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f107008b780 con 0x7f107019d8e0 2021-02-10T11:13:07.299 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.295 7f10627fc700 1 -- 172.21.2.101:0/2114763290 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1064004580 con 0x7f107019d8e0 2021-02-10T11:13:07.423 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.419 7f1077d76700 1 -- 172.21.2.101:0/2114763290 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1070084b60 con 0x7f107019d8e0 2021-02-10T11:13:07.423 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.420 7f10627fc700 1 -- 172.21.2.101:0/2114763290 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+9772 (secure 0 0 0) 0x7f1064004790 con 0x7f107019d8e0 2021-02-10T11:13:07.424 INFO:teuthology.misc.health.gibba001.stdout: 2021-02-10T11:13:07.424 INFO:teuthology.misc.health.gibba001.stdout:{"epoch":13,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:13:03.215004","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:13:07.426 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.421 7f1077d76700 1 -- 172.21.2.101:0/2114763290 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10580186c0 msgr2=0x7f105801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:07.426 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.421 7f1077d76700 1 --2- 172.21.2.101:0/2114763290 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10580186c0 0x7f105801abc0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:07.426 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.421 7f1077d76700 1 -- 172.21.2.101:0/2114763290 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f107019d8e0 msgr2=0x7f107019cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:07.426 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.421 7f1077d76700 1 --2- 172.21.2.101:0/2114763290 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f107019d8e0 0x7f107019cdf0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f106400a5d0 tx=0x7f106400a970).stop 2021-02-10T11:13:07.426 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.422 7f1077d76700 1 -- 172.21.2.101:0/2114763290 shutdown_connections 2021-02-10T11:13:07.426 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.422 7f1077d76700 1 --2- 172.21.2.101:0/2114763290 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10580186c0 0x7f105801abc0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:07.426 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.422 7f1077d76700 1 --2- 172.21.2.101:0/2114763290 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1070199340 0x7f107008a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:07.426 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.422 7f1077d76700 1 --2- 172.21.2.101:0/2114763290 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f107019d410 0x7f1070198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:07.426 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.422 7f1077d76700 1 --2- 172.21.2.101:0/2114763290 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f107019d8e0 0x7f107019cdf0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:07.426 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.422 7f1077d76700 1 -- 172.21.2.101:0/2114763290 >> 172.21.2.101:0/2114763290 conn(0x7f1070086770 msgr2=0x7f1070089740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:07.426 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.422 7f1077d76700 1 -- 172.21.2.101:0/2114763290 shutdown_connections 2021-02-10T11:13:07.426 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:07.422 7f1077d76700 1 -- 172.21.2.101:0/2114763290 wait complete. 2021-02-10T11:13:07.433 DEBUG:teuthology.misc:8 of 8 OSDs are up 2021-02-10T11:13:07.433 INFO:tasks.ceph:Creating RBD pool 2021-02-10T11:13:07.434 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --cluster ceph osd pool create rbd 8 2021-02-10T11:13:07.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.840 7ff274ca8700 1 Processor -- start 2021-02-10T11:13:07.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.840 7ff274ca8700 1 -- start start 2021-02-10T11:13:07.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.841 7ff274ca8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff2701b0870 0x7ff2700f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:07.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.841 7ff274ca8700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2701b0c70 con 0x7ff2700efe00 2021-02-10T11:13:07.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.841 7ff274ca8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2700f0b50 con 0x7ff2701b0470 2021-02-10T11:13:07.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.841 7ff274ca8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2700e5ea0 con 0x7ff2701b0870 2021-02-10T11:13:07.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.841 7ff26f05f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff2700efe00 0x7ff2700f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44266/0 (socket says 172.21.2.101:44266) 2021-02-10T11:13:07.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.841 7ff26f05f700 1 -- 172.21.2.101:0/1146753869 learned_addr learned my addr 172.21.2.101:0/1146753869 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:07.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.841 7ff26d85c700 1 -- 172.21.2.101:0/1146753869 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2593076672 0 0) 0x7ff268001500 con 0x7ff2700efe00 2021-02-10T11:13:07.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.841 7ff26d85c700 1 -- 172.21.2.101:0/1146753869 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff25c002fe0 con 0x7ff2700efe00 2021-02-10T11:13:07.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.841 7ff26e05d700 1 --2- 172.21.2.101:0/1146753869 >> v2:172.21.2.108:3300/0 conn(0x7ff2701b0870 0x7ff2700f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:07.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26d85c700 1 -- 172.21.2.101:0/1146753869 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 887777401 0 0) 0x7ff268001460 con 0x7ff2700efe00 2021-02-10T11:13:07.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26d85c700 1 -- 172.21.2.101:0/1146753869 >> v2:172.21.2.108:3300/0 conn(0x7ff2701b0870 msgr2=0x7ff2700f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:07.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26d85c700 1 --2- 172.21.2.101:0/1146753869 >> v2:172.21.2.108:3300/0 conn(0x7ff2701b0870 0x7ff2700f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:07.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26d85c700 1 -- 172.21.2.101:0/1146753869 >> v1:172.21.2.108:6789/0 conn(0x7ff2701b0470 legacy=0x7ff2700ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:07.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26d85c700 1 -- 172.21.2.101:0/1146753869 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2700e52e0 con 0x7ff2700efe00 2021-02-10T11:13:07.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26d85c700 1 -- 172.21.2.101:0/1146753869 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7ff268001980 con 0x7ff2700efe00 2021-02-10T11:13:07.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26d85c700 1 -- 172.21.2.101:0/1146753869 >> v1:172.21.2.101:6789/0 conn(0x7ff2700efe00 legacy=0x7ff2700f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:07.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26d85c700 1 --2- 172.21.2.101:0/1146753869 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff25c003b20 0x7ff25c005f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:07.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26d85c700 1 --2- 172.21.2.101:0/1146753869 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff25c0067b0 0x7ff25c008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:07.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26d85c700 1 --2- 172.21.2.101:0/1146753869 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff25c009590 0x7ff25c00bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:07.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26d85c700 1 -- 172.21.2.101:0/1146753869 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c00c300 con 0x7ff25c009590 2021-02-10T11:13:07.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26d85c700 1 -- 172.21.2.101:0/1146753869 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c00c520 con 0x7ff25c003b20 2021-02-10T11:13:07.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26d85c700 1 -- 172.21.2.101:0/1146753869 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c00c740 con 0x7ff25c0067b0 2021-02-10T11:13:07.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26f05f700 1 --2- 172.21.2.101:0/1146753869 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff25c009590 0x7ff25c00bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:07.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26e85e700 1 --2- 172.21.2.101:0/1146753869 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff25c003b20 0x7ff25c005f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:07.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26e05d700 1 --2- 172.21.2.101:0/1146753869 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff25c0067b0 0x7ff25c008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:07.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26f05f700 1 -- 172.21.2.101:0/1146753869 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff25c0067b0 msgr2=0x7ff25c008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:07.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26f05f700 1 --2- 172.21.2.101:0/1146753869 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff25c0067b0 0x7ff25c008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:07.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26f05f700 1 -- 172.21.2.101:0/1146753869 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff25c003b20 msgr2=0x7ff25c005f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:07.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26f05f700 1 --2- 172.21.2.101:0/1146753869 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff25c003b20 0x7ff25c005f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:07.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26f05f700 1 -- 172.21.2.101:0/1146753869 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c0509b0 con 0x7ff25c009590 2021-02-10T11:13:07.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26e05d700 1 --2- 172.21.2.101:0/1146753869 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff25c0067b0 0x7ff25c008c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:13:07.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26f05f700 1 -- 172.21.2.101:0/1146753869 reap_dead start 2021-02-10T11:13:07.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26f05f700 1 --2- 172.21.2.101:0/1146753869 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff25c009590 0x7ff25c00bab0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7ff2680018f0 tx=0x7ff26800a310).ready entity=mon.0 client_cookie=4f99cab535308f91 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:07.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26d85c700 1 -- 172.21.2.101:0/1146753869 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff268007340 con 0x7ff25c009590 2021-02-10T11:13:07.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26d85c700 1 -- 172.21.2.101:0/1146753869 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff268001930 con 0x7ff25c009590 2021-02-10T11:13:07.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff26d85c700 1 -- 172.21.2.101:0/1146753869 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff268004910 con 0x7ff25c009590 2021-02-10T11:13:07.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff274ca8700 1 -- 172.21.2.101:0/1146753869 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff25c009590 msgr2=0x7ff25c00bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:07.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.842 7ff274ca8700 1 --2- 172.21.2.101:0/1146753869 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff25c009590 0x7ff25c00bab0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7ff2680018f0 tx=0x7ff26800a310).stop 2021-02-10T11:13:07.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.843 7ff274ca8700 1 -- 172.21.2.101:0/1146753869 shutdown_connections 2021-02-10T11:13:07.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.843 7ff274ca8700 1 --2- 172.21.2.101:0/1146753869 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff25c009590 0x7ff25c00bab0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:07.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.843 7ff274ca8700 1 -- 172.21.2.101:0/1146753869 >> 172.21.2.101:0/1146753869 conn(0x7ff2700ce3e0 msgr2=0x7ff2700e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:07.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.843 7ff274ca8700 1 -- 172.21.2.101:0/1146753869 shutdown_connections 2021-02-10T11:13:07.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.843 7ff274ca8700 1 -- 172.21.2.101:0/1146753869 wait complete. 2021-02-10T11:13:07.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.858 7ff274ca8700 1 Processor -- start 2021-02-10T11:13:07.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.858 7ff274ca8700 1 -- start start 2021-02-10T11:13:07.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.858 7ff274ca8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff27009b930 0x7ff2700b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:07.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.858 7ff274ca8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2700b8410 0x7ff2700b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:07.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.858 7ff274ca8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2700c1f80 0x7ff2700c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:07.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.858 7ff274ca8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2700c1a00 con 0x7ff2700c1f80 2021-02-10T11:13:07.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.858 7ff274ca8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2700b8e30 con 0x7ff27009b930 2021-02-10T11:13:07.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.858 7ff274ca8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2700b9050 con 0x7ff2700b8410 2021-02-10T11:13:07.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.858 7ff26f05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2700c1f80 0x7ff2700c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:07.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.858 7ff26f05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2700c1f80 0x7ff2700c2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43780/0 (socket says 172.21.2.101:43780) 2021-02-10T11:13:07.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.858 7ff26f05f700 1 -- 172.21.2.101:0/2379610971 learned_addr learned my addr 172.21.2.101:0/2379610971 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:07.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.859 7ff26f05f700 1 -- 172.21.2.101:0/2379610971 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2700b8410 msgr2=0x7ff2700b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:07.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.859 7ff26f05f700 1 --2- 172.21.2.101:0/2379610971 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2700b8410 0x7ff2700b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:07.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.859 7ff26f05f700 1 -- 172.21.2.101:0/2379610971 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff27009b930 msgr2=0x7ff2700b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:07.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.859 7ff26f05f700 1 --2- 172.21.2.101:0/2379610971 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff27009b930 0x7ff2700b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:07.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.859 7ff26f05f700 1 -- 172.21.2.101:0/2379610971 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2700c29a0 con 0x7ff2700c1f80 2021-02-10T11:13:07.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.859 7ff26f05f700 1 --2- 172.21.2.101:0/2379610971 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2700c1f80 0x7ff2700c2380 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7ff268008fc0 tx=0x7ff268007330).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:07.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.859 7ff2677fe700 1 -- 172.21.2.101:0/2379610971 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff26800b310 con 0x7ff2700c1f80 2021-02-10T11:13:07.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.859 7ff2677fe700 1 -- 172.21.2.101:0/2379610971 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff268001bc0 con 0x7ff2700c1f80 2021-02-10T11:13:07.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.859 7ff274ca8700 1 -- 172.21.2.101:0/2379610971 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2700c29a0 con 0x7ff2700c1f80 2021-02-10T11:13:07.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.859 7ff2677fe700 1 -- 172.21.2.101:0/2379610971 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff26800b810 con 0x7ff2700c1f80 2021-02-10T11:13:07.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.859 7ff274ca8700 1 -- 172.21.2.101:0/2379610971 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2700c2f30 con 0x7ff2700c1f80 2021-02-10T11:13:07.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.860 7ff2677fe700 1 -- 172.21.2.101:0/2379610971 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff268009020 con 0x7ff2700c1f80 2021-02-10T11:13:07.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.860 7ff2677fe700 1 --2- 172.21.2.101:0/2379610971 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2540186c0 0x7ff25401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:07.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.860 7ff2677fe700 1 -- 172.21.2.101:0/2379610971 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4400+0+0 (secure 0 0 0) 0x7ff2680322c0 con 0x7ff2700c1f80 2021-02-10T11:13:07.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.860 7ff26e85e700 1 --2- 172.21.2.101:0/2379610971 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2540186c0 0x7ff25401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:07.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.860 7ff26e85e700 1 --2- 172.21.2.101:0/2379610971 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2540186c0 0x7ff25401abc0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:07.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.890 7ff274ca8700 1 -- 172.21.2.101:0/2379610971 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff270105b30 con 0x7ff2700c1f80 2021-02-10T11:13:07.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:07.893 7ff2677fe700 1 -- 172.21.2.101:0/2379610971 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff26802e430 con 0x7ff2700c1f80 2021-02-10T11:13:08.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.020 7ff274ca8700 1 -- 172.21.2.101:0/2379610971 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "rbd"} v 0) v1 -- 0x7ff2700f0b50 con 0x7ff2700c1f80 2021-02-10T11:13:08.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.480 7ff2677fe700 1 -- 172.21.2.101:0/2379610971 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "rbd"}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7ff26802e430 con 0x7ff2700c1f80 2021-02-10T11:13:08.490 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'rbd' created 2021-02-10T11:13:08.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.488 7ff274ca8700 1 -- 172.21.2.101:0/2379610971 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2540186c0 msgr2=0x7ff25401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:08.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.488 7ff274ca8700 1 --2- 172.21.2.101:0/2379610971 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2540186c0 0x7ff25401abc0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:08.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.488 7ff274ca8700 1 -- 172.21.2.101:0/2379610971 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2700c1f80 msgr2=0x7ff2700c2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:08.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.488 7ff274ca8700 1 --2- 172.21.2.101:0/2379610971 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2700c1f80 0x7ff2700c2380 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7ff268008fc0 tx=0x7ff268007330).stop 2021-02-10T11:13:08.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.488 7ff274ca8700 1 -- 172.21.2.101:0/2379610971 shutdown_connections 2021-02-10T11:13:08.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.488 7ff274ca8700 1 --2- 172.21.2.101:0/2379610971 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2540186c0 0x7ff25401abc0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:08.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.488 7ff274ca8700 1 --2- 172.21.2.101:0/2379610971 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2700b8410 0x7ff2700b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:08.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.488 7ff274ca8700 1 --2- 172.21.2.101:0/2379610971 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2700c1f80 0x7ff2700c2380 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:08.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.488 7ff274ca8700 1 --2- 172.21.2.101:0/2379610971 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff27009b930 0x7ff2700b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:08.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.488 7ff274ca8700 1 -- 172.21.2.101:0/2379610971 >> 172.21.2.101:0/2379610971 conn(0x7ff270094e10 msgr2=0x7ff2700a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:08.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.488 7ff274ca8700 1 -- 172.21.2.101:0/2379610971 shutdown_connections 2021-02-10T11:13:08.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.488 7ff274ca8700 1 -- 172.21.2.101:0/2379610971 wait complete. 2021-02-10T11:13:08.500 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2021-02-10T11:13:08.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.907 7f8262a5b700 1 Processor -- start 2021-02-10T11:13:08.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.907 7f8262a5b700 1 -- start start 2021-02-10T11:13:08.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.907 7f8262a5b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f825c0d1860 0x7f825c1b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:08.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.907 7f8262a5b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f825c1b0a00 0x7f825c0cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:08.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.907 7f8262a5b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f825c0d1c60 0x7f825c0cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:08.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.907 7f8262a5b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f825c0fe220 con 0x7f825c0d1c60 2021-02-10T11:13:08.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.907 7f8262a5b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f825c0e5ec0 con 0x7f825c1b0a00 2021-02-10T11:13:08.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.907 7f8262a5b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825c0f0b00 con 0x7f825c0d1860 2021-02-10T11:13:08.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.907 7f8261078700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f825c0d1c60 0x7f825c0cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:08.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.907 7f8261078700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f825c0d1c60 0x7f825c0cf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43788/0 (socket says 172.21.2.101:43788) 2021-02-10T11:13:08.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.907 7f8261078700 1 -- 172.21.2.101:0/3107665799 learned_addr learned my addr 172.21.2.101:0/3107665799 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:08.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.908 7f8261078700 1 -- 172.21.2.101:0/3107665799 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f825c0d1860 msgr2=0x7f825c1b03e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:13:08.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.908 7f8261078700 1 --2- 172.21.2.101:0/3107665799 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f825c0d1860 0x7f825c1b03e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:08.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.908 7f8261078700 1 -- 172.21.2.101:0/3107665799 >> v2:172.21.2.108:3300/0 conn(0x7f825c1b0a00 msgr2=0x7f825c0cf320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:13:08.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.908 7f8261078700 1 --2- 172.21.2.101:0/3107665799 >> v2:172.21.2.108:3300/0 conn(0x7f825c1b0a00 0x7f825c0cf320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:08.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.908 7f8261078700 1 -- 172.21.2.101:0/3107665799 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c0cff60 con 0x7f825c0d1c60 2021-02-10T11:13:08.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.908 7f8261078700 1 --2- 172.21.2.101:0/3107665799 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f825c0d1c60 0x7f825c0cf940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f8254006f00 tx=0x7f8254009f10).ready entity=mon.0 client_cookie=a05bef7be6221de0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:08.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.908 7f825b7fe700 1 -- 172.21.2.101:0/3107665799 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f825400b3b0 con 0x7f825c0d1c60 2021-02-10T11:13:08.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.908 7f825b7fe700 1 -- 172.21.2.101:0/3107665799 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8254000d30 con 0x7f825c0d1c60 2021-02-10T11:13:08.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.908 7f825b7fe700 1 -- 172.21.2.101:0/3107665799 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f825400b820 con 0x7f825c0d1c60 2021-02-10T11:13:08.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.908 7f8262a5b700 1 -- 172.21.2.101:0/3107665799 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f825c0d1c60 msgr2=0x7f825c0cf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:08.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.908 7f8262a5b700 1 --2- 172.21.2.101:0/3107665799 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f825c0d1c60 0x7f825c0cf940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f8254006f00 tx=0x7f8254009f10).stop 2021-02-10T11:13:08.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.909 7f8262a5b700 1 -- 172.21.2.101:0/3107665799 shutdown_connections 2021-02-10T11:13:08.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.909 7f8262a5b700 1 --2- 172.21.2.101:0/3107665799 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f825c0d1c60 0x7f825c0cf940 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:08.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.909 7f8262a5b700 1 --2- 172.21.2.101:0/3107665799 >> v2:172.21.2.108:3300/0 conn(0x7f825c1b0a00 0x7f825c0cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:08.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.909 7f8262a5b700 1 --2- 172.21.2.101:0/3107665799 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f825c0d1860 0x7f825c1b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:08.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.909 7f8262a5b700 1 -- 172.21.2.101:0/3107665799 >> 172.21.2.101:0/3107665799 conn(0x7f825c0ce440 msgr2=0x7f825c0d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:08.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.909 7f8262a5b700 1 -- 172.21.2.101:0/3107665799 shutdown_connections 2021-02-10T11:13:08.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.909 7f8262a5b700 1 -- 172.21.2.101:0/3107665799 wait complete. 2021-02-10T11:13:08.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.924 7f8262a5b700 1 Processor -- start 2021-02-10T11:13:08.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.924 7f8262a5b700 1 -- start start 2021-02-10T11:13:08.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.924 7f8262a5b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f825c18e420 0x7f825c18ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:08.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.924 7f8262a5b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f825c1af2d0 0x7f825c1ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:08.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.924 7f8262a5b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f825c18f440 0x7f825c18f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:08.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.924 7f8262a5b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c1aed50 con 0x7f825c1af2d0 2021-02-10T11:13:08.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.924 7f8262a5b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c18f0f0 con 0x7f825c18e420 2021-02-10T11:13:08.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.924 7f8262a5b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825c1ae9e0 con 0x7f825c18f440 2021-02-10T11:13:08.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.924 7f825bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f825c1af2d0 0x7f825c1ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:08.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.924 7f825bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f825c1af2d0 0x7f825c1ae3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43794/0 (socket says 172.21.2.101:43794) 2021-02-10T11:13:08.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.924 7f825bfff700 1 -- 172.21.2.101:0/34836107 learned_addr learned my addr 172.21.2.101:0/34836107 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:08.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.925 7f825bfff700 1 -- 172.21.2.101:0/34836107 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f825c18f440 msgr2=0x7f825c18f840 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:08.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.925 7f825bfff700 1 --2- 172.21.2.101:0/34836107 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f825c18f440 0x7f825c18f840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:08.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.925 7f825bfff700 1 -- 172.21.2.101:0/34836107 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f825c18e420 msgr2=0x7f825c18ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:08.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.925 7f825bfff700 1 --2- 172.21.2.101:0/34836107 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f825c18e420 0x7f825c18ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:08.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.925 7f825bfff700 1 -- 172.21.2.101:0/34836107 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c18fe60 con 0x7f825c1af2d0 2021-02-10T11:13:08.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.925 7f825bfff700 1 --2- 172.21.2.101:0/34836107 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f825c1af2d0 0x7f825c1ae3c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f824c00b560 tx=0x7f824c00b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:08.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.925 7f82597fa700 1 -- 172.21.2.101:0/34836107 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f824c00bcf0 con 0x7f825c1af2d0 2021-02-10T11:13:08.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.925 7f82597fa700 1 -- 172.21.2.101:0/34836107 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824c006030 con 0x7f825c1af2d0 2021-02-10T11:13:08.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.925 7f8262a5b700 1 -- 172.21.2.101:0/34836107 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f825c18fe60 con 0x7f825c1af2d0 2021-02-10T11:13:08.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.925 7f8262a5b700 1 -- 172.21.2.101:0/34836107 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f825c18fe60 con 0x7f825c1af2d0 2021-02-10T11:13:08.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.925 7f82597fa700 1 -- 172.21.2.101:0/34836107 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f824c0054d0 con 0x7f825c1af2d0 2021-02-10T11:13:08.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.926 7f82597fa700 1 -- 172.21.2.101:0/34836107 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f824c006210 con 0x7f825c1af2d0 2021-02-10T11:13:08.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.926 7f82597fa700 1 --2- 172.21.2.101:0/34836107 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82400186c0 0x7f824001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:08.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.926 7f82597fa700 1 -- 172.21.2.101:0/34836107 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4742+0+0 (secure 0 0 0) 0x7f824c032490 con 0x7f825c1af2d0 2021-02-10T11:13:08.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.926 7f8260877700 1 --2- 172.21.2.101:0/34836107 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82400186c0 0x7f824001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:08.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.926 7f8260877700 1 --2- 172.21.2.101:0/34836107 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82400186c0 0x7f824001abc0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:08.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.957 7f8262a5b700 1 -- 172.21.2.101:0/34836107 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f825c0ebbe0 con 0x7f825c1af2d0 2021-02-10T11:13:08.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:08.959 7f82597fa700 1 -- 172.21.2.101:0/34836107 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f824c006b80 con 0x7f825c1af2d0 2021-02-10T11:13:09.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.083 7f8262a5b700 1 -- 172.21.2.101:0/34836107 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool application enable", "app": "rbd", "pool": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f825c0f0b00 con 0x7f825c1af2d0 2021-02-10T11:13:09.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.483 7f82597fa700 1 -- 172.21.2.101:0/34836107 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "app": "rbd", "pool": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7f824c006030 con 0x7f825c1af2d0 2021-02-10T11:13:09.503 INFO:teuthology.orchestra.run.gibba001.stderr:enabled application 'rbd' on pool 'rbd' 2021-02-10T11:13:09.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.501 7f8262a5b700 1 -- 172.21.2.101:0/34836107 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82400186c0 msgr2=0x7f824001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:09.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.501 7f8262a5b700 1 --2- 172.21.2.101:0/34836107 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82400186c0 0x7f824001abc0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:09.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.501 7f8262a5b700 1 -- 172.21.2.101:0/34836107 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f825c1af2d0 msgr2=0x7f825c1ae3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:09.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.501 7f8262a5b700 1 --2- 172.21.2.101:0/34836107 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f825c1af2d0 0x7f825c1ae3c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f824c00b560 tx=0x7f824c00b900).stop 2021-02-10T11:13:09.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.501 7f8262a5b700 1 -- 172.21.2.101:0/34836107 shutdown_connections 2021-02-10T11:13:09.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.501 7f8262a5b700 1 --2- 172.21.2.101:0/34836107 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82400186c0 0x7f824001abc0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:09.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.501 7f8262a5b700 1 --2- 172.21.2.101:0/34836107 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f825c18f440 0x7f825c18f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:09.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.501 7f8262a5b700 1 --2- 172.21.2.101:0/34836107 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f825c1af2d0 0x7f825c1ae3c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:09.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.501 7f8262a5b700 1 --2- 172.21.2.101:0/34836107 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f825c18e420 0x7f825c18ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:09.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.501 7f8262a5b700 1 -- 172.21.2.101:0/34836107 >> 172.21.2.101:0/34836107 conn(0x7f825c0ed7f0 msgr2=0x7f825c094990 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:09.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.501 7f8262a5b700 1 -- 172.21.2.101:0/34836107 shutdown_connections 2021-02-10T11:13:09.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.501 7f8262a5b700 1 -- 172.21.2.101:0/34836107 wait complete. 2021-02-10T11:13:09.513 INFO:tasks.ceph:Setting up CephFS filesystem... 2021-02-10T11:13:09.513 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:13:09.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.919 7fa594410700 1 Processor -- start 2021-02-10T11:13:09.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.919 7fa594410700 1 -- start start 2021-02-10T11:13:09.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.919 7fa594410700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa58c0e5070 0x7fa58c1b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:09.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.919 7fa594410700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa58c0e5470 0x7fa58c080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:09.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.919 7fa594410700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa58c07ed80 con 0x7fa58c1b3e40 2021-02-10T11:13:09.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.919 7fa594410700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa58c0d1790 con 0x7fa58c0e5070 2021-02-10T11:13:09.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.919 7fa594410700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c0d19b0 con 0x7fa58c0e5470 2021-02-10T11:13:09.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.920 7fa591a2b700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa58c1b3e40 0x7fa58c07d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44296/0 (socket says 172.21.2.101:44296) 2021-02-10T11:13:09.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.920 7fa591a2b700 1 -- 172.21.2.101:0/2192902309 learned_addr learned my addr 172.21.2.101:0/2192902309 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:09.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.920 7fa59222c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa58c0e5070 0x7fa58c1b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:09.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.920 7fa59222c700 1 -- 172.21.2.101:0/2192902309 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa58c0e5470 msgr2=0x7fa58c080310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:13:09.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.920 7fa59222c700 1 --2- 172.21.2.101:0/2192902309 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa58c0e5470 0x7fa58c080310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:09.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.920 7fa59222c700 1 -- 172.21.2.101:0/2192902309 >> v1:172.21.2.101:6789/0 conn(0x7fa58c1b3e40 legacy=0x7fa58c07d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:09.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.920 7fa59222c700 1 -- 172.21.2.101:0/2192902309 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58c0d1bd0 con 0x7fa58c0e5070 2021-02-10T11:13:09.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.920 7fa59222c700 1 --2- 172.21.2.101:0/2192902309 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa58c0e5070 0x7fa58c1b3820 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fa588006650 tx=0x7fa588009a80).ready entity=mon.0 client_cookie=ee00a2181f24e934 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:09.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.920 7fa59122a700 1 -- 172.21.2.101:0/2192902309 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa58800c0c0 con 0x7fa58c0e5070 2021-02-10T11:13:09.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.920 7fa59122a700 1 -- 172.21.2.101:0/2192902309 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58800c340 con 0x7fa58c0e5070 2021-02-10T11:13:09.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.920 7fa59122a700 1 -- 172.21.2.101:0/2192902309 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa58800c860 con 0x7fa58c0e5070 2021-02-10T11:13:09.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.920 7fa594410700 1 -- 172.21.2.101:0/2192902309 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa58c0e5070 msgr2=0x7fa58c1b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:09.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.920 7fa594410700 1 --2- 172.21.2.101:0/2192902309 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa58c0e5070 0x7fa58c1b3820 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fa588006650 tx=0x7fa588009a80).stop 2021-02-10T11:13:09.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.921 7fa594410700 1 -- 172.21.2.101:0/2192902309 shutdown_connections 2021-02-10T11:13:09.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.921 7fa594410700 1 --2- 172.21.2.101:0/2192902309 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa58c0e5470 0x7fa58c080310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:09.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.921 7fa594410700 1 --2- 172.21.2.101:0/2192902309 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa58c0e5070 0x7fa58c1b3820 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:09.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.921 7fa594410700 1 -- 172.21.2.101:0/2192902309 >> 172.21.2.101:0/2192902309 conn(0x7fa58c0ce690 msgr2=0x7fa58c0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:09.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.921 7fa594410700 1 -- 172.21.2.101:0/2192902309 shutdown_connections 2021-02-10T11:13:09.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.921 7fa594410700 1 -- 172.21.2.101:0/2192902309 wait complete. 2021-02-10T11:13:09.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.936 7fa594410700 1 Processor -- start 2021-02-10T11:13:09.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.936 7fa594410700 1 -- start start 2021-02-10T11:13:09.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.936 7fa594410700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa58c1b19d0 0x7fa58c08cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:09.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.936 7fa594410700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa58c1b0d00 0x7fa58c1b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:09.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.936 7fa594410700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa58c1a2b50 0x7fa58c1a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:09.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.936 7fa594410700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c1b1720 con 0x7fa58c1a2b50 2021-02-10T11:13:09.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.936 7fa594410700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c08c470 con 0x7fa58c1b0d00 2021-02-10T11:13:09.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.936 7fa594410700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c1b1f50 con 0x7fa58c1b19d0 2021-02-10T11:13:09.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.937 7fa592a2d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa58c1a2b50 0x7fa58c1a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:09.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.937 7fa592a2d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa58c1a2b50 0x7fa58c1a2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43810/0 (socket says 172.21.2.101:43810) 2021-02-10T11:13:09.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.937 7fa592a2d700 1 -- 172.21.2.101:0/334369222 learned_addr learned my addr 172.21.2.101:0/334369222 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:09.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.937 7fa592a2d700 1 -- 172.21.2.101:0/334369222 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa58c1b19d0 msgr2=0x7fa58c08cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:09.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.937 7fa592a2d700 1 --2- 172.21.2.101:0/334369222 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa58c1b19d0 0x7fa58c08cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:09.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.937 7fa592a2d700 1 -- 172.21.2.101:0/334369222 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa58c1b0d00 msgr2=0x7fa58c1b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:09.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.937 7fa592a2d700 1 --2- 172.21.2.101:0/334369222 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa58c1b0d00 0x7fa58c1b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:09.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.937 7fa592a2d700 1 -- 172.21.2.101:0/334369222 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58c1a3570 con 0x7fa58c1a2b50 2021-02-10T11:13:09.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.937 7fa592a2d700 1 --2- 172.21.2.101:0/334369222 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa58c1a2b50 0x7fa58c1a2f50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fa584009960 tx=0x7fa58400bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:09.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.937 7fa57effd700 1 -- 172.21.2.101:0/334369222 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa584000d10 con 0x7fa58c1a2b50 2021-02-10T11:13:09.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.937 7fa57effd700 1 -- 172.21.2.101:0/334369222 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa584003030 con 0x7fa58c1a2b50 2021-02-10T11:13:09.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.937 7fa594410700 1 -- 172.21.2.101:0/334369222 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa58c1a3570 con 0x7fa58c1a2b50 2021-02-10T11:13:09.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.937 7fa57effd700 1 -- 172.21.2.101:0/334369222 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa584001590 con 0x7fa58c1a2b50 2021-02-10T11:13:09.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.937 7fa594410700 1 -- 172.21.2.101:0/334369222 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58c1a3570 con 0x7fa58c1a2b50 2021-02-10T11:13:09.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.938 7fa57effd700 1 -- 172.21.2.101:0/334369222 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa584000d10 con 0x7fa58c1a2b50 2021-02-10T11:13:09.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.938 7fa57effd700 1 --2- 172.21.2.101:0/334369222 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5740186d0 0x7fa57401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:09.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.938 7fa57effd700 1 -- 172.21.2.101:0/334369222 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fa5840326c0 con 0x7fa58c1a2b50 2021-02-10T11:13:09.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.938 7fa59222c700 1 --2- 172.21.2.101:0/334369222 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5740186d0 0x7fa57401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:09.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.938 7fa59222c700 1 --2- 172.21.2.101:0/334369222 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5740186d0 0x7fa57401abd0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:09.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.969 7fa594410700 1 -- 172.21.2.101:0/334369222 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa58c1b1720 con 0x7fa58c1a2b50 2021-02-10T11:13:09.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:09.972 7fa57effd700 1 -- 172.21.2.101:0/334369222 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa58400be30 con 0x7fa58c1a2b50 2021-02-10T11:13:10.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.096 7fa594410700 1 -- 172.21.2.101:0/334369222 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa58c1b1720 con 0x7fa58c1a2b50 2021-02-10T11:13:10.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.097 7fa57effd700 1 -- 172.21.2.101:0/334369222 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+11155 (secure 0 0 0) 0x7fa584003030 con 0x7fa58c1a2b50 2021-02-10T11:13:10.102 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:13:10.102 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":15,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:13:09.480289","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:13:10.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.099 7fa594410700 1 -- 172.21.2.101:0/334369222 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5740186d0 msgr2=0x7fa57401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:10.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.099 7fa594410700 1 --2- 172.21.2.101:0/334369222 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5740186d0 0x7fa57401abd0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:10.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.099 7fa594410700 1 -- 172.21.2.101:0/334369222 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa58c1a2b50 msgr2=0x7fa58c1a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:10.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.099 7fa594410700 1 --2- 172.21.2.101:0/334369222 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa58c1a2b50 0x7fa58c1a2f50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fa584009960 tx=0x7fa58400bcf0).stop 2021-02-10T11:13:10.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.099 7fa594410700 1 -- 172.21.2.101:0/334369222 shutdown_connections 2021-02-10T11:13:10.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.099 7fa594410700 1 --2- 172.21.2.101:0/334369222 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa58c1a2b50 0x7fa58c1a2f50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:10.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.099 7fa594410700 1 --2- 172.21.2.101:0/334369222 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa58c1b0d00 0x7fa58c1b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:10.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.099 7fa594410700 1 --2- 172.21.2.101:0/334369222 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5740186d0 0x7fa57401abd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:10.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.099 7fa594410700 1 --2- 172.21.2.101:0/334369222 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa58c1b19d0 0x7fa58c08cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:10.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.099 7fa594410700 1 -- 172.21.2.101:0/334369222 >> 172.21.2.101:0/334369222 conn(0x7fa58c093720 msgr2=0x7fa58c084b80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:10.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.099 7fa594410700 1 -- 172.21.2.101:0/334369222 shutdown_connections 2021-02-10T11:13:10.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.099 7fa594410700 1 -- 172.21.2.101:0/334369222 wait complete. 2021-02-10T11:13:10.112 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-10T11:13:10.112 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:13:10.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.523 7f9465660700 1 Processor -- start 2021-02-10T11:13:10.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.523 7f9465660700 1 -- start start 2021-02-10T11:13:10.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.523 7f9465660700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f94601b0470 0x7f94601b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:10.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.523 7f9465660700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94600f0b50 con 0x7f94600ef610 2021-02-10T11:13:10.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.523 7f9465660700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94600f24e0 con 0x7f94600f20e0 2021-02-10T11:13:10.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.523 7f9465660700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94600e5ea0 con 0x7f94601b0470 2021-02-10T11:13:10.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.523 7f945eabe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f94600ef610 0x7f94600efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44308/0 (socket says 172.21.2.101:44308) 2021-02-10T11:13:10.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.523 7f945eabe700 1 -- 172.21.2.101:0/955100886 learned_addr learned my addr 172.21.2.101:0/955100886 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:10.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.523 7f945f2bf700 1 --2- 172.21.2.101:0/955100886 >> v2:172.21.2.108:3300/0 conn(0x7f94601b0470 0x7f94601b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:10.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.524 7f945e2bd700 1 -- 172.21.2.101:0/955100886 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763387400 0 0) 0x7f94500010b0 con 0x7f94600ef610 2021-02-10T11:13:10.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.524 7f945e2bd700 1 -- 172.21.2.101:0/955100886 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f944c002fe0 con 0x7f94600ef610 2021-02-10T11:13:10.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.524 7f945e2bd700 1 -- 172.21.2.101:0/955100886 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 761548190 0 0) 0x7f9458001400 con 0x7f94600f20e0 2021-02-10T11:13:10.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.524 7f945e2bd700 1 -- 172.21.2.101:0/955100886 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f944c003210 con 0x7f94600f20e0 2021-02-10T11:13:10.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.524 7f945f2bf700 1 -- 172.21.2.101:0/955100886 >> v1:172.21.2.108:6789/0 conn(0x7f94600f20e0 legacy=0x7f94600f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:10.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.524 7f945f2bf700 1 -- 172.21.2.101:0/955100886 >> v1:172.21.2.101:6789/0 conn(0x7f94600ef610 legacy=0x7f94600efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:10.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.524 7f945f2bf700 1 -- 172.21.2.101:0/955100886 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94600e52e0 con 0x7f94601b0470 2021-02-10T11:13:10.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.524 7f945f2bf700 1 --2- 172.21.2.101:0/955100886 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94601b0470 0x7f94601b0870 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f9448006650 tx=0x7f9448009a80).ready entity=mon.1 client_cookie=cdaf915a4c24b722 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:10.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.524 7f945e2bd700 1 -- 172.21.2.101:0/955100886 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f944800c0c0 con 0x7f94601b0470 2021-02-10T11:13:10.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.524 7f945e2bd700 1 -- 172.21.2.101:0/955100886 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944800c340 con 0x7f94601b0470 2021-02-10T11:13:10.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.524 7f945e2bd700 1 -- 172.21.2.101:0/955100886 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f944800c820 con 0x7f94601b0470 2021-02-10T11:13:10.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.524 7f9465660700 1 -- 172.21.2.101:0/955100886 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94601b0470 msgr2=0x7f94601b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:10.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.524 7f9465660700 1 --2- 172.21.2.101:0/955100886 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94601b0470 0x7f94601b0870 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f9448006650 tx=0x7f9448009a80).stop 2021-02-10T11:13:10.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.525 7f9465660700 1 -- 172.21.2.101:0/955100886 shutdown_connections 2021-02-10T11:13:10.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.525 7f9465660700 1 --2- 172.21.2.101:0/955100886 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94601b0470 0x7f94601b0870 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:10.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.525 7f9465660700 1 -- 172.21.2.101:0/955100886 >> 172.21.2.101:0/955100886 conn(0x7f94600ce3e0 msgr2=0x7f94600e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:10.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.525 7f9465660700 1 -- 172.21.2.101:0/955100886 shutdown_connections 2021-02-10T11:13:10.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.525 7f9465660700 1 -- 172.21.2.101:0/955100886 wait complete. 2021-02-10T11:13:10.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.540 7f9465660700 1 Processor -- start 2021-02-10T11:13:10.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.540 7f9465660700 1 -- start start 2021-02-10T11:13:10.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.540 7f9465660700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94600a0420 0x7f94600a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:10.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.540 7f9465660700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9460189d00 0x7f946018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:10.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.540 7f9465660700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f946018a720 0x7f94600b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:10.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.540 7f9465660700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9460189780 con 0x7f946018a720 2021-02-10T11:13:10.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.540 7f9465660700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946018ab20 con 0x7f9460189d00 2021-02-10T11:13:10.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.540 7f9465660700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94600a1140 con 0x7f94600a0420 2021-02-10T11:13:10.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.540 7f945fac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f946018a720 0x7f94600b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:10.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.540 7f945f2bf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94600a0420 0x7f94600a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:10.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.540 7f945fac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f946018a720 0x7f94600b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43820/0 (socket says 172.21.2.101:43820) 2021-02-10T11:13:10.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.541 7f945fac0700 1 -- 172.21.2.101:0/249789271 learned_addr learned my addr 172.21.2.101:0/249789271 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:10.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.541 7f945eabe700 1 --2- 172.21.2.101:0/249789271 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9460189d00 0x7f946018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:10.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.541 7f945fac0700 1 -- 172.21.2.101:0/249789271 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94600a0420 msgr2=0x7f94600a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:10.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.541 7f945fac0700 1 --2- 172.21.2.101:0/249789271 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94600a0420 0x7f94600a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:10.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.541 7f945fac0700 1 -- 172.21.2.101:0/249789271 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9460189d00 msgr2=0x7f946018a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:10.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.541 7f945fac0700 1 --2- 172.21.2.101:0/249789271 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9460189d00 0x7f946018a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:10.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.541 7f945fac0700 1 -- 172.21.2.101:0/249789271 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94600b8200 con 0x7f946018a720 2021-02-10T11:13:10.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.541 7f945f2bf700 1 --2- 172.21.2.101:0/249789271 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94600a0420 0x7f94600a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:13:10.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.541 7f945fac0700 1 --2- 172.21.2.101:0/249789271 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f946018a720 0x7f94600b7be0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f9458009940 tx=0x7f945800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:10.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.541 7f9457fff700 1 -- 172.21.2.101:0/249789271 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9458000f40 con 0x7f946018a720 2021-02-10T11:13:10.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.541 7f9457fff700 1 -- 172.21.2.101:0/249789271 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945800c2f0 con 0x7f946018a720 2021-02-10T11:13:10.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.541 7f9457fff700 1 -- 172.21.2.101:0/249789271 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f945800c810 con 0x7f946018a720 2021-02-10T11:13:10.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.541 7f9465660700 1 -- 172.21.2.101:0/249789271 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94600b8200 con 0x7f946018a720 2021-02-10T11:13:10.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.541 7f9465660700 1 -- 172.21.2.101:0/249789271 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94600b8790 con 0x7f946018a720 2021-02-10T11:13:10.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.542 7f9457fff700 1 -- 172.21.2.101:0/249789271 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9458001250 con 0x7f946018a720 2021-02-10T11:13:10.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.542 7f9457fff700 1 --2- 172.21.2.101:0/249789271 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94440186c0 0x7f944401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:10.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.542 7f9457fff700 1 -- 172.21.2.101:0/249789271 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f94580325f0 con 0x7f946018a720 2021-02-10T11:13:10.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.542 7f945f2bf700 1 --2- 172.21.2.101:0/249789271 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94440186c0 0x7f944401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:10.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.542 7f945f2bf700 1 --2- 172.21.2.101:0/249789271 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94440186c0 0x7f944401abc0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:10.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.573 7f9465660700 1 -- 172.21.2.101:0/249789271 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9460105ae0 con 0x7f946018a720 2021-02-10T11:13:10.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.575 7f9457fff700 1 -- 172.21.2.101:0/249789271 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9458003080 con 0x7f946018a720 2021-02-10T11:13:10.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.703 7f9465660700 1 -- 172.21.2.101:0/249789271 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f94600f0b50 con 0x7f946018a720 2021-02-10T11:13:10.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.704 7f9457fff700 1 -- 172.21.2.101:0/249789271 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9458001250 con 0x7f946018a720 2021-02-10T11:13:10.708 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:13:10.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.705 7f9465660700 1 -- 172.21.2.101:0/249789271 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94440186c0 msgr2=0x7f944401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:10.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.705 7f9465660700 1 --2- 172.21.2.101:0/249789271 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94440186c0 0x7f944401abc0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:10.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.705 7f9465660700 1 -- 172.21.2.101:0/249789271 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f946018a720 msgr2=0x7f94600b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:10.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.706 7f9465660700 1 --2- 172.21.2.101:0/249789271 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f946018a720 0x7f94600b7be0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f9458009940 tx=0x7f945800bcf0).stop 2021-02-10T11:13:10.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.706 7f9465660700 1 -- 172.21.2.101:0/249789271 shutdown_connections 2021-02-10T11:13:10.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.706 7f9465660700 1 --2- 172.21.2.101:0/249789271 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f946018a720 0x7f94600b7be0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:10.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.706 7f9465660700 1 --2- 172.21.2.101:0/249789271 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9460189d00 0x7f946018a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:10.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.706 7f9465660700 1 --2- 172.21.2.101:0/249789271 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94440186c0 0x7f944401abc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:10.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.706 7f9465660700 1 --2- 172.21.2.101:0/249789271 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94600a0420 0x7f94600a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:10.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.706 7f9465660700 1 -- 172.21.2.101:0/249789271 >> 172.21.2.101:0/249789271 conn(0x7f9460094e10 msgr2=0x7f94600a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:10.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.706 7f9465660700 1 -- 172.21.2.101:0/249789271 shutdown_connections 2021-02-10T11:13:10.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:10.706 7f9465660700 1 -- 172.21.2.101:0/249789271 wait complete. 2021-02-10T11:13:10.719 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --format=json-pretty osd lspools 2021-02-10T11:13:11.026 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:13:11.030 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:13:11.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.123 7f8320d64700 1 Processor -- start 2021-02-10T11:13:11.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.123 7f8320d64700 1 -- start start 2021-02-10T11:13:11.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.124 7f8320d64700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f831c0e5470 0x7f831c07e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:11.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.124 7f8320d64700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f831c07f300 0x7f831c07f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.124 7f8320d64700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f831c07ed80 con 0x7f831c0e5070 2021-02-10T11:13:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.124 7f8320d64700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f831c1b4010 con 0x7f831c0e5470 2021-02-10T11:13:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.124 7f8320d64700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f831c0d1790 con 0x7f831c07f300 2021-02-10T11:13:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.124 7f831a312700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f831c0e5470 0x7f831c07e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.124 7f831a312700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f831c0e5470 0x7f831c07e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43826/0 (socket says 172.21.2.101:43826) 2021-02-10T11:13:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.124 7f831a312700 1 -- 172.21.2.101:0/2257149964 learned_addr learned my addr 172.21.2.101:0/2257149964 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.124 7f831a312700 1 -- 172.21.2.101:0/2257149964 >> v2:172.21.2.108:3300/0 conn(0x7f831c07f300 msgr2=0x7f831c07f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.124 7f831a312700 1 --2- 172.21.2.101:0/2257149964 >> v2:172.21.2.108:3300/0 conn(0x7f831c07f300 0x7f831c07f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.124 7f831a312700 1 -- 172.21.2.101:0/2257149964 >> v1:172.21.2.108:6789/0 conn(0x7f831c0e5070 legacy=0x7f831c1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.124 7f831a312700 1 -- 172.21.2.101:0/2257149964 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c0d19b0 con 0x7f831c0e5470 2021-02-10T11:13:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.125 7f831a312700 1 --2- 172.21.2.101:0/2257149964 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0e5470 0x7f831c07e6a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f830c006700 tx=0x7f830c009b80).ready entity=mon.0 client_cookie=4cb44456eb159b6d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.125 7f8319b11700 1 -- 172.21.2.101:0/2257149964 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f830c00c0c0 con 0x7f831c0e5470 2021-02-10T11:13:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.125 7f8319b11700 1 -- 172.21.2.101:0/2257149964 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830c00c380 con 0x7f831c0e5470 2021-02-10T11:13:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.125 7f8319b11700 1 -- 172.21.2.101:0/2257149964 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f830c00c8a0 con 0x7f831c0e5470 2021-02-10T11:13:11.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.125 7f8320d64700 1 -- 172.21.2.101:0/2257149964 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0e5470 msgr2=0x7f831c07e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:11.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.125 7f8320d64700 1 --2- 172.21.2.101:0/2257149964 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0e5470 0x7f831c07e6a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f830c006700 tx=0x7f830c009b80).stop 2021-02-10T11:13:11.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.126 7f8320d64700 1 -- 172.21.2.101:0/2257149964 shutdown_connections 2021-02-10T11:13:11.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.126 7f8320d64700 1 --2- 172.21.2.101:0/2257149964 >> v2:172.21.2.108:3300/0 conn(0x7f831c07f300 0x7f831c07f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:11.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.126 7f8320d64700 1 --2- 172.21.2.101:0/2257149964 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0e5470 0x7f831c07e6a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:11.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.126 7f8320d64700 1 -- 172.21.2.101:0/2257149964 >> 172.21.2.101:0/2257149964 conn(0x7f831c0ce690 msgr2=0x7f831c0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:11.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.126 7f8320d64700 1 -- 172.21.2.101:0/2257149964 shutdown_connections 2021-02-10T11:13:11.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.126 7f8320d64700 1 -- 172.21.2.101:0/2257149964 wait complete. 2021-02-10T11:13:11.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.141 7f8320d64700 1 Processor -- start 2021-02-10T11:13:11.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.141 7f8320d64700 1 -- start start 2021-02-10T11:13:11.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.141 7f8320d64700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f831c1a2a90 0x7f831c1aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:11.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.141 7f8320d64700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f831c1aa700 0x7f831c1a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:11.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.141 7f8320d64700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0a55e0 0x7f831c1a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:11.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.141 7f8320d64700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831c1a34a0 con 0x7f831c0a55e0 2021-02-10T11:13:11.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.141 7f8320d64700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831c1a2e90 con 0x7f831c1a2a90 2021-02-10T11:13:11.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.141 7f8320d64700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831c1a30b0 con 0x7f831c1aa700 2021-02-10T11:13:11.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.141 7f831b314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0a55e0 0x7f831c1a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:11.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.141 7f831b314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0a55e0 0x7f831c1a4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43834/0 (socket says 172.21.2.101:43834) 2021-02-10T11:13:11.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.141 7f831b314700 1 -- 172.21.2.101:0/1028793080 learned_addr learned my addr 172.21.2.101:0/1028793080 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:11.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.141 7f831b314700 1 -- 172.21.2.101:0/1028793080 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f831c1aa700 msgr2=0x7f831c1a3a20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:11.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.141 7f831b314700 1 --2- 172.21.2.101:0/1028793080 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f831c1aa700 0x7f831c1a3a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:11.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.141 7f831b314700 1 -- 172.21.2.101:0/1028793080 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f831c1a2a90 msgr2=0x7f831c1aa0e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:11.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.141 7f831b314700 1 --2- 172.21.2.101:0/1028793080 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f831c1a2a90 0x7f831c1aa0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:11.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.141 7f831b314700 1 -- 172.21.2.101:0/1028793080 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c1a4660 con 0x7f831c0a55e0 2021-02-10T11:13:11.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.142 7f831b314700 1 --2- 172.21.2.101:0/1028793080 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0a55e0 0x7f831c1a4040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f8314009980 tx=0x7f831400bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:11.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.142 7f83137fe700 1 -- 172.21.2.101:0/1028793080 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8314005870 con 0x7f831c0a55e0 2021-02-10T11:13:11.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.142 7f83137fe700 1 -- 172.21.2.101:0/1028793080 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8314005af0 con 0x7f831c0a55e0 2021-02-10T11:13:11.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.142 7f83137fe700 1 -- 172.21.2.101:0/1028793080 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8314006030 con 0x7f831c0a55e0 2021-02-10T11:13:11.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.142 7f8320d64700 1 -- 172.21.2.101:0/1028793080 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f831c1a4660 con 0x7f831c0a55e0 2021-02-10T11:13:11.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.142 7f8320d64700 1 -- 172.21.2.101:0/1028793080 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f831c1a4bf0 con 0x7f831c0a55e0 2021-02-10T11:13:11.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.142 7f83137fe700 1 -- 172.21.2.101:0/1028793080 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8314006030 con 0x7f831c0a55e0 2021-02-10T11:13:11.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.143 7f83137fe700 1 --2- 172.21.2.101:0/1028793080 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83000186c0 0x7f830001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:11.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.143 7f83137fe700 1 -- 172.21.2.101:0/1028793080 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f83140325e0 con 0x7f831c0a55e0 2021-02-10T11:13:11.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.143 7f831ab13700 1 --2- 172.21.2.101:0/1028793080 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83000186c0 0x7f830001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:11.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.143 7f831ab13700 1 --2- 172.21.2.101:0/1028793080 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83000186c0 0x7f830001abc0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:11.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.174 7f8320d64700 1 -- 172.21.2.101:0/1028793080 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f831c083da0 con 0x7f831c0a55e0 2021-02-10T11:13:11.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.177 7f83137fe700 1 -- 172.21.2.101:0/1028793080 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8314005cd0 con 0x7f831c0a55e0 2021-02-10T11:13:11.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.302 7f8320d64700 1 -- 172.21.2.101:0/1028793080 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f831c07ed80 con 0x7f831c0a55e0 2021-02-10T11:13:11.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.302 7f83137fe700 1 -- 172.21.2.101:0/1028793080 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+64 (secure 0 0 0) 0x7f8314006f20 con 0x7f831c0a55e0 2021-02-10T11:13:11.307 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:13:11.307 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:13:11.307 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:11.307 INFO:teuthology.orchestra.run.gibba001.stdout: "poolnum": 1, 2021-02-10T11:13:11.307 INFO:teuthology.orchestra.run.gibba001.stdout: "poolname": "rbd" 2021-02-10T11:13:11.307 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:11.307 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:13:11.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.304 7f8320d64700 1 -- 172.21.2.101:0/1028793080 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83000186c0 msgr2=0x7f830001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:11.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.304 7f8320d64700 1 --2- 172.21.2.101:0/1028793080 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83000186c0 0x7f830001abc0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:11.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.304 7f8320d64700 1 -- 172.21.2.101:0/1028793080 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0a55e0 msgr2=0x7f831c1a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:11.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.304 7f8320d64700 1 --2- 172.21.2.101:0/1028793080 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0a55e0 0x7f831c1a4040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f8314009980 tx=0x7f831400bcd0).stop 2021-02-10T11:13:11.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.304 7f8320d64700 1 -- 172.21.2.101:0/1028793080 shutdown_connections 2021-02-10T11:13:11.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.304 7f8320d64700 1 --2- 172.21.2.101:0/1028793080 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83000186c0 0x7f830001abc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:11.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.304 7f8320d64700 1 --2- 172.21.2.101:0/1028793080 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f831c1aa700 0x7f831c1a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:11.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.304 7f8320d64700 1 --2- 172.21.2.101:0/1028793080 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0a55e0 0x7f831c1a4040 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:11.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.305 7f8320d64700 1 --2- 172.21.2.101:0/1028793080 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f831c1a2a90 0x7f831c1aa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:11.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.305 7f8320d64700 1 -- 172.21.2.101:0/1028793080 >> 172.21.2.101:0/1028793080 conn(0x7f831c08fe10 msgr2=0x7f831c19c020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:11.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.305 7f8320d64700 1 -- 172.21.2.101:0/1028793080 shutdown_connections 2021-02-10T11:13:11.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.305 7f8320d64700 1 -- 172.21.2.101:0/1028793080 wait complete. 2021-02-10T11:13:11.317 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-02-10T11:13:11.317 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-02-10T11:13:11.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.726 7f93c96dd700 1 Processor -- start 2021-02-10T11:13:11.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.726 7f93c96dd700 1 -- start start 2021-02-10T11:13:11.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.726 7f93c96dd700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f93c41b0470 0x7f93c41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:11.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.726 7f93c96dd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93c40ef610 0x7f93c40efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:11.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.726 7f93c96dd700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c40f0b50 con 0x7f93c40f0030 2021-02-10T11:13:11.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.726 7f93c96dd700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c40e52e0 con 0x7f93c41b0470 2021-02-10T11:13:11.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.726 7f93c96dd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40e54d0 con 0x7f93c40ef610 2021-02-10T11:13:11.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.727 7f93c3ac0700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f93c40f0030 0x7f93c40f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44332/0 (socket says 172.21.2.101:44332) 2021-02-10T11:13:11.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.727 7f93c3ac0700 1 -- 172.21.2.101:0/2325974695 learned_addr learned my addr 172.21.2.101:0/2325974695 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:11.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.727 7f93c22bd700 1 -- 172.21.2.101:0/2325974695 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 689486219 0 0) 0x7f93bc001500 con 0x7f93c40f0030 2021-02-10T11:13:11.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.727 7f93c22bd700 1 -- 172.21.2.101:0/2325974695 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93b0002fe0 con 0x7f93c40f0030 2021-02-10T11:13:11.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.727 7f93c32bf700 1 --2- 172.21.2.101:0/2325974695 >> v2:172.21.2.108:3300/0 conn(0x7f93c41b0470 0x7f93c41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:11.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.727 7f93c2abe700 1 --2- 172.21.2.101:0/2325974695 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93c40ef610 0x7f93c40efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:11.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.727 7f93c22bd700 1 -- 172.21.2.101:0/2325974695 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 63649573 0 0) 0x7f93bc001460 con 0x7f93c40f0030 2021-02-10T11:13:11.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.727 7f93c22bd700 1 -- 172.21.2.101:0/2325974695 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93c40ef610 msgr2=0x7f93c40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:11.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.727 7f93c22bd700 1 --2- 172.21.2.101:0/2325974695 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93c40ef610 0x7f93c40efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:11.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.727 7f93c22bd700 1 -- 172.21.2.101:0/2325974695 >> v2:172.21.2.108:3300/0 conn(0x7f93c41b0470 msgr2=0x7f93c41b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:11.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.727 7f93c22bd700 1 --2- 172.21.2.101:0/2325974695 >> v2:172.21.2.108:3300/0 conn(0x7f93c41b0470 0x7f93c41b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:11.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.727 7f93c22bd700 1 -- 172.21.2.101:0/2325974695 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c40f0430 con 0x7f93c40f0030 2021-02-10T11:13:11.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.727 7f93c22bd700 1 -- 172.21.2.101:0/2325974695 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f93bc001980 con 0x7f93c40f0030 2021-02-10T11:13:11.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.727 7f93c22bd700 1 -- 172.21.2.101:0/2325974695 >> v1:172.21.2.101:6789/0 conn(0x7f93c40f0030 legacy=0x7f93c40f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:11.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.728 7f93c22bd700 1 --2- 172.21.2.101:0/2325974695 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93b0003a40 0x7f93b0005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:11.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.728 7f93c22bd700 1 --2- 172.21.2.101:0/2325974695 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93b0006790 0x7f93b0008c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:11.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.728 7f93c22bd700 1 --2- 172.21.2.101:0/2325974695 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93b0009500 0x7f93b000ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:11.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.728 7f93c22bd700 1 -- 172.21.2.101:0/2325974695 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b000c2d0 con 0x7f93b0003a40 2021-02-10T11:13:11.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.728 7f93c22bd700 1 -- 172.21.2.101:0/2325974695 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b00504c0 con 0x7f93b0009500 2021-02-10T11:13:11.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.728 7f93c22bd700 1 -- 172.21.2.101:0/2325974695 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b00506e0 con 0x7f93b0006790 2021-02-10T11:13:11.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.728 7f93c32bf700 1 --2- 172.21.2.101:0/2325974695 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93b0003a40 0x7f93b0005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:11.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.728 7f93c3ac0700 1 --2- 172.21.2.101:0/2325974695 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93b0009500 0x7f93b000ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:11.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.728 7f93c2abe700 1 --2- 172.21.2.101:0/2325974695 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93b0006790 0x7f93b0008c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:11.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.728 7f93c32bf700 1 -- 172.21.2.101:0/2325974695 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93b0006790 msgr2=0x7f93b0008c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:11.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.728 7f93c32bf700 1 --2- 172.21.2.101:0/2325974695 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93b0006790 0x7f93b0008c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:11.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.728 7f93c32bf700 1 -- 172.21.2.101:0/2325974695 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93b0009500 msgr2=0x7f93b000ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:11.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.728 7f93c32bf700 1 --2- 172.21.2.101:0/2325974695 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93b0009500 0x7f93b000ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:11.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.728 7f93c32bf700 1 -- 172.21.2.101:0/2325974695 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b0050980 con 0x7f93b0003a40 2021-02-10T11:13:11.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.728 7f93c32bf700 1 --2- 172.21.2.101:0/2325974695 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93b0003a40 0x7f93b0005ef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f93ac006360 tx=0x7f93ac009c30).ready entity=mon.0 client_cookie=73643d39a64fddba server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:11.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.728 7f93c22bd700 1 -- 172.21.2.101:0/2325974695 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f93ac0009b0 con 0x7f93b0003a40 2021-02-10T11:13:11.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.728 7f93c22bd700 1 -- 172.21.2.101:0/2325974695 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93ac000c90 con 0x7f93b0003a40 2021-02-10T11:13:11.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.728 7f93c22bd700 1 -- 172.21.2.101:0/2325974695 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f93ac002220 con 0x7f93b0003a40 2021-02-10T11:13:11.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.729 7f93c96dd700 1 -- 172.21.2.101:0/2325974695 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93b0003a40 msgr2=0x7f93b0005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:11.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.729 7f93c96dd700 1 --2- 172.21.2.101:0/2325974695 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93b0003a40 0x7f93b0005ef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f93ac006360 tx=0x7f93ac009c30).stop 2021-02-10T11:13:11.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.729 7f93c3ac0700 1 -- 172.21.2.101:0/2325974695 reap_dead start 2021-02-10T11:13:11.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.729 7f93c96dd700 1 -- 172.21.2.101:0/2325974695 shutdown_connections 2021-02-10T11:13:11.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.729 7f93c96dd700 1 -- 172.21.2.101:0/2325974695 >> 172.21.2.101:0/2325974695 conn(0x7f93c40ce3e0 msgr2=0x7f93c40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:11.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.729 7f93c96dd700 1 -- 172.21.2.101:0/2325974695 shutdown_connections 2021-02-10T11:13:11.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.729 7f93c96dd700 1 -- 172.21.2.101:0/2325974695 wait complete. 2021-02-10T11:13:11.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.744 7f93c96dd700 1 Processor -- start 2021-02-10T11:13:11.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.744 7f93c96dd700 1 -- start start 2021-02-10T11:13:11.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.744 7f93c96dd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93c409b930 0x7f93c40b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:11.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.744 7f93c96dd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93c40b8410 0x7f93c40b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:11.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.744 7f93c96dd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93c40c1f80 0x7f93c40c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:11.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.744 7f93c96dd700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40c1a00 con 0x7f93c40b8410 2021-02-10T11:13:11.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.744 7f93c96dd700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40b8e30 con 0x7f93c40c1f80 2021-02-10T11:13:11.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.744 7f93c96dd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40b9050 con 0x7f93c409b930 2021-02-10T11:13:11.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.745 7f93c2abe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93c40b8410 0x7f93c40b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:11.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.745 7f93c2abe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93c40b8410 0x7f93c40b8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43852/0 (socket says 172.21.2.101:43852) 2021-02-10T11:13:11.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.745 7f93c2abe700 1 -- 172.21.2.101:0/1071356642 learned_addr learned my addr 172.21.2.101:0/1071356642 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:11.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.745 7f93c32bf700 1 --2- 172.21.2.101:0/1071356642 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93c409b930 0x7f93c40b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:11.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.745 7f93c3ac0700 1 --2- 172.21.2.101:0/1071356642 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93c40c1f80 0x7f93c40c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:11.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.745 7f93c2abe700 1 -- 172.21.2.101:0/1071356642 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93c409b930 msgr2=0x7f93c40b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:11.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.745 7f93c2abe700 1 --2- 172.21.2.101:0/1071356642 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93c409b930 0x7f93c40b7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:11.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.745 7f93c2abe700 1 -- 172.21.2.101:0/1071356642 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93c40c1f80 msgr2=0x7f93c40c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:11.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.745 7f93c2abe700 1 --2- 172.21.2.101:0/1071356642 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93c40c1f80 0x7f93c40c2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:11.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.745 7f93c2abe700 1 -- 172.21.2.101:0/1071356642 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c40c29a0 con 0x7f93c40b8410 2021-02-10T11:13:11.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.745 7f93c2abe700 1 --2- 172.21.2.101:0/1071356642 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93c40b8410 0x7f93c40b8810 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f93b400b5e0 tx=0x7f93b400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:11.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.746 7f93bbfff700 1 -- 172.21.2.101:0/1071356642 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f93b400bd70 con 0x7f93c40b8410 2021-02-10T11:13:11.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.746 7f93bbfff700 1 -- 172.21.2.101:0/1071356642 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b4002030 con 0x7f93c40b8410 2021-02-10T11:13:11.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.746 7f93bbfff700 1 -- 172.21.2.101:0/1071356642 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f93b4002550 con 0x7f93c40b8410 2021-02-10T11:13:11.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.746 7f93c96dd700 1 -- 172.21.2.101:0/1071356642 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c40c29a0 con 0x7f93c40b8410 2021-02-10T11:13:11.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.746 7f93c96dd700 1 -- 172.21.2.101:0/1071356642 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c40c2f30 con 0x7f93c40b8410 2021-02-10T11:13:11.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.746 7f93bbfff700 1 -- 172.21.2.101:0/1071356642 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f93b4000c90 con 0x7f93c40b8410 2021-02-10T11:13:11.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.746 7f93bbfff700 1 --2- 172.21.2.101:0/1071356642 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93a80186c0 0x7f93a801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:11.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.746 7f93bbfff700 1 -- 172.21.2.101:0/1071356642 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f93b4032240 con 0x7f93c40b8410 2021-02-10T11:13:11.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.746 7f93c32bf700 1 --2- 172.21.2.101:0/1071356642 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93a80186c0 0x7f93a801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:11.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.747 7f93c32bf700 1 --2- 172.21.2.101:0/1071356642 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93a80186c0 0x7f93a801abc0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:11.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.776 7f93c96dd700 1 -- 172.21.2.101:0/1071356642 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c4105b30 con 0x7f93c40b8410 2021-02-10T11:13:11.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.779 7f93bbfff700 1 -- 172.21.2.101:0/1071356642 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f93b4000940 con 0x7f93c40b8410 2021-02-10T11:13:11.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:11.905 7f93c96dd700 1 -- 172.21.2.101:0/1071356642 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"} v 0) v1 -- 0x7f93c40f0b50 con 0x7f93c40b8410 2021-02-10T11:13:12.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.498 7f93bbfff700 1 -- 172.21.2.101:0/1071356642 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"}]=0 pool 'cephfs_metadata' created v17) v1 ==== 133+0+0 (secure 0 0 0) 0x7f93b4000c90 con 0x7f93c40b8410 2021-02-10T11:13:12.526 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' created 2021-02-10T11:13:12.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.524 7f93c96dd700 1 -- 172.21.2.101:0/1071356642 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93a80186c0 msgr2=0x7f93a801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:12.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.524 7f93c96dd700 1 --2- 172.21.2.101:0/1071356642 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93a80186c0 0x7f93a801abc0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:12.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.524 7f93c96dd700 1 -- 172.21.2.101:0/1071356642 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93c40b8410 msgr2=0x7f93c40b8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:12.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.524 7f93c96dd700 1 --2- 172.21.2.101:0/1071356642 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93c40b8410 0x7f93c40b8810 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f93b400b5e0 tx=0x7f93b400b980).stop 2021-02-10T11:13:12.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.524 7f93c96dd700 1 -- 172.21.2.101:0/1071356642 shutdown_connections 2021-02-10T11:13:12.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.524 7f93c96dd700 1 --2- 172.21.2.101:0/1071356642 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93c40c1f80 0x7f93c40c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:12.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.524 7f93c96dd700 1 --2- 172.21.2.101:0/1071356642 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93c40b8410 0x7f93c40b8810 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:12.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.524 7f93c96dd700 1 --2- 172.21.2.101:0/1071356642 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93a80186c0 0x7f93a801abc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:12.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.524 7f93c96dd700 1 --2- 172.21.2.101:0/1071356642 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93c409b930 0x7f93c40b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:12.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.524 7f93c96dd700 1 -- 172.21.2.101:0/1071356642 >> 172.21.2.101:0/1071356642 conn(0x7f93c4094e10 msgr2=0x7f93c40a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:12.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.524 7f93c96dd700 1 -- 172.21.2.101:0/1071356642 shutdown_connections 2021-02-10T11:13:12.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.525 7f93c96dd700 1 -- 172.21.2.101:0/1071356642 wait complete. 2021-02-10T11:13:12.537 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_data 8 2021-02-10T11:13:12.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.945 7fb2f48f1700 1 Processor -- start 2021-02-10T11:13:12.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.945 7fb2f48f1700 1 -- start start 2021-02-10T11:13:12.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.946 7fb2f48f1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb2f01b0470 0x7fb2f01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:12.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.946 7fb2f48f1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2f00f20e0 0x7fb2f00f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:12.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.946 7fb2f48f1700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f00f0b50 con 0x7fb2f00ef610 2021-02-10T11:13:12.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.946 7fb2f48f1700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f00f0820 con 0x7fb2f01b0470 2021-02-10T11:13:12.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.946 7fb2f48f1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f00f24e0 con 0x7fb2f00f20e0 2021-02-10T11:13:12.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.946 7fb2ee61c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb2f01b0470 0x7fb2f01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:12.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.946 7fb2ede1b700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb2f00ef610 0x7fb2f00efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44350/0 (socket says 172.21.2.101:44350) 2021-02-10T11:13:12.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.946 7fb2ede1b700 1 -- 172.21.2.101:0/3747290233 learned_addr learned my addr 172.21.2.101:0/3747290233 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:12.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.946 7fb2ee61c700 1 -- 172.21.2.101:0/3747290233 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2f00f20e0 msgr2=0x7fb2f00f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:12.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.946 7fb2ee61c700 1 --2- 172.21.2.101:0/3747290233 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2f00f20e0 0x7fb2f00f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:12.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.946 7fb2ee61c700 1 -- 172.21.2.101:0/3747290233 >> v1:172.21.2.101:6789/0 conn(0x7fb2f00ef610 legacy=0x7fb2f00efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:12.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.946 7fb2ee61c700 1 -- 172.21.2.101:0/3747290233 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f00e52e0 con 0x7fb2f01b0470 2021-02-10T11:13:12.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.946 7fb2ee61c700 1 --2- 172.21.2.101:0/3747290233 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2f01b0470 0x7fb2f01b0870 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fb2d8006650 tx=0x7fb2d8009a80).ready entity=mon.0 client_cookie=eb752420815cd68d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:12.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.947 7fb2ed61a700 1 -- 172.21.2.101:0/3747290233 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb2d800c0c0 con 0x7fb2f01b0470 2021-02-10T11:13:12.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.947 7fb2ed61a700 1 -- 172.21.2.101:0/3747290233 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d800c0c0 con 0x7fb2f01b0470 2021-02-10T11:13:12.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.947 7fb2ed61a700 1 -- 172.21.2.101:0/3747290233 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb2d800c340 con 0x7fb2f01b0470 2021-02-10T11:13:12.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.947 7fb2f48f1700 1 -- 172.21.2.101:0/3747290233 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2f01b0470 msgr2=0x7fb2f01b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:12.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.947 7fb2f48f1700 1 --2- 172.21.2.101:0/3747290233 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2f01b0470 0x7fb2f01b0870 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fb2d8006650 tx=0x7fb2d8009a80).stop 2021-02-10T11:13:12.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.947 7fb2f48f1700 1 -- 172.21.2.101:0/3747290233 shutdown_connections 2021-02-10T11:13:12.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.947 7fb2f48f1700 1 --2- 172.21.2.101:0/3747290233 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2f00f20e0 0x7fb2f00f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:12.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.947 7fb2f48f1700 1 --2- 172.21.2.101:0/3747290233 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2f01b0470 0x7fb2f01b0870 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:12.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.947 7fb2f48f1700 1 -- 172.21.2.101:0/3747290233 >> 172.21.2.101:0/3747290233 conn(0x7fb2f00ce3e0 msgr2=0x7fb2f00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:12.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.947 7fb2f48f1700 1 -- 172.21.2.101:0/3747290233 shutdown_connections 2021-02-10T11:13:12.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.947 7fb2f48f1700 1 -- 172.21.2.101:0/3747290233 wait complete. 2021-02-10T11:13:12.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.962 7fb2f48f1700 1 Processor -- start 2021-02-10T11:13:12.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.962 7fb2f48f1700 1 -- start start 2021-02-10T11:13:12.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.962 7fb2f48f1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2f00c36a0 0x7fb2f00c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:12.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.962 7fb2f48f1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2f00c3190 0x7fb2f00b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:12.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.962 7fb2f48f1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2f00b2370 0x7fb2f00b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:12.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.962 7fb2f48f1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f00b27c0 con 0x7fb2f00b2370 2021-02-10T11:13:12.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.962 7fb2f48f1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f00b3360 con 0x7fb2f00c36a0 2021-02-10T11:13:12.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.962 7fb2f48f1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f00b3540 con 0x7fb2f00c3190 2021-02-10T11:13:12.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.962 7fb2eee1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2f00b2370 0x7fb2f00b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:12.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.962 7fb2eee1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2f00b2370 0x7fb2f00b2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43860/0 (socket says 172.21.2.101:43860) 2021-02-10T11:13:12.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.962 7fb2eee1d700 1 -- 172.21.2.101:0/3810883643 learned_addr learned my addr 172.21.2.101:0/3810883643 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:12.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.962 7fb2eee1d700 1 -- 172.21.2.101:0/3810883643 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2f00c3190 msgr2=0x7fb2f00b1d50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:12.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.962 7fb2eee1d700 1 --2- 172.21.2.101:0/3810883643 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2f00c3190 0x7fb2f00b1d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:12.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.962 7fb2eee1d700 1 -- 172.21.2.101:0/3810883643 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2f00c36a0 msgr2=0x7fb2f00c2b70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:12.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.962 7fb2eee1d700 1 --2- 172.21.2.101:0/3810883643 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2f00c36a0 0x7fb2f00c2b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:12.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.962 7fb2eee1d700 1 -- 172.21.2.101:0/3810883643 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f00b3760 con 0x7fb2f00b2370 2021-02-10T11:13:12.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.963 7fb2eee1d700 1 --2- 172.21.2.101:0/3810883643 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2f00b2370 0x7fb2f00b2d40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fb2e8009980 tx=0x7fb2e800bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:12.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.963 7fb2e6ffd700 1 -- 172.21.2.101:0/3810883643 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb2e8005af0 con 0x7fb2f00b2370 2021-02-10T11:13:12.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.963 7fb2e6ffd700 1 -- 172.21.2.101:0/3810883643 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e8005d70 con 0x7fb2f00b2370 2021-02-10T11:13:12.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.963 7fb2e6ffd700 1 -- 172.21.2.101:0/3810883643 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb2e80062b0 con 0x7fb2f00b2370 2021-02-10T11:13:12.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.963 7fb2f48f1700 1 -- 172.21.2.101:0/3810883643 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f00b3760 con 0x7fb2f00b2370 2021-02-10T11:13:12.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.963 7fb2f48f1700 1 -- 172.21.2.101:0/3810883643 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f00b3760 con 0x7fb2f00b2370 2021-02-10T11:13:12.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.963 7fb2e6ffd700 1 -- 172.21.2.101:0/3810883643 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb2e8006030 con 0x7fb2f00b2370 2021-02-10T11:13:12.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.963 7fb2e6ffd700 1 --2- 172.21.2.101:0/3810883643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2d40186c0 0x7fb2d401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:12.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.964 7fb2e6ffd700 1 -- 172.21.2.101:0/3810883643 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5107+0+0 (secure 0 0 0) 0x7fb2e8032710 con 0x7fb2f00b2370 2021-02-10T11:13:12.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.964 7fb2ee61c700 1 --2- 172.21.2.101:0/3810883643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2d40186c0 0x7fb2d401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:12.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.964 7fb2ee61c700 1 --2- 172.21.2.101:0/3810883643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2d40186c0 0x7fb2d401abc0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:12.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.995 7fb2f48f1700 1 -- 172.21.2.101:0/3810883643 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f00edd10 con 0x7fb2f00b2370 2021-02-10T11:13:13.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:12.998 7fb2e6ffd700 1 -- 172.21.2.101:0/3810883643 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb2e8005af0 con 0x7fb2f00b2370 2021-02-10T11:13:13.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.124 7fb2f48f1700 1 -- 172.21.2.101:0/3810883643 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"} v 0) v1 -- 0x7fb2f00edd10 con 0x7fb2f00b2370 2021-02-10T11:13:13.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.504 7fb2e6ffd700 1 -- 172.21.2.101:0/3810883643 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"}]=0 pool 'cephfs_data' created v18) v1 ==== 125+0+0 (secure 0 0 0) 0x7fb2e8006030 con 0x7fb2f00b2370 2021-02-10T11:13:13.544 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' created 2021-02-10T11:13:13.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.542 7fb2f48f1700 1 -- 172.21.2.101:0/3810883643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2d40186c0 msgr2=0x7fb2d401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:13.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.542 7fb2f48f1700 1 --2- 172.21.2.101:0/3810883643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2d40186c0 0x7fb2d401abc0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:13.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.542 7fb2f48f1700 1 -- 172.21.2.101:0/3810883643 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2f00b2370 msgr2=0x7fb2f00b2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:13.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.542 7fb2f48f1700 1 --2- 172.21.2.101:0/3810883643 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2f00b2370 0x7fb2f00b2d40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fb2e8009980 tx=0x7fb2e800bcd0).stop 2021-02-10T11:13:13.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.542 7fb2f48f1700 1 -- 172.21.2.101:0/3810883643 shutdown_connections 2021-02-10T11:13:13.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.542 7fb2f48f1700 1 --2- 172.21.2.101:0/3810883643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2d40186c0 0x7fb2d401abc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:13.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.542 7fb2f48f1700 1 --2- 172.21.2.101:0/3810883643 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2f00c3190 0x7fb2f00b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:13.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.542 7fb2f48f1700 1 --2- 172.21.2.101:0/3810883643 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2f00b2370 0x7fb2f00b2d40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:13.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.542 7fb2f48f1700 1 --2- 172.21.2.101:0/3810883643 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2f00c36a0 0x7fb2f00c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:13.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.542 7fb2f48f1700 1 -- 172.21.2.101:0/3810883643 >> 172.21.2.101:0/3810883643 conn(0x7fb2f009f630 msgr2=0x7fb2f00a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:13.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.542 7fb2f48f1700 1 -- 172.21.2.101:0/3810883643 shutdown_connections 2021-02-10T11:13:13.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.542 7fb2f48f1700 1 -- 172.21.2.101:0/3810883643 wait complete. 2021-02-10T11:13:13.554 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-02-10T11:13:13.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.963 7f3761fdc700 1 Processor -- start 2021-02-10T11:13:13.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.964 7f3761fdc700 1 -- start start 2021-02-10T11:13:13.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.964 7f3761fdc700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f375c0d1c70 0x7f375c0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:13.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.964 7f3761fdc700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375c0d0170 con 0x7f375c0d1870 2021-02-10T11:13:13.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.964 7f3761fdc700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375c0fe1c0 con 0x7f375c0cf580 2021-02-10T11:13:13.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.964 7f3761fdc700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f375c1b0cb0 con 0x7f375c0d1c70 2021-02-10T11:13:13.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.964 7f375b7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f375c0d1870 0x7f375c1b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44360/0 (socket says 172.21.2.101:44360) 2021-02-10T11:13:13.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.964 7f375b7fe700 1 -- 172.21.2.101:0/3162246767 learned_addr learned my addr 172.21.2.101:0/3162246767 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:13.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.965 7f375a7fc700 1 -- 172.21.2.101:0/3162246767 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4269402862 0 0) 0x7f3744000ce0 con 0x7f375c0d1870 2021-02-10T11:13:13.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.965 7f375a7fc700 1 -- 172.21.2.101:0/3162246767 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3748002fe0 con 0x7f375c0d1870 2021-02-10T11:13:13.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.965 7f375affd700 1 --2- 172.21.2.101:0/3162246767 >> v2:172.21.2.108:3300/0 conn(0x7f375c0d1c70 0x7f375c0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:13.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.965 7f375a7fc700 1 -- 172.21.2.101:0/3162246767 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2447816356 0 0) 0x7f3744001180 con 0x7f375c0d1870 2021-02-10T11:13:13.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.965 7f375a7fc700 1 -- 172.21.2.101:0/3162246767 >> v2:172.21.2.108:3300/0 conn(0x7f375c0d1c70 msgr2=0x7f375c0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:13.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.965 7f375a7fc700 1 --2- 172.21.2.101:0/3162246767 >> v2:172.21.2.108:3300/0 conn(0x7f375c0d1c70 0x7f375c0cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:13.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.965 7f375a7fc700 1 -- 172.21.2.101:0/3162246767 >> v1:172.21.2.108:6789/0 conn(0x7f375c0cf580 legacy=0x7f375c0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:13.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.965 7f375a7fc700 1 -- 172.21.2.101:0/3162246767 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c109a60 con 0x7f375c0d1870 2021-02-10T11:13:13.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.965 7f375a7fc700 1 -- 172.21.2.101:0/3162246767 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f3744001680 con 0x7f375c0d1870 2021-02-10T11:13:13.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.965 7f375a7fc700 1 -- 172.21.2.101:0/3162246767 >> v1:172.21.2.101:6789/0 conn(0x7f375c0d1870 legacy=0x7f375c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:13.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.965 7f375a7fc700 1 --2- 172.21.2.101:0/3162246767 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3748003a60 0x7f3748005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:13.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.965 7f375a7fc700 1 --2- 172.21.2.101:0/3162246767 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3748006780 0x7f3748008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:13.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.965 7f375a7fc700 1 --2- 172.21.2.101:0/3162246767 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3748009580 0x7f374800baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:13.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.965 7f375a7fc700 1 -- 172.21.2.101:0/3162246767 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374800c2f0 con 0x7f3748006780 2021-02-10T11:13:13.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.965 7f375a7fc700 1 -- 172.21.2.101:0/3162246767 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374800c510 con 0x7f3748009580 2021-02-10T11:13:13.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.965 7f375a7fc700 1 -- 172.21.2.101:0/3162246767 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374800c730 con 0x7f3748003a60 2021-02-10T11:13:13.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.965 7f375affd700 1 --2- 172.21.2.101:0/3162246767 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3748006780 0x7f3748008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:13.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.966 7f375b7fe700 1 --2- 172.21.2.101:0/3162246767 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3748003a60 0x7f3748005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:13.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.966 7f375bfff700 1 --2- 172.21.2.101:0/3162246767 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3748009580 0x7f374800baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:13.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.966 7f375affd700 1 -- 172.21.2.101:0/3162246767 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3748003a60 msgr2=0x7f3748005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:13.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.966 7f375affd700 1 --2- 172.21.2.101:0/3162246767 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3748003a60 0x7f3748005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:13.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.966 7f375affd700 1 -- 172.21.2.101:0/3162246767 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3748009580 msgr2=0x7f374800baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:13.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.966 7f375affd700 1 --2- 172.21.2.101:0/3162246767 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3748009580 0x7f374800baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:13.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.966 7f375bfff700 1 -- 172.21.2.101:0/3162246767 reap_dead start 2021-02-10T11:13:13.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.966 7f375affd700 1 -- 172.21.2.101:0/3162246767 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374800c9d0 con 0x7f3748006780 2021-02-10T11:13:13.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.966 7f375affd700 1 --2- 172.21.2.101:0/3162246767 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3748006780 0x7f3748008c80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f374c006a40 tx=0x7f374c006820).ready entity=mon.0 client_cookie=121dc2aa773a699 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:13.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.966 7f375a7fc700 1 -- 172.21.2.101:0/3162246767 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f374c000c90 con 0x7f3748006780 2021-02-10T11:13:13.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.966 7f375a7fc700 1 -- 172.21.2.101:0/3162246767 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374c002030 con 0x7f3748006780 2021-02-10T11:13:13.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.966 7f375a7fc700 1 -- 172.21.2.101:0/3162246767 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f374c002510 con 0x7f3748006780 2021-02-10T11:13:13.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.966 7f3761fdc700 1 -- 172.21.2.101:0/3162246767 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3748006780 msgr2=0x7f3748008c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:13.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.966 7f3761fdc700 1 --2- 172.21.2.101:0/3162246767 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3748006780 0x7f3748008c80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f374c006a40 tx=0x7f374c006820).stop 2021-02-10T11:13:13.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.967 7f3761fdc700 1 -- 172.21.2.101:0/3162246767 shutdown_connections 2021-02-10T11:13:13.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.967 7f3761fdc700 1 --2- 172.21.2.101:0/3162246767 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3748006780 0x7f3748008c80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:13.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.967 7f3761fdc700 1 -- 172.21.2.101:0/3162246767 >> 172.21.2.101:0/3162246767 conn(0x7f375c0ce420 msgr2=0x7f375c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:13.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.967 7f3761fdc700 1 -- 172.21.2.101:0/3162246767 shutdown_connections 2021-02-10T11:13:13.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.967 7f3761fdc700 1 -- 172.21.2.101:0/3162246767 wait complete. 2021-02-10T11:13:13.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.981 7f3761fdc700 1 Processor -- start 2021-02-10T11:13:13.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.982 7f3761fdc700 1 -- start start 2021-02-10T11:13:13.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.982 7f3761fdc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f375c096700 0x7f375c095b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:13.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.982 7f3761fdc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f375c096b90 0x7f375c096f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:13.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.982 7f3761fdc700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f375c0975b0 0x7f375c0c20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:13.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.982 7f3761fdc700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c0961f0 con 0x7f375c096700 2021-02-10T11:13:13.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.982 7f3761fdc700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c0c2700 con 0x7f375c0975b0 2021-02-10T11:13:13.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.982 7f3761fdc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375c0c2920 con 0x7f375c096b90 2021-02-10T11:13:13.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.982 7f375b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f375c096700 0x7f375c095b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:13.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.982 7f375b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f375c096700 0x7f375c095b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43880/0 (socket says 172.21.2.101:43880) 2021-02-10T11:13:13.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.982 7f375b7fe700 1 -- 172.21.2.101:0/3741949872 learned_addr learned my addr 172.21.2.101:0/3741949872 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:13.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.983 7f375b7fe700 1 -- 172.21.2.101:0/3741949872 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f375c096b90 msgr2=0x7f375c096f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:13.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.983 7f375b7fe700 1 --2- 172.21.2.101:0/3741949872 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f375c096b90 0x7f375c096f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:13.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.983 7f375b7fe700 1 -- 172.21.2.101:0/3741949872 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f375c0975b0 msgr2=0x7f375c0c20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:13.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.983 7f375b7fe700 1 --2- 172.21.2.101:0/3741949872 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f375c0975b0 0x7f375c0c20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:13.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.983 7f375b7fe700 1 -- 172.21.2.101:0/3741949872 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c0c2b40 con 0x7f375c096700 2021-02-10T11:13:13.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.983 7f375b7fe700 1 --2- 172.21.2.101:0/3741949872 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f375c096700 0x7f375c095b40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f3744007f90 tx=0x7f374400a930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:13.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.983 7f3753fff700 1 -- 172.21.2.101:0/3741949872 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3744007480 con 0x7f375c096700 2021-02-10T11:13:13.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.983 7f3753fff700 1 -- 172.21.2.101:0/3741949872 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374400ad20 con 0x7f375c096700 2021-02-10T11:13:13.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.983 7f3753fff700 1 -- 172.21.2.101:0/3741949872 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3744004620 con 0x7f375c096700 2021-02-10T11:13:13.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.983 7f3761fdc700 1 -- 172.21.2.101:0/3741949872 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375c0c2b40 con 0x7f375c096700 2021-02-10T11:13:13.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.983 7f3761fdc700 1 -- 172.21.2.101:0/3741949872 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375c0c2b40 con 0x7f375c096700 2021-02-10T11:13:13.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.984 7f3753fff700 1 -- 172.21.2.101:0/3741949872 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f37440043a0 con 0x7f375c096700 2021-02-10T11:13:13.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.984 7f3753fff700 1 --2- 172.21.2.101:0/3741949872 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37400186c0 0x7f374001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:13.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.984 7f3753fff700 1 -- 172.21.2.101:0/3741949872 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5457+0+0 (secure 0 0 0) 0x7f37440324d0 con 0x7f375c096700 2021-02-10T11:13:13.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.984 7f375affd700 1 --2- 172.21.2.101:0/3741949872 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37400186c0 0x7f374001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:13.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:13.984 7f375affd700 1 --2- 172.21.2.101:0/3741949872 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37400186c0 0x7f374001abc0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:14.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.014 7f3761fdc700 1 -- 172.21.2.101:0/3741949872 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f375c09dae0 con 0x7f375c096700 2021-02-10T11:13:14.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.017 7f3753fff700 1 -- 172.21.2.101:0/3741949872 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3744007480 con 0x7f375c096700 2021-02-10T11:13:14.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.145 7f3761fdc700 1 -- 172.21.2.101:0/3741949872 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"} v 0) v1 -- 0x7f375c09dae0 con 0x7f375c096700 2021-02-10T11:13:14.150 INFO:tasks.ceph.mon.a.gibba001.stderr:2021-02-10 11:13:14.146 7f974a848700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-02-10T11:13:14.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.150 7f3753fff700 1 -- 172.21.2.101:0/3741949872 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"}]=0 new fs with metadata pool 2 and data pool 3 v2) v1 ==== 187+0+0 (secure 0 0 0) 0x7f3744007480 con 0x7f375c096700 2021-02-10T11:13:14.156 INFO:teuthology.orchestra.run.gibba001.stderr:new fs with metadata pool 2 and data pool 3 2021-02-10T11:13:14.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.154 7f3761fdc700 1 -- 172.21.2.101:0/3741949872 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37400186c0 msgr2=0x7f374001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:14.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.154 7f3761fdc700 1 --2- 172.21.2.101:0/3741949872 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37400186c0 0x7f374001abc0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:14.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.154 7f3761fdc700 1 -- 172.21.2.101:0/3741949872 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f375c096700 msgr2=0x7f375c095b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:14.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.154 7f3761fdc700 1 --2- 172.21.2.101:0/3741949872 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f375c096700 0x7f375c095b40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f3744007f90 tx=0x7f374400a930).stop 2021-02-10T11:13:14.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.154 7f3761fdc700 1 -- 172.21.2.101:0/3741949872 shutdown_connections 2021-02-10T11:13:14.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.154 7f3761fdc700 1 --2- 172.21.2.101:0/3741949872 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37400186c0 0x7f374001abc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:14.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.154 7f3761fdc700 1 --2- 172.21.2.101:0/3741949872 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f375c096b90 0x7f375c096f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:14.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.154 7f3761fdc700 1 --2- 172.21.2.101:0/3741949872 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f375c0975b0 0x7f375c0c20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:14.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.154 7f3761fdc700 1 --2- 172.21.2.101:0/3741949872 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f375c096700 0x7f375c095b40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:14.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.154 7f3761fdc700 1 -- 172.21.2.101:0/3741949872 >> 172.21.2.101:0/3741949872 conn(0x7f375c08a460 msgr2=0x7f375c082b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:14.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.154 7f3761fdc700 1 -- 172.21.2.101:0/3741949872 shutdown_connections 2021-02-10T11:13:14.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.154 7f3761fdc700 1 -- 172.21.2.101:0/3741949872 wait complete. 2021-02-10T11:13:14.168 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:13:14.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.574 7f6313c2b700 1 Processor -- start 2021-02-10T11:13:14.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.574 7f6313c2b700 1 -- start start 2021-02-10T11:13:14.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.574 7f6313c2b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f630c0e5470 0x7f630c07e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:14.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.574 7f6313c2b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f630c07ed80 con 0x7f630c07f300 2021-02-10T11:13:14.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.574 7f6313c2b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f630c1b4010 con 0x7f630c0e5070 2021-02-10T11:13:14.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.574 7f6313c2b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f630c0d1790 con 0x7f630c0e5470 2021-02-10T11:13:14.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.575 7f6312248700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f630c07f300 0x7f630c07f700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44382/0 (socket says 172.21.2.101:44382) 2021-02-10T11:13:14.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.575 7f6312248700 1 -- 172.21.2.101:0/2058900214 learned_addr learned my addr 172.21.2.101:0/2058900214 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:14.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.575 7f6310a45700 1 -- 172.21.2.101:0/2058900214 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3039801776 0 0) 0x7f6304001500 con 0x7f630c07f300 2021-02-10T11:13:14.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.575 7f6310a45700 1 -- 172.21.2.101:0/2058900214 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62f8002fe0 con 0x7f630c07f300 2021-02-10T11:13:14.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.575 7f6311246700 1 --2- 172.21.2.101:0/2058900214 >> v2:172.21.2.108:3300/0 conn(0x7f630c0e5470 0x7f630c07e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:14.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.576 7f6310a45700 1 -- 172.21.2.101:0/2058900214 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2495438938 0 0) 0x7f6304001500 con 0x7f630c07f300 2021-02-10T11:13:14.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.576 7f6311246700 1 -- 172.21.2.101:0/2058900214 >> v1:172.21.2.108:6789/0 conn(0x7f630c0e5070 legacy=0x7f630c1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:14.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.576 7f6311246700 1 -- 172.21.2.101:0/2058900214 >> v1:172.21.2.101:6789/0 conn(0x7f630c07f300 legacy=0x7f630c07f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:14.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.576 7f6311246700 1 -- 172.21.2.101:0/2058900214 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c0d19b0 con 0x7f630c0e5470 2021-02-10T11:13:14.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.576 7f6311246700 1 --2- 172.21.2.101:0/2058900214 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f630c0e5470 0x7f630c07e6a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f6308006650 tx=0x7f6308009a80).ready entity=mon.1 client_cookie=93983c961104a84c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:14.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.576 7f6310a45700 1 -- 172.21.2.101:0/2058900214 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f630800c0c0 con 0x7f630c0e5470 2021-02-10T11:13:14.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.576 7f6310a45700 1 -- 172.21.2.101:0/2058900214 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630800c340 con 0x7f630c0e5470 2021-02-10T11:13:14.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.576 7f6310a45700 1 -- 172.21.2.101:0/2058900214 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f630800c860 con 0x7f630c0e5470 2021-02-10T11:13:14.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.576 7f6313c2b700 1 -- 172.21.2.101:0/2058900214 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f630c0e5470 msgr2=0x7f630c07e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:14.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.576 7f6313c2b700 1 --2- 172.21.2.101:0/2058900214 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f630c0e5470 0x7f630c07e6a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f6308006650 tx=0x7f6308009a80).stop 2021-02-10T11:13:14.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.577 7f6313c2b700 1 -- 172.21.2.101:0/2058900214 shutdown_connections 2021-02-10T11:13:14.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.577 7f6313c2b700 1 --2- 172.21.2.101:0/2058900214 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f630c0e5470 0x7f630c07e6a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:14.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.577 7f6313c2b700 1 -- 172.21.2.101:0/2058900214 >> 172.21.2.101:0/2058900214 conn(0x7f630c0ce690 msgr2=0x7f630c0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:14.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.577 7f6313c2b700 1 -- 172.21.2.101:0/2058900214 shutdown_connections 2021-02-10T11:13:14.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.577 7f6313c2b700 1 -- 172.21.2.101:0/2058900214 wait complete. 2021-02-10T11:13:14.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.592 7f6313c2b700 1 Processor -- start 2021-02-10T11:13:14.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.592 7f6313c2b700 1 -- start start 2021-02-10T11:13:14.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.592 7f6313c2b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f630c18c700 0x7f630c096c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:14.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.592 7f6313c2b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f630c0ae400 0x7f630c0ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:14.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.592 7f6313c2b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f630c192380 0x7f630c192780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:14.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.592 7f6313c2b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c097290 con 0x7f630c0ae400 2021-02-10T11:13:14.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.592 7f6313c2b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c192da0 con 0x7f630c18c700 2021-02-10T11:13:14.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.592 7f6313c2b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630c192f80 con 0x7f630c192380 2021-02-10T11:13:14.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.592 7f6311246700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f630c0ae400 0x7f630c0ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:14.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.592 7f6311246700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f630c0ae400 0x7f630c0ae800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43890/0 (socket says 172.21.2.101:43890) 2021-02-10T11:13:14.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.592 7f6311246700 1 -- 172.21.2.101:0/117187204 learned_addr learned my addr 172.21.2.101:0/117187204 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:14.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.592 7f6311a47700 1 --2- 172.21.2.101:0/117187204 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f630c18c700 0x7f630c096c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:14.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.592 7f6312248700 1 --2- 172.21.2.101:0/117187204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f630c192380 0x7f630c192780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:14.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.593 7f6311246700 1 -- 172.21.2.101:0/117187204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f630c192380 msgr2=0x7f630c192780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:14.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.593 7f6311246700 1 --2- 172.21.2.101:0/117187204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f630c192380 0x7f630c192780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:14.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.593 7f6311246700 1 -- 172.21.2.101:0/117187204 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f630c18c700 msgr2=0x7f630c096c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:14.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.593 7f6311246700 1 --2- 172.21.2.101:0/117187204 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f630c18c700 0x7f630c096c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:14.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.593 7f6311246700 1 -- 172.21.2.101:0/117187204 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c1931a0 con 0x7f630c0ae400 2021-02-10T11:13:14.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.593 7f6311a47700 1 --2- 172.21.2.101:0/117187204 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f630c18c700 0x7f630c096c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:13:14.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.593 7f6312248700 1 --2- 172.21.2.101:0/117187204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f630c192380 0x7f630c192780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:13:14.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.593 7f6311246700 1 --2- 172.21.2.101:0/117187204 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f630c0ae400 0x7f630c0ae800 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f63080093a0 tx=0x7f6308002670).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:14.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.593 7f62fe7fc700 1 -- 172.21.2.101:0/117187204 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6308006070 con 0x7f630c0ae400 2021-02-10T11:13:14.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.593 7f62fe7fc700 1 -- 172.21.2.101:0/117187204 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6308006ae0 con 0x7f630c0ae400 2021-02-10T11:13:14.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.593 7f62fe7fc700 1 -- 172.21.2.101:0/117187204 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6308007030 con 0x7f630c0ae400 2021-02-10T11:13:14.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.593 7f6313c2b700 1 -- 172.21.2.101:0/117187204 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f630c1931a0 con 0x7f630c0ae400 2021-02-10T11:13:14.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.593 7f6313c2b700 1 -- 172.21.2.101:0/117187204 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f630c1931a0 con 0x7f630c0ae400 2021-02-10T11:13:14.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.594 7f62fe7fc700 1 -- 172.21.2.101:0/117187204 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6308006cc0 con 0x7f630c0ae400 2021-02-10T11:13:14.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.594 7f62fe7fc700 1 --2- 172.21.2.101:0/117187204 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f62f40186c0 0x7f62f401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:14.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.594 7f62fe7fc700 1 -- 172.21.2.101:0/117187204 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6308032610 con 0x7f630c0ae400 2021-02-10T11:13:14.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.594 7f6311a47700 1 --2- 172.21.2.101:0/117187204 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f62f40186c0 0x7f62f401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:14.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.594 7f6311a47700 1 --2- 172.21.2.101:0/117187204 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f62f40186c0 0x7f62f401abc0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:14.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.624 7f6313c2b700 1 -- 172.21.2.101:0/117187204 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f630c107120 con 0x7f630c0ae400 2021-02-10T11:13:14.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.627 7f62fe7fc700 1 -- 172.21.2.101:0/117187204 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6308002a60 con 0x7f630c0ae400 2021-02-10T11:13:14.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.751 7f6313c2b700 1 -- 172.21.2.101:0/117187204 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f630c107120 con 0x7f630c0ae400 2021-02-10T11:13:14.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.752 7f62fe7fc700 1 -- 172.21.2.101:0/117187204 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14006 (secure 0 0 0) 0x7f6308006ca0 con 0x7f630c0ae400 2021-02-10T11:13:14.756 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:13:14.756 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":19,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:13:14.146932","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:13:11.922272","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2021-02-10 11:13:13.141733","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:13:14.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.753 7f6313c2b700 1 -- 172.21.2.101:0/117187204 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f62f40186c0 msgr2=0x7f62f401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:14.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.753 7f6313c2b700 1 --2- 172.21.2.101:0/117187204 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f62f40186c0 0x7f62f401abc0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:14.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.753 7f6313c2b700 1 -- 172.21.2.101:0/117187204 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f630c0ae400 msgr2=0x7f630c0ae800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:14.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.753 7f6313c2b700 1 --2- 172.21.2.101:0/117187204 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f630c0ae400 0x7f630c0ae800 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f63080093a0 tx=0x7f6308002670).stop 2021-02-10T11:13:14.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.754 7f6313c2b700 1 -- 172.21.2.101:0/117187204 shutdown_connections 2021-02-10T11:13:14.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.754 7f6313c2b700 1 --2- 172.21.2.101:0/117187204 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f62f40186c0 0x7f62f401abc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:14.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.754 7f6313c2b700 1 --2- 172.21.2.101:0/117187204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f630c192380 0x7f630c192780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:14.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.754 7f6313c2b700 1 --2- 172.21.2.101:0/117187204 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f630c0ae400 0x7f630c0ae800 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:14.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.754 7f6313c2b700 1 --2- 172.21.2.101:0/117187204 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f630c18c700 0x7f630c096c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:14.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.754 7f6313c2b700 1 -- 172.21.2.101:0/117187204 >> 172.21.2.101:0/117187204 conn(0x7f630c0891f0 msgr2=0x7f630c1a0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:14.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.754 7f6313c2b700 1 -- 172.21.2.101:0/117187204 shutdown_connections 2021-02-10T11:13:14.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:14.754 7f6313c2b700 1 -- 172.21.2.101:0/117187204 wait complete. 2021-02-10T11:13:14.766 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:13:15.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.172 7f27f9010700 1 Processor -- start 2021-02-10T11:13:15.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.172 7f27f9010700 1 -- start start 2021-02-10T11:13:15.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.172 7f27f9010700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f27f40e5470 0x7f27f407e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:15.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.172 7f27f9010700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27f407f300 0x7f27f407f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:15.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.172 7f27f9010700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f407ed80 con 0x7f27f40e5070 2021-02-10T11:13:15.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.172 7f27f9010700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f41b4010 con 0x7f27f40e5470 2021-02-10T11:13:15.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.172 7f27f9010700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f40d1790 con 0x7f27f407f300 2021-02-10T11:13:15.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.173 7f27f259c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f27f40e5470 0x7f27f407e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:15.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.173 7f27f259c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f27f40e5470 0x7f27f407e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43900/0 (socket says 172.21.2.101:43900) 2021-02-10T11:13:15.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.173 7f27f259c700 1 -- 172.21.2.101:0/4263254039 learned_addr learned my addr 172.21.2.101:0/4263254039 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:15.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.173 7f27f259c700 1 -- 172.21.2.101:0/4263254039 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27f407f300 msgr2=0x7f27f407f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:15.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.173 7f27f259c700 1 --2- 172.21.2.101:0/4263254039 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27f407f300 0x7f27f407f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:15.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.173 7f27f259c700 1 -- 172.21.2.101:0/4263254039 >> v1:172.21.2.108:6789/0 conn(0x7f27f40e5070 legacy=0x7f27f41b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:15.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.173 7f27f259c700 1 -- 172.21.2.101:0/4263254039 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f40d19b0 con 0x7f27f40e5470 2021-02-10T11:13:15.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.173 7f27f259c700 1 --2- 172.21.2.101:0/4263254039 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27f40e5470 0x7f27f407e6a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f27e0006700 tx=0x7f27e0009b80).ready entity=mon.0 client_cookie=efd78d4ca1a0c5c4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:15.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.173 7f27f1d9b700 1 -- 172.21.2.101:0/4263254039 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f27e000c0c0 con 0x7f27f40e5470 2021-02-10T11:13:15.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.173 7f27f1d9b700 1 -- 172.21.2.101:0/4263254039 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e000c340 con 0x7f27f40e5470 2021-02-10T11:13:15.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.173 7f27f1d9b700 1 -- 172.21.2.101:0/4263254039 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f27e000c860 con 0x7f27f40e5470 2021-02-10T11:13:15.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.173 7f27f9010700 1 -- 172.21.2.101:0/4263254039 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27f40e5470 msgr2=0x7f27f407e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:15.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.173 7f27f9010700 1 --2- 172.21.2.101:0/4263254039 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27f40e5470 0x7f27f407e6a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f27e0006700 tx=0x7f27e0009b80).stop 2021-02-10T11:13:15.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.174 7f27f9010700 1 -- 172.21.2.101:0/4263254039 shutdown_connections 2021-02-10T11:13:15.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.174 7f27f9010700 1 --2- 172.21.2.101:0/4263254039 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27f407f300 0x7f27f407f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:15.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.174 7f27f9010700 1 --2- 172.21.2.101:0/4263254039 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27f40e5470 0x7f27f407e6a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:15.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.174 7f27f9010700 1 -- 172.21.2.101:0/4263254039 >> 172.21.2.101:0/4263254039 conn(0x7f27f40ce690 msgr2=0x7f27f40e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:15.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.174 7f27f9010700 1 -- 172.21.2.101:0/4263254039 shutdown_connections 2021-02-10T11:13:15.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.174 7f27f9010700 1 -- 172.21.2.101:0/4263254039 wait complete. 2021-02-10T11:13:15.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.189 7f27f9010700 1 Processor -- start 2021-02-10T11:13:15.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.189 7f27f9010700 1 -- start start 2021-02-10T11:13:15.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.189 7f27f9010700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27f41a2a90 0x7f27f41aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:15.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.189 7f27f9010700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27f41aa700 0x7f27f41a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:15.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.189 7f27f9010700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f27f40a55e0 0x7f27f41a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:15.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.189 7f27f9010700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f41a34a0 con 0x7f27f41a2a90 2021-02-10T11:13:15.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.189 7f27f9010700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f41a2e90 con 0x7f27f40a55e0 2021-02-10T11:13:15.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.189 7f27f9010700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f41a30b0 con 0x7f27f41aa700 2021-02-10T11:13:15.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.190 7f27f2d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27f41a2a90 0x7f27f41aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:15.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.190 7f27f2d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27f41a2a90 0x7f27f41aa0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43904/0 (socket says 172.21.2.101:43904) 2021-02-10T11:13:15.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.190 7f27f2d9d700 1 -- 172.21.2.101:0/915285907 learned_addr learned my addr 172.21.2.101:0/915285907 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:15.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.190 7f27f259c700 1 --2- 172.21.2.101:0/915285907 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27f41aa700 0x7f27f41a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:15.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.190 7f27f359e700 1 --2- 172.21.2.101:0/915285907 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f27f40a55e0 0x7f27f41a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:15.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.190 7f27f2d9d700 1 -- 172.21.2.101:0/915285907 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27f41aa700 msgr2=0x7f27f41a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:15.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.190 7f27f2d9d700 1 --2- 172.21.2.101:0/915285907 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27f41aa700 0x7f27f41a3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:15.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.190 7f27f2d9d700 1 -- 172.21.2.101:0/915285907 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f27f40a55e0 msgr2=0x7f27f41a4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:15.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.190 7f27f2d9d700 1 --2- 172.21.2.101:0/915285907 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f27f40a55e0 0x7f27f41a4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:15.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.190 7f27f2d9d700 1 -- 172.21.2.101:0/915285907 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f41a4660 con 0x7f27f41a2a90 2021-02-10T11:13:15.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.190 7f27f2d9d700 1 --2- 172.21.2.101:0/915285907 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27f41a2a90 0x7f27f41aa0e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f27e400b5b0 tx=0x7f27e400b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:15.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.191 7f27eb7fe700 1 -- 172.21.2.101:0/915285907 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f27e400bd40 con 0x7f27f41a2a90 2021-02-10T11:13:15.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.191 7f27eb7fe700 1 -- 172.21.2.101:0/915285907 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e4006240 con 0x7f27f41a2a90 2021-02-10T11:13:15.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.191 7f27eb7fe700 1 -- 172.21.2.101:0/915285907 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f27e4005720 con 0x7f27f41a2a90 2021-02-10T11:13:15.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.191 7f27f9010700 1 -- 172.21.2.101:0/915285907 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f41a4660 con 0x7f27f41a2a90 2021-02-10T11:13:15.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.191 7f27f9010700 1 -- 172.21.2.101:0/915285907 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f41a4660 con 0x7f27f41a2a90 2021-02-10T11:13:15.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.191 7f27eb7fe700 1 -- 172.21.2.101:0/915285907 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f27e40054a0 con 0x7f27f41a2a90 2021-02-10T11:13:15.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.191 7f27eb7fe700 1 --2- 172.21.2.101:0/915285907 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f27d00186c0 0x7f27d001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:15.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.191 7f27eb7fe700 1 -- 172.21.2.101:0/915285907 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f27e4032420 con 0x7f27f41a2a90 2021-02-10T11:13:15.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.192 7f27f259c700 1 --2- 172.21.2.101:0/915285907 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f27d00186c0 0x7f27d001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:15.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.192 7f27f259c700 1 --2- 172.21.2.101:0/915285907 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f27d00186c0 0x7f27d001abc0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:15.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.222 7f27f9010700 1 -- 172.21.2.101:0/915285907 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f4083da0 con 0x7f27f41a2a90 2021-02-10T11:13:15.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.224 7f27eb7fe700 1 -- 172.21.2.101:0/915285907 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f27e401ecc0 con 0x7f27f41a2a90 2021-02-10T11:13:15.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.349 7f27f9010700 1 -- 172.21.2.101:0/915285907 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27f407ed80 con 0x7f27f41a2a90 2021-02-10T11:13:15.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.350 7f27eb7fe700 1 -- 172.21.2.101:0/915285907 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+13993 (secure 0 0 0) 0x7f27e401ecc0 con 0x7f27f41a2a90 2021-02-10T11:13:15.354 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:13:15.355 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":20,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:13:15.149234","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:13:11.922272","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2021-02-10 11:13:13.141733","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:13:15.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.352 7f27f9010700 1 -- 172.21.2.101:0/915285907 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f27d00186c0 msgr2=0x7f27d001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:15.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.352 7f27f9010700 1 --2- 172.21.2.101:0/915285907 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f27d00186c0 0x7f27d001abc0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:15.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.352 7f27f9010700 1 -- 172.21.2.101:0/915285907 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27f41a2a90 msgr2=0x7f27f41aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:15.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.352 7f27f9010700 1 --2- 172.21.2.101:0/915285907 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27f41a2a90 0x7f27f41aa0e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f27e400b5b0 tx=0x7f27e400b950).stop 2021-02-10T11:13:15.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.352 7f27f9010700 1 -- 172.21.2.101:0/915285907 shutdown_connections 2021-02-10T11:13:15.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.352 7f27f9010700 1 --2- 172.21.2.101:0/915285907 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f27d00186c0 0x7f27d001abc0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:15.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.352 7f27f9010700 1 --2- 172.21.2.101:0/915285907 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27f41aa700 0x7f27f41a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:15.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.352 7f27f9010700 1 --2- 172.21.2.101:0/915285907 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f27f40a55e0 0x7f27f41a4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:15.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.352 7f27f9010700 1 --2- 172.21.2.101:0/915285907 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27f41a2a90 0x7f27f41aa0e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:15.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.352 7f27f9010700 1 -- 172.21.2.101:0/915285907 >> 172.21.2.101:0/915285907 conn(0x7f27f408fe10 msgr2=0x7f27f419c020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:15.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.352 7f27f9010700 1 -- 172.21.2.101:0/915285907 shutdown_connections 2021-02-10T11:13:15.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.352 7f27f9010700 1 -- 172.21.2.101:0/915285907 wait complete. 2021-02-10T11:13:15.365 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-02-10T11:13:15.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.773 7fc7b5d88700 1 Processor -- start 2021-02-10T11:13:15.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.773 7fc7b5d88700 1 -- start start 2021-02-10T11:13:15.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.773 7fc7b5d88700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7b00f05f0 0x7fc7b00f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:15.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.773 7fc7b5d88700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b01b0c70 con 0x7fc7b01b0470 2021-02-10T11:13:15.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.773 7fc7b5d88700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b00f0b50 con 0x7fc7b01b0870 2021-02-10T11:13:15.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.773 7fc7b5d88700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b00e5ea0 con 0x7fc7b00f05f0 2021-02-10T11:13:15.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.774 7fc7af7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc7b01b0470 0x7fc7b00ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44406/0 (socket says 172.21.2.101:44406) 2021-02-10T11:13:15.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.774 7fc7af7fe700 1 -- 172.21.2.101:0/1859685003 learned_addr learned my addr 172.21.2.101:0/1859685003 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:15.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.774 7fc7ae7fc700 1 -- 172.21.2.101:0/1859685003 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1559222783 0 0) 0x7fc798001180 con 0x7fc7b01b0470 2021-02-10T11:13:15.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.774 7fc7affff700 1 --2- 172.21.2.101:0/1859685003 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7b00f05f0 0x7fc7b00f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:15.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.774 7fc7ae7fc700 1 -- 172.21.2.101:0/1859685003 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc79c002fe0 con 0x7fc7b01b0470 2021-02-10T11:13:15.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.775 7fc7ae7fc700 1 -- 172.21.2.101:0/1859685003 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2110950750 0 0) 0x7fc7a0001080 con 0x7fc7b01b0870 2021-02-10T11:13:15.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.775 7fc7ae7fc700 1 -- 172.21.2.101:0/1859685003 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc79c003210 con 0x7fc7b01b0870 2021-02-10T11:13:15.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.775 7fc7affff700 1 -- 172.21.2.101:0/1859685003 >> v1:172.21.2.108:6789/0 conn(0x7fc7b01b0870 legacy=0x7fc7b00efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:15.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.775 7fc7affff700 1 -- 172.21.2.101:0/1859685003 >> v1:172.21.2.101:6789/0 conn(0x7fc7b01b0470 legacy=0x7fc7b00ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:15.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.775 7fc7affff700 1 -- 172.21.2.101:0/1859685003 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b00e52e0 con 0x7fc7b00f05f0 2021-02-10T11:13:15.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.775 7fc7affff700 1 --2- 172.21.2.101:0/1859685003 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7b00f05f0 0x7fc7b00f20e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fc7a8006e50 tx=0x7fc7a8009e70).ready entity=mon.2 client_cookie=d2f57471958d2630 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:15.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.775 7fc7ae7fc700 1 -- 172.21.2.101:0/1859685003 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc7a8006a20 con 0x7fc7b00f05f0 2021-02-10T11:13:15.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.775 7fc7ae7fc700 1 -- 172.21.2.101:0/1859685003 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a8006790 con 0x7fc7b00f05f0 2021-02-10T11:13:15.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.775 7fc7ae7fc700 1 -- 172.21.2.101:0/1859685003 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc7a800b700 con 0x7fc7b00f05f0 2021-02-10T11:13:15.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.775 7fc7b5d88700 1 -- 172.21.2.101:0/1859685003 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7b00f05f0 msgr2=0x7fc7b00f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:15.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.775 7fc7b5d88700 1 --2- 172.21.2.101:0/1859685003 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7b00f05f0 0x7fc7b00f20e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fc7a8006e50 tx=0x7fc7a8009e70).stop 2021-02-10T11:13:15.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.776 7fc7b5d88700 1 -- 172.21.2.101:0/1859685003 shutdown_connections 2021-02-10T11:13:15.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.776 7fc7b5d88700 1 --2- 172.21.2.101:0/1859685003 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7b00f05f0 0x7fc7b00f20e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:15.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.776 7fc7b5d88700 1 -- 172.21.2.101:0/1859685003 >> 172.21.2.101:0/1859685003 conn(0x7fc7b00ce3e0 msgr2=0x7fc7b00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:15.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.776 7fc7b5d88700 1 -- 172.21.2.101:0/1859685003 shutdown_connections 2021-02-10T11:13:15.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.776 7fc7b5d88700 1 -- 172.21.2.101:0/1859685003 wait complete. 2021-02-10T11:13:15.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.791 7fc7b5d88700 1 Processor -- start 2021-02-10T11:13:15.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.791 7fc7b5d88700 1 -- start start 2021-02-10T11:13:15.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.791 7fc7b5d88700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7b00a0420 0x7fc7b00a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:15.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.791 7fc7b5d88700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7b0189d00 0x7fc7b018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:15.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.791 7fc7b5d88700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7b018a720 0x7fc7b00b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:15.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.791 7fc7b5d88700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b0189780 con 0x7fc7b0189d00 2021-02-10T11:13:15.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.791 7fc7b5d88700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b018ab20 con 0x7fc7b018a720 2021-02-10T11:13:15.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.791 7fc7b5d88700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b00a1140 con 0x7fc7b00a0420 2021-02-10T11:13:15.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.792 7fc7aeffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7b0189d00 0x7fc7b018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:15.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.792 7fc7aeffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7b0189d00 0x7fc7b018a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43920/0 (socket says 172.21.2.101:43920) 2021-02-10T11:13:15.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.792 7fc7aeffd700 1 -- 172.21.2.101:0/2876456423 learned_addr learned my addr 172.21.2.101:0/2876456423 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:15.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.792 7fc7af7fe700 1 --2- 172.21.2.101:0/2876456423 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7b00a0420 0x7fc7b00a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:15.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.792 7fc7affff700 1 --2- 172.21.2.101:0/2876456423 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7b018a720 0x7fc7b00b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:15.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.792 7fc7aeffd700 1 -- 172.21.2.101:0/2876456423 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7b00a0420 msgr2=0x7fc7b00a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:15.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.792 7fc7aeffd700 1 --2- 172.21.2.101:0/2876456423 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7b00a0420 0x7fc7b00a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:15.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.792 7fc7aeffd700 1 -- 172.21.2.101:0/2876456423 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7b018a720 msgr2=0x7fc7b00b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:15.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.792 7fc7aeffd700 1 --2- 172.21.2.101:0/2876456423 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7b018a720 0x7fc7b00b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:15.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.792 7fc7aeffd700 1 -- 172.21.2.101:0/2876456423 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b00b8200 con 0x7fc7b0189d00 2021-02-10T11:13:15.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.792 7fc7aeffd700 1 --2- 172.21.2.101:0/2876456423 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7b0189d00 0x7fc7b018a100 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fc7a0006f00 tx=0x7fc7a000b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:15.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.792 7fc7a7fff700 1 -- 172.21.2.101:0/2876456423 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc7a000bd30 con 0x7fc7b0189d00 2021-02-10T11:13:15.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.792 7fc7a7fff700 1 -- 172.21.2.101:0/2876456423 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a0000e20 con 0x7fc7b0189d00 2021-02-10T11:13:15.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.792 7fc7b5d88700 1 -- 172.21.2.101:0/2876456423 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b00b8200 con 0x7fc7b0189d00 2021-02-10T11:13:15.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.792 7fc7a7fff700 1 -- 172.21.2.101:0/2876456423 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc7a00054f0 con 0x7fc7b0189d00 2021-02-10T11:13:15.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.793 7fc7b5d88700 1 -- 172.21.2.101:0/2876456423 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b00b8200 con 0x7fc7b0189d00 2021-02-10T11:13:15.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.793 7fc7a7fff700 1 -- 172.21.2.101:0/2876456423 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc7a0006240 con 0x7fc7b0189d00 2021-02-10T11:13:15.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.793 7fc7a7fff700 1 --2- 172.21.2.101:0/2876456423 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc794018730 0x7fc79401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:15.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.793 7fc7a7fff700 1 -- 172.21.2.101:0/2876456423 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc7a0032670 con 0x7fc7b0189d00 2021-02-10T11:13:15.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.793 7fc7af7fe700 1 --2- 172.21.2.101:0/2876456423 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc794018730 0x7fc79401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:15.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.794 7fc7af7fe700 1 --2- 172.21.2.101:0/2876456423 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc794018730 0x7fc79401ac10 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:15.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.823 7fc7b5d88700 1 -- 172.21.2.101:0/2876456423 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b0105ae0 con 0x7fc7b0189d00 2021-02-10T11:13:15.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.826 7fc7a7fff700 1 -- 172.21.2.101:0/2876456423 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc7a0006030 con 0x7fc7b0189d00 2021-02-10T11:13:15.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:15.963 7fc7b5d88700 1 -- 172.21.2.101:0/2876456423 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"} v 0) v1 -- 0x7fc7b00f0b50 con 0x7fc7b0189d00 2021-02-10T11:13:16.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.520 7fc7a7fff700 1 -- 172.21.2.101:0/2876456423 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"}]=0 v3) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc7a0000e20 con 0x7fc7b0189d00 2021-02-10T11:13:16.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.533 7fc7b5d88700 1 -- 172.21.2.101:0/2876456423 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc794018730 msgr2=0x7fc79401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:16.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.533 7fc7b5d88700 1 --2- 172.21.2.101:0/2876456423 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc794018730 0x7fc79401ac10 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:16.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.533 7fc7b5d88700 1 -- 172.21.2.101:0/2876456423 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7b0189d00 msgr2=0x7fc7b018a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:16.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.533 7fc7b5d88700 1 --2- 172.21.2.101:0/2876456423 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7b0189d00 0x7fc7b018a100 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fc7a0006f00 tx=0x7fc7a000b940).stop 2021-02-10T11:13:16.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.533 7fc7b5d88700 1 -- 172.21.2.101:0/2876456423 shutdown_connections 2021-02-10T11:13:16.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.533 7fc7b5d88700 1 --2- 172.21.2.101:0/2876456423 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7b018a720 0x7fc7b00b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:16.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.533 7fc7b5d88700 1 --2- 172.21.2.101:0/2876456423 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7b0189d00 0x7fc7b018a100 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:16.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.533 7fc7b5d88700 1 --2- 172.21.2.101:0/2876456423 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc794018730 0x7fc79401ac10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:16.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.533 7fc7b5d88700 1 --2- 172.21.2.101:0/2876456423 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7b00a0420 0x7fc7b00a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:16.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.533 7fc7b5d88700 1 -- 172.21.2.101:0/2876456423 >> 172.21.2.101:0/2876456423 conn(0x7fc7b0094e10 msgr2=0x7fc7b00a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:16.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.534 7fc7b5d88700 1 -- 172.21.2.101:0/2876456423 shutdown_connections 2021-02-10T11:13:16.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.534 7fc7b5d88700 1 -- 172.21.2.101:0/2876456423 wait complete. 2021-02-10T11:13:16.546 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:13:16.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.952 7fad61822700 1 Processor -- start 2021-02-10T11:13:16.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.952 7fad61822700 1 -- start start 2021-02-10T11:13:16.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.952 7fad61822700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad5c0e5070 0x7fad5c1b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:16.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.953 7fad61822700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fad5c0e5470 0x7fad5c07d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:16.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.953 7fad61822700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fad5c1b3e40 0x7fad5c07dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:16.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.953 7fad61822700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5c07ed80 con 0x7fad5c1b3e40 2021-02-10T11:13:16.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.953 7fad61822700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5c07e2d0 con 0x7fad5c0e5470 2021-02-10T11:13:16.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.953 7fad61822700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c07e4c0 con 0x7fad5c0e5070 2021-02-10T11:13:16.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.953 7fad5bd75700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fad5c1b3e40 0x7fad5c07dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:16.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.953 7fad5bd75700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fad5c1b3e40 0x7fad5c07dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43926/0 (socket says 172.21.2.101:43926) 2021-02-10T11:13:16.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.953 7fad5bd75700 1 -- 172.21.2.101:0/2912502220 learned_addr learned my addr 172.21.2.101:0/2912502220 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:16.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.953 7fad5bd75700 1 -- 172.21.2.101:0/2912502220 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad5c0e5070 msgr2=0x7fad5c1b3820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:13:16.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.953 7fad5bd75700 1 --2- 172.21.2.101:0/2912502220 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad5c0e5070 0x7fad5c1b3820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:16.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.953 7fad5bd75700 1 -- 172.21.2.101:0/2912502220 >> v2:172.21.2.108:3300/0 conn(0x7fad5c0e5470 msgr2=0x7fad5c07d690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:13:16.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.953 7fad5bd75700 1 --2- 172.21.2.101:0/2912502220 >> v2:172.21.2.108:3300/0 conn(0x7fad5c0e5470 0x7fad5c07d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:16.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.953 7fad5bd75700 1 -- 172.21.2.101:0/2912502220 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5c07e6e0 con 0x7fad5c1b3e40 2021-02-10T11:13:16.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.953 7fad5bd75700 1 --2- 172.21.2.101:0/2912502220 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad5c1b3e40 0x7fad5c07dcb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fad54006f00 tx=0x7fad54009f10).ready entity=mon.0 client_cookie=fd71c96d7cfb1842 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:16.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.953 7fad5a572700 1 -- 172.21.2.101:0/2912502220 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fad5400b3b0 con 0x7fad5c1b3e40 2021-02-10T11:13:16.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.953 7fad5a572700 1 -- 172.21.2.101:0/2912502220 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5400b630 con 0x7fad5c1b3e40 2021-02-10T11:13:16.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.953 7fad5a572700 1 -- 172.21.2.101:0/2912502220 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fad5400bb50 con 0x7fad5c1b3e40 2021-02-10T11:13:16.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.954 7fad61822700 1 -- 172.21.2.101:0/2912502220 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad5c1b3e40 msgr2=0x7fad5c07dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:16.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.954 7fad61822700 1 --2- 172.21.2.101:0/2912502220 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad5c1b3e40 0x7fad5c07dcb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fad54006f00 tx=0x7fad54009f10).stop 2021-02-10T11:13:16.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.954 7fad61822700 1 -- 172.21.2.101:0/2912502220 shutdown_connections 2021-02-10T11:13:16.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.954 7fad61822700 1 --2- 172.21.2.101:0/2912502220 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad5c1b3e40 0x7fad5c07dcb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:16.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.954 7fad61822700 1 --2- 172.21.2.101:0/2912502220 >> v2:172.21.2.108:3300/0 conn(0x7fad5c0e5470 0x7fad5c07d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:16.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.954 7fad61822700 1 --2- 172.21.2.101:0/2912502220 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad5c0e5070 0x7fad5c1b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:16.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.954 7fad61822700 1 -- 172.21.2.101:0/2912502220 >> 172.21.2.101:0/2912502220 conn(0x7fad5c0ce690 msgr2=0x7fad5c0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:16.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.954 7fad61822700 1 -- 172.21.2.101:0/2912502220 shutdown_connections 2021-02-10T11:13:16.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.954 7fad61822700 1 -- 172.21.2.101:0/2912502220 wait complete. 2021-02-10T11:13:16.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.968 7fad61822700 1 Processor -- start 2021-02-10T11:13:16.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.968 7fad61822700 1 -- start start 2021-02-10T11:13:16.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.969 7fad61822700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fad5c192960 0x7fad5c192d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:16.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.969 7fad61822700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad5c0ad110 0x7fad5c0ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:16.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.969 7fad61822700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad5c0adb30 0x7fad5c08e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:16.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.969 7fad61822700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c1a03e0 con 0x7fad5c0ad110 2021-02-10T11:13:16.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.969 7fad61822700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c193380 con 0x7fad5c192960 2021-02-10T11:13:16.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.969 7fad61822700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c08ea80 con 0x7fad5c0adb30 2021-02-10T11:13:16.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.969 7fad5ad73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad5c0ad110 0x7fad5c0ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:16.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.969 7fad5ad73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad5c0ad110 0x7fad5c0ad510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43934/0 (socket says 172.21.2.101:43934) 2021-02-10T11:13:16.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.969 7fad5ad73700 1 -- 172.21.2.101:0/3963761543 learned_addr learned my addr 172.21.2.101:0/3963761543 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:16.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.969 7fad5b574700 1 --2- 172.21.2.101:0/3963761543 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fad5c192960 0x7fad5c192d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:16.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.969 7fad5bd75700 1 --2- 172.21.2.101:0/3963761543 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad5c0adb30 0x7fad5c08e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:16.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.970 7fad5ad73700 1 -- 172.21.2.101:0/3963761543 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad5c0adb30 msgr2=0x7fad5c08e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:16.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.970 7fad5ad73700 1 --2- 172.21.2.101:0/3963761543 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad5c0adb30 0x7fad5c08e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:16.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.970 7fad5ad73700 1 -- 172.21.2.101:0/3963761543 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fad5c192960 msgr2=0x7fad5c192d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:16.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.970 7fad5ad73700 1 --2- 172.21.2.101:0/3963761543 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fad5c192960 0x7fad5c192d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:16.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.970 7fad5ad73700 1 -- 172.21.2.101:0/3963761543 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5c08eca0 con 0x7fad5c0ad110 2021-02-10T11:13:16.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.970 7fad5ad73700 1 --2- 172.21.2.101:0/3963761543 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad5c0ad110 0x7fad5c0ad510 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fad4400b560 tx=0x7fad4400b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:16.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.970 7fad53fff700 1 -- 172.21.2.101:0/3963761543 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fad4400bcf0 con 0x7fad5c0ad110 2021-02-10T11:13:16.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.970 7fad53fff700 1 -- 172.21.2.101:0/3963761543 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad44006030 con 0x7fad5c0ad110 2021-02-10T11:13:16.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.970 7fad61822700 1 -- 172.21.2.101:0/3963761543 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5c08eca0 con 0x7fad5c0ad110 2021-02-10T11:13:16.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.970 7fad61822700 1 -- 172.21.2.101:0/3963761543 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5c08f230 con 0x7fad5c0ad110 2021-02-10T11:13:16.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.970 7fad53fff700 1 -- 172.21.2.101:0/3963761543 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fad440054d0 con 0x7fad5c0ad110 2021-02-10T11:13:16.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.971 7fad53fff700 1 -- 172.21.2.101:0/3963761543 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fad44006210 con 0x7fad5c0ad110 2021-02-10T11:13:16.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.971 7fad53fff700 1 --2- 172.21.2.101:0/3963761543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fad400186c0 0x7fad4001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:16.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.971 7fad53fff700 1 -- 172.21.2.101:0/3963761543 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fad44032560 con 0x7fad5c0ad110 2021-02-10T11:13:16.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.971 7fad5b574700 1 --2- 172.21.2.101:0/3963761543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fad400186c0 0x7fad4001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:16.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:16.971 7fad5b574700 1 --2- 172.21.2.101:0/3963761543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fad400186c0 0x7fad4001abc0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:17.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.003 7fad61822700 1 -- 172.21.2.101:0/3963761543 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad5c107120 con 0x7fad5c0ad110 2021-02-10T11:13:17.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.005 7fad53fff700 1 -- 172.21.2.101:0/3963761543 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fad44000940 con 0x7fad5c0ad110 2021-02-10T11:13:17.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.132 7fad61822700 1 -- 172.21.2.101:0/3963761543 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad5c07ed80 con 0x7fad5c0ad110 2021-02-10T11:13:17.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.132 7fad53fff700 1 -- 172.21.2.101:0/3963761543 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+1415 (secure 0 0 0) 0x7fad44006210 con 0x7fad5c0ad110 2021-02-10T11:13:17.137 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 3 2021-02-10T11:13:17.137 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:13:17.137 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":3,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":3,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:13:14.147021","modified":"2021-02-10 11:13:16.516513","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-02-10T11:13:17.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.134 7fad61822700 1 -- 172.21.2.101:0/3963761543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fad400186c0 msgr2=0x7fad4001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:17.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.134 7fad61822700 1 --2- 172.21.2.101:0/3963761543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fad400186c0 0x7fad4001abc0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:17.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.134 7fad61822700 1 -- 172.21.2.101:0/3963761543 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad5c0ad110 msgr2=0x7fad5c0ad510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:17.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.134 7fad61822700 1 --2- 172.21.2.101:0/3963761543 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad5c0ad110 0x7fad5c0ad510 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fad4400b560 tx=0x7fad4400b900).stop 2021-02-10T11:13:17.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.134 7fad61822700 1 -- 172.21.2.101:0/3963761543 shutdown_connections 2021-02-10T11:13:17.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.134 7fad61822700 1 --2- 172.21.2.101:0/3963761543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fad400186c0 0x7fad4001abc0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:17.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.134 7fad61822700 1 --2- 172.21.2.101:0/3963761543 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad5c0adb30 0x7fad5c08e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:17.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.134 7fad61822700 1 --2- 172.21.2.101:0/3963761543 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad5c0ad110 0x7fad5c0ad510 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:17.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.134 7fad61822700 1 --2- 172.21.2.101:0/3963761543 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fad5c192960 0x7fad5c192d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:17.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.134 7fad61822700 1 -- 172.21.2.101:0/3963761543 >> 172.21.2.101:0/3963761543 conn(0x7fad5c0891f0 msgr2=0x7fad5c1a0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:17.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.134 7fad61822700 1 -- 172.21.2.101:0/3963761543 shutdown_connections 2021-02-10T11:13:17.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.135 7fad61822700 1 -- 172.21.2.101:0/3963761543 wait complete. 2021-02-10T11:13:17.147 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:13:17.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.554 7f413e742700 1 Processor -- start 2021-02-10T11:13:17.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.554 7f413e742700 1 -- start start 2021-02-10T11:13:17.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.554 7f413e742700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41380e5070 0x7f41381b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:17.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.554 7f413e742700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f413807ed80 con 0x7f41381b3e40 2021-02-10T11:13:17.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.554 7f413e742700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f413807e670 con 0x7f41380e5470 2021-02-10T11:13:17.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.554 7f413e742700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413807e940 con 0x7f41380e5070 2021-02-10T11:13:17.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.554 7f413cd5f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f41381b3e40 0x7f413807de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44436/0 (socket says 172.21.2.101:44436) 2021-02-10T11:13:17.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.554 7f413cd5f700 1 -- 172.21.2.101:0/4061723620 learned_addr learned my addr 172.21.2.101:0/4061723620 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:17.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.555 7f4137fff700 1 --2- 172.21.2.101:0/4061723620 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41380e5070 0x7f41381b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:17.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.555 7f4136ffd700 1 -- 172.21.2.101:0/4061723620 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4066564154 0 0) 0x7f4128000c10 con 0x7f41381b3e40 2021-02-10T11:13:17.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.555 7f4136ffd700 1 -- 172.21.2.101:0/4061723620 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4120002fe0 con 0x7f41381b3e40 2021-02-10T11:13:17.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.555 7f4136ffd700 1 -- 172.21.2.101:0/4061723620 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2043763914 0 0) 0x7f4128001140 con 0x7f41381b3e40 2021-02-10T11:13:17.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.555 7f4136ffd700 1 -- 172.21.2.101:0/4061723620 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41380e5070 msgr2=0x7f41381b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:17.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.555 7f4136ffd700 1 --2- 172.21.2.101:0/4061723620 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41380e5070 0x7f41381b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:17.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.555 7f4136ffd700 1 -- 172.21.2.101:0/4061723620 >> v1:172.21.2.108:6789/0 conn(0x7f41380e5470 legacy=0x7f413807d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:17.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.555 7f4136ffd700 1 -- 172.21.2.101:0/4061723620 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413807eb30 con 0x7f41381b3e40 2021-02-10T11:13:17.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.555 7f4136ffd700 1 -- 172.21.2.101:0/4061723620 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f4128001640 con 0x7f41381b3e40 2021-02-10T11:13:17.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.555 7f4136ffd700 1 -- 172.21.2.101:0/4061723620 >> v1:172.21.2.101:6789/0 conn(0x7f41381b3e40 legacy=0x7f413807de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:17.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.555 7f4136ffd700 1 --2- 172.21.2.101:0/4061723620 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4120003aa0 0x7f4120005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:17.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.556 7f4136ffd700 1 --2- 172.21.2.101:0/4061723620 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4120006790 0x7f4120008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:17.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.556 7f4136ffd700 1 --2- 172.21.2.101:0/4061723620 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4120009590 0x7f412000ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:17.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.556 7f4136ffd700 1 -- 172.21.2.101:0/4061723620 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412000c2d0 con 0x7f4120003aa0 2021-02-10T11:13:17.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.556 7f4136ffd700 1 -- 172.21.2.101:0/4061723620 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412000c4f0 con 0x7f4120006790 2021-02-10T11:13:17.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.556 7f4136ffd700 1 -- 172.21.2.101:0/4061723620 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4120050720 con 0x7f4120009590 2021-02-10T11:13:17.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.556 7f4137fff700 1 --2- 172.21.2.101:0/4061723620 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4120003aa0 0x7f4120005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:17.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.556 7f41377fe700 1 --2- 172.21.2.101:0/4061723620 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4120006790 0x7f4120008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:17.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.556 7f413cd5f700 1 --2- 172.21.2.101:0/4061723620 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4120009590 0x7f412000ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:17.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.556 7f4137fff700 1 -- 172.21.2.101:0/4061723620 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4120009590 msgr2=0x7f412000ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:17.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.556 7f4137fff700 1 --2- 172.21.2.101:0/4061723620 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4120009590 0x7f412000ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:17.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.556 7f4137fff700 1 -- 172.21.2.101:0/4061723620 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4120006790 msgr2=0x7f4120008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:17.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.556 7f4137fff700 1 --2- 172.21.2.101:0/4061723620 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4120006790 0x7f4120008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:17.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.556 7f4137fff700 1 -- 172.21.2.101:0/4061723620 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4120050950 con 0x7f4120003aa0 2021-02-10T11:13:17.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.556 7f41377fe700 1 --2- 172.21.2.101:0/4061723620 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4120006790 0x7f4120008c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:13:17.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.556 7f4137fff700 1 --2- 172.21.2.101:0/4061723620 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4120003aa0 0x7f4120005ef0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f4130006f80 tx=0x7f4130009e20).ready entity=mon.0 client_cookie=797a98f7d9b8e48a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:17.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.556 7f4136ffd700 1 -- 172.21.2.101:0/4061723620 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f413000b510 con 0x7f4120003aa0 2021-02-10T11:13:17.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.556 7f4136ffd700 1 -- 172.21.2.101:0/4061723620 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413000b790 con 0x7f4120003aa0 2021-02-10T11:13:17.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.556 7f4136ffd700 1 -- 172.21.2.101:0/4061723620 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f413000bc70 con 0x7f4120003aa0 2021-02-10T11:13:17.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.556 7f413e742700 1 -- 172.21.2.101:0/4061723620 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4120003aa0 msgr2=0x7f4120005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:17.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.556 7f413e742700 1 --2- 172.21.2.101:0/4061723620 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4120003aa0 0x7f4120005ef0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f4130006f80 tx=0x7f4130009e20).stop 2021-02-10T11:13:17.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.556 7f413cd5f700 1 -- 172.21.2.101:0/4061723620 reap_dead start 2021-02-10T11:13:17.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.557 7f413e742700 1 -- 172.21.2.101:0/4061723620 shutdown_connections 2021-02-10T11:13:17.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.557 7f413e742700 1 -- 172.21.2.101:0/4061723620 >> 172.21.2.101:0/4061723620 conn(0x7f41380ce690 msgr2=0x7f41380e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:17.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.557 7f413e742700 1 -- 172.21.2.101:0/4061723620 shutdown_connections 2021-02-10T11:13:17.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.557 7f413e742700 1 -- 172.21.2.101:0/4061723620 wait complete. 2021-02-10T11:13:17.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.572 7f413e742700 1 Processor -- start 2021-02-10T11:13:17.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.572 7f413e742700 1 -- start start 2021-02-10T11:13:17.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.572 7f413e742700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4138085a30 0x7f41380b8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:17.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.572 7f413e742700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4138198a40 0x7f41380b8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:17.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.572 7f413e742700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f413818a220 0x7f413818a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:17.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.572 7f413e742700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4138189ca0 con 0x7f4138198a40 2021-02-10T11:13:17.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.572 7f413e742700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41380b8db0 con 0x7f4138085a30 2021-02-10T11:13:17.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.572 7f413e742700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41380b8fd0 con 0x7f413818a220 2021-02-10T11:13:17.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.572 7f41377fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4138198a40 0x7f41380b8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:17.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.572 7f41377fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4138198a40 0x7f41380b8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43952/0 (socket says 172.21.2.101:43952) 2021-02-10T11:13:17.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.572 7f41377fe700 1 -- 172.21.2.101:0/479311644 learned_addr learned my addr 172.21.2.101:0/479311644 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:17.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.573 7f41377fe700 1 -- 172.21.2.101:0/479311644 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f413818a220 msgr2=0x7f413818a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:17.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.573 7f41377fe700 1 --2- 172.21.2.101:0/479311644 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f413818a220 0x7f413818a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:17.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.573 7f41377fe700 1 -- 172.21.2.101:0/479311644 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4138085a30 msgr2=0x7f41380b8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:17.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.573 7f41377fe700 1 --2- 172.21.2.101:0/479311644 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4138085a30 0x7f41380b8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:17.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.573 7f41377fe700 1 -- 172.21.2.101:0/479311644 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413818ac40 con 0x7f4138198a40 2021-02-10T11:13:17.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.573 7f41377fe700 1 --2- 172.21.2.101:0/479311644 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4138198a40 0x7f41380b8790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f412c006f30 tx=0x7f412c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:17.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.573 7f4134ff9700 1 -- 172.21.2.101:0/479311644 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f412c006bc0 con 0x7f4138198a40 2021-02-10T11:13:17.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.573 7f4134ff9700 1 -- 172.21.2.101:0/479311644 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412c00bd80 con 0x7f4138198a40 2021-02-10T11:13:17.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.573 7f413e742700 1 -- 172.21.2.101:0/479311644 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f413818ac40 con 0x7f4138198a40 2021-02-10T11:13:17.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.573 7f4134ff9700 1 -- 172.21.2.101:0/479311644 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f412c002220 con 0x7f4138198a40 2021-02-10T11:13:17.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.573 7f413e742700 1 -- 172.21.2.101:0/479311644 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f413818ac40 con 0x7f4138198a40 2021-02-10T11:13:17.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.574 7f4134ff9700 1 -- 172.21.2.101:0/479311644 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f412c002020 con 0x7f4138198a40 2021-02-10T11:13:17.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.574 7f4134ff9700 1 --2- 172.21.2.101:0/479311644 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f41180186c0 0x7f411801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:17.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.574 7f4134ff9700 1 -- 172.21.2.101:0/479311644 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f412c0327a0 con 0x7f4138198a40 2021-02-10T11:13:17.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.574 7f4137fff700 1 --2- 172.21.2.101:0/479311644 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f41180186c0 0x7f411801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:17.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.574 7f4137fff700 1 --2- 172.21.2.101:0/479311644 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f41180186c0 0x7f411801abc0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:17.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.604 7f413e742700 1 -- 172.21.2.101:0/479311644 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4138084b20 con 0x7f4138198a40 2021-02-10T11:13:17.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.607 7f4134ff9700 1 -- 172.21.2.101:0/479311644 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f412c006bc0 con 0x7f4138198a40 2021-02-10T11:13:17.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.732 7f413e742700 1 -- 172.21.2.101:0/479311644 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4138084b20 con 0x7f4138198a40 2021-02-10T11:13:17.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.732 7f4134ff9700 1 -- 172.21.2.101:0/479311644 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+13993 (secure 0 0 0) 0x7f412c002000 con 0x7f4138198a40 2021-02-10T11:13:17.737 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:13:17.737 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":20,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:13:15.149234","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:13:11.922272","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2021-02-10 11:13:13.141733","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:13:17.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.734 7f413e742700 1 -- 172.21.2.101:0/479311644 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f41180186c0 msgr2=0x7f411801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:17.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.734 7f413e742700 1 --2- 172.21.2.101:0/479311644 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f41180186c0 0x7f411801abc0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:17.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.734 7f413e742700 1 -- 172.21.2.101:0/479311644 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4138198a40 msgr2=0x7f41380b8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:17.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.734 7f413e742700 1 --2- 172.21.2.101:0/479311644 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4138198a40 0x7f41380b8790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f412c006f30 tx=0x7f412c00b990).stop 2021-02-10T11:13:17.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.734 7f413e742700 1 -- 172.21.2.101:0/479311644 shutdown_connections 2021-02-10T11:13:17.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.734 7f413e742700 1 --2- 172.21.2.101:0/479311644 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f41180186c0 0x7f411801abc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:17.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.734 7f413e742700 1 --2- 172.21.2.101:0/479311644 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f413818a220 0x7f413818a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:17.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.734 7f413e742700 1 --2- 172.21.2.101:0/479311644 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4138198a40 0x7f41380b8790 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:17.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.734 7f413e742700 1 --2- 172.21.2.101:0/479311644 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4138085a30 0x7f41380b8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:17.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.734 7f413e742700 1 -- 172.21.2.101:0/479311644 >> 172.21.2.101:0/479311644 conn(0x7f41380855a0 msgr2=0x7f41381996d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:17.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.734 7f413e742700 1 -- 172.21.2.101:0/479311644 shutdown_connections 2021-02-10T11:13:17.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:17.735 7f413e742700 1 -- 172.21.2.101:0/479311644 wait complete. 2021-02-10T11:13:17.747 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2021-02-10T11:13:17.747 INFO:tasks.ceph.mds.a:Restarting daemon 2021-02-10T11:13:17.747 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-02-10T11:13:17.750 INFO:tasks.ceph.mds.a:Started 2021-02-10T11:13:17.750 INFO:tasks.ceph.mds.b:Restarting daemon 2021-02-10T11:13:17.750 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-02-10T11:13:17.754 INFO:tasks.ceph.mds.b:Started 2021-02-10T11:13:17.754 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:13:18.157 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:13:18.158 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:13:18.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.155 7f0bb8e0a700 1 Processor -- start 2021-02-10T11:13:18.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.155 7f0bb8e0a700 1 -- start start 2021-02-10T11:13:18.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.156 7f0bb8e0a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0bb407e6a0 0x7f0bb40d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:18.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.156 7f0bb8e0a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb407ed80 con 0x7f0bb40e5470 2021-02-10T11:13:18.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.156 7f0bb8e0a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb407eaa0 con 0x7f0bb40e5070 2021-02-10T11:13:18.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.156 7f0bb8e0a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bb40e6090 con 0x7f0bb407e6a0 2021-02-10T11:13:18.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.156 7f0bb3314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0bb407e6a0 0x7f0bb40d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:18.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.156 7f0bb3314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0bb407e6a0 0x7f0bb40d1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43976/0 (socket says 172.21.2.101:43976) 2021-02-10T11:13:18.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.156 7f0bb3314700 1 -- 172.21.2.101:0/1279545029 learned_addr learned my addr 172.21.2.101:0/1279545029 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:18.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.156 7f0bb3314700 1 -- 172.21.2.101:0/1279545029 >> v1:172.21.2.108:6789/0 conn(0x7f0bb40e5070 legacy=0x7f0bb41b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:18.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.156 7f0bb3314700 1 -- 172.21.2.101:0/1279545029 >> v1:172.21.2.101:6789/0 conn(0x7f0bb40e5470 legacy=0x7f0bb407f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:18.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.156 7f0bb3314700 1 -- 172.21.2.101:0/1279545029 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb41b4040 con 0x7f0bb407e6a0 2021-02-10T11:13:18.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.156 7f0bb3314700 1 --2- 172.21.2.101:0/1279545029 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0bb407e6a0 0x7f0bb40d1790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f0ba4006b80 tx=0x7f0ba4009b90).ready entity=mon.0 client_cookie=a689251e51a5ccaf server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:18.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.157 7f0bb1b11700 1 -- 172.21.2.101:0/1279545029 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0ba40022e0 con 0x7f0bb407e6a0 2021-02-10T11:13:18.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.157 7f0bb1b11700 1 -- 172.21.2.101:0/1279545029 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba4002620 con 0x7f0bb407e6a0 2021-02-10T11:13:18.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.157 7f0bb1b11700 1 -- 172.21.2.101:0/1279545029 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0ba4002b00 con 0x7f0bb407e6a0 2021-02-10T11:13:18.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.157 7f0bb8e0a700 1 -- 172.21.2.101:0/1279545029 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0bb407e6a0 msgr2=0x7f0bb40d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:18.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.157 7f0bb8e0a700 1 --2- 172.21.2.101:0/1279545029 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0bb407e6a0 0x7f0bb40d1790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f0ba4006b80 tx=0x7f0ba4009b90).stop 2021-02-10T11:13:18.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.157 7f0bb8e0a700 1 -- 172.21.2.101:0/1279545029 shutdown_connections 2021-02-10T11:13:18.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.157 7f0bb8e0a700 1 --2- 172.21.2.101:0/1279545029 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0bb407e6a0 0x7f0bb40d1790 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:18.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.157 7f0bb8e0a700 1 -- 172.21.2.101:0/1279545029 >> 172.21.2.101:0/1279545029 conn(0x7f0bb40ce690 msgr2=0x7f0bb40e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:18.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.157 7f0bb8e0a700 1 -- 172.21.2.101:0/1279545029 shutdown_connections 2021-02-10T11:13:18.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.157 7f0bb8e0a700 1 -- 172.21.2.101:0/1279545029 wait complete. 2021-02-10T11:13:18.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.172 7f0bb8e0a700 1 Processor -- start 2021-02-10T11:13:18.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.172 7f0bb8e0a700 1 -- start start 2021-02-10T11:13:18.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.172 7f0bb8e0a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0bb409e320 0x7f0bb40a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:18.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.172 7f0bb8e0a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0bb4108230 0x7f0bb40b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:18.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.172 7f0bb8e0a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0bb4108630 0x7f0bb40a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:18.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.172 7f0bb8e0a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb40a0060 con 0x7f0bb4108630 2021-02-10T11:13:18.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.172 7f0bb8e0a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb40b77a0 con 0x7f0bb409e320 2021-02-10T11:13:18.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.172 7f0bb8e0a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb40b7980 con 0x7f0bb4108230 2021-02-10T11:13:18.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.173 7f0bb3314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0bb4108630 0x7f0bb40a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:18.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.173 7f0bb3314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0bb4108630 0x7f0bb40a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43982/0 (socket says 172.21.2.101:43982) 2021-02-10T11:13:18.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.173 7f0bb3314700 1 -- 172.21.2.101:0/1264738146 learned_addr learned my addr 172.21.2.101:0/1264738146 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:18.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.173 7f0bb2b13700 1 --2- 172.21.2.101:0/1264738146 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0bb409e320 0x7f0bb40a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:18.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.173 7f0bb2312700 1 --2- 172.21.2.101:0/1264738146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0bb4108230 0x7f0bb40b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:18.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.173 7f0bb3314700 1 -- 172.21.2.101:0/1264738146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0bb4108230 msgr2=0x7f0bb40b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:18.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.173 7f0bb3314700 1 --2- 172.21.2.101:0/1264738146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0bb4108230 0x7f0bb40b7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:18.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.173 7f0bb3314700 1 -- 172.21.2.101:0/1264738146 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0bb409e320 msgr2=0x7f0bb40a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:18.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.173 7f0bb3314700 1 --2- 172.21.2.101:0/1264738146 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0bb409e320 0x7f0bb40a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:18.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.173 7f0bb3314700 1 -- 172.21.2.101:0/1264738146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb40a0c00 con 0x7f0bb4108630 2021-02-10T11:13:18.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.173 7f0bb3314700 1 --2- 172.21.2.101:0/1264738146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0bb4108630 0x7f0bb40a05e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f0ba4006f10 tx=0x7f0ba400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:18.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.173 7f0ba37fe700 1 -- 172.21.2.101:0/1264738146 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0ba4009100 con 0x7f0bb4108630 2021-02-10T11:13:18.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.174 7f0ba37fe700 1 -- 172.21.2.101:0/1264738146 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba4000b90 con 0x7f0bb4108630 2021-02-10T11:13:18.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.174 7f0bb8e0a700 1 -- 172.21.2.101:0/1264738146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb40a0c00 con 0x7f0bb4108630 2021-02-10T11:13:18.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.174 7f0ba37fe700 1 -- 172.21.2.101:0/1264738146 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0ba4002080 con 0x7f0bb4108630 2021-02-10T11:13:18.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.174 7f0bb8e0a700 1 -- 172.21.2.101:0/1264738146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb409d650 con 0x7f0bb4108630 2021-02-10T11:13:18.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.174 7f0ba37fe700 1 -- 172.21.2.101:0/1264738146 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0ba4002080 con 0x7f0bb4108630 2021-02-10T11:13:18.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.174 7f0ba37fe700 1 --2- 172.21.2.101:0/1264738146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b980186c0 0x7f0b9801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:18.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.174 7f0ba37fe700 1 -- 172.21.2.101:0/1264738146 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0ba4032560 con 0x7f0bb4108630 2021-02-10T11:13:18.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.175 7f0bb2b13700 1 --2- 172.21.2.101:0/1264738146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b980186c0 0x7f0b9801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:18.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.175 7f0bb2b13700 1 --2- 172.21.2.101:0/1264738146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b980186c0 0x7f0b9801abc0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:18.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.205 7f0bb8e0a700 1 -- 172.21.2.101:0/1264738146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb40891c0 con 0x7f0bb4108630 2021-02-10T11:13:18.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.208 7f0ba37fe700 1 -- 172.21.2.101:0/1264738146 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0ba4022390 con 0x7f0bb4108630 2021-02-10T11:13:18.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.336 7f0bb8e0a700 1 -- 172.21.2.101:0/1264738146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bb40ce690 con 0x7f0bb4108630 2021-02-10T11:13:18.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.336 7f0ba37fe700 1 -- 172.21.2.101:0/1264738146 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+13993 (secure 0 0 0) 0x7f0ba4022390 con 0x7f0bb4108630 2021-02-10T11:13:18.341 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:13:18.341 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":20,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:13:15.149234","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:13:11.922272","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2021-02-10 11:13:13.141733","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:13:18.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.338 7f0bb8e0a700 1 -- 172.21.2.101:0/1264738146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b980186c0 msgr2=0x7f0b9801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:18.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.338 7f0bb8e0a700 1 --2- 172.21.2.101:0/1264738146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b980186c0 0x7f0b9801abc0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:18.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.338 7f0bb8e0a700 1 -- 172.21.2.101:0/1264738146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0bb4108630 msgr2=0x7f0bb40a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:18.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.338 7f0bb8e0a700 1 --2- 172.21.2.101:0/1264738146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0bb4108630 0x7f0bb40a05e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f0ba4006f10 tx=0x7f0ba400b960).stop 2021-02-10T11:13:18.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.338 7f0bb8e0a700 1 -- 172.21.2.101:0/1264738146 shutdown_connections 2021-02-10T11:13:18.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.338 7f0bb8e0a700 1 --2- 172.21.2.101:0/1264738146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b980186c0 0x7f0b9801abc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:18.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.338 7f0bb8e0a700 1 --2- 172.21.2.101:0/1264738146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0bb4108230 0x7f0bb40b7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:18.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.338 7f0bb8e0a700 1 --2- 172.21.2.101:0/1264738146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0bb4108630 0x7f0bb40a05e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:18.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.338 7f0bb8e0a700 1 --2- 172.21.2.101:0/1264738146 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0bb409e320 0x7f0bb40a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:18.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.338 7f0bb8e0a700 1 -- 172.21.2.101:0/1264738146 >> 172.21.2.101:0/1264738146 conn(0x7f0bb4094440 msgr2=0x7f0bb40ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:18.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.338 7f0bb8e0a700 1 -- 172.21.2.101:0/1264738146 shutdown_connections 2021-02-10T11:13:18.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.338 7f0bb8e0a700 1 -- 172.21.2.101:0/1264738146 wait complete. 2021-02-10T11:13:18.351 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:13:11.922272', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:13:13.141733', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:13:18.351 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:13:18.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.760 7fbebc430700 1 Processor -- start 2021-02-10T11:13:18.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.760 7fbebc430700 1 -- start start 2021-02-10T11:13:18.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.761 7fbebc430700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbeb41b0470 0x7fbeb41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:18.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.761 7fbebc430700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbeb40f20e0 0x7fbeb40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:18.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.761 7fbebc430700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb40f0b50 con 0x7fbeb40ef610 2021-02-10T11:13:18.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.761 7fbebc430700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb40f0820 con 0x7fbeb41b0470 2021-02-10T11:13:18.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.761 7fbebc430700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb40f24e0 con 0x7fbeb40f20e0 2021-02-10T11:13:18.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.761 7fbeba24c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbeb41b0470 0x7fbeb41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:18.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.761 7fbeba24c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbeb41b0470 0x7fbeb41b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43986/0 (socket says 172.21.2.101:43986) 2021-02-10T11:13:18.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.761 7fbeba24c700 1 -- 172.21.2.101:0/1238205077 learned_addr learned my addr 172.21.2.101:0/1238205077 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:18.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.761 7fbeba24c700 1 -- 172.21.2.101:0/1238205077 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbeb40f20e0 msgr2=0x7fbeb40f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:18.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.761 7fbeba24c700 1 --2- 172.21.2.101:0/1238205077 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbeb40f20e0 0x7fbeb40f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:18.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.761 7fbeba24c700 1 -- 172.21.2.101:0/1238205077 >> v1:172.21.2.108:6789/0 conn(0x7fbeb40ef610 legacy=0x7fbeb40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:18.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.761 7fbeba24c700 1 -- 172.21.2.101:0/1238205077 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb40e52e0 con 0x7fbeb41b0470 2021-02-10T11:13:18.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.761 7fbeba24c700 1 --2- 172.21.2.101:0/1238205077 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbeb41b0470 0x7fbeb41b0870 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fbeb0006700 tx=0x7fbeb0009b80).ready entity=mon.0 client_cookie=4ed418cfd202f395 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:18.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.762 7fbeb924a700 1 -- 172.21.2.101:0/1238205077 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbeb000c0c0 con 0x7fbeb41b0470 2021-02-10T11:13:18.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.762 7fbeb924a700 1 -- 172.21.2.101:0/1238205077 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb000c440 con 0x7fbeb41b0470 2021-02-10T11:13:18.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.762 7fbeb924a700 1 -- 172.21.2.101:0/1238205077 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbeb000c960 con 0x7fbeb41b0470 2021-02-10T11:13:18.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.762 7fbebc430700 1 -- 172.21.2.101:0/1238205077 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbeb41b0470 msgr2=0x7fbeb41b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:18.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.762 7fbebc430700 1 --2- 172.21.2.101:0/1238205077 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbeb41b0470 0x7fbeb41b0870 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fbeb0006700 tx=0x7fbeb0009b80).stop 2021-02-10T11:13:18.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.762 7fbebc430700 1 -- 172.21.2.101:0/1238205077 shutdown_connections 2021-02-10T11:13:18.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.762 7fbebc430700 1 --2- 172.21.2.101:0/1238205077 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbeb40f20e0 0x7fbeb40f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:18.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.762 7fbebc430700 1 --2- 172.21.2.101:0/1238205077 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbeb41b0470 0x7fbeb41b0870 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:18.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.762 7fbebc430700 1 -- 172.21.2.101:0/1238205077 >> 172.21.2.101:0/1238205077 conn(0x7fbeb40ce3e0 msgr2=0x7fbeb40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:18.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.763 7fbebc430700 1 -- 172.21.2.101:0/1238205077 shutdown_connections 2021-02-10T11:13:18.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.763 7fbebc430700 1 -- 172.21.2.101:0/1238205077 wait complete. 2021-02-10T11:13:18.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.777 7fbebc430700 1 Processor -- start 2021-02-10T11:13:18.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.777 7fbebc430700 1 -- start start 2021-02-10T11:13:18.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.777 7fbebc430700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbeb40a6c90 0x7fbeb41924c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:18.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.778 7fbebc430700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbeb4192ae0 0x7fbeb4192ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:18.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.778 7fbebc430700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbeb40919e0 0x7fbeb4091de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:18.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.778 7fbebc430700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb419d2e0 con 0x7fbeb4192ae0 2021-02-10T11:13:18.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.778 7fbebc430700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb40914d0 con 0x7fbeb40919e0 2021-02-10T11:13:18.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.778 7fbebc430700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb40a6820 con 0x7fbeb40a6c90 2021-02-10T11:13:18.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.778 7fbeb9a4b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbeb4192ae0 0x7fbeb4192ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:18.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.778 7fbeb9a4b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbeb4192ae0 0x7fbeb4192ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:43992/0 (socket says 172.21.2.101:43992) 2021-02-10T11:13:18.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.778 7fbeb9a4b700 1 -- 172.21.2.101:0/3519511569 learned_addr learned my addr 172.21.2.101:0/3519511569 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:18.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.778 7fbeb9a4b700 1 -- 172.21.2.101:0/3519511569 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbeb40a6c90 msgr2=0x7fbeb41924c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:18.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.778 7fbeb9a4b700 1 --2- 172.21.2.101:0/3519511569 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbeb40a6c90 0x7fbeb41924c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:18.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.778 7fbeb9a4b700 1 -- 172.21.2.101:0/3519511569 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbeb40919e0 msgr2=0x7fbeb4091de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:18.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.778 7fbeb9a4b700 1 --2- 172.21.2.101:0/3519511569 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbeb40919e0 0x7fbeb4091de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:18.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.778 7fbeb9a4b700 1 -- 172.21.2.101:0/3519511569 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb4092400 con 0x7fbeb4192ae0 2021-02-10T11:13:18.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.778 7fbeb9a4b700 1 --2- 172.21.2.101:0/3519511569 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbeb4192ae0 0x7fbeb4192ee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fbea800b530 tx=0x7fbea800b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:18.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.778 7fbea6ffd700 1 -- 172.21.2.101:0/3519511569 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbea800bcc0 con 0x7fbeb4192ae0 2021-02-10T11:13:18.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.778 7fbea6ffd700 1 -- 172.21.2.101:0/3519511569 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea8000e20 con 0x7fbeb4192ae0 2021-02-10T11:13:18.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.778 7fbebc430700 1 -- 172.21.2.101:0/3519511569 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb4092400 con 0x7fbeb4192ae0 2021-02-10T11:13:18.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.778 7fbea6ffd700 1 -- 172.21.2.101:0/3519511569 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbea8006210 con 0x7fbeb4192ae0 2021-02-10T11:13:18.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.778 7fbebc430700 1 -- 172.21.2.101:0/3519511569 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb4092990 con 0x7fbeb4192ae0 2021-02-10T11:13:18.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.779 7fbea6ffd700 1 -- 172.21.2.101:0/3519511569 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbea800a000 con 0x7fbeb4192ae0 2021-02-10T11:13:18.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.779 7fbea6ffd700 1 --2- 172.21.2.101:0/3519511569 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe9c018730 0x7fbe9c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:18.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.779 7fbea6ffd700 1 -- 172.21.2.101:0/3519511569 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fbea8032650 con 0x7fbeb4192ae0 2021-02-10T11:13:18.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.779 7fbeba24c700 1 --2- 172.21.2.101:0/3519511569 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe9c018730 0x7fbe9c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:18.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.779 7fbeba24c700 1 --2- 172.21.2.101:0/3519511569 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe9c018730 0x7fbe9c01ac10 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:18.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.810 7fbebc430700 1 -- 172.21.2.101:0/3519511569 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb40e52e0 con 0x7fbeb4192ae0 2021-02-10T11:13:18.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.813 7fbea6ffd700 1 -- 172.21.2.101:0/3519511569 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbea8000940 con 0x7fbeb4192ae0 2021-02-10T11:13:18.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.939 7fbebc430700 1 -- 172.21.2.101:0/3519511569 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fbeb40e52e0 con 0x7fbeb4192ae0 2021-02-10T11:13:18.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.939 7fbea6ffd700 1 -- 172.21.2.101:0/3519511569 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbea8000e00 con 0x7fbeb4192ae0 2021-02-10T11:13:18.944 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:13:18.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.941 7fbebc430700 1 -- 172.21.2.101:0/3519511569 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe9c018730 msgr2=0x7fbe9c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:18.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.941 7fbebc430700 1 --2- 172.21.2.101:0/3519511569 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe9c018730 0x7fbe9c01ac10 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:18.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.941 7fbebc430700 1 -- 172.21.2.101:0/3519511569 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbeb4192ae0 msgr2=0x7fbeb4192ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:18.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.941 7fbebc430700 1 --2- 172.21.2.101:0/3519511569 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbeb4192ae0 0x7fbeb4192ee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fbea800b530 tx=0x7fbea800b8d0).stop 2021-02-10T11:13:18.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.942 7fbebc430700 1 -- 172.21.2.101:0/3519511569 shutdown_connections 2021-02-10T11:13:18.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.942 7fbebc430700 1 --2- 172.21.2.101:0/3519511569 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbeb40919e0 0x7fbeb4091de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:18.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.942 7fbebc430700 1 --2- 172.21.2.101:0/3519511569 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbeb4192ae0 0x7fbeb4192ee0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:18.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.942 7fbebc430700 1 --2- 172.21.2.101:0/3519511569 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe9c018730 0x7fbe9c01ac10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:18.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.942 7fbebc430700 1 --2- 172.21.2.101:0/3519511569 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbeb40a6c90 0x7fbeb41924c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:18.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.942 7fbebc430700 1 -- 172.21.2.101:0/3519511569 >> 172.21.2.101:0/3519511569 conn(0x7fbeb40ac550 msgr2=0x7fbeb40be1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:18.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.942 7fbebc430700 1 -- 172.21.2.101:0/3519511569 shutdown_connections 2021-02-10T11:13:18.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:18.942 7fbebc430700 1 -- 172.21.2.101:0/3519511569 wait complete. 2021-02-10T11:13:18.954 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:13:19.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.366 7fd2def50700 1 Processor -- start 2021-02-10T11:13:19.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.366 7fd2def50700 1 -- start start 2021-02-10T11:13:19.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.366 7fd2def50700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd2d81b0470 0x7fd2d81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:19.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.366 7fd2def50700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d80f0b50 con 0x7fd2d80ef610 2021-02-10T11:13:19.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.366 7fd2def50700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d80f24e0 con 0x7fd2d80f20e0 2021-02-10T11:13:19.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.366 7fd2def50700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d80e5ea0 con 0x7fd2d81b0470 2021-02-10T11:13:19.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.367 7fd2d7fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd2d80ef610 0x7fd2d80efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44492/0 (socket says 172.21.2.101:44492) 2021-02-10T11:13:19.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.367 7fd2d7fff700 1 -- 172.21.2.101:0/2268675649 learned_addr learned my addr 172.21.2.101:0/2268675649 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:19.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.367 7fd2d77fe700 1 -- 172.21.2.101:0/2268675649 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1090408673 0 0) 0x7fd2c8001180 con 0x7fd2d80ef610 2021-02-10T11:13:19.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.367 7fd2d77fe700 1 -- 172.21.2.101:0/2268675649 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2c4002fe0 con 0x7fd2d80ef610 2021-02-10T11:13:19.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.367 7fd2d77fe700 1 -- 172.21.2.101:0/2268675649 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 444696417 0 0) 0x7fd2c8001180 con 0x7fd2d80ef610 2021-02-10T11:13:19.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.367 7fd2d77fe700 1 -- 172.21.2.101:0/2268675649 >> v2:172.21.2.108:3300/0 conn(0x7fd2d81b0470 msgr2=0x7fd2d81b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:19.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.367 7fd2d77fe700 1 --2- 172.21.2.101:0/2268675649 >> v2:172.21.2.108:3300/0 conn(0x7fd2d81b0470 0x7fd2d81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:19.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.367 7fd2d77fe700 1 -- 172.21.2.101:0/2268675649 >> v1:172.21.2.108:6789/0 conn(0x7fd2d80f20e0 legacy=0x7fd2d80f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:19.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.367 7fd2d77fe700 1 -- 172.21.2.101:0/2268675649 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d80e52e0 con 0x7fd2d80ef610 2021-02-10T11:13:19.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.367 7fd2d77fe700 1 -- 172.21.2.101:0/2268675649 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fd2c8001680 con 0x7fd2d80ef610 2021-02-10T11:13:19.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.367 7fd2d77fe700 1 -- 172.21.2.101:0/2268675649 >> v1:172.21.2.101:6789/0 conn(0x7fd2d80ef610 legacy=0x7fd2d80efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:19.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.367 7fd2d77fe700 1 --2- 172.21.2.101:0/2268675649 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2c4003af0 0x7fd2c4005f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:19.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.367 7fd2d77fe700 1 --2- 172.21.2.101:0/2268675649 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2c4006780 0x7fd2c4008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:19.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.367 7fd2d77fe700 1 --2- 172.21.2.101:0/2268675649 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2c4009580 0x7fd2c400baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:19.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.367 7fd2d77fe700 1 -- 172.21.2.101:0/2268675649 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c400c2f0 con 0x7fd2c4009580 2021-02-10T11:13:19.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.367 7fd2d77fe700 1 -- 172.21.2.101:0/2268675649 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c400c510 con 0x7fd2c4003af0 2021-02-10T11:13:19.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.367 7fd2d77fe700 1 -- 172.21.2.101:0/2268675649 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c400c730 con 0x7fd2c4006780 2021-02-10T11:13:19.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.368 7fd2dd56d700 1 --2- 172.21.2.101:0/2268675649 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2c4009580 0x7fd2c400baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:19.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.368 7fd2dcd6c700 1 --2- 172.21.2.101:0/2268675649 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2c4003af0 0x7fd2c4005f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:19.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.368 7fd2d7fff700 1 --2- 172.21.2.101:0/2268675649 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2c4006780 0x7fd2c4008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:19.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.368 7fd2dd56d700 1 -- 172.21.2.101:0/2268675649 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2c4006780 msgr2=0x7fd2c4008c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:19.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.368 7fd2dd56d700 1 --2- 172.21.2.101:0/2268675649 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2c4006780 0x7fd2c4008c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:19.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.368 7fd2dd56d700 1 -- 172.21.2.101:0/2268675649 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2c4003af0 msgr2=0x7fd2c4005f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:19.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.368 7fd2dd56d700 1 --2- 172.21.2.101:0/2268675649 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2c4003af0 0x7fd2c4005f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:19.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.368 7fd2dd56d700 1 -- 172.21.2.101:0/2268675649 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c400c9d0 con 0x7fd2c4009580 2021-02-10T11:13:19.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.368 7fd2d7fff700 1 --2- 172.21.2.101:0/2268675649 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2c4006780 0x7fd2c4008c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:13:19.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.368 7fd2dd56d700 1 -- 172.21.2.101:0/2268675649 reap_dead start 2021-02-10T11:13:19.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.368 7fd2dd56d700 1 --2- 172.21.2.101:0/2268675649 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2c4009580 0x7fd2c400baa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fd2d0006e30 tx=0x7fd2d004fed0).ready entity=mon.0 client_cookie=7c57b0bbd0cb1222 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:19.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.368 7fd2d77fe700 1 -- 172.21.2.101:0/2268675649 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2d0003020 con 0x7fd2c4009580 2021-02-10T11:13:19.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.368 7fd2d77fe700 1 -- 172.21.2.101:0/2268675649 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d0001400 con 0x7fd2c4009580 2021-02-10T11:13:19.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.368 7fd2d77fe700 1 -- 172.21.2.101:0/2268675649 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2d0006a70 con 0x7fd2c4009580 2021-02-10T11:13:19.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.368 7fd2def50700 1 -- 172.21.2.101:0/2268675649 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2c4009580 msgr2=0x7fd2c400baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:19.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.368 7fd2def50700 1 --2- 172.21.2.101:0/2268675649 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2c4009580 0x7fd2c400baa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fd2d0006e30 tx=0x7fd2d004fed0).stop 2021-02-10T11:13:19.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.369 7fd2def50700 1 -- 172.21.2.101:0/2268675649 shutdown_connections 2021-02-10T11:13:19.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.369 7fd2def50700 1 --2- 172.21.2.101:0/2268675649 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2c4009580 0x7fd2c400baa0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:19.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.369 7fd2def50700 1 -- 172.21.2.101:0/2268675649 >> 172.21.2.101:0/2268675649 conn(0x7fd2d80ce3e0 msgr2=0x7fd2d80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:19.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.369 7fd2def50700 1 -- 172.21.2.101:0/2268675649 shutdown_connections 2021-02-10T11:13:19.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.369 7fd2def50700 1 -- 172.21.2.101:0/2268675649 wait complete. 2021-02-10T11:13:19.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.383 7fd2def50700 1 Processor -- start 2021-02-10T11:13:19.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.383 7fd2def50700 1 -- start start 2021-02-10T11:13:19.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.384 7fd2def50700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2d80b7db0 0x7fd2d80b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:19.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.384 7fd2def50700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2d80b8880 0x7fd2d81a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:19.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.384 7fd2def50700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2d80b8c80 0x7fd2d81a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:19.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.384 7fd2def50700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d80b79b0 con 0x7fd2d80b7db0 2021-02-10T11:13:19.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.384 7fd2def50700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d81a8090 con 0x7fd2d80b8c80 2021-02-10T11:13:19.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.384 7fd2def50700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d81a82b0 con 0x7fd2d80b8880 2021-02-10T11:13:19.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.384 7fd2dcd6c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2d80b7db0 0x7fd2d80b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:19.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.384 7fd2dcd6c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2d80b7db0 0x7fd2d80b8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44014/0 (socket says 172.21.2.101:44014) 2021-02-10T11:13:19.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.384 7fd2dcd6c700 1 -- 172.21.2.101:0/874052835 learned_addr learned my addr 172.21.2.101:0/874052835 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:19.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.384 7fd2dd56d700 1 --2- 172.21.2.101:0/874052835 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2d80b8c80 0x7fd2d81a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:19.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.384 7fd2d7fff700 1 --2- 172.21.2.101:0/874052835 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2d80b8880 0x7fd2d81a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:19.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.384 7fd2dcd6c700 1 -- 172.21.2.101:0/874052835 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2d80b8880 msgr2=0x7fd2d81a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:19.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.384 7fd2dcd6c700 1 --2- 172.21.2.101:0/874052835 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2d80b8880 0x7fd2d81a7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:19.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.384 7fd2dcd6c700 1 -- 172.21.2.101:0/874052835 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2d80b8c80 msgr2=0x7fd2d81a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:19.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.385 7fd2dcd6c700 1 --2- 172.21.2.101:0/874052835 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2d80b8c80 0x7fd2d81a7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:19.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.385 7fd2dcd6c700 1 -- 172.21.2.101:0/874052835 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d8190c10 con 0x7fd2d80b7db0 2021-02-10T11:13:19.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.385 7fd2dd56d700 1 --2- 172.21.2.101:0/874052835 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2d80b8c80 0x7fd2d81a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:13:19.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.385 7fd2dcd6c700 1 --2- 172.21.2.101:0/874052835 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2d80b7db0 0x7fd2d80b8260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fd2c000b590 tx=0x7fd2c000b930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:19.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.385 7fd2d57fa700 1 -- 172.21.2.101:0/874052835 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2c000bd20 con 0x7fd2d80b7db0 2021-02-10T11:13:19.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.385 7fd2d57fa700 1 -- 172.21.2.101:0/874052835 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c0006030 con 0x7fd2d80b7db0 2021-02-10T11:13:19.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.385 7fd2d57fa700 1 -- 172.21.2.101:0/874052835 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2c0005520 con 0x7fd2d80b7db0 2021-02-10T11:13:19.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.385 7fd2def50700 1 -- 172.21.2.101:0/874052835 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d8190c10 con 0x7fd2d80b7db0 2021-02-10T11:13:19.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.385 7fd2def50700 1 -- 172.21.2.101:0/874052835 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d80c1eb0 con 0x7fd2d80b7db0 2021-02-10T11:13:19.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.386 7fd2d57fa700 1 -- 172.21.2.101:0/874052835 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd2c0006210 con 0x7fd2d80b7db0 2021-02-10T11:13:19.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.386 7fd2d57fa700 1 --2- 172.21.2.101:0/874052835 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2bc0186c0 0x7fd2bc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:19.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.386 7fd2d57fa700 1 -- 172.21.2.101:0/874052835 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd2c00325a0 con 0x7fd2d80b7db0 2021-02-10T11:13:19.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.386 7fd2d7fff700 1 --2- 172.21.2.101:0/874052835 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2bc0186c0 0x7fd2bc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:19.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.386 7fd2d7fff700 1 --2- 172.21.2.101:0/874052835 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2bc0186c0 0x7fd2bc01abc0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:19.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.416 7fd2def50700 1 -- 172.21.2.101:0/874052835 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d8105b30 con 0x7fd2d80b7db0 2021-02-10T11:13:19.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.419 7fd2d57fa700 1 -- 172.21.2.101:0/874052835 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd2c0000940 con 0x7fd2d80b7db0 2021-02-10T11:13:19.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.546 7fd2def50700 1 -- 172.21.2.101:0/874052835 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7fd2d80f0b50 con 0x7fd2d80b7db0 2021-02-10T11:13:19.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.547 7fd2d57fa700 1 -- 172.21.2.101:0/874052835 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v20) v1 ==== 104+0+10 (secure 0 0 0) 0x7fd2c00061d0 con 0x7fd2d80b7db0 2021-02-10T11:13:19.551 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:13:19.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.549 7fd2def50700 1 -- 172.21.2.101:0/874052835 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2bc0186c0 msgr2=0x7fd2bc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:19.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.549 7fd2def50700 1 --2- 172.21.2.101:0/874052835 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2bc0186c0 0x7fd2bc01abc0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:19.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.549 7fd2def50700 1 -- 172.21.2.101:0/874052835 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2d80b7db0 msgr2=0x7fd2d80b8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:19.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.549 7fd2def50700 1 --2- 172.21.2.101:0/874052835 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2d80b7db0 0x7fd2d80b8260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fd2c000b590 tx=0x7fd2c000b930).stop 2021-02-10T11:13:19.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.549 7fd2def50700 1 -- 172.21.2.101:0/874052835 shutdown_connections 2021-02-10T11:13:19.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.549 7fd2def50700 1 --2- 172.21.2.101:0/874052835 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2bc0186c0 0x7fd2bc01abc0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:19.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.549 7fd2def50700 1 --2- 172.21.2.101:0/874052835 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2d80b8880 0x7fd2d81a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:19.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.549 7fd2def50700 1 --2- 172.21.2.101:0/874052835 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2d80b8c80 0x7fd2d81a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:19.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.549 7fd2def50700 1 --2- 172.21.2.101:0/874052835 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2d80b7db0 0x7fd2d80b8260 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:19.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.549 7fd2def50700 1 -- 172.21.2.101:0/874052835 >> 172.21.2.101:0/874052835 conn(0x7fd2d8094e10 msgr2=0x7fd2d80a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:19.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.549 7fd2def50700 1 -- 172.21.2.101:0/874052835 shutdown_connections 2021-02-10T11:13:19.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.549 7fd2def50700 1 -- 172.21.2.101:0/874052835 wait complete. 2021-02-10T11:13:19.561 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:13:19.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.971 7f5163166700 1 Processor -- start 2021-02-10T11:13:19.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.971 7f5163166700 1 -- start start 2021-02-10T11:13:19.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.971 7f5163166700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f515c1b0470 0x7f515c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:19.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.971 7f5163166700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f515c0f20e0 0x7f515c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:19.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.971 7f5163166700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f515c0f0b50 con 0x7f515c0ef610 2021-02-10T11:13:19.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.971 7f5163166700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f515c0f0820 con 0x7f515c1b0470 2021-02-10T11:13:19.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.972 7f5163166700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f515c0f24e0 con 0x7f515c0f20e0 2021-02-10T11:13:19.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.972 7f5160f82700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f515c1b0470 0x7f515c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:19.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.972 7f515bfff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f515c0ef610 0x7f515c0efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44512/0 (socket says 172.21.2.101:44512) 2021-02-10T11:13:19.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.972 7f515bfff700 1 -- 172.21.2.101:0/3619725618 learned_addr learned my addr 172.21.2.101:0/3619725618 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:19.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.972 7f5160f82700 1 -- 172.21.2.101:0/3619725618 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f515c0f20e0 msgr2=0x7f515c0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:19.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.972 7f5160f82700 1 --2- 172.21.2.101:0/3619725618 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f515c0f20e0 0x7f515c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:19.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.972 7f5160f82700 1 -- 172.21.2.101:0/3619725618 >> v1:172.21.2.101:6789/0 conn(0x7f515c0ef610 legacy=0x7f515c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:19.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.972 7f515b7fe700 1 -- 172.21.2.101:0/3619725618 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1947917404 0 0) 0x7f514c001060 con 0x7f515c0ef610 2021-02-10T11:13:19.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.972 7f5160f82700 1 -- 172.21.2.101:0/3619725618 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515c0e52e0 con 0x7f515c1b0470 2021-02-10T11:13:19.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.973 7f5160f82700 1 --2- 172.21.2.101:0/3619725618 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f515c1b0470 0x7f515c1b0870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f5144006650 tx=0x7f5144009a00).ready entity=mon.0 client_cookie=b962ef79b727729e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:19.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.973 7f515b7fe700 1 -- 172.21.2.101:0/3619725618 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f51440009b0 con 0x7f515c1b0470 2021-02-10T11:13:19.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.973 7f515b7fe700 1 -- 172.21.2.101:0/3619725618 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514400c0c0 con 0x7f515c1b0470 2021-02-10T11:13:19.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.973 7f515b7fe700 1 -- 172.21.2.101:0/3619725618 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f514400c5a0 con 0x7f515c1b0470 2021-02-10T11:13:19.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.973 7f5163166700 1 -- 172.21.2.101:0/3619725618 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f515c1b0470 msgr2=0x7f515c1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:19.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.973 7f5163166700 1 --2- 172.21.2.101:0/3619725618 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f515c1b0470 0x7f515c1b0870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f5144006650 tx=0x7f5144009a00).stop 2021-02-10T11:13:19.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.973 7f5163166700 1 -- 172.21.2.101:0/3619725618 shutdown_connections 2021-02-10T11:13:19.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.973 7f5163166700 1 --2- 172.21.2.101:0/3619725618 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f515c0f20e0 0x7f515c0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:19.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.974 7f5163166700 1 --2- 172.21.2.101:0/3619725618 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f515c1b0470 0x7f515c1b0870 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:19.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.974 7f5163166700 1 -- 172.21.2.101:0/3619725618 >> 172.21.2.101:0/3619725618 conn(0x7f515c0ce3e0 msgr2=0x7f515c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:19.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.974 7f5163166700 1 -- 172.21.2.101:0/3619725618 shutdown_connections 2021-02-10T11:13:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.974 7f5163166700 1 -- 172.21.2.101:0/3619725618 wait complete. 2021-02-10T11:13:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.988 7f5163166700 1 Processor -- start 2021-02-10T11:13:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.988 7f5163166700 1 -- start start 2021-02-10T11:13:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.989 7f5163166700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f515c0bb980 0x7f515c19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.989 7f5163166700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f515c19c140 0x7f515c0acf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:19.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.989 7f5163166700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f515c0ad570 0x7f515c0a78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:19.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.989 7f5163166700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515c0c3e00 con 0x7f515c0ad570 2021-02-10T11:13:19.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.989 7f5163166700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515c0ad970 con 0x7f515c0bb980 2021-02-10T11:13:19.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.989 7f5163166700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f515c0adcb0 con 0x7f515c19c140 2021-02-10T11:13:19.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.989 7f5161783700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f515c0ad570 0x7f515c0a78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:19.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.989 7f5161783700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f515c0ad570 0x7f515c0a78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44026/0 (socket says 172.21.2.101:44026) 2021-02-10T11:13:19.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.989 7f5161783700 1 -- 172.21.2.101:0/3392358011 learned_addr learned my addr 172.21.2.101:0/3392358011 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:19.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.989 7f5160f82700 1 --2- 172.21.2.101:0/3392358011 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f515c0bb980 0x7f515c19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:19.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.990 7f5161783700 1 -- 172.21.2.101:0/3392358011 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f515c19c140 msgr2=0x7f515c0acf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:19.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.990 7f515bfff700 1 --2- 172.21.2.101:0/3392358011 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f515c19c140 0x7f515c0acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:19.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.990 7f5161783700 1 --2- 172.21.2.101:0/3392358011 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f515c19c140 0x7f515c0acf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:19.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.990 7f5161783700 1 -- 172.21.2.101:0/3392358011 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f515c0bb980 msgr2=0x7f515c19bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:19.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.990 7f5161783700 1 --2- 172.21.2.101:0/3392358011 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f515c0bb980 0x7f515c19bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:19.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.990 7f5161783700 1 -- 172.21.2.101:0/3392358011 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515c0a7f10 con 0x7f515c0ad570 2021-02-10T11:13:19.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.990 7f5160f82700 1 --2- 172.21.2.101:0/3392358011 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f515c0bb980 0x7f515c19bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:13:19.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.990 7f515bfff700 1 --2- 172.21.2.101:0/3392358011 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f515c19c140 0x7f515c0acf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:13:19.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.990 7f5161783700 1 --2- 172.21.2.101:0/3392358011 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f515c0ad570 0x7f515c0a78f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f5154006f10 tx=0x7f515400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:19.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.990 7f51597fa700 1 -- 172.21.2.101:0/3392358011 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5154006240 con 0x7f515c0ad570 2021-02-10T11:13:19.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.990 7f51597fa700 1 -- 172.21.2.101:0/3392358011 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5154000c10 con 0x7f515c0ad570 2021-02-10T11:13:19.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.990 7f51597fa700 1 -- 172.21.2.101:0/3392358011 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f51540056e0 con 0x7f515c0ad570 2021-02-10T11:13:19.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.990 7f5163166700 1 -- 172.21.2.101:0/3392358011 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515c0a7f10 con 0x7f515c0ad570 2021-02-10T11:13:19.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.990 7f5163166700 1 -- 172.21.2.101:0/3392358011 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f515c0a84a0 con 0x7f515c0ad570 2021-02-10T11:13:19.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.991 7f51597fa700 1 -- 172.21.2.101:0/3392358011 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5154006030 con 0x7f515c0ad570 2021-02-10T11:13:19.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.991 7f51597fa700 1 --2- 172.21.2.101:0/3392358011 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51400186c0 0x7f514001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:19.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.991 7f51597fa700 1 -- 172.21.2.101:0/3392358011 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5154032570 con 0x7f515c0ad570 2021-02-10T11:13:19.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.991 7f5160f82700 1 --2- 172.21.2.101:0/3392358011 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51400186c0 0x7f514001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:19.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:19.991 7f5160f82700 1 --2- 172.21.2.101:0/3392358011 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51400186c0 0x7f514001abc0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:20.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.022 7f5163166700 1 -- 172.21.2.101:0/3392358011 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f515c105af0 con 0x7f515c0ad570 2021-02-10T11:13:20.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.024 7f51597fa700 1 -- 172.21.2.101:0/3392358011 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5154030d20 con 0x7f515c0ad570 2021-02-10T11:13:20.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.152 7f5163166700 1 -- 172.21.2.101:0/3392358011 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f515c0f0b50 con 0x7f515c0ad570 2021-02-10T11:13:20.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.152 7f51597fa700 1 -- 172.21.2.101:0/3392358011 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v20) v1 ==== 100+0+10 (secure 0 0 0) 0x7f515400bd80 con 0x7f515c0ad570 2021-02-10T11:13:20.157 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:13:20.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.154 7f5163166700 1 -- 172.21.2.101:0/3392358011 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51400186c0 msgr2=0x7f514001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:20.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.154 7f5163166700 1 --2- 172.21.2.101:0/3392358011 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51400186c0 0x7f514001abc0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:20.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.154 7f5163166700 1 -- 172.21.2.101:0/3392358011 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f515c0ad570 msgr2=0x7f515c0a78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:20.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.154 7f5163166700 1 --2- 172.21.2.101:0/3392358011 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f515c0ad570 0x7f515c0a78f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f5154006f10 tx=0x7f515400b990).stop 2021-02-10T11:13:20.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.155 7f5163166700 1 -- 172.21.2.101:0/3392358011 shutdown_connections 2021-02-10T11:13:20.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.155 7f5163166700 1 --2- 172.21.2.101:0/3392358011 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51400186c0 0x7f514001abc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:20.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.155 7f5163166700 1 --2- 172.21.2.101:0/3392358011 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f515c19c140 0x7f515c0acf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:20.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.155 7f5163166700 1 --2- 172.21.2.101:0/3392358011 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f515c0ad570 0x7f515c0a78f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:20.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.155 7f5163166700 1 --2- 172.21.2.101:0/3392358011 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f515c0bb980 0x7f515c19bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:20.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.155 7f5163166700 1 -- 172.21.2.101:0/3392358011 >> 172.21.2.101:0/3392358011 conn(0x7f515c0bf4e0 msgr2=0x7f515c0bfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:20.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.155 7f5163166700 1 -- 172.21.2.101:0/3392358011 shutdown_connections 2021-02-10T11:13:20.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.155 7f5163166700 1 -- 172.21.2.101:0/3392358011 wait complete. 2021-02-10T11:13:20.167 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2021-02-10T11:13:20.167 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2021-02-10T11:13:20.167 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph mgr dump --format=json 2021-02-10T11:13:20.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.574 7f3a82c93700 1 Processor -- start 2021-02-10T11:13:20.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.574 7f3a82c93700 1 -- start start 2021-02-10T11:13:20.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.575 7f3a82c93700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3a7c0e5070 0x7f3a7c1b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:20.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.575 7f3a82c93700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a7c1b3e40 0x7f3a7c07d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:20.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.575 7f3a82c93700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a7c07ed80 con 0x7f3a7c0e5470 2021-02-10T11:13:20.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.575 7f3a82c93700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a7c080310 con 0x7f3a7c0e5070 2021-02-10T11:13:20.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.575 7f3a82c93700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c080500 con 0x7f3a7c1b3e40 2021-02-10T11:13:20.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.575 7f3a812b0700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f3a7c0e5470 0x7f3a7c07dcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44524/0 (socket says 172.21.2.101:44524) 2021-02-10T11:13:20.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.575 7f3a812b0700 1 -- 172.21.2.101:0/738619550 learned_addr learned my addr 172.21.2.101:0/738619550 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:20.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.575 7f3a80aaf700 1 --2- 172.21.2.101:0/738619550 >> v2:172.21.2.108:3300/0 conn(0x7f3a7c0e5070 0x7f3a7c1b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:20.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.575 7f3a7bfff700 1 --2- 172.21.2.101:0/738619550 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a7c1b3e40 0x7f3a7c07d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:20.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.576 7f3a7b7fe700 1 -- 172.21.2.101:0/738619550 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746882934 0 0) 0x7f3a74001500 con 0x7f3a7c0e5470 2021-02-10T11:13:20.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.576 7f3a7b7fe700 1 -- 172.21.2.101:0/738619550 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a68002fe0 con 0x7f3a7c0e5470 2021-02-10T11:13:20.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.576 7f3a80aaf700 1 -- 172.21.2.101:0/738619550 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a7c1b3e40 msgr2=0x7f3a7c07d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:20.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.576 7f3a80aaf700 1 --2- 172.21.2.101:0/738619550 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a7c1b3e40 0x7f3a7c07d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:20.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.576 7f3a80aaf700 1 -- 172.21.2.101:0/738619550 >> v1:172.21.2.101:6789/0 conn(0x7f3a7c0e5470 legacy=0x7f3a7c07dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:20.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.576 7f3a7b7fe700 1 -- 172.21.2.101:0/738619550 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1716379491 0 0) 0x7f3a74001500 con 0x7f3a7c0e5470 2021-02-10T11:13:20.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.576 7f3a80aaf700 1 -- 172.21.2.101:0/738619550 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7c0806f0 con 0x7f3a7c0e5070 2021-02-10T11:13:20.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.576 7f3a80aaf700 1 --2- 172.21.2.101:0/738619550 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3a7c0e5070 0x7f3a7c1b3820 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f3a64006650 tx=0x7f3a64009a00).ready entity=mon.1 client_cookie=e6863afdd300845f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:20.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.577 7f3a7b7fe700 1 -- 172.21.2.101:0/738619550 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3a64009df0 con 0x7f3a7c0e5070 2021-02-10T11:13:20.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.577 7f3a7b7fe700 1 -- 172.21.2.101:0/738619550 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a640009b0 con 0x7f3a7c0e5070 2021-02-10T11:13:20.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.577 7f3a7b7fe700 1 -- 172.21.2.101:0/738619550 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3a6400c2b0 con 0x7f3a7c0e5070 2021-02-10T11:13:20.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.577 7f3a82c93700 1 -- 172.21.2.101:0/738619550 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3a7c0e5070 msgr2=0x7f3a7c1b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:20.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.577 7f3a82c93700 1 --2- 172.21.2.101:0/738619550 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3a7c0e5070 0x7f3a7c1b3820 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f3a64006650 tx=0x7f3a64009a00).stop 2021-02-10T11:13:20.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.577 7f3a82c93700 1 -- 172.21.2.101:0/738619550 shutdown_connections 2021-02-10T11:13:20.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.577 7f3a82c93700 1 --2- 172.21.2.101:0/738619550 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a7c1b3e40 0x7f3a7c07d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:20.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.577 7f3a82c93700 1 --2- 172.21.2.101:0/738619550 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3a7c0e5070 0x7f3a7c1b3820 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:20.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.577 7f3a82c93700 1 -- 172.21.2.101:0/738619550 >> 172.21.2.101:0/738619550 conn(0x7f3a7c0ce690 msgr2=0x7f3a7c0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:20.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.578 7f3a82c93700 1 -- 172.21.2.101:0/738619550 shutdown_connections 2021-02-10T11:13:20.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.578 7f3a82c93700 1 -- 172.21.2.101:0/738619550 wait complete. 2021-02-10T11:13:20.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.592 7f3a82c93700 1 Processor -- start 2021-02-10T11:13:20.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.592 7f3a82c93700 1 -- start start 2021-02-10T11:13:20.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.592 7f3a82c93700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a7c1918e0 0x7f3a7c189c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:20.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.592 7f3a82c93700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3a7c18a2a0 0x7f3a7c18a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:20.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.592 7f3a82c93700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a7c18acc0 0x7f3a7c190070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:20.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.592 7f3a82c93700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c1a2f00 con 0x7f3a7c1918e0 2021-02-10T11:13:20.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.592 7f3a82c93700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c191ce0 con 0x7f3a7c18a2a0 2021-02-10T11:13:20.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.592 7f3a82c93700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c191f00 con 0x7f3a7c18acc0 2021-02-10T11:13:20.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.592 7f3a80aaf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a7c1918e0 0x7f3a7c189c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:20.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.593 7f3a80aaf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a7c1918e0 0x7f3a7c189c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44034/0 (socket says 172.21.2.101:44034) 2021-02-10T11:13:20.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.593 7f3a80aaf700 1 -- 172.21.2.101:0/3175754543 learned_addr learned my addr 172.21.2.101:0/3175754543 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:20.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.593 7f3a80aaf700 1 -- 172.21.2.101:0/3175754543 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a7c18acc0 msgr2=0x7f3a7c190070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:20.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.593 7f3a80aaf700 1 --2- 172.21.2.101:0/3175754543 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a7c18acc0 0x7f3a7c190070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:20.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.593 7f3a80aaf700 1 -- 172.21.2.101:0/3175754543 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3a7c18a2a0 msgr2=0x7f3a7c18a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:20.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.593 7f3a80aaf700 1 --2- 172.21.2.101:0/3175754543 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3a7c18a2a0 0x7f3a7c18a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:20.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.593 7f3a80aaf700 1 -- 172.21.2.101:0/3175754543 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7c190690 con 0x7f3a7c1918e0 2021-02-10T11:13:20.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.593 7f3a80aaf700 1 --2- 172.21.2.101:0/3175754543 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a7c1918e0 0x7f3a7c189c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f3a640096a0 tx=0x7f3a6400b9b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:20.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.593 7f3a797fa700 1 -- 172.21.2.101:0/3175754543 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3a640093a0 con 0x7f3a7c1918e0 2021-02-10T11:13:20.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.593 7f3a797fa700 1 -- 172.21.2.101:0/3175754543 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a64002030 con 0x7f3a7c1918e0 2021-02-10T11:13:20.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.593 7f3a82c93700 1 -- 172.21.2.101:0/3175754543 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7c190690 con 0x7f3a7c1918e0 2021-02-10T11:13:20.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.593 7f3a82c93700 1 -- 172.21.2.101:0/3175754543 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7c190c20 con 0x7f3a7c1918e0 2021-02-10T11:13:20.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.593 7f3a797fa700 1 -- 172.21.2.101:0/3175754543 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3a64002550 con 0x7f3a7c1918e0 2021-02-10T11:13:20.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.594 7f3a797fa700 1 -- 172.21.2.101:0/3175754543 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3a64000c90 con 0x7f3a7c1918e0 2021-02-10T11:13:20.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.594 7f3a797fa700 1 --2- 172.21.2.101:0/3175754543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a600186d0 0x7f3a6001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:20.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.594 7f3a797fa700 1 -- 172.21.2.101:0/3175754543 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3a64032780 con 0x7f3a7c1918e0 2021-02-10T11:13:20.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.594 7f3a7bfff700 1 --2- 172.21.2.101:0/3175754543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a600186d0 0x7f3a6001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:20.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.594 7f3a7bfff700 1 --2- 172.21.2.101:0/3175754543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a600186d0 0x7f3a6001abd0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:20.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.625 7f3a82c93700 1 -- 172.21.2.101:0/3175754543 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7c189750 con 0x7f3a7c1918e0 2021-02-10T11:13:20.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.628 7f3a797fa700 1 -- 172.21.2.101:0/3175754543 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3a640093a0 con 0x7f3a7c1918e0 2021-02-10T11:13:20.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.791 7f3a82c93700 1 -- 172.21.2.101:0/3175754543 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f3a7c189750 con 0x7f3a7c1918e0 2021-02-10T11:13:20.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.792 7f3a797fa700 1 -- 172.21.2.101:0/3175754543 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v3) v1 ==== 74+0+70186 (secure 0 0 0) 0x7f3a64000c90 con 0x7f3a7c1918e0 2021-02-10T11:13:20.799 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:13:20.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.795 7f3a82c93700 1 -- 172.21.2.101:0/3175754543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a600186d0 msgr2=0x7f3a6001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:20.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.795 7f3a82c93700 1 --2- 172.21.2.101:0/3175754543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a600186d0 0x7f3a6001abd0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:20.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.795 7f3a82c93700 1 -- 172.21.2.101:0/3175754543 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a7c1918e0 msgr2=0x7f3a7c189c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:20.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.795 7f3a82c93700 1 --2- 172.21.2.101:0/3175754543 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a7c1918e0 0x7f3a7c189c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f3a640096a0 tx=0x7f3a6400b9b0).stop 2021-02-10T11:13:20.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.796 7f3a82c93700 1 -- 172.21.2.101:0/3175754543 shutdown_connections 2021-02-10T11:13:20.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.796 7f3a82c93700 1 --2- 172.21.2.101:0/3175754543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a600186d0 0x7f3a6001abd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:20.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.796 7f3a82c93700 1 --2- 172.21.2.101:0/3175754543 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a7c18acc0 0x7f3a7c190070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:20.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.796 7f3a82c93700 1 --2- 172.21.2.101:0/3175754543 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3a7c18a2a0 0x7f3a7c18a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:20.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.796 7f3a82c93700 1 --2- 172.21.2.101:0/3175754543 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a7c1918e0 0x7f3a7c189c80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:20.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.796 7f3a82c93700 1 -- 172.21.2.101:0/3175754543 >> 172.21.2.101:0/3175754543 conn(0x7f3a7c1a31d0 msgr2=0x7f3a7c08b830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:20.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.796 7f3a82c93700 1 -- 172.21.2.101:0/3175754543 shutdown_connections 2021-02-10T11:13:20.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:20.796 7f3a82c93700 1 -- 172.21.2.101:0/3175754543 wait complete. 2021-02-10T11:13:20.808 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":3,"active_gid":4100,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6800","nonce":24165},{"type":"v1","addr":"172.21.2.101:6801","nonce":24165}]},"active_addr":"172.21.2.101:6801/24165","active_change":"2021-02-10 11:12:38.124918","available":true,"standbys":[{"gid":4106,"name":"x","available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_url":{"name":"server_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_iterations":{"name":"upmap_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimization iterations","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{}},{"name":"iostat","can_run":true,"error_string":"","module_options":{}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"ssh","can_run":true,"error_string":"","module_options":{"inventory_cache_timeout_min":{"name":"inventory_cache_timeout_min","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_url":{"name":"server_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_iterations":{"name":"upmap_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimization iterations","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{}},{"name":"iostat","can_run":true,"error_string":"","module_options":{}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"ssh","can_run":true,"error_string":"","module_options":{"inventory_cache_timeout_min":{"name":"inventory_cache_timeout_min","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"]}} 2021-02-10T11:13:20.810 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2021-02-10T11:13:21.617 DEBUG:teuthology.orchestra.run.gibba001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2021-02-10T11:13:22.034 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.014 7efede09c700 1 Processor -- start 2021-02-10T11:13:22.034 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.014 7efede09c700 1 -- start start 2021-02-10T11:13:22.034 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.014 7efede09c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efed80e5070 0x7efed81b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:22.035 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.014 7efede09c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7efed81b3e40 0x7efed8080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:22.035 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.014 7efede09c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed807ed80 con 0x7efed80e5470 2021-02-10T11:13:22.035 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.014 7efede09c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed80d1790 con 0x7efed81b3e40 2021-02-10T11:13:22.035 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.014 7efede09c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed80d1980 con 0x7efed80e5070 2021-02-10T11:13:22.035 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.015 7efed6ffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7efed80e5470 0x7efed807d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44536/0 (socket says 172.21.2.101:44536) 2021-02-10T11:13:22.035 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.015 7efed6ffd700 1 -- 172.21.2.101:0/2706522752 learned_addr learned my addr 172.21.2.101:0/2706522752 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:22.035 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.015 7efed7fff700 1 --2- 172.21.2.101:0/2706522752 >> v2:172.21.2.101:3300/0 conn(0x7efed81b3e40 0x7efed8080310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:22.035 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.015 7efed67fc700 1 -- 172.21.2.101:0/2706522752 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875266599 0 0) 0x7efec00010b0 con 0x7efed80e5470 2021-02-10T11:13:22.035 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.015 7efed67fc700 1 -- 172.21.2.101:0/2706522752 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efec4002fe0 con 0x7efed80e5470 2021-02-10T11:13:22.035 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.015 7efed7fff700 1 -- 172.21.2.101:0/2706522752 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efed80e5070 msgr2=0x7efed81b3820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:13:22.035 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.015 7efed7fff700 1 --2- 172.21.2.101:0/2706522752 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efed80e5070 0x7efed81b3820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:22.035 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.015 7efed7fff700 1 -- 172.21.2.101:0/2706522752 >> v1:172.21.2.101:6789/0 conn(0x7efed80e5470 legacy=0x7efed807d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:22.035 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.015 7efed7fff700 1 -- 172.21.2.101:0/2706522752 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed80d1ba0 con 0x7efed81b3e40 2021-02-10T11:13:22.036 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.015 7efed7fff700 1 --2- 172.21.2.101:0/2706522752 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efed81b3e40 0x7efed8080310 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7efed0006e30 tx=0x7efed0009e60).ready entity=mon.0 client_cookie=23f1ec3cafd1e728 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:22.036 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.015 7efed67fc700 1 -- 172.21.2.101:0/2706522752 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efed0006a50 con 0x7efed81b3e40 2021-02-10T11:13:22.036 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.015 7efed67fc700 1 -- 172.21.2.101:0/2706522752 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed000b300 con 0x7efed81b3e40 2021-02-10T11:13:22.036 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.015 7efed67fc700 1 -- 172.21.2.101:0/2706522752 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efed000b820 con 0x7efed81b3e40 2021-02-10T11:13:22.036 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.016 7efede09c700 1 -- 172.21.2.101:0/2706522752 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efed81b3e40 msgr2=0x7efed8080310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:22.036 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.016 7efede09c700 1 --2- 172.21.2.101:0/2706522752 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efed81b3e40 0x7efed8080310 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7efed0006e30 tx=0x7efed0009e60).stop 2021-02-10T11:13:22.036 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.016 7efede09c700 1 -- 172.21.2.101:0/2706522752 shutdown_connections 2021-02-10T11:13:22.036 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.016 7efede09c700 1 --2- 172.21.2.101:0/2706522752 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efed81b3e40 0x7efed8080310 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:22.036 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.016 7efede09c700 1 --2- 172.21.2.101:0/2706522752 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efed80e5070 0x7efed81b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:22.036 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.016 7efede09c700 1 -- 172.21.2.101:0/2706522752 >> 172.21.2.101:0/2706522752 conn(0x7efed80ce690 msgr2=0x7efed80e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:22.036 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.016 7efede09c700 1 -- 172.21.2.101:0/2706522752 shutdown_connections 2021-02-10T11:13:22.036 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.016 7efede09c700 1 -- 172.21.2.101:0/2706522752 wait complete. 2021-02-10T11:13:22.036 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.030 7efede09c700 1 Processor -- start 2021-02-10T11:13:22.037 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.030 7efede09c700 1 -- start start 2021-02-10T11:13:22.038 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.031 7efede09c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efed8092d00 0x7efed81076b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:22.038 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.031 7efede09c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efed8107cd0 0x7efed8198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:22.038 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.031 7efede09c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efed81991d0 0x7efed81995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:22.038 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.031 7efede09c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed81986b0 con 0x7efed8092d00 2021-02-10T11:13:22.038 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.031 7efede09c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed8198300 con 0x7efed81991d0 2021-02-10T11:13:22.038 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.031 7efede09c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed81aa0d0 con 0x7efed8107cd0 2021-02-10T11:13:22.039 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.031 7efed77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efed8092d00 0x7efed81076b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:22.039 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.031 7efed77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efed8092d00 0x7efed81076b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44046/0 (socket says 172.21.2.101:44046) 2021-02-10T11:13:22.039 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.031 7efed77fe700 1 -- 172.21.2.101:0/3739820163 learned_addr learned my addr 172.21.2.101:0/3739820163 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:22.039 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.031 7efed77fe700 1 -- 172.21.2.101:0/3739820163 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efed8107cd0 msgr2=0x7efed8198bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:22.039 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.031 7efed77fe700 1 --2- 172.21.2.101:0/3739820163 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efed8107cd0 0x7efed8198bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:22.039 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.031 7efed77fe700 1 -- 172.21.2.101:0/3739820163 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efed81991d0 msgr2=0x7efed81995d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:22.039 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.031 7efed77fe700 1 --2- 172.21.2.101:0/3739820163 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efed81991d0 0x7efed81995d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:22.039 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.031 7efed77fe700 1 -- 172.21.2.101:0/3739820163 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed81080d0 con 0x7efed8092d00 2021-02-10T11:13:22.039 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.032 7efed77fe700 1 --2- 172.21.2.101:0/3739820163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efed8092d00 0x7efed81076b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7efec800b5c0 tx=0x7efec800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:22.039 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.032 7efecffff700 1 -- 172.21.2.101:0/3739820163 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efec800bd50 con 0x7efed8092d00 2021-02-10T11:13:22.039 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.032 7efecffff700 1 -- 172.21.2.101:0/3739820163 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec8006030 con 0x7efed8092d00 2021-02-10T11:13:22.040 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.032 7efecffff700 1 -- 172.21.2.101:0/3739820163 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efec80054d0 con 0x7efed8092d00 2021-02-10T11:13:22.040 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.032 7efede09c700 1 -- 172.21.2.101:0/3739820163 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed81080d0 con 0x7efed8092d00 2021-02-10T11:13:22.040 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.032 7efede09c700 1 -- 172.21.2.101:0/3739820163 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed8199ee0 con 0x7efed8092d00 2021-02-10T11:13:22.040 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.032 7efecffff700 1 -- 172.21.2.101:0/3739820163 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7efec8006210 con 0x7efed8092d00 2021-02-10T11:13:22.040 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.033 7efecffff700 1 --2- 172.21.2.101:0/3739820163 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efebc0186c0 0x7efebc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:22.040 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.033 7efecffff700 1 -- 172.21.2.101:0/3739820163 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7efec80325d0 con 0x7efed8092d00 2021-02-10T11:13:22.040 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.033 7efed6ffd700 1 --2- 172.21.2.101:0/3739820163 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efebc0186c0 0x7efebc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:22.040 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.033 7efed6ffd700 1 --2- 172.21.2.101:0/3739820163 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efebc0186c0 0x7efebc01abc0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:22.069 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.065 7efede09c700 1 -- 172.21.2.101:0/3739820163 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed807ed80 con 0x7efed8092d00 2021-02-10T11:13:22.072 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.067 7efecffff700 1 -- 172.21.2.101:0/3739820163 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7efec8000940 con 0x7efed8092d00 2021-02-10T11:13:22.195 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.192 7efede09c700 1 -- 172.21.2.101:0/3739820163 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efed81982c0 con 0x7efed8092d00 2021-02-10T11:13:22.196 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.192 7efecffff700 1 -- 172.21.2.101:0/3739820163 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+13993 (secure 0 0 0) 0x7efec8006210 con 0x7efed8092d00 2021-02-10T11:13:22.197 INFO:teuthology.misc.health.gibba001.stdout: 2021-02-10T11:13:22.197 INFO:teuthology.misc.health.gibba001.stdout:{"epoch":20,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:13:15.149234","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:13:11.922272","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2021-02-10 11:13:13.141733","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:13:22.198 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.194 7efede09c700 1 -- 172.21.2.101:0/3739820163 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efebc0186c0 msgr2=0x7efebc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:22.198 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.194 7efede09c700 1 --2- 172.21.2.101:0/3739820163 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efebc0186c0 0x7efebc01abc0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:22.198 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.194 7efede09c700 1 -- 172.21.2.101:0/3739820163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efed8092d00 msgr2=0x7efed81076b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:22.198 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.194 7efede09c700 1 --2- 172.21.2.101:0/3739820163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efed8092d00 0x7efed81076b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7efec800b5c0 tx=0x7efec800b960).stop 2021-02-10T11:13:22.199 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.194 7efede09c700 1 -- 172.21.2.101:0/3739820163 shutdown_connections 2021-02-10T11:13:22.199 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.194 7efede09c700 1 --2- 172.21.2.101:0/3739820163 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efebc0186c0 0x7efebc01abc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:22.199 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.194 7efede09c700 1 --2- 172.21.2.101:0/3739820163 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efed8107cd0 0x7efed8198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:22.199 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.194 7efede09c700 1 --2- 172.21.2.101:0/3739820163 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efed81991d0 0x7efed81995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:22.199 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.194 7efede09c700 1 --2- 172.21.2.101:0/3739820163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efed8092d00 0x7efed81076b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:22.199 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.194 7efede09c700 1 -- 172.21.2.101:0/3739820163 >> 172.21.2.101:0/3739820163 conn(0x7efed81aa5d0 msgr2=0x7efed819f180 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:22.199 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.195 7efede09c700 1 -- 172.21.2.101:0/3739820163 shutdown_connections 2021-02-10T11:13:22.199 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:22.195 7efede09c700 1 -- 172.21.2.101:0/3739820163 wait complete. 2021-02-10T11:13:22.206 DEBUG:teuthology.misc:8 of 8 OSDs are up 2021-02-10T11:13:22.206 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:13:22.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.614 7fdf0d70c700 1 Processor -- start 2021-02-10T11:13:22.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.615 7fdf0d70c700 1 -- start start 2021-02-10T11:13:22.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.615 7fdf0d70c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fdf080e5470 0x7fdf0807e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:22.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.615 7fdf0d70c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0807ed80 con 0x7fdf080e5070 2021-02-10T11:13:22.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.615 7fdf0d70c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf081b4010 con 0x7fdf0807f300 2021-02-10T11:13:22.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.615 7fdf0d70c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf080d1790 con 0x7fdf080e5470 2021-02-10T11:13:22.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.615 7fdf072bf700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fdf080e5070 0x7fdf081b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44558/0 (socket says 172.21.2.101:44558) 2021-02-10T11:13:22.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.615 7fdf072bf700 1 -- 172.21.2.101:0/675844024 learned_addr learned my addr 172.21.2.101:0/675844024 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:22.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.615 7fdf06abe700 1 --2- 172.21.2.101:0/675844024 >> v2:172.21.2.108:3300/0 conn(0x7fdf080e5470 0x7fdf0807e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:22.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.616 7fdf06abe700 1 -- 172.21.2.101:0/675844024 >> v1:172.21.2.108:6789/0 conn(0x7fdf0807f300 legacy=0x7fdf0807f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:22.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.616 7fdf06abe700 1 -- 172.21.2.101:0/675844024 >> v1:172.21.2.101:6789/0 conn(0x7fdf080e5070 legacy=0x7fdf081b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:22.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.616 7fdf06abe700 1 -- 172.21.2.101:0/675844024 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf080d19b0 con 0x7fdf080e5470 2021-02-10T11:13:22.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.616 7fdf06abe700 1 --2- 172.21.2.101:0/675844024 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf080e5470 0x7fdf0807e6a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fdef0006650 tx=0x7fdef0009a80).ready entity=mon.1 client_cookie=3b338680fab9a450 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:22.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.616 7fdf062bd700 1 -- 172.21.2.101:0/675844024 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdef000c0c0 con 0x7fdf080e5470 2021-02-10T11:13:22.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.616 7fdf062bd700 1 -- 172.21.2.101:0/675844024 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef000c340 con 0x7fdf080e5470 2021-02-10T11:13:22.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.616 7fdf062bd700 1 -- 172.21.2.101:0/675844024 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdef000c860 con 0x7fdf080e5470 2021-02-10T11:13:22.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.616 7fdf0d70c700 1 -- 172.21.2.101:0/675844024 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf080e5470 msgr2=0x7fdf0807e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:22.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.616 7fdf0d70c700 1 --2- 172.21.2.101:0/675844024 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf080e5470 0x7fdf0807e6a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fdef0006650 tx=0x7fdef0009a80).stop 2021-02-10T11:13:22.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.617 7fdf0d70c700 1 -- 172.21.2.101:0/675844024 shutdown_connections 2021-02-10T11:13:22.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.617 7fdf0d70c700 1 --2- 172.21.2.101:0/675844024 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf080e5470 0x7fdf0807e6a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:22.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.617 7fdf0d70c700 1 -- 172.21.2.101:0/675844024 >> 172.21.2.101:0/675844024 conn(0x7fdf080ce690 msgr2=0x7fdf080e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:22.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.617 7fdf0d70c700 1 -- 172.21.2.101:0/675844024 shutdown_connections 2021-02-10T11:13:22.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.617 7fdf0d70c700 1 -- 172.21.2.101:0/675844024 wait complete. 2021-02-10T11:13:22.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.632 7fdf0d70c700 1 Processor -- start 2021-02-10T11:13:22.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.632 7fdf0d70c700 1 -- start start 2021-02-10T11:13:22.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.632 7fdf0d70c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf081c55a0 0x7fdf081c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:22.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.632 7fdf0d70c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf080bfdc0 0x7fdf080c01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:22.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.632 7fdf0d70c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf08099900 0x7fdf08099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:22.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.632 7fdf0d70c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf080c09f0 con 0x7fdf081c55a0 2021-02-10T11:13:22.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.632 7fdf0d70c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0809a320 con 0x7fdf08099900 2021-02-10T11:13:22.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.632 7fdf0d70c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0809a540 con 0x7fdf080bfdc0 2021-02-10T11:13:22.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.633 7fdf072bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf081c55a0 0x7fdf081c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:22.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.633 7fdf072bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf081c55a0 0x7fdf081c59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44070/0 (socket says 172.21.2.101:44070) 2021-02-10T11:13:22.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.633 7fdf072bf700 1 -- 172.21.2.101:0/2044521585 learned_addr learned my addr 172.21.2.101:0/2044521585 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:22.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.633 7fdf072bf700 1 -- 172.21.2.101:0/2044521585 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf080bfdc0 msgr2=0x7fdf080c01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:22.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.633 7fdf072bf700 1 --2- 172.21.2.101:0/2044521585 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf080bfdc0 0x7fdf080c01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:22.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.633 7fdf072bf700 1 -- 172.21.2.101:0/2044521585 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf08099900 msgr2=0x7fdf08099d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:22.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.633 7fdf072bf700 1 --2- 172.21.2.101:0/2044521585 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf08099900 0x7fdf08099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:22.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.633 7fdf072bf700 1 -- 172.21.2.101:0/2044521585 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0809a760 con 0x7fdf081c55a0 2021-02-10T11:13:22.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.633 7fdf072bf700 1 --2- 172.21.2.101:0/2044521585 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf081c55a0 0x7fdf081c59a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fdef8001c10 tx=0x7fdef800a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:22.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.633 7fdefffff700 1 -- 172.21.2.101:0/2044521585 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdef800acc0 con 0x7fdf081c55a0 2021-02-10T11:13:22.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.633 7fdefffff700 1 -- 172.21.2.101:0/2044521585 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef80009b0 con 0x7fdf081c55a0 2021-02-10T11:13:22.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.633 7fdefffff700 1 -- 172.21.2.101:0/2044521585 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdef8004540 con 0x7fdf081c55a0 2021-02-10T11:13:22.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.633 7fdf0d70c700 1 -- 172.21.2.101:0/2044521585 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0809a760 con 0x7fdf081c55a0 2021-02-10T11:13:22.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.633 7fdf0d70c700 1 -- 172.21.2.101:0/2044521585 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf080c0ef0 con 0x7fdf081c55a0 2021-02-10T11:13:22.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.634 7fdefffff700 1 -- 172.21.2.101:0/2044521585 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdef8004140 con 0x7fdf081c55a0 2021-02-10T11:13:22.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.634 7fdefffff700 1 --2- 172.21.2.101:0/2044521585 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdeec0186c0 0x7fdeec01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:22.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.634 7fdefffff700 1 -- 172.21.2.101:0/2044521585 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fdef8032600 con 0x7fdf081c55a0 2021-02-10T11:13:22.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.634 7fdf06abe700 1 --2- 172.21.2.101:0/2044521585 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdeec0186c0 0x7fdeec01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:22.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.634 7fdf06abe700 1 --2- 172.21.2.101:0/2044521585 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdeec0186c0 0x7fdeec01abc0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:22.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.665 7fdf0d70c700 1 -- 172.21.2.101:0/2044521585 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf080ce690 con 0x7fdf081c55a0 2021-02-10T11:13:22.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.667 7fdefffff700 1 -- 172.21.2.101:0/2044521585 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdef8001050 con 0x7fdf081c55a0 2021-02-10T11:13:22.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.792 7fdf0d70c700 1 -- 172.21.2.101:0/2044521585 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf080ce690 con 0x7fdf081c55a0 2021-02-10T11:13:22.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.793 7fdefffff700 1 -- 172.21.2.101:0/2044521585 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+13993 (secure 0 0 0) 0x7fdef801f4c0 con 0x7fdf081c55a0 2021-02-10T11:13:22.797 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:13:22.797 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":20,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:13:15.149234","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:13:11.922272","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2021-02-10 11:13:13.141733","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:13:22.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.794 7fdf0d70c700 1 -- 172.21.2.101:0/2044521585 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdeec0186c0 msgr2=0x7fdeec01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:22.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.794 7fdf0d70c700 1 --2- 172.21.2.101:0/2044521585 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdeec0186c0 0x7fdeec01abc0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:22.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.794 7fdf0d70c700 1 -- 172.21.2.101:0/2044521585 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf081c55a0 msgr2=0x7fdf081c59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:22.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.794 7fdf0d70c700 1 --2- 172.21.2.101:0/2044521585 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf081c55a0 0x7fdf081c59a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fdef8001c10 tx=0x7fdef800a8d0).stop 2021-02-10T11:13:22.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.795 7fdf0d70c700 1 -- 172.21.2.101:0/2044521585 shutdown_connections 2021-02-10T11:13:22.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.795 7fdf0d70c700 1 --2- 172.21.2.101:0/2044521585 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdeec0186c0 0x7fdeec01abc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:22.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.795 7fdf0d70c700 1 --2- 172.21.2.101:0/2044521585 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf080bfdc0 0x7fdf080c01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:22.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.795 7fdf0d70c700 1 --2- 172.21.2.101:0/2044521585 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf08099900 0x7fdf08099d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:22.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.795 7fdf0d70c700 1 --2- 172.21.2.101:0/2044521585 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf081c55a0 0x7fdf081c59a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:22.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.795 7fdf0d70c700 1 -- 172.21.2.101:0/2044521585 >> 172.21.2.101:0/2044521585 conn(0x7fdf080976b0 msgr2=0x7fdf080962b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:22.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.795 7fdf0d70c700 1 -- 172.21.2.101:0/2044521585 shutdown_connections 2021-02-10T11:13:22.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:22.795 7fdf0d70c700 1 -- 172.21.2.101:0/2044521585 wait complete. 2021-02-10T11:13:22.807 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell osd.0 flush_pg_stats 2021-02-10T11:13:23.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.212 7f404ba32700 1 Processor -- start 2021-02-10T11:13:23.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.212 7f404ba32700 1 -- start start 2021-02-10T11:13:23.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.212 7f404ba32700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f40440e5470 0x7f404407e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:23.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.212 7f404ba32700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f404407ed80 con 0x7f40440e5070 2021-02-10T11:13:23.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.212 7f404ba32700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40441b4010 con 0x7f404407f300 2021-02-10T11:13:23.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.212 7f404ba32700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40440d1790 con 0x7f40440e5470 2021-02-10T11:13:23.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.213 7f404904d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f40440e5470 0x7f404407e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:23.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.213 7f404984e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f40440e5070 0x7f40441b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44572/0 (socket says 172.21.2.101:44572) 2021-02-10T11:13:23.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.213 7f404984e700 1 -- 172.21.2.101:0/1445052505 learned_addr learned my addr 172.21.2.101:0/1445052505 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:23.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.213 7f404904d700 1 -- 172.21.2.101:0/1445052505 >> v1:172.21.2.108:6789/0 conn(0x7f404407f300 legacy=0x7f404407f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:23.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.213 7f404904d700 1 -- 172.21.2.101:0/1445052505 >> v1:172.21.2.101:6789/0 conn(0x7f40440e5070 legacy=0x7f40441b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:23.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.213 7f404904d700 1 -- 172.21.2.101:0/1445052505 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40440d19b0 con 0x7f40440e5470 2021-02-10T11:13:23.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.213 7f404904d700 1 --2- 172.21.2.101:0/1445052505 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f40440e5470 0x7f404407e6a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f4040006650 tx=0x7f4040009a80).ready entity=mon.0 client_cookie=84022c19c5341ed2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:23.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.213 7f404884c700 1 -- 172.21.2.101:0/1445052505 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f404000c0c0 con 0x7f40440e5470 2021-02-10T11:13:23.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.213 7f404884c700 1 -- 172.21.2.101:0/1445052505 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404000c340 con 0x7f40440e5470 2021-02-10T11:13:23.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.213 7f404884c700 1 -- 172.21.2.101:0/1445052505 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f404000c860 con 0x7f40440e5470 2021-02-10T11:13:23.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.213 7f404ba32700 1 -- 172.21.2.101:0/1445052505 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f40440e5470 msgr2=0x7f404407e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:23.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.213 7f404ba32700 1 --2- 172.21.2.101:0/1445052505 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f40440e5470 0x7f404407e6a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f4040006650 tx=0x7f4040009a80).stop 2021-02-10T11:13:23.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.214 7f404ba32700 1 -- 172.21.2.101:0/1445052505 shutdown_connections 2021-02-10T11:13:23.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.214 7f404ba32700 1 --2- 172.21.2.101:0/1445052505 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f40440e5470 0x7f404407e6a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:23.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.214 7f404ba32700 1 -- 172.21.2.101:0/1445052505 >> 172.21.2.101:0/1445052505 conn(0x7f40440ce690 msgr2=0x7f40440e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:23.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.214 7f404ba32700 1 -- 172.21.2.101:0/1445052505 shutdown_connections 2021-02-10T11:13:23.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.214 7f404ba32700 1 -- 172.21.2.101:0/1445052505 wait complete. 2021-02-10T11:13:23.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.229 7f404ba32700 1 Processor -- start 2021-02-10T11:13:23.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.229 7f404ba32700 1 -- start start 2021-02-10T11:13:23.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.229 7f404ba32700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f40441009c0 0x7f40440981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:23.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.229 7f404ba32700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4044101200 0x7f40441c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:23.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.229 7f404ba32700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4044101600 0x7f40441c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:23.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.229 7f404ba32700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4044098820 con 0x7f40441009c0 2021-02-10T11:13:23.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.229 7f404ba32700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4044100dc0 con 0x7f4044101200 2021-02-10T11:13:23.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.229 7f404ba32700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40441c9f60 con 0x7f4044101600 2021-02-10T11:13:23.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.229 7f404984e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f40441009c0 0x7f40440981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:23.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.229 7f404984e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f40441009c0 0x7f40440981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44086/0 (socket says 172.21.2.101:44086) 2021-02-10T11:13:23.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.229 7f404984e700 1 -- 172.21.2.101:0/2738072132 learned_addr learned my addr 172.21.2.101:0/2738072132 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:23.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.230 7f404984e700 1 -- 172.21.2.101:0/2738072132 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4044101600 msgr2=0x7f40441c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:23.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.230 7f404984e700 1 --2- 172.21.2.101:0/2738072132 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4044101600 0x7f40441c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:23.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.230 7f404984e700 1 -- 172.21.2.101:0/2738072132 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4044101200 msgr2=0x7f40441c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:23.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.230 7f404984e700 1 --2- 172.21.2.101:0/2738072132 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4044101200 0x7f40441c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:23.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.230 7f404984e700 1 -- 172.21.2.101:0/2738072132 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4044098da0 con 0x7f40441009c0 2021-02-10T11:13:23.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.230 7f404984e700 1 --2- 172.21.2.101:0/2738072132 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f40441009c0 0x7f40440981a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f4038001c60 tx=0x7f403800a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:23.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.230 7f40367fc700 1 -- 172.21.2.101:0/2738072132 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f403800acc0 con 0x7f40441009c0 2021-02-10T11:13:23.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.230 7f40367fc700 1 -- 172.21.2.101:0/2738072132 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4038004140 con 0x7f40441009c0 2021-02-10T11:13:23.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.230 7f40367fc700 1 -- 172.21.2.101:0/2738072132 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4038004620 con 0x7f40441009c0 2021-02-10T11:13:23.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.230 7f404ba32700 1 -- 172.21.2.101:0/2738072132 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4044098da0 con 0x7f40441009c0 2021-02-10T11:13:23.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.231 7f404ba32700 1 -- 172.21.2.101:0/2738072132 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4044098da0 con 0x7f40441009c0 2021-02-10T11:13:23.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.231 7f40367fc700 1 -- 172.21.2.101:0/2738072132 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4038000ce0 con 0x7f40441009c0 2021-02-10T11:13:23.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.231 7f40367fc700 1 --2- 172.21.2.101:0/2738072132 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f402c018730 0x7f402c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:23.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.231 7f40367fc700 1 -- 172.21.2.101:0/2738072132 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4038032760 con 0x7f40441009c0 2021-02-10T11:13:23.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.231 7f404904d700 1 --2- 172.21.2.101:0/2738072132 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f402c018730 0x7f402c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:23.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.232 7f404904d700 1 --2- 172.21.2.101:0/2738072132 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f402c018730 0x7f402c01ac10 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:23.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.262 7f404ba32700 1 --2- 172.21.2.101:0/2738072132 >> [v2:172.21.2.101:6802/25093,v1:172.21.2.101:6805/25093] conn(0x7f40440c45a0 0x7f4044099990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:23.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.262 7f404ba32700 1 -- 172.21.2.101:0/2738072132 --> [v2:172.21.2.101:6802/25093,v1:172.21.2.101:6805/25093] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40441c9f60 con 0x7f40440c45a0 2021-02-10T11:13:23.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.262 7f404a04f700 1 --2- 172.21.2.101:0/2738072132 >> [v2:172.21.2.101:6802/25093,v1:172.21.2.101:6805/25093] conn(0x7f40440c45a0 0x7f4044099990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:23.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.263 7f404a04f700 1 --2- 172.21.2.101:0/2738072132 >> [v2:172.21.2.101:6802/25093,v1:172.21.2.101:6805/25093] conn(0x7f40440c45a0 0x7f4044099990 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:23.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.263 7f40367fc700 1 -- 172.21.2.101:0/2738072132 <== osd.0 v2:172.21.2.101:6802/25093 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (crc 0 0 0) 0x7f403c00b050 con 0x7f40440c45a0 2021-02-10T11:13:23.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.270 7f404ba32700 1 -- 172.21.2.101:0/2738072132 --> [v2:172.21.2.101:6802/25093,v1:172.21.2.101:6805/25093] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4044089810 con 0x7f40440c45a0 2021-02-10T11:13:23.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.270 7f40367fc700 1 -- 172.21.2.101:0/2738072132 <== osd.0 v2:172.21.2.101:6802/25093 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f403c006da0 con 0x7f40440c45a0 2021-02-10T11:13:23.275 INFO:teuthology.orchestra.run.gibba001.stdout:55834574854 2021-02-10T11:13:23.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.271 7f404ba32700 1 -- 172.21.2.101:0/2738072132 >> [v2:172.21.2.101:6802/25093,v1:172.21.2.101:6805/25093] conn(0x7f40440c45a0 msgr2=0x7f4044099990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:23.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.271 7f404ba32700 1 --2- 172.21.2.101:0/2738072132 >> [v2:172.21.2.101:6802/25093,v1:172.21.2.101:6805/25093] conn(0x7f40440c45a0 0x7f4044099990 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:23.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.271 7f404ba32700 1 -- 172.21.2.101:0/2738072132 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f402c018730 msgr2=0x7f402c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:23.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.271 7f404ba32700 1 --2- 172.21.2.101:0/2738072132 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f402c018730 0x7f402c01ac10 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:23.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.271 7f404ba32700 1 -- 172.21.2.101:0/2738072132 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f40441009c0 msgr2=0x7f40440981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:23.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.271 7f404ba32700 1 --2- 172.21.2.101:0/2738072132 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f40441009c0 0x7f40440981a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f4038001c60 tx=0x7f403800a8d0).stop 2021-02-10T11:13:23.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.271 7f404a04f700 1 -- 172.21.2.101:0/2738072132 reap_dead start 2021-02-10T11:13:23.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.272 7f404ba32700 1 -- 172.21.2.101:0/2738072132 shutdown_connections 2021-02-10T11:13:23.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.272 7f404ba32700 1 -- 172.21.2.101:0/2738072132 >> 172.21.2.101:0/2738072132 conn(0x7f404410c980 msgr2=0x7f404409dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:23.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.272 7f404ba32700 1 -- 172.21.2.101:0/2738072132 shutdown_connections 2021-02-10T11:13:23.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.272 7f404ba32700 1 -- 172.21.2.101:0/2738072132 wait complete. 2021-02-10T11:13:23.282 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell osd.1 flush_pg_stats 2021-02-10T11:13:23.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.686 7f962c793700 1 Processor -- start 2021-02-10T11:13:23.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.686 7f962c793700 1 -- start start 2021-02-10T11:13:23.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.686 7f962c793700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f96240e5070 0x7f96241b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:23.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.686 7f962c793700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f96240e5470 0x7f9624080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:23.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.686 7f962c793700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f962407ed80 con 0x7f96241b3e40 2021-02-10T11:13:23.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.686 7f962c793700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96240d1790 con 0x7f96240e5470 2021-02-10T11:13:23.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.686 7f962c793700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96240d19b0 con 0x7f96240e5070 2021-02-10T11:13:23.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.686 7f962adb0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f96240e5470 0x7f9624080310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:23.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.686 7f962adb0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f96240e5470 0x7f9624080310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44094/0 (socket says 172.21.2.101:44094) 2021-02-10T11:13:23.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.686 7f962adb0700 1 -- 172.21.2.101:0/3695420087 learned_addr learned my addr 172.21.2.101:0/3695420087 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:23.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.686 7f9629dae700 1 --1- 172.21.2.101:0/3695420087 >> v1:172.21.2.101:6789/0 conn(0x7f96241b3e40 0x7f962407d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44588/0 (socket says 172.21.2.101:44588) 2021-02-10T11:13:23.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.687 7f962a5af700 1 --2- 172.21.2.101:0/3695420087 >> v2:172.21.2.108:3300/0 conn(0x7f96240e5070 0x7f96241b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:23.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.687 7f962adb0700 1 -- 172.21.2.101:0/3695420087 >> v2:172.21.2.108:3300/0 conn(0x7f96240e5070 msgr2=0x7f96241b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:23.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.687 7f962adb0700 1 --2- 172.21.2.101:0/3695420087 >> v2:172.21.2.108:3300/0 conn(0x7f96240e5070 0x7f96241b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:23.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.687 7f962adb0700 1 -- 172.21.2.101:0/3695420087 >> v1:172.21.2.101:6789/0 conn(0x7f96241b3e40 legacy=0x7f962407d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:23.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.687 7f962adb0700 1 -- 172.21.2.101:0/3695420087 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96240d1bd0 con 0x7f96240e5470 2021-02-10T11:13:23.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.687 7f962adb0700 1 --2- 172.21.2.101:0/3695420087 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96240e5470 0x7f9624080310 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f961c006f00 tx=0x7f961c009e90).ready entity=mon.0 client_cookie=36ebd25a8cd2283 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:23.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.688 7f96295ad700 1 -- 172.21.2.101:0/3695420087 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f961c00b300 con 0x7f96240e5470 2021-02-10T11:13:23.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.688 7f96295ad700 1 -- 172.21.2.101:0/3695420087 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961c00b580 con 0x7f96240e5470 2021-02-10T11:13:23.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.688 7f96295ad700 1 -- 172.21.2.101:0/3695420087 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f961c00b9d0 con 0x7f96240e5470 2021-02-10T11:13:23.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.688 7f962c793700 1 -- 172.21.2.101:0/3695420087 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96240e5470 msgr2=0x7f9624080310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:23.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.688 7f962c793700 1 --2- 172.21.2.101:0/3695420087 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96240e5470 0x7f9624080310 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f961c006f00 tx=0x7f961c009e90).stop 2021-02-10T11:13:23.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.688 7f962c793700 1 -- 172.21.2.101:0/3695420087 shutdown_connections 2021-02-10T11:13:23.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.688 7f962c793700 1 --2- 172.21.2.101:0/3695420087 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96240e5470 0x7f9624080310 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:23.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.688 7f962c793700 1 --2- 172.21.2.101:0/3695420087 >> v2:172.21.2.108:3300/0 conn(0x7f96240e5070 0x7f96241b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:23.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.688 7f962c793700 1 -- 172.21.2.101:0/3695420087 >> 172.21.2.101:0/3695420087 conn(0x7f96240ce690 msgr2=0x7f96240e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:23.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.689 7f962c793700 1 -- 172.21.2.101:0/3695420087 shutdown_connections 2021-02-10T11:13:23.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.689 7f962c793700 1 -- 172.21.2.101:0/3695420087 wait complete. 2021-02-10T11:13:23.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.703 7f962c793700 1 Processor -- start 2021-02-10T11:13:23.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.703 7f962c793700 1 -- start start 2021-02-10T11:13:23.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.703 7f962c793700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96240be120 0x7f96240be720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:23.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.703 7f962c793700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96240896d0 0x7f9624198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:23.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.704 7f962c793700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9624199150 0x7f96240ba680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:23.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.704 7f962c793700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96240baca0 con 0x7f9624199150 2021-02-10T11:13:23.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.704 7f962c793700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96240baec0 con 0x7f96240be120 2021-02-10T11:13:23.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.704 7f962c793700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9624089ad0 con 0x7f96240896d0 2021-02-10T11:13:23.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.704 7f962adb0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9624199150 0x7f96240ba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:23.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.704 7f962adb0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9624199150 0x7f96240ba680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44102/0 (socket says 172.21.2.101:44102) 2021-02-10T11:13:23.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.704 7f962adb0700 1 -- 172.21.2.101:0/942294659 learned_addr learned my addr 172.21.2.101:0/942294659 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:23.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.704 7f962a5af700 1 --2- 172.21.2.101:0/942294659 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96240be120 0x7f96240be720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:23.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.704 7f9629dae700 1 --2- 172.21.2.101:0/942294659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96240896d0 0x7f9624198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:23.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.704 7f962adb0700 1 -- 172.21.2.101:0/942294659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96240896d0 msgr2=0x7f9624198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:23.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.704 7f962adb0700 1 --2- 172.21.2.101:0/942294659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96240896d0 0x7f9624198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:23.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.704 7f962adb0700 1 -- 172.21.2.101:0/942294659 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96240be120 msgr2=0x7f96240be720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:23.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.704 7f962adb0700 1 --2- 172.21.2.101:0/942294659 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96240be120 0x7f96240be720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:23.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.704 7f962adb0700 1 -- 172.21.2.101:0/942294659 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9624089cf0 con 0x7f9624199150 2021-02-10T11:13:23.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.704 7f962adb0700 1 --2- 172.21.2.101:0/942294659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9624199150 0x7f96240ba680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f961c00b910 tx=0x7f961c00bcb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:23.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.705 7f9616ffd700 1 -- 172.21.2.101:0/942294659 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f961c000cc0 con 0x7f9624199150 2021-02-10T11:13:23.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.705 7f9616ffd700 1 -- 172.21.2.101:0/942294659 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961c00bdf0 con 0x7f9624199150 2021-02-10T11:13:23.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.705 7f962c793700 1 -- 172.21.2.101:0/942294659 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9624089cf0 con 0x7f9624199150 2021-02-10T11:13:23.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.705 7f962c793700 1 -- 172.21.2.101:0/942294659 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96240bf0e0 con 0x7f9624199150 2021-02-10T11:13:23.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.705 7f9616ffd700 1 -- 172.21.2.101:0/942294659 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f961c005a10 con 0x7f9624199150 2021-02-10T11:13:23.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.705 7f9616ffd700 1 -- 172.21.2.101:0/942294659 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f961c005820 con 0x7f9624199150 2021-02-10T11:13:23.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.705 7f9616ffd700 1 --2- 172.21.2.101:0/942294659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f960c018730 0x7f960c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:23.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.705 7f9616ffd700 1 -- 172.21.2.101:0/942294659 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f961c032b70 con 0x7f9624199150 2021-02-10T11:13:23.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.705 7f962a5af700 1 --2- 172.21.2.101:0/942294659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f960c018730 0x7f960c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:23.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.706 7f962a5af700 1 --2- 172.21.2.101:0/942294659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f960c018730 0x7f960c01ac10 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:23.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.736 7f962c793700 1 --2- 172.21.2.101:0/942294659 >> [v2:172.21.2.101:6803/25092,v1:172.21.2.101:6804/25092] conn(0x7f96240c45a0 0x7f96240bf970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:23.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.736 7f962c793700 1 -- 172.21.2.101:0/942294659 --> [v2:172.21.2.101:6803/25092,v1:172.21.2.101:6804/25092] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f96240bf0e0 con 0x7f96240c45a0 2021-02-10T11:13:23.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.736 7f9629dae700 1 --2- 172.21.2.101:0/942294659 >> [v2:172.21.2.101:6803/25092,v1:172.21.2.101:6804/25092] conn(0x7f96240c45a0 0x7f96240bf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:23.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.736 7f9629dae700 1 --2- 172.21.2.101:0/942294659 >> [v2:172.21.2.101:6803/25092,v1:172.21.2.101:6804/25092] conn(0x7f96240c45a0 0x7f96240bf970 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:23.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.737 7f9616ffd700 1 -- 172.21.2.101:0/942294659 <== osd.1 v2:172.21.2.101:6803/25092 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (crc 0 0 0) 0x7f9620001970 con 0x7f96240c45a0 2021-02-10T11:13:23.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.744 7f962c793700 1 -- 172.21.2.101:0/942294659 --> [v2:172.21.2.101:6803/25092,v1:172.21.2.101:6804/25092] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f96240bf0e0 con 0x7f96240c45a0 2021-02-10T11:13:23.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.744 7f9616ffd700 1 -- 172.21.2.101:0/942294659 <== osd.1 v2:172.21.2.101:6803/25092 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9620000f80 con 0x7f96240c45a0 2021-02-10T11:13:23.749 INFO:teuthology.orchestra.run.gibba001.stdout:55834574854 2021-02-10T11:13:23.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.745 7f962c793700 1 -- 172.21.2.101:0/942294659 >> [v2:172.21.2.101:6803/25092,v1:172.21.2.101:6804/25092] conn(0x7f96240c45a0 msgr2=0x7f96240bf970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:23.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.745 7f962c793700 1 --2- 172.21.2.101:0/942294659 >> [v2:172.21.2.101:6803/25092,v1:172.21.2.101:6804/25092] conn(0x7f96240c45a0 0x7f96240bf970 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:23.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.745 7f962c793700 1 -- 172.21.2.101:0/942294659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f960c018730 msgr2=0x7f960c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:23.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.745 7f962c793700 1 --2- 172.21.2.101:0/942294659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f960c018730 0x7f960c01ac10 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:23.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.745 7f962c793700 1 -- 172.21.2.101:0/942294659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9624199150 msgr2=0x7f96240ba680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:23.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.745 7f962c793700 1 --2- 172.21.2.101:0/942294659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9624199150 0x7f96240ba680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f961c00b910 tx=0x7f961c00bcb0).stop 2021-02-10T11:13:23.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.745 7f962adb0700 1 -- 172.21.2.101:0/942294659 reap_dead start 2021-02-10T11:13:23.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.746 7f962c793700 1 -- 172.21.2.101:0/942294659 shutdown_connections 2021-02-10T11:13:23.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.746 7f962c793700 1 -- 172.21.2.101:0/942294659 >> 172.21.2.101:0/942294659 conn(0x7f9624098740 msgr2=0x7f96240c3590 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:23.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.746 7f962c793700 1 -- 172.21.2.101:0/942294659 shutdown_connections 2021-02-10T11:13:23.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:23.746 7f962c793700 1 -- 172.21.2.101:0/942294659 wait complete. 2021-02-10T11:13:23.758 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell osd.2 flush_pg_stats 2021-02-10T11:13:24.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.164 7f52c65c2700 1 Processor -- start 2021-02-10T11:13:24.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.164 7f52c65c2700 1 -- start start 2021-02-10T11:13:24.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.165 7f52c65c2700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f52c00e5470 0x7f52c007e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:24.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.165 7f52c65c2700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c007ed80 con 0x7f52c00e5070 2021-02-10T11:13:24.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.165 7f52c65c2700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c01b4010 con 0x7f52c007f300 2021-02-10T11:13:24.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.165 7f52c65c2700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c00d1790 con 0x7f52c00e5470 2021-02-10T11:13:24.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.165 7f52bffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f52c00e5070 0x7f52c01b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44602/0 (socket says 172.21.2.101:44602) 2021-02-10T11:13:24.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.165 7f52bffff700 1 -- 172.21.2.101:0/920101751 learned_addr learned my addr 172.21.2.101:0/920101751 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:24.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52beffd700 1 -- 172.21.2.101:0/920101751 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1333969697 0 0) 0x7f52b0000ce0 con 0x7f52c00e5070 2021-02-10T11:13:24.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52bf7fe700 1 --2- 172.21.2.101:0/920101751 >> v2:172.21.2.108:3300/0 conn(0x7f52c00e5470 0x7f52c007e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:24.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52beffd700 1 -- 172.21.2.101:0/920101751 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52a8002fe0 con 0x7f52c00e5070 2021-02-10T11:13:24.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52beffd700 1 -- 172.21.2.101:0/920101751 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2625475687 0 0) 0x7f52b0001180 con 0x7f52c00e5070 2021-02-10T11:13:24.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52beffd700 1 -- 172.21.2.101:0/920101751 >> v2:172.21.2.108:3300/0 conn(0x7f52c00e5470 msgr2=0x7f52c007e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:24.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52beffd700 1 --2- 172.21.2.101:0/920101751 >> v2:172.21.2.108:3300/0 conn(0x7f52c00e5470 0x7f52c007e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:24.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52beffd700 1 -- 172.21.2.101:0/920101751 >> v1:172.21.2.108:6789/0 conn(0x7f52c007f300 legacy=0x7f52c007f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:24.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52beffd700 1 -- 172.21.2.101:0/920101751 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c00d19b0 con 0x7f52c00e5070 2021-02-10T11:13:24.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52beffd700 1 -- 172.21.2.101:0/920101751 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f52b0001680 con 0x7f52c00e5070 2021-02-10T11:13:24.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52beffd700 1 -- 172.21.2.101:0/920101751 >> v1:172.21.2.101:6789/0 conn(0x7f52c00e5070 legacy=0x7f52c01b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:24.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52beffd700 1 --2- 172.21.2.101:0/920101751 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52a8003ae0 0x7f52a8006000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:24.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52beffd700 1 --2- 172.21.2.101:0/920101751 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f52a8006820 0x7f52a8008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:24.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52beffd700 1 --2- 172.21.2.101:0/920101751 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f52a8009640 0x7f52a800bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:24.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52beffd700 1 -- 172.21.2.101:0/920101751 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a800c390 con 0x7f52a8003ae0 2021-02-10T11:13:24.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52beffd700 1 -- 172.21.2.101:0/920101751 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a800c5b0 con 0x7f52a8006820 2021-02-10T11:13:24.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52beffd700 1 -- 172.21.2.101:0/920101751 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a800c7d0 con 0x7f52a8009640 2021-02-10T11:13:24.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52bffff700 1 --2- 172.21.2.101:0/920101751 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52a8003ae0 0x7f52a8006000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:24.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52bf7fe700 1 --2- 172.21.2.101:0/920101751 >> v2:172.21.2.108:3300/0 conn(0x7f52c00e5470 0x7f52c007e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:13:24.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52bffff700 1 -- 172.21.2.101:0/920101751 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f52a8009640 msgr2=0x7f52a800bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:24.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52bffff700 1 --2- 172.21.2.101:0/920101751 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f52a8009640 0x7f52a800bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:24.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52bffff700 1 -- 172.21.2.101:0/920101751 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f52a8006820 msgr2=0x7f52a8008d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:24.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52bffff700 1 --2- 172.21.2.101:0/920101751 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f52a8006820 0x7f52a8008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:24.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52bffff700 1 -- 172.21.2.101:0/920101751 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a8050a40 con 0x7f52a8003ae0 2021-02-10T11:13:24.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.166 7f52c4bdf700 1 -- 172.21.2.101:0/920101751 reap_dead start 2021-02-10T11:13:24.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.167 7f52bffff700 1 --2- 172.21.2.101:0/920101751 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52a8003ae0 0x7f52a8006000 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f52b0000b60 tx=0x7f52b0007d20).ready entity=mon.0 client_cookie=f2e2e78cd217e109 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:24.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.167 7f52beffd700 1 -- 172.21.2.101:0/920101751 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f52b00070c0 con 0x7f52a8003ae0 2021-02-10T11:13:24.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.167 7f52beffd700 1 -- 172.21.2.101:0/920101751 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b00043a0 con 0x7f52a8003ae0 2021-02-10T11:13:24.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.167 7f52beffd700 1 -- 172.21.2.101:0/920101751 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f52b00048c0 con 0x7f52a8003ae0 2021-02-10T11:13:24.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.167 7f52c65c2700 1 -- 172.21.2.101:0/920101751 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52a8003ae0 msgr2=0x7f52a8006000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:24.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.167 7f52c65c2700 1 --2- 172.21.2.101:0/920101751 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52a8003ae0 0x7f52a8006000 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f52b0000b60 tx=0x7f52b0007d20).stop 2021-02-10T11:13:24.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.167 7f52c65c2700 1 -- 172.21.2.101:0/920101751 shutdown_connections 2021-02-10T11:13:24.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.167 7f52c65c2700 1 --2- 172.21.2.101:0/920101751 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52a8003ae0 0x7f52a8006000 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:24.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.167 7f52c65c2700 1 -- 172.21.2.101:0/920101751 >> 172.21.2.101:0/920101751 conn(0x7f52c00ce690 msgr2=0x7f52c00e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:24.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.167 7f52c65c2700 1 -- 172.21.2.101:0/920101751 shutdown_connections 2021-02-10T11:13:24.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.167 7f52c65c2700 1 -- 172.21.2.101:0/920101751 wait complete. 2021-02-10T11:13:24.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.182 7f52c65c2700 1 Processor -- start 2021-02-10T11:13:24.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.182 7f52c65c2700 1 -- start start 2021-02-10T11:13:24.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.182 7f52c65c2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f52c008f500 0x7f52c019f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:24.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.182 7f52c65c2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52c008fd90 0x7f52c0090190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:24.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.182 7f52c65c2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f52c00907b0 0x7f52c0094370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:24.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.182 7f52c65c2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c008f900 con 0x7f52c008fd90 2021-02-10T11:13:24.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.182 7f52c65c2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c019ff20 con 0x7f52c00907b0 2021-02-10T11:13:24.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.182 7f52c65c2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c01a0140 con 0x7f52c008f500 2021-02-10T11:13:24.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.182 7f52bf7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52c008fd90 0x7f52c0090190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:24.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.182 7f52bf7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52c008fd90 0x7f52c0090190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44122/0 (socket says 172.21.2.101:44122) 2021-02-10T11:13:24.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.182 7f52bf7fe700 1 -- 172.21.2.101:0/1085249324 learned_addr learned my addr 172.21.2.101:0/1085249324 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:24.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.183 7f52bf7fe700 1 -- 172.21.2.101:0/1085249324 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f52c008f500 msgr2=0x7f52c019f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:24.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.183 7f52bf7fe700 1 --2- 172.21.2.101:0/1085249324 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f52c008f500 0x7f52c019f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:24.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.183 7f52bf7fe700 1 -- 172.21.2.101:0/1085249324 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f52c00907b0 msgr2=0x7f52c0094370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:24.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.183 7f52bf7fe700 1 --2- 172.21.2.101:0/1085249324 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f52c00907b0 0x7f52c0094370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:24.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.183 7f52bf7fe700 1 -- 172.21.2.101:0/1085249324 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c0094990 con 0x7f52c008fd90 2021-02-10T11:13:24.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.183 7f52bf7fe700 1 --2- 172.21.2.101:0/1085249324 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52c008fd90 0x7f52c0090190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f52b400b5e0 tx=0x7f52b400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:24.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.183 7f52bcff9700 1 -- 172.21.2.101:0/1085249324 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f52b4006740 con 0x7f52c008fd90 2021-02-10T11:13:24.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.183 7f52bcff9700 1 -- 172.21.2.101:0/1085249324 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b4000940 con 0x7f52c008fd90 2021-02-10T11:13:24.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.183 7f52c65c2700 1 -- 172.21.2.101:0/1085249324 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c0094990 con 0x7f52c008fd90 2021-02-10T11:13:24.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.183 7f52bcff9700 1 -- 172.21.2.101:0/1085249324 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f52b40054d0 con 0x7f52c008fd90 2021-02-10T11:13:24.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.183 7f52c65c2700 1 -- 172.21.2.101:0/1085249324 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c0094990 con 0x7f52c008fd90 2021-02-10T11:13:24.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.184 7f52bcff9700 1 -- 172.21.2.101:0/1085249324 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f52b4006030 con 0x7f52c008fd90 2021-02-10T11:13:24.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.184 7f52bcff9700 1 --2- 172.21.2.101:0/1085249324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52a00186c0 0x7f52a001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:24.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.184 7f52bcff9700 1 -- 172.21.2.101:0/1085249324 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f52b40324d0 con 0x7f52c008fd90 2021-02-10T11:13:24.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.184 7f52bffff700 1 --2- 172.21.2.101:0/1085249324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52a00186c0 0x7f52a001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:24.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.185 7f52bffff700 1 --2- 172.21.2.101:0/1085249324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52a00186c0 0x7f52a001abc0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:24.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.215 7f52c65c2700 1 --2- 172.21.2.101:0/1085249324 >> [v2:172.21.2.101:6811/25094,v1:172.21.2.101:6813/25094] conn(0x7f52c00c45a0 0x7f52c00627c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:24.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.215 7f52c65c2700 1 -- 172.21.2.101:0/1085249324 --> [v2:172.21.2.101:6811/25094,v1:172.21.2.101:6813/25094] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52c019ff20 con 0x7f52c00c45a0 2021-02-10T11:13:24.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.215 7f52c4bdf700 1 --2- 172.21.2.101:0/1085249324 >> [v2:172.21.2.101:6811/25094,v1:172.21.2.101:6813/25094] conn(0x7f52c00c45a0 0x7f52c00627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:24.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.215 7f52c4bdf700 1 --2- 172.21.2.101:0/1085249324 >> [v2:172.21.2.101:6811/25094,v1:172.21.2.101:6813/25094] conn(0x7f52c00c45a0 0x7f52c00627c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:24.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.216 7f52bcff9700 1 -- 172.21.2.101:0/1085249324 <== osd.2 v2:172.21.2.101:6811/25094 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (crc 0 0 0) 0x7f52b8007ac0 con 0x7f52c00c45a0 2021-02-10T11:13:24.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.221 7f52c65c2700 1 -- 172.21.2.101:0/1085249324 --> [v2:172.21.2.101:6811/25094,v1:172.21.2.101:6813/25094] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f52c0084a90 con 0x7f52c00c45a0 2021-02-10T11:13:24.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.222 7f52bcff9700 1 -- 172.21.2.101:0/1085249324 <== osd.2 v2:172.21.2.101:6811/25094 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f52b8007ac0 con 0x7f52c00c45a0 2021-02-10T11:13:24.227 INFO:teuthology.orchestra.run.gibba001.stdout:55834574856 2021-02-10T11:13:24.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.223 7f52c65c2700 1 -- 172.21.2.101:0/1085249324 >> [v2:172.21.2.101:6811/25094,v1:172.21.2.101:6813/25094] conn(0x7f52c00c45a0 msgr2=0x7f52c00627c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:24.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.223 7f52c65c2700 1 --2- 172.21.2.101:0/1085249324 >> [v2:172.21.2.101:6811/25094,v1:172.21.2.101:6813/25094] conn(0x7f52c00c45a0 0x7f52c00627c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:24.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.223 7f52c65c2700 1 -- 172.21.2.101:0/1085249324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52a00186c0 msgr2=0x7f52a001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:24.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.223 7f52c65c2700 1 --2- 172.21.2.101:0/1085249324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52a00186c0 0x7f52a001abc0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:24.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.223 7f52c65c2700 1 -- 172.21.2.101:0/1085249324 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52c008fd90 msgr2=0x7f52c0090190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:24.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.223 7f52c65c2700 1 --2- 172.21.2.101:0/1085249324 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52c008fd90 0x7f52c0090190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f52b400b5e0 tx=0x7f52b400b980).stop 2021-02-10T11:13:24.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.223 7f52c4bdf700 1 -- 172.21.2.101:0/1085249324 reap_dead start 2021-02-10T11:13:24.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.223 7f52c65c2700 1 -- 172.21.2.101:0/1085249324 shutdown_connections 2021-02-10T11:13:24.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.223 7f52c65c2700 1 -- 172.21.2.101:0/1085249324 >> 172.21.2.101:0/1085249324 conn(0x7f52c00b5560 msgr2=0x7f52c00a6200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:24.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.223 7f52c65c2700 1 -- 172.21.2.101:0/1085249324 shutdown_connections 2021-02-10T11:13:24.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.223 7f52c65c2700 1 -- 172.21.2.101:0/1085249324 wait complete. 2021-02-10T11:13:24.236 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell osd.3 flush_pg_stats 2021-02-10T11:13:24.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.641 7ff60aefd700 1 Processor -- start 2021-02-10T11:13:24.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.641 7ff60aefd700 1 -- start start 2021-02-10T11:13:24.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.642 7ff60aefd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6040e5070 0x7ff6041b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:24.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.642 7ff60aefd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff6040e5470 0x7ff60407d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:24.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.642 7ff60aefd700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60407ed80 con 0x7ff6041b3e40 2021-02-10T11:13:24.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.642 7ff60aefd700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff604080310 con 0x7ff6040e5470 2021-02-10T11:13:24.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.642 7ff60aefd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff604080500 con 0x7ff6040e5070 2021-02-10T11:13:24.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.642 7ff603fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff6040e5470 0x7ff60407d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:24.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.642 7ff603fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff6040e5470 0x7ff60407d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44130/0 (socket says 172.21.2.101:44130) 2021-02-10T11:13:24.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.642 7ff603fff700 1 -- 172.21.2.101:0/1402609212 learned_addr learned my addr 172.21.2.101:0/1402609212 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:24.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.642 7ff60951a700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff6041b3e40 0x7ff60407dcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44628/0 (socket says 172.21.2.101:44628) 2021-02-10T11:13:24.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.642 7ff608d19700 1 --2- 172.21.2.101:0/1402609212 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6040e5070 0x7ff6041b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:24.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.642 7ff603fff700 1 -- 172.21.2.101:0/1402609212 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6040e5070 msgr2=0x7ff6041b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:24.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.642 7ff603fff700 1 --2- 172.21.2.101:0/1402609212 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6040e5070 0x7ff6041b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:24.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.642 7ff603fff700 1 -- 172.21.2.101:0/1402609212 >> v1:172.21.2.101:6789/0 conn(0x7ff6041b3e40 legacy=0x7ff60407dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:24.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.642 7ff603fff700 1 -- 172.21.2.101:0/1402609212 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6040806f0 con 0x7ff6040e5470 2021-02-10T11:13:24.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.643 7ff603fff700 1 --2- 172.21.2.101:0/1402609212 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6040e5470 0x7ff60407d690 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7ff5f4006700 tx=0x7ff5f4009b00).ready entity=mon.0 client_cookie=557b97f4747d6875 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:24.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.643 7ff6037fe700 1 -- 172.21.2.101:0/1402609212 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff5f400c0c0 con 0x7ff6040e5470 2021-02-10T11:13:24.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.643 7ff6037fe700 1 -- 172.21.2.101:0/1402609212 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f400c400 con 0x7ff6040e5470 2021-02-10T11:13:24.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.643 7ff6037fe700 1 -- 172.21.2.101:0/1402609212 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff5f400c8e0 con 0x7ff6040e5470 2021-02-10T11:13:24.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.643 7ff60aefd700 1 -- 172.21.2.101:0/1402609212 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6040e5470 msgr2=0x7ff60407d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:24.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.643 7ff60aefd700 1 --2- 172.21.2.101:0/1402609212 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6040e5470 0x7ff60407d690 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7ff5f4006700 tx=0x7ff5f4009b00).stop 2021-02-10T11:13:24.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.644 7ff60aefd700 1 -- 172.21.2.101:0/1402609212 shutdown_connections 2021-02-10T11:13:24.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.644 7ff60aefd700 1 --2- 172.21.2.101:0/1402609212 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6040e5470 0x7ff60407d690 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:24.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.644 7ff60aefd700 1 --2- 172.21.2.101:0/1402609212 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6040e5070 0x7ff6041b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:24.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.644 7ff60aefd700 1 -- 172.21.2.101:0/1402609212 >> 172.21.2.101:0/1402609212 conn(0x7ff6040ce690 msgr2=0x7ff6040e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:24.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.644 7ff60aefd700 1 -- 172.21.2.101:0/1402609212 shutdown_connections 2021-02-10T11:13:24.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.644 7ff60aefd700 1 -- 172.21.2.101:0/1402609212 wait complete. 2021-02-10T11:13:24.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.658 7ff60aefd700 1 Processor -- start 2021-02-10T11:13:24.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.658 7ff60aefd700 1 -- start start 2021-02-10T11:13:24.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.658 7ff60aefd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff60408f500 0x7ff60419f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:24.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.658 7ff60aefd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff60408fd90 0x7ff604090190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:24.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.658 7ff60aefd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6040907b0 0x7ff604094370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:24.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.658 7ff60aefd700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60408f900 con 0x7ff6040907b0 2021-02-10T11:13:24.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.658 7ff60aefd700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60419ff20 con 0x7ff60408f500 2021-02-10T11:13:24.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.658 7ff60aefd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6041a0140 con 0x7ff60408fd90 2021-02-10T11:13:24.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.659 7ff608d19700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff60408f500 0x7ff60419f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:24.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.659 7ff608d19700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff60408f500 0x7ff60419f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:54622/0 (socket says 172.21.2.101:54622) 2021-02-10T11:13:24.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.659 7ff608d19700 1 -- 172.21.2.101:0/1555019716 learned_addr learned my addr 172.21.2.101:0/1555019716 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:24.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.659 7ff603fff700 1 --2- 172.21.2.101:0/1555019716 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff60408fd90 0x7ff604090190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:24.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.659 7ff60951a700 1 --2- 172.21.2.101:0/1555019716 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6040907b0 0x7ff604094370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:24.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.659 7ff608d19700 1 -- 172.21.2.101:0/1555019716 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff60408fd90 msgr2=0x7ff604090190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:24.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.659 7ff608d19700 1 --2- 172.21.2.101:0/1555019716 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff60408fd90 0x7ff604090190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:24.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.659 7ff608d19700 1 -- 172.21.2.101:0/1555019716 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6040907b0 msgr2=0x7ff604094370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:24.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.659 7ff608d19700 1 --2- 172.21.2.101:0/1555019716 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6040907b0 0x7ff604094370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:24.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.659 7ff608d19700 1 -- 172.21.2.101:0/1555019716 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff604094990 con 0x7ff60408f500 2021-02-10T11:13:24.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.659 7ff608d19700 1 --2- 172.21.2.101:0/1555019716 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff60408f500 0x7ff60419f900 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7ff5ec006f60 tx=0x7ff5ec00b970).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:24.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.660 7ff6017fa700 1 -- 172.21.2.101:0/1555019716 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff5ec0009b0 con 0x7ff60408f500 2021-02-10T11:13:24.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.660 7ff6017fa700 1 -- 172.21.2.101:0/1555019716 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ec00bdf0 con 0x7ff60408f500 2021-02-10T11:13:24.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.660 7ff6017fa700 1 -- 172.21.2.101:0/1555019716 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff5ec002030 con 0x7ff60408f500 2021-02-10T11:13:24.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.660 7ff60aefd700 1 -- 172.21.2.101:0/1555019716 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff604094990 con 0x7ff60408f500 2021-02-10T11:13:24.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.660 7ff60aefd700 1 -- 172.21.2.101:0/1555019716 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff604094f20 con 0x7ff60408f500 2021-02-10T11:13:24.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.661 7ff6017fa700 1 -- 172.21.2.101:0/1555019716 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff5ec006b00 con 0x7ff60408f500 2021-02-10T11:13:24.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.661 7ff6017fa700 1 --2- 172.21.2.101:0/1555019716 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5e80186c0 0x7ff5e801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:24.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.661 7ff6017fa700 1 -- 172.21.2.101:0/1555019716 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff5ec032510 con 0x7ff60408f500 2021-02-10T11:13:24.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.661 7ff603fff700 1 --2- 172.21.2.101:0/1555019716 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5e80186c0 0x7ff5e801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:24.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.661 7ff603fff700 1 --2- 172.21.2.101:0/1555019716 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5e80186c0 0x7ff5e801abc0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:24.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.692 7ff60aefd700 1 --2- 172.21.2.101:0/1555019716 >> [v2:172.21.2.101:6824/25095,v1:172.21.2.101:6826/25095] conn(0x7ff6040c45a0 0x7ff6040627c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:24.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.692 7ff60aefd700 1 -- 172.21.2.101:0/1555019716 --> [v2:172.21.2.101:6824/25095,v1:172.21.2.101:6826/25095] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff604094f20 con 0x7ff6040c45a0 2021-02-10T11:13:24.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.692 7ff60951a700 1 --2- 172.21.2.101:0/1555019716 >> [v2:172.21.2.101:6824/25095,v1:172.21.2.101:6826/25095] conn(0x7ff6040c45a0 0x7ff6040627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:24.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.692 7ff60951a700 1 --2- 172.21.2.101:0/1555019716 >> [v2:172.21.2.101:6824/25095,v1:172.21.2.101:6826/25095] conn(0x7ff6040c45a0 0x7ff6040627c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:24.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.693 7ff6017fa700 1 -- 172.21.2.101:0/1555019716 <== osd.3 v2:172.21.2.101:6824/25095 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (crc 0 0 0) 0x7ff5fc006920 con 0x7ff6040c45a0 2021-02-10T11:13:24.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.700 7ff60aefd700 1 -- 172.21.2.101:0/1555019716 --> [v2:172.21.2.101:6824/25095,v1:172.21.2.101:6826/25095] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff604094f20 con 0x7ff6040c45a0 2021-02-10T11:13:24.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.700 7ff6017fa700 1 -- 172.21.2.101:0/1555019716 <== osd.3 v2:172.21.2.101:6824/25095 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff5fc006920 con 0x7ff6040c45a0 2021-02-10T11:13:24.705 INFO:teuthology.orchestra.run.gibba001.stdout:55834574856 2021-02-10T11:13:24.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.701 7ff60aefd700 1 -- 172.21.2.101:0/1555019716 >> [v2:172.21.2.101:6824/25095,v1:172.21.2.101:6826/25095] conn(0x7ff6040c45a0 msgr2=0x7ff6040627c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:24.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.701 7ff60aefd700 1 --2- 172.21.2.101:0/1555019716 >> [v2:172.21.2.101:6824/25095,v1:172.21.2.101:6826/25095] conn(0x7ff6040c45a0 0x7ff6040627c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:24.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.701 7ff60aefd700 1 -- 172.21.2.101:0/1555019716 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5e80186c0 msgr2=0x7ff5e801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:24.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.701 7ff60aefd700 1 --2- 172.21.2.101:0/1555019716 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5e80186c0 0x7ff5e801abc0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:24.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.701 7ff60aefd700 1 -- 172.21.2.101:0/1555019716 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff60408f500 msgr2=0x7ff60419f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:24.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.701 7ff60aefd700 1 --2- 172.21.2.101:0/1555019716 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff60408f500 0x7ff60419f900 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7ff5ec006f60 tx=0x7ff5ec00b970).stop 2021-02-10T11:13:24.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.701 7ff60951a700 1 -- 172.21.2.101:0/1555019716 reap_dead start 2021-02-10T11:13:24.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.702 7ff60aefd700 1 -- 172.21.2.101:0/1555019716 shutdown_connections 2021-02-10T11:13:24.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.702 7ff60aefd700 1 -- 172.21.2.101:0/1555019716 >> 172.21.2.101:0/1555019716 conn(0x7ff6040b5560 msgr2=0x7ff6040a6200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:24.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.702 7ff60aefd700 1 -- 172.21.2.101:0/1555019716 shutdown_connections 2021-02-10T11:13:24.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:24.702 7ff60aefd700 1 -- 172.21.2.101:0/1555019716 wait complete. 2021-02-10T11:13:24.713 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell osd.4 flush_pg_stats 2021-02-10T11:13:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.120 7f352be92700 1 Processor -- start 2021-02-10T11:13:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.120 7f352be92700 1 -- start start 2021-02-10T11:13:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.121 7f352be92700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35240e5070 0x7f35241b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.121 7f352be92700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f35240e5470 0x7f352407d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.121 7f352be92700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f35241b3e40 0x7f352407dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.121 7f352be92700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f352407ed80 con 0x7f35241b3e40 2021-02-10T11:13:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.121 7f352be92700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f352407e2d0 con 0x7f35240e5470 2021-02-10T11:13:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.121 7f352be92700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f352407e4c0 con 0x7f35240e5070 2021-02-10T11:13:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.121 7f352a4af700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f35241b3e40 0x7f352407dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.121 7f352a4af700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f35241b3e40 0x7f352407dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44150/0 (socket says 172.21.2.101:44150) 2021-02-10T11:13:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.121 7f352a4af700 1 -- 172.21.2.101:0/2975833678 learned_addr learned my addr 172.21.2.101:0/2975833678 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.121 7f352a4af700 1 -- 172.21.2.101:0/2975833678 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35240e5070 msgr2=0x7f35241b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.121 7f352a4af700 1 --2- 172.21.2.101:0/2975833678 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35240e5070 0x7f35241b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.121 7f352a4af700 1 -- 172.21.2.101:0/2975833678 >> v2:172.21.2.108:3300/0 conn(0x7f35240e5470 msgr2=0x7f352407d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.121 7f35294ad700 1 --2- 172.21.2.101:0/2975833678 >> v2:172.21.2.108:3300/0 conn(0x7f35240e5470 0x7f352407d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.121 7f352a4af700 1 --2- 172.21.2.101:0/2975833678 >> v2:172.21.2.108:3300/0 conn(0x7f35240e5470 0x7f352407d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.121 7f352a4af700 1 -- 172.21.2.101:0/2975833678 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352407e6e0 con 0x7f35241b3e40 2021-02-10T11:13:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.121 7f352a4af700 1 --2- 172.21.2.101:0/2975833678 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35241b3e40 0x7f352407dcb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f351c006f00 tx=0x7f351c009f10).ready entity=mon.0 client_cookie=1bc4a850a8fa788d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.121 7f3528cac700 1 -- 172.21.2.101:0/2975833678 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f351c00b3b0 con 0x7f35241b3e40 2021-02-10T11:13:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.121 7f3528cac700 1 -- 172.21.2.101:0/2975833678 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351c00b730 con 0x7f35241b3e40 2021-02-10T11:13:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.121 7f3528cac700 1 -- 172.21.2.101:0/2975833678 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f351c00bc10 con 0x7f35241b3e40 2021-02-10T11:13:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.121 7f352be92700 1 -- 172.21.2.101:0/2975833678 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35241b3e40 msgr2=0x7f352407dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.121 7f352be92700 1 --2- 172.21.2.101:0/2975833678 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35241b3e40 0x7f352407dcb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f351c006f00 tx=0x7f351c009f10).stop 2021-02-10T11:13:25.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.122 7f352be92700 1 -- 172.21.2.101:0/2975833678 shutdown_connections 2021-02-10T11:13:25.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.122 7f352be92700 1 --2- 172.21.2.101:0/2975833678 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35241b3e40 0x7f352407dcb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:25.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.122 7f352be92700 1 --2- 172.21.2.101:0/2975833678 >> v2:172.21.2.108:3300/0 conn(0x7f35240e5470 0x7f352407d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:25.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.122 7f352be92700 1 --2- 172.21.2.101:0/2975833678 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35240e5070 0x7f35241b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:25.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.122 7f352be92700 1 -- 172.21.2.101:0/2975833678 >> 172.21.2.101:0/2975833678 conn(0x7f35240ce690 msgr2=0x7f35240e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:25.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.122 7f352be92700 1 -- 172.21.2.101:0/2975833678 shutdown_connections 2021-02-10T11:13:25.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.122 7f352be92700 1 -- 172.21.2.101:0/2975833678 wait complete. 2021-02-10T11:13:25.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.137 7f352be92700 1 Processor -- start 2021-02-10T11:13:25.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.137 7f352be92700 1 -- start start 2021-02-10T11:13:25.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.137 7f352be92700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f352408f500 0x7f352419f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:25.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.137 7f352be92700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f352408fd90 0x7f3524090190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:25.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.137 7f352be92700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35240907b0 0x7f3524094370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:25.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.137 7f352be92700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352408f900 con 0x7f35240907b0 2021-02-10T11:13:25.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.137 7f352be92700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352419ff20 con 0x7f352408fd90 2021-02-10T11:13:25.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.137 7f352be92700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35241a0140 con 0x7f352408f500 2021-02-10T11:13:25.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.138 7f352a4af700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35240907b0 0x7f3524094370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:25.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.138 7f352a4af700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35240907b0 0x7f3524094370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44152/0 (socket says 172.21.2.101:44152) 2021-02-10T11:13:25.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.138 7f352a4af700 1 -- 172.21.2.101:0/2092369785 learned_addr learned my addr 172.21.2.101:0/2092369785 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:25.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.138 7f352a4af700 1 -- 172.21.2.101:0/2092369785 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f352408f500 msgr2=0x7f352419f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:25.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.138 7f352a4af700 1 --2- 172.21.2.101:0/2092369785 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f352408f500 0x7f352419f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:25.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.138 7f352a4af700 1 -- 172.21.2.101:0/2092369785 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f352408fd90 msgr2=0x7f3524090190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:25.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.138 7f352a4af700 1 --2- 172.21.2.101:0/2092369785 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f352408fd90 0x7f3524090190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:25.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.138 7f352a4af700 1 -- 172.21.2.101:0/2092369785 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3524094990 con 0x7f35240907b0 2021-02-10T11:13:25.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.138 7f352a4af700 1 --2- 172.21.2.101:0/2092369785 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35240907b0 0x7f3524094370 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f351c00b9b0 tx=0x7f351c00bd50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:25.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.138 7f35167fc700 1 -- 172.21.2.101:0/2092369785 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f351c005820 con 0x7f35240907b0 2021-02-10T11:13:25.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.138 7f35167fc700 1 -- 172.21.2.101:0/2092369785 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351c005aa0 con 0x7f35240907b0 2021-02-10T11:13:25.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.138 7f352be92700 1 -- 172.21.2.101:0/2092369785 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3524094990 con 0x7f35240907b0 2021-02-10T11:13:25.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.138 7f35167fc700 1 -- 172.21.2.101:0/2092369785 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f351c006030 con 0x7f35240907b0 2021-02-10T11:13:25.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.139 7f352be92700 1 -- 172.21.2.101:0/2092369785 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3524094990 con 0x7f35240907b0 2021-02-10T11:13:25.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.139 7f35167fc700 1 -- 172.21.2.101:0/2092369785 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f351c005c80 con 0x7f35240907b0 2021-02-10T11:13:25.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.139 7f35167fc700 1 --2- 172.21.2.101:0/2092369785 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f350c018730 0x7f350c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:25.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.139 7f35167fc700 1 -- 172.21.2.101:0/2092369785 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f351c032990 con 0x7f35240907b0 2021-02-10T11:13:25.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.139 7f3529cae700 1 --2- 172.21.2.101:0/2092369785 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f350c018730 0x7f350c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:25.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.140 7f3529cae700 1 --2- 172.21.2.101:0/2092369785 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f350c018730 0x7f350c01ac10 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:25.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.170 7f352be92700 1 --2- 172.21.2.101:0/2092369785 >> [v2:172.21.2.108:6816/36380,v1:172.21.2.108:6818/36380] conn(0x7f35240c45a0 0x7f35240627c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:25.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.170 7f352be92700 1 -- 172.21.2.101:0/2092369785 --> [v2:172.21.2.108:6816/36380,v1:172.21.2.108:6818/36380] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f352419ff20 con 0x7f35240c45a0 2021-02-10T11:13:25.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.171 7f35294ad700 1 --2- 172.21.2.101:0/2092369785 >> [v2:172.21.2.108:6816/36380,v1:172.21.2.108:6818/36380] conn(0x7f35240c45a0 0x7f35240627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:25.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.171 7f35294ad700 1 --2- 172.21.2.101:0/2092369785 >> [v2:172.21.2.108:6816/36380,v1:172.21.2.108:6818/36380] conn(0x7f35240c45a0 0x7f35240627c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:25.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.172 7f35167fc700 1 -- 172.21.2.101:0/2092369785 <== osd.4 v2:172.21.2.108:6816/36380 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (crc 0 0 0) 0x7f3520007940 con 0x7f35240c45a0 2021-02-10T11:13:25.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.178 7f352be92700 1 -- 172.21.2.101:0/2092369785 --> [v2:172.21.2.108:6816/36380,v1:172.21.2.108:6818/36380] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3524084a90 con 0x7f35240c45a0 2021-02-10T11:13:25.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.179 7f35167fc700 1 -- 172.21.2.101:0/2092369785 <== osd.4 v2:172.21.2.108:6816/36380 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f35200009b0 con 0x7f35240c45a0 2021-02-10T11:13:25.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.180 7f352be92700 1 -- 172.21.2.101:0/2092369785 >> [v2:172.21.2.108:6816/36380,v1:172.21.2.108:6818/36380] conn(0x7f35240c45a0 msgr2=0x7f35240627c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:25.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.180 7f352be92700 1 --2- 172.21.2.101:0/2092369785 >> [v2:172.21.2.108:6816/36380,v1:172.21.2.108:6818/36380] conn(0x7f35240c45a0 0x7f35240627c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:25.184 INFO:teuthology.orchestra.run.gibba001.stdout:55834574856 2021-02-10T11:13:25.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.180 7f352be92700 1 -- 172.21.2.101:0/2092369785 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f350c018730 msgr2=0x7f350c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:25.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.180 7f352be92700 1 --2- 172.21.2.101:0/2092369785 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f350c018730 0x7f350c01ac10 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:25.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.180 7f352be92700 1 -- 172.21.2.101:0/2092369785 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35240907b0 msgr2=0x7f3524094370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:25.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.180 7f352be92700 1 --2- 172.21.2.101:0/2092369785 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35240907b0 0x7f3524094370 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f351c00b9b0 tx=0x7f351c00bd50).stop 2021-02-10T11:13:25.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.180 7f352a4af700 1 -- 172.21.2.101:0/2092369785 reap_dead start 2021-02-10T11:13:25.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.180 7f352be92700 1 -- 172.21.2.101:0/2092369785 shutdown_connections 2021-02-10T11:13:25.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.180 7f352be92700 1 -- 172.21.2.101:0/2092369785 >> 172.21.2.101:0/2092369785 conn(0x7f35240b5560 msgr2=0x7f35240a6200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:25.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.180 7f352be92700 1 -- 172.21.2.101:0/2092369785 shutdown_connections 2021-02-10T11:13:25.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.180 7f352be92700 1 -- 172.21.2.101:0/2092369785 wait complete. 2021-02-10T11:13:25.192 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell osd.5 flush_pg_stats 2021-02-10T11:13:25.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.598 7f26f6c62700 1 Processor -- start 2021-02-10T11:13:25.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.598 7f26f6c62700 1 -- start start 2021-02-10T11:13:25.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.598 7f26f6c62700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f26f00e5070 0x7f26f01b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:25.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.598 7f26f6c62700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f26f01b3e40 0x7f26f007d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:25.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.598 7f26f6c62700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f007ed80 con 0x7f26f00e5470 2021-02-10T11:13:25.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.598 7f26f6c62700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f0080310 con 0x7f26f00e5070 2021-02-10T11:13:25.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.598 7f26f6c62700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f0080530 con 0x7f26f01b3e40 2021-02-10T11:13:25.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.598 7f26f4a7e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f26f00e5070 0x7f26f01b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:25.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.598 7f26f4a7e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f26f00e5070 0x7f26f01b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44162/0 (socket says 172.21.2.101:44162) 2021-02-10T11:13:25.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.598 7f26f4a7e700 1 -- 172.21.2.101:0/2492487980 learned_addr learned my addr 172.21.2.101:0/2492487980 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:25.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.599 7f26f4a7e700 1 -- 172.21.2.101:0/2492487980 >> v2:172.21.2.108:3300/0 conn(0x7f26f01b3e40 msgr2=0x7f26f007d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:25.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.599 7f26f4a7e700 1 --2- 172.21.2.101:0/2492487980 >> v2:172.21.2.108:3300/0 conn(0x7f26f01b3e40 0x7f26f007d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:25.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.599 7f26f4a7e700 1 -- 172.21.2.101:0/2492487980 >> v1:172.21.2.108:6789/0 conn(0x7f26f00e5470 legacy=0x7f26f007dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:25.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.599 7f26f4a7e700 1 -- 172.21.2.101:0/2492487980 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f0080750 con 0x7f26f00e5070 2021-02-10T11:13:25.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.599 7f26f4a7e700 1 --2- 172.21.2.101:0/2492487980 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26f00e5070 0x7f26f01b3820 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f26d8006700 tx=0x7f26d8009b80).ready entity=mon.0 client_cookie=d932cb57c6c03eb8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:25.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.599 7f26ef7fe700 1 -- 172.21.2.101:0/2492487980 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f26d800c0c0 con 0x7f26f00e5070 2021-02-10T11:13:25.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.599 7f26ef7fe700 1 -- 172.21.2.101:0/2492487980 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d800c340 con 0x7f26f00e5070 2021-02-10T11:13:25.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.599 7f26ef7fe700 1 -- 172.21.2.101:0/2492487980 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f26d800c860 con 0x7f26f00e5070 2021-02-10T11:13:25.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.599 7f26f6c62700 1 -- 172.21.2.101:0/2492487980 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26f00e5070 msgr2=0x7f26f01b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:25.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.599 7f26f6c62700 1 --2- 172.21.2.101:0/2492487980 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26f00e5070 0x7f26f01b3820 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f26d8006700 tx=0x7f26d8009b80).stop 2021-02-10T11:13:25.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.600 7f26f6c62700 1 -- 172.21.2.101:0/2492487980 shutdown_connections 2021-02-10T11:13:25.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.600 7f26f6c62700 1 --2- 172.21.2.101:0/2492487980 >> v2:172.21.2.108:3300/0 conn(0x7f26f01b3e40 0x7f26f007d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:25.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.600 7f26f6c62700 1 --2- 172.21.2.101:0/2492487980 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26f00e5070 0x7f26f01b3820 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:25.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.600 7f26f6c62700 1 -- 172.21.2.101:0/2492487980 >> 172.21.2.101:0/2492487980 conn(0x7f26f00ce690 msgr2=0x7f26f00e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:25.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.600 7f26f6c62700 1 -- 172.21.2.101:0/2492487980 shutdown_connections 2021-02-10T11:13:25.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.600 7f26f6c62700 1 -- 172.21.2.101:0/2492487980 wait complete. 2021-02-10T11:13:25.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.614 7f26f6c62700 1 Processor -- start 2021-02-10T11:13:25.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.614 7f26f6c62700 1 -- start start 2021-02-10T11:13:25.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.615 7f26f6c62700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f26f00be580 0x7f26f00be980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:25.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.615 7f26f6c62700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26f0097ed0 0x7f26f00982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:25.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.615 7f26f6c62700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26f0099180 0x7f26f0099580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:25.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.615 7f26f6c62700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f00988f0 con 0x7f26f0097ed0 2021-02-10T11:13:25.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.615 7f26f6c62700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f00c1110 con 0x7f26f00be580 2021-02-10T11:13:25.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.615 7f26f6c62700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f0099ba0 con 0x7f26f0099180 2021-02-10T11:13:25.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.615 7f26effff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26f0097ed0 0x7f26f00982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:25.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.615 7f26effff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26f0097ed0 0x7f26f00982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44168/0 (socket says 172.21.2.101:44168) 2021-02-10T11:13:25.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.615 7f26effff700 1 -- 172.21.2.101:0/3471778883 learned_addr learned my addr 172.21.2.101:0/3471778883 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:25.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.615 7f26f4a7e700 1 --2- 172.21.2.101:0/3471778883 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f26f00be580 0x7f26f00be980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:25.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.615 7f26effff700 1 -- 172.21.2.101:0/3471778883 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26f0099180 msgr2=0x7f26f0099580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:25.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.615 7f26f527f700 1 --2- 172.21.2.101:0/3471778883 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26f0099180 0x7f26f0099580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:25.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.615 7f26effff700 1 --2- 172.21.2.101:0/3471778883 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26f0099180 0x7f26f0099580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:25.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.615 7f26effff700 1 -- 172.21.2.101:0/3471778883 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f26f00be580 msgr2=0x7f26f00be980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:25.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.615 7f26effff700 1 --2- 172.21.2.101:0/3471778883 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f26f00be580 0x7f26f00be980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:25.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.615 7f26effff700 1 -- 172.21.2.101:0/3471778883 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f0099dc0 con 0x7f26f0097ed0 2021-02-10T11:13:25.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.616 7f26f527f700 1 --2- 172.21.2.101:0/3471778883 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26f0099180 0x7f26f0099580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:13:25.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.616 7f26f4a7e700 1 --2- 172.21.2.101:0/3471778883 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f26f00be580 0x7f26f00be980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:13:25.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.616 7f26effff700 1 --2- 172.21.2.101:0/3471778883 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26f0097ed0 0x7f26f00982d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f26e0006f00 tx=0x7f26e000b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:25.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.616 7f26ed7fa700 1 -- 172.21.2.101:0/3471778883 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f26e00009b0 con 0x7f26f0097ed0 2021-02-10T11:13:25.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.616 7f26ed7fa700 1 -- 172.21.2.101:0/3471778883 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e0006350 con 0x7f26f0097ed0 2021-02-10T11:13:25.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.616 7f26ed7fa700 1 -- 172.21.2.101:0/3471778883 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f26e0000c90 con 0x7f26f0097ed0 2021-02-10T11:13:25.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.616 7f26f6c62700 1 -- 172.21.2.101:0/3471778883 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f0099dc0 con 0x7f26f0097ed0 2021-02-10T11:13:25.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.616 7f26f6c62700 1 -- 172.21.2.101:0/3471778883 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f0099dc0 con 0x7f26f0097ed0 2021-02-10T11:13:25.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.616 7f26ed7fa700 1 -- 172.21.2.101:0/3471778883 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f26e0000c90 con 0x7f26f0097ed0 2021-02-10T11:13:25.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.617 7f26ed7fa700 1 --2- 172.21.2.101:0/3471778883 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26d40186c0 0x7f26d401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:25.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.617 7f26ed7fa700 1 -- 172.21.2.101:0/3471778883 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f26e0032440 con 0x7f26f0097ed0 2021-02-10T11:13:25.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.617 7f26f4a7e700 1 --2- 172.21.2.101:0/3471778883 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26d40186c0 0x7f26d401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:25.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.617 7f26f4a7e700 1 --2- 172.21.2.101:0/3471778883 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26d40186c0 0x7f26d401abc0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:25.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.648 7f26f6c62700 1 --2- 172.21.2.101:0/3471778883 >> [v2:172.21.2.108:6808/36383,v1:172.21.2.108:6809/36383] conn(0x7f26f00c45a0 0x7f26f00627c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:25.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.648 7f26f6c62700 1 -- 172.21.2.101:0/3471778883 --> [v2:172.21.2.108:6808/36383,v1:172.21.2.108:6809/36383] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f26f0098c50 con 0x7f26f00c45a0 2021-02-10T11:13:25.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.649 7f26f527f700 1 --2- 172.21.2.101:0/3471778883 >> [v2:172.21.2.108:6808/36383,v1:172.21.2.108:6809/36383] conn(0x7f26f00c45a0 0x7f26f00627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:25.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.650 7f26f527f700 1 --2- 172.21.2.101:0/3471778883 >> [v2:172.21.2.108:6808/36383,v1:172.21.2.108:6809/36383] conn(0x7f26f00c45a0 0x7f26f00627c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:25.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.651 7f26ed7fa700 1 -- 172.21.2.101:0/3471778883 <== osd.5 v2:172.21.2.108:6808/36383 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (crc 0 0 0) 0x7f26e8006c20 con 0x7f26f00c45a0 2021-02-10T11:13:25.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.656 7f26f6c62700 1 -- 172.21.2.101:0/3471778883 --> [v2:172.21.2.108:6808/36383,v1:172.21.2.108:6809/36383] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f26f00988f0 con 0x7f26f00c45a0 2021-02-10T11:13:25.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.657 7f26ed7fa700 1 -- 172.21.2.101:0/3471778883 <== osd.5 v2:172.21.2.108:6808/36383 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f26e8006c20 con 0x7f26f00c45a0 2021-02-10T11:13:25.661 INFO:teuthology.orchestra.run.gibba001.stdout:55834574856 2021-02-10T11:13:25.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.657 7f26f6c62700 1 -- 172.21.2.101:0/3471778883 >> [v2:172.21.2.108:6808/36383,v1:172.21.2.108:6809/36383] conn(0x7f26f00c45a0 msgr2=0x7f26f00627c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:25.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.657 7f26f6c62700 1 --2- 172.21.2.101:0/3471778883 >> [v2:172.21.2.108:6808/36383,v1:172.21.2.108:6809/36383] conn(0x7f26f00c45a0 0x7f26f00627c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:25.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.658 7f26f6c62700 1 -- 172.21.2.101:0/3471778883 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26d40186c0 msgr2=0x7f26d401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:25.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.658 7f26f6c62700 1 --2- 172.21.2.101:0/3471778883 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26d40186c0 0x7f26d401abc0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:25.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.658 7f26f6c62700 1 -- 172.21.2.101:0/3471778883 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26f0097ed0 msgr2=0x7f26f00982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:25.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.658 7f26f6c62700 1 --2- 172.21.2.101:0/3471778883 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26f0097ed0 0x7f26f00982d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f26e0006f00 tx=0x7f26e000b940).stop 2021-02-10T11:13:25.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.658 7f26f527f700 1 -- 172.21.2.101:0/3471778883 reap_dead start 2021-02-10T11:13:25.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.658 7f26f6c62700 1 -- 172.21.2.101:0/3471778883 shutdown_connections 2021-02-10T11:13:25.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.658 7f26f6c62700 1 -- 172.21.2.101:0/3471778883 >> 172.21.2.101:0/3471778883 conn(0x7f26f00bde70 msgr2=0x7f26f00c3970 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:25.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.658 7f26f6c62700 1 -- 172.21.2.101:0/3471778883 shutdown_connections 2021-02-10T11:13:25.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:25.658 7f26f6c62700 1 -- 172.21.2.101:0/3471778883 wait complete. 2021-02-10T11:13:25.669 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell osd.6 flush_pg_stats 2021-02-10T11:13:26.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.073 7f7eeaef7700 1 Processor -- start 2021-02-10T11:13:26.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.074 7f7eeaef7700 1 -- start start 2021-02-10T11:13:26.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.074 7f7eeaef7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7ee40e5070 0x7f7ee41b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:26.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.074 7f7eeaef7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7ee41b3e40 0x7f7ee407d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:26.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.074 7f7eeaef7700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee407ed80 con 0x7f7ee40e5470 2021-02-10T11:13:26.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.074 7f7eeaef7700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee4080310 con 0x7f7ee41b3e40 2021-02-10T11:13:26.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.074 7f7eeaef7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee4080530 con 0x7f7ee40e5070 2021-02-10T11:13:26.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.074 7f7ee3fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7ee41b3e40 0x7f7ee407d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:26.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.074 7f7ee9514700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f7ee40e5470 0x7f7ee407dcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44674/0 (socket says 172.21.2.101:44674) 2021-02-10T11:13:26.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.074 7f7ee9514700 1 -- 172.21.2.101:0/3086268266 learned_addr learned my addr 172.21.2.101:0/3086268266 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:26.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.074 7f7ee8d13700 1 --2- 172.21.2.101:0/3086268266 >> v2:172.21.2.108:3300/0 conn(0x7f7ee40e5070 0x7f7ee41b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:26.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.075 7f7ee3fff700 1 -- 172.21.2.101:0/3086268266 >> v2:172.21.2.108:3300/0 conn(0x7f7ee40e5070 msgr2=0x7f7ee41b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:26.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.075 7f7ee3fff700 1 --2- 172.21.2.101:0/3086268266 >> v2:172.21.2.108:3300/0 conn(0x7f7ee40e5070 0x7f7ee41b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:26.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.075 7f7ee3fff700 1 -- 172.21.2.101:0/3086268266 >> v1:172.21.2.101:6789/0 conn(0x7f7ee40e5470 legacy=0x7f7ee407dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:26.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.075 7f7ee3fff700 1 -- 172.21.2.101:0/3086268266 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee4080750 con 0x7f7ee41b3e40 2021-02-10T11:13:26.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.075 7f7ee8d13700 1 --2- 172.21.2.101:0/3086268266 >> v2:172.21.2.108:3300/0 conn(0x7f7ee40e5070 0x7f7ee41b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:13:26.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.075 7f7ee3fff700 1 --2- 172.21.2.101:0/3086268266 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ee41b3e40 0x7f7ee407d690 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f7ed0006650 tx=0x7f7ed0009a80).ready entity=mon.0 client_cookie=4965c7504c6adf02 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:26.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.075 7f7ee37fe700 1 -- 172.21.2.101:0/3086268266 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ed00009b0 con 0x7f7ee41b3e40 2021-02-10T11:13:26.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.075 7f7ee37fe700 1 -- 172.21.2.101:0/3086268266 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed000c0c0 con 0x7f7ee41b3e40 2021-02-10T11:13:26.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.075 7f7ee37fe700 1 -- 172.21.2.101:0/3086268266 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ed000c520 con 0x7f7ee41b3e40 2021-02-10T11:13:26.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.075 7f7eeaef7700 1 -- 172.21.2.101:0/3086268266 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ee41b3e40 msgr2=0x7f7ee407d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:26.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.075 7f7eeaef7700 1 --2- 172.21.2.101:0/3086268266 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ee41b3e40 0x7f7ee407d690 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f7ed0006650 tx=0x7f7ed0009a80).stop 2021-02-10T11:13:26.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.076 7f7eeaef7700 1 -- 172.21.2.101:0/3086268266 shutdown_connections 2021-02-10T11:13:26.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.076 7f7eeaef7700 1 --2- 172.21.2.101:0/3086268266 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ee41b3e40 0x7f7ee407d690 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:26.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.076 7f7eeaef7700 1 --2- 172.21.2.101:0/3086268266 >> v2:172.21.2.108:3300/0 conn(0x7f7ee40e5070 0x7f7ee41b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:26.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.076 7f7eeaef7700 1 -- 172.21.2.101:0/3086268266 >> 172.21.2.101:0/3086268266 conn(0x7f7ee40ce690 msgr2=0x7f7ee40e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:26.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.076 7f7eeaef7700 1 -- 172.21.2.101:0/3086268266 shutdown_connections 2021-02-10T11:13:26.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.076 7f7eeaef7700 1 -- 172.21.2.101:0/3086268266 wait complete. 2021-02-10T11:13:26.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.090 7f7eeaef7700 1 Processor -- start 2021-02-10T11:13:26.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.090 7f7eeaef7700 1 -- start start 2021-02-10T11:13:26.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.091 7f7eeaef7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ee40b6570 0x7f7ee40a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:26.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.091 7f7eeaef7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ee409edf0 0x7f7ee409f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:26.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.091 7f7eeaef7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ee41a6d50 0x7f7ee41a7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:26.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.091 7f7eeaef7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee409f810 con 0x7f7ee409edf0 2021-02-10T11:13:26.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.091 7f7eeaef7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee40a6570 con 0x7f7ee41a6d50 2021-02-10T11:13:26.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.091 7f7eeaef7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee41a7770 con 0x7f7ee40b6570 2021-02-10T11:13:26.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.091 7f7ee9514700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ee41a6d50 0x7f7ee41a7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:26.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.091 7f7ee3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ee409edf0 0x7f7ee409f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:26.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.091 7f7ee3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ee409edf0 0x7f7ee409f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44184/0 (socket says 172.21.2.101:44184) 2021-02-10T11:13:26.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.091 7f7ee3fff700 1 -- 172.21.2.101:0/2576074073 learned_addr learned my addr 172.21.2.101:0/2576074073 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:26.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.091 7f7ee8d13700 1 --2- 172.21.2.101:0/2576074073 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ee40b6570 0x7f7ee40a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:26.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.092 7f7ee8d13700 1 -- 172.21.2.101:0/2576074073 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ee41a6d50 msgr2=0x7f7ee41a7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:26.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.092 7f7ee8d13700 1 --2- 172.21.2.101:0/2576074073 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ee41a6d50 0x7f7ee41a7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:26.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.092 7f7ee8d13700 1 -- 172.21.2.101:0/2576074073 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ee409edf0 msgr2=0x7f7ee409f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:26.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.092 7f7ee8d13700 1 --2- 172.21.2.101:0/2576074073 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ee409edf0 0x7f7ee409f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:26.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.092 7f7ee8d13700 1 -- 172.21.2.101:0/2576074073 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee41a7950 con 0x7f7ee40b6570 2021-02-10T11:13:26.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.092 7f7ee9514700 1 --2- 172.21.2.101:0/2576074073 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ee41a6d50 0x7f7ee41a7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:13:26.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.092 7f7ee8d13700 1 --2- 172.21.2.101:0/2576074073 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ee40b6570 0x7f7ee40a6af0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f7ed4009540 tx=0x7f7ed400b8e0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:26.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.092 7f7ee17fa700 1 -- 172.21.2.101:0/2576074073 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ed400bcd0 con 0x7f7ee40b6570 2021-02-10T11:13:26.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.092 7f7ee17fa700 1 -- 172.21.2.101:0/2576074073 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed40063e0 con 0x7f7ee40b6570 2021-02-10T11:13:26.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.092 7f7ee17fa700 1 -- 172.21.2.101:0/2576074073 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ed40022b0 con 0x7f7ee40b6570 2021-02-10T11:13:26.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.092 7f7eeaef7700 1 -- 172.21.2.101:0/2576074073 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee41a7950 con 0x7f7ee40b6570 2021-02-10T11:13:26.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.092 7f7eeaef7700 1 -- 172.21.2.101:0/2576074073 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee41a7950 con 0x7f7ee40b6570 2021-02-10T11:13:26.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.093 7f7ee17fa700 1 -- 172.21.2.101:0/2576074073 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7ed4009fe0 con 0x7f7ee40b6570 2021-02-10T11:13:26.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.093 7f7ee17fa700 1 --2- 172.21.2.101:0/2576074073 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ec00186c0 0x7f7ec001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:26.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.093 7f7ee17fa700 1 -- 172.21.2.101:0/2576074073 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7ed4032350 con 0x7f7ee40b6570 2021-02-10T11:13:26.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.093 7f7ee3fff700 1 --2- 172.21.2.101:0/2576074073 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ec00186c0 0x7f7ec001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:26.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.093 7f7ee3fff700 1 --2- 172.21.2.101:0/2576074073 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ec00186c0 0x7f7ec001abc0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:26.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.123 7f7eeaef7700 1 --2- 172.21.2.101:0/2576074073 >> [v2:172.21.2.108:6800/36381,v1:172.21.2.108:6801/36381] conn(0x7f7ee40c45a0 0x7f7ee40a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:26.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.123 7f7eeaef7700 1 -- 172.21.2.101:0/2576074073 --> [v2:172.21.2.108:6800/36381,v1:172.21.2.108:6801/36381] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7ee41a7770 con 0x7f7ee40c45a0 2021-02-10T11:13:26.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.124 7f7ee9514700 1 --2- 172.21.2.101:0/2576074073 >> [v2:172.21.2.108:6800/36381,v1:172.21.2.108:6801/36381] conn(0x7f7ee40c45a0 0x7f7ee40a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:26.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.124 7f7ee9514700 1 --2- 172.21.2.101:0/2576074073 >> [v2:172.21.2.108:6800/36381,v1:172.21.2.108:6801/36381] conn(0x7f7ee40c45a0 0x7f7ee40a3930 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:26.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.125 7f7ee17fa700 1 -- 172.21.2.101:0/2576074073 <== osd.6 v2:172.21.2.108:6800/36381 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (crc 0 0 0) 0x7f7edc000ff0 con 0x7f7ee40c45a0 2021-02-10T11:13:26.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.132 7f7eeaef7700 1 -- 172.21.2.101:0/2576074073 --> [v2:172.21.2.108:6800/36381,v1:172.21.2.108:6801/36381] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7ee407ed80 con 0x7f7ee40c45a0 2021-02-10T11:13:26.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.132 7f7ee17fa700 1 -- 172.21.2.101:0/2576074073 <== osd.6 v2:172.21.2.108:6800/36381 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7edc000ff0 con 0x7f7ee40c45a0 2021-02-10T11:13:26.137 INFO:teuthology.orchestra.run.gibba001.stdout:55834574856 2021-02-10T11:13:26.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.133 7f7eeaef7700 1 -- 172.21.2.101:0/2576074073 >> [v2:172.21.2.108:6800/36381,v1:172.21.2.108:6801/36381] conn(0x7f7ee40c45a0 msgr2=0x7f7ee40a3930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:26.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.133 7f7eeaef7700 1 --2- 172.21.2.101:0/2576074073 >> [v2:172.21.2.108:6800/36381,v1:172.21.2.108:6801/36381] conn(0x7f7ee40c45a0 0x7f7ee40a3930 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:26.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.133 7f7eeaef7700 1 -- 172.21.2.101:0/2576074073 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ec00186c0 msgr2=0x7f7ec001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:26.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.133 7f7eeaef7700 1 --2- 172.21.2.101:0/2576074073 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ec00186c0 0x7f7ec001abc0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:26.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.133 7f7eeaef7700 1 -- 172.21.2.101:0/2576074073 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ee40b6570 msgr2=0x7f7ee40a6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:26.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.133 7f7eeaef7700 1 --2- 172.21.2.101:0/2576074073 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ee40b6570 0x7f7ee40a6af0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f7ed4009540 tx=0x7f7ed400b8e0).stop 2021-02-10T11:13:26.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.133 7f7ee9514700 1 -- 172.21.2.101:0/2576074073 reap_dead start 2021-02-10T11:13:26.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.133 7f7eeaef7700 1 -- 172.21.2.101:0/2576074073 shutdown_connections 2021-02-10T11:13:26.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.133 7f7eeaef7700 1 -- 172.21.2.101:0/2576074073 >> 172.21.2.101:0/2576074073 conn(0x7f7ee409be90 msgr2=0x7f7ee409cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:26.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.134 7f7eeaef7700 1 -- 172.21.2.101:0/2576074073 shutdown_connections 2021-02-10T11:13:26.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.134 7f7eeaef7700 1 -- 172.21.2.101:0/2576074073 wait complete. 2021-02-10T11:13:26.145 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell osd.7 flush_pg_stats 2021-02-10T11:13:26.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.550 7fd707cbc700 1 Processor -- start 2021-02-10T11:13:26.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.550 7fd707cbc700 1 -- start start 2021-02-10T11:13:26.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.550 7fd707cbc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd70007e6a0 0x7fd7000d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:26.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.550 7fd707cbc700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd70007ed80 con 0x7fd7000e5470 2021-02-10T11:13:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.550 7fd707cbc700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd70007eb10 con 0x7fd7000e5070 2021-02-10T11:13:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.550 7fd707cbc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7001b4080 con 0x7fd70007e6a0 2021-02-10T11:13:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.550 7fd7052d7700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd7000e5470 0x7fd70007f300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44688/0 (socket says 172.21.2.101:44688) 2021-02-10T11:13:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.551 7fd7052d7700 1 -- 172.21.2.101:0/567177935 learned_addr learned my addr 172.21.2.101:0/567177935 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.551 7fd704ad6700 1 -- 172.21.2.101:0/567177935 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070357320 0 0) 0x7fd6fc0010b0 con 0x7fd7000e5470 2021-02-10T11:13:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.551 7fd7062d9700 1 --2- 172.21.2.101:0/567177935 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd70007e6a0 0x7fd7000d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.551 7fd704ad6700 1 -- 172.21.2.101:0/567177935 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6ec002fe0 con 0x7fd7000e5470 2021-02-10T11:13:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.551 7fd704ad6700 1 -- 172.21.2.101:0/567177935 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 120547018 0 0) 0x7fd6fc0010b0 con 0x7fd7000e5470 2021-02-10T11:13:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.551 7fd704ad6700 1 -- 172.21.2.101:0/567177935 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd70007e6a0 msgr2=0x7fd7000d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.551 7fd704ad6700 1 --2- 172.21.2.101:0/567177935 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd70007e6a0 0x7fd7000d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.551 7fd704ad6700 1 -- 172.21.2.101:0/567177935 >> v1:172.21.2.108:6789/0 conn(0x7fd7000e5070 legacy=0x7fd7001b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.551 7fd704ad6700 1 -- 172.21.2.101:0/567177935 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7000d1db0 con 0x7fd7000e5470 2021-02-10T11:13:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.551 7fd704ad6700 1 -- 172.21.2.101:0/567177935 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fd6fc0015b0 con 0x7fd7000e5470 2021-02-10T11:13:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.551 7fd704ad6700 1 -- 172.21.2.101:0/567177935 >> v1:172.21.2.101:6789/0 conn(0x7fd7000e5470 legacy=0x7fd70007f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.551 7fd704ad6700 1 --2- 172.21.2.101:0/567177935 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6ec003a70 0x7fd6ec005f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.551 7fd704ad6700 1 --2- 172.21.2.101:0/567177935 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6ec006810 0x7fd6ec008d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.551 7fd704ad6700 1 --2- 172.21.2.101:0/567177935 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0095a0 0x7fd6ec00bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.551 7fd704ad6700 1 -- 172.21.2.101:0/567177935 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec00c2f0 con 0x7fd6ec0095a0 2021-02-10T11:13:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.551 7fd704ad6700 1 -- 172.21.2.101:0/567177935 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec00c510 con 0x7fd6ec003a70 2021-02-10T11:13:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.551 7fd704ad6700 1 -- 172.21.2.101:0/567177935 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec00c730 con 0x7fd6ec006810 2021-02-10T11:13:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.551 7fd7062d9700 1 --2- 172.21.2.101:0/567177935 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0095a0 0x7fd6ec00bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.551 7fd705ad8700 1 --2- 172.21.2.101:0/567177935 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6ec003a70 0x7fd6ec005f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:26.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.551 7fd7052d7700 1 --2- 172.21.2.101:0/567177935 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6ec006810 0x7fd6ec008d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:26.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.552 7fd7062d9700 1 -- 172.21.2.101:0/567177935 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6ec006810 msgr2=0x7fd6ec008d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:26.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.552 7fd7062d9700 1 --2- 172.21.2.101:0/567177935 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6ec006810 0x7fd6ec008d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:26.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.552 7fd7062d9700 1 -- 172.21.2.101:0/567177935 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6ec003a70 msgr2=0x7fd6ec005f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:26.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.552 7fd7062d9700 1 --2- 172.21.2.101:0/567177935 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6ec003a70 0x7fd6ec005f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:26.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.552 7fd7062d9700 1 -- 172.21.2.101:0/567177935 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec0509a0 con 0x7fd6ec0095a0 2021-02-10T11:13:26.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.552 7fd7062d9700 1 --2- 172.21.2.101:0/567177935 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0095a0 0x7fd6ec00bac0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fd6f8009a40 tx=0x7fd6f8009de0).ready entity=mon.0 client_cookie=c967b312489fdba server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:26.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.552 7fd704ad6700 1 -- 172.21.2.101:0/567177935 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6f800b300 con 0x7fd6ec0095a0 2021-02-10T11:13:26.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.552 7fd704ad6700 1 -- 172.21.2.101:0/567177935 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f8003030 con 0x7fd6ec0095a0 2021-02-10T11:13:26.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.552 7fd704ad6700 1 -- 172.21.2.101:0/567177935 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6f800b6f0 con 0x7fd6ec0095a0 2021-02-10T11:13:26.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.552 7fd707cbc700 1 -- 172.21.2.101:0/567177935 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0095a0 msgr2=0x7fd6ec00bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:26.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.552 7fd707cbc700 1 --2- 172.21.2.101:0/567177935 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0095a0 0x7fd6ec00bac0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fd6f8009a40 tx=0x7fd6f8009de0).stop 2021-02-10T11:13:26.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.552 7fd7062d9700 1 -- 172.21.2.101:0/567177935 reap_dead start 2021-02-10T11:13:26.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.552 7fd707cbc700 1 -- 172.21.2.101:0/567177935 shutdown_connections 2021-02-10T11:13:26.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.552 7fd707cbc700 1 -- 172.21.2.101:0/567177935 >> 172.21.2.101:0/567177935 conn(0x7fd7000ce690 msgr2=0x7fd7000e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:26.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.553 7fd707cbc700 1 -- 172.21.2.101:0/567177935 shutdown_connections 2021-02-10T11:13:26.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.553 7fd707cbc700 1 -- 172.21.2.101:0/567177935 wait complete. 2021-02-10T11:13:26.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.567 7fd707cbc700 1 Processor -- start 2021-02-10T11:13:26.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.567 7fd707cbc700 1 -- start start 2021-02-10T11:13:26.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.568 7fd707cbc700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd70008f500 0x7fd70019f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:26.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.568 7fd707cbc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd70008fd90 0x7fd700090190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:26.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.568 7fd707cbc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7000907b0 0x7fd700094370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:26.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.568 7fd707cbc700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70008f900 con 0x7fd70008fd90 2021-02-10T11:13:26.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.568 7fd707cbc700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70019ff20 con 0x7fd70008f500 2021-02-10T11:13:26.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.568 7fd707cbc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7001a0140 con 0x7fd7000907b0 2021-02-10T11:13:26.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.568 7fd7052d7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd70008fd90 0x7fd700090190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:26.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.568 7fd7052d7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd70008fd90 0x7fd700090190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44206/0 (socket says 172.21.2.101:44206) 2021-02-10T11:13:26.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.568 7fd7052d7700 1 -- 172.21.2.101:0/2345798534 learned_addr learned my addr 172.21.2.101:0/2345798534 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:26.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.568 7fd7052d7700 1 -- 172.21.2.101:0/2345798534 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7000907b0 msgr2=0x7fd700094370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:26.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.568 7fd7052d7700 1 --2- 172.21.2.101:0/2345798534 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7000907b0 0x7fd700094370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:26.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.568 7fd7052d7700 1 -- 172.21.2.101:0/2345798534 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd70008f500 msgr2=0x7fd70019f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:26.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.568 7fd7052d7700 1 --2- 172.21.2.101:0/2345798534 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd70008f500 0x7fd70019f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:26.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.568 7fd7052d7700 1 -- 172.21.2.101:0/2345798534 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd700094990 con 0x7fd70008fd90 2021-02-10T11:13:26.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.569 7fd7052d7700 1 --2- 172.21.2.101:0/2345798534 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd70008fd90 0x7fd700090190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fd6fc00a590 tx=0x7fd6fc00a930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:26.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.569 7fd6f27fc700 1 -- 172.21.2.101:0/2345798534 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6fc0010b0 con 0x7fd70008fd90 2021-02-10T11:13:26.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.569 7fd6f27fc700 1 -- 172.21.2.101:0/2345798534 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6fc004350 con 0x7fd70008fd90 2021-02-10T11:13:26.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.569 7fd707cbc700 1 -- 172.21.2.101:0/2345798534 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd700094990 con 0x7fd70008fd90 2021-02-10T11:13:26.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.569 7fd6f27fc700 1 -- 172.21.2.101:0/2345798534 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6fc004870 con 0x7fd70008fd90 2021-02-10T11:13:26.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.569 7fd707cbc700 1 -- 172.21.2.101:0/2345798534 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd700094990 con 0x7fd70008fd90 2021-02-10T11:13:26.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.569 7fd6f27fc700 1 -- 172.21.2.101:0/2345798534 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd6fc001090 con 0x7fd70008fd90 2021-02-10T11:13:26.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.570 7fd6f27fc700 1 --2- 172.21.2.101:0/2345798534 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6e80186d0 0x7fd6e801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:26.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.570 7fd6f27fc700 1 -- 172.21.2.101:0/2345798534 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd6fc032780 con 0x7fd70008fd90 2021-02-10T11:13:26.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.570 7fd705ad8700 1 --2- 172.21.2.101:0/2345798534 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6e80186d0 0x7fd6e801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:26.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.570 7fd705ad8700 1 --2- 172.21.2.101:0/2345798534 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6e80186d0 0x7fd6e801abd0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:26.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.600 7fd707cbc700 1 --2- 172.21.2.101:0/2345798534 >> [v2:172.21.2.108:6817/36382,v1:172.21.2.108:6819/36382] conn(0x7fd7000c45a0 0x7fd7000627c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:26.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.600 7fd707cbc700 1 -- 172.21.2.101:0/2345798534 --> [v2:172.21.2.108:6817/36382,v1:172.21.2.108:6819/36382] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd70019ff20 con 0x7fd7000c45a0 2021-02-10T11:13:26.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.600 7fd7062d9700 1 --2- 172.21.2.101:0/2345798534 >> [v2:172.21.2.108:6817/36382,v1:172.21.2.108:6819/36382] conn(0x7fd7000c45a0 0x7fd7000627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:26.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.601 7fd7062d9700 1 --2- 172.21.2.101:0/2345798534 >> [v2:172.21.2.108:6817/36382,v1:172.21.2.108:6819/36382] conn(0x7fd7000c45a0 0x7fd7000627c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:26.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.602 7fd6f27fc700 1 -- 172.21.2.101:0/2345798534 <== osd.7 v2:172.21.2.108:6817/36382 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (crc 0 0 0) 0x7fd6f80066f0 con 0x7fd7000c45a0 2021-02-10T11:13:26.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.608 7fd707cbc700 1 -- 172.21.2.101:0/2345798534 --> [v2:172.21.2.108:6817/36382,v1:172.21.2.108:6819/36382] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd700084a90 con 0x7fd7000c45a0 2021-02-10T11:13:26.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.609 7fd6f27fc700 1 -- 172.21.2.101:0/2345798534 <== osd.7 v2:172.21.2.108:6817/36382 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd6f8009e30 con 0x7fd7000c45a0 2021-02-10T11:13:26.613 INFO:teuthology.orchestra.run.gibba001.stdout:55834574856 2021-02-10T11:13:26.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.610 7fd707cbc700 1 -- 172.21.2.101:0/2345798534 >> [v2:172.21.2.108:6817/36382,v1:172.21.2.108:6819/36382] conn(0x7fd7000c45a0 msgr2=0x7fd7000627c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:26.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.610 7fd707cbc700 1 --2- 172.21.2.101:0/2345798534 >> [v2:172.21.2.108:6817/36382,v1:172.21.2.108:6819/36382] conn(0x7fd7000c45a0 0x7fd7000627c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:26.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.610 7fd707cbc700 1 -- 172.21.2.101:0/2345798534 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6e80186d0 msgr2=0x7fd6e801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:26.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.610 7fd707cbc700 1 --2- 172.21.2.101:0/2345798534 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6e80186d0 0x7fd6e801abd0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:26.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.610 7fd707cbc700 1 -- 172.21.2.101:0/2345798534 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd70008fd90 msgr2=0x7fd700090190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:26.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.610 7fd707cbc700 1 --2- 172.21.2.101:0/2345798534 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd70008fd90 0x7fd700090190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fd6fc00a590 tx=0x7fd6fc00a930).stop 2021-02-10T11:13:26.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.610 7fd7062d9700 1 -- 172.21.2.101:0/2345798534 reap_dead start 2021-02-10T11:13:26.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.610 7fd707cbc700 1 -- 172.21.2.101:0/2345798534 shutdown_connections 2021-02-10T11:13:26.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.610 7fd707cbc700 1 -- 172.21.2.101:0/2345798534 >> 172.21.2.101:0/2345798534 conn(0x7fd7000b5560 msgr2=0x7fd7000a6200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:26.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.610 7fd707cbc700 1 -- 172.21.2.101:0/2345798534 shutdown_connections 2021-02-10T11:13:26.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:26.610 7fd707cbc700 1 -- 172.21.2.101:0/2345798534 wait complete. 2021-02-10T11:13:26.622 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd last-stat-seq osd.0 2021-02-10T11:13:27.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.025 7fd7c341d700 1 Processor -- start 2021-02-10T11:13:27.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.025 7fd7c341d700 1 -- start start 2021-02-10T11:13:27.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.025 7fd7c341d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd7bc0e5070 0x7fd7bc1b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:27.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.025 7fd7c341d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd7bc1b3e40 0x7fd7bc07d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:27.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.025 7fd7c341d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7bc07ed80 con 0x7fd7bc0e5470 2021-02-10T11:13:27.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.025 7fd7c341d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7bc080310 con 0x7fd7bc0e5070 2021-02-10T11:13:27.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.025 7fd7c341d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7bc080530 con 0x7fd7bc1b3e40 2021-02-10T11:13:27.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.026 7fd7c1239700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd7bc0e5070 0x7fd7bc1b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:27.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.026 7fd7c1239700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd7bc0e5070 0x7fd7bc1b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44214/0 (socket says 172.21.2.101:44214) 2021-02-10T11:13:27.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.026 7fd7c1239700 1 -- 172.21.2.101:0/1854789295 learned_addr learned my addr 172.21.2.101:0/1854789295 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:27.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.026 7fd7c0a38700 1 --2- 172.21.2.101:0/1854789295 >> v2:172.21.2.108:3300/0 conn(0x7fd7bc1b3e40 0x7fd7bc07d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:27.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.026 7fd7c1239700 1 -- 172.21.2.101:0/1854789295 >> v2:172.21.2.108:3300/0 conn(0x7fd7bc1b3e40 msgr2=0x7fd7bc07d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:27.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.026 7fd7c1239700 1 --2- 172.21.2.101:0/1854789295 >> v2:172.21.2.108:3300/0 conn(0x7fd7bc1b3e40 0x7fd7bc07d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:27.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.026 7fd7c1239700 1 -- 172.21.2.101:0/1854789295 >> v1:172.21.2.108:6789/0 conn(0x7fd7bc0e5470 legacy=0x7fd7bc07dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:27.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.026 7fd7c1239700 1 -- 172.21.2.101:0/1854789295 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc080750 con 0x7fd7bc0e5070 2021-02-10T11:13:27.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.026 7fd7c1239700 1 --2- 172.21.2.101:0/1854789295 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7bc0e5070 0x7fd7bc1b3820 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fd7b0006700 tx=0x7fd7b0009b80).ready entity=mon.0 client_cookie=f0a0d7345ca1dba3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:27.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.026 7fd7affff700 1 -- 172.21.2.101:0/1854789295 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd7b000c0c0 con 0x7fd7bc0e5070 2021-02-10T11:13:27.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.026 7fd7affff700 1 -- 172.21.2.101:0/1854789295 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b000c400 con 0x7fd7bc0e5070 2021-02-10T11:13:27.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.026 7fd7affff700 1 -- 172.21.2.101:0/1854789295 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd7b000c860 con 0x7fd7bc0e5070 2021-02-10T11:13:27.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.027 7fd7c341d700 1 -- 172.21.2.101:0/1854789295 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7bc0e5070 msgr2=0x7fd7bc1b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:27.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.027 7fd7c341d700 1 --2- 172.21.2.101:0/1854789295 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7bc0e5070 0x7fd7bc1b3820 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fd7b0006700 tx=0x7fd7b0009b80).stop 2021-02-10T11:13:27.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.027 7fd7c341d700 1 -- 172.21.2.101:0/1854789295 shutdown_connections 2021-02-10T11:13:27.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.027 7fd7c341d700 1 --2- 172.21.2.101:0/1854789295 >> v2:172.21.2.108:3300/0 conn(0x7fd7bc1b3e40 0x7fd7bc07d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:27.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.027 7fd7c341d700 1 --2- 172.21.2.101:0/1854789295 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7bc0e5070 0x7fd7bc1b3820 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:27.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.027 7fd7c341d700 1 -- 172.21.2.101:0/1854789295 >> 172.21.2.101:0/1854789295 conn(0x7fd7bc0ce690 msgr2=0x7fd7bc0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:27.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.027 7fd7c341d700 1 -- 172.21.2.101:0/1854789295 shutdown_connections 2021-02-10T11:13:27.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.027 7fd7c341d700 1 -- 172.21.2.101:0/1854789295 wait complete. 2021-02-10T11:13:27.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.042 7fd7c341d700 1 Processor -- start 2021-02-10T11:13:27.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.042 7fd7c341d700 1 -- start start 2021-02-10T11:13:27.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.042 7fd7c341d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7bc090e20 0x7fd7bc094ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:27.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.042 7fd7c341d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7bc0849f0 0x7fd7bc095100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:27.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.042 7fd7c341d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7bc095720 0x7fd7bc07f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:27.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.042 7fd7c341d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc084fe0 con 0x7fd7bc090e20 2021-02-10T11:13:27.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.042 7fd7c341d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc091350 con 0x7fd7bc095720 2021-02-10T11:13:27.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.042 7fd7c341d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc07f840 con 0x7fd7bc0849f0 2021-02-10T11:13:27.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.042 7fd7c1239700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7bc090e20 0x7fd7bc094ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:27.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.042 7fd7c1239700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7bc090e20 0x7fd7bc094ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44218/0 (socket says 172.21.2.101:44218) 2021-02-10T11:13:27.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.042 7fd7c1239700 1 -- 172.21.2.101:0/636343021 learned_addr learned my addr 172.21.2.101:0/636343021 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:27.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.043 7fd7c1239700 1 -- 172.21.2.101:0/636343021 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7bc0849f0 msgr2=0x7fd7bc095100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:27.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.043 7fd7c1239700 1 --2- 172.21.2.101:0/636343021 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7bc0849f0 0x7fd7bc095100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:27.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.043 7fd7c1239700 1 -- 172.21.2.101:0/636343021 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7bc095720 msgr2=0x7fd7bc07f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:27.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.043 7fd7c1239700 1 --2- 172.21.2.101:0/636343021 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7bc095720 0x7fd7bc07f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:27.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.043 7fd7c1239700 1 -- 172.21.2.101:0/636343021 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc07fa60 con 0x7fd7bc090e20 2021-02-10T11:13:27.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.043 7fd7c1239700 1 --2- 172.21.2.101:0/636343021 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7bc090e20 0x7fd7bc094ae0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fd7b000b5e0 tx=0x7fd7b000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:27.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.043 7fd7adffb700 1 -- 172.21.2.101:0/636343021 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd7b000bd70 con 0x7fd7bc090e20 2021-02-10T11:13:27.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.043 7fd7adffb700 1 -- 172.21.2.101:0/636343021 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b0002030 con 0x7fd7bc090e20 2021-02-10T11:13:27.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.043 7fd7adffb700 1 -- 172.21.2.101:0/636343021 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd7b0002550 con 0x7fd7bc090e20 2021-02-10T11:13:27.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.043 7fd7c341d700 1 -- 172.21.2.101:0/636343021 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7bc07fa60 con 0x7fd7bc090e20 2021-02-10T11:13:27.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.043 7fd7c341d700 1 -- 172.21.2.101:0/636343021 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7bc07fff0 con 0x7fd7bc090e20 2021-02-10T11:13:27.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.044 7fd7adffb700 1 -- 172.21.2.101:0/636343021 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd7b0000c90 con 0x7fd7bc090e20 2021-02-10T11:13:27.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.044 7fd7adffb700 1 --2- 172.21.2.101:0/636343021 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd79c0186c0 0x7fd79c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:27.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.044 7fd7adffb700 1 -- 172.21.2.101:0/636343021 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd7b00325d0 con 0x7fd7bc090e20 2021-02-10T11:13:27.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.044 7fd7c0a38700 1 --2- 172.21.2.101:0/636343021 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd79c0186c0 0x7fd79c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:27.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.044 7fd7c0a38700 1 --2- 172.21.2.101:0/636343021 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd79c0186c0 0x7fd79c01abc0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:27.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.075 7fd7c341d700 1 -- 172.21.2.101:0/636343021 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7bc107180 con 0x7fd7bc090e20 2021-02-10T11:13:27.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.078 7fd7adffb700 1 -- 172.21.2.101:0/636343021 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd7b00009b0 con 0x7fd7bc090e20 2021-02-10T11:13:27.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.217 7fd7c341d700 1 -- 172.21.2.101:0/636343021 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd7bc107180 con 0x7fd7bc090e20 2021-02-10T11:13:27.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.217 7fd7adffb700 1 -- 172.21.2.101:0/636343021 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd7b0000c90 con 0x7fd7bc090e20 2021-02-10T11:13:27.222 INFO:teuthology.orchestra.run.gibba001.stdout:55834574855 2021-02-10T11:13:27.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.219 7fd7c341d700 1 -- 172.21.2.101:0/636343021 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd79c0186c0 msgr2=0x7fd79c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:27.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.219 7fd7c341d700 1 --2- 172.21.2.101:0/636343021 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd79c0186c0 0x7fd79c01abc0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:27.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.219 7fd7c341d700 1 -- 172.21.2.101:0/636343021 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7bc090e20 msgr2=0x7fd7bc094ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:27.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.219 7fd7c341d700 1 --2- 172.21.2.101:0/636343021 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7bc090e20 0x7fd7bc094ae0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fd7b000b5e0 tx=0x7fd7b000b980).stop 2021-02-10T11:13:27.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.219 7fd7c341d700 1 -- 172.21.2.101:0/636343021 shutdown_connections 2021-02-10T11:13:27.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.219 7fd7c341d700 1 --2- 172.21.2.101:0/636343021 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd79c0186c0 0x7fd79c01abc0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:27.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.219 7fd7c341d700 1 --2- 172.21.2.101:0/636343021 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7bc0849f0 0x7fd7bc095100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:27.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.219 7fd7c341d700 1 --2- 172.21.2.101:0/636343021 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7bc095720 0x7fd7bc07f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:27.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.219 7fd7c341d700 1 --2- 172.21.2.101:0/636343021 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7bc090e20 0x7fd7bc094ae0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:27.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.219 7fd7c341d700 1 -- 172.21.2.101:0/636343021 >> 172.21.2.101:0/636343021 conn(0x7fd7bc107d80 msgr2=0x7fd7bc1073b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:27.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.219 7fd7c341d700 1 -- 172.21.2.101:0/636343021 shutdown_connections 2021-02-10T11:13:27.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.220 7fd7c341d700 1 -- 172.21.2.101:0/636343021 wait complete. 2021-02-10T11:13:27.232 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.0 2021-02-10T11:13:27.232 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd last-stat-seq osd.1 2021-02-10T11:13:27.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.637 7fc38b797700 1 Processor -- start 2021-02-10T11:13:27.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.637 7fc38b797700 1 -- start start 2021-02-10T11:13:27.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.637 7fc38b797700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc3840e5070 0x7fc3841b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:27.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.637 7fc38b797700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc3841b3e40 0x7fc38407d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:27.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.637 7fc38b797700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc38407ed80 con 0x7fc3840e5470 2021-02-10T11:13:27.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.637 7fc38b797700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc384080310 con 0x7fc3841b3e40 2021-02-10T11:13:27.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.637 7fc38b797700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc384080530 con 0x7fc3840e5070 2021-02-10T11:13:27.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.637 7fc388db2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc3841b3e40 0x7fc38407d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:27.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.637 7fc388db2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc3841b3e40 0x7fc38407d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44226/0 (socket says 172.21.2.101:44226) 2021-02-10T11:13:27.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.637 7fc388db2700 1 -- 172.21.2.101:0/3973978084 learned_addr learned my addr 172.21.2.101:0/3973978084 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:27.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.638 7fc3895b3700 1 --2- 172.21.2.101:0/3973978084 >> v2:172.21.2.108:3300/0 conn(0x7fc3840e5070 0x7fc3841b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:27.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.638 7fc388db2700 1 -- 172.21.2.101:0/3973978084 >> v2:172.21.2.108:3300/0 conn(0x7fc3840e5070 msgr2=0x7fc3841b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:27.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.638 7fc388db2700 1 --2- 172.21.2.101:0/3973978084 >> v2:172.21.2.108:3300/0 conn(0x7fc3840e5070 0x7fc3841b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:27.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.638 7fc388db2700 1 -- 172.21.2.101:0/3973978084 >> v1:172.21.2.108:6789/0 conn(0x7fc3840e5470 legacy=0x7fc38407dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:27.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.638 7fc388db2700 1 -- 172.21.2.101:0/3973978084 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc384080750 con 0x7fc3841b3e40 2021-02-10T11:13:27.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.638 7fc3895b3700 1 --2- 172.21.2.101:0/3973978084 >> v2:172.21.2.108:3300/0 conn(0x7fc3840e5070 0x7fc3841b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:13:27.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.638 7fc388db2700 1 --2- 172.21.2.101:0/3973978084 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3841b3e40 0x7fc38407d690 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fc380006700 tx=0x7fc380009b80).ready entity=mon.0 client_cookie=742a96f35f9102f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:27.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.638 7fc377fff700 1 -- 172.21.2.101:0/3973978084 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc38000c0c0 con 0x7fc3841b3e40 2021-02-10T11:13:27.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.638 7fc377fff700 1 -- 172.21.2.101:0/3973978084 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38000c440 con 0x7fc3841b3e40 2021-02-10T11:13:27.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.638 7fc377fff700 1 -- 172.21.2.101:0/3973978084 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc38000c960 con 0x7fc3841b3e40 2021-02-10T11:13:27.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.638 7fc38b797700 1 -- 172.21.2.101:0/3973978084 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3841b3e40 msgr2=0x7fc38407d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:27.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.638 7fc38b797700 1 --2- 172.21.2.101:0/3973978084 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3841b3e40 0x7fc38407d690 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fc380006700 tx=0x7fc380009b80).stop 2021-02-10T11:13:27.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.639 7fc38b797700 1 -- 172.21.2.101:0/3973978084 shutdown_connections 2021-02-10T11:13:27.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.639 7fc38b797700 1 --2- 172.21.2.101:0/3973978084 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3841b3e40 0x7fc38407d690 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:27.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.639 7fc38b797700 1 --2- 172.21.2.101:0/3973978084 >> v2:172.21.2.108:3300/0 conn(0x7fc3840e5070 0x7fc3841b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:27.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.639 7fc38b797700 1 -- 172.21.2.101:0/3973978084 >> 172.21.2.101:0/3973978084 conn(0x7fc3840ce690 msgr2=0x7fc3840e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:27.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.639 7fc38b797700 1 -- 172.21.2.101:0/3973978084 shutdown_connections 2021-02-10T11:13:27.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.639 7fc38b797700 1 -- 172.21.2.101:0/3973978084 wait complete. 2021-02-10T11:13:27.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.654 7fc38b797700 1 Processor -- start 2021-02-10T11:13:27.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.654 7fc38b797700 1 -- start start 2021-02-10T11:13:27.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.654 7fc38b797700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc384198ea0 0x7fc3841992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:27.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.654 7fc38b797700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3840c0720 0x7fc3840c0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:27.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.654 7fc38b797700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc38407f220 0x7fc38407f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:27.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.654 7fc38b797700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3840bb1b0 con 0x7fc38407f220 2021-02-10T11:13:27.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.654 7fc38b797700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3840c1140 con 0x7fc3840c0720 2021-02-10T11:13:27.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.654 7fc38b797700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38407fc40 con 0x7fc384198ea0 2021-02-10T11:13:27.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.655 7fc389db4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc38407f220 0x7fc38407f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:27.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.655 7fc389db4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc38407f220 0x7fc38407f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44236/0 (socket says 172.21.2.101:44236) 2021-02-10T11:13:27.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.655 7fc389db4700 1 -- 172.21.2.101:0/3295097905 learned_addr learned my addr 172.21.2.101:0/3295097905 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:27.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.655 7fc389db4700 1 -- 172.21.2.101:0/3295097905 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc384198ea0 msgr2=0x7fc3841992a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:27.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.655 7fc389db4700 1 --2- 172.21.2.101:0/3295097905 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc384198ea0 0x7fc3841992a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:27.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.655 7fc389db4700 1 -- 172.21.2.101:0/3295097905 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3840c0720 msgr2=0x7fc3840c0b20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:27.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.655 7fc389db4700 1 --2- 172.21.2.101:0/3295097905 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3840c0720 0x7fc3840c0b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:27.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.655 7fc389db4700 1 -- 172.21.2.101:0/3295097905 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38407fe60 con 0x7fc38407f220 2021-02-10T11:13:27.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.655 7fc389db4700 1 --2- 172.21.2.101:0/3295097905 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc38407f220 0x7fc38407f620 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fc37c009940 tx=0x7fc37c00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:27.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.655 7fc375ffb700 1 -- 172.21.2.101:0/3295097905 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc37c000f90 con 0x7fc38407f220 2021-02-10T11:13:27.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.655 7fc375ffb700 1 -- 172.21.2.101:0/3295097905 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc37c00c2f0 con 0x7fc38407f220 2021-02-10T11:13:27.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.655 7fc38b797700 1 -- 172.21.2.101:0/3295097905 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc38407fe60 con 0x7fc38407f220 2021-02-10T11:13:27.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.655 7fc38b797700 1 -- 172.21.2.101:0/3295097905 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3840ba280 con 0x7fc38407f220 2021-02-10T11:13:27.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.655 7fc375ffb700 1 -- 172.21.2.101:0/3295097905 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc37c00c810 con 0x7fc38407f220 2021-02-10T11:13:27.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.656 7fc375ffb700 1 -- 172.21.2.101:0/3295097905 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc37c001220 con 0x7fc38407f220 2021-02-10T11:13:27.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.656 7fc375ffb700 1 --2- 172.21.2.101:0/3295097905 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3640186d0 0x7fc36401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:27.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.656 7fc375ffb700 1 -- 172.21.2.101:0/3295097905 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc37c0329a0 con 0x7fc38407f220 2021-02-10T11:13:27.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.656 7fc3895b3700 1 --2- 172.21.2.101:0/3295097905 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3640186d0 0x7fc36401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:27.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.656 7fc3895b3700 1 --2- 172.21.2.101:0/3295097905 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3640186d0 0x7fc36401abd0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:27.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.687 7fc38b797700 1 -- 172.21.2.101:0/3295097905 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc384080100 con 0x7fc38407f220 2021-02-10T11:13:27.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.690 7fc375ffb700 1 -- 172.21.2.101:0/3295097905 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc37c000f40 con 0x7fc38407f220 2021-02-10T11:13:27.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.831 7fc38b797700 1 -- 172.21.2.101:0/3295097905 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc384107150 con 0x7fc38407f220 2021-02-10T11:13:27.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.831 7fc375ffb700 1 -- 172.21.2.101:0/3295097905 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc37c01f860 con 0x7fc38407f220 2021-02-10T11:13:27.835 INFO:teuthology.orchestra.run.gibba001.stdout:55834574855 2021-02-10T11:13:27.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.833 7fc38b797700 1 -- 172.21.2.101:0/3295097905 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3640186d0 msgr2=0x7fc36401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:27.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.833 7fc38b797700 1 --2- 172.21.2.101:0/3295097905 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3640186d0 0x7fc36401abd0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:27.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.833 7fc38b797700 1 -- 172.21.2.101:0/3295097905 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc38407f220 msgr2=0x7fc38407f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:27.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.833 7fc38b797700 1 --2- 172.21.2.101:0/3295097905 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc38407f220 0x7fc38407f620 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fc37c009940 tx=0x7fc37c00bcf0).stop 2021-02-10T11:13:27.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.833 7fc38b797700 1 -- 172.21.2.101:0/3295097905 shutdown_connections 2021-02-10T11:13:27.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.833 7fc38b797700 1 --2- 172.21.2.101:0/3295097905 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc38407f220 0x7fc38407f620 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:27.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.833 7fc38b797700 1 --2- 172.21.2.101:0/3295097905 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3840c0720 0x7fc3840c0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:27.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.833 7fc38b797700 1 --2- 172.21.2.101:0/3295097905 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3640186d0 0x7fc36401abd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:27.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.833 7fc38b797700 1 --2- 172.21.2.101:0/3295097905 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc384198ea0 0x7fc3841992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:27.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.833 7fc38b797700 1 -- 172.21.2.101:0/3295097905 >> 172.21.2.101:0/3295097905 conn(0x7fc384107d50 msgr2=0x7fc3840bd220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:27.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.833 7fc38b797700 1 -- 172.21.2.101:0/3295097905 shutdown_connections 2021-02-10T11:13:27.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:27.833 7fc38b797700 1 -- 172.21.2.101:0/3295097905 wait complete. 2021-02-10T11:13:27.845 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.1 2021-02-10T11:13:27.845 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd last-stat-seq osd.2 2021-02-10T11:13:28.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.251 7fa410091700 1 Processor -- start 2021-02-10T11:13:28.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.251 7fa410091700 1 -- start start 2021-02-10T11:13:28.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.251 7fa410091700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa40807e6a0 0x7fa4080d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:28.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.251 7fa410091700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa40807ed80 con 0x7fa4080e5470 2021-02-10T11:13:28.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.251 7fa410091700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa40807eaa0 con 0x7fa4080e5070 2021-02-10T11:13:28.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.251 7fa410091700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4081b4080 con 0x7fa40807e6a0 2021-02-10T11:13:28.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.251 7fa40d6ac700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa4080e5470 0x7fa40807f300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44734/0 (socket says 172.21.2.101:44734) 2021-02-10T11:13:28.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.251 7fa40d6ac700 1 -- 172.21.2.101:0/248243546 learned_addr learned my addr 172.21.2.101:0/248243546 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:28.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.252 7fa40ceab700 1 -- 172.21.2.101:0/248243546 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 994539316 0 0) 0x7fa4040010b0 con 0x7fa4080e5470 2021-02-10T11:13:28.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.252 7fa40ceab700 1 -- 172.21.2.101:0/248243546 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3f4002fe0 con 0x7fa4080e5470 2021-02-10T11:13:28.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.252 7fa40e6ae700 1 --2- 172.21.2.101:0/248243546 >> v2:172.21.2.108:3300/0 conn(0x7fa40807e6a0 0x7fa4080d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:28.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.252 7fa40ceab700 1 -- 172.21.2.101:0/248243546 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 829057015 0 0) 0x7fa4040010b0 con 0x7fa4080e5470 2021-02-10T11:13:28.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.252 7fa40ceab700 1 -- 172.21.2.101:0/248243546 >> v2:172.21.2.108:3300/0 conn(0x7fa40807e6a0 msgr2=0x7fa4080d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:28.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.252 7fa40ceab700 1 --2- 172.21.2.101:0/248243546 >> v2:172.21.2.108:3300/0 conn(0x7fa40807e6a0 0x7fa4080d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:28.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.252 7fa40ceab700 1 -- 172.21.2.101:0/248243546 >> v1:172.21.2.108:6789/0 conn(0x7fa4080e5070 legacy=0x7fa4081b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:28.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.252 7fa40ceab700 1 -- 172.21.2.101:0/248243546 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4080d1db0 con 0x7fa4080e5470 2021-02-10T11:13:28.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.252 7fa40ceab700 1 -- 172.21.2.101:0/248243546 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fa4040015b0 con 0x7fa4080e5470 2021-02-10T11:13:28.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.252 7fa40ceab700 1 -- 172.21.2.101:0/248243546 >> v1:172.21.2.101:6789/0 conn(0x7fa4080e5470 legacy=0x7fa40807f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:28.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.252 7fa40ceab700 1 --2- 172.21.2.101:0/248243546 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3f4003a60 0x7fa3f4005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:28.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.252 7fa40ceab700 1 --2- 172.21.2.101:0/248243546 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa3f4006780 0x7fa3f4008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:28.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.252 7fa40ceab700 1 --2- 172.21.2.101:0/248243546 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3f4009580 0x7fa3f400baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:28.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.252 7fa40ceab700 1 -- 172.21.2.101:0/248243546 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f400c2f0 con 0x7fa3f4006780 2021-02-10T11:13:28.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.252 7fa40ceab700 1 -- 172.21.2.101:0/248243546 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f400c510 con 0x7fa3f4009580 2021-02-10T11:13:28.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.252 7fa40ceab700 1 -- 172.21.2.101:0/248243546 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f400c730 con 0x7fa3f4003a60 2021-02-10T11:13:28.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.252 7fa40d6ac700 1 --2- 172.21.2.101:0/248243546 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa3f4006780 0x7fa3f4008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:28.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.253 7fa40e6ae700 1 --2- 172.21.2.101:0/248243546 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3f4009580 0x7fa3f400baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:28.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.253 7fa40dead700 1 --2- 172.21.2.101:0/248243546 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3f4003a60 0x7fa3f4005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:28.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.253 7fa40d6ac700 1 -- 172.21.2.101:0/248243546 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3f4003a60 msgr2=0x7fa3f4005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:28.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.253 7fa40d6ac700 1 --2- 172.21.2.101:0/248243546 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3f4003a60 0x7fa3f4005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:28.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.253 7fa40d6ac700 1 -- 172.21.2.101:0/248243546 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3f4009580 msgr2=0x7fa3f400baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:28.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.253 7fa40d6ac700 1 --2- 172.21.2.101:0/248243546 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3f4009580 0x7fa3f400baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:28.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.253 7fa40d6ac700 1 -- 172.21.2.101:0/248243546 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f40509a0 con 0x7fa3f4006780 2021-02-10T11:13:28.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.253 7fa40dead700 1 --2- 172.21.2.101:0/248243546 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3f4003a60 0x7fa3f4005f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:13:28.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.253 7fa40e6ae700 1 -- 172.21.2.101:0/248243546 reap_dead start 2021-02-10T11:13:28.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.253 7fa40d6ac700 1 --2- 172.21.2.101:0/248243546 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa3f4006780 0x7fa3f4008c80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fa4040017d0 tx=0x7fa404007d30).ready entity=mon.0 client_cookie=1cafee6351d83257 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:28.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.253 7fa40ceab700 1 -- 172.21.2.101:0/248243546 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa404001010 con 0x7fa3f4006780 2021-02-10T11:13:28.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.253 7fa40ceab700 1 -- 172.21.2.101:0/248243546 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40400a310 con 0x7fa3f4006780 2021-02-10T11:13:28.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.253 7fa40ceab700 1 -- 172.21.2.101:0/248243546 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa40400a780 con 0x7fa3f4006780 2021-02-10T11:13:28.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.253 7fa410091700 1 -- 172.21.2.101:0/248243546 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa3f4006780 msgr2=0x7fa3f4008c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:28.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.253 7fa410091700 1 --2- 172.21.2.101:0/248243546 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa3f4006780 0x7fa3f4008c80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fa4040017d0 tx=0x7fa404007d30).stop 2021-02-10T11:13:28.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.253 7fa410091700 1 -- 172.21.2.101:0/248243546 shutdown_connections 2021-02-10T11:13:28.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.253 7fa410091700 1 --2- 172.21.2.101:0/248243546 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa3f4006780 0x7fa3f4008c80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:28.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.253 7fa410091700 1 -- 172.21.2.101:0/248243546 >> 172.21.2.101:0/248243546 conn(0x7fa4080ce690 msgr2=0x7fa4080e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:28.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.254 7fa410091700 1 -- 172.21.2.101:0/248243546 shutdown_connections 2021-02-10T11:13:28.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.254 7fa410091700 1 -- 172.21.2.101:0/248243546 wait complete. 2021-02-10T11:13:28.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.268 7fa410091700 1 Processor -- start 2021-02-10T11:13:28.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.268 7fa410091700 1 -- start start 2021-02-10T11:13:28.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.269 7fa410091700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa40808f500 0x7fa40819f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:28.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.269 7fa410091700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa40808fd90 0x7fa408090190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:28.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.269 7fa410091700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa4080907b0 0x7fa408094370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:28.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.269 7fa410091700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40808f900 con 0x7fa40808f500 2021-02-10T11:13:28.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.269 7fa410091700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40819ff20 con 0x7fa4080907b0 2021-02-10T11:13:28.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.269 7fa410091700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4081a0140 con 0x7fa40808fd90 2021-02-10T11:13:28.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.269 7fa40dead700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa40808f500 0x7fa40819f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:28.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.269 7fa40dead700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa40808f500 0x7fa40819f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44254/0 (socket says 172.21.2.101:44254) 2021-02-10T11:13:28.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.269 7fa40dead700 1 -- 172.21.2.101:0/4540503 learned_addr learned my addr 172.21.2.101:0/4540503 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:28.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.269 7fa40dead700 1 -- 172.21.2.101:0/4540503 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa40808fd90 msgr2=0x7fa408090190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:28.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.269 7fa40dead700 1 --2- 172.21.2.101:0/4540503 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa40808fd90 0x7fa408090190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:28.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.269 7fa40dead700 1 -- 172.21.2.101:0/4540503 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa4080907b0 msgr2=0x7fa408094370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:28.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.269 7fa40dead700 1 --2- 172.21.2.101:0/4540503 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa4080907b0 0x7fa408094370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:28.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.269 7fa40dead700 1 -- 172.21.2.101:0/4540503 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa408094990 con 0x7fa40808f500 2021-02-10T11:13:28.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.269 7fa40dead700 1 --2- 172.21.2.101:0/4540503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa40808f500 0x7fa40819f900 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fa3fc00b580 tx=0x7fa3fc00b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:28.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.270 7fa3fa7fc700 1 -- 172.21.2.101:0/4540503 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa3fc0068e0 con 0x7fa40808f500 2021-02-10T11:13:28.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.270 7fa3fa7fc700 1 -- 172.21.2.101:0/4540503 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3fc000de0 con 0x7fa40808f500 2021-02-10T11:13:28.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.270 7fa3fa7fc700 1 -- 172.21.2.101:0/4540503 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa3fc00bd10 con 0x7fa40808f500 2021-02-10T11:13:28.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.270 7fa410091700 1 -- 172.21.2.101:0/4540503 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa408094990 con 0x7fa40808f500 2021-02-10T11:13:28.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.270 7fa410091700 1 -- 172.21.2.101:0/4540503 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa408094990 con 0x7fa40808f500 2021-02-10T11:13:28.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.270 7fa3fa7fc700 1 -- 172.21.2.101:0/4540503 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa3fc00a000 con 0x7fa40808f500 2021-02-10T11:13:28.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.270 7fa3fa7fc700 1 --2- 172.21.2.101:0/4540503 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa3f00186c0 0x7fa3f001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:28.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.271 7fa3fa7fc700 1 -- 172.21.2.101:0/4540503 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa3fc00a000 con 0x7fa40808f500 2021-02-10T11:13:28.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.271 7fa40d6ac700 1 --2- 172.21.2.101:0/4540503 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa3f00186c0 0x7fa3f001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:28.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.271 7fa40d6ac700 1 --2- 172.21.2.101:0/4540503 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa3f00186c0 0x7fa3f001abc0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:28.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.301 7fa410091700 1 -- 172.21.2.101:0/4540503 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa408093e40 con 0x7fa40808f500 2021-02-10T11:13:28.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.304 7fa3fa7fc700 1 -- 172.21.2.101:0/4540503 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa3fc006c10 con 0x7fa40808f500 2021-02-10T11:13:28.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.443 7fa410091700 1 -- 172.21.2.101:0/4540503 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa408093e40 con 0x7fa40808f500 2021-02-10T11:13:28.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.443 7fa3fa7fc700 1 -- 172.21.2.101:0/4540503 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa3fc00a000 con 0x7fa40808f500 2021-02-10T11:13:28.448 INFO:teuthology.orchestra.run.gibba001.stdout:55834574856 2021-02-10T11:13:28.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.445 7fa410091700 1 -- 172.21.2.101:0/4540503 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa3f00186c0 msgr2=0x7fa3f001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:28.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.445 7fa410091700 1 --2- 172.21.2.101:0/4540503 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa3f00186c0 0x7fa3f001abc0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:28.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.445 7fa410091700 1 -- 172.21.2.101:0/4540503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa40808f500 msgr2=0x7fa40819f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:28.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.445 7fa410091700 1 --2- 172.21.2.101:0/4540503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa40808f500 0x7fa40819f900 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fa3fc00b580 tx=0x7fa3fc00b920).stop 2021-02-10T11:13:28.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.445 7fa410091700 1 -- 172.21.2.101:0/4540503 shutdown_connections 2021-02-10T11:13:28.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.445 7fa410091700 1 --2- 172.21.2.101:0/4540503 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa3f00186c0 0x7fa3f001abc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:28.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.445 7fa410091700 1 --2- 172.21.2.101:0/4540503 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa40808fd90 0x7fa408090190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:28.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.445 7fa410091700 1 --2- 172.21.2.101:0/4540503 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa4080907b0 0x7fa408094370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:28.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.445 7fa410091700 1 --2- 172.21.2.101:0/4540503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa40808f500 0x7fa40819f900 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:28.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.445 7fa410091700 1 -- 172.21.2.101:0/4540503 >> 172.21.2.101:0/4540503 conn(0x7fa4080b5560 msgr2=0x7fa4080a6200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:28.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.445 7fa410091700 1 -- 172.21.2.101:0/4540503 shutdown_connections 2021-02-10T11:13:28.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.445 7fa410091700 1 -- 172.21.2.101:0/4540503 wait complete. 2021-02-10T11:13:28.457 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2021-02-10T11:13:28.457 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd last-stat-seq osd.3 2021-02-10T11:13:28.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.863 7f8e0e57b700 1 Processor -- start 2021-02-10T11:13:28.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.863 7f8e0e57b700 1 -- start start 2021-02-10T11:13:28.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.863 7f8e0e57b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8e080e5070 0x7f8e081b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:28.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.863 7f8e0e57b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8e080e5470 0x7f8e08080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:28.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.863 7f8e0e57b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e0807ed80 con 0x7f8e081b3e40 2021-02-10T11:13:28.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.863 7f8e0e57b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e080d1790 con 0x7f8e080e5070 2021-02-10T11:13:28.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.863 7f8e0e57b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e080d19b0 con 0x7f8e080e5470 2021-02-10T11:13:28.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.863 7f8e07fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8e080e5070 0x7f8e081b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:28.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.863 7f8e07fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8e080e5070 0x7f8e081b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44260/0 (socket says 172.21.2.101:44260) 2021-02-10T11:13:28.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.864 7f8e07fff700 1 -- 172.21.2.101:0/4216126113 learned_addr learned my addr 172.21.2.101:0/4216126113 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:28.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.864 7f8e07fff700 1 -- 172.21.2.101:0/4216126113 >> v2:172.21.2.108:3300/0 conn(0x7f8e080e5470 msgr2=0x7f8e08080310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:28.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.864 7f8e07fff700 1 --2- 172.21.2.101:0/4216126113 >> v2:172.21.2.108:3300/0 conn(0x7f8e080e5470 0x7f8e08080310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:28.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.864 7f8e07fff700 1 -- 172.21.2.101:0/4216126113 >> v1:172.21.2.108:6789/0 conn(0x7f8e081b3e40 legacy=0x7f8e0807d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:28.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.864 7f8e07fff700 1 -- 172.21.2.101:0/4216126113 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e080d1bd0 con 0x7f8e080e5070 2021-02-10T11:13:28.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.864 7f8e07fff700 1 --2- 172.21.2.101:0/4216126113 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e080e5070 0x7f8e081b3820 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f8df8006700 tx=0x7f8df8009b80).ready entity=mon.0 client_cookie=8c4e179084a41cbb server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:28.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.864 7f8e06ffd700 1 -- 172.21.2.101:0/4216126113 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8df800c0c0 con 0x7f8e080e5070 2021-02-10T11:13:28.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.864 7f8e06ffd700 1 -- 172.21.2.101:0/4216126113 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df800c440 con 0x7f8e080e5070 2021-02-10T11:13:28.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.864 7f8e06ffd700 1 -- 172.21.2.101:0/4216126113 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8df800c960 con 0x7f8e080e5070 2021-02-10T11:13:28.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.865 7f8e0e57b700 1 -- 172.21.2.101:0/4216126113 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e080e5070 msgr2=0x7f8e081b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:28.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.865 7f8e0e57b700 1 --2- 172.21.2.101:0/4216126113 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e080e5070 0x7f8e081b3820 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f8df8006700 tx=0x7f8df8009b80).stop 2021-02-10T11:13:28.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.865 7f8e0e57b700 1 -- 172.21.2.101:0/4216126113 shutdown_connections 2021-02-10T11:13:28.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.865 7f8e0e57b700 1 --2- 172.21.2.101:0/4216126113 >> v2:172.21.2.108:3300/0 conn(0x7f8e080e5470 0x7f8e08080310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:28.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.865 7f8e0e57b700 1 --2- 172.21.2.101:0/4216126113 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e080e5070 0x7f8e081b3820 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:28.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.865 7f8e0e57b700 1 -- 172.21.2.101:0/4216126113 >> 172.21.2.101:0/4216126113 conn(0x7f8e080ce690 msgr2=0x7f8e080e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:28.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.865 7f8e0e57b700 1 -- 172.21.2.101:0/4216126113 shutdown_connections 2021-02-10T11:13:28.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.865 7f8e0e57b700 1 -- 172.21.2.101:0/4216126113 wait complete. 2021-02-10T11:13:28.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.880 7f8e0e57b700 1 Processor -- start 2021-02-10T11:13:28.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.880 7f8e0e57b700 1 -- start start 2021-02-10T11:13:28.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.880 7f8e0e57b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8e0808fbd0 0x7f8e0808c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:28.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.880 7f8e0e57b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e0808ffd0 0x7f8e0808c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:28.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.880 7f8e0e57b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8e0808d570 0x7f8e0808d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:28.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.880 7f8e0e57b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e080879d0 con 0x7f8e0808ffd0 2021-02-10T11:13:28.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.880 7f8e0e57b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e08087df0 con 0x7f8e0808fbd0 2021-02-10T11:13:28.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.880 7f8e0e57b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0808df90 con 0x7f8e0808d570 2021-02-10T11:13:28.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.880 7f8e077fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e0808ffd0 0x7f8e0808c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:28.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.881 7f8e077fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e0808ffd0 0x7f8e0808c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44266/0 (socket says 172.21.2.101:44266) 2021-02-10T11:13:28.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.881 7f8e077fe700 1 -- 172.21.2.101:0/1490736418 learned_addr learned my addr 172.21.2.101:0/1490736418 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:28.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.881 7f8e077fe700 1 -- 172.21.2.101:0/1490736418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8e0808d570 msgr2=0x7f8e0808d970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:28.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.881 7f8e077fe700 1 --2- 172.21.2.101:0/1490736418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8e0808d570 0x7f8e0808d970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:28.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.881 7f8e077fe700 1 -- 172.21.2.101:0/1490736418 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8e0808fbd0 msgr2=0x7f8e0808c2f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:28.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.881 7f8e077fe700 1 --2- 172.21.2.101:0/1490736418 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8e0808fbd0 0x7f8e0808c2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:28.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.881 7f8e077fe700 1 -- 172.21.2.101:0/1490736418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0808e1b0 con 0x7f8e0808ffd0 2021-02-10T11:13:28.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.881 7f8e077fe700 1 --2- 172.21.2.101:0/1490736418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e0808ffd0 0x7f8e0808c910 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f8dfc00b5b0 tx=0x7f8dfc00b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:28.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.881 7f8e04ff9700 1 -- 172.21.2.101:0/1490736418 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8dfc00bd40 con 0x7f8e0808ffd0 2021-02-10T11:13:28.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.881 7f8e04ff9700 1 -- 172.21.2.101:0/1490736418 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dfc000e20 con 0x7f8e0808ffd0 2021-02-10T11:13:28.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.881 7f8e04ff9700 1 -- 172.21.2.101:0/1490736418 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8dfc0054a0 con 0x7f8e0808ffd0 2021-02-10T11:13:28.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.881 7f8e0e57b700 1 -- 172.21.2.101:0/1490736418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e0808e1b0 con 0x7f8e0808ffd0 2021-02-10T11:13:28.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.882 7f8e0e57b700 1 -- 172.21.2.101:0/1490736418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e0808e740 con 0x7f8e0808ffd0 2021-02-10T11:13:28.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.882 7f8e04ff9700 1 -- 172.21.2.101:0/1490736418 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8dfc006030 con 0x7f8e0808ffd0 2021-02-10T11:13:28.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.882 7f8e04ff9700 1 --2- 172.21.2.101:0/1490736418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8de80186c0 0x7f8de801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:28.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.882 7f8e04ff9700 1 -- 172.21.2.101:0/1490736418 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8dfc032560 con 0x7f8e0808ffd0 2021-02-10T11:13:28.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.882 7f8e07fff700 1 --2- 172.21.2.101:0/1490736418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8de80186c0 0x7f8de801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:28.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.883 7f8e07fff700 1 --2- 172.21.2.101:0/1490736418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8de80186c0 0x7f8de801abc0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:28.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.914 7f8e0e57b700 1 -- 172.21.2.101:0/1490736418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e0807ed80 con 0x7f8e0808ffd0 2021-02-10T11:13:28.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:28.916 7f8e04ff9700 1 -- 172.21.2.101:0/1490736418 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8dfc02df50 con 0x7f8e0808ffd0 2021-02-10T11:13:29.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.055 7f8e0e57b700 1 -- 172.21.2.101:0/1490736418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8e0807ed80 con 0x7f8e0808ffd0 2021-02-10T11:13:29.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.055 7f8e04ff9700 1 -- 172.21.2.101:0/1490736418 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8dfc02df50 con 0x7f8e0808ffd0 2021-02-10T11:13:29.060 INFO:teuthology.orchestra.run.gibba001.stdout:55834574856 2021-02-10T11:13:29.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.057 7f8e0e57b700 1 -- 172.21.2.101:0/1490736418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8de80186c0 msgr2=0x7f8de801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:29.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.057 7f8e0e57b700 1 --2- 172.21.2.101:0/1490736418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8de80186c0 0x7f8de801abc0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:29.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.057 7f8e0e57b700 1 -- 172.21.2.101:0/1490736418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e0808ffd0 msgr2=0x7f8e0808c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:29.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.057 7f8e0e57b700 1 --2- 172.21.2.101:0/1490736418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e0808ffd0 0x7f8e0808c910 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f8dfc00b5b0 tx=0x7f8dfc00b950).stop 2021-02-10T11:13:29.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.057 7f8e0e57b700 1 -- 172.21.2.101:0/1490736418 shutdown_connections 2021-02-10T11:13:29.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.057 7f8e0e57b700 1 --2- 172.21.2.101:0/1490736418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8de80186c0 0x7f8de801abc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:29.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.057 7f8e0e57b700 1 --2- 172.21.2.101:0/1490736418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8e0808d570 0x7f8e0808d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:29.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.057 7f8e0e57b700 1 --2- 172.21.2.101:0/1490736418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e0808ffd0 0x7f8e0808c910 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:29.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.057 7f8e0e57b700 1 --2- 172.21.2.101:0/1490736418 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8e0808fbd0 0x7f8e0808c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:29.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.057 7f8e0e57b700 1 -- 172.21.2.101:0/1490736418 >> 172.21.2.101:0/1490736418 conn(0x7f8e08085a20 msgr2=0x7f8e080b44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:29.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.058 7f8e0e57b700 1 -- 172.21.2.101:0/1490736418 shutdown_connections 2021-02-10T11:13:29.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.058 7f8e0e57b700 1 -- 172.21.2.101:0/1490736418 wait complete. 2021-02-10T11:13:29.070 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2021-02-10T11:13:29.070 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd last-stat-seq osd.4 2021-02-10T11:13:29.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.476 7f29eab8d700 1 Processor -- start 2021-02-10T11:13:29.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.476 7f29eab8d700 1 -- start start 2021-02-10T11:13:29.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.477 7f29eab8d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29e40e5070 0x7f29e41b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:29.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.477 7f29eab8d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f29e40e5470 0x7f29e407d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:29.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.477 7f29eab8d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e407ed80 con 0x7f29e41b3e40 2021-02-10T11:13:29.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.477 7f29eab8d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e4080310 con 0x7f29e40e5470 2021-02-10T11:13:29.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.477 7f29eab8d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e4080500 con 0x7f29e40e5070 2021-02-10T11:13:29.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.477 7f29e91aa700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f29e41b3e40 0x7f29e407dcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44768/0 (socket says 172.21.2.101:44768) 2021-02-10T11:13:29.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.477 7f29e91aa700 1 -- 172.21.2.101:0/4162924734 learned_addr learned my addr 172.21.2.101:0/4162924734 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:29.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.478 7f29e37fe700 1 -- 172.21.2.101:0/4162924734 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128565247 0 0) 0x7f29dc000f90 con 0x7f29e41b3e40 2021-02-10T11:13:29.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.478 7f29e37fe700 1 -- 172.21.2.101:0/4162924734 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29d0002fe0 con 0x7f29e41b3e40 2021-02-10T11:13:29.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.478 7f29e3fff700 1 --2- 172.21.2.101:0/4162924734 >> v2:172.21.2.108:3300/0 conn(0x7f29e40e5470 0x7f29e407d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:29.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.478 7f29e89a9700 1 --2- 172.21.2.101:0/4162924734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29e40e5070 0x7f29e41b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:29.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.478 7f29e37fe700 1 -- 172.21.2.101:0/4162924734 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3217879988 0 0) 0x7f29dc001460 con 0x7f29e41b3e40 2021-02-10T11:13:29.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.478 7f29e37fe700 1 -- 172.21.2.101:0/4162924734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29e40e5070 msgr2=0x7f29e41b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:29.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.478 7f29e37fe700 1 --2- 172.21.2.101:0/4162924734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29e40e5070 0x7f29e41b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:29.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.478 7f29e37fe700 1 -- 172.21.2.101:0/4162924734 >> v2:172.21.2.108:3300/0 conn(0x7f29e40e5470 msgr2=0x7f29e407d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:29.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.478 7f29e37fe700 1 --2- 172.21.2.101:0/4162924734 >> v2:172.21.2.108:3300/0 conn(0x7f29e40e5470 0x7f29e407d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:29.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.478 7f29e37fe700 1 -- 172.21.2.101:0/4162924734 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e40806f0 con 0x7f29e41b3e40 2021-02-10T11:13:29.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.478 7f29e37fe700 1 -- 172.21.2.101:0/4162924734 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f29dc0019e0 con 0x7f29e41b3e40 2021-02-10T11:13:29.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.478 7f29e37fe700 1 -- 172.21.2.101:0/4162924734 >> v1:172.21.2.101:6789/0 conn(0x7f29e41b3e40 legacy=0x7f29e407dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:29.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.478 7f29e37fe700 1 --2- 172.21.2.101:0/4162924734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29d0003af0 0x7f29d0005f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:29.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.478 7f29e37fe700 1 --2- 172.21.2.101:0/4162924734 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29d00067e0 0x7f29d0008d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:29.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.478 7f29e37fe700 1 --2- 172.21.2.101:0/4162924734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29d00095a0 0x7f29d000bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:29.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.478 7f29e37fe700 1 -- 172.21.2.101:0/4162924734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d0050380 con 0x7f29d00095a0 2021-02-10T11:13:29.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.478 7f29e37fe700 1 -- 172.21.2.101:0/4162924734 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d00505a0 con 0x7f29d00067e0 2021-02-10T11:13:29.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.478 7f29e37fe700 1 -- 172.21.2.101:0/4162924734 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d00507c0 con 0x7f29d0003af0 2021-02-10T11:13:29.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.478 7f29e3fff700 1 --2- 172.21.2.101:0/4162924734 >> v2:172.21.2.108:3300/0 conn(0x7f29e40e5470 0x7f29e407d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:13:29.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.478 7f29e91aa700 1 --2- 172.21.2.101:0/4162924734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29d00095a0 0x7f29d000bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:29.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.479 7f29e3fff700 1 --2- 172.21.2.101:0/4162924734 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29d00067e0 0x7f29d0008d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:29.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.479 7f29e89a9700 1 --2- 172.21.2.101:0/4162924734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29d0003af0 0x7f29d0005f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:29.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.479 7f29e91aa700 1 -- 172.21.2.101:0/4162924734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29d0003af0 msgr2=0x7f29d0005f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:29.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.479 7f29e91aa700 1 --2- 172.21.2.101:0/4162924734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29d0003af0 0x7f29d0005f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:29.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.479 7f29e91aa700 1 -- 172.21.2.101:0/4162924734 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29d00067e0 msgr2=0x7f29d0008d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:29.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.479 7f29e91aa700 1 --2- 172.21.2.101:0/4162924734 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29d00067e0 0x7f29d0008d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:29.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.479 7f29e91aa700 1 -- 172.21.2.101:0/4162924734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d0050a60 con 0x7f29d00095a0 2021-02-10T11:13:29.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.479 7f29e89a9700 1 --2- 172.21.2.101:0/4162924734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29d0003af0 0x7f29d0005f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:13:29.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.479 7f29e91aa700 1 --2- 172.21.2.101:0/4162924734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29d00095a0 0x7f29d000bac0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f29dc001140 tx=0x7f29dc00a310).ready entity=mon.0 client_cookie=b3b456713a611f2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:29.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.479 7f29e37fe700 1 -- 172.21.2.101:0/4162924734 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f29dc000e60 con 0x7f29d00095a0 2021-02-10T11:13:29.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.479 7f29e37fe700 1 -- 172.21.2.101:0/4162924734 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc0070c0 con 0x7f29d00095a0 2021-02-10T11:13:29.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.479 7f29e37fe700 1 -- 172.21.2.101:0/4162924734 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f29dc007520 con 0x7f29d00095a0 2021-02-10T11:13:29.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.479 7f29eab8d700 1 -- 172.21.2.101:0/4162924734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29d00095a0 msgr2=0x7f29d000bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:29.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.479 7f29eab8d700 1 --2- 172.21.2.101:0/4162924734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29d00095a0 0x7f29d000bac0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f29dc001140 tx=0x7f29dc00a310).stop 2021-02-10T11:13:29.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.479 7f29e91aa700 1 -- 172.21.2.101:0/4162924734 reap_dead start 2021-02-10T11:13:29.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.480 7f29eab8d700 1 -- 172.21.2.101:0/4162924734 shutdown_connections 2021-02-10T11:13:29.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.480 7f29eab8d700 1 -- 172.21.2.101:0/4162924734 >> 172.21.2.101:0/4162924734 conn(0x7f29e40ce690 msgr2=0x7f29e40e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:29.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.480 7f29eab8d700 1 -- 172.21.2.101:0/4162924734 shutdown_connections 2021-02-10T11:13:29.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.480 7f29eab8d700 1 -- 172.21.2.101:0/4162924734 wait complete. 2021-02-10T11:13:29.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.494 7f29eab8d700 1 Processor -- start 2021-02-10T11:13:29.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.494 7f29eab8d700 1 -- start start 2021-02-10T11:13:29.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.495 7f29eab8d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29e40b4ad0 0x7f29e40a78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:29.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.495 7f29eab8d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29e40a7ef0 0x7f29e41a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:29.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.495 7f29eab8d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29e40a82f0 0x7f29e41a73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:29.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.495 7f29eab8d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40aa3c0 con 0x7f29e40a7ef0 2021-02-10T11:13:29.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.495 7f29eab8d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40a88c0 con 0x7f29e40a82f0 2021-02-10T11:13:29.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.495 7f29eab8d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40b4ed0 con 0x7f29e40b4ad0 2021-02-10T11:13:29.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.495 7f29e3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29e40a7ef0 0x7f29e41a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:29.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.495 7f29e3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29e40a7ef0 0x7f29e41a6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44286/0 (socket says 172.21.2.101:44286) 2021-02-10T11:13:29.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.495 7f29e3fff700 1 -- 172.21.2.101:0/3952398843 learned_addr learned my addr 172.21.2.101:0/3952398843 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:29.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.495 7f29e89a9700 1 --2- 172.21.2.101:0/3952398843 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29e40b4ad0 0x7f29e40a78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:29.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.496 7f29e91aa700 1 --2- 172.21.2.101:0/3952398843 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29e40a82f0 0x7f29e41a73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:29.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.496 7f29e3fff700 1 -- 172.21.2.101:0/3952398843 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29e40b4ad0 msgr2=0x7f29e40a78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:29.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.496 7f29e3fff700 1 --2- 172.21.2.101:0/3952398843 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29e40b4ad0 0x7f29e40a78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:29.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.496 7f29e3fff700 1 -- 172.21.2.101:0/3952398843 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29e40a82f0 msgr2=0x7f29e41a73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:29.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.496 7f29e3fff700 1 --2- 172.21.2.101:0/3952398843 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29e40a82f0 0x7f29e41a73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:29.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.496 7f29e3fff700 1 -- 172.21.2.101:0/3952398843 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e41a79d0 con 0x7f29e40a7ef0 2021-02-10T11:13:29.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.496 7f29e3fff700 1 --2- 172.21.2.101:0/3952398843 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29e40a7ef0 0x7f29e41a6d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f29cc006f70 tx=0x7f29cc00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:29.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.496 7f29e17fa700 1 -- 172.21.2.101:0/3952398843 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f29cc006890 con 0x7f29e40a7ef0 2021-02-10T11:13:29.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.496 7f29e17fa700 1 -- 172.21.2.101:0/3952398843 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc006030 con 0x7f29e40a7ef0 2021-02-10T11:13:29.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.496 7f29e17fa700 1 -- 172.21.2.101:0/3952398843 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f29cc005560 con 0x7f29e40a7ef0 2021-02-10T11:13:29.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.496 7f29eab8d700 1 -- 172.21.2.101:0/3952398843 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e41a79d0 con 0x7f29e40a7ef0 2021-02-10T11:13:29.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.496 7f29eab8d700 1 -- 172.21.2.101:0/3952398843 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e41a79d0 con 0x7f29e40a7ef0 2021-02-10T11:13:29.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.497 7f29e17fa700 1 -- 172.21.2.101:0/3952398843 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f29cc006890 con 0x7f29e40a7ef0 2021-02-10T11:13:29.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.497 7f29e17fa700 1 --2- 172.21.2.101:0/3952398843 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29c80186c0 0x7f29c801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:29.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.497 7f29e17fa700 1 -- 172.21.2.101:0/3952398843 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f29cc032570 con 0x7f29e40a7ef0 2021-02-10T11:13:29.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.497 7f29e89a9700 1 --2- 172.21.2.101:0/3952398843 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29c80186c0 0x7f29c801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:29.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.498 7f29e89a9700 1 --2- 172.21.2.101:0/3952398843 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29c80186c0 0x7f29c801abc0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:29.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.527 7f29eab8d700 1 -- 172.21.2.101:0/3952398843 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e4084a90 con 0x7f29e40a7ef0 2021-02-10T11:13:29.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.529 7f29e17fa700 1 -- 172.21.2.101:0/3952398843 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f29cc0065a0 con 0x7f29e40a7ef0 2021-02-10T11:13:29.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.668 7f29eab8d700 1 -- 172.21.2.101:0/3952398843 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f29e407ed80 con 0x7f29e40a7ef0 2021-02-10T11:13:29.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.668 7f29e17fa700 1 -- 172.21.2.101:0/3952398843 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f29cc006890 con 0x7f29e40a7ef0 2021-02-10T11:13:29.672 INFO:teuthology.orchestra.run.gibba001.stdout:55834574856 2021-02-10T11:13:29.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.670 7f29eab8d700 1 -- 172.21.2.101:0/3952398843 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29c80186c0 msgr2=0x7f29c801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:29.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.670 7f29eab8d700 1 --2- 172.21.2.101:0/3952398843 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29c80186c0 0x7f29c801abc0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:29.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.670 7f29eab8d700 1 -- 172.21.2.101:0/3952398843 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29e40a7ef0 msgr2=0x7f29e41a6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:29.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.670 7f29eab8d700 1 --2- 172.21.2.101:0/3952398843 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29e40a7ef0 0x7f29e41a6d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f29cc006f70 tx=0x7f29cc00b960).stop 2021-02-10T11:13:29.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.670 7f29eab8d700 1 -- 172.21.2.101:0/3952398843 shutdown_connections 2021-02-10T11:13:29.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.670 7f29eab8d700 1 --2- 172.21.2.101:0/3952398843 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29e40a82f0 0x7f29e41a73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:29.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.670 7f29eab8d700 1 --2- 172.21.2.101:0/3952398843 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29e40a7ef0 0x7f29e41a6d90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:29.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.670 7f29eab8d700 1 --2- 172.21.2.101:0/3952398843 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29c80186c0 0x7f29c801abc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:29.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.670 7f29eab8d700 1 --2- 172.21.2.101:0/3952398843 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29e40b4ad0 0x7f29e40a78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:29.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.670 7f29eab8d700 1 -- 172.21.2.101:0/3952398843 >> 172.21.2.101:0/3952398843 conn(0x7f29e40b5560 msgr2=0x7f29e40a6200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:29.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.670 7f29eab8d700 1 -- 172.21.2.101:0/3952398843 shutdown_connections 2021-02-10T11:13:29.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:29.670 7f29eab8d700 1 -- 172.21.2.101:0/3952398843 wait complete. 2021-02-10T11:13:29.682 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2021-02-10T11:13:29.683 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd last-stat-seq osd.5 2021-02-10T11:13:30.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.087 7fd42d901700 1 Processor -- start 2021-02-10T11:13:30.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.088 7fd42d901700 1 -- start start 2021-02-10T11:13:30.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.088 7fd42d901700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd4280e5070 0x7fd4281b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:30.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.088 7fd42d901700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd42807ed80 con 0x7fd4281b3e40 2021-02-10T11:13:30.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.088 7fd42d901700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd42807e670 con 0x7fd4280e5470 2021-02-10T11:13:30.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.088 7fd42d901700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd42807e890 con 0x7fd4280e5070 2021-02-10T11:13:30.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.088 7fd426d73700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd4281b3e40 0x7fd42807d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44786/0 (socket says 172.21.2.101:44786) 2021-02-10T11:13:30.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.088 7fd426d73700 1 -- 172.21.2.101:0/3013936432 learned_addr learned my addr 172.21.2.101:0/3013936432 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:30.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.089 7fd426572700 1 -- 172.21.2.101:0/3013936432 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265010365 0 0) 0x7fd41c0009b0 con 0x7fd4281b3e40 2021-02-10T11:13:30.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.089 7fd426572700 1 -- 172.21.2.101:0/3013936432 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd410002fe0 con 0x7fd4281b3e40 2021-02-10T11:13:30.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.089 7fd427574700 1 --2- 172.21.2.101:0/3013936432 >> v2:172.21.2.108:3300/0 conn(0x7fd4280e5070 0x7fd4281b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:30.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.089 7fd426572700 1 -- 172.21.2.101:0/3013936432 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2007943866 0 0) 0x7fd41c001010 con 0x7fd4281b3e40 2021-02-10T11:13:30.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.089 7fd426572700 1 -- 172.21.2.101:0/3013936432 >> v2:172.21.2.108:3300/0 conn(0x7fd4280e5070 msgr2=0x7fd4281b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:30.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.089 7fd426572700 1 --2- 172.21.2.101:0/3013936432 >> v2:172.21.2.108:3300/0 conn(0x7fd4280e5070 0x7fd4281b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:30.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.089 7fd426572700 1 -- 172.21.2.101:0/3013936432 >> v1:172.21.2.108:6789/0 conn(0x7fd4280e5470 legacy=0x7fd42807de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:30.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.089 7fd426572700 1 -- 172.21.2.101:0/3013936432 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42807eab0 con 0x7fd4281b3e40 2021-02-10T11:13:30.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.089 7fd426572700 1 -- 172.21.2.101:0/3013936432 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fd41c001590 con 0x7fd4281b3e40 2021-02-10T11:13:30.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.089 7fd426572700 1 -- 172.21.2.101:0/3013936432 >> v1:172.21.2.101:6789/0 conn(0x7fd4281b3e40 legacy=0x7fd42807d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:30.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.089 7fd426572700 1 --2- 172.21.2.101:0/3013936432 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd410003a60 0x7fd410005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:30.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.089 7fd426572700 1 --2- 172.21.2.101:0/3013936432 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd410006800 0x7fd410008d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:30.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.089 7fd426572700 1 --2- 172.21.2.101:0/3013936432 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4100095a0 0x7fd41000bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:30.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.089 7fd426572700 1 -- 172.21.2.101:0/3013936432 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41000c310 con 0x7fd4100095a0 2021-02-10T11:13:30.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.089 7fd426572700 1 -- 172.21.2.101:0/3013936432 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41000c530 con 0x7fd410006800 2021-02-10T11:13:30.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.089 7fd426572700 1 -- 172.21.2.101:0/3013936432 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd410050760 con 0x7fd410003a60 2021-02-10T11:13:30.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.090 7fd427d75700 1 --2- 172.21.2.101:0/3013936432 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4100095a0 0x7fd41000bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:30.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.090 7fd427574700 1 --2- 172.21.2.101:0/3013936432 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd410003a60 0x7fd410005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:30.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.090 7fd426d73700 1 --2- 172.21.2.101:0/3013936432 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd410006800 0x7fd410008d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:30.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.090 7fd427d75700 1 -- 172.21.2.101:0/3013936432 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd410003a60 msgr2=0x7fd410005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:30.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.090 7fd427d75700 1 --2- 172.21.2.101:0/3013936432 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd410003a60 0x7fd410005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:30.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.090 7fd427d75700 1 -- 172.21.2.101:0/3013936432 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd410006800 msgr2=0x7fd410008d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:30.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.090 7fd427d75700 1 --2- 172.21.2.101:0/3013936432 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd410006800 0x7fd410008d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:30.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.090 7fd427d75700 1 -- 172.21.2.101:0/3013936432 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd410050990 con 0x7fd4100095a0 2021-02-10T11:13:30.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.090 7fd426d73700 1 --2- 172.21.2.101:0/3013936432 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd410006800 0x7fd410008d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:13:30.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.090 7fd427d75700 1 -- 172.21.2.101:0/3013936432 reap_dead start 2021-02-10T11:13:30.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.090 7fd427d75700 1 --2- 172.21.2.101:0/3013936432 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4100095a0 0x7fd41000bac0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fd420006e40 tx=0x7fd420009e30).ready entity=mon.0 client_cookie=1932413a7fe825e8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:30.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.090 7fd426572700 1 -- 172.21.2.101:0/3013936432 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd42000b300 con 0x7fd4100095a0 2021-02-10T11:13:30.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.090 7fd426572700 1 -- 172.21.2.101:0/3013936432 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42000b580 con 0x7fd4100095a0 2021-02-10T11:13:30.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.090 7fd426572700 1 -- 172.21.2.101:0/3013936432 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd42000ba80 con 0x7fd4100095a0 2021-02-10T11:13:30.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.090 7fd42d901700 1 -- 172.21.2.101:0/3013936432 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4100095a0 msgr2=0x7fd41000bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:30.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.090 7fd42d901700 1 --2- 172.21.2.101:0/3013936432 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4100095a0 0x7fd41000bac0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fd420006e40 tx=0x7fd420009e30).stop 2021-02-10T11:13:30.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.090 7fd42d901700 1 -- 172.21.2.101:0/3013936432 shutdown_connections 2021-02-10T11:13:30.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.091 7fd42d901700 1 --2- 172.21.2.101:0/3013936432 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4100095a0 0x7fd41000bac0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:30.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.091 7fd42d901700 1 -- 172.21.2.101:0/3013936432 >> 172.21.2.101:0/3013936432 conn(0x7fd4280ce690 msgr2=0x7fd4280e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:30.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.091 7fd42d901700 1 -- 172.21.2.101:0/3013936432 shutdown_connections 2021-02-10T11:13:30.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.091 7fd42d901700 1 -- 172.21.2.101:0/3013936432 wait complete. 2021-02-10T11:13:30.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.105 7fd42d901700 1 Processor -- start 2021-02-10T11:13:30.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.105 7fd42d901700 1 -- start start 2021-02-10T11:13:30.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.105 7fd42d901700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd42808f500 0x7fd42819f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:30.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.105 7fd42d901700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd42808fd90 0x7fd428090190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:30.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.105 7fd42d901700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4280907b0 0x7fd428094370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:30.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.105 7fd42d901700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42808f900 con 0x7fd42808f500 2021-02-10T11:13:30.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.105 7fd42d901700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42819ff20 con 0x7fd42808fd90 2021-02-10T11:13:30.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.105 7fd42d901700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4281a0140 con 0x7fd4280907b0 2021-02-10T11:13:30.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.106 7fd427574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd42808f500 0x7fd42819f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:30.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.106 7fd427574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd42808f500 0x7fd42819f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44302/0 (socket says 172.21.2.101:44302) 2021-02-10T11:13:30.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.106 7fd427574700 1 -- 172.21.2.101:0/1619820216 learned_addr learned my addr 172.21.2.101:0/1619820216 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:30.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.106 7fd427574700 1 -- 172.21.2.101:0/1619820216 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4280907b0 msgr2=0x7fd428094370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:30.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.106 7fd427574700 1 --2- 172.21.2.101:0/1619820216 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4280907b0 0x7fd428094370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:30.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.106 7fd427574700 1 -- 172.21.2.101:0/1619820216 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd42808fd90 msgr2=0x7fd428090190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:30.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.106 7fd427574700 1 --2- 172.21.2.101:0/1619820216 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd42808fd90 0x7fd428090190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:30.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.106 7fd427574700 1 -- 172.21.2.101:0/1619820216 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd428094990 con 0x7fd42808f500 2021-02-10T11:13:30.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.106 7fd427574700 1 --2- 172.21.2.101:0/1619820216 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd42808f500 0x7fd42819f900 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fd418006f60 tx=0x7fd41800b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:30.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.106 7fd40ffff700 1 -- 172.21.2.101:0/1619820216 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd41800bd60 con 0x7fd42808f500 2021-02-10T11:13:30.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.106 7fd42d901700 1 -- 172.21.2.101:0/1619820216 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd428094990 con 0x7fd42808f500 2021-02-10T11:13:30.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.106 7fd40ffff700 1 -- 172.21.2.101:0/1619820216 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41800bd60 con 0x7fd42808f500 2021-02-10T11:13:30.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.106 7fd42d901700 1 -- 172.21.2.101:0/1619820216 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd428094f20 con 0x7fd42808f500 2021-02-10T11:13:30.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.106 7fd40ffff700 1 -- 172.21.2.101:0/1619820216 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd418000c90 con 0x7fd42808f500 2021-02-10T11:13:30.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.107 7fd40ffff700 1 -- 172.21.2.101:0/1619820216 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd418006b00 con 0x7fd42808f500 2021-02-10T11:13:30.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.107 7fd40ffff700 1 --2- 172.21.2.101:0/1619820216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4080186c0 0x7fd40801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:30.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.107 7fd40ffff700 1 -- 172.21.2.101:0/1619820216 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd4180327c0 con 0x7fd42808f500 2021-02-10T11:13:30.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.107 7fd426d73700 1 --2- 172.21.2.101:0/1619820216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4080186c0 0x7fd40801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:30.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.107 7fd426d73700 1 --2- 172.21.2.101:0/1619820216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4080186c0 0x7fd40801abc0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:30.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.138 7fd42d901700 1 -- 172.21.2.101:0/1619820216 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4280957b0 con 0x7fd42808f500 2021-02-10T11:13:30.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.140 7fd40ffff700 1 -- 172.21.2.101:0/1619820216 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd41801f500 con 0x7fd42808f500 2021-02-10T11:13:30.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.280 7fd42d901700 1 -- 172.21.2.101:0/1619820216 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fd428093e40 con 0x7fd42808f500 2021-02-10T11:13:30.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.281 7fd40ffff700 1 -- 172.21.2.101:0/1619820216 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd41800bd60 con 0x7fd42808f500 2021-02-10T11:13:30.285 INFO:teuthology.orchestra.run.gibba001.stdout:55834574857 2021-02-10T11:13:30.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.283 7fd42d901700 1 -- 172.21.2.101:0/1619820216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4080186c0 msgr2=0x7fd40801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:30.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.283 7fd42d901700 1 --2- 172.21.2.101:0/1619820216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4080186c0 0x7fd40801abc0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:30.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.283 7fd42d901700 1 -- 172.21.2.101:0/1619820216 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd42808f500 msgr2=0x7fd42819f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:30.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.283 7fd42d901700 1 --2- 172.21.2.101:0/1619820216 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd42808f500 0x7fd42819f900 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fd418006f60 tx=0x7fd41800b970).stop 2021-02-10T11:13:30.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.283 7fd42d901700 1 -- 172.21.2.101:0/1619820216 shutdown_connections 2021-02-10T11:13:30.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.283 7fd42d901700 1 --2- 172.21.2.101:0/1619820216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4080186c0 0x7fd40801abc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:30.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.283 7fd42d901700 1 --2- 172.21.2.101:0/1619820216 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4280907b0 0x7fd428094370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:30.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.283 7fd42d901700 1 --2- 172.21.2.101:0/1619820216 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd42808fd90 0x7fd428090190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:30.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.283 7fd42d901700 1 --2- 172.21.2.101:0/1619820216 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd42808f500 0x7fd42819f900 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:30.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.283 7fd42d901700 1 -- 172.21.2.101:0/1619820216 >> 172.21.2.101:0/1619820216 conn(0x7fd4280b5560 msgr2=0x7fd4280a6200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:30.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.283 7fd42d901700 1 -- 172.21.2.101:0/1619820216 shutdown_connections 2021-02-10T11:13:30.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.283 7fd42d901700 1 -- 172.21.2.101:0/1619820216 wait complete. 2021-02-10T11:13:30.295 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574857 for osd.5 2021-02-10T11:13:30.296 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd last-stat-seq osd.6 2021-02-10T11:13:30.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.700 7f4ee553b700 1 Processor -- start 2021-02-10T11:13:30.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.700 7f4ee553b700 1 -- start start 2021-02-10T11:13:30.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.701 7f4ee553b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f4ee007e6a0 0x7f4ee00d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:30.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.701 7f4ee553b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee007ed80 con 0x7f4ee00e5070 2021-02-10T11:13:30.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.701 7f4ee553b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee007eaa0 con 0x7f4ee00e5470 2021-02-10T11:13:30.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.701 7f4ee553b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ee01b4080 con 0x7f4ee007e6a0 2021-02-10T11:13:30.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.701 7f4edfac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f4ee007e6a0 0x7f4ee00d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:30.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.701 7f4edfac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f4ee007e6a0 0x7f4ee00d1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44310/0 (socket says 172.21.2.101:44310) 2021-02-10T11:13:30.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.701 7f4edfac0700 1 -- 172.21.2.101:0/49488895 learned_addr learned my addr 172.21.2.101:0/49488895 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:30.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.701 7f4edf2bf700 1 --1- 172.21.2.101:0/49488895 >> v1:172.21.2.101:6789/0 conn(0x7f4ee00e5070 0x7f4ee01b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44802/0 (socket says 172.21.2.101:44802) 2021-02-10T11:13:30.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.701 7f4edfac0700 1 -- 172.21.2.101:0/49488895 >> v1:172.21.2.108:6789/0 conn(0x7f4ee00e5470 legacy=0x7f4ee007f300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:13:30.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.701 7f4edfac0700 1 -- 172.21.2.101:0/49488895 >> v1:172.21.2.101:6789/0 conn(0x7f4ee00e5070 legacy=0x7f4ee01b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:30.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.701 7f4edfac0700 1 -- 172.21.2.101:0/49488895 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee00d1db0 con 0x7f4ee007e6a0 2021-02-10T11:13:30.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.701 7f4edfac0700 1 --2- 172.21.2.101:0/49488895 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ee007e6a0 0x7f4ee00d1790 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f4ed8006f00 tx=0x7f4ed8009f10).ready entity=mon.0 client_cookie=b3ec827a89252ca8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:30.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.701 7f4ede2bd700 1 -- 172.21.2.101:0/49488895 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4ed800b3b0 con 0x7f4ee007e6a0 2021-02-10T11:13:30.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.701 7f4ede2bd700 1 -- 172.21.2.101:0/49488895 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed800b730 con 0x7f4ee007e6a0 2021-02-10T11:13:30.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.701 7f4ede2bd700 1 -- 172.21.2.101:0/49488895 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4ed800bc10 con 0x7f4ee007e6a0 2021-02-10T11:13:30.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.701 7f4ee553b700 1 -- 172.21.2.101:0/49488895 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ee007e6a0 msgr2=0x7f4ee00d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:30.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.701 7f4ee553b700 1 --2- 172.21.2.101:0/49488895 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ee007e6a0 0x7f4ee00d1790 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f4ed8006f00 tx=0x7f4ed8009f10).stop 2021-02-10T11:13:30.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.702 7f4ee553b700 1 -- 172.21.2.101:0/49488895 shutdown_connections 2021-02-10T11:13:30.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.702 7f4ee553b700 1 --2- 172.21.2.101:0/49488895 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ee007e6a0 0x7f4ee00d1790 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:30.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.702 7f4ee553b700 1 -- 172.21.2.101:0/49488895 >> 172.21.2.101:0/49488895 conn(0x7f4ee00ce690 msgr2=0x7f4ee00e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:30.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.702 7f4ee553b700 1 -- 172.21.2.101:0/49488895 shutdown_connections 2021-02-10T11:13:30.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.702 7f4ee553b700 1 -- 172.21.2.101:0/49488895 wait complete. 2021-02-10T11:13:30.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.716 7f4ee553b700 1 Processor -- start 2021-02-10T11:13:30.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.716 7f4ee553b700 1 -- start start 2021-02-10T11:13:30.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.717 7f4ee553b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ee009fb20 0x7f4ee00c2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:30.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.717 7f4ee553b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ee01c6890 0x7f4ee00c2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:30.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.717 7f4ee553b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ee00b8d70 0x7f4ee009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:30.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.717 7f4ee553b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee009f320 con 0x7f4ee00b8d70 2021-02-10T11:13:30.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.717 7f4ee553b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee009f540 con 0x7f4ee01c6890 2021-02-10T11:13:30.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.717 7f4ee553b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee009f760 con 0x7f4ee009fb20 2021-02-10T11:13:30.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.717 7f4edfac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ee00b8d70 0x7f4ee009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:30.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.717 7f4edfac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ee00b8d70 0x7f4ee009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44316/0 (socket says 172.21.2.101:44316) 2021-02-10T11:13:30.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.717 7f4edfac0700 1 -- 172.21.2.101:0/4070550860 learned_addr learned my addr 172.21.2.101:0/4070550860 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:30.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.717 7f4edfac0700 1 -- 172.21.2.101:0/4070550860 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ee009fb20 msgr2=0x7f4ee00c2110 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:30.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.717 7f4edfac0700 1 --2- 172.21.2.101:0/4070550860 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ee009fb20 0x7f4ee00c2110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:30.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.717 7f4edfac0700 1 -- 172.21.2.101:0/4070550860 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ee01c6890 msgr2=0x7f4ee00c2730 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:30.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.717 7f4edfac0700 1 --2- 172.21.2.101:0/4070550860 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ee01c6890 0x7f4ee00c2730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:30.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.717 7f4edfac0700 1 -- 172.21.2.101:0/4070550860 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee00c3320 con 0x7f4ee00b8d70 2021-02-10T11:13:30.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.718 7f4edfac0700 1 --2- 172.21.2.101:0/4070550860 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ee00b8d70 0x7f4ee009ed00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f4ed800b8e0 tx=0x7f4ed800bc80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:30.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.718 7f4ed7fff700 1 -- 172.21.2.101:0/4070550860 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4ed8009450 con 0x7f4ee00b8d70 2021-02-10T11:13:30.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.718 7f4ed7fff700 1 -- 172.21.2.101:0/4070550860 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed80058b0 con 0x7f4ee00b8d70 2021-02-10T11:13:30.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.718 7f4ed7fff700 1 -- 172.21.2.101:0/4070550860 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4ed8005d60 con 0x7f4ee00b8d70 2021-02-10T11:13:30.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.718 7f4ee553b700 1 -- 172.21.2.101:0/4070550860 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee00c3320 con 0x7f4ee00b8d70 2021-02-10T11:13:30.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.718 7f4ee553b700 1 -- 172.21.2.101:0/4070550860 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee00c38b0 con 0x7f4ee00b8d70 2021-02-10T11:13:30.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.718 7f4ed7fff700 1 -- 172.21.2.101:0/4070550860 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4ed8009450 con 0x7f4ee00b8d70 2021-02-10T11:13:30.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.719 7f4ed7fff700 1 --2- 172.21.2.101:0/4070550860 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4ec40186c0 0x7f4ec401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:30.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.719 7f4ed7fff700 1 -- 172.21.2.101:0/4070550860 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4ed8032940 con 0x7f4ee00b8d70 2021-02-10T11:13:30.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.719 7f4edf2bf700 1 --2- 172.21.2.101:0/4070550860 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4ec40186c0 0x7f4ec401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:30.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.719 7f4edf2bf700 1 --2- 172.21.2.101:0/4070550860 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4ec40186c0 0x7f4ec401abc0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:30.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.751 7f4ee553b700 1 -- 172.21.2.101:0/4070550860 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee009f760 con 0x7f4ee00b8d70 2021-02-10T11:13:30.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.753 7f4ed7fff700 1 -- 172.21.2.101:0/4070550860 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4ed8000f90 con 0x7f4ee00b8d70 2021-02-10T11:13:30.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.890 7f4ee553b700 1 -- 172.21.2.101:0/4070550860 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f4ee009f760 con 0x7f4ee00b8d70 2021-02-10T11:13:30.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.891 7f4ed7fff700 1 -- 172.21.2.101:0/4070550860 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4ed8009450 con 0x7f4ee00b8d70 2021-02-10T11:13:30.895 INFO:teuthology.orchestra.run.gibba001.stdout:55834574857 2021-02-10T11:13:30.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.893 7f4ee553b700 1 -- 172.21.2.101:0/4070550860 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4ec40186c0 msgr2=0x7f4ec401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:30.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.893 7f4ee553b700 1 --2- 172.21.2.101:0/4070550860 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4ec40186c0 0x7f4ec401abc0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:30.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.893 7f4ee553b700 1 -- 172.21.2.101:0/4070550860 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ee00b8d70 msgr2=0x7f4ee009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:30.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.893 7f4ee553b700 1 --2- 172.21.2.101:0/4070550860 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ee00b8d70 0x7f4ee009ed00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f4ed800b8e0 tx=0x7f4ed800bc80).stop 2021-02-10T11:13:30.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.893 7f4ee553b700 1 -- 172.21.2.101:0/4070550860 shutdown_connections 2021-02-10T11:13:30.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.893 7f4ee553b700 1 --2- 172.21.2.101:0/4070550860 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ee00b8d70 0x7f4ee009ed00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:30.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.893 7f4ee553b700 1 --2- 172.21.2.101:0/4070550860 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ee01c6890 0x7f4ee00c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:30.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.893 7f4ee553b700 1 --2- 172.21.2.101:0/4070550860 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4ec40186c0 0x7f4ec401abc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:30.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.893 7f4ee553b700 1 --2- 172.21.2.101:0/4070550860 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ee009fb20 0x7f4ee00c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:30.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.893 7f4ee553b700 1 -- 172.21.2.101:0/4070550860 >> 172.21.2.101:0/4070550860 conn(0x7f4ee00bc2f0 msgr2=0x7f4ee00afea0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:30.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.893 7f4ee553b700 1 -- 172.21.2.101:0/4070550860 shutdown_connections 2021-02-10T11:13:30.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:30.893 7f4ee553b700 1 -- 172.21.2.101:0/4070550860 wait complete. 2021-02-10T11:13:30.905 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574857 for osd.6 2021-02-10T11:13:30.905 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd last-stat-seq osd.7 2021-02-10T11:13:31.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.310 7f2ecc769700 1 Processor -- start 2021-02-10T11:13:31.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.310 7f2ecc769700 1 -- start start 2021-02-10T11:13:31.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.311 7f2ecc769700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2ec407e6a0 0x7f2ec40d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:31.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.311 7f2ecc769700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec407ed80 con 0x7f2ec40e5070 2021-02-10T11:13:31.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.311 7f2ecc769700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec407eaa0 con 0x7f2ec40e5470 2021-02-10T11:13:31.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.311 7f2ecc769700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ec41b4080 con 0x7f2ec407e6a0 2021-02-10T11:13:31.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.311 7f2eca585700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f2ec40e5070 0x7f2ec41b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44814/0 (socket says 172.21.2.101:44814) 2021-02-10T11:13:31.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.311 7f2eca585700 1 -- 172.21.2.101:0/1269476325 learned_addr learned my addr 172.21.2.101:0/1269476325 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:31.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.312 7f2ecad86700 1 --2- 172.21.2.101:0/1269476325 >> v2:172.21.2.108:3300/0 conn(0x7f2ec407e6a0 0x7f2ec40d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:31.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.312 7f2ec9583700 1 -- 172.21.2.101:0/1269476325 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1359752275 0 0) 0x7f2ec0001180 con 0x7f2ec40e5070 2021-02-10T11:13:31.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.312 7f2ec9583700 1 -- 172.21.2.101:0/1269476325 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2eb0002fe0 con 0x7f2ec40e5070 2021-02-10T11:13:31.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.312 7f2ec9583700 1 -- 172.21.2.101:0/1269476325 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3393800941 0 0) 0x7f2ec0001180 con 0x7f2ec40e5070 2021-02-10T11:13:31.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.312 7f2ec9583700 1 -- 172.21.2.101:0/1269476325 >> v2:172.21.2.108:3300/0 conn(0x7f2ec407e6a0 msgr2=0x7f2ec40d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:31.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.312 7f2ec9583700 1 --2- 172.21.2.101:0/1269476325 >> v2:172.21.2.108:3300/0 conn(0x7f2ec407e6a0 0x7f2ec40d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:31.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.312 7f2ec9583700 1 -- 172.21.2.101:0/1269476325 >> v1:172.21.2.108:6789/0 conn(0x7f2ec40e5470 legacy=0x7f2ec407f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:31.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.312 7f2ec9583700 1 -- 172.21.2.101:0/1269476325 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec40d1db0 con 0x7f2ec40e5070 2021-02-10T11:13:31.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.312 7f2ec9583700 1 -- 172.21.2.101:0/1269476325 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f2ec0001680 con 0x7f2ec40e5070 2021-02-10T11:13:31.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.312 7f2ec9583700 1 -- 172.21.2.101:0/1269476325 >> v1:172.21.2.101:6789/0 conn(0x7f2ec40e5070 legacy=0x7f2ec41b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:31.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.312 7f2ec9583700 1 --2- 172.21.2.101:0/1269476325 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2eb0003b10 0x7f2eb0005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:31.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.312 7f2ec9583700 1 --2- 172.21.2.101:0/1269476325 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2eb0006790 0x7f2eb0008cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:31.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.312 7f2ec9583700 1 --2- 172.21.2.101:0/1269476325 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2eb00095b0 0x7f2eb000bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:31.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.312 7f2ec9583700 1 -- 172.21.2.101:0/1269476325 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb000c300 con 0x7f2eb0006790 2021-02-10T11:13:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.312 7f2ec9583700 1 -- 172.21.2.101:0/1269476325 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb000c520 con 0x7f2eb0003b10 2021-02-10T11:13:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.312 7f2ec9583700 1 -- 172.21.2.101:0/1269476325 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb0050750 con 0x7f2eb00095b0 2021-02-10T11:13:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.312 7f2eca585700 1 --2- 172.21.2.101:0/1269476325 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2eb0003b10 0x7f2eb0005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.312 7f2ec9d84700 1 --2- 172.21.2.101:0/1269476325 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2eb0006790 0x7f2eb0008cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.312 7f2ecad86700 1 --2- 172.21.2.101:0/1269476325 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2eb00095b0 0x7f2eb000bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.313 7f2eca585700 1 -- 172.21.2.101:0/1269476325 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2eb00095b0 msgr2=0x7f2eb000bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.313 7f2eca585700 1 --2- 172.21.2.101:0/1269476325 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2eb00095b0 0x7f2eb000bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.313 7f2eca585700 1 -- 172.21.2.101:0/1269476325 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2eb0006790 msgr2=0x7f2eb0008cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.313 7f2eca585700 1 --2- 172.21.2.101:0/1269476325 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2eb0006790 0x7f2eb0008cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.313 7f2ecad86700 1 -- 172.21.2.101:0/1269476325 reap_dead start 2021-02-10T11:13:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.313 7f2eca585700 1 -- 172.21.2.101:0/1269476325 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb00509b0 con 0x7f2eb0003b10 2021-02-10T11:13:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.313 7f2ec9d84700 1 --2- 172.21.2.101:0/1269476325 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2eb0006790 0x7f2eb0008cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:13:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.313 7f2eca585700 1 --2- 172.21.2.101:0/1269476325 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2eb0003b10 0x7f2eb0005fa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f2ec0001880 tx=0x7f2ec0007cc0).ready entity=mon.1 client_cookie=b24a7c892744deac server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:31.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.313 7f2ec9583700 1 -- 172.21.2.101:0/1269476325 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2ec0000c90 con 0x7f2eb0003b10 2021-02-10T11:13:31.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.313 7f2ec9583700 1 -- 172.21.2.101:0/1269476325 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec00070c0 con 0x7f2eb0003b10 2021-02-10T11:13:31.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.313 7f2ec9583700 1 -- 172.21.2.101:0/1269476325 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2ec000a710 con 0x7f2eb0003b10 2021-02-10T11:13:31.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.313 7f2ecc769700 1 -- 172.21.2.101:0/1269476325 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2eb0003b10 msgr2=0x7f2eb0005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:31.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.313 7f2ecc769700 1 --2- 172.21.2.101:0/1269476325 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2eb0003b10 0x7f2eb0005fa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f2ec0001880 tx=0x7f2ec0007cc0).stop 2021-02-10T11:13:31.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.314 7f2ecc769700 1 -- 172.21.2.101:0/1269476325 shutdown_connections 2021-02-10T11:13:31.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.314 7f2ecc769700 1 --2- 172.21.2.101:0/1269476325 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2eb0003b10 0x7f2eb0005fa0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:31.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.314 7f2ecc769700 1 -- 172.21.2.101:0/1269476325 >> 172.21.2.101:0/1269476325 conn(0x7f2ec40ce690 msgr2=0x7f2ec40e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:31.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.314 7f2ecc769700 1 -- 172.21.2.101:0/1269476325 shutdown_connections 2021-02-10T11:13:31.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.314 7f2ecc769700 1 -- 172.21.2.101:0/1269476325 wait complete. 2021-02-10T11:13:31.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.329 7f2ecc769700 1 Processor -- start 2021-02-10T11:13:31.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.329 7f2ecc769700 1 -- start start 2021-02-10T11:13:31.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.329 7f2ecc769700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ec419ec20 0x7f2ec419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:31.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.329 7f2ecc769700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ec4091c60 0x7f2ec4092060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:31.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.329 7f2ecc769700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ec40afb70 0x7f2ec40aff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:31.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.329 7f2ecc769700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec4092870 con 0x7f2ec40afb70 2021-02-10T11:13:31.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.329 7f2ecc769700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec40b0590 con 0x7f2ec4091c60 2021-02-10T11:13:31.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.329 7f2ecc769700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec40b0770 con 0x7f2ec419ec20 2021-02-10T11:13:31.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.329 7f2ecad86700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ec40afb70 0x7f2ec40aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:31.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.329 7f2ecad86700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ec40afb70 0x7f2ec40aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44334/0 (socket says 172.21.2.101:44334) 2021-02-10T11:13:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.329 7f2ecad86700 1 -- 172.21.2.101:0/2226854188 learned_addr learned my addr 172.21.2.101:0/2226854188 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.330 7f2eca585700 1 --2- 172.21.2.101:0/2226854188 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ec419ec20 0x7f2ec419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.330 7f2ecad86700 1 -- 172.21.2.101:0/2226854188 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ec419ec20 msgr2=0x7f2ec419f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.330 7f2ecad86700 1 --2- 172.21.2.101:0/2226854188 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ec419ec20 0x7f2ec419f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.330 7f2ecad86700 1 -- 172.21.2.101:0/2226854188 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ec4091c60 msgr2=0x7f2ec4092060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.330 7f2ecad86700 1 --2- 172.21.2.101:0/2226854188 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ec4091c60 0x7f2ec4092060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.330 7f2ecad86700 1 -- 172.21.2.101:0/2226854188 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec40b0990 con 0x7f2ec40afb70 2021-02-10T11:13:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.330 7f2ecad86700 1 --2- 172.21.2.101:0/2226854188 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ec40afb70 0x7f2ec40aff70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f2ebc00b8e0 tx=0x7f2ebc00bc80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.330 7f2eb6ffd700 1 -- 172.21.2.101:0/2226854188 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2ebc00bdc0 con 0x7f2ec40afb70 2021-02-10T11:13:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.330 7f2eb6ffd700 1 -- 172.21.2.101:0/2226854188 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ebc00c2f0 con 0x7f2ec40afb70 2021-02-10T11:13:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.330 7f2eb6ffd700 1 -- 172.21.2.101:0/2226854188 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2ebc00c810 con 0x7f2ec40afb70 2021-02-10T11:13:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.330 7f2ecc769700 1 -- 172.21.2.101:0/2226854188 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec40b0990 con 0x7f2ec40afb70 2021-02-10T11:13:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.330 7f2ecc769700 1 -- 172.21.2.101:0/2226854188 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec40b0990 con 0x7f2ec40afb70 2021-02-10T11:13:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.331 7f2eb6ffd700 1 -- 172.21.2.101:0/2226854188 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2ebc000df0 con 0x7f2ec40afb70 2021-02-10T11:13:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.331 7f2eb6ffd700 1 --2- 172.21.2.101:0/2226854188 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2eac0186c0 0x7f2eac01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.331 7f2eb6ffd700 1 -- 172.21.2.101:0/2226854188 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2ebc032940 con 0x7f2ec40afb70 2021-02-10T11:13:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.331 7f2eca585700 1 --2- 172.21.2.101:0/2226854188 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2eac0186c0 0x7f2eac01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.331 7f2eca585700 1 --2- 172.21.2.101:0/2226854188 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2eac0186c0 0x7f2eac01abc0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:31.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.361 7f2ecc769700 1 -- 172.21.2.101:0/2226854188 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec4084a90 con 0x7f2ec40afb70 2021-02-10T11:13:31.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.364 7f2eb6ffd700 1 -- 172.21.2.101:0/2226854188 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2ebc00bdc0 con 0x7f2ec40afb70 2021-02-10T11:13:31.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.504 7f2ecc769700 1 -- 172.21.2.101:0/2226854188 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2ec407ed80 con 0x7f2ec40afb70 2021-02-10T11:13:31.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.504 7f2eb6ffd700 1 -- 172.21.2.101:0/2226854188 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2ebc000df0 con 0x7f2ec40afb70 2021-02-10T11:13:31.509 INFO:teuthology.orchestra.run.gibba001.stdout:55834574857 2021-02-10T11:13:31.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.506 7f2ecc769700 1 -- 172.21.2.101:0/2226854188 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2eac0186c0 msgr2=0x7f2eac01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:31.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.506 7f2ecc769700 1 --2- 172.21.2.101:0/2226854188 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2eac0186c0 0x7f2eac01abc0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:31.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.506 7f2ecc769700 1 -- 172.21.2.101:0/2226854188 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ec40afb70 msgr2=0x7f2ec40aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:31.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.506 7f2ecc769700 1 --2- 172.21.2.101:0/2226854188 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ec40afb70 0x7f2ec40aff70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f2ebc00b8e0 tx=0x7f2ebc00bc80).stop 2021-02-10T11:13:31.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.507 7f2ecc769700 1 -- 172.21.2.101:0/2226854188 shutdown_connections 2021-02-10T11:13:31.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.507 7f2ecc769700 1 --2- 172.21.2.101:0/2226854188 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ec40afb70 0x7f2ec40aff70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:31.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.507 7f2ecc769700 1 --2- 172.21.2.101:0/2226854188 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ec4091c60 0x7f2ec4092060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:31.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.507 7f2ecc769700 1 --2- 172.21.2.101:0/2226854188 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2eac0186c0 0x7f2eac01abc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:31.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.507 7f2ecc769700 1 --2- 172.21.2.101:0/2226854188 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ec419ec20 0x7f2ec419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:31.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.507 7f2ecc769700 1 -- 172.21.2.101:0/2226854188 >> 172.21.2.101:0/2226854188 conn(0x7f2ec409be90 msgr2=0x7f2ec40a6500 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:31.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.507 7f2ecc769700 1 -- 172.21.2.101:0/2226854188 shutdown_connections 2021-02-10T11:13:31.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.507 7f2ecc769700 1 -- 172.21.2.101:0/2226854188 wait complete. 2021-02-10T11:13:31.519 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574857 for osd.7 2021-02-10T11:13:31.519 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2021-02-10T11:13:31.519 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph pg dump --format=json 2021-02-10T11:13:31.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.925 7f15fc21c700 1 Processor -- start 2021-02-10T11:13:31.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.925 7f15fc21c700 1 -- start start 2021-02-10T11:13:31.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.926 7f15fc21c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f15f40d17f0 0x7f15f41b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:31.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.926 7f15fc21c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f15f40d1bf0 0x7f15f41b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:31.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.926 7f15fc21c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15f41b3e20 0x7f15f40ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:31.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.926 7f15fc21c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f40e5eb0 con 0x7f15f40d1bf0 2021-02-10T11:13:31.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.926 7f15fc21c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f40f0b20 con 0x7f15f40d17f0 2021-02-10T11:13:31.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.926 7f15fc21c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f40cf510 con 0x7f15f41b3e20 2021-02-10T11:13:31.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.926 7f15f9837700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f15f40d1bf0 0x7f15f41b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:31.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.926 7f15f9837700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f15f40d1bf0 0x7f15f41b3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44340/0 (socket says 172.21.2.101:44340) 2021-02-10T11:13:31.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.926 7f15f9837700 1 -- 172.21.2.101:0/1744864537 learned_addr learned my addr 172.21.2.101:0/1744864537 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:31.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.927 7f15f9837700 1 -- 172.21.2.101:0/1744864537 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15f41b3e20 msgr2=0x7f15f40ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:31.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.927 7f15f9837700 1 --2- 172.21.2.101:0/1744864537 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15f41b3e20 0x7f15f40ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:31.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.927 7f15f9837700 1 -- 172.21.2.101:0/1744864537 >> v2:172.21.2.108:3300/0 conn(0x7f15f40d17f0 msgr2=0x7f15f41b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:31.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.927 7f15f9837700 1 --2- 172.21.2.101:0/1744864537 >> v2:172.21.2.108:3300/0 conn(0x7f15f40d17f0 0x7f15f41b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:31.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.927 7f15f9837700 1 -- 172.21.2.101:0/1744864537 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f40cf6f0 con 0x7f15f40d1bf0 2021-02-10T11:13:31.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.927 7f15f9837700 1 --2- 172.21.2.101:0/1744864537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15f40d1bf0 0x7f15f41b3800 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f15f0006700 tx=0x7f15f0009b80).ready entity=mon.0 client_cookie=cfcb12f77b6b87ad server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:31.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.927 7f15f9036700 1 -- 172.21.2.101:0/1744864537 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f15f0000c90 con 0x7f15f40d1bf0 2021-02-10T11:13:31.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.927 7f15f9036700 1 -- 172.21.2.101:0/1744864537 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f000c0c0 con 0x7f15f40d1bf0 2021-02-10T11:13:31.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.927 7f15f9036700 1 -- 172.21.2.101:0/1744864537 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f15f000c5e0 con 0x7f15f40d1bf0 2021-02-10T11:13:31.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.927 7f15fc21c700 1 -- 172.21.2.101:0/1744864537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15f40d1bf0 msgr2=0x7f15f41b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:31.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.927 7f15fc21c700 1 --2- 172.21.2.101:0/1744864537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15f40d1bf0 0x7f15f41b3800 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f15f0006700 tx=0x7f15f0009b80).stop 2021-02-10T11:13:31.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.928 7f15fc21c700 1 -- 172.21.2.101:0/1744864537 shutdown_connections 2021-02-10T11:13:31.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.928 7f15fc21c700 1 --2- 172.21.2.101:0/1744864537 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15f41b3e20 0x7f15f40ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:31.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.928 7f15fc21c700 1 --2- 172.21.2.101:0/1744864537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15f40d1bf0 0x7f15f41b3800 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:31.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.928 7f15fc21c700 1 --2- 172.21.2.101:0/1744864537 >> v2:172.21.2.108:3300/0 conn(0x7f15f40d17f0 0x7f15f41b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:31.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.928 7f15fc21c700 1 -- 172.21.2.101:0/1744864537 >> 172.21.2.101:0/1744864537 conn(0x7f15f40fde40 msgr2=0x7f15f40ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:31.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.928 7f15fc21c700 1 -- 172.21.2.101:0/1744864537 shutdown_connections 2021-02-10T11:13:31.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.928 7f15fc21c700 1 -- 172.21.2.101:0/1744864537 wait complete. 2021-02-10T11:13:31.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.943 7f15fc21c700 1 Processor -- start 2021-02-10T11:13:31.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.943 7f15fc21c700 1 -- start start 2021-02-10T11:13:31.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.943 7f15fc21c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15f40ae810 0x7f15f418ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:31.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.943 7f15fc21c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15f419e390 0x7f15f419e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:31.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.943 7f15fc21c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15f4197af0 0x7f15f4197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:31.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.943 7f15fc21c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f419de10 con 0x7f15f4197af0 2021-02-10T11:13:31.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.943 7f15fc21c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f418d4d0 con 0x7f15f419e390 2021-02-10T11:13:31.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.943 7f15fc21c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f4198510 con 0x7f15f40ae810 2021-02-10T11:13:31.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.944 7f15fa839700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15f4197af0 0x7f15f4197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:31.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.944 7f15fa839700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15f4197af0 0x7f15f4197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44348/0 (socket says 172.21.2.101:44348) 2021-02-10T11:13:31.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.944 7f15fa839700 1 -- 172.21.2.101:0/2384821943 learned_addr learned my addr 172.21.2.101:0/2384821943 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:31.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.944 7f15f9837700 1 --2- 172.21.2.101:0/2384821943 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15f419e390 0x7f15f419e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:31.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.944 7f15fa839700 1 -- 172.21.2.101:0/2384821943 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15f40ae810 msgr2=0x7f15f418ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:31.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.944 7f15fa038700 1 --2- 172.21.2.101:0/2384821943 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15f40ae810 0x7f15f418ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:31.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.944 7f15fa839700 1 --2- 172.21.2.101:0/2384821943 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15f40ae810 0x7f15f418ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:31.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.944 7f15fa839700 1 -- 172.21.2.101:0/2384821943 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15f419e390 msgr2=0x7f15f419e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:31.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.944 7f15fa839700 1 --2- 172.21.2.101:0/2384821943 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15f419e390 0x7f15f419e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:31.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.944 7f15fa839700 1 -- 172.21.2.101:0/2384821943 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f419cab0 con 0x7f15f4197af0 2021-02-10T11:13:31.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.944 7f15f9837700 1 --2- 172.21.2.101:0/2384821943 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15f419e390 0x7f15f419e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:13:31.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.944 7f15fa839700 1 --2- 172.21.2.101:0/2384821943 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15f4197af0 0x7f15f4197ef0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f15ec009980 tx=0x7f15ec00bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:31.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.944 7f15e6ffd700 1 -- 172.21.2.101:0/2384821943 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f15ec003020 con 0x7f15f4197af0 2021-02-10T11:13:31.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.944 7f15e6ffd700 1 -- 172.21.2.101:0/2384821943 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ec005a80 con 0x7f15f4197af0 2021-02-10T11:13:31.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.944 7f15e6ffd700 1 -- 172.21.2.101:0/2384821943 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f15ec006030 con 0x7f15f4197af0 2021-02-10T11:13:31.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.944 7f15fc21c700 1 -- 172.21.2.101:0/2384821943 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f419cab0 con 0x7f15f4197af0 2021-02-10T11:13:31.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.945 7f15fc21c700 1 -- 172.21.2.101:0/2384821943 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f419d040 con 0x7f15f4197af0 2021-02-10T11:13:31.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.945 7f15e6ffd700 1 -- 172.21.2.101:0/2384821943 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f15ec006030 con 0x7f15f4197af0 2021-02-10T11:13:31.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.945 7f15e6ffd700 1 --2- 172.21.2.101:0/2384821943 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15dc0186c0 0x7f15dc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:31.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.945 7f15e6ffd700 1 -- 172.21.2.101:0/2384821943 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f15ec0328e0 con 0x7f15f4197af0 2021-02-10T11:13:31.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.945 7f15fa038700 1 --2- 172.21.2.101:0/2384821943 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15dc0186c0 0x7f15dc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:31.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.946 7f15fa038700 1 --2- 172.21.2.101:0/2384821943 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15dc0186c0 0x7f15dc01abc0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:31.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.975 7f15fc21c700 1 -- 172.21.2.101:0/2384821943 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f40f0b20 con 0x7f15f4197af0 2021-02-10T11:13:31.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:31.978 7f15e6ffd700 1 -- 172.21.2.101:0/2384821943 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f15ec005c60 con 0x7f15f4197af0 2021-02-10T11:13:32.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.112 7f15fc21c700 1 -- 172.21.2.101:0/2384821943 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "pg dump", "target": ["mgr", ""], "format": "json"}) v1 -- 0x7f15f419d040 con 0x7f15dc0186c0 2021-02-10T11:13:32.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.114 7f15e6ffd700 1 -- 172.21.2.101:0/2384821943 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 dumped all) v1 ==== 18+0+77203 (crc 0 0 0) 0x7f15e8000940 con 0x7f15dc0186c0 2021-02-10T11:13:32.120 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:13:32.121 INFO:teuthology.orchestra.run.gibba001.stderr:dumped all 2021-02-10T11:13:32.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.117 7f15fc21c700 1 -- 172.21.2.101:0/2384821943 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15dc0186c0 msgr2=0x7f15dc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:32.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.117 7f15fc21c700 1 --2- 172.21.2.101:0/2384821943 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15dc0186c0 0x7f15dc01abc0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:32.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.117 7f15fc21c700 1 -- 172.21.2.101:0/2384821943 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15f4197af0 msgr2=0x7f15f4197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:32.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.117 7f15fc21c700 1 --2- 172.21.2.101:0/2384821943 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15f4197af0 0x7f15f4197ef0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f15ec009980 tx=0x7f15ec00bcd0).stop 2021-02-10T11:13:32.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.117 7f15fc21c700 1 -- 172.21.2.101:0/2384821943 shutdown_connections 2021-02-10T11:13:32.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.117 7f15fc21c700 1 --2- 172.21.2.101:0/2384821943 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15f4197af0 0x7f15f4197ef0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:32.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.117 7f15fc21c700 1 --2- 172.21.2.101:0/2384821943 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15f419e390 0x7f15f419e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:32.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.117 7f15fc21c700 1 --2- 172.21.2.101:0/2384821943 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15dc0186c0 0x7f15dc01abc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:32.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.117 7f15fc21c700 1 --2- 172.21.2.101:0/2384821943 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15f40ae810 0x7f15f418ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:32.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.117 7f15fc21c700 1 -- 172.21.2.101:0/2384821943 >> 172.21.2.101:0/2384821943 conn(0x7f15f41a3780 msgr2=0x7f15f4198c50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:32.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.117 7f15fc21c700 1 -- 172.21.2.101:0/2384821943 shutdown_connections 2021-02-10T11:13:32.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.118 7f15fc21c700 1 -- 172.21.2.101:0/2384821943 wait complete. 2021-02-10T11:13:32.129 INFO:teuthology.orchestra.run.gibba001.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2021-02-10 11:13:30.137552","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2286,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"kb":754974720,"kb_used":8390336,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584384,"statfs":{"total":773094113280,"available":764502409216,"internally_reserved":8589934592,"allocated":1703936,"data_stored":554356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":760,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001116"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"13","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.350023","last_change":"2021-02-10 11:13:09.495109","last_active":"2021-02-10 11:13:14.350023","last_peered":"2021-02-10 11:13:14.350023","last_clean":"2021-02-10 11:13:14.350023","last_became_active":"2021-02-10 11:13:09.494906","last_became_peered":"2021-02-10 11:13:09.494906","last_unstale":"2021-02-10 11:13:14.350023","last_undegraded":"2021-02-10 11:13:14.350023","last_fullsized":"2021-02-10 11:13:14.350023","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"20'8","reported_seq":"18","reported_epoch":"20","state":"active+clean","last_fresh":"2021-02-10 11:13:22.554526","last_change":"2021-02-10 11:13:13.509995","last_active":"2021-02-10 11:13:22.554526","last_peered":"2021-02-10 11:13:22.554526","last_clean":"2021-02-10 11:13:22.554526","last_became_active":"2021-02-10 11:13:13.509815","last_became_peered":"2021-02-10 11:13:13.509815","last_unstale":"2021-02-10 11:13:22.554526","last_undegraded":"2021-02-10 11:13:22.554526","last_fullsized":"2021-02-10 11:13:22.554526","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:12.494038","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:12.494038","last_clean_scrub_stamp":"2021-02-10 11:13:12.494038","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1526,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"9","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.161251","last_change":"2021-02-10 11:13:14.160844","last_active":"2021-02-10 11:13:14.161251","last_peered":"2021-02-10 11:13:14.161251","last_clean":"2021-02-10 11:13:14.161251","last_became_active":"2021-02-10 11:13:14.160633","last_became_peered":"2021-02-10 11:13:14.160633","last_unstale":"2021-02-10 11:13:14.161251","last_undegraded":"2021-02-10 11:13:14.161251","last_fullsized":"2021-02-10 11:13:14.161251","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:13.501437","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:13.501437","last_clean_scrub_stamp":"2021-02-10 11:13:13.501437","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"13","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.350019","last_change":"2021-02-10 11:13:09.495461","last_active":"2021-02-10 11:13:14.350019","last_peered":"2021-02-10 11:13:14.350019","last_clean":"2021-02-10 11:13:14.350019","last_became_active":"2021-02-10 11:13:09.495226","last_became_peered":"2021-02-10 11:13:09.495226","last_unstale":"2021-02-10 11:13:14.350019","last_undegraded":"2021-02-10 11:13:14.350019","last_fullsized":"2021-02-10 11:13:14.350019","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"20'6","reported_seq":"16","reported_epoch":"20","state":"active+clean","last_fresh":"2021-02-10 11:13:22.561499","last_change":"2021-02-10 11:13:13.516702","last_active":"2021-02-10 11:13:22.561499","last_peered":"2021-02-10 11:13:22.561499","last_clean":"2021-02-10 11:13:22.561499","last_became_active":"2021-02-10 11:13:13.516524","last_became_peered":"2021-02-10 11:13:13.516524","last_unstale":"2021-02-10 11:13:22.561499","last_undegraded":"2021-02-10 11:13:22.561499","last_fullsized":"2021-02-10 11:13:22.561499","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:12.494038","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:12.494038","last_clean_scrub_stamp":"2021-02-10 11:13:12.494038","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"9","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.155525","last_change":"2021-02-10 11:13:14.154896","last_active":"2021-02-10 11:13:14.155525","last_peered":"2021-02-10 11:13:14.155525","last_clean":"2021-02-10 11:13:14.155525","last_became_active":"2021-02-10 11:13:14.154653","last_became_peered":"2021-02-10 11:13:14.154653","last_unstale":"2021-02-10 11:13:14.155525","last_undegraded":"2021-02-10 11:13:14.155525","last_fullsized":"2021-02-10 11:13:14.155525","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:13.501437","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:13.501437","last_clean_scrub_stamp":"2021-02-10 11:13:13.501437","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":"9","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.155920","last_change":"2021-02-10 11:13:14.155520","last_active":"2021-02-10 11:13:14.155920","last_peered":"2021-02-10 11:13:14.155920","last_clean":"2021-02-10 11:13:14.155920","last_became_active":"2021-02-10 11:13:14.155328","last_became_peered":"2021-02-10 11:13:14.155328","last_unstale":"2021-02-10 11:13:14.155920","last_undegraded":"2021-02-10 11:13:14.155920","last_fullsized":"2021-02-10 11:13:14.155920","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:13.501437","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:13.501437","last_clean_scrub_stamp":"2021-02-10 11:13:13.501437","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"13","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.159455","last_change":"2021-02-10 11:13:09.495287","last_active":"2021-02-10 11:13:14.159455","last_peered":"2021-02-10 11:13:14.159455","last_clean":"2021-02-10 11:13:14.159455","last_became_active":"2021-02-10 11:13:09.495041","last_became_peered":"2021-02-10 11:13:09.495041","last_unstale":"2021-02-10 11:13:14.159455","last_undegraded":"2021-02-10 11:13:14.159455","last_fullsized":"2021-02-10 11:13:14.159455","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"20'4","reported_seq":"14","reported_epoch":"20","state":"active+clean","last_fresh":"2021-02-10 11:13:22.553337","last_change":"2021-02-10 11:13:13.509418","last_active":"2021-02-10 11:13:22.553337","last_peered":"2021-02-10 11:13:22.553337","last_clean":"2021-02-10 11:13:22.553337","last_became_active":"2021-02-10 11:13:13.509192","last_became_peered":"2021-02-10 11:13:13.509192","last_unstale":"2021-02-10 11:13:22.553337","last_undegraded":"2021-02-10 11:13:22.553337","last_fullsized":"2021-02-10 11:13:22.553337","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:12.494038","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:12.494038","last_clean_scrub_stamp":"2021-02-10 11:13:12.494038","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"9","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.154409","last_change":"2021-02-10 11:13:14.153836","last_active":"2021-02-10 11:13:14.154409","last_peered":"2021-02-10 11:13:14.154409","last_clean":"2021-02-10 11:13:14.154409","last_became_active":"2021-02-10 11:13:14.153619","last_became_peered":"2021-02-10 11:13:14.153619","last_unstale":"2021-02-10 11:13:14.154409","last_undegraded":"2021-02-10 11:13:14.154409","last_fullsized":"2021-02-10 11:13:14.154409","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:13.501437","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:13.501437","last_clean_scrub_stamp":"2021-02-10 11:13:13.501437","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"12","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.152601","last_change":"2021-02-10 11:13:09.487892","last_active":"2021-02-10 11:13:14.152601","last_peered":"2021-02-10 11:13:14.152601","last_clean":"2021-02-10 11:13:14.152601","last_became_active":"2021-02-10 11:13:09.487627","last_became_peered":"2021-02-10 11:13:09.487627","last_unstale":"2021-02-10 11:13:14.152601","last_undegraded":"2021-02-10 11:13:14.152601","last_fullsized":"2021-02-10 11:13:14.152601","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'7","reported_seq":"17","reported_epoch":"20","state":"active+clean","last_fresh":"2021-02-10 11:13:22.560718","last_change":"2021-02-10 11:13:13.515774","last_active":"2021-02-10 11:13:22.560718","last_peered":"2021-02-10 11:13:22.560718","last_clean":"2021-02-10 11:13:22.560718","last_became_active":"2021-02-10 11:13:13.515458","last_became_peered":"2021-02-10 11:13:13.515458","last_unstale":"2021-02-10 11:13:22.560718","last_undegraded":"2021-02-10 11:13:22.560718","last_fullsized":"2021-02-10 11:13:22.560718","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:12.494038","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:12.494038","last_clean_scrub_stamp":"2021-02-10 11:13:12.494038","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":566,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"13","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.160750","last_change":"2021-02-10 11:13:09.495589","last_active":"2021-02-10 11:13:14.160750","last_peered":"2021-02-10 11:13:14.160750","last_clean":"2021-02-10 11:13:14.160750","last_became_active":"2021-02-10 11:13:09.495399","last_became_peered":"2021-02-10 11:13:09.495399","last_unstale":"2021-02-10 11:13:14.160750","last_undegraded":"2021-02-10 11:13:14.160750","last_fullsized":"2021-02-10 11:13:14.160750","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'5","reported_seq":"15","reported_epoch":"20","state":"active+clean","last_fresh":"2021-02-10 11:13:22.559807","last_change":"2021-02-10 11:13:13.515340","last_active":"2021-02-10 11:13:22.559807","last_peered":"2021-02-10 11:13:22.559807","last_clean":"2021-02-10 11:13:22.559807","last_became_active":"2021-02-10 11:13:13.515156","last_became_peered":"2021-02-10 11:13:13.515156","last_unstale":"2021-02-10 11:13:22.559807","last_undegraded":"2021-02-10 11:13:22.559807","last_fullsized":"2021-02-10 11:13:22.559807","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:12.494038","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:12.494038","last_clean_scrub_stamp":"2021-02-10 11:13:12.494038","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"9","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.155464","last_change":"2021-02-10 11:13:14.155032","last_active":"2021-02-10 11:13:14.155464","last_peered":"2021-02-10 11:13:14.155464","last_clean":"2021-02-10 11:13:14.155464","last_became_active":"2021-02-10 11:13:14.154776","last_became_peered":"2021-02-10 11:13:14.154776","last_unstale":"2021-02-10 11:13:14.155464","last_undegraded":"2021-02-10 11:13:14.155464","last_fullsized":"2021-02-10 11:13:14.155464","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:13.501437","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:13.501437","last_clean_scrub_stamp":"2021-02-10 11:13:13.501437","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"13","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.159207","last_change":"2021-02-10 11:13:09.495570","last_active":"2021-02-10 11:13:14.159207","last_peered":"2021-02-10 11:13:14.159207","last_clean":"2021-02-10 11:13:14.159207","last_became_active":"2021-02-10 11:13:09.495383","last_became_peered":"2021-02-10 11:13:09.495383","last_unstale":"2021-02-10 11:13:14.159207","last_undegraded":"2021-02-10 11:13:14.159207","last_fullsized":"2021-02-10 11:13:14.159207","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"20'1","reported_seq":"11","reported_epoch":"20","state":"active+clean","last_fresh":"2021-02-10 11:13:15.160893","last_change":"2021-02-10 11:13:13.515945","last_active":"2021-02-10 11:13:15.160893","last_peered":"2021-02-10 11:13:15.160893","last_clean":"2021-02-10 11:13:15.160893","last_became_active":"2021-02-10 11:13:13.515644","last_became_peered":"2021-02-10 11:13:13.515644","last_unstale":"2021-02-10 11:13:15.160893","last_undegraded":"2021-02-10 11:13:15.160893","last_fullsized":"2021-02-10 11:13:15.160893","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:12.494038","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:12.494038","last_clean_scrub_stamp":"2021-02-10 11:13:12.494038","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"9","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.161457","last_change":"2021-02-10 11:13:14.161055","last_active":"2021-02-10 11:13:14.161457","last_peered":"2021-02-10 11:13:14.161457","last_clean":"2021-02-10 11:13:14.161457","last_became_active":"2021-02-10 11:13:14.160843","last_became_peered":"2021-02-10 11:13:14.160843","last_unstale":"2021-02-10 11:13:14.161457","last_undegraded":"2021-02-10 11:13:14.161457","last_fullsized":"2021-02-10 11:13:14.161457","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:13.501437","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:13.501437","last_clean_scrub_stamp":"2021-02-10 11:13:13.501437","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.153763","last_change":"2021-02-10 11:13:13.509094","last_active":"2021-02-10 11:13:14.153763","last_peered":"2021-02-10 11:13:14.153763","last_clean":"2021-02-10 11:13:14.153763","last_became_active":"2021-02-10 11:13:13.508842","last_became_peered":"2021-02-10 11:13:13.508842","last_unstale":"2021-02-10 11:13:14.153763","last_undegraded":"2021-02-10 11:13:14.153763","last_fullsized":"2021-02-10 11:13:14.153763","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:12.494038","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:12.494038","last_clean_scrub_stamp":"2021-02-10 11:13:12.494038","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"13","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.160741","last_change":"2021-02-10 11:13:09.495605","last_active":"2021-02-10 11:13:14.160741","last_peered":"2021-02-10 11:13:14.160741","last_clean":"2021-02-10 11:13:14.160741","last_became_active":"2021-02-10 11:13:09.495261","last_became_peered":"2021-02-10 11:13:09.495261","last_unstale":"2021-02-10 11:13:14.160741","last_undegraded":"2021-02-10 11:13:14.160741","last_fullsized":"2021-02-10 11:13:14.160741","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"9","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.155460","last_change":"2021-02-10 11:13:14.154981","last_active":"2021-02-10 11:13:14.155460","last_peered":"2021-02-10 11:13:14.155460","last_clean":"2021-02-10 11:13:14.155460","last_became_active":"2021-02-10 11:13:14.154718","last_became_peered":"2021-02-10 11:13:14.154718","last_unstale":"2021-02-10 11:13:14.155460","last_undegraded":"2021-02-10 11:13:14.155460","last_fullsized":"2021-02-10 11:13:14.155460","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:13.501437","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:13.501437","last_clean_scrub_stamp":"2021-02-10 11:13:13.501437","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"20'8","reported_seq":"18","reported_epoch":"20","state":"active+clean","last_fresh":"2021-02-10 11:13:22.561363","last_change":"2021-02-10 11:13:13.516117","last_active":"2021-02-10 11:13:22.561363","last_peered":"2021-02-10 11:13:22.561363","last_clean":"2021-02-10 11:13:22.561363","last_became_active":"2021-02-10 11:13:13.515901","last_became_peered":"2021-02-10 11:13:13.515901","last_unstale":"2021-02-10 11:13:22.561363","last_undegraded":"2021-02-10 11:13:22.561363","last_fullsized":"2021-02-10 11:13:22.561363","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:12.494038","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:12.494038","last_clean_scrub_stamp":"2021-02-10 11:13:12.494038","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"13","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.152417","last_change":"2021-02-10 11:13:09.488300","last_active":"2021-02-10 11:13:14.152417","last_peered":"2021-02-10 11:13:14.152417","last_clean":"2021-02-10 11:13:14.152417","last_became_active":"2021-02-10 11:13:09.488122","last_became_peered":"2021-02-10 11:13:09.488122","last_unstale":"2021-02-10 11:13:14.152417","last_undegraded":"2021-02-10 11:13:14.152417","last_fullsized":"2021-02-10 11:13:14.152417","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"9","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.155114","last_change":"2021-02-10 11:13:14.154576","last_active":"2021-02-10 11:13:14.155114","last_peered":"2021-02-10 11:13:14.155114","last_clean":"2021-02-10 11:13:14.155114","last_became_active":"2021-02-10 11:13:14.154349","last_became_peered":"2021-02-10 11:13:14.154349","last_unstale":"2021-02-10 11:13:14.155114","last_undegraded":"2021-02-10 11:13:14.155114","last_fullsized":"2021-02-10 11:13:14.155114","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:13.501437","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:13.501437","last_clean_scrub_stamp":"2021-02-10 11:13:13.501437","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":2286,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048800,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323040,"statfs":{"total":96636764160,"available":95562792960,"internally_reserved":1073741824,"allocated":221184,"data_stored":69449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048792,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323048,"statfs":{"total":96636764160,"available":95562801152,"internally_reserved":1073741824,"allocated":212992,"data_stored":69289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048788,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323052,"statfs":{"total":96636764160,"available":95562805248,"internally_reserved":1073741824,"allocated":208896,"data_stored":68757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048784,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323056,"statfs":{"total":96636764160,"available":95562809344,"internally_reserved":1073741824,"allocated":204800,"data_stored":68723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048804,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323036,"statfs":{"total":96636764160,"available":95562788864,"internally_reserved":1073741824,"allocated":225280,"data_stored":70409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048796,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323044,"statfs":{"total":96636764160,"available":95562797056,"internally_reserved":1073741824,"allocated":217088,"data_stored":70249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048788,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323052,"statfs":{"total":96636764160,"available":95562805248,"internally_reserved":1073741824,"allocated":208896,"data_stored":68757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048784,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323056,"statfs":{"total":96636764160,"available":95562809344,"internally_reserved":1073741824,"allocated":204800,"data_stored":68723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]}]}]}} 2021-02-10T11:13:32.131 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph pg dump --format=json 2021-02-10T11:13:32.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.537 7ffaa95ba700 1 Processor -- start 2021-02-10T11:13:32.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.537 7ffaa95ba700 1 -- start start 2021-02-10T11:13:32.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.538 7ffaa95ba700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ffaa407e6a0 0x7ffaa40d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:32.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.538 7ffaa95ba700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa407ed80 con 0x7ffaa40e5470 2021-02-10T11:13:32.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.538 7ffaa95ba700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa407eaa0 con 0x7ffaa40e5070 2021-02-10T11:13:32.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.538 7ffaa95ba700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa41b4080 con 0x7ffaa407e6a0 2021-02-10T11:13:32.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.538 7ffaa2abe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ffaa40e5470 0x7ffaa407f300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44848/0 (socket says 172.21.2.101:44848) 2021-02-10T11:13:32.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.538 7ffaa2abe700 1 -- 172.21.2.101:0/2695355301 learned_addr learned my addr 172.21.2.101:0/2695355301 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:32.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.538 7ffaa3ac0700 1 --2- 172.21.2.101:0/2695355301 >> v2:172.21.2.101:3300/0 conn(0x7ffaa407e6a0 0x7ffaa40d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:32.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.538 7ffaa3ac0700 1 -- 172.21.2.101:0/2695355301 >> v1:172.21.2.108:6789/0 conn(0x7ffaa40e5070 legacy=0x7ffaa41b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:32.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.538 7ffaa3ac0700 1 -- 172.21.2.101:0/2695355301 >> v1:172.21.2.101:6789/0 conn(0x7ffaa40e5470 legacy=0x7ffaa407f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:32.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.538 7ffaa3ac0700 1 -- 172.21.2.101:0/2695355301 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa40d1db0 con 0x7ffaa407e6a0 2021-02-10T11:13:32.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.539 7ffaa3ac0700 1 --2- 172.21.2.101:0/2695355301 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffaa407e6a0 0x7ffaa40d1790 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7ffa9c006e30 tx=0x7ffa9c009e60).ready entity=mon.0 client_cookie=872ee23e11557621 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:32.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.539 7ffaa22bd700 1 -- 172.21.2.101:0/2695355301 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffa9c006a50 con 0x7ffaa407e6a0 2021-02-10T11:13:32.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.539 7ffaa22bd700 1 -- 172.21.2.101:0/2695355301 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9c00b300 con 0x7ffaa407e6a0 2021-02-10T11:13:32.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.539 7ffaa22bd700 1 -- 172.21.2.101:0/2695355301 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffa9c00b820 con 0x7ffaa407e6a0 2021-02-10T11:13:32.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.539 7ffaa95ba700 1 -- 172.21.2.101:0/2695355301 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffaa407e6a0 msgr2=0x7ffaa40d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:32.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.539 7ffaa95ba700 1 --2- 172.21.2.101:0/2695355301 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffaa407e6a0 0x7ffaa40d1790 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7ffa9c006e30 tx=0x7ffa9c009e60).stop 2021-02-10T11:13:32.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.539 7ffaa95ba700 1 -- 172.21.2.101:0/2695355301 shutdown_connections 2021-02-10T11:13:32.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.539 7ffaa95ba700 1 --2- 172.21.2.101:0/2695355301 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffaa407e6a0 0x7ffaa40d1790 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:32.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.539 7ffaa95ba700 1 -- 172.21.2.101:0/2695355301 >> 172.21.2.101:0/2695355301 conn(0x7ffaa40ce690 msgr2=0x7ffaa40e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:32.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.540 7ffaa95ba700 1 -- 172.21.2.101:0/2695355301 shutdown_connections 2021-02-10T11:13:32.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.540 7ffaa95ba700 1 -- 172.21.2.101:0/2695355301 wait complete. 2021-02-10T11:13:32.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.554 7ffaa95ba700 1 Processor -- start 2021-02-10T11:13:32.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.554 7ffaa95ba700 1 -- start start 2021-02-10T11:13:32.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.555 7ffaa95ba700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffaa4096380 0x7ffaa4090ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:32.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.555 7ffaa95ba700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffaa4096780 0x7ffaa4091100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:32.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.555 7ffaa95ba700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffaa4091720 0x7ffaa4097ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:32.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.555 7ffaa95ba700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa4108870 con 0x7ffaa4091720 2021-02-10T11:13:32.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.555 7ffaa95ba700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa4096b80 con 0x7ffaa4096380 2021-02-10T11:13:32.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.555 7ffaa95ba700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa4090560 con 0x7ffaa4096780 2021-02-10T11:13:32.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.555 7ffaa3ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffaa4091720 0x7ffaa4097ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:32.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.555 7ffaa3ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffaa4091720 0x7ffaa4097ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44364/0 (socket says 172.21.2.101:44364) 2021-02-10T11:13:32.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.555 7ffaa3ac0700 1 -- 172.21.2.101:0/2857306926 learned_addr learned my addr 172.21.2.101:0/2857306926 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:32.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.555 7ffaa32bf700 1 --2- 172.21.2.101:0/2857306926 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffaa4096380 0x7ffaa4090ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:32.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.555 7ffaa3ac0700 1 -- 172.21.2.101:0/2857306926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffaa4096780 msgr2=0x7ffaa4091100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:32.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.555 7ffaa3ac0700 1 --2- 172.21.2.101:0/2857306926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffaa4096780 0x7ffaa4091100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:32.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.555 7ffaa3ac0700 1 -- 172.21.2.101:0/2857306926 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffaa4096380 msgr2=0x7ffaa4090ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:32.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.555 7ffaa3ac0700 1 --2- 172.21.2.101:0/2857306926 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffaa4096380 0x7ffaa4090ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:32.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.555 7ffaa3ac0700 1 -- 172.21.2.101:0/2857306926 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa4108440 con 0x7ffaa4091720 2021-02-10T11:13:32.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.555 7ffaa3ac0700 1 --2- 172.21.2.101:0/2857306926 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffaa4091720 0x7ffaa4097ad0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7ffa9c00b970 tx=0x7ffa9c00bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:32.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.556 7ffa9bfff700 1 -- 172.21.2.101:0/2857306926 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffa9c009380 con 0x7ffaa4091720 2021-02-10T11:13:32.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.556 7ffa9bfff700 1 -- 172.21.2.101:0/2857306926 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9c005820 con 0x7ffaa4091720 2021-02-10T11:13:32.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.556 7ffa9bfff700 1 -- 172.21.2.101:0/2857306926 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffa9c005c80 con 0x7ffaa4091720 2021-02-10T11:13:32.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.556 7ffaa95ba700 1 -- 172.21.2.101:0/2857306926 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa4108440 con 0x7ffaa4091720 2021-02-10T11:13:32.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.556 7ffaa95ba700 1 -- 172.21.2.101:0/2857306926 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa40983e0 con 0x7ffaa4091720 2021-02-10T11:13:32.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.556 7ffa9bfff700 1 -- 172.21.2.101:0/2857306926 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ffa9c005a00 con 0x7ffaa4091720 2021-02-10T11:13:32.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.556 7ffa9bfff700 1 --2- 172.21.2.101:0/2857306926 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffa880186c0 0x7ffa8801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:32.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.556 7ffa9bfff700 1 -- 172.21.2.101:0/2857306926 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ffa9c032920 con 0x7ffaa4091720 2021-02-10T11:13:32.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.556 7ffaa32bf700 1 --2- 172.21.2.101:0/2857306926 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffa880186c0 0x7ffa8801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:32.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.557 7ffaa32bf700 1 --2- 172.21.2.101:0/2857306926 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffa880186c0 0x7ffa8801abc0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:32.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.588 7ffaa95ba700 1 -- 172.21.2.101:0/2857306926 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa40891c0 con 0x7ffaa4091720 2021-02-10T11:13:32.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.591 7ffa9bfff700 1 -- 172.21.2.101:0/2857306926 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ffa9c009380 con 0x7ffaa4091720 2021-02-10T11:13:32.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.721 7ffaa95ba700 1 -- 172.21.2.101:0/2857306926 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "pg dump", "target": ["mgr", ""], "format": "json"}) v1 -- 0x7ffaa4096b80 con 0x7ffa880186c0 2021-02-10T11:13:32.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.723 7ffa9bfff700 1 -- 172.21.2.101:0/2857306926 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 dumped all) v1 ==== 18+0+77203 (crc 0 0 0) 0x7ffa94000960 con 0x7ffa880186c0 2021-02-10T11:13:32.730 INFO:teuthology.orchestra.run.gibba001.stderr:dumped all 2021-02-10T11:13:32.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.726 7ffaa95ba700 1 -- 172.21.2.101:0/2857306926 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffa880186c0 msgr2=0x7ffa8801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:32.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.726 7ffaa95ba700 1 --2- 172.21.2.101:0/2857306926 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffa880186c0 0x7ffa8801abc0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:32.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.726 7ffaa95ba700 1 -- 172.21.2.101:0/2857306926 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffaa4091720 msgr2=0x7ffaa4097ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:32.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.726 7ffaa95ba700 1 --2- 172.21.2.101:0/2857306926 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffaa4091720 0x7ffaa4097ad0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7ffa9c00b970 tx=0x7ffa9c00bd10).stop 2021-02-10T11:13:32.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.726 7ffaa95ba700 1 -- 172.21.2.101:0/2857306926 shutdown_connections 2021-02-10T11:13:32.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.726 7ffaa95ba700 1 --2- 172.21.2.101:0/2857306926 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffa880186c0 0x7ffa8801abc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:32.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.726 7ffaa95ba700 1 --2- 172.21.2.101:0/2857306926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffaa4096780 0x7ffaa4091100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:32.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.726 7ffaa95ba700 1 --2- 172.21.2.101:0/2857306926 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffaa4091720 0x7ffaa4097ad0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:32.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.726 7ffaa95ba700 1 --2- 172.21.2.101:0/2857306926 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffaa4096380 0x7ffaa4090ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:32.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.726 7ffaa95ba700 1 -- 172.21.2.101:0/2857306926 >> 172.21.2.101:0/2857306926 conn(0x7ffaa40b07f0 msgr2=0x7ffaa4107250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:32.731 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:13:32.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.726 7ffaa95ba700 1 -- 172.21.2.101:0/2857306926 shutdown_connections 2021-02-10T11:13:32.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:32.727 7ffaa95ba700 1 -- 172.21.2.101:0/2857306926 wait complete. 2021-02-10T11:13:32.739 INFO:teuthology.orchestra.run.gibba001.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2021-02-10 11:13:32.137857","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2286,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"kb":754974720,"kb_used":8390336,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584384,"statfs":{"total":773094113280,"available":764502409216,"internally_reserved":8589934592,"allocated":1703936,"data_stored":554356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":760,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001422"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"13","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.350023","last_change":"2021-02-10 11:13:09.495109","last_active":"2021-02-10 11:13:14.350023","last_peered":"2021-02-10 11:13:14.350023","last_clean":"2021-02-10 11:13:14.350023","last_became_active":"2021-02-10 11:13:09.494906","last_became_peered":"2021-02-10 11:13:09.494906","last_unstale":"2021-02-10 11:13:14.350023","last_undegraded":"2021-02-10 11:13:14.350023","last_fullsized":"2021-02-10 11:13:14.350023","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"20'8","reported_seq":"18","reported_epoch":"20","state":"active+clean","last_fresh":"2021-02-10 11:13:22.554526","last_change":"2021-02-10 11:13:13.509995","last_active":"2021-02-10 11:13:22.554526","last_peered":"2021-02-10 11:13:22.554526","last_clean":"2021-02-10 11:13:22.554526","last_became_active":"2021-02-10 11:13:13.509815","last_became_peered":"2021-02-10 11:13:13.509815","last_unstale":"2021-02-10 11:13:22.554526","last_undegraded":"2021-02-10 11:13:22.554526","last_fullsized":"2021-02-10 11:13:22.554526","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:12.494038","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:12.494038","last_clean_scrub_stamp":"2021-02-10 11:13:12.494038","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1526,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"9","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.161251","last_change":"2021-02-10 11:13:14.160844","last_active":"2021-02-10 11:13:14.161251","last_peered":"2021-02-10 11:13:14.161251","last_clean":"2021-02-10 11:13:14.161251","last_became_active":"2021-02-10 11:13:14.160633","last_became_peered":"2021-02-10 11:13:14.160633","last_unstale":"2021-02-10 11:13:14.161251","last_undegraded":"2021-02-10 11:13:14.161251","last_fullsized":"2021-02-10 11:13:14.161251","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:13.501437","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:13.501437","last_clean_scrub_stamp":"2021-02-10 11:13:13.501437","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"13","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.350019","last_change":"2021-02-10 11:13:09.495461","last_active":"2021-02-10 11:13:14.350019","last_peered":"2021-02-10 11:13:14.350019","last_clean":"2021-02-10 11:13:14.350019","last_became_active":"2021-02-10 11:13:09.495226","last_became_peered":"2021-02-10 11:13:09.495226","last_unstale":"2021-02-10 11:13:14.350019","last_undegraded":"2021-02-10 11:13:14.350019","last_fullsized":"2021-02-10 11:13:14.350019","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"20'6","reported_seq":"16","reported_epoch":"20","state":"active+clean","last_fresh":"2021-02-10 11:13:22.561499","last_change":"2021-02-10 11:13:13.516702","last_active":"2021-02-10 11:13:22.561499","last_peered":"2021-02-10 11:13:22.561499","last_clean":"2021-02-10 11:13:22.561499","last_became_active":"2021-02-10 11:13:13.516524","last_became_peered":"2021-02-10 11:13:13.516524","last_unstale":"2021-02-10 11:13:22.561499","last_undegraded":"2021-02-10 11:13:22.561499","last_fullsized":"2021-02-10 11:13:22.561499","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:12.494038","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:12.494038","last_clean_scrub_stamp":"2021-02-10 11:13:12.494038","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"9","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.155525","last_change":"2021-02-10 11:13:14.154896","last_active":"2021-02-10 11:13:14.155525","last_peered":"2021-02-10 11:13:14.155525","last_clean":"2021-02-10 11:13:14.155525","last_became_active":"2021-02-10 11:13:14.154653","last_became_peered":"2021-02-10 11:13:14.154653","last_unstale":"2021-02-10 11:13:14.155525","last_undegraded":"2021-02-10 11:13:14.155525","last_fullsized":"2021-02-10 11:13:14.155525","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:13.501437","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:13.501437","last_clean_scrub_stamp":"2021-02-10 11:13:13.501437","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":"9","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.155920","last_change":"2021-02-10 11:13:14.155520","last_active":"2021-02-10 11:13:14.155920","last_peered":"2021-02-10 11:13:14.155920","last_clean":"2021-02-10 11:13:14.155920","last_became_active":"2021-02-10 11:13:14.155328","last_became_peered":"2021-02-10 11:13:14.155328","last_unstale":"2021-02-10 11:13:14.155920","last_undegraded":"2021-02-10 11:13:14.155920","last_fullsized":"2021-02-10 11:13:14.155920","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:13.501437","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:13.501437","last_clean_scrub_stamp":"2021-02-10 11:13:13.501437","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"13","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.159455","last_change":"2021-02-10 11:13:09.495287","last_active":"2021-02-10 11:13:14.159455","last_peered":"2021-02-10 11:13:14.159455","last_clean":"2021-02-10 11:13:14.159455","last_became_active":"2021-02-10 11:13:09.495041","last_became_peered":"2021-02-10 11:13:09.495041","last_unstale":"2021-02-10 11:13:14.159455","last_undegraded":"2021-02-10 11:13:14.159455","last_fullsized":"2021-02-10 11:13:14.159455","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"20'4","reported_seq":"14","reported_epoch":"20","state":"active+clean","last_fresh":"2021-02-10 11:13:22.553337","last_change":"2021-02-10 11:13:13.509418","last_active":"2021-02-10 11:13:22.553337","last_peered":"2021-02-10 11:13:22.553337","last_clean":"2021-02-10 11:13:22.553337","last_became_active":"2021-02-10 11:13:13.509192","last_became_peered":"2021-02-10 11:13:13.509192","last_unstale":"2021-02-10 11:13:22.553337","last_undegraded":"2021-02-10 11:13:22.553337","last_fullsized":"2021-02-10 11:13:22.553337","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:12.494038","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:12.494038","last_clean_scrub_stamp":"2021-02-10 11:13:12.494038","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"9","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.154409","last_change":"2021-02-10 11:13:14.153836","last_active":"2021-02-10 11:13:14.154409","last_peered":"2021-02-10 11:13:14.154409","last_clean":"2021-02-10 11:13:14.154409","last_became_active":"2021-02-10 11:13:14.153619","last_became_peered":"2021-02-10 11:13:14.153619","last_unstale":"2021-02-10 11:13:14.154409","last_undegraded":"2021-02-10 11:13:14.154409","last_fullsized":"2021-02-10 11:13:14.154409","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:13.501437","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:13.501437","last_clean_scrub_stamp":"2021-02-10 11:13:13.501437","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"12","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.152601","last_change":"2021-02-10 11:13:09.487892","last_active":"2021-02-10 11:13:14.152601","last_peered":"2021-02-10 11:13:14.152601","last_clean":"2021-02-10 11:13:14.152601","last_became_active":"2021-02-10 11:13:09.487627","last_became_peered":"2021-02-10 11:13:09.487627","last_unstale":"2021-02-10 11:13:14.152601","last_undegraded":"2021-02-10 11:13:14.152601","last_fullsized":"2021-02-10 11:13:14.152601","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'7","reported_seq":"17","reported_epoch":"20","state":"active+clean","last_fresh":"2021-02-10 11:13:22.560718","last_change":"2021-02-10 11:13:13.515774","last_active":"2021-02-10 11:13:22.560718","last_peered":"2021-02-10 11:13:22.560718","last_clean":"2021-02-10 11:13:22.560718","last_became_active":"2021-02-10 11:13:13.515458","last_became_peered":"2021-02-10 11:13:13.515458","last_unstale":"2021-02-10 11:13:22.560718","last_undegraded":"2021-02-10 11:13:22.560718","last_fullsized":"2021-02-10 11:13:22.560718","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:12.494038","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:12.494038","last_clean_scrub_stamp":"2021-02-10 11:13:12.494038","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":566,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"13","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.160750","last_change":"2021-02-10 11:13:09.495589","last_active":"2021-02-10 11:13:14.160750","last_peered":"2021-02-10 11:13:14.160750","last_clean":"2021-02-10 11:13:14.160750","last_became_active":"2021-02-10 11:13:09.495399","last_became_peered":"2021-02-10 11:13:09.495399","last_unstale":"2021-02-10 11:13:14.160750","last_undegraded":"2021-02-10 11:13:14.160750","last_fullsized":"2021-02-10 11:13:14.160750","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'5","reported_seq":"15","reported_epoch":"20","state":"active+clean","last_fresh":"2021-02-10 11:13:22.559807","last_change":"2021-02-10 11:13:13.515340","last_active":"2021-02-10 11:13:22.559807","last_peered":"2021-02-10 11:13:22.559807","last_clean":"2021-02-10 11:13:22.559807","last_became_active":"2021-02-10 11:13:13.515156","last_became_peered":"2021-02-10 11:13:13.515156","last_unstale":"2021-02-10 11:13:22.559807","last_undegraded":"2021-02-10 11:13:22.559807","last_fullsized":"2021-02-10 11:13:22.559807","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:12.494038","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:12.494038","last_clean_scrub_stamp":"2021-02-10 11:13:12.494038","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"9","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.155464","last_change":"2021-02-10 11:13:14.155032","last_active":"2021-02-10 11:13:14.155464","last_peered":"2021-02-10 11:13:14.155464","last_clean":"2021-02-10 11:13:14.155464","last_became_active":"2021-02-10 11:13:14.154776","last_became_peered":"2021-02-10 11:13:14.154776","last_unstale":"2021-02-10 11:13:14.155464","last_undegraded":"2021-02-10 11:13:14.155464","last_fullsized":"2021-02-10 11:13:14.155464","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:13.501437","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:13.501437","last_clean_scrub_stamp":"2021-02-10 11:13:13.501437","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"13","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.159207","last_change":"2021-02-10 11:13:09.495570","last_active":"2021-02-10 11:13:14.159207","last_peered":"2021-02-10 11:13:14.159207","last_clean":"2021-02-10 11:13:14.159207","last_became_active":"2021-02-10 11:13:09.495383","last_became_peered":"2021-02-10 11:13:09.495383","last_unstale":"2021-02-10 11:13:14.159207","last_undegraded":"2021-02-10 11:13:14.159207","last_fullsized":"2021-02-10 11:13:14.159207","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"20'1","reported_seq":"11","reported_epoch":"20","state":"active+clean","last_fresh":"2021-02-10 11:13:15.160893","last_change":"2021-02-10 11:13:13.515945","last_active":"2021-02-10 11:13:15.160893","last_peered":"2021-02-10 11:13:15.160893","last_clean":"2021-02-10 11:13:15.160893","last_became_active":"2021-02-10 11:13:13.515644","last_became_peered":"2021-02-10 11:13:13.515644","last_unstale":"2021-02-10 11:13:15.160893","last_undegraded":"2021-02-10 11:13:15.160893","last_fullsized":"2021-02-10 11:13:15.160893","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:12.494038","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:12.494038","last_clean_scrub_stamp":"2021-02-10 11:13:12.494038","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"9","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.161457","last_change":"2021-02-10 11:13:14.161055","last_active":"2021-02-10 11:13:14.161457","last_peered":"2021-02-10 11:13:14.161457","last_clean":"2021-02-10 11:13:14.161457","last_became_active":"2021-02-10 11:13:14.160843","last_became_peered":"2021-02-10 11:13:14.160843","last_unstale":"2021-02-10 11:13:14.161457","last_undegraded":"2021-02-10 11:13:14.161457","last_fullsized":"2021-02-10 11:13:14.161457","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:13.501437","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:13.501437","last_clean_scrub_stamp":"2021-02-10 11:13:13.501437","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.153763","last_change":"2021-02-10 11:13:13.509094","last_active":"2021-02-10 11:13:14.153763","last_peered":"2021-02-10 11:13:14.153763","last_clean":"2021-02-10 11:13:14.153763","last_became_active":"2021-02-10 11:13:13.508842","last_became_peered":"2021-02-10 11:13:13.508842","last_unstale":"2021-02-10 11:13:14.153763","last_undegraded":"2021-02-10 11:13:14.153763","last_fullsized":"2021-02-10 11:13:14.153763","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:12.494038","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:12.494038","last_clean_scrub_stamp":"2021-02-10 11:13:12.494038","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"13","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.160741","last_change":"2021-02-10 11:13:09.495605","last_active":"2021-02-10 11:13:14.160741","last_peered":"2021-02-10 11:13:14.160741","last_clean":"2021-02-10 11:13:14.160741","last_became_active":"2021-02-10 11:13:09.495261","last_became_peered":"2021-02-10 11:13:09.495261","last_unstale":"2021-02-10 11:13:14.160741","last_undegraded":"2021-02-10 11:13:14.160741","last_fullsized":"2021-02-10 11:13:14.160741","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"9","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.155460","last_change":"2021-02-10 11:13:14.154981","last_active":"2021-02-10 11:13:14.155460","last_peered":"2021-02-10 11:13:14.155460","last_clean":"2021-02-10 11:13:14.155460","last_became_active":"2021-02-10 11:13:14.154718","last_became_peered":"2021-02-10 11:13:14.154718","last_unstale":"2021-02-10 11:13:14.155460","last_undegraded":"2021-02-10 11:13:14.155460","last_fullsized":"2021-02-10 11:13:14.155460","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:13.501437","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:13.501437","last_clean_scrub_stamp":"2021-02-10 11:13:13.501437","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"20'8","reported_seq":"18","reported_epoch":"20","state":"active+clean","last_fresh":"2021-02-10 11:13:22.561363","last_change":"2021-02-10 11:13:13.516117","last_active":"2021-02-10 11:13:22.561363","last_peered":"2021-02-10 11:13:22.561363","last_clean":"2021-02-10 11:13:22.561363","last_became_active":"2021-02-10 11:13:13.515901","last_became_peered":"2021-02-10 11:13:13.515901","last_unstale":"2021-02-10 11:13:22.561363","last_undegraded":"2021-02-10 11:13:22.561363","last_fullsized":"2021-02-10 11:13:22.561363","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:12.494038","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:12.494038","last_clean_scrub_stamp":"2021-02-10 11:13:12.494038","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"13","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.152417","last_change":"2021-02-10 11:13:09.488300","last_active":"2021-02-10 11:13:14.152417","last_peered":"2021-02-10 11:13:14.152417","last_clean":"2021-02-10 11:13:14.152417","last_became_active":"2021-02-10 11:13:09.488122","last_became_peered":"2021-02-10 11:13:09.488122","last_unstale":"2021-02-10 11:13:14.152417","last_undegraded":"2021-02-10 11:13:14.152417","last_fullsized":"2021-02-10 11:13:14.152417","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"9","reported_epoch":"19","state":"active+clean","last_fresh":"2021-02-10 11:13:14.155114","last_change":"2021-02-10 11:13:14.154576","last_active":"2021-02-10 11:13:14.155114","last_peered":"2021-02-10 11:13:14.155114","last_clean":"2021-02-10 11:13:14.155114","last_became_active":"2021-02-10 11:13:14.154349","last_became_peered":"2021-02-10 11:13:14.154349","last_unstale":"2021-02-10 11:13:14.155114","last_undegraded":"2021-02-10 11:13:14.155114","last_fullsized":"2021-02-10 11:13:14.155114","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:13.501437","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:13.501437","last_clean_scrub_stamp":"2021-02-10 11:13:13.501437","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":2286,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048800,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323040,"statfs":{"total":96636764160,"available":95562792960,"internally_reserved":1073741824,"allocated":221184,"data_stored":69449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048792,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323048,"statfs":{"total":96636764160,"available":95562801152,"internally_reserved":1073741824,"allocated":212992,"data_stored":69289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048788,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323052,"statfs":{"total":96636764160,"available":95562805248,"internally_reserved":1073741824,"allocated":208896,"data_stored":68757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048784,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323056,"statfs":{"total":96636764160,"available":95562809344,"internally_reserved":1073741824,"allocated":204800,"data_stored":68723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048804,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323036,"statfs":{"total":96636764160,"available":95562788864,"internally_reserved":1073741824,"allocated":225280,"data_stored":70409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048796,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323044,"statfs":{"total":96636764160,"available":95562797056,"internally_reserved":1073741824,"allocated":217088,"data_stored":70249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048788,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323052,"statfs":{"total":96636764160,"available":95562805248,"internally_reserved":1073741824,"allocated":208896,"data_stored":68757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048784,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323056,"statfs":{"total":96636764160,"available":95562809344,"internally_reserved":1073741824,"allocated":204800,"data_stored":68723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]}]}]}} 2021-02-10T11:13:32.741 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph status --format=json-pretty 2021-02-10T11:13:33.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.145 7f9cdd533700 1 Processor -- start 2021-02-10T11:13:33.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.145 7f9cdd533700 1 -- start start 2021-02-10T11:13:33.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.146 7f9cdd533700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9cd81b0470 0x7f9cd81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:33.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.146 7f9cdd533700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9cd80ef610 0x7f9cd80efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:33.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.146 7f9cdd533700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd80f0b50 con 0x7f9cd80f0030 2021-02-10T11:13:33.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.146 7f9cdd533700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd80e52e0 con 0x7f9cd80ef610 2021-02-10T11:13:33.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.146 7f9cdd533700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd80e5490 con 0x7f9cd81b0470 2021-02-10T11:13:33.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.146 7f9cd6abe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9cd80ef610 0x7f9cd80efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:33.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.146 7f9cd6abe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9cd80ef610 0x7f9cd80efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44368/0 (socket says 172.21.2.101:44368) 2021-02-10T11:13:33.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.146 7f9cd6abe700 1 -- 172.21.2.101:0/2726629721 learned_addr learned my addr 172.21.2.101:0/2726629721 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:33.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.146 7f9cd72bf700 1 --2- 172.21.2.101:0/2726629721 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9cd81b0470 0x7f9cd81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:33.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.146 7f9cd6abe700 1 -- 172.21.2.101:0/2726629721 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9cd81b0470 msgr2=0x7f9cd81b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:33.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.146 7f9cd6abe700 1 --2- 172.21.2.101:0/2726629721 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9cd81b0470 0x7f9cd81b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:33.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.146 7f9cd6abe700 1 -- 172.21.2.101:0/2726629721 >> v1:172.21.2.108:6789/0 conn(0x7f9cd80f0030 legacy=0x7f9cd80f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:33.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.146 7f9cd6abe700 1 -- 172.21.2.101:0/2726629721 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd80f0430 con 0x7f9cd80ef610 2021-02-10T11:13:33.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.146 7f9cd72bf700 1 --2- 172.21.2.101:0/2726629721 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9cd81b0470 0x7f9cd81b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:13:33.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.147 7f9cd6abe700 1 --2- 172.21.2.101:0/2726629721 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9cd80ef610 0x7f9cd80efa10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f9cc0006700 tx=0x7f9cc0009b80).ready entity=mon.0 client_cookie=3feab9ae96defe73 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:33.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.147 7f9cd62bd700 1 -- 172.21.2.101:0/2726629721 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9cc000c0c0 con 0x7f9cd80ef610 2021-02-10T11:13:33.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.147 7f9cd62bd700 1 -- 172.21.2.101:0/2726629721 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc000c440 con 0x7f9cd80ef610 2021-02-10T11:13:33.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.147 7f9cd62bd700 1 -- 172.21.2.101:0/2726629721 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9cc000c960 con 0x7f9cd80ef610 2021-02-10T11:13:33.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.147 7f9cdd533700 1 -- 172.21.2.101:0/2726629721 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9cd80ef610 msgr2=0x7f9cd80efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:33.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.147 7f9cdd533700 1 --2- 172.21.2.101:0/2726629721 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9cd80ef610 0x7f9cd80efa10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f9cc0006700 tx=0x7f9cc0009b80).stop 2021-02-10T11:13:33.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.147 7f9cdd533700 1 -- 172.21.2.101:0/2726629721 shutdown_connections 2021-02-10T11:13:33.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.147 7f9cdd533700 1 --2- 172.21.2.101:0/2726629721 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9cd80ef610 0x7f9cd80efa10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:33.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.147 7f9cdd533700 1 --2- 172.21.2.101:0/2726629721 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9cd81b0470 0x7f9cd81b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:33.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.147 7f9cdd533700 1 -- 172.21.2.101:0/2726629721 >> 172.21.2.101:0/2726629721 conn(0x7f9cd80ce3e0 msgr2=0x7f9cd80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:33.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.147 7f9cdd533700 1 -- 172.21.2.101:0/2726629721 shutdown_connections 2021-02-10T11:13:33.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.147 7f9cdd533700 1 -- 172.21.2.101:0/2726629721 wait complete. 2021-02-10T11:13:33.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.162 7f9cdd533700 1 Processor -- start 2021-02-10T11:13:33.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.162 7f9cdd533700 1 -- start start 2021-02-10T11:13:33.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.163 7f9cdd533700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9cd8093db0 0x7f9cd81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:33.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.163 7f9cdd533700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9cd80aa360 0x7f9cd81a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:33.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.163 7f9cdd533700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9cd80aa760 0x7f9cd81a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:33.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.163 7f9cdd533700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd81a4e40 con 0x7f9cd8093db0 2021-02-10T11:13:33.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.163 7f9cdd533700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd8198c10 con 0x7f9cd80aa360 2021-02-10T11:13:33.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.163 7f9cdd533700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd80941b0 con 0x7f9cd80aa760 2021-02-10T11:13:33.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.163 7f9cd72bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9cd8093db0 0x7f9cd81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:33.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.163 7f9cd72bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9cd8093db0 0x7f9cd81985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44374/0 (socket says 172.21.2.101:44374) 2021-02-10T11:13:33.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.163 7f9cd72bf700 1 -- 172.21.2.101:0/1140528604 learned_addr learned my addr 172.21.2.101:0/1140528604 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:33.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.163 7f9cd7ac0700 1 --2- 172.21.2.101:0/1140528604 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9cd80aa760 0x7f9cd81a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:33.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.163 7f9cd6abe700 1 --2- 172.21.2.101:0/1140528604 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9cd80aa360 0x7f9cd81a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:33.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.163 7f9cd72bf700 1 -- 172.21.2.101:0/1140528604 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9cd80aa760 msgr2=0x7f9cd81a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:33.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.163 7f9cd72bf700 1 --2- 172.21.2.101:0/1140528604 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9cd80aa760 0x7f9cd81a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:33.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.163 7f9cd72bf700 1 -- 172.21.2.101:0/1140528604 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9cd80aa360 msgr2=0x7f9cd81a5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:33.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.163 7f9cd72bf700 1 --2- 172.21.2.101:0/1140528604 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9cd80aa360 0x7f9cd81a5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:33.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.163 7f9cd72bf700 1 -- 172.21.2.101:0/1140528604 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd81a28f0 con 0x7f9cd8093db0 2021-02-10T11:13:33.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.164 7f9cd7ac0700 1 --2- 172.21.2.101:0/1140528604 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9cd80aa760 0x7f9cd81a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:13:33.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.164 7f9cd6abe700 1 --2- 172.21.2.101:0/1140528604 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9cd80aa360 0x7f9cd81a5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:13:33.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.164 7f9cd72bf700 1 --2- 172.21.2.101:0/1140528604 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9cd8093db0 0x7f9cd81985b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f9cc8006f20 tx=0x7f9cc800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:33.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.164 7f9ccffff700 1 -- 172.21.2.101:0/1140528604 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9cc800bd80 con 0x7f9cd8093db0 2021-02-10T11:13:33.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.164 7f9ccffff700 1 -- 172.21.2.101:0/1140528604 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc8000c90 con 0x7f9cd8093db0 2021-02-10T11:13:33.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.164 7f9ccffff700 1 -- 172.21.2.101:0/1140528604 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9cc80022b0 con 0x7f9cd8093db0 2021-02-10T11:13:33.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.164 7f9cdd533700 1 -- 172.21.2.101:0/1140528604 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd81a28f0 con 0x7f9cd8093db0 2021-02-10T11:13:33.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.164 7f9cdd533700 1 -- 172.21.2.101:0/1140528604 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd81a28f0 con 0x7f9cd8093db0 2021-02-10T11:13:33.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.165 7f9ccffff700 1 -- 172.21.2.101:0/1140528604 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9cc800a000 con 0x7f9cd8093db0 2021-02-10T11:13:33.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.165 7f9ccffff700 1 --2- 172.21.2.101:0/1140528604 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9cbc0186c0 0x7f9cbc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:33.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.165 7f9ccffff700 1 -- 172.21.2.101:0/1140528604 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9cc8032400 con 0x7f9cd8093db0 2021-02-10T11:13:33.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.165 7f9cd6abe700 1 --2- 172.21.2.101:0/1140528604 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9cbc0186c0 0x7f9cbc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:33.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.165 7f9cd6abe700 1 --2- 172.21.2.101:0/1140528604 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9cbc0186c0 0x7f9cbc01abc0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:33.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.196 7f9cdd533700 1 -- 172.21.2.101:0/1140528604 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd80941b0 con 0x7f9cd8093db0 2021-02-10T11:13:33.202 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.198 7f9ccffff700 1 -- 172.21.2.101:0/1140528604 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9cc8006bb0 con 0x7f9cd8093db0 2021-02-10T11:13:33.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.331 7f9cdd533700 1 -- 172.21.2.101:0/1140528604 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9cd80941b0 con 0x7f9cd8093db0 2021-02-10T11:13:33.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.333 7f9ccffff700 1 -- 172.21.2.101:0/1140528604 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+211028 (secure 0 0 0) 0x7f9cc8030d30 con 0x7f9cd8093db0 2021-02-10T11:13:33.341 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:13:33.341 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:13:33.341 INFO:teuthology.orchestra.run.gibba001.stdout: "fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:13:33.341 INFO:teuthology.orchestra.run.gibba001.stdout: "health": { 2021-02-10T11:13:33.341 INFO:teuthology.orchestra.run.gibba001.stdout: "checks": {}, 2021-02-10T11:13:33.341 INFO:teuthology.orchestra.run.gibba001.stdout: "status": "HEALTH_OK" 2021-02-10T11:13:33.342 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.342 INFO:teuthology.orchestra.run.gibba001.stdout: "election_epoch": 4, 2021-02-10T11:13:33.342 INFO:teuthology.orchestra.run.gibba001.stdout: "quorum": [ 2021-02-10T11:13:33.342 INFO:teuthology.orchestra.run.gibba001.stdout: 0, 2021-02-10T11:13:33.342 INFO:teuthology.orchestra.run.gibba001.stdout: 1, 2021-02-10T11:13:33.342 INFO:teuthology.orchestra.run.gibba001.stdout: 2 2021-02-10T11:13:33.342 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:13:33.342 INFO:teuthology.orchestra.run.gibba001.stdout: "quorum_names": [ 2021-02-10T11:13:33.342 INFO:teuthology.orchestra.run.gibba001.stdout: "a", 2021-02-10T11:13:33.342 INFO:teuthology.orchestra.run.gibba001.stdout: "b", 2021-02-10T11:13:33.342 INFO:teuthology.orchestra.run.gibba001.stdout: "c" 2021-02-10T11:13:33.342 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:13:33.342 INFO:teuthology.orchestra.run.gibba001.stdout: "quorum_age": 56, 2021-02-10T11:13:33.342 INFO:teuthology.orchestra.run.gibba001.stdout: "monmap": { 2021-02-10T11:13:33.343 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 1, 2021-02-10T11:13:33.343 INFO:teuthology.orchestra.run.gibba001.stdout: "min_mon_release_name": "14", 2021-02-10T11:13:33.343 INFO:teuthology.orchestra.run.gibba001.stdout: "num_mons": 3 2021-02-10T11:13:33.343 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.343 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap": { 2021-02-10T11:13:33.343 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap": { 2021-02-10T11:13:33.343 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 20, 2021-02-10T11:13:33.343 INFO:teuthology.orchestra.run.gibba001.stdout: "num_osds": 8, 2021-02-10T11:13:33.343 INFO:teuthology.orchestra.run.gibba001.stdout: "num_up_osds": 8, 2021-02-10T11:13:33.343 INFO:teuthology.orchestra.run.gibba001.stdout: "num_in_osds": 8, 2021-02-10T11:13:33.343 INFO:teuthology.orchestra.run.gibba001.stdout: "num_remapped_pgs": 0 2021-02-10T11:13:33.343 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.343 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.344 INFO:teuthology.orchestra.run.gibba001.stdout: "pgmap": { 2021-02-10T11:13:33.344 INFO:teuthology.orchestra.run.gibba001.stdout: "pgs_by_state": [ 2021-02-10T11:13:33.344 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.344 INFO:teuthology.orchestra.run.gibba001.stdout: "state_name": "active+clean", 2021-02-10T11:13:33.344 INFO:teuthology.orchestra.run.gibba001.stdout: "count": 24 2021-02-10T11:13:33.344 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.344 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:13:33.344 INFO:teuthology.orchestra.run.gibba001.stdout: "num_pgs": 24, 2021-02-10T11:13:33.344 INFO:teuthology.orchestra.run.gibba001.stdout: "num_pools": 3, 2021-02-10T11:13:33.344 INFO:teuthology.orchestra.run.gibba001.stdout: "num_objects": 22, 2021-02-10T11:13:33.344 INFO:teuthology.orchestra.run.gibba001.stdout: "data_bytes": 2286, 2021-02-10T11:13:33.344 INFO:teuthology.orchestra.run.gibba001.stdout: "bytes_used": 8591704064, 2021-02-10T11:13:33.344 INFO:teuthology.orchestra.run.gibba001.stdout: "bytes_avail": 764502409216, 2021-02-10T11:13:33.344 INFO:teuthology.orchestra.run.gibba001.stdout: "bytes_total": 773094113280, 2021-02-10T11:13:33.345 INFO:teuthology.orchestra.run.gibba001.stdout: "write_bytes_sec": 1279, 2021-02-10T11:13:33.345 INFO:teuthology.orchestra.run.gibba001.stdout: "read_op_per_sec": 0, 2021-02-10T11:13:33.345 INFO:teuthology.orchestra.run.gibba001.stdout: "write_op_per_sec": 4 2021-02-10T11:13:33.345 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.345 INFO:teuthology.orchestra.run.gibba001.stdout: "fsmap": { 2021-02-10T11:13:33.345 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 6, 2021-02-10T11:13:33.345 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 1, 2021-02-10T11:13:33.345 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:13:33.345 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:13:33.345 INFO:teuthology.orchestra.run.gibba001.stdout: "max": 1, 2021-02-10T11:13:33.345 INFO:teuthology.orchestra.run.gibba001.stdout: "by_rank": [ 2021-02-10T11:13:33.345 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.345 INFO:teuthology.orchestra.run.gibba001.stdout: "filesystem_id": 1, 2021-02-10T11:13:33.345 INFO:teuthology.orchestra.run.gibba001.stdout: "rank": 0, 2021-02-10T11:13:33.346 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "a", 2021-02-10T11:13:33.346 INFO:teuthology.orchestra.run.gibba001.stdout: "status": "up:active", 2021-02-10T11:13:33.346 INFO:teuthology.orchestra.run.gibba001.stdout: "gid": 4299 2021-02-10T11:13:33.346 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.346 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:13:33.346 INFO:teuthology.orchestra.run.gibba001.stdout: "up:standby": 1 2021-02-10T11:13:33.346 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.346 INFO:teuthology.orchestra.run.gibba001.stdout: "mgrmap": { 2021-02-10T11:13:33.346 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 3, 2021-02-10T11:13:33.346 INFO:teuthology.orchestra.run.gibba001.stdout: "active_gid": 4100, 2021-02-10T11:13:33.346 INFO:teuthology.orchestra.run.gibba001.stdout: "active_name": "y", 2021-02-10T11:13:33.346 INFO:teuthology.orchestra.run.gibba001.stdout: "active_addrs": { 2021-02-10T11:13:33.346 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:13:33.347 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.347 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:13:33.347 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6800", 2021-02-10T11:13:33.347 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 24165 2021-02-10T11:13:33.347 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.347 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.347 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:13:33.347 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6801", 2021-02-10T11:13:33.347 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 24165 2021-02-10T11:13:33.347 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.347 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:13:33.347 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.347 INFO:teuthology.orchestra.run.gibba001.stdout: "active_addr": "172.21.2.101:6801/24165", 2021-02-10T11:13:33.348 INFO:teuthology.orchestra.run.gibba001.stdout: "active_change": "2021-02-10 11:12:38.124918", 2021-02-10T11:13:33.348 INFO:teuthology.orchestra.run.gibba001.stdout: "available": true, 2021-02-10T11:13:33.348 INFO:teuthology.orchestra.run.gibba001.stdout: "standbys": [ 2021-02-10T11:13:33.348 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.348 INFO:teuthology.orchestra.run.gibba001.stdout: "gid": 4106, 2021-02-10T11:13:33.348 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "x", 2021-02-10T11:13:33.348 INFO:teuthology.orchestra.run.gibba001.stdout: "available_modules": [ 2021-02-10T11:13:33.348 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.348 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "alerts", 2021-02-10T11:13:33.348 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.348 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.348 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.348 INFO:teuthology.orchestra.run.gibba001.stdout: "interval": { 2021-02-10T11:13:33.349 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "interval", 2021-02-10T11:13:33.349 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.349 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.349 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.349 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "60", 2021-02-10T11:13:33.349 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.349 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.349 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.349 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "How frequently to reexamine health status", 2021-02-10T11:13:33.349 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.349 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.349 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.349 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.349 INFO:teuthology.orchestra.run.gibba001.stdout: "smtp_destination": { 2021-02-10T11:13:33.350 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "smtp_destination", 2021-02-10T11:13:33.350 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.350 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.350 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.350 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.350 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.350 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.350 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.350 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "Email address to send alerts to", 2021-02-10T11:13:33.350 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.350 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.350 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.350 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.351 INFO:teuthology.orchestra.run.gibba001.stdout: "smtp_from_name": { 2021-02-10T11:13:33.351 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "smtp_from_name", 2021-02-10T11:13:33.351 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.351 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.351 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.351 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "Ceph", 2021-02-10T11:13:33.351 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.351 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.351 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.351 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "Email From: name", 2021-02-10T11:13:33.351 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.351 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.351 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.352 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.352 INFO:teuthology.orchestra.run.gibba001.stdout: "smtp_host": { 2021-02-10T11:13:33.352 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "smtp_host", 2021-02-10T11:13:33.352 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.352 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.352 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.352 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.352 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.352 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.352 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.352 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "SMTP server", 2021-02-10T11:13:33.352 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.352 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.352 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.353 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.353 INFO:teuthology.orchestra.run.gibba001.stdout: "smtp_password": { 2021-02-10T11:13:33.353 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "smtp_password", 2021-02-10T11:13:33.353 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.353 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.353 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.353 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.353 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.353 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.353 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.353 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "Password to authenticate with", 2021-02-10T11:13:33.353 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.353 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.354 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.354 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.354 INFO:teuthology.orchestra.run.gibba001.stdout: "smtp_port": { 2021-02-10T11:13:33.354 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "smtp_port", 2021-02-10T11:13:33.354 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.354 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.354 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.354 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "465", 2021-02-10T11:13:33.354 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.354 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.354 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.354 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "SMTP port", 2021-02-10T11:13:33.354 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.355 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.355 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.355 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.355 INFO:teuthology.orchestra.run.gibba001.stdout: "smtp_sender": { 2021-02-10T11:13:33.355 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "smtp_sender", 2021-02-10T11:13:33.355 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.355 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.355 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.355 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.355 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.355 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.355 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.356 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "SMTP envelope sender", 2021-02-10T11:13:33.356 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.356 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.356 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.356 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.356 INFO:teuthology.orchestra.run.gibba001.stdout: "smtp_ssl": { 2021-02-10T11:13:33.356 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "smtp_ssl", 2021-02-10T11:13:33.356 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.356 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.356 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.356 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.356 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.356 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.357 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.357 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "Use SSL to connect to SMTP server", 2021-02-10T11:13:33.357 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.357 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.357 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.357 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.357 INFO:teuthology.orchestra.run.gibba001.stdout: "smtp_user": { 2021-02-10T11:13:33.357 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "smtp_user", 2021-02-10T11:13:33.357 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.357 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.357 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.357 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.357 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.358 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.358 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.358 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "User to authenticate as", 2021-02-10T11:13:33.358 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.358 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.358 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.358 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.358 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.359 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.359 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.359 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "ansible", 2021-02-10T11:13:33.359 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.359 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.359 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.359 INFO:teuthology.orchestra.run.gibba001.stdout: "password": { 2021-02-10T11:13:33.359 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "password", 2021-02-10T11:13:33.359 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.359 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.359 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.359 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.359 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.360 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.360 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.360 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.360 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.360 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.360 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.360 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.360 INFO:teuthology.orchestra.run.gibba001.stdout: "server_url": { 2021-02-10T11:13:33.360 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "server_url", 2021-02-10T11:13:33.360 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.360 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.360 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.360 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.361 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.361 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.361 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.361 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.361 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.361 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.361 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.361 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.361 INFO:teuthology.orchestra.run.gibba001.stdout: "username": { 2021-02-10T11:13:33.361 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "username", 2021-02-10T11:13:33.361 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.362 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.362 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.362 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.362 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.362 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.362 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.362 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.362 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.363 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.363 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.363 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.363 INFO:teuthology.orchestra.run.gibba001.stdout: "verify_server": { 2021-02-10T11:13:33.363 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "verify_server", 2021-02-10T11:13:33.363 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.363 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.363 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.363 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.363 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.363 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.363 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.363 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.364 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.364 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.364 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.364 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.364 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.364 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.364 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.364 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "balancer", 2021-02-10T11:13:33.364 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.364 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.364 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.364 INFO:teuthology.orchestra.run.gibba001.stdout: "active": { 2021-02-10T11:13:33.364 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "active", 2021-02-10T11:13:33.364 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.365 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.365 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.365 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "False", 2021-02-10T11:13:33.365 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.365 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.365 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.365 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "automatically balance PGs across cluster", 2021-02-10T11:13:33.365 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.365 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.365 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.365 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.365 INFO:teuthology.orchestra.run.gibba001.stdout: "begin_time": { 2021-02-10T11:13:33.365 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "begin_time", 2021-02-10T11:13:33.366 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.366 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.366 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.366 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "0000", 2021-02-10T11:13:33.366 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.366 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.366 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.366 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "beginning time of day to automatically balance", 2021-02-10T11:13:33.366 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2021-02-10T11:13:33.366 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.366 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.366 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.366 INFO:teuthology.orchestra.run.gibba001.stdout: "begin_weekday": { 2021-02-10T11:13:33.366 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "begin_weekday", 2021-02-10T11:13:33.367 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "uint", 2021-02-10T11:13:33.367 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.367 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.367 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "0", 2021-02-10T11:13:33.367 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "0", 2021-02-10T11:13:33.367 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "7", 2021-02-10T11:13:33.367 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.367 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2021-02-10T11:13:33.367 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2021-02-10T11:13:33.367 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.367 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.367 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.367 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_compat_max_iterations": { 2021-02-10T11:13:33.367 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "crush_compat_max_iterations", 2021-02-10T11:13:33.368 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "uint", 2021-02-10T11:13:33.368 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.368 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.368 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "25", 2021-02-10T11:13:33.368 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "1", 2021-02-10T11:13:33.368 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "250", 2021-02-10T11:13:33.368 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.368 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "maximum number of iterations to attempt optimization", 2021-02-10T11:13:33.368 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.368 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.368 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.368 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.368 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_compat_metrics": { 2021-02-10T11:13:33.369 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "crush_compat_metrics", 2021-02-10T11:13:33.369 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.369 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.369 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.369 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "pgs,objects,bytes", 2021-02-10T11:13:33.369 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.369 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.369 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.369 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "metrics with which to calculate OSD utilization", 2021-02-10T11:13:33.369 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2021-02-10T11:13:33.369 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.369 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.369 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.369 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_compat_step": { 2021-02-10T11:13:33.370 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "crush_compat_step", 2021-02-10T11:13:33.370 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "float", 2021-02-10T11:13:33.370 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.370 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.370 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "0.5", 2021-02-10T11:13:33.370 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "0.001", 2021-02-10T11:13:33.370 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "0.999", 2021-02-10T11:13:33.370 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.370 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "aggressiveness of optimization", 2021-02-10T11:13:33.370 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2021-02-10T11:13:33.370 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.370 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.370 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.371 INFO:teuthology.orchestra.run.gibba001.stdout: "end_time": { 2021-02-10T11:13:33.371 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "end_time", 2021-02-10T11:13:33.371 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.371 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.371 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.371 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "2400", 2021-02-10T11:13:33.371 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.371 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.371 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.371 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "ending time of day to automatically balance", 2021-02-10T11:13:33.371 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2021-02-10T11:13:33.371 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.371 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.371 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.372 INFO:teuthology.orchestra.run.gibba001.stdout: "end_weekday": { 2021-02-10T11:13:33.372 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "end_weekday", 2021-02-10T11:13:33.372 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "uint", 2021-02-10T11:13:33.372 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.372 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.372 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "7", 2021-02-10T11:13:33.372 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "0", 2021-02-10T11:13:33.372 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "7", 2021-02-10T11:13:33.372 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.372 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2021-02-10T11:13:33.372 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2021-02-10T11:13:33.372 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.372 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.373 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.373 INFO:teuthology.orchestra.run.gibba001.stdout: "min_score": { 2021-02-10T11:13:33.373 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "min_score", 2021-02-10T11:13:33.373 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "float", 2021-02-10T11:13:33.373 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.373 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.373 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "0", 2021-02-10T11:13:33.373 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.373 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.373 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.373 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "minimum score, below which no optimization is attempted", 2021-02-10T11:13:33.373 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.373 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.373 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.374 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.374 INFO:teuthology.orchestra.run.gibba001.stdout: "mode": { 2021-02-10T11:13:33.374 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "mode", 2021-02-10T11:13:33.374 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.374 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.374 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.374 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "none", 2021-02-10T11:13:33.374 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.374 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.374 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [ 2021-02-10T11:13:33.374 INFO:teuthology.orchestra.run.gibba001.stdout: "crush-compat", 2021-02-10T11:13:33.374 INFO:teuthology.orchestra.run.gibba001.stdout: "none", 2021-02-10T11:13:33.374 INFO:teuthology.orchestra.run.gibba001.stdout: "upmap" 2021-02-10T11:13:33.375 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:13:33.375 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "Balancer mode", 2021-02-10T11:13:33.375 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.375 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.375 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.375 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.375 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_ids": { 2021-02-10T11:13:33.375 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "pool_ids", 2021-02-10T11:13:33.375 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.375 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.375 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.375 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.375 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.376 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.376 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.376 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "pools which the automatic balancing will be limited to", 2021-02-10T11:13:33.376 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.376 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.376 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.376 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.376 INFO:teuthology.orchestra.run.gibba001.stdout: "sleep_interval": { 2021-02-10T11:13:33.376 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "sleep_interval", 2021-02-10T11:13:33.376 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.376 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.376 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.376 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "60", 2021-02-10T11:13:33.376 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.377 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.377 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.377 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "how frequently to wake up and attempt optimization", 2021-02-10T11:13:33.377 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.377 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.377 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.377 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.377 INFO:teuthology.orchestra.run.gibba001.stdout: "upmap_max_deviation": { 2021-02-10T11:13:33.377 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "upmap_max_deviation", 2021-02-10T11:13:33.377 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.377 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.377 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.377 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "5", 2021-02-10T11:13:33.378 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "1", 2021-02-10T11:13:33.378 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.378 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.378 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "deviation below which no optimization is attempted", 2021-02-10T11:13:33.378 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2021-02-10T11:13:33.378 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.378 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.378 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.378 INFO:teuthology.orchestra.run.gibba001.stdout: "upmap_max_iterations": { 2021-02-10T11:13:33.378 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "upmap_max_iterations", 2021-02-10T11:13:33.378 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "uint", 2021-02-10T11:13:33.378 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.378 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.379 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "10", 2021-02-10T11:13:33.379 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.379 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.379 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.379 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "maximum upmap optimization iterations", 2021-02-10T11:13:33.379 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.379 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.379 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.379 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.379 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.379 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.379 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.380 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "crash", 2021-02-10T11:13:33.380 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.380 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.380 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.380 INFO:teuthology.orchestra.run.gibba001.stdout: "retain_interval": { 2021-02-10T11:13:33.380 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "retain_interval", 2021-02-10T11:13:33.380 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.380 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.380 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.380 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "31536000", 2021-02-10T11:13:33.380 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.380 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.381 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.381 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "how long to retain crashes before pruning them", 2021-02-10T11:13:33.381 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.381 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.381 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.381 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.381 INFO:teuthology.orchestra.run.gibba001.stdout: "warn_recent_interval": { 2021-02-10T11:13:33.381 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "warn_recent_interval", 2021-02-10T11:13:33.381 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.381 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.381 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.381 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "1209600", 2021-02-10T11:13:33.381 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.382 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.382 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.382 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "time interval in which to warn about recent crashes", 2021-02-10T11:13:33.382 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.382 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.382 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.382 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.382 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.382 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.382 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.382 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "dashboard", 2021-02-10T11:13:33.382 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.382 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.383 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.383 INFO:teuthology.orchestra.run.gibba001.stdout: "ALERTMANAGER_API_HOST": { 2021-02-10T11:13:33.383 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "ALERTMANAGER_API_HOST", 2021-02-10T11:13:33.383 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.383 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.383 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.383 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.383 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.383 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.383 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.383 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.383 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.383 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.384 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.384 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.384 INFO:teuthology.orchestra.run.gibba001.stdout: "AUDIT_API_ENABLED": { 2021-02-10T11:13:33.384 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "AUDIT_API_ENABLED", 2021-02-10T11:13:33.384 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.384 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.384 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.384 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "False", 2021-02-10T11:13:33.384 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.384 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.384 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.385 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.385 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.385 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.385 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.385 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.385 INFO:teuthology.orchestra.run.gibba001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2021-02-10T11:13:33.385 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2021-02-10T11:13:33.385 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.385 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.385 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.385 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.385 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.386 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.386 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.386 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.386 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.386 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.386 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.386 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.386 INFO:teuthology.orchestra.run.gibba001.stdout: "ENABLE_BROWSABLE_API": { 2021-02-10T11:13:33.386 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "ENABLE_BROWSABLE_API", 2021-02-10T11:13:33.386 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.386 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.386 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.386 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.387 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.387 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.387 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.387 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.387 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.387 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.387 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.387 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.387 INFO:teuthology.orchestra.run.gibba001.stdout: "FEATURE_TOGGLE_cephfs": { 2021-02-10T11:13:33.387 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "FEATURE_TOGGLE_cephfs", 2021-02-10T11:13:33.387 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.387 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.387 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.387 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.388 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.388 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.388 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.388 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.388 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.388 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.388 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.388 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.388 INFO:teuthology.orchestra.run.gibba001.stdout: "FEATURE_TOGGLE_iscsi": { 2021-02-10T11:13:33.388 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "FEATURE_TOGGLE_iscsi", 2021-02-10T11:13:33.388 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.388 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.388 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.389 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.389 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.389 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.389 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.389 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.389 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.389 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.389 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.389 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.389 INFO:teuthology.orchestra.run.gibba001.stdout: "FEATURE_TOGGLE_mirroring": { 2021-02-10T11:13:33.389 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "FEATURE_TOGGLE_mirroring", 2021-02-10T11:13:33.389 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.389 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.389 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.389 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.390 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.390 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.390 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.390 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.390 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.390 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.390 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.390 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.390 INFO:teuthology.orchestra.run.gibba001.stdout: "FEATURE_TOGGLE_rbd": { 2021-02-10T11:13:33.390 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "FEATURE_TOGGLE_rbd", 2021-02-10T11:13:33.390 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.390 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.390 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.391 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.391 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.391 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.391 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.391 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.391 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.391 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.391 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.391 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.391 INFO:teuthology.orchestra.run.gibba001.stdout: "FEATURE_TOGGLE_rgw": { 2021-02-10T11:13:33.391 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "FEATURE_TOGGLE_rgw", 2021-02-10T11:13:33.391 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.391 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.391 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.392 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.392 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.392 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.392 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.392 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.392 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.392 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.392 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.392 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.392 INFO:teuthology.orchestra.run.gibba001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2021-02-10T11:13:33.392 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2021-02-10T11:13:33.392 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.392 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.392 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.393 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.393 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.393 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.393 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.393 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.393 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.393 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.393 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.393 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.393 INFO:teuthology.orchestra.run.gibba001.stdout: "GRAFANA_API_PASSWORD": { 2021-02-10T11:13:33.393 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "GRAFANA_API_PASSWORD", 2021-02-10T11:13:33.393 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.393 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.393 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.394 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "admin", 2021-02-10T11:13:33.394 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.394 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.394 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.394 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.394 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.394 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.394 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.394 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.394 INFO:teuthology.orchestra.run.gibba001.stdout: "GRAFANA_API_SSL_VERIFY": { 2021-02-10T11:13:33.394 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2021-02-10T11:13:33.394 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.394 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.395 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.395 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.395 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.395 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.395 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.395 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.395 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.395 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.395 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.395 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.395 INFO:teuthology.orchestra.run.gibba001.stdout: "GRAFANA_API_URL": { 2021-02-10T11:13:33.395 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "GRAFANA_API_URL", 2021-02-10T11:13:33.395 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.395 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.396 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.396 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.396 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.396 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.396 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.396 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.396 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.396 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.396 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.396 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.396 INFO:teuthology.orchestra.run.gibba001.stdout: "GRAFANA_API_USERNAME": { 2021-02-10T11:13:33.396 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "GRAFANA_API_USERNAME", 2021-02-10T11:13:33.396 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.397 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.397 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.397 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "admin", 2021-02-10T11:13:33.397 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.397 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.397 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.397 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.397 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.397 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.397 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.397 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.397 INFO:teuthology.orchestra.run.gibba001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2021-02-10T11:13:33.397 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2021-02-10T11:13:33.397 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.398 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.398 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.398 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "False", 2021-02-10T11:13:33.398 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.398 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.398 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.398 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.398 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.398 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.398 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.398 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.398 INFO:teuthology.orchestra.run.gibba001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2021-02-10T11:13:33.398 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2021-02-10T11:13:33.399 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.399 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.399 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.399 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.399 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.399 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.399 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.399 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.399 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.399 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.399 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.399 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.399 INFO:teuthology.orchestra.run.gibba001.stdout: "PROMETHEUS_API_HOST": { 2021-02-10T11:13:33.399 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "PROMETHEUS_API_HOST", 2021-02-10T11:13:33.400 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.400 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.400 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.400 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.400 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.400 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.400 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.400 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.400 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.400 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.400 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.400 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.401 INFO:teuthology.orchestra.run.gibba001.stdout: "REST_REQUESTS_TIMEOUT": { 2021-02-10T11:13:33.401 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2021-02-10T11:13:33.401 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.401 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.401 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.401 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "45", 2021-02-10T11:13:33.401 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.401 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.401 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.401 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.401 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.401 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.402 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.402 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.402 INFO:teuthology.orchestra.run.gibba001.stdout: "RGW_API_ACCESS_KEY": { 2021-02-10T11:13:33.402 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "RGW_API_ACCESS_KEY", 2021-02-10T11:13:33.402 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.402 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.402 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.402 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.402 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.402 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.402 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.402 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.402 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.403 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.403 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.403 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.403 INFO:teuthology.orchestra.run.gibba001.stdout: "RGW_API_ADMIN_RESOURCE": { 2021-02-10T11:13:33.403 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2021-02-10T11:13:33.403 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.403 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.403 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.403 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "admin", 2021-02-10T11:13:33.403 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.403 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.403 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.403 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.404 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.404 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.404 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.404 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.404 INFO:teuthology.orchestra.run.gibba001.stdout: "RGW_API_HOST": { 2021-02-10T11:13:33.404 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "RGW_API_HOST", 2021-02-10T11:13:33.404 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.404 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.404 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.404 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.404 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.404 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.404 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.404 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.405 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.405 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.405 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.405 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.405 INFO:teuthology.orchestra.run.gibba001.stdout: "RGW_API_PORT": { 2021-02-10T11:13:33.405 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "RGW_API_PORT", 2021-02-10T11:13:33.405 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.405 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.405 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.405 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "80", 2021-02-10T11:13:33.405 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.405 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.405 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.406 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.406 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.406 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.406 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.406 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.406 INFO:teuthology.orchestra.run.gibba001.stdout: "RGW_API_SCHEME": { 2021-02-10T11:13:33.406 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "RGW_API_SCHEME", 2021-02-10T11:13:33.406 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.406 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.406 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.406 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "http", 2021-02-10T11:13:33.406 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.406 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.407 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.407 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.407 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.407 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.407 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.407 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.407 INFO:teuthology.orchestra.run.gibba001.stdout: "RGW_API_SECRET_KEY": { 2021-02-10T11:13:33.407 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "RGW_API_SECRET_KEY", 2021-02-10T11:13:33.407 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.407 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.407 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.407 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.408 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.408 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.408 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.408 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.408 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.408 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.408 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.408 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.408 INFO:teuthology.orchestra.run.gibba001.stdout: "RGW_API_SSL_VERIFY": { 2021-02-10T11:13:33.408 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "RGW_API_SSL_VERIFY", 2021-02-10T11:13:33.408 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.408 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.408 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.409 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.409 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.409 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.409 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.409 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.409 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.409 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.409 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.409 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.409 INFO:teuthology.orchestra.run.gibba001.stdout: "RGW_API_USER_ID": { 2021-02-10T11:13:33.409 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "RGW_API_USER_ID", 2021-02-10T11:13:33.409 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.409 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.410 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.410 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.410 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.410 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.410 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.410 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.410 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.410 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.410 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.410 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.410 INFO:teuthology.orchestra.run.gibba001.stdout: "crt_file": { 2021-02-10T11:13:33.410 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "crt_file", 2021-02-10T11:13:33.411 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.411 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.411 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.411 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.411 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.411 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.411 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.411 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.411 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.411 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.411 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.411 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.412 INFO:teuthology.orchestra.run.gibba001.stdout: "debug": { 2021-02-10T11:13:33.412 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "debug", 2021-02-10T11:13:33.412 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.412 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.412 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.412 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "False", 2021-02-10T11:13:33.412 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.412 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.412 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.412 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "Enable/disable debug options", 2021-02-10T11:13:33.412 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.412 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.413 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.413 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.413 INFO:teuthology.orchestra.run.gibba001.stdout: "jwt_token_ttl": { 2021-02-10T11:13:33.413 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "jwt_token_ttl", 2021-02-10T11:13:33.413 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.413 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.413 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.413 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "28800", 2021-02-10T11:13:33.413 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.413 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.413 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.413 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.413 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.413 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.414 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.414 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.414 INFO:teuthology.orchestra.run.gibba001.stdout: "key_file": { 2021-02-10T11:13:33.414 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "key_file", 2021-02-10T11:13:33.414 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.414 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.414 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.414 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.414 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.414 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.414 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.414 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.414 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.415 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.415 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.415 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.415 INFO:teuthology.orchestra.run.gibba001.stdout: "password": { 2021-02-10T11:13:33.415 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "password", 2021-02-10T11:13:33.415 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.415 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.415 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.415 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.415 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.415 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.415 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.415 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.415 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.416 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.416 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.416 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.416 INFO:teuthology.orchestra.run.gibba001.stdout: "server_addr": { 2021-02-10T11:13:33.416 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "server_addr", 2021-02-10T11:13:33.416 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.416 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.416 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.416 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "::", 2021-02-10T11:13:33.416 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.416 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.416 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.416 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.416 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.417 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.417 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.417 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.417 INFO:teuthology.orchestra.run.gibba001.stdout: "server_port": { 2021-02-10T11:13:33.417 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "server_port", 2021-02-10T11:13:33.417 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.417 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.417 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.417 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "8080", 2021-02-10T11:13:33.417 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.417 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.417 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.417 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.417 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.418 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.418 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.418 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.418 INFO:teuthology.orchestra.run.gibba001.stdout: "ssl": { 2021-02-10T11:13:33.418 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "ssl", 2021-02-10T11:13:33.418 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.418 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.418 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.418 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.418 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.418 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.418 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.419 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.419 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.419 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.419 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.419 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.419 INFO:teuthology.orchestra.run.gibba001.stdout: "ssl_server_port": { 2021-02-10T11:13:33.419 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "ssl_server_port", 2021-02-10T11:13:33.419 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.419 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.419 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.419 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "8443", 2021-02-10T11:13:33.419 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.420 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.420 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.420 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.420 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.420 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.420 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.420 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.420 INFO:teuthology.orchestra.run.gibba001.stdout: "standby_behaviour": { 2021-02-10T11:13:33.420 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "standby_behaviour", 2021-02-10T11:13:33.420 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.420 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.420 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.420 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "redirect", 2021-02-10T11:13:33.421 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.421 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.421 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [ 2021-02-10T11:13:33.421 INFO:teuthology.orchestra.run.gibba001.stdout: "error", 2021-02-10T11:13:33.421 INFO:teuthology.orchestra.run.gibba001.stdout: "redirect" 2021-02-10T11:13:33.421 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:13:33.421 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.421 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.421 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.421 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.421 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.421 INFO:teuthology.orchestra.run.gibba001.stdout: "standby_error_status_code": { 2021-02-10T11:13:33.422 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "standby_error_status_code", 2021-02-10T11:13:33.422 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.422 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.422 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.422 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "500", 2021-02-10T11:13:33.422 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "400", 2021-02-10T11:13:33.422 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "599", 2021-02-10T11:13:33.422 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.422 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.422 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.422 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.422 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.422 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.423 INFO:teuthology.orchestra.run.gibba001.stdout: "url_prefix": { 2021-02-10T11:13:33.423 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "url_prefix", 2021-02-10T11:13:33.423 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.423 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.423 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.423 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.423 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.423 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.423 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.423 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.423 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.423 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.423 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.424 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.424 INFO:teuthology.orchestra.run.gibba001.stdout: "username": { 2021-02-10T11:13:33.424 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "username", 2021-02-10T11:13:33.424 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.424 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.424 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.424 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.424 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.424 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.424 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.424 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.424 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.424 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.425 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.425 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.425 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.425 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.425 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.425 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "deepsea", 2021-02-10T11:13:33.425 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.425 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.425 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.425 INFO:teuthology.orchestra.run.gibba001.stdout: "salt_api_eauth": { 2021-02-10T11:13:33.425 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "salt_api_eauth", 2021-02-10T11:13:33.425 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.425 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.426 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.426 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "sharedsecret", 2021-02-10T11:13:33.426 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.426 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.426 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.426 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.426 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.426 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.426 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.426 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.426 INFO:teuthology.orchestra.run.gibba001.stdout: "salt_api_password": { 2021-02-10T11:13:33.426 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "salt_api_password", 2021-02-10T11:13:33.427 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.427 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.427 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.427 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.427 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.427 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.427 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.427 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.427 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.427 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.427 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.427 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.428 INFO:teuthology.orchestra.run.gibba001.stdout: "salt_api_url": { 2021-02-10T11:13:33.428 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "salt_api_url", 2021-02-10T11:13:33.428 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.428 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.428 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.428 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.428 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.428 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.428 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.428 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.428 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.428 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.429 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.429 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.429 INFO:teuthology.orchestra.run.gibba001.stdout: "salt_api_username": { 2021-02-10T11:13:33.429 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "salt_api_username", 2021-02-10T11:13:33.429 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.429 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.429 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.429 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.429 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.429 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.429 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.429 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.429 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.429 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.430 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.430 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.430 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.430 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.430 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.430 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "devicehealth", 2021-02-10T11:13:33.430 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.430 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.430 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.430 INFO:teuthology.orchestra.run.gibba001.stdout: "enable_monitoring": { 2021-02-10T11:13:33.430 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "enable_monitoring", 2021-02-10T11:13:33.430 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.430 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.430 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.431 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "False", 2021-02-10T11:13:33.431 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.431 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.431 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.431 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "monitor device health metrics", 2021-02-10T11:13:33.431 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.431 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.431 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.431 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.431 INFO:teuthology.orchestra.run.gibba001.stdout: "mark_out_threshold": { 2021-02-10T11:13:33.431 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "mark_out_threshold", 2021-02-10T11:13:33.431 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.431 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.432 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.432 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "2419200", 2021-02-10T11:13:33.432 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.432 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.432 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.432 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2021-02-10T11:13:33.432 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.432 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.432 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.432 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.432 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": { 2021-02-10T11:13:33.432 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "pool_name", 2021-02-10T11:13:33.432 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.432 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.433 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.433 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "device_health_metrics", 2021-02-10T11:13:33.433 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.433 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.433 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.433 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "name of pool in which to store device health metrics", 2021-02-10T11:13:33.433 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.433 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.433 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.433 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.433 INFO:teuthology.orchestra.run.gibba001.stdout: "retention_period": { 2021-02-10T11:13:33.433 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "retention_period", 2021-02-10T11:13:33.433 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.433 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.434 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.434 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "15552000", 2021-02-10T11:13:33.434 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.434 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.434 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.434 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "how long to retain device health metrics", 2021-02-10T11:13:33.434 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.434 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.434 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.434 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.434 INFO:teuthology.orchestra.run.gibba001.stdout: "scrape_frequency": { 2021-02-10T11:13:33.434 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "scrape_frequency", 2021-02-10T11:13:33.434 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.434 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.435 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.435 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "86400", 2021-02-10T11:13:33.435 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.435 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.435 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.435 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "how frequently to scrape device health metrics", 2021-02-10T11:13:33.435 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.435 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.435 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.435 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.435 INFO:teuthology.orchestra.run.gibba001.stdout: "self_heal": { 2021-02-10T11:13:33.435 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "self_heal", 2021-02-10T11:13:33.436 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.436 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.436 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.436 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.436 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.436 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.436 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.436 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2021-02-10T11:13:33.436 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.436 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.436 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.437 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.437 INFO:teuthology.orchestra.run.gibba001.stdout: "sleep_interval": { 2021-02-10T11:13:33.437 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "sleep_interval", 2021-02-10T11:13:33.437 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.437 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.437 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.437 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "600", 2021-02-10T11:13:33.437 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.437 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.437 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.437 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "how frequently to wake up and check device health", 2021-02-10T11:13:33.437 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.438 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.438 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.438 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.438 INFO:teuthology.orchestra.run.gibba001.stdout: "warn_threshold": { 2021-02-10T11:13:33.438 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "warn_threshold", 2021-02-10T11:13:33.438 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.438 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.438 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.438 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "7257600", 2021-02-10T11:13:33.438 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.438 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.438 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.438 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "raise health warning if OSD may fail before this long", 2021-02-10T11:13:33.439 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.439 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.439 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.439 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.439 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.439 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.439 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.439 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_cloud", 2021-02-10T11:13:33.439 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.439 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.439 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.439 INFO:teuthology.orchestra.run.gibba001.stdout: "diskprediction_cert_context": { 2021-02-10T11:13:33.439 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_cert_context", 2021-02-10T11:13:33.439 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.440 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.440 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.440 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.440 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.440 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.440 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.440 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.440 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.440 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.440 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.440 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.440 INFO:teuthology.orchestra.run.gibba001.stdout: "diskprediction_default_authority": { 2021-02-10T11:13:33.440 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_default_authority", 2021-02-10T11:13:33.441 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.441 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.441 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.441 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "localhost", 2021-02-10T11:13:33.441 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.441 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.441 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.441 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.441 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.441 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.441 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.441 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.441 INFO:teuthology.orchestra.run.gibba001.stdout: "diskprediction_password": { 2021-02-10T11:13:33.442 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_password", 2021-02-10T11:13:33.442 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.442 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.442 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.442 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.442 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.442 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.442 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.442 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.442 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.442 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.442 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.442 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.443 INFO:teuthology.orchestra.run.gibba001.stdout: "diskprediction_port": { 2021-02-10T11:13:33.443 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_port", 2021-02-10T11:13:33.443 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.443 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.443 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.443 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "31400", 2021-02-10T11:13:33.443 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.443 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.443 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.443 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.443 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.443 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.443 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.443 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.444 INFO:teuthology.orchestra.run.gibba001.stdout: "diskprediction_retrieve_prediction_interval": { 2021-02-10T11:13:33.444 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_retrieve_prediction_interval", 2021-02-10T11:13:33.444 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.444 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.444 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.444 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "43200", 2021-02-10T11:13:33.444 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.444 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.444 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.444 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.444 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.444 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.444 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.445 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.445 INFO:teuthology.orchestra.run.gibba001.stdout: "diskprediction_server": { 2021-02-10T11:13:33.445 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_server", 2021-02-10T11:13:33.445 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.445 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.445 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.445 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.445 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.445 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.445 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.445 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.445 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.445 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.445 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.446 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.446 INFO:teuthology.orchestra.run.gibba001.stdout: "diskprediction_ssl_target_name_override": { 2021-02-10T11:13:33.446 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_ssl_target_name_override", 2021-02-10T11:13:33.446 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.446 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.446 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.446 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "localhost", 2021-02-10T11:13:33.446 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.446 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.446 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.446 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.446 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.446 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.447 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.447 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.447 INFO:teuthology.orchestra.run.gibba001.stdout: "diskprediction_upload_metrics_interval": { 2021-02-10T11:13:33.447 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_upload_metrics_interval", 2021-02-10T11:13:33.447 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.447 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.447 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.447 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "600", 2021-02-10T11:13:33.447 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.447 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.447 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.447 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.447 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.447 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.448 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.448 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.448 INFO:teuthology.orchestra.run.gibba001.stdout: "diskprediction_upload_smart_interval": { 2021-02-10T11:13:33.448 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_upload_smart_interval", 2021-02-10T11:13:33.448 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.448 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.448 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.448 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "43200", 2021-02-10T11:13:33.448 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.448 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.448 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.448 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.448 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.448 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.449 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.449 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.449 INFO:teuthology.orchestra.run.gibba001.stdout: "diskprediction_user": { 2021-02-10T11:13:33.449 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_user", 2021-02-10T11:13:33.449 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.449 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.449 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.449 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.449 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.449 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.449 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.450 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.450 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.450 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.450 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.450 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.450 INFO:teuthology.orchestra.run.gibba001.stdout: "sleep_interval": { 2021-02-10T11:13:33.450 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "sleep_interval", 2021-02-10T11:13:33.450 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.450 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.450 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.450 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "600", 2021-02-10T11:13:33.451 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.451 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.451 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.451 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.451 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.451 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.451 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.451 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.451 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.451 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.451 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.451 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_local", 2021-02-10T11:13:33.452 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.452 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.452 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.452 INFO:teuthology.orchestra.run.gibba001.stdout: "predict_interval": { 2021-02-10T11:13:33.452 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "predict_interval", 2021-02-10T11:13:33.452 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.452 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.452 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.452 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "86400", 2021-02-10T11:13:33.452 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.452 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.452 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.452 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.453 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.453 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.453 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.453 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.453 INFO:teuthology.orchestra.run.gibba001.stdout: "sleep_interval": { 2021-02-10T11:13:33.453 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "sleep_interval", 2021-02-10T11:13:33.453 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.453 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.453 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.453 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "600", 2021-02-10T11:13:33.453 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.453 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.454 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.454 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.454 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.454 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.454 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.454 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.454 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.454 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.454 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.454 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "influx", 2021-02-10T11:13:33.454 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": false, 2021-02-10T11:13:33.454 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "influxdb python module not found", 2021-02-10T11:13:33.455 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.455 INFO:teuthology.orchestra.run.gibba001.stdout: "batch_size": { 2021-02-10T11:13:33.455 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "batch_size", 2021-02-10T11:13:33.455 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.455 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.455 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.455 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "5000", 2021-02-10T11:13:33.455 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.455 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.455 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.455 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.455 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.456 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.456 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.456 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.456 INFO:teuthology.orchestra.run.gibba001.stdout: "database": { 2021-02-10T11:13:33.456 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "database", 2021-02-10T11:13:33.456 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.456 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.456 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.456 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "ceph", 2021-02-10T11:13:33.456 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.456 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.456 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.456 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.457 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.457 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.457 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.457 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.457 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": { 2021-02-10T11:13:33.457 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "hostname", 2021-02-10T11:13:33.457 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.457 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.457 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.457 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "None", 2021-02-10T11:13:33.457 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.457 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.457 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.457 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.458 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.458 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.458 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.458 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.458 INFO:teuthology.orchestra.run.gibba001.stdout: "interval": { 2021-02-10T11:13:33.458 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "interval", 2021-02-10T11:13:33.458 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.458 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.458 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.458 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "30", 2021-02-10T11:13:33.458 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.458 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.458 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.459 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.459 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.459 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.459 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.459 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.459 INFO:teuthology.orchestra.run.gibba001.stdout: "password": { 2021-02-10T11:13:33.459 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "password", 2021-02-10T11:13:33.459 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.459 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.459 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.459 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "None", 2021-02-10T11:13:33.459 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.459 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.460 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.460 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.460 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.460 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.460 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.460 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.460 INFO:teuthology.orchestra.run.gibba001.stdout: "port": { 2021-02-10T11:13:33.460 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "port", 2021-02-10T11:13:33.460 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.460 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.460 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.460 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "8086", 2021-02-10T11:13:33.460 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.461 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.461 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.461 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.461 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.461 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.461 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.461 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.461 INFO:teuthology.orchestra.run.gibba001.stdout: "ssl": { 2021-02-10T11:13:33.461 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "ssl", 2021-02-10T11:13:33.461 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.461 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.461 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.461 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "false", 2021-02-10T11:13:33.462 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.462 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.462 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.462 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.462 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.462 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.462 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.462 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.462 INFO:teuthology.orchestra.run.gibba001.stdout: "threads": { 2021-02-10T11:13:33.462 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "threads", 2021-02-10T11:13:33.462 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.462 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.462 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.462 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "5", 2021-02-10T11:13:33.463 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.463 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.463 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.463 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.463 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.463 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.463 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.463 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.463 INFO:teuthology.orchestra.run.gibba001.stdout: "username": { 2021-02-10T11:13:33.463 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "username", 2021-02-10T11:13:33.463 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.463 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "None", 2021-02-10T11:13:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "verify_ssl": { 2021-02-10T11:13:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "verify_ssl", 2021-02-10T11:13:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "true", 2021-02-10T11:13:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "insights", 2021-02-10T11:13:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": {} 2021-02-10T11:13:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "iostat", 2021-02-10T11:13:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": {} 2021-02-10T11:13:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "localpool", 2021-02-10T11:13:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "failure_domain": { 2021-02-10T11:13:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "failure_domain", 2021-02-10T11:13:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "host", 2021-02-10T11:13:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "failure domain for any created local pool", 2021-02-10T11:13:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": { 2021-02-10T11:13:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "min_size", 2021-02-10T11:13:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "default min_size for any created local pool", 2021-02-10T11:13:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "num_rep": { 2021-02-10T11:13:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "num_rep", 2021-02-10T11:13:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "3", 2021-02-10T11:13:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "default replica count for any created local pool", 2021-02-10T11:13:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": { 2021-02-10T11:13:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "pg_num", 2021-02-10T11:13:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "128", 2021-02-10T11:13:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "default pg_num for any created local pool", 2021-02-10T11:13:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: "prefix": { 2021-02-10T11:13:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "prefix", 2021-02-10T11:13:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "name prefix for any created local pool", 2021-02-10T11:13:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "subtree": { 2021-02-10T11:13:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "subtree", 2021-02-10T11:13:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "rack", 2021-02-10T11:13:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "CRUSH level for which to create a local pool", 2021-02-10T11:13:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "orchestrator_cli", 2021-02-10T11:13:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "orchestrator": { 2021-02-10T11:13:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "orchestrator", 2021-02-10T11:13:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "pg_autoscaler", 2021-02-10T11:13:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: "sleep_interval": { 2021-02-10T11:13:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "sleep_interval", 2021-02-10T11:13:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "60", 2021-02-10T11:13:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "progress", 2021-02-10T11:13:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "enabled": { 2021-02-10T11:13:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "enabled", 2021-02-10T11:13:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "max_completed_events": { 2021-02-10T11:13:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "max_completed_events", 2021-02-10T11:13:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "50", 2021-02-10T11:13:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "number of past completed events to remember", 2021-02-10T11:13:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "persist_interval": { 2021-02-10T11:13:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "persist_interval", 2021-02-10T11:13:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "5", 2021-02-10T11:13:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "how frequently to persist completed events", 2021-02-10T11:13:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "prometheus", 2021-02-10T11:13:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd_stats_pools": { 2021-02-10T11:13:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "rbd_stats_pools", 2021-02-10T11:13:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd_stats_pools_refresh_interval": { 2021-02-10T11:13:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "rbd_stats_pools_refresh_interval", 2021-02-10T11:13:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "300", 2021-02-10T11:13:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "scrape_interval": { 2021-02-10T11:13:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "scrape_interval", 2021-02-10T11:13:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: "server_addr": { 2021-02-10T11:13:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "server_addr", 2021-02-10T11:13:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: "server_port": { 2021-02-10T11:13:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "server_port", 2021-02-10T11:13:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: "stale_cache_strategy": { 2021-02-10T11:13:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "stale_cache_strategy", 2021-02-10T11:13:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.487 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.487 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.487 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.487 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.487 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.487 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.487 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.487 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.487 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.487 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.487 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.487 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "rbd_support", 2021-02-10T11:13:33.487 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.488 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.488 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": {} 2021-02-10T11:13:33.488 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.488 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.488 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "restful", 2021-02-10T11:13:33.488 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.488 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.488 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.488 INFO:teuthology.orchestra.run.gibba001.stdout: "key_file": { 2021-02-10T11:13:33.488 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "key_file", 2021-02-10T11:13:33.488 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.488 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.488 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.488 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.489 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.489 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.489 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.489 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.489 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.489 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.489 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.489 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.489 INFO:teuthology.orchestra.run.gibba001.stdout: "server_addr": { 2021-02-10T11:13:33.489 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "server_addr", 2021-02-10T11:13:33.489 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.489 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.489 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.489 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: "server_port": { 2021-02-10T11:13:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "server_port", 2021-02-10T11:13:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "rook", 2021-02-10T11:13:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": {} 2021-02-10T11:13:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "selftest", 2021-02-10T11:13:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: "roption1": { 2021-02-10T11:13:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "roption1", 2021-02-10T11:13:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: "roption2": { 2021-02-10T11:13:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "roption2", 2021-02-10T11:13:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "xyz", 2021-02-10T11:13:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: "rwoption1": { 2021-02-10T11:13:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "rwoption1", 2021-02-10T11:13:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: "rwoption2": { 2021-02-10T11:13:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "rwoption2", 2021-02-10T11:13:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: "rwoption3": { 2021-02-10T11:13:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "rwoption3", 2021-02-10T11:13:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "float", 2021-02-10T11:13:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "rwoption4": { 2021-02-10T11:13:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "rwoption4", 2021-02-10T11:13:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: "rwoption5": { 2021-02-10T11:13:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "rwoption5", 2021-02-10T11:13:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: "rwoption6": { 2021-02-10T11:13:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "rwoption6", 2021-02-10T11:13:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: "testkey": { 2021-02-10T11:13:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "testkey", 2021-02-10T11:13:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: "testlkey": { 2021-02-10T11:13:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "testlkey", 2021-02-10T11:13:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: "testnewline": { 2021-02-10T11:13:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "testnewline", 2021-02-10T11:13:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "ssh", 2021-02-10T11:13:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: "inventory_cache_timeout_min": { 2021-02-10T11:13:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "inventory_cache_timeout_min", 2021-02-10T11:13:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: "ssh_config_file": { 2021-02-10T11:13:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "ssh_config_file", 2021-02-10T11:13:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "status", 2021-02-10T11:13:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": {} 2021-02-10T11:13:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "telegraf", 2021-02-10T11:13:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.508 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.508 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.508 INFO:teuthology.orchestra.run.gibba001.stdout: "address": { 2021-02-10T11:13:33.508 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "address", 2021-02-10T11:13:33.508 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.508 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.508 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.508 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2021-02-10T11:13:33.508 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.508 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.508 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.508 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.508 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: "interval": { 2021-02-10T11:13:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "interval", 2021-02-10T11:13:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "15", 2021-02-10T11:13:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "telemetry", 2021-02-10T11:13:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "channel_basic": { 2021-02-10T11:13:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "channel_basic", 2021-02-10T11:13:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "Share basic cluster information (size, version)", 2021-02-10T11:13:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: "channel_crash": { 2021-02-10T11:13:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "channel_crash", 2021-02-10T11:13:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: "channel_device": { 2021-02-10T11:13:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "channel_device", 2021-02-10T11:13:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: "channel_ident": { 2021-02-10T11:13:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "channel_ident", 2021-02-10T11:13:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "False", 2021-02-10T11:13:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: "contact": { 2021-02-10T11:13:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "contact", 2021-02-10T11:13:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "None", 2021-02-10T11:13:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: "description": { 2021-02-10T11:13:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "description", 2021-02-10T11:13:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "None", 2021-02-10T11:13:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "device_url": { 2021-02-10T11:13:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "device_url", 2021-02-10T11:13:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "https://telemetry.ceph.com/device", 2021-02-10T11:13:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: "enabled": { 2021-02-10T11:13:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "enabled", 2021-02-10T11:13:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "False", 2021-02-10T11:13:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: "interval": { 2021-02-10T11:13:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "interval", 2021-02-10T11:13:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "24", 2021-02-10T11:13:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "8", 2021-02-10T11:13:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: "last_opt_revision": { 2021-02-10T11:13:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "last_opt_revision", 2021-02-10T11:13:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "1", 2021-02-10T11:13:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: "leaderboard": { 2021-02-10T11:13:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "leaderboard", 2021-02-10T11:13:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "False", 2021-02-10T11:13:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: "organization": { 2021-02-10T11:13:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "organization", 2021-02-10T11:13:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "None", 2021-02-10T11:13:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "proxy": { 2021-02-10T11:13:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "proxy", 2021-02-10T11:13:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "None", 2021-02-10T11:13:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: "url": { 2021-02-10T11:13:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "url", 2021-02-10T11:13:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "https://telemetry.ceph.com/report", 2021-02-10T11:13:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "test_orchestrator", 2021-02-10T11:13:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": {} 2021-02-10T11:13:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "volumes", 2021-02-10T11:13:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: "max_concurrent_clones": { 2021-02-10T11:13:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "max_concurrent_clones", 2021-02-10T11:13:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "4", 2021-02-10T11:13:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "Number of asynchronous cloner threads", 2021-02-10T11:13:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "zabbix", 2021-02-10T11:13:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: "identifier": { 2021-02-10T11:13:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "identifier", 2021-02-10T11:13:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: "interval": { 2021-02-10T11:13:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "interval", 2021-02-10T11:13:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "60", 2021-02-10T11:13:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "zabbix_host": { 2021-02-10T11:13:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "zabbix_host", 2021-02-10T11:13:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "None", 2021-02-10T11:13:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: "zabbix_port": { 2021-02-10T11:13:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "zabbix_port", 2021-02-10T11:13:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "10051", 2021-02-10T11:13:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: "zabbix_sender": { 2021-02-10T11:13:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "zabbix_sender", 2021-02-10T11:13:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "/usr/bin/zabbix_sender", 2021-02-10T11:13:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:13:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:13:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: "modules": [ 2021-02-10T11:13:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: "iostat", 2021-02-10T11:13:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: "restful" 2021-02-10T11:13:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:13:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: "available_modules": [ 2021-02-10T11:13:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "alerts", 2021-02-10T11:13:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: "interval": { 2021-02-10T11:13:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "interval", 2021-02-10T11:13:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "60", 2021-02-10T11:13:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "How frequently to reexamine health status", 2021-02-10T11:13:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "smtp_destination": { 2021-02-10T11:13:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "smtp_destination", 2021-02-10T11:13:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "Email address to send alerts to", 2021-02-10T11:13:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: "smtp_from_name": { 2021-02-10T11:13:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "smtp_from_name", 2021-02-10T11:13:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "Ceph", 2021-02-10T11:13:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "Email From: name", 2021-02-10T11:13:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: "smtp_host": { 2021-02-10T11:13:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "smtp_host", 2021-02-10T11:13:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "SMTP server", 2021-02-10T11:13:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: "smtp_password": { 2021-02-10T11:13:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "smtp_password", 2021-02-10T11:13:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "Password to authenticate with", 2021-02-10T11:13:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: "smtp_port": { 2021-02-10T11:13:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "smtp_port", 2021-02-10T11:13:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "465", 2021-02-10T11:13:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "SMTP port", 2021-02-10T11:13:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: "smtp_sender": { 2021-02-10T11:13:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "smtp_sender", 2021-02-10T11:13:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "SMTP envelope sender", 2021-02-10T11:13:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.542 INFO:teuthology.orchestra.run.gibba001.stdout: "smtp_ssl": { 2021-02-10T11:13:33.542 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "smtp_ssl", 2021-02-10T11:13:33.542 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.542 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.542 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.542 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.542 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.542 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.542 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.542 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "Use SSL to connect to SMTP server", 2021-02-10T11:13:33.542 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.542 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.543 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.543 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.543 INFO:teuthology.orchestra.run.gibba001.stdout: "smtp_user": { 2021-02-10T11:13:33.543 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "smtp_user", 2021-02-10T11:13:33.543 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.543 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.543 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.543 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.543 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.543 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.543 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.543 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "User to authenticate as", 2021-02-10T11:13:33.543 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.544 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.544 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.544 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.544 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.544 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.544 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.544 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "ansible", 2021-02-10T11:13:33.544 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.544 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.544 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.544 INFO:teuthology.orchestra.run.gibba001.stdout: "password": { 2021-02-10T11:13:33.544 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "password", 2021-02-10T11:13:33.545 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.545 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.545 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.545 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.545 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.545 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.545 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.545 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.545 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.545 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.545 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.545 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.546 INFO:teuthology.orchestra.run.gibba001.stdout: "server_url": { 2021-02-10T11:13:33.546 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "server_url", 2021-02-10T11:13:33.546 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.546 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.546 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.546 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.546 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.546 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.546 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.546 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.546 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.546 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.546 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.547 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.547 INFO:teuthology.orchestra.run.gibba001.stdout: "username": { 2021-02-10T11:13:33.547 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "username", 2021-02-10T11:13:33.547 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.547 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.547 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.547 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.547 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.547 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.547 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.547 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.547 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.548 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.548 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.548 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.548 INFO:teuthology.orchestra.run.gibba001.stdout: "verify_server": { 2021-02-10T11:13:33.548 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "verify_server", 2021-02-10T11:13:33.548 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.548 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.548 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.548 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.548 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.548 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.548 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.549 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.549 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.549 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.549 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.549 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.549 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.549 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.549 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.549 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "balancer", 2021-02-10T11:13:33.549 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.549 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.549 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.549 INFO:teuthology.orchestra.run.gibba001.stdout: "active": { 2021-02-10T11:13:33.550 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "active", 2021-02-10T11:13:33.550 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.550 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.550 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.550 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "False", 2021-02-10T11:13:33.550 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.550 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.550 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.550 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "automatically balance PGs across cluster", 2021-02-10T11:13:33.550 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.550 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.550 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.550 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.551 INFO:teuthology.orchestra.run.gibba001.stdout: "begin_time": { 2021-02-10T11:13:33.551 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "begin_time", 2021-02-10T11:13:33.551 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.551 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.551 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.551 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "0000", 2021-02-10T11:13:33.551 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.551 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.551 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.551 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "beginning time of day to automatically balance", 2021-02-10T11:13:33.551 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2021-02-10T11:13:33.551 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.551 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.551 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.552 INFO:teuthology.orchestra.run.gibba001.stdout: "begin_weekday": { 2021-02-10T11:13:33.552 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "begin_weekday", 2021-02-10T11:13:33.552 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "uint", 2021-02-10T11:13:33.552 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.552 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.552 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "0", 2021-02-10T11:13:33.552 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "0", 2021-02-10T11:13:33.552 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "7", 2021-02-10T11:13:33.552 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.552 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2021-02-10T11:13:33.552 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2021-02-10T11:13:33.552 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.552 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_compat_max_iterations": { 2021-02-10T11:13:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "crush_compat_max_iterations", 2021-02-10T11:13:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "uint", 2021-02-10T11:13:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "25", 2021-02-10T11:13:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "1", 2021-02-10T11:13:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "250", 2021-02-10T11:13:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "maximum number of iterations to attempt optimization", 2021-02-10T11:13:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_compat_metrics": { 2021-02-10T11:13:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "crush_compat_metrics", 2021-02-10T11:13:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "pgs,objects,bytes", 2021-02-10T11:13:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "metrics with which to calculate OSD utilization", 2021-02-10T11:13:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2021-02-10T11:13:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_compat_step": { 2021-02-10T11:13:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "crush_compat_step", 2021-02-10T11:13:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "float", 2021-02-10T11:13:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "0.5", 2021-02-10T11:13:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "0.001", 2021-02-10T11:13:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "0.999", 2021-02-10T11:13:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "aggressiveness of optimization", 2021-02-10T11:13:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2021-02-10T11:13:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: "end_time": { 2021-02-10T11:13:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "end_time", 2021-02-10T11:13:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "2400", 2021-02-10T11:13:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "ending time of day to automatically balance", 2021-02-10T11:13:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2021-02-10T11:13:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "end_weekday": { 2021-02-10T11:13:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "end_weekday", 2021-02-10T11:13:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "uint", 2021-02-10T11:13:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "7", 2021-02-10T11:13:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "0", 2021-02-10T11:13:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "7", 2021-02-10T11:13:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2021-02-10T11:13:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2021-02-10T11:13:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: "min_score": { 2021-02-10T11:13:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "min_score", 2021-02-10T11:13:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "float", 2021-02-10T11:13:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "0", 2021-02-10T11:13:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "minimum score, below which no optimization is attempted", 2021-02-10T11:13:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: "mode": { 2021-02-10T11:13:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "mode", 2021-02-10T11:13:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "none", 2021-02-10T11:13:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [ 2021-02-10T11:13:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: "crush-compat", 2021-02-10T11:13:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: "none", 2021-02-10T11:13:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: "upmap" 2021-02-10T11:13:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:13:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "Balancer mode", 2021-02-10T11:13:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_ids": { 2021-02-10T11:13:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "pool_ids", 2021-02-10T11:13:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "pools which the automatic balancing will be limited to", 2021-02-10T11:13:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: "sleep_interval": { 2021-02-10T11:13:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "sleep_interval", 2021-02-10T11:13:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "60", 2021-02-10T11:13:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "how frequently to wake up and attempt optimization", 2021-02-10T11:13:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: "upmap_max_deviation": { 2021-02-10T11:13:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "upmap_max_deviation", 2021-02-10T11:13:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "5", 2021-02-10T11:13:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "1", 2021-02-10T11:13:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "deviation below which no optimization is attempted", 2021-02-10T11:13:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2021-02-10T11:13:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: "upmap_max_iterations": { 2021-02-10T11:13:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "upmap_max_iterations", 2021-02-10T11:13:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "uint", 2021-02-10T11:13:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "10", 2021-02-10T11:13:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.564 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.564 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.564 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "maximum upmap optimization iterations", 2021-02-10T11:13:33.564 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.564 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.564 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.564 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.564 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.564 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.564 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.564 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "crash", 2021-02-10T11:13:33.564 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.564 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.565 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.565 INFO:teuthology.orchestra.run.gibba001.stdout: "retain_interval": { 2021-02-10T11:13:33.565 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "retain_interval", 2021-02-10T11:13:33.565 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.565 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.565 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.565 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "31536000", 2021-02-10T11:13:33.565 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.565 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.565 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.565 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "how long to retain crashes before pruning them", 2021-02-10T11:13:33.565 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.565 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.565 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.566 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.566 INFO:teuthology.orchestra.run.gibba001.stdout: "warn_recent_interval": { 2021-02-10T11:13:33.566 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "warn_recent_interval", 2021-02-10T11:13:33.566 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.566 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.566 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.566 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "1209600", 2021-02-10T11:13:33.566 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.566 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.566 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.566 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "time interval in which to warn about recent crashes", 2021-02-10T11:13:33.566 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.566 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.567 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.567 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.567 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.567 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.567 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.567 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "dashboard", 2021-02-10T11:13:33.567 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.567 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.567 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.567 INFO:teuthology.orchestra.run.gibba001.stdout: "ALERTMANAGER_API_HOST": { 2021-02-10T11:13:33.567 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "ALERTMANAGER_API_HOST", 2021-02-10T11:13:33.567 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.567 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.568 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.568 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.568 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.568 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.568 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.568 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.568 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.568 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.568 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.568 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.568 INFO:teuthology.orchestra.run.gibba001.stdout: "AUDIT_API_ENABLED": { 2021-02-10T11:13:33.568 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "AUDIT_API_ENABLED", 2021-02-10T11:13:33.568 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.568 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.569 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.569 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "False", 2021-02-10T11:13:33.569 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.569 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.569 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.569 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.569 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.569 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.569 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.569 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.569 INFO:teuthology.orchestra.run.gibba001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2021-02-10T11:13:33.569 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2021-02-10T11:13:33.569 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.570 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.570 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.570 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.570 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.570 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.570 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.570 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.570 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.570 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.570 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.570 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.570 INFO:teuthology.orchestra.run.gibba001.stdout: "ENABLE_BROWSABLE_API": { 2021-02-10T11:13:33.570 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "ENABLE_BROWSABLE_API", 2021-02-10T11:13:33.571 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.571 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.571 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.571 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.571 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.571 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.571 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.571 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.571 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.571 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.571 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.571 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.571 INFO:teuthology.orchestra.run.gibba001.stdout: "FEATURE_TOGGLE_cephfs": { 2021-02-10T11:13:33.572 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "FEATURE_TOGGLE_cephfs", 2021-02-10T11:13:33.572 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.572 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.572 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.572 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.572 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.572 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.572 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.572 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.572 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.572 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.573 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.573 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.573 INFO:teuthology.orchestra.run.gibba001.stdout: "FEATURE_TOGGLE_iscsi": { 2021-02-10T11:13:33.573 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "FEATURE_TOGGLE_iscsi", 2021-02-10T11:13:33.573 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.573 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.573 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.573 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.573 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.573 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.573 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.573 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.573 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.574 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.574 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.574 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.574 INFO:teuthology.orchestra.run.gibba001.stdout: "FEATURE_TOGGLE_mirroring": { 2021-02-10T11:13:33.574 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "FEATURE_TOGGLE_mirroring", 2021-02-10T11:13:33.574 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.574 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.574 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.574 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.574 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.574 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.574 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.575 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.575 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.575 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.575 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.575 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.575 INFO:teuthology.orchestra.run.gibba001.stdout: "FEATURE_TOGGLE_rbd": { 2021-02-10T11:13:33.575 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "FEATURE_TOGGLE_rbd", 2021-02-10T11:13:33.575 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.575 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.575 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.575 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.575 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.576 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.576 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.576 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.576 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.576 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.576 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.576 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.576 INFO:teuthology.orchestra.run.gibba001.stdout: "FEATURE_TOGGLE_rgw": { 2021-02-10T11:13:33.576 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "FEATURE_TOGGLE_rgw", 2021-02-10T11:13:33.576 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.576 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.576 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.576 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.577 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.577 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.577 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.577 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.577 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.577 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.577 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.577 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.577 INFO:teuthology.orchestra.run.gibba001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2021-02-10T11:13:33.577 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2021-02-10T11:13:33.577 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.577 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.577 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.578 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.578 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.578 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.578 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.578 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.578 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.578 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.578 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.578 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.578 INFO:teuthology.orchestra.run.gibba001.stdout: "GRAFANA_API_PASSWORD": { 2021-02-10T11:13:33.578 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "GRAFANA_API_PASSWORD", 2021-02-10T11:13:33.578 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.578 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.579 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.579 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "admin", 2021-02-10T11:13:33.579 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.579 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.579 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.579 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.579 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.579 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.579 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.579 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.579 INFO:teuthology.orchestra.run.gibba001.stdout: "GRAFANA_API_SSL_VERIFY": { 2021-02-10T11:13:33.579 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2021-02-10T11:13:33.579 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.579 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.580 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.580 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.580 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.580 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.580 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.580 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.580 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.580 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.580 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.580 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.580 INFO:teuthology.orchestra.run.gibba001.stdout: "GRAFANA_API_URL": { 2021-02-10T11:13:33.580 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "GRAFANA_API_URL", 2021-02-10T11:13:33.580 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.581 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.581 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.581 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.581 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.581 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.581 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.581 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.581 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.581 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.581 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.581 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.581 INFO:teuthology.orchestra.run.gibba001.stdout: "GRAFANA_API_USERNAME": { 2021-02-10T11:13:33.581 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "GRAFANA_API_USERNAME", 2021-02-10T11:13:33.581 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.582 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.582 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.582 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "admin", 2021-02-10T11:13:33.582 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.582 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.582 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.582 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.582 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.582 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.582 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.582 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.582 INFO:teuthology.orchestra.run.gibba001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2021-02-10T11:13:33.582 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2021-02-10T11:13:33.583 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.583 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.583 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.583 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "False", 2021-02-10T11:13:33.583 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.583 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.583 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.583 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.583 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.583 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.583 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.583 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.583 INFO:teuthology.orchestra.run.gibba001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2021-02-10T11:13:33.583 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2021-02-10T11:13:33.584 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.584 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.584 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.584 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.584 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.584 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.584 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.584 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.584 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.584 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.584 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.584 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.584 INFO:teuthology.orchestra.run.gibba001.stdout: "PROMETHEUS_API_HOST": { 2021-02-10T11:13:33.584 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "PROMETHEUS_API_HOST", 2021-02-10T11:13:33.585 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.585 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.585 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.585 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.585 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.585 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.585 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.585 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.585 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.585 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.585 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.585 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.585 INFO:teuthology.orchestra.run.gibba001.stdout: "REST_REQUESTS_TIMEOUT": { 2021-02-10T11:13:33.585 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2021-02-10T11:13:33.586 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.586 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.586 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.586 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "45", 2021-02-10T11:13:33.586 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.586 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.586 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.586 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.586 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.586 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.586 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.586 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.586 INFO:teuthology.orchestra.run.gibba001.stdout: "RGW_API_ACCESS_KEY": { 2021-02-10T11:13:33.587 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "RGW_API_ACCESS_KEY", 2021-02-10T11:13:33.587 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.587 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.587 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.587 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.587 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.587 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.587 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.587 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.587 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.587 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.587 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.587 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.588 INFO:teuthology.orchestra.run.gibba001.stdout: "RGW_API_ADMIN_RESOURCE": { 2021-02-10T11:13:33.588 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2021-02-10T11:13:33.588 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.588 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.588 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.588 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "admin", 2021-02-10T11:13:33.588 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.588 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.588 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.588 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.588 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.588 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.588 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.588 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.589 INFO:teuthology.orchestra.run.gibba001.stdout: "RGW_API_HOST": { 2021-02-10T11:13:33.589 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "RGW_API_HOST", 2021-02-10T11:13:33.589 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.589 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.589 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.589 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.589 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.589 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.589 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.589 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.589 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.589 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.589 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.590 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.590 INFO:teuthology.orchestra.run.gibba001.stdout: "RGW_API_PORT": { 2021-02-10T11:13:33.590 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "RGW_API_PORT", 2021-02-10T11:13:33.590 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.590 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.590 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.590 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "80", 2021-02-10T11:13:33.590 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.590 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.590 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.590 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.590 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.590 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.590 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.591 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.591 INFO:teuthology.orchestra.run.gibba001.stdout: "RGW_API_SCHEME": { 2021-02-10T11:13:33.591 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "RGW_API_SCHEME", 2021-02-10T11:13:33.591 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.591 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.591 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.591 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "http", 2021-02-10T11:13:33.591 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.591 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.591 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.591 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.591 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.591 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.591 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.592 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.592 INFO:teuthology.orchestra.run.gibba001.stdout: "RGW_API_SECRET_KEY": { 2021-02-10T11:13:33.592 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "RGW_API_SECRET_KEY", 2021-02-10T11:13:33.592 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.592 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.592 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.592 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.592 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.592 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.592 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.592 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.592 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.592 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.593 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.593 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.593 INFO:teuthology.orchestra.run.gibba001.stdout: "RGW_API_SSL_VERIFY": { 2021-02-10T11:13:33.593 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "RGW_API_SSL_VERIFY", 2021-02-10T11:13:33.593 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.593 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.593 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.593 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.593 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.593 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.593 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.593 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.593 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.594 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.594 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.594 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.594 INFO:teuthology.orchestra.run.gibba001.stdout: "RGW_API_USER_ID": { 2021-02-10T11:13:33.594 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "RGW_API_USER_ID", 2021-02-10T11:13:33.594 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.594 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.594 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.594 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.594 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.594 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.594 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.594 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.595 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.595 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.595 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.595 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.595 INFO:teuthology.orchestra.run.gibba001.stdout: "crt_file": { 2021-02-10T11:13:33.595 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "crt_file", 2021-02-10T11:13:33.595 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.595 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.595 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.595 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.595 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.595 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.595 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.595 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.596 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.596 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.596 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.596 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.596 INFO:teuthology.orchestra.run.gibba001.stdout: "debug": { 2021-02-10T11:13:33.596 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "debug", 2021-02-10T11:13:33.596 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.596 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.596 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.596 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "False", 2021-02-10T11:13:33.596 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.596 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.596 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.597 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "Enable/disable debug options", 2021-02-10T11:13:33.597 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.597 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.597 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.597 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.597 INFO:teuthology.orchestra.run.gibba001.stdout: "jwt_token_ttl": { 2021-02-10T11:13:33.597 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "jwt_token_ttl", 2021-02-10T11:13:33.597 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.597 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.597 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.597 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "28800", 2021-02-10T11:13:33.597 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.597 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.597 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.598 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.598 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.598 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.598 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.598 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.598 INFO:teuthology.orchestra.run.gibba001.stdout: "key_file": { 2021-02-10T11:13:33.598 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "key_file", 2021-02-10T11:13:33.598 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.598 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.598 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.598 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.598 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.598 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.599 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.599 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.599 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.599 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.599 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.599 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.599 INFO:teuthology.orchestra.run.gibba001.stdout: "password": { 2021-02-10T11:13:33.599 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "password", 2021-02-10T11:13:33.599 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.599 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.599 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.599 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.599 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.600 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.600 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.600 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.600 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.600 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.600 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.600 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.600 INFO:teuthology.orchestra.run.gibba001.stdout: "server_addr": { 2021-02-10T11:13:33.600 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "server_addr", 2021-02-10T11:13:33.600 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.600 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.600 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.600 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "::", 2021-02-10T11:13:33.601 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.601 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.601 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.601 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.601 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.601 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.601 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.601 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.601 INFO:teuthology.orchestra.run.gibba001.stdout: "server_port": { 2021-02-10T11:13:33.601 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "server_port", 2021-02-10T11:13:33.601 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.601 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.602 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.602 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "8080", 2021-02-10T11:13:33.602 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.602 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.602 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.602 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.602 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.602 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.602 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.602 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.602 INFO:teuthology.orchestra.run.gibba001.stdout: "ssl": { 2021-02-10T11:13:33.602 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "ssl", 2021-02-10T11:13:33.602 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.603 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.603 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.603 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.603 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.603 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.603 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.603 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.603 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.603 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.603 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.603 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.603 INFO:teuthology.orchestra.run.gibba001.stdout: "ssl_server_port": { 2021-02-10T11:13:33.603 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "ssl_server_port", 2021-02-10T11:13:33.604 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.604 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.604 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.604 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "8443", 2021-02-10T11:13:33.604 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.604 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.604 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.604 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.604 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.604 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.604 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.604 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.605 INFO:teuthology.orchestra.run.gibba001.stdout: "standby_behaviour": { 2021-02-10T11:13:33.605 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "standby_behaviour", 2021-02-10T11:13:33.605 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.605 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.605 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.605 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "redirect", 2021-02-10T11:13:33.605 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.605 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.605 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [ 2021-02-10T11:13:33.605 INFO:teuthology.orchestra.run.gibba001.stdout: "error", 2021-02-10T11:13:33.605 INFO:teuthology.orchestra.run.gibba001.stdout: "redirect" 2021-02-10T11:13:33.605 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:13:33.605 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.606 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.606 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.606 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.606 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.606 INFO:teuthology.orchestra.run.gibba001.stdout: "standby_error_status_code": { 2021-02-10T11:13:33.606 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "standby_error_status_code", 2021-02-10T11:13:33.606 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.606 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.606 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.606 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "500", 2021-02-10T11:13:33.606 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "400", 2021-02-10T11:13:33.607 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "599", 2021-02-10T11:13:33.607 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.607 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.607 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.607 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.607 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.607 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.607 INFO:teuthology.orchestra.run.gibba001.stdout: "url_prefix": { 2021-02-10T11:13:33.607 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "url_prefix", 2021-02-10T11:13:33.607 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.607 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.607 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.608 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.608 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.608 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.608 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.608 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.608 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.608 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.608 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.608 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.608 INFO:teuthology.orchestra.run.gibba001.stdout: "username": { 2021-02-10T11:13:33.608 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "username", 2021-02-10T11:13:33.608 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.608 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.609 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.609 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.609 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.609 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.609 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.609 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.609 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.609 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.609 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.609 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.609 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.609 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.610 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.610 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "deepsea", 2021-02-10T11:13:33.610 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.610 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.610 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.610 INFO:teuthology.orchestra.run.gibba001.stdout: "salt_api_eauth": { 2021-02-10T11:13:33.610 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "salt_api_eauth", 2021-02-10T11:13:33.610 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.610 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.610 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.610 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "sharedsecret", 2021-02-10T11:13:33.610 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.610 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.611 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.611 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.611 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.611 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.611 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.611 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.611 INFO:teuthology.orchestra.run.gibba001.stdout: "salt_api_password": { 2021-02-10T11:13:33.611 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "salt_api_password", 2021-02-10T11:13:33.611 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.611 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.611 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.611 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.612 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.612 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.612 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.612 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.612 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.612 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.612 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.612 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.612 INFO:teuthology.orchestra.run.gibba001.stdout: "salt_api_url": { 2021-02-10T11:13:33.612 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "salt_api_url", 2021-02-10T11:13:33.612 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.612 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.612 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.613 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.613 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.613 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.613 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.613 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.613 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.613 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.613 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.613 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.613 INFO:teuthology.orchestra.run.gibba001.stdout: "salt_api_username": { 2021-02-10T11:13:33.613 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "salt_api_username", 2021-02-10T11:13:33.613 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.614 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.614 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.614 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.614 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.614 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.614 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.614 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.614 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.614 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.614 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.614 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.614 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.615 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.615 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.615 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "devicehealth", 2021-02-10T11:13:33.615 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.615 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.615 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.615 INFO:teuthology.orchestra.run.gibba001.stdout: "enable_monitoring": { 2021-02-10T11:13:33.615 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "enable_monitoring", 2021-02-10T11:13:33.615 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.615 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.615 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.615 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "False", 2021-02-10T11:13:33.616 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.616 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.616 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.616 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "monitor device health metrics", 2021-02-10T11:13:33.616 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.616 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.616 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.616 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.616 INFO:teuthology.orchestra.run.gibba001.stdout: "mark_out_threshold": { 2021-02-10T11:13:33.616 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "mark_out_threshold", 2021-02-10T11:13:33.616 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.616 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.616 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.617 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "2419200", 2021-02-10T11:13:33.617 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.617 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.617 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.617 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2021-02-10T11:13:33.617 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.617 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.617 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.617 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.617 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": { 2021-02-10T11:13:33.617 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "pool_name", 2021-02-10T11:13:33.617 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.617 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.618 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.618 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "device_health_metrics", 2021-02-10T11:13:33.618 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.618 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.618 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.618 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "name of pool in which to store device health metrics", 2021-02-10T11:13:33.618 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.618 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.618 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.618 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.618 INFO:teuthology.orchestra.run.gibba001.stdout: "retention_period": { 2021-02-10T11:13:33.618 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "retention_period", 2021-02-10T11:13:33.619 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.619 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.619 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.619 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "15552000", 2021-02-10T11:13:33.619 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.619 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.619 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.619 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "how long to retain device health metrics", 2021-02-10T11:13:33.619 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.619 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.619 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.619 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.619 INFO:teuthology.orchestra.run.gibba001.stdout: "scrape_frequency": { 2021-02-10T11:13:33.620 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "scrape_frequency", 2021-02-10T11:13:33.620 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.620 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.620 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.620 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "86400", 2021-02-10T11:13:33.620 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.620 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.620 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.620 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "how frequently to scrape device health metrics", 2021-02-10T11:13:33.620 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.620 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.620 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.620 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.621 INFO:teuthology.orchestra.run.gibba001.stdout: "self_heal": { 2021-02-10T11:13:33.621 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "self_heal", 2021-02-10T11:13:33.621 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.621 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.621 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.621 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.621 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.621 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.621 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.621 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2021-02-10T11:13:33.621 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.621 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.621 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.622 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.622 INFO:teuthology.orchestra.run.gibba001.stdout: "sleep_interval": { 2021-02-10T11:13:33.622 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "sleep_interval", 2021-02-10T11:13:33.622 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.622 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.622 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.622 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "600", 2021-02-10T11:13:33.622 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.622 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.622 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.622 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "how frequently to wake up and check device health", 2021-02-10T11:13:33.622 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.623 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.623 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.623 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.623 INFO:teuthology.orchestra.run.gibba001.stdout: "warn_threshold": { 2021-02-10T11:13:33.623 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "warn_threshold", 2021-02-10T11:13:33.623 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.623 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.623 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.623 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "7257600", 2021-02-10T11:13:33.623 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.623 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.623 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.623 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "raise health warning if OSD may fail before this long", 2021-02-10T11:13:33.624 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.624 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.624 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.624 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.624 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.624 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.624 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.624 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_cloud", 2021-02-10T11:13:33.627 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.627 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.627 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.627 INFO:teuthology.orchestra.run.gibba001.stdout: "diskprediction_cert_context": { 2021-02-10T11:13:33.627 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_cert_context", 2021-02-10T11:13:33.627 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.627 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.627 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.627 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.627 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.627 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.628 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.628 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.628 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.628 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.628 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.628 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.628 INFO:teuthology.orchestra.run.gibba001.stdout: "diskprediction_default_authority": { 2021-02-10T11:13:33.628 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_default_authority", 2021-02-10T11:13:33.628 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.628 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.628 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.628 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "localhost", 2021-02-10T11:13:33.628 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.629 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.629 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.629 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.629 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.629 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.629 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.629 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.629 INFO:teuthology.orchestra.run.gibba001.stdout: "diskprediction_password": { 2021-02-10T11:13:33.629 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_password", 2021-02-10T11:13:33.629 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.629 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.629 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.629 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.630 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.630 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.630 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.630 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.630 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.630 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.630 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.630 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.630 INFO:teuthology.orchestra.run.gibba001.stdout: "diskprediction_port": { 2021-02-10T11:13:33.630 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_port", 2021-02-10T11:13:33.630 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.630 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.631 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.631 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "31400", 2021-02-10T11:13:33.631 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.631 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.631 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.631 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.631 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.631 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.631 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.631 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.631 INFO:teuthology.orchestra.run.gibba001.stdout: "diskprediction_retrieve_prediction_interval": { 2021-02-10T11:13:33.631 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_retrieve_prediction_interval", 2021-02-10T11:13:33.631 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.631 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.632 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.632 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "43200", 2021-02-10T11:13:33.632 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.632 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.632 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.632 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.632 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.632 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.632 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.632 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.632 INFO:teuthology.orchestra.run.gibba001.stdout: "diskprediction_server": { 2021-02-10T11:13:33.632 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_server", 2021-02-10T11:13:33.632 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.632 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.633 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.633 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.633 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.633 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.633 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.633 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.633 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.633 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.633 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.633 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.633 INFO:teuthology.orchestra.run.gibba001.stdout: "diskprediction_ssl_target_name_override": { 2021-02-10T11:13:33.633 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_ssl_target_name_override", 2021-02-10T11:13:33.633 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.634 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.634 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.634 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "localhost", 2021-02-10T11:13:33.634 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.634 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.634 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.634 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.634 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.634 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.634 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.634 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.634 INFO:teuthology.orchestra.run.gibba001.stdout: "diskprediction_upload_metrics_interval": { 2021-02-10T11:13:33.634 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_upload_metrics_interval", 2021-02-10T11:13:33.634 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.635 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.635 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.635 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "600", 2021-02-10T11:13:33.635 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.635 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.635 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.635 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.635 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.636 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.636 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.636 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.636 INFO:teuthology.orchestra.run.gibba001.stdout: "diskprediction_upload_smart_interval": { 2021-02-10T11:13:33.636 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_upload_smart_interval", 2021-02-10T11:13:33.636 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.636 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.636 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.636 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "43200", 2021-02-10T11:13:33.637 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.637 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.637 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.637 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.637 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.637 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.637 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.637 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.637 INFO:teuthology.orchestra.run.gibba001.stdout: "diskprediction_user": { 2021-02-10T11:13:33.638 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_user", 2021-02-10T11:13:33.638 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.638 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.638 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.638 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.638 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.638 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.638 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.638 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.638 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.638 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.638 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.639 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.639 INFO:teuthology.orchestra.run.gibba001.stdout: "sleep_interval": { 2021-02-10T11:13:33.639 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "sleep_interval", 2021-02-10T11:13:33.639 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.639 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.639 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.639 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "600", 2021-02-10T11:13:33.639 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.639 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.639 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.639 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.639 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.639 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.639 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.640 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.640 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.640 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.640 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.640 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "diskprediction_local", 2021-02-10T11:13:33.640 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.640 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.640 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.640 INFO:teuthology.orchestra.run.gibba001.stdout: "predict_interval": { 2021-02-10T11:13:33.640 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "predict_interval", 2021-02-10T11:13:33.640 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.640 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.640 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.640 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "86400", 2021-02-10T11:13:33.641 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.641 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.641 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.641 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.641 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.641 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.641 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.641 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.641 INFO:teuthology.orchestra.run.gibba001.stdout: "sleep_interval": { 2021-02-10T11:13:33.641 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "sleep_interval", 2021-02-10T11:13:33.641 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.641 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.641 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.641 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "600", 2021-02-10T11:13:33.642 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.642 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.642 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.642 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.642 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.642 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.642 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.642 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.642 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.642 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.642 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.642 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "influx", 2021-02-10T11:13:33.642 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": false, 2021-02-10T11:13:33.643 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "influxdb python module not found", 2021-02-10T11:13:33.643 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.643 INFO:teuthology.orchestra.run.gibba001.stdout: "batch_size": { 2021-02-10T11:13:33.643 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "batch_size", 2021-02-10T11:13:33.643 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.643 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.643 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.643 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "5000", 2021-02-10T11:13:33.643 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.643 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.643 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.643 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.643 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.643 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.644 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.644 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.644 INFO:teuthology.orchestra.run.gibba001.stdout: "database": { 2021-02-10T11:13:33.644 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "database", 2021-02-10T11:13:33.644 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.644 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.644 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.644 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "ceph", 2021-02-10T11:13:33.644 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.644 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.644 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.644 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.644 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.644 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.645 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.645 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.645 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": { 2021-02-10T11:13:33.645 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "hostname", 2021-02-10T11:13:33.645 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.645 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.645 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.645 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "None", 2021-02-10T11:13:33.645 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.645 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.645 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.645 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.645 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.646 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.646 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.646 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.646 INFO:teuthology.orchestra.run.gibba001.stdout: "interval": { 2021-02-10T11:13:33.646 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "interval", 2021-02-10T11:13:33.646 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.646 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.646 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.646 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "30", 2021-02-10T11:13:33.647 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.647 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.647 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.647 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.647 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.647 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.648 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.648 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.648 INFO:teuthology.orchestra.run.gibba001.stdout: "password": { 2021-02-10T11:13:33.648 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "password", 2021-02-10T11:13:33.648 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.648 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.648 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.648 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "None", 2021-02-10T11:13:33.648 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.648 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.648 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.648 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.649 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.649 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.649 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.649 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.649 INFO:teuthology.orchestra.run.gibba001.stdout: "port": { 2021-02-10T11:13:33.649 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "port", 2021-02-10T11:13:33.649 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.649 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.649 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.649 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "8086", 2021-02-10T11:13:33.649 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.649 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.649 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.650 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.650 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.650 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.650 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.650 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.650 INFO:teuthology.orchestra.run.gibba001.stdout: "ssl": { 2021-02-10T11:13:33.650 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "ssl", 2021-02-10T11:13:33.650 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.650 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.650 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.650 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "false", 2021-02-10T11:13:33.650 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.650 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.650 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.651 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.651 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.651 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.651 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.651 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.651 INFO:teuthology.orchestra.run.gibba001.stdout: "threads": { 2021-02-10T11:13:33.651 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "threads", 2021-02-10T11:13:33.651 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.651 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.651 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.651 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "5", 2021-02-10T11:13:33.651 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.651 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.652 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.652 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.652 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.652 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.652 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.652 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.652 INFO:teuthology.orchestra.run.gibba001.stdout: "username": { 2021-02-10T11:13:33.652 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "username", 2021-02-10T11:13:33.652 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.652 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.652 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.652 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "None", 2021-02-10T11:13:33.652 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.652 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.653 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.653 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.653 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.653 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.653 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.653 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.653 INFO:teuthology.orchestra.run.gibba001.stdout: "verify_ssl": { 2021-02-10T11:13:33.653 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "verify_ssl", 2021-02-10T11:13:33.653 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.653 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.653 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.654 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "true", 2021-02-10T11:13:33.654 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.654 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.654 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.654 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.654 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.654 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.654 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.654 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.655 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.655 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.655 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.655 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "insights", 2021-02-10T11:13:33.655 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.655 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.655 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": {} 2021-02-10T11:13:33.655 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.655 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.655 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "iostat", 2021-02-10T11:13:33.655 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.655 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.656 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": {} 2021-02-10T11:13:33.656 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.656 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.656 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "localpool", 2021-02-10T11:13:33.656 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.656 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.656 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.656 INFO:teuthology.orchestra.run.gibba001.stdout: "failure_domain": { 2021-02-10T11:13:33.656 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "failure_domain", 2021-02-10T11:13:33.657 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.657 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.657 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.657 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "host", 2021-02-10T11:13:33.657 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.657 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.657 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.657 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "failure domain for any created local pool", 2021-02-10T11:13:33.657 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.658 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.658 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.658 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.658 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": { 2021-02-10T11:13:33.658 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "min_size", 2021-02-10T11:13:33.658 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.658 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.658 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.659 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.659 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.659 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.659 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.659 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "default min_size for any created local pool", 2021-02-10T11:13:33.659 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.659 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.659 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.659 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.659 INFO:teuthology.orchestra.run.gibba001.stdout: "num_rep": { 2021-02-10T11:13:33.659 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "num_rep", 2021-02-10T11:13:33.659 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.659 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.660 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.660 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "3", 2021-02-10T11:13:33.660 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.660 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.660 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.660 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "default replica count for any created local pool", 2021-02-10T11:13:33.660 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.660 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.660 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.660 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.660 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": { 2021-02-10T11:13:33.660 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "pg_num", 2021-02-10T11:13:33.660 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.660 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.661 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.661 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "128", 2021-02-10T11:13:33.661 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.661 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.661 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.661 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "default pg_num for any created local pool", 2021-02-10T11:13:33.661 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.661 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.661 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.661 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.661 INFO:teuthology.orchestra.run.gibba001.stdout: "prefix": { 2021-02-10T11:13:33.661 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "prefix", 2021-02-10T11:13:33.661 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.661 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.662 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.662 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.662 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.662 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.662 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.662 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "name prefix for any created local pool", 2021-02-10T11:13:33.662 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.662 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.662 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.662 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.663 INFO:teuthology.orchestra.run.gibba001.stdout: "subtree": { 2021-02-10T11:13:33.663 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "subtree", 2021-02-10T11:13:33.663 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.663 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.663 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.663 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "rack", 2021-02-10T11:13:33.663 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.663 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.663 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.663 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "CRUSH level for which to create a local pool", 2021-02-10T11:13:33.663 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.663 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.663 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.664 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.664 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.664 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.664 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.664 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "orchestrator_cli", 2021-02-10T11:13:33.664 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.664 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.664 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.664 INFO:teuthology.orchestra.run.gibba001.stdout: "orchestrator": { 2021-02-10T11:13:33.664 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "orchestrator", 2021-02-10T11:13:33.664 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.664 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.664 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.664 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.665 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.665 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.665 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.665 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.665 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.665 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.665 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.665 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.665 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.665 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.665 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.665 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "pg_autoscaler", 2021-02-10T11:13:33.665 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.666 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.666 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.666 INFO:teuthology.orchestra.run.gibba001.stdout: "sleep_interval": { 2021-02-10T11:13:33.666 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "sleep_interval", 2021-02-10T11:13:33.666 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.666 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.666 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.666 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "60", 2021-02-10T11:13:33.666 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.666 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.666 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.666 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.666 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.666 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.667 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.667 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.667 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.667 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.667 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.667 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "progress", 2021-02-10T11:13:33.667 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.667 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.667 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.667 INFO:teuthology.orchestra.run.gibba001.stdout: "enabled": { 2021-02-10T11:13:33.667 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "enabled", 2021-02-10T11:13:33.667 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.667 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.667 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.668 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.668 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.668 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.668 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.668 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.668 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.668 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.668 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.668 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.668 INFO:teuthology.orchestra.run.gibba001.stdout: "max_completed_events": { 2021-02-10T11:13:33.668 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "max_completed_events", 2021-02-10T11:13:33.668 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.668 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.669 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.669 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "50", 2021-02-10T11:13:33.669 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.669 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.669 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.669 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "number of past completed events to remember", 2021-02-10T11:13:33.669 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.669 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.669 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.669 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.669 INFO:teuthology.orchestra.run.gibba001.stdout: "persist_interval": { 2021-02-10T11:13:33.669 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "persist_interval", 2021-02-10T11:13:33.670 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.670 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.670 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:13:33.670 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "5", 2021-02-10T11:13:33.670 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.670 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.670 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.671 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "how frequently to persist completed events", 2021-02-10T11:13:33.671 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.671 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.671 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.671 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.671 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.671 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.671 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.671 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "prometheus", 2021-02-10T11:13:33.671 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.671 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.672 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.672 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd_stats_pools": { 2021-02-10T11:13:33.672 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "rbd_stats_pools", 2021-02-10T11:13:33.672 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.672 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.672 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.672 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.672 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.672 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.672 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.672 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.672 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.672 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.672 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.673 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.673 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd_stats_pools_refresh_interval": { 2021-02-10T11:13:33.673 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "rbd_stats_pools_refresh_interval", 2021-02-10T11:13:33.673 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.673 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.673 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.673 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "300", 2021-02-10T11:13:33.673 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.673 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.673 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.673 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.673 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.673 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.673 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.674 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.674 INFO:teuthology.orchestra.run.gibba001.stdout: "scrape_interval": { 2021-02-10T11:13:33.674 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "scrape_interval", 2021-02-10T11:13:33.674 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.674 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.674 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.674 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.674 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.674 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.674 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.674 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.674 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.675 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.675 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.675 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.675 INFO:teuthology.orchestra.run.gibba001.stdout: "server_addr": { 2021-02-10T11:13:33.675 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "server_addr", 2021-02-10T11:13:33.675 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.675 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.675 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.675 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.675 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.675 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.675 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.675 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.676 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.676 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.676 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.676 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.676 INFO:teuthology.orchestra.run.gibba001.stdout: "server_port": { 2021-02-10T11:13:33.676 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "server_port", 2021-02-10T11:13:33.676 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.676 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.676 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.676 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.676 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.676 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.676 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.676 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.677 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.677 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.677 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.677 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.677 INFO:teuthology.orchestra.run.gibba001.stdout: "stale_cache_strategy": { 2021-02-10T11:13:33.677 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "stale_cache_strategy", 2021-02-10T11:13:33.677 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.677 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.677 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.677 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.677 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.677 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.677 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.677 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.678 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.678 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.678 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.678 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.678 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.678 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.678 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.678 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "rbd_support", 2021-02-10T11:13:33.678 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.678 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.678 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": {} 2021-02-10T11:13:33.678 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.678 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.678 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "restful", 2021-02-10T11:13:33.679 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.679 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.679 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.679 INFO:teuthology.orchestra.run.gibba001.stdout: "key_file": { 2021-02-10T11:13:33.679 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "key_file", 2021-02-10T11:13:33.679 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.679 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.679 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.679 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.679 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.679 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.679 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.679 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.679 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.680 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.680 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.680 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.680 INFO:teuthology.orchestra.run.gibba001.stdout: "server_addr": { 2021-02-10T11:13:33.680 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "server_addr", 2021-02-10T11:13:33.680 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.680 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.680 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.680 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.680 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.681 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.681 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.681 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.681 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.681 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.681 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.681 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.681 INFO:teuthology.orchestra.run.gibba001.stdout: "server_port": { 2021-02-10T11:13:33.681 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "server_port", 2021-02-10T11:13:33.681 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.681 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.681 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.682 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.682 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.682 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.682 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.682 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.682 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.682 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.682 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.682 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.682 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.682 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.682 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.682 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "rook", 2021-02-10T11:13:33.682 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.683 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.683 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": {} 2021-02-10T11:13:33.683 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.683 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.683 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "selftest", 2021-02-10T11:13:33.683 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.683 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.683 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.683 INFO:teuthology.orchestra.run.gibba001.stdout: "roption1": { 2021-02-10T11:13:33.683 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "roption1", 2021-02-10T11:13:33.683 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.683 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.683 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.684 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.684 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.684 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.684 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.684 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.684 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.684 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.684 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.684 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.684 INFO:teuthology.orchestra.run.gibba001.stdout: "roption2": { 2021-02-10T11:13:33.684 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "roption2", 2021-02-10T11:13:33.684 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.684 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.685 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.685 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "xyz", 2021-02-10T11:13:33.685 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.685 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.685 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.685 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.685 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.685 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.685 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.685 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.686 INFO:teuthology.orchestra.run.gibba001.stdout: "rwoption1": { 2021-02-10T11:13:33.686 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "rwoption1", 2021-02-10T11:13:33.686 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.686 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.686 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.686 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.686 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.687 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.687 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.687 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.687 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.687 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.687 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.687 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.688 INFO:teuthology.orchestra.run.gibba001.stdout: "rwoption2": { 2021-02-10T11:13:33.688 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "rwoption2", 2021-02-10T11:13:33.688 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.688 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.688 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.688 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.688 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.688 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.688 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.689 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.689 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.689 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.689 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.689 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.689 INFO:teuthology.orchestra.run.gibba001.stdout: "rwoption3": { 2021-02-10T11:13:33.689 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "rwoption3", 2021-02-10T11:13:33.689 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "float", 2021-02-10T11:13:33.689 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.689 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.689 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.690 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.690 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.690 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.690 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.690 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.690 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.690 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.690 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.690 INFO:teuthology.orchestra.run.gibba001.stdout: "rwoption4": { 2021-02-10T11:13:33.690 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "rwoption4", 2021-02-10T11:13:33.690 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.690 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.691 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.691 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.691 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.691 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.691 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.691 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.691 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.691 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.691 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.691 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.691 INFO:teuthology.orchestra.run.gibba001.stdout: "rwoption5": { 2021-02-10T11:13:33.691 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "rwoption5", 2021-02-10T11:13:33.691 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.692 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.692 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.692 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.692 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.692 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.692 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.692 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.692 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.692 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.692 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.692 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.692 INFO:teuthology.orchestra.run.gibba001.stdout: "rwoption6": { 2021-02-10T11:13:33.692 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "rwoption6", 2021-02-10T11:13:33.692 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.692 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.693 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.693 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.693 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.693 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.693 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.693 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.693 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.693 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.693 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.693 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.693 INFO:teuthology.orchestra.run.gibba001.stdout: "testkey": { 2021-02-10T11:13:33.693 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "testkey", 2021-02-10T11:13:33.693 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.694 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.694 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.694 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.694 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.694 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.694 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.694 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.694 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.694 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.694 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.695 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.695 INFO:teuthology.orchestra.run.gibba001.stdout: "testlkey": { 2021-02-10T11:13:33.695 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "testlkey", 2021-02-10T11:13:33.695 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.695 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.695 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.695 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.695 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.695 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.696 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.696 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.696 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.696 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.696 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.696 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.696 INFO:teuthology.orchestra.run.gibba001.stdout: "testnewline": { 2021-02-10T11:13:33.697 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "testnewline", 2021-02-10T11:13:33.697 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.697 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.697 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.697 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.697 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.697 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.697 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.697 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.697 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.697 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.697 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.697 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.697 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.698 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.698 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.698 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "ssh", 2021-02-10T11:13:33.698 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.698 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.698 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.698 INFO:teuthology.orchestra.run.gibba001.stdout: "inventory_cache_timeout_min": { 2021-02-10T11:13:33.698 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "inventory_cache_timeout_min", 2021-02-10T11:13:33.698 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.698 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.699 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.699 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.699 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.699 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.699 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.699 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.699 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.699 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.700 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.700 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.700 INFO:teuthology.orchestra.run.gibba001.stdout: "ssh_config_file": { 2021-02-10T11:13:33.700 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "ssh_config_file", 2021-02-10T11:13:33.700 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.700 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.700 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.700 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.700 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.701 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.701 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.701 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.701 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.701 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.702 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.702 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.702 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.702 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.702 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.702 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "status", 2021-02-10T11:13:33.702 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.702 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.702 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": {} 2021-02-10T11:13:33.703 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.703 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.703 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "telegraf", 2021-02-10T11:13:33.703 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.703 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.703 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.703 INFO:teuthology.orchestra.run.gibba001.stdout: "address": { 2021-02-10T11:13:33.703 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "address", 2021-02-10T11:13:33.703 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.703 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.703 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.703 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2021-02-10T11:13:33.704 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.704 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.704 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.704 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.704 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.704 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.704 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.704 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.704 INFO:teuthology.orchestra.run.gibba001.stdout: "interval": { 2021-02-10T11:13:33.704 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "interval", 2021-02-10T11:13:33.704 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.704 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.704 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.705 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "15", 2021-02-10T11:13:33.705 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.705 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.705 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.705 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.705 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.705 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.705 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.705 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.705 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.705 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.705 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.705 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "telemetry", 2021-02-10T11:13:33.705 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.706 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.706 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.706 INFO:teuthology.orchestra.run.gibba001.stdout: "channel_basic": { 2021-02-10T11:13:33.706 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "channel_basic", 2021-02-10T11:13:33.706 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.706 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.706 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.706 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.707 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.707 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.707 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.707 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "Share basic cluster information (size, version)", 2021-02-10T11:13:33.707 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.707 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.707 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.707 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.707 INFO:teuthology.orchestra.run.gibba001.stdout: "channel_crash": { 2021-02-10T11:13:33.707 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "channel_crash", 2021-02-10T11:13:33.707 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.708 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.708 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.708 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.708 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.708 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.708 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.708 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.708 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.708 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.708 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.708 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.708 INFO:teuthology.orchestra.run.gibba001.stdout: "channel_device": { 2021-02-10T11:13:33.708 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "channel_device", 2021-02-10T11:13:33.709 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.709 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.709 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.709 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "True", 2021-02-10T11:13:33.709 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.709 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.709 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.709 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.709 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.709 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.709 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.709 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.710 INFO:teuthology.orchestra.run.gibba001.stdout: "channel_ident": { 2021-02-10T11:13:33.710 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "channel_ident", 2021-02-10T11:13:33.710 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.710 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.710 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.710 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "False", 2021-02-10T11:13:33.710 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.710 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.711 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.711 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.711 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.711 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.711 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.711 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.711 INFO:teuthology.orchestra.run.gibba001.stdout: "contact": { 2021-02-10T11:13:33.711 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "contact", 2021-02-10T11:13:33.711 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.711 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.712 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.712 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "None", 2021-02-10T11:13:33.712 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.712 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.712 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.712 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.712 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.712 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.713 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.713 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.713 INFO:teuthology.orchestra.run.gibba001.stdout: "description": { 2021-02-10T11:13:33.713 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "description", 2021-02-10T11:13:33.713 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.713 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.713 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.713 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "None", 2021-02-10T11:13:33.713 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.713 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.714 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.714 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.714 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.714 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.714 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.714 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.714 INFO:teuthology.orchestra.run.gibba001.stdout: "device_url": { 2021-02-10T11:13:33.714 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "device_url", 2021-02-10T11:13:33.714 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.714 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.714 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.714 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "https://telemetry.ceph.com/device", 2021-02-10T11:13:33.714 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.715 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.715 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.715 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.715 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.715 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.715 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.715 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.715 INFO:teuthology.orchestra.run.gibba001.stdout: "enabled": { 2021-02-10T11:13:33.715 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "enabled", 2021-02-10T11:13:33.715 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.715 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.715 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.715 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "False", 2021-02-10T11:13:33.716 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.716 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.716 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.716 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.716 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.716 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.716 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.716 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.716 INFO:teuthology.orchestra.run.gibba001.stdout: "interval": { 2021-02-10T11:13:33.716 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "interval", 2021-02-10T11:13:33.717 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.717 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.717 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.717 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "24", 2021-02-10T11:13:33.717 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "8", 2021-02-10T11:13:33.717 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.717 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.717 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.717 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.717 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.717 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.717 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.718 INFO:teuthology.orchestra.run.gibba001.stdout: "last_opt_revision": { 2021-02-10T11:13:33.718 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "last_opt_revision", 2021-02-10T11:13:33.718 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.718 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.718 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.718 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "1", 2021-02-10T11:13:33.718 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.718 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.718 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.719 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.719 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.719 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.719 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.719 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.719 INFO:teuthology.orchestra.run.gibba001.stdout: "leaderboard": { 2021-02-10T11:13:33.719 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "leaderboard", 2021-02-10T11:13:33.719 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "bool", 2021-02-10T11:13:33.719 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.719 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.719 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "False", 2021-02-10T11:13:33.720 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.720 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.720 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.720 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.720 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.720 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.720 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.720 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.720 INFO:teuthology.orchestra.run.gibba001.stdout: "organization": { 2021-02-10T11:13:33.720 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "organization", 2021-02-10T11:13:33.720 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.720 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.720 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.720 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "None", 2021-02-10T11:13:33.721 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.721 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.721 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.721 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.721 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.721 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.721 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.722 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.722 INFO:teuthology.orchestra.run.gibba001.stdout: "proxy": { 2021-02-10T11:13:33.722 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "proxy", 2021-02-10T11:13:33.722 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.722 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.722 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.722 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "None", 2021-02-10T11:13:33.722 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.722 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.722 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.723 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.723 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.723 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.723 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.723 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.723 INFO:teuthology.orchestra.run.gibba001.stdout: "url": { 2021-02-10T11:13:33.723 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "url", 2021-02-10T11:13:33.724 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.724 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.724 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.724 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "https://telemetry.ceph.com/report", 2021-02-10T11:13:33.724 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.724 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.724 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.725 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.725 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.725 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.725 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.725 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.725 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.725 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.725 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.725 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "test_orchestrator", 2021-02-10T11:13:33.725 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.725 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.726 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": {} 2021-02-10T11:13:33.726 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.726 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.726 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "volumes", 2021-02-10T11:13:33.726 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.726 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.726 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.726 INFO:teuthology.orchestra.run.gibba001.stdout: "max_concurrent_clones": { 2021-02-10T11:13:33.726 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "max_concurrent_clones", 2021-02-10T11:13:33.726 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.726 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.726 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.726 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "4", 2021-02-10T11:13:33.727 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.727 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.727 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.727 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "Number of asynchronous cloner threads", 2021-02-10T11:13:33.727 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.727 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.727 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.728 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.728 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.728 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.728 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:13:33.728 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "zabbix", 2021-02-10T11:13:33.728 INFO:teuthology.orchestra.run.gibba001.stdout: "can_run": true, 2021-02-10T11:13:33.728 INFO:teuthology.orchestra.run.gibba001.stdout: "error_string": "", 2021-02-10T11:13:33.728 INFO:teuthology.orchestra.run.gibba001.stdout: "module_options": { 2021-02-10T11:13:33.728 INFO:teuthology.orchestra.run.gibba001.stdout: "identifier": { 2021-02-10T11:13:33.729 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "identifier", 2021-02-10T11:13:33.729 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.729 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.729 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.729 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "", 2021-02-10T11:13:33.729 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.729 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.729 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.729 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.729 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.730 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.730 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.730 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.730 INFO:teuthology.orchestra.run.gibba001.stdout: "interval": { 2021-02-10T11:13:33.730 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "interval", 2021-02-10T11:13:33.730 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "secs", 2021-02-10T11:13:33.730 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.730 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.730 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "60", 2021-02-10T11:13:33.730 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.730 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.730 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.731 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.731 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.731 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.731 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.731 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.731 INFO:teuthology.orchestra.run.gibba001.stdout: "zabbix_host": { 2021-02-10T11:13:33.731 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "zabbix_host", 2021-02-10T11:13:33.731 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.731 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.731 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.732 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "None", 2021-02-10T11:13:33.732 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.732 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.732 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.732 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.732 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.732 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.732 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.732 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.732 INFO:teuthology.orchestra.run.gibba001.stdout: "zabbix_port": { 2021-02-10T11:13:33.732 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "zabbix_port", 2021-02-10T11:13:33.732 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "int", 2021-02-10T11:13:33.733 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.733 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.733 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "10051", 2021-02-10T11:13:33.733 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.733 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.733 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.733 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.733 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.733 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.733 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.733 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.733 INFO:teuthology.orchestra.run.gibba001.stdout: "zabbix_sender": { 2021-02-10T11:13:33.733 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "zabbix_sender", 2021-02-10T11:13:33.733 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "str", 2021-02-10T11:13:33.734 INFO:teuthology.orchestra.run.gibba001.stdout: "level": "advanced", 2021-02-10T11:13:33.734 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 0, 2021-02-10T11:13:33.734 INFO:teuthology.orchestra.run.gibba001.stdout: "default_value": "/usr/bin/zabbix_sender", 2021-02-10T11:13:33.734 INFO:teuthology.orchestra.run.gibba001.stdout: "min": "", 2021-02-10T11:13:33.734 INFO:teuthology.orchestra.run.gibba001.stdout: "max": "", 2021-02-10T11:13:33.734 INFO:teuthology.orchestra.run.gibba001.stdout: "enum_allowed": [], 2021-02-10T11:13:33.734 INFO:teuthology.orchestra.run.gibba001.stdout: "desc": "", 2021-02-10T11:13:33.734 INFO:teuthology.orchestra.run.gibba001.stdout: "long_desc": "", 2021-02-10T11:13:33.734 INFO:teuthology.orchestra.run.gibba001.stdout: "tags": [], 2021-02-10T11:13:33.734 INFO:teuthology.orchestra.run.gibba001.stdout: "see_also": [] 2021-02-10T11:13:33.734 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.734 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.734 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.735 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:13:33.735 INFO:teuthology.orchestra.run.gibba001.stdout: "services": {}, 2021-02-10T11:13:33.735 INFO:teuthology.orchestra.run.gibba001.stdout: "always_on_modules": { 2021-02-10T11:13:33.735 INFO:teuthology.orchestra.run.gibba001.stdout: "nautilus": [ 2021-02-10T11:13:33.735 INFO:teuthology.orchestra.run.gibba001.stdout: "balancer", 2021-02-10T11:13:33.735 INFO:teuthology.orchestra.run.gibba001.stdout: "crash", 2021-02-10T11:13:33.735 INFO:teuthology.orchestra.run.gibba001.stdout: "devicehealth", 2021-02-10T11:13:33.735 INFO:teuthology.orchestra.run.gibba001.stdout: "orchestrator_cli", 2021-02-10T11:13:33.735 INFO:teuthology.orchestra.run.gibba001.stdout: "progress", 2021-02-10T11:13:33.735 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd_support", 2021-02-10T11:13:33.735 INFO:teuthology.orchestra.run.gibba001.stdout: "status", 2021-02-10T11:13:33.735 INFO:teuthology.orchestra.run.gibba001.stdout: "volumes" 2021-02-10T11:13:33.735 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:13:33.735 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.736 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.736 INFO:teuthology.orchestra.run.gibba001.stdout: "servicemap": { 2021-02-10T11:13:33.736 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 2, 2021-02-10T11:13:33.736 INFO:teuthology.orchestra.run.gibba001.stdout: "modified": "2021-02-10 11:13:28.137169", 2021-02-10T11:13:33.736 INFO:teuthology.orchestra.run.gibba001.stdout: "services": { 2021-02-10T11:13:33.736 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": { 2021-02-10T11:13:33.736 INFO:teuthology.orchestra.run.gibba001.stdout: "daemons": { 2021-02-10T11:13:33.736 INFO:teuthology.orchestra.run.gibba001.stdout: "summary": "", 2021-02-10T11:13:33.736 INFO:teuthology.orchestra.run.gibba001.stdout: "a": { 2021-02-10T11:13:33.737 INFO:teuthology.orchestra.run.gibba001.stdout: "start_epoch": 0, 2021-02-10T11:13:33.737 INFO:teuthology.orchestra.run.gibba001.stdout: "start_stamp": "0.000000", 2021-02-10T11:13:33.737 INFO:teuthology.orchestra.run.gibba001.stdout: "gid": 0, 2021-02-10T11:13:33.737 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": ":/0", 2021-02-10T11:13:33.737 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": {}, 2021-02-10T11:13:33.737 INFO:teuthology.orchestra.run.gibba001.stdout: "task_status": { 2021-02-10T11:13:33.737 INFO:teuthology.orchestra.run.gibba001.stdout: "scrub status": "idle" 2021-02-10T11:13:33.737 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.737 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.737 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.738 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.738 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:33.738 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:33.738 INFO:teuthology.orchestra.run.gibba001.stdout: "progress_events": {} 2021-02-10T11:13:33.738 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:13:33.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.336 7f9cdd533700 1 -- 172.21.2.101:0/1140528604 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9cbc0186c0 msgr2=0x7f9cbc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:33.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.336 7f9cdd533700 1 --2- 172.21.2.101:0/1140528604 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9cbc0186c0 0x7f9cbc01abc0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:33.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.336 7f9cdd533700 1 -- 172.21.2.101:0/1140528604 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9cd8093db0 msgr2=0x7f9cd81985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:33.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.336 7f9cdd533700 1 --2- 172.21.2.101:0/1140528604 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9cd8093db0 0x7f9cd81985b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f9cc8006f20 tx=0x7f9cc800b990).stop 2021-02-10T11:13:33.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.337 7f9cdd533700 1 -- 172.21.2.101:0/1140528604 shutdown_connections 2021-02-10T11:13:33.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.337 7f9cdd533700 1 --2- 172.21.2.101:0/1140528604 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9cbc0186c0 0x7f9cbc01abc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:33.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.337 7f9cdd533700 1 --2- 172.21.2.101:0/1140528604 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9cd80aa760 0x7f9cd81a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:33.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.337 7f9cdd533700 1 --2- 172.21.2.101:0/1140528604 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9cd80aa360 0x7f9cd81a5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:33.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.337 7f9cdd533700 1 --2- 172.21.2.101:0/1140528604 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9cd8093db0 0x7f9cd81985b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:33.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.337 7f9cdd533700 1 -- 172.21.2.101:0/1140528604 >> 172.21.2.101:0/1140528604 conn(0x7f9cd819f7c0 msgr2=0x7f9cd818c230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:33.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.337 7f9cdd533700 1 -- 172.21.2.101:0/1140528604 shutdown_connections 2021-02-10T11:13:33.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:33.337 7f9cdd533700 1 -- 172.21.2.101:0/1140528604 wait complete. 2021-02-10T11:13:33.741 INFO:tasks.ceph.ceph_manager.ceph:{'fsid': 'a6663ba9-2653-4542-8b6a-9323b4e7b545', 'health': {'checks': {}, 'status': 'HEALTH_OK'}, 'election_epoch': 4, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 56, 'monmap': {'epoch': 1, 'min_mon_release_name': '14', 'num_mons': 3}, 'osdmap': {'osdmap': {'epoch': 20, 'num_osds': 8, 'num_up_osds': 8, 'num_in_osds': 8, 'num_remapped_pgs': 0}}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 24}], 'num_pgs': 24, 'num_pools': 3, 'num_objects': 22, 'data_bytes': 2286, 'bytes_used': 8591704064, 'bytes_avail': 764502409216, 'bytes_total': 773094113280, 'write_bytes_sec': 1279, 'read_op_per_sec': 0, 'write_op_per_sec': 4}, 'fsmap': {'epoch': 6, 'id': 1, 'up': 1, 'in': 1, 'max': 1, 'by_rank': [{'filesystem_id': 1, 'rank': 0, 'name': 'a', 'status': 'up:active', 'gid': 4299}], 'up:standby': 1}, 'mgrmap': {'epoch': 3, 'active_gid': 4100, 'active_name': 'y', 'active_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.101:6800', 'nonce': 24165}, {'type': 'v1', 'addr': '172.21.2.101:6801', 'nonce': 24165}]}, 'active_addr': '172.21.2.101:6801/24165', 'active_change': '2021-02-10 11:12:38.124918', 'available': True, 'standbys': [{'gid': 4106, 'name': 'x', 'available_modules': [{'name': 'alerts', 'can_run': True, 'error_string': '', 'module_options': {'interval': {'name': 'interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'How frequently to reexamine health status', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_destination': {'name': 'smtp_destination', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Email address to send alerts to', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_from_name': {'name': 'smtp_from_name', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'Ceph', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Email From: name', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_host': {'name': 'smtp_host', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'SMTP server', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_password': {'name': 'smtp_password', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Password to authenticate with', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_port': {'name': 'smtp_port', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '465', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'SMTP port', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_sender': {'name': 'smtp_sender', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'SMTP envelope sender', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_ssl': {'name': 'smtp_ssl', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Use SSL to connect to SMTP server', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_user': {'name': 'smtp_user', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'User to authenticate as', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'ansible', 'can_run': True, 'error_string': '', 'module_options': {'password': {'name': 'password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_url': {'name': 'server_url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'username': {'name': 'username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'verify_server': {'name': 'verify_server', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'balancer', 'can_run': True, 'error_string': '', 'module_options': {'active': {'name': 'active', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'automatically balance PGs across cluster', 'long_desc': '', 'tags': [], 'see_also': []}, 'begin_time': {'name': 'begin_time', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '0000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'beginning time of day to automatically balance', 'long_desc': 'This is a time of day in the format HHMM.', 'tags': [], 'see_also': []}, 'begin_weekday': {'name': 'begin_weekday', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '0', 'min': '0', 'max': '7', 'enum_allowed': [], 'desc': 'Restrict automatic balancing to this day of the week or later', 'long_desc': '0 or 7 = Sunday, 1 = Monday, etc.', 'tags': [], 'see_also': []}, 'crush_compat_max_iterations': {'name': 'crush_compat_max_iterations', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '25', 'min': '1', 'max': '250', 'enum_allowed': [], 'desc': 'maximum number of iterations to attempt optimization', 'long_desc': '', 'tags': [], 'see_also': []}, 'crush_compat_metrics': {'name': 'crush_compat_metrics', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'pgs,objects,bytes', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'metrics with which to calculate OSD utilization', 'long_desc': 'Value is a list of one or more of "pgs", "objects", or "bytes", and indicates which metrics to use to balance utilization.', 'tags': [], 'see_also': []}, 'crush_compat_step': {'name': 'crush_compat_step', 'type': 'float', 'level': 'advanced', 'flags': 1, 'default_value': '0.5', 'min': '0.001', 'max': '0.999', 'enum_allowed': [], 'desc': 'aggressiveness of optimization', 'long_desc': '.99 is very aggressive, .01 is less aggressive', 'tags': [], 'see_also': []}, 'end_time': {'name': 'end_time', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '2400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'ending time of day to automatically balance', 'long_desc': 'This is a time of day in the format HHMM.', 'tags': [], 'see_also': []}, 'end_weekday': {'name': 'end_weekday', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '7', 'min': '0', 'max': '7', 'enum_allowed': [], 'desc': 'Restrict automatic balancing to days of the week earlier than this', 'long_desc': '0 or 7 = Sunday, 1 = Monday, etc.', 'tags': [], 'see_also': []}, 'min_score': {'name': 'min_score', 'type': 'float', 'level': 'advanced', 'flags': 1, 'default_value': '0', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'minimum score, below which no optimization is attempted', 'long_desc': '', 'tags': [], 'see_also': []}, 'mode': {'name': 'mode', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'none', 'min': '', 'max': '', 'enum_allowed': ['crush-compat', 'none', 'upmap'], 'desc': 'Balancer mode', 'long_desc': '', 'tags': [], 'see_also': []}, 'pool_ids': {'name': 'pool_ids', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'pools which the automatic balancing will be limited to', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to wake up and attempt optimization', 'long_desc': '', 'tags': [], 'see_also': []}, 'upmap_max_deviation': {'name': 'upmap_max_deviation', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '5', 'min': '1', 'max': '', 'enum_allowed': [], 'desc': 'deviation below which no optimization is attempted', 'long_desc': 'If the number of PGs are within this count then no optimization is attempted', 'tags': [], 'see_also': []}, 'upmap_max_iterations': {'name': 'upmap_max_iterations', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '10', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'maximum upmap optimization iterations', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'crash', 'can_run': True, 'error_string': '', 'module_options': {'retain_interval': {'name': 'retain_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '31536000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how long to retain crashes before pruning them', 'long_desc': '', 'tags': [], 'see_also': []}, 'warn_recent_interval': {'name': 'warn_recent_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '1209600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'time interval in which to warn about recent crashes', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'dashboard', 'can_run': True, 'error_string': '', 'module_options': {'ALERTMANAGER_API_HOST': {'name': 'ALERTMANAGER_API_HOST', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'AUDIT_API_ENABLED': {'name': 'AUDIT_API_ENABLED', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'AUDIT_API_LOG_PAYLOAD': {'name': 'AUDIT_API_LOG_PAYLOAD', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ENABLE_BROWSABLE_API': {'name': 'ENABLE_BROWSABLE_API', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_cephfs': {'name': 'FEATURE_TOGGLE_cephfs', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_iscsi': {'name': 'FEATURE_TOGGLE_iscsi', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_mirroring': {'name': 'FEATURE_TOGGLE_mirroring', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_rbd': {'name': 'FEATURE_TOGGLE_rbd', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_rgw': {'name': 'FEATURE_TOGGLE_rgw', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE': {'name': 'GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_PASSWORD': {'name': 'GRAFANA_API_PASSWORD', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'admin', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_SSL_VERIFY': {'name': 'GRAFANA_API_SSL_VERIFY', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_URL': {'name': 'GRAFANA_API_URL', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_USERNAME': {'name': 'GRAFANA_API_USERNAME', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'admin', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_UPDATE_DASHBOARDS': {'name': 'GRAFANA_UPDATE_DASHBOARDS', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ISCSI_API_SSL_VERIFICATION': {'name': 'ISCSI_API_SSL_VERIFICATION', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'PROMETHEUS_API_HOST': {'name': 'PROMETHEUS_API_HOST', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'REST_REQUESTS_TIMEOUT': {'name': 'REST_REQUESTS_TIMEOUT', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '45', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_ACCESS_KEY': {'name': 'RGW_API_ACCESS_KEY', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_ADMIN_RESOURCE': {'name': 'RGW_API_ADMIN_RESOURCE', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'admin', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_HOST': {'name': 'RGW_API_HOST', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_PORT': {'name': 'RGW_API_PORT', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '80', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_SCHEME': {'name': 'RGW_API_SCHEME', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'http', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_SECRET_KEY': {'name': 'RGW_API_SECRET_KEY', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_SSL_VERIFY': {'name': 'RGW_API_SSL_VERIFY', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_USER_ID': {'name': 'RGW_API_USER_ID', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'crt_file': {'name': 'crt_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'debug': {'name': 'debug', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Enable/disable debug options', 'long_desc': '', 'tags': [], 'see_also': []}, 'jwt_token_ttl': {'name': 'jwt_token_ttl', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '28800', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'key_file': {'name': 'key_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'password': {'name': 'password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_addr': {'name': 'server_addr', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '::', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_port': {'name': 'server_port', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '8080', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssl': {'name': 'ssl', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssl_server_port': {'name': 'ssl_server_port', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '8443', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'standby_behaviour': {'name': 'standby_behaviour', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'redirect', 'min': '', 'max': '', 'enum_allowed': ['error', 'redirect'], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'standby_error_status_code': {'name': 'standby_error_status_code', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '500', 'min': '400', 'max': '599', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'url_prefix': {'name': 'url_prefix', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'username': {'name': 'username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'deepsea', 'can_run': True, 'error_string': '', 'module_options': {'salt_api_eauth': {'name': 'salt_api_eauth', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'sharedsecret', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'salt_api_password': {'name': 'salt_api_password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'salt_api_url': {'name': 'salt_api_url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'salt_api_username': {'name': 'salt_api_username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'devicehealth', 'can_run': True, 'error_string': '', 'module_options': {'enable_monitoring': {'name': 'enable_monitoring', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'monitor device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'mark_out_threshold': {'name': 'mark_out_threshold', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '2419200', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'automatically mark OSD if it may fail before this long', 'long_desc': '', 'tags': [], 'see_also': []}, 'pool_name': {'name': 'pool_name', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'device_health_metrics', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'name of pool in which to store device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'retention_period': {'name': 'retention_period', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '15552000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how long to retain device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'scrape_frequency': {'name': 'scrape_frequency', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '86400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to scrape device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'self_heal': {'name': 'self_heal', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'preemptively heal cluster around devices that may fail', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to wake up and check device health', 'long_desc': '', 'tags': [], 'see_also': []}, 'warn_threshold': {'name': 'warn_threshold', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '7257600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'raise health warning if OSD may fail before this long', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'diskprediction_cloud', 'can_run': True, 'error_string': '', 'module_options': {'diskprediction_cert_context': {'name': 'diskprediction_cert_context', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_default_authority': {'name': 'diskprediction_default_authority', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'localhost', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_password': {'name': 'diskprediction_password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_port': {'name': 'diskprediction_port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '31400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_retrieve_prediction_interval': {'name': 'diskprediction_retrieve_prediction_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '43200', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_server': {'name': 'diskprediction_server', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_ssl_target_name_override': {'name': 'diskprediction_ssl_target_name_override', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'localhost', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_upload_metrics_interval': {'name': 'diskprediction_upload_metrics_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_upload_smart_interval': {'name': 'diskprediction_upload_smart_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '43200', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_user': {'name': 'diskprediction_user', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'diskprediction_local', 'can_run': True, 'error_string': '', 'module_options': {'predict_interval': {'name': 'predict_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '86400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'influx', 'can_run': False, 'error_string': 'influxdb python module not found', 'module_options': {'batch_size': {'name': 'batch_size', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '5000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'database': {'name': 'database', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'ceph', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'hostname': {'name': 'hostname', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '30', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'password': {'name': 'password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'port': {'name': 'port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '8086', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssl': {'name': 'ssl', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'false', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'threads': {'name': 'threads', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '5', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'username': {'name': 'username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'verify_ssl': {'name': 'verify_ssl', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'true', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'insights', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'iostat', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'localpool', 'can_run': True, 'error_string': '', 'module_options': {'failure_domain': {'name': 'failure_domain', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'host', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'failure domain for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'min_size': {'name': 'min_size', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'default min_size for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'num_rep': {'name': 'num_rep', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '3', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'default replica count for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'pg_num': {'name': 'pg_num', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '128', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'default pg_num for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'prefix': {'name': 'prefix', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'name prefix for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'subtree': {'name': 'subtree', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'rack', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'CRUSH level for which to create a local pool', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'orchestrator_cli', 'can_run': True, 'error_string': '', 'module_options': {'orchestrator': {'name': 'orchestrator', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'pg_autoscaler', 'can_run': True, 'error_string': '', 'module_options': {'sleep_interval': {'name': 'sleep_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'progress', 'can_run': True, 'error_string': '', 'module_options': {'enabled': {'name': 'enabled', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'max_completed_events': {'name': 'max_completed_events', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '50', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'number of past completed events to remember', 'long_desc': '', 'tags': [], 'see_also': []}, 'persist_interval': {'name': 'persist_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '5', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to persist completed events', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'prometheus', 'can_run': True, 'error_string': '', 'module_options': {'rbd_stats_pools': {'name': 'rbd_stats_pools', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rbd_stats_pools_refresh_interval': {'name': 'rbd_stats_pools_refresh_interval', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '300', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'scrape_interval': {'name': 'scrape_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_addr': {'name': 'server_addr', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_port': {'name': 'server_port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'stale_cache_strategy': {'name': 'stale_cache_strategy', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'rbd_support', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'restful', 'can_run': True, 'error_string': '', 'module_options': {'key_file': {'name': 'key_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_addr': {'name': 'server_addr', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_port': {'name': 'server_port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'rook', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'selftest', 'can_run': True, 'error_string': '', 'module_options': {'roption1': {'name': 'roption1', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'roption2': {'name': 'roption2', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'xyz', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption1': {'name': 'rwoption1', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption2': {'name': 'rwoption2', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption3': {'name': 'rwoption3', 'type': 'float', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption4': {'name': 'rwoption4', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption5': {'name': 'rwoption5', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption6': {'name': 'rwoption6', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'testkey': {'name': 'testkey', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'testlkey': {'name': 'testlkey', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'testnewline': {'name': 'testnewline', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'ssh', 'can_run': True, 'error_string': '', 'module_options': {'inventory_cache_timeout_min': {'name': 'inventory_cache_timeout_min', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssh_config_file': {'name': 'ssh_config_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'status', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'telegraf', 'can_run': True, 'error_string': '', 'module_options': {'address': {'name': 'address', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'unixgram:///tmp/telegraf.sock', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'secs', 'level': 'advanced', 'flags': 0, 'default_value': '15', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'telemetry', 'can_run': True, 'error_string': '', 'module_options': {'channel_basic': {'name': 'channel_basic', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Share basic cluster information (size, version)', 'long_desc': '', 'tags': [], 'see_also': []}, 'channel_crash': {'name': 'channel_crash', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'channel_device': {'name': 'channel_device', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'channel_ident': {'name': 'channel_ident', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'contact': {'name': 'contact', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'description': {'name': 'description', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'device_url': {'name': 'device_url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'https://telemetry.ceph.com/device', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'enabled': {'name': 'enabled', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '24', 'min': '8', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'last_opt_revision': {'name': 'last_opt_revision', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '1', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'leaderboard': {'name': 'leaderboard', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'organization': {'name': 'organization', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'proxy': {'name': 'proxy', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'url': {'name': 'url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'https://telemetry.ceph.com/report', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'test_orchestrator', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'volumes', 'can_run': True, 'error_string': '', 'module_options': {'max_concurrent_clones': {'name': 'max_concurrent_clones', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '4', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Number of asynchronous cloner threads', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'zabbix', 'can_run': True, 'error_string': '', 'module_options': {'identifier': {'name': 'identifier', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'secs', 'level': 'advanced', 'flags': 0, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'zabbix_host': {'name': 'zabbix_host', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'zabbix_port': {'name': 'zabbix_port', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '10051', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'zabbix_sender': {'name': 'zabbix_sender', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '/usr/bin/zabbix_sender', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}]}], 'modules': ['iostat', 'restful'], 'available_modules': [{'name': 'alerts', 'can_run': True, 'error_string': '', 'module_options': {'interval': {'name': 'interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'How frequently to reexamine health status', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_destination': {'name': 'smtp_destination', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Email address to send alerts to', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_from_name': {'name': 'smtp_from_name', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'Ceph', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Email From: name', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_host': {'name': 'smtp_host', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'SMTP server', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_password': {'name': 'smtp_password', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Password to authenticate with', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_port': {'name': 'smtp_port', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '465', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'SMTP port', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_sender': {'name': 'smtp_sender', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'SMTP envelope sender', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_ssl': {'name': 'smtp_ssl', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Use SSL to connect to SMTP server', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_user': {'name': 'smtp_user', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'User to authenticate as', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'ansible', 'can_run': True, 'error_string': '', 'module_options': {'password': {'name': 'password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_url': {'name': 'server_url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'username': {'name': 'username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'verify_server': {'name': 'verify_server', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'balancer', 'can_run': True, 'error_string': '', 'module_options': {'active': {'name': 'active', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'automatically balance PGs across cluster', 'long_desc': '', 'tags': [], 'see_also': []}, 'begin_time': {'name': 'begin_time', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '0000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'beginning time of day to automatically balance', 'long_desc': 'This is a time of day in the format HHMM.', 'tags': [], 'see_also': []}, 'begin_weekday': {'name': 'begin_weekday', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '0', 'min': '0', 'max': '7', 'enum_allowed': [], 'desc': 'Restrict automatic balancing to this day of the week or later', 'long_desc': '0 or 7 = Sunday, 1 = Monday, etc.', 'tags': [], 'see_also': []}, 'crush_compat_max_iterations': {'name': 'crush_compat_max_iterations', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '25', 'min': '1', 'max': '250', 'enum_allowed': [], 'desc': 'maximum number of iterations to attempt optimization', 'long_desc': '', 'tags': [], 'see_also': []}, 'crush_compat_metrics': {'name': 'crush_compat_metrics', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'pgs,objects,bytes', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'metrics with which to calculate OSD utilization', 'long_desc': 'Value is a list of one or more of "pgs", "objects", or "bytes", and indicates which metrics to use to balance utilization.', 'tags': [], 'see_also': []}, 'crush_compat_step': {'name': 'crush_compat_step', 'type': 'float', 'level': 'advanced', 'flags': 1, 'default_value': '0.5', 'min': '0.001', 'max': '0.999', 'enum_allowed': [], 'desc': 'aggressiveness of optimization', 'long_desc': '.99 is very aggressive, .01 is less aggressive', 'tags': [], 'see_also': []}, 'end_time': {'name': 'end_time', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '2400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'ending time of day to automatically balance', 'long_desc': 'This is a time of day in the format HHMM.', 'tags': [], 'see_also': []}, 'end_weekday': {'name': 'end_weekday', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '7', 'min': '0', 'max': '7', 'enum_allowed': [], 'desc': 'Restrict automatic balancing to days of the week earlier than this', 'long_desc': '0 or 7 = Sunday, 1 = Monday, etc.', 'tags': [], 'see_also': []}, 'min_score': {'name': 'min_score', 'type': 'float', 'level': 'advanced', 'flags': 1, 'default_value': '0', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'minimum score, below which no optimization is attempted', 'long_desc': '', 'tags': [], 'see_also': []}, 'mode': {'name': 'mode', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'none', 'min': '', 'max': '', 'enum_allowed': ['crush-compat', 'none', 'upmap'], 'desc': 'Balancer mode', 'long_desc': '', 'tags': [], 'see_also': []}, 'pool_ids': {'name': 'pool_ids', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'pools which the automatic balancing will be limited to', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to wake up and attempt optimization', 'long_desc': '', 'tags': [], 'see_also': []}, 'upmap_max_deviation': {'name': 'upmap_max_deviation', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '5', 'min': '1', 'max': '', 'enum_allowed': [], 'desc': 'deviation below which no optimization is attempted', 'long_desc': 'If the number of PGs are within this count then no optimization is attempted', 'tags': [], 'see_also': []}, 'upmap_max_iterations': {'name': 'upmap_max_iterations', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '10', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'maximum upmap optimization iterations', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'crash', 'can_run': True, 'error_string': '', 'module_options': {'retain_interval': {'name': 'retain_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '31536000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how long to retain crashes before pruning them', 'long_desc': '', 'tags': [], 'see_also': []}, 'warn_recent_interval': {'name': 'warn_recent_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '1209600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'time interval in which to warn about recent crashes', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'dashboard', 'can_run': True, 'error_string': '', 'module_options': {'ALERTMANAGER_API_HOST': {'name': 'ALERTMANAGER_API_HOST', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'AUDIT_API_ENABLED': {'name': 'AUDIT_API_ENABLED', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'AUDIT_API_LOG_PAYLOAD': {'name': 'AUDIT_API_LOG_PAYLOAD', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ENABLE_BROWSABLE_API': {'name': 'ENABLE_BROWSABLE_API', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_cephfs': {'name': 'FEATURE_TOGGLE_cephfs', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_iscsi': {'name': 'FEATURE_TOGGLE_iscsi', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_mirroring': {'name': 'FEATURE_TOGGLE_mirroring', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_rbd': {'name': 'FEATURE_TOGGLE_rbd', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_rgw': {'name': 'FEATURE_TOGGLE_rgw', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE': {'name': 'GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_PASSWORD': {'name': 'GRAFANA_API_PASSWORD', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'admin', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_SSL_VERIFY': {'name': 'GRAFANA_API_SSL_VERIFY', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_URL': {'name': 'GRAFANA_API_URL', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_USERNAME': {'name': 'GRAFANA_API_USERNAME', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'admin', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_UPDATE_DASHBOARDS': {'name': 'GRAFANA_UPDATE_DASHBOARDS', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ISCSI_API_SSL_VERIFICATION': {'name': 'ISCSI_API_SSL_VERIFICATION', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'PROMETHEUS_API_HOST': {'name': 'PROMETHEUS_API_HOST', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'REST_REQUESTS_TIMEOUT': {'name': 'REST_REQUESTS_TIMEOUT', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '45', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_ACCESS_KEY': {'name': 'RGW_API_ACCESS_KEY', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_ADMIN_RESOURCE': {'name': 'RGW_API_ADMIN_RESOURCE', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'admin', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_HOST': {'name': 'RGW_API_HOST', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_PORT': {'name': 'RGW_API_PORT', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '80', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_SCHEME': {'name': 'RGW_API_SCHEME', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'http', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_SECRET_KEY': {'name': 'RGW_API_SECRET_KEY', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_SSL_VERIFY': {'name': 'RGW_API_SSL_VERIFY', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_USER_ID': {'name': 'RGW_API_USER_ID', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'crt_file': {'name': 'crt_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'debug': {'name': 'debug', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Enable/disable debug options', 'long_desc': '', 'tags': [], 'see_also': []}, 'jwt_token_ttl': {'name': 'jwt_token_ttl', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '28800', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'key_file': {'name': 'key_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'password': {'name': 'password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_addr': {'name': 'server_addr', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '::', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_port': {'name': 'server_port', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '8080', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssl': {'name': 'ssl', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssl_server_port': {'name': 'ssl_server_port', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '8443', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'standby_behaviour': {'name': 'standby_behaviour', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'redirect', 'min': '', 'max': '', 'enum_allowed': ['error', 'redirect'], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'standby_error_status_code': {'name': 'standby_error_status_code', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '500', 'min': '400', 'max': '599', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'url_prefix': {'name': 'url_prefix', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'username': {'name': 'username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'deepsea', 'can_run': True, 'error_string': '', 'module_options': {'salt_api_eauth': {'name': 'salt_api_eauth', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'sharedsecret', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'salt_api_password': {'name': 'salt_api_password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'salt_api_url': {'name': 'salt_api_url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'salt_api_username': {'name': 'salt_api_username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'devicehealth', 'can_run': True, 'error_string': '', 'module_options': {'enable_monitoring': {'name': 'enable_monitoring', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'monitor device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'mark_out_threshold': {'name': 'mark_out_threshold', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '2419200', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'automatically mark OSD if it may fail before this long', 'long_desc': '', 'tags': [], 'see_also': []}, 'pool_name': {'name': 'pool_name', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'device_health_metrics', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'name of pool in which to store device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'retention_period': {'name': 'retention_period', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '15552000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how long to retain device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'scrape_frequency': {'name': 'scrape_frequency', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '86400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to scrape device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'self_heal': {'name': 'self_heal', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'preemptively heal cluster around devices that may fail', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to wake up and check device health', 'long_desc': '', 'tags': [], 'see_also': []}, 'warn_threshold': {'name': 'warn_threshold', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '7257600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'raise health warning if OSD may fail before this long', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'diskprediction_cloud', 'can_run': True, 'error_string': '', 'module_options': {'diskprediction_cert_context': {'name': 'diskprediction_cert_context', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_default_authority': {'name': 'diskprediction_default_authority', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'localhost', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_password': {'name': 'diskprediction_password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_port': {'name': 'diskprediction_port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '31400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_retrieve_prediction_interval': {'name': 'diskprediction_retrieve_prediction_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '43200', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_server': {'name': 'diskprediction_server', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_ssl_target_name_override': {'name': 'diskprediction_ssl_target_name_override', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'localhost', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_upload_metrics_interval': {'name': 'diskprediction_upload_metrics_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_upload_smart_interval': {'name': 'diskprediction_upload_smart_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '43200', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_user': {'name': 'diskprediction_user', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'diskprediction_local', 'can_run': True, 'error_string': '', 'module_options': {'predict_interval': {'name': 'predict_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '86400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'influx', 'can_run': False, 'error_string': 'influxdb python module not found', 'module_options': {'batch_size': {'name': 'batch_size', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '5000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'database': {'name': 'database', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'ceph', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'hostname': {'name': 'hostname', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '30', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'password': {'name': 'password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'port': {'name': 'port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '8086', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssl': {'name': 'ssl', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'false', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'threads': {'name': 'threads', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '5', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'username': {'name': 'username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'verify_ssl': {'name': 'verify_ssl', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'true', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'insights', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'iostat', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'localpool', 'can_run': True, 'error_string': '', 'module_options': {'failure_domain': {'name': 'failure_domain', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'host', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'failure domain for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'min_size': {'name': 'min_size', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'default min_size for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'num_rep': {'name': 'num_rep', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '3', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'default replica count for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'pg_num': {'name': 'pg_num', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '128', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'default pg_num for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'prefix': {'name': 'prefix', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'name prefix for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'subtree': {'name': 'subtree', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'rack', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'CRUSH level for which to create a local pool', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'orchestrator_cli', 'can_run': True, 'error_string': '', 'module_options': {'orchestrator': {'name': 'orchestrator', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'pg_autoscaler', 'can_run': True, 'error_string': '', 'module_options': {'sleep_interval': {'name': 'sleep_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'progress', 'can_run': True, 'error_string': '', 'module_options': {'enabled': {'name': 'enabled', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'max_completed_events': {'name': 'max_completed_events', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '50', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'number of past completed events to remember', 'long_desc': '', 'tags': [], 'see_also': []}, 'persist_interval': {'name': 'persist_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '5', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to persist completed events', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'prometheus', 'can_run': True, 'error_string': '', 'module_options': {'rbd_stats_pools': {'name': 'rbd_stats_pools', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rbd_stats_pools_refresh_interval': {'name': 'rbd_stats_pools_refresh_interval', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '300', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'scrape_interval': {'name': 'scrape_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_addr': {'name': 'server_addr', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_port': {'name': 'server_port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'stale_cache_strategy': {'name': 'stale_cache_strategy', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'rbd_support', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'restful', 'can_run': True, 'error_string': '', 'module_options': {'key_file': {'name': 'key_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_addr': {'name': 'server_addr', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_port': {'name': 'server_port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'rook', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'selftest', 'can_run': True, 'error_string': '', 'module_options': {'roption1': {'name': 'roption1', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'roption2': {'name': 'roption2', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'xyz', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption1': {'name': 'rwoption1', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption2': {'name': 'rwoption2', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption3': {'name': 'rwoption3', 'type': 'float', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption4': {'name': 'rwoption4', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption5': {'name': 'rwoption5', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption6': {'name': 'rwoption6', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'testkey': {'name': 'testkey', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'testlkey': {'name': 'testlkey', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'testnewline': {'name': 'testnewline', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'ssh', 'can_run': True, 'error_string': '', 'module_options': {'inventory_cache_timeout_min': {'name': 'inventory_cache_timeout_min', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssh_config_file': {'name': 'ssh_config_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'status', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'telegraf', 'can_run': True, 'error_string': '', 'module_options': {'address': {'name': 'address', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'unixgram:///tmp/telegraf.sock', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'secs', 'level': 'advanced', 'flags': 0, 'default_value': '15', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'telemetry', 'can_run': True, 'error_string': '', 'module_options': {'channel_basic': {'name': 'channel_basic', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Share basic cluster information (size, version)', 'long_desc': '', 'tags': [], 'see_also': []}, 'channel_crash': {'name': 'channel_crash', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'channel_device': {'name': 'channel_device', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'channel_ident': {'name': 'channel_ident', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'contact': {'name': 'contact', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'description': {'name': 'description', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'device_url': {'name': 'device_url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'https://telemetry.ceph.com/device', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'enabled': {'name': 'enabled', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '24', 'min': '8', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'last_opt_revision': {'name': 'last_opt_revision', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '1', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'leaderboard': {'name': 'leaderboard', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'organization': {'name': 'organization', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'proxy': {'name': 'proxy', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'url': {'name': 'url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'https://telemetry.ceph.com/report', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'test_orchestrator', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'volumes', 'can_run': True, 'error_string': '', 'module_options': {'max_concurrent_clones': {'name': 'max_concurrent_clones', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '4', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Number of asynchronous cloner threads', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'zabbix', 'can_run': True, 'error_string': '', 'module_options': {'identifier': {'name': 'identifier', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'secs', 'level': 'advanced', 'flags': 0, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'zabbix_host': {'name': 'zabbix_host', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'zabbix_port': {'name': 'zabbix_port', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '10051', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'zabbix_sender': {'name': 'zabbix_sender', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '/usr/bin/zabbix_sender', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}], 'services': {}, 'always_on_modules': {'nautilus': ['balancer', 'crash', 'devicehealth', 'orchestrator_cli', 'progress', 'rbd_support', 'status', 'volumes']}}, 'servicemap': {'epoch': 2, 'modified': '2021-02-10 11:13:28.137169', 'services': {'mds': {'daemons': {'summary': '', 'a': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': ':/0', 'metadata': {}, 'task_status': {'scrub status': 'idle'}}}}}}, 'progress_events': {}} 2021-02-10T11:13:33.743 INFO:tasks.ceph.ceph_manager.ceph:clean! 2021-02-10T11:13:33.743 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2021-02-10T11:13:33.744 DEBUG:teuthology.orchestra.run.gibba001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph health 2021-02-10T11:13:34.162 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.139 7fc6f47ed700 1 Processor -- start 2021-02-10T11:13:34.162 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.139 7fc6f47ed700 1 -- start start 2021-02-10T11:13:34.162 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.140 7fc6f47ed700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6ec1b0870 0x7fc6ec0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:34.163 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.140 7fc6f47ed700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6ec1b0c70 con 0x7fc6ec1b0470 2021-02-10T11:13:34.163 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.140 7fc6f47ed700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6ec0f0b50 con 0x7fc6ec0efe00 2021-02-10T11:13:34.163 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.140 7fc6f47ed700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec0e5ea0 con 0x7fc6ec1b0870 2021-02-10T11:13:34.163 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.140 7fc6f2609700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc6ec1b0470 0x7fc6ec0ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44876/0 (socket says 172.21.2.101:44876) 2021-02-10T11:13:34.163 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.140 7fc6f2609700 1 -- 172.21.2.101:0/1206541346 learned_addr learned my addr 172.21.2.101:0/1206541346 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:34.163 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.141 7fc6f1607700 1 -- 172.21.2.101:0/1206541346 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1685577804 0 0) 0x7fc6e0001180 con 0x7fc6ec1b0470 2021-02-10T11:13:34.163 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.141 7fc6f1e08700 1 --2- 172.21.2.101:0/1206541346 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6ec1b0870 0x7fc6ec0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:34.163 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.141 7fc6f1607700 1 -- 172.21.2.101:0/1206541346 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6d8002fe0 con 0x7fc6ec1b0470 2021-02-10T11:13:34.163 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.141 7fc6f1607700 1 -- 172.21.2.101:0/1206541346 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1371429036 0 0) 0x7fc6e00010e0 con 0x7fc6ec1b0470 2021-02-10T11:13:34.163 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.141 7fc6f1607700 1 -- 172.21.2.101:0/1206541346 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6ec1b0870 msgr2=0x7fc6ec0f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:34.163 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.141 7fc6f1607700 1 --2- 172.21.2.101:0/1206541346 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6ec1b0870 0x7fc6ec0f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:34.163 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.141 7fc6f1607700 1 -- 172.21.2.101:0/1206541346 >> v1:172.21.2.108:6789/0 conn(0x7fc6ec0efe00 legacy=0x7fc6ec0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:34.163 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.141 7fc6f1607700 1 -- 172.21.2.101:0/1206541346 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ec0e52e0 con 0x7fc6ec1b0470 2021-02-10T11:13:34.164 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.141 7fc6f1607700 1 -- 172.21.2.101:0/1206541346 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fc6e0001600 con 0x7fc6ec1b0470 2021-02-10T11:13:34.164 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.141 7fc6f1607700 1 -- 172.21.2.101:0/1206541346 >> v1:172.21.2.101:6789/0 conn(0x7fc6ec1b0470 legacy=0x7fc6ec0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:34.164 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.141 7fc6f1607700 1 --2- 172.21.2.101:0/1206541346 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6d8003a60 0x7fc6d8005ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:34.164 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.141 7fc6f1607700 1 --2- 172.21.2.101:0/1206541346 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6d8006770 0x7fc6d8008c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:34.164 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.141 7fc6f1607700 1 --2- 172.21.2.101:0/1206541346 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6d8009500 0x7fc6d800ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:34.164 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.141 7fc6f1607700 1 -- 172.21.2.101:0/1206541346 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d800c2d0 con 0x7fc6d8003a60 2021-02-10T11:13:34.164 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.141 7fc6f1607700 1 -- 172.21.2.101:0/1206541346 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d800c4f0 con 0x7fc6d8009500 2021-02-10T11:13:34.164 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.141 7fc6f1607700 1 -- 172.21.2.101:0/1206541346 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d800c710 con 0x7fc6d8006770 2021-02-10T11:13:34.164 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.141 7fc6f2609700 1 --2- 172.21.2.101:0/1206541346 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6d8003a60 0x7fc6d8005ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:34.164 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.141 7fc6f1e08700 1 --2- 172.21.2.101:0/1206541346 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6d8006770 0x7fc6d8008c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:34.164 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.141 7fc6f2609700 1 -- 172.21.2.101:0/1206541346 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6d8006770 msgr2=0x7fc6d8008c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:34.164 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.141 7fc6f2e0a700 1 --2- 172.21.2.101:0/1206541346 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6d8009500 0x7fc6d800ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:34.164 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.142 7fc6f2609700 1 --2- 172.21.2.101:0/1206541346 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6d8006770 0x7fc6d8008c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:34.165 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.142 7fc6f2609700 1 -- 172.21.2.101:0/1206541346 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6d8009500 msgr2=0x7fc6d800ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:34.165 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.142 7fc6f2609700 1 --2- 172.21.2.101:0/1206541346 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6d8009500 0x7fc6d800ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:34.165 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.142 7fc6f2609700 1 -- 172.21.2.101:0/1206541346 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d8050980 con 0x7fc6d8003a60 2021-02-10T11:13:34.165 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.142 7fc6f1e08700 1 --2- 172.21.2.101:0/1206541346 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6d8006770 0x7fc6d8008c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:13:34.165 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.142 7fc6f2e0a700 1 --2- 172.21.2.101:0/1206541346 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6d8009500 0x7fc6d800ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:13:34.165 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.142 7fc6f2609700 1 --2- 172.21.2.101:0/1206541346 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6d8003a60 0x7fc6d8005ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fc6e0001770 tx=0x7fc6e0007cf0).ready entity=mon.0 client_cookie=4668affb6c51e005 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:34.165 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.142 7fc6f1607700 1 -- 172.21.2.101:0/1206541346 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc6e00010e0 con 0x7fc6d8003a60 2021-02-10T11:13:34.165 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.142 7fc6f1607700 1 -- 172.21.2.101:0/1206541346 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e000a520 con 0x7fc6d8003a60 2021-02-10T11:13:34.165 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.142 7fc6f1607700 1 -- 172.21.2.101:0/1206541346 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc6e000a970 con 0x7fc6d8003a60 2021-02-10T11:13:34.165 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.142 7fc6f47ed700 1 -- 172.21.2.101:0/1206541346 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6d8003a60 msgr2=0x7fc6d8005ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:34.165 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.142 7fc6f47ed700 1 --2- 172.21.2.101:0/1206541346 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6d8003a60 0x7fc6d8005ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fc6e0001770 tx=0x7fc6e0007cf0).stop 2021-02-10T11:13:34.166 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.142 7fc6f2e0a700 1 -- 172.21.2.101:0/1206541346 reap_dead start 2021-02-10T11:13:34.166 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.142 7fc6f47ed700 1 -- 172.21.2.101:0/1206541346 shutdown_connections 2021-02-10T11:13:34.166 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.143 7fc6f47ed700 1 -- 172.21.2.101:0/1206541346 >> 172.21.2.101:0/1206541346 conn(0x7fc6ec0ce3e0 msgr2=0x7fc6ec0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:34.166 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.143 7fc6f47ed700 1 -- 172.21.2.101:0/1206541346 shutdown_connections 2021-02-10T11:13:34.166 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.143 7fc6f47ed700 1 -- 172.21.2.101:0/1206541346 wait complete. 2021-02-10T11:13:34.166 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.157 7fc6f47ed700 1 Processor -- start 2021-02-10T11:13:34.166 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.157 7fc6f47ed700 1 -- start start 2021-02-10T11:13:34.166 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.158 7fc6f47ed700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6ec09d5f0 0x7fc6ec0b65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:34.166 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.158 7fc6f47ed700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6ec0b6bf0 0x7fc6ec0905c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:34.166 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.158 7fc6f47ed700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6ec090be0 0x7fc6ec090fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:34.166 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.158 7fc6f47ed700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec091600 con 0x7fc6ec09d5f0 2021-02-10T11:13:34.166 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.158 7fc6f47ed700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec0b6ff0 con 0x7fc6ec0b6bf0 2021-02-10T11:13:34.166 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.158 7fc6f47ed700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec09d9f0 con 0x7fc6ec090be0 2021-02-10T11:13:34.167 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.158 7fc6f2609700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6ec09d5f0 0x7fc6ec0b65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:34.167 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.158 7fc6f2609700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6ec09d5f0 0x7fc6ec0b65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44394/0 (socket says 172.21.2.101:44394) 2021-02-10T11:13:34.167 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.158 7fc6f2609700 1 -- 172.21.2.101:0/3619607675 learned_addr learned my addr 172.21.2.101:0/3619607675 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:34.167 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.158 7fc6f2e0a700 1 --2- 172.21.2.101:0/3619607675 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6ec090be0 0x7fc6ec090fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:34.167 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.158 7fc6f1e08700 1 --2- 172.21.2.101:0/3619607675 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6ec0b6bf0 0x7fc6ec0905c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:34.168 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.158 7fc6f2609700 1 -- 172.21.2.101:0/3619607675 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6ec090be0 msgr2=0x7fc6ec090fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:34.168 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.158 7fc6f2609700 1 --2- 172.21.2.101:0/3619607675 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6ec090be0 0x7fc6ec090fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:34.168 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.158 7fc6f2609700 1 -- 172.21.2.101:0/3619607675 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6ec0b6bf0 msgr2=0x7fc6ec0905c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:34.168 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.158 7fc6f2609700 1 --2- 172.21.2.101:0/3619607675 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6ec0b6bf0 0x7fc6ec0905c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:34.169 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.159 7fc6f2609700 1 -- 172.21.2.101:0/3619607675 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ec08f110 con 0x7fc6ec09d5f0 2021-02-10T11:13:34.169 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.159 7fc6f2609700 1 --2- 172.21.2.101:0/3619607675 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6ec09d5f0 0x7fc6ec0b65d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fc6e000b540 tx=0x7fc6e000b8e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:34.169 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.159 7fc6deffd700 1 -- 172.21.2.101:0/3619607675 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc6e000bcd0 con 0x7fc6ec09d5f0 2021-02-10T11:13:34.169 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.159 7fc6deffd700 1 -- 172.21.2.101:0/3619607675 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e00043a0 con 0x7fc6ec09d5f0 2021-02-10T11:13:34.169 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.159 7fc6deffd700 1 -- 172.21.2.101:0/3619607675 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc6e00048c0 con 0x7fc6ec09d5f0 2021-02-10T11:13:34.169 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.159 7fc6f47ed700 1 -- 172.21.2.101:0/3619607675 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ec08f110 con 0x7fc6ec09d5f0 2021-02-10T11:13:34.169 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.159 7fc6f47ed700 1 -- 172.21.2.101:0/3619607675 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ec08f110 con 0x7fc6ec09d5f0 2021-02-10T11:13:34.169 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.159 7fc6deffd700 1 -- 172.21.2.101:0/3619607675 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc6e0009000 con 0x7fc6ec09d5f0 2021-02-10T11:13:34.169 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.160 7fc6deffd700 1 --2- 172.21.2.101:0/3619607675 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6d4018730 0x7fc6d401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:34.169 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.160 7fc6deffd700 1 -- 172.21.2.101:0/3619607675 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc6e0032600 con 0x7fc6ec09d5f0 2021-02-10T11:13:34.169 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.160 7fc6f1e08700 1 --2- 172.21.2.101:0/3619607675 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6d4018730 0x7fc6d401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:34.169 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.160 7fc6f1e08700 1 --2- 172.21.2.101:0/3619607675 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6d4018730 0x7fc6d401abc0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:34.194 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.190 7fc6f47ed700 1 -- 172.21.2.101:0/3619607675 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6ec105a50 con 0x7fc6ec09d5f0 2021-02-10T11:13:34.196 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.192 7fc6deffd700 1 -- 172.21.2.101:0/3619607675 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc6e0000c90 con 0x7fc6ec09d5f0 2021-02-10T11:13:34.331 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.327 7fc6f47ed700 1 -- 172.21.2.101:0/3619607675 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "health"} v 0) v1 -- 0x7fc6ec0f0b50 con 0x7fc6ec09d5f0 2021-02-10T11:13:34.332 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.328 7fc6deffd700 1 -- 172.21.2.101:0/3619607675 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "health"}]=0 v0) v1 ==== 54+0+10 (secure 0 0 0) 0x7fc6e00043a0 con 0x7fc6ec09d5f0 2021-02-10T11:13:34.332 INFO:teuthology.misc.health.gibba001.stdout:HEALTH_OK 2021-02-10T11:13:34.334 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.330 7fc6f47ed700 1 -- 172.21.2.101:0/3619607675 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6d4018730 msgr2=0x7fc6d401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:34.334 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.330 7fc6f47ed700 1 --2- 172.21.2.101:0/3619607675 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6d4018730 0x7fc6d401abc0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:34.334 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.330 7fc6f47ed700 1 -- 172.21.2.101:0/3619607675 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6ec09d5f0 msgr2=0x7fc6ec0b65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:34.334 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.330 7fc6f47ed700 1 --2- 172.21.2.101:0/3619607675 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6ec09d5f0 0x7fc6ec0b65d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fc6e000b540 tx=0x7fc6e000b8e0).stop 2021-02-10T11:13:34.334 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.330 7fc6f47ed700 1 -- 172.21.2.101:0/3619607675 shutdown_connections 2021-02-10T11:13:34.335 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.330 7fc6f47ed700 1 --2- 172.21.2.101:0/3619607675 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6d4018730 0x7fc6d401abc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:34.335 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.330 7fc6f47ed700 1 --2- 172.21.2.101:0/3619607675 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6ec090be0 0x7fc6ec090fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:34.335 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.330 7fc6f47ed700 1 --2- 172.21.2.101:0/3619607675 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6ec0b6bf0 0x7fc6ec0905c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:34.335 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.330 7fc6f47ed700 1 --2- 172.21.2.101:0/3619607675 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6ec09d5f0 0x7fc6ec0b65d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:34.335 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.330 7fc6f47ed700 1 -- 172.21.2.101:0/3619607675 >> 172.21.2.101:0/3619607675 conn(0x7fc6ec0b9e10 msgr2=0x7fc6ec0c3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:34.335 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.330 7fc6f47ed700 1 -- 172.21.2.101:0/3619607675 shutdown_connections 2021-02-10T11:13:34.335 INFO:teuthology.misc.health.gibba001.stderr:2021-02-10 11:13:34.330 7fc6f47ed700 1 -- 172.21.2.101:0/3619607675 wait complete. 2021-02-10T11:13:34.341 DEBUG:teuthology.misc:Ceph health: HEALTH_OK 2021-02-10T11:13:34.342 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:13:34.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.747 7fec2309e700 1 Processor -- start 2021-02-10T11:13:34.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.747 7fec2309e700 1 -- start start 2021-02-10T11:13:34.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.748 7fec2309e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fec1c0e5070 0x7fec1c1b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:34.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.748 7fec2309e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fec1c1b3e40 0x7fec1c07d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:34.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.748 7fec2309e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fec1c0e5470 0x7fec1c07dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:34.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.748 7fec2309e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec1c07ed80 con 0x7fec1c1b3e40 2021-02-10T11:13:34.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.748 7fec2309e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec1c07e2d0 con 0x7fec1c0e5070 2021-02-10T11:13:34.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.748 7fec2309e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c07e4f0 con 0x7fec1c0e5470 2021-02-10T11:13:34.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.748 7fec1bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fec1c1b3e40 0x7fec1c07d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:34.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.748 7fec1bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fec1c1b3e40 0x7fec1c07d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44402/0 (socket says 172.21.2.101:44402) 2021-02-10T11:13:34.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.748 7fec1bfff700 1 -- 172.21.2.101:0/3972567859 learned_addr learned my addr 172.21.2.101:0/3972567859 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:34.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.748 7fec1bfff700 1 -- 172.21.2.101:0/3972567859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fec1c0e5470 msgr2=0x7fec1c07dcb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:13:34.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.748 7fec1bfff700 1 --2- 172.21.2.101:0/3972567859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fec1c0e5470 0x7fec1c07dcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:34.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.748 7fec1bfff700 1 -- 172.21.2.101:0/3972567859 >> v2:172.21.2.108:3300/0 conn(0x7fec1c0e5070 msgr2=0x7fec1c1b3820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:13:34.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.748 7fec1bfff700 1 --2- 172.21.2.101:0/3972567859 >> v2:172.21.2.108:3300/0 conn(0x7fec1c0e5070 0x7fec1c1b3820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:34.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.748 7fec1bfff700 1 -- 172.21.2.101:0/3972567859 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1c07e710 con 0x7fec1c1b3e40 2021-02-10T11:13:34.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.748 7fec1bfff700 1 --2- 172.21.2.101:0/3972567859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fec1c1b3e40 0x7fec1c07d690 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fec0c006700 tx=0x7fec0c009b80).ready entity=mon.0 client_cookie=8d58c97e4ac85699 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:34.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.748 7fec1b7fe700 1 -- 172.21.2.101:0/3972567859 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fec0c000c90 con 0x7fec1c1b3e40 2021-02-10T11:13:34.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.748 7fec1b7fe700 1 -- 172.21.2.101:0/3972567859 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0c00c0c0 con 0x7fec1c1b3e40 2021-02-10T11:13:34.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.748 7fec1b7fe700 1 -- 172.21.2.101:0/3972567859 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fec0c00c5e0 con 0x7fec1c1b3e40 2021-02-10T11:13:34.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.748 7fec2309e700 1 -- 172.21.2.101:0/3972567859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fec1c1b3e40 msgr2=0x7fec1c07d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:34.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.748 7fec2309e700 1 --2- 172.21.2.101:0/3972567859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fec1c1b3e40 0x7fec1c07d690 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fec0c006700 tx=0x7fec0c009b80).stop 2021-02-10T11:13:34.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.749 7fec2309e700 1 -- 172.21.2.101:0/3972567859 shutdown_connections 2021-02-10T11:13:34.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.749 7fec2309e700 1 --2- 172.21.2.101:0/3972567859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fec1c0e5470 0x7fec1c07dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:34.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.749 7fec2309e700 1 --2- 172.21.2.101:0/3972567859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fec1c1b3e40 0x7fec1c07d690 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:34.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.749 7fec2309e700 1 --2- 172.21.2.101:0/3972567859 >> v2:172.21.2.108:3300/0 conn(0x7fec1c0e5070 0x7fec1c1b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:34.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.749 7fec2309e700 1 -- 172.21.2.101:0/3972567859 >> 172.21.2.101:0/3972567859 conn(0x7fec1c0ce690 msgr2=0x7fec1c0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:34.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.749 7fec2309e700 1 -- 172.21.2.101:0/3972567859 shutdown_connections 2021-02-10T11:13:34.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.749 7fec2309e700 1 -- 172.21.2.101:0/3972567859 wait complete. 2021-02-10T11:13:34.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.763 7fec2309e700 1 Processor -- start 2021-02-10T11:13:34.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.763 7fec2309e700 1 -- start start 2021-02-10T11:13:34.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.764 7fec2309e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fec1c192960 0x7fec1c192d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:34.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.764 7fec2309e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fec1c0ad110 0x7fec1c0ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:34.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.764 7fec2309e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fec1c0adb30 0x7fec1c08e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:34.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.764 7fec2309e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c1a03e0 con 0x7fec1c192960 2021-02-10T11:13:34.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.764 7fec2309e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c193380 con 0x7fec1c0ad110 2021-02-10T11:13:34.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.764 7fec2309e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c08ea80 con 0x7fec1c0adb30 2021-02-10T11:13:34.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.764 7fec20eba700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fec1c192960 0x7fec1c192d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:34.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.764 7fec20eba700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fec1c192960 0x7fec1c192d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44408/0 (socket says 172.21.2.101:44408) 2021-02-10T11:13:34.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.764 7fec20eba700 1 -- 172.21.2.101:0/3666288240 learned_addr learned my addr 172.21.2.101:0/3666288240 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:34.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.764 7fec20eba700 1 -- 172.21.2.101:0/3666288240 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fec1c0adb30 msgr2=0x7fec1c08e460 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:34.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.764 7fec20eba700 1 --2- 172.21.2.101:0/3666288240 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fec1c0adb30 0x7fec1c08e460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:34.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.764 7fec20eba700 1 -- 172.21.2.101:0/3666288240 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fec1c0ad110 msgr2=0x7fec1c0ad510 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:34.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.764 7fec20eba700 1 --2- 172.21.2.101:0/3666288240 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fec1c0ad110 0x7fec1c0ad510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:34.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.764 7fec20eba700 1 -- 172.21.2.101:0/3666288240 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1c08eca0 con 0x7fec1c192960 2021-02-10T11:13:34.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.765 7fec20eba700 1 --2- 172.21.2.101:0/3666288240 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fec1c192960 0x7fec1c192d60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fec0400b5c0 tx=0x7fec0400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:34.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.765 7fec197fa700 1 -- 172.21.2.101:0/3666288240 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fec0400bd50 con 0x7fec1c192960 2021-02-10T11:13:34.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.765 7fec197fa700 1 -- 172.21.2.101:0/3666288240 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec04006030 con 0x7fec1c192960 2021-02-10T11:13:34.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.765 7fec197fa700 1 -- 172.21.2.101:0/3666288240 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fec040054d0 con 0x7fec1c192960 2021-02-10T11:13:34.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.765 7fec2309e700 1 -- 172.21.2.101:0/3666288240 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1c08eca0 con 0x7fec1c192960 2021-02-10T11:13:34.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.765 7fec2309e700 1 -- 172.21.2.101:0/3666288240 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1c08f230 con 0x7fec1c192960 2021-02-10T11:13:34.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.765 7fec197fa700 1 -- 172.21.2.101:0/3666288240 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fec04006210 con 0x7fec1c192960 2021-02-10T11:13:34.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.765 7fec197fa700 1 --2- 172.21.2.101:0/3666288240 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fec000186c0 0x7fec0001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:34.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.766 7fec197fa700 1 -- 172.21.2.101:0/3666288240 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fec04032800 con 0x7fec1c192960 2021-02-10T11:13:34.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.766 7fec1bfff700 1 --2- 172.21.2.101:0/3666288240 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fec000186c0 0x7fec0001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:34.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.766 7fec1bfff700 1 --2- 172.21.2.101:0/3666288240 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fec000186c0 0x7fec0001abc0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:34.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.797 7fec2309e700 1 -- 172.21.2.101:0/3666288240 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec1c107120 con 0x7fec1c192960 2021-02-10T11:13:34.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.800 7fec197fa700 1 -- 172.21.2.101:0/3666288240 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fec04000940 con 0x7fec1c192960 2021-02-10T11:13:34.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.923 7fec2309e700 1 -- 172.21.2.101:0/3666288240 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec1c07ed80 con 0x7fec1c192960 2021-02-10T11:13:34.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.924 7fec197fa700 1 -- 172.21.2.101:0/3666288240 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+13993 (secure 0 0 0) 0x7fec04006030 con 0x7fec1c192960 2021-02-10T11:13:34.928 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:13:34.929 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":20,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:13:15.149234","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:13:11.922272","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2021-02-10 11:13:13.141733","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:13:34.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.925 7fec2309e700 1 -- 172.21.2.101:0/3666288240 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fec000186c0 msgr2=0x7fec0001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:34.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.925 7fec2309e700 1 --2- 172.21.2.101:0/3666288240 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fec000186c0 0x7fec0001abc0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:34.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.925 7fec2309e700 1 -- 172.21.2.101:0/3666288240 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fec1c192960 msgr2=0x7fec1c192d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:34.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.925 7fec2309e700 1 --2- 172.21.2.101:0/3666288240 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fec1c192960 0x7fec1c192d60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fec0400b5c0 tx=0x7fec0400b960).stop 2021-02-10T11:13:34.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.926 7fec2309e700 1 -- 172.21.2.101:0/3666288240 shutdown_connections 2021-02-10T11:13:34.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.926 7fec2309e700 1 --2- 172.21.2.101:0/3666288240 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fec000186c0 0x7fec0001abc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:34.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.926 7fec2309e700 1 --2- 172.21.2.101:0/3666288240 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fec1c0adb30 0x7fec1c08e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:34.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.926 7fec2309e700 1 --2- 172.21.2.101:0/3666288240 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fec1c0ad110 0x7fec1c0ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:34.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.926 7fec2309e700 1 --2- 172.21.2.101:0/3666288240 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fec1c192960 0x7fec1c192d60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:34.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.926 7fec2309e700 1 -- 172.21.2.101:0/3666288240 >> 172.21.2.101:0/3666288240 conn(0x7fec1c0891f0 msgr2=0x7fec1c1a0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:34.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.926 7fec2309e700 1 -- 172.21.2.101:0/3666288240 shutdown_connections 2021-02-10T11:13:34.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:34.926 7fec2309e700 1 -- 172.21.2.101:0/3666288240 wait complete. 2021-02-10T11:13:34.938 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:13:11.922272', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:13:13.141733', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:13:34.939 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:13:35.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.347 7f220a627700 1 Processor -- start 2021-02-10T11:13:35.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.347 7f220a627700 1 -- start start 2021-02-10T11:13:35.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.347 7f220a627700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f22041b0470 0x7f22041b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:35.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.347 7f220a627700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22040f20e0 0x7f22040f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:35.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.347 7f220a627700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22040f0b50 con 0x7f22040ef610 2021-02-10T11:13:35.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.347 7f220a627700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22040f0820 con 0x7f22041b0470 2021-02-10T11:13:35.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.347 7f220a627700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22040f24e0 con 0x7f22040f20e0 2021-02-10T11:13:35.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.348 7f22037fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f22040ef610 0x7f22040efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44912/0 (socket says 172.21.2.101:44912) 2021-02-10T11:13:35.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.348 7f22037fe700 1 -- 172.21.2.101:0/872473744 learned_addr learned my addr 172.21.2.101:0/872473744 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:35.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.348 7f2202ffd700 1 -- 172.21.2.101:0/872473744 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2291167580 0 0) 0x7f21f80009b0 con 0x7f22040ef610 2021-02-10T11:13:35.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.348 7f2202ffd700 1 -- 172.21.2.101:0/872473744 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21ec002fe0 con 0x7f22040ef610 2021-02-10T11:13:35.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.348 7f2202ffd700 1 -- 172.21.2.101:0/872473744 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 601163848 0 0) 0x7f21f8001010 con 0x7f22040ef610 2021-02-10T11:13:35.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.348 7f2202ffd700 1 -- 172.21.2.101:0/872473744 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22040f20e0 msgr2=0x7f22040f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:35.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.348 7f2208c44700 1 --2- 172.21.2.101:0/872473744 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22040f20e0 0x7f22040f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:35.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.348 7f2203fff700 1 --2- 172.21.2.101:0/872473744 >> v2:172.21.2.108:3300/0 conn(0x7f22041b0470 0x7f22041b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:35.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.348 7f2202ffd700 1 --2- 172.21.2.101:0/872473744 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22040f20e0 0x7f22040f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:35.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.348 7f2202ffd700 1 -- 172.21.2.101:0/872473744 >> v2:172.21.2.108:3300/0 conn(0x7f22041b0470 msgr2=0x7f22041b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:35.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.348 7f2202ffd700 1 --2- 172.21.2.101:0/872473744 >> v2:172.21.2.108:3300/0 conn(0x7f22041b0470 0x7f22041b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:35.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.348 7f2202ffd700 1 -- 172.21.2.101:0/872473744 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22040e52e0 con 0x7f22040ef610 2021-02-10T11:13:35.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.348 7f2202ffd700 1 -- 172.21.2.101:0/872473744 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f21f8001590 con 0x7f22040ef610 2021-02-10T11:13:35.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.348 7f2202ffd700 1 -- 172.21.2.101:0/872473744 >> v1:172.21.2.101:6789/0 conn(0x7f22040ef610 legacy=0x7f22040efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:35.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.348 7f2202ffd700 1 --2- 172.21.2.101:0/872473744 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21ec003b90 0x7f21ec0060b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:35.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.348 7f2202ffd700 1 --2- 172.21.2.101:0/872473744 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21ec0068d0 0x7f21ec008df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:35.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.348 7f2202ffd700 1 --2- 172.21.2.101:0/872473744 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21ec0096f0 0x7f21ec00bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:35.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.348 7f2202ffd700 1 -- 172.21.2.101:0/872473744 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ec00c410 con 0x7f21ec003b90 2021-02-10T11:13:35.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.348 7f2202ffd700 1 -- 172.21.2.101:0/872473744 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ec00c630 con 0x7f21ec0068d0 2021-02-10T11:13:35.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.348 7f2202ffd700 1 -- 172.21.2.101:0/872473744 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ec00c850 con 0x7f21ec0096f0 2021-02-10T11:13:35.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.349 7f2208c44700 1 --2- 172.21.2.101:0/872473744 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22040f20e0 0x7f22040f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:13:35.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.349 7f2203fff700 1 --2- 172.21.2.101:0/872473744 >> v2:172.21.2.108:3300/0 conn(0x7f22041b0470 0x7f22041b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:13:35.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.349 7f2203fff700 1 --2- 172.21.2.101:0/872473744 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21ec003b90 0x7f21ec0060b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:35.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.349 7f2208c44700 1 --2- 172.21.2.101:0/872473744 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21ec0096f0 0x7f21ec00bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:35.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.349 7f22037fe700 1 --2- 172.21.2.101:0/872473744 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21ec0068d0 0x7f21ec008df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:35.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.349 7f2203fff700 1 -- 172.21.2.101:0/872473744 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21ec0096f0 msgr2=0x7f21ec00bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:35.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.349 7f2203fff700 1 --2- 172.21.2.101:0/872473744 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21ec0096f0 0x7f21ec00bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:35.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.349 7f2203fff700 1 -- 172.21.2.101:0/872473744 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21ec0068d0 msgr2=0x7f21ec008df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:35.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.349 7f2203fff700 1 --2- 172.21.2.101:0/872473744 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21ec0068d0 0x7f21ec008df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:35.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.349 7f2203fff700 1 -- 172.21.2.101:0/872473744 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ec050b00 con 0x7f21ec003b90 2021-02-10T11:13:35.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.349 7f2203fff700 1 --2- 172.21.2.101:0/872473744 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21ec003b90 0x7f21ec0060b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f21f4009970 tx=0x7f21f4009d10).ready entity=mon.0 client_cookie=41b7386c40d6dd39 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:35.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.349 7f2202ffd700 1 -- 172.21.2.101:0/872473744 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21f4006960 con 0x7f21ec003b90 2021-02-10T11:13:35.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.349 7f2202ffd700 1 -- 172.21.2.101:0/872473744 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f4000c90 con 0x7f21ec003b90 2021-02-10T11:13:35.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.349 7f2202ffd700 1 -- 172.21.2.101:0/872473744 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21f4002220 con 0x7f21ec003b90 2021-02-10T11:13:35.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.349 7f220a627700 1 -- 172.21.2.101:0/872473744 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21ec003b90 msgr2=0x7f21ec0060b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:35.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.349 7f220a627700 1 --2- 172.21.2.101:0/872473744 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21ec003b90 0x7f21ec0060b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f21f4009970 tx=0x7f21f4009d10).stop 2021-02-10T11:13:35.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.349 7f2208c44700 1 -- 172.21.2.101:0/872473744 reap_dead start 2021-02-10T11:13:35.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.350 7f220a627700 1 -- 172.21.2.101:0/872473744 shutdown_connections 2021-02-10T11:13:35.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.350 7f220a627700 1 -- 172.21.2.101:0/872473744 >> 172.21.2.101:0/872473744 conn(0x7f22040ce3e0 msgr2=0x7f22040e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:35.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.350 7f220a627700 1 -- 172.21.2.101:0/872473744 shutdown_connections 2021-02-10T11:13:35.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.350 7f220a627700 1 -- 172.21.2.101:0/872473744 wait complete. 2021-02-10T11:13:35.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.365 7f220a627700 1 Processor -- start 2021-02-10T11:13:35.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.365 7f220a627700 1 -- start start 2021-02-10T11:13:35.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.365 7f220a627700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f220409b930 0x7f22040b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:35.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.365 7f220a627700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22040b8410 0x7f22040b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:35.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.365 7f220a627700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22040c1f80 0x7f22040c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:35.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.365 7f220a627700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22040c1a00 con 0x7f220409b930 2021-02-10T11:13:35.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.365 7f220a627700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22040b8e30 con 0x7f22040b8410 2021-02-10T11:13:35.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.365 7f220a627700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22040b9050 con 0x7f22040c1f80 2021-02-10T11:13:35.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.366 7f2203fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f220409b930 0x7f22040b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:35.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.366 7f2203fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f220409b930 0x7f22040b7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44426/0 (socket says 172.21.2.101:44426) 2021-02-10T11:13:35.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.366 7f2203fff700 1 -- 172.21.2.101:0/1377350998 learned_addr learned my addr 172.21.2.101:0/1377350998 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:35.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.366 7f22037fe700 1 --2- 172.21.2.101:0/1377350998 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22040b8410 0x7f22040b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:35.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.366 7f2208c44700 1 --2- 172.21.2.101:0/1377350998 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22040c1f80 0x7f22040c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:35.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.366 7f2203fff700 1 -- 172.21.2.101:0/1377350998 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22040c1f80 msgr2=0x7f22040c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:35.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.366 7f2203fff700 1 --2- 172.21.2.101:0/1377350998 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22040c1f80 0x7f22040c2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:35.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.366 7f2203fff700 1 -- 172.21.2.101:0/1377350998 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22040b8410 msgr2=0x7f22040b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:35.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.366 7f2203fff700 1 --2- 172.21.2.101:0/1377350998 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22040b8410 0x7f22040b8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:35.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.366 7f2203fff700 1 -- 172.21.2.101:0/1377350998 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22040c29a0 con 0x7f220409b930 2021-02-10T11:13:35.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.366 7f2203fff700 1 --2- 172.21.2.101:0/1377350998 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f220409b930 0x7f22040b7c20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f21f4009fc0 tx=0x7f21f400b9f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:35.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.366 7f2200ff9700 1 -- 172.21.2.101:0/1377350998 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21f4000c90 con 0x7f220409b930 2021-02-10T11:13:35.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.366 7f2200ff9700 1 -- 172.21.2.101:0/1377350998 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f4002030 con 0x7f220409b930 2021-02-10T11:13:35.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.366 7f220a627700 1 -- 172.21.2.101:0/1377350998 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22040c29a0 con 0x7f220409b930 2021-02-10T11:13:35.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.366 7f2200ff9700 1 -- 172.21.2.101:0/1377350998 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21f4002550 con 0x7f220409b930 2021-02-10T11:13:35.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.366 7f220a627700 1 -- 172.21.2.101:0/1377350998 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22040c29a0 con 0x7f220409b930 2021-02-10T11:13:35.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.367 7f2200ff9700 1 -- 172.21.2.101:0/1377350998 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f21f4002210 con 0x7f220409b930 2021-02-10T11:13:35.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.367 7f2200ff9700 1 --2- 172.21.2.101:0/1377350998 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21e4018730 0x7f21e401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:35.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.367 7f2200ff9700 1 -- 172.21.2.101:0/1377350998 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f21f4032670 con 0x7f220409b930 2021-02-10T11:13:35.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.367 7f22037fe700 1 --2- 172.21.2.101:0/1377350998 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21e4018730 0x7f21e401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:35.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.368 7f22037fe700 1 --2- 172.21.2.101:0/1377350998 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21e4018730 0x7f21e401ac10 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:35.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.397 7f220a627700 1 -- 172.21.2.101:0/1377350998 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2204105b30 con 0x7f220409b930 2021-02-10T11:13:35.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.400 7f2200ff9700 1 -- 172.21.2.101:0/1377350998 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f21f4006620 con 0x7f220409b930 2021-02-10T11:13:35.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.527 7f220a627700 1 -- 172.21.2.101:0/1377350998 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f22040f0b50 con 0x7f220409b930 2021-02-10T11:13:35.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.528 7f2200ff9700 1 -- 172.21.2.101:0/1377350998 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f21f400bde0 con 0x7f220409b930 2021-02-10T11:13:35.533 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:13:35.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.529 7f220a627700 1 -- 172.21.2.101:0/1377350998 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21e4018730 msgr2=0x7f21e401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:35.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.529 7f220a627700 1 --2- 172.21.2.101:0/1377350998 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21e4018730 0x7f21e401ac10 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:35.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.529 7f220a627700 1 -- 172.21.2.101:0/1377350998 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f220409b930 msgr2=0x7f22040b7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:35.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.529 7f220a627700 1 --2- 172.21.2.101:0/1377350998 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f220409b930 0x7f22040b7c20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f21f4009fc0 tx=0x7f21f400b9f0).stop 2021-02-10T11:13:35.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.530 7f220a627700 1 -- 172.21.2.101:0/1377350998 shutdown_connections 2021-02-10T11:13:35.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.530 7f220a627700 1 --2- 172.21.2.101:0/1377350998 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21e4018730 0x7f21e401ac10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:35.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.530 7f220a627700 1 --2- 172.21.2.101:0/1377350998 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22040c1f80 0x7f22040c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:35.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.530 7f220a627700 1 --2- 172.21.2.101:0/1377350998 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22040b8410 0x7f22040b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:35.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.530 7f220a627700 1 --2- 172.21.2.101:0/1377350998 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f220409b930 0x7f22040b7c20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:35.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.530 7f220a627700 1 -- 172.21.2.101:0/1377350998 >> 172.21.2.101:0/1377350998 conn(0x7f2204094e10 msgr2=0x7f22040a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:35.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.530 7f220a627700 1 -- 172.21.2.101:0/1377350998 shutdown_connections 2021-02-10T11:13:35.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.530 7f220a627700 1 -- 172.21.2.101:0/1377350998 wait complete. 2021-02-10T11:13:35.542 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:13:35.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.951 7f4ddda6c700 1 Processor -- start 2021-02-10T11:13:35.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.951 7f4ddda6c700 1 -- start start 2021-02-10T11:13:35.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.951 7f4ddda6c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4dd81b0470 0x7f4dd81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:35.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.951 7f4ddda6c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd80f0b50 con 0x7f4dd80ef610 2021-02-10T11:13:35.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.951 7f4ddda6c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd80f24e0 con 0x7f4dd80f20e0 2021-02-10T11:13:35.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.951 7f4ddda6c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd80e5ea0 con 0x7f4dd81b0470 2021-02-10T11:13:35.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.952 7f4dd6ffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f4dd80ef610 0x7f4dd80efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44928/0 (socket says 172.21.2.101:44928) 2021-02-10T11:13:35.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.952 7f4dd6ffd700 1 -- 172.21.2.101:0/2519693675 learned_addr learned my addr 172.21.2.101:0/2519693675 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:35.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.952 7f4dd67fc700 1 -- 172.21.2.101:0/2519693675 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3162709333 0 0) 0x7f4dc00010b0 con 0x7f4dd80ef610 2021-02-10T11:13:35.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.952 7f4dd67fc700 1 -- 172.21.2.101:0/2519693675 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dc4002fe0 con 0x7f4dd80ef610 2021-02-10T11:13:35.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.952 7f4dd77fe700 1 --2- 172.21.2.101:0/2519693675 >> v2:172.21.2.108:3300/0 conn(0x7f4dd81b0470 0x7f4dd81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:35.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.952 7f4dd67fc700 1 -- 172.21.2.101:0/2519693675 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 29331812 0 0) 0x7f4dc00010b0 con 0x7f4dd80ef610 2021-02-10T11:13:35.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.952 7f4dd67fc700 1 -- 172.21.2.101:0/2519693675 >> v2:172.21.2.108:3300/0 conn(0x7f4dd81b0470 msgr2=0x7f4dd81b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:35.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.952 7f4dd67fc700 1 --2- 172.21.2.101:0/2519693675 >> v2:172.21.2.108:3300/0 conn(0x7f4dd81b0470 0x7f4dd81b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:35.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.952 7f4dd67fc700 1 -- 172.21.2.101:0/2519693675 >> v1:172.21.2.108:6789/0 conn(0x7f4dd80f20e0 legacy=0x7f4dd80f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:35.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.952 7f4dd67fc700 1 -- 172.21.2.101:0/2519693675 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd80e52e0 con 0x7f4dd80ef610 2021-02-10T11:13:35.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.952 7f4dd67fc700 1 -- 172.21.2.101:0/2519693675 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f4dc00015b0 con 0x7f4dd80ef610 2021-02-10T11:13:35.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.952 7f4dd67fc700 1 -- 172.21.2.101:0/2519693675 >> v1:172.21.2.101:6789/0 conn(0x7f4dd80ef610 legacy=0x7f4dd80efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:35.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.952 7f4dd67fc700 1 --2- 172.21.2.101:0/2519693675 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dc4003a40 0x7f4dc4005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:35.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.952 7f4dd67fc700 1 --2- 172.21.2.101:0/2519693675 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dc4006800 0x7f4dc4008d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:35.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.952 7f4dd67fc700 1 --2- 172.21.2.101:0/2519693675 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4dc4009620 0x7f4dc400bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:35.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.952 7f4dd67fc700 1 -- 172.21.2.101:0/2519693675 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc400c310 con 0x7f4dc4006800 2021-02-10T11:13:35.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.952 7f4dd67fc700 1 -- 172.21.2.101:0/2519693675 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc400c530 con 0x7f4dc4009620 2021-02-10T11:13:35.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.952 7f4dd67fc700 1 -- 172.21.2.101:0/2519693675 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc400c750 con 0x7f4dc4003a40 2021-02-10T11:13:35.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.953 7f4dd6ffd700 1 --2- 172.21.2.101:0/2519693675 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dc4006800 0x7f4dc4008d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:35.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.953 7f4dd77fe700 1 --2- 172.21.2.101:0/2519693675 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dc4003a40 0x7f4dc4005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:35.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.953 7f4dd7fff700 1 --2- 172.21.2.101:0/2519693675 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4dc4009620 0x7f4dc400bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:35.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.953 7f4dd6ffd700 1 -- 172.21.2.101:0/2519693675 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dc4003a40 msgr2=0x7f4dc4005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:35.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.953 7f4dd6ffd700 1 --2- 172.21.2.101:0/2519693675 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dc4003a40 0x7f4dc4005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:35.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.953 7f4dd6ffd700 1 -- 172.21.2.101:0/2519693675 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4dc4009620 msgr2=0x7f4dc400bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:35.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.953 7f4dd6ffd700 1 --2- 172.21.2.101:0/2519693675 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4dc4009620 0x7f4dc400bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:35.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.953 7f4dd6ffd700 1 -- 172.21.2.101:0/2519693675 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc400c9f0 con 0x7f4dc4006800 2021-02-10T11:13:35.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.953 7f4dd7fff700 1 --2- 172.21.2.101:0/2519693675 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4dc4009620 0x7f4dc400bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:13:35.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.953 7f4dd7fff700 1 -- 172.21.2.101:0/2519693675 reap_dead start 2021-02-10T11:13:35.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.953 7f4dd6ffd700 1 --2- 172.21.2.101:0/2519693675 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dc4006800 0x7f4dc4008d20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f4dc0007560 tx=0x7f4dc0007c90).ready entity=mon.0 client_cookie=caf1ea4274d11f62 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:35.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.953 7f4dd67fc700 1 -- 172.21.2.101:0/2519693675 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4dc00010b0 con 0x7f4dc4006800 2021-02-10T11:13:35.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.953 7f4dd67fc700 1 -- 172.21.2.101:0/2519693675 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc0001400 con 0x7f4dc4006800 2021-02-10T11:13:35.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.953 7f4dd67fc700 1 -- 172.21.2.101:0/2519693675 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4dc004e510 con 0x7f4dc4006800 2021-02-10T11:13:35.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.953 7f4ddda6c700 1 -- 172.21.2.101:0/2519693675 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dc4006800 msgr2=0x7f4dc4008d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:35.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.953 7f4ddda6c700 1 --2- 172.21.2.101:0/2519693675 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dc4006800 0x7f4dc4008d20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f4dc0007560 tx=0x7f4dc0007c90).stop 2021-02-10T11:13:35.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.954 7f4ddda6c700 1 -- 172.21.2.101:0/2519693675 shutdown_connections 2021-02-10T11:13:35.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.954 7f4ddda6c700 1 --2- 172.21.2.101:0/2519693675 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dc4006800 0x7f4dc4008d20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:35.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.954 7f4ddda6c700 1 -- 172.21.2.101:0/2519693675 >> 172.21.2.101:0/2519693675 conn(0x7f4dd80ce3e0 msgr2=0x7f4dd80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:35.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.954 7f4ddda6c700 1 -- 172.21.2.101:0/2519693675 shutdown_connections 2021-02-10T11:13:35.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.954 7f4ddda6c700 1 -- 172.21.2.101:0/2519693675 wait complete. 2021-02-10T11:13:35.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.969 7f4ddda6c700 1 Processor -- start 2021-02-10T11:13:35.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.969 7f4ddda6c700 1 -- start start 2021-02-10T11:13:35.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.969 7f4ddda6c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4dd80bf920 0x7f4dd80bfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:35.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.969 7f4ddda6c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dd80b6570 0x7f4dd80b6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:35.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.969 7f4ddda6c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dd81a7430 0x7f4dd81a7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.969 7f4ddda6c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd80c0440 con 0x7f4dd81a7430 2021-02-10T11:13:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.969 7f4ddda6c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd80924b0 con 0x7f4dd80bf920 2021-02-10T11:13:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.969 7f4ddda6c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd8092970 con 0x7f4dd80b6570 2021-02-10T11:13:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.970 7f4dd7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dd81a7430 0x7f4dd81a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.970 7f4dd7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dd81a7430 0x7f4dd81a7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44444/0 (socket says 172.21.2.101:44444) 2021-02-10T11:13:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.970 7f4dd7fff700 1 -- 172.21.2.101:0/137220931 learned_addr learned my addr 172.21.2.101:0/137220931 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.970 7f4dd7fff700 1 -- 172.21.2.101:0/137220931 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dd80b6570 msgr2=0x7f4dd80b6970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.970 7f4dd7fff700 1 --2- 172.21.2.101:0/137220931 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dd80b6570 0x7f4dd80b6970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.970 7f4dd7fff700 1 -- 172.21.2.101:0/137220931 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4dd80bf920 msgr2=0x7f4dd80bfd20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.970 7f4dd7fff700 1 --2- 172.21.2.101:0/137220931 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4dd80bf920 0x7f4dd80bfd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.970 7f4dd7fff700 1 -- 172.21.2.101:0/137220931 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd81a7e50 con 0x7f4dd81a7430 2021-02-10T11:13:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.970 7f4dd7fff700 1 --2- 172.21.2.101:0/137220931 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dd81a7430 0x7f4dd81a7830 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f4dd00099a0 tx=0x7f4dd000bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:35.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.970 7f4dcffff700 1 -- 172.21.2.101:0/137220931 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4dd0000ce0 con 0x7f4dd81a7430 2021-02-10T11:13:35.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.970 7f4dcffff700 1 -- 172.21.2.101:0/137220931 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd0006030 con 0x7f4dd81a7430 2021-02-10T11:13:35.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.970 7f4dcffff700 1 -- 172.21.2.101:0/137220931 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4dd0006550 con 0x7f4dd81a7430 2021-02-10T11:13:35.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.970 7f4ddda6c700 1 -- 172.21.2.101:0/137220931 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd81a7e50 con 0x7f4dd81a7430 2021-02-10T11:13:35.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.970 7f4ddda6c700 1 -- 172.21.2.101:0/137220931 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd80c09c0 con 0x7f4dd81a7430 2021-02-10T11:13:35.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.971 7f4dcffff700 1 -- 172.21.2.101:0/137220931 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4dd0006210 con 0x7f4dd81a7430 2021-02-10T11:13:35.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.971 7f4dcffff700 1 --2- 172.21.2.101:0/137220931 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4dbc0186c0 0x7f4dbc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:35.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.971 7f4dcffff700 1 -- 172.21.2.101:0/137220931 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4dd00328c0 con 0x7f4dd81a7430 2021-02-10T11:13:35.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.971 7f4dd77fe700 1 --2- 172.21.2.101:0/137220931 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4dbc0186c0 0x7f4dbc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:35.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:35.972 7f4dd77fe700 1 --2- 172.21.2.101:0/137220931 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4dbc0186c0 0x7f4dbc01abc0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:36.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.001 7f4ddda6c700 1 -- 172.21.2.101:0/137220931 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd8105b40 con 0x7f4dd81a7430 2021-02-10T11:13:36.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.004 7f4dcffff700 1 -- 172.21.2.101:0/137220931 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4dd0000cc0 con 0x7f4dd81a7430 2021-02-10T11:13:36.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.130 7f4ddda6c700 1 -- 172.21.2.101:0/137220931 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f4dd80f0b50 con 0x7f4dd81a7430 2021-02-10T11:13:36.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.130 7f4dcffff700 1 -- 172.21.2.101:0/137220931 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v20) v1 ==== 104+0+10 (secure 0 0 0) 0x7f4dd0000cc0 con 0x7f4dd81a7430 2021-02-10T11:13:36.135 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:13:36.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.132 7f4ddda6c700 1 -- 172.21.2.101:0/137220931 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4dbc0186c0 msgr2=0x7f4dbc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:36.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.132 7f4ddda6c700 1 --2- 172.21.2.101:0/137220931 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4dbc0186c0 0x7f4dbc01abc0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:36.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.132 7f4ddda6c700 1 -- 172.21.2.101:0/137220931 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dd81a7430 msgr2=0x7f4dd81a7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:36.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.132 7f4ddda6c700 1 --2- 172.21.2.101:0/137220931 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dd81a7430 0x7f4dd81a7830 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f4dd00099a0 tx=0x7f4dd000bcf0).stop 2021-02-10T11:13:36.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.132 7f4ddda6c700 1 -- 172.21.2.101:0/137220931 shutdown_connections 2021-02-10T11:13:36.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.132 7f4ddda6c700 1 --2- 172.21.2.101:0/137220931 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4dbc0186c0 0x7f4dbc01abc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:36.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.132 7f4ddda6c700 1 --2- 172.21.2.101:0/137220931 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dd80b6570 0x7f4dd80b6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:36.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.132 7f4ddda6c700 1 --2- 172.21.2.101:0/137220931 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dd81a7430 0x7f4dd81a7830 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:36.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.132 7f4ddda6c700 1 --2- 172.21.2.101:0/137220931 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4dd80bf920 0x7f4dd80bfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:36.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.132 7f4ddda6c700 1 -- 172.21.2.101:0/137220931 >> 172.21.2.101:0/137220931 conn(0x7f4dd80b50c0 msgr2=0x7f4dd8092d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:36.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.133 7f4ddda6c700 1 -- 172.21.2.101:0/137220931 shutdown_connections 2021-02-10T11:13:36.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.133 7f4ddda6c700 1 -- 172.21.2.101:0/137220931 wait complete. 2021-02-10T11:13:36.145 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:13:36.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.552 7f3fcedbf700 1 Processor -- start 2021-02-10T11:13:36.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.552 7f3fcedbf700 1 -- start start 2021-02-10T11:13:36.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.553 7f3fcedbf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3fc80f05f0 0x7f3fc80f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:36.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.553 7f3fcedbf700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc81b0c70 con 0x7f3fc81b0470 2021-02-10T11:13:36.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.553 7f3fcedbf700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc80f0b50 con 0x7f3fc81b0870 2021-02-10T11:13:36.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.553 7f3fcedbf700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc80e5ea0 con 0x7f3fc80f05f0 2021-02-10T11:13:36.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.553 7f3fccbdb700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f3fc81b0470 0x7f3fc80ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44944/0 (socket says 172.21.2.101:44944) 2021-02-10T11:13:36.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.553 7f3fccbdb700 1 -- 172.21.2.101:0/1947220859 learned_addr learned my addr 172.21.2.101:0/1947220859 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:36.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.553 7f3fc77fe700 1 -- 172.21.2.101:0/1947220859 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3758071109 0 0) 0x7f3fb8001180 con 0x7f3fc81b0470 2021-02-10T11:13:36.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fcd3dc700 1 --2- 172.21.2.101:0/1947220859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3fc80f05f0 0x7f3fc80f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:36.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fc77fe700 1 -- 172.21.2.101:0/1947220859 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fac002fe0 con 0x7f3fc81b0470 2021-02-10T11:13:36.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fc77fe700 1 -- 172.21.2.101:0/1947220859 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3605717513 0 0) 0x7f3fb8001180 con 0x7f3fc81b0470 2021-02-10T11:13:36.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fc77fe700 1 -- 172.21.2.101:0/1947220859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3fc80f05f0 msgr2=0x7f3fc80f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:36.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fc77fe700 1 --2- 172.21.2.101:0/1947220859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3fc80f05f0 0x7f3fc80f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:36.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fc77fe700 1 -- 172.21.2.101:0/1947220859 >> v1:172.21.2.108:6789/0 conn(0x7f3fc81b0870 legacy=0x7f3fc80efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:36.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fc77fe700 1 -- 172.21.2.101:0/1947220859 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc80e52e0 con 0x7f3fc81b0470 2021-02-10T11:13:36.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fc77fe700 1 -- 172.21.2.101:0/1947220859 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f3fb8001680 con 0x7f3fc81b0470 2021-02-10T11:13:36.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fc77fe700 1 -- 172.21.2.101:0/1947220859 >> v1:172.21.2.101:6789/0 conn(0x7f3fc81b0470 legacy=0x7f3fc80ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:36.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fc77fe700 1 --2- 172.21.2.101:0/1947220859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fac003aa0 0x7f3fac005ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:36.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fc77fe700 1 --2- 172.21.2.101:0/1947220859 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3fac006770 0x7f3fac008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:36.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fc77fe700 1 --2- 172.21.2.101:0/1947220859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3fac009590 0x7f3fac00ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:36.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fc77fe700 1 -- 172.21.2.101:0/1947220859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac00c2d0 con 0x7f3fac003aa0 2021-02-10T11:13:36.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fc77fe700 1 -- 172.21.2.101:0/1947220859 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac00c4f0 con 0x7f3fac006770 2021-02-10T11:13:36.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fc77fe700 1 -- 172.21.2.101:0/1947220859 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac00c710 con 0x7f3fac009590 2021-02-10T11:13:36.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fccbdb700 1 --2- 172.21.2.101:0/1947220859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fac003aa0 0x7f3fac005ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:36.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fcd3dc700 1 --2- 172.21.2.101:0/1947220859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3fac009590 0x7f3fac00ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:36.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fc7fff700 1 --2- 172.21.2.101:0/1947220859 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3fac006770 0x7f3fac008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:36.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fccbdb700 1 -- 172.21.2.101:0/1947220859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3fac009590 msgr2=0x7f3fac00ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:36.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fccbdb700 1 --2- 172.21.2.101:0/1947220859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3fac009590 0x7f3fac00ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:36.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fccbdb700 1 -- 172.21.2.101:0/1947220859 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3fac006770 msgr2=0x7f3fac008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:36.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fccbdb700 1 --2- 172.21.2.101:0/1947220859 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3fac006770 0x7f3fac008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:36.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fccbdb700 1 -- 172.21.2.101:0/1947220859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fac050980 con 0x7f3fac003aa0 2021-02-10T11:13:36.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.554 7f3fc7fff700 1 --2- 172.21.2.101:0/1947220859 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3fac006770 0x7f3fac008c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:13:36.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.555 7f3fccbdb700 1 --2- 172.21.2.101:0/1947220859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fac003aa0 0x7f3fac005ed0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f3fb8000e80 tx=0x7f3fb8007d00).ready entity=mon.0 client_cookie=5ab03b89e56a3783 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:36.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.555 7f3fc77fe700 1 -- 172.21.2.101:0/1947220859 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3fb8001180 con 0x7f3fac003aa0 2021-02-10T11:13:36.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.555 7f3fc77fe700 1 -- 172.21.2.101:0/1947220859 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb800a520 con 0x7f3fac003aa0 2021-02-10T11:13:36.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.555 7f3fc77fe700 1 -- 172.21.2.101:0/1947220859 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3fb800a970 con 0x7f3fac003aa0 2021-02-10T11:13:36.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.555 7f3fcedbf700 1 -- 172.21.2.101:0/1947220859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fac003aa0 msgr2=0x7f3fac005ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:36.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.555 7f3fcedbf700 1 --2- 172.21.2.101:0/1947220859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fac003aa0 0x7f3fac005ed0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f3fb8000e80 tx=0x7f3fb8007d00).stop 2021-02-10T11:13:36.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.555 7f3fcd3dc700 1 -- 172.21.2.101:0/1947220859 reap_dead start 2021-02-10T11:13:36.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.555 7f3fcedbf700 1 -- 172.21.2.101:0/1947220859 shutdown_connections 2021-02-10T11:13:36.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.555 7f3fcedbf700 1 -- 172.21.2.101:0/1947220859 >> 172.21.2.101:0/1947220859 conn(0x7f3fc80ce3e0 msgr2=0x7f3fc80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:36.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.555 7f3fcedbf700 1 -- 172.21.2.101:0/1947220859 shutdown_connections 2021-02-10T11:13:36.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.555 7f3fcedbf700 1 -- 172.21.2.101:0/1947220859 wait complete. 2021-02-10T11:13:36.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.570 7f3fcedbf700 1 Processor -- start 2021-02-10T11:13:36.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.570 7f3fcedbf700 1 -- start start 2021-02-10T11:13:36.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.570 7f3fcedbf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fc80b7db0 0x7f3fc80b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:36.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.570 7f3fcedbf700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3fc80b8880 0x7f3fc81a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:36.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.570 7f3fcedbf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3fc80b8c80 0x7f3fc81a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:36.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.570 7f3fcedbf700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc80b79b0 con 0x7f3fc80b7db0 2021-02-10T11:13:36.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.570 7f3fcedbf700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc81a8090 con 0x7f3fc80b8880 2021-02-10T11:13:36.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.570 7f3fcedbf700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc81a82b0 con 0x7f3fc80b8c80 2021-02-10T11:13:36.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.571 7f3fccbdb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fc80b7db0 0x7f3fc80b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:36.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.571 7f3fccbdb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fc80b7db0 0x7f3fc80b8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44466/0 (socket says 172.21.2.101:44466) 2021-02-10T11:13:36.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.571 7f3fccbdb700 1 -- 172.21.2.101:0/82406950 learned_addr learned my addr 172.21.2.101:0/82406950 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:36.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.571 7f3fccbdb700 1 -- 172.21.2.101:0/82406950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3fc80b8c80 msgr2=0x7f3fc81a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:36.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.571 7f3fccbdb700 1 --2- 172.21.2.101:0/82406950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3fc80b8c80 0x7f3fc81a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:36.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.571 7f3fccbdb700 1 -- 172.21.2.101:0/82406950 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3fc80b8880 msgr2=0x7f3fc81a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:36.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.571 7f3fccbdb700 1 --2- 172.21.2.101:0/82406950 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3fc80b8880 0x7f3fc81a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:36.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.571 7f3fccbdb700 1 -- 172.21.2.101:0/82406950 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc8190c10 con 0x7f3fc80b7db0 2021-02-10T11:13:36.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.571 7f3fccbdb700 1 --2- 172.21.2.101:0/82406950 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fc80b7db0 0x7f3fc80b8260 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f3fb80055e0 tx=0x7f3fb8005980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:36.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.571 7f3fc57fa700 1 -- 172.21.2.101:0/82406950 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3fb8007970 con 0x7f3fc80b7db0 2021-02-10T11:13:36.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.571 7f3fc57fa700 1 -- 172.21.2.101:0/82406950 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb8007c70 con 0x7f3fc80b7db0 2021-02-10T11:13:36.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.571 7f3fcedbf700 1 -- 172.21.2.101:0/82406950 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc8190c10 con 0x7f3fc80b7db0 2021-02-10T11:13:36.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.571 7f3fc57fa700 1 -- 172.21.2.101:0/82406950 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3fb800a350 con 0x7f3fc80b7db0 2021-02-10T11:13:36.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.571 7f3fcedbf700 1 -- 172.21.2.101:0/82406950 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc80c1eb0 con 0x7f3fc80b7db0 2021-02-10T11:13:36.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.572 7f3fc57fa700 1 -- 172.21.2.101:0/82406950 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3fb8007970 con 0x7f3fc80b7db0 2021-02-10T11:13:36.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.572 7f3fc57fa700 1 --2- 172.21.2.101:0/82406950 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3fa40186c0 0x7f3fa401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:36.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.572 7f3fc57fa700 1 -- 172.21.2.101:0/82406950 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3fb80323c0 con 0x7f3fc80b7db0 2021-02-10T11:13:36.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.572 7f3fc7fff700 1 --2- 172.21.2.101:0/82406950 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3fa40186c0 0x7f3fa401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:36.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.572 7f3fc7fff700 1 --2- 172.21.2.101:0/82406950 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3fa40186c0 0x7f3fa401abc0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:36.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.603 7f3fcedbf700 1 -- 172.21.2.101:0/82406950 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc8105b30 con 0x7f3fc80b7db0 2021-02-10T11:13:36.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.605 7f3fc57fa700 1 -- 172.21.2.101:0/82406950 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3fb8030cf0 con 0x7f3fc80b7db0 2021-02-10T11:13:36.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.732 7f3fcedbf700 1 -- 172.21.2.101:0/82406950 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f3fc8105b30 con 0x7f3fc80b7db0 2021-02-10T11:13:36.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.732 7f3fc57fa700 1 -- 172.21.2.101:0/82406950 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v20) v1 ==== 100+0+10 (secure 0 0 0) 0x7f3fb8007970 con 0x7f3fc80b7db0 2021-02-10T11:13:36.737 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:13:36.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.734 7f3fcedbf700 1 -- 172.21.2.101:0/82406950 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3fa40186c0 msgr2=0x7f3fa401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:36.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.734 7f3fcedbf700 1 --2- 172.21.2.101:0/82406950 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3fa40186c0 0x7f3fa401abc0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:36.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.734 7f3fcedbf700 1 -- 172.21.2.101:0/82406950 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fc80b7db0 msgr2=0x7f3fc80b8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:36.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.734 7f3fcedbf700 1 --2- 172.21.2.101:0/82406950 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fc80b7db0 0x7f3fc80b8260 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f3fb80055e0 tx=0x7f3fb8005980).stop 2021-02-10T11:13:36.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.734 7f3fcedbf700 1 -- 172.21.2.101:0/82406950 shutdown_connections 2021-02-10T11:13:36.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.734 7f3fcedbf700 1 --2- 172.21.2.101:0/82406950 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3fa40186c0 0x7f3fa401abc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:36.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.734 7f3fcedbf700 1 --2- 172.21.2.101:0/82406950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3fc80b8c80 0x7f3fc81a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:36.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.734 7f3fcedbf700 1 --2- 172.21.2.101:0/82406950 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3fc80b8880 0x7f3fc81a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:36.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.734 7f3fcedbf700 1 --2- 172.21.2.101:0/82406950 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fc80b7db0 0x7f3fc80b8260 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:36.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.734 7f3fcedbf700 1 -- 172.21.2.101:0/82406950 >> 172.21.2.101:0/82406950 conn(0x7f3fc8094e10 msgr2=0x7f3fc80a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:36.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.734 7f3fcedbf700 1 -- 172.21.2.101:0/82406950 shutdown_connections 2021-02-10T11:13:36.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:36.734 7f3fcedbf700 1 -- 172.21.2.101:0/82406950 wait complete. 2021-02-10T11:13:36.747 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:13:37.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.151 7fa74dc54700 1 Processor -- start 2021-02-10T11:13:37.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.152 7fa74dc54700 1 -- start start 2021-02-10T11:13:37.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.152 7fa74dc54700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7480e5470 0x7fa74807e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:37.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.152 7fa74dc54700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa74807ed80 con 0x7fa74807f300 2021-02-10T11:13:37.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.152 7fa74dc54700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7481b4010 con 0x7fa7480e5070 2021-02-10T11:13:37.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.152 7fa74dc54700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7480d1790 con 0x7fa7480e5470 2021-02-10T11:13:37.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.152 7fa747fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa74807f300 0x7fa74807f700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:44968/0 (socket says 172.21.2.101:44968) 2021-02-10T11:13:37.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.152 7fa747fff700 1 -- 172.21.2.101:0/3022783047 learned_addr learned my addr 172.21.2.101:0/3022783047 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:37.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.152 7fa746ffd700 1 --2- 172.21.2.101:0/3022783047 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7480e5470 0x7fa74807e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:37.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.153 7fa7467fc700 1 -- 172.21.2.101:0/3022783047 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635618986 0 0) 0x7fa73c000c60 con 0x7fa7480e5070 2021-02-10T11:13:37.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.153 7fa7467fc700 1 -- 172.21.2.101:0/3022783047 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa730002fe0 con 0x7fa7480e5070 2021-02-10T11:13:37.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.153 7fa7467fc700 1 -- 172.21.2.101:0/3022783047 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1457404621 0 0) 0x7fa738001180 con 0x7fa74807f300 2021-02-10T11:13:37.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.153 7fa7467fc700 1 -- 172.21.2.101:0/3022783047 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa730002fe0 con 0x7fa74807f300 2021-02-10T11:13:37.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.153 7fa746ffd700 1 -- 172.21.2.101:0/3022783047 >> v1:172.21.2.108:6789/0 conn(0x7fa7480e5070 legacy=0x7fa7481b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:37.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.153 7fa746ffd700 1 -- 172.21.2.101:0/3022783047 >> v1:172.21.2.101:6789/0 conn(0x7fa74807f300 legacy=0x7fa74807f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:37.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.153 7fa746ffd700 1 -- 172.21.2.101:0/3022783047 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7480d1980 con 0x7fa7480e5470 2021-02-10T11:13:37.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.153 7fa746ffd700 1 --2- 172.21.2.101:0/3022783047 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7480e5470 0x7fa74807e6a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fa7400069d0 tx=0x7fa740009dc0).ready entity=mon.2 client_cookie=34feffbf4e95a584 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:37.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.153 7fa7467fc700 1 -- 172.21.2.101:0/3022783047 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa74000b510 con 0x7fa7480e5470 2021-02-10T11:13:37.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.153 7fa7467fc700 1 -- 172.21.2.101:0/3022783047 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74000b790 con 0x7fa7480e5470 2021-02-10T11:13:37.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.153 7fa7467fc700 1 -- 172.21.2.101:0/3022783047 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa74000bbf0 con 0x7fa7480e5470 2021-02-10T11:13:37.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.153 7fa74dc54700 1 -- 172.21.2.101:0/3022783047 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7480e5470 msgr2=0x7fa74807e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:37.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.154 7fa74dc54700 1 --2- 172.21.2.101:0/3022783047 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7480e5470 0x7fa74807e6a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fa7400069d0 tx=0x7fa740009dc0).stop 2021-02-10T11:13:37.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.154 7fa74dc54700 1 -- 172.21.2.101:0/3022783047 shutdown_connections 2021-02-10T11:13:37.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.154 7fa74dc54700 1 --2- 172.21.2.101:0/3022783047 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7480e5470 0x7fa74807e6a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:37.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.154 7fa74dc54700 1 -- 172.21.2.101:0/3022783047 >> 172.21.2.101:0/3022783047 conn(0x7fa7480ce690 msgr2=0x7fa7480e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:37.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.154 7fa74dc54700 1 -- 172.21.2.101:0/3022783047 shutdown_connections 2021-02-10T11:13:37.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.154 7fa74dc54700 1 -- 172.21.2.101:0/3022783047 wait complete. 2021-02-10T11:13:37.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.169 7fa74dc54700 1 Processor -- start 2021-02-10T11:13:37.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.169 7fa74dc54700 1 -- start start 2021-02-10T11:13:37.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.169 7fa74dc54700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa74808d400 0x7fa748096c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:37.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.169 7fa74dc54700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7480b1450 0x7fa7480b1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:37.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.169 7fa74dc54700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7480b1e70 0x7fa7481b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:37.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.169 7fa74dc54700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa748097310 con 0x7fa7480b1450 2021-02-10T11:13:37.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.169 7fa74dc54700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74808d800 con 0x7fa74808d400 2021-02-10T11:13:37.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.169 7fa74dc54700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa74808d9f0 con 0x7fa7480b1e70 2021-02-10T11:13:37.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.170 7fa746ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7480b1450 0x7fa7480b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:37.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.170 7fa746ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7480b1450 0x7fa7480b1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44480/0 (socket says 172.21.2.101:44480) 2021-02-10T11:13:37.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.170 7fa746ffd700 1 -- 172.21.2.101:0/829121240 learned_addr learned my addr 172.21.2.101:0/829121240 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:37.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.170 7fa7477fe700 1 --2- 172.21.2.101:0/829121240 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa74808d400 0x7fa748096c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:37.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.170 7fa747fff700 1 --2- 172.21.2.101:0/829121240 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7480b1e70 0x7fa7481b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:37.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.170 7fa746ffd700 1 -- 172.21.2.101:0/829121240 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7480b1e70 msgr2=0x7fa7481b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:37.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.170 7fa746ffd700 1 --2- 172.21.2.101:0/829121240 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7480b1e70 0x7fa7481b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:37.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.170 7fa746ffd700 1 -- 172.21.2.101:0/829121240 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa74808d400 msgr2=0x7fa748096c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:37.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.170 7fa746ffd700 1 --2- 172.21.2.101:0/829121240 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa74808d400 0x7fa748096c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:37.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.170 7fa746ffd700 1 -- 172.21.2.101:0/829121240 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7481b25f0 con 0x7fa7480b1450 2021-02-10T11:13:37.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.170 7fa7477fe700 1 --2- 172.21.2.101:0/829121240 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa74808d400 0x7fa748096c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:13:37.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.170 7fa746ffd700 1 --2- 172.21.2.101:0/829121240 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7480b1450 0x7fa7480b1850 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fa7400099c0 tx=0x7fa74000bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:37.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.170 7fa72ffff700 1 -- 172.21.2.101:0/829121240 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa740006240 con 0x7fa7480b1450 2021-02-10T11:13:37.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.170 7fa72ffff700 1 -- 172.21.2.101:0/829121240 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa740006540 con 0x7fa7480b1450 2021-02-10T11:13:37.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.170 7fa74dc54700 1 -- 172.21.2.101:0/829121240 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7481b25f0 con 0x7fa7480b1450 2021-02-10T11:13:37.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.170 7fa74dc54700 1 -- 172.21.2.101:0/829121240 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7481b2b80 con 0x7fa7480b1450 2021-02-10T11:13:37.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.170 7fa72ffff700 1 -- 172.21.2.101:0/829121240 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa740005850 con 0x7fa7480b1450 2021-02-10T11:13:37.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.171 7fa72ffff700 1 -- 172.21.2.101:0/829121240 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa740006030 con 0x7fa7480b1450 2021-02-10T11:13:37.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.171 7fa72ffff700 1 --2- 172.21.2.101:0/829121240 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7280186c0 0x7fa72801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:37.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.171 7fa72ffff700 1 -- 172.21.2.101:0/829121240 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa740032b60 con 0x7fa7480b1450 2021-02-10T11:13:37.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.171 7fa7477fe700 1 --2- 172.21.2.101:0/829121240 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7280186c0 0x7fa72801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:37.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.171 7fa7477fe700 1 --2- 172.21.2.101:0/829121240 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7280186c0 0x7fa72801abc0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:37.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.202 7fa74dc54700 1 -- 172.21.2.101:0/829121240 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa74807ed80 con 0x7fa7480b1450 2021-02-10T11:13:37.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.205 7fa72ffff700 1 -- 172.21.2.101:0/829121240 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa740000d30 con 0x7fa7480b1450 2021-02-10T11:13:37.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.334 7fa74dc54700 1 -- 172.21.2.101:0/829121240 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa748097290 con 0x7fa7480b1450 2021-02-10T11:13:37.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.334 7fa72ffff700 1 -- 172.21.2.101:0/829121240 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2097 (secure 0 0 0) 0x7fa740006540 con 0x7fa7480b1450 2021-02-10T11:13:37.339 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:13:37.339 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/3184957761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3184957761},{"type":"v1","addr":"172.21.2.108:6833","nonce":3184957761}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:13:14.147021","modified":"2021-02-10 11:13:23.547179","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":5,"state":"up:active","state_seq":3,"addr":"172.21.2.101:6835/3530104431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3530104431},{"type":"v1","addr":"172.21.2.101:6835","nonce":3530104431}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-02-10T11:13:37.339 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 6 2021-02-10T11:13:37.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.336 7fa74dc54700 1 -- 172.21.2.101:0/829121240 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7280186c0 msgr2=0x7fa72801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:37.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.336 7fa74dc54700 1 --2- 172.21.2.101:0/829121240 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7280186c0 0x7fa72801abc0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:37.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.336 7fa74dc54700 1 -- 172.21.2.101:0/829121240 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7480b1450 msgr2=0x7fa7480b1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:37.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.336 7fa74dc54700 1 --2- 172.21.2.101:0/829121240 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7480b1450 0x7fa7480b1850 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fa7400099c0 tx=0x7fa74000bd40).stop 2021-02-10T11:13:37.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.336 7fa74dc54700 1 -- 172.21.2.101:0/829121240 shutdown_connections 2021-02-10T11:13:37.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.336 7fa74dc54700 1 --2- 172.21.2.101:0/829121240 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7280186c0 0x7fa72801abc0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:37.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.336 7fa74dc54700 1 --2- 172.21.2.101:0/829121240 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7480b1e70 0x7fa7481b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:37.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.336 7fa74dc54700 1 --2- 172.21.2.101:0/829121240 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7480b1450 0x7fa7480b1850 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:37.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.336 7fa74dc54700 1 --2- 172.21.2.101:0/829121240 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa74808d400 0x7fa748096c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:37.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.336 7fa74dc54700 1 -- 172.21.2.101:0/829121240 >> 172.21.2.101:0/829121240 conn(0x7fa7481ac620 msgr2=0x7fa7480896e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:37.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.336 7fa74dc54700 1 -- 172.21.2.101:0/829121240 shutdown_connections 2021-02-10T11:13:37.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:37.336 7fa74dc54700 1 -- 172.21.2.101:0/829121240 wait complete. 2021-02-10T11:13:37.550 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:13:14.147021', 'modified': '2021-02-10 11:13:23.547179', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4299': {'gid': 4299, 'name': 'a', 'rank': 0, 'incarnation': 5, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.101:6835/3530104431', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.101:6834', 'nonce': 3530104431}, {'type': 'v1', 'addr': '172.21.2.101:6835', 'nonce': 3530104431}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:13:37.550 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:13:37.550 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok status 2021-02-10T11:13:37.947 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:13:37.948 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:13:37.948 INFO:teuthology.orchestra.run.gibba001.stdout: "whoami": 0, 2021-02-10T11:13:37.948 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 4299, 2021-02-10T11:13:37.948 INFO:teuthology.orchestra.run.gibba001.stdout: "want_state": "up:active", 2021-02-10T11:13:37.948 INFO:teuthology.orchestra.run.gibba001.stdout: "state": "up:active", 2021-02-10T11:13:37.948 INFO:teuthology.orchestra.run.gibba001.stdout: "rank_uptime": 15.396639113999999, 2021-02-10T11:13:37.948 INFO:teuthology.orchestra.run.gibba001.stdout: "mdsmap_epoch": 6, 2021-02-10T11:13:37.948 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch": 20, 2021-02-10T11:13:37.948 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch_barrier": 20, 2021-02-10T11:13:37.948 INFO:teuthology.orchestra.run.gibba001.stdout: "uptime": 19.788570367999998 2021-02-10T11:13:37.948 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:13:37.951 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 4299, "want_state": "up:active", "state": "up:active", "rank_uptime": 15.396639113999999, "mdsmap_epoch": 6, "osdmap_epoch": 20, "osdmap_epoch_barrier": 20, "uptime": 19.788570367999998 } 2021-02-10T11:13:37.952 INFO:teuthology.run_tasks:Running task ceph-fuse... 2021-02-10T11:13:37.979 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2021-02-10T11:13:37.979 INFO:tasks.ceph_fuse:config is None 2021-02-10T11:13:37.979 INFO:tasks.ceph_fuse:new config is {'client.0': None, 'client.1': None, 'client.2': None, 'client.3': None} 2021-02-10T11:13:37.979 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2021-02-10T11:13:37.981 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:13:38.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.388 7fa4f7cbb700 1 Processor -- start 2021-02-10T11:13:38.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.388 7fa4f7cbb700 1 -- start start 2021-02-10T11:13:38.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.388 7fa4f7cbb700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa4f00e5070 0x7fa4f01b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:38.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.389 7fa4f7cbb700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa4f01b3e40 0x7fa4f007d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:38.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.389 7fa4f7cbb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4f00e5470 0x7fa4f007dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:38.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.389 7fa4f7cbb700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f007ed80 con 0x7fa4f01b3e40 2021-02-10T11:13:38.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.389 7fa4f7cbb700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f007e2d0 con 0x7fa4f00e5070 2021-02-10T11:13:38.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.389 7fa4f7cbb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f007e4f0 con 0x7fa4f00e5470 2021-02-10T11:13:38.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.389 7fa4f52d6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa4f01b3e40 0x7fa4f007d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:38.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.389 7fa4f52d6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa4f01b3e40 0x7fa4f007d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44484/0 (socket says 172.21.2.101:44484) 2021-02-10T11:13:38.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.389 7fa4f52d6700 1 -- 172.21.2.101:0/2178760988 learned_addr learned my addr 172.21.2.101:0/2178760988 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:38.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.389 7fa4f52d6700 1 -- 172.21.2.101:0/2178760988 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4f00e5470 msgr2=0x7fa4f007dcb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:13:38.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.389 7fa4f52d6700 1 --2- 172.21.2.101:0/2178760988 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4f00e5470 0x7fa4f007dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:38.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.389 7fa4f52d6700 1 -- 172.21.2.101:0/2178760988 >> v2:172.21.2.108:3300/0 conn(0x7fa4f00e5070 msgr2=0x7fa4f01b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:38.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.389 7fa4f52d6700 1 --2- 172.21.2.101:0/2178760988 >> v2:172.21.2.108:3300/0 conn(0x7fa4f00e5070 0x7fa4f01b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:38.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.389 7fa4f52d6700 1 -- 172.21.2.101:0/2178760988 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f007e710 con 0x7fa4f01b3e40 2021-02-10T11:13:38.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.389 7fa4f52d6700 1 --2- 172.21.2.101:0/2178760988 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4f01b3e40 0x7fa4f007d690 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fa4e0006700 tx=0x7fa4e0009b80).ready entity=mon.0 client_cookie=868990fc48f73263 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:38.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.389 7fa4f4ad5700 1 -- 172.21.2.101:0/2178760988 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa4e000c0c0 con 0x7fa4f01b3e40 2021-02-10T11:13:38.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.389 7fa4f4ad5700 1 -- 172.21.2.101:0/2178760988 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e000c380 con 0x7fa4f01b3e40 2021-02-10T11:13:38.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.389 7fa4f4ad5700 1 -- 172.21.2.101:0/2178760988 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa4e000c880 con 0x7fa4f01b3e40 2021-02-10T11:13:38.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.390 7fa4f7cbb700 1 -- 172.21.2.101:0/2178760988 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4f01b3e40 msgr2=0x7fa4f007d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:38.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.390 7fa4f7cbb700 1 --2- 172.21.2.101:0/2178760988 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4f01b3e40 0x7fa4f007d690 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fa4e0006700 tx=0x7fa4e0009b80).stop 2021-02-10T11:13:38.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.390 7fa4f7cbb700 1 -- 172.21.2.101:0/2178760988 shutdown_connections 2021-02-10T11:13:38.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.390 7fa4f7cbb700 1 --2- 172.21.2.101:0/2178760988 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4f00e5470 0x7fa4f007dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:38.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.390 7fa4f7cbb700 1 --2- 172.21.2.101:0/2178760988 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4f01b3e40 0x7fa4f007d690 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:38.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.390 7fa4f7cbb700 1 --2- 172.21.2.101:0/2178760988 >> v2:172.21.2.108:3300/0 conn(0x7fa4f00e5070 0x7fa4f01b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:38.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.390 7fa4f7cbb700 1 -- 172.21.2.101:0/2178760988 >> 172.21.2.101:0/2178760988 conn(0x7fa4f00ce690 msgr2=0x7fa4f00e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:38.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.390 7fa4f7cbb700 1 -- 172.21.2.101:0/2178760988 shutdown_connections 2021-02-10T11:13:38.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.390 7fa4f7cbb700 1 -- 172.21.2.101:0/2178760988 wait complete. 2021-02-10T11:13:38.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.405 7fa4f7cbb700 1 Processor -- start 2021-02-10T11:13:38.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.405 7fa4f7cbb700 1 -- start start 2021-02-10T11:13:38.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.405 7fa4f7cbb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4f0192960 0x7fa4f0192d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:38.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.405 7fa4f7cbb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa4f00ad110 0x7fa4f00ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:38.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.405 7fa4f7cbb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4f00adb30 0x7fa4f008e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:38.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.405 7fa4f7cbb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f01a03e0 con 0x7fa4f00adb30 2021-02-10T11:13:38.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.405 7fa4f7cbb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f0193380 con 0x7fa4f00ad110 2021-02-10T11:13:38.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.405 7fa4f7cbb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f008ea80 con 0x7fa4f0192960 2021-02-10T11:13:38.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.406 7fa4f62d8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4f00adb30 0x7fa4f008e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:38.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.406 7fa4f62d8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4f00adb30 0x7fa4f008e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44492/0 (socket says 172.21.2.101:44492) 2021-02-10T11:13:38.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.406 7fa4f62d8700 1 -- 172.21.2.101:0/4272695942 learned_addr learned my addr 172.21.2.101:0/4272695942 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:38.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.406 7fa4f5ad7700 1 --2- 172.21.2.101:0/4272695942 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4f0192960 0x7fa4f0192d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:38.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.406 7fa4f52d6700 1 --2- 172.21.2.101:0/4272695942 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa4f00ad110 0x7fa4f00ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:38.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.406 7fa4f62d8700 1 -- 172.21.2.101:0/4272695942 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4f0192960 msgr2=0x7fa4f0192d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:38.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.406 7fa4f62d8700 1 --2- 172.21.2.101:0/4272695942 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4f0192960 0x7fa4f0192d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:38.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.406 7fa4f62d8700 1 -- 172.21.2.101:0/4272695942 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa4f00ad110 msgr2=0x7fa4f00ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:38.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.406 7fa4f62d8700 1 --2- 172.21.2.101:0/4272695942 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa4f00ad110 0x7fa4f00ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:38.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.406 7fa4f62d8700 1 -- 172.21.2.101:0/4272695942 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f008eca0 con 0x7fa4f00adb30 2021-02-10T11:13:38.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.406 7fa4f62d8700 1 --2- 172.21.2.101:0/4272695942 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4f00adb30 0x7fa4f008e460 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fa4e8009980 tx=0x7fa4e800bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:38.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.406 7fa4e67fc700 1 -- 172.21.2.101:0/4272695942 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa4e8005870 con 0x7fa4f00adb30 2021-02-10T11:13:38.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.406 7fa4e67fc700 1 -- 172.21.2.101:0/4272695942 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e8005b30 con 0x7fa4f00adb30 2021-02-10T11:13:38.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.406 7fa4f7cbb700 1 -- 172.21.2.101:0/4272695942 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f008eca0 con 0x7fa4f00adb30 2021-02-10T11:13:38.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.406 7fa4e67fc700 1 -- 172.21.2.101:0/4272695942 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa4e8006030 con 0x7fa4f00adb30 2021-02-10T11:13:38.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.407 7fa4f7cbb700 1 -- 172.21.2.101:0/4272695942 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f008eca0 con 0x7fa4f00adb30 2021-02-10T11:13:38.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.407 7fa4e67fc700 1 -- 172.21.2.101:0/4272695942 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa4e8005d10 con 0x7fa4f00adb30 2021-02-10T11:13:38.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.407 7fa4e67fc700 1 --2- 172.21.2.101:0/4272695942 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa4d80186d0 0x7fa4d801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:38.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.407 7fa4e67fc700 1 -- 172.21.2.101:0/4272695942 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa4e8032ba0 con 0x7fa4f00adb30 2021-02-10T11:13:38.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.407 7fa4f5ad7700 1 --2- 172.21.2.101:0/4272695942 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa4d80186d0 0x7fa4d801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:38.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.408 7fa4f5ad7700 1 --2- 172.21.2.101:0/4272695942 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa4d80186d0 0x7fa4d801abd0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:38.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.438 7fa4f7cbb700 1 -- 172.21.2.101:0/4272695942 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f0107120 con 0x7fa4f00adb30 2021-02-10T11:13:38.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.441 7fa4e67fc700 1 -- 172.21.2.101:0/4272695942 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa4e800be10 con 0x7fa4f00adb30 2021-02-10T11:13:38.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.565 7fa4f7cbb700 1 -- 172.21.2.101:0/4272695942 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4f007ed80 con 0x7fa4f00adb30 2021-02-10T11:13:38.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.565 7fa4e67fc700 1 -- 172.21.2.101:0/4272695942 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+13993 (secure 0 0 0) 0x7fa4e8006f20 con 0x7fa4f00adb30 2021-02-10T11:13:38.570 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:13:38.570 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":20,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:13:15.149234","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:13:11.922272","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2021-02-10 11:13:13.141733","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:13:38.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.567 7fa4f7cbb700 1 -- 172.21.2.101:0/4272695942 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa4d80186d0 msgr2=0x7fa4d801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:38.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.567 7fa4f7cbb700 1 --2- 172.21.2.101:0/4272695942 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa4d80186d0 0x7fa4d801abd0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:38.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.567 7fa4f7cbb700 1 -- 172.21.2.101:0/4272695942 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4f00adb30 msgr2=0x7fa4f008e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:38.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.567 7fa4f7cbb700 1 --2- 172.21.2.101:0/4272695942 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4f00adb30 0x7fa4f008e460 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fa4e8009980 tx=0x7fa4e800bcd0).stop 2021-02-10T11:13:38.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.567 7fa4f7cbb700 1 -- 172.21.2.101:0/4272695942 shutdown_connections 2021-02-10T11:13:38.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.567 7fa4f7cbb700 1 --2- 172.21.2.101:0/4272695942 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4f00adb30 0x7fa4f008e460 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:38.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.567 7fa4f7cbb700 1 --2- 172.21.2.101:0/4272695942 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa4f00ad110 0x7fa4f00ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:38.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.567 7fa4f7cbb700 1 --2- 172.21.2.101:0/4272695942 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa4d80186d0 0x7fa4d801abd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:38.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.567 7fa4f7cbb700 1 --2- 172.21.2.101:0/4272695942 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4f0192960 0x7fa4f0192d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:38.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.567 7fa4f7cbb700 1 -- 172.21.2.101:0/4272695942 >> 172.21.2.101:0/4272695942 conn(0x7fa4f00891f0 msgr2=0x7fa4f01a0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:38.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.567 7fa4f7cbb700 1 -- 172.21.2.101:0/4272695942 shutdown_connections 2021-02-10T11:13:38.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.567 7fa4f7cbb700 1 -- 172.21.2.101:0/4272695942 wait complete. 2021-02-10T11:13:38.580 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:13:11.922272', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:13:13.141733', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:13:38.580 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:13:39.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.989 7f2c80b3c700 1 Processor -- start 2021-02-10T11:13:39.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.989 7f2c80b3c700 1 -- start start 2021-02-10T11:13:39.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.990 7f2c80b3c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2c7c0f05f0 0x7f2c7c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:39.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.990 7f2c80b3c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c7c1b0c70 con 0x7f2c7c1b0870 2021-02-10T11:13:39.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.990 7f2c80b3c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c7c0f0b50 con 0x7f2c7c1b0470 2021-02-10T11:13:39.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.990 7f2c80b3c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c7c0e5ea0 con 0x7f2c7c0f05f0 2021-02-10T11:13:39.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.990 7f2c7b05f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2c7c0f05f0 0x7f2c7c0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:39.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.990 7f2c7b05f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2c7c0f05f0 0x7f2c7c0f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44498/0 (socket says 172.21.2.101:44498) 2021-02-10T11:13:39.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.990 7f2c7b05f700 1 -- 172.21.2.101:0/1319821033 learned_addr learned my addr 172.21.2.101:0/1319821033 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:39.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.990 7f2c7985c700 1 -- 172.21.2.101:0/1319821033 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2741564483 0 0) 0x7f2c70000ff0 con 0x7f2c7c1b0870 2021-02-10T11:13:39.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.990 7f2c7b05f700 1 -- 172.21.2.101:0/1319821033 >> v1:172.21.2.108:6789/0 conn(0x7f2c7c1b0470 legacy=0x7f2c7c0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:39.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.990 7f2c7b05f700 1 -- 172.21.2.101:0/1319821033 >> v1:172.21.2.101:6789/0 conn(0x7f2c7c1b0870 legacy=0x7f2c7c0efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:39.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.990 7f2c7b05f700 1 -- 172.21.2.101:0/1319821033 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7c0e52e0 con 0x7f2c7c0f05f0 2021-02-10T11:13:39.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.991 7f2c7b05f700 1 --2- 172.21.2.101:0/1319821033 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c7c0f05f0 0x7f2c7c0f20e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f2c74006f00 tx=0x7f2c74009e90).ready entity=mon.0 client_cookie=c4a84931f61d18aa server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:39.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.991 7f2c7985c700 1 -- 172.21.2.101:0/1319821033 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2c7400b300 con 0x7f2c7c0f05f0 2021-02-10T11:13:39.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.991 7f2c7985c700 1 -- 172.21.2.101:0/1319821033 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7400b640 con 0x7f2c7c0f05f0 2021-02-10T11:13:39.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.991 7f2c7985c700 1 -- 172.21.2.101:0/1319821033 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2c7400ba90 con 0x7f2c7c0f05f0 2021-02-10T11:13:39.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.991 7f2c80b3c700 1 -- 172.21.2.101:0/1319821033 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c7c0f05f0 msgr2=0x7f2c7c0f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:39.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.991 7f2c80b3c700 1 --2- 172.21.2.101:0/1319821033 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c7c0f05f0 0x7f2c7c0f20e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f2c74006f00 tx=0x7f2c74009e90).stop 2021-02-10T11:13:39.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.992 7f2c80b3c700 1 -- 172.21.2.101:0/1319821033 shutdown_connections 2021-02-10T11:13:39.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.992 7f2c80b3c700 1 --2- 172.21.2.101:0/1319821033 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c7c0f05f0 0x7f2c7c0f20e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:39.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.992 7f2c80b3c700 1 -- 172.21.2.101:0/1319821033 >> 172.21.2.101:0/1319821033 conn(0x7f2c7c0ce3e0 msgr2=0x7f2c7c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:39.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.992 7f2c80b3c700 1 -- 172.21.2.101:0/1319821033 shutdown_connections 2021-02-10T11:13:39.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:38.992 7f2c80b3c700 1 -- 172.21.2.101:0/1319821033 wait complete. 2021-02-10T11:13:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.006 7f2c80b3c700 1 Processor -- start 2021-02-10T11:13:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.006 7f2c80b3c700 1 -- start start 2021-02-10T11:13:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.007 7f2c80b3c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2c7c0bdbf0 0x7f2c7c1a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.007 7f2c80b3c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c7c0bdff0 0x7f2c7c1a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.007 7f2c80b3c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c7c0abcc0 0x7f2c7c0ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.007 7f2c80b3c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c0be590 con 0x7f2c7c0bdff0 2021-02-10T11:13:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.007 7f2c80b3c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c1a8090 con 0x7f2c7c0bdbf0 2021-02-10T11:13:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.007 7f2c80b3c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c1a82b0 con 0x7f2c7c0abcc0 2021-02-10T11:13:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.007 7f2c7a05d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c7c0bdff0 0x7f2c7c1a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:39.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.007 7f2c7a05d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c7c0bdff0 0x7f2c7c1a7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44504/0 (socket says 172.21.2.101:44504) 2021-02-10T11:13:39.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.007 7f2c7a05d700 1 -- 172.21.2.101:0/2269626921 learned_addr learned my addr 172.21.2.101:0/2269626921 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:39.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.007 7f2c7a85e700 1 --2- 172.21.2.101:0/2269626921 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2c7c0bdbf0 0x7f2c7c1a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:39.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.007 7f2c7b05f700 1 --2- 172.21.2.101:0/2269626921 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c7c0abcc0 0x7f2c7c0ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:39.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.007 7f2c7b05f700 1 -- 172.21.2.101:0/2269626921 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2c7c0bdbf0 msgr2=0x7f2c7c1a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:39.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.007 7f2c7b05f700 1 --2- 172.21.2.101:0/2269626921 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2c7c0bdbf0 0x7f2c7c1a7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:39.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.007 7f2c7b05f700 1 -- 172.21.2.101:0/2269626921 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c7c0bdff0 msgr2=0x7f2c7c1a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:39.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.007 7f2c7b05f700 1 --2- 172.21.2.101:0/2269626921 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c7c0bdff0 0x7f2c7c1a7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:39.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.007 7f2c7b05f700 1 -- 172.21.2.101:0/2269626921 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7c0ac750 con 0x7f2c7c0abcc0 2021-02-10T11:13:39.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.007 7f2c7a05d700 1 --2- 172.21.2.101:0/2269626921 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c7c0bdff0 0x7f2c7c1a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:13:39.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.008 7f2c7b05f700 1 --2- 172.21.2.101:0/2269626921 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c7c0abcc0 0x7f2c7c0ac0c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f2c7400b870 tx=0x7f2c7400bc10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:39.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.008 7f2c6b7fe700 1 -- 172.21.2.101:0/2269626921 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2c74003030 con 0x7f2c7c0abcc0 2021-02-10T11:13:39.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.008 7f2c6b7fe700 1 -- 172.21.2.101:0/2269626921 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c74001270 con 0x7f2c7c0abcc0 2021-02-10T11:13:39.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.008 7f2c80b3c700 1 -- 172.21.2.101:0/2269626921 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7c0ac750 con 0x7f2c7c0abcc0 2021-02-10T11:13:39.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.008 7f2c80b3c700 1 -- 172.21.2.101:0/2269626921 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c7c0ac750 con 0x7f2c7c0abcc0 2021-02-10T11:13:39.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.008 7f2c6b7fe700 1 -- 172.21.2.101:0/2269626921 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2c74001680 con 0x7f2c7c0abcc0 2021-02-10T11:13:39.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.008 7f2c6b7fe700 1 -- 172.21.2.101:0/2269626921 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2c74006fd0 con 0x7f2c7c0abcc0 2021-02-10T11:13:39.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.008 7f2c6b7fe700 1 --2- 172.21.2.101:0/2269626921 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2c60018730 0x7f2c6001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:39.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.008 7f2c6b7fe700 1 -- 172.21.2.101:0/2269626921 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2c74032aa0 con 0x7f2c7c0abcc0 2021-02-10T11:13:39.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.008 7f2c7a85e700 1 --2- 172.21.2.101:0/2269626921 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2c60018730 0x7f2c6001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:39.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.009 7f2c7a85e700 1 --2- 172.21.2.101:0/2269626921 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2c60018730 0x7f2c6001ac10 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:39.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.040 7f2c80b3c700 1 -- 172.21.2.101:0/2269626921 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c7c0edcc0 con 0x7f2c7c0abcc0 2021-02-10T11:13:39.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.044 7f2c6b7fe700 1 -- 172.21.2.101:0/2269626921 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2c74006d00 con 0x7f2c7c0abcc0 2021-02-10T11:13:39.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.173 7f2c80b3c700 1 -- 172.21.2.101:0/2269626921 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f2c7c0edcc0 con 0x7f2c7c0abcc0 2021-02-10T11:13:39.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.174 7f2c6b7fe700 1 -- 172.21.2.101:0/2269626921 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2c74006fd0 con 0x7f2c7c0abcc0 2021-02-10T11:13:39.178 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:13:39.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.176 7f2c80b3c700 1 -- 172.21.2.101:0/2269626921 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2c60018730 msgr2=0x7f2c6001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:39.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.176 7f2c80b3c700 1 --2- 172.21.2.101:0/2269626921 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2c60018730 0x7f2c6001ac10 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:39.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.176 7f2c80b3c700 1 -- 172.21.2.101:0/2269626921 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c7c0abcc0 msgr2=0x7f2c7c0ac0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:39.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.176 7f2c80b3c700 1 --2- 172.21.2.101:0/2269626921 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c7c0abcc0 0x7f2c7c0ac0c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f2c7400b870 tx=0x7f2c7400bc10).stop 2021-02-10T11:13:39.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.176 7f2c80b3c700 1 -- 172.21.2.101:0/2269626921 shutdown_connections 2021-02-10T11:13:39.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.176 7f2c80b3c700 1 --2- 172.21.2.101:0/2269626921 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2c60018730 0x7f2c6001ac10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:39.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.176 7f2c80b3c700 1 --2- 172.21.2.101:0/2269626921 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c7c0abcc0 0x7f2c7c0ac0c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:39.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.176 7f2c80b3c700 1 --2- 172.21.2.101:0/2269626921 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c7c0bdff0 0x7f2c7c1a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:39.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.176 7f2c80b3c700 1 --2- 172.21.2.101:0/2269626921 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2c7c0bdbf0 0x7f2c7c1a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:39.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.176 7f2c80b3c700 1 -- 172.21.2.101:0/2269626921 >> 172.21.2.101:0/2269626921 conn(0x7f2c7c095a70 msgr2=0x7f2c7c0bf040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:39.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.176 7f2c80b3c700 1 -- 172.21.2.101:0/2269626921 shutdown_connections 2021-02-10T11:13:39.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.176 7f2c80b3c700 1 -- 172.21.2.101:0/2269626921 wait complete. 2021-02-10T11:13:39.189 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:13:39.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.597 7f8c33324700 1 Processor -- start 2021-02-10T11:13:39.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.598 7f8c33324700 1 -- start start 2021-02-10T11:13:39.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.598 7f8c33324700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c2c1b0470 0x7f8c2c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:39.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.598 7f8c33324700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8c2c0ef610 0x7f8c2c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:39.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.598 7f8c33324700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c2c0f0b50 con 0x7f8c2c0f0030 2021-02-10T11:13:39.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.598 7f8c33324700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c2c0e52e0 con 0x7f8c2c0ef610 2021-02-10T11:13:39.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.598 7f8c33324700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c0e5490 con 0x7f8c2c1b0470 2021-02-10T11:13:39.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.598 7f8c3093f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8c2c0ef610 0x7f8c2c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:39.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.598 7f8c3093f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8c2c0ef610 0x7f8c2c0efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44510/0 (socket says 172.21.2.101:44510) 2021-02-10T11:13:39.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.598 7f8c3093f700 1 -- 172.21.2.101:0/3096129155 learned_addr learned my addr 172.21.2.101:0/3096129155 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:39.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.598 7f8c31140700 1 --2- 172.21.2.101:0/3096129155 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c2c1b0470 0x7f8c2c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:39.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.599 7f8c3093f700 1 -- 172.21.2.101:0/3096129155 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c2c1b0470 msgr2=0x7f8c2c1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:39.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.599 7f8c3093f700 1 --2- 172.21.2.101:0/3096129155 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c2c1b0470 0x7f8c2c1b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:39.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.599 7f8c3093f700 1 -- 172.21.2.101:0/3096129155 >> v1:172.21.2.108:6789/0 conn(0x7f8c2c0f0030 legacy=0x7f8c2c0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:39.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.599 7f8c3093f700 1 -- 172.21.2.101:0/3096129155 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2c0f0430 con 0x7f8c2c0ef610 2021-02-10T11:13:39.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.599 7f8c3093f700 1 --2- 172.21.2.101:0/3096129155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c2c0ef610 0x7f8c2c0efa10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f8c28006700 tx=0x7f8c28009b00).ready entity=mon.0 client_cookie=24e6b655b9ff1e5d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:39.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.599 7f8c1ffff700 1 -- 172.21.2.101:0/3096129155 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c2800c0c0 con 0x7f8c2c0ef610 2021-02-10T11:13:39.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.599 7f8c1ffff700 1 -- 172.21.2.101:0/3096129155 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2800c340 con 0x7f8c2c0ef610 2021-02-10T11:13:39.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.599 7f8c1ffff700 1 -- 172.21.2.101:0/3096129155 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c2800c7a0 con 0x7f8c2c0ef610 2021-02-10T11:13:39.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.599 7f8c33324700 1 -- 172.21.2.101:0/3096129155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c2c0ef610 msgr2=0x7f8c2c0efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:39.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.599 7f8c33324700 1 --2- 172.21.2.101:0/3096129155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c2c0ef610 0x7f8c2c0efa10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f8c28006700 tx=0x7f8c28009b00).stop 2021-02-10T11:13:39.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.600 7f8c33324700 1 -- 172.21.2.101:0/3096129155 shutdown_connections 2021-02-10T11:13:39.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.600 7f8c33324700 1 --2- 172.21.2.101:0/3096129155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c2c0ef610 0x7f8c2c0efa10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:39.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.600 7f8c33324700 1 --2- 172.21.2.101:0/3096129155 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c2c1b0470 0x7f8c2c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:39.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.600 7f8c33324700 1 -- 172.21.2.101:0/3096129155 >> 172.21.2.101:0/3096129155 conn(0x7f8c2c0ce3e0 msgr2=0x7f8c2c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:39.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.600 7f8c33324700 1 -- 172.21.2.101:0/3096129155 shutdown_connections 2021-02-10T11:13:39.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.600 7f8c33324700 1 -- 172.21.2.101:0/3096129155 wait complete. 2021-02-10T11:13:39.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.614 7f8c33324700 1 Processor -- start 2021-02-10T11:13:39.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.614 7f8c33324700 1 -- start start 2021-02-10T11:13:39.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.614 7f8c33324700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c2c105d10 0x7f8c2c09f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:39.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.615 7f8c33324700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c2c0961f0 0x7f8c2c0965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:39.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.615 7f8c33324700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c2c106890 0x7f8c2c106c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:39.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.615 7f8c33324700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c106110 con 0x7f8c2c0961f0 2021-02-10T11:13:39.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.615 7f8c33324700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c106380 con 0x7f8c2c106890 2021-02-10T11:13:39.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.615 7f8c33324700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c0a6620 con 0x7f8c2c105d10 2021-02-10T11:13:39.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.615 7f8c3093f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c2c0961f0 0x7f8c2c0965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:39.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.615 7f8c3093f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c2c0961f0 0x7f8c2c0965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44516/0 (socket says 172.21.2.101:44516) 2021-02-10T11:13:39.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.615 7f8c3093f700 1 -- 172.21.2.101:0/354673570 learned_addr learned my addr 172.21.2.101:0/354673570 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:39.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.615 7f8c31941700 1 --2- 172.21.2.101:0/354673570 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c2c106890 0x7f8c2c106c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:39.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.615 7f8c31140700 1 --2- 172.21.2.101:0/354673570 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c2c105d10 0x7f8c2c09f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:39.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.615 7f8c3093f700 1 -- 172.21.2.101:0/354673570 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c2c105d10 msgr2=0x7f8c2c09f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:39.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.615 7f8c3093f700 1 --2- 172.21.2.101:0/354673570 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c2c105d10 0x7f8c2c09f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:39.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.615 7f8c3093f700 1 -- 172.21.2.101:0/354673570 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c2c106890 msgr2=0x7f8c2c106c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:39.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.615 7f8c3093f700 1 --2- 172.21.2.101:0/354673570 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c2c106890 0x7f8c2c106c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:39.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.615 7f8c3093f700 1 -- 172.21.2.101:0/354673570 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2c1072b0 con 0x7f8c2c0961f0 2021-02-10T11:13:39.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.615 7f8c3093f700 1 --2- 172.21.2.101:0/354673570 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c2c0961f0 0x7f8c2c0965f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f8c28006f20 tx=0x7f8c2800b9b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:39.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.615 7f8c1dffb700 1 -- 172.21.2.101:0/354673570 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c28009100 con 0x7f8c2c0961f0 2021-02-10T11:13:39.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.615 7f8c1dffb700 1 -- 172.21.2.101:0/354673570 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2800bda0 con 0x7f8c2c0961f0 2021-02-10T11:13:39.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.615 7f8c33324700 1 -- 172.21.2.101:0/354673570 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c2c1072b0 con 0x7f8c2c0961f0 2021-02-10T11:13:39.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.615 7f8c1dffb700 1 -- 172.21.2.101:0/354673570 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c280052e0 con 0x7f8c2c0961f0 2021-02-10T11:13:39.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.615 7f8c33324700 1 -- 172.21.2.101:0/354673570 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2c107840 con 0x7f8c2c0961f0 2021-02-10T11:13:39.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.616 7f8c1dffb700 1 -- 172.21.2.101:0/354673570 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8c28009390 con 0x7f8c2c0961f0 2021-02-10T11:13:39.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.616 7f8c1dffb700 1 --2- 172.21.2.101:0/354673570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c0c018730 0x7f8c0c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:39.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.616 7f8c1dffb700 1 -- 172.21.2.101:0/354673570 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8c28032640 con 0x7f8c2c0961f0 2021-02-10T11:13:39.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.616 7f8c31140700 1 --2- 172.21.2.101:0/354673570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c0c018730 0x7f8c0c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:39.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.616 7f8c31140700 1 --2- 172.21.2.101:0/354673570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c0c018730 0x7f8c0c01abc0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:39.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.648 7f8c33324700 1 -- 172.21.2.101:0/354673570 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c2c105ae0 con 0x7f8c2c0961f0 2021-02-10T11:13:39.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.650 7f8c1dffb700 1 -- 172.21.2.101:0/354673570 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8c28006470 con 0x7f8c2c0961f0 2021-02-10T11:13:39.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.781 7f8c33324700 1 -- 172.21.2.101:0/354673570 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f8c2c0f0b50 con 0x7f8c2c0961f0 2021-02-10T11:13:39.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.781 7f8c1dffb700 1 -- 172.21.2.101:0/354673570 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v20) v1 ==== 104+0+10 (secure 0 0 0) 0x7f8c2800bda0 con 0x7f8c2c0961f0 2021-02-10T11:13:39.786 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:13:39.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.783 7f8c33324700 1 -- 172.21.2.101:0/354673570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c0c018730 msgr2=0x7f8c0c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:39.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.783 7f8c33324700 1 --2- 172.21.2.101:0/354673570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c0c018730 0x7f8c0c01abc0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:39.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.783 7f8c33324700 1 -- 172.21.2.101:0/354673570 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c2c0961f0 msgr2=0x7f8c2c0965f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:39.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.783 7f8c33324700 1 --2- 172.21.2.101:0/354673570 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c2c0961f0 0x7f8c2c0965f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f8c28006f20 tx=0x7f8c2800b9b0).stop 2021-02-10T11:13:39.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.783 7f8c33324700 1 -- 172.21.2.101:0/354673570 shutdown_connections 2021-02-10T11:13:39.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.783 7f8c33324700 1 --2- 172.21.2.101:0/354673570 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c2c106890 0x7f8c2c106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:39.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.783 7f8c33324700 1 --2- 172.21.2.101:0/354673570 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c2c0961f0 0x7f8c2c0965f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:39.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.783 7f8c33324700 1 --2- 172.21.2.101:0/354673570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c0c018730 0x7f8c0c01abc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:39.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.783 7f8c33324700 1 --2- 172.21.2.101:0/354673570 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c2c105d10 0x7f8c2c09f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:39.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.783 7f8c33324700 1 -- 172.21.2.101:0/354673570 >> 172.21.2.101:0/354673570 conn(0x7f8c2c095a70 msgr2=0x7f8c2c0bf050 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:39.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.784 7f8c33324700 1 -- 172.21.2.101:0/354673570 shutdown_connections 2021-02-10T11:13:39.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:39.784 7f8c33324700 1 -- 172.21.2.101:0/354673570 wait complete. 2021-02-10T11:13:39.796 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:13:40.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.205 7f9477180700 1 Processor -- start 2021-02-10T11:13:40.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.206 7f9477180700 1 -- start start 2021-02-10T11:13:40.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.206 7f9477180700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94701b0470 0x7f94701b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:40.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.206 7f9477180700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f94700f20e0 0x7f94700f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:40.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.206 7f9477180700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94700f0b50 con 0x7f94700ef610 2021-02-10T11:13:40.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.206 7f9477180700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94700f0820 con 0x7f94700f20e0 2021-02-10T11:13:40.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.206 7f9477180700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94700f24e0 con 0x7f94701b0470 2021-02-10T11:13:40.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.207 7f9474f9c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94701b0470 0x7f94701b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:40.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.207 7f947579d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f94700f20e0 0x7f94700f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:40.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.207 7f9474f9c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94701b0470 0x7f94701b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:37256/0 (socket says 172.21.2.101:37256) 2021-02-10T11:13:40.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.207 7f9474f9c700 1 -- 172.21.2.101:0/2040424106 learned_addr learned my addr 172.21.2.101:0/2040424106 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:40.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.207 7f946f7fe700 1 -- 172.21.2.101:0/2040424106 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3961377446 0 0) 0x7f9460000fe0 con 0x7f94700ef610 2021-02-10T11:13:40.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.207 7f9474f9c700 1 -- 172.21.2.101:0/2040424106 >> v2:172.21.2.108:3300/0 conn(0x7f94700f20e0 msgr2=0x7f94700f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:40.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.207 7f9474f9c700 1 --2- 172.21.2.101:0/2040424106 >> v2:172.21.2.108:3300/0 conn(0x7f94700f20e0 0x7f94700f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:40.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.207 7f9474f9c700 1 -- 172.21.2.101:0/2040424106 >> v1:172.21.2.108:6789/0 conn(0x7f94700ef610 legacy=0x7f94700efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:40.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.207 7f9474f9c700 1 -- 172.21.2.101:0/2040424106 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94700e52e0 con 0x7f94701b0470 2021-02-10T11:13:40.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.207 7f947579d700 1 --2- 172.21.2.101:0/2040424106 >> v2:172.21.2.108:3300/0 conn(0x7f94700f20e0 0x7f94700f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:13:40.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.207 7f9474f9c700 1 --2- 172.21.2.101:0/2040424106 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94701b0470 0x7f94701b0870 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f9458006700 tx=0x7f9458009b80).ready entity=mon.2 client_cookie=fe8cc9239fa2a2c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:40.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.207 7f946f7fe700 1 -- 172.21.2.101:0/2040424106 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9458006340 con 0x7f94701b0470 2021-02-10T11:13:40.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.208 7f946f7fe700 1 -- 172.21.2.101:0/2040424106 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945800c0c0 con 0x7f94701b0470 2021-02-10T11:13:40.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.208 7f946f7fe700 1 -- 172.21.2.101:0/2040424106 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f945800c5e0 con 0x7f94701b0470 2021-02-10T11:13:40.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.208 7f9477180700 1 -- 172.21.2.101:0/2040424106 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94701b0470 msgr2=0x7f94701b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:40.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.208 7f9477180700 1 --2- 172.21.2.101:0/2040424106 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94701b0470 0x7f94701b0870 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f9458006700 tx=0x7f9458009b80).stop 2021-02-10T11:13:40.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.208 7f9477180700 1 -- 172.21.2.101:0/2040424106 shutdown_connections 2021-02-10T11:13:40.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.208 7f9477180700 1 --2- 172.21.2.101:0/2040424106 >> v2:172.21.2.108:3300/0 conn(0x7f94700f20e0 0x7f94700f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:40.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.208 7f9477180700 1 --2- 172.21.2.101:0/2040424106 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94701b0470 0x7f94701b0870 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:40.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.208 7f9477180700 1 -- 172.21.2.101:0/2040424106 >> 172.21.2.101:0/2040424106 conn(0x7f94700ce3e0 msgr2=0x7f94700e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:40.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.208 7f9477180700 1 -- 172.21.2.101:0/2040424106 shutdown_connections 2021-02-10T11:13:40.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.208 7f9477180700 1 -- 172.21.2.101:0/2040424106 wait complete. 2021-02-10T11:13:40.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.223 7f9477180700 1 Processor -- start 2021-02-10T11:13:40.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.223 7f9477180700 1 -- start start 2021-02-10T11:13:40.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.223 7f9477180700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f947009b930 0x7f94700b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:40.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.223 7f9477180700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94700b8410 0x7f94700b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:40.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.223 7f9477180700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94700c1f80 0x7f94700c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:40.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.223 7f9477180700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94700c1a00 con 0x7f947009b930 2021-02-10T11:13:40.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.223 7f9477180700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94700b8e30 con 0x7f94700b8410 2021-02-10T11:13:40.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.223 7f9477180700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94700b9050 con 0x7f94700c1f80 2021-02-10T11:13:40.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.224 7f9474f9c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f947009b930 0x7f94700b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:40.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.224 7f9474f9c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f947009b930 0x7f94700b7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44528/0 (socket says 172.21.2.101:44528) 2021-02-10T11:13:40.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.224 7f9474f9c700 1 -- 172.21.2.101:0/1014878041 learned_addr learned my addr 172.21.2.101:0/1014878041 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:40.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.224 7f9474f9c700 1 -- 172.21.2.101:0/1014878041 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94700c1f80 msgr2=0x7f94700c2380 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:40.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.224 7f9474f9c700 1 --2- 172.21.2.101:0/1014878041 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94700c1f80 0x7f94700c2380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:40.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.224 7f9474f9c700 1 -- 172.21.2.101:0/1014878041 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94700b8410 msgr2=0x7f94700b8810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:40.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.224 7f9474f9c700 1 --2- 172.21.2.101:0/1014878041 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94700b8410 0x7f94700b8810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:40.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.224 7f9474f9c700 1 -- 172.21.2.101:0/1014878041 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94700c29a0 con 0x7f947009b930 2021-02-10T11:13:40.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.224 7f9474f9c700 1 --2- 172.21.2.101:0/1014878041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f947009b930 0x7f94700b7c20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f945800b600 tx=0x7f945800b9a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:40.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.224 7f946d7fa700 1 -- 172.21.2.101:0/1014878041 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9458002030 con 0x7f947009b930 2021-02-10T11:13:40.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.224 7f946d7fa700 1 -- 172.21.2.101:0/1014878041 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9458002370 con 0x7f947009b930 2021-02-10T11:13:40.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.224 7f946d7fa700 1 -- 172.21.2.101:0/1014878041 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f94580027d0 con 0x7f947009b930 2021-02-10T11:13:40.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.224 7f9477180700 1 -- 172.21.2.101:0/1014878041 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94700c29a0 con 0x7f947009b930 2021-02-10T11:13:40.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.224 7f9477180700 1 -- 172.21.2.101:0/1014878041 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94700c29a0 con 0x7f947009b930 2021-02-10T11:13:40.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.225 7f946d7fa700 1 -- 172.21.2.101:0/1014878041 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9458002020 con 0x7f947009b930 2021-02-10T11:13:40.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.225 7f946d7fa700 1 --2- 172.21.2.101:0/1014878041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94540186c0 0x7f945401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:40.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.225 7f946d7fa700 1 -- 172.21.2.101:0/1014878041 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9458032560 con 0x7f947009b930 2021-02-10T11:13:40.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.225 7f946ffff700 1 --2- 172.21.2.101:0/1014878041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94540186c0 0x7f945401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:40.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.225 7f946ffff700 1 --2- 172.21.2.101:0/1014878041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94540186c0 0x7f945401abc0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:40.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.256 7f9477180700 1 -- 172.21.2.101:0/1014878041 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9470105b30 con 0x7f947009b930 2021-02-10T11:13:40.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.258 7f946d7fa700 1 -- 172.21.2.101:0/1014878041 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9458030d90 con 0x7f947009b930 2021-02-10T11:13:40.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.385 7f9477180700 1 -- 172.21.2.101:0/1014878041 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f94700f0b50 con 0x7f947009b930 2021-02-10T11:13:40.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.386 7f946d7fa700 1 -- 172.21.2.101:0/1014878041 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v20) v1 ==== 100+0+10 (secure 0 0 0) 0x7f945800bd90 con 0x7f947009b930 2021-02-10T11:13:40.390 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:13:40.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.387 7f9477180700 1 -- 172.21.2.101:0/1014878041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94540186c0 msgr2=0x7f945401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:40.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.387 7f9477180700 1 --2- 172.21.2.101:0/1014878041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94540186c0 0x7f945401abc0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:40.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.387 7f9477180700 1 -- 172.21.2.101:0/1014878041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f947009b930 msgr2=0x7f94700b7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:40.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.387 7f9477180700 1 --2- 172.21.2.101:0/1014878041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f947009b930 0x7f94700b7c20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f945800b600 tx=0x7f945800b9a0).stop 2021-02-10T11:13:40.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.388 7f9477180700 1 -- 172.21.2.101:0/1014878041 shutdown_connections 2021-02-10T11:13:40.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.388 7f9477180700 1 --2- 172.21.2.101:0/1014878041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94540186c0 0x7f945401abc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:40.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.388 7f9477180700 1 --2- 172.21.2.101:0/1014878041 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94700c1f80 0x7f94700c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:40.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.388 7f9477180700 1 --2- 172.21.2.101:0/1014878041 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94700b8410 0x7f94700b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:40.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.388 7f9477180700 1 --2- 172.21.2.101:0/1014878041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f947009b930 0x7f94700b7c20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:40.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.388 7f9477180700 1 -- 172.21.2.101:0/1014878041 >> 172.21.2.101:0/1014878041 conn(0x7f9470094e10 msgr2=0x7f94700a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:40.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.388 7f9477180700 1 -- 172.21.2.101:0/1014878041 shutdown_connections 2021-02-10T11:13:40.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.388 7f9477180700 1 -- 172.21.2.101:0/1014878041 wait complete. 2021-02-10T11:13:40.400 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:13:40.400 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:13:40.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.806 7f530c2d3700 1 Processor -- start 2021-02-10T11:13:40.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.806 7f530c2d3700 1 -- start start 2021-02-10T11:13:40.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.807 7f530c2d3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53040e5070 0x7f53041b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:40.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.807 7f530c2d3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f53040e5470 0x7f530407d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:40.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.807 7f530c2d3700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530407ed80 con 0x7f53041b3e40 2021-02-10T11:13:40.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.807 7f530c2d3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5304080310 con 0x7f53040e5470 2021-02-10T11:13:40.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.807 7f530c2d3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5304080500 con 0x7f53040e5070 2021-02-10T11:13:40.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.807 7f53098ee700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f53040e5470 0x7f530407d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:40.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.807 7f53098ee700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f53040e5470 0x7f530407d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44536/0 (socket says 172.21.2.101:44536) 2021-02-10T11:13:40.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.807 7f53098ee700 1 -- 172.21.2.101:0/2798851754 learned_addr learned my addr 172.21.2.101:0/2798851754 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:40.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.807 7f53098ee700 1 -- 172.21.2.101:0/2798851754 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53040e5070 msgr2=0x7f53041b3820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:13:40.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.807 7f53098ee700 1 --2- 172.21.2.101:0/2798851754 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53040e5070 0x7f53041b3820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:40.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.807 7f53098ee700 1 -- 172.21.2.101:0/2798851754 >> v1:172.21.2.108:6789/0 conn(0x7f53041b3e40 legacy=0x7f530407dcb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:13:40.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.807 7f53098ee700 1 -- 172.21.2.101:0/2798851754 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53040806f0 con 0x7f53040e5470 2021-02-10T11:13:40.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.807 7f53098ee700 1 --2- 172.21.2.101:0/2798851754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53040e5470 0x7f530407d690 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f5300006700 tx=0x7f5300009b00).ready entity=mon.0 client_cookie=b3a31e5065acba39 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:40.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.807 7f53090ed700 1 -- 172.21.2.101:0/2798851754 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f530000c0c0 con 0x7f53040e5470 2021-02-10T11:13:40.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.807 7f53090ed700 1 -- 172.21.2.101:0/2798851754 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530000c340 con 0x7f53040e5470 2021-02-10T11:13:40.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.807 7f53090ed700 1 -- 172.21.2.101:0/2798851754 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f530000c820 con 0x7f53040e5470 2021-02-10T11:13:40.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.807 7f530c2d3700 1 -- 172.21.2.101:0/2798851754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53040e5470 msgr2=0x7f530407d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:40.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.807 7f530c2d3700 1 --2- 172.21.2.101:0/2798851754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53040e5470 0x7f530407d690 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f5300006700 tx=0x7f5300009b00).stop 2021-02-10T11:13:40.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.808 7f530c2d3700 1 -- 172.21.2.101:0/2798851754 shutdown_connections 2021-02-10T11:13:40.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.808 7f530c2d3700 1 --2- 172.21.2.101:0/2798851754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53040e5470 0x7f530407d690 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:40.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.808 7f530c2d3700 1 --2- 172.21.2.101:0/2798851754 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53040e5070 0x7f53041b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:40.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.808 7f530c2d3700 1 -- 172.21.2.101:0/2798851754 >> 172.21.2.101:0/2798851754 conn(0x7f53040ce690 msgr2=0x7f53040e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:40.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.808 7f530c2d3700 1 -- 172.21.2.101:0/2798851754 shutdown_connections 2021-02-10T11:13:40.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.808 7f530c2d3700 1 -- 172.21.2.101:0/2798851754 wait complete. 2021-02-10T11:13:40.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.823 7f530c2d3700 1 Processor -- start 2021-02-10T11:13:40.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.823 7f530c2d3700 1 -- start start 2021-02-10T11:13:40.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.823 7f530c2d3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53041a2a90 0x7f53041aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:40.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.823 7f530c2d3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53041aa700 0x7f53041a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:40.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.823 7f530c2d3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53040a55e0 0x7f53041a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:40.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.823 7f530c2d3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53041a34a0 con 0x7f53040a55e0 2021-02-10T11:13:40.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.823 7f530c2d3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53041a2e90 con 0x7f53041a2a90 2021-02-10T11:13:40.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.823 7f530c2d3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53041a30b0 con 0x7f53041aa700 2021-02-10T11:13:40.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.823 7f530a8f0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53040a55e0 0x7f53041a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:40.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.823 7f530a8f0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53040a55e0 0x7f53041a4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44540/0 (socket says 172.21.2.101:44540) 2021-02-10T11:13:40.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.823 7f530a8f0700 1 -- 172.21.2.101:0/2197356919 learned_addr learned my addr 172.21.2.101:0/2197356919 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:40.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.824 7f530a8f0700 1 -- 172.21.2.101:0/2197356919 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53041aa700 msgr2=0x7f53041a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:40.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.824 7f530a8f0700 1 --2- 172.21.2.101:0/2197356919 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53041aa700 0x7f53041a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:40.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.824 7f530a8f0700 1 -- 172.21.2.101:0/2197356919 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53041a2a90 msgr2=0x7f53041aa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:40.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.824 7f530a8f0700 1 --2- 172.21.2.101:0/2197356919 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53041a2a90 0x7f53041aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:40.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.824 7f530a8f0700 1 -- 172.21.2.101:0/2197356919 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53041a4660 con 0x7f53040a55e0 2021-02-10T11:13:40.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.824 7f530a8f0700 1 --2- 172.21.2.101:0/2197356919 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53040a55e0 0x7f53041a4040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f52fc009940 tx=0x7f52fc00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:40.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.824 7f52f6ffd700 1 -- 172.21.2.101:0/2197356919 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f52fc003030 con 0x7f53040a55e0 2021-02-10T11:13:40.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.824 7f52f6ffd700 1 -- 172.21.2.101:0/2197356919 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc001270 con 0x7f53040a55e0 2021-02-10T11:13:40.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.824 7f52f6ffd700 1 -- 172.21.2.101:0/2197356919 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f52fc001790 con 0x7f53040a55e0 2021-02-10T11:13:40.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.824 7f530c2d3700 1 -- 172.21.2.101:0/2197356919 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53041a4660 con 0x7f53040a55e0 2021-02-10T11:13:40.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.824 7f530c2d3700 1 -- 172.21.2.101:0/2197356919 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53041a4bf0 con 0x7f53040a55e0 2021-02-10T11:13:40.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.825 7f52f6ffd700 1 -- 172.21.2.101:0/2197356919 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f52fc001450 con 0x7f53040a55e0 2021-02-10T11:13:40.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.825 7f52f6ffd700 1 --2- 172.21.2.101:0/2197356919 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52ec0186c0 0x7f52ec01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:40.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.825 7f52f6ffd700 1 -- 172.21.2.101:0/2197356919 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f52fc032980 con 0x7f53040a55e0 2021-02-10T11:13:40.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.825 7f530a0ef700 1 --2- 172.21.2.101:0/2197356919 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52ec0186c0 0x7f52ec01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:40.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.825 7f530a0ef700 1 --2- 172.21.2.101:0/2197356919 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52ec0186c0 0x7f52ec01abc0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:40.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.856 7f530c2d3700 1 -- 172.21.2.101:0/2197356919 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5304083da0 con 0x7f53040a55e0 2021-02-10T11:13:40.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.859 7f52f6ffd700 1 -- 172.21.2.101:0/2197356919 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f52fc000f40 con 0x7f53040a55e0 2021-02-10T11:13:40.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.987 7f530c2d3700 1 -- 172.21.2.101:0/2197356919 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f530407ed80 con 0x7f53040a55e0 2021-02-10T11:13:40.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.988 7f52f6ffd700 1 -- 172.21.2.101:0/2197356919 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2097 (secure 0 0 0) 0x7f52fc01f720 con 0x7f53040a55e0 2021-02-10T11:13:40.993 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:13:40.993 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/3184957761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3184957761},{"type":"v1","addr":"172.21.2.108:6833","nonce":3184957761}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:13:14.147021","modified":"2021-02-10 11:13:23.547179","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":5,"state":"up:active","state_seq":3,"addr":"172.21.2.101:6835/3530104431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3530104431},{"type":"v1","addr":"172.21.2.101:6835","nonce":3530104431}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-02-10T11:13:40.993 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 6 2021-02-10T11:13:40.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.990 7f530c2d3700 1 -- 172.21.2.101:0/2197356919 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52ec0186c0 msgr2=0x7f52ec01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:40.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.990 7f530c2d3700 1 --2- 172.21.2.101:0/2197356919 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52ec0186c0 0x7f52ec01abc0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:40.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.990 7f530c2d3700 1 -- 172.21.2.101:0/2197356919 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53040a55e0 msgr2=0x7f53041a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:40.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.990 7f530c2d3700 1 --2- 172.21.2.101:0/2197356919 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53040a55e0 0x7f53041a4040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f52fc009940 tx=0x7f52fc00bcf0).stop 2021-02-10T11:13:40.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.990 7f530c2d3700 1 -- 172.21.2.101:0/2197356919 shutdown_connections 2021-02-10T11:13:40.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.990 7f530c2d3700 1 --2- 172.21.2.101:0/2197356919 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52ec0186c0 0x7f52ec01abc0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:40.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.990 7f530c2d3700 1 --2- 172.21.2.101:0/2197356919 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53041aa700 0x7f53041a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:40.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.990 7f530c2d3700 1 --2- 172.21.2.101:0/2197356919 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53040a55e0 0x7f53041a4040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:40.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.990 7f530c2d3700 1 --2- 172.21.2.101:0/2197356919 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53041a2a90 0x7f53041aa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:40.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.990 7f530c2d3700 1 -- 172.21.2.101:0/2197356919 >> 172.21.2.101:0/2197356919 conn(0x7f530408fe10 msgr2=0x7f530419c020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:40.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.990 7f530c2d3700 1 -- 172.21.2.101:0/2197356919 shutdown_connections 2021-02-10T11:13:40.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:40.990 7f530c2d3700 1 -- 172.21.2.101:0/2197356919 wait complete. 2021-02-10T11:13:41.204 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:13:14.147021', 'modified': '2021-02-10 11:13:23.547179', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4299': {'gid': 4299, 'name': 'a', 'rank': 0, 'incarnation': 5, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.101:6835/3530104431', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.101:6834', 'nonce': 3530104431}, {'type': 'v1', 'addr': '172.21.2.101:6835', 'nonce': 3530104431}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:13:41.204 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:13:41.204 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok status 2021-02-10T11:13:41.396 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:13:41.399 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:13:41.603 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:13:41.603 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:13:41.603 INFO:teuthology.orchestra.run.gibba001.stdout: "whoami": 0, 2021-02-10T11:13:41.603 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 4299, 2021-02-10T11:13:41.603 INFO:teuthology.orchestra.run.gibba001.stdout: "want_state": "up:active", 2021-02-10T11:13:41.603 INFO:teuthology.orchestra.run.gibba001.stdout: "state": "up:active", 2021-02-10T11:13:41.604 INFO:teuthology.orchestra.run.gibba001.stdout: "rank_uptime": 19.052188831999999, 2021-02-10T11:13:41.604 INFO:teuthology.orchestra.run.gibba001.stdout: "mdsmap_epoch": 6, 2021-02-10T11:13:41.604 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch": 20, 2021-02-10T11:13:41.604 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch_barrier": 20, 2021-02-10T11:13:41.604 INFO:teuthology.orchestra.run.gibba001.stdout: "uptime": 23.444120001000002 2021-02-10T11:13:41.604 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:13:41.607 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 4299, "want_state": "up:active", "state": "up:active", "rank_uptime": 19.052188831999999, "mdsmap_epoch": 6, "osdmap_epoch": 20, "osdmap_epoch_barrier": 20, "uptime": 23.444120001000002 } 2021-02-10T11:13:41.607 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:13:41.608 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-02-10T11:13:41.608 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-02-10T11:13:41.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:13:41.608 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:13:41.942 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:13:42.309 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:13:42.309 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:13:42.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:13:42.655 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:13:42.990 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-02-10T11:13:42.990 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:13:42.993 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:13:43.335 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:13:43.336 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:13:43.402 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:2021-02-10 11:13:43.398 7f3e057cbf80 -1 init, newargv = 0x55ad657d2aa0 newargc=9 2021-02-10T11:13:43.402 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[28943]: starting ceph client 2021-02-10T11:13:43.406 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[28943]: starting fuse 2021-02-10T11:13:43.678 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:13:43.679 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:13:43.679 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:13:43.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:13:43.680 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:13:44.014 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:13:44.014 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2021-02-10T11:13:44.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:13:44.015 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:13:44.015 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:13:44.015 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:13:44.015 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:13:44.015 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:13:44.015 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:13:44.015 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:13:44.015 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-02-10T11:13:44.015 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:13:44.015 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:13:44.015 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:13:44.016 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:13:44.016 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:13:44.016 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:13:44.016 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:13:44.016 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:13:44.016 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:13:44.016 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:13:44.016 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:13:44.016 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:13:44.016 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:13:44.016 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.0")) 2021-02-10T11:13:44.016 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:13:44.395 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.0.28943.asok 2021-02-10T11:13:44.398 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.28943.asok 2021-02-10T11:13:44.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:13:44.398 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.28943.asok status 2021-02-10T11:13:44.788 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:13:44.788 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:13:44.788 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:13:44.788 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:13:44.788 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "0", 2021-02-10T11:13:44.789 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:13:44.789 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-02-10T11:13:44.789 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "28943", 2021-02-10T11:13:44.789 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:13:44.789 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:44.789 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:13:44.789 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:13:44.789 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 4554, 2021-02-10T11:13:44.789 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:13:44.789 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:13:44.789 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:13:44.789 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 4554 2021-02-10T11:13:44.789 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:44.789 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:13:44.790 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:13:44.790 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:13:44.790 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 554664949 2021-02-10T11:13:44.790 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:44.790 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:44.790 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:13:44.790 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:13:44.790 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:13:44.790 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 554664949 2021-02-10T11:13:44.790 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:44.790 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.4554 172.21.2.101:0/554664949", 2021-02-10T11:13:44.790 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/554664949", 2021-02-10T11:13:44.791 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:13:44.791 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 6, 2021-02-10T11:13:44.791 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 20, 2021-02-10T11:13:44.791 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:13:44.791 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:13:44.791 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:13:44.793 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:13:45.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.198 7f969ec47700 1 Processor -- start 2021-02-10T11:13:45.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.198 7f969ec47700 1 -- start start 2021-02-10T11:13:45.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.198 7f969ec47700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f96980e5470 0x7f969807e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:45.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.198 7f969ec47700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f969807f300 0x7f969807f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:45.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.198 7f969ec47700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969807ed80 con 0x7f96980e5070 2021-02-10T11:13:45.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.198 7f969ec47700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96981b4010 con 0x7f969807f300 2021-02-10T11:13:45.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.198 7f969ec47700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96980d1790 con 0x7f96980e5470 2021-02-10T11:13:45.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.198 7f969ca63700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f96980e5070 0x7f96981b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45054/0 (socket says 172.21.2.101:45054) 2021-02-10T11:13:45.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.198 7f969ca63700 1 -- 172.21.2.101:0/526548692 learned_addr learned my addr 172.21.2.101:0/526548692 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:45.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.198 7f969d264700 1 --2- 172.21.2.101:0/526548692 >> v2:172.21.2.101:3300/0 conn(0x7f969807f300 0x7f969807f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:45.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.199 7f969d264700 1 -- 172.21.2.101:0/526548692 >> v2:172.21.2.108:3300/0 conn(0x7f96980e5470 msgr2=0x7f969807e6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:13:45.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.199 7f969d264700 1 --2- 172.21.2.101:0/526548692 >> v2:172.21.2.108:3300/0 conn(0x7f96980e5470 0x7f969807e6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:45.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.199 7f969d264700 1 -- 172.21.2.101:0/526548692 >> v1:172.21.2.101:6789/0 conn(0x7f96980e5070 legacy=0x7f96981b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:45.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.199 7f969d264700 1 -- 172.21.2.101:0/526548692 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96980d19b0 con 0x7f969807f300 2021-02-10T11:13:45.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.199 7f969d264700 1 --2- 172.21.2.101:0/526548692 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f969807f300 0x7f969807f700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f9690006e30 tx=0x7f9690009e60).ready entity=mon.0 client_cookie=2f4418bb02578ad6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:45.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.199 7f96977fe700 1 -- 172.21.2.101:0/526548692 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9690006a50 con 0x7f969807f300 2021-02-10T11:13:45.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.199 7f96977fe700 1 -- 172.21.2.101:0/526548692 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969000b300 con 0x7f969807f300 2021-02-10T11:13:45.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.199 7f96977fe700 1 -- 172.21.2.101:0/526548692 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f969000b820 con 0x7f969807f300 2021-02-10T11:13:45.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.199 7f969ec47700 1 -- 172.21.2.101:0/526548692 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f969807f300 msgr2=0x7f969807f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:45.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.199 7f969ec47700 1 --2- 172.21.2.101:0/526548692 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f969807f300 0x7f969807f700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f9690006e30 tx=0x7f9690009e60).stop 2021-02-10T11:13:45.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.200 7f969ec47700 1 -- 172.21.2.101:0/526548692 shutdown_connections 2021-02-10T11:13:45.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.200 7f969ec47700 1 --2- 172.21.2.101:0/526548692 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f969807f300 0x7f969807f700 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:45.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.200 7f969ec47700 1 --2- 172.21.2.101:0/526548692 >> v2:172.21.2.108:3300/0 conn(0x7f96980e5470 0x7f969807e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:45.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.200 7f969ec47700 1 -- 172.21.2.101:0/526548692 >> 172.21.2.101:0/526548692 conn(0x7f96980ce690 msgr2=0x7f96980e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:45.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.200 7f969ec47700 1 -- 172.21.2.101:0/526548692 shutdown_connections 2021-02-10T11:13:45.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.200 7f969ec47700 1 -- 172.21.2.101:0/526548692 wait complete. 2021-02-10T11:13:45.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.215 7f969ec47700 1 Processor -- start 2021-02-10T11:13:45.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.215 7f969ec47700 1 -- start start 2021-02-10T11:13:45.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.215 7f969ec47700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96981a2a90 0x7f96981aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:45.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.215 7f969ec47700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96981aa700 0x7f96981a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:45.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.215 7f969ec47700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96980a55e0 0x7f96981a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:45.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.215 7f969ec47700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96981a34a0 con 0x7f96981aa700 2021-02-10T11:13:45.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.215 7f969ec47700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96981a2e90 con 0x7f96981a2a90 2021-02-10T11:13:45.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.215 7f969ec47700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96981a30b0 con 0x7f96980a55e0 2021-02-10T11:13:45.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.215 7f9697fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96981aa700 0x7f96981a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:45.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.215 7f9697fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96981aa700 0x7f96981a3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44568/0 (socket says 172.21.2.101:44568) 2021-02-10T11:13:45.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.215 7f9697fff700 1 -- 172.21.2.101:0/2822712629 learned_addr learned my addr 172.21.2.101:0/2822712629 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:45.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.216 7f9697fff700 1 -- 172.21.2.101:0/2822712629 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96980a55e0 msgr2=0x7f96981a4040 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:45.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.216 7f9697fff700 1 --2- 172.21.2.101:0/2822712629 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96980a55e0 0x7f96981a4040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:45.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.216 7f9697fff700 1 -- 172.21.2.101:0/2822712629 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96981a2a90 msgr2=0x7f96981aa0e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:45.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.216 7f9697fff700 1 --2- 172.21.2.101:0/2822712629 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96981a2a90 0x7f96981aa0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:45.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.216 7f9697fff700 1 -- 172.21.2.101:0/2822712629 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96981a4660 con 0x7f96981aa700 2021-02-10T11:13:45.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.216 7f9697fff700 1 --2- 172.21.2.101:0/2822712629 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96981aa700 0x7f96981a3a20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f968800b5c0 tx=0x7f968800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:45.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.216 7f96957fa700 1 -- 172.21.2.101:0/2822712629 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f968800bd50 con 0x7f96981aa700 2021-02-10T11:13:45.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.216 7f96957fa700 1 -- 172.21.2.101:0/2822712629 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9688006030 con 0x7f96981aa700 2021-02-10T11:13:45.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.216 7f96957fa700 1 -- 172.21.2.101:0/2822712629 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96880054d0 con 0x7f96981aa700 2021-02-10T11:13:45.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.216 7f969ec47700 1 -- 172.21.2.101:0/2822712629 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96981a4660 con 0x7f96981aa700 2021-02-10T11:13:45.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.216 7f969ec47700 1 -- 172.21.2.101:0/2822712629 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96981a4660 con 0x7f96981aa700 2021-02-10T11:13:45.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.217 7f96957fa700 1 -- 172.21.2.101:0/2822712629 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9688006210 con 0x7f96981aa700 2021-02-10T11:13:45.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.217 7f96957fa700 1 --2- 172.21.2.101:0/2822712629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f967c0186c0 0x7f967c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:45.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.217 7f96957fa700 1 -- 172.21.2.101:0/2822712629 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f96880327d0 con 0x7f96981aa700 2021-02-10T11:13:45.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.217 7f969ca63700 1 --2- 172.21.2.101:0/2822712629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f967c0186c0 0x7f967c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:45.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.217 7f969ca63700 1 --2- 172.21.2.101:0/2822712629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f967c0186c0 0x7f967c01abc0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:45.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.248 7f969ec47700 1 -- 172.21.2.101:0/2822712629 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9698083da0 con 0x7f96981aa700 2021-02-10T11:13:45.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.250 7f96957fa700 1 -- 172.21.2.101:0/2822712629 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9688006b80 con 0x7f96981aa700 2021-02-10T11:13:45.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.376 7f969ec47700 1 -- 172.21.2.101:0/2822712629 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f969807ed80 con 0x7f96981aa700 2021-02-10T11:13:45.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.377 7f96957fa700 1 -- 172.21.2.101:0/2822712629 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+13993 (secure 0 0 0) 0x7f9688006030 con 0x7f96981aa700 2021-02-10T11:13:45.381 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:13:45.381 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":20,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:13:15.149234","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:13:11.922272","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2021-02-10 11:13:13.141733","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:13:45.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.378 7f969ec47700 1 -- 172.21.2.101:0/2822712629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f967c0186c0 msgr2=0x7f967c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:45.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.378 7f969ec47700 1 --2- 172.21.2.101:0/2822712629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f967c0186c0 0x7f967c01abc0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:45.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.378 7f969ec47700 1 -- 172.21.2.101:0/2822712629 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96981aa700 msgr2=0x7f96981a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:45.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.378 7f969ec47700 1 --2- 172.21.2.101:0/2822712629 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96981aa700 0x7f96981a3a20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f968800b5c0 tx=0x7f968800b960).stop 2021-02-10T11:13:45.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.378 7f969ec47700 1 -- 172.21.2.101:0/2822712629 shutdown_connections 2021-02-10T11:13:45.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.378 7f969ec47700 1 --2- 172.21.2.101:0/2822712629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f967c0186c0 0x7f967c01abc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:45.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.378 7f969ec47700 1 --2- 172.21.2.101:0/2822712629 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96980a55e0 0x7f96981a4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:45.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.379 7f969ec47700 1 --2- 172.21.2.101:0/2822712629 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96981aa700 0x7f96981a3a20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:45.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.379 7f969ec47700 1 --2- 172.21.2.101:0/2822712629 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96981a2a90 0x7f96981aa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:45.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.379 7f969ec47700 1 -- 172.21.2.101:0/2822712629 >> 172.21.2.101:0/2822712629 conn(0x7f969808fe10 msgr2=0x7f969819c020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:45.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.379 7f969ec47700 1 -- 172.21.2.101:0/2822712629 shutdown_connections 2021-02-10T11:13:45.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.379 7f969ec47700 1 -- 172.21.2.101:0/2822712629 wait complete. 2021-02-10T11:13:45.391 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:13:11.922272', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:13:13.141733', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:13:45.391 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:13:45.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.801 7f6ea2125700 1 Processor -- start 2021-02-10T11:13:45.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.801 7f6ea2125700 1 -- start start 2021-02-10T11:13:45.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.801 7f6ea2125700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6e9c1b0470 0x7f6e9c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:45.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.801 7f6ea2125700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6e9c0ef610 0x7f6e9c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:45.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.801 7f6ea2125700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e9c0f0b50 con 0x7f6e9c0f0030 2021-02-10T11:13:45.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.801 7f6ea2125700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e9c0e52e0 con 0x7f6e9c1b0470 2021-02-10T11:13:45.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.801 7f6ea2125700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e9c0e5490 con 0x7f6e9c0ef610 2021-02-10T11:13:45.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.801 7f6e9b7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6e9c1b0470 0x7f6e9c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:45.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.801 7f6e9b7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6e9c1b0470 0x7f6e9c1b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44574/0 (socket says 172.21.2.101:44574) 2021-02-10T11:13:45.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.801 7f6e9b7fe700 1 -- 172.21.2.101:0/1216263563 learned_addr learned my addr 172.21.2.101:0/1216263563 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:45.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.802 7f6e9affd700 1 --2- 172.21.2.101:0/1216263563 >> v2:172.21.2.108:3300/0 conn(0x7f6e9c0ef610 0x7f6e9c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:45.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.802 7f6e9b7fe700 1 -- 172.21.2.101:0/1216263563 >> v2:172.21.2.108:3300/0 conn(0x7f6e9c0ef610 msgr2=0x7f6e9c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:45.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.802 7f6e9b7fe700 1 --2- 172.21.2.101:0/1216263563 >> v2:172.21.2.108:3300/0 conn(0x7f6e9c0ef610 0x7f6e9c0efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:45.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.802 7f6e9b7fe700 1 -- 172.21.2.101:0/1216263563 >> v1:172.21.2.108:6789/0 conn(0x7f6e9c0f0030 legacy=0x7f6e9c0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:45.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.802 7f6e9b7fe700 1 -- 172.21.2.101:0/1216263563 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c0f0430 con 0x7f6e9c1b0470 2021-02-10T11:13:45.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.802 7f6e9affd700 1 --2- 172.21.2.101:0/1216263563 >> v2:172.21.2.108:3300/0 conn(0x7f6e9c0ef610 0x7f6e9c0efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:13:45.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.802 7f6e9b7fe700 1 --2- 172.21.2.101:0/1216263563 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e9c1b0470 0x7f6e9c1b0870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f6e84006700 tx=0x7f6e84009b80).ready entity=mon.0 client_cookie=23f40504431b2755 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:45.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.802 7f6e9a7fc700 1 -- 172.21.2.101:0/1216263563 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e84006340 con 0x7f6e9c1b0470 2021-02-10T11:13:45.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.802 7f6e9a7fc700 1 -- 172.21.2.101:0/1216263563 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8400c0c0 con 0x7f6e9c1b0470 2021-02-10T11:13:45.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.802 7f6e9a7fc700 1 -- 172.21.2.101:0/1216263563 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e8400c5e0 con 0x7f6e9c1b0470 2021-02-10T11:13:45.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.802 7f6ea2125700 1 -- 172.21.2.101:0/1216263563 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e9c1b0470 msgr2=0x7f6e9c1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:45.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.802 7f6ea2125700 1 --2- 172.21.2.101:0/1216263563 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e9c1b0470 0x7f6e9c1b0870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f6e84006700 tx=0x7f6e84009b80).stop 2021-02-10T11:13:45.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.803 7f6ea2125700 1 -- 172.21.2.101:0/1216263563 shutdown_connections 2021-02-10T11:13:45.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.803 7f6ea2125700 1 --2- 172.21.2.101:0/1216263563 >> v2:172.21.2.108:3300/0 conn(0x7f6e9c0ef610 0x7f6e9c0efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:45.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.803 7f6ea2125700 1 --2- 172.21.2.101:0/1216263563 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e9c1b0470 0x7f6e9c1b0870 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:45.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.803 7f6ea2125700 1 -- 172.21.2.101:0/1216263563 >> 172.21.2.101:0/1216263563 conn(0x7f6e9c0ce3e0 msgr2=0x7f6e9c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:45.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.803 7f6ea2125700 1 -- 172.21.2.101:0/1216263563 shutdown_connections 2021-02-10T11:13:45.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.803 7f6ea2125700 1 -- 172.21.2.101:0/1216263563 wait complete. 2021-02-10T11:13:45.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.818 7f6ea2125700 1 Processor -- start 2021-02-10T11:13:45.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.818 7f6ea2125700 1 -- start start 2021-02-10T11:13:45.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.818 7f6ea2125700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e9c190920 0x7f6e9c08f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:45.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.818 7f6ea2125700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e9c190d20 0x7f6e9c08fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:45.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.818 7f6ea2125700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e9c193270 0x7f6e9c193670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:45.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.818 7f6ea2125700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c090470 con 0x7f6e9c190d20 2021-02-10T11:13:45.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.818 7f6ea2125700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c193c90 con 0x7f6e9c190920 2021-02-10T11:13:45.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.818 7f6ea2125700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c193e70 con 0x7f6e9c193270 2021-02-10T11:13:45.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.818 7f6e9affd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e9c190d20 0x7f6e9c08fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:45.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.819 7f6e9affd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e9c190d20 0x7f6e9c08fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44576/0 (socket says 172.21.2.101:44576) 2021-02-10T11:13:45.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.819 7f6e9affd700 1 -- 172.21.2.101:0/3996600563 learned_addr learned my addr 172.21.2.101:0/3996600563 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:45.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.819 7f6e9affd700 1 -- 172.21.2.101:0/3996600563 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e9c193270 msgr2=0x7f6e9c193670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:45.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.819 7f6e9affd700 1 --2- 172.21.2.101:0/3996600563 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e9c193270 0x7f6e9c193670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:45.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.819 7f6e9affd700 1 -- 172.21.2.101:0/3996600563 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e9c190920 msgr2=0x7f6e9c08f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:45.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.819 7f6e9affd700 1 --2- 172.21.2.101:0/3996600563 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e9c190920 0x7f6e9c08f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:45.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.819 7f6e9affd700 1 -- 172.21.2.101:0/3996600563 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c194090 con 0x7f6e9c190d20 2021-02-10T11:13:45.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.819 7f6e9affd700 1 --2- 172.21.2.101:0/3996600563 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e9c190d20 0x7f6e9c08fc20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f6e8c006f20 tx=0x7f6e8c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:45.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.819 7f6e93fff700 1 -- 172.21.2.101:0/3996600563 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e8c006bb0 con 0x7f6e9c190d20 2021-02-10T11:13:45.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.819 7f6e93fff700 1 -- 172.21.2.101:0/3996600563 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8c00bd80 con 0x7f6e9c190d20 2021-02-10T11:13:45.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.819 7f6e93fff700 1 -- 172.21.2.101:0/3996600563 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e8c0022b0 con 0x7f6e9c190d20 2021-02-10T11:13:45.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.819 7f6ea2125700 1 -- 172.21.2.101:0/3996600563 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e9c194090 con 0x7f6e9c190d20 2021-02-10T11:13:45.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.819 7f6ea2125700 1 -- 172.21.2.101:0/3996600563 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9c194090 con 0x7f6e9c190d20 2021-02-10T11:13:45.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.820 7f6e93fff700 1 -- 172.21.2.101:0/3996600563 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6e8c000c90 con 0x7f6e9c190d20 2021-02-10T11:13:45.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.820 7f6e93fff700 1 --2- 172.21.2.101:0/3996600563 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e800186c0 0x7f6e8001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:45.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.820 7f6e93fff700 1 -- 172.21.2.101:0/3996600563 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6e8c032580 con 0x7f6e9c190d20 2021-02-10T11:13:45.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.820 7f6e9b7fe700 1 --2- 172.21.2.101:0/3996600563 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e800186c0 0x7f6e8001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:45.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.820 7f6e9b7fe700 1 --2- 172.21.2.101:0/3996600563 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e800186c0 0x7f6e8001abc0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:45.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.851 7f6ea2125700 1 -- 172.21.2.101:0/3996600563 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e9c105af0 con 0x7f6e9c190d20 2021-02-10T11:13:45.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.854 7f6e93fff700 1 -- 172.21.2.101:0/3996600563 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6e8c006bb0 con 0x7f6e9c190d20 2021-02-10T11:13:45.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.982 7f6ea2125700 1 -- 172.21.2.101:0/3996600563 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f6e9c0f0b50 con 0x7f6e9c190d20 2021-02-10T11:13:45.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.982 7f6e93fff700 1 -- 172.21.2.101:0/3996600563 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6e8c000c90 con 0x7f6e9c190d20 2021-02-10T11:13:45.987 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:13:45.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.984 7f6ea2125700 1 -- 172.21.2.101:0/3996600563 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e800186c0 msgr2=0x7f6e8001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:45.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.984 7f6ea2125700 1 --2- 172.21.2.101:0/3996600563 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e800186c0 0x7f6e8001abc0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:45.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.984 7f6ea2125700 1 -- 172.21.2.101:0/3996600563 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e9c190d20 msgr2=0x7f6e9c08fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:45.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.984 7f6ea2125700 1 --2- 172.21.2.101:0/3996600563 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e9c190d20 0x7f6e9c08fc20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f6e8c006f20 tx=0x7f6e8c00b990).stop 2021-02-10T11:13:45.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.984 7f6ea2125700 1 -- 172.21.2.101:0/3996600563 shutdown_connections 2021-02-10T11:13:45.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.984 7f6ea2125700 1 --2- 172.21.2.101:0/3996600563 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e800186c0 0x7f6e8001abc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:45.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.984 7f6ea2125700 1 --2- 172.21.2.101:0/3996600563 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e9c193270 0x7f6e9c193670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:45.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.984 7f6ea2125700 1 --2- 172.21.2.101:0/3996600563 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e9c190d20 0x7f6e9c08fc20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:45.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.984 7f6ea2125700 1 --2- 172.21.2.101:0/3996600563 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e9c190920 0x7f6e9c08f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:45.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.984 7f6ea2125700 1 -- 172.21.2.101:0/3996600563 >> 172.21.2.101:0/3996600563 conn(0x7f6e9c0bf4e0 msgr2=0x7f6e9c0be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:45.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.984 7f6ea2125700 1 -- 172.21.2.101:0/3996600563 shutdown_connections 2021-02-10T11:13:45.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:45.984 7f6ea2125700 1 -- 172.21.2.101:0/3996600563 wait complete. 2021-02-10T11:13:45.996 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:13:46.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.405 7f2fc6e6f700 1 Processor -- start 2021-02-10T11:13:46.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.405 7f2fc6e6f700 1 -- start start 2021-02-10T11:13:46.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.405 7f2fc6e6f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2fc01b0470 0x7f2fc01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:46.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.405 7f2fc6e6f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2fc00ef610 0x7f2fc00efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:46.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.405 7f2fc6e6f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc00f0b50 con 0x7f2fc00f0030 2021-02-10T11:13:46.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.405 7f2fc6e6f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc00e52e0 con 0x7f2fc00ef610 2021-02-10T11:13:46.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.405 7f2fc6e6f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc00e5490 con 0x7f2fc01b0470 2021-02-10T11:13:46.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.406 7f2fbffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2fc00ef610 0x7f2fc00efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:46.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.406 7f2fc548c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f2fc00f0030 0x7f2fc00f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45082/0 (socket says 172.21.2.101:45082) 2021-02-10T11:13:46.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.406 7f2fc548c700 1 -- 172.21.2.101:0/30949908 learned_addr learned my addr 172.21.2.101:0/30949908 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:46.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.406 7f2fbffff700 1 -- 172.21.2.101:0/30949908 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2fc01b0470 msgr2=0x7f2fc01b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:46.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.406 7f2fbffff700 1 --2- 172.21.2.101:0/30949908 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2fc01b0470 0x7f2fc01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:46.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.406 7f2fbffff700 1 -- 172.21.2.101:0/30949908 >> v1:172.21.2.101:6789/0 conn(0x7f2fc00f0030 legacy=0x7f2fc00f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:46.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.406 7f2fbffff700 1 -- 172.21.2.101:0/30949908 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc00f0430 con 0x7f2fc00ef610 2021-02-10T11:13:46.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.406 7f2fbffff700 1 --2- 172.21.2.101:0/30949908 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fc00ef610 0x7f2fc00efa10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f2fb0006650 tx=0x7f2fb0009a80).ready entity=mon.0 client_cookie=56bd7a72086f1a86 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:46.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.407 7f2fbf7fe700 1 -- 172.21.2.101:0/30949908 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2fb000c0c0 con 0x7f2fc00ef610 2021-02-10T11:13:46.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.407 7f2fbf7fe700 1 -- 172.21.2.101:0/30949908 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb000c340 con 0x7f2fc00ef610 2021-02-10T11:13:46.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.407 7f2fbf7fe700 1 -- 172.21.2.101:0/30949908 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2fb000c860 con 0x7f2fc00ef610 2021-02-10T11:13:46.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.407 7f2fc6e6f700 1 -- 172.21.2.101:0/30949908 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fc00ef610 msgr2=0x7f2fc00efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:46.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.407 7f2fc6e6f700 1 --2- 172.21.2.101:0/30949908 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fc00ef610 0x7f2fc00efa10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f2fb0006650 tx=0x7f2fb0009a80).stop 2021-02-10T11:13:46.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.407 7f2fc6e6f700 1 -- 172.21.2.101:0/30949908 shutdown_connections 2021-02-10T11:13:46.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.407 7f2fc6e6f700 1 --2- 172.21.2.101:0/30949908 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fc00ef610 0x7f2fc00efa10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:46.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.407 7f2fc6e6f700 1 --2- 172.21.2.101:0/30949908 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2fc01b0470 0x7f2fc01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:46.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.407 7f2fc6e6f700 1 -- 172.21.2.101:0/30949908 >> 172.21.2.101:0/30949908 conn(0x7f2fc00ce3e0 msgr2=0x7f2fc00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:46.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.407 7f2fc6e6f700 1 -- 172.21.2.101:0/30949908 shutdown_connections 2021-02-10T11:13:46.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.407 7f2fc6e6f700 1 -- 172.21.2.101:0/30949908 wait complete. 2021-02-10T11:13:46.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.422 7f2fc6e6f700 1 Processor -- start 2021-02-10T11:13:46.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.422 7f2fc6e6f700 1 -- start start 2021-02-10T11:13:46.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.422 7f2fc6e6f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2fc00c07e0 0x7f2fc00988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:46.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.422 7f2fc6e6f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fc00aceb0 0x7f2fc0098f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:46.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.422 7f2fc6e6f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2fc00c1240 0x7f2fc00c1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:46.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.422 7f2fc6e6f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc0199f00 con 0x7f2fc00aceb0 2021-02-10T11:13:46.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.422 7f2fc6e6f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc00c0cc0 con 0x7f2fc00c07e0 2021-02-10T11:13:46.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.422 7f2fc6e6f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc0095240 con 0x7f2fc00c1240 2021-02-10T11:13:46.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.423 7f2fbffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fc00aceb0 0x7f2fc0098f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:46.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.423 7f2fbffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fc00aceb0 0x7f2fc0098f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44594/0 (socket says 172.21.2.101:44594) 2021-02-10T11:13:46.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.423 7f2fbffff700 1 -- 172.21.2.101:0/3018929813 learned_addr learned my addr 172.21.2.101:0/3018929813 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:46.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.423 7f2fbffff700 1 -- 172.21.2.101:0/3018929813 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2fc00c1240 msgr2=0x7f2fc00c1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:46.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.423 7f2fbffff700 1 --2- 172.21.2.101:0/3018929813 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2fc00c1240 0x7f2fc00c1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:46.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.423 7f2fbffff700 1 -- 172.21.2.101:0/3018929813 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2fc00c07e0 msgr2=0x7f2fc00988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:46.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.423 7f2fbffff700 1 --2- 172.21.2.101:0/3018929813 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2fc00c07e0 0x7f2fc00988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:46.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.423 7f2fbffff700 1 -- 172.21.2.101:0/3018929813 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc00c1c60 con 0x7f2fc00aceb0 2021-02-10T11:13:46.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.423 7f2fbffff700 1 --2- 172.21.2.101:0/3018929813 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fc00aceb0 0x7f2fc0098f10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f2fb000d760 tx=0x7f2fb000db00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:46.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.423 7f2fbd7fa700 1 -- 172.21.2.101:0/3018929813 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2fb0000c90 con 0x7f2fc00aceb0 2021-02-10T11:13:46.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.423 7f2fbd7fa700 1 -- 172.21.2.101:0/3018929813 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb0002030 con 0x7f2fc00aceb0 2021-02-10T11:13:46.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.424 7f2fbd7fa700 1 -- 172.21.2.101:0/3018929813 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2fb0002550 con 0x7f2fc00aceb0 2021-02-10T11:13:46.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.424 7f2fc6e6f700 1 -- 172.21.2.101:0/3018929813 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc00c1c60 con 0x7f2fc00aceb0 2021-02-10T11:13:46.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.424 7f2fc6e6f700 1 -- 172.21.2.101:0/3018929813 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc00c1c60 con 0x7f2fc00aceb0 2021-02-10T11:13:46.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.424 7f2fbd7fa700 1 -- 172.21.2.101:0/3018929813 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2fb0002210 con 0x7f2fc00aceb0 2021-02-10T11:13:46.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.424 7f2fbd7fa700 1 --2- 172.21.2.101:0/3018929813 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2fa401cad0 0x7f2fa401eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:46.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.424 7f2fbd7fa700 1 -- 172.21.2.101:0/3018929813 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2fb00325a0 con 0x7f2fc00aceb0 2021-02-10T11:13:46.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.424 7f2fc4c8b700 1 --2- 172.21.2.101:0/3018929813 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2fa401cad0 0x7f2fa401eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:46.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.425 7f2fc4c8b700 1 --2- 172.21.2.101:0/3018929813 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2fa401cad0 0x7f2fa401eff0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:46.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.455 7f2fc6e6f700 1 -- 172.21.2.101:0/3018929813 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc00edd10 con 0x7f2fc00aceb0 2021-02-10T11:13:46.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.458 7f2fbd7fa700 1 -- 172.21.2.101:0/3018929813 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2fb0002420 con 0x7f2fc00aceb0 2021-02-10T11:13:46.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.584 7f2fc6e6f700 1 -- 172.21.2.101:0/3018929813 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f2fc00e52e0 con 0x7f2fc00aceb0 2021-02-10T11:13:46.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.585 7f2fbd7fa700 1 -- 172.21.2.101:0/3018929813 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v20) v1 ==== 104+0+10 (secure 0 0 0) 0x7f2fb00021f0 con 0x7f2fc00aceb0 2021-02-10T11:13:46.589 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:13:46.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.587 7f2fc6e6f700 1 -- 172.21.2.101:0/3018929813 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2fa401cad0 msgr2=0x7f2fa401eff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:46.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.587 7f2fc6e6f700 1 --2- 172.21.2.101:0/3018929813 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2fa401cad0 0x7f2fa401eff0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:46.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.587 7f2fc6e6f700 1 -- 172.21.2.101:0/3018929813 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fc00aceb0 msgr2=0x7f2fc0098f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:46.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.587 7f2fc6e6f700 1 --2- 172.21.2.101:0/3018929813 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fc00aceb0 0x7f2fc0098f10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f2fb000d760 tx=0x7f2fb000db00).stop 2021-02-10T11:13:46.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.587 7f2fc6e6f700 1 -- 172.21.2.101:0/3018929813 shutdown_connections 2021-02-10T11:13:46.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.587 7f2fc6e6f700 1 --2- 172.21.2.101:0/3018929813 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2fa401cad0 0x7f2fa401eff0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:46.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.587 7f2fc6e6f700 1 --2- 172.21.2.101:0/3018929813 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2fc00c1240 0x7f2fc00c1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:46.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.587 7f2fc6e6f700 1 --2- 172.21.2.101:0/3018929813 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fc00aceb0 0x7f2fc0098f10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:46.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.587 7f2fc6e6f700 1 --2- 172.21.2.101:0/3018929813 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2fc00c07e0 0x7f2fc00988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:46.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.587 7f2fc6e6f700 1 -- 172.21.2.101:0/3018929813 >> 172.21.2.101:0/3018929813 conn(0x7f2fc00c40e0 msgr2=0x7f2fc00f22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:46.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.587 7f2fc6e6f700 1 -- 172.21.2.101:0/3018929813 shutdown_connections 2021-02-10T11:13:46.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:46.587 7f2fc6e6f700 1 -- 172.21.2.101:0/3018929813 wait complete. 2021-02-10T11:13:46.599 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:13:47.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.008 7f3108076700 1 Processor -- start 2021-02-10T11:13:47.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.008 7f3108076700 1 -- start start 2021-02-10T11:13:47.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.008 7f3108076700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f31001b0870 0x7f31000f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:47.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.008 7f3108076700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31001b0c70 con 0x7f31001b0470 2021-02-10T11:13:47.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.008 7f3108076700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31000f0b50 con 0x7f31000efe00 2021-02-10T11:13:47.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.008 7f3108076700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31000e5ea0 con 0x7f31001b0870 2021-02-10T11:13:47.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.008 7f3105691700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f31001b0870 0x7f31000f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:47.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.008 7f3105e92700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f31001b0470 0x7f31000ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45092/0 (socket says 172.21.2.101:45092) 2021-02-10T11:13:47.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.009 7f3105e92700 1 -- 172.21.2.101:0/116565224 learned_addr learned my addr 172.21.2.101:0/116565224 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:47.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.009 7f3105691700 1 -- 172.21.2.101:0/116565224 >> v1:172.21.2.108:6789/0 conn(0x7f31000efe00 legacy=0x7f31000f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:47.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.009 7f3105691700 1 -- 172.21.2.101:0/116565224 >> v1:172.21.2.101:6789/0 conn(0x7f31001b0470 legacy=0x7f31000ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:47.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.009 7f3105691700 1 -- 172.21.2.101:0/116565224 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31000e52e0 con 0x7f31001b0870 2021-02-10T11:13:47.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.009 7f3105691700 1 --2- 172.21.2.101:0/116565224 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f31001b0870 0x7f31000f20e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f30fc006650 tx=0x7f30fc009a00).ready entity=mon.0 client_cookie=23335ed2765fc7c1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:47.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.009 7f3104e90700 1 -- 172.21.2.101:0/116565224 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f30fc0009b0 con 0x7f31001b0870 2021-02-10T11:13:47.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.009 7f3104e90700 1 -- 172.21.2.101:0/116565224 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30fc00c0c0 con 0x7f31001b0870 2021-02-10T11:13:47.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.009 7f3104e90700 1 -- 172.21.2.101:0/116565224 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f30fc00c5a0 con 0x7f31001b0870 2021-02-10T11:13:47.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.009 7f3108076700 1 -- 172.21.2.101:0/116565224 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f31001b0870 msgr2=0x7f31000f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:47.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.009 7f3108076700 1 --2- 172.21.2.101:0/116565224 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f31001b0870 0x7f31000f20e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f30fc006650 tx=0x7f30fc009a00).stop 2021-02-10T11:13:47.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.010 7f3108076700 1 -- 172.21.2.101:0/116565224 shutdown_connections 2021-02-10T11:13:47.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.010 7f3108076700 1 --2- 172.21.2.101:0/116565224 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f31001b0870 0x7f31000f20e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:47.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.010 7f3108076700 1 -- 172.21.2.101:0/116565224 >> 172.21.2.101:0/116565224 conn(0x7f31000ce3e0 msgr2=0x7f31000e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:47.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.010 7f3108076700 1 -- 172.21.2.101:0/116565224 shutdown_connections 2021-02-10T11:13:47.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.010 7f3108076700 1 -- 172.21.2.101:0/116565224 wait complete. 2021-02-10T11:13:47.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.024 7f3108076700 1 Processor -- start 2021-02-10T11:13:47.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.025 7f3108076700 1 -- start start 2021-02-10T11:13:47.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.025 7f3108076700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3100082170 0x7f31000a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:47.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.025 7f3108076700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f31000894a0 0x7f31000898a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:47.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.025 7f3108076700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3100082f20 0x7f3100083320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:47.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.025 7f3108076700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3100089ec0 con 0x7f31000894a0 2021-02-10T11:13:47.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.025 7f3108076700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3100087fa0 con 0x7f3100082170 2021-02-10T11:13:47.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.025 7f3108076700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3100088520 con 0x7f3100082f20 2021-02-10T11:13:47.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.025 7f3105691700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f31000894a0 0x7f31000898a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:47.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.025 7f3105691700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f31000894a0 0x7f31000898a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44608/0 (socket says 172.21.2.101:44608) 2021-02-10T11:13:47.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.025 7f3105691700 1 -- 172.21.2.101:0/2321865772 learned_addr learned my addr 172.21.2.101:0/2321865772 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:47.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.025 7f3106693700 1 --2- 172.21.2.101:0/2321865772 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3100082f20 0x7f3100083320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:47.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.025 7f3105e92700 1 --2- 172.21.2.101:0/2321865772 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3100082170 0x7f31000a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:47.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.025 7f3105691700 1 -- 172.21.2.101:0/2321865772 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3100082f20 msgr2=0x7f3100083320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:47.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.025 7f3105691700 1 --2- 172.21.2.101:0/2321865772 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3100082f20 0x7f3100083320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:47.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.025 7f3105691700 1 -- 172.21.2.101:0/2321865772 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3100082170 msgr2=0x7f31000a6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:47.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.025 7f3105691700 1 --2- 172.21.2.101:0/2321865772 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3100082170 0x7f31000a6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:47.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.025 7f3105691700 1 -- 172.21.2.101:0/2321865772 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31000a6a20 con 0x7f31000894a0 2021-02-10T11:13:47.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.025 7f3105e92700 1 --2- 172.21.2.101:0/2321865772 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3100082170 0x7f31000a6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:13:47.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.026 7f3105691700 1 --2- 172.21.2.101:0/2321865772 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f31000894a0 0x7f31000898a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f30fc0096a0 tx=0x7f30fc00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:47.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.026 7f30f27fc700 1 -- 172.21.2.101:0/2321865772 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f30fc00bd80 con 0x7f31000894a0 2021-02-10T11:13:47.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.026 7f30f27fc700 1 -- 172.21.2.101:0/2321865772 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30fc002080 con 0x7f31000894a0 2021-02-10T11:13:47.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.026 7f30f27fc700 1 -- 172.21.2.101:0/2321865772 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f30fc0025a0 con 0x7f31000894a0 2021-02-10T11:13:47.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.026 7f3108076700 1 -- 172.21.2.101:0/2321865772 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31000a6a20 con 0x7f31000894a0 2021-02-10T11:13:47.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.026 7f3108076700 1 -- 172.21.2.101:0/2321865772 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31000a6a20 con 0x7f31000894a0 2021-02-10T11:13:47.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.027 7f30f27fc700 1 -- 172.21.2.101:0/2321865772 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f30fc009300 con 0x7f31000894a0 2021-02-10T11:13:47.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.027 7f30f27fc700 1 --2- 172.21.2.101:0/2321865772 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30e80186c0 0x7f30e801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:47.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.027 7f30f27fc700 1 -- 172.21.2.101:0/2321865772 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f30fc032480 con 0x7f31000894a0 2021-02-10T11:13:47.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.027 7f3105e92700 1 --2- 172.21.2.101:0/2321865772 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30e80186c0 0x7f30e801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:47.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.027 7f3105e92700 1 --2- 172.21.2.101:0/2321865772 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30e80186c0 0x7f30e801abc0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:47.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.058 7f3108076700 1 -- 172.21.2.101:0/2321865772 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3100089ec0 con 0x7f31000894a0 2021-02-10T11:13:47.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.061 7f30f27fc700 1 -- 172.21.2.101:0/2321865772 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f30fc0009b0 con 0x7f31000894a0 2021-02-10T11:13:47.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.188 7f3108076700 1 -- 172.21.2.101:0/2321865772 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f3100089ec0 con 0x7f31000894a0 2021-02-10T11:13:47.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.188 7f30f27fc700 1 -- 172.21.2.101:0/2321865772 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v20) v1 ==== 100+0+10 (secure 0 0 0) 0x7f30fc00bd80 con 0x7f31000894a0 2021-02-10T11:13:47.193 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:13:47.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.190 7f3108076700 1 -- 172.21.2.101:0/2321865772 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30e80186c0 msgr2=0x7f30e801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:47.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.190 7f3108076700 1 --2- 172.21.2.101:0/2321865772 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30e80186c0 0x7f30e801abc0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:47.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.190 7f3108076700 1 -- 172.21.2.101:0/2321865772 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f31000894a0 msgr2=0x7f31000898a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:47.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.190 7f3108076700 1 --2- 172.21.2.101:0/2321865772 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f31000894a0 0x7f31000898a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f30fc0096a0 tx=0x7f30fc00b990).stop 2021-02-10T11:13:47.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.190 7f3108076700 1 -- 172.21.2.101:0/2321865772 shutdown_connections 2021-02-10T11:13:47.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.190 7f3108076700 1 --2- 172.21.2.101:0/2321865772 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30e80186c0 0x7f30e801abc0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:47.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.190 7f3108076700 1 --2- 172.21.2.101:0/2321865772 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3100082f20 0x7f3100083320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:47.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.190 7f3108076700 1 --2- 172.21.2.101:0/2321865772 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f31000894a0 0x7f31000898a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:47.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.190 7f3108076700 1 --2- 172.21.2.101:0/2321865772 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3100082170 0x7f31000a6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:47.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.190 7f3108076700 1 -- 172.21.2.101:0/2321865772 >> 172.21.2.101:0/2321865772 conn(0x7f3100088d10 msgr2=0x7f31000afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:47.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.190 7f3108076700 1 -- 172.21.2.101:0/2321865772 shutdown_connections 2021-02-10T11:13:47.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.191 7f3108076700 1 -- 172.21.2.101:0/2321865772 wait complete. 2021-02-10T11:13:47.203 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:13:47.203 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:13:47.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.607 7f86c2d22700 1 Processor -- start 2021-02-10T11:13:47.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.607 7f86c2d22700 1 -- start start 2021-02-10T11:13:47.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.607 7f86c2d22700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86bc0e5070 0x7f86bc1b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:47.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.607 7f86c2d22700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f86bc0e5470 0x7f86bc07d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:47.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.607 7f86c2d22700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f86bc1b3e40 0x7f86bc07dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:47.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.607 7f86c2d22700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86bc07ed80 con 0x7f86bc0e5470 2021-02-10T11:13:47.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.607 7f86c2d22700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86bc07e2d0 con 0x7f86bc1b3e40 2021-02-10T11:13:47.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.607 7f86c2d22700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc07e4c0 con 0x7f86bc0e5070 2021-02-10T11:13:47.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.607 7f86bbfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f86bc0e5470 0x7f86bc07d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:47.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.607 7f86bbfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f86bc0e5470 0x7f86bc07d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44614/0 (socket says 172.21.2.101:44614) 2021-02-10T11:13:47.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.607 7f86bbfff700 1 -- 172.21.2.101:0/577363778 learned_addr learned my addr 172.21.2.101:0/577363778 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:47.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.608 7f86bbfff700 1 -- 172.21.2.101:0/577363778 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86bc0e5070 msgr2=0x7f86bc1b3820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:13:47.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.608 7f86bbfff700 1 --2- 172.21.2.101:0/577363778 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86bc0e5070 0x7f86bc1b3820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:47.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.608 7f86bbfff700 1 -- 172.21.2.101:0/577363778 >> v2:172.21.2.108:3300/0 conn(0x7f86bc1b3e40 msgr2=0x7f86bc07dcb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:13:47.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.608 7f86bbfff700 1 --2- 172.21.2.101:0/577363778 >> v2:172.21.2.108:3300/0 conn(0x7f86bc1b3e40 0x7f86bc07dcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:47.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.608 7f86bbfff700 1 -- 172.21.2.101:0/577363778 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc07e6e0 con 0x7f86bc0e5470 2021-02-10T11:13:47.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.608 7f86bbfff700 1 --2- 172.21.2.101:0/577363778 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86bc0e5470 0x7f86bc07d690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f86ac006700 tx=0x7f86ac009b80).ready entity=mon.0 client_cookie=2ffdf54d53f0456e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:47.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.608 7f86bb7fe700 1 -- 172.21.2.101:0/577363778 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f86ac00c0c0 con 0x7f86bc0e5470 2021-02-10T11:13:47.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.608 7f86bb7fe700 1 -- 172.21.2.101:0/577363778 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ac00c340 con 0x7f86bc0e5470 2021-02-10T11:13:47.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.608 7f86bb7fe700 1 -- 172.21.2.101:0/577363778 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f86ac00c860 con 0x7f86bc0e5470 2021-02-10T11:13:47.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.608 7f86c2d22700 1 -- 172.21.2.101:0/577363778 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86bc0e5470 msgr2=0x7f86bc07d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:47.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.608 7f86c2d22700 1 --2- 172.21.2.101:0/577363778 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86bc0e5470 0x7f86bc07d690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f86ac006700 tx=0x7f86ac009b80).stop 2021-02-10T11:13:47.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.609 7f86c2d22700 1 -- 172.21.2.101:0/577363778 shutdown_connections 2021-02-10T11:13:47.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.609 7f86c2d22700 1 --2- 172.21.2.101:0/577363778 >> v2:172.21.2.108:3300/0 conn(0x7f86bc1b3e40 0x7f86bc07dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:47.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.609 7f86c2d22700 1 --2- 172.21.2.101:0/577363778 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86bc0e5470 0x7f86bc07d690 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:47.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.609 7f86c2d22700 1 --2- 172.21.2.101:0/577363778 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86bc0e5070 0x7f86bc1b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:47.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.609 7f86c2d22700 1 -- 172.21.2.101:0/577363778 >> 172.21.2.101:0/577363778 conn(0x7f86bc0ce690 msgr2=0x7f86bc0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:47.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.609 7f86c2d22700 1 -- 172.21.2.101:0/577363778 shutdown_connections 2021-02-10T11:13:47.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.609 7f86c2d22700 1 -- 172.21.2.101:0/577363778 wait complete. 2021-02-10T11:13:47.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.624 7f86c2d22700 1 Processor -- start 2021-02-10T11:13:47.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.624 7f86c2d22700 1 -- start start 2021-02-10T11:13:47.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.624 7f86c2d22700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86bc192960 0x7f86bc192d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:47.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.624 7f86c2d22700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86bc0ad110 0x7f86bc0ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:47.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.624 7f86c2d22700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86bc0adb30 0x7f86bc08e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:47.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.624 7f86c2d22700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc1a03e0 con 0x7f86bc0ad110 2021-02-10T11:13:47.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.624 7f86c2d22700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc193380 con 0x7f86bc192960 2021-02-10T11:13:47.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.624 7f86c2d22700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc08ea80 con 0x7f86bc0adb30 2021-02-10T11:13:47.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.624 7f86bbfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86bc0ad110 0x7f86bc0ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:47.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.624 7f86bbfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86bc0ad110 0x7f86bc0ad510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44620/0 (socket says 172.21.2.101:44620) 2021-02-10T11:13:47.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.624 7f86bbfff700 1 -- 172.21.2.101:0/4269672080 learned_addr learned my addr 172.21.2.101:0/4269672080 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:47.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.624 7f86bbfff700 1 -- 172.21.2.101:0/4269672080 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86bc0adb30 msgr2=0x7f86bc08e460 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:47.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.624 7f86bbfff700 1 --2- 172.21.2.101:0/4269672080 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86bc0adb30 0x7f86bc08e460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:47.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.624 7f86bbfff700 1 -- 172.21.2.101:0/4269672080 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86bc192960 msgr2=0x7f86bc192d60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:47.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.624 7f86bbfff700 1 --2- 172.21.2.101:0/4269672080 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86bc192960 0x7f86bc192d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:47.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.624 7f86bbfff700 1 -- 172.21.2.101:0/4269672080 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc08eca0 con 0x7f86bc0ad110 2021-02-10T11:13:47.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.625 7f86bbfff700 1 --2- 172.21.2.101:0/4269672080 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86bc0ad110 0x7f86bc0ad510 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f86ac00b5e0 tx=0x7f86ac00b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:47.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.625 7f86b97fa700 1 -- 172.21.2.101:0/4269672080 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f86ac00bd70 con 0x7f86bc0ad110 2021-02-10T11:13:47.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.625 7f86b97fa700 1 -- 172.21.2.101:0/4269672080 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ac002030 con 0x7f86bc0ad110 2021-02-10T11:13:47.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.625 7f86c2d22700 1 -- 172.21.2.101:0/4269672080 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86bc08eca0 con 0x7f86bc0ad110 2021-02-10T11:13:47.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.625 7f86c2d22700 1 -- 172.21.2.101:0/4269672080 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86bc08eca0 con 0x7f86bc0ad110 2021-02-10T11:13:47.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.625 7f86b97fa700 1 -- 172.21.2.101:0/4269672080 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f86ac002550 con 0x7f86bc0ad110 2021-02-10T11:13:47.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.625 7f86b97fa700 1 -- 172.21.2.101:0/4269672080 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f86ac000c90 con 0x7f86bc0ad110 2021-02-10T11:13:47.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.625 7f86b97fa700 1 --2- 172.21.2.101:0/4269672080 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86a00186d0 0x7f86a001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:47.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.625 7f86b97fa700 1 -- 172.21.2.101:0/4269672080 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f86ac032860 con 0x7f86bc0ad110 2021-02-10T11:13:47.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.626 7f86c0b3e700 1 --2- 172.21.2.101:0/4269672080 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86a00186d0 0x7f86a001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:47.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.626 7f86c0b3e700 1 --2- 172.21.2.101:0/4269672080 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86a00186d0 0x7f86a001abd0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:47.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.657 7f86c2d22700 1 -- 172.21.2.101:0/4269672080 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86bc107120 con 0x7f86bc0ad110 2021-02-10T11:13:47.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.660 7f86b97fa700 1 -- 172.21.2.101:0/4269672080 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f86ac01f5a0 con 0x7f86bc0ad110 2021-02-10T11:13:47.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.789 7f86c2d22700 1 -- 172.21.2.101:0/4269672080 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86bc107120 con 0x7f86bc0ad110 2021-02-10T11:13:47.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.789 7f86b97fa700 1 -- 172.21.2.101:0/4269672080 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2097 (secure 0 0 0) 0x7f86ac00bd70 con 0x7f86bc0ad110 2021-02-10T11:13:47.794 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:13:47.794 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/3184957761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3184957761},{"type":"v1","addr":"172.21.2.108:6833","nonce":3184957761}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:13:14.147021","modified":"2021-02-10 11:13:23.547179","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":5,"state":"up:active","state_seq":3,"addr":"172.21.2.101:6835/3530104431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3530104431},{"type":"v1","addr":"172.21.2.101:6835","nonce":3530104431}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-02-10T11:13:47.794 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 6 2021-02-10T11:13:47.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.791 7f86c2d22700 1 -- 172.21.2.101:0/4269672080 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86a00186d0 msgr2=0x7f86a001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:47.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.791 7f86c2d22700 1 --2- 172.21.2.101:0/4269672080 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86a00186d0 0x7f86a001abd0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:47.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.791 7f86c2d22700 1 -- 172.21.2.101:0/4269672080 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86bc0ad110 msgr2=0x7f86bc0ad510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:47.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.791 7f86c2d22700 1 --2- 172.21.2.101:0/4269672080 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86bc0ad110 0x7f86bc0ad510 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f86ac00b5e0 tx=0x7f86ac00b980).stop 2021-02-10T11:13:47.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.791 7f86c2d22700 1 -- 172.21.2.101:0/4269672080 shutdown_connections 2021-02-10T11:13:47.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.791 7f86c2d22700 1 --2- 172.21.2.101:0/4269672080 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86a00186d0 0x7f86a001abd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:47.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.791 7f86c2d22700 1 --2- 172.21.2.101:0/4269672080 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86bc0adb30 0x7f86bc08e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:47.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.791 7f86c2d22700 1 --2- 172.21.2.101:0/4269672080 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86bc0ad110 0x7f86bc0ad510 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:47.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.791 7f86c2d22700 1 --2- 172.21.2.101:0/4269672080 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86bc192960 0x7f86bc192d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:47.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.791 7f86c2d22700 1 -- 172.21.2.101:0/4269672080 >> 172.21.2.101:0/4269672080 conn(0x7f86bc0891f0 msgr2=0x7f86bc1a0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:47.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.791 7f86c2d22700 1 -- 172.21.2.101:0/4269672080 shutdown_connections 2021-02-10T11:13:47.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:47.791 7f86c2d22700 1 -- 172.21.2.101:0/4269672080 wait complete. 2021-02-10T11:13:48.005 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:13:14.147021', 'modified': '2021-02-10 11:13:23.547179', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4299': {'gid': 4299, 'name': 'a', 'rank': 0, 'incarnation': 5, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.101:6835/3530104431', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.101:6834', 'nonce': 3530104431}, {'type': 'v1', 'addr': '172.21.2.101:6835', 'nonce': 3530104431}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:13:48.279 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:13:48.279 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok status 2021-02-10T11:13:48.679 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:13:48.679 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:13:48.679 INFO:teuthology.orchestra.run.gibba001.stdout: "whoami": 0, 2021-02-10T11:13:48.679 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 4299, 2021-02-10T11:13:48.679 INFO:teuthology.orchestra.run.gibba001.stdout: "want_state": "up:active", 2021-02-10T11:13:48.679 INFO:teuthology.orchestra.run.gibba001.stdout: "state": "up:active", 2021-02-10T11:13:48.679 INFO:teuthology.orchestra.run.gibba001.stdout: "rank_uptime": 26.127879272000001, 2021-02-10T11:13:48.679 INFO:teuthology.orchestra.run.gibba001.stdout: "mdsmap_epoch": 6, 2021-02-10T11:13:48.680 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch": 20, 2021-02-10T11:13:48.680 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch_barrier": 20, 2021-02-10T11:13:48.680 INFO:teuthology.orchestra.run.gibba001.stdout: "uptime": 30.519810033999999 2021-02-10T11:13:48.680 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:13:48.683 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 4299, "want_state": "up:active", "state": "up:active", "rank_uptime": 26.127879272000001, "mdsmap_epoch": 6, "osdmap_epoch": 20, "osdmap_epoch_barrier": 20, "uptime": 30.519810033999999 } 2021-02-10T11:13:48.683 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:13:48.683 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-02-10T11:13:48.683 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:13:48.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:13:48.684 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:13:49.017 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:13:49.362 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:13:49.362 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:13:49.705 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:13:49.705 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:13:49.706 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:13:49.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:13:49.706 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:13:50.040 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:13:50.041 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2021-02-10T11:13:50.041 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:13:50.044 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:13:50.388 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:13:50.388 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:13:50.453 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:2021-02-10 11:13:50.449 7f87e0646f80 -1 init, newargv = 0x564cc565f100 newargc=9 2021-02-10T11:13:50.453 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[29562]: starting ceph client 2021-02-10T11:13:50.457 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[29562]: starting fuse 2021-02-10T11:13:50.730 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:13:50.730 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:13:50.731 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:13:50.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:13:50.731 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:13:51.088 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:13:51.089 INFO:teuthology.orchestra.run.gibba001.stdout:44 2021-02-10T11:13:51.089 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2021-02-10T11:13:51.089 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:13:51.091 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:13:51.092 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:13:51.092 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:13:51.092 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:13:51.092 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:13:51.092 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:13:51.092 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:13:51.093 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-02-10T11:13:51.093 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:13:51.093 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:13:51.093 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:13:51.093 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:13:51.093 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:13:51.093 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:13:51.094 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:13:51.094 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:13:51.094 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:13:51.094 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:13:51.094 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:13:51.094 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:13:51.095 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:13:51.095 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.1")) 2021-02-10T11:13:51.095 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:13:51.466 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.1.29562.asok 2021-02-10T11:13:51.468 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.29562.asok 2021-02-10T11:13:51.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:13:51.469 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.29562.asok status 2021-02-10T11:13:51.892 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:13:51.892 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:13:51.892 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:13:51.892 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:13:51.892 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "1", 2021-02-10T11:13:51.892 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:13:51.892 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:13:51.892 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "29562", 2021-02-10T11:13:51.892 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:13:51.893 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:51.893 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:13:51.893 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:13:51.893 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 4602, 2021-02-10T11:13:51.893 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:13:51.893 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:13:51.893 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:13:51.893 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 4602 2021-02-10T11:13:51.893 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:51.893 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:13:51.893 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:13:51.893 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:13:51.893 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 767671656 2021-02-10T11:13:51.893 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:13:51.894 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:51.894 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:13:51.894 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:13:51.894 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:13:51.894 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 767671656 2021-02-10T11:13:51.894 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:13:51.894 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.4602 172.21.2.101:0/767671656", 2021-02-10T11:13:51.894 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/767671656", 2021-02-10T11:13:51.894 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:13:51.894 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 6, 2021-02-10T11:13:51.894 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 20, 2021-02-10T11:13:51.894 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:13:51.894 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:13:51.894 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:13:51.896 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:13:52.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.309 7f364489c700 1 Processor -- start 2021-02-10T11:13:52.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.309 7f364489c700 1 -- start start 2021-02-10T11:13:52.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.309 7f364489c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f36400e5470 0x7f364007e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:52.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.309 7f364489c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f364007ed80 con 0x7f36400e5070 2021-02-10T11:13:52.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.309 7f364489c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36401b4010 con 0x7f364007f300 2021-02-10T11:13:52.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.309 7f364489c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36400d1790 con 0x7f36400e5470 2021-02-10T11:13:52.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.310 7f363e61c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f36400e5070 0x7f36401b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45132/0 (socket says 172.21.2.101:45132) 2021-02-10T11:13:52.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.310 7f363e61c700 1 -- 172.21.2.101:0/3033239054 learned_addr learned my addr 172.21.2.101:0/3033239054 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:52.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.310 7f363de1b700 1 --2- 172.21.2.101:0/3033239054 >> v2:172.21.2.108:3300/0 conn(0x7f36400e5470 0x7f364007e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:52.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.310 7f363d61a700 1 -- 172.21.2.101:0/3033239054 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1814554754 0 0) 0x7f36280009b0 con 0x7f36400e5070 2021-02-10T11:13:52.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.310 7f363d61a700 1 -- 172.21.2.101:0/3033239054 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f362c002fe0 con 0x7f36400e5070 2021-02-10T11:13:52.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.310 7f363d61a700 1 -- 172.21.2.101:0/3033239054 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1693487927 0 0) 0x7f3628001010 con 0x7f36400e5070 2021-02-10T11:13:52.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.310 7f363d61a700 1 -- 172.21.2.101:0/3033239054 >> v2:172.21.2.108:3300/0 conn(0x7f36400e5470 msgr2=0x7f364007e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:52.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.310 7f363d61a700 1 --2- 172.21.2.101:0/3033239054 >> v2:172.21.2.108:3300/0 conn(0x7f36400e5470 0x7f364007e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:52.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.310 7f363d61a700 1 -- 172.21.2.101:0/3033239054 >> v1:172.21.2.108:6789/0 conn(0x7f364007f300 legacy=0x7f364007f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:52.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.310 7f363d61a700 1 -- 172.21.2.101:0/3033239054 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36400d19b0 con 0x7f36400e5070 2021-02-10T11:13:52.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.310 7f363d61a700 1 -- 172.21.2.101:0/3033239054 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f3628001530 con 0x7f36400e5070 2021-02-10T11:13:52.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.310 7f363d61a700 1 -- 172.21.2.101:0/3033239054 >> v1:172.21.2.101:6789/0 conn(0x7f36400e5070 legacy=0x7f36401b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:52.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.310 7f363d61a700 1 --2- 172.21.2.101:0/3033239054 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f362c003a60 0x7f362c005f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:52.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.310 7f363d61a700 1 --2- 172.21.2.101:0/3033239054 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f362c0067b0 0x7f362c008cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:52.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.310 7f363d61a700 1 --2- 172.21.2.101:0/3033239054 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f362c009590 0x7f362c00bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:52.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.311 7f363d61a700 1 -- 172.21.2.101:0/3033239054 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c00c300 con 0x7f362c003a60 2021-02-10T11:13:52.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.311 7f363d61a700 1 -- 172.21.2.101:0/3033239054 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c00c520 con 0x7f362c009590 2021-02-10T11:13:52.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.311 7f363d61a700 1 -- 172.21.2.101:0/3033239054 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362c00c740 con 0x7f362c0067b0 2021-02-10T11:13:52.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.311 7f363e61c700 1 --2- 172.21.2.101:0/3033239054 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f362c003a60 0x7f362c005f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:52.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.311 7f363de1b700 1 --2- 172.21.2.101:0/3033239054 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f362c0067b0 0x7f362c008cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:52.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.311 7f363ee1d700 1 --2- 172.21.2.101:0/3033239054 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f362c009590 0x7f362c00bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:52.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.311 7f363e61c700 1 -- 172.21.2.101:0/3033239054 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f362c0067b0 msgr2=0x7f362c008cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:52.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.311 7f363e61c700 1 --2- 172.21.2.101:0/3033239054 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f362c0067b0 0x7f362c008cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:52.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.311 7f363e61c700 1 -- 172.21.2.101:0/3033239054 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f362c009590 msgr2=0x7f362c00bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:52.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.311 7f363e61c700 1 --2- 172.21.2.101:0/3033239054 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f362c009590 0x7f362c00bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:52.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.311 7f363e61c700 1 -- 172.21.2.101:0/3033239054 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c0509b0 con 0x7f362c003a60 2021-02-10T11:13:52.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.311 7f363ee1d700 1 --2- 172.21.2.101:0/3033239054 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f362c009590 0x7f362c00bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:13:52.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.311 7f363ee1d700 1 -- 172.21.2.101:0/3033239054 reap_dead start 2021-02-10T11:13:52.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.311 7f363e61c700 1 --2- 172.21.2.101:0/3033239054 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f362c003a60 0x7f362c005f10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f36280017d0 tx=0x7f3628007d30).ready entity=mon.0 client_cookie=435679ddb75cde32 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:52.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.311 7f363d61a700 1 -- 172.21.2.101:0/3033239054 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3628001380 con 0x7f362c003a60 2021-02-10T11:13:52.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.311 7f363d61a700 1 -- 172.21.2.101:0/3033239054 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362800a310 con 0x7f362c003a60 2021-02-10T11:13:52.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.311 7f363d61a700 1 -- 172.21.2.101:0/3033239054 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f362800a760 con 0x7f362c003a60 2021-02-10T11:13:52.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.311 7f364489c700 1 -- 172.21.2.101:0/3033239054 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f362c003a60 msgr2=0x7f362c005f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:52.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.311 7f364489c700 1 --2- 172.21.2.101:0/3033239054 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f362c003a60 0x7f362c005f10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f36280017d0 tx=0x7f3628007d30).stop 2021-02-10T11:13:52.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.312 7f364489c700 1 -- 172.21.2.101:0/3033239054 shutdown_connections 2021-02-10T11:13:52.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.312 7f364489c700 1 --2- 172.21.2.101:0/3033239054 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f362c003a60 0x7f362c005f10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:52.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.312 7f364489c700 1 -- 172.21.2.101:0/3033239054 >> 172.21.2.101:0/3033239054 conn(0x7f36400ce690 msgr2=0x7f36400e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:52.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.312 7f364489c700 1 -- 172.21.2.101:0/3033239054 shutdown_connections 2021-02-10T11:13:52.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.312 7f364489c700 1 -- 172.21.2.101:0/3033239054 wait complete. 2021-02-10T11:13:52.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.327 7f364489c700 1 Processor -- start 2021-02-10T11:13:52.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.327 7f364489c700 1 -- start start 2021-02-10T11:13:52.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.327 7f364489c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f364018d210 0x7f36401ab320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:52.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.327 7f364489c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36400984c0 0x7f3640097290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:52.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.327 7f364489c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f36400978b0 0x7f3640097cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:52.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.327 7f364489c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364009a540 con 0x7f36400984c0 2021-02-10T11:13:52.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.327 7f364489c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36401aba60 con 0x7f36400978b0 2021-02-10T11:13:52.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.327 7f364489c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364009aac0 con 0x7f364018d210 2021-02-10T11:13:52.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.327 7f363de1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36400984c0 0x7f3640097290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:52.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.327 7f363de1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36400984c0 0x7f3640097290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44654/0 (socket says 172.21.2.101:44654) 2021-02-10T11:13:52.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.327 7f363de1b700 1 -- 172.21.2.101:0/913564948 learned_addr learned my addr 172.21.2.101:0/913564948 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:52.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.327 7f363e61c700 1 --2- 172.21.2.101:0/913564948 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f364018d210 0x7f36401ab320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:52.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.327 7f363ee1d700 1 --2- 172.21.2.101:0/913564948 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f36400978b0 0x7f3640097cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:52.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.328 7f363de1b700 1 -- 172.21.2.101:0/913564948 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f364018d210 msgr2=0x7f36401ab320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:52.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.328 7f363de1b700 1 --2- 172.21.2.101:0/913564948 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f364018d210 0x7f36401ab320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:52.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.328 7f363de1b700 1 -- 172.21.2.101:0/913564948 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f36400978b0 msgr2=0x7f3640097cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:52.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.328 7f363de1b700 1 --2- 172.21.2.101:0/913564948 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f36400978b0 0x7f3640097cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:52.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.328 7f363de1b700 1 -- 172.21.2.101:0/913564948 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364009ace0 con 0x7f36400984c0 2021-02-10T11:13:52.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.328 7f363e61c700 1 --2- 172.21.2.101:0/913564948 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f364018d210 0x7f36401ab320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:13:52.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.328 7f363de1b700 1 --2- 172.21.2.101:0/913564948 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36400984c0 0x7f3640097290 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f3630006f60 tx=0x7f363000b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:52.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.328 7f3636ffd700 1 -- 172.21.2.101:0/913564948 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f363000bd60 con 0x7f36400984c0 2021-02-10T11:13:52.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.328 7f3636ffd700 1 -- 172.21.2.101:0/913564948 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3630000c90 con 0x7f36400984c0 2021-02-10T11:13:52.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.328 7f3636ffd700 1 -- 172.21.2.101:0/913564948 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f36300022b0 con 0x7f36400984c0 2021-02-10T11:13:52.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.328 7f364489c700 1 -- 172.21.2.101:0/913564948 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364009ace0 con 0x7f36400984c0 2021-02-10T11:13:52.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.328 7f364489c700 1 -- 172.21.2.101:0/913564948 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364009b270 con 0x7f36400984c0 2021-02-10T11:13:52.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.329 7f3636ffd700 1 -- 172.21.2.101:0/913564948 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3630006b00 con 0x7f36400984c0 2021-02-10T11:13:52.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.329 7f3636ffd700 1 --2- 172.21.2.101:0/913564948 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36240186c0 0x7f362401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:52.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.329 7f3636ffd700 1 -- 172.21.2.101:0/913564948 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3630032520 con 0x7f36400984c0 2021-02-10T11:13:52.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.329 7f363e61c700 1 --2- 172.21.2.101:0/913564948 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36240186c0 0x7f362401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:52.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.329 7f363e61c700 1 --2- 172.21.2.101:0/913564948 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36240186c0 0x7f362401abc0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:52.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.359 7f364489c700 1 -- 172.21.2.101:0/913564948 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36401aba40 con 0x7f36400984c0 2021-02-10T11:13:52.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.362 7f3636ffd700 1 -- 172.21.2.101:0/913564948 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3630000960 con 0x7f36400984c0 2021-02-10T11:13:52.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.487 7f364489c700 1 -- 172.21.2.101:0/913564948 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36401aba40 con 0x7f36400984c0 2021-02-10T11:13:52.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.487 7f3636ffd700 1 -- 172.21.2.101:0/913564948 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+13993 (secure 0 0 0) 0x7f3630006b00 con 0x7f36400984c0 2021-02-10T11:13:52.494 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:13:52.495 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":20,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:13:15.149234","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:13:11.922272","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2021-02-10 11:13:13.141733","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:13:52.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.489 7f364489c700 1 -- 172.21.2.101:0/913564948 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36240186c0 msgr2=0x7f362401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:52.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.489 7f364489c700 1 --2- 172.21.2.101:0/913564948 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36240186c0 0x7f362401abc0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:52.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.489 7f364489c700 1 -- 172.21.2.101:0/913564948 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36400984c0 msgr2=0x7f3640097290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:52.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.489 7f364489c700 1 --2- 172.21.2.101:0/913564948 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36400984c0 0x7f3640097290 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f3630006f60 tx=0x7f363000b970).stop 2021-02-10T11:13:52.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.489 7f364489c700 1 -- 172.21.2.101:0/913564948 shutdown_connections 2021-02-10T11:13:52.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.489 7f364489c700 1 --2- 172.21.2.101:0/913564948 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f36400978b0 0x7f3640097cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:52.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.489 7f364489c700 1 --2- 172.21.2.101:0/913564948 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36400984c0 0x7f3640097290 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:52.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.489 7f364489c700 1 --2- 172.21.2.101:0/913564948 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36240186c0 0x7f362401abc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:52.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.489 7f364489c700 1 --2- 172.21.2.101:0/913564948 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f364018d210 0x7f36401ab320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:52.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.489 7f364489c700 1 -- 172.21.2.101:0/913564948 >> 172.21.2.101:0/913564948 conn(0x7f36401b2a40 msgr2=0x7f3640198bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:52.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.489 7f364489c700 1 -- 172.21.2.101:0/913564948 shutdown_connections 2021-02-10T11:13:52.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.489 7f364489c700 1 -- 172.21.2.101:0/913564948 wait complete. 2021-02-10T11:13:52.502 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:13:11.922272', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:13:13.141733', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:13:52.502 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:13:52.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.916 7fe5cc5b6700 1 Processor -- start 2021-02-10T11:13:52.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.916 7fe5cc5b6700 1 -- start start 2021-02-10T11:13:52.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.917 7fe5cc5b6700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe5c41b0470 0x7fe5c41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:52.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.917 7fe5cc5b6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe5c40f20e0 0x7fe5c40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:52.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.917 7fe5cc5b6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c40f0b50 con 0x7fe5c40ef610 2021-02-10T11:13:52.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.917 7fe5cc5b6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c40f0820 con 0x7fe5c40f20e0 2021-02-10T11:13:52.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.917 7fe5cc5b6700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c40f24e0 con 0x7fe5c41b0470 2021-02-10T11:13:52.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.917 7fe5cabd3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe5c40f20e0 0x7fe5c40f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:52.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.917 7fe5cabd3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe5c40f20e0 0x7fe5c40f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44658/0 (socket says 172.21.2.101:44658) 2021-02-10T11:13:52.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.917 7fe5cabd3700 1 -- 172.21.2.101:0/3885873429 learned_addr learned my addr 172.21.2.101:0/3885873429 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:52.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.917 7fe5cabd3700 1 -- 172.21.2.101:0/3885873429 >> v2:172.21.2.108:3300/0 conn(0x7fe5c41b0470 msgr2=0x7fe5c41b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:13:52.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.917 7fe5cabd3700 1 --2- 172.21.2.101:0/3885873429 >> v2:172.21.2.108:3300/0 conn(0x7fe5c41b0470 0x7fe5c41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:52.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.917 7fe5cabd3700 1 -- 172.21.2.101:0/3885873429 >> v1:172.21.2.108:6789/0 conn(0x7fe5c40ef610 legacy=0x7fe5c40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:52.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.917 7fe5cabd3700 1 -- 172.21.2.101:0/3885873429 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c40e52e0 con 0x7fe5c40f20e0 2021-02-10T11:13:52.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.918 7fe5cabd3700 1 --2- 172.21.2.101:0/3885873429 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5c40f20e0 0x7fe5c40f0200 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fe5bc006f00 tx=0x7fe5bc009e90).ready entity=mon.0 client_cookie=e59e2e84518abbd2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:52.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.918 7fe5c93d0700 1 -- 172.21.2.101:0/3885873429 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5bc00b300 con 0x7fe5c40f20e0 2021-02-10T11:13:52.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.918 7fe5c93d0700 1 -- 172.21.2.101:0/3885873429 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc00b640 con 0x7fe5c40f20e0 2021-02-10T11:13:52.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.918 7fe5c93d0700 1 -- 172.21.2.101:0/3885873429 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5bc00ba90 con 0x7fe5c40f20e0 2021-02-10T11:13:52.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.918 7fe5cc5b6700 1 -- 172.21.2.101:0/3885873429 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5c40f20e0 msgr2=0x7fe5c40f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:52.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.918 7fe5cc5b6700 1 --2- 172.21.2.101:0/3885873429 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5c40f20e0 0x7fe5c40f0200 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fe5bc006f00 tx=0x7fe5bc009e90).stop 2021-02-10T11:13:52.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.918 7fe5cc5b6700 1 -- 172.21.2.101:0/3885873429 shutdown_connections 2021-02-10T11:13:52.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.918 7fe5cc5b6700 1 --2- 172.21.2.101:0/3885873429 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5c40f20e0 0x7fe5c40f0200 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:52.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.918 7fe5cc5b6700 1 --2- 172.21.2.101:0/3885873429 >> v2:172.21.2.108:3300/0 conn(0x7fe5c41b0470 0x7fe5c41b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:52.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.918 7fe5cc5b6700 1 -- 172.21.2.101:0/3885873429 >> 172.21.2.101:0/3885873429 conn(0x7fe5c40ce3e0 msgr2=0x7fe5c40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:52.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.919 7fe5cc5b6700 1 -- 172.21.2.101:0/3885873429 shutdown_connections 2021-02-10T11:13:52.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.919 7fe5cc5b6700 1 -- 172.21.2.101:0/3885873429 wait complete. 2021-02-10T11:13:52.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.933 7fe5cc5b6700 1 Processor -- start 2021-02-10T11:13:52.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.933 7fe5cc5b6700 1 -- start start 2021-02-10T11:13:52.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.934 7fe5cc5b6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5c40a6c90 0x7fe5c41924c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:52.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.934 7fe5cc5b6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5c4192ae0 0x7fe5c4192ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:52.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.934 7fe5cc5b6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5c40919e0 0x7fe5c4091de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:52.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.934 7fe5cc5b6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c419d2e0 con 0x7fe5c40919e0 2021-02-10T11:13:52.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.934 7fe5cc5b6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c40914d0 con 0x7fe5c40a6c90 2021-02-10T11:13:52.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.934 7fe5cc5b6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c40a6820 con 0x7fe5c4192ae0 2021-02-10T11:13:52.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.934 7fe5cabd3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5c40919e0 0x7fe5c4091de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:52.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.934 7fe5cabd3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5c40919e0 0x7fe5c4091de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44664/0 (socket says 172.21.2.101:44664) 2021-02-10T11:13:52.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.934 7fe5cabd3700 1 -- 172.21.2.101:0/4283335980 learned_addr learned my addr 172.21.2.101:0/4283335980 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:52.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.934 7fe5cabd3700 1 -- 172.21.2.101:0/4283335980 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5c4192ae0 msgr2=0x7fe5c4192ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:52.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.934 7fe5cabd3700 1 --2- 172.21.2.101:0/4283335980 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5c4192ae0 0x7fe5c4192ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:52.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.934 7fe5cabd3700 1 -- 172.21.2.101:0/4283335980 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5c40a6c90 msgr2=0x7fe5c41924c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:52.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.934 7fe5cabd3700 1 --2- 172.21.2.101:0/4283335980 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5c40a6c90 0x7fe5c41924c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:52.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.934 7fe5cabd3700 1 -- 172.21.2.101:0/4283335980 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c4092400 con 0x7fe5c40919e0 2021-02-10T11:13:52.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.934 7fe5cabd3700 1 --2- 172.21.2.101:0/4283335980 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5c40919e0 0x7fe5c4091de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fe5bc0099f0 tx=0x7fe5bc00bce0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:52.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.935 7fe5b6ffd700 1 -- 172.21.2.101:0/4283335980 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5bc003030 con 0x7fe5c40919e0 2021-02-10T11:13:52.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.935 7fe5b6ffd700 1 -- 172.21.2.101:0/4283335980 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc000d50 con 0x7fe5c40919e0 2021-02-10T11:13:52.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.935 7fe5b6ffd700 1 -- 172.21.2.101:0/4283335980 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5bc0014a0 con 0x7fe5c40919e0 2021-02-10T11:13:52.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.935 7fe5cc5b6700 1 -- 172.21.2.101:0/4283335980 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c4092400 con 0x7fe5c40919e0 2021-02-10T11:13:52.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.935 7fe5cc5b6700 1 -- 172.21.2.101:0/4283335980 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c4092400 con 0x7fe5c40919e0 2021-02-10T11:13:52.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.935 7fe5b6ffd700 1 -- 172.21.2.101:0/4283335980 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe5bc005820 con 0x7fe5c40919e0 2021-02-10T11:13:52.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.935 7fe5b6ffd700 1 --2- 172.21.2.101:0/4283335980 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5ac0186c0 0x7fe5ac01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:52.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.936 7fe5b6ffd700 1 -- 172.21.2.101:0/4283335980 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe5bc032b40 con 0x7fe5c40919e0 2021-02-10T11:13:52.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.936 7fe5ca3d2700 1 --2- 172.21.2.101:0/4283335980 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5ac0186c0 0x7fe5ac01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:52.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.936 7fe5ca3d2700 1 --2- 172.21.2.101:0/4283335980 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5ac0186c0 0x7fe5ac01abc0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:52.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.966 7fe5cc5b6700 1 -- 172.21.2.101:0/4283335980 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c40e52e0 con 0x7fe5c40919e0 2021-02-10T11:13:52.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:52.969 7fe5b6ffd700 1 -- 172.21.2.101:0/4283335980 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe5bc00be20 con 0x7fe5c40919e0 2021-02-10T11:13:53.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.095 7fe5cc5b6700 1 -- 172.21.2.101:0/4283335980 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fe5c40e52e0 con 0x7fe5c40919e0 2021-02-10T11:13:53.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.096 7fe5b6ffd700 1 -- 172.21.2.101:0/4283335980 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe5bc01f880 con 0x7fe5c40919e0 2021-02-10T11:13:53.100 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:13:53.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.098 7fe5cc5b6700 1 -- 172.21.2.101:0/4283335980 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5ac0186c0 msgr2=0x7fe5ac01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:53.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.098 7fe5cc5b6700 1 --2- 172.21.2.101:0/4283335980 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5ac0186c0 0x7fe5ac01abc0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:53.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.098 7fe5cc5b6700 1 -- 172.21.2.101:0/4283335980 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5c40919e0 msgr2=0x7fe5c4091de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:53.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.098 7fe5cc5b6700 1 --2- 172.21.2.101:0/4283335980 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5c40919e0 0x7fe5c4091de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fe5bc0099f0 tx=0x7fe5bc00bce0).stop 2021-02-10T11:13:53.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.098 7fe5cc5b6700 1 -- 172.21.2.101:0/4283335980 shutdown_connections 2021-02-10T11:13:53.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.098 7fe5cc5b6700 1 --2- 172.21.2.101:0/4283335980 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5ac0186c0 0x7fe5ac01abc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:53.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.098 7fe5cc5b6700 1 --2- 172.21.2.101:0/4283335980 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5c4192ae0 0x7fe5c4192ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:53.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.098 7fe5cc5b6700 1 --2- 172.21.2.101:0/4283335980 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5c40919e0 0x7fe5c4091de0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:53.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.098 7fe5cc5b6700 1 --2- 172.21.2.101:0/4283335980 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5c40a6c90 0x7fe5c41924c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:53.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.098 7fe5cc5b6700 1 -- 172.21.2.101:0/4283335980 >> 172.21.2.101:0/4283335980 conn(0x7fe5c40ac550 msgr2=0x7fe5c40be1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:53.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.098 7fe5cc5b6700 1 -- 172.21.2.101:0/4283335980 shutdown_connections 2021-02-10T11:13:53.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.098 7fe5cc5b6700 1 -- 172.21.2.101:0/4283335980 wait complete. 2021-02-10T11:13:53.110 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:13:53.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.519 7fe625e2f700 1 Processor -- start 2021-02-10T11:13:53.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.519 7fe625e2f700 1 -- start start 2021-02-10T11:13:53.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.519 7fe625e2f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe6201b0470 0x7fe6201b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:53.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.519 7fe625e2f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6200ef610 0x7fe6200efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:53.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.519 7fe625e2f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe6200f0030 0x7fe6200f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:53.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.519 7fe625e2f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6200e5ea0 con 0x7fe6200f0030 2021-02-10T11:13:53.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.519 7fe625e2f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6200f0b50 con 0x7fe6201b0470 2021-02-10T11:13:53.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.519 7fe625e2f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6200e52e0 con 0x7fe6200ef610 2021-02-10T11:13:53.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.520 7fe61ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe6200f0030 0x7fe6200f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:53.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.520 7fe61ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe6200f0030 0x7fe6200f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44670/0 (socket says 172.21.2.101:44670) 2021-02-10T11:13:53.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.520 7fe61ffff700 1 -- 172.21.2.101:0/401073000 learned_addr learned my addr 172.21.2.101:0/401073000 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:53.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.520 7fe61ffff700 1 -- 172.21.2.101:0/401073000 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6200ef610 msgr2=0x7fe6200efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:13:53.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.520 7fe61ffff700 1 --2- 172.21.2.101:0/401073000 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6200ef610 0x7fe6200efa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:53.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.520 7fe61ffff700 1 -- 172.21.2.101:0/401073000 >> v2:172.21.2.108:3300/0 conn(0x7fe6201b0470 msgr2=0x7fe6201b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:53.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.520 7fe61ffff700 1 --2- 172.21.2.101:0/401073000 >> v2:172.21.2.108:3300/0 conn(0x7fe6201b0470 0x7fe6201b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:53.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.520 7fe61ffff700 1 -- 172.21.2.101:0/401073000 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6200f10d0 con 0x7fe6200f0030 2021-02-10T11:13:53.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.520 7fe61ffff700 1 --2- 172.21.2.101:0/401073000 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6200f0030 0x7fe6200f0430 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fe618006f00 tx=0x7fe618009f10).ready entity=mon.0 client_cookie=54bc1227bba91514 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:53.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.520 7fe61e7fc700 1 -- 172.21.2.101:0/401073000 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe61800b3b0 con 0x7fe6200f0030 2021-02-10T11:13:53.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.520 7fe61e7fc700 1 -- 172.21.2.101:0/401073000 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61800b630 con 0x7fe6200f0030 2021-02-10T11:13:53.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.520 7fe61e7fc700 1 -- 172.21.2.101:0/401073000 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe61800bb50 con 0x7fe6200f0030 2021-02-10T11:13:53.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.520 7fe625e2f700 1 -- 172.21.2.101:0/401073000 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6200f0030 msgr2=0x7fe6200f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:53.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.520 7fe625e2f700 1 --2- 172.21.2.101:0/401073000 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6200f0030 0x7fe6200f0430 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fe618006f00 tx=0x7fe618009f10).stop 2021-02-10T11:13:53.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.521 7fe625e2f700 1 -- 172.21.2.101:0/401073000 shutdown_connections 2021-02-10T11:13:53.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.521 7fe625e2f700 1 --2- 172.21.2.101:0/401073000 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6200ef610 0x7fe6200efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:53.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.521 7fe625e2f700 1 --2- 172.21.2.101:0/401073000 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6200f0030 0x7fe6200f0430 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:53.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.521 7fe625e2f700 1 --2- 172.21.2.101:0/401073000 >> v2:172.21.2.108:3300/0 conn(0x7fe6201b0470 0x7fe6201b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:53.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.521 7fe625e2f700 1 -- 172.21.2.101:0/401073000 >> 172.21.2.101:0/401073000 conn(0x7fe6200ce3e0 msgr2=0x7fe6200e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:53.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.521 7fe625e2f700 1 -- 172.21.2.101:0/401073000 shutdown_connections 2021-02-10T11:13:53.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.521 7fe625e2f700 1 -- 172.21.2.101:0/401073000 wait complete. 2021-02-10T11:13:53.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.535 7fe625e2f700 1 Processor -- start 2021-02-10T11:13:53.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.535 7fe625e2f700 1 -- start start 2021-02-10T11:13:53.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.536 7fe625e2f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe620193230 0x7fe6200a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:53.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.536 7fe625e2f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe62008f5b0 0x7fe62008f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:53.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.536 7fe625e2f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe62008e6a0 0x7fe62008eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:53.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.536 7fe625e2f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62008ffd0 con 0x7fe62008e6a0 2021-02-10T11:13:53.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.536 7fe625e2f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe620090240 con 0x7fe62008f5b0 2021-02-10T11:13:53.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.536 7fe625e2f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe62008f0c0 con 0x7fe620193230 2021-02-10T11:13:53.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.536 7fe61ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe62008e6a0 0x7fe62008eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:53.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.536 7fe61ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe62008e6a0 0x7fe62008eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44678/0 (socket says 172.21.2.101:44678) 2021-02-10T11:13:53.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.536 7fe61ffff700 1 -- 172.21.2.101:0/2361139603 learned_addr learned my addr 172.21.2.101:0/2361139603 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:53.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.536 7fe61ffff700 1 -- 172.21.2.101:0/2361139603 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe620193230 msgr2=0x7fe6200a6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:53.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.536 7fe61f7fe700 1 --2- 172.21.2.101:0/2361139603 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe620193230 0x7fe6200a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:53.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.536 7fe61ffff700 1 --2- 172.21.2.101:0/2361139603 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe620193230 0x7fe6200a6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:53.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.536 7fe61ffff700 1 -- 172.21.2.101:0/2361139603 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe62008f5b0 msgr2=0x7fe62008f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:53.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.536 7fe61ffff700 1 --2- 172.21.2.101:0/2361139603 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe62008f5b0 0x7fe62008f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:53.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.536 7fe61ffff700 1 -- 172.21.2.101:0/2361139603 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62008f2a0 con 0x7fe62008e6a0 2021-02-10T11:13:53.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.536 7fe61f7fe700 1 --2- 172.21.2.101:0/2361139603 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe620193230 0x7fe6200a6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:13:53.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.537 7fe61ffff700 1 --2- 172.21.2.101:0/2361139603 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe62008e6a0 0x7fe62008eaa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fe618009390 tx=0x7fe61800bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:53.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.537 7fe617fff700 1 -- 172.21.2.101:0/2361139603 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe618003030 con 0x7fe62008e6a0 2021-02-10T11:13:53.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.537 7fe617fff700 1 -- 172.21.2.101:0/2361139603 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6180093d0 con 0x7fe62008e6a0 2021-02-10T11:13:53.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.537 7fe617fff700 1 -- 172.21.2.101:0/2361139603 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe618001480 con 0x7fe62008e6a0 2021-02-10T11:13:53.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.537 7fe625e2f700 1 -- 172.21.2.101:0/2361139603 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe62008f2a0 con 0x7fe62008e6a0 2021-02-10T11:13:53.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.537 7fe625e2f700 1 -- 172.21.2.101:0/2361139603 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6201924f0 con 0x7fe62008e6a0 2021-02-10T11:13:53.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.537 7fe617fff700 1 -- 172.21.2.101:0/2361139603 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe618001270 con 0x7fe62008e6a0 2021-02-10T11:13:53.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.537 7fe617fff700 1 --2- 172.21.2.101:0/2361139603 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6040186c0 0x7fe60401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:53.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.538 7fe617fff700 1 -- 172.21.2.101:0/2361139603 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe618032b30 con 0x7fe62008e6a0 2021-02-10T11:13:53.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.538 7fe61f7fe700 1 --2- 172.21.2.101:0/2361139603 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6040186c0 0x7fe60401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:53.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.538 7fe61f7fe700 1 --2- 172.21.2.101:0/2361139603 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6040186c0 0x7fe60401abc0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:53.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.569 7fe625e2f700 1 -- 172.21.2.101:0/2361139603 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6200f0b50 con 0x7fe62008e6a0 2021-02-10T11:13:53.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.572 7fe617fff700 1 -- 172.21.2.101:0/2361139603 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe6180068e0 con 0x7fe62008e6a0 2021-02-10T11:13:53.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.700 7fe625e2f700 1 -- 172.21.2.101:0/2361139603 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7fe6200f0b50 con 0x7fe62008e6a0 2021-02-10T11:13:53.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.700 7fe617fff700 1 -- 172.21.2.101:0/2361139603 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v20) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe618001270 con 0x7fe62008e6a0 2021-02-10T11:13:53.705 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:13:53.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.702 7fe625e2f700 1 -- 172.21.2.101:0/2361139603 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6040186c0 msgr2=0x7fe60401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:53.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.702 7fe625e2f700 1 --2- 172.21.2.101:0/2361139603 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6040186c0 0x7fe60401abc0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:53.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.702 7fe625e2f700 1 -- 172.21.2.101:0/2361139603 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe62008e6a0 msgr2=0x7fe62008eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:53.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.702 7fe625e2f700 1 --2- 172.21.2.101:0/2361139603 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe62008e6a0 0x7fe62008eaa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fe618009390 tx=0x7fe61800bd40).stop 2021-02-10T11:13:53.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.702 7fe625e2f700 1 -- 172.21.2.101:0/2361139603 shutdown_connections 2021-02-10T11:13:53.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.702 7fe625e2f700 1 --2- 172.21.2.101:0/2361139603 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe62008e6a0 0x7fe62008eaa0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:53.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.702 7fe625e2f700 1 --2- 172.21.2.101:0/2361139603 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe62008f5b0 0x7fe62008f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:53.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.702 7fe625e2f700 1 --2- 172.21.2.101:0/2361139603 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6040186c0 0x7fe60401abc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:53.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.702 7fe625e2f700 1 --2- 172.21.2.101:0/2361139603 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe620193230 0x7fe6200a6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:53.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.702 7fe625e2f700 1 -- 172.21.2.101:0/2361139603 >> 172.21.2.101:0/2361139603 conn(0x7fe6200bf4e0 msgr2=0x7fe6200be160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:53.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.703 7fe625e2f700 1 -- 172.21.2.101:0/2361139603 shutdown_connections 2021-02-10T11:13:53.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:53.703 7fe625e2f700 1 -- 172.21.2.101:0/2361139603 wait complete. 2021-02-10T11:13:53.715 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:13:54.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.125 7f5025582700 1 Processor -- start 2021-02-10T11:13:54.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.125 7f5025582700 1 -- start start 2021-02-10T11:13:54.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.125 7f5025582700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f50201b0470 0x7f50201b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:54.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.125 7f5025582700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f50200ef610 0x7f50200efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:54.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.125 7f5025582700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50200f0030 0x7f50200f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:54.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.125 7f5025582700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50200e5ea0 con 0x7f50200ef610 2021-02-10T11:13:54.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.125 7f5025582700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50200f0b50 con 0x7f50201b0470 2021-02-10T11:13:54.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.125 7f5025582700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50200e52e0 con 0x7f50200f0030 2021-02-10T11:13:54.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.125 7f501eabe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f50200ef610 0x7f50200efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:54.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.125 7f501eabe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f50200ef610 0x7f50200efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44682/0 (socket says 172.21.2.101:44682) 2021-02-10T11:13:54.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.125 7f501eabe700 1 -- 172.21.2.101:0/4000383265 learned_addr learned my addr 172.21.2.101:0/4000383265 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:54.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.126 7f501eabe700 1 -- 172.21.2.101:0/4000383265 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50200f0030 msgr2=0x7f50200f0430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:13:54.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.126 7f501eabe700 1 --2- 172.21.2.101:0/4000383265 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50200f0030 0x7f50200f0430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:54.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.126 7f501eabe700 1 -- 172.21.2.101:0/4000383265 >> v2:172.21.2.108:3300/0 conn(0x7f50201b0470 msgr2=0x7f50201b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:13:54.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.126 7f501eabe700 1 --2- 172.21.2.101:0/4000383265 >> v2:172.21.2.108:3300/0 conn(0x7f50201b0470 0x7f50201b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:54.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.126 7f501eabe700 1 -- 172.21.2.101:0/4000383265 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50200f10d0 con 0x7f50200ef610 2021-02-10T11:13:54.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.126 7f501eabe700 1 --2- 172.21.2.101:0/4000383265 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50200ef610 0x7f50200efa10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f5008006700 tx=0x7f5008009b80).ready entity=mon.0 client_cookie=2ce6d1ef1826ad71 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:54.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.126 7f501e2bd700 1 -- 172.21.2.101:0/4000383265 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f500800c0c0 con 0x7f50200ef610 2021-02-10T11:13:54.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.126 7f501e2bd700 1 -- 172.21.2.101:0/4000383265 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500800c340 con 0x7f50200ef610 2021-02-10T11:13:54.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.126 7f501e2bd700 1 -- 172.21.2.101:0/4000383265 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f500800c820 con 0x7f50200ef610 2021-02-10T11:13:54.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.126 7f5025582700 1 -- 172.21.2.101:0/4000383265 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50200ef610 msgr2=0x7f50200efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:54.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.126 7f5025582700 1 --2- 172.21.2.101:0/4000383265 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50200ef610 0x7f50200efa10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f5008006700 tx=0x7f5008009b80).stop 2021-02-10T11:13:54.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.126 7f5025582700 1 -- 172.21.2.101:0/4000383265 shutdown_connections 2021-02-10T11:13:54.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.127 7f5025582700 1 --2- 172.21.2.101:0/4000383265 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50200f0030 0x7f50200f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:54.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.127 7f5025582700 1 --2- 172.21.2.101:0/4000383265 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50200ef610 0x7f50200efa10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:54.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.127 7f5025582700 1 --2- 172.21.2.101:0/4000383265 >> v2:172.21.2.108:3300/0 conn(0x7f50201b0470 0x7f50201b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:54.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.127 7f5025582700 1 -- 172.21.2.101:0/4000383265 >> 172.21.2.101:0/4000383265 conn(0x7f50200ce3e0 msgr2=0x7f50200e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:54.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.127 7f5025582700 1 -- 172.21.2.101:0/4000383265 shutdown_connections 2021-02-10T11:13:54.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.127 7f5025582700 1 -- 172.21.2.101:0/4000383265 wait complete. 2021-02-10T11:13:54.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.141 7f5025582700 1 Processor -- start 2021-02-10T11:13:54.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.141 7f5025582700 1 -- start start 2021-02-10T11:13:54.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.142 7f5025582700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f50200b20b0 0x7f50200ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:54.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.142 7f5025582700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50200b24b0 0x7f5020191dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:54.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.142 7f5025582700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50200acf90 0x7f50201923e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:54.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.142 7f5025582700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50200b1b30 con 0x7f50200acf90 2021-02-10T11:13:54.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.142 7f5025582700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50201a04e0 con 0x7f50200b20b0 2021-02-10T11:13:54.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.142 7f5025582700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50201a06c0 con 0x7f50200b24b0 2021-02-10T11:13:54.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.142 7f501fac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50200acf90 0x7f50201923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:54.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.142 7f501fac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50200acf90 0x7f50201923e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44690/0 (socket says 172.21.2.101:44690) 2021-02-10T11:13:54.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.142 7f501fac0700 1 -- 172.21.2.101:0/1737532063 learned_addr learned my addr 172.21.2.101:0/1737532063 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:54.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.142 7f501fac0700 1 -- 172.21.2.101:0/1737532063 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50200b24b0 msgr2=0x7f5020191dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:54.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.142 7f501fac0700 1 --2- 172.21.2.101:0/1737532063 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50200b24b0 0x7f5020191dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:54.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.142 7f501fac0700 1 -- 172.21.2.101:0/1737532063 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f50200b20b0 msgr2=0x7f50200ac970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:54.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.142 7f501fac0700 1 --2- 172.21.2.101:0/1737532063 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f50200b20b0 0x7f50200ac970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:54.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.142 7f501fac0700 1 -- 172.21.2.101:0/1737532063 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5020192be0 con 0x7f50200acf90 2021-02-10T11:13:54.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.142 7f501fac0700 1 --2- 172.21.2.101:0/1737532063 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50200acf90 0x7f50201923e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f5018009960 tx=0x7f501800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:54.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.143 7f5017fff700 1 -- 172.21.2.101:0/1737532063 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5018000d10 con 0x7f50200acf90 2021-02-10T11:13:54.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.143 7f5017fff700 1 -- 172.21.2.101:0/1737532063 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5018003030 con 0x7f50200acf90 2021-02-10T11:13:54.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.143 7f5025582700 1 -- 172.21.2.101:0/1737532063 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5020192be0 con 0x7f50200acf90 2021-02-10T11:13:54.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.143 7f5025582700 1 -- 172.21.2.101:0/1737532063 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5020193170 con 0x7f50200acf90 2021-02-10T11:13:54.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.143 7f5017fff700 1 -- 172.21.2.101:0/1737532063 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5018001590 con 0x7f50200acf90 2021-02-10T11:13:54.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.143 7f5017fff700 1 -- 172.21.2.101:0/1737532063 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5018001310 con 0x7f50200acf90 2021-02-10T11:13:54.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.143 7f5017fff700 1 --2- 172.21.2.101:0/1737532063 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50040186c0 0x7f500401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:54.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.144 7f5017fff700 1 -- 172.21.2.101:0/1737532063 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5018032910 con 0x7f50200acf90 2021-02-10T11:13:54.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.144 7f501f2bf700 1 --2- 172.21.2.101:0/1737532063 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50040186c0 0x7f500401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:54.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.144 7f501f2bf700 1 --2- 172.21.2.101:0/1737532063 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50040186c0 0x7f500401abc0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:54.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.175 7f5025582700 1 -- 172.21.2.101:0/1737532063 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5020105af0 con 0x7f50200acf90 2021-02-10T11:13:54.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.178 7f5017fff700 1 -- 172.21.2.101:0/1737532063 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f50180030f0 con 0x7f50200acf90 2021-02-10T11:13:54.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.307 7f5025582700 1 -- 172.21.2.101:0/1737532063 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f50200f0b50 con 0x7f50200acf90 2021-02-10T11:13:54.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.308 7f5017fff700 1 -- 172.21.2.101:0/1737532063 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v20) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5018000f40 con 0x7f50200acf90 2021-02-10T11:13:54.313 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:13:54.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.310 7f5025582700 1 -- 172.21.2.101:0/1737532063 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50040186c0 msgr2=0x7f500401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.310 7f5025582700 1 --2- 172.21.2.101:0/1737532063 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50040186c0 0x7f500401abc0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.310 7f5025582700 1 -- 172.21.2.101:0/1737532063 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50200acf90 msgr2=0x7f50201923e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.310 7f5025582700 1 --2- 172.21.2.101:0/1737532063 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50200acf90 0x7f50201923e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f5018009960 tx=0x7f501800bcf0).stop 2021-02-10T11:13:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.310 7f5025582700 1 -- 172.21.2.101:0/1737532063 shutdown_connections 2021-02-10T11:13:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.310 7f5025582700 1 --2- 172.21.2.101:0/1737532063 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50040186c0 0x7f500401abc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.310 7f5025582700 1 --2- 172.21.2.101:0/1737532063 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50200b24b0 0x7f5020191dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.310 7f5025582700 1 --2- 172.21.2.101:0/1737532063 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50200acf90 0x7f50201923e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.310 7f5025582700 1 --2- 172.21.2.101:0/1737532063 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f50200b20b0 0x7f50200ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.310 7f5025582700 1 -- 172.21.2.101:0/1737532063 >> 172.21.2.101:0/1737532063 conn(0x7f50200bf4e0 msgr2=0x7f50200be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.310 7f5025582700 1 -- 172.21.2.101:0/1737532063 shutdown_connections 2021-02-10T11:13:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.310 7f5025582700 1 -- 172.21.2.101:0/1737532063 wait complete. 2021-02-10T11:13:54.323 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:13:54.323 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:13:54.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.727 7f1716c8a700 1 Processor -- start 2021-02-10T11:13:54.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.727 7f1716c8a700 1 -- start start 2021-02-10T11:13:54.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.728 7f1716c8a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f17100e5470 0x7f171007e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:54.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.728 7f1716c8a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f171007ed80 con 0x7f171007f300 2021-02-10T11:13:54.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.728 7f1716c8a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17101b4010 con 0x7f17100e5070 2021-02-10T11:13:54.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.728 7f1716c8a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17100d1790 con 0x7f17100e5470 2021-02-10T11:13:54.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.728 7f17152a7700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f171007f300 0x7f171007f700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45188/0 (socket says 172.21.2.101:45188) 2021-02-10T11:13:54.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.728 7f17152a7700 1 -- 172.21.2.101:0/1789984316 learned_addr learned my addr 172.21.2.101:0/1789984316 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:54.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.728 7f170f7fe700 1 -- 172.21.2.101:0/1789984316 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 663267804 0 0) 0x7f1708000d30 con 0x7f171007f300 2021-02-10T11:13:54.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.728 7f170f7fe700 1 -- 172.21.2.101:0/1789984316 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16fc002fe0 con 0x7f171007f300 2021-02-10T11:13:54.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f170ffff700 1 --2- 172.21.2.101:0/1789984316 >> v2:172.21.2.108:3300/0 conn(0x7f17100e5470 0x7f171007e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:54.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f170f7fe700 1 -- 172.21.2.101:0/1789984316 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3522009678 0 0) 0x7f1708001270 con 0x7f171007f300 2021-02-10T11:13:54.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f170f7fe700 1 -- 172.21.2.101:0/1789984316 >> v2:172.21.2.108:3300/0 conn(0x7f17100e5470 msgr2=0x7f171007e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:54.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f170f7fe700 1 --2- 172.21.2.101:0/1789984316 >> v2:172.21.2.108:3300/0 conn(0x7f17100e5470 0x7f171007e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:54.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f170f7fe700 1 -- 172.21.2.101:0/1789984316 >> v1:172.21.2.108:6789/0 conn(0x7f17100e5070 legacy=0x7f17101b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:54.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f170f7fe700 1 -- 172.21.2.101:0/1789984316 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17100d19b0 con 0x7f171007f300 2021-02-10T11:13:54.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f170f7fe700 1 -- 172.21.2.101:0/1789984316 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f1708001980 con 0x7f171007f300 2021-02-10T11:13:54.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f170f7fe700 1 -- 172.21.2.101:0/1789984316 >> v1:172.21.2.101:6789/0 conn(0x7f171007f300 legacy=0x7f171007f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:54.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f170f7fe700 1 --2- 172.21.2.101:0/1789984316 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16fc003b90 0x7f16fc006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:54.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f170f7fe700 1 --2- 172.21.2.101:0/1789984316 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f16fc0067b0 0x7f16fc008cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:54.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f170f7fe700 1 --2- 172.21.2.101:0/1789984316 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16fc0095d0 0x7f16fc00bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:54.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f170f7fe700 1 -- 172.21.2.101:0/1789984316 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc00c300 con 0x7f16fc0095d0 2021-02-10T11:13:54.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f170f7fe700 1 -- 172.21.2.101:0/1789984316 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc00c520 con 0x7f16fc0067b0 2021-02-10T11:13:54.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f170f7fe700 1 -- 172.21.2.101:0/1789984316 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc00c740 con 0x7f16fc003b90 2021-02-10T11:13:54.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f17152a7700 1 --2- 172.21.2.101:0/1789984316 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16fc0095d0 0x7f16fc00bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:54.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f170ffff700 1 --2- 172.21.2.101:0/1789984316 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f16fc0067b0 0x7f16fc008cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:54.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f1714aa6700 1 --2- 172.21.2.101:0/1789984316 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16fc003b90 0x7f16fc006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:54.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f17152a7700 1 -- 172.21.2.101:0/1789984316 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16fc003b90 msgr2=0x7f16fc006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:54.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f17152a7700 1 --2- 172.21.2.101:0/1789984316 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16fc003b90 0x7f16fc006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:54.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f17152a7700 1 -- 172.21.2.101:0/1789984316 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f16fc0067b0 msgr2=0x7f16fc008cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:54.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f17152a7700 1 --2- 172.21.2.101:0/1789984316 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f16fc0067b0 0x7f16fc008cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:54.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f17152a7700 1 -- 172.21.2.101:0/1789984316 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc00c9e0 con 0x7f16fc0095d0 2021-02-10T11:13:54.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f17152a7700 1 -- 172.21.2.101:0/1789984316 reap_dead start 2021-02-10T11:13:54.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.729 7f17152a7700 1 --2- 172.21.2.101:0/1789984316 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16fc0095d0 0x7f16fc00bad0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f1708007bc0 tx=0x7f170804e320).ready entity=mon.0 client_cookie=3d385afc88da4495 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:54.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.730 7f170f7fe700 1 -- 172.21.2.101:0/1789984316 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1708007800 con 0x7f16fc0095d0 2021-02-10T11:13:54.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.730 7f170f7fe700 1 -- 172.21.2.101:0/1789984316 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1708001270 con 0x7f16fc0095d0 2021-02-10T11:13:54.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.730 7f170f7fe700 1 -- 172.21.2.101:0/1789984316 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f17080072d0 con 0x7f16fc0095d0 2021-02-10T11:13:54.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.730 7f1716c8a700 1 -- 172.21.2.101:0/1789984316 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16fc0095d0 msgr2=0x7f16fc00bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:54.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.730 7f1716c8a700 1 --2- 172.21.2.101:0/1789984316 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16fc0095d0 0x7f16fc00bad0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f1708007bc0 tx=0x7f170804e320).stop 2021-02-10T11:13:54.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.730 7f1716c8a700 1 -- 172.21.2.101:0/1789984316 shutdown_connections 2021-02-10T11:13:54.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.730 7f1716c8a700 1 --2- 172.21.2.101:0/1789984316 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16fc0095d0 0x7f16fc00bad0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:54.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.730 7f1716c8a700 1 -- 172.21.2.101:0/1789984316 >> 172.21.2.101:0/1789984316 conn(0x7f17100ce690 msgr2=0x7f17100e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:54.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.730 7f1716c8a700 1 -- 172.21.2.101:0/1789984316 shutdown_connections 2021-02-10T11:13:54.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.730 7f1716c8a700 1 -- 172.21.2.101:0/1789984316 wait complete. 2021-02-10T11:13:54.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.745 7f1716c8a700 1 Processor -- start 2021-02-10T11:13:54.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.745 7f1716c8a700 1 -- start start 2021-02-10T11:13:54.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.745 7f1716c8a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f171018d210 0x7f17101ab320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:54.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.745 7f1716c8a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f17100984c0 0x7f1710097290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:54.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.745 7f1716c8a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17100978b0 0x7f1710097cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:54.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.745 7f1716c8a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171009a540 con 0x7f171018d210 2021-02-10T11:13:54.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.745 7f1716c8a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17101aba60 con 0x7f17100984c0 2021-02-10T11:13:54.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.745 7f1716c8a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171009aac0 con 0x7f17100978b0 2021-02-10T11:13:54.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.745 7f1714aa6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f171018d210 0x7f17101ab320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:54.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.745 7f1714aa6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f171018d210 0x7f17101ab320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44708/0 (socket says 172.21.2.101:44708) 2021-02-10T11:13:54.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.745 7f1714aa6700 1 -- 172.21.2.101:0/680882619 learned_addr learned my addr 172.21.2.101:0/680882619 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:54.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.746 7f1714aa6700 1 -- 172.21.2.101:0/680882619 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17100978b0 msgr2=0x7f1710097cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:54.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.746 7f1714aa6700 1 --2- 172.21.2.101:0/680882619 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17100978b0 0x7f1710097cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:54.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.746 7f1714aa6700 1 -- 172.21.2.101:0/680882619 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f17100984c0 msgr2=0x7f1710097290 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:54.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.746 7f1714aa6700 1 --2- 172.21.2.101:0/680882619 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f17100984c0 0x7f1710097290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:54.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.746 7f1714aa6700 1 -- 172.21.2.101:0/680882619 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171009ace0 con 0x7f171018d210 2021-02-10T11:13:54.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.746 7f1714aa6700 1 --2- 172.21.2.101:0/680882619 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f171018d210 0x7f17101ab320 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f16f8006990 tx=0x7f16f800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:54.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.746 7f170d7fa700 1 -- 172.21.2.101:0/680882619 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f16f8006a80 con 0x7f171018d210 2021-02-10T11:13:54.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.746 7f170d7fa700 1 -- 172.21.2.101:0/680882619 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f8002030 con 0x7f171018d210 2021-02-10T11:13:54.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.746 7f170d7fa700 1 -- 172.21.2.101:0/680882619 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f16f8002550 con 0x7f171018d210 2021-02-10T11:13:54.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.746 7f1716c8a700 1 -- 172.21.2.101:0/680882619 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171009ace0 con 0x7f171018d210 2021-02-10T11:13:54.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.746 7f1716c8a700 1 -- 172.21.2.101:0/680882619 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171009b270 con 0x7f171018d210 2021-02-10T11:13:54.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.747 7f170d7fa700 1 -- 172.21.2.101:0/680882619 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f16f8002210 con 0x7f171018d210 2021-02-10T11:13:54.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.747 7f170d7fa700 1 --2- 172.21.2.101:0/680882619 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16f40186c0 0x7f16f401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:54.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.747 7f170d7fa700 1 -- 172.21.2.101:0/680882619 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f16f8032420 con 0x7f171018d210 2021-02-10T11:13:54.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.747 7f170ffff700 1 --2- 172.21.2.101:0/680882619 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16f40186c0 0x7f16f401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:54.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.747 7f170ffff700 1 --2- 172.21.2.101:0/680882619 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16f40186c0 0x7f16f401abc0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:54.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.778 7f1716c8a700 1 -- 172.21.2.101:0/680882619 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1710084b20 con 0x7f171018d210 2021-02-10T11:13:54.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.780 7f170d7fa700 1 -- 172.21.2.101:0/680882619 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f16f8030d50 con 0x7f171018d210 2021-02-10T11:13:54.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.907 7f1716c8a700 1 -- 172.21.2.101:0/680882619 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f171007ed80 con 0x7f171018d210 2021-02-10T11:13:54.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.907 7f170d7fa700 1 -- 172.21.2.101:0/680882619 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2097 (secure 0 0 0) 0x7f16f80021f0 con 0x7f171018d210 2021-02-10T11:13:54.912 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 6 2021-02-10T11:13:54.912 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:13:54.912 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/3184957761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3184957761},{"type":"v1","addr":"172.21.2.108:6833","nonce":3184957761}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:13:14.147021","modified":"2021-02-10 11:13:23.547179","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":5,"state":"up:active","state_seq":3,"addr":"172.21.2.101:6835/3530104431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3530104431},{"type":"v1","addr":"172.21.2.101:6835","nonce":3530104431}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-02-10T11:13:54.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.909 7f1716c8a700 1 -- 172.21.2.101:0/680882619 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16f40186c0 msgr2=0x7f16f401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:54.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.909 7f1716c8a700 1 --2- 172.21.2.101:0/680882619 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16f40186c0 0x7f16f401abc0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:54.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.909 7f1716c8a700 1 -- 172.21.2.101:0/680882619 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f171018d210 msgr2=0x7f17101ab320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:54.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.909 7f1716c8a700 1 --2- 172.21.2.101:0/680882619 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f171018d210 0x7f17101ab320 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f16f8006990 tx=0x7f16f800b990).stop 2021-02-10T11:13:54.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.909 7f1716c8a700 1 -- 172.21.2.101:0/680882619 shutdown_connections 2021-02-10T11:13:54.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.909 7f1716c8a700 1 --2- 172.21.2.101:0/680882619 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16f40186c0 0x7f16f401abc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:54.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.909 7f1716c8a700 1 --2- 172.21.2.101:0/680882619 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17100978b0 0x7f1710097cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:54.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.909 7f1716c8a700 1 --2- 172.21.2.101:0/680882619 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f17100984c0 0x7f1710097290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:54.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.909 7f1716c8a700 1 --2- 172.21.2.101:0/680882619 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f171018d210 0x7f17101ab320 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:54.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.909 7f1716c8a700 1 -- 172.21.2.101:0/680882619 >> 172.21.2.101:0/680882619 conn(0x7f17101b2a40 msgr2=0x7f1710198bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:54.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.909 7f1716c8a700 1 -- 172.21.2.101:0/680882619 shutdown_connections 2021-02-10T11:13:54.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:54.909 7f1716c8a700 1 -- 172.21.2.101:0/680882619 wait complete. 2021-02-10T11:13:55.124 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:13:14.147021', 'modified': '2021-02-10 11:13:23.547179', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4299': {'gid': 4299, 'name': 'a', 'rank': 0, 'incarnation': 5, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.101:6835/3530104431', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.101:6834', 'nonce': 3530104431}, {'type': 'v1', 'addr': '172.21.2.101:6835', 'nonce': 3530104431}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:13:55.124 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:13:55.124 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok status 2021-02-10T11:13:55.523 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:13:55.523 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:13:55.523 INFO:teuthology.orchestra.run.gibba001.stdout: "whoami": 0, 2021-02-10T11:13:55.523 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 4299, 2021-02-10T11:13:55.523 INFO:teuthology.orchestra.run.gibba001.stdout: "want_state": "up:active", 2021-02-10T11:13:55.523 INFO:teuthology.orchestra.run.gibba001.stdout: "state": "up:active", 2021-02-10T11:13:55.524 INFO:teuthology.orchestra.run.gibba001.stdout: "rank_uptime": 32.972162891000004, 2021-02-10T11:13:55.524 INFO:teuthology.orchestra.run.gibba001.stdout: "mdsmap_epoch": 6, 2021-02-10T11:13:55.524 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch": 20, 2021-02-10T11:13:55.524 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch_barrier": 20, 2021-02-10T11:13:55.524 INFO:teuthology.orchestra.run.gibba001.stdout: "uptime": 37.364093902999997 2021-02-10T11:13:55.524 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:13:55.527 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 4299, "want_state": "up:active", "state": "up:active", "rank_uptime": 32.972162891000004, "mdsmap_epoch": 6, "osdmap_epoch": 20, "osdmap_epoch_barrier": 20, "uptime": 37.364093902999997 } 2021-02-10T11:13:55.528 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:13:55.528 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2021-02-10T11:13:55.528 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.2 at ubuntu@gibba008.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.2... 2021-02-10T11:13:55.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:13:55.528 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:13:55.862 DEBUG:teuthology.orchestra.run.gibba008:> sudo modprobe fuse 2021-02-10T11:13:56.234 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:13:56.234 DEBUG:teuthology.orchestra.run.gibba008:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:13:56.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:13:56.579 DEBUG:teuthology.orchestra.run.gibba008:> ls /sys/fs/fuse/connections 2021-02-10T11:13:56.912 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-02-10T11:13:56.912 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.2 /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:13:56.915 DEBUG:teuthology.orchestra.run.gibba008:> sudo modprobe fuse 2021-02-10T11:13:57.257 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:13:57.258 DEBUG:teuthology.orchestra.run.gibba008:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:13:57.323 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.gibba008.stderr:2021-02-10 11:13:57.325 7f910eb75f80 -1 init, newargv = 0x5591789c7110 newargc=9 2021-02-10T11:13:57.323 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.gibba008.stderr:ceph-fuse[37201]: starting ceph client 2021-02-10T11:13:57.328 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.gibba008.stderr:ceph-fuse[37201]: starting fuse 2021-02-10T11:13:57.600 INFO:teuthology.orchestra.run.gibba008.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:13:57.600 INFO:teuthology.orchestra.run.gibba008.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:13:57.601 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:13:57.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:13:57.601 DEBUG:teuthology.orchestra.run.gibba008:> ls /sys/fs/fuse/connections 2021-02-10T11:13:57.934 INFO:teuthology.orchestra.run.gibba008.stdout:43 2021-02-10T11:13:57.934 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2021-02-10T11:13:57.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:13:57.935 DEBUG:teuthology.orchestra.run.gibba008:> sudo python3 -c ' 2021-02-10T11:13:57.935 DEBUG:teuthology.orchestra.run.gibba008:> import glob 2021-02-10T11:13:57.935 DEBUG:teuthology.orchestra.run.gibba008:> import re 2021-02-10T11:13:57.935 DEBUG:teuthology.orchestra.run.gibba008:> import os 2021-02-10T11:13:57.935 DEBUG:teuthology.orchestra.run.gibba008:> import subprocess 2021-02-10T11:13:57.935 DEBUG:teuthology.orchestra.run.gibba008:> 2021-02-10T11:13:57.935 DEBUG:teuthology.orchestra.run.gibba008:> def find_socket(client_name): 2021-02-10T11:13:57.935 DEBUG:teuthology.orchestra.run.gibba008:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2021-02-10T11:13:57.935 DEBUG:teuthology.orchestra.run.gibba008:> files = glob.glob(asok_path) 2021-02-10T11:13:57.935 DEBUG:teuthology.orchestra.run.gibba008:> 2021-02-10T11:13:57.936 DEBUG:teuthology.orchestra.run.gibba008:> # Given a non-glob path, it better be there 2021-02-10T11:13:57.936 DEBUG:teuthology.orchestra.run.gibba008:> if "*" not in asok_path: 2021-02-10T11:13:57.936 DEBUG:teuthology.orchestra.run.gibba008:> assert(len(files) == 1) 2021-02-10T11:13:57.936 DEBUG:teuthology.orchestra.run.gibba008:> return files[0] 2021-02-10T11:13:57.936 DEBUG:teuthology.orchestra.run.gibba008:> 2021-02-10T11:13:57.936 DEBUG:teuthology.orchestra.run.gibba008:> for f in files: 2021-02-10T11:13:57.936 DEBUG:teuthology.orchestra.run.gibba008:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:13:57.936 DEBUG:teuthology.orchestra.run.gibba008:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:13:57.936 DEBUG:teuthology.orchestra.run.gibba008:> return f 2021-02-10T11:13:57.936 DEBUG:teuthology.orchestra.run.gibba008:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:13:57.936 DEBUG:teuthology.orchestra.run.gibba008:> 2021-02-10T11:13:57.936 DEBUG:teuthology.orchestra.run.gibba008:> print(find_socket("client.2")) 2021-02-10T11:13:57.936 DEBUG:teuthology.orchestra.run.gibba008:> ' 2021-02-10T11:13:58.307 INFO:teuthology.orchestra.run.gibba008.stdout:/var/run/ceph/ceph-client.2.37201.asok 2021-02-10T11:13:58.310 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.37201.asok 2021-02-10T11:13:58.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:13:58.310 DEBUG:teuthology.orchestra.run.gibba008:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.37201.asok status 2021-02-10T11:13:58.700 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:13:58.701 INFO:teuthology.orchestra.run.gibba008.stdout: "metadata": { 2021-02-10T11:13:58.701 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:13:58.701 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:13:58.701 INFO:teuthology.orchestra.run.gibba008.stdout: "entity_id": "2", 2021-02-10T11:13:58.701 INFO:teuthology.orchestra.run.gibba008.stdout: "hostname": "gibba008", 2021-02-10T11:13:58.701 INFO:teuthology.orchestra.run.gibba008.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2021-02-10T11:13:58.701 INFO:teuthology.orchestra.run.gibba008.stdout: "pid": "37201", 2021-02-10T11:13:58.701 INFO:teuthology.orchestra.run.gibba008.stdout: "root": "/" 2021-02-10T11:13:58.702 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:13:58.702 INFO:teuthology.orchestra.run.gibba008.stdout: "dentry_count": 0, 2021-02-10T11:13:58.702 INFO:teuthology.orchestra.run.gibba008.stdout: "dentry_pinned_count": 0, 2021-02-10T11:13:58.702 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 4304, 2021-02-10T11:13:58.702 INFO:teuthology.orchestra.run.gibba008.stdout: "inst": { 2021-02-10T11:13:58.702 INFO:teuthology.orchestra.run.gibba008.stdout: "name": { 2021-02-10T11:13:58.702 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "client", 2021-02-10T11:13:58.703 INFO:teuthology.orchestra.run.gibba008.stdout: "num": 4304 2021-02-10T11:13:58.703 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:13:58.703 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": { 2021-02-10T11:13:58.703 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "v1", 2021-02-10T11:13:58.703 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": "172.21.2.108:0", 2021-02-10T11:13:58.703 INFO:teuthology.orchestra.run.gibba008.stdout: "nonce": 1986700013 2021-02-10T11:13:58.703 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:13:58.703 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:13:58.703 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": { 2021-02-10T11:13:58.704 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "v1", 2021-02-10T11:13:58.704 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": "172.21.2.108:0", 2021-02-10T11:13:58.704 INFO:teuthology.orchestra.run.gibba008.stdout: "nonce": 1986700013 2021-02-10T11:13:58.704 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:13:58.704 INFO:teuthology.orchestra.run.gibba008.stdout: "inst_str": "client.4304 172.21.2.108:0/1986700013", 2021-02-10T11:13:58.704 INFO:teuthology.orchestra.run.gibba008.stdout: "addr_str": "172.21.2.108:0/1986700013", 2021-02-10T11:13:58.704 INFO:teuthology.orchestra.run.gibba008.stdout: "inode_count": 1, 2021-02-10T11:13:58.704 INFO:teuthology.orchestra.run.gibba008.stdout: "mds_epoch": 6, 2021-02-10T11:13:58.704 INFO:teuthology.orchestra.run.gibba008.stdout: "osd_epoch": 20, 2021-02-10T11:13:58.704 INFO:teuthology.orchestra.run.gibba008.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:13:58.704 INFO:teuthology.orchestra.run.gibba008.stdout: "blacklisted": false 2021-02-10T11:13:58.705 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:13:58.706 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:13:59.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.110 7fd6609b7700 1 Processor -- start 2021-02-10T11:13:59.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.110 7fd6609b7700 1 -- start start 2021-02-10T11:13:59.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.110 7fd6609b7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd65c0e5070 0x7fd65c1b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:59.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.110 7fd6609b7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd65c1b3e40 0x7fd65c07d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:59.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.110 7fd6609b7700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65c07ed80 con 0x7fd65c0e5470 2021-02-10T11:13:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.110 7fd6609b7700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd65c080310 con 0x7fd65c0e5070 2021-02-10T11:13:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.110 7fd6609b7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd65c080530 con 0x7fd65c1b3e40 2021-02-10T11:13:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.110 7fd65a61c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd65c0e5070 0x7fd65c1b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.110 7fd65a61c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd65c0e5070 0x7fd65c1b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44712/0 (socket says 172.21.2.101:44712) 2021-02-10T11:13:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.110 7fd65a61c700 1 -- 172.21.2.101:0/2966315402 learned_addr learned my addr 172.21.2.101:0/2966315402 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.111 7fd65a61c700 1 -- 172.21.2.101:0/2966315402 >> v2:172.21.2.108:3300/0 conn(0x7fd65c1b3e40 msgr2=0x7fd65c07d690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:13:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.111 7fd65a61c700 1 --2- 172.21.2.101:0/2966315402 >> v2:172.21.2.108:3300/0 conn(0x7fd65c1b3e40 0x7fd65c07d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.111 7fd65a61c700 1 -- 172.21.2.101:0/2966315402 >> v1:172.21.2.108:6789/0 conn(0x7fd65c0e5470 legacy=0x7fd65c07dcb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:13:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.111 7fd65a61c700 1 -- 172.21.2.101:0/2966315402 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c080750 con 0x7fd65c0e5070 2021-02-10T11:13:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.111 7fd65a61c700 1 --2- 172.21.2.101:0/2966315402 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd65c0e5070 0x7fd65c1b3820 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fd64c006700 tx=0x7fd64c009b00).ready entity=mon.0 client_cookie=2d083d6ebbd1feee server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.111 7fd65961a700 1 -- 172.21.2.101:0/2966315402 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd64c00c0c0 con 0x7fd65c0e5070 2021-02-10T11:13:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.111 7fd65961a700 1 -- 172.21.2.101:0/2966315402 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64c00c340 con 0x7fd65c0e5070 2021-02-10T11:13:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.111 7fd65961a700 1 -- 172.21.2.101:0/2966315402 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd64c00c820 con 0x7fd65c0e5070 2021-02-10T11:13:59.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.111 7fd6609b7700 1 -- 172.21.2.101:0/2966315402 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd65c0e5070 msgr2=0x7fd65c1b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:59.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.111 7fd6609b7700 1 --2- 172.21.2.101:0/2966315402 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd65c0e5070 0x7fd65c1b3820 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fd64c006700 tx=0x7fd64c009b00).stop 2021-02-10T11:13:59.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.112 7fd6609b7700 1 -- 172.21.2.101:0/2966315402 shutdown_connections 2021-02-10T11:13:59.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.112 7fd6609b7700 1 --2- 172.21.2.101:0/2966315402 >> v2:172.21.2.108:3300/0 conn(0x7fd65c1b3e40 0x7fd65c07d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:59.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.112 7fd6609b7700 1 --2- 172.21.2.101:0/2966315402 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd65c0e5070 0x7fd65c1b3820 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:59.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.112 7fd6609b7700 1 -- 172.21.2.101:0/2966315402 >> 172.21.2.101:0/2966315402 conn(0x7fd65c0ce690 msgr2=0x7fd65c0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:59.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.112 7fd6609b7700 1 -- 172.21.2.101:0/2966315402 shutdown_connections 2021-02-10T11:13:59.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.112 7fd6609b7700 1 -- 172.21.2.101:0/2966315402 wait complete. 2021-02-10T11:13:59.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.126 7fd6609b7700 1 Processor -- start 2021-02-10T11:13:59.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.126 7fd6609b7700 1 -- start start 2021-02-10T11:13:59.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.127 7fd6609b7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd65c1a2a90 0x7fd65c1aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:59.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.127 7fd6609b7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd65c1aa700 0x7fd65c1a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.127 7fd6609b7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd65c0a55e0 0x7fd65c1a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.127 7fd6609b7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c1a34a0 con 0x7fd65c1a2a90 2021-02-10T11:13:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.127 7fd6609b7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c1a2e90 con 0x7fd65c1aa700 2021-02-10T11:13:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.127 7fd6609b7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c1a30b0 con 0x7fd65c0a55e0 2021-02-10T11:13:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.127 7fd65a61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd65c1a2a90 0x7fd65c1aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.127 7fd65a61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd65c1a2a90 0x7fd65c1aa0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44714/0 (socket says 172.21.2.101:44714) 2021-02-10T11:13:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.127 7fd65a61c700 1 -- 172.21.2.101:0/1082860663 learned_addr learned my addr 172.21.2.101:0/1082860663 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.127 7fd65a61c700 1 -- 172.21.2.101:0/1082860663 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd65c0a55e0 msgr2=0x7fd65c1a4040 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.127 7fd65a61c700 1 --2- 172.21.2.101:0/1082860663 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd65c0a55e0 0x7fd65c1a4040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.127 7fd65a61c700 1 -- 172.21.2.101:0/1082860663 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd65c1aa700 msgr2=0x7fd65c1a3a20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:13:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.127 7fd65a61c700 1 --2- 172.21.2.101:0/1082860663 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd65c1aa700 0x7fd65c1a3a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.127 7fd65a61c700 1 -- 172.21.2.101:0/1082860663 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c1a4660 con 0x7fd65c1a2a90 2021-02-10T11:13:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.127 7fd65a61c700 1 --2- 172.21.2.101:0/1082860663 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd65c1a2a90 0x7fd65c1aa0e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fd64c006f20 tx=0x7fd64c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:59.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.128 7fd652ffd700 1 -- 172.21.2.101:0/1082860663 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd64c000c90 con 0x7fd65c1a2a90 2021-02-10T11:13:59.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.128 7fd652ffd700 1 -- 172.21.2.101:0/1082860663 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64c009100 con 0x7fd65c1a2a90 2021-02-10T11:13:59.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.128 7fd6609b7700 1 -- 172.21.2.101:0/1082860663 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65c1a4660 con 0x7fd65c1a2a90 2021-02-10T11:13:59.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.128 7fd6609b7700 1 -- 172.21.2.101:0/1082860663 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65c1a4bf0 con 0x7fd65c1a2a90 2021-02-10T11:13:59.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.128 7fd652ffd700 1 -- 172.21.2.101:0/1082860663 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd64c002220 con 0x7fd65c1a2a90 2021-02-10T11:13:59.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.128 7fd652ffd700 1 -- 172.21.2.101:0/1082860663 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd64c000c90 con 0x7fd65c1a2a90 2021-02-10T11:13:59.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.129 7fd652ffd700 1 --2- 172.21.2.101:0/1082860663 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd640018730 0x7fd64001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:59.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.129 7fd652ffd700 1 -- 172.21.2.101:0/1082860663 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd64c032810 con 0x7fd65c1a2a90 2021-02-10T11:13:59.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.129 7fd659e1b700 1 --2- 172.21.2.101:0/1082860663 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd640018730 0x7fd64001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:59.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.129 7fd659e1b700 1 --2- 172.21.2.101:0/1082860663 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd640018730 0x7fd64001ac10 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:59.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.160 7fd6609b7700 1 -- 172.21.2.101:0/1082860663 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd65c083da0 con 0x7fd65c1a2a90 2021-02-10T11:13:59.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.163 7fd652ffd700 1 -- 172.21.2.101:0/1082860663 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd64c009100 con 0x7fd65c1a2a90 2021-02-10T11:13:59.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.287 7fd6609b7700 1 -- 172.21.2.101:0/1082860663 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd65c07ed80 con 0x7fd65c1a2a90 2021-02-10T11:13:59.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.287 7fd652ffd700 1 -- 172.21.2.101:0/1082860663 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+13993 (secure 0 0 0) 0x7fd64c000c90 con 0x7fd65c1a2a90 2021-02-10T11:13:59.292 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:13:59.292 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":20,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:13:15.149234","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:13:11.922272","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2021-02-10 11:13:13.141733","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:13:59.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.289 7fd6609b7700 1 -- 172.21.2.101:0/1082860663 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd640018730 msgr2=0x7fd64001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:59.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.289 7fd6609b7700 1 --2- 172.21.2.101:0/1082860663 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd640018730 0x7fd64001ac10 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:59.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.289 7fd6609b7700 1 -- 172.21.2.101:0/1082860663 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd65c1a2a90 msgr2=0x7fd65c1aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:59.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.289 7fd6609b7700 1 --2- 172.21.2.101:0/1082860663 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd65c1a2a90 0x7fd65c1aa0e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fd64c006f20 tx=0x7fd64c00b990).stop 2021-02-10T11:13:59.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.289 7fd6609b7700 1 -- 172.21.2.101:0/1082860663 shutdown_connections 2021-02-10T11:13:59.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.289 7fd6609b7700 1 --2- 172.21.2.101:0/1082860663 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd640018730 0x7fd64001ac10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:59.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.289 7fd6609b7700 1 --2- 172.21.2.101:0/1082860663 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd65c0a55e0 0x7fd65c1a4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:59.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.289 7fd6609b7700 1 --2- 172.21.2.101:0/1082860663 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd65c1aa700 0x7fd65c1a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:13:59.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.289 7fd6609b7700 1 --2- 172.21.2.101:0/1082860663 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd65c1a2a90 0x7fd65c1aa0e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:59.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.289 7fd6609b7700 1 -- 172.21.2.101:0/1082860663 >> 172.21.2.101:0/1082860663 conn(0x7fd65c08fe10 msgr2=0x7fd65c19c020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:59.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.289 7fd6609b7700 1 -- 172.21.2.101:0/1082860663 shutdown_connections 2021-02-10T11:13:59.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.289 7fd6609b7700 1 -- 172.21.2.101:0/1082860663 wait complete. 2021-02-10T11:13:59.301 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:13:11.922272', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:13:13.141733', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:13:59.302 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:13:59.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.709 7f9da6930700 1 Processor -- start 2021-02-10T11:13:59.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.709 7f9da6930700 1 -- start start 2021-02-10T11:13:59.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.709 7f9da6930700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9da01b0470 0x7f9da01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:59.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.710 7f9da6930700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da00f0b50 con 0x7f9da00f20e0 2021-02-10T11:13:59.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.710 7f9da6930700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da00f24e0 con 0x7f9da00ef610 2021-02-10T11:13:59.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.710 7f9da6930700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da00e5ea0 con 0x7f9da01b0470 2021-02-10T11:13:59.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.710 7f9da4f4d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f9da00f20e0 0x7f9da00f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45218/0 (socket says 172.21.2.101:45218) 2021-02-10T11:13:59.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.710 7f9da4f4d700 1 -- 172.21.2.101:0/2868413979 learned_addr learned my addr 172.21.2.101:0/2868413979 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:13:59.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.710 7f9d9effd700 1 -- 172.21.2.101:0/2868413979 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3712790717 0 0) 0x7f9d98001500 con 0x7f9da00f20e0 2021-02-10T11:13:59.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.710 7f9d9effd700 1 -- 172.21.2.101:0/2868413979 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d88002fe0 con 0x7f9da00f20e0 2021-02-10T11:13:59.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.710 7f9d9ffff700 1 --2- 172.21.2.101:0/2868413979 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9da01b0470 0x7f9da01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:59.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.710 7f9d9effd700 1 -- 172.21.2.101:0/2868413979 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1124699183 0 0) 0x7f9d98001500 con 0x7f9da00f20e0 2021-02-10T11:13:59.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.710 7f9d9effd700 1 -- 172.21.2.101:0/2868413979 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9da01b0470 msgr2=0x7f9da01b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:59.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.710 7f9d9effd700 1 --2- 172.21.2.101:0/2868413979 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9da01b0470 0x7f9da01b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:59.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.710 7f9d9effd700 1 -- 172.21.2.101:0/2868413979 >> v1:172.21.2.108:6789/0 conn(0x7f9da00ef610 legacy=0x7f9da00efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:59.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.710 7f9d9effd700 1 -- 172.21.2.101:0/2868413979 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da00e52e0 con 0x7f9da00f20e0 2021-02-10T11:13:59.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.710 7f9d9effd700 1 -- 172.21.2.101:0/2868413979 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f9d98001a00 con 0x7f9da00f20e0 2021-02-10T11:13:59.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.710 7f9d9effd700 1 -- 172.21.2.101:0/2868413979 >> v1:172.21.2.101:6789/0 conn(0x7f9da00f20e0 legacy=0x7f9da00f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:59.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.710 7f9d9effd700 1 --2- 172.21.2.101:0/2868413979 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d88003aa0 0x7f9d88005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:59.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.710 7f9d9effd700 1 --2- 172.21.2.101:0/2868413979 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d88006770 0x7f9d88008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:59.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.710 7f9d9effd700 1 --2- 172.21.2.101:0/2868413979 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d88009590 0x7f9d8800ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:59.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.710 7f9d9effd700 1 -- 172.21.2.101:0/2868413979 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8800c2d0 con 0x7f9d88003aa0 2021-02-10T11:13:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.710 7f9d9effd700 1 -- 172.21.2.101:0/2868413979 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8800c4f0 con 0x7f9d88006770 2021-02-10T11:13:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.711 7f9d9effd700 1 -- 172.21.2.101:0/2868413979 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d88050720 con 0x7f9d88009590 2021-02-10T11:13:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.711 7f9d9ffff700 1 --2- 172.21.2.101:0/2868413979 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d88003aa0 0x7f9d88005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.711 7f9da4f4d700 1 --2- 172.21.2.101:0/2868413979 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d88009590 0x7f9d8800ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.711 7f9d9f7fe700 1 --2- 172.21.2.101:0/2868413979 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d88006770 0x7f9d88008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.711 7f9d9ffff700 1 -- 172.21.2.101:0/2868413979 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d88009590 msgr2=0x7f9d8800ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.711 7f9d9ffff700 1 --2- 172.21.2.101:0/2868413979 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d88009590 0x7f9d8800ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.711 7f9d9ffff700 1 -- 172.21.2.101:0/2868413979 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d88006770 msgr2=0x7f9d88008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:13:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.711 7f9d9ffff700 1 --2- 172.21.2.101:0/2868413979 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d88006770 0x7f9d88008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.711 7f9d9ffff700 1 -- 172.21.2.101:0/2868413979 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d88050950 con 0x7f9d88003aa0 2021-02-10T11:13:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.711 7f9d9ffff700 1 --2- 172.21.2.101:0/2868413979 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d88003aa0 0x7f9d88005ef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f9d90006ac0 tx=0x7f9d90009ab0).ready entity=mon.0 client_cookie=a502dc37a986c4fe server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.711 7f9d9effd700 1 -- 172.21.2.101:0/2868413979 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d9000c0c0 con 0x7f9d88003aa0 2021-02-10T11:13:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.711 7f9d9effd700 1 -- 172.21.2.101:0/2868413979 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9000c3c0 con 0x7f9d88003aa0 2021-02-10T11:13:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.711 7f9d9effd700 1 -- 172.21.2.101:0/2868413979 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d9000c8a0 con 0x7f9d88003aa0 2021-02-10T11:13:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.711 7f9da6930700 1 -- 172.21.2.101:0/2868413979 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d88003aa0 msgr2=0x7f9d88005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.711 7f9da6930700 1 --2- 172.21.2.101:0/2868413979 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d88003aa0 0x7f9d88005ef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f9d90006ac0 tx=0x7f9d90009ab0).stop 2021-02-10T11:13:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.711 7f9da4f4d700 1 -- 172.21.2.101:0/2868413979 reap_dead start 2021-02-10T11:13:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.712 7f9da6930700 1 -- 172.21.2.101:0/2868413979 shutdown_connections 2021-02-10T11:13:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.712 7f9da6930700 1 -- 172.21.2.101:0/2868413979 >> 172.21.2.101:0/2868413979 conn(0x7f9da00ce3e0 msgr2=0x7f9da00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.712 7f9da6930700 1 -- 172.21.2.101:0/2868413979 shutdown_connections 2021-02-10T11:13:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.712 7f9da6930700 1 -- 172.21.2.101:0/2868413979 wait complete. 2021-02-10T11:13:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.726 7f9da6930700 1 Processor -- start 2021-02-10T11:13:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.727 7f9da6930700 1 -- start start 2021-02-10T11:13:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.727 7f9da6930700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9da00b7db0 0x7f9da00b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.727 7f9da6930700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9da00b8880 0x7f9da01a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.727 7f9da6930700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9da00b8c80 0x7f9da01a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.727 7f9da6930700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da00b79b0 con 0x7f9da00b8c80 2021-02-10T11:13:59.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.727 7f9da6930700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da01a8090 con 0x7f9da00b7db0 2021-02-10T11:13:59.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.727 7f9da6930700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da01a82b0 con 0x7f9da00b8880 2021-02-10T11:13:59.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.727 7f9da4f4d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9da00b8c80 0x7f9da01a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:59.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.727 7f9da4f4d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9da00b8c80 0x7f9da01a7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44738/0 (socket says 172.21.2.101:44738) 2021-02-10T11:13:59.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.727 7f9da4f4d700 1 -- 172.21.2.101:0/2652097319 learned_addr learned my addr 172.21.2.101:0/2652097319 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:13:59.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.727 7f9d9ffff700 1 --2- 172.21.2.101:0/2652097319 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9da00b7db0 0x7f9da00b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:59.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.727 7f9d9f7fe700 1 --2- 172.21.2.101:0/2652097319 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9da00b8880 0x7f9da01a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:59.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.728 7f9da4f4d700 1 -- 172.21.2.101:0/2652097319 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9da00b8880 msgr2=0x7f9da01a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:59.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.728 7f9da4f4d700 1 --2- 172.21.2.101:0/2652097319 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9da00b8880 0x7f9da01a7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:59.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.728 7f9da4f4d700 1 -- 172.21.2.101:0/2652097319 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9da00b7db0 msgr2=0x7f9da00b8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:59.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.728 7f9da4f4d700 1 --2- 172.21.2.101:0/2652097319 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9da00b7db0 0x7f9da00b8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:59.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.728 7f9da4f4d700 1 -- 172.21.2.101:0/2652097319 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da0190c10 con 0x7f9da00b8c80 2021-02-10T11:13:59.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.728 7f9d9ffff700 1 --2- 172.21.2.101:0/2652097319 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9da00b7db0 0x7f9da00b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:13:59.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.728 7f9da4f4d700 1 --2- 172.21.2.101:0/2652097319 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9da00b8c80 0x7f9da01a7a70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f9d98001ff0 tx=0x7f9d9800ac50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:59.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.728 7f9d9cff9700 1 -- 172.21.2.101:0/2652097319 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d980017f0 con 0x7f9da00b8c80 2021-02-10T11:13:59.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.728 7f9d9cff9700 1 -- 172.21.2.101:0/2652097319 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d98004930 con 0x7f9da00b8c80 2021-02-10T11:13:59.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.728 7f9d9cff9700 1 -- 172.21.2.101:0/2652097319 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d98004d80 con 0x7f9da00b8c80 2021-02-10T11:13:59.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.728 7f9da6930700 1 -- 172.21.2.101:0/2652097319 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da0190c10 con 0x7f9da00b8c80 2021-02-10T11:13:59.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.728 7f9da6930700 1 -- 172.21.2.101:0/2652097319 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da00c1eb0 con 0x7f9da00b8c80 2021-02-10T11:13:59.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.729 7f9d9cff9700 1 -- 172.21.2.101:0/2652097319 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9d98001470 con 0x7f9da00b8c80 2021-02-10T11:13:59.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.729 7f9d9cff9700 1 --2- 172.21.2.101:0/2652097319 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d800186c0 0x7f9d8001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:13:59.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.729 7f9d9cff9700 1 -- 172.21.2.101:0/2652097319 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9d98032ae0 con 0x7f9da00b8c80 2021-02-10T11:13:59.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.729 7f9d9ffff700 1 --2- 172.21.2.101:0/2652097319 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d800186c0 0x7f9d8001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:13:59.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.729 7f9d9ffff700 1 --2- 172.21.2.101:0/2652097319 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d800186c0 0x7f9d8001abc0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:13:59.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.759 7f9da6930700 1 -- 172.21.2.101:0/2652097319 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da0105b30 con 0x7f9da00b8c80 2021-02-10T11:13:59.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.762 7f9d9cff9700 1 -- 172.21.2.101:0/2652097319 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9d98004b90 con 0x7f9da00b8c80 2021-02-10T11:13:59.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.889 7f9da6930700 1 -- 172.21.2.101:0/2652097319 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f9da00f0b50 con 0x7f9da00b8c80 2021-02-10T11:13:59.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.890 7f9d9cff9700 1 -- 172.21.2.101:0/2652097319 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9d98004b40 con 0x7f9da00b8c80 2021-02-10T11:13:59.894 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:13:59.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.892 7f9da6930700 1 -- 172.21.2.101:0/2652097319 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d800186c0 msgr2=0x7f9d8001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:59.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.892 7f9da6930700 1 --2- 172.21.2.101:0/2652097319 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d800186c0 0x7f9d8001abc0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:59.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.892 7f9da6930700 1 -- 172.21.2.101:0/2652097319 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9da00b8c80 msgr2=0x7f9da01a7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:13:59.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.892 7f9da6930700 1 --2- 172.21.2.101:0/2652097319 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9da00b8c80 0x7f9da01a7a70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f9d98001ff0 tx=0x7f9d9800ac50).stop 2021-02-10T11:13:59.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.892 7f9da6930700 1 -- 172.21.2.101:0/2652097319 shutdown_connections 2021-02-10T11:13:59.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.892 7f9da6930700 1 --2- 172.21.2.101:0/2652097319 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d800186c0 0x7f9d8001abc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:59.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.892 7f9da6930700 1 --2- 172.21.2.101:0/2652097319 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9da00b8880 0x7f9da01a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:59.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.892 7f9da6930700 1 --2- 172.21.2.101:0/2652097319 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9da00b8c80 0x7f9da01a7a70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:59.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.892 7f9da6930700 1 --2- 172.21.2.101:0/2652097319 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9da00b7db0 0x7f9da00b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:13:59.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.892 7f9da6930700 1 -- 172.21.2.101:0/2652097319 >> 172.21.2.101:0/2652097319 conn(0x7f9da0094e10 msgr2=0x7f9da00a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:13:59.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.892 7f9da6930700 1 -- 172.21.2.101:0/2652097319 shutdown_connections 2021-02-10T11:13:59.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:13:59.892 7f9da6930700 1 -- 172.21.2.101:0/2652097319 wait complete. 2021-02-10T11:13:59.904 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:14:00.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.312 7f30bf313700 1 Processor -- start 2021-02-10T11:14:00.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.312 7f30bf313700 1 -- start start 2021-02-10T11:14:00.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.313 7f30bf313700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f30b80f05f0 0x7f30b80f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:00.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.313 7f30bf313700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30b81b0c70 con 0x7f30b81b0470 2021-02-10T11:14:00.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.313 7f30bf313700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30b80f0b50 con 0x7f30b81b0870 2021-02-10T11:14:00.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.313 7f30bf313700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b80e5ea0 con 0x7f30b80f05f0 2021-02-10T11:14:00.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.313 7f30bd930700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f30b80f05f0 0x7f30b80f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:00.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.313 7f30bd930700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f30b80f05f0 0x7f30b80f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44746/0 (socket says 172.21.2.101:44746) 2021-02-10T11:14:00.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.313 7f30bd930700 1 -- 172.21.2.101:0/817359080 learned_addr learned my addr 172.21.2.101:0/817359080 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:00.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.313 7f30bd12f700 1 --1- 172.21.2.101:0/817359080 >> v1:172.21.2.101:6789/0 conn(0x7f30b81b0470 0x7f30b80ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45236/0 (socket says 172.21.2.101:45236) 2021-02-10T11:14:00.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.313 7f30bd930700 1 -- 172.21.2.101:0/817359080 >> v1:172.21.2.108:6789/0 conn(0x7f30b81b0870 legacy=0x7f30b80efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:00.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.313 7f30bd930700 1 -- 172.21.2.101:0/817359080 >> v1:172.21.2.101:6789/0 conn(0x7f30b81b0470 legacy=0x7f30b80ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:00.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.313 7f30bd930700 1 -- 172.21.2.101:0/817359080 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b80e52e0 con 0x7f30b80f05f0 2021-02-10T11:14:00.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.313 7f30bd930700 1 --2- 172.21.2.101:0/817359080 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30b80f05f0 0x7f30b80f20e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f30b0006f00 tx=0x7f30b0009f10).ready entity=mon.0 client_cookie=190f15bd767c9ca2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:00.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.313 7f30abfff700 1 -- 172.21.2.101:0/817359080 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f30b000b3b0 con 0x7f30b80f05f0 2021-02-10T11:14:00.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.314 7f30abfff700 1 -- 172.21.2.101:0/817359080 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b0000d30 con 0x7f30b80f05f0 2021-02-10T11:14:00.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.314 7f30abfff700 1 -- 172.21.2.101:0/817359080 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f30b000b8b0 con 0x7f30b80f05f0 2021-02-10T11:14:00.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.314 7f30bf313700 1 -- 172.21.2.101:0/817359080 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30b80f05f0 msgr2=0x7f30b80f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:00.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.314 7f30bf313700 1 --2- 172.21.2.101:0/817359080 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30b80f05f0 0x7f30b80f20e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f30b0006f00 tx=0x7f30b0009f10).stop 2021-02-10T11:14:00.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.314 7f30bf313700 1 -- 172.21.2.101:0/817359080 shutdown_connections 2021-02-10T11:14:00.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.314 7f30bf313700 1 --2- 172.21.2.101:0/817359080 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30b80f05f0 0x7f30b80f20e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:00.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.314 7f30bf313700 1 -- 172.21.2.101:0/817359080 >> 172.21.2.101:0/817359080 conn(0x7f30b80ce3e0 msgr2=0x7f30b80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:00.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.314 7f30bf313700 1 -- 172.21.2.101:0/817359080 shutdown_connections 2021-02-10T11:14:00.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.314 7f30bf313700 1 -- 172.21.2.101:0/817359080 wait complete. 2021-02-10T11:14:00.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.329 7f30bf313700 1 Processor -- start 2021-02-10T11:14:00.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.329 7f30bf313700 1 -- start start 2021-02-10T11:14:00.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.329 7f30bf313700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f30b80a3b50 0x7f30b81b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:00.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.329 7f30bf313700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30b81c25b0 0x7f30b81b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:00.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.329 7f30bf313700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30b81b4350 0x7f30b81b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:00.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.329 7f30bf313700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b81b3190 con 0x7f30b81b4350 2021-02-10T11:14:00.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.329 7f30bf313700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b8087750 con 0x7f30b80a3b50 2021-02-10T11:14:00.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.329 7f30bf313700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b80a6bf0 con 0x7f30b81c25b0 2021-02-10T11:14:00.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.330 7f30bd930700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30b81b4350 0x7f30b81b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:00.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.330 7f30bd930700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30b81b4350 0x7f30b81b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44748/0 (socket says 172.21.2.101:44748) 2021-02-10T11:14:00.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.330 7f30bd930700 1 -- 172.21.2.101:0/2501630780 learned_addr learned my addr 172.21.2.101:0/2501630780 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:00.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.330 7f30bd12f700 1 --2- 172.21.2.101:0/2501630780 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f30b80a3b50 0x7f30b81b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:00.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.330 7f30bc92e700 1 --2- 172.21.2.101:0/2501630780 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30b81c25b0 0x7f30b81b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:00.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.330 7f30bd930700 1 -- 172.21.2.101:0/2501630780 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30b81c25b0 msgr2=0x7f30b81b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:00.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.330 7f30bd930700 1 --2- 172.21.2.101:0/2501630780 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30b81c25b0 0x7f30b81b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:00.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.330 7f30bd930700 1 -- 172.21.2.101:0/2501630780 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f30b80a3b50 msgr2=0x7f30b81b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:00.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.330 7f30bd930700 1 --2- 172.21.2.101:0/2501630780 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f30b80a3b50 0x7f30b81b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:00.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.330 7f30bd930700 1 -- 172.21.2.101:0/2501630780 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b81b1ec0 con 0x7f30b81b4350 2021-02-10T11:14:00.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.330 7f30bc92e700 1 --2- 172.21.2.101:0/2501630780 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30b81c25b0 0x7f30b81b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:14:00.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.330 7f30bd930700 1 --2- 172.21.2.101:0/2501630780 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30b81b4350 0x7f30b81b18a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f30b000b990 tx=0x7f30b000bd30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:00.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.330 7f30a9ffb700 1 -- 172.21.2.101:0/2501630780 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f30b0000d30 con 0x7f30b81b4350 2021-02-10T11:14:00.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.330 7f30a9ffb700 1 -- 172.21.2.101:0/2501630780 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b0001480 con 0x7f30b81b4350 2021-02-10T11:14:00.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.330 7f30a9ffb700 1 -- 172.21.2.101:0/2501630780 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f30b00018d0 con 0x7f30b81b4350 2021-02-10T11:14:00.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.330 7f30bf313700 1 -- 172.21.2.101:0/2501630780 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b81b1ec0 con 0x7f30b81b4350 2021-02-10T11:14:00.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.330 7f30bf313700 1 -- 172.21.2.101:0/2501630780 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b81b2450 con 0x7f30b81b4350 2021-02-10T11:14:00.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.331 7f30a9ffb700 1 -- 172.21.2.101:0/2501630780 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f30b0001660 con 0x7f30b81b4350 2021-02-10T11:14:00.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.331 7f30a9ffb700 1 --2- 172.21.2.101:0/2501630780 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30980186d0 0x7f309801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:00.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.331 7f30a9ffb700 1 -- 172.21.2.101:0/2501630780 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f30b0032b70 con 0x7f30b81b4350 2021-02-10T11:14:00.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.331 7f30bd12f700 1 --2- 172.21.2.101:0/2501630780 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30980186d0 0x7f309801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:00.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.331 7f30bd12f700 1 --2- 172.21.2.101:0/2501630780 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30980186d0 0x7f309801abd0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:00.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.362 7f30bf313700 1 -- 172.21.2.101:0/2501630780 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b807e6b0 con 0x7f30b81b4350 2021-02-10T11:14:00.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.364 7f30a9ffb700 1 -- 172.21.2.101:0/2501630780 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f30b0003060 con 0x7f30b81b4350 2021-02-10T11:14:00.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.493 7f30bf313700 1 -- 172.21.2.101:0/2501630780 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f30b80ce3e0 con 0x7f30b81b4350 2021-02-10T11:14:00.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.494 7f30a9ffb700 1 -- 172.21.2.101:0/2501630780 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v20) v1 ==== 104+0+10 (secure 0 0 0) 0x7f30b0001480 con 0x7f30b81b4350 2021-02-10T11:14:00.499 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:14:00.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.496 7f30bf313700 1 -- 172.21.2.101:0/2501630780 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30980186d0 msgr2=0x7f309801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:00.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.496 7f30bf313700 1 --2- 172.21.2.101:0/2501630780 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30980186d0 0x7f309801abd0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:00.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.496 7f30bf313700 1 -- 172.21.2.101:0/2501630780 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30b81b4350 msgr2=0x7f30b81b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:00.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.496 7f30bf313700 1 --2- 172.21.2.101:0/2501630780 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30b81b4350 0x7f30b81b18a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f30b000b990 tx=0x7f30b000bd30).stop 2021-02-10T11:14:00.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.496 7f30bf313700 1 -- 172.21.2.101:0/2501630780 shutdown_connections 2021-02-10T11:14:00.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.496 7f30bf313700 1 --2- 172.21.2.101:0/2501630780 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30980186d0 0x7f309801abd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:00.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.496 7f30bf313700 1 --2- 172.21.2.101:0/2501630780 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30b81c25b0 0x7f30b81b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:00.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.496 7f30bf313700 1 --2- 172.21.2.101:0/2501630780 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30b81b4350 0x7f30b81b18a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:00.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.496 7f30bf313700 1 --2- 172.21.2.101:0/2501630780 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f30b80a3b50 0x7f30b81b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:00.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.496 7f30bf313700 1 -- 172.21.2.101:0/2501630780 >> 172.21.2.101:0/2501630780 conn(0x7f30b80886d0 msgr2=0x7f30b8081c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:00.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.496 7f30bf313700 1 -- 172.21.2.101:0/2501630780 shutdown_connections 2021-02-10T11:14:00.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.496 7f30bf313700 1 -- 172.21.2.101:0/2501630780 wait complete. 2021-02-10T11:14:00.508 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:14:00.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.915 7f289910d700 1 Processor -- start 2021-02-10T11:14:00.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.915 7f289910d700 1 -- start start 2021-02-10T11:14:00.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.916 7f289910d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28941b0870 0x7f28940f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:00.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.916 7f289910d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28941b0c70 con 0x7f28940efe00 2021-02-10T11:14:00.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.916 7f289910d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28940f0b50 con 0x7f28941b0470 2021-02-10T11:14:00.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.916 7f289910d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28940e5ea0 con 0x7f28941b0870 2021-02-10T11:14:00.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.916 7f289359e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f28940efe00 0x7f28940f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45254/0 (socket says 172.21.2.101:45254) 2021-02-10T11:14:00.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.916 7f289359e700 1 -- 172.21.2.101:0/2587840041 learned_addr learned my addr 172.21.2.101:0/2587840041 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:14:00.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f2891d9b700 1 -- 172.21.2.101:0/2587840041 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020577497 0 0) 0x7f288c001500 con 0x7f28940efe00 2021-02-10T11:14:00.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f2891d9b700 1 -- 172.21.2.101:0/2587840041 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2880002fe0 con 0x7f28940efe00 2021-02-10T11:14:00.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f289259c700 1 --2- 172.21.2.101:0/2587840041 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28941b0870 0x7f28940f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:00.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f2891d9b700 1 -- 172.21.2.101:0/2587840041 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 74430090 0 0) 0x7f288c001500 con 0x7f28940efe00 2021-02-10T11:14:00.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f2891d9b700 1 -- 172.21.2.101:0/2587840041 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28941b0870 msgr2=0x7f28940f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:00.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f2891d9b700 1 --2- 172.21.2.101:0/2587840041 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28941b0870 0x7f28940f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:00.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f2891d9b700 1 -- 172.21.2.101:0/2587840041 >> v1:172.21.2.108:6789/0 conn(0x7f28941b0470 legacy=0x7f28940ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:00.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f2891d9b700 1 -- 172.21.2.101:0/2587840041 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28940e52e0 con 0x7f28940efe00 2021-02-10T11:14:00.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f2891d9b700 1 -- 172.21.2.101:0/2587840041 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f288c001a00 con 0x7f28940efe00 2021-02-10T11:14:00.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f2891d9b700 1 -- 172.21.2.101:0/2587840041 >> v1:172.21.2.101:6789/0 conn(0x7f28940efe00 legacy=0x7f28940f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:00.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f2891d9b700 1 --2- 172.21.2.101:0/2587840041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2880003af0 0x7f2880005f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:00.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f2891d9b700 1 --2- 172.21.2.101:0/2587840041 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28800067a0 0x7f2880008c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:00.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f2891d9b700 1 --2- 172.21.2.101:0/2587840041 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f28800094f0 0x7f288000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:00.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f2891d9b700 1 -- 172.21.2.101:0/2587840041 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288000c2c0 con 0x7f2880003af0 2021-02-10T11:14:00.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f2891d9b700 1 -- 172.21.2.101:0/2587840041 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288000c4e0 con 0x7f28800094f0 2021-02-10T11:14:00.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f2891d9b700 1 -- 172.21.2.101:0/2587840041 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f288000c700 con 0x7f28800067a0 2021-02-10T11:14:00.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f2892d9d700 1 --2- 172.21.2.101:0/2587840041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2880003af0 0x7f2880005f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:00.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f289259c700 1 --2- 172.21.2.101:0/2587840041 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28800067a0 0x7f2880008c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:00.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f289359e700 1 --2- 172.21.2.101:0/2587840041 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f28800094f0 0x7f288000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:00.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f2892d9d700 1 -- 172.21.2.101:0/2587840041 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28800067a0 msgr2=0x7f2880008c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:00.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f2892d9d700 1 --2- 172.21.2.101:0/2587840041 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28800067a0 0x7f2880008c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:00.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f2892d9d700 1 -- 172.21.2.101:0/2587840041 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f28800094f0 msgr2=0x7f288000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:00.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f2892d9d700 1 --2- 172.21.2.101:0/2587840041 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f28800094f0 0x7f288000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:00.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f2892d9d700 1 -- 172.21.2.101:0/2587840041 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2880050970 con 0x7f2880003af0 2021-02-10T11:14:00.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.917 7f289359e700 1 --2- 172.21.2.101:0/2587840041 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f28800094f0 0x7f288000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:14:00.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.918 7f2892d9d700 1 --2- 172.21.2.101:0/2587840041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2880003af0 0x7f2880005f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f287c006ab0 tx=0x7f287c009ac0).ready entity=mon.0 client_cookie=82871ff9e06e3c0d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:00.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.918 7f2891d9b700 1 -- 172.21.2.101:0/2587840041 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f287c0066d0 con 0x7f2880003af0 2021-02-10T11:14:00.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.918 7f2891d9b700 1 -- 172.21.2.101:0/2587840041 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287c0022e0 con 0x7f2880003af0 2021-02-10T11:14:00.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.918 7f2891d9b700 1 -- 172.21.2.101:0/2587840041 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f287c0027c0 con 0x7f2880003af0 2021-02-10T11:14:00.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.918 7f289910d700 1 -- 172.21.2.101:0/2587840041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2880003af0 msgr2=0x7f2880005f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:00.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.918 7f289910d700 1 --2- 172.21.2.101:0/2587840041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2880003af0 0x7f2880005f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f287c006ab0 tx=0x7f287c009ac0).stop 2021-02-10T11:14:00.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.918 7f289359e700 1 -- 172.21.2.101:0/2587840041 reap_dead start 2021-02-10T11:14:00.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.918 7f289910d700 1 -- 172.21.2.101:0/2587840041 shutdown_connections 2021-02-10T11:14:00.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.918 7f289910d700 1 -- 172.21.2.101:0/2587840041 >> 172.21.2.101:0/2587840041 conn(0x7f28940ce3e0 msgr2=0x7f28940e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:00.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.918 7f289910d700 1 -- 172.21.2.101:0/2587840041 shutdown_connections 2021-02-10T11:14:00.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.919 7f289910d700 1 -- 172.21.2.101:0/2587840041 wait complete. 2021-02-10T11:14:00.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.933 7f289910d700 1 Processor -- start 2021-02-10T11:14:00.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.933 7f289910d700 1 -- start start 2021-02-10T11:14:00.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.933 7f289910d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f289409b930 0x7f28940b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:00.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.933 7f289910d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28940b8410 0x7f28940b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:00.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.933 7f289910d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28940c1f80 0x7f28940c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:00.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.933 7f289910d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28940c1a00 con 0x7f28940b8410 2021-02-10T11:14:00.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.933 7f289910d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28940b8e30 con 0x7f289409b930 2021-02-10T11:14:00.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.933 7f289910d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28940b9050 con 0x7f28940c1f80 2021-02-10T11:14:00.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.934 7f289259c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28940b8410 0x7f28940b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.934 7f289259c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28940b8410 0x7f28940b8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44770/0 (socket says 172.21.2.101:44770) 2021-02-10T11:14:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.934 7f289259c700 1 -- 172.21.2.101:0/3082457364 learned_addr learned my addr 172.21.2.101:0/3082457364 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.934 7f289259c700 1 -- 172.21.2.101:0/3082457364 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28940c1f80 msgr2=0x7f28940c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.934 7f289259c700 1 --2- 172.21.2.101:0/3082457364 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28940c1f80 0x7f28940c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.934 7f289259c700 1 -- 172.21.2.101:0/3082457364 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f289409b930 msgr2=0x7f28940b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.934 7f289259c700 1 --2- 172.21.2.101:0/3082457364 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f289409b930 0x7f28940b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.934 7f289259c700 1 -- 172.21.2.101:0/3082457364 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28940c29a0 con 0x7f28940b8410 2021-02-10T11:14:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.934 7f289259c700 1 --2- 172.21.2.101:0/3082457364 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28940b8410 0x7f28940b8810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f2884006f60 tx=0x7f288400b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.934 7f288b7fe700 1 -- 172.21.2.101:0/3082457364 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f288400bd60 con 0x7f28940b8410 2021-02-10T11:14:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.934 7f288b7fe700 1 -- 172.21.2.101:0/3082457364 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2884000c90 con 0x7f28940b8410 2021-02-10T11:14:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.934 7f288b7fe700 1 -- 172.21.2.101:0/3082457364 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f28840022b0 con 0x7f28940b8410 2021-02-10T11:14:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.934 7f289910d700 1 -- 172.21.2.101:0/3082457364 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28940c29a0 con 0x7f28940b8410 2021-02-10T11:14:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.934 7f289910d700 1 -- 172.21.2.101:0/3082457364 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28940c29a0 con 0x7f28940b8410 2021-02-10T11:14:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.935 7f288b7fe700 1 -- 172.21.2.101:0/3082457364 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f288400a000 con 0x7f28940b8410 2021-02-10T11:14:00.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.935 7f288b7fe700 1 --2- 172.21.2.101:0/3082457364 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28780186c0 0x7f287801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:00.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.935 7f288b7fe700 1 -- 172.21.2.101:0/3082457364 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f28840327b0 con 0x7f28940b8410 2021-02-10T11:14:00.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.935 7f2892d9d700 1 --2- 172.21.2.101:0/3082457364 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28780186c0 0x7f287801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:00.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.935 7f2892d9d700 1 --2- 172.21.2.101:0/3082457364 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28780186c0 0x7f287801abc0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:00.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.966 7f289910d700 1 -- 172.21.2.101:0/3082457364 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2894105b30 con 0x7f28940b8410 2021-02-10T11:14:00.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:00.968 7f288b7fe700 1 -- 172.21.2.101:0/3082457364 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2884006b60 con 0x7f28940b8410 2021-02-10T11:14:01.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.096 7f289910d700 1 -- 172.21.2.101:0/3082457364 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f2894105b30 con 0x7f28940b8410 2021-02-10T11:14:01.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.097 7f288b7fe700 1 -- 172.21.2.101:0/3082457364 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v20) v1 ==== 100+0+10 (secure 0 0 0) 0x7f288400bd60 con 0x7f28940b8410 2021-02-10T11:14:01.101 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:14:01.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.099 7f289910d700 1 -- 172.21.2.101:0/3082457364 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28780186c0 msgr2=0x7f287801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:01.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.099 7f289910d700 1 --2- 172.21.2.101:0/3082457364 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28780186c0 0x7f287801abc0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:01.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.099 7f289910d700 1 -- 172.21.2.101:0/3082457364 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28940b8410 msgr2=0x7f28940b8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:01.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.099 7f289910d700 1 --2- 172.21.2.101:0/3082457364 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28940b8410 0x7f28940b8810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f2884006f60 tx=0x7f288400b970).stop 2021-02-10T11:14:01.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.099 7f289910d700 1 -- 172.21.2.101:0/3082457364 shutdown_connections 2021-02-10T11:14:01.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.099 7f289910d700 1 --2- 172.21.2.101:0/3082457364 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28780186c0 0x7f287801abc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:01.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.099 7f289910d700 1 --2- 172.21.2.101:0/3082457364 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28940c1f80 0x7f28940c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:01.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.099 7f289910d700 1 --2- 172.21.2.101:0/3082457364 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28940b8410 0x7f28940b8810 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:01.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.099 7f289910d700 1 --2- 172.21.2.101:0/3082457364 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f289409b930 0x7f28940b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:01.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.099 7f289910d700 1 -- 172.21.2.101:0/3082457364 >> 172.21.2.101:0/3082457364 conn(0x7f2894094e10 msgr2=0x7f28940a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:01.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.099 7f289910d700 1 -- 172.21.2.101:0/3082457364 shutdown_connections 2021-02-10T11:14:01.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.099 7f289910d700 1 -- 172.21.2.101:0/3082457364 wait complete. 2021-02-10T11:14:01.111 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:14:01.112 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:14:01.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.516 7fe46cbf6700 1 Processor -- start 2021-02-10T11:14:01.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.516 7fe46cbf6700 1 -- start start 2021-02-10T11:14:01.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.517 7fe46cbf6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe4680e5070 0x7fe4681b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:01.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.517 7fe46cbf6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe4680e5470 0x7fe468080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:01.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.517 7fe46cbf6700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46807ed80 con 0x7fe4681b3e40 2021-02-10T11:14:01.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.517 7fe46cbf6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4680d1790 con 0x7fe4680e5070 2021-02-10T11:14:01.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.517 7fe46cbf6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4680d19b0 con 0x7fe4680e5470 2021-02-10T11:14:01.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.517 7fe46685e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe4680e5070 0x7fe4681b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:01.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.517 7fe46685e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe4680e5070 0x7fe4681b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44776/0 (socket says 172.21.2.101:44776) 2021-02-10T11:14:01.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.517 7fe46685e700 1 -- 172.21.2.101:0/1745096619 learned_addr learned my addr 172.21.2.101:0/1745096619 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:01.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.517 7fe46605d700 1 --1- 172.21.2.101:0/1745096619 >> v1:172.21.2.101:6789/0 conn(0x7fe4681b3e40 0x7fe46807d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45272/0 (socket says 172.21.2.101:45272) 2021-02-10T11:14:01.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.517 7fe46685e700 1 -- 172.21.2.101:0/1745096619 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe4680e5470 msgr2=0x7fe468080310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:01.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.517 7fe46685e700 1 --2- 172.21.2.101:0/1745096619 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe4680e5470 0x7fe468080310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:01.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.517 7fe46685e700 1 -- 172.21.2.101:0/1745096619 >> v1:172.21.2.101:6789/0 conn(0x7fe4681b3e40 legacy=0x7fe46807d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:01.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.517 7fe46685e700 1 -- 172.21.2.101:0/1745096619 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4680d1bd0 con 0x7fe4680e5070 2021-02-10T11:14:01.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.517 7fe46685e700 1 --2- 172.21.2.101:0/1745096619 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe4680e5070 0x7fe4681b3820 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fe458006700 tx=0x7fe458009b80).ready entity=mon.0 client_cookie=6d18c59736471acc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:01.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.517 7fe46585c700 1 -- 172.21.2.101:0/1745096619 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe45800c0c0 con 0x7fe4680e5070 2021-02-10T11:14:01.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.517 7fe46585c700 1 -- 172.21.2.101:0/1745096619 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45800c400 con 0x7fe4680e5070 2021-02-10T11:14:01.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.517 7fe46585c700 1 -- 172.21.2.101:0/1745096619 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe45800c8e0 con 0x7fe4680e5070 2021-02-10T11:14:01.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.517 7fe46cbf6700 1 -- 172.21.2.101:0/1745096619 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe4680e5070 msgr2=0x7fe4681b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:01.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.517 7fe46cbf6700 1 --2- 172.21.2.101:0/1745096619 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe4680e5070 0x7fe4681b3820 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fe458006700 tx=0x7fe458009b80).stop 2021-02-10T11:14:01.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.518 7fe46cbf6700 1 -- 172.21.2.101:0/1745096619 shutdown_connections 2021-02-10T11:14:01.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.518 7fe46cbf6700 1 --2- 172.21.2.101:0/1745096619 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe4680e5470 0x7fe468080310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:01.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.518 7fe46cbf6700 1 --2- 172.21.2.101:0/1745096619 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe4680e5070 0x7fe4681b3820 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:01.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.518 7fe46cbf6700 1 -- 172.21.2.101:0/1745096619 >> 172.21.2.101:0/1745096619 conn(0x7fe4680ce690 msgr2=0x7fe4680e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:01.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.518 7fe46cbf6700 1 -- 172.21.2.101:0/1745096619 shutdown_connections 2021-02-10T11:14:01.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.518 7fe46cbf6700 1 -- 172.21.2.101:0/1745096619 wait complete. 2021-02-10T11:14:01.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.532 7fe46cbf6700 1 Processor -- start 2021-02-10T11:14:01.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.532 7fe46cbf6700 1 -- start start 2021-02-10T11:14:01.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.533 7fe46cbf6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe46818c700 0x7fe468096c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:01.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.533 7fe46cbf6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe4680ae400 0x7fe4680ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:01.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.533 7fe46cbf6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe468192380 0x7fe468192780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:01.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.533 7fe46cbf6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe468097290 con 0x7fe46818c700 2021-02-10T11:14:01.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.533 7fe46cbf6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe468192da0 con 0x7fe468192380 2021-02-10T11:14:01.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.533 7fe46cbf6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe468192f80 con 0x7fe4680ae400 2021-02-10T11:14:01.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.533 7fe46685e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe46818c700 0x7fe468096c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:01.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.533 7fe46685e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe46818c700 0x7fe468096c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44786/0 (socket says 172.21.2.101:44786) 2021-02-10T11:14:01.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.533 7fe46685e700 1 -- 172.21.2.101:0/3328057426 learned_addr learned my addr 172.21.2.101:0/3328057426 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:01.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.533 7fe46685e700 1 -- 172.21.2.101:0/3328057426 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe4680ae400 msgr2=0x7fe4680ae800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:01.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.533 7fe46685e700 1 --2- 172.21.2.101:0/3328057426 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe4680ae400 0x7fe4680ae800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:01.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.533 7fe46685e700 1 -- 172.21.2.101:0/3328057426 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe468192380 msgr2=0x7fe468192780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:01.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.533 7fe46685e700 1 --2- 172.21.2.101:0/3328057426 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe468192380 0x7fe468192780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:01.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.533 7fe46685e700 1 -- 172.21.2.101:0/3328057426 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4681931a0 con 0x7fe46818c700 2021-02-10T11:14:01.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.533 7fe46685e700 1 --2- 172.21.2.101:0/3328057426 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe46818c700 0x7fe468096c70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fe4580066a0 tx=0x7fe45800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:01.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.534 7fe45f7fe700 1 -- 172.21.2.101:0/3328057426 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe45800bd80 con 0x7fe46818c700 2021-02-10T11:14:01.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.534 7fe45f7fe700 1 -- 172.21.2.101:0/3328057426 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4580052e0 con 0x7fe46818c700 2021-02-10T11:14:01.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.534 7fe45f7fe700 1 -- 172.21.2.101:0/3328057426 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe4580057c0 con 0x7fe46818c700 2021-02-10T11:14:01.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.534 7fe46cbf6700 1 -- 172.21.2.101:0/3328057426 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4681931a0 con 0x7fe46818c700 2021-02-10T11:14:01.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.534 7fe46cbf6700 1 -- 172.21.2.101:0/3328057426 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4681931a0 con 0x7fe46818c700 2021-02-10T11:14:01.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.534 7fe45f7fe700 1 -- 172.21.2.101:0/3328057426 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe458006240 con 0x7fe46818c700 2021-02-10T11:14:01.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.534 7fe45f7fe700 1 --2- 172.21.2.101:0/3328057426 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe44c0186c0 0x7fe44c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:01.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.534 7fe45f7fe700 1 -- 172.21.2.101:0/3328057426 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe458032580 con 0x7fe46818c700 2021-02-10T11:14:01.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.535 7fe46605d700 1 --2- 172.21.2.101:0/3328057426 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe44c0186c0 0x7fe44c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:01.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.535 7fe46605d700 1 --2- 172.21.2.101:0/3328057426 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe44c0186c0 0x7fe44c01abc0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:01.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.566 7fe46cbf6700 1 -- 172.21.2.101:0/3328057426 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe468107120 con 0x7fe46818c700 2021-02-10T11:14:01.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.569 7fe45f7fe700 1 -- 172.21.2.101:0/3328057426 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe458006030 con 0x7fe46818c700 2021-02-10T11:14:01.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.697 7fe46cbf6700 1 -- 172.21.2.101:0/3328057426 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe468107120 con 0x7fe46818c700 2021-02-10T11:14:01.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.697 7fe45f7fe700 1 -- 172.21.2.101:0/3328057426 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2097 (secure 0 0 0) 0x7fe45800bd80 con 0x7fe46818c700 2021-02-10T11:14:01.702 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 6 2021-02-10T11:14:01.702 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:14:01.703 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/3184957761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3184957761},{"type":"v1","addr":"172.21.2.108:6833","nonce":3184957761}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:13:14.147021","modified":"2021-02-10 11:13:23.547179","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":5,"state":"up:active","state_seq":3,"addr":"172.21.2.101:6835/3530104431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3530104431},{"type":"v1","addr":"172.21.2.101:6835","nonce":3530104431}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-02-10T11:14:01.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.699 7fe46cbf6700 1 -- 172.21.2.101:0/3328057426 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe44c0186c0 msgr2=0x7fe44c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:01.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.699 7fe46cbf6700 1 --2- 172.21.2.101:0/3328057426 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe44c0186c0 0x7fe44c01abc0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:01.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.699 7fe46cbf6700 1 -- 172.21.2.101:0/3328057426 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe46818c700 msgr2=0x7fe468096c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:01.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.699 7fe46cbf6700 1 --2- 172.21.2.101:0/3328057426 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe46818c700 0x7fe468096c70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fe4580066a0 tx=0x7fe45800b990).stop 2021-02-10T11:14:01.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.700 7fe46cbf6700 1 -- 172.21.2.101:0/3328057426 shutdown_connections 2021-02-10T11:14:01.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.700 7fe46cbf6700 1 --2- 172.21.2.101:0/3328057426 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe44c0186c0 0x7fe44c01abc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:01.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.700 7fe46cbf6700 1 --2- 172.21.2.101:0/3328057426 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe4680ae400 0x7fe4680ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:01.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.700 7fe46cbf6700 1 --2- 172.21.2.101:0/3328057426 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe468192380 0x7fe468192780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:01.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.700 7fe46cbf6700 1 --2- 172.21.2.101:0/3328057426 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe46818c700 0x7fe468096c70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:01.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.700 7fe46cbf6700 1 -- 172.21.2.101:0/3328057426 >> 172.21.2.101:0/3328057426 conn(0x7fe4680891f0 msgr2=0x7fe4681a0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:01.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.700 7fe46cbf6700 1 -- 172.21.2.101:0/3328057426 shutdown_connections 2021-02-10T11:14:01.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:01.700 7fe46cbf6700 1 -- 172.21.2.101:0/3328057426 wait complete. 2021-02-10T11:14:01.914 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:13:14.147021', 'modified': '2021-02-10 11:13:23.547179', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4299': {'gid': 4299, 'name': 'a', 'rank': 0, 'incarnation': 5, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.101:6835/3530104431', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.101:6834', 'nonce': 3530104431}, {'type': 'v1', 'addr': '172.21.2.101:6835', 'nonce': 3530104431}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:14:01.914 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:14:01.914 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok status 2021-02-10T11:14:02.312 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:14:02.312 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:14:02.312 INFO:teuthology.orchestra.run.gibba001.stdout: "whoami": 0, 2021-02-10T11:14:02.312 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 4299, 2021-02-10T11:14:02.312 INFO:teuthology.orchestra.run.gibba001.stdout: "want_state": "up:active", 2021-02-10T11:14:02.312 INFO:teuthology.orchestra.run.gibba001.stdout: "state": "up:active", 2021-02-10T11:14:02.313 INFO:teuthology.orchestra.run.gibba001.stdout: "rank_uptime": 39.761078703999999, 2021-02-10T11:14:02.313 INFO:teuthology.orchestra.run.gibba001.stdout: "mdsmap_epoch": 6, 2021-02-10T11:14:02.313 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch": 20, 2021-02-10T11:14:02.313 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch_barrier": 20, 2021-02-10T11:14:02.313 INFO:teuthology.orchestra.run.gibba001.stdout: "uptime": 44.153009976 2021-02-10T11:14:02.313 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:14:02.316 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 4299, "want_state": "up:active", "state": "up:active", "rank_uptime": 39.761078703999999, "mdsmap_epoch": 6, "osdmap_epoch": 20, "osdmap_epoch_barrier": 20, "uptime": 44.153009976 } 2021-02-10T11:14:02.316 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:14:02.316 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2021-02-10T11:14:02.316 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.3 at ubuntu@gibba008.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.3... 2021-02-10T11:14:02.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:02.317 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:14:02.652 DEBUG:teuthology.orchestra.run.gibba008:> sudo modprobe fuse 2021-02-10T11:14:02.995 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:14:02.995 DEBUG:teuthology.orchestra.run.gibba008:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:14:03.338 INFO:teuthology.orchestra.run.gibba008.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:14:03.338 INFO:teuthology.orchestra.run.gibba008.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:14:03.339 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:14:03.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:03.339 DEBUG:teuthology.orchestra.run.gibba008:> ls /sys/fs/fuse/connections 2021-02-10T11:14:03.673 INFO:teuthology.orchestra.run.gibba008.stdout:43 2021-02-10T11:14:03.674 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2021-02-10T11:14:03.674 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.3 /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:14:03.677 DEBUG:teuthology.orchestra.run.gibba008:> sudo modprobe fuse 2021-02-10T11:14:04.020 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:14:04.021 DEBUG:teuthology.orchestra.run.gibba008:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:14:04.087 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.gibba008.stderr:ceph-fuse[37501]: starting ceph client 2021-02-10T11:14:04.087 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.gibba008.stderr:2021-02-10 11:14:04.090 7f29a09e7f80 -1 init, newargv = 0x557c78e9a110 newargc=9 2021-02-10T11:14:04.092 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.gibba008.stderr:ceph-fuse[37501]: starting fuse 2021-02-10T11:14:04.362 INFO:teuthology.orchestra.run.gibba008.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:14:04.363 INFO:teuthology.orchestra.run.gibba008.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:14:04.363 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:14:04.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:04.364 DEBUG:teuthology.orchestra.run.gibba008:> ls /sys/fs/fuse/connections 2021-02-10T11:14:04.698 INFO:teuthology.orchestra.run.gibba008.stdout:43 2021-02-10T11:14:04.699 INFO:teuthology.orchestra.run.gibba008.stdout:44 2021-02-10T11:14:04.699 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2021-02-10T11:14:04.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:04.699 DEBUG:teuthology.orchestra.run.gibba008:> sudo python3 -c ' 2021-02-10T11:14:04.699 DEBUG:teuthology.orchestra.run.gibba008:> import glob 2021-02-10T11:14:04.699 DEBUG:teuthology.orchestra.run.gibba008:> import re 2021-02-10T11:14:04.700 DEBUG:teuthology.orchestra.run.gibba008:> import os 2021-02-10T11:14:04.700 DEBUG:teuthology.orchestra.run.gibba008:> import subprocess 2021-02-10T11:14:04.700 DEBUG:teuthology.orchestra.run.gibba008:> 2021-02-10T11:14:04.700 DEBUG:teuthology.orchestra.run.gibba008:> def find_socket(client_name): 2021-02-10T11:14:04.700 DEBUG:teuthology.orchestra.run.gibba008:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2021-02-10T11:14:04.700 DEBUG:teuthology.orchestra.run.gibba008:> files = glob.glob(asok_path) 2021-02-10T11:14:04.700 DEBUG:teuthology.orchestra.run.gibba008:> 2021-02-10T11:14:04.700 DEBUG:teuthology.orchestra.run.gibba008:> # Given a non-glob path, it better be there 2021-02-10T11:14:04.700 DEBUG:teuthology.orchestra.run.gibba008:> if "*" not in asok_path: 2021-02-10T11:14:04.700 DEBUG:teuthology.orchestra.run.gibba008:> assert(len(files) == 1) 2021-02-10T11:14:04.700 DEBUG:teuthology.orchestra.run.gibba008:> return files[0] 2021-02-10T11:14:04.700 DEBUG:teuthology.orchestra.run.gibba008:> 2021-02-10T11:14:04.700 DEBUG:teuthology.orchestra.run.gibba008:> for f in files: 2021-02-10T11:14:04.700 DEBUG:teuthology.orchestra.run.gibba008:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:14:04.701 DEBUG:teuthology.orchestra.run.gibba008:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:14:04.701 DEBUG:teuthology.orchestra.run.gibba008:> return f 2021-02-10T11:14:04.701 DEBUG:teuthology.orchestra.run.gibba008:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:14:04.701 DEBUG:teuthology.orchestra.run.gibba008:> 2021-02-10T11:14:04.701 DEBUG:teuthology.orchestra.run.gibba008:> print(find_socket("client.3")) 2021-02-10T11:14:04.701 DEBUG:teuthology.orchestra.run.gibba008:> ' 2021-02-10T11:14:05.062 INFO:teuthology.orchestra.run.gibba008.stdout:/var/run/ceph/ceph-client.3.37501.asok 2021-02-10T11:14:05.064 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.37501.asok 2021-02-10T11:14:05.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:05.065 DEBUG:teuthology.orchestra.run.gibba008:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.37501.asok status 2021-02-10T11:14:05.453 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:14:05.454 INFO:teuthology.orchestra.run.gibba008.stdout: "metadata": { 2021-02-10T11:14:05.454 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:14:05.454 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:14:05.454 INFO:teuthology.orchestra.run.gibba008.stdout: "entity_id": "3", 2021-02-10T11:14:05.454 INFO:teuthology.orchestra.run.gibba008.stdout: "hostname": "gibba008", 2021-02-10T11:14:05.454 INFO:teuthology.orchestra.run.gibba008.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2021-02-10T11:14:05.454 INFO:teuthology.orchestra.run.gibba008.stdout: "pid": "37501", 2021-02-10T11:14:05.454 INFO:teuthology.orchestra.run.gibba008.stdout: "root": "/" 2021-02-10T11:14:05.454 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:14:05.454 INFO:teuthology.orchestra.run.gibba008.stdout: "dentry_count": 0, 2021-02-10T11:14:05.454 INFO:teuthology.orchestra.run.gibba008.stdout: "dentry_pinned_count": 0, 2021-02-10T11:14:05.454 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 4432, 2021-02-10T11:14:05.455 INFO:teuthology.orchestra.run.gibba008.stdout: "inst": { 2021-02-10T11:14:05.455 INFO:teuthology.orchestra.run.gibba008.stdout: "name": { 2021-02-10T11:14:05.455 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "client", 2021-02-10T11:14:05.455 INFO:teuthology.orchestra.run.gibba008.stdout: "num": 4432 2021-02-10T11:14:05.455 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:14:05.455 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": { 2021-02-10T11:14:05.455 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "v1", 2021-02-10T11:14:05.455 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": "172.21.2.108:0", 2021-02-10T11:14:05.455 INFO:teuthology.orchestra.run.gibba008.stdout: "nonce": 3893725958 2021-02-10T11:14:05.455 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:14:05.455 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:14:05.455 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": { 2021-02-10T11:14:05.455 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "v1", 2021-02-10T11:14:05.456 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": "172.21.2.108:0", 2021-02-10T11:14:05.456 INFO:teuthology.orchestra.run.gibba008.stdout: "nonce": 3893725958 2021-02-10T11:14:05.456 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:14:05.456 INFO:teuthology.orchestra.run.gibba008.stdout: "inst_str": "client.4432 172.21.2.108:0/3893725958", 2021-02-10T11:14:05.456 INFO:teuthology.orchestra.run.gibba008.stdout: "addr_str": "172.21.2.108:0/3893725958", 2021-02-10T11:14:05.456 INFO:teuthology.orchestra.run.gibba008.stdout: "inode_count": 1, 2021-02-10T11:14:05.456 INFO:teuthology.orchestra.run.gibba008.stdout: "mds_epoch": 6, 2021-02-10T11:14:05.456 INFO:teuthology.orchestra.run.gibba008.stdout: "osd_epoch": 20, 2021-02-10T11:14:05.456 INFO:teuthology.orchestra.run.gibba008.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:14:05.456 INFO:teuthology.orchestra.run.gibba008.stdout: "blacklisted": false 2021-02-10T11:14:05.456 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:14:05.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:05.458 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:14:05.458 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:14:05.795 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:14:05.795 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:14:05.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:05.795 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:14:06.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:06.144 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:14:06.144 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:14:06.478 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:14:06.478 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:14:06.478 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:06.478 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:14:06.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:06.825 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:14:06.825 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:14:07.160 INFO:teuthology.orchestra.run.gibba008.stdout:fuseblk 2021-02-10T11:14:07.160 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2021-02-10T11:14:07.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:07.160 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:14:07.509 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:07.510 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:14:07.510 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:14:07.845 INFO:teuthology.orchestra.run.gibba008.stdout:fuseblk 2021-02-10T11:14:07.846 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2021-02-10T11:14:07.846 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:07.846 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:14:08.192 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2021-02-10T11:14:08.219 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:14:08.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.627 7f6dc2d7f700 1 Processor -- start 2021-02-10T11:14:08.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.627 7f6dc2d7f700 1 -- start start 2021-02-10T11:14:08.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.628 7f6dc2d7f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6dbc0e5470 0x7f6dbc07e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:08.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.628 7f6dc2d7f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dbc07ed80 con 0x7f6dbc0e5070 2021-02-10T11:14:08.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.628 7f6dc2d7f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dbc1b4010 con 0x7f6dbc07f300 2021-02-10T11:14:08.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.628 7f6dc2d7f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dbc0d1790 con 0x7f6dbc0e5470 2021-02-10T11:14:08.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.628 7f6dc0b9b700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f6dbc0e5070 0x7f6dbc1b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45284/0 (socket says 172.21.2.101:45284) 2021-02-10T11:14:08.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.628 7f6dc0b9b700 1 -- 172.21.2.101:0/1240521555 learned_addr learned my addr 172.21.2.101:0/1240521555 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:14:08.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.628 7f6dbbfff700 1 --2- 172.21.2.101:0/1240521555 >> v2:172.21.2.108:3300/0 conn(0x7f6dbc0e5470 0x7f6dbc07e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:08.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.628 7f6dbb7fe700 1 -- 172.21.2.101:0/1240521555 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1107649585 0 0) 0x7f6da40009b0 con 0x7f6dbc0e5070 2021-02-10T11:14:08.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.628 7f6dbb7fe700 1 -- 172.21.2.101:0/1240521555 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6da8002fe0 con 0x7f6dbc0e5070 2021-02-10T11:14:08.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.628 7f6dbbfff700 1 -- 172.21.2.101:0/1240521555 >> v1:172.21.2.108:6789/0 conn(0x7f6dbc07f300 legacy=0x7f6dbc07f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:08.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.628 7f6dbbfff700 1 -- 172.21.2.101:0/1240521555 >> v1:172.21.2.101:6789/0 conn(0x7f6dbc0e5070 legacy=0x7f6dbc1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:08.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.628 7f6dbbfff700 1 -- 172.21.2.101:0/1240521555 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc0d19b0 con 0x7f6dbc0e5470 2021-02-10T11:14:08.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.628 7f6dbbfff700 1 --2- 172.21.2.101:0/1240521555 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6dbc0e5470 0x7f6dbc07e6a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f6dac006650 tx=0x7f6dac009a80).ready entity=mon.1 client_cookie=7703bc82d08d4931 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:08.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.629 7f6dbb7fe700 1 -- 172.21.2.101:0/1240521555 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6dac00c0c0 con 0x7f6dbc0e5470 2021-02-10T11:14:08.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.629 7f6dbb7fe700 1 -- 172.21.2.101:0/1240521555 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac00c340 con 0x7f6dbc0e5470 2021-02-10T11:14:08.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.629 7f6dbb7fe700 1 -- 172.21.2.101:0/1240521555 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6dac00c860 con 0x7f6dbc0e5470 2021-02-10T11:14:08.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.629 7f6dc2d7f700 1 -- 172.21.2.101:0/1240521555 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6dbc0e5470 msgr2=0x7f6dbc07e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:08.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.629 7f6dc2d7f700 1 --2- 172.21.2.101:0/1240521555 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6dbc0e5470 0x7f6dbc07e6a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f6dac006650 tx=0x7f6dac009a80).stop 2021-02-10T11:14:08.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.629 7f6dc2d7f700 1 -- 172.21.2.101:0/1240521555 shutdown_connections 2021-02-10T11:14:08.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.629 7f6dc2d7f700 1 --2- 172.21.2.101:0/1240521555 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6dbc0e5470 0x7f6dbc07e6a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:08.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.629 7f6dc2d7f700 1 -- 172.21.2.101:0/1240521555 >> 172.21.2.101:0/1240521555 conn(0x7f6dbc0ce690 msgr2=0x7f6dbc0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:08.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.629 7f6dc2d7f700 1 -- 172.21.2.101:0/1240521555 shutdown_connections 2021-02-10T11:14:08.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.629 7f6dc2d7f700 1 -- 172.21.2.101:0/1240521555 wait complete. 2021-02-10T11:14:08.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.644 7f6dc2d7f700 1 Processor -- start 2021-02-10T11:14:08.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.644 7f6dc2d7f700 1 -- start start 2021-02-10T11:14:08.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.644 7f6dc2d7f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6dbc08a460 0x7f6dbc1a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:08.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.644 7f6dc2d7f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6dbc19d980 0x7f6dbc19dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:08.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.644 7f6dc2d7f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6dbc19e3a0 0x7f6dbc19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:08.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.644 7f6dc2d7f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc08ac90 con 0x7f6dbc19d980 2021-02-10T11:14:08.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.644 7f6dc2d7f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc1a1520 con 0x7f6dbc08a460 2021-02-10T11:14:08.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.644 7f6dc2d7f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc1a20c0 con 0x7f6dbc19e3a0 2021-02-10T11:14:08.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.644 7f6dbbfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6dbc19d980 0x7f6dbc19dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:08.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.645 7f6dbbfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6dbc19d980 0x7f6dbc19dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44796/0 (socket says 172.21.2.101:44796) 2021-02-10T11:14:08.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.645 7f6dbbfff700 1 -- 172.21.2.101:0/4174708414 learned_addr learned my addr 172.21.2.101:0/4174708414 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:08.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.645 7f6dc0b9b700 1 --2- 172.21.2.101:0/4174708414 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6dbc08a460 0x7f6dbc1a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:08.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.645 7f6dc139c700 1 --2- 172.21.2.101:0/4174708414 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6dbc19e3a0 0x7f6dbc19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:08.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.645 7f6dbbfff700 1 -- 172.21.2.101:0/4174708414 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6dbc19e3a0 msgr2=0x7f6dbc19e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:08.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.645 7f6dbbfff700 1 --2- 172.21.2.101:0/4174708414 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6dbc19e3a0 0x7f6dbc19e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:08.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.645 7f6dbbfff700 1 -- 172.21.2.101:0/4174708414 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6dbc08a460 msgr2=0x7f6dbc1a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:08.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.645 7f6dbbfff700 1 --2- 172.21.2.101:0/4174708414 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6dbc08a460 0x7f6dbc1a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:08.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.645 7f6dbbfff700 1 -- 172.21.2.101:0/4174708414 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc08a860 con 0x7f6dbc19d980 2021-02-10T11:14:08.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.645 7f6dc139c700 1 --2- 172.21.2.101:0/4174708414 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6dbc19e3a0 0x7f6dbc19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:14:08.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.645 7f6dbbfff700 1 --2- 172.21.2.101:0/4174708414 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6dbc19d980 0x7f6dbc19dd80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f6dac009570 tx=0x7f6dac002670).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:08.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.645 7f6db97fa700 1 -- 172.21.2.101:0/4174708414 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6dac006030 con 0x7f6dbc19d980 2021-02-10T11:14:08.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.645 7f6db97fa700 1 -- 172.21.2.101:0/4174708414 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac0052e0 con 0x7f6dbc19d980 2021-02-10T11:14:08.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.645 7f6dc2d7f700 1 -- 172.21.2.101:0/4174708414 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dbc08a860 con 0x7f6dbc19d980 2021-02-10T11:14:08.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.645 7f6dc2d7f700 1 -- 172.21.2.101:0/4174708414 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dbc19f0b0 con 0x7f6dbc19d980 2021-02-10T11:14:08.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.645 7f6db97fa700 1 -- 172.21.2.101:0/4174708414 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6dac005730 con 0x7f6dbc19d980 2021-02-10T11:14:08.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.646 7f6db97fa700 1 -- 172.21.2.101:0/4174708414 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6dac0054c0 con 0x7f6dbc19d980 2021-02-10T11:14:08.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.646 7f6db97fa700 1 --2- 172.21.2.101:0/4174708414 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6da00186c0 0x7f6da001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:08.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.646 7f6db97fa700 1 -- 172.21.2.101:0/4174708414 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6dac0325e0 con 0x7f6dbc19d980 2021-02-10T11:14:08.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.646 7f6dc0b9b700 1 --2- 172.21.2.101:0/4174708414 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6da00186c0 0x7f6da001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:08.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.646 7f6dc0b9b700 1 --2- 172.21.2.101:0/4174708414 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6da00186c0 0x7f6da001abc0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:08.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.677 7f6dc2d7f700 1 -- 172.21.2.101:0/4174708414 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dbc083e80 con 0x7f6dbc19d980 2021-02-10T11:14:08.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.680 7f6db97fa700 1 -- 172.21.2.101:0/4174708414 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6dac0009b0 con 0x7f6dbc19d980 2021-02-10T11:14:08.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.805 7f6dc2d7f700 1 -- 172.21.2.101:0/4174708414 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6dbc07ed80 con 0x7f6dbc19d980 2021-02-10T11:14:08.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.806 7f6db97fa700 1 -- 172.21.2.101:0/4174708414 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+13993 (secure 0 0 0) 0x7f6dac0054a0 con 0x7f6dbc19d980 2021-02-10T11:14:08.811 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:14:08.811 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":20,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:13:15.149234","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:13:11.922272","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2021-02-10 11:13:13.141733","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:14:08.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.807 7f6dc2d7f700 1 -- 172.21.2.101:0/4174708414 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6da00186c0 msgr2=0x7f6da001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:08.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.807 7f6dc2d7f700 1 --2- 172.21.2.101:0/4174708414 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6da00186c0 0x7f6da001abc0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:08.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.807 7f6dc2d7f700 1 -- 172.21.2.101:0/4174708414 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6dbc19d980 msgr2=0x7f6dbc19dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:08.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.807 7f6dc2d7f700 1 --2- 172.21.2.101:0/4174708414 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6dbc19d980 0x7f6dbc19dd80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f6dac009570 tx=0x7f6dac002670).stop 2021-02-10T11:14:08.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.808 7f6dc2d7f700 1 -- 172.21.2.101:0/4174708414 shutdown_connections 2021-02-10T11:14:08.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.808 7f6dc2d7f700 1 --2- 172.21.2.101:0/4174708414 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6da00186c0 0x7f6da001abc0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:08.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.808 7f6dc2d7f700 1 --2- 172.21.2.101:0/4174708414 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6dbc19e3a0 0x7f6dbc19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:08.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.808 7f6dc2d7f700 1 --2- 172.21.2.101:0/4174708414 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6dbc19d980 0x7f6dbc19dd80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:08.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.808 7f6dc2d7f700 1 --2- 172.21.2.101:0/4174708414 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6dbc08a460 0x7f6dbc1a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:08.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.808 7f6dc2d7f700 1 -- 172.21.2.101:0/4174708414 >> 172.21.2.101:0/4174708414 conn(0x7f6dbc089d10 msgr2=0x7f6dbc090610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:08.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.808 7f6dc2d7f700 1 -- 172.21.2.101:0/4174708414 shutdown_connections 2021-02-10T11:14:08.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:08.808 7f6dc2d7f700 1 -- 172.21.2.101:0/4174708414 wait complete. 2021-02-10T11:14:08.823 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:13:11.922272', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:13:13.141733', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:14:08.825 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:14:09.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.248 7f7be3ae3700 1 Processor -- start 2021-02-10T11:14:09.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.248 7f7be3ae3700 1 -- start start 2021-02-10T11:14:09.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.248 7f7be3ae3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7bdc1b0470 0x7f7bdc1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:09.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.248 7f7be3ae3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7bdc0ef610 0x7f7bdc0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:09.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.248 7f7be3ae3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7bdc0f0030 0x7f7bdc0f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:09.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.248 7f7be3ae3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bdc0e5ea0 con 0x7f7bdc0f0030 2021-02-10T11:14:09.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.248 7f7be3ae3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bdc0f0b50 con 0x7f7bdc0ef610 2021-02-10T11:14:09.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.249 7f7be3ae3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc0e52e0 con 0x7f7bdc1b0470 2021-02-10T11:14:09.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.249 7f7be2100700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7bdc0f0030 0x7f7bdc0f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:09.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.249 7f7be2100700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7bdc0f0030 0x7f7bdc0f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44808/0 (socket says 172.21.2.101:44808) 2021-02-10T11:14:09.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.249 7f7be2100700 1 -- 172.21.2.101:0/2039945605 learned_addr learned my addr 172.21.2.101:0/2039945605 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:09.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.249 7f7be2100700 1 -- 172.21.2.101:0/2039945605 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7bdc1b0470 msgr2=0x7f7bdc1b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:09.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.249 7f7be2100700 1 --2- 172.21.2.101:0/2039945605 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7bdc1b0470 0x7f7bdc1b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:09.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.249 7f7be2100700 1 -- 172.21.2.101:0/2039945605 >> v2:172.21.2.108:3300/0 conn(0x7f7bdc0ef610 msgr2=0x7f7bdc0efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:09.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.249 7f7be2100700 1 --2- 172.21.2.101:0/2039945605 >> v2:172.21.2.108:3300/0 conn(0x7f7bdc0ef610 0x7f7bdc0efa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:09.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.249 7f7be2100700 1 -- 172.21.2.101:0/2039945605 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bdc0f10d0 con 0x7f7bdc0f0030 2021-02-10T11:14:09.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.249 7f7be2100700 1 --2- 172.21.2.101:0/2039945605 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bdc0f0030 0x7f7bdc0f0430 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f7bd8006b80 tx=0x7f7bd8009b90).ready entity=mon.0 client_cookie=2bbad9afc1686b62 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:09.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.249 7f7be08fd700 1 -- 172.21.2.101:0/2039945605 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7bd80022e0 con 0x7f7bdc0f0030 2021-02-10T11:14:09.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.249 7f7be08fd700 1 -- 172.21.2.101:0/2039945605 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd8002620 con 0x7f7bdc0f0030 2021-02-10T11:14:09.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.249 7f7be08fd700 1 -- 172.21.2.101:0/2039945605 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7bd8002b00 con 0x7f7bdc0f0030 2021-02-10T11:14:09.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.249 7f7be3ae3700 1 -- 172.21.2.101:0/2039945605 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bdc0f0030 msgr2=0x7f7bdc0f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:09.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.249 7f7be3ae3700 1 --2- 172.21.2.101:0/2039945605 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bdc0f0030 0x7f7bdc0f0430 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f7bd8006b80 tx=0x7f7bd8009b90).stop 2021-02-10T11:14:09.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.250 7f7be3ae3700 1 -- 172.21.2.101:0/2039945605 shutdown_connections 2021-02-10T11:14:09.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.250 7f7be3ae3700 1 --2- 172.21.2.101:0/2039945605 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bdc0f0030 0x7f7bdc0f0430 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:09.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.250 7f7be3ae3700 1 --2- 172.21.2.101:0/2039945605 >> v2:172.21.2.108:3300/0 conn(0x7f7bdc0ef610 0x7f7bdc0efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:09.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.250 7f7be3ae3700 1 --2- 172.21.2.101:0/2039945605 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7bdc1b0470 0x7f7bdc1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:09.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.250 7f7be3ae3700 1 -- 172.21.2.101:0/2039945605 >> 172.21.2.101:0/2039945605 conn(0x7f7bdc0ce3e0 msgr2=0x7f7bdc0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:09.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.250 7f7be3ae3700 1 -- 172.21.2.101:0/2039945605 shutdown_connections 2021-02-10T11:14:09.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.250 7f7be3ae3700 1 -- 172.21.2.101:0/2039945605 wait complete. 2021-02-10T11:14:09.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.264 7f7be3ae3700 1 Processor -- start 2021-02-10T11:14:09.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.265 7f7be3ae3700 1 -- start start 2021-02-10T11:14:09.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.265 7f7be3ae3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7bdc0c36a0 0x7f7bdc0c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:09.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.265 7f7be3ae3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bdc0c3190 0x7f7bdc0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:09.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.265 7f7be3ae3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7bdc0b2370 0x7f7bdc0b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:09.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.265 7f7be3ae3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc0b27c0 con 0x7f7bdc0c3190 2021-02-10T11:14:09.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.265 7f7be3ae3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc0b3360 con 0x7f7bdc0c36a0 2021-02-10T11:14:09.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.265 7f7be3ae3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc0b3540 con 0x7f7bdc0b2370 2021-02-10T11:14:09.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.265 7f7be10fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bdc0c3190 0x7f7bdc0b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:09.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.265 7f7be10fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bdc0c3190 0x7f7bdc0b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44810/0 (socket says 172.21.2.101:44810) 2021-02-10T11:14:09.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.265 7f7be10fe700 1 -- 172.21.2.101:0/3897056706 learned_addr learned my addr 172.21.2.101:0/3897056706 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:09.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.265 7f7be2100700 1 --2- 172.21.2.101:0/3897056706 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7bdc0b2370 0x7f7bdc0b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:09.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.265 7f7be18ff700 1 --2- 172.21.2.101:0/3897056706 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7bdc0c36a0 0x7f7bdc0c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:09.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.266 7f7be10fe700 1 -- 172.21.2.101:0/3897056706 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7bdc0b2370 msgr2=0x7f7bdc0b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:09.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.266 7f7be10fe700 1 --2- 172.21.2.101:0/3897056706 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7bdc0b2370 0x7f7bdc0b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:09.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.266 7f7be10fe700 1 -- 172.21.2.101:0/3897056706 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7bdc0c36a0 msgr2=0x7f7bdc0c2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:09.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.266 7f7be10fe700 1 --2- 172.21.2.101:0/3897056706 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7bdc0c36a0 0x7f7bdc0c2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:09.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.266 7f7be10fe700 1 -- 172.21.2.101:0/3897056706 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bdc0b3760 con 0x7f7bdc0c3190 2021-02-10T11:14:09.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.266 7f7be10fe700 1 --2- 172.21.2.101:0/3897056706 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bdc0c3190 0x7f7bdc0b1d50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f7bd4009fc0 tx=0x7f7bd400bc40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:09.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.266 7f7bd27fc700 1 -- 172.21.2.101:0/3897056706 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7bd4003020 con 0x7f7bdc0c3190 2021-02-10T11:14:09.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.266 7f7bd27fc700 1 -- 172.21.2.101:0/3897056706 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd4005850 con 0x7f7bdc0c3190 2021-02-10T11:14:09.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.266 7f7bd27fc700 1 -- 172.21.2.101:0/3897056706 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7bd4005d30 con 0x7f7bdc0c3190 2021-02-10T11:14:09.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.266 7f7be3ae3700 1 -- 172.21.2.101:0/3897056706 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bdc0b3760 con 0x7f7bdc0c3190 2021-02-10T11:14:09.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.266 7f7be3ae3700 1 -- 172.21.2.101:0/3897056706 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bdc0b3760 con 0x7f7bdc0c3190 2021-02-10T11:14:09.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.267 7f7bd27fc700 1 -- 172.21.2.101:0/3897056706 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7bd4005a30 con 0x7f7bdc0c3190 2021-02-10T11:14:09.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.267 7f7bd27fc700 1 --2- 172.21.2.101:0/3897056706 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7bc40186c0 0x7f7bc401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:09.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.267 7f7bd27fc700 1 -- 172.21.2.101:0/3897056706 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7bd4032780 con 0x7f7bdc0c3190 2021-02-10T11:14:09.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.267 7f7be18ff700 1 --2- 172.21.2.101:0/3897056706 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7bc40186c0 0x7f7bc401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:09.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.267 7f7be18ff700 1 --2- 172.21.2.101:0/3897056706 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7bc40186c0 0x7f7bc401abc0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:09.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.298 7f7be3ae3700 1 -- 172.21.2.101:0/3897056706 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bdc0edd10 con 0x7f7bdc0c3190 2021-02-10T11:14:09.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.301 7f7bd27fc700 1 -- 172.21.2.101:0/3897056706 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7bd4024f10 con 0x7f7bdc0c3190 2021-02-10T11:14:09.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.428 7f7be3ae3700 1 -- 172.21.2.101:0/3897056706 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f7bdc0e52e0 con 0x7f7bdc0c3190 2021-02-10T11:14:09.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.428 7f7bd27fc700 1 -- 172.21.2.101:0/3897056706 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7bd4024f10 con 0x7f7bdc0c3190 2021-02-10T11:14:09.433 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:14:09.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.430 7f7be3ae3700 1 -- 172.21.2.101:0/3897056706 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7bc40186c0 msgr2=0x7f7bc401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:09.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.430 7f7be3ae3700 1 --2- 172.21.2.101:0/3897056706 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7bc40186c0 0x7f7bc401abc0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:09.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.430 7f7be3ae3700 1 -- 172.21.2.101:0/3897056706 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bdc0c3190 msgr2=0x7f7bdc0b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:09.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.430 7f7be3ae3700 1 --2- 172.21.2.101:0/3897056706 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bdc0c3190 0x7f7bdc0b1d50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f7bd4009fc0 tx=0x7f7bd400bc40).stop 2021-02-10T11:14:09.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.430 7f7be3ae3700 1 -- 172.21.2.101:0/3897056706 shutdown_connections 2021-02-10T11:14:09.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.430 7f7be3ae3700 1 --2- 172.21.2.101:0/3897056706 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7bc40186c0 0x7f7bc401abc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:09.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.430 7f7be3ae3700 1 --2- 172.21.2.101:0/3897056706 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7bdc0b2370 0x7f7bdc0b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:09.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.430 7f7be3ae3700 1 --2- 172.21.2.101:0/3897056706 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bdc0c3190 0x7f7bdc0b1d50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:09.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.431 7f7be3ae3700 1 --2- 172.21.2.101:0/3897056706 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7bdc0c36a0 0x7f7bdc0c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:09.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.431 7f7be3ae3700 1 -- 172.21.2.101:0/3897056706 >> 172.21.2.101:0/3897056706 conn(0x7f7bdc09f630 msgr2=0x7f7bdc0a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:09.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.431 7f7be3ae3700 1 -- 172.21.2.101:0/3897056706 shutdown_connections 2021-02-10T11:14:09.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.431 7f7be3ae3700 1 -- 172.21.2.101:0/3897056706 wait complete. 2021-02-10T11:14:09.443 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:14:09.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.853 7f8a85301700 1 Processor -- start 2021-02-10T11:14:09.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.853 7f8a85301700 1 -- start start 2021-02-10T11:14:09.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.854 7f8a85301700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8a801b0870 0x7f8a800f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:09.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.854 7f8a85301700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a801b0c70 con 0x7f8a800efe00 2021-02-10T11:14:09.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.854 7f8a85301700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a800f0b50 con 0x7f8a801b0470 2021-02-10T11:14:09.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.854 7f8a85301700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a800e5ea0 con 0x7f8a801b0870 2021-02-10T11:14:09.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.854 7f8a7f87e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8a800efe00 0x7f8a800f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45316/0 (socket says 172.21.2.101:45316) 2021-02-10T11:14:09.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.854 7f8a7f87e700 1 -- 172.21.2.101:0/468898815 learned_addr learned my addr 172.21.2.101:0/468898815 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:14:09.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.854 7f8a7e07b700 1 -- 172.21.2.101:0/468898815 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2623520494 0 0) 0x7f8a78001500 con 0x7f8a800efe00 2021-02-10T11:14:09.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.854 7f8a7e87c700 1 --2- 172.21.2.101:0/468898815 >> v2:172.21.2.108:3300/0 conn(0x7f8a801b0870 0x7f8a800f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:09.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.854 7f8a7e07b700 1 -- 172.21.2.101:0/468898815 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a6c002fe0 con 0x7f8a800efe00 2021-02-10T11:14:09.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7e07b700 1 -- 172.21.2.101:0/468898815 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2079989004 0 0) 0x7f8a78001270 con 0x7f8a800efe00 2021-02-10T11:14:09.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7e07b700 1 -- 172.21.2.101:0/468898815 >> v2:172.21.2.108:3300/0 conn(0x7f8a801b0870 msgr2=0x7f8a800f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:09.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7e07b700 1 --2- 172.21.2.101:0/468898815 >> v2:172.21.2.108:3300/0 conn(0x7f8a801b0870 0x7f8a800f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:09.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7e07b700 1 -- 172.21.2.101:0/468898815 >> v1:172.21.2.108:6789/0 conn(0x7f8a801b0470 legacy=0x7f8a800ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:09.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7e07b700 1 -- 172.21.2.101:0/468898815 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a800e52e0 con 0x7f8a800efe00 2021-02-10T11:14:09.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7e07b700 1 -- 172.21.2.101:0/468898815 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f8a78001b50 con 0x7f8a800efe00 2021-02-10T11:14:09.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7e87c700 1 --2- 172.21.2.101:0/468898815 >> v2:172.21.2.108:3300/0 conn(0x7f8a801b0870 0x7f8a800f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:14:09.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7e07b700 1 -- 172.21.2.101:0/468898815 >> v1:172.21.2.101:6789/0 conn(0x7f8a800efe00 legacy=0x7f8a800f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:09.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7e07b700 1 --2- 172.21.2.101:0/468898815 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a6c003aa0 0x7f8a6c005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:09.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7e07b700 1 --2- 172.21.2.101:0/468898815 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a6c006790 0x7f8a6c008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:09.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7e07b700 1 --2- 172.21.2.101:0/468898815 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a6c009590 0x7f8a6c00bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:09.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7e07b700 1 -- 172.21.2.101:0/468898815 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c00c300 con 0x7f8a6c003aa0 2021-02-10T11:14:09.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7e07b700 1 -- 172.21.2.101:0/468898815 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c00c520 con 0x7f8a6c006790 2021-02-10T11:14:09.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7e07b700 1 -- 172.21.2.101:0/468898815 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c00c740 con 0x7f8a6c009590 2021-02-10T11:14:09.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7f07d700 1 --2- 172.21.2.101:0/468898815 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a6c003aa0 0x7f8a6c005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:09.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7e87c700 1 --2- 172.21.2.101:0/468898815 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a6c006790 0x7f8a6c008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:09.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7f87e700 1 --2- 172.21.2.101:0/468898815 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a6c009590 0x7f8a6c00bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:09.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7f07d700 1 -- 172.21.2.101:0/468898815 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a6c009590 msgr2=0x7f8a6c00bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:09.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7f07d700 1 --2- 172.21.2.101:0/468898815 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a6c009590 0x7f8a6c00bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:09.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7f07d700 1 -- 172.21.2.101:0/468898815 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a6c006790 msgr2=0x7f8a6c008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:09.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7f07d700 1 --2- 172.21.2.101:0/468898815 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a6c006790 0x7f8a6c008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:09.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7f07d700 1 -- 172.21.2.101:0/468898815 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c00c9e0 con 0x7f8a6c003aa0 2021-02-10T11:14:09.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7f87e700 1 -- 172.21.2.101:0/468898815 reap_dead start 2021-02-10T11:14:09.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7f07d700 1 --2- 172.21.2.101:0/468898815 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a6c003aa0 0x7f8a6c005ef0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f8a6804aac0 tx=0x7f8a6804daf0).ready entity=mon.0 client_cookie=58fb8a4ef07dc9b5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:09.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7e07b700 1 -- 172.21.2.101:0/468898815 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8a680022e0 con 0x7f8a6c003aa0 2021-02-10T11:14:09.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7e07b700 1 -- 172.21.2.101:0/468898815 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a68002560 con 0x7f8a6c003aa0 2021-02-10T11:14:09.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a7e07b700 1 -- 172.21.2.101:0/468898815 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8a68002a80 con 0x7f8a6c003aa0 2021-02-10T11:14:09.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a85301700 1 -- 172.21.2.101:0/468898815 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a6c003aa0 msgr2=0x7f8a6c005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:09.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.855 7f8a85301700 1 --2- 172.21.2.101:0/468898815 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a6c003aa0 0x7f8a6c005ef0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f8a6804aac0 tx=0x7f8a6804daf0).stop 2021-02-10T11:14:09.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.856 7f8a85301700 1 -- 172.21.2.101:0/468898815 shutdown_connections 2021-02-10T11:14:09.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.856 7f8a85301700 1 --2- 172.21.2.101:0/468898815 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a6c003aa0 0x7f8a6c005ef0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:09.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.856 7f8a85301700 1 -- 172.21.2.101:0/468898815 >> 172.21.2.101:0/468898815 conn(0x7f8a800ce3e0 msgr2=0x7f8a800e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:09.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.856 7f8a85301700 1 -- 172.21.2.101:0/468898815 shutdown_connections 2021-02-10T11:14:09.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.856 7f8a85301700 1 -- 172.21.2.101:0/468898815 wait complete. 2021-02-10T11:14:09.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.870 7f8a85301700 1 Processor -- start 2021-02-10T11:14:09.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.871 7f8a85301700 1 -- start start 2021-02-10T11:14:09.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.871 7f8a85301700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a800924b0 0x7f8a800928b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:09.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.871 7f8a85301700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a80092ed0 0x7f8a800932d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:09.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.871 7f8a85301700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a8008d4a0 0x7f8a8008d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:09.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.871 7f8a85301700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a80091020 con 0x7f8a8008d4a0 2021-02-10T11:14:09.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.871 7f8a85301700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8008dec0 con 0x7f8a800924b0 2021-02-10T11:14:09.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.871 7f8a85301700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8008e0b0 con 0x7f8a80092ed0 2021-02-10T11:14:09.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.871 7f8a7f87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a8008d4a0 0x7f8a8008d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:09.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.871 7f8a7f87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a8008d4a0 0x7f8a8008d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44834/0 (socket says 172.21.2.101:44834) 2021-02-10T11:14:09.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.871 7f8a7f87e700 1 -- 172.21.2.101:0/2973183340 learned_addr learned my addr 172.21.2.101:0/2973183340 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:09.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.872 7f8a7e87c700 1 --2- 172.21.2.101:0/2973183340 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a80092ed0 0x7f8a800932d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:09.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.872 7f8a7f87e700 1 -- 172.21.2.101:0/2973183340 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a80092ed0 msgr2=0x7f8a800932d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:09.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.872 7f8a7f87e700 1 --2- 172.21.2.101:0/2973183340 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a80092ed0 0x7f8a800932d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:09.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.872 7f8a7f87e700 1 -- 172.21.2.101:0/2973183340 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a800924b0 msgr2=0x7f8a800928b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:09.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.872 7f8a7f87e700 1 --2- 172.21.2.101:0/2973183340 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a800924b0 0x7f8a800928b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:09.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.872 7f8a7f87e700 1 -- 172.21.2.101:0/2973183340 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8008e2d0 con 0x7f8a8008d4a0 2021-02-10T11:14:09.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.872 7f8a7e87c700 1 --2- 172.21.2.101:0/2973183340 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a80092ed0 0x7f8a800932d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:14:09.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.872 7f8a7f87e700 1 --2- 172.21.2.101:0/2973183340 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a8008d4a0 0x7f8a8008d8a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f8a78007f90 tx=0x7f8a7800ac50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:09.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.872 7f8a77fff700 1 -- 172.21.2.101:0/2973183340 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8a78000e90 con 0x7f8a8008d4a0 2021-02-10T11:14:09.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.872 7f8a77fff700 1 -- 172.21.2.101:0/2973183340 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a780070d0 con 0x7f8a8008d4a0 2021-02-10T11:14:09.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.872 7f8a77fff700 1 -- 172.21.2.101:0/2973183340 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8a78004720 con 0x7f8a8008d4a0 2021-02-10T11:14:09.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.872 7f8a85301700 1 -- 172.21.2.101:0/2973183340 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8008e2d0 con 0x7f8a8008d4a0 2021-02-10T11:14:09.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.872 7f8a85301700 1 -- 172.21.2.101:0/2973183340 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8008e2d0 con 0x7f8a8008d4a0 2021-02-10T11:14:09.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.873 7f8a77fff700 1 -- 172.21.2.101:0/2973183340 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8a78000e20 con 0x7f8a8008d4a0 2021-02-10T11:14:09.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.873 7f8a77fff700 1 --2- 172.21.2.101:0/2973183340 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a640186c0 0x7f8a6401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:09.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.873 7f8a77fff700 1 -- 172.21.2.101:0/2973183340 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8a780327f0 con 0x7f8a8008d4a0 2021-02-10T11:14:09.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.873 7f8a7f07d700 1 --2- 172.21.2.101:0/2973183340 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a640186c0 0x7f8a6401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:09.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.873 7f8a7f07d700 1 --2- 172.21.2.101:0/2973183340 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a640186c0 0x7f8a6401abc0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:09.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.904 7f8a85301700 1 -- 172.21.2.101:0/2973183340 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a80105840 con 0x7f8a8008d4a0 2021-02-10T11:14:09.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:09.906 7f8a77fff700 1 -- 172.21.2.101:0/2973183340 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8a78007d00 con 0x7f8a8008d4a0 2021-02-10T11:14:10.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.032 7f8a85301700 1 -- 172.21.2.101:0/2973183340 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f8a80105840 con 0x7f8a8008d4a0 2021-02-10T11:14:10.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.033 7f8a77fff700 1 -- 172.21.2.101:0/2973183340 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v20) v1 ==== 104+0+10 (secure 0 0 0) 0x7f8a78000e20 con 0x7f8a8008d4a0 2021-02-10T11:14:10.038 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:14:10.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.034 7f8a85301700 1 -- 172.21.2.101:0/2973183340 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a640186c0 msgr2=0x7f8a6401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:10.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.034 7f8a85301700 1 --2- 172.21.2.101:0/2973183340 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a640186c0 0x7f8a6401abc0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:10.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.034 7f8a85301700 1 -- 172.21.2.101:0/2973183340 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a8008d4a0 msgr2=0x7f8a8008d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:10.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.034 7f8a85301700 1 --2- 172.21.2.101:0/2973183340 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a8008d4a0 0x7f8a8008d8a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f8a78007f90 tx=0x7f8a7800ac50).stop 2021-02-10T11:14:10.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.035 7f8a85301700 1 -- 172.21.2.101:0/2973183340 shutdown_connections 2021-02-10T11:14:10.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.035 7f8a85301700 1 --2- 172.21.2.101:0/2973183340 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a640186c0 0x7f8a6401abc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:10.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.035 7f8a85301700 1 --2- 172.21.2.101:0/2973183340 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a80092ed0 0x7f8a800932d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:10.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.035 7f8a85301700 1 --2- 172.21.2.101:0/2973183340 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a8008d4a0 0x7f8a8008d8a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:10.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.035 7f8a85301700 1 --2- 172.21.2.101:0/2973183340 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a800924b0 0x7f8a800928b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:10.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.035 7f8a85301700 1 -- 172.21.2.101:0/2973183340 >> 172.21.2.101:0/2973183340 conn(0x7f8a8009c070 msgr2=0x7f8a800a6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:10.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.035 7f8a85301700 1 -- 172.21.2.101:0/2973183340 shutdown_connections 2021-02-10T11:14:10.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.035 7f8a85301700 1 -- 172.21.2.101:0/2973183340 wait complete. 2021-02-10T11:14:10.047 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:14:10.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.456 7fc721b79700 1 Processor -- start 2021-02-10T11:14:10.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.456 7fc721b79700 1 -- start start 2021-02-10T11:14:10.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.456 7fc721b79700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc71c1b0470 0x7fc71c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:10.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.456 7fc721b79700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc71c0f20e0 0x7fc71c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:10.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.456 7fc721b79700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc71c0f0b50 con 0x7fc71c0ef610 2021-02-10T11:14:10.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.456 7fc721b79700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc71c0f0820 con 0x7fc71c1b0470 2021-02-10T11:14:10.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.456 7fc721b79700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc71c0f24e0 con 0x7fc71c0f20e0 2021-02-10T11:14:10.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.456 7fc71b7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc71c1b0470 0x7fc71c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:10.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.457 7fc71b7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc71c1b0470 0x7fc71c1b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44838/0 (socket says 172.21.2.101:44838) 2021-02-10T11:14:10.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.457 7fc71b7fe700 1 -- 172.21.2.101:0/130908694 learned_addr learned my addr 172.21.2.101:0/130908694 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:10.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.457 7fc71b7fe700 1 -- 172.21.2.101:0/130908694 >> v2:172.21.2.108:3300/0 conn(0x7fc71c0f20e0 msgr2=0x7fc71c0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:10.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.457 7fc71b7fe700 1 --2- 172.21.2.101:0/130908694 >> v2:172.21.2.108:3300/0 conn(0x7fc71c0f20e0 0x7fc71c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:10.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.457 7fc71b7fe700 1 -- 172.21.2.101:0/130908694 >> v1:172.21.2.108:6789/0 conn(0x7fc71c0ef610 legacy=0x7fc71c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:10.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.457 7fc71b7fe700 1 -- 172.21.2.101:0/130908694 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71c0e52e0 con 0x7fc71c1b0470 2021-02-10T11:14:10.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.457 7fc71b7fe700 1 --2- 172.21.2.101:0/130908694 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc71c1b0470 0x7fc71c1b0870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fc704006700 tx=0x7fc704009b80).ready entity=mon.0 client_cookie=1c9e275633d90183 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:10.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.457 7fc71a7fc700 1 -- 172.21.2.101:0/130908694 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc70400c0c0 con 0x7fc71c1b0470 2021-02-10T11:14:10.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.457 7fc71a7fc700 1 -- 172.21.2.101:0/130908694 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70400c340 con 0x7fc71c1b0470 2021-02-10T11:14:10.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.457 7fc71a7fc700 1 -- 172.21.2.101:0/130908694 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc70400c860 con 0x7fc71c1b0470 2021-02-10T11:14:10.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.457 7fc721b79700 1 -- 172.21.2.101:0/130908694 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc71c1b0470 msgr2=0x7fc71c1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:10.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.457 7fc721b79700 1 --2- 172.21.2.101:0/130908694 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc71c1b0470 0x7fc71c1b0870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fc704006700 tx=0x7fc704009b80).stop 2021-02-10T11:14:10.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.458 7fc721b79700 1 -- 172.21.2.101:0/130908694 shutdown_connections 2021-02-10T11:14:10.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.458 7fc721b79700 1 --2- 172.21.2.101:0/130908694 >> v2:172.21.2.108:3300/0 conn(0x7fc71c0f20e0 0x7fc71c0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:10.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.458 7fc721b79700 1 --2- 172.21.2.101:0/130908694 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc71c1b0470 0x7fc71c1b0870 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:10.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.458 7fc721b79700 1 -- 172.21.2.101:0/130908694 >> 172.21.2.101:0/130908694 conn(0x7fc71c0ce3e0 msgr2=0x7fc71c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:10.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.458 7fc721b79700 1 -- 172.21.2.101:0/130908694 shutdown_connections 2021-02-10T11:14:10.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.458 7fc721b79700 1 -- 172.21.2.101:0/130908694 wait complete. 2021-02-10T11:14:10.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.473 7fc721b79700 1 Processor -- start 2021-02-10T11:14:10.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.473 7fc721b79700 1 -- start start 2021-02-10T11:14:10.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.473 7fc721b79700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc71c0c36a0 0x7fc71c0c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:10.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.473 7fc721b79700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc71c0c3190 0x7fc71c0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:10.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.473 7fc721b79700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc71c0b2370 0x7fc71c0b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:10.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.473 7fc721b79700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c0b27c0 con 0x7fc71c0c3190 2021-02-10T11:14:10.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.473 7fc721b79700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c0b3360 con 0x7fc71c0b2370 2021-02-10T11:14:10.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.473 7fc721b79700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c0b3540 con 0x7fc71c0c36a0 2021-02-10T11:14:10.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.473 7fc71affd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc71c0c3190 0x7fc71c0b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:10.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.473 7fc71affd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc71c0c3190 0x7fc71c0b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44842/0 (socket says 172.21.2.101:44842) 2021-02-10T11:14:10.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.473 7fc71affd700 1 -- 172.21.2.101:0/230968558 learned_addr learned my addr 172.21.2.101:0/230968558 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:10.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.474 7fc71affd700 1 -- 172.21.2.101:0/230968558 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc71c0c36a0 msgr2=0x7fc71c0c2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:10.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.474 7fc71affd700 1 --2- 172.21.2.101:0/230968558 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc71c0c36a0 0x7fc71c0c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:10.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.474 7fc71affd700 1 -- 172.21.2.101:0/230968558 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc71c0b2370 msgr2=0x7fc71c0b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:10.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.474 7fc71affd700 1 --2- 172.21.2.101:0/230968558 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc71c0b2370 0x7fc71c0b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:10.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.474 7fc71affd700 1 -- 172.21.2.101:0/230968558 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71c0b3760 con 0x7fc71c0c3190 2021-02-10T11:14:10.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.474 7fc71affd700 1 --2- 172.21.2.101:0/230968558 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc71c0c3190 0x7fc71c0b1d50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fc70c00b5b0 tx=0x7fc70c00b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:10.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.474 7fc713fff700 1 -- 172.21.2.101:0/230968558 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc70c0009b0 con 0x7fc71c0c3190 2021-02-10T11:14:10.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.474 7fc713fff700 1 -- 172.21.2.101:0/230968558 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70c00bd40 con 0x7fc71c0c3190 2021-02-10T11:14:10.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.474 7fc713fff700 1 -- 172.21.2.101:0/230968558 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc70c006210 con 0x7fc71c0c3190 2021-02-10T11:14:10.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.474 7fc721b79700 1 -- 172.21.2.101:0/230968558 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc71c0b3760 con 0x7fc71c0c3190 2021-02-10T11:14:10.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.474 7fc721b79700 1 -- 172.21.2.101:0/230968558 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71c0b3cf0 con 0x7fc71c0c3190 2021-02-10T11:14:10.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.475 7fc713fff700 1 -- 172.21.2.101:0/230968558 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc70c006020 con 0x7fc71c0c3190 2021-02-10T11:14:10.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.475 7fc713fff700 1 --2- 172.21.2.101:0/230968558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7000186c0 0x7fc70001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:10.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.475 7fc713fff700 1 -- 172.21.2.101:0/230968558 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc70c032570 con 0x7fc71c0c3190 2021-02-10T11:14:10.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.475 7fc71b7fe700 1 --2- 172.21.2.101:0/230968558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7000186c0 0x7fc70001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:10.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.475 7fc71b7fe700 1 --2- 172.21.2.101:0/230968558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7000186c0 0x7fc70001abc0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:10.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.506 7fc721b79700 1 -- 172.21.2.101:0/230968558 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc71c0edd10 con 0x7fc71c0c3190 2021-02-10T11:14:10.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.509 7fc713fff700 1 -- 172.21.2.101:0/230968558 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc70c0009b0 con 0x7fc71c0c3190 2021-02-10T11:14:10.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.637 7fc721b79700 1 -- 172.21.2.101:0/230968558 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7fc71c0e52e0 con 0x7fc71c0c3190 2021-02-10T11:14:10.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.637 7fc713fff700 1 -- 172.21.2.101:0/230968558 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v20) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc70c006000 con 0x7fc71c0c3190 2021-02-10T11:14:10.642 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:14:10.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.639 7fc721b79700 1 -- 172.21.2.101:0/230968558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7000186c0 msgr2=0x7fc70001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:10.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.639 7fc721b79700 1 --2- 172.21.2.101:0/230968558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7000186c0 0x7fc70001abc0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:10.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.639 7fc721b79700 1 -- 172.21.2.101:0/230968558 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc71c0c3190 msgr2=0x7fc71c0b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:10.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.639 7fc721b79700 1 --2- 172.21.2.101:0/230968558 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc71c0c3190 0x7fc71c0b1d50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fc70c00b5b0 tx=0x7fc70c00b950).stop 2021-02-10T11:14:10.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.639 7fc721b79700 1 -- 172.21.2.101:0/230968558 shutdown_connections 2021-02-10T11:14:10.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.639 7fc721b79700 1 --2- 172.21.2.101:0/230968558 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc71c0b2370 0x7fc71c0b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:10.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.639 7fc721b79700 1 --2- 172.21.2.101:0/230968558 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc71c0c3190 0x7fc71c0b1d50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:10.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.639 7fc721b79700 1 --2- 172.21.2.101:0/230968558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7000186c0 0x7fc70001abc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:10.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.639 7fc721b79700 1 --2- 172.21.2.101:0/230968558 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc71c0c36a0 0x7fc71c0c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:10.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.639 7fc721b79700 1 -- 172.21.2.101:0/230968558 >> 172.21.2.101:0/230968558 conn(0x7fc71c09f630 msgr2=0x7fc71c0a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:10.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.640 7fc721b79700 1 -- 172.21.2.101:0/230968558 shutdown_connections 2021-02-10T11:14:10.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:10.640 7fc721b79700 1 -- 172.21.2.101:0/230968558 wait complete. 2021-02-10T11:14:10.653 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:14:11.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.058 7fb919435700 1 Processor -- start 2021-02-10T11:14:11.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.058 7fb919435700 1 -- start start 2021-02-10T11:14:11.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.058 7fb919435700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb9140e5070 0x7fb9141b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:11.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.058 7fb919435700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9140e5470 0x7fb914080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:11.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.058 7fb919435700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb91407ed80 con 0x7fb9141b3e40 2021-02-10T11:14:11.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.059 7fb919435700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9140d1790 con 0x7fb9140e5070 2021-02-10T11:14:11.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.059 7fb919435700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9140d19b0 con 0x7fb9140e5470 2021-02-10T11:14:11.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.059 7fb91307d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb9140e5070 0x7fb9141b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:11.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.059 7fb91307d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb9140e5070 0x7fb9141b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44850/0 (socket says 172.21.2.101:44850) 2021-02-10T11:14:11.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.059 7fb91307d700 1 -- 172.21.2.101:0/7087172 learned_addr learned my addr 172.21.2.101:0/7087172 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:11.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.059 7fb91307d700 1 -- 172.21.2.101:0/7087172 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9140e5470 msgr2=0x7fb914080310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:11.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.059 7fb91307d700 1 --2- 172.21.2.101:0/7087172 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9140e5470 0x7fb914080310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:11.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.059 7fb91307d700 1 -- 172.21.2.101:0/7087172 >> v1:172.21.2.108:6789/0 conn(0x7fb9141b3e40 legacy=0x7fb91407d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:11.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.059 7fb91307d700 1 -- 172.21.2.101:0/7087172 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9140d1bd0 con 0x7fb9140e5070 2021-02-10T11:14:11.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.059 7fb91307d700 1 --2- 172.21.2.101:0/7087172 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9140e5070 0x7fb9141b3820 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fb8fc006700 tx=0x7fb8fc009b80).ready entity=mon.0 client_cookie=fae60a55bdccd25b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:11.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.060 7fb91207b700 1 -- 172.21.2.101:0/7087172 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb8fc00c0c0 con 0x7fb9140e5070 2021-02-10T11:14:11.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.060 7fb91207b700 1 -- 172.21.2.101:0/7087172 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8fc00c340 con 0x7fb9140e5070 2021-02-10T11:14:11.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.060 7fb91207b700 1 -- 172.21.2.101:0/7087172 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb8fc00c860 con 0x7fb9140e5070 2021-02-10T11:14:11.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.060 7fb919435700 1 -- 172.21.2.101:0/7087172 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9140e5070 msgr2=0x7fb9141b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:11.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.060 7fb919435700 1 --2- 172.21.2.101:0/7087172 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9140e5070 0x7fb9141b3820 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fb8fc006700 tx=0x7fb8fc009b80).stop 2021-02-10T11:14:11.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.060 7fb919435700 1 -- 172.21.2.101:0/7087172 shutdown_connections 2021-02-10T11:14:11.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.060 7fb919435700 1 --2- 172.21.2.101:0/7087172 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9140e5470 0x7fb914080310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:11.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.060 7fb919435700 1 --2- 172.21.2.101:0/7087172 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9140e5070 0x7fb9141b3820 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:11.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.060 7fb919435700 1 -- 172.21.2.101:0/7087172 >> 172.21.2.101:0/7087172 conn(0x7fb9140ce690 msgr2=0x7fb9140e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:11.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.060 7fb919435700 1 -- 172.21.2.101:0/7087172 shutdown_connections 2021-02-10T11:14:11.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.060 7fb919435700 1 -- 172.21.2.101:0/7087172 wait complete. 2021-02-10T11:14:11.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.075 7fb919435700 1 Processor -- start 2021-02-10T11:14:11.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.075 7fb919435700 1 -- start start 2021-02-10T11:14:11.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.075 7fb919435700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9141aa290 0x7fb9141aa690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:11.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.075 7fb919435700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9140b7270 0x7fb9140b7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:11.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.075 7fb919435700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9140b7c90 0x7fb914084e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:11.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.075 7fb919435700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9140b8090 con 0x7fb9140b7c90 2021-02-10T11:14:11.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.075 7fb919435700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9141a9df0 con 0x7fb9140b7270 2021-02-10T11:14:11.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.075 7fb919435700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb91409cc30 con 0x7fb9141aa290 2021-02-10T11:14:11.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.075 7fb91387e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9140b7c90 0x7fb914084e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:11.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.076 7fb91387e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9140b7c90 0x7fb914084e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44858/0 (socket says 172.21.2.101:44858) 2021-02-10T11:14:11.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.076 7fb91387e700 1 -- 172.21.2.101:0/2323714154 learned_addr learned my addr 172.21.2.101:0/2323714154 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:11.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.076 7fb91307d700 1 --2- 172.21.2.101:0/2323714154 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9141aa290 0x7fb9141aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:11.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.076 7fb91387e700 1 -- 172.21.2.101:0/2323714154 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9141aa290 msgr2=0x7fb9141aa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:11.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.076 7fb91387e700 1 --2- 172.21.2.101:0/2323714154 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9141aa290 0x7fb9141aa690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:11.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.076 7fb91387e700 1 -- 172.21.2.101:0/2323714154 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9140b7270 msgr2=0x7fb9140b7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:11.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.076 7fb91287c700 1 --2- 172.21.2.101:0/2323714154 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9140b7270 0x7fb9140b7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:11.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.076 7fb91387e700 1 --2- 172.21.2.101:0/2323714154 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9140b7270 0x7fb9140b7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:11.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.076 7fb91387e700 1 -- 172.21.2.101:0/2323714154 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb914085430 con 0x7fb9140b7c90 2021-02-10T11:14:11.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.076 7fb91307d700 1 --2- 172.21.2.101:0/2323714154 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9141aa290 0x7fb9141aa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:14:11.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.076 7fb91287c700 1 --2- 172.21.2.101:0/2323714154 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9140b7270 0x7fb9140b7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:14:11.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.076 7fb91387e700 1 --2- 172.21.2.101:0/2323714154 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9140b7c90 0x7fb914084e10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fb90c009980 tx=0x7fb90c00bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:11.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.076 7fb90bfff700 1 -- 172.21.2.101:0/2323714154 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb90c005870 con 0x7fb9140b7c90 2021-02-10T11:14:11.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.076 7fb90bfff700 1 -- 172.21.2.101:0/2323714154 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb90c005af0 con 0x7fb9140b7c90 2021-02-10T11:14:11.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.076 7fb90bfff700 1 -- 172.21.2.101:0/2323714154 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb90c006030 con 0x7fb9140b7c90 2021-02-10T11:14:11.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.076 7fb919435700 1 -- 172.21.2.101:0/2323714154 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb914085430 con 0x7fb9140b7c90 2021-02-10T11:14:11.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.076 7fb919435700 1 -- 172.21.2.101:0/2323714154 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb914085430 con 0x7fb9140b7c90 2021-02-10T11:14:11.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.077 7fb90bfff700 1 -- 172.21.2.101:0/2323714154 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb90c005cd0 con 0x7fb9140b7c90 2021-02-10T11:14:11.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.077 7fb90bfff700 1 --2- 172.21.2.101:0/2323714154 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8f80186d0 0x7fb8f801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:11.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.077 7fb90bfff700 1 -- 172.21.2.101:0/2323714154 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb90c0329c0 con 0x7fb9140b7c90 2021-02-10T11:14:11.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.077 7fb91307d700 1 --2- 172.21.2.101:0/2323714154 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8f80186d0 0x7fb8f801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:11.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.077 7fb91307d700 1 --2- 172.21.2.101:0/2323714154 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8f80186d0 0x7fb8f801abd0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:11.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.108 7fb919435700 1 -- 172.21.2.101:0/2323714154 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9140b8090 con 0x7fb9140b7c90 2021-02-10T11:14:11.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.111 7fb90bfff700 1 -- 172.21.2.101:0/2323714154 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb90c00be10 con 0x7fb9140b7c90 2021-02-10T11:14:11.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.236 7fb919435700 1 -- 172.21.2.101:0/2323714154 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9140b8090 con 0x7fb9140b7c90 2021-02-10T11:14:11.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.237 7fb90bfff700 1 -- 172.21.2.101:0/2323714154 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+13993 (secure 0 0 0) 0x7fb90c006f20 con 0x7fb9140b7c90 2021-02-10T11:14:11.242 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:14:11.242 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":20,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:13:15.149234","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:13:11.922272","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2021-02-10 11:13:13.141733","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:14:11.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.238 7fb919435700 1 -- 172.21.2.101:0/2323714154 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8f80186d0 msgr2=0x7fb8f801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:11.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.238 7fb919435700 1 --2- 172.21.2.101:0/2323714154 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8f80186d0 0x7fb8f801abd0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:11.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.238 7fb919435700 1 -- 172.21.2.101:0/2323714154 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9140b7c90 msgr2=0x7fb914084e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:11.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.238 7fb919435700 1 --2- 172.21.2.101:0/2323714154 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9140b7c90 0x7fb914084e10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fb90c009980 tx=0x7fb90c00bcd0).stop 2021-02-10T11:14:11.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.239 7fb919435700 1 -- 172.21.2.101:0/2323714154 shutdown_connections 2021-02-10T11:14:11.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.239 7fb919435700 1 --2- 172.21.2.101:0/2323714154 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9140b7c90 0x7fb914084e10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:11.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.239 7fb919435700 1 --2- 172.21.2.101:0/2323714154 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9140b7270 0x7fb9140b7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:11.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.239 7fb919435700 1 --2- 172.21.2.101:0/2323714154 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8f80186d0 0x7fb8f801abd0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:11.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.239 7fb919435700 1 --2- 172.21.2.101:0/2323714154 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9141aa290 0x7fb9141aa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:11.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.239 7fb919435700 1 -- 172.21.2.101:0/2323714154 >> 172.21.2.101:0/2323714154 conn(0x7fb91408fe10 msgr2=0x7fb91419c020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:11.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.239 7fb919435700 1 -- 172.21.2.101:0/2323714154 shutdown_connections 2021-02-10T11:14:11.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.239 7fb919435700 1 -- 172.21.2.101:0/2323714154 wait complete. 2021-02-10T11:14:11.252 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:13:11.922272', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:13:13.141733', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:14:11.253 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:14:11.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.661 7fd1275f0700 1 Processor -- start 2021-02-10T11:14:11.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.661 7fd1275f0700 1 -- start start 2021-02-10T11:14:11.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.662 7fd1275f0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd1201b0470 0x7fd1201b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:11.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.662 7fd1275f0700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd1200ef610 0x7fd1200efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:11.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.662 7fd1275f0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1200f0b50 con 0x7fd1200f0030 2021-02-10T11:14:11.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.662 7fd1275f0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1200e52e0 con 0x7fd1201b0470 2021-02-10T11:14:11.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.662 7fd1275f0700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1200e5490 con 0x7fd1200ef610 2021-02-10T11:14:11.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.662 7fd12540c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd1201b0470 0x7fd1201b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:11.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.662 7fd12540c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd1201b0470 0x7fd1201b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44862/0 (socket says 172.21.2.101:44862) 2021-02-10T11:14:11.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.662 7fd12540c700 1 -- 172.21.2.101:0/2327448047 learned_addr learned my addr 172.21.2.101:0/2327448047 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:11.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.662 7fd12540c700 1 -- 172.21.2.101:0/2327448047 >> v2:172.21.2.108:3300/0 conn(0x7fd1200ef610 msgr2=0x7fd1200efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:11.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.662 7fd12540c700 1 --2- 172.21.2.101:0/2327448047 >> v2:172.21.2.108:3300/0 conn(0x7fd1200ef610 0x7fd1200efa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:11.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.662 7fd12540c700 1 -- 172.21.2.101:0/2327448047 >> v1:172.21.2.108:6789/0 conn(0x7fd1200f0030 legacy=0x7fd1200f10d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:11.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.662 7fd12540c700 1 -- 172.21.2.101:0/2327448047 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1200f0430 con 0x7fd1201b0470 2021-02-10T11:14:11.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.663 7fd12540c700 1 --2- 172.21.2.101:0/2327448047 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1201b0470 0x7fd1201b0870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fd114006700 tx=0x7fd114009b00).ready entity=mon.0 client_cookie=347f8ab870db3bfb server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:11.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.663 7fd113fff700 1 -- 172.21.2.101:0/2327448047 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd11400c0c0 con 0x7fd1201b0470 2021-02-10T11:14:11.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.663 7fd113fff700 1 -- 172.21.2.101:0/2327448047 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11400c340 con 0x7fd1201b0470 2021-02-10T11:14:11.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.663 7fd113fff700 1 -- 172.21.2.101:0/2327448047 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd11400c820 con 0x7fd1201b0470 2021-02-10T11:14:11.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.663 7fd1275f0700 1 -- 172.21.2.101:0/2327448047 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1201b0470 msgr2=0x7fd1201b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:11.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.663 7fd1275f0700 1 --2- 172.21.2.101:0/2327448047 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1201b0470 0x7fd1201b0870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fd114006700 tx=0x7fd114009b00).stop 2021-02-10T11:14:11.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.663 7fd1275f0700 1 -- 172.21.2.101:0/2327448047 shutdown_connections 2021-02-10T11:14:11.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.663 7fd1275f0700 1 --2- 172.21.2.101:0/2327448047 >> v2:172.21.2.108:3300/0 conn(0x7fd1200ef610 0x7fd1200efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:11.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.663 7fd1275f0700 1 --2- 172.21.2.101:0/2327448047 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1201b0470 0x7fd1201b0870 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:11.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.663 7fd1275f0700 1 -- 172.21.2.101:0/2327448047 >> 172.21.2.101:0/2327448047 conn(0x7fd1200ce3e0 msgr2=0x7fd1200e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:11.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.664 7fd1275f0700 1 -- 172.21.2.101:0/2327448047 shutdown_connections 2021-02-10T11:14:11.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.664 7fd1275f0700 1 -- 172.21.2.101:0/2327448047 wait complete. 2021-02-10T11:14:11.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.678 7fd1275f0700 1 Processor -- start 2021-02-10T11:14:11.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.678 7fd1275f0700 1 -- start start 2021-02-10T11:14:11.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.678 7fd1275f0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1200b2500 0x7fd1200b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:11.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.678 7fd1275f0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1200b3080 0x7fd1200b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:11.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.678 7fd1275f0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1200b3aa0 0x7fd1200f0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:11.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.678 7fd1275f0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1200effb0 con 0x7fd1200b3aa0 2021-02-10T11:14:11.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.678 7fd1275f0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1200f0b50 con 0x7fd1200b3080 2021-02-10T11:14:11.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.678 7fd1275f0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1200f0d30 con 0x7fd1200b2500 2021-02-10T11:14:11.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.679 7fd125c0d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1200b3aa0 0x7fd1200f0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:11.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.679 7fd125c0d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1200b3aa0 0x7fd1200f0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44866/0 (socket says 172.21.2.101:44866) 2021-02-10T11:14:11.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.679 7fd125c0d700 1 -- 172.21.2.101:0/4154146473 learned_addr learned my addr 172.21.2.101:0/4154146473 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:11.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.679 7fd125c0d700 1 -- 172.21.2.101:0/4154146473 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1200b2500 msgr2=0x7fd1200b2900 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:11.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.679 7fd125c0d700 1 --2- 172.21.2.101:0/4154146473 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1200b2500 0x7fd1200b2900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:11.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.679 7fd125c0d700 1 -- 172.21.2.101:0/4154146473 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1200b3080 msgr2=0x7fd1200b3480 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:11.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.679 7fd125c0d700 1 --2- 172.21.2.101:0/4154146473 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1200b3080 0x7fd1200b3480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:11.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.679 7fd125c0d700 1 -- 172.21.2.101:0/4154146473 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1200f0f50 con 0x7fd1200b3aa0 2021-02-10T11:14:11.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.679 7fd125c0d700 1 --2- 172.21.2.101:0/4154146473 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1200b3aa0 0x7fd1200f0530 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fd118009940 tx=0x7fd11800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:11.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.679 7fd111ffb700 1 -- 172.21.2.101:0/4154146473 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd118003030 con 0x7fd1200b3aa0 2021-02-10T11:14:11.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.679 7fd111ffb700 1 -- 172.21.2.101:0/4154146473 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd118001270 con 0x7fd1200b3aa0 2021-02-10T11:14:11.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.679 7fd111ffb700 1 -- 172.21.2.101:0/4154146473 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd118001790 con 0x7fd1200b3aa0 2021-02-10T11:14:11.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.679 7fd1275f0700 1 -- 172.21.2.101:0/4154146473 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1200f0f50 con 0x7fd1200b3aa0 2021-02-10T11:14:11.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.679 7fd1275f0700 1 -- 172.21.2.101:0/4154146473 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1200f14e0 con 0x7fd1200b3aa0 2021-02-10T11:14:11.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.680 7fd111ffb700 1 -- 172.21.2.101:0/4154146473 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd11800a000 con 0x7fd1200b3aa0 2021-02-10T11:14:11.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.680 7fd111ffb700 1 --2- 172.21.2.101:0/4154146473 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1000186c0 0x7fd10001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:11.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.680 7fd111ffb700 1 -- 172.21.2.101:0/4154146473 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd1180328d0 con 0x7fd1200b3aa0 2021-02-10T11:14:11.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.680 7fd12540c700 1 --2- 172.21.2.101:0/4154146473 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1000186c0 0x7fd10001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:11.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.680 7fd12540c700 1 --2- 172.21.2.101:0/4154146473 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1000186c0 0x7fd10001abc0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:11.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.711 7fd1275f0700 1 -- 172.21.2.101:0/4154146473 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1200b3ef0 con 0x7fd1200b3aa0 2021-02-10T11:14:11.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.714 7fd111ffb700 1 -- 172.21.2.101:0/4154146473 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd118000f90 con 0x7fd1200b3aa0 2021-02-10T11:14:11.783 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:14:11.786 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:14:11.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.844 7fd1275f0700 1 -- 172.21.2.101:0/4154146473 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fd1200b3ef0 con 0x7fd1200b3aa0 2021-02-10T11:14:11.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.844 7fd111ffb700 1 -- 172.21.2.101:0/4154146473 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd11801f860 con 0x7fd1200b3aa0 2021-02-10T11:14:11.849 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:14:11.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.846 7fd1275f0700 1 -- 172.21.2.101:0/4154146473 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1000186c0 msgr2=0x7fd10001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:11.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.846 7fd1275f0700 1 --2- 172.21.2.101:0/4154146473 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1000186c0 0x7fd10001abc0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:11.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.846 7fd1275f0700 1 -- 172.21.2.101:0/4154146473 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1200b3aa0 msgr2=0x7fd1200f0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:11.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.846 7fd1275f0700 1 --2- 172.21.2.101:0/4154146473 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1200b3aa0 0x7fd1200f0530 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fd118009940 tx=0x7fd11800bcf0).stop 2021-02-10T11:14:11.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.847 7fd1275f0700 1 -- 172.21.2.101:0/4154146473 shutdown_connections 2021-02-10T11:14:11.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.847 7fd1275f0700 1 --2- 172.21.2.101:0/4154146473 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1200b3aa0 0x7fd1200f0530 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:11.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.847 7fd1275f0700 1 --2- 172.21.2.101:0/4154146473 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1200b3080 0x7fd1200b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:11.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.847 7fd1275f0700 1 --2- 172.21.2.101:0/4154146473 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1000186c0 0x7fd10001abc0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:11.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.847 7fd1275f0700 1 --2- 172.21.2.101:0/4154146473 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1200b2500 0x7fd1200b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:11.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.847 7fd1275f0700 1 -- 172.21.2.101:0/4154146473 >> 172.21.2.101:0/4154146473 conn(0x7fd1200c3c70 msgr2=0x7fd120094c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:11.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.847 7fd1275f0700 1 -- 172.21.2.101:0/4154146473 shutdown_connections 2021-02-10T11:14:11.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:11.847 7fd1275f0700 1 -- 172.21.2.101:0/4154146473 wait complete. 2021-02-10T11:14:11.859 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:14:12.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.191 7f26dc33d700 1 Processor -- start 2021-02-10T11:14:12.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.191 7f26dc33d700 1 -- start start 2021-02-10T11:14:12.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.192 7f26dc33d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26d41b0470 0x7f26d41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:12.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.192 7f26dc33d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f26d40f20e0 0x7f26d40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:12.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.192 7f26dc33d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26d40f0b50 con 0x7f26d40ef610 2021-02-10T11:14:12.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.192 7f26dc33d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26d40f0820 con 0x7f26d40f20e0 2021-02-10T11:14:12.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.192 7f26dc33d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d40f24e0 con 0x7f26d41b0470 2021-02-10T11:14:12.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.192 7f26d9958700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f26d40ef610 0x7f26d40efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45368/0 (socket says 172.21.2.101:45368) 2021-02-10T11:14:12.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.192 7f26d9958700 1 -- 172.21.2.101:0/2299165588 learned_addr learned my addr 172.21.2.101:0/2299165588 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:14:12.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.193 7f26d9157700 1 -- 172.21.2.101:0/2299165588 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3319187121 0 0) 0x7f26c40010b0 con 0x7f26d40ef610 2021-02-10T11:14:12.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.193 7f26d9157700 1 -- 172.21.2.101:0/2299165588 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26c0002fe0 con 0x7f26d40ef610 2021-02-10T11:14:12.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.193 7f26da159700 1 --2- 172.21.2.101:0/2299165588 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26d41b0470 0x7f26d41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:12.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.193 7f26da95a700 1 --2- 172.21.2.101:0/2299165588 >> v2:172.21.2.108:3300/0 conn(0x7f26d40f20e0 0x7f26d40f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:12.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.193 7f26da159700 1 -- 172.21.2.101:0/2299165588 >> v2:172.21.2.108:3300/0 conn(0x7f26d40f20e0 msgr2=0x7f26d40f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:12.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.193 7f26da159700 1 --2- 172.21.2.101:0/2299165588 >> v2:172.21.2.108:3300/0 conn(0x7f26d40f20e0 0x7f26d40f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:12.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.193 7f26da159700 1 -- 172.21.2.101:0/2299165588 >> v1:172.21.2.101:6789/0 conn(0x7f26d40ef610 legacy=0x7f26d40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:12.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.193 7f26da159700 1 -- 172.21.2.101:0/2299165588 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d40e52e0 con 0x7f26d41b0470 2021-02-10T11:14:12.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.193 7f26da159700 1 --2- 172.21.2.101:0/2299165588 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26d41b0470 0x7f26d41b0870 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f26d0006650 tx=0x7f26d0009a80).ready entity=mon.2 client_cookie=cb69e9a5ff30dd20 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:12.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.194 7f26d9157700 1 -- 172.21.2.101:0/2299165588 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f26d00009b0 con 0x7f26d41b0470 2021-02-10T11:14:12.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.194 7f26d9157700 1 -- 172.21.2.101:0/2299165588 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d000c0c0 con 0x7f26d41b0470 2021-02-10T11:14:12.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.194 7f26d9157700 1 -- 172.21.2.101:0/2299165588 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f26d000c5e0 con 0x7f26d41b0470 2021-02-10T11:14:12.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.194 7f26dc33d700 1 -- 172.21.2.101:0/2299165588 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26d41b0470 msgr2=0x7f26d41b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:12.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.194 7f26dc33d700 1 --2- 172.21.2.101:0/2299165588 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26d41b0470 0x7f26d41b0870 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f26d0006650 tx=0x7f26d0009a80).stop 2021-02-10T11:14:12.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.195 7f26dc33d700 1 -- 172.21.2.101:0/2299165588 shutdown_connections 2021-02-10T11:14:12.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.195 7f26dc33d700 1 --2- 172.21.2.101:0/2299165588 >> v2:172.21.2.108:3300/0 conn(0x7f26d40f20e0 0x7f26d40f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:12.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.195 7f26dc33d700 1 --2- 172.21.2.101:0/2299165588 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26d41b0470 0x7f26d41b0870 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:12.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.195 7f26dc33d700 1 -- 172.21.2.101:0/2299165588 >> 172.21.2.101:0/2299165588 conn(0x7f26d40ce3e0 msgr2=0x7f26d40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:12.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.195 7f26dc33d700 1 -- 172.21.2.101:0/2299165588 shutdown_connections 2021-02-10T11:14:12.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.195 7f26dc33d700 1 -- 172.21.2.101:0/2299165588 wait complete. 2021-02-10T11:14:12.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.210 7f26dc33d700 1 Processor -- start 2021-02-10T11:14:12.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.210 7f26dc33d700 1 -- start start 2021-02-10T11:14:12.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.210 7f26dc33d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26d4192ad0 0x7f26d4192ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:12.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.210 7f26dc33d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26d40a78f0 0x7f26d40a7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:12.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.210 7f26dc33d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f26d40a8310 0x7f26d40a9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:12.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.210 7f26dc33d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d40a8ae0 con 0x7f26d40a78f0 2021-02-10T11:14:12.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.210 7f26dc33d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d40a8710 con 0x7f26d40a8310 2021-02-10T11:14:12.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.210 7f26dc33d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d40a88f0 con 0x7f26d4192ad0 2021-02-10T11:14:12.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.211 7f26d9958700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26d40a78f0 0x7f26d40a7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:12.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.211 7f26d9958700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26d40a78f0 0x7f26d40a7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44880/0 (socket says 172.21.2.101:44880) 2021-02-10T11:14:12.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.211 7f26d9958700 1 -- 172.21.2.101:0/3363803762 learned_addr learned my addr 172.21.2.101:0/3363803762 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:12.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.211 7f26da95a700 1 --2- 172.21.2.101:0/3363803762 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f26d40a8310 0x7f26d40a9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.211 7f26da159700 1 --2- 172.21.2.101:0/3363803762 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26d4192ad0 0x7f26d4192ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.211 7f26d9958700 1 -- 172.21.2.101:0/3363803762 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26d4192ad0 msgr2=0x7f26d4192ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.211 7f26d9958700 1 --2- 172.21.2.101:0/3363803762 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26d4192ad0 0x7f26d4192ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.211 7f26d9958700 1 -- 172.21.2.101:0/3363803762 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f26d40a8310 msgr2=0x7f26d40a9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.211 7f26d9958700 1 --2- 172.21.2.101:0/3363803762 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f26d40a8310 0x7f26d40a9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.211 7f26d9958700 1 -- 172.21.2.101:0/3363803762 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d40a9680 con 0x7f26d40a78f0 2021-02-10T11:14:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.211 7f26da95a700 1 --2- 172.21.2.101:0/3363803762 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f26d40a8310 0x7f26d40a9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:14:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.211 7f26d9958700 1 --2- 172.21.2.101:0/3363803762 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26d40a78f0 0x7f26d40a7cf0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f26c4001c60 tx=0x7f26c400a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.211 7f26caffd700 1 -- 172.21.2.101:0/3363803762 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f26c400acc0 con 0x7f26d40a78f0 2021-02-10T11:14:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.211 7f26caffd700 1 -- 172.21.2.101:0/3363803762 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c4004140 con 0x7f26d40a78f0 2021-02-10T11:14:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.211 7f26dc33d700 1 -- 172.21.2.101:0/3363803762 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d40a9680 con 0x7f26d40a78f0 2021-02-10T11:14:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.211 7f26caffd700 1 -- 172.21.2.101:0/3363803762 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f26c4004620 con 0x7f26d40a78f0 2021-02-10T11:14:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.211 7f26dc33d700 1 -- 172.21.2.101:0/3363803762 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d40a9680 con 0x7f26d40a78f0 2021-02-10T11:14:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.212 7f26caffd700 1 -- 172.21.2.101:0/3363803762 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f26c4004320 con 0x7f26d40a78f0 2021-02-10T11:14:12.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.212 7f26caffd700 1 --2- 172.21.2.101:0/3363803762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26bc018730 0x7f26bc01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:12.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.212 7f26caffd700 1 -- 172.21.2.101:0/3363803762 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f26c4032610 con 0x7f26d40a78f0 2021-02-10T11:14:12.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.212 7f26da159700 1 --2- 172.21.2.101:0/3363803762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26bc018730 0x7f26bc01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:12.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.212 7f26da159700 1 --2- 172.21.2.101:0/3363803762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26bc018730 0x7f26bc01ac10 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:12.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.242 7f26dc33d700 1 -- 172.21.2.101:0/3363803762 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d40f0b50 con 0x7f26d40a78f0 2021-02-10T11:14:12.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.244 7f26caffd700 1 -- 172.21.2.101:0/3363803762 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f26c400acc0 con 0x7f26d40a78f0 2021-02-10T11:14:12.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.373 7f26dc33d700 1 -- 172.21.2.101:0/3363803762 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f26d40f0b50 con 0x7f26d40a78f0 2021-02-10T11:14:12.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.373 7f26caffd700 1 -- 172.21.2.101:0/3363803762 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v20) v1 ==== 104+0+10 (secure 0 0 0) 0x7f26c4004140 con 0x7f26d40a78f0 2021-02-10T11:14:12.378 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:14:12.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.375 7f26dc33d700 1 -- 172.21.2.101:0/3363803762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26bc018730 msgr2=0x7f26bc01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:12.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.375 7f26dc33d700 1 --2- 172.21.2.101:0/3363803762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26bc018730 0x7f26bc01ac10 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:12.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.375 7f26dc33d700 1 -- 172.21.2.101:0/3363803762 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26d40a78f0 msgr2=0x7f26d40a7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:12.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.375 7f26dc33d700 1 --2- 172.21.2.101:0/3363803762 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26d40a78f0 0x7f26d40a7cf0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f26c4001c60 tx=0x7f26c400a8d0).stop 2021-02-10T11:14:12.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.375 7f26dc33d700 1 -- 172.21.2.101:0/3363803762 shutdown_connections 2021-02-10T11:14:12.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.375 7f26dc33d700 1 --2- 172.21.2.101:0/3363803762 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f26d40a8310 0x7f26d40a9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:12.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.375 7f26dc33d700 1 --2- 172.21.2.101:0/3363803762 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26d40a78f0 0x7f26d40a7cf0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:12.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.375 7f26dc33d700 1 --2- 172.21.2.101:0/3363803762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26bc018730 0x7f26bc01ac10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:12.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.375 7f26dc33d700 1 --2- 172.21.2.101:0/3363803762 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26d4192ad0 0x7f26d4192ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:12.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.375 7f26dc33d700 1 -- 172.21.2.101:0/3363803762 >> 172.21.2.101:0/3363803762 conn(0x7f26d40bf4f0 msgr2=0x7f26d40be250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:12.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.375 7f26dc33d700 1 -- 172.21.2.101:0/3363803762 shutdown_connections 2021-02-10T11:14:12.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.375 7f26dc33d700 1 -- 172.21.2.101:0/3363803762 wait complete. 2021-02-10T11:14:12.387 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:14:12.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.795 7f88ab828700 1 Processor -- start 2021-02-10T11:14:12.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.795 7f88ab828700 1 -- start start 2021-02-10T11:14:12.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.796 7f88ab828700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88a41b0870 0x7f88a40f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:12.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.796 7f88ab828700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f88a40efe00 0x7f88a40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:12.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.796 7f88ab828700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a41b0c70 con 0x7f88a41b0470 2021-02-10T11:14:12.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.796 7f88ab828700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a40f0b50 con 0x7f88a40efe00 2021-02-10T11:14:12.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.796 7f88ab828700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a40f0820 con 0x7f88a41b0870 2021-02-10T11:14:12.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.796 7f88a9e45700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f88a40efe00 0x7f88a40f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:12.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.796 7f88a9e45700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f88a40efe00 0x7f88a40f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44890/0 (socket says 172.21.2.101:44890) 2021-02-10T11:14:12.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.796 7f88a9e45700 1 -- 172.21.2.101:0/1541451208 learned_addr learned my addr 172.21.2.101:0/1541451208 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:12.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.796 7f88a9644700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f88a41b0470 0x7f88a40ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:38790/0 (socket says 172.21.2.101:38790) 2021-02-10T11:14:12.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.796 7f88a8e43700 1 --2- 172.21.2.101:0/1541451208 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88a41b0870 0x7f88a40f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:12.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.796 7f88a9e45700 1 -- 172.21.2.101:0/1541451208 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88a41b0870 msgr2=0x7f88a40f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:12.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.796 7f88a9e45700 1 --2- 172.21.2.101:0/1541451208 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88a41b0870 0x7f88a40f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:12.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.796 7f88a9e45700 1 -- 172.21.2.101:0/1541451208 >> v1:172.21.2.108:6789/0 conn(0x7f88a41b0470 legacy=0x7f88a40ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:12.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.796 7f8897fff700 1 -- 172.21.2.101:0/1541451208 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901663799 0 0) 0x7f88980010c0 con 0x7f88a41b0470 2021-02-10T11:14:12.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.796 7f88a9e45700 1 -- 172.21.2.101:0/1541451208 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a40e52e0 con 0x7f88a40efe00 2021-02-10T11:14:12.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.797 7f88a9e45700 1 --2- 172.21.2.101:0/1541451208 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88a40efe00 0x7f88a40f0200 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f889c006f00 tx=0x7f889c009f10).ready entity=mon.0 client_cookie=4f9074636b51e779 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:12.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.797 7f8897fff700 1 -- 172.21.2.101:0/1541451208 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f889c00b3b0 con 0x7f88a40efe00 2021-02-10T11:14:12.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.797 7f8897fff700 1 -- 172.21.2.101:0/1541451208 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889c00b730 con 0x7f88a40efe00 2021-02-10T11:14:12.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.797 7f8897fff700 1 -- 172.21.2.101:0/1541451208 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f889c00bc50 con 0x7f88a40efe00 2021-02-10T11:14:12.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.797 7f88ab828700 1 -- 172.21.2.101:0/1541451208 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88a40efe00 msgr2=0x7f88a40f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:12.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.797 7f88ab828700 1 --2- 172.21.2.101:0/1541451208 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88a40efe00 0x7f88a40f0200 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f889c006f00 tx=0x7f889c009f10).stop 2021-02-10T11:14:12.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.798 7f88ab828700 1 -- 172.21.2.101:0/1541451208 shutdown_connections 2021-02-10T11:14:12.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.798 7f88ab828700 1 --2- 172.21.2.101:0/1541451208 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88a41b0870 0x7f88a40f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:12.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.798 7f88ab828700 1 --2- 172.21.2.101:0/1541451208 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88a40efe00 0x7f88a40f0200 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:12.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.798 7f88ab828700 1 -- 172.21.2.101:0/1541451208 >> 172.21.2.101:0/1541451208 conn(0x7f88a40ce3e0 msgr2=0x7f88a40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:12.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.798 7f88ab828700 1 -- 172.21.2.101:0/1541451208 shutdown_connections 2021-02-10T11:14:12.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.798 7f88ab828700 1 -- 172.21.2.101:0/1541451208 wait complete. 2021-02-10T11:14:12.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.812 7f88ab828700 1 Processor -- start 2021-02-10T11:14:12.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.812 7f88ab828700 1 -- start start 2021-02-10T11:14:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.813 7f88ab828700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f88a40a0420 0x7f88a40a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.813 7f88ab828700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88a4189d00 0x7f88a418a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.813 7f88ab828700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88a418a720 0x7f88a40b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.813 7f88ab828700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a4189800 con 0x7f88a4189d00 2021-02-10T11:14:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.813 7f88ab828700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a418ab20 con 0x7f88a40a0420 2021-02-10T11:14:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.813 7f88ab828700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a40a1140 con 0x7f88a418a720 2021-02-10T11:14:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.813 7f88a8e43700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88a4189d00 0x7f88a418a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.813 7f88a8e43700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88a4189d00 0x7f88a418a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44892/0 (socket says 172.21.2.101:44892) 2021-02-10T11:14:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.813 7f88a8e43700 1 -- 172.21.2.101:0/2437675795 learned_addr learned my addr 172.21.2.101:0/2437675795 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.813 7f88a9644700 1 --2- 172.21.2.101:0/2437675795 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f88a40a0420 0x7f88a40a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:12.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.813 7f88a9e45700 1 --2- 172.21.2.101:0/2437675795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88a418a720 0x7f88a40b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:12.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.813 7f88a8e43700 1 -- 172.21.2.101:0/2437675795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88a418a720 msgr2=0x7f88a40b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:12.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.813 7f88a8e43700 1 --2- 172.21.2.101:0/2437675795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88a418a720 0x7f88a40b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:12.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.813 7f88a8e43700 1 -- 172.21.2.101:0/2437675795 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f88a40a0420 msgr2=0x7f88a40a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:12.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.813 7f88a8e43700 1 --2- 172.21.2.101:0/2437675795 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f88a40a0420 0x7f88a40a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:12.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.813 7f88a8e43700 1 -- 172.21.2.101:0/2437675795 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a40b8200 con 0x7f88a4189d00 2021-02-10T11:14:12.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.813 7f88a8e43700 1 --2- 172.21.2.101:0/2437675795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88a4189d00 0x7f88a418a100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f88a0006f60 tx=0x7f88a000b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:12.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.814 7f8895ffb700 1 -- 172.21.2.101:0/2437675795 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f88a000bd60 con 0x7f88a4189d00 2021-02-10T11:14:12.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.814 7f8895ffb700 1 -- 172.21.2.101:0/2437675795 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a0000c90 con 0x7f88a4189d00 2021-02-10T11:14:12.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.814 7f88ab828700 1 -- 172.21.2.101:0/2437675795 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a40b8200 con 0x7f88a4189d00 2021-02-10T11:14:12.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.814 7f88ab828700 1 -- 172.21.2.101:0/2437675795 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a40b8200 con 0x7f88a4189d00 2021-02-10T11:14:12.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.814 7f8895ffb700 1 -- 172.21.2.101:0/2437675795 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f88a0002220 con 0x7f88a4189d00 2021-02-10T11:14:12.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.814 7f8895ffb700 1 -- 172.21.2.101:0/2437675795 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f88a00052e0 con 0x7f88a4189d00 2021-02-10T11:14:12.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.814 7f8895ffb700 1 --2- 172.21.2.101:0/2437675795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8884018730 0x7f888401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:12.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.814 7f8895ffb700 1 -- 172.21.2.101:0/2437675795 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f88a00327c0 con 0x7f88a4189d00 2021-02-10T11:14:12.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.814 7f88a9644700 1 --2- 172.21.2.101:0/2437675795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8884018730 0x7f888401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:12.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.815 7f88a9644700 1 --2- 172.21.2.101:0/2437675795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8884018730 0x7f888401abc0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:12.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.846 7f88ab828700 1 -- 172.21.2.101:0/2437675795 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88a418ab20 con 0x7f88a4189d00 2021-02-10T11:14:12.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.848 7f8895ffb700 1 -- 172.21.2.101:0/2437675795 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f88a0000c90 con 0x7f88a4189d00 2021-02-10T11:14:12.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.976 7f88ab828700 1 -- 172.21.2.101:0/2437675795 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f88a418ab20 con 0x7f88a4189d00 2021-02-10T11:14:12.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.977 7f8895ffb700 1 -- 172.21.2.101:0/2437675795 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v20) v1 ==== 100+0+10 (secure 0 0 0) 0x7f88a00052e0 con 0x7f88a4189d00 2021-02-10T11:14:12.982 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:14:12.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.979 7f88ab828700 1 -- 172.21.2.101:0/2437675795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8884018730 msgr2=0x7f888401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:12.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.979 7f88ab828700 1 --2- 172.21.2.101:0/2437675795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8884018730 0x7f888401abc0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:12.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.979 7f88ab828700 1 -- 172.21.2.101:0/2437675795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88a4189d00 msgr2=0x7f88a418a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:12.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.979 7f88ab828700 1 --2- 172.21.2.101:0/2437675795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88a4189d00 0x7f88a418a100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f88a0006f60 tx=0x7f88a000b970).stop 2021-02-10T11:14:12.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.979 7f88ab828700 1 -- 172.21.2.101:0/2437675795 shutdown_connections 2021-02-10T11:14:12.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.979 7f88ab828700 1 --2- 172.21.2.101:0/2437675795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8884018730 0x7f888401abc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:12.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.979 7f88ab828700 1 --2- 172.21.2.101:0/2437675795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88a418a720 0x7f88a40b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:12.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.979 7f88ab828700 1 --2- 172.21.2.101:0/2437675795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88a4189d00 0x7f88a418a100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:12.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.979 7f88ab828700 1 --2- 172.21.2.101:0/2437675795 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f88a40a0420 0x7f88a40a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:12.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.979 7f88ab828700 1 -- 172.21.2.101:0/2437675795 >> 172.21.2.101:0/2437675795 conn(0x7f88a4094e10 msgr2=0x7f88a40a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:12.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.979 7f88ab828700 1 -- 172.21.2.101:0/2437675795 shutdown_connections 2021-02-10T11:14:12.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:12.979 7f88ab828700 1 -- 172.21.2.101:0/2437675795 wait complete. 2021-02-10T11:14:12.992 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:14:13.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.397 7ff68ee31700 1 Processor -- start 2021-02-10T11:14:13.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.397 7ff68ee31700 1 -- start start 2021-02-10T11:14:13.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.397 7ff68ee31700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff6880e5070 0x7ff6881b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:13.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.397 7ff68ee31700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff6881b3e40 0x7ff68807d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:13.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.397 7ff68ee31700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff68807ed80 con 0x7ff6880e5470 2021-02-10T11:14:13.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.397 7ff68ee31700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff688080310 con 0x7ff6880e5070 2021-02-10T11:14:13.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.397 7ff68ee31700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff688080530 con 0x7ff6881b3e40 2021-02-10T11:14:13.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.398 7ff68cc4d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff6880e5070 0x7ff6881b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:13.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.398 7ff68cc4d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff6880e5070 0x7ff6881b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44900/0 (socket says 172.21.2.101:44900) 2021-02-10T11:14:13.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.398 7ff68cc4d700 1 -- 172.21.2.101:0/138594681 learned_addr learned my addr 172.21.2.101:0/138594681 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:13.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.398 7ff68cc4d700 1 -- 172.21.2.101:0/138594681 >> v2:172.21.2.108:3300/0 conn(0x7ff6881b3e40 msgr2=0x7ff68807d690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:13.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.398 7ff68cc4d700 1 --2- 172.21.2.101:0/138594681 >> v2:172.21.2.108:3300/0 conn(0x7ff6881b3e40 0x7ff68807d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:13.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.398 7ff68cc4d700 1 -- 172.21.2.101:0/138594681 >> v1:172.21.2.108:6789/0 conn(0x7ff6880e5470 legacy=0x7ff68807dcb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:13.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.398 7ff68cc4d700 1 -- 172.21.2.101:0/138594681 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff688080750 con 0x7ff6880e5070 2021-02-10T11:14:13.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.398 7ff68cc4d700 1 --2- 172.21.2.101:0/138594681 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6880e5070 0x7ff6881b3820 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7ff678006700 tx=0x7ff678009b00).ready entity=mon.0 client_cookie=b0d38d65de84d40e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:13.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.398 7ff6877fe700 1 -- 172.21.2.101:0/138594681 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff67800c0c0 con 0x7ff6880e5070 2021-02-10T11:14:13.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.398 7ff6877fe700 1 -- 172.21.2.101:0/138594681 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67800c400 con 0x7ff6880e5070 2021-02-10T11:14:13.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.398 7ff6877fe700 1 -- 172.21.2.101:0/138594681 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff67800c920 con 0x7ff6880e5070 2021-02-10T11:14:13.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.398 7ff68ee31700 1 -- 172.21.2.101:0/138594681 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6880e5070 msgr2=0x7ff6881b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:13.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.398 7ff68ee31700 1 --2- 172.21.2.101:0/138594681 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6880e5070 0x7ff6881b3820 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7ff678006700 tx=0x7ff678009b00).stop 2021-02-10T11:14:13.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.399 7ff68ee31700 1 -- 172.21.2.101:0/138594681 shutdown_connections 2021-02-10T11:14:13.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.399 7ff68ee31700 1 --2- 172.21.2.101:0/138594681 >> v2:172.21.2.108:3300/0 conn(0x7ff6881b3e40 0x7ff68807d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:13.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.399 7ff68ee31700 1 --2- 172.21.2.101:0/138594681 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6880e5070 0x7ff6881b3820 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:13.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.399 7ff68ee31700 1 -- 172.21.2.101:0/138594681 >> 172.21.2.101:0/138594681 conn(0x7ff6880ce690 msgr2=0x7ff6880e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:13.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.399 7ff68ee31700 1 -- 172.21.2.101:0/138594681 shutdown_connections 2021-02-10T11:14:13.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.399 7ff68ee31700 1 -- 172.21.2.101:0/138594681 wait complete. 2021-02-10T11:14:13.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.413 7ff68ee31700 1 Processor -- start 2021-02-10T11:14:13.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.414 7ff68ee31700 1 -- start start 2021-02-10T11:14:13.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.414 7ff68ee31700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6880b7a90 0x7ff6880b7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:13.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.414 7ff68ee31700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff688084180 0x7ff688084580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:13.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.414 7ff68ee31700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff688084ba0 0x7ff688084fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:13.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.414 7ff68ee31700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68809d0f0 con 0x7ff688084ba0 2021-02-10T11:14:13.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.414 7ff68ee31700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff688191ed0 con 0x7ff688084180 2021-02-10T11:14:13.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.414 7ff68ee31700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6881a9190 con 0x7ff6880b7a90 2021-02-10T11:14:13.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.414 7ff68d44e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff688084ba0 0x7ff688084fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:13.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.414 7ff68d44e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff688084ba0 0x7ff688084fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44906/0 (socket says 172.21.2.101:44906) 2021-02-10T11:14:13.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.414 7ff68d44e700 1 -- 172.21.2.101:0/3798154826 learned_addr learned my addr 172.21.2.101:0/3798154826 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:13.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.414 7ff68cc4d700 1 --2- 172.21.2.101:0/3798154826 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6880b7a90 0x7ff6880b7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:13.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.414 7ff687fff700 1 --2- 172.21.2.101:0/3798154826 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff688084180 0x7ff688084580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:13.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.415 7ff68d44e700 1 -- 172.21.2.101:0/3798154826 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6880b7a90 msgr2=0x7ff6880b7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:13.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.415 7ff68d44e700 1 --2- 172.21.2.101:0/3798154826 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6880b7a90 0x7ff6880b7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:13.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.415 7ff68d44e700 1 -- 172.21.2.101:0/3798154826 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff688084180 msgr2=0x7ff688084580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:13.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.415 7ff68d44e700 1 --2- 172.21.2.101:0/3798154826 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff688084180 0x7ff688084580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:13.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.415 7ff68d44e700 1 -- 172.21.2.101:0/3798154826 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6881a93b0 con 0x7ff688084ba0 2021-02-10T11:14:13.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.415 7ff68d44e700 1 --2- 172.21.2.101:0/3798154826 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff688084ba0 0x7ff688084fa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7ff680009940 tx=0x7ff68000bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:13.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.415 7ff6857fa700 1 -- 172.21.2.101:0/3798154826 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff680000f90 con 0x7ff688084ba0 2021-02-10T11:14:13.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.415 7ff6857fa700 1 -- 172.21.2.101:0/3798154826 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff680001270 con 0x7ff688084ba0 2021-02-10T11:14:13.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.415 7ff68ee31700 1 -- 172.21.2.101:0/3798154826 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6881a93b0 con 0x7ff688084ba0 2021-02-10T11:14:13.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.415 7ff68ee31700 1 -- 172.21.2.101:0/3798154826 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6881a9940 con 0x7ff688084ba0 2021-02-10T11:14:13.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.415 7ff6857fa700 1 -- 172.21.2.101:0/3798154826 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff680001790 con 0x7ff688084ba0 2021-02-10T11:14:13.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.416 7ff6857fa700 1 -- 172.21.2.101:0/3798154826 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff680001450 con 0x7ff688084ba0 2021-02-10T11:14:13.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.416 7ff6857fa700 1 --2- 172.21.2.101:0/3798154826 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6640186c0 0x7ff66401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:13.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.416 7ff6857fa700 1 -- 172.21.2.101:0/3798154826 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff6800329e0 con 0x7ff688084ba0 2021-02-10T11:14:13.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.416 7ff68cc4d700 1 --2- 172.21.2.101:0/3798154826 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6640186c0 0x7ff66401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:13.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.416 7ff68cc4d700 1 --2- 172.21.2.101:0/3798154826 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6640186c0 0x7ff66401abc0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:13.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.447 7ff68ee31700 1 -- 172.21.2.101:0/3798154826 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6881aa850 con 0x7ff688084ba0 2021-02-10T11:14:13.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.450 7ff6857fa700 1 -- 172.21.2.101:0/3798154826 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff680000d30 con 0x7ff688084ba0 2021-02-10T11:14:13.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.575 7ff68ee31700 1 -- 172.21.2.101:0/3798154826 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6881aa850 con 0x7ff688084ba0 2021-02-10T11:14:13.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.575 7ff6857fa700 1 -- 172.21.2.101:0/3798154826 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+13993 (secure 0 0 0) 0x7ff680001270 con 0x7ff688084ba0 2021-02-10T11:14:13.580 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:14:13.580 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":20,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:13:15.149234","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:13:11.922272","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2021-02-10 11:13:13.141733","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:14:13.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.577 7ff68ee31700 1 -- 172.21.2.101:0/3798154826 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6640186c0 msgr2=0x7ff66401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:13.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.577 7ff68ee31700 1 --2- 172.21.2.101:0/3798154826 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6640186c0 0x7ff66401abc0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:13.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.577 7ff68ee31700 1 -- 172.21.2.101:0/3798154826 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff688084ba0 msgr2=0x7ff688084fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:13.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.577 7ff68ee31700 1 --2- 172.21.2.101:0/3798154826 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff688084ba0 0x7ff688084fa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7ff680009940 tx=0x7ff68000bcf0).stop 2021-02-10T11:14:13.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.577 7ff68ee31700 1 -- 172.21.2.101:0/3798154826 shutdown_connections 2021-02-10T11:14:13.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.577 7ff68ee31700 1 --2- 172.21.2.101:0/3798154826 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff688084ba0 0x7ff688084fa0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:13.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.577 7ff68ee31700 1 --2- 172.21.2.101:0/3798154826 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff688084180 0x7ff688084580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:13.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.577 7ff68ee31700 1 --2- 172.21.2.101:0/3798154826 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6640186c0 0x7ff66401abc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:13.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.577 7ff68ee31700 1 --2- 172.21.2.101:0/3798154826 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6880b7a90 0x7ff6880b7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:13.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.577 7ff68ee31700 1 -- 172.21.2.101:0/3798154826 >> 172.21.2.101:0/3798154826 conn(0x7ff6881b2dc0 msgr2=0x7ff688092c70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:13.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.577 7ff68ee31700 1 -- 172.21.2.101:0/3798154826 shutdown_connections 2021-02-10T11:14:13.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.577 7ff68ee31700 1 -- 172.21.2.101:0/3798154826 wait complete. 2021-02-10T11:14:13.590 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:13:11.922272', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:13:13.141733', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:14:13.590 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:14:14.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.998 7fd0eb51f700 1 Processor -- start 2021-02-10T11:14:14.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.998 7fd0eb51f700 1 -- start start 2021-02-10T11:14:14.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.998 7fd0eb51f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd0e41b0870 0x7fd0e40f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:14.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.998 7fd0eb51f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0e40efe00 0x7fd0e40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:14.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.998 7fd0eb51f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e41b0c70 con 0x7fd0e41b0470 2021-02-10T11:14:14.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.998 7fd0eb51f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e40f0b50 con 0x7fd0e41b0870 2021-02-10T11:14:14.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.998 7fd0eb51f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e40f0820 con 0x7fd0e40efe00 2021-02-10T11:14:14.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.998 7fd0e933b700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd0e41b0470 0x7fd0e40ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45406/0 (socket says 172.21.2.101:45406) 2021-02-10T11:14:14.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.998 7fd0e933b700 1 -- 172.21.2.101:0/3752252562 learned_addr learned my addr 172.21.2.101:0/3752252562 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:14:14.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.999 7fd0e9b3c700 1 --2- 172.21.2.101:0/3752252562 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0e40efe00 0x7fd0e40f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.999 7fd0e8b3a700 1 --2- 172.21.2.101:0/3752252562 >> v2:172.21.2.108:3300/0 conn(0x7fd0e41b0870 0x7fd0e40f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.999 7fd0d7fff700 1 -- 172.21.2.101:0/3752252562 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2979423332 0 0) 0x7fd0e00010b0 con 0x7fd0e41b0470 2021-02-10T11:14:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.999 7fd0d7fff700 1 -- 172.21.2.101:0/3752252562 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0d0002fe0 con 0x7fd0e41b0470 2021-02-10T11:14:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.999 7fd0e9b3c700 1 -- 172.21.2.101:0/3752252562 >> v2:172.21.2.108:3300/0 conn(0x7fd0e41b0870 msgr2=0x7fd0e40f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.999 7fd0e9b3c700 1 --2- 172.21.2.101:0/3752252562 >> v2:172.21.2.108:3300/0 conn(0x7fd0e41b0870 0x7fd0e40f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.999 7fd0e9b3c700 1 -- 172.21.2.101:0/3752252562 >> v1:172.21.2.101:6789/0 conn(0x7fd0e41b0470 legacy=0x7fd0e40ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.999 7fd0e9b3c700 1 -- 172.21.2.101:0/3752252562 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e40e52e0 con 0x7fd0e40efe00 2021-02-10T11:14:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.999 7fd0e8b3a700 1 --2- 172.21.2.101:0/3752252562 >> v2:172.21.2.108:3300/0 conn(0x7fd0e41b0870 0x7fd0e40f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:14:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.999 7fd0e9b3c700 1 --2- 172.21.2.101:0/3752252562 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0e40efe00 0x7fd0e40f0200 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fd0dc006e50 tx=0x7fd0dc009e70).ready entity=mon.2 client_cookie=314156e9ac5d51d1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:13.999 7fd0d7fff700 1 -- 172.21.2.101:0/3752252562 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd0dc00b510 con 0x7fd0e40efe00 2021-02-10T11:14:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.000 7fd0d7fff700 1 -- 172.21.2.101:0/3752252562 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0dc006790 con 0x7fd0e40efe00 2021-02-10T11:14:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.000 7fd0d7fff700 1 -- 172.21.2.101:0/3752252562 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd0dc00b940 con 0x7fd0e40efe00 2021-02-10T11:14:14.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.000 7fd0eb51f700 1 -- 172.21.2.101:0/3752252562 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0e40efe00 msgr2=0x7fd0e40f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:14.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.000 7fd0eb51f700 1 --2- 172.21.2.101:0/3752252562 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0e40efe00 0x7fd0e40f0200 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fd0dc006e50 tx=0x7fd0dc009e70).stop 2021-02-10T11:14:14.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.000 7fd0eb51f700 1 -- 172.21.2.101:0/3752252562 shutdown_connections 2021-02-10T11:14:14.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.000 7fd0eb51f700 1 --2- 172.21.2.101:0/3752252562 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0e40efe00 0x7fd0e40f0200 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:14.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.000 7fd0eb51f700 1 --2- 172.21.2.101:0/3752252562 >> v2:172.21.2.108:3300/0 conn(0x7fd0e41b0870 0x7fd0e40f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:14.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.000 7fd0eb51f700 1 -- 172.21.2.101:0/3752252562 >> 172.21.2.101:0/3752252562 conn(0x7fd0e40ce3e0 msgr2=0x7fd0e40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:14.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.000 7fd0eb51f700 1 -- 172.21.2.101:0/3752252562 shutdown_connections 2021-02-10T11:14:14.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.000 7fd0eb51f700 1 -- 172.21.2.101:0/3752252562 wait complete. 2021-02-10T11:14:14.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.015 7fd0eb51f700 1 Processor -- start 2021-02-10T11:14:14.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.015 7fd0eb51f700 1 -- start start 2021-02-10T11:14:14.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.015 7fd0eb51f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0e40a0420 0x7fd0e40a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:14.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.015 7fd0eb51f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0e4189d00 0x7fd0e418a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:14.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.015 7fd0eb51f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0e418a720 0x7fd0e40b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:14.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.015 7fd0eb51f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e4189800 con 0x7fd0e418a720 2021-02-10T11:14:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.015 7fd0eb51f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e418ab20 con 0x7fd0e4189d00 2021-02-10T11:14:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.015 7fd0eb51f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e40a1140 con 0x7fd0e40a0420 2021-02-10T11:14:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.015 7fd0e9b3c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0e418a720 0x7fd0e40b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.016 7fd0e9b3c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0e418a720 0x7fd0e40b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44922/0 (socket says 172.21.2.101:44922) 2021-02-10T11:14:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.016 7fd0e9b3c700 1 -- 172.21.2.101:0/2099087389 learned_addr learned my addr 172.21.2.101:0/2099087389 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.016 7fd0e9b3c700 1 -- 172.21.2.101:0/2099087389 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0e40a0420 msgr2=0x7fd0e40a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.016 7fd0e9b3c700 1 --2- 172.21.2.101:0/2099087389 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0e40a0420 0x7fd0e40a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.016 7fd0e9b3c700 1 -- 172.21.2.101:0/2099087389 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0e4189d00 msgr2=0x7fd0e418a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.016 7fd0e9b3c700 1 --2- 172.21.2.101:0/2099087389 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0e4189d00 0x7fd0e418a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.016 7fd0e9b3c700 1 -- 172.21.2.101:0/2099087389 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e40b8200 con 0x7fd0e418a720 2021-02-10T11:14:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.016 7fd0e9b3c700 1 --2- 172.21.2.101:0/2099087389 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0e418a720 0x7fd0e40b7be0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fd0dc00b970 tx=0x7fd0dc00bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.016 7fd0d5ffb700 1 -- 172.21.2.101:0/2099087389 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd0dc0093d0 con 0x7fd0e418a720 2021-02-10T11:14:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.016 7fd0d5ffb700 1 -- 172.21.2.101:0/2099087389 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0dc007240 con 0x7fd0e418a720 2021-02-10T11:14:14.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.016 7fd0d5ffb700 1 -- 172.21.2.101:0/2099087389 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd0dc007760 con 0x7fd0e418a720 2021-02-10T11:14:14.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.016 7fd0eb51f700 1 -- 172.21.2.101:0/2099087389 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e40b8200 con 0x7fd0e418a720 2021-02-10T11:14:14.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.016 7fd0eb51f700 1 -- 172.21.2.101:0/2099087389 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e40b8200 con 0x7fd0e418a720 2021-02-10T11:14:14.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.017 7fd0d5ffb700 1 -- 172.21.2.101:0/2099087389 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd0dc007420 con 0x7fd0e418a720 2021-02-10T11:14:14.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.017 7fd0d5ffb700 1 --2- 172.21.2.101:0/2099087389 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd0c4018730 0x7fd0c401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:14.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.017 7fd0d5ffb700 1 -- 172.21.2.101:0/2099087389 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd0dc0329f0 con 0x7fd0e418a720 2021-02-10T11:14:14.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.017 7fd0e933b700 1 --2- 172.21.2.101:0/2099087389 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd0c4018730 0x7fd0c401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:14.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.017 7fd0e933b700 1 --2- 172.21.2.101:0/2099087389 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd0c4018730 0x7fd0c401ac10 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:14.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.048 7fd0eb51f700 1 -- 172.21.2.101:0/2099087389 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e4105ae0 con 0x7fd0e418a720 2021-02-10T11:14:14.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.051 7fd0d5ffb700 1 -- 172.21.2.101:0/2099087389 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd0dc007030 con 0x7fd0e418a720 2021-02-10T11:14:14.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.180 7fd0eb51f700 1 -- 172.21.2.101:0/2099087389 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fd0e40f0b50 con 0x7fd0e418a720 2021-02-10T11:14:14.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.180 7fd0d5ffb700 1 -- 172.21.2.101:0/2099087389 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd0dc007240 con 0x7fd0e418a720 2021-02-10T11:14:14.185 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:14:14.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.182 7fd0eb51f700 1 -- 172.21.2.101:0/2099087389 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd0c4018730 msgr2=0x7fd0c401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:14.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.182 7fd0eb51f700 1 --2- 172.21.2.101:0/2099087389 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd0c4018730 0x7fd0c401ac10 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:14.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.182 7fd0eb51f700 1 -- 172.21.2.101:0/2099087389 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0e418a720 msgr2=0x7fd0e40b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:14.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.182 7fd0eb51f700 1 --2- 172.21.2.101:0/2099087389 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0e418a720 0x7fd0e40b7be0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fd0dc00b970 tx=0x7fd0dc00bd10).stop 2021-02-10T11:14:14.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.182 7fd0eb51f700 1 -- 172.21.2.101:0/2099087389 shutdown_connections 2021-02-10T11:14:14.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.182 7fd0eb51f700 1 --2- 172.21.2.101:0/2099087389 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0e418a720 0x7fd0e40b7be0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:14.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.182 7fd0eb51f700 1 --2- 172.21.2.101:0/2099087389 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0e4189d00 0x7fd0e418a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:14.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.182 7fd0eb51f700 1 --2- 172.21.2.101:0/2099087389 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd0c4018730 0x7fd0c401ac10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:14.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.182 7fd0eb51f700 1 --2- 172.21.2.101:0/2099087389 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0e40a0420 0x7fd0e40a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:14.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.182 7fd0eb51f700 1 -- 172.21.2.101:0/2099087389 >> 172.21.2.101:0/2099087389 conn(0x7fd0e4094e10 msgr2=0x7fd0e40a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:14.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.182 7fd0eb51f700 1 -- 172.21.2.101:0/2099087389 shutdown_connections 2021-02-10T11:14:14.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.182 7fd0eb51f700 1 -- 172.21.2.101:0/2099087389 wait complete. 2021-02-10T11:14:14.194 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:14:14.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.602 7f5f6cb0d700 1 Processor -- start 2021-02-10T11:14:14.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.602 7f5f6cb0d700 1 -- start start 2021-02-10T11:14:14.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.603 7f5f6cb0d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5f680f05f0 0x7f5f680f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:14.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.603 7f5f6cb0d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f681b0c70 con 0x7f5f681b0870 2021-02-10T11:14:14.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.603 7f5f6cb0d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f680f0b50 con 0x7f5f681b0470 2021-02-10T11:14:14.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.603 7f5f6cb0d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f680e5ea0 con 0x7f5f680f05f0 2021-02-10T11:14:14.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.603 7f5f6705f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5f680f05f0 0x7f5f680f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:14.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.603 7f5f6605d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f5f681b0870 0x7f5f680efe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45422/0 (socket says 172.21.2.101:45422) 2021-02-10T11:14:14.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.603 7f5f6605d700 1 -- 172.21.2.101:0/2438475676 learned_addr learned my addr 172.21.2.101:0/2438475676 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:14:14.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.603 7f5f6705f700 1 -- 172.21.2.101:0/2438475676 >> v1:172.21.2.108:6789/0 conn(0x7f5f681b0470 legacy=0x7f5f680ef610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:14.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.603 7f5f6705f700 1 -- 172.21.2.101:0/2438475676 >> v1:172.21.2.101:6789/0 conn(0x7f5f681b0870 legacy=0x7f5f680efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:14.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.603 7f5f6705f700 1 -- 172.21.2.101:0/2438475676 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f680e52e0 con 0x7f5f680f05f0 2021-02-10T11:14:14.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.603 7f5f6705f700 1 --2- 172.21.2.101:0/2438475676 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f680f05f0 0x7f5f680f20e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f5f60006e30 tx=0x7f5f60009e60).ready entity=mon.0 client_cookie=9afe67e97cb8f086 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:14.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.603 7f5f6585c700 1 -- 172.21.2.101:0/2438475676 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5f60006a50 con 0x7f5f680f05f0 2021-02-10T11:14:14.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.603 7f5f6585c700 1 -- 172.21.2.101:0/2438475676 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6000b300 con 0x7f5f680f05f0 2021-02-10T11:14:14.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.603 7f5f6585c700 1 -- 172.21.2.101:0/2438475676 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5f6000b820 con 0x7f5f680f05f0 2021-02-10T11:14:14.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.603 7f5f6cb0d700 1 -- 172.21.2.101:0/2438475676 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f680f05f0 msgr2=0x7f5f680f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:14.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.604 7f5f6cb0d700 1 --2- 172.21.2.101:0/2438475676 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f680f05f0 0x7f5f680f20e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f5f60006e30 tx=0x7f5f60009e60).stop 2021-02-10T11:14:14.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.604 7f5f6cb0d700 1 -- 172.21.2.101:0/2438475676 shutdown_connections 2021-02-10T11:14:14.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.604 7f5f6cb0d700 1 --2- 172.21.2.101:0/2438475676 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f680f05f0 0x7f5f680f20e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:14.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.604 7f5f6cb0d700 1 -- 172.21.2.101:0/2438475676 >> 172.21.2.101:0/2438475676 conn(0x7f5f680ce3e0 msgr2=0x7f5f680e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:14.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.604 7f5f6cb0d700 1 -- 172.21.2.101:0/2438475676 shutdown_connections 2021-02-10T11:14:14.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.604 7f5f6cb0d700 1 -- 172.21.2.101:0/2438475676 wait complete. 2021-02-10T11:14:14.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.618 7f5f6cb0d700 1 Processor -- start 2021-02-10T11:14:14.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.618 7f5f6cb0d700 1 -- start start 2021-02-10T11:14:14.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.619 7f5f6cb0d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5f68082170 0x7f5f680894a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:14.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.619 7f5f6cb0d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5f681b01a0 0x7f5f681b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:14.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.619 7f5f6cb0d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f681b1460 0x7f5f681b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:14.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.619 7f5f6cb0d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f681bf2a0 con 0x7f5f681b1460 2021-02-10T11:14:14.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.619 7f5f6cb0d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f681b0ee0 con 0x7f5f68082170 2021-02-10T11:14:14.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.619 7f5f6cb0d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f681b60c0 con 0x7f5f681b01a0 2021-02-10T11:14:14.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.619 7f5f6705f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f681b1460 0x7f5f681b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:14.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.619 7f5f6705f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f681b1460 0x7f5f681b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44936/0 (socket says 172.21.2.101:44936) 2021-02-10T11:14:14.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.619 7f5f6705f700 1 -- 172.21.2.101:0/537422691 learned_addr learned my addr 172.21.2.101:0/537422691 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:14.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.619 7f5f6705f700 1 -- 172.21.2.101:0/537422691 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5f681b01a0 msgr2=0x7f5f681b05a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:14.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.619 7f5f6705f700 1 --2- 172.21.2.101:0/537422691 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5f681b01a0 0x7f5f681b05a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:14.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.619 7f5f6705f700 1 -- 172.21.2.101:0/537422691 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5f68082170 msgr2=0x7f5f680894a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:14.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.619 7f5f6705f700 1 --2- 172.21.2.101:0/537422691 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5f68082170 0x7f5f680894a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:14.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.619 7f5f6705f700 1 -- 172.21.2.101:0/537422691 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f681b0bc0 con 0x7f5f681b1460 2021-02-10T11:14:14.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.619 7f5f6705f700 1 --2- 172.21.2.101:0/537422691 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f681b1460 0x7f5f681b1860 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f5f6000b970 tx=0x7f5f6000bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:14.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.620 7f5f5f7fe700 1 -- 172.21.2.101:0/537422691 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5f60009380 con 0x7f5f681b1460 2021-02-10T11:14:14.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.620 7f5f5f7fe700 1 -- 172.21.2.101:0/537422691 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f60000ff0 con 0x7f5f681b1460 2021-02-10T11:14:14.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.620 7f5f5f7fe700 1 -- 172.21.2.101:0/537422691 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5f60005a10 con 0x7f5f681b1460 2021-02-10T11:14:14.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.620 7f5f6cb0d700 1 -- 172.21.2.101:0/537422691 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f681b0bc0 con 0x7f5f681b1460 2021-02-10T11:14:14.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.620 7f5f6cb0d700 1 -- 172.21.2.101:0/537422691 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f681b0bc0 con 0x7f5f681b1460 2021-02-10T11:14:14.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.620 7f5f5f7fe700 1 -- 172.21.2.101:0/537422691 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5f60005820 con 0x7f5f681b1460 2021-02-10T11:14:14.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.620 7f5f5f7fe700 1 --2- 172.21.2.101:0/537422691 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5f4c0186c0 0x7f5f4c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:14.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.620 7f5f5f7fe700 1 -- 172.21.2.101:0/537422691 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5f600327c0 con 0x7f5f681b1460 2021-02-10T11:14:14.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.621 7f5f6685e700 1 --2- 172.21.2.101:0/537422691 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5f4c0186c0 0x7f5f4c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:14.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.621 7f5f6685e700 1 --2- 172.21.2.101:0/537422691 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5f4c0186c0 0x7f5f4c01abc0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:14.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.653 7f5f6cb0d700 1 -- 172.21.2.101:0/537422691 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f681bf2a0 con 0x7f5f681b1460 2021-02-10T11:14:14.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.656 7f5f5f7fe700 1 -- 172.21.2.101:0/537422691 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5f60009380 con 0x7f5f681b1460 2021-02-10T11:14:14.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.782 7f5f6cb0d700 1 -- 172.21.2.101:0/537422691 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f5f681bf2a0 con 0x7f5f681b1460 2021-02-10T11:14:14.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.782 7f5f5f7fe700 1 -- 172.21.2.101:0/537422691 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v20) v1 ==== 104+0+10 (secure 0 0 0) 0x7f5f60005820 con 0x7f5f681b1460 2021-02-10T11:14:14.787 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:14:14.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.784 7f5f6cb0d700 1 -- 172.21.2.101:0/537422691 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5f4c0186c0 msgr2=0x7f5f4c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:14.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.784 7f5f6cb0d700 1 --2- 172.21.2.101:0/537422691 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5f4c0186c0 0x7f5f4c01abc0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:14.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.784 7f5f6cb0d700 1 -- 172.21.2.101:0/537422691 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f681b1460 msgr2=0x7f5f681b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:14.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.784 7f5f6cb0d700 1 --2- 172.21.2.101:0/537422691 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f681b1460 0x7f5f681b1860 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f5f6000b970 tx=0x7f5f6000bd10).stop 2021-02-10T11:14:14.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.785 7f5f6cb0d700 1 -- 172.21.2.101:0/537422691 shutdown_connections 2021-02-10T11:14:14.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.785 7f5f6cb0d700 1 --2- 172.21.2.101:0/537422691 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5f4c0186c0 0x7f5f4c01abc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:14.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.785 7f5f6cb0d700 1 --2- 172.21.2.101:0/537422691 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5f681b01a0 0x7f5f681b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:14.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.785 7f5f6cb0d700 1 --2- 172.21.2.101:0/537422691 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f681b1460 0x7f5f681b1860 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:14.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.785 7f5f6cb0d700 1 --2- 172.21.2.101:0/537422691 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5f68082170 0x7f5f680894a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:14.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.785 7f5f6cb0d700 1 -- 172.21.2.101:0/537422691 >> 172.21.2.101:0/537422691 conn(0x7f5f68088d10 msgr2=0x7f5f680afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:14.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.785 7f5f6cb0d700 1 -- 172.21.2.101:0/537422691 shutdown_connections 2021-02-10T11:14:14.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:14.785 7f5f6cb0d700 1 -- 172.21.2.101:0/537422691 wait complete. 2021-02-10T11:14:14.797 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:14:15.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.205 7f25a046d700 1 Processor -- start 2021-02-10T11:14:15.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.205 7f25a046d700 1 -- start start 2021-02-10T11:14:15.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.206 7f25a046d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25981b0870 0x7f25980f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:15.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.206 7f25a046d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25981b0c70 con 0x7f25981b0470 2021-02-10T11:14:15.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.206 7f25a046d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25980f0b50 con 0x7f25980efe00 2021-02-10T11:14:15.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.206 7f25a046d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25980e5ea0 con 0x7f25981b0870 2021-02-10T11:14:15.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.206 7f259e289700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f25981b0470 0x7f25980ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45434/0 (socket says 172.21.2.101:45434) 2021-02-10T11:14:15.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.206 7f259e289700 1 -- 172.21.2.101:0/3414605796 learned_addr learned my addr 172.21.2.101:0/3414605796 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:14:15.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.206 7f259d287700 1 -- 172.21.2.101:0/3414605796 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 915869637 0 0) 0x7f25940010b0 con 0x7f25981b0470 2021-02-10T11:14:15.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.206 7f259d287700 1 -- 172.21.2.101:0/3414605796 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2584002fe0 con 0x7f25981b0470 2021-02-10T11:14:15.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259da88700 1 --2- 172.21.2.101:0/3414605796 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25981b0870 0x7f25980f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:15.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259d287700 1 -- 172.21.2.101:0/3414605796 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3230421262 0 0) 0x7f25940010b0 con 0x7f25981b0470 2021-02-10T11:14:15.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259d287700 1 -- 172.21.2.101:0/3414605796 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25981b0870 msgr2=0x7f25980f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:15.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259d287700 1 --2- 172.21.2.101:0/3414605796 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25981b0870 0x7f25980f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:15.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259d287700 1 -- 172.21.2.101:0/3414605796 >> v1:172.21.2.108:6789/0 conn(0x7f25980efe00 legacy=0x7f25980f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:15.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259d287700 1 -- 172.21.2.101:0/3414605796 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25980e52e0 con 0x7f25981b0470 2021-02-10T11:14:15.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259d287700 1 -- 172.21.2.101:0/3414605796 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f25940015b0 con 0x7f25981b0470 2021-02-10T11:14:15.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259d287700 1 -- 172.21.2.101:0/3414605796 >> v1:172.21.2.101:6789/0 conn(0x7f25981b0470 legacy=0x7f25980ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:15.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259d287700 1 --2- 172.21.2.101:0/3414605796 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2584003b90 0x7f2584005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:15.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259d287700 1 --2- 172.21.2.101:0/3414605796 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2584006840 0x7f2584008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:15.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259d287700 1 --2- 172.21.2.101:0/3414605796 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2584009640 0x7f258400bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:15.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259d287700 1 -- 172.21.2.101:0/3414605796 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258400c380 con 0x7f2584006840 2021-02-10T11:14:15.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259d287700 1 -- 172.21.2.101:0/3414605796 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258400c5a0 con 0x7f2584003b90 2021-02-10T11:14:15.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259d287700 1 -- 172.21.2.101:0/3414605796 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258400c7c0 con 0x7f2584009640 2021-02-10T11:14:15.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259da88700 1 --2- 172.21.2.101:0/3414605796 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25981b0870 0x7f25980f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:14:15.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259da88700 1 --2- 172.21.2.101:0/3414605796 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2584006840 0x7f2584008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:15.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259e289700 1 --2- 172.21.2.101:0/3414605796 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2584003b90 0x7f2584005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:15.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259ea8a700 1 --2- 172.21.2.101:0/3414605796 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2584009640 0x7f258400bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:15.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259da88700 1 -- 172.21.2.101:0/3414605796 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2584009640 msgr2=0x7f258400bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:15.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259da88700 1 --2- 172.21.2.101:0/3414605796 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2584009640 0x7f258400bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:15.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259da88700 1 -- 172.21.2.101:0/3414605796 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2584003b90 msgr2=0x7f2584005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:15.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259da88700 1 --2- 172.21.2.101:0/3414605796 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2584003b90 0x7f2584005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:15.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259da88700 1 -- 172.21.2.101:0/3414605796 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258400ca60 con 0x7f2584006840 2021-02-10T11:14:15.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259da88700 1 --2- 172.21.2.101:0/3414605796 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2584006840 0x7f2584008d40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f258c002e00 tx=0x7f258c04dac0).ready entity=mon.0 client_cookie=be520f63019aec07 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:15.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259d287700 1 -- 172.21.2.101:0/3414605796 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f258c000c90 con 0x7f2584006840 2021-02-10T11:14:15.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259d287700 1 -- 172.21.2.101:0/3414605796 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c002030 con 0x7f2584006840 2021-02-10T11:14:15.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259d287700 1 -- 172.21.2.101:0/3414605796 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f258c002550 con 0x7f2584006840 2021-02-10T11:14:15.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f25a046d700 1 -- 172.21.2.101:0/3414605796 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2584006840 msgr2=0x7f2584008d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:15.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f25a046d700 1 --2- 172.21.2.101:0/3414605796 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2584006840 0x7f2584008d40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f258c002e00 tx=0x7f258c04dac0).stop 2021-02-10T11:14:15.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.207 7f259ea8a700 1 -- 172.21.2.101:0/3414605796 reap_dead start 2021-02-10T11:14:15.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.208 7f25a046d700 1 -- 172.21.2.101:0/3414605796 shutdown_connections 2021-02-10T11:14:15.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.208 7f25a046d700 1 -- 172.21.2.101:0/3414605796 >> 172.21.2.101:0/3414605796 conn(0x7f25980ce3e0 msgr2=0x7f25980e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:15.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.208 7f25a046d700 1 -- 172.21.2.101:0/3414605796 shutdown_connections 2021-02-10T11:14:15.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.208 7f25a046d700 1 -- 172.21.2.101:0/3414605796 wait complete. 2021-02-10T11:14:15.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.223 7f25a046d700 1 Processor -- start 2021-02-10T11:14:15.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.223 7f25a046d700 1 -- start start 2021-02-10T11:14:15.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.223 7f25a046d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f259809b930 0x7f25980b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:15.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.224 7f25a046d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25980b8410 0x7f25980b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:15.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.224 7f25a046d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25980c1f80 0x7f25980c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:15.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.224 7f25a046d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25980c1a00 con 0x7f259809b930 2021-02-10T11:14:15.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.224 7f25a046d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25980b8e30 con 0x7f25980c1f80 2021-02-10T11:14:15.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.224 7f25a046d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25980b9050 con 0x7f25980b8410 2021-02-10T11:14:15.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.224 7f259e289700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f259809b930 0x7f25980b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:15.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.224 7f259e289700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f259809b930 0x7f25980b7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44954/0 (socket says 172.21.2.101:44954) 2021-02-10T11:14:15.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.224 7f259e289700 1 -- 172.21.2.101:0/1357305256 learned_addr learned my addr 172.21.2.101:0/1357305256 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:15.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.224 7f259da88700 1 --2- 172.21.2.101:0/1357305256 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25980b8410 0x7f25980b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:15.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.224 7f259ea8a700 1 --2- 172.21.2.101:0/1357305256 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25980c1f80 0x7f25980c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:15.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.224 7f259e289700 1 -- 172.21.2.101:0/1357305256 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25980b8410 msgr2=0x7f25980b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:15.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.224 7f259e289700 1 --2- 172.21.2.101:0/1357305256 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25980b8410 0x7f25980b8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:15.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.224 7f259e289700 1 -- 172.21.2.101:0/1357305256 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25980c1f80 msgr2=0x7f25980c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:15.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.224 7f259e289700 1 --2- 172.21.2.101:0/1357305256 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25980c1f80 0x7f25980c2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:15.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.224 7f259e289700 1 -- 172.21.2.101:0/1357305256 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25980c29a0 con 0x7f259809b930 2021-02-10T11:14:15.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.224 7f259e289700 1 --2- 172.21.2.101:0/1357305256 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f259809b930 0x7f25980b7c20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f2594001660 tx=0x7f259400a930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:15.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.224 7f258affd700 1 -- 172.21.2.101:0/1357305256 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f259400ad20 con 0x7f259809b930 2021-02-10T11:14:15.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.224 7f258affd700 1 -- 172.21.2.101:0/1357305256 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25940016a0 con 0x7f259809b930 2021-02-10T11:14:15.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.224 7f25a046d700 1 -- 172.21.2.101:0/1357305256 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25980c29a0 con 0x7f259809b930 2021-02-10T11:14:15.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.224 7f258affd700 1 -- 172.21.2.101:0/1357305256 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2594004330 con 0x7f259809b930 2021-02-10T11:14:15.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.225 7f25a046d700 1 -- 172.21.2.101:0/1357305256 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25980c29a0 con 0x7f259809b930 2021-02-10T11:14:15.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.225 7f258affd700 1 -- 172.21.2.101:0/1357305256 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2594004140 con 0x7f259809b930 2021-02-10T11:14:15.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.225 7f258affd700 1 --2- 172.21.2.101:0/1357305256 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2580018730 0x7f258001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:15.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.225 7f258affd700 1 -- 172.21.2.101:0/1357305256 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2594032660 con 0x7f259809b930 2021-02-10T11:14:15.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.225 7f259da88700 1 --2- 172.21.2.101:0/1357305256 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2580018730 0x7f258001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:15.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.225 7f259da88700 1 --2- 172.21.2.101:0/1357305256 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2580018730 0x7f258001ac10 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:15.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.256 7f25a046d700 1 -- 172.21.2.101:0/1357305256 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2598105b30 con 0x7f259809b930 2021-02-10T11:14:15.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.258 7f258affd700 1 -- 172.21.2.101:0/1357305256 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2594001180 con 0x7f259809b930 2021-02-10T11:14:15.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.388 7f25a046d700 1 -- 172.21.2.101:0/1357305256 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f25980f0b50 con 0x7f259809b930 2021-02-10T11:14:15.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.388 7f258affd700 1 -- 172.21.2.101:0/1357305256 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v20) v1 ==== 100+0+10 (secure 0 0 0) 0x7f2594004140 con 0x7f259809b930 2021-02-10T11:14:15.393 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:14:15.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.390 7f25a046d700 1 -- 172.21.2.101:0/1357305256 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2580018730 msgr2=0x7f258001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:15.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.390 7f25a046d700 1 --2- 172.21.2.101:0/1357305256 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2580018730 0x7f258001ac10 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:15.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.390 7f25a046d700 1 -- 172.21.2.101:0/1357305256 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f259809b930 msgr2=0x7f25980b7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:15.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.390 7f25a046d700 1 --2- 172.21.2.101:0/1357305256 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f259809b930 0x7f25980b7c20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f2594001660 tx=0x7f259400a930).stop 2021-02-10T11:14:15.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.390 7f25a046d700 1 -- 172.21.2.101:0/1357305256 shutdown_connections 2021-02-10T11:14:15.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.390 7f25a046d700 1 --2- 172.21.2.101:0/1357305256 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2580018730 0x7f258001ac10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:15.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.390 7f25a046d700 1 --2- 172.21.2.101:0/1357305256 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25980b8410 0x7f25980b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:15.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.390 7f25a046d700 1 --2- 172.21.2.101:0/1357305256 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25980c1f80 0x7f25980c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:15.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.390 7f25a046d700 1 --2- 172.21.2.101:0/1357305256 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f259809b930 0x7f25980b7c20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:15.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.390 7f25a046d700 1 -- 172.21.2.101:0/1357305256 >> 172.21.2.101:0/1357305256 conn(0x7f2598094e10 msgr2=0x7f25980a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:15.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.390 7f25a046d700 1 -- 172.21.2.101:0/1357305256 shutdown_connections 2021-02-10T11:14:15.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.390 7f25a046d700 1 -- 172.21.2.101:0/1357305256 wait complete. 2021-02-10T11:14:15.403 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:14:15.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.807 7f075466d700 1 Processor -- start 2021-02-10T11:14:15.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.807 7f075466d700 1 -- start start 2021-02-10T11:14:15.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.808 7f075466d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f074c0e5470 0x7f074c07e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:15.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.808 7f075466d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f074c07ed80 con 0x7f074c0e5070 2021-02-10T11:14:15.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.808 7f075466d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f074c1b4010 con 0x7f074c07f300 2021-02-10T11:14:15.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.808 7f075466d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f074c0d1790 con 0x7f074c0e5470 2021-02-10T11:14:15.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.808 7f0751c88700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f074c0e5470 0x7f074c07e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:15.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.808 7f0752489700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f074c0e5070 0x7f074c1b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45454/0 (socket says 172.21.2.101:45454) 2021-02-10T11:14:15.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.808 7f0752489700 1 -- 172.21.2.101:0/1564496231 learned_addr learned my addr 172.21.2.101:0/1564496231 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:14:15.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.808 7f0751c88700 1 -- 172.21.2.101:0/1564496231 >> v1:172.21.2.108:6789/0 conn(0x7f074c07f300 legacy=0x7f074c07f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:15.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.808 7f0751c88700 1 -- 172.21.2.101:0/1564496231 >> v1:172.21.2.101:6789/0 conn(0x7f074c0e5070 legacy=0x7f074c1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:15.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.808 7f0751c88700 1 -- 172.21.2.101:0/1564496231 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074c0d19b0 con 0x7f074c0e5470 2021-02-10T11:14:15.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.808 7f0751c88700 1 --2- 172.21.2.101:0/1564496231 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f074c0e5470 0x7f074c07e6a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f0748006650 tx=0x7f0748009a80).ready entity=mon.0 client_cookie=5406da06cb8df7d0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:15.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.808 7f0751487700 1 -- 172.21.2.101:0/1564496231 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f074800c0c0 con 0x7f074c0e5470 2021-02-10T11:14:15.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.808 7f0751487700 1 -- 172.21.2.101:0/1564496231 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074800c340 con 0x7f074c0e5470 2021-02-10T11:14:15.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.808 7f0751487700 1 -- 172.21.2.101:0/1564496231 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f074800c7a0 con 0x7f074c0e5470 2021-02-10T11:14:15.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.808 7f075466d700 1 -- 172.21.2.101:0/1564496231 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f074c0e5470 msgr2=0x7f074c07e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:15.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.808 7f075466d700 1 --2- 172.21.2.101:0/1564496231 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f074c0e5470 0x7f074c07e6a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f0748006650 tx=0x7f0748009a80).stop 2021-02-10T11:14:15.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.809 7f075466d700 1 -- 172.21.2.101:0/1564496231 shutdown_connections 2021-02-10T11:14:15.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.809 7f075466d700 1 --2- 172.21.2.101:0/1564496231 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f074c0e5470 0x7f074c07e6a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:15.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.809 7f075466d700 1 -- 172.21.2.101:0/1564496231 >> 172.21.2.101:0/1564496231 conn(0x7f074c0ce690 msgr2=0x7f074c0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:15.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.809 7f075466d700 1 -- 172.21.2.101:0/1564496231 shutdown_connections 2021-02-10T11:14:15.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.809 7f075466d700 1 -- 172.21.2.101:0/1564496231 wait complete. 2021-02-10T11:14:15.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.823 7f075466d700 1 Processor -- start 2021-02-10T11:14:15.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.823 7f075466d700 1 -- start start 2021-02-10T11:14:15.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.824 7f075466d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f074c1c55a0 0x7f074c1c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:15.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.824 7f075466d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f074c0bfdc0 0x7f074c0c01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:15.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.824 7f075466d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f074c099900 0x7f074c099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:15.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.824 7f075466d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c0c09f0 con 0x7f074c099900 2021-02-10T11:14:15.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.824 7f075466d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c09a320 con 0x7f074c1c55a0 2021-02-10T11:14:15.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.824 7f075466d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f074c09a540 con 0x7f074c0bfdc0 2021-02-10T11:14:15.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.824 7f0752c8a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f074c099900 0x7f074c099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:15.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.824 7f0752c8a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f074c099900 0x7f074c099d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44970/0 (socket says 172.21.2.101:44970) 2021-02-10T11:14:15.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.824 7f0752c8a700 1 -- 172.21.2.101:0/4122403070 learned_addr learned my addr 172.21.2.101:0/4122403070 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:15.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.824 7f0752c8a700 1 -- 172.21.2.101:0/4122403070 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f074c0bfdc0 msgr2=0x7f074c0c01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:15.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.824 7f0752c8a700 1 --2- 172.21.2.101:0/4122403070 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f074c0bfdc0 0x7f074c0c01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:15.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.824 7f0752c8a700 1 -- 172.21.2.101:0/4122403070 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f074c1c55a0 msgr2=0x7f074c1c59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:15.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.824 7f0752c8a700 1 --2- 172.21.2.101:0/4122403070 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f074c1c55a0 0x7f074c1c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:15.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.824 7f0752c8a700 1 -- 172.21.2.101:0/4122403070 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074c09a760 con 0x7f074c099900 2021-02-10T11:14:15.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.825 7f0752c8a700 1 --2- 172.21.2.101:0/4122403070 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f074c099900 0x7f074c099d00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f0744009940 tx=0x7f074400bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:15.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.825 7f073effd700 1 -- 172.21.2.101:0/4122403070 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0744000f90 con 0x7f074c099900 2021-02-10T11:14:15.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.825 7f073effd700 1 -- 172.21.2.101:0/4122403070 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074400c2f0 con 0x7f074c099900 2021-02-10T11:14:15.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.825 7f073effd700 1 -- 172.21.2.101:0/4122403070 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f074400c750 con 0x7f074c099900 2021-02-10T11:14:15.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.825 7f075466d700 1 -- 172.21.2.101:0/4122403070 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f074c09a760 con 0x7f074c099900 2021-02-10T11:14:15.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.825 7f075466d700 1 -- 172.21.2.101:0/4122403070 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f074c09a760 con 0x7f074c099900 2021-02-10T11:14:15.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.825 7f073effd700 1 -- 172.21.2.101:0/4122403070 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0744001220 con 0x7f074c099900 2021-02-10T11:14:15.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.825 7f073effd700 1 --2- 172.21.2.101:0/4122403070 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07340186c0 0x7f073401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:15.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.826 7f073effd700 1 -- 172.21.2.101:0/4122403070 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0744032910 con 0x7f074c099900 2021-02-10T11:14:15.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.826 7f0752489700 1 --2- 172.21.2.101:0/4122403070 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07340186c0 0x7f073401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:15.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.826 7f0752489700 1 --2- 172.21.2.101:0/4122403070 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07340186c0 0x7f073401abc0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:15.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.857 7f075466d700 1 -- 172.21.2.101:0/4122403070 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f074c0ce690 con 0x7f074c099900 2021-02-10T11:14:15.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.860 7f073effd700 1 -- 172.21.2.101:0/4122403070 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0744000d30 con 0x7f074c099900 2021-02-10T11:14:15.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.984 7f075466d700 1 -- 172.21.2.101:0/4122403070 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f074c0ce690 con 0x7f074c099900 2021-02-10T11:14:15.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.985 7f073effd700 1 -- 172.21.2.101:0/4122403070 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+13993 (secure 0 0 0) 0x7f0744001220 con 0x7f074c099900 2021-02-10T11:14:15.990 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:14:15.990 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":20,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:13:15.149234","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:13:11.922272","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2021-02-10 11:13:13.141733","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:14:15.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.987 7f075466d700 1 -- 172.21.2.101:0/4122403070 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07340186c0 msgr2=0x7f073401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:15.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.987 7f075466d700 1 --2- 172.21.2.101:0/4122403070 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07340186c0 0x7f073401abc0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:15.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.987 7f075466d700 1 -- 172.21.2.101:0/4122403070 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f074c099900 msgr2=0x7f074c099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:15.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.987 7f075466d700 1 --2- 172.21.2.101:0/4122403070 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f074c099900 0x7f074c099d00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f0744009940 tx=0x7f074400bcf0).stop 2021-02-10T11:14:15.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.987 7f075466d700 1 -- 172.21.2.101:0/4122403070 shutdown_connections 2021-02-10T11:14:15.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.987 7f075466d700 1 --2- 172.21.2.101:0/4122403070 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07340186c0 0x7f073401abc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:15.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.987 7f075466d700 1 --2- 172.21.2.101:0/4122403070 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f074c0bfdc0 0x7f074c0c01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:15.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.987 7f075466d700 1 --2- 172.21.2.101:0/4122403070 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f074c099900 0x7f074c099d00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:15.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.987 7f075466d700 1 --2- 172.21.2.101:0/4122403070 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f074c1c55a0 0x7f074c1c59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:15.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.987 7f075466d700 1 -- 172.21.2.101:0/4122403070 >> 172.21.2.101:0/4122403070 conn(0x7f074c0976b0 msgr2=0x7f074c0962b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:15.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.987 7f075466d700 1 -- 172.21.2.101:0/4122403070 shutdown_connections 2021-02-10T11:14:15.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:15.987 7f075466d700 1 -- 172.21.2.101:0/4122403070 wait complete. 2021-02-10T11:14:16.000 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:13:11.922272', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:13:13.141733', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:14:16.000 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:14:16.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.408 7fdc61885700 1 Processor -- start 2021-02-10T11:14:16.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.408 7fdc61885700 1 -- start start 2021-02-10T11:14:16.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.408 7fdc61885700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fdc5c1b0470 0x7fdc5c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:16.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.408 7fdc61885700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc5c0ef610 0x7fdc5c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:16.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.408 7fdc61885700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc5c0f0b50 con 0x7fdc5c0f0030 2021-02-10T11:14:16.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.408 7fdc61885700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc5c0e52e0 con 0x7fdc5c1b0470 2021-02-10T11:14:16.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.408 7fdc61885700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c0e54d0 con 0x7fdc5c0ef610 2021-02-10T11:14:16.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.408 7fdc5bd75700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fdc5c0f0030 0x7fdc5c0f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45472/0 (socket says 172.21.2.101:45472) 2021-02-10T11:14:16.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.408 7fdc5bd75700 1 -- 172.21.2.101:0/88561979 learned_addr learned my addr 172.21.2.101:0/88561979 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:14:16.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.409 7fdc5ad73700 1 --2- 172.21.2.101:0/88561979 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc5c0ef610 0x7fdc5c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:16.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.409 7fdc5b574700 1 --2- 172.21.2.101:0/88561979 >> v2:172.21.2.108:3300/0 conn(0x7fdc5c1b0470 0x7fdc5c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:16.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.409 7fdc5a572700 1 -- 172.21.2.101:0/88561979 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996203155 0 0) 0x7fdc54001500 con 0x7fdc5c0f0030 2021-02-10T11:14:16.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.409 7fdc5a572700 1 -- 172.21.2.101:0/88561979 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc48002fe0 con 0x7fdc5c0f0030 2021-02-10T11:14:16.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.409 7fdc5ad73700 1 -- 172.21.2.101:0/88561979 >> v2:172.21.2.108:3300/0 conn(0x7fdc5c1b0470 msgr2=0x7fdc5c1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:16.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.409 7fdc5ad73700 1 --2- 172.21.2.101:0/88561979 >> v2:172.21.2.108:3300/0 conn(0x7fdc5c1b0470 0x7fdc5c1b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:16.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.409 7fdc5ad73700 1 -- 172.21.2.101:0/88561979 >> v1:172.21.2.101:6789/0 conn(0x7fdc5c0f0030 legacy=0x7fdc5c0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:16.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.409 7fdc5ad73700 1 -- 172.21.2.101:0/88561979 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc5c0f0430 con 0x7fdc5c0ef610 2021-02-10T11:14:16.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.409 7fdc5b574700 1 --2- 172.21.2.101:0/88561979 >> v2:172.21.2.108:3300/0 conn(0x7fdc5c1b0470 0x7fdc5c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:14:16.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.409 7fdc5ad73700 1 --2- 172.21.2.101:0/88561979 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc5c0ef610 0x7fdc5c0efa10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fdc4c006650 tx=0x7fdc4c009a80).ready entity=mon.2 client_cookie=18a9b4da789a719c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:16.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.410 7fdc5a572700 1 -- 172.21.2.101:0/88561979 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc4c00c0c0 con 0x7fdc5c0ef610 2021-02-10T11:14:16.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.410 7fdc5a572700 1 -- 172.21.2.101:0/88561979 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4c00c340 con 0x7fdc5c0ef610 2021-02-10T11:14:16.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.410 7fdc5a572700 1 -- 172.21.2.101:0/88561979 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc4c00c860 con 0x7fdc5c0ef610 2021-02-10T11:14:16.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.410 7fdc61885700 1 -- 172.21.2.101:0/88561979 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc5c0ef610 msgr2=0x7fdc5c0efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:16.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.410 7fdc61885700 1 --2- 172.21.2.101:0/88561979 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc5c0ef610 0x7fdc5c0efa10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fdc4c006650 tx=0x7fdc4c009a80).stop 2021-02-10T11:14:16.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.410 7fdc61885700 1 -- 172.21.2.101:0/88561979 shutdown_connections 2021-02-10T11:14:16.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.410 7fdc61885700 1 --2- 172.21.2.101:0/88561979 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc5c0ef610 0x7fdc5c0efa10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:16.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.410 7fdc61885700 1 --2- 172.21.2.101:0/88561979 >> v2:172.21.2.108:3300/0 conn(0x7fdc5c1b0470 0x7fdc5c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:16.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.410 7fdc61885700 1 -- 172.21.2.101:0/88561979 >> 172.21.2.101:0/88561979 conn(0x7fdc5c0ce3e0 msgr2=0x7fdc5c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:16.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.410 7fdc61885700 1 -- 172.21.2.101:0/88561979 shutdown_connections 2021-02-10T11:14:16.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.410 7fdc61885700 1 -- 172.21.2.101:0/88561979 wait complete. 2021-02-10T11:14:16.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.425 7fdc61885700 1 Processor -- start 2021-02-10T11:14:16.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.425 7fdc61885700 1 -- start start 2021-02-10T11:14:16.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.425 7fdc61885700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc5c0a0420 0x7fdc5c0a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:16.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.425 7fdc61885700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc5c189d00 0x7fdc5c18a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:16.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.425 7fdc61885700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc5c18a720 0x7fdc5c0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:16.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.425 7fdc61885700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c189800 con 0x7fdc5c189d00 2021-02-10T11:14:16.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.425 7fdc61885700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c18ab20 con 0x7fdc5c0a0420 2021-02-10T11:14:16.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.425 7fdc61885700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c0a1140 con 0x7fdc5c18a720 2021-02-10T11:14:16.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.426 7fdc5ad73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc5c189d00 0x7fdc5c18a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:16.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.426 7fdc5ad73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc5c189d00 0x7fdc5c18a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44982/0 (socket says 172.21.2.101:44982) 2021-02-10T11:14:16.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.426 7fdc5ad73700 1 -- 172.21.2.101:0/4076993754 learned_addr learned my addr 172.21.2.101:0/4076993754 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:16.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.426 7fdc5ad73700 1 -- 172.21.2.101:0/4076993754 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc5c18a720 msgr2=0x7fdc5c0b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:16.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.426 7fdc5ad73700 1 --2- 172.21.2.101:0/4076993754 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc5c18a720 0x7fdc5c0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:16.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.426 7fdc5ad73700 1 -- 172.21.2.101:0/4076993754 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc5c0a0420 msgr2=0x7fdc5c0a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:16.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.426 7fdc5ad73700 1 --2- 172.21.2.101:0/4076993754 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc5c0a0420 0x7fdc5c0a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:16.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.426 7fdc5ad73700 1 -- 172.21.2.101:0/4076993754 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc5c0b8200 con 0x7fdc5c189d00 2021-02-10T11:14:16.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.426 7fdc5ad73700 1 --2- 172.21.2.101:0/4076993754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc5c189d00 0x7fdc5c18a100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fdc4c0096a0 tx=0x7fdc4c00b8e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:16.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.426 7fdc53fff700 1 -- 172.21.2.101:0/4076993754 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc4c0093a0 con 0x7fdc5c189d00 2021-02-10T11:14:16.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.426 7fdc53fff700 1 -- 172.21.2.101:0/4076993754 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4c0067f0 con 0x7fdc5c189d00 2021-02-10T11:14:16.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.426 7fdc53fff700 1 -- 172.21.2.101:0/4076993754 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc4c002220 con 0x7fdc5c189d00 2021-02-10T11:14:16.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.426 7fdc61885700 1 -- 172.21.2.101:0/4076993754 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc5c0b8200 con 0x7fdc5c189d00 2021-02-10T11:14:16.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.426 7fdc61885700 1 -- 172.21.2.101:0/4076993754 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5c0b8790 con 0x7fdc5c189d00 2021-02-10T11:14:16.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.427 7fdc53fff700 1 -- 172.21.2.101:0/4076993754 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdc4c000c90 con 0x7fdc5c189d00 2021-02-10T11:14:16.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.427 7fdc53fff700 1 --2- 172.21.2.101:0/4076993754 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc400186c0 0x7fdc4001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:16.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.427 7fdc53fff700 1 -- 172.21.2.101:0/4076993754 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fdc4c032500 con 0x7fdc5c189d00 2021-02-10T11:14:16.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.427 7fdc5b574700 1 --2- 172.21.2.101:0/4076993754 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc400186c0 0x7fdc4001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:16.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.427 7fdc5b574700 1 --2- 172.21.2.101:0/4076993754 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc400186c0 0x7fdc4001abc0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:16.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.458 7fdc61885700 1 -- 172.21.2.101:0/4076993754 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc5c105ae0 con 0x7fdc5c189d00 2021-02-10T11:14:16.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.461 7fdc53fff700 1 -- 172.21.2.101:0/4076993754 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdc4c0093a0 con 0x7fdc5c189d00 2021-02-10T11:14:16.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.589 7fdc61885700 1 -- 172.21.2.101:0/4076993754 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fdc5c0f0b50 con 0x7fdc5c189d00 2021-02-10T11:14:16.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.589 7fdc53fff700 1 -- 172.21.2.101:0/4076993754 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdc4c000c90 con 0x7fdc5c189d00 2021-02-10T11:14:16.594 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:14:16.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.591 7fdc61885700 1 -- 172.21.2.101:0/4076993754 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc400186c0 msgr2=0x7fdc4001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:16.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.591 7fdc61885700 1 --2- 172.21.2.101:0/4076993754 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc400186c0 0x7fdc4001abc0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:16.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.591 7fdc61885700 1 -- 172.21.2.101:0/4076993754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc5c189d00 msgr2=0x7fdc5c18a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:16.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.591 7fdc61885700 1 --2- 172.21.2.101:0/4076993754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc5c189d00 0x7fdc5c18a100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fdc4c0096a0 tx=0x7fdc4c00b8e0).stop 2021-02-10T11:14:16.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.591 7fdc61885700 1 -- 172.21.2.101:0/4076993754 shutdown_connections 2021-02-10T11:14:16.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.591 7fdc61885700 1 --2- 172.21.2.101:0/4076993754 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc400186c0 0x7fdc4001abc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:16.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.591 7fdc61885700 1 --2- 172.21.2.101:0/4076993754 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc5c18a720 0x7fdc5c0b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:16.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.591 7fdc61885700 1 --2- 172.21.2.101:0/4076993754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc5c189d00 0x7fdc5c18a100 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:16.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.591 7fdc61885700 1 --2- 172.21.2.101:0/4076993754 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc5c0a0420 0x7fdc5c0a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:16.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.591 7fdc61885700 1 -- 172.21.2.101:0/4076993754 >> 172.21.2.101:0/4076993754 conn(0x7fdc5c094e10 msgr2=0x7fdc5c0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:16.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.591 7fdc61885700 1 -- 172.21.2.101:0/4076993754 shutdown_connections 2021-02-10T11:14:16.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:16.591 7fdc61885700 1 -- 172.21.2.101:0/4076993754 wait complete. 2021-02-10T11:14:16.604 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:14:17.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.012 7f2612450700 1 Processor -- start 2021-02-10T11:14:17.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.012 7f2612450700 1 -- start start 2021-02-10T11:14:17.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.012 7f2612450700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f260c1b0870 0x7f260c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:17.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.012 7f2612450700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f260c0efe00 0x7f260c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:17.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.012 7f2612450700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f260c1b0c70 con 0x7f260c1b0470 2021-02-10T11:14:17.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.012 7f2612450700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f260c0f0b50 con 0x7f260c1b0870 2021-02-10T11:14:17.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.012 7f2612450700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260c0f0820 con 0x7f260c0efe00 2021-02-10T11:14:17.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.012 7f260b7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f260c1b0870 0x7f260c0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:17.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.012 7f260b7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f260c1b0870 0x7f260c0f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44988/0 (socket says 172.21.2.101:44988) 2021-02-10T11:14:17.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.012 7f260b7fe700 1 -- 172.21.2.101:0/2424412184 learned_addr learned my addr 172.21.2.101:0/2424412184 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:17.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.013 7f260b7fe700 1 -- 172.21.2.101:0/2424412184 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f260c0efe00 msgr2=0x7f260c0f0200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:17.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.013 7f260b7fe700 1 --2- 172.21.2.101:0/2424412184 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f260c0efe00 0x7f260c0f0200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:17.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.013 7f260b7fe700 1 -- 172.21.2.101:0/2424412184 >> v1:172.21.2.108:6789/0 conn(0x7f260c1b0470 legacy=0x7f260c0ef610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:17.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.013 7f260b7fe700 1 -- 172.21.2.101:0/2424412184 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260c0e52e0 con 0x7f260c1b0870 2021-02-10T11:14:17.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.013 7f260b7fe700 1 --2- 172.21.2.101:0/2424412184 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f260c1b0870 0x7f260c0f20e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f25f4006700 tx=0x7f25f4009b80).ready entity=mon.0 client_cookie=d466552cc32e105e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:17.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.013 7f260affd700 1 -- 172.21.2.101:0/2424412184 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f25f400c0c0 con 0x7f260c1b0870 2021-02-10T11:14:17.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.013 7f260affd700 1 -- 172.21.2.101:0/2424412184 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f400c340 con 0x7f260c1b0870 2021-02-10T11:14:17.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.013 7f260affd700 1 -- 172.21.2.101:0/2424412184 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f25f400c860 con 0x7f260c1b0870 2021-02-10T11:14:17.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.013 7f2612450700 1 -- 172.21.2.101:0/2424412184 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f260c1b0870 msgr2=0x7f260c0f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:17.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.013 7f2612450700 1 --2- 172.21.2.101:0/2424412184 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f260c1b0870 0x7f260c0f20e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f25f4006700 tx=0x7f25f4009b80).stop 2021-02-10T11:14:17.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.013 7f2612450700 1 -- 172.21.2.101:0/2424412184 shutdown_connections 2021-02-10T11:14:17.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.013 7f2612450700 1 --2- 172.21.2.101:0/2424412184 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f260c0efe00 0x7f260c0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:17.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.013 7f2612450700 1 --2- 172.21.2.101:0/2424412184 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f260c1b0870 0x7f260c0f20e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:17.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.013 7f2612450700 1 -- 172.21.2.101:0/2424412184 >> 172.21.2.101:0/2424412184 conn(0x7f260c0ce3e0 msgr2=0x7f260c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:17.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.014 7f2612450700 1 -- 172.21.2.101:0/2424412184 shutdown_connections 2021-02-10T11:14:17.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.014 7f2612450700 1 -- 172.21.2.101:0/2424412184 wait complete. 2021-02-10T11:14:17.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.028 7f2612450700 1 Processor -- start 2021-02-10T11:14:17.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.028 7f2612450700 1 -- start start 2021-02-10T11:14:17.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.029 7f2612450700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f260c0c36a0 0x7f260c0c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:17.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.029 7f2612450700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f260c0c3190 0x7f260c0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:17.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.029 7f2612450700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f260c0b2370 0x7f260c0b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:17.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.029 7f2612450700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260c0b27c0 con 0x7f260c0b2370 2021-02-10T11:14:17.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.029 7f2612450700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260c0b3360 con 0x7f260c0c36a0 2021-02-10T11:14:17.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.029 7f2612450700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260c0b3540 con 0x7f260c0c3190 2021-02-10T11:14:17.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.029 7f2610a6d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f260c0b2370 0x7f260c0b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:17.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.029 7f2610a6d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f260c0b2370 0x7f260c0b2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:44994/0 (socket says 172.21.2.101:44994) 2021-02-10T11:14:17.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.029 7f2610a6d700 1 -- 172.21.2.101:0/4055980345 learned_addr learned my addr 172.21.2.101:0/4055980345 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:17.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.029 7f2610a6d700 1 -- 172.21.2.101:0/4055980345 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f260c0c3190 msgr2=0x7f260c0b1d50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:17.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.029 7f2610a6d700 1 --2- 172.21.2.101:0/4055980345 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f260c0c3190 0x7f260c0b1d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:17.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.029 7f2610a6d700 1 -- 172.21.2.101:0/4055980345 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f260c0c36a0 msgr2=0x7f260c0c2b70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:17.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.029 7f2610a6d700 1 --2- 172.21.2.101:0/4055980345 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f260c0c36a0 0x7f260c0c2b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:17.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.029 7f2610a6d700 1 -- 172.21.2.101:0/4055980345 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260c0b3760 con 0x7f260c0b2370 2021-02-10T11:14:17.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.029 7f2610a6d700 1 --2- 172.21.2.101:0/4055980345 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f260c0b2370 0x7f260c0b2d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f260400b5c0 tx=0x7f260400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:17.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.030 7f2608ff9700 1 -- 172.21.2.101:0/4055980345 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2604006b80 con 0x7f260c0b2370 2021-02-10T11:14:17.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.030 7f2608ff9700 1 -- 172.21.2.101:0/4055980345 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260400bd50 con 0x7f260c0b2370 2021-02-10T11:14:17.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.030 7f2608ff9700 1 -- 172.21.2.101:0/4055980345 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f26040054d0 con 0x7f260c0b2370 2021-02-10T11:14:17.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.030 7f2612450700 1 -- 172.21.2.101:0/4055980345 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f260c0b3760 con 0x7f260c0b2370 2021-02-10T11:14:17.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.030 7f2612450700 1 -- 172.21.2.101:0/4055980345 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f260c0b3760 con 0x7f260c0b2370 2021-02-10T11:14:17.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.030 7f2608ff9700 1 -- 172.21.2.101:0/4055980345 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2604000a10 con 0x7f260c0b2370 2021-02-10T11:14:17.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.030 7f2608ff9700 1 --2- 172.21.2.101:0/4055980345 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25f00186c0 0x7f25f001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:17.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.030 7f2608ff9700 1 -- 172.21.2.101:0/4055980345 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f26040325f0 con 0x7f260c0b2370 2021-02-10T11:14:17.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.030 7f260bfff700 1 --2- 172.21.2.101:0/4055980345 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25f00186c0 0x7f25f001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:17.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.031 7f260bfff700 1 --2- 172.21.2.101:0/4055980345 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25f00186c0 0x7f25f001abc0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:17.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.062 7f2612450700 1 -- 172.21.2.101:0/4055980345 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f260c0edd10 con 0x7f260c0b2370 2021-02-10T11:14:17.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.064 7f2608ff9700 1 -- 172.21.2.101:0/4055980345 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2604030da0 con 0x7f260c0b2370 2021-02-10T11:14:17.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.193 7f2612450700 1 -- 172.21.2.101:0/4055980345 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f260c0edd10 con 0x7f260c0b2370 2021-02-10T11:14:17.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.194 7f2608ff9700 1 -- 172.21.2.101:0/4055980345 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v20) v1 ==== 104+0+10 (secure 0 0 0) 0x7f26040009f0 con 0x7f260c0b2370 2021-02-10T11:14:17.198 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:14:17.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.195 7f2612450700 1 -- 172.21.2.101:0/4055980345 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25f00186c0 msgr2=0x7f25f001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:17.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.195 7f2612450700 1 --2- 172.21.2.101:0/4055980345 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25f00186c0 0x7f25f001abc0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:17.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.196 7f2612450700 1 -- 172.21.2.101:0/4055980345 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f260c0b2370 msgr2=0x7f260c0b2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:17.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.196 7f2612450700 1 --2- 172.21.2.101:0/4055980345 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f260c0b2370 0x7f260c0b2d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f260400b5c0 tx=0x7f260400b960).stop 2021-02-10T11:14:17.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.196 7f2612450700 1 -- 172.21.2.101:0/4055980345 shutdown_connections 2021-02-10T11:14:17.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.196 7f2612450700 1 --2- 172.21.2.101:0/4055980345 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25f00186c0 0x7f25f001abc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:17.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.196 7f2612450700 1 --2- 172.21.2.101:0/4055980345 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f260c0c3190 0x7f260c0b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:17.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.196 7f2612450700 1 --2- 172.21.2.101:0/4055980345 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f260c0b2370 0x7f260c0b2d40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:17.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.196 7f2612450700 1 --2- 172.21.2.101:0/4055980345 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f260c0c36a0 0x7f260c0c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:17.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.196 7f2612450700 1 -- 172.21.2.101:0/4055980345 >> 172.21.2.101:0/4055980345 conn(0x7f260c09f630 msgr2=0x7f260c0a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:17.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.196 7f2612450700 1 -- 172.21.2.101:0/4055980345 shutdown_connections 2021-02-10T11:14:17.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.196 7f2612450700 1 -- 172.21.2.101:0/4055980345 wait complete. 2021-02-10T11:14:17.208 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:14:17.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.618 7fde1eb4e700 1 Processor -- start 2021-02-10T11:14:17.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.618 7fde1eb4e700 1 -- start start 2021-02-10T11:14:17.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.618 7fde1eb4e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fde181b0470 0x7fde181b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:17.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.618 7fde1eb4e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fde180f20e0 0x7fde180f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:17.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.618 7fde1eb4e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde180f0b50 con 0x7fde180ef610 2021-02-10T11:14:17.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.618 7fde1eb4e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde180f0820 con 0x7fde181b0470 2021-02-10T11:14:17.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.618 7fde1eb4e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde180f24e0 con 0x7fde180f20e0 2021-02-10T11:14:17.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.618 7fde1c96a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fde181b0470 0x7fde181b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:17.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.618 7fde1c96a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fde181b0470 0x7fde181b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45002/0 (socket says 172.21.2.101:45002) 2021-02-10T11:14:17.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.618 7fde1c96a700 1 -- 172.21.2.101:0/3401483232 learned_addr learned my addr 172.21.2.101:0/3401483232 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:17.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.618 7fde1c96a700 1 -- 172.21.2.101:0/3401483232 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fde180f20e0 msgr2=0x7fde180f0200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:17.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.618 7fde1c96a700 1 --2- 172.21.2.101:0/3401483232 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fde180f20e0 0x7fde180f0200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:17.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.618 7fde1c96a700 1 -- 172.21.2.101:0/3401483232 >> v1:172.21.2.101:6789/0 conn(0x7fde180ef610 legacy=0x7fde180efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:17.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.618 7fde1c96a700 1 -- 172.21.2.101:0/3401483232 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde180e52e0 con 0x7fde181b0470 2021-02-10T11:14:17.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.619 7fde1c96a700 1 --2- 172.21.2.101:0/3401483232 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fde181b0470 0x7fde181b0870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fde00006700 tx=0x7fde00009b80).ready entity=mon.0 client_cookie=80e8b36bd7162f58 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:17.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.619 7fde177fe700 1 -- 172.21.2.101:0/3401483232 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fde0000c0c0 con 0x7fde181b0470 2021-02-10T11:14:17.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.619 7fde177fe700 1 -- 172.21.2.101:0/3401483232 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0000c340 con 0x7fde181b0470 2021-02-10T11:14:17.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.619 7fde177fe700 1 -- 172.21.2.101:0/3401483232 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fde0000c860 con 0x7fde181b0470 2021-02-10T11:14:17.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.619 7fde1eb4e700 1 -- 172.21.2.101:0/3401483232 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fde181b0470 msgr2=0x7fde181b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:17.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.619 7fde1eb4e700 1 --2- 172.21.2.101:0/3401483232 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fde181b0470 0x7fde181b0870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fde00006700 tx=0x7fde00009b80).stop 2021-02-10T11:14:17.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.619 7fde1eb4e700 1 -- 172.21.2.101:0/3401483232 shutdown_connections 2021-02-10T11:14:17.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.619 7fde1eb4e700 1 --2- 172.21.2.101:0/3401483232 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fde180f20e0 0x7fde180f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:17.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.619 7fde1eb4e700 1 --2- 172.21.2.101:0/3401483232 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fde181b0470 0x7fde181b0870 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:17.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.619 7fde1eb4e700 1 -- 172.21.2.101:0/3401483232 >> 172.21.2.101:0/3401483232 conn(0x7fde180ce3e0 msgr2=0x7fde180e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:17.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.619 7fde1eb4e700 1 -- 172.21.2.101:0/3401483232 shutdown_connections 2021-02-10T11:14:17.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.619 7fde1eb4e700 1 -- 172.21.2.101:0/3401483232 wait complete. 2021-02-10T11:14:17.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.634 7fde1eb4e700 1 Processor -- start 2021-02-10T11:14:17.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.634 7fde1eb4e700 1 -- start start 2021-02-10T11:14:17.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.634 7fde1eb4e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fde180c36a0 0x7fde180c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:17.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.634 7fde1eb4e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fde180c3190 0x7fde180b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:17.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.634 7fde1eb4e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fde180b2370 0x7fde180b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:17.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.634 7fde1eb4e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde180b27c0 con 0x7fde180c3190 2021-02-10T11:14:17.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.634 7fde1eb4e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde180b3360 con 0x7fde180c36a0 2021-02-10T11:14:17.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.634 7fde1eb4e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde180b3540 con 0x7fde180b2370 2021-02-10T11:14:17.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.634 7fde17fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fde180c3190 0x7fde180b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:17.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.634 7fde17fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fde180c3190 0x7fde180b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45012/0 (socket says 172.21.2.101:45012) 2021-02-10T11:14:17.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.634 7fde17fff700 1 -- 172.21.2.101:0/2934918235 learned_addr learned my addr 172.21.2.101:0/2934918235 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:17.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.635 7fde17fff700 1 -- 172.21.2.101:0/2934918235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fde180b2370 msgr2=0x7fde180b2d40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:17.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.635 7fde17fff700 1 --2- 172.21.2.101:0/2934918235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fde180b2370 0x7fde180b2d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:17.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.635 7fde17fff700 1 -- 172.21.2.101:0/2934918235 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fde180c36a0 msgr2=0x7fde180c2b70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:17.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.635 7fde17fff700 1 --2- 172.21.2.101:0/2934918235 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fde180c36a0 0x7fde180c2b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:17.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.635 7fde17fff700 1 -- 172.21.2.101:0/2934918235 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde180b3760 con 0x7fde180c3190 2021-02-10T11:14:17.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.635 7fde17fff700 1 --2- 172.21.2.101:0/2934918235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fde180c3190 0x7fde180b1d50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fde08006f00 tx=0x7fde0800b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:17.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.635 7fde157fa700 1 -- 172.21.2.101:0/2934918235 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fde08007070 con 0x7fde180c3190 2021-02-10T11:14:17.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.635 7fde157fa700 1 -- 172.21.2.101:0/2934918235 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde08007270 con 0x7fde180c3190 2021-02-10T11:14:17.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.635 7fde157fa700 1 -- 172.21.2.101:0/2934918235 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fde08007790 con 0x7fde180c3190 2021-02-10T11:14:17.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.635 7fde1eb4e700 1 -- 172.21.2.101:0/2934918235 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde180b3760 con 0x7fde180c3190 2021-02-10T11:14:17.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.635 7fde1eb4e700 1 -- 172.21.2.101:0/2934918235 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde180b3cf0 con 0x7fde180c3190 2021-02-10T11:14:17.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.636 7fde157fa700 1 -- 172.21.2.101:0/2934918235 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fde08002020 con 0x7fde180c3190 2021-02-10T11:14:17.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.636 7fde157fa700 1 --2- 172.21.2.101:0/2934918235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fddfc0186c0 0x7fddfc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:17.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.636 7fde157fa700 1 -- 172.21.2.101:0/2934918235 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fde08032580 con 0x7fde180c3190 2021-02-10T11:14:17.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.636 7fde1c96a700 1 --2- 172.21.2.101:0/2934918235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fddfc0186c0 0x7fddfc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:17.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.636 7fde1c96a700 1 --2- 172.21.2.101:0/2934918235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fddfc0186c0 0x7fddfc01abc0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:17.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.668 7fde1eb4e700 1 -- 172.21.2.101:0/2934918235 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde180edd10 con 0x7fde180c3190 2021-02-10T11:14:17.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.670 7fde157fa700 1 -- 172.21.2.101:0/2934918235 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fde08030d30 con 0x7fde180c3190 2021-02-10T11:14:17.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.796 7fde1eb4e700 1 -- 172.21.2.101:0/2934918235 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7fde180e52e0 con 0x7fde180c3190 2021-02-10T11:14:17.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.797 7fde157fa700 1 -- 172.21.2.101:0/2934918235 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v20) v1 ==== 100+0+10 (secure 0 0 0) 0x7fde08002000 con 0x7fde180c3190 2021-02-10T11:14:17.802 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:14:17.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.799 7fde1eb4e700 1 -- 172.21.2.101:0/2934918235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fddfc0186c0 msgr2=0x7fddfc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:17.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.799 7fde1eb4e700 1 --2- 172.21.2.101:0/2934918235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fddfc0186c0 0x7fddfc01abc0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:17.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.799 7fde1eb4e700 1 -- 172.21.2.101:0/2934918235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fde180c3190 msgr2=0x7fde180b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:17.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.799 7fde1eb4e700 1 --2- 172.21.2.101:0/2934918235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fde180c3190 0x7fde180b1d50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fde08006f00 tx=0x7fde0800b940).stop 2021-02-10T11:14:17.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.799 7fde1eb4e700 1 -- 172.21.2.101:0/2934918235 shutdown_connections 2021-02-10T11:14:17.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.799 7fde1eb4e700 1 --2- 172.21.2.101:0/2934918235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fddfc0186c0 0x7fddfc01abc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:17.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.799 7fde1eb4e700 1 --2- 172.21.2.101:0/2934918235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fde180b2370 0x7fde180b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:17.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.799 7fde1eb4e700 1 --2- 172.21.2.101:0/2934918235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fde180c3190 0x7fde180b1d50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:17.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.799 7fde1eb4e700 1 --2- 172.21.2.101:0/2934918235 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fde180c36a0 0x7fde180c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:17.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.799 7fde1eb4e700 1 -- 172.21.2.101:0/2934918235 >> 172.21.2.101:0/2934918235 conn(0x7fde1809f630 msgr2=0x7fde180a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:17.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.799 7fde1eb4e700 1 -- 172.21.2.101:0/2934918235 shutdown_connections 2021-02-10T11:14:17.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:17.799 7fde1eb4e700 1 -- 172.21.2.101:0/2934918235 wait complete. 2021-02-10T11:14:17.850 INFO:tasks.cephfs_test_runner:Starting test: test_async_subvolume_rm (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:14:17.850 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_async_subvolume_rm' 2021-02-10T11:14:18.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.256 7f2c045eb700 1 Processor -- start 2021-02-10T11:14:18.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.256 7f2c045eb700 1 -- start start 2021-02-10T11:14:18.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.256 7f2c045eb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bfc0e5070 0x7f2bfc1b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:18.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.256 7f2c045eb700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2bfc1b3e40 0x7f2bfc080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:18.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.256 7f2c045eb700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bfc07ed80 con 0x7f2bfc0e5470 2021-02-10T11:14:18.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.256 7f2c045eb700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bfc0d1790 con 0x7f2bfc1b3e40 2021-02-10T11:14:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.256 7f2c045eb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0d1980 con 0x7f2bfc0e5070 2021-02-10T11:14:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.256 7f2c02c08700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2bfc1b3e40 0x7f2bfc080310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.256 7f2c02c08700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2bfc1b3e40 0x7f2bfc080310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45016/0 (socket says 172.21.2.101:45016) 2021-02-10T11:14:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.256 7f2c02c08700 1 -- 172.21.2.101:0/2646469093 learned_addr learned my addr 172.21.2.101:0/2646469093 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.256 7f2c02407700 1 --2- 172.21.2.101:0/2646469093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bfc0e5070 0x7f2bfc1b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.257 7f2c02c08700 1 -- 172.21.2.101:0/2646469093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bfc0e5070 msgr2=0x7f2bfc1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.257 7f2c02c08700 1 --2- 172.21.2.101:0/2646469093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bfc0e5070 0x7f2bfc1b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.257 7f2c02c08700 1 -- 172.21.2.101:0/2646469093 >> v1:172.21.2.108:6789/0 conn(0x7f2bfc0e5470 legacy=0x7f2bfc07d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.257 7f2c02c08700 1 -- 172.21.2.101:0/2646469093 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc0d1ba0 con 0x7f2bfc1b3e40 2021-02-10T11:14:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.257 7f2c02c08700 1 --2- 172.21.2.101:0/2646469093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2bfc1b3e40 0x7f2bfc080310 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f2bf4006f00 tx=0x7f2bf4009f10).ready entity=mon.0 client_cookie=dc0412052e697843 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.257 7f2c01405700 1 -- 172.21.2.101:0/2646469093 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2bf400b3b0 con 0x7f2bfc1b3e40 2021-02-10T11:14:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.257 7f2c01405700 1 -- 172.21.2.101:0/2646469093 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf400b730 con 0x7f2bfc1b3e40 2021-02-10T11:14:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.257 7f2c01405700 1 -- 172.21.2.101:0/2646469093 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2bf400bc10 con 0x7f2bfc1b3e40 2021-02-10T11:14:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.257 7f2c045eb700 1 -- 172.21.2.101:0/2646469093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2bfc1b3e40 msgr2=0x7f2bfc080310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.257 7f2c045eb700 1 --2- 172.21.2.101:0/2646469093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2bfc1b3e40 0x7f2bfc080310 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f2bf4006f00 tx=0x7f2bf4009f10).stop 2021-02-10T11:14:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.258 7f2c045eb700 1 -- 172.21.2.101:0/2646469093 shutdown_connections 2021-02-10T11:14:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.258 7f2c045eb700 1 --2- 172.21.2.101:0/2646469093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2bfc1b3e40 0x7f2bfc080310 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.258 7f2c045eb700 1 --2- 172.21.2.101:0/2646469093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bfc0e5070 0x7f2bfc1b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.258 7f2c045eb700 1 -- 172.21.2.101:0/2646469093 >> 172.21.2.101:0/2646469093 conn(0x7f2bfc0ce690 msgr2=0x7f2bfc0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.258 7f2c045eb700 1 -- 172.21.2.101:0/2646469093 shutdown_connections 2021-02-10T11:14:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.258 7f2c045eb700 1 -- 172.21.2.101:0/2646469093 wait complete. 2021-02-10T11:14:18.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.272 7f2c045eb700 1 Processor -- start 2021-02-10T11:14:18.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.272 7f2c045eb700 1 -- start start 2021-02-10T11:14:18.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.272 7f2c045eb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2bfc1a06e0 0x7f2bfc18d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:18.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.272 7f2c045eb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bfc1ab940 0x7f2bfc18d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:18.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.272 7f2c045eb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2bfc18dd70 0x7f2bfc0b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:18.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.272 7f2c045eb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc094120 con 0x7f2bfc1a06e0 2021-02-10T11:14:18.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.272 7f2c045eb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc094340 con 0x7f2bfc18dd70 2021-02-10T11:14:18.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.272 7f2c045eb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0b1810 con 0x7f2bfc1ab940 2021-02-10T11:14:18.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.273 7f2c02407700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2bfc1a06e0 0x7f2bfc18d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.273 7f2c02407700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2bfc1a06e0 0x7f2bfc18d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45022/0 (socket says 172.21.2.101:45022) 2021-02-10T11:14:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.273 7f2c02407700 1 -- 172.21.2.101:0/3067994765 learned_addr learned my addr 172.21.2.101:0/3067994765 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.273 7f2c02c08700 1 --2- 172.21.2.101:0/3067994765 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2bfc18dd70 0x7f2bfc0b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.273 7f2c01c06700 1 --2- 172.21.2.101:0/3067994765 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bfc1ab940 0x7f2bfc18d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.273 7f2c02407700 1 -- 172.21.2.101:0/3067994765 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bfc1ab940 msgr2=0x7f2bfc18d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.273 7f2c02407700 1 --2- 172.21.2.101:0/3067994765 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bfc1ab940 0x7f2bfc18d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.273 7f2c02407700 1 -- 172.21.2.101:0/3067994765 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2bfc18dd70 msgr2=0x7f2bfc0b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.273 7f2c02407700 1 --2- 172.21.2.101:0/3067994765 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2bfc18dd70 0x7f2bfc0b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.273 7f2c02407700 1 -- 172.21.2.101:0/3067994765 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc0b1a30 con 0x7f2bfc1a06e0 2021-02-10T11:14:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.273 7f2c02407700 1 --2- 172.21.2.101:0/3067994765 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2bfc1a06e0 0x7f2bfc18d130 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f2bf800b530 tx=0x7f2bf800b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.273 7f2beeffd700 1 -- 172.21.2.101:0/3067994765 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2bf800bcc0 con 0x7f2bfc1a06e0 2021-02-10T11:14:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.273 7f2c045eb700 1 -- 172.21.2.101:0/3067994765 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bfc0b1a30 con 0x7f2bfc1a06e0 2021-02-10T11:14:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.273 7f2c045eb700 1 -- 172.21.2.101:0/3067994765 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bfc0b1fc0 con 0x7f2bfc1a06e0 2021-02-10T11:14:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.273 7f2beeffd700 1 -- 172.21.2.101:0/3067994765 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf80009b0 con 0x7f2bfc1a06e0 2021-02-10T11:14:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.273 7f2beeffd700 1 -- 172.21.2.101:0/3067994765 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2bf8002030 con 0x7f2bfc1a06e0 2021-02-10T11:14:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.274 7f2beeffd700 1 -- 172.21.2.101:0/3067994765 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2bf8000c90 con 0x7f2bfc1a06e0 2021-02-10T11:14:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.274 7f2beeffd700 1 --2- 172.21.2.101:0/3067994765 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2be4018730 0x7f2be401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.274 7f2beeffd700 1 -- 172.21.2.101:0/3067994765 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2bf80327a0 con 0x7f2bfc1a06e0 2021-02-10T11:14:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.274 7f2c01c06700 1 --2- 172.21.2.101:0/3067994765 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2be4018730 0x7f2be401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.274 7f2c01c06700 1 --2- 172.21.2.101:0/3067994765 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2be4018730 0x7f2be401abc0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:18.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.306 7f2c045eb700 1 -- 172.21.2.101:0/3067994765 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bfc07ed80 con 0x7f2bfc1a06e0 2021-02-10T11:14:18.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.309 7f2beeffd700 1 -- 172.21.2.101:0/3067994765 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2bf8000940 con 0x7f2bfc1a06e0 2021-02-10T11:14:18.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:18.443 7f2c045eb700 1 -- 172.21.2.101:0/3067994765 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_async_subvolume_rm"]} v 0) v1 -- 0x7f2bfc07ed80 con 0x7f2bfc1a06e0 2021-02-10T11:14:19.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.233 7f2beeffd700 1 -- 172.21.2.101:0/3067994765 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_async_subvolume_rm"]}]=0 v70) v1 ==== 143+0+0 (secure 0 0 0) 0x7f2bf8000c90 con 0x7f2bfc1a06e0 2021-02-10T11:14:19.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.263 7f2c045eb700 1 -- 172.21.2.101:0/3067994765 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2be4018730 msgr2=0x7f2be401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:19.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.263 7f2c045eb700 1 --2- 172.21.2.101:0/3067994765 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2be4018730 0x7f2be401abc0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:19.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.263 7f2c045eb700 1 -- 172.21.2.101:0/3067994765 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2bfc1a06e0 msgr2=0x7f2bfc18d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:19.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.263 7f2c045eb700 1 --2- 172.21.2.101:0/3067994765 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2bfc1a06e0 0x7f2bfc18d130 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f2bf800b530 tx=0x7f2bf800b8d0).stop 2021-02-10T11:14:19.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.263 7f2c045eb700 1 -- 172.21.2.101:0/3067994765 shutdown_connections 2021-02-10T11:14:19.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.263 7f2c045eb700 1 --2- 172.21.2.101:0/3067994765 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2be4018730 0x7f2be401abc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:19.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.263 7f2c045eb700 1 --2- 172.21.2.101:0/3067994765 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bfc1ab940 0x7f2bfc18d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:19.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.263 7f2c045eb700 1 --2- 172.21.2.101:0/3067994765 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2bfc18dd70 0x7f2bfc0b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:19.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.263 7f2c045eb700 1 --2- 172.21.2.101:0/3067994765 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2bfc1a06e0 0x7f2bfc18d130 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:19.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.263 7f2c045eb700 1 -- 172.21.2.101:0/3067994765 >> 172.21.2.101:0/3067994765 conn(0x7f2bfc1994b0 msgr2=0x7f2bfc190680 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:19.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.263 7f2c045eb700 1 -- 172.21.2.101:0/3067994765 shutdown_connections 2021-02-10T11:14:19.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.263 7f2c045eb700 1 -- 172.21.2.101:0/3067994765 wait complete. 2021-02-10T11:14:19.275 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:14:19.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.684 7fb1fea04700 1 Processor -- start 2021-02-10T11:14:19.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.684 7fb1fea04700 1 -- start start 2021-02-10T11:14:19.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.684 7fb1fea04700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1f81b0470 0x7fb1f81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:19.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.684 7fb1fea04700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb1f80f20e0 0x7fb1f80f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:19.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.684 7fb1fea04700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f80f0b50 con 0x7fb1f80ef610 2021-02-10T11:14:19.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.684 7fb1fea04700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f80f0820 con 0x7fb1f81b0470 2021-02-10T11:14:19.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.684 7fb1fea04700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f80f24e0 con 0x7fb1f80f20e0 2021-02-10T11:14:19.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.685 7fb1fc820700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1f81b0470 0x7fb1f81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:19.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.685 7fb1fc820700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1f81b0470 0x7fb1f81b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45028/0 (socket says 172.21.2.101:45028) 2021-02-10T11:14:19.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.685 7fb1fc820700 1 -- 172.21.2.101:0/227022092 learned_addr learned my addr 172.21.2.101:0/227022092 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:19.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.685 7fb1fc820700 1 -- 172.21.2.101:0/227022092 >> v2:172.21.2.108:3300/0 conn(0x7fb1f80f20e0 msgr2=0x7fb1f80f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:19.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.685 7fb1fc820700 1 --2- 172.21.2.101:0/227022092 >> v2:172.21.2.108:3300/0 conn(0x7fb1f80f20e0 0x7fb1f80f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:19.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.685 7fb1fc820700 1 -- 172.21.2.101:0/227022092 >> v1:172.21.2.108:6789/0 conn(0x7fb1f80ef610 legacy=0x7fb1f80efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:19.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.685 7fb1fc820700 1 -- 172.21.2.101:0/227022092 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f80e52e0 con 0x7fb1f81b0470 2021-02-10T11:14:19.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.685 7fb1fc820700 1 --2- 172.21.2.101:0/227022092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1f81b0470 0x7fb1f81b0870 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fb1e0006700 tx=0x7fb1e0009b80).ready entity=mon.0 client_cookie=6a4c64f251beb93b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:19.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.685 7fb1f77fe700 1 -- 172.21.2.101:0/227022092 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1e000c0c0 con 0x7fb1f81b0470 2021-02-10T11:14:19.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.685 7fb1f77fe700 1 -- 172.21.2.101:0/227022092 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e000c440 con 0x7fb1f81b0470 2021-02-10T11:14:19.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.685 7fb1f77fe700 1 -- 172.21.2.101:0/227022092 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1e000c8a0 con 0x7fb1f81b0470 2021-02-10T11:14:19.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.685 7fb1fea04700 1 -- 172.21.2.101:0/227022092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1f81b0470 msgr2=0x7fb1f81b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:19.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.685 7fb1fea04700 1 --2- 172.21.2.101:0/227022092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1f81b0470 0x7fb1f81b0870 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fb1e0006700 tx=0x7fb1e0009b80).stop 2021-02-10T11:14:19.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.686 7fb1fea04700 1 -- 172.21.2.101:0/227022092 shutdown_connections 2021-02-10T11:14:19.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.686 7fb1fea04700 1 --2- 172.21.2.101:0/227022092 >> v2:172.21.2.108:3300/0 conn(0x7fb1f80f20e0 0x7fb1f80f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:19.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.686 7fb1fea04700 1 --2- 172.21.2.101:0/227022092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1f81b0470 0x7fb1f81b0870 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:19.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.686 7fb1fea04700 1 -- 172.21.2.101:0/227022092 >> 172.21.2.101:0/227022092 conn(0x7fb1f80ce3e0 msgr2=0x7fb1f80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:19.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.686 7fb1fea04700 1 -- 172.21.2.101:0/227022092 shutdown_connections 2021-02-10T11:14:19.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.686 7fb1fea04700 1 -- 172.21.2.101:0/227022092 wait complete. 2021-02-10T11:14:19.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.701 7fb1fea04700 1 Processor -- start 2021-02-10T11:14:19.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.701 7fb1fea04700 1 -- start start 2021-02-10T11:14:19.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.701 7fb1fea04700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1f80c36a0 0x7fb1f80c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:19.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.701 7fb1fea04700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1f80c3190 0x7fb1f80b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:19.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.701 7fb1fea04700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1f80b2370 0x7fb1f80b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:19.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.701 7fb1fea04700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f80b27c0 con 0x7fb1f80b2370 2021-02-10T11:14:19.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.701 7fb1fea04700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f80b3360 con 0x7fb1f80c36a0 2021-02-10T11:14:19.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.701 7fb1fea04700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f80b3540 con 0x7fb1f80c3190 2021-02-10T11:14:19.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.701 7fb1fd021700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1f80b2370 0x7fb1f80b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:19.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.701 7fb1fd021700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1f80b2370 0x7fb1f80b2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45032/0 (socket says 172.21.2.101:45032) 2021-02-10T11:14:19.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.701 7fb1fd021700 1 -- 172.21.2.101:0/2990534126 learned_addr learned my addr 172.21.2.101:0/2990534126 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:19.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.702 7fb1fd021700 1 -- 172.21.2.101:0/2990534126 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1f80c3190 msgr2=0x7fb1f80b1d50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:19.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.702 7fb1fd021700 1 --2- 172.21.2.101:0/2990534126 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1f80c3190 0x7fb1f80b1d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:19.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.702 7fb1fd021700 1 -- 172.21.2.101:0/2990534126 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1f80c36a0 msgr2=0x7fb1f80c2b70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:19.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.702 7fb1fd021700 1 --2- 172.21.2.101:0/2990534126 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1f80c36a0 0x7fb1f80c2b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:19.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.702 7fb1fd021700 1 -- 172.21.2.101:0/2990534126 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f80b3760 con 0x7fb1f80b2370 2021-02-10T11:14:19.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.702 7fb1fd021700 1 --2- 172.21.2.101:0/2990534126 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1f80b2370 0x7fb1f80b2d40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fb1f0009980 tx=0x7fb1f000bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:19.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.702 7fb1f57fa700 1 -- 172.21.2.101:0/2990534126 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1f0005870 con 0x7fb1f80b2370 2021-02-10T11:14:19.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.702 7fb1f57fa700 1 -- 172.21.2.101:0/2990534126 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f0005b30 con 0x7fb1f80b2370 2021-02-10T11:14:19.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.702 7fb1fea04700 1 -- 172.21.2.101:0/2990534126 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f80b3760 con 0x7fb1f80b2370 2021-02-10T11:14:19.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.702 7fb1fea04700 1 -- 172.21.2.101:0/2990534126 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f80b3cf0 con 0x7fb1f80b2370 2021-02-10T11:14:19.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.702 7fb1f57fa700 1 -- 172.21.2.101:0/2990534126 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1f0006030 con 0x7fb1f80b2370 2021-02-10T11:14:19.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.702 7fb1f57fa700 1 -- 172.21.2.101:0/2990534126 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb1f0005d10 con 0x7fb1f80b2370 2021-02-10T11:14:19.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.702 7fb1f57fa700 1 --2- 172.21.2.101:0/2990534126 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1dc0186c0 0x7fb1dc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:19.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.703 7fb1f57fa700 1 -- 172.21.2.101:0/2990534126 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb1f0032890 con 0x7fb1f80b2370 2021-02-10T11:14:19.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.703 7fb1fc820700 1 --2- 172.21.2.101:0/2990534126 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1dc0186c0 0x7fb1dc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:19.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.703 7fb1fc820700 1 --2- 172.21.2.101:0/2990534126 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1dc0186c0 0x7fb1dc01abc0 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:19.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.734 7fb1fea04700 1 -- 172.21.2.101:0/2990534126 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f80edd10 con 0x7fb1f80b2370 2021-02-10T11:14:19.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.737 7fb1f57fa700 1 -- 172.21.2.101:0/2990534126 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb1f00062f0 con 0x7fb1f80b2370 2021-02-10T11:14:19.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.887 7fb1fea04700 1 -- 172.21.2.101:0/2990534126 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb1f80e52e0 con 0x7fb1f80b2370 2021-02-10T11:14:19.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.891 7fb1f57fa700 1 -- 172.21.2.101:0/2990534126 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb1f0006f20 con 0x7fb1f80b2370 2021-02-10T11:14:19.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.896 7fb1fea04700 1 -- 172.21.2.101:0/2990534126 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1dc0186c0 msgr2=0x7fb1dc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:19.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.896 7fb1fea04700 1 --2- 172.21.2.101:0/2990534126 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1dc0186c0 0x7fb1dc01abc0 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:19.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.896 7fb1fea04700 1 -- 172.21.2.101:0/2990534126 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1f80b2370 msgr2=0x7fb1f80b2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:19.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.896 7fb1fea04700 1 --2- 172.21.2.101:0/2990534126 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1f80b2370 0x7fb1f80b2d40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fb1f0009980 tx=0x7fb1f000bcd0).stop 2021-02-10T11:14:19.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.896 7fb1fea04700 1 -- 172.21.2.101:0/2990534126 shutdown_connections 2021-02-10T11:14:19.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.896 7fb1fea04700 1 --2- 172.21.2.101:0/2990534126 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1dc0186c0 0x7fb1dc01abc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:19.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.896 7fb1fea04700 1 --2- 172.21.2.101:0/2990534126 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1f80c3190 0x7fb1f80b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:19.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.896 7fb1fea04700 1 --2- 172.21.2.101:0/2990534126 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1f80b2370 0x7fb1f80b2d40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:19.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.896 7fb1fea04700 1 --2- 172.21.2.101:0/2990534126 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1f80c36a0 0x7fb1f80c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:19.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.896 7fb1fea04700 1 -- 172.21.2.101:0/2990534126 >> 172.21.2.101:0/2990534126 conn(0x7fb1f809f630 msgr2=0x7fb1f80a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:19.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.896 7fb1fea04700 1 -- 172.21.2.101:0/2990534126 shutdown_connections 2021-02-10T11:14:19.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:19.896 7fb1fea04700 1 -- 172.21.2.101:0/2990534126 wait complete. 2021-02-10T11:14:19.908 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:14:19.909 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:14:20.283 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:14:20.641 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:14:20.641 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:20.641 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:14:20.641 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:14:20.975 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:14:20.976 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:14:20.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:20.976 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:14:20.976 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:14:21.309 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.0’: Transport endpoint is not connected 2021-02-10T11:14:21.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:14:21.310 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2021-02-10T11:14:21.310 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:14:21.310 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:14:21.310 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:14:21.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:21.677 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:14:21.677 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:14:22.011 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:14:22.012 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:14:22.012 INFO:teuthology.orchestra.run:waiting for 900 2021-02-10T11:14:22.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:14:22.012 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:14:22.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:22.359 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:14:22.359 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:14:22.694 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:14:22.694 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:14:22.694 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:22.695 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:14:22.695 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:14:23.029 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: Transport endpoint is not connected 2021-02-10T11:14:23.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:14:23.030 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.1 2021-02-10T11:14:23.030 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:14:23.030 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:14:23.030 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:14:23.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:23.388 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:14:23.388 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:14:23.721 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:14:23.722 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:14:23.722 INFO:teuthology.orchestra.run:waiting for 900 2021-02-10T11:14:23.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:14:23.722 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:14:24.058 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:24.058 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:14:24.058 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:14:24.394 INFO:teuthology.orchestra.run.gibba008.stdout:fuseblk 2021-02-10T11:14:26.596 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2021-02-10T11:14:26.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:26.596 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:14:26.596 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:14:26.932 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: Transport endpoint is not connected 2021-02-10T11:14:26.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:14:26.933 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.2 2021-02-10T11:14:26.933 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba008.front.sepia.ceph.com... 2021-02-10T11:14:26.933 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:14:26.933 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:14:27.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:29.820 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:14:29.820 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:14:30.161 INFO:teuthology.orchestra.run.gibba008.stdout:ext2/ext3 2021-02-10T11:14:30.161 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:14:30.161 INFO:teuthology.orchestra.run:waiting for 900 2021-02-10T11:14:30.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:14:30.162 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:14:30.517 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:30.517 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:14:30.518 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:14:30.853 INFO:teuthology.orchestra.run.gibba008.stdout:fuseblk 2021-02-10T11:14:30.854 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2021-02-10T11:14:30.854 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:30.854 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:14:30.854 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:14:31.189 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: Transport endpoint is not connected 2021-02-10T11:14:31.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:14:31.189 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.3 2021-02-10T11:14:31.190 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba008.front.sepia.ceph.com... 2021-02-10T11:14:31.191 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:14:31.191 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:14:31.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:14:31.547 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:14:31.547 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:14:31.881 INFO:teuthology.orchestra.run.gibba008.stdout:ext2/ext3 2021-02-10T11:14:31.881 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:14:31.881 INFO:teuthology.orchestra.run:waiting for 900 2021-02-10T11:14:31.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:14:31.882 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:14:32.216 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json-pretty 2021-02-10T11:14:32.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.623 7f54f1386700 1 Processor -- start 2021-02-10T11:14:32.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.623 7f54f1386700 1 -- start start 2021-02-10T11:14:32.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.624 7f54f1386700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54ec0e5070 0x7f54ec1b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:32.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.624 7f54f1386700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54ec07ed80 con 0x7f54ec1b3e40 2021-02-10T11:14:32.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.624 7f54f1386700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54ec07e670 con 0x7f54ec0e5470 2021-02-10T11:14:32.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.624 7f54f1386700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec07e940 con 0x7f54ec0e5070 2021-02-10T11:14:32.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.624 7f54eb87e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f54ec1b3e40 0x7f54ec07de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45536/0 (socket says 172.21.2.101:45536) 2021-02-10T11:14:32.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.624 7f54eb87e700 1 -- 172.21.2.101:0/2149169851 learned_addr learned my addr 172.21.2.101:0/2149169851 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:14:32.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.624 7f54eb07d700 1 --2- 172.21.2.101:0/2149169851 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54ec0e5070 0x7f54ec1b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:32.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.624 7f54ea07b700 1 -- 172.21.2.101:0/2149169851 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1002361807 0 0) 0x7f54e4001500 con 0x7f54ec1b3e40 2021-02-10T11:14:32.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.624 7f54ea07b700 1 -- 172.21.2.101:0/2149169851 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54d8002fe0 con 0x7f54ec1b3e40 2021-02-10T11:14:32.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54ea07b700 1 -- 172.21.2.101:0/2149169851 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1765764530 0 0) 0x7f54e4001460 con 0x7f54ec1b3e40 2021-02-10T11:14:32.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54ea07b700 1 -- 172.21.2.101:0/2149169851 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54ec0e5070 msgr2=0x7f54ec1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:32.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54ea07b700 1 --2- 172.21.2.101:0/2149169851 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54ec0e5070 0x7f54ec1b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:32.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54ea07b700 1 -- 172.21.2.101:0/2149169851 >> v1:172.21.2.108:6789/0 conn(0x7f54ec0e5470 legacy=0x7f54ec07d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:32.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54ea07b700 1 -- 172.21.2.101:0/2149169851 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec07eb30 con 0x7f54ec1b3e40 2021-02-10T11:14:32.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54ea07b700 1 -- 172.21.2.101:0/2149169851 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f54e4001980 con 0x7f54ec1b3e40 2021-02-10T11:14:32.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54ea07b700 1 -- 172.21.2.101:0/2149169851 >> v1:172.21.2.101:6789/0 conn(0x7f54ec1b3e40 legacy=0x7f54ec07de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:32.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54ea07b700 1 --2- 172.21.2.101:0/2149169851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54d8003a70 0x7f54d8005f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:32.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54ea07b700 1 --2- 172.21.2.101:0/2149169851 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54d8006790 0x7f54d8008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:32.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54ea07b700 1 --2- 172.21.2.101:0/2149169851 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54d8009500 0x7f54d800ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:32.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54ea07b700 1 -- 172.21.2.101:0/2149169851 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d800c2d0 con 0x7f54d8003a70 2021-02-10T11:14:32.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54ea07b700 1 -- 172.21.2.101:0/2149169851 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d800c4f0 con 0x7f54d8009500 2021-02-10T11:14:32.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54ea07b700 1 -- 172.21.2.101:0/2149169851 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d8050720 con 0x7f54d8006790 2021-02-10T11:14:32.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54eb07d700 1 --2- 172.21.2.101:0/2149169851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54d8003a70 0x7f54d8005f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:32.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54ea87c700 1 --2- 172.21.2.101:0/2149169851 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54d8006790 0x7f54d8008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:32.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54eb87e700 1 --2- 172.21.2.101:0/2149169851 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54d8009500 0x7f54d800ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:32.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54eb07d700 1 -- 172.21.2.101:0/2149169851 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54d8006790 msgr2=0x7f54d8008c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:32.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54eb07d700 1 --2- 172.21.2.101:0/2149169851 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54d8006790 0x7f54d8008c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:32.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54eb07d700 1 -- 172.21.2.101:0/2149169851 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54d8009500 msgr2=0x7f54d800ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:32.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54eb07d700 1 --2- 172.21.2.101:0/2149169851 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54d8009500 0x7f54d800ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:32.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54eb07d700 1 -- 172.21.2.101:0/2149169851 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d8050980 con 0x7f54d8003a70 2021-02-10T11:14:32.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54eb87e700 1 --2- 172.21.2.101:0/2149169851 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54d8009500 0x7f54d800ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:14:32.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.625 7f54eb07d700 1 --2- 172.21.2.101:0/2149169851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54d8003a70 0x7f54d8005f70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f54d4006ac0 tx=0x7f54d4009a30).ready entity=mon.0 client_cookie=4160dc13769ea026 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:32.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.626 7f54ea07b700 1 -- 172.21.2.101:0/2149169851 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f54d400c0c0 con 0x7f54d8003a70 2021-02-10T11:14:32.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.626 7f54ea07b700 1 -- 172.21.2.101:0/2149169851 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d4009e20 con 0x7f54d8003a70 2021-02-10T11:14:32.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.626 7f54ea07b700 1 -- 172.21.2.101:0/2149169851 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f54d400c6c0 con 0x7f54d8003a70 2021-02-10T11:14:32.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.626 7f54f1386700 1 -- 172.21.2.101:0/2149169851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54d8003a70 msgr2=0x7f54d8005f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:32.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.626 7f54f1386700 1 --2- 172.21.2.101:0/2149169851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54d8003a70 0x7f54d8005f70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f54d4006ac0 tx=0x7f54d4009a30).stop 2021-02-10T11:14:32.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.626 7f54eb87e700 1 -- 172.21.2.101:0/2149169851 reap_dead start 2021-02-10T11:14:32.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.626 7f54f1386700 1 -- 172.21.2.101:0/2149169851 shutdown_connections 2021-02-10T11:14:32.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.626 7f54f1386700 1 -- 172.21.2.101:0/2149169851 >> 172.21.2.101:0/2149169851 conn(0x7f54ec0ce690 msgr2=0x7f54ec0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:32.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.626 7f54f1386700 1 -- 172.21.2.101:0/2149169851 shutdown_connections 2021-02-10T11:14:32.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.626 7f54f1386700 1 -- 172.21.2.101:0/2149169851 wait complete. 2021-02-10T11:14:32.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.641 7f54f1386700 1 Processor -- start 2021-02-10T11:14:32.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.641 7f54f1386700 1 -- start start 2021-02-10T11:14:32.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.641 7f54f1386700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54ec19acd0 0x7f54ec0a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:32.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.641 7f54f1386700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54ec0a09f0 0x7f54ec0b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:32.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.641 7f54f1386700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54ec0b40e0 0x7f54ec0b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:32.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.641 7f54f1386700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec0a1240 con 0x7f54ec0b40e0 2021-02-10T11:14:32.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.641 7f54f1386700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec19b1e0 con 0x7f54ec19acd0 2021-02-10T11:14:32.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.641 7f54f1386700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec0a0df0 con 0x7f54ec0a09f0 2021-02-10T11:14:32.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.642 7f54eb87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54ec0b40e0 0x7f54ec0b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:32.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.642 7f54eb87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54ec0b40e0 0x7f54ec0b44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45052/0 (socket says 172.21.2.101:45052) 2021-02-10T11:14:32.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.642 7f54eb87e700 1 -- 172.21.2.101:0/917983338 learned_addr learned my addr 172.21.2.101:0/917983338 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:32.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.642 7f54eb07d700 1 --2- 172.21.2.101:0/917983338 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54ec19acd0 0x7f54ec0a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:32.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.642 7f54ea87c700 1 --2- 172.21.2.101:0/917983338 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54ec0a09f0 0x7f54ec0b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:32.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.642 7f54eb87e700 1 -- 172.21.2.101:0/917983338 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54ec0a09f0 msgr2=0x7f54ec0b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:32.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.642 7f54eb87e700 1 --2- 172.21.2.101:0/917983338 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54ec0a09f0 0x7f54ec0b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:32.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.642 7f54eb87e700 1 -- 172.21.2.101:0/917983338 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54ec19acd0 msgr2=0x7f54ec0a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:32.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.642 7f54eb87e700 1 --2- 172.21.2.101:0/917983338 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54ec19acd0 0x7f54ec0a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:32.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.642 7f54eb87e700 1 -- 172.21.2.101:0/917983338 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec0a1740 con 0x7f54ec0b40e0 2021-02-10T11:14:32.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.642 7f54eb07d700 1 --2- 172.21.2.101:0/917983338 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54ec19acd0 0x7f54ec0a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:14:32.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.642 7f54eb87e700 1 --2- 172.21.2.101:0/917983338 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54ec0b40e0 0x7f54ec0b44e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f54e4001c30 tx=0x7f54e400ac60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:32.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.642 7f54e3fff700 1 -- 172.21.2.101:0/917983338 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f54e4004930 con 0x7f54ec0b40e0 2021-02-10T11:14:32.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.642 7f54e3fff700 1 -- 172.21.2.101:0/917983338 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e4004bf0 con 0x7f54ec0b40e0 2021-02-10T11:14:32.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.642 7f54e3fff700 1 -- 172.21.2.101:0/917983338 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f54e40072a0 con 0x7f54ec0b40e0 2021-02-10T11:14:32.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.642 7f54f1386700 1 -- 172.21.2.101:0/917983338 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ec0a1740 con 0x7f54ec0b40e0 2021-02-10T11:14:32.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.642 7f54f1386700 1 -- 172.21.2.101:0/917983338 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ec0a1740 con 0x7f54ec0b40e0 2021-02-10T11:14:32.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.643 7f54e3fff700 1 -- 172.21.2.101:0/917983338 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f54e4004720 con 0x7f54ec0b40e0 2021-02-10T11:14:32.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.643 7f54e3fff700 1 --2- 172.21.2.101:0/917983338 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54d0018730 0x7f54d001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:32.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.643 7f54e3fff700 1 -- 172.21.2.101:0/917983338 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f54e4032960 con 0x7f54ec0b40e0 2021-02-10T11:14:32.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.643 7f54eb07d700 1 --2- 172.21.2.101:0/917983338 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54d0018730 0x7f54d001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:32.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.643 7f54eb07d700 1 --2- 172.21.2.101:0/917983338 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54d0018730 0x7f54d001abc0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:32.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.674 7f54f1386700 1 -- 172.21.2.101:0/917983338 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ec07ed80 con 0x7f54ec0b40e0 2021-02-10T11:14:32.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.676 7f54e3fff700 1 -- 172.21.2.101:0/917983338 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f54e4004bf0 con 0x7f54ec0b40e0 2021-02-10T11:14:32.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.800 7f54f1386700 1 -- 172.21.2.101:0/917983338 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f54ec07ed80 con 0x7f54ec0b40e0 2021-02-10T11:14:32.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.801 7f54e3fff700 1 -- 172.21.2.101:0/917983338 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v20) v1 ==== 81+0+30360 (secure 0 0 0) 0x7f54e4004e00 con 0x7f54ec0b40e0 2021-02-10T11:14:32.805 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:14:32.805 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:14:32.805 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 20, 2021-02-10T11:14:32.805 INFO:teuthology.orchestra.run.gibba001.stdout: "fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:14:32.805 INFO:teuthology.orchestra.run.gibba001.stdout: "created": "2021-02-10 11:12:36.788366", 2021-02-10T11:14:32.805 INFO:teuthology.orchestra.run.gibba001.stdout: "modified": "2021-02-10 11:13:15.149234", 2021-02-10T11:14:32.806 INFO:teuthology.orchestra.run.gibba001.stdout: "last_up_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:14:32.806 INFO:teuthology.orchestra.run.gibba001.stdout: "last_in_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:14:32.806 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-02-10T11:14:32.806 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_num": 5799936, 2021-02-10T11:14:32.806 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_set": [ 2021-02-10T11:14:32.806 INFO:teuthology.orchestra.run.gibba001.stdout: "pglog_hardlimit", 2021-02-10T11:14:32.806 INFO:teuthology.orchestra.run.gibba001.stdout: "purged_snapdirs", 2021-02-10T11:14:32.806 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_deletes", 2021-02-10T11:14:32.806 INFO:teuthology.orchestra.run.gibba001.stdout: "sortbitwise" 2021-02-10T11:14:32.806 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:14:32.806 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_version": 4, 2021-02-10T11:14:32.806 INFO:teuthology.orchestra.run.gibba001.stdout: "full_ratio": 0.94999998807907104, 2021-02-10T11:14:32.807 INFO:teuthology.orchestra.run.gibba001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-02-10T11:14:32.807 INFO:teuthology.orchestra.run.gibba001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-02-10T11:14:32.807 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_snapshot": "", 2021-02-10T11:14:32.807 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_max": 3, 2021-02-10T11:14:32.807 INFO:teuthology.orchestra.run.gibba001.stdout: "max_osd": 8, 2021-02-10T11:14:32.807 INFO:teuthology.orchestra.run.gibba001.stdout: "require_min_compat_client": "jewel", 2021-02-10T11:14:32.807 INFO:teuthology.orchestra.run.gibba001.stdout: "min_compat_client": "jewel", 2021-02-10T11:14:32.807 INFO:teuthology.orchestra.run.gibba001.stdout: "require_osd_release": "nautilus", 2021-02-10T11:14:32.807 INFO:teuthology.orchestra.run.gibba001.stdout: "pools": [ 2021-02-10T11:14:32.807 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.807 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 1, 2021-02-10T11:14:32.807 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "rbd", 2021-02-10T11:14:32.807 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:13:08.037472", 2021-02-10T11:14:32.807 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:14:32.808 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:14:32.808 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:14:32.808 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:14:32.808 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:14:32.808 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:14:32.808 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:14:32.808 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:14:32.808 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:14:32.808 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:14:32.808 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:14:32.808 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:14:32.808 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:14:32.808 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:14:32.809 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:14:32.809 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:14:32.809 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:14:32.809 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:14:32.809 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:14:32.809 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:14:32.809 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.809 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "16", 2021-02-10T11:14:32.809 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:14:32.809 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:14:32.809 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:14:32.809 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:14:32.809 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:14:32.809 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:14:32.810 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:14:32.810 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:14:32.810 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:14:32.810 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:14:32.810 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:14:32.810 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:14:32.810 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:14:32.810 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:14:32.810 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:14:32.810 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:14:32.810 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:14:32.810 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:14:32.810 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:14:32.811 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:14:32.811 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:14:32.811 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:14:32.811 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:14:32.811 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:14:32.811 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:14:32.811 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:14:32.811 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.811 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:14:32.811 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:14:32.811 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:14:32.811 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:14:32.811 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:14:32.812 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:14:32.812 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:14:32.812 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:14:32.812 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:14:32.812 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:14:32.812 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:14:32.812 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:14:32.812 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:14:32.812 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd": {} 2021-02-10T11:14:32.812 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.812 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.812 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.812 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 2, 2021-02-10T11:14:32.812 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_metadata", 2021-02-10T11:14:32.813 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:13:11.922272", 2021-02-10T11:14:32.813 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:14:32.813 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:14:32.813 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:14:32.813 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:14:32.813 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:14:32.813 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:14:32.813 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:14:32.813 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:14:32.813 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:14:32.813 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:14:32.813 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:14:32.813 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:14:32.814 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:14:32.814 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:14:32.814 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:14:32.814 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:14:32.814 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:14:32.814 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:14:32.814 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:14:32.814 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:14:32.814 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.814 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "19", 2021-02-10T11:14:32.814 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:14:32.814 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:14:32.814 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:14:32.815 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:14:32.815 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:14:32.815 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:14:32.815 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:14:32.815 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:14:32.815 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:14:32.815 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:14:32.815 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:14:32.815 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:14:32.815 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:14:32.815 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:14:32.815 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:14:32.815 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:14:32.816 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:14:32.816 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:14:32.816 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:14:32.816 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:14:32.816 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:14:32.816 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:14:32.816 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:14:32.816 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:14:32.816 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:14:32.816 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:14:32.816 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.816 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:14:32.816 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:14:32.816 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:14:32.817 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:14:32.817 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:14:32.817 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:14:32.817 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:14:32.817 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:14:32.817 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:14:32.817 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:14:32.817 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:14:32.817 INFO:teuthology.orchestra.run.gibba001.stdout: "options": { 2021-02-10T11:14:32.817 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_bias": 4, 2021-02-10T11:14:32.817 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_min": 16, 2021-02-10T11:14:32.817 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_priority": 5 2021-02-10T11:14:32.817 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.817 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:14:32.818 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:14:32.818 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": "cephfs" 2021-02-10T11:14:32.818 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.818 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.818 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.818 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.818 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 3, 2021-02-10T11:14:32.818 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_data", 2021-02-10T11:14:32.818 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:13:13.141733", 2021-02-10T11:14:32.818 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:14:32.818 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:14:32.818 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:14:32.818 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:14:32.819 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:14:32.819 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:14:32.819 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:14:32.819 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:14:32.819 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:14:32.819 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:14:32.819 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:14:32.819 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:14:32.819 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:14:32.819 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:14:32.819 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:14:32.820 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:14:32.820 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:14:32.820 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:14:32.820 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:14:32.820 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:14:32.820 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.820 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "20", 2021-02-10T11:14:32.820 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:14:32.820 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:14:32.820 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:14:32.820 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:14:32.820 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:14:32.820 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:14:32.820 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:14:32.821 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:14:32.821 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:14:32.821 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:14:32.821 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:14:32.821 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:14:32.821 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:14:32.821 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:14:32.821 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:14:32.821 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:14:32.821 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:14:32.821 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:14:32.821 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:14:32.821 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:14:32.822 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:14:32.822 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:14:32.822 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:14:32.822 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:14:32.822 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:14:32.822 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:14:32.822 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.822 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:14:32.822 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:14:32.822 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:14:32.822 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:14:32.822 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:14:32.822 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:14:32.822 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:14:32.823 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:14:32.823 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:14:32.823 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:14:32.823 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:14:32.823 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:14:32.823 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:14:32.823 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:14:32.823 INFO:teuthology.orchestra.run.gibba001.stdout: "data": "cephfs" 2021-02-10T11:14:32.823 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.823 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.823 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.823 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:14:32.823 INFO:teuthology.orchestra.run.gibba001.stdout: "osds": [ 2021-02-10T11:14:32.824 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.824 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:14:32.824 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5", 2021-02-10T11:14:32.824 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:14:32.824 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:14:32.824 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:14:32.824 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:14:32.824 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:14:32.824 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:14:32.824 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:14:32.824 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 18, 2021-02-10T11:14:32.824 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:14:32.824 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:14:32.824 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:14:32.825 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.825 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.825 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.825 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6802", 2021-02-10T11:14:32.825 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:14:32.825 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.825 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.825 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.825 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6805", 2021-02-10T11:14:32.825 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:14:32.825 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.825 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.825 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.826 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:14:32.826 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.826 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.826 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.826 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6807", 2021-02-10T11:14:32.826 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:14:32.826 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.826 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.826 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.826 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6809", 2021-02-10T11:14:32.826 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:14:32.826 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.826 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.827 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.827 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:14:32.827 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.827 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.827 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.827 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6817", 2021-02-10T11:14:32.827 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:14:32.827 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.827 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.827 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.827 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6820", 2021-02-10T11:14:32.828 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:14:32.828 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.828 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.828 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.828 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:14:32.828 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.828 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.828 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.828 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6812", 2021-02-10T11:14:32.828 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:14:32.828 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.828 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.828 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.829 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6814", 2021-02-10T11:14:32.829 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:14:32.829 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.829 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.829 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.829 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6805/25093", 2021-02-10T11:14:32.829 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6809/25093", 2021-02-10T11:14:32.829 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6820/25093", 2021-02-10T11:14:32.829 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6814/25093", 2021-02-10T11:14:32.829 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:14:32.829 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:14:32.829 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:14:32.830 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.830 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.830 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.830 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:14:32.830 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "ddc04ce3-b313-4707-8270-585391e38ecc", 2021-02-10T11:14:32.830 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:14:32.830 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:14:32.830 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:14:32.830 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:14:32.830 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:14:32.830 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:14:32.830 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:14:32.831 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 18, 2021-02-10T11:14:32.831 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:14:32.831 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:14:32.831 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:14:32.831 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.831 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.831 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.831 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6803", 2021-02-10T11:14:32.831 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:14:32.831 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.831 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.831 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.832 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6804", 2021-02-10T11:14:32.832 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:14:32.832 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.832 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.832 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.832 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:14:32.832 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.832 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.832 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.832 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6806", 2021-02-10T11:14:32.832 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:14:32.832 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.833 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.833 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.833 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6808", 2021-02-10T11:14:32.833 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:14:32.833 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.833 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.833 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.833 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:14:32.833 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.833 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.834 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.834 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6818", 2021-02-10T11:14:32.834 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:14:32.834 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.834 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.834 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.834 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6821", 2021-02-10T11:14:32.834 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:14:32.834 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.834 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.834 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.835 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:14:32.835 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.835 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.835 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.835 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6810", 2021-02-10T11:14:32.835 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:14:32.835 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.835 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.836 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.836 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6815", 2021-02-10T11:14:32.836 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:14:32.836 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.836 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.836 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.836 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6804/25092", 2021-02-10T11:14:32.836 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6808/25092", 2021-02-10T11:14:32.837 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6821/25092", 2021-02-10T11:14:32.837 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6815/25092", 2021-02-10T11:14:32.837 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:14:32.837 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:14:32.837 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:14:32.837 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.837 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.837 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.837 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:14:32.837 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3", 2021-02-10T11:14:32.837 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:14:32.837 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:14:32.837 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:14:32.838 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:14:32.838 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:14:32.838 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:14:32.838 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:14:32.838 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 17, 2021-02-10T11:14:32.838 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:14:32.838 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:14:32.838 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:14:32.838 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.838 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.839 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.839 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6811", 2021-02-10T11:14:32.839 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:14:32.839 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.839 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.839 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.839 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6813", 2021-02-10T11:14:32.839 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:14:32.839 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.839 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.839 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.839 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:14:32.839 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.839 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.840 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.840 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6816", 2021-02-10T11:14:32.840 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:14:32.840 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.840 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.840 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.840 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6819", 2021-02-10T11:14:32.840 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:14:32.840 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.840 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.840 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.840 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:14:32.840 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.841 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.841 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.841 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6825", 2021-02-10T11:14:32.841 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:14:32.841 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.841 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.841 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.841 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6827", 2021-02-10T11:14:32.841 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:14:32.841 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.841 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.841 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.841 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:14:32.842 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.842 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.842 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.842 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6822", 2021-02-10T11:14:32.842 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:14:32.842 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.842 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.842 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.842 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6823", 2021-02-10T11:14:32.842 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:14:32.842 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.843 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.844 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.844 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6813/25094", 2021-02-10T11:14:32.844 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6819/25094", 2021-02-10T11:14:32.844 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6827/25094", 2021-02-10T11:14:32.844 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6823/25094", 2021-02-10T11:14:32.844 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:14:32.844 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:14:32.844 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:14:32.845 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.845 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.845 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.845 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:14:32.845 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "0d9ef9b1-98aa-4d37-aa16-6305c7d46d52", 2021-02-10T11:14:32.845 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:14:32.845 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:14:32.845 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:14:32.845 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:14:32.845 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:14:32.846 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:14:32.846 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:14:32.846 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 18, 2021-02-10T11:14:32.846 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:14:32.846 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:14:32.846 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:14:32.846 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.846 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.846 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.846 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6824", 2021-02-10T11:14:32.846 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:14:32.847 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.847 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.847 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.847 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6826", 2021-02-10T11:14:32.847 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:14:32.847 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.847 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.847 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.847 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:14:32.847 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.847 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.847 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.847 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6828", 2021-02-10T11:14:32.848 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:14:32.848 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.848 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.848 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.848 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6829", 2021-02-10T11:14:32.848 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:14:32.848 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.848 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.848 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.848 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:14:32.848 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.848 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.848 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.849 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6832", 2021-02-10T11:14:32.849 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:14:32.849 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.849 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.849 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.849 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6833", 2021-02-10T11:14:32.849 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:14:32.849 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.849 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.849 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.849 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:14:32.849 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.849 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.850 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.850 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6830", 2021-02-10T11:14:32.850 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:14:32.850 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.850 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.850 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.850 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6831", 2021-02-10T11:14:32.850 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:14:32.850 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.850 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.850 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.850 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6826/25095", 2021-02-10T11:14:32.850 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6829/25095", 2021-02-10T11:14:32.851 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6833/25095", 2021-02-10T11:14:32.851 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6831/25095", 2021-02-10T11:14:32.851 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:14:32.851 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:14:32.851 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:14:32.851 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.851 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.851 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.851 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:14:32.851 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "36742c21-bfeb-4e11-bce9-8e745378b605", 2021-02-10T11:14:32.851 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:14:32.851 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:14:32.851 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:14:32.852 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:14:32.852 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:14:32.852 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:14:32.852 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:14:32.852 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 18, 2021-02-10T11:14:32.852 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:14:32.852 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:14:32.852 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:14:32.852 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.852 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.852 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.852 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6816", 2021-02-10T11:14:32.852 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:14:32.852 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.853 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.853 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.853 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6818", 2021-02-10T11:14:32.853 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:14:32.853 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.853 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.853 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.853 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:14:32.853 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.853 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.853 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.853 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6820", 2021-02-10T11:14:32.853 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:14:32.854 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.854 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.854 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.854 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6822", 2021-02-10T11:14:32.854 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:14:32.854 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.854 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.854 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.854 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:14:32.854 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.854 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.854 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.855 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6828", 2021-02-10T11:14:32.855 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:14:32.855 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.855 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.855 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.855 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6830", 2021-02-10T11:14:32.855 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:14:32.855 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.855 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.855 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.855 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:14:32.855 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.855 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.855 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.856 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6824", 2021-02-10T11:14:32.856 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:14:32.856 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.856 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.856 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.856 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6826", 2021-02-10T11:14:32.856 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:14:32.856 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.856 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.856 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.856 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6818/36380", 2021-02-10T11:14:32.856 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6822/36380", 2021-02-10T11:14:32.856 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6830/36380", 2021-02-10T11:14:32.857 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6826/36380", 2021-02-10T11:14:32.857 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:14:32.857 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:14:32.857 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:14:32.857 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.857 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.857 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.857 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:14:32.857 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "88e694e8-fe98-4d1b-88c8-8c9907ece60b", 2021-02-10T11:14:32.857 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:14:32.857 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:14:32.857 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:14:32.858 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:14:32.858 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:14:32.858 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:14:32.858 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:14:32.858 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 18, 2021-02-10T11:14:32.858 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:14:32.858 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:14:32.858 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:14:32.858 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.858 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.858 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.858 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6808", 2021-02-10T11:14:32.858 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:14:32.859 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.859 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.859 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.859 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6809", 2021-02-10T11:14:32.859 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:14:32.859 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.859 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.859 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.859 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:14:32.859 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.859 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.859 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.859 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6810", 2021-02-10T11:14:32.860 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:14:32.860 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.860 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.860 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.860 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6811", 2021-02-10T11:14:32.860 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:14:32.860 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.860 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.860 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.860 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:14:32.860 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.860 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.860 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.860 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6814", 2021-02-10T11:14:32.861 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:14:32.861 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.861 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.861 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.861 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6815", 2021-02-10T11:14:32.861 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:14:32.861 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.861 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.861 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.861 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:14:32.861 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.861 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.861 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.862 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6812", 2021-02-10T11:14:32.862 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:14:32.862 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.862 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.862 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.862 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6813", 2021-02-10T11:14:32.862 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:14:32.862 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.862 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.862 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.862 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6809/36383", 2021-02-10T11:14:32.862 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6811/36383", 2021-02-10T11:14:32.862 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6815/36383", 2021-02-10T11:14:32.863 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6813/36383", 2021-02-10T11:14:32.863 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:14:32.863 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:14:32.863 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:14:32.863 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.863 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.863 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.863 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:14:32.863 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "87a13c6c-cf85-4aaf-916b-e73913484a49", 2021-02-10T11:14:32.863 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:14:32.863 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:14:32.863 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:14:32.863 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:14:32.864 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:14:32.864 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:14:32.864 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:14:32.864 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 17, 2021-02-10T11:14:32.864 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:14:32.864 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:14:32.864 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:14:32.864 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.864 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.864 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.864 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6800", 2021-02-10T11:14:32.864 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:14:32.864 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.865 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.865 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.865 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6801", 2021-02-10T11:14:32.865 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:14:32.865 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.865 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.865 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.865 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:14:32.865 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.865 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.865 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.865 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6802", 2021-02-10T11:14:32.865 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:14:32.866 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.866 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.866 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.866 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6803", 2021-02-10T11:14:32.866 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:14:32.866 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.866 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.866 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.866 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:14:32.866 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.866 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.866 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.866 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6806", 2021-02-10T11:14:32.867 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:14:32.867 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.867 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.867 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.867 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6807", 2021-02-10T11:14:32.867 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:14:32.867 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.867 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.867 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.867 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:14:32.867 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.867 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.868 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.868 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6804", 2021-02-10T11:14:32.868 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:14:32.868 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.868 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.868 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.868 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6805", 2021-02-10T11:14:32.868 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:14:32.868 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.868 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.868 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.868 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6801/36381", 2021-02-10T11:14:32.868 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6803/36381", 2021-02-10T11:14:32.868 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6807/36381", 2021-02-10T11:14:32.869 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6805/36381", 2021-02-10T11:14:32.869 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:14:32.869 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:14:32.869 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:14:32.869 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.869 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.869 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.869 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:14:32.869 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "7b90823f-854c-4592-a95b-a7fa98f93254", 2021-02-10T11:14:32.869 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:14:32.869 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:14:32.869 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:14:32.869 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:14:32.870 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:14:32.870 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:14:32.870 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:14:32.870 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 17, 2021-02-10T11:14:32.870 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:14:32.870 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:14:32.870 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:14:32.870 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.870 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.870 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.870 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6817", 2021-02-10T11:14:32.870 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:14:32.871 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.871 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.871 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.871 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6819", 2021-02-10T11:14:32.871 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:14:32.871 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.871 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.871 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.871 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:14:32.871 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.871 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.871 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.871 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6821", 2021-02-10T11:14:32.872 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:14:32.872 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.872 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.872 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.872 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6823", 2021-02-10T11:14:32.872 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:14:32.872 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.872 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.872 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.872 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:14:32.872 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.872 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.872 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.873 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6829", 2021-02-10T11:14:32.873 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:14:32.873 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.873 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.873 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.873 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6831", 2021-02-10T11:14:32.873 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:14:32.873 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.873 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.873 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.873 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:14:32.873 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:14:32.873 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.874 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:14:32.874 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6825", 2021-02-10T11:14:32.874 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:14:32.874 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.874 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.874 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:14:32.874 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6827", 2021-02-10T11:14:32.874 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:14:32.874 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.874 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.874 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.874 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6819/36382", 2021-02-10T11:14:32.874 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6823/36382", 2021-02-10T11:14:32.875 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6831/36382", 2021-02-10T11:14:32.875 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6827/36382", 2021-02-10T11:14:32.875 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:14:32.875 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:14:32.875 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:14:32.875 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:32.875 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.875 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:14:32.875 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_xinfo": [ 2021-02-10T11:14:32.875 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.875 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:14:32.875 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:14:32.876 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:14:32.876 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:14:32.876 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:14:32.876 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:14:32.876 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.876 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.876 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:14:32.876 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:14:32.876 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:14:32.876 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:14:32.876 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:14:32.876 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:14:32.876 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.877 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.877 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:14:32.877 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:14:32.877 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:14:32.877 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:14:32.877 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:14:32.877 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:14:32.877 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.877 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.877 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:14:32.877 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:14:32.878 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:14:32.878 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:14:32.878 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:14:32.878 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:14:32.878 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.878 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.878 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:14:32.878 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:14:32.878 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:14:32.878 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:14:32.878 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:14:32.878 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:14:32.879 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.879 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.879 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:14:32.879 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:14:32.879 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:14:32.879 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:14:32.879 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:14:32.879 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:14:32.879 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.879 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.879 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:14:32.879 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:14:32.879 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:14:32.879 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:14:32.880 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:14:32.880 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:14:32.880 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.880 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:32.880 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:14:32.880 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:14:32.880 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:14:32.880 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:14:32.880 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:14:32.880 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:14:32.880 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.880 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:14:32.880 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap": [], 2021-02-10T11:14:32.881 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap_items": [], 2021-02-10T11:14:32.881 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_temp": [], 2021-02-10T11:14:32.881 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_temp": [], 2021-02-10T11:14:32.881 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklist": {}, 2021-02-10T11:14:32.881 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profiles": { 2021-02-10T11:14:32.881 INFO:teuthology.orchestra.run.gibba001.stdout: "default": { 2021-02-10T11:14:32.881 INFO:teuthology.orchestra.run.gibba001.stdout: "crush-failure-domain": "osd", 2021-02-10T11:14:32.881 INFO:teuthology.orchestra.run.gibba001.stdout: "k": "2", 2021-02-10T11:14:32.881 INFO:teuthology.orchestra.run.gibba001.stdout: "m": "1", 2021-02-10T11:14:32.881 INFO:teuthology.orchestra.run.gibba001.stdout: "plugin": "jerasure", 2021-02-10T11:14:32.881 INFO:teuthology.orchestra.run.gibba001.stdout: "ruleset-failure-domain": "osd", 2021-02-10T11:14:32.881 INFO:teuthology.orchestra.run.gibba001.stdout: "technique": "reed_sol_van" 2021-02-10T11:14:32.881 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:32.882 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:32.882 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps_queue": [], 2021-02-10T11:14:32.882 INFO:teuthology.orchestra.run.gibba001.stdout: "new_removed_snaps": [], 2021-02-10T11:14:32.882 INFO:teuthology.orchestra.run.gibba001.stdout: "new_purged_snaps": [], 2021-02-10T11:14:32.882 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_node_flags": {}, 2021-02-10T11:14:32.882 INFO:teuthology.orchestra.run.gibba001.stdout: "device_class_flags": {} 2021-02-10T11:14:32.882 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:14:32.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.803 7f54f1386700 1 -- 172.21.2.101:0/917983338 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54d0018730 msgr2=0x7f54d001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:32.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.803 7f54f1386700 1 --2- 172.21.2.101:0/917983338 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54d0018730 0x7f54d001abc0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.803 7f54f1386700 1 -- 172.21.2.101:0/917983338 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54ec0b40e0 msgr2=0x7f54ec0b44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.803 7f54f1386700 1 --2- 172.21.2.101:0/917983338 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54ec0b40e0 0x7f54ec0b44e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f54e4001c30 tx=0x7f54e400ac60).stop 2021-02-10T11:14:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.803 7f54f1386700 1 -- 172.21.2.101:0/917983338 shutdown_connections 2021-02-10T11:14:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.803 7f54f1386700 1 --2- 172.21.2.101:0/917983338 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54d0018730 0x7f54d001abc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.803 7f54f1386700 1 --2- 172.21.2.101:0/917983338 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54ec0a09f0 0x7f54ec0b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.803 7f54f1386700 1 --2- 172.21.2.101:0/917983338 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54ec0b40e0 0x7f54ec0b44e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.803 7f54f1386700 1 --2- 172.21.2.101:0/917983338 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54ec19acd0 0x7f54ec0a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.803 7f54f1386700 1 -- 172.21.2.101:0/917983338 >> 172.21.2.101:0/917983338 conn(0x7f54ec1a31d0 msgr2=0x7f54ec08b830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.803 7f54f1386700 1 -- 172.21.2.101:0/917983338 shutdown_connections 2021-02-10T11:14:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:32.803 7f54f1386700 1 -- 172.21.2.101:0/917983338 wait complete. 2021-02-10T11:14:32.885 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:14:33.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.291 7f04741ca700 1 Processor -- start 2021-02-10T11:14:33.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.291 7f04741ca700 1 -- start start 2021-02-10T11:14:33.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.291 7f04741ca700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f046c0e5470 0x7f046c07e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:33.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.291 7f04741ca700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f046c07f300 0x7f046c07f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:33.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.291 7f04741ca700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f046c07ed80 con 0x7f046c0e5070 2021-02-10T11:14:33.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.291 7f04741ca700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f046c1b4010 con 0x7f046c07f300 2021-02-10T11:14:33.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.292 7f04741ca700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f046c0d1790 con 0x7f046c0e5470 2021-02-10T11:14:33.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.292 7f04727e7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f046c07f300 0x7f046c07f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:33.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.292 7f0471fe6700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f046c0e5070 0x7f046c1b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45552/0 (socket says 172.21.2.101:45552) 2021-02-10T11:14:33.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.292 7f0471fe6700 1 -- 172.21.2.101:0/3484097613 learned_addr learned my addr 172.21.2.101:0/3484097613 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:14:33.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.292 7f0470fe4700 1 -- 172.21.2.101:0/3484097613 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3220603059 0 0) 0x7f0460001180 con 0x7f046c0e5070 2021-02-10T11:14:33.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.292 7f04727e7700 1 -- 172.21.2.101:0/3484097613 >> v2:172.21.2.108:3300/0 conn(0x7f046c0e5470 msgr2=0x7f046c07e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:33.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.292 7f04727e7700 1 --2- 172.21.2.101:0/3484097613 >> v2:172.21.2.108:3300/0 conn(0x7f046c0e5470 0x7f046c07e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:33.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.292 7f04727e7700 1 -- 172.21.2.101:0/3484097613 >> v1:172.21.2.101:6789/0 conn(0x7f046c0e5070 legacy=0x7f046c1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:33.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.292 7f04727e7700 1 -- 172.21.2.101:0/3484097613 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046c0d19b0 con 0x7f046c07f300 2021-02-10T11:14:33.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.292 7f04727e7700 1 --2- 172.21.2.101:0/3484097613 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f046c07f300 0x7f046c07f700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f0464006e30 tx=0x7f0464009e60).ready entity=mon.0 client_cookie=9abe2c690bb4fb9f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:33.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.293 7f0470fe4700 1 -- 172.21.2.101:0/3484097613 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0464006a50 con 0x7f046c07f300 2021-02-10T11:14:33.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.293 7f0470fe4700 1 -- 172.21.2.101:0/3484097613 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046400b300 con 0x7f046c07f300 2021-02-10T11:14:33.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.293 7f0470fe4700 1 -- 172.21.2.101:0/3484097613 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f046400b7e0 con 0x7f046c07f300 2021-02-10T11:14:33.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.293 7f04741ca700 1 -- 172.21.2.101:0/3484097613 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f046c07f300 msgr2=0x7f046c07f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:33.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.293 7f04741ca700 1 --2- 172.21.2.101:0/3484097613 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f046c07f300 0x7f046c07f700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f0464006e30 tx=0x7f0464009e60).stop 2021-02-10T11:14:33.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.293 7f04741ca700 1 -- 172.21.2.101:0/3484097613 shutdown_connections 2021-02-10T11:14:33.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.293 7f04741ca700 1 --2- 172.21.2.101:0/3484097613 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f046c07f300 0x7f046c07f700 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:33.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.293 7f04741ca700 1 --2- 172.21.2.101:0/3484097613 >> v2:172.21.2.108:3300/0 conn(0x7f046c0e5470 0x7f046c07e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:33.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.293 7f04741ca700 1 -- 172.21.2.101:0/3484097613 >> 172.21.2.101:0/3484097613 conn(0x7f046c0ce690 msgr2=0x7f046c0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:33.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.293 7f04741ca700 1 -- 172.21.2.101:0/3484097613 shutdown_connections 2021-02-10T11:14:33.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.293 7f04741ca700 1 -- 172.21.2.101:0/3484097613 wait complete. 2021-02-10T11:14:33.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.308 7f04741ca700 1 Processor -- start 2021-02-10T11:14:33.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.308 7f04741ca700 1 -- start start 2021-02-10T11:14:33.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.308 7f04741ca700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f046c08d1c0 0x7f046c081230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:33.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.308 7f04741ca700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f046c081850 0x7f046c0ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:33.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.308 7f04741ca700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f046c0ad6f0 0x7f046c1a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:33.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.308 7f04741ca700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046c087740 con 0x7f046c0ad6f0 2021-02-10T11:14:33.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.308 7f04741ca700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046c081db0 con 0x7f046c08d1c0 2021-02-10T11:14:33.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.308 7f04741ca700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f046c08d5c0 con 0x7f046c081850 2021-02-10T11:14:33.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.308 7f04727e7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f046c0ad6f0 0x7f046c1a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:33.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.308 7f04727e7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f046c0ad6f0 0x7f046c1a1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45064/0 (socket says 172.21.2.101:45064) 2021-02-10T11:14:33.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.308 7f04727e7700 1 -- 172.21.2.101:0/2497513225 learned_addr learned my addr 172.21.2.101:0/2497513225 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:33.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.309 7f04727e7700 1 -- 172.21.2.101:0/2497513225 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f046c081850 msgr2=0x7f046c0ad0d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:33.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.309 7f04727e7700 1 --2- 172.21.2.101:0/2497513225 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f046c081850 0x7f046c0ad0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:33.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.309 7f04727e7700 1 -- 172.21.2.101:0/2497513225 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f046c08d1c0 msgr2=0x7f046c081230 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:33.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.309 7f04727e7700 1 --2- 172.21.2.101:0/2497513225 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f046c08d1c0 0x7f046c081230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:33.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.309 7f04727e7700 1 -- 172.21.2.101:0/2497513225 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046c08da00 con 0x7f046c0ad6f0 2021-02-10T11:14:33.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.309 7f04727e7700 1 --2- 172.21.2.101:0/2497513225 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f046c0ad6f0 0x7f046c1a1d30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f046400b970 tx=0x7f046400bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:33.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.309 7f045e7fc700 1 -- 172.21.2.101:0/2497513225 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0464009380 con 0x7f046c0ad6f0 2021-02-10T11:14:33.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.309 7f045e7fc700 1 -- 172.21.2.101:0/2497513225 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0464005820 con 0x7f046c0ad6f0 2021-02-10T11:14:33.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.309 7f045e7fc700 1 -- 172.21.2.101:0/2497513225 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0464005c80 con 0x7f046c0ad6f0 2021-02-10T11:14:33.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.309 7f04741ca700 1 -- 172.21.2.101:0/2497513225 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f046c08da00 con 0x7f046c0ad6f0 2021-02-10T11:14:33.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.309 7f04741ca700 1 -- 172.21.2.101:0/2497513225 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f046c08da00 con 0x7f046c0ad6f0 2021-02-10T11:14:33.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.310 7f045e7fc700 1 -- 172.21.2.101:0/2497513225 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0464005a00 con 0x7f046c0ad6f0 2021-02-10T11:14:33.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.310 7f045e7fc700 1 --2- 172.21.2.101:0/2497513225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04540186c0 0x7f045401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:33.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.310 7f045e7fc700 1 -- 172.21.2.101:0/2497513225 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0464032800 con 0x7f046c0ad6f0 2021-02-10T11:14:33.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.310 7f0471fe6700 1 --2- 172.21.2.101:0/2497513225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04540186c0 0x7f045401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:33.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.310 7f0471fe6700 1 --2- 172.21.2.101:0/2497513225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04540186c0 0x7f045401abc0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:33.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.342 7f04741ca700 1 -- 172.21.2.101:0/2497513225 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f046c087740 con 0x7f046c0ad6f0 2021-02-10T11:14:33.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.344 7f045e7fc700 1 -- 172.21.2.101:0/2497513225 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0464009380 con 0x7f046c0ad6f0 2021-02-10T11:14:33.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.473 7f04741ca700 1 -- 172.21.2.101:0/2497513225 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f046c087740 con 0x7f046c0ad6f0 2021-02-10T11:14:33.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.473 7f045e7fc700 1 -- 172.21.2.101:0/2497513225 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2097 (secure 0 0 0) 0x7f04640059e0 con 0x7f046c0ad6f0 2021-02-10T11:14:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:14:33.477 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/3184957761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3184957761},{"type":"v1","addr":"172.21.2.108:6833","nonce":3184957761}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:13:14.147021","modified":"2021-02-10 11:13:23.547179","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":5,"state":"up:active","state_seq":3,"addr":"172.21.2.101:6835/3530104431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3530104431},{"type":"v1","addr":"172.21.2.101:6835","nonce":3530104431}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-02-10T11:14:33.477 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 6 2021-02-10T11:14:33.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.475 7f04741ca700 1 -- 172.21.2.101:0/2497513225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04540186c0 msgr2=0x7f045401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:33.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.475 7f04741ca700 1 --2- 172.21.2.101:0/2497513225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04540186c0 0x7f045401abc0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:33.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.475 7f04741ca700 1 -- 172.21.2.101:0/2497513225 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f046c0ad6f0 msgr2=0x7f046c1a1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:33.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.475 7f04741ca700 1 --2- 172.21.2.101:0/2497513225 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f046c0ad6f0 0x7f046c1a1d30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f046400b970 tx=0x7f046400bd10).stop 2021-02-10T11:14:33.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.475 7f04741ca700 1 -- 172.21.2.101:0/2497513225 shutdown_connections 2021-02-10T11:14:33.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.475 7f04741ca700 1 --2- 172.21.2.101:0/2497513225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04540186c0 0x7f045401abc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:33.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.475 7f04741ca700 1 --2- 172.21.2.101:0/2497513225 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f046c081850 0x7f046c0ad0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:33.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.475 7f04741ca700 1 --2- 172.21.2.101:0/2497513225 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f046c0ad6f0 0x7f046c1a1d30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:33.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.475 7f04741ca700 1 --2- 172.21.2.101:0/2497513225 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f046c08d1c0 0x7f046c081230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:33.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.475 7f04741ca700 1 -- 172.21.2.101:0/2497513225 >> 172.21.2.101:0/2497513225 conn(0x7f046c19b930 msgr2=0x7f046c1a0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:33.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.475 7f04741ca700 1 -- 172.21.2.101:0/2497513225 shutdown_connections 2021-02-10T11:14:33.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.475 7f04741ca700 1 -- 172.21.2.101:0/2497513225 wait complete. 2021-02-10T11:14:33.487 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs fail cephfs 2021-02-10T11:14:33.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.892 7fe68d678700 1 Processor -- start 2021-02-10T11:14:33.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.892 7fe68d678700 1 -- start start 2021-02-10T11:14:33.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.893 7fe68d678700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6880e5070 0x7fe6881b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:33.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.893 7fe68d678700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe6881b3e40 0x7fe688080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:33.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.893 7fe68d678700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe68807ed80 con 0x7fe6880e5470 2021-02-10T11:14:33.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.893 7fe68d678700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6880d1790 con 0x7fe6881b3e40 2021-02-10T11:14:33.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.893 7fe68d678700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6880d1980 con 0x7fe6880e5070 2021-02-10T11:14:33.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.893 7fe687ac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe6881b3e40 0x7fe688080310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:33.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.893 7fe686abe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe6880e5470 0x7fe68807d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45566/0 (socket says 172.21.2.101:45566) 2021-02-10T11:14:33.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.893 7fe686abe700 1 -- 172.21.2.101:0/1086020136 learned_addr learned my addr 172.21.2.101:0/1086020136 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:14:33.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.893 7fe6862bd700 1 -- 172.21.2.101:0/1086020136 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3685824646 0 0) 0x7fe67c0010b0 con 0x7fe6880e5470 2021-02-10T11:14:33.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.893 7fe6862bd700 1 -- 172.21.2.101:0/1086020136 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe670002fe0 con 0x7fe6880e5470 2021-02-10T11:14:33.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.893 7fe687ac0700 1 -- 172.21.2.101:0/1086020136 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6880e5070 msgr2=0x7fe6881b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:33.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.893 7fe687ac0700 1 --2- 172.21.2.101:0/1086020136 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6880e5070 0x7fe6881b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:33.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.893 7fe687ac0700 1 -- 172.21.2.101:0/1086020136 >> v1:172.21.2.101:6789/0 conn(0x7fe6880e5470 legacy=0x7fe68807d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:33.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.893 7fe687ac0700 1 -- 172.21.2.101:0/1086020136 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6880d1ba0 con 0x7fe6881b3e40 2021-02-10T11:14:33.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.894 7fe687ac0700 1 --2- 172.21.2.101:0/1086020136 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6881b3e40 0x7fe688080310 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fe680006e30 tx=0x7fe680009de0).ready entity=mon.0 client_cookie=75936d366d82361a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:33.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.894 7fe6862bd700 1 -- 172.21.2.101:0/1086020136 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe68000b300 con 0x7fe6881b3e40 2021-02-10T11:14:33.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.894 7fe6862bd700 1 -- 172.21.2.101:0/1086020136 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68000b680 con 0x7fe6881b3e40 2021-02-10T11:14:33.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.894 7fe6862bd700 1 -- 172.21.2.101:0/1086020136 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe68000bba0 con 0x7fe6881b3e40 2021-02-10T11:14:33.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.894 7fe68d678700 1 -- 172.21.2.101:0/1086020136 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6881b3e40 msgr2=0x7fe688080310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:33.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.894 7fe68d678700 1 --2- 172.21.2.101:0/1086020136 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6881b3e40 0x7fe688080310 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fe680006e30 tx=0x7fe680009de0).stop 2021-02-10T11:14:33.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.894 7fe68d678700 1 -- 172.21.2.101:0/1086020136 shutdown_connections 2021-02-10T11:14:33.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.894 7fe68d678700 1 --2- 172.21.2.101:0/1086020136 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6881b3e40 0x7fe688080310 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:33.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.894 7fe68d678700 1 --2- 172.21.2.101:0/1086020136 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6880e5070 0x7fe6881b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:33.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.894 7fe68d678700 1 -- 172.21.2.101:0/1086020136 >> 172.21.2.101:0/1086020136 conn(0x7fe6880ce690 msgr2=0x7fe6880e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:33.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.894 7fe68d678700 1 -- 172.21.2.101:0/1086020136 shutdown_connections 2021-02-10T11:14:33.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.894 7fe68d678700 1 -- 172.21.2.101:0/1086020136 wait complete. 2021-02-10T11:14:33.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.909 7fe68d678700 1 Processor -- start 2021-02-10T11:14:33.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.909 7fe68d678700 1 -- start start 2021-02-10T11:14:33.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.909 7fe68d678700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6881a3e50 0x7fe6881a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:33.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.909 7fe68d678700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6880a3d00 0x7fe6880a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:33.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.909 7fe68d678700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6880a4720 0x7fe6880a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:33.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.909 7fe68d678700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6880a39f0 con 0x7fe6880a4720 2021-02-10T11:14:33.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.909 7fe68d678700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6880be010 con 0x7fe6880a3d00 2021-02-10T11:14:33.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.909 7fe68d678700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6880c41d0 con 0x7fe6881a3e50 2021-02-10T11:14:33.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.910 7fe687ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6880a4720 0x7fe6880a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:33.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.910 7fe687ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6880a4720 0x7fe6880a4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45080/0 (socket says 172.21.2.101:45080) 2021-02-10T11:14:33.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.910 7fe687ac0700 1 -- 172.21.2.101:0/3089142282 learned_addr learned my addr 172.21.2.101:0/3089142282 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:33.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.910 7fe687ac0700 1 -- 172.21.2.101:0/3089142282 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6881a3e50 msgr2=0x7fe6881a4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:33.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.910 7fe687ac0700 1 --2- 172.21.2.101:0/3089142282 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6881a3e50 0x7fe6881a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:33.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.910 7fe687ac0700 1 -- 172.21.2.101:0/3089142282 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6880a3d00 msgr2=0x7fe6880a4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:33.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.910 7fe687ac0700 1 --2- 172.21.2.101:0/3089142282 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6880a3d00 0x7fe6880a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:33.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.910 7fe687ac0700 1 -- 172.21.2.101:0/3089142282 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6880a57b0 con 0x7fe6880a4720 2021-02-10T11:14:33.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.910 7fe687ac0700 1 --2- 172.21.2.101:0/3089142282 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6880a4720 0x7fe6880a4b20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fe68000b970 tx=0x7fe68000bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:33.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.910 7fe66ffff700 1 -- 172.21.2.101:0/3089142282 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe680009380 con 0x7fe6880a4720 2021-02-10T11:14:33.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.910 7fe66ffff700 1 -- 172.21.2.101:0/3089142282 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe680005820 con 0x7fe6880a4720 2021-02-10T11:14:33.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.910 7fe68d678700 1 -- 172.21.2.101:0/3089142282 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6880a57b0 con 0x7fe6880a4720 2021-02-10T11:14:33.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.910 7fe66ffff700 1 -- 172.21.2.101:0/3089142282 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe680005d40 con 0x7fe6880a4720 2021-02-10T11:14:33.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.910 7fe68d678700 1 -- 172.21.2.101:0/3089142282 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6880a5d40 con 0x7fe6880a4720 2021-02-10T11:14:33.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.911 7fe66ffff700 1 -- 172.21.2.101:0/3089142282 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe680005cd0 con 0x7fe6880a4720 2021-02-10T11:14:33.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.911 7fe66ffff700 1 --2- 172.21.2.101:0/3089142282 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6680186c0 0x7fe66801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:33.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.911 7fe66ffff700 1 -- 172.21.2.101:0/3089142282 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe680032ba0 con 0x7fe6880a4720 2021-02-10T11:14:33.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.911 7fe6872bf700 1 --2- 172.21.2.101:0/3089142282 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6680186c0 0x7fe66801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:33.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.911 7fe6872bf700 1 --2- 172.21.2.101:0/3089142282 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6680186c0 0x7fe66801abc0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:33.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.943 7fe68d678700 1 -- 172.21.2.101:0/3089142282 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe688084b00 con 0x7fe6880a4720 2021-02-10T11:14:33.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:33.945 7fe66ffff700 1 -- 172.21.2.101:0/3089142282 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe680005ac0 con 0x7fe6880a4720 2021-02-10T11:14:34.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.073 7fe68d678700 1 -- 172.21.2.101:0/3089142282 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe68807ed80 con 0x7fe6880a4720 2021-02-10T11:14:34.212 INFO:tasks.ceph.mon.a.gibba001.stderr:2021-02-10 11:14:34.208 7f974d04d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-02-10T11:14:34.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.214 7fe66ffff700 1 -- 172.21.2.101:0/3089142282 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe680005820 con 0x7fe6880a4720 2021-02-10T11:14:34.240 INFO:teuthology.orchestra.run.gibba001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-02-10T11:14:34.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.239 7fe68d678700 1 -- 172.21.2.101:0/3089142282 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6680186c0 msgr2=0x7fe66801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:34.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.239 7fe68d678700 1 --2- 172.21.2.101:0/3089142282 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6680186c0 0x7fe66801abc0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:34.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.239 7fe68d678700 1 -- 172.21.2.101:0/3089142282 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6880a4720 msgr2=0x7fe6880a4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:34.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.239 7fe68d678700 1 --2- 172.21.2.101:0/3089142282 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6880a4720 0x7fe6880a4b20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fe68000b970 tx=0x7fe68000bd10).stop 2021-02-10T11:14:34.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.239 7fe68d678700 1 -- 172.21.2.101:0/3089142282 shutdown_connections 2021-02-10T11:14:34.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.239 7fe68d678700 1 --2- 172.21.2.101:0/3089142282 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6880a4720 0x7fe6880a4b20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:34.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.239 7fe68d678700 1 --2- 172.21.2.101:0/3089142282 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6880a3d00 0x7fe6880a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:34.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.239 7fe68d678700 1 --2- 172.21.2.101:0/3089142282 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6680186c0 0x7fe66801abc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:34.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.239 7fe68d678700 1 --2- 172.21.2.101:0/3089142282 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6881a3e50 0x7fe6881a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:34.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.239 7fe68d678700 1 -- 172.21.2.101:0/3089142282 >> 172.21.2.101:0/3089142282 conn(0x7fe688090990 msgr2=0x7fe6880adcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:34.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.239 7fe68d678700 1 -- 172.21.2.101:0/3089142282 shutdown_connections 2021-02-10T11:14:34.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.239 7fe68d678700 1 -- 172.21.2.101:0/3089142282 wait complete. 2021-02-10T11:14:34.251 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:14:34.262 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:14:34.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.656 7f4ec81a5700 1 Processor -- start 2021-02-10T11:14:34.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.656 7f4ec81a5700 1 -- start start 2021-02-10T11:14:34.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.657 7f4ec81a5700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f4ec00e5070 0x7f4ec01b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:34.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.657 7f4ec81a5700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec007ed80 con 0x7f4ec01b3e40 2021-02-10T11:14:34.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.657 7f4ec81a5700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec007e670 con 0x7f4ec00e5470 2021-02-10T11:14:34.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.657 7f4ec81a5700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec007e890 con 0x7f4ec00e5070 2021-02-10T11:14:34.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.657 7f4ec5fc1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f4ec00e5070 0x7f4ec01b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:34.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.657 7f4ec57c0700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f4ec01b3e40 0x7f4ec007d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45588/0 (socket says 172.21.2.101:45588) 2021-02-10T11:14:34.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.657 7f4ec57c0700 1 -- 172.21.2.101:0/3003030589 learned_addr learned my addr 172.21.2.101:0/3003030589 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:14:34.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.657 7f4ec5fc1700 1 -- 172.21.2.101:0/3003030589 >> v1:172.21.2.108:6789/0 conn(0x7f4ec00e5470 legacy=0x7f4ec007de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:34.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.658 7f4ec5fc1700 1 -- 172.21.2.101:0/3003030589 >> v1:172.21.2.101:6789/0 conn(0x7f4ec01b3e40 legacy=0x7f4ec007d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:34.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.658 7f4ec5fc1700 1 -- 172.21.2.101:0/3003030589 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec007eab0 con 0x7f4ec00e5070 2021-02-10T11:14:34.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.658 7f4ec5fc1700 1 --2- 172.21.2.101:0/3003030589 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ec00e5070 0x7f4ec01b3820 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f4eb4006650 tx=0x7f4eb4009a80).ready entity=mon.0 client_cookie=a25cb01f5ced79b1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:34.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.658 7f4ec4fbf700 1 -- 172.21.2.101:0/3003030589 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4eb400c0c0 con 0x7f4ec00e5070 2021-02-10T11:14:34.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.658 7f4ec4fbf700 1 -- 172.21.2.101:0/3003030589 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb400c340 con 0x7f4ec00e5070 2021-02-10T11:14:34.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.658 7f4ec4fbf700 1 -- 172.21.2.101:0/3003030589 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4eb400c860 con 0x7f4ec00e5070 2021-02-10T11:14:34.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.658 7f4ec81a5700 1 -- 172.21.2.101:0/3003030589 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ec00e5070 msgr2=0x7f4ec01b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:34.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.658 7f4ec81a5700 1 --2- 172.21.2.101:0/3003030589 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ec00e5070 0x7f4ec01b3820 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f4eb4006650 tx=0x7f4eb4009a80).stop 2021-02-10T11:14:34.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.659 7f4ec81a5700 1 -- 172.21.2.101:0/3003030589 shutdown_connections 2021-02-10T11:14:34.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.659 7f4ec81a5700 1 --2- 172.21.2.101:0/3003030589 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ec00e5070 0x7f4ec01b3820 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:34.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.659 7f4ec81a5700 1 -- 172.21.2.101:0/3003030589 >> 172.21.2.101:0/3003030589 conn(0x7f4ec00ce690 msgr2=0x7f4ec00e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:34.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.659 7f4ec81a5700 1 -- 172.21.2.101:0/3003030589 shutdown_connections 2021-02-10T11:14:34.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.659 7f4ec81a5700 1 -- 172.21.2.101:0/3003030589 wait complete. 2021-02-10T11:14:34.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.673 7f4ec81a5700 1 Processor -- start 2021-02-10T11:14:34.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.674 7f4ec81a5700 1 -- start start 2021-02-10T11:14:34.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.674 7f4ec81a5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ec0092dc0 0x7f4ec00931c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:34.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.674 7f4ec81a5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ec00937e0 0x7f4ec00ab250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:34.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.674 7f4ec81a5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ec00ab870 0x7f4ec00942e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:34.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.674 7f4ec81a5700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec0093de0 con 0x7f4ec00ab870 2021-02-10T11:14:34.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.674 7f4ec81a5700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec01c2500 con 0x7f4ec0092dc0 2021-02-10T11:14:34.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.674 7f4ec81a5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec01c2720 con 0x7f4ec00937e0 2021-02-10T11:14:34.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.674 7f4ec67c2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ec00ab870 0x7f4ec00942e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:34.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.674 7f4ec67c2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ec00ab870 0x7f4ec00942e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45100/0 (socket says 172.21.2.101:45100) 2021-02-10T11:14:34.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.674 7f4ec67c2700 1 -- 172.21.2.101:0/4091664502 learned_addr learned my addr 172.21.2.101:0/4091664502 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:34.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.675 7f4ec67c2700 1 -- 172.21.2.101:0/4091664502 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ec00937e0 msgr2=0x7f4ec00ab250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:34.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.675 7f4ec67c2700 1 --2- 172.21.2.101:0/4091664502 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ec00937e0 0x7f4ec00ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:34.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.675 7f4ec67c2700 1 -- 172.21.2.101:0/4091664502 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ec0092dc0 msgr2=0x7f4ec00931c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:34.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.675 7f4ec67c2700 1 --2- 172.21.2.101:0/4091664502 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ec0092dc0 0x7f4ec00931c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:34.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.675 7f4ec67c2700 1 -- 172.21.2.101:0/4091664502 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec00abce0 con 0x7f4ec00ab870 2021-02-10T11:14:34.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.675 7f4ec67c2700 1 --2- 172.21.2.101:0/4091664502 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ec00ab870 0x7f4ec00942e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f4eb8009940 tx=0x7f4eb800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:34.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.675 7f4eb27fc700 1 -- 172.21.2.101:0/4091664502 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4eb8000f90 con 0x7f4ec00ab870 2021-02-10T11:14:34.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.675 7f4eb27fc700 1 -- 172.21.2.101:0/4091664502 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb800c2f0 con 0x7f4ec00ab870 2021-02-10T11:14:34.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.675 7f4eb27fc700 1 -- 172.21.2.101:0/4091664502 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4eb800c810 con 0x7f4ec00ab870 2021-02-10T11:14:34.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.675 7f4ec81a5700 1 -- 172.21.2.101:0/4091664502 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec00abce0 con 0x7f4ec00ab870 2021-02-10T11:14:34.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.675 7f4ec81a5700 1 -- 172.21.2.101:0/4091664502 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec00abce0 con 0x7f4ec00ab870 2021-02-10T11:14:34.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.676 7f4eb27fc700 1 -- 172.21.2.101:0/4091664502 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4eb8001220 con 0x7f4ec00ab870 2021-02-10T11:14:34.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.676 7f4eb27fc700 1 --2- 172.21.2.101:0/4091664502 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4ea80186c0 0x7f4ea801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:34.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.676 7f4eb27fc700 1 -- 172.21.2.101:0/4091664502 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f4eb8032990 con 0x7f4ec00ab870 2021-02-10T11:14:34.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.676 7f4ec5fc1700 1 --2- 172.21.2.101:0/4091664502 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4ea80186c0 0x7f4ea801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:34.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.676 7f4ec5fc1700 1 --2- 172.21.2.101:0/4091664502 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4ea80186c0 0x7f4ea801abc0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:34.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.707 7f4ec81a5700 1 -- 172.21.2.101:0/4091664502 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec00ce690 con 0x7f4ec00ab870 2021-02-10T11:14:34.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.709 7f4eb27fc700 1 -- 172.21.2.101:0/4091664502 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4eb800be30 con 0x7f4ec00ab870 2021-02-10T11:14:34.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.837 7f4ec81a5700 1 -- 172.21.2.101:0/4091664502 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ec00ce690 con 0x7f4ec00ab870 2021-02-10T11:14:34.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.837 7f4eb27fc700 1 -- 172.21.2.101:0/4091664502 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+1753 (secure 0 0 0) 0x7f4eb801f8c0 con 0x7f4ec00ab870 2021-02-10T11:14:34.841 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 7 2021-02-10T11:14:34.841 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:14:34.841 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/3184957761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3184957761},{"type":"v1","addr":"172.21.2.108:6833","nonce":3184957761}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":7,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:13:14.147021","modified":"2021-02-10 11:14:34.208536","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":21,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-02-10T11:14:34.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.839 7f4ec81a5700 1 -- 172.21.2.101:0/4091664502 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4ea80186c0 msgr2=0x7f4ea801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:34.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.839 7f4ec81a5700 1 --2- 172.21.2.101:0/4091664502 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4ea80186c0 0x7f4ea801abc0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:34.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.839 7f4ec81a5700 1 -- 172.21.2.101:0/4091664502 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ec00ab870 msgr2=0x7f4ec00942e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:34.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.839 7f4ec81a5700 1 --2- 172.21.2.101:0/4091664502 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ec00ab870 0x7f4ec00942e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f4eb8009940 tx=0x7f4eb800bcf0).stop 2021-02-10T11:14:34.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.839 7f4ec81a5700 1 -- 172.21.2.101:0/4091664502 shutdown_connections 2021-02-10T11:14:34.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.839 7f4ec81a5700 1 --2- 172.21.2.101:0/4091664502 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4ea80186c0 0x7f4ea801abc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:34.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.839 7f4ec81a5700 1 --2- 172.21.2.101:0/4091664502 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ec00937e0 0x7f4ec00ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:34.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.839 7f4ec81a5700 1 --2- 172.21.2.101:0/4091664502 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ec00ab870 0x7f4ec00942e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:34.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.839 7f4ec81a5700 1 --2- 172.21.2.101:0/4091664502 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ec0092dc0 0x7f4ec00931c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:34.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.839 7f4ec81a5700 1 -- 172.21.2.101:0/4091664502 >> 172.21.2.101:0/4091664502 conn(0x7f4ec0106df0 msgr2=0x7f4ec00962d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:34.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.839 7f4ec81a5700 1 -- 172.21.2.101:0/4091664502 shutdown_connections 2021-02-10T11:14:34.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:34.839 7f4ec81a5700 1 -- 172.21.2.101:0/4091664502 wait complete. 2021-02-10T11:14:34.852 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-02-10T11:14:35.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.301 7f0e407f8700 1 Processor -- start 2021-02-10T11:14:35.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.301 7f0e407f8700 1 -- start start 2021-02-10T11:14:35.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.301 7f0e407f8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0e380d17f0 0x7f0e381b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:35.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.301 7f0e407f8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e380d1bf0 0x7f0e381b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:35.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.301 7f0e407f8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e380f0b20 con 0x7f0e381b3e20 2021-02-10T11:14:35.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.301 7f0e407f8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e380e5eb0 con 0x7f0e380d17f0 2021-02-10T11:14:35.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.301 7f0e407f8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e380f0630 con 0x7f0e380d1bf0 2021-02-10T11:14:35.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.302 7f0e3e614700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0e380d17f0 0x7f0e381b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:35.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.302 7f0e3e614700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0e380d17f0 0x7f0e381b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45104/0 (socket says 172.21.2.101:45104) 2021-02-10T11:14:35.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.302 7f0e3e614700 1 -- 172.21.2.101:0/2028786126 learned_addr learned my addr 172.21.2.101:0/2028786126 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:35.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.302 7f0e3e614700 1 -- 172.21.2.101:0/2028786126 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e380d1bf0 msgr2=0x7f0e381b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:35.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.302 7f0e3e614700 1 --2- 172.21.2.101:0/2028786126 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e380d1bf0 0x7f0e381b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:35.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.302 7f0e3e614700 1 -- 172.21.2.101:0/2028786126 >> v1:172.21.2.108:6789/0 conn(0x7f0e381b3e20 legacy=0x7f0e380ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:35.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.302 7f0e3e614700 1 -- 172.21.2.101:0/2028786126 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e380f0820 con 0x7f0e380d17f0 2021-02-10T11:14:35.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.302 7f0e3e614700 1 --2- 172.21.2.101:0/2028786126 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e380d17f0 0x7f0e381b31e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f0e2c006700 tx=0x7f0e2c009b80).ready entity=mon.0 client_cookie=3b61117555feecac server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:35.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.302 7f0e3d612700 1 -- 172.21.2.101:0/2028786126 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e2c000c90 con 0x7f0e380d17f0 2021-02-10T11:14:35.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.302 7f0e3d612700 1 -- 172.21.2.101:0/2028786126 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2c00c0c0 con 0x7f0e380d17f0 2021-02-10T11:14:35.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.302 7f0e3d612700 1 -- 172.21.2.101:0/2028786126 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e2c00c520 con 0x7f0e380d17f0 2021-02-10T11:14:35.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.302 7f0e407f8700 1 -- 172.21.2.101:0/2028786126 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e380d17f0 msgr2=0x7f0e381b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:35.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.303 7f0e407f8700 1 --2- 172.21.2.101:0/2028786126 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e380d17f0 0x7f0e381b31e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f0e2c006700 tx=0x7f0e2c009b80).stop 2021-02-10T11:14:35.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.303 7f0e407f8700 1 -- 172.21.2.101:0/2028786126 shutdown_connections 2021-02-10T11:14:35.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.303 7f0e407f8700 1 --2- 172.21.2.101:0/2028786126 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e380d1bf0 0x7f0e381b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:35.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.303 7f0e407f8700 1 --2- 172.21.2.101:0/2028786126 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e380d17f0 0x7f0e381b31e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:35.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.303 7f0e407f8700 1 -- 172.21.2.101:0/2028786126 >> 172.21.2.101:0/2028786126 conn(0x7f0e380fde40 msgr2=0x7f0e380ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:35.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.303 7f0e407f8700 1 -- 172.21.2.101:0/2028786126 shutdown_connections 2021-02-10T11:14:35.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.303 7f0e407f8700 1 -- 172.21.2.101:0/2028786126 wait complete. 2021-02-10T11:14:35.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.318 7f0e407f8700 1 Processor -- start 2021-02-10T11:14:35.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.318 7f0e407f8700 1 -- start start 2021-02-10T11:14:35.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.318 7f0e407f8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e381960f0 0x7f0e3819b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:35.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.318 7f0e407f8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e3818ecc0 0x7f0e38195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:35.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.318 7f0e407f8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e3819c280 0x7f0e3819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:35.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.318 7f0e407f8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3818f0c0 con 0x7f0e3819c280 2021-02-10T11:14:35.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.318 7f0e407f8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e38195d60 con 0x7f0e3818ecc0 2021-02-10T11:14:35.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.318 7f0e407f8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3819cca0 con 0x7f0e381960f0 2021-02-10T11:14:35.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.319 7f0e3ee15700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e3819c280 0x7f0e3819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:35.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.319 7f0e3de13700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e3818ecc0 0x7f0e38195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:35.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.319 7f0e3de13700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e3818ecc0 0x7f0e38195740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:55594/0 (socket says 172.21.2.101:55594) 2021-02-10T11:14:35.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.319 7f0e3ee15700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e3819c280 0x7f0e3819c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45112/0 (socket says 172.21.2.101:45112) 2021-02-10T11:14:35.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.319 7f0e3de13700 1 -- 172.21.2.101:0/1766452981 learned_addr learned my addr 172.21.2.101:0/1766452981 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:35.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.319 7f0e3e614700 1 --2- 172.21.2.101:0/1766452981 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e381960f0 0x7f0e3819b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:35.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.319 7f0e3ee15700 1 -- 172.21.2.101:0/1766452981 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e381960f0 msgr2=0x7f0e3819b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:35.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.319 7f0e3ee15700 1 --2- 172.21.2.101:0/1766452981 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e381960f0 0x7f0e3819b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:35.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.319 7f0e3ee15700 1 -- 172.21.2.101:0/1766452981 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e3818ecc0 msgr2=0x7f0e38195740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:35.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.319 7f0e3ee15700 1 --2- 172.21.2.101:0/1766452981 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e3818ecc0 0x7f0e38195740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:35.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.319 7f0e3ee15700 1 -- 172.21.2.101:0/1766452981 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3819cec0 con 0x7f0e3819c280 2021-02-10T11:14:35.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.319 7f0e3ee15700 1 --2- 172.21.2.101:0/1766452981 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e3819c280 0x7f0e3819c680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f0e3000b890 tx=0x7f0e3000bc30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:35.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.319 7f0e2affd700 1 -- 172.21.2.101:0/1766452981 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e30006a30 con 0x7f0e3819c280 2021-02-10T11:14:35.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.319 7f0e2affd700 1 -- 172.21.2.101:0/1766452981 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3000bd70 con 0x7f0e3819c280 2021-02-10T11:14:35.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.319 7f0e2affd700 1 -- 172.21.2.101:0/1766452981 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e3000c570 con 0x7f0e3819c280 2021-02-10T11:14:35.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.319 7f0e407f8700 1 -- 172.21.2.101:0/1766452981 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3819cec0 con 0x7f0e3819c280 2021-02-10T11:14:35.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.319 7f0e407f8700 1 -- 172.21.2.101:0/1766452981 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3819d450 con 0x7f0e3819c280 2021-02-10T11:14:35.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.320 7f0e2affd700 1 -- 172.21.2.101:0/1766452981 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0e30001220 con 0x7f0e3819c280 2021-02-10T11:14:35.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.320 7f0e2affd700 1 --2- 172.21.2.101:0/1766452981 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e20018730 0x7f0e2001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:35.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.320 7f0e2affd700 1 -- 172.21.2.101:0/1766452981 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f0e30032970 con 0x7f0e3819c280 2021-02-10T11:14:35.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.320 7f0e3e614700 1 --2- 172.21.2.101:0/1766452981 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e20018730 0x7f0e2001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:35.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.322 7f0e3e614700 1 --2- 172.21.2.101:0/1766452981 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e20018730 0x7f0e2001abc0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:35.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.351 7f0e407f8700 1 -- 172.21.2.101:0/1766452981 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e3819bd50 con 0x7f0e3819c280 2021-02-10T11:14:35.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.355 7f0e2affd700 1 -- 172.21.2.101:0/1766452981 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0e30000f90 con 0x7f0e3819c280 2021-02-10T11:14:35.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:35.481 7f0e407f8700 1 -- 172.21.2.101:0/1766452981 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0e3819bd50 con 0x7f0e3819c280 2021-02-10T11:14:36.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.220 7f0e2affd700 1 -- 172.21.2.101:0/1766452981 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0e3001f7e0 con 0x7f0e3819c280 2021-02-10T11:14:36.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.252 7f0e407f8700 1 -- 172.21.2.101:0/1766452981 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e20018730 msgr2=0x7f0e2001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:36.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.252 7f0e407f8700 1 --2- 172.21.2.101:0/1766452981 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e20018730 0x7f0e2001abc0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:36.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.252 7f0e407f8700 1 -- 172.21.2.101:0/1766452981 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e3819c280 msgr2=0x7f0e3819c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:36.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.252 7f0e407f8700 1 --2- 172.21.2.101:0/1766452981 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e3819c280 0x7f0e3819c680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f0e3000b890 tx=0x7f0e3000bc30).stop 2021-02-10T11:14:36.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.252 7f0e407f8700 1 -- 172.21.2.101:0/1766452981 shutdown_connections 2021-02-10T11:14:36.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.252 7f0e407f8700 1 --2- 172.21.2.101:0/1766452981 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e3819c280 0x7f0e3819c680 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:36.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.252 7f0e407f8700 1 --2- 172.21.2.101:0/1766452981 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e3818ecc0 0x7f0e38195740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:36.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.252 7f0e407f8700 1 --2- 172.21.2.101:0/1766452981 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e20018730 0x7f0e2001abc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:36.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.252 7f0e407f8700 1 --2- 172.21.2.101:0/1766452981 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e381960f0 0x7f0e3819b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:36.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.252 7f0e407f8700 1 -- 172.21.2.101:0/1766452981 >> 172.21.2.101:0/1766452981 conn(0x7f0e3819f7e0 msgr2=0x7f0e3818c240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:36.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.252 7f0e407f8700 1 -- 172.21.2.101:0/1766452981 shutdown_connections 2021-02-10T11:14:36.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.252 7f0e407f8700 1 -- 172.21.2.101:0/1766452981 wait complete. 2021-02-10T11:14:36.264 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-02-10T11:14:36.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.674 7f3e288e0700 1 Processor -- start 2021-02-10T11:14:36.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.674 7f3e288e0700 1 -- start start 2021-02-10T11:14:36.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.675 7f3e288e0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e240d1870 0x7f3e241b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:36.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.675 7f3e288e0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3e241b0ae0 0x7f3e24108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:36.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.675 7f3e288e0700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e24109a60 con 0x7f3e240d1c70 2021-02-10T11:14:36.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.675 7f3e288e0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e240e5ef0 con 0x7f3e241b0ae0 2021-02-10T11:14:36.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.675 7f3e288e0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e24109750 con 0x7f3e240d1870 2021-02-10T11:14:36.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.675 7f3e21e1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3e241b0ae0 0x7f3e24108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:36.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.675 7f3e22e1d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f3e240d1c70 0x7f3e24108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45610/0 (socket says 172.21.2.101:45610) 2021-02-10T11:14:36.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.675 7f3e21e1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3e241b0ae0 0x7f3e24108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45116/0 (socket says 172.21.2.101:45116) 2021-02-10T11:14:36.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.675 7f3e22e1d700 1 -- 172.21.2.101:0/2931771417 learned_addr learned my addr 172.21.2.101:0/2931771417 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:14:36.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.675 7f3e21e1b700 1 -- 172.21.2.101:0/2931771417 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e240d1870 msgr2=0x7f3e241b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:36.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.675 7f3e21e1b700 1 --2- 172.21.2.101:0/2931771417 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e240d1870 0x7f3e241b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:36.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.675 7f3e21e1b700 1 -- 172.21.2.101:0/2931771417 >> v1:172.21.2.101:6789/0 conn(0x7f3e240d1c70 legacy=0x7f3e24108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:36.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.675 7f3e21e1b700 1 -- 172.21.2.101:0/2931771417 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e24088020 con 0x7f3e241b0ae0 2021-02-10T11:14:36.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.676 7f3e21e1b700 1 --2- 172.21.2.101:0/2931771417 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e241b0ae0 0x7f3e24108940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f3e18006650 tx=0x7f3e18009a80).ready entity=mon.0 client_cookie=334d2b4cd177a32e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:36.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.676 7f3e2161a700 1 -- 172.21.2.101:0/2931771417 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3e1800c0c0 con 0x7f3e241b0ae0 2021-02-10T11:14:36.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.676 7f3e2161a700 1 -- 172.21.2.101:0/2931771417 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1800c340 con 0x7f3e241b0ae0 2021-02-10T11:14:36.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.676 7f3e2161a700 1 -- 172.21.2.101:0/2931771417 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3e1800c860 con 0x7f3e241b0ae0 2021-02-10T11:14:36.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.676 7f3e288e0700 1 -- 172.21.2.101:0/2931771417 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e241b0ae0 msgr2=0x7f3e24108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:36.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.676 7f3e288e0700 1 --2- 172.21.2.101:0/2931771417 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e241b0ae0 0x7f3e24108940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f3e18006650 tx=0x7f3e18009a80).stop 2021-02-10T11:14:36.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.676 7f3e288e0700 1 -- 172.21.2.101:0/2931771417 shutdown_connections 2021-02-10T11:14:36.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.676 7f3e288e0700 1 --2- 172.21.2.101:0/2931771417 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e241b0ae0 0x7f3e24108940 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:36.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.676 7f3e288e0700 1 --2- 172.21.2.101:0/2931771417 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e240d1870 0x7f3e241b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:36.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.676 7f3e288e0700 1 -- 172.21.2.101:0/2931771417 >> 172.21.2.101:0/2931771417 conn(0x7f3e240ce420 msgr2=0x7f3e240d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:36.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.677 7f3e288e0700 1 -- 172.21.2.101:0/2931771417 shutdown_connections 2021-02-10T11:14:36.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.677 7f3e288e0700 1 -- 172.21.2.101:0/2931771417 wait complete. 2021-02-10T11:14:36.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.691 7f3e288e0700 1 Processor -- start 2021-02-10T11:14:36.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.691 7f3e288e0700 1 -- start start 2021-02-10T11:14:36.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.692 7f3e288e0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e24187e70 0x7f3e240ac030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:36.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.692 7f3e288e0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3e240ac650 0x7f3e241889a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:36.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.692 7f3e288e0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e24190fd0 0x7f3e24188fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:36.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.692 7f3e288e0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e24090130 con 0x7f3e24190fd0 2021-02-10T11:14:36.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.692 7f3e288e0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e24188420 con 0x7f3e240ac650 2021-02-10T11:14:36.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.692 7f3e288e0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e24099f60 con 0x7f3e24187e70 2021-02-10T11:14:36.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.692 7f3e22e1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e24190fd0 0x7f3e24188fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:36.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.692 7f3e22e1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e24190fd0 0x7f3e24188fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45120/0 (socket says 172.21.2.101:45120) 2021-02-10T11:14:36.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.692 7f3e22e1d700 1 -- 172.21.2.101:0/2313291407 learned_addr learned my addr 172.21.2.101:0/2313291407 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:36.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.692 7f3e22e1d700 1 -- 172.21.2.101:0/2313291407 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e24187e70 msgr2=0x7f3e240ac030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:36.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.692 7f3e22e1d700 1 --2- 172.21.2.101:0/2313291407 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e24187e70 0x7f3e240ac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:36.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.692 7f3e22e1d700 1 -- 172.21.2.101:0/2313291407 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3e240ac650 msgr2=0x7f3e241889a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:36.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.692 7f3e22e1d700 1 --2- 172.21.2.101:0/2313291407 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3e240ac650 0x7f3e241889a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:36.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.692 7f3e22e1d700 1 -- 172.21.2.101:0/2313291407 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2408fcc0 con 0x7f3e24190fd0 2021-02-10T11:14:36.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.692 7f3e22e1d700 1 --2- 172.21.2.101:0/2313291407 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e24190fd0 0x7f3e24188fc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f3e1c00a900 tx=0x7f3e1c00aca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:36.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.693 7f3e12ffd700 1 -- 172.21.2.101:0/2313291407 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3e1c000ce0 con 0x7f3e24190fd0 2021-02-10T11:14:36.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.693 7f3e12ffd700 1 -- 172.21.2.101:0/2313291407 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1c0070b0 con 0x7f3e24190fd0 2021-02-10T11:14:36.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.693 7f3e288e0700 1 -- 172.21.2.101:0/2313291407 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e2408fcc0 con 0x7f3e24190fd0 2021-02-10T11:14:36.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.693 7f3e288e0700 1 -- 172.21.2.101:0/2313291407 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e241895e0 con 0x7f3e24190fd0 2021-02-10T11:14:36.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.693 7f3e12ffd700 1 -- 172.21.2.101:0/2313291407 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3e1c0074c0 con 0x7f3e24190fd0 2021-02-10T11:14:36.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.693 7f3e12ffd700 1 -- 172.21.2.101:0/2313291407 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3e1c000cc0 con 0x7f3e24190fd0 2021-02-10T11:14:36.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.693 7f3e12ffd700 1 --2- 172.21.2.101:0/2313291407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3e08018730 0x7f3e0801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:36.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.693 7f3e12ffd700 1 -- 172.21.2.101:0/2313291407 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f3e1c032b70 con 0x7f3e24190fd0 2021-02-10T11:14:36.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.695 7f3e2261c700 1 --2- 172.21.2.101:0/2313291407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3e08018730 0x7f3e0801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:36.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.695 7f3e2261c700 1 --2- 172.21.2.101:0/2313291407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3e08018730 0x7f3e0801ac10 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:36.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.725 7f3e288e0700 1 -- 172.21.2.101:0/2313291407 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e240828e0 con 0x7f3e24190fd0 2021-02-10T11:14:36.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.727 7f3e12ffd700 1 -- 172.21.2.101:0/2313291407 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3e1c000f90 con 0x7f3e24190fd0 2021-02-10T11:14:36.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:36.854 7f3e288e0700 1 -- 172.21.2.101:0/2313291407 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3e240828e0 con 0x7f3e24190fd0 2021-02-10T11:14:37.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.225 7f3e12ffd700 1 -- 172.21.2.101:0/2313291407 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v22) v1 ==== 185+0+0 (secure 0 0 0) 0x7f3e1c0070b0 con 0x7f3e24190fd0 2021-02-10T11:14:37.275 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' removed 2021-02-10T11:14:37.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.273 7f3e288e0700 1 -- 172.21.2.101:0/2313291407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3e08018730 msgr2=0x7f3e0801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:37.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.273 7f3e288e0700 1 --2- 172.21.2.101:0/2313291407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3e08018730 0x7f3e0801ac10 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:37.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.273 7f3e288e0700 1 -- 172.21.2.101:0/2313291407 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e24190fd0 msgr2=0x7f3e24188fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:37.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.273 7f3e288e0700 1 --2- 172.21.2.101:0/2313291407 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e24190fd0 0x7f3e24188fc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f3e1c00a900 tx=0x7f3e1c00aca0).stop 2021-02-10T11:14:37.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.273 7f3e288e0700 1 -- 172.21.2.101:0/2313291407 shutdown_connections 2021-02-10T11:14:37.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.273 7f3e288e0700 1 --2- 172.21.2.101:0/2313291407 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e24190fd0 0x7f3e24188fc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:37.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.273 7f3e288e0700 1 --2- 172.21.2.101:0/2313291407 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3e240ac650 0x7f3e241889a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:37.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.273 7f3e288e0700 1 --2- 172.21.2.101:0/2313291407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3e08018730 0x7f3e0801ac10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:37.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.273 7f3e288e0700 1 --2- 172.21.2.101:0/2313291407 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e24187e70 0x7f3e240ac030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:37.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.273 7f3e288e0700 1 -- 172.21.2.101:0/2313291407 >> 172.21.2.101:0/2313291407 conn(0x7f3e2419f820 msgr2=0x7f3e241a5340 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:37.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.273 7f3e288e0700 1 -- 172.21.2.101:0/2313291407 shutdown_connections 2021-02-10T11:14:37.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.273 7f3e288e0700 1 -- 172.21.2.101:0/2313291407 wait complete. 2021-02-10T11:14:37.286 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-02-10T11:14:37.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.695 7fa8a7bf0700 1 Processor -- start 2021-02-10T11:14:37.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.696 7fa8a7bf0700 1 -- start start 2021-02-10T11:14:37.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.696 7fa8a7bf0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa8a00d1c70 0x7fa8a00cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:37.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.696 7fa8a7bf0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8a00cf580 0x7fa8a00cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:37.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.696 7fa8a7bf0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a00fe1c0 con 0x7fa8a00d1870 2021-02-10T11:14:37.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.696 7fa8a7bf0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a01b0cb0 con 0x7fa8a00d1c70 2021-02-10T11:14:37.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.696 7fa8a7bf0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a0109a60 con 0x7fa8a00cf580 2021-02-10T11:14:37.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.696 7fa8a520b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa8a00d1c70 0x7fa8a00cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:37.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.696 7fa8a520b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa8a00d1c70 0x7fa8a00cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45126/0 (socket says 172.21.2.101:45126) 2021-02-10T11:14:37.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.696 7fa8a520b700 1 -- 172.21.2.101:0/2018037900 learned_addr learned my addr 172.21.2.101:0/2018037900 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:37.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.697 7fa8a520b700 1 -- 172.21.2.101:0/2018037900 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8a00cf580 msgr2=0x7fa8a00cf980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:37.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.697 7fa8a520b700 1 --2- 172.21.2.101:0/2018037900 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8a00cf580 0x7fa8a00cf980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:37.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.697 7fa8a520b700 1 -- 172.21.2.101:0/2018037900 >> v1:172.21.2.108:6789/0 conn(0x7fa8a00d1870 legacy=0x7fa8a01b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:37.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.697 7fa8a520b700 1 -- 172.21.2.101:0/2018037900 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a0088020 con 0x7fa8a00d1c70 2021-02-10T11:14:37.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.697 7fa8a520b700 1 --2- 172.21.2.101:0/2018037900 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8a00d1c70 0x7fa8a00cef60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fa894006700 tx=0x7fa894009b80).ready entity=mon.0 client_cookie=7900145e0473c188 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:37.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.697 7fa8a4a0a700 1 -- 172.21.2.101:0/2018037900 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa89400c0c0 con 0x7fa8a00d1c70 2021-02-10T11:14:37.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.697 7fa8a4a0a700 1 -- 172.21.2.101:0/2018037900 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89400c340 con 0x7fa8a00d1c70 2021-02-10T11:14:37.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.697 7fa8a4a0a700 1 -- 172.21.2.101:0/2018037900 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa89400c860 con 0x7fa8a00d1c70 2021-02-10T11:14:37.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.697 7fa8a7bf0700 1 -- 172.21.2.101:0/2018037900 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8a00d1c70 msgr2=0x7fa8a00cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:37.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.697 7fa8a7bf0700 1 --2- 172.21.2.101:0/2018037900 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8a00d1c70 0x7fa8a00cef60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fa894006700 tx=0x7fa894009b80).stop 2021-02-10T11:14:37.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.698 7fa8a7bf0700 1 -- 172.21.2.101:0/2018037900 shutdown_connections 2021-02-10T11:14:37.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.698 7fa8a7bf0700 1 --2- 172.21.2.101:0/2018037900 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8a00cf580 0x7fa8a00cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:37.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.698 7fa8a7bf0700 1 --2- 172.21.2.101:0/2018037900 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8a00d1c70 0x7fa8a00cef60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:37.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.698 7fa8a7bf0700 1 -- 172.21.2.101:0/2018037900 >> 172.21.2.101:0/2018037900 conn(0x7fa8a00ce420 msgr2=0x7fa8a00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:37.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.698 7fa8a7bf0700 1 -- 172.21.2.101:0/2018037900 shutdown_connections 2021-02-10T11:14:37.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.698 7fa8a7bf0700 1 -- 172.21.2.101:0/2018037900 wait complete. 2021-02-10T11:14:37.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.713 7fa8a7bf0700 1 Processor -- start 2021-02-10T11:14:37.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.713 7fa8a7bf0700 1 -- start start 2021-02-10T11:14:37.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.713 7fa8a7bf0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8a0198c50 0x7fa8a0199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:37.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.713 7fa8a7bf0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8a0199b60 0x7fa8a018f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:37.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.713 7fa8a7bf0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8a018fe20 0x7fa8a0190220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:37.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.713 7fa8a7bf0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a018ef90 con 0x7fa8a0199b60 2021-02-10T11:14:37.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.713 7fa8a7bf0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a0190f30 con 0x7fa8a0198c50 2021-02-10T11:14:37.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.713 7fa8a7bf0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a0199f60 con 0x7fa8a018fe20 2021-02-10T11:14:37.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.713 7fa8a520b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8a0199b60 0x7fa8a018f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:37.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.713 7fa8a520b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8a0199b60 0x7fa8a018f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45136/0 (socket says 172.21.2.101:45136) 2021-02-10T11:14:37.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.713 7fa8a520b700 1 -- 172.21.2.101:0/2785733864 learned_addr learned my addr 172.21.2.101:0/2785733864 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:37.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.713 7fa8a520b700 1 -- 172.21.2.101:0/2785733864 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8a018fe20 msgr2=0x7fa8a0190220 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:37.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.713 7fa8a520b700 1 --2- 172.21.2.101:0/2785733864 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8a018fe20 0x7fa8a0190220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:37.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.713 7fa8a520b700 1 -- 172.21.2.101:0/2785733864 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8a0198c50 msgr2=0x7fa8a0199540 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:37.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.713 7fa8a520b700 1 --2- 172.21.2.101:0/2785733864 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8a0198c50 0x7fa8a0199540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:37.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.713 7fa8a520b700 1 -- 172.21.2.101:0/2785733864 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a0199050 con 0x7fa8a0199b60 2021-02-10T11:14:37.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.714 7fa8a520b700 1 --2- 172.21.2.101:0/2785733864 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8a0199b60 0x7fa8a018f800 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7fa89400b5e0 tx=0x7fa89400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:37.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.714 7fa8927fc700 1 -- 172.21.2.101:0/2785733864 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa89400bd70 con 0x7fa8a0199b60 2021-02-10T11:14:37.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.714 7fa8927fc700 1 -- 172.21.2.101:0/2785733864 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa894002030 con 0x7fa8a0199b60 2021-02-10T11:14:37.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.714 7fa8927fc700 1 -- 172.21.2.101:0/2785733864 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa894002550 con 0x7fa8a0199b60 2021-02-10T11:14:37.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.714 7fa8a7bf0700 1 -- 172.21.2.101:0/2785733864 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a0199050 con 0x7fa8a0199b60 2021-02-10T11:14:37.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.714 7fa8a7bf0700 1 -- 172.21.2.101:0/2785733864 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a0199050 con 0x7fa8a0199b60 2021-02-10T11:14:37.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.714 7fa8927fc700 1 -- 172.21.2.101:0/2785733864 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa894000c90 con 0x7fa8a0199b60 2021-02-10T11:14:37.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.715 7fa8927fc700 1 --2- 172.21.2.101:0/2785733864 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8880186c0 0x7fa88801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:37.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.715 7fa8927fc700 1 -- 172.21.2.101:0/2785733864 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5221+0+0 (secure 0 0 0) 0x7fa894032490 con 0x7fa8a0199b60 2021-02-10T11:14:37.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.715 7fa8a5a0c700 1 --2- 172.21.2.101:0/2785733864 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8880186c0 0x7fa88801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:37.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.715 7fa8a5a0c700 1 --2- 172.21.2.101:0/2785733864 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8880186c0 0x7fa88801abc0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:37.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.746 7fa8a7bf0700 1 -- 172.21.2.101:0/2785733864 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a00828e0 con 0x7fa8a0199b60 2021-02-10T11:14:37.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.748 7fa8927fc700 1 -- 172.21.2.101:0/2785733864 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa894030da0 con 0x7fa8a0199b60 2021-02-10T11:14:37.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:37.875 7fa8a7bf0700 1 -- 172.21.2.101:0/2785733864 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa8a00828e0 con 0x7fa8a0199b60 2021-02-10T11:14:38.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.230 7fa8927fc700 1 -- 172.21.2.101:0/2785733864 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v23) v1 ==== 173+0+0 (secure 0 0 0) 0x7fa894000c90 con 0x7fa8a0199b60 2021-02-10T11:14:38.246 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' removed 2021-02-10T11:14:38.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.244 7fa8a7bf0700 1 -- 172.21.2.101:0/2785733864 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8880186c0 msgr2=0x7fa88801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:38.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.244 7fa8a7bf0700 1 --2- 172.21.2.101:0/2785733864 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8880186c0 0x7fa88801abc0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:38.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.244 7fa8a7bf0700 1 -- 172.21.2.101:0/2785733864 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8a0199b60 msgr2=0x7fa8a018f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:38.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.244 7fa8a7bf0700 1 --2- 172.21.2.101:0/2785733864 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8a0199b60 0x7fa8a018f800 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7fa89400b5e0 tx=0x7fa89400b980).stop 2021-02-10T11:14:38.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.244 7fa8a7bf0700 1 -- 172.21.2.101:0/2785733864 shutdown_connections 2021-02-10T11:14:38.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.244 7fa8a7bf0700 1 --2- 172.21.2.101:0/2785733864 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8880186c0 0x7fa88801abc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:38.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.244 7fa8a7bf0700 1 --2- 172.21.2.101:0/2785733864 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8a018fe20 0x7fa8a0190220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:38.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.244 7fa8a7bf0700 1 --2- 172.21.2.101:0/2785733864 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8a0199b60 0x7fa8a018f800 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:38.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.244 7fa8a7bf0700 1 --2- 172.21.2.101:0/2785733864 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8a0198c50 0x7fa8a0199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:38.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.244 7fa8a7bf0700 1 -- 172.21.2.101:0/2785733864 >> 172.21.2.101:0/2785733864 conn(0x7fa8a019f820 msgr2=0x7fa8a01aafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:38.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.244 7fa8a7bf0700 1 -- 172.21.2.101:0/2785733864 shutdown_connections 2021-02-10T11:14:38.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.244 7fa8a7bf0700 1 -- 172.21.2.101:0/2785733864 wait complete. 2021-02-10T11:14:38.257 INFO:tasks.ceph.mds.a:Restarting daemon 2021-02-10T11:14:38.257 INFO:tasks.ceph.mds.a:Stopping old one... 2021-02-10T11:14:38.257 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-02-10T11:14:38.257 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:14:38.258 INFO:tasks.ceph.mds.b:Restarting daemon 2021-02-10T11:14:38.258 INFO:tasks.ceph.mds.b:Stopping old one... 2021-02-10T11:14:38.258 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-02-10T11:14:38.258 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:14:38.263 INFO:tasks.ceph.mds.a:Stopped 2021-02-10T11:14:38.264 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-02-10T11:14:38.265 INFO:tasks.ceph.mds.b:Stopped 2021-02-10T11:14:38.265 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-02-10T11:14:38.305 INFO:tasks.ceph.mds.a:Started 2021-02-10T11:14:38.306 DEBUG:teuthology.parallel:result is None 2021-02-10T11:14:38.306 INFO:tasks.ceph.mds.b:Started 2021-02-10T11:14:38.306 DEBUG:teuthology.parallel:result is None 2021-02-10T11:14:38.307 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd blacklist clear 2021-02-10T11:14:38.710 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:14:38.712 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:14:38.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.714 7f63dc921700 1 Processor -- start 2021-02-10T11:14:38.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.714 7f63dc921700 1 -- start start 2021-02-10T11:14:38.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.714 7f63dc921700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f63d807e6a0 0x7f63d80d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:38.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.714 7f63dc921700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d807ed80 con 0x7f63d80e5070 2021-02-10T11:14:38.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.714 7f63dc921700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d807eaa0 con 0x7f63d80e5470 2021-02-10T11:14:38.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.714 7f63dc921700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d81b4080 con 0x7f63d807e6a0 2021-02-10T11:14:38.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.714 7f63d661c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f63d80e5070 0x7f63d81b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45646/0 (socket says 172.21.2.101:45646) 2021-02-10T11:14:38.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.714 7f63d661c700 1 -- 172.21.2.101:0/519780779 learned_addr learned my addr 172.21.2.101:0/519780779 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:14:38.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.716 7f63d6e1d700 1 --2- 172.21.2.101:0/519780779 >> v2:172.21.2.108:3300/0 conn(0x7f63d807e6a0 0x7f63d80d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:38.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.716 7f63d561a700 1 -- 172.21.2.101:0/519780779 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3994880242 0 0) 0x7f63c8001180 con 0x7f63d80e5070 2021-02-10T11:14:38.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.716 7f63d561a700 1 -- 172.21.2.101:0/519780779 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63c4002fe0 con 0x7f63d80e5070 2021-02-10T11:14:38.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.716 7f63d561a700 1 -- 172.21.2.101:0/519780779 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 119136060 0 0) 0x7f63c8000ed0 con 0x7f63d80e5070 2021-02-10T11:14:38.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.716 7f63d6e1d700 1 -- 172.21.2.101:0/519780779 >> v1:172.21.2.108:6789/0 conn(0x7f63d80e5470 legacy=0x7f63d807f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:38.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.716 7f63d6e1d700 1 -- 172.21.2.101:0/519780779 >> v1:172.21.2.101:6789/0 conn(0x7f63d80e5070 legacy=0x7f63d81b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:38.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.716 7f63d6e1d700 1 -- 172.21.2.101:0/519780779 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d80d1db0 con 0x7f63d807e6a0 2021-02-10T11:14:38.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.716 7f63d6e1d700 1 --2- 172.21.2.101:0/519780779 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63d807e6a0 0x7f63d80d1790 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f63d0006e50 tx=0x7f63d0009e70).ready entity=mon.1 client_cookie=f384f843ef7e5b95 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:38.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.716 7f63d561a700 1 -- 172.21.2.101:0/519780779 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f63d000b510 con 0x7f63d807e6a0 2021-02-10T11:14:38.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.716 7f63d561a700 1 -- 172.21.2.101:0/519780779 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d0006790 con 0x7f63d807e6a0 2021-02-10T11:14:38.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.716 7f63d561a700 1 -- 172.21.2.101:0/519780779 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f63d000b940 con 0x7f63d807e6a0 2021-02-10T11:14:38.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.716 7f63dc921700 1 -- 172.21.2.101:0/519780779 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63d807e6a0 msgr2=0x7f63d80d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:38.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.716 7f63dc921700 1 --2- 172.21.2.101:0/519780779 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63d807e6a0 0x7f63d80d1790 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f63d0006e50 tx=0x7f63d0009e70).stop 2021-02-10T11:14:38.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.716 7f63dc921700 1 -- 172.21.2.101:0/519780779 shutdown_connections 2021-02-10T11:14:38.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.716 7f63dc921700 1 --2- 172.21.2.101:0/519780779 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63d807e6a0 0x7f63d80d1790 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:38.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.716 7f63dc921700 1 -- 172.21.2.101:0/519780779 >> 172.21.2.101:0/519780779 conn(0x7f63d80ce690 msgr2=0x7f63d80e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:38.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.716 7f63dc921700 1 -- 172.21.2.101:0/519780779 shutdown_connections 2021-02-10T11:14:38.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.716 7f63dc921700 1 -- 172.21.2.101:0/519780779 wait complete. 2021-02-10T11:14:38.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.731 7f63dc921700 1 Processor -- start 2021-02-10T11:14:38.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.731 7f63dc921700 1 -- start start 2021-02-10T11:14:38.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.731 7f63dc921700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63d80b2ca0 0x7f63d80b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:38.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.731 7f63dc921700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63d80b3150 0x7f63d80b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:38.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.731 7f63dc921700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63d80bb0a0 0x7f63d80bb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:38.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.731 7f63dc921700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d80ba910 con 0x7f63d80bb0a0 2021-02-10T11:14:38.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.731 7f63dc921700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d80bab20 con 0x7f63d80b3150 2021-02-10T11:14:38.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.731 7f63dc921700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d80b2720 con 0x7f63d80b2ca0 2021-02-10T11:14:38.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.731 7f63d6e1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63d80bb0a0 0x7f63d80bb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:38.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.731 7f63d6e1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63d80bb0a0 0x7f63d80bb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45160/0 (socket says 172.21.2.101:45160) 2021-02-10T11:14:38.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.731 7f63d6e1d700 1 -- 172.21.2.101:0/3024958035 learned_addr learned my addr 172.21.2.101:0/3024958035 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:38.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.731 7f63d661c700 1 --2- 172.21.2.101:0/3024958035 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63d80b2ca0 0x7f63d80b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:38.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.731 7f63d5e1b700 1 --2- 172.21.2.101:0/3024958035 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63d80b3150 0x7f63d80b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:38.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.732 7f63d6e1d700 1 -- 172.21.2.101:0/3024958035 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63d80b2ca0 msgr2=0x7f63d80b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:38.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.732 7f63d6e1d700 1 --2- 172.21.2.101:0/3024958035 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63d80b2ca0 0x7f63d80b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:38.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.732 7f63d6e1d700 1 -- 172.21.2.101:0/3024958035 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63d80b3150 msgr2=0x7f63d80b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:38.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.732 7f63d6e1d700 1 --2- 172.21.2.101:0/3024958035 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63d80b3150 0x7f63d80b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:38.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.732 7f63d6e1d700 1 -- 172.21.2.101:0/3024958035 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d80b2910 con 0x7f63d80bb0a0 2021-02-10T11:14:38.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.732 7f63d5e1b700 1 --2- 172.21.2.101:0/3024958035 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63d80b3150 0x7f63d80b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:14:38.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.732 7f63d6e1d700 1 --2- 172.21.2.101:0/3024958035 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63d80bb0a0 0x7f63d80bb4a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f63d0009a90 tx=0x7f63d000bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:38.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.732 7f63ceffd700 1 -- 172.21.2.101:0/3024958035 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f63d0002ef0 con 0x7f63d80bb0a0 2021-02-10T11:14:38.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.732 7f63ceffd700 1 -- 172.21.2.101:0/3024958035 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d0009270 con 0x7f63d80bb0a0 2021-02-10T11:14:38.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.732 7f63dc921700 1 -- 172.21.2.101:0/3024958035 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d80b2910 con 0x7f63d80bb0a0 2021-02-10T11:14:38.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.732 7f63ceffd700 1 -- 172.21.2.101:0/3024958035 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f63d000c4e0 con 0x7f63d80bb0a0 2021-02-10T11:14:38.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.732 7f63dc921700 1 -- 172.21.2.101:0/3024958035 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d80bbdf0 con 0x7f63d80bb0a0 2021-02-10T11:14:38.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.733 7f63ceffd700 1 -- 172.21.2.101:0/3024958035 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f63d0009450 con 0x7f63d80bb0a0 2021-02-10T11:14:38.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.733 7f63ceffd700 1 --2- 172.21.2.101:0/3024958035 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f63bc0186d0 0x7f63bc01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:38.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.733 7f63ceffd700 1 -- 172.21.2.101:0/3024958035 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4839+0+0 (secure 0 0 0) 0x7f63d0032710 con 0x7f63d80bb0a0 2021-02-10T11:14:38.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.733 7f63d661c700 1 --2- 172.21.2.101:0/3024958035 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f63bc0186d0 0x7f63bc01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:38.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.733 7f63d661c700 1 --2- 172.21.2.101:0/3024958035 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f63bc0186d0 0x7f63bc01abd0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:38.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.764 7f63dc921700 1 -- 172.21.2.101:0/3024958035 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d81071c0 con 0x7f63d80bb0a0 2021-02-10T11:14:38.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.767 7f63ceffd700 1 -- 172.21.2.101:0/3024958035 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f63d000be30 con 0x7f63d80bb0a0 2021-02-10T11:14:38.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:38.905 7f63dc921700 1 -- 172.21.2.101:0/3024958035 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f63d807ed80 con 0x7f63d80bb0a0 2021-02-10T11:14:39.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.231 7f63ceffd700 1 -- 172.21.2.101:0/3024958035 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v24) v1 ==== 97+0+0 (secure 0 0 0) 0x7f63d0009170 con 0x7f63d80bb0a0 2021-02-10T11:14:39.276 INFO:teuthology.orchestra.run.gibba001.stderr: removed all blacklist entries 2021-02-10T11:14:39.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.273 7f63dc921700 1 -- 172.21.2.101:0/3024958035 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f63bc0186d0 msgr2=0x7f63bc01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:39.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.273 7f63dc921700 1 --2- 172.21.2.101:0/3024958035 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f63bc0186d0 0x7f63bc01abd0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:39.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.273 7f63dc921700 1 -- 172.21.2.101:0/3024958035 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63d80bb0a0 msgr2=0x7f63d80bb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:39.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.273 7f63dc921700 1 --2- 172.21.2.101:0/3024958035 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63d80bb0a0 0x7f63d80bb4a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f63d0009a90 tx=0x7f63d000bcf0).stop 2021-02-10T11:14:39.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.274 7f63dc921700 1 -- 172.21.2.101:0/3024958035 shutdown_connections 2021-02-10T11:14:39.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.274 7f63dc921700 1 --2- 172.21.2.101:0/3024958035 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63d80bb0a0 0x7f63d80bb4a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:39.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.274 7f63dc921700 1 --2- 172.21.2.101:0/3024958035 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63d80b3150 0x7f63d80b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:39.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.274 7f63dc921700 1 --2- 172.21.2.101:0/3024958035 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f63bc0186d0 0x7f63bc01abd0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:39.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.274 7f63dc921700 1 --2- 172.21.2.101:0/3024958035 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63d80b2ca0 0x7f63d80b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:39.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.274 7f63dc921700 1 -- 172.21.2.101:0/3024958035 >> 172.21.2.101:0/3024958035 conn(0x7f63d80bca20 msgr2=0x7f63d8098170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:39.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.274 7f63dc921700 1 -- 172.21.2.101:0/3024958035 shutdown_connections 2021-02-10T11:14:39.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.274 7f63dc921700 1 -- 172.21.2.101:0/3024958035 wait complete. 2021-02-10T11:14:39.287 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2021-02-10T11:14:39.287 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:14:39.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.691 7f6ce22b4700 1 Processor -- start 2021-02-10T11:14:39.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.691 7f6ce22b4700 1 -- start start 2021-02-10T11:14:39.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.691 7f6ce22b4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6cdc0e5070 0x7f6cdc1b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:39.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.691 7f6ce22b4700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6cdc1b3e40 0x7f6cdc07d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:39.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.692 7f6ce22b4700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cdc07ed80 con 0x7f6cdc0e5470 2021-02-10T11:14:39.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.692 7f6ce22b4700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cdc080310 con 0x7f6cdc0e5070 2021-02-10T11:14:39.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.692 7f6ce22b4700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cdc080530 con 0x7f6cdc1b3e40 2021-02-10T11:14:39.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.692 7f6cdbfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6cdc0e5070 0x7f6cdc1b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:39.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.692 7f6cdbfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6cdc0e5070 0x7f6cdc1b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45166/0 (socket says 172.21.2.101:45166) 2021-02-10T11:14:39.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.692 7f6cdbfff700 1 -- 172.21.2.101:0/159396471 learned_addr learned my addr 172.21.2.101:0/159396471 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:39.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.692 7f6cdbfff700 1 -- 172.21.2.101:0/159396471 >> v2:172.21.2.108:3300/0 conn(0x7f6cdc1b3e40 msgr2=0x7f6cdc07d690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:39.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.692 7f6cdbfff700 1 --2- 172.21.2.101:0/159396471 >> v2:172.21.2.108:3300/0 conn(0x7f6cdc1b3e40 0x7f6cdc07d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:39.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.692 7f6cdbfff700 1 -- 172.21.2.101:0/159396471 >> v1:172.21.2.108:6789/0 conn(0x7f6cdc0e5470 legacy=0x7f6cdc07dcb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:39.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.692 7f6cdbfff700 1 -- 172.21.2.101:0/159396471 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc080750 con 0x7f6cdc0e5070 2021-02-10T11:14:39.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.692 7f6cdbfff700 1 --2- 172.21.2.101:0/159396471 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cdc0e5070 0x7f6cdc1b3820 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f6ccc006700 tx=0x7f6ccc009b80).ready entity=mon.0 client_cookie=35e9cf02ba851a78 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:39.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.692 7f6cdaffd700 1 -- 172.21.2.101:0/159396471 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ccc00c0c0 con 0x7f6cdc0e5070 2021-02-10T11:14:39.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.692 7f6cdaffd700 1 -- 172.21.2.101:0/159396471 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ccc00c340 con 0x7f6cdc0e5070 2021-02-10T11:14:39.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.692 7f6cdaffd700 1 -- 172.21.2.101:0/159396471 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ccc00c7a0 con 0x7f6cdc0e5070 2021-02-10T11:14:39.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.692 7f6ce22b4700 1 -- 172.21.2.101:0/159396471 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cdc0e5070 msgr2=0x7f6cdc1b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:39.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.692 7f6ce22b4700 1 --2- 172.21.2.101:0/159396471 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cdc0e5070 0x7f6cdc1b3820 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f6ccc006700 tx=0x7f6ccc009b80).stop 2021-02-10T11:14:39.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.693 7f6ce22b4700 1 -- 172.21.2.101:0/159396471 shutdown_connections 2021-02-10T11:14:39.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.693 7f6ce22b4700 1 --2- 172.21.2.101:0/159396471 >> v2:172.21.2.108:3300/0 conn(0x7f6cdc1b3e40 0x7f6cdc07d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:39.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.693 7f6ce22b4700 1 --2- 172.21.2.101:0/159396471 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cdc0e5070 0x7f6cdc1b3820 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:39.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.693 7f6ce22b4700 1 -- 172.21.2.101:0/159396471 >> 172.21.2.101:0/159396471 conn(0x7f6cdc0ce690 msgr2=0x7f6cdc0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:39.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.693 7f6ce22b4700 1 -- 172.21.2.101:0/159396471 shutdown_connections 2021-02-10T11:14:39.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.693 7f6ce22b4700 1 -- 172.21.2.101:0/159396471 wait complete. 2021-02-10T11:14:39.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.708 7f6ce22b4700 1 Processor -- start 2021-02-10T11:14:39.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.708 7f6ce22b4700 1 -- start start 2021-02-10T11:14:39.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.708 7f6ce22b4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cdc1aad40 0x7f6cdc192110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:39.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.708 7f6ce22b4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6cdc1a1aa0 0x7f6cdc0b7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:39.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.708 7f6ce22b4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6cdc192730 0x7f6cdc0b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:39.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.708 7f6ce22b4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc192d70 con 0x7f6cdc1aad40 2021-02-10T11:14:39.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.708 7f6ce22b4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc1a1590 con 0x7f6cdc1a1aa0 2021-02-10T11:14:39.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.708 7f6ce22b4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc1a11b0 con 0x7f6cdc192730 2021-02-10T11:14:39.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.708 7f6cdbfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cdc1aad40 0x7f6cdc192110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:39.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.708 7f6cdbfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cdc1aad40 0x7f6cdc192110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45172/0 (socket says 172.21.2.101:45172) 2021-02-10T11:14:39.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.708 7f6cdbfff700 1 -- 172.21.2.101:0/677521703 learned_addr learned my addr 172.21.2.101:0/677521703 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:39.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.708 7f6ce08d1700 1 --2- 172.21.2.101:0/677521703 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6cdc192730 0x7f6cdc0b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:39.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.708 7f6cdb7fe700 1 --2- 172.21.2.101:0/677521703 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6cdc1a1aa0 0x7f6cdc0b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:39.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.709 7f6cdbfff700 1 -- 172.21.2.101:0/677521703 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6cdc192730 msgr2=0x7f6cdc0b7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:39.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.709 7f6cdbfff700 1 --2- 172.21.2.101:0/677521703 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6cdc192730 0x7f6cdc0b7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:39.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.709 7f6cdbfff700 1 -- 172.21.2.101:0/677521703 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6cdc1a1aa0 msgr2=0x7f6cdc0b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:39.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.709 7f6cdbfff700 1 --2- 172.21.2.101:0/677521703 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6cdc1a1aa0 0x7f6cdc0b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:39.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.709 7f6cdbfff700 1 -- 172.21.2.101:0/677521703 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc0b8170 con 0x7f6cdc1aad40 2021-02-10T11:14:39.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.709 7f6cdbfff700 1 --2- 172.21.2.101:0/677521703 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cdc1aad40 0x7f6cdc192110 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f6ccc00b580 tx=0x7f6ccc00b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:39.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.709 7f6cd8ff9700 1 -- 172.21.2.101:0/677521703 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ccc00bd10 con 0x7f6cdc1aad40 2021-02-10T11:14:39.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.709 7f6cd8ff9700 1 -- 172.21.2.101:0/677521703 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ccc006500 con 0x7f6cdc1aad40 2021-02-10T11:14:39.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.709 7f6ce22b4700 1 -- 172.21.2.101:0/677521703 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cdc0b8170 con 0x7f6cdc1aad40 2021-02-10T11:14:39.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.709 7f6cd8ff9700 1 -- 172.21.2.101:0/677521703 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ccc002220 con 0x7f6cdc1aad40 2021-02-10T11:14:39.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.709 7f6ce22b4700 1 -- 172.21.2.101:0/677521703 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cdc1935e0 con 0x7f6cdc1aad40 2021-02-10T11:14:39.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.710 7f6cd8ff9700 1 -- 172.21.2.101:0/677521703 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6ccc0052e0 con 0x7f6cdc1aad40 2021-02-10T11:14:39.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.710 7f6cd8ff9700 1 --2- 172.21.2.101:0/677521703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6cbc0186c0 0x7f6cbc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:39.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.710 7f6cd8ff9700 1 -- 172.21.2.101:0/677521703 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f6ccc032130 con 0x7f6cdc1aad40 2021-02-10T11:14:39.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.710 7f6cdb7fe700 1 --2- 172.21.2.101:0/677521703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6cbc0186c0 0x7f6cbc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:39.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.710 7f6cdb7fe700 1 --2- 172.21.2.101:0/677521703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6cbc0186c0 0x7f6cbc01abc0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:39.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.742 7f6ce22b4700 1 -- 172.21.2.101:0/677521703 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cdc083da0 con 0x7f6cdc1aad40 2021-02-10T11:14:39.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.745 7f6cd8ff9700 1 -- 172.21.2.101:0/677521703 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6ccc0300b0 con 0x7f6cdc1aad40 2021-02-10T11:14:39.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.877 7f6ce22b4700 1 -- 172.21.2.101:0/677521703 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6cdc07ed80 con 0x7f6cdc1aad40 2021-02-10T11:14:39.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.877 7f6cd8ff9700 1 -- 172.21.2.101:0/677521703 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+5917 (secure 0 0 0) 0x7f6ccc0300b0 con 0x7f6cdc1aad40 2021-02-10T11:14:39.882 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:14:39.882 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:14:39.882 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:14:39.882 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.882 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:14:39.883 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:14:39.883 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.883 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:14:39.883 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:14:39.883 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:14:39.883 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:14:39.883 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.883 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:39.883 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.883 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:14:39.883 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:14:39.883 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.884 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:14:39.884 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:14:39.884 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:14:39.884 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:14:39.884 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.884 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:39.884 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.884 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:14:39.884 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:14:39.884 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.884 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:14:39.884 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:14:39.884 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:14:39.884 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.885 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:39.885 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.885 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:14:39.885 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:14:39.885 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.885 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:14:39.885 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:14:39.885 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:14:39.885 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.885 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:39.885 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.885 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:14:39.886 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:14:39.886 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.886 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:14:39.886 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:14:39.886 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:14:39.886 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.886 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:39.886 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.886 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:14:39.886 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:14:39.886 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.886 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:14:39.886 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:14:39.887 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:14:39.887 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.887 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:39.887 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.887 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:14:39.887 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:14:39.887 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.887 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:14:39.887 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:14:39.887 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:14:39.887 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.887 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:39.887 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.888 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:14:39.888 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:14:39.888 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.888 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:14:39.888 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:14:39.888 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:14:39.888 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.888 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:39.888 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.888 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:14:39.888 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:14:39.889 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.889 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:14:39.889 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:14:39.889 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:14:39.889 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.889 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:39.889 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.889 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:14:39.889 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:14:39.889 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.890 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:14:39.890 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:14:39.890 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:14:39.890 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.890 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:39.890 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.890 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:14:39.890 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:14:39.890 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.890 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:14:39.890 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow r", 2021-02-10T11:14:39.890 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow rw", 2021-02-10T11:14:39.891 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rwx" 2021-02-10T11:14:39.891 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.891 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:39.891 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.891 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:14:39.891 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:14:39.891 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.891 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:14:39.891 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow r", 2021-02-10T11:14:39.891 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow rw", 2021-02-10T11:14:39.891 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rwx" 2021-02-10T11:14:39.891 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.891 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:39.891 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.892 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:14:39.892 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:14:39.892 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.892 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:14:39.892 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow r", 2021-02-10T11:14:39.892 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow rw", 2021-02-10T11:14:39.892 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rwx" 2021-02-10T11:14:39.892 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.892 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:39.892 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.892 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:14:39.892 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:14:39.892 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.893 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:14:39.893 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow r", 2021-02-10T11:14:39.893 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow rw", 2021-02-10T11:14:39.893 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rwx" 2021-02-10T11:14:39.893 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.893 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:39.893 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.893 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:14:39.893 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:14:39.893 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.893 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:14:39.893 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:14:39.893 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:14:39.894 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:14:39.894 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.894 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:39.894 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.894 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.bootstrap-mds", 2021-02-10T11:14:39.894 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQCkvyNgctgBLxAAs3vtjHAWDDOUDrqoYlKuZw==", 2021-02-10T11:14:39.894 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.894 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile bootstrap-mds" 2021-02-10T11:14:39.894 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.894 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:39.894 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.894 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.bootstrap-mgr", 2021-02-10T11:14:39.894 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQCkvyNgluEBLxAAAuSbdlAACz2EB299J2bUHQ==", 2021-02-10T11:14:39.895 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.895 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile bootstrap-mgr" 2021-02-10T11:14:39.895 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.895 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:39.895 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.895 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.bootstrap-osd", 2021-02-10T11:14:39.895 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQCkvyNgs+oBLxAAvUumoxLmO8gnjjGihTpIjA==", 2021-02-10T11:14:39.895 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.895 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile bootstrap-osd" 2021-02-10T11:14:39.895 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.895 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:39.895 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.895 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.bootstrap-rbd", 2021-02-10T11:14:39.896 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQCkvyNgqPMBLxAAwqOdShCG/O3V2Z8RtBHBHg==", 2021-02-10T11:14:39.896 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.896 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile bootstrap-rbd" 2021-02-10T11:14:39.896 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.896 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:39.896 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.896 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.bootstrap-rbd-mirror", 2021-02-10T11:14:39.896 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQCkvyNg7vwBLxAAD19LHXeLG+FaY+vjJg7m1w==", 2021-02-10T11:14:39.896 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.896 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2021-02-10T11:14:39.896 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.896 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:39.896 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.897 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.bootstrap-rgw", 2021-02-10T11:14:39.897 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQCkvyNgBgYCLxAAETMUKcn9sI5nOywiAzuYWg==", 2021-02-10T11:14:39.897 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.897 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile bootstrap-rgw" 2021-02-10T11:14:39.897 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.897 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:39.897 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.897 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:14:39.897 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:14:39.897 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.897 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:14:39.897 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:14:39.898 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:14:39.898 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.898 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:14:39.898 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:39.898 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:14:39.898 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:14:39.898 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:14:39.898 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:14:39.898 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:14:39.898 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:14:39.898 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.898 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:39.898 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:14:39.899 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:14:39.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.879 7f6ce22b4700 1 -- 172.21.2.101:0/677521703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6cbc0186c0 msgr2=0x7f6cbc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:39.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.879 7f6ce22b4700 1 --2- 172.21.2.101:0/677521703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6cbc0186c0 0x7f6cbc01abc0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:39.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.879 7f6ce22b4700 1 -- 172.21.2.101:0/677521703 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cdc1aad40 msgr2=0x7f6cdc192110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:39.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.879 7f6ce22b4700 1 --2- 172.21.2.101:0/677521703 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cdc1aad40 0x7f6cdc192110 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f6ccc00b580 tx=0x7f6ccc00b920).stop 2021-02-10T11:14:39.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.879 7f6ce22b4700 1 -- 172.21.2.101:0/677521703 shutdown_connections 2021-02-10T11:14:39.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.879 7f6ce22b4700 1 --2- 172.21.2.101:0/677521703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6cbc0186c0 0x7f6cbc01abc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:39.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.879 7f6ce22b4700 1 --2- 172.21.2.101:0/677521703 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6cdc192730 0x7f6cdc0b7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:39.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.879 7f6ce22b4700 1 --2- 172.21.2.101:0/677521703 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6cdc1a1aa0 0x7f6cdc0b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:39.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.879 7f6ce22b4700 1 --2- 172.21.2.101:0/677521703 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cdc1aad40 0x7f6cdc192110 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:39.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.879 7f6ce22b4700 1 -- 172.21.2.101:0/677521703 >> 172.21.2.101:0/677521703 conn(0x7f6cdc1a2470 msgr2=0x7f6cdc194f50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:39.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.879 7f6ce22b4700 1 -- 172.21.2.101:0/677521703 shutdown_connections 2021-02-10T11:14:39.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:39.879 7f6ce22b4700 1 -- 172.21.2.101:0/677521703 wait complete. 2021-02-10T11:14:39.901 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth del client.bootstrap-mds 2021-02-10T11:14:40.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.306 7fcd94910700 1 Processor -- start 2021-02-10T11:14:40.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.306 7fcd94910700 1 -- start start 2021-02-10T11:14:40.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.307 7fcd94910700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fcd900e5070 0x7fcd901b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:40.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.307 7fcd94910700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd9007ed80 con 0x7fcd900e5470 2021-02-10T11:14:40.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.307 7fcd94910700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd9007e670 con 0x7fcd901b3e40 2021-02-10T11:14:40.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.307 7fcd94910700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd9007e890 con 0x7fcd900e5070 2021-02-10T11:14:40.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.307 7fcd8ee1d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fcd900e5470 0x7fcd9007de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45674/0 (socket says 172.21.2.101:45674) 2021-02-10T11:14:40.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.307 7fcd8ee1d700 1 -- 172.21.2.101:0/1269338116 learned_addr learned my addr 172.21.2.101:0/1269338116 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:14:40.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.307 7fcd8d61a700 1 -- 172.21.2.101:0/1269338116 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997146542 0 0) 0x7fcd88001500 con 0x7fcd900e5470 2021-02-10T11:14:40.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.307 7fcd8d61a700 1 -- 172.21.2.101:0/1269338116 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd7c002fe0 con 0x7fcd900e5470 2021-02-10T11:14:40.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.307 7fcd8d61a700 1 -- 172.21.2.101:0/1269338116 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3836138832 0 0) 0x7fcd88001460 con 0x7fcd900e5470 2021-02-10T11:14:40.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.307 7fcd8d61a700 1 -- 172.21.2.101:0/1269338116 >> v2:172.21.2.108:3300/0 conn(0x7fcd900e5070 msgr2=0x7fcd901b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:40.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.307 7fcd8d61a700 1 --2- 172.21.2.101:0/1269338116 >> v2:172.21.2.108:3300/0 conn(0x7fcd900e5070 0x7fcd901b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:40.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.307 7fcd8d61a700 1 -- 172.21.2.101:0/1269338116 >> v1:172.21.2.108:6789/0 conn(0x7fcd901b3e40 legacy=0x7fcd9007d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:40.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.307 7fcd8d61a700 1 -- 172.21.2.101:0/1269338116 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9007eab0 con 0x7fcd900e5470 2021-02-10T11:14:40.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.307 7fcd8d61a700 1 -- 172.21.2.101:0/1269338116 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fcd88001980 con 0x7fcd900e5470 2021-02-10T11:14:40.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.307 7fcd8d61a700 1 -- 172.21.2.101:0/1269338116 >> v1:172.21.2.101:6789/0 conn(0x7fcd900e5470 legacy=0x7fcd9007de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:40.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.307 7fcd8d61a700 1 --2- 172.21.2.101:0/1269338116 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd7c003ac0 0x7fcd7c005ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:40.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.308 7fcd8d61a700 1 --2- 172.21.2.101:0/1269338116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd7c006770 0x7fcd7c008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:40.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.308 7fcd8d61a700 1 --2- 172.21.2.101:0/1269338116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd7c009590 0x7fcd7c00bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:40.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.308 7fcd8d61a700 1 -- 172.21.2.101:0/1269338116 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c00c300 con 0x7fcd7c006770 2021-02-10T11:14:40.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.308 7fcd8d61a700 1 -- 172.21.2.101:0/1269338116 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c00c520 con 0x7fcd7c003ac0 2021-02-10T11:14:40.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.308 7fcd8d61a700 1 -- 172.21.2.101:0/1269338116 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c00c740 con 0x7fcd7c009590 2021-02-10T11:14:40.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.308 7fcd8de1b700 1 --2- 172.21.2.101:0/1269338116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd7c006770 0x7fcd7c008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:40.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.308 7fcd8ee1d700 1 --2- 172.21.2.101:0/1269338116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd7c009590 0x7fcd7c00bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:40.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.308 7fcd8e61c700 1 --2- 172.21.2.101:0/1269338116 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd7c003ac0 0x7fcd7c005ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:40.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.308 7fcd8de1b700 1 -- 172.21.2.101:0/1269338116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd7c009590 msgr2=0x7fcd7c00bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:40.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.308 7fcd8de1b700 1 --2- 172.21.2.101:0/1269338116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd7c009590 0x7fcd7c00bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:40.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.308 7fcd8de1b700 1 -- 172.21.2.101:0/1269338116 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd7c003ac0 msgr2=0x7fcd7c005ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:40.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.308 7fcd8de1b700 1 --2- 172.21.2.101:0/1269338116 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd7c003ac0 0x7fcd7c005ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:40.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.308 7fcd8de1b700 1 -- 172.21.2.101:0/1269338116 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c00c9e0 con 0x7fcd7c006770 2021-02-10T11:14:40.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.308 7fcd8ee1d700 1 -- 172.21.2.101:0/1269338116 reap_dead start 2021-02-10T11:14:40.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.308 7fcd8de1b700 1 --2- 172.21.2.101:0/1269338116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd7c006770 0x7fcd7c008c90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fcd80006ab0 tx=0x7fcd80009b40).ready entity=mon.0 client_cookie=636aae94542de100 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:40.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.308 7fcd8d61a700 1 -- 172.21.2.101:0/1269338116 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd800066a0 con 0x7fcd7c006770 2021-02-10T11:14:40.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.308 7fcd8d61a700 1 -- 172.21.2.101:0/1269338116 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd80000e20 con 0x7fcd7c006770 2021-02-10T11:14:40.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.308 7fcd8d61a700 1 -- 172.21.2.101:0/1269338116 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd8000b4d0 con 0x7fcd7c006770 2021-02-10T11:14:40.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.308 7fcd94910700 1 -- 172.21.2.101:0/1269338116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd7c006770 msgr2=0x7fcd7c008c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:40.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.308 7fcd94910700 1 --2- 172.21.2.101:0/1269338116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd7c006770 0x7fcd7c008c90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fcd80006ab0 tx=0x7fcd80009b40).stop 2021-02-10T11:14:40.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.309 7fcd94910700 1 -- 172.21.2.101:0/1269338116 shutdown_connections 2021-02-10T11:14:40.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.309 7fcd94910700 1 --2- 172.21.2.101:0/1269338116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd7c006770 0x7fcd7c008c90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:40.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.309 7fcd94910700 1 -- 172.21.2.101:0/1269338116 >> 172.21.2.101:0/1269338116 conn(0x7fcd900ce690 msgr2=0x7fcd900e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:40.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.309 7fcd94910700 1 -- 172.21.2.101:0/1269338116 shutdown_connections 2021-02-10T11:14:40.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.309 7fcd94910700 1 -- 172.21.2.101:0/1269338116 wait complete. 2021-02-10T11:14:40.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.323 7fcd94910700 1 Processor -- start 2021-02-10T11:14:40.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.323 7fcd94910700 1 -- start start 2021-02-10T11:14:40.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.324 7fcd94910700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd9008f500 0x7fcd9019f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:40.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.324 7fcd94910700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd9008fd90 0x7fcd90090190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:40.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.324 7fcd94910700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd900907b0 0x7fcd90094370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:40.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.324 7fcd94910700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9008f900 con 0x7fcd900907b0 2021-02-10T11:14:40.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.324 7fcd94910700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9019ff20 con 0x7fcd9008f500 2021-02-10T11:14:40.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.324 7fcd94910700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd901a0140 con 0x7fcd9008fd90 2021-02-10T11:14:40.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.324 7fcd8ee1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd900907b0 0x7fcd90094370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:40.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.324 7fcd8ee1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd900907b0 0x7fcd90094370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45190/0 (socket says 172.21.2.101:45190) 2021-02-10T11:14:40.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.324 7fcd8ee1d700 1 -- 172.21.2.101:0/3968121352 learned_addr learned my addr 172.21.2.101:0/3968121352 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:40.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.324 7fcd8ee1d700 1 -- 172.21.2.101:0/3968121352 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd9008fd90 msgr2=0x7fcd90090190 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:40.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.324 7fcd8ee1d700 1 --2- 172.21.2.101:0/3968121352 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd9008fd90 0x7fcd90090190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:40.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.324 7fcd8ee1d700 1 -- 172.21.2.101:0/3968121352 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd9008f500 msgr2=0x7fcd9019f900 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:40.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.324 7fcd8ee1d700 1 --2- 172.21.2.101:0/3968121352 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd9008f500 0x7fcd9019f900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:40.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.324 7fcd8ee1d700 1 -- 172.21.2.101:0/3968121352 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd90094990 con 0x7fcd900907b0 2021-02-10T11:14:40.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.324 7fcd8ee1d700 1 --2- 172.21.2.101:0/3968121352 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd900907b0 0x7fcd90094370 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fcd88001640 tx=0x7fcd8804eca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:40.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.324 7fcd86ffd700 1 -- 172.21.2.101:0/3968121352 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd88001f60 con 0x7fcd900907b0 2021-02-10T11:14:40.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.324 7fcd86ffd700 1 -- 172.21.2.101:0/3968121352 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd88006030 con 0x7fcd900907b0 2021-02-10T11:14:40.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.324 7fcd86ffd700 1 -- 172.21.2.101:0/3968121352 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd88006490 con 0x7fcd900907b0 2021-02-10T11:14:40.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.324 7fcd94910700 1 -- 172.21.2.101:0/3968121352 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd90094990 con 0x7fcd900907b0 2021-02-10T11:14:40.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.325 7fcd94910700 1 -- 172.21.2.101:0/3968121352 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd90094f20 con 0x7fcd900907b0 2021-02-10T11:14:40.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.325 7fcd86ffd700 1 -- 172.21.2.101:0/3968121352 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fcd88006420 con 0x7fcd900907b0 2021-02-10T11:14:40.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.325 7fcd86ffd700 1 --2- 172.21.2.101:0/3968121352 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcd740186c0 0x7fcd7401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:40.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.325 7fcd86ffd700 1 -- 172.21.2.101:0/3968121352 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fcd880765c0 con 0x7fcd900907b0 2021-02-10T11:14:40.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.325 7fcd8e61c700 1 --2- 172.21.2.101:0/3968121352 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcd740186c0 0x7fcd7401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:40.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.325 7fcd8e61c700 1 --2- 172.21.2.101:0/3968121352 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcd740186c0 0x7fcd7401abc0 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:40.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.356 7fcd94910700 1 -- 172.21.2.101:0/3968121352 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd90093e40 con 0x7fcd900907b0 2021-02-10T11:14:40.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.360 7fcd86ffd700 1 -- 172.21.2.101:0/3968121352 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fcd88006210 con 0x7fcd900907b0 2021-02-10T11:14:40.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.491 7fcd94910700 1 -- 172.21.2.101:0/3968121352 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fcd900957b0 con 0x7fcd900907b0 2021-02-10T11:14:40.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.494 7fcd86ffd700 1 -- 172.21.2.101:0/3968121352 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 updated v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcd88006420 con 0x7fcd900907b0 2021-02-10T11:14:40.498 INFO:teuthology.orchestra.run.gibba001.stderr:updated 2021-02-10T11:14:40.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.495 7fcd94910700 1 -- 172.21.2.101:0/3968121352 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcd740186c0 msgr2=0x7fcd7401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:40.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.495 7fcd94910700 1 --2- 172.21.2.101:0/3968121352 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcd740186c0 0x7fcd7401abc0 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:40.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.495 7fcd94910700 1 -- 172.21.2.101:0/3968121352 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd900907b0 msgr2=0x7fcd90094370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:40.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.495 7fcd94910700 1 --2- 172.21.2.101:0/3968121352 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd900907b0 0x7fcd90094370 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fcd88001640 tx=0x7fcd8804eca0).stop 2021-02-10T11:14:40.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.495 7fcd94910700 1 -- 172.21.2.101:0/3968121352 shutdown_connections 2021-02-10T11:14:40.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.495 7fcd94910700 1 --2- 172.21.2.101:0/3968121352 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcd740186c0 0x7fcd7401abc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:40.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.495 7fcd94910700 1 --2- 172.21.2.101:0/3968121352 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd9008fd90 0x7fcd90090190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:40.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.495 7fcd94910700 1 --2- 172.21.2.101:0/3968121352 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd900907b0 0x7fcd90094370 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:40.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.495 7fcd94910700 1 --2- 172.21.2.101:0/3968121352 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd9008f500 0x7fcd9019f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:40.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.495 7fcd94910700 1 -- 172.21.2.101:0/3968121352 >> 172.21.2.101:0/3968121352 conn(0x7fcd900b5560 msgr2=0x7fcd900a6200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:40.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.496 7fcd94910700 1 -- 172.21.2.101:0/3968121352 shutdown_connections 2021-02-10T11:14:40.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.496 7fcd94910700 1 -- 172.21.2.101:0/3968121352 wait complete. 2021-02-10T11:14:40.508 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth del client.bootstrap-mgr 2021-02-10T11:14:40.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.911 7f03560ab700 1 Processor -- start 2021-02-10T11:14:40.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.911 7f03560ab700 1 -- start start 2021-02-10T11:14:40.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.912 7f03560ab700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f03500e5470 0x7f035007e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:40.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.912 7f03560ab700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f035007f300 0x7f035007f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:40.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.912 7f03560ab700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f035007ed80 con 0x7f03500e5070 2021-02-10T11:14:40.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.912 7f03560ab700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03501b4010 con 0x7f035007f300 2021-02-10T11:14:40.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.912 7f03560ab700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03500d1790 con 0x7f03500e5470 2021-02-10T11:14:40.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.912 7f034ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f035007f300 0x7f035007f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.912 7f034ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f035007f300 0x7f035007f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45200/0 (socket says 172.21.2.101:45200) 2021-02-10T11:14:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.912 7f034f7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f03500e5070 0x7f03501b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45690/0 (socket says 172.21.2.101:45690) 2021-02-10T11:14:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.912 7f034ffff700 1 -- 172.21.2.101:0/3004816162 learned_addr learned my addr 172.21.2.101:0/3004816162 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.912 7f034ffff700 1 -- 172.21.2.101:0/3004816162 >> v2:172.21.2.108:3300/0 conn(0x7f03500e5470 msgr2=0x7f035007e6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.912 7f034ffff700 1 --2- 172.21.2.101:0/3004816162 >> v2:172.21.2.108:3300/0 conn(0x7f03500e5470 0x7f035007e6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.912 7f034ffff700 1 -- 172.21.2.101:0/3004816162 >> v1:172.21.2.101:6789/0 conn(0x7f03500e5070 legacy=0x7f03501b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.912 7f034ffff700 1 -- 172.21.2.101:0/3004816162 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03500d19b0 con 0x7f035007f300 2021-02-10T11:14:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.912 7f034ffff700 1 --2- 172.21.2.101:0/3004816162 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f035007f300 0x7f035007f700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f0348006f00 tx=0x7f0348009f10).ready entity=mon.0 client_cookie=d47b4b4d832fd758 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.913 7f034e7fc700 1 -- 172.21.2.101:0/3004816162 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f034800b3b0 con 0x7f035007f300 2021-02-10T11:14:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.913 7f034e7fc700 1 -- 172.21.2.101:0/3004816162 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034800b670 con 0x7f035007f300 2021-02-10T11:14:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.913 7f034e7fc700 1 -- 172.21.2.101:0/3004816162 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f034800bb90 con 0x7f035007f300 2021-02-10T11:14:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.913 7f03560ab700 1 -- 172.21.2.101:0/3004816162 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f035007f300 msgr2=0x7f035007f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.913 7f03560ab700 1 --2- 172.21.2.101:0/3004816162 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f035007f300 0x7f035007f700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f0348006f00 tx=0x7f0348009f10).stop 2021-02-10T11:14:40.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.913 7f03560ab700 1 -- 172.21.2.101:0/3004816162 shutdown_connections 2021-02-10T11:14:40.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.913 7f03560ab700 1 --2- 172.21.2.101:0/3004816162 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f035007f300 0x7f035007f700 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:40.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.913 7f03560ab700 1 --2- 172.21.2.101:0/3004816162 >> v2:172.21.2.108:3300/0 conn(0x7f03500e5470 0x7f035007e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:40.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.913 7f03560ab700 1 -- 172.21.2.101:0/3004816162 >> 172.21.2.101:0/3004816162 conn(0x7f03500ce690 msgr2=0x7f03500e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:40.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.913 7f03560ab700 1 -- 172.21.2.101:0/3004816162 shutdown_connections 2021-02-10T11:14:40.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.913 7f03560ab700 1 -- 172.21.2.101:0/3004816162 wait complete. 2021-02-10T11:14:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.928 7f03560ab700 1 Processor -- start 2021-02-10T11:14:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.928 7f03560ab700 1 -- start start 2021-02-10T11:14:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.928 7f03560ab700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f03500c15b0 0x7f0350088eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.928 7f03560ab700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f03500c19b0 0x7f03500894d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.928 7f03560ab700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f03500c2470 0x7f03500c2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.928 7f03560ab700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03500c1ef0 con 0x7f03500c19b0 2021-02-10T11:14:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.928 7f03560ab700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03500b2760 con 0x7f03500c15b0 2021-02-10T11:14:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.928 7f03560ab700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0350089af0 con 0x7f03500c2470 2021-02-10T11:14:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.929 7f034effd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f03500c19b0 0x7f03500894d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.929 7f034effd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f03500c19b0 0x7f03500894d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45206/0 (socket says 172.21.2.101:45206) 2021-02-10T11:14:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.929 7f034effd700 1 -- 172.21.2.101:0/2437754727 learned_addr learned my addr 172.21.2.101:0/2437754727 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.929 7f034ffff700 1 --2- 172.21.2.101:0/2437754727 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f03500c2470 0x7f03500c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:40.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.929 7f034f7fe700 1 --2- 172.21.2.101:0/2437754727 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f03500c15b0 0x7f0350088eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:40.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.929 7f034effd700 1 -- 172.21.2.101:0/2437754727 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f03500c2470 msgr2=0x7f03500c2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:40.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.929 7f034effd700 1 --2- 172.21.2.101:0/2437754727 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f03500c2470 0x7f03500c2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:40.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.929 7f034effd700 1 -- 172.21.2.101:0/2437754727 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f03500c15b0 msgr2=0x7f0350088eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:40.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.929 7f034effd700 1 --2- 172.21.2.101:0/2437754727 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f03500c15b0 0x7f0350088eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:40.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.929 7f034effd700 1 -- 172.21.2.101:0/2437754727 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03500b22c0 con 0x7f03500c19b0 2021-02-10T11:14:40.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.929 7f034effd700 1 --2- 172.21.2.101:0/2437754727 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f03500c19b0 0x7f03500894d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f033800b5c0 tx=0x7f033800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:40.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.929 7f0347fff700 1 -- 172.21.2.101:0/2437754727 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0338006b80 con 0x7f03500c19b0 2021-02-10T11:14:40.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.929 7f0347fff700 1 -- 172.21.2.101:0/2437754727 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033800bd50 con 0x7f03500c19b0 2021-02-10T11:14:40.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.929 7f0347fff700 1 -- 172.21.2.101:0/2437754727 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f03380054d0 con 0x7f03500c19b0 2021-02-10T11:14:40.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.929 7f03560ab700 1 -- 172.21.2.101:0/2437754727 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03500b22c0 con 0x7f03500c19b0 2021-02-10T11:14:40.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.929 7f03560ab700 1 -- 172.21.2.101:0/2437754727 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03500b22c0 con 0x7f03500c19b0 2021-02-10T11:14:40.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.930 7f0347fff700 1 -- 172.21.2.101:0/2437754727 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0338006030 con 0x7f03500c19b0 2021-02-10T11:14:40.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.930 7f0347fff700 1 --2- 172.21.2.101:0/2437754727 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f03340186d0 0x7f033401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:40.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.930 7f0347fff700 1 -- 172.21.2.101:0/2437754727 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f03380324e0 con 0x7f03500c19b0 2021-02-10T11:14:40.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.930 7f034f7fe700 1 --2- 172.21.2.101:0/2437754727 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f03340186d0 0x7f033401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:40.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.930 7f034f7fe700 1 --2- 172.21.2.101:0/2437754727 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f03340186d0 0x7f033401abd0 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:40.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.962 7f03560ab700 1 -- 172.21.2.101:0/2437754727 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03501071c0 con 0x7f03500c19b0 2021-02-10T11:14:40.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:40.964 7f0347fff700 1 -- 172.21.2.101:0/2437754727 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0338006b80 con 0x7f03500c19b0 2021-02-10T11:14:41.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.094 7f03560ab700 1 -- 172.21.2.101:0/2437754727 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f035007ed80 con 0x7f03500c19b0 2021-02-10T11:14:41.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.098 7f0347fff700 1 -- 172.21.2.101:0/2437754727 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 updated v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0338006030 con 0x7f03500c19b0 2021-02-10T11:14:41.106 INFO:teuthology.orchestra.run.gibba001.stderr:updated 2021-02-10T11:14:41.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.104 7f03560ab700 1 -- 172.21.2.101:0/2437754727 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f03340186d0 msgr2=0x7f033401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:41.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.104 7f03560ab700 1 --2- 172.21.2.101:0/2437754727 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f03340186d0 0x7f033401abd0 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:41.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.104 7f03560ab700 1 -- 172.21.2.101:0/2437754727 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f03500c19b0 msgr2=0x7f03500894d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:41.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.104 7f03560ab700 1 --2- 172.21.2.101:0/2437754727 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f03500c19b0 0x7f03500894d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f033800b5c0 tx=0x7f033800b960).stop 2021-02-10T11:14:41.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.104 7f03560ab700 1 -- 172.21.2.101:0/2437754727 shutdown_connections 2021-02-10T11:14:41.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.104 7f03560ab700 1 --2- 172.21.2.101:0/2437754727 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f03340186d0 0x7f033401abd0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:41.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.104 7f03560ab700 1 --2- 172.21.2.101:0/2437754727 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f03500c2470 0x7f03500c2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:41.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.104 7f03560ab700 1 --2- 172.21.2.101:0/2437754727 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f03500c19b0 0x7f03500894d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:41.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.104 7f03560ab700 1 --2- 172.21.2.101:0/2437754727 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f03500c15b0 0x7f0350088eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:41.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.104 7f03560ab700 1 -- 172.21.2.101:0/2437754727 >> 172.21.2.101:0/2437754727 conn(0x7f0350082130 msgr2=0x7f0350087150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:41.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.104 7f03560ab700 1 -- 172.21.2.101:0/2437754727 shutdown_connections 2021-02-10T11:14:41.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.104 7f03560ab700 1 -- 172.21.2.101:0/2437754727 wait complete. 2021-02-10T11:14:41.116 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth del client.bootstrap-osd 2021-02-10T11:14:41.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.520 7f9817cf4700 1 Processor -- start 2021-02-10T11:14:41.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.520 7f9817cf4700 1 -- start start 2021-02-10T11:14:41.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.520 7f9817cf4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f98100e5070 0x7f98101b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:41.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.520 7f9817cf4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98101b3e40 0x7f981007d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:41.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.520 7f9817cf4700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981007ed80 con 0x7f98100e5470 2021-02-10T11:14:41.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.520 7f9817cf4700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9810080310 con 0x7f98100e5070 2021-02-10T11:14:41.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.520 7f9817cf4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9810080500 con 0x7f98101b3e40 2021-02-10T11:14:41.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.520 7f9815b10700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f98100e5070 0x7f98101b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:41.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.520 7f9815b10700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f98100e5070 0x7f98101b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45210/0 (socket says 172.21.2.101:45210) 2021-02-10T11:14:41.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.521 7f9815b10700 1 -- 172.21.2.101:0/2748952904 learned_addr learned my addr 172.21.2.101:0/2748952904 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:41.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.521 7f9815b10700 1 -- 172.21.2.101:0/2748952904 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98101b3e40 msgr2=0x7f981007d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:41.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.521 7f9815b10700 1 --2- 172.21.2.101:0/2748952904 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98101b3e40 0x7f981007d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:41.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.521 7f9815b10700 1 -- 172.21.2.101:0/2748952904 >> v1:172.21.2.101:6789/0 conn(0x7f98100e5470 legacy=0x7f981007dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:41.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.521 7f9815b10700 1 -- 172.21.2.101:0/2748952904 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98100806f0 con 0x7f98100e5070 2021-02-10T11:14:41.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.521 7f9815b10700 1 --2- 172.21.2.101:0/2748952904 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98100e5070 0x7f98101b3820 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f9804006700 tx=0x7f9804009b80).ready entity=mon.0 client_cookie=73e1cdbd8b8f3063 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:41.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.521 7f9814b0e700 1 -- 172.21.2.101:0/2748952904 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f980400c0c0 con 0x7f98100e5070 2021-02-10T11:14:41.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.521 7f9814b0e700 1 -- 172.21.2.101:0/2748952904 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980400c340 con 0x7f98100e5070 2021-02-10T11:14:41.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.521 7f9814b0e700 1 -- 172.21.2.101:0/2748952904 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f980400c7a0 con 0x7f98100e5070 2021-02-10T11:14:41.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.521 7f9817cf4700 1 -- 172.21.2.101:0/2748952904 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98100e5070 msgr2=0x7f98101b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:41.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.521 7f9817cf4700 1 --2- 172.21.2.101:0/2748952904 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98100e5070 0x7f98101b3820 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f9804006700 tx=0x7f9804009b80).stop 2021-02-10T11:14:41.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.522 7f9817cf4700 1 -- 172.21.2.101:0/2748952904 shutdown_connections 2021-02-10T11:14:41.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.522 7f9817cf4700 1 --2- 172.21.2.101:0/2748952904 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98101b3e40 0x7f981007d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:41.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.522 7f9817cf4700 1 --2- 172.21.2.101:0/2748952904 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98100e5070 0x7f98101b3820 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:41.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.522 7f9817cf4700 1 -- 172.21.2.101:0/2748952904 >> 172.21.2.101:0/2748952904 conn(0x7f98100ce690 msgr2=0x7f98100e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:41.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.522 7f9817cf4700 1 -- 172.21.2.101:0/2748952904 shutdown_connections 2021-02-10T11:14:41.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.522 7f9817cf4700 1 -- 172.21.2.101:0/2748952904 wait complete. 2021-02-10T11:14:41.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.537 7f9817cf4700 1 Processor -- start 2021-02-10T11:14:41.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.537 7f9817cf4700 1 -- start start 2021-02-10T11:14:41.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.537 7f9817cf4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98100be580 0x7f98100be980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.537 7f9817cf4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9810097ed0 0x7f98100982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.537 7f9817cf4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9810099180 0x7f9810099580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.537 7f9817cf4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98100988f0 con 0x7f9810099180 2021-02-10T11:14:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.537 7f9817cf4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98100c1110 con 0x7f98100be580 2021-02-10T11:14:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.537 7f9817cf4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9810099ba0 con 0x7f9810097ed0 2021-02-10T11:14:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.537 7f9816311700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9810099180 0x7f9810099580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.537 7f9816311700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9810099180 0x7f9810099580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45216/0 (socket says 172.21.2.101:45216) 2021-02-10T11:14:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.537 7f9816311700 1 -- 172.21.2.101:0/412875887 learned_addr learned my addr 172.21.2.101:0/412875887 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.538 7f9816311700 1 -- 172.21.2.101:0/412875887 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9810097ed0 msgr2=0x7f98100982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.538 7f9816311700 1 --2- 172.21.2.101:0/412875887 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9810097ed0 0x7f98100982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.538 7f9816311700 1 -- 172.21.2.101:0/412875887 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98100be580 msgr2=0x7f98100be980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.538 7f9816311700 1 --2- 172.21.2.101:0/412875887 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98100be580 0x7f98100be980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.538 7f9816311700 1 -- 172.21.2.101:0/412875887 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9810099dc0 con 0x7f9810099180 2021-02-10T11:14:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.538 7f9816311700 1 --2- 172.21.2.101:0/412875887 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9810099180 0x7f9810099580 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f9808009980 tx=0x7f980800bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:41.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.538 7f98027fc700 1 -- 172.21.2.101:0/412875887 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9808005af0 con 0x7f9810099180 2021-02-10T11:14:41.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.538 7f98027fc700 1 -- 172.21.2.101:0/412875887 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9808005d70 con 0x7f9810099180 2021-02-10T11:14:41.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.538 7f9817cf4700 1 -- 172.21.2.101:0/412875887 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9810099dc0 con 0x7f9810099180 2021-02-10T11:14:41.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.538 7f98027fc700 1 -- 172.21.2.101:0/412875887 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9808006220 con 0x7f9810099180 2021-02-10T11:14:41.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.538 7f9817cf4700 1 -- 172.21.2.101:0/412875887 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9810099dc0 con 0x7f9810099180 2021-02-10T11:14:41.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.539 7f98027fc700 1 -- 172.21.2.101:0/412875887 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9808005870 con 0x7f9810099180 2021-02-10T11:14:41.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.539 7f98027fc700 1 --2- 172.21.2.101:0/412875887 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97f8018730 0x7f97f801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:41.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.539 7f98027fc700 1 -- 172.21.2.101:0/412875887 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f9808032870 con 0x7f9810099180 2021-02-10T11:14:41.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.539 7f9815b10700 1 --2- 172.21.2.101:0/412875887 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97f8018730 0x7f97f801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:41.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.539 7f9815b10700 1 --2- 172.21.2.101:0/412875887 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97f8018730 0x7f97f801ac10 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:41.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.571 7f9817cf4700 1 -- 172.21.2.101:0/412875887 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9810098c50 con 0x7f9810099180 2021-02-10T11:14:41.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.573 7f98027fc700 1 -- 172.21.2.101:0/412875887 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f980800be10 con 0x7f9810099180 2021-02-10T11:14:41.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.704 7f9817cf4700 1 -- 172.21.2.101:0/412875887 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f9810098c50 con 0x7f9810099180 2021-02-10T11:14:41.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.707 7f98027fc700 1 -- 172.21.2.101:0/412875887 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 updated v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f980800be10 con 0x7f9810099180 2021-02-10T11:14:41.715 INFO:teuthology.orchestra.run.gibba001.stderr:updated 2021-02-10T11:14:41.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.713 7f9817cf4700 1 -- 172.21.2.101:0/412875887 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97f8018730 msgr2=0x7f97f801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:41.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.713 7f9817cf4700 1 --2- 172.21.2.101:0/412875887 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97f8018730 0x7f97f801ac10 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:41.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.713 7f9817cf4700 1 -- 172.21.2.101:0/412875887 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9810099180 msgr2=0x7f9810099580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:41.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.713 7f9817cf4700 1 --2- 172.21.2.101:0/412875887 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9810099180 0x7f9810099580 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f9808009980 tx=0x7f980800bcd0).stop 2021-02-10T11:14:41.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.713 7f9817cf4700 1 -- 172.21.2.101:0/412875887 shutdown_connections 2021-02-10T11:14:41.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.713 7f9817cf4700 1 --2- 172.21.2.101:0/412875887 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97f8018730 0x7f97f801ac10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:41.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.713 7f9817cf4700 1 --2- 172.21.2.101:0/412875887 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9810097ed0 0x7f98100982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:41.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.713 7f9817cf4700 1 --2- 172.21.2.101:0/412875887 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9810099180 0x7f9810099580 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:41.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.713 7f9817cf4700 1 --2- 172.21.2.101:0/412875887 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98100be580 0x7f98100be980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:41.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.713 7f9817cf4700 1 -- 172.21.2.101:0/412875887 >> 172.21.2.101:0/412875887 conn(0x7f98100bde70 msgr2=0x7f98100c3970 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:41.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.713 7f9817cf4700 1 -- 172.21.2.101:0/412875887 shutdown_connections 2021-02-10T11:14:41.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:41.713 7f9817cf4700 1 -- 172.21.2.101:0/412875887 wait complete. 2021-02-10T11:14:41.725 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth del client.bootstrap-rbd 2021-02-10T11:14:42.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.130 7f96de368700 1 Processor -- start 2021-02-10T11:14:42.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.130 7f96de368700 1 -- start start 2021-02-10T11:14:42.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.130 7f96de368700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f96d80e5470 0x7f96d807e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:42.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.130 7f96de368700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96d807f300 0x7f96d807f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:42.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.130 7f96de368700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d807ed80 con 0x7f96d80e5070 2021-02-10T11:14:42.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.130 7f96de368700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d81b4010 con 0x7f96d80e5470 2021-02-10T11:14:42.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.130 7f96de368700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d80d1790 con 0x7f96d807f300 2021-02-10T11:14:42.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.131 7f96dc985700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96d807f300 0x7f96d807f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:42.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.131 7f96d77fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f96d80e5470 0x7f96d807e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:42.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.131 7f96d7fff700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f96d80e5070 0x7f96d81b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:39126/0 (socket says 172.21.2.101:39126) 2021-02-10T11:14:42.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.131 7f96d7fff700 1 -- 172.21.2.101:0/1526072799 learned_addr learned my addr 172.21.2.101:0/1526072799 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:14:42.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.132 7f96d6ffd700 1 -- 172.21.2.101:0/1526072799 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4023746135 0 0) 0x7f96c8001190 con 0x7f96d80e5070 2021-02-10T11:14:42.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.132 7f96d6ffd700 1 -- 172.21.2.101:0/1526072799 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96c0002fe0 con 0x7f96d80e5070 2021-02-10T11:14:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.132 7f96d77fe700 1 -- 172.21.2.101:0/1526072799 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96d807f300 msgr2=0x7f96d807f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.132 7f96d77fe700 1 --2- 172.21.2.101:0/1526072799 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96d807f300 0x7f96d807f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.132 7f96d77fe700 1 -- 172.21.2.101:0/1526072799 >> v1:172.21.2.108:6789/0 conn(0x7f96d80e5070 legacy=0x7f96d81b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.132 7f96d77fe700 1 -- 172.21.2.101:0/1526072799 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d80d19b0 con 0x7f96d80e5470 2021-02-10T11:14:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.132 7f96d77fe700 1 --2- 172.21.2.101:0/1526072799 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96d80e5470 0x7f96d807e6a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f96cc006650 tx=0x7f96cc009a80).ready entity=mon.1 client_cookie=d62bc401cdf71ac1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.132 7f96d6ffd700 1 -- 172.21.2.101:0/1526072799 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96cc00c0c0 con 0x7f96d80e5470 2021-02-10T11:14:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.133 7f96d6ffd700 1 -- 172.21.2.101:0/1526072799 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96cc00c340 con 0x7f96d80e5470 2021-02-10T11:14:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.133 7f96d6ffd700 1 -- 172.21.2.101:0/1526072799 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96cc00c860 con 0x7f96d80e5470 2021-02-10T11:14:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.133 7f96de368700 1 -- 172.21.2.101:0/1526072799 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96d80e5470 msgr2=0x7f96d807e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.133 7f96de368700 1 --2- 172.21.2.101:0/1526072799 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96d80e5470 0x7f96d807e6a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f96cc006650 tx=0x7f96cc009a80).stop 2021-02-10T11:14:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.133 7f96de368700 1 -- 172.21.2.101:0/1526072799 shutdown_connections 2021-02-10T11:14:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.133 7f96de368700 1 --2- 172.21.2.101:0/1526072799 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96d807f300 0x7f96d807f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.133 7f96de368700 1 --2- 172.21.2.101:0/1526072799 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96d80e5470 0x7f96d807e6a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:42.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.133 7f96de368700 1 -- 172.21.2.101:0/1526072799 >> 172.21.2.101:0/1526072799 conn(0x7f96d80ce690 msgr2=0x7f96d80e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:42.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.133 7f96de368700 1 -- 172.21.2.101:0/1526072799 shutdown_connections 2021-02-10T11:14:42.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.133 7f96de368700 1 -- 172.21.2.101:0/1526072799 wait complete. 2021-02-10T11:14:42.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.147 7f96de368700 1 Processor -- start 2021-02-10T11:14:42.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.148 7f96de368700 1 -- start start 2021-02-10T11:14:42.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.148 7f96de368700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96d80b4ad0 0x7f96d80a78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:42.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.148 7f96de368700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96d80a7ef0 0x7f96d81a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:42.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.148 7f96de368700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96d80a82f0 0x7f96d81a73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:42.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.148 7f96de368700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d80aa3c0 con 0x7f96d80b4ad0 2021-02-10T11:14:42.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.148 7f96de368700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d80a88c0 con 0x7f96d80a82f0 2021-02-10T11:14:42.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.148 7f96de368700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d80b4ed0 con 0x7f96d80a7ef0 2021-02-10T11:14:42.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.148 7f96d7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96d80b4ad0 0x7f96d80a78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:42.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.148 7f96d7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96d80b4ad0 0x7f96d80a78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45228/0 (socket says 172.21.2.101:45228) 2021-02-10T11:14:42.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.148 7f96d7fff700 1 -- 172.21.2.101:0/1901412571 learned_addr learned my addr 172.21.2.101:0/1901412571 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:42.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.149 7f96d7fff700 1 -- 172.21.2.101:0/1901412571 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96d80a7ef0 msgr2=0x7f96d81a6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:42.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.149 7f96d7fff700 1 --2- 172.21.2.101:0/1901412571 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96d80a7ef0 0x7f96d81a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:42.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.149 7f96d7fff700 1 -- 172.21.2.101:0/1901412571 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96d80a82f0 msgr2=0x7f96d81a73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:42.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.149 7f96d7fff700 1 --2- 172.21.2.101:0/1901412571 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96d80a82f0 0x7f96d81a73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:42.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.149 7f96d7fff700 1 -- 172.21.2.101:0/1901412571 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d81a79d0 con 0x7f96d80b4ad0 2021-02-10T11:14:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.149 7f96d7fff700 1 --2- 172.21.2.101:0/1901412571 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96d80b4ad0 0x7f96d80a78d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f96c800a580 tx=0x7f96c800a920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.149 7f96d4ff9700 1 -- 172.21.2.101:0/1901412571 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96c80070d0 con 0x7f96d80b4ad0 2021-02-10T11:14:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.149 7f96d4ff9700 1 -- 172.21.2.101:0/1901412571 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c800ad10 con 0x7f96d80b4ad0 2021-02-10T11:14:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.149 7f96d4ff9700 1 -- 172.21.2.101:0/1901412571 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96c8004510 con 0x7f96d80b4ad0 2021-02-10T11:14:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.149 7f96de368700 1 -- 172.21.2.101:0/1901412571 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d81a79d0 con 0x7f96d80b4ad0 2021-02-10T11:14:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.149 7f96de368700 1 -- 172.21.2.101:0/1901412571 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d80a9130 con 0x7f96d80b4ad0 2021-02-10T11:14:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.150 7f96d4ff9700 1 -- 172.21.2.101:0/1901412571 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f96c8004320 con 0x7f96d80b4ad0 2021-02-10T11:14:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.150 7f96d4ff9700 1 --2- 172.21.2.101:0/1901412571 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96b80186c0 0x7f96b801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.150 7f96d4ff9700 1 -- 172.21.2.101:0/1901412571 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f96c8032080 con 0x7f96d80b4ad0 2021-02-10T11:14:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.150 7f96d77fe700 1 --2- 172.21.2.101:0/1901412571 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96b80186c0 0x7f96b801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.150 7f96d77fe700 1 --2- 172.21.2.101:0/1901412571 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96b80186c0 0x7f96b801abc0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:42.166 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:14:42.169 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:14:42.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.180 7f96de368700 1 -- 172.21.2.101:0/1901412571 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d8084a90 con 0x7f96d80b4ad0 2021-02-10T11:14:42.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.183 7f96d4ff9700 1 -- 172.21.2.101:0/1901412571 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f96c8000c10 con 0x7f96d80b4ad0 2021-02-10T11:14:42.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.314 7f96de368700 1 -- 172.21.2.101:0/1901412571 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f96d8084a90 con 0x7f96d80b4ad0 2021-02-10T11:14:42.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.317 7f96d4ff9700 1 -- 172.21.2.101:0/1901412571 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 updated v10) v1 ==== 97+0+0 (secure 0 0 0) 0x7f96c8004320 con 0x7f96d80b4ad0 2021-02-10T11:14:42.326 INFO:teuthology.orchestra.run.gibba001.stderr:updated 2021-02-10T11:14:42.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.323 7f96de368700 1 -- 172.21.2.101:0/1901412571 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96b80186c0 msgr2=0x7f96b801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:42.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.323 7f96de368700 1 --2- 172.21.2.101:0/1901412571 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96b80186c0 0x7f96b801abc0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:42.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.323 7f96de368700 1 -- 172.21.2.101:0/1901412571 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96d80b4ad0 msgr2=0x7f96d80a78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:42.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.323 7f96de368700 1 --2- 172.21.2.101:0/1901412571 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96d80b4ad0 0x7f96d80a78d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f96c800a580 tx=0x7f96c800a920).stop 2021-02-10T11:14:42.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.323 7f96de368700 1 -- 172.21.2.101:0/1901412571 shutdown_connections 2021-02-10T11:14:42.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.323 7f96de368700 1 --2- 172.21.2.101:0/1901412571 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96b80186c0 0x7f96b801abc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:42.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.323 7f96de368700 1 --2- 172.21.2.101:0/1901412571 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96d80a7ef0 0x7f96d81a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:42.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.323 7f96de368700 1 --2- 172.21.2.101:0/1901412571 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96d80a82f0 0x7f96d81a73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:42.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.323 7f96de368700 1 --2- 172.21.2.101:0/1901412571 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96d80b4ad0 0x7f96d80a78d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:42.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.323 7f96de368700 1 -- 172.21.2.101:0/1901412571 >> 172.21.2.101:0/1901412571 conn(0x7f96d80b5560 msgr2=0x7f96d80a6200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:42.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.323 7f96de368700 1 -- 172.21.2.101:0/1901412571 shutdown_connections 2021-02-10T11:14:42.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.323 7f96de368700 1 -- 172.21.2.101:0/1901412571 wait complete. 2021-02-10T11:14:42.336 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2021-02-10T11:14:42.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.782 7f2dacfa6700 1 Processor -- start 2021-02-10T11:14:42.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.782 7f2dacfa6700 1 -- start start 2021-02-10T11:14:42.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.783 7f2dacfa6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2da80e5070 0x7f2da81b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:42.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.783 7f2dacfa6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2da81b3e40 0x7f2da807d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:42.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.783 7f2dacfa6700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da807ed80 con 0x7f2da80e5470 2021-02-10T11:14:42.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.783 7f2dacfa6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da8080310 con 0x7f2da80e5070 2021-02-10T11:14:42.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.783 7f2dacfa6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da8080500 con 0x7f2da81b3e40 2021-02-10T11:14:42.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.783 7f2da6d9d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2da80e5070 0x7f2da81b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:42.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.783 7f2da6d9d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2da80e5070 0x7f2da81b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45236/0 (socket says 172.21.2.101:45236) 2021-02-10T11:14:42.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.783 7f2da6d9d700 1 -- 172.21.2.101:0/1715386078 learned_addr learned my addr 172.21.2.101:0/1715386078 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:42.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.783 7f2da659c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2da81b3e40 0x7f2da807d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:42.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.783 7f2da659c700 1 -- 172.21.2.101:0/1715386078 >> v2:172.21.2.101:3300/0 conn(0x7f2da80e5070 msgr2=0x7f2da81b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:42.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.783 7f2da659c700 1 --2- 172.21.2.101:0/1715386078 >> v2:172.21.2.101:3300/0 conn(0x7f2da80e5070 0x7f2da81b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:42.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.783 7f2da659c700 1 -- 172.21.2.101:0/1715386078 >> v1:172.21.2.101:6789/0 conn(0x7f2da80e5470 legacy=0x7f2da807dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:42.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.783 7f2da659c700 1 -- 172.21.2.101:0/1715386078 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da80806f0 con 0x7f2da81b3e40 2021-02-10T11:14:42.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.783 7f2da6d9d700 1 --2- 172.21.2.101:0/1715386078 >> v2:172.21.2.101:3300/0 conn(0x7f2da80e5070 0x7f2da81b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:14:42.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.784 7f2da659c700 1 --2- 172.21.2.101:0/1715386078 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2da81b3e40 0x7f2da807d690 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f2d9c006650 tx=0x7f2d9c009a80).ready entity=mon.2 client_cookie=96a4cf3f61b3ccec server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:42.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.784 7f2da5d9b700 1 -- 172.21.2.101:0/1715386078 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d9c006390 con 0x7f2da81b3e40 2021-02-10T11:14:42.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.784 7f2da5d9b700 1 -- 172.21.2.101:0/1715386078 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9c0009b0 con 0x7f2da81b3e40 2021-02-10T11:14:42.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.784 7f2da5d9b700 1 -- 172.21.2.101:0/1715386078 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d9c00c2b0 con 0x7f2da81b3e40 2021-02-10T11:14:42.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.784 7f2dacfa6700 1 -- 172.21.2.101:0/1715386078 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2da81b3e40 msgr2=0x7f2da807d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:42.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.784 7f2dacfa6700 1 --2- 172.21.2.101:0/1715386078 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2da81b3e40 0x7f2da807d690 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f2d9c006650 tx=0x7f2d9c009a80).stop 2021-02-10T11:14:42.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.785 7f2dacfa6700 1 -- 172.21.2.101:0/1715386078 shutdown_connections 2021-02-10T11:14:42.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.785 7f2dacfa6700 1 --2- 172.21.2.101:0/1715386078 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2da81b3e40 0x7f2da807d690 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:42.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.785 7f2dacfa6700 1 --2- 172.21.2.101:0/1715386078 >> v2:172.21.2.101:3300/0 conn(0x7f2da80e5070 0x7f2da81b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:42.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.785 7f2dacfa6700 1 -- 172.21.2.101:0/1715386078 >> 172.21.2.101:0/1715386078 conn(0x7f2da80ce690 msgr2=0x7f2da80e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:42.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.785 7f2dacfa6700 1 -- 172.21.2.101:0/1715386078 shutdown_connections 2021-02-10T11:14:42.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.785 7f2dacfa6700 1 -- 172.21.2.101:0/1715386078 wait complete. 2021-02-10T11:14:42.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.799 7f2dacfa6700 1 Processor -- start 2021-02-10T11:14:42.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.799 7f2dacfa6700 1 -- start start 2021-02-10T11:14:42.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.800 7f2dacfa6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2da8082520 0x7f2da8087ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:42.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.800 7f2dacfa6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2da80880d0 0x7f2da80b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:42.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.800 7f2dacfa6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2da80884d0 0x7f2da80b2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:42.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.800 7f2dacfa6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da80b17d0 con 0x7f2da80880d0 2021-02-10T11:14:42.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.800 7f2dacfa6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da80b2990 con 0x7f2da80884d0 2021-02-10T11:14:42.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.800 7f2dacfa6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da80b2b70 con 0x7f2da8082520 2021-02-10T11:14:42.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.800 7f2da659c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2da80880d0 0x7f2da80b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:42.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.800 7f2da659c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2da80880d0 0x7f2da80b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45238/0 (socket says 172.21.2.101:45238) 2021-02-10T11:14:42.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.800 7f2da659c700 1 -- 172.21.2.101:0/1288514204 learned_addr learned my addr 172.21.2.101:0/1288514204 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:42.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.800 7f2da659c700 1 -- 172.21.2.101:0/1288514204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2da8082520 msgr2=0x7f2da8087ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:42.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.800 7f2da659c700 1 --2- 172.21.2.101:0/1288514204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2da8082520 0x7f2da8087ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:42.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.800 7f2da659c700 1 -- 172.21.2.101:0/1288514204 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2da80884d0 msgr2=0x7f2da80b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:42.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.800 7f2da659c700 1 --2- 172.21.2.101:0/1288514204 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2da80884d0 0x7f2da80b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:42.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.800 7f2da659c700 1 -- 172.21.2.101:0/1288514204 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da8088a70 con 0x7f2da80880d0 2021-02-10T11:14:42.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.800 7f2da659c700 1 --2- 172.21.2.101:0/1288514204 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2da80880d0 0x7f2da80b1d50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f2d9c006f70 tx=0x7f2d9c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:42.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.801 7f2d977fe700 1 -- 172.21.2.101:0/1288514204 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d9c0064a0 con 0x7f2da80880d0 2021-02-10T11:14:42.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.801 7f2d977fe700 1 -- 172.21.2.101:0/1288514204 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9c006c50 con 0x7f2da80880d0 2021-02-10T11:14:42.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.801 7f2d977fe700 1 -- 172.21.2.101:0/1288514204 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d9c0052e0 con 0x7f2da80880d0 2021-02-10T11:14:42.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.801 7f2dacfa6700 1 -- 172.21.2.101:0/1288514204 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da8088a70 con 0x7f2da80880d0 2021-02-10T11:14:42.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.801 7f2dacfa6700 1 -- 172.21.2.101:0/1288514204 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da8088fd0 con 0x7f2da80880d0 2021-02-10T11:14:42.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.801 7f2d977fe700 1 -- 172.21.2.101:0/1288514204 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2d9c0052e0 con 0x7f2da80880d0 2021-02-10T11:14:42.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.802 7f2d977fe700 1 --2- 172.21.2.101:0/1288514204 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d8c0186c0 0x7f2d8c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:42.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.802 7f2d977fe700 1 -- 172.21.2.101:0/1288514204 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f2d9c032200 con 0x7f2da80880d0 2021-02-10T11:14:42.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.802 7f2da6d9d700 1 --2- 172.21.2.101:0/1288514204 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d8c0186c0 0x7f2d8c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:42.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.802 7f2da6d9d700 1 --2- 172.21.2.101:0/1288514204 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d8c0186c0 0x7f2d8c01abc0 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:42.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.833 7f2dacfa6700 1 -- 172.21.2.101:0/1288514204 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da8107180 con 0x7f2da80880d0 2021-02-10T11:14:42.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.835 7f2d977fe700 1 -- 172.21.2.101:0/1288514204 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2d9c0009b0 con 0x7f2da80880d0 2021-02-10T11:14:42.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.967 7f2dacfa6700 1 -- 172.21.2.101:0/1288514204 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f2da807ed80 con 0x7f2da80880d0 2021-02-10T11:14:42.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.971 7f2d977fe700 1 -- 172.21.2.101:0/1288514204 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 updated v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2d9c0052e0 con 0x7f2da80880d0 2021-02-10T11:14:42.979 INFO:teuthology.orchestra.run.gibba001.stderr:updated 2021-02-10T11:14:42.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.977 7f2dacfa6700 1 -- 172.21.2.101:0/1288514204 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d8c0186c0 msgr2=0x7f2d8c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:42.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.977 7f2dacfa6700 1 --2- 172.21.2.101:0/1288514204 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d8c0186c0 0x7f2d8c01abc0 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:42.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.977 7f2dacfa6700 1 -- 172.21.2.101:0/1288514204 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2da80880d0 msgr2=0x7f2da80b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:42.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.977 7f2dacfa6700 1 --2- 172.21.2.101:0/1288514204 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2da80880d0 0x7f2da80b1d50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f2d9c006f70 tx=0x7f2d9c00b960).stop 2021-02-10T11:14:42.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.977 7f2dacfa6700 1 -- 172.21.2.101:0/1288514204 shutdown_connections 2021-02-10T11:14:42.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.977 7f2dacfa6700 1 --2- 172.21.2.101:0/1288514204 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2da80884d0 0x7f2da80b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:42.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.977 7f2dacfa6700 1 --2- 172.21.2.101:0/1288514204 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2da80880d0 0x7f2da80b1d50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:42.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.977 7f2dacfa6700 1 --2- 172.21.2.101:0/1288514204 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d8c0186c0 0x7f2d8c01abc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:42.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.977 7f2dacfa6700 1 --2- 172.21.2.101:0/1288514204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2da8082520 0x7f2da8087ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:42.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.977 7f2dacfa6700 1 -- 172.21.2.101:0/1288514204 >> 172.21.2.101:0/1288514204 conn(0x7f2da8098740 msgr2=0x7f2da80c3590 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:42.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.977 7f2dacfa6700 1 -- 172.21.2.101:0/1288514204 shutdown_connections 2021-02-10T11:14:42.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:42.977 7f2dacfa6700 1 -- 172.21.2.101:0/1288514204 wait complete. 2021-02-10T11:14:42.989 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth del client.bootstrap-rgw 2021-02-10T11:14:43.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.392 7f331b30b700 1 Processor -- start 2021-02-10T11:14:43.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.392 7f331b30b700 1 -- start start 2021-02-10T11:14:43.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.393 7f331b30b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f33140e5070 0x7f33141b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:43.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.393 7f331b30b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33141b3e40 0x7f331407d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:43.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.393 7f331b30b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f331407ed80 con 0x7f33140e5470 2021-02-10T11:14:43.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.393 7f331b30b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3314080310 con 0x7f33140e5070 2021-02-10T11:14:43.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.393 7f331b30b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3314080500 con 0x7f33141b3e40 2021-02-10T11:14:43.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.393 7f3319127700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f33140e5070 0x7f33141b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:43.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.393 7f3319127700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f33140e5070 0x7f33141b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45246/0 (socket says 172.21.2.101:45246) 2021-02-10T11:14:43.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.393 7f3319127700 1 -- 172.21.2.101:0/3938120703 learned_addr learned my addr 172.21.2.101:0/3938120703 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:43.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.393 7f3319127700 1 -- 172.21.2.101:0/3938120703 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33141b3e40 msgr2=0x7f331407d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:43.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.393 7f3318926700 1 --2- 172.21.2.101:0/3938120703 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33141b3e40 0x7f331407d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:43.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.393 7f3319127700 1 --2- 172.21.2.101:0/3938120703 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33141b3e40 0x7f331407d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:43.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.394 7f3319127700 1 -- 172.21.2.101:0/3938120703 >> v1:172.21.2.108:6789/0 conn(0x7f33140e5470 legacy=0x7f331407dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:43.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.394 7f3319127700 1 -- 172.21.2.101:0/3938120703 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33140806f0 con 0x7f33140e5070 2021-02-10T11:14:43.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.394 7f3318926700 1 --2- 172.21.2.101:0/3938120703 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33141b3e40 0x7f331407d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:14:43.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.394 7f3319127700 1 --2- 172.21.2.101:0/3938120703 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33140e5070 0x7f33141b3820 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f3310006700 tx=0x7f3310009b80).ready entity=mon.0 client_cookie=748dd236c392a8b3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:43.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.394 7f3307fff700 1 -- 172.21.2.101:0/3938120703 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f331000c0c0 con 0x7f33140e5070 2021-02-10T11:14:43.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.394 7f3307fff700 1 -- 172.21.2.101:0/3938120703 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331000c400 con 0x7f33140e5070 2021-02-10T11:14:43.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.394 7f3307fff700 1 -- 172.21.2.101:0/3938120703 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f331000c920 con 0x7f33140e5070 2021-02-10T11:14:43.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.394 7f331b30b700 1 -- 172.21.2.101:0/3938120703 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33140e5070 msgr2=0x7f33141b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:43.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.394 7f331b30b700 1 --2- 172.21.2.101:0/3938120703 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33140e5070 0x7f33141b3820 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f3310006700 tx=0x7f3310009b80).stop 2021-02-10T11:14:43.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.395 7f331b30b700 1 -- 172.21.2.101:0/3938120703 shutdown_connections 2021-02-10T11:14:43.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.395 7f331b30b700 1 --2- 172.21.2.101:0/3938120703 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33141b3e40 0x7f331407d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:43.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.395 7f331b30b700 1 --2- 172.21.2.101:0/3938120703 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33140e5070 0x7f33141b3820 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:43.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.395 7f331b30b700 1 -- 172.21.2.101:0/3938120703 >> 172.21.2.101:0/3938120703 conn(0x7f33140ce690 msgr2=0x7f33140e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:43.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.395 7f331b30b700 1 -- 172.21.2.101:0/3938120703 shutdown_connections 2021-02-10T11:14:43.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.395 7f331b30b700 1 -- 172.21.2.101:0/3938120703 wait complete. 2021-02-10T11:14:43.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.409 7f331b30b700 1 Processor -- start 2021-02-10T11:14:43.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.410 7f331b30b700 1 -- start start 2021-02-10T11:14:43.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.410 7f331b30b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3314094a30 0x7f3314094e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:43.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.410 7f331b30b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f33140956e0 0x7f3314095ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:43.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.410 7f331b30b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3314096100 0x7f3314097070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:43.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.410 7f331b30b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3314096af0 con 0x7f3314094a30 2021-02-10T11:14:43.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.410 7f331b30b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3314096500 con 0x7f33140956e0 2021-02-10T11:14:43.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.410 7f331b30b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33140966e0 con 0x7f3314096100 2021-02-10T11:14:43.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.410 7f3319127700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3314094a30 0x7f3314094e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:43.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.410 7f3319127700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3314094a30 0x7f3314094e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45250/0 (socket says 172.21.2.101:45250) 2021-02-10T11:14:43.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.410 7f3319127700 1 -- 172.21.2.101:0/690603288 learned_addr learned my addr 172.21.2.101:0/690603288 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:43.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.411 7f3319127700 1 -- 172.21.2.101:0/690603288 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3314096100 msgr2=0x7f3314097070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:43.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.411 7f3319127700 1 --2- 172.21.2.101:0/690603288 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3314096100 0x7f3314097070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:43.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.411 7f3319127700 1 -- 172.21.2.101:0/690603288 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f33140956e0 msgr2=0x7f3314095ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:43.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.411 7f3319127700 1 --2- 172.21.2.101:0/690603288 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f33140956e0 0x7f3314095ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:43.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.411 7f3319127700 1 -- 172.21.2.101:0/690603288 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3314097690 con 0x7f3314094a30 2021-02-10T11:14:43.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.411 7f3319127700 1 --2- 172.21.2.101:0/690603288 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3314094a30 0x7f3314094e30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f33100066a0 tx=0x7f331000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:43.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.411 7f3305ffb700 1 -- 172.21.2.101:0/690603288 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f331000bd80 con 0x7f3314094a30 2021-02-10T11:14:43.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.411 7f3305ffb700 1 -- 172.21.2.101:0/690603288 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33100052e0 con 0x7f3314094a30 2021-02-10T11:14:43.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.411 7f3305ffb700 1 -- 172.21.2.101:0/690603288 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f33100057c0 con 0x7f3314094a30 2021-02-10T11:14:43.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.411 7f331b30b700 1 -- 172.21.2.101:0/690603288 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3314097690 con 0x7f3314094a30 2021-02-10T11:14:43.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.411 7f331b30b700 1 -- 172.21.2.101:0/690603288 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3314097c20 con 0x7f3314094a30 2021-02-10T11:14:43.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.412 7f3305ffb700 1 -- 172.21.2.101:0/690603288 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3310006030 con 0x7f3314094a30 2021-02-10T11:14:43.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.412 7f3305ffb700 1 --2- 172.21.2.101:0/690603288 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32f40186d0 0x7f32f401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:43.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.412 7f3305ffb700 1 -- 172.21.2.101:0/690603288 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f3310032530 con 0x7f3314094a30 2021-02-10T11:14:43.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.412 7f3318926700 1 --2- 172.21.2.101:0/690603288 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32f40186d0 0x7f32f401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:43.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.412 7f3318926700 1 --2- 172.21.2.101:0/690603288 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32f40186d0 0x7f32f401abd0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:43.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.443 7f331b30b700 1 -- 172.21.2.101:0/690603288 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3314107150 con 0x7f3314094a30 2021-02-10T11:14:43.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.445 7f3305ffb700 1 -- 172.21.2.101:0/690603288 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3310005a80 con 0x7f3314094a30 2021-02-10T11:14:43.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.575 7f331b30b700 1 -- 172.21.2.101:0/690603288 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f331407ed80 con 0x7f3314094a30 2021-02-10T11:14:43.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.579 7f3305ffb700 1 -- 172.21.2.101:0/690603288 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 updated v12) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3310006030 con 0x7f3314094a30 2021-02-10T11:14:43.587 INFO:teuthology.orchestra.run.gibba001.stderr:updated 2021-02-10T11:14:43.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.585 7f331b30b700 1 -- 172.21.2.101:0/690603288 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32f40186d0 msgr2=0x7f32f401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:43.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.585 7f331b30b700 1 --2- 172.21.2.101:0/690603288 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32f40186d0 0x7f32f401abd0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:43.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.585 7f331b30b700 1 -- 172.21.2.101:0/690603288 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3314094a30 msgr2=0x7f3314094e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:43.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.585 7f331b30b700 1 --2- 172.21.2.101:0/690603288 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3314094a30 0x7f3314094e30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f33100066a0 tx=0x7f331000b990).stop 2021-02-10T11:14:43.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.585 7f331b30b700 1 -- 172.21.2.101:0/690603288 shutdown_connections 2021-02-10T11:14:43.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.585 7f331b30b700 1 --2- 172.21.2.101:0/690603288 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32f40186d0 0x7f32f401abd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:43.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.585 7f331b30b700 1 --2- 172.21.2.101:0/690603288 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3314096100 0x7f3314097070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:43.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.585 7f331b30b700 1 --2- 172.21.2.101:0/690603288 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f33140956e0 0x7f3314095ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:43.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.585 7f331b30b700 1 --2- 172.21.2.101:0/690603288 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3314094a30 0x7f3314094e30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:43.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.585 7f331b30b700 1 -- 172.21.2.101:0/690603288 >> 172.21.2.101:0/690603288 conn(0x7f33140a6c50 msgr2=0x7f33140a6230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:43.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.585 7f331b30b700 1 -- 172.21.2.101:0/690603288 shutdown_connections 2021-02-10T11:14:43.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:43.585 7f331b30b700 1 -- 172.21.2.101:0/690603288 wait complete. 2021-02-10T11:14:43.598 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:14:44.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.003 7ff0e5dfd700 1 Processor -- start 2021-02-10T11:14:44.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.003 7ff0e5dfd700 1 -- start start 2021-02-10T11:14:44.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.003 7ff0e5dfd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff0e00e5070 0x7ff0e01b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:44.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.003 7ff0e5dfd700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff0e00e5470 0x7ff0e0080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:44.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.003 7ff0e5dfd700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e007ed80 con 0x7ff0e01b3e40 2021-02-10T11:14:44.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.003 7ff0e5dfd700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e00d1790 con 0x7ff0e00e5070 2021-02-10T11:14:44.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.003 7ff0e5dfd700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e00d19b0 con 0x7ff0e00e5470 2021-02-10T11:14:44.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.003 7ff0df7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff0e00e5070 0x7ff0e01b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:44.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.004 7ff0deffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff0e01b3e40 0x7ff0e007d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45754/0 (socket says 172.21.2.101:45754) 2021-02-10T11:14:44.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.004 7ff0deffd700 1 -- 172.21.2.101:0/1950860041 learned_addr learned my addr 172.21.2.101:0/1950860041 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:14:44.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.004 7ff0dffff700 1 --2- 172.21.2.101:0/1950860041 >> v2:172.21.2.108:3300/0 conn(0x7ff0e00e5470 0x7ff0e0080310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:44.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.004 7ff0df7fe700 1 -- 172.21.2.101:0/1950860041 >> v2:172.21.2.108:3300/0 conn(0x7ff0e00e5470 msgr2=0x7ff0e0080310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:44.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.004 7ff0df7fe700 1 --2- 172.21.2.101:0/1950860041 >> v2:172.21.2.108:3300/0 conn(0x7ff0e00e5470 0x7ff0e0080310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:44.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.004 7ff0df7fe700 1 -- 172.21.2.101:0/1950860041 >> v1:172.21.2.101:6789/0 conn(0x7ff0e01b3e40 legacy=0x7ff0e007d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:44.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.004 7ff0df7fe700 1 -- 172.21.2.101:0/1950860041 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e00d1bd0 con 0x7ff0e00e5070 2021-02-10T11:14:44.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.004 7ff0df7fe700 1 --2- 172.21.2.101:0/1950860041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0e00e5070 0x7ff0e01b3820 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7ff0d4006650 tx=0x7ff0d4009a80).ready entity=mon.0 client_cookie=996ccc5d86095472 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:44.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.004 7ff0de7fc700 1 -- 172.21.2.101:0/1950860041 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0d400c0c0 con 0x7ff0e00e5070 2021-02-10T11:14:44.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.004 7ff0de7fc700 1 -- 172.21.2.101:0/1950860041 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d400c340 con 0x7ff0e00e5070 2021-02-10T11:14:44.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.004 7ff0de7fc700 1 -- 172.21.2.101:0/1950860041 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0d400c860 con 0x7ff0e00e5070 2021-02-10T11:14:44.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.004 7ff0e5dfd700 1 -- 172.21.2.101:0/1950860041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0e00e5070 msgr2=0x7ff0e01b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:44.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.004 7ff0e5dfd700 1 --2- 172.21.2.101:0/1950860041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0e00e5070 0x7ff0e01b3820 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7ff0d4006650 tx=0x7ff0d4009a80).stop 2021-02-10T11:14:44.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.005 7ff0e5dfd700 1 -- 172.21.2.101:0/1950860041 shutdown_connections 2021-02-10T11:14:44.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.005 7ff0e5dfd700 1 --2- 172.21.2.101:0/1950860041 >> v2:172.21.2.108:3300/0 conn(0x7ff0e00e5470 0x7ff0e0080310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:44.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.005 7ff0e5dfd700 1 --2- 172.21.2.101:0/1950860041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0e00e5070 0x7ff0e01b3820 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:44.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.005 7ff0e5dfd700 1 -- 172.21.2.101:0/1950860041 >> 172.21.2.101:0/1950860041 conn(0x7ff0e00ce690 msgr2=0x7ff0e00e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:44.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.005 7ff0e5dfd700 1 -- 172.21.2.101:0/1950860041 shutdown_connections 2021-02-10T11:14:44.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.005 7ff0e5dfd700 1 -- 172.21.2.101:0/1950860041 wait complete. 2021-02-10T11:14:44.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.019 7ff0e5dfd700 1 Processor -- start 2021-02-10T11:14:44.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.020 7ff0e5dfd700 1 -- start start 2021-02-10T11:14:44.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.020 7ff0e5dfd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0e0191bb0 0x7ff0e00a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:44.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.020 7ff0e5dfd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0e00a29a0 0x7ff0e00a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:44.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.020 7ff0e5dfd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0e00a33c0 0x7ff0e00a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:44.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.020 7ff0e5dfd700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00adf90 con 0x7ff0e00a33c0 2021-02-10T11:14:44.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.020 7ff0e5dfd700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00a1e00 con 0x7ff0e0191bb0 2021-02-10T11:14:44.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.020 7ff0e5dfd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00a09f0 con 0x7ff0e00a29a0 2021-02-10T11:14:44.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.020 7ff0dffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0e00a33c0 0x7ff0e00a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:44.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.020 7ff0dffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0e00a33c0 0x7ff0e00a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45270/0 (socket says 172.21.2.101:45270) 2021-02-10T11:14:44.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.020 7ff0dffff700 1 -- 172.21.2.101:0/3324269203 learned_addr learned my addr 172.21.2.101:0/3324269203 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:44.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.020 7ff0df7fe700 1 --2- 172.21.2.101:0/3324269203 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0e0191bb0 0x7ff0e00a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:44.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.020 7ff0deffd700 1 --2- 172.21.2.101:0/3324269203 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0e00a29a0 0x7ff0e00a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:44.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.021 7ff0dffff700 1 -- 172.21.2.101:0/3324269203 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0e00a29a0 msgr2=0x7ff0e00a2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:44.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.021 7ff0dffff700 1 --2- 172.21.2.101:0/3324269203 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0e00a29a0 0x7ff0e00a2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:44.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.021 7ff0dffff700 1 -- 172.21.2.101:0/3324269203 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0e0191bb0 msgr2=0x7ff0e00a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:44.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.021 7ff0dffff700 1 --2- 172.21.2.101:0/3324269203 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0e0191bb0 0x7ff0e00a2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:44.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.021 7ff0dffff700 1 -- 172.21.2.101:0/3324269203 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e00a0bd0 con 0x7ff0e00a33c0 2021-02-10T11:14:44.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.021 7ff0dffff700 1 --2- 172.21.2.101:0/3324269203 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0e00a33c0 0x7ff0e00a03d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7ff0d0006f20 tx=0x7ff0d000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:44.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.021 7ff0c7fff700 1 -- 172.21.2.101:0/3324269203 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0d0006bb0 con 0x7ff0e00a33c0 2021-02-10T11:14:44.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.021 7ff0c7fff700 1 -- 172.21.2.101:0/3324269203 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d000bd80 con 0x7ff0e00a33c0 2021-02-10T11:14:44.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.021 7ff0c7fff700 1 -- 172.21.2.101:0/3324269203 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0d00022b0 con 0x7ff0e00a33c0 2021-02-10T11:14:44.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.021 7ff0e5dfd700 1 -- 172.21.2.101:0/3324269203 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e00a0bd0 con 0x7ff0e00a33c0 2021-02-10T11:14:44.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.021 7ff0e5dfd700 1 -- 172.21.2.101:0/3324269203 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e00a1160 con 0x7ff0e00a33c0 2021-02-10T11:14:44.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.021 7ff0c7fff700 1 -- 172.21.2.101:0/3324269203 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff0d0002020 con 0x7ff0e00a33c0 2021-02-10T11:14:44.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.022 7ff0c7fff700 1 --2- 172.21.2.101:0/3324269203 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0c00186c0 0x7ff0c001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:44.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.022 7ff0c7fff700 1 -- 172.21.2.101:0/3324269203 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4753+0+0 (secure 0 0 0) 0x7ff0d0032260 con 0x7ff0e00a33c0 2021-02-10T11:14:44.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.022 7ff0df7fe700 1 --2- 172.21.2.101:0/3324269203 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0c00186c0 0x7ff0c001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:44.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.022 7ff0df7fe700 1 --2- 172.21.2.101:0/3324269203 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0c00186c0 0x7ff0c001abc0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:44.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.054 7ff0e5dfd700 1 -- 172.21.2.101:0/3324269203 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e00a1e00 con 0x7ff0e00a33c0 2021-02-10T11:14:44.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.056 7ff0c7fff700 1 -- 172.21.2.101:0/3324269203 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff0d0000c10 con 0x7ff0e00a33c0 2021-02-10T11:14:44.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.181 7ff0e5dfd700 1 -- 172.21.2.101:0/3324269203 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0e00a1e00 con 0x7ff0e00a33c0 2021-02-10T11:14:52.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.181 7ff0c7fff700 1 -- 172.21.2.101:0/3324269203 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+11144 (secure 0 0 0) 0x7ff0d0000e70 con 0x7ff0e00a33c0 2021-02-10T11:14:52.833 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:14:52.833 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":25,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:14:42.711296","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:14:52.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.183 7ff0e5dfd700 1 -- 172.21.2.101:0/3324269203 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0c00186c0 msgr2=0x7ff0c001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:52.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.183 7ff0e5dfd700 1 --2- 172.21.2.101:0/3324269203 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0c00186c0 0x7ff0c001abc0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:52.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.183 7ff0e5dfd700 1 -- 172.21.2.101:0/3324269203 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0e00a33c0 msgr2=0x7ff0e00a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:52.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.183 7ff0e5dfd700 1 --2- 172.21.2.101:0/3324269203 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0e00a33c0 0x7ff0e00a03d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7ff0d0006f20 tx=0x7ff0d000b990).stop 2021-02-10T11:14:52.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.183 7ff0e5dfd700 1 -- 172.21.2.101:0/3324269203 shutdown_connections 2021-02-10T11:14:52.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.183 7ff0e5dfd700 1 --2- 172.21.2.101:0/3324269203 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0c00186c0 0x7ff0c001abc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:52.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.183 7ff0e5dfd700 1 --2- 172.21.2.101:0/3324269203 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0e00a29a0 0x7ff0e00a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:52.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.183 7ff0e5dfd700 1 --2- 172.21.2.101:0/3324269203 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0e00a33c0 0x7ff0e00a03d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:52.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.183 7ff0e5dfd700 1 --2- 172.21.2.101:0/3324269203 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0e0191bb0 0x7ff0e00a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:52.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.183 7ff0e5dfd700 1 -- 172.21.2.101:0/3324269203 >> 172.21.2.101:0/3324269203 conn(0x7ff0e01a31d0 msgr2=0x7ff0e008b830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:52.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.184 7ff0e5dfd700 1 -- 172.21.2.101:0/3324269203 shutdown_connections 2021-02-10T11:14:52.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:44.184 7ff0e5dfd700 1 -- 172.21.2.101:0/3324269203 wait complete. 2021-02-10T11:14:52.835 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-10T11:14:52.835 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:14:53.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.245 7fd066528700 1 Processor -- start 2021-02-10T11:14:53.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.245 7fd066528700 1 -- start start 2021-02-10T11:14:53.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.245 7fd066528700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0601b0470 0x7fd0601b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:53.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.245 7fd066528700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd0600f20e0 0x7fd0600f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:53.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.245 7fd066528700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0600f0b50 con 0x7fd0600ef610 2021-02-10T11:14:53.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.245 7fd066528700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0600f0820 con 0x7fd0600f20e0 2021-02-10T11:14:53.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.245 7fd066528700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0600f24e0 con 0x7fd0601b0470 2021-02-10T11:14:53.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.245 7fd064b45700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd0600f20e0 0x7fd0600f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:53.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.245 7fd064b45700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd0600f20e0 0x7fd0600f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45274/0 (socket says 172.21.2.101:45274) 2021-02-10T11:14:53.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.245 7fd05f7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd0600ef610 0x7fd0600efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45770/0 (socket says 172.21.2.101:45770) 2021-02-10T11:14:53.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.246 7fd05f7fe700 1 -- 172.21.2.101:0/3840504463 learned_addr learned my addr 172.21.2.101:0/3840504463 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:14:53.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.246 7fd064b45700 1 -- 172.21.2.101:0/3840504463 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0601b0470 msgr2=0x7fd0601b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:53.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.246 7fd064b45700 1 --2- 172.21.2.101:0/3840504463 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0601b0470 0x7fd0601b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:53.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.246 7fd064b45700 1 -- 172.21.2.101:0/3840504463 >> v1:172.21.2.101:6789/0 conn(0x7fd0600ef610 legacy=0x7fd0600efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:53.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.246 7fd05effd700 1 -- 172.21.2.101:0/3840504463 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1684587960 0 0) 0x7fd0500010b0 con 0x7fd0600ef610 2021-02-10T11:14:53.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.246 7fd064b45700 1 -- 172.21.2.101:0/3840504463 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0600e52e0 con 0x7fd0600f20e0 2021-02-10T11:14:53.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.246 7fd064b45700 1 --2- 172.21.2.101:0/3840504463 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0600f20e0 0x7fd0600f0200 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fd058006e30 tx=0x7fd058009e60).ready entity=mon.0 client_cookie=fb8c0679ddce216d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:53.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.246 7fd05effd700 1 -- 172.21.2.101:0/3840504463 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd058006a50 con 0x7fd0600f20e0 2021-02-10T11:14:53.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.246 7fd05effd700 1 -- 172.21.2.101:0/3840504463 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05800b300 con 0x7fd0600f20e0 2021-02-10T11:14:53.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.246 7fd05effd700 1 -- 172.21.2.101:0/3840504463 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd05800b820 con 0x7fd0600f20e0 2021-02-10T11:14:53.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.246 7fd066528700 1 -- 172.21.2.101:0/3840504463 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0600f20e0 msgr2=0x7fd0600f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:53.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.246 7fd066528700 1 --2- 172.21.2.101:0/3840504463 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0600f20e0 0x7fd0600f0200 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fd058006e30 tx=0x7fd058009e60).stop 2021-02-10T11:14:53.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.247 7fd066528700 1 -- 172.21.2.101:0/3840504463 shutdown_connections 2021-02-10T11:14:53.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.247 7fd066528700 1 --2- 172.21.2.101:0/3840504463 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0600f20e0 0x7fd0600f0200 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:53.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.247 7fd066528700 1 --2- 172.21.2.101:0/3840504463 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0601b0470 0x7fd0601b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:53.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.247 7fd066528700 1 -- 172.21.2.101:0/3840504463 >> 172.21.2.101:0/3840504463 conn(0x7fd0600ce3e0 msgr2=0x7fd0600e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:53.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.247 7fd066528700 1 -- 172.21.2.101:0/3840504463 shutdown_connections 2021-02-10T11:14:53.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.247 7fd066528700 1 -- 172.21.2.101:0/3840504463 wait complete. 2021-02-10T11:14:53.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.262 7fd066528700 1 Processor -- start 2021-02-10T11:14:53.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.262 7fd066528700 1 -- start start 2021-02-10T11:14:53.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.262 7fd066528700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0600b20b0 0x7fd0600ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:53.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.262 7fd066528700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0600b24b0 0x7fd060191dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:53.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.262 7fd066528700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0600acf90 0x7fd0601923e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:53.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.262 7fd066528700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0600b1b30 con 0x7fd0600b24b0 2021-02-10T11:14:53.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.262 7fd066528700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0601a04e0 con 0x7fd0600b20b0 2021-02-10T11:14:53.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.262 7fd066528700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0601a06c0 con 0x7fd0600acf90 2021-02-10T11:14:53.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.262 7fd05f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0600b24b0 0x7fd060191dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:53.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.262 7fd05f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0600b24b0 0x7fd060191dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45282/0 (socket says 172.21.2.101:45282) 2021-02-10T11:14:53.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.262 7fd05f7fe700 1 -- 172.21.2.101:0/4059919874 learned_addr learned my addr 172.21.2.101:0/4059919874 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:53.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.263 7fd05f7fe700 1 -- 172.21.2.101:0/4059919874 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0600acf90 msgr2=0x7fd0601923e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:53.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.263 7fd05f7fe700 1 --2- 172.21.2.101:0/4059919874 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0600acf90 0x7fd0601923e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:53.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.263 7fd05f7fe700 1 -- 172.21.2.101:0/4059919874 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0600b20b0 msgr2=0x7fd0600ac970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:53.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.263 7fd05f7fe700 1 --2- 172.21.2.101:0/4059919874 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0600b20b0 0x7fd0600ac970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:53.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.263 7fd05f7fe700 1 -- 172.21.2.101:0/4059919874 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd060192be0 con 0x7fd0600b24b0 2021-02-10T11:14:53.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.263 7fd05f7fe700 1 --2- 172.21.2.101:0/4059919874 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0600b24b0 0x7fd060191dc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fd050001c10 tx=0x7fd05000a8f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:53.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.263 7fd05cff9700 1 -- 172.21.2.101:0/4059919874 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd0500009b0 con 0x7fd0600b24b0 2021-02-10T11:14:53.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.263 7fd05cff9700 1 -- 172.21.2.101:0/4059919874 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0500012c0 con 0x7fd0600b24b0 2021-02-10T11:14:53.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.263 7fd05cff9700 1 -- 172.21.2.101:0/4059919874 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd050004350 con 0x7fd0600b24b0 2021-02-10T11:14:53.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.263 7fd066528700 1 -- 172.21.2.101:0/4059919874 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd060192be0 con 0x7fd0600b24b0 2021-02-10T11:14:53.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.263 7fd066528700 1 -- 172.21.2.101:0/4059919874 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd060193170 con 0x7fd0600b24b0 2021-02-10T11:14:53.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.264 7fd05cff9700 1 -- 172.21.2.101:0/4059919874 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd050004140 con 0x7fd0600b24b0 2021-02-10T11:14:53.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.264 7fd05cff9700 1 --2- 172.21.2.101:0/4059919874 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd0440186c0 0x7fd04401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:53.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.264 7fd05cff9700 1 -- 172.21.2.101:0/4059919874 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fd050032230 con 0x7fd0600b24b0 2021-02-10T11:14:53.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.264 7fd05ffff700 1 --2- 172.21.2.101:0/4059919874 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd0440186c0 0x7fd04401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:53.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.264 7fd05ffff700 1 --2- 172.21.2.101:0/4059919874 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd0440186c0 0x7fd04401abc0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:53.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.295 7fd066528700 1 -- 172.21.2.101:0/4059919874 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd060105af0 con 0x7fd0600b24b0 2021-02-10T11:14:53.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.297 7fd05cff9700 1 -- 172.21.2.101:0/4059919874 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd0500009b0 con 0x7fd0600b24b0 2021-02-10T11:14:53.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.431 7fd066528700 1 -- 172.21.2.101:0/4059919874 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fd0600f0b50 con 0x7fd0600b24b0 2021-02-10T11:14:53.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.432 7fd05cff9700 1 -- 172.21.2.101:0/4059919874 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd050004140 con 0x7fd0600b24b0 2021-02-10T11:14:53.437 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:14:53.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.433 7fd066528700 1 -- 172.21.2.101:0/4059919874 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd0440186c0 msgr2=0x7fd04401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:53.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.433 7fd066528700 1 --2- 172.21.2.101:0/4059919874 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd0440186c0 0x7fd04401abc0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:53.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.434 7fd066528700 1 -- 172.21.2.101:0/4059919874 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0600b24b0 msgr2=0x7fd060191dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:53.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.434 7fd066528700 1 --2- 172.21.2.101:0/4059919874 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0600b24b0 0x7fd060191dc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fd050001c10 tx=0x7fd05000a8f0).stop 2021-02-10T11:14:53.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.434 7fd066528700 1 -- 172.21.2.101:0/4059919874 shutdown_connections 2021-02-10T11:14:53.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.434 7fd066528700 1 --2- 172.21.2.101:0/4059919874 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd0440186c0 0x7fd04401abc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:53.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.434 7fd066528700 1 --2- 172.21.2.101:0/4059919874 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0600acf90 0x7fd0601923e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:53.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.434 7fd066528700 1 --2- 172.21.2.101:0/4059919874 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0600b24b0 0x7fd060191dc0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:53.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.434 7fd066528700 1 --2- 172.21.2.101:0/4059919874 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0600b20b0 0x7fd0600ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:53.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.434 7fd066528700 1 -- 172.21.2.101:0/4059919874 >> 172.21.2.101:0/4059919874 conn(0x7fd0600bf4e0 msgr2=0x7fd0600be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:53.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.434 7fd066528700 1 -- 172.21.2.101:0/4059919874 shutdown_connections 2021-02-10T11:14:53.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.434 7fd066528700 1 -- 172.21.2.101:0/4059919874 wait complete. 2021-02-10T11:14:53.447 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --format=json-pretty osd lspools 2021-02-10T11:14:53.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.852 7fb16ce09700 1 Processor -- start 2021-02-10T11:14:53.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.852 7fb16ce09700 1 -- start start 2021-02-10T11:14:53.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.853 7fb16ce09700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1680e5070 0x7fb1681b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:53.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.853 7fb16ce09700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb1680e5470 0x7fb16807d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:53.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.853 7fb16ce09700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1681b3e40 0x7fb16807dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:53.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.853 7fb16ce09700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb16807ed80 con 0x7fb1681b3e40 2021-02-10T11:14:53.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.853 7fb16ce09700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb16807e2d0 con 0x7fb1680e5470 2021-02-10T11:14:53.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.853 7fb16ce09700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb16807e4c0 con 0x7fb1680e5070 2021-02-10T11:14:53.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.853 7fb167314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1681b3e40 0x7fb16807dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:53.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.853 7fb167314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1681b3e40 0x7fb16807dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45290/0 (socket says 172.21.2.101:45290) 2021-02-10T11:14:53.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.853 7fb167314700 1 -- 172.21.2.101:0/621777075 learned_addr learned my addr 172.21.2.101:0/621777075 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:53.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.853 7fb167314700 1 -- 172.21.2.101:0/621777075 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1680e5070 msgr2=0x7fb1681b3820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:53.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.853 7fb167314700 1 --2- 172.21.2.101:0/621777075 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1680e5070 0x7fb1681b3820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:53.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.853 7fb167314700 1 -- 172.21.2.101:0/621777075 >> v2:172.21.2.108:3300/0 conn(0x7fb1680e5470 msgr2=0x7fb16807d690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:53.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.853 7fb167314700 1 --2- 172.21.2.101:0/621777075 >> v2:172.21.2.108:3300/0 conn(0x7fb1680e5470 0x7fb16807d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:53.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.853 7fb167314700 1 -- 172.21.2.101:0/621777075 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb16807e6e0 con 0x7fb1681b3e40 2021-02-10T11:14:53.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.854 7fb167314700 1 --2- 172.21.2.101:0/621777075 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1681b3e40 0x7fb16807dcb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fb160006f00 tx=0x7fb160009f10).ready entity=mon.0 client_cookie=5232a773ae55dccc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:53.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.854 7fb165b11700 1 -- 172.21.2.101:0/621777075 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb16000b3b0 con 0x7fb1681b3e40 2021-02-10T11:14:53.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.854 7fb165b11700 1 -- 172.21.2.101:0/621777075 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16000b630 con 0x7fb1681b3e40 2021-02-10T11:14:53.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.854 7fb165b11700 1 -- 172.21.2.101:0/621777075 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb16000bb50 con 0x7fb1681b3e40 2021-02-10T11:14:53.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.854 7fb16ce09700 1 -- 172.21.2.101:0/621777075 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1681b3e40 msgr2=0x7fb16807dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:53.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.854 7fb16ce09700 1 --2- 172.21.2.101:0/621777075 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1681b3e40 0x7fb16807dcb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fb160006f00 tx=0x7fb160009f10).stop 2021-02-10T11:14:53.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.854 7fb16ce09700 1 -- 172.21.2.101:0/621777075 shutdown_connections 2021-02-10T11:14:53.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.854 7fb16ce09700 1 --2- 172.21.2.101:0/621777075 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1681b3e40 0x7fb16807dcb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:53.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.854 7fb16ce09700 1 --2- 172.21.2.101:0/621777075 >> v2:172.21.2.108:3300/0 conn(0x7fb1680e5470 0x7fb16807d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:53.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.855 7fb16ce09700 1 --2- 172.21.2.101:0/621777075 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1680e5070 0x7fb1681b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:53.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.855 7fb16ce09700 1 -- 172.21.2.101:0/621777075 >> 172.21.2.101:0/621777075 conn(0x7fb1680ce690 msgr2=0x7fb1680e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:53.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.855 7fb16ce09700 1 -- 172.21.2.101:0/621777075 shutdown_connections 2021-02-10T11:14:53.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.855 7fb16ce09700 1 -- 172.21.2.101:0/621777075 wait complete. 2021-02-10T11:14:53.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.869 7fb16ce09700 1 Processor -- start 2021-02-10T11:14:53.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.869 7fb16ce09700 1 -- start start 2021-02-10T11:14:53.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.870 7fb16ce09700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1681a6ab0 0x7fb1681a6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:53.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.870 7fb16ce09700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1681a74d0 0x7fb1681a8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:53.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.870 7fb16ce09700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1681a78d0 0x7fb1681a8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:53.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.870 7fb16ce09700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1681a7f00 con 0x7fb1681a78d0 2021-02-10T11:14:53.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.870 7fb16ce09700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb168080ab0 con 0x7fb1681a74d0 2021-02-10T11:14:53.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.870 7fb16ce09700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb168080580 con 0x7fb1681a6ab0 2021-02-10T11:14:53.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.870 7fb167314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1681a78d0 0x7fb1681a8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:53.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.870 7fb167314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1681a78d0 0x7fb1681a8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45292/0 (socket says 172.21.2.101:45292) 2021-02-10T11:14:53.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.870 7fb167314700 1 -- 172.21.2.101:0/2309034182 learned_addr learned my addr 172.21.2.101:0/2309034182 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:53.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.870 7fb167314700 1 -- 172.21.2.101:0/2309034182 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1681a6ab0 msgr2=0x7fb1681a6eb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:53.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.870 7fb167314700 1 --2- 172.21.2.101:0/2309034182 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1681a6ab0 0x7fb1681a6eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:53.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.870 7fb167314700 1 -- 172.21.2.101:0/2309034182 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1681a74d0 msgr2=0x7fb1681a8400 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:53.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.870 7fb167314700 1 --2- 172.21.2.101:0/2309034182 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1681a74d0 0x7fb1681a8400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:53.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.870 7fb167314700 1 -- 172.21.2.101:0/2309034182 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1681a9040 con 0x7fb1681a78d0 2021-02-10T11:14:53.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.870 7fb167314700 1 --2- 172.21.2.101:0/2309034182 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1681a78d0 0x7fb1681a8a20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fb160009390 tx=0x7fb16000bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:53.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.870 7fb15f7fe700 1 -- 172.21.2.101:0/2309034182 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb160009450 con 0x7fb1681a78d0 2021-02-10T11:14:53.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.870 7fb15f7fe700 1 -- 172.21.2.101:0/2309034182 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb160003030 con 0x7fb1681a78d0 2021-02-10T11:14:53.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.870 7fb16ce09700 1 -- 172.21.2.101:0/2309034182 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1681a9040 con 0x7fb1681a78d0 2021-02-10T11:14:53.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.870 7fb15f7fe700 1 -- 172.21.2.101:0/2309034182 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb160001480 con 0x7fb1681a78d0 2021-02-10T11:14:53.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.871 7fb16ce09700 1 -- 172.21.2.101:0/2309034182 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1681a95d0 con 0x7fb1681a78d0 2021-02-10T11:14:53.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.871 7fb15f7fe700 1 -- 172.21.2.101:0/2309034182 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb160005820 con 0x7fb1681a78d0 2021-02-10T11:14:53.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.871 7fb15f7fe700 1 --2- 172.21.2.101:0/2309034182 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb14c0186c0 0x7fb14c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:53.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.871 7fb15f7fe700 1 -- 172.21.2.101:0/2309034182 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fb1600325b0 con 0x7fb1681a78d0 2021-02-10T11:14:53.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.871 7fb166b13700 1 --2- 172.21.2.101:0/2309034182 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb14c0186c0 0x7fb14c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:53.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.872 7fb166b13700 1 --2- 172.21.2.101:0/2309034182 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb14c0186c0 0x7fb14c01abc0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:53.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.903 7fb16ce09700 1 -- 172.21.2.101:0/2309034182 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb168107120 con 0x7fb1681a78d0 2021-02-10T11:14:53.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:53.905 7fb15f7fe700 1 -- 172.21.2.101:0/2309034182 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb160009450 con 0x7fb1681a78d0 2021-02-10T11:14:54.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.029 7fb16ce09700 1 -- 172.21.2.101:0/2309034182 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb16807ed80 con 0x7fb1681a78d0 2021-02-10T11:14:54.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.029 7fb15f7fe700 1 -- 172.21.2.101:0/2309034182 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v25) v1 ==== 84+0+64 (secure 0 0 0) 0x7fb160005820 con 0x7fb1681a78d0 2021-02-10T11:14:54.034 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:14:54.034 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:14:54.035 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:14:54.035 INFO:teuthology.orchestra.run.gibba001.stdout: "poolnum": 1, 2021-02-10T11:14:54.035 INFO:teuthology.orchestra.run.gibba001.stdout: "poolname": "rbd" 2021-02-10T11:14:54.035 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:14:54.035 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:14:54.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.031 7fb16ce09700 1 -- 172.21.2.101:0/2309034182 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb14c0186c0 msgr2=0x7fb14c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:54.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.031 7fb16ce09700 1 --2- 172.21.2.101:0/2309034182 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb14c0186c0 0x7fb14c01abc0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:54.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.031 7fb16ce09700 1 -- 172.21.2.101:0/2309034182 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1681a78d0 msgr2=0x7fb1681a8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:54.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.031 7fb16ce09700 1 --2- 172.21.2.101:0/2309034182 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1681a78d0 0x7fb1681a8a20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fb160009390 tx=0x7fb16000bd40).stop 2021-02-10T11:14:54.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.032 7fb16ce09700 1 -- 172.21.2.101:0/2309034182 shutdown_connections 2021-02-10T11:14:54.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.032 7fb16ce09700 1 --2- 172.21.2.101:0/2309034182 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1681a78d0 0x7fb1681a8a20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:54.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.032 7fb16ce09700 1 --2- 172.21.2.101:0/2309034182 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1681a74d0 0x7fb1681a8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:54.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.032 7fb16ce09700 1 --2- 172.21.2.101:0/2309034182 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb14c0186c0 0x7fb14c01abc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:54.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.032 7fb16ce09700 1 --2- 172.21.2.101:0/2309034182 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1681a6ab0 0x7fb1681a6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:54.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.032 7fb16ce09700 1 -- 172.21.2.101:0/2309034182 >> 172.21.2.101:0/2309034182 conn(0x7fb1681ac620 msgr2=0x7fb1681a0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:54.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.032 7fb16ce09700 1 -- 172.21.2.101:0/2309034182 shutdown_connections 2021-02-10T11:14:54.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.032 7fb16ce09700 1 -- 172.21.2.101:0/2309034182 wait complete. 2021-02-10T11:14:54.045 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-02-10T11:14:54.045 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-02-10T11:14:54.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.453 7f3a9c4ac700 1 Processor -- start 2021-02-10T11:14:54.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.453 7f3a9c4ac700 1 -- start start 2021-02-10T11:14:54.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.453 7f3a9c4ac700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a941b0470 0x7f3a941b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:54.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.453 7f3a9c4ac700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3a940ef610 0x7f3a940efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:54.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.453 7f3a9c4ac700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a940f0b50 con 0x7f3a940f0030 2021-02-10T11:14:54.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.453 7f3a9c4ac700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a940e52e0 con 0x7f3a940ef610 2021-02-10T11:14:54.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.453 7f3a9c4ac700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a940e5490 con 0x7f3a941b0470 2021-02-10T11:14:54.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.454 7f3a9aac9700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f3a940f0030 0x7f3a940f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45794/0 (socket says 172.21.2.101:45794) 2021-02-10T11:14:54.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.454 7f3a99ac7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3a940ef610 0x7f3a940efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:54.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.454 7f3a9aac9700 1 -- 172.21.2.101:0/4214117879 learned_addr learned my addr 172.21.2.101:0/4214117879 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:14:54.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.454 7f3a9a2c8700 1 --2- 172.21.2.101:0/4214117879 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a941b0470 0x7f3a941b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:54.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.454 7f3a99ac7700 1 -- 172.21.2.101:0/4214117879 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a941b0470 msgr2=0x7f3a941b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:54.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.454 7f3a99ac7700 1 --2- 172.21.2.101:0/4214117879 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a941b0470 0x7f3a941b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:54.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.454 7f3a99ac7700 1 -- 172.21.2.101:0/4214117879 >> v1:172.21.2.101:6789/0 conn(0x7f3a940f0030 legacy=0x7f3a940f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:54.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.454 7f3a99ac7700 1 -- 172.21.2.101:0/4214117879 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a940f0430 con 0x7f3a940ef610 2021-02-10T11:14:54.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.454 7f3a992c6700 1 -- 172.21.2.101:0/4214117879 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1463007127 0 0) 0x7f3a8c000f90 con 0x7f3a940f0030 2021-02-10T11:14:54.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.454 7f3a9a2c8700 1 --2- 172.21.2.101:0/4214117879 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a941b0470 0x7f3a941b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:14:54.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.454 7f3a99ac7700 1 --2- 172.21.2.101:0/4214117879 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a940ef610 0x7f3a940efa10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f3a90006650 tx=0x7f3a90009a80).ready entity=mon.0 client_cookie=d5469475446275b9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:54.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.454 7f3a992c6700 1 -- 172.21.2.101:0/4214117879 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3a90006390 con 0x7f3a940ef610 2021-02-10T11:14:54.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.454 7f3a992c6700 1 -- 172.21.2.101:0/4214117879 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a90006740 con 0x7f3a940ef610 2021-02-10T11:14:54.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.454 7f3a992c6700 1 -- 172.21.2.101:0/4214117879 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3a90006390 con 0x7f3a940ef610 2021-02-10T11:14:54.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.454 7f3a9c4ac700 1 -- 172.21.2.101:0/4214117879 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a940ef610 msgr2=0x7f3a940efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:54.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.454 7f3a9c4ac700 1 --2- 172.21.2.101:0/4214117879 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a940ef610 0x7f3a940efa10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f3a90006650 tx=0x7f3a90009a80).stop 2021-02-10T11:14:54.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.455 7f3a9c4ac700 1 -- 172.21.2.101:0/4214117879 shutdown_connections 2021-02-10T11:14:54.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.455 7f3a9c4ac700 1 --2- 172.21.2.101:0/4214117879 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a940ef610 0x7f3a940efa10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:54.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.455 7f3a9c4ac700 1 --2- 172.21.2.101:0/4214117879 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a941b0470 0x7f3a941b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:54.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.455 7f3a9c4ac700 1 -- 172.21.2.101:0/4214117879 >> 172.21.2.101:0/4214117879 conn(0x7f3a940ce3e0 msgr2=0x7f3a940e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:54.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.455 7f3a9c4ac700 1 -- 172.21.2.101:0/4214117879 shutdown_connections 2021-02-10T11:14:54.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.455 7f3a9c4ac700 1 -- 172.21.2.101:0/4214117879 wait complete. 2021-02-10T11:14:54.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.470 7f3a9c4ac700 1 Processor -- start 2021-02-10T11:14:54.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.470 7f3a9c4ac700 1 -- start start 2021-02-10T11:14:54.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.470 7f3a9c4ac700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a940a0420 0x7f3a940a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:54.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.470 7f3a9c4ac700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3a94189d00 0x7f3a9418a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:54.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.470 7f3a9c4ac700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a9418a720 0x7f3a940b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:54.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.470 7f3a9c4ac700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a94189800 con 0x7f3a940a0420 2021-02-10T11:14:54.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.470 7f3a9c4ac700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9418ab20 con 0x7f3a94189d00 2021-02-10T11:14:54.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.470 7f3a9c4ac700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a940a1140 con 0x7f3a9418a720 2021-02-10T11:14:54.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.470 7f3a9a2c8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a940a0420 0x7f3a940a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:54.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.470 7f3a9a2c8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a940a0420 0x7f3a940a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45308/0 (socket says 172.21.2.101:45308) 2021-02-10T11:14:54.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.470 7f3a9a2c8700 1 -- 172.21.2.101:0/1518751471 learned_addr learned my addr 172.21.2.101:0/1518751471 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:54.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.470 7f3a9aac9700 1 --2- 172.21.2.101:0/1518751471 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a9418a720 0x7f3a940b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:54.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.471 7f3a99ac7700 1 --2- 172.21.2.101:0/1518751471 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3a94189d00 0x7f3a9418a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:54.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.471 7f3a9a2c8700 1 -- 172.21.2.101:0/1518751471 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a9418a720 msgr2=0x7f3a940b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:54.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.471 7f3a9a2c8700 1 --2- 172.21.2.101:0/1518751471 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a9418a720 0x7f3a940b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:54.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.471 7f3a9a2c8700 1 -- 172.21.2.101:0/1518751471 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3a94189d00 msgr2=0x7f3a9418a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:54.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.471 7f3a9a2c8700 1 --2- 172.21.2.101:0/1518751471 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3a94189d00 0x7f3a9418a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:54.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.471 7f3a9a2c8700 1 -- 172.21.2.101:0/1518751471 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a940b8200 con 0x7f3a940a0420 2021-02-10T11:14:54.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.471 7f3a9a2c8700 1 --2- 172.21.2.101:0/1518751471 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a940a0420 0x7f3a940a0820 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f3a88006f20 tx=0x7f3a8800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:54.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.471 7f3a86ffd700 1 -- 172.21.2.101:0/1518751471 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3a8800bd80 con 0x7f3a940a0420 2021-02-10T11:14:54.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.471 7f3a86ffd700 1 -- 172.21.2.101:0/1518751471 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a88000c90 con 0x7f3a940a0420 2021-02-10T11:14:54.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.471 7f3a86ffd700 1 -- 172.21.2.101:0/1518751471 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3a880022b0 con 0x7f3a940a0420 2021-02-10T11:14:54.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.471 7f3a9c4ac700 1 -- 172.21.2.101:0/1518751471 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a940b8200 con 0x7f3a940a0420 2021-02-10T11:14:54.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.471 7f3a9c4ac700 1 -- 172.21.2.101:0/1518751471 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a940b8200 con 0x7f3a940a0420 2021-02-10T11:14:54.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.472 7f3a86ffd700 1 -- 172.21.2.101:0/1518751471 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3a8800a000 con 0x7f3a940a0420 2021-02-10T11:14:54.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.472 7f3a86ffd700 1 --2- 172.21.2.101:0/1518751471 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a7c0186c0 0x7f3a7c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:54.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.472 7f3a86ffd700 1 -- 172.21.2.101:0/1518751471 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f3a880322e0 con 0x7f3a940a0420 2021-02-10T11:14:54.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.472 7f3a99ac7700 1 --2- 172.21.2.101:0/1518751471 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a7c0186c0 0x7f3a7c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:54.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.472 7f3a99ac7700 1 --2- 172.21.2.101:0/1518751471 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a7c0186c0 0x7f3a7c01abc0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:54.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.503 7f3a9c4ac700 1 -- 172.21.2.101:0/1518751471 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a94105ae0 con 0x7f3a940a0420 2021-02-10T11:14:54.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.506 7f3a86ffd700 1 -- 172.21.2.101:0/1518751471 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3a88006bb0 con 0x7f3a940a0420 2021-02-10T11:14:54.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:54.635 7f3a9c4ac700 1 -- 172.21.2.101:0/1518751471 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"} v 0) v1 -- 0x7f3a940f0b50 con 0x7f3a940a0420 2021-02-10T11:14:55.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.494 7f3a86ffd700 1 -- 172.21.2.101:0/1518751471 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"}]=0 pool 'cephfs_metadata' created v26) v1 ==== 133+0+0 (secure 0 0 0) 0x7f3a8800a000 con 0x7f3a940a0420 2021-02-10T11:14:55.509 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' created 2021-02-10T11:14:55.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.506 7f3a9c4ac700 1 -- 172.21.2.101:0/1518751471 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a7c0186c0 msgr2=0x7f3a7c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:55.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.506 7f3a9c4ac700 1 --2- 172.21.2.101:0/1518751471 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a7c0186c0 0x7f3a7c01abc0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:55.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.506 7f3a9c4ac700 1 -- 172.21.2.101:0/1518751471 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a940a0420 msgr2=0x7f3a940a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:55.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.506 7f3a9c4ac700 1 --2- 172.21.2.101:0/1518751471 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a940a0420 0x7f3a940a0820 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f3a88006f20 tx=0x7f3a8800b990).stop 2021-02-10T11:14:55.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.506 7f3a9c4ac700 1 -- 172.21.2.101:0/1518751471 shutdown_connections 2021-02-10T11:14:55.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.506 7f3a9c4ac700 1 --2- 172.21.2.101:0/1518751471 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a7c0186c0 0x7f3a7c01abc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:55.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.506 7f3a9c4ac700 1 --2- 172.21.2.101:0/1518751471 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a9418a720 0x7f3a940b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:55.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.506 7f3a9c4ac700 1 --2- 172.21.2.101:0/1518751471 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3a94189d00 0x7f3a9418a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:55.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.506 7f3a9c4ac700 1 --2- 172.21.2.101:0/1518751471 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a940a0420 0x7f3a940a0820 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:55.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.506 7f3a9c4ac700 1 -- 172.21.2.101:0/1518751471 >> 172.21.2.101:0/1518751471 conn(0x7f3a94094e10 msgr2=0x7f3a940a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:55.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.506 7f3a9c4ac700 1 -- 172.21.2.101:0/1518751471 shutdown_connections 2021-02-10T11:14:55.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.506 7f3a9c4ac700 1 -- 172.21.2.101:0/1518751471 wait complete. 2021-02-10T11:14:55.519 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_data 8 2021-02-10T11:14:55.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.927 7f2b7003b700 1 Processor -- start 2021-02-10T11:14:55.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.927 7f2b7003b700 1 -- start start 2021-02-10T11:14:55.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.927 7f2b7003b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2b681b0470 0x7f2b681b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:55.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.927 7f2b7003b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2b680f20e0 0x7f2b680f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:55.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.927 7f2b7003b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b680f0b50 con 0x7f2b680ef610 2021-02-10T11:14:55.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.927 7f2b7003b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b680f0820 con 0x7f2b680f20e0 2021-02-10T11:14:55.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.927 7f2b7003b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b680f24e0 con 0x7f2b681b0470 2021-02-10T11:14:55.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.927 7f2b6e658700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2b680f20e0 0x7f2b680f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:55.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.927 7f2b6e658700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2b680f20e0 0x7f2b680f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45314/0 (socket says 172.21.2.101:45314) 2021-02-10T11:14:55.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.927 7f2b6e658700 1 -- 172.21.2.101:0/4169035697 learned_addr learned my addr 172.21.2.101:0/4169035697 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:55.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.928 7f2b6e658700 1 -- 172.21.2.101:0/4169035697 >> v2:172.21.2.108:3300/0 conn(0x7f2b681b0470 msgr2=0x7f2b681b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:55.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.928 7f2b6e658700 1 --2- 172.21.2.101:0/4169035697 >> v2:172.21.2.108:3300/0 conn(0x7f2b681b0470 0x7f2b681b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:55.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.928 7f2b6e658700 1 -- 172.21.2.101:0/4169035697 >> v1:172.21.2.101:6789/0 conn(0x7f2b680ef610 legacy=0x7f2b680efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:55.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.928 7f2b6e658700 1 -- 172.21.2.101:0/4169035697 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b680e52e0 con 0x7f2b680f20e0 2021-02-10T11:14:55.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.928 7f2b6e658700 1 --2- 172.21.2.101:0/4169035697 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b680f20e0 0x7f2b680f0200 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f2b60006f00 tx=0x7f2b60009f10).ready entity=mon.0 client_cookie=a7131f7794a3f79c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:55.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.928 7f2b6ce55700 1 -- 172.21.2.101:0/4169035697 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b6000b3b0 con 0x7f2b680f20e0 2021-02-10T11:14:55.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.928 7f2b6ce55700 1 -- 172.21.2.101:0/4169035697 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6000b730 con 0x7f2b680f20e0 2021-02-10T11:14:55.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.928 7f2b6ce55700 1 -- 172.21.2.101:0/4169035697 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b6000bb90 con 0x7f2b680f20e0 2021-02-10T11:14:55.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.928 7f2b7003b700 1 -- 172.21.2.101:0/4169035697 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b680f20e0 msgr2=0x7f2b680f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:55.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.928 7f2b7003b700 1 --2- 172.21.2.101:0/4169035697 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b680f20e0 0x7f2b680f0200 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f2b60006f00 tx=0x7f2b60009f10).stop 2021-02-10T11:14:55.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.928 7f2b7003b700 1 -- 172.21.2.101:0/4169035697 shutdown_connections 2021-02-10T11:14:55.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.928 7f2b7003b700 1 --2- 172.21.2.101:0/4169035697 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b680f20e0 0x7f2b680f0200 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:55.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.928 7f2b7003b700 1 --2- 172.21.2.101:0/4169035697 >> v2:172.21.2.108:3300/0 conn(0x7f2b681b0470 0x7f2b681b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:55.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.928 7f2b7003b700 1 -- 172.21.2.101:0/4169035697 >> 172.21.2.101:0/4169035697 conn(0x7f2b680ce3e0 msgr2=0x7f2b680e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:55.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.929 7f2b7003b700 1 -- 172.21.2.101:0/4169035697 shutdown_connections 2021-02-10T11:14:55.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.929 7f2b7003b700 1 -- 172.21.2.101:0/4169035697 wait complete. 2021-02-10T11:14:55.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.943 7f2b7003b700 1 Processor -- start 2021-02-10T11:14:55.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.943 7f2b7003b700 1 -- start start 2021-02-10T11:14:55.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.944 7f2b7003b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b680c36a0 0x7f2b680c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:55.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.944 7f2b7003b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b680c3190 0x7f2b680b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:55.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.944 7f2b7003b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b680b2370 0x7f2b680b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:55.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.944 7f2b7003b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b680b27c0 con 0x7f2b680c3190 2021-02-10T11:14:55.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.944 7f2b7003b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b680b3360 con 0x7f2b680c36a0 2021-02-10T11:14:55.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.944 7f2b7003b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b680b3540 con 0x7f2b680b2370 2021-02-10T11:14:55.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.944 7f2b6d656700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b680c3190 0x7f2b680b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:55.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.944 7f2b6d656700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b680c3190 0x7f2b680b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45318/0 (socket says 172.21.2.101:45318) 2021-02-10T11:14:55.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.944 7f2b6d656700 1 -- 172.21.2.101:0/2114401167 learned_addr learned my addr 172.21.2.101:0/2114401167 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:55.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.944 7f2b6d656700 1 -- 172.21.2.101:0/2114401167 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b680b2370 msgr2=0x7f2b680b2d40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:55.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.944 7f2b6d656700 1 --2- 172.21.2.101:0/2114401167 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b680b2370 0x7f2b680b2d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:55.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.944 7f2b6d656700 1 -- 172.21.2.101:0/2114401167 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b680c36a0 msgr2=0x7f2b680c2b70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:55.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.944 7f2b6d656700 1 --2- 172.21.2.101:0/2114401167 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b680c36a0 0x7f2b680c2b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:55.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.944 7f2b6d656700 1 -- 172.21.2.101:0/2114401167 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b680b3760 con 0x7f2b680c3190 2021-02-10T11:14:55.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.944 7f2b6d656700 1 --2- 172.21.2.101:0/2114401167 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b680c3190 0x7f2b680b1d50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f2b58006ed0 tx=0x7f2b5800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:55.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.944 7f2b5e7fc700 1 -- 172.21.2.101:0/2114401167 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b58006020 con 0x7f2b680c3190 2021-02-10T11:14:55.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.944 7f2b5e7fc700 1 -- 172.21.2.101:0/2114401167 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b580062a0 con 0x7f2b680c3190 2021-02-10T11:14:55.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.944 7f2b5e7fc700 1 -- 172.21.2.101:0/2114401167 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b58005730 con 0x7f2b680c3190 2021-02-10T11:14:55.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.944 7f2b7003b700 1 -- 172.21.2.101:0/2114401167 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b680b3760 con 0x7f2b680c3190 2021-02-10T11:14:55.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.944 7f2b7003b700 1 -- 172.21.2.101:0/2114401167 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b680b3760 con 0x7f2b680c3190 2021-02-10T11:14:55.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.945 7f2b5e7fc700 1 -- 172.21.2.101:0/2114401167 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2b580054b0 con 0x7f2b680c3190 2021-02-10T11:14:55.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.945 7f2b5e7fc700 1 --2- 172.21.2.101:0/2114401167 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b500186c0 0x7f2b5001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:55.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.945 7f2b5e7fc700 1 -- 172.21.2.101:0/2114401167 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5107+0+0 (secure 0 0 0) 0x7f2b580323d0 con 0x7f2b680c3190 2021-02-10T11:14:55.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.945 7f2b6de57700 1 --2- 172.21.2.101:0/2114401167 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b500186c0 0x7f2b5001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:55.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.945 7f2b6de57700 1 --2- 172.21.2.101:0/2114401167 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b500186c0 0x7f2b5001abc0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:55.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.977 7f2b7003b700 1 -- 172.21.2.101:0/2114401167 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b680edd10 con 0x7f2b680c3190 2021-02-10T11:14:55.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:55.980 7f2b5e7fc700 1 -- 172.21.2.101:0/2114401167 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2b580009b0 con 0x7f2b680c3190 2021-02-10T11:14:56.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.107 7f2b7003b700 1 -- 172.21.2.101:0/2114401167 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"} v 0) v1 -- 0x7f2b680e52e0 con 0x7f2b680c3190 2021-02-10T11:14:56.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.496 7f2b5e7fc700 1 -- 172.21.2.101:0/2114401167 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"}]=0 pool 'cephfs_data' created v27) v1 ==== 125+0+0 (secure 0 0 0) 0x7f2b580054b0 con 0x7f2b680c3190 2021-02-10T11:14:56.527 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' created 2021-02-10T11:14:56.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.524 7f2b7003b700 1 -- 172.21.2.101:0/2114401167 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b500186c0 msgr2=0x7f2b5001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:56.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.524 7f2b7003b700 1 --2- 172.21.2.101:0/2114401167 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b500186c0 0x7f2b5001abc0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:56.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.524 7f2b7003b700 1 -- 172.21.2.101:0/2114401167 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b680c3190 msgr2=0x7f2b680b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:56.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.524 7f2b7003b700 1 --2- 172.21.2.101:0/2114401167 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b680c3190 0x7f2b680b1d50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f2b58006ed0 tx=0x7f2b5800b960).stop 2021-02-10T11:14:56.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.524 7f2b7003b700 1 -- 172.21.2.101:0/2114401167 shutdown_connections 2021-02-10T11:14:56.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.524 7f2b7003b700 1 --2- 172.21.2.101:0/2114401167 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b500186c0 0x7f2b5001abc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:56.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.524 7f2b7003b700 1 --2- 172.21.2.101:0/2114401167 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b680b2370 0x7f2b680b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:56.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.524 7f2b7003b700 1 --2- 172.21.2.101:0/2114401167 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b680c3190 0x7f2b680b1d50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:56.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.524 7f2b7003b700 1 --2- 172.21.2.101:0/2114401167 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b680c36a0 0x7f2b680c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:56.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.524 7f2b7003b700 1 -- 172.21.2.101:0/2114401167 >> 172.21.2.101:0/2114401167 conn(0x7f2b6809f630 msgr2=0x7f2b680a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:56.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.524 7f2b7003b700 1 -- 172.21.2.101:0/2114401167 shutdown_connections 2021-02-10T11:14:56.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.524 7f2b7003b700 1 -- 172.21.2.101:0/2114401167 wait complete. 2021-02-10T11:14:56.538 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-02-10T11:14:56.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.948 7fe9bdb14700 1 Processor -- start 2021-02-10T11:14:56.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.948 7fe9bdb14700 1 -- start start 2021-02-10T11:14:56.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.948 7fe9bdb14700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe9b80d1870 0x7fe9b81b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:56.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.948 7fe9bdb14700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9b81b0ae0 0x7fe9b8108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:56.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.948 7fe9bdb14700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b80fe1c0 con 0x7fe9b80d1c70 2021-02-10T11:14:56.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.948 7fe9bdb14700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b8109a60 con 0x7fe9b80d1870 2021-02-10T11:14:56.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.948 7fe9bdb14700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b8109750 con 0x7fe9b81b0ae0 2021-02-10T11:14:56.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.948 7fe9b77fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe9b80d1870 0x7fe9b81b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:56.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.948 7fe9b77fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe9b80d1870 0x7fe9b81b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45326/0 (socket says 172.21.2.101:45326) 2021-02-10T11:14:56.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.948 7fe9b77fe700 1 -- 172.21.2.101:0/3442771435 learned_addr learned my addr 172.21.2.101:0/3442771435 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:56.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.949 7fe9b77fe700 1 -- 172.21.2.101:0/3442771435 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9b81b0ae0 msgr2=0x7fe9b8108940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:56.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.949 7fe9b77fe700 1 --2- 172.21.2.101:0/3442771435 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9b81b0ae0 0x7fe9b8108940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:56.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.949 7fe9b77fe700 1 -- 172.21.2.101:0/3442771435 >> v1:172.21.2.108:6789/0 conn(0x7fe9b80d1c70 legacy=0x7fe9b8108f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:56.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.949 7fe9b77fe700 1 -- 172.21.2.101:0/3442771435 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b8088020 con 0x7fe9b80d1870 2021-02-10T11:14:56.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.949 7fe9b77fe700 1 --2- 172.21.2.101:0/3442771435 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9b80d1870 0x7fe9b81b04c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7fe9a8006700 tx=0x7fe9a8009b00).ready entity=mon.0 client_cookie=d6616d29d309fbe server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:56.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.949 7fe9b67fc700 1 -- 172.21.2.101:0/3442771435 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe9a800c0c0 con 0x7fe9b80d1870 2021-02-10T11:14:56.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.949 7fe9b67fc700 1 -- 172.21.2.101:0/3442771435 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a800c340 con 0x7fe9b80d1870 2021-02-10T11:14:56.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.949 7fe9b67fc700 1 -- 172.21.2.101:0/3442771435 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe9a800c820 con 0x7fe9b80d1870 2021-02-10T11:14:56.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.949 7fe9bdb14700 1 -- 172.21.2.101:0/3442771435 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9b80d1870 msgr2=0x7fe9b81b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:56.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.949 7fe9bdb14700 1 --2- 172.21.2.101:0/3442771435 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9b80d1870 0x7fe9b81b04c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7fe9a8006700 tx=0x7fe9a8009b00).stop 2021-02-10T11:14:56.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.950 7fe9bdb14700 1 -- 172.21.2.101:0/3442771435 shutdown_connections 2021-02-10T11:14:56.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.950 7fe9bdb14700 1 --2- 172.21.2.101:0/3442771435 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9b81b0ae0 0x7fe9b8108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:56.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.950 7fe9bdb14700 1 --2- 172.21.2.101:0/3442771435 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9b80d1870 0x7fe9b81b04c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:56.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.950 7fe9bdb14700 1 -- 172.21.2.101:0/3442771435 >> 172.21.2.101:0/3442771435 conn(0x7fe9b80ce420 msgr2=0x7fe9b80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:56.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.950 7fe9bdb14700 1 -- 172.21.2.101:0/3442771435 shutdown_connections 2021-02-10T11:14:56.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.950 7fe9bdb14700 1 -- 172.21.2.101:0/3442771435 wait complete. 2021-02-10T11:14:56.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.964 7fe9bdb14700 1 Processor -- start 2021-02-10T11:14:56.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.964 7fe9bdb14700 1 -- start start 2021-02-10T11:14:56.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.965 7fe9bdb14700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9b80b7d30 0x7fe9b80b8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:56.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.965 7fe9bdb14700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe9b8095780 0x7fe9b8095b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:56.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.965 7fe9bdb14700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9b80961a0 0x7fe9b80965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:56.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.965 7fe9bdb14700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b819f110 con 0x7fe9b80961a0 2021-02-10T11:14:56.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.965 7fe9bdb14700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b81abb60 con 0x7fe9b8095780 2021-02-10T11:14:56.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.965 7fe9bdb14700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b8194a70 con 0x7fe9b80b7d30 2021-02-10T11:14:56.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.965 7fe9b7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9b80961a0 0x7fe9b80965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:56.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.965 7fe9b7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9b80961a0 0x7fe9b80965a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45330/0 (socket says 172.21.2.101:45330) 2021-02-10T11:14:56.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.965 7fe9b7fff700 1 -- 172.21.2.101:0/1880811825 learned_addr learned my addr 172.21.2.101:0/1880811825 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:56.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.965 7fe9b7fff700 1 -- 172.21.2.101:0/1880811825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9b80b7d30 msgr2=0x7fe9b80b8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:56.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.965 7fe9b7fff700 1 --2- 172.21.2.101:0/1880811825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9b80b7d30 0x7fe9b80b8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:56.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.965 7fe9b7fff700 1 -- 172.21.2.101:0/1880811825 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe9b8095780 msgr2=0x7fe9b8095b80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:56.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.965 7fe9b7fff700 1 --2- 172.21.2.101:0/1880811825 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe9b8095780 0x7fe9b8095b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:56.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.965 7fe9b7fff700 1 -- 172.21.2.101:0/1880811825 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b8194c90 con 0x7fe9b80961a0 2021-02-10T11:14:56.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.965 7fe9b7fff700 1 --2- 172.21.2.101:0/1880811825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9b80961a0 0x7fe9b80965a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7fe9b000b8e0 tx=0x7fe9b000bc80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:56.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.966 7fe9affff700 1 -- 172.21.2.101:0/1880811825 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe9b0003030 con 0x7fe9b80961a0 2021-02-10T11:14:56.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.966 7fe9affff700 1 -- 172.21.2.101:0/1880811825 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b0001270 con 0x7fe9b80961a0 2021-02-10T11:14:56.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.966 7fe9affff700 1 -- 172.21.2.101:0/1880811825 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe9b0001790 con 0x7fe9b80961a0 2021-02-10T11:14:56.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.966 7fe9bdb14700 1 -- 172.21.2.101:0/1880811825 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b8194c90 con 0x7fe9b80961a0 2021-02-10T11:14:56.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.966 7fe9bdb14700 1 -- 172.21.2.101:0/1880811825 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b8195220 con 0x7fe9b80961a0 2021-02-10T11:14:56.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.966 7fe9affff700 1 -- 172.21.2.101:0/1880811825 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe9b0001450 con 0x7fe9b80961a0 2021-02-10T11:14:56.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.966 7fe9affff700 1 --2- 172.21.2.101:0/1880811825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe99c0186c0 0x7fe99c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:56.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.966 7fe9affff700 1 -- 172.21.2.101:0/1880811825 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5457+0+0 (secure 0 0 0) 0x7fe9b0032870 con 0x7fe9b80961a0 2021-02-10T11:14:56.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.967 7fe9b77fe700 1 --2- 172.21.2.101:0/1880811825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe99c0186c0 0x7fe99c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:56.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.967 7fe9b77fe700 1 --2- 172.21.2.101:0/1880811825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe99c0186c0 0x7fe99c01abc0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:57.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:56.998 7fe9bdb14700 1 -- 172.21.2.101:0/1880811825 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b81961b0 con 0x7fe9b80961a0 2021-02-10T11:14:57.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.001 7fe9affff700 1 -- 172.21.2.101:0/1880811825 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe9b0000f90 con 0x7fe9b80961a0 2021-02-10T11:14:57.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.128 7fe9bdb14700 1 -- 172.21.2.101:0/1880811825 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"} v 0) v1 -- 0x7fe9b81961b0 con 0x7fe9b80961a0 2021-02-10T11:14:57.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.133 7fe9affff700 1 -- 172.21.2.101:0/1880811825 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"}]=0 new fs with metadata pool 4 and data pool 5 v10) v1 ==== 187+0+0 (secure 0 0 0) 0x7fe9b001f820 con 0x7fe9b80961a0 2021-02-10T11:14:57.140 INFO:teuthology.orchestra.run.gibba001.stderr:new fs with metadata pool 4 and data pool 5 2021-02-10T11:14:57.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.138 7fe9bdb14700 1 -- 172.21.2.101:0/1880811825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe99c0186c0 msgr2=0x7fe99c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:57.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.138 7fe9bdb14700 1 --2- 172.21.2.101:0/1880811825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe99c0186c0 0x7fe99c01abc0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:57.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.138 7fe9bdb14700 1 -- 172.21.2.101:0/1880811825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9b80961a0 msgr2=0x7fe9b80965a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:57.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.138 7fe9bdb14700 1 --2- 172.21.2.101:0/1880811825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9b80961a0 0x7fe9b80965a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7fe9b000b8e0 tx=0x7fe9b000bc80).stop 2021-02-10T11:14:57.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.138 7fe9bdb14700 1 -- 172.21.2.101:0/1880811825 shutdown_connections 2021-02-10T11:14:57.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.138 7fe9bdb14700 1 --2- 172.21.2.101:0/1880811825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9b80961a0 0x7fe9b80965a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:57.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.138 7fe9bdb14700 1 --2- 172.21.2.101:0/1880811825 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe9b8095780 0x7fe9b8095b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:57.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.138 7fe9bdb14700 1 --2- 172.21.2.101:0/1880811825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe99c0186c0 0x7fe99c01abc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:57.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.138 7fe9bdb14700 1 --2- 172.21.2.101:0/1880811825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9b80b7d30 0x7fe9b80b8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:57.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.138 7fe9bdb14700 1 -- 172.21.2.101:0/1880811825 >> 172.21.2.101:0/1880811825 conn(0x7fe9b819f820 msgr2=0x7fe9b8082b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:57.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.138 7fe9bdb14700 1 -- 172.21.2.101:0/1880811825 shutdown_connections 2021-02-10T11:14:57.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.139 7fe9bdb14700 1 -- 172.21.2.101:0/1880811825 wait complete. 2021-02-10T11:14:57.151 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:14:57.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.555 7f2e42981700 1 Processor -- start 2021-02-10T11:14:57.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.555 7f2e42981700 1 -- start start 2021-02-10T11:14:57.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.556 7f2e42981700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2e3c0cff00 0x7f2e3c1b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:57.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.556 7f2e42981700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2e3c0d1bf0 0x7f2e3c0ef620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:57.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.556 7f2e42981700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e3c0f0b20 con 0x7f2e3c0d17f0 2021-02-10T11:14:57.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.556 7f2e42981700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e3c0e5eb0 con 0x7f2e3c0cff00 2021-02-10T11:14:57.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.556 7f2e42981700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e3c1b4020 con 0x7f2e3c0d1bf0 2021-02-10T11:14:57.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.556 7f2e3b7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2e3c0cff00 0x7f2e3c1b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:57.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.556 7f2e3b7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2e3c0cff00 0x7f2e3c1b39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45340/0 (socket says 172.21.2.101:45340) 2021-02-10T11:14:57.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.556 7f2e3b7fe700 1 -- 172.21.2.101:0/3402402577 learned_addr learned my addr 172.21.2.101:0/3402402577 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:57.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.556 7f2e40f9e700 1 --2- 172.21.2.101:0/3402402577 >> v2:172.21.2.108:3300/0 conn(0x7f2e3c0d1bf0 0x7f2e3c0ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:57.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.556 7f2e3b7fe700 1 -- 172.21.2.101:0/3402402577 >> v2:172.21.2.108:3300/0 conn(0x7f2e3c0d1bf0 msgr2=0x7f2e3c0ef620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:57.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.556 7f2e3b7fe700 1 --2- 172.21.2.101:0/3402402577 >> v2:172.21.2.108:3300/0 conn(0x7f2e3c0d1bf0 0x7f2e3c0ef620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:57.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.556 7f2e3b7fe700 1 -- 172.21.2.101:0/3402402577 >> v1:172.21.2.108:6789/0 conn(0x7f2e3c0d17f0 legacy=0x7f2e3c1b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:57.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.556 7f2e3b7fe700 1 -- 172.21.2.101:0/3402402577 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3c0f20b0 con 0x7f2e3c0cff00 2021-02-10T11:14:57.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.556 7f2e3b7fe700 1 --2- 172.21.2.101:0/3402402577 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2e3c0cff00 0x7f2e3c1b39d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f2e30006700 tx=0x7f2e30009b80).ready entity=mon.0 client_cookie=58413907e30af570 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:57.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.557 7f2e3affd700 1 -- 172.21.2.101:0/3402402577 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2e3000c0c0 con 0x7f2e3c0cff00 2021-02-10T11:14:57.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.557 7f2e3affd700 1 -- 172.21.2.101:0/3402402577 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3000c440 con 0x7f2e3c0cff00 2021-02-10T11:14:57.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.557 7f2e3affd700 1 -- 172.21.2.101:0/3402402577 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2e3000c960 con 0x7f2e3c0cff00 2021-02-10T11:14:57.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.557 7f2e42981700 1 -- 172.21.2.101:0/3402402577 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2e3c0cff00 msgr2=0x7f2e3c1b39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:57.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.557 7f2e42981700 1 --2- 172.21.2.101:0/3402402577 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2e3c0cff00 0x7f2e3c1b39d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f2e30006700 tx=0x7f2e30009b80).stop 2021-02-10T11:14:57.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.557 7f2e42981700 1 -- 172.21.2.101:0/3402402577 shutdown_connections 2021-02-10T11:14:57.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.557 7f2e42981700 1 --2- 172.21.2.101:0/3402402577 >> v2:172.21.2.108:3300/0 conn(0x7f2e3c0d1bf0 0x7f2e3c0ef620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:57.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.557 7f2e42981700 1 --2- 172.21.2.101:0/3402402577 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2e3c0cff00 0x7f2e3c1b39d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:57.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.557 7f2e42981700 1 -- 172.21.2.101:0/3402402577 >> 172.21.2.101:0/3402402577 conn(0x7f2e3c0fde40 msgr2=0x7f2e3c0ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:57.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.557 7f2e42981700 1 -- 172.21.2.101:0/3402402577 shutdown_connections 2021-02-10T11:14:57.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.557 7f2e42981700 1 -- 172.21.2.101:0/3402402577 wait complete. 2021-02-10T11:14:57.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.572 7f2e42981700 1 Processor -- start 2021-02-10T11:14:57.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.572 7f2e42981700 1 -- start start 2021-02-10T11:14:57.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.572 7f2e42981700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2e3c107ee0 0x7f2e3c08e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:57.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.572 7f2e42981700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2e3c08ee90 0x7f2e3c08f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:57.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.572 7f2e42981700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2e3c0b6d20 0x7f2e3c0b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:57.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.572 7f2e42981700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c0b67a0 con 0x7f2e3c08ee90 2021-02-10T11:14:57.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.572 7f2e42981700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c1082e0 con 0x7f2e3c0b6d20 2021-02-10T11:14:57.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.572 7f2e42981700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c08f8b0 con 0x7f2e3c107ee0 2021-02-10T11:14:57.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.573 7f2e3b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2e3c08ee90 0x7f2e3c08f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:57.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.573 7f2e3b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2e3c08ee90 0x7f2e3c08f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45348/0 (socket says 172.21.2.101:45348) 2021-02-10T11:14:57.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.573 7f2e3b7fe700 1 -- 172.21.2.101:0/1975869019 learned_addr learned my addr 172.21.2.101:0/1975869019 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:57.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.573 7f2e3b7fe700 1 -- 172.21.2.101:0/1975869019 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2e3c107ee0 msgr2=0x7f2e3c08e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:57.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.573 7f2e3b7fe700 1 --2- 172.21.2.101:0/1975869019 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2e3c107ee0 0x7f2e3c08e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:57.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.573 7f2e3b7fe700 1 -- 172.21.2.101:0/1975869019 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2e3c0b6d20 msgr2=0x7f2e3c0b7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:57.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.573 7f2e3b7fe700 1 --2- 172.21.2.101:0/1975869019 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2e3c0b6d20 0x7f2e3c0b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:57.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.573 7f2e3b7fe700 1 -- 172.21.2.101:0/1975869019 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3c0b7740 con 0x7f2e3c08ee90 2021-02-10T11:14:57.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.573 7f2e3b7fe700 1 --2- 172.21.2.101:0/1975869019 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2e3c08ee90 0x7f2e3c08f290 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f2e3000b5e0 tx=0x7f2e3000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:57.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.573 7f2e38ff9700 1 -- 172.21.2.101:0/1975869019 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2e3000bd70 con 0x7f2e3c08ee90 2021-02-10T11:14:57.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.573 7f2e38ff9700 1 -- 172.21.2.101:0/1975869019 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e30002030 con 0x7f2e3c08ee90 2021-02-10T11:14:57.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.574 7f2e38ff9700 1 -- 172.21.2.101:0/1975869019 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2e30002550 con 0x7f2e3c08ee90 2021-02-10T11:14:57.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.574 7f2e42981700 1 -- 172.21.2.101:0/1975869019 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3c0b7740 con 0x7f2e3c08ee90 2021-02-10T11:14:57.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.574 7f2e42981700 1 -- 172.21.2.101:0/1975869019 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3c0b7cd0 con 0x7f2e3c08ee90 2021-02-10T11:14:57.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.574 7f2e38ff9700 1 -- 172.21.2.101:0/1975869019 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2e30000c90 con 0x7f2e3c08ee90 2021-02-10T11:14:57.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.574 7f2e38ff9700 1 --2- 172.21.2.101:0/1975869019 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2e1c0186c0 0x7f2e1c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:57.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.574 7f2e38ff9700 1 -- 172.21.2.101:0/1975869019 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2e300325f0 con 0x7f2e3c08ee90 2021-02-10T11:14:57.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.574 7f2e3bfff700 1 --2- 172.21.2.101:0/1975869019 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2e1c0186c0 0x7f2e1c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:57.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.575 7f2e3bfff700 1 --2- 172.21.2.101:0/1975869019 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2e1c0186c0 0x7f2e1c01abc0 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:57.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.606 7f2e42981700 1 -- 172.21.2.101:0/1975869019 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e3c1036a0 con 0x7f2e3c08ee90 2021-02-10T11:14:57.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.608 7f2e38ff9700 1 -- 172.21.2.101:0/1975869019 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2e3002cf70 con 0x7f2e3c08ee90 2021-02-10T11:14:57.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.733 7f2e42981700 1 -- 172.21.2.101:0/1975869019 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e3c0f0b20 con 0x7f2e3c08ee90 2021-02-10T11:14:57.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.734 7f2e38ff9700 1 -- 172.21.2.101:0/1975869019 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+14006 (secure 0 0 0) 0x7f2e3002cf70 con 0x7f2e3c08ee90 2021-02-10T11:14:57.739 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:14:57.739 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":28,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:14:57.130221","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:14:54.652789","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2021-02-10 11:14:56.125641","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:14:57.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.735 7f2e42981700 1 -- 172.21.2.101:0/1975869019 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2e1c0186c0 msgr2=0x7f2e1c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:57.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.735 7f2e42981700 1 --2- 172.21.2.101:0/1975869019 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2e1c0186c0 0x7f2e1c01abc0 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:57.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.735 7f2e42981700 1 -- 172.21.2.101:0/1975869019 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2e3c08ee90 msgr2=0x7f2e3c08f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:57.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.735 7f2e42981700 1 --2- 172.21.2.101:0/1975869019 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2e3c08ee90 0x7f2e3c08f290 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f2e3000b5e0 tx=0x7f2e3000b980).stop 2021-02-10T11:14:57.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.736 7f2e42981700 1 -- 172.21.2.101:0/1975869019 shutdown_connections 2021-02-10T11:14:57.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.736 7f2e42981700 1 --2- 172.21.2.101:0/1975869019 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2e3c0b6d20 0x7f2e3c0b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:57.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.736 7f2e42981700 1 --2- 172.21.2.101:0/1975869019 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2e3c08ee90 0x7f2e3c08f290 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:57.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.736 7f2e42981700 1 --2- 172.21.2.101:0/1975869019 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2e1c0186c0 0x7f2e1c01abc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:57.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.736 7f2e42981700 1 --2- 172.21.2.101:0/1975869019 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2e3c107ee0 0x7f2e3c08e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:57.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.736 7f2e42981700 1 -- 172.21.2.101:0/1975869019 >> 172.21.2.101:0/1975869019 conn(0x7f2e3c09ca80 msgr2=0x7f2e3c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:57.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.736 7f2e42981700 1 -- 172.21.2.101:0/1975869019 shutdown_connections 2021-02-10T11:14:57.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:57.736 7f2e42981700 1 -- 172.21.2.101:0/1975869019 wait complete. 2021-02-10T11:14:57.749 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:14:58.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.155 7f890e551700 1 Processor -- start 2021-02-10T11:14:58.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.155 7f890e551700 1 -- start start 2021-02-10T11:14:58.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.155 7f890e551700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f89080e5070 0x7f89081b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:58.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.155 7f890e551700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89081b3e40 0x7f890807d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:58.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.156 7f890e551700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f890807ed80 con 0x7f89080e5470 2021-02-10T11:14:58.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.156 7f890e551700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8908080310 con 0x7f89080e5070 2021-02-10T11:14:58.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.156 7f890e551700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8908080500 con 0x7f89081b3e40 2021-02-10T11:14:58.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.156 7f89077fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89081b3e40 0x7f890807d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:58.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.156 7f89077fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89081b3e40 0x7f890807d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:38088/0 (socket says 172.21.2.101:38088) 2021-02-10T11:14:58.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.156 7f89077fe700 1 -- 172.21.2.101:0/4285735217 learned_addr learned my addr 172.21.2.101:0/4285735217 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:58.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.156 7f8907fff700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f89080e5070 0x7f89081b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:58.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.156 7f890cb6e700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f89080e5470 0x7f890807dcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:39258/0 (socket says 172.21.2.101:39258) 2021-02-10T11:14:58.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.156 7f8907fff700 1 -- 172.21.2.101:0/4285735217 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89081b3e40 msgr2=0x7f890807d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:14:58.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.156 7f8907fff700 1 --2- 172.21.2.101:0/4285735217 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89081b3e40 0x7f890807d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:58.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.156 7f8907fff700 1 -- 172.21.2.101:0/4285735217 >> v1:172.21.2.108:6789/0 conn(0x7f89080e5470 legacy=0x7f890807dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:58.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.156 7f8907fff700 1 -- 172.21.2.101:0/4285735217 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89080e6090 con 0x7f89080e5070 2021-02-10T11:14:58.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.156 7f8906ffd700 1 -- 172.21.2.101:0/4285735217 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2546877378 0 0) 0x7f8900000d30 con 0x7f89080e5470 2021-02-10T11:14:58.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.156 7f89077fe700 1 --2- 172.21.2.101:0/4285735217 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89081b3e40 0x7f890807d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:14:58.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.156 7f8907fff700 1 --2- 172.21.2.101:0/4285735217 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89080e5070 0x7f89081b3820 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f88f8006650 tx=0x7f88f8009a80).ready entity=mon.1 client_cookie=fcf28397377465e4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:58.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.157 7f8906ffd700 1 -- 172.21.2.101:0/4285735217 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f88f80009b0 con 0x7f89080e5070 2021-02-10T11:14:58.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.157 7f8906ffd700 1 -- 172.21.2.101:0/4285735217 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f800c0c0 con 0x7f89080e5070 2021-02-10T11:14:58.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.157 7f8906ffd700 1 -- 172.21.2.101:0/4285735217 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f88f800c5e0 con 0x7f89080e5070 2021-02-10T11:14:58.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.157 7f890e551700 1 -- 172.21.2.101:0/4285735217 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89080e5070 msgr2=0x7f89081b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:58.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.157 7f890e551700 1 --2- 172.21.2.101:0/4285735217 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89080e5070 0x7f89081b3820 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f88f8006650 tx=0x7f88f8009a80).stop 2021-02-10T11:14:58.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.157 7f890e551700 1 -- 172.21.2.101:0/4285735217 shutdown_connections 2021-02-10T11:14:58.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.157 7f890e551700 1 --2- 172.21.2.101:0/4285735217 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89081b3e40 0x7f890807d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:58.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.157 7f890e551700 1 --2- 172.21.2.101:0/4285735217 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89080e5070 0x7f89081b3820 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:58.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.157 7f890e551700 1 -- 172.21.2.101:0/4285735217 >> 172.21.2.101:0/4285735217 conn(0x7f89080ce690 msgr2=0x7f89080e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:58.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.157 7f890e551700 1 -- 172.21.2.101:0/4285735217 shutdown_connections 2021-02-10T11:14:58.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.157 7f890e551700 1 -- 172.21.2.101:0/4285735217 wait complete. 2021-02-10T11:14:58.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.172 7f890e551700 1 Processor -- start 2021-02-10T11:14:58.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.172 7f890e551700 1 -- start start 2021-02-10T11:14:58.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.172 7f890e551700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8908191bb0 0x7f89080a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:58.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.172 7f890e551700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89080a29a0 0x7f89080a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:58.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.172 7f890e551700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89080a33c0 0x7f89080a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:58.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.172 7f890e551700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89080adf90 con 0x7f89080a33c0 2021-02-10T11:14:58.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.172 7f890e551700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89080a1e00 con 0x7f89080a29a0 2021-02-10T11:14:58.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.172 7f890e551700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89080a09f0 con 0x7f8908191bb0 2021-02-10T11:14:58.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.172 7f890cb6e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89080a33c0 0x7f89080a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:58.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.172 7f890cb6e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89080a33c0 0x7f89080a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45358/0 (socket says 172.21.2.101:45358) 2021-02-10T11:14:58.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.172 7f890cb6e700 1 -- 172.21.2.101:0/466153598 learned_addr learned my addr 172.21.2.101:0/466153598 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:58.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.173 7f8907fff700 1 --2- 172.21.2.101:0/466153598 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8908191bb0 0x7f89080a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:58.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.173 7f89077fe700 1 --2- 172.21.2.101:0/466153598 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89080a29a0 0x7f89080a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:58.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.173 7f890cb6e700 1 -- 172.21.2.101:0/466153598 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8908191bb0 msgr2=0x7f89080a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:58.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.173 7f890cb6e700 1 --2- 172.21.2.101:0/466153598 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8908191bb0 0x7f89080a2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:58.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.173 7f890cb6e700 1 -- 172.21.2.101:0/466153598 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89080a29a0 msgr2=0x7f89080a2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:58.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.173 7f890cb6e700 1 --2- 172.21.2.101:0/466153598 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89080a29a0 0x7f89080a2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:58.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.173 7f890cb6e700 1 -- 172.21.2.101:0/466153598 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89080a0bd0 con 0x7f89080a33c0 2021-02-10T11:14:58.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.173 7f8907fff700 1 --2- 172.21.2.101:0/466153598 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8908191bb0 0x7f89080a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:14:58.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.173 7f890cb6e700 1 --2- 172.21.2.101:0/466153598 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89080a33c0 0x7f89080a03d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f890000a950 tx=0x7f890000acf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:58.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.173 7f8904ff9700 1 -- 172.21.2.101:0/466153598 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f89000011f0 con 0x7f89080a33c0 2021-02-10T11:14:58.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.173 7f8904ff9700 1 -- 172.21.2.101:0/466153598 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8900007160 con 0x7f89080a33c0 2021-02-10T11:14:58.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.173 7f8904ff9700 1 -- 172.21.2.101:0/466153598 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8900007530 con 0x7f89080a33c0 2021-02-10T11:14:58.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.173 7f890e551700 1 -- 172.21.2.101:0/466153598 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89080a0bd0 con 0x7f89080a33c0 2021-02-10T11:14:58.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.173 7f890e551700 1 -- 172.21.2.101:0/466153598 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89080a1160 con 0x7f89080a33c0 2021-02-10T11:14:58.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.174 7f8904ff9700 1 -- 172.21.2.101:0/466153598 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8900007340 con 0x7f89080a33c0 2021-02-10T11:14:58.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.174 7f8904ff9700 1 --2- 172.21.2.101:0/466153598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f88e80186c0 0x7f88e801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:58.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.174 7f8904ff9700 1 -- 172.21.2.101:0/466153598 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8900032790 con 0x7f89080a33c0 2021-02-10T11:14:58.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.174 7f8907fff700 1 --2- 172.21.2.101:0/466153598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f88e80186c0 0x7f88e801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:58.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.174 7f8907fff700 1 --2- 172.21.2.101:0/466153598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f88e80186c0 0x7f88e801abc0 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:58.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.205 7f890e551700 1 -- 172.21.2.101:0/466153598 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89080a1160 con 0x7f89080a33c0 2021-02-10T11:14:58.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.208 7f8904ff9700 1 -- 172.21.2.101:0/466153598 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f89000011f0 con 0x7f89080a33c0 2021-02-10T11:14:58.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.332 7f890e551700 1 -- 172.21.2.101:0/466153598 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89080a1160 con 0x7f89080a33c0 2021-02-10T11:14:58.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.333 7f8904ff9700 1 -- 172.21.2.101:0/466153598 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+13993 (secure 0 0 0) 0x7f8900007340 con 0x7f89080a33c0 2021-02-10T11:14:58.338 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:14:58.339 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":29,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:14:58.132054","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:14:54.652789","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2021-02-10 11:14:56.125641","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:14:58.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.334 7f890e551700 1 -- 172.21.2.101:0/466153598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f88e80186c0 msgr2=0x7f88e801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:58.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.334 7f890e551700 1 --2- 172.21.2.101:0/466153598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f88e80186c0 0x7f88e801abc0 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:58.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.334 7f890e551700 1 -- 172.21.2.101:0/466153598 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89080a33c0 msgr2=0x7f89080a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:58.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.334 7f890e551700 1 --2- 172.21.2.101:0/466153598 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89080a33c0 0x7f89080a03d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f890000a950 tx=0x7f890000acf0).stop 2021-02-10T11:14:58.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.335 7f890e551700 1 -- 172.21.2.101:0/466153598 shutdown_connections 2021-02-10T11:14:58.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.335 7f890e551700 1 --2- 172.21.2.101:0/466153598 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89080a33c0 0x7f89080a03d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:58.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.335 7f890e551700 1 --2- 172.21.2.101:0/466153598 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89080a29a0 0x7f89080a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:58.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.335 7f890e551700 1 --2- 172.21.2.101:0/466153598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f88e80186c0 0x7f88e801abc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:58.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.335 7f890e551700 1 --2- 172.21.2.101:0/466153598 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8908191bb0 0x7f89080a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:58.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.335 7f890e551700 1 -- 172.21.2.101:0/466153598 >> 172.21.2.101:0/466153598 conn(0x7f89081a31d0 msgr2=0x7f890808b830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:58.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.335 7f890e551700 1 -- 172.21.2.101:0/466153598 shutdown_connections 2021-02-10T11:14:58.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.335 7f890e551700 1 -- 172.21.2.101:0/466153598 wait complete. 2021-02-10T11:14:58.348 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-02-10T11:14:58.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.757 7f24fb2f5700 1 Processor -- start 2021-02-10T11:14:58.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.757 7f24fb2f5700 1 -- start start 2021-02-10T11:14:58.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.757 7f24fb2f5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24f41b0470 0x7f24f41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:58.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.757 7f24fb2f5700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f24f40ef610 0x7f24f40efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:58.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.757 7f24fb2f5700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f40f0b50 con 0x7f24f40f0030 2021-02-10T11:14:58.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.757 7f24fb2f5700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f40e52e0 con 0x7f24f40ef610 2021-02-10T11:14:58.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.757 7f24fb2f5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f40e5490 con 0x7f24f41b0470 2021-02-10T11:14:58.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.757 7f24f8910700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f24f40ef610 0x7f24f40efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:58.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.757 7f24f8910700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f24f40ef610 0x7f24f40efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45366/0 (socket says 172.21.2.101:45366) 2021-02-10T11:14:58.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.757 7f24f8910700 1 -- 172.21.2.101:0/546211177 learned_addr learned my addr 172.21.2.101:0/546211177 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:58.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.758 7f24f8910700 1 -- 172.21.2.101:0/546211177 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24f41b0470 msgr2=0x7f24f41b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:58.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.758 7f24f8910700 1 --2- 172.21.2.101:0/546211177 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24f41b0470 0x7f24f41b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:58.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.758 7f24f8910700 1 -- 172.21.2.101:0/546211177 >> v1:172.21.2.108:6789/0 conn(0x7f24f40f0030 legacy=0x7f24f40f10d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:14:58.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.758 7f24f8910700 1 -- 172.21.2.101:0/546211177 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f40f0430 con 0x7f24f40ef610 2021-02-10T11:14:58.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.758 7f24f8910700 1 --2- 172.21.2.101:0/546211177 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24f40ef610 0x7f24f40efa10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f24e8006700 tx=0x7f24e8009b00).ready entity=mon.0 client_cookie=a585b926c668ef9a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:58.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.758 7f24e7fff700 1 -- 172.21.2.101:0/546211177 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f24e800c0c0 con 0x7f24f40ef610 2021-02-10T11:14:58.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.758 7f24e7fff700 1 -- 172.21.2.101:0/546211177 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e800c340 con 0x7f24f40ef610 2021-02-10T11:14:58.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.758 7f24e7fff700 1 -- 172.21.2.101:0/546211177 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f24e800c820 con 0x7f24f40ef610 2021-02-10T11:14:58.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.758 7f24fb2f5700 1 -- 172.21.2.101:0/546211177 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24f40ef610 msgr2=0x7f24f40efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:58.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.758 7f24fb2f5700 1 --2- 172.21.2.101:0/546211177 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24f40ef610 0x7f24f40efa10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f24e8006700 tx=0x7f24e8009b00).stop 2021-02-10T11:14:58.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.759 7f24fb2f5700 1 -- 172.21.2.101:0/546211177 shutdown_connections 2021-02-10T11:14:58.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.759 7f24fb2f5700 1 --2- 172.21.2.101:0/546211177 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24f40ef610 0x7f24f40efa10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:58.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.759 7f24fb2f5700 1 --2- 172.21.2.101:0/546211177 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24f41b0470 0x7f24f41b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:58.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.759 7f24fb2f5700 1 -- 172.21.2.101:0/546211177 >> 172.21.2.101:0/546211177 conn(0x7f24f40ce3e0 msgr2=0x7f24f40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:58.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.759 7f24fb2f5700 1 -- 172.21.2.101:0/546211177 shutdown_connections 2021-02-10T11:14:58.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.759 7f24fb2f5700 1 -- 172.21.2.101:0/546211177 wait complete. 2021-02-10T11:14:58.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.773 7f24fb2f5700 1 Processor -- start 2021-02-10T11:14:58.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.773 7f24fb2f5700 1 -- start start 2021-02-10T11:14:58.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.774 7f24fb2f5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24f40b2840 0x7f24f41a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:58.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.774 7f24fb2f5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24f40b30d0 0x7f24f40b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:58.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.774 7f24fb2f5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24f40b3af0 0x7f24f40f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:58.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.774 7f24fb2f5700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f40b2c40 con 0x7f24f40b2840 2021-02-10T11:14:58.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.774 7f24fb2f5700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f41a4920 con 0x7f24f40b30d0 2021-02-10T11:14:58.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.774 7f24fb2f5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f40f0ba0 con 0x7f24f40b3af0 2021-02-10T11:14:58.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.774 7f24f9111700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24f40b2840 0x7f24f41a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:58.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.774 7f24f9111700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24f40b2840 0x7f24f41a4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45370/0 (socket says 172.21.2.101:45370) 2021-02-10T11:14:58.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.774 7f24f9111700 1 -- 172.21.2.101:0/4195108832 learned_addr learned my addr 172.21.2.101:0/4195108832 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:14:58.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.774 7f24f9111700 1 -- 172.21.2.101:0/4195108832 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24f40b3af0 msgr2=0x7f24f40f0580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:58.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.774 7f24f9111700 1 --2- 172.21.2.101:0/4195108832 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24f40b3af0 0x7f24f40f0580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:58.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.774 7f24f9111700 1 -- 172.21.2.101:0/4195108832 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24f40b30d0 msgr2=0x7f24f40b34d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:14:58.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.774 7f24f9111700 1 --2- 172.21.2.101:0/4195108832 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24f40b30d0 0x7f24f40b34d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:58.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.774 7f24f9111700 1 -- 172.21.2.101:0/4195108832 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f40f0dc0 con 0x7f24f40b2840 2021-02-10T11:14:58.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.774 7f24f9111700 1 --2- 172.21.2.101:0/4195108832 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24f40b2840 0x7f24f41a4300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f24f000b5c0 tx=0x7f24f000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:58.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.775 7f24e5ffb700 1 -- 172.21.2.101:0/4195108832 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f24f000bd50 con 0x7f24f40b2840 2021-02-10T11:14:58.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.775 7f24e5ffb700 1 -- 172.21.2.101:0/4195108832 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f0006030 con 0x7f24f40b2840 2021-02-10T11:14:58.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.775 7f24e5ffb700 1 -- 172.21.2.101:0/4195108832 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f24f00054d0 con 0x7f24f40b2840 2021-02-10T11:14:58.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.775 7f24fb2f5700 1 -- 172.21.2.101:0/4195108832 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f40f0dc0 con 0x7f24f40b2840 2021-02-10T11:14:58.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.775 7f24fb2f5700 1 -- 172.21.2.101:0/4195108832 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f40f1350 con 0x7f24f40b2840 2021-02-10T11:14:58.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.775 7f24e5ffb700 1 -- 172.21.2.101:0/4195108832 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f24f0006210 con 0x7f24f40b2840 2021-02-10T11:14:58.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.775 7f24e5ffb700 1 --2- 172.21.2.101:0/4195108832 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24d40186c0 0x7f24d401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:14:58.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.775 7f24e5ffb700 1 -- 172.21.2.101:0/4195108832 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f24f0032430 con 0x7f24f40b2840 2021-02-10T11:14:58.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.775 7f24f8910700 1 --2- 172.21.2.101:0/4195108832 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24d40186c0 0x7f24d401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:14:58.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.776 7f24f8910700 1 --2- 172.21.2.101:0/4195108832 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24d40186c0 0x7f24d401abc0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:14:58.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.807 7f24fb2f5700 1 -- 172.21.2.101:0/4195108832 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f40f18f0 con 0x7f24f40b2840 2021-02-10T11:14:58.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.810 7f24e5ffb700 1 -- 172.21.2.101:0/4195108832 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f24f0000940 con 0x7f24f40b2840 2021-02-10T11:14:58.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:58.945 7f24fb2f5700 1 -- 172.21.2.101:0/4195108832 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"} v 0) v1 -- 0x7f24f40f0050 con 0x7f24f40b2840 2021-02-10T11:14:59.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.512 7f24e5ffb700 1 -- 172.21.2.101:0/4195108832 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"}]=0 v13) v1 ==== 118+0+0 (secure 0 0 0) 0x7f24f0006210 con 0x7f24f40b2840 2021-02-10T11:14:59.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.564 7f24fb2f5700 1 -- 172.21.2.101:0/4195108832 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24d40186c0 msgr2=0x7f24d401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:59.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.564 7f24fb2f5700 1 --2- 172.21.2.101:0/4195108832 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24d40186c0 0x7f24d401abc0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:59.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.564 7f24fb2f5700 1 -- 172.21.2.101:0/4195108832 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24f40b2840 msgr2=0x7f24f41a4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:14:59.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.564 7f24fb2f5700 1 --2- 172.21.2.101:0/4195108832 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24f40b2840 0x7f24f41a4300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f24f000b5c0 tx=0x7f24f000b960).stop 2021-02-10T11:14:59.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.565 7f24fb2f5700 1 -- 172.21.2.101:0/4195108832 shutdown_connections 2021-02-10T11:14:59.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.565 7f24fb2f5700 1 --2- 172.21.2.101:0/4195108832 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24d40186c0 0x7f24d401abc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:59.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.565 7f24fb2f5700 1 --2- 172.21.2.101:0/4195108832 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24f40b3af0 0x7f24f40f0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:59.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.565 7f24fb2f5700 1 --2- 172.21.2.101:0/4195108832 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24f40b30d0 0x7f24f40b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:14:59.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.565 7f24fb2f5700 1 --2- 172.21.2.101:0/4195108832 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24f40b2840 0x7f24f41a4300 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:14:59.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.565 7f24fb2f5700 1 -- 172.21.2.101:0/4195108832 >> 172.21.2.101:0/4195108832 conn(0x7f24f40c36f0 msgr2=0x7f24f4094c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:14:59.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.565 7f24fb2f5700 1 -- 172.21.2.101:0/4195108832 shutdown_connections 2021-02-10T11:14:59.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.565 7f24fb2f5700 1 -- 172.21.2.101:0/4195108832 wait complete. 2021-02-10T11:14:59.578 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:15:00.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.983 7faf4e00b700 1 Processor -- start 2021-02-10T11:15:00.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.983 7faf4e00b700 1 -- start start 2021-02-10T11:15:00.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.984 7faf4e00b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faf480e5070 0x7faf481b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:00.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.984 7faf4e00b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7faf481b3e40 0x7faf48080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:00.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.984 7faf4e00b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf4807ed80 con 0x7faf480e5470 2021-02-10T11:15:00.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.984 7faf4e00b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf480d1790 con 0x7faf481b3e40 2021-02-10T11:15:00.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.984 7faf4e00b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf480d1980 con 0x7faf480e5070 2021-02-10T11:15:00.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.984 7faf47fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7faf481b3e40 0x7faf48080310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:00.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.984 7faf47fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7faf481b3e40 0x7faf48080310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45378/0 (socket says 172.21.2.101:45378) 2021-02-10T11:15:00.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.984 7faf46ffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7faf480e5470 0x7faf4807d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45872/0 (socket says 172.21.2.101:45872) 2021-02-10T11:15:00.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.984 7faf47fff700 1 -- 172.21.2.101:0/404751761 learned_addr learned my addr 172.21.2.101:0/404751761 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:00.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.984 7faf477fe700 1 --2- 172.21.2.101:0/404751761 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faf480e5070 0x7faf481b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:00.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.984 7faf47fff700 1 -- 172.21.2.101:0/404751761 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faf480e5070 msgr2=0x7faf481b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:00.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.984 7faf47fff700 1 --2- 172.21.2.101:0/404751761 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faf480e5070 0x7faf481b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:00.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.984 7faf47fff700 1 -- 172.21.2.101:0/404751761 >> v1:172.21.2.101:6789/0 conn(0x7faf480e5470 legacy=0x7faf4807d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:00.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.984 7faf47fff700 1 -- 172.21.2.101:0/404751761 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf480d1ba0 con 0x7faf481b3e40 2021-02-10T11:15:00.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.984 7faf477fe700 1 --2- 172.21.2.101:0/404751761 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faf480e5070 0x7faf481b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:15:00.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.984 7faf47fff700 1 --2- 172.21.2.101:0/404751761 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faf481b3e40 0x7faf48080310 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7faf40006f00 tx=0x7faf40009f10).ready entity=mon.0 client_cookie=e4cbc83d6a62d574 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:00.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.985 7faf467fc700 1 -- 172.21.2.101:0/404751761 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faf4000b3b0 con 0x7faf481b3e40 2021-02-10T11:15:00.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.985 7faf467fc700 1 -- 172.21.2.101:0/404751761 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf40000d30 con 0x7faf481b3e40 2021-02-10T11:15:00.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.985 7faf467fc700 1 -- 172.21.2.101:0/404751761 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faf4000b8a0 con 0x7faf481b3e40 2021-02-10T11:15:00.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.985 7faf4e00b700 1 -- 172.21.2.101:0/404751761 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faf481b3e40 msgr2=0x7faf48080310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:00.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.985 7faf4e00b700 1 --2- 172.21.2.101:0/404751761 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faf481b3e40 0x7faf48080310 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7faf40006f00 tx=0x7faf40009f10).stop 2021-02-10T11:15:00.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.985 7faf4e00b700 1 -- 172.21.2.101:0/404751761 shutdown_connections 2021-02-10T11:15:00.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.985 7faf4e00b700 1 --2- 172.21.2.101:0/404751761 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faf481b3e40 0x7faf48080310 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:00.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.985 7faf4e00b700 1 --2- 172.21.2.101:0/404751761 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faf480e5070 0x7faf481b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:00.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.985 7faf4e00b700 1 -- 172.21.2.101:0/404751761 >> 172.21.2.101:0/404751761 conn(0x7faf480ce690 msgr2=0x7faf480e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:00.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.985 7faf4e00b700 1 -- 172.21.2.101:0/404751761 shutdown_connections 2021-02-10T11:15:00.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:14:59.985 7faf4e00b700 1 -- 172.21.2.101:0/404751761 wait complete. 2021-02-10T11:15:00.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.000 7faf4e00b700 1 Processor -- start 2021-02-10T11:15:00.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.000 7faf4e00b700 1 -- start start 2021-02-10T11:15:00.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.000 7faf4e00b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faf48097250 0x7faf481a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:00.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.000 7faf4e00b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faf4808d5a0 0x7faf480ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:00.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.000 7faf4e00b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faf480ae9c0 0x7faf481a2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:00.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.000 7faf4e00b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf481a22f0 con 0x7faf48097250 2021-02-10T11:15:00.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.000 7faf4e00b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf481b2d40 con 0x7faf4808d5a0 2021-02-10T11:15:00.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.000 7faf4e00b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf481b2f20 con 0x7faf480ae9c0 2021-02-10T11:15:00.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.001 7faf477fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faf48097250 0x7faf481a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:00.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.001 7faf46ffd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faf4808d5a0 0x7faf480ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:00.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.001 7faf477fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faf48097250 0x7faf481a1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45386/0 (socket says 172.21.2.101:45386) 2021-02-10T11:15:00.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.001 7faf477fe700 1 -- 172.21.2.101:0/1190075340 learned_addr learned my addr 172.21.2.101:0/1190075340 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:00.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.001 7faf47fff700 1 --2- 172.21.2.101:0/1190075340 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faf480ae9c0 0x7faf481a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:00.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.001 7faf477fe700 1 -- 172.21.2.101:0/1190075340 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faf480ae9c0 msgr2=0x7faf481a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:00.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.001 7faf477fe700 1 --2- 172.21.2.101:0/1190075340 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faf480ae9c0 0x7faf481a2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:00.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.001 7faf477fe700 1 -- 172.21.2.101:0/1190075340 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faf4808d5a0 msgr2=0x7faf480ae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:00.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.001 7faf477fe700 1 --2- 172.21.2.101:0/1190075340 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faf4808d5a0 0x7faf480ae3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:00.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.001 7faf477fe700 1 -- 172.21.2.101:0/1190075340 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4808d9d0 con 0x7faf48097250 2021-02-10T11:15:00.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.001 7faf477fe700 1 --2- 172.21.2.101:0/1190075340 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faf48097250 0x7faf481a1b90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7faf38006f20 tx=0x7faf3800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:00.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.001 7faf2ffff700 1 -- 172.21.2.101:0/1190075340 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faf3800bd80 con 0x7faf48097250 2021-02-10T11:15:00.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.001 7faf2ffff700 1 -- 172.21.2.101:0/1190075340 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf38000c90 con 0x7faf48097250 2021-02-10T11:15:00.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.001 7faf2ffff700 1 -- 172.21.2.101:0/1190075340 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faf380022b0 con 0x7faf48097250 2021-02-10T11:15:00.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.001 7faf4e00b700 1 -- 172.21.2.101:0/1190075340 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4808d9d0 con 0x7faf48097250 2021-02-10T11:15:00.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.001 7faf4e00b700 1 -- 172.21.2.101:0/1190075340 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4808d9d0 con 0x7faf48097250 2021-02-10T11:15:00.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.002 7faf2ffff700 1 -- 172.21.2.101:0/1190075340 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7faf3800a000 con 0x7faf48097250 2021-02-10T11:15:00.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.002 7faf2ffff700 1 --2- 172.21.2.101:0/1190075340 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faf28018730 0x7faf2801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:00.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.002 7faf2ffff700 1 -- 172.21.2.101:0/1190075340 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7faf38032640 con 0x7faf48097250 2021-02-10T11:15:00.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.002 7faf46ffd700 1 --2- 172.21.2.101:0/1190075340 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faf28018730 0x7faf2801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:00.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.002 7faf46ffd700 1 --2- 172.21.2.101:0/1190075340 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faf28018730 0x7faf2801ac10 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:00.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.034 7faf4e00b700 1 -- 172.21.2.101:0/1190075340 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf48107170 con 0x7faf48097250 2021-02-10T11:15:00.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.036 7faf2ffff700 1 -- 172.21.2.101:0/1190075340 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7faf38006bb0 con 0x7faf48097250 2021-02-10T11:15:00.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.164 7faf4e00b700 1 -- 172.21.2.101:0/1190075340 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf4807ed80 con 0x7faf48097250 2021-02-10T11:15:00.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.164 7faf2ffff700 1 -- 172.21.2.101:0/1190075340 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+2100 (secure 0 0 0) 0x7faf3800bd80 con 0x7faf48097250 2021-02-10T11:15:00.170 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:15:00.170 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/3226463341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3226463341},{"type":"v1","addr":"172.21.2.101:6835","nonce":3226463341}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:14:57.130312","modified":"2021-02-10 11:14:59.506936","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4373":{"gid":4373,"name":"b","rank":0,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.2.108:6833/3230549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3230549207},{"type":"v1","addr":"172.21.2.108:6833","nonce":3230549207}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-02-10T11:15:00.170 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 13 2021-02-10T11:15:00.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.166 7faf4e00b700 1 -- 172.21.2.101:0/1190075340 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faf28018730 msgr2=0x7faf2801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:00.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.166 7faf4e00b700 1 --2- 172.21.2.101:0/1190075340 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faf28018730 0x7faf2801ac10 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:00.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.166 7faf4e00b700 1 -- 172.21.2.101:0/1190075340 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faf48097250 msgr2=0x7faf481a1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:00.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.166 7faf4e00b700 1 --2- 172.21.2.101:0/1190075340 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faf48097250 0x7faf481a1b90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7faf38006f20 tx=0x7faf3800b990).stop 2021-02-10T11:15:00.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.166 7faf4e00b700 1 -- 172.21.2.101:0/1190075340 shutdown_connections 2021-02-10T11:15:00.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.166 7faf4e00b700 1 --2- 172.21.2.101:0/1190075340 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faf28018730 0x7faf2801ac10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:00.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.166 7faf4e00b700 1 --2- 172.21.2.101:0/1190075340 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faf480ae9c0 0x7faf481a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:00.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.166 7faf4e00b700 1 --2- 172.21.2.101:0/1190075340 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faf4808d5a0 0x7faf480ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:00.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.167 7faf4e00b700 1 --2- 172.21.2.101:0/1190075340 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faf48097250 0x7faf481a1b90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:00.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.167 7faf4e00b700 1 -- 172.21.2.101:0/1190075340 >> 172.21.2.101:0/1190075340 conn(0x7faf481ac620 msgr2=0x7faf480896e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:00.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.167 7faf4e00b700 1 -- 172.21.2.101:0/1190075340 shutdown_connections 2021-02-10T11:15:00.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.167 7faf4e00b700 1 -- 172.21.2.101:0/1190075340 wait complete. 2021-02-10T11:15:00.180 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:15:00.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.587 7fa9f6cb2700 1 Processor -- start 2021-02-10T11:15:00.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.587 7fa9f6cb2700 1 -- start start 2021-02-10T11:15:00.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.587 7fa9f6cb2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa9f00e5070 0x7fa9f01b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:00.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.587 7fa9f6cb2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa9f00e5470 0x7fa9f007d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:00.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.587 7fa9f6cb2700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f007ed80 con 0x7fa9f01b3e40 2021-02-10T11:15:00.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.587 7fa9f6cb2700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f0080310 con 0x7fa9f00e5470 2021-02-10T11:15:00.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.587 7fa9f6cb2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f0080500 con 0x7fa9f00e5070 2021-02-10T11:15:00.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.588 7fa9f52cf700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa9f01b3e40 0x7fa9f007dcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45888/0 (socket says 172.21.2.101:45888) 2021-02-10T11:15:00.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.588 7fa9f52cf700 1 -- 172.21.2.101:0/818769364 learned_addr learned my addr 172.21.2.101:0/818769364 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:15:00.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.588 7fa9effff700 1 --2- 172.21.2.101:0/818769364 >> v2:172.21.2.101:3300/0 conn(0x7fa9f00e5470 0x7fa9f007d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:00.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.588 7fa9f4ace700 1 --2- 172.21.2.101:0/818769364 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa9f00e5070 0x7fa9f01b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:00.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.588 7fa9effff700 1 -- 172.21.2.101:0/818769364 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa9f00e5070 msgr2=0x7fa9f01b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:00.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.588 7fa9effff700 1 --2- 172.21.2.101:0/818769364 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa9f00e5070 0x7fa9f01b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:00.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.588 7fa9effff700 1 -- 172.21.2.101:0/818769364 >> v1:172.21.2.101:6789/0 conn(0x7fa9f01b3e40 legacy=0x7fa9f007dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:00.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.588 7fa9effff700 1 -- 172.21.2.101:0/818769364 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f00806f0 con 0x7fa9f00e5470 2021-02-10T11:15:00.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.588 7fa9f4ace700 1 --2- 172.21.2.101:0/818769364 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa9f00e5070 0x7fa9f01b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:15:00.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.588 7fa9effff700 1 --2- 172.21.2.101:0/818769364 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa9f00e5470 0x7fa9f007d690 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fa9d8006650 tx=0x7fa9d8009a80).ready entity=mon.0 client_cookie=d759f56f489c6e1e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:00.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.588 7fa9ef7fe700 1 -- 172.21.2.101:0/818769364 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa9d8006390 con 0x7fa9f00e5470 2021-02-10T11:15:00.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.588 7fa9ef7fe700 1 -- 172.21.2.101:0/818769364 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d80009b0 con 0x7fa9f00e5470 2021-02-10T11:15:00.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.588 7fa9ef7fe700 1 -- 172.21.2.101:0/818769364 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa9d800c2b0 con 0x7fa9f00e5470 2021-02-10T11:15:00.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.588 7fa9f6cb2700 1 -- 172.21.2.101:0/818769364 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa9f00e5470 msgr2=0x7fa9f007d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:00.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.588 7fa9f6cb2700 1 --2- 172.21.2.101:0/818769364 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa9f00e5470 0x7fa9f007d690 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fa9d8006650 tx=0x7fa9d8009a80).stop 2021-02-10T11:15:00.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.589 7fa9f6cb2700 1 -- 172.21.2.101:0/818769364 shutdown_connections 2021-02-10T11:15:00.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.589 7fa9f6cb2700 1 --2- 172.21.2.101:0/818769364 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa9f00e5470 0x7fa9f007d690 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:00.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.589 7fa9f6cb2700 1 --2- 172.21.2.101:0/818769364 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa9f00e5070 0x7fa9f01b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:00.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.589 7fa9f6cb2700 1 -- 172.21.2.101:0/818769364 >> 172.21.2.101:0/818769364 conn(0x7fa9f00ce690 msgr2=0x7fa9f00e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:00.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.589 7fa9f6cb2700 1 -- 172.21.2.101:0/818769364 shutdown_connections 2021-02-10T11:15:00.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.589 7fa9f6cb2700 1 -- 172.21.2.101:0/818769364 wait complete. 2021-02-10T11:15:00.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.604 7fa9f6cb2700 1 Processor -- start 2021-02-10T11:15:00.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.604 7fa9f6cb2700 1 -- start start 2021-02-10T11:15:00.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.604 7fa9f6cb2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa9f00969a0 0x7fa9f0096110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:00.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.604 7fa9f6cb2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa9f008a460 0x7fa9f01a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:00.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.604 7fa9f6cb2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa9f00b1440 0x7fa9f00b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:00.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.604 7fa9f6cb2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00b1e60 con 0x7fa9f008a460 2021-02-10T11:15:00.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.604 7fa9f6cb2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00b2080 con 0x7fa9f00b1440 2021-02-10T11:15:00.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.604 7fa9f6cb2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00b22a0 con 0x7fa9f00969a0 2021-02-10T11:15:00.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.605 7fa9effff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa9f008a460 0x7fa9f01a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:00.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.605 7fa9effff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa9f008a460 0x7fa9f01a1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45400/0 (socket says 172.21.2.101:45400) 2021-02-10T11:15:00.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.605 7fa9effff700 1 -- 172.21.2.101:0/3243060941 learned_addr learned my addr 172.21.2.101:0/3243060941 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:00.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.605 7fa9effff700 1 -- 172.21.2.101:0/3243060941 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa9f00969a0 msgr2=0x7fa9f0096110 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:00.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.605 7fa9effff700 1 --2- 172.21.2.101:0/3243060941 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa9f00969a0 0x7fa9f0096110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:00.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.605 7fa9effff700 1 -- 172.21.2.101:0/3243060941 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa9f00b1440 msgr2=0x7fa9f00b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:00.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.605 7fa9effff700 1 --2- 172.21.2.101:0/3243060941 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa9f00b1440 0x7fa9f00b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:00.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.605 7fa9effff700 1 -- 172.21.2.101:0/3243060941 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f008ae90 con 0x7fa9f008a460 2021-02-10T11:15:00.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.605 7fa9effff700 1 --2- 172.21.2.101:0/3243060941 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa9f008a460 0x7fa9f01a1440 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fa9d800b660 tx=0x7fa9d800ba00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:00.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.605 7fa9ed7fa700 1 -- 172.21.2.101:0/3243060941 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa9d80009b0 con 0x7fa9f008a460 2021-02-10T11:15:00.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.605 7fa9ed7fa700 1 -- 172.21.2.101:0/3243060941 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d8006760 con 0x7fa9f008a460 2021-02-10T11:15:00.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.605 7fa9ed7fa700 1 -- 172.21.2.101:0/3243060941 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa9d8002030 con 0x7fa9f008a460 2021-02-10T11:15:00.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.605 7fa9f6cb2700 1 -- 172.21.2.101:0/3243060941 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f008ae90 con 0x7fa9f008a460 2021-02-10T11:15:00.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.605 7fa9f6cb2700 1 -- 172.21.2.101:0/3243060941 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f008ae90 con 0x7fa9f008a460 2021-02-10T11:15:00.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.606 7fa9ed7fa700 1 -- 172.21.2.101:0/3243060941 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa9d8000c90 con 0x7fa9f008a460 2021-02-10T11:15:00.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.606 7fa9ed7fa700 1 --2- 172.21.2.101:0/3243060941 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa9d40186c0 0x7fa9d401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:00.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.606 7fa9ed7fa700 1 -- 172.21.2.101:0/3243060941 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa9d8032690 con 0x7fa9f008a460 2021-02-10T11:15:00.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.606 7fa9f4ace700 1 --2- 172.21.2.101:0/3243060941 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa9d40186c0 0x7fa9d401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:00.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.606 7fa9f4ace700 1 --2- 172.21.2.101:0/3243060941 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa9d40186c0 0x7fa9d401abc0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:00.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.637 7fa9f6cb2700 1 -- 172.21.2.101:0/3243060941 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f008a960 con 0x7fa9f008a460 2021-02-10T11:15:00.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.639 7fa9ed7fa700 1 -- 172.21.2.101:0/3243060941 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa9d8006740 con 0x7fa9f008a460 2021-02-10T11:15:00.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.764 7fa9f6cb2700 1 -- 172.21.2.101:0/3243060941 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9f008a960 con 0x7fa9f008a460 2021-02-10T11:15:00.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.765 7fa9ed7fa700 1 -- 172.21.2.101:0/3243060941 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+13993 (secure 0 0 0) 0x7fa9d8006740 con 0x7fa9f008a460 2021-02-10T11:15:00.770 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:15:00.771 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":29,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:14:58.132054","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:14:54.652789","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2021-02-10 11:14:56.125641","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:15:00.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.767 7fa9f6cb2700 1 -- 172.21.2.101:0/3243060941 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa9d40186c0 msgr2=0x7fa9d401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:00.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.767 7fa9f6cb2700 1 --2- 172.21.2.101:0/3243060941 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa9d40186c0 0x7fa9d401abc0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:00.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.767 7fa9f6cb2700 1 -- 172.21.2.101:0/3243060941 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa9f008a460 msgr2=0x7fa9f01a1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:00.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.767 7fa9f6cb2700 1 --2- 172.21.2.101:0/3243060941 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa9f008a460 0x7fa9f01a1440 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fa9d800b660 tx=0x7fa9d800ba00).stop 2021-02-10T11:15:00.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.767 7fa9f6cb2700 1 -- 172.21.2.101:0/3243060941 shutdown_connections 2021-02-10T11:15:00.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.767 7fa9f6cb2700 1 --2- 172.21.2.101:0/3243060941 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa9f00b1440 0x7fa9f00b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:00.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.767 7fa9f6cb2700 1 --2- 172.21.2.101:0/3243060941 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa9f008a460 0x7fa9f01a1440 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:00.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.767 7fa9f6cb2700 1 --2- 172.21.2.101:0/3243060941 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa9d40186c0 0x7fa9d401abc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:00.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.767 7fa9f6cb2700 1 --2- 172.21.2.101:0/3243060941 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa9f00969a0 0x7fa9f0096110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:00.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.767 7fa9f6cb2700 1 -- 172.21.2.101:0/3243060941 >> 172.21.2.101:0/3243060941 conn(0x7fa9f01909e0 msgr2=0x7fa9f00895a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:00.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.767 7fa9f6cb2700 1 -- 172.21.2.101:0/3243060941 shutdown_connections 2021-02-10T11:15:00.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:00.767 7fa9f6cb2700 1 -- 172.21.2.101:0/3243060941 wait complete. 2021-02-10T11:15:00.781 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:15:01.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.197 7fedd1669700 1 Processor -- start 2021-02-10T11:15:01.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.197 7fedd1669700 1 -- start start 2021-02-10T11:15:01.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.198 7fedd1669700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fedcc0f05f0 0x7fedcc0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:01.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.198 7fedd1669700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedcc1b0c70 con 0x7fedcc1b0870 2021-02-10T11:15:01.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.198 7fedd1669700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedcc0f0b50 con 0x7fedcc1b0470 2021-02-10T11:15:01.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.198 7fedd1669700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedcc0e5ea0 con 0x7fedcc0f05f0 2021-02-10T11:15:01.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.198 7fedcbac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fedcc0f05f0 0x7fedcc0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:01.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.198 7fedcbac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fedcc0f05f0 0x7fedcc0f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45406/0 (socket says 172.21.2.101:45406) 2021-02-10T11:15:01.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.198 7fedcbac0700 1 -- 172.21.2.101:0/2109456801 learned_addr learned my addr 172.21.2.101:0/2109456801 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:01.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.198 7fedcbac0700 1 -- 172.21.2.101:0/2109456801 >> v1:172.21.2.108:6789/0 conn(0x7fedcc1b0470 legacy=0x7fedcc0ef610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:15:01.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.198 7fedcbac0700 1 -- 172.21.2.101:0/2109456801 >> v1:172.21.2.101:6789/0 conn(0x7fedcc1b0870 legacy=0x7fedcc0efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:01.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.198 7fedcbac0700 1 -- 172.21.2.101:0/2109456801 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc0e52e0 con 0x7fedcc0f05f0 2021-02-10T11:15:01.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.198 7fedcbac0700 1 --2- 172.21.2.101:0/2109456801 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedcc0f05f0 0x7fedcc0f20e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fedc4006f00 tx=0x7fedc4009f10).ready entity=mon.0 client_cookie=bdda1d877e2baf99 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:01.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.199 7fedca2bd700 1 -- 172.21.2.101:0/2109456801 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fedc400b3b0 con 0x7fedcc0f05f0 2021-02-10T11:15:01.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.199 7fedca2bd700 1 -- 172.21.2.101:0/2109456801 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc400b730 con 0x7fedcc0f05f0 2021-02-10T11:15:01.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.199 7fedca2bd700 1 -- 172.21.2.101:0/2109456801 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fedc400bc10 con 0x7fedcc0f05f0 2021-02-10T11:15:01.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.199 7fedd1669700 1 -- 172.21.2.101:0/2109456801 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedcc0f05f0 msgr2=0x7fedcc0f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:01.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.199 7fedd1669700 1 --2- 172.21.2.101:0/2109456801 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedcc0f05f0 0x7fedcc0f20e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fedc4006f00 tx=0x7fedc4009f10).stop 2021-02-10T11:15:01.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.199 7fedd1669700 1 -- 172.21.2.101:0/2109456801 shutdown_connections 2021-02-10T11:15:01.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.199 7fedd1669700 1 --2- 172.21.2.101:0/2109456801 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedcc0f05f0 0x7fedcc0f20e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:01.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.199 7fedd1669700 1 -- 172.21.2.101:0/2109456801 >> 172.21.2.101:0/2109456801 conn(0x7fedcc0ce3e0 msgr2=0x7fedcc0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:01.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.199 7fedd1669700 1 -- 172.21.2.101:0/2109456801 shutdown_connections 2021-02-10T11:15:01.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.199 7fedd1669700 1 -- 172.21.2.101:0/2109456801 wait complete. 2021-02-10T11:15:01.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.214 7fedd1669700 1 Processor -- start 2021-02-10T11:15:01.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.214 7fedd1669700 1 -- start start 2021-02-10T11:15:01.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.214 7fedd1669700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedcc09f300 0x7fedcc0a73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:01.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.214 7fedd1669700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fedcc0aa870 0x7fedcc0aac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:01.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.214 7fedd1669700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedcc0ab290 0x7fedcc0a8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:01.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.214 7fedd1669700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc081940 con 0x7fedcc0ab290 2021-02-10T11:15:01.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.214 7fedd1669700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0a7bd0 con 0x7fedcc0aa870 2021-02-10T11:15:01.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.214 7fedd1669700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0ab690 con 0x7fedcc09f300 2021-02-10T11:15:01.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.214 7fedcbac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedcc0ab290 0x7fedcc0a8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:01.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.214 7fedcbac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedcc0ab290 0x7fedcc0a8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45410/0 (socket says 172.21.2.101:45410) 2021-02-10T11:15:01.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.214 7fedcbac0700 1 -- 172.21.2.101:0/949250606 learned_addr learned my addr 172.21.2.101:0/949250606 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:01.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.215 7fedcbac0700 1 -- 172.21.2.101:0/949250606 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedcc09f300 msgr2=0x7fedcc0a73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:01.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.215 7fedcbac0700 1 --2- 172.21.2.101:0/949250606 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedcc09f300 0x7fedcc0a73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:01.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.215 7fedcbac0700 1 -- 172.21.2.101:0/949250606 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fedcc0aa870 msgr2=0x7fedcc0aac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:01.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.215 7fedcbac0700 1 --2- 172.21.2.101:0/949250606 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fedcc0aa870 0x7fedcc0aac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:01.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.215 7fedcbac0700 1 -- 172.21.2.101:0/949250606 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc09f700 con 0x7fedcc0ab290 2021-02-10T11:15:01.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.215 7fedcbac0700 1 --2- 172.21.2.101:0/949250606 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedcc0ab290 0x7fedcc0a8150 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7fedc4009390 tx=0x7fedc400bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:01.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.215 7fedb3fff700 1 -- 172.21.2.101:0/949250606 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fedc4003030 con 0x7fedcc0ab290 2021-02-10T11:15:01.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.215 7fedb3fff700 1 -- 172.21.2.101:0/949250606 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc40068e0 con 0x7fedcc0ab290 2021-02-10T11:15:01.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.215 7fedb3fff700 1 -- 172.21.2.101:0/949250606 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fedc400c570 con 0x7fedcc0ab290 2021-02-10T11:15:01.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.215 7fedd1669700 1 -- 172.21.2.101:0/949250606 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedcc09f700 con 0x7fedcc0ab290 2021-02-10T11:15:01.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.215 7fedd1669700 1 -- 172.21.2.101:0/949250606 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedcc0a8a60 con 0x7fedcc0ab290 2021-02-10T11:15:01.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.216 7fedb3fff700 1 -- 172.21.2.101:0/949250606 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fedc4001270 con 0x7fedcc0ab290 2021-02-10T11:15:01.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.216 7fedb3fff700 1 --2- 172.21.2.101:0/949250606 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fedac0186c0 0x7fedac01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:01.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.216 7fedb3fff700 1 -- 172.21.2.101:0/949250606 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fedc40328d0 con 0x7fedcc0ab290 2021-02-10T11:15:01.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.216 7fedcb2bf700 1 --2- 172.21.2.101:0/949250606 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fedac0186c0 0x7fedac01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:01.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.217 7fedcb2bf700 1 --2- 172.21.2.101:0/949250606 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fedac0186c0 0x7fedac01abc0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:01.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.248 7fedd1669700 1 -- 172.21.2.101:0/949250606 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedcc07ed20 con 0x7fedcc0ab290 2021-02-10T11:15:01.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.250 7fedb3fff700 1 -- 172.21.2.101:0/949250606 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fedc4006c80 con 0x7fedcc0ab290 2021-02-10T11:15:01.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.379 7fedd1669700 1 -- 172.21.2.101:0/949250606 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fedcc0ce3e0 con 0x7fedcc0ab290 2021-02-10T11:15:01.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.382 7fedb3fff700 1 -- 172.21.2.101:0/949250606 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v13) v1 ==== 184+0+0 (secure 0 0 0) 0x7fedc401f610 con 0x7fedcc0ab290 2021-02-10T11:15:01.391 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.0 2021-02-10T11:15:01.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.387 7fedd1669700 1 -- 172.21.2.101:0/949250606 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fedac0186c0 msgr2=0x7fedac01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:01.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.387 7fedd1669700 1 --2- 172.21.2.101:0/949250606 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fedac0186c0 0x7fedac01abc0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:01.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.387 7fedd1669700 1 -- 172.21.2.101:0/949250606 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedcc0ab290 msgr2=0x7fedcc0a8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:01.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.387 7fedd1669700 1 --2- 172.21.2.101:0/949250606 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedcc0ab290 0x7fedcc0a8150 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7fedc4009390 tx=0x7fedc400bd40).stop 2021-02-10T11:15:01.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.388 7fedd1669700 1 -- 172.21.2.101:0/949250606 shutdown_connections 2021-02-10T11:15:01.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.388 7fedd1669700 1 --2- 172.21.2.101:0/949250606 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedcc0ab290 0x7fedcc0a8150 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:01.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.388 7fedd1669700 1 --2- 172.21.2.101:0/949250606 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fedcc0aa870 0x7fedcc0aac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:01.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.388 7fedd1669700 1 --2- 172.21.2.101:0/949250606 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fedac0186c0 0x7fedac01abc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:01.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.388 7fedd1669700 1 --2- 172.21.2.101:0/949250606 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedcc09f300 0x7fedcc0a73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:01.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.388 7fedd1669700 1 -- 172.21.2.101:0/949250606 >> 172.21.2.101:0/949250606 conn(0x7fedcc09fab0 msgr2=0x7fedcc089220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:01.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.388 7fedd1669700 1 -- 172.21.2.101:0/949250606 shutdown_connections 2021-02-10T11:15:01.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.388 7fedd1669700 1 -- 172.21.2.101:0/949250606 wait complete. 2021-02-10T11:15:01.400 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:15:01.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.817 7f7bc5d9b700 1 Processor -- start 2021-02-10T11:15:01.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.817 7f7bc5d9b700 1 -- start start 2021-02-10T11:15:01.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.818 7f7bc5d9b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7bc01b0470 0x7f7bc01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:01.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.818 7f7bc5d9b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7bc00ef610 0x7f7bc00efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:01.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.818 7f7bc5d9b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc00f0b50 con 0x7f7bc00f0030 2021-02-10T11:15:01.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.818 7f7bc5d9b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc00e52e0 con 0x7f7bc01b0470 2021-02-10T11:15:01.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.818 7f7bc5d9b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc00e5490 con 0x7f7bc00ef610 2021-02-10T11:15:01.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.818 7f7bbf7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7bc01b0470 0x7f7bc01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:01.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.818 7f7bbf7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7bc01b0470 0x7f7bc01b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45416/0 (socket says 172.21.2.101:45416) 2021-02-10T11:15:01.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.818 7f7bbf7fe700 1 -- 172.21.2.101:0/3559000599 learned_addr learned my addr 172.21.2.101:0/3559000599 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:01.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.818 7f7bbf7fe700 1 -- 172.21.2.101:0/3559000599 >> v2:172.21.2.108:3300/0 conn(0x7f7bc00ef610 msgr2=0x7f7bc00efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:01.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.818 7f7bbf7fe700 1 --2- 172.21.2.101:0/3559000599 >> v2:172.21.2.108:3300/0 conn(0x7f7bc00ef610 0x7f7bc00efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:01.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.818 7f7bbf7fe700 1 -- 172.21.2.101:0/3559000599 >> v1:172.21.2.101:6789/0 conn(0x7f7bc00f0030 legacy=0x7f7bc00f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:01.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.818 7f7bbf7fe700 1 -- 172.21.2.101:0/3559000599 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc00f0430 con 0x7f7bc01b0470 2021-02-10T11:15:01.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.818 7f7bbf7fe700 1 --2- 172.21.2.101:0/3559000599 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bc01b0470 0x7f7bc01b0870 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f7bb0006700 tx=0x7f7bb0009b80).ready entity=mon.0 client_cookie=8f8436f3245899ff server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:01.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.819 7f7bbe7fc700 1 -- 172.21.2.101:0/3559000599 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7bb000c0c0 con 0x7f7bc01b0470 2021-02-10T11:15:01.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.819 7f7bbe7fc700 1 -- 172.21.2.101:0/3559000599 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb000c400 con 0x7f7bc01b0470 2021-02-10T11:15:01.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.819 7f7bbe7fc700 1 -- 172.21.2.101:0/3559000599 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7bb000c920 con 0x7f7bc01b0470 2021-02-10T11:15:01.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.819 7f7bc5d9b700 1 -- 172.21.2.101:0/3559000599 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bc01b0470 msgr2=0x7f7bc01b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:01.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.819 7f7bc5d9b700 1 --2- 172.21.2.101:0/3559000599 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bc01b0470 0x7f7bc01b0870 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f7bb0006700 tx=0x7f7bb0009b80).stop 2021-02-10T11:15:01.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.819 7f7bc5d9b700 1 -- 172.21.2.101:0/3559000599 shutdown_connections 2021-02-10T11:15:01.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.819 7f7bc5d9b700 1 --2- 172.21.2.101:0/3559000599 >> v2:172.21.2.108:3300/0 conn(0x7f7bc00ef610 0x7f7bc00efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:01.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.819 7f7bc5d9b700 1 --2- 172.21.2.101:0/3559000599 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bc01b0470 0x7f7bc01b0870 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:01.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.819 7f7bc5d9b700 1 -- 172.21.2.101:0/3559000599 >> 172.21.2.101:0/3559000599 conn(0x7f7bc00ce3e0 msgr2=0x7f7bc00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:01.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.819 7f7bc5d9b700 1 -- 172.21.2.101:0/3559000599 shutdown_connections 2021-02-10T11:15:01.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.819 7f7bc5d9b700 1 -- 172.21.2.101:0/3559000599 wait complete. 2021-02-10T11:15:01.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.834 7f7bc5d9b700 1 Processor -- start 2021-02-10T11:15:01.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.834 7f7bc5d9b700 1 -- start start 2021-02-10T11:15:01.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.834 7f7bc5d9b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7bc018ecd0 0x7f7bc018e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:01.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.834 7f7bc5d9b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7bc01a3d90 0x7f7bc01ab630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:01.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.834 7f7bc5d9b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bc01a4190 0x7f7bc01ac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:01.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.834 7f7bc5d9b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc018f150 con 0x7f7bc01a4190 2021-02-10T11:15:01.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.834 7f7bc5d9b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc01abc50 con 0x7f7bc01a3d90 2021-02-10T11:15:01.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.834 7f7bc5d9b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc01abe70 con 0x7f7bc018ecd0 2021-02-10T11:15:01.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.835 7f7bbffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bc01a4190 0x7f7bc01ac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:01.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.835 7f7bbffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bc01a4190 0x7f7bc01ac5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45426/0 (socket says 172.21.2.101:45426) 2021-02-10T11:15:01.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.835 7f7bbffff700 1 -- 172.21.2.101:0/3396056406 learned_addr learned my addr 172.21.2.101:0/3396056406 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:01.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.835 7f7bbffff700 1 -- 172.21.2.101:0/3396056406 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7bc018ecd0 msgr2=0x7f7bc018e4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:01.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.835 7f7bbffff700 1 --2- 172.21.2.101:0/3396056406 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7bc018ecd0 0x7f7bc018e4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:01.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.835 7f7bbffff700 1 -- 172.21.2.101:0/3396056406 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7bc01a3d90 msgr2=0x7f7bc01ab630 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:01.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.835 7f7bbffff700 1 --2- 172.21.2.101:0/3396056406 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7bc01a3d90 0x7f7bc01ab630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:01.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.835 7f7bbffff700 1 -- 172.21.2.101:0/3396056406 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc01acc30 con 0x7f7bc01a4190 2021-02-10T11:15:01.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.835 7f7bbffff700 1 --2- 172.21.2.101:0/3396056406 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bc01a4190 0x7f7bc01ac5e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f7bb8009980 tx=0x7f7bb800bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:01.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.835 7f7bb7fff700 1 -- 172.21.2.101:0/3396056406 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7bb8005870 con 0x7f7bc01a4190 2021-02-10T11:15:01.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.835 7f7bb7fff700 1 -- 172.21.2.101:0/3396056406 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb8005a70 con 0x7f7bc01a4190 2021-02-10T11:15:01.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.835 7f7bb7fff700 1 -- 172.21.2.101:0/3396056406 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7bb8006030 con 0x7f7bc01a4190 2021-02-10T11:15:01.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.835 7f7bc5d9b700 1 -- 172.21.2.101:0/3396056406 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc01acc30 con 0x7f7bc01a4190 2021-02-10T11:15:01.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.835 7f7bc5d9b700 1 -- 172.21.2.101:0/3396056406 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc01acc30 con 0x7f7bc01a4190 2021-02-10T11:15:01.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.836 7f7bb7fff700 1 -- 172.21.2.101:0/3396056406 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7bb8005c50 con 0x7f7bc01a4190 2021-02-10T11:15:01.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.836 7f7bb7fff700 1 --2- 172.21.2.101:0/3396056406 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ba40186c0 0x7f7ba401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:01.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.836 7f7bb7fff700 1 -- 172.21.2.101:0/3396056406 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7bb8032b60 con 0x7f7bc01a4190 2021-02-10T11:15:01.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.836 7f7bbf7fe700 1 --2- 172.21.2.101:0/3396056406 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ba40186c0 0x7f7ba401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:01.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.837 7f7bbf7fe700 1 --2- 172.21.2.101:0/3396056406 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ba40186c0 0x7f7ba401abc0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:01.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.868 7f7bc5d9b700 1 -- 172.21.2.101:0/3396056406 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc00edd30 con 0x7f7bc01a4190 2021-02-10T11:15:01.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:01.870 7f7bb7fff700 1 -- 172.21.2.101:0/3396056406 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7bb800be10 con 0x7f7bc01a4190 2021-02-10T11:15:02.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.000 7f7bc5d9b700 1 -- 172.21.2.101:0/3396056406 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f7bc00edd30 con 0x7f7bc01a4190 2021-02-10T11:15:02.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.004 7f7bb7fff700 1 -- 172.21.2.101:0/3396056406 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v14) v1 ==== 184+0+0 (secure 0 0 0) 0x7f7bb8005a70 con 0x7f7bc01a4190 2021-02-10T11:15:02.012 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.1 2021-02-10T11:15:02.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.008 7f7bc5d9b700 1 -- 172.21.2.101:0/3396056406 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ba40186c0 msgr2=0x7f7ba401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:02.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.008 7f7bc5d9b700 1 --2- 172.21.2.101:0/3396056406 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ba40186c0 0x7f7ba401abc0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:02.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.009 7f7bc5d9b700 1 -- 172.21.2.101:0/3396056406 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bc01a4190 msgr2=0x7f7bc01ac5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:02.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.009 7f7bc5d9b700 1 --2- 172.21.2.101:0/3396056406 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bc01a4190 0x7f7bc01ac5e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f7bb8009980 tx=0x7f7bb800bcd0).stop 2021-02-10T11:15:02.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.009 7f7bc5d9b700 1 -- 172.21.2.101:0/3396056406 shutdown_connections 2021-02-10T11:15:02.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.009 7f7bc5d9b700 1 --2- 172.21.2.101:0/3396056406 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bc01a4190 0x7f7bc01ac5e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:02.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.009 7f7bc5d9b700 1 --2- 172.21.2.101:0/3396056406 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7bc01a3d90 0x7f7bc01ab630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:02.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.009 7f7bc5d9b700 1 --2- 172.21.2.101:0/3396056406 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ba40186c0 0x7f7ba401abc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:02.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.009 7f7bc5d9b700 1 --2- 172.21.2.101:0/3396056406 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7bc018ecd0 0x7f7bc018e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:02.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.009 7f7bc5d9b700 1 -- 172.21.2.101:0/3396056406 >> 172.21.2.101:0/3396056406 conn(0x7f7bc01a90c0 msgr2=0x7f7bc01b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:02.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.009 7f7bc5d9b700 1 -- 172.21.2.101:0/3396056406 shutdown_connections 2021-02-10T11:15:02.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.009 7f7bc5d9b700 1 -- 172.21.2.101:0/3396056406 wait complete. 2021-02-10T11:15:02.022 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:15:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.438 7f633d29d700 1 Processor -- start 2021-02-10T11:15:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.438 7f633d29d700 1 -- start start 2021-02-10T11:15:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.439 7f633d29d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f63381b0470 0x7f63381b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.439 7f633d29d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63380f0b50 con 0x7f63380ef610 2021-02-10T11:15:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.439 7f633d29d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63380f24e0 con 0x7f63380f20e0 2021-02-10T11:15:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.439 7f633d29d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63380e5ea0 con 0x7f63381b0470 2021-02-10T11:15:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.439 7f633707d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f63381b0470 0x7f63381b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.439 7f633687c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f63380ef610 0x7f63380efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45924/0 (socket says 172.21.2.101:45924) 2021-02-10T11:15:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.439 7f633687c700 1 -- 172.21.2.101:0/1508200897 learned_addr learned my addr 172.21.2.101:0/1508200897 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:15:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.439 7f633707d700 1 -- 172.21.2.101:0/1508200897 >> v1:172.21.2.108:6789/0 conn(0x7f63380f20e0 legacy=0x7f63380f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.439 7f633707d700 1 -- 172.21.2.101:0/1508200897 >> v1:172.21.2.101:6789/0 conn(0x7f63380ef610 legacy=0x7f63380efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.439 7f633607b700 1 -- 172.21.2.101:0/1508200897 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3726115656 0 0) 0x7f63280009b0 con 0x7f63380ef610 2021-02-10T11:15:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.439 7f633707d700 1 -- 172.21.2.101:0/1508200897 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63380e52e0 con 0x7f63381b0470 2021-02-10T11:15:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.439 7f633707d700 1 --2- 172.21.2.101:0/1508200897 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63381b0470 0x7f63381b0870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f6320006650 tx=0x7f6320009a00).ready entity=mon.0 client_cookie=62201b38cb3774ba server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.440 7f633607b700 1 -- 172.21.2.101:0/1508200897 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6320009df0 con 0x7f63381b0470 2021-02-10T11:15:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.440 7f633607b700 1 -- 172.21.2.101:0/1508200897 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63200009b0 con 0x7f63381b0470 2021-02-10T11:15:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.440 7f633607b700 1 -- 172.21.2.101:0/1508200897 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f632000c2b0 con 0x7f63381b0470 2021-02-10T11:15:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.440 7f633d29d700 1 -- 172.21.2.101:0/1508200897 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63381b0470 msgr2=0x7f63381b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.440 7f633d29d700 1 --2- 172.21.2.101:0/1508200897 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63381b0470 0x7f63381b0870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f6320006650 tx=0x7f6320009a00).stop 2021-02-10T11:15:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.440 7f633d29d700 1 -- 172.21.2.101:0/1508200897 shutdown_connections 2021-02-10T11:15:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.440 7f633d29d700 1 --2- 172.21.2.101:0/1508200897 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63381b0470 0x7f63381b0870 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.440 7f633d29d700 1 -- 172.21.2.101:0/1508200897 >> 172.21.2.101:0/1508200897 conn(0x7f63380ce3e0 msgr2=0x7f63380e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:02.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.440 7f633d29d700 1 -- 172.21.2.101:0/1508200897 shutdown_connections 2021-02-10T11:15:02.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.440 7f633d29d700 1 -- 172.21.2.101:0/1508200897 wait complete. 2021-02-10T11:15:02.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.455 7f633d29d700 1 Processor -- start 2021-02-10T11:15:02.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.455 7f633d29d700 1 -- start start 2021-02-10T11:15:02.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.455 7f633d29d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f633809fb70 0x7f633818e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:02.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.455 7f633d29d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f633818ea60 0x7f63381b1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:02.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.455 7f633d29d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63381b1e70 0x7f63381b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:02.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.455 7f633d29d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63381b2270 con 0x7f633818ea60 2021-02-10T11:15:02.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.455 7f633d29d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633818ef20 con 0x7f63381b1e70 2021-02-10T11:15:02.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.455 7f633d29d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63381b24c0 con 0x7f633809fb70 2021-02-10T11:15:02.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.456 7f633687c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f633818ea60 0x7f63381b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:02.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.456 7f633687c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f633818ea60 0x7f63381b1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45438/0 (socket says 172.21.2.101:45438) 2021-02-10T11:15:02.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.456 7f633687c700 1 -- 172.21.2.101:0/1958666639 learned_addr learned my addr 172.21.2.101:0/1958666639 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:02.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.456 7f633687c700 1 -- 172.21.2.101:0/1958666639 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f633809fb70 msgr2=0x7f633818e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:02.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.456 7f633687c700 1 --2- 172.21.2.101:0/1958666639 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f633809fb70 0x7f633818e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:02.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.456 7f633687c700 1 -- 172.21.2.101:0/1958666639 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63381b1e70 msgr2=0x7f63381b2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:02.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.456 7f633687c700 1 --2- 172.21.2.101:0/1958666639 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63381b1e70 0x7f63381b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:02.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.456 7f633687c700 1 -- 172.21.2.101:0/1958666639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633809ff70 con 0x7f633818ea60 2021-02-10T11:15:02.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.456 7f633687c700 1 --2- 172.21.2.101:0/1958666639 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f633818ea60 0x7f63381b1850 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f6328001c60 tx=0x7f632800a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:02.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.456 7f632ffff700 1 -- 172.21.2.101:0/1958666639 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f632800acc0 con 0x7f633818ea60 2021-02-10T11:15:02.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.456 7f632ffff700 1 -- 172.21.2.101:0/1958666639 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6328004140 con 0x7f633818ea60 2021-02-10T11:15:02.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.456 7f632ffff700 1 -- 172.21.2.101:0/1958666639 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6328004660 con 0x7f633818ea60 2021-02-10T11:15:02.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.456 7f633d29d700 1 -- 172.21.2.101:0/1958666639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633809ff70 con 0x7f633818ea60 2021-02-10T11:15:02.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.456 7f633d29d700 1 -- 172.21.2.101:0/1958666639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633809ff70 con 0x7f633818ea60 2021-02-10T11:15:02.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.457 7f632ffff700 1 -- 172.21.2.101:0/1958666639 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6328000dd0 con 0x7f633818ea60 2021-02-10T11:15:02.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.457 7f632ffff700 1 --2- 172.21.2.101:0/1958666639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f631c018730 0x7f631c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:02.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.457 7f632ffff700 1 -- 172.21.2.101:0/1958666639 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6328032760 con 0x7f633818ea60 2021-02-10T11:15:02.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.457 7f633707d700 1 --2- 172.21.2.101:0/1958666639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f631c018730 0x7f631c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:02.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.458 7f633707d700 1 --2- 172.21.2.101:0/1958666639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f631c018730 0x7f631c01abc0 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:02.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.489 7f633d29d700 1 -- 172.21.2.101:0/1958666639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63381b2270 con 0x7f633818ea60 2021-02-10T11:15:02.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.491 7f632ffff700 1 -- 172.21.2.101:0/1958666639 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f63280009b0 con 0x7f633818ea60 2021-02-10T11:15:02.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.621 7f633d29d700 1 -- 172.21.2.101:0/1958666639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f63381b2270 con 0x7f633818ea60 2021-02-10T11:15:02.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.625 7f632ffff700 1 -- 172.21.2.101:0/1958666639 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v15) v1 ==== 184+0+0 (secure 0 0 0) 0x7f6328000dd0 con 0x7f633818ea60 2021-02-10T11:15:02.633 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.2 2021-02-10T11:15:02.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.631 7f633d29d700 1 -- 172.21.2.101:0/1958666639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f631c018730 msgr2=0x7f631c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:02.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.631 7f633d29d700 1 --2- 172.21.2.101:0/1958666639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f631c018730 0x7f631c01abc0 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:02.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.631 7f633d29d700 1 -- 172.21.2.101:0/1958666639 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f633818ea60 msgr2=0x7f63381b1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:02.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.631 7f633d29d700 1 --2- 172.21.2.101:0/1958666639 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f633818ea60 0x7f63381b1850 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f6328001c60 tx=0x7f632800a8d0).stop 2021-02-10T11:15:02.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.631 7f633d29d700 1 -- 172.21.2.101:0/1958666639 shutdown_connections 2021-02-10T11:15:02.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.631 7f633d29d700 1 --2- 172.21.2.101:0/1958666639 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63381b1e70 0x7f63381b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:02.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.631 7f633d29d700 1 --2- 172.21.2.101:0/1958666639 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f633818ea60 0x7f63381b1850 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:02.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.631 7f633d29d700 1 --2- 172.21.2.101:0/1958666639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f631c018730 0x7f631c01abc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:02.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.631 7f633d29d700 1 --2- 172.21.2.101:0/1958666639 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f633809fb70 0x7f633818e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:02.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.631 7f633d29d700 1 -- 172.21.2.101:0/1958666639 >> 172.21.2.101:0/1958666639 conn(0x7f633819f770 msgr2=0x7f633819ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:02.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.631 7f633d29d700 1 -- 172.21.2.101:0/1958666639 shutdown_connections 2021-02-10T11:15:02.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:02.631 7f633d29d700 1 -- 172.21.2.101:0/1958666639 wait complete. 2021-02-10T11:15:02.644 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:15:03.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.062 7f3f2b38a700 1 Processor -- start 2021-02-10T11:15:04.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.062 7f3f2b38a700 1 -- start start 2021-02-10T11:15:04.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.063 7f3f2b38a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3f241b0470 0x7f3f241b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:04.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.063 7f3f2b38a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3f240f20e0 0x7f3f240f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:04.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.063 7f3f2b38a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f240f0b50 con 0x7f3f240ef610 2021-02-10T11:15:04.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.063 7f3f2b38a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f240f0820 con 0x7f3f241b0470 2021-02-10T11:15:04.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.063 7f3f2b38a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f240f24e0 con 0x7f3f240f20e0 2021-02-10T11:15:04.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.063 7f3f291a6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3f241b0470 0x7f3f241b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:04.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.063 7f3f289a5700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f3f240ef610 0x7f3f240efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45936/0 (socket says 172.21.2.101:45936) 2021-02-10T11:15:04.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.063 7f3f289a5700 1 -- 172.21.2.101:0/451061530 learned_addr learned my addr 172.21.2.101:0/451061530 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:15:04.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.063 7f3f299a7700 1 --2- 172.21.2.101:0/451061530 >> v2:172.21.2.108:3300/0 conn(0x7f3f240f20e0 0x7f3f240f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:04.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.063 7f3f291a6700 1 -- 172.21.2.101:0/451061530 >> v2:172.21.2.108:3300/0 conn(0x7f3f240f20e0 msgr2=0x7f3f240f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:04.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.063 7f3f291a6700 1 --2- 172.21.2.101:0/451061530 >> v2:172.21.2.108:3300/0 conn(0x7f3f240f20e0 0x7f3f240f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:04.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.063 7f3f291a6700 1 -- 172.21.2.101:0/451061530 >> v1:172.21.2.101:6789/0 conn(0x7f3f240ef610 legacy=0x7f3f240efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:04.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.064 7f3f291a6700 1 -- 172.21.2.101:0/451061530 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f240e52e0 con 0x7f3f241b0470 2021-02-10T11:15:04.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.064 7f3f299a7700 1 --2- 172.21.2.101:0/451061530 >> v2:172.21.2.108:3300/0 conn(0x7f3f240f20e0 0x7f3f240f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:15:04.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.064 7f3f291a6700 1 --2- 172.21.2.101:0/451061530 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f241b0470 0x7f3f241b0870 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f3f20006650 tx=0x7f3f20009a80).ready entity=mon.0 client_cookie=b36ddb5214ff20f1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:04.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.064 7f3f17fff700 1 -- 172.21.2.101:0/451061530 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3f200009b0 con 0x7f3f241b0470 2021-02-10T11:15:04.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.064 7f3f17fff700 1 -- 172.21.2.101:0/451061530 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2000c0c0 con 0x7f3f241b0470 2021-02-10T11:15:04.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.064 7f3f17fff700 1 -- 172.21.2.101:0/451061530 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3f2000c5e0 con 0x7f3f241b0470 2021-02-10T11:15:04.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.064 7f3f2b38a700 1 -- 172.21.2.101:0/451061530 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f241b0470 msgr2=0x7f3f241b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:04.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.064 7f3f2b38a700 1 --2- 172.21.2.101:0/451061530 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f241b0470 0x7f3f241b0870 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f3f20006650 tx=0x7f3f20009a80).stop 2021-02-10T11:15:04.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.065 7f3f2b38a700 1 -- 172.21.2.101:0/451061530 shutdown_connections 2021-02-10T11:15:04.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.065 7f3f2b38a700 1 --2- 172.21.2.101:0/451061530 >> v2:172.21.2.108:3300/0 conn(0x7f3f240f20e0 0x7f3f240f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:04.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.065 7f3f2b38a700 1 --2- 172.21.2.101:0/451061530 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f241b0470 0x7f3f241b0870 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:04.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.065 7f3f2b38a700 1 -- 172.21.2.101:0/451061530 >> 172.21.2.101:0/451061530 conn(0x7f3f240ce3e0 msgr2=0x7f3f240e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:04.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.065 7f3f2b38a700 1 -- 172.21.2.101:0/451061530 shutdown_connections 2021-02-10T11:15:04.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.065 7f3f2b38a700 1 -- 172.21.2.101:0/451061530 wait complete. 2021-02-10T11:15:04.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.079 7f3f2b38a700 1 Processor -- start 2021-02-10T11:15:04.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.080 7f3f2b38a700 1 -- start start 2021-02-10T11:15:04.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.080 7f3f2b38a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f241b6de0 0x7f3f241b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:04.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.080 7f3f2b38a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3f2418f3f0 0x7f3f2418f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:04.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.080 7f3f2b38a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f2418fe10 0x7f3f2418ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:04.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.080 7f3f2b38a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2418d950 con 0x7f3f2418fe10 2021-02-10T11:15:04.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.080 7f3f2b38a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f241b71e0 con 0x7f3f2418f3f0 2021-02-10T11:15:04.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.080 7f3f2b38a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f241b7400 con 0x7f3f241b6de0 2021-02-10T11:15:04.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.080 7f3f299a7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f2418fe10 0x7f3f2418ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:04.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.080 7f3f299a7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f2418fe10 0x7f3f2418ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45450/0 (socket says 172.21.2.101:45450) 2021-02-10T11:15:04.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.080 7f3f299a7700 1 -- 172.21.2.101:0/417092877 learned_addr learned my addr 172.21.2.101:0/417092877 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:04.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.080 7f3f299a7700 1 -- 172.21.2.101:0/417092877 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f241b6de0 msgr2=0x7f3f241b63f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:04.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.080 7f3f299a7700 1 --2- 172.21.2.101:0/417092877 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f241b6de0 0x7f3f241b63f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:04.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.080 7f3f299a7700 1 -- 172.21.2.101:0/417092877 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3f2418f3f0 msgr2=0x7f3f2418f7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:04.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.080 7f3f299a7700 1 --2- 172.21.2.101:0/417092877 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3f2418f3f0 0x7f3f2418f7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:04.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.080 7f3f299a7700 1 -- 172.21.2.101:0/417092877 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f24190210 con 0x7f3f2418fe10 2021-02-10T11:15:04.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.080 7f3f299a7700 1 --2- 172.21.2.101:0/417092877 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f2418fe10 0x7f3f2418ddf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f3f1c009980 tx=0x7f3f1c00bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:04.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.081 7f3f15ffb700 1 -- 172.21.2.101:0/417092877 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3f1c00c2f0 con 0x7f3f2418fe10 2021-02-10T11:15:04.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.081 7f3f15ffb700 1 -- 172.21.2.101:0/417092877 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1c006790 con 0x7f3f2418fe10 2021-02-10T11:15:04.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.081 7f3f15ffb700 1 -- 172.21.2.101:0/417092877 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3f1c00c870 con 0x7f3f2418fe10 2021-02-10T11:15:04.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.081 7f3f2b38a700 1 -- 172.21.2.101:0/417092877 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f24190210 con 0x7f3f2418fe10 2021-02-10T11:15:04.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.081 7f3f2b38a700 1 -- 172.21.2.101:0/417092877 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f24190210 con 0x7f3f2418fe10 2021-02-10T11:15:04.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.081 7f3f15ffb700 1 -- 172.21.2.101:0/417092877 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3f1c0011f0 con 0x7f3f2418fe10 2021-02-10T11:15:04.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.081 7f3f15ffb700 1 --2- 172.21.2.101:0/417092877 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f04018700 0x7f3f0401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:04.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.081 7f3f15ffb700 1 -- 172.21.2.101:0/417092877 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3f1c032bd0 con 0x7f3f2418fe10 2021-02-10T11:15:04.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.081 7f3f291a6700 1 --2- 172.21.2.101:0/417092877 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f04018700 0x7f3f0401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:04.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.082 7f3f291a6700 1 --2- 172.21.2.101:0/417092877 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f04018700 0x7f3f0401abc0 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:04.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.113 7f3f2b38a700 1 -- 172.21.2.101:0/417092877 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f241059e0 con 0x7f3f2418fe10 2021-02-10T11:15:04.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.115 7f3f15ffb700 1 -- 172.21.2.101:0/417092877 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3f1c00be10 con 0x7f3f2418fe10 2021-02-10T11:15:04.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.245 7f3f2b38a700 1 -- 172.21.2.101:0/417092877 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f3f240f0b50 con 0x7f3f2418fe10 2021-02-10T11:15:04.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.249 7f3f15ffb700 1 -- 172.21.2.101:0/417092877 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v16) v1 ==== 184+0+0 (secure 0 0 0) 0x7f3f1c0066e0 con 0x7f3f2418fe10 2021-02-10T11:15:04.807 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.3 2021-02-10T11:15:04.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.255 7f3f2b38a700 1 -- 172.21.2.101:0/417092877 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f04018700 msgr2=0x7f3f0401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:04.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.255 7f3f2b38a700 1 --2- 172.21.2.101:0/417092877 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f04018700 0x7f3f0401abc0 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:04.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.255 7f3f2b38a700 1 -- 172.21.2.101:0/417092877 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f2418fe10 msgr2=0x7f3f2418ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:04.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.255 7f3f2b38a700 1 --2- 172.21.2.101:0/417092877 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f2418fe10 0x7f3f2418ddf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f3f1c009980 tx=0x7f3f1c00bcd0).stop 2021-02-10T11:15:04.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.255 7f3f2b38a700 1 -- 172.21.2.101:0/417092877 shutdown_connections 2021-02-10T11:15:04.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.255 7f3f2b38a700 1 --2- 172.21.2.101:0/417092877 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f2418fe10 0x7f3f2418ddf0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:04.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.255 7f3f2b38a700 1 --2- 172.21.2.101:0/417092877 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3f2418f3f0 0x7f3f2418f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:04.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.255 7f3f2b38a700 1 --2- 172.21.2.101:0/417092877 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f04018700 0x7f3f0401abc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:04.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.255 7f3f2b38a700 1 --2- 172.21.2.101:0/417092877 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f241b6de0 0x7f3f241b63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:04.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.255 7f3f2b38a700 1 -- 172.21.2.101:0/417092877 >> 172.21.2.101:0/417092877 conn(0x7f3f241a37a0 msgr2=0x7f3f2418c230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:04.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.255 7f3f2b38a700 1 -- 172.21.2.101:0/417092877 shutdown_connections 2021-02-10T11:15:04.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:03.255 7f3f2b38a700 1 -- 172.21.2.101:0/417092877 wait complete. 2021-02-10T11:15:04.808 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:15:05.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.214 7fc5a9309700 1 Processor -- start 2021-02-10T11:15:05.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.214 7fc5a9309700 1 -- start start 2021-02-10T11:15:05.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.214 7fc5a9309700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc5a40e5070 0x7fc5a41b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:05.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.214 7fc5a9309700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5a41b3e40 0x7fc5a407d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:05.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.214 7fc5a9309700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc5a40e5470 0x7fc5a407dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:05.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.214 7fc5a9309700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a407ed80 con 0x7fc5a40e5070 2021-02-10T11:15:05.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.214 7fc5a9309700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a407e2d0 con 0x7fc5a40e5470 2021-02-10T11:15:05.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.214 7fc5a9309700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a407e4c0 con 0x7fc5a41b3e40 2021-02-10T11:15:05.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.214 7fc5a307d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc5a40e5070 0x7fc5a41b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:05.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.215 7fc5a307d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc5a40e5070 0x7fc5a41b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45454/0 (socket says 172.21.2.101:45454) 2021-02-10T11:15:05.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.215 7fc5a307d700 1 -- 172.21.2.101:0/996509657 learned_addr learned my addr 172.21.2.101:0/996509657 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:05.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.215 7fc5a307d700 1 -- 172.21.2.101:0/996509657 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5a41b3e40 msgr2=0x7fc5a407d690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:15:05.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.215 7fc5a307d700 1 --2- 172.21.2.101:0/996509657 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5a41b3e40 0x7fc5a407d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:05.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.215 7fc5a307d700 1 -- 172.21.2.101:0/996509657 >> v2:172.21.2.108:3300/0 conn(0x7fc5a40e5470 msgr2=0x7fc5a407dcb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:15:05.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.215 7fc5a307d700 1 --2- 172.21.2.101:0/996509657 >> v2:172.21.2.108:3300/0 conn(0x7fc5a40e5470 0x7fc5a407dcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:05.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.215 7fc5a307d700 1 -- 172.21.2.101:0/996509657 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a407e6e0 con 0x7fc5a40e5070 2021-02-10T11:15:05.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.215 7fc5a307d700 1 --2- 172.21.2.101:0/996509657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5a40e5070 0x7fc5a41b3820 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7fc58c006700 tx=0x7fc58c009b00).ready entity=mon.0 client_cookie=c8c2eefe48a86dd7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:05.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.215 7fc5a207b700 1 -- 172.21.2.101:0/996509657 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc58c00c0c0 con 0x7fc5a40e5070 2021-02-10T11:15:05.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.215 7fc5a207b700 1 -- 172.21.2.101:0/996509657 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58c00c340 con 0x7fc5a40e5070 2021-02-10T11:15:05.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.215 7fc5a207b700 1 -- 172.21.2.101:0/996509657 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc58c00c820 con 0x7fc5a40e5070 2021-02-10T11:15:05.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.215 7fc5a9309700 1 -- 172.21.2.101:0/996509657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5a40e5070 msgr2=0x7fc5a41b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:05.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.215 7fc5a9309700 1 --2- 172.21.2.101:0/996509657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5a40e5070 0x7fc5a41b3820 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7fc58c006700 tx=0x7fc58c009b00).stop 2021-02-10T11:15:05.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.216 7fc5a9309700 1 -- 172.21.2.101:0/996509657 shutdown_connections 2021-02-10T11:15:05.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.216 7fc5a9309700 1 --2- 172.21.2.101:0/996509657 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5a41b3e40 0x7fc5a407d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:05.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.216 7fc5a9309700 1 --2- 172.21.2.101:0/996509657 >> v2:172.21.2.108:3300/0 conn(0x7fc5a40e5470 0x7fc5a407dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:05.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.216 7fc5a9309700 1 --2- 172.21.2.101:0/996509657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5a40e5070 0x7fc5a41b3820 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:05.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.216 7fc5a9309700 1 -- 172.21.2.101:0/996509657 >> 172.21.2.101:0/996509657 conn(0x7fc5a40ce690 msgr2=0x7fc5a40e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:05.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.216 7fc5a9309700 1 -- 172.21.2.101:0/996509657 shutdown_connections 2021-02-10T11:15:05.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.216 7fc5a9309700 1 -- 172.21.2.101:0/996509657 wait complete. 2021-02-10T11:15:05.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.230 7fc5a9309700 1 Processor -- start 2021-02-10T11:15:05.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.230 7fc5a9309700 1 -- start start 2021-02-10T11:15:05.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.231 7fc5a9309700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5a4192960 0x7fc5a4192d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:05.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.231 7fc5a9309700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc5a40ad110 0x7fc5a40ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:05.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.231 7fc5a9309700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5a40adb30 0x7fc5a408e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:05.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.231 7fc5a9309700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a41a03e0 con 0x7fc5a40adb30 2021-02-10T11:15:05.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.231 7fc5a9309700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a4193380 con 0x7fc5a40ad110 2021-02-10T11:15:05.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.231 7fc5a9309700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a408ea80 con 0x7fc5a4192960 2021-02-10T11:15:05.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.231 7fc5a387e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5a40adb30 0x7fc5a408e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:05.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.231 7fc5a387e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5a40adb30 0x7fc5a408e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45464/0 (socket says 172.21.2.101:45464) 2021-02-10T11:15:05.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.231 7fc5a387e700 1 -- 172.21.2.101:0/2900395995 learned_addr learned my addr 172.21.2.101:0/2900395995 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:05.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.231 7fc5a387e700 1 -- 172.21.2.101:0/2900395995 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5a4192960 msgr2=0x7fc5a4192d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:05.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.231 7fc5a387e700 1 --2- 172.21.2.101:0/2900395995 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5a4192960 0x7fc5a4192d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:05.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.231 7fc5a387e700 1 -- 172.21.2.101:0/2900395995 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc5a40ad110 msgr2=0x7fc5a40ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:05.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.231 7fc5a387e700 1 --2- 172.21.2.101:0/2900395995 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc5a40ad110 0x7fc5a40ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:05.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.231 7fc5a387e700 1 -- 172.21.2.101:0/2900395995 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a408eca0 con 0x7fc5a40adb30 2021-02-10T11:15:05.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.232 7fc5a387e700 1 --2- 172.21.2.101:0/2900395995 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5a40adb30 0x7fc5a408e460 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fc59c009960 tx=0x7fc59c00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:05.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.232 7fc59bfff700 1 -- 172.21.2.101:0/2900395995 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc59c003030 con 0x7fc5a40adb30 2021-02-10T11:15:05.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.232 7fc59bfff700 1 -- 172.21.2.101:0/2900395995 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59c001310 con 0x7fc5a40adb30 2021-02-10T11:15:05.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.232 7fc5a9309700 1 -- 172.21.2.101:0/2900395995 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a408eca0 con 0x7fc5a40adb30 2021-02-10T11:15:05.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.232 7fc5a9309700 1 -- 172.21.2.101:0/2900395995 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a408f230 con 0x7fc5a40adb30 2021-02-10T11:15:05.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.232 7fc59bfff700 1 -- 172.21.2.101:0/2900395995 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc59c001830 con 0x7fc5a40adb30 2021-02-10T11:15:05.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.232 7fc59bfff700 1 -- 172.21.2.101:0/2900395995 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc59c0014f0 con 0x7fc5a40adb30 2021-02-10T11:15:05.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.233 7fc59bfff700 1 --2- 172.21.2.101:0/2900395995 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5880186d0 0x7fc58801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:05.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.233 7fc59bfff700 1 -- 172.21.2.101:0/2900395995 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc59c032b70 con 0x7fc5a40adb30 2021-02-10T11:15:05.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.233 7fc5a307d700 1 --2- 172.21.2.101:0/2900395995 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5880186d0 0x7fc58801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:05.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.233 7fc5a307d700 1 --2- 172.21.2.101:0/2900395995 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5880186d0 0x7fc58801abd0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:05.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.265 7fc5a9309700 1 -- 172.21.2.101:0/2900395995 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a4107120 con 0x7fc5a40adb30 2021-02-10T11:15:05.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.267 7fc59bfff700 1 -- 172.21.2.101:0/2900395995 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc59c001ee0 con 0x7fc5a40adb30 2021-02-10T11:15:05.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.395 7fc5a9309700 1 -- 172.21.2.101:0/2900395995 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5a407ed80 con 0x7fc5a40adb30 2021-02-10T11:15:05.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.395 7fc59bfff700 1 -- 172.21.2.101:0/2900395995 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+2100 (secure 0 0 0) 0x7fc59c000f40 con 0x7fc5a40adb30 2021-02-10T11:15:05.401 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:15:05.401 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/3226463341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3226463341},{"type":"v1","addr":"172.21.2.101:6835","nonce":3226463341}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:14:57.130312","modified":"2021-02-10 11:14:59.506936","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4373":{"gid":4373,"name":"b","rank":0,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.2.108:6833/3230549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3230549207},{"type":"v1","addr":"172.21.2.108:6833","nonce":3230549207}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-02-10T11:15:05.401 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 13 2021-02-10T11:15:05.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.397 7fc5a9309700 1 -- 172.21.2.101:0/2900395995 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5880186d0 msgr2=0x7fc58801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:05.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.397 7fc5a9309700 1 --2- 172.21.2.101:0/2900395995 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5880186d0 0x7fc58801abd0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:05.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.397 7fc5a9309700 1 -- 172.21.2.101:0/2900395995 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5a40adb30 msgr2=0x7fc5a408e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:05.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.397 7fc5a9309700 1 --2- 172.21.2.101:0/2900395995 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5a40adb30 0x7fc5a408e460 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fc59c009960 tx=0x7fc59c00bcf0).stop 2021-02-10T11:15:05.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.397 7fc5a9309700 1 -- 172.21.2.101:0/2900395995 shutdown_connections 2021-02-10T11:15:05.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.397 7fc5a9309700 1 --2- 172.21.2.101:0/2900395995 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5a40adb30 0x7fc5a408e460 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:05.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.397 7fc5a9309700 1 --2- 172.21.2.101:0/2900395995 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc5a40ad110 0x7fc5a40ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:05.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.397 7fc5a9309700 1 --2- 172.21.2.101:0/2900395995 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5880186d0 0x7fc58801abd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:05.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.397 7fc5a9309700 1 --2- 172.21.2.101:0/2900395995 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5a4192960 0x7fc5a4192d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:05.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.397 7fc5a9309700 1 -- 172.21.2.101:0/2900395995 >> 172.21.2.101:0/2900395995 conn(0x7fc5a40891f0 msgr2=0x7fc5a41a0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:05.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.398 7fc5a9309700 1 -- 172.21.2.101:0/2900395995 shutdown_connections 2021-02-10T11:15:05.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:05.398 7fc5a9309700 1 -- 172.21.2.101:0/2900395995 wait complete. 2021-02-10T11:15:05.611 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:14:57.130312', 'modified': '2021-02-10 11:14:59.506936', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4373}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4373': {'gid': 4373, 'name': 'b', 'rank': 0, 'incarnation': 10, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.2.108:6833/3230549207', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 3230549207}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 3230549207}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:15:05.612 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:15:05.612 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:15:06.008 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:15:14.382 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:15:14.382 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:15:14.382 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 4373, 2021-02-10T11:15:14.382 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:15:14.382 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:15:14.383 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 8.8718807319999993, 2021-02-10T11:15:14.383 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 13, 2021-02-10T11:15:14.383 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 29, 2021-02-10T11:15:14.383 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 29, 2021-02-10T11:15:14.383 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 27.296966208000001 2021-02-10T11:15:14.383 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:15:14.384 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:15:14.385 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 4373, "want_state": "up:active", "state": "up:active", "rank_uptime": 8.8718807319999993, "mdsmap_epoch": 13, "osdmap_epoch": 29, "osdmap_epoch_barrier": 29, "uptime": 27.296966208000001 } 2021-02-10T11:15:14.385 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:15:14.388 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:15:14.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.794 7f15aca6f700 1 Processor -- start 2021-02-10T11:15:14.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.794 7f15aca6f700 1 -- start start 2021-02-10T11:15:14.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.794 7f15aca6f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15a80d17f0 0x7f15a81b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:14.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.794 7f15aca6f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f15a80d1bf0 0x7f15a81b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:14.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.794 7f15aca6f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f15a81b3e20 0x7f15a80ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:14.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.794 7f15aca6f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a80e5eb0 con 0x7f15a80d1bf0 2021-02-10T11:15:14.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.794 7f15aca6f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a80f0b20 con 0x7f15a81b3e20 2021-02-10T11:15:14.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.794 7f15aca6f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a80cf510 con 0x7f15a80d17f0 2021-02-10T11:15:14.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.794 7f15a605d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f15a80d1bf0 0x7f15a81b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:14.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.794 7f15a605d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f15a80d1bf0 0x7f15a81b3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45468/0 (socket says 172.21.2.101:45468) 2021-02-10T11:15:14.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.794 7f15a605d700 1 -- 172.21.2.101:0/4114670389 learned_addr learned my addr 172.21.2.101:0/4114670389 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:14.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.795 7f15a605d700 1 -- 172.21.2.101:0/4114670389 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15a80d17f0 msgr2=0x7f15a81b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:14.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.795 7f15a605d700 1 --2- 172.21.2.101:0/4114670389 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15a80d17f0 0x7f15a81b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:14.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.795 7f15a605d700 1 -- 172.21.2.101:0/4114670389 >> v2:172.21.2.108:3300/0 conn(0x7f15a81b3e20 msgr2=0x7f15a80ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:14.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.795 7f15a605d700 1 --2- 172.21.2.101:0/4114670389 >> v2:172.21.2.108:3300/0 conn(0x7f15a81b3e20 0x7f15a80ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:14.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.795 7f15a605d700 1 -- 172.21.2.101:0/4114670389 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a80cf730 con 0x7f15a80d1bf0 2021-02-10T11:15:14.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.795 7f15a605d700 1 --2- 172.21.2.101:0/4114670389 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15a80d1bf0 0x7f15a81b3800 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f159c006700 tx=0x7f159c009b80).ready entity=mon.0 client_cookie=e1943e5f10c85dfc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:14.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.795 7f15a585c700 1 -- 172.21.2.101:0/4114670389 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f159c00c0c0 con 0x7f15a80d1bf0 2021-02-10T11:15:14.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.795 7f15a585c700 1 -- 172.21.2.101:0/4114670389 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159c00c340 con 0x7f15a80d1bf0 2021-02-10T11:15:14.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.795 7f15a585c700 1 -- 172.21.2.101:0/4114670389 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f159c00c7a0 con 0x7f15a80d1bf0 2021-02-10T11:15:14.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.795 7f15aca6f700 1 -- 172.21.2.101:0/4114670389 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15a80d1bf0 msgr2=0x7f15a81b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:14.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.795 7f15aca6f700 1 --2- 172.21.2.101:0/4114670389 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15a80d1bf0 0x7f15a81b3800 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f159c006700 tx=0x7f159c009b80).stop 2021-02-10T11:15:14.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.796 7f15aca6f700 1 -- 172.21.2.101:0/4114670389 shutdown_connections 2021-02-10T11:15:14.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.796 7f15aca6f700 1 --2- 172.21.2.101:0/4114670389 >> v2:172.21.2.108:3300/0 conn(0x7f15a81b3e20 0x7f15a80ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:14.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.796 7f15aca6f700 1 --2- 172.21.2.101:0/4114670389 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15a80d1bf0 0x7f15a81b3800 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:14.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.796 7f15aca6f700 1 --2- 172.21.2.101:0/4114670389 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15a80d17f0 0x7f15a81b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:14.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.796 7f15aca6f700 1 -- 172.21.2.101:0/4114670389 >> 172.21.2.101:0/4114670389 conn(0x7f15a80fde40 msgr2=0x7f15a80ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:14.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.796 7f15aca6f700 1 -- 172.21.2.101:0/4114670389 shutdown_connections 2021-02-10T11:15:14.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.796 7f15aca6f700 1 -- 172.21.2.101:0/4114670389 wait complete. 2021-02-10T11:15:14.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.811 7f15aca6f700 1 Processor -- start 2021-02-10T11:15:14.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.811 7f15aca6f700 1 -- start start 2021-02-10T11:15:14.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.812 7f15aca6f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15a81b2c30 0x7f15a81a65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:14.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.812 7f15aca6f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15a81a7110 0x7f15a81b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:14.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.812 7f15aca6f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15a81b2130 0x7f15a819fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:14.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.812 7f15aca6f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a81a0ff0 con 0x7f15a81b2c30 2021-02-10T11:15:14.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.812 7f15aca6f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a81b26e0 con 0x7f15a81a7110 2021-02-10T11:15:14.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.812 7f15aca6f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a81a6be0 con 0x7f15a81b2130 2021-02-10T11:15:14.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.812 7f15a685e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15a81b2c30 0x7f15a81a65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:14.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.812 7f15a685e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15a81b2c30 0x7f15a81a65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45472/0 (socket says 172.21.2.101:45472) 2021-02-10T11:15:14.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.812 7f15a685e700 1 -- 172.21.2.101:0/3029850252 learned_addr learned my addr 172.21.2.101:0/3029850252 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:14.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.812 7f15a685e700 1 -- 172.21.2.101:0/3029850252 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15a81b2130 msgr2=0x7f15a819fed0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:14.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.812 7f15a685e700 1 --2- 172.21.2.101:0/3029850252 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15a81b2130 0x7f15a819fed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:14.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.812 7f15a685e700 1 -- 172.21.2.101:0/3029850252 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15a81a7110 msgr2=0x7f15a81b1b10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:14.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.812 7f15a685e700 1 --2- 172.21.2.101:0/3029850252 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15a81a7110 0x7f15a81b1b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:14.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.812 7f15a685e700 1 -- 172.21.2.101:0/3029850252 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a81a04f0 con 0x7f15a81b2c30 2021-02-10T11:15:14.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.812 7f15a685e700 1 --2- 172.21.2.101:0/3029850252 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15a81b2c30 0x7f15a81a65c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f1598006f00 tx=0x7f159800b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:14.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.812 7f15977fe700 1 -- 172.21.2.101:0/3029850252 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1598006b90 con 0x7f15a81b2c30 2021-02-10T11:15:14.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.812 7f15977fe700 1 -- 172.21.2.101:0/3029850252 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159800bd30 con 0x7f15a81b2c30 2021-02-10T11:15:14.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.812 7f15977fe700 1 -- 172.21.2.101:0/3029850252 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f15980020d0 con 0x7f15a81b2c30 2021-02-10T11:15:14.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.813 7f15aca6f700 1 -- 172.21.2.101:0/3029850252 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a81a04f0 con 0x7f15a81b2c30 2021-02-10T11:15:14.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.813 7f15aca6f700 1 -- 172.21.2.101:0/3029850252 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a81a0a80 con 0x7f15a81b2c30 2021-02-10T11:15:14.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.813 7f15977fe700 1 -- 172.21.2.101:0/3029850252 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1598007020 con 0x7f15a81b2c30 2021-02-10T11:15:14.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.813 7f15977fe700 1 --2- 172.21.2.101:0/3029850252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f158c0186c0 0x7f158c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:14.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.813 7f15977fe700 1 -- 172.21.2.101:0/3029850252 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f15980323c0 con 0x7f15a81b2c30 2021-02-10T11:15:14.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.813 7f15a605d700 1 --2- 172.21.2.101:0/3029850252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f158c0186c0 0x7f158c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:14.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.813 7f15a605d700 1 --2- 172.21.2.101:0/3029850252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f158c0186c0 0x7f158c01abc0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:14.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.844 7f15aca6f700 1 -- 172.21.2.101:0/3029850252 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a81036d0 con 0x7f15a81b2c30 2021-02-10T11:15:14.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.847 7f15977fe700 1 -- 172.21.2.101:0/3029850252 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1598007ef0 con 0x7f15a81b2c30 2021-02-10T11:15:14.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.970 7f15aca6f700 1 -- 172.21.2.101:0/3029850252 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15a80f0b20 con 0x7f15a81b2c30 2021-02-10T11:15:14.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.971 7f15977fe700 1 -- 172.21.2.101:0/3029850252 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+13993 (secure 0 0 0) 0x7f1598000940 con 0x7f15a81b2c30 2021-02-10T11:15:14.976 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:15:14.976 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":29,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:14:58.132054","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:14:54.652789","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2021-02-10 11:14:56.125641","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:15:14.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.973 7f15aca6f700 1 -- 172.21.2.101:0/3029850252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f158c0186c0 msgr2=0x7f158c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:14.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.973 7f15aca6f700 1 --2- 172.21.2.101:0/3029850252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f158c0186c0 0x7f158c01abc0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:14.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.973 7f15aca6f700 1 -- 172.21.2.101:0/3029850252 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15a81b2c30 msgr2=0x7f15a81a65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:14.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.973 7f15aca6f700 1 --2- 172.21.2.101:0/3029850252 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15a81b2c30 0x7f15a81a65c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f1598006f00 tx=0x7f159800b940).stop 2021-02-10T11:15:14.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.973 7f15aca6f700 1 -- 172.21.2.101:0/3029850252 shutdown_connections 2021-02-10T11:15:14.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.973 7f15aca6f700 1 --2- 172.21.2.101:0/3029850252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f158c0186c0 0x7f158c01abc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:14.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.973 7f15aca6f700 1 --2- 172.21.2.101:0/3029850252 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15a81b2130 0x7f15a819fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:14.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.973 7f15aca6f700 1 --2- 172.21.2.101:0/3029850252 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15a81a7110 0x7f15a81b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:14.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.973 7f15aca6f700 1 --2- 172.21.2.101:0/3029850252 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15a81b2c30 0x7f15a81a65c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:14.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.973 7f15aca6f700 1 -- 172.21.2.101:0/3029850252 >> 172.21.2.101:0/3029850252 conn(0x7f15a81a37b0 msgr2=0x7f15a8103900 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:14.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.973 7f15aca6f700 1 -- 172.21.2.101:0/3029850252 shutdown_connections 2021-02-10T11:15:14.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:14.973 7f15aca6f700 1 -- 172.21.2.101:0/3029850252 wait complete. 2021-02-10T11:15:14.986 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:14:54.652789', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:14:56.125641', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:15:14.987 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:15:15.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.433 7f77bddc3700 1 Processor -- start 2021-02-10T11:15:15.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.433 7f77bddc3700 1 -- start start 2021-02-10T11:15:15.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.434 7f77bddc3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77b81b0470 0x7f77b81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:15.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.434 7f77bddc3700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b80f0b50 con 0x7f77b80f20e0 2021-02-10T11:15:15.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.434 7f77bddc3700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b80f24e0 con 0x7f77b80ef610 2021-02-10T11:15:15.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.434 7f77bddc3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b80e5ea0 con 0x7f77b81b0470 2021-02-10T11:15:15.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.434 7f77b7fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f77b80f20e0 0x7f77b80f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:45974/0 (socket says 172.21.2.101:45974) 2021-02-10T11:15:15.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.434 7f77b7fff700 1 -- 172.21.2.101:0/1676415534 learned_addr learned my addr 172.21.2.101:0/1676415534 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:15:15.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.434 7f77b67fc700 1 -- 172.21.2.101:0/1676415534 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2014304321 0 0) 0x7f77b0001500 con 0x7f77b80f20e0 2021-02-10T11:15:15.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.434 7f77b67fc700 1 -- 172.21.2.101:0/1676415534 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77a4002fe0 con 0x7f77b80f20e0 2021-02-10T11:15:15.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.434 7f77b77fe700 1 --2- 172.21.2.101:0/1676415534 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77b81b0470 0x7f77b81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:15.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.434 7f77b67fc700 1 -- 172.21.2.101:0/1676415534 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2414341521 0 0) 0x7f77b0001460 con 0x7f77b80f20e0 2021-02-10T11:15:15.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.434 7f77b67fc700 1 -- 172.21.2.101:0/1676415534 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77b81b0470 msgr2=0x7f77b81b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:15.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.434 7f77b67fc700 1 --2- 172.21.2.101:0/1676415534 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77b81b0470 0x7f77b81b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:15.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.434 7f77b67fc700 1 -- 172.21.2.101:0/1676415534 >> v1:172.21.2.108:6789/0 conn(0x7f77b80ef610 legacy=0x7f77b80efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:15.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.434 7f77b67fc700 1 -- 172.21.2.101:0/1676415534 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b80e52e0 con 0x7f77b80f20e0 2021-02-10T11:15:15.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.434 7f77b67fc700 1 -- 172.21.2.101:0/1676415534 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f77b0001980 con 0x7f77b80f20e0 2021-02-10T11:15:15.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.434 7f77b67fc700 1 -- 172.21.2.101:0/1676415534 >> v1:172.21.2.101:6789/0 conn(0x7f77b80f20e0 legacy=0x7f77b80f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:15.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.434 7f77b67fc700 1 --2- 172.21.2.101:0/1676415534 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77a4003b10 0x7f77a4005f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:15.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.434 7f77b67fc700 1 --2- 172.21.2.101:0/1676415534 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77a4006740 0x7f77a4008c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:15.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.434 7f77b67fc700 1 --2- 172.21.2.101:0/1676415534 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77a4009500 0x7f77a400ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:15.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.434 7f77b67fc700 1 -- 172.21.2.101:0/1676415534 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a400c2d0 con 0x7f77a4006740 2021-02-10T11:15:15.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.434 7f77b67fc700 1 -- 172.21.2.101:0/1676415534 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a400c4f0 con 0x7f77a4009500 2021-02-10T11:15:15.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.434 7f77b67fc700 1 -- 172.21.2.101:0/1676415534 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a400c710 con 0x7f77a4003b10 2021-02-10T11:15:15.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.435 7f77b6ffd700 1 --2- 172.21.2.101:0/1676415534 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77a4006740 0x7f77a4008c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:15.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.435 7f77b77fe700 1 --2- 172.21.2.101:0/1676415534 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77a4003b10 0x7f77a4005f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:15.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.435 7f77b7fff700 1 --2- 172.21.2.101:0/1676415534 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77a4009500 0x7f77a400ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:15.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.435 7f77b6ffd700 1 -- 172.21.2.101:0/1676415534 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77a4003b10 msgr2=0x7f77a4005f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:15.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.435 7f77b6ffd700 1 --2- 172.21.2.101:0/1676415534 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77a4003b10 0x7f77a4005f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:15.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.435 7f77b6ffd700 1 -- 172.21.2.101:0/1676415534 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77a4009500 msgr2=0x7f77a400ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:15.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.435 7f77b6ffd700 1 --2- 172.21.2.101:0/1676415534 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77a4009500 0x7f77a400ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:15.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.435 7f77b6ffd700 1 -- 172.21.2.101:0/1676415534 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a400c9b0 con 0x7f77a4006740 2021-02-10T11:15:15.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.435 7f77b6ffd700 1 --2- 172.21.2.101:0/1676415534 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77a4006740 0x7f77a4008c60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f77a804aac0 tx=0x7f77a804daf0).ready entity=mon.0 client_cookie=5e782ed4ceb5f012 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:15.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.435 7f77b67fc700 1 -- 172.21.2.101:0/1676415534 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f77a804f2e0 con 0x7f77a4006740 2021-02-10T11:15:15.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.435 7f77b67fc700 1 -- 172.21.2.101:0/1676415534 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a8000e20 con 0x7f77a4006740 2021-02-10T11:15:15.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.435 7f77b67fc700 1 -- 172.21.2.101:0/1676415534 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f77a804f5e0 con 0x7f77a4006740 2021-02-10T11:15:15.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.435 7f77bddc3700 1 -- 172.21.2.101:0/1676415534 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77a4006740 msgr2=0x7f77a4008c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:15.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.435 7f77bddc3700 1 --2- 172.21.2.101:0/1676415534 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77a4006740 0x7f77a4008c60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f77a804aac0 tx=0x7f77a804daf0).stop 2021-02-10T11:15:15.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.435 7f77b7fff700 1 -- 172.21.2.101:0/1676415534 reap_dead start 2021-02-10T11:15:15.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.436 7f77bddc3700 1 -- 172.21.2.101:0/1676415534 shutdown_connections 2021-02-10T11:15:15.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.436 7f77bddc3700 1 -- 172.21.2.101:0/1676415534 >> 172.21.2.101:0/1676415534 conn(0x7f77b80ce3e0 msgr2=0x7f77b80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:15.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.436 7f77bddc3700 1 -- 172.21.2.101:0/1676415534 shutdown_connections 2021-02-10T11:15:15.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.436 7f77bddc3700 1 -- 172.21.2.101:0/1676415534 wait complete. 2021-02-10T11:15:15.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.451 7f77bddc3700 1 Processor -- start 2021-02-10T11:15:15.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.451 7f77bddc3700 1 -- start start 2021-02-10T11:15:15.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.451 7f77bddc3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77b809b930 0x7f77b80b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:15.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.451 7f77bddc3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77b80b8410 0x7f77b80b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:15.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.451 7f77bddc3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77b80c1f80 0x7f77b80c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:15.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.451 7f77bddc3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b80c1a00 con 0x7f77b80b8410 2021-02-10T11:15:15.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.451 7f77bddc3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b80b8e30 con 0x7f77b80c1f80 2021-02-10T11:15:15.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.451 7f77bddc3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b80b9050 con 0x7f77b809b930 2021-02-10T11:15:15.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.451 7f77b6ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77b80b8410 0x7f77b80b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:15.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.451 7f77b6ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77b80b8410 0x7f77b80b8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45492/0 (socket says 172.21.2.101:45492) 2021-02-10T11:15:15.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.451 7f77b6ffd700 1 -- 172.21.2.101:0/2029851705 learned_addr learned my addr 172.21.2.101:0/2029851705 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:15.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.452 7f77b6ffd700 1 -- 172.21.2.101:0/2029851705 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77b809b930 msgr2=0x7f77b80b7c20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:15.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.452 7f77b6ffd700 1 --2- 172.21.2.101:0/2029851705 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77b809b930 0x7f77b80b7c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:15.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.452 7f77b6ffd700 1 -- 172.21.2.101:0/2029851705 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77b80c1f80 msgr2=0x7f77b80c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:15.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.452 7f77b6ffd700 1 --2- 172.21.2.101:0/2029851705 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77b80c1f80 0x7f77b80c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:15.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.452 7f77b6ffd700 1 -- 172.21.2.101:0/2029851705 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b80c29a0 con 0x7f77b80b8410 2021-02-10T11:15:15.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.452 7f77b6ffd700 1 --2- 172.21.2.101:0/2029851705 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77b80b8410 0x7f77b80b8810 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f77a804a5f0 tx=0x7f77a804f960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:15.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.452 7f77affff700 1 -- 172.21.2.101:0/2029851705 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f77a804d3e0 con 0x7f77b80b8410 2021-02-10T11:15:15.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.452 7f77affff700 1 -- 172.21.2.101:0/2029851705 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a80022e0 con 0x7f77b80b8410 2021-02-10T11:15:15.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.452 7f77bddc3700 1 -- 172.21.2.101:0/2029851705 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b80c29a0 con 0x7f77b80b8410 2021-02-10T11:15:15.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.452 7f77bddc3700 1 -- 172.21.2.101:0/2029851705 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b80c2f30 con 0x7f77b80b8410 2021-02-10T11:15:15.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.452 7f77affff700 1 -- 172.21.2.101:0/2029851705 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f77a80026f0 con 0x7f77b80b8410 2021-02-10T11:15:15.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.453 7f77affff700 1 -- 172.21.2.101:0/2029851705 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f77a80024c0 con 0x7f77b80b8410 2021-02-10T11:15:15.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.453 7f77affff700 1 --2- 172.21.2.101:0/2029851705 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f779c018700 0x7f779c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:15.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.453 7f77affff700 1 -- 172.21.2.101:0/2029851705 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f77a80767c0 con 0x7f77b80b8410 2021-02-10T11:15:15.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.453 7f77b77fe700 1 --2- 172.21.2.101:0/2029851705 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f779c018700 0x7f779c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:15.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.453 7f77b77fe700 1 --2- 172.21.2.101:0/2029851705 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f779c018700 0x7f779c01abc0 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:15.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.483 7f77bddc3700 1 -- 172.21.2.101:0/2029851705 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b8105b30 con 0x7f77b80b8410 2021-02-10T11:15:15.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.486 7f77affff700 1 -- 172.21.2.101:0/2029851705 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f77a804d3e0 con 0x7f77b80b8410 2021-02-10T11:15:15.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.615 7f77bddc3700 1 -- 172.21.2.101:0/2029851705 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f77b80f0b50 con 0x7f77b80b8410 2021-02-10T11:15:15.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.615 7f77affff700 1 -- 172.21.2.101:0/2029851705 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f77a804fd50 con 0x7f77b80b8410 2021-02-10T11:15:15.620 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:15:15.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.617 7f77bddc3700 1 -- 172.21.2.101:0/2029851705 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f779c018700 msgr2=0x7f779c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:15.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.617 7f77bddc3700 1 --2- 172.21.2.101:0/2029851705 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f779c018700 0x7f779c01abc0 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:15.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.617 7f77bddc3700 1 -- 172.21.2.101:0/2029851705 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77b80b8410 msgr2=0x7f77b80b8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:15.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.617 7f77bddc3700 1 --2- 172.21.2.101:0/2029851705 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77b80b8410 0x7f77b80b8810 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f77a804a5f0 tx=0x7f77a804f960).stop 2021-02-10T11:15:15.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.617 7f77bddc3700 1 -- 172.21.2.101:0/2029851705 shutdown_connections 2021-02-10T11:15:15.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.617 7f77bddc3700 1 --2- 172.21.2.101:0/2029851705 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77b80c1f80 0x7f77b80c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:15.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.617 7f77bddc3700 1 --2- 172.21.2.101:0/2029851705 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77b80b8410 0x7f77b80b8810 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:15.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.617 7f77bddc3700 1 --2- 172.21.2.101:0/2029851705 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f779c018700 0x7f779c01abc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:15.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.617 7f77bddc3700 1 --2- 172.21.2.101:0/2029851705 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77b809b930 0x7f77b80b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:15.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.617 7f77bddc3700 1 -- 172.21.2.101:0/2029851705 >> 172.21.2.101:0/2029851705 conn(0x7f77b8094e10 msgr2=0x7f77b80a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:15.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.617 7f77bddc3700 1 -- 172.21.2.101:0/2029851705 shutdown_connections 2021-02-10T11:15:15.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:15.617 7f77bddc3700 1 -- 172.21.2.101:0/2029851705 wait complete. 2021-02-10T11:15:15.630 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:15:16.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.040 7fc9fd05e700 1 Processor -- start 2021-02-10T11:15:16.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.040 7fc9fd05e700 1 -- start start 2021-02-10T11:15:16.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.040 7fc9fd05e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc9f81b0470 0x7fc9f81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:16.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.040 7fc9fd05e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9f80ef610 0x7fc9f80efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:16.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.040 7fc9fd05e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc9f80f0030 0x7fc9f80f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:16.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.040 7fc9fd05e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f80e5ea0 con 0x7fc9f81b0470 2021-02-10T11:15:16.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.040 7fc9fd05e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f80f0b50 con 0x7fc9f80f0030 2021-02-10T11:15:16.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.040 7fc9fd05e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80e52e0 con 0x7fc9f80ef610 2021-02-10T11:15:16.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.041 7fc9f6d9d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc9f81b0470 0x7fc9f81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:16.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.041 7fc9f6d9d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc9f81b0470 0x7fc9f81b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45496/0 (socket says 172.21.2.101:45496) 2021-02-10T11:15:16.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.041 7fc9f6d9d700 1 -- 172.21.2.101:0/2799985620 learned_addr learned my addr 172.21.2.101:0/2799985620 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:16.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.041 7fc9f6d9d700 1 -- 172.21.2.101:0/2799985620 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9f80ef610 msgr2=0x7fc9f80efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:16.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.041 7fc9f6d9d700 1 --2- 172.21.2.101:0/2799985620 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9f80ef610 0x7fc9f80efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:16.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.041 7fc9f6d9d700 1 -- 172.21.2.101:0/2799985620 >> v2:172.21.2.108:3300/0 conn(0x7fc9f80f0030 msgr2=0x7fc9f80f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:16.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.041 7fc9f6d9d700 1 --2- 172.21.2.101:0/2799985620 >> v2:172.21.2.108:3300/0 conn(0x7fc9f80f0030 0x7fc9f80f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:16.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.041 7fc9f6d9d700 1 -- 172.21.2.101:0/2799985620 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f80f10d0 con 0x7fc9f81b0470 2021-02-10T11:15:16.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.041 7fc9f6d9d700 1 --2- 172.21.2.101:0/2799985620 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9f81b0470 0x7fc9f81b0870 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7fc9e0006700 tx=0x7fc9e0009b80).ready entity=mon.0 client_cookie=83c5f5b263f56e29 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:16.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.041 7fc9f5d9b700 1 -- 172.21.2.101:0/2799985620 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc9e000c0c0 con 0x7fc9f81b0470 2021-02-10T11:15:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.041 7fc9f5d9b700 1 -- 172.21.2.101:0/2799985620 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e000c340 con 0x7fc9f81b0470 2021-02-10T11:15:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.041 7fc9f5d9b700 1 -- 172.21.2.101:0/2799985620 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc9e000c860 con 0x7fc9f81b0470 2021-02-10T11:15:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.041 7fc9fd05e700 1 -- 172.21.2.101:0/2799985620 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9f81b0470 msgr2=0x7fc9f81b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.041 7fc9fd05e700 1 --2- 172.21.2.101:0/2799985620 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9f81b0470 0x7fc9f81b0870 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7fc9e0006700 tx=0x7fc9e0009b80).stop 2021-02-10T11:15:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.042 7fc9fd05e700 1 -- 172.21.2.101:0/2799985620 shutdown_connections 2021-02-10T11:15:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.042 7fc9fd05e700 1 --2- 172.21.2.101:0/2799985620 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9f80ef610 0x7fc9f80efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.042 7fc9fd05e700 1 --2- 172.21.2.101:0/2799985620 >> v2:172.21.2.108:3300/0 conn(0x7fc9f80f0030 0x7fc9f80f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.042 7fc9fd05e700 1 --2- 172.21.2.101:0/2799985620 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9f81b0470 0x7fc9f81b0870 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.042 7fc9fd05e700 1 -- 172.21.2.101:0/2799985620 >> 172.21.2.101:0/2799985620 conn(0x7fc9f80ce3e0 msgr2=0x7fc9f80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.042 7fc9fd05e700 1 -- 172.21.2.101:0/2799985620 shutdown_connections 2021-02-10T11:15:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.042 7fc9fd05e700 1 -- 172.21.2.101:0/2799985620 wait complete. 2021-02-10T11:15:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.056 7fc9fd05e700 1 Processor -- start 2021-02-10T11:15:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.056 7fc9fd05e700 1 -- start start 2021-02-10T11:15:16.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.057 7fc9fd05e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9f80b20b0 0x7fc9f80ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:16.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.057 7fc9fd05e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9f80b24b0 0x7fc9f8191dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:16.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.057 7fc9fd05e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9f80acf90 0x7fc9f81923e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:16.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.057 7fc9fd05e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80b1b30 con 0x7fc9f80b20b0 2021-02-10T11:15:16.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.057 7fc9fd05e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f81a04e0 con 0x7fc9f80acf90 2021-02-10T11:15:16.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.057 7fc9fd05e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f81a06c0 con 0x7fc9f80b24b0 2021-02-10T11:15:16.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.057 7fc9f6d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9f80b20b0 0x7fc9f80ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:16.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.057 7fc9f6d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9f80b20b0 0x7fc9f80ac970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45504/0 (socket says 172.21.2.101:45504) 2021-02-10T11:15:16.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.057 7fc9f6d9d700 1 -- 172.21.2.101:0/220027982 learned_addr learned my addr 172.21.2.101:0/220027982 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:16.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.057 7fc9f6d9d700 1 -- 172.21.2.101:0/220027982 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9f80b24b0 msgr2=0x7fc9f8191dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:16.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.057 7fc9f6d9d700 1 --2- 172.21.2.101:0/220027982 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9f80b24b0 0x7fc9f8191dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:16.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.057 7fc9f6d9d700 1 -- 172.21.2.101:0/220027982 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9f80acf90 msgr2=0x7fc9f81923e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:16.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.057 7fc9f6d9d700 1 --2- 172.21.2.101:0/220027982 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9f80acf90 0x7fc9f81923e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:16.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.057 7fc9f6d9d700 1 -- 172.21.2.101:0/220027982 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f8192be0 con 0x7fc9f80b20b0 2021-02-10T11:15:16.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.057 7fc9f6d9d700 1 --2- 172.21.2.101:0/220027982 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9f80b20b0 0x7fc9f80ac970 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fc9e000b5e0 tx=0x7fc9e000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:16.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.058 7fc9ef7fe700 1 -- 172.21.2.101:0/220027982 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc9e000bd70 con 0x7fc9f80b20b0 2021-02-10T11:15:16.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.058 7fc9ef7fe700 1 -- 172.21.2.101:0/220027982 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e0002030 con 0x7fc9f80b20b0 2021-02-10T11:15:16.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.058 7fc9ef7fe700 1 -- 172.21.2.101:0/220027982 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc9e0002550 con 0x7fc9f80b20b0 2021-02-10T11:15:16.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.058 7fc9fd05e700 1 -- 172.21.2.101:0/220027982 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f8192be0 con 0x7fc9f80b20b0 2021-02-10T11:15:16.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.058 7fc9fd05e700 1 -- 172.21.2.101:0/220027982 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f8193170 con 0x7fc9f80b20b0 2021-02-10T11:15:16.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.059 7fc9ef7fe700 1 -- 172.21.2.101:0/220027982 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc9e0000c90 con 0x7fc9f80b20b0 2021-02-10T11:15:16.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.059 7fc9ef7fe700 1 --2- 172.21.2.101:0/220027982 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9dc0186c0 0x7fc9dc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:16.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.059 7fc9ef7fe700 1 -- 172.21.2.101:0/220027982 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc9e0032570 con 0x7fc9f80b20b0 2021-02-10T11:15:16.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.059 7fc9f659c700 1 --2- 172.21.2.101:0/220027982 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9dc0186c0 0x7fc9dc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:16.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.059 7fc9f659c700 1 --2- 172.21.2.101:0/220027982 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9dc0186c0 0x7fc9dc01abc0 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:16.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.090 7fc9fd05e700 1 -- 172.21.2.101:0/220027982 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f8105af0 con 0x7fc9f80b20b0 2021-02-10T11:15:16.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.093 7fc9ef7fe700 1 -- 172.21.2.101:0/220027982 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc9e0000940 con 0x7fc9f80b20b0 2021-02-10T11:15:16.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.222 7fc9fd05e700 1 -- 172.21.2.101:0/220027982 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7fc9f80f0b50 con 0x7fc9f80b20b0 2021-02-10T11:15:16.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.223 7fc9ef7fe700 1 -- 172.21.2.101:0/220027982 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v29) v1 ==== 104+0+10 (secure 0 0 0) 0x7fc9e0000c90 con 0x7fc9f80b20b0 2021-02-10T11:15:16.228 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:15:16.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.225 7fc9fd05e700 1 -- 172.21.2.101:0/220027982 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9dc0186c0 msgr2=0x7fc9dc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:16.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.225 7fc9fd05e700 1 --2- 172.21.2.101:0/220027982 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9dc0186c0 0x7fc9dc01abc0 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:16.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.225 7fc9fd05e700 1 -- 172.21.2.101:0/220027982 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9f80b20b0 msgr2=0x7fc9f80ac970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:16.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.225 7fc9fd05e700 1 --2- 172.21.2.101:0/220027982 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9f80b20b0 0x7fc9f80ac970 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fc9e000b5e0 tx=0x7fc9e000b980).stop 2021-02-10T11:15:16.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.225 7fc9fd05e700 1 -- 172.21.2.101:0/220027982 shutdown_connections 2021-02-10T11:15:16.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.225 7fc9fd05e700 1 --2- 172.21.2.101:0/220027982 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9dc0186c0 0x7fc9dc01abc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:16.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.225 7fc9fd05e700 1 --2- 172.21.2.101:0/220027982 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9f80b24b0 0x7fc9f8191dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:16.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.225 7fc9fd05e700 1 --2- 172.21.2.101:0/220027982 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9f80acf90 0x7fc9f81923e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:16.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.225 7fc9fd05e700 1 --2- 172.21.2.101:0/220027982 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9f80b20b0 0x7fc9f80ac970 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:16.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.225 7fc9fd05e700 1 -- 172.21.2.101:0/220027982 >> 172.21.2.101:0/220027982 conn(0x7fc9f80bf4e0 msgr2=0x7fc9f80be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:16.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.225 7fc9fd05e700 1 -- 172.21.2.101:0/220027982 shutdown_connections 2021-02-10T11:15:16.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.225 7fc9fd05e700 1 -- 172.21.2.101:0/220027982 wait complete. 2021-02-10T11:15:16.238 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:15:16.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.647 7f7663d43700 1 Processor -- start 2021-02-10T11:15:16.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.647 7f7663d43700 1 -- start start 2021-02-10T11:15:16.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.647 7f7663d43700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f765c1b0470 0x7f765c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:16.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.647 7f7663d43700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f765c0ef610 0x7f765c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:16.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.647 7f7663d43700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f765c0f0b50 con 0x7f765c0f0030 2021-02-10T11:15:16.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.647 7f7663d43700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f765c0e52e0 con 0x7f765c1b0470 2021-02-10T11:15:16.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.647 7f7663d43700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765c0e54d0 con 0x7f765c0ef610 2021-02-10T11:15:16.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.648 7f766135e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f765c0ef610 0x7f765c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:16.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.648 7f766135e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f765c0ef610 0x7f765c0efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:38242/0 (socket says 172.21.2.101:38242) 2021-02-10T11:15:16.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.648 7f766135e700 1 -- 172.21.2.101:0/642200641 learned_addr learned my addr 172.21.2.101:0/642200641 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:16.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.648 7f7661b5f700 1 --2- 172.21.2.101:0/642200641 >> v2:172.21.2.108:3300/0 conn(0x7f765c1b0470 0x7f765c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:16.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.649 7f7661b5f700 1 -- 172.21.2.101:0/642200641 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f765c0ef610 msgr2=0x7f765c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:16.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.649 7f7661b5f700 1 --2- 172.21.2.101:0/642200641 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f765c0ef610 0x7f765c0efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:16.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.649 7f7661b5f700 1 -- 172.21.2.101:0/642200641 >> v1:172.21.2.108:6789/0 conn(0x7f765c0f0030 legacy=0x7f765c0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:16.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.649 7f7661b5f700 1 -- 172.21.2.101:0/642200641 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765c0f0430 con 0x7f765c1b0470 2021-02-10T11:15:16.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.649 7f766135e700 1 --2- 172.21.2.101:0/642200641 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f765c0ef610 0x7f765c0efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:15:16.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.649 7f7661b5f700 1 --2- 172.21.2.101:0/642200641 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f765c1b0470 0x7f765c1b0870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f7650006650 tx=0x7f7650009a80).ready entity=mon.1 client_cookie=8e5715dab55d7e35 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:16.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.650 7f7660b5d700 1 -- 172.21.2.101:0/642200641 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f76500009b0 con 0x7f765c1b0470 2021-02-10T11:15:16.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.650 7f7660b5d700 1 -- 172.21.2.101:0/642200641 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765000c0c0 con 0x7f765c1b0470 2021-02-10T11:15:16.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.650 7f7660b5d700 1 -- 172.21.2.101:0/642200641 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f765000c5e0 con 0x7f765c1b0470 2021-02-10T11:15:16.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.650 7f7663d43700 1 -- 172.21.2.101:0/642200641 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f765c1b0470 msgr2=0x7f765c1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:16.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.650 7f7663d43700 1 --2- 172.21.2.101:0/642200641 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f765c1b0470 0x7f765c1b0870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f7650006650 tx=0x7f7650009a80).stop 2021-02-10T11:15:16.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.651 7f7663d43700 1 -- 172.21.2.101:0/642200641 shutdown_connections 2021-02-10T11:15:16.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.651 7f7663d43700 1 --2- 172.21.2.101:0/642200641 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f765c0ef610 0x7f765c0efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:16.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.651 7f7663d43700 1 --2- 172.21.2.101:0/642200641 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f765c1b0470 0x7f765c1b0870 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:16.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.651 7f7663d43700 1 -- 172.21.2.101:0/642200641 >> 172.21.2.101:0/642200641 conn(0x7f765c0ce3e0 msgr2=0x7f765c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:16.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.651 7f7663d43700 1 -- 172.21.2.101:0/642200641 shutdown_connections 2021-02-10T11:15:16.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.651 7f7663d43700 1 -- 172.21.2.101:0/642200641 wait complete. 2021-02-10T11:15:16.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.666 7f7663d43700 1 Processor -- start 2021-02-10T11:15:16.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.666 7f7663d43700 1 -- start start 2021-02-10T11:15:16.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.666 7f7663d43700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f765c0a0420 0x7f765c0a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:16.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.666 7f7663d43700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f765c189d00 0x7f765c18a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:16.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.666 7f7663d43700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f765c18a720 0x7f765c0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:16.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.666 7f7663d43700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c189780 con 0x7f765c18a720 2021-02-10T11:15:16.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.666 7f7663d43700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c18ab20 con 0x7f765c0a0420 2021-02-10T11:15:16.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.666 7f7663d43700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765c0a1140 con 0x7f765c189d00 2021-02-10T11:15:16.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.667 7f7662360700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f765c18a720 0x7f765c0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:16.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.667 7f7662360700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f765c18a720 0x7f765c0b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45516/0 (socket says 172.21.2.101:45516) 2021-02-10T11:15:16.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.667 7f766135e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f765c189d00 0x7f765c18a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:16.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.667 7f7662360700 1 -- 172.21.2.101:0/3945200238 learned_addr learned my addr 172.21.2.101:0/3945200238 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:16.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.667 7f7661b5f700 1 --2- 172.21.2.101:0/3945200238 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f765c0a0420 0x7f765c0a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:16.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.667 7f7662360700 1 -- 172.21.2.101:0/3945200238 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f765c189d00 msgr2=0x7f765c18a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:16.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.667 7f7662360700 1 --2- 172.21.2.101:0/3945200238 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f765c189d00 0x7f765c18a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:16.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.667 7f7662360700 1 -- 172.21.2.101:0/3945200238 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f765c0a0420 msgr2=0x7f765c0a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:16.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.667 7f7662360700 1 --2- 172.21.2.101:0/3945200238 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f765c0a0420 0x7f765c0a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:16.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.667 7f7662360700 1 -- 172.21.2.101:0/3945200238 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765c0b8200 con 0x7f765c18a720 2021-02-10T11:15:16.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.667 7f7661b5f700 1 --2- 172.21.2.101:0/3945200238 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f765c0a0420 0x7f765c0a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:15:16.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.667 7f7662360700 1 --2- 172.21.2.101:0/3945200238 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f765c18a720 0x7f765c0b7be0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f7654009940 tx=0x7f765400bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:16.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.667 7f764e7fc700 1 -- 172.21.2.101:0/3945200238 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f765400c2f0 con 0x7f765c18a720 2021-02-10T11:15:16.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.667 7f764e7fc700 1 -- 172.21.2.101:0/3945200238 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765400c570 con 0x7f765c18a720 2021-02-10T11:15:16.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.667 7f764e7fc700 1 -- 172.21.2.101:0/3945200238 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f765400c9d0 con 0x7f765c18a720 2021-02-10T11:15:16.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.667 7f7663d43700 1 -- 172.21.2.101:0/3945200238 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765c0b8200 con 0x7f765c18a720 2021-02-10T11:15:16.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.667 7f7663d43700 1 -- 172.21.2.101:0/3945200238 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f765c0b8200 con 0x7f765c18a720 2021-02-10T11:15:16.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.668 7f764e7fc700 1 -- 172.21.2.101:0/3945200238 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7654002030 con 0x7f765c18a720 2021-02-10T11:15:16.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.668 7f764e7fc700 1 --2- 172.21.2.101:0/3945200238 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76440186c0 0x7f764401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:16.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.668 7f764e7fc700 1 -- 172.21.2.101:0/3945200238 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7654032850 con 0x7f765c18a720 2021-02-10T11:15:16.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.668 7f7661b5f700 1 --2- 172.21.2.101:0/3945200238 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76440186c0 0x7f764401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:16.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.668 7f7661b5f700 1 --2- 172.21.2.101:0/3945200238 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76440186c0 0x7f764401abc0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:16.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.697 7f7663d43700 1 -- 172.21.2.101:0/3945200238 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f765c105ae0 con 0x7f765c18a720 2021-02-10T11:15:16.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.700 7f764e7fc700 1 -- 172.21.2.101:0/3945200238 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f765400be30 con 0x7f765c18a720 2021-02-10T11:15:16.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.828 7f7663d43700 1 -- 172.21.2.101:0/3945200238 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f765c105ae0 con 0x7f765c18a720 2021-02-10T11:15:16.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.828 7f764e7fc700 1 -- 172.21.2.101:0/3945200238 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v29) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7654002f00 con 0x7f765c18a720 2021-02-10T11:15:16.834 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:15:16.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.830 7f7663d43700 1 -- 172.21.2.101:0/3945200238 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76440186c0 msgr2=0x7f764401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:16.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.830 7f7663d43700 1 --2- 172.21.2.101:0/3945200238 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76440186c0 0x7f764401abc0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:16.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.830 7f7663d43700 1 -- 172.21.2.101:0/3945200238 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f765c18a720 msgr2=0x7f765c0b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:16.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.830 7f7663d43700 1 --2- 172.21.2.101:0/3945200238 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f765c18a720 0x7f765c0b7be0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f7654009940 tx=0x7f765400bcf0).stop 2021-02-10T11:15:16.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.830 7f7663d43700 1 -- 172.21.2.101:0/3945200238 shutdown_connections 2021-02-10T11:15:16.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.831 7f7663d43700 1 --2- 172.21.2.101:0/3945200238 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76440186c0 0x7f764401abc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:16.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.831 7f7663d43700 1 --2- 172.21.2.101:0/3945200238 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f765c189d00 0x7f765c18a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:16.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.831 7f7663d43700 1 --2- 172.21.2.101:0/3945200238 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f765c18a720 0x7f765c0b7be0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:16.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.831 7f7663d43700 1 --2- 172.21.2.101:0/3945200238 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f765c0a0420 0x7f765c0a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:16.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.831 7f7663d43700 1 -- 172.21.2.101:0/3945200238 >> 172.21.2.101:0/3945200238 conn(0x7f765c094e10 msgr2=0x7f765c0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:16.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.831 7f7663d43700 1 -- 172.21.2.101:0/3945200238 shutdown_connections 2021-02-10T11:15:16.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:16.831 7f7663d43700 1 -- 172.21.2.101:0/3945200238 wait complete. 2021-02-10T11:15:16.843 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:15:16.844 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:15:17.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.250 7f8502cde700 1 Processor -- start 2021-02-10T11:15:17.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.250 7f8502cde700 1 -- start start 2021-02-10T11:15:17.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.251 7f8502cde700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f84fc0e5070 0x7f84fc1b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:17.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.251 7f8502cde700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f84fc1b3e40 0x7f84fc07d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:17.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.251 7f8502cde700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f84fc0e5470 0x7f84fc07dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:17.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.251 7f8502cde700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84fc07ed80 con 0x7f84fc1b3e40 2021-02-10T11:15:17.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.251 7f8502cde700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84fc07e2d0 con 0x7f84fc0e5070 2021-02-10T11:15:17.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.251 7f8502cde700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc07e4f0 con 0x7f84fc0e5470 2021-02-10T11:15:17.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.251 7f84fbfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f84fc1b3e40 0x7f84fc07d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:17.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.251 7f84fbfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f84fc1b3e40 0x7f84fc07d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45520/0 (socket says 172.21.2.101:45520) 2021-02-10T11:15:17.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.251 7f84fbfff700 1 -- 172.21.2.101:0/2614959884 learned_addr learned my addr 172.21.2.101:0/2614959884 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:17.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.251 7f85012fb700 1 --2- 172.21.2.101:0/2614959884 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f84fc0e5470 0x7f84fc07dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:17.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.251 7f8500afa700 1 --2- 172.21.2.101:0/2614959884 >> v2:172.21.2.108:3300/0 conn(0x7f84fc0e5070 0x7f84fc1b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:17.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.251 7f84fbfff700 1 -- 172.21.2.101:0/2614959884 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f84fc0e5470 msgr2=0x7f84fc07dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:17.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.251 7f84fbfff700 1 --2- 172.21.2.101:0/2614959884 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f84fc0e5470 0x7f84fc07dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:17.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.251 7f84fbfff700 1 -- 172.21.2.101:0/2614959884 >> v2:172.21.2.108:3300/0 conn(0x7f84fc0e5070 msgr2=0x7f84fc1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:17.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.251 7f84fbfff700 1 --2- 172.21.2.101:0/2614959884 >> v2:172.21.2.108:3300/0 conn(0x7f84fc0e5070 0x7f84fc1b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:17.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.251 7f84fbfff700 1 -- 172.21.2.101:0/2614959884 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84fc07e710 con 0x7f84fc1b3e40 2021-02-10T11:15:17.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.251 7f84fbfff700 1 --2- 172.21.2.101:0/2614959884 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84fc1b3e40 0x7f84fc07d690 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f84ec006700 tx=0x7f84ec009b80).ready entity=mon.0 client_cookie=1e1388943089d8ee server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:17.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.252 7f84fb7fe700 1 -- 172.21.2.101:0/2614959884 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f84ec000c90 con 0x7f84fc1b3e40 2021-02-10T11:15:17.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.252 7f84fb7fe700 1 -- 172.21.2.101:0/2614959884 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ec00c0c0 con 0x7f84fc1b3e40 2021-02-10T11:15:17.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.252 7f84fb7fe700 1 -- 172.21.2.101:0/2614959884 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f84ec00c5e0 con 0x7f84fc1b3e40 2021-02-10T11:15:17.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.252 7f8502cde700 1 -- 172.21.2.101:0/2614959884 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84fc1b3e40 msgr2=0x7f84fc07d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:17.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.252 7f8502cde700 1 --2- 172.21.2.101:0/2614959884 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84fc1b3e40 0x7f84fc07d690 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f84ec006700 tx=0x7f84ec009b80).stop 2021-02-10T11:15:17.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.252 7f8502cde700 1 -- 172.21.2.101:0/2614959884 shutdown_connections 2021-02-10T11:15:17.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.252 7f8502cde700 1 --2- 172.21.2.101:0/2614959884 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f84fc0e5470 0x7f84fc07dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:17.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.252 7f8502cde700 1 --2- 172.21.2.101:0/2614959884 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84fc1b3e40 0x7f84fc07d690 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:17.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.252 7f8502cde700 1 --2- 172.21.2.101:0/2614959884 >> v2:172.21.2.108:3300/0 conn(0x7f84fc0e5070 0x7f84fc1b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:17.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.252 7f8502cde700 1 -- 172.21.2.101:0/2614959884 >> 172.21.2.101:0/2614959884 conn(0x7f84fc0ce690 msgr2=0x7f84fc0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:17.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.252 7f8502cde700 1 -- 172.21.2.101:0/2614959884 shutdown_connections 2021-02-10T11:15:17.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.252 7f8502cde700 1 -- 172.21.2.101:0/2614959884 wait complete. 2021-02-10T11:15:17.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.267 7f8502cde700 1 Processor -- start 2021-02-10T11:15:17.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.267 7f8502cde700 1 -- start start 2021-02-10T11:15:17.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.267 7f8502cde700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84fc1918e0 0x7f84fc189c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:17.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.267 7f8502cde700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f84fc18a2a0 0x7f84fc18a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:17.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.267 7f8502cde700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f84fc18acc0 0x7f84fc190070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:17.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.267 7f8502cde700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc1a2f00 con 0x7f84fc1918e0 2021-02-10T11:15:17.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.267 7f8502cde700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc191ce0 con 0x7f84fc18a2a0 2021-02-10T11:15:17.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.267 7f8502cde700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc191f00 con 0x7f84fc18acc0 2021-02-10T11:15:17.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.268 7f8500afa700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84fc1918e0 0x7f84fc189c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:17.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.268 7f8500afa700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84fc1918e0 0x7f84fc189c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45522/0 (socket says 172.21.2.101:45522) 2021-02-10T11:15:17.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.268 7f8500afa700 1 -- 172.21.2.101:0/2503313259 learned_addr learned my addr 172.21.2.101:0/2503313259 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:17.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.268 7f85012fb700 1 --2- 172.21.2.101:0/2503313259 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f84fc18acc0 0x7f84fc190070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:17.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.268 7f84fbfff700 1 --2- 172.21.2.101:0/2503313259 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f84fc18a2a0 0x7f84fc18a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:17.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.268 7f8500afa700 1 -- 172.21.2.101:0/2503313259 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f84fc18acc0 msgr2=0x7f84fc190070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:17.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.268 7f8500afa700 1 --2- 172.21.2.101:0/2503313259 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f84fc18acc0 0x7f84fc190070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:17.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.268 7f8500afa700 1 -- 172.21.2.101:0/2503313259 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f84fc18a2a0 msgr2=0x7f84fc18a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:17.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.268 7f8500afa700 1 --2- 172.21.2.101:0/2503313259 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f84fc18a2a0 0x7f84fc18a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:17.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.268 7f8500afa700 1 -- 172.21.2.101:0/2503313259 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84fc190690 con 0x7f84fc1918e0 2021-02-10T11:15:17.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.268 7f8500afa700 1 --2- 172.21.2.101:0/2503313259 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84fc1918e0 0x7f84fc189c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f84e4006f60 tx=0x7f84e400b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:17.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.268 7f84f97fa700 1 -- 172.21.2.101:0/2503313259 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f84e40009b0 con 0x7f84fc1918e0 2021-02-10T11:15:17.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.268 7f84f97fa700 1 -- 172.21.2.101:0/2503313259 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e400bd60 con 0x7f84fc1918e0 2021-02-10T11:15:17.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.268 7f8502cde700 1 -- 172.21.2.101:0/2503313259 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84fc190690 con 0x7f84fc1918e0 2021-02-10T11:15:17.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.268 7f8502cde700 1 -- 172.21.2.101:0/2503313259 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84fc190c20 con 0x7f84fc1918e0 2021-02-10T11:15:17.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.268 7f84f97fa700 1 -- 172.21.2.101:0/2503313259 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f84e4002030 con 0x7f84fc1918e0 2021-02-10T11:15:17.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.269 7f84f97fa700 1 -- 172.21.2.101:0/2503313259 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f84e4000c90 con 0x7f84fc1918e0 2021-02-10T11:15:17.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.269 7f84f97fa700 1 --2- 172.21.2.101:0/2503313259 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84e00186c0 0x7f84e001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:17.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.269 7f84f97fa700 1 -- 172.21.2.101:0/2503313259 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f84e40327c0 con 0x7f84fc1918e0 2021-02-10T11:15:17.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.269 7f84fbfff700 1 --2- 172.21.2.101:0/2503313259 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84e00186c0 0x7f84e001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:17.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.270 7f84fbfff700 1 --2- 172.21.2.101:0/2503313259 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84e00186c0 0x7f84e001abc0 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:17.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.300 7f8502cde700 1 -- 172.21.2.101:0/2503313259 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84fc189750 con 0x7f84fc1918e0 2021-02-10T11:15:17.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.303 7f84f97fa700 1 -- 172.21.2.101:0/2503313259 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f84e40009b0 con 0x7f84fc1918e0 2021-02-10T11:15:17.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.432 7f8502cde700 1 -- 172.21.2.101:0/2503313259 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84fc189750 con 0x7f84fc1918e0 2021-02-10T11:15:17.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.432 7f84f97fa700 1 -- 172.21.2.101:0/2503313259 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+2100 (secure 0 0 0) 0x7f84e400bd60 con 0x7f84fc1918e0 2021-02-10T11:15:17.437 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:15:17.437 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/3226463341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3226463341},{"type":"v1","addr":"172.21.2.101:6835","nonce":3226463341}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:14:57.130312","modified":"2021-02-10 11:14:59.506936","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4373":{"gid":4373,"name":"b","rank":0,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.2.108:6833/3230549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3230549207},{"type":"v1","addr":"172.21.2.108:6833","nonce":3230549207}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-02-10T11:15:17.438 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 13 2021-02-10T11:15:17.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.434 7f8502cde700 1 -- 172.21.2.101:0/2503313259 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84e00186c0 msgr2=0x7f84e001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:17.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.434 7f8502cde700 1 --2- 172.21.2.101:0/2503313259 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84e00186c0 0x7f84e001abc0 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:17.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.434 7f8502cde700 1 -- 172.21.2.101:0/2503313259 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84fc1918e0 msgr2=0x7f84fc189c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:17.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.434 7f8502cde700 1 --2- 172.21.2.101:0/2503313259 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84fc1918e0 0x7f84fc189c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f84e4006f60 tx=0x7f84e400b970).stop 2021-02-10T11:15:17.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.434 7f8502cde700 1 -- 172.21.2.101:0/2503313259 shutdown_connections 2021-02-10T11:15:17.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.434 7f8502cde700 1 --2- 172.21.2.101:0/2503313259 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84e00186c0 0x7f84e001abc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:17.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.434 7f8502cde700 1 --2- 172.21.2.101:0/2503313259 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f84fc18acc0 0x7f84fc190070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:17.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.434 7f8502cde700 1 --2- 172.21.2.101:0/2503313259 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f84fc18a2a0 0x7f84fc18a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:17.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.434 7f8502cde700 1 --2- 172.21.2.101:0/2503313259 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84fc1918e0 0x7f84fc189c80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:17.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.434 7f8502cde700 1 -- 172.21.2.101:0/2503313259 >> 172.21.2.101:0/2503313259 conn(0x7f84fc1a31d0 msgr2=0x7f84fc08b830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:17.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.434 7f8502cde700 1 -- 172.21.2.101:0/2503313259 shutdown_connections 2021-02-10T11:15:17.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:17.434 7f8502cde700 1 -- 172.21.2.101:0/2503313259 wait complete. 2021-02-10T11:15:17.649 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:14:57.130312', 'modified': '2021-02-10 11:14:59.506936', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4373}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4373': {'gid': 4373, 'name': 'b', 'rank': 0, 'incarnation': 10, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.2.108:6833/3230549207', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 3230549207}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 3230549207}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:15:17.649 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:15:17.650 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:15:18.048 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:15:18.048 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:15:18.049 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:15:18.049 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 4373, 2021-02-10T11:15:18.049 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:15:18.049 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:15:18.049 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 20.911790521, 2021-02-10T11:15:18.049 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 13, 2021-02-10T11:15:18.049 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 29, 2021-02-10T11:15:18.049 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 29, 2021-02-10T11:15:18.049 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 39.336876193999998 2021-02-10T11:15:18.049 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:15:18.052 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 4373, "want_state": "up:active", "state": "up:active", "rank_uptime": 20.911790521, "mdsmap_epoch": 13, "osdmap_epoch": 29, "osdmap_epoch_barrier": 29, "uptime": 39.336876193999998 } 2021-02-10T11:15:18.052 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:15:18.053 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-02-10T11:15:18.053 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-02-10T11:15:18.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:18.053 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:18.388 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:15:18.732 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:15:18.733 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:15:19.075 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:15:19.075 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:15:19.075 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:15:19.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:19.076 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:15:19.410 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-02-10T11:15:19.410 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:19.412 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:15:19.758 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:15:19.759 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:15:19.825 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:2021-02-10 11:15:19.820 7f3e68b8af80 -1 init, newargv = 0x55a6bdf4f120 newargc=9 2021-02-10T11:15:19.825 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[34129]: starting ceph client 2021-02-10T11:15:19.830 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[34129]: starting fuse 2021-02-10T11:15:20.100 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:15:20.100 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:15:20.101 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:15:20.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:20.101 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:15:20.434 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:15:20.435 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2021-02-10T11:15:20.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:20.435 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:15:20.435 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:15:20.435 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:15:20.435 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:15:20.435 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:15:20.436 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:15:20.436 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:15:20.436 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-02-10T11:15:20.436 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:15:20.436 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:15:20.436 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:15:20.436 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:15:20.436 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:15:20.436 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:15:20.436 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:15:20.436 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:15:20.436 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:15:20.436 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:15:20.436 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:15:20.436 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:15:20.437 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:15:20.437 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.0")) 2021-02-10T11:15:20.437 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:15:20.797 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.0.34129.asok 2021-02-10T11:15:20.799 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.34129.asok 2021-02-10T11:15:20.800 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:20.800 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.34129.asok status 2021-02-10T11:15:21.187 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:15:21.187 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:15:21.187 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:15:21.187 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:15:21.187 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "0", 2021-02-10T11:15:21.187 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:15:21.187 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-02-10T11:15:21.187 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "34129", 2021-02-10T11:15:21.187 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:15:21.187 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:15:21.188 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:15:21.188 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:15:21.188 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 5085, 2021-02-10T11:15:21.188 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:15:21.188 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:15:21.188 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:15:21.188 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 5085 2021-02-10T11:15:21.188 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:15:21.188 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:15:21.188 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:15:21.188 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:15:21.188 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 3980499065 2021-02-10T11:15:21.188 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:15:21.189 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:15:21.189 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:15:21.189 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:15:21.189 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:15:21.189 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 3980499065 2021-02-10T11:15:21.189 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:15:21.189 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.5085 172.21.2.101:0/3980499065", 2021-02-10T11:15:21.189 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/3980499065", 2021-02-10T11:15:21.189 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:15:21.189 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 13, 2021-02-10T11:15:21.189 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 29, 2021-02-10T11:15:21.189 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:15:21.189 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:15:21.190 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:15:21.191 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:21.191 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:21.191 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:21.525 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:21.526 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:21.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:21.526 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:21.872 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:15:22.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.278 7fd2f4d66700 1 Processor -- start 2021-02-10T11:15:22.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.278 7fd2f4d66700 1 -- start start 2021-02-10T11:15:22.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.278 7fd2f4d66700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd2f00e5470 0x7fd2f007e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:22.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.278 7fd2f4d66700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2f007f300 0x7fd2f007f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:22.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.278 7fd2f4d66700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f007ed80 con 0x7fd2f00e5070 2021-02-10T11:15:22.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.278 7fd2f4d66700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f01b4010 con 0x7fd2f00e5470 2021-02-10T11:15:22.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.278 7fd2f4d66700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f00d1790 con 0x7fd2f007f300 2021-02-10T11:15:22.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.278 7fd2ef314700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2f007f300 0x7fd2f007f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:22.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.278 7fd2eeb13700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd2f00e5070 0x7fd2f01b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46038/0 (socket says 172.21.2.101:46038) 2021-02-10T11:15:22.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.278 7fd2eeb13700 1 -- 172.21.2.101:0/1851219038 learned_addr learned my addr 172.21.2.101:0/1851219038 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:15:22.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.278 7fd2ee312700 1 --2- 172.21.2.101:0/1851219038 >> v2:172.21.2.108:3300/0 conn(0x7fd2f00e5470 0x7fd2f007e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:22.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.279 7fd2ef314700 1 -- 172.21.2.101:0/1851219038 >> v2:172.21.2.108:3300/0 conn(0x7fd2f00e5470 msgr2=0x7fd2f007e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:22.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.279 7fd2ef314700 1 --2- 172.21.2.101:0/1851219038 >> v2:172.21.2.108:3300/0 conn(0x7fd2f00e5470 0x7fd2f007e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:22.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.279 7fd2ef314700 1 -- 172.21.2.101:0/1851219038 >> v1:172.21.2.101:6789/0 conn(0x7fd2f00e5070 legacy=0x7fd2f01b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:22.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.279 7fd2ef314700 1 -- 172.21.2.101:0/1851219038 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f00d19b0 con 0x7fd2f007f300 2021-02-10T11:15:22.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.279 7fd2ee312700 1 --2- 172.21.2.101:0/1851219038 >> v2:172.21.2.108:3300/0 conn(0x7fd2f00e5470 0x7fd2f007e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:15:22.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.279 7fd2ef314700 1 --2- 172.21.2.101:0/1851219038 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2f007f300 0x7fd2f007f700 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fd2e8006e50 tx=0x7fd2e8009e70).ready entity=mon.2 client_cookie=1a52502fbad7f35 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:22.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.279 7fd2edb11700 1 -- 172.21.2.101:0/1851219038 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2e8006a20 con 0x7fd2f007f300 2021-02-10T11:15:22.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.280 7fd2edb11700 1 -- 172.21.2.101:0/1851219038 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e8006790 con 0x7fd2f007f300 2021-02-10T11:15:22.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.280 7fd2edb11700 1 -- 172.21.2.101:0/1851219038 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2e800b700 con 0x7fd2f007f300 2021-02-10T11:15:22.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.280 7fd2f4d66700 1 -- 172.21.2.101:0/1851219038 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2f007f300 msgr2=0x7fd2f007f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:22.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.280 7fd2f4d66700 1 --2- 172.21.2.101:0/1851219038 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2f007f300 0x7fd2f007f700 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fd2e8006e50 tx=0x7fd2e8009e70).stop 2021-02-10T11:15:22.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.280 7fd2f4d66700 1 -- 172.21.2.101:0/1851219038 shutdown_connections 2021-02-10T11:15:22.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.280 7fd2f4d66700 1 --2- 172.21.2.101:0/1851219038 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2f007f300 0x7fd2f007f700 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:22.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.280 7fd2f4d66700 1 --2- 172.21.2.101:0/1851219038 >> v2:172.21.2.108:3300/0 conn(0x7fd2f00e5470 0x7fd2f007e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:22.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.280 7fd2f4d66700 1 -- 172.21.2.101:0/1851219038 >> 172.21.2.101:0/1851219038 conn(0x7fd2f00ce690 msgr2=0x7fd2f00e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:22.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.280 7fd2f4d66700 1 -- 172.21.2.101:0/1851219038 shutdown_connections 2021-02-10T11:15:22.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.280 7fd2f4d66700 1 -- 172.21.2.101:0/1851219038 wait complete. 2021-02-10T11:15:22.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.295 7fd2f4d66700 1 Processor -- start 2021-02-10T11:15:22.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.295 7fd2f4d66700 1 -- start start 2021-02-10T11:15:22.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.295 7fd2f4d66700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2f008d400 0x7fd2f0081230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:22.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.295 7fd2f4d66700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2f0081850 0x7fd2f00acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:22.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.295 7fd2f4d66700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2f0081c50 0x7fd2f00ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:22.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.295 7fd2f4d66700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f0082110 con 0x7fd2f0081c50 2021-02-10T11:15:22.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.295 7fd2f4d66700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f008d800 con 0x7fd2f0081850 2021-02-10T11:15:22.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.295 7fd2f4d66700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f00adbc0 con 0x7fd2f008d400 2021-02-10T11:15:22.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.295 7fd2ef314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2f0081c50 0x7fd2f00ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:22.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.295 7fd2ef314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2f0081c50 0x7fd2f00ad5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45546/0 (socket says 172.21.2.101:45546) 2021-02-10T11:15:22.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.295 7fd2ef314700 1 -- 172.21.2.101:0/178445053 learned_addr learned my addr 172.21.2.101:0/178445053 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:22.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.295 7fd2eeb13700 1 --2- 172.21.2.101:0/178445053 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2f008d400 0x7fd2f0081230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:22.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.295 7fd2ee312700 1 --2- 172.21.2.101:0/178445053 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2f0081850 0x7fd2f00acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:22.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.296 7fd2ef314700 1 -- 172.21.2.101:0/178445053 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2f008d400 msgr2=0x7fd2f0081230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:22.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.296 7fd2ef314700 1 --2- 172.21.2.101:0/178445053 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2f008d400 0x7fd2f0081230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:22.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.296 7fd2ef314700 1 -- 172.21.2.101:0/178445053 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2f0081850 msgr2=0x7fd2f00acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:22.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.296 7fd2ef314700 1 --2- 172.21.2.101:0/178445053 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2f0081850 0x7fd2f00acf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:22.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.296 7fd2ef314700 1 -- 172.21.2.101:0/178445053 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f0187fc0 con 0x7fd2f0081c50 2021-02-10T11:15:22.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.296 7fd2ef314700 1 --2- 172.21.2.101:0/178445053 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2f0081c50 0x7fd2f00ad5a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fd2e8006e60 tx=0x7fd2e800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:22.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.296 7fd2e77fe700 1 -- 172.21.2.101:0/178445053 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2e8009130 con 0x7fd2f0081c50 2021-02-10T11:15:22.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.296 7fd2e77fe700 1 -- 172.21.2.101:0/178445053 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e8000e80 con 0x7fd2f0081c50 2021-02-10T11:15:22.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.296 7fd2e77fe700 1 -- 172.21.2.101:0/178445053 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2e8001390 con 0x7fd2f0081c50 2021-02-10T11:15:22.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.296 7fd2f4d66700 1 -- 172.21.2.101:0/178445053 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f0187fc0 con 0x7fd2f0081c50 2021-02-10T11:15:22.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.296 7fd2f4d66700 1 -- 172.21.2.101:0/178445053 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f0187fc0 con 0x7fd2f0081c50 2021-02-10T11:15:22.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.297 7fd2e77fe700 1 -- 172.21.2.101:0/178445053 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd2e8009130 con 0x7fd2f0081c50 2021-02-10T11:15:22.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.297 7fd2e77fe700 1 --2- 172.21.2.101:0/178445053 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2d4018730 0x7fd2d401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:22.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.297 7fd2e77fe700 1 -- 172.21.2.101:0/178445053 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd2e8032b00 con 0x7fd2f0081c50 2021-02-10T11:15:22.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.297 7fd2eeb13700 1 --2- 172.21.2.101:0/178445053 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2d4018730 0x7fd2d401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:22.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.297 7fd2eeb13700 1 --2- 172.21.2.101:0/178445053 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2d4018730 0x7fd2d401ac10 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:22.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.328 7fd2f4d66700 1 -- 172.21.2.101:0/178445053 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f008d800 con 0x7fd2f0081c50 2021-02-10T11:15:22.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.330 7fd2e77fe700 1 -- 172.21.2.101:0/178445053 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd2e800be30 con 0x7fd2f0081c50 2021-02-10T11:15:22.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.456 7fd2f4d66700 1 -- 172.21.2.101:0/178445053 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2f008d800 con 0x7fd2f0081c50 2021-02-10T11:15:22.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.457 7fd2e77fe700 1 -- 172.21.2.101:0/178445053 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+13993 (secure 0 0 0) 0x7fd2e8000df0 con 0x7fd2f0081c50 2021-02-10T11:15:22.462 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:15:22.462 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":29,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:14:58.132054","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:14:54.652789","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2021-02-10 11:14:56.125641","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:15:22.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.459 7fd2f4d66700 1 -- 172.21.2.101:0/178445053 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2d4018730 msgr2=0x7fd2d401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:22.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.459 7fd2f4d66700 1 --2- 172.21.2.101:0/178445053 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2d4018730 0x7fd2d401ac10 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:22.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.459 7fd2f4d66700 1 -- 172.21.2.101:0/178445053 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2f0081c50 msgr2=0x7fd2f00ad5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:22.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.459 7fd2f4d66700 1 --2- 172.21.2.101:0/178445053 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2f0081c50 0x7fd2f00ad5a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fd2e8006e60 tx=0x7fd2e800bcf0).stop 2021-02-10T11:15:22.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.459 7fd2f4d66700 1 -- 172.21.2.101:0/178445053 shutdown_connections 2021-02-10T11:15:22.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.459 7fd2f4d66700 1 --2- 172.21.2.101:0/178445053 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2f0081c50 0x7fd2f00ad5a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:22.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.459 7fd2f4d66700 1 --2- 172.21.2.101:0/178445053 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2f0081850 0x7fd2f00acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:22.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.459 7fd2f4d66700 1 --2- 172.21.2.101:0/178445053 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2d4018730 0x7fd2d401ac10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:22.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.459 7fd2f4d66700 1 --2- 172.21.2.101:0/178445053 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2f008d400 0x7fd2f0081230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:22.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.459 7fd2f4d66700 1 -- 172.21.2.101:0/178445053 >> 172.21.2.101:0/178445053 conn(0x7fd2f01ac620 msgr2=0x7fd2f00896e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:22.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.459 7fd2f4d66700 1 -- 172.21.2.101:0/178445053 shutdown_connections 2021-02-10T11:15:22.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.459 7fd2f4d66700 1 -- 172.21.2.101:0/178445053 wait complete. 2021-02-10T11:15:22.473 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:14:54.652789', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:14:56.125641', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:15:22.473 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:15:22.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.881 7ff17ec2d700 1 Processor -- start 2021-02-10T11:15:22.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.881 7ff17ec2d700 1 -- start start 2021-02-10T11:15:22.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.881 7ff17ec2d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff1781b0470 0x7ff1781b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:22.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.881 7ff17ec2d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff1780ef610 0x7ff1780efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:22.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.881 7ff17ec2d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1780f0b50 con 0x7ff1780f0030 2021-02-10T11:15:22.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.881 7ff17ec2d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1780e52e0 con 0x7ff1781b0470 2021-02-10T11:15:22.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.881 7ff17ec2d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1780e5490 con 0x7ff1780ef610 2021-02-10T11:15:22.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.882 7ff17ca49700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff1781b0470 0x7ff1781b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:22.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.882 7ff17ca49700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff1781b0470 0x7ff1781b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45554/0 (socket says 172.21.2.101:45554) 2021-02-10T11:15:22.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.882 7ff17ca49700 1 -- 172.21.2.101:0/2042251121 learned_addr learned my addr 172.21.2.101:0/2042251121 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:22.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.882 7ff17ca49700 1 -- 172.21.2.101:0/2042251121 >> v2:172.21.2.108:3300/0 conn(0x7ff1780ef610 msgr2=0x7ff1780efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:22.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.882 7ff17ca49700 1 --2- 172.21.2.101:0/2042251121 >> v2:172.21.2.108:3300/0 conn(0x7ff1780ef610 0x7ff1780efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:22.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.882 7ff17ca49700 1 -- 172.21.2.101:0/2042251121 >> v1:172.21.2.108:6789/0 conn(0x7ff1780f0030 legacy=0x7ff1780f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:22.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.882 7ff17ca49700 1 -- 172.21.2.101:0/2042251121 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1780f0430 con 0x7ff1781b0470 2021-02-10T11:15:22.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.882 7ff17ca49700 1 --2- 172.21.2.101:0/2042251121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1781b0470 0x7ff1781b0870 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7ff168006700 tx=0x7ff168009b80).ready entity=mon.0 client_cookie=4cf7ff112739540 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:22.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.883 7ff1777fe700 1 -- 172.21.2.101:0/2042251121 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff16800c0c0 con 0x7ff1781b0470 2021-02-10T11:15:22.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.883 7ff1777fe700 1 -- 172.21.2.101:0/2042251121 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16800c440 con 0x7ff1781b0470 2021-02-10T11:15:22.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.883 7ff1777fe700 1 -- 172.21.2.101:0/2042251121 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff16800c960 con 0x7ff1781b0470 2021-02-10T11:15:22.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.883 7ff17ec2d700 1 -- 172.21.2.101:0/2042251121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1781b0470 msgr2=0x7ff1781b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:22.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.883 7ff17ec2d700 1 --2- 172.21.2.101:0/2042251121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1781b0470 0x7ff1781b0870 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7ff168006700 tx=0x7ff168009b80).stop 2021-02-10T11:15:22.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.883 7ff17ec2d700 1 -- 172.21.2.101:0/2042251121 shutdown_connections 2021-02-10T11:15:22.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.883 7ff17ec2d700 1 --2- 172.21.2.101:0/2042251121 >> v2:172.21.2.108:3300/0 conn(0x7ff1780ef610 0x7ff1780efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:22.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.883 7ff17ec2d700 1 --2- 172.21.2.101:0/2042251121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1781b0470 0x7ff1781b0870 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:22.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.883 7ff17ec2d700 1 -- 172.21.2.101:0/2042251121 >> 172.21.2.101:0/2042251121 conn(0x7ff1780ce3e0 msgr2=0x7ff1780e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:22.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.883 7ff17ec2d700 1 -- 172.21.2.101:0/2042251121 shutdown_connections 2021-02-10T11:15:22.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.883 7ff17ec2d700 1 -- 172.21.2.101:0/2042251121 wait complete. 2021-02-10T11:15:22.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.898 7ff17ec2d700 1 Processor -- start 2021-02-10T11:15:22.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.898 7ff17ec2d700 1 -- start start 2021-02-10T11:15:22.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.898 7ff17ec2d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1780a0960 0x7ff1780a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:22.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.898 7ff17ec2d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1781924c0 0x7ff1781928c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:22.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.898 7ff17ec2d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff178192ee0 0x7ff1780a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:22.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.898 7ff17ec2d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1780a6e10 con 0x7ff1781924c0 2021-02-10T11:15:22.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.898 7ff17ec2d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff178190630 con 0x7ff1780a0960 2021-02-10T11:15:22.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.898 7ff17ec2d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1781932e0 con 0x7ff178192ee0 2021-02-10T11:15:22.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.899 7ff177fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1781924c0 0x7ff1781928c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:22.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.899 7ff177fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1781924c0 0x7ff1781928c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45558/0 (socket says 172.21.2.101:45558) 2021-02-10T11:15:22.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.899 7ff177fff700 1 -- 172.21.2.101:0/3812131688 learned_addr learned my addr 172.21.2.101:0/3812131688 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:22.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.899 7ff17ca49700 1 --2- 172.21.2.101:0/3812131688 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1780a0960 0x7ff1780a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:22.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.899 7ff177fff700 1 -- 172.21.2.101:0/3812131688 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff178192ee0 msgr2=0x7ff1780a29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:22.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.899 7ff17d24a700 1 --2- 172.21.2.101:0/3812131688 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff178192ee0 0x7ff1780a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:22.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.899 7ff177fff700 1 --2- 172.21.2.101:0/3812131688 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff178192ee0 0x7ff1780a29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:22.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.899 7ff177fff700 1 -- 172.21.2.101:0/3812131688 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1780a0960 msgr2=0x7ff1780a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:22.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.899 7ff177fff700 1 --2- 172.21.2.101:0/3812131688 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1780a0960 0x7ff1780a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:22.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.899 7ff177fff700 1 -- 172.21.2.101:0/3812131688 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff178193500 con 0x7ff1781924c0 2021-02-10T11:15:22.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.899 7ff17ca49700 1 --2- 172.21.2.101:0/3812131688 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1780a0960 0x7ff1780a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:15:22.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.899 7ff17d24a700 1 --2- 172.21.2.101:0/3812131688 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff178192ee0 0x7ff1780a29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:15:22.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.899 7ff177fff700 1 --2- 172.21.2.101:0/3812131688 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1781924c0 0x7ff1781928c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7ff160006f00 tx=0x7ff16000b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:22.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.900 7ff1757fa700 1 -- 172.21.2.101:0/3812131688 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1600009b0 con 0x7ff1781924c0 2021-02-10T11:15:22.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.900 7ff1757fa700 1 -- 172.21.2.101:0/3812131688 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff160006350 con 0x7ff1781924c0 2021-02-10T11:15:22.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.900 7ff1757fa700 1 -- 172.21.2.101:0/3812131688 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff160000c90 con 0x7ff1781924c0 2021-02-10T11:15:22.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.900 7ff17ec2d700 1 -- 172.21.2.101:0/3812131688 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff178193500 con 0x7ff1781924c0 2021-02-10T11:15:22.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.900 7ff17ec2d700 1 -- 172.21.2.101:0/3812131688 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff178193500 con 0x7ff1781924c0 2021-02-10T11:15:22.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.900 7ff1757fa700 1 -- 172.21.2.101:0/3812131688 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff160007000 con 0x7ff1781924c0 2021-02-10T11:15:22.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.900 7ff1757fa700 1 --2- 172.21.2.101:0/3812131688 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff15c0186c0 0x7ff15c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:22.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.900 7ff1757fa700 1 -- 172.21.2.101:0/3812131688 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff160032560 con 0x7ff1781924c0 2021-02-10T11:15:22.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.901 7ff17ca49700 1 --2- 172.21.2.101:0/3812131688 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff15c0186c0 0x7ff15c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:22.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.901 7ff17ca49700 1 --2- 172.21.2.101:0/3812131688 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff15c0186c0 0x7ff15c01abc0 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:22.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.931 7ff17ec2d700 1 -- 172.21.2.101:0/3812131688 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1780a24c0 con 0x7ff1781924c0 2021-02-10T11:15:22.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:22.934 7ff1757fa700 1 -- 172.21.2.101:0/3812131688 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff16001ab30 con 0x7ff1781924c0 2021-02-10T11:15:23.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.061 7ff17ec2d700 1 -- 172.21.2.101:0/3812131688 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7ff1780a24c0 con 0x7ff1781924c0 2021-02-10T11:15:23.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.061 7ff1757fa700 1 -- 172.21.2.101:0/3812131688 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff16001ab30 con 0x7ff1781924c0 2021-02-10T11:15:23.067 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:15:23.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.063 7ff17ec2d700 1 -- 172.21.2.101:0/3812131688 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff15c0186c0 msgr2=0x7ff15c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:23.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.063 7ff17ec2d700 1 --2- 172.21.2.101:0/3812131688 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff15c0186c0 0x7ff15c01abc0 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:23.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.063 7ff17ec2d700 1 -- 172.21.2.101:0/3812131688 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1781924c0 msgr2=0x7ff1781928c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:23.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.063 7ff17ec2d700 1 --2- 172.21.2.101:0/3812131688 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1781924c0 0x7ff1781928c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7ff160006f00 tx=0x7ff16000b940).stop 2021-02-10T11:15:23.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.063 7ff17ec2d700 1 -- 172.21.2.101:0/3812131688 shutdown_connections 2021-02-10T11:15:23.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.063 7ff17ec2d700 1 --2- 172.21.2.101:0/3812131688 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff15c0186c0 0x7ff15c01abc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:23.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.063 7ff17ec2d700 1 --2- 172.21.2.101:0/3812131688 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff178192ee0 0x7ff1780a29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:23.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.063 7ff17ec2d700 1 --2- 172.21.2.101:0/3812131688 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1781924c0 0x7ff1781928c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:23.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.063 7ff17ec2d700 1 --2- 172.21.2.101:0/3812131688 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1780a0960 0x7ff1780a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:23.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.063 7ff17ec2d700 1 -- 172.21.2.101:0/3812131688 >> 172.21.2.101:0/3812131688 conn(0x7ff1780ac550 msgr2=0x7ff1780be1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:23.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.064 7ff17ec2d700 1 -- 172.21.2.101:0/3812131688 shutdown_connections 2021-02-10T11:15:23.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.064 7ff17ec2d700 1 -- 172.21.2.101:0/3812131688 wait complete. 2021-02-10T11:15:23.076 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:15:23.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.484 7ff3dd202700 1 Processor -- start 2021-02-10T11:15:23.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.484 7ff3dd202700 1 -- start start 2021-02-10T11:15:23.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.484 7ff3dd202700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff3d81b0470 0x7ff3d81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:23.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.484 7ff3dd202700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff3d80ef610 0x7ff3d80efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:23.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.485 7ff3dd202700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d80f0b50 con 0x7ff3d80f0030 2021-02-10T11:15:23.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.485 7ff3dd202700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d80e52e0 con 0x7ff3d81b0470 2021-02-10T11:15:23.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.485 7ff3dd202700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d80e5490 con 0x7ff3d80ef610 2021-02-10T11:15:23.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.485 7ff3d6d9d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff3d81b0470 0x7ff3d81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:23.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.485 7ff3d6d9d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff3d81b0470 0x7ff3d81b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45566/0 (socket says 172.21.2.101:45566) 2021-02-10T11:15:23.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.485 7ff3d6d9d700 1 -- 172.21.2.101:0/783275421 learned_addr learned my addr 172.21.2.101:0/783275421 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:23.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.485 7ff3d6d9d700 1 -- 172.21.2.101:0/783275421 >> v2:172.21.2.108:3300/0 conn(0x7ff3d80ef610 msgr2=0x7ff3d80efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:23.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.485 7ff3d6d9d700 1 --2- 172.21.2.101:0/783275421 >> v2:172.21.2.108:3300/0 conn(0x7ff3d80ef610 0x7ff3d80efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:23.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.485 7ff3d6d9d700 1 -- 172.21.2.101:0/783275421 >> v1:172.21.2.108:6789/0 conn(0x7ff3d80f0030 legacy=0x7ff3d80f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:23.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.485 7ff3d6d9d700 1 -- 172.21.2.101:0/783275421 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d80f0430 con 0x7ff3d81b0470 2021-02-10T11:15:23.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.485 7ff3d6d9d700 1 --2- 172.21.2.101:0/783275421 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3d81b0470 0x7ff3d81b0870 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7ff3c8006700 tx=0x7ff3c8009b80).ready entity=mon.0 client_cookie=a0a8049b6aef57c9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:23.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.486 7ff3d5d9b700 1 -- 172.21.2.101:0/783275421 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff3c800c0c0 con 0x7ff3d81b0470 2021-02-10T11:15:23.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.486 7ff3d5d9b700 1 -- 172.21.2.101:0/783275421 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c800c440 con 0x7ff3d81b0470 2021-02-10T11:15:23.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.486 7ff3d5d9b700 1 -- 172.21.2.101:0/783275421 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff3c800c920 con 0x7ff3d81b0470 2021-02-10T11:15:23.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.486 7ff3dd202700 1 -- 172.21.2.101:0/783275421 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3d81b0470 msgr2=0x7ff3d81b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:23.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.486 7ff3dd202700 1 --2- 172.21.2.101:0/783275421 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3d81b0470 0x7ff3d81b0870 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7ff3c8006700 tx=0x7ff3c8009b80).stop 2021-02-10T11:15:23.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.486 7ff3dd202700 1 -- 172.21.2.101:0/783275421 shutdown_connections 2021-02-10T11:15:23.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.486 7ff3dd202700 1 --2- 172.21.2.101:0/783275421 >> v2:172.21.2.108:3300/0 conn(0x7ff3d80ef610 0x7ff3d80efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:23.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.486 7ff3dd202700 1 --2- 172.21.2.101:0/783275421 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3d81b0470 0x7ff3d81b0870 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:23.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.486 7ff3dd202700 1 -- 172.21.2.101:0/783275421 >> 172.21.2.101:0/783275421 conn(0x7ff3d80ce3e0 msgr2=0x7ff3d80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:23.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.487 7ff3dd202700 1 -- 172.21.2.101:0/783275421 shutdown_connections 2021-02-10T11:15:23.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.487 7ff3dd202700 1 -- 172.21.2.101:0/783275421 wait complete. 2021-02-10T11:15:23.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.501 7ff3dd202700 1 Processor -- start 2021-02-10T11:15:23.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.501 7ff3dd202700 1 -- start start 2021-02-10T11:15:23.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.502 7ff3dd202700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff3d80c36a0 0x7ff3d80c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:23.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.502 7ff3dd202700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3d80c3190 0x7ff3d80b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:23.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.502 7ff3dd202700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3d80b2370 0x7ff3d80b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:23.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.502 7ff3dd202700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d80b27c0 con 0x7ff3d80c3190 2021-02-10T11:15:23.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.502 7ff3dd202700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d80b3360 con 0x7ff3d80c36a0 2021-02-10T11:15:23.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.502 7ff3dd202700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d80b3540 con 0x7ff3d80b2370 2021-02-10T11:15:23.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.502 7ff3d659c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3d80c3190 0x7ff3d80b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:23.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.502 7ff3d659c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3d80c3190 0x7ff3d80b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45574/0 (socket says 172.21.2.101:45574) 2021-02-10T11:15:23.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.502 7ff3d659c700 1 -- 172.21.2.101:0/1546794846 learned_addr learned my addr 172.21.2.101:0/1546794846 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:23.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.502 7ff3d659c700 1 -- 172.21.2.101:0/1546794846 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3d80b2370 msgr2=0x7ff3d80b2d40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:23.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.502 7ff3d659c700 1 --2- 172.21.2.101:0/1546794846 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3d80b2370 0x7ff3d80b2d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:23.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.502 7ff3d659c700 1 -- 172.21.2.101:0/1546794846 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff3d80c36a0 msgr2=0x7ff3d80c2b70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:23.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.502 7ff3d659c700 1 --2- 172.21.2.101:0/1546794846 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff3d80c36a0 0x7ff3d80c2b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:23.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.502 7ff3d659c700 1 -- 172.21.2.101:0/1546794846 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d80b3760 con 0x7ff3d80c3190 2021-02-10T11:15:23.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.502 7ff3d659c700 1 --2- 172.21.2.101:0/1546794846 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3d80c3190 0x7ff3d80b1d50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7ff3cc006f00 tx=0x7ff3cc00b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:23.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.502 7ff3c77fe700 1 -- 172.21.2.101:0/1546794846 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff3cc006350 con 0x7ff3d80c3190 2021-02-10T11:15:23.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.502 7ff3c77fe700 1 -- 172.21.2.101:0/1546794846 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3cc00bdc0 con 0x7ff3d80c3190 2021-02-10T11:15:23.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.502 7ff3dd202700 1 -- 172.21.2.101:0/1546794846 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d80b3760 con 0x7ff3d80c3190 2021-02-10T11:15:23.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.502 7ff3c77fe700 1 -- 172.21.2.101:0/1546794846 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff3cc002020 con 0x7ff3d80c3190 2021-02-10T11:15:23.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.503 7ff3dd202700 1 -- 172.21.2.101:0/1546794846 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d80b3760 con 0x7ff3d80c3190 2021-02-10T11:15:23.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.503 7ff3c77fe700 1 -- 172.21.2.101:0/1546794846 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff3cc007020 con 0x7ff3d80c3190 2021-02-10T11:15:23.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.503 7ff3c77fe700 1 --2- 172.21.2.101:0/1546794846 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff3bc0186c0 0x7ff3bc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:23.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.503 7ff3c77fe700 1 -- 172.21.2.101:0/1546794846 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff3cc0327a0 con 0x7ff3d80c3190 2021-02-10T11:15:23.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.503 7ff3d6d9d700 1 --2- 172.21.2.101:0/1546794846 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff3bc0186c0 0x7ff3bc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:23.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.503 7ff3d6d9d700 1 --2- 172.21.2.101:0/1546794846 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff3bc0186c0 0x7ff3bc01abc0 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:23.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.534 7ff3dd202700 1 -- 172.21.2.101:0/1546794846 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d80edd10 con 0x7ff3d80c3190 2021-02-10T11:15:23.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.537 7ff3c77fe700 1 -- 172.21.2.101:0/1546794846 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff3cc006b90 con 0x7ff3d80c3190 2021-02-10T11:15:23.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.666 7ff3dd202700 1 -- 172.21.2.101:0/1546794846 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7ff3d80e52e0 con 0x7ff3d80c3190 2021-02-10T11:15:23.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.666 7ff3c77fe700 1 -- 172.21.2.101:0/1546794846 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v29) v1 ==== 104+0+10 (secure 0 0 0) 0x7ff3cc00bd30 con 0x7ff3d80c3190 2021-02-10T11:15:23.671 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:15:23.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.668 7ff3dd202700 1 -- 172.21.2.101:0/1546794846 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff3bc0186c0 msgr2=0x7ff3bc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:23.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.668 7ff3dd202700 1 --2- 172.21.2.101:0/1546794846 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff3bc0186c0 0x7ff3bc01abc0 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:23.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.668 7ff3dd202700 1 -- 172.21.2.101:0/1546794846 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3d80c3190 msgr2=0x7ff3d80b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:23.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.668 7ff3dd202700 1 --2- 172.21.2.101:0/1546794846 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3d80c3190 0x7ff3d80b1d50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7ff3cc006f00 tx=0x7ff3cc00b940).stop 2021-02-10T11:15:23.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.668 7ff3dd202700 1 -- 172.21.2.101:0/1546794846 shutdown_connections 2021-02-10T11:15:23.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.668 7ff3dd202700 1 --2- 172.21.2.101:0/1546794846 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff3bc0186c0 0x7ff3bc01abc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:23.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.668 7ff3dd202700 1 --2- 172.21.2.101:0/1546794846 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3d80b2370 0x7ff3d80b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:23.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.668 7ff3dd202700 1 --2- 172.21.2.101:0/1546794846 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3d80c3190 0x7ff3d80b1d50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:23.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.668 7ff3dd202700 1 --2- 172.21.2.101:0/1546794846 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff3d80c36a0 0x7ff3d80c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:23.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.668 7ff3dd202700 1 -- 172.21.2.101:0/1546794846 >> 172.21.2.101:0/1546794846 conn(0x7ff3d809f630 msgr2=0x7ff3d80a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:23.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.669 7ff3dd202700 1 -- 172.21.2.101:0/1546794846 shutdown_connections 2021-02-10T11:15:23.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:23.669 7ff3dd202700 1 -- 172.21.2.101:0/1546794846 wait complete. 2021-02-10T11:15:23.681 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:15:24.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.088 7ff1f1254700 1 Processor -- start 2021-02-10T11:15:24.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.088 7ff1f1254700 1 -- start start 2021-02-10T11:15:24.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.089 7ff1f1254700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ec1b0870 0x7ff1ec0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.089 7ff1f1254700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff1ec0efe00 0x7ff1ec0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.089 7ff1f1254700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ec1b0c70 con 0x7ff1ec1b0470 2021-02-10T11:15:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.089 7ff1f1254700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ec0f0b50 con 0x7ff1ec0efe00 2021-02-10T11:15:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.089 7ff1f1254700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec0f0820 con 0x7ff1ec1b0870 2021-02-10T11:15:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.090 7ff1eb59e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff1ec0efe00 0x7ff1ec0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.090 7ff1ea59c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ec1b0870 0x7ff1ec0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.090 7ff1ead9d700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7ff1ec1b0470 0x7ff1ec0ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:39482/0 (socket says 172.21.2.101:39482) 2021-02-10T11:15:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.090 7ff1ead9d700 1 -- 172.21.2.101:0/4281252324 learned_addr learned my addr 172.21.2.101:0/4281252324 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:15:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.091 7ff1e9d9b700 1 -- 172.21.2.101:0/4281252324 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1047318421 0 0) 0x7ff1dc001190 con 0x7ff1ec1b0470 2021-02-10T11:15:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.091 7ff1ea59c700 1 -- 172.21.2.101:0/4281252324 >> v2:172.21.2.108:3300/0 conn(0x7ff1ec0efe00 msgr2=0x7ff1ec0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.091 7ff1ea59c700 1 --2- 172.21.2.101:0/4281252324 >> v2:172.21.2.108:3300/0 conn(0x7ff1ec0efe00 0x7ff1ec0f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.091 7ff1ea59c700 1 -- 172.21.2.101:0/4281252324 >> v1:172.21.2.108:6789/0 conn(0x7ff1ec1b0470 legacy=0x7ff1ec0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.091 7ff1ea59c700 1 -- 172.21.2.101:0/4281252324 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec0e52e0 con 0x7ff1ec1b0870 2021-02-10T11:15:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.091 7ff1eb59e700 1 --2- 172.21.2.101:0/4281252324 >> v2:172.21.2.108:3300/0 conn(0x7ff1ec0efe00 0x7ff1ec0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:15:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.091 7ff1ea59c700 1 --2- 172.21.2.101:0/4281252324 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ec1b0870 0x7ff1ec0f20e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7ff1d4006650 tx=0x7ff1d4009a80).ready entity=mon.2 client_cookie=60a577759bf753d8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.091 7ff1e9d9b700 1 -- 172.21.2.101:0/4281252324 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1d4006390 con 0x7ff1ec1b0870 2021-02-10T11:15:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.091 7ff1e9d9b700 1 -- 172.21.2.101:0/4281252324 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d40009b0 con 0x7ff1ec1b0870 2021-02-10T11:15:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.091 7ff1e9d9b700 1 -- 172.21.2.101:0/4281252324 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1d400c2b0 con 0x7ff1ec1b0870 2021-02-10T11:15:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.091 7ff1f1254700 1 -- 172.21.2.101:0/4281252324 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ec1b0870 msgr2=0x7ff1ec0f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.091 7ff1f1254700 1 --2- 172.21.2.101:0/4281252324 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ec1b0870 0x7ff1ec0f20e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7ff1d4006650 tx=0x7ff1d4009a80).stop 2021-02-10T11:15:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.092 7ff1f1254700 1 -- 172.21.2.101:0/4281252324 shutdown_connections 2021-02-10T11:15:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.092 7ff1f1254700 1 --2- 172.21.2.101:0/4281252324 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ec1b0870 0x7ff1ec0f20e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.092 7ff1f1254700 1 --2- 172.21.2.101:0/4281252324 >> v2:172.21.2.108:3300/0 conn(0x7ff1ec0efe00 0x7ff1ec0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.092 7ff1f1254700 1 -- 172.21.2.101:0/4281252324 >> 172.21.2.101:0/4281252324 conn(0x7ff1ec0ce3e0 msgr2=0x7ff1ec0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.092 7ff1f1254700 1 -- 172.21.2.101:0/4281252324 shutdown_connections 2021-02-10T11:15:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.092 7ff1f1254700 1 -- 172.21.2.101:0/4281252324 wait complete. 2021-02-10T11:15:24.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.107 7ff1f1254700 1 Processor -- start 2021-02-10T11:15:24.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.107 7ff1f1254700 1 -- start start 2021-02-10T11:15:24.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.107 7ff1f1254700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ec09f140 0x7ff1ec0c1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:24.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.107 7ff1f1254700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec0c1f60 0x7ff1ec09e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:24.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.107 7ff1f1254700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1ec09e920 0x7ff1ec0c2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:24.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.107 7ff1f1254700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec0c23e0 con 0x7ff1ec0c1f60 2021-02-10T11:15:24.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.107 7ff1f1254700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec09ed20 con 0x7ff1ec09e920 2021-02-10T11:15:24.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.107 7ff1f1254700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec105d10 con 0x7ff1ec09f140 2021-02-10T11:15:24.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.108 7ff1ea59c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec0c1f60 0x7ff1ec09e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:24.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.108 7ff1ea59c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec0c1f60 0x7ff1ec09e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45582/0 (socket says 172.21.2.101:45582) 2021-02-10T11:15:24.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.108 7ff1ea59c700 1 -- 172.21.2.101:0/3402837371 learned_addr learned my addr 172.21.2.101:0/3402837371 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:24.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.108 7ff1ea59c700 1 -- 172.21.2.101:0/3402837371 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ec09f140 msgr2=0x7ff1ec0c1940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:24.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.108 7ff1ea59c700 1 --2- 172.21.2.101:0/3402837371 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ec09f140 0x7ff1ec0c1940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:24.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.108 7ff1ea59c700 1 -- 172.21.2.101:0/3402837371 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1ec09e920 msgr2=0x7ff1ec0c2960 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:24.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.108 7ff1ea59c700 1 --2- 172.21.2.101:0/3402837371 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1ec09e920 0x7ff1ec0c2960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:24.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.108 7ff1ea59c700 1 -- 172.21.2.101:0/3402837371 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec1a4910 con 0x7ff1ec0c1f60 2021-02-10T11:15:24.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.108 7ff1ea59c700 1 --2- 172.21.2.101:0/3402837371 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec0c1f60 0x7ff1ec09e300 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7ff1d4006f70 tx=0x7ff1d400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:24.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.108 7ff1e37fe700 1 -- 172.21.2.101:0/3402837371 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1d40064a0 con 0x7ff1ec0c1f60 2021-02-10T11:15:24.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.108 7ff1e37fe700 1 -- 172.21.2.101:0/3402837371 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d400bde0 con 0x7ff1ec0c1f60 2021-02-10T11:15:24.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.108 7ff1f1254700 1 -- 172.21.2.101:0/3402837371 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ec1a4910 con 0x7ff1ec0c1f60 2021-02-10T11:15:24.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.108 7ff1f1254700 1 -- 172.21.2.101:0/3402837371 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec0c3270 con 0x7ff1ec0c1f60 2021-02-10T11:15:24.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.108 7ff1e37fe700 1 -- 172.21.2.101:0/3402837371 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1d40052e0 con 0x7ff1ec0c1f60 2021-02-10T11:15:24.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.109 7ff1e37fe700 1 -- 172.21.2.101:0/3402837371 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff1d4006030 con 0x7ff1ec0c1f60 2021-02-10T11:15:24.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.109 7ff1e37fe700 1 --2- 172.21.2.101:0/3402837371 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1d00186d0 0x7ff1d001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:24.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.109 7ff1e37fe700 1 -- 172.21.2.101:0/3402837371 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff1d40327a0 con 0x7ff1ec0c1f60 2021-02-10T11:15:24.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.109 7ff1ead9d700 1 --2- 172.21.2.101:0/3402837371 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1d00186d0 0x7ff1d001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:24.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.109 7ff1ead9d700 1 --2- 172.21.2.101:0/3402837371 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1d00186d0 0x7ff1d001abd0 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:24.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.139 7ff1f1254700 1 -- 172.21.2.101:0/3402837371 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ec0f0b50 con 0x7ff1ec0c1f60 2021-02-10T11:15:24.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.141 7ff1e37fe700 1 -- 172.21.2.101:0/3402837371 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff1d4000940 con 0x7ff1ec0c1f60 2021-02-10T11:15:24.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.271 7ff1f1254700 1 -- 172.21.2.101:0/3402837371 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7ff1ec0f0b50 con 0x7ff1ec0c1f60 2021-02-10T11:15:24.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.271 7ff1e37fe700 1 -- 172.21.2.101:0/3402837371 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v29) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff1d4006030 con 0x7ff1ec0c1f60 2021-02-10T11:15:24.276 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:15:24.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.273 7ff1f1254700 1 -- 172.21.2.101:0/3402837371 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1d00186d0 msgr2=0x7ff1d001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:24.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.273 7ff1f1254700 1 --2- 172.21.2.101:0/3402837371 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1d00186d0 0x7ff1d001abd0 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:24.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.273 7ff1f1254700 1 -- 172.21.2.101:0/3402837371 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec0c1f60 msgr2=0x7ff1ec09e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:24.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.273 7ff1f1254700 1 --2- 172.21.2.101:0/3402837371 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec0c1f60 0x7ff1ec09e300 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7ff1d4006f70 tx=0x7ff1d400b960).stop 2021-02-10T11:15:24.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.274 7ff1f1254700 1 -- 172.21.2.101:0/3402837371 shutdown_connections 2021-02-10T11:15:24.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.274 7ff1f1254700 1 --2- 172.21.2.101:0/3402837371 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1ec09e920 0x7ff1ec0c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:24.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.274 7ff1f1254700 1 --2- 172.21.2.101:0/3402837371 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec0c1f60 0x7ff1ec09e300 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:24.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.274 7ff1f1254700 1 --2- 172.21.2.101:0/3402837371 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1d00186d0 0x7ff1d001abd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:24.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.274 7ff1f1254700 1 --2- 172.21.2.101:0/3402837371 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ec09f140 0x7ff1ec0c1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:24.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.274 7ff1f1254700 1 -- 172.21.2.101:0/3402837371 >> 172.21.2.101:0/3402837371 conn(0x7ff1ec09fc10 msgr2=0x7ff1ec0b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:24.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.274 7ff1f1254700 1 -- 172.21.2.101:0/3402837371 shutdown_connections 2021-02-10T11:15:24.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.274 7ff1f1254700 1 -- 172.21.2.101:0/3402837371 wait complete. 2021-02-10T11:15:24.286 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:15:24.287 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:15:24.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.692 7fa7e923e700 1 Processor -- start 2021-02-10T11:15:24.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.692 7fa7e923e700 1 -- start start 2021-02-10T11:15:24.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.693 7fa7e923e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7e40e5070 0x7fa7e41b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:24.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.693 7fa7e923e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa7e40e5470 0x7fa7e407d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:24.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.693 7fa7e923e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa7e41b3e40 0x7fa7e407dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:24.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.693 7fa7e923e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e407ed80 con 0x7fa7e41b3e40 2021-02-10T11:15:24.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.693 7fa7e923e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e407e2d0 con 0x7fa7e40e5470 2021-02-10T11:15:24.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.693 7fa7e923e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e407e4c0 con 0x7fa7e40e5070 2021-02-10T11:15:24.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.693 7fa7e359e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa7e41b3e40 0x7fa7e407dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:24.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.693 7fa7e359e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa7e41b3e40 0x7fa7e407dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45590/0 (socket says 172.21.2.101:45590) 2021-02-10T11:15:24.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.693 7fa7e359e700 1 -- 172.21.2.101:0/1864545190 learned_addr learned my addr 172.21.2.101:0/1864545190 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:24.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.693 7fa7e359e700 1 -- 172.21.2.101:0/1864545190 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7e40e5070 msgr2=0x7fa7e41b3820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:15:24.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.693 7fa7e359e700 1 --2- 172.21.2.101:0/1864545190 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7e40e5070 0x7fa7e41b3820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:24.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.693 7fa7e359e700 1 -- 172.21.2.101:0/1864545190 >> v2:172.21.2.108:3300/0 conn(0x7fa7e40e5470 msgr2=0x7fa7e407d690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:15:24.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.693 7fa7e359e700 1 --2- 172.21.2.101:0/1864545190 >> v2:172.21.2.108:3300/0 conn(0x7fa7e40e5470 0x7fa7e407d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:24.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.693 7fa7e359e700 1 -- 172.21.2.101:0/1864545190 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e407e6e0 con 0x7fa7e41b3e40 2021-02-10T11:15:24.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.694 7fa7e359e700 1 --2- 172.21.2.101:0/1864545190 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7e41b3e40 0x7fa7e407dcb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7fa7dc006f00 tx=0x7fa7dc009f10).ready entity=mon.0 client_cookie=56b5217bc07f8162 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:24.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.694 7fa7e1d9b700 1 -- 172.21.2.101:0/1864545190 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa7dc00b3b0 con 0x7fa7e41b3e40 2021-02-10T11:15:24.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.694 7fa7e1d9b700 1 -- 172.21.2.101:0/1864545190 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7dc00b3b0 con 0x7fa7e41b3e40 2021-02-10T11:15:24.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.694 7fa7e1d9b700 1 -- 172.21.2.101:0/1864545190 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa7dc00b630 con 0x7fa7e41b3e40 2021-02-10T11:15:24.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.694 7fa7e923e700 1 -- 172.21.2.101:0/1864545190 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7e41b3e40 msgr2=0x7fa7e407dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:24.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.694 7fa7e923e700 1 --2- 172.21.2.101:0/1864545190 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7e41b3e40 0x7fa7e407dcb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7fa7dc006f00 tx=0x7fa7dc009f10).stop 2021-02-10T11:15:24.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.695 7fa7e923e700 1 -- 172.21.2.101:0/1864545190 shutdown_connections 2021-02-10T11:15:24.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.695 7fa7e923e700 1 --2- 172.21.2.101:0/1864545190 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7e41b3e40 0x7fa7e407dcb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:24.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.695 7fa7e923e700 1 --2- 172.21.2.101:0/1864545190 >> v2:172.21.2.108:3300/0 conn(0x7fa7e40e5470 0x7fa7e407d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:24.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.695 7fa7e923e700 1 --2- 172.21.2.101:0/1864545190 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7e40e5070 0x7fa7e41b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:24.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.695 7fa7e923e700 1 -- 172.21.2.101:0/1864545190 >> 172.21.2.101:0/1864545190 conn(0x7fa7e40ce690 msgr2=0x7fa7e40e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:24.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.695 7fa7e923e700 1 -- 172.21.2.101:0/1864545190 shutdown_connections 2021-02-10T11:15:24.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.695 7fa7e923e700 1 -- 172.21.2.101:0/1864545190 wait complete. 2021-02-10T11:15:24.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.709 7fa7e923e700 1 Processor -- start 2021-02-10T11:15:24.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.709 7fa7e923e700 1 -- start start 2021-02-10T11:15:24.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.710 7fa7e923e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7e4192960 0x7fa7e4192d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:24.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.710 7fa7e923e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7e40ad110 0x7fa7e40ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:24.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.710 7fa7e923e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7e40adb30 0x7fa7e408e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:24.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.710 7fa7e923e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e41a03e0 con 0x7fa7e40adb30 2021-02-10T11:15:24.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.710 7fa7e923e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e4193380 con 0x7fa7e4192960 2021-02-10T11:15:24.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.710 7fa7e923e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e408ea80 con 0x7fa7e40ad110 2021-02-10T11:15:24.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.710 7fa7e359e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7e40adb30 0x7fa7e408e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:24.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.710 7fa7e359e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7e40adb30 0x7fa7e408e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45594/0 (socket says 172.21.2.101:45594) 2021-02-10T11:15:24.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.710 7fa7e359e700 1 -- 172.21.2.101:0/1524379992 learned_addr learned my addr 172.21.2.101:0/1524379992 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:24.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.710 7fa7e359e700 1 -- 172.21.2.101:0/1524379992 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7e40ad110 msgr2=0x7fa7e40ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:24.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.710 7fa7e359e700 1 --2- 172.21.2.101:0/1524379992 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7e40ad110 0x7fa7e40ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:24.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.710 7fa7e359e700 1 -- 172.21.2.101:0/1524379992 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7e4192960 msgr2=0x7fa7e4192d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:24.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.710 7fa7e359e700 1 --2- 172.21.2.101:0/1524379992 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7e4192960 0x7fa7e4192d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:24.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.710 7fa7e359e700 1 -- 172.21.2.101:0/1524379992 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e408eca0 con 0x7fa7e40adb30 2021-02-10T11:15:24.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.710 7fa7e359e700 1 --2- 172.21.2.101:0/1524379992 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7e40adb30 0x7fa7e408e460 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7fa7dc009390 tx=0x7fa7dc00bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:24.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.711 7fa7d37fe700 1 -- 172.21.2.101:0/1524379992 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa7dc003030 con 0x7fa7e40adb30 2021-02-10T11:15:24.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.711 7fa7d37fe700 1 -- 172.21.2.101:0/1524379992 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7dc0068e0 con 0x7fa7e40adb30 2021-02-10T11:15:24.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.711 7fa7d37fe700 1 -- 172.21.2.101:0/1524379992 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa7dc00c570 con 0x7fa7e40adb30 2021-02-10T11:15:24.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.711 7fa7e923e700 1 -- 172.21.2.101:0/1524379992 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e408eca0 con 0x7fa7e40adb30 2021-02-10T11:15:24.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.711 7fa7e923e700 1 -- 172.21.2.101:0/1524379992 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e408f230 con 0x7fa7e40adb30 2021-02-10T11:15:24.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.711 7fa7d37fe700 1 -- 172.21.2.101:0/1524379992 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa7dc001270 con 0x7fa7e40adb30 2021-02-10T11:15:24.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.711 7fa7d37fe700 1 --2- 172.21.2.101:0/1524379992 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7c80186c0 0x7fa7c801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:24.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.711 7fa7d37fe700 1 -- 172.21.2.101:0/1524379992 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa7dc032930 con 0x7fa7e40adb30 2021-02-10T11:15:24.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.712 7fa7e2d9d700 1 --2- 172.21.2.101:0/1524379992 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7c80186c0 0x7fa7c801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:24.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.712 7fa7e2d9d700 1 --2- 172.21.2.101:0/1524379992 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7c80186c0 0x7fa7c801abc0 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:24.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.743 7fa7e923e700 1 -- 172.21.2.101:0/1524379992 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e4107120 con 0x7fa7e40adb30 2021-02-10T11:15:24.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.745 7fa7d37fe700 1 -- 172.21.2.101:0/1524379992 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa7dc027310 con 0x7fa7e40adb30 2021-02-10T11:15:24.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.874 7fa7e923e700 1 -- 172.21.2.101:0/1524379992 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7e407ed80 con 0x7fa7e40adb30 2021-02-10T11:15:24.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.874 7fa7d37fe700 1 -- 172.21.2.101:0/1524379992 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+2100 (secure 0 0 0) 0x7fa7dc027310 con 0x7fa7e40adb30 2021-02-10T11:15:24.879 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:15:24.880 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/3226463341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3226463341},{"type":"v1","addr":"172.21.2.101:6835","nonce":3226463341}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:14:57.130312","modified":"2021-02-10 11:14:59.506936","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4373":{"gid":4373,"name":"b","rank":0,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.2.108:6833/3230549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3230549207},{"type":"v1","addr":"172.21.2.108:6833","nonce":3230549207}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-02-10T11:15:24.880 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 13 2021-02-10T11:15:24.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.876 7fa7e923e700 1 -- 172.21.2.101:0/1524379992 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7c80186c0 msgr2=0x7fa7c801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:24.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.876 7fa7e923e700 1 --2- 172.21.2.101:0/1524379992 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7c80186c0 0x7fa7c801abc0 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:24.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.876 7fa7e923e700 1 -- 172.21.2.101:0/1524379992 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7e40adb30 msgr2=0x7fa7e408e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:24.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.876 7fa7e923e700 1 --2- 172.21.2.101:0/1524379992 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7e40adb30 0x7fa7e408e460 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7fa7dc009390 tx=0x7fa7dc00bd40).stop 2021-02-10T11:15:24.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.876 7fa7e923e700 1 -- 172.21.2.101:0/1524379992 shutdown_connections 2021-02-10T11:15:24.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.876 7fa7e923e700 1 --2- 172.21.2.101:0/1524379992 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7c80186c0 0x7fa7c801abc0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:24.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.876 7fa7e923e700 1 --2- 172.21.2.101:0/1524379992 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7e40ad110 0x7fa7e40ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:24.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.876 7fa7e923e700 1 --2- 172.21.2.101:0/1524379992 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7e40adb30 0x7fa7e408e460 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:24.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.876 7fa7e923e700 1 --2- 172.21.2.101:0/1524379992 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7e4192960 0x7fa7e4192d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:24.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.876 7fa7e923e700 1 -- 172.21.2.101:0/1524379992 >> 172.21.2.101:0/1524379992 conn(0x7fa7e40891f0 msgr2=0x7fa7e41a0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:24.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.876 7fa7e923e700 1 -- 172.21.2.101:0/1524379992 shutdown_connections 2021-02-10T11:15:24.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:24.876 7fa7e923e700 1 -- 172.21.2.101:0/1524379992 wait complete. 2021-02-10T11:15:25.090 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:14:57.130312', 'modified': '2021-02-10 11:14:59.506936', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4373}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4373': {'gid': 4373, 'name': 'b', 'rank': 0, 'incarnation': 10, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.2.108:6833/3230549207', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 3230549207}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 3230549207}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:15:25.091 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:15:25.091 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:15:25.488 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:15:25.488 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:15:25.488 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:15:25.488 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 4373, 2021-02-10T11:15:25.489 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:15:25.489 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:15:25.489 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 28.351723903, 2021-02-10T11:15:25.489 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 13, 2021-02-10T11:15:25.489 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 29, 2021-02-10T11:15:25.489 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 29, 2021-02-10T11:15:25.489 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 46.776810040999997 2021-02-10T11:15:25.489 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:15:25.492 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 4373, "want_state": "up:active", "state": "up:active", "rank_uptime": 28.351723903, "mdsmap_epoch": 13, "osdmap_epoch": 29, "osdmap_epoch_barrier": 29, "uptime": 46.776810040999997 } 2021-02-10T11:15:25.492 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:15:25.493 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-02-10T11:15:25.493 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:15:25.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:25.493 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:15:25.828 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:15:26.172 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:15:26.173 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:15:26.514 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:15:26.515 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:15:26.515 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:15:26.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:26.516 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:15:26.848 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:15:26.849 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2021-02-10T11:15:26.849 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:15:26.852 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:15:27.198 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:15:27.198 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:15:27.262 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[34770]: starting ceph client 2021-02-10T11:15:27.262 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:2021-02-10 11:15:27.257 7f483cec0f80 -1 init, newargv = 0x55a5ee2ff100 newargc=9 2021-02-10T11:15:27.267 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[34770]: starting fuse 2021-02-10T11:15:27.543 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:15:27.543 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:15:27.544 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:15:27.544 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:27.544 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:15:27.877 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:15:27.878 INFO:teuthology.orchestra.run.gibba001.stdout:44 2021-02-10T11:15:27.878 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2021-02-10T11:15:27.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:27.878 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:15:27.878 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:15:27.878 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:15:27.878 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:15:27.879 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:15:27.879 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:15:27.879 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:15:27.879 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-02-10T11:15:27.879 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:15:27.879 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:15:27.879 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:15:27.879 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:15:27.879 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:15:27.879 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:15:27.879 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:15:27.879 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:15:27.879 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:15:27.879 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:15:27.879 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:15:27.880 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:15:27.880 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:15:27.880 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.1")) 2021-02-10T11:15:27.880 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:15:28.240 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.1.34770.asok 2021-02-10T11:15:28.243 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34770.asok 2021-02-10T11:15:28.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:28.243 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34770.asok status 2021-02-10T11:15:28.631 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:15:28.631 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:15:28.631 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:15:28.631 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:15:28.632 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "1", 2021-02-10T11:15:28.632 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:15:28.632 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:15:28.632 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "34770", 2021-02-10T11:15:28.632 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:15:28.632 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:15:28.632 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:15:28.632 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:15:28.632 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 5118, 2021-02-10T11:15:28.632 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:15:28.632 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:15:28.632 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:15:28.633 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 5118 2021-02-10T11:15:28.633 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:15:28.633 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:15:28.633 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:15:28.633 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:15:28.633 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 3187518039 2021-02-10T11:15:28.633 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:15:28.633 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:15:28.633 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:15:28.633 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:15:28.633 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:15:28.633 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 3187518039 2021-02-10T11:15:28.633 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:15:28.634 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.5118 172.21.2.101:0/3187518039", 2021-02-10T11:15:28.634 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/3187518039", 2021-02-10T11:15:28.634 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:15:28.634 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 13, 2021-02-10T11:15:28.634 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 29, 2021-02-10T11:15:28.634 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:15:28.634 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:15:28.634 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:15:28.635 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:28.635 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:28.635 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:15:28.969 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:28.970 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:15:28.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:28.970 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:15:29.315 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2021-02-10T11:15:29.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.735 7f1434f11700 1 Processor -- start 2021-02-10T11:15:29.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.735 7f1434f11700 1 -- start start 2021-02-10T11:15:29.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.735 7f1434f11700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14300d1c70 0x7f14300cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:29.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.735 7f1434f11700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f14300cf580 0x7f14300cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:29.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.736 7f1434f11700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14300fe1c0 con 0x7f14300d1870 2021-02-10T11:15:29.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.736 7f1434f11700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14301b0cb0 con 0x7f14300cf580 2021-02-10T11:15:29.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.736 7f1434f11700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1430109a60 con 0x7f14300d1c70 2021-02-10T11:15:29.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.736 7f142eb13700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f14300d1870 0x7f14301b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46108/0 (socket says 172.21.2.101:46108) 2021-02-10T11:15:29.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.736 7f142f314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f14300cf580 0x7f14300cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:29.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.736 7f142eb13700 1 -- 172.21.2.101:0/1203168352 learned_addr learned my addr 172.21.2.101:0/1203168352 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:15:29.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.736 7f142f314700 1 -- 172.21.2.101:0/1203168352 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14300d1c70 msgr2=0x7f14300cef60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:15:29.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.736 7f142f314700 1 --2- 172.21.2.101:0/1203168352 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14300d1c70 0x7f14300cef60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:29.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.736 7f142f314700 1 -- 172.21.2.101:0/1203168352 >> v1:172.21.2.101:6789/0 conn(0x7f14300d1870 legacy=0x7f14301b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:29.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.736 7f142f314700 1 -- 172.21.2.101:0/1203168352 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1430088020 con 0x7f14300cf580 2021-02-10T11:15:29.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.736 7f142f314700 1 --2- 172.21.2.101:0/1203168352 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14300cf580 0x7f14300cf980 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f1428006e30 tx=0x7f1428009e60).ready entity=mon.0 client_cookie=9e47a27b68dc9819 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:29.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.736 7f142db11700 1 -- 172.21.2.101:0/1203168352 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1428006a50 con 0x7f14300cf580 2021-02-10T11:15:29.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.736 7f142db11700 1 -- 172.21.2.101:0/1203168352 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142800b300 con 0x7f14300cf580 2021-02-10T11:15:29.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.736 7f142db11700 1 -- 172.21.2.101:0/1203168352 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f142800b760 con 0x7f14300cf580 2021-02-10T11:15:29.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.736 7f1434f11700 1 -- 172.21.2.101:0/1203168352 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14300cf580 msgr2=0x7f14300cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:29.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.736 7f1434f11700 1 --2- 172.21.2.101:0/1203168352 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14300cf580 0x7f14300cf980 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f1428006e30 tx=0x7f1428009e60).stop 2021-02-10T11:15:29.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.737 7f1434f11700 1 -- 172.21.2.101:0/1203168352 shutdown_connections 2021-02-10T11:15:29.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.737 7f1434f11700 1 --2- 172.21.2.101:0/1203168352 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14300d1c70 0x7f14300cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:29.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.737 7f1434f11700 1 --2- 172.21.2.101:0/1203168352 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14300cf580 0x7f14300cf980 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:29.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.737 7f1434f11700 1 -- 172.21.2.101:0/1203168352 >> 172.21.2.101:0/1203168352 conn(0x7f14300ce420 msgr2=0x7f14300d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:29.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.737 7f1434f11700 1 -- 172.21.2.101:0/1203168352 shutdown_connections 2021-02-10T11:15:29.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.737 7f1434f11700 1 -- 172.21.2.101:0/1203168352 wait complete. 2021-02-10T11:15:29.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.752 7f1434f11700 1 Processor -- start 2021-02-10T11:15:29.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.752 7f1434f11700 1 -- start start 2021-02-10T11:15:29.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.752 7f1434f11700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14300b4d40 0x7f14301a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:29.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.752 7f1434f11700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14301a5b00 0x7f1430082b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:29.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.752 7f1434f11700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1430083130 0x7f14301a6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:29.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.752 7f1434f11700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14301a6160 con 0x7f1430083130 2021-02-10T11:15:29.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.752 7f1434f11700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1430083930 con 0x7f14301a5b00 2021-02-10T11:15:29.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.752 7f1434f11700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1430083530 con 0x7f14300b4d40 2021-02-10T11:15:29.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.752 7f142f314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1430083130 0x7f14301a6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:29.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.752 7f142f314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1430083130 0x7f14301a6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45622/0 (socket says 172.21.2.101:45622) 2021-02-10T11:15:29.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.752 7f142f314700 1 -- 172.21.2.101:0/1051838655 learned_addr learned my addr 172.21.2.101:0/1051838655 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:29.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.753 7f142f314700 1 -- 172.21.2.101:0/1051838655 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14300b4d40 msgr2=0x7f14301a54e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:29.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.753 7f142f314700 1 --2- 172.21.2.101:0/1051838655 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14300b4d40 0x7f14301a54e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:29.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.753 7f142f314700 1 -- 172.21.2.101:0/1051838655 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14301a5b00 msgr2=0x7f1430082b10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:29.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.753 7f142f314700 1 --2- 172.21.2.101:0/1051838655 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14301a5b00 0x7f1430082b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:29.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.753 7f142f314700 1 -- 172.21.2.101:0/1051838655 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14301a7170 con 0x7f1430083130 2021-02-10T11:15:29.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.753 7f142f314700 1 --2- 172.21.2.101:0/1051838655 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1430083130 0x7f14301a6b50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f142800b970 tx=0x7f142800bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:29.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.753 7f14277fe700 1 -- 172.21.2.101:0/1051838655 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1428009380 con 0x7f1430083130 2021-02-10T11:15:29.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.753 7f14277fe700 1 -- 172.21.2.101:0/1051838655 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1428005820 con 0x7f1430083130 2021-02-10T11:15:29.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.753 7f14277fe700 1 -- 172.21.2.101:0/1051838655 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1428005d40 con 0x7f1430083130 2021-02-10T11:15:29.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.753 7f1434f11700 1 -- 172.21.2.101:0/1051838655 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14301a7170 con 0x7f1430083130 2021-02-10T11:15:29.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.753 7f1434f11700 1 -- 172.21.2.101:0/1051838655 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14301a7700 con 0x7f1430083130 2021-02-10T11:15:29.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.754 7f14277fe700 1 -- 172.21.2.101:0/1051838655 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1428005cd0 con 0x7f1430083130 2021-02-10T11:15:29.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.754 7f14277fe700 1 --2- 172.21.2.101:0/1051838655 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14140186c0 0x7f141401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:29.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.754 7f14277fe700 1 -- 172.21.2.101:0/1051838655 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1428032770 con 0x7f1430083130 2021-02-10T11:15:29.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.754 7f142eb13700 1 --2- 172.21.2.101:0/1051838655 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14140186c0 0x7f141401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:29.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.754 7f142eb13700 1 --2- 172.21.2.101:0/1051838655 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14140186c0 0x7f141401abc0 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:29.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.785 7f1434f11700 1 -- 172.21.2.101:0/1051838655 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14300fe1c0 con 0x7f1430083130 2021-02-10T11:15:29.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.788 7f14277fe700 1 -- 172.21.2.101:0/1051838655 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1428005ac0 con 0x7f1430083130 2021-02-10T11:15:29.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:29.926 7f1434f11700 1 -- 172.21.2.101:0/1051838655 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f14300fe1c0 con 0x7f1430083130 2021-02-10T11:15:30.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:30.881 7f14277fe700 1 -- 172.21.2.101:0/1051838655 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v14) v1 ==== 136+0+0 (secure 0 0 0) 0x7f1428005cd0 con 0x7f1430083130 2021-02-10T11:15:30.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:30.900 7f1434f11700 1 -- 172.21.2.101:0/1051838655 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14140186c0 msgr2=0x7f141401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:30.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:30.900 7f1434f11700 1 --2- 172.21.2.101:0/1051838655 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14140186c0 0x7f141401abc0 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:30.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:30.900 7f1434f11700 1 -- 172.21.2.101:0/1051838655 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1430083130 msgr2=0x7f14301a6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:30.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:30.900 7f1434f11700 1 --2- 172.21.2.101:0/1051838655 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1430083130 0x7f14301a6b50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f142800b970 tx=0x7f142800bd10).stop 2021-02-10T11:15:30.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:30.900 7f1434f11700 1 -- 172.21.2.101:0/1051838655 shutdown_connections 2021-02-10T11:15:30.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:30.900 7f1434f11700 1 --2- 172.21.2.101:0/1051838655 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1430083130 0x7f14301a6b50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:30.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:30.900 7f1434f11700 1 --2- 172.21.2.101:0/1051838655 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14301a5b00 0x7f1430082b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:30.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:30.900 7f1434f11700 1 --2- 172.21.2.101:0/1051838655 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14140186c0 0x7f141401abc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:30.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:30.900 7f1434f11700 1 --2- 172.21.2.101:0/1051838655 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14300b4d40 0x7f14301a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:30.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:30.900 7f1434f11700 1 -- 172.21.2.101:0/1051838655 >> 172.21.2.101:0/1051838655 conn(0x7f143019d4f0 msgr2=0x7f14301aee10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:30.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:30.900 7f1434f11700 1 -- 172.21.2.101:0/1051838655 shutdown_connections 2021-02-10T11:15:30.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:30.900 7f1434f11700 1 -- 172.21.2.101:0/1051838655 wait complete. 2021-02-10T11:15:30.913 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs volume ls 2021-02-10T11:15:31.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.317 7fd03a92a700 1 Processor -- start 2021-02-10T11:15:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.317 7fd03a92a700 1 -- start start 2021-02-10T11:15:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.317 7fd03a92a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd0340e5070 0x7fd0341b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.317 7fd03a92a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0341b3e40 0x7fd03407d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.317 7fd03a92a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd0340e5470 0x7fd03407dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.317 7fd03a92a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03407ed80 con 0x7fd0340e5070 2021-02-10T11:15:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.317 7fd03a92a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03407e2d0 con 0x7fd0340e5470 2021-02-10T11:15:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.317 7fd03a92a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03407e4c0 con 0x7fd0341b3e40 2021-02-10T11:15:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.318 7fd033fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd0340e5070 0x7fd0341b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.318 7fd033fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd0340e5070 0x7fd0341b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45628/0 (socket says 172.21.2.101:45628) 2021-02-10T11:15:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.318 7fd033fff700 1 -- 172.21.2.101:0/2127343337 learned_addr learned my addr 172.21.2.101:0/2127343337 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.318 7fd0337fe700 1 --2- 172.21.2.101:0/2127343337 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0341b3e40 0x7fd03407d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.318 7fd038f47700 1 --2- 172.21.2.101:0/2127343337 >> v2:172.21.2.108:3300/0 conn(0x7fd0340e5470 0x7fd03407dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.318 7fd033fff700 1 -- 172.21.2.101:0/2127343337 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0341b3e40 msgr2=0x7fd03407d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.318 7fd033fff700 1 --2- 172.21.2.101:0/2127343337 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0341b3e40 0x7fd03407d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.318 7fd033fff700 1 -- 172.21.2.101:0/2127343337 >> v2:172.21.2.108:3300/0 conn(0x7fd0340e5470 msgr2=0x7fd03407dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.318 7fd033fff700 1 --2- 172.21.2.101:0/2127343337 >> v2:172.21.2.108:3300/0 conn(0x7fd0340e5470 0x7fd03407dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.318 7fd033fff700 1 -- 172.21.2.101:0/2127343337 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03407e6e0 con 0x7fd0340e5070 2021-02-10T11:15:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.318 7fd033fff700 1 --2- 172.21.2.101:0/2127343337 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0340e5070 0x7fd0341b3820 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7fd024006a80 tx=0x7fd024009f00).ready entity=mon.0 client_cookie=c35cd9b2fc40dfa0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.319 7fd032ffd700 1 -- 172.21.2.101:0/2127343337 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd02400b5c0 con 0x7fd0340e5070 2021-02-10T11:15:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.319 7fd032ffd700 1 -- 172.21.2.101:0/2127343337 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd02400b5c0 con 0x7fd0340e5070 2021-02-10T11:15:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.319 7fd032ffd700 1 -- 172.21.2.101:0/2127343337 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd02400b8c0 con 0x7fd0340e5070 2021-02-10T11:15:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.319 7fd03a92a700 1 -- 172.21.2.101:0/2127343337 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0340e5070 msgr2=0x7fd0341b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.319 7fd03a92a700 1 --2- 172.21.2.101:0/2127343337 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0340e5070 0x7fd0341b3820 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7fd024006a80 tx=0x7fd024009f00).stop 2021-02-10T11:15:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.319 7fd03a92a700 1 -- 172.21.2.101:0/2127343337 shutdown_connections 2021-02-10T11:15:31.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.319 7fd03a92a700 1 --2- 172.21.2.101:0/2127343337 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0341b3e40 0x7fd03407d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:31.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.319 7fd03a92a700 1 --2- 172.21.2.101:0/2127343337 >> v2:172.21.2.108:3300/0 conn(0x7fd0340e5470 0x7fd03407dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:31.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.319 7fd03a92a700 1 --2- 172.21.2.101:0/2127343337 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0340e5070 0x7fd0341b3820 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:31.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.319 7fd03a92a700 1 -- 172.21.2.101:0/2127343337 >> 172.21.2.101:0/2127343337 conn(0x7fd0340ce690 msgr2=0x7fd0340e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:31.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.319 7fd03a92a700 1 -- 172.21.2.101:0/2127343337 shutdown_connections 2021-02-10T11:15:31.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.319 7fd03a92a700 1 -- 172.21.2.101:0/2127343337 wait complete. 2021-02-10T11:15:31.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.334 7fd03a92a700 1 Processor -- start 2021-02-10T11:15:31.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.334 7fd03a92a700 1 -- start start 2021-02-10T11:15:31.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.334 7fd03a92a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd034093780 0x7fd034093b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:31.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.334 7fd03a92a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0340941a0 0x7fd0340945a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:31.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.334 7fd03a92a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd034091c60 0x7fd034092060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:31.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.334 7fd03a92a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd034093280 con 0x7fd034091c60 2021-02-10T11:15:31.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.334 7fd03a92a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd034089a10 con 0x7fd0340941a0 2021-02-10T11:15:31.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.334 7fd03a92a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd034089c30 con 0x7fd034093780 2021-02-10T11:15:31.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.334 7fd038f47700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd034091c60 0x7fd034092060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:31.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.334 7fd038f47700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd034091c60 0x7fd034092060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45634/0 (socket says 172.21.2.101:45634) 2021-02-10T11:15:31.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.334 7fd038f47700 1 -- 172.21.2.101:0/133426403 learned_addr learned my addr 172.21.2.101:0/133426403 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:31.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.335 7fd038f47700 1 -- 172.21.2.101:0/133426403 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd034093780 msgr2=0x7fd034093b80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:31.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.335 7fd038f47700 1 --2- 172.21.2.101:0/133426403 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd034093780 0x7fd034093b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:31.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.335 7fd038f47700 1 -- 172.21.2.101:0/133426403 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0340941a0 msgr2=0x7fd0340945a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:31.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.335 7fd038f47700 1 --2- 172.21.2.101:0/133426403 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0340941a0 0x7fd0340945a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:31.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.335 7fd038f47700 1 -- 172.21.2.101:0/133426403 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0340926b0 con 0x7fd034091c60 2021-02-10T11:15:31.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.335 7fd038f47700 1 --2- 172.21.2.101:0/133426403 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd034091c60 0x7fd034092060 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fd02c006f20 tx=0x7fd02c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:31.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.335 7fd030ff9700 1 -- 172.21.2.101:0/133426403 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd02c006720 con 0x7fd034091c60 2021-02-10T11:15:31.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.335 7fd030ff9700 1 -- 172.21.2.101:0/133426403 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd02c00be10 con 0x7fd034091c60 2021-02-10T11:15:31.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.335 7fd030ff9700 1 -- 172.21.2.101:0/133426403 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd02c005330 con 0x7fd034091c60 2021-02-10T11:15:31.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.335 7fd03a92a700 1 -- 172.21.2.101:0/133426403 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0340926b0 con 0x7fd034091c60 2021-02-10T11:15:31.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.335 7fd03a92a700 1 -- 172.21.2.101:0/133426403 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd034092c40 con 0x7fd034091c60 2021-02-10T11:15:31.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.336 7fd030ff9700 1 -- 172.21.2.101:0/133426403 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd02c006640 con 0x7fd034091c60 2021-02-10T11:15:31.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.336 7fd030ff9700 1 --2- 172.21.2.101:0/133426403 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd0140186c0 0x7fd01401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:31.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.336 7fd030ff9700 1 -- 172.21.2.101:0/133426403 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd02c0325f0 con 0x7fd034091c60 2021-02-10T11:15:31.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.336 7fd033fff700 1 --2- 172.21.2.101:0/133426403 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd0140186c0 0x7fd01401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:31.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.337 7fd033fff700 1 --2- 172.21.2.101:0/133426403 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd0140186c0 0x7fd01401abc0 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:31.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.367 7fd03a92a700 1 -- 172.21.2.101:0/133426403 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd034084ab0 con 0x7fd034091c60 2021-02-10T11:15:31.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.370 7fd030ff9700 1 -- 172.21.2.101:0/133426403 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd02c014890 con 0x7fd034091c60 2021-02-10T11:15:31.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.505 7fd03a92a700 1 -- 172.21.2.101:0/133426403 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs volume ls", "target": ["mgr", ""]}) v1 -- 0x7fd03407ed80 con 0x7fd0140186c0 2021-02-10T11:15:31.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.506 7fd030ff9700 1 -- 172.21.2.101:0/133426403 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7fd024009430 con 0x7fd0140186c0 2021-02-10T11:15:31.511 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:15:31.511 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:15:31.511 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "cephfs" 2021-02-10T11:15:31.511 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:15:31.511 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:15:31.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.507 7fd03a92a700 1 -- 172.21.2.101:0/133426403 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd0140186c0 msgr2=0x7fd01401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:31.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.507 7fd03a92a700 1 --2- 172.21.2.101:0/133426403 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd0140186c0 0x7fd01401abc0 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:31.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.507 7fd03a92a700 1 -- 172.21.2.101:0/133426403 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd034091c60 msgr2=0x7fd034092060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:31.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.507 7fd03a92a700 1 --2- 172.21.2.101:0/133426403 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd034091c60 0x7fd034092060 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fd02c006f20 tx=0x7fd02c00b990).stop 2021-02-10T11:15:31.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.507 7fd03a92a700 1 -- 172.21.2.101:0/133426403 shutdown_connections 2021-02-10T11:15:31.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.507 7fd03a92a700 1 --2- 172.21.2.101:0/133426403 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd034091c60 0x7fd034092060 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:31.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.507 7fd03a92a700 1 --2- 172.21.2.101:0/133426403 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0340941a0 0x7fd0340945a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:31.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.507 7fd03a92a700 1 --2- 172.21.2.101:0/133426403 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd0140186c0 0x7fd01401abc0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:31.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.507 7fd03a92a700 1 --2- 172.21.2.101:0/133426403 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd034093780 0x7fd034093b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:31.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.507 7fd03a92a700 1 -- 172.21.2.101:0/133426403 >> 172.21.2.101:0/133426403 conn(0x7fd0340b5560 msgr2=0x7fd0340a6200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:31.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.508 7fd03a92a700 1 -- 172.21.2.101:0/133426403 shutdown_connections 2021-02-10T11:15:31.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.508 7fd03a92a700 1 -- 172.21.2.101:0/133426403 wait complete. 2021-02-10T11:15:31.522 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:15:31.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.932 7f756b4f9700 1 Processor -- start 2021-02-10T11:15:31.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.932 7f756b4f9700 1 -- start start 2021-02-10T11:15:31.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.933 7f756b4f9700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f75641b0470 0x7f75641b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:31.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.933 7f756b4f9700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75640f0b50 con 0x7f75640f20e0 2021-02-10T11:15:31.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.933 7f756b4f9700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75640f24e0 con 0x7f75640ef610 2021-02-10T11:15:31.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.933 7f756b4f9700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75640e5ea0 con 0x7f75641b0470 2021-02-10T11:15:31.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.933 7f7569b16700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f75640f20e0 0x7f75640f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46136/0 (socket says 172.21.2.101:46136) 2021-02-10T11:15:31.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.933 7f7569b16700 1 -- 172.21.2.101:0/1037559732 learned_addr learned my addr 172.21.2.101:0/1037559732 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:15:31.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.933 7f7569315700 1 --2- 172.21.2.101:0/1037559732 >> v2:172.21.2.108:3300/0 conn(0x7f75641b0470 0x7f75641b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:31.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.933 7f7557fff700 1 -- 172.21.2.101:0/1037559732 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 139016061 0 0) 0x7f755c000f90 con 0x7f75640f20e0 2021-02-10T11:15:31.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.933 7f7557fff700 1 -- 172.21.2.101:0/1037559732 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7550002fe0 con 0x7f75640f20e0 2021-02-10T11:15:31.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.933 7f7569315700 1 -- 172.21.2.101:0/1037559732 >> v1:172.21.2.108:6789/0 conn(0x7f75640ef610 legacy=0x7f75640efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:31.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.933 7f7569315700 1 -- 172.21.2.101:0/1037559732 >> v1:172.21.2.101:6789/0 conn(0x7f75640f20e0 legacy=0x7f75640f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:31.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.933 7f7569315700 1 -- 172.21.2.101:0/1037559732 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75640e52e0 con 0x7f75641b0470 2021-02-10T11:15:31.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.933 7f7569315700 1 --2- 172.21.2.101:0/1037559732 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75641b0470 0x7f75641b0870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f7558006650 tx=0x7f7558009a80).ready entity=mon.1 client_cookie=d411327819cf68b4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:31.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.934 7f7557fff700 1 -- 172.21.2.101:0/1037559732 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f755800c0c0 con 0x7f75641b0470 2021-02-10T11:15:31.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.934 7f7557fff700 1 -- 172.21.2.101:0/1037559732 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755800c340 con 0x7f75641b0470 2021-02-10T11:15:31.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.934 7f7557fff700 1 -- 172.21.2.101:0/1037559732 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f755800c860 con 0x7f75641b0470 2021-02-10T11:15:31.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.934 7f756b4f9700 1 -- 172.21.2.101:0/1037559732 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75641b0470 msgr2=0x7f75641b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:31.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.934 7f756b4f9700 1 --2- 172.21.2.101:0/1037559732 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75641b0470 0x7f75641b0870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f7558006650 tx=0x7f7558009a80).stop 2021-02-10T11:15:31.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.934 7f756b4f9700 1 -- 172.21.2.101:0/1037559732 shutdown_connections 2021-02-10T11:15:31.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.934 7f756b4f9700 1 --2- 172.21.2.101:0/1037559732 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75641b0470 0x7f75641b0870 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:31.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.934 7f756b4f9700 1 -- 172.21.2.101:0/1037559732 >> 172.21.2.101:0/1037559732 conn(0x7f75640ce3e0 msgr2=0x7f75640e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:31.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.935 7f756b4f9700 1 -- 172.21.2.101:0/1037559732 shutdown_connections 2021-02-10T11:15:31.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.935 7f756b4f9700 1 -- 172.21.2.101:0/1037559732 wait complete. 2021-02-10T11:15:31.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.949 7f756b4f9700 1 Processor -- start 2021-02-10T11:15:31.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.949 7f756b4f9700 1 -- start start 2021-02-10T11:15:31.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.949 7f756b4f9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75640b2840 0x7f75641a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:31.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.949 7f756b4f9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75640b30d0 0x7f75640b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:31.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.949 7f756b4f9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75640b3af0 0x7f75640f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:31.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.949 7f756b4f9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75640b2c40 con 0x7f75640b30d0 2021-02-10T11:15:31.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.949 7f756b4f9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75641a4920 con 0x7f75640b2840 2021-02-10T11:15:31.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.949 7f756b4f9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75640f0ba0 con 0x7f75640b3af0 2021-02-10T11:15:31.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.950 7f7568b14700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75640b30d0 0x7f75640b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:31.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.950 7f7568b14700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75640b30d0 0x7f75640b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45650/0 (socket says 172.21.2.101:45650) 2021-02-10T11:15:31.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.950 7f7568b14700 1 -- 172.21.2.101:0/559145687 learned_addr learned my addr 172.21.2.101:0/559145687 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:31.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.950 7f7568b14700 1 -- 172.21.2.101:0/559145687 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75640b3af0 msgr2=0x7f75640f0580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:31.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.950 7f7568b14700 1 --2- 172.21.2.101:0/559145687 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75640b3af0 0x7f75640f0580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:31.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.950 7f7568b14700 1 -- 172.21.2.101:0/559145687 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75640b2840 msgr2=0x7f75641a4300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:31.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.950 7f7568b14700 1 --2- 172.21.2.101:0/559145687 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75640b2840 0x7f75641a4300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:31.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.950 7f7568b14700 1 -- 172.21.2.101:0/559145687 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75640f0dc0 con 0x7f75640b30d0 2021-02-10T11:15:31.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.950 7f7568b14700 1 --2- 172.21.2.101:0/559145687 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75640b30d0 0x7f75640b34d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f7560006ed0 tx=0x7f756000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:31.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.950 7f7555ffb700 1 -- 172.21.2.101:0/559145687 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7560006020 con 0x7f75640b30d0 2021-02-10T11:15:31.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.950 7f7555ffb700 1 -- 172.21.2.101:0/559145687 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75600063a0 con 0x7f75640b30d0 2021-02-10T11:15:31.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.950 7f7555ffb700 1 -- 172.21.2.101:0/559145687 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7560005730 con 0x7f75640b30d0 2021-02-10T11:15:31.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.950 7f756b4f9700 1 -- 172.21.2.101:0/559145687 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75640f0dc0 con 0x7f75640b30d0 2021-02-10T11:15:31.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.950 7f756b4f9700 1 -- 172.21.2.101:0/559145687 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75640f1350 con 0x7f75640b30d0 2021-02-10T11:15:31.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.951 7f7555ffb700 1 -- 172.21.2.101:0/559145687 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7560006020 con 0x7f75640b30d0 2021-02-10T11:15:31.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.951 7f7555ffb700 1 --2- 172.21.2.101:0/559145687 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75440186c0 0x7f754401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:31.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.951 7f7555ffb700 1 -- 172.21.2.101:0/559145687 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f75600323d0 con 0x7f75640b30d0 2021-02-10T11:15:31.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.951 7f7569315700 1 --2- 172.21.2.101:0/559145687 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75440186c0 0x7f754401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:31.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.951 7f7569315700 1 --2- 172.21.2.101:0/559145687 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75440186c0 0x7f754401abc0 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:31.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.982 7f756b4f9700 1 -- 172.21.2.101:0/559145687 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75640f18f0 con 0x7f75640b30d0 2021-02-10T11:15:31.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:31.985 7f7555ffb700 1 -- 172.21.2.101:0/559145687 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7560030d00 con 0x7f75640b30d0 2021-02-10T11:15:32.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.132 7f756b4f9700 1 -- 172.21.2.101:0/559145687 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f75640f0050 con 0x7f75640b30d0 2021-02-10T11:15:32.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.133 7f7555ffb700 1 -- 172.21.2.101:0/559145687 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7560006020 con 0x7f75640b30d0 2021-02-10T11:15:32.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.135 7f756b4f9700 1 -- 172.21.2.101:0/559145687 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75440186c0 msgr2=0x7f754401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:32.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.135 7f756b4f9700 1 --2- 172.21.2.101:0/559145687 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75440186c0 0x7f754401abc0 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:32.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.135 7f756b4f9700 1 -- 172.21.2.101:0/559145687 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75640b30d0 msgr2=0x7f75640b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:32.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.135 7f756b4f9700 1 --2- 172.21.2.101:0/559145687 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75640b30d0 0x7f75640b34d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f7560006ed0 tx=0x7f756000b960).stop 2021-02-10T11:15:32.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.135 7f756b4f9700 1 -- 172.21.2.101:0/559145687 shutdown_connections 2021-02-10T11:15:32.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.135 7f756b4f9700 1 --2- 172.21.2.101:0/559145687 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75440186c0 0x7f754401abc0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:32.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.135 7f756b4f9700 1 --2- 172.21.2.101:0/559145687 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75640b3af0 0x7f75640f0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:32.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.135 7f756b4f9700 1 --2- 172.21.2.101:0/559145687 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75640b30d0 0x7f75640b34d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:32.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.135 7f756b4f9700 1 --2- 172.21.2.101:0/559145687 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75640b2840 0x7f75641a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:32.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.135 7f756b4f9700 1 -- 172.21.2.101:0/559145687 >> 172.21.2.101:0/559145687 conn(0x7f75640c36f0 msgr2=0x7f7564094c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:32.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.135 7f756b4f9700 1 -- 172.21.2.101:0/559145687 shutdown_connections 2021-02-10T11:15:32.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.135 7f756b4f9700 1 -- 172.21.2.101:0/559145687 wait complete. 2021-02-10T11:15:32.148 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839239 2021-02-10T11:15:32.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.557 7fdd968b6700 1 Processor -- start 2021-02-10T11:15:32.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.557 7fdd968b6700 1 -- start start 2021-02-10T11:15:32.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.557 7fdd968b6700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fdd901b0470 0x7fdd901b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:32.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.557 7fdd968b6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd900ef610 0x7fdd900efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:32.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.557 7fdd968b6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdd900f0030 0x7fdd900f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:32.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.557 7fdd968b6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd900e5ea0 con 0x7fdd900f0030 2021-02-10T11:15:32.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.557 7fdd968b6700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd900f0b50 con 0x7fdd901b0470 2021-02-10T11:15:32.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.557 7fdd968b6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd900e52e0 con 0x7fdd900ef610 2021-02-10T11:15:32.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.558 7fdd94ed3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdd900f0030 0x7fdd900f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:32.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.558 7fdd94ed3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdd900f0030 0x7fdd900f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45658/0 (socket says 172.21.2.101:45658) 2021-02-10T11:15:32.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.558 7fdd94ed3700 1 -- 172.21.2.101:0/3792631298 learned_addr learned my addr 172.21.2.101:0/3792631298 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:32.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.558 7fdd94ed3700 1 -- 172.21.2.101:0/3792631298 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd900ef610 msgr2=0x7fdd900efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:15:32.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.558 7fdd94ed3700 1 --2- 172.21.2.101:0/3792631298 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd900ef610 0x7fdd900efa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:32.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.558 7fdd94ed3700 1 -- 172.21.2.101:0/3792631298 >> v2:172.21.2.108:3300/0 conn(0x7fdd901b0470 msgr2=0x7fdd901b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:15:32.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.558 7fdd94ed3700 1 --2- 172.21.2.101:0/3792631298 >> v2:172.21.2.108:3300/0 conn(0x7fdd901b0470 0x7fdd901b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:32.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.558 7fdd94ed3700 1 -- 172.21.2.101:0/3792631298 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd900f10d0 con 0x7fdd900f0030 2021-02-10T11:15:32.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.558 7fdd94ed3700 1 --2- 172.21.2.101:0/3792631298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd900f0030 0x7fdd900f0430 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7fdd88006f00 tx=0x7fdd88009f10).ready entity=mon.0 client_cookie=706157324bc5519 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:32.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.558 7fdd8effd700 1 -- 172.21.2.101:0/3792631298 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd8800b3b0 con 0x7fdd900f0030 2021-02-10T11:15:32.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.558 7fdd8effd700 1 -- 172.21.2.101:0/3792631298 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd88000d30 con 0x7fdd900f0030 2021-02-10T11:15:32.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.558 7fdd8effd700 1 -- 172.21.2.101:0/3792631298 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd8800b9b0 con 0x7fdd900f0030 2021-02-10T11:15:32.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.558 7fdd968b6700 1 -- 172.21.2.101:0/3792631298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd900f0030 msgr2=0x7fdd900f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:32.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.558 7fdd968b6700 1 --2- 172.21.2.101:0/3792631298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd900f0030 0x7fdd900f0430 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7fdd88006f00 tx=0x7fdd88009f10).stop 2021-02-10T11:15:32.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.559 7fdd968b6700 1 -- 172.21.2.101:0/3792631298 shutdown_connections 2021-02-10T11:15:32.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.559 7fdd968b6700 1 --2- 172.21.2.101:0/3792631298 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd900ef610 0x7fdd900efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:32.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.559 7fdd968b6700 1 --2- 172.21.2.101:0/3792631298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd900f0030 0x7fdd900f0430 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:32.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.559 7fdd968b6700 1 --2- 172.21.2.101:0/3792631298 >> v2:172.21.2.108:3300/0 conn(0x7fdd901b0470 0x7fdd901b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:32.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.559 7fdd968b6700 1 -- 172.21.2.101:0/3792631298 >> 172.21.2.101:0/3792631298 conn(0x7fdd900ce3e0 msgr2=0x7fdd900e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:32.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.559 7fdd968b6700 1 -- 172.21.2.101:0/3792631298 shutdown_connections 2021-02-10T11:15:32.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.559 7fdd968b6700 1 -- 172.21.2.101:0/3792631298 wait complete. 2021-02-10T11:15:32.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.573 7fdd968b6700 1 Processor -- start 2021-02-10T11:15:32.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.574 7fdd968b6700 1 -- start start 2021-02-10T11:15:32.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.574 7fdd968b6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd900bf910 0x7fdd900bfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:32.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.574 7fdd968b6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd900c0330 0x7fdd900c0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:32.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.574 7fdd968b6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd900c13d0 0x7fdd900c17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:32.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.574 7fdd968b6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd90109930 con 0x7fdd900c0330 2021-02-10T11:15:32.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.574 7fdd968b6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd90109b50 con 0x7fdd900c13d0 2021-02-10T11:15:32.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.574 7fdd968b6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd900c1df0 con 0x7fdd900bf910 2021-02-10T11:15:32.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.574 7fdd8f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd900c0330 0x7fdd900c0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:32.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.574 7fdd8f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd900c0330 0x7fdd900c0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45662/0 (socket says 172.21.2.101:45662) 2021-02-10T11:15:32.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.574 7fdd8f7fe700 1 -- 172.21.2.101:0/3490608209 learned_addr learned my addr 172.21.2.101:0/3490608209 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:32.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.575 7fdd8f7fe700 1 -- 172.21.2.101:0/3490608209 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd900bf910 msgr2=0x7fdd900bfd10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:32.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.575 7fdd8f7fe700 1 --2- 172.21.2.101:0/3490608209 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd900bf910 0x7fdd900bfd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:32.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.575 7fdd8f7fe700 1 -- 172.21.2.101:0/3490608209 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd900c13d0 msgr2=0x7fdd900c17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:32.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.575 7fdd8f7fe700 1 --2- 172.21.2.101:0/3490608209 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd900c13d0 0x7fdd900c17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:32.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.575 7fdd8f7fe700 1 -- 172.21.2.101:0/3490608209 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd900c1fd0 con 0x7fdd900c0330 2021-02-10T11:15:32.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.575 7fdd8f7fe700 1 --2- 172.21.2.101:0/3490608209 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd900c0330 0x7fdd900c0730 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fdd8400b5c0 tx=0x7fdd8400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:32.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.575 7fdd8cff9700 1 -- 172.21.2.101:0/3490608209 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd8400bd50 con 0x7fdd900c0330 2021-02-10T11:15:32.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.575 7fdd8cff9700 1 -- 172.21.2.101:0/3490608209 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd84006030 con 0x7fdd900c0330 2021-02-10T11:15:32.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.575 7fdd968b6700 1 -- 172.21.2.101:0/3490608209 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd900c1fd0 con 0x7fdd900c0330 2021-02-10T11:15:32.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.575 7fdd968b6700 1 -- 172.21.2.101:0/3490608209 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd900c2560 con 0x7fdd900c0330 2021-02-10T11:15:32.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.575 7fdd8cff9700 1 -- 172.21.2.101:0/3490608209 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd840054d0 con 0x7fdd900c0330 2021-02-10T11:15:32.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.576 7fdd8cff9700 1 -- 172.21.2.101:0/3490608209 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdd84006210 con 0x7fdd900c0330 2021-02-10T11:15:32.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.576 7fdd8cff9700 1 --2- 172.21.2.101:0/3490608209 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd700186c0 0x7fdd7001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:32.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.576 7fdd8cff9700 1 -- 172.21.2.101:0/3490608209 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fdd84032800 con 0x7fdd900c0330 2021-02-10T11:15:32.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.576 7fdd8ffff700 1 --2- 172.21.2.101:0/3490608209 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd700186c0 0x7fdd7001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:32.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.576 7fdd8ffff700 1 --2- 172.21.2.101:0/3490608209 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd700186c0 0x7fdd7001abc0 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:32.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.607 7fdd968b6700 1 -- 172.21.2.101:0/3490608209 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd900f0b50 con 0x7fdd900c0330 2021-02-10T11:15:32.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.609 7fdd8cff9700 1 -- 172.21.2.101:0/3490608209 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdd840009b0 con 0x7fdd900c0330 2021-02-10T11:15:32.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.735 7fdd968b6700 1 -- 172.21.2.101:0/3490608209 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839239", "target": ["mgr", ""]}) v1 -- 0x7fdd900c2560 con 0x7fdd700186c0 2021-02-10T11:15:32.743 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:15:32.738 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:15:32.743 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:15:32.738 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:15:32.743 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:15:32.738 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:15:32.743 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:15:32.738 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:15:32.743 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:15:32.738 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:15:32.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.770 7fdd8cff9700 1 -- 172.21.2.101:0/3490608209 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fdd800009b0 con 0x7fdd700186c0 2021-02-10T11:15:32.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.800 7fdd968b6700 1 -- 172.21.2.101:0/3490608209 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd700186c0 msgr2=0x7fdd7001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:32.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.800 7fdd968b6700 1 --2- 172.21.2.101:0/3490608209 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd700186c0 0x7fdd7001abc0 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:32.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.800 7fdd968b6700 1 -- 172.21.2.101:0/3490608209 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd900c0330 msgr2=0x7fdd900c0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:32.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.800 7fdd968b6700 1 --2- 172.21.2.101:0/3490608209 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd900c0330 0x7fdd900c0730 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fdd8400b5c0 tx=0x7fdd8400b960).stop 2021-02-10T11:15:32.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.800 7fdd968b6700 1 -- 172.21.2.101:0/3490608209 shutdown_connections 2021-02-10T11:15:32.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.800 7fdd968b6700 1 --2- 172.21.2.101:0/3490608209 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd900c13d0 0x7fdd900c17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:32.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.800 7fdd968b6700 1 --2- 172.21.2.101:0/3490608209 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd900c0330 0x7fdd900c0730 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:32.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.800 7fdd968b6700 1 --2- 172.21.2.101:0/3490608209 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd700186c0 0x7fdd7001abc0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:32.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.800 7fdd968b6700 1 --2- 172.21.2.101:0/3490608209 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd900bf910 0x7fdd900bfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:32.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.800 7fdd968b6700 1 -- 172.21.2.101:0/3490608209 >> 172.21.2.101:0/3490608209 conn(0x7fdd900bf4e0 msgr2=0x7fdd90191190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:32.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.800 7fdd968b6700 1 -- 172.21.2.101:0/3490608209 shutdown_connections 2021-02-10T11:15:32.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:32.800 7fdd968b6700 1 -- 172.21.2.101:0/3490608209 wait complete. 2021-02-10T11:15:32.813 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839239 2021-02-10T11:15:33.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.220 7fbfe67f1700 1 Processor -- start 2021-02-10T11:15:33.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.220 7fbfe67f1700 1 -- start start 2021-02-10T11:15:33.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.221 7fbfe67f1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbfe00f05f0 0x7fbfe00f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:33.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.221 7fbfe67f1700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe01b0c70 con 0x7fbfe01b0470 2021-02-10T11:15:33.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.221 7fbfe67f1700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe00f0b50 con 0x7fbfe01b0870 2021-02-10T11:15:33.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.221 7fbfe67f1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe00e5ea0 con 0x7fbfe00f05f0 2021-02-10T11:15:33.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.221 7fbfdffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fbfe01b0470 0x7fbfe00ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46178/0 (socket says 172.21.2.101:46178) 2021-02-10T11:15:33.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.221 7fbfdffff700 1 -- 172.21.2.101:0/3466215771 learned_addr learned my addr 172.21.2.101:0/3466215771 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:15:33.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.221 7fbfdeffd700 1 -- 172.21.2.101:0/3466215771 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 8155855 0 0) 0x7fbfd0000ce0 con 0x7fbfe01b0470 2021-02-10T11:15:33.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.221 7fbfdeffd700 1 -- 172.21.2.101:0/3466215771 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfc8002fe0 con 0x7fbfe01b0470 2021-02-10T11:15:33.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.221 7fbfe4e0e700 1 --2- 172.21.2.101:0/3466215771 >> v2:172.21.2.108:3300/0 conn(0x7fbfe00f05f0 0x7fbfe00f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:33.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfdeffd700 1 -- 172.21.2.101:0/3466215771 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1519719555 0 0) 0x7fbfd00010e0 con 0x7fbfe01b0470 2021-02-10T11:15:33.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfdeffd700 1 -- 172.21.2.101:0/3466215771 >> v2:172.21.2.108:3300/0 conn(0x7fbfe00f05f0 msgr2=0x7fbfe00f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:33.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfdeffd700 1 --2- 172.21.2.101:0/3466215771 >> v2:172.21.2.108:3300/0 conn(0x7fbfe00f05f0 0x7fbfe00f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:33.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfdeffd700 1 -- 172.21.2.101:0/3466215771 >> v1:172.21.2.108:6789/0 conn(0x7fbfe01b0870 legacy=0x7fbfe00efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:33.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfdeffd700 1 -- 172.21.2.101:0/3466215771 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe00e52e0 con 0x7fbfe01b0470 2021-02-10T11:15:33.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfdeffd700 1 -- 172.21.2.101:0/3466215771 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fbfd0001660 con 0x7fbfe01b0470 2021-02-10T11:15:33.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfdeffd700 1 -- 172.21.2.101:0/3466215771 >> v1:172.21.2.101:6789/0 conn(0x7fbfe01b0470 legacy=0x7fbfe00ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:33.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfdeffd700 1 --2- 172.21.2.101:0/3466215771 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbfc8003a60 0x7fbfc8005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:33.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfdeffd700 1 --2- 172.21.2.101:0/3466215771 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbfc8006790 0x7fbfc8008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:33.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfdeffd700 1 --2- 172.21.2.101:0/3466215771 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbfc8009590 0x7fbfc800bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:33.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfdeffd700 1 -- 172.21.2.101:0/3466215771 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc800c300 con 0x7fbfc8006790 2021-02-10T11:15:33.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfdeffd700 1 -- 172.21.2.101:0/3466215771 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc800c520 con 0x7fbfc8009590 2021-02-10T11:15:33.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfdeffd700 1 -- 172.21.2.101:0/3466215771 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8050750 con 0x7fbfc8003a60 2021-02-10T11:15:33.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfdf7fe700 1 --2- 172.21.2.101:0/3466215771 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbfc8006790 0x7fbfc8008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:33.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfdffff700 1 --2- 172.21.2.101:0/3466215771 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbfc8003a60 0x7fbfc8005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:33.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfe4e0e700 1 --2- 172.21.2.101:0/3466215771 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbfc8009590 0x7fbfc800bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:33.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfdf7fe700 1 -- 172.21.2.101:0/3466215771 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbfc8003a60 msgr2=0x7fbfc8005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:33.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfdf7fe700 1 --2- 172.21.2.101:0/3466215771 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbfc8003a60 0x7fbfc8005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:33.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfdf7fe700 1 -- 172.21.2.101:0/3466215771 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbfc8009590 msgr2=0x7fbfc800bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:33.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfdf7fe700 1 --2- 172.21.2.101:0/3466215771 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbfc8009590 0x7fbfc800bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:33.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfdf7fe700 1 -- 172.21.2.101:0/3466215771 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc80509b0 con 0x7fbfc8006790 2021-02-10T11:15:33.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfe4e0e700 1 --2- 172.21.2.101:0/3466215771 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbfc8009590 0x7fbfc800bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:15:33.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.222 7fbfe4e0e700 1 -- 172.21.2.101:0/3466215771 reap_dead start 2021-02-10T11:15:33.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.223 7fbfdf7fe700 1 --2- 172.21.2.101:0/3466215771 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbfc8006790 0x7fbfc8008c90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fbfd4006ac0 tx=0x7fbfd4009af0).ready entity=mon.0 client_cookie=c6b7c5287b061388 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:33.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.223 7fbfdeffd700 1 -- 172.21.2.101:0/3466215771 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbfd4007280 con 0x7fbfc8006790 2021-02-10T11:15:33.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.223 7fbfdeffd700 1 -- 172.21.2.101:0/3466215771 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd4007600 con 0x7fbfc8006790 2021-02-10T11:15:33.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.223 7fbfdeffd700 1 -- 172.21.2.101:0/3466215771 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbfd4007a60 con 0x7fbfc8006790 2021-02-10T11:15:33.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.223 7fbfe67f1700 1 -- 172.21.2.101:0/3466215771 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbfc8006790 msgr2=0x7fbfc8008c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:33.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.223 7fbfe67f1700 1 --2- 172.21.2.101:0/3466215771 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbfc8006790 0x7fbfc8008c90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fbfd4006ac0 tx=0x7fbfd4009af0).stop 2021-02-10T11:15:33.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.223 7fbfe67f1700 1 -- 172.21.2.101:0/3466215771 shutdown_connections 2021-02-10T11:15:33.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.223 7fbfe67f1700 1 --2- 172.21.2.101:0/3466215771 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbfc8006790 0x7fbfc8008c90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:33.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.223 7fbfe67f1700 1 -- 172.21.2.101:0/3466215771 >> 172.21.2.101:0/3466215771 conn(0x7fbfe00ce3e0 msgr2=0x7fbfe00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:33.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.224 7fbfe67f1700 1 -- 172.21.2.101:0/3466215771 shutdown_connections 2021-02-10T11:15:33.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.224 7fbfe67f1700 1 -- 172.21.2.101:0/3466215771 wait complete. 2021-02-10T11:15:33.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.238 7fbfe67f1700 1 Processor -- start 2021-02-10T11:15:33.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.238 7fbfe67f1700 1 -- start start 2021-02-10T11:15:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.239 7fbfe67f1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbfe00b7db0 0x7fbfe00b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.239 7fbfe67f1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbfe00b8880 0x7fbfe01a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.239 7fbfe67f1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbfe00b8c80 0x7fbfe01a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.239 7fbfe67f1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe00b79b0 con 0x7fbfe00b8880 2021-02-10T11:15:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.239 7fbfe67f1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe01a8090 con 0x7fbfe00b7db0 2021-02-10T11:15:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.239 7fbfe67f1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe01a82b0 con 0x7fbfe00b8c80 2021-02-10T11:15:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.239 7fbfdf7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbfe00b8880 0x7fbfe01a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.239 7fbfdf7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbfe00b8880 0x7fbfe01a7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45696/0 (socket says 172.21.2.101:45696) 2021-02-10T11:15:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.239 7fbfdf7fe700 1 -- 172.21.2.101:0/1906766388 learned_addr learned my addr 172.21.2.101:0/1906766388 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.239 7fbfdffff700 1 --2- 172.21.2.101:0/1906766388 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbfe00b7db0 0x7fbfe00b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.239 7fbfe4e0e700 1 --2- 172.21.2.101:0/1906766388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbfe00b8c80 0x7fbfe01a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.239 7fbfdf7fe700 1 -- 172.21.2.101:0/1906766388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbfe00b8c80 msgr2=0x7fbfe01a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.239 7fbfdf7fe700 1 --2- 172.21.2.101:0/1906766388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbfe00b8c80 0x7fbfe01a7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.239 7fbfdf7fe700 1 -- 172.21.2.101:0/1906766388 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbfe00b7db0 msgr2=0x7fbfe00b8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.239 7fbfdf7fe700 1 --2- 172.21.2.101:0/1906766388 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbfe00b7db0 0x7fbfe00b8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.239 7fbfdf7fe700 1 -- 172.21.2.101:0/1906766388 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe0190c10 con 0x7fbfe00b8880 2021-02-10T11:15:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.239 7fbfdffff700 1 --2- 172.21.2.101:0/1906766388 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbfe00b7db0 0x7fbfe00b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:15:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.240 7fbfdf7fe700 1 --2- 172.21.2.101:0/1906766388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbfe00b8880 0x7fbfe01a7450 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7fbfd4009da0 tx=0x7fbfd40075e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.240 7fbfdcff9700 1 -- 172.21.2.101:0/1906766388 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbfd4001020 con 0x7fbfe00b8880 2021-02-10T11:15:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.240 7fbfdcff9700 1 -- 172.21.2.101:0/1906766388 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd40012a0 con 0x7fbfe00b8880 2021-02-10T11:15:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.240 7fbfe67f1700 1 -- 172.21.2.101:0/1906766388 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe0190c10 con 0x7fbfe00b8880 2021-02-10T11:15:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.240 7fbfe67f1700 1 -- 172.21.2.101:0/1906766388 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe00c1eb0 con 0x7fbfe00b8880 2021-02-10T11:15:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.240 7fbfdcff9700 1 -- 172.21.2.101:0/1906766388 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbfd4001730 con 0x7fbfe00b8880 2021-02-10T11:15:33.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.241 7fbfdcff9700 1 -- 172.21.2.101:0/1906766388 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbfd4001480 con 0x7fbfe00b8880 2021-02-10T11:15:33.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.241 7fbfdcff9700 1 --2- 172.21.2.101:0/1906766388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbfc00186c0 0x7fbfc001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:33.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.241 7fbfdcff9700 1 -- 172.21.2.101:0/1906766388 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fbfd4032560 con 0x7fbfe00b8880 2021-02-10T11:15:33.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.241 7fbfdffff700 1 --2- 172.21.2.101:0/1906766388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbfc00186c0 0x7fbfc001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:33.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.241 7fbfdffff700 1 --2- 172.21.2.101:0/1906766388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbfc00186c0 0x7fbfc001abc0 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:33.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.271 7fbfe67f1700 1 -- 172.21.2.101:0/1906766388 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe0105b30 con 0x7fbfe00b8880 2021-02-10T11:15:33.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.273 7fbfdcff9700 1 -- 172.21.2.101:0/1906766388 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbfd4000d00 con 0x7fbfe00b8880 2021-02-10T11:15:33.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.402 7fbfe67f1700 1 -- 172.21.2.101:0/1906766388 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839239", "target": ["mgr", ""]}) v1 -- 0x7fbfe00c1eb0 con 0x7fbfc00186c0 2021-02-10T11:15:33.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.406 7fbfdcff9700 1 -- 172.21.2.101:0/1906766388 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fbfd00012d0 con 0x7fbfc00186c0 2021-02-10T11:15:33.414 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839239/fa875cd0-c3a7-4fa6-9481-60c012a9ffa0 2021-02-10T11:15:33.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.411 7fbfe67f1700 1 -- 172.21.2.101:0/1906766388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbfc00186c0 msgr2=0x7fbfc001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:33.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.411 7fbfe67f1700 1 --2- 172.21.2.101:0/1906766388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbfc00186c0 0x7fbfc001abc0 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:33.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.411 7fbfe67f1700 1 -- 172.21.2.101:0/1906766388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbfe00b8880 msgr2=0x7fbfe01a7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:33.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.411 7fbfe67f1700 1 --2- 172.21.2.101:0/1906766388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbfe00b8880 0x7fbfe01a7450 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7fbfd4009da0 tx=0x7fbfd40075e0).stop 2021-02-10T11:15:33.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.411 7fbfe67f1700 1 -- 172.21.2.101:0/1906766388 shutdown_connections 2021-02-10T11:15:33.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.411 7fbfe67f1700 1 --2- 172.21.2.101:0/1906766388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbfc00186c0 0x7fbfc001abc0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:33.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.411 7fbfe67f1700 1 --2- 172.21.2.101:0/1906766388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbfe00b8c80 0x7fbfe01a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:33.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.411 7fbfe67f1700 1 --2- 172.21.2.101:0/1906766388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbfe00b8880 0x7fbfe01a7450 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:33.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.411 7fbfe67f1700 1 --2- 172.21.2.101:0/1906766388 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbfe00b7db0 0x7fbfe00b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:33.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.411 7fbfe67f1700 1 -- 172.21.2.101:0/1906766388 >> 172.21.2.101:0/1906766388 conn(0x7fbfe0094e10 msgr2=0x7fbfe00a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:33.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.412 7fbfe67f1700 1 -- 172.21.2.101:0/1906766388 shutdown_connections 2021-02-10T11:15:33.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:33.412 7fbfe67f1700 1 -- 172.21.2.101:0/1906766388 wait complete. 2021-02-10T11:15:33.424 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839239 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839239/fa875cd0-c3a7-4fa6-9481-60c012a9ffa0 2021-02-10T11:15:33.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:33.424 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:33.424 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:33.759 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:33.759 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:33.760 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839239/fa875cd0-c3a7-4fa6-9481-60c012a9ffa0/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:34.141 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:34.141 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:34.141 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0256195 s, 40.9 MB/s 2021-02-10T11:15:34.142 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:34.142 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:34.142 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:34.477 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:34.478 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:34.478 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839239/fa875cd0-c3a7-4fa6-9481-60c012a9ffa0/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:34.850 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:34.851 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:34.851 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024326 s, 43.1 MB/s 2021-02-10T11:15:34.852 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:34.852 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:34.852 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:35.187 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:35.188 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:35.188 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839239/fa875cd0-c3a7-4fa6-9481-60c012a9ffa0/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:35.563 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:35.563 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:35.563 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231323 s, 45.3 MB/s 2021-02-10T11:15:35.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:35.569 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:35.569 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:35.906 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:35.906 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:35.906 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839239/fa875cd0-c3a7-4fa6-9481-60c012a9ffa0/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:36.281 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:36.281 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:36.281 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0226359 s, 46.3 MB/s 2021-02-10T11:15:36.282 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:36.282 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:36.282 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:36.618 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:36.618 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:36.618 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839239/fa875cd0-c3a7-4fa6-9481-60c012a9ffa0/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:36.992 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:36.992 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:36.992 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0223325 s, 47.0 MB/s 2021-02-10T11:15:36.994 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:36.994 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:36.994 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:37.329 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:37.330 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:37.330 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839239/fa875cd0-c3a7-4fa6-9481-60c012a9ffa0/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:37.703 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:37.703 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:37.703 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244081 s, 43.0 MB/s 2021-02-10T11:15:37.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:37.705 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:37.705 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:38.040 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:38.040 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:38.040 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839239/fa875cd0-c3a7-4fa6-9481-60c012a9ffa0/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:38.412 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:38.412 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:38.412 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0226684 s, 46.3 MB/s 2021-02-10T11:15:38.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:38.413 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:38.413 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:38.748 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:38.748 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:38.749 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839239/fa875cd0-c3a7-4fa6-9481-60c012a9ffa0/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:39.123 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:39.123 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:39.123 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0265582 s, 39.5 MB/s 2021-02-10T11:15:39.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:39.124 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:39.124 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:39.457 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:39.458 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:39.458 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839239/fa875cd0-c3a7-4fa6-9481-60c012a9ffa0/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:39.832 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:39.832 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:39.832 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023685 s, 44.3 MB/s 2021-02-10T11:15:39.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:39.833 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:39.834 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:40.168 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:40.168 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:40.168 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839239/fa875cd0-c3a7-4fa6-9481-60c012a9ffa0/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:40.544 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:40.544 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:40.544 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252346 s, 41.6 MB/s 2021-02-10T11:15:40.545 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839240 2021-02-10T11:15:40.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.953 7f73b308d700 1 Processor -- start 2021-02-10T11:15:40.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.953 7f73b308d700 1 -- start start 2021-02-10T11:15:40.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.954 7f73b308d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f73ac1b0470 0x7f73ac1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:40.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.954 7f73b308d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f73ac0f20e0 0x7f73ac0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:40.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.954 7f73b308d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73ac0f0b50 con 0x7f73ac0ef610 2021-02-10T11:15:40.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.954 7f73b308d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73ac0f0820 con 0x7f73ac0f20e0 2021-02-10T11:15:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.954 7f73b308d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73ac0f24e0 con 0x7f73ac1b0470 2021-02-10T11:15:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.954 7f73b16aa700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f73ac0f20e0 0x7f73ac0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.954 7f73a3fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f73ac0ef610 0x7f73ac0efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46208/0 (socket says 172.21.2.101:46208) 2021-02-10T11:15:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.954 7f73a3fff700 1 -- 172.21.2.101:0/1722786125 learned_addr learned my addr 172.21.2.101:0/1722786125 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:15:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.954 7f73b0ea9700 1 --2- 172.21.2.101:0/1722786125 >> v2:172.21.2.108:3300/0 conn(0x7f73ac1b0470 0x7f73ac1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.954 7f73b16aa700 1 -- 172.21.2.101:0/1722786125 >> v2:172.21.2.108:3300/0 conn(0x7f73ac1b0470 msgr2=0x7f73ac1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.954 7f73b16aa700 1 --2- 172.21.2.101:0/1722786125 >> v2:172.21.2.108:3300/0 conn(0x7f73ac1b0470 0x7f73ac1b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.954 7f73b16aa700 1 -- 172.21.2.101:0/1722786125 >> v1:172.21.2.101:6789/0 conn(0x7f73ac0ef610 legacy=0x7f73ac0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.955 7f73b16aa700 1 -- 172.21.2.101:0/1722786125 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac0e52e0 con 0x7f73ac0f20e0 2021-02-10T11:15:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.955 7f73b16aa700 1 --2- 172.21.2.101:0/1722786125 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73ac0f20e0 0x7f73ac0f0200 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f73a4006e30 tx=0x7f73a4009e60).ready entity=mon.0 client_cookie=a5966b9cd68ee7c7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.955 7f73a37fe700 1 -- 172.21.2.101:0/1722786125 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73a4006a50 con 0x7f73ac0f20e0 2021-02-10T11:15:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.955 7f73a37fe700 1 -- 172.21.2.101:0/1722786125 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a400b300 con 0x7f73ac0f20e0 2021-02-10T11:15:40.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.955 7f73a37fe700 1 -- 172.21.2.101:0/1722786125 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73a400b820 con 0x7f73ac0f20e0 2021-02-10T11:15:40.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.955 7f73b308d700 1 -- 172.21.2.101:0/1722786125 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73ac0f20e0 msgr2=0x7f73ac0f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:40.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.955 7f73b308d700 1 --2- 172.21.2.101:0/1722786125 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73ac0f20e0 0x7f73ac0f0200 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f73a4006e30 tx=0x7f73a4009e60).stop 2021-02-10T11:15:40.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.955 7f73b308d700 1 -- 172.21.2.101:0/1722786125 shutdown_connections 2021-02-10T11:15:40.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.955 7f73b308d700 1 --2- 172.21.2.101:0/1722786125 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73ac0f20e0 0x7f73ac0f0200 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:40.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.955 7f73b308d700 1 --2- 172.21.2.101:0/1722786125 >> v2:172.21.2.108:3300/0 conn(0x7f73ac1b0470 0x7f73ac1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:40.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.955 7f73b308d700 1 -- 172.21.2.101:0/1722786125 >> 172.21.2.101:0/1722786125 conn(0x7f73ac0ce3e0 msgr2=0x7f73ac0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:40.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.956 7f73b308d700 1 -- 172.21.2.101:0/1722786125 shutdown_connections 2021-02-10T11:15:40.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.956 7f73b308d700 1 -- 172.21.2.101:0/1722786125 wait complete. 2021-02-10T11:15:40.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.970 7f73b308d700 1 Processor -- start 2021-02-10T11:15:40.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.970 7f73b308d700 1 -- start start 2021-02-10T11:15:40.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.971 7f73b308d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73ac08ece0 0x7f73ac0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:40.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.971 7f73b308d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73ac0a6860 0x7f73ac0a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:40.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.971 7f73b308d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73ac0a7280 0x7f73ac0a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:40.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.971 7f73b308d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac0a5cc0 con 0x7f73ac0a6860 2021-02-10T11:15:40.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.971 7f73b308d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac08f0e0 con 0x7f73ac08ece0 2021-02-10T11:15:40.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.971 7f73b308d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac08f2c0 con 0x7f73ac0a7280 2021-02-10T11:15:40.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.971 7f73a3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73ac0a6860 0x7f73ac0a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:40.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.971 7f73a3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73ac0a6860 0x7f73ac0a6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45720/0 (socket says 172.21.2.101:45720) 2021-02-10T11:15:40.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.971 7f73a3fff700 1 -- 172.21.2.101:0/1641795344 learned_addr learned my addr 172.21.2.101:0/1641795344 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:40.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.971 7f73a3fff700 1 -- 172.21.2.101:0/1641795344 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73ac0a7280 msgr2=0x7f73ac0a4590 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:40.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.971 7f73a3fff700 1 --2- 172.21.2.101:0/1641795344 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73ac0a7280 0x7f73ac0a4590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:40.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.971 7f73a3fff700 1 -- 172.21.2.101:0/1641795344 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73ac08ece0 msgr2=0x7f73ac0a6240 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:15:40.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.971 7f73a3fff700 1 --2- 172.21.2.101:0/1641795344 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73ac08ece0 0x7f73ac0a6240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:40.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.971 7f73a3fff700 1 -- 172.21.2.101:0/1641795344 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac0a4bb0 con 0x7f73ac0a6860 2021-02-10T11:15:40.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.971 7f73a3fff700 1 --2- 172.21.2.101:0/1641795344 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73ac0a6860 0x7f73ac0a6c60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f739c001400 tx=0x7f739c00a900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:40.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.971 7f73a17fa700 1 -- 172.21.2.101:0/1641795344 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f739c00acf0 con 0x7f73ac0a6860 2021-02-10T11:15:40.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.971 7f73a17fa700 1 -- 172.21.2.101:0/1641795344 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739c0009b0 con 0x7f73ac0a6860 2021-02-10T11:15:40.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.971 7f73b308d700 1 -- 172.21.2.101:0/1641795344 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73ac0a4bb0 con 0x7f73ac0a6860 2021-02-10T11:15:40.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.971 7f73b308d700 1 -- 172.21.2.101:0/1641795344 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73ac0a5140 con 0x7f73ac0a6860 2021-02-10T11:15:40.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.972 7f73a17fa700 1 -- 172.21.2.101:0/1641795344 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f739c004590 con 0x7f73ac0a6860 2021-02-10T11:15:40.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.972 7f73a17fa700 1 -- 172.21.2.101:0/1641795344 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f739c0043a0 con 0x7f73ac0a6860 2021-02-10T11:15:40.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.972 7f73a17fa700 1 --2- 172.21.2.101:0/1641795344 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f738c0186d0 0x7f738c01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:40.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.972 7f73a17fa700 1 -- 172.21.2.101:0/1641795344 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f739c032610 con 0x7f73ac0a6860 2021-02-10T11:15:40.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.972 7f73b0ea9700 1 --2- 172.21.2.101:0/1641795344 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f738c0186d0 0x7f738c01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:40.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:40.972 7f73b0ea9700 1 --2- 172.21.2.101:0/1641795344 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f738c0186d0 0x7f738c01abd0 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:41.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.003 7f73b308d700 1 -- 172.21.2.101:0/1641795344 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73ac19d1a0 con 0x7f73ac0a6860 2021-02-10T11:15:41.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.005 7f73a17fa700 1 -- 172.21.2.101:0/1641795344 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f739c000ed0 con 0x7f73ac0a6860 2021-02-10T11:15:41.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.132 7f73b308d700 1 -- 172.21.2.101:0/1641795344 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839240", "target": ["mgr", ""]}) v1 -- 0x7f73ac0a5cc0 con 0x7f738c0186d0 2021-02-10T11:15:41.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.156 7f73a17fa700 1 -- 172.21.2.101:0/1641795344 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f73a8000c90 con 0x7f738c0186d0 2021-02-10T11:15:41.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.165 7f73b308d700 1 -- 172.21.2.101:0/1641795344 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f738c0186d0 msgr2=0x7f738c01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:41.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.165 7f73b308d700 1 --2- 172.21.2.101:0/1641795344 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f738c0186d0 0x7f738c01abd0 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:41.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.165 7f73b308d700 1 -- 172.21.2.101:0/1641795344 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73ac0a6860 msgr2=0x7f73ac0a6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:41.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.165 7f73b308d700 1 --2- 172.21.2.101:0/1641795344 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73ac0a6860 0x7f73ac0a6c60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f739c001400 tx=0x7f739c00a900).stop 2021-02-10T11:15:41.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.166 7f73b308d700 1 -- 172.21.2.101:0/1641795344 shutdown_connections 2021-02-10T11:15:41.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.166 7f73b308d700 1 --2- 172.21.2.101:0/1641795344 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f738c0186d0 0x7f738c01abd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:41.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.166 7f73b308d700 1 --2- 172.21.2.101:0/1641795344 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73ac0a7280 0x7f73ac0a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:41.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.166 7f73b308d700 1 --2- 172.21.2.101:0/1641795344 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73ac0a6860 0x7f73ac0a6c60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:41.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.166 7f73b308d700 1 --2- 172.21.2.101:0/1641795344 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73ac08ece0 0x7f73ac0a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:41.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.166 7f73b308d700 1 -- 172.21.2.101:0/1641795344 >> 172.21.2.101:0/1641795344 conn(0x7f73ac08ffe0 msgr2=0x7f73ac0bf050 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:41.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.166 7f73b308d700 1 -- 172.21.2.101:0/1641795344 shutdown_connections 2021-02-10T11:15:41.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.166 7f73b308d700 1 -- 172.21.2.101:0/1641795344 wait complete. 2021-02-10T11:15:41.179 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839240 2021-02-10T11:15:41.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.587 7fbaeb351700 1 Processor -- start 2021-02-10T11:15:41.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.588 7fbaeb351700 1 -- start start 2021-02-10T11:15:41.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.588 7fbaeb351700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbae41b0470 0x7fbae41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:41.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.588 7fbaeb351700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbae40ef610 0x7fbae40efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:41.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.588 7fbaeb351700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbae40f0b50 con 0x7fbae40f0030 2021-02-10T11:15:41.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.588 7fbaeb351700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae40e52e0 con 0x7fbae40ef610 2021-02-10T11:15:41.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.588 7fbaeb351700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae40e5490 con 0x7fbae41b0470 2021-02-10T11:15:41.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.588 7fbae896c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbae40ef610 0x7fbae40efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:41.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.588 7fbae896c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbae40ef610 0x7fbae40efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45726/0 (socket says 172.21.2.101:45726) 2021-02-10T11:15:41.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.588 7fbae896c700 1 -- 172.21.2.101:0/2630258264 learned_addr learned my addr 172.21.2.101:0/2630258264 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:41.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.589 7fbae896c700 1 -- 172.21.2.101:0/2630258264 >> v2:172.21.2.108:3300/0 conn(0x7fbae41b0470 msgr2=0x7fbae41b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:41.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.589 7fbae896c700 1 --2- 172.21.2.101:0/2630258264 >> v2:172.21.2.108:3300/0 conn(0x7fbae41b0470 0x7fbae41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:41.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.589 7fbae896c700 1 -- 172.21.2.101:0/2630258264 >> v1:172.21.2.108:6789/0 conn(0x7fbae40f0030 legacy=0x7fbae40f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:41.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.589 7fbae896c700 1 -- 172.21.2.101:0/2630258264 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae40f0430 con 0x7fbae40ef610 2021-02-10T11:15:41.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.589 7fbae896c700 1 --2- 172.21.2.101:0/2630258264 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbae40ef610 0x7fbae40efa10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fbad4006700 tx=0x7fbad4009b80).ready entity=mon.0 client_cookie=3f0282107f48ce33 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:41.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.589 7fbadbfff700 1 -- 172.21.2.101:0/2630258264 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbad400c0c0 con 0x7fbae40ef610 2021-02-10T11:15:41.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.589 7fbadbfff700 1 -- 172.21.2.101:0/2630258264 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad400c380 con 0x7fbae40ef610 2021-02-10T11:15:41.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.589 7fbadbfff700 1 -- 172.21.2.101:0/2630258264 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbad400c8a0 con 0x7fbae40ef610 2021-02-10T11:15:41.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.590 7fbaeb351700 1 -- 172.21.2.101:0/2630258264 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbae40ef610 msgr2=0x7fbae40efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:41.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.590 7fbaeb351700 1 --2- 172.21.2.101:0/2630258264 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbae40ef610 0x7fbae40efa10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fbad4006700 tx=0x7fbad4009b80).stop 2021-02-10T11:15:41.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.590 7fbaeb351700 1 -- 172.21.2.101:0/2630258264 shutdown_connections 2021-02-10T11:15:41.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.590 7fbaeb351700 1 --2- 172.21.2.101:0/2630258264 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbae40ef610 0x7fbae40efa10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:41.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.590 7fbaeb351700 1 --2- 172.21.2.101:0/2630258264 >> v2:172.21.2.108:3300/0 conn(0x7fbae41b0470 0x7fbae41b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:41.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.590 7fbaeb351700 1 -- 172.21.2.101:0/2630258264 >> 172.21.2.101:0/2630258264 conn(0x7fbae40ce3e0 msgr2=0x7fbae40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:41.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.590 7fbaeb351700 1 -- 172.21.2.101:0/2630258264 shutdown_connections 2021-02-10T11:15:41.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.590 7fbaeb351700 1 -- 172.21.2.101:0/2630258264 wait complete. 2021-02-10T11:15:41.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.605 7fbaeb351700 1 Processor -- start 2021-02-10T11:15:41.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.605 7fbaeb351700 1 -- start start 2021-02-10T11:15:41.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.606 7fbaeb351700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbae40c36a0 0x7fbae40c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:41.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.606 7fbaeb351700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbae40c3190 0x7fbae40b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:41.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.606 7fbaeb351700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbae40b2370 0x7fbae40b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:41.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.606 7fbaeb351700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae40b27c0 con 0x7fbae40b2370 2021-02-10T11:15:41.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.606 7fbaeb351700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae40b3360 con 0x7fbae40c3190 2021-02-10T11:15:41.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.606 7fbaeb351700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae40b3540 con 0x7fbae40c36a0 2021-02-10T11:15:41.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.606 7fbae996e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbae40b2370 0x7fbae40b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:41.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.606 7fbae896c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbae40c3190 0x7fbae40b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:41.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.606 7fbae996e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbae40b2370 0x7fbae40b2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45732/0 (socket says 172.21.2.101:45732) 2021-02-10T11:15:41.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.606 7fbae996e700 1 -- 172.21.2.101:0/3204554198 learned_addr learned my addr 172.21.2.101:0/3204554198 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:41.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.606 7fbae916d700 1 --2- 172.21.2.101:0/3204554198 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbae40c36a0 0x7fbae40c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:41.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.606 7fbae996e700 1 -- 172.21.2.101:0/3204554198 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbae40c36a0 msgr2=0x7fbae40c2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:41.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.606 7fbae996e700 1 --2- 172.21.2.101:0/3204554198 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbae40c36a0 0x7fbae40c2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:41.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.606 7fbae996e700 1 -- 172.21.2.101:0/3204554198 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbae40c3190 msgr2=0x7fbae40b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:41.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.606 7fbae996e700 1 --2- 172.21.2.101:0/3204554198 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbae40c3190 0x7fbae40b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:41.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.606 7fbae996e700 1 -- 172.21.2.101:0/3204554198 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae40b3760 con 0x7fbae40b2370 2021-02-10T11:15:41.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.607 7fbae996e700 1 --2- 172.21.2.101:0/3204554198 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbae40b2370 0x7fbae40b2d40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7fbadc009940 tx=0x7fbadc00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:41.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.607 7fbad9ffb700 1 -- 172.21.2.101:0/3204554198 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbadc000f90 con 0x7fbae40b2370 2021-02-10T11:15:41.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.607 7fbad9ffb700 1 -- 172.21.2.101:0/3204554198 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbadc00c2f0 con 0x7fbae40b2370 2021-02-10T11:15:41.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.607 7fbad9ffb700 1 -- 172.21.2.101:0/3204554198 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbadc00c810 con 0x7fbae40b2370 2021-02-10T11:15:41.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.607 7fbaeb351700 1 -- 172.21.2.101:0/3204554198 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae40b3760 con 0x7fbae40b2370 2021-02-10T11:15:41.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.607 7fbaeb351700 1 -- 172.21.2.101:0/3204554198 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae40b3cf0 con 0x7fbae40b2370 2021-02-10T11:15:41.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.608 7fbad9ffb700 1 -- 172.21.2.101:0/3204554198 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbadc001220 con 0x7fbae40b2370 2021-02-10T11:15:41.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.608 7fbad9ffb700 1 --2- 172.21.2.101:0/3204554198 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbac40186c0 0x7fbac401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:41.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.608 7fbad9ffb700 1 -- 172.21.2.101:0/3204554198 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fbadc0328e0 con 0x7fbae40b2370 2021-02-10T11:15:41.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.608 7fbae916d700 1 --2- 172.21.2.101:0/3204554198 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbac40186c0 0x7fbac401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:41.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.608 7fbae916d700 1 --2- 172.21.2.101:0/3204554198 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbac40186c0 0x7fbac401abc0 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:41.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.638 7fbaeb351700 1 -- 172.21.2.101:0/3204554198 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae40edd10 con 0x7fbae40b2370 2021-02-10T11:15:41.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.641 7fbad9ffb700 1 -- 172.21.2.101:0/3204554198 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbadc000f40 con 0x7fbae40b2370 2021-02-10T11:15:41.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.768 7fbaeb351700 1 -- 172.21.2.101:0/3204554198 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839240", "target": ["mgr", ""]}) v1 -- 0x7fbae40b3cf0 con 0x7fbac40186c0 2021-02-10T11:15:41.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.772 7fbad9ffb700 1 -- 172.21.2.101:0/3204554198 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fbae0006550 con 0x7fbac40186c0 2021-02-10T11:15:41.781 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839240/17222ff8-9db4-46f6-bce2-b1316f942edd 2021-02-10T11:15:41.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.777 7fbaeb351700 1 -- 172.21.2.101:0/3204554198 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbac40186c0 msgr2=0x7fbac401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:41.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.777 7fbaeb351700 1 --2- 172.21.2.101:0/3204554198 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbac40186c0 0x7fbac401abc0 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:41.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.777 7fbaeb351700 1 -- 172.21.2.101:0/3204554198 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbae40b2370 msgr2=0x7fbae40b2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:41.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.777 7fbaeb351700 1 --2- 172.21.2.101:0/3204554198 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbae40b2370 0x7fbae40b2d40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7fbadc009940 tx=0x7fbadc00bcf0).stop 2021-02-10T11:15:41.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.777 7fbaeb351700 1 -- 172.21.2.101:0/3204554198 shutdown_connections 2021-02-10T11:15:41.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.777 7fbaeb351700 1 --2- 172.21.2.101:0/3204554198 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbae40b2370 0x7fbae40b2d40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:41.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.777 7fbaeb351700 1 --2- 172.21.2.101:0/3204554198 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbae40c3190 0x7fbae40b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:41.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.777 7fbaeb351700 1 --2- 172.21.2.101:0/3204554198 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbac40186c0 0x7fbac401abc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:41.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.777 7fbaeb351700 1 --2- 172.21.2.101:0/3204554198 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbae40c36a0 0x7fbae40c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:41.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.777 7fbaeb351700 1 -- 172.21.2.101:0/3204554198 >> 172.21.2.101:0/3204554198 conn(0x7fbae409f630 msgr2=0x7fbae40a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:41.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.777 7fbaeb351700 1 -- 172.21.2.101:0/3204554198 shutdown_connections 2021-02-10T11:15:41.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:41.777 7fbaeb351700 1 -- 172.21.2.101:0/3204554198 wait complete. 2021-02-10T11:15:41.790 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839240 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839240/17222ff8-9db4-46f6-bce2-b1316f942edd 2021-02-10T11:15:41.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:41.790 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:41.791 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:42.126 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:42.126 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:42.126 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839240/17222ff8-9db4-46f6-bce2-b1316f942edd/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:42.507 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:42.507 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:42.507 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.025104 s, 41.8 MB/s 2021-02-10T11:15:42.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:42.508 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:42.508 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:42.841 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:42.842 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:42.842 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839240/17222ff8-9db4-46f6-bce2-b1316f942edd/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:43.215 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:43.215 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:43.215 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234233 s, 44.8 MB/s 2021-02-10T11:15:43.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:43.217 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:43.217 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:43.550 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:43.550 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:43.551 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839240/17222ff8-9db4-46f6-bce2-b1316f942edd/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:43.921 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:43.921 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:43.921 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0223989 s, 46.8 MB/s 2021-02-10T11:15:43.922 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:43.922 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:43.922 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:44.257 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:44.258 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:44.258 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839240/17222ff8-9db4-46f6-bce2-b1316f942edd/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:44.630 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:44.631 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:44.631 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0224396 s, 46.7 MB/s 2021-02-10T11:15:44.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:44.632 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:44.632 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:44.804 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:15:44.807 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:15:44.965 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:44.966 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:44.966 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839240/17222ff8-9db4-46f6-bce2-b1316f942edd/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:45.341 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:45.341 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:45.341 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242729 s, 43.2 MB/s 2021-02-10T11:15:45.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:45.342 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:45.343 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:45.676 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:45.677 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:45.677 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839240/17222ff8-9db4-46f6-bce2-b1316f942edd/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:46.051 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:46.052 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:46.052 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244666 s, 42.9 MB/s 2021-02-10T11:15:46.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:46.053 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:46.053 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:46.387 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:46.388 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:46.388 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839240/17222ff8-9db4-46f6-bce2-b1316f942edd/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:46.764 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:46.764 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:46.764 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0254047 s, 41.3 MB/s 2021-02-10T11:15:46.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:46.765 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:46.765 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:47.098 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:47.099 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:47.099 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839240/17222ff8-9db4-46f6-bce2-b1316f942edd/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:47.473 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:47.473 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:47.473 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233173 s, 45.0 MB/s 2021-02-10T11:15:47.474 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:47.474 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:47.474 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:47.810 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:47.810 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:47.810 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839240/17222ff8-9db4-46f6-bce2-b1316f942edd/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:48.185 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:48.185 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:48.185 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244696 s, 42.9 MB/s 2021-02-10T11:15:48.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:48.186 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:48.187 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:48.521 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:48.521 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:48.521 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839240/17222ff8-9db4-46f6-bce2-b1316f942edd/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:48.896 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:48.896 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:48.896 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0269228 s, 38.9 MB/s 2021-02-10T11:15:48.897 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839241 2021-02-10T11:15:49.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.305 7f3c2ad20700 1 Processor -- start 2021-02-10T11:15:49.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.305 7f3c2ad20700 1 -- start start 2021-02-10T11:15:49.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.306 7f3c2ad20700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3c241b0870 0x7f3c240f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:49.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.306 7f3c2ad20700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c241b0c70 con 0x7f3c240efe00 2021-02-10T11:15:49.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.306 7f3c2ad20700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c240f0b50 con 0x7f3c241b0470 2021-02-10T11:15:49.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.306 7f3c2ad20700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c240e5ea0 con 0x7f3c241b0870 2021-02-10T11:15:49.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.306 7f3c2933d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f3c240efe00 0x7f3c240f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46238/0 (socket says 172.21.2.101:46238) 2021-02-10T11:15:49.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.306 7f3c2933d700 1 -- 172.21.2.101:0/2372269591 learned_addr learned my addr 172.21.2.101:0/2372269591 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:15:49.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.306 7f3c237fe700 1 -- 172.21.2.101:0/2372269591 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4152543403 0 0) 0x7f3c1c001500 con 0x7f3c240efe00 2021-02-10T11:15:49.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.306 7f3c237fe700 1 -- 172.21.2.101:0/2372269591 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c10002fe0 con 0x7f3c240efe00 2021-02-10T11:15:49.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c23fff700 1 --2- 172.21.2.101:0/2372269591 >> v2:172.21.2.108:3300/0 conn(0x7f3c241b0870 0x7f3c240f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:49.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c237fe700 1 -- 172.21.2.101:0/2372269591 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2886751821 0 0) 0x7f3c1c001500 con 0x7f3c240efe00 2021-02-10T11:15:49.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c237fe700 1 -- 172.21.2.101:0/2372269591 >> v2:172.21.2.108:3300/0 conn(0x7f3c241b0870 msgr2=0x7f3c240f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:49.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c237fe700 1 --2- 172.21.2.101:0/2372269591 >> v2:172.21.2.108:3300/0 conn(0x7f3c241b0870 0x7f3c240f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:49.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c237fe700 1 -- 172.21.2.101:0/2372269591 >> v1:172.21.2.108:6789/0 conn(0x7f3c241b0470 legacy=0x7f3c240ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:49.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c237fe700 1 -- 172.21.2.101:0/2372269591 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c240e52e0 con 0x7f3c240efe00 2021-02-10T11:15:49.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c237fe700 1 -- 172.21.2.101:0/2372269591 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f3c1c001a00 con 0x7f3c240efe00 2021-02-10T11:15:49.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c23fff700 1 --2- 172.21.2.101:0/2372269591 >> v2:172.21.2.108:3300/0 conn(0x7f3c241b0870 0x7f3c240f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:15:49.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c237fe700 1 -- 172.21.2.101:0/2372269591 >> v1:172.21.2.101:6789/0 conn(0x7f3c240efe00 legacy=0x7f3c240f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:49.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c237fe700 1 --2- 172.21.2.101:0/2372269591 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c10003aa0 0x7f3c10005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:49.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c237fe700 1 --2- 172.21.2.101:0/2372269591 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c10006770 0x7f3c10008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:49.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c237fe700 1 --2- 172.21.2.101:0/2372269591 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c10009590 0x7f3c1000bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:49.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c237fe700 1 -- 172.21.2.101:0/2372269591 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1000c300 con 0x7f3c10009590 2021-02-10T11:15:49.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c237fe700 1 -- 172.21.2.101:0/2372269591 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1000c520 con 0x7f3c10006770 2021-02-10T11:15:49.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c237fe700 1 -- 172.21.2.101:0/2372269591 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1000c740 con 0x7f3c10003aa0 2021-02-10T11:15:49.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c2933d700 1 --2- 172.21.2.101:0/2372269591 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c10009590 0x7f3c1000bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:49.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c28b3c700 1 --2- 172.21.2.101:0/2372269591 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c10003aa0 0x7f3c10005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:49.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c23fff700 1 --2- 172.21.2.101:0/2372269591 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c10006770 0x7f3c10008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:49.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c2933d700 1 -- 172.21.2.101:0/2372269591 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c10003aa0 msgr2=0x7f3c10005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:49.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c2933d700 1 --2- 172.21.2.101:0/2372269591 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c10003aa0 0x7f3c10005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:49.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c2933d700 1 -- 172.21.2.101:0/2372269591 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c10006770 msgr2=0x7f3c10008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:49.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c2933d700 1 --2- 172.21.2.101:0/2372269591 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c10006770 0x7f3c10008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:49.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.307 7f3c2933d700 1 -- 172.21.2.101:0/2372269591 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c1000c9e0 con 0x7f3c10009590 2021-02-10T11:15:49.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.308 7f3c2933d700 1 -- 172.21.2.101:0/2372269591 reap_dead start 2021-02-10T11:15:49.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.308 7f3c2933d700 1 --2- 172.21.2.101:0/2372269591 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c10009590 0x7f3c1000bab0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f3c1c001160 tx=0x7f3c1c04e310).ready entity=mon.0 client_cookie=31855b7a8e5cd701 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:49.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.308 7f3c237fe700 1 -- 172.21.2.101:0/2372269591 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3c1c04bde0 con 0x7f3c10009590 2021-02-10T11:15:49.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.308 7f3c237fe700 1 -- 172.21.2.101:0/2372269591 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1c04e700 con 0x7f3c10009590 2021-02-10T11:15:49.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.308 7f3c237fe700 1 -- 172.21.2.101:0/2372269591 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3c1c04ec20 con 0x7f3c10009590 2021-02-10T11:15:49.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.308 7f3c2ad20700 1 -- 172.21.2.101:0/2372269591 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c10009590 msgr2=0x7f3c1000bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:49.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.308 7f3c2ad20700 1 --2- 172.21.2.101:0/2372269591 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c10009590 0x7f3c1000bab0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f3c1c001160 tx=0x7f3c1c04e310).stop 2021-02-10T11:15:49.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.308 7f3c2ad20700 1 -- 172.21.2.101:0/2372269591 shutdown_connections 2021-02-10T11:15:49.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.308 7f3c2ad20700 1 --2- 172.21.2.101:0/2372269591 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c10009590 0x7f3c1000bab0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:49.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.308 7f3c2ad20700 1 -- 172.21.2.101:0/2372269591 >> 172.21.2.101:0/2372269591 conn(0x7f3c240ce3e0 msgr2=0x7f3c240e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:49.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.308 7f3c2ad20700 1 -- 172.21.2.101:0/2372269591 shutdown_connections 2021-02-10T11:15:49.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.308 7f3c2ad20700 1 -- 172.21.2.101:0/2372269591 wait complete. 2021-02-10T11:15:49.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.323 7f3c2ad20700 1 Processor -- start 2021-02-10T11:15:49.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.323 7f3c2ad20700 1 -- start start 2021-02-10T11:15:49.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.324 7f3c2ad20700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c2409b930 0x7f3c240b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:49.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.324 7f3c2ad20700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c240b8410 0x7f3c240b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:49.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.324 7f3c2ad20700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c240c1f80 0x7f3c240c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:49.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.324 7f3c2ad20700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c240c1a00 con 0x7f3c2409b930 2021-02-10T11:15:49.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.324 7f3c2ad20700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c240b8e30 con 0x7f3c240b8410 2021-02-10T11:15:49.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.324 7f3c2ad20700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c240b9050 con 0x7f3c240c1f80 2021-02-10T11:15:49.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.324 7f3c28b3c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c2409b930 0x7f3c240b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:49.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.324 7f3c28b3c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c2409b930 0x7f3c240b7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45756/0 (socket says 172.21.2.101:45756) 2021-02-10T11:15:49.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.324 7f3c28b3c700 1 -- 172.21.2.101:0/1145740443 learned_addr learned my addr 172.21.2.101:0/1145740443 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:49.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.324 7f3c28b3c700 1 -- 172.21.2.101:0/1145740443 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c240c1f80 msgr2=0x7f3c240c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:49.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.324 7f3c28b3c700 1 --2- 172.21.2.101:0/1145740443 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c240c1f80 0x7f3c240c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:49.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.324 7f3c28b3c700 1 -- 172.21.2.101:0/1145740443 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c240b8410 msgr2=0x7f3c240b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:49.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.324 7f3c28b3c700 1 --2- 172.21.2.101:0/1145740443 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c240b8410 0x7f3c240b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:49.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.324 7f3c28b3c700 1 -- 172.21.2.101:0/1145740443 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c240c29a0 con 0x7f3c2409b930 2021-02-10T11:15:49.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.324 7f3c28b3c700 1 --2- 172.21.2.101:0/1145740443 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c2409b930 0x7f3c240b7c20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f3c0c00b590 tx=0x7f3c0c00b930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:49.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.325 7f3c217fa700 1 -- 172.21.2.101:0/1145740443 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3c0c00bd20 con 0x7f3c2409b930 2021-02-10T11:15:49.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.325 7f3c217fa700 1 -- 172.21.2.101:0/1145740443 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0c000d90 con 0x7f3c2409b930 2021-02-10T11:15:49.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.325 7f3c217fa700 1 -- 172.21.2.101:0/1145740443 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3c0c006430 con 0x7f3c2409b930 2021-02-10T11:15:49.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.325 7f3c2ad20700 1 -- 172.21.2.101:0/1145740443 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c240c29a0 con 0x7f3c2409b930 2021-02-10T11:15:49.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.325 7f3c2ad20700 1 -- 172.21.2.101:0/1145740443 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c240c29a0 con 0x7f3c2409b930 2021-02-10T11:15:49.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.325 7f3c217fa700 1 -- 172.21.2.101:0/1145740443 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3c0c006030 con 0x7f3c2409b930 2021-02-10T11:15:49.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.325 7f3c217fa700 1 --2- 172.21.2.101:0/1145740443 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3c080186c0 0x7f3c0801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:49.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.325 7f3c217fa700 1 -- 172.21.2.101:0/1145740443 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3c0c0325c0 con 0x7f3c2409b930 2021-02-10T11:15:49.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.326 7f3c23fff700 1 --2- 172.21.2.101:0/1145740443 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3c080186c0 0x7f3c0801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:49.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.326 7f3c23fff700 1 --2- 172.21.2.101:0/1145740443 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3c080186c0 0x7f3c0801abc0 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:49.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.356 7f3c2ad20700 1 -- 172.21.2.101:0/1145740443 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c24105b30 con 0x7f3c2409b930 2021-02-10T11:15:49.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.358 7f3c217fa700 1 -- 172.21.2.101:0/1145740443 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3c0c021040 con 0x7f3c2409b930 2021-02-10T11:15:49.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.482 7f3c2ad20700 1 -- 172.21.2.101:0/1145740443 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839241", "target": ["mgr", ""]}) v1 -- 0x7f3c240b8e30 con 0x7f3c080186c0 2021-02-10T11:15:49.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.508 7f3c217fa700 1 -- 172.21.2.101:0/1145740443 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f3c140009b0 con 0x7f3c080186c0 2021-02-10T11:15:49.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.515 7f3c2ad20700 1 -- 172.21.2.101:0/1145740443 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3c080186c0 msgr2=0x7f3c0801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:49.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.515 7f3c2ad20700 1 --2- 172.21.2.101:0/1145740443 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3c080186c0 0x7f3c0801abc0 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:49.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.515 7f3c2ad20700 1 -- 172.21.2.101:0/1145740443 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c2409b930 msgr2=0x7f3c240b7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:49.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.515 7f3c2ad20700 1 --2- 172.21.2.101:0/1145740443 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c2409b930 0x7f3c240b7c20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f3c0c00b590 tx=0x7f3c0c00b930).stop 2021-02-10T11:15:49.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.516 7f3c2ad20700 1 -- 172.21.2.101:0/1145740443 shutdown_connections 2021-02-10T11:15:49.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.516 7f3c2ad20700 1 --2- 172.21.2.101:0/1145740443 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3c080186c0 0x7f3c0801abc0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:49.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.516 7f3c2ad20700 1 --2- 172.21.2.101:0/1145740443 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c240c1f80 0x7f3c240c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:49.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.516 7f3c2ad20700 1 --2- 172.21.2.101:0/1145740443 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c240b8410 0x7f3c240b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:49.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.516 7f3c2ad20700 1 --2- 172.21.2.101:0/1145740443 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c2409b930 0x7f3c240b7c20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:49.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.516 7f3c2ad20700 1 -- 172.21.2.101:0/1145740443 >> 172.21.2.101:0/1145740443 conn(0x7f3c24094e10 msgr2=0x7f3c240a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:49.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.516 7f3c2ad20700 1 -- 172.21.2.101:0/1145740443 shutdown_connections 2021-02-10T11:15:49.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.516 7f3c2ad20700 1 -- 172.21.2.101:0/1145740443 wait complete. 2021-02-10T11:15:49.573 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839241 2021-02-10T11:15:50.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.983 7f7607ed9700 1 Processor -- start 2021-02-10T11:15:50.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.984 7f7607ed9700 1 -- start start 2021-02-10T11:15:50.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.984 7f7607ed9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f76001b0870 0x7f76000f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:50.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.984 7f7607ed9700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76001b0c70 con 0x7f76000efe00 2021-02-10T11:15:50.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.984 7f7607ed9700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76000f0b50 con 0x7f76001b0470 2021-02-10T11:15:50.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.984 7f7607ed9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76000e5ea0 con 0x7f76001b0870 2021-02-10T11:15:50.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.984 7f76064f6700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f76000efe00 0x7f76000f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46260/0 (socket says 172.21.2.101:46260) 2021-02-10T11:15:50.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.984 7f76064f6700 1 -- 172.21.2.101:0/683260529 learned_addr learned my addr 172.21.2.101:0/683260529 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:15:50.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.985 7f7604cf3700 1 -- 172.21.2.101:0/683260529 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912877994 0 0) 0x7f75f8000f90 con 0x7f76000efe00 2021-02-10T11:15:50.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.985 7f7604cf3700 1 -- 172.21.2.101:0/683260529 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75ec002fe0 con 0x7f76000efe00 2021-02-10T11:15:50.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.985 7f7604cf3700 1 -- 172.21.2.101:0/683260529 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3030632169 0 0) 0x7f75f8001460 con 0x7f76000efe00 2021-02-10T11:15:50.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.985 7f7604cf3700 1 -- 172.21.2.101:0/683260529 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f76001b0870 msgr2=0x7f76000f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:50.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.985 7f76054f4700 1 --2- 172.21.2.101:0/683260529 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f76001b0870 0x7f76000f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:50.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.985 7f7604cf3700 1 --2- 172.21.2.101:0/683260529 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f76001b0870 0x7f76000f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:50.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.985 7f7604cf3700 1 -- 172.21.2.101:0/683260529 >> v1:172.21.2.108:6789/0 conn(0x7f76001b0470 legacy=0x7f76000ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:50.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.985 7f7604cf3700 1 -- 172.21.2.101:0/683260529 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76000e52e0 con 0x7f76000efe00 2021-02-10T11:15:50.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.985 7f7604cf3700 1 -- 172.21.2.101:0/683260529 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f75f8001980 con 0x7f76000efe00 2021-02-10T11:15:50.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.985 7f7604cf3700 1 -- 172.21.2.101:0/683260529 >> v1:172.21.2.101:6789/0 conn(0x7f76000efe00 legacy=0x7f76000f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:50.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.985 7f7604cf3700 1 --2- 172.21.2.101:0/683260529 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75ec003b20 0x7f75ec005f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:50.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.985 7f7604cf3700 1 --2- 172.21.2.101:0/683260529 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75ec006830 0x7f75ec008d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:50.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.985 7f7604cf3700 1 --2- 172.21.2.101:0/683260529 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75ec0095a0 0x7f75ec00bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:50.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.985 7f7604cf3700 1 -- 172.21.2.101:0/683260529 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec00c370 con 0x7f75ec003b20 2021-02-10T11:15:50.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.985 7f7604cf3700 1 -- 172.21.2.101:0/683260529 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec00c590 con 0x7f75ec0095a0 2021-02-10T11:15:50.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.985 7f7604cf3700 1 -- 172.21.2.101:0/683260529 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec00c7b0 con 0x7f75ec006830 2021-02-10T11:15:50.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.985 7f76054f4700 1 --2- 172.21.2.101:0/683260529 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f76001b0870 0x7f76000f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:15:50.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.985 7f7605cf5700 1 --2- 172.21.2.101:0/683260529 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75ec003b20 0x7f75ec005f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:50.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.985 7f76064f6700 1 --2- 172.21.2.101:0/683260529 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75ec0095a0 0x7f75ec00bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:50.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.985 7f76054f4700 1 --2- 172.21.2.101:0/683260529 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75ec006830 0x7f75ec008d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:50.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.986 7f7605cf5700 1 -- 172.21.2.101:0/683260529 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75ec006830 msgr2=0x7f75ec008d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:50.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.986 7f7605cf5700 1 --2- 172.21.2.101:0/683260529 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75ec006830 0x7f75ec008d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:50.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.986 7f7605cf5700 1 -- 172.21.2.101:0/683260529 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75ec0095a0 msgr2=0x7f75ec00bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:50.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.986 7f7605cf5700 1 --2- 172.21.2.101:0/683260529 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75ec0095a0 0x7f75ec00bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:50.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.986 7f7605cf5700 1 -- 172.21.2.101:0/683260529 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec00ca50 con 0x7f75ec003b20 2021-02-10T11:15:50.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.986 7f7605cf5700 1 --2- 172.21.2.101:0/683260529 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75ec003b20 0x7f75ec005f90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f75f4006740 tx=0x7f75f404da90).ready entity=mon.0 client_cookie=42dfe850e89ba3f6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:50.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.986 7f7604cf3700 1 -- 172.21.2.101:0/683260529 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f75f40500c0 con 0x7f75ec003b20 2021-02-10T11:15:50.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.986 7f7604cf3700 1 -- 172.21.2.101:0/683260529 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f40503c0 con 0x7f75ec003b20 2021-02-10T11:15:50.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.986 7f7604cf3700 1 -- 172.21.2.101:0/683260529 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f75f40508e0 con 0x7f75ec003b20 2021-02-10T11:15:50.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.986 7f7607ed9700 1 -- 172.21.2.101:0/683260529 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75ec003b20 msgr2=0x7f75ec005f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:50.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.986 7f7607ed9700 1 --2- 172.21.2.101:0/683260529 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75ec003b20 0x7f75ec005f90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f75f4006740 tx=0x7f75f404da90).stop 2021-02-10T11:15:50.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.986 7f76064f6700 1 -- 172.21.2.101:0/683260529 reap_dead start 2021-02-10T11:15:50.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.986 7f7607ed9700 1 -- 172.21.2.101:0/683260529 shutdown_connections 2021-02-10T11:15:50.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.986 7f7607ed9700 1 -- 172.21.2.101:0/683260529 >> 172.21.2.101:0/683260529 conn(0x7f76000ce3e0 msgr2=0x7f76000e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:50.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.987 7f7607ed9700 1 -- 172.21.2.101:0/683260529 shutdown_connections 2021-02-10T11:15:50.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:49.987 7f7607ed9700 1 -- 172.21.2.101:0/683260529 wait complete. 2021-02-10T11:15:50.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.001 7f7607ed9700 1 Processor -- start 2021-02-10T11:15:50.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.001 7f7607ed9700 1 -- start start 2021-02-10T11:15:50.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.001 7f7607ed9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f760009e560 0x7f76000a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:50.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.001 7f7607ed9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f760009e960 0x7f76000a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:50.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.001 7f7607ed9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f76000a0d70 0x7f76000a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:50.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.001 7f7607ed9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76000c38f0 con 0x7f760009e560 2021-02-10T11:15:50.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.001 7f7607ed9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76000a1170 con 0x7f76000a0d70 2021-02-10T11:15:50.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.001 7f7607ed9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76000a1390 con 0x7f760009e960 2021-02-10T11:15:50.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.002 7f7605cf5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f760009e560 0x7f76000a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:50.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.002 7f7605cf5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f760009e560 0x7f76000a0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45774/0 (socket says 172.21.2.101:45774) 2021-02-10T11:15:50.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.002 7f7605cf5700 1 -- 172.21.2.101:0/2867587868 learned_addr learned my addr 172.21.2.101:0/2867587868 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:50.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.002 7f7605cf5700 1 -- 172.21.2.101:0/2867587868 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f760009e960 msgr2=0x7f76000a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:50.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.002 7f7605cf5700 1 --2- 172.21.2.101:0/2867587868 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f760009e960 0x7f76000a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:50.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.002 7f7605cf5700 1 -- 172.21.2.101:0/2867587868 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f76000a0d70 msgr2=0x7f76000a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:50.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.002 7f7605cf5700 1 --2- 172.21.2.101:0/2867587868 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f76000a0d70 0x7f76000a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:50.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.002 7f7605cf5700 1 -- 172.21.2.101:0/2867587868 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76000a2110 con 0x7f760009e560 2021-02-10T11:15:50.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.002 7f7605cf5700 1 --2- 172.21.2.101:0/2867587868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f760009e560 0x7f76000a0130 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f75f404df00 tx=0x7f75f404f960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:50.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.002 7f75f27fc700 1 -- 172.21.2.101:0/2867587868 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f75f404fd50 con 0x7f760009e560 2021-02-10T11:15:50.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.002 7f75f27fc700 1 -- 172.21.2.101:0/2867587868 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f40022e0 con 0x7f760009e560 2021-02-10T11:15:50.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.002 7f75f27fc700 1 -- 172.21.2.101:0/2867587868 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f75f4002740 con 0x7f760009e560 2021-02-10T11:15:50.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.002 7f7607ed9700 1 -- 172.21.2.101:0/2867587868 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76000a2110 con 0x7f760009e560 2021-02-10T11:15:50.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.003 7f7607ed9700 1 -- 172.21.2.101:0/2867587868 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76000a2110 con 0x7f760009e560 2021-02-10T11:15:50.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.003 7f75f27fc700 1 -- 172.21.2.101:0/2867587868 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f75f40024c0 con 0x7f760009e560 2021-02-10T11:15:50.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.003 7f75f27fc700 1 --2- 172.21.2.101:0/2867587868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75e80186c0 0x7f75e801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:50.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.003 7f75f27fc700 1 -- 172.21.2.101:0/2867587868 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f75f40765c0 con 0x7f760009e560 2021-02-10T11:15:50.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.003 7f76054f4700 1 --2- 172.21.2.101:0/2867587868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75e80186c0 0x7f75e801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:50.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.004 7f76054f4700 1 --2- 172.21.2.101:0/2867587868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75e80186c0 0x7f75e801abc0 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:50.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.034 7f7607ed9700 1 -- 172.21.2.101:0/2867587868 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7600105b30 con 0x7f760009e560 2021-02-10T11:15:50.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.037 7f75f27fc700 1 -- 172.21.2.101:0/2867587868 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f75f404d030 con 0x7f760009e560 2021-02-10T11:15:50.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.165 7f7607ed9700 1 -- 172.21.2.101:0/2867587868 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839241", "target": ["mgr", ""]}) v1 -- 0x7f76000f0b50 con 0x7f75e80186c0 2021-02-10T11:15:50.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.168 7f75f27fc700 1 -- 172.21.2.101:0/2867587868 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f75fc000940 con 0x7f75e80186c0 2021-02-10T11:15:50.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.169 7f7607ed9700 1 -- 172.21.2.101:0/2867587868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75e80186c0 msgr2=0x7f75e801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:50.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.169 7f7607ed9700 1 --2- 172.21.2.101:0/2867587868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75e80186c0 0x7f75e801abc0 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:50.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.169 7f7607ed9700 1 -- 172.21.2.101:0/2867587868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f760009e560 msgr2=0x7f76000a0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:50.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.169 7f7607ed9700 1 --2- 172.21.2.101:0/2867587868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f760009e560 0x7f76000a0130 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f75f404df00 tx=0x7f75f404f960).stop 2021-02-10T11:15:50.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.170 7f7607ed9700 1 -- 172.21.2.101:0/2867587868 shutdown_connections 2021-02-10T11:15:50.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.170 7f7607ed9700 1 --2- 172.21.2.101:0/2867587868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75e80186c0 0x7f75e801abc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:50.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.170 7f7607ed9700 1 --2- 172.21.2.101:0/2867587868 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f760009e960 0x7f76000a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:50.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.170 7f7607ed9700 1 --2- 172.21.2.101:0/2867587868 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f76000a0d70 0x7f76000a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:50.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.170 7f7607ed9700 1 --2- 172.21.2.101:0/2867587868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f760009e560 0x7f76000a0130 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:50.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.170 7f7607ed9700 1 -- 172.21.2.101:0/2867587868 >> 172.21.2.101:0/2867587868 conn(0x7f760009fc10 msgr2=0x7f7600094ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:50.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.170 7f7607ed9700 1 -- 172.21.2.101:0/2867587868 shutdown_connections 2021-02-10T11:15:50.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:50.170 7f7607ed9700 1 -- 172.21.2.101:0/2867587868 wait complete. 2021-02-10T11:15:50.176 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839241/5a0de426-9690-4c83-b14e-f110c22c8efb 2021-02-10T11:15:50.183 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839241 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839241/5a0de426-9690-4c83-b14e-f110c22c8efb 2021-02-10T11:15:50.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:50.183 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:50.183 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:50.518 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:50.518 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:50.518 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839241/5a0de426-9690-4c83-b14e-f110c22c8efb/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:50.901 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:50.901 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:50.901 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0267941 s, 39.1 MB/s 2021-02-10T11:15:50.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:50.902 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:50.902 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:51.236 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:51.236 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:51.236 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839241/5a0de426-9690-4c83-b14e-f110c22c8efb/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:51.616 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:51.617 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:51.617 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0261534 s, 40.1 MB/s 2021-02-10T11:15:51.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:51.617 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:51.617 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:51.957 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:51.958 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:51.958 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839241/5a0de426-9690-4c83-b14e-f110c22c8efb/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:52.335 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:52.335 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:52.335 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242332 s, 43.3 MB/s 2021-02-10T11:15:52.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:52.336 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:52.336 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:52.680 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:52.681 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:52.681 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839241/5a0de426-9690-4c83-b14e-f110c22c8efb/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:53.065 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:53.066 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:53.066 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241287 s, 43.5 MB/s 2021-02-10T11:15:53.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:53.067 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:53.067 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:53.407 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:53.408 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:53.408 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839241/5a0de426-9690-4c83-b14e-f110c22c8efb/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:53.785 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:53.786 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:53.786 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241297 s, 43.5 MB/s 2021-02-10T11:15:53.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:53.787 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:53.787 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:54.121 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:54.122 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:54.122 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839241/5a0de426-9690-4c83-b14e-f110c22c8efb/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:54.496 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:54.496 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:54.496 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247221 s, 42.4 MB/s 2021-02-10T11:15:54.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:54.498 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:54.498 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:54.833 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:54.833 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:54.833 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839241/5a0de426-9690-4c83-b14e-f110c22c8efb/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:55.208 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:55.209 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:55.209 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248938 s, 42.1 MB/s 2021-02-10T11:15:55.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:55.210 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:55.210 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:55.544 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:55.545 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:55.545 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839241/5a0de426-9690-4c83-b14e-f110c22c8efb/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:55.920 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:55.920 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:55.920 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243868 s, 43.0 MB/s 2021-02-10T11:15:55.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:55.920 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:55.921 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:56.254 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:56.255 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:56.255 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839241/5a0de426-9690-4c83-b14e-f110c22c8efb/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:56.629 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:56.629 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:56.629 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251604 s, 41.7 MB/s 2021-02-10T11:15:56.630 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:56.630 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:56.630 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:56.966 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:56.966 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:56.966 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839241/5a0de426-9690-4c83-b14e-f110c22c8efb/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:57.339 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:57.339 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:57.339 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243449 s, 43.1 MB/s 2021-02-10T11:15:57.340 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839242 2021-02-10T11:15:57.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.747 7f7d8ab5b700 1 Processor -- start 2021-02-10T11:15:57.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.747 7f7d8ab5b700 1 -- start start 2021-02-10T11:15:57.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.748 7f7d8ab5b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d841b0870 0x7f7d840f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:57.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.748 7f7d8ab5b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7d840efe00 0x7f7d840f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:57.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.748 7f7d8ab5b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d841b0c70 con 0x7f7d841b0470 2021-02-10T11:15:57.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.748 7f7d8ab5b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d840f0b50 con 0x7f7d840efe00 2021-02-10T11:15:57.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.748 7f7d8ab5b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d840f0820 con 0x7f7d841b0870 2021-02-10T11:15:57.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.748 7f7d88977700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f7d841b0470 0x7f7d840ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46278/0 (socket says 172.21.2.101:46278) 2021-02-10T11:15:57.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.748 7f7d88977700 1 -- 172.21.2.101:0/2258922310 learned_addr learned my addr 172.21.2.101:0/2258922310 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:15:57.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.748 7f7d837fe700 1 -- 172.21.2.101:0/2258922310 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 186857811 0 0) 0x7f7d6c0009b0 con 0x7f7d841b0470 2021-02-10T11:15:57.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.748 7f7d837fe700 1 -- 172.21.2.101:0/2258922310 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d70002fe0 con 0x7f7d841b0470 2021-02-10T11:15:57.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.748 7f7d83fff700 1 --2- 172.21.2.101:0/2258922310 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d841b0870 0x7f7d840f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:57.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.748 7f7d89178700 1 --2- 172.21.2.101:0/2258922310 >> v2:172.21.2.108:3300/0 conn(0x7f7d840efe00 0x7f7d840f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:57.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d837fe700 1 -- 172.21.2.101:0/2258922310 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2646749331 0 0) 0x7f7d6c001010 con 0x7f7d841b0470 2021-02-10T11:15:57.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d837fe700 1 -- 172.21.2.101:0/2258922310 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d841b0870 msgr2=0x7f7d840f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:57.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d837fe700 1 --2- 172.21.2.101:0/2258922310 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d841b0870 0x7f7d840f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:57.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d837fe700 1 -- 172.21.2.101:0/2258922310 >> v2:172.21.2.108:3300/0 conn(0x7f7d840efe00 msgr2=0x7f7d840f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:57.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d837fe700 1 --2- 172.21.2.101:0/2258922310 >> v2:172.21.2.108:3300/0 conn(0x7f7d840efe00 0x7f7d840f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:57.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d837fe700 1 -- 172.21.2.101:0/2258922310 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d840e52e0 con 0x7f7d841b0470 2021-02-10T11:15:57.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d837fe700 1 -- 172.21.2.101:0/2258922310 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f7d6c001530 con 0x7f7d841b0470 2021-02-10T11:15:57.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d89178700 1 --2- 172.21.2.101:0/2258922310 >> v2:172.21.2.108:3300/0 conn(0x7f7d840efe00 0x7f7d840f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:15:57.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d837fe700 1 -- 172.21.2.101:0/2258922310 >> v1:172.21.2.101:6789/0 conn(0x7f7d841b0470 legacy=0x7f7d840ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:57.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d837fe700 1 --2- 172.21.2.101:0/2258922310 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d70003a60 0x7f7d70005e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:57.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d837fe700 1 --2- 172.21.2.101:0/2258922310 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7d70006730 0x7f7d70008c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:57.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d837fe700 1 --2- 172.21.2.101:0/2258922310 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d7004d500 0x7f7d7004fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:57.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d837fe700 1 -- 172.21.2.101:0/2258922310 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d700502d0 con 0x7f7d7004d500 2021-02-10T11:15:57.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d837fe700 1 -- 172.21.2.101:0/2258922310 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d700504f0 con 0x7f7d70006730 2021-02-10T11:15:57.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d837fe700 1 -- 172.21.2.101:0/2258922310 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d70050710 con 0x7f7d70003a60 2021-02-10T11:15:57.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d88977700 1 --2- 172.21.2.101:0/2258922310 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d70003a60 0x7f7d70005e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:57.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d89178700 1 --2- 172.21.2.101:0/2258922310 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d7004d500 0x7f7d7004fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:57.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d83fff700 1 --2- 172.21.2.101:0/2258922310 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7d70006730 0x7f7d70008c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:57.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d89178700 1 -- 172.21.2.101:0/2258922310 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d70003a60 msgr2=0x7f7d70005e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:57.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d89178700 1 --2- 172.21.2.101:0/2258922310 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d70003a60 0x7f7d70005e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:57.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d89178700 1 -- 172.21.2.101:0/2258922310 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7d70006730 msgr2=0x7f7d70008c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:57.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d89178700 1 --2- 172.21.2.101:0/2258922310 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7d70006730 0x7f7d70008c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:57.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d89178700 1 -- 172.21.2.101:0/2258922310 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d700509b0 con 0x7f7d7004d500 2021-02-10T11:15:57.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d83fff700 1 --2- 172.21.2.101:0/2258922310 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7d70006730 0x7f7d70008c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:15:57.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d89178700 1 --2- 172.21.2.101:0/2258922310 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d7004d500 0x7f7d7004fa20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f7d7c0099a0 tx=0x7f7d7c009d40).ready entity=mon.0 client_cookie=a3e50264106e9236 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:57.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d837fe700 1 -- 172.21.2.101:0/2258922310 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7d7c0065d0 con 0x7f7d7004d500 2021-02-10T11:15:57.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d837fe700 1 -- 172.21.2.101:0/2258922310 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d7c00b300 con 0x7f7d7004d500 2021-02-10T11:15:57.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d837fe700 1 -- 172.21.2.101:0/2258922310 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7d7c00b7e0 con 0x7f7d7004d500 2021-02-10T11:15:57.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d8ab5b700 1 -- 172.21.2.101:0/2258922310 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d7004d500 msgr2=0x7f7d7004fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:57.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d8ab5b700 1 --2- 172.21.2.101:0/2258922310 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d7004d500 0x7f7d7004fa20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f7d7c0099a0 tx=0x7f7d7c009d40).stop 2021-02-10T11:15:57.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.749 7f7d89178700 1 -- 172.21.2.101:0/2258922310 reap_dead start 2021-02-10T11:15:57.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.750 7f7d8ab5b700 1 -- 172.21.2.101:0/2258922310 shutdown_connections 2021-02-10T11:15:57.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.750 7f7d8ab5b700 1 -- 172.21.2.101:0/2258922310 >> 172.21.2.101:0/2258922310 conn(0x7f7d840ce3e0 msgr2=0x7f7d840e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:57.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.750 7f7d8ab5b700 1 -- 172.21.2.101:0/2258922310 shutdown_connections 2021-02-10T11:15:57.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.750 7f7d8ab5b700 1 -- 172.21.2.101:0/2258922310 wait complete. 2021-02-10T11:15:57.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.765 7f7d8ab5b700 1 Processor -- start 2021-02-10T11:15:57.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.765 7f7d8ab5b700 1 -- start start 2021-02-10T11:15:57.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.765 7f7d8ab5b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d8409b930 0x7f7d840b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:57.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.765 7f7d8ab5b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d840b8410 0x7f7d840b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:57.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.765 7f7d8ab5b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7d840c1f80 0x7f7d840c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:57.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.765 7f7d8ab5b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d840c1a00 con 0x7f7d840b8410 2021-02-10T11:15:57.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.765 7f7d8ab5b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d840b8e30 con 0x7f7d840c1f80 2021-02-10T11:15:57.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.765 7f7d8ab5b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d840b9050 con 0x7f7d8409b930 2021-02-10T11:15:57.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.765 7f7d83fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d840b8410 0x7f7d840b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:57.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.765 7f7d83fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d840b8410 0x7f7d840b8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45796/0 (socket says 172.21.2.101:45796) 2021-02-10T11:15:57.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.765 7f7d83fff700 1 -- 172.21.2.101:0/643693943 learned_addr learned my addr 172.21.2.101:0/643693943 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:57.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.766 7f7d83fff700 1 -- 172.21.2.101:0/643693943 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d8409b930 msgr2=0x7f7d840b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:57.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.766 7f7d83fff700 1 --2- 172.21.2.101:0/643693943 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d8409b930 0x7f7d840b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:57.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.766 7f7d83fff700 1 -- 172.21.2.101:0/643693943 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7d840c1f80 msgr2=0x7f7d840c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:57.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.766 7f7d83fff700 1 --2- 172.21.2.101:0/643693943 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7d840c1f80 0x7f7d840c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:57.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.766 7f7d83fff700 1 -- 172.21.2.101:0/643693943 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d840c29a0 con 0x7f7d840b8410 2021-02-10T11:15:57.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.766 7f7d83fff700 1 --2- 172.21.2.101:0/643693943 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d840b8410 0x7f7d840b8810 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f7d7400b970 tx=0x7f7d7400bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:57.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.766 7f7d817fa700 1 -- 172.21.2.101:0/643693943 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7d74000d30 con 0x7f7d840b8410 2021-02-10T11:15:57.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.766 7f7d817fa700 1 -- 172.21.2.101:0/643693943 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d74006b70 con 0x7f7d840b8410 2021-02-10T11:15:57.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.766 7f7d817fa700 1 -- 172.21.2.101:0/643693943 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7d74006210 con 0x7f7d840b8410 2021-02-10T11:15:57.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.766 7f7d8ab5b700 1 -- 172.21.2.101:0/643693943 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d840c29a0 con 0x7f7d840b8410 2021-02-10T11:15:57.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.766 7f7d8ab5b700 1 -- 172.21.2.101:0/643693943 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d840c2f30 con 0x7f7d840b8410 2021-02-10T11:15:57.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.767 7f7d817fa700 1 -- 172.21.2.101:0/643693943 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7d74006020 con 0x7f7d840b8410 2021-02-10T11:15:57.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.767 7f7d817fa700 1 --2- 172.21.2.101:0/643693943 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7d680186c0 0x7f7d6801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:57.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.767 7f7d817fa700 1 -- 172.21.2.101:0/643693943 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7d740328e0 con 0x7f7d840b8410 2021-02-10T11:15:57.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.767 7f7d88977700 1 --2- 172.21.2.101:0/643693943 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7d680186c0 0x7f7d6801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:57.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.767 7f7d88977700 1 --2- 172.21.2.101:0/643693943 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7d680186c0 0x7f7d6801abc0 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:57.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.797 7f7d8ab5b700 1 -- 172.21.2.101:0/643693943 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d84105b30 con 0x7f7d840b8410 2021-02-10T11:15:57.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.800 7f7d817fa700 1 -- 172.21.2.101:0/643693943 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7d74000d30 con 0x7f7d840b8410 2021-02-10T11:15:57.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.927 7f7d8ab5b700 1 -- 172.21.2.101:0/643693943 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839242", "target": ["mgr", ""]}) v1 -- 0x7f7d840f0b50 con 0x7f7d680186c0 2021-02-10T11:15:57.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.952 7f7d817fa700 1 -- 172.21.2.101:0/643693943 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f7d6c000e20 con 0x7f7d680186c0 2021-02-10T11:15:57.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.960 7f7d8ab5b700 1 -- 172.21.2.101:0/643693943 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7d680186c0 msgr2=0x7f7d6801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:57.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.960 7f7d8ab5b700 1 --2- 172.21.2.101:0/643693943 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7d680186c0 0x7f7d6801abc0 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:57.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.960 7f7d8ab5b700 1 -- 172.21.2.101:0/643693943 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d840b8410 msgr2=0x7f7d840b8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:57.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.960 7f7d8ab5b700 1 --2- 172.21.2.101:0/643693943 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d840b8410 0x7f7d840b8810 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f7d7400b970 tx=0x7f7d7400bd10).stop 2021-02-10T11:15:57.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.960 7f7d8ab5b700 1 -- 172.21.2.101:0/643693943 shutdown_connections 2021-02-10T11:15:57.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.960 7f7d8ab5b700 1 --2- 172.21.2.101:0/643693943 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7d840c1f80 0x7f7d840c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:57.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.960 7f7d8ab5b700 1 --2- 172.21.2.101:0/643693943 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d840b8410 0x7f7d840b8810 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:57.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.960 7f7d8ab5b700 1 --2- 172.21.2.101:0/643693943 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7d680186c0 0x7f7d6801abc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:57.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.960 7f7d8ab5b700 1 --2- 172.21.2.101:0/643693943 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d8409b930 0x7f7d840b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:57.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.960 7f7d8ab5b700 1 -- 172.21.2.101:0/643693943 >> 172.21.2.101:0/643693943 conn(0x7f7d84094e10 msgr2=0x7f7d840a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:57.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.960 7f7d8ab5b700 1 -- 172.21.2.101:0/643693943 shutdown_connections 2021-02-10T11:15:57.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:57.960 7f7d8ab5b700 1 -- 172.21.2.101:0/643693943 wait complete. 2021-02-10T11:15:57.973 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839242 2021-02-10T11:15:58.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.380 7f090014e700 1 Processor -- start 2021-02-10T11:15:58.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.380 7f090014e700 1 -- start start 2021-02-10T11:15:58.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.381 7f090014e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f08f81b0470 0x7f08f81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:58.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.381 7f090014e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08f80ef610 0x7f08f80efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:58.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.381 7f090014e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f80f0b50 con 0x7f08f80f0030 2021-02-10T11:15:58.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.381 7f090014e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f80e52e0 con 0x7f08f81b0470 2021-02-10T11:15:58.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.381 7f090014e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f80e54d0 con 0x7f08f80ef610 2021-02-10T11:15:58.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.382 7f08fd769700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08f80ef610 0x7f08f80efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:58.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.382 7f08fd769700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08f80ef610 0x7f08f80efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:38536/0 (socket says 172.21.2.101:38536) 2021-02-10T11:15:58.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.382 7f08fd769700 1 -- 172.21.2.101:0/2827750232 learned_addr learned my addr 172.21.2.101:0/2827750232 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:58.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.382 7f08fdf6a700 1 --2- 172.21.2.101:0/2827750232 >> v2:172.21.2.108:3300/0 conn(0x7f08f81b0470 0x7f08f81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:58.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.382 7f08fd769700 1 -- 172.21.2.101:0/2827750232 >> v2:172.21.2.108:3300/0 conn(0x7f08f81b0470 msgr2=0x7f08f81b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:15:58.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.382 7f08fd769700 1 --2- 172.21.2.101:0/2827750232 >> v2:172.21.2.108:3300/0 conn(0x7f08f81b0470 0x7f08f81b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:58.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.382 7f08fd769700 1 -- 172.21.2.101:0/2827750232 >> v1:172.21.2.108:6789/0 conn(0x7f08f80f0030 legacy=0x7f08f80f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:58.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.382 7f08fd769700 1 -- 172.21.2.101:0/2827750232 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f80f0430 con 0x7f08f80ef610 2021-02-10T11:15:58.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.382 7f08fcf68700 1 -- 172.21.2.101:0/2827750232 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928251893 0 0) 0x7f08f0001400 con 0x7f08f80f0030 2021-02-10T11:15:58.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.382 7f08fd769700 1 --2- 172.21.2.101:0/2827750232 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08f80ef610 0x7f08f80efa10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f08f4006700 tx=0x7f08f4009b80).ready entity=mon.2 client_cookie=be58352a28b6ee39 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:58.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.383 7f08fcf68700 1 -- 172.21.2.101:0/2827750232 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f08f400c0c0 con 0x7f08f80ef610 2021-02-10T11:15:58.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.383 7f08fcf68700 1 -- 172.21.2.101:0/2827750232 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f400c340 con 0x7f08f80ef610 2021-02-10T11:15:58.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.383 7f08fcf68700 1 -- 172.21.2.101:0/2827750232 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f08f400c820 con 0x7f08f80ef610 2021-02-10T11:15:58.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.383 7f090014e700 1 -- 172.21.2.101:0/2827750232 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08f80ef610 msgr2=0x7f08f80efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:58.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.383 7f090014e700 1 --2- 172.21.2.101:0/2827750232 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08f80ef610 0x7f08f80efa10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f08f4006700 tx=0x7f08f4009b80).stop 2021-02-10T11:15:58.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.383 7f090014e700 1 -- 172.21.2.101:0/2827750232 shutdown_connections 2021-02-10T11:15:58.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.383 7f090014e700 1 --2- 172.21.2.101:0/2827750232 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08f80ef610 0x7f08f80efa10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:58.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.383 7f090014e700 1 --2- 172.21.2.101:0/2827750232 >> v2:172.21.2.108:3300/0 conn(0x7f08f81b0470 0x7f08f81b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:58.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.383 7f090014e700 1 -- 172.21.2.101:0/2827750232 >> 172.21.2.101:0/2827750232 conn(0x7f08f80ce3e0 msgr2=0x7f08f80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:58.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.384 7f090014e700 1 -- 172.21.2.101:0/2827750232 shutdown_connections 2021-02-10T11:15:58.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.384 7f090014e700 1 -- 172.21.2.101:0/2827750232 wait complete. 2021-02-10T11:15:58.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.398 7f090014e700 1 Processor -- start 2021-02-10T11:15:58.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.398 7f090014e700 1 -- start start 2021-02-10T11:15:58.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.398 7f090014e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08f80a0420 0x7f08f80a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:58.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.398 7f090014e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f08f8189d00 0x7f08f818a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:58.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.398 7f090014e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08f818a720 0x7f08f80b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:58.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.398 7f090014e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f8189800 con 0x7f08f80a0420 2021-02-10T11:15:58.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.398 7f090014e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f818ab20 con 0x7f08f8189d00 2021-02-10T11:15:58.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.398 7f090014e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f80a1140 con 0x7f08f818a720 2021-02-10T11:15:58.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.398 7f08fdf6a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08f80a0420 0x7f08f80a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:58.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.398 7f08fdf6a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08f80a0420 0x7f08f80a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45810/0 (socket says 172.21.2.101:45810) 2021-02-10T11:15:58.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.398 7f08fdf6a700 1 -- 172.21.2.101:0/3920835732 learned_addr learned my addr 172.21.2.101:0/3920835732 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:15:58.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.399 7f08fdf6a700 1 -- 172.21.2.101:0/3920835732 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08f818a720 msgr2=0x7f08f80b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:58.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.399 7f08fdf6a700 1 --2- 172.21.2.101:0/3920835732 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08f818a720 0x7f08f80b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:58.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.399 7f08fdf6a700 1 -- 172.21.2.101:0/3920835732 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f08f8189d00 msgr2=0x7f08f818a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:58.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.399 7f08fdf6a700 1 --2- 172.21.2.101:0/3920835732 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f08f8189d00 0x7f08f818a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:58.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.399 7f08fdf6a700 1 -- 172.21.2.101:0/3920835732 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f80b8200 con 0x7f08f80a0420 2021-02-10T11:15:58.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.399 7f08fdf6a700 1 --2- 172.21.2.101:0/3920835732 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08f80a0420 0x7f08f80a0820 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f08ec006f60 tx=0x7f08ec00b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:58.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.399 7f08ea7fc700 1 -- 172.21.2.101:0/3920835732 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f08ec0009b0 con 0x7f08f80a0420 2021-02-10T11:15:58.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.399 7f08ea7fc700 1 -- 172.21.2.101:0/3920835732 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec00bd60 con 0x7f08f80a0420 2021-02-10T11:15:58.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.399 7f08ea7fc700 1 -- 172.21.2.101:0/3920835732 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f08ec002030 con 0x7f08f80a0420 2021-02-10T11:15:58.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.399 7f090014e700 1 -- 172.21.2.101:0/3920835732 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f80b8200 con 0x7f08f80a0420 2021-02-10T11:15:58.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.399 7f090014e700 1 -- 172.21.2.101:0/3920835732 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f80b8200 con 0x7f08f80a0420 2021-02-10T11:15:58.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.400 7f08ea7fc700 1 -- 172.21.2.101:0/3920835732 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f08ec000c90 con 0x7f08f80a0420 2021-02-10T11:15:58.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.400 7f08ea7fc700 1 --2- 172.21.2.101:0/3920835732 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f08e00186c0 0x7f08e001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:15:58.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.400 7f08ea7fc700 1 -- 172.21.2.101:0/3920835732 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f08ec032680 con 0x7f08f80a0420 2021-02-10T11:15:58.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.400 7f08fd769700 1 --2- 172.21.2.101:0/3920835732 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f08e00186c0 0x7f08e001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:15:58.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.400 7f08fd769700 1 --2- 172.21.2.101:0/3920835732 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f08e00186c0 0x7f08e001abc0 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:15:58.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.431 7f090014e700 1 -- 172.21.2.101:0/3920835732 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f8105ae0 con 0x7f08f80a0420 2021-02-10T11:15:58.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.433 7f08ea7fc700 1 -- 172.21.2.101:0/3920835732 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f08ec006b00 con 0x7f08f80a0420 2021-02-10T11:15:58.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.561 7f090014e700 1 -- 172.21.2.101:0/3920835732 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839242", "target": ["mgr", ""]}) v1 -- 0x7f08f818ab20 con 0x7f08e00186c0 2021-02-10T11:15:58.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.564 7f08ea7fc700 1 -- 172.21.2.101:0/3920835732 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f08f4000970 con 0x7f08e00186c0 2021-02-10T11:15:58.573 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839242/72b0cf6c-db76-41f7-b57e-0d74633c89af 2021-02-10T11:15:58.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.569 7f090014e700 1 -- 172.21.2.101:0/3920835732 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f08e00186c0 msgr2=0x7f08e001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:58.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.569 7f090014e700 1 --2- 172.21.2.101:0/3920835732 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f08e00186c0 0x7f08e001abc0 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:58.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.569 7f090014e700 1 -- 172.21.2.101:0/3920835732 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08f80a0420 msgr2=0x7f08f80a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:15:58.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.569 7f090014e700 1 --2- 172.21.2.101:0/3920835732 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08f80a0420 0x7f08f80a0820 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f08ec006f60 tx=0x7f08ec00b970).stop 2021-02-10T11:15:58.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.570 7f090014e700 1 -- 172.21.2.101:0/3920835732 shutdown_connections 2021-02-10T11:15:58.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.570 7f090014e700 1 --2- 172.21.2.101:0/3920835732 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f08e00186c0 0x7f08e001abc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:58.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.570 7f090014e700 1 --2- 172.21.2.101:0/3920835732 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08f818a720 0x7f08f80b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:58.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.570 7f090014e700 1 --2- 172.21.2.101:0/3920835732 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f08f8189d00 0x7f08f818a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:15:58.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.570 7f090014e700 1 --2- 172.21.2.101:0/3920835732 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08f80a0420 0x7f08f80a0820 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:15:58.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.570 7f090014e700 1 -- 172.21.2.101:0/3920835732 >> 172.21.2.101:0/3920835732 conn(0x7f08f8094e10 msgr2=0x7f08f80a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:15:58.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.570 7f090014e700 1 -- 172.21.2.101:0/3920835732 shutdown_connections 2021-02-10T11:15:58.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:15:58.570 7f090014e700 1 -- 172.21.2.101:0/3920835732 wait complete. 2021-02-10T11:15:58.583 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839242 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839242/72b0cf6c-db76-41f7-b57e-0d74633c89af 2021-02-10T11:15:58.583 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:58.583 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:58.583 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:58.918 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:58.918 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:58.918 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839242/72b0cf6c-db76-41f7-b57e-0d74633c89af/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:15:59.297 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:15:59.297 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:15:59.297 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0255059 s, 41.1 MB/s 2021-02-10T11:15:59.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:15:59.298 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:15:59.298 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:15:59.631 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:15:59.632 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:15:59.632 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839242/72b0cf6c-db76-41f7-b57e-0d74633c89af/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:00.006 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:00.007 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:00.007 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238888 s, 43.9 MB/s 2021-02-10T11:16:00.008 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:00.008 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:00.008 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:00.343 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:00.343 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:00.344 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839242/72b0cf6c-db76-41f7-b57e-0d74633c89af/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:00.718 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:00.718 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:00.718 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023373 s, 44.9 MB/s 2021-02-10T11:16:00.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:00.719 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:00.719 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:01.053 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:01.053 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:01.054 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839242/72b0cf6c-db76-41f7-b57e-0d74633c89af/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:01.428 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:01.428 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:01.429 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238158 s, 44.0 MB/s 2021-02-10T11:16:01.429 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:01.430 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:01.430 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:01.763 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:01.764 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:01.764 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839242/72b0cf6c-db76-41f7-b57e-0d74633c89af/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:02.138 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:02.139 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:02.139 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024768 s, 42.3 MB/s 2021-02-10T11:16:02.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:02.140 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:02.140 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:02.474 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:02.474 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:02.474 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839242/72b0cf6c-db76-41f7-b57e-0d74633c89af/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:02.849 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:02.849 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:02.849 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244914 s, 42.8 MB/s 2021-02-10T11:16:02.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:02.850 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:02.850 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:03.184 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:03.185 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:03.185 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839242/72b0cf6c-db76-41f7-b57e-0d74633c89af/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:03.560 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:03.560 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:03.560 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024791 s, 42.3 MB/s 2021-02-10T11:16:03.561 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:03.561 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:03.561 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:03.895 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:03.896 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:03.896 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839242/72b0cf6c-db76-41f7-b57e-0d74633c89af/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:04.266 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:04.266 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:04.267 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233846 s, 44.8 MB/s 2021-02-10T11:16:04.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:04.268 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:04.268 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:04.603 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:04.603 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:04.603 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839242/72b0cf6c-db76-41f7-b57e-0d74633c89af/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:04.978 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:04.979 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:04.979 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252743 s, 41.5 MB/s 2021-02-10T11:16:04.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:04.980 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:04.980 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:05.315 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:05.315 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:05.315 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839242/72b0cf6c-db76-41f7-b57e-0d74633c89af/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:05.689 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:05.689 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:05.689 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237167 s, 44.2 MB/s 2021-02-10T11:16:05.690 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839243 2021-02-10T11:16:06.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.096 7f85ce53e700 1 Processor -- start 2021-02-10T11:16:06.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.097 7f85ce53e700 1 -- start start 2021-02-10T11:16:06.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.097 7f85ce53e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f85c81b0470 0x7f85c81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:06.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.097 7f85ce53e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f85c80f20e0 0x7f85c80f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:06.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.097 7f85ce53e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c80f0b50 con 0x7f85c80ef610 2021-02-10T11:16:06.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.097 7f85ce53e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c80f0820 con 0x7f85c80f20e0 2021-02-10T11:16:06.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.097 7f85ce53e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c80f24e0 con 0x7f85c81b0470 2021-02-10T11:16:06.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.097 7f85c77fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f85c80ef610 0x7f85c80efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46310/0 (socket says 172.21.2.101:46310) 2021-02-10T11:16:06.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.097 7f85c77fe700 1 -- 172.21.2.101:0/87732376 learned_addr learned my addr 172.21.2.101:0/87732376 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:16:06.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c6ffd700 1 -- 172.21.2.101:0/87732376 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788613771 0 0) 0x7f85bc0010b0 con 0x7f85c80ef610 2021-02-10T11:16:06.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c6ffd700 1 -- 172.21.2.101:0/87732376 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85b0002fe0 con 0x7f85c80ef610 2021-02-10T11:16:06.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c7fff700 1 --2- 172.21.2.101:0/87732376 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f85c81b0470 0x7f85c81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:06.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85ccb5b700 1 --2- 172.21.2.101:0/87732376 >> v2:172.21.2.108:3300/0 conn(0x7f85c80f20e0 0x7f85c80f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:06.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c6ffd700 1 -- 172.21.2.101:0/87732376 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 435135715 0 0) 0x7f85bc0010b0 con 0x7f85c80ef610 2021-02-10T11:16:06.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c6ffd700 1 -- 172.21.2.101:0/87732376 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f85c81b0470 msgr2=0x7f85c81b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:16:06.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c6ffd700 1 --2- 172.21.2.101:0/87732376 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f85c81b0470 0x7f85c81b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:06.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c6ffd700 1 -- 172.21.2.101:0/87732376 >> v2:172.21.2.108:3300/0 conn(0x7f85c80f20e0 msgr2=0x7f85c80f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:16:06.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c6ffd700 1 --2- 172.21.2.101:0/87732376 >> v2:172.21.2.108:3300/0 conn(0x7f85c80f20e0 0x7f85c80f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:06.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c6ffd700 1 -- 172.21.2.101:0/87732376 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c80e52e0 con 0x7f85c80ef610 2021-02-10T11:16:06.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c6ffd700 1 -- 172.21.2.101:0/87732376 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f85bc0015b0 con 0x7f85c80ef610 2021-02-10T11:16:06.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c6ffd700 1 -- 172.21.2.101:0/87732376 >> v1:172.21.2.101:6789/0 conn(0x7f85c80ef610 legacy=0x7f85c80efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:06.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c6ffd700 1 --2- 172.21.2.101:0/87732376 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85b0003a40 0x7f85b0005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:06.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c6ffd700 1 --2- 172.21.2.101:0/87732376 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f85b004a7a0 0x7f85b004cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:06.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c6ffd700 1 --2- 172.21.2.101:0/87732376 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f85b004d510 0x7f85b004fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:06.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c6ffd700 1 -- 172.21.2.101:0/87732376 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b00502e0 con 0x7f85b0003a40 2021-02-10T11:16:06.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c6ffd700 1 -- 172.21.2.101:0/87732376 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b0050500 con 0x7f85b004d510 2021-02-10T11:16:06.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c6ffd700 1 -- 172.21.2.101:0/87732376 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b0050720 con 0x7f85b004a7a0 2021-02-10T11:16:06.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c7fff700 1 --2- 172.21.2.101:0/87732376 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85b0003a40 0x7f85b0005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:06.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c7fff700 1 -- 172.21.2.101:0/87732376 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f85b004a7a0 msgr2=0x7f85b004cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:16:06.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c77fe700 1 --2- 172.21.2.101:0/87732376 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f85b004a7a0 0x7f85b004cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:06.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c7fff700 1 --2- 172.21.2.101:0/87732376 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f85b004a7a0 0x7f85b004cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:06.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c7fff700 1 -- 172.21.2.101:0/87732376 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f85b004d510 msgr2=0x7f85b004fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:16:06.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c7fff700 1 --2- 172.21.2.101:0/87732376 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f85b004d510 0x7f85b004fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:06.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c7fff700 1 -- 172.21.2.101:0/87732376 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b00509c0 con 0x7f85b0003a40 2021-02-10T11:16:06.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c77fe700 1 --2- 172.21.2.101:0/87732376 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f85b004a7a0 0x7f85b004cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:16:06.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.098 7f85c7fff700 1 --2- 172.21.2.101:0/87732376 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85b0003a40 0x7f85b0005ef0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f85b8006ac0 tx=0x7f85b8009ab0).ready entity=mon.0 client_cookie=cd44b0237c0077ce server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:06.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.099 7f85c6ffd700 1 -- 172.21.2.101:0/87732376 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f85b800c0c0 con 0x7f85b0003a40 2021-02-10T11:16:06.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.099 7f85c6ffd700 1 -- 172.21.2.101:0/87732376 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b800c380 con 0x7f85b0003a40 2021-02-10T11:16:06.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.099 7f85c6ffd700 1 -- 172.21.2.101:0/87732376 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f85b800c8a0 con 0x7f85b0003a40 2021-02-10T11:16:06.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.099 7f85ce53e700 1 -- 172.21.2.101:0/87732376 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85b0003a40 msgr2=0x7f85b0005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:06.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.099 7f85ce53e700 1 --2- 172.21.2.101:0/87732376 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85b0003a40 0x7f85b0005ef0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f85b8006ac0 tx=0x7f85b8009ab0).stop 2021-02-10T11:16:06.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.099 7f85ccb5b700 1 -- 172.21.2.101:0/87732376 reap_dead start 2021-02-10T11:16:06.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.099 7f85ce53e700 1 -- 172.21.2.101:0/87732376 shutdown_connections 2021-02-10T11:16:06.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.099 7f85ce53e700 1 -- 172.21.2.101:0/87732376 >> 172.21.2.101:0/87732376 conn(0x7f85c80ce3e0 msgr2=0x7f85c80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:06.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.099 7f85ce53e700 1 -- 172.21.2.101:0/87732376 shutdown_connections 2021-02-10T11:16:06.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.099 7f85ce53e700 1 -- 172.21.2.101:0/87732376 wait complete. 2021-02-10T11:16:06.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.114 7f85ce53e700 1 Processor -- start 2021-02-10T11:16:06.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.114 7f85ce53e700 1 -- start start 2021-02-10T11:16:06.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.114 7f85ce53e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f85c809b930 0x7f85c80b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:06.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.114 7f85ce53e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85c80b8410 0x7f85c80b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:06.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.114 7f85ce53e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f85c80c1f80 0x7f85c80c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:06.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.114 7f85ce53e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c80c1a00 con 0x7f85c80b8410 2021-02-10T11:16:06.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.114 7f85ce53e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c80b8e30 con 0x7f85c80c1f80 2021-02-10T11:16:06.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.114 7f85ce53e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c80b9050 con 0x7f85c809b930 2021-02-10T11:16:06.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.114 7f85c77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85c80b8410 0x7f85c80b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:06.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.114 7f85c77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85c80b8410 0x7f85c80b8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45826/0 (socket says 172.21.2.101:45826) 2021-02-10T11:16:06.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.114 7f85c77fe700 1 -- 172.21.2.101:0/931938975 learned_addr learned my addr 172.21.2.101:0/931938975 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:16:06.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.115 7f85c77fe700 1 -- 172.21.2.101:0/931938975 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f85c809b930 msgr2=0x7f85c80b7c20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:16:06.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.115 7f85c77fe700 1 --2- 172.21.2.101:0/931938975 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f85c809b930 0x7f85c80b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:06.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.115 7f85c77fe700 1 -- 172.21.2.101:0/931938975 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f85c80c1f80 msgr2=0x7f85c80c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:06.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.115 7f85c77fe700 1 --2- 172.21.2.101:0/931938975 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f85c80c1f80 0x7f85c80c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:06.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.115 7f85c77fe700 1 -- 172.21.2.101:0/931938975 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c80c29a0 con 0x7f85c80b8410 2021-02-10T11:16:06.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.115 7f85c77fe700 1 --2- 172.21.2.101:0/931938975 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85c80b8410 0x7f85c80b8810 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f85bc001c10 tx=0x7f85bc00a8f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:06.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.115 7f85c4ff9700 1 -- 172.21.2.101:0/931938975 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f85bc000c90 con 0x7f85c80b8410 2021-02-10T11:16:06.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.115 7f85c4ff9700 1 -- 172.21.2.101:0/931938975 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85bc00ace0 con 0x7f85c80b8410 2021-02-10T11:16:06.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.115 7f85ce53e700 1 -- 172.21.2.101:0/931938975 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c80c29a0 con 0x7f85c80b8410 2021-02-10T11:16:06.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.115 7f85c4ff9700 1 -- 172.21.2.101:0/931938975 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f85bc0045d0 con 0x7f85c80b8410 2021-02-10T11:16:06.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.115 7f85ce53e700 1 -- 172.21.2.101:0/931938975 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c80c29a0 con 0x7f85c80b8410 2021-02-10T11:16:06.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.116 7f85c4ff9700 1 -- 172.21.2.101:0/931938975 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f85bc004140 con 0x7f85c80b8410 2021-02-10T11:16:06.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.116 7f85c4ff9700 1 --2- 172.21.2.101:0/931938975 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f85a8018730 0x7f85a801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:06.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.116 7f85c4ff9700 1 -- 172.21.2.101:0/931938975 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f85bc032620 con 0x7f85c80b8410 2021-02-10T11:16:06.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.116 7f85c7fff700 1 --2- 172.21.2.101:0/931938975 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f85a8018730 0x7f85a801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:06.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.116 7f85c7fff700 1 --2- 172.21.2.101:0/931938975 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f85a8018730 0x7f85a801ac10 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:06.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.147 7f85ce53e700 1 -- 172.21.2.101:0/931938975 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c8105b30 con 0x7f85c80b8410 2021-02-10T11:16:06.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.149 7f85c4ff9700 1 -- 172.21.2.101:0/931938975 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f85bc000c90 con 0x7f85c80b8410 2021-02-10T11:16:06.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.277 7f85ce53e700 1 -- 172.21.2.101:0/931938975 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839243", "target": ["mgr", ""]}) v1 -- 0x7f85c8105b30 con 0x7f85a8018730 2021-02-10T11:16:06.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.301 7f85c4ff9700 1 -- 172.21.2.101:0/931938975 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f85b8006740 con 0x7f85a8018730 2021-02-10T11:16:06.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.310 7f85ce53e700 1 -- 172.21.2.101:0/931938975 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f85a8018730 msgr2=0x7f85a801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:06.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.310 7f85ce53e700 1 --2- 172.21.2.101:0/931938975 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f85a8018730 0x7f85a801ac10 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:06.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.310 7f85ce53e700 1 -- 172.21.2.101:0/931938975 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85c80b8410 msgr2=0x7f85c80b8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:06.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.310 7f85ce53e700 1 --2- 172.21.2.101:0/931938975 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85c80b8410 0x7f85c80b8810 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f85bc001c10 tx=0x7f85bc00a8f0).stop 2021-02-10T11:16:06.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.310 7f85ce53e700 1 -- 172.21.2.101:0/931938975 shutdown_connections 2021-02-10T11:16:06.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.310 7f85ce53e700 1 --2- 172.21.2.101:0/931938975 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f85c80c1f80 0x7f85c80c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:06.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.310 7f85ce53e700 1 --2- 172.21.2.101:0/931938975 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85c80b8410 0x7f85c80b8810 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:06.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.310 7f85ce53e700 1 --2- 172.21.2.101:0/931938975 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f85a8018730 0x7f85a801ac10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:06.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.310 7f85ce53e700 1 --2- 172.21.2.101:0/931938975 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f85c809b930 0x7f85c80b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:06.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.310 7f85ce53e700 1 -- 172.21.2.101:0/931938975 >> 172.21.2.101:0/931938975 conn(0x7f85c8094e10 msgr2=0x7f85c80a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:06.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.310 7f85ce53e700 1 -- 172.21.2.101:0/931938975 shutdown_connections 2021-02-10T11:16:06.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.311 7f85ce53e700 1 -- 172.21.2.101:0/931938975 wait complete. 2021-02-10T11:16:06.323 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839243 2021-02-10T11:16:06.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.732 7ff0bb54e700 1 Processor -- start 2021-02-10T11:16:06.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.732 7ff0bb54e700 1 -- start start 2021-02-10T11:16:06.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.733 7ff0bb54e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff0b41b0470 0x7ff0b41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:06.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.733 7ff0bb54e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0b40ef610 0x7ff0b40efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:06.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.733 7ff0bb54e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b40f0b50 con 0x7ff0b40f0030 2021-02-10T11:16:06.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.733 7ff0bb54e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b40e52e0 con 0x7ff0b41b0470 2021-02-10T11:16:06.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.733 7ff0bb54e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b40e54d0 con 0x7ff0b40ef610 2021-02-10T11:16:06.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.733 7ff0b9b6b700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff0b40f0030 0x7ff0b40f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46328/0 (socket says 172.21.2.101:46328) 2021-02-10T11:16:06.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.733 7ff0b9b6b700 1 -- 172.21.2.101:0/732614115 learned_addr learned my addr 172.21.2.101:0/732614115 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:16:06.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.733 7ff0b936a700 1 --2- 172.21.2.101:0/732614115 >> v2:172.21.2.101:3300/0 conn(0x7ff0b41b0470 0x7ff0b41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:06.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.733 7ff0a7fff700 1 -- 172.21.2.101:0/732614115 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2356653343 0 0) 0x7ff0ac001500 con 0x7ff0b40f0030 2021-02-10T11:16:06.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.733 7ff0b936a700 1 -- 172.21.2.101:0/732614115 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0b40ef610 msgr2=0x7ff0b40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:16:06.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.733 7ff0b936a700 1 --2- 172.21.2.101:0/732614115 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0b40ef610 0x7ff0b40efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:06.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.733 7ff0b936a700 1 -- 172.21.2.101:0/732614115 >> v1:172.21.2.101:6789/0 conn(0x7ff0b40f0030 legacy=0x7ff0b40f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:06.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.733 7ff0b936a700 1 -- 172.21.2.101:0/732614115 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b40f0430 con 0x7ff0b41b0470 2021-02-10T11:16:06.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.734 7ff0b936a700 1 --2- 172.21.2.101:0/732614115 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0b41b0470 0x7ff0b41b0870 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7ff0a8006650 tx=0x7ff0a8009a80).ready entity=mon.0 client_cookie=58854cc8ccd46e80 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:06.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.734 7ff0a7fff700 1 -- 172.21.2.101:0/732614115 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0a80009b0 con 0x7ff0b41b0470 2021-02-10T11:16:06.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.734 7ff0a7fff700 1 -- 172.21.2.101:0/732614115 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a800c0c0 con 0x7ff0b41b0470 2021-02-10T11:16:06.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.734 7ff0a7fff700 1 -- 172.21.2.101:0/732614115 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0a800c5a0 con 0x7ff0b41b0470 2021-02-10T11:16:06.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.734 7ff0bb54e700 1 -- 172.21.2.101:0/732614115 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0b41b0470 msgr2=0x7ff0b41b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:06.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.734 7ff0bb54e700 1 --2- 172.21.2.101:0/732614115 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0b41b0470 0x7ff0b41b0870 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7ff0a8006650 tx=0x7ff0a8009a80).stop 2021-02-10T11:16:06.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.734 7ff0bb54e700 1 -- 172.21.2.101:0/732614115 shutdown_connections 2021-02-10T11:16:06.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.734 7ff0bb54e700 1 --2- 172.21.2.101:0/732614115 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0b40ef610 0x7ff0b40efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:06.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.734 7ff0bb54e700 1 --2- 172.21.2.101:0/732614115 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0b41b0470 0x7ff0b41b0870 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:06.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.734 7ff0bb54e700 1 -- 172.21.2.101:0/732614115 >> 172.21.2.101:0/732614115 conn(0x7ff0b40ce3e0 msgr2=0x7ff0b40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:06.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.735 7ff0bb54e700 1 -- 172.21.2.101:0/732614115 shutdown_connections 2021-02-10T11:16:06.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.735 7ff0bb54e700 1 -- 172.21.2.101:0/732614115 wait complete. 2021-02-10T11:16:06.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.749 7ff0bb54e700 1 Processor -- start 2021-02-10T11:16:06.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.749 7ff0bb54e700 1 -- start start 2021-02-10T11:16:06.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.750 7ff0bb54e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0b40bf910 0x7ff0b40bfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:06.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.750 7ff0bb54e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0b40c0330 0x7ff0b40c0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:06.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.750 7ff0bb54e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0b40c13d0 0x7ff0b40c17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:06.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.750 7ff0bb54e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b4109930 con 0x7ff0b40c0330 2021-02-10T11:16:06.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.750 7ff0bb54e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b4109b50 con 0x7ff0b40c13d0 2021-02-10T11:16:06.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.750 7ff0bb54e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b40c1df0 con 0x7ff0b40bf910 2021-02-10T11:16:06.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.750 7ff0b8b69700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0b40c0330 0x7ff0b40c0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:06.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.750 7ff0b8b69700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0b40c0330 0x7ff0b40c0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45838/0 (socket says 172.21.2.101:45838) 2021-02-10T11:16:06.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.750 7ff0b8b69700 1 -- 172.21.2.101:0/2453259904 learned_addr learned my addr 172.21.2.101:0/2453259904 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:16:06.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.750 7ff0b8b69700 1 -- 172.21.2.101:0/2453259904 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0b40bf910 msgr2=0x7ff0b40bfd10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:16:06.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.750 7ff0b8b69700 1 --2- 172.21.2.101:0/2453259904 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0b40bf910 0x7ff0b40bfd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:06.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.750 7ff0b8b69700 1 -- 172.21.2.101:0/2453259904 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0b40c13d0 msgr2=0x7ff0b40c17d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:16:06.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.750 7ff0b8b69700 1 --2- 172.21.2.101:0/2453259904 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0b40c13d0 0x7ff0b40c17d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:06.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.750 7ff0b8b69700 1 -- 172.21.2.101:0/2453259904 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b40c1fd0 con 0x7ff0b40c0330 2021-02-10T11:16:06.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.750 7ff0b8b69700 1 --2- 172.21.2.101:0/2453259904 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0b40c0330 0x7ff0b40c0730 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7ff0b0006f00 tx=0x7ff0b000b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:06.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.750 7ff0a5ffb700 1 -- 172.21.2.101:0/2453259904 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0b000bd30 con 0x7ff0b40c0330 2021-02-10T11:16:06.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.750 7ff0a5ffb700 1 -- 172.21.2.101:0/2453259904 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b0000c90 con 0x7ff0b40c0330 2021-02-10T11:16:06.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.750 7ff0bb54e700 1 -- 172.21.2.101:0/2453259904 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b40c1fd0 con 0x7ff0b40c0330 2021-02-10T11:16:06.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.750 7ff0bb54e700 1 -- 172.21.2.101:0/2453259904 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b40c1fd0 con 0x7ff0b40c0330 2021-02-10T11:16:06.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.750 7ff0a5ffb700 1 -- 172.21.2.101:0/2453259904 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0b00022a0 con 0x7ff0b40c0330 2021-02-10T11:16:06.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.751 7ff0a5ffb700 1 -- 172.21.2.101:0/2453259904 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff0b0005310 con 0x7ff0b40c0330 2021-02-10T11:16:06.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.751 7ff0a5ffb700 1 --2- 172.21.2.101:0/2453259904 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff094018730 0x7ff09401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:06.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.751 7ff0a5ffb700 1 -- 172.21.2.101:0/2453259904 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff0b0032770 con 0x7ff0b40c0330 2021-02-10T11:16:06.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.751 7ff0b936a700 1 --2- 172.21.2.101:0/2453259904 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff094018730 0x7ff09401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:06.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.751 7ff0b936a700 1 --2- 172.21.2.101:0/2453259904 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff094018730 0x7ff09401ac10 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:06.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.782 7ff0bb54e700 1 -- 172.21.2.101:0/2453259904 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b40f0b50 con 0x7ff0b40c0330 2021-02-10T11:16:06.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.785 7ff0a5ffb700 1 -- 172.21.2.101:0/2453259904 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff0b0006b90 con 0x7ff0b40c0330 2021-02-10T11:16:06.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.914 7ff0bb54e700 1 -- 172.21.2.101:0/2453259904 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839243", "target": ["mgr", ""]}) v1 -- 0x7ff0b40f0b50 con 0x7ff094018730 2021-02-10T11:16:06.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.917 7ff0a5ffb700 1 -- 172.21.2.101:0/2453259904 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7ff0a80093a0 con 0x7ff094018730 2021-02-10T11:16:06.922 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839243/179750e3-297d-49a1-95e1-e7ae01642eaf 2021-02-10T11:16:06.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.919 7ff0bb54e700 1 -- 172.21.2.101:0/2453259904 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff094018730 msgr2=0x7ff09401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:06.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.919 7ff0bb54e700 1 --2- 172.21.2.101:0/2453259904 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff094018730 0x7ff09401ac10 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:06.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.919 7ff0bb54e700 1 -- 172.21.2.101:0/2453259904 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0b40c0330 msgr2=0x7ff0b40c0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:06.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.919 7ff0bb54e700 1 --2- 172.21.2.101:0/2453259904 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0b40c0330 0x7ff0b40c0730 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7ff0b0006f00 tx=0x7ff0b000b940).stop 2021-02-10T11:16:06.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.919 7ff0bb54e700 1 -- 172.21.2.101:0/2453259904 shutdown_connections 2021-02-10T11:16:06.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.919 7ff0bb54e700 1 --2- 172.21.2.101:0/2453259904 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0b40c13d0 0x7ff0b40c17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:06.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.919 7ff0bb54e700 1 --2- 172.21.2.101:0/2453259904 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0b40c0330 0x7ff0b40c0730 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:06.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.919 7ff0bb54e700 1 --2- 172.21.2.101:0/2453259904 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff094018730 0x7ff09401ac10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:06.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.919 7ff0bb54e700 1 --2- 172.21.2.101:0/2453259904 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0b40bf910 0x7ff0b40bfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:06.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.919 7ff0bb54e700 1 -- 172.21.2.101:0/2453259904 >> 172.21.2.101:0/2453259904 conn(0x7ff0b40bf4e0 msgr2=0x7ff0b4191190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:06.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.919 7ff0bb54e700 1 -- 172.21.2.101:0/2453259904 shutdown_connections 2021-02-10T11:16:06.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:06.919 7ff0bb54e700 1 -- 172.21.2.101:0/2453259904 wait complete. 2021-02-10T11:16:06.932 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839243 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839243/179750e3-297d-49a1-95e1-e7ae01642eaf 2021-02-10T11:16:06.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:06.932 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:06.933 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:07.267 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:07.267 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:07.267 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839243/179750e3-297d-49a1-95e1-e7ae01642eaf/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:07.646 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:07.646 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:07.646 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236775 s, 44.3 MB/s 2021-02-10T11:16:07.647 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:07.647 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:07.647 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:07.981 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:07.982 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:07.982 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839243/179750e3-297d-49a1-95e1-e7ae01642eaf/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:08.358 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:08.358 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:08.359 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251852 s, 41.6 MB/s 2021-02-10T11:16:08.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:08.365 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:08.365 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:08.700 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:08.700 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:08.700 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839243/179750e3-297d-49a1-95e1-e7ae01642eaf/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:09.072 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:09.072 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:09.072 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024612 s, 42.6 MB/s 2021-02-10T11:16:09.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:09.074 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:09.074 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:09.410 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:09.497 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:09.499 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839243/179750e3-297d-49a1-95e1-e7ae01642eaf/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:09.885 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:09.885 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:09.885 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0254008 s, 41.3 MB/s 2021-02-10T11:16:09.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:09.886 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:09.887 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:10.221 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:10.221 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:10.221 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839243/179750e3-297d-49a1-95e1-e7ae01642eaf/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:10.595 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:10.595 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:10.595 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245644 s, 42.7 MB/s 2021-02-10T11:16:10.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:10.596 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:10.597 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:10.930 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:10.931 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:10.931 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839243/179750e3-297d-49a1-95e1-e7ae01642eaf/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:11.303 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:11.303 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:11.304 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231547 s, 45.3 MB/s 2021-02-10T11:16:11.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:11.305 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:11.305 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:11.639 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:11.639 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:11.639 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839243/179750e3-297d-49a1-95e1-e7ae01642eaf/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:12.014 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:14.553 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:14.553 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249944 s, 42.0 MB/s 2021-02-10T11:16:14.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:14.554 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:14.554 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:14.893 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:14.894 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:14.894 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839243/179750e3-297d-49a1-95e1-e7ae01642eaf/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:15.176 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:16:15.179 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:16:15.268 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:15.269 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:15.269 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231963 s, 45.2 MB/s 2021-02-10T11:16:15.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:15.270 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:15.270 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:15.511 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:15.511 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:15.512 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839243/179750e3-297d-49a1-95e1-e7ae01642eaf/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:15.886 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:15.887 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:15.887 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0250246 s, 41.9 MB/s 2021-02-10T11:16:15.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:15.888 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:15.888 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:16.222 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:16.222 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:16.222 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839243/179750e3-297d-49a1-95e1-e7ae01642eaf/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:16.596 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:16.597 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:16.597 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235456 s, 44.5 MB/s 2021-02-10T11:16:16.598 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839244 2021-02-10T11:16:17.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.004 7f44b616e700 1 Processor -- start 2021-02-10T11:16:17.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.004 7f44b616e700 1 -- start start 2021-02-10T11:16:17.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.005 7f44b616e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44b01b0470 0x7f44b01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:17.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.005 7f44b616e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f44b00ef610 0x7f44b00efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:17.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.005 7f44b616e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f44b00f0030 0x7f44b00f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:17.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.005 7f44b616e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b00e5ea0 con 0x7f44b00f0030 2021-02-10T11:16:17.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.005 7f44b616e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b00f0b50 con 0x7f44b00ef610 2021-02-10T11:16:17.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.005 7f44b616e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b00e52e0 con 0x7f44b01b0470 2021-02-10T11:16:17.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.005 7f44affff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f44b00f0030 0x7f44b00f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:17.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.005 7f44affff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f44b00f0030 0x7f44b00f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45842/0 (socket says 172.21.2.101:45842) 2021-02-10T11:16:17.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.005 7f44affff700 1 -- 172.21.2.101:0/3092339197 learned_addr learned my addr 172.21.2.101:0/3092339197 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:16:17.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.005 7f44affff700 1 -- 172.21.2.101:0/3092339197 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44b01b0470 msgr2=0x7f44b01b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:16:17.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.005 7f44aeffd700 1 --2- 172.21.2.101:0/3092339197 >> v2:172.21.2.108:3300/0 conn(0x7f44b00ef610 0x7f44b00efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:17.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.005 7f44af7fe700 1 --2- 172.21.2.101:0/3092339197 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44b01b0470 0x7f44b01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:17.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.005 7f44affff700 1 --2- 172.21.2.101:0/3092339197 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44b01b0470 0x7f44b01b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:17.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.005 7f44affff700 1 -- 172.21.2.101:0/3092339197 >> v2:172.21.2.108:3300/0 conn(0x7f44b00ef610 msgr2=0x7f44b00efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:16:17.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.005 7f44affff700 1 --2- 172.21.2.101:0/3092339197 >> v2:172.21.2.108:3300/0 conn(0x7f44b00ef610 0x7f44b00efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:17.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.005 7f44affff700 1 -- 172.21.2.101:0/3092339197 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b00f10d0 con 0x7f44b00f0030 2021-02-10T11:16:17.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.005 7f44af7fe700 1 --2- 172.21.2.101:0/3092339197 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44b01b0470 0x7f44b01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:16:17.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.005 7f44aeffd700 1 --2- 172.21.2.101:0/3092339197 >> v2:172.21.2.108:3300/0 conn(0x7f44b00ef610 0x7f44b00efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:16:17.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.005 7f44affff700 1 --2- 172.21.2.101:0/3092339197 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44b00f0030 0x7f44b00f0430 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f44a8006f00 tx=0x7f44a8009f10).ready entity=mon.0 client_cookie=8506982f53ca5e94 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:17.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.006 7f44ae7fc700 1 -- 172.21.2.101:0/3092339197 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f44a800b3b0 con 0x7f44b00f0030 2021-02-10T11:16:17.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.006 7f44ae7fc700 1 -- 172.21.2.101:0/3092339197 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a800b6b0 con 0x7f44b00f0030 2021-02-10T11:16:17.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.006 7f44ae7fc700 1 -- 172.21.2.101:0/3092339197 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f44a800bbd0 con 0x7f44b00f0030 2021-02-10T11:16:17.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.006 7f44b616e700 1 -- 172.21.2.101:0/3092339197 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44b00f0030 msgr2=0x7f44b00f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:17.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.006 7f44b616e700 1 --2- 172.21.2.101:0/3092339197 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44b00f0030 0x7f44b00f0430 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f44a8006f00 tx=0x7f44a8009f10).stop 2021-02-10T11:16:17.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.006 7f44b616e700 1 -- 172.21.2.101:0/3092339197 shutdown_connections 2021-02-10T11:16:17.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.006 7f44b616e700 1 --2- 172.21.2.101:0/3092339197 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44b00f0030 0x7f44b00f0430 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:17.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.006 7f44b616e700 1 --2- 172.21.2.101:0/3092339197 >> v2:172.21.2.108:3300/0 conn(0x7f44b00ef610 0x7f44b00efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:17.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.006 7f44b616e700 1 --2- 172.21.2.101:0/3092339197 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44b01b0470 0x7f44b01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:17.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.006 7f44b616e700 1 -- 172.21.2.101:0/3092339197 >> 172.21.2.101:0/3092339197 conn(0x7f44b00ce3e0 msgr2=0x7f44b00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:17.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.006 7f44b616e700 1 -- 172.21.2.101:0/3092339197 shutdown_connections 2021-02-10T11:16:17.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.006 7f44b616e700 1 -- 172.21.2.101:0/3092339197 wait complete. 2021-02-10T11:16:17.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.072 7f44b616e700 1 Processor -- start 2021-02-10T11:16:17.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.073 7f44b616e700 1 -- start start 2021-02-10T11:16:17.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.073 7f44b616e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44b00b7c20 0x7f44b00b8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:17.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.073 7f44b616e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44b00b8a30 0x7f44b00c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:17.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.073 7f44b616e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44b00c25a0 0x7f44b00c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:17.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.073 7f44b616e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b00c1a80 con 0x7f44b00b8a30 2021-02-10T11:16:17.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.073 7f44b616e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b00b8020 con 0x7f44b00b7c20 2021-02-10T11:16:17.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.073 7f44b616e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b009b9c0 con 0x7f44b00c25a0 2021-02-10T11:16:17.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.073 7f44aeffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44b00b8a30 0x7f44b00c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:17.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.073 7f44af7fe700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44b00b7c20 0x7f44b00b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:17.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.073 7f44aeffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44b00b8a30 0x7f44b00c1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45846/0 (socket says 172.21.2.101:45846) 2021-02-10T11:16:17.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.073 7f44aeffd700 1 -- 172.21.2.101:0/2091341527 learned_addr learned my addr 172.21.2.101:0/2091341527 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:16:17.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.073 7f44affff700 1 --2- 172.21.2.101:0/2091341527 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44b00c25a0 0x7f44b00c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:17.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.074 7f44aeffd700 1 -- 172.21.2.101:0/2091341527 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44b00c25a0 msgr2=0x7f44b00c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:17.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.074 7f44aeffd700 1 --2- 172.21.2.101:0/2091341527 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44b00c25a0 0x7f44b00c29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:17.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.074 7f44aeffd700 1 -- 172.21.2.101:0/2091341527 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44b00b7c20 msgr2=0x7f44b00b8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:17.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.074 7f44aeffd700 1 --2- 172.21.2.101:0/2091341527 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44b00b7c20 0x7f44b00b8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:17.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.074 7f44aeffd700 1 -- 172.21.2.101:0/2091341527 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b00b8e30 con 0x7f44b00b8a30 2021-02-10T11:16:17.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.074 7f44aeffd700 1 --2- 172.21.2.101:0/2091341527 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44b00b8a30 0x7f44b00c1f80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f4498006f20 tx=0x7f449800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:17.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.074 7f44a7fff700 1 -- 172.21.2.101:0/2091341527 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f449800bd80 con 0x7f44b00b8a30 2021-02-10T11:16:17.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.074 7f44a7fff700 1 -- 172.21.2.101:0/2091341527 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4498000c90 con 0x7f44b00b8a30 2021-02-10T11:16:17.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.074 7f44a7fff700 1 -- 172.21.2.101:0/2091341527 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4498002220 con 0x7f44b00b8a30 2021-02-10T11:16:17.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.074 7f44b616e700 1 -- 172.21.2.101:0/2091341527 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b00b8e30 con 0x7f44b00b8a30 2021-02-10T11:16:17.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.074 7f44b616e700 1 -- 172.21.2.101:0/2091341527 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b00c32b0 con 0x7f44b00b8a30 2021-02-10T11:16:17.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.075 7f44a7fff700 1 -- 172.21.2.101:0/2091341527 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f449800a000 con 0x7f44b00b8a30 2021-02-10T11:16:17.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.075 7f44a7fff700 1 --2- 172.21.2.101:0/2091341527 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44940186d0 0x7f449401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:17.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.075 7f44a7fff700 1 -- 172.21.2.101:0/2091341527 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4498032670 con 0x7f44b00b8a30 2021-02-10T11:16:17.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.075 7f44af7fe700 1 --2- 172.21.2.101:0/2091341527 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44940186d0 0x7f449401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:17.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.075 7f44af7fe700 1 --2- 172.21.2.101:0/2091341527 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44940186d0 0x7f449401abd0 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:17.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.105 7f44b616e700 1 -- 172.21.2.101:0/2091341527 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b0105b30 con 0x7f44b00b8a30 2021-02-10T11:16:17.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.107 7f44a7fff700 1 -- 172.21.2.101:0/2091341527 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f44980009b0 con 0x7f44b00b8a30 2021-02-10T11:16:17.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.233 7f44b616e700 1 -- 172.21.2.101:0/2091341527 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839244", "target": ["mgr", ""]}) v1 -- 0x7f44b0105b30 con 0x7f44940186d0 2021-02-10T11:16:17.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.256 7f44a7fff700 1 -- 172.21.2.101:0/2091341527 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f44a0006540 con 0x7f44940186d0 2021-02-10T11:16:17.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.266 7f44b616e700 1 -- 172.21.2.101:0/2091341527 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44940186d0 msgr2=0x7f449401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:17.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.266 7f44b616e700 1 --2- 172.21.2.101:0/2091341527 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44940186d0 0x7f449401abd0 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:17.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.266 7f44b616e700 1 -- 172.21.2.101:0/2091341527 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44b00b8a30 msgr2=0x7f44b00c1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:17.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.266 7f44b616e700 1 --2- 172.21.2.101:0/2091341527 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44b00b8a30 0x7f44b00c1f80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f4498006f20 tx=0x7f449800b990).stop 2021-02-10T11:16:17.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.266 7f44b616e700 1 -- 172.21.2.101:0/2091341527 shutdown_connections 2021-02-10T11:16:17.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.266 7f44b616e700 1 --2- 172.21.2.101:0/2091341527 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44940186d0 0x7f449401abd0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:17.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.266 7f44b616e700 1 --2- 172.21.2.101:0/2091341527 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44b00c25a0 0x7f44b00c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:17.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.266 7f44b616e700 1 --2- 172.21.2.101:0/2091341527 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44b00b8a30 0x7f44b00c1f80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:17.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.266 7f44b616e700 1 --2- 172.21.2.101:0/2091341527 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44b00b7c20 0x7f44b00b8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:17.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.266 7f44b616e700 1 -- 172.21.2.101:0/2091341527 >> 172.21.2.101:0/2091341527 conn(0x7f44b0094e10 msgr2=0x7f44b00a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:17.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.266 7f44b616e700 1 -- 172.21.2.101:0/2091341527 shutdown_connections 2021-02-10T11:16:17.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.266 7f44b616e700 1 -- 172.21.2.101:0/2091341527 wait complete. 2021-02-10T11:16:17.279 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839244 2021-02-10T11:16:17.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.686 7fb320026700 1 Processor -- start 2021-02-10T11:16:17.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.686 7fb320026700 1 -- start start 2021-02-10T11:16:17.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.687 7fb320026700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb3181b0870 0x7fb3180f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:17.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.687 7fb320026700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3181b0c70 con 0x7fb3181b0470 2021-02-10T11:16:17.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.687 7fb320026700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3180f0b50 con 0x7fb3180efe00 2021-02-10T11:16:17.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.687 7fb320026700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3180e5ea0 con 0x7fb3181b0870 2021-02-10T11:16:17.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.687 7fb31de42700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb3181b0470 0x7fb3180ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46348/0 (socket says 172.21.2.101:46348) 2021-02-10T11:16:17.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.687 7fb31de42700 1 -- 172.21.2.101:0/1197324844 learned_addr learned my addr 172.21.2.101:0/1197324844 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:16:17.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.687 7fb31d641700 1 --2- 172.21.2.101:0/1197324844 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb3181b0870 0x7fb3180f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:17.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.688 7fb31ce40700 1 -- 172.21.2.101:0/1197324844 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3435108969 0 0) 0x7fb30c000ce0 con 0x7fb3181b0470 2021-02-10T11:16:17.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.688 7fb31ce40700 1 -- 172.21.2.101:0/1197324844 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb304002fe0 con 0x7fb3181b0470 2021-02-10T11:16:17.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.688 7fb31d641700 1 -- 172.21.2.101:0/1197324844 >> v1:172.21.2.108:6789/0 conn(0x7fb3180efe00 legacy=0x7fb3180f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:17.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.688 7fb31ce40700 1 -- 172.21.2.101:0/1197324844 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800107447 0 0) 0x7fb310001400 con 0x7fb3180efe00 2021-02-10T11:16:17.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.688 7fb31d641700 1 -- 172.21.2.101:0/1197324844 >> v1:172.21.2.101:6789/0 conn(0x7fb3181b0470 legacy=0x7fb3180ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:17.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.688 7fb31d641700 1 -- 172.21.2.101:0/1197324844 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3180e52e0 con 0x7fb3181b0870 2021-02-10T11:16:17.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.688 7fb31d641700 1 --2- 172.21.2.101:0/1197324844 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb3181b0870 0x7fb3180f20e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fb314006650 tx=0x7fb314009a80).ready entity=mon.2 client_cookie=16f4896ac501d321 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:17.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.688 7fb31ce40700 1 -- 172.21.2.101:0/1197324844 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb31400c0c0 con 0x7fb3181b0870 2021-02-10T11:16:17.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.688 7fb31ce40700 1 -- 172.21.2.101:0/1197324844 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31400c340 con 0x7fb3181b0870 2021-02-10T11:16:17.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.689 7fb31ce40700 1 -- 172.21.2.101:0/1197324844 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb31400c860 con 0x7fb3181b0870 2021-02-10T11:16:17.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.689 7fb320026700 1 -- 172.21.2.101:0/1197324844 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb3181b0870 msgr2=0x7fb3180f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:17.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.689 7fb320026700 1 --2- 172.21.2.101:0/1197324844 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb3181b0870 0x7fb3180f20e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fb314006650 tx=0x7fb314009a80).stop 2021-02-10T11:16:17.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.689 7fb320026700 1 -- 172.21.2.101:0/1197324844 shutdown_connections 2021-02-10T11:16:17.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.689 7fb320026700 1 --2- 172.21.2.101:0/1197324844 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb3181b0870 0x7fb3180f20e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:17.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.689 7fb320026700 1 -- 172.21.2.101:0/1197324844 >> 172.21.2.101:0/1197324844 conn(0x7fb3180ce3e0 msgr2=0x7fb3180e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:17.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.689 7fb320026700 1 -- 172.21.2.101:0/1197324844 shutdown_connections 2021-02-10T11:16:17.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.689 7fb320026700 1 -- 172.21.2.101:0/1197324844 wait complete. 2021-02-10T11:16:17.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.704 7fb320026700 1 Processor -- start 2021-02-10T11:16:17.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.704 7fb320026700 1 -- start start 2021-02-10T11:16:17.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.704 7fb320026700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3181a42f0 0x7fb3181a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:17.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.704 7fb320026700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb318196170 0x7fb318196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:17.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.704 7fb320026700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb31808f5b0 0x7fb31808f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:17.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.704 7fb320026700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb318195bf0 con 0x7fb3181a42f0 2021-02-10T11:16:17.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.704 7fb320026700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3180c4130 con 0x7fb31808f5b0 2021-02-10T11:16:17.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.704 7fb320026700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3180a6e70 con 0x7fb318196170 2021-02-10T11:16:17.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.704 7fb31de42700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3181a42f0 0x7fb3181a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:17.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.704 7fb31de42700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3181a42f0 0x7fb3181a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45860/0 (socket says 172.21.2.101:45860) 2021-02-10T11:16:17.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.704 7fb31de42700 1 -- 172.21.2.101:0/405140121 learned_addr learned my addr 172.21.2.101:0/405140121 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:16:17.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.705 7fb31de42700 1 -- 172.21.2.101:0/405140121 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb318196170 msgr2=0x7fb318196570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:16:17.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.705 7fb31de42700 1 --2- 172.21.2.101:0/405140121 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb318196170 0x7fb318196570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:17.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.705 7fb31de42700 1 -- 172.21.2.101:0/405140121 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb31808f5b0 msgr2=0x7fb31808f9b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:16:17.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.705 7fb31de42700 1 --2- 172.21.2.101:0/405140121 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb31808f5b0 0x7fb31808f9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:17.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.705 7fb31de42700 1 -- 172.21.2.101:0/405140121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3180a6a80 con 0x7fb3181a42f0 2021-02-10T11:16:17.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.705 7fb31de42700 1 --2- 172.21.2.101:0/405140121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3181a42f0 0x7fb3181a46f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7fb30c00a580 tx=0x7fb30c00a920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:17.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.705 7fb30a7fc700 1 -- 172.21.2.101:0/405140121 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb30c001940 con 0x7fb3181a42f0 2021-02-10T11:16:17.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.705 7fb30a7fc700 1 -- 172.21.2.101:0/405140121 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c0043a0 con 0x7fb3181a42f0 2021-02-10T11:16:17.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.705 7fb30a7fc700 1 -- 172.21.2.101:0/405140121 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb30c0048c0 con 0x7fb3181a42f0 2021-02-10T11:16:17.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.705 7fb320026700 1 -- 172.21.2.101:0/405140121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3180a6a80 con 0x7fb3181a42f0 2021-02-10T11:16:17.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.705 7fb320026700 1 -- 172.21.2.101:0/405140121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3180a6a80 con 0x7fb3181a42f0 2021-02-10T11:16:17.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.706 7fb30a7fc700 1 -- 172.21.2.101:0/405140121 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb30c0018f0 con 0x7fb3181a42f0 2021-02-10T11:16:17.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.706 7fb30a7fc700 1 --2- 172.21.2.101:0/405140121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb300018730 0x7fb30001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:17.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.706 7fb30a7fc700 1 -- 172.21.2.101:0/405140121 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb30c032610 con 0x7fb3181a42f0 2021-02-10T11:16:17.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.706 7fb31d641700 1 --2- 172.21.2.101:0/405140121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb300018730 0x7fb30001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:17.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.706 7fb31d641700 1 --2- 172.21.2.101:0/405140121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb300018730 0x7fb30001ac10 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:17.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.737 7fb320026700 1 -- 172.21.2.101:0/405140121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3180f0b50 con 0x7fb3181a42f0 2021-02-10T11:16:17.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.739 7fb30a7fc700 1 -- 172.21.2.101:0/405140121 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb30c001870 con 0x7fb3181a42f0 2021-02-10T11:16:17.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.868 7fb320026700 1 -- 172.21.2.101:0/405140121 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839244", "target": ["mgr", ""]}) v1 -- 0x7fb3180f0b50 con 0x7fb300018730 2021-02-10T11:16:17.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.872 7fb30a7fc700 1 -- 172.21.2.101:0/405140121 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fb314006830 con 0x7fb300018730 2021-02-10T11:16:17.881 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839244/c8ec57b0-49ca-40eb-85f2-1207149f4417 2021-02-10T11:16:17.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.877 7fb320026700 1 -- 172.21.2.101:0/405140121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb300018730 msgr2=0x7fb30001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:17.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.877 7fb320026700 1 --2- 172.21.2.101:0/405140121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb300018730 0x7fb30001ac10 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:17.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.877 7fb320026700 1 -- 172.21.2.101:0/405140121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3181a42f0 msgr2=0x7fb3181a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:17.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.877 7fb320026700 1 --2- 172.21.2.101:0/405140121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3181a42f0 0x7fb3181a46f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7fb30c00a580 tx=0x7fb30c00a920).stop 2021-02-10T11:16:17.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.878 7fb320026700 1 -- 172.21.2.101:0/405140121 shutdown_connections 2021-02-10T11:16:17.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.878 7fb320026700 1 --2- 172.21.2.101:0/405140121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb300018730 0x7fb30001ac10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:17.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.878 7fb320026700 1 --2- 172.21.2.101:0/405140121 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb318196170 0x7fb318196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:17.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.878 7fb320026700 1 --2- 172.21.2.101:0/405140121 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb31808f5b0 0x7fb31808f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:17.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.878 7fb320026700 1 --2- 172.21.2.101:0/405140121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3181a42f0 0x7fb3181a46f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:17.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.878 7fb320026700 1 -- 172.21.2.101:0/405140121 >> 172.21.2.101:0/405140121 conn(0x7fb3180bf4e0 msgr2=0x7fb3180be160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:17.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.878 7fb320026700 1 -- 172.21.2.101:0/405140121 shutdown_connections 2021-02-10T11:16:17.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:17.878 7fb320026700 1 -- 172.21.2.101:0/405140121 wait complete. 2021-02-10T11:16:17.891 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839244 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839244/c8ec57b0-49ca-40eb-85f2-1207149f4417 2021-02-10T11:16:17.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:17.891 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:17.891 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:18.225 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:18.226 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:18.226 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839244/c8ec57b0-49ca-40eb-85f2-1207149f4417/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:18.605 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:18.605 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:18.605 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0223384 s, 46.9 MB/s 2021-02-10T11:16:18.606 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:18.606 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:18.606 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:18.940 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:18.941 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:18.941 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839244/c8ec57b0-49ca-40eb-85f2-1207149f4417/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:19.314 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:19.314 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:19.314 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246025 s, 42.6 MB/s 2021-02-10T11:16:19.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:19.315 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:19.315 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:19.650 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:19.650 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:19.650 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839244/c8ec57b0-49ca-40eb-85f2-1207149f4417/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:20.024 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:20.024 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:20.024 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241349 s, 43.4 MB/s 2021-02-10T11:16:20.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:20.025 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:20.026 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:20.361 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:20.361 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:20.362 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839244/c8ec57b0-49ca-40eb-85f2-1207149f4417/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:20.737 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:20.737 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:20.737 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246017 s, 42.6 MB/s 2021-02-10T11:16:20.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:20.739 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:20.739 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:21.072 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:21.072 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:21.073 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839244/c8ec57b0-49ca-40eb-85f2-1207149f4417/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:21.447 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:21.448 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:21.448 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245707 s, 42.7 MB/s 2021-02-10T11:16:21.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:21.449 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:21.449 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:21.783 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:21.783 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:21.783 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839244/c8ec57b0-49ca-40eb-85f2-1207149f4417/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:22.158 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:22.158 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:22.158 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251794 s, 41.6 MB/s 2021-02-10T11:16:22.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:22.159 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:22.159 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:22.494 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:22.494 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:22.494 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839244/c8ec57b0-49ca-40eb-85f2-1207149f4417/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:22.870 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:22.870 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:22.870 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244697 s, 42.9 MB/s 2021-02-10T11:16:22.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:22.871 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:22.871 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:23.205 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:23.205 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:23.205 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839244/c8ec57b0-49ca-40eb-85f2-1207149f4417/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:23.578 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:23.579 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:23.579 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0227885 s, 46.0 MB/s 2021-02-10T11:16:23.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:23.580 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:23.580 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:23.914 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:23.915 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:23.915 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839244/c8ec57b0-49ca-40eb-85f2-1207149f4417/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:24.288 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:24.288 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:24.288 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251967 s, 41.6 MB/s 2021-02-10T11:16:24.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:24.289 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:24.289 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:24.624 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:24.625 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:24.625 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839244/c8ec57b0-49ca-40eb-85f2-1207149f4417/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:24.998 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:24.998 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:24.998 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0224119 s, 46.8 MB/s 2021-02-10T11:16:24.999 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839245 2021-02-10T11:16:25.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.406 7f29825b2700 1 Processor -- start 2021-02-10T11:16:25.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.406 7f29825b2700 1 -- start start 2021-02-10T11:16:25.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.407 7f29825b2700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f297c1b0470 0x7f297c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:25.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.407 7f29825b2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f297c0f20e0 0x7f297c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:25.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.407 7f29825b2700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f297c0f0b50 con 0x7f297c0ef610 2021-02-10T11:16:25.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.407 7f29825b2700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f297c0f0820 con 0x7f297c1b0470 2021-02-10T11:16:25.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.407 7f29825b2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297c0f24e0 con 0x7f297c0f20e0 2021-02-10T11:16:25.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.407 7f297b7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f297c0ef610 0x7f297c0efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46362/0 (socket says 172.21.2.101:46362) 2021-02-10T11:16:25.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.407 7f297b7fe700 1 -- 172.21.2.101:0/3446412750 learned_addr learned my addr 172.21.2.101:0/3446412750 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:16:25.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.407 7f2980bcf700 1 --2- 172.21.2.101:0/3446412750 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f297c0f20e0 0x7f297c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:25.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.407 7f297bfff700 1 --2- 172.21.2.101:0/3446412750 >> v2:172.21.2.108:3300/0 conn(0x7f297c1b0470 0x7f297c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:25.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.407 7f297affd700 1 -- 172.21.2.101:0/3446412750 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1059168437 0 0) 0x7f29700010b0 con 0x7f297c0ef610 2021-02-10T11:16:25.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.407 7f297affd700 1 -- 172.21.2.101:0/3446412750 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2964002fe0 con 0x7f297c0ef610 2021-02-10T11:16:25.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.408 7f297affd700 1 -- 172.21.2.101:0/3446412750 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 436988943 0 0) 0x7f29700010b0 con 0x7f297c0ef610 2021-02-10T11:16:25.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.408 7f2980bcf700 1 -- 172.21.2.101:0/3446412750 >> v2:172.21.2.108:3300/0 conn(0x7f297c1b0470 msgr2=0x7f297c1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:16:25.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.408 7f2980bcf700 1 --2- 172.21.2.101:0/3446412750 >> v2:172.21.2.108:3300/0 conn(0x7f297c1b0470 0x7f297c1b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:25.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.408 7f2980bcf700 1 -- 172.21.2.101:0/3446412750 >> v1:172.21.2.101:6789/0 conn(0x7f297c0ef610 legacy=0x7f297c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:25.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.408 7f2980bcf700 1 -- 172.21.2.101:0/3446412750 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297c0e52e0 con 0x7f297c0f20e0 2021-02-10T11:16:25.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.408 7f2980bcf700 1 --2- 172.21.2.101:0/3446412750 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f297c0f20e0 0x7f297c0f0200 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2974006e50 tx=0x7f2974009e70).ready entity=mon.2 client_cookie=927c30d964e7cd2c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:25.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.408 7f297affd700 1 -- 172.21.2.101:0/3446412750 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f297400b510 con 0x7f297c0f20e0 2021-02-10T11:16:25.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.408 7f297affd700 1 -- 172.21.2.101:0/3446412750 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2974006790 con 0x7f297c0f20e0 2021-02-10T11:16:25.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.408 7f297affd700 1 -- 172.21.2.101:0/3446412750 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f297400b790 con 0x7f297c0f20e0 2021-02-10T11:16:25.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.408 7f29825b2700 1 -- 172.21.2.101:0/3446412750 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f297c0f20e0 msgr2=0x7f297c0f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:25.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.408 7f29825b2700 1 --2- 172.21.2.101:0/3446412750 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f297c0f20e0 0x7f297c0f0200 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2974006e50 tx=0x7f2974009e70).stop 2021-02-10T11:16:25.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.409 7f29825b2700 1 -- 172.21.2.101:0/3446412750 shutdown_connections 2021-02-10T11:16:25.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.409 7f29825b2700 1 --2- 172.21.2.101:0/3446412750 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f297c0f20e0 0x7f297c0f0200 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:25.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.409 7f29825b2700 1 --2- 172.21.2.101:0/3446412750 >> v2:172.21.2.108:3300/0 conn(0x7f297c1b0470 0x7f297c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:25.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.409 7f29825b2700 1 -- 172.21.2.101:0/3446412750 >> 172.21.2.101:0/3446412750 conn(0x7f297c0ce3e0 msgr2=0x7f297c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:25.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.409 7f29825b2700 1 -- 172.21.2.101:0/3446412750 shutdown_connections 2021-02-10T11:16:25.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.409 7f29825b2700 1 -- 172.21.2.101:0/3446412750 wait complete. 2021-02-10T11:16:25.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.423 7f29825b2700 1 Processor -- start 2021-02-10T11:16:25.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.423 7f29825b2700 1 -- start start 2021-02-10T11:16:25.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.424 7f29825b2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f297c190920 0x7f297c0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:25.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.424 7f29825b2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f297c0b8200 0x7f297c0b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:25.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.424 7f29825b2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f297c0c1d80 0x7f297c0c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:25.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.424 7f29825b2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c190da0 con 0x7f297c0c1d80 2021-02-10T11:16:25.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.424 7f29825b2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c0b8c20 con 0x7f297c190920 2021-02-10T11:16:25.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.424 7f29825b2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297c0b8e10 con 0x7f297c0b8200 2021-02-10T11:16:25.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.424 7f2980bcf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f297c0c1d80 0x7f297c0c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:25.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.424 7f2980bcf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f297c0c1d80 0x7f297c0c2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45872/0 (socket says 172.21.2.101:45872) 2021-02-10T11:16:25.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.424 7f2980bcf700 1 -- 172.21.2.101:0/148524181 learned_addr learned my addr 172.21.2.101:0/148524181 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:16:25.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.424 7f2980bcf700 1 -- 172.21.2.101:0/148524181 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f297c0b8200 msgr2=0x7f297c0b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:25.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.424 7f2980bcf700 1 --2- 172.21.2.101:0/148524181 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f297c0b8200 0x7f297c0b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:25.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.424 7f2980bcf700 1 -- 172.21.2.101:0/148524181 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f297c190920 msgr2=0x7f297c0b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:25.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.424 7f2980bcf700 1 --2- 172.21.2.101:0/148524181 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f297c190920 0x7f297c0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:25.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.424 7f2980bcf700 1 -- 172.21.2.101:0/148524181 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297c0c27d0 con 0x7f297c0c1d80 2021-02-10T11:16:25.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.424 7f2980bcf700 1 --2- 172.21.2.101:0/148524181 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f297c0c1d80 0x7f297c0c2180 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f29740099f0 tx=0x7f297400bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:25.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.425 7f2978ff9700 1 -- 172.21.2.101:0/148524181 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2974009170 con 0x7f297c0c1d80 2021-02-10T11:16:25.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.425 7f2978ff9700 1 -- 172.21.2.101:0/148524181 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2974006880 con 0x7f297c0c1d80 2021-02-10T11:16:25.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.425 7f2978ff9700 1 -- 172.21.2.101:0/148524181 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f297400c4e0 con 0x7f297c0c1d80 2021-02-10T11:16:25.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.425 7f29825b2700 1 -- 172.21.2.101:0/148524181 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297c0c27d0 con 0x7f297c0c1d80 2021-02-10T11:16:25.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.425 7f29825b2700 1 -- 172.21.2.101:0/148524181 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297c0c2d60 con 0x7f297c0c1d80 2021-02-10T11:16:25.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.425 7f2978ff9700 1 -- 172.21.2.101:0/148524181 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2974009170 con 0x7f297c0c1d80 2021-02-10T11:16:25.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.425 7f2978ff9700 1 --2- 172.21.2.101:0/148524181 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f295c0186c0 0x7f295c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:25.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.425 7f2978ff9700 1 -- 172.21.2.101:0/148524181 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f29740328f0 con 0x7f297c0c1d80 2021-02-10T11:16:25.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.425 7f297bfff700 1 --2- 172.21.2.101:0/148524181 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f295c0186c0 0x7f295c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:25.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.426 7f297bfff700 1 --2- 172.21.2.101:0/148524181 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f295c0186c0 0x7f295c01abc0 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:25.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.456 7f29825b2700 1 -- 172.21.2.101:0/148524181 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f297c105ae0 con 0x7f297c0c1d80 2021-02-10T11:16:25.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.459 7f2978ff9700 1 -- 172.21.2.101:0/148524181 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f297401f6c0 con 0x7f297c0c1d80 2021-02-10T11:16:25.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.585 7f29825b2700 1 -- 172.21.2.101:0/148524181 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839245", "target": ["mgr", ""]}) v1 -- 0x7f297c0c2d60 con 0x7f295c0186c0 2021-02-10T11:16:25.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.610 7f2978ff9700 1 -- 172.21.2.101:0/148524181 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f296c006e70 con 0x7f295c0186c0 2021-02-10T11:16:25.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.618 7f29825b2700 1 -- 172.21.2.101:0/148524181 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f295c0186c0 msgr2=0x7f295c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:25.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.618 7f29825b2700 1 --2- 172.21.2.101:0/148524181 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f295c0186c0 0x7f295c01abc0 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:25.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.618 7f29825b2700 1 -- 172.21.2.101:0/148524181 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f297c0c1d80 msgr2=0x7f297c0c2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:25.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.618 7f29825b2700 1 --2- 172.21.2.101:0/148524181 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f297c0c1d80 0x7f297c0c2180 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f29740099f0 tx=0x7f297400bd10).stop 2021-02-10T11:16:25.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.618 7f29825b2700 1 -- 172.21.2.101:0/148524181 shutdown_connections 2021-02-10T11:16:25.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.618 7f29825b2700 1 --2- 172.21.2.101:0/148524181 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f295c0186c0 0x7f295c01abc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:25.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.618 7f29825b2700 1 --2- 172.21.2.101:0/148524181 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f297c0b8200 0x7f297c0b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:25.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.618 7f29825b2700 1 --2- 172.21.2.101:0/148524181 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f297c0c1d80 0x7f297c0c2180 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:25.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.618 7f29825b2700 1 --2- 172.21.2.101:0/148524181 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f297c190920 0x7f297c0b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:25.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.618 7f29825b2700 1 -- 172.21.2.101:0/148524181 >> 172.21.2.101:0/148524181 conn(0x7f297c094e10 msgr2=0x7f297c0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:25.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.619 7f29825b2700 1 -- 172.21.2.101:0/148524181 shutdown_connections 2021-02-10T11:16:25.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:25.619 7f29825b2700 1 -- 172.21.2.101:0/148524181 wait complete. 2021-02-10T11:16:25.632 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839245 2021-02-10T11:16:26.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.039 7f2b808eb700 1 Processor -- start 2021-02-10T11:16:26.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.039 7f2b808eb700 1 -- start start 2021-02-10T11:16:26.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.040 7f2b808eb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b7c1b0470 0x7f2b7c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:26.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.040 7f2b808eb700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2b7c0ef610 0x7f2b7c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:26.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.040 7f2b808eb700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2b7c0f0030 0x7f2b7c0f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:26.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.040 7f2b808eb700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b7c0e5ea0 con 0x7f2b7c0ef610 2021-02-10T11:16:26.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.040 7f2b808eb700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b7c0f0b50 con 0x7f2b7c0f0030 2021-02-10T11:16:26.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.040 7f2b808eb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c0e52e0 con 0x7f2b7c1b0470 2021-02-10T11:16:26.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.040 7f2b79e1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2b7c0ef610 0x7f2b7c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:26.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.040 7f2b79e1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2b7c0ef610 0x7f2b7c0efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45880/0 (socket says 172.21.2.101:45880) 2021-02-10T11:16:26.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.040 7f2b79e1b700 1 -- 172.21.2.101:0/1680426971 learned_addr learned my addr 172.21.2.101:0/1680426971 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:16:26.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.041 7f2b79e1b700 1 -- 172.21.2.101:0/1680426971 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b7c1b0470 msgr2=0x7f2b7c1b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:16:26.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.041 7f2b79e1b700 1 --2- 172.21.2.101:0/1680426971 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b7c1b0470 0x7f2b7c1b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:26.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.041 7f2b79e1b700 1 -- 172.21.2.101:0/1680426971 >> v2:172.21.2.108:3300/0 conn(0x7f2b7c0f0030 msgr2=0x7f2b7c0f0430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:16:26.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.041 7f2b79e1b700 1 --2- 172.21.2.101:0/1680426971 >> v2:172.21.2.108:3300/0 conn(0x7f2b7c0f0030 0x7f2b7c0f0430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:26.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.041 7f2b79e1b700 1 -- 172.21.2.101:0/1680426971 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7c0f10d0 con 0x7f2b7c0ef610 2021-02-10T11:16:26.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.041 7f2b79e1b700 1 --2- 172.21.2.101:0/1680426971 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b7c0ef610 0x7f2b7c0efa10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f2b6c006700 tx=0x7f2b6c009b00).ready entity=mon.0 client_cookie=59b5ed3c86fdeb9b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:26.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.041 7f2b7961a700 1 -- 172.21.2.101:0/1680426971 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b6c00c0c0 con 0x7f2b7c0ef610 2021-02-10T11:16:26.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.041 7f2b7961a700 1 -- 172.21.2.101:0/1680426971 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6c00c340 con 0x7f2b7c0ef610 2021-02-10T11:16:26.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.041 7f2b7961a700 1 -- 172.21.2.101:0/1680426971 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b6c00c820 con 0x7f2b7c0ef610 2021-02-10T11:16:26.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.041 7f2b808eb700 1 -- 172.21.2.101:0/1680426971 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b7c0ef610 msgr2=0x7f2b7c0efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:26.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.041 7f2b808eb700 1 --2- 172.21.2.101:0/1680426971 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b7c0ef610 0x7f2b7c0efa10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f2b6c006700 tx=0x7f2b6c009b00).stop 2021-02-10T11:16:26.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.042 7f2b808eb700 1 -- 172.21.2.101:0/1680426971 shutdown_connections 2021-02-10T11:16:26.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.042 7f2b808eb700 1 --2- 172.21.2.101:0/1680426971 >> v2:172.21.2.108:3300/0 conn(0x7f2b7c0f0030 0x7f2b7c0f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:26.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.042 7f2b808eb700 1 --2- 172.21.2.101:0/1680426971 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b7c0ef610 0x7f2b7c0efa10 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:26.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.042 7f2b808eb700 1 --2- 172.21.2.101:0/1680426971 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b7c1b0470 0x7f2b7c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:26.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.042 7f2b808eb700 1 -- 172.21.2.101:0/1680426971 >> 172.21.2.101:0/1680426971 conn(0x7f2b7c0ce3e0 msgr2=0x7f2b7c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:26.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.042 7f2b808eb700 1 -- 172.21.2.101:0/1680426971 shutdown_connections 2021-02-10T11:16:26.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.042 7f2b808eb700 1 -- 172.21.2.101:0/1680426971 wait complete. 2021-02-10T11:16:26.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.057 7f2b808eb700 1 Processor -- start 2021-02-10T11:16:26.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.057 7f2b808eb700 1 -- start start 2021-02-10T11:16:26.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.057 7f2b808eb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b7c190920 0x7f2b7c08f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:26.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.057 7f2b808eb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b7c190d20 0x7f2b7c08fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:26.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.057 7f2b808eb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b7c193270 0x7f2b7c193670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:26.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.057 7f2b808eb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c090470 con 0x7f2b7c193270 2021-02-10T11:16:26.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.057 7f2b808eb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c193c90 con 0x7f2b7c190920 2021-02-10T11:16:26.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.057 7f2b808eb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c193e70 con 0x7f2b7c190d20 2021-02-10T11:16:26.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.057 7f2b7ae1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b7c193270 0x7f2b7c193670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:26.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.057 7f2b7ae1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b7c193270 0x7f2b7c193670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45888/0 (socket says 172.21.2.101:45888) 2021-02-10T11:16:26.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.057 7f2b7ae1d700 1 -- 172.21.2.101:0/722559773 learned_addr learned my addr 172.21.2.101:0/722559773 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:16:26.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.058 7f2b79e1b700 1 --2- 172.21.2.101:0/722559773 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b7c190d20 0x7f2b7c08fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:26.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.058 7f2b7a61c700 1 --2- 172.21.2.101:0/722559773 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b7c190920 0x7f2b7c08f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:26.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.058 7f2b7ae1d700 1 -- 172.21.2.101:0/722559773 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b7c190d20 msgr2=0x7f2b7c08fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:26.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.058 7f2b7ae1d700 1 --2- 172.21.2.101:0/722559773 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b7c190d20 0x7f2b7c08fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:26.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.058 7f2b7ae1d700 1 -- 172.21.2.101:0/722559773 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b7c190920 msgr2=0x7f2b7c08f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:26.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.058 7f2b7ae1d700 1 --2- 172.21.2.101:0/722559773 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b7c190920 0x7f2b7c08f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:26.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.058 7f2b7ae1d700 1 -- 172.21.2.101:0/722559773 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7c194090 con 0x7f2b7c193270 2021-02-10T11:16:26.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.058 7f2b7a61c700 1 --2- 172.21.2.101:0/722559773 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b7c190920 0x7f2b7c08f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:16:26.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.058 7f2b79e1b700 1 --2- 172.21.2.101:0/722559773 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b7c190d20 0x7f2b7c08fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:16:26.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.058 7f2b7ae1d700 1 --2- 172.21.2.101:0/722559773 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b7c193270 0x7f2b7c193670 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f2b74009960 tx=0x7f2b7400bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:26.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.058 7f2b72ffd700 1 -- 172.21.2.101:0/722559773 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b74000d10 con 0x7f2b7c193270 2021-02-10T11:16:26.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.058 7f2b72ffd700 1 -- 172.21.2.101:0/722559773 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b74003020 con 0x7f2b7c193270 2021-02-10T11:16:26.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.058 7f2b72ffd700 1 -- 172.21.2.101:0/722559773 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b74005ad0 con 0x7f2b7c193270 2021-02-10T11:16:26.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.058 7f2b808eb700 1 -- 172.21.2.101:0/722559773 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7c194090 con 0x7f2b7c193270 2021-02-10T11:16:26.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.058 7f2b808eb700 1 -- 172.21.2.101:0/722559773 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7c194090 con 0x7f2b7c193270 2021-02-10T11:16:26.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.059 7f2b72ffd700 1 -- 172.21.2.101:0/722559773 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2b74000d10 con 0x7f2b7c193270 2021-02-10T11:16:26.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.059 7f2b72ffd700 1 --2- 172.21.2.101:0/722559773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b600186d0 0x7f2b6001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:26.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.059 7f2b72ffd700 1 -- 172.21.2.101:0/722559773 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2b74032b30 con 0x7f2b7c193270 2021-02-10T11:16:26.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.059 7f2b7a61c700 1 --2- 172.21.2.101:0/722559773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b600186d0 0x7f2b6001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:26.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.059 7f2b7a61c700 1 --2- 172.21.2.101:0/722559773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b600186d0 0x7f2b6001abd0 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:26.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.090 7f2b808eb700 1 -- 172.21.2.101:0/722559773 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b7c193c90 con 0x7f2b7c193270 2021-02-10T11:16:26.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.092 7f2b72ffd700 1 -- 172.21.2.101:0/722559773 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2b7400be30 con 0x7f2b7c193270 2021-02-10T11:16:26.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.221 7f2b808eb700 1 -- 172.21.2.101:0/722559773 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839245", "target": ["mgr", ""]}) v1 -- 0x7f2b7c193c90 con 0x7f2b600186d0 2021-02-10T11:16:26.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.224 7f2b72ffd700 1 -- 172.21.2.101:0/722559773 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f2b64006580 con 0x7f2b600186d0 2021-02-10T11:16:26.230 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839245/47cf9ebc-3b45-4c98-a3ff-aad3ae72aba6 2021-02-10T11:16:26.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.226 7f2b808eb700 1 -- 172.21.2.101:0/722559773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b600186d0 msgr2=0x7f2b6001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:26.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.226 7f2b808eb700 1 --2- 172.21.2.101:0/722559773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b600186d0 0x7f2b6001abd0 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:26.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.226 7f2b808eb700 1 -- 172.21.2.101:0/722559773 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b7c193270 msgr2=0x7f2b7c193670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:26.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.226 7f2b808eb700 1 --2- 172.21.2.101:0/722559773 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b7c193270 0x7f2b7c193670 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f2b74009960 tx=0x7f2b7400bcf0).stop 2021-02-10T11:16:26.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.226 7f2b808eb700 1 -- 172.21.2.101:0/722559773 shutdown_connections 2021-02-10T11:16:26.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.226 7f2b808eb700 1 --2- 172.21.2.101:0/722559773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b600186d0 0x7f2b6001abd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:26.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.226 7f2b808eb700 1 --2- 172.21.2.101:0/722559773 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b7c190d20 0x7f2b7c08fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:26.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.226 7f2b808eb700 1 --2- 172.21.2.101:0/722559773 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b7c193270 0x7f2b7c193670 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:26.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.226 7f2b808eb700 1 --2- 172.21.2.101:0/722559773 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b7c190920 0x7f2b7c08f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:26.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.226 7f2b808eb700 1 -- 172.21.2.101:0/722559773 >> 172.21.2.101:0/722559773 conn(0x7f2b7c0bf4e0 msgr2=0x7f2b7c0be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:26.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.226 7f2b808eb700 1 -- 172.21.2.101:0/722559773 shutdown_connections 2021-02-10T11:16:26.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:26.226 7f2b808eb700 1 -- 172.21.2.101:0/722559773 wait complete. 2021-02-10T11:16:26.240 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839245 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839245/47cf9ebc-3b45-4c98-a3ff-aad3ae72aba6 2021-02-10T11:16:26.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:26.240 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:26.240 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:26.575 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:26.575 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:26.575 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839245/47cf9ebc-3b45-4c98-a3ff-aad3ae72aba6/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:26.956 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:26.956 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:26.957 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253887 s, 41.3 MB/s 2021-02-10T11:16:26.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:26.958 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:26.958 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:27.293 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:27.293 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:27.293 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839245/47cf9ebc-3b45-4c98-a3ff-aad3ae72aba6/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:27.666 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:27.666 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:27.666 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237381 s, 44.2 MB/s 2021-02-10T11:16:27.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:27.667 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:27.667 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:28.003 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:28.004 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:28.004 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839245/47cf9ebc-3b45-4c98-a3ff-aad3ae72aba6/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:28.376 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:28.376 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:28.377 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238266 s, 44.0 MB/s 2021-02-10T11:16:28.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:28.378 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:28.378 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:28.712 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:28.712 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:28.712 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839245/47cf9ebc-3b45-4c98-a3ff-aad3ae72aba6/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:29.083 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:29.083 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:29.083 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0229216 s, 45.7 MB/s 2021-02-10T11:16:29.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:29.084 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:29.085 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:29.418 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:29.418 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:29.418 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839245/47cf9ebc-3b45-4c98-a3ff-aad3ae72aba6/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:29.806 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:29.806 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:29.806 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023969 s, 43.7 MB/s 2021-02-10T11:16:29.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:29.806 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:29.807 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:30.148 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:30.148 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:30.149 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839245/47cf9ebc-3b45-4c98-a3ff-aad3ae72aba6/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:30.525 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:30.526 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:30.526 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0225838 s, 46.4 MB/s 2021-02-10T11:16:30.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:30.527 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:30.527 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:30.865 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:30.865 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:30.865 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839245/47cf9ebc-3b45-4c98-a3ff-aad3ae72aba6/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:31.245 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:31.245 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:31.245 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236373 s, 44.4 MB/s 2021-02-10T11:16:31.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:31.246 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:31.246 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:31.581 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:31.582 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:31.582 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839245/47cf9ebc-3b45-4c98-a3ff-aad3ae72aba6/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:31.958 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:31.959 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:31.959 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248985 s, 42.1 MB/s 2021-02-10T11:16:31.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:31.960 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:31.960 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:32.294 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:32.294 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:32.294 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839245/47cf9ebc-3b45-4c98-a3ff-aad3ae72aba6/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:32.667 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:32.667 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:32.667 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237921 s, 44.1 MB/s 2021-02-10T11:16:32.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:32.668 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:32.668 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:33.003 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:33.003 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:33.003 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839245/47cf9ebc-3b45-4c98-a3ff-aad3ae72aba6/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:33.377 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:33.377 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:33.377 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0224115 s, 46.8 MB/s 2021-02-10T11:16:33.378 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839246 2021-02-10T11:16:33.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.786 7f19408f3700 1 Processor -- start 2021-02-10T11:16:33.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.786 7f19408f3700 1 -- start start 2021-02-10T11:16:33.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.787 7f19408f3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f193c1b0870 0x7f193c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:33.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.787 7f19408f3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f193c0efe00 0x7f193c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:33.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.787 7f19408f3700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f193c1b0c70 con 0x7f193c1b0470 2021-02-10T11:16:33.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.787 7f19408f3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f193c0f0b50 con 0x7f193c1b0870 2021-02-10T11:16:33.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.787 7f19408f3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193c0f0820 con 0x7f193c0efe00 2021-02-10T11:16:33.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.787 7f193ae1d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f193c0efe00 0x7f193c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.787 7f1939e1b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f193c1b0870 0x7f193c0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.787 7f193a61c700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f193c1b0470 0x7f193c0ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:39798/0 (socket says 172.21.2.101:39798) 2021-02-10T11:16:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.787 7f193ae1d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f193c0efe00 0x7f193c0f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:38630/0 (socket says 172.21.2.101:38630) 2021-02-10T11:16:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.787 7f193ae1d700 1 -- 172.21.2.101:0/2659991248 learned_addr learned my addr 172.21.2.101:0/2659991248 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:16:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.788 7f193ae1d700 1 -- 172.21.2.101:0/2659991248 >> v2:172.21.2.108:3300/0 conn(0x7f193c1b0870 msgr2=0x7f193c0f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:16:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.788 7f193ae1d700 1 --2- 172.21.2.101:0/2659991248 >> v2:172.21.2.108:3300/0 conn(0x7f193c1b0870 0x7f193c0f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.788 7f193ae1d700 1 -- 172.21.2.101:0/2659991248 >> v1:172.21.2.108:6789/0 conn(0x7f193c1b0470 legacy=0x7f193c0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.788 7f193ae1d700 1 -- 172.21.2.101:0/2659991248 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193c0e52e0 con 0x7f193c0efe00 2021-02-10T11:16:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.788 7f1939e1b700 1 --2- 172.21.2.101:0/2659991248 >> v2:172.21.2.108:3300/0 conn(0x7f193c1b0870 0x7f193c0f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:16:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.788 7f193ae1d700 1 --2- 172.21.2.101:0/2659991248 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f193c0efe00 0x7f193c0f0200 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f1934006aa0 tx=0x7f1934009f20).ready entity=mon.2 client_cookie=ba90f3e91ab9ca8f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.788 7f193961a700 1 -- 172.21.2.101:0/2659991248 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f193400b5c0 con 0x7f193c0efe00 2021-02-10T11:16:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.788 7f193961a700 1 -- 172.21.2.101:0/2659991248 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1934003030 con 0x7f193c0efe00 2021-02-10T11:16:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.788 7f193961a700 1 -- 172.21.2.101:0/2659991248 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f193400bab0 con 0x7f193c0efe00 2021-02-10T11:16:33.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.789 7f19408f3700 1 -- 172.21.2.101:0/2659991248 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f193c0efe00 msgr2=0x7f193c0f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:33.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.789 7f19408f3700 1 --2- 172.21.2.101:0/2659991248 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f193c0efe00 0x7f193c0f0200 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f1934006aa0 tx=0x7f1934009f20).stop 2021-02-10T11:16:33.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.789 7f19408f3700 1 -- 172.21.2.101:0/2659991248 shutdown_connections 2021-02-10T11:16:33.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.789 7f19408f3700 1 --2- 172.21.2.101:0/2659991248 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f193c0efe00 0x7f193c0f0200 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:33.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.789 7f19408f3700 1 --2- 172.21.2.101:0/2659991248 >> v2:172.21.2.108:3300/0 conn(0x7f193c1b0870 0x7f193c0f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:33.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.789 7f19408f3700 1 -- 172.21.2.101:0/2659991248 >> 172.21.2.101:0/2659991248 conn(0x7f193c0ce3e0 msgr2=0x7f193c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:33.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.789 7f19408f3700 1 -- 172.21.2.101:0/2659991248 shutdown_connections 2021-02-10T11:16:33.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.789 7f19408f3700 1 -- 172.21.2.101:0/2659991248 wait complete. 2021-02-10T11:16:33.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.804 7f19408f3700 1 Processor -- start 2021-02-10T11:16:33.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.804 7f19408f3700 1 -- start start 2021-02-10T11:16:33.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.804 7f19408f3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f193c0a0420 0x7f193c0a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:33.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.804 7f19408f3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f193c189d00 0x7f193c18a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:33.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.804 7f19408f3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f193c18a720 0x7f193c0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:33.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.804 7f19408f3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c189780 con 0x7f193c189d00 2021-02-10T11:16:33.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.804 7f19408f3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c18ab20 con 0x7f193c0a0420 2021-02-10T11:16:33.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.804 7f19408f3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193c0a1140 con 0x7f193c18a720 2021-02-10T11:16:33.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.805 7f1939e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f193c189d00 0x7f193c18a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:33.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.805 7f193ae1d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f193c18a720 0x7f193c0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:33.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.805 7f1939e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f193c189d00 0x7f193c18a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45900/0 (socket says 172.21.2.101:45900) 2021-02-10T11:16:33.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.805 7f1939e1b700 1 -- 172.21.2.101:0/3156366934 learned_addr learned my addr 172.21.2.101:0/3156366934 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:16:33.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.805 7f193a61c700 1 --2- 172.21.2.101:0/3156366934 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f193c0a0420 0x7f193c0a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:33.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.805 7f1939e1b700 1 -- 172.21.2.101:0/3156366934 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f193c18a720 msgr2=0x7f193c0b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:33.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.805 7f1939e1b700 1 --2- 172.21.2.101:0/3156366934 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f193c18a720 0x7f193c0b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:33.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.805 7f1939e1b700 1 -- 172.21.2.101:0/3156366934 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f193c0a0420 msgr2=0x7f193c0a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:33.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.805 7f1939e1b700 1 --2- 172.21.2.101:0/3156366934 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f193c0a0420 0x7f193c0a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:33.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.805 7f1939e1b700 1 -- 172.21.2.101:0/3156366934 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193c0b8200 con 0x7f193c189d00 2021-02-10T11:16:33.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.805 7f1939e1b700 1 --2- 172.21.2.101:0/3156366934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f193c189d00 0x7f193c18a100 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f1930006f40 tx=0x7f193000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:33.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.805 7f192affd700 1 -- 172.21.2.101:0/3156366934 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1930006bd0 con 0x7f193c189d00 2021-02-10T11:16:33.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.805 7f192affd700 1 -- 172.21.2.101:0/3156366934 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193000bd80 con 0x7f193c189d00 2021-02-10T11:16:33.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.805 7f192affd700 1 -- 172.21.2.101:0/3156366934 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19300022b0 con 0x7f193c189d00 2021-02-10T11:16:33.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.805 7f19408f3700 1 -- 172.21.2.101:0/3156366934 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f193c0b8200 con 0x7f193c189d00 2021-02-10T11:16:33.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.806 7f19408f3700 1 -- 172.21.2.101:0/3156366934 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193c0b8200 con 0x7f193c189d00 2021-02-10T11:16:33.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.806 7f192affd700 1 -- 172.21.2.101:0/3156366934 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1930000c90 con 0x7f193c189d00 2021-02-10T11:16:33.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.806 7f192affd700 1 --2- 172.21.2.101:0/3156366934 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19200186c0 0x7f192001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:33.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.806 7f192affd700 1 -- 172.21.2.101:0/3156366934 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1930032590 con 0x7f193c189d00 2021-02-10T11:16:33.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.806 7f193a61c700 1 --2- 172.21.2.101:0/3156366934 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19200186c0 0x7f192001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:33.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.807 7f193a61c700 1 --2- 172.21.2.101:0/3156366934 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19200186c0 0x7f192001abc0 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:33.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.837 7f19408f3700 1 -- 172.21.2.101:0/3156366934 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f193c105ae0 con 0x7f193c189d00 2021-02-10T11:16:33.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.839 7f192affd700 1 -- 172.21.2.101:0/3156366934 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f19300273f0 con 0x7f193c189d00 2021-02-10T11:16:33.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.966 7f19408f3700 1 -- 172.21.2.101:0/3156366934 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839246", "target": ["mgr", ""]}) v1 -- 0x7f193c189780 con 0x7f19200186c0 2021-02-10T11:16:33.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.990 7f192affd700 1 -- 172.21.2.101:0/3156366934 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f192c007cc0 con 0x7f19200186c0 2021-02-10T11:16:34.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.999 7f19408f3700 1 -- 172.21.2.101:0/3156366934 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19200186c0 msgr2=0x7f192001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:34.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.999 7f19408f3700 1 --2- 172.21.2.101:0/3156366934 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19200186c0 0x7f192001abc0 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:34.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.999 7f19408f3700 1 -- 172.21.2.101:0/3156366934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f193c189d00 msgr2=0x7f193c18a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:34.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.999 7f19408f3700 1 --2- 172.21.2.101:0/3156366934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f193c189d00 0x7f193c18a100 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f1930006f40 tx=0x7f193000b990).stop 2021-02-10T11:16:34.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.999 7f19408f3700 1 -- 172.21.2.101:0/3156366934 shutdown_connections 2021-02-10T11:16:34.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.999 7f19408f3700 1 --2- 172.21.2.101:0/3156366934 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19200186c0 0x7f192001abc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:34.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.999 7f19408f3700 1 --2- 172.21.2.101:0/3156366934 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f193c18a720 0x7f193c0b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:34.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.999 7f19408f3700 1 --2- 172.21.2.101:0/3156366934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f193c189d00 0x7f193c18a100 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:34.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.999 7f19408f3700 1 --2- 172.21.2.101:0/3156366934 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f193c0a0420 0x7f193c0a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:34.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.999 7f19408f3700 1 -- 172.21.2.101:0/3156366934 >> 172.21.2.101:0/3156366934 conn(0x7f193c094e10 msgr2=0x7f193c0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:34.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.999 7f19408f3700 1 -- 172.21.2.101:0/3156366934 shutdown_connections 2021-02-10T11:16:34.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:33.999 7f19408f3700 1 -- 172.21.2.101:0/3156366934 wait complete. 2021-02-10T11:16:34.013 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839246 2021-02-10T11:16:34.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.420 7fa8ffbe0700 1 Processor -- start 2021-02-10T11:16:34.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.420 7fa8ffbe0700 1 -- start start 2021-02-10T11:16:34.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.421 7fa8ffbe0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8f81b0870 0x7fa8f80f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:34.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.421 7fa8ffbe0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa8f80efe00 0x7fa8f80f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:34.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.421 7fa8ffbe0700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f81b0c70 con 0x7fa8f81b0470 2021-02-10T11:16:34.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.421 7fa8ffbe0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8f80f0b50 con 0x7fa8f80efe00 2021-02-10T11:16:34.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.421 7fa8ffbe0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f80f0820 con 0x7fa8f81b0870 2021-02-10T11:16:34.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.421 7fa8fe1fd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa8f80efe00 0x7fa8f80f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:34.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.421 7fa8fe1fd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa8f80efe00 0x7fa8f80f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45910/0 (socket says 172.21.2.101:45910) 2021-02-10T11:16:34.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.421 7fa8fe1fd700 1 -- 172.21.2.101:0/1367056537 learned_addr learned my addr 172.21.2.101:0/1367056537 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:16:34.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.421 7fa8fe1fd700 1 -- 172.21.2.101:0/1367056537 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8f81b0870 msgr2=0x7fa8f80f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:16:34.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.421 7fa8fd1fb700 1 --2- 172.21.2.101:0/1367056537 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8f81b0870 0x7fa8f80f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:34.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.421 7fa8fe1fd700 1 --2- 172.21.2.101:0/1367056537 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8f81b0870 0x7fa8f80f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:34.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.421 7fa8fe1fd700 1 -- 172.21.2.101:0/1367056537 >> v1:172.21.2.101:6789/0 conn(0x7fa8f81b0470 legacy=0x7fa8f80ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:34.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.421 7fa8fe1fd700 1 -- 172.21.2.101:0/1367056537 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f80e52e0 con 0x7fa8f80efe00 2021-02-10T11:16:34.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.421 7fa8fd1fb700 1 --2- 172.21.2.101:0/1367056537 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8f81b0870 0x7fa8f80f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:16:34.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.421 7fa8fe1fd700 1 --2- 172.21.2.101:0/1367056537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8f80efe00 0x7fa8f80f0200 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fa8f0006f00 tx=0x7fa8f0009f10).ready entity=mon.0 client_cookie=ffa290e395266643 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:34.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.422 7fa8fc9fa700 1 -- 172.21.2.101:0/1367056537 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa8f000b3b0 con 0x7fa8f80efe00 2021-02-10T11:16:34.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.422 7fa8fc9fa700 1 -- 172.21.2.101:0/1367056537 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f000b730 con 0x7fa8f80efe00 2021-02-10T11:16:34.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.422 7fa8fc9fa700 1 -- 172.21.2.101:0/1367056537 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa8f000bc10 con 0x7fa8f80efe00 2021-02-10T11:16:34.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.422 7fa8ffbe0700 1 -- 172.21.2.101:0/1367056537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8f80efe00 msgr2=0x7fa8f80f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:34.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.422 7fa8ffbe0700 1 --2- 172.21.2.101:0/1367056537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8f80efe00 0x7fa8f80f0200 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fa8f0006f00 tx=0x7fa8f0009f10).stop 2021-02-10T11:16:34.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.422 7fa8ffbe0700 1 -- 172.21.2.101:0/1367056537 shutdown_connections 2021-02-10T11:16:34.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.422 7fa8ffbe0700 1 --2- 172.21.2.101:0/1367056537 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8f81b0870 0x7fa8f80f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:34.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.422 7fa8ffbe0700 1 --2- 172.21.2.101:0/1367056537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8f80efe00 0x7fa8f80f0200 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:34.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.422 7fa8ffbe0700 1 -- 172.21.2.101:0/1367056537 >> 172.21.2.101:0/1367056537 conn(0x7fa8f80ce3e0 msgr2=0x7fa8f80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:34.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.422 7fa8ffbe0700 1 -- 172.21.2.101:0/1367056537 shutdown_connections 2021-02-10T11:16:34.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.422 7fa8ffbe0700 1 -- 172.21.2.101:0/1367056537 wait complete. 2021-02-10T11:16:34.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.437 7fa8ffbe0700 1 Processor -- start 2021-02-10T11:16:34.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.437 7fa8ffbe0700 1 -- start start 2021-02-10T11:16:34.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.437 7fa8ffbe0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8f819d1f0 0x7fa8f80a6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:34.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.437 7fa8ffbe0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8f80a7230 0x7fa8f8097d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:34.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.437 7fa8ffbe0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8f8098330 0x7fa8f8098f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:34.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.437 7fa8ffbe0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f8098730 con 0x7fa8f80a7230 2021-02-10T11:16:34.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.437 7fa8ffbe0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f80989a0 con 0x7fa8f8098330 2021-02-10T11:16:34.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.437 7fa8ffbe0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f80bc4d0 con 0x7fa8f819d1f0 2021-02-10T11:16:34.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.437 7fa8fd1fb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8f80a7230 0x7fa8f8097d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:34.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.437 7fa8fd1fb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8f80a7230 0x7fa8f8097d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45914/0 (socket says 172.21.2.101:45914) 2021-02-10T11:16:34.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.437 7fa8fd1fb700 1 -- 172.21.2.101:0/1213498840 learned_addr learned my addr 172.21.2.101:0/1213498840 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:16:34.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.438 7fa8fd1fb700 1 -- 172.21.2.101:0/1213498840 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8f819d1f0 msgr2=0x7fa8f80a6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:34.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.438 7fa8fd1fb700 1 --2- 172.21.2.101:0/1213498840 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8f819d1f0 0x7fa8f80a6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:34.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.438 7fa8fd1fb700 1 -- 172.21.2.101:0/1213498840 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8f8098330 msgr2=0x7fa8f8098f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:34.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.438 7fa8fd1fb700 1 --2- 172.21.2.101:0/1213498840 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8f8098330 0x7fa8f8098f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:34.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.438 7fa8fd1fb700 1 -- 172.21.2.101:0/1213498840 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f80bc6b0 con 0x7fa8f80a7230 2021-02-10T11:16:34.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.438 7fa8fd1fb700 1 --2- 172.21.2.101:0/1213498840 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8f80a7230 0x7fa8f8097d10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fa8f4006f20 tx=0x7fa8f400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:34.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.438 7fa8ea7fc700 1 -- 172.21.2.101:0/1213498840 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa8f400bd80 con 0x7fa8f80a7230 2021-02-10T11:16:34.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.438 7fa8ea7fc700 1 -- 172.21.2.101:0/1213498840 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f4000c90 con 0x7fa8f80a7230 2021-02-10T11:16:34.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.438 7fa8ffbe0700 1 -- 172.21.2.101:0/1213498840 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f80bc6b0 con 0x7fa8f80a7230 2021-02-10T11:16:34.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.438 7fa8ea7fc700 1 -- 172.21.2.101:0/1213498840 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa8f40022b0 con 0x7fa8f80a7230 2021-02-10T11:16:34.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.438 7fa8ffbe0700 1 -- 172.21.2.101:0/1213498840 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f8099830 con 0x7fa8f80a7230 2021-02-10T11:16:34.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.439 7fa8ea7fc700 1 -- 172.21.2.101:0/1213498840 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa8f400a000 con 0x7fa8f80a7230 2021-02-10T11:16:34.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.439 7fa8ea7fc700 1 --2- 172.21.2.101:0/1213498840 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8e0018730 0x7fa8e001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:34.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.439 7fa8ea7fc700 1 -- 172.21.2.101:0/1213498840 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa8f4032640 con 0x7fa8f80a7230 2021-02-10T11:16:34.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.439 7fa8fd9fc700 1 --2- 172.21.2.101:0/1213498840 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8e0018730 0x7fa8e001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:34.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.439 7fa8fd9fc700 1 --2- 172.21.2.101:0/1213498840 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8e0018730 0x7fa8e001ac10 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:34.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.471 7fa8ffbe0700 1 -- 172.21.2.101:0/1213498840 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f8105800 con 0x7fa8f80a7230 2021-02-10T11:16:34.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.473 7fa8ea7fc700 1 -- 172.21.2.101:0/1213498840 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa8f4006bb0 con 0x7fa8f80a7230 2021-02-10T11:16:34.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.600 7fa8ffbe0700 1 -- 172.21.2.101:0/1213498840 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839246", "target": ["mgr", ""]}) v1 -- 0x7fa8f8099830 con 0x7fa8e0018730 2021-02-10T11:16:34.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.603 7fa8ea7fc700 1 -- 172.21.2.101:0/1213498840 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fa8ec006a20 con 0x7fa8e0018730 2021-02-10T11:16:34.609 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839246/748d8a62-56d6-4de4-ad05-71655f9ebe77 2021-02-10T11:16:34.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.605 7fa8ffbe0700 1 -- 172.21.2.101:0/1213498840 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8e0018730 msgr2=0x7fa8e001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:34.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.605 7fa8ffbe0700 1 --2- 172.21.2.101:0/1213498840 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8e0018730 0x7fa8e001ac10 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:34.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.605 7fa8ffbe0700 1 -- 172.21.2.101:0/1213498840 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8f80a7230 msgr2=0x7fa8f8097d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:34.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.605 7fa8ffbe0700 1 --2- 172.21.2.101:0/1213498840 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8f80a7230 0x7fa8f8097d10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fa8f4006f20 tx=0x7fa8f400b990).stop 2021-02-10T11:16:34.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.605 7fa8ffbe0700 1 -- 172.21.2.101:0/1213498840 shutdown_connections 2021-02-10T11:16:34.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.605 7fa8ffbe0700 1 --2- 172.21.2.101:0/1213498840 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8f8098330 0x7fa8f8098f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:34.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.605 7fa8ffbe0700 1 --2- 172.21.2.101:0/1213498840 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8f80a7230 0x7fa8f8097d10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:34.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.605 7fa8ffbe0700 1 --2- 172.21.2.101:0/1213498840 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8e0018730 0x7fa8e001ac10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:34.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.605 7fa8ffbe0700 1 --2- 172.21.2.101:0/1213498840 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8f819d1f0 0x7fa8f80a6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:34.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.605 7fa8ffbe0700 1 -- 172.21.2.101:0/1213498840 >> 172.21.2.101:0/1213498840 conn(0x7fa8f80afe70 msgr2=0x7fa8f80a1580 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:34.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.605 7fa8ffbe0700 1 -- 172.21.2.101:0/1213498840 shutdown_connections 2021-02-10T11:16:34.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:34.605 7fa8ffbe0700 1 -- 172.21.2.101:0/1213498840 wait complete. 2021-02-10T11:16:34.618 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839246 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839246/748d8a62-56d6-4de4-ad05-71655f9ebe77 2021-02-10T11:16:34.618 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:34.619 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:34.619 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:34.953 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:34.954 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:34.954 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839246/748d8a62-56d6-4de4-ad05-71655f9ebe77/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:35.333 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:35.334 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:35.334 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0250283 s, 41.9 MB/s 2021-02-10T11:16:35.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:35.335 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:35.335 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:35.670 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:35.671 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:35.671 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839246/748d8a62-56d6-4de4-ad05-71655f9ebe77/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:36.044 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:36.045 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:36.045 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0222665 s, 47.1 MB/s 2021-02-10T11:16:36.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:36.047 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:36.047 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:36.381 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:36.381 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:36.382 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839246/748d8a62-56d6-4de4-ad05-71655f9ebe77/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:36.754 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:36.754 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:36.754 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237811 s, 44.1 MB/s 2021-02-10T11:16:36.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:36.756 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:36.756 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:37.091 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:37.092 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:37.092 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839246/748d8a62-56d6-4de4-ad05-71655f9ebe77/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:37.466 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:37.466 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:37.466 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235615 s, 44.5 MB/s 2021-02-10T11:16:37.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:37.468 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:37.468 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:37.803 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:37.804 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:37.804 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839246/748d8a62-56d6-4de4-ad05-71655f9ebe77/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:38.178 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:38.178 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:38.179 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241167 s, 43.5 MB/s 2021-02-10T11:16:38.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:38.180 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:38.180 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:38.514 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:38.514 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:38.514 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839246/748d8a62-56d6-4de4-ad05-71655f9ebe77/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:38.883 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:38.883 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:38.883 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0226694 s, 46.3 MB/s 2021-02-10T11:16:38.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:38.884 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:38.884 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:39.218 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:39.218 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:39.219 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839246/748d8a62-56d6-4de4-ad05-71655f9ebe77/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:39.594 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:39.594 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:39.594 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242303 s, 43.3 MB/s 2021-02-10T11:16:39.595 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:39.595 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:39.595 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:39.930 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:39.930 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:39.930 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839246/748d8a62-56d6-4de4-ad05-71655f9ebe77/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:40.307 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:40.307 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:40.307 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0262025 s, 40.0 MB/s 2021-02-10T11:16:40.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:40.308 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:40.308 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:40.643 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:40.643 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:40.643 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839246/748d8a62-56d6-4de4-ad05-71655f9ebe77/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:41.015 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:41.015 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:41.015 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0227357 s, 46.1 MB/s 2021-02-10T11:16:41.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:41.016 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:41.016 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:41.351 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:41.351 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:41.351 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839246/748d8a62-56d6-4de4-ad05-71655f9ebe77/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:41.724 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:41.724 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:41.724 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0229143 s, 45.8 MB/s 2021-02-10T11:16:41.726 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839247 2021-02-10T11:16:42.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.133 7f7d09f48700 1 Processor -- start 2021-02-10T11:16:42.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.133 7f7d09f48700 1 -- start start 2021-02-10T11:16:42.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.134 7f7d09f48700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7d041b0470 0x7f7d041b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:42.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.134 7f7d09f48700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d040f20e0 0x7f7d040f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:42.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.134 7f7d09f48700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d040f0b50 con 0x7f7d040ef610 2021-02-10T11:16:42.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.134 7f7d09f48700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d040f0820 con 0x7f7d041b0470 2021-02-10T11:16:42.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.134 7f7d09f48700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d040f24e0 con 0x7f7d040f20e0 2021-02-10T11:16:42.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.134 7f7d02ffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f7d040ef610 0x7f7d040efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46414/0 (socket says 172.21.2.101:46414) 2021-02-10T11:16:42.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.134 7f7d02ffd700 1 -- 172.21.2.101:0/2446105267 learned_addr learned my addr 172.21.2.101:0/2446105267 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:16:42.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.134 7f7d027fc700 1 -- 172.21.2.101:0/2446105267 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4142150028 0 0) 0x7f7cec0010b0 con 0x7f7d040ef610 2021-02-10T11:16:42.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.134 7f7d027fc700 1 -- 172.21.2.101:0/2446105267 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cf0002fe0 con 0x7f7d040ef610 2021-02-10T11:16:42.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.134 7f7d03fff700 1 --2- 172.21.2.101:0/2446105267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d040f20e0 0x7f7d040f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:42.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d037fe700 1 --2- 172.21.2.101:0/2446105267 >> v2:172.21.2.108:3300/0 conn(0x7f7d041b0470 0x7f7d041b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:42.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d027fc700 1 -- 172.21.2.101:0/2446105267 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4096285499 0 0) 0x7f7cec0010b0 con 0x7f7d040ef610 2021-02-10T11:16:42.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d027fc700 1 -- 172.21.2.101:0/2446105267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d040f20e0 msgr2=0x7f7d040f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:16:42.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d027fc700 1 --2- 172.21.2.101:0/2446105267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d040f20e0 0x7f7d040f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:42.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d027fc700 1 -- 172.21.2.101:0/2446105267 >> v2:172.21.2.108:3300/0 conn(0x7f7d041b0470 msgr2=0x7f7d041b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:16:42.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d027fc700 1 --2- 172.21.2.101:0/2446105267 >> v2:172.21.2.108:3300/0 conn(0x7f7d041b0470 0x7f7d041b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:42.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d027fc700 1 -- 172.21.2.101:0/2446105267 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d040e52e0 con 0x7f7d040ef610 2021-02-10T11:16:42.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d027fc700 1 -- 172.21.2.101:0/2446105267 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f7cec0015b0 con 0x7f7d040ef610 2021-02-10T11:16:42.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d027fc700 1 -- 172.21.2.101:0/2446105267 >> v1:172.21.2.101:6789/0 conn(0x7f7d040ef610 legacy=0x7f7d040efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:42.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d027fc700 1 --2- 172.21.2.101:0/2446105267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7cf0003b10 0x7f7cf0005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:42.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d027fc700 1 --2- 172.21.2.101:0/2446105267 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7cf004a7e0 0x7f7cf004cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:42.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d027fc700 1 --2- 172.21.2.101:0/2446105267 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7cf004d500 0x7f7cf004fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:42.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d027fc700 1 -- 172.21.2.101:0/2446105267 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf00502d0 con 0x7f7cf004a7e0 2021-02-10T11:16:42.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d027fc700 1 -- 172.21.2.101:0/2446105267 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf00504f0 con 0x7f7cf004d500 2021-02-10T11:16:42.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d027fc700 1 -- 172.21.2.101:0/2446105267 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf0050710 con 0x7f7cf0003b10 2021-02-10T11:16:42.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d037fe700 1 --2- 172.21.2.101:0/2446105267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7cf0003b10 0x7f7cf0005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:42.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d02ffd700 1 --2- 172.21.2.101:0/2446105267 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7cf004a7e0 0x7f7cf004cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:42.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d03fff700 1 --2- 172.21.2.101:0/2446105267 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7cf004d500 0x7f7cf004fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:42.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d037fe700 1 -- 172.21.2.101:0/2446105267 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7cf004d500 msgr2=0x7f7cf004fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:16:42.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d037fe700 1 --2- 172.21.2.101:0/2446105267 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7cf004d500 0x7f7cf004fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:42.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d037fe700 1 -- 172.21.2.101:0/2446105267 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7cf004a7e0 msgr2=0x7f7cf004cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:16:42.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d037fe700 1 --2- 172.21.2.101:0/2446105267 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7cf004a7e0 0x7f7cf004cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:42.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.135 7f7d037fe700 1 -- 172.21.2.101:0/2446105267 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf00509b0 con 0x7f7cf0003b10 2021-02-10T11:16:42.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.136 7f7d037fe700 1 --2- 172.21.2.101:0/2446105267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7cf0003b10 0x7f7cf0005f80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f7cf400c870 tx=0x7f7cf400cc10).ready entity=mon.2 client_cookie=5da2fa1ef71b909b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:42.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.136 7f7d027fc700 1 -- 172.21.2.101:0/2446105267 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7cf40090d0 con 0x7f7cf0003b10 2021-02-10T11:16:42.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.136 7f7d027fc700 1 -- 172.21.2.101:0/2446105267 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf4009350 con 0x7f7cf0003b10 2021-02-10T11:16:42.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.136 7f7d027fc700 1 -- 172.21.2.101:0/2446105267 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7cf40097b0 con 0x7f7cf0003b10 2021-02-10T11:16:42.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.136 7f7d09f48700 1 -- 172.21.2.101:0/2446105267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7cf0003b10 msgr2=0x7f7cf0005f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:42.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.136 7f7d09f48700 1 --2- 172.21.2.101:0/2446105267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7cf0003b10 0x7f7cf0005f80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f7cf400c870 tx=0x7f7cf400cc10).stop 2021-02-10T11:16:42.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.136 7f7d03fff700 1 -- 172.21.2.101:0/2446105267 reap_dead start 2021-02-10T11:16:42.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.136 7f7d09f48700 1 -- 172.21.2.101:0/2446105267 shutdown_connections 2021-02-10T11:16:42.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.136 7f7d09f48700 1 -- 172.21.2.101:0/2446105267 >> 172.21.2.101:0/2446105267 conn(0x7f7d040ce3e0 msgr2=0x7f7d040e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:42.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.136 7f7d09f48700 1 -- 172.21.2.101:0/2446105267 shutdown_connections 2021-02-10T11:16:42.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.136 7f7d09f48700 1 -- 172.21.2.101:0/2446105267 wait complete. 2021-02-10T11:16:42.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.151 7f7d09f48700 1 Processor -- start 2021-02-10T11:16:42.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.151 7f7d09f48700 1 -- start start 2021-02-10T11:16:42.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.152 7f7d09f48700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d040bf920 0x7f7d040bfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:42.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.152 7f7d09f48700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d040b6570 0x7f7d040b6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:42.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.152 7f7d09f48700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7d041a7430 0x7f7d041a7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:42.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.152 7f7d09f48700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d040c0440 con 0x7f7d040b6570 2021-02-10T11:16:42.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.152 7f7d09f48700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d040924b0 con 0x7f7d041a7430 2021-02-10T11:16:42.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.152 7f7d09f48700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d04092970 con 0x7f7d040bf920 2021-02-10T11:16:42.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.152 7f7d02ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d040b6570 0x7f7d040b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:42.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.152 7f7d02ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d040b6570 0x7f7d040b6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45934/0 (socket says 172.21.2.101:45934) 2021-02-10T11:16:42.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.152 7f7d02ffd700 1 -- 172.21.2.101:0/177906639 learned_addr learned my addr 172.21.2.101:0/177906639 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:16:42.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.152 7f7d03fff700 1 --2- 172.21.2.101:0/177906639 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7d041a7430 0x7f7d041a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:42.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.152 7f7d037fe700 1 --2- 172.21.2.101:0/177906639 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d040bf920 0x7f7d040bfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:42.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.152 7f7d02ffd700 1 -- 172.21.2.101:0/177906639 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d040bf920 msgr2=0x7f7d040bfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:42.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.152 7f7d02ffd700 1 --2- 172.21.2.101:0/177906639 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d040bf920 0x7f7d040bfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:42.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.152 7f7d02ffd700 1 -- 172.21.2.101:0/177906639 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7d041a7430 msgr2=0x7f7d041a7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:42.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.152 7f7d02ffd700 1 --2- 172.21.2.101:0/177906639 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7d041a7430 0x7f7d041a7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:42.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.152 7f7d02ffd700 1 -- 172.21.2.101:0/177906639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d041a7e50 con 0x7f7d040b6570 2021-02-10T11:16:42.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.152 7f7d02ffd700 1 --2- 172.21.2.101:0/177906639 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d040b6570 0x7f7d040b6970 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f7cec001bf0 tx=0x7f7cec00a900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:42.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.153 7f7cfbfff700 1 -- 172.21.2.101:0/177906639 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7cec00acf0 con 0x7f7d040b6570 2021-02-10T11:16:42.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.153 7f7cfbfff700 1 -- 172.21.2.101:0/177906639 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cec0012d0 con 0x7f7d040b6570 2021-02-10T11:16:42.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.153 7f7cfbfff700 1 -- 172.21.2.101:0/177906639 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7cec004330 con 0x7f7d040b6570 2021-02-10T11:16:42.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.153 7f7d09f48700 1 -- 172.21.2.101:0/177906639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d041a7e50 con 0x7f7d040b6570 2021-02-10T11:16:42.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.153 7f7d09f48700 1 -- 172.21.2.101:0/177906639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d040c09c0 con 0x7f7d040b6570 2021-02-10T11:16:42.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.153 7f7cfbfff700 1 -- 172.21.2.101:0/177906639 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7cec004140 con 0x7f7d040b6570 2021-02-10T11:16:42.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.153 7f7cfbfff700 1 --2- 172.21.2.101:0/177906639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ce80186c0 0x7f7ce801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:42.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.153 7f7cfbfff700 1 -- 172.21.2.101:0/177906639 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7cec0323a0 con 0x7f7d040b6570 2021-02-10T11:16:42.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.154 7f7d037fe700 1 --2- 172.21.2.101:0/177906639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ce80186c0 0x7f7ce801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:42.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.154 7f7d037fe700 1 --2- 172.21.2.101:0/177906639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ce80186c0 0x7f7ce801abc0 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:42.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.184 7f7d09f48700 1 -- 172.21.2.101:0/177906639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d04105b40 con 0x7f7d040b6570 2021-02-10T11:16:42.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.186 7f7cfbfff700 1 -- 172.21.2.101:0/177906639 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7cec01c530 con 0x7f7d040b6570 2021-02-10T11:16:42.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.313 7f7d09f48700 1 -- 172.21.2.101:0/177906639 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839247", "target": ["mgr", ""]}) v1 -- 0x7f7d040c0440 con 0x7f7ce80186c0 2021-02-10T11:16:42.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.339 7f7cfbfff700 1 -- 172.21.2.101:0/177906639 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f7cf40009b0 con 0x7f7ce80186c0 2021-02-10T11:16:42.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.346 7f7d09f48700 1 -- 172.21.2.101:0/177906639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ce80186c0 msgr2=0x7f7ce801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:42.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.346 7f7d09f48700 1 --2- 172.21.2.101:0/177906639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ce80186c0 0x7f7ce801abc0 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:42.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.346 7f7d09f48700 1 -- 172.21.2.101:0/177906639 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d040b6570 msgr2=0x7f7d040b6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:42.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.346 7f7d09f48700 1 --2- 172.21.2.101:0/177906639 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d040b6570 0x7f7d040b6970 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f7cec001bf0 tx=0x7f7cec00a900).stop 2021-02-10T11:16:42.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.347 7f7d09f48700 1 -- 172.21.2.101:0/177906639 shutdown_connections 2021-02-10T11:16:42.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.347 7f7d09f48700 1 --2- 172.21.2.101:0/177906639 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7d041a7430 0x7f7d041a7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:42.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.347 7f7d09f48700 1 --2- 172.21.2.101:0/177906639 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d040b6570 0x7f7d040b6970 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:42.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.347 7f7d09f48700 1 --2- 172.21.2.101:0/177906639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ce80186c0 0x7f7ce801abc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:42.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.347 7f7d09f48700 1 --2- 172.21.2.101:0/177906639 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d040bf920 0x7f7d040bfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:42.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.347 7f7d09f48700 1 -- 172.21.2.101:0/177906639 >> 172.21.2.101:0/177906639 conn(0x7f7d040b50c0 msgr2=0x7f7d04092d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:42.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.347 7f7d09f48700 1 -- 172.21.2.101:0/177906639 shutdown_connections 2021-02-10T11:16:42.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.347 7f7d09f48700 1 -- 172.21.2.101:0/177906639 wait complete. 2021-02-10T11:16:42.360 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839247 2021-02-10T11:16:42.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.767 7f04526b4700 1 Processor -- start 2021-02-10T11:16:42.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.768 7f04526b4700 1 -- start start 2021-02-10T11:16:42.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.768 7f04526b4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f044c1b0470 0x7f044c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:42.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.768 7f04526b4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f044c0f20e0 0x7f044c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:42.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.768 7f04526b4700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f044c0f0b50 con 0x7f044c0ef610 2021-02-10T11:16:42.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.768 7f04526b4700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f044c0f0820 con 0x7f044c1b0470 2021-02-10T11:16:42.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.768 7f04526b4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f044c0f24e0 con 0x7f044c0f20e0 2021-02-10T11:16:42.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.768 7f044bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f044c1b0470 0x7f044c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:42.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.768 7f044bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f044c1b0470 0x7f044c1b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45940/0 (socket says 172.21.2.101:45940) 2021-02-10T11:16:42.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.768 7f044bfff700 1 -- 172.21.2.101:0/2780530425 learned_addr learned my addr 172.21.2.101:0/2780530425 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:16:42.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.768 7f0450cd1700 1 --2- 172.21.2.101:0/2780530425 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f044c0f20e0 0x7f044c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:42.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.769 7f044bfff700 1 -- 172.21.2.101:0/2780530425 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f044c0f20e0 msgr2=0x7f044c0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:16:42.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.769 7f044bfff700 1 --2- 172.21.2.101:0/2780530425 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f044c0f20e0 0x7f044c0f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:42.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.769 7f044bfff700 1 -- 172.21.2.101:0/2780530425 >> v1:172.21.2.108:6789/0 conn(0x7f044c0ef610 legacy=0x7f044c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:42.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.769 7f044bfff700 1 -- 172.21.2.101:0/2780530425 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044c0e52e0 con 0x7f044c1b0470 2021-02-10T11:16:42.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.769 7f044bfff700 1 --2- 172.21.2.101:0/2780530425 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f044c1b0470 0x7f044c1b0870 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f043c006700 tx=0x7f043c009b80).ready entity=mon.0 client_cookie=fbac45bb677eeefd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:42.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.769 7f044affd700 1 -- 172.21.2.101:0/2780530425 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f043c00c0c0 con 0x7f044c1b0470 2021-02-10T11:16:42.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.769 7f044affd700 1 -- 172.21.2.101:0/2780530425 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043c00c440 con 0x7f044c1b0470 2021-02-10T11:16:42.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.769 7f044affd700 1 -- 172.21.2.101:0/2780530425 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f043c00c960 con 0x7f044c1b0470 2021-02-10T11:16:42.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.769 7f04526b4700 1 -- 172.21.2.101:0/2780530425 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f044c1b0470 msgr2=0x7f044c1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:42.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.769 7f04526b4700 1 --2- 172.21.2.101:0/2780530425 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f044c1b0470 0x7f044c1b0870 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f043c006700 tx=0x7f043c009b80).stop 2021-02-10T11:16:42.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.770 7f04526b4700 1 -- 172.21.2.101:0/2780530425 shutdown_connections 2021-02-10T11:16:42.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.770 7f04526b4700 1 --2- 172.21.2.101:0/2780530425 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f044c0f20e0 0x7f044c0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:42.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.770 7f04526b4700 1 --2- 172.21.2.101:0/2780530425 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f044c1b0470 0x7f044c1b0870 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:42.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.770 7f04526b4700 1 -- 172.21.2.101:0/2780530425 >> 172.21.2.101:0/2780530425 conn(0x7f044c0ce3e0 msgr2=0x7f044c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:42.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.770 7f04526b4700 1 -- 172.21.2.101:0/2780530425 shutdown_connections 2021-02-10T11:16:42.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.770 7f04526b4700 1 -- 172.21.2.101:0/2780530425 wait complete. 2021-02-10T11:16:42.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.784 7f04526b4700 1 Processor -- start 2021-02-10T11:16:42.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.785 7f04526b4700 1 -- start start 2021-02-10T11:16:42.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.785 7f04526b4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f044c0a0420 0x7f044c0a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:42.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.785 7f04526b4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f044c189d00 0x7f044c18a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:42.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.785 7f04526b4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f044c18a720 0x7f044c0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:42.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.785 7f04526b4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044c189780 con 0x7f044c0a0420 2021-02-10T11:16:42.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.785 7f04526b4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044c18ab20 con 0x7f044c189d00 2021-02-10T11:16:42.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.785 7f04526b4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f044c0a1140 con 0x7f044c18a720 2021-02-10T11:16:42.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.785 7f044bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f044c0a0420 0x7f044c0a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:42.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.785 7f044bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f044c0a0420 0x7f044c0a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45944/0 (socket says 172.21.2.101:45944) 2021-02-10T11:16:42.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.785 7f044bfff700 1 -- 172.21.2.101:0/164804698 learned_addr learned my addr 172.21.2.101:0/164804698 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:16:42.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.786 7f044b7fe700 1 --2- 172.21.2.101:0/164804698 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f044c189d00 0x7f044c18a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:42.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.786 7f0450cd1700 1 --2- 172.21.2.101:0/164804698 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f044c18a720 0x7f044c0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:42.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.786 7f044bfff700 1 -- 172.21.2.101:0/164804698 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f044c18a720 msgr2=0x7f044c0b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:42.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.786 7f044bfff700 1 --2- 172.21.2.101:0/164804698 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f044c18a720 0x7f044c0b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:42.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.786 7f044bfff700 1 -- 172.21.2.101:0/164804698 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f044c189d00 msgr2=0x7f044c18a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:42.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.786 7f044bfff700 1 --2- 172.21.2.101:0/164804698 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f044c189d00 0x7f044c18a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:42.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.786 7f044bfff700 1 -- 172.21.2.101:0/164804698 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044c0b8200 con 0x7f044c0a0420 2021-02-10T11:16:42.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.786 7f0450cd1700 1 --2- 172.21.2.101:0/164804698 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f044c18a720 0x7f044c0b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:16:42.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.786 7f044bfff700 1 --2- 172.21.2.101:0/164804698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f044c0a0420 0x7f044c0a0820 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f043c00b5e0 tx=0x7f043c00b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:42.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.786 7f0448ff9700 1 -- 172.21.2.101:0/164804698 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f043c00bd70 con 0x7f044c0a0420 2021-02-10T11:16:42.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.786 7f0448ff9700 1 -- 172.21.2.101:0/164804698 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043c002030 con 0x7f044c0a0420 2021-02-10T11:16:42.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.786 7f0448ff9700 1 -- 172.21.2.101:0/164804698 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f043c002550 con 0x7f044c0a0420 2021-02-10T11:16:42.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.786 7f04526b4700 1 -- 172.21.2.101:0/164804698 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f044c0b8200 con 0x7f044c0a0420 2021-02-10T11:16:42.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.786 7f04526b4700 1 -- 172.21.2.101:0/164804698 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f044c0b8200 con 0x7f044c0a0420 2021-02-10T11:16:42.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.787 7f0448ff9700 1 -- 172.21.2.101:0/164804698 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f043c000c90 con 0x7f044c0a0420 2021-02-10T11:16:42.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.787 7f0448ff9700 1 --2- 172.21.2.101:0/164804698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04280186c0 0x7f042801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:42.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.787 7f0448ff9700 1 -- 172.21.2.101:0/164804698 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f043c032630 con 0x7f044c0a0420 2021-02-10T11:16:42.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.787 7f044b7fe700 1 --2- 172.21.2.101:0/164804698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04280186c0 0x7f042801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:42.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.788 7f044b7fe700 1 --2- 172.21.2.101:0/164804698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04280186c0 0x7f042801abc0 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:42.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.818 7f04526b4700 1 -- 172.21.2.101:0/164804698 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f044c105ae0 con 0x7f044c0a0420 2021-02-10T11:16:42.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.821 7f0448ff9700 1 -- 172.21.2.101:0/164804698 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f043c030de0 con 0x7f044c0a0420 2021-02-10T11:16:42.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.946 7f04526b4700 1 -- 172.21.2.101:0/164804698 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839247", "target": ["mgr", ""]}) v1 -- 0x7f044c189780 con 0x7f04280186c0 2021-02-10T11:16:42.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.950 7f0448ff9700 1 -- 172.21.2.101:0/164804698 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f0438006a90 con 0x7f04280186c0 2021-02-10T11:16:42.959 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839247/8f3efd75-3f80-4053-b813-1d487083bb72 2021-02-10T11:16:42.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.955 7f04526b4700 1 -- 172.21.2.101:0/164804698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04280186c0 msgr2=0x7f042801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:42.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.955 7f04526b4700 1 --2- 172.21.2.101:0/164804698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04280186c0 0x7f042801abc0 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:42.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.955 7f04526b4700 1 -- 172.21.2.101:0/164804698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f044c0a0420 msgr2=0x7f044c0a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:42.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.955 7f04526b4700 1 --2- 172.21.2.101:0/164804698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f044c0a0420 0x7f044c0a0820 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f043c00b5e0 tx=0x7f043c00b980).stop 2021-02-10T11:16:42.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.955 7f04526b4700 1 -- 172.21.2.101:0/164804698 shutdown_connections 2021-02-10T11:16:42.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.955 7f04526b4700 1 --2- 172.21.2.101:0/164804698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04280186c0 0x7f042801abc0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:42.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.955 7f04526b4700 1 --2- 172.21.2.101:0/164804698 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f044c18a720 0x7f044c0b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:42.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.955 7f04526b4700 1 --2- 172.21.2.101:0/164804698 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f044c189d00 0x7f044c18a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:42.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.955 7f04526b4700 1 --2- 172.21.2.101:0/164804698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f044c0a0420 0x7f044c0a0820 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:42.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.955 7f04526b4700 1 -- 172.21.2.101:0/164804698 >> 172.21.2.101:0/164804698 conn(0x7f044c094e10 msgr2=0x7f044c0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:42.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.955 7f04526b4700 1 -- 172.21.2.101:0/164804698 shutdown_connections 2021-02-10T11:16:42.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:42.955 7f04526b4700 1 -- 172.21.2.101:0/164804698 wait complete. 2021-02-10T11:16:42.968 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839247 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839247/8f3efd75-3f80-4053-b813-1d487083bb72 2021-02-10T11:16:42.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:42.969 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:42.969 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:43.304 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:43.304 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:43.305 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839247/8f3efd75-3f80-4053-b813-1d487083bb72/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:43.685 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:43.685 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:43.685 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0254044 s, 41.3 MB/s 2021-02-10T11:16:43.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:43.686 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:43.686 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:44.020 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:44.020 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:44.020 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839247/8f3efd75-3f80-4053-b813-1d487083bb72/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:44.391 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:44.392 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:44.392 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251019 s, 41.8 MB/s 2021-02-10T11:16:44.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:44.393 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:44.393 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:44.728 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:44.728 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:44.729 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839247/8f3efd75-3f80-4053-b813-1d487083bb72/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:45.105 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:45.105 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:45.105 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247789 s, 42.3 MB/s 2021-02-10T11:16:45.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:45.106 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:45.106 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:45.440 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:45.440 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:45.440 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839247/8f3efd75-3f80-4053-b813-1d487083bb72/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:45.546 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:16:45.548 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:16:45.815 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:45.815 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:45.815 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243195 s, 43.1 MB/s 2021-02-10T11:16:45.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:45.817 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:45.817 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:46.151 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:46.151 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:46.152 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839247/8f3efd75-3f80-4053-b813-1d487083bb72/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:46.527 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:46.527 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:46.527 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245526 s, 42.7 MB/s 2021-02-10T11:16:46.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:46.528 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:46.528 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:46.863 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:46.864 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:46.864 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839247/8f3efd75-3f80-4053-b813-1d487083bb72/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:47.238 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:47.239 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:47.239 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023695 s, 44.3 MB/s 2021-02-10T11:16:47.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:47.240 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:47.240 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:47.574 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:47.574 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:47.575 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839247/8f3efd75-3f80-4053-b813-1d487083bb72/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:47.948 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:47.948 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:47.948 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240312 s, 43.6 MB/s 2021-02-10T11:16:47.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:47.949 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:47.950 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:48.285 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:48.285 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:48.285 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839247/8f3efd75-3f80-4053-b813-1d487083bb72/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:48.658 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:48.658 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:48.658 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0230265 s, 45.5 MB/s 2021-02-10T11:16:48.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:48.659 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:48.660 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:48.994 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:48.994 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:48.995 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839247/8f3efd75-3f80-4053-b813-1d487083bb72/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:49.366 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:49.366 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:49.366 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0232752 s, 45.1 MB/s 2021-02-10T11:16:49.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:49.367 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:49.367 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:49.703 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:49.703 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:49.704 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839247/8f3efd75-3f80-4053-b813-1d487083bb72/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:50.080 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:50.784 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:50.784 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0250382 s, 41.9 MB/s 2021-02-10T11:16:50.786 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839248 2021-02-10T11:16:51.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.193 7f2793c8f700 1 Processor -- start 2021-02-10T11:16:51.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.193 7f2793c8f700 1 -- start start 2021-02-10T11:16:51.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.194 7f2793c8f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f278c1b0470 0x7f278c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:51.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.194 7f2793c8f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f278c0f0b50 con 0x7f278c0f20e0 2021-02-10T11:16:51.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.194 7f2793c8f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f278c0f24e0 con 0x7f278c0ef610 2021-02-10T11:16:51.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.194 7f2793c8f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f278c0e5ea0 con 0x7f278c1b0470 2021-02-10T11:16:51.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.194 7f2791aab700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f278c1b0470 0x7f278c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:51.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.194 7f27922ac700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f278c0f20e0 0x7f278c0f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46448/0 (socket says 172.21.2.101:46448) 2021-02-10T11:16:51.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.194 7f27922ac700 1 -- 172.21.2.101:0/1062636410 learned_addr learned my addr 172.21.2.101:0/1062636410 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:16:51.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.195 7f2791aab700 1 -- 172.21.2.101:0/1062636410 >> v1:172.21.2.108:6789/0 conn(0x7f278c0ef610 legacy=0x7f278c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:16:51.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.195 7f2791aab700 1 -- 172.21.2.101:0/1062636410 >> v1:172.21.2.101:6789/0 conn(0x7f278c0f20e0 legacy=0x7f278c0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:51.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.195 7f2791aab700 1 -- 172.21.2.101:0/1062636410 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278c0e52e0 con 0x7f278c1b0470 2021-02-10T11:16:51.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.195 7f2791aab700 1 --2- 172.21.2.101:0/1062636410 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f278c1b0470 0x7f278c1b0870 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f2788006650 tx=0x7f2788009a80).ready entity=mon.0 client_cookie=406d25c979e40fc8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:51.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.195 7f2790aa9700 1 -- 172.21.2.101:0/1062636410 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f27880009b0 con 0x7f278c1b0470 2021-02-10T11:16:51.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.195 7f2790aa9700 1 -- 172.21.2.101:0/1062636410 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278800c0c0 con 0x7f278c1b0470 2021-02-10T11:16:51.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.195 7f2790aa9700 1 -- 172.21.2.101:0/1062636410 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f278800c5e0 con 0x7f278c1b0470 2021-02-10T11:16:51.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.195 7f2793c8f700 1 -- 172.21.2.101:0/1062636410 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f278c1b0470 msgr2=0x7f278c1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:51.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.195 7f2793c8f700 1 --2- 172.21.2.101:0/1062636410 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f278c1b0470 0x7f278c1b0870 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f2788006650 tx=0x7f2788009a80).stop 2021-02-10T11:16:51.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.196 7f2793c8f700 1 -- 172.21.2.101:0/1062636410 shutdown_connections 2021-02-10T11:16:51.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.196 7f2793c8f700 1 --2- 172.21.2.101:0/1062636410 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f278c1b0470 0x7f278c1b0870 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:51.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.196 7f2793c8f700 1 -- 172.21.2.101:0/1062636410 >> 172.21.2.101:0/1062636410 conn(0x7f278c0ce3e0 msgr2=0x7f278c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:51.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.196 7f2793c8f700 1 -- 172.21.2.101:0/1062636410 shutdown_connections 2021-02-10T11:16:51.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.196 7f2793c8f700 1 -- 172.21.2.101:0/1062636410 wait complete. 2021-02-10T11:16:51.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.211 7f2793c8f700 1 Processor -- start 2021-02-10T11:16:51.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.211 7f2793c8f700 1 -- start start 2021-02-10T11:16:51.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.211 7f2793c8f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f278c087510 0x7f278c0aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:51.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.211 7f2793c8f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f278c1c68f0 0x7f278c1b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:51.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.211 7f2793c8f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f278c08c4d0 0x7f278c1b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:51.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.211 7f2793c8f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c0a2d50 con 0x7f278c1c68f0 2021-02-10T11:16:51.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.211 7f2793c8f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c0a2f70 con 0x7f278c087510 2021-02-10T11:16:51.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.211 7f2793c8f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278c0a3190 con 0x7f278c08c4d0 2021-02-10T11:16:51.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.211 7f27912aa700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f278c1c68f0 0x7f278c1b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:51.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.211 7f27912aa700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f278c1c68f0 0x7f278c1b1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45956/0 (socket says 172.21.2.101:45956) 2021-02-10T11:16:51.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.211 7f27912aa700 1 -- 172.21.2.101:0/3098846367 learned_addr learned my addr 172.21.2.101:0/3098846367 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:16:51.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.212 7f27912aa700 1 -- 172.21.2.101:0/3098846367 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f278c08c4d0 msgr2=0x7f278c1b18a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:16:51.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.212 7f27912aa700 1 --2- 172.21.2.101:0/3098846367 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f278c08c4d0 0x7f278c1b18a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:51.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.212 7f27912aa700 1 -- 172.21.2.101:0/3098846367 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f278c087510 msgr2=0x7f278c0aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:51.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.212 7f27912aa700 1 --2- 172.21.2.101:0/3098846367 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f278c087510 0x7f278c0aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:51.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.212 7f27912aa700 1 -- 172.21.2.101:0/3098846367 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278c0a33b0 con 0x7f278c1c68f0 2021-02-10T11:16:51.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.212 7f27912aa700 1 --2- 172.21.2.101:0/3098846367 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f278c1c68f0 0x7f278c1b1f60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f277c00b5b0 tx=0x7f277c00b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:51.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.212 7f27827fc700 1 -- 172.21.2.101:0/3098846367 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f277c00bd40 con 0x7f278c1c68f0 2021-02-10T11:16:51.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.212 7f27827fc700 1 -- 172.21.2.101:0/3098846367 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277c000e20 con 0x7f278c1c68f0 2021-02-10T11:16:51.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.212 7f27827fc700 1 -- 172.21.2.101:0/3098846367 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f277c0054a0 con 0x7f278c1c68f0 2021-02-10T11:16:51.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.212 7f2793c8f700 1 -- 172.21.2.101:0/3098846367 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f278c0a33b0 con 0x7f278c1c68f0 2021-02-10T11:16:51.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.212 7f2793c8f700 1 -- 172.21.2.101:0/3098846367 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278c0a33b0 con 0x7f278c1c68f0 2021-02-10T11:16:51.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.213 7f27827fc700 1 -- 172.21.2.101:0/3098846367 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f277c006030 con 0x7f278c1c68f0 2021-02-10T11:16:51.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.213 7f27827fc700 1 --2- 172.21.2.101:0/3098846367 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f27740186c0 0x7f277401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:51.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.213 7f27827fc700 1 -- 172.21.2.101:0/3098846367 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f277c032590 con 0x7f278c1c68f0 2021-02-10T11:16:51.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.213 7f2791aab700 1 --2- 172.21.2.101:0/3098846367 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f27740186c0 0x7f277401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:51.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.213 7f2791aab700 1 --2- 172.21.2.101:0/3098846367 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f27740186c0 0x7f277401abc0 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:51.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.244 7f2793c8f700 1 -- 172.21.2.101:0/3098846367 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f278c07e6b0 con 0x7f278c1c68f0 2021-02-10T11:16:51.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.247 7f27827fc700 1 -- 172.21.2.101:0/3098846367 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f277c00bd40 con 0x7f278c1c68f0 2021-02-10T11:16:51.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.376 7f2793c8f700 1 -- 172.21.2.101:0/3098846367 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839248", "target": ["mgr", ""]}) v1 -- 0x7f278c07e6b0 con 0x7f27740186c0 2021-02-10T11:16:51.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.401 7f27827fc700 1 -- 172.21.2.101:0/3098846367 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f2788000940 con 0x7f27740186c0 2021-02-10T11:16:51.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.410 7f2793c8f700 1 -- 172.21.2.101:0/3098846367 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f27740186c0 msgr2=0x7f277401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:51.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.410 7f2793c8f700 1 --2- 172.21.2.101:0/3098846367 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f27740186c0 0x7f277401abc0 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:51.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.410 7f2793c8f700 1 -- 172.21.2.101:0/3098846367 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f278c1c68f0 msgr2=0x7f278c1b1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:51.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.410 7f2793c8f700 1 --2- 172.21.2.101:0/3098846367 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f278c1c68f0 0x7f278c1b1f60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f277c00b5b0 tx=0x7f277c00b950).stop 2021-02-10T11:16:51.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.410 7f2793c8f700 1 -- 172.21.2.101:0/3098846367 shutdown_connections 2021-02-10T11:16:51.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.410 7f2793c8f700 1 --2- 172.21.2.101:0/3098846367 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f27740186c0 0x7f277401abc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:51.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.410 7f2793c8f700 1 --2- 172.21.2.101:0/3098846367 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f278c08c4d0 0x7f278c1b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:51.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.410 7f2793c8f700 1 --2- 172.21.2.101:0/3098846367 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f278c1c68f0 0x7f278c1b1f60 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:51.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.410 7f2793c8f700 1 --2- 172.21.2.101:0/3098846367 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f278c087510 0x7f278c0aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:51.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.410 7f2793c8f700 1 -- 172.21.2.101:0/3098846367 >> 172.21.2.101:0/3098846367 conn(0x7f278c0886d0 msgr2=0x7f278c081c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:51.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.410 7f2793c8f700 1 -- 172.21.2.101:0/3098846367 shutdown_connections 2021-02-10T11:16:51.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.410 7f2793c8f700 1 -- 172.21.2.101:0/3098846367 wait complete. 2021-02-10T11:16:51.423 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839248 2021-02-10T11:16:51.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.830 7f820381a700 1 Processor -- start 2021-02-10T11:16:51.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.830 7f820381a700 1 -- start start 2021-02-10T11:16:51.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.831 7f820381a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81fc1b0470 0x7f81fc1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:51.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.831 7f820381a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f81fc0ef610 0x7f81fc0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:51.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.831 7f820381a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81fc0f0b50 con 0x7f81fc0f0030 2021-02-10T11:16:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.831 7f820381a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81fc0e52e0 con 0x7f81fc0ef610 2021-02-10T11:16:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.831 7f820381a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc0e5490 con 0x7f81fc1b0470 2021-02-10T11:16:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.831 7f8200e35700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f81fc0ef610 0x7f81fc0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.831 7f8200e35700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f81fc0ef610 0x7f81fc0efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45964/0 (socket says 172.21.2.101:45964) 2021-02-10T11:16:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.831 7f8200e35700 1 -- 172.21.2.101:0/1038115389 learned_addr learned my addr 172.21.2.101:0/1038115389 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:16:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.831 7f8200e35700 1 -- 172.21.2.101:0/1038115389 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81fc1b0470 msgr2=0x7f81fc1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:16:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.831 7f8200e35700 1 --2- 172.21.2.101:0/1038115389 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81fc1b0470 0x7f81fc1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.831 7f8200e35700 1 -- 172.21.2.101:0/1038115389 >> v1:172.21.2.108:6789/0 conn(0x7f81fc0f0030 legacy=0x7f81fc0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:16:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.831 7f8200e35700 1 -- 172.21.2.101:0/1038115389 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc0f0430 con 0x7f81fc0ef610 2021-02-10T11:16:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.831 7f8200e35700 1 --2- 172.21.2.101:0/1038115389 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81fc0ef610 0x7f81fc0efa10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f81f0006700 tx=0x7f81f0009b80).ready entity=mon.0 client_cookie=f8ea1f1b419be1e9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.832 7f81effff700 1 -- 172.21.2.101:0/1038115389 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81f000c0c0 con 0x7f81fc0ef610 2021-02-10T11:16:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.832 7f81effff700 1 -- 172.21.2.101:0/1038115389 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f000c400 con 0x7f81fc0ef610 2021-02-10T11:16:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.832 7f81effff700 1 -- 172.21.2.101:0/1038115389 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81f000c920 con 0x7f81fc0ef610 2021-02-10T11:16:51.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.832 7f820381a700 1 -- 172.21.2.101:0/1038115389 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81fc0ef610 msgr2=0x7f81fc0efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:51.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.832 7f820381a700 1 --2- 172.21.2.101:0/1038115389 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81fc0ef610 0x7f81fc0efa10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f81f0006700 tx=0x7f81f0009b80).stop 2021-02-10T11:16:51.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.832 7f820381a700 1 -- 172.21.2.101:0/1038115389 shutdown_connections 2021-02-10T11:16:51.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.832 7f820381a700 1 --2- 172.21.2.101:0/1038115389 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81fc0ef610 0x7f81fc0efa10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:51.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.832 7f820381a700 1 --2- 172.21.2.101:0/1038115389 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81fc1b0470 0x7f81fc1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:51.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.832 7f820381a700 1 -- 172.21.2.101:0/1038115389 >> 172.21.2.101:0/1038115389 conn(0x7f81fc0ce3e0 msgr2=0x7f81fc0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:51.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.833 7f820381a700 1 -- 172.21.2.101:0/1038115389 shutdown_connections 2021-02-10T11:16:51.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.833 7f820381a700 1 -- 172.21.2.101:0/1038115389 wait complete. 2021-02-10T11:16:51.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.847 7f820381a700 1 Processor -- start 2021-02-10T11:16:51.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.847 7f820381a700 1 -- start start 2021-02-10T11:16:51.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.848 7f820381a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81fc0a6c90 0x7f81fc1924c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:51.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.848 7f820381a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81fc192ae0 0x7f81fc192ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:51.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.848 7f820381a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81fc0919e0 0x7f81fc091de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:51.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.848 7f820381a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc19d2e0 con 0x7f81fc0a6c90 2021-02-10T11:16:51.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.848 7f820381a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc0914d0 con 0x7f81fc192ae0 2021-02-10T11:16:51.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.848 7f820381a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc0a6820 con 0x7f81fc0919e0 2021-02-10T11:16:51.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.848 7f8201636700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81fc0a6c90 0x7f81fc1924c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:51.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.848 7f8201636700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81fc0a6c90 0x7f81fc1924c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45968/0 (socket says 172.21.2.101:45968) 2021-02-10T11:16:51.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.848 7f8201636700 1 -- 172.21.2.101:0/1008794645 learned_addr learned my addr 172.21.2.101:0/1008794645 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:16:51.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.848 7f8201e37700 1 --2- 172.21.2.101:0/1008794645 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81fc0919e0 0x7f81fc091de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:51.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.848 7f8200e35700 1 --2- 172.21.2.101:0/1008794645 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81fc192ae0 0x7f81fc192ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:51.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.848 7f8201e37700 1 -- 172.21.2.101:0/1008794645 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81fc192ae0 msgr2=0x7f81fc192ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:51.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.848 7f8201e37700 1 --2- 172.21.2.101:0/1008794645 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81fc192ae0 0x7f81fc192ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:51.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.848 7f8201e37700 1 -- 172.21.2.101:0/1008794645 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81fc0a6c90 msgr2=0x7f81fc1924c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:51.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.849 7f8201e37700 1 --2- 172.21.2.101:0/1008794645 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81fc0a6c90 0x7f81fc1924c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:51.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.849 7f8201e37700 1 -- 172.21.2.101:0/1008794645 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc092400 con 0x7f81fc0919e0 2021-02-10T11:16:51.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.849 7f8201636700 1 --2- 172.21.2.101:0/1008794645 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81fc0a6c90 0x7f81fc1924c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:16:51.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.849 7f8201e37700 1 --2- 172.21.2.101:0/1008794645 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81fc0919e0 0x7f81fc091de0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f81f4009fc0 tx=0x7f81f400bc90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:51.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.849 7f81edffb700 1 -- 172.21.2.101:0/1008794645 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81f4007070 con 0x7f81fc0919e0 2021-02-10T11:16:51.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.849 7f81edffb700 1 -- 172.21.2.101:0/1008794645 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f40073b0 con 0x7f81fc0919e0 2021-02-10T11:16:51.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.849 7f81edffb700 1 -- 172.21.2.101:0/1008794645 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81f4007800 con 0x7f81fc0919e0 2021-02-10T11:16:51.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.849 7f820381a700 1 -- 172.21.2.101:0/1008794645 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81fc092400 con 0x7f81fc0919e0 2021-02-10T11:16:51.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.849 7f820381a700 1 -- 172.21.2.101:0/1008794645 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81fc092400 con 0x7f81fc0919e0 2021-02-10T11:16:51.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.850 7f81edffb700 1 -- 172.21.2.101:0/1008794645 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f81f4007000 con 0x7f81fc0919e0 2021-02-10T11:16:51.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.850 7f81edffb700 1 --2- 172.21.2.101:0/1008794645 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81dc0186c0 0x7f81dc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:51.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.850 7f81edffb700 1 -- 172.21.2.101:0/1008794645 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f81f4032850 con 0x7f81fc0919e0 2021-02-10T11:16:51.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.850 7f8201636700 1 --2- 172.21.2.101:0/1008794645 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81dc0186c0 0x7f81dc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:51.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.850 7f8201636700 1 --2- 172.21.2.101:0/1008794645 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81dc0186c0 0x7f81dc01abc0 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:51.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.881 7f820381a700 1 -- 172.21.2.101:0/1008794645 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81fc0e52e0 con 0x7f81fc0919e0 2021-02-10T11:16:51.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:51.884 7f81edffb700 1 -- 172.21.2.101:0/1008794645 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f81f400bdd0 con 0x7f81fc0919e0 2021-02-10T11:16:52.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:52.018 7f820381a700 1 -- 172.21.2.101:0/1008794645 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839248", "target": ["mgr", ""]}) v1 -- 0x7f81fc0e52e0 con 0x7f81dc0186c0 2021-02-10T11:16:52.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:52.022 7f81edffb700 1 -- 172.21.2.101:0/1008794645 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f81f8006ac0 con 0x7f81dc0186c0 2021-02-10T11:16:52.030 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839248/5f26f55a-adcd-4e45-9505-b5fb98ac71c8 2021-02-10T11:16:52.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:52.027 7f820381a700 1 -- 172.21.2.101:0/1008794645 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81dc0186c0 msgr2=0x7f81dc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:52.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:52.027 7f820381a700 1 --2- 172.21.2.101:0/1008794645 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81dc0186c0 0x7f81dc01abc0 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:52.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:52.027 7f820381a700 1 -- 172.21.2.101:0/1008794645 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81fc0919e0 msgr2=0x7f81fc091de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:52.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:52.027 7f820381a700 1 --2- 172.21.2.101:0/1008794645 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81fc0919e0 0x7f81fc091de0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f81f4009fc0 tx=0x7f81f400bc90).stop 2021-02-10T11:16:52.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:52.027 7f820381a700 1 -- 172.21.2.101:0/1008794645 shutdown_connections 2021-02-10T11:16:52.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:52.027 7f820381a700 1 --2- 172.21.2.101:0/1008794645 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81dc0186c0 0x7f81dc01abc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:52.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:52.027 7f820381a700 1 --2- 172.21.2.101:0/1008794645 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81fc0919e0 0x7f81fc091de0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:52.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:52.027 7f820381a700 1 --2- 172.21.2.101:0/1008794645 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81fc192ae0 0x7f81fc192ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:52.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:52.027 7f820381a700 1 --2- 172.21.2.101:0/1008794645 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81fc0a6c90 0x7f81fc1924c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:52.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:52.027 7f820381a700 1 -- 172.21.2.101:0/1008794645 >> 172.21.2.101:0/1008794645 conn(0x7f81fc0ac550 msgr2=0x7f81fc0be1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:52.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:52.027 7f820381a700 1 -- 172.21.2.101:0/1008794645 shutdown_connections 2021-02-10T11:16:52.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:52.027 7f820381a700 1 -- 172.21.2.101:0/1008794645 wait complete. 2021-02-10T11:16:52.041 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839248 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839248/5f26f55a-adcd-4e45-9505-b5fb98ac71c8 2021-02-10T11:16:52.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:52.041 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:52.041 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:52.377 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:52.378 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:52.378 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839248/5f26f55a-adcd-4e45-9505-b5fb98ac71c8/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:52.759 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:52.759 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:52.759 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239391 s, 43.8 MB/s 2021-02-10T11:16:52.759 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:52.760 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:52.760 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:53.095 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:53.095 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:53.095 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839248/5f26f55a-adcd-4e45-9505-b5fb98ac71c8/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:53.470 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:53.470 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:53.470 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248432 s, 42.2 MB/s 2021-02-10T11:16:53.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:53.471 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:53.471 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:53.807 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:53.808 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:53.808 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839248/5f26f55a-adcd-4e45-9505-b5fb98ac71c8/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:54.180 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:54.180 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:54.180 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253178 s, 41.4 MB/s 2021-02-10T11:16:54.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:54.181 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:54.181 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:54.515 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:54.515 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:54.515 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839248/5f26f55a-adcd-4e45-9505-b5fb98ac71c8/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:54.890 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:54.891 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:54.891 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248646 s, 42.2 MB/s 2021-02-10T11:16:54.892 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:54.892 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:54.892 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:55.228 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:55.228 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:55.229 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839248/5f26f55a-adcd-4e45-9505-b5fb98ac71c8/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:55.605 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:55.605 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:55.605 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0255756 s, 41.0 MB/s 2021-02-10T11:16:55.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:55.607 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:55.607 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:55.948 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:55.949 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:55.949 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839248/5f26f55a-adcd-4e45-9505-b5fb98ac71c8/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:56.325 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:56.325 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:56.325 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242803 s, 43.2 MB/s 2021-02-10T11:16:56.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:56.326 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:56.327 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:56.662 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:56.662 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:56.662 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839248/5f26f55a-adcd-4e45-9505-b5fb98ac71c8/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:57.037 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:57.038 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:57.038 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244111 s, 43.0 MB/s 2021-02-10T11:16:57.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:57.039 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:57.039 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:57.375 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:57.376 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:57.376 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839248/5f26f55a-adcd-4e45-9505-b5fb98ac71c8/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:57.753 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:57.753 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:57.753 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249639 s, 42.0 MB/s 2021-02-10T11:16:57.754 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:57.754 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:57.754 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:58.087 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:58.088 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:58.088 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839248/5f26f55a-adcd-4e45-9505-b5fb98ac71c8/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:58.460 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:58.461 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:58.461 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0227624 s, 46.1 MB/s 2021-02-10T11:16:58.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:16:58.462 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:16:58.462 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:16:58.798 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:16:58.798 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:16:58.798 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839248/5f26f55a-adcd-4e45-9505-b5fb98ac71c8/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:16:59.172 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:16:59.172 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:16:59.172 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253192 s, 41.4 MB/s 2021-02-10T11:16:59.174 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839249 2021-02-10T11:16:59.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.581 7fef6e1d3700 1 Processor -- start 2021-02-10T11:16:59.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.581 7fef6e1d3700 1 -- start start 2021-02-10T11:16:59.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.581 7fef6e1d3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fef681b0470 0x7fef681b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:59.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.581 7fef6e1d3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fef680f20e0 0x7fef680f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:59.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.581 7fef6e1d3700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef680f0b50 con 0x7fef680ef610 2021-02-10T11:16:59.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.581 7fef6e1d3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef680f0820 con 0x7fef680f20e0 2021-02-10T11:16:59.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.581 7fef6e1d3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef680f24e0 con 0x7fef681b0470 2021-02-10T11:16:59.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.582 7fef67fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fef680f20e0 0x7fef680f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:59.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.582 7fef66ffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fef680ef610 0x7fef680efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46470/0 (socket says 172.21.2.101:46470) 2021-02-10T11:16:59.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.582 7fef66ffd700 1 -- 172.21.2.101:0/2074043101 learned_addr learned my addr 172.21.2.101:0/2074043101 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:16:59.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.582 7fef67fff700 1 -- 172.21.2.101:0/2074043101 >> v2:172.21.2.108:3300/0 conn(0x7fef681b0470 msgr2=0x7fef681b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:16:59.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.582 7fef67fff700 1 --2- 172.21.2.101:0/2074043101 >> v2:172.21.2.108:3300/0 conn(0x7fef681b0470 0x7fef681b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:59.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.582 7fef67fff700 1 -- 172.21.2.101:0/2074043101 >> v1:172.21.2.101:6789/0 conn(0x7fef680ef610 legacy=0x7fef680efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:59.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.582 7fef67fff700 1 -- 172.21.2.101:0/2074043101 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef680e52e0 con 0x7fef680f20e0 2021-02-10T11:16:59.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.582 7fef667fc700 1 -- 172.21.2.101:0/2074043101 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1980136494 0 0) 0x7fef5c0009b0 con 0x7fef680ef610 2021-02-10T11:16:59.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.582 7fef67fff700 1 --2- 172.21.2.101:0/2074043101 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef680f20e0 0x7fef680f0200 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7fef60006e30 tx=0x7fef60009de0).ready entity=mon.0 client_cookie=5e7a1ed44d150e6f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:59.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.582 7fef667fc700 1 -- 172.21.2.101:0/2074043101 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fef6000b300 con 0x7fef680f20e0 2021-02-10T11:16:59.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.582 7fef667fc700 1 -- 172.21.2.101:0/2074043101 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6000b580 con 0x7fef680f20e0 2021-02-10T11:16:59.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.582 7fef667fc700 1 -- 172.21.2.101:0/2074043101 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fef6000baa0 con 0x7fef680f20e0 2021-02-10T11:16:59.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.583 7fef6e1d3700 1 -- 172.21.2.101:0/2074043101 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef680f20e0 msgr2=0x7fef680f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:59.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.583 7fef6e1d3700 1 --2- 172.21.2.101:0/2074043101 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef680f20e0 0x7fef680f0200 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7fef60006e30 tx=0x7fef60009de0).stop 2021-02-10T11:16:59.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.583 7fef6e1d3700 1 -- 172.21.2.101:0/2074043101 shutdown_connections 2021-02-10T11:16:59.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.583 7fef6e1d3700 1 --2- 172.21.2.101:0/2074043101 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef680f20e0 0x7fef680f0200 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:59.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.583 7fef6e1d3700 1 --2- 172.21.2.101:0/2074043101 >> v2:172.21.2.108:3300/0 conn(0x7fef681b0470 0x7fef681b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:59.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.583 7fef6e1d3700 1 -- 172.21.2.101:0/2074043101 >> 172.21.2.101:0/2074043101 conn(0x7fef680ce3e0 msgr2=0x7fef680e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:59.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.583 7fef6e1d3700 1 -- 172.21.2.101:0/2074043101 shutdown_connections 2021-02-10T11:16:59.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.583 7fef6e1d3700 1 -- 172.21.2.101:0/2074043101 wait complete. 2021-02-10T11:16:59.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.598 7fef6e1d3700 1 Processor -- start 2021-02-10T11:16:59.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.598 7fef6e1d3700 1 -- start start 2021-02-10T11:16:59.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.598 7fef6e1d3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef68190920 0x7fef6808f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:59.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.598 7fef6e1d3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef68190d20 0x7fef6808fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:59.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.598 7fef6e1d3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef68193270 0x7fef68193670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:59.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.598 7fef6e1d3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef68090470 con 0x7fef68193270 2021-02-10T11:16:59.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.598 7fef6e1d3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef68193c90 con 0x7fef68190920 2021-02-10T11:16:59.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.598 7fef6e1d3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef68193e70 con 0x7fef68190d20 2021-02-10T11:16:59.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.598 7fef67fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef68193270 0x7fef68193670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:59.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.598 7fef67fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef68193270 0x7fef68193670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45984/0 (socket says 172.21.2.101:45984) 2021-02-10T11:16:59.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.598 7fef67fff700 1 -- 172.21.2.101:0/3425745281 learned_addr learned my addr 172.21.2.101:0/3425745281 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:16:59.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.599 7fef67fff700 1 -- 172.21.2.101:0/3425745281 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef68190d20 msgr2=0x7fef6808fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:59.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.599 7fef67fff700 1 --2- 172.21.2.101:0/3425745281 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef68190d20 0x7fef6808fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:59.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.599 7fef67fff700 1 -- 172.21.2.101:0/3425745281 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef68190920 msgr2=0x7fef6808f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:59.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.599 7fef67fff700 1 --2- 172.21.2.101:0/3425745281 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef68190920 0x7fef6808f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:59.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.599 7fef67fff700 1 -- 172.21.2.101:0/3425745281 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef68194090 con 0x7fef68193270 2021-02-10T11:16:59.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.599 7fef67fff700 1 --2- 172.21.2.101:0/3425745281 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef68193270 0x7fef68193670 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7fef6000b970 tx=0x7fef6000bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:59.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.599 7fef4ffff700 1 -- 172.21.2.101:0/3425745281 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fef60009380 con 0x7fef68193270 2021-02-10T11:16:59.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.599 7fef4ffff700 1 -- 172.21.2.101:0/3425745281 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef60001270 con 0x7fef68193270 2021-02-10T11:16:59.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.599 7fef6e1d3700 1 -- 172.21.2.101:0/3425745281 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef68194090 con 0x7fef68193270 2021-02-10T11:16:59.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.599 7fef4ffff700 1 -- 172.21.2.101:0/3425745281 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fef60001750 con 0x7fef68193270 2021-02-10T11:16:59.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.599 7fef6e1d3700 1 -- 172.21.2.101:0/3425745281 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef681920b0 con 0x7fef68193270 2021-02-10T11:16:59.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.600 7fef4ffff700 1 -- 172.21.2.101:0/3425745281 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fef60001c20 con 0x7fef68193270 2021-02-10T11:16:59.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.600 7fef4ffff700 1 --2- 172.21.2.101:0/3425745281 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fef48018730 0x7fef4801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:16:59.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.600 7fef4ffff700 1 -- 172.21.2.101:0/3425745281 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fef60032b30 con 0x7fef68193270 2021-02-10T11:16:59.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.600 7fef677fe700 1 --2- 172.21.2.101:0/3425745281 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fef48018730 0x7fef4801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:16:59.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.600 7fef677fe700 1 --2- 172.21.2.101:0/3425745281 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fef48018730 0x7fef4801abc0 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:16:59.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.631 7fef6e1d3700 1 -- 172.21.2.101:0/3425745281 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef68193c90 con 0x7fef68193270 2021-02-10T11:16:59.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.634 7fef4ffff700 1 -- 172.21.2.101:0/3425745281 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fef60001a10 con 0x7fef68193270 2021-02-10T11:16:59.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.760 7fef6e1d3700 1 -- 172.21.2.101:0/3425745281 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839249", "target": ["mgr", ""]}) v1 -- 0x7fef681920b0 con 0x7fef48018730 2021-02-10T11:16:59.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.786 7fef4ffff700 1 -- 172.21.2.101:0/3425745281 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fef580009b0 con 0x7fef48018730 2021-02-10T11:16:59.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.794 7fef6e1d3700 1 -- 172.21.2.101:0/3425745281 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fef48018730 msgr2=0x7fef4801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:59.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.794 7fef6e1d3700 1 --2- 172.21.2.101:0/3425745281 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fef48018730 0x7fef4801abc0 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:59.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.794 7fef6e1d3700 1 -- 172.21.2.101:0/3425745281 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef68193270 msgr2=0x7fef68193670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:16:59.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.794 7fef6e1d3700 1 --2- 172.21.2.101:0/3425745281 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef68193270 0x7fef68193670 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7fef6000b970 tx=0x7fef6000bd10).stop 2021-02-10T11:16:59.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.794 7fef6e1d3700 1 -- 172.21.2.101:0/3425745281 shutdown_connections 2021-02-10T11:16:59.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.794 7fef6e1d3700 1 --2- 172.21.2.101:0/3425745281 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fef48018730 0x7fef4801abc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:59.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.794 7fef6e1d3700 1 --2- 172.21.2.101:0/3425745281 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef68190d20 0x7fef6808fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:59.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.794 7fef6e1d3700 1 --2- 172.21.2.101:0/3425745281 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef68193270 0x7fef68193670 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:16:59.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.794 7fef6e1d3700 1 --2- 172.21.2.101:0/3425745281 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef68190920 0x7fef6808f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:16:59.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.794 7fef6e1d3700 1 -- 172.21.2.101:0/3425745281 >> 172.21.2.101:0/3425745281 conn(0x7fef680bf4e0 msgr2=0x7fef680be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:16:59.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.794 7fef6e1d3700 1 -- 172.21.2.101:0/3425745281 shutdown_connections 2021-02-10T11:16:59.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:16:59.794 7fef6e1d3700 1 -- 172.21.2.101:0/3425745281 wait complete. 2021-02-10T11:16:59.807 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839249 2021-02-10T11:17:00.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.215 7fa923632700 1 Processor -- start 2021-02-10T11:17:00.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.215 7fa923632700 1 -- start start 2021-02-10T11:17:00.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.215 7fa923632700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa91c1b0470 0x7fa91c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:00.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.215 7fa923632700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa91c0ef610 0x7fa91c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:00.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.215 7fa923632700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91c0f0b50 con 0x7fa91c0f0030 2021-02-10T11:17:00.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.215 7fa923632700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa91c0e52e0 con 0x7fa91c0ef610 2021-02-10T11:17:00.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.215 7fa923632700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c0e5490 con 0x7fa91c1b0470 2021-02-10T11:17:00.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.215 7fa920c4d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa91c0ef610 0x7fa91c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:00.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.216 7fa920c4d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa91c0ef610 0x7fa91c0efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45988/0 (socket says 172.21.2.101:45988) 2021-02-10T11:17:00.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.216 7fa920c4d700 1 -- 172.21.2.101:0/1352673425 learned_addr learned my addr 172.21.2.101:0/1352673425 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:17:00.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.216 7fa920c4d700 1 -- 172.21.2.101:0/1352673425 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa91c1b0470 msgr2=0x7fa91c1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:00.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.216 7fa920c4d700 1 --2- 172.21.2.101:0/1352673425 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa91c1b0470 0x7fa91c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:00.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.216 7fa920c4d700 1 -- 172.21.2.101:0/1352673425 >> v1:172.21.2.108:6789/0 conn(0x7fa91c0f0030 legacy=0x7fa91c0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:00.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.216 7fa920c4d700 1 -- 172.21.2.101:0/1352673425 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91c0f0430 con 0x7fa91c0ef610 2021-02-10T11:17:00.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.216 7fa920c4d700 1 --2- 172.21.2.101:0/1352673425 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa91c0ef610 0x7fa91c0efa10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7fa918006700 tx=0x7fa918009b80).ready entity=mon.0 client_cookie=f8db32ef9a711921 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:00.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.216 7fa90ffff700 1 -- 172.21.2.101:0/1352673425 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa91800c0c0 con 0x7fa91c0ef610 2021-02-10T11:17:00.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.216 7fa90ffff700 1 -- 172.21.2.101:0/1352673425 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91800c3c0 con 0x7fa91c0ef610 2021-02-10T11:17:00.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.216 7fa90ffff700 1 -- 172.21.2.101:0/1352673425 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa91800c900 con 0x7fa91c0ef610 2021-02-10T11:17:00.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.216 7fa923632700 1 -- 172.21.2.101:0/1352673425 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa91c0ef610 msgr2=0x7fa91c0efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:00.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.216 7fa923632700 1 --2- 172.21.2.101:0/1352673425 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa91c0ef610 0x7fa91c0efa10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7fa918006700 tx=0x7fa918009b80).stop 2021-02-10T11:17:00.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.217 7fa923632700 1 -- 172.21.2.101:0/1352673425 shutdown_connections 2021-02-10T11:17:00.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.217 7fa923632700 1 --2- 172.21.2.101:0/1352673425 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa91c0ef610 0x7fa91c0efa10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:00.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.217 7fa923632700 1 --2- 172.21.2.101:0/1352673425 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa91c1b0470 0x7fa91c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:00.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.217 7fa923632700 1 -- 172.21.2.101:0/1352673425 >> 172.21.2.101:0/1352673425 conn(0x7fa91c0ce3e0 msgr2=0x7fa91c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:00.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.217 7fa923632700 1 -- 172.21.2.101:0/1352673425 shutdown_connections 2021-02-10T11:17:00.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.217 7fa923632700 1 -- 172.21.2.101:0/1352673425 wait complete. 2021-02-10T11:17:00.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.232 7fa923632700 1 Processor -- start 2021-02-10T11:17:00.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.232 7fa923632700 1 -- start start 2021-02-10T11:17:00.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.232 7fa923632700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa91c0a6c90 0x7fa91c1924c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:00.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.232 7fa923632700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa91c192ae0 0x7fa91c192ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:00.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.232 7fa923632700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa91c0919e0 0x7fa91c091de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:00.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.232 7fa923632700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c19d2e0 con 0x7fa91c192ae0 2021-02-10T11:17:00.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.232 7fa923632700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c0914d0 con 0x7fa91c0919e0 2021-02-10T11:17:00.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.232 7fa923632700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c0a6820 con 0x7fa91c0a6c90 2021-02-10T11:17:00.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.233 7fa920c4d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa91c192ae0 0x7fa91c192ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:00.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.233 7fa920c4d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa91c192ae0 0x7fa91c192ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:45994/0 (socket says 172.21.2.101:45994) 2021-02-10T11:17:00.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.233 7fa920c4d700 1 -- 172.21.2.101:0/483913625 learned_addr learned my addr 172.21.2.101:0/483913625 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:17:00.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.233 7fa920c4d700 1 -- 172.21.2.101:0/483913625 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa91c0a6c90 msgr2=0x7fa91c1924c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:17:00.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.233 7fa920c4d700 1 --2- 172.21.2.101:0/483913625 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa91c0a6c90 0x7fa91c1924c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:00.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.233 7fa920c4d700 1 -- 172.21.2.101:0/483913625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa91c0919e0 msgr2=0x7fa91c091de0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:17:00.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.233 7fa920c4d700 1 --2- 172.21.2.101:0/483913625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa91c0919e0 0x7fa91c091de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:00.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.233 7fa920c4d700 1 -- 172.21.2.101:0/483913625 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91c092400 con 0x7fa91c192ae0 2021-02-10T11:17:00.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.233 7fa920c4d700 1 --2- 172.21.2.101:0/483913625 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa91c192ae0 0x7fa91c192ee0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fa91800b5e0 tx=0x7fa91800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:00.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.233 7fa90dffb700 1 -- 172.21.2.101:0/483913625 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa918009440 con 0x7fa91c192ae0 2021-02-10T11:17:00.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.233 7fa90dffb700 1 -- 172.21.2.101:0/483913625 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91800bd70 con 0x7fa91c192ae0 2021-02-10T11:17:00.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.233 7fa90dffb700 1 -- 172.21.2.101:0/483913625 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa918002300 con 0x7fa91c192ae0 2021-02-10T11:17:00.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.233 7fa923632700 1 -- 172.21.2.101:0/483913625 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91c092400 con 0x7fa91c192ae0 2021-02-10T11:17:00.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.233 7fa923632700 1 -- 172.21.2.101:0/483913625 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91c092990 con 0x7fa91c192ae0 2021-02-10T11:17:00.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.234 7fa90dffb700 1 -- 172.21.2.101:0/483913625 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa918000c90 con 0x7fa91c192ae0 2021-02-10T11:17:00.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.234 7fa90dffb700 1 --2- 172.21.2.101:0/483913625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8fc0186c0 0x7fa8fc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:00.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.234 7fa90dffb700 1 -- 172.21.2.101:0/483913625 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa918032570 con 0x7fa91c192ae0 2021-02-10T11:17:00.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.234 7fa92144e700 1 --2- 172.21.2.101:0/483913625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8fc0186c0 0x7fa8fc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:00.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.234 7fa92144e700 1 --2- 172.21.2.101:0/483913625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8fc0186c0 0x7fa8fc01abc0 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:00.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.265 7fa923632700 1 -- 172.21.2.101:0/483913625 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91c0e52e0 con 0x7fa91c192ae0 2021-02-10T11:17:00.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.268 7fa90dffb700 1 -- 172.21.2.101:0/483913625 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa918009440 con 0x7fa91c192ae0 2021-02-10T11:17:00.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.398 7fa923632700 1 -- 172.21.2.101:0/483913625 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839249", "target": ["mgr", ""]}) v1 -- 0x7fa91c092990 con 0x7fa8fc0186c0 2021-02-10T11:17:00.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.401 7fa90dffb700 1 -- 172.21.2.101:0/483913625 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fa910006960 con 0x7fa8fc0186c0 2021-02-10T11:17:00.410 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839249/1d42911c-4611-4d89-88bc-e649914d5023 2021-02-10T11:17:00.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.406 7fa923632700 1 -- 172.21.2.101:0/483913625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8fc0186c0 msgr2=0x7fa8fc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:00.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.406 7fa923632700 1 --2- 172.21.2.101:0/483913625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8fc0186c0 0x7fa8fc01abc0 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:00.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.406 7fa923632700 1 -- 172.21.2.101:0/483913625 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa91c192ae0 msgr2=0x7fa91c192ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:00.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.406 7fa923632700 1 --2- 172.21.2.101:0/483913625 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa91c192ae0 0x7fa91c192ee0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fa91800b5e0 tx=0x7fa91800b980).stop 2021-02-10T11:17:00.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.406 7fa923632700 1 -- 172.21.2.101:0/483913625 shutdown_connections 2021-02-10T11:17:00.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.406 7fa923632700 1 --2- 172.21.2.101:0/483913625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa91c0919e0 0x7fa91c091de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:00.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.406 7fa923632700 1 --2- 172.21.2.101:0/483913625 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa91c192ae0 0x7fa91c192ee0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:00.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.406 7fa923632700 1 --2- 172.21.2.101:0/483913625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8fc0186c0 0x7fa8fc01abc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:00.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.406 7fa923632700 1 --2- 172.21.2.101:0/483913625 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa91c0a6c90 0x7fa91c1924c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:00.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.406 7fa923632700 1 -- 172.21.2.101:0/483913625 >> 172.21.2.101:0/483913625 conn(0x7fa91c0ac550 msgr2=0x7fa91c0be1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:00.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.407 7fa923632700 1 -- 172.21.2.101:0/483913625 shutdown_connections 2021-02-10T11:17:00.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:00.407 7fa923632700 1 -- 172.21.2.101:0/483913625 wait complete. 2021-02-10T11:17:00.420 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839249 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839249/1d42911c-4611-4d89-88bc-e649914d5023 2021-02-10T11:17:00.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:00.420 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:00.420 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:00.756 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:00.756 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:00.757 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839249/1d42911c-4611-4d89-88bc-e649914d5023/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:01.136 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:01.136 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:01.136 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241553 s, 43.4 MB/s 2021-02-10T11:17:01.137 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:01.137 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:01.137 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:01.471 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:01.471 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:01.472 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839249/1d42911c-4611-4d89-88bc-e649914d5023/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:01.844 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:01.844 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:01.844 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0228742 s, 45.8 MB/s 2021-02-10T11:17:01.845 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:01.845 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:01.845 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:02.181 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:02.181 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:02.181 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839249/1d42911c-4611-4d89-88bc-e649914d5023/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:02.555 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:02.556 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:02.556 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023884 s, 43.9 MB/s 2021-02-10T11:17:02.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:02.557 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:02.557 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:02.891 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:02.891 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:02.891 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839249/1d42911c-4611-4d89-88bc-e649914d5023/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:03.267 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:03.267 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:03.268 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247694 s, 42.3 MB/s 2021-02-10T11:17:03.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:03.269 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:03.269 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:03.604 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:03.604 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:03.604 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839249/1d42911c-4611-4d89-88bc-e649914d5023/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:03.975 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:03.976 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:03.976 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238535 s, 44.0 MB/s 2021-02-10T11:17:03.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:03.977 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:03.977 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:04.311 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:04.311 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:04.311 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839249/1d42911c-4611-4d89-88bc-e649914d5023/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:04.685 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:04.685 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:04.685 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236685 s, 44.3 MB/s 2021-02-10T11:17:04.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:04.687 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:04.687 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:05.022 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:05.022 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:05.022 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839249/1d42911c-4611-4d89-88bc-e649914d5023/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:05.399 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:05.399 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:05.400 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.025699 s, 40.8 MB/s 2021-02-10T11:17:05.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:05.401 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:05.401 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:05.736 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:05.736 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:05.737 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839249/1d42911c-4611-4d89-88bc-e649914d5023/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:06.110 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:06.111 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:06.111 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241827 s, 43.4 MB/s 2021-02-10T11:17:06.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:06.112 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:06.112 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:06.447 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:06.447 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:06.448 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839249/1d42911c-4611-4d89-88bc-e649914d5023/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:06.824 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:06.824 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:06.825 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240345 s, 43.6 MB/s 2021-02-10T11:17:06.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:06.825 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:06.825 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:07.160 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:07.160 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:07.161 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839249/1d42911c-4611-4d89-88bc-e649914d5023/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:07.535 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:07.535 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:07.535 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248538 s, 42.2 MB/s 2021-02-10T11:17:07.536 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839250 2021-02-10T11:17:07.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.943 7f9bb131e700 1 Processor -- start 2021-02-10T11:17:07.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.943 7f9bb131e700 1 -- start start 2021-02-10T11:17:07.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.943 7f9bb131e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9bac1b0470 0x7f9bac1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:07.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.943 7f9bb131e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9bac0f20e0 0x7f9bac0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:07.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.943 7f9bb131e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bac0f0b50 con 0x7f9bac0ef610 2021-02-10T11:17:07.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.943 7f9bb131e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bac0f0820 con 0x7f9bac1b0470 2021-02-10T11:17:07.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.943 7f9bb131e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac0f24e0 con 0x7f9bac0f20e0 2021-02-10T11:17:07.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.944 7f9bab07d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9bac1b0470 0x7f9bac1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:07.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.944 7f9baa87c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f9bac0ef610 0x7f9bac0efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46494/0 (socket says 172.21.2.101:46494) 2021-02-10T11:17:07.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.944 7f9bab87e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9bac0f20e0 0x7f9bac0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:07.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.944 7f9baa87c700 1 -- 172.21.2.101:0/3596112371 learned_addr learned my addr 172.21.2.101:0/3596112371 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:17:07.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.944 7f9bab07d700 1 -- 172.21.2.101:0/3596112371 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9bac0f20e0 msgr2=0x7f9bac0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:07.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.944 7f9bab07d700 1 --2- 172.21.2.101:0/3596112371 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9bac0f20e0 0x7f9bac0f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:07.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.944 7f9bab07d700 1 -- 172.21.2.101:0/3596112371 >> v1:172.21.2.101:6789/0 conn(0x7f9bac0ef610 legacy=0x7f9bac0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:07.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.944 7f9bab07d700 1 -- 172.21.2.101:0/3596112371 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bac0e52e0 con 0x7f9bac1b0470 2021-02-10T11:17:07.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.944 7f9bab07d700 1 --2- 172.21.2.101:0/3596112371 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9bac1b0470 0x7f9bac1b0870 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f9b94006650 tx=0x7f9b94009a80).ready entity=mon.0 client_cookie=81e74ad0e826a28f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:07.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.944 7f9baa07b700 1 -- 172.21.2.101:0/3596112371 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9b9400c0c0 con 0x7f9bac1b0470 2021-02-10T11:17:07.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.944 7f9baa07b700 1 -- 172.21.2.101:0/3596112371 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9400c340 con 0x7f9bac1b0470 2021-02-10T11:17:07.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.944 7f9baa07b700 1 -- 172.21.2.101:0/3596112371 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9b9400c860 con 0x7f9bac1b0470 2021-02-10T11:17:07.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.945 7f9bb131e700 1 -- 172.21.2.101:0/3596112371 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9bac1b0470 msgr2=0x7f9bac1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:07.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.945 7f9bb131e700 1 --2- 172.21.2.101:0/3596112371 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9bac1b0470 0x7f9bac1b0870 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f9b94006650 tx=0x7f9b94009a80).stop 2021-02-10T11:17:07.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.945 7f9bb131e700 1 -- 172.21.2.101:0/3596112371 shutdown_connections 2021-02-10T11:17:07.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.945 7f9bb131e700 1 --2- 172.21.2.101:0/3596112371 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9bac0f20e0 0x7f9bac0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:07.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.945 7f9bb131e700 1 --2- 172.21.2.101:0/3596112371 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9bac1b0470 0x7f9bac1b0870 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:07.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.945 7f9bb131e700 1 -- 172.21.2.101:0/3596112371 >> 172.21.2.101:0/3596112371 conn(0x7f9bac0ce3e0 msgr2=0x7f9bac0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:07.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.945 7f9bb131e700 1 -- 172.21.2.101:0/3596112371 shutdown_connections 2021-02-10T11:17:07.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.945 7f9bb131e700 1 -- 172.21.2.101:0/3596112371 wait complete. 2021-02-10T11:17:07.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.960 7f9bb131e700 1 Processor -- start 2021-02-10T11:17:07.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.961 7f9bb131e700 1 -- start start 2021-02-10T11:17:07.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.961 7f9bb131e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9bac190920 0x7f9bac0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:07.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.961 7f9bb131e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9bac0b8200 0x7f9bac0b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:07.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.961 7f9bb131e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9bac0c1d80 0x7f9bac0c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:07.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.961 7f9bb131e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac190da0 con 0x7f9bac190920 2021-02-10T11:17:07.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.961 7f9bb131e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac0b8c20 con 0x7f9bac0c1d80 2021-02-10T11:17:07.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.961 7f9bb131e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac0b8e10 con 0x7f9bac0b8200 2021-02-10T11:17:07.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.961 7f9baa87c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9bac0b8200 0x7f9bac0b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:07.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.961 7f9bab07d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9bac190920 0x7f9bac0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:07.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.961 7f9baa87c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9bac0b8200 0x7f9bac0b8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:38736/0 (socket says 172.21.2.101:38736) 2021-02-10T11:17:07.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.961 7f9baa87c700 1 -- 172.21.2.101:0/1394392719 learned_addr learned my addr 172.21.2.101:0/1394392719 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:17:07.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.961 7f9bab07d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9bac190920 0x7f9bac0b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46008/0 (socket says 172.21.2.101:46008) 2021-02-10T11:17:07.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.961 7f9bab87e700 1 --2- 172.21.2.101:0/1394392719 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9bac0c1d80 0x7f9bac0c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:07.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.962 7f9bab07d700 1 -- 172.21.2.101:0/1394392719 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9bac0b8200 msgr2=0x7f9bac0b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:07.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.962 7f9bab07d700 1 --2- 172.21.2.101:0/1394392719 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9bac0b8200 0x7f9bac0b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:07.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.962 7f9bab07d700 1 -- 172.21.2.101:0/1394392719 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9bac0c1d80 msgr2=0x7f9bac0c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:07.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.962 7f9bab07d700 1 --2- 172.21.2.101:0/1394392719 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9bac0c1d80 0x7f9bac0c2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:07.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.962 7f9bab07d700 1 -- 172.21.2.101:0/1394392719 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bac0c27d0 con 0x7f9bac190920 2021-02-10T11:17:07.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.962 7f9bab87e700 1 --2- 172.21.2.101:0/1394392719 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9bac0c1d80 0x7f9bac0c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:17:07.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.962 7f9bab07d700 1 --2- 172.21.2.101:0/1394392719 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9bac190920 0x7f9bac0b7be0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f9b9400b530 tx=0x7f9b9400b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:07.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.962 7f9ba3fff700 1 -- 172.21.2.101:0/1394392719 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9b94000c90 con 0x7f9bac190920 2021-02-10T11:17:07.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.962 7f9ba3fff700 1 -- 172.21.2.101:0/1394392719 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b94002030 con 0x7f9bac190920 2021-02-10T11:17:07.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.962 7f9bb131e700 1 -- 172.21.2.101:0/1394392719 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bac0c27d0 con 0x7f9bac190920 2021-02-10T11:17:07.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.962 7f9ba3fff700 1 -- 172.21.2.101:0/1394392719 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9b94002490 con 0x7f9bac190920 2021-02-10T11:17:07.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.962 7f9bb131e700 1 -- 172.21.2.101:0/1394392719 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bac0c27d0 con 0x7f9bac190920 2021-02-10T11:17:07.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.963 7f9ba3fff700 1 -- 172.21.2.101:0/1394392719 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9b94002210 con 0x7f9bac190920 2021-02-10T11:17:07.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.963 7f9ba3fff700 1 --2- 172.21.2.101:0/1394392719 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b90018730 0x7f9b9001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:07.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.963 7f9ba3fff700 1 -- 172.21.2.101:0/1394392719 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9b940325b0 con 0x7f9bac190920 2021-02-10T11:17:07.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.963 7f9baa87c700 1 --2- 172.21.2.101:0/1394392719 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b90018730 0x7f9b9001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:07.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.963 7f9baa87c700 1 --2- 172.21.2.101:0/1394392719 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b90018730 0x7f9b9001ac10 crc :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:07.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.993 7f9bb131e700 1 -- 172.21.2.101:0/1394392719 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bac105ae0 con 0x7f9bac190920 2021-02-10T11:17:08.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:07.996 7f9ba3fff700 1 -- 172.21.2.101:0/1394392719 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9b94000940 con 0x7f9bac190920 2021-02-10T11:17:08.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.123 7f9bb131e700 1 -- 172.21.2.101:0/1394392719 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839250", "target": ["mgr", ""]}) v1 -- 0x7f9bac0f0b50 con 0x7f9b90018730 2021-02-10T11:17:08.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.148 7f9ba3fff700 1 -- 172.21.2.101:0/1394392719 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f9b9c0019e0 con 0x7f9b90018730 2021-02-10T11:17:08.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.156 7f9bb131e700 1 -- 172.21.2.101:0/1394392719 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b90018730 msgr2=0x7f9b9001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:08.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.156 7f9bb131e700 1 --2- 172.21.2.101:0/1394392719 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b90018730 0x7f9b9001ac10 crc :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:08.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.156 7f9bb131e700 1 -- 172.21.2.101:0/1394392719 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9bac190920 msgr2=0x7f9bac0b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:08.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.156 7f9bb131e700 1 --2- 172.21.2.101:0/1394392719 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9bac190920 0x7f9bac0b7be0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f9b9400b530 tx=0x7f9b9400b8d0).stop 2021-02-10T11:17:08.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.157 7f9bb131e700 1 -- 172.21.2.101:0/1394392719 shutdown_connections 2021-02-10T11:17:08.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.157 7f9bb131e700 1 --2- 172.21.2.101:0/1394392719 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b90018730 0x7f9b9001ac10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:08.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.157 7f9bb131e700 1 --2- 172.21.2.101:0/1394392719 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9bac0b8200 0x7f9bac0b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:08.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.157 7f9bb131e700 1 --2- 172.21.2.101:0/1394392719 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9bac0c1d80 0x7f9bac0c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:08.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.157 7f9bb131e700 1 --2- 172.21.2.101:0/1394392719 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9bac190920 0x7f9bac0b7be0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:08.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.157 7f9bb131e700 1 -- 172.21.2.101:0/1394392719 >> 172.21.2.101:0/1394392719 conn(0x7f9bac094e10 msgr2=0x7f9bac0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:08.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.157 7f9bb131e700 1 -- 172.21.2.101:0/1394392719 shutdown_connections 2021-02-10T11:17:08.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.157 7f9bb131e700 1 -- 172.21.2.101:0/1394392719 wait complete. 2021-02-10T11:17:08.170 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839250 2021-02-10T11:17:08.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.579 7fc9b3a70700 1 Processor -- start 2021-02-10T11:17:08.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.579 7fc9b3a70700 1 -- start start 2021-02-10T11:17:08.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.580 7fc9b3a70700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc9ac1b0870 0x7fc9ac0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:08.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.580 7fc9b3a70700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9ac0efe00 0x7fc9ac0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:08.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.580 7fc9b3a70700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ac1b0c70 con 0x7fc9ac1b0470 2021-02-10T11:17:08.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.580 7fc9b3a70700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ac0f0b50 con 0x7fc9ac1b0870 2021-02-10T11:17:08.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.580 7fc9b3a70700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac0f0820 con 0x7fc9ac0efe00 2021-02-10T11:17:08.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.580 7fc9b188c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc9ac1b0470 0x7fc9ac0ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46506/0 (socket says 172.21.2.101:46506) 2021-02-10T11:17:08.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.580 7fc9b188c700 1 -- 172.21.2.101:0/2623246952 learned_addr learned my addr 172.21.2.101:0/2623246952 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:17:08.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.580 7fc9b088a700 1 -- 172.21.2.101:0/2623246952 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1543905217 0 0) 0x7fc9a8001180 con 0x7fc9ac1b0470 2021-02-10T11:17:08.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.580 7fc9b088a700 1 -- 172.21.2.101:0/2623246952 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc998002fe0 con 0x7fc9ac1b0470 2021-02-10T11:17:08.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.580 7fc9b208d700 1 --2- 172.21.2.101:0/2623246952 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9ac0efe00 0x7fc9ac0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:08.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.580 7fc9b108b700 1 --2- 172.21.2.101:0/2623246952 >> v2:172.21.2.108:3300/0 conn(0x7fc9ac1b0870 0x7fc9ac0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:08.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.581 7fc9b088a700 1 -- 172.21.2.101:0/2623246952 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3650647096 0 0) 0x7fc9a8001180 con 0x7fc9ac1b0470 2021-02-10T11:17:08.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.581 7fc9b088a700 1 -- 172.21.2.101:0/2623246952 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9ac0efe00 msgr2=0x7fc9ac0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:08.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.581 7fc9b088a700 1 --2- 172.21.2.101:0/2623246952 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9ac0efe00 0x7fc9ac0f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:08.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.581 7fc9b088a700 1 -- 172.21.2.101:0/2623246952 >> v2:172.21.2.108:3300/0 conn(0x7fc9ac1b0870 msgr2=0x7fc9ac0f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:08.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.581 7fc9b088a700 1 --2- 172.21.2.101:0/2623246952 >> v2:172.21.2.108:3300/0 conn(0x7fc9ac1b0870 0x7fc9ac0f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:08.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.581 7fc9b088a700 1 -- 172.21.2.101:0/2623246952 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac0e52e0 con 0x7fc9ac1b0470 2021-02-10T11:17:08.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.581 7fc9b088a700 1 -- 172.21.2.101:0/2623246952 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fc9a8001680 con 0x7fc9ac1b0470 2021-02-10T11:17:08.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.581 7fc9b088a700 1 -- 172.21.2.101:0/2623246952 >> v1:172.21.2.101:6789/0 conn(0x7fc9ac1b0470 legacy=0x7fc9ac0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:08.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.581 7fc9b088a700 1 --2- 172.21.2.101:0/2623246952 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc998003af0 0x7fc998005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:08.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.581 7fc9b088a700 1 --2- 172.21.2.101:0/2623246952 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc99804a780 0x7fc99804cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:08.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.581 7fc9b088a700 1 --2- 172.21.2.101:0/2623246952 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc99804d500 0x7fc99804fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:08.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.581 7fc9b088a700 1 -- 172.21.2.101:0/2623246952 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9980502d0 con 0x7fc99804d500 2021-02-10T11:17:08.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.581 7fc9b088a700 1 -- 172.21.2.101:0/2623246952 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9980504f0 con 0x7fc99804a780 2021-02-10T11:17:08.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.581 7fc9b088a700 1 -- 172.21.2.101:0/2623246952 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc998050710 con 0x7fc998003af0 2021-02-10T11:17:08.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.581 7fc9b208d700 1 --2- 172.21.2.101:0/2623246952 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc99804d500 0x7fc99804fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:08.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.581 7fc9b108b700 1 --2- 172.21.2.101:0/2623246952 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc99804a780 0x7fc99804cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:08.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.581 7fc9b188c700 1 --2- 172.21.2.101:0/2623246952 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc998003af0 0x7fc998005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:08.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.581 7fc9b208d700 1 -- 172.21.2.101:0/2623246952 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc998003af0 msgr2=0x7fc998005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:08.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.581 7fc9b208d700 1 --2- 172.21.2.101:0/2623246952 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc998003af0 0x7fc998005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:08.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.581 7fc9b208d700 1 -- 172.21.2.101:0/2623246952 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc99804a780 msgr2=0x7fc99804cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:08.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.581 7fc9b208d700 1 --2- 172.21.2.101:0/2623246952 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc99804a780 0x7fc99804cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:08.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.581 7fc9b208d700 1 -- 172.21.2.101:0/2623246952 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9980509b0 con 0x7fc99804d500 2021-02-10T11:17:08.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.582 7fc9b208d700 1 --2- 172.21.2.101:0/2623246952 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc99804d500 0x7fc99804fa20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fc9a4009b10 tx=0x7fc9a4009eb0).ready entity=mon.0 client_cookie=1646f1e3be49f069 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:08.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.582 7fc9b088a700 1 -- 172.21.2.101:0/2623246952 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc9a400b510 con 0x7fc99804d500 2021-02-10T11:17:08.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.582 7fc9b088a700 1 -- 172.21.2.101:0/2623246952 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a4003030 con 0x7fc99804d500 2021-02-10T11:17:08.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.582 7fc9b088a700 1 -- 172.21.2.101:0/2623246952 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc9a400b790 con 0x7fc99804d500 2021-02-10T11:17:08.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.582 7fc9b3a70700 1 -- 172.21.2.101:0/2623246952 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc99804d500 msgr2=0x7fc99804fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:08.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.582 7fc9b3a70700 1 --2- 172.21.2.101:0/2623246952 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc99804d500 0x7fc99804fa20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fc9a4009b10 tx=0x7fc9a4009eb0).stop 2021-02-10T11:17:08.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.582 7fc9b208d700 1 -- 172.21.2.101:0/2623246952 reap_dead start 2021-02-10T11:17:08.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.582 7fc9b3a70700 1 -- 172.21.2.101:0/2623246952 shutdown_connections 2021-02-10T11:17:08.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.582 7fc9b3a70700 1 -- 172.21.2.101:0/2623246952 >> 172.21.2.101:0/2623246952 conn(0x7fc9ac0ce3e0 msgr2=0x7fc9ac0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:08.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.582 7fc9b3a70700 1 -- 172.21.2.101:0/2623246952 shutdown_connections 2021-02-10T11:17:08.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.582 7fc9b3a70700 1 -- 172.21.2.101:0/2623246952 wait complete. 2021-02-10T11:17:08.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.597 7fc9b3a70700 1 Processor -- start 2021-02-10T11:17:08.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.597 7fc9b3a70700 1 -- start start 2021-02-10T11:17:08.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.597 7fc9b3a70700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9ac0b7db0 0x7fc9ac0b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:08.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.597 7fc9b3a70700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9ac0b8880 0x7fc9ac1a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:08.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.597 7fc9b3a70700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9ac0b8c80 0x7fc9ac1a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:08.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.597 7fc9b3a70700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac0b79b0 con 0x7fc9ac0b7db0 2021-02-10T11:17:08.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.597 7fc9b3a70700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac1a8090 con 0x7fc9ac0b8c80 2021-02-10T11:17:08.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.597 7fc9b3a70700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac1a82b0 con 0x7fc9ac0b8880 2021-02-10T11:17:08.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.597 7fc9b188c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9ac0b7db0 0x7fc9ac0b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:08.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.598 7fc9b188c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9ac0b7db0 0x7fc9ac0b8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46028/0 (socket says 172.21.2.101:46028) 2021-02-10T11:17:08.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.598 7fc9b188c700 1 -- 172.21.2.101:0/2209801550 learned_addr learned my addr 172.21.2.101:0/2209801550 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:17:08.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.598 7fc9b188c700 1 -- 172.21.2.101:0/2209801550 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9ac0b8880 msgr2=0x7fc9ac1a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:08.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.598 7fc9b188c700 1 --2- 172.21.2.101:0/2209801550 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9ac0b8880 0x7fc9ac1a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:08.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.598 7fc9b188c700 1 -- 172.21.2.101:0/2209801550 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9ac0b8c80 msgr2=0x7fc9ac1a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:08.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.598 7fc9b188c700 1 --2- 172.21.2.101:0/2209801550 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9ac0b8c80 0x7fc9ac1a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:08.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.598 7fc9b188c700 1 -- 172.21.2.101:0/2209801550 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac190c10 con 0x7fc9ac0b7db0 2021-02-10T11:17:08.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.598 7fc9b188c700 1 --2- 172.21.2.101:0/2209801550 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9ac0b7db0 0x7fc9ac0b8260 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7fc9a8006f00 tx=0x7fc9a800b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:08.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.598 7fc9a27fc700 1 -- 172.21.2.101:0/2209801550 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc9a800bd30 con 0x7fc9ac0b7db0 2021-02-10T11:17:08.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.598 7fc9a27fc700 1 -- 172.21.2.101:0/2209801550 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a80060d0 con 0x7fc9ac0b7db0 2021-02-10T11:17:08.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.598 7fc9b3a70700 1 -- 172.21.2.101:0/2209801550 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ac190c10 con 0x7fc9ac0b7db0 2021-02-10T11:17:08.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.598 7fc9a27fc700 1 -- 172.21.2.101:0/2209801550 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc9a80043a0 con 0x7fc9ac0b7db0 2021-02-10T11:17:08.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.598 7fc9b3a70700 1 -- 172.21.2.101:0/2209801550 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ac190c10 con 0x7fc9ac0b7db0 2021-02-10T11:17:08.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.599 7fc9a27fc700 1 -- 172.21.2.101:0/2209801550 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc9a80062b0 con 0x7fc9ac0b7db0 2021-02-10T11:17:08.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.599 7fc9a27fc700 1 --2- 172.21.2.101:0/2209801550 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc994018730 0x7fc99401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:08.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.599 7fc9a27fc700 1 -- 172.21.2.101:0/2209801550 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc9a8032650 con 0x7fc9ac0b7db0 2021-02-10T11:17:08.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.599 7fc9b108b700 1 --2- 172.21.2.101:0/2209801550 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc994018730 0x7fc99401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:08.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.599 7fc9b108b700 1 --2- 172.21.2.101:0/2209801550 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc994018730 0x7fc99401ac10 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:08.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.629 7fc9b3a70700 1 -- 172.21.2.101:0/2209801550 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ac105b30 con 0x7fc9ac0b7db0 2021-02-10T11:17:08.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.632 7fc9a27fc700 1 -- 172.21.2.101:0/2209801550 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc9a8000970 con 0x7fc9ac0b7db0 2021-02-10T11:17:08.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.765 7fc9b3a70700 1 -- 172.21.2.101:0/2209801550 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839250", "target": ["mgr", ""]}) v1 -- 0x7fc9ac0f0b50 con 0x7fc994018730 2021-02-10T11:17:08.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.768 7fc9a27fc700 1 -- 172.21.2.101:0/2209801550 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fc99c006e70 con 0x7fc994018730 2021-02-10T11:17:08.773 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839250/16e9d1c0-d8b7-4265-9f37-f35c0fb8a20e 2021-02-10T11:17:08.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.770 7fc9b3a70700 1 -- 172.21.2.101:0/2209801550 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc994018730 msgr2=0x7fc99401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:08.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.770 7fc9b3a70700 1 --2- 172.21.2.101:0/2209801550 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc994018730 0x7fc99401ac10 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:08.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.770 7fc9b3a70700 1 -- 172.21.2.101:0/2209801550 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9ac0b7db0 msgr2=0x7fc9ac0b8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:08.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.770 7fc9b3a70700 1 --2- 172.21.2.101:0/2209801550 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9ac0b7db0 0x7fc9ac0b8260 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7fc9a8006f00 tx=0x7fc9a800b940).stop 2021-02-10T11:17:08.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.770 7fc9b3a70700 1 -- 172.21.2.101:0/2209801550 shutdown_connections 2021-02-10T11:17:08.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.770 7fc9b3a70700 1 --2- 172.21.2.101:0/2209801550 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc994018730 0x7fc99401ac10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:08.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.770 7fc9b3a70700 1 --2- 172.21.2.101:0/2209801550 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9ac0b8880 0x7fc9ac1a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:08.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.770 7fc9b3a70700 1 --2- 172.21.2.101:0/2209801550 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9ac0b8c80 0x7fc9ac1a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:08.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.770 7fc9b3a70700 1 --2- 172.21.2.101:0/2209801550 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9ac0b7db0 0x7fc9ac0b8260 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:08.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.770 7fc9b3a70700 1 -- 172.21.2.101:0/2209801550 >> 172.21.2.101:0/2209801550 conn(0x7fc9ac094e10 msgr2=0x7fc9ac0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:08.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.770 7fc9b3a70700 1 -- 172.21.2.101:0/2209801550 shutdown_connections 2021-02-10T11:17:08.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:08.770 7fc9b3a70700 1 -- 172.21.2.101:0/2209801550 wait complete. 2021-02-10T11:17:08.783 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839250 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839250/16e9d1c0-d8b7-4265-9f37-f35c0fb8a20e 2021-02-10T11:17:08.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:08.783 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:08.784 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:09.119 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:09.119 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:09.120 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839250/16e9d1c0-d8b7-4265-9f37-f35c0fb8a20e/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:09.496 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:09.496 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:09.496 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0227364 s, 46.1 MB/s 2021-02-10T11:17:09.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:09.497 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:09.498 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:09.833 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:09.833 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:09.834 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839250/16e9d1c0-d8b7-4265-9f37-f35c0fb8a20e/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:10.207 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:10.207 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:10.207 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0225478 s, 46.5 MB/s 2021-02-10T11:17:10.208 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:10.208 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:10.208 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:10.543 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:10.544 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:10.544 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839250/16e9d1c0-d8b7-4265-9f37-f35c0fb8a20e/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:10.920 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:10.920 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:10.920 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251023 s, 41.8 MB/s 2021-02-10T11:17:10.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:10.921 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:10.921 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:11.256 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:11.256 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:11.257 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839250/16e9d1c0-d8b7-4265-9f37-f35c0fb8a20e/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:11.632 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:11.632 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:11.632 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249881 s, 42.0 MB/s 2021-02-10T11:17:11.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:11.634 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:11.634 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:11.967 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:11.968 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:11.968 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839250/16e9d1c0-d8b7-4265-9f37-f35c0fb8a20e/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:12.341 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:12.342 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:12.342 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235694 s, 44.5 MB/s 2021-02-10T11:17:12.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:12.343 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:12.343 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:12.678 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:12.678 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:12.678 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839250/16e9d1c0-d8b7-4265-9f37-f35c0fb8a20e/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:13.053 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:13.053 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:13.054 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246593 s, 42.5 MB/s 2021-02-10T11:17:13.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:13.055 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:13.055 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:13.388 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:13.388 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:13.389 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839250/16e9d1c0-d8b7-4265-9f37-f35c0fb8a20e/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:13.761 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:13.762 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:13.762 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252515 s, 41.5 MB/s 2021-02-10T11:17:13.763 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:13.763 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:13.763 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:14.099 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:14.099 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:14.100 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839250/16e9d1c0-d8b7-4265-9f37-f35c0fb8a20e/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:14.472 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:14.472 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:14.472 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240849 s, 43.5 MB/s 2021-02-10T11:17:14.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:14.473 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:14.473 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:14.808 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:14.809 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:14.809 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839250/16e9d1c0-d8b7-4265-9f37-f35c0fb8a20e/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:15.184 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:15.185 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:15.185 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251966 s, 41.6 MB/s 2021-02-10T11:17:15.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:15.186 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:15.186 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:15.520 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:15.521 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:15.521 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839250/16e9d1c0-d8b7-4265-9f37-f35c0fb8a20e/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:15.895 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:15.895 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:15.895 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243187 s, 43.1 MB/s 2021-02-10T11:17:15.896 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839251 2021-02-10T11:17:15.926 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:17:15.942 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:17:16.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.306 7f4180562700 1 Processor -- start 2021-02-10T11:17:16.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.306 7f4180562700 1 -- start start 2021-02-10T11:17:16.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.306 7f4180562700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41781b0470 0x7f41781b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:16.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.306 7f4180562700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f41780ef610 0x7f41780efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:16.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.306 7f4180562700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41780f0b50 con 0x7f41780f0030 2021-02-10T11:17:16.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.306 7f4180562700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41780e52e0 con 0x7f41780ef610 2021-02-10T11:17:16.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.306 7f4180562700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41780e5490 con 0x7f41781b0470 2021-02-10T11:17:16.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.307 7f417eb7f700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f41780f0030 0x7f41780f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:39936/0 (socket says 172.21.2.101:39936) 2021-02-10T11:17:16.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.307 7f417db7d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f41780ef610 0x7f41780efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:16.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.307 7f417eb7f700 1 -- 172.21.2.101:0/125984719 learned_addr learned my addr 172.21.2.101:0/125984719 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:17:16.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.307 7f417e37e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41781b0470 0x7f41781b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:16.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.308 7f417db7d700 1 -- 172.21.2.101:0/125984719 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41781b0470 msgr2=0x7f41781b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:16.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.308 7f417db7d700 1 --2- 172.21.2.101:0/125984719 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41781b0470 0x7f41781b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:16.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.308 7f417db7d700 1 -- 172.21.2.101:0/125984719 >> v1:172.21.2.108:6789/0 conn(0x7f41780f0030 legacy=0x7f41780f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:16.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.308 7f417d37c700 1 -- 172.21.2.101:0/125984719 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 263482573 0 0) 0x7f4170001510 con 0x7f41780f0030 2021-02-10T11:17:16.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.308 7f417db7d700 1 -- 172.21.2.101:0/125984719 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41780f0430 con 0x7f41780ef610 2021-02-10T11:17:16.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.308 7f417e37e700 1 --2- 172.21.2.101:0/125984719 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41781b0470 0x7f41781b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:17:16.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.309 7f417db7d700 1 --2- 172.21.2.101:0/125984719 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f41780ef610 0x7f41780efa10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f4174006650 tx=0x7f4174009a80).ready entity=mon.1 client_cookie=87ecb8dced08ddda server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:16.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.309 7f417d37c700 1 -- 172.21.2.101:0/125984719 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f41740009b0 con 0x7f41780ef610 2021-02-10T11:17:16.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.309 7f417d37c700 1 -- 172.21.2.101:0/125984719 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417400c0c0 con 0x7f41780ef610 2021-02-10T11:17:16.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.309 7f417d37c700 1 -- 172.21.2.101:0/125984719 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f417400c5e0 con 0x7f41780ef610 2021-02-10T11:17:16.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.309 7f4180562700 1 -- 172.21.2.101:0/125984719 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f41780ef610 msgr2=0x7f41780efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:16.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.309 7f4180562700 1 --2- 172.21.2.101:0/125984719 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f41780ef610 0x7f41780efa10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f4174006650 tx=0x7f4174009a80).stop 2021-02-10T11:17:16.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.310 7f4180562700 1 -- 172.21.2.101:0/125984719 shutdown_connections 2021-02-10T11:17:16.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.310 7f4180562700 1 --2- 172.21.2.101:0/125984719 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f41780ef610 0x7f41780efa10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:16.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.310 7f4180562700 1 --2- 172.21.2.101:0/125984719 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41781b0470 0x7f41781b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:16.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.310 7f4180562700 1 -- 172.21.2.101:0/125984719 >> 172.21.2.101:0/125984719 conn(0x7f41780ce3e0 msgr2=0x7f41780e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:16.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.310 7f4180562700 1 -- 172.21.2.101:0/125984719 shutdown_connections 2021-02-10T11:17:16.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.310 7f4180562700 1 -- 172.21.2.101:0/125984719 wait complete. 2021-02-10T11:17:16.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.325 7f4180562700 1 Processor -- start 2021-02-10T11:17:16.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.325 7f4180562700 1 -- start start 2021-02-10T11:17:16.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.325 7f4180562700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4178105d10 0x7f417809f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:16.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.325 7f4180562700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41780961f0 0x7f41780965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:16.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.325 7f4180562700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4178106890 0x7f4178106c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:16.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.325 7f4180562700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4178106110 con 0x7f41780961f0 2021-02-10T11:17:16.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.325 7f4180562700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4178106380 con 0x7f4178106890 2021-02-10T11:17:16.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.325 7f4180562700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41780a6620 con 0x7f4178105d10 2021-02-10T11:17:16.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.326 7f417db7d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41780961f0 0x7f41780965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:16.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.326 7f417db7d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41780961f0 0x7f41780965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46036/0 (socket says 172.21.2.101:46036) 2021-02-10T11:17:16.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.326 7f417db7d700 1 -- 172.21.2.101:0/2501842483 learned_addr learned my addr 172.21.2.101:0/2501842483 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:17:16.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.326 7f417eb7f700 1 --2- 172.21.2.101:0/2501842483 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4178106890 0x7f4178106c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:16.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.326 7f417e37e700 1 --2- 172.21.2.101:0/2501842483 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4178105d10 0x7f417809f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:16.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.326 7f417db7d700 1 -- 172.21.2.101:0/2501842483 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4178105d10 msgr2=0x7f417809f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:16.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.326 7f417db7d700 1 --2- 172.21.2.101:0/2501842483 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4178105d10 0x7f417809f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:16.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.326 7f417db7d700 1 -- 172.21.2.101:0/2501842483 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4178106890 msgr2=0x7f4178106c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:16.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.326 7f417db7d700 1 --2- 172.21.2.101:0/2501842483 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4178106890 0x7f4178106c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:16.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.326 7f417db7d700 1 -- 172.21.2.101:0/2501842483 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41781072b0 con 0x7f41780961f0 2021-02-10T11:17:16.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.326 7f417db7d700 1 --2- 172.21.2.101:0/2501842483 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41780961f0 0x7f41780965f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f4174009630 tx=0x7f417400b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:16.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.326 7f416affd700 1 -- 172.21.2.101:0/2501842483 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f41740093a0 con 0x7f41780961f0 2021-02-10T11:17:16.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.326 7f416affd700 1 -- 172.21.2.101:0/2501842483 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417400bcf0 con 0x7f41780961f0 2021-02-10T11:17:16.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.326 7f4180562700 1 -- 172.21.2.101:0/2501842483 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41781072b0 con 0x7f41780961f0 2021-02-10T11:17:16.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.326 7f416affd700 1 -- 172.21.2.101:0/2501842483 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4174005560 con 0x7f41780961f0 2021-02-10T11:17:16.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.326 7f4180562700 1 -- 172.21.2.101:0/2501842483 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4178107840 con 0x7f41780961f0 2021-02-10T11:17:16.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.327 7f416affd700 1 -- 172.21.2.101:0/2501842483 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f417400a000 con 0x7f41780961f0 2021-02-10T11:17:16.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.327 7f416affd700 1 --2- 172.21.2.101:0/2501842483 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4160018730 0x7f416001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:16.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.327 7f416affd700 1 -- 172.21.2.101:0/2501842483 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f41740327e0 con 0x7f41780961f0 2021-02-10T11:17:16.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.327 7f417e37e700 1 --2- 172.21.2.101:0/2501842483 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4160018730 0x7f416001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:16.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.327 7f417e37e700 1 --2- 172.21.2.101:0/2501842483 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4160018730 0x7f416001abc0 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:16.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.356 7f4180562700 1 -- 172.21.2.101:0/2501842483 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4178105ae0 con 0x7f41780961f0 2021-02-10T11:17:16.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.359 7f416affd700 1 -- 172.21.2.101:0/2501842483 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4174009340 con 0x7f41780961f0 2021-02-10T11:17:16.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.486 7f4180562700 1 -- 172.21.2.101:0/2501842483 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839251", "target": ["mgr", ""]}) v1 -- 0x7f4178107840 con 0x7f4160018730 2021-02-10T11:17:16.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.510 7f416affd700 1 -- 172.21.2.101:0/2501842483 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f416c0009b0 con 0x7f4160018730 2021-02-10T11:17:16.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.518 7f4180562700 1 -- 172.21.2.101:0/2501842483 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4160018730 msgr2=0x7f416001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:16.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.518 7f4180562700 1 --2- 172.21.2.101:0/2501842483 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4160018730 0x7f416001abc0 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:16.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.518 7f4180562700 1 -- 172.21.2.101:0/2501842483 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41780961f0 msgr2=0x7f41780965f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:16.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.518 7f4180562700 1 --2- 172.21.2.101:0/2501842483 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41780961f0 0x7f41780965f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f4174009630 tx=0x7f417400b900).stop 2021-02-10T11:17:16.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.519 7f4180562700 1 -- 172.21.2.101:0/2501842483 shutdown_connections 2021-02-10T11:17:16.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.519 7f4180562700 1 --2- 172.21.2.101:0/2501842483 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4178106890 0x7f4178106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:16.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.519 7f4180562700 1 --2- 172.21.2.101:0/2501842483 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41780961f0 0x7f41780965f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:16.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.519 7f4180562700 1 --2- 172.21.2.101:0/2501842483 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4160018730 0x7f416001abc0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:16.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.519 7f4180562700 1 --2- 172.21.2.101:0/2501842483 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4178105d10 0x7f417809f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:16.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.519 7f4180562700 1 -- 172.21.2.101:0/2501842483 >> 172.21.2.101:0/2501842483 conn(0x7f4178095a70 msgr2=0x7f41780bf050 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:16.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.519 7f4180562700 1 -- 172.21.2.101:0/2501842483 shutdown_connections 2021-02-10T11:17:16.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.519 7f4180562700 1 -- 172.21.2.101:0/2501842483 wait complete. 2021-02-10T11:17:16.532 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839251 2021-02-10T11:17:16.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.939 7f7f19e88700 1 Processor -- start 2021-02-10T11:17:16.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.939 7f7f19e88700 1 -- start start 2021-02-10T11:17:16.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.940 7f7f19e88700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7f141b0870 0x7f7f140f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:16.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.940 7f7f19e88700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7f140efe00 0x7f7f140f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:16.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.940 7f7f19e88700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f141b0c70 con 0x7f7f141b0470 2021-02-10T11:17:16.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.940 7f7f19e88700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f140e5ea0 con 0x7f7f141b0870 2021-02-10T11:17:16.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.940 7f7f19e88700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f140f0b50 con 0x7f7f140efe00 2021-02-10T11:17:16.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.940 7f7f12ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7f141b0870 0x7f7f140f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:16.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.940 7f7f12ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7f141b0870 0x7f7f140f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46046/0 (socket says 172.21.2.101:46046) 2021-02-10T11:17:16.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.940 7f7f12ffd700 1 -- 172.21.2.101:0/356936332 learned_addr learned my addr 172.21.2.101:0/356936332 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:17:16.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.940 7f7f12ffd700 1 -- 172.21.2.101:0/356936332 >> v2:172.21.2.108:3300/0 conn(0x7f7f140efe00 msgr2=0x7f7f140f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:16.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.940 7f7f12ffd700 1 --2- 172.21.2.101:0/356936332 >> v2:172.21.2.108:3300/0 conn(0x7f7f140efe00 0x7f7f140f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:16.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.940 7f7f12ffd700 1 -- 172.21.2.101:0/356936332 >> v1:172.21.2.108:6789/0 conn(0x7f7f141b0470 legacy=0x7f7f140ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:16.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.940 7f7f12ffd700 1 -- 172.21.2.101:0/356936332 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f140f0820 con 0x7f7f141b0870 2021-02-10T11:17:16.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.941 7f7f12ffd700 1 --2- 172.21.2.101:0/356936332 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f141b0870 0x7f7f140f20e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f7efc006700 tx=0x7f7efc009b80).ready entity=mon.0 client_cookie=349e7cf4cfc23718 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:16.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.941 7f7f127fc700 1 -- 172.21.2.101:0/356936332 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7efc00c0c0 con 0x7f7f141b0870 2021-02-10T11:17:16.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.941 7f7f127fc700 1 -- 172.21.2.101:0/356936332 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7efc00c340 con 0x7f7f141b0870 2021-02-10T11:17:16.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.941 7f7f127fc700 1 -- 172.21.2.101:0/356936332 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7efc00c860 con 0x7f7f141b0870 2021-02-10T11:17:16.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.941 7f7f19e88700 1 -- 172.21.2.101:0/356936332 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f141b0870 msgr2=0x7f7f140f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:16.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.941 7f7f19e88700 1 --2- 172.21.2.101:0/356936332 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f141b0870 0x7f7f140f20e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f7efc006700 tx=0x7f7efc009b80).stop 2021-02-10T11:17:16.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.941 7f7f19e88700 1 -- 172.21.2.101:0/356936332 shutdown_connections 2021-02-10T11:17:16.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.941 7f7f19e88700 1 --2- 172.21.2.101:0/356936332 >> v2:172.21.2.108:3300/0 conn(0x7f7f140efe00 0x7f7f140f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:16.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.941 7f7f19e88700 1 --2- 172.21.2.101:0/356936332 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f141b0870 0x7f7f140f20e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:16.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.941 7f7f19e88700 1 -- 172.21.2.101:0/356936332 >> 172.21.2.101:0/356936332 conn(0x7f7f140ce3e0 msgr2=0x7f7f140e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:16.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.942 7f7f19e88700 1 -- 172.21.2.101:0/356936332 shutdown_connections 2021-02-10T11:17:16.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.942 7f7f19e88700 1 -- 172.21.2.101:0/356936332 wait complete. 2021-02-10T11:17:16.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.956 7f7f19e88700 1 Processor -- start 2021-02-10T11:17:16.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.956 7f7f19e88700 1 -- start start 2021-02-10T11:17:16.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.956 7f7f19e88700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7f140b2840 0x7f7f141a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:16.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.956 7f7f19e88700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7f140b30d0 0x7f7f140b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:16.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.956 7f7f19e88700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f140b3af0 0x7f7f140f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:16.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.957 7f7f19e88700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f140b2c40 con 0x7f7f140b3af0 2021-02-10T11:17:16.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.957 7f7f19e88700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f141a4920 con 0x7f7f140b2840 2021-02-10T11:17:16.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.957 7f7f19e88700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f140f0ba0 con 0x7f7f140b30d0 2021-02-10T11:17:16.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.957 7f7f13fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f140b3af0 0x7f7f140f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:16.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.957 7f7f13fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f140b3af0 0x7f7f140f0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46054/0 (socket says 172.21.2.101:46054) 2021-02-10T11:17:16.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.957 7f7f13fff700 1 -- 172.21.2.101:0/2933348284 learned_addr learned my addr 172.21.2.101:0/2933348284 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:17:16.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.957 7f7f13fff700 1 -- 172.21.2.101:0/2933348284 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7f140b30d0 msgr2=0x7f7f140b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:16.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.957 7f7f13fff700 1 --2- 172.21.2.101:0/2933348284 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7f140b30d0 0x7f7f140b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:16.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.957 7f7f13fff700 1 -- 172.21.2.101:0/2933348284 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7f140b2840 msgr2=0x7f7f141a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:16.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.957 7f7f13fff700 1 --2- 172.21.2.101:0/2933348284 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7f140b2840 0x7f7f141a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:16.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.957 7f7f13fff700 1 -- 172.21.2.101:0/2933348284 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f140f0dc0 con 0x7f7f140b3af0 2021-02-10T11:17:16.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.957 7f7f13fff700 1 --2- 172.21.2.101:0/2933348284 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f140b3af0 0x7f7f140f0580 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f7f0c009980 tx=0x7f7f0c00bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:16.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.957 7f7f0bfff700 1 -- 172.21.2.101:0/2933348284 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7f0c003020 con 0x7f7f140b3af0 2021-02-10T11:17:16.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.957 7f7f0bfff700 1 -- 172.21.2.101:0/2933348284 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0c005a80 con 0x7f7f140b3af0 2021-02-10T11:17:16.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.957 7f7f19e88700 1 -- 172.21.2.101:0/2933348284 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f140f0dc0 con 0x7f7f140b3af0 2021-02-10T11:17:16.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.957 7f7f19e88700 1 -- 172.21.2.101:0/2933348284 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f140f1350 con 0x7f7f140b3af0 2021-02-10T11:17:16.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.957 7f7f0bfff700 1 -- 172.21.2.101:0/2933348284 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7f0c006030 con 0x7f7f140b3af0 2021-02-10T11:17:16.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.958 7f7f0bfff700 1 -- 172.21.2.101:0/2933348284 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7f0c005c60 con 0x7f7f140b3af0 2021-02-10T11:17:16.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.958 7f7f0bfff700 1 --2- 172.21.2.101:0/2933348284 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ef80186c0 0x7f7ef801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:16.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.958 7f7f0bfff700 1 -- 172.21.2.101:0/2933348284 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7f0c032910 con 0x7f7f140b3af0 2021-02-10T11:17:16.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.958 7f7f137fe700 1 --2- 172.21.2.101:0/2933348284 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ef80186c0 0x7f7ef801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:16.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.958 7f7f137fe700 1 --2- 172.21.2.101:0/2933348284 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ef80186c0 0x7f7ef801abc0 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:16.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.990 7f7f19e88700 1 -- 172.21.2.101:0/2933348284 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f140f18f0 con 0x7f7f140b3af0 2021-02-10T11:17:16.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:16.992 7f7f0bfff700 1 -- 172.21.2.101:0/2933348284 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7f0c006230 con 0x7f7f140b3af0 2021-02-10T11:17:17.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:17.119 7f7f19e88700 1 -- 172.21.2.101:0/2933348284 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839251", "target": ["mgr", ""]}) v1 -- 0x7f7f140f1350 con 0x7f7ef80186c0 2021-02-10T11:17:17.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:17.122 7f7f0bfff700 1 -- 172.21.2.101:0/2933348284 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f7f04006e40 con 0x7f7ef80186c0 2021-02-10T11:17:17.132 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839251/bc20b967-116d-4403-a5fc-84c0e02c7571 2021-02-10T11:17:17.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:17.128 7f7f19e88700 1 -- 172.21.2.101:0/2933348284 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ef80186c0 msgr2=0x7f7ef801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:17.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:17.128 7f7f19e88700 1 --2- 172.21.2.101:0/2933348284 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ef80186c0 0x7f7ef801abc0 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:17.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:17.128 7f7f19e88700 1 -- 172.21.2.101:0/2933348284 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f140b3af0 msgr2=0x7f7f140f0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:17.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:17.128 7f7f19e88700 1 --2- 172.21.2.101:0/2933348284 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f140b3af0 0x7f7f140f0580 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f7f0c009980 tx=0x7f7f0c00bcd0).stop 2021-02-10T11:17:17.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:17.128 7f7f19e88700 1 -- 172.21.2.101:0/2933348284 shutdown_connections 2021-02-10T11:17:17.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:17.128 7f7f19e88700 1 --2- 172.21.2.101:0/2933348284 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ef80186c0 0x7f7ef801abc0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:17.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:17.128 7f7f19e88700 1 --2- 172.21.2.101:0/2933348284 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7f140b30d0 0x7f7f140b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:17.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:17.128 7f7f19e88700 1 --2- 172.21.2.101:0/2933348284 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f140b3af0 0x7f7f140f0580 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:17.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:17.128 7f7f19e88700 1 --2- 172.21.2.101:0/2933348284 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7f140b2840 0x7f7f141a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:17.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:17.128 7f7f19e88700 1 -- 172.21.2.101:0/2933348284 >> 172.21.2.101:0/2933348284 conn(0x7f7f140c36f0 msgr2=0x7f7f14094c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:17.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:17.128 7f7f19e88700 1 -- 172.21.2.101:0/2933348284 shutdown_connections 2021-02-10T11:17:17.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:17.128 7f7f19e88700 1 -- 172.21.2.101:0/2933348284 wait complete. 2021-02-10T11:17:17.141 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839251 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839251/bc20b967-116d-4403-a5fc-84c0e02c7571 2021-02-10T11:17:17.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:17.142 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:17.142 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:17.477 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:17.477 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:17.478 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839251/bc20b967-116d-4403-a5fc-84c0e02c7571/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:17.857 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:17.857 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:17.857 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247662 s, 42.3 MB/s 2021-02-10T11:17:17.858 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:17.858 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:17.858 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:18.194 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:18.195 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:18.195 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839251/bc20b967-116d-4403-a5fc-84c0e02c7571/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:18.570 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:18.570 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:18.570 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245527 s, 42.7 MB/s 2021-02-10T11:17:18.571 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:18.571 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:18.571 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:18.907 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:18.907 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:18.908 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839251/bc20b967-116d-4403-a5fc-84c0e02c7571/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:19.280 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:19.281 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:19.281 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246096 s, 42.6 MB/s 2021-02-10T11:17:19.282 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:19.282 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:19.282 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:19.617 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:19.617 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:19.618 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839251/bc20b967-116d-4403-a5fc-84c0e02c7571/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:19.992 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:19.992 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:19.992 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0228035 s, 46.0 MB/s 2021-02-10T11:17:19.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:19.993 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:19.993 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:20.327 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:20.327 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:20.328 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839251/bc20b967-116d-4403-a5fc-84c0e02c7571/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:20.702 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:20.702 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:20.702 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242635 s, 43.2 MB/s 2021-02-10T11:17:20.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:20.703 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:20.703 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:21.039 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:21.039 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:21.040 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839251/bc20b967-116d-4403-a5fc-84c0e02c7571/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:21.413 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:21.414 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:21.414 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240499 s, 43.6 MB/s 2021-02-10T11:17:21.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:21.415 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:21.415 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:21.750 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:21.750 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:21.751 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839251/bc20b967-116d-4403-a5fc-84c0e02c7571/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:22.124 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:22.135 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:22.135 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238044 s, 44.0 MB/s 2021-02-10T11:17:22.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:22.136 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:22.136 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:22.470 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:22.471 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:22.471 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839251/bc20b967-116d-4403-a5fc-84c0e02c7571/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:22.843 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:22.843 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:22.843 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0230489 s, 45.5 MB/s 2021-02-10T11:17:22.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:22.844 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:22.845 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:23.180 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:23.180 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:23.180 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839251/bc20b967-116d-4403-a5fc-84c0e02c7571/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:23.555 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:23.555 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:23.555 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023584 s, 44.5 MB/s 2021-02-10T11:17:23.556 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:23.556 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:23.556 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:23.890 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:23.968 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:23.969 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839251/bc20b967-116d-4403-a5fc-84c0e02c7571/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:24.339 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:24.340 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:24.340 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237748 s, 44.1 MB/s 2021-02-10T11:17:24.341 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839252 2021-02-10T11:17:24.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.748 7f7574b12700 1 Processor -- start 2021-02-10T11:17:24.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.748 7f7574b12700 1 -- start start 2021-02-10T11:17:24.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.749 7f7574b12700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f75701b0870 0x7f75700f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:24.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.749 7f7574b12700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75700efe00 0x7f75700f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:24.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.749 7f7574b12700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75701b0c70 con 0x7f75701b0470 2021-02-10T11:17:24.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.749 7f7574b12700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75700f0b50 con 0x7f75701b0870 2021-02-10T11:17:24.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.749 7f7574b12700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75700f0820 con 0x7f75700efe00 2021-02-10T11:17:24.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.749 7f756e85e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f75701b0470 0x7f75700ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46554/0 (socket says 172.21.2.101:46554) 2021-02-10T11:17:24.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.749 7f756e85e700 1 -- 172.21.2.101:0/3529819400 learned_addr learned my addr 172.21.2.101:0/3529819400 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:17:24.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.749 7f756d85c700 1 -- 172.21.2.101:0/3529819400 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055683631 0 0) 0x7f7558001180 con 0x7f75701b0470 2021-02-10T11:17:24.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.749 7f756d85c700 1 -- 172.21.2.101:0/3529819400 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f755c002fe0 con 0x7f75701b0470 2021-02-10T11:17:24.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.749 7f756e05d700 1 --2- 172.21.2.101:0/3529819400 >> v2:172.21.2.108:3300/0 conn(0x7f75701b0870 0x7f75700f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:24.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.749 7f756f05f700 1 --2- 172.21.2.101:0/3529819400 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75700efe00 0x7f75700f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:24.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.749 7f756d85c700 1 -- 172.21.2.101:0/3529819400 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2300861235 0 0) 0x7f7558001180 con 0x7f75701b0470 2021-02-10T11:17:24.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.749 7f756d85c700 1 -- 172.21.2.101:0/3529819400 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75700efe00 msgr2=0x7f75700f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:24.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.749 7f756d85c700 1 --2- 172.21.2.101:0/3529819400 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75700efe00 0x7f75700f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:24.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.749 7f756d85c700 1 -- 172.21.2.101:0/3529819400 >> v2:172.21.2.108:3300/0 conn(0x7f75701b0870 msgr2=0x7f75700f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:24.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.749 7f756d85c700 1 --2- 172.21.2.101:0/3529819400 >> v2:172.21.2.108:3300/0 conn(0x7f75701b0870 0x7f75700f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:24.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.749 7f756d85c700 1 -- 172.21.2.101:0/3529819400 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75700e52e0 con 0x7f75701b0470 2021-02-10T11:17:24.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.749 7f756d85c700 1 -- 172.21.2.101:0/3529819400 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f7558001680 con 0x7f75701b0470 2021-02-10T11:17:24.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.750 7f756d85c700 1 -- 172.21.2.101:0/3529819400 >> v1:172.21.2.101:6789/0 conn(0x7f75701b0470 legacy=0x7f75700ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:24.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.750 7f756d85c700 1 --2- 172.21.2.101:0/3529819400 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f755c003b90 0x7f755c006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:24.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.750 7f756d85c700 1 --2- 172.21.2.101:0/3529819400 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f755c0068e0 0x7f755c008de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:24.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.750 7f756d85c700 1 --2- 172.21.2.101:0/3529819400 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f755c0096e0 0x7f755c00bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:24.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.750 7f756d85c700 1 -- 172.21.2.101:0/3529819400 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c050430 con 0x7f755c003b90 2021-02-10T11:17:24.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.750 7f756d85c700 1 -- 172.21.2.101:0/3529819400 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c050610 con 0x7f755c0068e0 2021-02-10T11:17:24.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.750 7f756d85c700 1 -- 172.21.2.101:0/3529819400 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f755c050830 con 0x7f755c0096e0 2021-02-10T11:17:24.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.750 7f756f05f700 1 --2- 172.21.2.101:0/3529819400 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75700efe00 0x7f75700f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:17:24.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.750 7f756e05d700 1 --2- 172.21.2.101:0/3529819400 >> v2:172.21.2.108:3300/0 conn(0x7f75701b0870 0x7f75700f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:17:24.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.750 7f756e85e700 1 --2- 172.21.2.101:0/3529819400 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f755c003b90 0x7f755c006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:24.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.750 7f756e85e700 1 -- 172.21.2.101:0/3529819400 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f755c0096e0 msgr2=0x7f755c00bbd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:17:24.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.750 7f756e85e700 1 --2- 172.21.2.101:0/3529819400 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f755c0096e0 0x7f755c00bbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:24.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.750 7f756e85e700 1 -- 172.21.2.101:0/3529819400 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f755c0068e0 msgr2=0x7f755c008de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:17:24.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.750 7f756e85e700 1 --2- 172.21.2.101:0/3529819400 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f755c0068e0 0x7f755c008de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:24.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.750 7f756e85e700 1 -- 172.21.2.101:0/3529819400 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c050ad0 con 0x7f755c003b90 2021-02-10T11:17:24.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.750 7f756e85e700 1 --2- 172.21.2.101:0/3529819400 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f755c003b90 0x7f755c006040 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f7558000e80 tx=0x7f7558007c80).ready entity=mon.0 client_cookie=d11ce54a3bb2ce06 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:24.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.750 7f756d85c700 1 -- 172.21.2.101:0/3529819400 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f75580070c0 con 0x7f755c003b90 2021-02-10T11:17:24.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.750 7f756d85c700 1 -- 172.21.2.101:0/3529819400 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755800a520 con 0x7f755c003b90 2021-02-10T11:17:24.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.750 7f756d85c700 1 -- 172.21.2.101:0/3529819400 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f755800a9b0 con 0x7f755c003b90 2021-02-10T11:17:24.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.750 7f7574b12700 1 -- 172.21.2.101:0/3529819400 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f755c003b90 msgr2=0x7f755c006040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:24.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.750 7f7574b12700 1 --2- 172.21.2.101:0/3529819400 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f755c003b90 0x7f755c006040 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f7558000e80 tx=0x7f7558007c80).stop 2021-02-10T11:17:24.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.750 7f756f05f700 1 -- 172.21.2.101:0/3529819400 reap_dead start 2021-02-10T11:17:24.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.751 7f7574b12700 1 -- 172.21.2.101:0/3529819400 shutdown_connections 2021-02-10T11:17:24.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.751 7f7574b12700 1 -- 172.21.2.101:0/3529819400 >> 172.21.2.101:0/3529819400 conn(0x7f75700ce3e0 msgr2=0x7f75700e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:24.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.751 7f7574b12700 1 -- 172.21.2.101:0/3529819400 shutdown_connections 2021-02-10T11:17:24.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.751 7f7574b12700 1 -- 172.21.2.101:0/3529819400 wait complete. 2021-02-10T11:17:24.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.766 7f7574b12700 1 Processor -- start 2021-02-10T11:17:24.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.766 7f7574b12700 1 -- start start 2021-02-10T11:17:24.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.766 7f7574b12700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75700a6e70 0x7f75700965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:24.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.766 7f7574b12700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7570096c10 0x7f7570097010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:24.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.766 7f7574b12700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75700c0ef0 0x7f75700c12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:24.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.766 7f7574b12700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75700a7270 con 0x7f75700c0ef0 2021-02-10T11:17:24.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.766 7f7574b12700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7570190c50 con 0x7f7570096c10 2021-02-10T11:17:24.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.766 7f7574b12700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75700c1910 con 0x7f75700a6e70 2021-02-10T11:17:24.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.767 7f756f05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75700c0ef0 0x7f75700c12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:24.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.767 7f756f05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75700c0ef0 0x7f75700c12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46070/0 (socket says 172.21.2.101:46070) 2021-02-10T11:17:24.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.767 7f756f05f700 1 -- 172.21.2.101:0/1669893835 learned_addr learned my addr 172.21.2.101:0/1669893835 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:17:24.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.767 7f756e05d700 1 --2- 172.21.2.101:0/1669893835 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7570096c10 0x7f7570097010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:24.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.767 7f756f05f700 1 -- 172.21.2.101:0/1669893835 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75700a6e70 msgr2=0x7f75700965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:24.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.767 7f756f05f700 1 --2- 172.21.2.101:0/1669893835 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75700a6e70 0x7f75700965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:24.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.767 7f756f05f700 1 -- 172.21.2.101:0/1669893835 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7570096c10 msgr2=0x7f7570097010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:24.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.767 7f756f05f700 1 --2- 172.21.2.101:0/1669893835 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7570096c10 0x7f7570097010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:24.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.767 7f756f05f700 1 -- 172.21.2.101:0/1669893835 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75700c1b30 con 0x7f75700c0ef0 2021-02-10T11:17:24.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.767 7f756e05d700 1 --2- 172.21.2.101:0/1669893835 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7570096c10 0x7f7570097010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:17:24.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.767 7f756f05f700 1 --2- 172.21.2.101:0/1669893835 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75700c0ef0 0x7f75700c12f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f75680099a0 tx=0x7f756800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:24.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.767 7f75677fe700 1 -- 172.21.2.101:0/1669893835 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f75680066f0 con 0x7f75700c0ef0 2021-02-10T11:17:24.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.767 7f75677fe700 1 -- 172.21.2.101:0/1669893835 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7568003030 con 0x7f75700c0ef0 2021-02-10T11:17:24.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.767 7f75677fe700 1 -- 172.21.2.101:0/1669893835 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f75680013e0 con 0x7f75700c0ef0 2021-02-10T11:17:24.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.767 7f7574b12700 1 -- 172.21.2.101:0/1669893835 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75700c1b30 con 0x7f75700c0ef0 2021-02-10T11:17:24.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.767 7f7574b12700 1 -- 172.21.2.101:0/1669893835 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75700c1b30 con 0x7f75700c0ef0 2021-02-10T11:17:24.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.768 7f75677fe700 1 -- 172.21.2.101:0/1669893835 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7568000df0 con 0x7f75700c0ef0 2021-02-10T11:17:24.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.768 7f75677fe700 1 --2- 172.21.2.101:0/1669893835 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75540186c0 0x7f755401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:24.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.768 7f75677fe700 1 -- 172.21.2.101:0/1669893835 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7568032900 con 0x7f75700c0ef0 2021-02-10T11:17:24.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.768 7f756e85e700 1 --2- 172.21.2.101:0/1669893835 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75540186c0 0x7f755401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:24.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.769 7f756e85e700 1 --2- 172.21.2.101:0/1669893835 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75540186c0 0x7f755401abc0 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:24.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.798 7f7574b12700 1 -- 172.21.2.101:0/1669893835 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7570105b30 con 0x7f75700c0ef0 2021-02-10T11:17:24.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.801 7f75677fe700 1 -- 172.21.2.101:0/1669893835 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f756800be30 con 0x7f75700c0ef0 2021-02-10T11:17:24.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.928 7f7574b12700 1 -- 172.21.2.101:0/1669893835 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839252", "target": ["mgr", ""]}) v1 -- 0x7f7570105b30 con 0x7f75540186c0 2021-02-10T11:17:24.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.952 7f75677fe700 1 -- 172.21.2.101:0/1669893835 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f7558000c90 con 0x7f75540186c0 2021-02-10T11:17:24.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.961 7f7574b12700 1 -- 172.21.2.101:0/1669893835 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75540186c0 msgr2=0x7f755401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:24.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.961 7f7574b12700 1 --2- 172.21.2.101:0/1669893835 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75540186c0 0x7f755401abc0 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:24.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.961 7f7574b12700 1 -- 172.21.2.101:0/1669893835 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75700c0ef0 msgr2=0x7f75700c12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:24.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.961 7f7574b12700 1 --2- 172.21.2.101:0/1669893835 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75700c0ef0 0x7f75700c12f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f75680099a0 tx=0x7f756800bcf0).stop 2021-02-10T11:17:24.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.961 7f7574b12700 1 -- 172.21.2.101:0/1669893835 shutdown_connections 2021-02-10T11:17:24.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.961 7f7574b12700 1 --2- 172.21.2.101:0/1669893835 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75700c0ef0 0x7f75700c12f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:24.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.961 7f7574b12700 1 --2- 172.21.2.101:0/1669893835 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7570096c10 0x7f7570097010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:24.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.961 7f7574b12700 1 --2- 172.21.2.101:0/1669893835 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75540186c0 0x7f755401abc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:24.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.961 7f7574b12700 1 --2- 172.21.2.101:0/1669893835 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75700a6e70 0x7f75700965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:24.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.961 7f7574b12700 1 -- 172.21.2.101:0/1669893835 >> 172.21.2.101:0/1669893835 conn(0x7f75700c40e0 msgr2=0x7f75700957c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:24.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.961 7f7574b12700 1 -- 172.21.2.101:0/1669893835 shutdown_connections 2021-02-10T11:17:24.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:24.961 7f7574b12700 1 -- 172.21.2.101:0/1669893835 wait complete. 2021-02-10T11:17:24.975 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839252 2021-02-10T11:17:25.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.381 7fba306bf700 1 Processor -- start 2021-02-10T11:17:25.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.381 7fba306bf700 1 -- start start 2021-02-10T11:17:25.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.382 7fba306bf700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fba281b0470 0x7fba281b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:25.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.382 7fba306bf700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba280f0b50 con 0x7fba280f20e0 2021-02-10T11:17:25.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.382 7fba306bf700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba280f24e0 con 0x7fba280ef610 2021-02-10T11:17:25.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.382 7fba306bf700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba280e5ea0 con 0x7fba281b0470 2021-02-10T11:17:25.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.382 7fba2ecdc700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fba280f20e0 0x7fba280f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46570/0 (socket says 172.21.2.101:46570) 2021-02-10T11:17:25.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.382 7fba2ecdc700 1 -- 172.21.2.101:0/4145275758 learned_addr learned my addr 172.21.2.101:0/4145275758 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:17:25.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.382 7fba2e4db700 1 --2- 172.21.2.101:0/4145275758 >> v2:172.21.2.108:3300/0 conn(0x7fba281b0470 0x7fba281b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:25.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.383 7fba2d4d9700 1 -- 172.21.2.101:0/4145275758 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3323351593 0 0) 0x7fba20000d30 con 0x7fba280f20e0 2021-02-10T11:17:25.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.383 7fba2d4d9700 1 -- 172.21.2.101:0/4145275758 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba14002fe0 con 0x7fba280f20e0 2021-02-10T11:17:25.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.383 7fba2e4db700 1 -- 172.21.2.101:0/4145275758 >> v1:172.21.2.108:6789/0 conn(0x7fba280ef610 legacy=0x7fba280efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:25.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.383 7fba2e4db700 1 -- 172.21.2.101:0/4145275758 >> v1:172.21.2.101:6789/0 conn(0x7fba280f20e0 legacy=0x7fba280f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:25.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.383 7fba2d4d9700 1 -- 172.21.2.101:0/4145275758 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 617842953 0 0) 0x7fba20001500 con 0x7fba280f20e0 2021-02-10T11:17:25.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.383 7fba2e4db700 1 -- 172.21.2.101:0/4145275758 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba280e52e0 con 0x7fba281b0470 2021-02-10T11:17:25.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.383 7fba2e4db700 1 --2- 172.21.2.101:0/4145275758 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba281b0470 0x7fba281b0870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fba24006650 tx=0x7fba24009a80).ready entity=mon.1 client_cookie=d0e650af3de08800 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:25.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.383 7fba2d4d9700 1 -- 172.21.2.101:0/4145275758 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fba240009b0 con 0x7fba281b0470 2021-02-10T11:17:25.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.383 7fba2d4d9700 1 -- 172.21.2.101:0/4145275758 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2400c0c0 con 0x7fba281b0470 2021-02-10T11:17:25.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.383 7fba2d4d9700 1 -- 172.21.2.101:0/4145275758 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fba2400c520 con 0x7fba281b0470 2021-02-10T11:17:25.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.384 7fba306bf700 1 -- 172.21.2.101:0/4145275758 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba281b0470 msgr2=0x7fba281b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:25.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.384 7fba306bf700 1 --2- 172.21.2.101:0/4145275758 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba281b0470 0x7fba281b0870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fba24006650 tx=0x7fba24009a80).stop 2021-02-10T11:17:25.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.384 7fba306bf700 1 -- 172.21.2.101:0/4145275758 shutdown_connections 2021-02-10T11:17:25.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.384 7fba306bf700 1 --2- 172.21.2.101:0/4145275758 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba281b0470 0x7fba281b0870 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:25.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.384 7fba306bf700 1 -- 172.21.2.101:0/4145275758 >> 172.21.2.101:0/4145275758 conn(0x7fba280ce3e0 msgr2=0x7fba280e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:25.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.384 7fba306bf700 1 -- 172.21.2.101:0/4145275758 shutdown_connections 2021-02-10T11:17:25.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.384 7fba306bf700 1 -- 172.21.2.101:0/4145275758 wait complete. 2021-02-10T11:17:25.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.399 7fba306bf700 1 Processor -- start 2021-02-10T11:17:25.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.399 7fba306bf700 1 -- start start 2021-02-10T11:17:25.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.399 7fba306bf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba28193230 0x7fba280a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:25.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.399 7fba306bf700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba2808f5b0 0x7fba2808f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:25.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.399 7fba306bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba2808e6a0 0x7fba2808eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:25.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.399 7fba306bf700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2808ffd0 con 0x7fba2808e6a0 2021-02-10T11:17:25.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.399 7fba306bf700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba28090240 con 0x7fba2808f5b0 2021-02-10T11:17:25.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.399 7fba306bf700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2808f0c0 con 0x7fba28193230 2021-02-10T11:17:25.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.399 7fba2ecdc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba2808e6a0 0x7fba2808eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:25.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.399 7fba2ecdc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba2808e6a0 0x7fba2808eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46080/0 (socket says 172.21.2.101:46080) 2021-02-10T11:17:25.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.399 7fba2ecdc700 1 -- 172.21.2.101:0/3783793481 learned_addr learned my addr 172.21.2.101:0/3783793481 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:17:25.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.399 7fba2ecdc700 1 -- 172.21.2.101:0/3783793481 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba28193230 msgr2=0x7fba280a6a80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:17:25.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.399 7fba2ecdc700 1 --2- 172.21.2.101:0/3783793481 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba28193230 0x7fba280a6a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:25.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.399 7fba2ecdc700 1 -- 172.21.2.101:0/3783793481 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba2808f5b0 msgr2=0x7fba2808f9b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:17:25.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.399 7fba2ecdc700 1 --2- 172.21.2.101:0/3783793481 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba2808f5b0 0x7fba2808f9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:25.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.399 7fba2ecdc700 1 -- 172.21.2.101:0/3783793481 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2808f2a0 con 0x7fba2808e6a0 2021-02-10T11:17:25.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.400 7fba2ecdc700 1 --2- 172.21.2.101:0/3783793481 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba2808e6a0 0x7fba2808eaa0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fba2000a8f0 tx=0x7fba2000ac90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:25.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.400 7fba1effd700 1 -- 172.21.2.101:0/3783793481 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fba20000d30 con 0x7fba2808e6a0 2021-02-10T11:17:25.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.400 7fba1effd700 1 -- 172.21.2.101:0/3783793481 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba20000f70 con 0x7fba2808e6a0 2021-02-10T11:17:25.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.400 7fba306bf700 1 -- 172.21.2.101:0/3783793481 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2808f2a0 con 0x7fba2808e6a0 2021-02-10T11:17:25.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.400 7fba306bf700 1 -- 172.21.2.101:0/3783793481 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2808f2a0 con 0x7fba2808e6a0 2021-02-10T11:17:25.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.400 7fba1effd700 1 -- 172.21.2.101:0/3783793481 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fba20004b60 con 0x7fba2808e6a0 2021-02-10T11:17:25.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.401 7fba1effd700 1 -- 172.21.2.101:0/3783793481 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fba2000add0 con 0x7fba2808e6a0 2021-02-10T11:17:25.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.401 7fba1effd700 1 --2- 172.21.2.101:0/3783793481 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba10018730 0x7fba1001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:25.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.401 7fba1effd700 1 -- 172.21.2.101:0/3783793481 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fba20032b50 con 0x7fba2808e6a0 2021-02-10T11:17:25.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.401 7fba2e4db700 1 --2- 172.21.2.101:0/3783793481 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba10018730 0x7fba1001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:25.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.401 7fba2e4db700 1 --2- 172.21.2.101:0/3783793481 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba10018730 0x7fba1001ac10 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:25.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.431 7fba306bf700 1 -- 172.21.2.101:0/3783793481 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba280f0b50 con 0x7fba2808e6a0 2021-02-10T11:17:25.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.434 7fba1effd700 1 -- 172.21.2.101:0/3783793481 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fba20004720 con 0x7fba2808e6a0 2021-02-10T11:17:25.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.563 7fba306bf700 1 -- 172.21.2.101:0/3783793481 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839252", "target": ["mgr", ""]}) v1 -- 0x7fba280f0b50 con 0x7fba10018730 2021-02-10T11:17:25.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.567 7fba1effd700 1 -- 172.21.2.101:0/3783793481 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fba24006390 con 0x7fba10018730 2021-02-10T11:17:25.575 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839252/a2163b5f-5671-4172-8eb7-d245cfc0e0e6 2021-02-10T11:17:25.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.572 7fba306bf700 1 -- 172.21.2.101:0/3783793481 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba10018730 msgr2=0x7fba1001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:25.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.572 7fba306bf700 1 --2- 172.21.2.101:0/3783793481 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba10018730 0x7fba1001ac10 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:25.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.572 7fba306bf700 1 -- 172.21.2.101:0/3783793481 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba2808e6a0 msgr2=0x7fba2808eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:25.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.572 7fba306bf700 1 --2- 172.21.2.101:0/3783793481 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba2808e6a0 0x7fba2808eaa0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fba2000a8f0 tx=0x7fba2000ac90).stop 2021-02-10T11:17:25.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.572 7fba306bf700 1 -- 172.21.2.101:0/3783793481 shutdown_connections 2021-02-10T11:17:25.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.572 7fba306bf700 1 --2- 172.21.2.101:0/3783793481 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba2808e6a0 0x7fba2808eaa0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:25.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.572 7fba306bf700 1 --2- 172.21.2.101:0/3783793481 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba2808f5b0 0x7fba2808f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:25.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.572 7fba306bf700 1 --2- 172.21.2.101:0/3783793481 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba10018730 0x7fba1001ac10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:25.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.572 7fba306bf700 1 --2- 172.21.2.101:0/3783793481 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba28193230 0x7fba280a6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:25.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.572 7fba306bf700 1 -- 172.21.2.101:0/3783793481 >> 172.21.2.101:0/3783793481 conn(0x7fba280bf4e0 msgr2=0x7fba280be160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:25.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.572 7fba306bf700 1 -- 172.21.2.101:0/3783793481 shutdown_connections 2021-02-10T11:17:25.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:25.572 7fba306bf700 1 -- 172.21.2.101:0/3783793481 wait complete. 2021-02-10T11:17:25.585 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839252 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839252/a2163b5f-5671-4172-8eb7-d245cfc0e0e6 2021-02-10T11:17:25.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:25.586 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:25.586 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:25.918 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:25.919 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:25.919 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839252/a2163b5f-5671-4172-8eb7-d245cfc0e0e6/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:26.298 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:26.299 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:26.299 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236834 s, 44.3 MB/s 2021-02-10T11:17:26.300 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:26.300 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:26.300 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:26.634 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:26.635 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:26.635 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839252/a2163b5f-5671-4172-8eb7-d245cfc0e0e6/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:27.014 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:27.014 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:27.015 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248442 s, 42.2 MB/s 2021-02-10T11:17:27.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:27.016 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:27.016 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:27.351 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:27.351 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:27.351 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839252/a2163b5f-5671-4172-8eb7-d245cfc0e0e6/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:27.728 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:27.728 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:27.728 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249094 s, 42.1 MB/s 2021-02-10T11:17:27.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:27.729 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:27.729 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:28.065 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:28.065 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:28.066 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839252/a2163b5f-5671-4172-8eb7-d245cfc0e0e6/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:28.442 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:28.443 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:28.443 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0256587 s, 40.9 MB/s 2021-02-10T11:17:28.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:28.444 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:28.444 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:28.781 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:28.781 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:28.782 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839252/a2163b5f-5671-4172-8eb7-d245cfc0e0e6/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:29.156 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:29.156 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:29.157 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.025399 s, 41.3 MB/s 2021-02-10T11:17:29.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:29.158 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:29.158 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:29.492 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:29.493 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:29.493 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839252/a2163b5f-5671-4172-8eb7-d245cfc0e0e6/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:29.869 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:29.869 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:29.869 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244335 s, 42.9 MB/s 2021-02-10T11:17:29.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:29.871 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:29.871 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:30.206 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:30.206 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:30.206 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839252/a2163b5f-5671-4172-8eb7-d245cfc0e0e6/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:30.581 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:30.581 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:30.581 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237748 s, 44.1 MB/s 2021-02-10T11:17:30.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:30.583 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:30.583 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:30.917 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:30.917 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:30.917 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839252/a2163b5f-5671-4172-8eb7-d245cfc0e0e6/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:31.291 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:31.291 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:31.291 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0232263 s, 45.1 MB/s 2021-02-10T11:17:31.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:31.292 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:31.292 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:31.627 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:31.628 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:31.628 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839252/a2163b5f-5671-4172-8eb7-d245cfc0e0e6/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:32.003 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:32.003 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:32.003 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242202 s, 43.3 MB/s 2021-02-10T11:17:32.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:32.004 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:32.004 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:32.339 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:32.339 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:32.340 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839252/a2163b5f-5671-4172-8eb7-d245cfc0e0e6/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:32.712 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:32.712 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:32.712 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233733 s, 44.9 MB/s 2021-02-10T11:17:32.713 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839253 2021-02-10T11:17:33.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.121 7f4bcbb3d700 1 Processor -- start 2021-02-10T11:17:33.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.121 7f4bcbb3d700 1 -- start start 2021-02-10T11:17:33.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.122 7f4bcbb3d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bc41b0870 0x7f4bc40f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:33.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.122 7f4bcbb3d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc41b0c70 con 0x7f4bc41b0470 2021-02-10T11:17:33.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.122 7f4bcbb3d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc40f0b50 con 0x7f4bc40efe00 2021-02-10T11:17:33.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.122 7f4bcbb3d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc40e5ea0 con 0x7f4bc41b0870 2021-02-10T11:17:33.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.122 7f4bc9959700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f4bc41b0470 0x7f4bc40ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46582/0 (socket says 172.21.2.101:46582) 2021-02-10T11:17:33.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.122 7f4bc9959700 1 -- 172.21.2.101:0/4253362305 learned_addr learned my addr 172.21.2.101:0/4253362305 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:17:33.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.122 7f4bc8957700 1 -- 172.21.2.101:0/4253362305 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3015484764 0 0) 0x7f4bb80010b0 con 0x7f4bc41b0470 2021-02-10T11:17:33.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bc8957700 1 -- 172.21.2.101:0/4253362305 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bb0002fe0 con 0x7f4bc41b0470 2021-02-10T11:17:33.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bc9158700 1 --2- 172.21.2.101:0/4253362305 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bc41b0870 0x7f4bc40f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:33.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bc8957700 1 -- 172.21.2.101:0/4253362305 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 999132097 0 0) 0x7f4bb80010b0 con 0x7f4bc41b0470 2021-02-10T11:17:33.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bc8957700 1 -- 172.21.2.101:0/4253362305 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bc41b0870 msgr2=0x7f4bc40f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:33.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bc8957700 1 --2- 172.21.2.101:0/4253362305 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bc41b0870 0x7f4bc40f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:33.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bc8957700 1 -- 172.21.2.101:0/4253362305 >> v1:172.21.2.108:6789/0 conn(0x7f4bc40efe00 legacy=0x7f4bc40f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:33.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bc8957700 1 -- 172.21.2.101:0/4253362305 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc40e52e0 con 0x7f4bc41b0470 2021-02-10T11:17:33.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bc8957700 1 -- 172.21.2.101:0/4253362305 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f4bb80015b0 con 0x7f4bc41b0470 2021-02-10T11:17:33.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bc8957700 1 -- 172.21.2.101:0/4253362305 >> v1:172.21.2.101:6789/0 conn(0x7f4bc41b0470 legacy=0x7f4bc40ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:33.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bc8957700 1 --2- 172.21.2.101:0/4253362305 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bb0003af0 0x7f4bb0005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:33.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bc8957700 1 --2- 172.21.2.101:0/4253362305 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bb0006840 0x7f4bb0008d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:33.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bc8957700 1 --2- 172.21.2.101:0/4253362305 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4bb0009530 0x7f4bb000ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:33.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bc8957700 1 -- 172.21.2.101:0/4253362305 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb000c300 con 0x7f4bb0003af0 2021-02-10T11:17:33.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bc8957700 1 -- 172.21.2.101:0/4253362305 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb000c520 con 0x7f4bb0009530 2021-02-10T11:17:33.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bc8957700 1 -- 172.21.2.101:0/4253362305 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb000c740 con 0x7f4bb0006840 2021-02-10T11:17:33.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bc9959700 1 --2- 172.21.2.101:0/4253362305 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bb0003af0 0x7f4bb0005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:33.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bc9158700 1 --2- 172.21.2.101:0/4253362305 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bb0006840 0x7f4bb0008d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:33.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bca15a700 1 --2- 172.21.2.101:0/4253362305 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4bb0009530 0x7f4bb000ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:33.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bc9959700 1 -- 172.21.2.101:0/4253362305 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bb0006840 msgr2=0x7f4bb0008d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:33.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bc9959700 1 --2- 172.21.2.101:0/4253362305 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bb0006840 0x7f4bb0008d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:33.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bc9959700 1 -- 172.21.2.101:0/4253362305 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4bb0009530 msgr2=0x7f4bb000ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:33.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bc9959700 1 --2- 172.21.2.101:0/4253362305 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4bb0009530 0x7f4bb000ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:33.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.123 7f4bc9959700 1 -- 172.21.2.101:0/4253362305 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb000c9e0 con 0x7f4bb0003af0 2021-02-10T11:17:33.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.124 7f4bc9959700 1 --2- 172.21.2.101:0/4253362305 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bb0003af0 0x7f4bb0005fa0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f4bb804b590 tx=0x7f4bb804bd10).ready entity=mon.0 client_cookie=9108d298f0538b8a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:33.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.124 7f4bc8957700 1 -- 172.21.2.101:0/4253362305 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4bb8004410 con 0x7f4bb0003af0 2021-02-10T11:17:33.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.124 7f4bc8957700 1 -- 172.21.2.101:0/4253362305 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb8001850 con 0x7f4bb0003af0 2021-02-10T11:17:33.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.124 7f4bc8957700 1 -- 172.21.2.101:0/4253362305 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4bb8004a10 con 0x7f4bb0003af0 2021-02-10T11:17:33.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.124 7f4bcbb3d700 1 -- 172.21.2.101:0/4253362305 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bb0003af0 msgr2=0x7f4bb0005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:33.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.124 7f4bcbb3d700 1 --2- 172.21.2.101:0/4253362305 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bb0003af0 0x7f4bb0005fa0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f4bb804b590 tx=0x7f4bb804bd10).stop 2021-02-10T11:17:33.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.124 7f4bca15a700 1 -- 172.21.2.101:0/4253362305 reap_dead start 2021-02-10T11:17:33.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.124 7f4bcbb3d700 1 -- 172.21.2.101:0/4253362305 shutdown_connections 2021-02-10T11:17:33.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.124 7f4bcbb3d700 1 -- 172.21.2.101:0/4253362305 >> 172.21.2.101:0/4253362305 conn(0x7f4bc40ce3e0 msgr2=0x7f4bc40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:33.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.124 7f4bcbb3d700 1 -- 172.21.2.101:0/4253362305 shutdown_connections 2021-02-10T11:17:33.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.125 7f4bcbb3d700 1 -- 172.21.2.101:0/4253362305 wait complete. 2021-02-10T11:17:33.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.139 7f4bcbb3d700 1 Processor -- start 2021-02-10T11:17:33.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.139 7f4bcbb3d700 1 -- start start 2021-02-10T11:17:33.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.140 7f4bcbb3d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bc409b930 0x7f4bc40b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:33.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.140 7f4bcbb3d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4bc40b8410 0x7f4bc40b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:33.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.140 7f4bcbb3d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bc40c1f80 0x7f4bc40c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:33.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.140 7f4bcbb3d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc40c1a00 con 0x7f4bc409b930 2021-02-10T11:17:33.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.140 7f4bcbb3d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc40b8e30 con 0x7f4bc40b8410 2021-02-10T11:17:33.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.140 7f4bcbb3d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc40b9050 con 0x7f4bc40c1f80 2021-02-10T11:17:33.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.140 7f4bc9959700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bc409b930 0x7f4bc40b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:33.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.140 7f4bc9959700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bc409b930 0x7f4bc40b7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46098/0 (socket says 172.21.2.101:46098) 2021-02-10T11:17:33.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.140 7f4bc9959700 1 -- 172.21.2.101:0/1320526836 learned_addr learned my addr 172.21.2.101:0/1320526836 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:17:33.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.140 7f4bca15a700 1 --2- 172.21.2.101:0/1320526836 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bc40c1f80 0x7f4bc40c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:33.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.140 7f4bc9158700 1 --2- 172.21.2.101:0/1320526836 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4bc40b8410 0x7f4bc40b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:33.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.140 7f4bc9959700 1 -- 172.21.2.101:0/1320526836 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bc40c1f80 msgr2=0x7f4bc40c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:33.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.140 7f4bc9959700 1 --2- 172.21.2.101:0/1320526836 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bc40c1f80 0x7f4bc40c2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:33.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.140 7f4bc9959700 1 -- 172.21.2.101:0/1320526836 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4bc40b8410 msgr2=0x7f4bc40b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:33.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.140 7f4bc9959700 1 --2- 172.21.2.101:0/1320526836 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4bc40b8410 0x7f4bc40b8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:33.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.140 7f4bc9959700 1 -- 172.21.2.101:0/1320526836 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc40c29a0 con 0x7f4bc409b930 2021-02-10T11:17:33.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.140 7f4bc9158700 1 --2- 172.21.2.101:0/1320526836 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4bc40b8410 0x7f4bc40b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:17:33.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.140 7f4bc9959700 1 --2- 172.21.2.101:0/1320526836 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bc409b930 0x7f4bc40b7c20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f4bb804e590 tx=0x7f4bb804e930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:33.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.141 7f4bb67fc700 1 -- 172.21.2.101:0/1320526836 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4bb804b5f0 con 0x7f4bc409b930 2021-02-10T11:17:33.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.141 7f4bb67fc700 1 -- 172.21.2.101:0/1320526836 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb804ed20 con 0x7f4bc409b930 2021-02-10T11:17:33.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.141 7f4bb67fc700 1 -- 172.21.2.101:0/1320526836 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4bb8050350 con 0x7f4bc409b930 2021-02-10T11:17:33.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.141 7f4bcbb3d700 1 -- 172.21.2.101:0/1320526836 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc40c29a0 con 0x7f4bc409b930 2021-02-10T11:17:33.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.141 7f4bcbb3d700 1 -- 172.21.2.101:0/1320526836 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc40c29a0 con 0x7f4bc409b930 2021-02-10T11:17:33.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.141 7f4bb67fc700 1 -- 172.21.2.101:0/1320526836 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4bb804c030 con 0x7f4bc409b930 2021-02-10T11:17:33.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.141 7f4bb67fc700 1 --2- 172.21.2.101:0/1320526836 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bac0186c0 0x7f4bac01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:33.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.141 7f4bb67fc700 1 -- 172.21.2.101:0/1320526836 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4bb8076550 con 0x7f4bc409b930 2021-02-10T11:17:33.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.142 7f4bc9158700 1 --2- 172.21.2.101:0/1320526836 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bac0186c0 0x7f4bac01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:33.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.142 7f4bc9158700 1 --2- 172.21.2.101:0/1320526836 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bac0186c0 0x7f4bac01abc0 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:33.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.172 7f4bcbb3d700 1 -- 172.21.2.101:0/1320526836 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc4105b30 con 0x7f4bc409b930 2021-02-10T11:17:33.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.174 7f4bb67fc700 1 -- 172.21.2.101:0/1320526836 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4bb804ba90 con 0x7f4bc409b930 2021-02-10T11:17:33.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.301 7f4bcbb3d700 1 -- 172.21.2.101:0/1320526836 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839253", "target": ["mgr", ""]}) v1 -- 0x7f4bc40b8e30 con 0x7f4bac0186c0 2021-02-10T11:17:33.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.329 7f4bb67fc700 1 -- 172.21.2.101:0/1320526836 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f4bc0006980 con 0x7f4bac0186c0 2021-02-10T11:17:33.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.335 7f4bcbb3d700 1 -- 172.21.2.101:0/1320526836 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bac0186c0 msgr2=0x7f4bac01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:33.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.335 7f4bcbb3d700 1 --2- 172.21.2.101:0/1320526836 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bac0186c0 0x7f4bac01abc0 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:33.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.335 7f4bcbb3d700 1 -- 172.21.2.101:0/1320526836 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bc409b930 msgr2=0x7f4bc40b7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:33.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.335 7f4bcbb3d700 1 --2- 172.21.2.101:0/1320526836 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bc409b930 0x7f4bc40b7c20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f4bb804e590 tx=0x7f4bb804e930).stop 2021-02-10T11:17:33.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.335 7f4bcbb3d700 1 -- 172.21.2.101:0/1320526836 shutdown_connections 2021-02-10T11:17:33.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.335 7f4bcbb3d700 1 --2- 172.21.2.101:0/1320526836 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bac0186c0 0x7f4bac01abc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:33.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.335 7f4bcbb3d700 1 --2- 172.21.2.101:0/1320526836 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bc40c1f80 0x7f4bc40c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:33.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.335 7f4bcbb3d700 1 --2- 172.21.2.101:0/1320526836 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4bc40b8410 0x7f4bc40b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:33.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.335 7f4bcbb3d700 1 --2- 172.21.2.101:0/1320526836 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bc409b930 0x7f4bc40b7c20 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:33.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.335 7f4bcbb3d700 1 -- 172.21.2.101:0/1320526836 >> 172.21.2.101:0/1320526836 conn(0x7f4bc4094e10 msgr2=0x7f4bc40a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:33.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.335 7f4bcbb3d700 1 -- 172.21.2.101:0/1320526836 shutdown_connections 2021-02-10T11:17:33.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.335 7f4bcbb3d700 1 -- 172.21.2.101:0/1320526836 wait complete. 2021-02-10T11:17:33.348 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839253 2021-02-10T11:17:33.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.756 7f276bbab700 1 Processor -- start 2021-02-10T11:17:33.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.756 7f276bbab700 1 -- start start 2021-02-10T11:17:33.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.756 7f276bbab700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27641b0470 0x7f27641b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:33.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.756 7f276bbab700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f27640f20e0 0x7f27640f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:33.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.756 7f276bbab700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27640f0b50 con 0x7f27640ef610 2021-02-10T11:17:33.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.756 7f276bbab700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27640f0820 con 0x7f27640f20e0 2021-02-10T11:17:33.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.756 7f276bbab700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27640f24e0 con 0x7f27641b0470 2021-02-10T11:17:33.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.757 7f276a1c8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f27640f20e0 0x7f27640f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:33.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.757 7f276a1c8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f27640f20e0 0x7f27640f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46108/0 (socket says 172.21.2.101:46108) 2021-02-10T11:17:33.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.757 7f276a1c8700 1 -- 172.21.2.101:0/83310113 learned_addr learned my addr 172.21.2.101:0/83310113 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:17:33.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.757 7f27691c6700 1 --1- 172.21.2.101:0/83310113 >> v1:172.21.2.101:6789/0 conn(0x7f27640ef610 0x7f27640efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46604/0 (socket says 172.21.2.101:46604) 2021-02-10T11:17:33.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.757 7f276a1c8700 1 -- 172.21.2.101:0/83310113 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27641b0470 msgr2=0x7f27641b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:33.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.757 7f276a1c8700 1 --2- 172.21.2.101:0/83310113 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27641b0470 0x7f27641b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:33.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.757 7f276a1c8700 1 -- 172.21.2.101:0/83310113 >> v1:172.21.2.101:6789/0 conn(0x7f27640ef610 legacy=0x7f27640efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:33.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.757 7f276a1c8700 1 -- 172.21.2.101:0/83310113 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27640e52e0 con 0x7f27640f20e0 2021-02-10T11:17:33.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.757 7f276a1c8700 1 --2- 172.21.2.101:0/83310113 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27640f20e0 0x7f27640f0200 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f275c006f00 tx=0x7f275c009f10).ready entity=mon.0 client_cookie=a0f94bc019d5d7cf server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:33.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.757 7f27689c5700 1 -- 172.21.2.101:0/83310113 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f275c00b3b0 con 0x7f27640f20e0 2021-02-10T11:17:33.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.757 7f27689c5700 1 -- 172.21.2.101:0/83310113 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275c00b630 con 0x7f27640f20e0 2021-02-10T11:17:33.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.757 7f27689c5700 1 -- 172.21.2.101:0/83310113 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f275c00bb50 con 0x7f27640f20e0 2021-02-10T11:17:33.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.757 7f276bbab700 1 -- 172.21.2.101:0/83310113 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27640f20e0 msgr2=0x7f27640f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:33.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.757 7f276bbab700 1 --2- 172.21.2.101:0/83310113 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27640f20e0 0x7f27640f0200 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f275c006f00 tx=0x7f275c009f10).stop 2021-02-10T11:17:33.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.758 7f276bbab700 1 -- 172.21.2.101:0/83310113 shutdown_connections 2021-02-10T11:17:33.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.758 7f276bbab700 1 --2- 172.21.2.101:0/83310113 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27640f20e0 0x7f27640f0200 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:33.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.758 7f276bbab700 1 --2- 172.21.2.101:0/83310113 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27641b0470 0x7f27641b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:33.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.758 7f276bbab700 1 -- 172.21.2.101:0/83310113 >> 172.21.2.101:0/83310113 conn(0x7f27640ce3e0 msgr2=0x7f27640e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:33.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.758 7f276bbab700 1 -- 172.21.2.101:0/83310113 shutdown_connections 2021-02-10T11:17:33.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.758 7f276bbab700 1 -- 172.21.2.101:0/83310113 wait complete. 2021-02-10T11:17:33.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.773 7f276bbab700 1 Processor -- start 2021-02-10T11:17:33.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.773 7f276bbab700 1 -- start start 2021-02-10T11:17:33.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.773 7f276bbab700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f27640bf910 0x7f27640bfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:33.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.773 7f276bbab700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27640c0330 0x7f27640c0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:33.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.773 7f276bbab700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27640c13d0 0x7f27640c17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:33.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.773 7f276bbab700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2764109930 con 0x7f27640c13d0 2021-02-10T11:17:33.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.773 7f276bbab700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2764109b50 con 0x7f27640bf910 2021-02-10T11:17:33.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.773 7f276bbab700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27640c1df0 con 0x7f27640c0330 2021-02-10T11:17:33.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.773 7f276a1c8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27640c13d0 0x7f27640c17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:33.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.773 7f276a1c8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27640c13d0 0x7f27640c17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46116/0 (socket says 172.21.2.101:46116) 2021-02-10T11:17:33.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.773 7f276a1c8700 1 -- 172.21.2.101:0/2618676789 learned_addr learned my addr 172.21.2.101:0/2618676789 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:17:33.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.774 7f276a1c8700 1 -- 172.21.2.101:0/2618676789 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27640c0330 msgr2=0x7f27640c0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:33.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.774 7f276a1c8700 1 --2- 172.21.2.101:0/2618676789 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27640c0330 0x7f27640c0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:33.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.774 7f276a1c8700 1 -- 172.21.2.101:0/2618676789 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f27640bf910 msgr2=0x7f27640bfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:33.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.774 7f276a1c8700 1 --2- 172.21.2.101:0/2618676789 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f27640bf910 0x7f27640bfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:33.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.774 7f276a1c8700 1 -- 172.21.2.101:0/2618676789 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27640c1fd0 con 0x7f27640c13d0 2021-02-10T11:17:33.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.774 7f276a1c8700 1 --2- 172.21.2.101:0/2618676789 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27640c13d0 0x7f27640c17d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f275c009390 tx=0x7f275c00bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:33.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.774 7f275a7fc700 1 -- 172.21.2.101:0/2618676789 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f275c003030 con 0x7f27640c13d0 2021-02-10T11:17:33.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.774 7f275a7fc700 1 -- 172.21.2.101:0/2618676789 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275c0068e0 con 0x7f27640c13d0 2021-02-10T11:17:33.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.774 7f275a7fc700 1 -- 172.21.2.101:0/2618676789 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f275c00c570 con 0x7f27640c13d0 2021-02-10T11:17:33.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.774 7f276bbab700 1 -- 172.21.2.101:0/2618676789 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27640c1fd0 con 0x7f27640c13d0 2021-02-10T11:17:33.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.775 7f276bbab700 1 -- 172.21.2.101:0/2618676789 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27640c1fd0 con 0x7f27640c13d0 2021-02-10T11:17:33.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.775 7f275a7fc700 1 -- 172.21.2.101:0/2618676789 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f275c005820 con 0x7f27640c13d0 2021-02-10T11:17:33.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.775 7f275a7fc700 1 --2- 172.21.2.101:0/2618676789 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f274c018730 0x7f274c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:33.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.775 7f275a7fc700 1 -- 172.21.2.101:0/2618676789 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f275c005820 con 0x7f27640c13d0 2021-02-10T11:17:33.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.775 7f27699c7700 1 --2- 172.21.2.101:0/2618676789 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f274c018730 0x7f274c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:33.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.776 7f27699c7700 1 --2- 172.21.2.101:0/2618676789 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f274c018730 0x7f274c01ac10 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:33.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.806 7f276bbab700 1 -- 172.21.2.101:0/2618676789 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27640c28d0 con 0x7f27640c13d0 2021-02-10T11:17:33.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.809 7f275a7fc700 1 -- 172.21.2.101:0/2618676789 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f275c005820 con 0x7f27640c13d0 2021-02-10T11:17:33.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.941 7f276bbab700 1 -- 172.21.2.101:0/2618676789 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839253", "target": ["mgr", ""]}) v1 -- 0x7f27640c28d0 con 0x7f274c018730 2021-02-10T11:17:33.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.944 7f275a7fc700 1 -- 172.21.2.101:0/2618676789 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f2760006e40 con 0x7f274c018730 2021-02-10T11:17:33.953 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839253/8d5f2d19-a3c7-4fb7-84d8-663471e37cac 2021-02-10T11:17:33.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.949 7f276bbab700 1 -- 172.21.2.101:0/2618676789 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f274c018730 msgr2=0x7f274c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:33.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.949 7f276bbab700 1 --2- 172.21.2.101:0/2618676789 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f274c018730 0x7f274c01ac10 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:33.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.949 7f276bbab700 1 -- 172.21.2.101:0/2618676789 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27640c13d0 msgr2=0x7f27640c17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:33.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.949 7f276bbab700 1 --2- 172.21.2.101:0/2618676789 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27640c13d0 0x7f27640c17d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f275c009390 tx=0x7f275c00bd40).stop 2021-02-10T11:17:33.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.950 7f276bbab700 1 -- 172.21.2.101:0/2618676789 shutdown_connections 2021-02-10T11:17:33.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.950 7f276bbab700 1 --2- 172.21.2.101:0/2618676789 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f274c018730 0x7f274c01ac10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:33.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.950 7f276bbab700 1 --2- 172.21.2.101:0/2618676789 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27640c0330 0x7f27640c0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:33.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.950 7f276bbab700 1 --2- 172.21.2.101:0/2618676789 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27640c13d0 0x7f27640c17d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:33.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.950 7f276bbab700 1 --2- 172.21.2.101:0/2618676789 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f27640bf910 0x7f27640bfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:33.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.950 7f276bbab700 1 -- 172.21.2.101:0/2618676789 >> 172.21.2.101:0/2618676789 conn(0x7f27640bf4e0 msgr2=0x7f2764191190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:33.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.950 7f276bbab700 1 -- 172.21.2.101:0/2618676789 shutdown_connections 2021-02-10T11:17:33.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:33.950 7f276bbab700 1 -- 172.21.2.101:0/2618676789 wait complete. 2021-02-10T11:17:33.963 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839253 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839253/8d5f2d19-a3c7-4fb7-84d8-663471e37cac 2021-02-10T11:17:33.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:33.963 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:33.963 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:34.299 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:34.299 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:34.300 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839253/8d5f2d19-a3c7-4fb7-84d8-663471e37cac/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:34.679 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:34.679 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:34.679 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0259628 s, 40.4 MB/s 2021-02-10T11:17:34.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:34.680 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:34.680 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:35.016 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:35.016 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:35.017 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839253/8d5f2d19-a3c7-4fb7-84d8-663471e37cac/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:35.391 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:35.391 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:35.391 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0228096 s, 46.0 MB/s 2021-02-10T11:17:35.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:35.392 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:35.392 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:35.728 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:35.729 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:35.729 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839253/8d5f2d19-a3c7-4fb7-84d8-663471e37cac/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:36.102 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:36.102 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:36.103 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241521 s, 43.4 MB/s 2021-02-10T11:17:36.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:36.104 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:36.104 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:36.439 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:36.440 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:36.440 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839253/8d5f2d19-a3c7-4fb7-84d8-663471e37cac/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:36.816 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:36.816 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:36.816 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244771 s, 42.8 MB/s 2021-02-10T11:17:36.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:36.817 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:36.817 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:37.151 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:37.151 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:37.152 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839253/8d5f2d19-a3c7-4fb7-84d8-663471e37cac/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:37.526 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:37.526 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:37.526 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240151 s, 43.7 MB/s 2021-02-10T11:17:37.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:37.527 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:37.527 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:37.862 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:37.863 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:37.863 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839253/8d5f2d19-a3c7-4fb7-84d8-663471e37cac/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:38.240 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:38.240 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:38.240 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.025978 s, 40.4 MB/s 2021-02-10T11:17:38.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:38.241 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:38.241 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:38.576 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:38.577 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:38.577 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839253/8d5f2d19-a3c7-4fb7-84d8-663471e37cac/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:38.947 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:38.947 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:38.947 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231092 s, 45.4 MB/s 2021-02-10T11:17:38.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:38.948 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:38.948 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:39.284 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:39.284 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:39.284 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839253/8d5f2d19-a3c7-4fb7-84d8-663471e37cac/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:39.661 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:39.661 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:39.662 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0262936 s, 39.9 MB/s 2021-02-10T11:17:39.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:39.663 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:39.663 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:39.997 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:39.997 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:39.997 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839253/8d5f2d19-a3c7-4fb7-84d8-663471e37cac/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:40.373 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:40.373 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:40.373 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253554 s, 41.4 MB/s 2021-02-10T11:17:40.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:40.374 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:40.374 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:40.710 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:40.711 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:40.711 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839253/8d5f2d19-a3c7-4fb7-84d8-663471e37cac/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:41.088 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:41.088 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:41.088 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0254115 s, 41.3 MB/s 2021-02-10T11:17:41.089 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839254 2021-02-10T11:17:41.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.496 7f4dda07c700 1 Processor -- start 2021-02-10T11:17:41.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.496 7f4dda07c700 1 -- start start 2021-02-10T11:17:41.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.497 7f4dda07c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dd41b0470 0x7f4dd41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:41.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.497 7f4dda07c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4dd40f20e0 0x7f4dd40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:41.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.497 7f4dda07c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd40f0b50 con 0x7f4dd40ef610 2021-02-10T11:17:41.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.497 7f4dda07c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd40f0820 con 0x7f4dd40f20e0 2021-02-10T11:17:41.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.497 7f4dda07c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd40f24e0 con 0x7f4dd41b0470 2021-02-10T11:17:41.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.498 7f4dd37fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dd41b0470 0x7f4dd41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:41.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.498 7f4dd37fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dd41b0470 0x7f4dd41b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:38852/0 (socket says 172.21.2.101:38852) 2021-02-10T11:17:41.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.498 7f4dd37fe700 1 -- 172.21.2.101:0/1907122012 learned_addr learned my addr 172.21.2.101:0/1907122012 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:17:41.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.498 7f4dd3fff700 1 --2- 172.21.2.101:0/1907122012 >> v2:172.21.2.108:3300/0 conn(0x7f4dd40f20e0 0x7f4dd40f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:41.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.498 7f4dd37fe700 1 -- 172.21.2.101:0/1907122012 >> v2:172.21.2.108:3300/0 conn(0x7f4dd40f20e0 msgr2=0x7f4dd40f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:41.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.498 7f4dd37fe700 1 --2- 172.21.2.101:0/1907122012 >> v2:172.21.2.108:3300/0 conn(0x7f4dd40f20e0 0x7f4dd40f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:41.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.498 7f4dd37fe700 1 -- 172.21.2.101:0/1907122012 >> v1:172.21.2.108:6789/0 conn(0x7f4dd40ef610 legacy=0x7f4dd40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:41.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.498 7f4dd27fc700 1 -- 172.21.2.101:0/1907122012 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3688711574 0 0) 0x7f4dbc000ff0 con 0x7f4dd40ef610 2021-02-10T11:17:41.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.498 7f4dd37fe700 1 -- 172.21.2.101:0/1907122012 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd40e52e0 con 0x7f4dd41b0470 2021-02-10T11:17:41.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.498 7f4dd3fff700 1 --2- 172.21.2.101:0/1907122012 >> v2:172.21.2.108:3300/0 conn(0x7f4dd40f20e0 0x7f4dd40f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:17:41.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.498 7f4dd37fe700 1 --2- 172.21.2.101:0/1907122012 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dd41b0470 0x7f4dd41b0870 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f4dc4006700 tx=0x7f4dc4009b80).ready entity=mon.2 client_cookie=48515e0e0aa7aad3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:41.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.499 7f4dd27fc700 1 -- 172.21.2.101:0/1907122012 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4dc4006340 con 0x7f4dd41b0470 2021-02-10T11:17:41.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.499 7f4dd27fc700 1 -- 172.21.2.101:0/1907122012 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc400c0c0 con 0x7f4dd41b0470 2021-02-10T11:17:41.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.499 7f4dd27fc700 1 -- 172.21.2.101:0/1907122012 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4dc400c520 con 0x7f4dd41b0470 2021-02-10T11:17:41.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.499 7f4dda07c700 1 -- 172.21.2.101:0/1907122012 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dd41b0470 msgr2=0x7f4dd41b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:41.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.499 7f4dda07c700 1 --2- 172.21.2.101:0/1907122012 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dd41b0470 0x7f4dd41b0870 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f4dc4006700 tx=0x7f4dc4009b80).stop 2021-02-10T11:17:41.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.499 7f4dda07c700 1 -- 172.21.2.101:0/1907122012 shutdown_connections 2021-02-10T11:17:41.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.499 7f4dda07c700 1 --2- 172.21.2.101:0/1907122012 >> v2:172.21.2.108:3300/0 conn(0x7f4dd40f20e0 0x7f4dd40f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:41.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.499 7f4dda07c700 1 --2- 172.21.2.101:0/1907122012 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dd41b0470 0x7f4dd41b0870 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:41.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.499 7f4dda07c700 1 -- 172.21.2.101:0/1907122012 >> 172.21.2.101:0/1907122012 conn(0x7f4dd40ce3e0 msgr2=0x7f4dd40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:41.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.499 7f4dda07c700 1 -- 172.21.2.101:0/1907122012 shutdown_connections 2021-02-10T11:17:41.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.500 7f4dda07c700 1 -- 172.21.2.101:0/1907122012 wait complete. 2021-02-10T11:17:41.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.514 7f4dda07c700 1 Processor -- start 2021-02-10T11:17:41.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.514 7f4dda07c700 1 -- start start 2021-02-10T11:17:41.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.514 7f4dda07c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4dd409b930 0x7f4dd40b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:41.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.514 7f4dda07c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dd40b8410 0x7f4dd40b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:41.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.514 7f4dda07c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dd40c1f80 0x7f4dd40c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:41.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.514 7f4dda07c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd40c1a00 con 0x7f4dd40b8410 2021-02-10T11:17:41.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.514 7f4dda07c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd40b8e30 con 0x7f4dd409b930 2021-02-10T11:17:41.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.514 7f4dda07c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd40b9050 con 0x7f4dd40c1f80 2021-02-10T11:17:41.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.514 7f4dd2ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dd40b8410 0x7f4dd40b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:41.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.514 7f4dd3fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dd40c1f80 0x7f4dd40c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:41.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.514 7f4dd3fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dd40c1f80 0x7f4dd40c2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:38858/0 (socket says 172.21.2.101:38858) 2021-02-10T11:17:41.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.514 7f4dd2ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dd40b8410 0x7f4dd40b8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46130/0 (socket says 172.21.2.101:46130) 2021-02-10T11:17:41.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.514 7f4dd3fff700 1 -- 172.21.2.101:0/1164048088 learned_addr learned my addr 172.21.2.101:0/1164048088 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:17:41.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.515 7f4dd37fe700 1 --2- 172.21.2.101:0/1164048088 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4dd409b930 0x7f4dd40b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:41.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.515 7f4dd2ffd700 1 -- 172.21.2.101:0/1164048088 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dd40c1f80 msgr2=0x7f4dd40c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:41.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.515 7f4dd2ffd700 1 --2- 172.21.2.101:0/1164048088 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dd40c1f80 0x7f4dd40c2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:41.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.515 7f4dd2ffd700 1 -- 172.21.2.101:0/1164048088 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4dd409b930 msgr2=0x7f4dd40b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:41.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.515 7f4dd2ffd700 1 --2- 172.21.2.101:0/1164048088 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4dd409b930 0x7f4dd40b7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:41.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.515 7f4dd2ffd700 1 -- 172.21.2.101:0/1164048088 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd40c29a0 con 0x7f4dd40b8410 2021-02-10T11:17:41.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.515 7f4dd3fff700 1 --2- 172.21.2.101:0/1164048088 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dd40c1f80 0x7f4dd40c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:17:41.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.515 7f4dd2ffd700 1 --2- 172.21.2.101:0/1164048088 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dd40b8410 0x7f4dd40b8810 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f4dbc00a4a0 tx=0x7f4dbc00a840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:41.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.515 7f4dcbfff700 1 -- 172.21.2.101:0/1164048088 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4dbc00ac30 con 0x7f4dd40b8410 2021-02-10T11:17:41.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.515 7f4dcbfff700 1 -- 172.21.2.101:0/1164048088 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dbc007240 con 0x7f4dd40b8410 2021-02-10T11:17:41.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.515 7f4dcbfff700 1 -- 172.21.2.101:0/1164048088 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4dbc006540 con 0x7f4dd40b8410 2021-02-10T11:17:41.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.515 7f4dda07c700 1 -- 172.21.2.101:0/1164048088 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd40c29a0 con 0x7f4dd40b8410 2021-02-10T11:17:41.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.515 7f4dda07c700 1 -- 172.21.2.101:0/1164048088 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd40c2f30 con 0x7f4dd40b8410 2021-02-10T11:17:41.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.516 7f4dcbfff700 1 -- 172.21.2.101:0/1164048088 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4dbc007420 con 0x7f4dd40b8410 2021-02-10T11:17:41.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.516 7f4dcbfff700 1 --2- 172.21.2.101:0/1164048088 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4db80186c0 0x7f4db801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:41.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.516 7f4dcbfff700 1 -- 172.21.2.101:0/1164048088 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4dbc032560 con 0x7f4dd40b8410 2021-02-10T11:17:41.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.516 7f4dd37fe700 1 --2- 172.21.2.101:0/1164048088 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4db80186c0 0x7f4db801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:41.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.516 7f4dd37fe700 1 --2- 172.21.2.101:0/1164048088 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4db80186c0 0x7f4db801abc0 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:41.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.547 7f4dda07c700 1 -- 172.21.2.101:0/1164048088 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd4105b30 con 0x7f4dd40b8410 2021-02-10T11:17:41.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.549 7f4dcbfff700 1 -- 172.21.2.101:0/1164048088 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4dbc006700 con 0x7f4dd40b8410 2021-02-10T11:17:41.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.676 7f4dda07c700 1 -- 172.21.2.101:0/1164048088 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839254", "target": ["mgr", ""]}) v1 -- 0x7f4dd40c2f30 con 0x7f4db80186c0 2021-02-10T11:17:41.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.700 7f4dcbfff700 1 -- 172.21.2.101:0/1164048088 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f4dc40009b0 con 0x7f4db80186c0 2021-02-10T11:17:41.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.709 7f4dda07c700 1 -- 172.21.2.101:0/1164048088 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4db80186c0 msgr2=0x7f4db801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:41.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.709 7f4dda07c700 1 --2- 172.21.2.101:0/1164048088 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4db80186c0 0x7f4db801abc0 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:41.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.709 7f4dda07c700 1 -- 172.21.2.101:0/1164048088 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dd40b8410 msgr2=0x7f4dd40b8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:41.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.709 7f4dda07c700 1 --2- 172.21.2.101:0/1164048088 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dd40b8410 0x7f4dd40b8810 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f4dbc00a4a0 tx=0x7f4dbc00a840).stop 2021-02-10T11:17:41.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.709 7f4dda07c700 1 -- 172.21.2.101:0/1164048088 shutdown_connections 2021-02-10T11:17:41.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.709 7f4dda07c700 1 --2- 172.21.2.101:0/1164048088 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4db80186c0 0x7f4db801abc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:41.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.709 7f4dda07c700 1 --2- 172.21.2.101:0/1164048088 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dd40c1f80 0x7f4dd40c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:41.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.709 7f4dda07c700 1 --2- 172.21.2.101:0/1164048088 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dd40b8410 0x7f4dd40b8810 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:41.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.709 7f4dda07c700 1 --2- 172.21.2.101:0/1164048088 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4dd409b930 0x7f4dd40b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:41.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.709 7f4dda07c700 1 -- 172.21.2.101:0/1164048088 >> 172.21.2.101:0/1164048088 conn(0x7f4dd4094e10 msgr2=0x7f4dd40a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:41.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.709 7f4dda07c700 1 -- 172.21.2.101:0/1164048088 shutdown_connections 2021-02-10T11:17:41.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:41.709 7f4dda07c700 1 -- 172.21.2.101:0/1164048088 wait complete. 2021-02-10T11:17:41.723 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839254 2021-02-10T11:17:42.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.130 7fd01826e700 1 Processor -- start 2021-02-10T11:17:42.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.130 7fd01826e700 1 -- start start 2021-02-10T11:17:42.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.130 7fd01826e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0100f05f0 0x7fd0100f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.130 7fd01826e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0101b0c70 con 0x7fd0101b0470 2021-02-10T11:17:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.130 7fd01826e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0100f0b50 con 0x7fd0101b0870 2021-02-10T11:17:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.130 7fd01826e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0100e5ea0 con 0x7fd0100f05f0 2021-02-10T11:17:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.131 7fd01608a700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd0101b0470 0x7fd0100ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46628/0 (socket says 172.21.2.101:46628) 2021-02-10T11:17:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.131 7fd01608a700 1 -- 172.21.2.101:0/1910628992 learned_addr learned my addr 172.21.2.101:0/1910628992 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:17:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.131 7fd01688b700 1 --2- 172.21.2.101:0/1910628992 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0100f05f0 0x7fd0100f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.131 7fd015088700 1 -- 172.21.2.101:0/1910628992 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 626195255 0 0) 0x7fd0040010b0 con 0x7fd0101b0470 2021-02-10T11:17:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.131 7fd015088700 1 -- 172.21.2.101:0/1910628992 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcffc002fe0 con 0x7fd0101b0470 2021-02-10T11:17:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.131 7fd01688b700 1 -- 172.21.2.101:0/1910628992 >> v1:172.21.2.108:6789/0 conn(0x7fd0101b0870 legacy=0x7fd0100efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.131 7fd01688b700 1 -- 172.21.2.101:0/1910628992 >> v1:172.21.2.101:6789/0 conn(0x7fd0101b0470 legacy=0x7fd0100ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:42.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.131 7fd01688b700 1 -- 172.21.2.101:0/1910628992 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0100e52e0 con 0x7fd0100f05f0 2021-02-10T11:17:42.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.131 7fd01688b700 1 --2- 172.21.2.101:0/1910628992 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0100f05f0 0x7fd0100f20e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fd008006e50 tx=0x7fd008009e70).ready entity=mon.2 client_cookie=11b53351d7a41f85 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:42.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.132 7fd015088700 1 -- 172.21.2.101:0/1910628992 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd00800b510 con 0x7fd0100f05f0 2021-02-10T11:17:42.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.132 7fd015088700 1 -- 172.21.2.101:0/1910628992 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd008006790 con 0x7fd0100f05f0 2021-02-10T11:17:42.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.132 7fd015088700 1 -- 172.21.2.101:0/1910628992 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd00800b940 con 0x7fd0100f05f0 2021-02-10T11:17:42.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.132 7fd01826e700 1 -- 172.21.2.101:0/1910628992 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0100f05f0 msgr2=0x7fd0100f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:42.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.132 7fd01826e700 1 --2- 172.21.2.101:0/1910628992 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0100f05f0 0x7fd0100f20e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fd008006e50 tx=0x7fd008009e70).stop 2021-02-10T11:17:42.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.132 7fd01826e700 1 -- 172.21.2.101:0/1910628992 shutdown_connections 2021-02-10T11:17:42.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.132 7fd01826e700 1 --2- 172.21.2.101:0/1910628992 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0100f05f0 0x7fd0100f20e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:42.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.132 7fd01826e700 1 -- 172.21.2.101:0/1910628992 >> 172.21.2.101:0/1910628992 conn(0x7fd0100ce3e0 msgr2=0x7fd0100e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:42.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.132 7fd01826e700 1 -- 172.21.2.101:0/1910628992 shutdown_connections 2021-02-10T11:17:42.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.132 7fd01826e700 1 -- 172.21.2.101:0/1910628992 wait complete. 2021-02-10T11:17:42.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.147 7fd01826e700 1 Processor -- start 2021-02-10T11:17:42.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.147 7fd01826e700 1 -- start start 2021-02-10T11:17:42.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.147 7fd01826e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0100c36a0 0x7fd0100c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:42.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.147 7fd01826e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0100c3190 0x7fd0100b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:42.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.147 7fd01826e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0100b2370 0x7fd0100b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:42.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.147 7fd01826e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0100b27c0 con 0x7fd0100c36a0 2021-02-10T11:17:42.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.147 7fd01826e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0100b3360 con 0x7fd0100b2370 2021-02-10T11:17:42.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.147 7fd01826e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0100b3540 con 0x7fd0100c3190 2021-02-10T11:17:42.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.147 7fd01608a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0100c36a0 0x7fd0100c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:42.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.147 7fd01608a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0100c36a0 0x7fd0100c2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46140/0 (socket says 172.21.2.101:46140) 2021-02-10T11:17:42.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.147 7fd01608a700 1 -- 172.21.2.101:0/1550275086 learned_addr learned my addr 172.21.2.101:0/1550275086 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:17:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.148 7fd01608a700 1 -- 172.21.2.101:0/1550275086 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0100c3190 msgr2=0x7fd0100b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.148 7fd01608a700 1 --2- 172.21.2.101:0/1550275086 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0100c3190 0x7fd0100b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.148 7fd01608a700 1 -- 172.21.2.101:0/1550275086 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0100b2370 msgr2=0x7fd0100b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.148 7fd01608a700 1 --2- 172.21.2.101:0/1550275086 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0100b2370 0x7fd0100b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.148 7fd01608a700 1 -- 172.21.2.101:0/1550275086 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0100b3760 con 0x7fd0100c36a0 2021-02-10T11:17:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.148 7fd01608a700 1 --2- 172.21.2.101:0/1550275086 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0100c36a0 0x7fd0100c2b70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7fd004001c60 tx=0x7fd00400a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.148 7fd002ffd700 1 -- 172.21.2.101:0/1550275086 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd00400acc0 con 0x7fd0100c36a0 2021-02-10T11:17:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.148 7fd002ffd700 1 -- 172.21.2.101:0/1550275086 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd004004350 con 0x7fd0100c36a0 2021-02-10T11:17:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.148 7fd002ffd700 1 -- 172.21.2.101:0/1550275086 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd004004870 con 0x7fd0100c36a0 2021-02-10T11:17:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.148 7fd01826e700 1 -- 172.21.2.101:0/1550275086 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0100b3760 con 0x7fd0100c36a0 2021-02-10T11:17:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.148 7fd01826e700 1 -- 172.21.2.101:0/1550275086 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0100b3760 con 0x7fd0100c36a0 2021-02-10T11:17:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.149 7fd002ffd700 1 -- 172.21.2.101:0/1550275086 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd004004530 con 0x7fd0100c36a0 2021-02-10T11:17:42.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.149 7fd002ffd700 1 --2- 172.21.2.101:0/1550275086 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcff80186c0 0x7fcff801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:42.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.149 7fd002ffd700 1 -- 172.21.2.101:0/1550275086 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd004032580 con 0x7fd0100c36a0 2021-02-10T11:17:42.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.149 7fd015889700 1 --2- 172.21.2.101:0/1550275086 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcff80186c0 0x7fcff801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:42.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.149 7fd015889700 1 --2- 172.21.2.101:0/1550275086 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcff80186c0 0x7fcff801abc0 crc :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:42.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.180 7fd01826e700 1 -- 172.21.2.101:0/1550275086 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0100edd10 con 0x7fd0100c36a0 2021-02-10T11:17:42.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.183 7fd002ffd700 1 -- 172.21.2.101:0/1550275086 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd004004140 con 0x7fd0100c36a0 2021-02-10T11:17:42.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.312 7fd01826e700 1 -- 172.21.2.101:0/1550275086 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839254", "target": ["mgr", ""]}) v1 -- 0x7fd0100edd10 con 0x7fcff80186c0 2021-02-10T11:17:42.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.315 7fd002ffd700 1 -- 172.21.2.101:0/1550275086 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fd00c0009b0 con 0x7fcff80186c0 2021-02-10T11:17:42.320 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839254/525e5d20-9bd1-494f-9251-e6f6dfde37cf 2021-02-10T11:17:42.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.316 7fd01826e700 1 -- 172.21.2.101:0/1550275086 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcff80186c0 msgr2=0x7fcff801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:42.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.316 7fd01826e700 1 --2- 172.21.2.101:0/1550275086 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcff80186c0 0x7fcff801abc0 crc :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:42.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.316 7fd01826e700 1 -- 172.21.2.101:0/1550275086 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0100c36a0 msgr2=0x7fd0100c2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:42.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.316 7fd01826e700 1 --2- 172.21.2.101:0/1550275086 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0100c36a0 0x7fd0100c2b70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7fd004001c60 tx=0x7fd00400a8d0).stop 2021-02-10T11:17:42.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.316 7fd01826e700 1 -- 172.21.2.101:0/1550275086 shutdown_connections 2021-02-10T11:17:42.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.316 7fd01826e700 1 --2- 172.21.2.101:0/1550275086 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcff80186c0 0x7fcff801abc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:42.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.316 7fd01826e700 1 --2- 172.21.2.101:0/1550275086 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0100c3190 0x7fd0100b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:42.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.316 7fd01826e700 1 --2- 172.21.2.101:0/1550275086 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0100b2370 0x7fd0100b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:42.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.316 7fd01826e700 1 --2- 172.21.2.101:0/1550275086 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0100c36a0 0x7fd0100c2b70 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:42.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.316 7fd01826e700 1 -- 172.21.2.101:0/1550275086 >> 172.21.2.101:0/1550275086 conn(0x7fd01009f630 msgr2=0x7fd0100a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:42.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.316 7fd01826e700 1 -- 172.21.2.101:0/1550275086 shutdown_connections 2021-02-10T11:17:42.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:42.316 7fd01826e700 1 -- 172.21.2.101:0/1550275086 wait complete. 2021-02-10T11:17:42.330 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839254 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839254/525e5d20-9bd1-494f-9251-e6f6dfde37cf 2021-02-10T11:17:42.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:42.330 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:42.330 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:42.665 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:42.666 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:42.666 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839254/525e5d20-9bd1-494f-9251-e6f6dfde37cf/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:43.046 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:43.046 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:43.047 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.025367 s, 41.3 MB/s 2021-02-10T11:17:43.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:43.048 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:43.048 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:43.384 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:43.384 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:43.385 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839254/525e5d20-9bd1-494f-9251-e6f6dfde37cf/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:43.757 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:43.757 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:43.757 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242157 s, 43.3 MB/s 2021-02-10T11:17:43.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:43.764 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:43.764 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:44.099 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:44.100 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:44.100 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839254/525e5d20-9bd1-494f-9251-e6f6dfde37cf/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:44.477 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:44.477 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:44.477 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0267994 s, 39.1 MB/s 2021-02-10T11:17:44.478 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:44.478 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:44.478 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:44.812 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:44.813 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:44.813 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839254/525e5d20-9bd1-494f-9251-e6f6dfde37cf/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:45.188 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:45.188 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:45.188 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252455 s, 41.5 MB/s 2021-02-10T11:17:45.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:45.190 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:45.190 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:45.525 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:45.526 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:45.526 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839254/525e5d20-9bd1-494f-9251-e6f6dfde37cf/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:45.899 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:45.900 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:45.900 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239203 s, 43.8 MB/s 2021-02-10T11:17:45.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:45.901 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:45.901 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:46.235 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:46.236 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:46.236 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839254/525e5d20-9bd1-494f-9251-e6f6dfde37cf/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:46.301 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:17:46.305 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:17:46.611 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:46.611 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:46.612 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251831 s, 41.6 MB/s 2021-02-10T11:17:46.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:46.613 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:46.613 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:46.947 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:46.948 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:46.948 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839254/525e5d20-9bd1-494f-9251-e6f6dfde37cf/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:47.324 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:47.325 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:47.325 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245848 s, 42.7 MB/s 2021-02-10T11:17:47.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:47.326 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:47.326 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:47.661 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:47.661 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:47.662 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839254/525e5d20-9bd1-494f-9251-e6f6dfde37cf/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:48.038 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:48.038 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:48.038 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253493 s, 41.4 MB/s 2021-02-10T11:17:48.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:48.039 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:48.039 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:48.373 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:48.373 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:48.374 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839254/525e5d20-9bd1-494f-9251-e6f6dfde37cf/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:48.750 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:48.750 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:48.750 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0259429 s, 40.4 MB/s 2021-02-10T11:17:48.751 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:48.751 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:48.751 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:49.086 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:49.087 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:49.087 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839254/525e5d20-9bd1-494f-9251-e6f6dfde37cf/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:49.457 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:49.457 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:49.457 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235822 s, 44.5 MB/s 2021-02-10T11:17:49.458 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839255 2021-02-10T11:17:49.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.865 7f51e386c700 1 Processor -- start 2021-02-10T11:17:49.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.865 7f51e386c700 1 -- start start 2021-02-10T11:17:49.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.866 7f51e386c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51dc1b0470 0x7f51dc1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:49.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.866 7f51e386c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f51dc0ef610 0x7f51dc0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:49.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.866 7f51e386c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f51dc0f0030 0x7f51dc0f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:49.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.866 7f51e386c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51dc0e5ea0 con 0x7f51dc0ef610 2021-02-10T11:17:49.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.866 7f51e386c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51dc0f0b50 con 0x7f51dc0f0030 2021-02-10T11:17:49.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.866 7f51e386c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0e52e0 con 0x7f51dc1b0470 2021-02-10T11:17:49.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.866 7f51e0e87700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f51dc0ef610 0x7f51dc0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:49.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.866 7f51e0e87700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f51dc0ef610 0x7f51dc0efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46148/0 (socket says 172.21.2.101:46148) 2021-02-10T11:17:49.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.866 7f51e0e87700 1 -- 172.21.2.101:0/858823680 learned_addr learned my addr 172.21.2.101:0/858823680 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:17:49.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.866 7f51e1688700 1 --2- 172.21.2.101:0/858823680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51dc1b0470 0x7f51dc1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:49.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.866 7f51e1e89700 1 --2- 172.21.2.101:0/858823680 >> v2:172.21.2.108:3300/0 conn(0x7f51dc0f0030 0x7f51dc0f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:49.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.867 7f51e0e87700 1 -- 172.21.2.101:0/858823680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51dc1b0470 msgr2=0x7f51dc1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:49.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.867 7f51e0e87700 1 --2- 172.21.2.101:0/858823680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51dc1b0470 0x7f51dc1b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:49.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.867 7f51e0e87700 1 -- 172.21.2.101:0/858823680 >> v2:172.21.2.108:3300/0 conn(0x7f51dc0f0030 msgr2=0x7f51dc0f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:49.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.867 7f51e0e87700 1 --2- 172.21.2.101:0/858823680 >> v2:172.21.2.108:3300/0 conn(0x7f51dc0f0030 0x7f51dc0f0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:49.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.867 7f51e0e87700 1 -- 172.21.2.101:0/858823680 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc0f10d0 con 0x7f51dc0ef610 2021-02-10T11:17:49.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.867 7f51e0e87700 1 --2- 172.21.2.101:0/858823680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51dc0ef610 0x7f51dc0efa10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f51d0006700 tx=0x7f51d0009b80).ready entity=mon.0 client_cookie=48405471cdcfce04 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:49.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.867 7f51cffff700 1 -- 172.21.2.101:0/858823680 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f51d000c0c0 con 0x7f51dc0ef610 2021-02-10T11:17:49.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.867 7f51cffff700 1 -- 172.21.2.101:0/858823680 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d000c340 con 0x7f51dc0ef610 2021-02-10T11:17:49.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.867 7f51cffff700 1 -- 172.21.2.101:0/858823680 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f51d000c860 con 0x7f51dc0ef610 2021-02-10T11:17:49.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.867 7f51e386c700 1 -- 172.21.2.101:0/858823680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51dc0ef610 msgr2=0x7f51dc0efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:49.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.867 7f51e386c700 1 --2- 172.21.2.101:0/858823680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51dc0ef610 0x7f51dc0efa10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f51d0006700 tx=0x7f51d0009b80).stop 2021-02-10T11:17:49.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.868 7f51e386c700 1 -- 172.21.2.101:0/858823680 shutdown_connections 2021-02-10T11:17:49.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.868 7f51e386c700 1 --2- 172.21.2.101:0/858823680 >> v2:172.21.2.108:3300/0 conn(0x7f51dc0f0030 0x7f51dc0f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:49.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.868 7f51e386c700 1 --2- 172.21.2.101:0/858823680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51dc0ef610 0x7f51dc0efa10 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:49.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.868 7f51e386c700 1 --2- 172.21.2.101:0/858823680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51dc1b0470 0x7f51dc1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:49.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.868 7f51e386c700 1 -- 172.21.2.101:0/858823680 >> 172.21.2.101:0/858823680 conn(0x7f51dc0ce3e0 msgr2=0x7f51dc0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:49.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.868 7f51e386c700 1 -- 172.21.2.101:0/858823680 shutdown_connections 2021-02-10T11:17:49.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.868 7f51e386c700 1 -- 172.21.2.101:0/858823680 wait complete. 2021-02-10T11:17:49.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.883 7f51e386c700 1 Processor -- start 2021-02-10T11:17:49.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.883 7f51e386c700 1 -- start start 2021-02-10T11:17:49.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.884 7f51e386c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51dc09ba50 0x7f51dc10a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:49.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.884 7f51e386c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f51dc0c27f0 0x7f51dc1095c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:49.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.884 7f51e386c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51dc0c32d0 0x7f51dc0c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:49.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.884 7f51e386c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc199f50 con 0x7f51dc0c32d0 2021-02-10T11:17:49.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.884 7f51e386c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0c2d50 con 0x7f51dc0c27f0 2021-02-10T11:17:49.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.884 7f51e386c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc10a730 con 0x7f51dc09ba50 2021-02-10T11:17:49.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.884 7f51e1e89700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51dc0c32d0 0x7f51dc0c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:49.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.884 7f51e1e89700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51dc0c32d0 0x7f51dc0c36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46156/0 (socket says 172.21.2.101:46156) 2021-02-10T11:17:49.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.884 7f51e1e89700 1 -- 172.21.2.101:0/1459255499 learned_addr learned my addr 172.21.2.101:0/1459255499 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:17:49.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.885 7f51e1688700 1 --2- 172.21.2.101:0/1459255499 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51dc09ba50 0x7f51dc10a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:49.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.885 7f51e0e87700 1 --2- 172.21.2.101:0/1459255499 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f51dc0c27f0 0x7f51dc1095c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:49.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.885 7f51e1e89700 1 -- 172.21.2.101:0/1459255499 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51dc09ba50 msgr2=0x7f51dc10a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:49.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.885 7f51e1e89700 1 --2- 172.21.2.101:0/1459255499 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51dc09ba50 0x7f51dc10a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:49.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.885 7f51e1e89700 1 -- 172.21.2.101:0/1459255499 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f51dc0c27f0 msgr2=0x7f51dc1095c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:49.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.885 7f51e1e89700 1 --2- 172.21.2.101:0/1459255499 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f51dc0c27f0 0x7f51dc1095c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:49.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.885 7f51e1e89700 1 -- 172.21.2.101:0/1459255499 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc09be50 con 0x7f51dc0c32d0 2021-02-10T11:17:49.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.885 7f51e0e87700 1 --2- 172.21.2.101:0/1459255499 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f51dc0c27f0 0x7f51dc1095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:17:49.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.885 7f51e1e89700 1 --2- 172.21.2.101:0/1459255499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51dc0c32d0 0x7f51dc0c36d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f51d400b910 tx=0x7f51d400bcb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:49.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.885 7f51cdffb700 1 -- 172.21.2.101:0/1459255499 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f51d40066f0 con 0x7f51dc0c32d0 2021-02-10T11:17:49.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.885 7f51cdffb700 1 -- 172.21.2.101:0/1459255499 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d4003030 con 0x7f51dc0c32d0 2021-02-10T11:17:49.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.885 7f51e386c700 1 -- 172.21.2.101:0/1459255499 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51dc09be50 con 0x7f51dc0c32d0 2021-02-10T11:17:49.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.885 7f51cdffb700 1 -- 172.21.2.101:0/1459255499 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f51d4001470 con 0x7f51dc0c32d0 2021-02-10T11:17:49.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.885 7f51e386c700 1 -- 172.21.2.101:0/1459255499 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51dc0c3cf0 con 0x7f51dc0c32d0 2021-02-10T11:17:49.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.886 7f51cdffb700 1 -- 172.21.2.101:0/1459255499 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f51d40011f0 con 0x7f51dc0c32d0 2021-02-10T11:17:49.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.886 7f51cdffb700 1 --2- 172.21.2.101:0/1459255499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51bc01a900 0x7f51bc01cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:49.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.886 7f51cdffb700 1 -- 172.21.2.101:0/1459255499 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f51d40329b0 con 0x7f51dc0c32d0 2021-02-10T11:17:49.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.886 7f51e1688700 1 --2- 172.21.2.101:0/1459255499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51bc01a900 0x7f51bc01cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:49.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.886 7f51e1688700 1 --2- 172.21.2.101:0/1459255499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51bc01a900 0x7f51bc01cde0 crc :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:49.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.916 7f51e386c700 1 -- 172.21.2.101:0/1459255499 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51dc199f50 con 0x7f51dc0c32d0 2021-02-10T11:17:49.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:49.918 7f51cdffb700 1 -- 172.21.2.101:0/1459255499 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f51d4000f40 con 0x7f51dc0c32d0 2021-02-10T11:17:50.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.043 7f51e386c700 1 -- 172.21.2.101:0/1459255499 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839255", "target": ["mgr", ""]}) v1 -- 0x7f51dc0c3cf0 con 0x7f51bc01a900 2021-02-10T11:17:50.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.070 7f51cdffb700 1 -- 172.21.2.101:0/1459255499 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f51d8006a70 con 0x7f51bc01a900 2021-02-10T11:17:50.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.076 7f51e386c700 1 -- 172.21.2.101:0/1459255499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51bc01a900 msgr2=0x7f51bc01cde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:50.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.076 7f51e386c700 1 --2- 172.21.2.101:0/1459255499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51bc01a900 0x7f51bc01cde0 crc :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:50.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.076 7f51e386c700 1 -- 172.21.2.101:0/1459255499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51dc0c32d0 msgr2=0x7f51dc0c36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:50.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.076 7f51e386c700 1 --2- 172.21.2.101:0/1459255499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51dc0c32d0 0x7f51dc0c36d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f51d400b910 tx=0x7f51d400bcb0).stop 2021-02-10T11:17:50.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.076 7f51e386c700 1 -- 172.21.2.101:0/1459255499 shutdown_connections 2021-02-10T11:17:50.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.076 7f51e386c700 1 --2- 172.21.2.101:0/1459255499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51dc0c32d0 0x7f51dc0c36d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:50.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.076 7f51e386c700 1 --2- 172.21.2.101:0/1459255499 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f51dc0c27f0 0x7f51dc1095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:50.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.076 7f51e386c700 1 --2- 172.21.2.101:0/1459255499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51bc01a900 0x7f51bc01cde0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:50.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.076 7f51e386c700 1 --2- 172.21.2.101:0/1459255499 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51dc09ba50 0x7f51dc10a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:50.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.076 7f51e386c700 1 -- 172.21.2.101:0/1459255499 >> 172.21.2.101:0/1459255499 conn(0x7f51dc0c40e0 msgr2=0x7f51dc0bf620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:50.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.077 7f51e386c700 1 -- 172.21.2.101:0/1459255499 shutdown_connections 2021-02-10T11:17:50.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.077 7f51e386c700 1 -- 172.21.2.101:0/1459255499 wait complete. 2021-02-10T11:17:50.090 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839255 2021-02-10T11:17:50.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.497 7f335f9c6700 1 Processor -- start 2021-02-10T11:17:50.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.497 7f335f9c6700 1 -- start start 2021-02-10T11:17:50.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.497 7f335f9c6700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f33580f05f0 0x7f33580f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:50.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.497 7f335f9c6700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33581b0c70 con 0x7f33581b0470 2021-02-10T11:17:50.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.497 7f335f9c6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33580f0b50 con 0x7f33581b0870 2021-02-10T11:17:50.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.497 7f335f9c6700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33580e5ea0 con 0x7f33580f05f0 2021-02-10T11:17:50.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.497 7f335d7e2700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f33581b0470 0x7f33580ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46654/0 (socket says 172.21.2.101:46654) 2021-02-10T11:17:50.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.497 7f335d7e2700 1 -- 172.21.2.101:0/562120259 learned_addr learned my addr 172.21.2.101:0/562120259 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:17:50.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.498 7f334bfff700 1 -- 172.21.2.101:0/562120259 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360918227 0 0) 0x7f334c001180 con 0x7f33581b0470 2021-02-10T11:17:50.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.498 7f334bfff700 1 -- 172.21.2.101:0/562120259 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3344002fe0 con 0x7f33581b0470 2021-02-10T11:17:50.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.498 7f335dfe3700 1 --2- 172.21.2.101:0/562120259 >> v2:172.21.2.108:3300/0 conn(0x7f33580f05f0 0x7f33580f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:50.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.498 7f334bfff700 1 -- 172.21.2.101:0/562120259 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1753439862 0 0) 0x7f334c001180 con 0x7f33581b0470 2021-02-10T11:17:50.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.498 7f334bfff700 1 -- 172.21.2.101:0/562120259 >> v2:172.21.2.108:3300/0 conn(0x7f33580f05f0 msgr2=0x7f33580f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:50.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.498 7f334bfff700 1 --2- 172.21.2.101:0/562120259 >> v2:172.21.2.108:3300/0 conn(0x7f33580f05f0 0x7f33580f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:50.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.498 7f334bfff700 1 -- 172.21.2.101:0/562120259 >> v1:172.21.2.108:6789/0 conn(0x7f33581b0870 legacy=0x7f33580efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:50.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.498 7f334bfff700 1 -- 172.21.2.101:0/562120259 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33580e52e0 con 0x7f33581b0470 2021-02-10T11:17:50.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.498 7f334bfff700 1 -- 172.21.2.101:0/562120259 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f334c001680 con 0x7f33581b0470 2021-02-10T11:17:50.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.498 7f334bfff700 1 -- 172.21.2.101:0/562120259 >> v1:172.21.2.101:6789/0 conn(0x7f33581b0470 legacy=0x7f33580ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:50.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.498 7f334bfff700 1 --2- 172.21.2.101:0/562120259 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3344003aa0 0x7f3344005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:50.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.498 7f334bfff700 1 --2- 172.21.2.101:0/562120259 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3344006780 0x7f3344008ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:50.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.498 7f334bfff700 1 --2- 172.21.2.101:0/562120259 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3344009580 0x7f334400baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:50.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.498 7f334bfff700 1 -- 172.21.2.101:0/562120259 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334400c2f0 con 0x7f3344009580 2021-02-10T11:17:50.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.498 7f334bfff700 1 -- 172.21.2.101:0/562120259 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334400c510 con 0x7f3344003aa0 2021-02-10T11:17:50.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.498 7f334bfff700 1 -- 172.21.2.101:0/562120259 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3344050740 con 0x7f3344006780 2021-02-10T11:17:50.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.499 7f335dfe3700 1 --2- 172.21.2.101:0/562120259 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3344009580 0x7f334400baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:50.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.499 7f335d7e2700 1 --2- 172.21.2.101:0/562120259 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3344003aa0 0x7f3344005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:50.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.499 7f335cfe1700 1 --2- 172.21.2.101:0/562120259 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3344006780 0x7f3344008ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:50.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.499 7f335dfe3700 1 -- 172.21.2.101:0/562120259 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3344006780 msgr2=0x7f3344008ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:50.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.499 7f335dfe3700 1 --2- 172.21.2.101:0/562120259 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3344006780 0x7f3344008ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:50.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.499 7f335dfe3700 1 -- 172.21.2.101:0/562120259 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3344003aa0 msgr2=0x7f3344005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:50.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.499 7f335dfe3700 1 --2- 172.21.2.101:0/562120259 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3344003aa0 0x7f3344005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:50.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.499 7f335dfe3700 1 -- 172.21.2.101:0/562120259 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3344050970 con 0x7f3344009580 2021-02-10T11:17:50.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.499 7f335dfe3700 1 -- 172.21.2.101:0/562120259 reap_dead start 2021-02-10T11:17:50.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.499 7f335dfe3700 1 --2- 172.21.2.101:0/562120259 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3344009580 0x7f334400baa0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f3350006a40 tx=0x7f335000c0c0).ready entity=mon.0 client_cookie=cd8569a82da4e5dd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:50.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.499 7f334bfff700 1 -- 172.21.2.101:0/562120259 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3350006b00 con 0x7f3344009580 2021-02-10T11:17:50.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.499 7f334bfff700 1 -- 172.21.2.101:0/562120259 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3350007070 con 0x7f3344009580 2021-02-10T11:17:50.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.499 7f334bfff700 1 -- 172.21.2.101:0/562120259 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f33500074d0 con 0x7f3344009580 2021-02-10T11:17:50.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.499 7f335f9c6700 1 -- 172.21.2.101:0/562120259 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3344009580 msgr2=0x7f334400baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:50.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.499 7f335f9c6700 1 --2- 172.21.2.101:0/562120259 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3344009580 0x7f334400baa0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f3350006a40 tx=0x7f335000c0c0).stop 2021-02-10T11:17:50.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.500 7f335f9c6700 1 -- 172.21.2.101:0/562120259 shutdown_connections 2021-02-10T11:17:50.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.500 7f335f9c6700 1 --2- 172.21.2.101:0/562120259 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3344009580 0x7f334400baa0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:50.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.500 7f335f9c6700 1 -- 172.21.2.101:0/562120259 >> 172.21.2.101:0/562120259 conn(0x7f33580ce3e0 msgr2=0x7f33580e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:50.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.500 7f335f9c6700 1 -- 172.21.2.101:0/562120259 shutdown_connections 2021-02-10T11:17:50.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.500 7f335f9c6700 1 -- 172.21.2.101:0/562120259 wait complete. 2021-02-10T11:17:50.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.515 7f335f9c6700 1 Processor -- start 2021-02-10T11:17:50.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.515 7f335f9c6700 1 -- start start 2021-02-10T11:17:50.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.515 7f335f9c6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f335809b930 0x7f33580b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:50.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.515 7f335f9c6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33580b8410 0x7f33580b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:50.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.515 7f335f9c6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33580c1f80 0x7f33580c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:50.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.515 7f335f9c6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33580c1a00 con 0x7f33580b8410 2021-02-10T11:17:50.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.515 7f335f9c6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33580b8e30 con 0x7f335809b930 2021-02-10T11:17:50.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.515 7f335f9c6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33580b9050 con 0x7f33580c1f80 2021-02-10T11:17:50.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.515 7f335cfe1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33580b8410 0x7f33580b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:50.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.515 7f335cfe1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33580b8410 0x7f33580b8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46174/0 (socket says 172.21.2.101:46174) 2021-02-10T11:17:50.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.515 7f335cfe1700 1 -- 172.21.2.101:0/1842535194 learned_addr learned my addr 172.21.2.101:0/1842535194 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:17:50.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.516 7f335cfe1700 1 -- 172.21.2.101:0/1842535194 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33580c1f80 msgr2=0x7f33580c2380 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:17:50.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.516 7f335cfe1700 1 --2- 172.21.2.101:0/1842535194 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33580c1f80 0x7f33580c2380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:50.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.516 7f335cfe1700 1 -- 172.21.2.101:0/1842535194 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f335809b930 msgr2=0x7f33580b7c20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:17:50.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.516 7f335cfe1700 1 --2- 172.21.2.101:0/1842535194 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f335809b930 0x7f33580b7c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:50.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.516 7f335cfe1700 1 -- 172.21.2.101:0/1842535194 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33580c29a0 con 0x7f33580b8410 2021-02-10T11:17:50.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.516 7f335cfe1700 1 --2- 172.21.2.101:0/1842535194 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33580b8410 0x7f33580b8810 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f3354006f10 tx=0x7f335400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:50.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.516 7f3349ffb700 1 -- 172.21.2.101:0/1842535194 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f335400bd80 con 0x7f33580b8410 2021-02-10T11:17:50.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.516 7f3349ffb700 1 -- 172.21.2.101:0/1842535194 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3354006030 con 0x7f33580b8410 2021-02-10T11:17:50.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.516 7f3349ffb700 1 -- 172.21.2.101:0/1842535194 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3354002020 con 0x7f33580b8410 2021-02-10T11:17:50.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.516 7f335f9c6700 1 -- 172.21.2.101:0/1842535194 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33580c29a0 con 0x7f33580b8410 2021-02-10T11:17:50.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.516 7f335f9c6700 1 -- 172.21.2.101:0/1842535194 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33580c2f30 con 0x7f33580b8410 2021-02-10T11:17:50.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.517 7f3349ffb700 1 -- 172.21.2.101:0/1842535194 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3354006210 con 0x7f33580b8410 2021-02-10T11:17:50.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.517 7f3349ffb700 1 --2- 172.21.2.101:0/1842535194 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f33380186c0 0x7f333801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:50.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.517 7f3349ffb700 1 -- 172.21.2.101:0/1842535194 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3354032570 con 0x7f33580b8410 2021-02-10T11:17:50.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.517 7f335d7e2700 1 --2- 172.21.2.101:0/1842535194 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f33380186c0 0x7f333801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:50.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.517 7f335d7e2700 1 --2- 172.21.2.101:0/1842535194 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f33380186c0 0x7f333801abc0 crc :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:50.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.547 7f335f9c6700 1 -- 172.21.2.101:0/1842535194 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3358105b30 con 0x7f33580b8410 2021-02-10T11:17:50.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.550 7f3349ffb700 1 -- 172.21.2.101:0/1842535194 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3354000940 con 0x7f33580b8410 2021-02-10T11:17:50.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.679 7f335f9c6700 1 -- 172.21.2.101:0/1842535194 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839255", "target": ["mgr", ""]}) v1 -- 0x7f33580c2f30 con 0x7f33380186c0 2021-02-10T11:17:50.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.682 7f3349ffb700 1 -- 172.21.2.101:0/1842535194 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f334c000a00 con 0x7f33380186c0 2021-02-10T11:17:50.688 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839255/ba3be31b-25cc-4b69-a790-f7b70a4bddf6 2021-02-10T11:17:50.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.684 7f335f9c6700 1 -- 172.21.2.101:0/1842535194 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f33380186c0 msgr2=0x7f333801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:50.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.684 7f335f9c6700 1 --2- 172.21.2.101:0/1842535194 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f33380186c0 0x7f333801abc0 crc :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:50.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.684 7f335f9c6700 1 -- 172.21.2.101:0/1842535194 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33580b8410 msgr2=0x7f33580b8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:50.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.684 7f335f9c6700 1 --2- 172.21.2.101:0/1842535194 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33580b8410 0x7f33580b8810 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f3354006f10 tx=0x7f335400b990).stop 2021-02-10T11:17:50.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.684 7f335f9c6700 1 -- 172.21.2.101:0/1842535194 shutdown_connections 2021-02-10T11:17:50.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.684 7f335f9c6700 1 --2- 172.21.2.101:0/1842535194 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f33380186c0 0x7f333801abc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:50.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.684 7f335f9c6700 1 --2- 172.21.2.101:0/1842535194 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33580c1f80 0x7f33580c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:50.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.684 7f335f9c6700 1 --2- 172.21.2.101:0/1842535194 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33580b8410 0x7f33580b8810 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:50.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.684 7f335f9c6700 1 --2- 172.21.2.101:0/1842535194 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f335809b930 0x7f33580b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:50.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.684 7f335f9c6700 1 -- 172.21.2.101:0/1842535194 >> 172.21.2.101:0/1842535194 conn(0x7f3358094e10 msgr2=0x7f33580a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:50.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.684 7f335f9c6700 1 -- 172.21.2.101:0/1842535194 shutdown_connections 2021-02-10T11:17:50.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:50.684 7f335f9c6700 1 -- 172.21.2.101:0/1842535194 wait complete. 2021-02-10T11:17:50.698 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839255 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839255/ba3be31b-25cc-4b69-a790-f7b70a4bddf6 2021-02-10T11:17:50.698 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:50.698 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:50.698 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:51.033 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:51.034 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:51.034 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839255/ba3be31b-25cc-4b69-a790-f7b70a4bddf6/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:51.414 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:51.414 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:51.414 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240693 s, 43.6 MB/s 2021-02-10T11:17:51.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:51.416 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:51.416 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:51.749 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:51.750 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:51.750 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839255/ba3be31b-25cc-4b69-a790-f7b70a4bddf6/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:52.122 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:52.231 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:52.231 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0222675 s, 47.1 MB/s 2021-02-10T11:17:52.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:52.232 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:52.232 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:52.566 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:52.566 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:52.567 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839255/ba3be31b-25cc-4b69-a790-f7b70a4bddf6/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:52.939 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:52.939 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:52.939 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0230843 s, 45.4 MB/s 2021-02-10T11:17:52.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:52.941 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:52.941 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:53.275 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:53.275 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:53.276 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839255/ba3be31b-25cc-4b69-a790-f7b70a4bddf6/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:53.650 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:53.650 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:53.650 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024127 s, 43.5 MB/s 2021-02-10T11:17:53.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:53.652 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:53.652 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:53.987 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:53.988 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:53.988 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839255/ba3be31b-25cc-4b69-a790-f7b70a4bddf6/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:54.361 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:54.361 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:54.361 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246352 s, 42.6 MB/s 2021-02-10T11:17:54.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:54.362 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:54.362 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:54.700 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:54.701 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:54.701 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839255/ba3be31b-25cc-4b69-a790-f7b70a4bddf6/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:55.090 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:55.090 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:55.090 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240224 s, 43.6 MB/s 2021-02-10T11:17:55.091 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:55.091 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:55.091 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:55.428 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:55.428 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:55.428 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839255/ba3be31b-25cc-4b69-a790-f7b70a4bddf6/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:55.803 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:55.804 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:55.804 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0232452 s, 45.1 MB/s 2021-02-10T11:17:55.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:55.805 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:55.805 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:56.138 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:56.139 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:56.139 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839255/ba3be31b-25cc-4b69-a790-f7b70a4bddf6/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:56.513 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:56.513 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:56.513 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239113 s, 43.9 MB/s 2021-02-10T11:17:56.514 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:56.514 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:56.514 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:56.854 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:56.854 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:56.855 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839255/ba3be31b-25cc-4b69-a790-f7b70a4bddf6/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:57.229 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:57.229 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:57.229 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023666 s, 44.3 MB/s 2021-02-10T11:17:57.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:57.231 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:57.231 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:57.564 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:57.564 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:57.565 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839255/ba3be31b-25cc-4b69-a790-f7b70a4bddf6/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:57.942 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:57.942 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:57.942 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0265044 s, 39.6 MB/s 2021-02-10T11:17:57.943 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839256 2021-02-10T11:17:58.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.349 7f84b1482700 1 Processor -- start 2021-02-10T11:17:58.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.349 7f84b1482700 1 -- start start 2021-02-10T11:17:58.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.350 7f84b1482700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f84ac1b0870 0x7f84ac0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:58.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.350 7f84b1482700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84ac1b0c70 con 0x7f84ac1b0470 2021-02-10T11:17:58.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.350 7f84b1482700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84ac0f0b50 con 0x7f84ac0efe00 2021-02-10T11:17:58.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.350 7f84b1482700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84ac0e5ea0 con 0x7f84ac1b0870 2021-02-10T11:17:58.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.350 7f84ab07d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f84ac1b0470 0x7f84ac0ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46672/0 (socket says 172.21.2.101:46672) 2021-02-10T11:17:58.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.350 7f84ab07d700 1 -- 172.21.2.101:0/3604071526 learned_addr learned my addr 172.21.2.101:0/3604071526 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:17:58.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.350 7f84aa07b700 1 -- 172.21.2.101:0/3604071526 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 892153532 0 0) 0x7f84940010b0 con 0x7f84ac1b0470 2021-02-10T11:17:58.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.350 7f84aa07b700 1 -- 172.21.2.101:0/3604071526 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8498002fe0 con 0x7f84ac1b0470 2021-02-10T11:17:58.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.350 7f84aa87c700 1 --2- 172.21.2.101:0/3604071526 >> v2:172.21.2.108:3300/0 conn(0x7f84ac1b0870 0x7f84ac0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:58.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.350 7f84aa07b700 1 -- 172.21.2.101:0/3604071526 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3364407273 0 0) 0x7f8494001010 con 0x7f84ac1b0470 2021-02-10T11:17:58.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.350 7f84aa07b700 1 -- 172.21.2.101:0/3604071526 >> v2:172.21.2.108:3300/0 conn(0x7f84ac1b0870 msgr2=0x7f84ac0f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:58.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.350 7f84aa07b700 1 --2- 172.21.2.101:0/3604071526 >> v2:172.21.2.108:3300/0 conn(0x7f84ac1b0870 0x7f84ac0f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:58.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.350 7f84aa07b700 1 -- 172.21.2.101:0/3604071526 >> v1:172.21.2.108:6789/0 conn(0x7f84ac0efe00 legacy=0x7f84ac0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:58.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.350 7f84aa07b700 1 -- 172.21.2.101:0/3604071526 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ac0e52e0 con 0x7f84ac1b0470 2021-02-10T11:17:58.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84aa07b700 1 -- 172.21.2.101:0/3604071526 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f8494001530 con 0x7f84ac1b0470 2021-02-10T11:17:58.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84aa07b700 1 -- 172.21.2.101:0/3604071526 >> v1:172.21.2.101:6789/0 conn(0x7f84ac1b0470 legacy=0x7f84ac0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:58.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84aa07b700 1 --2- 172.21.2.101:0/3604071526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8498003a40 0x7f8498005ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:58.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84aa07b700 1 --2- 172.21.2.101:0/3604071526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8498006770 0x7f8498008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:58.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84aa07b700 1 --2- 172.21.2.101:0/3604071526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8498009590 0x7f849800bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:58.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84aa07b700 1 -- 172.21.2.101:0/3604071526 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849800c300 con 0x7f8498003a40 2021-02-10T11:17:58.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84aa07b700 1 -- 172.21.2.101:0/3604071526 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849800c520 con 0x7f8498009590 2021-02-10T11:17:58.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84aa07b700 1 -- 172.21.2.101:0/3604071526 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f849800c740 con 0x7f8498006770 2021-02-10T11:17:58.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84ab07d700 1 --2- 172.21.2.101:0/3604071526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8498003a40 0x7f8498005ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:58.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84aa87c700 1 --2- 172.21.2.101:0/3604071526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8498006770 0x7f8498008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:58.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84ab87e700 1 --2- 172.21.2.101:0/3604071526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8498009590 0x7f849800bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:58.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84ab07d700 1 -- 172.21.2.101:0/3604071526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8498006770 msgr2=0x7f8498008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:58.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84ab07d700 1 --2- 172.21.2.101:0/3604071526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8498006770 0x7f8498008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:58.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84ab07d700 1 -- 172.21.2.101:0/3604071526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8498009590 msgr2=0x7f849800bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:17:58.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84ab07d700 1 --2- 172.21.2.101:0/3604071526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8498009590 0x7f849800bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:58.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84ab07d700 1 -- 172.21.2.101:0/3604071526 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849800c9e0 con 0x7f8498003a40 2021-02-10T11:17:58.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84ab87e700 1 --2- 172.21.2.101:0/3604071526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8498009590 0x7f849800bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:17:58.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84ab87e700 1 -- 172.21.2.101:0/3604071526 reap_dead start 2021-02-10T11:17:58.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84ab07d700 1 --2- 172.21.2.101:0/3604071526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8498003a40 0x7f8498005ed0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f84940014a0 tx=0x7f849404bd70).ready entity=mon.0 client_cookie=d6860b0d1a6a454b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:58.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84aa07b700 1 -- 172.21.2.101:0/3604071526 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8494000940 con 0x7f8498003a40 2021-02-10T11:17:58.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84aa07b700 1 -- 172.21.2.101:0/3604071526 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84940014e0 con 0x7f8498003a40 2021-02-10T11:17:58.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84aa07b700 1 -- 172.21.2.101:0/3604071526 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f849404e500 con 0x7f8498003a40 2021-02-10T11:17:58.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84b1482700 1 -- 172.21.2.101:0/3604071526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8498003a40 msgr2=0x7f8498005ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:58.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.351 7f84b1482700 1 --2- 172.21.2.101:0/3604071526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8498003a40 0x7f8498005ed0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f84940014a0 tx=0x7f849404bd70).stop 2021-02-10T11:17:58.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.352 7f84b1482700 1 -- 172.21.2.101:0/3604071526 shutdown_connections 2021-02-10T11:17:58.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.352 7f84b1482700 1 --2- 172.21.2.101:0/3604071526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8498003a40 0x7f8498005ed0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:58.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.352 7f84b1482700 1 -- 172.21.2.101:0/3604071526 >> 172.21.2.101:0/3604071526 conn(0x7f84ac0ce3e0 msgr2=0x7f84ac0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:58.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.352 7f84b1482700 1 -- 172.21.2.101:0/3604071526 shutdown_connections 2021-02-10T11:17:58.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.352 7f84b1482700 1 -- 172.21.2.101:0/3604071526 wait complete. 2021-02-10T11:17:58.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.367 7f84b1482700 1 Processor -- start 2021-02-10T11:17:58.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.367 7f84b1482700 1 -- start start 2021-02-10T11:17:58.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.367 7f84b1482700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84ac0b7db0 0x7f84ac0b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:58.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.367 7f84b1482700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f84ac0b8880 0x7f84ac1a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:58.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.367 7f84b1482700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f84ac0b8c80 0x7f84ac1a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:58.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.367 7f84b1482700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac0b79b0 con 0x7f84ac0b7db0 2021-02-10T11:17:58.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.367 7f84b1482700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac1a8090 con 0x7f84ac0b8880 2021-02-10T11:17:58.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.367 7f84b1482700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac1a82b0 con 0x7f84ac0b8c80 2021-02-10T11:17:58.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.367 7f84ab07d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84ac0b7db0 0x7f84ac0b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:58.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.367 7f84ab07d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84ac0b7db0 0x7f84ac0b8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46194/0 (socket says 172.21.2.101:46194) 2021-02-10T11:17:58.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.367 7f84ab07d700 1 -- 172.21.2.101:0/2851724754 learned_addr learned my addr 172.21.2.101:0/2851724754 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:17:58.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.368 7f84ab07d700 1 -- 172.21.2.101:0/2851724754 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f84ac0b8c80 msgr2=0x7f84ac1a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:58.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.368 7f84ab07d700 1 --2- 172.21.2.101:0/2851724754 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f84ac0b8c80 0x7f84ac1a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:58.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.368 7f84ab07d700 1 -- 172.21.2.101:0/2851724754 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f84ac0b8880 msgr2=0x7f84ac1a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:58.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.368 7f84ab07d700 1 --2- 172.21.2.101:0/2851724754 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f84ac0b8880 0x7f84ac1a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:58.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.368 7f84ab07d700 1 -- 172.21.2.101:0/2851724754 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ac190c10 con 0x7f84ac0b7db0 2021-02-10T11:17:58.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.368 7f84ab07d700 1 --2- 172.21.2.101:0/2851724754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84ac0b7db0 0x7f84ac0b8260 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f8494001170 tx=0x7f849404e9a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:58.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.368 7f84a3fff700 1 -- 172.21.2.101:0/2851724754 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f849404b2e0 con 0x7f84ac0b7db0 2021-02-10T11:17:58.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.368 7f84a3fff700 1 -- 172.21.2.101:0/2851724754 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849404ed90 con 0x7f84ac0b7db0 2021-02-10T11:17:58.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.368 7f84a3fff700 1 -- 172.21.2.101:0/2851724754 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f84940043c0 con 0x7f84ac0b7db0 2021-02-10T11:17:58.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.368 7f84b1482700 1 -- 172.21.2.101:0/2851724754 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84ac190c10 con 0x7f84ac0b7db0 2021-02-10T11:17:58.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.368 7f84b1482700 1 -- 172.21.2.101:0/2851724754 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84ac0c1eb0 con 0x7f84ac0b7db0 2021-02-10T11:17:58.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.369 7f84a3fff700 1 -- 172.21.2.101:0/2851724754 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8494004140 con 0x7f84ac0b7db0 2021-02-10T11:17:58.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.369 7f84a3fff700 1 --2- 172.21.2.101:0/2851724754 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84900186c0 0x7f849001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:58.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.369 7f84a3fff700 1 -- 172.21.2.101:0/2851724754 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8494076400 con 0x7f84ac0b7db0 2021-02-10T11:17:58.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.369 7f84aa87c700 1 --2- 172.21.2.101:0/2851724754 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84900186c0 0x7f849001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:58.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.369 7f84aa87c700 1 --2- 172.21.2.101:0/2851724754 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84900186c0 0x7f849001abc0 crc :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:58.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.400 7f84b1482700 1 -- 172.21.2.101:0/2851724754 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84ac105b30 con 0x7f84ac0b7db0 2021-02-10T11:17:58.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.402 7f84a3fff700 1 -- 172.21.2.101:0/2851724754 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8494074d30 con 0x7f84ac0b7db0 2021-02-10T11:17:58.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.527 7f84b1482700 1 -- 172.21.2.101:0/2851724754 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839256", "target": ["mgr", ""]}) v1 -- 0x7f84ac0c1eb0 con 0x7f84900186c0 2021-02-10T11:17:58.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.552 7f84a3fff700 1 -- 172.21.2.101:0/2851724754 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f849c000c90 con 0x7f84900186c0 2021-02-10T11:17:58.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.560 7f84b1482700 1 -- 172.21.2.101:0/2851724754 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84900186c0 msgr2=0x7f849001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:58.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.560 7f84b1482700 1 --2- 172.21.2.101:0/2851724754 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84900186c0 0x7f849001abc0 crc :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:58.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.560 7f84b1482700 1 -- 172.21.2.101:0/2851724754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84ac0b7db0 msgr2=0x7f84ac0b8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:58.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.560 7f84b1482700 1 --2- 172.21.2.101:0/2851724754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84ac0b7db0 0x7f84ac0b8260 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f8494001170 tx=0x7f849404e9a0).stop 2021-02-10T11:17:58.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.560 7f84b1482700 1 -- 172.21.2.101:0/2851724754 shutdown_connections 2021-02-10T11:17:58.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.560 7f84b1482700 1 --2- 172.21.2.101:0/2851724754 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84900186c0 0x7f849001abc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:58.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.560 7f84b1482700 1 --2- 172.21.2.101:0/2851724754 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f84ac0b8c80 0x7f84ac1a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:58.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.560 7f84b1482700 1 --2- 172.21.2.101:0/2851724754 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f84ac0b8880 0x7f84ac1a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:58.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.560 7f84b1482700 1 --2- 172.21.2.101:0/2851724754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84ac0b7db0 0x7f84ac0b8260 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:58.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.560 7f84b1482700 1 -- 172.21.2.101:0/2851724754 >> 172.21.2.101:0/2851724754 conn(0x7f84ac094e10 msgr2=0x7f84ac0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:58.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.561 7f84b1482700 1 -- 172.21.2.101:0/2851724754 shutdown_connections 2021-02-10T11:17:58.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.561 7f84b1482700 1 -- 172.21.2.101:0/2851724754 wait complete. 2021-02-10T11:17:58.574 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839256 2021-02-10T11:17:59.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.983 7fc70fbf3700 1 Processor -- start 2021-02-10T11:17:59.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.983 7fc70fbf3700 1 -- start start 2021-02-10T11:17:59.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.983 7fc70fbf3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc7081b0870 0x7fc7080f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:59.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.984 7fc70fbf3700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7081b0c70 con 0x7fc7080efe00 2021-02-10T11:17:59.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.984 7fc70fbf3700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7080f0b50 con 0x7fc7081b0470 2021-02-10T11:17:59.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.984 7fc70fbf3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7080e5ea0 con 0x7fc7081b0870 2021-02-10T11:17:59.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.984 7fc70e210700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc7080efe00 0x7fc7080f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46692/0 (socket says 172.21.2.101:46692) 2021-02-10T11:17:59.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.984 7fc70e210700 1 -- 172.21.2.101:0/1521402757 learned_addr learned my addr 172.21.2.101:0/1521402757 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:17:59.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.984 7fc70d20e700 1 --2- 172.21.2.101:0/1521402757 >> v2:172.21.2.108:3300/0 conn(0x7fc7081b0870 0x7fc7080f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:59.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.984 7fc70ca0d700 1 -- 172.21.2.101:0/1521402757 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203459131 0 0) 0x7fc700000f90 con 0x7fc7080efe00 2021-02-10T11:17:59.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.984 7fc70ca0d700 1 -- 172.21.2.101:0/1521402757 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6f4002fe0 con 0x7fc7080efe00 2021-02-10T11:17:59.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.984 7fc70ca0d700 1 -- 172.21.2.101:0/1521402757 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1442916793 0 0) 0x7fc704000c60 con 0x7fc7081b0470 2021-02-10T11:17:59.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.984 7fc70ca0d700 1 -- 172.21.2.101:0/1521402757 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6f40042c0 con 0x7fc7081b0470 2021-02-10T11:17:59.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.984 7fc70d20e700 1 -- 172.21.2.101:0/1521402757 >> v1:172.21.2.108:6789/0 conn(0x7fc7081b0470 legacy=0x7fc7080ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:59.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.984 7fc70d20e700 1 -- 172.21.2.101:0/1521402757 >> v1:172.21.2.101:6789/0 conn(0x7fc7080efe00 legacy=0x7fc7080f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:59.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.984 7fc70d20e700 1 -- 172.21.2.101:0/1521402757 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7080e52e0 con 0x7fc7081b0870 2021-02-10T11:17:59.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.985 7fc70d20e700 1 --2- 172.21.2.101:0/1521402757 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7081b0870 0x7fc7080f20e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fc6f8006650 tx=0x7fc6f8009a80).ready entity=mon.1 client_cookie=78dade3a6efdf09f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:59.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.985 7fc70ca0d700 1 -- 172.21.2.101:0/1521402757 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc6f80009b0 con 0x7fc7081b0870 2021-02-10T11:17:59.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.985 7fc70ca0d700 1 -- 172.21.2.101:0/1521402757 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f800c0c0 con 0x7fc7081b0870 2021-02-10T11:17:59.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.985 7fc70ca0d700 1 -- 172.21.2.101:0/1521402757 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc6f800c5e0 con 0x7fc7081b0870 2021-02-10T11:17:59.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.985 7fc70fbf3700 1 -- 172.21.2.101:0/1521402757 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7081b0870 msgr2=0x7fc7080f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:59.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.985 7fc70fbf3700 1 --2- 172.21.2.101:0/1521402757 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7081b0870 0x7fc7080f20e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fc6f8006650 tx=0x7fc6f8009a80).stop 2021-02-10T11:17:59.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.986 7fc70fbf3700 1 -- 172.21.2.101:0/1521402757 shutdown_connections 2021-02-10T11:17:59.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.986 7fc70fbf3700 1 --2- 172.21.2.101:0/1521402757 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7081b0870 0x7fc7080f20e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:59.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.986 7fc70fbf3700 1 -- 172.21.2.101:0/1521402757 >> 172.21.2.101:0/1521402757 conn(0x7fc7080ce3e0 msgr2=0x7fc7080e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:59.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.986 7fc70fbf3700 1 -- 172.21.2.101:0/1521402757 shutdown_connections 2021-02-10T11:17:59.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:58.986 7fc70fbf3700 1 -- 172.21.2.101:0/1521402757 wait complete. 2021-02-10T11:17:59.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.001 7fc70fbf3700 1 Processor -- start 2021-02-10T11:17:59.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.001 7fc70fbf3700 1 -- start start 2021-02-10T11:17:59.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.001 7fc70fbf3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc708190920 0x7fc7080b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:59.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.001 7fc70fbf3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7080b8200 0x7fc7080b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:59.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.001 7fc70fbf3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7080c1d80 0x7fc7080c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:59.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.001 7fc70fbf3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc708190d20 con 0x7fc708190920 2021-02-10T11:17:59.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.001 7fc70fbf3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7080b8c20 con 0x7fc7080c1d80 2021-02-10T11:17:59.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.001 7fc70fbf3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7080c27a0 con 0x7fc7080b8200 2021-02-10T11:17:59.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.001 7fc70da0f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc708190920 0x7fc7080b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:59.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.001 7fc70da0f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc708190920 0x7fc7080b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46200/0 (socket says 172.21.2.101:46200) 2021-02-10T11:17:59.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.001 7fc70da0f700 1 -- 172.21.2.101:0/1745737427 learned_addr learned my addr 172.21.2.101:0/1745737427 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:17:59.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.002 7fc70da0f700 1 -- 172.21.2.101:0/1745737427 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7080b8200 msgr2=0x7fc7080b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:59.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.002 7fc70da0f700 1 --2- 172.21.2.101:0/1745737427 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7080b8200 0x7fc7080b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:59.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.002 7fc70da0f700 1 -- 172.21.2.101:0/1745737427 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7080c1d80 msgr2=0x7fc7080c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:59.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.002 7fc70da0f700 1 --2- 172.21.2.101:0/1745737427 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7080c1d80 0x7fc7080c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:59.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.002 7fc70da0f700 1 -- 172.21.2.101:0/1745737427 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7080c29c0 con 0x7fc708190920 2021-02-10T11:17:59.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.002 7fc70da0f700 1 --2- 172.21.2.101:0/1745737427 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc708190920 0x7fc7080b7be0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fc70400b580 tx=0x7fc70400b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:59.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.002 7fc6fe7fc700 1 -- 172.21.2.101:0/1745737427 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc70400bd10 con 0x7fc708190920 2021-02-10T11:17:59.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.002 7fc6fe7fc700 1 -- 172.21.2.101:0/1745737427 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc704000e20 con 0x7fc708190920 2021-02-10T11:17:59.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.002 7fc6fe7fc700 1 -- 172.21.2.101:0/1745737427 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc7040054f0 con 0x7fc708190920 2021-02-10T11:17:59.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.002 7fc70fbf3700 1 -- 172.21.2.101:0/1745737427 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7080c29c0 con 0x7fc708190920 2021-02-10T11:17:59.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.002 7fc70fbf3700 1 -- 172.21.2.101:0/1745737427 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7080c29c0 con 0x7fc708190920 2021-02-10T11:17:59.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.003 7fc6fe7fc700 1 -- 172.21.2.101:0/1745737427 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc704009fe0 con 0x7fc708190920 2021-02-10T11:17:59.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.003 7fc6fe7fc700 1 --2- 172.21.2.101:0/1745737427 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6f0018730 0x7fc6f001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:17:59.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.003 7fc6fe7fc700 1 -- 172.21.2.101:0/1745737427 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc7040327d0 con 0x7fc708190920 2021-02-10T11:17:59.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.003 7fc70d20e700 1 --2- 172.21.2.101:0/1745737427 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6f0018730 0x7fc6f001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:17:59.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.004 7fc70d20e700 1 --2- 172.21.2.101:0/1745737427 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6f0018730 0x7fc6f001ac10 crc :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:17:59.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.033 7fc70fbf3700 1 -- 172.21.2.101:0/1745737427 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc708105ae0 con 0x7fc708190920 2021-02-10T11:17:59.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.036 7fc6fe7fc700 1 -- 172.21.2.101:0/1745737427 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc704000e20 con 0x7fc708190920 2021-02-10T11:17:59.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.164 7fc70fbf3700 1 -- 172.21.2.101:0/1745737427 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839256", "target": ["mgr", ""]}) v1 -- 0x7fc7080c32c0 con 0x7fc6f0018730 2021-02-10T11:17:59.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.168 7fc6fe7fc700 1 -- 172.21.2.101:0/1745737427 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fc6f8006060 con 0x7fc6f0018730 2021-02-10T11:17:59.177 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839256/bfc9f6e6-12f2-498a-9e8a-0baf3aee4d2a 2021-02-10T11:17:59.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.173 7fc70fbf3700 1 -- 172.21.2.101:0/1745737427 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6f0018730 msgr2=0x7fc6f001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:59.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.173 7fc70fbf3700 1 --2- 172.21.2.101:0/1745737427 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6f0018730 0x7fc6f001ac10 crc :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:59.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.173 7fc70fbf3700 1 -- 172.21.2.101:0/1745737427 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc708190920 msgr2=0x7fc7080b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:17:59.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.173 7fc70fbf3700 1 --2- 172.21.2.101:0/1745737427 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc708190920 0x7fc7080b7be0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fc70400b580 tx=0x7fc70400b920).stop 2021-02-10T11:17:59.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.173 7fc70fbf3700 1 -- 172.21.2.101:0/1745737427 shutdown_connections 2021-02-10T11:17:59.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.173 7fc70fbf3700 1 --2- 172.21.2.101:0/1745737427 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6f0018730 0x7fc6f001ac10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:59.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.173 7fc70fbf3700 1 --2- 172.21.2.101:0/1745737427 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7080b8200 0x7fc7080b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:59.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.173 7fc70fbf3700 1 --2- 172.21.2.101:0/1745737427 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7080c1d80 0x7fc7080c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:17:59.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.173 7fc70fbf3700 1 --2- 172.21.2.101:0/1745737427 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc708190920 0x7fc7080b7be0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:17:59.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.173 7fc70fbf3700 1 -- 172.21.2.101:0/1745737427 >> 172.21.2.101:0/1745737427 conn(0x7fc708094e10 msgr2=0x7fc7080a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:17:59.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.173 7fc70fbf3700 1 -- 172.21.2.101:0/1745737427 shutdown_connections 2021-02-10T11:17:59.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:17:59.173 7fc70fbf3700 1 -- 172.21.2.101:0/1745737427 wait complete. 2021-02-10T11:17:59.187 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839256 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839256/bfc9f6e6-12f2-498a-9e8a-0baf3aee4d2a 2021-02-10T11:17:59.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:59.187 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:59.187 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:17:59.522 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:17:59.523 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:17:59.523 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839256/bfc9f6e6-12f2-498a-9e8a-0baf3aee4d2a/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:17:59.904 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:17:59.904 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:17:59.904 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0256318 s, 40.9 MB/s 2021-02-10T11:17:59.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:17:59.905 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:17:59.906 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:00.240 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:00.241 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:00.241 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839256/bfc9f6e6-12f2-498a-9e8a-0baf3aee4d2a/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:00.619 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:00.619 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:00.619 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251211 s, 41.7 MB/s 2021-02-10T11:18:00.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:00.619 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:00.620 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:00.954 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:00.954 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:00.954 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839256/bfc9f6e6-12f2-498a-9e8a-0baf3aee4d2a/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:01.330 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:01.330 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:01.330 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0259917 s, 40.3 MB/s 2021-02-10T11:18:01.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:01.331 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:01.331 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:01.666 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:01.667 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:01.667 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839256/bfc9f6e6-12f2-498a-9e8a-0baf3aee4d2a/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:02.042 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:02.042 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:02.042 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0255345 s, 41.1 MB/s 2021-02-10T11:18:02.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:02.043 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:02.044 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:02.378 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:02.378 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:02.379 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839256/bfc9f6e6-12f2-498a-9e8a-0baf3aee4d2a/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:02.754 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:02.754 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:02.754 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240049 s, 43.7 MB/s 2021-02-10T11:18:02.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:02.756 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:02.756 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:03.091 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:03.092 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:03.092 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839256/bfc9f6e6-12f2-498a-9e8a-0baf3aee4d2a/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:03.469 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:03.469 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:03.469 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0255062 s, 41.1 MB/s 2021-02-10T11:18:03.470 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:03.470 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:03.471 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:03.804 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:03.804 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:03.805 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839256/bfc9f6e6-12f2-498a-9e8a-0baf3aee4d2a/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:04.179 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:04.179 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:04.179 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0260915 s, 40.2 MB/s 2021-02-10T11:18:04.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:04.180 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:04.180 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:04.515 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:04.516 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:04.516 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839256/bfc9f6e6-12f2-498a-9e8a-0baf3aee4d2a/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:04.890 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:04.890 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:04.891 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243776 s, 43.0 MB/s 2021-02-10T11:18:04.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:04.892 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:04.892 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:05.226 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:05.226 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:05.227 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839256/bfc9f6e6-12f2-498a-9e8a-0baf3aee4d2a/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:05.601 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:05.601 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:05.601 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233722 s, 44.9 MB/s 2021-02-10T11:18:05.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:05.602 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:05.602 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:05.936 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:05.937 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:05.937 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839256/bfc9f6e6-12f2-498a-9e8a-0baf3aee4d2a/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:06.312 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:06.312 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:06.312 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238509 s, 44.0 MB/s 2021-02-10T11:18:06.313 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839257 2021-02-10T11:18:06.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.719 7f147f920700 1 Processor -- start 2021-02-10T11:18:06.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.719 7f147f920700 1 -- start start 2021-02-10T11:18:06.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.720 7f147f920700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f14781b0470 0x7f14781b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:06.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.720 7f147f920700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14780ef610 0x7f14780efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:06.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.720 7f147f920700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14780f0b50 con 0x7f14780f0030 2021-02-10T11:18:06.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.720 7f147f920700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14780e52e0 con 0x7f14781b0470 2021-02-10T11:18:06.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.720 7f147f920700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14780e54d0 con 0x7f14780ef610 2021-02-10T11:18:06.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.720 7f147d73c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f14781b0470 0x7f14781b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:06.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.720 7f147cf3b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14780ef610 0x7f14780efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:06.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.720 7f147d73c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f14781b0470 0x7f14781b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46208/0 (socket says 172.21.2.101:46208) 2021-02-10T11:18:06.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.720 7f147d73c700 1 -- 172.21.2.101:0/3228235248 learned_addr learned my addr 172.21.2.101:0/3228235248 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:18:06.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.721 7f147cf3b700 1 -- 172.21.2.101:0/3228235248 >> v2:172.21.2.101:3300/0 conn(0x7f14781b0470 msgr2=0x7f14781b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:18:06.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.721 7f147cf3b700 1 --2- 172.21.2.101:0/3228235248 >> v2:172.21.2.101:3300/0 conn(0x7f14781b0470 0x7f14781b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:06.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.721 7f147cf3b700 1 -- 172.21.2.101:0/3228235248 >> v1:172.21.2.108:6789/0 conn(0x7f14780f0030 legacy=0x7f14780f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:06.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.721 7f146bfff700 1 -- 172.21.2.101:0/3228235248 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4085561626 0 0) 0x7f1470001400 con 0x7f14780f0030 2021-02-10T11:18:06.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.721 7f147cf3b700 1 -- 172.21.2.101:0/3228235248 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14780f0430 con 0x7f14780ef610 2021-02-10T11:18:06.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.721 7f147d73c700 1 --2- 172.21.2.101:0/3228235248 >> v2:172.21.2.101:3300/0 conn(0x7f14781b0470 0x7f14781b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:18:06.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.721 7f147cf3b700 1 --2- 172.21.2.101:0/3228235248 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14780ef610 0x7f14780efa10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f146c006650 tx=0x7f146c009a80).ready entity=mon.2 client_cookie=5bd569e4c0e22ffe server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:06.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.721 7f146bfff700 1 -- 172.21.2.101:0/3228235248 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f146c00c0c0 con 0x7f14780ef610 2021-02-10T11:18:06.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.721 7f146bfff700 1 -- 172.21.2.101:0/3228235248 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146c00c340 con 0x7f14780ef610 2021-02-10T11:18:06.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.721 7f146bfff700 1 -- 172.21.2.101:0/3228235248 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f146c00c860 con 0x7f14780ef610 2021-02-10T11:18:06.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.721 7f147f920700 1 -- 172.21.2.101:0/3228235248 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14780ef610 msgr2=0x7f14780efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:06.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.721 7f147f920700 1 --2- 172.21.2.101:0/3228235248 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14780ef610 0x7f14780efa10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f146c006650 tx=0x7f146c009a80).stop 2021-02-10T11:18:06.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.722 7f147f920700 1 -- 172.21.2.101:0/3228235248 shutdown_connections 2021-02-10T11:18:06.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.722 7f147f920700 1 --2- 172.21.2.101:0/3228235248 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14780ef610 0x7f14780efa10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:06.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.722 7f147f920700 1 --2- 172.21.2.101:0/3228235248 >> v2:172.21.2.101:3300/0 conn(0x7f14781b0470 0x7f14781b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:06.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.722 7f147f920700 1 -- 172.21.2.101:0/3228235248 >> 172.21.2.101:0/3228235248 conn(0x7f14780ce3e0 msgr2=0x7f14780e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:06.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.722 7f147f920700 1 -- 172.21.2.101:0/3228235248 shutdown_connections 2021-02-10T11:18:06.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.722 7f147f920700 1 -- 172.21.2.101:0/3228235248 wait complete. 2021-02-10T11:18:06.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.736 7f147f920700 1 Processor -- start 2021-02-10T11:18:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.736 7f147f920700 1 -- start start 2021-02-10T11:18:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.737 7f147f920700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1478190920 0x7f14780b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.737 7f147f920700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14780b8200 0x7f14780b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.737 7f147f920700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14780c1d80 0x7f14780c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.737 7f147f920700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1478190da0 con 0x7f14780b8200 2021-02-10T11:18:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.737 7f147f920700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14780b8c20 con 0x7f1478190920 2021-02-10T11:18:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.737 7f147f920700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14780b8e10 con 0x7f14780c1d80 2021-02-10T11:18:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.737 7f147cf3b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14780b8200 0x7f14780b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.737 7f147cf3b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14780b8200 0x7f14780b8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46216/0 (socket says 172.21.2.101:46216) 2021-02-10T11:18:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.737 7f147cf3b700 1 -- 172.21.2.101:0/2003837124 learned_addr learned my addr 172.21.2.101:0/2003837124 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:18:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.737 7f147df3d700 1 --2- 172.21.2.101:0/2003837124 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14780c1d80 0x7f14780c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.737 7f147d73c700 1 --2- 172.21.2.101:0/2003837124 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1478190920 0x7f14780b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.738 7f147cf3b700 1 -- 172.21.2.101:0/2003837124 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14780c1d80 msgr2=0x7f14780c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:06.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.738 7f147cf3b700 1 --2- 172.21.2.101:0/2003837124 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14780c1d80 0x7f14780c2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:06.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.738 7f147cf3b700 1 -- 172.21.2.101:0/2003837124 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1478190920 msgr2=0x7f14780b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:06.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.738 7f147cf3b700 1 --2- 172.21.2.101:0/2003837124 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1478190920 0x7f14780b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:06.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.738 7f147cf3b700 1 -- 172.21.2.101:0/2003837124 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14780c27d0 con 0x7f14780b8200 2021-02-10T11:18:06.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.738 7f147d73c700 1 --2- 172.21.2.101:0/2003837124 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1478190920 0x7f14780b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:18:06.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.738 7f147cf3b700 1 --2- 172.21.2.101:0/2003837124 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14780b8200 0x7f14780b8600 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f146c009650 tx=0x7f146c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:06.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.738 7f1469ffb700 1 -- 172.21.2.101:0/2003837124 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f146c0093a0 con 0x7f14780b8200 2021-02-10T11:18:06.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.738 7f1469ffb700 1 -- 172.21.2.101:0/2003837124 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146c00bde0 con 0x7f14780b8200 2021-02-10T11:18:06.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.738 7f1469ffb700 1 -- 172.21.2.101:0/2003837124 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f146c0052e0 con 0x7f14780b8200 2021-02-10T11:18:06.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.738 7f147f920700 1 -- 172.21.2.101:0/2003837124 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14780c27d0 con 0x7f14780b8200 2021-02-10T11:18:06.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.738 7f147f920700 1 -- 172.21.2.101:0/2003837124 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14780c2d60 con 0x7f14780b8200 2021-02-10T11:18:06.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.739 7f1469ffb700 1 -- 172.21.2.101:0/2003837124 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f146c006030 con 0x7f14780b8200 2021-02-10T11:18:06.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.739 7f1469ffb700 1 --2- 172.21.2.101:0/2003837124 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14580186c0 0x7f145801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:06.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.739 7f1469ffb700 1 -- 172.21.2.101:0/2003837124 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f146c0325c0 con 0x7f14780b8200 2021-02-10T11:18:06.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.739 7f147d73c700 1 --2- 172.21.2.101:0/2003837124 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14580186c0 0x7f145801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:06.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.739 7f147d73c700 1 --2- 172.21.2.101:0/2003837124 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14580186c0 0x7f145801abc0 crc :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:06.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.769 7f147f920700 1 -- 172.21.2.101:0/2003837124 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1478105ae0 con 0x7f14780b8200 2021-02-10T11:18:06.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.772 7f1469ffb700 1 -- 172.21.2.101:0/2003837124 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f146c009360 con 0x7f14780b8200 2021-02-10T11:18:06.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.899 7f147f920700 1 -- 172.21.2.101:0/2003837124 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839257", "target": ["mgr", ""]}) v1 -- 0x7f14780c2d60 con 0x7f14580186c0 2021-02-10T11:18:06.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.923 7f1469ffb700 1 -- 172.21.2.101:0/2003837124 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f1474006e40 con 0x7f14580186c0 2021-02-10T11:18:06.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.931 7f147f920700 1 -- 172.21.2.101:0/2003837124 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14580186c0 msgr2=0x7f145801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:06.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.931 7f147f920700 1 --2- 172.21.2.101:0/2003837124 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14580186c0 0x7f145801abc0 crc :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:06.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.931 7f147f920700 1 -- 172.21.2.101:0/2003837124 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14780b8200 msgr2=0x7f14780b8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:06.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.931 7f147f920700 1 --2- 172.21.2.101:0/2003837124 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14780b8200 0x7f14780b8600 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f146c009650 tx=0x7f146c00b960).stop 2021-02-10T11:18:06.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.932 7f147f920700 1 -- 172.21.2.101:0/2003837124 shutdown_connections 2021-02-10T11:18:06.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.932 7f147f920700 1 --2- 172.21.2.101:0/2003837124 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14580186c0 0x7f145801abc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:06.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.932 7f147f920700 1 --2- 172.21.2.101:0/2003837124 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14780c1d80 0x7f14780c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:06.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.932 7f147f920700 1 --2- 172.21.2.101:0/2003837124 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14780b8200 0x7f14780b8600 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:06.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.932 7f147f920700 1 --2- 172.21.2.101:0/2003837124 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1478190920 0x7f14780b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:06.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.932 7f147f920700 1 -- 172.21.2.101:0/2003837124 >> 172.21.2.101:0/2003837124 conn(0x7f1478094e10 msgr2=0x7f14780a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:06.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.932 7f147f920700 1 -- 172.21.2.101:0/2003837124 shutdown_connections 2021-02-10T11:18:06.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:06.932 7f147f920700 1 -- 172.21.2.101:0/2003837124 wait complete. 2021-02-10T11:18:06.945 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839257 2021-02-10T11:18:07.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.354 7f8d38718700 1 Processor -- start 2021-02-10T11:18:07.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.354 7f8d38718700 1 -- start start 2021-02-10T11:18:07.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.354 7f8d38718700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8d301b0870 0x7f8d300f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:07.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.354 7f8d38718700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d300efe00 0x7f8d300f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:07.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.354 7f8d38718700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d301b0c70 con 0x7f8d301b0470 2021-02-10T11:18:07.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.354 7f8d38718700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d300f0b50 con 0x7f8d301b0870 2021-02-10T11:18:07.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.354 7f8d38718700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d300f0820 con 0x7f8d300efe00 2021-02-10T11:18:07.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.355 7f8d36d35700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d300efe00 0x7f8d300f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:07.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.355 7f8d36d35700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d300efe00 0x7f8d300f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:38958/0 (socket says 172.21.2.101:38958) 2021-02-10T11:18:07.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.355 7f8d36d35700 1 -- 172.21.2.101:0/2546453695 learned_addr learned my addr 172.21.2.101:0/2546453695 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:18:07.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.355 7f8d35d33700 1 --2- 172.21.2.101:0/2546453695 >> v2:172.21.2.108:3300/0 conn(0x7f8d301b0870 0x7f8d300f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:07.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.355 7f8d36d35700 1 -- 172.21.2.101:0/2546453695 >> v2:172.21.2.108:3300/0 conn(0x7f8d301b0870 msgr2=0x7f8d300f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:18:07.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.355 7f8d36d35700 1 --2- 172.21.2.101:0/2546453695 >> v2:172.21.2.108:3300/0 conn(0x7f8d301b0870 0x7f8d300f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:07.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.355 7f8d36d35700 1 -- 172.21.2.101:0/2546453695 >> v1:172.21.2.108:6789/0 conn(0x7f8d301b0470 legacy=0x7f8d300ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:07.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.355 7f8d36d35700 1 -- 172.21.2.101:0/2546453695 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d300e52e0 con 0x7f8d300efe00 2021-02-10T11:18:07.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.355 7f8d36d35700 1 --2- 172.21.2.101:0/2546453695 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d300efe00 0x7f8d300f0200 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f8d28006aa0 tx=0x7f8d28009f20).ready entity=mon.2 client_cookie=74cab702fe412fe9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:07.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.355 7f8d35532700 1 -- 172.21.2.101:0/2546453695 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d2800b5c0 con 0x7f8d300efe00 2021-02-10T11:18:07.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.356 7f8d35532700 1 -- 172.21.2.101:0/2546453695 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d28003030 con 0x7f8d300efe00 2021-02-10T11:18:07.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.356 7f8d35532700 1 -- 172.21.2.101:0/2546453695 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d2800ba30 con 0x7f8d300efe00 2021-02-10T11:18:07.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.356 7f8d38718700 1 -- 172.21.2.101:0/2546453695 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d300efe00 msgr2=0x7f8d300f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:07.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.356 7f8d38718700 1 --2- 172.21.2.101:0/2546453695 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d300efe00 0x7f8d300f0200 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f8d28006aa0 tx=0x7f8d28009f20).stop 2021-02-10T11:18:07.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.356 7f8d38718700 1 -- 172.21.2.101:0/2546453695 shutdown_connections 2021-02-10T11:18:07.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.356 7f8d38718700 1 --2- 172.21.2.101:0/2546453695 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d300efe00 0x7f8d300f0200 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:07.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.356 7f8d38718700 1 --2- 172.21.2.101:0/2546453695 >> v2:172.21.2.108:3300/0 conn(0x7f8d301b0870 0x7f8d300f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:07.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.356 7f8d38718700 1 -- 172.21.2.101:0/2546453695 >> 172.21.2.101:0/2546453695 conn(0x7f8d300ce3e0 msgr2=0x7f8d300e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:07.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.357 7f8d38718700 1 -- 172.21.2.101:0/2546453695 shutdown_connections 2021-02-10T11:18:07.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.357 7f8d38718700 1 -- 172.21.2.101:0/2546453695 wait complete. 2021-02-10T11:18:07.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.371 7f8d38718700 1 Processor -- start 2021-02-10T11:18:07.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.371 7f8d38718700 1 -- start start 2021-02-10T11:18:07.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.371 7f8d38718700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d300a0420 0x7f8d300a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:07.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.371 7f8d38718700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d30189d00 0x7f8d3018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:07.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.371 7f8d38718700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d3018a720 0x7f8d300b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:07.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.371 7f8d38718700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d30189800 con 0x7f8d3018a720 2021-02-10T11:18:07.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.371 7f8d38718700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3018ab20 con 0x7f8d30189d00 2021-02-10T11:18:07.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.371 7f8d38718700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d300a1140 con 0x7f8d300a0420 2021-02-10T11:18:07.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.371 7f8d36d35700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d3018a720 0x7f8d300b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:07.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.371 7f8d36d35700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d3018a720 0x7f8d300b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46228/0 (socket says 172.21.2.101:46228) 2021-02-10T11:18:07.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.372 7f8d36d35700 1 -- 172.21.2.101:0/855540108 learned_addr learned my addr 172.21.2.101:0/855540108 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:18:07.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.372 7f8d36534700 1 --2- 172.21.2.101:0/855540108 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d300a0420 0x7f8d300a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:07.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.372 7f8d35d33700 1 --2- 172.21.2.101:0/855540108 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d30189d00 0x7f8d3018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:07.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.372 7f8d36d35700 1 -- 172.21.2.101:0/855540108 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d300a0420 msgr2=0x7f8d300a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:07.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.372 7f8d36d35700 1 --2- 172.21.2.101:0/855540108 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d300a0420 0x7f8d300a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:07.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.372 7f8d36d35700 1 -- 172.21.2.101:0/855540108 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d30189d00 msgr2=0x7f8d3018a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:07.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.372 7f8d36d35700 1 --2- 172.21.2.101:0/855540108 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d30189d00 0x7f8d3018a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:07.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.372 7f8d36d35700 1 -- 172.21.2.101:0/855540108 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d300b8200 con 0x7f8d3018a720 2021-02-10T11:18:07.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.372 7f8d35d33700 1 --2- 172.21.2.101:0/855540108 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d30189d00 0x7f8d3018a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:18:07.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.372 7f8d36d35700 1 --2- 172.21.2.101:0/855540108 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d3018a720 0x7f8d300b7be0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f8d2800b980 tx=0x7f8d2800bd20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:07.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.372 7f8d26ffd700 1 -- 172.21.2.101:0/855540108 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d28009450 con 0x7f8d3018a720 2021-02-10T11:18:07.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.372 7f8d26ffd700 1 -- 172.21.2.101:0/855540108 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d28006bb0 con 0x7f8d3018a720 2021-02-10T11:18:07.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.372 7f8d38718700 1 -- 172.21.2.101:0/855540108 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d300b8200 con 0x7f8d3018a720 2021-02-10T11:18:07.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.372 7f8d26ffd700 1 -- 172.21.2.101:0/855540108 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d2800c2f0 con 0x7f8d3018a720 2021-02-10T11:18:07.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.372 7f8d38718700 1 -- 172.21.2.101:0/855540108 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d300b8200 con 0x7f8d3018a720 2021-02-10T11:18:07.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.373 7f8d26ffd700 1 -- 172.21.2.101:0/855540108 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8d28009430 con 0x7f8d3018a720 2021-02-10T11:18:07.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.373 7f8d26ffd700 1 --2- 172.21.2.101:0/855540108 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d180186d0 0x7f8d1801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:07.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.373 7f8d26ffd700 1 -- 172.21.2.101:0/855540108 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8d28032b10 con 0x7f8d3018a720 2021-02-10T11:18:07.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.373 7f8d36534700 1 --2- 172.21.2.101:0/855540108 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d180186d0 0x7f8d1801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:07.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.373 7f8d36534700 1 --2- 172.21.2.101:0/855540108 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d180186d0 0x7f8d1801abd0 crc :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:07.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.404 7f8d38718700 1 -- 172.21.2.101:0/855540108 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d30105ae0 con 0x7f8d3018a720 2021-02-10T11:18:07.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.406 7f8d26ffd700 1 -- 172.21.2.101:0/855540108 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8d28000df0 con 0x7f8d3018a720 2021-02-10T11:18:07.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.534 7f8d38718700 1 -- 172.21.2.101:0/855540108 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839257", "target": ["mgr", ""]}) v1 -- 0x7f8d3018ab20 con 0x7f8d180186d0 2021-02-10T11:18:07.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.538 7f8d26ffd700 1 -- 172.21.2.101:0/855540108 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f8d2c006a90 con 0x7f8d180186d0 2021-02-10T11:18:07.547 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839257/14a6cc23-3695-4570-8d0b-ad11fb88bea8 2021-02-10T11:18:07.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.543 7f8d38718700 1 -- 172.21.2.101:0/855540108 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d180186d0 msgr2=0x7f8d1801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:07.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.543 7f8d38718700 1 --2- 172.21.2.101:0/855540108 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d180186d0 0x7f8d1801abd0 crc :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:07.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.543 7f8d38718700 1 -- 172.21.2.101:0/855540108 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d3018a720 msgr2=0x7f8d300b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:07.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.543 7f8d38718700 1 --2- 172.21.2.101:0/855540108 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d3018a720 0x7f8d300b7be0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f8d2800b980 tx=0x7f8d2800bd20).stop 2021-02-10T11:18:07.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.543 7f8d38718700 1 -- 172.21.2.101:0/855540108 shutdown_connections 2021-02-10T11:18:07.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.543 7f8d38718700 1 --2- 172.21.2.101:0/855540108 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d3018a720 0x7f8d300b7be0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:07.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.543 7f8d38718700 1 --2- 172.21.2.101:0/855540108 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d30189d00 0x7f8d3018a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:07.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.543 7f8d38718700 1 --2- 172.21.2.101:0/855540108 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d180186d0 0x7f8d1801abd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:07.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.543 7f8d38718700 1 --2- 172.21.2.101:0/855540108 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d300a0420 0x7f8d300a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:07.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.543 7f8d38718700 1 -- 172.21.2.101:0/855540108 >> 172.21.2.101:0/855540108 conn(0x7f8d30094e10 msgr2=0x7f8d300a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:07.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.543 7f8d38718700 1 -- 172.21.2.101:0/855540108 shutdown_connections 2021-02-10T11:18:07.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:07.543 7f8d38718700 1 -- 172.21.2.101:0/855540108 wait complete. 2021-02-10T11:18:07.556 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839257 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839257/14a6cc23-3695-4570-8d0b-ad11fb88bea8 2021-02-10T11:18:07.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:07.557 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:07.557 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:07.891 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:07.891 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:07.891 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839257/14a6cc23-3695-4570-8d0b-ad11fb88bea8/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:08.272 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:08.272 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:08.272 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248709 s, 42.2 MB/s 2021-02-10T11:18:08.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:08.274 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:08.274 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:08.608 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:08.609 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:08.609 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839257/14a6cc23-3695-4570-8d0b-ad11fb88bea8/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:08.981 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:08.981 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:08.981 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243702 s, 43.0 MB/s 2021-02-10T11:18:08.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:08.982 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:08.982 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:09.317 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:09.317 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:09.318 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839257/14a6cc23-3695-4570-8d0b-ad11fb88bea8/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:09.693 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:09.693 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:09.693 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249917 s, 42.0 MB/s 2021-02-10T11:18:09.694 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:09.694 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:09.694 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:10.028 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:10.028 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:10.029 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839257/14a6cc23-3695-4570-8d0b-ad11fb88bea8/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:10.405 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:10.405 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:10.405 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251133 s, 41.8 MB/s 2021-02-10T11:18:10.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:10.406 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:10.406 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:10.740 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:10.740 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:10.741 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839257/14a6cc23-3695-4570-8d0b-ad11fb88bea8/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:11.115 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:12.700 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:12.700 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241922 s, 43.3 MB/s 2021-02-10T11:18:12.702 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:12.702 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:12.702 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:13.037 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:13.038 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:13.038 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839257/14a6cc23-3695-4570-8d0b-ad11fb88bea8/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:13.412 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:14.302 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:14.303 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248441 s, 42.2 MB/s 2021-02-10T11:18:14.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:14.304 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:14.304 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:14.640 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:14.640 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:14.640 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839257/14a6cc23-3695-4570-8d0b-ad11fb88bea8/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:15.013 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:15.013 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:15.013 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0232116 s, 45.2 MB/s 2021-02-10T11:18:15.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:15.014 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:15.014 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:15.351 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:15.351 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:15.351 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839257/14a6cc23-3695-4570-8d0b-ad11fb88bea8/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:15.731 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:15.731 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:15.731 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236241 s, 44.4 MB/s 2021-02-10T11:18:15.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:15.732 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:15.732 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:16.070 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:16.071 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:16.071 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839257/14a6cc23-3695-4570-8d0b-ad11fb88bea8/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:16.448 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:16.448 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:16.448 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241867 s, 43.4 MB/s 2021-02-10T11:18:16.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:16.449 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:16.449 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:16.670 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:18:16.674 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:18:16.788 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:16.788 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:16.789 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839257/14a6cc23-3695-4570-8d0b-ad11fb88bea8/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:17.166 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:17.166 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:17.166 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242722 s, 43.2 MB/s 2021-02-10T11:18:17.167 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839258 2021-02-10T11:18:17.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.575 7f8cba859700 1 Processor -- start 2021-02-10T11:18:17.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.575 7f8cba859700 1 -- start start 2021-02-10T11:18:17.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.576 7f8cba859700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8cb41b0470 0x7f8cb41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:17.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.576 7f8cba859700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8cb40ef610 0x7f8cb40efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:17.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.576 7f8cba859700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cb40f0b50 con 0x7f8cb40f0030 2021-02-10T11:18:17.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.576 7f8cba859700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cb40e52e0 con 0x7f8cb40ef610 2021-02-10T11:18:17.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.576 7f8cba859700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cb40e5490 con 0x7f8cb41b0470 2021-02-10T11:18:17.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.576 7f8cb37fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8cb40ef610 0x7f8cb40efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:17.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.576 7f8cb37fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8cb40ef610 0x7f8cb40efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46236/0 (socket says 172.21.2.101:46236) 2021-02-10T11:18:17.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.576 7f8cb37fe700 1 -- 172.21.2.101:0/335213996 learned_addr learned my addr 172.21.2.101:0/335213996 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:18:17.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.576 7f8cb37fe700 1 -- 172.21.2.101:0/335213996 >> v2:172.21.2.108:3300/0 conn(0x7f8cb41b0470 msgr2=0x7f8cb41b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:18:17.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.576 7f8cb37fe700 1 --2- 172.21.2.101:0/335213996 >> v2:172.21.2.108:3300/0 conn(0x7f8cb41b0470 0x7f8cb41b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:17.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.576 7f8cb37fe700 1 -- 172.21.2.101:0/335213996 >> v1:172.21.2.101:6789/0 conn(0x7f8cb40f0030 legacy=0x7f8cb40f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:17.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.576 7f8cb37fe700 1 -- 172.21.2.101:0/335213996 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb40f0430 con 0x7f8cb40ef610 2021-02-10T11:18:17.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.577 7f8cb37fe700 1 --2- 172.21.2.101:0/335213996 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8cb40ef610 0x7f8cb40efa10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f8ca0006700 tx=0x7f8ca0009b80).ready entity=mon.0 client_cookie=368de57d61744a09 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:17.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.577 7f8cb2ffd700 1 -- 172.21.2.101:0/335213996 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8ca000c0c0 con 0x7f8cb40ef610 2021-02-10T11:18:17.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.577 7f8cb2ffd700 1 -- 172.21.2.101:0/335213996 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca000c340 con 0x7f8cb40ef610 2021-02-10T11:18:17.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.577 7f8cb2ffd700 1 -- 172.21.2.101:0/335213996 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8ca000c860 con 0x7f8cb40ef610 2021-02-10T11:18:17.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.577 7f8cba859700 1 -- 172.21.2.101:0/335213996 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8cb40ef610 msgr2=0x7f8cb40efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:17.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.577 7f8cba859700 1 --2- 172.21.2.101:0/335213996 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8cb40ef610 0x7f8cb40efa10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f8ca0006700 tx=0x7f8ca0009b80).stop 2021-02-10T11:18:17.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.577 7f8cba859700 1 -- 172.21.2.101:0/335213996 shutdown_connections 2021-02-10T11:18:17.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.577 7f8cba859700 1 --2- 172.21.2.101:0/335213996 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8cb40ef610 0x7f8cb40efa10 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:17.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.577 7f8cba859700 1 --2- 172.21.2.101:0/335213996 >> v2:172.21.2.108:3300/0 conn(0x7f8cb41b0470 0x7f8cb41b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:17.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.577 7f8cba859700 1 -- 172.21.2.101:0/335213996 >> 172.21.2.101:0/335213996 conn(0x7f8cb40ce3e0 msgr2=0x7f8cb40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:17.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.578 7f8cba859700 1 -- 172.21.2.101:0/335213996 shutdown_connections 2021-02-10T11:18:17.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.578 7f8cba859700 1 -- 172.21.2.101:0/335213996 wait complete. 2021-02-10T11:18:17.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.592 7f8cba859700 1 Processor -- start 2021-02-10T11:18:17.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.592 7f8cba859700 1 -- start start 2021-02-10T11:18:17.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.593 7f8cba859700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8cb40a6c90 0x7f8cb41924c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:17.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.593 7f8cba859700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8cb4192ae0 0x7f8cb4192ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:17.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.593 7f8cba859700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8cb40919e0 0x7f8cb4091de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:17.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.593 7f8cba859700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb419d2e0 con 0x7f8cb4192ae0 2021-02-10T11:18:17.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.593 7f8cba859700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb40914d0 con 0x7f8cb40a6c90 2021-02-10T11:18:17.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.593 7f8cba859700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb40a6820 con 0x7f8cb40919e0 2021-02-10T11:18:17.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.593 7f8cb37fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8cb4192ae0 0x7f8cb4192ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:17.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.593 7f8cb37fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8cb4192ae0 0x7f8cb4192ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46240/0 (socket says 172.21.2.101:46240) 2021-02-10T11:18:17.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.593 7f8cb37fe700 1 -- 172.21.2.101:0/3275512723 learned_addr learned my addr 172.21.2.101:0/3275512723 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:18:17.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.593 7f8cb8e76700 1 --2- 172.21.2.101:0/3275512723 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8cb40919e0 0x7f8cb4091de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:17.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.593 7f8cb3fff700 1 --2- 172.21.2.101:0/3275512723 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8cb40a6c90 0x7f8cb41924c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:17.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.593 7f8cb37fe700 1 -- 172.21.2.101:0/3275512723 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8cb40919e0 msgr2=0x7f8cb4091de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:17.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.593 7f8cb37fe700 1 --2- 172.21.2.101:0/3275512723 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8cb40919e0 0x7f8cb4091de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:17.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.593 7f8cb37fe700 1 -- 172.21.2.101:0/3275512723 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8cb40a6c90 msgr2=0x7f8cb41924c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:17.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.593 7f8cb37fe700 1 --2- 172.21.2.101:0/3275512723 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8cb40a6c90 0x7f8cb41924c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:17.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.593 7f8cb37fe700 1 -- 172.21.2.101:0/3275512723 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb4092400 con 0x7f8cb4192ae0 2021-02-10T11:18:17.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.593 7f8cb37fe700 1 --2- 172.21.2.101:0/3275512723 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8cb4192ae0 0x7f8cb4192ee0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f8ca000b650 tx=0x7f8ca000b9f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:17.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.594 7f8cb0ff9700 1 -- 172.21.2.101:0/3275512723 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8ca000bde0 con 0x7f8cb4192ae0 2021-02-10T11:18:17.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.594 7f8cb0ff9700 1 -- 172.21.2.101:0/3275512723 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca00091e0 con 0x7f8cb4192ae0 2021-02-10T11:18:17.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.594 7f8cba859700 1 -- 172.21.2.101:0/3275512723 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb4092400 con 0x7f8cb4192ae0 2021-02-10T11:18:17.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.594 7f8cb0ff9700 1 -- 172.21.2.101:0/3275512723 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8ca00022b0 con 0x7f8cb4192ae0 2021-02-10T11:18:17.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.594 7f8cba859700 1 -- 172.21.2.101:0/3275512723 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb4092990 con 0x7f8cb4192ae0 2021-02-10T11:18:17.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.594 7f8cb0ff9700 1 -- 172.21.2.101:0/3275512723 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8ca0009f90 con 0x7f8cb4192ae0 2021-02-10T11:18:17.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.594 7f8cb0ff9700 1 --2- 172.21.2.101:0/3275512723 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c90018730 0x7f8c9001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:17.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.594 7f8cb0ff9700 1 -- 172.21.2.101:0/3275512723 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8ca0032610 con 0x7f8cb4192ae0 2021-02-10T11:18:17.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.595 7f8cb3fff700 1 --2- 172.21.2.101:0/3275512723 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c90018730 0x7f8c9001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:17.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.595 7f8cb3fff700 1 --2- 172.21.2.101:0/3275512723 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c90018730 0x7f8c9001abc0 crc :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:17.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.626 7f8cba859700 1 -- 172.21.2.101:0/3275512723 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cb40e52e0 con 0x7f8cb4192ae0 2021-02-10T11:18:17.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.628 7f8cb0ff9700 1 -- 172.21.2.101:0/3275512723 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8ca00009b0 con 0x7f8cb4192ae0 2021-02-10T11:18:17.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.753 7f8cba859700 1 -- 172.21.2.101:0/3275512723 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839258", "target": ["mgr", ""]}) v1 -- 0x7f8cb4092990 con 0x7f8c90018730 2021-02-10T11:18:17.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.780 7f8cb0ff9700 1 -- 172.21.2.101:0/3275512723 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f8ca4006e30 con 0x7f8c90018730 2021-02-10T11:18:17.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.786 7f8cba859700 1 -- 172.21.2.101:0/3275512723 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c90018730 msgr2=0x7f8c9001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:17.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.786 7f8cba859700 1 --2- 172.21.2.101:0/3275512723 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c90018730 0x7f8c9001abc0 crc :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:17.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.786 7f8cba859700 1 -- 172.21.2.101:0/3275512723 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8cb4192ae0 msgr2=0x7f8cb4192ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:17.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.786 7f8cba859700 1 --2- 172.21.2.101:0/3275512723 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8cb4192ae0 0x7f8cb4192ee0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f8ca000b650 tx=0x7f8ca000b9f0).stop 2021-02-10T11:18:17.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.786 7f8cba859700 1 -- 172.21.2.101:0/3275512723 shutdown_connections 2021-02-10T11:18:17.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.786 7f8cba859700 1 --2- 172.21.2.101:0/3275512723 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c90018730 0x7f8c9001abc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:17.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.786 7f8cba859700 1 --2- 172.21.2.101:0/3275512723 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8cb40919e0 0x7f8cb4091de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:17.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.786 7f8cba859700 1 --2- 172.21.2.101:0/3275512723 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8cb4192ae0 0x7f8cb4192ee0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:17.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.786 7f8cba859700 1 --2- 172.21.2.101:0/3275512723 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8cb40a6c90 0x7f8cb41924c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:17.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.786 7f8cba859700 1 -- 172.21.2.101:0/3275512723 >> 172.21.2.101:0/3275512723 conn(0x7f8cb40ac550 msgr2=0x7f8cb40be1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:17.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.787 7f8cba859700 1 -- 172.21.2.101:0/3275512723 shutdown_connections 2021-02-10T11:18:17.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:17.787 7f8cba859700 1 -- 172.21.2.101:0/3275512723 wait complete. 2021-02-10T11:18:17.800 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839258 2021-02-10T11:18:18.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.209 7fc330e47700 1 Processor -- start 2021-02-10T11:18:18.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.209 7fc330e47700 1 -- start start 2021-02-10T11:18:18.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.210 7fc330e47700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc32c1b0870 0x7fc32c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:18.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.210 7fc330e47700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc32c1b0c70 con 0x7fc32c0efe00 2021-02-10T11:18:18.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.210 7fc330e47700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc32c0f0b50 con 0x7fc32c1b0470 2021-02-10T11:18:18.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.210 7fc330e47700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc32c0e5ea0 con 0x7fc32c1b0870 2021-02-10T11:18:18.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.210 7fc32b314700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc32c0efe00 0x7fc32c0f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46746/0 (socket says 172.21.2.101:46746) 2021-02-10T11:18:18.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.210 7fc32b314700 1 -- 172.21.2.101:0/1627229466 learned_addr learned my addr 172.21.2.101:0/1627229466 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:18:18.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.210 7fc32a312700 1 --2- 172.21.2.101:0/1627229466 >> v2:172.21.2.101:3300/0 conn(0x7fc32c1b0870 0x7fc32c0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:18.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.210 7fc32a312700 1 -- 172.21.2.101:0/1627229466 >> v1:172.21.2.108:6789/0 conn(0x7fc32c1b0470 legacy=0x7fc32c0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:18:18.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.210 7fc32a312700 1 -- 172.21.2.101:0/1627229466 >> v1:172.21.2.101:6789/0 conn(0x7fc32c0efe00 legacy=0x7fc32c0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:18.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.210 7fc329b11700 1 -- 172.21.2.101:0/1627229466 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2135569888 0 0) 0x7fc324001500 con 0x7fc32c0efe00 2021-02-10T11:18:18.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.210 7fc32a312700 1 -- 172.21.2.101:0/1627229466 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32c0d2fb0 con 0x7fc32c1b0870 2021-02-10T11:18:18.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.211 7fc32a312700 1 --2- 172.21.2.101:0/1627229466 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc32c1b0870 0x7fc32c0f20e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fc31c006650 tx=0x7fc31c009a00).ready entity=mon.0 client_cookie=7c34d9aff2adc486 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:18.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.211 7fc329b11700 1 -- 172.21.2.101:0/1627229466 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc31c009df0 con 0x7fc32c1b0870 2021-02-10T11:18:18.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.211 7fc329b11700 1 -- 172.21.2.101:0/1627229466 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31c0009b0 con 0x7fc32c1b0870 2021-02-10T11:18:18.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.211 7fc329b11700 1 -- 172.21.2.101:0/1627229466 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc31c00c2b0 con 0x7fc32c1b0870 2021-02-10T11:18:18.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.211 7fc330e47700 1 -- 172.21.2.101:0/1627229466 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc32c1b0870 msgr2=0x7fc32c0f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:18.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.211 7fc330e47700 1 --2- 172.21.2.101:0/1627229466 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc32c1b0870 0x7fc32c0f20e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fc31c006650 tx=0x7fc31c009a00).stop 2021-02-10T11:18:18.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.211 7fc330e47700 1 -- 172.21.2.101:0/1627229466 shutdown_connections 2021-02-10T11:18:18.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.211 7fc330e47700 1 --2- 172.21.2.101:0/1627229466 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc32c1b0870 0x7fc32c0f20e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:18.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.211 7fc330e47700 1 -- 172.21.2.101:0/1627229466 >> 172.21.2.101:0/1627229466 conn(0x7fc32c0ce3e0 msgr2=0x7fc32c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:18.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.211 7fc330e47700 1 -- 172.21.2.101:0/1627229466 shutdown_connections 2021-02-10T11:18:18.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.211 7fc330e47700 1 -- 172.21.2.101:0/1627229466 wait complete. 2021-02-10T11:18:18.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.226 7fc330e47700 1 Processor -- start 2021-02-10T11:18:18.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.226 7fc330e47700 1 -- start start 2021-02-10T11:18:18.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.226 7fc330e47700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc32c0a1dd0 0x7fc32c190920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:18.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.226 7fc330e47700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc32c08fd30 0x7fc32c0effc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:18.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.226 7fc330e47700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc32c0f0e90 0x7fc32c0f1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:18.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.226 7fc330e47700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c0f06f0 con 0x7fc32c0a1dd0 2021-02-10T11:18:18.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.226 7fc330e47700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c090160 con 0x7fc32c08fd30 2021-02-10T11:18:18.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.226 7fc330e47700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c0f18b0 con 0x7fc32c0f0e90 2021-02-10T11:18:18.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.226 7fc32ab13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc32c0a1dd0 0x7fc32c190920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:18.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.226 7fc32ab13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc32c0a1dd0 0x7fc32c190920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46258/0 (socket says 172.21.2.101:46258) 2021-02-10T11:18:18.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.226 7fc32ab13700 1 -- 172.21.2.101:0/1305889175 learned_addr learned my addr 172.21.2.101:0/1305889175 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:18:18.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.227 7fc32ab13700 1 -- 172.21.2.101:0/1305889175 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc32c0f0e90 msgr2=0x7fc32c0f1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:18.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.227 7fc32ab13700 1 --2- 172.21.2.101:0/1305889175 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc32c0f0e90 0x7fc32c0f1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:18.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.227 7fc32ab13700 1 -- 172.21.2.101:0/1305889175 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc32c08fd30 msgr2=0x7fc32c0effc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:18.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.227 7fc32ab13700 1 --2- 172.21.2.101:0/1305889175 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc32c08fd30 0x7fc32c0effc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:18.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.227 7fc32ab13700 1 -- 172.21.2.101:0/1305889175 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32c0f1ad0 con 0x7fc32c0a1dd0 2021-02-10T11:18:18.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.227 7fc32ab13700 1 --2- 172.21.2.101:0/1305889175 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc32c0a1dd0 0x7fc32c190920 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fc31400b5b0 tx=0x7fc31400b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:18.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.227 7fc3237fe700 1 -- 172.21.2.101:0/1305889175 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc31400bd40 con 0x7fc32c0a1dd0 2021-02-10T11:18:18.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.227 7fc3237fe700 1 -- 172.21.2.101:0/1305889175 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3140009b0 con 0x7fc32c0a1dd0 2021-02-10T11:18:18.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.227 7fc3237fe700 1 -- 172.21.2.101:0/1305889175 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc3140062a0 con 0x7fc32c0a1dd0 2021-02-10T11:18:18.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.227 7fc330e47700 1 -- 172.21.2.101:0/1305889175 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc32c0f1ad0 con 0x7fc32c0a1dd0 2021-02-10T11:18:18.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.227 7fc330e47700 1 -- 172.21.2.101:0/1305889175 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc32c0f1ad0 con 0x7fc32c0a1dd0 2021-02-10T11:18:18.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.228 7fc3237fe700 1 -- 172.21.2.101:0/1305889175 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc314006020 con 0x7fc32c0a1dd0 2021-02-10T11:18:18.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.228 7fc3237fe700 1 --2- 172.21.2.101:0/1305889175 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3100186c0 0x7fc31001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:18.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.228 7fc3237fe700 1 -- 172.21.2.101:0/1305889175 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc314032590 con 0x7fc32c0a1dd0 2021-02-10T11:18:18.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.228 7fc32a312700 1 --2- 172.21.2.101:0/1305889175 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3100186c0 0x7fc31001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:18.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.228 7fc32a312700 1 --2- 172.21.2.101:0/1305889175 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3100186c0 0x7fc31001abc0 crc :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:18.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.259 7fc330e47700 1 -- 172.21.2.101:0/1305889175 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc32c090130 con 0x7fc32c0a1dd0 2021-02-10T11:18:18.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.262 7fc3237fe700 1 -- 172.21.2.101:0/1305889175 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc31400bd40 con 0x7fc32c0a1dd0 2021-02-10T11:18:18.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.390 7fc330e47700 1 -- 172.21.2.101:0/1305889175 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839258", "target": ["mgr", ""]}) v1 -- 0x7fc32c090130 con 0x7fc3100186c0 2021-02-10T11:18:18.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.393 7fc3237fe700 1 -- 172.21.2.101:0/1305889175 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fc31c006740 con 0x7fc3100186c0 2021-02-10T11:18:18.399 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839258/60716029-de44-4f7c-9286-3005bb8e07c5 2021-02-10T11:18:18.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.394 7fc330e47700 1 -- 172.21.2.101:0/1305889175 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3100186c0 msgr2=0x7fc31001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:18.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.394 7fc330e47700 1 --2- 172.21.2.101:0/1305889175 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3100186c0 0x7fc31001abc0 crc :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:18.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.394 7fc330e47700 1 -- 172.21.2.101:0/1305889175 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc32c0a1dd0 msgr2=0x7fc32c190920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:18.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.394 7fc330e47700 1 --2- 172.21.2.101:0/1305889175 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc32c0a1dd0 0x7fc32c190920 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fc31400b5b0 tx=0x7fc31400b950).stop 2021-02-10T11:18:18.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.395 7fc330e47700 1 -- 172.21.2.101:0/1305889175 shutdown_connections 2021-02-10T11:18:18.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.395 7fc330e47700 1 --2- 172.21.2.101:0/1305889175 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3100186c0 0x7fc31001abc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:18.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.395 7fc330e47700 1 --2- 172.21.2.101:0/1305889175 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc32c0f0e90 0x7fc32c0f1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:18.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.395 7fc330e47700 1 --2- 172.21.2.101:0/1305889175 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc32c08fd30 0x7fc32c0effc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:18.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.395 7fc330e47700 1 --2- 172.21.2.101:0/1305889175 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc32c0a1dd0 0x7fc32c190920 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:18.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.395 7fc330e47700 1 -- 172.21.2.101:0/1305889175 >> 172.21.2.101:0/1305889175 conn(0x7fc32c0958b0 msgr2=0x7fc32c0bf040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:18.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.395 7fc330e47700 1 -- 172.21.2.101:0/1305889175 shutdown_connections 2021-02-10T11:18:18.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:18.395 7fc330e47700 1 -- 172.21.2.101:0/1305889175 wait complete. 2021-02-10T11:18:18.409 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839258 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839258/60716029-de44-4f7c-9286-3005bb8e07c5 2021-02-10T11:18:18.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:18.409 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:18.409 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:18.744 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:18.744 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:18.745 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839258/60716029-de44-4f7c-9286-3005bb8e07c5/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:19.123 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:19.123 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:19.123 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248145 s, 42.3 MB/s 2021-02-10T11:18:19.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:19.124 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:19.124 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:19.459 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:19.460 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:19.460 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839258/60716029-de44-4f7c-9286-3005bb8e07c5/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:19.834 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:19.835 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:19.835 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236765 s, 44.3 MB/s 2021-02-10T11:18:19.836 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:19.836 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:19.836 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:20.171 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:20.171 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:20.171 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839258/60716029-de44-4f7c-9286-3005bb8e07c5/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:20.545 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:20.546 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:20.546 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244024 s, 43.0 MB/s 2021-02-10T11:18:20.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:20.547 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:20.547 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:20.881 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:20.882 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:20.882 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839258/60716029-de44-4f7c-9286-3005bb8e07c5/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:21.259 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:21.259 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:21.259 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0256556 s, 40.9 MB/s 2021-02-10T11:18:21.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:21.260 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:21.260 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:21.594 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:21.594 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:21.595 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839258/60716029-de44-4f7c-9286-3005bb8e07c5/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:21.970 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:21.971 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:21.971 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239743 s, 43.7 MB/s 2021-02-10T11:18:21.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:21.972 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:21.972 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:22.306 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:22.307 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:22.307 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839258/60716029-de44-4f7c-9286-3005bb8e07c5/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:22.683 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:22.683 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:22.683 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243733 s, 43.0 MB/s 2021-02-10T11:18:22.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:22.684 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:22.684 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:23.020 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:23.020 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:23.020 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839258/60716029-de44-4f7c-9286-3005bb8e07c5/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:23.395 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:23.395 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:23.396 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240594 s, 43.6 MB/s 2021-02-10T11:18:23.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:23.397 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:23.397 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:23.733 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:23.733 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:23.734 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839258/60716029-de44-4f7c-9286-3005bb8e07c5/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:24.105 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:24.106 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:24.106 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0232898 s, 45.0 MB/s 2021-02-10T11:18:24.107 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:24.107 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:24.107 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:24.441 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:24.441 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:24.441 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839258/60716029-de44-4f7c-9286-3005bb8e07c5/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:24.815 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:24.815 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:24.815 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024248 s, 43.2 MB/s 2021-02-10T11:18:24.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:24.816 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:24.816 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:25.151 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:25.151 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:25.151 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839258/60716029-de44-4f7c-9286-3005bb8e07c5/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:25.526 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:25.527 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:25.527 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249831 s, 42.0 MB/s 2021-02-10T11:18:25.528 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839259 2021-02-10T11:18:25.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.935 7f548b244700 1 Processor -- start 2021-02-10T11:18:25.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.935 7f548b244700 1 -- start start 2021-02-10T11:18:25.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.935 7f548b244700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54841b0470 0x7f54841b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:25.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.935 7f548b244700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f54840f20e0 0x7f54840f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:25.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.935 7f548b244700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54840f0b50 con 0x7f54840ef610 2021-02-10T11:18:25.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.935 7f548b244700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54840f0820 con 0x7f54840f20e0 2021-02-10T11:18:25.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.935 7f548b244700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54840f24e0 con 0x7f54841b0470 2021-02-10T11:18:25.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.936 7f5489861700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f54840f20e0 0x7f54840f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:25.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.936 7f5489861700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f54840f20e0 0x7f54840f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46262/0 (socket says 172.21.2.101:46262) 2021-02-10T11:18:25.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.936 7f5489861700 1 -- 172.21.2.101:0/925082317 learned_addr learned my addr 172.21.2.101:0/925082317 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:18:25.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.936 7f5489060700 1 --2- 172.21.2.101:0/925082317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54841b0470 0x7f54841b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:25.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.936 7f5489861700 1 -- 172.21.2.101:0/925082317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54841b0470 msgr2=0x7f54841b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:18:25.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.936 7f5489861700 1 --2- 172.21.2.101:0/925082317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54841b0470 0x7f54841b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:25.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.936 7f5489861700 1 -- 172.21.2.101:0/925082317 >> v1:172.21.2.101:6789/0 conn(0x7f54840ef610 legacy=0x7f54840efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:25.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.936 7f5489861700 1 -- 172.21.2.101:0/925082317 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54840e52e0 con 0x7f54840f20e0 2021-02-10T11:18:25.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.936 7f5489861700 1 --2- 172.21.2.101:0/925082317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54840f20e0 0x7f54840f0200 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f547c006f00 tx=0x7f547c009f10).ready entity=mon.0 client_cookie=1c48cdcb6e5afdfd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.936 7f5477fff700 1 -- 172.21.2.101:0/925082317 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f547c00b3b0 con 0x7f54840f20e0 2021-02-10T11:18:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.936 7f5477fff700 1 -- 172.21.2.101:0/925082317 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c00b630 con 0x7f54840f20e0 2021-02-10T11:18:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.936 7f5477fff700 1 -- 172.21.2.101:0/925082317 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f547c00bb50 con 0x7f54840f20e0 2021-02-10T11:18:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.936 7f548b244700 1 -- 172.21.2.101:0/925082317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54840f20e0 msgr2=0x7f54840f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.936 7f548b244700 1 --2- 172.21.2.101:0/925082317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54840f20e0 0x7f54840f0200 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f547c006f00 tx=0x7f547c009f10).stop 2021-02-10T11:18:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.937 7f548b244700 1 -- 172.21.2.101:0/925082317 shutdown_connections 2021-02-10T11:18:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.937 7f548b244700 1 --2- 172.21.2.101:0/925082317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54840f20e0 0x7f54840f0200 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.937 7f548b244700 1 --2- 172.21.2.101:0/925082317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54841b0470 0x7f54841b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.937 7f548b244700 1 -- 172.21.2.101:0/925082317 >> 172.21.2.101:0/925082317 conn(0x7f54840ce3e0 msgr2=0x7f54840e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.937 7f548b244700 1 -- 172.21.2.101:0/925082317 shutdown_connections 2021-02-10T11:18:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.937 7f548b244700 1 -- 172.21.2.101:0/925082317 wait complete. 2021-02-10T11:18:25.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.952 7f548b244700 1 Processor -- start 2021-02-10T11:18:25.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.952 7f548b244700 1 -- start start 2021-02-10T11:18:25.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.952 7f548b244700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54840b7c20 0x7f54840b8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:25.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.952 7f548b244700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54840b8a30 0x7f54840c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:25.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.952 7f548b244700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54840c25a0 0x7f54840c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:25.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.952 7f548b244700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54840c1a80 con 0x7f54840b7c20 2021-02-10T11:18:25.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.952 7f548b244700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54840b8020 con 0x7f54840b8a30 2021-02-10T11:18:25.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.952 7f548b244700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548409b9c0 con 0x7f54840c25a0 2021-02-10T11:18:25.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.952 7f5489060700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54840b7c20 0x7f54840b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:25.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.952 7f5489060700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54840b7c20 0x7f54840b8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46266/0 (socket says 172.21.2.101:46266) 2021-02-10T11:18:25.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.952 7f5489060700 1 -- 172.21.2.101:0/1417513013 learned_addr learned my addr 172.21.2.101:0/1417513013 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:18:25.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.953 7f5489060700 1 -- 172.21.2.101:0/1417513013 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54840c25a0 msgr2=0x7f54840c29a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:18:25.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.953 7f5489060700 1 --2- 172.21.2.101:0/1417513013 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54840c25a0 0x7f54840c29a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:25.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.953 7f5489060700 1 -- 172.21.2.101:0/1417513013 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54840b8a30 msgr2=0x7f54840c1f80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:18:25.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.953 7f5489060700 1 --2- 172.21.2.101:0/1417513013 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54840b8a30 0x7f54840c1f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:25.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.953 7f5489060700 1 -- 172.21.2.101:0/1417513013 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54840b8e30 con 0x7f54840b7c20 2021-02-10T11:18:25.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.953 7f5489060700 1 --2- 172.21.2.101:0/1417513013 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54840b7c20 0x7f54840b8410 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f5480006f60 tx=0x7f548000b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:25.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.953 7f5475ffb700 1 -- 172.21.2.101:0/1417513013 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f548000bd60 con 0x7f54840b7c20 2021-02-10T11:18:25.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.953 7f5475ffb700 1 -- 172.21.2.101:0/1417513013 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5480000c90 con 0x7f54840b7c20 2021-02-10T11:18:25.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.953 7f5475ffb700 1 -- 172.21.2.101:0/1417513013 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5480002220 con 0x7f54840b7c20 2021-02-10T11:18:25.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.953 7f548b244700 1 -- 172.21.2.101:0/1417513013 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54840b8e30 con 0x7f54840b7c20 2021-02-10T11:18:25.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.953 7f548b244700 1 -- 172.21.2.101:0/1417513013 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54840c32b0 con 0x7f54840b7c20 2021-02-10T11:18:25.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.954 7f5475ffb700 1 -- 172.21.2.101:0/1417513013 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f54800052e0 con 0x7f54840b7c20 2021-02-10T11:18:25.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.954 7f5475ffb700 1 --2- 172.21.2.101:0/1417513013 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54640186c0 0x7f546401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:25.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.954 7f5475ffb700 1 -- 172.21.2.101:0/1417513013 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5480032530 con 0x7f54840b7c20 2021-02-10T11:18:25.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.954 7f548885f700 1 --2- 172.21.2.101:0/1417513013 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54640186c0 0x7f546401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:25.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.954 7f548885f700 1 --2- 172.21.2.101:0/1417513013 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54640186c0 0x7f546401abc0 crc :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:25.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.985 7f548b244700 1 -- 172.21.2.101:0/1417513013 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5484105b30 con 0x7f54840b7c20 2021-02-10T11:18:25.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:25.988 7f5475ffb700 1 -- 172.21.2.101:0/1417513013 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5480000940 con 0x7f54840b7c20 2021-02-10T11:18:26.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.115 7f548b244700 1 -- 172.21.2.101:0/1417513013 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839259", "target": ["mgr", ""]}) v1 -- 0x7f54840b8020 con 0x7f54640186c0 2021-02-10T11:18:26.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.144 7f5475ffb700 1 -- 172.21.2.101:0/1417513013 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f5478006b80 con 0x7f54640186c0 2021-02-10T11:18:26.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.148 7f548b244700 1 -- 172.21.2.101:0/1417513013 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54640186c0 msgr2=0x7f546401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:26.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.148 7f548b244700 1 --2- 172.21.2.101:0/1417513013 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54640186c0 0x7f546401abc0 crc :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:26.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.148 7f548b244700 1 -- 172.21.2.101:0/1417513013 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54840b7c20 msgr2=0x7f54840b8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:26.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.148 7f548b244700 1 --2- 172.21.2.101:0/1417513013 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54840b7c20 0x7f54840b8410 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f5480006f60 tx=0x7f548000b970).stop 2021-02-10T11:18:26.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.148 7f548b244700 1 -- 172.21.2.101:0/1417513013 shutdown_connections 2021-02-10T11:18:26.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.148 7f548b244700 1 --2- 172.21.2.101:0/1417513013 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54640186c0 0x7f546401abc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:26.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.148 7f548b244700 1 --2- 172.21.2.101:0/1417513013 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54840c25a0 0x7f54840c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:26.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.148 7f548b244700 1 --2- 172.21.2.101:0/1417513013 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54840b8a30 0x7f54840c1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:26.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.148 7f548b244700 1 --2- 172.21.2.101:0/1417513013 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54840b7c20 0x7f54840b8410 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:26.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.148 7f548b244700 1 -- 172.21.2.101:0/1417513013 >> 172.21.2.101:0/1417513013 conn(0x7f5484094e10 msgr2=0x7f54840a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:26.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.148 7f548b244700 1 -- 172.21.2.101:0/1417513013 shutdown_connections 2021-02-10T11:18:26.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.148 7f548b244700 1 -- 172.21.2.101:0/1417513013 wait complete. 2021-02-10T11:18:26.162 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839259 2021-02-10T11:18:26.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.569 7f25a2b87700 1 Processor -- start 2021-02-10T11:18:26.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.569 7f25a2b87700 1 -- start start 2021-02-10T11:18:26.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.570 7f25a2b87700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f259c1b0470 0x7f259c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:26.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.570 7f25a2b87700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f259c0ef610 0x7f259c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:26.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.570 7f25a2b87700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f259c0f0030 0x7f259c0f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:26.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.570 7f25a2b87700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f259c0e5ea0 con 0x7f259c0ef610 2021-02-10T11:18:26.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.570 7f25a2b87700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f259c0f0b50 con 0x7f259c1b0470 2021-02-10T11:18:26.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.570 7f25a2b87700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259c0e52e0 con 0x7f259c0f0030 2021-02-10T11:18:26.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.570 7f259bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f259c0ef610 0x7f259c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:26.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.570 7f259bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f259c0ef610 0x7f259c0efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46276/0 (socket says 172.21.2.101:46276) 2021-02-10T11:18:26.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.570 7f259bfff700 1 -- 172.21.2.101:0/2199064129 learned_addr learned my addr 172.21.2.101:0/2199064129 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:18:26.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.570 7f25a11a4700 1 --2- 172.21.2.101:0/2199064129 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f259c0f0030 0x7f259c0f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:26.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.570 7f25a09a3700 1 --2- 172.21.2.101:0/2199064129 >> v2:172.21.2.108:3300/0 conn(0x7f259c1b0470 0x7f259c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:26.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.571 7f259bfff700 1 -- 172.21.2.101:0/2199064129 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f259c0f0030 msgr2=0x7f259c0f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:18:26.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.571 7f259bfff700 1 --2- 172.21.2.101:0/2199064129 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f259c0f0030 0x7f259c0f0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:26.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.571 7f259bfff700 1 -- 172.21.2.101:0/2199064129 >> v2:172.21.2.108:3300/0 conn(0x7f259c1b0470 msgr2=0x7f259c1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:18:26.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.571 7f259bfff700 1 --2- 172.21.2.101:0/2199064129 >> v2:172.21.2.108:3300/0 conn(0x7f259c1b0470 0x7f259c1b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:26.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.571 7f259bfff700 1 -- 172.21.2.101:0/2199064129 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259c0f10d0 con 0x7f259c0ef610 2021-02-10T11:18:26.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.571 7f25a09a3700 1 --2- 172.21.2.101:0/2199064129 >> v2:172.21.2.108:3300/0 conn(0x7f259c1b0470 0x7f259c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:18:26.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.571 7f259bfff700 1 --2- 172.21.2.101:0/2199064129 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f259c0ef610 0x7f259c0efa10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f258c006a80 tx=0x7f258c009f00).ready entity=mon.0 client_cookie=8ad95b775ca4f103 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:26.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.571 7f259b7fe700 1 -- 172.21.2.101:0/2199064129 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f258c00b5c0 con 0x7f259c0ef610 2021-02-10T11:18:26.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.571 7f259b7fe700 1 -- 172.21.2.101:0/2199064129 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c00b8c0 con 0x7f259c0ef610 2021-02-10T11:18:26.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.571 7f259b7fe700 1 -- 172.21.2.101:0/2199064129 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f258c00bde0 con 0x7f259c0ef610 2021-02-10T11:18:26.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.571 7f25a2b87700 1 -- 172.21.2.101:0/2199064129 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f259c0ef610 msgr2=0x7f259c0efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:26.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.571 7f25a2b87700 1 --2- 172.21.2.101:0/2199064129 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f259c0ef610 0x7f259c0efa10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f258c006a80 tx=0x7f258c009f00).stop 2021-02-10T11:18:26.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.572 7f25a2b87700 1 -- 172.21.2.101:0/2199064129 shutdown_connections 2021-02-10T11:18:26.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.572 7f25a2b87700 1 --2- 172.21.2.101:0/2199064129 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f259c0f0030 0x7f259c0f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:26.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.572 7f25a2b87700 1 --2- 172.21.2.101:0/2199064129 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f259c0ef610 0x7f259c0efa10 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:26.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.572 7f25a2b87700 1 --2- 172.21.2.101:0/2199064129 >> v2:172.21.2.108:3300/0 conn(0x7f259c1b0470 0x7f259c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:26.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.572 7f25a2b87700 1 -- 172.21.2.101:0/2199064129 >> 172.21.2.101:0/2199064129 conn(0x7f259c0ce3e0 msgr2=0x7f259c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:26.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.572 7f25a2b87700 1 -- 172.21.2.101:0/2199064129 shutdown_connections 2021-02-10T11:18:26.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.572 7f25a2b87700 1 -- 172.21.2.101:0/2199064129 wait complete. 2021-02-10T11:18:26.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.587 7f25a2b87700 1 Processor -- start 2021-02-10T11:18:26.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.587 7f25a2b87700 1 -- start start 2021-02-10T11:18:26.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.587 7f25a2b87700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f259c09b930 0x7f259c0b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:26.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.587 7f25a2b87700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f259c0b8410 0x7f259c0b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:26.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.587 7f25a2b87700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f259c0c1f80 0x7f259c0c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:26.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.587 7f25a2b87700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c0c1a00 con 0x7f259c0c1f80 2021-02-10T11:18:26.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.587 7f25a2b87700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c0b8e30 con 0x7f259c09b930 2021-02-10T11:18:26.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.587 7f25a2b87700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259c0b9050 con 0x7f259c0b8410 2021-02-10T11:18:26.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.587 7f25a11a4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f259c0c1f80 0x7f259c0c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:26.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.588 7f25a11a4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f259c0c1f80 0x7f259c0c2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46278/0 (socket says 172.21.2.101:46278) 2021-02-10T11:18:26.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.588 7f25a11a4700 1 -- 172.21.2.101:0/2597111894 learned_addr learned my addr 172.21.2.101:0/2597111894 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:18:26.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.588 7f25a11a4700 1 -- 172.21.2.101:0/2597111894 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f259c0b8410 msgr2=0x7f259c0b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:26.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.588 7f259bfff700 1 --2- 172.21.2.101:0/2597111894 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f259c0b8410 0x7f259c0b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:26.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.588 7f25a11a4700 1 --2- 172.21.2.101:0/2597111894 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f259c0b8410 0x7f259c0b8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:26.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.588 7f25a11a4700 1 -- 172.21.2.101:0/2597111894 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f259c09b930 msgr2=0x7f259c0b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:26.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.588 7f25a09a3700 1 --2- 172.21.2.101:0/2597111894 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f259c09b930 0x7f259c0b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:26.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.588 7f25a11a4700 1 --2- 172.21.2.101:0/2597111894 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f259c09b930 0x7f259c0b7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:26.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.588 7f25a11a4700 1 -- 172.21.2.101:0/2597111894 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259c0c29a0 con 0x7f259c0c1f80 2021-02-10T11:18:26.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.588 7f25a09a3700 1 --2- 172.21.2.101:0/2597111894 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f259c09b930 0x7f259c0b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:18:26.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.588 7f25a11a4700 1 --2- 172.21.2.101:0/2597111894 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f259c0c1f80 0x7f259c0c2380 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f259400b580 tx=0x7f259400b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:26.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.589 7f25997fa700 1 -- 172.21.2.101:0/2597111894 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2594006240 con 0x7f259c0c1f80 2021-02-10T11:18:26.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.589 7f25997fa700 1 -- 172.21.2.101:0/2597111894 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2594006640 con 0x7f259c0c1f80 2021-02-10T11:18:26.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.589 7f25997fa700 1 -- 172.21.2.101:0/2597111894 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2594005330 con 0x7f259c0c1f80 2021-02-10T11:18:26.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.589 7f25a2b87700 1 -- 172.21.2.101:0/2597111894 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259c0c29a0 con 0x7f259c0c1f80 2021-02-10T11:18:26.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.589 7f25a2b87700 1 -- 172.21.2.101:0/2597111894 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259c0c29a0 con 0x7f259c0c1f80 2021-02-10T11:18:26.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.589 7f25997fa700 1 -- 172.21.2.101:0/2597111894 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2594006030 con 0x7f259c0c1f80 2021-02-10T11:18:26.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.589 7f25997fa700 1 --2- 172.21.2.101:0/2597111894 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25800186c0 0x7f258001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:26.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.589 7f25997fa700 1 -- 172.21.2.101:0/2597111894 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2594032550 con 0x7f259c0c1f80 2021-02-10T11:18:26.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.589 7f25a09a3700 1 --2- 172.21.2.101:0/2597111894 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25800186c0 0x7f258001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:26.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.590 7f25a09a3700 1 --2- 172.21.2.101:0/2597111894 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25800186c0 0x7f258001abc0 crc :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:26.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.619 7f25a2b87700 1 -- 172.21.2.101:0/2597111894 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f259c105b30 con 0x7f259c0c1f80 2021-02-10T11:18:26.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.622 7f25997fa700 1 -- 172.21.2.101:0/2597111894 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2594006820 con 0x7f259c0c1f80 2021-02-10T11:18:26.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.749 7f25a2b87700 1 -- 172.21.2.101:0/2597111894 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839259", "target": ["mgr", ""]}) v1 -- 0x7f259c105b30 con 0x7f25800186c0 2021-02-10T11:18:26.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.752 7f25997fa700 1 -- 172.21.2.101:0/2597111894 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f2584006ad0 con 0x7f25800186c0 2021-02-10T11:18:26.758 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839259/8dec98c6-92a5-477e-a19d-725065b4f1f7 2021-02-10T11:18:26.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.753 7f25a2b87700 1 -- 172.21.2.101:0/2597111894 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25800186c0 msgr2=0x7f258001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:26.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.753 7f25a2b87700 1 --2- 172.21.2.101:0/2597111894 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25800186c0 0x7f258001abc0 crc :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:26.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.753 7f25a2b87700 1 -- 172.21.2.101:0/2597111894 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f259c0c1f80 msgr2=0x7f259c0c2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:26.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.753 7f25a2b87700 1 --2- 172.21.2.101:0/2597111894 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f259c0c1f80 0x7f259c0c2380 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f259400b580 tx=0x7f259400b920).stop 2021-02-10T11:18:26.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.753 7f25a2b87700 1 -- 172.21.2.101:0/2597111894 shutdown_connections 2021-02-10T11:18:26.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.753 7f25a2b87700 1 --2- 172.21.2.101:0/2597111894 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25800186c0 0x7f258001abc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:26.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.753 7f25a2b87700 1 --2- 172.21.2.101:0/2597111894 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f259c0b8410 0x7f259c0b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:26.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.753 7f25a2b87700 1 --2- 172.21.2.101:0/2597111894 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f259c0c1f80 0x7f259c0c2380 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:26.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.753 7f25a2b87700 1 --2- 172.21.2.101:0/2597111894 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f259c09b930 0x7f259c0b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:26.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.753 7f25a2b87700 1 -- 172.21.2.101:0/2597111894 >> 172.21.2.101:0/2597111894 conn(0x7f259c094e10 msgr2=0x7f259c0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:26.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.754 7f25a2b87700 1 -- 172.21.2.101:0/2597111894 shutdown_connections 2021-02-10T11:18:26.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:26.754 7f25a2b87700 1 -- 172.21.2.101:0/2597111894 wait complete. 2021-02-10T11:18:26.768 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839259 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839259/8dec98c6-92a5-477e-a19d-725065b4f1f7 2021-02-10T11:18:26.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:26.768 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:26.768 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:27.103 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:27.103 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:27.103 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839259/8dec98c6-92a5-477e-a19d-725065b4f1f7/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:27.485 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:27.486 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:27.486 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0256003 s, 41.0 MB/s 2021-02-10T11:18:27.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:27.487 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:27.487 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:27.823 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:27.823 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:27.823 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839259/8dec98c6-92a5-477e-a19d-725065b4f1f7/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:28.196 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:28.196 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:28.196 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239755 s, 43.7 MB/s 2021-02-10T11:18:28.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:28.197 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:28.197 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:28.531 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:28.532 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:28.532 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839259/8dec98c6-92a5-477e-a19d-725065b4f1f7/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:28.907 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:28.907 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:28.907 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0290451 s, 36.1 MB/s 2021-02-10T11:18:28.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:28.909 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:28.909 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:29.242 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:29.242 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:29.243 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839259/8dec98c6-92a5-477e-a19d-725065b4f1f7/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231505 s, 45.3 MB/s 2021-02-10T11:18:29.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:29.616 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:29.616 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:29.950 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:29.951 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:29.951 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839259/8dec98c6-92a5-477e-a19d-725065b4f1f7/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:30.326 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:30.327 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:30.327 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0260143 s, 40.3 MB/s 2021-02-10T11:18:30.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:30.328 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:30.328 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:30.661 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:30.662 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:30.662 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839259/8dec98c6-92a5-477e-a19d-725065b4f1f7/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:31.035 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:31.035 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:31.036 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243701 s, 43.0 MB/s 2021-02-10T11:18:31.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:31.037 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:31.037 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:31.370 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:31.371 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:31.371 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839259/8dec98c6-92a5-477e-a19d-725065b4f1f7/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:31.745 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:31.746 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:31.746 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242695 s, 43.2 MB/s 2021-02-10T11:18:31.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:31.747 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:31.747 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:32.080 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:33.748 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:33.748 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839259/8dec98c6-92a5-477e-a19d-725065b4f1f7/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:34.120 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:34.120 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:34.120 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248704 s, 42.2 MB/s 2021-02-10T11:18:34.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:34.121 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:34.121 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:34.455 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:34.456 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:34.456 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839259/8dec98c6-92a5-477e-a19d-725065b4f1f7/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:34.830 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:34.830 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:34.830 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0230818 s, 45.4 MB/s 2021-02-10T11:18:34.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:34.831 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:34.831 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:35.167 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:35.167 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:35.167 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839259/8dec98c6-92a5-477e-a19d-725065b4f1f7/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:35.545 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:35.546 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:35.546 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0265261 s, 39.5 MB/s 2021-02-10T11:18:35.547 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839260 2021-02-10T11:18:35.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.954 7f5481f86700 1 Processor -- start 2021-02-10T11:18:35.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.954 7f5481f86700 1 -- start start 2021-02-10T11:18:35.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.955 7f5481f86700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f547c1b0470 0x7f547c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:35.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.955 7f5481f86700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f547c0f20e0 0x7f547c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:35.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.955 7f5481f86700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f547c0f0b50 con 0x7f547c0ef610 2021-02-10T11:18:35.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.955 7f5481f86700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f547c0f0820 con 0x7f547c0f20e0 2021-02-10T11:18:35.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.955 7f5481f86700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547c0f24e0 con 0x7f547c1b0470 2021-02-10T11:18:35.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.955 7f547affd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f547c0ef610 0x7f547c0efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46780/0 (socket says 172.21.2.101:46780) 2021-02-10T11:18:35.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.955 7f547affd700 1 -- 172.21.2.101:0/3638162609 learned_addr learned my addr 172.21.2.101:0/3638162609 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:18:35.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.955 7f547b7fe700 1 --2- 172.21.2.101:0/3638162609 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f547c1b0470 0x7f547c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:35.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.955 7f547a7fc700 1 -- 172.21.2.101:0/3638162609 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176268159 0 0) 0x7f546c0010b0 con 0x7f547c0ef610 2021-02-10T11:18:35.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.955 7f547a7fc700 1 -- 172.21.2.101:0/3638162609 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5468002fe0 con 0x7f547c0ef610 2021-02-10T11:18:35.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.955 7f547bfff700 1 --2- 172.21.2.101:0/3638162609 >> v2:172.21.2.108:3300/0 conn(0x7f547c0f20e0 0x7f547c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:35.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.956 7f547bfff700 1 -- 172.21.2.101:0/3638162609 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f547c1b0470 msgr2=0x7f547c1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:18:35.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.956 7f547bfff700 1 --2- 172.21.2.101:0/3638162609 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f547c1b0470 0x7f547c1b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.956 7f547bfff700 1 -- 172.21.2.101:0/3638162609 >> v1:172.21.2.101:6789/0 conn(0x7f547c0ef610 legacy=0x7f547c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.956 7f547bfff700 1 -- 172.21.2.101:0/3638162609 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547c0e52e0 con 0x7f547c0f20e0 2021-02-10T11:18:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.956 7f547b7fe700 1 --2- 172.21.2.101:0/3638162609 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f547c1b0470 0x7f547c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:18:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.956 7f547bfff700 1 --2- 172.21.2.101:0/3638162609 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f547c0f20e0 0x7f547c0f0200 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f5474006ad0 tx=0x7f5474009ad0).ready entity=mon.1 client_cookie=fbad13f6d142feb1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.956 7f547a7fc700 1 -- 172.21.2.101:0/3638162609 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5474000c10 con 0x7f547c0f20e0 2021-02-10T11:18:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.956 7f547a7fc700 1 -- 172.21.2.101:0/3638162609 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547400c0c0 con 0x7f547c0f20e0 2021-02-10T11:18:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.956 7f547a7fc700 1 -- 172.21.2.101:0/3638162609 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f547400c520 con 0x7f547c0f20e0 2021-02-10T11:18:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.956 7f5481f86700 1 -- 172.21.2.101:0/3638162609 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f547c0f20e0 msgr2=0x7f547c0f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.956 7f5481f86700 1 --2- 172.21.2.101:0/3638162609 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f547c0f20e0 0x7f547c0f0200 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f5474006ad0 tx=0x7f5474009ad0).stop 2021-02-10T11:18:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.957 7f5481f86700 1 -- 172.21.2.101:0/3638162609 shutdown_connections 2021-02-10T11:18:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.957 7f5481f86700 1 --2- 172.21.2.101:0/3638162609 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f547c0f20e0 0x7f547c0f0200 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.957 7f5481f86700 1 --2- 172.21.2.101:0/3638162609 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f547c1b0470 0x7f547c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:35.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.957 7f5481f86700 1 -- 172.21.2.101:0/3638162609 >> 172.21.2.101:0/3638162609 conn(0x7f547c0ce3e0 msgr2=0x7f547c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:35.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.957 7f5481f86700 1 -- 172.21.2.101:0/3638162609 shutdown_connections 2021-02-10T11:18:35.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.957 7f5481f86700 1 -- 172.21.2.101:0/3638162609 wait complete. 2021-02-10T11:18:35.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.972 7f5481f86700 1 Processor -- start 2021-02-10T11:18:35.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.972 7f5481f86700 1 -- start start 2021-02-10T11:18:35.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.972 7f5481f86700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f547c0b7de0 0x7f547c190920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:35.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.972 7f5481f86700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f547c0b82f0 0x7f547c0b86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:35.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.972 7f5481f86700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f547c0b8d10 0x7f547c0bf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:35.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.972 7f5481f86700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c0b7930 con 0x7f547c0b82f0 2021-02-10T11:18:35.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.972 7f5481f86700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c0b7b20 con 0x7f547c0b7de0 2021-02-10T11:18:35.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.972 7f5481f86700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547c0bffd0 con 0x7f547c0b8d10 2021-02-10T11:18:35.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.972 7f547affd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f547c0b82f0 0x7f547c0b86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:35.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.972 7f547affd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f547c0b82f0 0x7f547c0b86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46292/0 (socket says 172.21.2.101:46292) 2021-02-10T11:18:35.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.972 7f547affd700 1 -- 172.21.2.101:0/2323857421 learned_addr learned my addr 172.21.2.101:0/2323857421 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:18:35.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.973 7f547b7fe700 1 --2- 172.21.2.101:0/2323857421 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f547c0b7de0 0x7f547c190920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:35.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.973 7f547bfff700 1 --2- 172.21.2.101:0/2323857421 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f547c0b8d10 0x7f547c0bf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:35.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.973 7f547affd700 1 -- 172.21.2.101:0/2323857421 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f547c0b8d10 msgr2=0x7f547c0bf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:35.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.973 7f547affd700 1 --2- 172.21.2.101:0/2323857421 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f547c0b8d10 0x7f547c0bf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:35.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.973 7f547affd700 1 -- 172.21.2.101:0/2323857421 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f547c0b7de0 msgr2=0x7f547c190920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:35.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.973 7f547affd700 1 --2- 172.21.2.101:0/2323857421 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f547c0b7de0 0x7f547c190920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:35.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.973 7f547affd700 1 -- 172.21.2.101:0/2323857421 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547c0c01c0 con 0x7f547c0b82f0 2021-02-10T11:18:35.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.973 7f547affd700 1 --2- 172.21.2.101:0/2323857421 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f547c0b82f0 0x7f547c0b86f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f546c001c60 tx=0x7f546c00a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:35.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.974 7f5473fff700 1 -- 172.21.2.101:0/2323857421 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f546c00acc0 con 0x7f547c0b82f0 2021-02-10T11:18:35.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.974 7f5473fff700 1 -- 172.21.2.101:0/2323857421 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546c004140 con 0x7f547c0b82f0 2021-02-10T11:18:35.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.974 7f5473fff700 1 -- 172.21.2.101:0/2323857421 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f546c004660 con 0x7f547c0b82f0 2021-02-10T11:18:35.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.974 7f5481f86700 1 -- 172.21.2.101:0/2323857421 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f547c0c01c0 con 0x7f547c0b82f0 2021-02-10T11:18:35.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.974 7f5481f86700 1 -- 172.21.2.101:0/2323857421 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547c0c01c0 con 0x7f547c0b82f0 2021-02-10T11:18:35.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.974 7f5473fff700 1 -- 172.21.2.101:0/2323857421 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f546c004320 con 0x7f547c0b82f0 2021-02-10T11:18:35.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.974 7f5473fff700 1 --2- 172.21.2.101:0/2323857421 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54600186d0 0x7f546001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:35.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.974 7f5473fff700 1 -- 172.21.2.101:0/2323857421 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f546c0327a0 con 0x7f547c0b82f0 2021-02-10T11:18:35.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.974 7f547b7fe700 1 --2- 172.21.2.101:0/2323857421 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54600186d0 0x7f546001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:35.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:35.975 7f547b7fe700 1 --2- 172.21.2.101:0/2323857421 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54600186d0 0x7f546001abd0 crc :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:36.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.005 7f5481f86700 1 -- 172.21.2.101:0/2323857421 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547c105b30 con 0x7f547c0b82f0 2021-02-10T11:18:36.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.008 7f5473fff700 1 -- 172.21.2.101:0/2323857421 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f546c000e20 con 0x7f547c0b82f0 2021-02-10T11:18:36.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.133 7f5481f86700 1 -- 172.21.2.101:0/2323857421 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839260", "target": ["mgr", ""]}) v1 -- 0x7f547c105b30 con 0x7f54600186d0 2021-02-10T11:18:36.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.157 7f5473fff700 1 -- 172.21.2.101:0/2323857421 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f54640068e0 con 0x7f54600186d0 2021-02-10T11:18:36.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.166 7f5481f86700 1 -- 172.21.2.101:0/2323857421 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54600186d0 msgr2=0x7f546001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:36.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.166 7f5481f86700 1 --2- 172.21.2.101:0/2323857421 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54600186d0 0x7f546001abd0 crc :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:36.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.166 7f5481f86700 1 -- 172.21.2.101:0/2323857421 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f547c0b82f0 msgr2=0x7f547c0b86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:36.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.166 7f5481f86700 1 --2- 172.21.2.101:0/2323857421 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f547c0b82f0 0x7f547c0b86f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f546c001c60 tx=0x7f546c00a8d0).stop 2021-02-10T11:18:36.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.166 7f5481f86700 1 -- 172.21.2.101:0/2323857421 shutdown_connections 2021-02-10T11:18:36.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.166 7f5481f86700 1 --2- 172.21.2.101:0/2323857421 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54600186d0 0x7f546001abd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:36.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.166 7f5481f86700 1 --2- 172.21.2.101:0/2323857421 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f547c0b8d10 0x7f547c0bf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:36.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.166 7f5481f86700 1 --2- 172.21.2.101:0/2323857421 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f547c0b82f0 0x7f547c0b86f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:36.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.166 7f5481f86700 1 --2- 172.21.2.101:0/2323857421 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f547c0b7de0 0x7f547c190920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:36.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.166 7f5481f86700 1 -- 172.21.2.101:0/2323857421 >> 172.21.2.101:0/2323857421 conn(0x7f547c094e10 msgr2=0x7f547c0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:36.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.166 7f5481f86700 1 -- 172.21.2.101:0/2323857421 shutdown_connections 2021-02-10T11:18:36.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.166 7f5481f86700 1 -- 172.21.2.101:0/2323857421 wait complete. 2021-02-10T11:18:36.180 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839260 2021-02-10T11:18:36.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.588 7f09ed346700 1 Processor -- start 2021-02-10T11:18:36.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.588 7f09ed346700 1 -- start start 2021-02-10T11:18:36.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.588 7f09ed346700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f09e81b0470 0x7f09e81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:36.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.589 7f09ed346700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09e80f20e0 0x7f09e80f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:36.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.589 7f09ed346700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e80f0b50 con 0x7f09e80ef610 2021-02-10T11:18:36.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.589 7f09ed346700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e80f0820 con 0x7f09e81b0470 2021-02-10T11:18:36.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.589 7f09ed346700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e80f24e0 con 0x7f09e80f20e0 2021-02-10T11:18:36.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.589 7f09e707d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f09e81b0470 0x7f09e81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:36.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.589 7f09e687c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f09e80ef610 0x7f09e80efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46794/0 (socket says 172.21.2.101:46794) 2021-02-10T11:18:36.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.589 7f09e687c700 1 -- 172.21.2.101:0/2821281228 learned_addr learned my addr 172.21.2.101:0/2821281228 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:18:36.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.589 7f09e787e700 1 --2- 172.21.2.101:0/2821281228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09e80f20e0 0x7f09e80f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:36.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.589 7f09e707d700 1 -- 172.21.2.101:0/2821281228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09e80f20e0 msgr2=0x7f09e80f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:18:36.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.589 7f09e607b700 1 -- 172.21.2.101:0/2821281228 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329862974 0 0) 0x7f09dc0010b0 con 0x7f09e80ef610 2021-02-10T11:18:36.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.589 7f09e707d700 1 --2- 172.21.2.101:0/2821281228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09e80f20e0 0x7f09e80f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:36.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.589 7f09e707d700 1 -- 172.21.2.101:0/2821281228 >> v1:172.21.2.101:6789/0 conn(0x7f09e80ef610 legacy=0x7f09e80efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:36.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.589 7f09e707d700 1 -- 172.21.2.101:0/2821281228 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e80e52e0 con 0x7f09e81b0470 2021-02-10T11:18:36.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.589 7f09e707d700 1 --2- 172.21.2.101:0/2821281228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09e81b0470 0x7f09e81b0870 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f09d8006650 tx=0x7f09d8009a80).ready entity=mon.0 client_cookie=40e5f572ef13d3c3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:36.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.590 7f09e607b700 1 -- 172.21.2.101:0/2821281228 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f09d80009b0 con 0x7f09e81b0470 2021-02-10T11:18:36.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.590 7f09e607b700 1 -- 172.21.2.101:0/2821281228 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d800c0c0 con 0x7f09e81b0470 2021-02-10T11:18:36.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.590 7f09e607b700 1 -- 172.21.2.101:0/2821281228 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f09d800c5e0 con 0x7f09e81b0470 2021-02-10T11:18:36.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.590 7f09ed346700 1 -- 172.21.2.101:0/2821281228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09e81b0470 msgr2=0x7f09e81b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:36.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.590 7f09ed346700 1 --2- 172.21.2.101:0/2821281228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09e81b0470 0x7f09e81b0870 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f09d8006650 tx=0x7f09d8009a80).stop 2021-02-10T11:18:36.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.591 7f09ed346700 1 -- 172.21.2.101:0/2821281228 shutdown_connections 2021-02-10T11:18:36.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.591 7f09ed346700 1 --2- 172.21.2.101:0/2821281228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09e80f20e0 0x7f09e80f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:36.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.591 7f09ed346700 1 --2- 172.21.2.101:0/2821281228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09e81b0470 0x7f09e81b0870 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:36.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.591 7f09ed346700 1 -- 172.21.2.101:0/2821281228 >> 172.21.2.101:0/2821281228 conn(0x7f09e80ce3e0 msgr2=0x7f09e80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:36.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.591 7f09ed346700 1 -- 172.21.2.101:0/2821281228 shutdown_connections 2021-02-10T11:18:36.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.591 7f09ed346700 1 -- 172.21.2.101:0/2821281228 wait complete. 2021-02-10T11:18:36.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.605 7f09ed346700 1 Processor -- start 2021-02-10T11:18:36.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.606 7f09ed346700 1 -- start start 2021-02-10T11:18:36.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.606 7f09ed346700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09e808ece0 0x7f09e80a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:36.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.606 7f09ed346700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09e80a6860 0x7f09e80a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:36.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.606 7f09ed346700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09e80a7280 0x7f09e80a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:36.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.606 7f09ed346700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e80a5cc0 con 0x7f09e80a6860 2021-02-10T11:18:36.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.606 7f09ed346700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e808f0e0 con 0x7f09e80a7280 2021-02-10T11:18:36.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.606 7f09ed346700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e808f2c0 con 0x7f09e808ece0 2021-02-10T11:18:36.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.606 7f09e687c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09e80a6860 0x7f09e80a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:36.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.606 7f09e687c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09e80a6860 0x7f09e80a6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46306/0 (socket says 172.21.2.101:46306) 2021-02-10T11:18:36.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.606 7f09e687c700 1 -- 172.21.2.101:0/1615137630 learned_addr learned my addr 172.21.2.101:0/1615137630 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:18:36.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.607 7f09e687c700 1 -- 172.21.2.101:0/1615137630 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09e808ece0 msgr2=0x7f09e80a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:36.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.607 7f09e687c700 1 --2- 172.21.2.101:0/1615137630 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09e808ece0 0x7f09e80a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:36.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.607 7f09e687c700 1 -- 172.21.2.101:0/1615137630 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09e80a7280 msgr2=0x7f09e80a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:36.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.607 7f09e687c700 1 --2- 172.21.2.101:0/1615137630 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09e80a7280 0x7f09e80a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:36.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.607 7f09e687c700 1 -- 172.21.2.101:0/1615137630 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e80a4bb0 con 0x7f09e80a6860 2021-02-10T11:18:36.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.607 7f09e687c700 1 --2- 172.21.2.101:0/1615137630 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09e80a6860 0x7f09e80a6c60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f09dc001c10 tx=0x7f09dc00a8f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:36.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.607 7f09cffff700 1 -- 172.21.2.101:0/1615137630 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f09dc0009b0 con 0x7f09e80a6860 2021-02-10T11:18:36.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.607 7f09cffff700 1 -- 172.21.2.101:0/1615137630 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09dc0012c0 con 0x7f09e80a6860 2021-02-10T11:18:36.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.607 7f09cffff700 1 -- 172.21.2.101:0/1615137630 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f09dc004350 con 0x7f09e80a6860 2021-02-10T11:18:36.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.607 7f09ed346700 1 -- 172.21.2.101:0/1615137630 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e80a4bb0 con 0x7f09e80a6860 2021-02-10T11:18:36.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.607 7f09ed346700 1 -- 172.21.2.101:0/1615137630 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e80a5140 con 0x7f09e80a6860 2021-02-10T11:18:36.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.608 7f09cffff700 1 -- 172.21.2.101:0/1615137630 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f09dc004140 con 0x7f09e80a6860 2021-02-10T11:18:36.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.608 7f09cffff700 1 --2- 172.21.2.101:0/1615137630 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09c80186c0 0x7f09c801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:36.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.608 7f09cffff700 1 -- 172.21.2.101:0/1615137630 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f09dc0323c0 con 0x7f09e80a6860 2021-02-10T11:18:36.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.608 7f09e707d700 1 --2- 172.21.2.101:0/1615137630 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09c80186c0 0x7f09c801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:36.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.608 7f09e707d700 1 --2- 172.21.2.101:0/1615137630 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09c80186c0 0x7f09c801abc0 crc :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:36.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.638 7f09ed346700 1 -- 172.21.2.101:0/1615137630 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e8105ae0 con 0x7f09e80a6860 2021-02-10T11:18:36.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.641 7f09cffff700 1 -- 172.21.2.101:0/1615137630 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f09dc024b80 con 0x7f09e80a6860 2021-02-10T11:18:36.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.769 7f09ed346700 1 -- 172.21.2.101:0/1615137630 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839260", "target": ["mgr", ""]}) v1 -- 0x7f09e80f0b50 con 0x7f09c80186c0 2021-02-10T11:18:36.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.772 7f09cffff700 1 -- 172.21.2.101:0/1615137630 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f09d8006060 con 0x7f09c80186c0 2021-02-10T11:18:36.778 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839260/156dbdb9-1adc-4af5-a604-8920c362e596 2021-02-10T11:18:36.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.774 7f09ed346700 1 -- 172.21.2.101:0/1615137630 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09c80186c0 msgr2=0x7f09c801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:36.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.774 7f09ed346700 1 --2- 172.21.2.101:0/1615137630 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09c80186c0 0x7f09c801abc0 crc :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:36.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.774 7f09ed346700 1 -- 172.21.2.101:0/1615137630 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09e80a6860 msgr2=0x7f09e80a6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:36.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.774 7f09ed346700 1 --2- 172.21.2.101:0/1615137630 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09e80a6860 0x7f09e80a6c60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f09dc001c10 tx=0x7f09dc00a8f0).stop 2021-02-10T11:18:36.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.774 7f09ed346700 1 -- 172.21.2.101:0/1615137630 shutdown_connections 2021-02-10T11:18:36.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.774 7f09ed346700 1 --2- 172.21.2.101:0/1615137630 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09e80a7280 0x7f09e80a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:36.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.774 7f09ed346700 1 --2- 172.21.2.101:0/1615137630 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09e80a6860 0x7f09e80a6c60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:36.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.774 7f09ed346700 1 --2- 172.21.2.101:0/1615137630 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09c80186c0 0x7f09c801abc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:36.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.774 7f09ed346700 1 --2- 172.21.2.101:0/1615137630 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09e808ece0 0x7f09e80a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:36.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.774 7f09ed346700 1 -- 172.21.2.101:0/1615137630 >> 172.21.2.101:0/1615137630 conn(0x7f09e808ffe0 msgr2=0x7f09e80bf050 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:36.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.774 7f09ed346700 1 -- 172.21.2.101:0/1615137630 shutdown_connections 2021-02-10T11:18:36.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:36.774 7f09ed346700 1 -- 172.21.2.101:0/1615137630 wait complete. 2021-02-10T11:18:36.788 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839260 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839260/156dbdb9-1adc-4af5-a604-8920c362e596 2021-02-10T11:18:36.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:36.788 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:36.788 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:37.124 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:37.124 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:37.124 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839260/156dbdb9-1adc-4af5-a604-8920c362e596/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:37.504 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:37.504 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:37.504 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234123 s, 44.8 MB/s 2021-02-10T11:18:37.505 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:37.506 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:37.506 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:37.849 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:37.849 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:37.849 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839260/156dbdb9-1adc-4af5-a604-8920c362e596/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:38.225 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:38.225 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:38.225 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0224782 s, 46.6 MB/s 2021-02-10T11:18:38.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:38.226 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:38.226 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:38.564 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:38.564 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:38.564 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839260/156dbdb9-1adc-4af5-a604-8920c362e596/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:38.939 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251263 s, 41.7 MB/s 2021-02-10T11:18:38.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:38.940 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:38.940 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:39.277 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:39.277 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:39.277 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839260/156dbdb9-1adc-4af5-a604-8920c362e596/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:39.651 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:39.652 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:39.652 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231348 s, 45.3 MB/s 2021-02-10T11:18:39.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:39.653 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:39.653 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:39.986 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:39.987 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:39.987 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839260/156dbdb9-1adc-4af5-a604-8920c362e596/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:40.362 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:40.362 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:40.362 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239822 s, 43.7 MB/s 2021-02-10T11:18:40.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:40.364 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:40.364 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:40.699 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:40.699 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:40.699 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839260/156dbdb9-1adc-4af5-a604-8920c362e596/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:41.073 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:41.073 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:41.073 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0230451 s, 45.5 MB/s 2021-02-10T11:18:41.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:41.074 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:41.074 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:41.409 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:41.410 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:41.410 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839260/156dbdb9-1adc-4af5-a604-8920c362e596/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:41.785 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:41.785 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:41.785 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243323 s, 43.1 MB/s 2021-02-10T11:18:41.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:41.787 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:41.787 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:42.122 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:42.122 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:42.122 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839260/156dbdb9-1adc-4af5-a604-8920c362e596/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:42.495 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:42.495 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:42.495 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0228054 s, 46.0 MB/s 2021-02-10T11:18:42.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:42.497 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:42.497 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:42.830 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:42.831 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:42.831 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839260/156dbdb9-1adc-4af5-a604-8920c362e596/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:43.204 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:43.205 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:43.205 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238538 s, 44.0 MB/s 2021-02-10T11:18:43.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:43.206 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:43.206 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:43.540 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:43.540 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:43.540 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839260/156dbdb9-1adc-4af5-a604-8920c362e596/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:43.911 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:43.912 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:43.913 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0232898 s, 45.0 MB/s 2021-02-10T11:18:43.913 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839261 2021-02-10T11:18:44.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.323 7f9412dc1700 1 Processor -- start 2021-02-10T11:18:44.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.323 7f9412dc1700 1 -- start start 2021-02-10T11:18:44.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.323 7f9412dc1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f940c1b0470 0x7f940c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:44.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.323 7f9412dc1700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f940c0f0b50 con 0x7f940c0f20e0 2021-02-10T11:18:44.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.323 7f9412dc1700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f940c0f24e0 con 0x7f940c0ef610 2021-02-10T11:18:44.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.323 7f9412dc1700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f940c0e5ea0 con 0x7f940c1b0470 2021-02-10T11:18:44.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.323 7f94113de700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f940c0f20e0 0x7f940c0f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46810/0 (socket says 172.21.2.101:46810) 2021-02-10T11:18:44.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.323 7f94113de700 1 -- 172.21.2.101:0/3934010278 learned_addr learned my addr 172.21.2.101:0/3934010278 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:18:44.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.323 7f9410bdd700 1 --2- 172.21.2.101:0/3934010278 >> v2:172.21.2.101:3300/0 conn(0x7f940c1b0470 0x7f940c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:44.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.324 7f940b7fe700 1 -- 172.21.2.101:0/3934010278 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 43652957 0 0) 0x7f9404001180 con 0x7f940c0f20e0 2021-02-10T11:18:44.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.324 7f940b7fe700 1 -- 172.21.2.101:0/3934010278 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93f8002fe0 con 0x7f940c0f20e0 2021-02-10T11:18:44.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.324 7f9410bdd700 1 -- 172.21.2.101:0/3934010278 >> v1:172.21.2.108:6789/0 conn(0x7f940c0ef610 legacy=0x7f940c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:44.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.324 7f9410bdd700 1 -- 172.21.2.101:0/3934010278 >> v1:172.21.2.101:6789/0 conn(0x7f940c0f20e0 legacy=0x7f940c0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:44.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.324 7f9410bdd700 1 -- 172.21.2.101:0/3934010278 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f940c0e52e0 con 0x7f940c1b0470 2021-02-10T11:18:44.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.324 7f9410bdd700 1 --2- 172.21.2.101:0/3934010278 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f940c1b0470 0x7f940c1b0870 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f93f4006650 tx=0x7f93f4009a80).ready entity=mon.0 client_cookie=b1c417253a89a4e0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:44.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.324 7f940b7fe700 1 -- 172.21.2.101:0/3934010278 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f93f400c0c0 con 0x7f940c1b0470 2021-02-10T11:18:44.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.324 7f940b7fe700 1 -- 172.21.2.101:0/3934010278 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f400c340 con 0x7f940c1b0470 2021-02-10T11:18:44.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.324 7f940b7fe700 1 -- 172.21.2.101:0/3934010278 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f93f400c860 con 0x7f940c1b0470 2021-02-10T11:18:44.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.324 7f9412dc1700 1 -- 172.21.2.101:0/3934010278 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f940c1b0470 msgr2=0x7f940c1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:44.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.324 7f9412dc1700 1 --2- 172.21.2.101:0/3934010278 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f940c1b0470 0x7f940c1b0870 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f93f4006650 tx=0x7f93f4009a80).stop 2021-02-10T11:18:44.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.325 7f9412dc1700 1 -- 172.21.2.101:0/3934010278 shutdown_connections 2021-02-10T11:18:44.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.325 7f9412dc1700 1 --2- 172.21.2.101:0/3934010278 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f940c1b0470 0x7f940c1b0870 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:44.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.325 7f9412dc1700 1 -- 172.21.2.101:0/3934010278 >> 172.21.2.101:0/3934010278 conn(0x7f940c0ce3e0 msgr2=0x7f940c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:44.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.325 7f9412dc1700 1 -- 172.21.2.101:0/3934010278 shutdown_connections 2021-02-10T11:18:44.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.325 7f9412dc1700 1 -- 172.21.2.101:0/3934010278 wait complete. 2021-02-10T11:18:44.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.339 7f9412dc1700 1 Processor -- start 2021-02-10T11:18:44.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.339 7f9412dc1700 1 -- start start 2021-02-10T11:18:44.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.340 7f9412dc1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f940c0c36a0 0x7f940c0c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:44.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.340 7f9412dc1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f940c0c3190 0x7f940c0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:44.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.340 7f9412dc1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f940c0b2370 0x7f940c0b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:44.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.340 7f9412dc1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c0b27c0 con 0x7f940c0b2370 2021-02-10T11:18:44.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.340 7f9412dc1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c0b3360 con 0x7f940c0c36a0 2021-02-10T11:18:44.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.340 7f9412dc1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f940c0b3540 con 0x7f940c0c3190 2021-02-10T11:18:44.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.340 7f94113de700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f940c0b2370 0x7f940c0b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:44.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.340 7f94113de700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f940c0b2370 0x7f940c0b2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46322/0 (socket says 172.21.2.101:46322) 2021-02-10T11:18:44.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.340 7f94113de700 1 -- 172.21.2.101:0/2920462385 learned_addr learned my addr 172.21.2.101:0/2920462385 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:18:44.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.340 7f9410bdd700 1 --2- 172.21.2.101:0/2920462385 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f940c0c36a0 0x7f940c0c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:44.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.340 7f940bfff700 1 --2- 172.21.2.101:0/2920462385 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f940c0c3190 0x7f940c0b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:44.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.341 7f94113de700 1 -- 172.21.2.101:0/2920462385 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f940c0c3190 msgr2=0x7f940c0b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:44.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.341 7f94113de700 1 --2- 172.21.2.101:0/2920462385 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f940c0c3190 0x7f940c0b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:44.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.341 7f94113de700 1 -- 172.21.2.101:0/2920462385 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f940c0c36a0 msgr2=0x7f940c0c2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.341 7f94113de700 1 --2- 172.21.2.101:0/2920462385 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f940c0c36a0 0x7f940c0c2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.341 7f94113de700 1 -- 172.21.2.101:0/2920462385 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f940c0b3760 con 0x7f940c0b2370 2021-02-10T11:18:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.341 7f9410bdd700 1 --2- 172.21.2.101:0/2920462385 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f940c0c36a0 0x7f940c0c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:18:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.341 7f94113de700 1 --2- 172.21.2.101:0/2920462385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f940c0b2370 0x7f940c0b2d40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f9404006f00 tx=0x7f940400b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.341 7f94097fa700 1 -- 172.21.2.101:0/2920462385 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9404006b90 con 0x7f940c0b2370 2021-02-10T11:18:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.341 7f94097fa700 1 -- 172.21.2.101:0/2920462385 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9404006030 con 0x7f940c0b2370 2021-02-10T11:18:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.341 7f9412dc1700 1 -- 172.21.2.101:0/2920462385 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f940c0b3760 con 0x7f940c0b2370 2021-02-10T11:18:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.341 7f94097fa700 1 -- 172.21.2.101:0/2920462385 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9404004590 con 0x7f940c0b2370 2021-02-10T11:18:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.341 7f9412dc1700 1 -- 172.21.2.101:0/2920462385 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f940c0b3cf0 con 0x7f940c0b2370 2021-02-10T11:18:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.341 7f94097fa700 1 -- 172.21.2.101:0/2920462385 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f94040043a0 con 0x7f940c0b2370 2021-02-10T11:18:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.341 7f94097fa700 1 --2- 172.21.2.101:0/2920462385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93f00186c0 0x7f93f001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.342 7f94097fa700 1 -- 172.21.2.101:0/2920462385 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f94040327a0 con 0x7f940c0b2370 2021-02-10T11:18:44.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.342 7f9410bdd700 1 --2- 172.21.2.101:0/2920462385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93f00186c0 0x7f93f001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:44.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.342 7f9410bdd700 1 --2- 172.21.2.101:0/2920462385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93f00186c0 0x7f93f001abc0 crc :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:44.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.372 7f9412dc1700 1 -- 172.21.2.101:0/2920462385 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f940c0edd10 con 0x7f940c0b2370 2021-02-10T11:18:44.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.375 7f94097fa700 1 -- 172.21.2.101:0/2920462385 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9404000bc0 con 0x7f940c0b2370 2021-02-10T11:18:44.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.502 7f9412dc1700 1 -- 172.21.2.101:0/2920462385 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839261", "target": ["mgr", ""]}) v1 -- 0x7f940c0b27c0 con 0x7f93f00186c0 2021-02-10T11:18:44.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.529 7f94097fa700 1 -- 172.21.2.101:0/2920462385 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f93f40093a0 con 0x7f93f00186c0 2021-02-10T11:18:44.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.535 7f9412dc1700 1 -- 172.21.2.101:0/2920462385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93f00186c0 msgr2=0x7f93f001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:44.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.535 7f9412dc1700 1 --2- 172.21.2.101:0/2920462385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93f00186c0 0x7f93f001abc0 crc :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:44.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.535 7f9412dc1700 1 -- 172.21.2.101:0/2920462385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f940c0b2370 msgr2=0x7f940c0b2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:44.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.535 7f9412dc1700 1 --2- 172.21.2.101:0/2920462385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f940c0b2370 0x7f940c0b2d40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f9404006f00 tx=0x7f940400b940).stop 2021-02-10T11:18:44.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.535 7f9412dc1700 1 -- 172.21.2.101:0/2920462385 shutdown_connections 2021-02-10T11:18:44.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.535 7f9412dc1700 1 --2- 172.21.2.101:0/2920462385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93f00186c0 0x7f93f001abc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:44.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.535 7f9412dc1700 1 --2- 172.21.2.101:0/2920462385 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f940c0c3190 0x7f940c0b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:44.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.535 7f9412dc1700 1 --2- 172.21.2.101:0/2920462385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f940c0b2370 0x7f940c0b2d40 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:44.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.535 7f9412dc1700 1 --2- 172.21.2.101:0/2920462385 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f940c0c36a0 0x7f940c0c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:44.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.535 7f9412dc1700 1 -- 172.21.2.101:0/2920462385 >> 172.21.2.101:0/2920462385 conn(0x7f940c09f630 msgr2=0x7f940c0a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:44.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.535 7f9412dc1700 1 -- 172.21.2.101:0/2920462385 shutdown_connections 2021-02-10T11:18:44.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.535 7f9412dc1700 1 -- 172.21.2.101:0/2920462385 wait complete. 2021-02-10T11:18:44.549 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839261 2021-02-10T11:18:44.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.956 7fe75f3f7700 1 Processor -- start 2021-02-10T11:18:44.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.956 7fe75f3f7700 1 -- start start 2021-02-10T11:18:44.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.957 7fe75f3f7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe7581b0470 0x7fe7581b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:44.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.957 7fe75f3f7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe7580ef610 0x7fe7580efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:44.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.957 7fe75f3f7700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7580f0b50 con 0x7fe7580f0030 2021-02-10T11:18:44.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.957 7fe75f3f7700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7580e52e0 con 0x7fe7581b0470 2021-02-10T11:18:44.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.957 7fe75f3f7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7580e5490 con 0x7fe7580ef610 2021-02-10T11:18:44.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.957 7fe75d213700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe7581b0470 0x7fe7581b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:44.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.957 7fe75d213700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe7581b0470 0x7fe7581b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46326/0 (socket says 172.21.2.101:46326) 2021-02-10T11:18:44.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.957 7fe75d213700 1 -- 172.21.2.101:0/876572851 learned_addr learned my addr 172.21.2.101:0/876572851 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:18:44.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.957 7fe75d213700 1 -- 172.21.2.101:0/876572851 >> v2:172.21.2.108:3300/0 conn(0x7fe7580ef610 msgr2=0x7fe7580efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:18:44.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.957 7fe75d213700 1 --2- 172.21.2.101:0/876572851 >> v2:172.21.2.108:3300/0 conn(0x7fe7580ef610 0x7fe7580efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:44.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.957 7fe75d213700 1 -- 172.21.2.101:0/876572851 >> v1:172.21.2.108:6789/0 conn(0x7fe7580f0030 legacy=0x7fe7580f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:18:44.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.957 7fe75d213700 1 -- 172.21.2.101:0/876572851 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7580f0430 con 0x7fe7581b0470 2021-02-10T11:18:44.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.958 7fe75d213700 1 --2- 172.21.2.101:0/876572851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7581b0470 0x7fe7581b0870 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7fe754006700 tx=0x7fe754009b00).ready entity=mon.0 client_cookie=15b2a36daf44dfb5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:44.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.958 7fe74bfff700 1 -- 172.21.2.101:0/876572851 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe75400c0c0 con 0x7fe7581b0470 2021-02-10T11:18:44.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.958 7fe74bfff700 1 -- 172.21.2.101:0/876572851 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75400c340 con 0x7fe7581b0470 2021-02-10T11:18:44.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.958 7fe74bfff700 1 -- 172.21.2.101:0/876572851 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe75400c820 con 0x7fe7581b0470 2021-02-10T11:18:44.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.958 7fe75f3f7700 1 -- 172.21.2.101:0/876572851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7581b0470 msgr2=0x7fe7581b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:44.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.958 7fe75f3f7700 1 --2- 172.21.2.101:0/876572851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7581b0470 0x7fe7581b0870 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7fe754006700 tx=0x7fe754009b00).stop 2021-02-10T11:18:44.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.958 7fe75f3f7700 1 -- 172.21.2.101:0/876572851 shutdown_connections 2021-02-10T11:18:44.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.958 7fe75f3f7700 1 --2- 172.21.2.101:0/876572851 >> v2:172.21.2.108:3300/0 conn(0x7fe7580ef610 0x7fe7580efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:44.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.958 7fe75f3f7700 1 --2- 172.21.2.101:0/876572851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7581b0470 0x7fe7581b0870 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:44.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.958 7fe75f3f7700 1 -- 172.21.2.101:0/876572851 >> 172.21.2.101:0/876572851 conn(0x7fe7580ce3e0 msgr2=0x7fe7580e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:44.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.959 7fe75f3f7700 1 -- 172.21.2.101:0/876572851 shutdown_connections 2021-02-10T11:18:44.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.959 7fe75f3f7700 1 -- 172.21.2.101:0/876572851 wait complete. 2021-02-10T11:18:44.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.973 7fe75f3f7700 1 Processor -- start 2021-02-10T11:18:44.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.973 7fe75f3f7700 1 -- start start 2021-02-10T11:18:44.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.974 7fe75f3f7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe758094fb0 0x7fe75809e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:44.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.974 7fe75f3f7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7580c2be0 0x7fe7580c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:44.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.974 7fe75f3f7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7580c3600 0x7fe7580a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.974 7fe75f3f7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75809ebf0 con 0x7fe7580c2be0 2021-02-10T11:18:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.974 7fe75f3f7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7580aa2a0 con 0x7fe758094fb0 2021-02-10T11:18:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.974 7fe75f3f7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7580aa4c0 con 0x7fe7580c3600 2021-02-10T11:18:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.974 7fe75ca12700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7580c2be0 0x7fe7580c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.974 7fe75ca12700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7580c2be0 0x7fe7580c2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46332/0 (socket says 172.21.2.101:46332) 2021-02-10T11:18:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.974 7fe75ca12700 1 -- 172.21.2.101:0/468230041 learned_addr learned my addr 172.21.2.101:0/468230041 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:18:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.974 7fe75ca12700 1 -- 172.21.2.101:0/468230041 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7580c3600 msgr2=0x7fe7580a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.974 7fe75ca12700 1 --2- 172.21.2.101:0/468230041 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7580c3600 0x7fe7580a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.974 7fe75ca12700 1 -- 172.21.2.101:0/468230041 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe758094fb0 msgr2=0x7fe75809e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.974 7fe75ca12700 1 --2- 172.21.2.101:0/468230041 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe758094fb0 0x7fe75809e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.974 7fe75ca12700 1 -- 172.21.2.101:0/468230041 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7580aa6e0 con 0x7fe7580c2be0 2021-02-10T11:18:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.974 7fe75ca12700 1 --2- 172.21.2.101:0/468230041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7580c2be0 0x7fe7580c2fe0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fe74c006f00 tx=0x7fe74c00b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.975 7fe749ffb700 1 -- 172.21.2.101:0/468230041 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe74c00bd30 con 0x7fe7580c2be0 2021-02-10T11:18:44.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.975 7fe749ffb700 1 -- 172.21.2.101:0/468230041 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c000c90 con 0x7fe7580c2be0 2021-02-10T11:18:44.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.975 7fe749ffb700 1 -- 172.21.2.101:0/468230041 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe74c0022a0 con 0x7fe7580c2be0 2021-02-10T11:18:44.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.975 7fe75f3f7700 1 -- 172.21.2.101:0/468230041 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7580aa6e0 con 0x7fe7580c2be0 2021-02-10T11:18:44.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.975 7fe75f3f7700 1 -- 172.21.2.101:0/468230041 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7580aa6e0 con 0x7fe7580c2be0 2021-02-10T11:18:44.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.975 7fe749ffb700 1 -- 172.21.2.101:0/468230041 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe74c005310 con 0x7fe7580c2be0 2021-02-10T11:18:44.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.975 7fe749ffb700 1 --2- 172.21.2.101:0/468230041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7380186c0 0x7fe73801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:44.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.975 7fe749ffb700 1 -- 172.21.2.101:0/468230041 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe74c0323e0 con 0x7fe7580c2be0 2021-02-10T11:18:44.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.976 7fe75d213700 1 --2- 172.21.2.101:0/468230041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7380186c0 0x7fe73801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:44.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:44.976 7fe75d213700 1 --2- 172.21.2.101:0/468230041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7380186c0 0x7fe73801abc0 crc :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:45.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:45.007 7fe75f3f7700 1 -- 172.21.2.101:0/468230041 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7580edd10 con 0x7fe7580c2be0 2021-02-10T11:18:45.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:45.009 7fe749ffb700 1 -- 172.21.2.101:0/468230041 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe74c02d0a0 con 0x7fe7580c2be0 2021-02-10T11:18:45.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:45.136 7fe75f3f7700 1 -- 172.21.2.101:0/468230041 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839261", "target": ["mgr", ""]}) v1 -- 0x7fe75809ebf0 con 0x7fe7380186c0 2021-02-10T11:18:45.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:45.140 7fe749ffb700 1 -- 172.21.2.101:0/468230041 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fe754009360 con 0x7fe7380186c0 2021-02-10T11:18:45.149 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839261/44678fdb-2a03-4a2d-9879-691971deacae 2021-02-10T11:18:45.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:45.145 7fe75f3f7700 1 -- 172.21.2.101:0/468230041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7380186c0 msgr2=0x7fe73801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:45.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:45.145 7fe75f3f7700 1 --2- 172.21.2.101:0/468230041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7380186c0 0x7fe73801abc0 crc :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:45.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:45.145 7fe75f3f7700 1 -- 172.21.2.101:0/468230041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7580c2be0 msgr2=0x7fe7580c2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:45.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:45.145 7fe75f3f7700 1 --2- 172.21.2.101:0/468230041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7580c2be0 0x7fe7580c2fe0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fe74c006f00 tx=0x7fe74c00b940).stop 2021-02-10T11:18:45.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:45.145 7fe75f3f7700 1 -- 172.21.2.101:0/468230041 shutdown_connections 2021-02-10T11:18:45.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:45.145 7fe75f3f7700 1 --2- 172.21.2.101:0/468230041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7380186c0 0x7fe73801abc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:45.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:45.145 7fe75f3f7700 1 --2- 172.21.2.101:0/468230041 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7580c3600 0x7fe7580a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:45.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:45.145 7fe75f3f7700 1 --2- 172.21.2.101:0/468230041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7580c2be0 0x7fe7580c2fe0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:45.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:45.145 7fe75f3f7700 1 --2- 172.21.2.101:0/468230041 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe758094fb0 0x7fe75809e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:45.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:45.145 7fe75f3f7700 1 -- 172.21.2.101:0/468230041 >> 172.21.2.101:0/468230041 conn(0x7fe75809f630 msgr2=0x7fe7580a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:45.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:45.145 7fe75f3f7700 1 -- 172.21.2.101:0/468230041 shutdown_connections 2021-02-10T11:18:45.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:45.146 7fe75f3f7700 1 -- 172.21.2.101:0/468230041 wait complete. 2021-02-10T11:18:45.159 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839261 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839261/44678fdb-2a03-4a2d-9879-691971deacae 2021-02-10T11:18:45.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:45.159 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:45.159 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:45.494 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:45.495 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:45.495 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839261/44678fdb-2a03-4a2d-9879-691971deacae/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:45.877 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:45.877 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:45.877 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236122 s, 44.4 MB/s 2021-02-10T11:18:45.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:45.878 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:45.878 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:46.214 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:46.214 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:46.215 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839261/44678fdb-2a03-4a2d-9879-691971deacae/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:46.590 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:46.590 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:46.590 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245583 s, 42.7 MB/s 2021-02-10T11:18:46.591 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:46.592 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:46.592 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:46.927 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:46.927 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:46.927 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839261/44678fdb-2a03-4a2d-9879-691971deacae/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:47.046 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:18:47.049 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:18:47.301 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:47.301 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:47.301 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243979 s, 43.0 MB/s 2021-02-10T11:18:47.302 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:47.302 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:47.303 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:47.637 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:47.637 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:47.637 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839261/44678fdb-2a03-4a2d-9879-691971deacae/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:48.012 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:48.013 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:48.013 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247615 s, 42.3 MB/s 2021-02-10T11:18:48.014 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:48.014 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:48.014 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:48.349 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:48.349 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:48.349 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839261/44678fdb-2a03-4a2d-9879-691971deacae/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:48.725 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:48.725 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:48.725 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247705 s, 42.3 MB/s 2021-02-10T11:18:48.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:48.726 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:48.726 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:49.060 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:49.061 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:49.061 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839261/44678fdb-2a03-4a2d-9879-691971deacae/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:49.435 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:49.435 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:49.435 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0255355 s, 41.1 MB/s 2021-02-10T11:18:49.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:49.436 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:49.436 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:49.769 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:49.770 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:49.770 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839261/44678fdb-2a03-4a2d-9879-691971deacae/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:50.144 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:50.144 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:50.145 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239577 s, 43.8 MB/s 2021-02-10T11:18:50.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:50.146 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:50.146 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:50.482 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:50.482 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:50.482 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839261/44678fdb-2a03-4a2d-9879-691971deacae/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:50.855 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:50.855 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:50.855 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023824 s, 44.0 MB/s 2021-02-10T11:18:50.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:50.856 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:50.857 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:51.191 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:51.191 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:51.192 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839261/44678fdb-2a03-4a2d-9879-691971deacae/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:51.567 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:51.567 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:51.567 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0250209 s, 41.9 MB/s 2021-02-10T11:18:51.568 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:51.568 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:51.569 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:51.904 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:51.904 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:51.904 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839261/44678fdb-2a03-4a2d-9879-691971deacae/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:52.278 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:52.278 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:52.278 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233373 s, 44.9 MB/s 2021-02-10T11:18:52.279 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839262 2021-02-10T11:18:52.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.686 7f6a83743700 1 Processor -- start 2021-02-10T11:18:52.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.687 7f6a83743700 1 -- start start 2021-02-10T11:18:52.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.687 7f6a83743700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6a7c1b0870 0x7f6a7c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:52.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.687 7f6a83743700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6a7c0efe00 0x7f6a7c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:52.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.687 7f6a83743700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c1b0c70 con 0x7f6a7c1b0470 2021-02-10T11:18:52.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.687 7f6a83743700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a7c0e5ea0 con 0x7f6a7c1b0870 2021-02-10T11:18:52.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.687 7f6a83743700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a7c0f0b50 con 0x7f6a7c0efe00 2021-02-10T11:18:52.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.687 7f6a80d5e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6a7c1b0870 0x7f6a7c0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:52.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.687 7f6a80d5e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6a7c1b0870 0x7f6a7c0f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46338/0 (socket says 172.21.2.101:46338) 2021-02-10T11:18:52.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.687 7f6a80d5e700 1 -- 172.21.2.101:0/490641486 learned_addr learned my addr 172.21.2.101:0/490641486 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:18:52.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.688 7f6a80d5e700 1 -- 172.21.2.101:0/490641486 >> v2:172.21.2.108:3300/0 conn(0x7f6a7c0efe00 msgr2=0x7f6a7c0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:18:52.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.688 7f6a80d5e700 1 --2- 172.21.2.101:0/490641486 >> v2:172.21.2.108:3300/0 conn(0x7f6a7c0efe00 0x7f6a7c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:52.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.688 7f6a80d5e700 1 -- 172.21.2.101:0/490641486 >> v1:172.21.2.101:6789/0 conn(0x7f6a7c1b0470 legacy=0x7f6a7c0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:52.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.688 7f6a80d5e700 1 -- 172.21.2.101:0/490641486 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c0f0820 con 0x7f6a7c1b0870 2021-02-10T11:18:52.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.688 7f6a80d5e700 1 --2- 172.21.2.101:0/490641486 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a7c1b0870 0x7f6a7c0f20e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f6a70006700 tx=0x7f6a70009b80).ready entity=mon.0 client_cookie=8cb1c67ea3f877ed server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:52.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.688 7f6a6ffff700 1 -- 172.21.2.101:0/490641486 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a7000c0c0 con 0x7f6a7c1b0870 2021-02-10T11:18:52.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.688 7f6a6ffff700 1 -- 172.21.2.101:0/490641486 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7000c340 con 0x7f6a7c1b0870 2021-02-10T11:18:52.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.688 7f6a6ffff700 1 -- 172.21.2.101:0/490641486 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a7000c860 con 0x7f6a7c1b0870 2021-02-10T11:18:52.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.688 7f6a83743700 1 -- 172.21.2.101:0/490641486 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a7c1b0870 msgr2=0x7f6a7c0f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:52.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.688 7f6a83743700 1 --2- 172.21.2.101:0/490641486 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a7c1b0870 0x7f6a7c0f20e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f6a70006700 tx=0x7f6a70009b80).stop 2021-02-10T11:18:52.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.689 7f6a83743700 1 -- 172.21.2.101:0/490641486 shutdown_connections 2021-02-10T11:18:52.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.689 7f6a83743700 1 --2- 172.21.2.101:0/490641486 >> v2:172.21.2.108:3300/0 conn(0x7f6a7c0efe00 0x7f6a7c0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:52.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.689 7f6a83743700 1 --2- 172.21.2.101:0/490641486 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a7c1b0870 0x7f6a7c0f20e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:52.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.689 7f6a83743700 1 -- 172.21.2.101:0/490641486 >> 172.21.2.101:0/490641486 conn(0x7f6a7c0ce3e0 msgr2=0x7f6a7c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:52.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.689 7f6a83743700 1 -- 172.21.2.101:0/490641486 shutdown_connections 2021-02-10T11:18:52.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.689 7f6a83743700 1 -- 172.21.2.101:0/490641486 wait complete. 2021-02-10T11:18:52.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.704 7f6a83743700 1 Processor -- start 2021-02-10T11:18:52.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.704 7f6a83743700 1 -- start start 2021-02-10T11:18:52.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.704 7f6a83743700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a7c094fb0 0x7f6a7c09e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:52.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.704 7f6a83743700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a7c0c2be0 0x7f6a7c0c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:52.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.704 7f6a83743700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a7c0c3600 0x7f6a7c0a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:52.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.704 7f6a83743700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c09ebf0 con 0x7f6a7c094fb0 2021-02-10T11:18:52.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.704 7f6a83743700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0aa2a0 con 0x7f6a7c0c2be0 2021-02-10T11:18:52.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.704 7f6a83743700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0aa4c0 con 0x7f6a7c0c3600 2021-02-10T11:18:52.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.705 7f6a8155f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a7c094fb0 0x7f6a7c09e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:52.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.705 7f6a8155f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a7c094fb0 0x7f6a7c09e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46344/0 (socket says 172.21.2.101:46344) 2021-02-10T11:18:52.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.705 7f6a8155f700 1 -- 172.21.2.101:0/2417008897 learned_addr learned my addr 172.21.2.101:0/2417008897 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:18:52.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.705 7f6a81d60700 1 --2- 172.21.2.101:0/2417008897 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a7c0c3600 0x7f6a7c0a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.705 7f6a80d5e700 1 --2- 172.21.2.101:0/2417008897 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a7c0c2be0 0x7f6a7c0c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.705 7f6a8155f700 1 -- 172.21.2.101:0/2417008897 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a7c0c3600 msgr2=0x7f6a7c0a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.705 7f6a8155f700 1 --2- 172.21.2.101:0/2417008897 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a7c0c3600 0x7f6a7c0a9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.705 7f6a8155f700 1 -- 172.21.2.101:0/2417008897 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a7c0c2be0 msgr2=0x7f6a7c0c2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.705 7f6a8155f700 1 --2- 172.21.2.101:0/2417008897 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a7c0c2be0 0x7f6a7c0c2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.705 7f6a8155f700 1 -- 172.21.2.101:0/2417008897 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c0aa6e0 con 0x7f6a7c094fb0 2021-02-10T11:18:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.705 7f6a8155f700 1 --2- 172.21.2.101:0/2417008897 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a7c094fb0 0x7f6a7c09e5d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f6a78006f00 tx=0x7f6a7800b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.706 7f6a6dffb700 1 -- 172.21.2.101:0/2417008897 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a78007070 con 0x7f6a7c094fb0 2021-02-10T11:18:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.706 7f6a6dffb700 1 -- 172.21.2.101:0/2417008897 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a78007330 con 0x7f6a7c094fb0 2021-02-10T11:18:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.706 7f6a6dffb700 1 -- 172.21.2.101:0/2417008897 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a78007790 con 0x7f6a7c094fb0 2021-02-10T11:18:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.706 7f6a83743700 1 -- 172.21.2.101:0/2417008897 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7c0aa6e0 con 0x7f6a7c094fb0 2021-02-10T11:18:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.706 7f6a83743700 1 -- 172.21.2.101:0/2417008897 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c0aac70 con 0x7f6a7c094fb0 2021-02-10T11:18:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.706 7f6a6dffb700 1 -- 172.21.2.101:0/2417008897 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6a78007000 con 0x7f6a7c094fb0 2021-02-10T11:18:52.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.706 7f6a6dffb700 1 --2- 172.21.2.101:0/2417008897 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6a5c018700 0x7f6a5c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:52.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.706 7f6a6dffb700 1 -- 172.21.2.101:0/2417008897 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6a780326c0 con 0x7f6a7c094fb0 2021-02-10T11:18:52.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.706 7f6a80d5e700 1 --2- 172.21.2.101:0/2417008897 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6a5c018700 0x7f6a5c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:52.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.707 7f6a80d5e700 1 --2- 172.21.2.101:0/2417008897 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6a5c018700 0x7f6a5c01abc0 crc :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:52.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.737 7f6a83743700 1 -- 172.21.2.101:0/2417008897 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7c0a9750 con 0x7f6a7c094fb0 2021-02-10T11:18:52.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.740 7f6a6dffb700 1 -- 172.21.2.101:0/2417008897 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6a78007210 con 0x7f6a7c094fb0 2021-02-10T11:18:52.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.865 7f6a83743700 1 -- 172.21.2.101:0/2417008897 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839262", "target": ["mgr", ""]}) v1 -- 0x7f6a7c0a9750 con 0x7f6a5c018700 2021-02-10T11:18:52.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.892 7f6a6dffb700 1 -- 172.21.2.101:0/2417008897 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f6a700009b0 con 0x7f6a5c018700 2021-02-10T11:18:52.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.898 7f6a83743700 1 -- 172.21.2.101:0/2417008897 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6a5c018700 msgr2=0x7f6a5c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:52.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.898 7f6a83743700 1 --2- 172.21.2.101:0/2417008897 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6a5c018700 0x7f6a5c01abc0 crc :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:52.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.898 7f6a83743700 1 -- 172.21.2.101:0/2417008897 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a7c094fb0 msgr2=0x7f6a7c09e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:52.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.898 7f6a83743700 1 --2- 172.21.2.101:0/2417008897 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a7c094fb0 0x7f6a7c09e5d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f6a78006f00 tx=0x7f6a7800b940).stop 2021-02-10T11:18:52.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.899 7f6a83743700 1 -- 172.21.2.101:0/2417008897 shutdown_connections 2021-02-10T11:18:52.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.899 7f6a83743700 1 --2- 172.21.2.101:0/2417008897 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6a5c018700 0x7f6a5c01abc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:52.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.899 7f6a83743700 1 --2- 172.21.2.101:0/2417008897 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a7c0c3600 0x7f6a7c0a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:52.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.899 7f6a83743700 1 --2- 172.21.2.101:0/2417008897 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a7c0c2be0 0x7f6a7c0c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:52.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.899 7f6a83743700 1 --2- 172.21.2.101:0/2417008897 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a7c094fb0 0x7f6a7c09e5d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:52.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.899 7f6a83743700 1 -- 172.21.2.101:0/2417008897 >> 172.21.2.101:0/2417008897 conn(0x7f6a7c09f630 msgr2=0x7f6a7c0a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:52.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.899 7f6a83743700 1 -- 172.21.2.101:0/2417008897 shutdown_connections 2021-02-10T11:18:52.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:52.899 7f6a83743700 1 -- 172.21.2.101:0/2417008897 wait complete. 2021-02-10T11:18:52.913 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839262 2021-02-10T11:18:53.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.320 7f5d9be7f700 1 Processor -- start 2021-02-10T11:18:53.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.320 7f5d9be7f700 1 -- start start 2021-02-10T11:18:53.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.320 7f5d9be7f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d941b0470 0x7f5d941b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:53.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.320 7f5d9be7f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5d940ef610 0x7f5d940efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:53.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.320 7f5d9be7f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d940f0b50 con 0x7f5d940f0030 2021-02-10T11:18:53.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.320 7f5d9be7f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d940e52e0 con 0x7f5d940ef610 2021-02-10T11:18:53.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.320 7f5d9be7f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d940e5490 con 0x7f5d941b0470 2021-02-10T11:18:53.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.320 7f5d9949a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5d940ef610 0x7f5d940efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:53.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.320 7f5d9949a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5d940ef610 0x7f5d940efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46350/0 (socket says 172.21.2.101:46350) 2021-02-10T11:18:53.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.320 7f5d9a49c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f5d940f0030 0x7f5d940f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46844/0 (socket says 172.21.2.101:46844) 2021-02-10T11:18:53.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.320 7f5d9949a700 1 -- 172.21.2.101:0/3466654163 learned_addr learned my addr 172.21.2.101:0/3466654163 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:18:53.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.321 7f5d9949a700 1 -- 172.21.2.101:0/3466654163 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d941b0470 msgr2=0x7f5d941b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:18:53.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.321 7f5d9949a700 1 --2- 172.21.2.101:0/3466654163 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d941b0470 0x7f5d941b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:53.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.321 7f5d9949a700 1 -- 172.21.2.101:0/3466654163 >> v1:172.21.2.101:6789/0 conn(0x7f5d940f0030 legacy=0x7f5d940f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:53.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.321 7f5d9949a700 1 -- 172.21.2.101:0/3466654163 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d940f0430 con 0x7f5d940ef610 2021-02-10T11:18:53.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.321 7f5d98c99700 1 -- 172.21.2.101:0/3466654163 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082258205 0 0) 0x7f5d8c001480 con 0x7f5d940f0030 2021-02-10T11:18:53.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.321 7f5d9949a700 1 --2- 172.21.2.101:0/3466654163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d940ef610 0x7f5d940efa10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f5d84006700 tx=0x7f5d84009b00).ready entity=mon.0 client_cookie=341ba294342fb52d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:53.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.321 7f5d98c99700 1 -- 172.21.2.101:0/3466654163 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5d8400c0c0 con 0x7f5d940ef610 2021-02-10T11:18:53.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.321 7f5d98c99700 1 -- 172.21.2.101:0/3466654163 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8400c400 con 0x7f5d940ef610 2021-02-10T11:18:53.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.321 7f5d98c99700 1 -- 172.21.2.101:0/3466654163 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5d8400c860 con 0x7f5d940ef610 2021-02-10T11:18:53.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.321 7f5d9be7f700 1 -- 172.21.2.101:0/3466654163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d940ef610 msgr2=0x7f5d940efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:53.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.321 7f5d9be7f700 1 --2- 172.21.2.101:0/3466654163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d940ef610 0x7f5d940efa10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f5d84006700 tx=0x7f5d84009b00).stop 2021-02-10T11:18:53.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.322 7f5d9be7f700 1 -- 172.21.2.101:0/3466654163 shutdown_connections 2021-02-10T11:18:53.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.322 7f5d9be7f700 1 --2- 172.21.2.101:0/3466654163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d940ef610 0x7f5d940efa10 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:53.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.322 7f5d9be7f700 1 --2- 172.21.2.101:0/3466654163 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d941b0470 0x7f5d941b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:53.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.322 7f5d9be7f700 1 -- 172.21.2.101:0/3466654163 >> 172.21.2.101:0/3466654163 conn(0x7f5d940ce3e0 msgr2=0x7f5d940e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:53.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.322 7f5d9be7f700 1 -- 172.21.2.101:0/3466654163 shutdown_connections 2021-02-10T11:18:53.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.322 7f5d9be7f700 1 -- 172.21.2.101:0/3466654163 wait complete. 2021-02-10T11:18:53.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.337 7f5d9be7f700 1 Processor -- start 2021-02-10T11:18:53.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.337 7f5d9be7f700 1 -- start start 2021-02-10T11:18:53.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.337 7f5d9be7f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d940bf910 0x7f5d940bfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:53.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.337 7f5d9be7f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d940c0330 0x7f5d940c0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:53.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.337 7f5d9be7f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d940c13d0 0x7f5d940c17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:53.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.337 7f5d9be7f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d94109930 con 0x7f5d940c0330 2021-02-10T11:18:53.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.337 7f5d9be7f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d94109b50 con 0x7f5d940bf910 2021-02-10T11:18:53.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.337 7f5d9be7f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d940c1df0 con 0x7f5d940c13d0 2021-02-10T11:18:53.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.337 7f5d9949a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d940c0330 0x7f5d940c0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:53.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.337 7f5d9949a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d940c0330 0x7f5d940c0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46356/0 (socket says 172.21.2.101:46356) 2021-02-10T11:18:53.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.337 7f5d9949a700 1 -- 172.21.2.101:0/3613534412 learned_addr learned my addr 172.21.2.101:0/3613534412 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:18:53.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.338 7f5d9a49c700 1 --2- 172.21.2.101:0/3613534412 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d940c13d0 0x7f5d940c17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.338 7f5d9949a700 1 -- 172.21.2.101:0/3613534412 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d940c13d0 msgr2=0x7f5d940c17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.338 7f5d9949a700 1 --2- 172.21.2.101:0/3613534412 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d940c13d0 0x7f5d940c17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.338 7f5d9949a700 1 -- 172.21.2.101:0/3613534412 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d940bf910 msgr2=0x7f5d940bfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.338 7f5d9949a700 1 --2- 172.21.2.101:0/3613534412 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d940bf910 0x7f5d940bfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.338 7f5d9949a700 1 -- 172.21.2.101:0/3613534412 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d940c1fd0 con 0x7f5d940c0330 2021-02-10T11:18:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.338 7f5d9949a700 1 --2- 172.21.2.101:0/3613534412 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d940c0330 0x7f5d940c0730 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f5d8400b590 tx=0x7f5d8400b930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.338 7f5d8a7fc700 1 -- 172.21.2.101:0/3613534412 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5d8400bd20 con 0x7f5d940c0330 2021-02-10T11:18:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.338 7f5d8a7fc700 1 -- 172.21.2.101:0/3613534412 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d84006030 con 0x7f5d940c0330 2021-02-10T11:18:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.338 7f5d8a7fc700 1 -- 172.21.2.101:0/3613534412 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5d84005370 con 0x7f5d940c0330 2021-02-10T11:18:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.338 7f5d9be7f700 1 -- 172.21.2.101:0/3613534412 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d940c1fd0 con 0x7f5d940c0330 2021-02-10T11:18:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.338 7f5d9be7f700 1 -- 172.21.2.101:0/3613534412 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d940c1fd0 con 0x7f5d940c0330 2021-02-10T11:18:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.339 7f5d8a7fc700 1 -- 172.21.2.101:0/3613534412 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5d840052e0 con 0x7f5d940c0330 2021-02-10T11:18:53.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.339 7f5d8a7fc700 1 --2- 172.21.2.101:0/3613534412 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5d7c0186c0 0x7f5d7c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:18:53.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.339 7f5d8a7fc700 1 -- 172.21.2.101:0/3613534412 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5d840325b0 con 0x7f5d940c0330 2021-02-10T11:18:53.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.339 7f5d99c9b700 1 --2- 172.21.2.101:0/3613534412 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5d7c0186c0 0x7f5d7c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:18:53.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.340 7f5d99c9b700 1 --2- 172.21.2.101:0/3613534412 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5d7c0186c0 0x7f5d7c01abc0 crc :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:18:53.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.370 7f5d9be7f700 1 -- 172.21.2.101:0/3613534412 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d940c28d0 con 0x7f5d940c0330 2021-02-10T11:18:53.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.373 7f5d8a7fc700 1 -- 172.21.2.101:0/3613534412 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5d84006390 con 0x7f5d940c0330 2021-02-10T11:18:53.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.502 7f5d9be7f700 1 -- 172.21.2.101:0/3613534412 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839262", "target": ["mgr", ""]}) v1 -- 0x7f5d940c28d0 con 0x7f5d7c0186c0 2021-02-10T11:18:53.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.506 7f5d8a7fc700 1 -- 172.21.2.101:0/3613534412 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f5d90006e40 con 0x7f5d7c0186c0 2021-02-10T11:18:53.516 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839262/04dd816e-9957-4ff2-89ba-ce3179854a21 2021-02-10T11:18:53.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.511 7f5d9be7f700 1 -- 172.21.2.101:0/3613534412 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5d7c0186c0 msgr2=0x7f5d7c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:53.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.511 7f5d9be7f700 1 --2- 172.21.2.101:0/3613534412 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5d7c0186c0 0x7f5d7c01abc0 crc :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:53.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.511 7f5d9be7f700 1 -- 172.21.2.101:0/3613534412 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d940c0330 msgr2=0x7f5d940c0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:18:53.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.511 7f5d9be7f700 1 --2- 172.21.2.101:0/3613534412 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d940c0330 0x7f5d940c0730 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f5d8400b590 tx=0x7f5d8400b930).stop 2021-02-10T11:18:53.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.511 7f5d9be7f700 1 -- 172.21.2.101:0/3613534412 shutdown_connections 2021-02-10T11:18:53.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.511 7f5d9be7f700 1 --2- 172.21.2.101:0/3613534412 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5d7c0186c0 0x7f5d7c01abc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:53.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.511 7f5d9be7f700 1 --2- 172.21.2.101:0/3613534412 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d940c13d0 0x7f5d940c17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:53.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.511 7f5d9be7f700 1 --2- 172.21.2.101:0/3613534412 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d940c0330 0x7f5d940c0730 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:18:53.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.511 7f5d9be7f700 1 --2- 172.21.2.101:0/3613534412 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d940bf910 0x7f5d940bfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:18:53.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.511 7f5d9be7f700 1 -- 172.21.2.101:0/3613534412 >> 172.21.2.101:0/3613534412 conn(0x7f5d940bf4e0 msgr2=0x7f5d94191190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:18:53.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.511 7f5d9be7f700 1 -- 172.21.2.101:0/3613534412 shutdown_connections 2021-02-10T11:18:53.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:18:53.511 7f5d9be7f700 1 -- 172.21.2.101:0/3613534412 wait complete. 2021-02-10T11:18:53.526 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839262 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839262/04dd816e-9957-4ff2-89ba-ce3179854a21 2021-02-10T11:18:53.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:53.527 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:53.527 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:53.862 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:53.863 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:53.863 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839262/04dd816e-9957-4ff2-89ba-ce3179854a21/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:54.240 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:54.240 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:54.240 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236891 s, 44.3 MB/s 2021-02-10T11:18:54.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:54.241 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:54.242 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:54.578 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:54.578 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:54.578 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839262/04dd816e-9957-4ff2-89ba-ce3179854a21/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:54.950 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:54.950 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:54.950 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231171 s, 45.4 MB/s 2021-02-10T11:18:54.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:54.952 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:54.952 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:55.287 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:55.287 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:55.287 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839262/04dd816e-9957-4ff2-89ba-ce3179854a21/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:55.663 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:55.663 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:55.663 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0256135 s, 40.9 MB/s 2021-02-10T11:18:55.664 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:55.664 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:55.665 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:56.000 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:56.000 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:56.001 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839262/04dd816e-9957-4ff2-89ba-ce3179854a21/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:56.376 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:56.377 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:56.377 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242868 s, 43.2 MB/s 2021-02-10T11:18:56.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:56.378 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:56.378 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:56.713 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:56.713 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:56.713 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839262/04dd816e-9957-4ff2-89ba-ce3179854a21/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:57.089 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:57.089 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:57.089 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240936 s, 43.5 MB/s 2021-02-10T11:18:57.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:57.090 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:57.091 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:57.426 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:57.426 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:57.426 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839262/04dd816e-9957-4ff2-89ba-ce3179854a21/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:57.800 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:57.801 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:57.801 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024661 s, 42.5 MB/s 2021-02-10T11:18:57.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:57.803 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:57.803 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:58.137 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:58.137 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:58.137 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839262/04dd816e-9957-4ff2-89ba-ce3179854a21/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:58.510 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:58.510 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:58.510 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242672 s, 43.2 MB/s 2021-02-10T11:18:58.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:58.511 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:58.511 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:58.846 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:58.846 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:58.846 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839262/04dd816e-9957-4ff2-89ba-ce3179854a21/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:59.219 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:59.219 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:59.219 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249323 s, 42.1 MB/s 2021-02-10T11:18:59.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:59.220 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:59.220 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:18:59.555 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:18:59.555 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:18:59.555 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839262/04dd816e-9957-4ff2-89ba-ce3179854a21/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:18:59.929 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:18:59.929 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:18:59.929 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0222916 s, 47.0 MB/s 2021-02-10T11:18:59.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:18:59.930 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:18:59.930 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:00.265 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:00.265 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:00.265 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839262/04dd816e-9957-4ff2-89ba-ce3179854a21/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:00.641 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:00.641 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:00.641 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0250216 s, 41.9 MB/s 2021-02-10T11:19:00.642 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839263 2021-02-10T11:19:01.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.050 7fbe4ba24700 1 Processor -- start 2021-02-10T11:19:01.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.050 7fbe4ba24700 1 -- start start 2021-02-10T11:19:01.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.050 7fbe4ba24700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe441b0470 0x7fbe441b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:01.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.050 7fbe4ba24700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbe440ef610 0x7fbe440efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:01.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.050 7fbe4ba24700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe440f0b50 con 0x7fbe440f0030 2021-02-10T11:19:01.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.050 7fbe4ba24700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe440e52e0 con 0x7fbe440ef610 2021-02-10T11:19:01.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.050 7fbe4ba24700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe440e5490 con 0x7fbe441b0470 2021-02-10T11:19:01.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.051 7fbe4a041700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fbe440f0030 0x7fbe440f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46856/0 (socket says 172.21.2.101:46856) 2021-02-10T11:19:01.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.051 7fbe4a041700 1 -- 172.21.2.101:0/2912023294 learned_addr learned my addr 172.21.2.101:0/2912023294 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:19:01.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.051 7fbe4883e700 1 -- 172.21.2.101:0/2912023294 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 236628811 0 0) 0x7fbe3c001500 con 0x7fbe440f0030 2021-02-10T11:19:01.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.051 7fbe4883e700 1 -- 172.21.2.101:0/2912023294 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe30002fe0 con 0x7fbe440f0030 2021-02-10T11:19:01.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.051 7fbe49840700 1 --2- 172.21.2.101:0/2912023294 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe441b0470 0x7fbe441b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:01.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.051 7fbe4903f700 1 --2- 172.21.2.101:0/2912023294 >> v2:172.21.2.108:3300/0 conn(0x7fbe440ef610 0x7fbe440efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:01.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe4883e700 1 -- 172.21.2.101:0/2912023294 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2380532889 0 0) 0x7fbe3c001500 con 0x7fbe440f0030 2021-02-10T11:19:01.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe4883e700 1 -- 172.21.2.101:0/2912023294 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe441b0470 msgr2=0x7fbe441b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:19:01.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe4883e700 1 --2- 172.21.2.101:0/2912023294 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe441b0470 0x7fbe441b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:01.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe4883e700 1 -- 172.21.2.101:0/2912023294 >> v2:172.21.2.108:3300/0 conn(0x7fbe440ef610 msgr2=0x7fbe440efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:19:01.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe4883e700 1 --2- 172.21.2.101:0/2912023294 >> v2:172.21.2.108:3300/0 conn(0x7fbe440ef610 0x7fbe440efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:01.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe4883e700 1 -- 172.21.2.101:0/2912023294 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe440f0430 con 0x7fbe440f0030 2021-02-10T11:19:01.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe4883e700 1 -- 172.21.2.101:0/2912023294 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fbe3c001a00 con 0x7fbe440f0030 2021-02-10T11:19:01.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe4883e700 1 -- 172.21.2.101:0/2912023294 >> v1:172.21.2.101:6789/0 conn(0x7fbe440f0030 legacy=0x7fbe440f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:01.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe4883e700 1 --2- 172.21.2.101:0/2912023294 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe30003a60 0x7fbe30005f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:01.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe4883e700 1 --2- 172.21.2.101:0/2912023294 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbe3004a760 0x7fbe3004cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe4883e700 1 --2- 172.21.2.101:0/2912023294 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe3004d500 0x7fbe3004fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe4883e700 1 -- 172.21.2.101:0/2912023294 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe300502d0 con 0x7fbe3004d500 2021-02-10T11:19:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe4883e700 1 -- 172.21.2.101:0/2912023294 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe300504f0 con 0x7fbe3004a760 2021-02-10T11:19:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe4883e700 1 -- 172.21.2.101:0/2912023294 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe30050710 con 0x7fbe30003a60 2021-02-10T11:19:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe4a041700 1 --2- 172.21.2.101:0/2912023294 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe3004d500 0x7fbe3004fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe49840700 1 --2- 172.21.2.101:0/2912023294 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe30003a60 0x7fbe30005f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe4903f700 1 --2- 172.21.2.101:0/2912023294 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbe3004a760 0x7fbe3004cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe49840700 1 -- 172.21.2.101:0/2912023294 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbe3004a760 msgr2=0x7fbe3004cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:19:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe49840700 1 --2- 172.21.2.101:0/2912023294 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbe3004a760 0x7fbe3004cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe49840700 1 -- 172.21.2.101:0/2912023294 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe3004d500 msgr2=0x7fbe3004fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:19:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe49840700 1 --2- 172.21.2.101:0/2912023294 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe3004d500 0x7fbe3004fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe49840700 1 -- 172.21.2.101:0/2912023294 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe300509b0 con 0x7fbe30003a60 2021-02-10T11:19:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.052 7fbe4a041700 1 --2- 172.21.2.101:0/2912023294 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe3004d500 0x7fbe3004fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:19:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.053 7fbe49840700 1 --2- 172.21.2.101:0/2912023294 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe30003a60 0x7fbe30005f30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fbe40009310 tx=0x7fbe40009a90).ready entity=mon.2 client_cookie=b967aaf42475538c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.053 7fbe4883e700 1 -- 172.21.2.101:0/2912023294 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbe4000c0c0 con 0x7fbe30003a60 2021-02-10T11:19:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.053 7fbe4883e700 1 -- 172.21.2.101:0/2912023294 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe40006740 con 0x7fbe30003a60 2021-02-10T11:19:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.053 7fbe4883e700 1 -- 172.21.2.101:0/2912023294 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbe4000c5c0 con 0x7fbe30003a60 2021-02-10T11:19:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.053 7fbe4ba24700 1 -- 172.21.2.101:0/2912023294 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe30003a60 msgr2=0x7fbe30005f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.053 7fbe4ba24700 1 --2- 172.21.2.101:0/2912023294 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe30003a60 0x7fbe30005f30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fbe40009310 tx=0x7fbe40009a90).stop 2021-02-10T11:19:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.053 7fbe4a041700 1 -- 172.21.2.101:0/2912023294 reap_dead start 2021-02-10T11:19:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.053 7fbe4ba24700 1 -- 172.21.2.101:0/2912023294 shutdown_connections 2021-02-10T11:19:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.053 7fbe4ba24700 1 -- 172.21.2.101:0/2912023294 >> 172.21.2.101:0/2912023294 conn(0x7fbe440ce3e0 msgr2=0x7fbe440e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.054 7fbe4ba24700 1 -- 172.21.2.101:0/2912023294 shutdown_connections 2021-02-10T11:19:01.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.054 7fbe4ba24700 1 -- 172.21.2.101:0/2912023294 wait complete. 2021-02-10T11:19:01.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.068 7fbe4ba24700 1 Processor -- start 2021-02-10T11:19:01.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.068 7fbe4ba24700 1 -- start start 2021-02-10T11:19:01.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.069 7fbe4ba24700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbe440b7db0 0x7fbe440b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:01.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.069 7fbe4ba24700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe440b8880 0x7fbe441a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:01.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.069 7fbe4ba24700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe440b8c80 0x7fbe441a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:01.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.069 7fbe4ba24700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe440b79b0 con 0x7fbe440b8c80 2021-02-10T11:19:01.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.069 7fbe4ba24700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe441a8090 con 0x7fbe440b7db0 2021-02-10T11:19:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.069 7fbe4ba24700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe441a82b0 con 0x7fbe440b8880 2021-02-10T11:19:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.069 7fbe4a041700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe440b8c80 0x7fbe441a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.069 7fbe4a041700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe440b8c80 0x7fbe441a7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46374/0 (socket says 172.21.2.101:46374) 2021-02-10T11:19:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.069 7fbe4a041700 1 -- 172.21.2.101:0/28077734 learned_addr learned my addr 172.21.2.101:0/28077734 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:19:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.069 7fbe4a041700 1 -- 172.21.2.101:0/28077734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe440b8880 msgr2=0x7fbe441a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.069 7fbe4a041700 1 --2- 172.21.2.101:0/28077734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe440b8880 0x7fbe441a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.069 7fbe4a041700 1 -- 172.21.2.101:0/28077734 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbe440b7db0 msgr2=0x7fbe440b8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.069 7fbe4a041700 1 --2- 172.21.2.101:0/28077734 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbe440b7db0 0x7fbe440b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.069 7fbe4a041700 1 -- 172.21.2.101:0/28077734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe44190c10 con 0x7fbe440b8c80 2021-02-10T11:19:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.069 7fbe4a041700 1 --2- 172.21.2.101:0/28077734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe440b8c80 0x7fbe441a7a70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7fbe3c00a900 tx=0x7fbe3c00aca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.070 7fbe367fc700 1 -- 172.21.2.101:0/28077734 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbe3c004930 con 0x7fbe440b8c80 2021-02-10T11:19:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.070 7fbe367fc700 1 -- 172.21.2.101:0/28077734 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3c001270 con 0x7fbe440b8c80 2021-02-10T11:19:01.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.070 7fbe367fc700 1 -- 172.21.2.101:0/28077734 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbe3c0070d0 con 0x7fbe440b8c80 2021-02-10T11:19:01.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.070 7fbe4ba24700 1 -- 172.21.2.101:0/28077734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe44190c10 con 0x7fbe440b8c80 2021-02-10T11:19:01.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.070 7fbe4ba24700 1 -- 172.21.2.101:0/28077734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe44190c10 con 0x7fbe440b8c80 2021-02-10T11:19:01.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.070 7fbe367fc700 1 -- 172.21.2.101:0/28077734 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbe3c007240 con 0x7fbe440b8c80 2021-02-10T11:19:01.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.070 7fbe367fc700 1 --2- 172.21.2.101:0/28077734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe2c0186c0 0x7fbe2c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:01.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.070 7fbe367fc700 1 -- 172.21.2.101:0/28077734 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fbe3c0328c0 con 0x7fbe440b8c80 2021-02-10T11:19:01.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.070 7fbe49840700 1 --2- 172.21.2.101:0/28077734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe2c0186c0 0x7fbe2c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:01.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.071 7fbe49840700 1 --2- 172.21.2.101:0/28077734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe2c0186c0 0x7fbe2c01abc0 crc :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:01.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.100 7fbe4ba24700 1 -- 172.21.2.101:0/28077734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe44105b30 con 0x7fbe440b8c80 2021-02-10T11:19:01.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.103 7fbe367fc700 1 -- 172.21.2.101:0/28077734 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbe3c026f60 con 0x7fbe440b8c80 2021-02-10T11:19:01.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.230 7fbe4ba24700 1 -- 172.21.2.101:0/28077734 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839263", "target": ["mgr", ""]}) v1 -- 0x7fbe440f0b50 con 0x7fbe2c0186c0 2021-02-10T11:19:01.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.255 7fbe367fc700 1 -- 172.21.2.101:0/28077734 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fbe40000bd0 con 0x7fbe2c0186c0 2021-02-10T11:19:01.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.263 7fbe4ba24700 1 -- 172.21.2.101:0/28077734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe2c0186c0 msgr2=0x7fbe2c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:01.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.263 7fbe4ba24700 1 --2- 172.21.2.101:0/28077734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe2c0186c0 0x7fbe2c01abc0 crc :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:01.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.263 7fbe4ba24700 1 -- 172.21.2.101:0/28077734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe440b8c80 msgr2=0x7fbe441a7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:01.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.263 7fbe4ba24700 1 --2- 172.21.2.101:0/28077734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe440b8c80 0x7fbe441a7a70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7fbe3c00a900 tx=0x7fbe3c00aca0).stop 2021-02-10T11:19:01.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.263 7fbe4ba24700 1 -- 172.21.2.101:0/28077734 shutdown_connections 2021-02-10T11:19:01.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.263 7fbe4ba24700 1 --2- 172.21.2.101:0/28077734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe2c0186c0 0x7fbe2c01abc0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:01.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.263 7fbe4ba24700 1 --2- 172.21.2.101:0/28077734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe440b8880 0x7fbe441a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:01.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.263 7fbe4ba24700 1 --2- 172.21.2.101:0/28077734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe440b8c80 0x7fbe441a7a70 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:01.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.263 7fbe4ba24700 1 --2- 172.21.2.101:0/28077734 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbe440b7db0 0x7fbe440b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:01.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.263 7fbe4ba24700 1 -- 172.21.2.101:0/28077734 >> 172.21.2.101:0/28077734 conn(0x7fbe44094e10 msgr2=0x7fbe440a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:01.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.264 7fbe4ba24700 1 -- 172.21.2.101:0/28077734 shutdown_connections 2021-02-10T11:19:01.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.264 7fbe4ba24700 1 -- 172.21.2.101:0/28077734 wait complete. 2021-02-10T11:19:01.278 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839263 2021-02-10T11:19:01.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.685 7f22e638a700 1 Processor -- start 2021-02-10T11:19:01.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.685 7f22e638a700 1 -- start start 2021-02-10T11:19:01.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.685 7f22e638a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f22e01b0470 0x7f22e01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:01.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.685 7f22e638a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22e00f20e0 0x7f22e00f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:01.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.685 7f22e638a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e00f0b50 con 0x7f22e00ef610 2021-02-10T11:19:01.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.685 7f22e638a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e00f0820 con 0x7f22e01b0470 2021-02-10T11:19:01.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.685 7f22e638a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e00f24e0 con 0x7f22e00f20e0 2021-02-10T11:19:01.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.686 7f22e49a7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22e00f20e0 0x7f22e00f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:01.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.686 7f22dffff700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f22e01b0470 0x7f22e01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:01.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.686 7f22e49a7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22e00f20e0 0x7f22e00f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:39112/0 (socket says 172.21.2.101:39112) 2021-02-10T11:19:01.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.686 7f22e49a7700 1 -- 172.21.2.101:0/1926828335 learned_addr learned my addr 172.21.2.101:0/1926828335 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:19:01.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.686 7f22e49a7700 1 -- 172.21.2.101:0/1926828335 >> v2:172.21.2.108:3300/0 conn(0x7f22e01b0470 msgr2=0x7f22e01b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:19:01.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.686 7f22e49a7700 1 --2- 172.21.2.101:0/1926828335 >> v2:172.21.2.108:3300/0 conn(0x7f22e01b0470 0x7f22e01b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:01.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.686 7f22e49a7700 1 -- 172.21.2.101:0/1926828335 >> v1:172.21.2.108:6789/0 conn(0x7f22e00ef610 legacy=0x7f22e00efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:01.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.686 7f22e49a7700 1 -- 172.21.2.101:0/1926828335 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e00e52e0 con 0x7f22e00f20e0 2021-02-10T11:19:01.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.686 7f22dffff700 1 --2- 172.21.2.101:0/1926828335 >> v2:172.21.2.108:3300/0 conn(0x7f22e01b0470 0x7f22e01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:19:01.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.687 7f22e49a7700 1 --2- 172.21.2.101:0/1926828335 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22e00f20e0 0x7f22e00f0200 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f22d8006aa0 tx=0x7f22d8009f20).ready entity=mon.2 client_cookie=45ec36a3163e8270 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:01.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.687 7f22deffd700 1 -- 172.21.2.101:0/1926828335 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f22d800b5c0 con 0x7f22e00f20e0 2021-02-10T11:19:01.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.687 7f22deffd700 1 -- 172.21.2.101:0/1926828335 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d8003030 con 0x7f22e00f20e0 2021-02-10T11:19:01.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.687 7f22deffd700 1 -- 172.21.2.101:0/1926828335 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f22d800bab0 con 0x7f22e00f20e0 2021-02-10T11:19:01.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.687 7f22e638a700 1 -- 172.21.2.101:0/1926828335 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22e00f20e0 msgr2=0x7f22e00f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:01.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.687 7f22e638a700 1 --2- 172.21.2.101:0/1926828335 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22e00f20e0 0x7f22e00f0200 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f22d8006aa0 tx=0x7f22d8009f20).stop 2021-02-10T11:19:01.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.688 7f22e638a700 1 -- 172.21.2.101:0/1926828335 shutdown_connections 2021-02-10T11:19:01.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.688 7f22e638a700 1 --2- 172.21.2.101:0/1926828335 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22e00f20e0 0x7f22e00f0200 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:01.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.688 7f22e638a700 1 --2- 172.21.2.101:0/1926828335 >> v2:172.21.2.108:3300/0 conn(0x7f22e01b0470 0x7f22e01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:01.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.688 7f22e638a700 1 -- 172.21.2.101:0/1926828335 >> 172.21.2.101:0/1926828335 conn(0x7f22e00ce3e0 msgr2=0x7f22e00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:01.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.688 7f22e638a700 1 -- 172.21.2.101:0/1926828335 shutdown_connections 2021-02-10T11:19:01.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.688 7f22e638a700 1 -- 172.21.2.101:0/1926828335 wait complete. 2021-02-10T11:19:01.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.702 7f22e638a700 1 Processor -- start 2021-02-10T11:19:01.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.702 7f22e638a700 1 -- start start 2021-02-10T11:19:01.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.703 7f22e638a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22e00a0420 0x7f22e00a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:01.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.703 7f22e638a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22e0189d00 0x7f22e018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:01.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.703 7f22e638a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22e018a720 0x7f22e00b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:01.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.703 7f22e638a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e0189800 con 0x7f22e00a0420 2021-02-10T11:19:01.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.703 7f22e638a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e018ab20 con 0x7f22e018a720 2021-02-10T11:19:01.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.703 7f22e638a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e00a1140 con 0x7f22e0189d00 2021-02-10T11:19:01.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.703 7f22dffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22e00a0420 0x7f22e00a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:01.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.703 7f22dffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22e00a0420 0x7f22e00a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46384/0 (socket says 172.21.2.101:46384) 2021-02-10T11:19:01.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.703 7f22dffff700 1 -- 172.21.2.101:0/3372700119 learned_addr learned my addr 172.21.2.101:0/3372700119 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:19:01.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.703 7f22dffff700 1 -- 172.21.2.101:0/3372700119 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22e0189d00 msgr2=0x7f22e018a100 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:19:01.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.703 7f22dffff700 1 --2- 172.21.2.101:0/3372700119 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22e0189d00 0x7f22e018a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:01.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.703 7f22dffff700 1 -- 172.21.2.101:0/3372700119 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22e018a720 msgr2=0x7f22e00b7be0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:19:01.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.703 7f22dffff700 1 --2- 172.21.2.101:0/3372700119 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22e018a720 0x7f22e00b7be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:01.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.703 7f22dffff700 1 -- 172.21.2.101:0/3372700119 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e00b8200 con 0x7f22e00a0420 2021-02-10T11:19:01.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.703 7f22dffff700 1 --2- 172.21.2.101:0/3372700119 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22e00a0420 0x7f22e00a0820 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f22d0006f40 tx=0x7f22d000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:01.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.704 7f22dcff9700 1 -- 172.21.2.101:0/3372700119 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f22d000bd80 con 0x7f22e00a0420 2021-02-10T11:19:01.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.704 7f22dcff9700 1 -- 172.21.2.101:0/3372700119 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d0000c90 con 0x7f22e00a0420 2021-02-10T11:19:01.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.704 7f22dcff9700 1 -- 172.21.2.101:0/3372700119 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f22d0002220 con 0x7f22e00a0420 2021-02-10T11:19:01.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.704 7f22e638a700 1 -- 172.21.2.101:0/3372700119 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e00b8200 con 0x7f22e00a0420 2021-02-10T11:19:01.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.704 7f22e638a700 1 -- 172.21.2.101:0/3372700119 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e00b8200 con 0x7f22e00a0420 2021-02-10T11:19:01.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.704 7f22dcff9700 1 -- 172.21.2.101:0/3372700119 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f22d000a000 con 0x7f22e00a0420 2021-02-10T11:19:01.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.704 7f22dcff9700 1 --2- 172.21.2.101:0/3372700119 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22c00186c0 0x7f22c001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:01.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.704 7f22dcff9700 1 -- 172.21.2.101:0/3372700119 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f22d00327c0 con 0x7f22e00a0420 2021-02-10T11:19:01.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.704 7f22df7fe700 1 --2- 172.21.2.101:0/3372700119 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22c00186c0 0x7f22c001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:01.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.705 7f22df7fe700 1 --2- 172.21.2.101:0/3372700119 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22c00186c0 0x7f22c001abc0 crc :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:01.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.735 7f22e638a700 1 -- 172.21.2.101:0/3372700119 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e0105ae0 con 0x7f22e00a0420 2021-02-10T11:19:01.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.737 7f22dcff9700 1 -- 172.21.2.101:0/3372700119 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f22d000a000 con 0x7f22e00a0420 2021-02-10T11:19:01.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.865 7f22e638a700 1 -- 172.21.2.101:0/3372700119 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839263", "target": ["mgr", ""]}) v1 -- 0x7f22e018ab20 con 0x7f22c00186c0 2021-02-10T11:19:01.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.868 7f22dcff9700 1 -- 172.21.2.101:0/3372700119 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f22d4006b00 con 0x7f22c00186c0 2021-02-10T11:19:01.878 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839263/65ad6626-d1d5-4ef3-b330-93505056dd71 2021-02-10T11:19:01.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.873 7f22e638a700 1 -- 172.21.2.101:0/3372700119 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22c00186c0 msgr2=0x7f22c001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:01.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.873 7f22e638a700 1 --2- 172.21.2.101:0/3372700119 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22c00186c0 0x7f22c001abc0 crc :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:01.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.873 7f22e638a700 1 -- 172.21.2.101:0/3372700119 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22e00a0420 msgr2=0x7f22e00a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:01.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.873 7f22e638a700 1 --2- 172.21.2.101:0/3372700119 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22e00a0420 0x7f22e00a0820 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f22d0006f40 tx=0x7f22d000b990).stop 2021-02-10T11:19:01.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.874 7f22e638a700 1 -- 172.21.2.101:0/3372700119 shutdown_connections 2021-02-10T11:19:01.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.874 7f22e638a700 1 --2- 172.21.2.101:0/3372700119 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22c00186c0 0x7f22c001abc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:01.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.874 7f22e638a700 1 --2- 172.21.2.101:0/3372700119 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22e0189d00 0x7f22e018a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:01.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.874 7f22e638a700 1 --2- 172.21.2.101:0/3372700119 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22e018a720 0x7f22e00b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:01.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.874 7f22e638a700 1 --2- 172.21.2.101:0/3372700119 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22e00a0420 0x7f22e00a0820 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:01.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.874 7f22e638a700 1 -- 172.21.2.101:0/3372700119 >> 172.21.2.101:0/3372700119 conn(0x7f22e0094e10 msgr2=0x7f22e00a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:01.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.874 7f22e638a700 1 -- 172.21.2.101:0/3372700119 shutdown_connections 2021-02-10T11:19:01.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:01.874 7f22e638a700 1 -- 172.21.2.101:0/3372700119 wait complete. 2021-02-10T11:19:01.888 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839263 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839263/65ad6626-d1d5-4ef3-b330-93505056dd71 2021-02-10T11:19:01.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:01.888 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:01.888 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:02.223 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:02.223 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:02.223 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839263/65ad6626-d1d5-4ef3-b330-93505056dd71/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:02.602 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:02.602 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:02.602 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231282 s, 45.3 MB/s 2021-02-10T11:19:02.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:02.603 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:02.603 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:02.937 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:02.937 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:02.938 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839263/65ad6626-d1d5-4ef3-b330-93505056dd71/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:03.310 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:03.311 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:03.311 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0224249 s, 46.8 MB/s 2021-02-10T11:19:03.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:03.312 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:03.313 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:03.647 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:03.648 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:03.648 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839263/65ad6626-d1d5-4ef3-b330-93505056dd71/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:04.020 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:04.020 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:04.021 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253951 s, 41.3 MB/s 2021-02-10T11:19:04.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:04.022 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:04.022 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:04.356 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:04.357 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:04.357 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839263/65ad6626-d1d5-4ef3-b330-93505056dd71/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:04.733 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:04.734 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:04.734 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0257834 s, 40.7 MB/s 2021-02-10T11:19:04.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:04.735 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:04.735 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:05.070 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:05.070 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:05.070 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839263/65ad6626-d1d5-4ef3-b330-93505056dd71/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:05.444 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:05.444 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:05.444 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237874 s, 44.1 MB/s 2021-02-10T11:19:05.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:05.445 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:05.445 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:05.780 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:05.780 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:05.780 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839263/65ad6626-d1d5-4ef3-b330-93505056dd71/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:06.156 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:06.157 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:06.157 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243024 s, 43.1 MB/s 2021-02-10T11:19:06.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:06.158 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:06.158 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:06.491 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:06.492 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:06.492 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839263/65ad6626-d1d5-4ef3-b330-93505056dd71/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:06.867 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:06.867 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:06.867 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252085 s, 41.6 MB/s 2021-02-10T11:19:06.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:06.868 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:06.868 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:07.204 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:07.204 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:07.204 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839263/65ad6626-d1d5-4ef3-b330-93505056dd71/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:07.576 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:07.576 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:07.576 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0223834 s, 46.8 MB/s 2021-02-10T11:19:07.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:07.577 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:07.577 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:07.913 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:07.913 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:07.914 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839263/65ad6626-d1d5-4ef3-b330-93505056dd71/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:08.291 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:08.291 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:08.291 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0261593 s, 40.1 MB/s 2021-02-10T11:19:08.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:08.292 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:08.292 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:08.627 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:08.630 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:08.630 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839263/65ad6626-d1d5-4ef3-b330-93505056dd71/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:09.003 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:09.003 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:09.003 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245033 s, 42.8 MB/s 2021-02-10T11:19:09.004 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839264 2021-02-10T11:19:09.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.414 7fb1bbbc9700 1 Processor -- start 2021-02-10T11:19:09.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.414 7fb1bbbc9700 1 -- start start 2021-02-10T11:19:09.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.415 7fb1bbbc9700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1b40f05f0 0x7fb1b40f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:09.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.415 7fb1bbbc9700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b41b0c70 con 0x7fb1b41b0870 2021-02-10T11:19:09.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.415 7fb1bbbc9700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b40f0b50 con 0x7fb1b41b0470 2021-02-10T11:19:09.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.415 7fb1bbbc9700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b40e5ea0 con 0x7fb1b40f05f0 2021-02-10T11:19:09.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.415 7fb1ba1e6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1b40f05f0 0x7fb1b40f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:09.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.415 7fb1b91e4700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb1b41b0870 0x7fb1b40efe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46886/0 (socket says 172.21.2.101:46886) 2021-02-10T11:19:09.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.415 7fb1b91e4700 1 -- 172.21.2.101:0/3756498046 learned_addr learned my addr 172.21.2.101:0/3756498046 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:19:09.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.415 7fb1ba1e6700 1 -- 172.21.2.101:0/3756498046 >> v1:172.21.2.108:6789/0 conn(0x7fb1b41b0470 legacy=0x7fb1b40ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:19:09.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.415 7fb1ba1e6700 1 -- 172.21.2.101:0/3756498046 >> v1:172.21.2.101:6789/0 conn(0x7fb1b41b0870 legacy=0x7fb1b40efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:09.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.415 7fb1b89e3700 1 -- 172.21.2.101:0/3756498046 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481616475 0 0) 0x7fb1b00010b0 con 0x7fb1b41b0870 2021-02-10T11:19:09.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.415 7fb1ba1e6700 1 -- 172.21.2.101:0/3756498046 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b40e52e0 con 0x7fb1b40f05f0 2021-02-10T11:19:09.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.416 7fb1ba1e6700 1 --2- 172.21.2.101:0/3756498046 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1b40f05f0 0x7fb1b40f20e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fb1ac006e30 tx=0x7fb1ac009de0).ready entity=mon.0 client_cookie=634744efaae229cf server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:09.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.416 7fb1b89e3700 1 -- 172.21.2.101:0/3756498046 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1ac00b300 con 0x7fb1b40f05f0 2021-02-10T11:19:09.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.416 7fb1b89e3700 1 -- 172.21.2.101:0/3756498046 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ac00b580 con 0x7fb1b40f05f0 2021-02-10T11:19:09.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.416 7fb1b89e3700 1 -- 172.21.2.101:0/3756498046 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1ac00baa0 con 0x7fb1b40f05f0 2021-02-10T11:19:09.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.416 7fb1bbbc9700 1 -- 172.21.2.101:0/3756498046 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1b40f05f0 msgr2=0x7fb1b40f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:09.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.416 7fb1bbbc9700 1 --2- 172.21.2.101:0/3756498046 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1b40f05f0 0x7fb1b40f20e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fb1ac006e30 tx=0x7fb1ac009de0).stop 2021-02-10T11:19:09.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.416 7fb1bbbc9700 1 -- 172.21.2.101:0/3756498046 shutdown_connections 2021-02-10T11:19:09.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.416 7fb1bbbc9700 1 --2- 172.21.2.101:0/3756498046 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1b40f05f0 0x7fb1b40f20e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:09.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.416 7fb1bbbc9700 1 -- 172.21.2.101:0/3756498046 >> 172.21.2.101:0/3756498046 conn(0x7fb1b40ce3e0 msgr2=0x7fb1b40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:09.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.417 7fb1bbbc9700 1 -- 172.21.2.101:0/3756498046 shutdown_connections 2021-02-10T11:19:09.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.417 7fb1bbbc9700 1 -- 172.21.2.101:0/3756498046 wait complete. 2021-02-10T11:19:09.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.431 7fb1bbbc9700 1 Processor -- start 2021-02-10T11:19:09.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.431 7fb1bbbc9700 1 -- start start 2021-02-10T11:19:09.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.432 7fb1bbbc9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1b40a1dd0 0x7fb1b4190920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:09.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.432 7fb1bbbc9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1b408fd30 0x7fb1b40effc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:09.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.432 7fb1bbbc9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1b40f0e90 0x7fb1b40f1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:09.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.432 7fb1bbbc9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b40f06f0 con 0x7fb1b40a1dd0 2021-02-10T11:19:09.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.432 7fb1bbbc9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b4090160 con 0x7fb1b40f0e90 2021-02-10T11:19:09.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.432 7fb1bbbc9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b40f18b0 con 0x7fb1b408fd30 2021-02-10T11:19:09.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.432 7fb1b99e5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1b40a1dd0 0x7fb1b4190920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:09.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.432 7fb1b99e5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1b40a1dd0 0x7fb1b4190920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46400/0 (socket says 172.21.2.101:46400) 2021-02-10T11:19:09.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.432 7fb1b99e5700 1 -- 172.21.2.101:0/179541432 learned_addr learned my addr 172.21.2.101:0/179541432 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:19:09.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.432 7fb1b99e5700 1 -- 172.21.2.101:0/179541432 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1b408fd30 msgr2=0x7fb1b40effc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:09.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.432 7fb1b99e5700 1 --2- 172.21.2.101:0/179541432 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1b408fd30 0x7fb1b40effc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:09.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.432 7fb1b99e5700 1 -- 172.21.2.101:0/179541432 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1b40f0e90 msgr2=0x7fb1b40f1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:09.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.432 7fb1b99e5700 1 --2- 172.21.2.101:0/179541432 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1b40f0e90 0x7fb1b40f1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:09.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.432 7fb1b99e5700 1 -- 172.21.2.101:0/179541432 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b40f1ad0 con 0x7fb1b40a1dd0 2021-02-10T11:19:09.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.433 7fb1b99e5700 1 --2- 172.21.2.101:0/179541432 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1b40a1dd0 0x7fb1b4190920 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fb1a800b5b0 tx=0x7fb1a800b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:09.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.433 7fb1a67fc700 1 -- 172.21.2.101:0/179541432 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1a800bd40 con 0x7fb1b40a1dd0 2021-02-10T11:19:09.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.433 7fb1a67fc700 1 -- 172.21.2.101:0/179541432 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a8000e20 con 0x7fb1b40a1dd0 2021-02-10T11:19:09.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.433 7fb1bbbc9700 1 -- 172.21.2.101:0/179541432 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b40f1ad0 con 0x7fb1b40a1dd0 2021-02-10T11:19:09.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.433 7fb1a67fc700 1 -- 172.21.2.101:0/179541432 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1a80054a0 con 0x7fb1b40a1dd0 2021-02-10T11:19:09.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.433 7fb1bbbc9700 1 -- 172.21.2.101:0/179541432 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b40f1ad0 con 0x7fb1b40a1dd0 2021-02-10T11:19:09.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.434 7fb1a67fc700 1 -- 172.21.2.101:0/179541432 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb1a8006030 con 0x7fb1b40a1dd0 2021-02-10T11:19:09.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.434 7fb1a67fc700 1 --2- 172.21.2.101:0/179541432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb19c018730 0x7fb19c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:09.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.434 7fb1a67fc700 1 -- 172.21.2.101:0/179541432 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb1a80327a0 con 0x7fb1b40a1dd0 2021-02-10T11:19:09.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.434 7fb1b91e4700 1 --2- 172.21.2.101:0/179541432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb19c018730 0x7fb19c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:09.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.434 7fb1b91e4700 1 --2- 172.21.2.101:0/179541432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb19c018730 0x7fb19c01abc0 crc :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:09.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.464 7fb1bbbc9700 1 -- 172.21.2.101:0/179541432 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b4090130 con 0x7fb1b40a1dd0 2021-02-10T11:19:09.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.467 7fb1a67fc700 1 -- 172.21.2.101:0/179541432 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb1a8000ce0 con 0x7fb1b40a1dd0 2021-02-10T11:19:09.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.593 7fb1bbbc9700 1 -- 172.21.2.101:0/179541432 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839264", "target": ["mgr", ""]}) v1 -- 0x7fb1b4090130 con 0x7fb19c018730 2021-02-10T11:19:09.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.620 7fb1a67fc700 1 -- 172.21.2.101:0/179541432 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fb1b0000940 con 0x7fb19c018730 2021-02-10T11:19:09.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.626 7fb1bbbc9700 1 -- 172.21.2.101:0/179541432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb19c018730 msgr2=0x7fb19c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:09.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.626 7fb1bbbc9700 1 --2- 172.21.2.101:0/179541432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb19c018730 0x7fb19c01abc0 crc :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:09.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.626 7fb1bbbc9700 1 -- 172.21.2.101:0/179541432 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1b40a1dd0 msgr2=0x7fb1b4190920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:09.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.626 7fb1bbbc9700 1 --2- 172.21.2.101:0/179541432 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1b40a1dd0 0x7fb1b4190920 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fb1a800b5b0 tx=0x7fb1a800b950).stop 2021-02-10T11:19:09.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.626 7fb1bbbc9700 1 -- 172.21.2.101:0/179541432 shutdown_connections 2021-02-10T11:19:09.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.626 7fb1bbbc9700 1 --2- 172.21.2.101:0/179541432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb19c018730 0x7fb19c01abc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:09.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.626 7fb1bbbc9700 1 --2- 172.21.2.101:0/179541432 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1b408fd30 0x7fb1b40effc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:09.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.626 7fb1bbbc9700 1 --2- 172.21.2.101:0/179541432 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1b40f0e90 0x7fb1b40f1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:09.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.626 7fb1bbbc9700 1 --2- 172.21.2.101:0/179541432 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1b40a1dd0 0x7fb1b4190920 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:09.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.626 7fb1bbbc9700 1 -- 172.21.2.101:0/179541432 >> 172.21.2.101:0/179541432 conn(0x7fb1b40958b0 msgr2=0x7fb1b40bf040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:09.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.627 7fb1bbbc9700 1 -- 172.21.2.101:0/179541432 shutdown_connections 2021-02-10T11:19:09.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:09.627 7fb1bbbc9700 1 -- 172.21.2.101:0/179541432 wait complete. 2021-02-10T11:19:09.641 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839264 2021-02-10T11:19:10.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.052 7fd481cc4700 1 Processor -- start 2021-02-10T11:19:10.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.052 7fd481cc4700 1 -- start start 2021-02-10T11:19:10.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.052 7fd481cc4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd47c1b0870 0x7fd47c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:10.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.052 7fd481cc4700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd47c1b0c70 con 0x7fd47c0efe00 2021-02-10T11:19:10.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.052 7fd481cc4700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd47c0f0b50 con 0x7fd47c1b0470 2021-02-10T11:19:10.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.052 7fd481cc4700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd47c0e5ea0 con 0x7fd47c1b0870 2021-02-10T11:19:10.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.053 7fd47bfff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd47c0efe00 0x7fd47c0f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46904/0 (socket says 172.21.2.101:46904) 2021-02-10T11:19:10.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.053 7fd47bfff700 1 -- 172.21.2.101:0/4005899366 learned_addr learned my addr 172.21.2.101:0/4005899366 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:19:10.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.053 7fd47affd700 1 --2- 172.21.2.101:0/4005899366 >> v2:172.21.2.101:3300/0 conn(0x7fd47c1b0870 0x7fd47c0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:10.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.053 7fd47a7fc700 1 -- 172.21.2.101:0/4005899366 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 195507959 0 0) 0x7fd474001500 con 0x7fd47c0efe00 2021-02-10T11:19:10.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.053 7fd47affd700 1 -- 172.21.2.101:0/4005899366 >> v1:172.21.2.108:6789/0 conn(0x7fd47c1b0470 legacy=0x7fd47c0ef610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:19:10.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.053 7fd47affd700 1 -- 172.21.2.101:0/4005899366 >> v1:172.21.2.101:6789/0 conn(0x7fd47c0efe00 legacy=0x7fd47c0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:10.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.053 7fd47affd700 1 -- 172.21.2.101:0/4005899366 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c0e52e0 con 0x7fd47c1b0870 2021-02-10T11:19:10.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.053 7fd47affd700 1 --2- 172.21.2.101:0/4005899366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd47c1b0870 0x7fd47c0f20e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7fd464006650 tx=0x7fd464009a80).ready entity=mon.0 client_cookie=4eb31243621d2956 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:10.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.053 7fd47a7fc700 1 -- 172.21.2.101:0/4005899366 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd46400c0c0 con 0x7fd47c1b0870 2021-02-10T11:19:10.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.053 7fd47a7fc700 1 -- 172.21.2.101:0/4005899366 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46400c340 con 0x7fd47c1b0870 2021-02-10T11:19:10.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.053 7fd47a7fc700 1 -- 172.21.2.101:0/4005899366 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd46400c860 con 0x7fd47c1b0870 2021-02-10T11:19:10.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.053 7fd481cc4700 1 -- 172.21.2.101:0/4005899366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd47c1b0870 msgr2=0x7fd47c0f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:10.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.053 7fd481cc4700 1 --2- 172.21.2.101:0/4005899366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd47c1b0870 0x7fd47c0f20e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7fd464006650 tx=0x7fd464009a80).stop 2021-02-10T11:19:10.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.054 7fd481cc4700 1 -- 172.21.2.101:0/4005899366 shutdown_connections 2021-02-10T11:19:10.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.054 7fd481cc4700 1 --2- 172.21.2.101:0/4005899366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd47c1b0870 0x7fd47c0f20e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:10.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.054 7fd481cc4700 1 -- 172.21.2.101:0/4005899366 >> 172.21.2.101:0/4005899366 conn(0x7fd47c0ce3e0 msgr2=0x7fd47c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:10.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.054 7fd481cc4700 1 -- 172.21.2.101:0/4005899366 shutdown_connections 2021-02-10T11:19:10.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.054 7fd481cc4700 1 -- 172.21.2.101:0/4005899366 wait complete. 2021-02-10T11:19:10.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.069 7fd481cc4700 1 Processor -- start 2021-02-10T11:19:10.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.069 7fd481cc4700 1 -- start start 2021-02-10T11:19:10.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.069 7fd481cc4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd47c1a0570 0x7fd47c0a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:10.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.069 7fd481cc4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd47c0f1fd0 0x7fd47c0f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:10.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.069 7fd481cc4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd47c0a5450 0x7fd47c0a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:10.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.069 7fd481cc4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c0a4bb0 con 0x7fd47c0a5450 2021-02-10T11:19:10.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.069 7fd481cc4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c0b35e0 con 0x7fd47c0f1fd0 2021-02-10T11:19:10.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.069 7fd481cc4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c0a1f00 con 0x7fd47c1a0570 2021-02-10T11:19:10.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.070 7fd47bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd47c0a5450 0x7fd47c0a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:10.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.070 7fd47bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd47c0a5450 0x7fd47c0a5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46414/0 (socket says 172.21.2.101:46414) 2021-02-10T11:19:10.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.070 7fd47bfff700 1 -- 172.21.2.101:0/1799746087 learned_addr learned my addr 172.21.2.101:0/1799746087 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:19:10.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.070 7fd47b7fe700 1 --2- 172.21.2.101:0/1799746087 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd47c1a0570 0x7fd47c0a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:10.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.070 7fd47bfff700 1 -- 172.21.2.101:0/1799746087 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd47c1a0570 msgr2=0x7fd47c0a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:10.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.070 7fd47affd700 1 --2- 172.21.2.101:0/1799746087 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd47c0f1fd0 0x7fd47c0f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:10.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.070 7fd47bfff700 1 --2- 172.21.2.101:0/1799746087 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd47c1a0570 0x7fd47c0a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:10.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.070 7fd47bfff700 1 -- 172.21.2.101:0/1799746087 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd47c0f1fd0 msgr2=0x7fd47c0f23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:10.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.070 7fd47bfff700 1 --2- 172.21.2.101:0/1799746087 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd47c0f1fd0 0x7fd47c0f23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:10.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.070 7fd47bfff700 1 -- 172.21.2.101:0/1799746087 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c0a5e70 con 0x7fd47c0a5450 2021-02-10T11:19:10.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.070 7fd47affd700 1 --2- 172.21.2.101:0/1799746087 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd47c0f1fd0 0x7fd47c0f23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:19:10.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.070 7fd47bfff700 1 --2- 172.21.2.101:0/1799746087 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd47c0a5450 0x7fd47c0a5850 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7fd474001c10 tx=0x7fd47400ac60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:10.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.071 7fd473fff700 1 -- 172.21.2.101:0/1799746087 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd474000f90 con 0x7fd47c0a5450 2021-02-10T11:19:10.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.071 7fd473fff700 1 -- 172.21.2.101:0/1799746087 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4740070d0 con 0x7fd47c0a5450 2021-02-10T11:19:10.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.071 7fd473fff700 1 -- 172.21.2.101:0/1799746087 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd474004720 con 0x7fd47c0a5450 2021-02-10T11:19:10.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.071 7fd481cc4700 1 -- 172.21.2.101:0/1799746087 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd47c0a5e70 con 0x7fd47c0a5450 2021-02-10T11:19:10.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.071 7fd481cc4700 1 -- 172.21.2.101:0/1799746087 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd47c0a6400 con 0x7fd47c0a5450 2021-02-10T11:19:10.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.071 7fd473fff700 1 -- 172.21.2.101:0/1799746087 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd4740072b0 con 0x7fd47c0a5450 2021-02-10T11:19:10.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.071 7fd473fff700 1 --2- 172.21.2.101:0/1799746087 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4600186c0 0x7fd46001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:10.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.071 7fd473fff700 1 -- 172.21.2.101:0/1799746087 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd4740328c0 con 0x7fd47c0a5450 2021-02-10T11:19:10.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.072 7fd47b7fe700 1 --2- 172.21.2.101:0/1799746087 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4600186c0 0x7fd46001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:10.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.072 7fd47b7fe700 1 --2- 172.21.2.101:0/1799746087 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4600186c0 0x7fd46001abc0 crc :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:10.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.102 7fd481cc4700 1 -- 172.21.2.101:0/1799746087 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd47c0e52e0 con 0x7fd47c0a5450 2021-02-10T11:19:10.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.105 7fd473fff700 1 -- 172.21.2.101:0/1799746087 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd474000f90 con 0x7fd47c0a5450 2021-02-10T11:19:10.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.234 7fd481cc4700 1 -- 172.21.2.101:0/1799746087 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839264", "target": ["mgr", ""]}) v1 -- 0x7fd47c0a6400 con 0x7fd4600186c0 2021-02-10T11:19:10.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.237 7fd473fff700 1 -- 172.21.2.101:0/1799746087 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fd46c006b40 con 0x7fd4600186c0 2021-02-10T11:19:10.247 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839264/2bc5280c-997b-425d-865c-5caf648c595d 2021-02-10T11:19:10.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.242 7fd481cc4700 1 -- 172.21.2.101:0/1799746087 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4600186c0 msgr2=0x7fd46001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:10.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.242 7fd481cc4700 1 --2- 172.21.2.101:0/1799746087 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4600186c0 0x7fd46001abc0 crc :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:10.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.242 7fd481cc4700 1 -- 172.21.2.101:0/1799746087 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd47c0a5450 msgr2=0x7fd47c0a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:10.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.242 7fd481cc4700 1 --2- 172.21.2.101:0/1799746087 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd47c0a5450 0x7fd47c0a5850 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7fd474001c10 tx=0x7fd47400ac60).stop 2021-02-10T11:19:10.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.242 7fd481cc4700 1 -- 172.21.2.101:0/1799746087 shutdown_connections 2021-02-10T11:19:10.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.242 7fd481cc4700 1 --2- 172.21.2.101:0/1799746087 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd47c0a5450 0x7fd47c0a5850 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:10.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.242 7fd481cc4700 1 --2- 172.21.2.101:0/1799746087 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd47c0f1fd0 0x7fd47c0f23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:10.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.242 7fd481cc4700 1 --2- 172.21.2.101:0/1799746087 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4600186c0 0x7fd46001abc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:10.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.242 7fd481cc4700 1 --2- 172.21.2.101:0/1799746087 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd47c1a0570 0x7fd47c0a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:10.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.242 7fd481cc4700 1 -- 172.21.2.101:0/1799746087 >> 172.21.2.101:0/1799746087 conn(0x7fd47c0958b0 msgr2=0x7fd47c0bf040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:10.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.243 7fd481cc4700 1 -- 172.21.2.101:0/1799746087 shutdown_connections 2021-02-10T11:19:10.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:10.243 7fd481cc4700 1 -- 172.21.2.101:0/1799746087 wait complete. 2021-02-10T11:19:10.257 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839264 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839264/2bc5280c-997b-425d-865c-5caf648c595d 2021-02-10T11:19:10.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:10.258 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:10.258 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:10.594 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:10.594 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:10.594 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839264/2bc5280c-997b-425d-865c-5caf648c595d/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:10.974 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:10.974 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:10.974 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0225691 s, 46.5 MB/s 2021-02-10T11:19:10.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:10.975 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:10.975 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:11.309 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:11.309 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:11.309 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839264/2bc5280c-997b-425d-865c-5caf648c595d/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:11.683 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:11.683 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:11.683 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0223552 s, 46.9 MB/s 2021-02-10T11:19:11.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:11.684 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:11.684 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:12.019 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:12.019 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:12.020 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839264/2bc5280c-997b-425d-865c-5caf648c595d/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:12.392 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:12.392 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:12.392 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0226942 s, 46.2 MB/s 2021-02-10T11:19:12.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:12.393 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:12.394 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:12.729 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:12.729 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:12.729 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839264/2bc5280c-997b-425d-865c-5caf648c595d/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:13.103 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:13.104 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:13.104 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233059 s, 45.0 MB/s 2021-02-10T11:19:13.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:13.105 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:13.105 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:13.440 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:13.440 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:13.440 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839264/2bc5280c-997b-425d-865c-5caf648c595d/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:13.811 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:13.812 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:13.812 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234143 s, 44.8 MB/s 2021-02-10T11:19:13.813 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:13.813 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:13.813 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:14.147 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:14.148 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:14.148 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839264/2bc5280c-997b-425d-865c-5caf648c595d/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:14.523 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:14.523 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:14.523 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247951 s, 42.3 MB/s 2021-02-10T11:19:14.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:14.524 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:14.524 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:14.860 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:14.860 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:14.860 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839264/2bc5280c-997b-425d-865c-5caf648c595d/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:15.236 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:15.237 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:15.237 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024773 s, 42.3 MB/s 2021-02-10T11:19:15.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:15.238 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:15.238 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:15.573 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:15.573 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:15.573 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839264/2bc5280c-997b-425d-865c-5caf648c595d/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:15.949 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:15.949 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:15.950 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240142 s, 43.7 MB/s 2021-02-10T11:19:15.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:15.951 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:15.951 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:16.284 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:16.285 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:16.285 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839264/2bc5280c-997b-425d-865c-5caf648c595d/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:16.661 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:16.661 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:16.661 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246892 s, 42.5 MB/s 2021-02-10T11:19:16.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:16.662 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:16.662 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:16.998 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:16.998 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:16.998 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839264/2bc5280c-997b-425d-865c-5caf648c595d/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:17.374 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:17.374 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:17.374 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251193 s, 41.7 MB/s 2021-02-10T11:19:17.375 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839265 2021-02-10T11:19:17.484 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:19:17.488 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:19:17.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.782 7fb1d534a700 1 Processor -- start 2021-02-10T11:19:17.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.782 7fb1d534a700 1 -- start start 2021-02-10T11:19:17.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.782 7fb1d534a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1d01b0470 0x7fb1d01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:17.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.782 7fb1d534a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1d00f20e0 0x7fb1d00f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:17.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.782 7fb1d534a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d00f0b50 con 0x7fb1d00ef610 2021-02-10T11:19:17.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.782 7fb1d534a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d00f0820 con 0x7fb1d00f20e0 2021-02-10T11:19:17.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.782 7fb1d534a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d00f24e0 con 0x7fb1d01b0470 2021-02-10T11:19:17.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.783 7fb1cf87e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1d00f20e0 0x7fb1d00f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:17.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.783 7fb1cf87e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1d00f20e0 0x7fb1d00f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46420/0 (socket says 172.21.2.101:46420) 2021-02-10T11:19:17.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.783 7fb1cf87e700 1 -- 172.21.2.101:0/1366586818 learned_addr learned my addr 172.21.2.101:0/1366586818 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:19:17.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.783 7fb1cf87e700 1 -- 172.21.2.101:0/1366586818 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1d01b0470 msgr2=0x7fb1d01b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:19:17.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.783 7fb1cf87e700 1 --2- 172.21.2.101:0/1366586818 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1d01b0470 0x7fb1d01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:17.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.783 7fb1cf87e700 1 -- 172.21.2.101:0/1366586818 >> v1:172.21.2.101:6789/0 conn(0x7fb1d00ef610 legacy=0x7fb1d00efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:17.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.783 7fb1cf87e700 1 -- 172.21.2.101:0/1366586818 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d00e52e0 con 0x7fb1d00f20e0 2021-02-10T11:19:17.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.783 7fb1cf87e700 1 --2- 172.21.2.101:0/1366586818 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1d00f20e0 0x7fb1d00f0200 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7fb1c8006f00 tx=0x7fb1c8009f10).ready entity=mon.0 client_cookie=3708537fab235ec0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:17.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.784 7fb1ce07b700 1 -- 172.21.2.101:0/1366586818 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1c800b3b0 con 0x7fb1d00f20e0 2021-02-10T11:19:17.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.784 7fb1ce07b700 1 -- 172.21.2.101:0/1366586818 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c800b730 con 0x7fb1d00f20e0 2021-02-10T11:19:17.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.784 7fb1ce07b700 1 -- 172.21.2.101:0/1366586818 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1c800bc50 con 0x7fb1d00f20e0 2021-02-10T11:19:17.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.784 7fb1d534a700 1 -- 172.21.2.101:0/1366586818 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1d00f20e0 msgr2=0x7fb1d00f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:17.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.784 7fb1d534a700 1 --2- 172.21.2.101:0/1366586818 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1d00f20e0 0x7fb1d00f0200 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7fb1c8006f00 tx=0x7fb1c8009f10).stop 2021-02-10T11:19:17.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.784 7fb1d534a700 1 -- 172.21.2.101:0/1366586818 shutdown_connections 2021-02-10T11:19:17.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.784 7fb1d534a700 1 --2- 172.21.2.101:0/1366586818 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1d00f20e0 0x7fb1d00f0200 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:17.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.784 7fb1d534a700 1 --2- 172.21.2.101:0/1366586818 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1d01b0470 0x7fb1d01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:17.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.784 7fb1d534a700 1 -- 172.21.2.101:0/1366586818 >> 172.21.2.101:0/1366586818 conn(0x7fb1d00ce3e0 msgr2=0x7fb1d00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:17.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.784 7fb1d534a700 1 -- 172.21.2.101:0/1366586818 shutdown_connections 2021-02-10T11:19:17.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.784 7fb1d534a700 1 -- 172.21.2.101:0/1366586818 wait complete. 2021-02-10T11:19:17.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.799 7fb1d534a700 1 Processor -- start 2021-02-10T11:19:17.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.799 7fb1d534a700 1 -- start start 2021-02-10T11:19:17.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.799 7fb1d534a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1d0094fb0 0x7fb1d009e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:17.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.799 7fb1d534a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1d00c2be0 0x7fb1d00c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:17.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.799 7fb1d534a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1d00c3600 0x7fb1d00a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:17.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.799 7fb1d534a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d009ebf0 con 0x7fb1d00c3600 2021-02-10T11:19:17.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.799 7fb1d534a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d00aa2a0 con 0x7fb1d00c2be0 2021-02-10T11:19:17.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.799 7fb1d534a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d00aa4c0 con 0x7fb1d0094fb0 2021-02-10T11:19:17.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.800 7fb1cf87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1d00c3600 0x7fb1d00a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:17.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.800 7fb1cf87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1d00c3600 0x7fb1d00a9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46428/0 (socket says 172.21.2.101:46428) 2021-02-10T11:19:17.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.800 7fb1cf87e700 1 -- 172.21.2.101:0/3410838414 learned_addr learned my addr 172.21.2.101:0/3410838414 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:19:17.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.800 7fb1cf87e700 1 -- 172.21.2.101:0/3410838414 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1d0094fb0 msgr2=0x7fb1d009e5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:19:17.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.800 7fb1cf87e700 1 --2- 172.21.2.101:0/3410838414 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1d0094fb0 0x7fb1d009e5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:17.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.800 7fb1cf87e700 1 -- 172.21.2.101:0/3410838414 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1d00c2be0 msgr2=0x7fb1d00c2fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:19:17.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.800 7fb1cf87e700 1 --2- 172.21.2.101:0/3410838414 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1d00c2be0 0x7fb1d00c2fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:17.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.800 7fb1cf87e700 1 -- 172.21.2.101:0/3410838414 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d00aa6e0 con 0x7fb1d00c3600 2021-02-10T11:19:17.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.800 7fb1cf87e700 1 --2- 172.21.2.101:0/3410838414 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1d00c3600 0x7fb1d00a9c80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fb1c8009390 tx=0x7fb1c800bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:17.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.800 7fb1b7fff700 1 -- 172.21.2.101:0/3410838414 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1c8003030 con 0x7fb1d00c3600 2021-02-10T11:19:17.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.800 7fb1b7fff700 1 -- 172.21.2.101:0/3410838414 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c80068e0 con 0x7fb1d00c3600 2021-02-10T11:19:17.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.800 7fb1d534a700 1 -- 172.21.2.101:0/3410838414 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d00aa6e0 con 0x7fb1d00c3600 2021-02-10T11:19:17.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.800 7fb1d534a700 1 -- 172.21.2.101:0/3410838414 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d00aac70 con 0x7fb1d00c3600 2021-02-10T11:19:17.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.800 7fb1b7fff700 1 -- 172.21.2.101:0/3410838414 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1c800c570 con 0x7fb1d00c3600 2021-02-10T11:19:17.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.801 7fb1b7fff700 1 -- 172.21.2.101:0/3410838414 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb1c8001270 con 0x7fb1d00c3600 2021-02-10T11:19:17.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.801 7fb1b7fff700 1 --2- 172.21.2.101:0/3410838414 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1b00186c0 0x7fb1b001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:17.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.801 7fb1b7fff700 1 -- 172.21.2.101:0/3410838414 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb1c80328d0 con 0x7fb1d00c3600 2021-02-10T11:19:17.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.801 7fb1cf07d700 1 --2- 172.21.2.101:0/3410838414 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1b00186c0 0x7fb1b001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:17.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.801 7fb1cf07d700 1 --2- 172.21.2.101:0/3410838414 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1b00186c0 0x7fb1b001abc0 crc :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:17.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.832 7fb1d534a700 1 -- 172.21.2.101:0/3410838414 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d00a9750 con 0x7fb1d00c3600 2021-02-10T11:19:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.835 7fb1b7fff700 1 -- 172.21.2.101:0/3410838414 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb1c8009430 con 0x7fb1d00c3600 2021-02-10T11:19:17.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.964 7fb1d534a700 1 -- 172.21.2.101:0/3410838414 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839265", "target": ["mgr", ""]}) v1 -- 0x7fb1d00aac70 con 0x7fb1b00186c0 2021-02-10T11:19:17.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.987 7fb1b7fff700 1 -- 172.21.2.101:0/3410838414 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fb1c0006960 con 0x7fb1b00186c0 2021-02-10T11:19:18.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.997 7fb1d534a700 1 -- 172.21.2.101:0/3410838414 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1b00186c0 msgr2=0x7fb1b001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:18.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.997 7fb1d534a700 1 --2- 172.21.2.101:0/3410838414 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1b00186c0 0x7fb1b001abc0 crc :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:18.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.997 7fb1d534a700 1 -- 172.21.2.101:0/3410838414 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1d00c3600 msgr2=0x7fb1d00a9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:18.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.997 7fb1d534a700 1 --2- 172.21.2.101:0/3410838414 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1d00c3600 0x7fb1d00a9c80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fb1c8009390 tx=0x7fb1c800bd40).stop 2021-02-10T11:19:18.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.997 7fb1d534a700 1 -- 172.21.2.101:0/3410838414 shutdown_connections 2021-02-10T11:19:18.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.997 7fb1d534a700 1 --2- 172.21.2.101:0/3410838414 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1d00c3600 0x7fb1d00a9c80 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:18.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.997 7fb1d534a700 1 --2- 172.21.2.101:0/3410838414 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1d00c2be0 0x7fb1d00c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:18.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.997 7fb1d534a700 1 --2- 172.21.2.101:0/3410838414 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1b00186c0 0x7fb1b001abc0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:18.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.997 7fb1d534a700 1 --2- 172.21.2.101:0/3410838414 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1d0094fb0 0x7fb1d009e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:18.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.997 7fb1d534a700 1 -- 172.21.2.101:0/3410838414 >> 172.21.2.101:0/3410838414 conn(0x7fb1d009f630 msgr2=0x7fb1d00a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:18.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.997 7fb1d534a700 1 -- 172.21.2.101:0/3410838414 shutdown_connections 2021-02-10T11:19:18.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:17.997 7fb1d534a700 1 -- 172.21.2.101:0/3410838414 wait complete. 2021-02-10T11:19:18.012 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839265 2021-02-10T11:19:18.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.419 7f994cd6b700 1 Processor -- start 2021-02-10T11:19:18.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.419 7f994cd6b700 1 -- start start 2021-02-10T11:19:18.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.419 7f994cd6b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f99481b0470 0x7f99481b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:18.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.419 7f994cd6b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99480ef610 0x7f99480efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:18.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.419 7f994cd6b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99480f0b50 con 0x7f99480f0030 2021-02-10T11:19:18.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.419 7f994cd6b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99480e52e0 con 0x7f99481b0470 2021-02-10T11:19:18.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.419 7f994cd6b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99480e54d0 con 0x7f99480ef610 2021-02-10T11:19:18.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.419 7f9946b13700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f99481b0470 0x7f99481b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:18.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.420 7f9947314700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f99480f0030 0x7f99480f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46932/0 (socket says 172.21.2.101:46932) 2021-02-10T11:19:18.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.420 7f9947314700 1 -- 172.21.2.101:0/2915258208 learned_addr learned my addr 172.21.2.101:0/2915258208 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:19:18.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.420 7f9946312700 1 --2- 172.21.2.101:0/2915258208 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99480ef610 0x7f99480efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:18.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.420 7f9946b13700 1 -- 172.21.2.101:0/2915258208 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99480ef610 msgr2=0x7f99480efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:19:18.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.420 7f9946b13700 1 --2- 172.21.2.101:0/2915258208 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99480ef610 0x7f99480efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:18.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.420 7f9946b13700 1 -- 172.21.2.101:0/2915258208 >> v1:172.21.2.101:6789/0 conn(0x7f99480f0030 legacy=0x7f99480f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:18.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.420 7f9946b13700 1 -- 172.21.2.101:0/2915258208 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99480f0430 con 0x7f99481b0470 2021-02-10T11:19:18.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.420 7f9946b13700 1 --2- 172.21.2.101:0/2915258208 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99481b0470 0x7f99481b0870 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f9938006650 tx=0x7f9938009a80).ready entity=mon.0 client_cookie=4ef56428653104cc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:18.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.420 7f9945b11700 1 -- 172.21.2.101:0/2915258208 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f993800c0c0 con 0x7f99481b0470 2021-02-10T11:19:18.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.420 7f9945b11700 1 -- 172.21.2.101:0/2915258208 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993800c340 con 0x7f99481b0470 2021-02-10T11:19:18.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.420 7f9945b11700 1 -- 172.21.2.101:0/2915258208 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f993800c860 con 0x7f99481b0470 2021-02-10T11:19:18.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.420 7f994cd6b700 1 -- 172.21.2.101:0/2915258208 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99481b0470 msgr2=0x7f99481b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:18.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.420 7f994cd6b700 1 --2- 172.21.2.101:0/2915258208 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99481b0470 0x7f99481b0870 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f9938006650 tx=0x7f9938009a80).stop 2021-02-10T11:19:18.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.421 7f994cd6b700 1 -- 172.21.2.101:0/2915258208 shutdown_connections 2021-02-10T11:19:18.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.421 7f994cd6b700 1 --2- 172.21.2.101:0/2915258208 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99480ef610 0x7f99480efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:18.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.421 7f994cd6b700 1 --2- 172.21.2.101:0/2915258208 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99481b0470 0x7f99481b0870 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:18.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.421 7f994cd6b700 1 -- 172.21.2.101:0/2915258208 >> 172.21.2.101:0/2915258208 conn(0x7f99480ce3e0 msgr2=0x7f99480e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:18.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.421 7f994cd6b700 1 -- 172.21.2.101:0/2915258208 shutdown_connections 2021-02-10T11:19:18.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.421 7f994cd6b700 1 -- 172.21.2.101:0/2915258208 wait complete. 2021-02-10T11:19:18.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.436 7f994cd6b700 1 Processor -- start 2021-02-10T11:19:18.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.436 7f994cd6b700 1 -- start start 2021-02-10T11:19:18.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.436 7f994cd6b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99480a0420 0x7f99480a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:18.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.436 7f994cd6b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9948189d00 0x7f994818a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:18.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.436 7f994cd6b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f994818a720 0x7f99480b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:18.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.436 7f994cd6b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9948189800 con 0x7f9948189d00 2021-02-10T11:19:18.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.436 7f994cd6b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994818ab20 con 0x7f99480a0420 2021-02-10T11:19:18.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.436 7f994cd6b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99480a1140 con 0x7f994818a720 2021-02-10T11:19:18.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.437 7f9946312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9948189d00 0x7f994818a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:18.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.437 7f9946312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9948189d00 0x7f994818a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46442/0 (socket says 172.21.2.101:46442) 2021-02-10T11:19:18.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.437 7f9946312700 1 -- 172.21.2.101:0/961793586 learned_addr learned my addr 172.21.2.101:0/961793586 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:19:18.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.437 7f9946312700 1 -- 172.21.2.101:0/961793586 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f994818a720 msgr2=0x7f99480b7be0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:19:18.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.437 7f9946312700 1 --2- 172.21.2.101:0/961793586 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f994818a720 0x7f99480b7be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:18.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.437 7f9946312700 1 -- 172.21.2.101:0/961793586 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99480a0420 msgr2=0x7f99480a0820 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:19:18.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.437 7f9946312700 1 --2- 172.21.2.101:0/961793586 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99480a0420 0x7f99480a0820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:18.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.437 7f9946312700 1 -- 172.21.2.101:0/961793586 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99480b8200 con 0x7f9948189d00 2021-02-10T11:19:18.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.437 7f9946312700 1 --2- 172.21.2.101:0/961793586 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9948189d00 0x7f994818a100 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f9930006f00 tx=0x7f993000b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:18.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.437 7f993f7fe700 1 -- 172.21.2.101:0/961793586 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f993000bd30 con 0x7f9948189d00 2021-02-10T11:19:18.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.437 7f993f7fe700 1 -- 172.21.2.101:0/961793586 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9930002030 con 0x7f9948189d00 2021-02-10T11:19:18.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.437 7f993f7fe700 1 -- 172.21.2.101:0/961793586 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9930002590 con 0x7f9948189d00 2021-02-10T11:19:18.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.437 7f994cd6b700 1 -- 172.21.2.101:0/961793586 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99480b8200 con 0x7f9948189d00 2021-02-10T11:19:18.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.437 7f994cd6b700 1 -- 172.21.2.101:0/961793586 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99480b8200 con 0x7f9948189d00 2021-02-10T11:19:18.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.438 7f993f7fe700 1 -- 172.21.2.101:0/961793586 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9930002210 con 0x7f9948189d00 2021-02-10T11:19:18.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.438 7f993f7fe700 1 --2- 172.21.2.101:0/961793586 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f992c0186c0 0x7f992c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:18.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.438 7f993f7fe700 1 -- 172.21.2.101:0/961793586 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9930032670 con 0x7f9948189d00 2021-02-10T11:19:18.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.438 7f9946b13700 1 --2- 172.21.2.101:0/961793586 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f992c0186c0 0x7f992c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:18.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.438 7f9946b13700 1 --2- 172.21.2.101:0/961793586 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f992c0186c0 0x7f992c01abc0 crc :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:18.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.469 7f994cd6b700 1 -- 172.21.2.101:0/961793586 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f994818ab20 con 0x7f9948189d00 2021-02-10T11:19:18.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.471 7f993f7fe700 1 -- 172.21.2.101:0/961793586 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9930000c90 con 0x7f9948189d00 2021-02-10T11:19:18.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.601 7f994cd6b700 1 -- 172.21.2.101:0/961793586 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839265", "target": ["mgr", ""]}) v1 -- 0x7f994818ab20 con 0x7f992c0186c0 2021-02-10T11:19:18.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.605 7f993f7fe700 1 -- 172.21.2.101:0/961793586 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f9938006390 con 0x7f992c0186c0 2021-02-10T11:19:18.614 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839265/77534c92-14a9-4f7d-a632-589a92b36558 2021-02-10T11:19:18.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.609 7f994cd6b700 1 -- 172.21.2.101:0/961793586 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f992c0186c0 msgr2=0x7f992c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:18.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.609 7f994cd6b700 1 --2- 172.21.2.101:0/961793586 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f992c0186c0 0x7f992c01abc0 crc :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:18.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.609 7f994cd6b700 1 -- 172.21.2.101:0/961793586 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9948189d00 msgr2=0x7f994818a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:18.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.609 7f994cd6b700 1 --2- 172.21.2.101:0/961793586 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9948189d00 0x7f994818a100 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f9930006f00 tx=0x7f993000b940).stop 2021-02-10T11:19:18.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.610 7f994cd6b700 1 -- 172.21.2.101:0/961793586 shutdown_connections 2021-02-10T11:19:18.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.610 7f994cd6b700 1 --2- 172.21.2.101:0/961793586 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f992c0186c0 0x7f992c01abc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:18.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.610 7f994cd6b700 1 --2- 172.21.2.101:0/961793586 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f994818a720 0x7f99480b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:18.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.610 7f994cd6b700 1 --2- 172.21.2.101:0/961793586 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9948189d00 0x7f994818a100 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:18.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.610 7f994cd6b700 1 --2- 172.21.2.101:0/961793586 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99480a0420 0x7f99480a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:18.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.610 7f994cd6b700 1 -- 172.21.2.101:0/961793586 >> 172.21.2.101:0/961793586 conn(0x7f9948094e10 msgr2=0x7f99480a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:18.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.610 7f994cd6b700 1 -- 172.21.2.101:0/961793586 shutdown_connections 2021-02-10T11:19:18.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:18.610 7f994cd6b700 1 -- 172.21.2.101:0/961793586 wait complete. 2021-02-10T11:19:18.624 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839265 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839265/77534c92-14a9-4f7d-a632-589a92b36558 2021-02-10T11:19:18.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:18.624 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:18.624 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:18.959 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:18.959 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:18.959 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839265/77534c92-14a9-4f7d-a632-589a92b36558/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:19.339 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:19.340 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:19.340 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249801 s, 42.0 MB/s 2021-02-10T11:19:19.341 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:19.341 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:19.341 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:19.676 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:19.677 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:19.677 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839265/77534c92-14a9-4f7d-a632-589a92b36558/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:20.053 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:20.053 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:20.053 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248412 s, 42.2 MB/s 2021-02-10T11:19:20.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:20.054 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:20.054 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:20.388 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:20.388 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:20.388 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839265/77534c92-14a9-4f7d-a632-589a92b36558/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:20.763 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:20.763 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:20.763 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023865 s, 43.9 MB/s 2021-02-10T11:19:20.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:20.765 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:20.765 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:21.099 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:21.099 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:21.099 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839265/77534c92-14a9-4f7d-a632-589a92b36558/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:21.472 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:21.472 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:21.473 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0226385 s, 46.3 MB/s 2021-02-10T11:19:21.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:21.473 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:21.474 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:21.808 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:21.808 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:21.808 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839265/77534c92-14a9-4f7d-a632-589a92b36558/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:22.183 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:22.183 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:22.183 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253699 s, 41.3 MB/s 2021-02-10T11:19:22.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:22.184 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:22.184 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:22.519 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:22.519 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:22.519 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839265/77534c92-14a9-4f7d-a632-589a92b36558/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:22.896 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:22.896 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:22.896 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0258082 s, 40.6 MB/s 2021-02-10T11:19:22.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:22.897 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:22.897 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:23.231 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:23.231 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:23.231 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839265/77534c92-14a9-4f7d-a632-589a92b36558/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:23.606 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:23.606 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:23.606 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0250243 s, 41.9 MB/s 2021-02-10T11:19:23.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:23.607 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:23.607 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:23.942 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:23.942 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:23.942 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839265/77534c92-14a9-4f7d-a632-589a92b36558/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:24.315 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:24.315 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:24.315 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249854 s, 42.0 MB/s 2021-02-10T11:19:24.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:24.316 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:24.316 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:24.651 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:24.651 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:24.652 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839265/77534c92-14a9-4f7d-a632-589a92b36558/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:25.028 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:25.028 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:25.028 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.026249 s, 39.9 MB/s 2021-02-10T11:19:25.029 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:25.029 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:25.029 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:25.364 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:25.364 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:25.365 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839265/77534c92-14a9-4f7d-a632-589a92b36558/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:25.739 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:25.740 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:25.740 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246886 s, 42.5 MB/s 2021-02-10T11:19:25.741 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839266 2021-02-10T11:19:26.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.149 7f03970d0700 1 Processor -- start 2021-02-10T11:19:26.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.149 7f03970d0700 1 -- start start 2021-02-10T11:19:26.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.149 7f03970d0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f03901b0470 0x7f03901b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:26.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.149 7f03970d0700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03900f0b50 con 0x7f03900ef610 2021-02-10T11:19:26.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.149 7f03970d0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03900f24e0 con 0x7f03900f20e0 2021-02-10T11:19:26.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.149 7f03970d0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03900e5ea0 con 0x7f03901b0470 2021-02-10T11:19:26.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.150 7f038ffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f03900ef610 0x7f03900efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46944/0 (socket says 172.21.2.101:46944) 2021-02-10T11:19:26.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.150 7f038ffff700 1 -- 172.21.2.101:0/1751609918 learned_addr learned my addr 172.21.2.101:0/1751609918 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:19:26.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.150 7f0394eec700 1 --2- 172.21.2.101:0/1751609918 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f03901b0470 0x7f03901b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:26.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.150 7f038f7fe700 1 -- 172.21.2.101:0/1751609918 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113516767 0 0) 0x7f03800009b0 con 0x7f03900ef610 2021-02-10T11:19:26.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.150 7f038f7fe700 1 -- 172.21.2.101:0/1751609918 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f037c002fe0 con 0x7f03900ef610 2021-02-10T11:19:26.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.150 7f0394eec700 1 -- 172.21.2.101:0/1751609918 >> v1:172.21.2.108:6789/0 conn(0x7f03900f20e0 legacy=0x7f03900f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:26.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.150 7f0394eec700 1 -- 172.21.2.101:0/1751609918 >> v1:172.21.2.101:6789/0 conn(0x7f03900ef610 legacy=0x7f03900efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:26.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.150 7f0394eec700 1 -- 172.21.2.101:0/1751609918 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03900e52e0 con 0x7f03901b0470 2021-02-10T11:19:26.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.151 7f0394eec700 1 --2- 172.21.2.101:0/1751609918 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f03901b0470 0x7f03901b0870 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f0378006650 tx=0x7f0378009a80).ready entity=mon.2 client_cookie=74aee57bb9fe89c7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:26.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.151 7f038f7fe700 1 -- 172.21.2.101:0/1751609918 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f037800c0c0 con 0x7f03901b0470 2021-02-10T11:19:26.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.151 7f038f7fe700 1 -- 172.21.2.101:0/1751609918 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037800c340 con 0x7f03901b0470 2021-02-10T11:19:26.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.151 7f038f7fe700 1 -- 172.21.2.101:0/1751609918 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f037800c860 con 0x7f03901b0470 2021-02-10T11:19:26.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.151 7f03970d0700 1 -- 172.21.2.101:0/1751609918 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f03901b0470 msgr2=0x7f03901b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:26.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.151 7f03970d0700 1 --2- 172.21.2.101:0/1751609918 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f03901b0470 0x7f03901b0870 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f0378006650 tx=0x7f0378009a80).stop 2021-02-10T11:19:26.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.151 7f03970d0700 1 -- 172.21.2.101:0/1751609918 shutdown_connections 2021-02-10T11:19:26.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.151 7f03970d0700 1 --2- 172.21.2.101:0/1751609918 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f03901b0470 0x7f03901b0870 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:26.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.151 7f03970d0700 1 -- 172.21.2.101:0/1751609918 >> 172.21.2.101:0/1751609918 conn(0x7f03900ce3e0 msgr2=0x7f03900e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:26.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.152 7f03970d0700 1 -- 172.21.2.101:0/1751609918 shutdown_connections 2021-02-10T11:19:26.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.152 7f03970d0700 1 -- 172.21.2.101:0/1751609918 wait complete. 2021-02-10T11:19:26.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.166 7f03970d0700 1 Processor -- start 2021-02-10T11:19:26.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.166 7f03970d0700 1 -- start start 2021-02-10T11:19:26.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.167 7f03970d0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f03900b2840 0x7f03901a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:26.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.167 7f03970d0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f03900b30d0 0x7f03900b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:26.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.167 7f03970d0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f03900b3af0 0x7f03900f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:26.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.167 7f03970d0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03900b2c40 con 0x7f03900b2840 2021-02-10T11:19:26.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.167 7f03970d0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03901a4920 con 0x7f03900b30d0 2021-02-10T11:19:26.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.167 7f03970d0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03900f0ba0 con 0x7f03900b3af0 2021-02-10T11:19:26.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.167 7f0394eec700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f03900b2840 0x7f03901a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:26.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.167 7f0394eec700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f03900b2840 0x7f03901a4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46454/0 (socket says 172.21.2.101:46454) 2021-02-10T11:19:26.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.167 7f0394eec700 1 -- 172.21.2.101:0/4019941698 learned_addr learned my addr 172.21.2.101:0/4019941698 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:19:26.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.167 7f0394eec700 1 -- 172.21.2.101:0/4019941698 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f03900b3af0 msgr2=0x7f03900f0580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:19:26.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.167 7f0394eec700 1 --2- 172.21.2.101:0/4019941698 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f03900b3af0 0x7f03900f0580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:26.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.167 7f0394eec700 1 -- 172.21.2.101:0/4019941698 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f03900b30d0 msgr2=0x7f03900b34d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:19:26.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.167 7f0394eec700 1 --2- 172.21.2.101:0/4019941698 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f03900b30d0 0x7f03900b34d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:26.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.167 7f0394eec700 1 -- 172.21.2.101:0/4019941698 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03900f0dc0 con 0x7f03900b2840 2021-02-10T11:19:26.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.167 7f0394eec700 1 --2- 172.21.2.101:0/4019941698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f03900b2840 0x7f03901a4300 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f0378009630 tx=0x7f037800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:26.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.168 7f038d7fa700 1 -- 172.21.2.101:0/4019941698 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f03780093a0 con 0x7f03900b2840 2021-02-10T11:19:26.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.168 7f038d7fa700 1 -- 172.21.2.101:0/4019941698 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037800bd70 con 0x7f03900b2840 2021-02-10T11:19:26.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.168 7f038d7fa700 1 -- 172.21.2.101:0/4019941698 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0378005560 con 0x7f03900b2840 2021-02-10T11:19:26.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.168 7f03970d0700 1 -- 172.21.2.101:0/4019941698 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03900f0dc0 con 0x7f03900b2840 2021-02-10T11:19:26.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.168 7f03970d0700 1 -- 172.21.2.101:0/4019941698 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03900f0dc0 con 0x7f03900b2840 2021-02-10T11:19:26.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.168 7f038d7fa700 1 -- 172.21.2.101:0/4019941698 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f037800a000 con 0x7f03900b2840 2021-02-10T11:19:26.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.168 7f038d7fa700 1 --2- 172.21.2.101:0/4019941698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f03740186c0 0x7f037401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:26.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.169 7f038d7fa700 1 -- 172.21.2.101:0/4019941698 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f03780325a0 con 0x7f03900b2840 2021-02-10T11:19:26.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.169 7f038ffff700 1 --2- 172.21.2.101:0/4019941698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f03740186c0 0x7f037401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:26.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.169 7f038ffff700 1 --2- 172.21.2.101:0/4019941698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f03740186c0 0x7f037401abc0 crc :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:26.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.199 7f03970d0700 1 -- 172.21.2.101:0/4019941698 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03900f0050 con 0x7f03900b2840 2021-02-10T11:19:26.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.202 7f038d7fa700 1 -- 172.21.2.101:0/4019941698 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0378027a10 con 0x7f03900b2840 2021-02-10T11:19:26.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.331 7f03970d0700 1 -- 172.21.2.101:0/4019941698 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839266", "target": ["mgr", ""]}) v1 -- 0x7f03900f0050 con 0x7f03740186c0 2021-02-10T11:19:26.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.356 7f038d7fa700 1 -- 172.21.2.101:0/4019941698 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f0380000dd0 con 0x7f03740186c0 2021-02-10T11:19:26.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.363 7f03970d0700 1 -- 172.21.2.101:0/4019941698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f03740186c0 msgr2=0x7f037401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:26.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.363 7f03970d0700 1 --2- 172.21.2.101:0/4019941698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f03740186c0 0x7f037401abc0 crc :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:26.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.363 7f03970d0700 1 -- 172.21.2.101:0/4019941698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f03900b2840 msgr2=0x7f03901a4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:26.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.363 7f03970d0700 1 --2- 172.21.2.101:0/4019941698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f03900b2840 0x7f03901a4300 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f0378009630 tx=0x7f037800b980).stop 2021-02-10T11:19:26.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.364 7f03970d0700 1 -- 172.21.2.101:0/4019941698 shutdown_connections 2021-02-10T11:19:26.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.364 7f03970d0700 1 --2- 172.21.2.101:0/4019941698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f03740186c0 0x7f037401abc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:26.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.364 7f03970d0700 1 --2- 172.21.2.101:0/4019941698 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f03900b3af0 0x7f03900f0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:26.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.364 7f03970d0700 1 --2- 172.21.2.101:0/4019941698 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f03900b30d0 0x7f03900b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:26.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.364 7f03970d0700 1 --2- 172.21.2.101:0/4019941698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f03900b2840 0x7f03901a4300 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:26.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.364 7f03970d0700 1 -- 172.21.2.101:0/4019941698 >> 172.21.2.101:0/4019941698 conn(0x7f03900c36f0 msgr2=0x7f0390094c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:26.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.364 7f03970d0700 1 -- 172.21.2.101:0/4019941698 shutdown_connections 2021-02-10T11:19:26.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.364 7f03970d0700 1 -- 172.21.2.101:0/4019941698 wait complete. 2021-02-10T11:19:26.378 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839266 2021-02-10T11:19:26.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.785 7fe6a9e17700 1 Processor -- start 2021-02-10T11:19:26.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.785 7fe6a9e17700 1 -- start start 2021-02-10T11:19:26.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.785 7fe6a9e17700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe6a41b0470 0x7fe6a41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:26.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.785 7fe6a9e17700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6a40ef610 0x7fe6a40efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:26.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.785 7fe6a9e17700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe6a40f0030 0x7fe6a40f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:26.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.785 7fe6a9e17700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a40e5ea0 con 0x7fe6a40f0030 2021-02-10T11:19:26.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.785 7fe6a9e17700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a40f0b50 con 0x7fe6a41b0470 2021-02-10T11:19:26.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.785 7fe6a9e17700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40e52e0 con 0x7fe6a40ef610 2021-02-10T11:19:26.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.786 7fe6a3fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe6a40f0030 0x7fe6a40f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:26.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.786 7fe6a3fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe6a40f0030 0x7fe6a40f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46462/0 (socket says 172.21.2.101:46462) 2021-02-10T11:19:26.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.786 7fe6a3fff700 1 -- 172.21.2.101:0/2080258420 learned_addr learned my addr 172.21.2.101:0/2080258420 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:19:26.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.786 7fe6a2ffd700 1 --2- 172.21.2.101:0/2080258420 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6a40ef610 0x7fe6a40efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:26.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.786 7fe6a37fe700 1 --2- 172.21.2.101:0/2080258420 >> v2:172.21.2.108:3300/0 conn(0x7fe6a41b0470 0x7fe6a41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:26.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.786 7fe6a3fff700 1 -- 172.21.2.101:0/2080258420 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6a40ef610 msgr2=0x7fe6a40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:19:26.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.786 7fe6a3fff700 1 --2- 172.21.2.101:0/2080258420 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6a40ef610 0x7fe6a40efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:26.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.786 7fe6a3fff700 1 -- 172.21.2.101:0/2080258420 >> v2:172.21.2.108:3300/0 conn(0x7fe6a41b0470 msgr2=0x7fe6a41b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:19:26.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.786 7fe6a3fff700 1 --2- 172.21.2.101:0/2080258420 >> v2:172.21.2.108:3300/0 conn(0x7fe6a41b0470 0x7fe6a41b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:26.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.786 7fe6a3fff700 1 -- 172.21.2.101:0/2080258420 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a40f10d0 con 0x7fe6a40f0030 2021-02-10T11:19:26.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.786 7fe6a37fe700 1 --2- 172.21.2.101:0/2080258420 >> v2:172.21.2.108:3300/0 conn(0x7fe6a41b0470 0x7fe6a41b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:19:26.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.786 7fe6a3fff700 1 --2- 172.21.2.101:0/2080258420 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6a40f0030 0x7fe6a40f0430 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7fe69c006f00 tx=0x7fe69c009f10).ready entity=mon.0 client_cookie=668c1589d38547ad server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:26.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.786 7fe6a27fc700 1 -- 172.21.2.101:0/2080258420 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe69c00b3b0 con 0x7fe6a40f0030 2021-02-10T11:19:26.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.786 7fe6a27fc700 1 -- 172.21.2.101:0/2080258420 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c00b730 con 0x7fe6a40f0030 2021-02-10T11:19:26.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.786 7fe6a27fc700 1 -- 172.21.2.101:0/2080258420 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe69c00bb90 con 0x7fe6a40f0030 2021-02-10T11:19:26.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.787 7fe6a9e17700 1 -- 172.21.2.101:0/2080258420 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6a40f0030 msgr2=0x7fe6a40f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:26.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.787 7fe6a9e17700 1 --2- 172.21.2.101:0/2080258420 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6a40f0030 0x7fe6a40f0430 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7fe69c006f00 tx=0x7fe69c009f10).stop 2021-02-10T11:19:26.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.787 7fe6a9e17700 1 -- 172.21.2.101:0/2080258420 shutdown_connections 2021-02-10T11:19:26.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.787 7fe6a9e17700 1 --2- 172.21.2.101:0/2080258420 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6a40ef610 0x7fe6a40efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:26.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.787 7fe6a9e17700 1 --2- 172.21.2.101:0/2080258420 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6a40f0030 0x7fe6a40f0430 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:26.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.787 7fe6a9e17700 1 --2- 172.21.2.101:0/2080258420 >> v2:172.21.2.108:3300/0 conn(0x7fe6a41b0470 0x7fe6a41b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:26.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.787 7fe6a9e17700 1 -- 172.21.2.101:0/2080258420 >> 172.21.2.101:0/2080258420 conn(0x7fe6a40ce3e0 msgr2=0x7fe6a40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:26.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.787 7fe6a9e17700 1 -- 172.21.2.101:0/2080258420 shutdown_connections 2021-02-10T11:19:26.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.787 7fe6a9e17700 1 -- 172.21.2.101:0/2080258420 wait complete. 2021-02-10T11:19:26.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.802 7fe6a9e17700 1 Processor -- start 2021-02-10T11:19:26.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.802 7fe6a9e17700 1 -- start start 2021-02-10T11:19:26.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.802 7fe6a9e17700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6a40b7db0 0x7fe6a40b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:26.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.802 7fe6a9e17700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6a40b8880 0x7fe6a41a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:26.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.802 7fe6a9e17700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6a40b8c80 0x7fe6a41a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:26.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.802 7fe6a9e17700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40b79b0 con 0x7fe6a40b8880 2021-02-10T11:19:26.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.802 7fe6a9e17700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a41a8090 con 0x7fe6a40b7db0 2021-02-10T11:19:26.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.802 7fe6a9e17700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a41a82b0 con 0x7fe6a40b8c80 2021-02-10T11:19:26.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.802 7fe6a2ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6a40b8880 0x7fe6a41a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:26.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.803 7fe6a2ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6a40b8880 0x7fe6a41a7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46470/0 (socket says 172.21.2.101:46470) 2021-02-10T11:19:26.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.803 7fe6a2ffd700 1 -- 172.21.2.101:0/818990593 learned_addr learned my addr 172.21.2.101:0/818990593 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:19:26.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.803 7fe6a2ffd700 1 -- 172.21.2.101:0/818990593 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6a40b8c80 msgr2=0x7fe6a41a7a70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:19:26.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.803 7fe6a2ffd700 1 --2- 172.21.2.101:0/818990593 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6a40b8c80 0x7fe6a41a7a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:26.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.803 7fe6a2ffd700 1 -- 172.21.2.101:0/818990593 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6a40b7db0 msgr2=0x7fe6a40b8260 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:19:26.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.803 7fe6a2ffd700 1 --2- 172.21.2.101:0/818990593 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6a40b7db0 0x7fe6a40b8260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:26.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.803 7fe6a2ffd700 1 -- 172.21.2.101:0/818990593 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a4190c10 con 0x7fe6a40b8880 2021-02-10T11:19:26.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.803 7fe6a2ffd700 1 --2- 172.21.2.101:0/818990593 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6a40b8880 0x7fe6a41a7450 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7fe698006f00 tx=0x7fe69800b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:26.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.803 7fe68bfff700 1 -- 172.21.2.101:0/818990593 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe69800bd30 con 0x7fe6a40b8880 2021-02-10T11:19:26.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.803 7fe68bfff700 1 -- 172.21.2.101:0/818990593 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe698002030 con 0x7fe6a40b8880 2021-02-10T11:19:26.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.803 7fe6a9e17700 1 -- 172.21.2.101:0/818990593 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a4190c10 con 0x7fe6a40b8880 2021-02-10T11:19:26.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.803 7fe6a9e17700 1 -- 172.21.2.101:0/818990593 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a40c1eb0 con 0x7fe6a40b8880 2021-02-10T11:19:26.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.803 7fe68bfff700 1 -- 172.21.2.101:0/818990593 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe698002510 con 0x7fe6a40b8880 2021-02-10T11:19:26.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.804 7fe68bfff700 1 -- 172.21.2.101:0/818990593 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe698002270 con 0x7fe6a40b8880 2021-02-10T11:19:26.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.804 7fe68bfff700 1 --2- 172.21.2.101:0/818990593 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6840186d0 0x7fe68401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:26.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.804 7fe68bfff700 1 -- 172.21.2.101:0/818990593 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe6980327b0 con 0x7fe6a40b8880 2021-02-10T11:19:26.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.804 7fe6a37fe700 1 --2- 172.21.2.101:0/818990593 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6840186d0 0x7fe68401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:26.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.805 7fe6a37fe700 1 --2- 172.21.2.101:0/818990593 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6840186d0 0x7fe68401abd0 crc :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:26.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.835 7fe6a9e17700 1 -- 172.21.2.101:0/818990593 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a4105b30 con 0x7fe6a40b8880 2021-02-10T11:19:26.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.838 7fe68bfff700 1 -- 172.21.2.101:0/818990593 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe698002010 con 0x7fe6a40b8880 2021-02-10T11:19:26.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.966 7fe6a9e17700 1 -- 172.21.2.101:0/818990593 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839266", "target": ["mgr", ""]}) v1 -- 0x7fe6a40c1eb0 con 0x7fe6840186d0 2021-02-10T11:19:26.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.969 7fe68bfff700 1 -- 172.21.2.101:0/818990593 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fe694000940 con 0x7fe6840186d0 2021-02-10T11:19:26.975 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839266/0a3a36f5-fb33-4a6e-92d2-d18d969b1762 2021-02-10T11:19:26.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.970 7fe6a9e17700 1 -- 172.21.2.101:0/818990593 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6840186d0 msgr2=0x7fe68401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:26.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.970 7fe6a9e17700 1 --2- 172.21.2.101:0/818990593 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6840186d0 0x7fe68401abd0 crc :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:26.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.970 7fe6a9e17700 1 -- 172.21.2.101:0/818990593 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6a40b8880 msgr2=0x7fe6a41a7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:26.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.970 7fe6a9e17700 1 --2- 172.21.2.101:0/818990593 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6a40b8880 0x7fe6a41a7450 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7fe698006f00 tx=0x7fe69800b940).stop 2021-02-10T11:19:26.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.970 7fe6a9e17700 1 -- 172.21.2.101:0/818990593 shutdown_connections 2021-02-10T11:19:26.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.970 7fe6a9e17700 1 --2- 172.21.2.101:0/818990593 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6840186d0 0x7fe68401abd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:26.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.970 7fe6a9e17700 1 --2- 172.21.2.101:0/818990593 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6a40b8c80 0x7fe6a41a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:26.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.970 7fe6a9e17700 1 --2- 172.21.2.101:0/818990593 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6a40b8880 0x7fe6a41a7450 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:26.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.970 7fe6a9e17700 1 --2- 172.21.2.101:0/818990593 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6a40b7db0 0x7fe6a40b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:26.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.970 7fe6a9e17700 1 -- 172.21.2.101:0/818990593 >> 172.21.2.101:0/818990593 conn(0x7fe6a4094e10 msgr2=0x7fe6a40a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:26.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.971 7fe6a9e17700 1 -- 172.21.2.101:0/818990593 shutdown_connections 2021-02-10T11:19:26.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:26.971 7fe6a9e17700 1 -- 172.21.2.101:0/818990593 wait complete. 2021-02-10T11:19:26.985 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839266 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839266/0a3a36f5-fb33-4a6e-92d2-d18d969b1762 2021-02-10T11:19:26.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:26.985 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:26.985 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:27.321 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:27.321 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:27.321 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839266/0a3a36f5-fb33-4a6e-92d2-d18d969b1762/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:27.701 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:27.701 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:27.701 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0223541 s, 46.9 MB/s 2021-02-10T11:19:27.702 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:27.702 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:27.702 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:28.036 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:28.037 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:28.037 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839266/0a3a36f5-fb33-4a6e-92d2-d18d969b1762/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:28.410 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:28.411 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:28.411 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0221224 s, 47.4 MB/s 2021-02-10T11:19:28.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:28.412 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:28.412 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:28.746 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:28.747 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:28.747 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839266/0a3a36f5-fb33-4a6e-92d2-d18d969b1762/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:29.120 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:29.121 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:29.121 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0256387 s, 40.9 MB/s 2021-02-10T11:19:29.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:29.122 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:29.122 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:29.457 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:29.457 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:29.458 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839266/0a3a36f5-fb33-4a6e-92d2-d18d969b1762/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:29.838 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:29.839 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:29.839 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234618 s, 44.7 MB/s 2021-02-10T11:19:29.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:29.840 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:29.840 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:30.179 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:30.180 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:30.180 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839266/0a3a36f5-fb33-4a6e-92d2-d18d969b1762/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:30.556 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:30.556 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:30.556 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247787 s, 42.3 MB/s 2021-02-10T11:19:30.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:30.558 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:30.558 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:30.894 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:30.894 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:30.894 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839266/0a3a36f5-fb33-4a6e-92d2-d18d969b1762/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:31.269 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:31.269 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:31.269 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235161 s, 44.6 MB/s 2021-02-10T11:19:31.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:31.270 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:31.270 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:31.604 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:31.604 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:31.605 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839266/0a3a36f5-fb33-4a6e-92d2-d18d969b1762/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:31.980 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:31.980 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:31.980 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242431 s, 43.3 MB/s 2021-02-10T11:19:31.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:31.981 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:31.981 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:32.316 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:32.317 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:32.317 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839266/0a3a36f5-fb33-4a6e-92d2-d18d969b1762/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:32.692 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:32.692 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:32.692 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241824 s, 43.4 MB/s 2021-02-10T11:19:32.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:32.693 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:32.693 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:33.028 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:33.028 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:33.029 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839266/0a3a36f5-fb33-4a6e-92d2-d18d969b1762/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:33.403 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:33.403 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:33.403 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237569 s, 44.1 MB/s 2021-02-10T11:19:33.404 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:33.405 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:33.405 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:33.738 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:33.739 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:33.739 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839266/0a3a36f5-fb33-4a6e-92d2-d18d969b1762/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:34.112 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:34.112 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:34.112 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0259709 s, 40.4 MB/s 2021-02-10T11:19:34.113 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839267 2021-02-10T11:19:34.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.520 7f44f2e60700 1 Processor -- start 2021-02-10T11:19:34.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.520 7f44f2e60700 1 -- start start 2021-02-10T11:19:34.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.520 7f44f2e60700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f44ec1b0870 0x7f44ec0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:34.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.520 7f44f2e60700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f44ec0efe00 0x7f44ec0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:34.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.520 7f44f2e60700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44ec1b0c70 con 0x7f44ec1b0470 2021-02-10T11:19:34.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.520 7f44f2e60700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44ec0e5ea0 con 0x7f44ec0efe00 2021-02-10T11:19:34.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.520 7f44f2e60700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44ec0f0b50 con 0x7f44ec1b0870 2021-02-10T11:19:34.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.520 7f44f147d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f44ec0efe00 0x7f44ec0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:34.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.520 7f44f147d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f44ec0efe00 0x7f44ec0f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46476/0 (socket says 172.21.2.101:46476) 2021-02-10T11:19:34.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.520 7f44f147d700 1 -- 172.21.2.101:0/868689098 learned_addr learned my addr 172.21.2.101:0/868689098 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:19:34.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.521 7f44f147d700 1 -- 172.21.2.101:0/868689098 >> v2:172.21.2.108:3300/0 conn(0x7f44ec1b0870 msgr2=0x7f44ec0f20e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:19:34.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.521 7f44f147d700 1 --2- 172.21.2.101:0/868689098 >> v2:172.21.2.108:3300/0 conn(0x7f44ec1b0870 0x7f44ec0f20e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:34.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.521 7f44f147d700 1 -- 172.21.2.101:0/868689098 >> v1:172.21.2.101:6789/0 conn(0x7f44ec1b0470 legacy=0x7f44ec0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:34.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.521 7f44f147d700 1 -- 172.21.2.101:0/868689098 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec0f0820 con 0x7f44ec0efe00 2021-02-10T11:19:34.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.521 7f44f147d700 1 --2- 172.21.2.101:0/868689098 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44ec0efe00 0x7f44ec0f0200 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f44e4006f00 tx=0x7f44e4009f10).ready entity=mon.0 client_cookie=de7cdb2bb442536d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:34.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.521 7f44eb7fe700 1 -- 172.21.2.101:0/868689098 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f44e400b3b0 con 0x7f44ec0efe00 2021-02-10T11:19:34.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.521 7f44eb7fe700 1 -- 172.21.2.101:0/868689098 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e400b630 con 0x7f44ec0efe00 2021-02-10T11:19:34.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.521 7f44eb7fe700 1 -- 172.21.2.101:0/868689098 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f44e400bb10 con 0x7f44ec0efe00 2021-02-10T11:19:34.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.521 7f44f2e60700 1 -- 172.21.2.101:0/868689098 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44ec0efe00 msgr2=0x7f44ec0f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:34.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.521 7f44f2e60700 1 --2- 172.21.2.101:0/868689098 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44ec0efe00 0x7f44ec0f0200 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f44e4006f00 tx=0x7f44e4009f10).stop 2021-02-10T11:19:34.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.521 7f44f2e60700 1 -- 172.21.2.101:0/868689098 shutdown_connections 2021-02-10T11:19:34.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.521 7f44f2e60700 1 --2- 172.21.2.101:0/868689098 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44ec0efe00 0x7f44ec0f0200 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:34.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.521 7f44f2e60700 1 --2- 172.21.2.101:0/868689098 >> v2:172.21.2.108:3300/0 conn(0x7f44ec1b0870 0x7f44ec0f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:34.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.521 7f44f2e60700 1 -- 172.21.2.101:0/868689098 >> 172.21.2.101:0/868689098 conn(0x7f44ec0ce3e0 msgr2=0x7f44ec0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:34.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.522 7f44f2e60700 1 -- 172.21.2.101:0/868689098 shutdown_connections 2021-02-10T11:19:34.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.522 7f44f2e60700 1 -- 172.21.2.101:0/868689098 wait complete. 2021-02-10T11:19:34.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.536 7f44f2e60700 1 Processor -- start 2021-02-10T11:19:34.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.536 7f44f2e60700 1 -- start start 2021-02-10T11:19:34.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.536 7f44f2e60700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44ec0c36a0 0x7f44ec0c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:34.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.536 7f44f2e60700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44ec0c3190 0x7f44ec0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:34.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.536 7f44f2e60700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44ec0b2370 0x7f44ec0b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:34.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.536 7f44f2e60700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec0b27c0 con 0x7f44ec0c3190 2021-02-10T11:19:34.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.536 7f44f2e60700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec0b3360 con 0x7f44ec0c36a0 2021-02-10T11:19:34.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.536 7f44f2e60700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec0b3540 con 0x7f44ec0b2370 2021-02-10T11:19:34.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.536 7f44ebfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44ec0c3190 0x7f44ec0b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:34.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.537 7f44ebfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44ec0c3190 0x7f44ec0b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46478/0 (socket says 172.21.2.101:46478) 2021-02-10T11:19:34.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.537 7f44ebfff700 1 -- 172.21.2.101:0/484364028 learned_addr learned my addr 172.21.2.101:0/484364028 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:19:34.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.537 7f44ebfff700 1 -- 172.21.2.101:0/484364028 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44ec0b2370 msgr2=0x7f44ec0b2d40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:19:34.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.537 7f44ebfff700 1 --2- 172.21.2.101:0/484364028 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44ec0b2370 0x7f44ec0b2d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:34.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.537 7f44ebfff700 1 -- 172.21.2.101:0/484364028 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44ec0c36a0 msgr2=0x7f44ec0c2b70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:19:34.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.537 7f44ebfff700 1 --2- 172.21.2.101:0/484364028 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44ec0c36a0 0x7f44ec0c2b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:34.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.537 7f44ebfff700 1 -- 172.21.2.101:0/484364028 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec0b3760 con 0x7f44ec0c3190 2021-02-10T11:19:34.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.537 7f44ebfff700 1 --2- 172.21.2.101:0/484364028 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44ec0c3190 0x7f44ec0b1d50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f44dc00b5c0 tx=0x7f44dc00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:34.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.537 7f44e97fa700 1 -- 172.21.2.101:0/484364028 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f44dc006b80 con 0x7f44ec0c3190 2021-02-10T11:19:34.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.537 7f44e97fa700 1 -- 172.21.2.101:0/484364028 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44dc00bd50 con 0x7f44ec0c3190 2021-02-10T11:19:34.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.537 7f44e97fa700 1 -- 172.21.2.101:0/484364028 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f44dc0054d0 con 0x7f44ec0c3190 2021-02-10T11:19:34.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.537 7f44f2e60700 1 -- 172.21.2.101:0/484364028 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44ec0b3760 con 0x7f44ec0c3190 2021-02-10T11:19:34.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.537 7f44f2e60700 1 -- 172.21.2.101:0/484364028 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44ec0b3760 con 0x7f44ec0c3190 2021-02-10T11:19:34.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.538 7f44e97fa700 1 -- 172.21.2.101:0/484364028 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f44dc006030 con 0x7f44ec0c3190 2021-02-10T11:19:34.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.538 7f44e97fa700 1 --2- 172.21.2.101:0/484364028 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44d00186c0 0x7f44d001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:34.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.538 7f44e97fa700 1 -- 172.21.2.101:0/484364028 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f44dc032580 con 0x7f44ec0c3190 2021-02-10T11:19:34.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.538 7f44f0c7c700 1 --2- 172.21.2.101:0/484364028 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44d00186c0 0x7f44d001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:34.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.538 7f44f0c7c700 1 --2- 172.21.2.101:0/484364028 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44d00186c0 0x7f44d001abc0 crc :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:34.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.570 7f44f2e60700 1 -- 172.21.2.101:0/484364028 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44ec0edd10 con 0x7f44ec0c3190 2021-02-10T11:19:34.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.573 7f44e97fa700 1 -- 172.21.2.101:0/484364028 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f44dc006b80 con 0x7f44ec0c3190 2021-02-10T11:19:34.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.698 7f44f2e60700 1 -- 172.21.2.101:0/484364028 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839267", "target": ["mgr", ""]}) v1 -- 0x7f44ec0e52e0 con 0x7f44d00186c0 2021-02-10T11:19:35.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.724 7f44e97fa700 1 -- 172.21.2.101:0/484364028 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f44d4006db0 con 0x7f44d00186c0 2021-02-10T11:19:35.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.731 7f44f2e60700 1 -- 172.21.2.101:0/484364028 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44d00186c0 msgr2=0x7f44d001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:35.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.731 7f44f2e60700 1 --2- 172.21.2.101:0/484364028 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44d00186c0 0x7f44d001abc0 crc :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:35.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.731 7f44f2e60700 1 -- 172.21.2.101:0/484364028 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44ec0c3190 msgr2=0x7f44ec0b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:35.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.731 7f44f2e60700 1 --2- 172.21.2.101:0/484364028 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44ec0c3190 0x7f44ec0b1d50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f44dc00b5c0 tx=0x7f44dc00b960).stop 2021-02-10T11:19:35.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.732 7f44f2e60700 1 -- 172.21.2.101:0/484364028 shutdown_connections 2021-02-10T11:19:35.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.732 7f44f2e60700 1 --2- 172.21.2.101:0/484364028 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44d00186c0 0x7f44d001abc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:35.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.732 7f44f2e60700 1 --2- 172.21.2.101:0/484364028 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44ec0b2370 0x7f44ec0b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:35.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.732 7f44f2e60700 1 --2- 172.21.2.101:0/484364028 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44ec0c3190 0x7f44ec0b1d50 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:35.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.732 7f44f2e60700 1 --2- 172.21.2.101:0/484364028 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44ec0c36a0 0x7f44ec0c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:19:35.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.732 7f44f2e60700 1 -- 172.21.2.101:0/484364028 >> 172.21.2.101:0/484364028 conn(0x7f44ec09f630 msgr2=0x7f44ec0a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:35.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.732 7f44f2e60700 1 -- 172.21.2.101:0/484364028 shutdown_connections 2021-02-10T11:19:35.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:34.732 7f44f2e60700 1 -- 172.21.2.101:0/484364028 wait complete. 2021-02-10T11:19:35.943 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839267 2021-02-10T11:19:36.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.351 7fea97d96700 1 Processor -- start 2021-02-10T11:19:36.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.351 7fea97d96700 1 -- start start 2021-02-10T11:19:36.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.351 7fea97d96700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea901b0870 0x7fea900f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:36.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.351 7fea97d96700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea901b0c70 con 0x7fea900efe00 2021-02-10T11:19:36.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.351 7fea97d96700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea900f0b50 con 0x7fea901b0470 2021-02-10T11:19:36.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.351 7fea97d96700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea900e5ea0 con 0x7fea901b0870 2021-02-10T11:19:36.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.351 7fea963b3700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fea900efe00 0x7fea900f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:46984/0 (socket says 172.21.2.101:46984) 2021-02-10T11:19:36.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.351 7fea963b3700 1 -- 172.21.2.101:0/3024914581 learned_addr learned my addr 172.21.2.101:0/3024914581 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:19:36.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.352 7fea94bb0700 1 -- 172.21.2.101:0/3024914581 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037785202 0 0) 0x7fea88001500 con 0x7fea900efe00 2021-02-10T11:19:36.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.352 7fea94bb0700 1 -- 172.21.2.101:0/3024914581 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea7c002fe0 con 0x7fea900efe00 2021-02-10T11:19:36.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.352 7fea94bb0700 1 -- 172.21.2.101:0/3024914581 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2830658557 0 0) 0x7fea88001270 con 0x7fea900efe00 2021-02-10T11:19:36.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.352 7fea953b1700 1 --2- 172.21.2.101:0/3024914581 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea901b0870 0x7fea900f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:36.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.352 7fea94bb0700 1 -- 172.21.2.101:0/3024914581 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea901b0870 msgr2=0x7fea900f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:19:36.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.352 7fea94bb0700 1 --2- 172.21.2.101:0/3024914581 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea901b0870 0x7fea900f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:36.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.352 7fea94bb0700 1 -- 172.21.2.101:0/3024914581 >> v1:172.21.2.108:6789/0 conn(0x7fea901b0470 legacy=0x7fea900ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:19:36.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.352 7fea94bb0700 1 -- 172.21.2.101:0/3024914581 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea900e52e0 con 0x7fea900efe00 2021-02-10T11:19:36.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.352 7fea94bb0700 1 -- 172.21.2.101:0/3024914581 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fea88001b50 con 0x7fea900efe00 2021-02-10T11:19:36.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.352 7fea94bb0700 1 -- 172.21.2.101:0/3024914581 >> v1:172.21.2.101:6789/0 conn(0x7fea900efe00 legacy=0x7fea900f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:36.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.352 7fea94bb0700 1 --2- 172.21.2.101:0/3024914581 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea7c003af0 0x7fea7c005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:36.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.352 7fea94bb0700 1 --2- 172.21.2.101:0/3024914581 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea7c006820 0x7fea7c008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:36.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.352 7fea94bb0700 1 --2- 172.21.2.101:0/3024914581 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea7c009640 0x7fea7c00bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:36.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.352 7fea94bb0700 1 -- 172.21.2.101:0/3024914581 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c00c380 con 0x7fea7c006820 2021-02-10T11:19:36.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.352 7fea94bb0700 1 -- 172.21.2.101:0/3024914581 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c00c5a0 con 0x7fea7c003af0 2021-02-10T11:19:36.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.352 7fea94bb0700 1 -- 172.21.2.101:0/3024914581 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c00c7c0 con 0x7fea7c009640 2021-02-10T11:19:36.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.352 7fea953b1700 1 --2- 172.21.2.101:0/3024914581 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea901b0870 0x7fea900f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:19:36.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.352 7fea953b1700 1 --2- 172.21.2.101:0/3024914581 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea7c006820 0x7fea7c008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:36.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.352 7fea95bb2700 1 --2- 172.21.2.101:0/3024914581 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea7c003af0 0x7fea7c005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:36.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.353 7fea963b3700 1 --2- 172.21.2.101:0/3024914581 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea7c009640 0x7fea7c00bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:36.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.353 7fea953b1700 1 -- 172.21.2.101:0/3024914581 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea7c009640 msgr2=0x7fea7c00bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:19:36.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.353 7fea953b1700 1 --2- 172.21.2.101:0/3024914581 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea7c009640 0x7fea7c00bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:36.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.353 7fea953b1700 1 -- 172.21.2.101:0/3024914581 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea7c003af0 msgr2=0x7fea7c005f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:19:36.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.353 7fea953b1700 1 --2- 172.21.2.101:0/3024914581 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea7c003af0 0x7fea7c005f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:36.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.353 7fea953b1700 1 -- 172.21.2.101:0/3024914581 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7c00ca60 con 0x7fea7c006820 2021-02-10T11:19:36.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.353 7fea953b1700 1 --2- 172.21.2.101:0/3024914581 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea7c006820 0x7fea7c008d40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7fea8c002e00 tx=0x7fea8c04dac0).ready entity=mon.0 client_cookie=1efe6409a2ba6a56 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:36.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.353 7fea94bb0700 1 -- 172.21.2.101:0/3024914581 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fea8c000c90 con 0x7fea7c006820 2021-02-10T11:19:36.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.353 7fea94bb0700 1 -- 172.21.2.101:0/3024914581 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8c002030 con 0x7fea7c006820 2021-02-10T11:19:36.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.353 7fea94bb0700 1 -- 172.21.2.101:0/3024914581 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fea8c002550 con 0x7fea7c006820 2021-02-10T11:19:36.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.353 7fea97d96700 1 -- 172.21.2.101:0/3024914581 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea7c006820 msgr2=0x7fea7c008d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:36.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.353 7fea97d96700 1 --2- 172.21.2.101:0/3024914581 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea7c006820 0x7fea7c008d40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7fea8c002e00 tx=0x7fea8c04dac0).stop 2021-02-10T11:19:36.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.353 7fea963b3700 1 -- 172.21.2.101:0/3024914581 reap_dead start 2021-02-10T11:19:36.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.354 7fea97d96700 1 -- 172.21.2.101:0/3024914581 shutdown_connections 2021-02-10T11:19:36.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.354 7fea97d96700 1 -- 172.21.2.101:0/3024914581 >> 172.21.2.101:0/3024914581 conn(0x7fea900ce3e0 msgr2=0x7fea900e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:36.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.354 7fea97d96700 1 -- 172.21.2.101:0/3024914581 shutdown_connections 2021-02-10T11:19:36.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.354 7fea97d96700 1 -- 172.21.2.101:0/3024914581 wait complete. 2021-02-10T11:19:36.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.369 7fea97d96700 1 Processor -- start 2021-02-10T11:19:36.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.369 7fea97d96700 1 -- start start 2021-02-10T11:19:36.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.369 7fea97d96700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea9009b930 0x7fea900b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:36.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.369 7fea97d96700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea900b8410 0x7fea900b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:36.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.369 7fea97d96700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea900c1f80 0x7fea900c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:36.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.369 7fea97d96700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea900c1a00 con 0x7fea9009b930 2021-02-10T11:19:36.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.369 7fea97d96700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea900b8e30 con 0x7fea900c1f80 2021-02-10T11:19:36.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.369 7fea97d96700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea900b9050 con 0x7fea900b8410 2021-02-10T11:19:36.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.369 7fea95bb2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea9009b930 0x7fea900b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:36.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.369 7fea95bb2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea9009b930 0x7fea900b7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46498/0 (socket says 172.21.2.101:46498) 2021-02-10T11:19:36.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.369 7fea95bb2700 1 -- 172.21.2.101:0/1168480993 learned_addr learned my addr 172.21.2.101:0/1168480993 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:19:36.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.369 7fea963b3700 1 --2- 172.21.2.101:0/1168480993 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea900c1f80 0x7fea900c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:36.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.369 7fea953b1700 1 --2- 172.21.2.101:0/1168480993 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea900b8410 0x7fea900b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:36.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.370 7fea95bb2700 1 -- 172.21.2.101:0/1168480993 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea900b8410 msgr2=0x7fea900b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:36.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.370 7fea95bb2700 1 --2- 172.21.2.101:0/1168480993 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea900b8410 0x7fea900b8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:36.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.370 7fea95bb2700 1 -- 172.21.2.101:0/1168480993 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea900c1f80 msgr2=0x7fea900c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:36.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.370 7fea95bb2700 1 --2- 172.21.2.101:0/1168480993 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea900c1f80 0x7fea900c2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:36.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.370 7fea95bb2700 1 -- 172.21.2.101:0/1168480993 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea900c29a0 con 0x7fea9009b930 2021-02-10T11:19:36.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.370 7fea95bb2700 1 --2- 172.21.2.101:0/1168480993 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea9009b930 0x7fea900b7c20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7fea8400b5e0 tx=0x7fea8400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:36.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.370 7fea827fc700 1 -- 172.21.2.101:0/1168480993 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fea8400bd70 con 0x7fea9009b930 2021-02-10T11:19:36.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.370 7fea827fc700 1 -- 172.21.2.101:0/1168480993 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea840052e0 con 0x7fea9009b930 2021-02-10T11:19:36.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.370 7fea827fc700 1 -- 172.21.2.101:0/1168480993 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fea84005740 con 0x7fea9009b930 2021-02-10T11:19:36.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.370 7fea97d96700 1 -- 172.21.2.101:0/1168480993 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea900c29a0 con 0x7fea9009b930 2021-02-10T11:19:36.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.370 7fea97d96700 1 -- 172.21.2.101:0/1168480993 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea900c29a0 con 0x7fea9009b930 2021-02-10T11:19:36.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.371 7fea827fc700 1 -- 172.21.2.101:0/1168480993 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fea840054c0 con 0x7fea9009b930 2021-02-10T11:19:36.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.371 7fea827fc700 1 --2- 172.21.2.101:0/1168480993 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea780186c0 0x7fea7801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:36.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.371 7fea827fc700 1 -- 172.21.2.101:0/1168480993 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fea84032450 con 0x7fea9009b930 2021-02-10T11:19:36.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.371 7fea953b1700 1 --2- 172.21.2.101:0/1168480993 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea780186c0 0x7fea7801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:36.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.371 7fea953b1700 1 --2- 172.21.2.101:0/1168480993 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea780186c0 0x7fea7801abc0 crc :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:36.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.401 7fea97d96700 1 -- 172.21.2.101:0/1168480993 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea90105b30 con 0x7fea9009b930 2021-02-10T11:19:36.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.403 7fea827fc700 1 -- 172.21.2.101:0/1168480993 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fea8402d2a0 con 0x7fea9009b930 2021-02-10T11:19:36.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.532 7fea97d96700 1 -- 172.21.2.101:0/1168480993 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839267", "target": ["mgr", ""]}) v1 -- 0x7fea900b8e30 con 0x7fea780186c0 2021-02-10T11:19:36.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.535 7fea827fc700 1 -- 172.21.2.101:0/1168480993 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fea8c04d260 con 0x7fea780186c0 2021-02-10T11:19:36.546 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839267/8c97bd07-8d47-4179-967a-ebcd14264c96 2021-02-10T11:19:36.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.541 7fea97d96700 1 -- 172.21.2.101:0/1168480993 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea780186c0 msgr2=0x7fea7801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:36.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.541 7fea97d96700 1 --2- 172.21.2.101:0/1168480993 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea780186c0 0x7fea7801abc0 crc :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:36.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.541 7fea97d96700 1 -- 172.21.2.101:0/1168480993 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea9009b930 msgr2=0x7fea900b7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:36.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.541 7fea97d96700 1 --2- 172.21.2.101:0/1168480993 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea9009b930 0x7fea900b7c20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7fea8400b5e0 tx=0x7fea8400b980).stop 2021-02-10T11:19:36.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.541 7fea97d96700 1 -- 172.21.2.101:0/1168480993 shutdown_connections 2021-02-10T11:19:36.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.541 7fea97d96700 1 --2- 172.21.2.101:0/1168480993 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea780186c0 0x7fea7801abc0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:36.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.541 7fea97d96700 1 --2- 172.21.2.101:0/1168480993 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea900b8410 0x7fea900b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:36.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.541 7fea97d96700 1 --2- 172.21.2.101:0/1168480993 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea900c1f80 0x7fea900c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:36.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.541 7fea97d96700 1 --2- 172.21.2.101:0/1168480993 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea9009b930 0x7fea900b7c20 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:36.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.541 7fea97d96700 1 -- 172.21.2.101:0/1168480993 >> 172.21.2.101:0/1168480993 conn(0x7fea90094e10 msgr2=0x7fea900a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:36.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.541 7fea97d96700 1 -- 172.21.2.101:0/1168480993 shutdown_connections 2021-02-10T11:19:36.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:36.541 7fea97d96700 1 -- 172.21.2.101:0/1168480993 wait complete. 2021-02-10T11:19:36.555 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839267 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839267/8c97bd07-8d47-4179-967a-ebcd14264c96 2021-02-10T11:19:36.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:36.556 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:36.556 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:36.893 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:36.893 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:36.894 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839267/8c97bd07-8d47-4179-967a-ebcd14264c96/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:37.274 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:37.275 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:37.275 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236438 s, 44.3 MB/s 2021-02-10T11:19:37.276 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:37.276 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:37.276 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:37.611 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:37.611 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:37.611 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839267/8c97bd07-8d47-4179-967a-ebcd14264c96/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:37.987 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:37.987 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:37.987 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245637 s, 42.7 MB/s 2021-02-10T11:19:37.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:37.988 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:37.988 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:38.324 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:38.324 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:38.325 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839267/8c97bd07-8d47-4179-967a-ebcd14264c96/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:38.699 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:38.700 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:38.700 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233896 s, 44.8 MB/s 2021-02-10T11:19:38.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:38.701 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:38.701 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:39.036 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:39.036 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:39.036 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839267/8c97bd07-8d47-4179-967a-ebcd14264c96/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:39.408 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:39.408 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:39.408 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244765 s, 42.8 MB/s 2021-02-10T11:19:39.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:39.410 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:39.410 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:39.744 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:39.745 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:39.745 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839267/8c97bd07-8d47-4179-967a-ebcd14264c96/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:40.124 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:40.124 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:40.124 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247379 s, 42.4 MB/s 2021-02-10T11:19:40.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:40.125 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:40.125 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:40.460 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:40.460 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:40.461 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839267/8c97bd07-8d47-4179-967a-ebcd14264c96/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:40.833 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:40.834 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:40.834 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246483 s, 42.5 MB/s 2021-02-10T11:19:40.835 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:40.835 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:40.835 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:41.170 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:41.170 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:41.171 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839267/8c97bd07-8d47-4179-967a-ebcd14264c96/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:41.548 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:41.548 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:41.548 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249485 s, 42.0 MB/s 2021-02-10T11:19:41.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:41.549 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:41.549 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:41.884 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:41.884 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:41.885 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839267/8c97bd07-8d47-4179-967a-ebcd14264c96/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:42.260 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:42.261 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:42.261 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246372 s, 42.6 MB/s 2021-02-10T11:19:42.262 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:42.262 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:42.262 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:42.598 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:42.598 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:42.599 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839267/8c97bd07-8d47-4179-967a-ebcd14264c96/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:42.974 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:42.974 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:42.974 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0250723 s, 41.8 MB/s 2021-02-10T11:19:42.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:42.975 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:42.976 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:43.309 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:43.310 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:43.310 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839267/8c97bd07-8d47-4179-967a-ebcd14264c96/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:43.683 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:43.683 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:43.683 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.022805 s, 46.0 MB/s 2021-02-10T11:19:43.684 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839268 2021-02-10T11:19:44.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.091 7f2d886d8700 1 Processor -- start 2021-02-10T11:19:44.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.091 7f2d886d8700 1 -- start start 2021-02-10T11:19:44.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.091 7f2d886d8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2d801b0870 0x7f2d800f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:44.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.091 7f2d886d8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d800efe00 0x7f2d800f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:44.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.091 7f2d886d8700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d801b0c70 con 0x7f2d801b0470 2021-02-10T11:19:44.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.091 7f2d886d8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d800f0b50 con 0x7f2d801b0870 2021-02-10T11:19:44.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.091 7f2d886d8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d800f0820 con 0x7f2d800efe00 2021-02-10T11:19:44.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.091 7f2d85cf3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2d801b0870 0x7f2d800f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:44.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.092 7f2d85cf3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2d801b0870 0x7f2d800f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46506/0 (socket says 172.21.2.101:46506) 2021-02-10T11:19:44.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.092 7f2d864f4700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f2d801b0470 0x7f2d800ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47000/0 (socket says 172.21.2.101:47000) 2021-02-10T11:19:44.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.092 7f2d85cf3700 1 -- 172.21.2.101:0/812307894 learned_addr learned my addr 172.21.2.101:0/812307894 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:19:44.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.092 7f2d86cf5700 1 --2- 172.21.2.101:0/812307894 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d800efe00 0x7f2d800f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:44.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.092 7f2d854f2700 1 -- 172.21.2.101:0/812307894 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2721361356 0 0) 0x7f2d7c0010b0 con 0x7f2d801b0470 2021-02-10T11:19:44.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.092 7f2d854f2700 1 -- 172.21.2.101:0/812307894 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d6c002fe0 con 0x7f2d801b0470 2021-02-10T11:19:44.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.092 7f2d85cf3700 1 -- 172.21.2.101:0/812307894 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d800efe00 msgr2=0x7f2d800f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:19:44.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.092 7f2d85cf3700 1 --2- 172.21.2.101:0/812307894 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d800efe00 0x7f2d800f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:44.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.092 7f2d85cf3700 1 -- 172.21.2.101:0/812307894 >> v1:172.21.2.101:6789/0 conn(0x7f2d801b0470 legacy=0x7f2d800ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:44.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.092 7f2d85cf3700 1 -- 172.21.2.101:0/812307894 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d800e52e0 con 0x7f2d801b0870 2021-02-10T11:19:44.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.092 7f2d86cf5700 1 --2- 172.21.2.101:0/812307894 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d800efe00 0x7f2d800f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:19:44.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.092 7f2d85cf3700 1 --2- 172.21.2.101:0/812307894 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d801b0870 0x7f2d800f20e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f2d70006700 tx=0x7f2d70009b80).ready entity=mon.0 client_cookie=aa6b8a7fcaf8035a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:44.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.093 7f2d854f2700 1 -- 172.21.2.101:0/812307894 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d7000c0c0 con 0x7f2d801b0870 2021-02-10T11:19:44.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.093 7f2d854f2700 1 -- 172.21.2.101:0/812307894 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7000c400 con 0x7f2d801b0870 2021-02-10T11:19:44.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.093 7f2d854f2700 1 -- 172.21.2.101:0/812307894 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d7000c920 con 0x7f2d801b0870 2021-02-10T11:19:44.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.093 7f2d886d8700 1 -- 172.21.2.101:0/812307894 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d801b0870 msgr2=0x7f2d800f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:44.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.093 7f2d886d8700 1 --2- 172.21.2.101:0/812307894 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d801b0870 0x7f2d800f20e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f2d70006700 tx=0x7f2d70009b80).stop 2021-02-10T11:19:44.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.093 7f2d886d8700 1 -- 172.21.2.101:0/812307894 shutdown_connections 2021-02-10T11:19:44.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.093 7f2d886d8700 1 --2- 172.21.2.101:0/812307894 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d800efe00 0x7f2d800f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:44.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.093 7f2d886d8700 1 --2- 172.21.2.101:0/812307894 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d801b0870 0x7f2d800f20e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:44.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.093 7f2d886d8700 1 -- 172.21.2.101:0/812307894 >> 172.21.2.101:0/812307894 conn(0x7f2d800ce3e0 msgr2=0x7f2d800e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:44.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.093 7f2d886d8700 1 -- 172.21.2.101:0/812307894 shutdown_connections 2021-02-10T11:19:44.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.093 7f2d886d8700 1 -- 172.21.2.101:0/812307894 wait complete. 2021-02-10T11:19:44.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.108 7f2d886d8700 1 Processor -- start 2021-02-10T11:19:44.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.108 7f2d886d8700 1 -- start start 2021-02-10T11:19:44.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.108 7f2d886d8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d800a0420 0x7f2d800a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:44.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.108 7f2d886d8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d80189d00 0x7f2d8018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:44.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.108 7f2d886d8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d8018a720 0x7f2d800b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:44.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.108 7f2d886d8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d80189800 con 0x7f2d80189d00 2021-02-10T11:19:44.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.108 7f2d886d8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8018ab20 con 0x7f2d8018a720 2021-02-10T11:19:44.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.108 7f2d886d8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d800a1140 con 0x7f2d800a0420 2021-02-10T11:19:44.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.109 7f2d85cf3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d80189d00 0x7f2d8018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:44.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.109 7f2d85cf3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d80189d00 0x7f2d8018a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46514/0 (socket says 172.21.2.101:46514) 2021-02-10T11:19:44.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.109 7f2d86cf5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d8018a720 0x7f2d800b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:44.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.109 7f2d85cf3700 1 -- 172.21.2.101:0/835294674 learned_addr learned my addr 172.21.2.101:0/835294674 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:19:44.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.109 7f2d864f4700 1 --2- 172.21.2.101:0/835294674 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d800a0420 0x7f2d800a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:44.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.109 7f2d85cf3700 1 -- 172.21.2.101:0/835294674 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d800a0420 msgr2=0x7f2d800a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:44.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.109 7f2d85cf3700 1 --2- 172.21.2.101:0/835294674 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d800a0420 0x7f2d800a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:44.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.109 7f2d85cf3700 1 -- 172.21.2.101:0/835294674 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d8018a720 msgr2=0x7f2d800b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:44.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.109 7f2d85cf3700 1 --2- 172.21.2.101:0/835294674 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d8018a720 0x7f2d800b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:44.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.109 7f2d85cf3700 1 -- 172.21.2.101:0/835294674 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d800b8200 con 0x7f2d80189d00 2021-02-10T11:19:44.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.109 7f2d85cf3700 1 --2- 172.21.2.101:0/835294674 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d80189d00 0x7f2d8018a100 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f2d7000b580 tx=0x7f2d7000b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:44.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.109 7f2d76ffd700 1 -- 172.21.2.101:0/835294674 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d7000bd10 con 0x7f2d80189d00 2021-02-10T11:19:44.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.109 7f2d76ffd700 1 -- 172.21.2.101:0/835294674 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d700052e0 con 0x7f2d80189d00 2021-02-10T11:19:44.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.109 7f2d76ffd700 1 -- 172.21.2.101:0/835294674 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d70005740 con 0x7f2d80189d00 2021-02-10T11:19:44.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.109 7f2d886d8700 1 -- 172.21.2.101:0/835294674 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d800b8200 con 0x7f2d80189d00 2021-02-10T11:19:44.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.110 7f2d886d8700 1 -- 172.21.2.101:0/835294674 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d800b8200 con 0x7f2d80189d00 2021-02-10T11:19:44.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.110 7f2d76ffd700 1 -- 172.21.2.101:0/835294674 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2d70006240 con 0x7f2d80189d00 2021-02-10T11:19:44.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.110 7f2d76ffd700 1 --2- 172.21.2.101:0/835294674 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d680186c0 0x7f2d6801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:44.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.110 7f2d76ffd700 1 -- 172.21.2.101:0/835294674 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2d700323e0 con 0x7f2d80189d00 2021-02-10T11:19:44.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.110 7f2d864f4700 1 --2- 172.21.2.101:0/835294674 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d680186c0 0x7f2d6801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:44.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.111 7f2d864f4700 1 --2- 172.21.2.101:0/835294674 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d680186c0 0x7f2d6801abc0 crc :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:44.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.141 7f2d886d8700 1 -- 172.21.2.101:0/835294674 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d80105ae0 con 0x7f2d80189d00 2021-02-10T11:19:44.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.144 7f2d76ffd700 1 -- 172.21.2.101:0/835294674 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2d7002d360 con 0x7f2d80189d00 2021-02-10T11:19:44.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.271 7f2d886d8700 1 -- 172.21.2.101:0/835294674 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839268", "target": ["mgr", ""]}) v1 -- 0x7f2d8018ab20 con 0x7f2d680186c0 2021-02-10T11:19:44.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.294 7f2d76ffd700 1 -- 172.21.2.101:0/835294674 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f2d7c0014b0 con 0x7f2d680186c0 2021-02-10T11:19:44.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.304 7f2d886d8700 1 -- 172.21.2.101:0/835294674 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d680186c0 msgr2=0x7f2d6801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:44.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.304 7f2d886d8700 1 --2- 172.21.2.101:0/835294674 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d680186c0 0x7f2d6801abc0 crc :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:44.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.304 7f2d886d8700 1 -- 172.21.2.101:0/835294674 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d80189d00 msgr2=0x7f2d8018a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:44.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.304 7f2d886d8700 1 --2- 172.21.2.101:0/835294674 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d80189d00 0x7f2d8018a100 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f2d7000b580 tx=0x7f2d7000b920).stop 2021-02-10T11:19:44.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.304 7f2d886d8700 1 -- 172.21.2.101:0/835294674 shutdown_connections 2021-02-10T11:19:44.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.304 7f2d886d8700 1 --2- 172.21.2.101:0/835294674 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d8018a720 0x7f2d800b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:44.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.304 7f2d886d8700 1 --2- 172.21.2.101:0/835294674 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d80189d00 0x7f2d8018a100 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:44.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.304 7f2d886d8700 1 --2- 172.21.2.101:0/835294674 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d680186c0 0x7f2d6801abc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:44.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.304 7f2d886d8700 1 --2- 172.21.2.101:0/835294674 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d800a0420 0x7f2d800a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:44.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.304 7f2d886d8700 1 -- 172.21.2.101:0/835294674 >> 172.21.2.101:0/835294674 conn(0x7f2d80094e10 msgr2=0x7f2d800a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:44.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.304 7f2d886d8700 1 -- 172.21.2.101:0/835294674 shutdown_connections 2021-02-10T11:19:44.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.304 7f2d886d8700 1 -- 172.21.2.101:0/835294674 wait complete. 2021-02-10T11:19:44.318 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839268 2021-02-10T11:19:44.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.726 7fcfd2c60700 1 Processor -- start 2021-02-10T11:19:44.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.726 7fcfd2c60700 1 -- start start 2021-02-10T11:19:44.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.727 7fcfd2c60700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcfcc1b0870 0x7fcfcc0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:44.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.727 7fcfd2c60700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcfcc0efe00 0x7fcfcc0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.727 7fcfd2c60700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfcc1b0c70 con 0x7fcfcc1b0470 2021-02-10T11:19:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.727 7fcfd2c60700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfcc0f0b50 con 0x7fcfcc1b0870 2021-02-10T11:19:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.727 7fcfd2c60700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0f0820 con 0x7fcfcc0efe00 2021-02-10T11:19:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.727 7fcfcbfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcfcc1b0870 0x7fcfcc0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.727 7fcfcbfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcfcc1b0870 0x7fcfcc0f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46520/0 (socket says 172.21.2.101:46520) 2021-02-10T11:19:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.727 7fcfcbfff700 1 -- 172.21.2.101:0/2654492856 learned_addr learned my addr 172.21.2.101:0/2654492856 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:19:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.727 7fcfd127d700 1 --2- 172.21.2.101:0/2654492856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcfcc0efe00 0x7fcfcc0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.727 7fcfcbfff700 1 -- 172.21.2.101:0/2654492856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcfcc0efe00 msgr2=0x7fcfcc0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:19:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.727 7fcfcbfff700 1 --2- 172.21.2.101:0/2654492856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcfcc0efe00 0x7fcfcc0f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.727 7fcfcbfff700 1 -- 172.21.2.101:0/2654492856 >> v1:172.21.2.108:6789/0 conn(0x7fcfcc1b0470 legacy=0x7fcfcc0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.727 7fcfcbfff700 1 -- 172.21.2.101:0/2654492856 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc0e52e0 con 0x7fcfcc1b0870 2021-02-10T11:19:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.728 7fcfcbfff700 1 --2- 172.21.2.101:0/2654492856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcfcc1b0870 0x7fcfcc0f20e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fcfbc006a80 tx=0x7fcfbc009e80).ready entity=mon.0 client_cookie=9c9b079119b210be server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.728 7fcfcb7fe700 1 -- 172.21.2.101:0/2654492856 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcfbc00b510 con 0x7fcfcc1b0870 2021-02-10T11:19:44.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.728 7fcfcb7fe700 1 -- 172.21.2.101:0/2654492856 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfbc00b7d0 con 0x7fcfcc1b0870 2021-02-10T11:19:44.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.728 7fcfcb7fe700 1 -- 172.21.2.101:0/2654492856 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcfbc00bc20 con 0x7fcfcc1b0870 2021-02-10T11:19:44.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.728 7fcfd2c60700 1 -- 172.21.2.101:0/2654492856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcfcc1b0870 msgr2=0x7fcfcc0f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:44.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.728 7fcfd2c60700 1 --2- 172.21.2.101:0/2654492856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcfcc1b0870 0x7fcfcc0f20e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fcfbc006a80 tx=0x7fcfbc009e80).stop 2021-02-10T11:19:44.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.728 7fcfd2c60700 1 -- 172.21.2.101:0/2654492856 shutdown_connections 2021-02-10T11:19:44.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.728 7fcfd2c60700 1 --2- 172.21.2.101:0/2654492856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcfcc0efe00 0x7fcfcc0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:44.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.728 7fcfd2c60700 1 --2- 172.21.2.101:0/2654492856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcfcc1b0870 0x7fcfcc0f20e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:44.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.729 7fcfd2c60700 1 -- 172.21.2.101:0/2654492856 >> 172.21.2.101:0/2654492856 conn(0x7fcfcc0ce3e0 msgr2=0x7fcfcc0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:44.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.729 7fcfd2c60700 1 -- 172.21.2.101:0/2654492856 shutdown_connections 2021-02-10T11:19:44.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.729 7fcfd2c60700 1 -- 172.21.2.101:0/2654492856 wait complete. 2021-02-10T11:19:44.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.743 7fcfd2c60700 1 Processor -- start 2021-02-10T11:19:44.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.743 7fcfd2c60700 1 -- start start 2021-02-10T11:19:44.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.744 7fcfd2c60700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcfcc0a0420 0x7fcfcc0a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:44.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.744 7fcfd2c60700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcfcc189d00 0x7fcfcc18a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:44.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.744 7fcfd2c60700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcfcc18a720 0x7fcfcc0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:44.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.744 7fcfd2c60700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc189800 con 0x7fcfcc18a720 2021-02-10T11:19:44.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.744 7fcfd2c60700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc18ab20 con 0x7fcfcc0a0420 2021-02-10T11:19:44.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.744 7fcfd2c60700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0a1140 con 0x7fcfcc189d00 2021-02-10T11:19:44.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.744 7fcfd127d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcfcc18a720 0x7fcfcc0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:44.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.744 7fcfcbfff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcfcc189d00 0x7fcfcc18a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:44.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.744 7fcfd127d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcfcc18a720 0x7fcfcc0b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46524/0 (socket says 172.21.2.101:46524) 2021-02-10T11:19:44.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.744 7fcfd127d700 1 -- 172.21.2.101:0/929982150 learned_addr learned my addr 172.21.2.101:0/929982150 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:19:44.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.744 7fcfd0a7c700 1 --2- 172.21.2.101:0/929982150 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcfcc0a0420 0x7fcfcc0a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:44.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.744 7fcfd127d700 1 -- 172.21.2.101:0/929982150 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcfcc189d00 msgr2=0x7fcfcc18a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:44.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.744 7fcfd127d700 1 --2- 172.21.2.101:0/929982150 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcfcc189d00 0x7fcfcc18a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:44.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.744 7fcfd127d700 1 -- 172.21.2.101:0/929982150 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcfcc0a0420 msgr2=0x7fcfcc0a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:44.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.744 7fcfd127d700 1 --2- 172.21.2.101:0/929982150 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcfcc0a0420 0x7fcfcc0a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:44.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.744 7fcfd127d700 1 -- 172.21.2.101:0/929982150 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc0b8200 con 0x7fcfcc18a720 2021-02-10T11:19:44.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.744 7fcfcbfff700 1 --2- 172.21.2.101:0/929982150 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcfcc189d00 0x7fcfcc18a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:19:44.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.744 7fcfd127d700 1 --2- 172.21.2.101:0/929982150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcfcc18a720 0x7fcfcc0b7be0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fcfc400b5e0 tx=0x7fcfc400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:44.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.745 7fcfc97fa700 1 -- 172.21.2.101:0/929982150 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcfc400bd70 con 0x7fcfcc18a720 2021-02-10T11:19:44.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.745 7fcfc97fa700 1 -- 172.21.2.101:0/929982150 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc4006720 con 0x7fcfcc18a720 2021-02-10T11:19:44.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.745 7fcfd2c60700 1 -- 172.21.2.101:0/929982150 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfcc0b8200 con 0x7fcfcc18a720 2021-02-10T11:19:44.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.745 7fcfd2c60700 1 -- 172.21.2.101:0/929982150 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfcc0b8200 con 0x7fcfcc18a720 2021-02-10T11:19:44.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.745 7fcfc97fa700 1 -- 172.21.2.101:0/929982150 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcfc40022b0 con 0x7fcfcc18a720 2021-02-10T11:19:44.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.745 7fcfc97fa700 1 -- 172.21.2.101:0/929982150 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fcfc4005330 con 0x7fcfcc18a720 2021-02-10T11:19:44.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.745 7fcfc97fa700 1 --2- 172.21.2.101:0/929982150 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcfb00186c0 0x7fcfb001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:44.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.745 7fcfc97fa700 1 -- 172.21.2.101:0/929982150 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fcfc4032800 con 0x7fcfcc18a720 2021-02-10T11:19:44.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.746 7fcfd0a7c700 1 --2- 172.21.2.101:0/929982150 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcfb00186c0 0x7fcfb001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:44.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.746 7fcfd0a7c700 1 --2- 172.21.2.101:0/929982150 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcfb00186c0 0x7fcfb001abc0 crc :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:44.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.776 7fcfd2c60700 1 -- 172.21.2.101:0/929982150 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfcc18ab20 con 0x7fcfcc18a720 2021-02-10T11:19:44.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.778 7fcfc97fa700 1 -- 172.21.2.101:0/929982150 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fcfc4006720 con 0x7fcfcc18a720 2021-02-10T11:19:44.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.907 7fcfd2c60700 1 -- 172.21.2.101:0/929982150 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839268", "target": ["mgr", ""]}) v1 -- 0x7fcfcc18ab20 con 0x7fcfb00186c0 2021-02-10T11:19:44.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.910 7fcfc97fa700 1 -- 172.21.2.101:0/929982150 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fcfb4006550 con 0x7fcfb00186c0 2021-02-10T11:19:44.916 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839268/58db331d-186d-4db5-9261-1ff0b00f3d1f 2021-02-10T11:19:44.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.911 7fcfd2c60700 1 -- 172.21.2.101:0/929982150 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcfb00186c0 msgr2=0x7fcfb001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:44.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.911 7fcfd2c60700 1 --2- 172.21.2.101:0/929982150 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcfb00186c0 0x7fcfb001abc0 crc :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:44.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.911 7fcfd2c60700 1 -- 172.21.2.101:0/929982150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcfcc18a720 msgr2=0x7fcfcc0b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:44.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.911 7fcfd2c60700 1 --2- 172.21.2.101:0/929982150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcfcc18a720 0x7fcfcc0b7be0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fcfc400b5e0 tx=0x7fcfc400b980).stop 2021-02-10T11:19:44.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.912 7fcfd2c60700 1 -- 172.21.2.101:0/929982150 shutdown_connections 2021-02-10T11:19:44.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.912 7fcfd2c60700 1 --2- 172.21.2.101:0/929982150 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcfb00186c0 0x7fcfb001abc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:44.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.912 7fcfd2c60700 1 --2- 172.21.2.101:0/929982150 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcfcc189d00 0x7fcfcc18a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:44.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.912 7fcfd2c60700 1 --2- 172.21.2.101:0/929982150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcfcc18a720 0x7fcfcc0b7be0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:44.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.912 7fcfd2c60700 1 --2- 172.21.2.101:0/929982150 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcfcc0a0420 0x7fcfcc0a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:44.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.912 7fcfd2c60700 1 -- 172.21.2.101:0/929982150 >> 172.21.2.101:0/929982150 conn(0x7fcfcc094e10 msgr2=0x7fcfcc0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:44.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.912 7fcfd2c60700 1 -- 172.21.2.101:0/929982150 shutdown_connections 2021-02-10T11:19:44.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:44.912 7fcfd2c60700 1 -- 172.21.2.101:0/929982150 wait complete. 2021-02-10T11:19:44.926 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839268 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839268/58db331d-186d-4db5-9261-1ff0b00f3d1f 2021-02-10T11:19:44.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:44.927 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:44.927 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:45.263 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:45.263 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:45.263 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839268/58db331d-186d-4db5-9261-1ff0b00f3d1f/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:45.645 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:45.645 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:45.645 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0255163 s, 41.1 MB/s 2021-02-10T11:19:45.646 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:45.647 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:45.647 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:45.981 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:45.981 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:45.982 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839268/58db331d-186d-4db5-9261-1ff0b00f3d1f/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:46.356 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:46.356 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:46.356 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235369 s, 44.6 MB/s 2021-02-10T11:19:46.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:46.357 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:46.357 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:46.692 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:46.692 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:46.693 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839268/58db331d-186d-4db5-9261-1ff0b00f3d1f/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:47.068 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:47.068 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:47.068 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238498 s, 44.0 MB/s 2021-02-10T11:19:47.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:47.069 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:47.069 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:47.404 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:47.404 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:47.405 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839268/58db331d-186d-4db5-9261-1ff0b00f3d1f/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:47.780 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:47.780 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:47.780 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0255258 s, 41.1 MB/s 2021-02-10T11:19:47.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:47.781 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:47.781 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:47.880 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:19:47.883 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:19:48.116 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:48.116 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:48.117 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839268/58db331d-186d-4db5-9261-1ff0b00f3d1f/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:48.491 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:48.491 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:48.491 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238308 s, 44.0 MB/s 2021-02-10T11:19:48.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:48.492 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:48.492 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:48.827 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:48.827 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:48.827 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839268/58db331d-186d-4db5-9261-1ff0b00f3d1f/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:49.199 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:49.200 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:49.200 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253671 s, 41.3 MB/s 2021-02-10T11:19:49.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:49.201 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:49.201 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:49.535 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:49.536 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:49.536 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839268/58db331d-186d-4db5-9261-1ff0b00f3d1f/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:49.909 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:49.909 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:49.909 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233992 s, 44.8 MB/s 2021-02-10T11:19:49.910 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:49.911 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:49.911 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:50.247 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:50.247 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:50.248 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839268/58db331d-186d-4db5-9261-1ff0b00f3d1f/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:50.619 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:50.619 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:50.619 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0219639 s, 47.7 MB/s 2021-02-10T11:19:50.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:50.620 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:50.620 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:50.954 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:50.955 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:50.955 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839268/58db331d-186d-4db5-9261-1ff0b00f3d1f/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:51.328 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:51.329 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:51.329 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231894 s, 45.2 MB/s 2021-02-10T11:19:51.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:51.330 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:51.330 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:51.666 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:51.667 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:51.667 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839268/58db331d-186d-4db5-9261-1ff0b00f3d1f/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:52.042 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:52.042 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:52.042 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249206 s, 42.1 MB/s 2021-02-10T11:19:52.043 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839269 2021-02-10T11:19:52.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.450 7f1c44d91700 1 Processor -- start 2021-02-10T11:19:52.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.450 7f1c44d91700 1 -- start start 2021-02-10T11:19:52.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.451 7f1c44d91700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1c401b0470 0x7f1c401b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:52.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.451 7f1c44d91700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1c400f20e0 0x7f1c400f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:52.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.451 7f1c44d91700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c400f0b50 con 0x7f1c400ef610 2021-02-10T11:19:52.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.451 7f1c44d91700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c400f0820 con 0x7f1c400f20e0 2021-02-10T11:19:52.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.451 7f1c44d91700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c400f24e0 con 0x7f1c401b0470 2021-02-10T11:19:52.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.451 7f1c3f314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1c400f20e0 0x7f1c400f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:52.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.451 7f1c3f314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1c400f20e0 0x7f1c400f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46536/0 (socket says 172.21.2.101:46536) 2021-02-10T11:19:52.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.451 7f1c3e312700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f1c400ef610 0x7f1c400efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47026/0 (socket says 172.21.2.101:47026) 2021-02-10T11:19:52.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.451 7f1c3f314700 1 -- 172.21.2.101:0/405818752 learned_addr learned my addr 172.21.2.101:0/405818752 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:19:52.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.451 7f1c3eb13700 1 --2- 172.21.2.101:0/405818752 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1c401b0470 0x7f1c401b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:52.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.451 7f1c3f314700 1 -- 172.21.2.101:0/405818752 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1c401b0470 msgr2=0x7f1c401b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:19:52.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.451 7f1c3f314700 1 --2- 172.21.2.101:0/405818752 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1c401b0470 0x7f1c401b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:52.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.451 7f1c3f314700 1 -- 172.21.2.101:0/405818752 >> v1:172.21.2.101:6789/0 conn(0x7f1c400ef610 legacy=0x7f1c400efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:52.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.451 7f1c3f314700 1 -- 172.21.2.101:0/405818752 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c400e52e0 con 0x7f1c400f20e0 2021-02-10T11:19:52.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.451 7f1c3eb13700 1 --2- 172.21.2.101:0/405818752 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1c401b0470 0x7f1c401b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:19:52.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.452 7f1c3f314700 1 --2- 172.21.2.101:0/405818752 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1c400f20e0 0x7f1c400f0200 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f1c38006f00 tx=0x7f1c38009f10).ready entity=mon.0 client_cookie=b0139afcedfd05ef server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:52.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.452 7f1c3db11700 1 -- 172.21.2.101:0/405818752 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1c3800b3b0 con 0x7f1c400f20e0 2021-02-10T11:19:52.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.452 7f1c3db11700 1 -- 172.21.2.101:0/405818752 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3800b730 con 0x7f1c400f20e0 2021-02-10T11:19:52.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.452 7f1c3db11700 1 -- 172.21.2.101:0/405818752 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1c3800bc50 con 0x7f1c400f20e0 2021-02-10T11:19:52.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.452 7f1c44d91700 1 -- 172.21.2.101:0/405818752 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1c400f20e0 msgr2=0x7f1c400f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:52.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.452 7f1c44d91700 1 --2- 172.21.2.101:0/405818752 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1c400f20e0 0x7f1c400f0200 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f1c38006f00 tx=0x7f1c38009f10).stop 2021-02-10T11:19:52.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.453 7f1c44d91700 1 -- 172.21.2.101:0/405818752 shutdown_connections 2021-02-10T11:19:52.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.453 7f1c44d91700 1 --2- 172.21.2.101:0/405818752 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1c400f20e0 0x7f1c400f0200 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:52.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.453 7f1c44d91700 1 --2- 172.21.2.101:0/405818752 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1c401b0470 0x7f1c401b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:52.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.453 7f1c44d91700 1 -- 172.21.2.101:0/405818752 >> 172.21.2.101:0/405818752 conn(0x7f1c400ce3e0 msgr2=0x7f1c400e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:52.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.453 7f1c44d91700 1 -- 172.21.2.101:0/405818752 shutdown_connections 2021-02-10T11:19:52.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.453 7f1c44d91700 1 -- 172.21.2.101:0/405818752 wait complete. 2021-02-10T11:19:52.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.468 7f1c44d91700 1 Processor -- start 2021-02-10T11:19:52.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.468 7f1c44d91700 1 -- start start 2021-02-10T11:19:52.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.468 7f1c44d91700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1c400b7de0 0x7f1c40190920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:52.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.468 7f1c44d91700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1c400b82f0 0x7f1c400b86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:52.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.468 7f1c44d91700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1c400b8d10 0x7f1c400bf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:52.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.468 7f1c44d91700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c400b7930 con 0x7f1c400b7de0 2021-02-10T11:19:52.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.468 7f1c44d91700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c400b7b20 con 0x7f1c400b82f0 2021-02-10T11:19:52.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.468 7f1c44d91700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c400bffd0 con 0x7f1c400b8d10 2021-02-10T11:19:52.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.469 7f1c3eb13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1c400b7de0 0x7f1c40190920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:52.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.469 7f1c3eb13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1c400b7de0 0x7f1c40190920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46538/0 (socket says 172.21.2.101:46538) 2021-02-10T11:19:52.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.469 7f1c3eb13700 1 -- 172.21.2.101:0/3713553365 learned_addr learned my addr 172.21.2.101:0/3713553365 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:19:52.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.469 7f1c3f314700 1 --2- 172.21.2.101:0/3713553365 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1c400b8d10 0x7f1c400bf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:52.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.469 7f1c3e312700 1 --2- 172.21.2.101:0/3713553365 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1c400b82f0 0x7f1c400b86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:52.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.469 7f1c3eb13700 1 -- 172.21.2.101:0/3713553365 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1c400b8d10 msgr2=0x7f1c400bf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:52.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.469 7f1c3eb13700 1 --2- 172.21.2.101:0/3713553365 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1c400b8d10 0x7f1c400bf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:52.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.469 7f1c3eb13700 1 -- 172.21.2.101:0/3713553365 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1c400b82f0 msgr2=0x7f1c400b86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:52.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.469 7f1c3eb13700 1 --2- 172.21.2.101:0/3713553365 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1c400b82f0 0x7f1c400b86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:52.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.469 7f1c3eb13700 1 -- 172.21.2.101:0/3713553365 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c400c01c0 con 0x7f1c400b7de0 2021-02-10T11:19:52.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.469 7f1c3e312700 1 --2- 172.21.2.101:0/3713553365 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1c400b82f0 0x7f1c400b86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:19:52.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.469 7f1c3eb13700 1 --2- 172.21.2.101:0/3713553365 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1c400b7de0 0x7f1c40190920 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f1c30006f20 tx=0x7f1c3000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:52.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.470 7f1c377fe700 1 -- 172.21.2.101:0/3713553365 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1c30006bb0 con 0x7f1c400b7de0 2021-02-10T11:19:52.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.470 7f1c377fe700 1 -- 172.21.2.101:0/3713553365 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3000bd80 con 0x7f1c400b7de0 2021-02-10T11:19:52.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.470 7f1c44d91700 1 -- 172.21.2.101:0/3713553365 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c400c01c0 con 0x7f1c400b7de0 2021-02-10T11:19:52.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.470 7f1c44d91700 1 -- 172.21.2.101:0/3713553365 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c400c0750 con 0x7f1c400b7de0 2021-02-10T11:19:52.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.470 7f1c377fe700 1 -- 172.21.2.101:0/3713553365 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1c30002220 con 0x7f1c400b7de0 2021-02-10T11:19:52.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.470 7f1c377fe700 1 -- 172.21.2.101:0/3713553365 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1c3000a000 con 0x7f1c400b7de0 2021-02-10T11:19:52.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.470 7f1c377fe700 1 --2- 172.21.2.101:0/3713553365 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1c24018730 0x7f1c2401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:52.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.471 7f1c377fe700 1 -- 172.21.2.101:0/3713553365 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1c300327e0 con 0x7f1c400b7de0 2021-02-10T11:19:52.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.471 7f1c3e312700 1 --2- 172.21.2.101:0/3713553365 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1c24018730 0x7f1c2401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:52.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.471 7f1c3e312700 1 --2- 172.21.2.101:0/3713553365 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1c24018730 0x7f1c2401ac10 crc :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:52.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.500 7f1c44d91700 1 -- 172.21.2.101:0/3713553365 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c40105b30 con 0x7f1c400b7de0 2021-02-10T11:19:52.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.503 7f1c377fe700 1 -- 172.21.2.101:0/3713553365 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1c300009b0 con 0x7f1c400b7de0 2021-02-10T11:19:52.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.629 7f1c44d91700 1 -- 172.21.2.101:0/3713553365 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839269", "target": ["mgr", ""]}) v1 -- 0x7f1c400f0b50 con 0x7f1c24018730 2021-02-10T11:19:52.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.654 7f1c377fe700 1 -- 172.21.2.101:0/3713553365 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f1c280079b0 con 0x7f1c24018730 2021-02-10T11:19:52.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.662 7f1c44d91700 1 -- 172.21.2.101:0/3713553365 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1c24018730 msgr2=0x7f1c2401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:52.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.662 7f1c44d91700 1 --2- 172.21.2.101:0/3713553365 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1c24018730 0x7f1c2401ac10 crc :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:52.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.662 7f1c44d91700 1 -- 172.21.2.101:0/3713553365 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1c400b7de0 msgr2=0x7f1c40190920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:52.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.662 7f1c44d91700 1 --2- 172.21.2.101:0/3713553365 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1c400b7de0 0x7f1c40190920 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f1c30006f20 tx=0x7f1c3000b990).stop 2021-02-10T11:19:52.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.662 7f1c44d91700 1 -- 172.21.2.101:0/3713553365 shutdown_connections 2021-02-10T11:19:52.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.662 7f1c44d91700 1 --2- 172.21.2.101:0/3713553365 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1c24018730 0x7f1c2401ac10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:52.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.662 7f1c44d91700 1 --2- 172.21.2.101:0/3713553365 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1c400b8d10 0x7f1c400bf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:52.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.662 7f1c44d91700 1 --2- 172.21.2.101:0/3713553365 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1c400b82f0 0x7f1c400b86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:52.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.662 7f1c44d91700 1 --2- 172.21.2.101:0/3713553365 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1c400b7de0 0x7f1c40190920 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:52.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.662 7f1c44d91700 1 -- 172.21.2.101:0/3713553365 >> 172.21.2.101:0/3713553365 conn(0x7f1c40094e10 msgr2=0x7f1c400a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:52.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.662 7f1c44d91700 1 -- 172.21.2.101:0/3713553365 shutdown_connections 2021-02-10T11:19:52.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:52.662 7f1c44d91700 1 -- 172.21.2.101:0/3713553365 wait complete. 2021-02-10T11:19:52.677 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839269 2021-02-10T11:19:53.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.086 7ff41b7db700 1 Processor -- start 2021-02-10T11:19:53.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.086 7ff41b7db700 1 -- start start 2021-02-10T11:19:53.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.086 7ff41b7db700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4140f05f0 0x7ff4140f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:53.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.086 7ff41b7db700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4141b0c70 con 0x7ff4141b0470 2021-02-10T11:19:53.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.086 7ff41b7db700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4140f0b50 con 0x7ff4141b0870 2021-02-10T11:19:53.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.086 7ff41b7db700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4140e5ea0 con 0x7ff4140f05f0 2021-02-10T11:19:53.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.087 7ff4195f7700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff4141b0470 0x7ff4140ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47040/0 (socket says 172.21.2.101:47040) 2021-02-10T11:19:53.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.087 7ff4195f7700 1 -- 172.21.2.101:0/2662496583 learned_addr learned my addr 172.21.2.101:0/2662496583 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:19:53.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.087 7ff407fff700 1 -- 172.21.2.101:0/2662496583 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1965705084 0 0) 0x7ff408001180 con 0x7ff4141b0470 2021-02-10T11:19:53.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.087 7ff407fff700 1 -- 172.21.2.101:0/2662496583 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff400002fe0 con 0x7ff4141b0470 2021-02-10T11:19:53.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.087 7ff419df8700 1 --2- 172.21.2.101:0/2662496583 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4140f05f0 0x7ff4140f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:53.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.087 7ff407fff700 1 -- 172.21.2.101:0/2662496583 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 26599572 0 0) 0x7ff408001180 con 0x7ff4141b0470 2021-02-10T11:19:53.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.087 7ff407fff700 1 -- 172.21.2.101:0/2662496583 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4140f05f0 msgr2=0x7ff4140f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:19:53.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.087 7ff407fff700 1 --2- 172.21.2.101:0/2662496583 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4140f05f0 0x7ff4140f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.087 7ff407fff700 1 -- 172.21.2.101:0/2662496583 >> v1:172.21.2.108:6789/0 conn(0x7ff4141b0870 legacy=0x7ff4140efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.088 7ff407fff700 1 -- 172.21.2.101:0/2662496583 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4140e52e0 con 0x7ff4141b0470 2021-02-10T11:19:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.088 7ff407fff700 1 -- 172.21.2.101:0/2662496583 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7ff408001680 con 0x7ff4141b0470 2021-02-10T11:19:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.088 7ff407fff700 1 -- 172.21.2.101:0/2662496583 >> v1:172.21.2.101:6789/0 conn(0x7ff4141b0470 legacy=0x7ff4140ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.088 7ff407fff700 1 --2- 172.21.2.101:0/2662496583 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff400003a60 0x7ff400005f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.088 7ff407fff700 1 --2- 172.21.2.101:0/2662496583 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff400006750 0x7ff400008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.088 7ff407fff700 1 --2- 172.21.2.101:0/2662496583 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff400009510 0x7ff40000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.088 7ff407fff700 1 -- 172.21.2.101:0/2662496583 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40000c2c0 con 0x7ff400009510 2021-02-10T11:19:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.088 7ff407fff700 1 -- 172.21.2.101:0/2662496583 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40000c4e0 con 0x7ff400006750 2021-02-10T11:19:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.088 7ff407fff700 1 -- 172.21.2.101:0/2662496583 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff40000c700 con 0x7ff400003a60 2021-02-10T11:19:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.088 7ff4195f7700 1 --2- 172.21.2.101:0/2662496583 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff400003a60 0x7ff400005f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.088 7ff418df6700 1 --2- 172.21.2.101:0/2662496583 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff400006750 0x7ff400008c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.088 7ff419df8700 1 --2- 172.21.2.101:0/2662496583 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff400009510 0x7ff40000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.088 7ff418df6700 1 -- 172.21.2.101:0/2662496583 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff400003a60 msgr2=0x7ff400005f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:19:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.088 7ff418df6700 1 --2- 172.21.2.101:0/2662496583 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff400003a60 0x7ff400005f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.088 7ff418df6700 1 -- 172.21.2.101:0/2662496583 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff400009510 msgr2=0x7ff40000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:19:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.088 7ff418df6700 1 --2- 172.21.2.101:0/2662496583 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff400009510 0x7ff40000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.088 7ff418df6700 1 -- 172.21.2.101:0/2662496583 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff400050940 con 0x7ff400006750 2021-02-10T11:19:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.088 7ff4195f7700 1 --2- 172.21.2.101:0/2662496583 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff400003a60 0x7ff400005f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:19:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.089 7ff418df6700 1 --2- 172.21.2.101:0/2662496583 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff400006750 0x7ff400008c70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7ff410006af0 tx=0x7ff410009b40).ready entity=mon.1 client_cookie=8be3858e5d1ee1ba server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.089 7ff407fff700 1 -- 172.21.2.101:0/2662496583 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff410007500 con 0x7ff400006750 2021-02-10T11:19:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.089 7ff407fff700 1 -- 172.21.2.101:0/2662496583 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff410006710 con 0x7ff400006750 2021-02-10T11:19:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.089 7ff407fff700 1 -- 172.21.2.101:0/2662496583 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff410007a30 con 0x7ff400006750 2021-02-10T11:19:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.089 7ff41b7db700 1 -- 172.21.2.101:0/2662496583 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff400006750 msgr2=0x7ff400008c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.089 7ff41b7db700 1 --2- 172.21.2.101:0/2662496583 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff400006750 0x7ff400008c70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7ff410006af0 tx=0x7ff410009b40).stop 2021-02-10T11:19:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.089 7ff419df8700 1 -- 172.21.2.101:0/2662496583 reap_dead start 2021-02-10T11:19:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.089 7ff41b7db700 1 -- 172.21.2.101:0/2662496583 shutdown_connections 2021-02-10T11:19:53.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.089 7ff41b7db700 1 -- 172.21.2.101:0/2662496583 >> 172.21.2.101:0/2662496583 conn(0x7ff4140ce3e0 msgr2=0x7ff4140e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:53.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.089 7ff41b7db700 1 -- 172.21.2.101:0/2662496583 shutdown_connections 2021-02-10T11:19:53.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.089 7ff41b7db700 1 -- 172.21.2.101:0/2662496583 wait complete. 2021-02-10T11:19:53.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.104 7ff41b7db700 1 Processor -- start 2021-02-10T11:19:53.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.104 7ff41b7db700 1 -- start start 2021-02-10T11:19:53.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.104 7ff41b7db700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff41409b930 0x7ff4140b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:53.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.104 7ff41b7db700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4140b8410 0x7ff4140b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:53.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.104 7ff41b7db700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4140c1f80 0x7ff4140c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:53.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.104 7ff41b7db700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4140c1a00 con 0x7ff4140b8410 2021-02-10T11:19:53.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.104 7ff41b7db700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4140b8e30 con 0x7ff41409b930 2021-02-10T11:19:53.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.104 7ff41b7db700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4140b9050 con 0x7ff4140c1f80 2021-02-10T11:19:53.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.105 7ff418df6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4140b8410 0x7ff4140b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:53.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.105 7ff418df6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4140b8410 0x7ff4140b8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46560/0 (socket says 172.21.2.101:46560) 2021-02-10T11:19:53.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.105 7ff418df6700 1 -- 172.21.2.101:0/277522505 learned_addr learned my addr 172.21.2.101:0/277522505 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:19:53.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.105 7ff419df8700 1 --2- 172.21.2.101:0/277522505 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4140c1f80 0x7ff4140c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:53.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.105 7ff4195f7700 1 --2- 172.21.2.101:0/277522505 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff41409b930 0x7ff4140b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:53.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.105 7ff418df6700 1 -- 172.21.2.101:0/277522505 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4140c1f80 msgr2=0x7ff4140c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:53.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.105 7ff418df6700 1 --2- 172.21.2.101:0/277522505 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4140c1f80 0x7ff4140c2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:53.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.105 7ff418df6700 1 -- 172.21.2.101:0/277522505 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff41409b930 msgr2=0x7ff4140b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:53.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.105 7ff418df6700 1 --2- 172.21.2.101:0/277522505 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff41409b930 0x7ff4140b7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:53.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.105 7ff418df6700 1 -- 172.21.2.101:0/277522505 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4140c29a0 con 0x7ff4140b8410 2021-02-10T11:19:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.105 7ff418df6700 1 --2- 172.21.2.101:0/277522505 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4140b8410 0x7ff4140b8810 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7ff410006860 tx=0x7ff41000b9e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.106 7ff405ffb700 1 -- 172.21.2.101:0/277522505 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff410006c50 con 0x7ff4140b8410 2021-02-10T11:19:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.106 7ff405ffb700 1 -- 172.21.2.101:0/277522505 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff410006030 con 0x7ff4140b8410 2021-02-10T11:19:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.106 7ff405ffb700 1 -- 172.21.2.101:0/277522505 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff410006490 con 0x7ff4140b8410 2021-02-10T11:19:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.106 7ff41b7db700 1 -- 172.21.2.101:0/277522505 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4140c29a0 con 0x7ff4140b8410 2021-02-10T11:19:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.106 7ff41b7db700 1 -- 172.21.2.101:0/277522505 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4140c2f30 con 0x7ff4140b8410 2021-02-10T11:19:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.106 7ff405ffb700 1 -- 172.21.2.101:0/277522505 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff410006c30 con 0x7ff4140b8410 2021-02-10T11:19:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.106 7ff405ffb700 1 --2- 172.21.2.101:0/277522505 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff3f40186c0 0x7ff3f401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:19:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.106 7ff405ffb700 1 -- 172.21.2.101:0/277522505 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff410032430 con 0x7ff4140b8410 2021-02-10T11:19:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.107 7ff4195f7700 1 --2- 172.21.2.101:0/277522505 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff3f40186c0 0x7ff3f401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:19:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.107 7ff4195f7700 1 --2- 172.21.2.101:0/277522505 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff3f40186c0 0x7ff3f401abc0 crc :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:19:53.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.136 7ff41b7db700 1 -- 172.21.2.101:0/277522505 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff414105b30 con 0x7ff4140b8410 2021-02-10T11:19:53.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.139 7ff405ffb700 1 -- 172.21.2.101:0/277522505 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff410030d60 con 0x7ff4140b8410 2021-02-10T11:19:53.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.268 7ff41b7db700 1 -- 172.21.2.101:0/277522505 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839269", "target": ["mgr", ""]}) v1 -- 0x7ff4140c2f30 con 0x7ff3f40186c0 2021-02-10T11:19:53.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.272 7ff405ffb700 1 -- 172.21.2.101:0/277522505 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7ff408000c90 con 0x7ff3f40186c0 2021-02-10T11:19:53.282 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839269/46860891-ef51-4825-998a-08c045f6d213 2021-02-10T11:19:53.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.277 7ff41b7db700 1 -- 172.21.2.101:0/277522505 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff3f40186c0 msgr2=0x7ff3f401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:53.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.277 7ff41b7db700 1 --2- 172.21.2.101:0/277522505 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff3f40186c0 0x7ff3f401abc0 crc :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:53.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.277 7ff41b7db700 1 -- 172.21.2.101:0/277522505 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4140b8410 msgr2=0x7ff4140b8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:19:53.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.277 7ff41b7db700 1 --2- 172.21.2.101:0/277522505 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4140b8410 0x7ff4140b8810 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7ff410006860 tx=0x7ff41000b9e0).stop 2021-02-10T11:19:53.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.277 7ff41b7db700 1 -- 172.21.2.101:0/277522505 shutdown_connections 2021-02-10T11:19:53.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.277 7ff41b7db700 1 --2- 172.21.2.101:0/277522505 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff3f40186c0 0x7ff3f401abc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:53.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.277 7ff41b7db700 1 --2- 172.21.2.101:0/277522505 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4140c1f80 0x7ff4140c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:53.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.277 7ff41b7db700 1 --2- 172.21.2.101:0/277522505 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4140b8410 0x7ff4140b8810 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:53.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.277 7ff41b7db700 1 --2- 172.21.2.101:0/277522505 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff41409b930 0x7ff4140b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:19:53.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.277 7ff41b7db700 1 -- 172.21.2.101:0/277522505 >> 172.21.2.101:0/277522505 conn(0x7ff414094e10 msgr2=0x7ff4140a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:19:53.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.277 7ff41b7db700 1 -- 172.21.2.101:0/277522505 shutdown_connections 2021-02-10T11:19:53.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:19:53.278 7ff41b7db700 1 -- 172.21.2.101:0/277522505 wait complete. 2021-02-10T11:19:53.292 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839269 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839269/46860891-ef51-4825-998a-08c045f6d213 2021-02-10T11:19:53.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:53.293 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:53.293 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:53.629 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:53.629 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:53.630 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839269/46860891-ef51-4825-998a-08c045f6d213/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:54.006 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:54.006 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:54.006 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235311 s, 44.6 MB/s 2021-02-10T11:19:54.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:54.007 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:54.007 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:54.341 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:54.342 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:54.342 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839269/46860891-ef51-4825-998a-08c045f6d213/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:54.716 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:54.717 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:54.717 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234672 s, 44.7 MB/s 2021-02-10T11:19:54.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:54.718 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:54.718 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:55.052 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:55.052 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:55.053 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839269/46860891-ef51-4825-998a-08c045f6d213/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:55.426 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:55.426 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:55.426 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0228997 s, 45.8 MB/s 2021-02-10T11:19:55.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:55.427 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:55.427 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:55.762 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:55.762 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:55.763 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839269/46860891-ef51-4825-998a-08c045f6d213/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:56.136 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:56.136 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:56.137 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0228363 s, 45.9 MB/s 2021-02-10T11:19:56.137 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:56.138 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:56.138 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:56.473 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:56.473 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:56.473 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839269/46860891-ef51-4825-998a-08c045f6d213/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:56.848 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:56.848 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:56.848 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239807 s, 43.7 MB/s 2021-02-10T11:19:56.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:56.849 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:56.849 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:57.184 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:57.184 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:57.185 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839269/46860891-ef51-4825-998a-08c045f6d213/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:57.558 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:57.558 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:57.558 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239106 s, 43.9 MB/s 2021-02-10T11:19:57.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:57.559 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:57.559 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:57.895 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:57.895 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:57.896 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839269/46860891-ef51-4825-998a-08c045f6d213/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:58.269 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:58.269 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:58.269 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0230612 s, 45.5 MB/s 2021-02-10T11:19:58.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:58.270 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:58.271 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:58.607 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:58.607 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:58.607 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839269/46860891-ef51-4825-998a-08c045f6d213/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:58.980 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:58.980 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:58.980 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253148 s, 41.4 MB/s 2021-02-10T11:19:58.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:58.981 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:58.982 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:19:59.316 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:19:59.316 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:19:59.317 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839269/46860891-ef51-4825-998a-08c045f6d213/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:19:59.691 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:19:59.692 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:19:59.692 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248013 s, 42.3 MB/s 2021-02-10T11:19:59.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:19:59.693 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:19:59.693 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:00.029 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:00.029 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:00.030 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839269/46860891-ef51-4825-998a-08c045f6d213/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:00.405 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:00.405 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:00.405 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233101 s, 45.0 MB/s 2021-02-10T11:20:00.406 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839270 2021-02-10T11:20:00.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.812 7f8da2169700 1 Processor -- start 2021-02-10T11:20:00.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.812 7f8da2169700 1 -- start start 2021-02-10T11:20:00.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.812 7f8da2169700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8d9c1b0470 0x7f8d9c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:00.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.812 7f8da2169700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8d9c0ef610 0x7f8d9c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:00.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.812 7f8da2169700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d9c0f0030 0x7f8d9c0f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:00.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.812 7f8da2169700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d9c0e5ea0 con 0x7f8d9c0ef610 2021-02-10T11:20:00.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.812 7f8da2169700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d9c0f0b50 con 0x7f8d9c1b0470 2021-02-10T11:20:00.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.812 7f8da2169700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c0e52e0 con 0x7f8d9c0f0030 2021-02-10T11:20:00.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.813 7f8d9affd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8d9c0ef610 0x7f8d9c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:00.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.813 7f8d9affd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8d9c0ef610 0x7f8d9c0efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46566/0 (socket says 172.21.2.101:46566) 2021-02-10T11:20:00.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.813 7f8d9affd700 1 -- 172.21.2.101:0/174510455 learned_addr learned my addr 172.21.2.101:0/174510455 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:20:00.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.813 7f8d9affd700 1 -- 172.21.2.101:0/174510455 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d9c0f0030 msgr2=0x7f8d9c0f0430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:20:00.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.813 7f8d9affd700 1 --2- 172.21.2.101:0/174510455 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d9c0f0030 0x7f8d9c0f0430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:00.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.813 7f8d9affd700 1 -- 172.21.2.101:0/174510455 >> v2:172.21.2.108:3300/0 conn(0x7f8d9c1b0470 msgr2=0x7f8d9c1b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:20:00.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.813 7f8d9affd700 1 --2- 172.21.2.101:0/174510455 >> v2:172.21.2.108:3300/0 conn(0x7f8d9c1b0470 0x7f8d9c1b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:00.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.813 7f8d9affd700 1 -- 172.21.2.101:0/174510455 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c0f10d0 con 0x7f8d9c0ef610 2021-02-10T11:20:00.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.813 7f8d9affd700 1 --2- 172.21.2.101:0/174510455 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d9c0ef610 0x7f8d9c0efa10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f8d8c006700 tx=0x7f8d8c009b80).ready entity=mon.0 client_cookie=5cd5d1b0c01d95fd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:00.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.813 7f8d9a7fc700 1 -- 172.21.2.101:0/174510455 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d8c00c0c0 con 0x7f8d9c0ef610 2021-02-10T11:20:00.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.813 7f8d9a7fc700 1 -- 172.21.2.101:0/174510455 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d8c00c440 con 0x7f8d9c0ef610 2021-02-10T11:20:00.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.813 7f8d9a7fc700 1 -- 172.21.2.101:0/174510455 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d8c00c960 con 0x7f8d9c0ef610 2021-02-10T11:20:00.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.813 7f8da2169700 1 -- 172.21.2.101:0/174510455 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d9c0ef610 msgr2=0x7f8d9c0efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:00.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.813 7f8da2169700 1 --2- 172.21.2.101:0/174510455 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d9c0ef610 0x7f8d9c0efa10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f8d8c006700 tx=0x7f8d8c009b80).stop 2021-02-10T11:20:00.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.814 7f8da2169700 1 -- 172.21.2.101:0/174510455 shutdown_connections 2021-02-10T11:20:00.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.814 7f8da2169700 1 --2- 172.21.2.101:0/174510455 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d9c0f0030 0x7f8d9c0f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:00.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.814 7f8da2169700 1 --2- 172.21.2.101:0/174510455 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d9c0ef610 0x7f8d9c0efa10 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:00.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.814 7f8da2169700 1 --2- 172.21.2.101:0/174510455 >> v2:172.21.2.108:3300/0 conn(0x7f8d9c1b0470 0x7f8d9c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:00.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.814 7f8da2169700 1 -- 172.21.2.101:0/174510455 >> 172.21.2.101:0/174510455 conn(0x7f8d9c0ce3e0 msgr2=0x7f8d9c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:00.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.814 7f8da2169700 1 -- 172.21.2.101:0/174510455 shutdown_connections 2021-02-10T11:20:00.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.814 7f8da2169700 1 -- 172.21.2.101:0/174510455 wait complete. 2021-02-10T11:20:00.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.828 7f8da2169700 1 Processor -- start 2021-02-10T11:20:00.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.829 7f8da2169700 1 -- start start 2021-02-10T11:20:00.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.829 7f8da2169700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d9c190920 0x7f8d9c08f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:00.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.829 7f8da2169700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d9c190d20 0x7f8d9c08fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:00.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.829 7f8da2169700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d9c193270 0x7f8d9c193670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:00.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.829 7f8da2169700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c090470 con 0x7f8d9c190920 2021-02-10T11:20:00.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.829 7f8da2169700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c193c90 con 0x7f8d9c193270 2021-02-10T11:20:00.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.829 7f8da2169700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c193e70 con 0x7f8d9c190d20 2021-02-10T11:20:00.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.829 7f8d9b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d9c190920 0x7f8d9c08f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:00.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.829 7f8d9b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d9c190920 0x7f8d9c08f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46572/0 (socket says 172.21.2.101:46572) 2021-02-10T11:20:00.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.829 7f8d9b7fe700 1 -- 172.21.2.101:0/1618789630 learned_addr learned my addr 172.21.2.101:0/1618789630 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:20:00.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.829 7f8d9b7fe700 1 -- 172.21.2.101:0/1618789630 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d9c190d20 msgr2=0x7f8d9c08fc20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:20:00.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.829 7f8d9b7fe700 1 --2- 172.21.2.101:0/1618789630 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d9c190d20 0x7f8d9c08fc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:00.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.829 7f8d9b7fe700 1 -- 172.21.2.101:0/1618789630 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d9c193270 msgr2=0x7f8d9c193670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:20:00.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.829 7f8d9b7fe700 1 --2- 172.21.2.101:0/1618789630 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d9c193270 0x7f8d9c193670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:00.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.829 7f8d9b7fe700 1 -- 172.21.2.101:0/1618789630 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c194090 con 0x7f8d9c190920 2021-02-10T11:20:00.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.830 7f8d9b7fe700 1 --2- 172.21.2.101:0/1618789630 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d9c190920 0x7f8d9c08f600 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f8d8400b5c0 tx=0x7f8d8400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:00.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.830 7f8d93fff700 1 -- 172.21.2.101:0/1618789630 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d8400bd50 con 0x7f8d9c190920 2021-02-10T11:20:00.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.830 7f8d93fff700 1 -- 172.21.2.101:0/1618789630 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d84006030 con 0x7f8d9c190920 2021-02-10T11:20:00.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.830 7f8da2169700 1 -- 172.21.2.101:0/1618789630 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9c194090 con 0x7f8d9c190920 2021-02-10T11:20:00.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.830 7f8d93fff700 1 -- 172.21.2.101:0/1618789630 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d840054d0 con 0x7f8d9c190920 2021-02-10T11:20:00.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.830 7f8da2169700 1 -- 172.21.2.101:0/1618789630 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9c1920b0 con 0x7f8d9c190920 2021-02-10T11:20:00.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.831 7f8d93fff700 1 -- 172.21.2.101:0/1618789630 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8d84006210 con 0x7f8d9c190920 2021-02-10T11:20:00.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.831 7f8d93fff700 1 --2- 172.21.2.101:0/1618789630 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d800186c0 0x7f8d8001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:00.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.831 7f8d93fff700 1 -- 172.21.2.101:0/1618789630 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8d84032420 con 0x7f8d9c190920 2021-02-10T11:20:00.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.831 7f8d9affd700 1 --2- 172.21.2.101:0/1618789630 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d800186c0 0x7f8d8001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:00.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.831 7f8d9affd700 1 --2- 172.21.2.101:0/1618789630 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d800186c0 0x7f8d8001abc0 crc :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:00.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.862 7f8da2169700 1 -- 172.21.2.101:0/1618789630 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d9c105af0 con 0x7f8d9c190920 2021-02-10T11:20:00.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.865 7f8d93fff700 1 -- 172.21.2.101:0/1618789630 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8d84000940 con 0x7f8d9c190920 2021-02-10T11:20:00.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:00.991 7f8da2169700 1 -- 172.21.2.101:0/1618789630 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839270", "target": ["mgr", ""]}) v1 -- 0x7f8d9c090470 con 0x7f8d800186c0 2021-02-10T11:20:01.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.014 7f8d93fff700 1 -- 172.21.2.101:0/1618789630 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f8d8c0095d0 con 0x7f8d800186c0 2021-02-10T11:20:01.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.024 7f8da2169700 1 -- 172.21.2.101:0/1618789630 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d800186c0 msgr2=0x7f8d8001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:01.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.024 7f8da2169700 1 --2- 172.21.2.101:0/1618789630 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d800186c0 0x7f8d8001abc0 crc :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:01.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.024 7f8da2169700 1 -- 172.21.2.101:0/1618789630 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d9c190920 msgr2=0x7f8d9c08f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:01.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.024 7f8da2169700 1 --2- 172.21.2.101:0/1618789630 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d9c190920 0x7f8d9c08f600 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f8d8400b5c0 tx=0x7f8d8400b960).stop 2021-02-10T11:20:01.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.024 7f8da2169700 1 -- 172.21.2.101:0/1618789630 shutdown_connections 2021-02-10T11:20:01.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.024 7f8da2169700 1 --2- 172.21.2.101:0/1618789630 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d800186c0 0x7f8d8001abc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:01.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.024 7f8da2169700 1 --2- 172.21.2.101:0/1618789630 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d9c190d20 0x7f8d9c08fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:01.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.024 7f8da2169700 1 --2- 172.21.2.101:0/1618789630 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d9c193270 0x7f8d9c193670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:01.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.024 7f8da2169700 1 --2- 172.21.2.101:0/1618789630 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d9c190920 0x7f8d9c08f600 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:01.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.024 7f8da2169700 1 -- 172.21.2.101:0/1618789630 >> 172.21.2.101:0/1618789630 conn(0x7f8d9c0bf4e0 msgr2=0x7f8d9c0be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:01.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.024 7f8da2169700 1 -- 172.21.2.101:0/1618789630 shutdown_connections 2021-02-10T11:20:01.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.024 7f8da2169700 1 -- 172.21.2.101:0/1618789630 wait complete. 2021-02-10T11:20:01.039 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839270 2021-02-10T11:20:01.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.446 7f057520c700 1 Processor -- start 2021-02-10T11:20:01.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.446 7f057520c700 1 -- start start 2021-02-10T11:20:01.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.447 7f057520c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f05701b0470 0x7f05701b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:01.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.447 7f057520c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05700f20e0 0x7f05700f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:01.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.447 7f057520c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05700f0b50 con 0x7f05700ef610 2021-02-10T11:20:01.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.447 7f057520c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05700f0820 con 0x7f05701b0470 2021-02-10T11:20:01.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.447 7f057520c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05700f24e0 con 0x7f05700f20e0 2021-02-10T11:20:01.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.447 7f056ed9d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f05701b0470 0x7f05701b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:01.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.447 7f056ed9d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f05701b0470 0x7f05701b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46580/0 (socket says 172.21.2.101:46580) 2021-02-10T11:20:01.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.447 7f056ed9d700 1 -- 172.21.2.101:0/1079906634 learned_addr learned my addr 172.21.2.101:0/1079906634 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:20:01.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.447 7f056ed9d700 1 -- 172.21.2.101:0/1079906634 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05700f20e0 msgr2=0x7f05700f0200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:20:01.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.447 7f056ed9d700 1 --2- 172.21.2.101:0/1079906634 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05700f20e0 0x7f05700f0200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:01.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.447 7f056ed9d700 1 -- 172.21.2.101:0/1079906634 >> v1:172.21.2.108:6789/0 conn(0x7f05700ef610 legacy=0x7f05700efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:20:01.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.447 7f056ed9d700 1 -- 172.21.2.101:0/1079906634 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05700e52e0 con 0x7f05701b0470 2021-02-10T11:20:01.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.447 7f056ed9d700 1 --2- 172.21.2.101:0/1079906634 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05701b0470 0x7f05701b0870 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f0558006700 tx=0x7f0558009b80).ready entity=mon.0 client_cookie=4c4415f962ee5b73 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:01.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.448 7f056dd9b700 1 -- 172.21.2.101:0/1079906634 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f055800c0c0 con 0x7f05701b0470 2021-02-10T11:20:01.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.448 7f056dd9b700 1 -- 172.21.2.101:0/1079906634 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055800c400 con 0x7f05701b0470 2021-02-10T11:20:01.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.448 7f056dd9b700 1 -- 172.21.2.101:0/1079906634 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f055800c920 con 0x7f05701b0470 2021-02-10T11:20:01.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.448 7f057520c700 1 -- 172.21.2.101:0/1079906634 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05701b0470 msgr2=0x7f05701b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:01.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.448 7f057520c700 1 --2- 172.21.2.101:0/1079906634 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05701b0470 0x7f05701b0870 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f0558006700 tx=0x7f0558009b80).stop 2021-02-10T11:20:01.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.448 7f057520c700 1 -- 172.21.2.101:0/1079906634 shutdown_connections 2021-02-10T11:20:01.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.448 7f057520c700 1 --2- 172.21.2.101:0/1079906634 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05700f20e0 0x7f05700f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:01.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.448 7f057520c700 1 --2- 172.21.2.101:0/1079906634 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05701b0470 0x7f05701b0870 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:01.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.448 7f057520c700 1 -- 172.21.2.101:0/1079906634 >> 172.21.2.101:0/1079906634 conn(0x7f05700ce3e0 msgr2=0x7f05700e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:01.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.448 7f057520c700 1 -- 172.21.2.101:0/1079906634 shutdown_connections 2021-02-10T11:20:01.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.448 7f057520c700 1 -- 172.21.2.101:0/1079906634 wait complete. 2021-02-10T11:20:01.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.463 7f057520c700 1 Processor -- start 2021-02-10T11:20:01.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.463 7f057520c700 1 -- start start 2021-02-10T11:20:01.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.463 7f057520c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f05700b2840 0x7f05701a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:01.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.463 7f057520c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05700b30d0 0x7f05700b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:01.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.463 7f057520c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05700b3af0 0x7f05700f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:01.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.463 7f057520c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05700b2c40 con 0x7f05700b3af0 2021-02-10T11:20:01.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.463 7f057520c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05701a4920 con 0x7f05700b2840 2021-02-10T11:20:01.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.463 7f057520c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05700f0ba0 con 0x7f05700b30d0 2021-02-10T11:20:01.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.463 7f056f59e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05700b3af0 0x7f05700f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:01.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.463 7f056f59e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05700b3af0 0x7f05700f0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46588/0 (socket says 172.21.2.101:46588) 2021-02-10T11:20:01.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.463 7f056f59e700 1 -- 172.21.2.101:0/791387467 learned_addr learned my addr 172.21.2.101:0/791387467 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:20:01.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.463 7f056f59e700 1 -- 172.21.2.101:0/791387467 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05700b30d0 msgr2=0x7f05700b34d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:20:01.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.463 7f056f59e700 1 --2- 172.21.2.101:0/791387467 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05700b30d0 0x7f05700b34d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:01.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.463 7f056f59e700 1 -- 172.21.2.101:0/791387467 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f05700b2840 msgr2=0x7f05701a4300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:20:01.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.463 7f056f59e700 1 --2- 172.21.2.101:0/791387467 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f05700b2840 0x7f05701a4300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:01.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.464 7f056f59e700 1 -- 172.21.2.101:0/791387467 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05700f0dc0 con 0x7f05700b3af0 2021-02-10T11:20:01.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.464 7f056f59e700 1 --2- 172.21.2.101:0/791387467 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05700b3af0 0x7f05700f0580 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f0568009960 tx=0x7f056800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:01.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.464 7f05677fe700 1 -- 172.21.2.101:0/791387467 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0568003030 con 0x7f05700b3af0 2021-02-10T11:20:01.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.464 7f05677fe700 1 -- 172.21.2.101:0/791387467 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0568001310 con 0x7f05700b3af0 2021-02-10T11:20:01.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.464 7f05677fe700 1 -- 172.21.2.101:0/791387467 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0568001770 con 0x7f05700b3af0 2021-02-10T11:20:01.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.464 7f057520c700 1 -- 172.21.2.101:0/791387467 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05700f0dc0 con 0x7f05700b3af0 2021-02-10T11:20:01.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.464 7f057520c700 1 -- 172.21.2.101:0/791387467 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05700f0dc0 con 0x7f05700b3af0 2021-02-10T11:20:01.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.465 7f05677fe700 1 -- 172.21.2.101:0/791387467 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0568000d10 con 0x7f05700b3af0 2021-02-10T11:20:01.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.465 7f05677fe700 1 --2- 172.21.2.101:0/791387467 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f05540186c0 0x7f055401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:01.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.465 7f05677fe700 1 -- 172.21.2.101:0/791387467 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0568032920 con 0x7f05700b3af0 2021-02-10T11:20:01.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.465 7f056ed9d700 1 --2- 172.21.2.101:0/791387467 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f05540186c0 0x7f055401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:01.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.465 7f056ed9d700 1 --2- 172.21.2.101:0/791387467 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f05540186c0 0x7f055401abc0 crc :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:01.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.497 7f057520c700 1 -- 172.21.2.101:0/791387467 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05700f0050 con 0x7f05700b3af0 2021-02-10T11:20:01.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.499 7f05677fe700 1 -- 172.21.2.101:0/791387467 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f056800be30 con 0x7f05700b3af0 2021-02-10T11:20:01.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.630 7f057520c700 1 -- 172.21.2.101:0/791387467 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839270", "target": ["mgr", ""]}) v1 -- 0x7f05700f0050 con 0x7f05540186c0 2021-02-10T11:20:01.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.633 7f05677fe700 1 -- 172.21.2.101:0/791387467 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f05580064f0 con 0x7f05540186c0 2021-02-10T11:20:01.640 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839270/016932ae-d267-47c7-8843-46da5f114e28 2021-02-10T11:20:01.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.634 7f057520c700 1 -- 172.21.2.101:0/791387467 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f05540186c0 msgr2=0x7f055401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:01.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.634 7f057520c700 1 --2- 172.21.2.101:0/791387467 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f05540186c0 0x7f055401abc0 crc :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:01.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.634 7f057520c700 1 -- 172.21.2.101:0/791387467 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05700b3af0 msgr2=0x7f05700f0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:01.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.634 7f057520c700 1 --2- 172.21.2.101:0/791387467 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05700b3af0 0x7f05700f0580 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f0568009960 tx=0x7f056800bcf0).stop 2021-02-10T11:20:01.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.635 7f057520c700 1 -- 172.21.2.101:0/791387467 shutdown_connections 2021-02-10T11:20:01.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.635 7f057520c700 1 --2- 172.21.2.101:0/791387467 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f05540186c0 0x7f055401abc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:01.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.635 7f057520c700 1 --2- 172.21.2.101:0/791387467 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05700b30d0 0x7f05700b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:01.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.635 7f057520c700 1 --2- 172.21.2.101:0/791387467 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05700b3af0 0x7f05700f0580 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:01.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.635 7f057520c700 1 --2- 172.21.2.101:0/791387467 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f05700b2840 0x7f05701a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:01.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.635 7f057520c700 1 -- 172.21.2.101:0/791387467 >> 172.21.2.101:0/791387467 conn(0x7f05700c36f0 msgr2=0x7f0570094c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:01.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.635 7f057520c700 1 -- 172.21.2.101:0/791387467 shutdown_connections 2021-02-10T11:20:01.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:01.635 7f057520c700 1 -- 172.21.2.101:0/791387467 wait complete. 2021-02-10T11:20:01.650 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839270 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839270/016932ae-d267-47c7-8843-46da5f114e28 2021-02-10T11:20:01.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:01.650 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:01.650 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:01.986 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:01.987 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:01.987 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839270/016932ae-d267-47c7-8843-46da5f114e28/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:02.354 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:02.354 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:02.355 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0230032 s, 45.6 MB/s 2021-02-10T11:20:02.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:02.356 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:02.356 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:02.692 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:02.693 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:02.693 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839270/016932ae-d267-47c7-8843-46da5f114e28/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:03.069 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:03.069 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:03.069 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246115 s, 42.6 MB/s 2021-02-10T11:20:03.070 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:03.071 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:03.071 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:03.407 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:03.407 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:03.408 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839270/016932ae-d267-47c7-8843-46da5f114e28/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:03.779 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:03.780 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:03.780 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243786 s, 43.0 MB/s 2021-02-10T11:20:03.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:03.781 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:03.781 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:04.115 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:04.116 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:04.116 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839270/016932ae-d267-47c7-8843-46da5f114e28/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:04.492 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:04.493 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:04.493 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249656 s, 42.0 MB/s 2021-02-10T11:20:04.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:04.494 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:04.494 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:04.829 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:04.829 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:04.829 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839270/016932ae-d267-47c7-8843-46da5f114e28/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:05.205 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:05.205 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:05.205 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242909 s, 43.2 MB/s 2021-02-10T11:20:05.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:05.206 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:05.206 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:05.540 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:05.540 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:05.541 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839270/016932ae-d267-47c7-8843-46da5f114e28/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:05.914 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:08.707 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:08.707 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231136 s, 45.4 MB/s 2021-02-10T11:20:08.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:08.709 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:08.709 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:09.045 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:09.046 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:09.046 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839270/016932ae-d267-47c7-8843-46da5f114e28/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:09.419 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:09.419 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:09.419 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240241 s, 43.6 MB/s 2021-02-10T11:20:09.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:09.420 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:09.420 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:09.755 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:09.756 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:09.756 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839270/016932ae-d267-47c7-8843-46da5f114e28/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:10.131 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:10.131 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:10.131 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239831 s, 43.7 MB/s 2021-02-10T11:20:10.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:10.132 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:10.132 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:10.467 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:10.468 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:10.468 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839270/016932ae-d267-47c7-8843-46da5f114e28/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:10.841 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:10.841 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:10.841 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0229562 s, 45.7 MB/s 2021-02-10T11:20:10.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:10.842 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:10.842 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:11.178 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:11.179 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:11.179 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839270/016932ae-d267-47c7-8843-46da5f114e28/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:11.554 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:11.554 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:11.554 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236215 s, 44.4 MB/s 2021-02-10T11:20:11.555 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839271 2021-02-10T11:20:11.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.962 7f83c948d700 1 Processor -- start 2021-02-10T11:20:11.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.962 7f83c948d700 1 -- start start 2021-02-10T11:20:11.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.963 7f83c948d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f83c41b0870 0x7f83c40f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:11.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.963 7f83c948d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f83c40efe00 0x7f83c40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:11.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.963 7f83c948d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c41b0c70 con 0x7f83c41b0470 2021-02-10T11:20:11.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.963 7f83c948d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c40e5ea0 con 0x7f83c40efe00 2021-02-10T11:20:11.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.963 7f83c948d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c40f0b50 con 0x7f83c41b0870 2021-02-10T11:20:11.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.963 7f83c387e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f83c40efe00 0x7f83c40f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:11.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.963 7f83c387e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f83c40efe00 0x7f83c40f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46594/0 (socket says 172.21.2.101:46594) 2021-02-10T11:20:11.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.963 7f83c387e700 1 -- 172.21.2.101:0/2734473170 learned_addr learned my addr 172.21.2.101:0/2734473170 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:20:11.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.963 7f83c387e700 1 -- 172.21.2.101:0/2734473170 >> v2:172.21.2.108:3300/0 conn(0x7f83c41b0870 msgr2=0x7f83c40f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:20:11.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.963 7f83c387e700 1 --2- 172.21.2.101:0/2734473170 >> v2:172.21.2.108:3300/0 conn(0x7f83c41b0870 0x7f83c40f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:11.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.964 7f83c387e700 1 -- 172.21.2.101:0/2734473170 >> v1:172.21.2.101:6789/0 conn(0x7f83c41b0470 legacy=0x7f83c40ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:11.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.964 7f83c207b700 1 -- 172.21.2.101:0/2734473170 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 531706068 0 0) 0x7f83ac000ce0 con 0x7f83c41b0470 2021-02-10T11:20:11.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.964 7f83c387e700 1 -- 172.21.2.101:0/2734473170 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c40f0820 con 0x7f83c40efe00 2021-02-10T11:20:11.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.964 7f83c387e700 1 --2- 172.21.2.101:0/2734473170 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83c40efe00 0x7f83c40f0200 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f83bc006f00 tx=0x7f83bc009e90).ready entity=mon.0 client_cookie=3fef65797c214680 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:11.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.964 7f83c207b700 1 -- 172.21.2.101:0/2734473170 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f83bc00b300 con 0x7f83c40efe00 2021-02-10T11:20:11.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.964 7f83c207b700 1 -- 172.21.2.101:0/2734473170 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83bc00b640 con 0x7f83c40efe00 2021-02-10T11:20:11.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.964 7f83c207b700 1 -- 172.21.2.101:0/2734473170 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f83bc00bb20 con 0x7f83c40efe00 2021-02-10T11:20:11.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.964 7f83c948d700 1 -- 172.21.2.101:0/2734473170 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83c40efe00 msgr2=0x7f83c40f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:11.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.964 7f83c948d700 1 --2- 172.21.2.101:0/2734473170 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83c40efe00 0x7f83c40f0200 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f83bc006f00 tx=0x7f83bc009e90).stop 2021-02-10T11:20:11.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.965 7f83c948d700 1 -- 172.21.2.101:0/2734473170 shutdown_connections 2021-02-10T11:20:11.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.965 7f83c948d700 1 --2- 172.21.2.101:0/2734473170 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83c40efe00 0x7f83c40f0200 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:11.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.965 7f83c948d700 1 --2- 172.21.2.101:0/2734473170 >> v2:172.21.2.108:3300/0 conn(0x7f83c41b0870 0x7f83c40f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:11.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.965 7f83c948d700 1 -- 172.21.2.101:0/2734473170 >> 172.21.2.101:0/2734473170 conn(0x7f83c40ce3e0 msgr2=0x7f83c40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:11.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.965 7f83c948d700 1 -- 172.21.2.101:0/2734473170 shutdown_connections 2021-02-10T11:20:11.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.965 7f83c948d700 1 -- 172.21.2.101:0/2734473170 wait complete. 2021-02-10T11:20:11.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.979 7f83c948d700 1 Processor -- start 2021-02-10T11:20:11.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.979 7f83c948d700 1 -- start start 2021-02-10T11:20:11.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.980 7f83c948d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83c41933f0 0x7f83c40a7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:11.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.980 7f83c948d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83c40a8230 0x7f83c40a8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:11.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.980 7f83c948d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83c40a9360 0x7f83c40a9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:11.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.980 7f83c948d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c40a87c0 con 0x7f83c40a9360 2021-02-10T11:20:11.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.980 7f83c948d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c41937f0 con 0x7f83c41933f0 2021-02-10T11:20:11.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.980 7f83c948d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c4194060 con 0x7f83c40a8230 2021-02-10T11:20:11.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.980 7f83c387e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83c40a9360 0x7f83c40a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:11.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.980 7f83c387e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83c40a9360 0x7f83c40a9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46604/0 (socket says 172.21.2.101:46604) 2021-02-10T11:20:11.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.980 7f83c387e700 1 -- 172.21.2.101:0/681705879 learned_addr learned my addr 172.21.2.101:0/681705879 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:20:11.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.980 7f83c387e700 1 -- 172.21.2.101:0/681705879 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83c40a8230 msgr2=0x7f83c40a8d40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:20:11.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.980 7f83c387e700 1 --2- 172.21.2.101:0/681705879 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83c40a8230 0x7f83c40a8d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:11.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.980 7f83c387e700 1 -- 172.21.2.101:0/681705879 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83c41933f0 msgr2=0x7f83c40a7c10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:20:11.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.980 7f83c387e700 1 --2- 172.21.2.101:0/681705879 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83c41933f0 0x7f83c40a7c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:11.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.981 7f83c387e700 1 -- 172.21.2.101:0/681705879 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c4193b50 con 0x7f83c40a9360 2021-02-10T11:20:11.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.981 7f83c387e700 1 --2- 172.21.2.101:0/681705879 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83c40a9360 0x7f83c40a9760 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f83bc00b890 tx=0x7f83bc00bc30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:11.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.981 7f83bbfff700 1 -- 172.21.2.101:0/681705879 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f83bc00bd70 con 0x7f83c40a9360 2021-02-10T11:20:11.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.981 7f83bbfff700 1 -- 172.21.2.101:0/681705879 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83bc005820 con 0x7f83c40a9360 2021-02-10T11:20:11.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.981 7f83bbfff700 1 -- 172.21.2.101:0/681705879 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f83bc005cb0 con 0x7f83c40a9360 2021-02-10T11:20:11.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.981 7f83c948d700 1 -- 172.21.2.101:0/681705879 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c4193b50 con 0x7f83c40a9360 2021-02-10T11:20:11.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.981 7f83c948d700 1 -- 172.21.2.101:0/681705879 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c4193b50 con 0x7f83c40a9360 2021-02-10T11:20:11.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.982 7f83bbfff700 1 -- 172.21.2.101:0/681705879 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f83bc00bd70 con 0x7f83c40a9360 2021-02-10T11:20:11.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.982 7f83bbfff700 1 --2- 172.21.2.101:0/681705879 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83a80186c0 0x7f83a801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:11.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.982 7f83bbfff700 1 -- 172.21.2.101:0/681705879 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f83bc032b70 con 0x7f83c40a9360 2021-02-10T11:20:11.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.982 7f83c307d700 1 --2- 172.21.2.101:0/681705879 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83a80186c0 0x7f83a801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:11.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:11.982 7f83c307d700 1 --2- 172.21.2.101:0/681705879 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83a80186c0 0x7f83a801abc0 crc :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:12.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.013 7f83c948d700 1 -- 172.21.2.101:0/681705879 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c40f0b50 con 0x7f83c40a9360 2021-02-10T11:20:12.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.015 7f83bbfff700 1 -- 172.21.2.101:0/681705879 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f83bc006ef0 con 0x7f83c40a9360 2021-02-10T11:20:12.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.140 7f83c948d700 1 -- 172.21.2.101:0/681705879 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839271", "target": ["mgr", ""]}) v1 -- 0x7f83c40f0b50 con 0x7f83a80186c0 2021-02-10T11:20:12.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.166 7f83bbfff700 1 -- 172.21.2.101:0/681705879 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f83ac000ce0 con 0x7f83a80186c0 2021-02-10T11:20:12.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.173 7f83c948d700 1 -- 172.21.2.101:0/681705879 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83a80186c0 msgr2=0x7f83a801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:12.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.173 7f83c948d700 1 --2- 172.21.2.101:0/681705879 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83a80186c0 0x7f83a801abc0 crc :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:12.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.173 7f83c948d700 1 -- 172.21.2.101:0/681705879 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83c40a9360 msgr2=0x7f83c40a9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:12.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.173 7f83c948d700 1 --2- 172.21.2.101:0/681705879 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83c40a9360 0x7f83c40a9760 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f83bc00b890 tx=0x7f83bc00bc30).stop 2021-02-10T11:20:12.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.173 7f83c948d700 1 -- 172.21.2.101:0/681705879 shutdown_connections 2021-02-10T11:20:12.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.173 7f83c948d700 1 --2- 172.21.2.101:0/681705879 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83a80186c0 0x7f83a801abc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:12.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.173 7f83c948d700 1 --2- 172.21.2.101:0/681705879 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83c40a8230 0x7f83c40a8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:12.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.173 7f83c948d700 1 --2- 172.21.2.101:0/681705879 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83c40a9360 0x7f83c40a9760 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:12.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.173 7f83c948d700 1 --2- 172.21.2.101:0/681705879 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83c41933f0 0x7f83c40a7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:12.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.173 7f83c948d700 1 -- 172.21.2.101:0/681705879 >> 172.21.2.101:0/681705879 conn(0x7f83c40bf4e0 msgr2=0x7f83c40937f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:12.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.174 7f83c948d700 1 -- 172.21.2.101:0/681705879 shutdown_connections 2021-02-10T11:20:12.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.174 7f83c948d700 1 -- 172.21.2.101:0/681705879 wait complete. 2021-02-10T11:20:12.189 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839271 2021-02-10T11:20:12.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.600 7f109915e700 1 Processor -- start 2021-02-10T11:20:12.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.600 7f109915e700 1 -- start start 2021-02-10T11:20:12.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.601 7f109915e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f10941b0470 0x7f10941b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:12.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.601 7f109915e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f10940ef610 0x7f10940efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:12.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.601 7f109915e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10940f0b50 con 0x7f10940f0030 2021-02-10T11:20:12.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.601 7f109915e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10940e52e0 con 0x7f10941b0470 2021-02-10T11:20:12.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.601 7f109915e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10940e5490 con 0x7f10940ef610 2021-02-10T11:20:12.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.601 7f1092d9d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f10941b0470 0x7f10941b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:12.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.601 7f109359e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f10940f0030 0x7f10940f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47106/0 (socket says 172.21.2.101:47106) 2021-02-10T11:20:12.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.601 7f109359e700 1 -- 172.21.2.101:0/1154188702 learned_addr learned my addr 172.21.2.101:0/1154188702 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:20:12.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.601 7f1091d9b700 1 -- 172.21.2.101:0/1154188702 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 184014489 0 0) 0x7f108c001500 con 0x7f10940f0030 2021-02-10T11:20:12.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.601 7f1091d9b700 1 -- 172.21.2.101:0/1154188702 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1080002fe0 con 0x7f10940f0030 2021-02-10T11:20:12.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.601 7f1092d9d700 1 -- 172.21.2.101:0/1154188702 >> v2:172.21.2.108:3300/0 conn(0x7f10940ef610 msgr2=0x7f10940efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:20:12.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.601 7f1092d9d700 1 --2- 172.21.2.101:0/1154188702 >> v2:172.21.2.108:3300/0 conn(0x7f10940ef610 0x7f10940efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:12.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.601 7f1092d9d700 1 -- 172.21.2.101:0/1154188702 >> v1:172.21.2.101:6789/0 conn(0x7f10940f0030 legacy=0x7f10940f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:12.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.601 7f1092d9d700 1 -- 172.21.2.101:0/1154188702 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10940f0430 con 0x7f10941b0470 2021-02-10T11:20:12.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.601 7f1092d9d700 1 --2- 172.21.2.101:0/1154188702 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f10941b0470 0x7f10941b0870 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f1084006650 tx=0x7f1084009a80).ready entity=mon.0 client_cookie=a7f944911c26be67 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:12.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.602 7f1091d9b700 1 -- 172.21.2.101:0/1154188702 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f10840009b0 con 0x7f10941b0470 2021-02-10T11:20:12.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.602 7f1091d9b700 1 -- 172.21.2.101:0/1154188702 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108400c0c0 con 0x7f10941b0470 2021-02-10T11:20:12.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.602 7f1091d9b700 1 -- 172.21.2.101:0/1154188702 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f108400c5e0 con 0x7f10941b0470 2021-02-10T11:20:12.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.602 7f109915e700 1 -- 172.21.2.101:0/1154188702 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f10941b0470 msgr2=0x7f10941b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:12.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.602 7f109915e700 1 --2- 172.21.2.101:0/1154188702 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f10941b0470 0x7f10941b0870 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f1084006650 tx=0x7f1084009a80).stop 2021-02-10T11:20:12.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.602 7f109915e700 1 -- 172.21.2.101:0/1154188702 shutdown_connections 2021-02-10T11:20:12.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.602 7f109915e700 1 --2- 172.21.2.101:0/1154188702 >> v2:172.21.2.108:3300/0 conn(0x7f10940ef610 0x7f10940efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:12.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.602 7f109915e700 1 --2- 172.21.2.101:0/1154188702 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f10941b0470 0x7f10941b0870 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:12.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.602 7f109915e700 1 -- 172.21.2.101:0/1154188702 >> 172.21.2.101:0/1154188702 conn(0x7f10940ce3e0 msgr2=0x7f10940e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:12.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.603 7f109915e700 1 -- 172.21.2.101:0/1154188702 shutdown_connections 2021-02-10T11:20:12.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.603 7f109915e700 1 -- 172.21.2.101:0/1154188702 wait complete. 2021-02-10T11:20:12.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.617 7f109915e700 1 Processor -- start 2021-02-10T11:20:12.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.617 7f109915e700 1 -- start start 2021-02-10T11:20:12.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.618 7f109915e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f109408ece0 0x7f10940a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:12.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.618 7f109915e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10940a6860 0x7f10940a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:12.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.618 7f109915e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f10940a7280 0x7f10940a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:12.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.618 7f109915e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10940a5cc0 con 0x7f10940a7280 2021-02-10T11:20:12.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.618 7f109915e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109408f0e0 con 0x7f109408ece0 2021-02-10T11:20:12.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.618 7f109915e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109408f2c0 con 0x7f10940a6860 2021-02-10T11:20:12.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.618 7f109359e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f10940a7280 0x7f10940a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:12.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.618 7f109359e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f10940a7280 0x7f10940a4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46616/0 (socket says 172.21.2.101:46616) 2021-02-10T11:20:12.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.618 7f109359e700 1 -- 172.21.2.101:0/1678819449 learned_addr learned my addr 172.21.2.101:0/1678819449 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:20:12.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.618 7f109359e700 1 -- 172.21.2.101:0/1678819449 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10940a6860 msgr2=0x7f10940a6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:12.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.618 7f109359e700 1 --2- 172.21.2.101:0/1678819449 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10940a6860 0x7f10940a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:12.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.618 7f109359e700 1 -- 172.21.2.101:0/1678819449 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f109408ece0 msgr2=0x7f10940a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:12.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.618 7f109359e700 1 --2- 172.21.2.101:0/1678819449 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f109408ece0 0x7f10940a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:12.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.618 7f109359e700 1 -- 172.21.2.101:0/1678819449 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10940a4bb0 con 0x7f10940a7280 2021-02-10T11:20:12.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.619 7f109359e700 1 --2- 172.21.2.101:0/1678819449 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f10940a7280 0x7f10940a4590 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f108c009980 tx=0x7f108c00bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:12.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.619 7f108b7fe700 1 -- 172.21.2.101:0/1678819449 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f108c00be10 con 0x7f10940a7280 2021-02-10T11:20:12.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.619 7f108b7fe700 1 -- 172.21.2.101:0/1678819449 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c006080 con 0x7f10940a7280 2021-02-10T11:20:12.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.619 7f108b7fe700 1 -- 172.21.2.101:0/1678819449 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f108c0064e0 con 0x7f10940a7280 2021-02-10T11:20:12.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.619 7f109915e700 1 -- 172.21.2.101:0/1678819449 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10940a4bb0 con 0x7f10940a7280 2021-02-10T11:20:12.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.619 7f109915e700 1 -- 172.21.2.101:0/1678819449 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10940a5140 con 0x7f10940a7280 2021-02-10T11:20:12.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.620 7f108b7fe700 1 -- 172.21.2.101:0/1678819449 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f108c006260 con 0x7f10940a7280 2021-02-10T11:20:12.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.620 7f108b7fe700 1 --2- 172.21.2.101:0/1678819449 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10780186c0 0x7f107801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:12.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.620 7f108b7fe700 1 -- 172.21.2.101:0/1678819449 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f108c032900 con 0x7f10940a7280 2021-02-10T11:20:12.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.620 7f1092d9d700 1 --2- 172.21.2.101:0/1678819449 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10780186c0 0x7f107801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:12.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.620 7f1092d9d700 1 --2- 172.21.2.101:0/1678819449 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10780186c0 0x7f107801abc0 crc :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:12.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.651 7f109915e700 1 -- 172.21.2.101:0/1678819449 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1094105ae0 con 0x7f10940a7280 2021-02-10T11:20:12.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.653 7f108b7fe700 1 -- 172.21.2.101:0/1678819449 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f108c000d30 con 0x7f10940a7280 2021-02-10T11:20:12.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.782 7f109915e700 1 -- 172.21.2.101:0/1678819449 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839271", "target": ["mgr", ""]}) v1 -- 0x7f10940a5140 con 0x7f10780186c0 2021-02-10T11:20:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.785 7f108b7fe700 1 -- 172.21.2.101:0/1678819449 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f1084006060 con 0x7f10780186c0 2021-02-10T11:20:12.792 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839271/2b733fec-3547-4d7d-af39-3705df8c4dba 2021-02-10T11:20:12.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.786 7f109915e700 1 -- 172.21.2.101:0/1678819449 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10780186c0 msgr2=0x7f107801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:12.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.786 7f109915e700 1 --2- 172.21.2.101:0/1678819449 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10780186c0 0x7f107801abc0 crc :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:12.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.786 7f109915e700 1 -- 172.21.2.101:0/1678819449 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f10940a7280 msgr2=0x7f10940a4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:12.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.786 7f109915e700 1 --2- 172.21.2.101:0/1678819449 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f10940a7280 0x7f10940a4590 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f108c009980 tx=0x7f108c00bcd0).stop 2021-02-10T11:20:12.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.786 7f109915e700 1 -- 172.21.2.101:0/1678819449 shutdown_connections 2021-02-10T11:20:12.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.786 7f109915e700 1 --2- 172.21.2.101:0/1678819449 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10780186c0 0x7f107801abc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:12.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.786 7f109915e700 1 --2- 172.21.2.101:0/1678819449 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10940a6860 0x7f10940a6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:12.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.786 7f109915e700 1 --2- 172.21.2.101:0/1678819449 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f10940a7280 0x7f10940a4590 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:12.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.786 7f109915e700 1 --2- 172.21.2.101:0/1678819449 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f109408ece0 0x7f10940a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:12.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.786 7f109915e700 1 -- 172.21.2.101:0/1678819449 >> 172.21.2.101:0/1678819449 conn(0x7f109408ffe0 msgr2=0x7f10940bf050 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:12.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.786 7f109915e700 1 -- 172.21.2.101:0/1678819449 shutdown_connections 2021-02-10T11:20:12.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:12.786 7f109915e700 1 -- 172.21.2.101:0/1678819449 wait complete. 2021-02-10T11:20:12.802 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839271 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839271/2b733fec-3547-4d7d-af39-3705df8c4dba 2021-02-10T11:20:12.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:12.803 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:12.803 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:13.143 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:13.144 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:13.144 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839271/2b733fec-3547-4d7d-af39-3705df8c4dba/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:13.528 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:13.528 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:13.528 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233175 s, 45.0 MB/s 2021-02-10T11:20:13.529 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:13.529 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:13.530 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:13.867 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:13.867 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:13.867 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839271/2b733fec-3547-4d7d-af39-3705df8c4dba/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:14.241 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:14.241 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:14.242 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0255642 s, 41.0 MB/s 2021-02-10T11:20:14.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:14.243 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:14.243 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:14.578 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:14.579 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:14.579 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839271/2b733fec-3547-4d7d-af39-3705df8c4dba/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:14.953 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:14.953 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:14.954 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238403 s, 44.0 MB/s 2021-02-10T11:20:14.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:14.955 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:14.955 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:15.290 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:15.290 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:15.291 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839271/2b733fec-3547-4d7d-af39-3705df8c4dba/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:15.664 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:15.664 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:15.664 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024416 s, 42.9 MB/s 2021-02-10T11:20:15.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:15.665 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:15.665 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:16.001 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:16.002 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:16.002 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839271/2b733fec-3547-4d7d-af39-3705df8c4dba/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:16.377 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:16.377 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:16.377 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243384 s, 43.1 MB/s 2021-02-10T11:20:16.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:16.378 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:16.378 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:16.714 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:16.715 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:16.715 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839271/2b733fec-3547-4d7d-af39-3705df8c4dba/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:17.089 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:17.089 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:17.089 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023838 s, 44.0 MB/s 2021-02-10T11:20:17.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:17.090 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:17.090 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:17.425 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:17.425 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:17.426 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839271/2b733fec-3547-4d7d-af39-3705df8c4dba/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:17.801 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:17.802 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:17.802 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.025242 s, 41.5 MB/s 2021-02-10T11:20:17.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:17.803 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:17.803 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:18.138 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:18.139 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:18.139 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839271/2b733fec-3547-4d7d-af39-3705df8c4dba/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:18.367 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:20:18.370 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:20:18.511 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:18.511 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:18.511 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233614 s, 44.9 MB/s 2021-02-10T11:20:18.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:18.512 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:18.512 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:18.847 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:18.848 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:18.848 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839271/2b733fec-3547-4d7d-af39-3705df8c4dba/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:19.219 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:19.220 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:19.220 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244607 s, 42.9 MB/s 2021-02-10T11:20:19.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:19.221 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:19.221 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:19.555 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:19.556 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:19.556 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839271/2b733fec-3547-4d7d-af39-3705df8c4dba/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:19.931 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:19.931 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:19.932 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241593 s, 43.4 MB/s 2021-02-10T11:20:19.932 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839272 2021-02-10T11:20:20.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.340 7f7e3f9d5700 1 Processor -- start 2021-02-10T11:20:20.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.340 7f7e3f9d5700 1 -- start start 2021-02-10T11:20:20.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.340 7f7e3f9d5700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7e381b0470 0x7f7e381b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:20.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.340 7f7e3f9d5700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e380f0b50 con 0x7f7e380ef610 2021-02-10T11:20:20.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.340 7f7e3f9d5700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e380f24e0 con 0x7f7e380f20e0 2021-02-10T11:20:20.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.340 7f7e3f9d5700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e380e5ea0 con 0x7f7e381b0470 2021-02-10T11:20:20.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.340 7f7e3cff0700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f7e380ef610 0x7f7e380efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47116/0 (socket says 172.21.2.101:47116) 2021-02-10T11:20:20.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.341 7f7e3cff0700 1 -- 172.21.2.101:0/647484114 learned_addr learned my addr 172.21.2.101:0/647484114 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:20:20.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.341 7f7e2bfff700 1 -- 172.21.2.101:0/647484114 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1282445841 0 0) 0x7f7e2c0010b0 con 0x7f7e380ef610 2021-02-10T11:20:20.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.341 7f7e2bfff700 1 -- 172.21.2.101:0/647484114 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e24002fe0 con 0x7f7e380ef610 2021-02-10T11:20:20.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.341 7f7e3d7f1700 1 --2- 172.21.2.101:0/647484114 >> v2:172.21.2.108:3300/0 conn(0x7f7e381b0470 0x7f7e381b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:20.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.341 7f7e2bfff700 1 -- 172.21.2.101:0/647484114 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3607194409 0 0) 0x7f7e2c0010b0 con 0x7f7e380ef610 2021-02-10T11:20:20.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.341 7f7e2bfff700 1 -- 172.21.2.101:0/647484114 >> v2:172.21.2.108:3300/0 conn(0x7f7e381b0470 msgr2=0x7f7e381b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:20:20.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.341 7f7e2bfff700 1 --2- 172.21.2.101:0/647484114 >> v2:172.21.2.108:3300/0 conn(0x7f7e381b0470 0x7f7e381b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:20.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.341 7f7e2bfff700 1 -- 172.21.2.101:0/647484114 >> v1:172.21.2.108:6789/0 conn(0x7f7e380f20e0 legacy=0x7f7e380f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:20.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.341 7f7e2bfff700 1 -- 172.21.2.101:0/647484114 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e380e52e0 con 0x7f7e380ef610 2021-02-10T11:20:20.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.341 7f7e2bfff700 1 -- 172.21.2.101:0/647484114 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f7e2c0015b0 con 0x7f7e380ef610 2021-02-10T11:20:20.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.341 7f7e2bfff700 1 -- 172.21.2.101:0/647484114 >> v1:172.21.2.101:6789/0 conn(0x7f7e380ef610 legacy=0x7f7e380efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:20.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.341 7f7e2bfff700 1 --2- 172.21.2.101:0/647484114 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7e24003b10 0x7f7e24005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:20.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.341 7f7e2bfff700 1 --2- 172.21.2.101:0/647484114 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e240067a0 0x7f7e24008ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:20.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.341 7f7e2bfff700 1 --2- 172.21.2.101:0/647484114 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e24009580 0x7f7e2400baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:20.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.341 7f7e2bfff700 1 -- 172.21.2.101:0/647484114 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2400c2f0 con 0x7f7e240067a0 2021-02-10T11:20:20.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.341 7f7e2bfff700 1 -- 172.21.2.101:0/647484114 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2400c510 con 0x7f7e24003b10 2021-02-10T11:20:20.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.341 7f7e2bfff700 1 -- 172.21.2.101:0/647484114 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2400c730 con 0x7f7e24009580 2021-02-10T11:20:20.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.342 7f7e3cff0700 1 --2- 172.21.2.101:0/647484114 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e240067a0 0x7f7e24008ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:20.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.342 7f7e3d7f1700 1 --2- 172.21.2.101:0/647484114 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7e24003b10 0x7f7e24005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:20.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.342 7f7e3dff2700 1 --2- 172.21.2.101:0/647484114 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e24009580 0x7f7e2400baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:20.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.342 7f7e3cff0700 1 -- 172.21.2.101:0/647484114 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e24009580 msgr2=0x7f7e2400baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:20:20.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.342 7f7e3cff0700 1 --2- 172.21.2.101:0/647484114 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e24009580 0x7f7e2400baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:20.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.342 7f7e3cff0700 1 -- 172.21.2.101:0/647484114 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7e24003b10 msgr2=0x7f7e24005f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:20:20.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.342 7f7e3cff0700 1 --2- 172.21.2.101:0/647484114 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7e24003b10 0x7f7e24005f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:20.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.342 7f7e3cff0700 1 -- 172.21.2.101:0/647484114 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e240509a0 con 0x7f7e240067a0 2021-02-10T11:20:20.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.342 7f7e3dff2700 1 --2- 172.21.2.101:0/647484114 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e24009580 0x7f7e2400baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:20:20.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.342 7f7e3dff2700 1 -- 172.21.2.101:0/647484114 reap_dead start 2021-02-10T11:20:20.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.342 7f7e3d7f1700 1 --2- 172.21.2.101:0/647484114 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7e24003b10 0x7f7e24005f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:20:20.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.342 7f7e3cff0700 1 --2- 172.21.2.101:0/647484114 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e240067a0 0x7f7e24008ca0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f7e2c001c60 tx=0x7f7e2c007d30).ready entity=mon.0 client_cookie=50a3dcd82961b7c0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:20.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.342 7f7e2bfff700 1 -- 172.21.2.101:0/647484114 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7e2c001010 con 0x7f7e240067a0 2021-02-10T11:20:20.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.342 7f7e2bfff700 1 -- 172.21.2.101:0/647484114 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c001360 con 0x7f7e240067a0 2021-02-10T11:20:20.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.342 7f7e2bfff700 1 -- 172.21.2.101:0/647484114 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7e2c00a580 con 0x7f7e240067a0 2021-02-10T11:20:20.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.342 7f7e3f9d5700 1 -- 172.21.2.101:0/647484114 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e240067a0 msgr2=0x7f7e24008ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:20.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.342 7f7e3f9d5700 1 --2- 172.21.2.101:0/647484114 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e240067a0 0x7f7e24008ca0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f7e2c001c60 tx=0x7f7e2c007d30).stop 2021-02-10T11:20:20.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.343 7f7e3f9d5700 1 -- 172.21.2.101:0/647484114 shutdown_connections 2021-02-10T11:20:20.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.343 7f7e3f9d5700 1 --2- 172.21.2.101:0/647484114 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e240067a0 0x7f7e24008ca0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:20.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.343 7f7e3f9d5700 1 -- 172.21.2.101:0/647484114 >> 172.21.2.101:0/647484114 conn(0x7f7e380ce3e0 msgr2=0x7f7e380e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:20.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.343 7f7e3f9d5700 1 -- 172.21.2.101:0/647484114 shutdown_connections 2021-02-10T11:20:20.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.343 7f7e3f9d5700 1 -- 172.21.2.101:0/647484114 wait complete. 2021-02-10T11:20:20.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.358 7f7e3f9d5700 1 Processor -- start 2021-02-10T11:20:20.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.358 7f7e3f9d5700 1 -- start start 2021-02-10T11:20:20.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.358 7f7e3f9d5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7e380b7db0 0x7f7e380b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:20.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.358 7f7e3f9d5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e380b8880 0x7f7e381a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:20.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.358 7f7e3f9d5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e380b8c80 0x7f7e381a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:20.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.358 7f7e3f9d5700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e380b79b0 con 0x7f7e380b8880 2021-02-10T11:20:20.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.358 7f7e3f9d5700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e381a8090 con 0x7f7e380b7db0 2021-02-10T11:20:20.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.358 7f7e3f9d5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e381a82b0 con 0x7f7e380b8c80 2021-02-10T11:20:20.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.358 7f7e3cff0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e380b8880 0x7f7e381a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:20.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.358 7f7e3cff0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e380b8880 0x7f7e381a7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46634/0 (socket says 172.21.2.101:46634) 2021-02-10T11:20:20.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.358 7f7e3cff0700 1 -- 172.21.2.101:0/324868762 learned_addr learned my addr 172.21.2.101:0/324868762 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:20:20.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.359 7f7e3cff0700 1 -- 172.21.2.101:0/324868762 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e380b8c80 msgr2=0x7f7e381a7a70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:20:20.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.359 7f7e3cff0700 1 --2- 172.21.2.101:0/324868762 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e380b8c80 0x7f7e381a7a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:20.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.359 7f7e3cff0700 1 -- 172.21.2.101:0/324868762 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7e380b7db0 msgr2=0x7f7e380b8260 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:20:20.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.359 7f7e3cff0700 1 --2- 172.21.2.101:0/324868762 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7e380b7db0 0x7f7e380b8260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:20.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.359 7f7e3cff0700 1 -- 172.21.2.101:0/324868762 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e38190c10 con 0x7f7e380b8880 2021-02-10T11:20:20.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.359 7f7e3cff0700 1 --2- 172.21.2.101:0/324868762 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e380b8880 0x7f7e381a7450 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f7e2c00a550 tx=0x7f7e2c00a8f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:20.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.359 7f7e29ffb700 1 -- 172.21.2.101:0/324868762 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7e2c007480 con 0x7f7e380b8880 2021-02-10T11:20:20.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.359 7f7e29ffb700 1 -- 172.21.2.101:0/324868762 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c000a30 con 0x7f7e380b8880 2021-02-10T11:20:20.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.359 7f7e29ffb700 1 -- 172.21.2.101:0/324868762 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7e2c0043c0 con 0x7f7e380b8880 2021-02-10T11:20:20.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.359 7f7e3f9d5700 1 -- 172.21.2.101:0/324868762 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e38190c10 con 0x7f7e380b8880 2021-02-10T11:20:20.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.359 7f7e3f9d5700 1 -- 172.21.2.101:0/324868762 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e380c1eb0 con 0x7f7e380b8880 2021-02-10T11:20:20.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.360 7f7e29ffb700 1 -- 172.21.2.101:0/324868762 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7e2c007480 con 0x7f7e380b8880 2021-02-10T11:20:20.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.360 7f7e29ffb700 1 --2- 172.21.2.101:0/324868762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7e180186c0 0x7f7e1801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:20.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.360 7f7e29ffb700 1 -- 172.21.2.101:0/324868762 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7e2c032400 con 0x7f7e380b8880 2021-02-10T11:20:20.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.360 7f7e3d7f1700 1 --2- 172.21.2.101:0/324868762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7e180186c0 0x7f7e1801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:20.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.360 7f7e3d7f1700 1 --2- 172.21.2.101:0/324868762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7e180186c0 0x7f7e1801abc0 crc :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:20.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.390 7f7e3f9d5700 1 -- 172.21.2.101:0/324868762 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e38105b30 con 0x7f7e380b8880 2021-02-10T11:20:20.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.393 7f7e29ffb700 1 -- 172.21.2.101:0/324868762 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7e2c030d30 con 0x7f7e380b8880 2021-02-10T11:20:20.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.519 7f7e3f9d5700 1 -- 172.21.2.101:0/324868762 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839272", "target": ["mgr", ""]}) v1 -- 0x7f7e38190c10 con 0x7f7e180186c0 2021-02-10T11:20:20.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.547 7f7e29ffb700 1 -- 172.21.2.101:0/324868762 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f7e34000c90 con 0x7f7e180186c0 2021-02-10T11:20:20.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.552 7f7e3f9d5700 1 -- 172.21.2.101:0/324868762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7e180186c0 msgr2=0x7f7e1801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:20.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.552 7f7e3f9d5700 1 --2- 172.21.2.101:0/324868762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7e180186c0 0x7f7e1801abc0 crc :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:20.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.552 7f7e3f9d5700 1 -- 172.21.2.101:0/324868762 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e380b8880 msgr2=0x7f7e381a7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:20.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.552 7f7e3f9d5700 1 --2- 172.21.2.101:0/324868762 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e380b8880 0x7f7e381a7450 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f7e2c00a550 tx=0x7f7e2c00a8f0).stop 2021-02-10T11:20:20.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.552 7f7e3f9d5700 1 -- 172.21.2.101:0/324868762 shutdown_connections 2021-02-10T11:20:20.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.552 7f7e3f9d5700 1 --2- 172.21.2.101:0/324868762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7e180186c0 0x7f7e1801abc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:20.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.552 7f7e3f9d5700 1 --2- 172.21.2.101:0/324868762 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e380b8c80 0x7f7e381a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:20.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.552 7f7e3f9d5700 1 --2- 172.21.2.101:0/324868762 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e380b8880 0x7f7e381a7450 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:20.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.552 7f7e3f9d5700 1 --2- 172.21.2.101:0/324868762 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7e380b7db0 0x7f7e380b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:20.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.552 7f7e3f9d5700 1 -- 172.21.2.101:0/324868762 >> 172.21.2.101:0/324868762 conn(0x7f7e38094e10 msgr2=0x7f7e380a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:20.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.552 7f7e3f9d5700 1 -- 172.21.2.101:0/324868762 shutdown_connections 2021-02-10T11:20:20.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.552 7f7e3f9d5700 1 -- 172.21.2.101:0/324868762 wait complete. 2021-02-10T11:20:20.567 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839272 2021-02-10T11:20:20.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.974 7fc308a8b700 1 Processor -- start 2021-02-10T11:20:20.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.974 7fc308a8b700 1 -- start start 2021-02-10T11:20:20.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.974 7fc308a8b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3041b0470 0x7fc3041b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:20.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.974 7fc308a8b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc3040ef610 0x7fc3040efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:20.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.974 7fc308a8b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3040f0b50 con 0x7fc3040f0030 2021-02-10T11:20:20.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.974 7fc308a8b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3040e52e0 con 0x7fc3040ef610 2021-02-10T11:20:20.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.974 7fc308a8b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3040e5490 con 0x7fc3041b0470 2021-02-10T11:20:20.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.975 7fc30305f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc3040f0030 0x7fc3040f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47138/0 (socket says 172.21.2.101:47138) 2021-02-10T11:20:20.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.975 7fc30305f700 1 -- 172.21.2.101:0/2415696669 learned_addr learned my addr 172.21.2.101:0/2415696669 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:20:20.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.975 7fc30185c700 1 -- 172.21.2.101:0/2415696669 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844585643 0 0) 0x7fc2fc001500 con 0x7fc3040f0030 2021-02-10T11:20:20.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.975 7fc30185c700 1 -- 172.21.2.101:0/2415696669 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2f0002fe0 con 0x7fc3040f0030 2021-02-10T11:20:20.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.975 7fc30285e700 1 --2- 172.21.2.101:0/2415696669 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3041b0470 0x7fc3041b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:20.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.975 7fc30205d700 1 --2- 172.21.2.101:0/2415696669 >> v2:172.21.2.108:3300/0 conn(0x7fc3040ef610 0x7fc3040efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:20.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.975 7fc30185c700 1 -- 172.21.2.101:0/2415696669 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 173558479 0 0) 0x7fc2fc001460 con 0x7fc3040f0030 2021-02-10T11:20:20.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.975 7fc30185c700 1 -- 172.21.2.101:0/2415696669 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3041b0470 msgr2=0x7fc3041b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:20:21.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.975 7fc30185c700 1 --2- 172.21.2.101:0/2415696669 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3041b0470 0x7fc3041b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:21.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.975 7fc30185c700 1 -- 172.21.2.101:0/2415696669 >> v2:172.21.2.108:3300/0 conn(0x7fc3040ef610 msgr2=0x7fc3040efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:20:21.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.975 7fc30185c700 1 --2- 172.21.2.101:0/2415696669 >> v2:172.21.2.108:3300/0 conn(0x7fc3040ef610 0x7fc3040efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:21.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.975 7fc30185c700 1 -- 172.21.2.101:0/2415696669 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3040f0430 con 0x7fc3040f0030 2021-02-10T11:20:21.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.975 7fc30185c700 1 -- 172.21.2.101:0/2415696669 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fc2fc001980 con 0x7fc3040f0030 2021-02-10T11:20:21.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.975 7fc30185c700 1 -- 172.21.2.101:0/2415696669 >> v1:172.21.2.101:6789/0 conn(0x7fc3040f0030 legacy=0x7fc3040f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:21.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.975 7fc30185c700 1 --2- 172.21.2.101:0/2415696669 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc2f0003af0 0x7fc2f0005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:21.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.975 7fc30185c700 1 --2- 172.21.2.101:0/2415696669 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2f0006820 0x7fc2f0008d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:21.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.975 7fc30185c700 1 --2- 172.21.2.101:0/2415696669 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2f00095b0 0x7fc2f000bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:21.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.975 7fc30185c700 1 -- 172.21.2.101:0/2415696669 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f000c380 con 0x7fc2f00095b0 2021-02-10T11:20:21.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.975 7fc30185c700 1 -- 172.21.2.101:0/2415696669 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f0050570 con 0x7fc2f0003af0 2021-02-10T11:20:21.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.975 7fc30185c700 1 -- 172.21.2.101:0/2415696669 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f0050790 con 0x7fc2f0006820 2021-02-10T11:20:21.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.975 7fc30205d700 1 --2- 172.21.2.101:0/2415696669 >> v2:172.21.2.108:3300/0 conn(0x7fc3040ef610 0x7fc3040efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:20:21.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.975 7fc30305f700 1 --2- 172.21.2.101:0/2415696669 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2f00095b0 0x7fc2f000bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:21.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.976 7fc30285e700 1 --2- 172.21.2.101:0/2415696669 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc2f0003af0 0x7fc2f0005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:21.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.976 7fc30205d700 1 --2- 172.21.2.101:0/2415696669 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2f0006820 0x7fc2f0008d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:21.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.976 7fc30305f700 1 -- 172.21.2.101:0/2415696669 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2f0006820 msgr2=0x7fc2f0008d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:20:21.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.976 7fc30305f700 1 --2- 172.21.2.101:0/2415696669 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2f0006820 0x7fc2f0008d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:21.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.976 7fc30305f700 1 -- 172.21.2.101:0/2415696669 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc2f0003af0 msgr2=0x7fc2f0005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:20:21.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.976 7fc30305f700 1 --2- 172.21.2.101:0/2415696669 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc2f0003af0 0x7fc2f0005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:21.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.976 7fc30305f700 1 -- 172.21.2.101:0/2415696669 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f0050a30 con 0x7fc2f00095b0 2021-02-10T11:20:21.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.976 7fc30305f700 1 --2- 172.21.2.101:0/2415696669 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2f00095b0 0x7fc2f000bad0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fc2fc001ae0 tx=0x7fc2fc00a310).ready entity=mon.0 client_cookie=f742c05ba1be3cb6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:21.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.976 7fc30185c700 1 -- 172.21.2.101:0/2415696669 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc2fc0070c0 con 0x7fc2f00095b0 2021-02-10T11:20:21.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.976 7fc30185c700 1 -- 172.21.2.101:0/2415696669 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2fc007400 con 0x7fc2f00095b0 2021-02-10T11:20:21.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.976 7fc30185c700 1 -- 172.21.2.101:0/2415696669 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc2fc0049a0 con 0x7fc2f00095b0 2021-02-10T11:20:21.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.976 7fc308a8b700 1 -- 172.21.2.101:0/2415696669 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2f00095b0 msgr2=0x7fc2f000bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:21.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.976 7fc308a8b700 1 --2- 172.21.2.101:0/2415696669 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2f00095b0 0x7fc2f000bad0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fc2fc001ae0 tx=0x7fc2fc00a310).stop 2021-02-10T11:20:21.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.976 7fc30305f700 1 -- 172.21.2.101:0/2415696669 reap_dead start 2021-02-10T11:20:21.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.977 7fc308a8b700 1 -- 172.21.2.101:0/2415696669 shutdown_connections 2021-02-10T11:20:21.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.977 7fc308a8b700 1 -- 172.21.2.101:0/2415696669 >> 172.21.2.101:0/2415696669 conn(0x7fc3040ce3e0 msgr2=0x7fc3040e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:21.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.977 7fc308a8b700 1 -- 172.21.2.101:0/2415696669 shutdown_connections 2021-02-10T11:20:21.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.977 7fc308a8b700 1 -- 172.21.2.101:0/2415696669 wait complete. 2021-02-10T11:20:21.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.992 7fc308a8b700 1 Processor -- start 2021-02-10T11:20:21.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.992 7fc308a8b700 1 -- start start 2021-02-10T11:20:21.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.992 7fc308a8b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3040924b0 0x7fc3041a3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:21.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.992 7fc308a8b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3041a3bb0 0x7fc3040bf920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:21.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.992 7fc308a8b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3040bff40 0x7fc3040c0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:21.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.992 7fc308a8b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30409a0a0 con 0x7fc3040924b0 2021-02-10T11:20:21.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.992 7fc308a8b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3040c0960 con 0x7fc3041a3bb0 2021-02-10T11:20:21.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.992 7fc308a8b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc304098420 con 0x7fc3040bff40 2021-02-10T11:20:21.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.992 7fc30285e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3040924b0 0x7fc3041a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:21.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.992 7fc30285e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3040924b0 0x7fc3041a3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46652/0 (socket says 172.21.2.101:46652) 2021-02-10T11:20:21.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.992 7fc30285e700 1 -- 172.21.2.101:0/2993598418 learned_addr learned my addr 172.21.2.101:0/2993598418 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:20:21.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.992 7fc30285e700 1 -- 172.21.2.101:0/2993598418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3040bff40 msgr2=0x7fc3040c0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:21.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.993 7fc30285e700 1 --2- 172.21.2.101:0/2993598418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3040bff40 0x7fc3040c0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:21.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.993 7fc30285e700 1 -- 172.21.2.101:0/2993598418 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3041a3bb0 msgr2=0x7fc3040bf920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:20:21.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.993 7fc30285e700 1 --2- 172.21.2.101:0/2993598418 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3041a3bb0 0x7fc3040bf920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:21.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.993 7fc30285e700 1 -- 172.21.2.101:0/2993598418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc304098640 con 0x7fc3040924b0 2021-02-10T11:20:21.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.993 7fc30285e700 1 --2- 172.21.2.101:0/2993598418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3040924b0 0x7fc3041a3590 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7fc2ec00b590 tx=0x7fc2ec00b930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:21.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.993 7fc2fb7fe700 1 -- 172.21.2.101:0/2993598418 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc2ec006bb0 con 0x7fc3040924b0 2021-02-10T11:20:21.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.993 7fc2fb7fe700 1 -- 172.21.2.101:0/2993598418 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec00bd20 con 0x7fc3040924b0 2021-02-10T11:20:21.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.993 7fc2fb7fe700 1 -- 172.21.2.101:0/2993598418 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc2ec0052e0 con 0x7fc3040924b0 2021-02-10T11:20:21.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.993 7fc308a8b700 1 -- 172.21.2.101:0/2993598418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc304098640 con 0x7fc3040924b0 2021-02-10T11:20:21.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.993 7fc308a8b700 1 -- 172.21.2.101:0/2993598418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc304098640 con 0x7fc3040924b0 2021-02-10T11:20:21.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.993 7fc2fb7fe700 1 -- 172.21.2.101:0/2993598418 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc2ec006030 con 0x7fc3040924b0 2021-02-10T11:20:21.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.994 7fc2fb7fe700 1 --2- 172.21.2.101:0/2993598418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc2e80186c0 0x7fc2e801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:21.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.994 7fc2fb7fe700 1 -- 172.21.2.101:0/2993598418 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc2ec032560 con 0x7fc3040924b0 2021-02-10T11:20:21.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.994 7fc30205d700 1 --2- 172.21.2.101:0/2993598418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc2e80186c0 0x7fc2e801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:21.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:20.994 7fc30205d700 1 --2- 172.21.2.101:0/2993598418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc2e80186c0 0x7fc2e801abc0 crc :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:21.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:21.024 7fc308a8b700 1 -- 172.21.2.101:0/2993598418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc304105b40 con 0x7fc3040924b0 2021-02-10T11:20:21.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:21.027 7fc2fb7fe700 1 -- 172.21.2.101:0/2993598418 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc2ec006bb0 con 0x7fc3040924b0 2021-02-10T11:20:21.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:21.157 7fc308a8b700 1 -- 172.21.2.101:0/2993598418 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839272", "target": ["mgr", ""]}) v1 -- 0x7fc3040c0960 con 0x7fc2e80186c0 2021-02-10T11:20:21.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:21.160 7fc2fb7fe700 1 -- 172.21.2.101:0/2993598418 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fc2f4006060 con 0x7fc2e80186c0 2021-02-10T11:20:21.168 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839272/419b4e95-3054-44a1-bb63-dcf934caaa28 2021-02-10T11:20:21.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:21.162 7fc308a8b700 1 -- 172.21.2.101:0/2993598418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc2e80186c0 msgr2=0x7fc2e801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:21.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:21.162 7fc308a8b700 1 --2- 172.21.2.101:0/2993598418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc2e80186c0 0x7fc2e801abc0 crc :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:21.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:21.162 7fc308a8b700 1 -- 172.21.2.101:0/2993598418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3040924b0 msgr2=0x7fc3041a3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:21.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:21.162 7fc308a8b700 1 --2- 172.21.2.101:0/2993598418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3040924b0 0x7fc3041a3590 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7fc2ec00b590 tx=0x7fc2ec00b930).stop 2021-02-10T11:20:21.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:21.162 7fc308a8b700 1 -- 172.21.2.101:0/2993598418 shutdown_connections 2021-02-10T11:20:21.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:21.162 7fc308a8b700 1 --2- 172.21.2.101:0/2993598418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc2e80186c0 0x7fc2e801abc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:21.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:21.162 7fc308a8b700 1 --2- 172.21.2.101:0/2993598418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3040bff40 0x7fc3040c0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:21.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:21.162 7fc308a8b700 1 --2- 172.21.2.101:0/2993598418 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3041a3bb0 0x7fc3040bf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:21.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:21.162 7fc308a8b700 1 --2- 172.21.2.101:0/2993598418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3040924b0 0x7fc3041a3590 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:21.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:21.162 7fc308a8b700 1 -- 172.21.2.101:0/2993598418 >> 172.21.2.101:0/2993598418 conn(0x7fc3040b50c0 msgr2=0x7fc304092d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:21.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:21.162 7fc308a8b700 1 -- 172.21.2.101:0/2993598418 shutdown_connections 2021-02-10T11:20:21.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:21.162 7fc308a8b700 1 -- 172.21.2.101:0/2993598418 wait complete. 2021-02-10T11:20:21.176 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839272 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839272/419b4e95-3054-44a1-bb63-dcf934caaa28 2021-02-10T11:20:21.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:21.177 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:21.177 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:21.511 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:21.511 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:21.512 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839272/419b4e95-3054-44a1-bb63-dcf934caaa28/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:21.891 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:21.891 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:21.891 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234354 s, 44.7 MB/s 2021-02-10T11:20:21.892 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:21.892 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:21.892 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:22.228 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:22.229 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:22.229 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839272/419b4e95-3054-44a1-bb63-dcf934caaa28/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:22.602 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:22.602 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:22.602 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0225885 s, 46.4 MB/s 2021-02-10T11:20:22.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:22.603 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:22.604 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:22.939 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:22.939 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:22.940 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839272/419b4e95-3054-44a1-bb63-dcf934caaa28/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:23.315 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:23.315 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:23.315 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239252 s, 43.8 MB/s 2021-02-10T11:20:23.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:23.316 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:23.316 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:23.651 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:23.652 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:23.652 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839272/419b4e95-3054-44a1-bb63-dcf934caaa28/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:24.025 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:24.025 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:24.025 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248189 s, 42.2 MB/s 2021-02-10T11:20:24.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:24.026 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:24.026 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:24.362 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:24.362 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:24.363 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839272/419b4e95-3054-44a1-bb63-dcf934caaa28/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:24.740 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:24.740 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:24.740 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241024 s, 43.5 MB/s 2021-02-10T11:20:24.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:24.741 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:24.741 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:25.075 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:25.075 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:25.076 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839272/419b4e95-3054-44a1-bb63-dcf934caaa28/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:25.453 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:25.453 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:25.453 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251446 s, 41.7 MB/s 2021-02-10T11:20:25.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:25.454 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:25.454 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:25.790 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:25.791 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:25.791 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839272/419b4e95-3054-44a1-bb63-dcf934caaa28/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:26.168 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:26.168 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:26.168 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246115 s, 42.6 MB/s 2021-02-10T11:20:26.169 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:26.169 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:26.169 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:26.504 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:26.505 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:26.505 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839272/419b4e95-3054-44a1-bb63-dcf934caaa28/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:26.877 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:26.877 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:26.877 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0232578 s, 45.1 MB/s 2021-02-10T11:20:26.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:26.878 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:26.878 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:27.212 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:27.213 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:27.213 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839272/419b4e95-3054-44a1-bb63-dcf934caaa28/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:27.588 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:27.588 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:27.588 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245791 s, 42.7 MB/s 2021-02-10T11:20:27.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:27.589 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:27.589 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:27.924 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:27.925 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:27.925 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839272/419b4e95-3054-44a1-bb63-dcf934caaa28/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:28.303 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:28.303 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:28.303 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0271029 s, 38.7 MB/s 2021-02-10T11:20:28.304 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839273 2021-02-10T11:20:28.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.710 7f888b2c1700 1 Processor -- start 2021-02-10T11:20:28.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.710 7f888b2c1700 1 -- start start 2021-02-10T11:20:28.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.711 7f888b2c1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f88841b0870 0x7f88840f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:28.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.711 7f888b2c1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88840efe00 0x7f88840f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:28.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.711 7f888b2c1700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88841b0c70 con 0x7f88841b0470 2021-02-10T11:20:28.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.711 7f888b2c1700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88840f0b50 con 0x7f88841b0870 2021-02-10T11:20:28.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.711 7f888b2c1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88840f0820 con 0x7f88840efe00 2021-02-10T11:20:28.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.711 7f88888dc700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f88841b0870 0x7f88840f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:28.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.711 7f88888dc700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f88841b0870 0x7f88840f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46660/0 (socket says 172.21.2.101:46660) 2021-02-10T11:20:28.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.711 7f88888dc700 1 -- 172.21.2.101:0/1291940791 learned_addr learned my addr 172.21.2.101:0/1291940791 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:20:28.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.711 7f88898de700 1 --2- 172.21.2.101:0/1291940791 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88840efe00 0x7f88840f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:28.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.711 7f88888dc700 1 -- 172.21.2.101:0/1291940791 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88840efe00 msgr2=0x7f88840f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:20:28.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.711 7f88888dc700 1 --2- 172.21.2.101:0/1291940791 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88840efe00 0x7f88840f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:28.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.711 7f88888dc700 1 -- 172.21.2.101:0/1291940791 >> v1:172.21.2.108:6789/0 conn(0x7f88841b0470 legacy=0x7f88840ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:28.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.711 7f88888dc700 1 -- 172.21.2.101:0/1291940791 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88840e52e0 con 0x7f88841b0870 2021-02-10T11:20:28.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.711 7f88888dc700 1 --2- 172.21.2.101:0/1291940791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88841b0870 0x7f88840f20e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f8878006700 tx=0x7f8878009b80).ready entity=mon.0 client_cookie=c2b20fbee58a60c3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:28.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.711 7f8877fff700 1 -- 172.21.2.101:0/1291940791 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f887800c0c0 con 0x7f88841b0870 2021-02-10T11:20:28.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.712 7f8877fff700 1 -- 172.21.2.101:0/1291940791 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887800c340 con 0x7f88841b0870 2021-02-10T11:20:28.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.712 7f8877fff700 1 -- 172.21.2.101:0/1291940791 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f887800c860 con 0x7f88841b0870 2021-02-10T11:20:28.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.712 7f888b2c1700 1 -- 172.21.2.101:0/1291940791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88841b0870 msgr2=0x7f88840f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:28.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.712 7f888b2c1700 1 --2- 172.21.2.101:0/1291940791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88841b0870 0x7f88840f20e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f8878006700 tx=0x7f8878009b80).stop 2021-02-10T11:20:28.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.712 7f888b2c1700 1 -- 172.21.2.101:0/1291940791 shutdown_connections 2021-02-10T11:20:28.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.712 7f888b2c1700 1 --2- 172.21.2.101:0/1291940791 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88840efe00 0x7f88840f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:28.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.712 7f888b2c1700 1 --2- 172.21.2.101:0/1291940791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88841b0870 0x7f88840f20e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:28.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.712 7f888b2c1700 1 -- 172.21.2.101:0/1291940791 >> 172.21.2.101:0/1291940791 conn(0x7f88840ce3e0 msgr2=0x7f88840e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:28.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.712 7f888b2c1700 1 -- 172.21.2.101:0/1291940791 shutdown_connections 2021-02-10T11:20:28.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.712 7f888b2c1700 1 -- 172.21.2.101:0/1291940791 wait complete. 2021-02-10T11:20:28.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.729 7f888b2c1700 1 Processor -- start 2021-02-10T11:20:28.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.729 7f888b2c1700 1 -- start start 2021-02-10T11:20:28.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.730 7f888b2c1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f88840b8950 0x7f88840a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:28.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.730 7f888b2c1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88840a6830 0x7f88841a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:28.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.730 7f888b2c1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88841a3400 0x7f88841a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:28.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.730 7f888b2c1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88840a6cb0 con 0x7f88841a3400 2021-02-10T11:20:28.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.730 7f888b2c1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88840b8d50 con 0x7f88840b8950 2021-02-10T11:20:28.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.730 7f888b2c1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88840a6f70 con 0x7f88840a6830 2021-02-10T11:20:28.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.730 7f88898de700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88841a3400 0x7f88841a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:28.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.730 7f88898de700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88841a3400 0x7f88841a3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46668/0 (socket says 172.21.2.101:46668) 2021-02-10T11:20:28.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.730 7f88898de700 1 -- 172.21.2.101:0/657948728 learned_addr learned my addr 172.21.2.101:0/657948728 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:20:28.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.730 7f88888dc700 1 --2- 172.21.2.101:0/657948728 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88840a6830 0x7f88841a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:28.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.730 7f88890dd700 1 --2- 172.21.2.101:0/657948728 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f88840b8950 0x7f88840a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:28.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.730 7f88898de700 1 -- 172.21.2.101:0/657948728 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88840a6830 msgr2=0x7f88841a2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:28.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.730 7f88898de700 1 --2- 172.21.2.101:0/657948728 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88840a6830 0x7f88841a2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:28.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.730 7f88898de700 1 -- 172.21.2.101:0/657948728 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f88840b8950 msgr2=0x7f88840a6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:28.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.730 7f88898de700 1 --2- 172.21.2.101:0/657948728 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f88840b8950 0x7f88840a6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:28.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.730 7f88898de700 1 -- 172.21.2.101:0/657948728 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88840a96a0 con 0x7f88841a3400 2021-02-10T11:20:28.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.730 7f88890dd700 1 --2- 172.21.2.101:0/657948728 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f88840b8950 0x7f88840a6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:20:28.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.731 7f88898de700 1 --2- 172.21.2.101:0/657948728 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88841a3400 0x7f88841a3800 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f887c00b910 tx=0x7f887c00bcb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:28.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.731 7f8875ffb700 1 -- 172.21.2.101:0/657948728 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f887c006aa0 con 0x7f88841a3400 2021-02-10T11:20:28.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.731 7f8875ffb700 1 -- 172.21.2.101:0/657948728 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887c003030 con 0x7f88841a3400 2021-02-10T11:20:28.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.731 7f888b2c1700 1 -- 172.21.2.101:0/657948728 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88840a96a0 con 0x7f88841a3400 2021-02-10T11:20:28.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.731 7f8875ffb700 1 -- 172.21.2.101:0/657948728 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f887c001470 con 0x7f88841a3400 2021-02-10T11:20:28.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.731 7f888b2c1700 1 -- 172.21.2.101:0/657948728 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88840a96a0 con 0x7f88841a3400 2021-02-10T11:20:28.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.731 7f8875ffb700 1 -- 172.21.2.101:0/657948728 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f887c000ec0 con 0x7f88841a3400 2021-02-10T11:20:28.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.731 7f8875ffb700 1 --2- 172.21.2.101:0/657948728 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8864018730 0x7f886401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:28.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.731 7f8875ffb700 1 -- 172.21.2.101:0/657948728 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f887c0329e0 con 0x7f88841a3400 2021-02-10T11:20:28.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.731 7f88890dd700 1 --2- 172.21.2.101:0/657948728 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8864018730 0x7f886401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:28.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.732 7f88890dd700 1 --2- 172.21.2.101:0/657948728 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8864018730 0x7f886401ac10 crc :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:28.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.760 7f888b2c1700 1 -- 172.21.2.101:0/657948728 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88840b8d50 con 0x7f88841a3400 2021-02-10T11:20:28.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.763 7f8875ffb700 1 -- 172.21.2.101:0/657948728 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f887c00bdf0 con 0x7f88841a3400 2021-02-10T11:20:28.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.893 7f888b2c1700 1 -- 172.21.2.101:0/657948728 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839273", "target": ["mgr", ""]}) v1 -- 0x7f88840b8d50 con 0x7f8864018730 2021-02-10T11:20:28.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.920 7f8875ffb700 1 -- 172.21.2.101:0/657948728 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f8880006be0 con 0x7f8864018730 2021-02-10T11:20:28.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.926 7f888b2c1700 1 -- 172.21.2.101:0/657948728 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8864018730 msgr2=0x7f886401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:28.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.926 7f888b2c1700 1 --2- 172.21.2.101:0/657948728 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8864018730 0x7f886401ac10 crc :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:28.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.926 7f888b2c1700 1 -- 172.21.2.101:0/657948728 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88841a3400 msgr2=0x7f88841a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:28.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.926 7f888b2c1700 1 --2- 172.21.2.101:0/657948728 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88841a3400 0x7f88841a3800 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f887c00b910 tx=0x7f887c00bcb0).stop 2021-02-10T11:20:28.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.926 7f888b2c1700 1 -- 172.21.2.101:0/657948728 shutdown_connections 2021-02-10T11:20:28.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.926 7f888b2c1700 1 --2- 172.21.2.101:0/657948728 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8864018730 0x7f886401ac10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:28.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.926 7f888b2c1700 1 --2- 172.21.2.101:0/657948728 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88840a6830 0x7f88841a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:28.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.926 7f888b2c1700 1 --2- 172.21.2.101:0/657948728 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88841a3400 0x7f88841a3800 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:28.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.926 7f888b2c1700 1 --2- 172.21.2.101:0/657948728 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f88840b8950 0x7f88840a6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:28.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.926 7f888b2c1700 1 -- 172.21.2.101:0/657948728 >> 172.21.2.101:0/657948728 conn(0x7f88840bbf60 msgr2=0x7f8884105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:28.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.926 7f888b2c1700 1 -- 172.21.2.101:0/657948728 shutdown_connections 2021-02-10T11:20:28.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:28.926 7f888b2c1700 1 -- 172.21.2.101:0/657948728 wait complete. 2021-02-10T11:20:28.941 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839273 2021-02-10T11:20:29.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.347 7f5d97fff700 1 Processor -- start 2021-02-10T11:20:29.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.347 7f5d97fff700 1 -- start start 2021-02-10T11:20:29.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.347 7f5d97fff700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f5d901b0470 0x7f5d901b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:29.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.347 7f5d97fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d900ef610 0x7f5d900efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:29.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.347 7f5d97fff700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d900f0b50 con 0x7f5d900f0030 2021-02-10T11:20:29.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.347 7f5d97fff700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d900e52e0 con 0x7f5d901b0470 2021-02-10T11:20:29.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.347 7f5d97fff700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d900e54d0 con 0x7f5d900ef610 2021-02-10T11:20:29.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.348 7f5d95e1b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f5d901b0470 0x7f5d901b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.348 7f5d9561a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d900ef610 0x7f5d900efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.348 7f5d95e1b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f5d901b0470 0x7f5d901b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:57158/0 (socket says 172.21.2.101:57158) 2021-02-10T11:20:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.348 7f5d9561a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d900ef610 0x7f5d900efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:39404/0 (socket says 172.21.2.101:39404) 2021-02-10T11:20:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.348 7f5d95e1b700 1 -- 172.21.2.101:0/1457139453 learned_addr learned my addr 172.21.2.101:0/1457139453 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:20:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.349 7f5d95e1b700 1 -- 172.21.2.101:0/1457139453 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d900ef610 msgr2=0x7f5d900efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:20:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.349 7f5d95e1b700 1 --2- 172.21.2.101:0/1457139453 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d900ef610 0x7f5d900efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.349 7f5d95e1b700 1 -- 172.21.2.101:0/1457139453 >> v1:172.21.2.108:6789/0 conn(0x7f5d900f0030 legacy=0x7f5d900f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.349 7f5d95e1b700 1 -- 172.21.2.101:0/1457139453 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d900f0430 con 0x7f5d901b0470 2021-02-10T11:20:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.349 7f5d95e1b700 1 --2- 172.21.2.101:0/1457139453 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d901b0470 0x7f5d901b0870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f5d84006700 tx=0x7f5d84009b80).ready entity=mon.1 client_cookie=e1796517cbd5f8b5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.349 7f5d94e19700 1 -- 172.21.2.101:0/1457139453 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5d8400c0c0 con 0x7f5d901b0470 2021-02-10T11:20:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.349 7f5d94e19700 1 -- 172.21.2.101:0/1457139453 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8400c300 con 0x7f5d901b0470 2021-02-10T11:20:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.349 7f5d94e19700 1 -- 172.21.2.101:0/1457139453 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5d8400c820 con 0x7f5d901b0470 2021-02-10T11:20:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.349 7f5d97fff700 1 -- 172.21.2.101:0/1457139453 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d901b0470 msgr2=0x7f5d901b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:29.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.349 7f5d97fff700 1 --2- 172.21.2.101:0/1457139453 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d901b0470 0x7f5d901b0870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f5d84006700 tx=0x7f5d84009b80).stop 2021-02-10T11:20:29.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.350 7f5d97fff700 1 -- 172.21.2.101:0/1457139453 shutdown_connections 2021-02-10T11:20:29.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.350 7f5d97fff700 1 --2- 172.21.2.101:0/1457139453 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d900ef610 0x7f5d900efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:29.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.350 7f5d97fff700 1 --2- 172.21.2.101:0/1457139453 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d901b0470 0x7f5d901b0870 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:29.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.350 7f5d97fff700 1 -- 172.21.2.101:0/1457139453 >> 172.21.2.101:0/1457139453 conn(0x7f5d900ce3e0 msgr2=0x7f5d900e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:29.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.350 7f5d97fff700 1 -- 172.21.2.101:0/1457139453 shutdown_connections 2021-02-10T11:20:29.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.350 7f5d97fff700 1 -- 172.21.2.101:0/1457139453 wait complete. 2021-02-10T11:20:29.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.364 7f5d97fff700 1 Processor -- start 2021-02-10T11:20:29.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.365 7f5d97fff700 1 -- start start 2021-02-10T11:20:29.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.365 7f5d97fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d900a0420 0x7f5d900a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:29.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.365 7f5d97fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d90189d00 0x7f5d9018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:29.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.365 7f5d97fff700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d9018a720 0x7f5d900b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:29.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.365 7f5d97fff700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d90189780 con 0x7f5d900a0420 2021-02-10T11:20:29.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.365 7f5d97fff700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9018ab20 con 0x7f5d9018a720 2021-02-10T11:20:29.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.365 7f5d97fff700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d900a1140 con 0x7f5d90189d00 2021-02-10T11:20:29.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.365 7f5d95e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d900a0420 0x7f5d900a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:29.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.365 7f5d95e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d900a0420 0x7f5d900a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46674/0 (socket says 172.21.2.101:46674) 2021-02-10T11:20:29.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.365 7f5d95e1b700 1 -- 172.21.2.101:0/2767429901 learned_addr learned my addr 172.21.2.101:0/2767429901 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:20:29.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.366 7f5d9561a700 1 --2- 172.21.2.101:0/2767429901 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d90189d00 0x7f5d9018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:29.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.366 7f5d9661c700 1 --2- 172.21.2.101:0/2767429901 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d9018a720 0x7f5d900b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:29.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.366 7f5d95e1b700 1 -- 172.21.2.101:0/2767429901 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d90189d00 msgr2=0x7f5d9018a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:29.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.366 7f5d95e1b700 1 --2- 172.21.2.101:0/2767429901 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d90189d00 0x7f5d9018a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:29.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.366 7f5d95e1b700 1 -- 172.21.2.101:0/2767429901 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d9018a720 msgr2=0x7f5d900b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:29.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.366 7f5d95e1b700 1 --2- 172.21.2.101:0/2767429901 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d9018a720 0x7f5d900b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:29.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.366 7f5d95e1b700 1 -- 172.21.2.101:0/2767429901 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d900b8200 con 0x7f5d900a0420 2021-02-10T11:20:29.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.366 7f5d9561a700 1 --2- 172.21.2.101:0/2767429901 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d90189d00 0x7f5d9018a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:20:29.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.366 7f5d9661c700 1 --2- 172.21.2.101:0/2767429901 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d9018a720 0x7f5d900b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:20:29.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.366 7f5d95e1b700 1 --2- 172.21.2.101:0/2767429901 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d900a0420 0x7f5d900a0820 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f5d8400b5b0 tx=0x7f5d8400b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:29.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.366 7f5d827fc700 1 -- 172.21.2.101:0/2767429901 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5d84009440 con 0x7f5d900a0420 2021-02-10T11:20:29.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.366 7f5d827fc700 1 -- 172.21.2.101:0/2767429901 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d84006030 con 0x7f5d900a0420 2021-02-10T11:20:29.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.366 7f5d827fc700 1 -- 172.21.2.101:0/2767429901 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5d84005580 con 0x7f5d900a0420 2021-02-10T11:20:29.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.366 7f5d97fff700 1 -- 172.21.2.101:0/2767429901 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d900b8200 con 0x7f5d900a0420 2021-02-10T11:20:29.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.366 7f5d97fff700 1 -- 172.21.2.101:0/2767429901 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d900b8200 con 0x7f5d900a0420 2021-02-10T11:20:29.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.367 7f5d827fc700 1 -- 172.21.2.101:0/2767429901 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5d84009440 con 0x7f5d900a0420 2021-02-10T11:20:29.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.367 7f5d827fc700 1 --2- 172.21.2.101:0/2767429901 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5d780186c0 0x7f5d7801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:29.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.367 7f5d827fc700 1 -- 172.21.2.101:0/2767429901 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5d84032550 con 0x7f5d900a0420 2021-02-10T11:20:29.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.367 7f5d9561a700 1 --2- 172.21.2.101:0/2767429901 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5d780186c0 0x7f5d7801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:29.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.367 7f5d9561a700 1 --2- 172.21.2.101:0/2767429901 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5d780186c0 0x7f5d7801abc0 crc :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:29.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.397 7f5d97fff700 1 -- 172.21.2.101:0/2767429901 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d90105ae0 con 0x7f5d900a0420 2021-02-10T11:20:29.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.400 7f5d827fc700 1 -- 172.21.2.101:0/2767429901 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5d8402ade0 con 0x7f5d900a0420 2021-02-10T11:20:29.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.529 7f5d97fff700 1 -- 172.21.2.101:0/2767429901 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839273", "target": ["mgr", ""]}) v1 -- 0x7f5d90105ae0 con 0x7f5d780186c0 2021-02-10T11:20:29.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.533 7f5d827fc700 1 -- 172.21.2.101:0/2767429901 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f5d8c006b30 con 0x7f5d780186c0 2021-02-10T11:20:29.543 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839273/f40693f7-6185-4224-be75-bb28efddf11a 2021-02-10T11:20:29.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.538 7f5d97fff700 1 -- 172.21.2.101:0/2767429901 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5d780186c0 msgr2=0x7f5d7801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:29.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.538 7f5d97fff700 1 --2- 172.21.2.101:0/2767429901 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5d780186c0 0x7f5d7801abc0 crc :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:29.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.538 7f5d97fff700 1 -- 172.21.2.101:0/2767429901 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d900a0420 msgr2=0x7f5d900a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:29.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.538 7f5d97fff700 1 --2- 172.21.2.101:0/2767429901 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d900a0420 0x7f5d900a0820 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f5d8400b5b0 tx=0x7f5d8400b950).stop 2021-02-10T11:20:29.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.538 7f5d97fff700 1 -- 172.21.2.101:0/2767429901 shutdown_connections 2021-02-10T11:20:29.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.538 7f5d97fff700 1 --2- 172.21.2.101:0/2767429901 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5d780186c0 0x7f5d7801abc0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:29.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.538 7f5d97fff700 1 --2- 172.21.2.101:0/2767429901 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d90189d00 0x7f5d9018a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:29.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.538 7f5d97fff700 1 --2- 172.21.2.101:0/2767429901 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d9018a720 0x7f5d900b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:29.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.538 7f5d97fff700 1 --2- 172.21.2.101:0/2767429901 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d900a0420 0x7f5d900a0820 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:29.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.538 7f5d97fff700 1 -- 172.21.2.101:0/2767429901 >> 172.21.2.101:0/2767429901 conn(0x7f5d90094e10 msgr2=0x7f5d900a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:29.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.539 7f5d97fff700 1 -- 172.21.2.101:0/2767429901 shutdown_connections 2021-02-10T11:20:29.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:29.539 7f5d97fff700 1 -- 172.21.2.101:0/2767429901 wait complete. 2021-02-10T11:20:29.553 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839273 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839273/f40693f7-6185-4224-be75-bb28efddf11a 2021-02-10T11:20:29.553 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:29.553 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:29.554 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:29.891 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:29.891 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:29.892 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839273/f40693f7-6185-4224-be75-bb28efddf11a/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:30.273 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:30.274 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:30.274 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252963 s, 41.5 MB/s 2021-02-10T11:20:30.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:30.275 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:30.276 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:30.611 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:30.612 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:30.612 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839273/f40693f7-6185-4224-be75-bb28efddf11a/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:30.987 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:30.988 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:30.988 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245304 s, 42.7 MB/s 2021-02-10T11:20:30.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:30.989 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:30.989 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:31.325 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:31.325 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:31.325 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839273/f40693f7-6185-4224-be75-bb28efddf11a/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:31.702 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:31.703 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:31.703 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0254469 s, 41.2 MB/s 2021-02-10T11:20:31.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:31.704 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:31.704 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:32.039 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:32.039 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:32.039 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839273/f40693f7-6185-4224-be75-bb28efddf11a/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:32.414 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:32.414 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:32.414 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231782 s, 45.2 MB/s 2021-02-10T11:20:32.414 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:32.415 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:32.415 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:32.750 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:32.751 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:32.751 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839273/f40693f7-6185-4224-be75-bb28efddf11a/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:33.133 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:33.133 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:33.133 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249313 s, 42.1 MB/s 2021-02-10T11:20:33.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:33.134 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:33.134 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:33.469 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:33.470 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:33.470 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839273/f40693f7-6185-4224-be75-bb28efddf11a/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:33.840 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:33.840 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:33.840 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023027 s, 45.5 MB/s 2021-02-10T11:20:33.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:33.841 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:33.841 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:34.177 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:34.177 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:34.178 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839273/f40693f7-6185-4224-be75-bb28efddf11a/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:34.553 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:34.553 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:34.553 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024871 s, 42.2 MB/s 2021-02-10T11:20:34.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:34.554 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:34.554 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:34.889 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:34.890 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:34.890 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839273/f40693f7-6185-4224-be75-bb28efddf11a/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:35.263 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:35.263 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:35.263 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0230494 s, 45.5 MB/s 2021-02-10T11:20:35.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:35.265 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:35.265 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:35.601 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:35.602 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:35.602 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839273/f40693f7-6185-4224-be75-bb28efddf11a/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:35.978 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:35.978 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:35.978 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245091 s, 42.8 MB/s 2021-02-10T11:20:35.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:35.979 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:35.979 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:36.315 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:36.315 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:36.315 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839273/f40693f7-6185-4224-be75-bb28efddf11a/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:36.692 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:36.693 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:36.693 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252856 s, 41.5 MB/s 2021-02-10T11:20:36.694 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839274 2021-02-10T11:20:37.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.100 7f9f9fe1c700 1 Processor -- start 2021-02-10T11:20:37.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.100 7f9f9fe1c700 1 -- start start 2021-02-10T11:20:37.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.101 7f9f9fe1c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9f980f05f0 0x7f9f980f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:37.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.101 7f9f9fe1c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f981b0c70 con 0x7f9f981b0870 2021-02-10T11:20:37.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.101 7f9f9fe1c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f980f0b50 con 0x7f9f981b0470 2021-02-10T11:20:37.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.101 7f9f9fe1c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f980e5ea0 con 0x7f9f980f05f0 2021-02-10T11:20:37.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.101 7f9f9e439700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9f980f05f0 0x7f9f980f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:37.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.101 7f9f9d437700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f9f981b0870 0x7f9f980efe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47176/0 (socket says 172.21.2.101:47176) 2021-02-10T11:20:37.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.101 7f9f9d437700 1 -- 172.21.2.101:0/3055326266 learned_addr learned my addr 172.21.2.101:0/3055326266 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:20:37.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.101 7f9f9e439700 1 -- 172.21.2.101:0/3055326266 >> v1:172.21.2.108:6789/0 conn(0x7f9f981b0470 legacy=0x7f9f980ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:20:37.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.101 7f9f9e439700 1 -- 172.21.2.101:0/3055326266 >> v1:172.21.2.101:6789/0 conn(0x7f9f981b0870 legacy=0x7f9f980efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:37.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.101 7f9f9cc36700 1 -- 172.21.2.101:0/3055326266 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769763308 0 0) 0x7f9f94000ce0 con 0x7f9f981b0870 2021-02-10T11:20:37.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.101 7f9f9e439700 1 -- 172.21.2.101:0/3055326266 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f980e52e0 con 0x7f9f980f05f0 2021-02-10T11:20:37.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.101 7f9f9e439700 1 --2- 172.21.2.101:0/3055326266 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f980f05f0 0x7f9f980f20e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f9f90006e30 tx=0x7f9f90009de0).ready entity=mon.0 client_cookie=ab7697a73fe798e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:37.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.101 7f9f9cc36700 1 -- 172.21.2.101:0/3055326266 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9f9000b300 con 0x7f9f980f05f0 2021-02-10T11:20:37.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.101 7f9f9cc36700 1 -- 172.21.2.101:0/3055326266 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9000b680 con 0x7f9f980f05f0 2021-02-10T11:20:37.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.101 7f9f9cc36700 1 -- 172.21.2.101:0/3055326266 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9f9000bae0 con 0x7f9f980f05f0 2021-02-10T11:20:37.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.102 7f9f9fe1c700 1 -- 172.21.2.101:0/3055326266 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f980f05f0 msgr2=0x7f9f980f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:37.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.102 7f9f9fe1c700 1 --2- 172.21.2.101:0/3055326266 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f980f05f0 0x7f9f980f20e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f9f90006e30 tx=0x7f9f90009de0).stop 2021-02-10T11:20:37.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.102 7f9f9fe1c700 1 -- 172.21.2.101:0/3055326266 shutdown_connections 2021-02-10T11:20:37.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.102 7f9f9fe1c700 1 --2- 172.21.2.101:0/3055326266 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f980f05f0 0x7f9f980f20e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:37.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.102 7f9f9fe1c700 1 -- 172.21.2.101:0/3055326266 >> 172.21.2.101:0/3055326266 conn(0x7f9f980ce3e0 msgr2=0x7f9f980e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:37.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.102 7f9f9fe1c700 1 -- 172.21.2.101:0/3055326266 shutdown_connections 2021-02-10T11:20:37.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.102 7f9f9fe1c700 1 -- 172.21.2.101:0/3055326266 wait complete. 2021-02-10T11:20:37.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.117 7f9f9fe1c700 1 Processor -- start 2021-02-10T11:20:37.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.117 7f9f9fe1c700 1 -- start start 2021-02-10T11:20:37.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.117 7f9f9fe1c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f980a1dd0 0x7f9f98190920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:37.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.117 7f9f9fe1c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f9808fd30 0x7f9f980effc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:37.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.117 7f9f9fe1c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f980f0e90 0x7f9f980f1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:37.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.117 7f9f9fe1c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980f06f0 con 0x7f9f9808fd30 2021-02-10T11:20:37.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.117 7f9f9fe1c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f98090160 con 0x7f9f980a1dd0 2021-02-10T11:20:37.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.117 7f9f9fe1c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980f18b0 con 0x7f9f980f0e90 2021-02-10T11:20:37.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.118 7f9f9d437700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f9808fd30 0x7f9f980effc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:37.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.118 7f9f9d437700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f9808fd30 0x7f9f980effc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46692/0 (socket says 172.21.2.101:46692) 2021-02-10T11:20:37.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.118 7f9f9d437700 1 -- 172.21.2.101:0/687665617 learned_addr learned my addr 172.21.2.101:0/687665617 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:20:37.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.118 7f9f9e439700 1 --2- 172.21.2.101:0/687665617 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f980f0e90 0x7f9f980f1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:37.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.118 7f9f9dc38700 1 --2- 172.21.2.101:0/687665617 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f980a1dd0 0x7f9f98190920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:37.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.118 7f9f9d437700 1 -- 172.21.2.101:0/687665617 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f980f0e90 msgr2=0x7f9f980f1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:37.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.118 7f9f9d437700 1 --2- 172.21.2.101:0/687665617 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f980f0e90 0x7f9f980f1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:37.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.118 7f9f9d437700 1 -- 172.21.2.101:0/687665617 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f980a1dd0 msgr2=0x7f9f98190920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:37.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.118 7f9f9d437700 1 --2- 172.21.2.101:0/687665617 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f980a1dd0 0x7f9f98190920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:37.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.118 7f9f9d437700 1 -- 172.21.2.101:0/687665617 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f980f1ad0 con 0x7f9f9808fd30 2021-02-10T11:20:37.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.118 7f9f9d437700 1 --2- 172.21.2.101:0/687665617 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f9808fd30 0x7f9f980effc0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f9f9400a590 tx=0x7f9f9400a930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:37.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.118 7f9f8e7fc700 1 -- 172.21.2.101:0/687665617 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9f9400ad20 con 0x7f9f9808fd30 2021-02-10T11:20:37.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.118 7f9f8e7fc700 1 -- 172.21.2.101:0/687665617 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f940043a0 con 0x7f9f9808fd30 2021-02-10T11:20:37.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.118 7f9f8e7fc700 1 -- 172.21.2.101:0/687665617 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9f94004770 con 0x7f9f9808fd30 2021-02-10T11:20:37.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.118 7f9f9fe1c700 1 -- 172.21.2.101:0/687665617 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f980f1ad0 con 0x7f9f9808fd30 2021-02-10T11:20:37.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.118 7f9f9fe1c700 1 -- 172.21.2.101:0/687665617 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f980f1ad0 con 0x7f9f9808fd30 2021-02-10T11:20:37.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.119 7f9f8e7fc700 1 -- 172.21.2.101:0/687665617 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9f94004580 con 0x7f9f9808fd30 2021-02-10T11:20:37.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.119 7f9f8e7fc700 1 --2- 172.21.2.101:0/687665617 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f800186c0 0x7f9f8001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:37.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.119 7f9f8e7fc700 1 -- 172.21.2.101:0/687665617 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9f94032430 con 0x7f9f9808fd30 2021-02-10T11:20:37.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.119 7f9f9dc38700 1 --2- 172.21.2.101:0/687665617 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f800186c0 0x7f9f8001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:37.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.119 7f9f9dc38700 1 --2- 172.21.2.101:0/687665617 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f800186c0 0x7f9f8001abc0 crc :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:37.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.150 7f9f9fe1c700 1 -- 172.21.2.101:0/687665617 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f980e52e0 con 0x7f9f9808fd30 2021-02-10T11:20:37.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.153 7f9f8e7fc700 1 -- 172.21.2.101:0/687665617 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9f94007070 con 0x7f9f9808fd30 2021-02-10T11:20:37.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.279 7f9f9fe1c700 1 -- 172.21.2.101:0/687665617 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839274", "target": ["mgr", ""]}) v1 -- 0x7f9f980f23d0 con 0x7f9f800186c0 2021-02-10T11:20:37.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.303 7f9f8e7fc700 1 -- 172.21.2.101:0/687665617 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f9f88000960 con 0x7f9f800186c0 2021-02-10T11:20:37.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.312 7f9f9fe1c700 1 -- 172.21.2.101:0/687665617 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f800186c0 msgr2=0x7f9f8001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:37.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.312 7f9f9fe1c700 1 --2- 172.21.2.101:0/687665617 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f800186c0 0x7f9f8001abc0 crc :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:37.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.312 7f9f9fe1c700 1 -- 172.21.2.101:0/687665617 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f9808fd30 msgr2=0x7f9f980effc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:37.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.312 7f9f9fe1c700 1 --2- 172.21.2.101:0/687665617 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f9808fd30 0x7f9f980effc0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f9f9400a590 tx=0x7f9f9400a930).stop 2021-02-10T11:20:37.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.312 7f9f9fe1c700 1 -- 172.21.2.101:0/687665617 shutdown_connections 2021-02-10T11:20:37.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.312 7f9f9fe1c700 1 --2- 172.21.2.101:0/687665617 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f800186c0 0x7f9f8001abc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:37.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.312 7f9f9fe1c700 1 --2- 172.21.2.101:0/687665617 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f980f0e90 0x7f9f980f1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:37.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.312 7f9f9fe1c700 1 --2- 172.21.2.101:0/687665617 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f9808fd30 0x7f9f980effc0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:37.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.312 7f9f9fe1c700 1 --2- 172.21.2.101:0/687665617 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f980a1dd0 0x7f9f98190920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:37.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.312 7f9f9fe1c700 1 -- 172.21.2.101:0/687665617 >> 172.21.2.101:0/687665617 conn(0x7f9f980958b0 msgr2=0x7f9f980bf040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:37.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.312 7f9f9fe1c700 1 -- 172.21.2.101:0/687665617 shutdown_connections 2021-02-10T11:20:37.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.312 7f9f9fe1c700 1 -- 172.21.2.101:0/687665617 wait complete. 2021-02-10T11:20:37.326 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839274 2021-02-10T11:20:37.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.733 7f135e339700 1 Processor -- start 2021-02-10T11:20:37.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.733 7f135e339700 1 -- start start 2021-02-10T11:20:37.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.734 7f135e339700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f13581b0870 0x7f13580f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:37.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.734 7f135e339700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13581b0c70 con 0x7f13580efe00 2021-02-10T11:20:37.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.734 7f135e339700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13580f0b50 con 0x7f13581b0470 2021-02-10T11:20:37.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.734 7f135e339700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13580e5ea0 con 0x7f13581b0870 2021-02-10T11:20:37.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.734 7f135c956700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f13580efe00 0x7f13580f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47192/0 (socket says 172.21.2.101:47192) 2021-02-10T11:20:37.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.734 7f135c956700 1 -- 172.21.2.101:0/3837374163 learned_addr learned my addr 172.21.2.101:0/3837374163 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:20:37.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.734 7f13577fe700 1 --2- 172.21.2.101:0/3837374163 >> v2:172.21.2.108:3300/0 conn(0x7f13581b0870 0x7f13580f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:37.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.734 7f1356ffd700 1 -- 172.21.2.101:0/3837374163 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722086529 0 0) 0x7f1350001500 con 0x7f13580efe00 2021-02-10T11:20:37.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.734 7f1356ffd700 1 -- 172.21.2.101:0/3837374163 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1340002fe0 con 0x7f13580efe00 2021-02-10T11:20:37.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.734 7f1356ffd700 1 -- 172.21.2.101:0/3837374163 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 289280158 0 0) 0x7f1348000ff0 con 0x7f13581b0470 2021-02-10T11:20:37.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.734 7f1356ffd700 1 -- 172.21.2.101:0/3837374163 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1340003210 con 0x7f13581b0470 2021-02-10T11:20:37.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.735 7f13577fe700 1 -- 172.21.2.101:0/3837374163 >> v1:172.21.2.108:6789/0 conn(0x7f13581b0470 legacy=0x7f13580ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:37.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.735 7f13577fe700 1 -- 172.21.2.101:0/3837374163 >> v1:172.21.2.101:6789/0 conn(0x7f13580efe00 legacy=0x7f13580f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:37.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.735 7f13577fe700 1 -- 172.21.2.101:0/3837374163 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13580e52e0 con 0x7f13581b0870 2021-02-10T11:20:37.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.735 7f13577fe700 1 --2- 172.21.2.101:0/3837374163 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f13581b0870 0x7f13580f20e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f134c006650 tx=0x7f134c009a00).ready entity=mon.1 client_cookie=7df5ca552f9806aa server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:37.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.735 7f1356ffd700 1 -- 172.21.2.101:0/3837374163 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f134c0009b0 con 0x7f13581b0870 2021-02-10T11:20:37.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.735 7f1356ffd700 1 -- 172.21.2.101:0/3837374163 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134c00c0c0 con 0x7f13581b0870 2021-02-10T11:20:37.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.735 7f1356ffd700 1 -- 172.21.2.101:0/3837374163 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f134c00c5e0 con 0x7f13581b0870 2021-02-10T11:20:37.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.735 7f135e339700 1 -- 172.21.2.101:0/3837374163 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f13581b0870 msgr2=0x7f13580f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:37.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.735 7f135e339700 1 --2- 172.21.2.101:0/3837374163 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f13581b0870 0x7f13580f20e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f134c006650 tx=0x7f134c009a00).stop 2021-02-10T11:20:37.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.736 7f135e339700 1 -- 172.21.2.101:0/3837374163 shutdown_connections 2021-02-10T11:20:37.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.736 7f135e339700 1 --2- 172.21.2.101:0/3837374163 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f13581b0870 0x7f13580f20e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:37.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.736 7f135e339700 1 -- 172.21.2.101:0/3837374163 >> 172.21.2.101:0/3837374163 conn(0x7f13580ce3e0 msgr2=0x7f13580e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:37.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.736 7f135e339700 1 -- 172.21.2.101:0/3837374163 shutdown_connections 2021-02-10T11:20:37.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.736 7f135e339700 1 -- 172.21.2.101:0/3837374163 wait complete. 2021-02-10T11:20:37.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.750 7f135e339700 1 Processor -- start 2021-02-10T11:20:37.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.750 7f135e339700 1 -- start start 2021-02-10T11:20:37.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.751 7f135e339700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1358190920 0x7f13580b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:37.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.751 7f135e339700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13580b8200 0x7f13580b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:37.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.751 7f135e339700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f13580c1d80 0x7f13580c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:37.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.751 7f135e339700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1358190d20 con 0x7f13580b8200 2021-02-10T11:20:37.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.751 7f135e339700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13580b8c20 con 0x7f13580c1d80 2021-02-10T11:20:37.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.751 7f135e339700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13580c27a0 con 0x7f1358190920 2021-02-10T11:20:37.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.751 7f13577fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13580b8200 0x7f13580b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:37.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.751 7f13577fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13580b8200 0x7f13580b8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46702/0 (socket says 172.21.2.101:46702) 2021-02-10T11:20:37.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.751 7f13577fe700 1 -- 172.21.2.101:0/1132311644 learned_addr learned my addr 172.21.2.101:0/1132311644 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:20:37.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.751 7f1357fff700 1 --2- 172.21.2.101:0/1132311644 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1358190920 0x7f13580b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:37.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.751 7f135c956700 1 --2- 172.21.2.101:0/1132311644 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f13580c1d80 0x7f13580c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:37.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.751 7f13577fe700 1 -- 172.21.2.101:0/1132311644 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1358190920 msgr2=0x7f13580b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:37.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.751 7f13577fe700 1 --2- 172.21.2.101:0/1132311644 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1358190920 0x7f13580b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:37.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.751 7f13577fe700 1 -- 172.21.2.101:0/1132311644 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f13580c1d80 msgr2=0x7f13580c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:37.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.751 7f13577fe700 1 --2- 172.21.2.101:0/1132311644 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f13580c1d80 0x7f13580c2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:37.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.751 7f13577fe700 1 -- 172.21.2.101:0/1132311644 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13580c29c0 con 0x7f13580b8200 2021-02-10T11:20:37.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.752 7f13577fe700 1 --2- 172.21.2.101:0/1132311644 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13580b8200 0x7f13580b8600 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f134c0096d0 tx=0x7f134c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:37.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.752 7f1354ff9700 1 -- 172.21.2.101:0/1132311644 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f134c002030 con 0x7f13580b8200 2021-02-10T11:20:37.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.752 7f1354ff9700 1 -- 172.21.2.101:0/1132311644 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134c000940 con 0x7f13580b8200 2021-02-10T11:20:37.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.752 7f1354ff9700 1 -- 172.21.2.101:0/1132311644 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f134c002570 con 0x7f13580b8200 2021-02-10T11:20:37.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.752 7f135e339700 1 -- 172.21.2.101:0/1132311644 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13580c29c0 con 0x7f13580b8200 2021-02-10T11:20:37.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.752 7f135e339700 1 -- 172.21.2.101:0/1132311644 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13580c2f50 con 0x7f13580b8200 2021-02-10T11:20:37.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.753 7f1354ff9700 1 -- 172.21.2.101:0/1132311644 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f134c000c90 con 0x7f13580b8200 2021-02-10T11:20:37.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.753 7f1354ff9700 1 --2- 172.21.2.101:0/1132311644 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13380186c0 0x7f133801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:37.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.753 7f1354ff9700 1 -- 172.21.2.101:0/1132311644 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f134c032550 con 0x7f13580b8200 2021-02-10T11:20:37.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.753 7f1357fff700 1 --2- 172.21.2.101:0/1132311644 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13380186c0 0x7f133801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:37.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.753 7f1357fff700 1 --2- 172.21.2.101:0/1132311644 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13380186c0 0x7f133801abc0 crc :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:37.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.783 7f135e339700 1 -- 172.21.2.101:0/1132311644 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1358105ae0 con 0x7f13580b8200 2021-02-10T11:20:37.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.786 7f1354ff9700 1 -- 172.21.2.101:0/1132311644 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f134c022520 con 0x7f13580b8200 2021-02-10T11:20:37.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.913 7f135e339700 1 -- 172.21.2.101:0/1132311644 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839274", "target": ["mgr", ""]}) v1 -- 0x7f1358190d20 con 0x7f13380186c0 2021-02-10T11:20:37.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.916 7f1354ff9700 1 -- 172.21.2.101:0/1132311644 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f1348006810 con 0x7f13380186c0 2021-02-10T11:20:37.923 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839274/1fe44bb7-04d9-488f-aefa-fdedf1026332 2021-02-10T11:20:37.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.918 7f135e339700 1 -- 172.21.2.101:0/1132311644 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13380186c0 msgr2=0x7f133801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:37.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.918 7f135e339700 1 --2- 172.21.2.101:0/1132311644 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13380186c0 0x7f133801abc0 crc :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:37.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.918 7f135e339700 1 -- 172.21.2.101:0/1132311644 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13580b8200 msgr2=0x7f13580b8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:37.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.918 7f135e339700 1 --2- 172.21.2.101:0/1132311644 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13580b8200 0x7f13580b8600 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f134c0096d0 tx=0x7f134c00b990).stop 2021-02-10T11:20:37.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.918 7f135e339700 1 -- 172.21.2.101:0/1132311644 shutdown_connections 2021-02-10T11:20:37.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.918 7f135e339700 1 --2- 172.21.2.101:0/1132311644 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f13580c1d80 0x7f13580c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:37.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.918 7f135e339700 1 --2- 172.21.2.101:0/1132311644 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13580b8200 0x7f13580b8600 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:37.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.918 7f135e339700 1 --2- 172.21.2.101:0/1132311644 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13380186c0 0x7f133801abc0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:37.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.918 7f135e339700 1 --2- 172.21.2.101:0/1132311644 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1358190920 0x7f13580b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:37.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.918 7f135e339700 1 -- 172.21.2.101:0/1132311644 >> 172.21.2.101:0/1132311644 conn(0x7f1358094e10 msgr2=0x7f13580a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:37.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.918 7f135e339700 1 -- 172.21.2.101:0/1132311644 shutdown_connections 2021-02-10T11:20:37.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:37.918 7f135e339700 1 -- 172.21.2.101:0/1132311644 wait complete. 2021-02-10T11:20:37.933 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839274 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839274/1fe44bb7-04d9-488f-aefa-fdedf1026332 2021-02-10T11:20:37.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:37.933 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:37.933 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:38.273 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:38.273 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:38.273 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839274/1fe44bb7-04d9-488f-aefa-fdedf1026332/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:38.670 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:38.671 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:38.671 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0256712 s, 40.8 MB/s 2021-02-10T11:20:38.672 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:38.672 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:38.672 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:39.014 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:39.014 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:39.014 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839274/1fe44bb7-04d9-488f-aefa-fdedf1026332/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:39.390 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:39.390 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:39.391 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236761 s, 44.3 MB/s 2021-02-10T11:20:39.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:39.392 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:39.393 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:39.731 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:39.732 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:39.732 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839274/1fe44bb7-04d9-488f-aefa-fdedf1026332/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249907 s, 42.0 MB/s 2021-02-10T11:20:40.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:40.111 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:40.111 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:40.445 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:40.445 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:40.445 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839274/1fe44bb7-04d9-488f-aefa-fdedf1026332/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:40.820 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:40.821 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:40.821 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233717 s, 44.9 MB/s 2021-02-10T11:20:40.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:40.822 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:40.822 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:41.157 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:41.157 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:41.158 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839274/1fe44bb7-04d9-488f-aefa-fdedf1026332/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:41.535 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:41.536 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:41.536 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251762 s, 41.6 MB/s 2021-02-10T11:20:41.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:41.537 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:41.537 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:41.871 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:41.872 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:41.872 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839274/1fe44bb7-04d9-488f-aefa-fdedf1026332/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:42.249 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:42.250 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:42.250 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.025935 s, 40.4 MB/s 2021-02-10T11:20:42.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:42.251 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:42.251 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:42.587 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:42.587 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:42.587 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839274/1fe44bb7-04d9-488f-aefa-fdedf1026332/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:42.962 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:42.963 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:42.963 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239879 s, 43.7 MB/s 2021-02-10T11:20:42.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:42.964 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:42.964 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:43.300 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:43.301 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:43.301 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839274/1fe44bb7-04d9-488f-aefa-fdedf1026332/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:43.675 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:43.676 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:43.676 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238814 s, 43.9 MB/s 2021-02-10T11:20:43.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:43.677 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:43.677 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:44.013 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:44.014 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:44.014 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839274/1fe44bb7-04d9-488f-aefa-fdedf1026332/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:44.387 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:44.387 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:44.387 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246624 s, 42.5 MB/s 2021-02-10T11:20:44.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:44.388 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:44.388 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:44.723 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:44.724 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:44.724 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839274/1fe44bb7-04d9-488f-aefa-fdedf1026332/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:45.099 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:45.100 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:45.100 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235687 s, 44.5 MB/s 2021-02-10T11:20:45.101 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839275 2021-02-10T11:20:45.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.507 7fcd3f212700 1 Processor -- start 2021-02-10T11:20:45.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.507 7fcd3f212700 1 -- start start 2021-02-10T11:20:45.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.508 7fcd3f212700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fcd381b0470 0x7fcd381b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:45.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.508 7fcd3f212700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcd380ef610 0x7fcd380efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:45.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.508 7fcd3f212700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd380f0b50 con 0x7fcd380f0030 2021-02-10T11:20:45.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.508 7fcd3f212700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd380e52e0 con 0x7fcd380ef610 2021-02-10T11:20:45.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.508 7fcd3f212700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd380e5490 con 0x7fcd381b0470 2021-02-10T11:20:45.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.508 7fcd3c82d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcd380ef610 0x7fcd380efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:45.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.508 7fcd3d82f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fcd380f0030 0x7fcd380f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47202/0 (socket says 172.21.2.101:47202) 2021-02-10T11:20:45.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.508 7fcd3d82f700 1 -- 172.21.2.101:0/644509607 learned_addr learned my addr 172.21.2.101:0/644509607 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:20:45.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.508 7fcd2bfff700 1 -- 172.21.2.101:0/644509607 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2162426472 0 0) 0x7fcd30001500 con 0x7fcd380f0030 2021-02-10T11:20:45.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.508 7fcd3c82d700 1 -- 172.21.2.101:0/644509607 >> v2:172.21.2.108:3300/0 conn(0x7fcd381b0470 msgr2=0x7fcd381b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:20:45.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.508 7fcd3c82d700 1 --2- 172.21.2.101:0/644509607 >> v2:172.21.2.108:3300/0 conn(0x7fcd381b0470 0x7fcd381b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:45.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.508 7fcd3c82d700 1 -- 172.21.2.101:0/644509607 >> v1:172.21.2.101:6789/0 conn(0x7fcd380f0030 legacy=0x7fcd380f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:45.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.508 7fcd3c82d700 1 -- 172.21.2.101:0/644509607 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd380f0430 con 0x7fcd380ef610 2021-02-10T11:20:45.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.508 7fcd3c82d700 1 --2- 172.21.2.101:0/644509607 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd380ef610 0x7fcd380efa10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fcd34006650 tx=0x7fcd34009a80).ready entity=mon.0 client_cookie=fa37a6bdce6d3fb8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:45.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.508 7fcd2bfff700 1 -- 172.21.2.101:0/644509607 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd340009b0 con 0x7fcd380ef610 2021-02-10T11:20:45.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.508 7fcd2bfff700 1 -- 172.21.2.101:0/644509607 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3400c0c0 con 0x7fcd380ef610 2021-02-10T11:20:45.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.508 7fcd2bfff700 1 -- 172.21.2.101:0/644509607 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd3400c520 con 0x7fcd380ef610 2021-02-10T11:20:45.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.508 7fcd3f212700 1 -- 172.21.2.101:0/644509607 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd380ef610 msgr2=0x7fcd380efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:45.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.508 7fcd3f212700 1 --2- 172.21.2.101:0/644509607 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd380ef610 0x7fcd380efa10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fcd34006650 tx=0x7fcd34009a80).stop 2021-02-10T11:20:45.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.509 7fcd3f212700 1 -- 172.21.2.101:0/644509607 shutdown_connections 2021-02-10T11:20:45.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.509 7fcd3f212700 1 --2- 172.21.2.101:0/644509607 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd380ef610 0x7fcd380efa10 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:45.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.509 7fcd3f212700 1 --2- 172.21.2.101:0/644509607 >> v2:172.21.2.108:3300/0 conn(0x7fcd381b0470 0x7fcd381b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:45.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.509 7fcd3f212700 1 -- 172.21.2.101:0/644509607 >> 172.21.2.101:0/644509607 conn(0x7fcd380ce3e0 msgr2=0x7fcd380e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:45.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.509 7fcd3f212700 1 -- 172.21.2.101:0/644509607 shutdown_connections 2021-02-10T11:20:45.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.509 7fcd3f212700 1 -- 172.21.2.101:0/644509607 wait complete. 2021-02-10T11:20:45.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.524 7fcd3f212700 1 Processor -- start 2021-02-10T11:20:45.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.524 7fcd3f212700 1 -- start start 2021-02-10T11:20:45.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.524 7fcd3f212700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd380c07e0 0x7fcd380988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:45.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.524 7fcd3f212700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd380aceb0 0x7fcd38098f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:45.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.524 7fcd3f212700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd380eec20 0x7fcd380ef020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:45.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.524 7fcd3f212700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd38199f00 con 0x7fcd380eec20 2021-02-10T11:20:45.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.524 7fcd3f212700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd380c0cc0 con 0x7fcd380aceb0 2021-02-10T11:20:45.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.524 7fcd3f212700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd38095240 con 0x7fcd380c07e0 2021-02-10T11:20:45.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.525 7fcd3d82f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd380eec20 0x7fcd380ef020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:45.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.525 7fcd3d82f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd380eec20 0x7fcd380ef020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46716/0 (socket says 172.21.2.101:46716) 2021-02-10T11:20:45.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.525 7fcd3d82f700 1 -- 172.21.2.101:0/1029332957 learned_addr learned my addr 172.21.2.101:0/1029332957 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:20:45.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.525 7fcd3d82f700 1 -- 172.21.2.101:0/1029332957 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd380c07e0 msgr2=0x7fcd380988f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:20:45.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.525 7fcd3d82f700 1 --2- 172.21.2.101:0/1029332957 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd380c07e0 0x7fcd380988f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:45.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.525 7fcd3d82f700 1 -- 172.21.2.101:0/1029332957 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd380aceb0 msgr2=0x7fcd38098f10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:20:45.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.525 7fcd3d82f700 1 --2- 172.21.2.101:0/1029332957 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd380aceb0 0x7fcd38098f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:45.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.525 7fcd3d82f700 1 -- 172.21.2.101:0/1029332957 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd380ef640 con 0x7fcd380eec20 2021-02-10T11:20:45.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.525 7fcd3d82f700 1 --2- 172.21.2.101:0/1029332957 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd380eec20 0x7fcd380ef020 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fcd30001bf0 tx=0x7fcd3000ac50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:45.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.525 7fcd29ffb700 1 -- 172.21.2.101:0/1029332957 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd300070d0 con 0x7fcd380eec20 2021-02-10T11:20:45.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.525 7fcd29ffb700 1 -- 172.21.2.101:0/1029332957 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd30007390 con 0x7fcd380eec20 2021-02-10T11:20:45.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.525 7fcd3f212700 1 -- 172.21.2.101:0/1029332957 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd380ef640 con 0x7fcd380eec20 2021-02-10T11:20:45.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.525 7fcd3f212700 1 -- 172.21.2.101:0/1029332957 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd38199f00 con 0x7fcd380eec20 2021-02-10T11:20:45.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.525 7fcd29ffb700 1 -- 172.21.2.101:0/1029332957 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd30004720 con 0x7fcd380eec20 2021-02-10T11:20:45.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.526 7fcd29ffb700 1 -- 172.21.2.101:0/1029332957 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fcd30007570 con 0x7fcd380eec20 2021-02-10T11:20:45.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.526 7fcd29ffb700 1 --2- 172.21.2.101:0/1029332957 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcd180186c0 0x7fcd1801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:45.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.526 7fcd29ffb700 1 -- 172.21.2.101:0/1029332957 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fcd30032af0 con 0x7fcd380eec20 2021-02-10T11:20:45.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.526 7fcd3d02e700 1 --2- 172.21.2.101:0/1029332957 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcd180186c0 0x7fcd1801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:45.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.526 7fcd3d02e700 1 --2- 172.21.2.101:0/1029332957 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcd180186c0 0x7fcd1801abc0 crc :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:45.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.557 7fcd3f212700 1 -- 172.21.2.101:0/1029332957 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd380edd10 con 0x7fcd380eec20 2021-02-10T11:20:45.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.560 7fcd29ffb700 1 -- 172.21.2.101:0/1029332957 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fcd300014e0 con 0x7fcd380eec20 2021-02-10T11:20:45.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.688 7fcd3f212700 1 -- 172.21.2.101:0/1029332957 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839275", "target": ["mgr", ""]}) v1 -- 0x7fcd38095240 con 0x7fcd180186c0 2021-02-10T11:20:45.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.713 7fcd29ffb700 1 -- 172.21.2.101:0/1029332957 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fcd2c006960 con 0x7fcd180186c0 2021-02-10T11:20:45.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.721 7fcd3f212700 1 -- 172.21.2.101:0/1029332957 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcd180186c0 msgr2=0x7fcd1801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:45.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.721 7fcd3f212700 1 --2- 172.21.2.101:0/1029332957 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcd180186c0 0x7fcd1801abc0 crc :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:45.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.721 7fcd3f212700 1 -- 172.21.2.101:0/1029332957 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd380eec20 msgr2=0x7fcd380ef020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:45.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.721 7fcd3f212700 1 --2- 172.21.2.101:0/1029332957 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd380eec20 0x7fcd380ef020 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fcd30001bf0 tx=0x7fcd3000ac50).stop 2021-02-10T11:20:45.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.721 7fcd3f212700 1 -- 172.21.2.101:0/1029332957 shutdown_connections 2021-02-10T11:20:45.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.721 7fcd3f212700 1 --2- 172.21.2.101:0/1029332957 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd380eec20 0x7fcd380ef020 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:45.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.721 7fcd3f212700 1 --2- 172.21.2.101:0/1029332957 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd380aceb0 0x7fcd38098f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:45.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.721 7fcd3f212700 1 --2- 172.21.2.101:0/1029332957 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcd180186c0 0x7fcd1801abc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:45.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.721 7fcd3f212700 1 --2- 172.21.2.101:0/1029332957 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd380c07e0 0x7fcd380988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:45.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.721 7fcd3f212700 1 -- 172.21.2.101:0/1029332957 >> 172.21.2.101:0/1029332957 conn(0x7fcd380c40e0 msgr2=0x7fcd380f22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:45.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.721 7fcd3f212700 1 -- 172.21.2.101:0/1029332957 shutdown_connections 2021-02-10T11:20:45.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:45.721 7fcd3f212700 1 -- 172.21.2.101:0/1029332957 wait complete. 2021-02-10T11:20:45.736 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839275 2021-02-10T11:20:46.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.143 7f8d7c17d700 1 Processor -- start 2021-02-10T11:20:46.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.143 7f8d7c17d700 1 -- start start 2021-02-10T11:20:46.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.143 7f8d7c17d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d741b0470 0x7f8d741b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:46.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.143 7f8d7c17d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8d740f20e0 0x7f8d740f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:46.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.143 7f8d7c17d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d740f0b50 con 0x7f8d740ef610 2021-02-10T11:20:46.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.143 7f8d7c17d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d740f0820 con 0x7f8d740f20e0 2021-02-10T11:20:46.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.143 7f8d7c17d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d740f24e0 con 0x7f8d741b0470 2021-02-10T11:20:46.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.143 7f8d79798700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8d740ef610 0x7f8d740efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47214/0 (socket says 172.21.2.101:47214) 2021-02-10T11:20:46.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.143 7f8d79798700 1 -- 172.21.2.101:0/3624166856 learned_addr learned my addr 172.21.2.101:0/3624166856 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:20:46.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.143 7f8d79f99700 1 --2- 172.21.2.101:0/3624166856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d741b0470 0x7f8d741b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:46.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.144 7f8d7a79a700 1 --2- 172.21.2.101:0/3624166856 >> v2:172.21.2.108:3300/0 conn(0x7f8d740f20e0 0x7f8d740f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:46.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.144 7f8d78f97700 1 -- 172.21.2.101:0/3624166856 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2930052442 0 0) 0x7f8d680009b0 con 0x7f8d740ef610 2021-02-10T11:20:46.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.144 7f8d78f97700 1 -- 172.21.2.101:0/3624166856 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d60002fe0 con 0x7f8d740ef610 2021-02-10T11:20:46.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.144 7f8d79f99700 1 -- 172.21.2.101:0/3624166856 >> v2:172.21.2.108:3300/0 conn(0x7f8d740f20e0 msgr2=0x7f8d740f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:20:46.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.144 7f8d79f99700 1 --2- 172.21.2.101:0/3624166856 >> v2:172.21.2.108:3300/0 conn(0x7f8d740f20e0 0x7f8d740f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:46.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.144 7f8d79f99700 1 -- 172.21.2.101:0/3624166856 >> v1:172.21.2.101:6789/0 conn(0x7f8d740ef610 legacy=0x7f8d740efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:46.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.144 7f8d79f99700 1 -- 172.21.2.101:0/3624166856 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d740e52e0 con 0x7f8d741b0470 2021-02-10T11:20:46.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.144 7f8d79f99700 1 --2- 172.21.2.101:0/3624166856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d741b0470 0x7f8d741b0870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f8d70006650 tx=0x7f8d70009a80).ready entity=mon.2 client_cookie=41b377b7542489e3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:46.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.144 7f8d78f97700 1 -- 172.21.2.101:0/3624166856 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d7000c0c0 con 0x7f8d741b0470 2021-02-10T11:20:46.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.144 7f8d78f97700 1 -- 172.21.2.101:0/3624166856 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7000c340 con 0x7f8d741b0470 2021-02-10T11:20:46.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.144 7f8d78f97700 1 -- 172.21.2.101:0/3624166856 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d7000c860 con 0x7f8d741b0470 2021-02-10T11:20:46.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.145 7f8d7c17d700 1 -- 172.21.2.101:0/3624166856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d741b0470 msgr2=0x7f8d741b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:46.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.145 7f8d7c17d700 1 --2- 172.21.2.101:0/3624166856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d741b0470 0x7f8d741b0870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f8d70006650 tx=0x7f8d70009a80).stop 2021-02-10T11:20:46.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.145 7f8d7c17d700 1 -- 172.21.2.101:0/3624166856 shutdown_connections 2021-02-10T11:20:46.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.145 7f8d7c17d700 1 --2- 172.21.2.101:0/3624166856 >> v2:172.21.2.108:3300/0 conn(0x7f8d740f20e0 0x7f8d740f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:46.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.145 7f8d7c17d700 1 --2- 172.21.2.101:0/3624166856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d741b0470 0x7f8d741b0870 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:46.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.145 7f8d7c17d700 1 -- 172.21.2.101:0/3624166856 >> 172.21.2.101:0/3624166856 conn(0x7f8d740ce3e0 msgr2=0x7f8d740e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:46.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.145 7f8d7c17d700 1 -- 172.21.2.101:0/3624166856 shutdown_connections 2021-02-10T11:20:46.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.145 7f8d7c17d700 1 -- 172.21.2.101:0/3624166856 wait complete. 2021-02-10T11:20:46.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.160 7f8d7c17d700 1 Processor -- start 2021-02-10T11:20:46.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.160 7f8d7c17d700 1 -- start start 2021-02-10T11:20:46.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.160 7f8d7c17d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d740b7db0 0x7f8d740b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:46.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.160 7f8d7c17d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d740b8880 0x7f8d741a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:46.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.160 7f8d7c17d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d740b8c80 0x7f8d741a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:46.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.160 7f8d7c17d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d740b79b0 con 0x7f8d740b8880 2021-02-10T11:20:46.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.160 7f8d7c17d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d741a8090 con 0x7f8d740b8c80 2021-02-10T11:20:46.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.160 7f8d7c17d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d741a82b0 con 0x7f8d740b7db0 2021-02-10T11:20:46.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.160 7f8d79798700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d740b8880 0x7f8d741a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:46.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.160 7f8d79f99700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d740b7db0 0x7f8d740b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:46.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.160 7f8d79798700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d740b8880 0x7f8d741a7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46728/0 (socket says 172.21.2.101:46728) 2021-02-10T11:20:46.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.160 7f8d79798700 1 -- 172.21.2.101:0/1775443778 learned_addr learned my addr 172.21.2.101:0/1775443778 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:20:46.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.161 7f8d7a79a700 1 --2- 172.21.2.101:0/1775443778 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d740b8c80 0x7f8d741a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:46.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.161 7f8d79798700 1 -- 172.21.2.101:0/1775443778 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d740b7db0 msgr2=0x7f8d740b8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:46.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.161 7f8d79798700 1 --2- 172.21.2.101:0/1775443778 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d740b7db0 0x7f8d740b8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:46.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.161 7f8d79798700 1 -- 172.21.2.101:0/1775443778 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d740b8c80 msgr2=0x7f8d741a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:46.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.161 7f8d79798700 1 --2- 172.21.2.101:0/1775443778 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d740b8c80 0x7f8d741a7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:46.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.161 7f8d79798700 1 -- 172.21.2.101:0/1775443778 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d74190c10 con 0x7f8d740b8880 2021-02-10T11:20:46.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.161 7f8d7a79a700 1 --2- 172.21.2.101:0/1775443778 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d740b8c80 0x7f8d741a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:20:46.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.161 7f8d79798700 1 --2- 172.21.2.101:0/1775443778 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d740b8880 0x7f8d741a7450 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f8d68001c60 tx=0x7f8d6800a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:46.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.161 7f8d667fc700 1 -- 172.21.2.101:0/1775443778 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d68000e00 con 0x7f8d740b8880 2021-02-10T11:20:46.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.161 7f8d667fc700 1 -- 172.21.2.101:0/1775443778 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6800acc0 con 0x7f8d740b8880 2021-02-10T11:20:46.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.161 7f8d667fc700 1 -- 172.21.2.101:0/1775443778 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d68004540 con 0x7f8d740b8880 2021-02-10T11:20:46.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.161 7f8d7c17d700 1 -- 172.21.2.101:0/1775443778 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d74190c10 con 0x7f8d740b8880 2021-02-10T11:20:46.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.161 7f8d7c17d700 1 -- 172.21.2.101:0/1775443778 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d74190c10 con 0x7f8d740b8880 2021-02-10T11:20:46.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.162 7f8d667fc700 1 -- 172.21.2.101:0/1775443778 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8d68004140 con 0x7f8d740b8880 2021-02-10T11:20:46.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.162 7f8d667fc700 1 --2- 172.21.2.101:0/1775443778 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d5c018730 0x7f8d5c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:46.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.162 7f8d667fc700 1 -- 172.21.2.101:0/1775443778 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8d68032610 con 0x7f8d740b8880 2021-02-10T11:20:46.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.162 7f8d79f99700 1 --2- 172.21.2.101:0/1775443778 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d5c018730 0x7f8d5c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:46.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.162 7f8d79f99700 1 --2- 172.21.2.101:0/1775443778 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d5c018730 0x7f8d5c01ac10 crc :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:46.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.193 7f8d7c17d700 1 -- 172.21.2.101:0/1775443778 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d74105b30 con 0x7f8d740b8880 2021-02-10T11:20:46.202 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.195 7f8d667fc700 1 -- 172.21.2.101:0/1775443778 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8d68000e00 con 0x7f8d740b8880 2021-02-10T11:20:46.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.323 7f8d7c17d700 1 -- 172.21.2.101:0/1775443778 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839275", "target": ["mgr", ""]}) v1 -- 0x7f8d741a8090 con 0x7f8d5c018730 2021-02-10T11:20:46.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.326 7f8d667fc700 1 -- 172.21.2.101:0/1775443778 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f8d70009180 con 0x7f8d5c018730 2021-02-10T11:20:46.333 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839275/111f6ac6-cf6f-485d-a754-93bce8976a32 2021-02-10T11:20:46.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.328 7f8d7c17d700 1 -- 172.21.2.101:0/1775443778 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d5c018730 msgr2=0x7f8d5c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:46.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.328 7f8d7c17d700 1 --2- 172.21.2.101:0/1775443778 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d5c018730 0x7f8d5c01ac10 crc :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:46.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.328 7f8d7c17d700 1 -- 172.21.2.101:0/1775443778 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d740b8880 msgr2=0x7f8d741a7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:46.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.328 7f8d7c17d700 1 --2- 172.21.2.101:0/1775443778 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d740b8880 0x7f8d741a7450 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f8d68001c60 tx=0x7f8d6800a8d0).stop 2021-02-10T11:20:46.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.328 7f8d7c17d700 1 -- 172.21.2.101:0/1775443778 shutdown_connections 2021-02-10T11:20:46.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.328 7f8d7c17d700 1 --2- 172.21.2.101:0/1775443778 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d740b8c80 0x7f8d741a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:46.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.328 7f8d7c17d700 1 --2- 172.21.2.101:0/1775443778 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d740b8880 0x7f8d741a7450 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:46.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.328 7f8d7c17d700 1 --2- 172.21.2.101:0/1775443778 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d5c018730 0x7f8d5c01ac10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:46.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.328 7f8d7c17d700 1 --2- 172.21.2.101:0/1775443778 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d740b7db0 0x7f8d740b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:46.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.328 7f8d7c17d700 1 -- 172.21.2.101:0/1775443778 >> 172.21.2.101:0/1775443778 conn(0x7f8d74094e10 msgr2=0x7f8d740a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:46.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.328 7f8d7c17d700 1 -- 172.21.2.101:0/1775443778 shutdown_connections 2021-02-10T11:20:46.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:46.328 7f8d7c17d700 1 -- 172.21.2.101:0/1775443778 wait complete. 2021-02-10T11:20:46.343 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839275 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839275/111f6ac6-cf6f-485d-a754-93bce8976a32 2021-02-10T11:20:46.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:46.343 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:46.343 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:46.678 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:46.679 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:46.679 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839275/111f6ac6-cf6f-485d-a754-93bce8976a32/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:47.060 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:47.060 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:47.060 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249297 s, 42.1 MB/s 2021-02-10T11:20:47.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:47.062 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:47.062 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:47.396 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:47.397 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:47.397 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839275/111f6ac6-cf6f-485d-a754-93bce8976a32/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:47.773 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:47.773 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:47.773 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243884 s, 43.0 MB/s 2021-02-10T11:20:47.774 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:47.774 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:47.774 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:48.111 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:48.112 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:48.112 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839275/111f6ac6-cf6f-485d-a754-93bce8976a32/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:48.487 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:48.487 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:48.487 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238088 s, 44.0 MB/s 2021-02-10T11:20:48.488 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:48.488 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:48.488 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:48.738 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:20:48.740 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:20:48.824 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:48.825 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:48.825 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839275/111f6ac6-cf6f-485d-a754-93bce8976a32/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:49.109 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:49.109 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:49.109 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024937 s, 42.0 MB/s 2021-02-10T11:20:49.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:49.110 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:49.110 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:49.484 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:49.485 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:49.485 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839275/111f6ac6-cf6f-485d-a754-93bce8976a32/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:49.862 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:49.862 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:49.862 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0261292 s, 40.1 MB/s 2021-02-10T11:20:49.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:49.864 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:49.864 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:50.199 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:50.200 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:50.200 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839275/111f6ac6-cf6f-485d-a754-93bce8976a32/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:50.574 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:50.574 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:50.574 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0223696 s, 46.9 MB/s 2021-02-10T11:20:50.575 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:50.576 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:50.576 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:50.910 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:50.910 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:50.911 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839275/111f6ac6-cf6f-485d-a754-93bce8976a32/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:51.286 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:51.286 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:51.286 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240512 s, 43.6 MB/s 2021-02-10T11:20:51.287 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:51.287 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:51.288 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:51.622 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:51.622 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:51.623 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839275/111f6ac6-cf6f-485d-a754-93bce8976a32/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:51.997 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:51.998 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:51.998 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245285 s, 42.7 MB/s 2021-02-10T11:20:51.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:51.999 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:51.999 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:52.333 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:52.334 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:52.334 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839275/111f6ac6-cf6f-485d-a754-93bce8976a32/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:52.706 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:52.706 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:52.706 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0224939 s, 46.6 MB/s 2021-02-10T11:20:52.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:52.708 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:52.708 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:53.042 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:53.042 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:53.043 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839275/111f6ac6-cf6f-485d-a754-93bce8976a32/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:53.417 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:53.418 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:53.418 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245657 s, 42.7 MB/s 2021-02-10T11:20:53.419 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839276 2021-02-10T11:20:53.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.826 7f0966779700 1 Processor -- start 2021-02-10T11:20:53.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.826 7f0966779700 1 -- start start 2021-02-10T11:20:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.826 7f0966779700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f09601b0470 0x7f09601b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.826 7f0966779700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09600f0b50 con 0x7f09600ef610 2021-02-10T11:20:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.826 7f0966779700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09600f24e0 con 0x7f09600f20e0 2021-02-10T11:20:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.826 7f0966779700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09600e5ea0 con 0x7f09601b0470 2021-02-10T11:20:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.827 7f095f7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f09600ef610 0x7f09600efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47230/0 (socket says 172.21.2.101:47230) 2021-02-10T11:20:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.827 7f095f7fe700 1 -- 172.21.2.101:0/3641541000 learned_addr learned my addr 172.21.2.101:0/3641541000 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:20:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.827 7f095effd700 1 -- 172.21.2.101:0/3641541000 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3348391093 0 0) 0x7f09540009b0 con 0x7f09600ef610 2021-02-10T11:20:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.827 7f095effd700 1 -- 172.21.2.101:0/3641541000 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0948002fe0 con 0x7f09600ef610 2021-02-10T11:20:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.827 7f095ffff700 1 --2- 172.21.2.101:0/3641541000 >> v2:172.21.2.108:3300/0 conn(0x7f09601b0470 0x7f09601b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.827 7f095effd700 1 -- 172.21.2.101:0/3641541000 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4255569252 0 0) 0x7f0954001010 con 0x7f09600ef610 2021-02-10T11:20:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.827 7f095effd700 1 -- 172.21.2.101:0/3641541000 >> v2:172.21.2.108:3300/0 conn(0x7f09601b0470 msgr2=0x7f09601b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:20:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.827 7f095effd700 1 --2- 172.21.2.101:0/3641541000 >> v2:172.21.2.108:3300/0 conn(0x7f09601b0470 0x7f09601b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.827 7f095effd700 1 -- 172.21.2.101:0/3641541000 >> v1:172.21.2.108:6789/0 conn(0x7f09600f20e0 legacy=0x7f09600f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:20:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.827 7f095effd700 1 -- 172.21.2.101:0/3641541000 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09600e52e0 con 0x7f09600ef610 2021-02-10T11:20:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.827 7f095effd700 1 -- 172.21.2.101:0/3641541000 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f0954001590 con 0x7f09600ef610 2021-02-10T11:20:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.827 7f095effd700 1 -- 172.21.2.101:0/3641541000 >> v1:172.21.2.101:6789/0 conn(0x7f09600ef610 legacy=0x7f09600efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.827 7f095effd700 1 --2- 172.21.2.101:0/3641541000 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0948003b90 0x7f0948006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.827 7f095effd700 1 --2- 172.21.2.101:0/3641541000 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09480068c0 0x7f0948008dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.827 7f095effd700 1 --2- 172.21.2.101:0/3641541000 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09480096c0 0x7f094800bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.827 7f095effd700 1 -- 172.21.2.101:0/3641541000 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094800c430 con 0x7f09480096c0 2021-02-10T11:20:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.827 7f095effd700 1 -- 172.21.2.101:0/3641541000 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094800c650 con 0x7f09480068c0 2021-02-10T11:20:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.827 7f095effd700 1 -- 172.21.2.101:0/3641541000 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f094800c870 con 0x7f0948003b90 2021-02-10T11:20:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.827 7f095ffff700 1 --2- 172.21.2.101:0/3641541000 >> v2:172.21.2.108:3300/0 conn(0x7f09601b0470 0x7f09601b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:20:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.827 7f0964d96700 1 --2- 172.21.2.101:0/3641541000 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09480096c0 0x7f094800bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.828 7f095f7fe700 1 --2- 172.21.2.101:0/3641541000 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09480068c0 0x7f0948008dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.828 7f0964d96700 1 -- 172.21.2.101:0/3641541000 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0948003b90 msgr2=0x7f0948006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:20:53.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.828 7f0964d96700 1 --2- 172.21.2.101:0/3641541000 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0948003b90 0x7f0948006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:53.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.828 7f0964d96700 1 -- 172.21.2.101:0/3641541000 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09480068c0 msgr2=0x7f0948008dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:20:53.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.828 7f0964d96700 1 --2- 172.21.2.101:0/3641541000 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09480068c0 0x7f0948008dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:53.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.828 7f0964d96700 1 -- 172.21.2.101:0/3641541000 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094800cb10 con 0x7f09480096c0 2021-02-10T11:20:53.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.828 7f0964d96700 1 -- 172.21.2.101:0/3641541000 reap_dead start 2021-02-10T11:20:53.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.828 7f0964d96700 1 --2- 172.21.2.101:0/3641541000 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09480096c0 0x7f094800bbe0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f0958006e30 tx=0x7f095804de60).ready entity=mon.0 client_cookie=a8ac73e4812a27c2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:53.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.828 7f095effd700 1 -- 172.21.2.101:0/3641541000 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0958003020 con 0x7f09480096c0 2021-02-10T11:20:53.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.828 7f095effd700 1 -- 172.21.2.101:0/3641541000 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0958001400 con 0x7f09480096c0 2021-02-10T11:20:53.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.828 7f095effd700 1 -- 172.21.2.101:0/3641541000 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0958001890 con 0x7f09480096c0 2021-02-10T11:20:53.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.828 7f0966779700 1 -- 172.21.2.101:0/3641541000 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09480096c0 msgr2=0x7f094800bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:53.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.828 7f0966779700 1 --2- 172.21.2.101:0/3641541000 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09480096c0 0x7f094800bbe0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f0958006e30 tx=0x7f095804de60).stop 2021-02-10T11:20:53.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.829 7f0966779700 1 -- 172.21.2.101:0/3641541000 shutdown_connections 2021-02-10T11:20:53.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.829 7f0966779700 1 --2- 172.21.2.101:0/3641541000 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09480096c0 0x7f094800bbe0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:53.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.829 7f0966779700 1 -- 172.21.2.101:0/3641541000 >> 172.21.2.101:0/3641541000 conn(0x7f09600ce3e0 msgr2=0x7f09600e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:53.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.829 7f0966779700 1 -- 172.21.2.101:0/3641541000 shutdown_connections 2021-02-10T11:20:53.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.829 7f0966779700 1 -- 172.21.2.101:0/3641541000 wait complete. 2021-02-10T11:20:53.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.843 7f0966779700 1 Processor -- start 2021-02-10T11:20:53.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.844 7f0966779700 1 -- start start 2021-02-10T11:20:53.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.844 7f0966779700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09600b7db0 0x7f09600b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:53.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.844 7f0966779700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09600b8880 0x7f09601a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:53.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.844 7f0966779700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09600b8c80 0x7f09601a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:53.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.844 7f0966779700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09600b79b0 con 0x7f09600b8880 2021-02-10T11:20:53.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.844 7f0966779700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09601a8090 con 0x7f09600b7db0 2021-02-10T11:20:53.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.844 7f0966779700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09601a82b0 con 0x7f09600b8c80 2021-02-10T11:20:53.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.844 7f095f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09600b8880 0x7f09601a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:53.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.844 7f095f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09600b8880 0x7f09601a7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46746/0 (socket says 172.21.2.101:46746) 2021-02-10T11:20:53.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.844 7f095f7fe700 1 -- 172.21.2.101:0/2844573012 learned_addr learned my addr 172.21.2.101:0/2844573012 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:20:53.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.845 7f0964d96700 1 --2- 172.21.2.101:0/2844573012 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09600b8c80 0x7f09601a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:53.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.845 7f095ffff700 1 --2- 172.21.2.101:0/2844573012 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09600b7db0 0x7f09600b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:53.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.845 7f095f7fe700 1 -- 172.21.2.101:0/2844573012 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09600b8c80 msgr2=0x7f09601a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:53.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.845 7f095f7fe700 1 --2- 172.21.2.101:0/2844573012 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09600b8c80 0x7f09601a7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:53.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.845 7f095f7fe700 1 -- 172.21.2.101:0/2844573012 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09600b7db0 msgr2=0x7f09600b8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:53.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.845 7f095f7fe700 1 --2- 172.21.2.101:0/2844573012 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09600b7db0 0x7f09600b8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:53.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.845 7f095f7fe700 1 -- 172.21.2.101:0/2844573012 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0960190c10 con 0x7f09600b8880 2021-02-10T11:20:53.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.845 7f095ffff700 1 --2- 172.21.2.101:0/2844573012 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09600b7db0 0x7f09600b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:20:53.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.845 7f095f7fe700 1 --2- 172.21.2.101:0/2844573012 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09600b8880 0x7f09601a7450 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f0954007650 tx=0x7f095400a900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:53.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.845 7f095cff9700 1 -- 172.21.2.101:0/2844573012 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f095400acf0 con 0x7f09600b8880 2021-02-10T11:20:53.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.845 7f095cff9700 1 -- 172.21.2.101:0/2844573012 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0954004350 con 0x7f09600b8880 2021-02-10T11:20:53.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.845 7f095cff9700 1 -- 172.21.2.101:0/2844573012 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f09540048b0 con 0x7f09600b8880 2021-02-10T11:20:53.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.845 7f0966779700 1 -- 172.21.2.101:0/2844573012 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0960190c10 con 0x7f09600b8880 2021-02-10T11:20:53.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.845 7f0966779700 1 -- 172.21.2.101:0/2844573012 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09600c1eb0 con 0x7f09600b8880 2021-02-10T11:20:53.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.846 7f095cff9700 1 -- 172.21.2.101:0/2844573012 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0954004530 con 0x7f09600b8880 2021-02-10T11:20:53.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.846 7f095cff9700 1 --2- 172.21.2.101:0/2844573012 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09400186c0 0x7f094001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:53.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.846 7f095cff9700 1 -- 172.21.2.101:0/2844573012 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f09540325c0 con 0x7f09600b8880 2021-02-10T11:20:53.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.846 7f095ffff700 1 --2- 172.21.2.101:0/2844573012 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09400186c0 0x7f094001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:53.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.846 7f095ffff700 1 --2- 172.21.2.101:0/2844573012 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09400186c0 0x7f094001abc0 crc :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:53.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.875 7f0966779700 1 -- 172.21.2.101:0/2844573012 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0960105b30 con 0x7f09600b8880 2021-02-10T11:20:53.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:53.878 7f095cff9700 1 -- 172.21.2.101:0/2844573012 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f095402a390 con 0x7f09600b8880 2021-02-10T11:20:54.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.007 7f0966779700 1 -- 172.21.2.101:0/2844573012 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839276", "target": ["mgr", ""]}) v1 -- 0x7f09600c1eb0 con 0x7f09400186c0 2021-02-10T11:20:54.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.032 7f095cff9700 1 -- 172.21.2.101:0/2844573012 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f0950006b00 con 0x7f09400186c0 2021-02-10T11:20:54.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.040 7f0966779700 1 -- 172.21.2.101:0/2844573012 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09400186c0 msgr2=0x7f094001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:54.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.040 7f0966779700 1 --2- 172.21.2.101:0/2844573012 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09400186c0 0x7f094001abc0 crc :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:54.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.040 7f0966779700 1 -- 172.21.2.101:0/2844573012 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09600b8880 msgr2=0x7f09601a7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:54.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.040 7f0966779700 1 --2- 172.21.2.101:0/2844573012 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09600b8880 0x7f09601a7450 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f0954007650 tx=0x7f095400a900).stop 2021-02-10T11:20:54.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.040 7f0966779700 1 -- 172.21.2.101:0/2844573012 shutdown_connections 2021-02-10T11:20:54.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.040 7f0966779700 1 --2- 172.21.2.101:0/2844573012 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09400186c0 0x7f094001abc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:54.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.040 7f0966779700 1 --2- 172.21.2.101:0/2844573012 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09600b8c80 0x7f09601a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:54.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.040 7f0966779700 1 --2- 172.21.2.101:0/2844573012 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09600b8880 0x7f09601a7450 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:54.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.040 7f0966779700 1 --2- 172.21.2.101:0/2844573012 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09600b7db0 0x7f09600b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:54.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.040 7f0966779700 1 -- 172.21.2.101:0/2844573012 >> 172.21.2.101:0/2844573012 conn(0x7f0960094e10 msgr2=0x7f09600a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:54.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.040 7f0966779700 1 -- 172.21.2.101:0/2844573012 shutdown_connections 2021-02-10T11:20:54.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.040 7f0966779700 1 -- 172.21.2.101:0/2844573012 wait complete. 2021-02-10T11:20:54.055 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839276 2021-02-10T11:20:54.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.462 7fdc6c7b7700 1 Processor -- start 2021-02-10T11:20:54.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.462 7fdc6c7b7700 1 -- start start 2021-02-10T11:20:54.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.463 7fdc6c7b7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc641b0470 0x7fdc641b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:54.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.463 7fdc6c7b7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdc640f20e0 0x7fdc640f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:54.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.463 7fdc6c7b7700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc640f0b50 con 0x7fdc640ef610 2021-02-10T11:20:54.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.463 7fdc6c7b7700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc640f0820 con 0x7fdc640f20e0 2021-02-10T11:20:54.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.463 7fdc6c7b7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc640f24e0 con 0x7fdc641b0470 2021-02-10T11:20:54.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.463 7fdc6add4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdc640f20e0 0x7fdc640f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:54.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.463 7fdc6add4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdc640f20e0 0x7fdc640f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46756/0 (socket says 172.21.2.101:46756) 2021-02-10T11:20:54.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.463 7fdc6add4700 1 -- 172.21.2.101:0/1088962931 learned_addr learned my addr 172.21.2.101:0/1088962931 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:20:54.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.463 7fdc6add4700 1 -- 172.21.2.101:0/1088962931 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc641b0470 msgr2=0x7fdc641b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:20:54.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.463 7fdc6add4700 1 --2- 172.21.2.101:0/1088962931 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc641b0470 0x7fdc641b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:54.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.463 7fdc6add4700 1 -- 172.21.2.101:0/1088962931 >> v1:172.21.2.108:6789/0 conn(0x7fdc640ef610 legacy=0x7fdc640efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:20:54.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.463 7fdc6add4700 1 -- 172.21.2.101:0/1088962931 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc640e52e0 con 0x7fdc640f20e0 2021-02-10T11:20:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.463 7fdc6add4700 1 --2- 172.21.2.101:0/1088962931 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc640f20e0 0x7fdc640f0200 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7fdc5c006f00 tx=0x7fdc5c009f10).ready entity=mon.0 client_cookie=9d2f45aaa0531135 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.463 7fdc695d1700 1 -- 172.21.2.101:0/1088962931 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc5c00b3b0 con 0x7fdc640f20e0 2021-02-10T11:20:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.463 7fdc695d1700 1 -- 172.21.2.101:0/1088962931 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5c00b630 con 0x7fdc640f20e0 2021-02-10T11:20:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.463 7fdc695d1700 1 -- 172.21.2.101:0/1088962931 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc5c00bb50 con 0x7fdc640f20e0 2021-02-10T11:20:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.463 7fdc6c7b7700 1 -- 172.21.2.101:0/1088962931 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc640f20e0 msgr2=0x7fdc640f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.463 7fdc6c7b7700 1 --2- 172.21.2.101:0/1088962931 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc640f20e0 0x7fdc640f0200 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7fdc5c006f00 tx=0x7fdc5c009f10).stop 2021-02-10T11:20:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.464 7fdc6c7b7700 1 -- 172.21.2.101:0/1088962931 shutdown_connections 2021-02-10T11:20:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.464 7fdc6c7b7700 1 --2- 172.21.2.101:0/1088962931 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc640f20e0 0x7fdc640f0200 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.464 7fdc6c7b7700 1 --2- 172.21.2.101:0/1088962931 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc641b0470 0x7fdc641b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.464 7fdc6c7b7700 1 -- 172.21.2.101:0/1088962931 >> 172.21.2.101:0/1088962931 conn(0x7fdc640ce3e0 msgr2=0x7fdc640e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.464 7fdc6c7b7700 1 -- 172.21.2.101:0/1088962931 shutdown_connections 2021-02-10T11:20:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.464 7fdc6c7b7700 1 -- 172.21.2.101:0/1088962931 wait complete. 2021-02-10T11:20:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.478 7fdc6c7b7700 1 Processor -- start 2021-02-10T11:20:54.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.478 7fdc6c7b7700 1 -- start start 2021-02-10T11:20:54.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.479 7fdc6c7b7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6409e140 0x7fdc6409e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:54.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.479 7fdc6c7b7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc6409ee40 0x7fdc640ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:54.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.479 7fdc6c7b7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc6409f240 0x7fdc640aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:54.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.479 7fdc6c7b7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6409e540 con 0x7fdc6409e140 2021-02-10T11:20:54.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.479 7fdc6c7b7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc641933c0 con 0x7fdc6409ee40 2021-02-10T11:20:54.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.479 7fdc6c7b7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc640af5e0 con 0x7fdc6409f240 2021-02-10T11:20:54.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.479 7fdc6a5d3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6409e140 0x7fdc6409e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:54.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.479 7fdc6a5d3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6409e140 0x7fdc6409e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46760/0 (socket says 172.21.2.101:46760) 2021-02-10T11:20:54.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.479 7fdc6a5d3700 1 -- 172.21.2.101:0/3775766374 learned_addr learned my addr 172.21.2.101:0/3775766374 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:20:54.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.479 7fdc6a5d3700 1 -- 172.21.2.101:0/3775766374 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc6409f240 msgr2=0x7fdc640aefc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:20:54.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.479 7fdc6a5d3700 1 --2- 172.21.2.101:0/3775766374 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc6409f240 0x7fdc640aefc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:54.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.479 7fdc6a5d3700 1 -- 172.21.2.101:0/3775766374 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc6409ee40 msgr2=0x7fdc640ae9a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:20:54.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.479 7fdc6a5d3700 1 --2- 172.21.2.101:0/3775766374 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc6409ee40 0x7fdc640ae9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:54.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.479 7fdc6a5d3700 1 -- 172.21.2.101:0/3775766374 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc640af800 con 0x7fdc6409e140 2021-02-10T11:20:54.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.479 7fdc6a5d3700 1 --2- 172.21.2.101:0/3775766374 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6409e140 0x7fdc6409e820 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7fdc5800b5c0 tx=0x7fdc5800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:54.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.480 7fdc56ffd700 1 -- 172.21.2.101:0/3775766374 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc58000c40 con 0x7fdc6409e140 2021-02-10T11:20:54.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.480 7fdc56ffd700 1 -- 172.21.2.101:0/3775766374 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5800be20 con 0x7fdc6409e140 2021-02-10T11:20:54.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.480 7fdc56ffd700 1 -- 172.21.2.101:0/3775766374 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc580062e0 con 0x7fdc6409e140 2021-02-10T11:20:54.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.480 7fdc6c7b7700 1 -- 172.21.2.101:0/3775766374 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc640af800 con 0x7fdc6409e140 2021-02-10T11:20:54.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.480 7fdc6c7b7700 1 -- 172.21.2.101:0/3775766374 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc640afd90 con 0x7fdc6409e140 2021-02-10T11:20:54.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.480 7fdc56ffd700 1 -- 172.21.2.101:0/3775766374 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdc58006030 con 0x7fdc6409e140 2021-02-10T11:20:54.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.480 7fdc56ffd700 1 --2- 172.21.2.101:0/3775766374 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc4c0186c0 0x7fdc4c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:20:54.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.480 7fdc56ffd700 1 -- 172.21.2.101:0/3775766374 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fdc58032600 con 0x7fdc6409e140 2021-02-10T11:20:54.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.481 7fdc69dd2700 1 --2- 172.21.2.101:0/3775766374 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc4c0186c0 0x7fdc4c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:20:54.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.481 7fdc69dd2700 1 --2- 172.21.2.101:0/3775766374 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc4c0186c0 0x7fdc4c01abc0 crc :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:20:54.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.512 7fdc6c7b7700 1 -- 172.21.2.101:0/3775766374 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc640edd10 con 0x7fdc6409e140 2021-02-10T11:20:54.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.515 7fdc56ffd700 1 -- 172.21.2.101:0/3775766374 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdc580009b0 con 0x7fdc6409e140 2021-02-10T11:20:54.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.642 7fdc6c7b7700 1 -- 172.21.2.101:0/3775766374 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839276", "target": ["mgr", ""]}) v1 -- 0x7fdc640afd90 con 0x7fdc4c0186c0 2021-02-10T11:20:54.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.645 7fdc56ffd700 1 -- 172.21.2.101:0/3775766374 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fdc60006970 con 0x7fdc4c0186c0 2021-02-10T11:20:54.656 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839276/345eb79e-7efd-4532-8903-82bc61858edf 2021-02-10T11:20:54.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.650 7fdc6c7b7700 1 -- 172.21.2.101:0/3775766374 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc4c0186c0 msgr2=0x7fdc4c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:54.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.650 7fdc6c7b7700 1 --2- 172.21.2.101:0/3775766374 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc4c0186c0 0x7fdc4c01abc0 crc :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:54.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.650 7fdc6c7b7700 1 -- 172.21.2.101:0/3775766374 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6409e140 msgr2=0x7fdc6409e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:20:54.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.650 7fdc6c7b7700 1 --2- 172.21.2.101:0/3775766374 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6409e140 0x7fdc6409e820 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7fdc5800b5c0 tx=0x7fdc5800b960).stop 2021-02-10T11:20:54.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.650 7fdc6c7b7700 1 -- 172.21.2.101:0/3775766374 shutdown_connections 2021-02-10T11:20:54.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.650 7fdc6c7b7700 1 --2- 172.21.2.101:0/3775766374 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc4c0186c0 0x7fdc4c01abc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:54.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.650 7fdc6c7b7700 1 --2- 172.21.2.101:0/3775766374 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc6409f240 0x7fdc640aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:54.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.650 7fdc6c7b7700 1 --2- 172.21.2.101:0/3775766374 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc6409ee40 0x7fdc640ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:20:54.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.650 7fdc6c7b7700 1 --2- 172.21.2.101:0/3775766374 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6409e140 0x7fdc6409e820 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:20:54.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.650 7fdc6c7b7700 1 -- 172.21.2.101:0/3775766374 >> 172.21.2.101:0/3775766374 conn(0x7fdc640eecf0 msgr2=0x7fdc640ee250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:20:54.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.651 7fdc6c7b7700 1 -- 172.21.2.101:0/3775766374 shutdown_connections 2021-02-10T11:20:54.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:20:54.651 7fdc6c7b7700 1 -- 172.21.2.101:0/3775766374 wait complete. 2021-02-10T11:20:54.665 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839276 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839276/345eb79e-7efd-4532-8903-82bc61858edf 2021-02-10T11:20:54.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:54.665 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:54.665 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:55.002 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:55.002 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:55.003 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839276/345eb79e-7efd-4532-8903-82bc61858edf/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:55.383 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:55.383 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:55.383 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241241 s, 43.5 MB/s 2021-02-10T11:20:55.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:55.385 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:55.385 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:55.719 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:55.719 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:55.719 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839276/345eb79e-7efd-4532-8903-82bc61858edf/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:56.096 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:56.096 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:56.096 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251034 s, 41.8 MB/s 2021-02-10T11:20:56.097 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:56.098 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:56.098 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:56.434 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:56.434 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:56.434 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839276/345eb79e-7efd-4532-8903-82bc61858edf/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:56.810 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:56.810 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:56.810 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0250512 s, 41.9 MB/s 2021-02-10T11:20:56.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:56.811 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:56.811 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:57.146 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:57.146 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:57.147 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839276/345eb79e-7efd-4532-8903-82bc61858edf/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:57.521 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:57.521 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:57.521 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0228309 s, 45.9 MB/s 2021-02-10T11:20:57.522 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:57.522 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:57.522 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:57.857 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:57.857 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:57.857 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839276/345eb79e-7efd-4532-8903-82bc61858edf/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:58.233 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:58.234 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:58.234 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246967 s, 42.5 MB/s 2021-02-10T11:20:58.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:58.235 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:58.235 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:58.571 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:58.571 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:58.571 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839276/345eb79e-7efd-4532-8903-82bc61858edf/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:58.942 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:58.942 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:58.942 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.022822 s, 45.9 MB/s 2021-02-10T11:20:58.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:58.943 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:58.944 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:59.277 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:59.277 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:59.278 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839276/345eb79e-7efd-4532-8903-82bc61858edf/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:20:59.653 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:20:59.653 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:20:59.653 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241668 s, 43.4 MB/s 2021-02-10T11:20:59.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:20:59.654 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:20:59.654 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:20:59.991 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:20:59.991 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:20:59.991 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839276/345eb79e-7efd-4532-8903-82bc61858edf/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:00.369 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:00.369 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:00.369 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253965 s, 41.3 MB/s 2021-02-10T11:21:00.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:00.370 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:00.370 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:00.706 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:00.706 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:00.706 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839276/345eb79e-7efd-4532-8903-82bc61858edf/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:01.082 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240358 s, 43.6 MB/s 2021-02-10T11:21:01.082 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:01.083 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:01.083 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:01.418 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:01.663 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:01.664 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839276/345eb79e-7efd-4532-8903-82bc61858edf/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:02.038 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:02.038 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:02.039 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023553 s, 44.5 MB/s 2021-02-10T11:21:02.039 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839277 2021-02-10T11:21:02.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.446 7efe4ba1d700 1 Processor -- start 2021-02-10T11:21:02.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.447 7efe4ba1d700 1 -- start start 2021-02-10T11:21:02.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.447 7efe4ba1d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7efe441b0870 0x7efe440f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:02.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.447 7efe4ba1d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe441b0c70 con 0x7efe441b0470 2021-02-10T11:21:02.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.447 7efe4ba1d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe440f0b50 con 0x7efe440efe00 2021-02-10T11:21:02.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.447 7efe4ba1d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe440e5ea0 con 0x7efe441b0870 2021-02-10T11:21:02.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.447 7efe49038700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7efe441b0870 0x7efe440f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:02.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.447 7efe49839700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7efe441b0470 0x7efe440ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47260/0 (socket says 172.21.2.101:47260) 2021-02-10T11:21:02.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.447 7efe49839700 1 -- 172.21.2.101:0/2523506372 learned_addr learned my addr 172.21.2.101:0/2523506372 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:21:02.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.448 7efe49038700 1 -- 172.21.2.101:0/2523506372 >> v1:172.21.2.108:6789/0 conn(0x7efe440efe00 legacy=0x7efe440f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:02.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.448 7efe49038700 1 -- 172.21.2.101:0/2523506372 >> v1:172.21.2.101:6789/0 conn(0x7efe441b0470 legacy=0x7efe440ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:02.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.448 7efe49038700 1 -- 172.21.2.101:0/2523506372 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe440e52e0 con 0x7efe441b0870 2021-02-10T11:21:02.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.448 7efe49038700 1 --2- 172.21.2.101:0/2523506372 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe441b0870 0x7efe440f20e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7efe38006650 tx=0x7efe38009a80).ready entity=mon.0 client_cookie=aae0ad82ddbeed56 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:02.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.448 7efe48837700 1 -- 172.21.2.101:0/2523506372 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efe380009b0 con 0x7efe441b0870 2021-02-10T11:21:02.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.448 7efe48837700 1 -- 172.21.2.101:0/2523506372 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3800c0c0 con 0x7efe441b0870 2021-02-10T11:21:02.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.448 7efe48837700 1 -- 172.21.2.101:0/2523506372 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efe3800c520 con 0x7efe441b0870 2021-02-10T11:21:02.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.448 7efe4ba1d700 1 -- 172.21.2.101:0/2523506372 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe441b0870 msgr2=0x7efe440f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:02.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.448 7efe4ba1d700 1 --2- 172.21.2.101:0/2523506372 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe441b0870 0x7efe440f20e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7efe38006650 tx=0x7efe38009a80).stop 2021-02-10T11:21:02.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.449 7efe4ba1d700 1 -- 172.21.2.101:0/2523506372 shutdown_connections 2021-02-10T11:21:02.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.449 7efe4ba1d700 1 --2- 172.21.2.101:0/2523506372 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe441b0870 0x7efe440f20e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:02.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.449 7efe4ba1d700 1 -- 172.21.2.101:0/2523506372 >> 172.21.2.101:0/2523506372 conn(0x7efe440ce3e0 msgr2=0x7efe440e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:02.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.449 7efe4ba1d700 1 -- 172.21.2.101:0/2523506372 shutdown_connections 2021-02-10T11:21:02.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.449 7efe4ba1d700 1 -- 172.21.2.101:0/2523506372 wait complete. 2021-02-10T11:21:02.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.464 7efe4ba1d700 1 Processor -- start 2021-02-10T11:21:02.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.464 7efe4ba1d700 1 -- start start 2021-02-10T11:21:02.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.464 7efe4ba1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe44087510 0x7efe440aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:02.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.464 7efe4ba1d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efe441c68f0 0x7efe441b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:02.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.464 7efe4ba1d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efe4408c4d0 0x7efe441b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:02.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.464 7efe4ba1d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe440a2d50 con 0x7efe44087510 2021-02-10T11:21:02.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.464 7efe4ba1d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe440a2f70 con 0x7efe4408c4d0 2021-02-10T11:21:02.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.464 7efe4ba1d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe440a3190 con 0x7efe441c68f0 2021-02-10T11:21:02.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.464 7efe49839700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe44087510 0x7efe440aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:02.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.464 7efe49839700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe44087510 0x7efe440aedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46770/0 (socket says 172.21.2.101:46770) 2021-02-10T11:21:02.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.464 7efe49839700 1 -- 172.21.2.101:0/3742306901 learned_addr learned my addr 172.21.2.101:0/3742306901 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:21:02.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.465 7efe49038700 1 --2- 172.21.2.101:0/3742306901 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efe441c68f0 0x7efe441b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:02.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.465 7efe4a03a700 1 --2- 172.21.2.101:0/3742306901 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efe4408c4d0 0x7efe441b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:02.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.465 7efe49839700 1 -- 172.21.2.101:0/3742306901 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efe441c68f0 msgr2=0x7efe441b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:02.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.465 7efe49839700 1 --2- 172.21.2.101:0/3742306901 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efe441c68f0 0x7efe441b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:02.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.465 7efe49839700 1 -- 172.21.2.101:0/3742306901 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efe4408c4d0 msgr2=0x7efe441b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:02.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.465 7efe49839700 1 --2- 172.21.2.101:0/3742306901 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efe4408c4d0 0x7efe441b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:02.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.465 7efe49839700 1 -- 172.21.2.101:0/3742306901 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe440a33b0 con 0x7efe44087510 2021-02-10T11:21:02.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.465 7efe49839700 1 --2- 172.21.2.101:0/3742306901 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe44087510 0x7efe440aedc0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7efe40001c60 tx=0x7efe4000a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:02.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.465 7efe367fc700 1 -- 172.21.2.101:0/3742306901 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efe400010b0 con 0x7efe44087510 2021-02-10T11:21:02.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.465 7efe367fc700 1 -- 172.21.2.101:0/3742306901 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe40004140 con 0x7efe44087510 2021-02-10T11:21:02.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.465 7efe367fc700 1 -- 172.21.2.101:0/3742306901 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efe40004660 con 0x7efe44087510 2021-02-10T11:21:02.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.465 7efe4ba1d700 1 -- 172.21.2.101:0/3742306901 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe440a33b0 con 0x7efe44087510 2021-02-10T11:21:02.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.465 7efe4ba1d700 1 -- 172.21.2.101:0/3742306901 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe440a33b0 con 0x7efe44087510 2021-02-10T11:21:02.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.466 7efe367fc700 1 -- 172.21.2.101:0/3742306901 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7efe400010b0 con 0x7efe44087510 2021-02-10T11:21:02.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.466 7efe367fc700 1 --2- 172.21.2.101:0/3742306901 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efe2c018730 0x7efe2c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:02.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.466 7efe367fc700 1 -- 172.21.2.101:0/3742306901 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7efe40032630 con 0x7efe44087510 2021-02-10T11:21:02.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.466 7efe49038700 1 --2- 172.21.2.101:0/3742306901 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efe2c018730 0x7efe2c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:02.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.466 7efe49038700 1 --2- 172.21.2.101:0/3742306901 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efe2c018730 0x7efe2c01abc0 crc :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:02.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.497 7efe4ba1d700 1 -- 172.21.2.101:0/3742306901 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe4407e6b0 con 0x7efe44087510 2021-02-10T11:21:02.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.499 7efe367fc700 1 -- 172.21.2.101:0/3742306901 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7efe400009b0 con 0x7efe44087510 2021-02-10T11:21:02.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.626 7efe4ba1d700 1 -- 172.21.2.101:0/3742306901 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839277", "target": ["mgr", ""]}) v1 -- 0x7efe4407e6b0 con 0x7efe2c018730 2021-02-10T11:21:02.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.653 7efe367fc700 1 -- 172.21.2.101:0/3742306901 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7efe38000c90 con 0x7efe2c018730 2021-02-10T11:21:02.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.659 7efe4ba1d700 1 -- 172.21.2.101:0/3742306901 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efe2c018730 msgr2=0x7efe2c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:02.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.659 7efe4ba1d700 1 --2- 172.21.2.101:0/3742306901 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efe2c018730 0x7efe2c01abc0 crc :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:02.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.659 7efe4ba1d700 1 -- 172.21.2.101:0/3742306901 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe44087510 msgr2=0x7efe440aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:02.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.659 7efe4ba1d700 1 --2- 172.21.2.101:0/3742306901 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe44087510 0x7efe440aedc0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7efe40001c60 tx=0x7efe4000a8d0).stop 2021-02-10T11:21:02.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.659 7efe4ba1d700 1 -- 172.21.2.101:0/3742306901 shutdown_connections 2021-02-10T11:21:02.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.659 7efe4ba1d700 1 --2- 172.21.2.101:0/3742306901 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efe2c018730 0x7efe2c01abc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:02.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.659 7efe4ba1d700 1 --2- 172.21.2.101:0/3742306901 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efe441c68f0 0x7efe441b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:02.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.659 7efe4ba1d700 1 --2- 172.21.2.101:0/3742306901 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efe4408c4d0 0x7efe441b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:02.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.659 7efe4ba1d700 1 --2- 172.21.2.101:0/3742306901 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe44087510 0x7efe440aedc0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:02.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.659 7efe4ba1d700 1 -- 172.21.2.101:0/3742306901 >> 172.21.2.101:0/3742306901 conn(0x7efe440886d0 msgr2=0x7efe44081c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:02.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.659 7efe4ba1d700 1 -- 172.21.2.101:0/3742306901 shutdown_connections 2021-02-10T11:21:02.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:02.659 7efe4ba1d700 1 -- 172.21.2.101:0/3742306901 wait complete. 2021-02-10T11:21:02.674 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839277 2021-02-10T11:21:03.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.081 7f96ffad0700 1 Processor -- start 2021-02-10T11:21:03.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.081 7f96ffad0700 1 -- start start 2021-02-10T11:21:03.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.082 7f96ffad0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f96f81b0470 0x7f96f81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:03.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.082 7f96ffad0700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f96f80f20e0 0x7f96f80f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:03.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.082 7f96ffad0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f80f0b50 con 0x7f96f80ef610 2021-02-10T11:21:03.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.082 7f96ffad0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f80f0820 con 0x7f96f81b0470 2021-02-10T11:21:03.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.082 7f96ffad0700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f80f24e0 con 0x7f96f80f20e0 2021-02-10T11:21:03.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.082 7f96fd8ec700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f96f81b0470 0x7f96f81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:03.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.082 7f96fd8ec700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f96f81b0470 0x7f96f81b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46778/0 (socket says 172.21.2.101:46778) 2021-02-10T11:21:03.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.082 7f96fd8ec700 1 -- 172.21.2.101:0/2136141128 learned_addr learned my addr 172.21.2.101:0/2136141128 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:21:03.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.082 7f96fd8ec700 1 -- 172.21.2.101:0/2136141128 >> v2:172.21.2.108:3300/0 conn(0x7f96f80f20e0 msgr2=0x7f96f80f0200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:21:03.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.082 7f96fd8ec700 1 --2- 172.21.2.101:0/2136141128 >> v2:172.21.2.108:3300/0 conn(0x7f96f80f20e0 0x7f96f80f0200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:03.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.082 7f96fd8ec700 1 -- 172.21.2.101:0/2136141128 >> v1:172.21.2.108:6789/0 conn(0x7f96f80ef610 legacy=0x7f96f80efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:21:03.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.082 7f96fd8ec700 1 -- 172.21.2.101:0/2136141128 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f80e52e0 con 0x7f96f81b0470 2021-02-10T11:21:03.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.082 7f96fd8ec700 1 --2- 172.21.2.101:0/2136141128 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96f81b0470 0x7f96f81b0870 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f96ec006700 tx=0x7f96ec009b80).ready entity=mon.0 client_cookie=dd2c54229de1ae7a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:03.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.083 7f96fc8ea700 1 -- 172.21.2.101:0/2136141128 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96ec00c0c0 con 0x7f96f81b0470 2021-02-10T11:21:03.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.083 7f96fc8ea700 1 -- 172.21.2.101:0/2136141128 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ec00c340 con 0x7f96f81b0470 2021-02-10T11:21:03.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.083 7f96fc8ea700 1 -- 172.21.2.101:0/2136141128 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96ec00c860 con 0x7f96f81b0470 2021-02-10T11:21:03.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.083 7f96ffad0700 1 -- 172.21.2.101:0/2136141128 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96f81b0470 msgr2=0x7f96f81b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:03.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.083 7f96ffad0700 1 --2- 172.21.2.101:0/2136141128 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96f81b0470 0x7f96f81b0870 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f96ec006700 tx=0x7f96ec009b80).stop 2021-02-10T11:21:03.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.083 7f96ffad0700 1 -- 172.21.2.101:0/2136141128 shutdown_connections 2021-02-10T11:21:03.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.083 7f96ffad0700 1 --2- 172.21.2.101:0/2136141128 >> v2:172.21.2.108:3300/0 conn(0x7f96f80f20e0 0x7f96f80f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:03.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.083 7f96ffad0700 1 --2- 172.21.2.101:0/2136141128 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96f81b0470 0x7f96f81b0870 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:03.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.083 7f96ffad0700 1 -- 172.21.2.101:0/2136141128 >> 172.21.2.101:0/2136141128 conn(0x7f96f80ce3e0 msgr2=0x7f96f80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:03.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.084 7f96ffad0700 1 -- 172.21.2.101:0/2136141128 shutdown_connections 2021-02-10T11:21:03.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.084 7f96ffad0700 1 -- 172.21.2.101:0/2136141128 wait complete. 2021-02-10T11:21:03.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.098 7f96ffad0700 1 Processor -- start 2021-02-10T11:21:03.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.098 7f96ffad0700 1 -- start start 2021-02-10T11:21:03.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.098 7f96ffad0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96f8094fb0 0x7f96f809e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:03.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.098 7f96ffad0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96f80c2be0 0x7f96f80c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:03.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.098 7f96ffad0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96f80c3600 0x7f96f80a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:03.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.098 7f96ffad0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f809ebf0 con 0x7f96f8094fb0 2021-02-10T11:21:03.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.098 7f96ffad0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f80aa2a0 con 0x7f96f80c3600 2021-02-10T11:21:03.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.098 7f96ffad0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f80aa4c0 con 0x7f96f80c2be0 2021-02-10T11:21:03.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.099 7f96fd8ec700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96f8094fb0 0x7f96f809e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:03.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.099 7f96fd8ec700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96f8094fb0 0x7f96f809e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46780/0 (socket says 172.21.2.101:46780) 2021-02-10T11:21:03.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.099 7f96fd8ec700 1 -- 172.21.2.101:0/1643450068 learned_addr learned my addr 172.21.2.101:0/1643450068 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:21:03.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.099 7f96fd8ec700 1 -- 172.21.2.101:0/1643450068 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96f80c2be0 msgr2=0x7f96f80c2fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:21:03.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.099 7f96fd8ec700 1 --2- 172.21.2.101:0/1643450068 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96f80c2be0 0x7f96f80c2fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:03.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.099 7f96fd8ec700 1 -- 172.21.2.101:0/1643450068 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96f80c3600 msgr2=0x7f96f80a9c80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:21:03.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.099 7f96fd8ec700 1 --2- 172.21.2.101:0/1643450068 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96f80c3600 0x7f96f80a9c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:03.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.099 7f96fd8ec700 1 -- 172.21.2.101:0/1643450068 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f80aa6e0 con 0x7f96f8094fb0 2021-02-10T11:21:03.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.099 7f96fd8ec700 1 --2- 172.21.2.101:0/1643450068 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96f8094fb0 0x7f96f809e5d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f96ec00b5e0 tx=0x7f96ec00b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:03.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.099 7f96ea7fc700 1 -- 172.21.2.101:0/1643450068 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96ec00bd70 con 0x7f96f8094fb0 2021-02-10T11:21:03.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.099 7f96ea7fc700 1 -- 172.21.2.101:0/1643450068 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ec002030 con 0x7f96f8094fb0 2021-02-10T11:21:03.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.099 7f96ffad0700 1 -- 172.21.2.101:0/1643450068 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f80aa6e0 con 0x7f96f8094fb0 2021-02-10T11:21:03.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.099 7f96ea7fc700 1 -- 172.21.2.101:0/1643450068 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96ec002550 con 0x7f96f8094fb0 2021-02-10T11:21:03.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.099 7f96ffad0700 1 -- 172.21.2.101:0/1643450068 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f80aa6e0 con 0x7f96f8094fb0 2021-02-10T11:21:03.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.100 7f96ea7fc700 1 -- 172.21.2.101:0/1643450068 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f96ec000c90 con 0x7f96f8094fb0 2021-02-10T11:21:03.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.100 7f96ea7fc700 1 --2- 172.21.2.101:0/1643450068 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96e00186d0 0x7f96e001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:03.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.100 7f96ea7fc700 1 -- 172.21.2.101:0/1643450068 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f96ec0327d0 con 0x7f96f8094fb0 2021-02-10T11:21:03.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.100 7f96fd0eb700 1 --2- 172.21.2.101:0/1643450068 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96e00186d0 0x7f96e001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:03.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.100 7f96fd0eb700 1 --2- 172.21.2.101:0/1643450068 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96e00186d0 0x7f96e001abd0 crc :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:03.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.132 7f96ffad0700 1 -- 172.21.2.101:0/1643450068 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f80a9750 con 0x7f96f8094fb0 2021-02-10T11:21:03.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.135 7f96ea7fc700 1 -- 172.21.2.101:0/1643450068 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f96ec01f510 con 0x7f96f8094fb0 2021-02-10T11:21:03.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.263 7f96ffad0700 1 -- 172.21.2.101:0/1643450068 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839277", "target": ["mgr", ""]}) v1 -- 0x7f96f80a9750 con 0x7f96e00186d0 2021-02-10T11:21:03.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.267 7f96ea7fc700 1 -- 172.21.2.101:0/1643450068 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f96f4006970 con 0x7f96e00186d0 2021-02-10T11:21:03.277 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839277/18c35ac5-776b-4d62-8334-296d0971ae87 2021-02-10T11:21:03.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.271 7f96ffad0700 1 -- 172.21.2.101:0/1643450068 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96e00186d0 msgr2=0x7f96e001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:03.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.271 7f96ffad0700 1 --2- 172.21.2.101:0/1643450068 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96e00186d0 0x7f96e001abd0 crc :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:03.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.271 7f96ffad0700 1 -- 172.21.2.101:0/1643450068 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96f8094fb0 msgr2=0x7f96f809e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:03.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.271 7f96ffad0700 1 --2- 172.21.2.101:0/1643450068 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96f8094fb0 0x7f96f809e5d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f96ec00b5e0 tx=0x7f96ec00b980).stop 2021-02-10T11:21:03.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.271 7f96ffad0700 1 -- 172.21.2.101:0/1643450068 shutdown_connections 2021-02-10T11:21:03.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.271 7f96ffad0700 1 --2- 172.21.2.101:0/1643450068 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96e00186d0 0x7f96e001abd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:03.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.271 7f96ffad0700 1 --2- 172.21.2.101:0/1643450068 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96f80c2be0 0x7f96f80c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:03.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.271 7f96ffad0700 1 --2- 172.21.2.101:0/1643450068 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96f80c3600 0x7f96f80a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:03.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.271 7f96ffad0700 1 --2- 172.21.2.101:0/1643450068 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96f8094fb0 0x7f96f809e5d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:03.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.271 7f96ffad0700 1 -- 172.21.2.101:0/1643450068 >> 172.21.2.101:0/1643450068 conn(0x7f96f809f630 msgr2=0x7f96f80a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:03.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.272 7f96ffad0700 1 -- 172.21.2.101:0/1643450068 shutdown_connections 2021-02-10T11:21:03.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:03.272 7f96ffad0700 1 -- 172.21.2.101:0/1643450068 wait complete. 2021-02-10T11:21:03.287 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839277 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839277/18c35ac5-776b-4d62-8334-296d0971ae87 2021-02-10T11:21:03.287 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:03.288 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:03.288 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:03.623 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:03.624 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:03.624 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839277/18c35ac5-776b-4d62-8334-296d0971ae87/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:04.004 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:04.004 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:04.004 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0258207 s, 40.6 MB/s 2021-02-10T11:21:04.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:04.006 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:04.006 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:04.343 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:04.343 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:04.344 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839277/18c35ac5-776b-4d62-8334-296d0971ae87/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:04.790 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:04.790 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:04.790 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253871 s, 41.3 MB/s 2021-02-10T11:21:04.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:04.792 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:04.792 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:05.131 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:05.131 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:05.131 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839277/18c35ac5-776b-4d62-8334-296d0971ae87/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:05.512 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:05.512 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:05.512 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0257141 s, 40.8 MB/s 2021-02-10T11:21:05.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:05.513 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:05.513 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:05.852 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:05.852 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:05.852 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839277/18c35ac5-776b-4d62-8334-296d0971ae87/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:06.231 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:06.231 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:06.231 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246879 s, 42.5 MB/s 2021-02-10T11:21:06.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:06.232 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:06.232 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:06.566 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:06.567 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:06.567 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839277/18c35ac5-776b-4d62-8334-296d0971ae87/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:06.943 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:06.943 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:06.943 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241691 s, 43.4 MB/s 2021-02-10T11:21:06.944 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:06.944 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:06.944 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:07.279 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:07.279 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:07.279 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839277/18c35ac5-776b-4d62-8334-296d0971ae87/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:07.654 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:07.654 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:07.654 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245941 s, 42.6 MB/s 2021-02-10T11:21:07.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:07.655 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:07.655 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:07.989 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:07.990 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:07.990 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839277/18c35ac5-776b-4d62-8334-296d0971ae87/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:08.363 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:08.364 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:08.364 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235181 s, 44.6 MB/s 2021-02-10T11:21:08.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:08.365 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:08.365 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:08.700 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:08.700 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:08.700 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839277/18c35ac5-776b-4d62-8334-296d0971ae87/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:09.071 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:09.072 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:09.072 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023896 s, 43.9 MB/s 2021-02-10T11:21:09.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:09.073 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:09.073 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:09.409 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:09.409 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:09.409 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839277/18c35ac5-776b-4d62-8334-296d0971ae87/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:09.786 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:09.786 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:09.786 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0254773 s, 41.2 MB/s 2021-02-10T11:21:09.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:09.787 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:09.787 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:10.123 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:10.123 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:10.123 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839277/18c35ac5-776b-4d62-8334-296d0971ae87/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:10.500 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:10.500 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:10.500 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247014 s, 42.4 MB/s 2021-02-10T11:21:10.501 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839278 2021-02-10T11:21:10.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.908 7f6850317700 1 Processor -- start 2021-02-10T11:21:10.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.908 7f6850317700 1 -- start start 2021-02-10T11:21:10.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.909 7f6850317700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68481b0870 0x7f68480f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:10.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.909 7f6850317700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68481b0c70 con 0x7f68481b0470 2021-02-10T11:21:10.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.909 7f6850317700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68480f0b50 con 0x7f68480efe00 2021-02-10T11:21:10.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.909 7f6850317700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68480e5ea0 con 0x7f68481b0870 2021-02-10T11:21:10.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.909 7f684e133700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f68481b0470 0x7f68480ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47282/0 (socket says 172.21.2.101:47282) 2021-02-10T11:21:10.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.909 7f684e133700 1 -- 172.21.2.101:0/1281744831 learned_addr learned my addr 172.21.2.101:0/1281744831 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:21:10.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.909 7f684d932700 1 --2- 172.21.2.101:0/1281744831 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68481b0870 0x7f68480f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:10.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.909 7f684d131700 1 -- 172.21.2.101:0/1281744831 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2694484080 0 0) 0x7f683c0009b0 con 0x7f68481b0470 2021-02-10T11:21:10.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.909 7f684d131700 1 -- 172.21.2.101:0/1281744831 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6834002fe0 con 0x7f68481b0470 2021-02-10T11:21:10.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.909 7f684d131700 1 -- 172.21.2.101:0/1281744831 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2413217519 0 0) 0x7f6840000f90 con 0x7f68480efe00 2021-02-10T11:21:10.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.910 7f684d131700 1 -- 172.21.2.101:0/1281744831 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6834003210 con 0x7f68480efe00 2021-02-10T11:21:10.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.910 7f684d932700 1 -- 172.21.2.101:0/1281744831 >> v1:172.21.2.108:6789/0 conn(0x7f68480efe00 legacy=0x7f68480f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:10.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.910 7f684d932700 1 -- 172.21.2.101:0/1281744831 >> v1:172.21.2.101:6789/0 conn(0x7f68481b0470 legacy=0x7f68480ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:10.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.910 7f684d932700 1 -- 172.21.2.101:0/1281744831 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68480e52e0 con 0x7f68481b0870 2021-02-10T11:21:10.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.910 7f684d932700 1 --2- 172.21.2.101:0/1281744831 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68481b0870 0x7f68480f20e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f6844006650 tx=0x7f6844009a80).ready entity=mon.2 client_cookie=df13330d4eb01fb6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:10.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.910 7f684d131700 1 -- 172.21.2.101:0/1281744831 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f684400c0c0 con 0x7f68481b0870 2021-02-10T11:21:10.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.910 7f684d131700 1 -- 172.21.2.101:0/1281744831 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684400c340 con 0x7f68481b0870 2021-02-10T11:21:10.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.910 7f684d131700 1 -- 172.21.2.101:0/1281744831 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f684400c820 con 0x7f68481b0870 2021-02-10T11:21:10.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.910 7f6850317700 1 -- 172.21.2.101:0/1281744831 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68481b0870 msgr2=0x7f68480f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:10.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.910 7f6850317700 1 --2- 172.21.2.101:0/1281744831 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68481b0870 0x7f68480f20e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f6844006650 tx=0x7f6844009a80).stop 2021-02-10T11:21:10.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.911 7f6850317700 1 -- 172.21.2.101:0/1281744831 shutdown_connections 2021-02-10T11:21:10.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.911 7f6850317700 1 --2- 172.21.2.101:0/1281744831 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68481b0870 0x7f68480f20e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:10.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.911 7f6850317700 1 -- 172.21.2.101:0/1281744831 >> 172.21.2.101:0/1281744831 conn(0x7f68480ce3e0 msgr2=0x7f68480e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:10.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.911 7f6850317700 1 -- 172.21.2.101:0/1281744831 shutdown_connections 2021-02-10T11:21:10.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.911 7f6850317700 1 -- 172.21.2.101:0/1281744831 wait complete. 2021-02-10T11:21:10.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.925 7f6850317700 1 Processor -- start 2021-02-10T11:21:10.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.925 7f6850317700 1 -- start start 2021-02-10T11:21:10.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.926 7f6850317700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68480b20b0 0x7f68480ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:10.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.926 7f6850317700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68480b24b0 0x7f6848191dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:10.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.926 7f6850317700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68480acf90 0x7f68481923e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:10.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.926 7f6850317700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68480b1b30 con 0x7f68480b20b0 2021-02-10T11:21:10.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.926 7f6850317700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68481a04e0 con 0x7f68480b24b0 2021-02-10T11:21:10.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.926 7f6850317700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68481a06c0 con 0x7f68480acf90 2021-02-10T11:21:10.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.926 7f684e133700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68480b20b0 0x7f68480ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:10.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.926 7f684e133700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68480b20b0 0x7f68480ac970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46792/0 (socket says 172.21.2.101:46792) 2021-02-10T11:21:10.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.926 7f684e133700 1 -- 172.21.2.101:0/3099936016 learned_addr learned my addr 172.21.2.101:0/3099936016 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:21:10.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.926 7f684e133700 1 -- 172.21.2.101:0/3099936016 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68480acf90 msgr2=0x7f68481923e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:10.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.926 7f684e934700 1 --2- 172.21.2.101:0/3099936016 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68480acf90 0x7f68481923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:10.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.926 7f684d932700 1 --2- 172.21.2.101:0/3099936016 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68480b24b0 0x7f6848191dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:10.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.926 7f684e133700 1 --2- 172.21.2.101:0/3099936016 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68480acf90 0x7f68481923e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:10.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.926 7f684e133700 1 -- 172.21.2.101:0/3099936016 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68480b24b0 msgr2=0x7f6848191dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:10.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.926 7f684e133700 1 --2- 172.21.2.101:0/3099936016 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68480b24b0 0x7f6848191dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:10.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.926 7f684e133700 1 -- 172.21.2.101:0/3099936016 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6848192be0 con 0x7f68480b20b0 2021-02-10T11:21:10.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.926 7f684e934700 1 --2- 172.21.2.101:0/3099936016 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68480acf90 0x7f68481923e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:21:10.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.926 7f684e133700 1 --2- 172.21.2.101:0/3099936016 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68480b20b0 0x7f68480ac970 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f683c001c60 tx=0x7f683c00a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:10.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.927 7f683affd700 1 -- 172.21.2.101:0/3099936016 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f683c00acc0 con 0x7f68480b20b0 2021-02-10T11:21:10.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.927 7f683affd700 1 -- 172.21.2.101:0/3099936016 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683c004140 con 0x7f68480b20b0 2021-02-10T11:21:10.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.927 7f683affd700 1 -- 172.21.2.101:0/3099936016 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f683c004620 con 0x7f68480b20b0 2021-02-10T11:21:10.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.927 7f6850317700 1 -- 172.21.2.101:0/3099936016 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6848192be0 con 0x7f68480b20b0 2021-02-10T11:21:10.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.927 7f6850317700 1 -- 172.21.2.101:0/3099936016 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6848193170 con 0x7f68480b20b0 2021-02-10T11:21:10.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.927 7f683affd700 1 -- 172.21.2.101:0/3099936016 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f683c004320 con 0x7f68480b20b0 2021-02-10T11:21:10.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.927 7f683affd700 1 --2- 172.21.2.101:0/3099936016 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68300186c0 0x7f683001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:10.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.927 7f683affd700 1 -- 172.21.2.101:0/3099936016 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f683c032560 con 0x7f68480b20b0 2021-02-10T11:21:10.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.928 7f684d932700 1 --2- 172.21.2.101:0/3099936016 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68300186c0 0x7f683001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:10.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.928 7f684d932700 1 --2- 172.21.2.101:0/3099936016 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68300186c0 0x7f683001abc0 crc :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:10.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.959 7f6850317700 1 -- 172.21.2.101:0/3099936016 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6848105af0 con 0x7f68480b20b0 2021-02-10T11:21:10.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:10.961 7f683affd700 1 -- 172.21.2.101:0/3099936016 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f683c000960 con 0x7f68480b20b0 2021-02-10T11:21:11.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.088 7f6850317700 1 -- 172.21.2.101:0/3099936016 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839278", "target": ["mgr", ""]}) v1 -- 0x7f68480b1b30 con 0x7f68300186c0 2021-02-10T11:21:11.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.117 7f683affd700 1 -- 172.21.2.101:0/3099936016 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f6844009150 con 0x7f68300186c0 2021-02-10T11:21:11.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.121 7f6850317700 1 -- 172.21.2.101:0/3099936016 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68300186c0 msgr2=0x7f683001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:11.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.121 7f6850317700 1 --2- 172.21.2.101:0/3099936016 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68300186c0 0x7f683001abc0 crc :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:11.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.121 7f6850317700 1 -- 172.21.2.101:0/3099936016 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68480b20b0 msgr2=0x7f68480ac970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:11.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.121 7f6850317700 1 --2- 172.21.2.101:0/3099936016 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68480b20b0 0x7f68480ac970 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f683c001c60 tx=0x7f683c00a8d0).stop 2021-02-10T11:21:11.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.121 7f6850317700 1 -- 172.21.2.101:0/3099936016 shutdown_connections 2021-02-10T11:21:11.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.121 7f6850317700 1 --2- 172.21.2.101:0/3099936016 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68300186c0 0x7f683001abc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:11.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.121 7f6850317700 1 --2- 172.21.2.101:0/3099936016 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68480acf90 0x7f68481923e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:11.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.121 7f6850317700 1 --2- 172.21.2.101:0/3099936016 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68480b24b0 0x7f6848191dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:11.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.121 7f6850317700 1 --2- 172.21.2.101:0/3099936016 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68480b20b0 0x7f68480ac970 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:11.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.121 7f6850317700 1 -- 172.21.2.101:0/3099936016 >> 172.21.2.101:0/3099936016 conn(0x7f68480bf4e0 msgr2=0x7f68480be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:11.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.121 7f6850317700 1 -- 172.21.2.101:0/3099936016 shutdown_connections 2021-02-10T11:21:11.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.121 7f6850317700 1 -- 172.21.2.101:0/3099936016 wait complete. 2021-02-10T11:21:11.137 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839278 2021-02-10T11:21:11.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.543 7fd99aec6700 1 Processor -- start 2021-02-10T11:21:11.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.543 7fd99aec6700 1 -- start start 2021-02-10T11:21:11.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.544 7fd99aec6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9941b0870 0x7fd9940f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:11.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.544 7fd99aec6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd9940efe00 0x7fd9940f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:11.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.544 7fd99aec6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9941b0c70 con 0x7fd9941b0470 2021-02-10T11:21:11.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.544 7fd99aec6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9940f0b50 con 0x7fd9940efe00 2021-02-10T11:21:11.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.544 7fd99aec6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9940f0820 con 0x7fd9941b0870 2021-02-10T11:21:11.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.544 7fd9994e3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd9940efe00 0x7fd9940f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:11.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.544 7fd9994e3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd9940efe00 0x7fd9940f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46802/0 (socket says 172.21.2.101:46802) 2021-02-10T11:21:11.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.544 7fd9994e3700 1 -- 172.21.2.101:0/3077046447 learned_addr learned my addr 172.21.2.101:0/3077046447 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:21:11.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.544 7fd9994e3700 1 -- 172.21.2.101:0/3077046447 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9941b0870 msgr2=0x7fd9940f20e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:21:11.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.544 7fd9994e3700 1 --2- 172.21.2.101:0/3077046447 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9941b0870 0x7fd9940f20e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:11.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.544 7fd9994e3700 1 -- 172.21.2.101:0/3077046447 >> v1:172.21.2.108:6789/0 conn(0x7fd9941b0470 legacy=0x7fd9940ef610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:21:11.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.544 7fd9994e3700 1 -- 172.21.2.101:0/3077046447 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9940e52e0 con 0x7fd9940efe00 2021-02-10T11:21:11.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.544 7fd9994e3700 1 --2- 172.21.2.101:0/3077046447 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9940efe00 0x7fd9940f0200 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7fd98c006f00 tx=0x7fd98c009f10).ready entity=mon.0 client_cookie=5484953746afe5d6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:11.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.544 7fd9937fe700 1 -- 172.21.2.101:0/3077046447 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd98c00b3b0 con 0x7fd9940efe00 2021-02-10T11:21:11.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.544 7fd9937fe700 1 -- 172.21.2.101:0/3077046447 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98c00b630 con 0x7fd9940efe00 2021-02-10T11:21:11.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.544 7fd9937fe700 1 -- 172.21.2.101:0/3077046447 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd98c00bb50 con 0x7fd9940efe00 2021-02-10T11:21:11.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.544 7fd99aec6700 1 -- 172.21.2.101:0/3077046447 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9940efe00 msgr2=0x7fd9940f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:11.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.544 7fd99aec6700 1 --2- 172.21.2.101:0/3077046447 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9940efe00 0x7fd9940f0200 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7fd98c006f00 tx=0x7fd98c009f10).stop 2021-02-10T11:21:11.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.545 7fd99aec6700 1 -- 172.21.2.101:0/3077046447 shutdown_connections 2021-02-10T11:21:11.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.545 7fd99aec6700 1 --2- 172.21.2.101:0/3077046447 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9941b0870 0x7fd9940f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:11.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.545 7fd99aec6700 1 --2- 172.21.2.101:0/3077046447 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9940efe00 0x7fd9940f0200 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:11.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.545 7fd99aec6700 1 -- 172.21.2.101:0/3077046447 >> 172.21.2.101:0/3077046447 conn(0x7fd9940ce3e0 msgr2=0x7fd9940e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:11.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.545 7fd99aec6700 1 -- 172.21.2.101:0/3077046447 shutdown_connections 2021-02-10T11:21:11.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.545 7fd99aec6700 1 -- 172.21.2.101:0/3077046447 wait complete. 2021-02-10T11:21:11.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.559 7fd99aec6700 1 Processor -- start 2021-02-10T11:21:11.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.560 7fd99aec6700 1 -- start start 2021-02-10T11:21:11.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.560 7fd99aec6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd994094fb0 0x7fd99409e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:11.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.560 7fd99aec6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd9940c2be0 0x7fd9940c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:11.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.560 7fd99aec6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9940c3600 0x7fd9940a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:11.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.560 7fd99aec6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99409ebf0 con 0x7fd9940c3600 2021-02-10T11:21:11.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.560 7fd99aec6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9940aa2a0 con 0x7fd9940c2be0 2021-02-10T11:21:11.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.560 7fd99aec6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9940aa4c0 con 0x7fd994094fb0 2021-02-10T11:21:11.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.560 7fd9994e3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9940c3600 0x7fd9940a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:11.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.560 7fd9994e3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9940c3600 0x7fd9940a9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46808/0 (socket says 172.21.2.101:46808) 2021-02-10T11:21:11.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.560 7fd9994e3700 1 -- 172.21.2.101:0/489089646 learned_addr learned my addr 172.21.2.101:0/489089646 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:21:11.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.560 7fd9994e3700 1 -- 172.21.2.101:0/489089646 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd994094fb0 msgr2=0x7fd99409e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:11.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.560 7fd998ce2700 1 --2- 172.21.2.101:0/489089646 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd994094fb0 0x7fd99409e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:11.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.561 7fd9994e3700 1 --2- 172.21.2.101:0/489089646 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd994094fb0 0x7fd99409e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:11.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.561 7fd9994e3700 1 -- 172.21.2.101:0/489089646 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd9940c2be0 msgr2=0x7fd9940c2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:11.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.561 7fd9994e3700 1 --2- 172.21.2.101:0/489089646 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd9940c2be0 0x7fd9940c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:11.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.561 7fd9994e3700 1 -- 172.21.2.101:0/489089646 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9940aa6e0 con 0x7fd9940c3600 2021-02-10T11:21:11.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.561 7fd998ce2700 1 --2- 172.21.2.101:0/489089646 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd994094fb0 0x7fd99409e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:21:11.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.561 7fd9994e3700 1 --2- 172.21.2.101:0/489089646 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9940c3600 0x7fd9940a9c80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7fd98c009390 tx=0x7fd98c00bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:11.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.561 7fd9917fa700 1 -- 172.21.2.101:0/489089646 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd98c003030 con 0x7fd9940c3600 2021-02-10T11:21:11.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.561 7fd9917fa700 1 -- 172.21.2.101:0/489089646 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98c0068e0 con 0x7fd9940c3600 2021-02-10T11:21:11.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.561 7fd9917fa700 1 -- 172.21.2.101:0/489089646 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd98c00c570 con 0x7fd9940c3600 2021-02-10T11:21:11.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.561 7fd99aec6700 1 -- 172.21.2.101:0/489089646 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9940aa6e0 con 0x7fd9940c3600 2021-02-10T11:21:11.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.561 7fd99aec6700 1 -- 172.21.2.101:0/489089646 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9940aa6e0 con 0x7fd9940c3600 2021-02-10T11:21:11.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.562 7fd9917fa700 1 -- 172.21.2.101:0/489089646 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd98c001270 con 0x7fd9940c3600 2021-02-10T11:21:11.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.562 7fd9917fa700 1 --2- 172.21.2.101:0/489089646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd9780186c0 0x7fd97801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:11.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.562 7fd9917fa700 1 -- 172.21.2.101:0/489089646 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd98c0328e0 con 0x7fd9940c3600 2021-02-10T11:21:11.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.562 7fd998ce2700 1 --2- 172.21.2.101:0/489089646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd9780186c0 0x7fd97801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:11.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.562 7fd998ce2700 1 --2- 172.21.2.101:0/489089646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd9780186c0 0x7fd97801abc0 crc :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:11.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.594 7fd99aec6700 1 -- 172.21.2.101:0/489089646 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9940edd10 con 0x7fd9940c3600 2021-02-10T11:21:11.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.596 7fd9917fa700 1 -- 172.21.2.101:0/489089646 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd98c0234d0 con 0x7fd9940c3600 2021-02-10T11:21:11.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.724 7fd99aec6700 1 -- 172.21.2.101:0/489089646 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839278", "target": ["mgr", ""]}) v1 -- 0x7fd99409ebf0 con 0x7fd9780186c0 2021-02-10T11:21:11.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.727 7fd9917fa700 1 -- 172.21.2.101:0/489089646 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fd97c006b20 con 0x7fd9780186c0 2021-02-10T11:21:11.738 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839278/af02cadb-f78e-4ef1-820b-2ec6971fb6bb 2021-02-10T11:21:11.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.733 7fd99aec6700 1 -- 172.21.2.101:0/489089646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd9780186c0 msgr2=0x7fd97801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:11.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.733 7fd99aec6700 1 --2- 172.21.2.101:0/489089646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd9780186c0 0x7fd97801abc0 crc :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:11.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.733 7fd99aec6700 1 -- 172.21.2.101:0/489089646 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9940c3600 msgr2=0x7fd9940a9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:11.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.733 7fd99aec6700 1 --2- 172.21.2.101:0/489089646 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9940c3600 0x7fd9940a9c80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7fd98c009390 tx=0x7fd98c00bd40).stop 2021-02-10T11:21:11.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.733 7fd99aec6700 1 -- 172.21.2.101:0/489089646 shutdown_connections 2021-02-10T11:21:11.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.733 7fd99aec6700 1 --2- 172.21.2.101:0/489089646 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9940c3600 0x7fd9940a9c80 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:11.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.733 7fd99aec6700 1 --2- 172.21.2.101:0/489089646 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd9940c2be0 0x7fd9940c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:11.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.733 7fd99aec6700 1 --2- 172.21.2.101:0/489089646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd9780186c0 0x7fd97801abc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:11.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.733 7fd99aec6700 1 --2- 172.21.2.101:0/489089646 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd994094fb0 0x7fd99409e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:11.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.733 7fd99aec6700 1 -- 172.21.2.101:0/489089646 >> 172.21.2.101:0/489089646 conn(0x7fd99409f630 msgr2=0x7fd9940a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:11.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.733 7fd99aec6700 1 -- 172.21.2.101:0/489089646 shutdown_connections 2021-02-10T11:21:11.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:11.733 7fd99aec6700 1 -- 172.21.2.101:0/489089646 wait complete. 2021-02-10T11:21:11.748 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839278 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839278/af02cadb-f78e-4ef1-820b-2ec6971fb6bb 2021-02-10T11:21:11.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:11.748 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:11.748 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:12.083 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:12.083 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:12.083 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839278/af02cadb-f78e-4ef1-820b-2ec6971fb6bb/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:12.464 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:12.464 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:12.464 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241239 s, 43.5 MB/s 2021-02-10T11:21:12.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:12.465 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:12.465 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:12.802 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:12.802 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:12.802 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839278/af02cadb-f78e-4ef1-820b-2ec6971fb6bb/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:13.177 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:13.178 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:13.178 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.02457 s, 42.7 MB/s 2021-02-10T11:21:13.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:13.179 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:13.179 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:13.515 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:13.515 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:13.515 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839278/af02cadb-f78e-4ef1-820b-2ec6971fb6bb/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:13.887 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:13.887 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:13.887 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240185 s, 43.7 MB/s 2021-02-10T11:21:13.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:13.888 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:13.888 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:14.223 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:14.223 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:14.223 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839278/af02cadb-f78e-4ef1-820b-2ec6971fb6bb/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:14.599 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:14.600 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:14.600 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249432 s, 42.0 MB/s 2021-02-10T11:21:14.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:14.601 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:14.601 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:14.936 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:14.936 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:14.937 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839278/af02cadb-f78e-4ef1-820b-2ec6971fb6bb/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:15.311 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:15.312 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:15.312 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238268 s, 44.0 MB/s 2021-02-10T11:21:15.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:15.313 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:15.313 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:15.648 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:15.648 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:15.648 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839278/af02cadb-f78e-4ef1-820b-2ec6971fb6bb/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:16.022 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:16.022 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:16.022 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0219615 s, 47.7 MB/s 2021-02-10T11:21:16.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:16.023 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:16.023 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:16.359 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:16.360 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:16.360 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839278/af02cadb-f78e-4ef1-820b-2ec6971fb6bb/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:16.736 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:16.736 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:16.736 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0255726 s, 41.0 MB/s 2021-02-10T11:21:16.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:16.737 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:16.737 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:17.072 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:17.072 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:17.072 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839278/af02cadb-f78e-4ef1-820b-2ec6971fb6bb/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:17.448 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:17.448 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:17.448 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251653 s, 41.7 MB/s 2021-02-10T11:21:17.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:17.449 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:17.449 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:17.783 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:17.784 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:17.784 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839278/af02cadb-f78e-4ef1-820b-2ec6971fb6bb/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:18.158 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:18.158 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:18.158 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231343 s, 45.3 MB/s 2021-02-10T11:21:18.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:18.159 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:18.159 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:18.496 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:18.496 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:18.496 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839278/af02cadb-f78e-4ef1-820b-2ec6971fb6bb/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:18.868 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:18.868 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:18.868 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244891 s, 42.8 MB/s 2021-02-10T11:21:18.869 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839279 2021-02-10T11:21:19.109 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:21:19.118 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:21:19.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.275 7f947583f700 1 Processor -- start 2021-02-10T11:21:19.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.275 7f947583f700 1 -- start start 2021-02-10T11:21:19.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.275 7f947583f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94700f05f0 0x7f94700f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:19.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.275 7f947583f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94701b0c70 con 0x7f94701b0470 2021-02-10T11:21:19.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.276 7f947583f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94700f0b50 con 0x7f94701b0870 2021-02-10T11:21:19.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.276 7f947583f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94700e5ea0 con 0x7f94700f05f0 2021-02-10T11:21:19.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.276 7f946f574700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f94701b0470 0x7f94700ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47306/0 (socket says 172.21.2.101:47306) 2021-02-10T11:21:19.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.276 7f946f574700 1 -- 172.21.2.101:0/1528309263 learned_addr learned my addr 172.21.2.101:0/1528309263 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:21:19.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.276 7f946e572700 1 -- 172.21.2.101:0/1528309263 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230067236 0 0) 0x7f9468001500 con 0x7f94701b0470 2021-02-10T11:21:19.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.276 7f946e572700 1 -- 172.21.2.101:0/1528309263 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9458002fe0 con 0x7f94701b0470 2021-02-10T11:21:19.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.276 7f946e572700 1 -- 172.21.2.101:0/1528309263 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1639297612 0 0) 0x7f9468001460 con 0x7f94701b0470 2021-02-10T11:21:19.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.276 7f946fd75700 1 --2- 172.21.2.101:0/1528309263 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94700f05f0 0x7f94700f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:19.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.276 7f946e572700 1 -- 172.21.2.101:0/1528309263 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94700f05f0 msgr2=0x7f94700f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:21:19.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.276 7f946e572700 1 --2- 172.21.2.101:0/1528309263 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94700f05f0 0x7f94700f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:19.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.276 7f946e572700 1 -- 172.21.2.101:0/1528309263 >> v1:172.21.2.108:6789/0 conn(0x7f94701b0870 legacy=0x7f94700efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:21:19.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.276 7f946e572700 1 -- 172.21.2.101:0/1528309263 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94700e52e0 con 0x7f94701b0470 2021-02-10T11:21:19.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.276 7f946e572700 1 -- 172.21.2.101:0/1528309263 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f9468001980 con 0x7f94701b0470 2021-02-10T11:21:19.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.276 7f946e572700 1 -- 172.21.2.101:0/1528309263 >> v1:172.21.2.101:6789/0 conn(0x7f94701b0470 legacy=0x7f94700ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:19.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.276 7f946e572700 1 --2- 172.21.2.101:0/1528309263 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9458003b20 0x7f9458005f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:19.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.276 7f946e572700 1 --2- 172.21.2.101:0/1528309263 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9458006830 0x7f9458008d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:19.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.276 7f946e572700 1 --2- 172.21.2.101:0/1528309263 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9458009630 0x7f945800bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:19.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.276 7f946e572700 1 -- 172.21.2.101:0/1528309263 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945800c370 con 0x7f9458003b20 2021-02-10T11:21:19.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.276 7f946e572700 1 -- 172.21.2.101:0/1528309263 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945800c590 con 0x7f9458006830 2021-02-10T11:21:19.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.276 7f946e572700 1 -- 172.21.2.101:0/1528309263 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945800c7b0 con 0x7f9458009630 2021-02-10T11:21:19.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.276 7f946fd75700 1 --2- 172.21.2.101:0/1528309263 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94700f05f0 0x7f94700f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:21:19.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.276 7f946f574700 1 --2- 172.21.2.101:0/1528309263 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9458003b20 0x7f9458005f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:19.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.277 7f946ed73700 1 --2- 172.21.2.101:0/1528309263 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9458006830 0x7f9458008d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:19.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.277 7f946fd75700 1 --2- 172.21.2.101:0/1528309263 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9458009630 0x7f945800bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:19.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.277 7f946f574700 1 -- 172.21.2.101:0/1528309263 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9458009630 msgr2=0x7f945800bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:21:19.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.277 7f946f574700 1 --2- 172.21.2.101:0/1528309263 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9458009630 0x7f945800bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:19.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.277 7f946f574700 1 -- 172.21.2.101:0/1528309263 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9458006830 msgr2=0x7f9458008d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:21:19.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.277 7f946f574700 1 --2- 172.21.2.101:0/1528309263 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9458006830 0x7f9458008d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:19.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.277 7f946f574700 1 -- 172.21.2.101:0/1528309263 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945800ca50 con 0x7f9458003b20 2021-02-10T11:21:19.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.277 7f946f574700 1 --2- 172.21.2.101:0/1528309263 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9458003b20 0x7f9458005f90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f9468002200 tx=0x7f946804e320).ready entity=mon.0 client_cookie=1e8a782e2f86e9ec server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:19.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.277 7f946e572700 1 -- 172.21.2.101:0/1528309263 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f94680070d0 con 0x7f9458003b20 2021-02-10T11:21:19.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.277 7f946e572700 1 -- 172.21.2.101:0/1528309263 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9468007350 con 0x7f9458003b20 2021-02-10T11:21:19.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.277 7f946e572700 1 -- 172.21.2.101:0/1528309263 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f94680049a0 con 0x7f9458003b20 2021-02-10T11:21:19.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.277 7f947583f700 1 -- 172.21.2.101:0/1528309263 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9458003b20 msgr2=0x7f9458005f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:19.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.277 7f947583f700 1 --2- 172.21.2.101:0/1528309263 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9458003b20 0x7f9458005f90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f9468002200 tx=0x7f946804e320).stop 2021-02-10T11:21:19.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.277 7f946fd75700 1 -- 172.21.2.101:0/1528309263 reap_dead start 2021-02-10T11:21:19.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.277 7f947583f700 1 -- 172.21.2.101:0/1528309263 shutdown_connections 2021-02-10T11:21:19.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.277 7f947583f700 1 -- 172.21.2.101:0/1528309263 >> 172.21.2.101:0/1528309263 conn(0x7f94700ce3e0 msgr2=0x7f94700e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:19.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.278 7f947583f700 1 -- 172.21.2.101:0/1528309263 shutdown_connections 2021-02-10T11:21:19.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.278 7f947583f700 1 -- 172.21.2.101:0/1528309263 wait complete. 2021-02-10T11:21:19.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.292 7f947583f700 1 Processor -- start 2021-02-10T11:21:19.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.292 7f947583f700 1 -- start start 2021-02-10T11:21:19.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.293 7f947583f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94700b7db0 0x7f94700b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:19.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.293 7f947583f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94700b8880 0x7f94701a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:19.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.293 7f947583f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94700b8c80 0x7f94701a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:19.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.293 7f947583f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94700b79b0 con 0x7f94700b8880 2021-02-10T11:21:19.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.293 7f947583f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94701a8090 con 0x7f94700b8c80 2021-02-10T11:21:19.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.293 7f947583f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94701a82b0 con 0x7f94700b7db0 2021-02-10T11:21:19.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.293 7f946f574700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94700b7db0 0x7f94700b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:19.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.293 7f946ed73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94700b8880 0x7f94701a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:19.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.293 7f946f574700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94700b7db0 0x7f94700b8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:39556/0 (socket says 172.21.2.101:39556) 2021-02-10T11:21:19.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.293 7f946f574700 1 -- 172.21.2.101:0/3851579609 learned_addr learned my addr 172.21.2.101:0/3851579609 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:21:19.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.293 7f946fd75700 1 --2- 172.21.2.101:0/3851579609 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94700b8c80 0x7f94701a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:19.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.293 7f946ed73700 1 -- 172.21.2.101:0/3851579609 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94700b7db0 msgr2=0x7f94700b8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:19.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.293 7f946ed73700 1 --2- 172.21.2.101:0/3851579609 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94700b7db0 0x7f94700b8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:19.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.293 7f946ed73700 1 -- 172.21.2.101:0/3851579609 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94700b8c80 msgr2=0x7f94701a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:19.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.293 7f946ed73700 1 --2- 172.21.2.101:0/3851579609 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94700b8c80 0x7f94701a7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:19.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.293 7f946ed73700 1 -- 172.21.2.101:0/3851579609 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9470190c10 con 0x7f94700b8880 2021-02-10T11:21:19.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.294 7f946ed73700 1 --2- 172.21.2.101:0/3851579609 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94700b8880 0x7f94701a7450 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f9464009540 tx=0x7f946400b890).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:19.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.294 7f9457fff700 1 -- 172.21.2.101:0/3851579609 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f946400bc80 con 0x7f94700b8880 2021-02-10T11:21:19.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.294 7f9457fff700 1 -- 172.21.2.101:0/3851579609 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9464006030 con 0x7f94700b8880 2021-02-10T11:21:19.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.294 7f947583f700 1 -- 172.21.2.101:0/3851579609 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9470190c10 con 0x7f94700b8880 2021-02-10T11:21:19.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.294 7f947583f700 1 -- 172.21.2.101:0/3851579609 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9470190c10 con 0x7f94700b8880 2021-02-10T11:21:19.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.294 7f9457fff700 1 -- 172.21.2.101:0/3851579609 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f94640052e0 con 0x7f94700b8880 2021-02-10T11:21:19.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.295 7f9457fff700 1 -- 172.21.2.101:0/3851579609 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9464006210 con 0x7f94700b8880 2021-02-10T11:21:19.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.295 7f9457fff700 1 --2- 172.21.2.101:0/3851579609 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9450018730 0x7f945001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:19.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.295 7f9457fff700 1 -- 172.21.2.101:0/3851579609 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f94640326a0 con 0x7f94700b8880 2021-02-10T11:21:19.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.295 7f946f574700 1 --2- 172.21.2.101:0/3851579609 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9450018730 0x7f945001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:19.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.295 7f946f574700 1 --2- 172.21.2.101:0/3851579609 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9450018730 0x7f945001abc0 crc :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:19.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.325 7f947583f700 1 -- 172.21.2.101:0/3851579609 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9470105b30 con 0x7f94700b8880 2021-02-10T11:21:19.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.328 7f9457fff700 1 -- 172.21.2.101:0/3851579609 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9464006030 con 0x7f94700b8880 2021-02-10T11:21:19.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.457 7f947583f700 1 -- 172.21.2.101:0/3851579609 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839279", "target": ["mgr", ""]}) v1 -- 0x7f94701a8090 con 0x7f9450018730 2021-02-10T11:21:19.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.483 7f9457fff700 1 -- 172.21.2.101:0/3851579609 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f9468007de0 con 0x7f9450018730 2021-02-10T11:21:19.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.490 7f947583f700 1 -- 172.21.2.101:0/3851579609 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9450018730 msgr2=0x7f945001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:19.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.490 7f947583f700 1 --2- 172.21.2.101:0/3851579609 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9450018730 0x7f945001abc0 crc :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:19.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.490 7f947583f700 1 -- 172.21.2.101:0/3851579609 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94700b8880 msgr2=0x7f94701a7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:19.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.491 7f947583f700 1 --2- 172.21.2.101:0/3851579609 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94700b8880 0x7f94701a7450 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f9464009540 tx=0x7f946400b890).stop 2021-02-10T11:21:19.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.491 7f947583f700 1 -- 172.21.2.101:0/3851579609 shutdown_connections 2021-02-10T11:21:19.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.491 7f947583f700 1 --2- 172.21.2.101:0/3851579609 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94700b8c80 0x7f94701a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:19.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.491 7f947583f700 1 --2- 172.21.2.101:0/3851579609 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94700b8880 0x7f94701a7450 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:19.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.491 7f947583f700 1 --2- 172.21.2.101:0/3851579609 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9450018730 0x7f945001abc0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:19.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.491 7f947583f700 1 --2- 172.21.2.101:0/3851579609 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94700b7db0 0x7f94700b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:19.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.491 7f947583f700 1 -- 172.21.2.101:0/3851579609 >> 172.21.2.101:0/3851579609 conn(0x7f9470094e10 msgr2=0x7f94700a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:19.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.491 7f947583f700 1 -- 172.21.2.101:0/3851579609 shutdown_connections 2021-02-10T11:21:19.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.491 7f947583f700 1 -- 172.21.2.101:0/3851579609 wait complete. 2021-02-10T11:21:19.506 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839279 2021-02-10T11:21:19.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.954 7f29880b6700 1 Processor -- start 2021-02-10T11:21:19.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.954 7f29880b6700 1 -- start start 2021-02-10T11:21:19.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.954 7f29880b6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f29801b0470 0x7f29801b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:19.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.954 7f29880b6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29800f20e0 0x7f29800f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:19.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.954 7f29880b6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29800f0b50 con 0x7f29800ef610 2021-02-10T11:21:19.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.954 7f29880b6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29800f0820 con 0x7f29801b0470 2021-02-10T11:21:19.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.954 7f29880b6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29800f24e0 con 0x7f29800f20e0 2021-02-10T11:21:19.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.955 7f2985ed2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f29801b0470 0x7f29801b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:19.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.955 7f2985ed2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f29801b0470 0x7f29801b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46832/0 (socket says 172.21.2.101:46832) 2021-02-10T11:21:19.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.955 7f2985ed2700 1 -- 172.21.2.101:0/2224483725 learned_addr learned my addr 172.21.2.101:0/2224483725 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:21:19.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.955 7f2985ed2700 1 -- 172.21.2.101:0/2224483725 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29800f20e0 msgr2=0x7f29800f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:21:19.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.955 7f2985ed2700 1 --2- 172.21.2.101:0/2224483725 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29800f20e0 0x7f29800f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:19.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.955 7f2985ed2700 1 -- 172.21.2.101:0/2224483725 >> v1:172.21.2.108:6789/0 conn(0x7f29800ef610 legacy=0x7f29800efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:21:19.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.955 7f2985ed2700 1 -- 172.21.2.101:0/2224483725 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29800e52e0 con 0x7f29801b0470 2021-02-10T11:21:19.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.955 7f2985ed2700 1 --2- 172.21.2.101:0/2224483725 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29801b0470 0x7f29801b0870 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f297c006700 tx=0x7f297c009b80).ready entity=mon.0 client_cookie=99a9f439e31cafee server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:19.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.955 7f2984ed0700 1 -- 172.21.2.101:0/2224483725 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f297c00c0c0 con 0x7f29801b0470 2021-02-10T11:21:19.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.955 7f2984ed0700 1 -- 172.21.2.101:0/2224483725 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297c00c380 con 0x7f29801b0470 2021-02-10T11:21:19.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.955 7f2984ed0700 1 -- 172.21.2.101:0/2224483725 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f297c00c8a0 con 0x7f29801b0470 2021-02-10T11:21:19.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.956 7f29880b6700 1 -- 172.21.2.101:0/2224483725 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29801b0470 msgr2=0x7f29801b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:19.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.956 7f29880b6700 1 --2- 172.21.2.101:0/2224483725 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29801b0470 0x7f29801b0870 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f297c006700 tx=0x7f297c009b80).stop 2021-02-10T11:21:19.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.956 7f29880b6700 1 -- 172.21.2.101:0/2224483725 shutdown_connections 2021-02-10T11:21:19.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.956 7f29880b6700 1 --2- 172.21.2.101:0/2224483725 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29800f20e0 0x7f29800f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:19.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.956 7f29880b6700 1 --2- 172.21.2.101:0/2224483725 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29801b0470 0x7f29801b0870 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:19.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.956 7f29880b6700 1 -- 172.21.2.101:0/2224483725 >> 172.21.2.101:0/2224483725 conn(0x7f29800ce3e0 msgr2=0x7f29800e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:19.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.956 7f29880b6700 1 -- 172.21.2.101:0/2224483725 shutdown_connections 2021-02-10T11:21:19.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.956 7f29880b6700 1 -- 172.21.2.101:0/2224483725 wait complete. 2021-02-10T11:21:19.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.971 7f29880b6700 1 Processor -- start 2021-02-10T11:21:19.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.971 7f29880b6700 1 -- start start 2021-02-10T11:21:19.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.971 7f29880b6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29800afa70 0x7f29800afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:19.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.971 7f29880b6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29800b0490 0x7f29800a78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:19.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.971 7f29880b6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29800a7f10 0x7f29800a8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:19.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.971 7f29880b6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29800a8a40 con 0x7f29800afa70 2021-02-10T11:21:19.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.971 7f29880b6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29800a8fc0 con 0x7f29800b0490 2021-02-10T11:21:19.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.971 7f29880b6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29800a91a0 con 0x7f29800a7f10 2021-02-10T11:21:19.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.972 7f2985ed2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29800afa70 0x7f29800afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:19.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.972 7f2985ed2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29800afa70 0x7f29800afe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46836/0 (socket says 172.21.2.101:46836) 2021-02-10T11:21:19.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.972 7f2985ed2700 1 -- 172.21.2.101:0/3521826205 learned_addr learned my addr 172.21.2.101:0/3521826205 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:21:19.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.972 7f2985ed2700 1 -- 172.21.2.101:0/3521826205 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29800a7f10 msgr2=0x7f29800a8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:19.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.972 7f2985ed2700 1 --2- 172.21.2.101:0/3521826205 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29800a7f10 0x7f29800a8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:19.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.972 7f2985ed2700 1 -- 172.21.2.101:0/3521826205 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29800b0490 msgr2=0x7f29800a78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:19.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.972 7f2985ed2700 1 --2- 172.21.2.101:0/3521826205 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29800b0490 0x7f29800a78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:19.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.972 7f2985ed2700 1 -- 172.21.2.101:0/3521826205 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29800a93c0 con 0x7f29800afa70 2021-02-10T11:21:19.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.972 7f2985ed2700 1 --2- 172.21.2.101:0/3521826205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29800afa70 0x7f29800afe70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f297c00b580 tx=0x7f297c00b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:19.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.972 7f29767fc700 1 -- 172.21.2.101:0/3521826205 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f297c00bd10 con 0x7f29800afa70 2021-02-10T11:21:19.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.972 7f29767fc700 1 -- 172.21.2.101:0/3521826205 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297c002030 con 0x7f29800afa70 2021-02-10T11:21:19.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.972 7f29767fc700 1 -- 172.21.2.101:0/3521826205 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f297c002480 con 0x7f29800afa70 2021-02-10T11:21:19.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.972 7f29880b6700 1 -- 172.21.2.101:0/3521826205 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29800a93c0 con 0x7f29800afa70 2021-02-10T11:21:19.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.973 7f29880b6700 1 -- 172.21.2.101:0/3521826205 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29800a93c0 con 0x7f29800afa70 2021-02-10T11:21:19.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.973 7f29767fc700 1 -- 172.21.2.101:0/3521826205 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f297c000c90 con 0x7f29800afa70 2021-02-10T11:21:19.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.973 7f29767fc700 1 --2- 172.21.2.101:0/3521826205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2968018730 0x7f296801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:19.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.973 7f29767fc700 1 -- 172.21.2.101:0/3521826205 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f297c032800 con 0x7f29800afa70 2021-02-10T11:21:19.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.973 7f29856d1700 1 --2- 172.21.2.101:0/3521826205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2968018730 0x7f296801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:19.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:19.974 7f29856d1700 1 --2- 172.21.2.101:0/3521826205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2968018730 0x7f296801ac10 crc :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:20.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:20.004 7f29880b6700 1 -- 172.21.2.101:0/3521826205 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29800e52e0 con 0x7f29800afa70 2021-02-10T11:21:20.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:20.007 7f29767fc700 1 -- 172.21.2.101:0/3521826205 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f297c01f540 con 0x7f29800afa70 2021-02-10T11:21:20.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:20.135 7f29880b6700 1 -- 172.21.2.101:0/3521826205 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839279", "target": ["mgr", ""]}) v1 -- 0x7f29800e52e0 con 0x7f2968018730 2021-02-10T11:21:20.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:20.138 7f29767fc700 1 -- 172.21.2.101:0/3521826205 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f2970006e40 con 0x7f2968018730 2021-02-10T11:21:20.145 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839279/5472b77c-4fe5-48bc-94c6-648f4b73ec44 2021-02-10T11:21:20.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:20.139 7f29880b6700 1 -- 172.21.2.101:0/3521826205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2968018730 msgr2=0x7f296801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:20.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:20.139 7f29880b6700 1 --2- 172.21.2.101:0/3521826205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2968018730 0x7f296801ac10 crc :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:20.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:20.139 7f29880b6700 1 -- 172.21.2.101:0/3521826205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29800afa70 msgr2=0x7f29800afe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:20.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:20.139 7f29880b6700 1 --2- 172.21.2.101:0/3521826205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29800afa70 0x7f29800afe70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f297c00b580 tx=0x7f297c00b920).stop 2021-02-10T11:21:20.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:20.139 7f29880b6700 1 -- 172.21.2.101:0/3521826205 shutdown_connections 2021-02-10T11:21:20.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:20.139 7f29880b6700 1 --2- 172.21.2.101:0/3521826205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2968018730 0x7f296801ac10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:20.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:20.139 7f29880b6700 1 --2- 172.21.2.101:0/3521826205 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29800a7f10 0x7f29800a8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:20.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:20.139 7f29880b6700 1 --2- 172.21.2.101:0/3521826205 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29800b0490 0x7f29800a78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:20.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:20.139 7f29880b6700 1 --2- 172.21.2.101:0/3521826205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29800afa70 0x7f29800afe70 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:20.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:20.139 7f29880b6700 1 -- 172.21.2.101:0/3521826205 >> 172.21.2.101:0/3521826205 conn(0x7f29800ac550 msgr2=0x7f29800acb70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:20.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:20.139 7f29880b6700 1 -- 172.21.2.101:0/3521826205 shutdown_connections 2021-02-10T11:21:20.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:20.140 7f29880b6700 1 -- 172.21.2.101:0/3521826205 wait complete. 2021-02-10T11:21:20.155 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839279 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839279/5472b77c-4fe5-48bc-94c6-648f4b73ec44 2021-02-10T11:21:20.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:20.155 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:20.155 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:20.490 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:20.490 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:20.490 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839279/5472b77c-4fe5-48bc-94c6-648f4b73ec44/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:20.874 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:20.874 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:20.874 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0259144 s, 40.5 MB/s 2021-02-10T11:21:20.875 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:20.875 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:20.875 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:21.211 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:21.211 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:21.211 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839279/5472b77c-4fe5-48bc-94c6-648f4b73ec44/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:21.586 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:21.587 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:21.587 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247586 s, 42.4 MB/s 2021-02-10T11:21:21.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:21.588 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:21.588 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:21.923 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:21.924 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:21.924 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839279/5472b77c-4fe5-48bc-94c6-648f4b73ec44/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:22.297 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:22.298 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:22.298 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0232377 s, 45.1 MB/s 2021-02-10T11:21:22.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:22.299 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:22.299 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:22.635 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:22.635 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:22.635 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839279/5472b77c-4fe5-48bc-94c6-648f4b73ec44/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:23.009 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:23.009 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:23.010 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.022165 s, 47.3 MB/s 2021-02-10T11:21:23.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:23.011 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:23.011 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:23.345 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:23.346 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:23.346 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839279/5472b77c-4fe5-48bc-94c6-648f4b73ec44/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:23.720 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:23.720 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:23.720 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0232527 s, 45.1 MB/s 2021-02-10T11:21:23.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:23.721 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:23.721 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:24.058 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:24.058 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:24.059 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839279/5472b77c-4fe5-48bc-94c6-648f4b73ec44/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:24.432 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:24.432 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:24.432 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244627 s, 42.9 MB/s 2021-02-10T11:21:24.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:24.433 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:24.433 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:24.768 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:24.806 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:24.807 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839279/5472b77c-4fe5-48bc-94c6-648f4b73ec44/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:25.183 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:25.184 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:25.184 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240281 s, 43.6 MB/s 2021-02-10T11:21:25.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:25.185 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:25.185 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:25.520 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:25.520 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:25.520 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839279/5472b77c-4fe5-48bc-94c6-648f4b73ec44/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:25.897 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:25.897 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:25.897 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.025496 s, 41.1 MB/s 2021-02-10T11:21:25.898 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:25.898 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:25.898 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:26.234 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:26.234 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:26.235 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839279/5472b77c-4fe5-48bc-94c6-648f4b73ec44/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:26.611 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:26.611 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:26.611 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252243 s, 41.6 MB/s 2021-02-10T11:21:26.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:26.612 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:26.612 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:26.946 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:26.946 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:26.946 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839279/5472b77c-4fe5-48bc-94c6-648f4b73ec44/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:27.321 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:27.321 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:27.321 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243464 s, 43.1 MB/s 2021-02-10T11:21:27.322 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839280 2021-02-10T11:21:27.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.728 7fcece208700 1 Processor -- start 2021-02-10T11:21:27.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.728 7fcece208700 1 -- start start 2021-02-10T11:21:27.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.728 7fcece208700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcec81b0870 0x7fcec80f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:27.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.729 7fcece208700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec81b0c70 con 0x7fcec80efe00 2021-02-10T11:21:27.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.729 7fcece208700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec80f0b50 con 0x7fcec81b0470 2021-02-10T11:21:27.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.729 7fcece208700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec80e5ea0 con 0x7fcec81b0870 2021-02-10T11:21:27.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.729 7fcecc825700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fcec80efe00 0x7fcec80f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47336/0 (socket says 172.21.2.101:47336) 2021-02-10T11:21:27.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.729 7fcecc825700 1 -- 172.21.2.101:0/4274131065 learned_addr learned my addr 172.21.2.101:0/4274131065 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:21:27.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.729 7fcec77fe700 1 --2- 172.21.2.101:0/4274131065 >> v2:172.21.2.101:3300/0 conn(0x7fcec81b0870 0x7fcec80f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:27.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.729 7fcec6ffd700 1 -- 172.21.2.101:0/4274131065 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706668816 0 0) 0x7fcec0001500 con 0x7fcec80efe00 2021-02-10T11:21:27.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.729 7fcec6ffd700 1 -- 172.21.2.101:0/4274131065 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fceb0002fe0 con 0x7fcec80efe00 2021-02-10T11:21:27.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.729 7fcec77fe700 1 -- 172.21.2.101:0/4274131065 >> v1:172.21.2.108:6789/0 conn(0x7fcec81b0470 legacy=0x7fcec80ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:21:27.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.729 7fcec77fe700 1 -- 172.21.2.101:0/4274131065 >> v1:172.21.2.101:6789/0 conn(0x7fcec80efe00 legacy=0x7fcec80f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:27.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.729 7fcec77fe700 1 -- 172.21.2.101:0/4274131065 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec80e52e0 con 0x7fcec81b0870 2021-02-10T11:21:27.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.729 7fcec77fe700 1 --2- 172.21.2.101:0/4274131065 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcec81b0870 0x7fcec80f20e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fcebc006650 tx=0x7fcebc009a80).ready entity=mon.0 client_cookie=6e57943db62e0bb4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:27.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.730 7fcec6ffd700 1 -- 172.21.2.101:0/4274131065 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcebc0009b0 con 0x7fcec81b0870 2021-02-10T11:21:27.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.730 7fcec6ffd700 1 -- 172.21.2.101:0/4274131065 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcebc00c0c0 con 0x7fcec81b0870 2021-02-10T11:21:27.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.730 7fcec6ffd700 1 -- 172.21.2.101:0/4274131065 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcebc00c5e0 con 0x7fcec81b0870 2021-02-10T11:21:27.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.730 7fcece208700 1 -- 172.21.2.101:0/4274131065 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcec81b0870 msgr2=0x7fcec80f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:27.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.730 7fcece208700 1 --2- 172.21.2.101:0/4274131065 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcec81b0870 0x7fcec80f20e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fcebc006650 tx=0x7fcebc009a80).stop 2021-02-10T11:21:27.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.730 7fcece208700 1 -- 172.21.2.101:0/4274131065 shutdown_connections 2021-02-10T11:21:27.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.730 7fcece208700 1 --2- 172.21.2.101:0/4274131065 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcec81b0870 0x7fcec80f20e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:27.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.730 7fcece208700 1 -- 172.21.2.101:0/4274131065 >> 172.21.2.101:0/4274131065 conn(0x7fcec80ce3e0 msgr2=0x7fcec80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:27.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.730 7fcece208700 1 -- 172.21.2.101:0/4274131065 shutdown_connections 2021-02-10T11:21:27.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.730 7fcece208700 1 -- 172.21.2.101:0/4274131065 wait complete. 2021-02-10T11:21:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.745 7fcece208700 1 Processor -- start 2021-02-10T11:21:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.745 7fcece208700 1 -- start start 2021-02-10T11:21:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.746 7fcece208700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcec8094fb0 0x7fcec809e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.746 7fcece208700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcec80c2be0 0x7fcec80c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.746 7fcece208700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcec80c3600 0x7fcec80a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.746 7fcece208700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec809ebf0 con 0x7fcec8094fb0 2021-02-10T11:21:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.746 7fcece208700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec80aa2a0 con 0x7fcec80c3600 2021-02-10T11:21:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.746 7fcece208700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec80aa4c0 con 0x7fcec80c2be0 2021-02-10T11:21:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.746 7fcec7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcec8094fb0 0x7fcec809e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.746 7fcec7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcec8094fb0 0x7fcec809e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46848/0 (socket says 172.21.2.101:46848) 2021-02-10T11:21:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.746 7fcec7fff700 1 -- 172.21.2.101:0/4245882411 learned_addr learned my addr 172.21.2.101:0/4245882411 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:21:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.746 7fcec7fff700 1 -- 172.21.2.101:0/4245882411 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcec80c2be0 msgr2=0x7fcec80c2fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:21:27.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.746 7fcec7fff700 1 --2- 172.21.2.101:0/4245882411 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcec80c2be0 0x7fcec80c2fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:27.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.746 7fcec7fff700 1 -- 172.21.2.101:0/4245882411 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcec80c3600 msgr2=0x7fcec80a9c80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:21:27.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.746 7fcec7fff700 1 --2- 172.21.2.101:0/4245882411 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcec80c3600 0x7fcec80a9c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:27.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.746 7fcec7fff700 1 -- 172.21.2.101:0/4245882411 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec80aa6e0 con 0x7fcec8094fb0 2021-02-10T11:21:27.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.746 7fcec7fff700 1 --2- 172.21.2.101:0/4245882411 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcec8094fb0 0x7fcec809e5d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fceb800b5b0 tx=0x7fceb800b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:27.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.746 7fcec4ff9700 1 -- 172.21.2.101:0/4245882411 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fceb800bd40 con 0x7fcec8094fb0 2021-02-10T11:21:27.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.747 7fcec4ff9700 1 -- 172.21.2.101:0/4245882411 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb8006020 con 0x7fcec8094fb0 2021-02-10T11:21:27.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.747 7fcec4ff9700 1 -- 172.21.2.101:0/4245882411 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fceb80054a0 con 0x7fcec8094fb0 2021-02-10T11:21:27.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.747 7fcece208700 1 -- 172.21.2.101:0/4245882411 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec80aa6e0 con 0x7fcec8094fb0 2021-02-10T11:21:27.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.747 7fcece208700 1 -- 172.21.2.101:0/4245882411 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec80aac70 con 0x7fcec8094fb0 2021-02-10T11:21:27.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.747 7fcec4ff9700 1 -- 172.21.2.101:0/4245882411 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fceb8006200 con 0x7fcec8094fb0 2021-02-10T11:21:27.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.747 7fcec4ff9700 1 --2- 172.21.2.101:0/4245882411 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcea80186c0 0x7fcea801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:27.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.747 7fcec4ff9700 1 -- 172.21.2.101:0/4245882411 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fceb8032440 con 0x7fcec8094fb0 2021-02-10T11:21:27.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.747 7fcec77fe700 1 --2- 172.21.2.101:0/4245882411 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcea80186c0 0x7fcea801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:27.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.748 7fcec77fe700 1 --2- 172.21.2.101:0/4245882411 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcea80186c0 0x7fcea801abc0 crc :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:27.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.778 7fcece208700 1 -- 172.21.2.101:0/4245882411 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec80a9750 con 0x7fcec8094fb0 2021-02-10T11:21:27.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.781 7fcec4ff9700 1 -- 172.21.2.101:0/4245882411 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fceb8000c10 con 0x7fcec8094fb0 2021-02-10T11:21:27.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.906 7fcece208700 1 -- 172.21.2.101:0/4245882411 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839280", "target": ["mgr", ""]}) v1 -- 0x7fcec80aa2a0 con 0x7fcea80186c0 2021-02-10T11:21:27.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.930 7fcec4ff9700 1 -- 172.21.2.101:0/4245882411 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fcebc000d10 con 0x7fcea80186c0 2021-02-10T11:21:27.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.939 7fcece208700 1 -- 172.21.2.101:0/4245882411 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcea80186c0 msgr2=0x7fcea801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:27.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.939 7fcece208700 1 --2- 172.21.2.101:0/4245882411 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcea80186c0 0x7fcea801abc0 crc :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:27.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.939 7fcece208700 1 -- 172.21.2.101:0/4245882411 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcec8094fb0 msgr2=0x7fcec809e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:27.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.939 7fcece208700 1 --2- 172.21.2.101:0/4245882411 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcec8094fb0 0x7fcec809e5d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fceb800b5b0 tx=0x7fceb800b950).stop 2021-02-10T11:21:27.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.940 7fcece208700 1 -- 172.21.2.101:0/4245882411 shutdown_connections 2021-02-10T11:21:27.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.940 7fcece208700 1 --2- 172.21.2.101:0/4245882411 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcea80186c0 0x7fcea801abc0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:27.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.940 7fcece208700 1 --2- 172.21.2.101:0/4245882411 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcec80c2be0 0x7fcec80c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:27.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.940 7fcece208700 1 --2- 172.21.2.101:0/4245882411 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcec80c3600 0x7fcec80a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:27.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.940 7fcece208700 1 --2- 172.21.2.101:0/4245882411 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcec8094fb0 0x7fcec809e5d0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:27.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.940 7fcece208700 1 -- 172.21.2.101:0/4245882411 >> 172.21.2.101:0/4245882411 conn(0x7fcec809f630 msgr2=0x7fcec80a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:27.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.940 7fcece208700 1 -- 172.21.2.101:0/4245882411 shutdown_connections 2021-02-10T11:21:27.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:27.940 7fcece208700 1 -- 172.21.2.101:0/4245882411 wait complete. 2021-02-10T11:21:27.955 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839280 2021-02-10T11:21:28.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.364 7f6c34177700 1 Processor -- start 2021-02-10T11:21:28.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.364 7f6c34177700 1 -- start start 2021-02-10T11:21:28.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.364 7f6c34177700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6c2c0f05f0 0x7f6c2c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:28.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.364 7f6c34177700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2c1b0c70 con 0x7f6c2c1b0470 2021-02-10T11:21:28.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.364 7f6c34177700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2c0f0b50 con 0x7f6c2c1b0870 2021-02-10T11:21:28.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.364 7f6c34177700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c2c0e5ea0 con 0x7f6c2c0f05f0 2021-02-10T11:21:28.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.364 7f6c31f93700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f6c2c1b0470 0x7f6c2c0ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47348/0 (socket says 172.21.2.101:47348) 2021-02-10T11:21:28.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.364 7f6c31f93700 1 -- 172.21.2.101:0/1377632538 learned_addr learned my addr 172.21.2.101:0/1377632538 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:21:28.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.365 7f6c30f91700 1 -- 172.21.2.101:0/1377632538 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212070458 0 0) 0x7f6c20001180 con 0x7f6c2c1b0470 2021-02-10T11:21:28.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.365 7f6c30f91700 1 -- 172.21.2.101:0/1377632538 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c18002fe0 con 0x7f6c2c1b0470 2021-02-10T11:21:28.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.365 7f6c32794700 1 --2- 172.21.2.101:0/1377632538 >> v2:172.21.2.108:3300/0 conn(0x7f6c2c0f05f0 0x7f6c2c0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:28.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.365 7f6c30f91700 1 -- 172.21.2.101:0/1377632538 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1245374935 0 0) 0x7f6c20001180 con 0x7f6c2c1b0470 2021-02-10T11:21:28.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.365 7f6c30f91700 1 -- 172.21.2.101:0/1377632538 >> v2:172.21.2.108:3300/0 conn(0x7f6c2c0f05f0 msgr2=0x7f6c2c0f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:21:28.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.365 7f6c30f91700 1 --2- 172.21.2.101:0/1377632538 >> v2:172.21.2.108:3300/0 conn(0x7f6c2c0f05f0 0x7f6c2c0f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:28.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.365 7f6c30f91700 1 -- 172.21.2.101:0/1377632538 >> v1:172.21.2.108:6789/0 conn(0x7f6c2c1b0870 legacy=0x7f6c2c0efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:28.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.365 7f6c30f91700 1 -- 172.21.2.101:0/1377632538 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2c0e52e0 con 0x7f6c2c1b0470 2021-02-10T11:21:28.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.365 7f6c30f91700 1 -- 172.21.2.101:0/1377632538 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f6c20001680 con 0x7f6c2c1b0470 2021-02-10T11:21:28.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.365 7f6c30f91700 1 -- 172.21.2.101:0/1377632538 >> v1:172.21.2.101:6789/0 conn(0x7f6c2c1b0470 legacy=0x7f6c2c0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:28.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.365 7f6c30f91700 1 --2- 172.21.2.101:0/1377632538 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c18003ac0 0x7f6c18005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:28.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.365 7f6c30f91700 1 --2- 172.21.2.101:0/1377632538 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c18006790 0x7f6c18008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:28.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.365 7f6c30f91700 1 --2- 172.21.2.101:0/1377632538 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c18009590 0x7f6c1800bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:28.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.365 7f6c30f91700 1 -- 172.21.2.101:0/1377632538 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1800c300 con 0x7f6c18003ac0 2021-02-10T11:21:28.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.365 7f6c30f91700 1 -- 172.21.2.101:0/1377632538 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1800c520 con 0x7f6c18009590 2021-02-10T11:21:28.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.365 7f6c30f91700 1 -- 172.21.2.101:0/1377632538 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1800c740 con 0x7f6c18006790 2021-02-10T11:21:28.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.365 7f6c31f93700 1 --2- 172.21.2.101:0/1377632538 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c18003ac0 0x7f6c18005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:28.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.365 7f6c32794700 1 --2- 172.21.2.101:0/1377632538 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c18009590 0x7f6c1800bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:28.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.366 7f6c31792700 1 --2- 172.21.2.101:0/1377632538 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c18006790 0x7f6c18008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:28.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.366 7f6c31f93700 1 -- 172.21.2.101:0/1377632538 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c18006790 msgr2=0x7f6c18008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:21:28.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.366 7f6c31f93700 1 --2- 172.21.2.101:0/1377632538 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c18006790 0x7f6c18008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:28.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.366 7f6c31f93700 1 -- 172.21.2.101:0/1377632538 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c18009590 msgr2=0x7f6c1800bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:21:28.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.366 7f6c31f93700 1 --2- 172.21.2.101:0/1377632538 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c18009590 0x7f6c1800bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:28.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.366 7f6c31f93700 1 -- 172.21.2.101:0/1377632538 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c180509b0 con 0x7f6c18003ac0 2021-02-10T11:21:28.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.366 7f6c32794700 1 --2- 172.21.2.101:0/1377632538 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c18009590 0x7f6c1800bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:21:28.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.366 7f6c32794700 1 -- 172.21.2.101:0/1377632538 reap_dead start 2021-02-10T11:21:28.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.366 7f6c31792700 1 --2- 172.21.2.101:0/1377632538 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c18006790 0x7f6c18008c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:21:28.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.366 7f6c31f93700 1 --2- 172.21.2.101:0/1377632538 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c18003ac0 0x7f6c18005ef0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f6c20000b60 tx=0x7f6c20007da0).ready entity=mon.0 client_cookie=7a6691685e104781 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:28.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.366 7f6c30f91700 1 -- 172.21.2.101:0/1377632538 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6c200070c0 con 0x7f6c18003ac0 2021-02-10T11:21:28.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.366 7f6c30f91700 1 -- 172.21.2.101:0/1377632538 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c200043a0 con 0x7f6c18003ac0 2021-02-10T11:21:28.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.366 7f6c30f91700 1 -- 172.21.2.101:0/1377632538 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6c200047f0 con 0x7f6c18003ac0 2021-02-10T11:21:28.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.366 7f6c34177700 1 -- 172.21.2.101:0/1377632538 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c18003ac0 msgr2=0x7f6c18005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:28.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.366 7f6c34177700 1 --2- 172.21.2.101:0/1377632538 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c18003ac0 0x7f6c18005ef0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f6c20000b60 tx=0x7f6c20007da0).stop 2021-02-10T11:21:28.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.367 7f6c34177700 1 -- 172.21.2.101:0/1377632538 shutdown_connections 2021-02-10T11:21:28.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.367 7f6c34177700 1 --2- 172.21.2.101:0/1377632538 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c18003ac0 0x7f6c18005ef0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:28.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.367 7f6c34177700 1 -- 172.21.2.101:0/1377632538 >> 172.21.2.101:0/1377632538 conn(0x7f6c2c0ce3e0 msgr2=0x7f6c2c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:28.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.367 7f6c34177700 1 -- 172.21.2.101:0/1377632538 shutdown_connections 2021-02-10T11:21:28.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.367 7f6c34177700 1 -- 172.21.2.101:0/1377632538 wait complete. 2021-02-10T11:21:28.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.382 7f6c34177700 1 Processor -- start 2021-02-10T11:21:28.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.382 7f6c34177700 1 -- start start 2021-02-10T11:21:28.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.382 7f6c34177700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c2c106d30 0x7f6c2c107130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:28.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.382 7f6c34177700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c2c0c29f0 0x7f6c2c0c2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:28.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.382 7f6c34177700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c2c0c3410 0x7f6c2c0b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:28.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.382 7f6c34177700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c0b0ea0 con 0x7f6c2c106d30 2021-02-10T11:21:28.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.382 7f6c34177700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c107750 con 0x7f6c2c0c3410 2021-02-10T11:21:28.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.382 7f6c34177700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c107930 con 0x7f6c2c0c29f0 2021-02-10T11:21:28.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.382 7f6c31f93700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c2c106d30 0x7f6c2c107130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:28.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.382 7f6c31f93700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c2c106d30 0x7f6c2c107130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46870/0 (socket says 172.21.2.101:46870) 2021-02-10T11:21:28.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.382 7f6c31f93700 1 -- 172.21.2.101:0/495297161 learned_addr learned my addr 172.21.2.101:0/495297161 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:21:28.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.383 7f6c31f93700 1 -- 172.21.2.101:0/495297161 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c2c0c29f0 msgr2=0x7f6c2c0c2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:28.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.383 7f6c31792700 1 --2- 172.21.2.101:0/495297161 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c2c0c29f0 0x7f6c2c0c2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:28.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.383 7f6c32794700 1 --2- 172.21.2.101:0/495297161 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c2c0c3410 0x7f6c2c0b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:28.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.383 7f6c31f93700 1 --2- 172.21.2.101:0/495297161 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c2c0c29f0 0x7f6c2c0c2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:28.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.383 7f6c31f93700 1 -- 172.21.2.101:0/495297161 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c2c0c3410 msgr2=0x7f6c2c0b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:28.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.383 7f6c31f93700 1 --2- 172.21.2.101:0/495297161 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c2c0c3410 0x7f6c2c0b13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:28.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.383 7f6c31f93700 1 -- 172.21.2.101:0/495297161 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2c0c3810 con 0x7f6c2c106d30 2021-02-10T11:21:28.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.383 7f6c31792700 1 --2- 172.21.2.101:0/495297161 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c2c0c29f0 0x7f6c2c0c2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:21:28.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.383 7f6c32794700 1 --2- 172.21.2.101:0/495297161 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c2c0c3410 0x7f6c2c0b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:21:28.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.383 7f6c31f93700 1 --2- 172.21.2.101:0/495297161 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c2c106d30 0x7f6c2c107130 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f6c20001910 tx=0x7f6c2000a940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:28.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.383 7f6c1e7fc700 1 -- 172.21.2.101:0/495297161 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6c2000ad30 con 0x7f6c2c106d30 2021-02-10T11:21:28.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.383 7f6c1e7fc700 1 -- 172.21.2.101:0/495297161 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c200043a0 con 0x7f6c2c106d30 2021-02-10T11:21:28.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.383 7f6c34177700 1 -- 172.21.2.101:0/495297161 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2c0c3810 con 0x7f6c2c106d30 2021-02-10T11:21:28.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.383 7f6c1e7fc700 1 -- 172.21.2.101:0/495297161 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6c200048c0 con 0x7f6c2c106d30 2021-02-10T11:21:28.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.384 7f6c34177700 1 -- 172.21.2.101:0/495297161 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2c0c3810 con 0x7f6c2c106d30 2021-02-10T11:21:28.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.384 7f6c1e7fc700 1 -- 172.21.2.101:0/495297161 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6c20008000 con 0x7f6c2c106d30 2021-02-10T11:21:28.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.384 7f6c1e7fc700 1 --2- 172.21.2.101:0/495297161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c14018730 0x7f6c1401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:28.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.384 7f6c1e7fc700 1 -- 172.21.2.101:0/495297161 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6c20032630 con 0x7f6c2c106d30 2021-02-10T11:21:28.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.384 7f6c31792700 1 --2- 172.21.2.101:0/495297161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c14018730 0x7f6c1401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:28.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.385 7f6c31792700 1 --2- 172.21.2.101:0/495297161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c14018730 0x7f6c1401ac10 crc :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:28.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.414 7f6c34177700 1 -- 172.21.2.101:0/495297161 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2c107930 con 0x7f6c2c106d30 2021-02-10T11:21:28.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.416 7f6c1e7fc700 1 -- 172.21.2.101:0/495297161 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6c200043a0 con 0x7f6c2c106d30 2021-02-10T11:21:28.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.545 7f6c34177700 1 -- 172.21.2.101:0/495297161 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839280", "target": ["mgr", ""]}) v1 -- 0x7f6c2c107930 con 0x7f6c14018730 2021-02-10T11:21:28.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.548 7f6c1e7fc700 1 -- 172.21.2.101:0/495297161 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f6c28000960 con 0x7f6c14018730 2021-02-10T11:21:28.555 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839280/6d3550da-9196-45c9-8497-b020460101a3 2021-02-10T11:21:28.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.549 7f6c34177700 1 -- 172.21.2.101:0/495297161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c14018730 msgr2=0x7f6c1401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:28.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.549 7f6c34177700 1 --2- 172.21.2.101:0/495297161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c14018730 0x7f6c1401ac10 crc :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:28.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.549 7f6c34177700 1 -- 172.21.2.101:0/495297161 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c2c106d30 msgr2=0x7f6c2c107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:28.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.549 7f6c34177700 1 --2- 172.21.2.101:0/495297161 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c2c106d30 0x7f6c2c107130 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f6c20001910 tx=0x7f6c2000a940).stop 2021-02-10T11:21:28.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.549 7f6c34177700 1 -- 172.21.2.101:0/495297161 shutdown_connections 2021-02-10T11:21:28.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.549 7f6c34177700 1 --2- 172.21.2.101:0/495297161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c14018730 0x7f6c1401ac10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:28.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.549 7f6c34177700 1 --2- 172.21.2.101:0/495297161 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c2c0c29f0 0x7f6c2c0c2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:28.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.549 7f6c34177700 1 --2- 172.21.2.101:0/495297161 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c2c0c3410 0x7f6c2c0b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:28.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.549 7f6c34177700 1 --2- 172.21.2.101:0/495297161 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c2c106d30 0x7f6c2c107130 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:28.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.549 7f6c34177700 1 -- 172.21.2.101:0/495297161 >> 172.21.2.101:0/495297161 conn(0x7f6c2c0c40e0 msgr2=0x7f6c2c0bf620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:28.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.550 7f6c34177700 1 -- 172.21.2.101:0/495297161 shutdown_connections 2021-02-10T11:21:28.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:28.550 7f6c34177700 1 -- 172.21.2.101:0/495297161 wait complete. 2021-02-10T11:21:28.565 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839280 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839280/6d3550da-9196-45c9-8497-b020460101a3 2021-02-10T11:21:28.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:28.565 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:28.565 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:28.904 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:28.905 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:28.905 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839280/6d3550da-9196-45c9-8497-b020460101a3/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:29.286 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:29.286 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:29.286 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253412 s, 41.4 MB/s 2021-02-10T11:21:29.287 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:29.287 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:29.287 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:29.623 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:29.623 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:29.624 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839280/6d3550da-9196-45c9-8497-b020460101a3/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:30.012 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:30.012 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:30.012 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239966 s, 43.7 MB/s 2021-02-10T11:21:30.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:30.013 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:30.013 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:30.358 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:30.358 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:30.358 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839280/6d3550da-9196-45c9-8497-b020460101a3/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:30.744 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:30.744 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:30.744 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242166 s, 43.3 MB/s 2021-02-10T11:21:30.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:30.745 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:30.745 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:31.084 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:31.084 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:31.084 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839280/6d3550da-9196-45c9-8497-b020460101a3/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:31.465 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:31.465 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:31.465 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243549 s, 43.1 MB/s 2021-02-10T11:21:31.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:31.466 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:31.466 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:31.802 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:31.802 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:31.802 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839280/6d3550da-9196-45c9-8497-b020460101a3/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:32.177 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:32.177 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:32.177 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243494 s, 43.1 MB/s 2021-02-10T11:21:32.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:32.179 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:32.179 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:32.514 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:32.515 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:32.515 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839280/6d3550da-9196-45c9-8497-b020460101a3/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:32.888 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:32.889 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:32.889 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233816 s, 44.8 MB/s 2021-02-10T11:21:32.890 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:32.890 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:32.890 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:33.225 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:33.225 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:33.225 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839280/6d3550da-9196-45c9-8497-b020460101a3/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:33.598 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:33.598 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:33.598 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0230952 s, 45.4 MB/s 2021-02-10T11:21:33.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:33.600 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:33.600 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:33.933 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:33.934 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:33.934 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839280/6d3550da-9196-45c9-8497-b020460101a3/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:34.304 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:34.305 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:34.305 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.022224 s, 47.2 MB/s 2021-02-10T11:21:34.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:34.306 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:34.306 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:34.641 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:34.641 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:34.641 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839280/6d3550da-9196-45c9-8497-b020460101a3/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:35.014 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:35.014 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:35.014 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0220464 s, 47.6 MB/s 2021-02-10T11:21:35.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:35.015 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:35.015 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:35.351 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:35.351 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:35.352 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839280/6d3550da-9196-45c9-8497-b020460101a3/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:35.727 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:35.727 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:35.727 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240107 s, 43.7 MB/s 2021-02-10T11:21:35.728 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839281 2021-02-10T11:21:36.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.136 7fefec808700 1 Processor -- start 2021-02-10T11:21:36.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.136 7fefec808700 1 -- start start 2021-02-10T11:21:36.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.136 7fefec808700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fefe81b0470 0x7fefe81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:36.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.136 7fefec808700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fefe80ef610 0x7fefe80efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:36.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.136 7fefec808700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe80f0b50 con 0x7fefe80f0030 2021-02-10T11:21:36.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.136 7fefec808700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe80e52e0 con 0x7fefe81b0470 2021-02-10T11:21:36.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.136 7fefec808700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe80e54d0 con 0x7fefe80ef610 2021-02-10T11:21:36.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.136 7fefe6e1d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fefe80f0030 0x7fefe80f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47372/0 (socket says 172.21.2.101:47372) 2021-02-10T11:21:36.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.137 7fefe6e1d700 1 -- 172.21.2.101:0/2673833431 learned_addr learned my addr 172.21.2.101:0/2673833431 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:21:36.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.137 7fefe661c700 1 --2- 172.21.2.101:0/2673833431 >> v2:172.21.2.108:3300/0 conn(0x7fefe81b0470 0x7fefe81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:36.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.137 7fefe5e1b700 1 --2- 172.21.2.101:0/2673833431 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fefe80ef610 0x7fefe80efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:36.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.137 7fefe561a700 1 -- 172.21.2.101:0/2673833431 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2640471400 0 0) 0x7fefe0000f90 con 0x7fefe80f0030 2021-02-10T11:21:36.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.137 7fefe561a700 1 -- 172.21.2.101:0/2673833431 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefd4002fe0 con 0x7fefe80f0030 2021-02-10T11:21:36.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.137 7fefe661c700 1 -- 172.21.2.101:0/2673833431 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fefe80ef610 msgr2=0x7fefe80efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:21:36.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.137 7fefe661c700 1 --2- 172.21.2.101:0/2673833431 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fefe80ef610 0x7fefe80efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:36.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.137 7fefe661c700 1 -- 172.21.2.101:0/2673833431 >> v1:172.21.2.101:6789/0 conn(0x7fefe80f0030 legacy=0x7fefe80f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:36.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.137 7fefe661c700 1 -- 172.21.2.101:0/2673833431 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe80f0430 con 0x7fefe81b0470 2021-02-10T11:21:36.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.137 7fefe661c700 1 --2- 172.21.2.101:0/2673833431 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fefe81b0470 0x7fefe81b0870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fefd0006650 tx=0x7fefd0009a80).ready entity=mon.1 client_cookie=b8493c4b3f08da8c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:36.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.138 7fefe561a700 1 -- 172.21.2.101:0/2673833431 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fefd00009b0 con 0x7fefe81b0470 2021-02-10T11:21:36.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.138 7fefe561a700 1 -- 172.21.2.101:0/2673833431 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd000c0c0 con 0x7fefe81b0470 2021-02-10T11:21:36.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.138 7fefe561a700 1 -- 172.21.2.101:0/2673833431 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fefd000c5e0 con 0x7fefe81b0470 2021-02-10T11:21:36.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.138 7fefec808700 1 -- 172.21.2.101:0/2673833431 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fefe81b0470 msgr2=0x7fefe81b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:36.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.138 7fefec808700 1 --2- 172.21.2.101:0/2673833431 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fefe81b0470 0x7fefe81b0870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fefd0006650 tx=0x7fefd0009a80).stop 2021-02-10T11:21:36.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.139 7fefec808700 1 -- 172.21.2.101:0/2673833431 shutdown_connections 2021-02-10T11:21:36.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.139 7fefec808700 1 --2- 172.21.2.101:0/2673833431 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fefe80ef610 0x7fefe80efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:36.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.139 7fefec808700 1 --2- 172.21.2.101:0/2673833431 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fefe81b0470 0x7fefe81b0870 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:36.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.139 7fefec808700 1 -- 172.21.2.101:0/2673833431 >> 172.21.2.101:0/2673833431 conn(0x7fefe80ce3e0 msgr2=0x7fefe80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:36.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.139 7fefec808700 1 -- 172.21.2.101:0/2673833431 shutdown_connections 2021-02-10T11:21:36.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.139 7fefec808700 1 -- 172.21.2.101:0/2673833431 wait complete. 2021-02-10T11:21:36.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.153 7fefec808700 1 Processor -- start 2021-02-10T11:21:36.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.153 7fefec808700 1 -- start start 2021-02-10T11:21:36.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.154 7fefec808700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fefe809f140 0x7fefe80c1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:36.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.154 7fefec808700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fefe80c1f60 0x7fefe809e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:36.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.154 7fefec808700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fefe809e920 0x7fefe80c2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:36.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.154 7fefec808700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe80c23e0 con 0x7fefe809e920 2021-02-10T11:21:36.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.154 7fefec808700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe809ed20 con 0x7fefe80c1f60 2021-02-10T11:21:36.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.154 7fefec808700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe8105d10 con 0x7fefe809f140 2021-02-10T11:21:36.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.154 7fefe6e1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fefe809e920 0x7fefe80c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:36.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.154 7fefe6e1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fefe809e920 0x7fefe80c2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46882/0 (socket says 172.21.2.101:46882) 2021-02-10T11:21:36.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.154 7fefe6e1d700 1 -- 172.21.2.101:0/3897570308 learned_addr learned my addr 172.21.2.101:0/3897570308 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:21:36.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.154 7fefe661c700 1 --2- 172.21.2.101:0/3897570308 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fefe809f140 0x7fefe80c1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:36.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.154 7fefe6e1d700 1 -- 172.21.2.101:0/3897570308 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fefe809f140 msgr2=0x7fefe80c1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:36.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.154 7fefe6e1d700 1 --2- 172.21.2.101:0/3897570308 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fefe809f140 0x7fefe80c1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:36.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.154 7fefe5e1b700 1 --2- 172.21.2.101:0/3897570308 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fefe80c1f60 0x7fefe809e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:36.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.154 7fefe6e1d700 1 -- 172.21.2.101:0/3897570308 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fefe80c1f60 msgr2=0x7fefe809e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:36.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.154 7fefe6e1d700 1 --2- 172.21.2.101:0/3897570308 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fefe80c1f60 0x7fefe809e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:36.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.154 7fefe6e1d700 1 -- 172.21.2.101:0/3897570308 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe81a4910 con 0x7fefe809e920 2021-02-10T11:21:36.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.154 7fefe5e1b700 1 --2- 172.21.2.101:0/3897570308 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fefe80c1f60 0x7fefe809e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:21:36.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.154 7fefe6e1d700 1 --2- 172.21.2.101:0/3897570308 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fefe809e920 0x7fefe80c2960 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fefe000a8a0 tx=0x7fefe000ac40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:36.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.155 7fefdeffd700 1 -- 172.21.2.101:0/3897570308 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fefe0007ce0 con 0x7fefe809e920 2021-02-10T11:21:36.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.155 7fefdeffd700 1 -- 172.21.2.101:0/3897570308 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe0004930 con 0x7fefe809e920 2021-02-10T11:21:36.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.155 7fefdeffd700 1 -- 172.21.2.101:0/3897570308 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fefe0007020 con 0x7fefe809e920 2021-02-10T11:21:36.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.155 7fefec808700 1 -- 172.21.2.101:0/3897570308 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe81a4910 con 0x7fefe809e920 2021-02-10T11:21:36.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.155 7fefec808700 1 -- 172.21.2.101:0/3897570308 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe80c3270 con 0x7fefe809e920 2021-02-10T11:21:36.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.155 7fefdeffd700 1 -- 172.21.2.101:0/3897570308 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fefe000ad80 con 0x7fefe809e920 2021-02-10T11:21:36.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.156 7fefdeffd700 1 --2- 172.21.2.101:0/3897570308 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fefcc0186c0 0x7fefcc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:36.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.156 7fefdeffd700 1 -- 172.21.2.101:0/3897570308 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fefe00328c0 con 0x7fefe809e920 2021-02-10T11:21:36.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.156 7fefe661c700 1 --2- 172.21.2.101:0/3897570308 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fefcc0186c0 0x7fefcc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:36.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.156 7fefe661c700 1 --2- 172.21.2.101:0/3897570308 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fefcc0186c0 0x7fefcc01abc0 crc :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:36.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.186 7fefec808700 1 -- 172.21.2.101:0/3897570308 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe80f0b50 con 0x7fefe809e920 2021-02-10T11:21:36.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.188 7fefdeffd700 1 -- 172.21.2.101:0/3897570308 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fefe0000f40 con 0x7fefe809e920 2021-02-10T11:21:36.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.317 7fefec808700 1 -- 172.21.2.101:0/3897570308 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839281", "target": ["mgr", ""]}) v1 -- 0x7fefe80c23e0 con 0x7fefcc0186c0 2021-02-10T11:21:36.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.343 7fefdeffd700 1 -- 172.21.2.101:0/3897570308 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fefd0000c90 con 0x7fefcc0186c0 2021-02-10T11:21:36.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.350 7fefec808700 1 -- 172.21.2.101:0/3897570308 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fefcc0186c0 msgr2=0x7fefcc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:36.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.350 7fefec808700 1 --2- 172.21.2.101:0/3897570308 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fefcc0186c0 0x7fefcc01abc0 crc :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:36.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.350 7fefec808700 1 -- 172.21.2.101:0/3897570308 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fefe809e920 msgr2=0x7fefe80c2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:36.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.350 7fefec808700 1 --2- 172.21.2.101:0/3897570308 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fefe809e920 0x7fefe80c2960 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fefe000a8a0 tx=0x7fefe000ac40).stop 2021-02-10T11:21:36.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.350 7fefec808700 1 -- 172.21.2.101:0/3897570308 shutdown_connections 2021-02-10T11:21:36.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.350 7fefec808700 1 --2- 172.21.2.101:0/3897570308 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fefe809e920 0x7fefe80c2960 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:36.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.350 7fefec808700 1 --2- 172.21.2.101:0/3897570308 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fefe80c1f60 0x7fefe809e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:36.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.350 7fefec808700 1 --2- 172.21.2.101:0/3897570308 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fefcc0186c0 0x7fefcc01abc0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:36.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.350 7fefec808700 1 --2- 172.21.2.101:0/3897570308 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fefe809f140 0x7fefe80c1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:36.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.350 7fefec808700 1 -- 172.21.2.101:0/3897570308 >> 172.21.2.101:0/3897570308 conn(0x7fefe809fc10 msgr2=0x7fefe80b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:36.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.350 7fefec808700 1 -- 172.21.2.101:0/3897570308 shutdown_connections 2021-02-10T11:21:36.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.350 7fefec808700 1 -- 172.21.2.101:0/3897570308 wait complete. 2021-02-10T11:21:36.365 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839281 2021-02-10T11:21:36.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.773 7f194fff8700 1 Processor -- start 2021-02-10T11:21:36.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.774 7f194fff8700 1 -- start start 2021-02-10T11:21:36.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.774 7f194fff8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f19481b0470 0x7f19481b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:36.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.774 7f194fff8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19480f20e0 0x7f19480f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:36.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.774 7f194fff8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19480f0b50 con 0x7f19480ef610 2021-02-10T11:21:36.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.774 7f194fff8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19480f0820 con 0x7f19481b0470 2021-02-10T11:21:36.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.774 7f194fff8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19480f24e0 con 0x7f19480f20e0 2021-02-10T11:21:36.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.774 7f194d613700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f19480ef610 0x7f19480efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:40792/0 (socket says 172.21.2.101:40792) 2021-02-10T11:21:36.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.774 7f194e615700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19480f20e0 0x7f19480f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:36.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.774 7f194de14700 1 --2- 172.21.2.101:0/2380740608 >> v2:172.21.2.108:3300/0 conn(0x7f19481b0470 0x7f19481b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:36.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.774 7f194d613700 1 -- 172.21.2.101:0/2380740608 learned_addr learned my addr 172.21.2.101:0/2380740608 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:21:36.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.775 7f194de14700 1 -- 172.21.2.101:0/2380740608 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19480f20e0 msgr2=0x7f19480f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:21:36.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.775 7f194de14700 1 --2- 172.21.2.101:0/2380740608 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19480f20e0 0x7f19480f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:36.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.775 7f194de14700 1 -- 172.21.2.101:0/2380740608 >> v1:172.21.2.108:6789/0 conn(0x7f19480ef610 legacy=0x7f19480efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:36.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.775 7f194de14700 1 -- 172.21.2.101:0/2380740608 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19480e52e0 con 0x7f19481b0470 2021-02-10T11:21:36.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.775 7f194e615700 1 --2- 172.21.2.101:0/2380740608 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19480f20e0 0x7f19480f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:21:36.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.775 7f194de14700 1 --2- 172.21.2.101:0/2380740608 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19481b0470 0x7f19481b0870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f1944006650 tx=0x7f1944009a80).ready entity=mon.1 client_cookie=38ac4ae88c26b3a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:36.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.775 7f194ce12700 1 -- 172.21.2.101:0/2380740608 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f194400c0c0 con 0x7f19481b0470 2021-02-10T11:21:36.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.775 7f194ce12700 1 -- 172.21.2.101:0/2380740608 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194400c340 con 0x7f19481b0470 2021-02-10T11:21:36.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.775 7f194ce12700 1 -- 172.21.2.101:0/2380740608 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f194400c790 con 0x7f19481b0470 2021-02-10T11:21:36.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.776 7f194fff8700 1 -- 172.21.2.101:0/2380740608 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19481b0470 msgr2=0x7f19481b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:36.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.776 7f194fff8700 1 --2- 172.21.2.101:0/2380740608 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19481b0470 0x7f19481b0870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f1944006650 tx=0x7f1944009a80).stop 2021-02-10T11:21:36.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.776 7f194fff8700 1 -- 172.21.2.101:0/2380740608 shutdown_connections 2021-02-10T11:21:36.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.776 7f194fff8700 1 --2- 172.21.2.101:0/2380740608 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19480f20e0 0x7f19480f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:36.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.776 7f194fff8700 1 --2- 172.21.2.101:0/2380740608 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19481b0470 0x7f19481b0870 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:36.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.776 7f194fff8700 1 -- 172.21.2.101:0/2380740608 >> 172.21.2.101:0/2380740608 conn(0x7f19480ce3e0 msgr2=0x7f19480e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:36.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.776 7f194fff8700 1 -- 172.21.2.101:0/2380740608 shutdown_connections 2021-02-10T11:21:36.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.776 7f194fff8700 1 -- 172.21.2.101:0/2380740608 wait complete. 2021-02-10T11:21:36.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.791 7f194fff8700 1 Processor -- start 2021-02-10T11:21:36.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.791 7f194fff8700 1 -- start start 2021-02-10T11:21:36.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.791 7f194fff8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19480a0420 0x7f19480a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:36.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.791 7f194fff8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1948189d00 0x7f194818a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:36.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.791 7f194fff8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f194818a720 0x7f19480b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:36.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.791 7f194fff8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1948189780 con 0x7f19480a0420 2021-02-10T11:21:36.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.791 7f194fff8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194818ab20 con 0x7f194818a720 2021-02-10T11:21:36.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.791 7f194fff8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19480a1140 con 0x7f1948189d00 2021-02-10T11:21:36.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.791 7f194de14700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19480a0420 0x7f19480a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:36.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.791 7f194de14700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19480a0420 0x7f19480a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46894/0 (socket says 172.21.2.101:46894) 2021-02-10T11:21:36.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.791 7f194de14700 1 -- 172.21.2.101:0/1367962917 learned_addr learned my addr 172.21.2.101:0/1367962917 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:21:36.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.791 7f194d613700 1 --2- 172.21.2.101:0/1367962917 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1948189d00 0x7f194818a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:36.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.791 7f194e615700 1 --2- 172.21.2.101:0/1367962917 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f194818a720 0x7f19480b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:36.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.792 7f194de14700 1 -- 172.21.2.101:0/1367962917 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1948189d00 msgr2=0x7f194818a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:36.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.792 7f194de14700 1 --2- 172.21.2.101:0/1367962917 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1948189d00 0x7f194818a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:36.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.792 7f194de14700 1 -- 172.21.2.101:0/1367962917 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f194818a720 msgr2=0x7f19480b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:36.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.792 7f194de14700 1 --2- 172.21.2.101:0/1367962917 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f194818a720 0x7f19480b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:36.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.792 7f194de14700 1 -- 172.21.2.101:0/1367962917 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19480b8200 con 0x7f19480a0420 2021-02-10T11:21:36.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.792 7f194de14700 1 --2- 172.21.2.101:0/1367962917 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19480a0420 0x7f19480a0820 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f1944006e90 tx=0x7f194400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:36.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.792 7f193e7fc700 1 -- 172.21.2.101:0/1367962917 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19440009b0 con 0x7f19480a0420 2021-02-10T11:21:36.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.792 7f193e7fc700 1 -- 172.21.2.101:0/1367962917 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194400bd80 con 0x7f19480a0420 2021-02-10T11:21:36.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.792 7f193e7fc700 1 -- 172.21.2.101:0/1367962917 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1944005560 con 0x7f19480a0420 2021-02-10T11:21:36.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.792 7f194fff8700 1 -- 172.21.2.101:0/1367962917 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19480b8200 con 0x7f19480a0420 2021-02-10T11:21:36.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.792 7f194fff8700 1 -- 172.21.2.101:0/1367962917 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19480b8200 con 0x7f19480a0420 2021-02-10T11:21:36.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.793 7f193e7fc700 1 -- 172.21.2.101:0/1367962917 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1944006030 con 0x7f19480a0420 2021-02-10T11:21:36.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.793 7f193e7fc700 1 --2- 172.21.2.101:0/1367962917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19300186c0 0x7f193001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:36.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.793 7f193e7fc700 1 -- 172.21.2.101:0/1367962917 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1944032560 con 0x7f19480a0420 2021-02-10T11:21:36.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.793 7f194d613700 1 --2- 172.21.2.101:0/1367962917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19300186c0 0x7f193001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:36.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.794 7f194d613700 1 --2- 172.21.2.101:0/1367962917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19300186c0 0x7f193001abc0 crc :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:36.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.824 7f194fff8700 1 -- 172.21.2.101:0/1367962917 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1948105ae0 con 0x7f19480a0420 2021-02-10T11:21:36.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.826 7f193e7fc700 1 -- 172.21.2.101:0/1367962917 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f19440063f0 con 0x7f19480a0420 2021-02-10T11:21:36.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.955 7f194fff8700 1 -- 172.21.2.101:0/1367962917 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839281", "target": ["mgr", ""]}) v1 -- 0x7f1948105ae0 con 0x7f19300186c0 2021-02-10T11:21:36.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.959 7f193e7fc700 1 -- 172.21.2.101:0/1367962917 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f1938001a00 con 0x7f19300186c0 2021-02-10T11:21:36.970 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839281/cb62b62c-fcea-487e-9fb3-aa88b442ebf3 2021-02-10T11:21:36.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.964 7f194fff8700 1 -- 172.21.2.101:0/1367962917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19300186c0 msgr2=0x7f193001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:36.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.964 7f194fff8700 1 --2- 172.21.2.101:0/1367962917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19300186c0 0x7f193001abc0 crc :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:36.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.964 7f194fff8700 1 -- 172.21.2.101:0/1367962917 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19480a0420 msgr2=0x7f19480a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:36.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.964 7f194fff8700 1 --2- 172.21.2.101:0/1367962917 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19480a0420 0x7f19480a0820 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f1944006e90 tx=0x7f194400b990).stop 2021-02-10T11:21:36.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.964 7f194fff8700 1 -- 172.21.2.101:0/1367962917 shutdown_connections 2021-02-10T11:21:36.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.964 7f194fff8700 1 --2- 172.21.2.101:0/1367962917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19300186c0 0x7f193001abc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:36.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.964 7f194fff8700 1 --2- 172.21.2.101:0/1367962917 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1948189d00 0x7f194818a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:36.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.964 7f194fff8700 1 --2- 172.21.2.101:0/1367962917 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f194818a720 0x7f19480b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:36.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.964 7f194fff8700 1 --2- 172.21.2.101:0/1367962917 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19480a0420 0x7f19480a0820 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:36.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.964 7f194fff8700 1 -- 172.21.2.101:0/1367962917 >> 172.21.2.101:0/1367962917 conn(0x7f1948094e10 msgr2=0x7f19480a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:36.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.965 7f194fff8700 1 -- 172.21.2.101:0/1367962917 shutdown_connections 2021-02-10T11:21:36.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:36.965 7f194fff8700 1 -- 172.21.2.101:0/1367962917 wait complete. 2021-02-10T11:21:36.980 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839281 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839281/cb62b62c-fcea-487e-9fb3-aa88b442ebf3 2021-02-10T11:21:36.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:36.981 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:36.981 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:37.316 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:37.317 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:37.317 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839281/cb62b62c-fcea-487e-9fb3-aa88b442ebf3/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:37.698 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:37.699 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:37.699 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235405 s, 44.5 MB/s 2021-02-10T11:21:37.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:37.700 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:37.700 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:38.035 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:38.036 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:38.036 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839281/cb62b62c-fcea-487e-9fb3-aa88b442ebf3/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:38.410 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:38.411 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:38.411 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233762 s, 44.9 MB/s 2021-02-10T11:21:38.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:38.412 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:38.412 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:38.748 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:38.749 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:38.749 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839281/cb62b62c-fcea-487e-9fb3-aa88b442ebf3/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:39.119 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:39.119 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:39.119 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0226039 s, 46.4 MB/s 2021-02-10T11:21:39.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:39.120 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:39.120 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:39.455 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:39.455 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:39.455 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839281/cb62b62c-fcea-487e-9fb3-aa88b442ebf3/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:39.832 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:39.832 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:39.832 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243477 s, 43.1 MB/s 2021-02-10T11:21:39.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:39.833 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:39.833 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:40.168 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:40.168 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:40.168 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839281/cb62b62c-fcea-487e-9fb3-aa88b442ebf3/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:40.544 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:40.544 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:40.544 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241523 s, 43.4 MB/s 2021-02-10T11:21:40.545 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:40.545 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:40.545 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:40.880 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:40.880 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:40.881 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839281/cb62b62c-fcea-487e-9fb3-aa88b442ebf3/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:41.259 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:41.259 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:41.259 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0257317 s, 40.8 MB/s 2021-02-10T11:21:41.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:41.260 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:41.260 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:41.596 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:41.597 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:41.597 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839281/cb62b62c-fcea-487e-9fb3-aa88b442ebf3/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:41.972 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:41.972 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:41.973 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.025259 s, 41.5 MB/s 2021-02-10T11:21:41.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:41.974 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:41.974 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:42.309 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:42.310 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:42.310 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839281/cb62b62c-fcea-487e-9fb3-aa88b442ebf3/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:42.685 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:42.685 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:42.686 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249909 s, 42.0 MB/s 2021-02-10T11:21:42.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:42.687 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:42.687 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:43.023 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:43.023 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:43.024 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839281/cb62b62c-fcea-487e-9fb3-aa88b442ebf3/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:43.397 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:43.398 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:43.398 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236275 s, 44.4 MB/s 2021-02-10T11:21:43.399 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:43.399 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:43.399 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:43.734 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:43.735 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:43.735 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839281/cb62b62c-fcea-487e-9fb3-aa88b442ebf3/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:44.108 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:44.108 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:44.108 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242107 s, 43.3 MB/s 2021-02-10T11:21:44.109 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839282 2021-02-10T11:21:44.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.515 7fa82459c700 1 Processor -- start 2021-02-10T11:21:44.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.516 7fa82459c700 1 -- start start 2021-02-10T11:21:44.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.516 7fa82459c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa81c1b0470 0x7fa81c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:44.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.516 7fa82459c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa81c0ef610 0x7fa81c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:44.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.516 7fa82459c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa81c0f0b50 con 0x7fa81c0f0030 2021-02-10T11:21:44.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.516 7fa82459c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa81c0e52e0 con 0x7fa81c0ef610 2021-02-10T11:21:44.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.516 7fa82459c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa81c0e5490 con 0x7fa81c1b0470 2021-02-10T11:21:44.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.516 7fa821bb7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa81c0ef610 0x7fa81c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:44.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.516 7fa821bb7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa81c0ef610 0x7fa81c0efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46902/0 (socket says 172.21.2.101:46902) 2021-02-10T11:21:44.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.516 7fa821bb7700 1 -- 172.21.2.101:0/3919851859 learned_addr learned my addr 172.21.2.101:0/3919851859 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:21:44.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.517 7fa821bb7700 1 -- 172.21.2.101:0/3919851859 >> v2:172.21.2.108:3300/0 conn(0x7fa81c1b0470 msgr2=0x7fa81c1b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:21:44.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.517 7fa821bb7700 1 --2- 172.21.2.101:0/3919851859 >> v2:172.21.2.108:3300/0 conn(0x7fa81c1b0470 0x7fa81c1b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:44.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.517 7fa821bb7700 1 -- 172.21.2.101:0/3919851859 >> v1:172.21.2.108:6789/0 conn(0x7fa81c0f0030 legacy=0x7fa81c0f10d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:21:44.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.517 7fa821bb7700 1 -- 172.21.2.101:0/3919851859 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c0f0430 con 0x7fa81c0ef610 2021-02-10T11:21:44.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.517 7fa821bb7700 1 --2- 172.21.2.101:0/3919851859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa81c0ef610 0x7fa81c0efa10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7fa80c006700 tx=0x7fa80c009b80).ready entity=mon.0 client_cookie=3454a1c76f409325 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:44.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.517 7fa8213b6700 1 -- 172.21.2.101:0/3919851859 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa80c00c0c0 con 0x7fa81c0ef610 2021-02-10T11:21:44.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.517 7fa8213b6700 1 -- 172.21.2.101:0/3919851859 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80c00c400 con 0x7fa81c0ef610 2021-02-10T11:21:44.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.517 7fa8213b6700 1 -- 172.21.2.101:0/3919851859 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa80c00c920 con 0x7fa81c0ef610 2021-02-10T11:21:44.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.517 7fa82459c700 1 -- 172.21.2.101:0/3919851859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa81c0ef610 msgr2=0x7fa81c0efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:44.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.517 7fa82459c700 1 --2- 172.21.2.101:0/3919851859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa81c0ef610 0x7fa81c0efa10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7fa80c006700 tx=0x7fa80c009b80).stop 2021-02-10T11:21:44.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.518 7fa82459c700 1 -- 172.21.2.101:0/3919851859 shutdown_connections 2021-02-10T11:21:44.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.518 7fa82459c700 1 --2- 172.21.2.101:0/3919851859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa81c0ef610 0x7fa81c0efa10 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:44.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.518 7fa82459c700 1 --2- 172.21.2.101:0/3919851859 >> v2:172.21.2.108:3300/0 conn(0x7fa81c1b0470 0x7fa81c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:44.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.518 7fa82459c700 1 -- 172.21.2.101:0/3919851859 >> 172.21.2.101:0/3919851859 conn(0x7fa81c0ce3e0 msgr2=0x7fa81c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:44.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.518 7fa82459c700 1 -- 172.21.2.101:0/3919851859 shutdown_connections 2021-02-10T11:21:44.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.518 7fa82459c700 1 -- 172.21.2.101:0/3919851859 wait complete. 2021-02-10T11:21:44.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.532 7fa82459c700 1 Processor -- start 2021-02-10T11:21:44.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.532 7fa82459c700 1 -- start start 2021-02-10T11:21:44.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.533 7fa82459c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa81c0c36a0 0x7fa81c0c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:44.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.533 7fa82459c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa81c0c3190 0x7fa81c0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:44.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.533 7fa82459c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa81c0b2370 0x7fa81c0b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:44.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.533 7fa82459c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c0b27c0 con 0x7fa81c0c36a0 2021-02-10T11:21:44.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.533 7fa82459c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c0b3360 con 0x7fa81c0c3190 2021-02-10T11:21:44.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.533 7fa82459c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c0b3540 con 0x7fa81c0b2370 2021-02-10T11:21:44.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.533 7fa8223b8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa81c0c36a0 0x7fa81c0c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:44.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.533 7fa8223b8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa81c0c36a0 0x7fa81c0c2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46906/0 (socket says 172.21.2.101:46906) 2021-02-10T11:21:44.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.533 7fa8223b8700 1 -- 172.21.2.101:0/1164827617 learned_addr learned my addr 172.21.2.101:0/1164827617 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:21:44.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.533 7fa8223b8700 1 -- 172.21.2.101:0/1164827617 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa81c0b2370 msgr2=0x7fa81c0b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:44.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.533 7fa8223b8700 1 --2- 172.21.2.101:0/1164827617 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa81c0b2370 0x7fa81c0b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:44.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.533 7fa8223b8700 1 -- 172.21.2.101:0/1164827617 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa81c0c3190 msgr2=0x7fa81c0b1d50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:21:44.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.533 7fa8223b8700 1 --2- 172.21.2.101:0/1164827617 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa81c0c3190 0x7fa81c0b1d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:44.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.533 7fa8223b8700 1 -- 172.21.2.101:0/1164827617 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c0b3760 con 0x7fa81c0c36a0 2021-02-10T11:21:44.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.534 7fa8223b8700 1 --2- 172.21.2.101:0/1164827617 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa81c0c36a0 0x7fa81c0c2b70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fa81800b560 tx=0x7fa81800b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:44.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.534 7fa812ffd700 1 -- 172.21.2.101:0/1164827617 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa81800bcf0 con 0x7fa81c0c36a0 2021-02-10T11:21:44.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.534 7fa812ffd700 1 -- 172.21.2.101:0/1164827617 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa818006030 con 0x7fa81c0c36a0 2021-02-10T11:21:44.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.534 7fa812ffd700 1 -- 172.21.2.101:0/1164827617 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa8180054d0 con 0x7fa81c0c36a0 2021-02-10T11:21:44.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.534 7fa82459c700 1 -- 172.21.2.101:0/1164827617 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa81c0b3760 con 0x7fa81c0c36a0 2021-02-10T11:21:44.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.534 7fa82459c700 1 -- 172.21.2.101:0/1164827617 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa81c0b3cf0 con 0x7fa81c0c36a0 2021-02-10T11:21:44.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.534 7fa812ffd700 1 -- 172.21.2.101:0/1164827617 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa818006210 con 0x7fa81c0c36a0 2021-02-10T11:21:44.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.535 7fa812ffd700 1 --2- 172.21.2.101:0/1164827617 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8040186c0 0x7fa80401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:44.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.535 7fa812ffd700 1 -- 172.21.2.101:0/1164827617 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa818032590 con 0x7fa81c0c36a0 2021-02-10T11:21:44.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.535 7fa821bb7700 1 --2- 172.21.2.101:0/1164827617 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8040186c0 0x7fa80401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:44.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.535 7fa821bb7700 1 --2- 172.21.2.101:0/1164827617 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8040186c0 0x7fa80401abc0 crc :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:44.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.566 7fa82459c700 1 -- 172.21.2.101:0/1164827617 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa81c0edd10 con 0x7fa81c0c36a0 2021-02-10T11:21:44.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.568 7fa812ffd700 1 -- 172.21.2.101:0/1164827617 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa818000960 con 0x7fa81c0c36a0 2021-02-10T11:21:44.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.694 7fa82459c700 1 -- 172.21.2.101:0/1164827617 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839282", "target": ["mgr", ""]}) v1 -- 0x7fa81c0b27c0 con 0x7fa8040186c0 2021-02-10T11:21:44.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.720 7fa812ffd700 1 -- 172.21.2.101:0/1164827617 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fa80c009110 con 0x7fa8040186c0 2021-02-10T11:21:44.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.727 7fa82459c700 1 -- 172.21.2.101:0/1164827617 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8040186c0 msgr2=0x7fa80401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:44.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.727 7fa82459c700 1 --2- 172.21.2.101:0/1164827617 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8040186c0 0x7fa80401abc0 crc :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:44.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.727 7fa82459c700 1 -- 172.21.2.101:0/1164827617 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa81c0c36a0 msgr2=0x7fa81c0c2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:44.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.727 7fa82459c700 1 --2- 172.21.2.101:0/1164827617 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa81c0c36a0 0x7fa81c0c2b70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fa81800b560 tx=0x7fa81800b900).stop 2021-02-10T11:21:44.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.727 7fa82459c700 1 -- 172.21.2.101:0/1164827617 shutdown_connections 2021-02-10T11:21:44.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.727 7fa82459c700 1 --2- 172.21.2.101:0/1164827617 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8040186c0 0x7fa80401abc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:44.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.727 7fa82459c700 1 --2- 172.21.2.101:0/1164827617 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa81c0b2370 0x7fa81c0b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:44.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.727 7fa82459c700 1 --2- 172.21.2.101:0/1164827617 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa81c0c3190 0x7fa81c0b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:44.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.727 7fa82459c700 1 --2- 172.21.2.101:0/1164827617 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa81c0c36a0 0x7fa81c0c2b70 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:44.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.727 7fa82459c700 1 -- 172.21.2.101:0/1164827617 >> 172.21.2.101:0/1164827617 conn(0x7fa81c09f630 msgr2=0x7fa81c0a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:44.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.727 7fa82459c700 1 -- 172.21.2.101:0/1164827617 shutdown_connections 2021-02-10T11:21:44.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:44.727 7fa82459c700 1 -- 172.21.2.101:0/1164827617 wait complete. 2021-02-10T11:21:44.742 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839282 2021-02-10T11:21:45.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.149 7f3d0c4f5700 1 Processor -- start 2021-02-10T11:21:45.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.149 7f3d0c4f5700 1 -- start start 2021-02-10T11:21:45.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.150 7f3d0c4f5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d041b0470 0x7f3d041b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:45.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.150 7f3d0c4f5700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3d040f20e0 0x7f3d040f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:45.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.150 7f3d0c4f5700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d040f0b50 con 0x7f3d040ef610 2021-02-10T11:21:45.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.150 7f3d0c4f5700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d040f0820 con 0x7f3d040f20e0 2021-02-10T11:21:45.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.150 7f3d0c4f5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d040f24e0 con 0x7f3d041b0470 2021-02-10T11:21:45.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.151 7f3d0a311700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d041b0470 0x7f3d041b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:45.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.151 7f3d0ab12700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3d040f20e0 0x7f3d040f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:45.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.151 7f3d09b10700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f3d040ef610 0x7f3d040efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:40818/0 (socket says 172.21.2.101:40818) 2021-02-10T11:21:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.151 7f3d09b10700 1 -- 172.21.2.101:0/317520294 learned_addr learned my addr 172.21.2.101:0/317520294 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:21:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.152 7f3d0ab12700 1 -- 172.21.2.101:0/317520294 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d041b0470 msgr2=0x7f3d041b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:21:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.152 7f3d0ab12700 1 --2- 172.21.2.101:0/317520294 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d041b0470 0x7f3d041b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.152 7f3d0ab12700 1 -- 172.21.2.101:0/317520294 >> v1:172.21.2.108:6789/0 conn(0x7f3d040ef610 legacy=0x7f3d040efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.152 7f3d0ab12700 1 -- 172.21.2.101:0/317520294 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d040e52e0 con 0x7f3d040f20e0 2021-02-10T11:21:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.152 7f3d0ab12700 1 --2- 172.21.2.101:0/317520294 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3d040f20e0 0x7f3d040f0200 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f3cfc006e50 tx=0x7f3cfc009e70).ready entity=mon.1 client_cookie=e9392ab6ab657d9e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.152 7f3d0930f700 1 -- 172.21.2.101:0/317520294 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3cfc00b300 con 0x7f3d040f20e0 2021-02-10T11:21:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.152 7f3d0930f700 1 -- 172.21.2.101:0/317520294 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cfc003030 con 0x7f3d040f20e0 2021-02-10T11:21:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.152 7f3d0930f700 1 -- 172.21.2.101:0/317520294 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3cfc00b730 con 0x7f3d040f20e0 2021-02-10T11:21:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.152 7f3d0c4f5700 1 -- 172.21.2.101:0/317520294 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3d040f20e0 msgr2=0x7f3d040f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.152 7f3d0c4f5700 1 --2- 172.21.2.101:0/317520294 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3d040f20e0 0x7f3d040f0200 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f3cfc006e50 tx=0x7f3cfc009e70).stop 2021-02-10T11:21:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.153 7f3d0c4f5700 1 -- 172.21.2.101:0/317520294 shutdown_connections 2021-02-10T11:21:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.153 7f3d0c4f5700 1 --2- 172.21.2.101:0/317520294 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3d040f20e0 0x7f3d040f0200 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.153 7f3d0c4f5700 1 --2- 172.21.2.101:0/317520294 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d041b0470 0x7f3d041b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:45.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.153 7f3d0c4f5700 1 -- 172.21.2.101:0/317520294 >> 172.21.2.101:0/317520294 conn(0x7f3d040ce3e0 msgr2=0x7f3d040e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:45.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.153 7f3d0c4f5700 1 -- 172.21.2.101:0/317520294 shutdown_connections 2021-02-10T11:21:45.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.153 7f3d0c4f5700 1 -- 172.21.2.101:0/317520294 wait complete. 2021-02-10T11:21:45.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.168 7f3d0c4f5700 1 Processor -- start 2021-02-10T11:21:45.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.168 7f3d0c4f5700 1 -- start start 2021-02-10T11:21:45.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.168 7f3d0c4f5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d040b7c20 0x7f3d040b8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:45.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.168 7f3d0c4f5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3d040b8a30 0x7f3d040c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:45.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.168 7f3d0c4f5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3d040c25a0 0x7f3d040c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:45.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.168 7f3d0c4f5700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d040c1a80 con 0x7f3d040b8a30 2021-02-10T11:21:45.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.168 7f3d0c4f5700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d040b8020 con 0x7f3d040c25a0 2021-02-10T11:21:45.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.168 7f3d0c4f5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0409b9c0 con 0x7f3d040b7c20 2021-02-10T11:21:45.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.168 7f3d09b10700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3d040b8a30 0x7f3d040c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:45.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.169 7f3d09b10700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3d040b8a30 0x7f3d040c1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46916/0 (socket says 172.21.2.101:46916) 2021-02-10T11:21:45.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.169 7f3d09b10700 1 -- 172.21.2.101:0/1111751733 learned_addr learned my addr 172.21.2.101:0/1111751733 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:21:45.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.169 7f3d09b10700 1 -- 172.21.2.101:0/1111751733 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d040b7c20 msgr2=0x7f3d040b8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:45.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.169 7f3d09b10700 1 --2- 172.21.2.101:0/1111751733 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d040b7c20 0x7f3d040b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:45.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.169 7f3d09b10700 1 -- 172.21.2.101:0/1111751733 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3d040c25a0 msgr2=0x7f3d040c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:45.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.169 7f3d09b10700 1 --2- 172.21.2.101:0/1111751733 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3d040c25a0 0x7f3d040c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:45.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.169 7f3d09b10700 1 -- 172.21.2.101:0/1111751733 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d040b8e30 con 0x7f3d040b8a30 2021-02-10T11:21:45.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.169 7f3d09b10700 1 --2- 172.21.2.101:0/1111751733 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3d040b8a30 0x7f3d040c1f80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f3d00001c60 tx=0x7f3d0000a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:45.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.169 7f3cf6ffd700 1 -- 172.21.2.101:0/1111751733 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3d0000acc0 con 0x7f3d040b8a30 2021-02-10T11:21:45.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.169 7f3cf6ffd700 1 -- 172.21.2.101:0/1111751733 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d00004140 con 0x7f3d040b8a30 2021-02-10T11:21:45.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.169 7f3cf6ffd700 1 -- 172.21.2.101:0/1111751733 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3d00004660 con 0x7f3d040b8a30 2021-02-10T11:21:45.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.169 7f3d0c4f5700 1 -- 172.21.2.101:0/1111751733 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d040b8e30 con 0x7f3d040b8a30 2021-02-10T11:21:45.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.169 7f3d0c4f5700 1 -- 172.21.2.101:0/1111751733 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d040c32b0 con 0x7f3d040b8a30 2021-02-10T11:21:45.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.170 7f3cf6ffd700 1 -- 172.21.2.101:0/1111751733 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3d00004320 con 0x7f3d040b8a30 2021-02-10T11:21:45.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.170 7f3cf6ffd700 1 --2- 172.21.2.101:0/1111751733 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cec0186c0 0x7f3cec01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:45.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.170 7f3cf6ffd700 1 -- 172.21.2.101:0/1111751733 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3d00032590 con 0x7f3d040b8a30 2021-02-10T11:21:45.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.170 7f3d0a311700 1 --2- 172.21.2.101:0/1111751733 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cec0186c0 0x7f3cec01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:45.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.171 7f3d0a311700 1 --2- 172.21.2.101:0/1111751733 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cec0186c0 0x7f3cec01abc0 crc :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:45.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.200 7f3d0c4f5700 1 -- 172.21.2.101:0/1111751733 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d04105b30 con 0x7f3d040b8a30 2021-02-10T11:21:45.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.203 7f3cf6ffd700 1 -- 172.21.2.101:0/1111751733 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3d00000dd0 con 0x7f3d040b8a30 2021-02-10T11:21:45.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.331 7f3d0c4f5700 1 -- 172.21.2.101:0/1111751733 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839282", "target": ["mgr", ""]}) v1 -- 0x7f3d040b8020 con 0x7f3cec0186c0 2021-02-10T11:21:45.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.333 7f3cf6ffd700 1 -- 172.21.2.101:0/1111751733 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f3cf8006e70 con 0x7f3cec0186c0 2021-02-10T11:21:45.341 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839282/179d0f29-d967-43a9-a5f7-89a76752d468 2021-02-10T11:21:45.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.335 7f3d0c4f5700 1 -- 172.21.2.101:0/1111751733 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cec0186c0 msgr2=0x7f3cec01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:45.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.335 7f3d0c4f5700 1 --2- 172.21.2.101:0/1111751733 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cec0186c0 0x7f3cec01abc0 crc :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:45.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.335 7f3d0c4f5700 1 -- 172.21.2.101:0/1111751733 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3d040b8a30 msgr2=0x7f3d040c1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:45.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.335 7f3d0c4f5700 1 --2- 172.21.2.101:0/1111751733 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3d040b8a30 0x7f3d040c1f80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f3d00001c60 tx=0x7f3d0000a8d0).stop 2021-02-10T11:21:45.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.335 7f3d0c4f5700 1 -- 172.21.2.101:0/1111751733 shutdown_connections 2021-02-10T11:21:45.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.335 7f3d0c4f5700 1 --2- 172.21.2.101:0/1111751733 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3d040c25a0 0x7f3d040c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:45.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.335 7f3d0c4f5700 1 --2- 172.21.2.101:0/1111751733 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3d040b8a30 0x7f3d040c1f80 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:45.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.335 7f3d0c4f5700 1 --2- 172.21.2.101:0/1111751733 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cec0186c0 0x7f3cec01abc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:45.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.335 7f3d0c4f5700 1 --2- 172.21.2.101:0/1111751733 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d040b7c20 0x7f3d040b8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:45.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.335 7f3d0c4f5700 1 -- 172.21.2.101:0/1111751733 >> 172.21.2.101:0/1111751733 conn(0x7f3d04094e10 msgr2=0x7f3d040a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:45.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.335 7f3d0c4f5700 1 -- 172.21.2.101:0/1111751733 shutdown_connections 2021-02-10T11:21:45.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:45.335 7f3d0c4f5700 1 -- 172.21.2.101:0/1111751733 wait complete. 2021-02-10T11:21:45.350 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839282 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839282/179d0f29-d967-43a9-a5f7-89a76752d468 2021-02-10T11:21:45.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:45.351 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:45.351 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:45.686 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:45.687 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:45.687 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839282/179d0f29-d967-43a9-a5f7-89a76752d468/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:46.068 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:46.068 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:46.068 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0228286 s, 45.9 MB/s 2021-02-10T11:21:46.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:46.069 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:46.069 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:46.404 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:46.404 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:46.404 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839282/179d0f29-d967-43a9-a5f7-89a76752d468/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:46.780 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:46.780 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:46.780 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238567 s, 44.0 MB/s 2021-02-10T11:21:46.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:46.781 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:46.781 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:47.116 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:47.117 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:47.117 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839282/179d0f29-d967-43a9-a5f7-89a76752d468/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:47.491 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:47.491 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:47.491 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235516 s, 44.5 MB/s 2021-02-10T11:21:47.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:47.492 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:47.492 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:47.827 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:47.827 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:47.827 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839282/179d0f29-d967-43a9-a5f7-89a76752d468/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:48.203 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:48.203 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:48.203 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239606 s, 43.8 MB/s 2021-02-10T11:21:48.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:48.204 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:48.204 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:48.541 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:48.541 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:48.541 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839282/179d0f29-d967-43a9-a5f7-89a76752d468/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:48.911 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:48.911 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:48.911 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0228501 s, 45.9 MB/s 2021-02-10T11:21:48.912 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:48.913 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:48.913 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:49.248 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:49.248 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:49.248 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839282/179d0f29-d967-43a9-a5f7-89a76752d468/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:49.492 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:21:49.496 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:21:49.623 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:49.623 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:49.624 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244109 s, 43.0 MB/s 2021-02-10T11:21:49.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:49.625 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:49.625 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:49.960 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:49.960 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:49.960 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839282/179d0f29-d967-43a9-a5f7-89a76752d468/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:50.336 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:50.336 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:50.336 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235842 s, 44.5 MB/s 2021-02-10T11:21:50.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:50.337 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:50.337 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:50.672 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:50.672 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:50.673 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839282/179d0f29-d967-43a9-a5f7-89a76752d468/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:51.047 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:51.048 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:51.048 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240026 s, 43.7 MB/s 2021-02-10T11:21:51.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:51.049 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:51.049 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:51.385 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:51.567 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:51.567 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839282/179d0f29-d967-43a9-a5f7-89a76752d468/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:51.943 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:51.943 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:51.943 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233073 s, 45.0 MB/s 2021-02-10T11:21:51.944 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:51.944 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:51.944 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:52.279 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:52.279 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:52.280 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839282/179d0f29-d967-43a9-a5f7-89a76752d468/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:52.656 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:52.656 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:52.656 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024965 s, 42.0 MB/s 2021-02-10T11:21:52.657 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839283 2021-02-10T11:21:53.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.063 7f03a4182700 1 Processor -- start 2021-02-10T11:21:53.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.063 7f03a4182700 1 -- start start 2021-02-10T11:21:53.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.064 7f03a4182700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f039c1b0470 0x7f039c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:53.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.064 7f03a4182700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f039c0ef610 0x7f039c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:53.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.064 7f03a4182700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f039c0f0b50 con 0x7f039c0f0030 2021-02-10T11:21:53.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.064 7f03a4182700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f039c0e52e0 con 0x7f039c0ef610 2021-02-10T11:21:53.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.064 7f03a4182700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f039c0e5490 con 0x7f039c1b0470 2021-02-10T11:21:53.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.064 7f03a179d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f039c0ef610 0x7f039c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:53.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.064 7f03a179d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f039c0ef610 0x7f039c0efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46924/0 (socket says 172.21.2.101:46924) 2021-02-10T11:21:53.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.064 7f03a179d700 1 -- 172.21.2.101:0/4289044831 learned_addr learned my addr 172.21.2.101:0/4289044831 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:21:53.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.065 7f03a179d700 1 -- 172.21.2.101:0/4289044831 >> v2:172.21.2.108:3300/0 conn(0x7f039c1b0470 msgr2=0x7f039c1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:21:53.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.065 7f03a1f9e700 1 --2- 172.21.2.101:0/4289044831 >> v2:172.21.2.108:3300/0 conn(0x7f039c1b0470 0x7f039c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:53.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.065 7f03a179d700 1 --2- 172.21.2.101:0/4289044831 >> v2:172.21.2.108:3300/0 conn(0x7f039c1b0470 0x7f039c1b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:53.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.065 7f03a179d700 1 -- 172.21.2.101:0/4289044831 >> v1:172.21.2.108:6789/0 conn(0x7f039c0f0030 legacy=0x7f039c0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:53.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.065 7f03a179d700 1 -- 172.21.2.101:0/4289044831 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c0f0430 con 0x7f039c0ef610 2021-02-10T11:21:53.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.065 7f03a1f9e700 1 --2- 172.21.2.101:0/4289044831 >> v2:172.21.2.108:3300/0 conn(0x7f039c1b0470 0x7f039c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:21:53.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.065 7f03a179d700 1 --2- 172.21.2.101:0/4289044831 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f039c0ef610 0x7f039c0efa10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f0398006700 tx=0x7f0398009b80).ready entity=mon.0 client_cookie=17c8978c85890e29 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:53.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.065 7f03a0f9c700 1 -- 172.21.2.101:0/4289044831 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f039800c0c0 con 0x7f039c0ef610 2021-02-10T11:21:53.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.065 7f03a0f9c700 1 -- 172.21.2.101:0/4289044831 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039800c440 con 0x7f039c0ef610 2021-02-10T11:21:53.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.065 7f03a0f9c700 1 -- 172.21.2.101:0/4289044831 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f039800c960 con 0x7f039c0ef610 2021-02-10T11:21:53.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.065 7f03a4182700 1 -- 172.21.2.101:0/4289044831 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f039c0ef610 msgr2=0x7f039c0efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:53.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.065 7f03a4182700 1 --2- 172.21.2.101:0/4289044831 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f039c0ef610 0x7f039c0efa10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f0398006700 tx=0x7f0398009b80).stop 2021-02-10T11:21:53.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.066 7f03a4182700 1 -- 172.21.2.101:0/4289044831 shutdown_connections 2021-02-10T11:21:53.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.066 7f03a4182700 1 --2- 172.21.2.101:0/4289044831 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f039c0ef610 0x7f039c0efa10 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:53.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.066 7f03a4182700 1 --2- 172.21.2.101:0/4289044831 >> v2:172.21.2.108:3300/0 conn(0x7f039c1b0470 0x7f039c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:53.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.066 7f03a4182700 1 -- 172.21.2.101:0/4289044831 >> 172.21.2.101:0/4289044831 conn(0x7f039c0ce3e0 msgr2=0x7f039c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:53.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.067 7f03a4182700 1 -- 172.21.2.101:0/4289044831 shutdown_connections 2021-02-10T11:21:53.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.067 7f03a4182700 1 -- 172.21.2.101:0/4289044831 wait complete. 2021-02-10T11:21:53.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.081 7f03a4182700 1 Processor -- start 2021-02-10T11:21:53.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.081 7f03a4182700 1 -- start start 2021-02-10T11:21:53.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.081 7f03a4182700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f039c0b20b0 0x7f039c0ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:53.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.081 7f03a4182700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f039c0b24b0 0x7f039c191dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:53.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.081 7f03a4182700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f039c0acf90 0x7f039c1923e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:53.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.081 7f03a4182700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c0b1b30 con 0x7f039c0acf90 2021-02-10T11:21:53.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.081 7f03a4182700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c1a04e0 con 0x7f039c0b24b0 2021-02-10T11:21:53.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.081 7f03a4182700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c1a06c0 con 0x7f039c0b20b0 2021-02-10T11:21:53.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.082 7f03a279f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f039c0acf90 0x7f039c1923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:53.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.082 7f03a279f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f039c0acf90 0x7f039c1923e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46930/0 (socket says 172.21.2.101:46930) 2021-02-10T11:21:53.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.082 7f03a279f700 1 -- 172.21.2.101:0/4055253581 learned_addr learned my addr 172.21.2.101:0/4055253581 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:21:53.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.082 7f03a279f700 1 -- 172.21.2.101:0/4055253581 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f039c0b20b0 msgr2=0x7f039c0ac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:53.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.082 7f03a279f700 1 --2- 172.21.2.101:0/4055253581 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f039c0b20b0 0x7f039c0ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:53.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.082 7f03a279f700 1 -- 172.21.2.101:0/4055253581 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f039c0b24b0 msgr2=0x7f039c191dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:53.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.082 7f03a279f700 1 --2- 172.21.2.101:0/4055253581 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f039c0b24b0 0x7f039c191dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:53.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.082 7f03a279f700 1 -- 172.21.2.101:0/4055253581 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c192be0 con 0x7f039c0acf90 2021-02-10T11:21:53.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.082 7f03a279f700 1 --2- 172.21.2.101:0/4055253581 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f039c0acf90 0x7f039c1923e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f0394006ed0 tx=0x7f039400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:53.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.083 7f038e7fc700 1 -- 172.21.2.101:0/4055253581 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f039400bd50 con 0x7f039c0acf90 2021-02-10T11:21:53.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.083 7f038e7fc700 1 -- 172.21.2.101:0/4055253581 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0394006020 con 0x7f039c0acf90 2021-02-10T11:21:53.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.083 7f038e7fc700 1 -- 172.21.2.101:0/4055253581 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f03940054b0 con 0x7f039c0acf90 2021-02-10T11:21:53.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.083 7f03a4182700 1 -- 172.21.2.101:0/4055253581 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f039c192be0 con 0x7f039c0acf90 2021-02-10T11:21:53.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.083 7f03a4182700 1 -- 172.21.2.101:0/4055253581 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f039c192be0 con 0x7f039c0acf90 2021-02-10T11:21:53.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.083 7f038e7fc700 1 -- 172.21.2.101:0/4055253581 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0394006200 con 0x7f039c0acf90 2021-02-10T11:21:53.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.083 7f038e7fc700 1 --2- 172.21.2.101:0/4055253581 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f03840186c0 0x7f038401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:53.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.083 7f038e7fc700 1 -- 172.21.2.101:0/4055253581 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0394032420 con 0x7f039c0acf90 2021-02-10T11:21:53.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.083 7f03a1f9e700 1 --2- 172.21.2.101:0/4055253581 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f03840186c0 0x7f038401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:53.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.084 7f03a1f9e700 1 --2- 172.21.2.101:0/4055253581 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f03840186c0 0x7f038401abc0 crc :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:53.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.114 7f03a4182700 1 -- 172.21.2.101:0/4055253581 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039c105af0 con 0x7f039c0acf90 2021-02-10T11:21:53.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.117 7f038e7fc700 1 -- 172.21.2.101:0/4055253581 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0394030d50 con 0x7f039c0acf90 2021-02-10T11:21:53.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.241 7f03a4182700 1 -- 172.21.2.101:0/4055253581 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839283", "target": ["mgr", ""]}) v1 -- 0x7f039c105af0 con 0x7f03840186c0 2021-02-10T11:21:53.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.270 7f038e7fc700 1 -- 172.21.2.101:0/4055253581 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f0390000d30 con 0x7f03840186c0 2021-02-10T11:21:53.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.274 7f03a4182700 1 -- 172.21.2.101:0/4055253581 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f03840186c0 msgr2=0x7f038401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:53.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.274 7f03a4182700 1 --2- 172.21.2.101:0/4055253581 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f03840186c0 0x7f038401abc0 crc :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:53.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.274 7f03a4182700 1 -- 172.21.2.101:0/4055253581 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f039c0acf90 msgr2=0x7f039c1923e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:53.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.274 7f03a4182700 1 --2- 172.21.2.101:0/4055253581 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f039c0acf90 0x7f039c1923e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f0394006ed0 tx=0x7f039400b960).stop 2021-02-10T11:21:53.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.274 7f03a4182700 1 -- 172.21.2.101:0/4055253581 shutdown_connections 2021-02-10T11:21:53.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.274 7f03a4182700 1 --2- 172.21.2.101:0/4055253581 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f039c0acf90 0x7f039c1923e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:53.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.274 7f03a4182700 1 --2- 172.21.2.101:0/4055253581 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f039c0b24b0 0x7f039c191dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:53.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.275 7f03a4182700 1 --2- 172.21.2.101:0/4055253581 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f03840186c0 0x7f038401abc0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:53.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.275 7f03a4182700 1 --2- 172.21.2.101:0/4055253581 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f039c0b20b0 0x7f039c0ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:53.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.275 7f03a4182700 1 -- 172.21.2.101:0/4055253581 >> 172.21.2.101:0/4055253581 conn(0x7f039c0bf4e0 msgr2=0x7f039c0be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:53.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.275 7f03a4182700 1 -- 172.21.2.101:0/4055253581 shutdown_connections 2021-02-10T11:21:53.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.275 7f03a4182700 1 -- 172.21.2.101:0/4055253581 wait complete. 2021-02-10T11:21:53.290 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839283 2021-02-10T11:21:53.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.698 7f3dd6118700 1 Processor -- start 2021-02-10T11:21:53.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.698 7f3dd6118700 1 -- start start 2021-02-10T11:21:53.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.698 7f3dd6118700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3dd01b0470 0x7f3dd01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:53.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.698 7f3dd6118700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3dd00ef610 0x7f3dd00efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:53.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.698 7f3dd6118700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd00f0b50 con 0x7f3dd00f0030 2021-02-10T11:21:53.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.698 7f3dd6118700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd00e52e0 con 0x7f3dd01b0470 2021-02-10T11:21:53.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.698 7f3dd6118700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd00e54d0 con 0x7f3dd00ef610 2021-02-10T11:21:53.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.698 7f3dceffd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3dd00ef610 0x7f3dd00efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:53.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.698 7f3dcf7fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3dd01b0470 0x7f3dd01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:53.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.698 7f3dcffff700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f3dd00f0030 0x7f3dd00f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:40844/0 (socket says 172.21.2.101:40844) 2021-02-10T11:21:53.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.698 7f3dceffd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3dd00ef610 0x7f3dd00efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:39670/0 (socket says 172.21.2.101:39670) 2021-02-10T11:21:53.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.698 7f3dcffff700 1 -- 172.21.2.101:0/2430624590 learned_addr learned my addr 172.21.2.101:0/2430624590 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:21:53.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.699 7f3dcf7fe700 1 -- 172.21.2.101:0/2430624590 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3dd00ef610 msgr2=0x7f3dd00efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:21:53.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.699 7f3dcf7fe700 1 --2- 172.21.2.101:0/2430624590 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3dd00ef610 0x7f3dd00efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:53.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.699 7f3dcf7fe700 1 -- 172.21.2.101:0/2430624590 >> v1:172.21.2.108:6789/0 conn(0x7f3dd00f0030 legacy=0x7f3dd00f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:53.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.699 7f3dcf7fe700 1 -- 172.21.2.101:0/2430624590 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd00f0430 con 0x7f3dd01b0470 2021-02-10T11:21:53.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.699 7f3dcf7fe700 1 --2- 172.21.2.101:0/2430624590 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3dd01b0470 0x7f3dd01b0870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f3dc8006650 tx=0x7f3dc8009a80).ready entity=mon.1 client_cookie=62e7f63aca0eca1b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:53.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.700 7f3dce7fc700 1 -- 172.21.2.101:0/2430624590 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3dc80009b0 con 0x7f3dd01b0470 2021-02-10T11:21:53.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.700 7f3dce7fc700 1 -- 172.21.2.101:0/2430624590 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc800c0c0 con 0x7f3dd01b0470 2021-02-10T11:21:53.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.700 7f3dce7fc700 1 -- 172.21.2.101:0/2430624590 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3dc800c520 con 0x7f3dd01b0470 2021-02-10T11:21:53.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.700 7f3dd6118700 1 -- 172.21.2.101:0/2430624590 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3dd01b0470 msgr2=0x7f3dd01b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:53.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.700 7f3dd6118700 1 --2- 172.21.2.101:0/2430624590 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3dd01b0470 0x7f3dd01b0870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f3dc8006650 tx=0x7f3dc8009a80).stop 2021-02-10T11:21:53.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.700 7f3dd6118700 1 -- 172.21.2.101:0/2430624590 shutdown_connections 2021-02-10T11:21:53.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.700 7f3dd6118700 1 --2- 172.21.2.101:0/2430624590 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3dd00ef610 0x7f3dd00efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:53.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.700 7f3dd6118700 1 --2- 172.21.2.101:0/2430624590 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3dd01b0470 0x7f3dd01b0870 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:53.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.700 7f3dd6118700 1 -- 172.21.2.101:0/2430624590 >> 172.21.2.101:0/2430624590 conn(0x7f3dd00ce3e0 msgr2=0x7f3dd00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:53.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.700 7f3dd6118700 1 -- 172.21.2.101:0/2430624590 shutdown_connections 2021-02-10T11:21:53.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.700 7f3dd6118700 1 -- 172.21.2.101:0/2430624590 wait complete. 2021-02-10T11:21:53.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.715 7f3dd6118700 1 Processor -- start 2021-02-10T11:21:53.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.715 7f3dd6118700 1 -- start start 2021-02-10T11:21:53.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.715 7f3dd6118700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3dd00a0420 0x7f3dd00a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:53.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.715 7f3dd6118700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3dd0189d00 0x7f3dd018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:53.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.715 7f3dd6118700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3dd018a720 0x7f3dd00b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:53.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.715 7f3dd6118700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd0189780 con 0x7f3dd00a0420 2021-02-10T11:21:53.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.715 7f3dd6118700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd018ab20 con 0x7f3dd0189d00 2021-02-10T11:21:53.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.715 7f3dd6118700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd00a1140 con 0x7f3dd018a720 2021-02-10T11:21:53.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.715 7f3dcf7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3dd00a0420 0x7f3dd00a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:53.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.715 7f3dcf7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3dd00a0420 0x7f3dd00a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46942/0 (socket says 172.21.2.101:46942) 2021-02-10T11:21:53.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.715 7f3dcf7fe700 1 -- 172.21.2.101:0/1840197730 learned_addr learned my addr 172.21.2.101:0/1840197730 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:21:53.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.716 7f3dcf7fe700 1 -- 172.21.2.101:0/1840197730 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3dd018a720 msgr2=0x7f3dd00b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:53.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.716 7f3dcf7fe700 1 --2- 172.21.2.101:0/1840197730 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3dd018a720 0x7f3dd00b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:53.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.716 7f3dcf7fe700 1 -- 172.21.2.101:0/1840197730 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3dd0189d00 msgr2=0x7f3dd018a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:53.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.716 7f3dcf7fe700 1 --2- 172.21.2.101:0/1840197730 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3dd0189d00 0x7f3dd018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:53.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.716 7f3dcf7fe700 1 -- 172.21.2.101:0/1840197730 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd00b8200 con 0x7f3dd00a0420 2021-02-10T11:21:53.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.716 7f3dcf7fe700 1 --2- 172.21.2.101:0/1840197730 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3dd00a0420 0x7f3dd00a0820 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f3dc8009630 tx=0x7f3dc800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:53.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.716 7f3db7fff700 1 -- 172.21.2.101:0/1840197730 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3dc80093a0 con 0x7f3dd00a0420 2021-02-10T11:21:53.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.716 7f3db7fff700 1 -- 172.21.2.101:0/1840197730 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc800bd70 con 0x7f3dd00a0420 2021-02-10T11:21:53.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.716 7f3dd6118700 1 -- 172.21.2.101:0/1840197730 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd00b8200 con 0x7f3dd00a0420 2021-02-10T11:21:53.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.716 7f3db7fff700 1 -- 172.21.2.101:0/1840197730 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3dc8005560 con 0x7f3dd00a0420 2021-02-10T11:21:53.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.716 7f3dd6118700 1 -- 172.21.2.101:0/1840197730 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd00b8790 con 0x7f3dd00a0420 2021-02-10T11:21:53.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.717 7f3db7fff700 1 -- 172.21.2.101:0/1840197730 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3dc8005740 con 0x7f3dd00a0420 2021-02-10T11:21:53.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.717 7f3db7fff700 1 --2- 172.21.2.101:0/1840197730 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3db00186c0 0x7f3db001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:21:53.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.717 7f3db7fff700 1 -- 172.21.2.101:0/1840197730 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3dc8032680 con 0x7f3dd00a0420 2021-02-10T11:21:53.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.717 7f3dceffd700 1 --2- 172.21.2.101:0/1840197730 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3db00186c0 0x7f3db001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:21:53.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.717 7f3dceffd700 1 --2- 172.21.2.101:0/1840197730 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3db00186c0 0x7f3db001abc0 crc :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:21:53.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.748 7f3dd6118700 1 -- 172.21.2.101:0/1840197730 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd0105ae0 con 0x7f3dd00a0420 2021-02-10T11:21:53.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.751 7f3db7fff700 1 -- 172.21.2.101:0/1840197730 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3dc8005530 con 0x7f3dd00a0420 2021-02-10T11:21:53.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.885 7f3dd6118700 1 -- 172.21.2.101:0/1840197730 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839283", "target": ["mgr", ""]}) v1 -- 0x7f3dd0189780 con 0x7f3db00186c0 2021-02-10T11:21:53.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.888 7f3db7fff700 1 -- 172.21.2.101:0/1840197730 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f3dc0006e00 con 0x7f3db00186c0 2021-02-10T11:21:53.895 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839283/3005e802-71d0-4245-98ab-254d779f6cbc 2021-02-10T11:21:53.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.889 7f3dd6118700 1 -- 172.21.2.101:0/1840197730 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3db00186c0 msgr2=0x7f3db001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:53.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.889 7f3dd6118700 1 --2- 172.21.2.101:0/1840197730 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3db00186c0 0x7f3db001abc0 crc :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:53.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.889 7f3dd6118700 1 -- 172.21.2.101:0/1840197730 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3dd00a0420 msgr2=0x7f3dd00a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:21:53.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.889 7f3dd6118700 1 --2- 172.21.2.101:0/1840197730 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3dd00a0420 0x7f3dd00a0820 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f3dc8009630 tx=0x7f3dc800b980).stop 2021-02-10T11:21:53.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.889 7f3dd6118700 1 -- 172.21.2.101:0/1840197730 shutdown_connections 2021-02-10T11:21:53.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.889 7f3dd6118700 1 --2- 172.21.2.101:0/1840197730 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3db00186c0 0x7f3db001abc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:53.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.889 7f3dd6118700 1 --2- 172.21.2.101:0/1840197730 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3dd018a720 0x7f3dd00b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:53.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.889 7f3dd6118700 1 --2- 172.21.2.101:0/1840197730 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3dd0189d00 0x7f3dd018a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:21:53.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.889 7f3dd6118700 1 --2- 172.21.2.101:0/1840197730 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3dd00a0420 0x7f3dd00a0820 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:21:53.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.889 7f3dd6118700 1 -- 172.21.2.101:0/1840197730 >> 172.21.2.101:0/1840197730 conn(0x7f3dd0094e10 msgr2=0x7f3dd00a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:21:53.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.889 7f3dd6118700 1 -- 172.21.2.101:0/1840197730 shutdown_connections 2021-02-10T11:21:53.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:21:53.889 7f3dd6118700 1 -- 172.21.2.101:0/1840197730 wait complete. 2021-02-10T11:21:53.905 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839283 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839283/3005e802-71d0-4245-98ab-254d779f6cbc 2021-02-10T11:21:53.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:53.906 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:53.906 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:54.243 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:54.243 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:54.243 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839283/3005e802-71d0-4245-98ab-254d779f6cbc/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:54.621 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:21:54.622 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:21:54.622 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242143 s, 43.3 MB/s 2021-02-10T11:21:54.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:21:54.623 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:21:54.623 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:21:54.958 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:21:54.958 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:21:54.959 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839283/3005e802-71d0-4245-98ab-254d779f6cbc/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:21:55.333 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:02.153 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:02.153 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237283 s, 44.2 MB/s 2021-02-10T11:22:02.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:02.154 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:02.154 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:02.489 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:02.489 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:02.490 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839283/3005e802-71d0-4245-98ab-254d779f6cbc/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:02.861 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:02.862 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:02.862 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244005 s, 43.0 MB/s 2021-02-10T11:22:02.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:02.863 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:02.863 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:03.199 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:03.199 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:03.200 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839283/3005e802-71d0-4245-98ab-254d779f6cbc/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:03.575 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:03.576 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:03.576 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239752 s, 43.7 MB/s 2021-02-10T11:22:03.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:03.577 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:03.577 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:03.914 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:03.914 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:03.915 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839283/3005e802-71d0-4245-98ab-254d779f6cbc/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:04.288 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:04.288 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:04.289 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244395 s, 42.9 MB/s 2021-02-10T11:22:04.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:04.290 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:04.290 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:04.626 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:04.627 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:04.627 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839283/3005e802-71d0-4245-98ab-254d779f6cbc/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:05.002 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:05.002 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:05.003 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.025156 s, 41.7 MB/s 2021-02-10T11:22:05.003 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:05.004 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:05.004 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:05.338 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:05.339 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:05.339 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839283/3005e802-71d0-4245-98ab-254d779f6cbc/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:05.713 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:05.714 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:05.714 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0232164 s, 45.2 MB/s 2021-02-10T11:22:05.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:05.715 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:05.715 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:06.050 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:06.050 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:06.051 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839283/3005e802-71d0-4245-98ab-254d779f6cbc/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:06.427 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:06.427 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:06.427 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249103 s, 42.1 MB/s 2021-02-10T11:22:06.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:06.428 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:06.428 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:06.762 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:06.762 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:06.762 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839283/3005e802-71d0-4245-98ab-254d779f6cbc/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:07.137 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:07.137 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:07.138 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248894 s, 42.1 MB/s 2021-02-10T11:22:07.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:07.139 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:07.139 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:07.473 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:07.473 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:07.474 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839283/3005e802-71d0-4245-98ab-254d779f6cbc/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:07.850 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:07.850 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:07.850 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236696 s, 44.3 MB/s 2021-02-10T11:22:07.851 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839284 2021-02-10T11:22:08.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.259 7f3a179e5700 1 Processor -- start 2021-02-10T11:22:08.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.259 7f3a179e5700 1 -- start start 2021-02-10T11:22:08.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.259 7f3a179e5700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3a101b0870 0x7f3a100f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:08.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.259 7f3a179e5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a100efe00 0x7f3a100f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:08.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.259 7f3a179e5700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a101b0c70 con 0x7f3a101b0470 2021-02-10T11:22:08.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.259 7f3a179e5700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a100f0b50 con 0x7f3a101b0870 2021-02-10T11:22:08.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.259 7f3a179e5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a100f0820 con 0x7f3a100efe00 2021-02-10T11:22:08.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.259 7f3a15000700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3a101b0870 0x7f3a100f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:08.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.260 7f3a15000700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3a101b0870 0x7f3a100f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46952/0 (socket says 172.21.2.101:46952) 2021-02-10T11:22:08.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.260 7f3a15000700 1 -- 172.21.2.101:0/3228023330 learned_addr learned my addr 172.21.2.101:0/3228023330 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:22:08.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.260 7f3a15801700 1 --1- 172.21.2.101:0/3228023330 >> v1:172.21.2.101:6789/0 conn(0x7f3a101b0470 0x7f3a100ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47444/0 (socket says 172.21.2.101:47444) 2021-02-10T11:22:08.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.260 7f3a16002700 1 --2- 172.21.2.101:0/3228023330 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a100efe00 0x7f3a100f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:08.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.260 7f3a15000700 1 -- 172.21.2.101:0/3228023330 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a100efe00 msgr2=0x7f3a100f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:22:08.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.260 7f3a15000700 1 --2- 172.21.2.101:0/3228023330 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a100efe00 0x7f3a100f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:08.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.260 7f3a15000700 1 -- 172.21.2.101:0/3228023330 >> v1:172.21.2.101:6789/0 conn(0x7f3a101b0470 legacy=0x7f3a100ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:08.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.260 7f3a07fff700 1 -- 172.21.2.101:0/3228023330 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3282494388 0 0) 0x7f3a00000ce0 con 0x7f3a101b0470 2021-02-10T11:22:08.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.260 7f3a15000700 1 -- 172.21.2.101:0/3228023330 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a100e52e0 con 0x7f3a101b0870 2021-02-10T11:22:08.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.260 7f3a15000700 1 --2- 172.21.2.101:0/3228023330 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a101b0870 0x7f3a100f20e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f3a0c006700 tx=0x7f3a0c009b00).ready entity=mon.0 client_cookie=8cd1314240af5f10 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:08.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.260 7f3a07fff700 1 -- 172.21.2.101:0/3228023330 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3a0c00c0c0 con 0x7f3a101b0870 2021-02-10T11:22:08.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.260 7f3a07fff700 1 -- 172.21.2.101:0/3228023330 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0c00c400 con 0x7f3a101b0870 2021-02-10T11:22:08.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.260 7f3a07fff700 1 -- 172.21.2.101:0/3228023330 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3a0c00c920 con 0x7f3a101b0870 2021-02-10T11:22:08.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.260 7f3a179e5700 1 -- 172.21.2.101:0/3228023330 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a101b0870 msgr2=0x7f3a100f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:08.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.260 7f3a179e5700 1 --2- 172.21.2.101:0/3228023330 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a101b0870 0x7f3a100f20e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f3a0c006700 tx=0x7f3a0c009b00).stop 2021-02-10T11:22:08.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.261 7f3a179e5700 1 -- 172.21.2.101:0/3228023330 shutdown_connections 2021-02-10T11:22:08.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.261 7f3a179e5700 1 --2- 172.21.2.101:0/3228023330 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a100efe00 0x7f3a100f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:08.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.261 7f3a179e5700 1 --2- 172.21.2.101:0/3228023330 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a101b0870 0x7f3a100f20e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:08.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.261 7f3a179e5700 1 -- 172.21.2.101:0/3228023330 >> 172.21.2.101:0/3228023330 conn(0x7f3a100ce3e0 msgr2=0x7f3a100e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:08.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.261 7f3a179e5700 1 -- 172.21.2.101:0/3228023330 shutdown_connections 2021-02-10T11:22:08.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.262 7f3a179e5700 1 -- 172.21.2.101:0/3228023330 wait complete. 2021-02-10T11:22:08.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.276 7f3a179e5700 1 Processor -- start 2021-02-10T11:22:08.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.276 7f3a179e5700 1 -- start start 2021-02-10T11:22:08.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.277 7f3a179e5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a100b8950 0x7f3a100a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:08.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.277 7f3a179e5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3a100a6830 0x7f3a101a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:08.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.277 7f3a179e5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a101a3400 0x7f3a101a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:08.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.277 7f3a179e5700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a100a6cb0 con 0x7f3a100b8950 2021-02-10T11:22:08.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.277 7f3a179e5700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a100b8d50 con 0x7f3a100a6830 2021-02-10T11:22:08.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.277 7f3a179e5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a100a6f70 con 0x7f3a101a3400 2021-02-10T11:22:08.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.277 7f3a15801700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a100b8950 0x7f3a100a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:08.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.277 7f3a15801700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a100b8950 0x7f3a100a6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46958/0 (socket says 172.21.2.101:46958) 2021-02-10T11:22:08.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.277 7f3a15801700 1 -- 172.21.2.101:0/1683846513 learned_addr learned my addr 172.21.2.101:0/1683846513 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:22:08.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.277 7f3a16002700 1 --2- 172.21.2.101:0/1683846513 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a101a3400 0x7f3a101a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:08.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.277 7f3a15000700 1 --2- 172.21.2.101:0/1683846513 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3a100a6830 0x7f3a101a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:08.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.277 7f3a15801700 1 -- 172.21.2.101:0/1683846513 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a101a3400 msgr2=0x7f3a101a3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:08.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.277 7f3a15801700 1 --2- 172.21.2.101:0/1683846513 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a101a3400 0x7f3a101a3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:08.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.277 7f3a15801700 1 -- 172.21.2.101:0/1683846513 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3a100a6830 msgr2=0x7f3a101a2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:08.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.277 7f3a15801700 1 --2- 172.21.2.101:0/1683846513 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3a100a6830 0x7f3a101a2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:08.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.277 7f3a15801700 1 -- 172.21.2.101:0/1683846513 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a100a96a0 con 0x7f3a100b8950 2021-02-10T11:22:08.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.277 7f3a15801700 1 --2- 172.21.2.101:0/1683846513 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a100b8950 0x7f3a100a6210 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f3a00006f00 tx=0x7f3a0000b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:08.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.278 7f3a05ffb700 1 -- 172.21.2.101:0/1683846513 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3a0000bd30 con 0x7f3a100b8950 2021-02-10T11:22:08.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.278 7f3a05ffb700 1 -- 172.21.2.101:0/1683846513 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a000060d0 con 0x7f3a100b8950 2021-02-10T11:22:08.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.278 7f3a05ffb700 1 -- 172.21.2.101:0/1683846513 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3a000043a0 con 0x7f3a100b8950 2021-02-10T11:22:08.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.278 7f3a179e5700 1 -- 172.21.2.101:0/1683846513 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a100a96a0 con 0x7f3a100b8950 2021-02-10T11:22:08.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.278 7f3a179e5700 1 -- 172.21.2.101:0/1683846513 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a100a96a0 con 0x7f3a100b8950 2021-02-10T11:22:08.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.278 7f3a05ffb700 1 -- 172.21.2.101:0/1683846513 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3a000062b0 con 0x7f3a100b8950 2021-02-10T11:22:08.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.279 7f3a05ffb700 1 --2- 172.21.2.101:0/1683846513 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f39f00186c0 0x7f39f001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:08.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.279 7f3a05ffb700 1 -- 172.21.2.101:0/1683846513 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3a00032550 con 0x7f3a100b8950 2021-02-10T11:22:08.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.279 7f3a15000700 1 --2- 172.21.2.101:0/1683846513 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f39f00186c0 0x7f39f001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:08.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.279 7f3a15000700 1 --2- 172.21.2.101:0/1683846513 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f39f00186c0 0x7f39f001abc0 crc :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:08.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.309 7f3a179e5700 1 -- 172.21.2.101:0/1683846513 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a100b8d50 con 0x7f3a100b8950 2021-02-10T11:22:08.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.312 7f3a05ffb700 1 -- 172.21.2.101:0/1683846513 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3a00006b90 con 0x7f3a100b8950 2021-02-10T11:22:08.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.438 7f3a179e5700 1 -- 172.21.2.101:0/1683846513 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839284", "target": ["mgr", ""]}) v1 -- 0x7f3a100b8d50 con 0x7f39f00186c0 2021-02-10T11:22:08.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.464 7f3a05ffb700 1 -- 172.21.2.101:0/1683846513 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f3a0c009440 con 0x7f39f00186c0 2021-02-10T11:22:08.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.471 7f3a179e5700 1 -- 172.21.2.101:0/1683846513 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f39f00186c0 msgr2=0x7f39f001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:08.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.471 7f3a179e5700 1 --2- 172.21.2.101:0/1683846513 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f39f00186c0 0x7f39f001abc0 crc :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:08.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.471 7f3a179e5700 1 -- 172.21.2.101:0/1683846513 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a100b8950 msgr2=0x7f3a100a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:08.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.471 7f3a179e5700 1 --2- 172.21.2.101:0/1683846513 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a100b8950 0x7f3a100a6210 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f3a00006f00 tx=0x7f3a0000b940).stop 2021-02-10T11:22:08.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.471 7f3a179e5700 1 -- 172.21.2.101:0/1683846513 shutdown_connections 2021-02-10T11:22:08.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.471 7f3a179e5700 1 --2- 172.21.2.101:0/1683846513 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f39f00186c0 0x7f39f001abc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:08.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.471 7f3a179e5700 1 --2- 172.21.2.101:0/1683846513 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a101a3400 0x7f3a101a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:08.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.471 7f3a179e5700 1 --2- 172.21.2.101:0/1683846513 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3a100a6830 0x7f3a101a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:08.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.471 7f3a179e5700 1 --2- 172.21.2.101:0/1683846513 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a100b8950 0x7f3a100a6210 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:08.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.471 7f3a179e5700 1 -- 172.21.2.101:0/1683846513 >> 172.21.2.101:0/1683846513 conn(0x7f3a100bbf60 msgr2=0x7f3a10105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:08.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.471 7f3a179e5700 1 -- 172.21.2.101:0/1683846513 shutdown_connections 2021-02-10T11:22:08.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.471 7f3a179e5700 1 -- 172.21.2.101:0/1683846513 wait complete. 2021-02-10T11:22:08.487 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839284 2021-02-10T11:22:08.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.895 7f125849b700 1 Processor -- start 2021-02-10T11:22:08.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.896 7f125849b700 1 -- start start 2021-02-10T11:22:08.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.896 7f125849b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f12501b0470 0x7f12501b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:08.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.896 7f125849b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12500f0b50 con 0x7f12500f20e0 2021-02-10T11:22:08.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.896 7f125849b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12500f24e0 con 0x7f12500ef610 2021-02-10T11:22:08.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.896 7f125849b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12500e5ea0 con 0x7f12501b0470 2021-02-10T11:22:08.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.896 7f1256ab8700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f12500f20e0 0x7f12500f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47458/0 (socket says 172.21.2.101:47458) 2021-02-10T11:22:08.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.896 7f1256ab8700 1 -- 172.21.2.101:0/2478588573 learned_addr learned my addr 172.21.2.101:0/2478588573 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:22:08.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.897 7f12552b5700 1 -- 172.21.2.101:0/2478588573 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3313463382 0 0) 0x7f1248001500 con 0x7f12500f20e0 2021-02-10T11:22:08.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.897 7f12552b5700 1 -- 172.21.2.101:0/2478588573 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f123c002fe0 con 0x7f12500f20e0 2021-02-10T11:22:08.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.897 7f12562b7700 1 --2- 172.21.2.101:0/2478588573 >> v2:172.21.2.108:3300/0 conn(0x7f12501b0470 0x7f12501b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:08.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.897 7f12552b5700 1 -- 172.21.2.101:0/2478588573 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3131834678 0 0) 0x7f1248001460 con 0x7f12500f20e0 2021-02-10T11:22:08.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.897 7f12552b5700 1 -- 172.21.2.101:0/2478588573 >> v2:172.21.2.108:3300/0 conn(0x7f12501b0470 msgr2=0x7f12501b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:22:08.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.897 7f12552b5700 1 --2- 172.21.2.101:0/2478588573 >> v2:172.21.2.108:3300/0 conn(0x7f12501b0470 0x7f12501b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:08.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.897 7f12552b5700 1 -- 172.21.2.101:0/2478588573 >> v1:172.21.2.108:6789/0 conn(0x7f12500ef610 legacy=0x7f12500efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:22:08.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.897 7f12552b5700 1 -- 172.21.2.101:0/2478588573 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12500e52e0 con 0x7f12500f20e0 2021-02-10T11:22:08.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.897 7f12552b5700 1 -- 172.21.2.101:0/2478588573 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f1248001980 con 0x7f12500f20e0 2021-02-10T11:22:08.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.897 7f12552b5700 1 -- 172.21.2.101:0/2478588573 >> v1:172.21.2.101:6789/0 conn(0x7f12500f20e0 legacy=0x7f12500f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:08.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.897 7f12552b5700 1 --2- 172.21.2.101:0/2478588573 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f123c003a70 0x7f123c005f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:08.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.897 7f12552b5700 1 --2- 172.21.2.101:0/2478588573 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f123c006790 0x7f123c008cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:08.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.897 7f12552b5700 1 --2- 172.21.2.101:0/2478588573 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f123c0095b0 0x7f123c00bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:08.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.897 7f12552b5700 1 -- 172.21.2.101:0/2478588573 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c00c300 con 0x7f123c003a70 2021-02-10T11:22:08.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.897 7f12552b5700 1 -- 172.21.2.101:0/2478588573 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c00c520 con 0x7f123c0095b0 2021-02-10T11:22:08.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.897 7f12552b5700 1 -- 172.21.2.101:0/2478588573 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f123c00c740 con 0x7f123c006790 2021-02-10T11:22:08.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.897 7f12562b7700 1 --2- 172.21.2.101:0/2478588573 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f123c003a70 0x7f123c005f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:08.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.898 7f1255ab6700 1 --2- 172.21.2.101:0/2478588573 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f123c006790 0x7f123c008cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:08.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.898 7f1256ab8700 1 --2- 172.21.2.101:0/2478588573 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f123c0095b0 0x7f123c00bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:08.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.898 7f12562b7700 1 -- 172.21.2.101:0/2478588573 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f123c006790 msgr2=0x7f123c008cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:22:08.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.898 7f12562b7700 1 --2- 172.21.2.101:0/2478588573 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f123c006790 0x7f123c008cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:08.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.898 7f12562b7700 1 -- 172.21.2.101:0/2478588573 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f123c0095b0 msgr2=0x7f123c00bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:22:08.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.898 7f12562b7700 1 --2- 172.21.2.101:0/2478588573 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f123c0095b0 0x7f123c00bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:08.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.898 7f12562b7700 1 -- 172.21.2.101:0/2478588573 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123c00c9e0 con 0x7f123c003a70 2021-02-10T11:22:08.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.898 7f1256ab8700 1 -- 172.21.2.101:0/2478588573 reap_dead start 2021-02-10T11:22:08.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.898 7f12562b7700 1 --2- 172.21.2.101:0/2478588573 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f123c003a70 0x7f123c005f70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f1244006a40 tx=0x7f1244006820).ready entity=mon.0 client_cookie=3d9935203479bf61 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:08.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.898 7f12552b5700 1 -- 172.21.2.101:0/2478588573 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1244000c90 con 0x7f123c003a70 2021-02-10T11:22:08.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.898 7f12552b5700 1 -- 172.21.2.101:0/2478588573 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1244002030 con 0x7f123c003a70 2021-02-10T11:22:08.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.898 7f12552b5700 1 -- 172.21.2.101:0/2478588573 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1244002550 con 0x7f123c003a70 2021-02-10T11:22:08.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.898 7f125849b700 1 -- 172.21.2.101:0/2478588573 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f123c003a70 msgr2=0x7f123c005f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:08.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.898 7f125849b700 1 --2- 172.21.2.101:0/2478588573 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f123c003a70 0x7f123c005f70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f1244006a40 tx=0x7f1244006820).stop 2021-02-10T11:22:08.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.899 7f125849b700 1 -- 172.21.2.101:0/2478588573 shutdown_connections 2021-02-10T11:22:08.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.899 7f125849b700 1 --2- 172.21.2.101:0/2478588573 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f123c003a70 0x7f123c005f70 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:08.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.899 7f125849b700 1 -- 172.21.2.101:0/2478588573 >> 172.21.2.101:0/2478588573 conn(0x7f12500ce3e0 msgr2=0x7f12500e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:08.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.899 7f125849b700 1 -- 172.21.2.101:0/2478588573 shutdown_connections 2021-02-10T11:22:08.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.899 7f125849b700 1 -- 172.21.2.101:0/2478588573 wait complete. 2021-02-10T11:22:08.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.914 7f125849b700 1 Processor -- start 2021-02-10T11:22:08.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.914 7f125849b700 1 -- start start 2021-02-10T11:22:08.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.914 7f125849b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f12500b7db0 0x7f12500b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:08.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.914 7f125849b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12500b8880 0x7f12501a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:08.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.914 7f125849b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f12500b8c80 0x7f12501a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:08.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.914 7f125849b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12500b79b0 con 0x7f12500b8880 2021-02-10T11:22:08.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.914 7f125849b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12501a8090 con 0x7f12500b8c80 2021-02-10T11:22:08.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.914 7f125849b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12501a82b0 con 0x7f12500b7db0 2021-02-10T11:22:08.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.914 7f1255ab6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12500b8880 0x7f12501a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:08.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.915 7f1255ab6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12500b8880 0x7f12501a7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46976/0 (socket says 172.21.2.101:46976) 2021-02-10T11:22:08.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.915 7f1255ab6700 1 -- 172.21.2.101:0/51060007 learned_addr learned my addr 172.21.2.101:0/51060007 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:22:08.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.915 7f1256ab8700 1 --2- 172.21.2.101:0/51060007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f12500b8c80 0x7f12501a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:08.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.915 7f12562b7700 1 --2- 172.21.2.101:0/51060007 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f12500b7db0 0x7f12500b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:08.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.915 7f1255ab6700 1 -- 172.21.2.101:0/51060007 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f12500b7db0 msgr2=0x7f12500b8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:08.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.915 7f1255ab6700 1 --2- 172.21.2.101:0/51060007 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f12500b7db0 0x7f12500b8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:08.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.915 7f1255ab6700 1 -- 172.21.2.101:0/51060007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f12500b8c80 msgr2=0x7f12501a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:08.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.915 7f1255ab6700 1 --2- 172.21.2.101:0/51060007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f12500b8c80 0x7f12501a7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:08.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.915 7f1255ab6700 1 -- 172.21.2.101:0/51060007 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1250190c10 con 0x7f12500b8880 2021-02-10T11:22:08.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.915 7f1256ab8700 1 --2- 172.21.2.101:0/51060007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f12500b8c80 0x7f12501a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:22:08.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.915 7f1255ab6700 1 --2- 172.21.2.101:0/51060007 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12500b8880 0x7f12501a7450 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f124c006f10 tx=0x7f124c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:08.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.915 7f1242ffd700 1 -- 172.21.2.101:0/51060007 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f124c00bd80 con 0x7f12500b8880 2021-02-10T11:22:08.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.915 7f1242ffd700 1 -- 172.21.2.101:0/51060007 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124c002030 con 0x7f12500b8880 2021-02-10T11:22:08.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.915 7f125849b700 1 -- 172.21.2.101:0/51060007 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1250190c10 con 0x7f12500b8880 2021-02-10T11:22:08.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.915 7f125849b700 1 -- 172.21.2.101:0/51060007 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1250190c10 con 0x7f12500b8880 2021-02-10T11:22:08.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.915 7f1242ffd700 1 -- 172.21.2.101:0/51060007 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f124c002480 con 0x7f12500b8880 2021-02-10T11:22:08.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.916 7f1242ffd700 1 -- 172.21.2.101:0/51060007 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f124c002210 con 0x7f12500b8880 2021-02-10T11:22:08.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.916 7f1242ffd700 1 --2- 172.21.2.101:0/51060007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1238018730 0x7f123801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:08.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.916 7f1242ffd700 1 -- 172.21.2.101:0/51060007 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f124c0327c0 con 0x7f12500b8880 2021-02-10T11:22:08.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.916 7f12562b7700 1 --2- 172.21.2.101:0/51060007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1238018730 0x7f123801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:08.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.917 7f12562b7700 1 --2- 172.21.2.101:0/51060007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1238018730 0x7f123801ac10 crc :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:08.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.945 7f125849b700 1 -- 172.21.2.101:0/51060007 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1250105b30 con 0x7f12500b8880 2021-02-10T11:22:08.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:08.948 7f1242ffd700 1 -- 172.21.2.101:0/51060007 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f124c002240 con 0x7f12500b8880 2021-02-10T11:22:09.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:09.079 7f125849b700 1 -- 172.21.2.101:0/51060007 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839284", "target": ["mgr", ""]}) v1 -- 0x7f12501a8090 con 0x7f1238018730 2021-02-10T11:22:09.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:09.082 7f1242ffd700 1 -- 172.21.2.101:0/51060007 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f124404e550 con 0x7f1238018730 2021-02-10T11:22:09.093 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839284/453d3c2e-f55a-46f8-9592-efea59dd706b 2021-02-10T11:22:09.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:09.087 7f125849b700 1 -- 172.21.2.101:0/51060007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1238018730 msgr2=0x7f123801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:09.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:09.087 7f125849b700 1 --2- 172.21.2.101:0/51060007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1238018730 0x7f123801ac10 crc :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:09.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:09.087 7f125849b700 1 -- 172.21.2.101:0/51060007 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12500b8880 msgr2=0x7f12501a7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:09.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:09.087 7f125849b700 1 --2- 172.21.2.101:0/51060007 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12500b8880 0x7f12501a7450 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f124c006f10 tx=0x7f124c00b990).stop 2021-02-10T11:22:09.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:09.087 7f125849b700 1 -- 172.21.2.101:0/51060007 shutdown_connections 2021-02-10T11:22:09.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:09.087 7f125849b700 1 --2- 172.21.2.101:0/51060007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f12500b8c80 0x7f12501a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:09.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:09.087 7f125849b700 1 --2- 172.21.2.101:0/51060007 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12500b8880 0x7f12501a7450 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:09.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:09.087 7f125849b700 1 --2- 172.21.2.101:0/51060007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1238018730 0x7f123801ac10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:09.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:09.087 7f125849b700 1 --2- 172.21.2.101:0/51060007 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f12500b7db0 0x7f12500b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:09.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:09.087 7f125849b700 1 -- 172.21.2.101:0/51060007 >> 172.21.2.101:0/51060007 conn(0x7f1250094e10 msgr2=0x7f12500a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:09.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:09.087 7f125849b700 1 -- 172.21.2.101:0/51060007 shutdown_connections 2021-02-10T11:22:09.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:09.087 7f125849b700 1 -- 172.21.2.101:0/51060007 wait complete. 2021-02-10T11:22:09.103 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839284 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839284/453d3c2e-f55a-46f8-9592-efea59dd706b 2021-02-10T11:22:09.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:09.103 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:09.103 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:09.438 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:09.439 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:09.439 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839284/453d3c2e-f55a-46f8-9592-efea59dd706b/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:09.816 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:09.816 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:09.816 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245431 s, 42.7 MB/s 2021-02-10T11:22:09.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:09.818 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:09.818 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:10.152 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:10.152 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:10.152 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839284/453d3c2e-f55a-46f8-9592-efea59dd706b/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:10.528 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:10.529 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:10.529 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240266 s, 43.6 MB/s 2021-02-10T11:22:10.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:10.530 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:10.530 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:10.866 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:10.866 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:10.866 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839284/453d3c2e-f55a-46f8-9592-efea59dd706b/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:11.241 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:11.241 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:11.241 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242008 s, 43.3 MB/s 2021-02-10T11:22:11.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:11.243 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:11.243 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:11.577 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:11.577 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:11.578 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839284/453d3c2e-f55a-46f8-9592-efea59dd706b/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:11.955 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:11.955 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:11.955 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247679 s, 42.3 MB/s 2021-02-10T11:22:11.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:11.956 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:11.956 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:12.291 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:12.292 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:12.292 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839284/453d3c2e-f55a-46f8-9592-efea59dd706b/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:12.668 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:12.668 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:12.668 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253595 s, 41.3 MB/s 2021-02-10T11:22:12.669 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:12.669 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:12.669 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:13.005 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:13.006 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:13.006 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839284/453d3c2e-f55a-46f8-9592-efea59dd706b/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:13.383 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:13.383 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:13.383 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0258697 s, 40.5 MB/s 2021-02-10T11:22:13.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:13.385 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:13.385 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:13.720 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:13.720 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:13.721 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839284/453d3c2e-f55a-46f8-9592-efea59dd706b/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:14.091 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:14.091 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:14.092 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0228127 s, 46.0 MB/s 2021-02-10T11:22:14.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:14.093 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:14.093 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:14.429 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:14.429 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:14.429 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839284/453d3c2e-f55a-46f8-9592-efea59dd706b/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:14.804 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:14.804 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:14.804 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0230179 s, 45.6 MB/s 2021-02-10T11:22:14.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:14.805 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:14.805 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:15.139 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:15.140 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:15.140 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839284/453d3c2e-f55a-46f8-9592-efea59dd706b/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:15.515 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:15.515 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:15.515 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238409 s, 44.0 MB/s 2021-02-10T11:22:15.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:15.516 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:15.516 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:15.850 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:15.851 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:15.851 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839284/453d3c2e-f55a-46f8-9592-efea59dd706b/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:16.227 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:16.227 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:16.227 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239548 s, 43.8 MB/s 2021-02-10T11:22:16.228 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839285 2021-02-10T11:22:16.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.636 7fd4d2c39700 1 Processor -- start 2021-02-10T11:22:16.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.636 7fd4d2c39700 1 -- start start 2021-02-10T11:22:16.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.636 7fd4d2c39700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4cc1b0470 0x7fd4cc1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:16.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.636 7fd4d2c39700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd4cc0f20e0 0x7fd4cc0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:16.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.636 7fd4d2c39700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4cc0f0b50 con 0x7fd4cc0ef610 2021-02-10T11:22:16.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.636 7fd4d2c39700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4cc0f0820 con 0x7fd4cc0f20e0 2021-02-10T11:22:16.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.636 7fd4d2c39700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc0f24e0 con 0x7fd4cc1b0470 2021-02-10T11:22:16.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.637 7fd4d0a55700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4cc1b0470 0x7fd4cc1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:16.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.637 7fd4d1256700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd4cc0f20e0 0x7fd4cc0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:16.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.637 7fd4cbfff700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fd4cc0ef610 0x7fd4cc0efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:40888/0 (socket says 172.21.2.101:40888) 2021-02-10T11:22:16.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.637 7fd4cbfff700 1 -- 172.21.2.101:0/3969729958 learned_addr learned my addr 172.21.2.101:0/3969729958 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:22:16.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.637 7fd4d0a55700 1 --2- 172.21.2.101:0/3969729958 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4cc1b0470 0x7fd4cc1b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:39714/0 (socket says 172.21.2.101:39714) 2021-02-10T11:22:16.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.637 7fd4d0a55700 1 -- 172.21.2.101:0/3969729958 >> v2:172.21.2.108:3300/0 conn(0x7fd4cc0f20e0 msgr2=0x7fd4cc0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:22:16.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.637 7fd4d0a55700 1 --2- 172.21.2.101:0/3969729958 >> v2:172.21.2.108:3300/0 conn(0x7fd4cc0f20e0 0x7fd4cc0f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:16.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.637 7fd4d0a55700 1 -- 172.21.2.101:0/3969729958 >> v1:172.21.2.108:6789/0 conn(0x7fd4cc0ef610 legacy=0x7fd4cc0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:16.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.637 7fd4d0a55700 1 -- 172.21.2.101:0/3969729958 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc0e52e0 con 0x7fd4cc1b0470 2021-02-10T11:22:16.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.637 7fd4d1256700 1 --2- 172.21.2.101:0/3969729958 >> v2:172.21.2.108:3300/0 conn(0x7fd4cc0f20e0 0x7fd4cc0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:22:16.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.638 7fd4d0a55700 1 --2- 172.21.2.101:0/3969729958 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4cc1b0470 0x7fd4cc1b0870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fd4b4006650 tx=0x7fd4b4009a80).ready entity=mon.2 client_cookie=3e0caf626d6d2bd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:16.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.638 7fd4cb7fe700 1 -- 172.21.2.101:0/3969729958 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd4b4006390 con 0x7fd4cc1b0470 2021-02-10T11:22:16.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.638 7fd4cb7fe700 1 -- 172.21.2.101:0/3969729958 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b40009b0 con 0x7fd4cc1b0470 2021-02-10T11:22:16.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.638 7fd4cb7fe700 1 -- 172.21.2.101:0/3969729958 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd4b400c2b0 con 0x7fd4cc1b0470 2021-02-10T11:22:16.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.638 7fd4d2c39700 1 -- 172.21.2.101:0/3969729958 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4cc1b0470 msgr2=0x7fd4cc1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:16.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.638 7fd4d2c39700 1 --2- 172.21.2.101:0/3969729958 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4cc1b0470 0x7fd4cc1b0870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fd4b4006650 tx=0x7fd4b4009a80).stop 2021-02-10T11:22:16.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.639 7fd4d2c39700 1 -- 172.21.2.101:0/3969729958 shutdown_connections 2021-02-10T11:22:16.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.639 7fd4d2c39700 1 --2- 172.21.2.101:0/3969729958 >> v2:172.21.2.108:3300/0 conn(0x7fd4cc0f20e0 0x7fd4cc0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:16.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.639 7fd4d2c39700 1 --2- 172.21.2.101:0/3969729958 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4cc1b0470 0x7fd4cc1b0870 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:16.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.639 7fd4d2c39700 1 -- 172.21.2.101:0/3969729958 >> 172.21.2.101:0/3969729958 conn(0x7fd4cc0ce3e0 msgr2=0x7fd4cc0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:16.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.639 7fd4d2c39700 1 -- 172.21.2.101:0/3969729958 shutdown_connections 2021-02-10T11:22:16.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.639 7fd4d2c39700 1 -- 172.21.2.101:0/3969729958 wait complete. 2021-02-10T11:22:16.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.653 7fd4d2c39700 1 Processor -- start 2021-02-10T11:22:16.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.653 7fd4d2c39700 1 -- start start 2021-02-10T11:22:16.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.654 7fd4d2c39700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4cc08edf0 0x7fd4cc192870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:16.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.654 7fd4d2c39700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4cc192e90 0x7fd4cc08ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:16.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.654 7fd4d2c39700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4cc193290 0x7fd4cc08d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:16.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.654 7fd4d2c39700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc08c4a0 con 0x7fd4cc192e90 2021-02-10T11:22:16.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.654 7fd4d2c39700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc08f1f0 con 0x7fd4cc193290 2021-02-10T11:22:16.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.654 7fd4d2c39700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc09fd30 con 0x7fd4cc08edf0 2021-02-10T11:22:16.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.654 7fd4cbfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4cc192e90 0x7fd4cc08ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:16.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.654 7fd4cbfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4cc192e90 0x7fd4cc08ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46990/0 (socket says 172.21.2.101:46990) 2021-02-10T11:22:16.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.654 7fd4cbfff700 1 -- 172.21.2.101:0/4073826181 learned_addr learned my addr 172.21.2.101:0/4073826181 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:22:16.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.654 7fd4d0a55700 1 --2- 172.21.2.101:0/4073826181 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4cc08edf0 0x7fd4cc192870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:16.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.655 7fd4d1256700 1 --2- 172.21.2.101:0/4073826181 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4cc193290 0x7fd4cc08d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:16.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.655 7fd4cbfff700 1 -- 172.21.2.101:0/4073826181 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4cc08edf0 msgr2=0x7fd4cc192870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:16.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.655 7fd4cbfff700 1 --2- 172.21.2.101:0/4073826181 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4cc08edf0 0x7fd4cc192870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:16.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.655 7fd4cbfff700 1 -- 172.21.2.101:0/4073826181 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4cc193290 msgr2=0x7fd4cc08d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:16.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.655 7fd4cbfff700 1 --2- 172.21.2.101:0/4073826181 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4cc193290 0x7fd4cc08d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:16.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.655 7fd4cbfff700 1 -- 172.21.2.101:0/4073826181 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc08d660 con 0x7fd4cc192e90 2021-02-10T11:22:16.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.655 7fd4d1256700 1 --2- 172.21.2.101:0/4073826181 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4cc193290 0x7fd4cc08d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:22:16.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.655 7fd4cbfff700 1 --2- 172.21.2.101:0/4073826181 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4cc192e90 0x7fd4cc08ca20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7fd4bc001c60 tx=0x7fd4bc00a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:16.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.655 7fd4c97fa700 1 -- 172.21.2.101:0/4073826181 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd4bc00acc0 con 0x7fd4cc192e90 2021-02-10T11:22:16.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.655 7fd4c97fa700 1 -- 172.21.2.101:0/4073826181 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4bc004140 con 0x7fd4cc192e90 2021-02-10T11:22:16.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.655 7fd4c97fa700 1 -- 172.21.2.101:0/4073826181 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd4bc004620 con 0x7fd4cc192e90 2021-02-10T11:22:16.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.655 7fd4d2c39700 1 -- 172.21.2.101:0/4073826181 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4cc08d660 con 0x7fd4cc192e90 2021-02-10T11:22:16.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.655 7fd4d2c39700 1 -- 172.21.2.101:0/4073826181 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4cc08d660 con 0x7fd4cc192e90 2021-02-10T11:22:16.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.656 7fd4c97fa700 1 -- 172.21.2.101:0/4073826181 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd4bc000d90 con 0x7fd4cc192e90 2021-02-10T11:22:16.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.656 7fd4c97fa700 1 --2- 172.21.2.101:0/4073826181 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4b00186c0 0x7fd4b001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:16.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.656 7fd4c97fa700 1 -- 172.21.2.101:0/4073826181 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd4bc032540 con 0x7fd4cc192e90 2021-02-10T11:22:16.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.656 7fd4d0a55700 1 --2- 172.21.2.101:0/4073826181 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4b00186c0 0x7fd4b001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:16.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.656 7fd4d0a55700 1 --2- 172.21.2.101:0/4073826181 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4b00186c0 0x7fd4b001abc0 crc :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:16.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.686 7fd4d2c39700 1 -- 172.21.2.101:0/4073826181 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4cc19d290 con 0x7fd4cc192e90 2021-02-10T11:22:16.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.688 7fd4c97fa700 1 -- 172.21.2.101:0/4073826181 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd4bc0009b0 con 0x7fd4cc192e90 2021-02-10T11:22:16.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.812 7fd4d2c39700 1 -- 172.21.2.101:0/4073826181 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839285", "target": ["mgr", ""]}) v1 -- 0x7fd4cc08f1f0 con 0x7fd4b00186c0 2021-02-10T11:22:16.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.839 7fd4c97fa700 1 -- 172.21.2.101:0/4073826181 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fd4b40009b0 con 0x7fd4b00186c0 2021-02-10T11:22:16.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.845 7fd4d2c39700 1 -- 172.21.2.101:0/4073826181 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4b00186c0 msgr2=0x7fd4b001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:16.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.845 7fd4d2c39700 1 --2- 172.21.2.101:0/4073826181 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4b00186c0 0x7fd4b001abc0 crc :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:16.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.845 7fd4d2c39700 1 -- 172.21.2.101:0/4073826181 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4cc192e90 msgr2=0x7fd4cc08ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:16.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.845 7fd4d2c39700 1 --2- 172.21.2.101:0/4073826181 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4cc192e90 0x7fd4cc08ca20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7fd4bc001c60 tx=0x7fd4bc00a8d0).stop 2021-02-10T11:22:16.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.846 7fd4d2c39700 1 -- 172.21.2.101:0/4073826181 shutdown_connections 2021-02-10T11:22:16.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.846 7fd4d2c39700 1 --2- 172.21.2.101:0/4073826181 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4cc193290 0x7fd4cc08d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:16.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.846 7fd4d2c39700 1 --2- 172.21.2.101:0/4073826181 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4cc192e90 0x7fd4cc08ca20 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:16.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.846 7fd4d2c39700 1 --2- 172.21.2.101:0/4073826181 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4b00186c0 0x7fd4b001abc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:16.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.846 7fd4d2c39700 1 --2- 172.21.2.101:0/4073826181 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4cc08edf0 0x7fd4cc192870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:16.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.846 7fd4d2c39700 1 -- 172.21.2.101:0/4073826181 >> 172.21.2.101:0/4073826181 conn(0x7fd4cc0bf4f0 msgr2=0x7fd4cc0be250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:16.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.846 7fd4d2c39700 1 -- 172.21.2.101:0/4073826181 shutdown_connections 2021-02-10T11:22:16.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:16.846 7fd4d2c39700 1 -- 172.21.2.101:0/4073826181 wait complete. 2021-02-10T11:22:16.861 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839285 2021-02-10T11:22:17.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.268 7f6b3e2c1700 1 Processor -- start 2021-02-10T11:22:17.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.268 7f6b3e2c1700 1 -- start start 2021-02-10T11:22:17.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.269 7f6b3e2c1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6b381b0470 0x7f6b381b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:17.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.269 7f6b3e2c1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6b380ef610 0x7f6b380efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:17.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.269 7f6b3e2c1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b380f0030 0x7f6b380f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:17.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.269 7f6b3e2c1700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b380e5ea0 con 0x7f6b381b0470 2021-02-10T11:22:17.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.269 7f6b3e2c1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b380f0b50 con 0x7f6b380ef610 2021-02-10T11:22:17.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.269 7f6b3e2c1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b380e52e0 con 0x7f6b380f0030 2021-02-10T11:22:17.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.269 7f6b37fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6b381b0470 0x7f6b381b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:17.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.269 7f6b37fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6b381b0470 0x7f6b381b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:46994/0 (socket says 172.21.2.101:46994) 2021-02-10T11:22:17.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.269 7f6b37fff700 1 -- 172.21.2.101:0/3746930882 learned_addr learned my addr 172.21.2.101:0/3746930882 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:22:17.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.269 7f6b37fff700 1 -- 172.21.2.101:0/3746930882 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b380f0030 msgr2=0x7f6b380f0430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:22:17.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.269 7f6b37fff700 1 --2- 172.21.2.101:0/3746930882 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b380f0030 0x7f6b380f0430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:17.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.269 7f6b37fff700 1 -- 172.21.2.101:0/3746930882 >> v2:172.21.2.108:3300/0 conn(0x7f6b380ef610 msgr2=0x7f6b380efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:22:17.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.269 7f6b37fff700 1 --2- 172.21.2.101:0/3746930882 >> v2:172.21.2.108:3300/0 conn(0x7f6b380ef610 0x7f6b380efa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:17.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.269 7f6b37fff700 1 -- 172.21.2.101:0/3746930882 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b380f10d0 con 0x7f6b381b0470 2021-02-10T11:22:17.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.270 7f6b37fff700 1 --2- 172.21.2.101:0/3746930882 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b381b0470 0x7f6b381b0870 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f6b28006700 tx=0x7f6b28009b80).ready entity=mon.0 client_cookie=867084a370685735 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:17.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.270 7f6b36ffd700 1 -- 172.21.2.101:0/3746930882 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6b2800c0c0 con 0x7f6b381b0470 2021-02-10T11:22:17.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.270 7f6b36ffd700 1 -- 172.21.2.101:0/3746930882 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2800c340 con 0x7f6b381b0470 2021-02-10T11:22:17.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.270 7f6b36ffd700 1 -- 172.21.2.101:0/3746930882 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6b2800c860 con 0x7f6b381b0470 2021-02-10T11:22:17.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.270 7f6b3e2c1700 1 -- 172.21.2.101:0/3746930882 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b381b0470 msgr2=0x7f6b381b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:17.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.270 7f6b3e2c1700 1 --2- 172.21.2.101:0/3746930882 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b381b0470 0x7f6b381b0870 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f6b28006700 tx=0x7f6b28009b80).stop 2021-02-10T11:22:17.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.271 7f6b3e2c1700 1 -- 172.21.2.101:0/3746930882 shutdown_connections 2021-02-10T11:22:17.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.271 7f6b3e2c1700 1 --2- 172.21.2.101:0/3746930882 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b380f0030 0x7f6b380f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:17.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.271 7f6b3e2c1700 1 --2- 172.21.2.101:0/3746930882 >> v2:172.21.2.108:3300/0 conn(0x7f6b380ef610 0x7f6b380efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:17.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.271 7f6b3e2c1700 1 --2- 172.21.2.101:0/3746930882 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b381b0470 0x7f6b381b0870 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:17.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.271 7f6b3e2c1700 1 -- 172.21.2.101:0/3746930882 >> 172.21.2.101:0/3746930882 conn(0x7f6b380ce3e0 msgr2=0x7f6b380e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:17.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.271 7f6b3e2c1700 1 -- 172.21.2.101:0/3746930882 shutdown_connections 2021-02-10T11:22:17.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.271 7f6b3e2c1700 1 -- 172.21.2.101:0/3746930882 wait complete. 2021-02-10T11:22:17.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.285 7f6b3e2c1700 1 Processor -- start 2021-02-10T11:22:17.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.286 7f6b3e2c1700 1 -- start start 2021-02-10T11:22:17.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.286 7f6b3e2c1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b380bf910 0x7f6b380bfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:17.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.286 7f6b3e2c1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b380c0330 0x7f6b380c0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:17.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.286 7f6b3e2c1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b380c13d0 0x7f6b380c17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:17.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.286 7f6b3e2c1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b38109930 con 0x7f6b380c13d0 2021-02-10T11:22:17.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.286 7f6b3e2c1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b38109b50 con 0x7f6b380bf910 2021-02-10T11:22:17.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.286 7f6b3e2c1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b380c1df0 con 0x7f6b380c0330 2021-02-10T11:22:17.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.286 7f6b3c8de700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b380c13d0 0x7f6b380c17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:17.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.286 7f6b3c8de700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b380c13d0 0x7f6b380c17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47002/0 (socket says 172.21.2.101:47002) 2021-02-10T11:22:17.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.286 7f6b3c8de700 1 -- 172.21.2.101:0/2532271708 learned_addr learned my addr 172.21.2.101:0/2532271708 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:22:17.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.287 7f6b3c8de700 1 -- 172.21.2.101:0/2532271708 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b380c0330 msgr2=0x7f6b380c0730 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:22:17.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.287 7f6b3c8de700 1 --2- 172.21.2.101:0/2532271708 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b380c0330 0x7f6b380c0730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:17.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.287 7f6b3c8de700 1 -- 172.21.2.101:0/2532271708 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b380bf910 msgr2=0x7f6b380bfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:17.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.287 7f6b3c8de700 1 --2- 172.21.2.101:0/2532271708 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b380bf910 0x7f6b380bfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:17.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.287 7f6b3c8de700 1 -- 172.21.2.101:0/2532271708 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b380c1fd0 con 0x7f6b380c13d0 2021-02-10T11:22:17.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.287 7f6b3c8de700 1 --2- 172.21.2.101:0/2532271708 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b380c13d0 0x7f6b380c17d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f6b30009960 tx=0x7f6b3000bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:17.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.287 7f6b34ff9700 1 -- 172.21.2.101:0/2532271708 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6b30000d10 con 0x7f6b380c13d0 2021-02-10T11:22:17.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.287 7f6b34ff9700 1 -- 172.21.2.101:0/2532271708 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b30003030 con 0x7f6b380c13d0 2021-02-10T11:22:17.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.287 7f6b34ff9700 1 -- 172.21.2.101:0/2532271708 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6b30001590 con 0x7f6b380c13d0 2021-02-10T11:22:17.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.287 7f6b3e2c1700 1 -- 172.21.2.101:0/2532271708 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b380c1fd0 con 0x7f6b380c13d0 2021-02-10T11:22:17.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.287 7f6b3e2c1700 1 -- 172.21.2.101:0/2532271708 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b380c2560 con 0x7f6b380c13d0 2021-02-10T11:22:17.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.288 7f6b34ff9700 1 -- 172.21.2.101:0/2532271708 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6b30001310 con 0x7f6b380c13d0 2021-02-10T11:22:17.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.288 7f6b34ff9700 1 --2- 172.21.2.101:0/2532271708 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b140186c0 0x7f6b1401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:17.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.288 7f6b34ff9700 1 -- 172.21.2.101:0/2532271708 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6b300328f0 con 0x7f6b380c13d0 2021-02-10T11:22:17.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.288 7f6b37fff700 1 --2- 172.21.2.101:0/2532271708 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b140186c0 0x7f6b1401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:17.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.288 7f6b37fff700 1 --2- 172.21.2.101:0/2532271708 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b140186c0 0x7f6b1401abc0 crc :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:17.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.319 7f6b3e2c1700 1 -- 172.21.2.101:0/2532271708 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b380f0b50 con 0x7f6b380c13d0 2021-02-10T11:22:17.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.321 7f6b34ff9700 1 -- 172.21.2.101:0/2532271708 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6b30000f40 con 0x7f6b380c13d0 2021-02-10T11:22:17.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.448 7f6b3e2c1700 1 -- 172.21.2.101:0/2532271708 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839285", "target": ["mgr", ""]}) v1 -- 0x7f6b380c2560 con 0x7f6b140186c0 2021-02-10T11:22:17.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.451 7f6b34ff9700 1 -- 172.21.2.101:0/2532271708 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f6b28000940 con 0x7f6b140186c0 2021-02-10T11:22:17.462 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839285/1f687477-8910-4bae-ac8e-cb22089cb99c 2021-02-10T11:22:17.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.456 7f6b3e2c1700 1 -- 172.21.2.101:0/2532271708 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b140186c0 msgr2=0x7f6b1401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:17.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.456 7f6b3e2c1700 1 --2- 172.21.2.101:0/2532271708 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b140186c0 0x7f6b1401abc0 crc :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:17.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.456 7f6b3e2c1700 1 -- 172.21.2.101:0/2532271708 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b380c13d0 msgr2=0x7f6b380c17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:17.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.456 7f6b3e2c1700 1 --2- 172.21.2.101:0/2532271708 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b380c13d0 0x7f6b380c17d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f6b30009960 tx=0x7f6b3000bcf0).stop 2021-02-10T11:22:17.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.456 7f6b3e2c1700 1 -- 172.21.2.101:0/2532271708 shutdown_connections 2021-02-10T11:22:17.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.456 7f6b3e2c1700 1 --2- 172.21.2.101:0/2532271708 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b140186c0 0x7f6b1401abc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:17.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.456 7f6b3e2c1700 1 --2- 172.21.2.101:0/2532271708 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b380c0330 0x7f6b380c0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:17.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.456 7f6b3e2c1700 1 --2- 172.21.2.101:0/2532271708 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b380c13d0 0x7f6b380c17d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:17.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.456 7f6b3e2c1700 1 --2- 172.21.2.101:0/2532271708 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b380bf910 0x7f6b380bfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:17.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.456 7f6b3e2c1700 1 -- 172.21.2.101:0/2532271708 >> 172.21.2.101:0/2532271708 conn(0x7f6b380bf4e0 msgr2=0x7f6b38191190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:17.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.457 7f6b3e2c1700 1 -- 172.21.2.101:0/2532271708 shutdown_connections 2021-02-10T11:22:17.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:17.457 7f6b3e2c1700 1 -- 172.21.2.101:0/2532271708 wait complete. 2021-02-10T11:22:17.472 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839285 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839285/1f687477-8910-4bae-ac8e-cb22089cb99c 2021-02-10T11:22:17.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:17.473 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:17.473 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:17.808 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:17.808 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:17.809 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839285/1f687477-8910-4bae-ac8e-cb22089cb99c/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:18.188 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:18.188 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:18.189 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0229648 s, 45.7 MB/s 2021-02-10T11:22:18.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:18.190 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:18.190 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:18.524 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:18.525 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:18.525 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839285/1f687477-8910-4bae-ac8e-cb22089cb99c/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:18.895 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:18.895 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:18.895 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237878 s, 44.1 MB/s 2021-02-10T11:22:18.896 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:18.896 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:18.896 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:19.231 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:19.231 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:19.232 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839285/1f687477-8910-4bae-ac8e-cb22089cb99c/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:19.609 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:19.609 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:19.609 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251302 s, 41.7 MB/s 2021-02-10T11:22:19.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:19.610 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:19.610 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:19.900 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:22:19.903 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:22:19.946 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:19.946 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:19.946 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839285/1f687477-8910-4bae-ac8e-cb22089cb99c/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:20.276 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:20.276 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:20.276 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253573 s, 41.4 MB/s 2021-02-10T11:22:20.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:20.278 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:20.278 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:20.612 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:22.381 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:22.382 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839285/1f687477-8910-4bae-ac8e-cb22089cb99c/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:22.759 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:22.759 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:22.760 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024577 s, 42.7 MB/s 2021-02-10T11:22:22.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:22.761 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:22.761 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:23.097 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:23.098 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:23.098 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839285/1f687477-8910-4bae-ac8e-cb22089cb99c/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:23.473 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:23.473 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:23.473 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237422 s, 44.2 MB/s 2021-02-10T11:22:23.474 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:23.474 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:23.474 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:23.810 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:23.810 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:23.810 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839285/1f687477-8910-4bae-ac8e-cb22089cb99c/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:24.186 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:24.186 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:24.187 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0254741 s, 41.2 MB/s 2021-02-10T11:22:24.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:24.187 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:24.187 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:24.530 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:24.530 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:24.531 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839285/1f687477-8910-4bae-ac8e-cb22089cb99c/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:24.951 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:24.951 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:24.951 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0223595 s, 46.9 MB/s 2021-02-10T11:22:24.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:24.952 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:24.952 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:25.288 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:25.289 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:25.289 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839285/1f687477-8910-4bae-ac8e-cb22089cb99c/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:25.679 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:25.679 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:25.680 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.02485 s, 42.2 MB/s 2021-02-10T11:22:25.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:25.681 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:25.681 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:26.016 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:26.016 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:26.017 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839285/1f687477-8910-4bae-ac8e-cb22089cb99c/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:26.393 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:26.394 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:26.394 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252299 s, 41.6 MB/s 2021-02-10T11:22:26.395 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839286 2021-02-10T11:22:26.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.801 7f058a3fe700 1 Processor -- start 2021-02-10T11:22:26.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.801 7f058a3fe700 1 -- start start 2021-02-10T11:22:26.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.801 7f058a3fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f05841b0470 0x7f05841b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:26.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.801 7f058a3fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05840ef610 0x7f05840efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:26.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.801 7f058a3fe700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05840f0b50 con 0x7f05840f0030 2021-02-10T11:22:26.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.801 7f058a3fe700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05840e52e0 con 0x7f05841b0470 2021-02-10T11:22:26.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.801 7f058a3fe700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05840e54d0 con 0x7f05840ef610 2021-02-10T11:22:26.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.802 7f0588a1b700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f05840f0030 0x7f05840f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47500/0 (socket says 172.21.2.101:47500) 2021-02-10T11:22:26.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.802 7f0588a1b700 1 -- 172.21.2.101:0/3309603132 learned_addr learned my addr 172.21.2.101:0/3309603132 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:22:26.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.802 7f0582ffd700 1 -- 172.21.2.101:0/3309603132 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819748565 0 0) 0x7f057c001500 con 0x7f05840f0030 2021-02-10T11:22:26.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.802 7f0582ffd700 1 -- 172.21.2.101:0/3309603132 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f056c002fe0 con 0x7f05840f0030 2021-02-10T11:22:26.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.802 7f05837fe700 1 --2- 172.21.2.101:0/3309603132 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05840ef610 0x7f05840efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:26.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.802 7f0582ffd700 1 -- 172.21.2.101:0/3309603132 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 698105109 0 0) 0x7f057c001500 con 0x7f05840f0030 2021-02-10T11:22:26.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.802 7f0583fff700 1 --2- 172.21.2.101:0/3309603132 >> v2:172.21.2.108:3300/0 conn(0x7f05841b0470 0x7f05841b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:26.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.802 7f0582ffd700 1 -- 172.21.2.101:0/3309603132 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05840ef610 msgr2=0x7f05840efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:22:26.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.802 7f0582ffd700 1 --2- 172.21.2.101:0/3309603132 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05840ef610 0x7f05840efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:26.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.802 7f0582ffd700 1 -- 172.21.2.101:0/3309603132 >> v2:172.21.2.108:3300/0 conn(0x7f05841b0470 msgr2=0x7f05841b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:22:26.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.802 7f0582ffd700 1 --2- 172.21.2.101:0/3309603132 >> v2:172.21.2.108:3300/0 conn(0x7f05841b0470 0x7f05841b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:26.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.802 7f0582ffd700 1 -- 172.21.2.101:0/3309603132 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05840f0430 con 0x7f05840f0030 2021-02-10T11:22:26.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.802 7f0582ffd700 1 -- 172.21.2.101:0/3309603132 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f057c001a00 con 0x7f05840f0030 2021-02-10T11:22:26.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.802 7f05837fe700 1 --2- 172.21.2.101:0/3309603132 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05840ef610 0x7f05840efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:22:26.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.802 7f0582ffd700 1 -- 172.21.2.101:0/3309603132 >> v1:172.21.2.101:6789/0 conn(0x7f05840f0030 legacy=0x7f05840f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:26.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.802 7f0582ffd700 1 --2- 172.21.2.101:0/3309603132 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f056c003a60 0x7f056c005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:26.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.802 7f0582ffd700 1 --2- 172.21.2.101:0/3309603132 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f056c006790 0x7f056c008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:26.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.802 7f0582ffd700 1 --2- 172.21.2.101:0/3309603132 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f056c009590 0x7f056c00ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:26.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.802 7f0582ffd700 1 -- 172.21.2.101:0/3309603132 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c00c2d0 con 0x7f056c003a60 2021-02-10T11:22:26.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.802 7f0582ffd700 1 -- 172.21.2.101:0/3309603132 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c00c4f0 con 0x7f056c006790 2021-02-10T11:22:26.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.802 7f0582ffd700 1 -- 172.21.2.101:0/3309603132 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056c050720 con 0x7f056c009590 2021-02-10T11:22:26.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.802 7f0583fff700 1 --2- 172.21.2.101:0/3309603132 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f056c003a60 0x7f056c005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:26.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.803 7f0583fff700 1 -- 172.21.2.101:0/3309603132 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f056c009590 msgr2=0x7f056c00ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:22:26.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.803 7f0583fff700 1 --2- 172.21.2.101:0/3309603132 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f056c009590 0x7f056c00ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:26.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.803 7f0583fff700 1 -- 172.21.2.101:0/3309603132 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f056c006790 msgr2=0x7f056c008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:22:26.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.803 7f0583fff700 1 --2- 172.21.2.101:0/3309603132 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f056c006790 0x7f056c008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:26.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.803 7f0583fff700 1 -- 172.21.2.101:0/3309603132 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c050950 con 0x7f056c003a60 2021-02-10T11:22:26.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.803 7f0583fff700 1 --2- 172.21.2.101:0/3309603132 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f056c003a60 0x7f056c005ef0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f05740098e0 tx=0x7f0574009c80).ready entity=mon.0 client_cookie=ab435266f4a343a3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:26.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.803 7f0582ffd700 1 -- 172.21.2.101:0/3309603132 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f05740009b0 con 0x7f056c003a60 2021-02-10T11:22:26.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.803 7f0582ffd700 1 -- 172.21.2.101:0/3309603132 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0574000c90 con 0x7f056c003a60 2021-02-10T11:22:26.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.803 7f0582ffd700 1 -- 172.21.2.101:0/3309603132 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f05740022b0 con 0x7f056c003a60 2021-02-10T11:22:26.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.803 7f058a3fe700 1 -- 172.21.2.101:0/3309603132 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f056c003a60 msgr2=0x7f056c005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:26.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.803 7f058a3fe700 1 --2- 172.21.2.101:0/3309603132 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f056c003a60 0x7f056c005ef0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f05740098e0 tx=0x7f0574009c80).stop 2021-02-10T11:22:26.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.803 7f0588a1b700 1 -- 172.21.2.101:0/3309603132 reap_dead start 2021-02-10T11:22:26.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.804 7f058a3fe700 1 -- 172.21.2.101:0/3309603132 shutdown_connections 2021-02-10T11:22:26.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.804 7f058a3fe700 1 -- 172.21.2.101:0/3309603132 >> 172.21.2.101:0/3309603132 conn(0x7f05840ce3e0 msgr2=0x7f05840e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:26.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.804 7f058a3fe700 1 -- 172.21.2.101:0/3309603132 shutdown_connections 2021-02-10T11:22:26.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.804 7f058a3fe700 1 -- 172.21.2.101:0/3309603132 wait complete. 2021-02-10T11:22:26.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.819 7f058a3fe700 1 Processor -- start 2021-02-10T11:22:26.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.819 7f058a3fe700 1 -- start start 2021-02-10T11:22:26.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.819 7f058a3fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05840c30c0 0x7f05841a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:26.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.819 7f058a3fe700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f05840c34c0 0x7f05841a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:26.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.819 7f058a3fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05840a93e0 0x7f05840a97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:26.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.819 7f058a3fe700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05841a8090 con 0x7f05840a93e0 2021-02-10T11:22:26.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.819 7f058a3fe700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05841a82b0 con 0x7f05840c34c0 2021-02-10T11:22:26.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.819 7f058a3fe700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05840a9e00 con 0x7f05840c30c0 2021-02-10T11:22:26.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.819 7f0588a1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05840a93e0 0x7f05840a97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:26.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.819 7f0588a1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05840a93e0 0x7f05840a97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47020/0 (socket says 172.21.2.101:47020) 2021-02-10T11:22:26.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.819 7f0588a1b700 1 -- 172.21.2.101:0/1343755933 learned_addr learned my addr 172.21.2.101:0/1343755933 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:22:26.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.820 7f0588a1b700 1 -- 172.21.2.101:0/1343755933 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05840c30c0 msgr2=0x7f05841a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:26.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.820 7f0588a1b700 1 --2- 172.21.2.101:0/1343755933 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05840c30c0 0x7f05841a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:26.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.820 7f0588a1b700 1 -- 172.21.2.101:0/1343755933 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f05840c34c0 msgr2=0x7f05841a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:26.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.820 7f0588a1b700 1 --2- 172.21.2.101:0/1343755933 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f05840c34c0 0x7f05841a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:26.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.820 7f0588a1b700 1 -- 172.21.2.101:0/1343755933 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05840aa020 con 0x7f05840a93e0 2021-02-10T11:22:26.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.820 7f0588a1b700 1 --2- 172.21.2.101:0/1343755933 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05840a93e0 0x7f05840a97e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f057c009980 tx=0x7f057c00bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:26.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.820 7f0580ff9700 1 -- 172.21.2.101:0/1343755933 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f057c0060c0 con 0x7f05840a93e0 2021-02-10T11:22:26.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.820 7f0580ff9700 1 -- 172.21.2.101:0/1343755933 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057c006340 con 0x7f05840a93e0 2021-02-10T11:22:26.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.820 7f058a3fe700 1 -- 172.21.2.101:0/1343755933 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05840aa020 con 0x7f05840a93e0 2021-02-10T11:22:26.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.820 7f058a3fe700 1 -- 172.21.2.101:0/1343755933 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05840aa5b0 con 0x7f05840a93e0 2021-02-10T11:22:26.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.820 7f0580ff9700 1 -- 172.21.2.101:0/1343755933 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f057c004720 con 0x7f05840a93e0 2021-02-10T11:22:26.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.821 7f0580ff9700 1 -- 172.21.2.101:0/1343755933 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f057c001270 con 0x7f05840a93e0 2021-02-10T11:22:26.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.821 7f0580ff9700 1 --2- 172.21.2.101:0/1343755933 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f05640186c0 0x7f056401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:26.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.821 7f0580ff9700 1 -- 172.21.2.101:0/1343755933 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f057c0328f0 con 0x7f05840a93e0 2021-02-10T11:22:26.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.821 7f0583fff700 1 --2- 172.21.2.101:0/1343755933 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f05640186c0 0x7f056401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:26.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.822 7f0583fff700 1 --2- 172.21.2.101:0/1343755933 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f05640186c0 0x7f056401abc0 crc :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:26.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.851 7f058a3fe700 1 -- 172.21.2.101:0/1343755933 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f055c002fd0 con 0x7f05840a93e0 2021-02-10T11:22:26.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.854 7f0580ff9700 1 -- 172.21.2.101:0/1343755933 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f057c02ef50 con 0x7f05840a93e0 2021-02-10T11:22:26.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:26.980 7f058a3fe700 1 -- 172.21.2.101:0/1343755933 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839286", "target": ["mgr", ""]}) v1 -- 0x7f055c002f80 con 0x7f05640186c0 2021-02-10T11:22:27.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.006 7f0580ff9700 1 -- 172.21.2.101:0/1343755933 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f0574009630 con 0x7f05640186c0 2021-02-10T11:22:27.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.013 7f058a3fe700 1 -- 172.21.2.101:0/1343755933 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f05640186c0 msgr2=0x7f056401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:27.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.013 7f058a3fe700 1 --2- 172.21.2.101:0/1343755933 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f05640186c0 0x7f056401abc0 crc :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:27.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.013 7f058a3fe700 1 -- 172.21.2.101:0/1343755933 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05840a93e0 msgr2=0x7f05840a97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:27.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.013 7f058a3fe700 1 --2- 172.21.2.101:0/1343755933 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05840a93e0 0x7f05840a97e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f057c009980 tx=0x7f057c00bcd0).stop 2021-02-10T11:22:27.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.013 7f058a3fe700 1 -- 172.21.2.101:0/1343755933 shutdown_connections 2021-02-10T11:22:27.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.013 7f058a3fe700 1 --2- 172.21.2.101:0/1343755933 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05840a93e0 0x7f05840a97e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:27.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.013 7f058a3fe700 1 --2- 172.21.2.101:0/1343755933 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f05840c34c0 0x7f05841a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:27.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.013 7f058a3fe700 1 --2- 172.21.2.101:0/1343755933 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f05640186c0 0x7f056401abc0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:27.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.013 7f058a3fe700 1 --2- 172.21.2.101:0/1343755933 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05840c30c0 0x7f05841a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:27.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.014 7f058a3fe700 1 -- 172.21.2.101:0/1343755933 >> 172.21.2.101:0/1343755933 conn(0x7f058409fc10 msgr2=0x7f05840b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:27.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.014 7f058a3fe700 1 -- 172.21.2.101:0/1343755933 shutdown_connections 2021-02-10T11:22:27.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.014 7f058a3fe700 1 -- 172.21.2.101:0/1343755933 wait complete. 2021-02-10T11:22:27.029 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839286 2021-02-10T11:22:27.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.435 7f95f3281700 1 Processor -- start 2021-02-10T11:22:27.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.435 7f95f3281700 1 -- start start 2021-02-10T11:22:27.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.436 7f95f3281700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f95ec1b0470 0x7f95ec1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:27.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.436 7f95f3281700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f95ec0ef610 0x7f95ec0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:27.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.436 7f95f3281700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ec0f0b50 con 0x7f95ec0f0030 2021-02-10T11:22:27.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.436 7f95f3281700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ec0e52e0 con 0x7f95ec1b0470 2021-02-10T11:22:27.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.436 7f95f3281700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ec0e5490 con 0x7f95ec0ef610 2021-02-10T11:22:27.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.436 7f95f109d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f95ec1b0470 0x7f95ec1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:27.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.436 7f95f109d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f95ec1b0470 0x7f95ec1b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47024/0 (socket says 172.21.2.101:47024) 2021-02-10T11:22:27.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.436 7f95f109d700 1 -- 172.21.2.101:0/705760453 learned_addr learned my addr 172.21.2.101:0/705760453 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:22:27.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.436 7f95f109d700 1 -- 172.21.2.101:0/705760453 >> v2:172.21.2.108:3300/0 conn(0x7f95ec0ef610 msgr2=0x7f95ec0efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:22:27.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.436 7f95f109d700 1 --2- 172.21.2.101:0/705760453 >> v2:172.21.2.108:3300/0 conn(0x7f95ec0ef610 0x7f95ec0efa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:27.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.436 7f95f109d700 1 -- 172.21.2.101:0/705760453 >> v1:172.21.2.108:6789/0 conn(0x7f95ec0f0030 legacy=0x7f95ec0f10d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:22:27.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.436 7f95f109d700 1 -- 172.21.2.101:0/705760453 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ec0f0430 con 0x7f95ec1b0470 2021-02-10T11:22:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.436 7f95f109d700 1 --2- 172.21.2.101:0/705760453 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95ec1b0470 0x7f95ec1b0870 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f95e0006700 tx=0x7f95e0009b80).ready entity=mon.0 client_cookie=3048d7ab3b6b8d41 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.437 7f95dffff700 1 -- 172.21.2.101:0/705760453 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f95e0000c90 con 0x7f95ec1b0470 2021-02-10T11:22:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.437 7f95dffff700 1 -- 172.21.2.101:0/705760453 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e0000c90 con 0x7f95ec1b0470 2021-02-10T11:22:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.437 7f95dffff700 1 -- 172.21.2.101:0/705760453 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f95e000c0c0 con 0x7f95ec1b0470 2021-02-10T11:22:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.437 7f95f3281700 1 -- 172.21.2.101:0/705760453 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95ec1b0470 msgr2=0x7f95ec1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.437 7f95f3281700 1 --2- 172.21.2.101:0/705760453 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95ec1b0470 0x7f95ec1b0870 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f95e0006700 tx=0x7f95e0009b80).stop 2021-02-10T11:22:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.437 7f95f3281700 1 -- 172.21.2.101:0/705760453 shutdown_connections 2021-02-10T11:22:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.437 7f95f3281700 1 --2- 172.21.2.101:0/705760453 >> v2:172.21.2.108:3300/0 conn(0x7f95ec0ef610 0x7f95ec0efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.437 7f95f3281700 1 --2- 172.21.2.101:0/705760453 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95ec1b0470 0x7f95ec1b0870 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.437 7f95f3281700 1 -- 172.21.2.101:0/705760453 >> 172.21.2.101:0/705760453 conn(0x7f95ec0ce3e0 msgr2=0x7f95ec0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.437 7f95f3281700 1 -- 172.21.2.101:0/705760453 shutdown_connections 2021-02-10T11:22:27.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.438 7f95f3281700 1 -- 172.21.2.101:0/705760453 wait complete. 2021-02-10T11:22:27.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.452 7f95f3281700 1 Processor -- start 2021-02-10T11:22:27.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.452 7f95f3281700 1 -- start start 2021-02-10T11:22:27.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.452 7f95f3281700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95ec0acce0 0x7f95ec0a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:27.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.452 7f95f3281700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f95ec0a4010 0x7f95ec0a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:27.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.452 7f95f3281700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95ec0eec20 0x7f95ec0ef020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:27.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.452 7f95f3281700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec0a5330 con 0x7f95ec0acce0 2021-02-10T11:22:27.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.452 7f95f3281700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec0ad0e0 con 0x7f95ec0a4010 2021-02-10T11:22:27.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.452 7f95f3281700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec0ef640 con 0x7f95ec0eec20 2021-02-10T11:22:27.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.452 7f95f109d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95ec0acce0 0x7f95ec0a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:27.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.452 7f95f109d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95ec0acce0 0x7f95ec0a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47026/0 (socket says 172.21.2.101:47026) 2021-02-10T11:22:27.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.452 7f95f109d700 1 -- 172.21.2.101:0/2035697839 learned_addr learned my addr 172.21.2.101:0/2035697839 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:22:27.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.453 7f95f189e700 1 --2- 172.21.2.101:0/2035697839 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95ec0eec20 0x7f95ec0ef020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:27.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.453 7f95f089c700 1 --2- 172.21.2.101:0/2035697839 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f95ec0a4010 0x7f95ec0a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:27.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.453 7f95f109d700 1 -- 172.21.2.101:0/2035697839 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95ec0eec20 msgr2=0x7f95ec0ef020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:27.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.453 7f95f109d700 1 --2- 172.21.2.101:0/2035697839 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95ec0eec20 0x7f95ec0ef020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:27.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.453 7f95f109d700 1 -- 172.21.2.101:0/2035697839 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f95ec0a4010 msgr2=0x7f95ec0a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:27.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.453 7f95f109d700 1 --2- 172.21.2.101:0/2035697839 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f95ec0a4010 0x7f95ec0a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:27.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.453 7f95f109d700 1 -- 172.21.2.101:0/2035697839 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ec0ef860 con 0x7f95ec0acce0 2021-02-10T11:22:27.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.453 7f95f189e700 1 --2- 172.21.2.101:0/2035697839 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95ec0eec20 0x7f95ec0ef020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:22:27.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.453 7f95f109d700 1 --2- 172.21.2.101:0/2035697839 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95ec0acce0 0x7f95ec0a4b00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f95e0006f00 tx=0x7f95e000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:27.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.453 7f95ddffb700 1 -- 172.21.2.101:0/2035697839 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f95e0006b10 con 0x7f95ec0acce0 2021-02-10T11:22:27.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.453 7f95ddffb700 1 -- 172.21.2.101:0/2035697839 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e000be10 con 0x7f95ec0acce0 2021-02-10T11:22:27.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.454 7f95ddffb700 1 -- 172.21.2.101:0/2035697839 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f95e00052e0 con 0x7f95ec0acce0 2021-02-10T11:22:27.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.454 7f95f3281700 1 -- 172.21.2.101:0/2035697839 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ec0ef860 con 0x7f95ec0acce0 2021-02-10T11:22:27.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.454 7f95f3281700 1 -- 172.21.2.101:0/2035697839 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ec0b6750 con 0x7f95ec0acce0 2021-02-10T11:22:27.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.454 7f95ddffb700 1 -- 172.21.2.101:0/2035697839 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f95e0006030 con 0x7f95ec0acce0 2021-02-10T11:22:27.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.454 7f95ddffb700 1 --2- 172.21.2.101:0/2035697839 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95cc0186c0 0x7f95cc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:27.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.454 7f95ddffb700 1 -- 172.21.2.101:0/2035697839 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f95e00323b0 con 0x7f95ec0acce0 2021-02-10T11:22:27.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.455 7f95f089c700 1 --2- 172.21.2.101:0/2035697839 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95cc0186c0 0x7f95cc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:27.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.455 7f95f089c700 1 --2- 172.21.2.101:0/2035697839 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95cc0186c0 0x7f95cc01abc0 crc :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:27.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.485 7f95f3281700 1 -- 172.21.2.101:0/2035697839 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ec0edd10 con 0x7f95ec0acce0 2021-02-10T11:22:27.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.488 7f95ddffb700 1 -- 172.21.2.101:0/2035697839 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f95e0000940 con 0x7f95ec0acce0 2021-02-10T11:22:27.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.616 7f95f3281700 1 -- 172.21.2.101:0/2035697839 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839286", "target": ["mgr", ""]}) v1 -- 0x7f95ec0b6750 con 0x7f95cc0186c0 2021-02-10T11:22:27.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.619 7f95ddffb700 1 -- 172.21.2.101:0/2035697839 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f95e8006e30 con 0x7f95cc0186c0 2021-02-10T11:22:27.627 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839286/73e5ca2c-7bc7-4769-a6de-514325889d96 2021-02-10T11:22:27.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.620 7f95f3281700 1 -- 172.21.2.101:0/2035697839 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95cc0186c0 msgr2=0x7f95cc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:27.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.620 7f95f3281700 1 --2- 172.21.2.101:0/2035697839 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95cc0186c0 0x7f95cc01abc0 crc :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:27.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.620 7f95f3281700 1 -- 172.21.2.101:0/2035697839 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95ec0acce0 msgr2=0x7f95ec0a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:27.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.620 7f95f3281700 1 --2- 172.21.2.101:0/2035697839 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95ec0acce0 0x7f95ec0a4b00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f95e0006f00 tx=0x7f95e000b990).stop 2021-02-10T11:22:27.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.620 7f95f3281700 1 -- 172.21.2.101:0/2035697839 shutdown_connections 2021-02-10T11:22:27.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.620 7f95f3281700 1 --2- 172.21.2.101:0/2035697839 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95cc0186c0 0x7f95cc01abc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:27.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.620 7f95f3281700 1 --2- 172.21.2.101:0/2035697839 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95ec0eec20 0x7f95ec0ef020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:27.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.620 7f95f3281700 1 --2- 172.21.2.101:0/2035697839 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f95ec0a4010 0x7f95ec0a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:27.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.620 7f95f3281700 1 --2- 172.21.2.101:0/2035697839 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95ec0acce0 0x7f95ec0a4b00 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:27.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.621 7f95f3281700 1 -- 172.21.2.101:0/2035697839 >> 172.21.2.101:0/2035697839 conn(0x7f95ec0c40e0 msgr2=0x7f95ec0f22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:27.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.621 7f95f3281700 1 -- 172.21.2.101:0/2035697839 shutdown_connections 2021-02-10T11:22:27.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:27.621 7f95f3281700 1 -- 172.21.2.101:0/2035697839 wait complete. 2021-02-10T11:22:27.636 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839286 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839286/73e5ca2c-7bc7-4769-a6de-514325889d96 2021-02-10T11:22:27.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:27.637 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:27.637 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:27.973 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:27.973 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:27.973 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839286/73e5ca2c-7bc7-4769-a6de-514325889d96/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:28.355 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:28.355 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:28.355 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0254713 s, 41.2 MB/s 2021-02-10T11:22:28.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:28.356 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:28.356 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:28.692 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:28.692 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:28.693 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839286/73e5ca2c-7bc7-4769-a6de-514325889d96/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:29.066 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:29.066 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:29.066 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0256552 s, 40.9 MB/s 2021-02-10T11:22:29.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:29.068 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:29.068 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:29.403 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:29.403 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:29.403 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839286/73e5ca2c-7bc7-4769-a6de-514325889d96/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:29.779 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:29.779 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:29.779 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252174 s, 41.6 MB/s 2021-02-10T11:22:29.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:29.780 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:29.780 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:30.116 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:30.117 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:30.117 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839286/73e5ca2c-7bc7-4769-a6de-514325889d96/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:30.492 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:30.492 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:30.492 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0228136 s, 46.0 MB/s 2021-02-10T11:22:30.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:30.493 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:30.493 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:30.827 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:30.827 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:30.828 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839286/73e5ca2c-7bc7-4769-a6de-514325889d96/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:31.202 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:31.203 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:31.203 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236093 s, 44.4 MB/s 2021-02-10T11:22:31.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:31.204 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:31.204 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:31.539 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:31.540 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:31.540 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839286/73e5ca2c-7bc7-4769-a6de-514325889d96/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:31.912 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:31.913 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:31.913 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0225006 s, 46.6 MB/s 2021-02-10T11:22:31.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:31.914 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:31.914 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:32.250 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:32.250 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:32.251 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839286/73e5ca2c-7bc7-4769-a6de-514325889d96/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:32.626 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:32.627 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:32.627 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0232413 s, 45.1 MB/s 2021-02-10T11:22:32.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:32.628 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:32.628 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:32.962 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:32.962 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:32.963 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839286/73e5ca2c-7bc7-4769-a6de-514325889d96/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:33.340 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:33.340 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:33.340 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0259067 s, 40.5 MB/s 2021-02-10T11:22:33.341 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:33.341 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:33.341 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:33.676 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:33.677 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:33.677 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839286/73e5ca2c-7bc7-4769-a6de-514325889d96/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:34.050 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:34.050 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:34.050 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023753 s, 44.1 MB/s 2021-02-10T11:22:34.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:34.051 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:34.051 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:34.387 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:34.387 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:34.387 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839286/73e5ca2c-7bc7-4769-a6de-514325889d96/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:34.761 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:34.761 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:34.762 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0232603 s, 45.1 MB/s 2021-02-10T11:22:34.763 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839287 2021-02-10T11:22:35.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.169 7f681e6af700 1 Processor -- start 2021-02-10T11:22:35.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.169 7f681e6af700 1 -- start start 2021-02-10T11:22:35.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.169 7f681e6af700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f68181b0870 0x7f68180f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:35.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.169 7f681e6af700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68181b0c70 con 0x7f68181b0470 2021-02-10T11:22:35.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.169 7f681e6af700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68180f0b50 con 0x7f68180efe00 2021-02-10T11:22:35.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.169 7f681e6af700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68180e5ea0 con 0x7f68181b0870 2021-02-10T11:22:35.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.169 7f68177fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f68181b0870 0x7f68180f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:35.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.169 7f6817fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f68181b0470 0x7f68180ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47528/0 (socket says 172.21.2.101:47528) 2021-02-10T11:22:35.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.170 7f6817fff700 1 -- 172.21.2.101:0/3720911756 learned_addr learned my addr 172.21.2.101:0/3720911756 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:22:35.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.170 7f68177fe700 1 -- 172.21.2.101:0/3720911756 >> v1:172.21.2.108:6789/0 conn(0x7f68180efe00 legacy=0x7f68180f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:22:35.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.170 7f68177fe700 1 -- 172.21.2.101:0/3720911756 >> v1:172.21.2.101:6789/0 conn(0x7f68181b0470 legacy=0x7f68180ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:35.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.170 7f68177fe700 1 -- 172.21.2.101:0/3720911756 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68180e52e0 con 0x7f68181b0870 2021-02-10T11:22:35.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.170 7f68177fe700 1 --2- 172.21.2.101:0/3720911756 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68181b0870 0x7f68180f20e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f680c006650 tx=0x7f680c009a80).ready entity=mon.0 client_cookie=9ed4647a98d26b05 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:35.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.170 7f6816ffd700 1 -- 172.21.2.101:0/3720911756 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f680c0009b0 con 0x7f68181b0870 2021-02-10T11:22:35.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.170 7f6816ffd700 1 -- 172.21.2.101:0/3720911756 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680c00c0c0 con 0x7f68181b0870 2021-02-10T11:22:35.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.170 7f6816ffd700 1 -- 172.21.2.101:0/3720911756 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f680c00c5e0 con 0x7f68181b0870 2021-02-10T11:22:35.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.170 7f681e6af700 1 -- 172.21.2.101:0/3720911756 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68181b0870 msgr2=0x7f68180f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:35.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.170 7f681e6af700 1 --2- 172.21.2.101:0/3720911756 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68181b0870 0x7f68180f20e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f680c006650 tx=0x7f680c009a80).stop 2021-02-10T11:22:35.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.171 7f681e6af700 1 -- 172.21.2.101:0/3720911756 shutdown_connections 2021-02-10T11:22:35.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.171 7f681e6af700 1 --2- 172.21.2.101:0/3720911756 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68181b0870 0x7f68180f20e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:35.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.171 7f681e6af700 1 -- 172.21.2.101:0/3720911756 >> 172.21.2.101:0/3720911756 conn(0x7f68180ce3e0 msgr2=0x7f68180e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:35.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.171 7f681e6af700 1 -- 172.21.2.101:0/3720911756 shutdown_connections 2021-02-10T11:22:35.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.171 7f681e6af700 1 -- 172.21.2.101:0/3720911756 wait complete. 2021-02-10T11:22:35.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.186 7f681e6af700 1 Processor -- start 2021-02-10T11:22:35.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.186 7f681e6af700 1 -- start start 2021-02-10T11:22:35.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.186 7f681e6af700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68180a3b50 0x7f68181b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:35.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.186 7f681e6af700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68181c25b0 0x7f68181b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:35.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.186 7f681e6af700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68181b4350 0x7f68181b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:35.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.186 7f681e6af700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68181b3190 con 0x7f68181b4350 2021-02-10T11:22:35.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.186 7f681e6af700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6818087750 con 0x7f68180a3b50 2021-02-10T11:22:35.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.186 7f681e6af700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68180a6bf0 con 0x7f68181c25b0 2021-02-10T11:22:35.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.187 7f681cccc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68181b4350 0x7f68181b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:35.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.187 7f681cccc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68181b4350 0x7f68181b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47044/0 (socket says 172.21.2.101:47044) 2021-02-10T11:22:35.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.187 7f681cccc700 1 -- 172.21.2.101:0/1970503145 learned_addr learned my addr 172.21.2.101:0/1970503145 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:22:35.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.187 7f6817fff700 1 --2- 172.21.2.101:0/1970503145 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68180a3b50 0x7f68181b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:35.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.187 7f68177fe700 1 --2- 172.21.2.101:0/1970503145 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68181c25b0 0x7f68181b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:35.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.187 7f681cccc700 1 -- 172.21.2.101:0/1970503145 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68181c25b0 msgr2=0x7f68181b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:35.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.187 7f681cccc700 1 --2- 172.21.2.101:0/1970503145 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68181c25b0 0x7f68181b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:35.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.187 7f681cccc700 1 -- 172.21.2.101:0/1970503145 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68180a3b50 msgr2=0x7f68181b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:35.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.187 7f681cccc700 1 --2- 172.21.2.101:0/1970503145 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68180a3b50 0x7f68181b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:35.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.187 7f681cccc700 1 -- 172.21.2.101:0/1970503145 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68181b1ec0 con 0x7f68181b4350 2021-02-10T11:22:35.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.187 7f6817fff700 1 --2- 172.21.2.101:0/1970503145 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68180a3b50 0x7f68181b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:22:35.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.187 7f681cccc700 1 --2- 172.21.2.101:0/1970503145 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68181b4350 0x7f68181b18a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f681000b8e0 tx=0x7f681000bc80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:35.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.188 7f6814ff9700 1 -- 172.21.2.101:0/1970503145 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6810000f90 con 0x7f68181b4350 2021-02-10T11:22:35.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.188 7f6814ff9700 1 -- 172.21.2.101:0/1970503145 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681000c2f0 con 0x7f68181b4350 2021-02-10T11:22:35.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.188 7f6814ff9700 1 -- 172.21.2.101:0/1970503145 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f681000c810 con 0x7f68181b4350 2021-02-10T11:22:35.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.188 7f681e6af700 1 -- 172.21.2.101:0/1970503145 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68181b1ec0 con 0x7f68181b4350 2021-02-10T11:22:35.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.188 7f681e6af700 1 -- 172.21.2.101:0/1970503145 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68181b2450 con 0x7f68181b4350 2021-02-10T11:22:35.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.188 7f6814ff9700 1 -- 172.21.2.101:0/1970503145 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6810001220 con 0x7f68181b4350 2021-02-10T11:22:35.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.188 7f6814ff9700 1 --2- 172.21.2.101:0/1970503145 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f67f80186c0 0x7f67f801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:35.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.188 7f6814ff9700 1 -- 172.21.2.101:0/1970503145 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6810032920 con 0x7f68181b4350 2021-02-10T11:22:35.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.189 7f6817fff700 1 --2- 172.21.2.101:0/1970503145 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f67f80186c0 0x7f67f801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:35.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.189 7f6817fff700 1 --2- 172.21.2.101:0/1970503145 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f67f80186c0 0x7f67f801abc0 crc :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:35.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.219 7f681e6af700 1 -- 172.21.2.101:0/1970503145 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f681807e6b0 con 0x7f68181b4350 2021-02-10T11:22:35.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.222 7f6814ff9700 1 -- 172.21.2.101:0/1970503145 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6810000f40 con 0x7f68181b4350 2021-02-10T11:22:35.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.345 7f681e6af700 1 -- 172.21.2.101:0/1970503145 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839287", "target": ["mgr", ""]}) v1 -- 0x7f68180ce3e0 con 0x7f67f80186c0 2021-02-10T11:22:35.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.372 7f6814ff9700 1 -- 172.21.2.101:0/1970503145 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f6808007d30 con 0x7f67f80186c0 2021-02-10T11:22:35.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.379 7f681e6af700 1 -- 172.21.2.101:0/1970503145 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f67f80186c0 msgr2=0x7f67f801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:35.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.379 7f681e6af700 1 --2- 172.21.2.101:0/1970503145 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f67f80186c0 0x7f67f801abc0 crc :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:35.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.379 7f681e6af700 1 -- 172.21.2.101:0/1970503145 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68181b4350 msgr2=0x7f68181b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:35.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.379 7f681e6af700 1 --2- 172.21.2.101:0/1970503145 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68181b4350 0x7f68181b18a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f681000b8e0 tx=0x7f681000bc80).stop 2021-02-10T11:22:35.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.379 7f681e6af700 1 -- 172.21.2.101:0/1970503145 shutdown_connections 2021-02-10T11:22:35.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.379 7f681e6af700 1 --2- 172.21.2.101:0/1970503145 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f67f80186c0 0x7f67f801abc0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:35.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.379 7f681e6af700 1 --2- 172.21.2.101:0/1970503145 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68181c25b0 0x7f68181b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:35.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.379 7f681e6af700 1 --2- 172.21.2.101:0/1970503145 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68181b4350 0x7f68181b18a0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:35.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.379 7f681e6af700 1 --2- 172.21.2.101:0/1970503145 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68180a3b50 0x7f68181b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:35.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.379 7f681e6af700 1 -- 172.21.2.101:0/1970503145 >> 172.21.2.101:0/1970503145 conn(0x7f68180886d0 msgr2=0x7f6818081c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:35.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.379 7f681e6af700 1 -- 172.21.2.101:0/1970503145 shutdown_connections 2021-02-10T11:22:35.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.379 7f681e6af700 1 -- 172.21.2.101:0/1970503145 wait complete. 2021-02-10T11:22:35.395 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839287 2021-02-10T11:22:35.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.802 7fb8914ce700 1 Processor -- start 2021-02-10T11:22:35.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.802 7fb8914ce700 1 -- start start 2021-02-10T11:22:35.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.803 7fb8914ce700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb88c1b0870 0x7fb88c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:35.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.803 7fb8914ce700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb88c0efe00 0x7fb88c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:35.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.803 7fb8914ce700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb88c1b0c70 con 0x7fb88c1b0470 2021-02-10T11:22:35.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.803 7fb8914ce700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb88c0f0b50 con 0x7fb88c1b0870 2021-02-10T11:22:35.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.803 7fb8914ce700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c0f0820 con 0x7fb88c0efe00 2021-02-10T11:22:35.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.803 7fb88aabe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb88c1b0870 0x7fb88c0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:35.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.803 7fb88b2bf700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb88c1b0470 0x7fb88c0ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47542/0 (socket says 172.21.2.101:47542) 2021-02-10T11:22:35.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.803 7fb88b2bf700 1 -- 172.21.2.101:0/1604277632 learned_addr learned my addr 172.21.2.101:0/1604277632 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:22:35.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.803 7fb88aabe700 1 -- 172.21.2.101:0/1604277632 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb88c0efe00 msgr2=0x7fb88c0f0200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:22:35.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.803 7fb88aabe700 1 --2- 172.21.2.101:0/1604277632 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb88c0efe00 0x7fb88c0f0200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:35.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.803 7fb88aabe700 1 -- 172.21.2.101:0/1604277632 >> v1:172.21.2.101:6789/0 conn(0x7fb88c1b0470 legacy=0x7fb88c0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:35.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.803 7fb88aabe700 1 -- 172.21.2.101:0/1604277632 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88c0e52e0 con 0x7fb88c1b0870 2021-02-10T11:22:35.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.804 7fb88aabe700 1 --2- 172.21.2.101:0/1604277632 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb88c1b0870 0x7fb88c0f20e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7fb874006650 tx=0x7fb874009a00).ready entity=mon.0 client_cookie=e5f16bb981ecce1f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:35.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.804 7fb88a2bd700 1 -- 172.21.2.101:0/1604277632 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb8740009b0 con 0x7fb88c1b0870 2021-02-10T11:22:35.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.804 7fb88a2bd700 1 -- 172.21.2.101:0/1604277632 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87400c0c0 con 0x7fb88c1b0870 2021-02-10T11:22:35.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.804 7fb88a2bd700 1 -- 172.21.2.101:0/1604277632 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb87400c5a0 con 0x7fb88c1b0870 2021-02-10T11:22:35.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.804 7fb8914ce700 1 -- 172.21.2.101:0/1604277632 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb88c1b0870 msgr2=0x7fb88c0f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:35.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.804 7fb8914ce700 1 --2- 172.21.2.101:0/1604277632 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb88c1b0870 0x7fb88c0f20e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7fb874006650 tx=0x7fb874009a00).stop 2021-02-10T11:22:35.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.804 7fb8914ce700 1 -- 172.21.2.101:0/1604277632 shutdown_connections 2021-02-10T11:22:35.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.804 7fb8914ce700 1 --2- 172.21.2.101:0/1604277632 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb88c0efe00 0x7fb88c0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:35.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.804 7fb8914ce700 1 --2- 172.21.2.101:0/1604277632 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb88c1b0870 0x7fb88c0f20e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:35.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.804 7fb8914ce700 1 -- 172.21.2.101:0/1604277632 >> 172.21.2.101:0/1604277632 conn(0x7fb88c0ce3e0 msgr2=0x7fb88c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:35.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.805 7fb8914ce700 1 -- 172.21.2.101:0/1604277632 shutdown_connections 2021-02-10T11:22:35.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.805 7fb8914ce700 1 -- 172.21.2.101:0/1604277632 wait complete. 2021-02-10T11:22:35.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.819 7fb8914ce700 1 Processor -- start 2021-02-10T11:22:35.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.819 7fb8914ce700 1 -- start start 2021-02-10T11:22:35.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.820 7fb8914ce700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb88c094fb0 0x7fb88c09e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:35.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.820 7fb8914ce700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb88c0c2be0 0x7fb88c0c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:35.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.820 7fb8914ce700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb88c0c3600 0x7fb88c0a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:35.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.820 7fb8914ce700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c09ebf0 con 0x7fb88c094fb0 2021-02-10T11:22:35.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.820 7fb8914ce700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c0aa2a0 con 0x7fb88c0c3600 2021-02-10T11:22:35.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.820 7fb8914ce700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c0aa4c0 con 0x7fb88c0c2be0 2021-02-10T11:22:35.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.820 7fb88b2bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb88c094fb0 0x7fb88c09e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:35.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.820 7fb88b2bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb88c094fb0 0x7fb88c09e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47054/0 (socket says 172.21.2.101:47054) 2021-02-10T11:22:35.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.820 7fb88b2bf700 1 -- 172.21.2.101:0/3235043039 learned_addr learned my addr 172.21.2.101:0/3235043039 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:22:35.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.820 7fb88b2bf700 1 -- 172.21.2.101:0/3235043039 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb88c0c2be0 msgr2=0x7fb88c0c2fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:22:35.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.820 7fb88b2bf700 1 --2- 172.21.2.101:0/3235043039 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb88c0c2be0 0x7fb88c0c2fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:35.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.820 7fb88b2bf700 1 -- 172.21.2.101:0/3235043039 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb88c0c3600 msgr2=0x7fb88c0a9c80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:22:35.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.820 7fb88b2bf700 1 --2- 172.21.2.101:0/3235043039 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb88c0c3600 0x7fb88c0a9c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:35.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.820 7fb88b2bf700 1 -- 172.21.2.101:0/3235043039 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88c0aa6e0 con 0x7fb88c094fb0 2021-02-10T11:22:35.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.820 7fb88b2bf700 1 --2- 172.21.2.101:0/3235043039 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb88c094fb0 0x7fb88c09e5d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7fb87c001c60 tx=0x7fb87c00a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:35.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.821 7fb883fff700 1 -- 172.21.2.101:0/3235043039 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb87c00acc0 con 0x7fb88c094fb0 2021-02-10T11:22:35.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.821 7fb8914ce700 1 -- 172.21.2.101:0/3235043039 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88c0aa6e0 con 0x7fb88c094fb0 2021-02-10T11:22:35.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.821 7fb8914ce700 1 -- 172.21.2.101:0/3235043039 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88c0aac70 con 0x7fb88c094fb0 2021-02-10T11:22:35.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.821 7fb883fff700 1 -- 172.21.2.101:0/3235043039 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87c00acc0 con 0x7fb88c094fb0 2021-02-10T11:22:35.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.821 7fb883fff700 1 -- 172.21.2.101:0/3235043039 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb87c004140 con 0x7fb88c094fb0 2021-02-10T11:22:35.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.821 7fb883fff700 1 -- 172.21.2.101:0/3235043039 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb87c0045d0 con 0x7fb88c094fb0 2021-02-10T11:22:35.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.821 7fb883fff700 1 --2- 172.21.2.101:0/3235043039 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb870018730 0x7fb87001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:35.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.821 7fb883fff700 1 -- 172.21.2.101:0/3235043039 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb87c0327d0 con 0x7fb88c094fb0 2021-02-10T11:22:35.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.821 7fb88aabe700 1 --2- 172.21.2.101:0/3235043039 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb870018730 0x7fb87001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:35.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.822 7fb88aabe700 1 --2- 172.21.2.101:0/3235043039 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb870018730 0x7fb87001abc0 crc :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:35.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.853 7fb8914ce700 1 -- 172.21.2.101:0/3235043039 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88c0a9750 con 0x7fb88c094fb0 2021-02-10T11:22:35.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.855 7fb883fff700 1 -- 172.21.2.101:0/3235043039 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb87c0043c0 con 0x7fb88c094fb0 2021-02-10T11:22:35.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.984 7fb8914ce700 1 -- 172.21.2.101:0/3235043039 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839287", "target": ["mgr", ""]}) v1 -- 0x7fb88c0a9750 con 0x7fb870018730 2021-02-10T11:22:35.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.987 7fb883fff700 1 -- 172.21.2.101:0/3235043039 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fb874000990 con 0x7fb870018730 2021-02-10T11:22:35.995 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839287/4395f51e-7e8f-4173-843b-bed6f9e4f21e 2021-02-10T11:22:35.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.989 7fb8914ce700 1 -- 172.21.2.101:0/3235043039 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb870018730 msgr2=0x7fb87001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:35.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.989 7fb8914ce700 1 --2- 172.21.2.101:0/3235043039 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb870018730 0x7fb87001abc0 crc :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:35.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.989 7fb8914ce700 1 -- 172.21.2.101:0/3235043039 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb88c094fb0 msgr2=0x7fb88c09e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:35.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.989 7fb8914ce700 1 --2- 172.21.2.101:0/3235043039 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb88c094fb0 0x7fb88c09e5d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7fb87c001c60 tx=0x7fb87c00a8d0).stop 2021-02-10T11:22:35.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.989 7fb8914ce700 1 -- 172.21.2.101:0/3235043039 shutdown_connections 2021-02-10T11:22:35.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.989 7fb8914ce700 1 --2- 172.21.2.101:0/3235043039 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb870018730 0x7fb87001abc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:35.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.989 7fb8914ce700 1 --2- 172.21.2.101:0/3235043039 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb88c0c2be0 0x7fb88c0c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:35.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.989 7fb8914ce700 1 --2- 172.21.2.101:0/3235043039 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb88c0c3600 0x7fb88c0a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:35.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.989 7fb8914ce700 1 --2- 172.21.2.101:0/3235043039 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb88c094fb0 0x7fb88c09e5d0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:35.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.989 7fb8914ce700 1 -- 172.21.2.101:0/3235043039 >> 172.21.2.101:0/3235043039 conn(0x7fb88c09f630 msgr2=0x7fb88c0a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:35.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.989 7fb8914ce700 1 -- 172.21.2.101:0/3235043039 shutdown_connections 2021-02-10T11:22:35.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:35.989 7fb8914ce700 1 -- 172.21.2.101:0/3235043039 wait complete. 2021-02-10T11:22:36.005 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839287 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839287/4395f51e-7e8f-4173-843b-bed6f9e4f21e 2021-02-10T11:22:36.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:36.005 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:36.005 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:36.340 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:36.340 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:36.341 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839287/4395f51e-7e8f-4173-843b-bed6f9e4f21e/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:36.723 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:36.723 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:36.723 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235129 s, 44.6 MB/s 2021-02-10T11:22:36.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:36.724 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:36.724 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:37.058 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:37.059 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:37.059 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839287/4395f51e-7e8f-4173-843b-bed6f9e4f21e/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:37.434 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:37.434 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:37.434 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0214847 s, 48.8 MB/s 2021-02-10T11:22:37.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:37.435 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:37.435 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:37.770 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:37.771 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:37.771 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839287/4395f51e-7e8f-4173-843b-bed6f9e4f21e/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:38.147 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:38.147 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:38.148 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234742 s, 44.7 MB/s 2021-02-10T11:22:38.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:38.149 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:38.149 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:38.484 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:38.484 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:38.485 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839287/4395f51e-7e8f-4173-843b-bed6f9e4f21e/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:38.857 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:38.857 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:38.857 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245674 s, 42.7 MB/s 2021-02-10T11:22:38.859 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:38.859 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:38.859 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:39.194 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:39.195 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:39.195 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839287/4395f51e-7e8f-4173-843b-bed6f9e4f21e/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:39.572 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:39.572 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:39.573 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253766 s, 41.3 MB/s 2021-02-10T11:22:39.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:39.574 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:39.574 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:39.909 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:39.909 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:39.909 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839287/4395f51e-7e8f-4173-843b-bed6f9e4f21e/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:40.285 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:40.285 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:40.285 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023751 s, 44.1 MB/s 2021-02-10T11:22:40.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:40.286 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:40.286 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:40.623 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:40.623 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:40.623 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839287/4395f51e-7e8f-4173-843b-bed6f9e4f21e/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:40.997 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:40.997 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:40.998 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236865 s, 44.3 MB/s 2021-02-10T11:22:40.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:40.999 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:40.999 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:41.333 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:41.334 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:41.334 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839287/4395f51e-7e8f-4173-843b-bed6f9e4f21e/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:41.710 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:41.710 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:41.710 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231537 s, 45.3 MB/s 2021-02-10T11:22:41.711 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:41.711 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:41.712 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:42.046 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:42.047 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:42.047 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839287/4395f51e-7e8f-4173-843b-bed6f9e4f21e/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:42.422 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:42.422 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:42.422 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236776 s, 44.3 MB/s 2021-02-10T11:22:42.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:42.423 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:42.423 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:42.758 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:42.759 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:42.759 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839287/4395f51e-7e8f-4173-843b-bed6f9e4f21e/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:43.135 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:43.136 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:43.136 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0254342 s, 41.2 MB/s 2021-02-10T11:22:43.137 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839288 2021-02-10T11:22:43.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.544 7fbed792c700 1 Processor -- start 2021-02-10T11:22:43.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.544 7fbed792c700 1 -- start start 2021-02-10T11:22:43.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.545 7fbed792c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbed01b0470 0x7fbed01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:43.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.545 7fbed792c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbed00f20e0 0x7fbed00f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:43.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.545 7fbed792c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed00f0b50 con 0x7fbed00ef610 2021-02-10T11:22:43.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.545 7fbed792c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed00f0820 con 0x7fbed00f20e0 2021-02-10T11:22:43.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.545 7fbed792c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed00f24e0 con 0x7fbed01b0470 2021-02-10T11:22:43.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.545 7fbed5f49700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbed00f20e0 0x7fbed00f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:43.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.545 7fbed5f49700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbed00f20e0 0x7fbed00f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47060/0 (socket says 172.21.2.101:47060) 2021-02-10T11:22:43.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.545 7fbed5f49700 1 -- 172.21.2.101:0/2569944304 learned_addr learned my addr 172.21.2.101:0/2569944304 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:22:43.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.545 7fbed5f49700 1 -- 172.21.2.101:0/2569944304 >> v2:172.21.2.108:3300/0 conn(0x7fbed01b0470 msgr2=0x7fbed01b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:22:43.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.545 7fbed5f49700 1 --2- 172.21.2.101:0/2569944304 >> v2:172.21.2.108:3300/0 conn(0x7fbed01b0470 0x7fbed01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:43.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.546 7fbed5f49700 1 -- 172.21.2.101:0/2569944304 >> v1:172.21.2.101:6789/0 conn(0x7fbed00ef610 legacy=0x7fbed00efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:43.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.546 7fbed5f49700 1 -- 172.21.2.101:0/2569944304 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed00e52e0 con 0x7fbed00f20e0 2021-02-10T11:22:43.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.546 7fbed5f49700 1 --2- 172.21.2.101:0/2569944304 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbed00f20e0 0x7fbed00f0200 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7fbec8006f00 tx=0x7fbec8009f10).ready entity=mon.0 client_cookie=17a1858e7c08e095 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:43.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.546 7fbec3fff700 1 -- 172.21.2.101:0/2569944304 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbec800b3b0 con 0x7fbed00f20e0 2021-02-10T11:22:43.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.546 7fbec3fff700 1 -- 172.21.2.101:0/2569944304 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec800b730 con 0x7fbed00f20e0 2021-02-10T11:22:43.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.546 7fbec3fff700 1 -- 172.21.2.101:0/2569944304 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbec800bb90 con 0x7fbed00f20e0 2021-02-10T11:22:43.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.546 7fbed792c700 1 -- 172.21.2.101:0/2569944304 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbed00f20e0 msgr2=0x7fbed00f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:43.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.546 7fbed792c700 1 --2- 172.21.2.101:0/2569944304 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbed00f20e0 0x7fbed00f0200 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7fbec8006f00 tx=0x7fbec8009f10).stop 2021-02-10T11:22:43.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.546 7fbed792c700 1 -- 172.21.2.101:0/2569944304 shutdown_connections 2021-02-10T11:22:43.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.547 7fbed792c700 1 --2- 172.21.2.101:0/2569944304 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbed00f20e0 0x7fbed00f0200 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:43.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.547 7fbed792c700 1 --2- 172.21.2.101:0/2569944304 >> v2:172.21.2.108:3300/0 conn(0x7fbed01b0470 0x7fbed01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:43.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.547 7fbed792c700 1 -- 172.21.2.101:0/2569944304 >> 172.21.2.101:0/2569944304 conn(0x7fbed00ce3e0 msgr2=0x7fbed00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:43.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.547 7fbed792c700 1 -- 172.21.2.101:0/2569944304 shutdown_connections 2021-02-10T11:22:43.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.547 7fbed792c700 1 -- 172.21.2.101:0/2569944304 wait complete. 2021-02-10T11:22:43.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.561 7fbed792c700 1 Processor -- start 2021-02-10T11:22:43.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.561 7fbed792c700 1 -- start start 2021-02-10T11:22:43.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.562 7fbed792c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbed00c36a0 0x7fbed00c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:43.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.562 7fbed792c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbed00c3190 0x7fbed00b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:43.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.562 7fbed792c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbed00b2370 0x7fbed00b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:43.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.562 7fbed792c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed00b27c0 con 0x7fbed00c36a0 2021-02-10T11:22:43.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.562 7fbed792c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed00b3360 con 0x7fbed00c3190 2021-02-10T11:22:43.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.562 7fbed792c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed00b3540 con 0x7fbed00b2370 2021-02-10T11:22:43.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.562 7fbed5748700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbed00c36a0 0x7fbed00c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:43.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.562 7fbed5748700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbed00c36a0 0x7fbed00c2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47066/0 (socket says 172.21.2.101:47066) 2021-02-10T11:22:43.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.562 7fbed5748700 1 -- 172.21.2.101:0/3040513287 learned_addr learned my addr 172.21.2.101:0/3040513287 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:22:43.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.562 7fbed4f47700 1 --2- 172.21.2.101:0/3040513287 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbed00c3190 0x7fbed00b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:43.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.562 7fbed5f49700 1 --2- 172.21.2.101:0/3040513287 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbed00b2370 0x7fbed00b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:43.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.562 7fbed5748700 1 -- 172.21.2.101:0/3040513287 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbed00b2370 msgr2=0x7fbed00b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:43.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.562 7fbed5748700 1 --2- 172.21.2.101:0/3040513287 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbed00b2370 0x7fbed00b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:43.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.562 7fbed5748700 1 -- 172.21.2.101:0/3040513287 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbed00c3190 msgr2=0x7fbed00b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:43.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.562 7fbed5748700 1 --2- 172.21.2.101:0/3040513287 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbed00c3190 0x7fbed00b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:43.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.562 7fbed5748700 1 -- 172.21.2.101:0/3040513287 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed00b3760 con 0x7fbed00c36a0 2021-02-10T11:22:43.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.562 7fbed4f47700 1 --2- 172.21.2.101:0/3040513287 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbed00c3190 0x7fbed00b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:22:43.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.563 7fbed5748700 1 --2- 172.21.2.101:0/3040513287 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbed00c36a0 0x7fbed00c2b70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fbec400b530 tx=0x7fbec400b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:43.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.563 7fbec1ffb700 1 -- 172.21.2.101:0/3040513287 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbec400bcc0 con 0x7fbed00c36a0 2021-02-10T11:22:43.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.563 7fbec1ffb700 1 -- 172.21.2.101:0/3040513287 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec4000c90 con 0x7fbed00c36a0 2021-02-10T11:22:43.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.563 7fbed792c700 1 -- 172.21.2.101:0/3040513287 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed00b3760 con 0x7fbed00c36a0 2021-02-10T11:22:43.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.563 7fbec1ffb700 1 -- 172.21.2.101:0/3040513287 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbec40022a0 con 0x7fbed00c36a0 2021-02-10T11:22:43.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.563 7fbed792c700 1 -- 172.21.2.101:0/3040513287 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed00b3760 con 0x7fbed00c36a0 2021-02-10T11:22:43.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.563 7fbec1ffb700 1 -- 172.21.2.101:0/3040513287 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbec4005310 con 0x7fbed00c36a0 2021-02-10T11:22:43.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.563 7fbec1ffb700 1 --2- 172.21.2.101:0/3040513287 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbeb00186d0 0x7fbeb001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:43.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.564 7fbec1ffb700 1 -- 172.21.2.101:0/3040513287 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fbec40325f0 con 0x7fbed00c36a0 2021-02-10T11:22:43.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.564 7fbed4f47700 1 --2- 172.21.2.101:0/3040513287 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbeb00186d0 0x7fbeb001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:43.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.564 7fbed4f47700 1 --2- 172.21.2.101:0/3040513287 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbeb00186d0 0x7fbeb001abd0 crc :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:43.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.595 7fbed792c700 1 -- 172.21.2.101:0/3040513287 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed00edd10 con 0x7fbed00c36a0 2021-02-10T11:22:43.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.597 7fbec1ffb700 1 -- 172.21.2.101:0/3040513287 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbec4006b90 con 0x7fbed00c36a0 2021-02-10T11:22:43.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.723 7fbed792c700 1 -- 172.21.2.101:0/3040513287 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839288", "target": ["mgr", ""]}) v1 -- 0x7fbed00edd10 con 0x7fbeb00186d0 2021-02-10T11:22:43.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.752 7fbec1ffb700 1 -- 172.21.2.101:0/3040513287 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fbecc006950 con 0x7fbeb00186d0 2021-02-10T11:22:43.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.756 7fbed792c700 1 -- 172.21.2.101:0/3040513287 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbeb00186d0 msgr2=0x7fbeb001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:43.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.756 7fbed792c700 1 --2- 172.21.2.101:0/3040513287 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbeb00186d0 0x7fbeb001abd0 crc :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:43.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.756 7fbed792c700 1 -- 172.21.2.101:0/3040513287 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbed00c36a0 msgr2=0x7fbed00c2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:43.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.756 7fbed792c700 1 --2- 172.21.2.101:0/3040513287 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbed00c36a0 0x7fbed00c2b70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fbec400b530 tx=0x7fbec400b8d0).stop 2021-02-10T11:22:43.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.756 7fbed792c700 1 -- 172.21.2.101:0/3040513287 shutdown_connections 2021-02-10T11:22:43.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.756 7fbed792c700 1 --2- 172.21.2.101:0/3040513287 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbeb00186d0 0x7fbeb001abd0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:43.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.757 7fbed792c700 1 --2- 172.21.2.101:0/3040513287 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbed00b2370 0x7fbed00b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:43.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.757 7fbed792c700 1 --2- 172.21.2.101:0/3040513287 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbed00c3190 0x7fbed00b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:43.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.757 7fbed792c700 1 --2- 172.21.2.101:0/3040513287 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbed00c36a0 0x7fbed00c2b70 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:43.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.757 7fbed792c700 1 -- 172.21.2.101:0/3040513287 >> 172.21.2.101:0/3040513287 conn(0x7fbed009f630 msgr2=0x7fbed00a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:43.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.757 7fbed792c700 1 -- 172.21.2.101:0/3040513287 shutdown_connections 2021-02-10T11:22:43.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:43.757 7fbed792c700 1 -- 172.21.2.101:0/3040513287 wait complete. 2021-02-10T11:22:43.773 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839288 2021-02-10T11:22:44.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.179 7fec8cef1700 1 Processor -- start 2021-02-10T11:22:44.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.179 7fec8cef1700 1 -- start start 2021-02-10T11:22:44.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.179 7fec8cef1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fec881b0870 0x7fec880f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:44.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.179 7fec8cef1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fec880efe00 0x7fec880f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:44.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.179 7fec8cef1700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec881b0c70 con 0x7fec881b0470 2021-02-10T11:22:44.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.179 7fec8cef1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec880f0b50 con 0x7fec880efe00 2021-02-10T11:22:44.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.179 7fec8cef1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec880f0820 con 0x7fec881b0870 2021-02-10T11:22:44.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.180 7fec86312700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fec881b0870 0x7fec880f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:44.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.180 7fec86312700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fec881b0870 0x7fec880f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:39806/0 (socket says 172.21.2.101:39806) 2021-02-10T11:22:44.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.180 7fec86312700 1 -- 172.21.2.101:0/833195386 learned_addr learned my addr 172.21.2.101:0/833195386 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:22:44.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.180 7fec87314700 1 --2- 172.21.2.101:0/833195386 >> v2:172.21.2.108:3300/0 conn(0x7fec880efe00 0x7fec880f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:44.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.181 7fec85b11700 1 -- 172.21.2.101:0/833195386 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3183828570 0 0) 0x7fec70000c60 con 0x7fec881b0470 2021-02-10T11:22:44.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.181 7fec85b11700 1 -- 172.21.2.101:0/833195386 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec74002fe0 con 0x7fec881b0470 2021-02-10T11:22:44.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.181 7fec86312700 1 -- 172.21.2.101:0/833195386 >> v2:172.21.2.108:3300/0 conn(0x7fec880efe00 msgr2=0x7fec880f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:22:44.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.181 7fec86312700 1 --2- 172.21.2.101:0/833195386 >> v2:172.21.2.108:3300/0 conn(0x7fec880efe00 0x7fec880f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:44.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.181 7fec86312700 1 -- 172.21.2.101:0/833195386 >> v1:172.21.2.108:6789/0 conn(0x7fec881b0470 legacy=0x7fec880ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:44.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.181 7fec86312700 1 -- 172.21.2.101:0/833195386 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec880e52e0 con 0x7fec881b0870 2021-02-10T11:22:44.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.181 7fec87314700 1 --2- 172.21.2.101:0/833195386 >> v2:172.21.2.108:3300/0 conn(0x7fec880efe00 0x7fec880f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:22:44.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.181 7fec86312700 1 --2- 172.21.2.101:0/833195386 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fec881b0870 0x7fec880f20e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fec78006700 tx=0x7fec78009b80).ready entity=mon.2 client_cookie=f5663ba9c0190aff server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:44.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.181 7fec85b11700 1 -- 172.21.2.101:0/833195386 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fec7800c0c0 con 0x7fec881b0870 2021-02-10T11:22:44.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.181 7fec85b11700 1 -- 172.21.2.101:0/833195386 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7800c340 con 0x7fec881b0870 2021-02-10T11:22:44.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.181 7fec85b11700 1 -- 172.21.2.101:0/833195386 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fec7800c860 con 0x7fec881b0870 2021-02-10T11:22:44.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.181 7fec8cef1700 1 -- 172.21.2.101:0/833195386 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fec881b0870 msgr2=0x7fec880f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:44.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.181 7fec8cef1700 1 --2- 172.21.2.101:0/833195386 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fec881b0870 0x7fec880f20e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fec78006700 tx=0x7fec78009b80).stop 2021-02-10T11:22:44.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.182 7fec8cef1700 1 -- 172.21.2.101:0/833195386 shutdown_connections 2021-02-10T11:22:44.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.182 7fec8cef1700 1 --2- 172.21.2.101:0/833195386 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fec881b0870 0x7fec880f20e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:44.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.182 7fec8cef1700 1 --2- 172.21.2.101:0/833195386 >> v2:172.21.2.108:3300/0 conn(0x7fec880efe00 0x7fec880f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:44.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.182 7fec8cef1700 1 -- 172.21.2.101:0/833195386 >> 172.21.2.101:0/833195386 conn(0x7fec880ce3e0 msgr2=0x7fec880e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:44.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.182 7fec8cef1700 1 -- 172.21.2.101:0/833195386 shutdown_connections 2021-02-10T11:22:44.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.182 7fec8cef1700 1 -- 172.21.2.101:0/833195386 wait complete. 2021-02-10T11:22:44.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.197 7fec8cef1700 1 Processor -- start 2021-02-10T11:22:44.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.197 7fec8cef1700 1 -- start start 2021-02-10T11:22:44.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.197 7fec8cef1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fec880a0420 0x7fec880a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:44.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.197 7fec8cef1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fec88189d00 0x7fec8818a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:44.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.197 7fec8cef1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fec8818a720 0x7fec880b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:44.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.197 7fec8cef1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec88189800 con 0x7fec880a0420 2021-02-10T11:22:44.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.197 7fec8cef1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8818ab20 con 0x7fec8818a720 2021-02-10T11:22:44.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.197 7fec8cef1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec880a1140 con 0x7fec88189d00 2021-02-10T11:22:44.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.198 7fec86b13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fec880a0420 0x7fec880a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:44.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.198 7fec86b13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fec880a0420 0x7fec880a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47080/0 (socket says 172.21.2.101:47080) 2021-02-10T11:22:44.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.198 7fec86b13700 1 -- 172.21.2.101:0/3915139955 learned_addr learned my addr 172.21.2.101:0/3915139955 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:22:44.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.198 7fec86312700 1 --2- 172.21.2.101:0/3915139955 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fec88189d00 0x7fec8818a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:44.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.198 7fec86b13700 1 -- 172.21.2.101:0/3915139955 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fec88189d00 msgr2=0x7fec8818a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:44.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.198 7fec86b13700 1 --2- 172.21.2.101:0/3915139955 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fec88189d00 0x7fec8818a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:44.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.198 7fec86b13700 1 -- 172.21.2.101:0/3915139955 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fec8818a720 msgr2=0x7fec880b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:44.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.198 7fec86b13700 1 --2- 172.21.2.101:0/3915139955 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fec8818a720 0x7fec880b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:44.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.198 7fec86b13700 1 -- 172.21.2.101:0/3915139955 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec880b8200 con 0x7fec880a0420 2021-02-10T11:22:44.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.198 7fec86312700 1 --2- 172.21.2.101:0/3915139955 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fec88189d00 0x7fec8818a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:22:44.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.198 7fec86b13700 1 --2- 172.21.2.101:0/3915139955 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fec880a0420 0x7fec880a0820 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7fec70006f10 tx=0x7fec7000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:44.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.198 7fec7f7fe700 1 -- 172.21.2.101:0/3915139955 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fec7000bd80 con 0x7fec880a0420 2021-02-10T11:22:44.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.198 7fec7f7fe700 1 -- 172.21.2.101:0/3915139955 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec70000e20 con 0x7fec880a0420 2021-02-10T11:22:44.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.198 7fec7f7fe700 1 -- 172.21.2.101:0/3915139955 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fec700054f0 con 0x7fec880a0420 2021-02-10T11:22:44.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.198 7fec8cef1700 1 -- 172.21.2.101:0/3915139955 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec880b8200 con 0x7fec880a0420 2021-02-10T11:22:44.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.198 7fec8cef1700 1 -- 172.21.2.101:0/3915139955 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec880b8790 con 0x7fec880a0420 2021-02-10T11:22:44.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.199 7fec7f7fe700 1 -- 172.21.2.101:0/3915139955 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fec70009fe0 con 0x7fec880a0420 2021-02-10T11:22:44.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.199 7fec7f7fe700 1 --2- 172.21.2.101:0/3915139955 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fec6c0186c0 0x7fec6c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:44.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.199 7fec7f7fe700 1 -- 172.21.2.101:0/3915139955 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fec70032570 con 0x7fec880a0420 2021-02-10T11:22:44.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.199 7fec86312700 1 --2- 172.21.2.101:0/3915139955 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fec6c0186c0 0x7fec6c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:44.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.200 7fec86312700 1 --2- 172.21.2.101:0/3915139955 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fec6c0186c0 0x7fec6c01abc0 crc :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:44.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.230 7fec8cef1700 1 -- 172.21.2.101:0/3915139955 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec88105ae0 con 0x7fec880a0420 2021-02-10T11:22:44.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.232 7fec7f7fe700 1 -- 172.21.2.101:0/3915139955 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fec70000bc0 con 0x7fec880a0420 2021-02-10T11:22:44.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.362 7fec8cef1700 1 -- 172.21.2.101:0/3915139955 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839288", "target": ["mgr", ""]}) v1 -- 0x7fec880b8790 con 0x7fec6c0186c0 2021-02-10T11:22:44.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.364 7fec7f7fe700 1 -- 172.21.2.101:0/3915139955 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fec78006ad0 con 0x7fec6c0186c0 2021-02-10T11:22:44.372 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839288/617f38bc-85d3-484b-92f7-9179dc460252 2021-02-10T11:22:44.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.366 7fec8cef1700 1 -- 172.21.2.101:0/3915139955 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fec6c0186c0 msgr2=0x7fec6c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:44.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.366 7fec8cef1700 1 --2- 172.21.2.101:0/3915139955 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fec6c0186c0 0x7fec6c01abc0 crc :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:44.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.366 7fec8cef1700 1 -- 172.21.2.101:0/3915139955 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fec880a0420 msgr2=0x7fec880a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:44.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.366 7fec8cef1700 1 --2- 172.21.2.101:0/3915139955 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fec880a0420 0x7fec880a0820 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7fec70006f10 tx=0x7fec7000b990).stop 2021-02-10T11:22:44.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.366 7fec8cef1700 1 -- 172.21.2.101:0/3915139955 shutdown_connections 2021-02-10T11:22:44.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.366 7fec8cef1700 1 --2- 172.21.2.101:0/3915139955 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fec6c0186c0 0x7fec6c01abc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:44.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.366 7fec8cef1700 1 --2- 172.21.2.101:0/3915139955 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fec88189d00 0x7fec8818a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:44.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.366 7fec8cef1700 1 --2- 172.21.2.101:0/3915139955 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fec8818a720 0x7fec880b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:44.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.366 7fec8cef1700 1 --2- 172.21.2.101:0/3915139955 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fec880a0420 0x7fec880a0820 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:44.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.366 7fec8cef1700 1 -- 172.21.2.101:0/3915139955 >> 172.21.2.101:0/3915139955 conn(0x7fec88094e10 msgr2=0x7fec880a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:44.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.367 7fec8cef1700 1 -- 172.21.2.101:0/3915139955 shutdown_connections 2021-02-10T11:22:44.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:44.367 7fec8cef1700 1 -- 172.21.2.101:0/3915139955 wait complete. 2021-02-10T11:22:44.382 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839288 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839288/617f38bc-85d3-484b-92f7-9179dc460252 2021-02-10T11:22:44.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:44.383 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:44.383 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:44.719 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:44.719 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:44.719 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839288/617f38bc-85d3-484b-92f7-9179dc460252/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:45.100 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:45.100 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:45.100 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233438 s, 44.9 MB/s 2021-02-10T11:22:45.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:45.101 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:45.101 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:45.437 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:45.438 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:45.438 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839288/617f38bc-85d3-484b-92f7-9179dc460252/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:45.815 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:45.816 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:45.816 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242961 s, 43.2 MB/s 2021-02-10T11:22:45.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:45.817 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:45.817 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:46.154 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:46.154 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:46.154 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839288/617f38bc-85d3-484b-92f7-9179dc460252/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:46.528 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:46.528 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:46.528 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242061 s, 43.3 MB/s 2021-02-10T11:22:46.529 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:46.529 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:46.529 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:46.865 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:46.865 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:46.865 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839288/617f38bc-85d3-484b-92f7-9179dc460252/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:47.241 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:47.241 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:47.241 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248712 s, 42.2 MB/s 2021-02-10T11:22:47.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:47.242 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:47.242 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:47.576 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:47.576 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:47.576 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839288/617f38bc-85d3-484b-92f7-9179dc460252/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:47.952 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:47.952 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:47.953 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248191 s, 42.2 MB/s 2021-02-10T11:22:47.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:47.954 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:47.954 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:48.289 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:48.289 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:48.289 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839288/617f38bc-85d3-484b-92f7-9179dc460252/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:48.664 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:48.665 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:48.665 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247514 s, 42.4 MB/s 2021-02-10T11:22:48.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:48.666 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:48.666 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:49.000 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:49.001 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:49.001 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839288/617f38bc-85d3-484b-92f7-9179dc460252/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:49.373 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:49.374 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:49.374 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235505 s, 44.5 MB/s 2021-02-10T11:22:49.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:49.375 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:49.375 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:49.711 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:49.711 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:49.711 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839288/617f38bc-85d3-484b-92f7-9179dc460252/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:50.089 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:50.332 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:50.332 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0256369 s, 40.9 MB/s 2021-02-10T11:22:50.333 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:22:50.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:50.334 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:50.334 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:50.336 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:22:50.674 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:50.674 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:50.674 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839288/617f38bc-85d3-484b-92f7-9179dc460252/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:51.054 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:51.054 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:51.054 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.026931 s, 38.9 MB/s 2021-02-10T11:22:51.055 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:51.055 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:51.055 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:51.391 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:51.392 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:51.392 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839288/617f38bc-85d3-484b-92f7-9179dc460252/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:51.769 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:51.769 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:51.769 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252932 s, 41.5 MB/s 2021-02-10T11:22:51.770 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839289 2021-02-10T11:22:52.202 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.177 7ff5d1f01700 1 Processor -- start 2021-02-10T11:22:52.202 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.177 7ff5d1f01700 1 -- start start 2021-02-10T11:22:52.202 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.178 7ff5d1f01700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff5cc1b0470 0x7ff5cc1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:52.202 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.178 7ff5d1f01700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff5cc0f20e0 0x7ff5cc0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:52.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.178 7ff5d1f01700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5cc0f0b50 con 0x7ff5cc0ef610 2021-02-10T11:22:52.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.178 7ff5d1f01700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5cc0f0820 con 0x7ff5cc0f20e0 2021-02-10T11:22:52.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.178 7ff5d1f01700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5cc0f24e0 con 0x7ff5cc1b0470 2021-02-10T11:22:52.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.178 7ff5cbfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff5cc0f20e0 0x7ff5cc0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:52.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.178 7ff5cbfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff5cc0f20e0 0x7ff5cc0f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47088/0 (socket says 172.21.2.101:47088) 2021-02-10T11:22:52.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.178 7ff5cbfff700 1 -- 172.21.2.101:0/2138239129 learned_addr learned my addr 172.21.2.101:0/2138239129 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:22:52.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.178 7ff5cb7fe700 1 --2- 172.21.2.101:0/2138239129 >> v2:172.21.2.108:3300/0 conn(0x7ff5cc1b0470 0x7ff5cc1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:52.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.179 7ff5cbfff700 1 -- 172.21.2.101:0/2138239129 >> v2:172.21.2.108:3300/0 conn(0x7ff5cc1b0470 msgr2=0x7ff5cc1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:22:52.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.179 7ff5cbfff700 1 --2- 172.21.2.101:0/2138239129 >> v2:172.21.2.108:3300/0 conn(0x7ff5cc1b0470 0x7ff5cc1b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:52.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.179 7ff5cbfff700 1 -- 172.21.2.101:0/2138239129 >> v1:172.21.2.101:6789/0 conn(0x7ff5cc0ef610 legacy=0x7ff5cc0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:52.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.179 7ff5cbfff700 1 -- 172.21.2.101:0/2138239129 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5cc0e52e0 con 0x7ff5cc0f20e0 2021-02-10T11:22:52.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.179 7ff5ca7fc700 1 -- 172.21.2.101:0/2138239129 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4294682752 0 0) 0x7ff5c0000ff0 con 0x7ff5cc0ef610 2021-02-10T11:22:52.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.179 7ff5cbfff700 1 --2- 172.21.2.101:0/2138239129 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5cc0f20e0 0x7ff5cc0f0200 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7ff5c4006f00 tx=0x7ff5c4009e90).ready entity=mon.0 client_cookie=77b93171dfd76401 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:52.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.179 7ff5ca7fc700 1 -- 172.21.2.101:0/2138239129 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff5c400b300 con 0x7ff5cc0f20e0 2021-02-10T11:22:52.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.179 7ff5ca7fc700 1 -- 172.21.2.101:0/2138239129 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c400b580 con 0x7ff5cc0f20e0 2021-02-10T11:22:52.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.179 7ff5ca7fc700 1 -- 172.21.2.101:0/2138239129 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff5c400b9d0 con 0x7ff5cc0f20e0 2021-02-10T11:22:52.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.179 7ff5d1f01700 1 -- 172.21.2.101:0/2138239129 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5cc0f20e0 msgr2=0x7ff5cc0f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:52.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.179 7ff5d1f01700 1 --2- 172.21.2.101:0/2138239129 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5cc0f20e0 0x7ff5cc0f0200 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7ff5c4006f00 tx=0x7ff5c4009e90).stop 2021-02-10T11:22:52.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.180 7ff5d1f01700 1 -- 172.21.2.101:0/2138239129 shutdown_connections 2021-02-10T11:22:52.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.180 7ff5d1f01700 1 --2- 172.21.2.101:0/2138239129 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5cc0f20e0 0x7ff5cc0f0200 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:52.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.180 7ff5d1f01700 1 --2- 172.21.2.101:0/2138239129 >> v2:172.21.2.108:3300/0 conn(0x7ff5cc1b0470 0x7ff5cc1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:52.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.180 7ff5d1f01700 1 -- 172.21.2.101:0/2138239129 >> 172.21.2.101:0/2138239129 conn(0x7ff5cc0ce3e0 msgr2=0x7ff5cc0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:52.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.180 7ff5d1f01700 1 -- 172.21.2.101:0/2138239129 shutdown_connections 2021-02-10T11:22:52.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.180 7ff5d1f01700 1 -- 172.21.2.101:0/2138239129 wait complete. 2021-02-10T11:22:52.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.195 7ff5d1f01700 1 Processor -- start 2021-02-10T11:22:52.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.195 7ff5d1f01700 1 -- start start 2021-02-10T11:22:52.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.195 7ff5d1f01700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5cc190920 0x7ff5cc0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:52.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.195 7ff5d1f01700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5cc0b8200 0x7ff5cc0b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:52.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.195 7ff5d1f01700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5cc0c1d80 0x7ff5cc0c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:52.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.195 7ff5d1f01700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc190da0 con 0x7ff5cc0b8200 2021-02-10T11:22:52.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.195 7ff5d1f01700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc0b8c20 con 0x7ff5cc0c1d80 2021-02-10T11:22:52.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.195 7ff5d1f01700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc0b8e10 con 0x7ff5cc190920 2021-02-10T11:22:52.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.195 7ff5caffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5cc0b8200 0x7ff5cc0b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:52.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.195 7ff5caffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5cc0b8200 0x7ff5cc0b8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47096/0 (socket says 172.21.2.101:47096) 2021-02-10T11:22:52.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.195 7ff5caffd700 1 -- 172.21.2.101:0/3048562861 learned_addr learned my addr 172.21.2.101:0/3048562861 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:22:52.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.196 7ff5cbfff700 1 --2- 172.21.2.101:0/3048562861 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5cc0c1d80 0x7ff5cc0c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:52.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.196 7ff5cb7fe700 1 --2- 172.21.2.101:0/3048562861 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5cc190920 0x7ff5cc0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:52.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.196 7ff5caffd700 1 -- 172.21.2.101:0/3048562861 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5cc190920 msgr2=0x7ff5cc0b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:52.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.196 7ff5caffd700 1 --2- 172.21.2.101:0/3048562861 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5cc190920 0x7ff5cc0b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:52.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.196 7ff5caffd700 1 -- 172.21.2.101:0/3048562861 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5cc0c1d80 msgr2=0x7ff5cc0c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:52.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.196 7ff5caffd700 1 --2- 172.21.2.101:0/3048562861 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5cc0c1d80 0x7ff5cc0c2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:52.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.196 7ff5caffd700 1 -- 172.21.2.101:0/3048562861 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5cc0c27d0 con 0x7ff5cc0b8200 2021-02-10T11:22:52.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.196 7ff5cb7fe700 1 --2- 172.21.2.101:0/3048562861 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5cc190920 0x7ff5cc0b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:22:52.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.196 7ff5caffd700 1 --2- 172.21.2.101:0/3048562861 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5cc0b8200 0x7ff5cc0b8600 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7ff5c0006f20 tx=0x7ff5c000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:52.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.196 7ff5b3fff700 1 -- 172.21.2.101:0/3048562861 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff5c00080d0 con 0x7ff5cc0b8200 2021-02-10T11:22:52.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.196 7ff5b3fff700 1 -- 172.21.2.101:0/3048562861 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c0008350 con 0x7ff5cc0b8200 2021-02-10T11:22:52.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.196 7ff5b3fff700 1 -- 172.21.2.101:0/3048562861 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff5c00087a0 con 0x7ff5cc0b8200 2021-02-10T11:22:52.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.196 7ff5d1f01700 1 -- 172.21.2.101:0/3048562861 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5cc0c27d0 con 0x7ff5cc0b8200 2021-02-10T11:22:52.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.196 7ff5d1f01700 1 -- 172.21.2.101:0/3048562861 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5cc0c27d0 con 0x7ff5cc0b8200 2021-02-10T11:22:52.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.197 7ff5b3fff700 1 -- 172.21.2.101:0/3048562861 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff5c0008530 con 0x7ff5cc0b8200 2021-02-10T11:22:52.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.197 7ff5b3fff700 1 --2- 172.21.2.101:0/3048562861 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5ac0186d0 0x7ff5ac01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:52.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.197 7ff5b3fff700 1 -- 172.21.2.101:0/3048562861 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff5c00327c0 con 0x7ff5cc0b8200 2021-02-10T11:22:52.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.197 7ff5cb7fe700 1 --2- 172.21.2.101:0/3048562861 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5ac0186d0 0x7ff5ac01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:52.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.197 7ff5cb7fe700 1 --2- 172.21.2.101:0/3048562861 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5ac0186d0 0x7ff5ac01abd0 crc :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:52.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.228 7ff5d1f01700 1 -- 172.21.2.101:0/3048562861 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5cc105ae0 con 0x7ff5cc0b8200 2021-02-10T11:22:52.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.230 7ff5b3fff700 1 -- 172.21.2.101:0/3048562861 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff5c00009b0 con 0x7ff5cc0b8200 2021-02-10T11:22:52.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.357 7ff5d1f01700 1 -- 172.21.2.101:0/3048562861 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839289", "target": ["mgr", ""]}) v1 -- 0x7ff5cc0f0b50 con 0x7ff5ac0186d0 2021-02-10T11:22:52.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.379 7ff5b3fff700 1 -- 172.21.2.101:0/3048562861 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7ff5bc0009b0 con 0x7ff5ac0186d0 2021-02-10T11:22:52.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.390 7ff5d1f01700 1 -- 172.21.2.101:0/3048562861 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5ac0186d0 msgr2=0x7ff5ac01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:52.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.390 7ff5d1f01700 1 --2- 172.21.2.101:0/3048562861 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5ac0186d0 0x7ff5ac01abd0 crc :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:52.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.390 7ff5d1f01700 1 -- 172.21.2.101:0/3048562861 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5cc0b8200 msgr2=0x7ff5cc0b8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:52.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.390 7ff5d1f01700 1 --2- 172.21.2.101:0/3048562861 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5cc0b8200 0x7ff5cc0b8600 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7ff5c0006f20 tx=0x7ff5c000b990).stop 2021-02-10T11:22:52.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.390 7ff5d1f01700 1 -- 172.21.2.101:0/3048562861 shutdown_connections 2021-02-10T11:22:52.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.390 7ff5d1f01700 1 --2- 172.21.2.101:0/3048562861 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5cc0c1d80 0x7ff5cc0c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:52.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.390 7ff5d1f01700 1 --2- 172.21.2.101:0/3048562861 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5cc0b8200 0x7ff5cc0b8600 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:52.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.390 7ff5d1f01700 1 --2- 172.21.2.101:0/3048562861 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5ac0186d0 0x7ff5ac01abd0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:52.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.390 7ff5d1f01700 1 --2- 172.21.2.101:0/3048562861 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5cc190920 0x7ff5cc0b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:52.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.390 7ff5d1f01700 1 -- 172.21.2.101:0/3048562861 >> 172.21.2.101:0/3048562861 conn(0x7ff5cc094e10 msgr2=0x7ff5cc0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:52.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.390 7ff5d1f01700 1 -- 172.21.2.101:0/3048562861 shutdown_connections 2021-02-10T11:22:52.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.390 7ff5d1f01700 1 -- 172.21.2.101:0/3048562861 wait complete. 2021-02-10T11:22:52.406 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839289 2021-02-10T11:22:52.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.814 7ff08de62700 1 Processor -- start 2021-02-10T11:22:52.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.814 7ff08de62700 1 -- start start 2021-02-10T11:22:52.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.815 7ff08de62700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff0881b0470 0x7ff0881b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:52.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.815 7ff08de62700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff0880ef610 0x7ff0880efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:52.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.815 7ff08de62700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0880f0b50 con 0x7ff0880f0030 2021-02-10T11:22:52.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.815 7ff08de62700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0880e52e0 con 0x7ff0881b0470 2021-02-10T11:22:52.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.815 7ff08de62700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0880e5490 con 0x7ff0880ef610 2021-02-10T11:22:52.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.815 7ff0877fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff0881b0470 0x7ff0881b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:52.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.815 7ff087fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff0880f0030 0x7ff0880f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47594/0 (socket says 172.21.2.101:47594) 2021-02-10T11:22:52.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.815 7ff087fff700 1 -- 172.21.2.101:0/2986533818 learned_addr learned my addr 172.21.2.101:0/2986533818 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:22:52.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.815 7ff0877fe700 1 -- 172.21.2.101:0/2986533818 >> v2:172.21.2.108:3300/0 conn(0x7ff0880ef610 msgr2=0x7ff0880efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:22:52.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.815 7ff0877fe700 1 --2- 172.21.2.101:0/2986533818 >> v2:172.21.2.108:3300/0 conn(0x7ff0880ef610 0x7ff0880efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:52.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.815 7ff0877fe700 1 -- 172.21.2.101:0/2986533818 >> v1:172.21.2.101:6789/0 conn(0x7ff0880f0030 legacy=0x7ff0880f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:52.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.815 7ff0877fe700 1 -- 172.21.2.101:0/2986533818 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0880f0430 con 0x7ff0881b0470 2021-02-10T11:22:52.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.816 7ff0877fe700 1 --2- 172.21.2.101:0/2986533818 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0881b0470 0x7ff0881b0870 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7ff078006650 tx=0x7ff078009a80).ready entity=mon.0 client_cookie=c8561a9bf884e314 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:52.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.816 7ff0867fc700 1 -- 172.21.2.101:0/2986533818 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0780009b0 con 0x7ff0881b0470 2021-02-10T11:22:52.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.816 7ff0867fc700 1 -- 172.21.2.101:0/2986533818 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07800c0c0 con 0x7ff0881b0470 2021-02-10T11:22:52.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.816 7ff0867fc700 1 -- 172.21.2.101:0/2986533818 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff07800c510 con 0x7ff0881b0470 2021-02-10T11:22:52.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.816 7ff08de62700 1 -- 172.21.2.101:0/2986533818 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0881b0470 msgr2=0x7ff0881b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:52.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.816 7ff08de62700 1 --2- 172.21.2.101:0/2986533818 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0881b0470 0x7ff0881b0870 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7ff078006650 tx=0x7ff078009a80).stop 2021-02-10T11:22:52.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.817 7ff08de62700 1 -- 172.21.2.101:0/2986533818 shutdown_connections 2021-02-10T11:22:52.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.817 7ff08de62700 1 --2- 172.21.2.101:0/2986533818 >> v2:172.21.2.108:3300/0 conn(0x7ff0880ef610 0x7ff0880efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:52.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.817 7ff08de62700 1 --2- 172.21.2.101:0/2986533818 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0881b0470 0x7ff0881b0870 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:52.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.817 7ff08de62700 1 -- 172.21.2.101:0/2986533818 >> 172.21.2.101:0/2986533818 conn(0x7ff0880ce3e0 msgr2=0x7ff0880e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:52.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.817 7ff08de62700 1 -- 172.21.2.101:0/2986533818 shutdown_connections 2021-02-10T11:22:52.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.817 7ff08de62700 1 -- 172.21.2.101:0/2986533818 wait complete. 2021-02-10T11:22:52.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.831 7ff08de62700 1 Processor -- start 2021-02-10T11:22:52.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.831 7ff08de62700 1 -- start start 2021-02-10T11:22:52.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.832 7ff08de62700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0880a6c90 0x7ff0881924c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:52.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.832 7ff08de62700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff088192ae0 0x7ff088192ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:52.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.832 7ff08de62700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0880919e0 0x7ff088091de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:52.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.832 7ff08de62700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08819d2e0 con 0x7ff0880a6c90 2021-02-10T11:22:52.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.832 7ff08de62700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0880914d0 con 0x7ff088192ae0 2021-02-10T11:22:52.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.832 7ff08de62700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0880a6820 con 0x7ff0880919e0 2021-02-10T11:22:52.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.832 7ff0877fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0880a6c90 0x7ff0881924c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:52.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.832 7ff0877fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0880a6c90 0x7ff0881924c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47108/0 (socket says 172.21.2.101:47108) 2021-02-10T11:22:52.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.832 7ff0877fe700 1 -- 172.21.2.101:0/1661790356 learned_addr learned my addr 172.21.2.101:0/1661790356 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:22:52.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.832 7ff0877fe700 1 -- 172.21.2.101:0/1661790356 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0880919e0 msgr2=0x7ff088091de0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:22:52.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.832 7ff0877fe700 1 --2- 172.21.2.101:0/1661790356 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0880919e0 0x7ff088091de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:52.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.832 7ff0877fe700 1 -- 172.21.2.101:0/1661790356 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff088192ae0 msgr2=0x7ff088192ee0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:22:52.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.832 7ff0877fe700 1 --2- 172.21.2.101:0/1661790356 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff088192ae0 0x7ff088192ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:52.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.832 7ff0877fe700 1 -- 172.21.2.101:0/1661790356 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff088092400 con 0x7ff0880a6c90 2021-02-10T11:22:52.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.832 7ff0877fe700 1 --2- 172.21.2.101:0/1661790356 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0880a6c90 0x7ff0881924c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7ff078009630 tx=0x7ff07800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:52.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.833 7ff07ffff700 1 -- 172.21.2.101:0/1661790356 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0780093a0 con 0x7ff0880a6c90 2021-02-10T11:22:52.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.833 7ff07ffff700 1 -- 172.21.2.101:0/1661790356 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07800bd70 con 0x7ff0880a6c90 2021-02-10T11:22:52.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.833 7ff07ffff700 1 -- 172.21.2.101:0/1661790356 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff078005560 con 0x7ff0880a6c90 2021-02-10T11:22:52.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.833 7ff08de62700 1 -- 172.21.2.101:0/1661790356 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff088092400 con 0x7ff0880a6c90 2021-02-10T11:22:52.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.833 7ff08de62700 1 -- 172.21.2.101:0/1661790356 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff088092400 con 0x7ff0880a6c90 2021-02-10T11:22:52.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.833 7ff07ffff700 1 -- 172.21.2.101:0/1661790356 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff0780054f0 con 0x7ff0880a6c90 2021-02-10T11:22:52.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.833 7ff07ffff700 1 --2- 172.21.2.101:0/1661790356 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff06c0186c0 0x7ff06c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:22:52.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.833 7ff07ffff700 1 -- 172.21.2.101:0/1661790356 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff078032430 con 0x7ff0880a6c90 2021-02-10T11:22:52.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.834 7ff086ffd700 1 --2- 172.21.2.101:0/1661790356 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff06c0186c0 0x7ff06c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:22:52.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.834 7ff086ffd700 1 --2- 172.21.2.101:0/1661790356 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff06c0186c0 0x7ff06c01abc0 crc :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:22:52.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.864 7ff08de62700 1 -- 172.21.2.101:0/1661790356 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0880e52e0 con 0x7ff0880a6c90 2021-02-10T11:22:52.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.867 7ff07ffff700 1 -- 172.21.2.101:0/1661790356 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff0780052e0 con 0x7ff0880a6c90 2021-02-10T11:22:53.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.996 7ff08de62700 1 -- 172.21.2.101:0/1661790356 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839289", "target": ["mgr", ""]}) v1 -- 0x7ff0880e52e0 con 0x7ff06c0186c0 2021-02-10T11:22:53.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:52.999 7ff07ffff700 1 -- 172.21.2.101:0/1661790356 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7ff070006960 con 0x7ff06c0186c0 2021-02-10T11:22:53.010 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839289/a9fedcf8-b5c0-49b8-a018-e7a5d33b58c2 2021-02-10T11:22:53.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:53.005 7ff08de62700 1 -- 172.21.2.101:0/1661790356 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff06c0186c0 msgr2=0x7ff06c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:53.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:53.005 7ff08de62700 1 --2- 172.21.2.101:0/1661790356 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff06c0186c0 0x7ff06c01abc0 crc :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:53.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:53.005 7ff08de62700 1 -- 172.21.2.101:0/1661790356 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0880a6c90 msgr2=0x7ff0881924c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:22:53.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:53.005 7ff08de62700 1 --2- 172.21.2.101:0/1661790356 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0880a6c90 0x7ff0881924c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7ff078009630 tx=0x7ff07800b980).stop 2021-02-10T11:22:53.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:53.005 7ff08de62700 1 -- 172.21.2.101:0/1661790356 shutdown_connections 2021-02-10T11:22:53.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:53.005 7ff08de62700 1 --2- 172.21.2.101:0/1661790356 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff06c0186c0 0x7ff06c01abc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:53.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:53.005 7ff08de62700 1 --2- 172.21.2.101:0/1661790356 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0880919e0 0x7ff088091de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:53.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:53.005 7ff08de62700 1 --2- 172.21.2.101:0/1661790356 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff088192ae0 0x7ff088192ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:22:53.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:53.005 7ff08de62700 1 --2- 172.21.2.101:0/1661790356 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0880a6c90 0x7ff0881924c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:22:53.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:53.005 7ff08de62700 1 -- 172.21.2.101:0/1661790356 >> 172.21.2.101:0/1661790356 conn(0x7ff0880ac550 msgr2=0x7ff0880be1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:22:53.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:53.005 7ff08de62700 1 -- 172.21.2.101:0/1661790356 shutdown_connections 2021-02-10T11:22:53.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:22:53.005 7ff08de62700 1 -- 172.21.2.101:0/1661790356 wait complete. 2021-02-10T11:22:53.021 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839289 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839289/a9fedcf8-b5c0-49b8-a018-e7a5d33b58c2 2021-02-10T11:22:53.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:53.021 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:53.021 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:53.358 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:53.358 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:53.358 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839289/a9fedcf8-b5c0-49b8-a018-e7a5d33b58c2/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:53.743 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:53.743 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:53.743 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0256115 s, 40.9 MB/s 2021-02-10T11:22:53.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:53.744 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:53.744 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:54.080 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:54.080 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:54.080 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839289/a9fedcf8-b5c0-49b8-a018-e7a5d33b58c2/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:54.468 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:54.468 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:54.468 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248901 s, 42.1 MB/s 2021-02-10T11:22:54.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:54.469 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:54.469 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:54.805 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:54.805 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:54.805 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839289/a9fedcf8-b5c0-49b8-a018-e7a5d33b58c2/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:55.181 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:55.181 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:55.181 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0226886 s, 46.2 MB/s 2021-02-10T11:22:55.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:55.182 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:55.182 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:55.519 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:55.519 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:55.519 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839289/a9fedcf8-b5c0-49b8-a018-e7a5d33b58c2/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:55.893 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:55.894 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:55.894 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239273 s, 43.8 MB/s 2021-02-10T11:22:55.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:55.895 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:55.895 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:56.231 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:56.231 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:56.231 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839289/a9fedcf8-b5c0-49b8-a018-e7a5d33b58c2/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:56.604 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:56.604 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:56.604 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0227979 s, 46.0 MB/s 2021-02-10T11:22:56.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:56.605 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:56.605 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:56.939 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:56.940 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:56.940 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839289/a9fedcf8-b5c0-49b8-a018-e7a5d33b58c2/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:57.312 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:57.313 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:57.313 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0221461 s, 47.3 MB/s 2021-02-10T11:22:57.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:57.314 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:57.314 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:57.734 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:57.734 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:57.734 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839289/a9fedcf8-b5c0-49b8-a018-e7a5d33b58c2/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:58.129 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:58.130 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:58.130 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0227509 s, 46.1 MB/s 2021-02-10T11:22:58.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:58.131 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:58.131 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:58.478 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:58.478 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:58.478 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839289/a9fedcf8-b5c0-49b8-a018-e7a5d33b58c2/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:58.860 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:58.861 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:58.861 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242666 s, 43.2 MB/s 2021-02-10T11:22:58.862 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:58.862 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:58.862 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:59.201 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:59.201 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:59.201 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839289/a9fedcf8-b5c0-49b8-a018-e7a5d33b58c2/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:22:59.577 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:22:59.577 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:22:59.577 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246468 s, 42.5 MB/s 2021-02-10T11:22:59.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:22:59.578 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:22:59.578 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:22:59.913 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:22:59.913 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:22:59.914 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839289/a9fedcf8-b5c0-49b8-a018-e7a5d33b58c2/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:00.291 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:00.291 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:00.291 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.025512 s, 41.1 MB/s 2021-02-10T11:23:00.292 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839290 2021-02-10T11:23:00.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.699 7fc1f1bd5700 1 Processor -- start 2021-02-10T11:23:00.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.699 7fc1f1bd5700 1 -- start start 2021-02-10T11:23:00.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.700 7fc1f1bd5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc1ec1b0470 0x7fc1ec1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:00.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.700 7fc1f1bd5700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc1ec0ef610 0x7fc1ec0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:00.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.700 7fc1f1bd5700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1ec0f0b50 con 0x7fc1ec0f0030 2021-02-10T11:23:00.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.700 7fc1f1bd5700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1ec0e52e0 con 0x7fc1ec0ef610 2021-02-10T11:23:00.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.700 7fc1f1bd5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec0e5490 con 0x7fc1ec1b0470 2021-02-10T11:23:00.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.700 7fc1ebfff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc1ec0f0030 0x7fc1ec0f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47606/0 (socket says 172.21.2.101:47606) 2021-02-10T11:23:00.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.700 7fc1ebfff700 1 -- 172.21.2.101:0/496018509 learned_addr learned my addr 172.21.2.101:0/496018509 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:23:00.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.700 7fc1eaffd700 1 --2- 172.21.2.101:0/496018509 >> v2:172.21.2.101:3300/0 conn(0x7fc1ec0ef610 0x7fc1ec0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:00.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.700 7fc1ea7fc700 1 -- 172.21.2.101:0/496018509 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198085961 0 0) 0x7fc1d4001180 con 0x7fc1ec0f0030 2021-02-10T11:23:00.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.700 7fc1eaffd700 1 -- 172.21.2.101:0/496018509 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc1ec1b0470 msgr2=0x7fc1ec1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:23:00.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.700 7fc1eaffd700 1 --2- 172.21.2.101:0/496018509 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc1ec1b0470 0x7fc1ec1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:00.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.700 7fc1eaffd700 1 -- 172.21.2.101:0/496018509 >> v1:172.21.2.101:6789/0 conn(0x7fc1ec0f0030 legacy=0x7fc1ec0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:00.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.700 7fc1eaffd700 1 -- 172.21.2.101:0/496018509 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ec0f0430 con 0x7fc1ec0ef610 2021-02-10T11:23:00.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.701 7fc1eaffd700 1 --2- 172.21.2.101:0/496018509 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1ec0ef610 0x7fc1ec0efa10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fc1dc0069d0 tx=0x7fc1dc009e40).ready entity=mon.0 client_cookie=392748a51f886e44 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:00.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.701 7fc1ea7fc700 1 -- 172.21.2.101:0/496018509 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc1dc00b510 con 0x7fc1ec0ef610 2021-02-10T11:23:00.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.701 7fc1ea7fc700 1 -- 172.21.2.101:0/496018509 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1dc00b890 con 0x7fc1ec0ef610 2021-02-10T11:23:00.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.701 7fc1ea7fc700 1 -- 172.21.2.101:0/496018509 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc1dc00bdb0 con 0x7fc1ec0ef610 2021-02-10T11:23:00.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.701 7fc1f1bd5700 1 -- 172.21.2.101:0/496018509 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1ec0ef610 msgr2=0x7fc1ec0efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:00.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.701 7fc1f1bd5700 1 --2- 172.21.2.101:0/496018509 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1ec0ef610 0x7fc1ec0efa10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fc1dc0069d0 tx=0x7fc1dc009e40).stop 2021-02-10T11:23:00.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.701 7fc1f1bd5700 1 -- 172.21.2.101:0/496018509 shutdown_connections 2021-02-10T11:23:00.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.701 7fc1f1bd5700 1 --2- 172.21.2.101:0/496018509 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1ec0ef610 0x7fc1ec0efa10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:00.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.701 7fc1f1bd5700 1 --2- 172.21.2.101:0/496018509 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc1ec1b0470 0x7fc1ec1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:00.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.701 7fc1f1bd5700 1 -- 172.21.2.101:0/496018509 >> 172.21.2.101:0/496018509 conn(0x7fc1ec0ce3e0 msgr2=0x7fc1ec0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:00.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.702 7fc1f1bd5700 1 -- 172.21.2.101:0/496018509 shutdown_connections 2021-02-10T11:23:00.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.702 7fc1f1bd5700 1 -- 172.21.2.101:0/496018509 wait complete. 2021-02-10T11:23:00.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.716 7fc1f1bd5700 1 Processor -- start 2021-02-10T11:23:00.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.716 7fc1f1bd5700 1 -- start start 2021-02-10T11:23:00.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.717 7fc1f1bd5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc1ec0b20b0 0x7fc1ec0ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:00.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.717 7fc1f1bd5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1ec0b24b0 0x7fc1ec191dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:00.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.717 7fc1f1bd5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc1ec0acf90 0x7fc1ec1923e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:00.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.717 7fc1f1bd5700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec0b1b30 con 0x7fc1ec0b24b0 2021-02-10T11:23:00.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.717 7fc1f1bd5700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec1a04e0 con 0x7fc1ec0acf90 2021-02-10T11:23:00.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.717 7fc1f1bd5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec1a06c0 con 0x7fc1ec0b20b0 2021-02-10T11:23:00.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.717 7fc1eaffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1ec0b24b0 0x7fc1ec191dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.717 7fc1eaffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1ec0b24b0 0x7fc1ec191dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47120/0 (socket says 172.21.2.101:47120) 2021-02-10T11:23:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.717 7fc1eaffd700 1 -- 172.21.2.101:0/3713365331 learned_addr learned my addr 172.21.2.101:0/3713365331 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.717 7fc1eaffd700 1 -- 172.21.2.101:0/3713365331 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc1ec0b20b0 msgr2=0x7fc1ec0ac970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:23:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.717 7fc1eaffd700 1 --2- 172.21.2.101:0/3713365331 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc1ec0b20b0 0x7fc1ec0ac970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.717 7fc1eaffd700 1 -- 172.21.2.101:0/3713365331 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc1ec0acf90 msgr2=0x7fc1ec1923e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:23:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.717 7fc1eaffd700 1 --2- 172.21.2.101:0/3713365331 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc1ec0acf90 0x7fc1ec1923e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.717 7fc1eaffd700 1 -- 172.21.2.101:0/3713365331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ec192be0 con 0x7fc1ec0b24b0 2021-02-10T11:23:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.717 7fc1eaffd700 1 --2- 172.21.2.101:0/3713365331 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1ec0b24b0 0x7fc1ec191dc0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7fc1dc0099e0 tx=0x7fc1dc00bd20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.717 7fc1e3fff700 1 -- 172.21.2.101:0/3713365331 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc1dc006240 con 0x7fc1ec0b24b0 2021-02-10T11:23:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.717 7fc1e3fff700 1 -- 172.21.2.101:0/3713365331 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1dc00be60 con 0x7fc1ec0b24b0 2021-02-10T11:23:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.718 7fc1e3fff700 1 -- 172.21.2.101:0/3713365331 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc1dc006630 con 0x7fc1ec0b24b0 2021-02-10T11:23:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.718 7fc1f1bd5700 1 -- 172.21.2.101:0/3713365331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1ec192be0 con 0x7fc1ec0b24b0 2021-02-10T11:23:00.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.718 7fc1f1bd5700 1 -- 172.21.2.101:0/3713365331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ec193170 con 0x7fc1ec0b24b0 2021-02-10T11:23:00.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.718 7fc1e3fff700 1 -- 172.21.2.101:0/3713365331 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc1dc005870 con 0x7fc1ec0b24b0 2021-02-10T11:23:00.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.718 7fc1e3fff700 1 --2- 172.21.2.101:0/3713365331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1d00186c0 0x7fc1d001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:00.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.718 7fc1e3fff700 1 -- 172.21.2.101:0/3713365331 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc1dc0328d0 con 0x7fc1ec0b24b0 2021-02-10T11:23:00.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.718 7fc1eb7fe700 1 --2- 172.21.2.101:0/3713365331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1d00186c0 0x7fc1d001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:00.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.719 7fc1eb7fe700 1 --2- 172.21.2.101:0/3713365331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1d00186c0 0x7fc1d001abc0 crc :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:00.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.750 7fc1f1bd5700 1 -- 172.21.2.101:0/3713365331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1ec105af0 con 0x7fc1ec0b24b0 2021-02-10T11:23:00.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.753 7fc1e3fff700 1 -- 172.21.2.101:0/3713365331 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc1dc01f8b0 con 0x7fc1ec0b24b0 2021-02-10T11:23:00.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.879 7fc1f1bd5700 1 -- 172.21.2.101:0/3713365331 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839290", "target": ["mgr", ""]}) v1 -- 0x7fc1ec1a06c0 con 0x7fc1d00186c0 2021-02-10T11:23:00.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.905 7fc1e3fff700 1 -- 172.21.2.101:0/3713365331 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fc1e4006960 con 0x7fc1d00186c0 2021-02-10T11:23:00.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.911 7fc1f1bd5700 1 -- 172.21.2.101:0/3713365331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1d00186c0 msgr2=0x7fc1d001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:00.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.911 7fc1f1bd5700 1 --2- 172.21.2.101:0/3713365331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1d00186c0 0x7fc1d001abc0 crc :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:00.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.911 7fc1f1bd5700 1 -- 172.21.2.101:0/3713365331 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1ec0b24b0 msgr2=0x7fc1ec191dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:00.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.911 7fc1f1bd5700 1 --2- 172.21.2.101:0/3713365331 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1ec0b24b0 0x7fc1ec191dc0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7fc1dc0099e0 tx=0x7fc1dc00bd20).stop 2021-02-10T11:23:00.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.912 7fc1f1bd5700 1 -- 172.21.2.101:0/3713365331 shutdown_connections 2021-02-10T11:23:00.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.912 7fc1f1bd5700 1 --2- 172.21.2.101:0/3713365331 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc1ec0acf90 0x7fc1ec1923e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:00.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.912 7fc1f1bd5700 1 --2- 172.21.2.101:0/3713365331 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1ec0b24b0 0x7fc1ec191dc0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:00.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.912 7fc1f1bd5700 1 --2- 172.21.2.101:0/3713365331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1d00186c0 0x7fc1d001abc0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:00.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.912 7fc1f1bd5700 1 --2- 172.21.2.101:0/3713365331 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc1ec0b20b0 0x7fc1ec0ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:00.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.912 7fc1f1bd5700 1 -- 172.21.2.101:0/3713365331 >> 172.21.2.101:0/3713365331 conn(0x7fc1ec0bf4e0 msgr2=0x7fc1ec0be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:00.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.912 7fc1f1bd5700 1 -- 172.21.2.101:0/3713365331 shutdown_connections 2021-02-10T11:23:00.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:00.912 7fc1f1bd5700 1 -- 172.21.2.101:0/3713365331 wait complete. 2021-02-10T11:23:00.927 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839290 2021-02-10T11:23:01.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.334 7f675d930700 1 Processor -- start 2021-02-10T11:23:01.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.334 7f675d930700 1 -- start start 2021-02-10T11:23:01.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.334 7f675d930700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f67581b0870 0x7f67580f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:01.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.334 7f675d930700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f67580efe00 0x7f67580f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:01.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.334 7f675d930700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67581b0c70 con 0x7f67581b0470 2021-02-10T11:23:01.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.334 7f675d930700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67580e5ea0 con 0x7f67580efe00 2021-02-10T11:23:01.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.334 7f675d930700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67580f0b50 con 0x7f67581b0870 2021-02-10T11:23:01.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.334 7f6757d75700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f67580efe00 0x7f67580f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:01.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.334 7f6757d75700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f67580efe00 0x7f67580f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47124/0 (socket says 172.21.2.101:47124) 2021-02-10T11:23:01.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.334 7f6757d75700 1 -- 172.21.2.101:0/3795580525 learned_addr learned my addr 172.21.2.101:0/3795580525 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:01.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.335 7f6757d75700 1 -- 172.21.2.101:0/3795580525 >> v2:172.21.2.108:3300/0 conn(0x7f67581b0870 msgr2=0x7f67580f20e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:23:01.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.335 7f6757d75700 1 --2- 172.21.2.101:0/3795580525 >> v2:172.21.2.108:3300/0 conn(0x7f67581b0870 0x7f67580f20e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:01.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.335 7f6757d75700 1 -- 172.21.2.101:0/3795580525 >> v1:172.21.2.101:6789/0 conn(0x7f67581b0470 legacy=0x7f67580ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:01.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.335 7f6757d75700 1 -- 172.21.2.101:0/3795580525 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67580f0820 con 0x7f67580efe00 2021-02-10T11:23:01.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.335 7f6757d75700 1 --2- 172.21.2.101:0/3795580525 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67580efe00 0x7f67580f0200 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f6750006f00 tx=0x7f6750009f10).ready entity=mon.0 client_cookie=8be42b23b209fdcc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:01.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.335 7f6756572700 1 -- 172.21.2.101:0/3795580525 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f675000b3b0 con 0x7f67580efe00 2021-02-10T11:23:01.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.335 7f6756572700 1 -- 172.21.2.101:0/3795580525 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675000b630 con 0x7f67580efe00 2021-02-10T11:23:01.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.335 7f6756572700 1 -- 172.21.2.101:0/3795580525 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f675000bb50 con 0x7f67580efe00 2021-02-10T11:23:01.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.335 7f675d930700 1 -- 172.21.2.101:0/3795580525 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67580efe00 msgr2=0x7f67580f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:01.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.335 7f675d930700 1 --2- 172.21.2.101:0/3795580525 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67580efe00 0x7f67580f0200 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f6750006f00 tx=0x7f6750009f10).stop 2021-02-10T11:23:01.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.335 7f675d930700 1 -- 172.21.2.101:0/3795580525 shutdown_connections 2021-02-10T11:23:01.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.335 7f675d930700 1 --2- 172.21.2.101:0/3795580525 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67580efe00 0x7f67580f0200 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:01.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.335 7f675d930700 1 --2- 172.21.2.101:0/3795580525 >> v2:172.21.2.108:3300/0 conn(0x7f67581b0870 0x7f67580f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:01.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.335 7f675d930700 1 -- 172.21.2.101:0/3795580525 >> 172.21.2.101:0/3795580525 conn(0x7f67580ce3e0 msgr2=0x7f67580e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:01.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.336 7f675d930700 1 -- 172.21.2.101:0/3795580525 shutdown_connections 2021-02-10T11:23:01.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.336 7f675d930700 1 -- 172.21.2.101:0/3795580525 wait complete. 2021-02-10T11:23:01.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.350 7f675d930700 1 Processor -- start 2021-02-10T11:23:01.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.350 7f675d930700 1 -- start start 2021-02-10T11:23:01.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.350 7f675d930700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f67580a9690 0x7f67580a9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:01.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.350 7f675d930700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67580aa0b0 0x7f67580aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:01.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.350 7f675d930700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67580f1800 0x7f67580f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:01.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.350 7f675d930700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6758092350 con 0x7f67580f1800 2021-02-10T11:23:01.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.350 7f675d930700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67580a6540 con 0x7f67580a9690 2021-02-10T11:23:01.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.350 7f675d930700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67580a6720 con 0x7f67580aa0b0 2021-02-10T11:23:01.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.351 7f6757d75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67580f1800 0x7f67580f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:01.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.351 7f6757d75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67580f1800 0x7f67580f1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47128/0 (socket says 172.21.2.101:47128) 2021-02-10T11:23:01.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.351 7f6757d75700 1 -- 172.21.2.101:0/2080200666 learned_addr learned my addr 172.21.2.101:0/2080200666 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:01.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.351 7f6757d75700 1 -- 172.21.2.101:0/2080200666 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67580aa0b0 msgr2=0x7f67580aa4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:23:01.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.351 7f6757d75700 1 --2- 172.21.2.101:0/2080200666 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67580aa0b0 0x7f67580aa4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:01.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.351 7f6757d75700 1 -- 172.21.2.101:0/2080200666 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f67580a9690 msgr2=0x7f67580a9a90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:23:01.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.351 7f6757d75700 1 --2- 172.21.2.101:0/2080200666 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f67580a9690 0x7f67580a9a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:01.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.351 7f6757d75700 1 -- 172.21.2.101:0/2080200666 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67580a69e0 con 0x7f67580f1800 2021-02-10T11:23:01.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.351 7f6757d75700 1 --2- 172.21.2.101:0/2080200666 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67580f1800 0x7f67580f1c00 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f675000b9b0 tx=0x7f675000bd50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:01.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.351 7f674ffff700 1 -- 172.21.2.101:0/2080200666 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6750003030 con 0x7f67580f1800 2021-02-10T11:23:01.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.351 7f674ffff700 1 -- 172.21.2.101:0/2080200666 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675000c2f0 con 0x7f67580f1800 2021-02-10T11:23:01.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.351 7f675d930700 1 -- 172.21.2.101:0/2080200666 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67580a69e0 con 0x7f67580f1800 2021-02-10T11:23:01.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.351 7f674ffff700 1 -- 172.21.2.101:0/2080200666 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f675000c7d0 con 0x7f67580f1800 2021-02-10T11:23:01.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.351 7f675d930700 1 -- 172.21.2.101:0/2080200666 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67580a69e0 con 0x7f67580f1800 2021-02-10T11:23:01.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.352 7f674ffff700 1 -- 172.21.2.101:0/2080200666 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6750001270 con 0x7f67580f1800 2021-02-10T11:23:01.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.352 7f674ffff700 1 --2- 172.21.2.101:0/2080200666 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f673c018730 0x7f673c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:01.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.352 7f674ffff700 1 -- 172.21.2.101:0/2080200666 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6750032b00 con 0x7f67580f1800 2021-02-10T11:23:01.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.352 7f6757574700 1 --2- 172.21.2.101:0/2080200666 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f673c018730 0x7f673c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:01.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.352 7f6757574700 1 --2- 172.21.2.101:0/2080200666 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f673c018730 0x7f673c01ac10 crc :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:01.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.384 7f675d930700 1 -- 172.21.2.101:0/2080200666 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67580edd10 con 0x7f67580f1800 2021-02-10T11:23:01.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.387 7f674ffff700 1 -- 172.21.2.101:0/2080200666 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6750000f90 con 0x7f67580f1800 2021-02-10T11:23:01.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.514 7f675d930700 1 -- 172.21.2.101:0/2080200666 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839290", "target": ["mgr", ""]}) v1 -- 0x7f67580e52e0 con 0x7f673c018730 2021-02-10T11:23:01.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.517 7f674ffff700 1 -- 172.21.2.101:0/2080200666 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f6740006980 con 0x7f673c018730 2021-02-10T11:23:01.529 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839290/a8a549a6-2b77-48ba-8527-83ce874e08d2 2021-02-10T11:23:01.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.522 7f675d930700 1 -- 172.21.2.101:0/2080200666 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f673c018730 msgr2=0x7f673c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:01.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.522 7f675d930700 1 --2- 172.21.2.101:0/2080200666 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f673c018730 0x7f673c01ac10 crc :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:01.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.522 7f675d930700 1 -- 172.21.2.101:0/2080200666 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67580f1800 msgr2=0x7f67580f1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:01.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.522 7f675d930700 1 --2- 172.21.2.101:0/2080200666 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67580f1800 0x7f67580f1c00 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f675000b9b0 tx=0x7f675000bd50).stop 2021-02-10T11:23:01.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.522 7f675d930700 1 -- 172.21.2.101:0/2080200666 shutdown_connections 2021-02-10T11:23:01.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.522 7f675d930700 1 --2- 172.21.2.101:0/2080200666 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f673c018730 0x7f673c01ac10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:01.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.522 7f675d930700 1 --2- 172.21.2.101:0/2080200666 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67580aa0b0 0x7f67580aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:01.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.522 7f675d930700 1 --2- 172.21.2.101:0/2080200666 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67580f1800 0x7f67580f1c00 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:01.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.522 7f675d930700 1 --2- 172.21.2.101:0/2080200666 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f67580a9690 0x7f67580a9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:01.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.522 7f675d930700 1 -- 172.21.2.101:0/2080200666 >> 172.21.2.101:0/2080200666 conn(0x7f67580c40e0 msgr2=0x7f6758094eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:01.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.523 7f675d930700 1 -- 172.21.2.101:0/2080200666 shutdown_connections 2021-02-10T11:23:01.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:01.523 7f675d930700 1 -- 172.21.2.101:0/2080200666 wait complete. 2021-02-10T11:23:01.538 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839290 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839290/a8a549a6-2b77-48ba-8527-83ce874e08d2 2021-02-10T11:23:01.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:01.539 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:01.539 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:01.875 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:01.875 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:01.875 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839290/a8a549a6-2b77-48ba-8527-83ce874e08d2/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:02.258 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:02.259 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:02.259 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248075 s, 42.3 MB/s 2021-02-10T11:23:02.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:02.260 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:02.260 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:02.595 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:02.595 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:02.595 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839290/a8a549a6-2b77-48ba-8527-83ce874e08d2/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:02.969 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:02.970 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:02.970 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.022734 s, 46.1 MB/s 2021-02-10T11:23:02.971 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:02.971 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:02.971 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:03.306 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:03.306 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:03.306 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839290/a8a549a6-2b77-48ba-8527-83ce874e08d2/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:03.681 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:03.681 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:03.681 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.022934 s, 45.7 MB/s 2021-02-10T11:23:03.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:03.683 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:03.683 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:04.019 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:04.019 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:04.019 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839290/a8a549a6-2b77-48ba-8527-83ce874e08d2/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:04.391 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:04.392 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:04.392 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234473 s, 44.7 MB/s 2021-02-10T11:23:04.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:04.393 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:04.393 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:04.727 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:04.728 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:04.728 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839290/a8a549a6-2b77-48ba-8527-83ce874e08d2/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:05.103 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:05.104 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:05.104 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245591 s, 42.7 MB/s 2021-02-10T11:23:05.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:05.105 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:05.105 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:05.441 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:05.441 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:05.441 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839290/a8a549a6-2b77-48ba-8527-83ce874e08d2/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:05.816 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:05.817 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:05.817 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243987 s, 43.0 MB/s 2021-02-10T11:23:05.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:05.818 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:05.818 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:06.154 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:06.154 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:06.154 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839290/a8a549a6-2b77-48ba-8527-83ce874e08d2/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:06.528 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:06.528 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:06.528 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.022299 s, 47.0 MB/s 2021-02-10T11:23:06.529 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:06.529 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:06.529 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:06.864 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:06.865 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:06.865 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839290/a8a549a6-2b77-48ba-8527-83ce874e08d2/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:07.242 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:07.243 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:07.243 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0261919 s, 40.0 MB/s 2021-02-10T11:23:07.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:07.244 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:07.244 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:07.579 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:07.579 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:07.580 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839290/a8a549a6-2b77-48ba-8527-83ce874e08d2/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:07.960 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:07.960 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:07.960 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0254524 s, 41.2 MB/s 2021-02-10T11:23:07.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:07.961 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:07.961 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:08.296 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:08.297 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:08.297 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839290/a8a549a6-2b77-48ba-8527-83ce874e08d2/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:08.673 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:08.673 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:08.673 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.02405 s, 43.6 MB/s 2021-02-10T11:23:08.674 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839291 2021-02-10T11:23:09.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.079 7f75ad007700 1 Processor -- start 2021-02-10T11:23:09.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.079 7f75ad007700 1 -- start start 2021-02-10T11:23:09.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.080 7f75ad007700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f75a81b0870 0x7f75a80f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:09.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.080 7f75ad007700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a81b0c70 con 0x7f75a80efe00 2021-02-10T11:23:09.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.080 7f75ad007700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a80f0b50 con 0x7f75a81b0470 2021-02-10T11:23:09.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.080 7f75ad007700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75a80e5ea0 con 0x7f75a81b0870 2021-02-10T11:23:09.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.080 7f75a659c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f75a81b0870 0x7f75a80f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:09.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.080 7f75a659c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f75a81b0870 0x7f75a80f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47136/0 (socket says 172.21.2.101:47136) 2021-02-10T11:23:09.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.080 7f75a759e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f75a80efe00 0x7f75a80f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47632/0 (socket says 172.21.2.101:47632) 2021-02-10T11:23:09.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.080 7f75a659c700 1 -- 172.21.2.101:0/2141865276 learned_addr learned my addr 172.21.2.101:0/2141865276 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:09.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.080 7f75a659c700 1 -- 172.21.2.101:0/2141865276 >> v1:172.21.2.108:6789/0 conn(0x7f75a81b0470 legacy=0x7f75a80ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:09.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.080 7f75a659c700 1 -- 172.21.2.101:0/2141865276 >> v1:172.21.2.101:6789/0 conn(0x7f75a80efe00 legacy=0x7f75a80f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:09.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.080 7f75a659c700 1 -- 172.21.2.101:0/2141865276 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a80e52e0 con 0x7f75a81b0870 2021-02-10T11:23:09.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.080 7f75a659c700 1 --2- 172.21.2.101:0/2141865276 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75a81b0870 0x7f75a80f20e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f7598006700 tx=0x7f7598009b80).ready entity=mon.0 client_cookie=90d3b889b4a41458 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:09.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.081 7f75a5d9b700 1 -- 172.21.2.101:0/2141865276 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f759800c0c0 con 0x7f75a81b0870 2021-02-10T11:23:09.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.081 7f75a5d9b700 1 -- 172.21.2.101:0/2141865276 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759800c340 con 0x7f75a81b0870 2021-02-10T11:23:09.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.081 7f75a5d9b700 1 -- 172.21.2.101:0/2141865276 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f759800c860 con 0x7f75a81b0870 2021-02-10T11:23:09.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.081 7f75ad007700 1 -- 172.21.2.101:0/2141865276 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75a81b0870 msgr2=0x7f75a80f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:09.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.081 7f75ad007700 1 --2- 172.21.2.101:0/2141865276 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75a81b0870 0x7f75a80f20e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f7598006700 tx=0x7f7598009b80).stop 2021-02-10T11:23:09.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.081 7f75ad007700 1 -- 172.21.2.101:0/2141865276 shutdown_connections 2021-02-10T11:23:09.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.081 7f75ad007700 1 --2- 172.21.2.101:0/2141865276 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75a81b0870 0x7f75a80f20e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:09.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.081 7f75ad007700 1 -- 172.21.2.101:0/2141865276 >> 172.21.2.101:0/2141865276 conn(0x7f75a80ce3e0 msgr2=0x7f75a80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:09.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.082 7f75ad007700 1 -- 172.21.2.101:0/2141865276 shutdown_connections 2021-02-10T11:23:09.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.082 7f75ad007700 1 -- 172.21.2.101:0/2141865276 wait complete. 2021-02-10T11:23:09.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.096 7f75ad007700 1 Processor -- start 2021-02-10T11:23:09.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.096 7f75ad007700 1 -- start start 2021-02-10T11:23:09.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.097 7f75ad007700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75a8087510 0x7f75a80aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:09.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.097 7f75ad007700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75a81c68f0 0x7f75a81b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:09.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.097 7f75ad007700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75a808c4d0 0x7f75a81b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:09.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.097 7f75ad007700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a80a2d50 con 0x7f75a81c68f0 2021-02-10T11:23:09.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.097 7f75ad007700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a80a2f70 con 0x7f75a8087510 2021-02-10T11:23:09.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.097 7f75ad007700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a80a3190 con 0x7f75a808c4d0 2021-02-10T11:23:09.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.097 7f75a659c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75a81c68f0 0x7f75a81b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:09.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.097 7f75a659c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75a81c68f0 0x7f75a81b1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47140/0 (socket says 172.21.2.101:47140) 2021-02-10T11:23:09.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.097 7f75a659c700 1 -- 172.21.2.101:0/25908946 learned_addr learned my addr 172.21.2.101:0/25908946 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:09.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.097 7f75a759e700 1 --2- 172.21.2.101:0/25908946 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75a808c4d0 0x7f75a81b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:09.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.097 7f75a6d9d700 1 --2- 172.21.2.101:0/25908946 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75a8087510 0x7f75a80aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:09.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.097 7f75a659c700 1 -- 172.21.2.101:0/25908946 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75a808c4d0 msgr2=0x7f75a81b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:09.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.097 7f75a659c700 1 --2- 172.21.2.101:0/25908946 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75a808c4d0 0x7f75a81b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:09.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.097 7f75a659c700 1 -- 172.21.2.101:0/25908946 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75a8087510 msgr2=0x7f75a80aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:09.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.097 7f75a659c700 1 --2- 172.21.2.101:0/25908946 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75a8087510 0x7f75a80aedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:09.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.097 7f75a659c700 1 -- 172.21.2.101:0/25908946 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a80a33b0 con 0x7f75a81c68f0 2021-02-10T11:23:09.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.097 7f75a659c700 1 --2- 172.21.2.101:0/25908946 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75a81c68f0 0x7f75a81b1f60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f759800b5d0 tx=0x7f759800b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:09.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.098 7f759f7fe700 1 -- 172.21.2.101:0/25908946 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f759800bd60 con 0x7f75a81c68f0 2021-02-10T11:23:09.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.098 7f759f7fe700 1 -- 172.21.2.101:0/25908946 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7598009020 con 0x7f75a81c68f0 2021-02-10T11:23:09.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.098 7f75ad007700 1 -- 172.21.2.101:0/25908946 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a80a33b0 con 0x7f75a81c68f0 2021-02-10T11:23:09.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.098 7f759f7fe700 1 -- 172.21.2.101:0/25908946 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7598002300 con 0x7f75a81c68f0 2021-02-10T11:23:09.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.098 7f75ad007700 1 -- 172.21.2.101:0/25908946 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a80a33b0 con 0x7f75a81c68f0 2021-02-10T11:23:09.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.098 7f759f7fe700 1 -- 172.21.2.101:0/25908946 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7598000c90 con 0x7f75a81c68f0 2021-02-10T11:23:09.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.098 7f759f7fe700 1 --2- 172.21.2.101:0/25908946 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f758c0186c0 0x7f758c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:09.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.098 7f759f7fe700 1 -- 172.21.2.101:0/25908946 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f75980325a0 con 0x7f75a81c68f0 2021-02-10T11:23:09.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.098 7f75a6d9d700 1 --2- 172.21.2.101:0/25908946 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f758c0186c0 0x7f758c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:09.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.099 7f75a6d9d700 1 --2- 172.21.2.101:0/25908946 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f758c0186c0 0x7f758c01abc0 crc :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:09.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.130 7f75ad007700 1 -- 172.21.2.101:0/25908946 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a807e6b0 con 0x7f75a81c68f0 2021-02-10T11:23:09.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.133 7f759f7fe700 1 -- 172.21.2.101:0/25908946 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f75980063a0 con 0x7f75a81c68f0 2021-02-10T11:23:09.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.258 7f75ad007700 1 -- 172.21.2.101:0/25908946 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839291", "target": ["mgr", ""]}) v1 -- 0x7f75a807e6b0 con 0x7f758c0186c0 2021-02-10T11:23:09.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.283 7f759f7fe700 1 -- 172.21.2.101:0/25908946 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f7590002ec0 con 0x7f758c0186c0 2021-02-10T11:23:09.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.292 7f75ad007700 1 -- 172.21.2.101:0/25908946 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f758c0186c0 msgr2=0x7f758c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:09.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.292 7f75ad007700 1 --2- 172.21.2.101:0/25908946 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f758c0186c0 0x7f758c01abc0 crc :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:09.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.292 7f75ad007700 1 -- 172.21.2.101:0/25908946 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75a81c68f0 msgr2=0x7f75a81b1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:09.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.292 7f75ad007700 1 --2- 172.21.2.101:0/25908946 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75a81c68f0 0x7f75a81b1f60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f759800b5d0 tx=0x7f759800b970).stop 2021-02-10T11:23:09.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.292 7f75ad007700 1 -- 172.21.2.101:0/25908946 shutdown_connections 2021-02-10T11:23:09.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.292 7f75ad007700 1 --2- 172.21.2.101:0/25908946 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f758c0186c0 0x7f758c01abc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:09.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.292 7f75ad007700 1 --2- 172.21.2.101:0/25908946 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75a808c4d0 0x7f75a81b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:09.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.292 7f75ad007700 1 --2- 172.21.2.101:0/25908946 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75a81c68f0 0x7f75a81b1f60 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:09.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.292 7f75ad007700 1 --2- 172.21.2.101:0/25908946 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75a8087510 0x7f75a80aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:09.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.292 7f75ad007700 1 -- 172.21.2.101:0/25908946 >> 172.21.2.101:0/25908946 conn(0x7f75a80886d0 msgr2=0x7f75a8081c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:09.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.292 7f75ad007700 1 -- 172.21.2.101:0/25908946 shutdown_connections 2021-02-10T11:23:09.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.292 7f75ad007700 1 -- 172.21.2.101:0/25908946 wait complete. 2021-02-10T11:23:09.308 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839291 2021-02-10T11:23:09.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.714 7fc05b319700 1 Processor -- start 2021-02-10T11:23:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.714 7fc05b319700 1 -- start start 2021-02-10T11:23:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.714 7fc05b319700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc0541b0470 0x7fc0541b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.714 7fc05b319700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc0540ef610 0x7fc0540efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.714 7fc05b319700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0540f0030 0x7fc0540f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.714 7fc05b319700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0540e5ea0 con 0x7fc0541b0470 2021-02-10T11:23:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.714 7fc05b319700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0540f0b50 con 0x7fc0540ef610 2021-02-10T11:23:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.714 7fc05b319700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0540e52e0 con 0x7fc0540f0030 2021-02-10T11:23:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.715 7fc059135700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc0541b0470 0x7fc0541b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.715 7fc059135700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc0541b0470 0x7fc0541b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47148/0 (socket says 172.21.2.101:47148) 2021-02-10T11:23:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.715 7fc059135700 1 -- 172.21.2.101:0/4203538705 learned_addr learned my addr 172.21.2.101:0/4203538705 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.715 7fc059135700 1 -- 172.21.2.101:0/4203538705 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0540f0030 msgr2=0x7fc0540f0430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:23:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.715 7fc059135700 1 --2- 172.21.2.101:0/4203538705 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0540f0030 0x7fc0540f0430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:09.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.715 7fc059135700 1 -- 172.21.2.101:0/4203538705 >> v2:172.21.2.108:3300/0 conn(0x7fc0540ef610 msgr2=0x7fc0540efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:23:09.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.715 7fc059135700 1 --2- 172.21.2.101:0/4203538705 >> v2:172.21.2.108:3300/0 conn(0x7fc0540ef610 0x7fc0540efa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:09.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.715 7fc059135700 1 -- 172.21.2.101:0/4203538705 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0540f10d0 con 0x7fc0541b0470 2021-02-10T11:23:09.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.715 7fc059135700 1 --2- 172.21.2.101:0/4203538705 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0541b0470 0x7fc0541b0870 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7fc048006700 tx=0x7fc048009b80).ready entity=mon.0 client_cookie=7b1b34e501f03fb7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:09.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.715 7fc047fff700 1 -- 172.21.2.101:0/4203538705 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc04800c0c0 con 0x7fc0541b0470 2021-02-10T11:23:09.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.715 7fc047fff700 1 -- 172.21.2.101:0/4203538705 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04800c440 con 0x7fc0541b0470 2021-02-10T11:23:09.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.715 7fc047fff700 1 -- 172.21.2.101:0/4203538705 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc04800c960 con 0x7fc0541b0470 2021-02-10T11:23:09.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.715 7fc05b319700 1 -- 172.21.2.101:0/4203538705 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0541b0470 msgr2=0x7fc0541b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:09.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.715 7fc05b319700 1 --2- 172.21.2.101:0/4203538705 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0541b0470 0x7fc0541b0870 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7fc048006700 tx=0x7fc048009b80).stop 2021-02-10T11:23:09.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.716 7fc05b319700 1 -- 172.21.2.101:0/4203538705 shutdown_connections 2021-02-10T11:23:09.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.716 7fc05b319700 1 --2- 172.21.2.101:0/4203538705 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0540f0030 0x7fc0540f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:09.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.716 7fc05b319700 1 --2- 172.21.2.101:0/4203538705 >> v2:172.21.2.108:3300/0 conn(0x7fc0540ef610 0x7fc0540efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:09.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.716 7fc05b319700 1 --2- 172.21.2.101:0/4203538705 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0541b0470 0x7fc0541b0870 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:09.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.716 7fc05b319700 1 -- 172.21.2.101:0/4203538705 >> 172.21.2.101:0/4203538705 conn(0x7fc0540ce3e0 msgr2=0x7fc0540e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:09.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.716 7fc05b319700 1 -- 172.21.2.101:0/4203538705 shutdown_connections 2021-02-10T11:23:09.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.716 7fc05b319700 1 -- 172.21.2.101:0/4203538705 wait complete. 2021-02-10T11:23:09.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.731 7fc05b319700 1 Processor -- start 2021-02-10T11:23:09.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.731 7fc05b319700 1 -- start start 2021-02-10T11:23:09.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.731 7fc05b319700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc054190920 0x7fc05408f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:09.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.731 7fc05b319700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc054190d20 0x7fc05408fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:09.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.731 7fc05b319700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc054193270 0x7fc054193670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:09.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.731 7fc05b319700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc054090470 con 0x7fc054190d20 2021-02-10T11:23:09.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.731 7fc05b319700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc054193c90 con 0x7fc054190920 2021-02-10T11:23:09.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.731 7fc05b319700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc054193e70 con 0x7fc054193270 2021-02-10T11:23:09.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.732 7fc058934700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc054190d20 0x7fc05408fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:09.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.732 7fc058934700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc054190d20 0x7fc05408fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47154/0 (socket says 172.21.2.101:47154) 2021-02-10T11:23:09.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.732 7fc058934700 1 -- 172.21.2.101:0/2709223441 learned_addr learned my addr 172.21.2.101:0/2709223441 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:09.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.732 7fc058934700 1 -- 172.21.2.101:0/2709223441 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc054193270 msgr2=0x7fc054193670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:09.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.732 7fc058934700 1 --2- 172.21.2.101:0/2709223441 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc054193270 0x7fc054193670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:09.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.732 7fc058934700 1 -- 172.21.2.101:0/2709223441 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc054190920 msgr2=0x7fc05408f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:09.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.732 7fc058934700 1 --2- 172.21.2.101:0/2709223441 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc054190920 0x7fc05408f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:09.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.732 7fc058934700 1 -- 172.21.2.101:0/2709223441 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc054194090 con 0x7fc054190d20 2021-02-10T11:23:09.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.732 7fc058934700 1 --2- 172.21.2.101:0/2709223441 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc054190d20 0x7fc05408fc20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fc05000b5c0 tx=0x7fc05000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:09.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.732 7fc045ffb700 1 -- 172.21.2.101:0/2709223441 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc05000bd50 con 0x7fc054190d20 2021-02-10T11:23:09.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.732 7fc045ffb700 1 -- 172.21.2.101:0/2709223441 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc050006030 con 0x7fc054190d20 2021-02-10T11:23:09.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.732 7fc045ffb700 1 -- 172.21.2.101:0/2709223441 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc050006360 con 0x7fc054190d20 2021-02-10T11:23:09.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.732 7fc05b319700 1 -- 172.21.2.101:0/2709223441 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc054194090 con 0x7fc054190d20 2021-02-10T11:23:09.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.732 7fc05b319700 1 -- 172.21.2.101:0/2709223441 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc054194090 con 0x7fc054190d20 2021-02-10T11:23:09.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.733 7fc045ffb700 1 -- 172.21.2.101:0/2709223441 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc050006210 con 0x7fc054190d20 2021-02-10T11:23:09.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.733 7fc045ffb700 1 --2- 172.21.2.101:0/2709223441 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc034018730 0x7fc03401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:09.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.733 7fc045ffb700 1 -- 172.21.2.101:0/2709223441 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc0500327c0 con 0x7fc054190d20 2021-02-10T11:23:09.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.733 7fc059135700 1 --2- 172.21.2.101:0/2709223441 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc034018730 0x7fc03401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:09.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.733 7fc059135700 1 --2- 172.21.2.101:0/2709223441 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc034018730 0x7fc03401ac10 crc :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:09.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.764 7fc05b319700 1 -- 172.21.2.101:0/2709223441 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc054105af0 con 0x7fc054190d20 2021-02-10T11:23:09.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.767 7fc045ffb700 1 -- 172.21.2.101:0/2709223441 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc050006b80 con 0x7fc054190d20 2021-02-10T11:23:09.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.895 7fc05b319700 1 -- 172.21.2.101:0/2709223441 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839291", "target": ["mgr", ""]}) v1 -- 0x7fc0540f0b50 con 0x7fc034018730 2021-02-10T11:23:09.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.898 7fc045ffb700 1 -- 172.21.2.101:0/2709223441 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fc048000970 con 0x7fc034018730 2021-02-10T11:23:09.906 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839291/07245497-8241-4d02-b49d-0c81c4dd1962 2021-02-10T11:23:09.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.899 7fc05b319700 1 -- 172.21.2.101:0/2709223441 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc034018730 msgr2=0x7fc03401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:09.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.899 7fc05b319700 1 --2- 172.21.2.101:0/2709223441 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc034018730 0x7fc03401ac10 crc :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:09.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.899 7fc05b319700 1 -- 172.21.2.101:0/2709223441 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc054190d20 msgr2=0x7fc05408fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:09.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.899 7fc05b319700 1 --2- 172.21.2.101:0/2709223441 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc054190d20 0x7fc05408fc20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fc05000b5c0 tx=0x7fc05000b960).stop 2021-02-10T11:23:09.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.900 7fc05b319700 1 -- 172.21.2.101:0/2709223441 shutdown_connections 2021-02-10T11:23:09.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.900 7fc05b319700 1 --2- 172.21.2.101:0/2709223441 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc034018730 0x7fc03401ac10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:09.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.900 7fc05b319700 1 --2- 172.21.2.101:0/2709223441 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc054193270 0x7fc054193670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:09.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.900 7fc05b319700 1 --2- 172.21.2.101:0/2709223441 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc054190d20 0x7fc05408fc20 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:09.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.900 7fc05b319700 1 --2- 172.21.2.101:0/2709223441 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc054190920 0x7fc05408f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:09.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.900 7fc05b319700 1 -- 172.21.2.101:0/2709223441 >> 172.21.2.101:0/2709223441 conn(0x7fc0540bf4e0 msgr2=0x7fc0540be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:09.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.900 7fc05b319700 1 -- 172.21.2.101:0/2709223441 shutdown_connections 2021-02-10T11:23:09.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:09.900 7fc05b319700 1 -- 172.21.2.101:0/2709223441 wait complete. 2021-02-10T11:23:09.916 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839291 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839291/07245497-8241-4d02-b49d-0c81c4dd1962 2021-02-10T11:23:09.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:09.916 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:09.916 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:10.251 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:10.252 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:10.252 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839291/07245497-8241-4d02-b49d-0c81c4dd1962/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:10.631 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:10.631 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:10.631 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236521 s, 44.3 MB/s 2021-02-10T11:23:10.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:10.632 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:10.633 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:10.966 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:10.966 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:10.967 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839291/07245497-8241-4d02-b49d-0c81c4dd1962/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:11.342 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:11.343 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:11.343 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252593 s, 41.5 MB/s 2021-02-10T11:23:11.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:11.344 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:11.344 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:11.678 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:11.679 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:11.679 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839291/07245497-8241-4d02-b49d-0c81c4dd1962/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:12.052 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:12.053 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:12.053 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0229747 s, 45.6 MB/s 2021-02-10T11:23:12.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:12.054 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:12.054 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:12.389 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:12.390 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:12.390 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839291/07245497-8241-4d02-b49d-0c81c4dd1962/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:12.768 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:12.768 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:12.769 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253111 s, 41.4 MB/s 2021-02-10T11:23:12.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:12.770 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:12.770 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:13.104 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:13.105 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:13.105 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839291/07245497-8241-4d02-b49d-0c81c4dd1962/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:13.481 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:13.481 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:13.481 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249073 s, 42.1 MB/s 2021-02-10T11:23:13.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:13.483 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:13.483 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:13.818 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:13.818 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:13.819 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839291/07245497-8241-4d02-b49d-0c81c4dd1962/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:14.194 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:14.194 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:14.194 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0258764 s, 40.5 MB/s 2021-02-10T11:23:14.195 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:14.195 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:14.195 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:14.530 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:14.531 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:14.531 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839291/07245497-8241-4d02-b49d-0c81c4dd1962/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:14.905 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:14.905 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:14.906 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0226347 s, 46.3 MB/s 2021-02-10T11:23:14.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:14.907 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:14.907 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:15.243 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:15.243 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:15.243 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839291/07245497-8241-4d02-b49d-0c81c4dd1962/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:15.617 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:15.618 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:15.618 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241339 s, 43.4 MB/s 2021-02-10T11:23:15.618 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:15.619 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:15.619 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:15.953 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:15.953 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:15.954 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839291/07245497-8241-4d02-b49d-0c81c4dd1962/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:16.331 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:16.332 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:16.332 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251815 s, 41.6 MB/s 2021-02-10T11:23:16.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:16.333 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:16.333 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:16.668 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:16.669 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:16.669 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839291/07245497-8241-4d02-b49d-0c81c4dd1962/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:17.048 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:17.048 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:17.048 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0260865 s, 40.2 MB/s 2021-02-10T11:23:17.049 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839292 2021-02-10T11:23:17.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.456 7f682d497700 1 Processor -- start 2021-02-10T11:23:17.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.456 7f682d497700 1 -- start start 2021-02-10T11:23:17.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.456 7f682d497700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68281b0470 0x7f68281b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:17.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.456 7f682d497700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f68280f20e0 0x7f68280f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:17.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.456 7f682d497700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68280f0b50 con 0x7f68280ef610 2021-02-10T11:23:17.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.456 7f682d497700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68280f0820 con 0x7f68280f20e0 2021-02-10T11:23:17.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.456 7f682d497700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68280f24e0 con 0x7f68281b0470 2021-02-10T11:23:17.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.457 7f682707d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68281b0470 0x7f68281b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:17.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.457 7f682787e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f68280f20e0 0x7f68280f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:17.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.457 7f682687c700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f68280ef610 0x7f68280efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:41064/0 (socket says 172.21.2.101:41064) 2021-02-10T11:23:17.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.457 7f682687c700 1 -- 172.21.2.101:0/3735262227 learned_addr learned my addr 172.21.2.101:0/3735262227 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:23:17.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.458 7f682607b700 1 -- 172.21.2.101:0/3735262227 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648204622 0 0) 0x7f68100009b0 con 0x7f68280ef610 2021-02-10T11:23:17.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.458 7f682787e700 1 -- 172.21.2.101:0/3735262227 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68281b0470 msgr2=0x7f68281b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:23:17.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.458 7f682787e700 1 --2- 172.21.2.101:0/3735262227 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68281b0470 0x7f68281b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:17.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.458 7f682787e700 1 -- 172.21.2.101:0/3735262227 >> v1:172.21.2.108:6789/0 conn(0x7f68280ef610 legacy=0x7f68280efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:17.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.458 7f682787e700 1 -- 172.21.2.101:0/3735262227 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68280e52e0 con 0x7f68280f20e0 2021-02-10T11:23:17.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.458 7f682787e700 1 --2- 172.21.2.101:0/3735262227 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68280f20e0 0x7f68280f0200 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f6820006e50 tx=0x7f6820009e70).ready entity=mon.1 client_cookie=9f174ff16aa81f0a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:17.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.459 7f682607b700 1 -- 172.21.2.101:0/3735262227 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f682000b510 con 0x7f68280f20e0 2021-02-10T11:23:17.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.459 7f682607b700 1 -- 172.21.2.101:0/3735262227 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6820006790 con 0x7f68280f20e0 2021-02-10T11:23:17.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.459 7f682607b700 1 -- 172.21.2.101:0/3735262227 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f682000b9d0 con 0x7f68280f20e0 2021-02-10T11:23:17.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.459 7f682d497700 1 -- 172.21.2.101:0/3735262227 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68280f20e0 msgr2=0x7f68280f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:17.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.459 7f682d497700 1 --2- 172.21.2.101:0/3735262227 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68280f20e0 0x7f68280f0200 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f6820006e50 tx=0x7f6820009e70).stop 2021-02-10T11:23:17.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.459 7f682d497700 1 -- 172.21.2.101:0/3735262227 shutdown_connections 2021-02-10T11:23:17.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.459 7f682d497700 1 --2- 172.21.2.101:0/3735262227 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68280f20e0 0x7f68280f0200 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:17.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.459 7f682d497700 1 --2- 172.21.2.101:0/3735262227 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68281b0470 0x7f68281b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:17.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.459 7f682d497700 1 -- 172.21.2.101:0/3735262227 >> 172.21.2.101:0/3735262227 conn(0x7f68280ce3e0 msgr2=0x7f68280e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:17.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.460 7f682d497700 1 -- 172.21.2.101:0/3735262227 shutdown_connections 2021-02-10T11:23:17.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.460 7f682d497700 1 -- 172.21.2.101:0/3735262227 wait complete. 2021-02-10T11:23:17.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.474 7f682d497700 1 Processor -- start 2021-02-10T11:23:17.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.474 7f682d497700 1 -- start start 2021-02-10T11:23:17.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.474 7f682d497700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68280b7c20 0x7f68280b8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:17.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.474 7f682d497700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68280b8a30 0x7f68280c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:17.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.474 7f682d497700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68280c25a0 0x7f68280c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:17.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.474 7f682d497700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68280c1a80 con 0x7f68280b8a30 2021-02-10T11:23:17.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.474 7f682d497700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68280b8020 con 0x7f68280c25a0 2021-02-10T11:23:17.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.474 7f682d497700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f682809b9c0 con 0x7f68280b7c20 2021-02-10T11:23:17.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.475 7f682687c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68280b8a30 0x7f68280c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:17.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.475 7f682687c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68280b8a30 0x7f68280c1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47168/0 (socket says 172.21.2.101:47168) 2021-02-10T11:23:17.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.475 7f682687c700 1 -- 172.21.2.101:0/784145855 learned_addr learned my addr 172.21.2.101:0/784145855 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:17.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.475 7f682707d700 1 --2- 172.21.2.101:0/784145855 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68280b7c20 0x7f68280b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:17.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.475 7f682787e700 1 --2- 172.21.2.101:0/784145855 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68280c25a0 0x7f68280c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:17.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.475 7f682687c700 1 -- 172.21.2.101:0/784145855 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68280b7c20 msgr2=0x7f68280b8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:17.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.475 7f682687c700 1 --2- 172.21.2.101:0/784145855 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68280b7c20 0x7f68280b8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:17.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.475 7f682687c700 1 -- 172.21.2.101:0/784145855 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68280c25a0 msgr2=0x7f68280c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:17.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.475 7f682687c700 1 --2- 172.21.2.101:0/784145855 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68280c25a0 0x7f68280c29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:17.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.475 7f682687c700 1 -- 172.21.2.101:0/784145855 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68280b8e30 con 0x7f68280b8a30 2021-02-10T11:23:17.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.475 7f682707d700 1 --2- 172.21.2.101:0/784145855 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68280b7c20 0x7f68280b8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:23:17.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.475 7f682787e700 1 --2- 172.21.2.101:0/784145855 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68280c25a0 0x7f68280c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:23:17.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.475 7f682687c700 1 --2- 172.21.2.101:0/784145855 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68280b8a30 0x7f68280c1f80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f6810001c60 tx=0x7f681000a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:17.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.476 7f681ffff700 1 -- 172.21.2.101:0/784145855 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f681000acc0 con 0x7f68280b8a30 2021-02-10T11:23:17.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.476 7f681ffff700 1 -- 172.21.2.101:0/784145855 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6810004140 con 0x7f68280b8a30 2021-02-10T11:23:17.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.476 7f682d497700 1 -- 172.21.2.101:0/784145855 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68280b8e30 con 0x7f68280b8a30 2021-02-10T11:23:17.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.476 7f682d497700 1 -- 172.21.2.101:0/784145855 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68280c32b0 con 0x7f68280b8a30 2021-02-10T11:23:17.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.476 7f681ffff700 1 -- 172.21.2.101:0/784145855 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6810004660 con 0x7f68280b8a30 2021-02-10T11:23:17.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.476 7f681ffff700 1 -- 172.21.2.101:0/784145855 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6810004630 con 0x7f68280b8a30 2021-02-10T11:23:17.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.476 7f681ffff700 1 --2- 172.21.2.101:0/784145855 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f680c0186c0 0x7f680c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:17.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.476 7f681ffff700 1 -- 172.21.2.101:0/784145855 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6810032570 con 0x7f68280b8a30 2021-02-10T11:23:17.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.477 7f682707d700 1 --2- 172.21.2.101:0/784145855 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f680c0186c0 0x7f680c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:17.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.477 7f682707d700 1 --2- 172.21.2.101:0/784145855 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f680c0186c0 0x7f680c01abc0 crc :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:17.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.506 7f682d497700 1 -- 172.21.2.101:0/784145855 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6828105b30 con 0x7f68280b8a30 2021-02-10T11:23:17.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.509 7f681ffff700 1 -- 172.21.2.101:0/784145855 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6810004220 con 0x7f68280b8a30 2021-02-10T11:23:17.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.636 7f682d497700 1 -- 172.21.2.101:0/784145855 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839292", "target": ["mgr", ""]}) v1 -- 0x7f68280b8020 con 0x7f680c0186c0 2021-02-10T11:23:17.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.662 7f681ffff700 1 -- 172.21.2.101:0/784145855 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f6818006b30 con 0x7f680c0186c0 2021-02-10T11:23:17.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.669 7f682d497700 1 -- 172.21.2.101:0/784145855 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f680c0186c0 msgr2=0x7f680c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:17.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.669 7f682d497700 1 --2- 172.21.2.101:0/784145855 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f680c0186c0 0x7f680c01abc0 crc :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:17.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.669 7f682d497700 1 -- 172.21.2.101:0/784145855 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68280b8a30 msgr2=0x7f68280c1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:17.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.669 7f682d497700 1 --2- 172.21.2.101:0/784145855 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68280b8a30 0x7f68280c1f80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f6810001c60 tx=0x7f681000a8d0).stop 2021-02-10T11:23:17.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.669 7f682d497700 1 -- 172.21.2.101:0/784145855 shutdown_connections 2021-02-10T11:23:17.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.669 7f682d497700 1 --2- 172.21.2.101:0/784145855 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68280c25a0 0x7f68280c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:17.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.669 7f682d497700 1 --2- 172.21.2.101:0/784145855 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68280b8a30 0x7f68280c1f80 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:17.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.669 7f682d497700 1 --2- 172.21.2.101:0/784145855 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f680c0186c0 0x7f680c01abc0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:17.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.669 7f682d497700 1 --2- 172.21.2.101:0/784145855 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68280b7c20 0x7f68280b8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:17.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.669 7f682d497700 1 -- 172.21.2.101:0/784145855 >> 172.21.2.101:0/784145855 conn(0x7f6828094e10 msgr2=0x7f68280a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:17.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.669 7f682d497700 1 -- 172.21.2.101:0/784145855 shutdown_connections 2021-02-10T11:23:17.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:17.669 7f682d497700 1 -- 172.21.2.101:0/784145855 wait complete. 2021-02-10T11:23:17.685 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839292 2021-02-10T11:23:18.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.092 7f6ede038700 1 Processor -- start 2021-02-10T11:23:18.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.092 7f6ede038700 1 -- start start 2021-02-10T11:23:18.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.093 7f6ede038700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6ed81b0470 0x7f6ed81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:18.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.093 7f6ede038700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6ed80f20e0 0x7f6ed80f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:18.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.093 7f6ede038700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ed80f0b50 con 0x7f6ed80ef610 2021-02-10T11:23:18.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.093 7f6ede038700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ed80f0820 con 0x7f6ed80f20e0 2021-02-10T11:23:18.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.093 7f6ede038700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ed80f24e0 con 0x7f6ed81b0470 2021-02-10T11:23:18.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.093 7f6ed7fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6ed80f20e0 0x7f6ed80f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:18.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.093 7f6ed7fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6ed80f20e0 0x7f6ed80f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47172/0 (socket says 172.21.2.101:47172) 2021-02-10T11:23:18.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.093 7f6ed7fff700 1 -- 172.21.2.101:0/1595952146 learned_addr learned my addr 172.21.2.101:0/1595952146 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:18.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.093 7f6ed7fff700 1 -- 172.21.2.101:0/1595952146 >> v2:172.21.2.108:3300/0 conn(0x7f6ed81b0470 msgr2=0x7f6ed81b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:23:18.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.093 7f6ed7fff700 1 --2- 172.21.2.101:0/1595952146 >> v2:172.21.2.108:3300/0 conn(0x7f6ed81b0470 0x7f6ed81b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:18.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.093 7f6ed7fff700 1 -- 172.21.2.101:0/1595952146 >> v1:172.21.2.108:6789/0 conn(0x7f6ed80ef610 legacy=0x7f6ed80efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:23:18.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.094 7f6ed7fff700 1 -- 172.21.2.101:0/1595952146 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed80e52e0 con 0x7f6ed80f20e0 2021-02-10T11:23:18.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.094 7f6ed7fff700 1 --2- 172.21.2.101:0/1595952146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ed80f20e0 0x7f6ed80f0200 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f6ed0006f00 tx=0x7f6ed0009f10).ready entity=mon.0 client_cookie=bdcf050b4d6f27b6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:18.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.094 7f6ed67fc700 1 -- 172.21.2.101:0/1595952146 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ed000b3b0 con 0x7f6ed80f20e0 2021-02-10T11:23:18.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.094 7f6ed67fc700 1 -- 172.21.2.101:0/1595952146 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed000b630 con 0x7f6ed80f20e0 2021-02-10T11:23:18.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.094 7f6ed67fc700 1 -- 172.21.2.101:0/1595952146 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ed000bb50 con 0x7f6ed80f20e0 2021-02-10T11:23:18.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.094 7f6ede038700 1 -- 172.21.2.101:0/1595952146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ed80f20e0 msgr2=0x7f6ed80f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:18.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.094 7f6ede038700 1 --2- 172.21.2.101:0/1595952146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ed80f20e0 0x7f6ed80f0200 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f6ed0006f00 tx=0x7f6ed0009f10).stop 2021-02-10T11:23:18.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.095 7f6ede038700 1 -- 172.21.2.101:0/1595952146 shutdown_connections 2021-02-10T11:23:18.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.095 7f6ede038700 1 --2- 172.21.2.101:0/1595952146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ed80f20e0 0x7f6ed80f0200 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:18.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.095 7f6ede038700 1 --2- 172.21.2.101:0/1595952146 >> v2:172.21.2.108:3300/0 conn(0x7f6ed81b0470 0x7f6ed81b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:18.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.095 7f6ede038700 1 -- 172.21.2.101:0/1595952146 >> 172.21.2.101:0/1595952146 conn(0x7f6ed80ce3e0 msgr2=0x7f6ed80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:18.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.095 7f6ede038700 1 -- 172.21.2.101:0/1595952146 shutdown_connections 2021-02-10T11:23:18.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.095 7f6ede038700 1 -- 172.21.2.101:0/1595952146 wait complete. 2021-02-10T11:23:18.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.109 7f6ede038700 1 Processor -- start 2021-02-10T11:23:18.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.109 7f6ede038700 1 -- start start 2021-02-10T11:23:18.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.110 7f6ede038700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ed80c36a0 0x7f6ed80c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:18.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.110 7f6ede038700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6ed80c3190 0x7f6ed80b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:18.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.110 7f6ede038700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ed80b2370 0x7f6ed80b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:18.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.110 7f6ede038700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80b27c0 con 0x7f6ed80b2370 2021-02-10T11:23:18.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.110 7f6ede038700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80b3360 con 0x7f6ed80c3190 2021-02-10T11:23:18.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.110 7f6ede038700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80b3540 con 0x7f6ed80c36a0 2021-02-10T11:23:18.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.110 7f6ed7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ed80b2370 0x7f6ed80b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:18.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.110 7f6ed7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ed80b2370 0x7f6ed80b2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47178/0 (socket says 172.21.2.101:47178) 2021-02-10T11:23:18.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.110 7f6ed7fff700 1 -- 172.21.2.101:0/2570425091 learned_addr learned my addr 172.21.2.101:0/2570425091 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:18.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.110 7f6ed7fff700 1 -- 172.21.2.101:0/2570425091 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ed80c36a0 msgr2=0x7f6ed80c2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:18.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.110 7f6ed7fff700 1 --2- 172.21.2.101:0/2570425091 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ed80c36a0 0x7f6ed80c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:18.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.110 7f6ed7fff700 1 -- 172.21.2.101:0/2570425091 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6ed80c3190 msgr2=0x7f6ed80b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:18.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.110 7f6ed7fff700 1 --2- 172.21.2.101:0/2570425091 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6ed80c3190 0x7f6ed80b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:18.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.110 7f6ed7fff700 1 -- 172.21.2.101:0/2570425091 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed80b3760 con 0x7f6ed80b2370 2021-02-10T11:23:18.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.111 7f6ed7fff700 1 --2- 172.21.2.101:0/2570425091 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ed80b2370 0x7f6ed80b2d40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f6ed0009390 tx=0x7f6ed000bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:18.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.111 7f6ecffff700 1 -- 172.21.2.101:0/2570425091 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ed0003030 con 0x7f6ed80b2370 2021-02-10T11:23:18.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.111 7f6ecffff700 1 -- 172.21.2.101:0/2570425091 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed00068e0 con 0x7f6ed80b2370 2021-02-10T11:23:18.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.111 7f6ecffff700 1 -- 172.21.2.101:0/2570425091 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ed000c570 con 0x7f6ed80b2370 2021-02-10T11:23:18.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.111 7f6ede038700 1 -- 172.21.2.101:0/2570425091 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ed80b3760 con 0x7f6ed80b2370 2021-02-10T11:23:18.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.111 7f6ede038700 1 -- 172.21.2.101:0/2570425091 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed80b3760 con 0x7f6ed80b2370 2021-02-10T11:23:18.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.111 7f6ecffff700 1 -- 172.21.2.101:0/2570425091 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6ed0005820 con 0x7f6ed80b2370 2021-02-10T11:23:18.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.112 7f6ecffff700 1 --2- 172.21.2.101:0/2570425091 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ebc0186c0 0x7f6ebc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:18.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.112 7f6ecffff700 1 -- 172.21.2.101:0/2570425091 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6ed00328f0 con 0x7f6ed80b2370 2021-02-10T11:23:18.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.112 7f6ed77fe700 1 --2- 172.21.2.101:0/2570425091 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ebc0186c0 0x7f6ebc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:18.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.112 7f6ed77fe700 1 --2- 172.21.2.101:0/2570425091 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ebc0186c0 0x7f6ebc01abc0 crc :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:18.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.143 7f6ede038700 1 -- 172.21.2.101:0/2570425091 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ed80edd10 con 0x7f6ed80b2370 2021-02-10T11:23:18.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.146 7f6ecffff700 1 -- 172.21.2.101:0/2570425091 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6ed0006c80 con 0x7f6ed80b2370 2021-02-10T11:23:18.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.276 7f6ede038700 1 -- 172.21.2.101:0/2570425091 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839292", "target": ["mgr", ""]}) v1 -- 0x7f6ed80edd10 con 0x7f6ebc0186c0 2021-02-10T11:23:18.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.279 7f6ecffff700 1 -- 172.21.2.101:0/2570425091 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f6ec0006e40 con 0x7f6ebc0186c0 2021-02-10T11:23:18.287 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839292/88688e19-1da3-4512-9ff4-d964d7010206 2021-02-10T11:23:18.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.280 7f6ede038700 1 -- 172.21.2.101:0/2570425091 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ebc0186c0 msgr2=0x7f6ebc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:18.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.280 7f6ede038700 1 --2- 172.21.2.101:0/2570425091 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ebc0186c0 0x7f6ebc01abc0 crc :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:18.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.280 7f6ede038700 1 -- 172.21.2.101:0/2570425091 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ed80b2370 msgr2=0x7f6ed80b2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:18.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.280 7f6ede038700 1 --2- 172.21.2.101:0/2570425091 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ed80b2370 0x7f6ed80b2d40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f6ed0009390 tx=0x7f6ed000bd40).stop 2021-02-10T11:23:18.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.280 7f6ede038700 1 -- 172.21.2.101:0/2570425091 shutdown_connections 2021-02-10T11:23:18.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.280 7f6ede038700 1 --2- 172.21.2.101:0/2570425091 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ed80b2370 0x7f6ed80b2d40 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:18.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.280 7f6ede038700 1 --2- 172.21.2.101:0/2570425091 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6ed80c3190 0x7f6ed80b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:18.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.280 7f6ede038700 1 --2- 172.21.2.101:0/2570425091 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ebc0186c0 0x7f6ebc01abc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:18.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.280 7f6ede038700 1 --2- 172.21.2.101:0/2570425091 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ed80c36a0 0x7f6ed80c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:18.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.280 7f6ede038700 1 -- 172.21.2.101:0/2570425091 >> 172.21.2.101:0/2570425091 conn(0x7f6ed809f630 msgr2=0x7f6ed80a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:18.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.280 7f6ede038700 1 -- 172.21.2.101:0/2570425091 shutdown_connections 2021-02-10T11:23:18.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:18.280 7f6ede038700 1 -- 172.21.2.101:0/2570425091 wait complete. 2021-02-10T11:23:18.297 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839292 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839292/88688e19-1da3-4512-9ff4-d964d7010206 2021-02-10T11:23:18.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:18.297 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:18.297 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:18.633 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:18.633 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:18.633 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839292/88688e19-1da3-4512-9ff4-d964d7010206/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:19.011 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:19.012 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:19.012 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0230734 s, 45.4 MB/s 2021-02-10T11:23:19.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:19.013 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:19.013 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:19.348 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:19.348 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:19.349 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839292/88688e19-1da3-4512-9ff4-d964d7010206/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:19.725 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:19.725 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:19.725 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024888 s, 42.1 MB/s 2021-02-10T11:23:19.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:19.726 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:19.726 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:20.060 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:20.061 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:20.061 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839292/88688e19-1da3-4512-9ff4-d964d7010206/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:20.438 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:20.439 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:20.439 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242247 s, 43.3 MB/s 2021-02-10T11:23:20.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:20.440 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:20.440 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:20.723 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:23:20.726 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:23:20.776 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:20.776 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:20.776 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839292/88688e19-1da3-4512-9ff4-d964d7010206/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:21.099 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:21.099 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:21.099 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249824 s, 42.0 MB/s 2021-02-10T11:23:21.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:21.100 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:21.101 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:21.476 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:21.476 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:21.476 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839292/88688e19-1da3-4512-9ff4-d964d7010206/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:21.850 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:21.851 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:21.851 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242198 s, 43.3 MB/s 2021-02-10T11:23:21.852 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:21.852 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:21.852 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:22.187 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:22.187 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:22.187 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839292/88688e19-1da3-4512-9ff4-d964d7010206/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:22.564 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:22.564 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:22.564 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249364 s, 42.1 MB/s 2021-02-10T11:23:22.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:22.565 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:22.565 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:22.901 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:22.901 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:22.901 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839292/88688e19-1da3-4512-9ff4-d964d7010206/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:23.274 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:23.336 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:23.336 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235071 s, 44.6 MB/s 2021-02-10T11:23:23.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:23.338 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:23.338 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:23.673 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:23.673 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:23.674 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839292/88688e19-1da3-4512-9ff4-d964d7010206/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:24.046 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:24.047 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:24.047 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0254959 s, 41.1 MB/s 2021-02-10T11:23:24.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:24.048 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:24.048 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:24.382 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:24.383 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:24.383 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839292/88688e19-1da3-4512-9ff4-d964d7010206/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:24.759 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:24.760 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:24.760 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251359 s, 41.7 MB/s 2021-02-10T11:23:24.760 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:24.761 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:24.761 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:25.098 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:25.098 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:25.098 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839292/88688e19-1da3-4512-9ff4-d964d7010206/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:25.474 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:25.474 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:25.474 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247128 s, 42.4 MB/s 2021-02-10T11:23:25.475 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839293 2021-02-10T11:23:25.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.881 7fb5dbda3700 1 Processor -- start 2021-02-10T11:23:25.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.881 7fb5dbda3700 1 -- start start 2021-02-10T11:23:25.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.882 7fb5dbda3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5d41b0470 0x7fb5d41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:25.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.882 7fb5dbda3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb5d40f20e0 0x7fb5d40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:25.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.882 7fb5dbda3700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d40f0b50 con 0x7fb5d40ef610 2021-02-10T11:23:25.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.882 7fb5dbda3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d40f0820 con 0x7fb5d40f20e0 2021-02-10T11:23:25.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.882 7fb5dbda3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d40f24e0 con 0x7fb5d41b0470 2021-02-10T11:23:25.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.882 7fb5da3c0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb5d40f20e0 0x7fb5d40f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:25.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.882 7fb5da3c0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb5d40f20e0 0x7fb5d40f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47186/0 (socket says 172.21.2.101:47186) 2021-02-10T11:23:25.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.882 7fb5d93be700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb5d40ef610 0x7fb5d40efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47680/0 (socket says 172.21.2.101:47680) 2021-02-10T11:23:25.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.882 7fb5da3c0700 1 -- 172.21.2.101:0/3352600619 learned_addr learned my addr 172.21.2.101:0/3352600619 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:25.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.882 7fb5da3c0700 1 -- 172.21.2.101:0/3352600619 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5d41b0470 msgr2=0x7fb5d41b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:23:25.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.882 7fb5da3c0700 1 --2- 172.21.2.101:0/3352600619 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5d41b0470 0x7fb5d41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:25.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.882 7fb5da3c0700 1 -- 172.21.2.101:0/3352600619 >> v1:172.21.2.101:6789/0 conn(0x7fb5d40ef610 legacy=0x7fb5d40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:25.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.882 7fb5da3c0700 1 -- 172.21.2.101:0/3352600619 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d40e52e0 con 0x7fb5d40f20e0 2021-02-10T11:23:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.882 7fb5da3c0700 1 --2- 172.21.2.101:0/3352600619 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5d40f20e0 0x7fb5d40f0200 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7fb5cc006f00 tx=0x7fb5cc009f10).ready entity=mon.0 client_cookie=f552c4d944ac0126 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.883 7fb5d8bbd700 1 -- 172.21.2.101:0/3352600619 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb5cc00b3b0 con 0x7fb5d40f20e0 2021-02-10T11:23:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.883 7fb5d8bbd700 1 -- 172.21.2.101:0/3352600619 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5cc00b730 con 0x7fb5d40f20e0 2021-02-10T11:23:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.883 7fb5d8bbd700 1 -- 172.21.2.101:0/3352600619 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb5cc00bc50 con 0x7fb5d40f20e0 2021-02-10T11:23:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.883 7fb5dbda3700 1 -- 172.21.2.101:0/3352600619 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5d40f20e0 msgr2=0x7fb5d40f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.883 7fb5dbda3700 1 --2- 172.21.2.101:0/3352600619 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5d40f20e0 0x7fb5d40f0200 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7fb5cc006f00 tx=0x7fb5cc009f10).stop 2021-02-10T11:23:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.884 7fb5dbda3700 1 -- 172.21.2.101:0/3352600619 shutdown_connections 2021-02-10T11:23:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.884 7fb5dbda3700 1 --2- 172.21.2.101:0/3352600619 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5d40f20e0 0x7fb5d40f0200 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:25.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.884 7fb5dbda3700 1 --2- 172.21.2.101:0/3352600619 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5d41b0470 0x7fb5d41b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:25.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.884 7fb5dbda3700 1 -- 172.21.2.101:0/3352600619 >> 172.21.2.101:0/3352600619 conn(0x7fb5d40ce3e0 msgr2=0x7fb5d40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:25.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.884 7fb5dbda3700 1 -- 172.21.2.101:0/3352600619 shutdown_connections 2021-02-10T11:23:25.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.884 7fb5dbda3700 1 -- 172.21.2.101:0/3352600619 wait complete. 2021-02-10T11:23:25.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.898 7fb5dbda3700 1 Processor -- start 2021-02-10T11:23:25.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.898 7fb5dbda3700 1 -- start start 2021-02-10T11:23:25.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.899 7fb5dbda3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb5d40b20b0 0x7fb5d40ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:25.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.899 7fb5dbda3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5d40b24b0 0x7fb5d4191dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:25.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.899 7fb5dbda3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5d40acf90 0x7fb5d41923e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:25.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.899 7fb5dbda3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d40b1b30 con 0x7fb5d40acf90 2021-02-10T11:23:25.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.899 7fb5dbda3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d41a04e0 con 0x7fb5d40b20b0 2021-02-10T11:23:25.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.899 7fb5dbda3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d41a06c0 con 0x7fb5d40b24b0 2021-02-10T11:23:25.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.899 7fb5da3c0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5d40acf90 0x7fb5d41923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:25.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.899 7fb5da3c0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5d40acf90 0x7fb5d41923e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47188/0 (socket says 172.21.2.101:47188) 2021-02-10T11:23:25.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.899 7fb5da3c0700 1 -- 172.21.2.101:0/1295547072 learned_addr learned my addr 172.21.2.101:0/1295547072 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:25.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.899 7fb5da3c0700 1 -- 172.21.2.101:0/1295547072 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5d40b24b0 msgr2=0x7fb5d4191dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:23:25.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.899 7fb5da3c0700 1 --2- 172.21.2.101:0/1295547072 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5d40b24b0 0x7fb5d4191dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:25.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.899 7fb5da3c0700 1 -- 172.21.2.101:0/1295547072 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb5d40b20b0 msgr2=0x7fb5d40ac970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:23:25.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.899 7fb5da3c0700 1 --2- 172.21.2.101:0/1295547072 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb5d40b20b0 0x7fb5d40ac970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:25.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.899 7fb5da3c0700 1 -- 172.21.2.101:0/1295547072 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d4192be0 con 0x7fb5d40acf90 2021-02-10T11:23:25.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.899 7fb5da3c0700 1 --2- 172.21.2.101:0/1295547072 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5d40acf90 0x7fb5d41923e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fb5cc009390 tx=0x7fb5cc00bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:25.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.900 7fb5ca7fc700 1 -- 172.21.2.101:0/1295547072 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb5cc000cc0 con 0x7fb5d40acf90 2021-02-10T11:23:25.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.900 7fb5ca7fc700 1 -- 172.21.2.101:0/1295547072 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5cc003030 con 0x7fb5d40acf90 2021-02-10T11:23:25.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.900 7fb5ca7fc700 1 -- 172.21.2.101:0/1295547072 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb5cc001480 con 0x7fb5d40acf90 2021-02-10T11:23:25.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.900 7fb5dbda3700 1 -- 172.21.2.101:0/1295547072 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d4192be0 con 0x7fb5d40acf90 2021-02-10T11:23:25.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.900 7fb5dbda3700 1 -- 172.21.2.101:0/1295547072 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d4192be0 con 0x7fb5d40acf90 2021-02-10T11:23:25.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.900 7fb5ca7fc700 1 -- 172.21.2.101:0/1295547072 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb5cc001270 con 0x7fb5d40acf90 2021-02-10T11:23:25.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.900 7fb5ca7fc700 1 --2- 172.21.2.101:0/1295547072 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5bc0186c0 0x7fb5bc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:25.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.900 7fb5ca7fc700 1 -- 172.21.2.101:0/1295547072 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb5cc032900 con 0x7fb5d40acf90 2021-02-10T11:23:25.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.900 7fb5d9bbf700 1 --2- 172.21.2.101:0/1295547072 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5bc0186c0 0x7fb5bc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:25.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.901 7fb5d9bbf700 1 --2- 172.21.2.101:0/1295547072 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5bc0186c0 0x7fb5bc01abc0 crc :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:25.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.931 7fb5dbda3700 1 -- 172.21.2.101:0/1295547072 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d4105af0 con 0x7fb5d40acf90 2021-02-10T11:23:25.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:25.934 7fb5ca7fc700 1 -- 172.21.2.101:0/1295547072 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb5cc000cc0 con 0x7fb5d40acf90 2021-02-10T11:23:26.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.059 7fb5dbda3700 1 -- 172.21.2.101:0/1295547072 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839293", "target": ["mgr", ""]}) v1 -- 0x7fb5d4105af0 con 0x7fb5bc0186c0 2021-02-10T11:23:26.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.084 7fb5ca7fc700 1 -- 172.21.2.101:0/1295547072 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fb5d0006960 con 0x7fb5bc0186c0 2021-02-10T11:23:26.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.092 7fb5dbda3700 1 -- 172.21.2.101:0/1295547072 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5bc0186c0 msgr2=0x7fb5bc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:26.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.092 7fb5dbda3700 1 --2- 172.21.2.101:0/1295547072 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5bc0186c0 0x7fb5bc01abc0 crc :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:26.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.092 7fb5dbda3700 1 -- 172.21.2.101:0/1295547072 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5d40acf90 msgr2=0x7fb5d41923e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:26.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.092 7fb5dbda3700 1 --2- 172.21.2.101:0/1295547072 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5d40acf90 0x7fb5d41923e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fb5cc009390 tx=0x7fb5cc00bd40).stop 2021-02-10T11:23:26.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.092 7fb5dbda3700 1 -- 172.21.2.101:0/1295547072 shutdown_connections 2021-02-10T11:23:26.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.092 7fb5dbda3700 1 --2- 172.21.2.101:0/1295547072 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5bc0186c0 0x7fb5bc01abc0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:26.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.092 7fb5dbda3700 1 --2- 172.21.2.101:0/1295547072 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5d40b24b0 0x7fb5d4191dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:26.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.092 7fb5dbda3700 1 --2- 172.21.2.101:0/1295547072 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5d40acf90 0x7fb5d41923e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:26.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.092 7fb5dbda3700 1 --2- 172.21.2.101:0/1295547072 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb5d40b20b0 0x7fb5d40ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:26.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.092 7fb5dbda3700 1 -- 172.21.2.101:0/1295547072 >> 172.21.2.101:0/1295547072 conn(0x7fb5d40bf4e0 msgr2=0x7fb5d40be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:26.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.092 7fb5dbda3700 1 -- 172.21.2.101:0/1295547072 shutdown_connections 2021-02-10T11:23:26.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.092 7fb5dbda3700 1 -- 172.21.2.101:0/1295547072 wait complete. 2021-02-10T11:23:26.108 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839293 2021-02-10T11:23:26.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.514 7ffaaad94700 1 Processor -- start 2021-02-10T11:23:26.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.514 7ffaaad94700 1 -- start start 2021-02-10T11:23:26.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.514 7ffaaad94700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ffaa41b0870 0x7ffaa40f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:26.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.515 7ffaaad94700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ffaa40efe00 0x7ffaa40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:26.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.515 7ffaaad94700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa41b0c70 con 0x7ffaa41b0470 2021-02-10T11:23:26.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.515 7ffaaad94700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa40e5ea0 con 0x7ffaa41b0870 2021-02-10T11:23:26.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.515 7ffaaad94700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa40f0b50 con 0x7ffaa40efe00 2021-02-10T11:23:26.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.515 7ffaa3fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ffaa41b0870 0x7ffaa40f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:26.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.515 7ffaa3fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ffaa41b0870 0x7ffaa40f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47196/0 (socket says 172.21.2.101:47196) 2021-02-10T11:23:26.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.515 7ffaa8bb0700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ffaa41b0470 0x7ffaa40ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47690/0 (socket says 172.21.2.101:47690) 2021-02-10T11:23:26.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.515 7ffaa3fff700 1 -- 172.21.2.101:0/1570814287 learned_addr learned my addr 172.21.2.101:0/1570814287 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:26.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.515 7ffaa93b1700 1 --2- 172.21.2.101:0/1570814287 >> v2:172.21.2.108:3300/0 conn(0x7ffaa40efe00 0x7ffaa40f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:26.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.515 7ffaa3fff700 1 -- 172.21.2.101:0/1570814287 >> v2:172.21.2.108:3300/0 conn(0x7ffaa40efe00 msgr2=0x7ffaa40f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:23:26.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.515 7ffaa3fff700 1 --2- 172.21.2.101:0/1570814287 >> v2:172.21.2.108:3300/0 conn(0x7ffaa40efe00 0x7ffaa40f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:26.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.515 7ffaa3fff700 1 -- 172.21.2.101:0/1570814287 >> v1:172.21.2.101:6789/0 conn(0x7ffaa41b0470 legacy=0x7ffaa40ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:26.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.515 7ffaa3fff700 1 -- 172.21.2.101:0/1570814287 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa40f0820 con 0x7ffaa41b0870 2021-02-10T11:23:26.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.515 7ffaa3fff700 1 --2- 172.21.2.101:0/1570814287 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffaa41b0870 0x7ffaa40f20e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7ffa94006a80 tx=0x7ffa94009f00).ready entity=mon.0 client_cookie=2cc5afae1a0c06e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:26.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.516 7ffaa37fe700 1 -- 172.21.2.101:0/1570814287 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffa94000d30 con 0x7ffaa41b0870 2021-02-10T11:23:26.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.516 7ffaa37fe700 1 -- 172.21.2.101:0/1570814287 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa94006710 con 0x7ffaa41b0870 2021-02-10T11:23:26.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.516 7ffaa37fe700 1 -- 172.21.2.101:0/1570814287 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffa9400b5c0 con 0x7ffaa41b0870 2021-02-10T11:23:26.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.516 7ffaaad94700 1 -- 172.21.2.101:0/1570814287 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffaa41b0870 msgr2=0x7ffaa40f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:26.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.516 7ffaaad94700 1 --2- 172.21.2.101:0/1570814287 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffaa41b0870 0x7ffaa40f20e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7ffa94006a80 tx=0x7ffa94009f00).stop 2021-02-10T11:23:26.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.516 7ffaaad94700 1 -- 172.21.2.101:0/1570814287 shutdown_connections 2021-02-10T11:23:26.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.516 7ffaaad94700 1 --2- 172.21.2.101:0/1570814287 >> v2:172.21.2.108:3300/0 conn(0x7ffaa40efe00 0x7ffaa40f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:26.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.516 7ffaaad94700 1 --2- 172.21.2.101:0/1570814287 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffaa41b0870 0x7ffaa40f20e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:26.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.516 7ffaaad94700 1 -- 172.21.2.101:0/1570814287 >> 172.21.2.101:0/1570814287 conn(0x7ffaa40ce3e0 msgr2=0x7ffaa40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:26.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.516 7ffaaad94700 1 -- 172.21.2.101:0/1570814287 shutdown_connections 2021-02-10T11:23:26.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.516 7ffaaad94700 1 -- 172.21.2.101:0/1570814287 wait complete. 2021-02-10T11:23:26.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.531 7ffaaad94700 1 Processor -- start 2021-02-10T11:23:26.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.531 7ffaaad94700 1 -- start start 2021-02-10T11:23:26.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.531 7ffaaad94700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffaa4190920 0x7ffaa40b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:26.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.531 7ffaaad94700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffaa40b8200 0x7ffaa40b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:26.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.531 7ffaaad94700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffaa40c1d80 0x7ffaa40c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:26.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.531 7ffaaad94700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa4190da0 con 0x7ffaa40c1d80 2021-02-10T11:23:26.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.531 7ffaaad94700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa40b8c20 con 0x7ffaa4190920 2021-02-10T11:23:26.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.531 7ffaaad94700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa40b8e10 con 0x7ffaa40b8200 2021-02-10T11:23:26.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.531 7ffaa93b1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffaa40c1d80 0x7ffaa40c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:26.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.531 7ffaa93b1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffaa40c1d80 0x7ffaa40c2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47204/0 (socket says 172.21.2.101:47204) 2021-02-10T11:23:26.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.531 7ffaa93b1700 1 -- 172.21.2.101:0/3099217606 learned_addr learned my addr 172.21.2.101:0/3099217606 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:26.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.532 7ffaa93b1700 1 -- 172.21.2.101:0/3099217606 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffaa40b8200 msgr2=0x7ffaa40b8600 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:23:26.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.532 7ffaa93b1700 1 --2- 172.21.2.101:0/3099217606 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffaa40b8200 0x7ffaa40b8600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:26.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.532 7ffaa93b1700 1 -- 172.21.2.101:0/3099217606 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffaa4190920 msgr2=0x7ffaa40b7be0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:23:26.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.532 7ffaa93b1700 1 --2- 172.21.2.101:0/3099217606 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffaa4190920 0x7ffaa40b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:26.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.532 7ffaa93b1700 1 -- 172.21.2.101:0/3099217606 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa40c27d0 con 0x7ffaa40c1d80 2021-02-10T11:23:26.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.532 7ffaa93b1700 1 --2- 172.21.2.101:0/3099217606 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffaa40c1d80 0x7ffaa40c2180 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7ffa9c00b580 tx=0x7ffa9c00b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:26.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.532 7ffaa17fa700 1 -- 172.21.2.101:0/3099217606 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffa9c00bd10 con 0x7ffaa40c1d80 2021-02-10T11:23:26.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.532 7ffaa17fa700 1 -- 172.21.2.101:0/3099217606 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9c006720 con 0x7ffaa40c1d80 2021-02-10T11:23:26.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.532 7ffaa17fa700 1 -- 172.21.2.101:0/3099217606 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffa9c002560 con 0x7ffaa40c1d80 2021-02-10T11:23:26.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.532 7ffaaad94700 1 -- 172.21.2.101:0/3099217606 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa40c27d0 con 0x7ffaa40c1d80 2021-02-10T11:23:26.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.532 7ffaaad94700 1 -- 172.21.2.101:0/3099217606 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa40c27d0 con 0x7ffaa40c1d80 2021-02-10T11:23:26.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.533 7ffaa17fa700 1 -- 172.21.2.101:0/3099217606 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ffa9c005330 con 0x7ffaa40c1d80 2021-02-10T11:23:26.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.533 7ffaa17fa700 1 --2- 172.21.2.101:0/3099217606 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffa880186c0 0x7ffa8801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:26.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.533 7ffaa17fa700 1 -- 172.21.2.101:0/3099217606 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ffa9c0327f0 con 0x7ffaa40c1d80 2021-02-10T11:23:26.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.533 7ffaa8bb0700 1 --2- 172.21.2.101:0/3099217606 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffa880186c0 0x7ffa8801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:26.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.533 7ffaa8bb0700 1 --2- 172.21.2.101:0/3099217606 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffa880186c0 0x7ffa8801abc0 crc :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.564 7ffaaad94700 1 -- 172.21.2.101:0/3099217606 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa4105ae0 con 0x7ffaa40c1d80 2021-02-10T11:23:26.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.567 7ffaa17fa700 1 -- 172.21.2.101:0/3099217606 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ffa9c006370 con 0x7ffaa40c1d80 2021-02-10T11:23:26.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.694 7ffaaad94700 1 -- 172.21.2.101:0/3099217606 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839293", "target": ["mgr", ""]}) v1 -- 0x7ffaa40f0b50 con 0x7ffa880186c0 2021-02-10T11:23:26.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.699 7ffaa17fa700 1 -- 172.21.2.101:0/3099217606 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7ffa8c001590 con 0x7ffa880186c0 2021-02-10T11:23:26.710 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839293/94125cd0-5e67-4ec9-adfb-39d9faf6e7bd 2021-02-10T11:23:26.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.703 7ffaaad94700 1 -- 172.21.2.101:0/3099217606 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffa880186c0 msgr2=0x7ffa8801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:26.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.703 7ffaaad94700 1 --2- 172.21.2.101:0/3099217606 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffa880186c0 0x7ffa8801abc0 crc :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:26.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.703 7ffaaad94700 1 -- 172.21.2.101:0/3099217606 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffaa40c1d80 msgr2=0x7ffaa40c2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:26.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.704 7ffaaad94700 1 --2- 172.21.2.101:0/3099217606 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffaa40c1d80 0x7ffaa40c2180 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7ffa9c00b580 tx=0x7ffa9c00b920).stop 2021-02-10T11:23:26.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.704 7ffaaad94700 1 -- 172.21.2.101:0/3099217606 shutdown_connections 2021-02-10T11:23:26.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.704 7ffaaad94700 1 --2- 172.21.2.101:0/3099217606 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffa880186c0 0x7ffa8801abc0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:26.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.704 7ffaaad94700 1 --2- 172.21.2.101:0/3099217606 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffaa40b8200 0x7ffaa40b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:26.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.704 7ffaaad94700 1 --2- 172.21.2.101:0/3099217606 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffaa40c1d80 0x7ffaa40c2180 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:26.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.704 7ffaaad94700 1 --2- 172.21.2.101:0/3099217606 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffaa4190920 0x7ffaa40b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:26.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.704 7ffaaad94700 1 -- 172.21.2.101:0/3099217606 >> 172.21.2.101:0/3099217606 conn(0x7ffaa4094e10 msgr2=0x7ffaa40a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:26.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.704 7ffaaad94700 1 -- 172.21.2.101:0/3099217606 shutdown_connections 2021-02-10T11:23:26.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:26.704 7ffaaad94700 1 -- 172.21.2.101:0/3099217606 wait complete. 2021-02-10T11:23:26.721 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839293 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839293/94125cd0-5e67-4ec9-adfb-39d9faf6e7bd 2021-02-10T11:23:26.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:26.721 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:26.721 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:27.056 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:27.056 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:27.057 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839293/94125cd0-5e67-4ec9-adfb-39d9faf6e7bd/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:27.435 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:27.436 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:27.436 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0230151 s, 45.6 MB/s 2021-02-10T11:23:27.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:27.437 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:27.437 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:27.771 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:27.772 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:27.772 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839293/94125cd0-5e67-4ec9-adfb-39d9faf6e7bd/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:28.150 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:28.150 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:28.150 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249257 s, 42.1 MB/s 2021-02-10T11:23:28.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:28.151 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:28.151 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:28.486 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:28.486 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:28.487 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839293/94125cd0-5e67-4ec9-adfb-39d9faf6e7bd/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:28.858 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:28.858 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:28.858 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0221745 s, 47.3 MB/s 2021-02-10T11:23:28.859 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:28.859 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:28.859 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:29.195 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:29.196 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:29.196 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839293/94125cd0-5e67-4ec9-adfb-39d9faf6e7bd/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:29.576 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:29.577 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:29.577 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.022618 s, 46.4 MB/s 2021-02-10T11:23:29.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:29.579 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:29.579 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:29.915 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:29.916 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:29.916 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839293/94125cd0-5e67-4ec9-adfb-39d9faf6e7bd/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:30.294 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:30.295 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:30.295 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0263127 s, 39.9 MB/s 2021-02-10T11:23:30.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:30.296 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:30.296 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:30.631 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:30.631 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:30.632 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839293/94125cd0-5e67-4ec9-adfb-39d9faf6e7bd/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:31.005 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:31.005 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:31.005 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0224842 s, 46.6 MB/s 2021-02-10T11:23:31.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:31.006 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:31.006 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:31.341 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:31.342 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:31.342 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839293/94125cd0-5e67-4ec9-adfb-39d9faf6e7bd/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:31.718 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:31.718 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:31.718 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.02479 s, 42.3 MB/s 2021-02-10T11:23:31.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:31.719 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:31.719 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:32.056 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:32.056 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:32.056 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839293/94125cd0-5e67-4ec9-adfb-39d9faf6e7bd/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:32.433 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:32.434 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:32.434 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0256043 s, 41.0 MB/s 2021-02-10T11:23:32.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:32.435 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:32.435 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:32.770 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:32.771 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:32.771 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839293/94125cd0-5e67-4ec9-adfb-39d9faf6e7bd/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:33.147 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:33.147 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:33.147 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239373 s, 43.8 MB/s 2021-02-10T11:23:33.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:33.148 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:33.148 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:33.484 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:33.484 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:33.484 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839293/94125cd0-5e67-4ec9-adfb-39d9faf6e7bd/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:33.857 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:33.857 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:33.857 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240305 s, 43.6 MB/s 2021-02-10T11:23:33.858 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839294 2021-02-10T11:23:34.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.268 7f029b2fd700 1 Processor -- start 2021-02-10T11:23:34.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.268 7f029b2fd700 1 -- start start 2021-02-10T11:23:34.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.269 7f029b2fd700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f02941b0470 0x7f02941b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:34.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.269 7f029b2fd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f02940ef610 0x7f02940efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:34.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.269 7f029b2fd700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02940f0b50 con 0x7f02940f0030 2021-02-10T11:23:34.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.269 7f029b2fd700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02940e52e0 con 0x7f02940ef610 2021-02-10T11:23:34.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.269 7f029b2fd700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02940e5490 con 0x7f02941b0470 2021-02-10T11:23:34.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.269 7f0298918700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f02940ef610 0x7f02940efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:34.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.269 7f0298918700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f02940ef610 0x7f02940efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47208/0 (socket says 172.21.2.101:47208) 2021-02-10T11:23:34.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.269 7f0298918700 1 -- 172.21.2.101:0/3265217884 learned_addr learned my addr 172.21.2.101:0/3265217884 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:34.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.269 7f029991a700 1 --1- 172.21.2.101:0/3265217884 >> v1:172.21.2.101:6789/0 conn(0x7f02940f0030 0x7f02940f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47702/0 (socket says 172.21.2.101:47702) 2021-02-10T11:23:34.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.269 7f0298918700 1 -- 172.21.2.101:0/3265217884 >> v2:172.21.2.108:3300/0 conn(0x7f02941b0470 msgr2=0x7f02941b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:23:34.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.269 7f0298918700 1 --2- 172.21.2.101:0/3265217884 >> v2:172.21.2.108:3300/0 conn(0x7f02941b0470 0x7f02941b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:34.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.269 7f0298918700 1 -- 172.21.2.101:0/3265217884 >> v1:172.21.2.101:6789/0 conn(0x7f02940f0030 legacy=0x7f02940f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:34.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.269 7f0298918700 1 -- 172.21.2.101:0/3265217884 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02940f0430 con 0x7f02940ef610 2021-02-10T11:23:34.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.270 7f0298918700 1 --2- 172.21.2.101:0/3265217884 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02940ef610 0x7f02940efa10 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f0290006700 tx=0x7f0290009b00).ready entity=mon.0 client_cookie=537af5d78d59b305 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:34.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.270 7f0287fff700 1 -- 172.21.2.101:0/3265217884 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f029000c0c0 con 0x7f02940ef610 2021-02-10T11:23:34.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.270 7f0287fff700 1 -- 172.21.2.101:0/3265217884 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029000c340 con 0x7f02940ef610 2021-02-10T11:23:34.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.270 7f0287fff700 1 -- 172.21.2.101:0/3265217884 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f029000c820 con 0x7f02940ef610 2021-02-10T11:23:34.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.270 7f029b2fd700 1 -- 172.21.2.101:0/3265217884 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02940ef610 msgr2=0x7f02940efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:34.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.270 7f029b2fd700 1 --2- 172.21.2.101:0/3265217884 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02940ef610 0x7f02940efa10 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f0290006700 tx=0x7f0290009b00).stop 2021-02-10T11:23:34.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.270 7f029b2fd700 1 -- 172.21.2.101:0/3265217884 shutdown_connections 2021-02-10T11:23:34.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.270 7f029b2fd700 1 --2- 172.21.2.101:0/3265217884 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02940ef610 0x7f02940efa10 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:34.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.270 7f029b2fd700 1 --2- 172.21.2.101:0/3265217884 >> v2:172.21.2.108:3300/0 conn(0x7f02941b0470 0x7f02941b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:34.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.271 7f029b2fd700 1 -- 172.21.2.101:0/3265217884 >> 172.21.2.101:0/3265217884 conn(0x7f02940ce3e0 msgr2=0x7f02940e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:34.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.271 7f029b2fd700 1 -- 172.21.2.101:0/3265217884 shutdown_connections 2021-02-10T11:23:34.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.271 7f029b2fd700 1 -- 172.21.2.101:0/3265217884 wait complete. 2021-02-10T11:23:34.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.285 7f029b2fd700 1 Processor -- start 2021-02-10T11:23:34.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.285 7f029b2fd700 1 -- start start 2021-02-10T11:23:34.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.286 7f029b2fd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02940a98f0 0x7f02941090e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:34.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.286 7f029b2fd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02940a9cf0 0x7f0294109700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:34.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.286 7f029b2fd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02940a02b0 0x7f02940a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:34.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.286 7f029b2fd700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0294108b60 con 0x7f02940a98f0 2021-02-10T11:23:34.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.286 7f029b2fd700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02940a0cd0 con 0x7f02940a9cf0 2021-02-10T11:23:34.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.286 7f029b2fd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02940a6870 con 0x7f02940a02b0 2021-02-10T11:23:34.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.286 7f0299119700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02940a98f0 0x7f02941090e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:34.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.286 7f0299119700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02940a98f0 0x7f02941090e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47212/0 (socket says 172.21.2.101:47212) 2021-02-10T11:23:34.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.286 7f0299119700 1 -- 172.21.2.101:0/1644100565 learned_addr learned my addr 172.21.2.101:0/1644100565 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:34.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.286 7f0299119700 1 -- 172.21.2.101:0/1644100565 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02940a02b0 msgr2=0x7f02940a06b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:23:34.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.286 7f0299119700 1 --2- 172.21.2.101:0/1644100565 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02940a02b0 0x7f02940a06b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:34.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.286 7f0299119700 1 -- 172.21.2.101:0/1644100565 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02940a9cf0 msgr2=0x7f0294109700 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:23:34.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.286 7f0299119700 1 --2- 172.21.2.101:0/1644100565 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02940a9cf0 0x7f0294109700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:34.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.286 7f0299119700 1 -- 172.21.2.101:0/1644100565 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0294107b10 con 0x7f02940a98f0 2021-02-10T11:23:34.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.287 7f0299119700 1 --2- 172.21.2.101:0/1644100565 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02940a98f0 0x7f02941090e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f028800b530 tx=0x7f028800b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:34.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.287 7f0285ffb700 1 -- 172.21.2.101:0/1644100565 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f028800bcc0 con 0x7f02940a98f0 2021-02-10T11:23:34.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.287 7f0285ffb700 1 -- 172.21.2.101:0/1644100565 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0288000c90 con 0x7f02940a98f0 2021-02-10T11:23:34.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.287 7f0285ffb700 1 -- 172.21.2.101:0/1644100565 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f02880022a0 con 0x7f02940a98f0 2021-02-10T11:23:34.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.287 7f029b2fd700 1 -- 172.21.2.101:0/1644100565 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0294107b10 con 0x7f02940a98f0 2021-02-10T11:23:34.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.287 7f029b2fd700 1 -- 172.21.2.101:0/1644100565 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02941080a0 con 0x7f02940a98f0 2021-02-10T11:23:34.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.287 7f0285ffb700 1 -- 172.21.2.101:0/1644100565 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0288005310 con 0x7f02940a98f0 2021-02-10T11:23:34.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.287 7f0285ffb700 1 --2- 172.21.2.101:0/1644100565 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02740186d0 0x7f027401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:34.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.287 7f0285ffb700 1 -- 172.21.2.101:0/1644100565 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0288032610 con 0x7f02940a98f0 2021-02-10T11:23:34.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.288 7f0298918700 1 --2- 172.21.2.101:0/1644100565 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02740186d0 0x7f027401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:34.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.288 7f0298918700 1 --2- 172.21.2.101:0/1644100565 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02740186d0 0x7f027401abd0 crc :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:34.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.318 7f029b2fd700 1 -- 172.21.2.101:0/1644100565 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02940f0b50 con 0x7f02940a98f0 2021-02-10T11:23:34.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.321 7f0285ffb700 1 -- 172.21.2.101:0/1644100565 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0288006b90 con 0x7f02940a98f0 2021-02-10T11:23:34.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.450 7f029b2fd700 1 -- 172.21.2.101:0/1644100565 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839294", "target": ["mgr", ""]}) v1 -- 0x7f02941080a0 con 0x7f02740186d0 2021-02-10T11:23:34.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.474 7f0285ffb700 1 -- 172.21.2.101:0/1644100565 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f0290009390 con 0x7f02740186d0 2021-02-10T11:23:34.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.483 7f029b2fd700 1 -- 172.21.2.101:0/1644100565 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02740186d0 msgr2=0x7f027401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:34.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.483 7f029b2fd700 1 --2- 172.21.2.101:0/1644100565 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02740186d0 0x7f027401abd0 crc :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:34.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.483 7f029b2fd700 1 -- 172.21.2.101:0/1644100565 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02940a98f0 msgr2=0x7f02941090e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:34.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.483 7f029b2fd700 1 --2- 172.21.2.101:0/1644100565 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02940a98f0 0x7f02941090e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f028800b530 tx=0x7f028800b8d0).stop 2021-02-10T11:23:34.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.483 7f029b2fd700 1 -- 172.21.2.101:0/1644100565 shutdown_connections 2021-02-10T11:23:34.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.483 7f029b2fd700 1 --2- 172.21.2.101:0/1644100565 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02740186d0 0x7f027401abd0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:34.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.483 7f029b2fd700 1 --2- 172.21.2.101:0/1644100565 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02940a02b0 0x7f02940a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:34.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.483 7f029b2fd700 1 --2- 172.21.2.101:0/1644100565 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02940a9cf0 0x7f0294109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:34.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.483 7f029b2fd700 1 --2- 172.21.2.101:0/1644100565 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02940a98f0 0x7f02941090e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:34.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.483 7f029b2fd700 1 -- 172.21.2.101:0/1644100565 >> 172.21.2.101:0/1644100565 conn(0x7f02940a9360 msgr2=0x7f02940b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:34.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.483 7f029b2fd700 1 -- 172.21.2.101:0/1644100565 shutdown_connections 2021-02-10T11:23:34.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.483 7f029b2fd700 1 -- 172.21.2.101:0/1644100565 wait complete. 2021-02-10T11:23:34.499 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839294 2021-02-10T11:23:34.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.905 7f28709b1700 1 Processor -- start 2021-02-10T11:23:34.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.905 7f28709b1700 1 -- start start 2021-02-10T11:23:34.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.905 7f28709b1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f286c1b0470 0x7f286c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:34.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.905 7f28709b1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f286c0ef610 0x7f286c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:34.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.906 7f28709b1700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f286c0f0b50 con 0x7f286c0f0030 2021-02-10T11:23:34.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.906 7f28709b1700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f286c0e52e0 con 0x7f286c0ef610 2021-02-10T11:23:34.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.906 7f28709b1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f286c0e5490 con 0x7f286c1b0470 2021-02-10T11:23:34.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.906 7f2869e1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f286c0ef610 0x7f286c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:34.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.906 7f2869e1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f286c0ef610 0x7f286c0efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47220/0 (socket says 172.21.2.101:47220) 2021-02-10T11:23:34.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.906 7f2869e1b700 1 -- 172.21.2.101:0/3449852429 learned_addr learned my addr 172.21.2.101:0/3449852429 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:34.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.906 7f2869e1b700 1 -- 172.21.2.101:0/3449852429 >> v2:172.21.2.108:3300/0 conn(0x7f286c1b0470 msgr2=0x7f286c1b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:23:34.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.906 7f2869e1b700 1 --2- 172.21.2.101:0/3449852429 >> v2:172.21.2.108:3300/0 conn(0x7f286c1b0470 0x7f286c1b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:34.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.906 7f2869e1b700 1 -- 172.21.2.101:0/3449852429 >> v1:172.21.2.108:6789/0 conn(0x7f286c0f0030 legacy=0x7f286c0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:23:34.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.906 7f2869e1b700 1 -- 172.21.2.101:0/3449852429 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286c0f0430 con 0x7f286c0ef610 2021-02-10T11:23:34.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.906 7f2869e1b700 1 --2- 172.21.2.101:0/3449852429 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f286c0ef610 0x7f286c0efa10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f285c006700 tx=0x7f285c009b80).ready entity=mon.0 client_cookie=ad943b98075ac12d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:34.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.907 7f286961a700 1 -- 172.21.2.101:0/3449852429 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f285c00c0c0 con 0x7f286c0ef610 2021-02-10T11:23:34.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.907 7f286961a700 1 -- 172.21.2.101:0/3449852429 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285c00c340 con 0x7f286c0ef610 2021-02-10T11:23:34.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.907 7f286961a700 1 -- 172.21.2.101:0/3449852429 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f285c00c860 con 0x7f286c0ef610 2021-02-10T11:23:34.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.907 7f28709b1700 1 -- 172.21.2.101:0/3449852429 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f286c0ef610 msgr2=0x7f286c0efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:34.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.907 7f28709b1700 1 --2- 172.21.2.101:0/3449852429 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f286c0ef610 0x7f286c0efa10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f285c006700 tx=0x7f285c009b80).stop 2021-02-10T11:23:34.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.907 7f28709b1700 1 -- 172.21.2.101:0/3449852429 shutdown_connections 2021-02-10T11:23:34.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.907 7f28709b1700 1 --2- 172.21.2.101:0/3449852429 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f286c0ef610 0x7f286c0efa10 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:34.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.907 7f28709b1700 1 --2- 172.21.2.101:0/3449852429 >> v2:172.21.2.108:3300/0 conn(0x7f286c1b0470 0x7f286c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:34.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.907 7f28709b1700 1 -- 172.21.2.101:0/3449852429 >> 172.21.2.101:0/3449852429 conn(0x7f286c0ce3e0 msgr2=0x7f286c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:34.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.908 7f28709b1700 1 -- 172.21.2.101:0/3449852429 shutdown_connections 2021-02-10T11:23:34.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.908 7f28709b1700 1 -- 172.21.2.101:0/3449852429 wait complete. 2021-02-10T11:23:34.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.922 7f28709b1700 1 Processor -- start 2021-02-10T11:23:34.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.922 7f28709b1700 1 -- start start 2021-02-10T11:23:34.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.923 7f28709b1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f286c094fb0 0x7f286c09e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:34.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.923 7f28709b1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f286c0c2be0 0x7f286c0c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:34.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.923 7f28709b1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f286c0c3600 0x7f286c0a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:34.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.923 7f28709b1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286c09ebf0 con 0x7f286c094fb0 2021-02-10T11:23:34.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.923 7f28709b1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286c0aa2a0 con 0x7f286c0c2be0 2021-02-10T11:23:34.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.923 7f28709b1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286c0aa4c0 con 0x7f286c0c3600 2021-02-10T11:23:34.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.923 7f286a61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f286c094fb0 0x7f286c09e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:34.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.923 7f286a61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f286c094fb0 0x7f286c09e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47224/0 (socket says 172.21.2.101:47224) 2021-02-10T11:23:34.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.923 7f286a61c700 1 -- 172.21.2.101:0/3353056230 learned_addr learned my addr 172.21.2.101:0/3353056230 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:34.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.923 7f286a61c700 1 -- 172.21.2.101:0/3353056230 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f286c0c3600 msgr2=0x7f286c0a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:34.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.923 7f286a61c700 1 --2- 172.21.2.101:0/3353056230 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f286c0c3600 0x7f286c0a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:34.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.923 7f286a61c700 1 -- 172.21.2.101:0/3353056230 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f286c0c2be0 msgr2=0x7f286c0c2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:34.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.923 7f286a61c700 1 --2- 172.21.2.101:0/3353056230 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f286c0c2be0 0x7f286c0c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:34.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.923 7f286a61c700 1 -- 172.21.2.101:0/3353056230 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286c0aa6e0 con 0x7f286c094fb0 2021-02-10T11:23:34.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.923 7f286a61c700 1 --2- 172.21.2.101:0/3353056230 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f286c094fb0 0x7f286c09e5d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f285400b5c0 tx=0x7f285400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:34.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.924 7f2862ffd700 1 -- 172.21.2.101:0/3353056230 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2854000c40 con 0x7f286c094fb0 2021-02-10T11:23:34.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.924 7f2862ffd700 1 -- 172.21.2.101:0/3353056230 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285400bd50 con 0x7f286c094fb0 2021-02-10T11:23:34.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.924 7f2862ffd700 1 -- 172.21.2.101:0/3353056230 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f28540054d0 con 0x7f286c094fb0 2021-02-10T11:23:34.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.924 7f28709b1700 1 -- 172.21.2.101:0/3353056230 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f286c0aa6e0 con 0x7f286c094fb0 2021-02-10T11:23:34.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.924 7f28709b1700 1 -- 172.21.2.101:0/3353056230 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286c0aac70 con 0x7f286c094fb0 2021-02-10T11:23:34.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.924 7f2862ffd700 1 -- 172.21.2.101:0/3353056230 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f28540054d0 con 0x7f286c094fb0 2021-02-10T11:23:34.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.924 7f2862ffd700 1 --2- 172.21.2.101:0/3353056230 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28500186c0 0x7f285001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:34.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.924 7f2862ffd700 1 -- 172.21.2.101:0/3353056230 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2854032600 con 0x7f286c094fb0 2021-02-10T11:23:34.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.924 7f2869e1b700 1 --2- 172.21.2.101:0/3353056230 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28500186c0 0x7f285001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:34.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.925 7f2869e1b700 1 --2- 172.21.2.101:0/3353056230 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28500186c0 0x7f285001abc0 crc :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:34.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.955 7f28709b1700 1 -- 172.21.2.101:0/3353056230 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f286c0a9750 con 0x7f286c094fb0 2021-02-10T11:23:34.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:34.958 7f2862ffd700 1 -- 172.21.2.101:0/3353056230 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2854030db0 con 0x7f286c094fb0 2021-02-10T11:23:35.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:35.086 7f28709b1700 1 -- 172.21.2.101:0/3353056230 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839294", "target": ["mgr", ""]}) v1 -- 0x7f286c0a9750 con 0x7f28500186c0 2021-02-10T11:23:35.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:35.089 7f2862ffd700 1 -- 172.21.2.101:0/3353056230 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f285c000970 con 0x7f28500186c0 2021-02-10T11:23:35.101 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839294/eb55c3d1-2f08-49be-aab1-26b90ed05e82 2021-02-10T11:23:35.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:35.094 7f28709b1700 1 -- 172.21.2.101:0/3353056230 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28500186c0 msgr2=0x7f285001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:35.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:35.094 7f28709b1700 1 --2- 172.21.2.101:0/3353056230 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28500186c0 0x7f285001abc0 crc :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:35.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:35.094 7f28709b1700 1 -- 172.21.2.101:0/3353056230 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f286c094fb0 msgr2=0x7f286c09e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:35.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:35.094 7f28709b1700 1 --2- 172.21.2.101:0/3353056230 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f286c094fb0 0x7f286c09e5d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f285400b5c0 tx=0x7f285400b960).stop 2021-02-10T11:23:35.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:35.094 7f28709b1700 1 -- 172.21.2.101:0/3353056230 shutdown_connections 2021-02-10T11:23:35.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:35.094 7f28709b1700 1 --2- 172.21.2.101:0/3353056230 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28500186c0 0x7f285001abc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:35.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:35.094 7f28709b1700 1 --2- 172.21.2.101:0/3353056230 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f286c0c3600 0x7f286c0a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:35.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:35.094 7f28709b1700 1 --2- 172.21.2.101:0/3353056230 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f286c0c2be0 0x7f286c0c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:35.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:35.094 7f28709b1700 1 --2- 172.21.2.101:0/3353056230 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f286c094fb0 0x7f286c09e5d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:35.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:35.094 7f28709b1700 1 -- 172.21.2.101:0/3353056230 >> 172.21.2.101:0/3353056230 conn(0x7f286c09f630 msgr2=0x7f286c0a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:35.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:35.094 7f28709b1700 1 -- 172.21.2.101:0/3353056230 shutdown_connections 2021-02-10T11:23:35.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:35.095 7f28709b1700 1 -- 172.21.2.101:0/3353056230 wait complete. 2021-02-10T11:23:35.110 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839294 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839294/eb55c3d1-2f08-49be-aab1-26b90ed05e82 2021-02-10T11:23:35.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:35.111 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:35.111 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:35.446 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:35.446 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:35.446 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839294/eb55c3d1-2f08-49be-aab1-26b90ed05e82/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:35.825 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:35.825 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:35.826 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0221229 s, 47.4 MB/s 2021-02-10T11:23:35.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:35.827 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:35.827 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:36.163 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:36.163 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:36.163 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839294/eb55c3d1-2f08-49be-aab1-26b90ed05e82/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:36.540 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:36.540 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:36.540 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237585 s, 44.1 MB/s 2021-02-10T11:23:36.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:36.541 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:36.541 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:36.875 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:36.875 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:36.876 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839294/eb55c3d1-2f08-49be-aab1-26b90ed05e82/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:37.250 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:37.250 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:37.250 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0232839 s, 45.0 MB/s 2021-02-10T11:23:37.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:37.251 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:37.251 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:37.587 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:37.587 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:37.587 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839294/eb55c3d1-2f08-49be-aab1-26b90ed05e82/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:37.964 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:37.964 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:37.964 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242565 s, 43.2 MB/s 2021-02-10T11:23:37.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:37.965 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:37.965 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:38.300 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:38.300 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:38.300 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839294/eb55c3d1-2f08-49be-aab1-26b90ed05e82/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:38.675 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:38.676 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:38.676 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024227 s, 43.3 MB/s 2021-02-10T11:23:38.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:38.677 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:38.677 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:39.012 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:39.012 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:39.012 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839294/eb55c3d1-2f08-49be-aab1-26b90ed05e82/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:39.384 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:39.384 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:39.384 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242726 s, 43.2 MB/s 2021-02-10T11:23:39.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:39.385 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:39.385 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:39.720 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:39.720 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:39.720 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839294/eb55c3d1-2f08-49be-aab1-26b90ed05e82/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:40.096 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:40.096 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:40.097 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.02358 s, 44.5 MB/s 2021-02-10T11:23:40.097 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:40.098 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:40.098 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:40.434 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:40.435 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:40.435 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839294/eb55c3d1-2f08-49be-aab1-26b90ed05e82/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:40.809 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:40.809 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:40.810 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.022613 s, 46.4 MB/s 2021-02-10T11:23:40.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:40.811 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:40.811 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:41.147 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:41.147 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:41.147 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839294/eb55c3d1-2f08-49be-aab1-26b90ed05e82/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:41.524 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:41.524 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:41.525 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247792 s, 42.3 MB/s 2021-02-10T11:23:41.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:41.526 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:41.526 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:41.862 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:41.862 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:41.863 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839294/eb55c3d1-2f08-49be-aab1-26b90ed05e82/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:42.238 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:42.238 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:42.238 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249511 s, 42.0 MB/s 2021-02-10T11:23:42.239 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839295 2021-02-10T11:23:42.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.646 7f5cda07e700 1 Processor -- start 2021-02-10T11:23:42.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.646 7f5cda07e700 1 -- start start 2021-02-10T11:23:42.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.646 7f5cda07e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5cd41b0470 0x7f5cd41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:42.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.646 7f5cda07e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5cd40ef610 0x7f5cd40efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:42.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.646 7f5cda07e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f5cd40f0030 0x7f5cd40f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:42.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.646 7f5cda07e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd40e5ea0 con 0x7f5cd41b0470 2021-02-10T11:23:42.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.646 7f5cda07e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd40f0b50 con 0x7f5cd40f0030 2021-02-10T11:23:42.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.646 7f5cda07e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd40e52e0 con 0x7f5cd40ef610 2021-02-10T11:23:42.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.647 7f5cd37fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5cd41b0470 0x7f5cd41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:42.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.647 7f5cd37fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5cd41b0470 0x7f5cd41b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47232/0 (socket says 172.21.2.101:47232) 2021-02-10T11:23:42.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.647 7f5cd37fe700 1 -- 172.21.2.101:0/2049340792 learned_addr learned my addr 172.21.2.101:0/2049340792 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:42.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.647 7f5cd37fe700 1 -- 172.21.2.101:0/2049340792 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5cd40ef610 msgr2=0x7f5cd40efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:23:42.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.647 7f5cd37fe700 1 --2- 172.21.2.101:0/2049340792 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5cd40ef610 0x7f5cd40efa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:42.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.647 7f5cd37fe700 1 -- 172.21.2.101:0/2049340792 >> v2:172.21.2.108:3300/0 conn(0x7f5cd40f0030 msgr2=0x7f5cd40f0430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:23:42.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.647 7f5cd37fe700 1 --2- 172.21.2.101:0/2049340792 >> v2:172.21.2.108:3300/0 conn(0x7f5cd40f0030 0x7f5cd40f0430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:42.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.647 7f5cd37fe700 1 -- 172.21.2.101:0/2049340792 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd40f10d0 con 0x7f5cd41b0470 2021-02-10T11:23:42.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.647 7f5cd37fe700 1 --2- 172.21.2.101:0/2049340792 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5cd41b0470 0x7f5cd41b0870 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f5cc4006700 tx=0x7f5cc4009b80).ready entity=mon.0 client_cookie=dce4bb95661f2cc9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:42.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.647 7f5cd27fc700 1 -- 172.21.2.101:0/2049340792 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5cc400c0c0 con 0x7f5cd41b0470 2021-02-10T11:23:42.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.648 7f5cd27fc700 1 -- 172.21.2.101:0/2049340792 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc400c380 con 0x7f5cd41b0470 2021-02-10T11:23:42.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.648 7f5cd27fc700 1 -- 172.21.2.101:0/2049340792 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5cc400c8a0 con 0x7f5cd41b0470 2021-02-10T11:23:42.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.648 7f5cda07e700 1 -- 172.21.2.101:0/2049340792 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5cd41b0470 msgr2=0x7f5cd41b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:42.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.648 7f5cda07e700 1 --2- 172.21.2.101:0/2049340792 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5cd41b0470 0x7f5cd41b0870 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f5cc4006700 tx=0x7f5cc4009b80).stop 2021-02-10T11:23:42.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.648 7f5cda07e700 1 -- 172.21.2.101:0/2049340792 shutdown_connections 2021-02-10T11:23:42.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.648 7f5cda07e700 1 --2- 172.21.2.101:0/2049340792 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5cd40ef610 0x7f5cd40efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:42.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.648 7f5cda07e700 1 --2- 172.21.2.101:0/2049340792 >> v2:172.21.2.108:3300/0 conn(0x7f5cd40f0030 0x7f5cd40f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:42.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.648 7f5cda07e700 1 --2- 172.21.2.101:0/2049340792 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5cd41b0470 0x7f5cd41b0870 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:42.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.648 7f5cda07e700 1 -- 172.21.2.101:0/2049340792 >> 172.21.2.101:0/2049340792 conn(0x7f5cd40ce3e0 msgr2=0x7f5cd40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:42.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.648 7f5cda07e700 1 -- 172.21.2.101:0/2049340792 shutdown_connections 2021-02-10T11:23:42.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.648 7f5cda07e700 1 -- 172.21.2.101:0/2049340792 wait complete. 2021-02-10T11:23:42.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.662 7f5cda07e700 1 Processor -- start 2021-02-10T11:23:42.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.663 7f5cda07e700 1 -- start start 2021-02-10T11:23:42.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.663 7f5cda07e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5cd41a42f0 0x7f5cd41a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:42.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.663 7f5cda07e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5cd4196170 0x7f5cd4196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:42.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.663 7f5cda07e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5cd408f5b0 0x7f5cd408f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:42.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.663 7f5cda07e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd4195bf0 con 0x7f5cd408f5b0 2021-02-10T11:23:42.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.663 7f5cda07e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd40c4130 con 0x7f5cd4196170 2021-02-10T11:23:42.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.663 7f5cda07e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd40a6e70 con 0x7f5cd41a42f0 2021-02-10T11:23:42.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.663 7f5cd3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5cd408f5b0 0x7f5cd408f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:42.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.663 7f5cd3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5cd408f5b0 0x7f5cd408f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47236/0 (socket says 172.21.2.101:47236) 2021-02-10T11:23:42.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.663 7f5cd3fff700 1 -- 172.21.2.101:0/1917601205 learned_addr learned my addr 172.21.2.101:0/1917601205 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:42.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.664 7f5cd3fff700 1 -- 172.21.2.101:0/1917601205 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5cd41a42f0 msgr2=0x7f5cd41a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:42.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.664 7f5cd3fff700 1 --2- 172.21.2.101:0/1917601205 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5cd41a42f0 0x7f5cd41a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:42.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.664 7f5cd3fff700 1 -- 172.21.2.101:0/1917601205 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5cd4196170 msgr2=0x7f5cd4196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:42.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.664 7f5cd3fff700 1 --2- 172.21.2.101:0/1917601205 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5cd4196170 0x7f5cd4196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:42.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.664 7f5cd3fff700 1 -- 172.21.2.101:0/1917601205 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd40a6a80 con 0x7f5cd408f5b0 2021-02-10T11:23:42.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.664 7f5cd3fff700 1 --2- 172.21.2.101:0/1917601205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5cd408f5b0 0x7f5cd408f9b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f5ccc009960 tx=0x7f5ccc00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:42.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.664 7f5ccbfff700 1 -- 172.21.2.101:0/1917601205 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5ccc000f70 con 0x7f5cd408f5b0 2021-02-10T11:23:42.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.664 7f5ccbfff700 1 -- 172.21.2.101:0/1917601205 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ccc003030 con 0x7f5cd408f5b0 2021-02-10T11:23:42.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.664 7f5ccbfff700 1 -- 172.21.2.101:0/1917601205 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5ccc001590 con 0x7f5cd408f5b0 2021-02-10T11:23:42.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.664 7f5cda07e700 1 -- 172.21.2.101:0/1917601205 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd40a6a80 con 0x7f5cd408f5b0 2021-02-10T11:23:42.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.664 7f5cda07e700 1 -- 172.21.2.101:0/1917601205 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd40a6a80 con 0x7f5cd408f5b0 2021-02-10T11:23:42.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.665 7f5ccbfff700 1 -- 172.21.2.101:0/1917601205 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5ccc001310 con 0x7f5cd408f5b0 2021-02-10T11:23:42.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.665 7f5ccbfff700 1 --2- 172.21.2.101:0/1917601205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5cb80186c0 0x7f5cb801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:42.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.665 7f5ccbfff700 1 -- 172.21.2.101:0/1917601205 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5ccc0328f0 con 0x7f5cd408f5b0 2021-02-10T11:23:42.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.665 7f5cd37fe700 1 --2- 172.21.2.101:0/1917601205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5cb80186c0 0x7f5cb801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:42.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.665 7f5cd37fe700 1 --2- 172.21.2.101:0/1917601205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5cb80186c0 0x7f5cb801abc0 crc :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:42.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.697 7f5cda07e700 1 -- 172.21.2.101:0/1917601205 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd40f0b50 con 0x7f5cd408f5b0 2021-02-10T11:23:42.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.699 7f5ccbfff700 1 -- 172.21.2.101:0/1917601205 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5ccc00be30 con 0x7f5cd408f5b0 2021-02-10T11:23:42.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.823 7f5cda07e700 1 -- 172.21.2.101:0/1917601205 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839295", "target": ["mgr", ""]}) v1 -- 0x7f5cd40f0b50 con 0x7f5cb80186c0 2021-02-10T11:23:42.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.850 7f5ccbfff700 1 -- 172.21.2.101:0/1917601205 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f5cc4000990 con 0x7f5cb80186c0 2021-02-10T11:23:42.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.857 7f5cda07e700 1 -- 172.21.2.101:0/1917601205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5cb80186c0 msgr2=0x7f5cb801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:42.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.857 7f5cda07e700 1 --2- 172.21.2.101:0/1917601205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5cb80186c0 0x7f5cb801abc0 crc :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:42.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.857 7f5cda07e700 1 -- 172.21.2.101:0/1917601205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5cd408f5b0 msgr2=0x7f5cd408f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:42.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.857 7f5cda07e700 1 --2- 172.21.2.101:0/1917601205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5cd408f5b0 0x7f5cd408f9b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f5ccc009960 tx=0x7f5ccc00bcf0).stop 2021-02-10T11:23:42.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.857 7f5cda07e700 1 -- 172.21.2.101:0/1917601205 shutdown_connections 2021-02-10T11:23:42.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.857 7f5cda07e700 1 --2- 172.21.2.101:0/1917601205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5cd408f5b0 0x7f5cd408f9b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:42.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.857 7f5cda07e700 1 --2- 172.21.2.101:0/1917601205 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5cd4196170 0x7f5cd4196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:42.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.857 7f5cda07e700 1 --2- 172.21.2.101:0/1917601205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5cb80186c0 0x7f5cb801abc0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:42.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.857 7f5cda07e700 1 --2- 172.21.2.101:0/1917601205 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5cd41a42f0 0x7f5cd41a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:42.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.857 7f5cda07e700 1 -- 172.21.2.101:0/1917601205 >> 172.21.2.101:0/1917601205 conn(0x7f5cd40bf4e0 msgr2=0x7f5cd40be160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:42.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.857 7f5cda07e700 1 -- 172.21.2.101:0/1917601205 shutdown_connections 2021-02-10T11:23:42.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:42.857 7f5cda07e700 1 -- 172.21.2.101:0/1917601205 wait complete. 2021-02-10T11:23:42.874 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839295 2021-02-10T11:23:43.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.280 7fe53a289700 1 Processor -- start 2021-02-10T11:23:43.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.280 7fe53a289700 1 -- start start 2021-02-10T11:23:43.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.280 7fe53a289700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe5341b0470 0x7fe5341b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:43.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.280 7fe53a289700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5340f0b50 con 0x7fe5340f20e0 2021-02-10T11:23:43.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.280 7fe53a289700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5340f24e0 con 0x7fe5340ef610 2021-02-10T11:23:43.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.280 7fe53a289700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5340e5ea0 con 0x7fe5341b0470 2021-02-10T11:23:43.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.281 7fe533fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe5341b0470 0x7fe5341b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:43.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.281 7fe5388a6700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe5340f20e0 0x7fe5340f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47736/0 (socket says 172.21.2.101:47736) 2021-02-10T11:23:43.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.281 7fe5388a6700 1 -- 172.21.2.101:0/3917098791 learned_addr learned my addr 172.21.2.101:0/3917098791 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:23:43.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.281 7fe532ffd700 1 -- 172.21.2.101:0/3917098791 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706195715 0 0) 0x7fe52c001500 con 0x7fe5340f20e0 2021-02-10T11:23:43.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.281 7fe532ffd700 1 -- 172.21.2.101:0/3917098791 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe51c002fe0 con 0x7fe5340f20e0 2021-02-10T11:23:43.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.281 7fe533fff700 1 -- 172.21.2.101:0/3917098791 >> v1:172.21.2.108:6789/0 conn(0x7fe5340ef610 legacy=0x7fe5340efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:23:43.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.281 7fe533fff700 1 -- 172.21.2.101:0/3917098791 >> v1:172.21.2.101:6789/0 conn(0x7fe5340f20e0 legacy=0x7fe5340f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:43.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.281 7fe533fff700 1 -- 172.21.2.101:0/3917098791 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5340e52e0 con 0x7fe5341b0470 2021-02-10T11:23:43.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.281 7fe533fff700 1 --2- 172.21.2.101:0/3917098791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5341b0470 0x7fe5341b0870 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7fe524006650 tx=0x7fe524009a80).ready entity=mon.0 client_cookie=4e10311ad9d0597a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:43.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.282 7fe532ffd700 1 -- 172.21.2.101:0/3917098791 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe52400c0c0 con 0x7fe5341b0470 2021-02-10T11:23:43.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.282 7fe532ffd700 1 -- 172.21.2.101:0/3917098791 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52400c340 con 0x7fe5341b0470 2021-02-10T11:23:43.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.282 7fe532ffd700 1 -- 172.21.2.101:0/3917098791 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe52400c860 con 0x7fe5341b0470 2021-02-10T11:23:43.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.282 7fe53a289700 1 -- 172.21.2.101:0/3917098791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5341b0470 msgr2=0x7fe5341b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:43.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.282 7fe53a289700 1 --2- 172.21.2.101:0/3917098791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5341b0470 0x7fe5341b0870 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7fe524006650 tx=0x7fe524009a80).stop 2021-02-10T11:23:43.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.282 7fe53a289700 1 -- 172.21.2.101:0/3917098791 shutdown_connections 2021-02-10T11:23:43.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.282 7fe53a289700 1 --2- 172.21.2.101:0/3917098791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5341b0470 0x7fe5341b0870 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:43.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.282 7fe53a289700 1 -- 172.21.2.101:0/3917098791 >> 172.21.2.101:0/3917098791 conn(0x7fe5340ce3e0 msgr2=0x7fe5340e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:43.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.282 7fe53a289700 1 -- 172.21.2.101:0/3917098791 shutdown_connections 2021-02-10T11:23:43.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.282 7fe53a289700 1 -- 172.21.2.101:0/3917098791 wait complete. 2021-02-10T11:23:43.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.297 7fe53a289700 1 Processor -- start 2021-02-10T11:23:43.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.297 7fe53a289700 1 -- start start 2021-02-10T11:23:43.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.298 7fe53a289700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5340a0960 0x7fe5340a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:43.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.298 7fe53a289700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5341924c0 0x7fe5341928c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:43.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.298 7fe53a289700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe534192ee0 0x7fe5340a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:43.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.298 7fe53a289700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5340a6e10 con 0x7fe534192ee0 2021-02-10T11:23:43.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.298 7fe53a289700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe534190630 con 0x7fe5341924c0 2021-02-10T11:23:43.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.298 7fe53a289700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5341932e0 con 0x7fe5340a0960 2021-02-10T11:23:43.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.298 7fe5388a6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe534192ee0 0x7fe5340a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:43.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.298 7fe5388a6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe534192ee0 0x7fe5340a29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47246/0 (socket says 172.21.2.101:47246) 2021-02-10T11:23:43.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.298 7fe5388a6700 1 -- 172.21.2.101:0/4243569977 learned_addr learned my addr 172.21.2.101:0/4243569977 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:43.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.298 7fe5388a6700 1 -- 172.21.2.101:0/4243569977 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5340a0960 msgr2=0x7fe5340a67f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:23:43.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.298 7fe5388a6700 1 --2- 172.21.2.101:0/4243569977 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5340a0960 0x7fe5340a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:43.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.298 7fe5388a6700 1 -- 172.21.2.101:0/4243569977 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5341924c0 msgr2=0x7fe5341928c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:23:43.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.298 7fe5388a6700 1 --2- 172.21.2.101:0/4243569977 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5341924c0 0x7fe5341928c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:43.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.298 7fe5388a6700 1 -- 172.21.2.101:0/4243569977 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe534193500 con 0x7fe534192ee0 2021-02-10T11:23:43.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.298 7fe5388a6700 1 --2- 172.21.2.101:0/4243569977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe534192ee0 0x7fe5340a29f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7fe52c00a8f0 tx=0x7fe52c00ac90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:43.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.299 7fe530ff9700 1 -- 172.21.2.101:0/4243569977 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe52c001270 con 0x7fe534192ee0 2021-02-10T11:23:43.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.299 7fe530ff9700 1 -- 172.21.2.101:0/4243569977 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52c0070d0 con 0x7fe534192ee0 2021-02-10T11:23:43.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.299 7fe53a289700 1 -- 172.21.2.101:0/4243569977 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe534193500 con 0x7fe534192ee0 2021-02-10T11:23:43.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.299 7fe53a289700 1 -- 172.21.2.101:0/4243569977 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5340a3300 con 0x7fe534192ee0 2021-02-10T11:23:43.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.299 7fe530ff9700 1 -- 172.21.2.101:0/4243569977 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe52c004720 con 0x7fe534192ee0 2021-02-10T11:23:43.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.299 7fe530ff9700 1 -- 172.21.2.101:0/4243569977 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe52c0072b0 con 0x7fe534192ee0 2021-02-10T11:23:43.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.299 7fe530ff9700 1 --2- 172.21.2.101:0/4243569977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5140186c0 0x7fe51401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:43.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.299 7fe530ff9700 1 -- 172.21.2.101:0/4243569977 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe52c032af0 con 0x7fe534192ee0 2021-02-10T11:23:43.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.299 7fe533fff700 1 --2- 172.21.2.101:0/4243569977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5140186c0 0x7fe51401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:43.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.300 7fe533fff700 1 --2- 172.21.2.101:0/4243569977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5140186c0 0x7fe51401abc0 crc :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:43.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.330 7fe53a289700 1 -- 172.21.2.101:0/4243569977 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5340a24c0 con 0x7fe534192ee0 2021-02-10T11:23:43.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.333 7fe530ff9700 1 -- 172.21.2.101:0/4243569977 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe52c001270 con 0x7fe534192ee0 2021-02-10T11:23:43.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.459 7fe53a289700 1 -- 172.21.2.101:0/4243569977 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839295", "target": ["mgr", ""]}) v1 -- 0x7fe5340a6e10 con 0x7fe5140186c0 2021-02-10T11:23:43.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.462 7fe530ff9700 1 -- 172.21.2.101:0/4243569977 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fe524006390 con 0x7fe5140186c0 2021-02-10T11:23:43.475 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839295/3eb4e8ca-ae3f-4e1f-ba46-a9b925d228e7 2021-02-10T11:23:43.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.468 7fe53a289700 1 -- 172.21.2.101:0/4243569977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5140186c0 msgr2=0x7fe51401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:43.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.468 7fe53a289700 1 --2- 172.21.2.101:0/4243569977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5140186c0 0x7fe51401abc0 crc :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:43.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.468 7fe53a289700 1 -- 172.21.2.101:0/4243569977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe534192ee0 msgr2=0x7fe5340a29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:43.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.468 7fe53a289700 1 --2- 172.21.2.101:0/4243569977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe534192ee0 0x7fe5340a29f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7fe52c00a8f0 tx=0x7fe52c00ac90).stop 2021-02-10T11:23:43.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.468 7fe53a289700 1 -- 172.21.2.101:0/4243569977 shutdown_connections 2021-02-10T11:23:43.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.468 7fe53a289700 1 --2- 172.21.2.101:0/4243569977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe534192ee0 0x7fe5340a29f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:43.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.468 7fe53a289700 1 --2- 172.21.2.101:0/4243569977 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5341924c0 0x7fe5341928c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:43.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.468 7fe53a289700 1 --2- 172.21.2.101:0/4243569977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5140186c0 0x7fe51401abc0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:43.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.468 7fe53a289700 1 --2- 172.21.2.101:0/4243569977 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5340a0960 0x7fe5340a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:43.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.468 7fe53a289700 1 -- 172.21.2.101:0/4243569977 >> 172.21.2.101:0/4243569977 conn(0x7fe5340ac550 msgr2=0x7fe5340be1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:43.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.468 7fe53a289700 1 -- 172.21.2.101:0/4243569977 shutdown_connections 2021-02-10T11:23:43.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:43.468 7fe53a289700 1 -- 172.21.2.101:0/4243569977 wait complete. 2021-02-10T11:23:43.485 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839295 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839295/3eb4e8ca-ae3f-4e1f-ba46-a9b925d228e7 2021-02-10T11:23:43.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:43.485 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:43.485 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:43.820 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:43.820 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:43.820 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839295/3eb4e8ca-ae3f-4e1f-ba46-a9b925d228e7/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:44.199 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:44.199 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:44.199 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241869 s, 43.4 MB/s 2021-02-10T11:23:44.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:44.200 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:44.200 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:44.539 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:44.539 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:44.539 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839295/3eb4e8ca-ae3f-4e1f-ba46-a9b925d228e7/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:44.921 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:44.922 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:44.922 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238722 s, 43.9 MB/s 2021-02-10T11:23:44.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:44.923 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:44.923 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:45.257 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:45.257 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:45.258 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839295/3eb4e8ca-ae3f-4e1f-ba46-a9b925d228e7/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:45.634 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:45.634 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:45.634 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245682 s, 42.7 MB/s 2021-02-10T11:23:45.635 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:45.636 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:45.636 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:45.971 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:45.971 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:45.971 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839295/3eb4e8ca-ae3f-4e1f-ba46-a9b925d228e7/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:46.348 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:46.349 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:46.349 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240179 s, 43.7 MB/s 2021-02-10T11:23:46.349 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:46.350 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:46.350 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:46.686 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:46.686 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:46.687 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839295/3eb4e8ca-ae3f-4e1f-ba46-a9b925d228e7/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:47.063 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:47.063 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:47.063 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252687 s, 41.5 MB/s 2021-02-10T11:23:47.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:47.064 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:47.064 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:47.399 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:47.399 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:47.400 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839295/3eb4e8ca-ae3f-4e1f-ba46-a9b925d228e7/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:47.772 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:47.773 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:47.773 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0228169 s, 46.0 MB/s 2021-02-10T11:23:47.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:47.774 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:47.774 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:48.110 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:48.111 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:48.111 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839295/3eb4e8ca-ae3f-4e1f-ba46-a9b925d228e7/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:48.488 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:48.488 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:48.488 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242871 s, 43.2 MB/s 2021-02-10T11:23:48.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:48.489 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:48.489 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:48.823 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:48.824 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:48.824 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839295/3eb4e8ca-ae3f-4e1f-ba46-a9b925d228e7/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:49.195 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:49.196 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:49.196 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238851 s, 43.9 MB/s 2021-02-10T11:23:49.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:49.197 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:49.197 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:49.531 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:49.531 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:49.531 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839295/3eb4e8ca-ae3f-4e1f-ba46-a9b925d228e7/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:49.908 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:49.908 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:49.908 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246875 s, 42.5 MB/s 2021-02-10T11:23:49.909 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:49.909 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:49.909 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:50.244 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:50.244 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:50.244 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839295/3eb4e8ca-ae3f-4e1f-ba46-a9b925d228e7/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:50.621 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:50.622 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:50.622 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247594 s, 42.4 MB/s 2021-02-10T11:23:50.623 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839296 2021-02-10T11:23:51.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.030 7f2ba0502700 1 Processor -- start 2021-02-10T11:23:51.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.030 7f2ba0502700 1 -- start start 2021-02-10T11:23:51.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.030 7f2ba0502700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2b981b0470 0x7f2b981b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:51.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.030 7f2ba0502700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b980f20e0 0x7f2b980f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:51.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.030 7f2ba0502700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b980f0b50 con 0x7f2b980ef610 2021-02-10T11:23:51.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.030 7f2ba0502700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b980f0820 con 0x7f2b981b0470 2021-02-10T11:23:51.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.030 7f2ba0502700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b980f24e0 con 0x7f2b980f20e0 2021-02-10T11:23:51.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.031 7f2b9eb1f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b980f20e0 0x7f2b980f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:51.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.031 7f2b9e31e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2b981b0470 0x7f2b981b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:51.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.031 7f2b9eb1f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b980f20e0 0x7f2b980f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:39990/0 (socket says 172.21.2.101:39990) 2021-02-10T11:23:51.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.031 7f2b9eb1f700 1 -- 172.21.2.101:0/369486065 learned_addr learned my addr 172.21.2.101:0/369486065 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:51.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.031 7f2b9eb1f700 1 -- 172.21.2.101:0/369486065 >> v2:172.21.2.108:3300/0 conn(0x7f2b981b0470 msgr2=0x7f2b981b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:23:51.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.031 7f2b9eb1f700 1 --2- 172.21.2.101:0/369486065 >> v2:172.21.2.108:3300/0 conn(0x7f2b981b0470 0x7f2b981b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:51.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.031 7f2b9eb1f700 1 -- 172.21.2.101:0/369486065 >> v1:172.21.2.108:6789/0 conn(0x7f2b980ef610 legacy=0x7f2b980efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:51.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.031 7f2b9eb1f700 1 -- 172.21.2.101:0/369486065 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b980e52e0 con 0x7f2b980f20e0 2021-02-10T11:23:51.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.031 7f2b9eb1f700 1 --2- 172.21.2.101:0/369486065 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b980f20e0 0x7f2b980f0200 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f2b90006aa0 tx=0x7f2b90009f20).ready entity=mon.2 client_cookie=a64ffc9201606ffd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:51.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.032 7f2b9d31c700 1 -- 172.21.2.101:0/369486065 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b9000b5c0 con 0x7f2b980f20e0 2021-02-10T11:23:51.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.032 7f2b9d31c700 1 -- 172.21.2.101:0/369486065 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b90003030 con 0x7f2b980f20e0 2021-02-10T11:23:51.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.032 7f2b9d31c700 1 -- 172.21.2.101:0/369486065 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b9000b9f0 con 0x7f2b980f20e0 2021-02-10T11:23:51.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.032 7f2ba0502700 1 -- 172.21.2.101:0/369486065 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b980f20e0 msgr2=0x7f2b980f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:51.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.032 7f2ba0502700 1 --2- 172.21.2.101:0/369486065 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b980f20e0 0x7f2b980f0200 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f2b90006aa0 tx=0x7f2b90009f20).stop 2021-02-10T11:23:51.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.032 7f2ba0502700 1 -- 172.21.2.101:0/369486065 shutdown_connections 2021-02-10T11:23:51.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.032 7f2ba0502700 1 --2- 172.21.2.101:0/369486065 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b980f20e0 0x7f2b980f0200 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:51.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.032 7f2ba0502700 1 --2- 172.21.2.101:0/369486065 >> v2:172.21.2.108:3300/0 conn(0x7f2b981b0470 0x7f2b981b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:51.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.032 7f2ba0502700 1 -- 172.21.2.101:0/369486065 >> 172.21.2.101:0/369486065 conn(0x7f2b980ce3e0 msgr2=0x7f2b980e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:51.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.033 7f2ba0502700 1 -- 172.21.2.101:0/369486065 shutdown_connections 2021-02-10T11:23:51.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.033 7f2ba0502700 1 -- 172.21.2.101:0/369486065 wait complete. 2021-02-10T11:23:51.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.047 7f2ba0502700 1 Processor -- start 2021-02-10T11:23:51.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.047 7f2ba0502700 1 -- start start 2021-02-10T11:23:51.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.047 7f2ba0502700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b98105d10 0x7f2b9809eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:51.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.047 7f2ba0502700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b98106110 0x7f2b9809f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:51.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.047 7f2ba0502700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b980b2a00 0x7f2b980b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:51.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.047 7f2ba0502700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9809e560 con 0x7f2b98106110 2021-02-10T11:23:51.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.047 7f2ba0502700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9809c090 con 0x7f2b980b2a00 2021-02-10T11:23:51.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.047 7f2ba0502700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b980b3420 con 0x7f2b98105d10 2021-02-10T11:23:51.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.047 7f2b9e31e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b98105d10 0x7f2b9809eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:51.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.048 7f2b9eb1f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b980b2a00 0x7f2b980b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:51.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.048 7f2b9db1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b98106110 0x7f2b9809f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:51.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.048 7f2b9e31e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b98105d10 0x7f2b9809eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:39992/0 (socket says 172.21.2.101:39992) 2021-02-10T11:23:51.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.048 7f2b9e31e700 1 -- 172.21.2.101:0/1315173715 learned_addr learned my addr 172.21.2.101:0/1315173715 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:51.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.048 7f2b9e31e700 1 -- 172.21.2.101:0/1315173715 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b980b2a00 msgr2=0x7f2b980b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:51.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.048 7f2b9e31e700 1 --2- 172.21.2.101:0/1315173715 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b980b2a00 0x7f2b980b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:51.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.048 7f2b9e31e700 1 -- 172.21.2.101:0/1315173715 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b98106110 msgr2=0x7f2b9809f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:51.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.048 7f2b9e31e700 1 --2- 172.21.2.101:0/1315173715 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b98106110 0x7f2b9809f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:51.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.048 7f2b9e31e700 1 -- 172.21.2.101:0/1315173715 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b980b3640 con 0x7f2b98105d10 2021-02-10T11:23:51.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.048 7f2b9e31e700 1 --2- 172.21.2.101:0/1315173715 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b98105d10 0x7f2b9809eae0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f2b94006f60 tx=0x7f2b9400b970).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:51.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.048 7f2b8affd700 1 -- 172.21.2.101:0/1315173715 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b940009b0 con 0x7f2b98105d10 2021-02-10T11:23:51.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.048 7f2b8affd700 1 -- 172.21.2.101:0/1315173715 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9400bd60 con 0x7f2b98105d10 2021-02-10T11:23:51.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.048 7f2b8affd700 1 -- 172.21.2.101:0/1315173715 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b94002030 con 0x7f2b98105d10 2021-02-10T11:23:51.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.048 7f2ba0502700 1 -- 172.21.2.101:0/1315173715 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b980b3640 con 0x7f2b98105d10 2021-02-10T11:23:51.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.048 7f2ba0502700 1 -- 172.21.2.101:0/1315173715 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b980b3bd0 con 0x7f2b98105d10 2021-02-10T11:23:51.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.049 7f2b8affd700 1 -- 172.21.2.101:0/1315173715 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2b94006b00 con 0x7f2b98105d10 2021-02-10T11:23:51.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.049 7f2b8affd700 1 --2- 172.21.2.101:0/1315173715 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b800186c0 0x7f2b8001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:51.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.049 7f2b8affd700 1 -- 172.21.2.101:0/1315173715 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2b94032520 con 0x7f2b98105d10 2021-02-10T11:23:51.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.049 7f2b9db1d700 1 --2- 172.21.2.101:0/1315173715 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b800186c0 0x7f2b8001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:51.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.049 7f2b9db1d700 1 --2- 172.21.2.101:0/1315173715 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b800186c0 0x7f2b8001abc0 crc :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:51.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.080 7f2ba0502700 1 -- 172.21.2.101:0/1315173715 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b980b3ec0 con 0x7f2b98105d10 2021-02-10T11:23:51.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.084 7f2b8affd700 1 -- 172.21.2.101:0/1315173715 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2b94006340 con 0x7f2b98105d10 2021-02-10T11:23:51.102 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:23:51.105 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:23:51.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.216 7f2ba0502700 1 -- 172.21.2.101:0/1315173715 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839296", "target": ["mgr", ""]}) v1 -- 0x7f2b981a16c0 con 0x7f2b800186c0 2021-02-10T11:23:51.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.242 7f2b8affd700 1 -- 172.21.2.101:0/1315173715 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f2b8c006a90 con 0x7f2b800186c0 2021-02-10T11:23:51.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.249 7f2ba0502700 1 -- 172.21.2.101:0/1315173715 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b800186c0 msgr2=0x7f2b8001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:51.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.249 7f2ba0502700 1 --2- 172.21.2.101:0/1315173715 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b800186c0 0x7f2b8001abc0 crc :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:51.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.249 7f2ba0502700 1 -- 172.21.2.101:0/1315173715 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b98105d10 msgr2=0x7f2b9809eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:51.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.249 7f2ba0502700 1 --2- 172.21.2.101:0/1315173715 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b98105d10 0x7f2b9809eae0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f2b94006f60 tx=0x7f2b9400b970).stop 2021-02-10T11:23:51.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.250 7f2ba0502700 1 -- 172.21.2.101:0/1315173715 shutdown_connections 2021-02-10T11:23:51.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.250 7f2ba0502700 1 --2- 172.21.2.101:0/1315173715 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b980b2a00 0x7f2b980b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:51.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.250 7f2ba0502700 1 --2- 172.21.2.101:0/1315173715 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b98106110 0x7f2b9809f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:51.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.250 7f2ba0502700 1 --2- 172.21.2.101:0/1315173715 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b800186c0 0x7f2b8001abc0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:51.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.250 7f2ba0502700 1 --2- 172.21.2.101:0/1315173715 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b98105d10 0x7f2b9809eae0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:51.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.250 7f2ba0502700 1 -- 172.21.2.101:0/1315173715 >> 172.21.2.101:0/1315173715 conn(0x7f2b980c36f0 msgr2=0x7f2b980bd640 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:51.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.250 7f2ba0502700 1 -- 172.21.2.101:0/1315173715 shutdown_connections 2021-02-10T11:23:51.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.250 7f2ba0502700 1 -- 172.21.2.101:0/1315173715 wait complete. 2021-02-10T11:23:51.266 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839296 2021-02-10T11:23:51.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.672 7f41fba04700 1 Processor -- start 2021-02-10T11:23:51.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.672 7f41fba04700 1 -- start start 2021-02-10T11:23:51.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.672 7f41fba04700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f41f41b0870 0x7f41f40f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:51.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.672 7f41fba04700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41f40efe00 0x7f41f40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:51.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.672 7f41fba04700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f41b0c70 con 0x7f41f41b0470 2021-02-10T11:23:51.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.672 7f41fba04700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41f40f0b50 con 0x7f41f41b0870 2021-02-10T11:23:51.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.672 7f41fba04700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f40f0820 con 0x7f41f40efe00 2021-02-10T11:23:51.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.673 7f41f9820700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f41f41b0470 0x7f41f40ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47762/0 (socket says 172.21.2.101:47762) 2021-02-10T11:23:51.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.673 7f41f9820700 1 -- 172.21.2.101:0/2840732345 learned_addr learned my addr 172.21.2.101:0/2840732345 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:23:51.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.673 7f41fa021700 1 --2- 172.21.2.101:0/2840732345 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41f40efe00 0x7f41f40f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:51.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.673 7f41f901f700 1 --2- 172.21.2.101:0/2840732345 >> v2:172.21.2.108:3300/0 conn(0x7f41f41b0870 0x7f41f40f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:51.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.673 7f41f881e700 1 -- 172.21.2.101:0/2840732345 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3151226296 0 0) 0x7f41f0000ce0 con 0x7f41f41b0470 2021-02-10T11:23:51.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.673 7f41f881e700 1 -- 172.21.2.101:0/2840732345 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41e0002fe0 con 0x7f41f41b0470 2021-02-10T11:23:51.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.674 7f41fa021700 1 -- 172.21.2.101:0/2840732345 >> v2:172.21.2.108:3300/0 conn(0x7f41f41b0870 msgr2=0x7f41f40f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:23:51.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.674 7f41fa021700 1 --2- 172.21.2.101:0/2840732345 >> v2:172.21.2.108:3300/0 conn(0x7f41f41b0870 0x7f41f40f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:51.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.674 7f41fa021700 1 -- 172.21.2.101:0/2840732345 >> v1:172.21.2.101:6789/0 conn(0x7f41f41b0470 legacy=0x7f41f40ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:51.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.674 7f41fa021700 1 -- 172.21.2.101:0/2840732345 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f40e52e0 con 0x7f41f40efe00 2021-02-10T11:23:51.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.674 7f41f901f700 1 --2- 172.21.2.101:0/2840732345 >> v2:172.21.2.108:3300/0 conn(0x7f41f41b0870 0x7f41f40f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:23:51.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.674 7f41fa021700 1 --2- 172.21.2.101:0/2840732345 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41f40efe00 0x7f41f40f0200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f41ec006e50 tx=0x7f41ec009e70).ready entity=mon.2 client_cookie=d112b9e1848b86a1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:51.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.674 7f41f881e700 1 -- 172.21.2.101:0/2840732345 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f41ec006a20 con 0x7f41f40efe00 2021-02-10T11:23:51.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.674 7f41f881e700 1 -- 172.21.2.101:0/2840732345 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ec006790 con 0x7f41f40efe00 2021-02-10T11:23:51.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.674 7f41f881e700 1 -- 172.21.2.101:0/2840732345 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f41ec00b700 con 0x7f41f40efe00 2021-02-10T11:23:51.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.674 7f41fba04700 1 -- 172.21.2.101:0/2840732345 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41f40efe00 msgr2=0x7f41f40f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:51.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.674 7f41fba04700 1 --2- 172.21.2.101:0/2840732345 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41f40efe00 0x7f41f40f0200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f41ec006e50 tx=0x7f41ec009e70).stop 2021-02-10T11:23:51.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.675 7f41fba04700 1 -- 172.21.2.101:0/2840732345 shutdown_connections 2021-02-10T11:23:51.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.675 7f41fba04700 1 --2- 172.21.2.101:0/2840732345 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41f40efe00 0x7f41f40f0200 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:51.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.675 7f41fba04700 1 --2- 172.21.2.101:0/2840732345 >> v2:172.21.2.108:3300/0 conn(0x7f41f41b0870 0x7f41f40f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:51.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.675 7f41fba04700 1 -- 172.21.2.101:0/2840732345 >> 172.21.2.101:0/2840732345 conn(0x7f41f40ce3e0 msgr2=0x7f41f40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:51.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.675 7f41fba04700 1 -- 172.21.2.101:0/2840732345 shutdown_connections 2021-02-10T11:23:51.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.675 7f41fba04700 1 -- 172.21.2.101:0/2840732345 wait complete. 2021-02-10T11:23:51.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.690 7f41fba04700 1 Processor -- start 2021-02-10T11:23:51.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.690 7f41fba04700 1 -- start start 2021-02-10T11:23:51.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.690 7f41fba04700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f41f4190920 0x7f41f40b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:51.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.690 7f41fba04700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41f40b8200 0x7f41f40b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:51.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.690 7f41fba04700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41f40c1d80 0x7f41f40c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:51.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.690 7f41fba04700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f4190d20 con 0x7f41f40b8200 2021-02-10T11:23:51.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.690 7f41fba04700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f40b8c20 con 0x7f41f4190920 2021-02-10T11:23:51.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.690 7f41fba04700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f40c27a0 con 0x7f41f40c1d80 2021-02-10T11:23:51.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.691 7f41f901f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41f40b8200 0x7f41f40b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:51.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.691 7f41f901f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41f40b8200 0x7f41f40b8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47276/0 (socket says 172.21.2.101:47276) 2021-02-10T11:23:51.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.691 7f41f901f700 1 -- 172.21.2.101:0/553028719 learned_addr learned my addr 172.21.2.101:0/553028719 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:51.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.691 7f41f901f700 1 -- 172.21.2.101:0/553028719 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41f40c1d80 msgr2=0x7f41f40c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:51.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.691 7f41f901f700 1 --2- 172.21.2.101:0/553028719 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41f40c1d80 0x7f41f40c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:51.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.691 7f41f901f700 1 -- 172.21.2.101:0/553028719 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f41f4190920 msgr2=0x7f41f40b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:51.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.691 7f41f901f700 1 --2- 172.21.2.101:0/553028719 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f41f4190920 0x7f41f40b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:51.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.691 7f41f901f700 1 -- 172.21.2.101:0/553028719 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f40c29c0 con 0x7f41f40b8200 2021-02-10T11:23:51.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.691 7f41f901f700 1 --2- 172.21.2.101:0/553028719 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41f40b8200 0x7f41f40b8600 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f41e4006f40 tx=0x7f41e400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:51.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.691 7f41ea7fc700 1 -- 172.21.2.101:0/553028719 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f41e400bd80 con 0x7f41f40b8200 2021-02-10T11:23:51.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.691 7f41ea7fc700 1 -- 172.21.2.101:0/553028719 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e4000c90 con 0x7f41f40b8200 2021-02-10T11:23:51.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.691 7f41fba04700 1 -- 172.21.2.101:0/553028719 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f40c29c0 con 0x7f41f40b8200 2021-02-10T11:23:51.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.691 7f41ea7fc700 1 -- 172.21.2.101:0/553028719 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f41e4002220 con 0x7f41f40b8200 2021-02-10T11:23:51.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.692 7f41fba04700 1 -- 172.21.2.101:0/553028719 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f40c29c0 con 0x7f41f40b8200 2021-02-10T11:23:51.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.692 7f41ea7fc700 1 -- 172.21.2.101:0/553028719 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f41e4006b00 con 0x7f41f40b8200 2021-02-10T11:23:51.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.692 7f41ea7fc700 1 --2- 172.21.2.101:0/553028719 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f41dc018730 0x7f41dc01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:51.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.692 7f41ea7fc700 1 -- 172.21.2.101:0/553028719 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f41e4032650 con 0x7f41f40b8200 2021-02-10T11:23:51.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.692 7f41f9820700 1 --2- 172.21.2.101:0/553028719 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f41dc018730 0x7f41dc01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:51.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.692 7f41f9820700 1 --2- 172.21.2.101:0/553028719 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f41dc018730 0x7f41dc01ac10 crc :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:51.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.722 7f41fba04700 1 -- 172.21.2.101:0/553028719 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f4105ae0 con 0x7f41f40b8200 2021-02-10T11:23:51.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.725 7f41ea7fc700 1 -- 172.21.2.101:0/553028719 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f41e4000c90 con 0x7f41f40b8200 2021-02-10T11:23:51.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.853 7f41fba04700 1 -- 172.21.2.101:0/553028719 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839296", "target": ["mgr", ""]}) v1 -- 0x7f41f40c32c0 con 0x7f41dc018730 2021-02-10T11:23:51.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.856 7f41ea7fc700 1 -- 172.21.2.101:0/553028719 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f41f0000c90 con 0x7f41dc018730 2021-02-10T11:23:51.868 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839296/41de6871-3723-4bb5-a95f-0add5a2076e8 2021-02-10T11:23:51.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.861 7f41fba04700 1 -- 172.21.2.101:0/553028719 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f41dc018730 msgr2=0x7f41dc01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:51.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.861 7f41fba04700 1 --2- 172.21.2.101:0/553028719 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f41dc018730 0x7f41dc01ac10 crc :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:51.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.861 7f41fba04700 1 -- 172.21.2.101:0/553028719 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41f40b8200 msgr2=0x7f41f40b8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:51.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.861 7f41fba04700 1 --2- 172.21.2.101:0/553028719 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41f40b8200 0x7f41f40b8600 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f41e4006f40 tx=0x7f41e400b990).stop 2021-02-10T11:23:51.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.861 7f41fba04700 1 -- 172.21.2.101:0/553028719 shutdown_connections 2021-02-10T11:23:51.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.861 7f41fba04700 1 --2- 172.21.2.101:0/553028719 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f41dc018730 0x7f41dc01ac10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:51.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.861 7f41fba04700 1 --2- 172.21.2.101:0/553028719 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41f40c1d80 0x7f41f40c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:51.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.861 7f41fba04700 1 --2- 172.21.2.101:0/553028719 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41f40b8200 0x7f41f40b8600 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:51.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.861 7f41fba04700 1 --2- 172.21.2.101:0/553028719 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f41f4190920 0x7f41f40b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:23:51.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.861 7f41fba04700 1 -- 172.21.2.101:0/553028719 >> 172.21.2.101:0/553028719 conn(0x7f41f4094e10 msgr2=0x7f41f40a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:51.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.861 7f41fba04700 1 -- 172.21.2.101:0/553028719 shutdown_connections 2021-02-10T11:23:51.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:51.862 7f41fba04700 1 -- 172.21.2.101:0/553028719 wait complete. 2021-02-10T11:23:51.878 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839296 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839296/41de6871-3723-4bb5-a95f-0add5a2076e8 2021-02-10T11:23:51.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:51.878 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:51.878 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:52.214 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:52.214 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:52.214 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839296/41de6871-3723-4bb5-a95f-0add5a2076e8/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:52.597 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:52.597 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:52.597 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239969 s, 43.7 MB/s 2021-02-10T11:23:52.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:52.598 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:52.598 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:52.934 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:52.935 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:52.935 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839296/41de6871-3723-4bb5-a95f-0add5a2076e8/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:53.309 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:53.309 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:53.309 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0226963 s, 46.2 MB/s 2021-02-10T11:23:53.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:53.311 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:53.311 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:53.648 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:53.648 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:53.649 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839296/41de6871-3723-4bb5-a95f-0add5a2076e8/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:54.022 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:54.022 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:54.022 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251235 s, 41.7 MB/s 2021-02-10T11:23:54.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:54.023 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:54.023 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:54.358 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:54.358 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:54.358 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839296/41de6871-3723-4bb5-a95f-0add5a2076e8/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:54.734 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:54.735 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:54.735 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240433 s, 43.6 MB/s 2021-02-10T11:23:54.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:54.736 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:54.736 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:55.072 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:55.072 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:55.073 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839296/41de6871-3723-4bb5-a95f-0add5a2076e8/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:55.448 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:55.448 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:55.448 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238977 s, 43.9 MB/s 2021-02-10T11:23:55.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:55.450 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:55.450 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:55.785 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:55.785 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:55.786 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839296/41de6871-3723-4bb5-a95f-0add5a2076e8/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:56.163 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:56.163 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:56.163 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247142 s, 42.4 MB/s 2021-02-10T11:23:56.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:56.164 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:56.164 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:56.499 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:56.499 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:56.500 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839296/41de6871-3723-4bb5-a95f-0add5a2076e8/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:56.877 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:56.877 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:56.877 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247727 s, 42.3 MB/s 2021-02-10T11:23:56.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:56.878 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:56.878 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:57.215 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:57.215 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:57.215 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839296/41de6871-3723-4bb5-a95f-0add5a2076e8/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:57.589 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:57.590 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:57.590 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239167 s, 43.8 MB/s 2021-02-10T11:23:57.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:57.591 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:57.591 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:57.927 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:57.927 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:57.927 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839296/41de6871-3723-4bb5-a95f-0add5a2076e8/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:58.304 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:58.304 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:58.304 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.02479 s, 42.3 MB/s 2021-02-10T11:23:58.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:23:58.305 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:23:58.305 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:23:58.641 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:23:58.641 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:23:58.642 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839296/41de6871-3723-4bb5-a95f-0add5a2076e8/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:23:59.013 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:23:59.014 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:23:59.014 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245413 s, 42.7 MB/s 2021-02-10T11:23:59.015 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839297 2021-02-10T11:23:59.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.421 7f507b679700 1 Processor -- start 2021-02-10T11:23:59.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.422 7f507b679700 1 -- start start 2021-02-10T11:23:59.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.422 7f507b679700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50741b0870 0x7f50740f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:59.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.422 7f507b679700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50741b0c70 con 0x7f50740efe00 2021-02-10T11:23:59.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.422 7f507b679700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50740f0b50 con 0x7f50741b0470 2021-02-10T11:23:59.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.422 7f507b679700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50740e5ea0 con 0x7f50741b0870 2021-02-10T11:23:59.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.422 7f5079c96700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f50740efe00 0x7f50740f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47778/0 (socket says 172.21.2.101:47778) 2021-02-10T11:23:59.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.422 7f5079c96700 1 -- 172.21.2.101:0/1326730987 learned_addr learned my addr 172.21.2.101:0/1326730987 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:23:59.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.422 7f5073fff700 1 -- 172.21.2.101:0/1326730987 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4027630838 0 0) 0x7f506c001500 con 0x7f50740efe00 2021-02-10T11:23:59.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5073fff700 1 -- 172.21.2.101:0/1326730987 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f505c002fe0 con 0x7f50740efe00 2021-02-10T11:23:59.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5078c94700 1 --2- 172.21.2.101:0/1326730987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50741b0870 0x7f50740f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:59.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5073fff700 1 -- 172.21.2.101:0/1326730987 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1487299417 0 0) 0x7f506c001460 con 0x7f50740efe00 2021-02-10T11:23:59.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5073fff700 1 -- 172.21.2.101:0/1326730987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50741b0870 msgr2=0x7f50740f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:23:59.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5073fff700 1 --2- 172.21.2.101:0/1326730987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50741b0870 0x7f50740f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:59.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5073fff700 1 -- 172.21.2.101:0/1326730987 >> v1:172.21.2.108:6789/0 conn(0x7f50741b0470 legacy=0x7f50740ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:23:59.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5073fff700 1 -- 172.21.2.101:0/1326730987 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50740e52e0 con 0x7f50740efe00 2021-02-10T11:23:59.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5073fff700 1 -- 172.21.2.101:0/1326730987 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f506c001980 con 0x7f50740efe00 2021-02-10T11:23:59.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5073fff700 1 -- 172.21.2.101:0/1326730987 >> v1:172.21.2.101:6789/0 conn(0x7f50740efe00 legacy=0x7f50740f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:59.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5073fff700 1 --2- 172.21.2.101:0/1326730987 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f505c003b10 0x7f505c005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:59.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5073fff700 1 --2- 172.21.2.101:0/1326730987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f505c006820 0x7f505c008cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:59.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5073fff700 1 --2- 172.21.2.101:0/1326730987 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f505c009590 0x7f505c00bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:59.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5073fff700 1 -- 172.21.2.101:0/1326730987 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c00c360 con 0x7f505c009590 2021-02-10T11:23:59.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5073fff700 1 -- 172.21.2.101:0/1326730987 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c00c580 con 0x7f505c003b10 2021-02-10T11:23:59.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5073fff700 1 -- 172.21.2.101:0/1326730987 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c00c7a0 con 0x7f505c006820 2021-02-10T11:23:59.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5078c94700 1 --2- 172.21.2.101:0/1326730987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50741b0870 0x7f50740f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:23:59.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5079c96700 1 --2- 172.21.2.101:0/1326730987 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f505c009590 0x7f505c00bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:59.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5078c94700 1 --2- 172.21.2.101:0/1326730987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f505c006820 0x7f505c008cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:59.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5079495700 1 --2- 172.21.2.101:0/1326730987 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f505c003b10 0x7f505c005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:59.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5079c96700 1 -- 172.21.2.101:0/1326730987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f505c006820 msgr2=0x7f505c008cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:23:59.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5079c96700 1 --2- 172.21.2.101:0/1326730987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f505c006820 0x7f505c008cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:59.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5079c96700 1 -- 172.21.2.101:0/1326730987 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f505c003b10 msgr2=0x7f505c005f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:23:59.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5079c96700 1 --2- 172.21.2.101:0/1326730987 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f505c003b10 0x7f505c005f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:59.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.423 7f5079c96700 1 -- 172.21.2.101:0/1326730987 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c00ca40 con 0x7f505c009590 2021-02-10T11:23:59.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.424 7f5079c96700 1 --2- 172.21.2.101:0/1326730987 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f505c009590 0x7f505c00bab0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f506c001c80 tx=0x7f506c04e320).ready entity=mon.0 client_cookie=f899a4affbb37c03 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:59.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.424 7f5073fff700 1 -- 172.21.2.101:0/1326730987 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f506c000f70 con 0x7f505c009590 2021-02-10T11:23:59.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.424 7f5073fff700 1 -- 172.21.2.101:0/1326730987 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506c0070d0 con 0x7f505c009590 2021-02-10T11:23:59.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.424 7f5073fff700 1 -- 172.21.2.101:0/1326730987 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f506c007520 con 0x7f505c009590 2021-02-10T11:23:59.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.424 7f507b679700 1 -- 172.21.2.101:0/1326730987 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f505c009590 msgr2=0x7f505c00bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:59.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.424 7f507b679700 1 --2- 172.21.2.101:0/1326730987 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f505c009590 0x7f505c00bab0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f506c001c80 tx=0x7f506c04e320).stop 2021-02-10T11:23:59.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.424 7f5079c96700 1 -- 172.21.2.101:0/1326730987 reap_dead start 2021-02-10T11:23:59.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.424 7f507b679700 1 -- 172.21.2.101:0/1326730987 shutdown_connections 2021-02-10T11:23:59.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.424 7f507b679700 1 -- 172.21.2.101:0/1326730987 >> 172.21.2.101:0/1326730987 conn(0x7f50740ce3e0 msgr2=0x7f50740e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:59.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.424 7f507b679700 1 -- 172.21.2.101:0/1326730987 shutdown_connections 2021-02-10T11:23:59.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.424 7f507b679700 1 -- 172.21.2.101:0/1326730987 wait complete. 2021-02-10T11:23:59.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.439 7f507b679700 1 Processor -- start 2021-02-10T11:23:59.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.439 7f507b679700 1 -- start start 2021-02-10T11:23:59.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.439 7f507b679700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50740b7db0 0x7f50740b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:59.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.439 7f507b679700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50740b8880 0x7f50741a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:59.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.439 7f507b679700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f50740b8c80 0x7f50741a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:59.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.439 7f507b679700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50740b79b0 con 0x7f50740b7db0 2021-02-10T11:23:59.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.439 7f507b679700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50741a8090 con 0x7f50740b8c80 2021-02-10T11:23:59.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.439 7f507b679700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50741a82b0 con 0x7f50740b8880 2021-02-10T11:23:59.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.440 7f5079495700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50740b7db0 0x7f50740b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:59.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.440 7f5079495700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50740b7db0 0x7f50740b8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47292/0 (socket says 172.21.2.101:47292) 2021-02-10T11:23:59.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.440 7f5079495700 1 -- 172.21.2.101:0/783053093 learned_addr learned my addr 172.21.2.101:0/783053093 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:23:59.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.440 7f5079c96700 1 --2- 172.21.2.101:0/783053093 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f50740b8c80 0x7f50741a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:59.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.440 7f5078c94700 1 --2- 172.21.2.101:0/783053093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50740b8880 0x7f50741a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:59.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.440 7f5079495700 1 -- 172.21.2.101:0/783053093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50740b8880 msgr2=0x7f50741a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:59.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.440 7f5079495700 1 --2- 172.21.2.101:0/783053093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50740b8880 0x7f50741a7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:59.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.440 7f5079495700 1 -- 172.21.2.101:0/783053093 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f50740b8c80 msgr2=0x7f50741a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:59.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.440 7f5079495700 1 --2- 172.21.2.101:0/783053093 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f50740b8c80 0x7f50741a7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:59.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.440 7f5079495700 1 -- 172.21.2.101:0/783053093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5074190c10 con 0x7f50740b7db0 2021-02-10T11:23:59.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.440 7f5079495700 1 --2- 172.21.2.101:0/783053093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50740b7db0 0x7f50740b8260 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f5064006990 tx=0x7f506400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:59.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.440 7f5071ffb700 1 -- 172.21.2.101:0/783053093 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5064006a80 con 0x7f50740b7db0 2021-02-10T11:23:59.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.440 7f5071ffb700 1 -- 172.21.2.101:0/783053093 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5064000c60 con 0x7f50740b7db0 2021-02-10T11:23:59.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.440 7f5071ffb700 1 -- 172.21.2.101:0/783053093 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5064002350 con 0x7f50740b7db0 2021-02-10T11:23:59.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.440 7f507b679700 1 -- 172.21.2.101:0/783053093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5074190c10 con 0x7f50740b7db0 2021-02-10T11:23:59.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.441 7f507b679700 1 -- 172.21.2.101:0/783053093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5074190c10 con 0x7f50740b7db0 2021-02-10T11:23:59.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.441 7f5071ffb700 1 -- 172.21.2.101:0/783053093 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5064006a80 con 0x7f50740b7db0 2021-02-10T11:23:59.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.441 7f5071ffb700 1 --2- 172.21.2.101:0/783053093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50540186c0 0x7f505401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:23:59.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.441 7f5071ffb700 1 -- 172.21.2.101:0/783053093 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5064032570 con 0x7f50740b7db0 2021-02-10T11:23:59.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.441 7f5078c94700 1 --2- 172.21.2.101:0/783053093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50540186c0 0x7f505401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:23:59.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.442 7f5078c94700 1 --2- 172.21.2.101:0/783053093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50540186c0 0x7f505401abc0 crc :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:23:59.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.472 7f507b679700 1 -- 172.21.2.101:0/783053093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5074105b30 con 0x7f50740b7db0 2021-02-10T11:23:59.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.474 7f5071ffb700 1 -- 172.21.2.101:0/783053093 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5064030d60 con 0x7f50740b7db0 2021-02-10T11:23:59.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.601 7f507b679700 1 -- 172.21.2.101:0/783053093 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839297", "target": ["mgr", ""]}) v1 -- 0x7f50740f0b50 con 0x7f50540186c0 2021-02-10T11:23:59.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.625 7f5071ffb700 1 -- 172.21.2.101:0/783053093 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f5068006a90 con 0x7f50540186c0 2021-02-10T11:23:59.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.634 7f507b679700 1 -- 172.21.2.101:0/783053093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50540186c0 msgr2=0x7f505401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:59.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.634 7f507b679700 1 --2- 172.21.2.101:0/783053093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50540186c0 0x7f505401abc0 crc :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:59.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.634 7f507b679700 1 -- 172.21.2.101:0/783053093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50740b7db0 msgr2=0x7f50740b8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:23:59.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.634 7f507b679700 1 --2- 172.21.2.101:0/783053093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50740b7db0 0x7f50740b8260 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f5064006990 tx=0x7f506400b990).stop 2021-02-10T11:23:59.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.635 7f507b679700 1 -- 172.21.2.101:0/783053093 shutdown_connections 2021-02-10T11:23:59.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.635 7f507b679700 1 --2- 172.21.2.101:0/783053093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50540186c0 0x7f505401abc0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:59.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.635 7f507b679700 1 --2- 172.21.2.101:0/783053093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50740b8880 0x7f50741a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:59.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.635 7f507b679700 1 --2- 172.21.2.101:0/783053093 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f50740b8c80 0x7f50741a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:59.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.635 7f507b679700 1 --2- 172.21.2.101:0/783053093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50740b7db0 0x7f50740b8260 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:23:59.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.635 7f507b679700 1 -- 172.21.2.101:0/783053093 >> 172.21.2.101:0/783053093 conn(0x7f5074094e10 msgr2=0x7f50740a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:23:59.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.635 7f507b679700 1 -- 172.21.2.101:0/783053093 shutdown_connections 2021-02-10T11:23:59.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:23:59.635 7f507b679700 1 -- 172.21.2.101:0/783053093 wait complete. 2021-02-10T11:23:59.651 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839297 2021-02-10T11:24:00.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.058 7effd96ec700 1 Processor -- start 2021-02-10T11:24:00.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.058 7effd96ec700 1 -- start start 2021-02-10T11:24:00.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.058 7effd96ec700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7effd41b0870 0x7effd40f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:00.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.058 7effd96ec700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7effd40efe00 0x7effd40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:00.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.058 7effd96ec700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effd41b0c70 con 0x7effd41b0470 2021-02-10T11:24:00.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.058 7effd96ec700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd40e5ea0 con 0x7effd40efe00 2021-02-10T11:24:00.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.058 7effd96ec700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd40f0b50 con 0x7effd41b0870 2021-02-10T11:24:00.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.059 7effd3ac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7effd40efe00 0x7effd40f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:00.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.059 7effd3ac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7effd40efe00 0x7effd40f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47302/0 (socket says 172.21.2.101:47302) 2021-02-10T11:24:00.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.059 7effd3ac0700 1 -- 172.21.2.101:0/678910770 learned_addr learned my addr 172.21.2.101:0/678910770 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:00.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.059 7effd3ac0700 1 -- 172.21.2.101:0/678910770 >> v2:172.21.2.108:3300/0 conn(0x7effd41b0870 msgr2=0x7effd40f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:24:00.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.059 7effd3ac0700 1 --2- 172.21.2.101:0/678910770 >> v2:172.21.2.108:3300/0 conn(0x7effd41b0870 0x7effd40f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:00.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.059 7effd3ac0700 1 -- 172.21.2.101:0/678910770 >> v1:172.21.2.108:6789/0 conn(0x7effd41b0470 legacy=0x7effd40ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:24:00.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.059 7effd3ac0700 1 -- 172.21.2.101:0/678910770 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd40f0820 con 0x7effd40efe00 2021-02-10T11:24:00.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.059 7effd3ac0700 1 --2- 172.21.2.101:0/678910770 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7effd40efe00 0x7effd40f0200 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7effcc006b80 tx=0x7effcc009b90).ready entity=mon.0 client_cookie=6d763a17a3e889d1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:00.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.059 7effd22bd700 1 -- 172.21.2.101:0/678910770 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7effcc0022e0 con 0x7effd40efe00 2021-02-10T11:24:00.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.060 7effd22bd700 1 -- 172.21.2.101:0/678910770 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effcc002620 con 0x7effd40efe00 2021-02-10T11:24:00.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.060 7effd22bd700 1 -- 172.21.2.101:0/678910770 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7effcc002b00 con 0x7effd40efe00 2021-02-10T11:24:00.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.060 7effd96ec700 1 -- 172.21.2.101:0/678910770 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7effd40efe00 msgr2=0x7effd40f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:00.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.060 7effd96ec700 1 --2- 172.21.2.101:0/678910770 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7effd40efe00 0x7effd40f0200 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7effcc006b80 tx=0x7effcc009b90).stop 2021-02-10T11:24:00.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.060 7effd96ec700 1 -- 172.21.2.101:0/678910770 shutdown_connections 2021-02-10T11:24:00.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.060 7effd96ec700 1 --2- 172.21.2.101:0/678910770 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7effd40efe00 0x7effd40f0200 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:00.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.060 7effd96ec700 1 --2- 172.21.2.101:0/678910770 >> v2:172.21.2.108:3300/0 conn(0x7effd41b0870 0x7effd40f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:00.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.060 7effd96ec700 1 -- 172.21.2.101:0/678910770 >> 172.21.2.101:0/678910770 conn(0x7effd40ce3e0 msgr2=0x7effd40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:00.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.060 7effd96ec700 1 -- 172.21.2.101:0/678910770 shutdown_connections 2021-02-10T11:24:00.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.060 7effd96ec700 1 -- 172.21.2.101:0/678910770 wait complete. 2021-02-10T11:24:00.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.075 7effd96ec700 1 Processor -- start 2021-02-10T11:24:00.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.075 7effd96ec700 1 -- start start 2021-02-10T11:24:00.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.075 7effd96ec700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7effd4094fb0 0x7effd409e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:00.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.075 7effd96ec700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7effd40c2be0 0x7effd40c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:00.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.075 7effd96ec700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7effd40c3600 0x7effd40a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:00.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.075 7effd96ec700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd409ebf0 con 0x7effd4094fb0 2021-02-10T11:24:00.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.075 7effd96ec700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd40aa2a0 con 0x7effd40c3600 2021-02-10T11:24:00.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.075 7effd96ec700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd40aa4c0 con 0x7effd40c2be0 2021-02-10T11:24:00.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.076 7effd32bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7effd4094fb0 0x7effd409e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:00.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.076 7effd32bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7effd4094fb0 0x7effd409e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47304/0 (socket says 172.21.2.101:47304) 2021-02-10T11:24:00.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.076 7effd32bf700 1 -- 172.21.2.101:0/2986634478 learned_addr learned my addr 172.21.2.101:0/2986634478 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:00.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.076 7effd32bf700 1 -- 172.21.2.101:0/2986634478 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7effd40c2be0 msgr2=0x7effd40c2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:00.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.076 7effd32bf700 1 --2- 172.21.2.101:0/2986634478 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7effd40c2be0 0x7effd40c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:00.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.076 7effd32bf700 1 -- 172.21.2.101:0/2986634478 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7effd40c3600 msgr2=0x7effd40a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:00.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.076 7effd32bf700 1 --2- 172.21.2.101:0/2986634478 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7effd40c3600 0x7effd40a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:00.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.076 7effd32bf700 1 -- 172.21.2.101:0/2986634478 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd40aa6e0 con 0x7effd4094fb0 2021-02-10T11:24:00.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.076 7effd32bf700 1 --2- 172.21.2.101:0/2986634478 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7effd4094fb0 0x7effd409e5d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7effbc00b8e0 tx=0x7effbc00bc80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:00.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.076 7effcbfff700 1 -- 172.21.2.101:0/2986634478 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7effbc000fe0 con 0x7effd4094fb0 2021-02-10T11:24:00.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.076 7effcbfff700 1 -- 172.21.2.101:0/2986634478 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effbc00c2f0 con 0x7effd4094fb0 2021-02-10T11:24:00.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.076 7effd96ec700 1 -- 172.21.2.101:0/2986634478 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd40aa6e0 con 0x7effd4094fb0 2021-02-10T11:24:00.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.077 7effcbfff700 1 -- 172.21.2.101:0/2986634478 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7effbc00c810 con 0x7effd4094fb0 2021-02-10T11:24:00.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.077 7effd96ec700 1 -- 172.21.2.101:0/2986634478 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd40aa6e0 con 0x7effd4094fb0 2021-02-10T11:24:00.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.077 7effcbfff700 1 -- 172.21.2.101:0/2986634478 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7effbc001230 con 0x7effd4094fb0 2021-02-10T11:24:00.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.077 7effcbfff700 1 --2- 172.21.2.101:0/2986634478 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7effb80186c0 0x7effb801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:00.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.077 7effcbfff700 1 -- 172.21.2.101:0/2986634478 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7effbc032980 con 0x7effd4094fb0 2021-02-10T11:24:00.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.077 7effd2abe700 1 --2- 172.21.2.101:0/2986634478 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7effb80186c0 0x7effb801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:00.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.078 7effd2abe700 1 --2- 172.21.2.101:0/2986634478 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7effb80186c0 0x7effb801abc0 crc :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:00.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.108 7effd96ec700 1 -- 172.21.2.101:0/2986634478 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd40edd10 con 0x7effd4094fb0 2021-02-10T11:24:00.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.111 7effcbfff700 1 -- 172.21.2.101:0/2986634478 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7effbc00bdc0 con 0x7effd4094fb0 2021-02-10T11:24:00.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.238 7effd96ec700 1 -- 172.21.2.101:0/2986634478 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839297", "target": ["mgr", ""]}) v1 -- 0x7effd409ebf0 con 0x7effb80186c0 2021-02-10T11:24:00.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.242 7effcbfff700 1 -- 172.21.2.101:0/2986634478 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7effc4006e40 con 0x7effb80186c0 2021-02-10T11:24:00.254 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839297/d77f2075-799a-44f8-b3cb-5231e30edcd0 2021-02-10T11:24:00.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.247 7effd96ec700 1 -- 172.21.2.101:0/2986634478 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7effb80186c0 msgr2=0x7effb801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:00.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.247 7effd96ec700 1 --2- 172.21.2.101:0/2986634478 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7effb80186c0 0x7effb801abc0 crc :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:00.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.247 7effd96ec700 1 -- 172.21.2.101:0/2986634478 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7effd4094fb0 msgr2=0x7effd409e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:00.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.247 7effd96ec700 1 --2- 172.21.2.101:0/2986634478 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7effd4094fb0 0x7effd409e5d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7effbc00b8e0 tx=0x7effbc00bc80).stop 2021-02-10T11:24:00.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.247 7effd96ec700 1 -- 172.21.2.101:0/2986634478 shutdown_connections 2021-02-10T11:24:00.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.247 7effd96ec700 1 --2- 172.21.2.101:0/2986634478 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7effb80186c0 0x7effb801abc0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:00.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.247 7effd96ec700 1 --2- 172.21.2.101:0/2986634478 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7effd40c2be0 0x7effd40c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:00.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.247 7effd96ec700 1 --2- 172.21.2.101:0/2986634478 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7effd40c3600 0x7effd40a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:00.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.247 7effd96ec700 1 --2- 172.21.2.101:0/2986634478 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7effd4094fb0 0x7effd409e5d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:00.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.247 7effd96ec700 1 -- 172.21.2.101:0/2986634478 >> 172.21.2.101:0/2986634478 conn(0x7effd409f630 msgr2=0x7effd40a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:00.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.247 7effd96ec700 1 -- 172.21.2.101:0/2986634478 shutdown_connections 2021-02-10T11:24:00.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:00.247 7effd96ec700 1 -- 172.21.2.101:0/2986634478 wait complete. 2021-02-10T11:24:00.263 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839297 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839297/d77f2075-799a-44f8-b3cb-5231e30edcd0 2021-02-10T11:24:00.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:00.264 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:00.264 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:00.600 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:00.600 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:00.601 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839297/d77f2075-799a-44f8-b3cb-5231e30edcd0/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:00.981 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:00.981 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:00.981 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237759 s, 44.1 MB/s 2021-02-10T11:24:00.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:00.983 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:00.983 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:01.319 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:01.319 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:01.320 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839297/d77f2075-799a-44f8-b3cb-5231e30edcd0/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:01.692 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:01.692 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:01.692 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0230322 s, 45.5 MB/s 2021-02-10T11:24:01.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:01.693 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:01.693 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:02.028 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:02.028 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:02.029 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839297/d77f2075-799a-44f8-b3cb-5231e30edcd0/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:02.406 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:02.406 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:02.407 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246696 s, 42.5 MB/s 2021-02-10T11:24:02.407 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:02.408 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:02.408 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:02.742 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:02.742 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:02.743 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839297/d77f2075-799a-44f8-b3cb-5231e30edcd0/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:03.119 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:03.120 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:03.120 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245992 s, 42.6 MB/s 2021-02-10T11:24:03.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:03.121 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:03.121 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:03.456 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:03.457 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:03.457 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839297/d77f2075-799a-44f8-b3cb-5231e30edcd0/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:03.828 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:03.829 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:03.829 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243519 s, 43.1 MB/s 2021-02-10T11:24:03.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:03.830 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:03.830 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:04.165 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:04.166 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:04.166 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839297/d77f2075-799a-44f8-b3cb-5231e30edcd0/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:04.540 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:04.541 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:04.541 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023878 s, 43.9 MB/s 2021-02-10T11:24:04.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:04.542 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:04.542 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:04.878 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:04.878 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:04.879 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839297/d77f2075-799a-44f8-b3cb-5231e30edcd0/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:05.279 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:05.279 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:05.279 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0263312 s, 39.8 MB/s 2021-02-10T11:24:05.280 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:05.281 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:05.281 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:05.622 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:05.622 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:05.622 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839297/d77f2075-799a-44f8-b3cb-5231e30edcd0/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:05.998 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:05.998 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:05.998 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237283 s, 44.2 MB/s 2021-02-10T11:24:05.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:06.000 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:06.000 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:06.335 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:06.336 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:06.336 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839297/d77f2075-799a-44f8-b3cb-5231e30edcd0/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:06.711 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:06.711 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:06.711 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251508 s, 41.7 MB/s 2021-02-10T11:24:06.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:06.713 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:06.713 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:07.047 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:07.047 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:07.048 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839297/d77f2075-799a-44f8-b3cb-5231e30edcd0/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:07.422 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:07.422 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:07.422 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0228147 s, 46.0 MB/s 2021-02-10T11:24:07.423 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839298 2021-02-10T11:24:07.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.828 7f8a6cc8f700 1 Processor -- start 2021-02-10T11:24:07.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.829 7f8a6cc8f700 1 -- start start 2021-02-10T11:24:07.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.829 7f8a6cc8f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a681b0470 0x7f8a681b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:07.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.829 7f8a6cc8f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8a680f20e0 0x7f8a680f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:07.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.829 7f8a6cc8f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a680f0b50 con 0x7f8a680ef610 2021-02-10T11:24:07.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.829 7f8a6cc8f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a680f0820 con 0x7f8a680f20e0 2021-02-10T11:24:07.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.829 7f8a6cc8f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a680f24e0 con 0x7f8a681b0470 2021-02-10T11:24:07.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.829 7f8a6605d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8a680ef610 0x7f8a680efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47808/0 (socket says 172.21.2.101:47808) 2021-02-10T11:24:07.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.829 7f8a6605d700 1 -- 172.21.2.101:0/1828415235 learned_addr learned my addr 172.21.2.101:0/1828415235 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:24:07.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.829 7f8a6705f700 1 --2- 172.21.2.101:0/1828415235 >> v2:172.21.2.101:3300/0 conn(0x7f8a680f20e0 0x7f8a680f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:07.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.829 7f8a6685e700 1 --2- 172.21.2.101:0/1828415235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a681b0470 0x7f8a681b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:07.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.830 7f8a6705f700 1 -- 172.21.2.101:0/1828415235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a681b0470 msgr2=0x7f8a681b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:24:07.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.830 7f8a6705f700 1 --2- 172.21.2.101:0/1828415235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a681b0470 0x7f8a681b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:07.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.830 7f8a6705f700 1 -- 172.21.2.101:0/1828415235 >> v1:172.21.2.101:6789/0 conn(0x7f8a680ef610 legacy=0x7f8a680efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:07.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.830 7f8a6705f700 1 -- 172.21.2.101:0/1828415235 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a680e52e0 con 0x7f8a680f20e0 2021-02-10T11:24:07.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.830 7f8a6685e700 1 --2- 172.21.2.101:0/1828415235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a681b0470 0x7f8a681b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:24:07.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.830 7f8a6705f700 1 --2- 172.21.2.101:0/1828415235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a680f20e0 0x7f8a680f0200 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f8a60006e30 tx=0x7f8a60009e60).ready entity=mon.0 client_cookie=67711690453de7f9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:07.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.830 7f8a6585c700 1 -- 172.21.2.101:0/1828415235 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8a60000f90 con 0x7f8a680f20e0 2021-02-10T11:24:07.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.830 7f8a6585c700 1 -- 172.21.2.101:0/1828415235 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6000b300 con 0x7f8a680f20e0 2021-02-10T11:24:07.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.830 7f8a6585c700 1 -- 172.21.2.101:0/1828415235 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8a6000b820 con 0x7f8a680f20e0 2021-02-10T11:24:07.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.830 7f8a6cc8f700 1 -- 172.21.2.101:0/1828415235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a680f20e0 msgr2=0x7f8a680f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:07.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.830 7f8a6cc8f700 1 --2- 172.21.2.101:0/1828415235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a680f20e0 0x7f8a680f0200 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f8a60006e30 tx=0x7f8a60009e60).stop 2021-02-10T11:24:07.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.831 7f8a6cc8f700 1 -- 172.21.2.101:0/1828415235 shutdown_connections 2021-02-10T11:24:07.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.831 7f8a6cc8f700 1 --2- 172.21.2.101:0/1828415235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a680f20e0 0x7f8a680f0200 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:07.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.831 7f8a6cc8f700 1 --2- 172.21.2.101:0/1828415235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a681b0470 0x7f8a681b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:07.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.831 7f8a6cc8f700 1 -- 172.21.2.101:0/1828415235 >> 172.21.2.101:0/1828415235 conn(0x7f8a680ce3e0 msgr2=0x7f8a680e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:07.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.831 7f8a6cc8f700 1 -- 172.21.2.101:0/1828415235 shutdown_connections 2021-02-10T11:24:07.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.831 7f8a6cc8f700 1 -- 172.21.2.101:0/1828415235 wait complete. 2021-02-10T11:24:07.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.846 7f8a6cc8f700 1 Processor -- start 2021-02-10T11:24:07.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.846 7f8a6cc8f700 1 -- start start 2021-02-10T11:24:07.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.846 7f8a6cc8f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a680b7c20 0x7f8a680b8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:07.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.846 7f8a6cc8f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a680b8a30 0x7f8a680c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:07.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.846 7f8a6cc8f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a680c25a0 0x7f8a680c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:07.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.846 7f8a6cc8f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a680c1a80 con 0x7f8a680b7c20 2021-02-10T11:24:07.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.846 7f8a6cc8f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a680b8020 con 0x7f8a680c25a0 2021-02-10T11:24:07.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.846 7f8a6cc8f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6809b9c0 con 0x7f8a680b8a30 2021-02-10T11:24:07.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.846 7f8a6685e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a680b7c20 0x7f8a680b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:07.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.846 7f8a6685e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a680b7c20 0x7f8a680b8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47320/0 (socket says 172.21.2.101:47320) 2021-02-10T11:24:07.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.846 7f8a6685e700 1 -- 172.21.2.101:0/2999323309 learned_addr learned my addr 172.21.2.101:0/2999323309 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:07.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.846 7f8a6605d700 1 --2- 172.21.2.101:0/2999323309 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a680b8a30 0x7f8a680c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:07.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.846 7f8a6705f700 1 --2- 172.21.2.101:0/2999323309 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a680c25a0 0x7f8a680c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:07.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.847 7f8a6685e700 1 -- 172.21.2.101:0/2999323309 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a680b8a30 msgr2=0x7f8a680c1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:07.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.847 7f8a6685e700 1 --2- 172.21.2.101:0/2999323309 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a680b8a30 0x7f8a680c1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:07.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.847 7f8a6685e700 1 -- 172.21.2.101:0/2999323309 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a680c25a0 msgr2=0x7f8a680c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:07.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.847 7f8a6685e700 1 --2- 172.21.2.101:0/2999323309 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a680c25a0 0x7f8a680c29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:07.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.847 7f8a6685e700 1 -- 172.21.2.101:0/2999323309 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a680b8e30 con 0x7f8a680b7c20 2021-02-10T11:24:07.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.847 7f8a6685e700 1 --2- 172.21.2.101:0/2999323309 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a680b7c20 0x7f8a680b8410 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f8a58006f20 tx=0x7f8a5800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:07.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.847 7f8a5f7fe700 1 -- 172.21.2.101:0/2999323309 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8a5800bd80 con 0x7f8a680b7c20 2021-02-10T11:24:07.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.847 7f8a5f7fe700 1 -- 172.21.2.101:0/2999323309 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a58000c90 con 0x7f8a680b7c20 2021-02-10T11:24:07.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.847 7f8a6cc8f700 1 -- 172.21.2.101:0/2999323309 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a680b8e30 con 0x7f8a680b7c20 2021-02-10T11:24:07.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.847 7f8a6cc8f700 1 -- 172.21.2.101:0/2999323309 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a680c32b0 con 0x7f8a680b7c20 2021-02-10T11:24:07.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.847 7f8a5f7fe700 1 -- 172.21.2.101:0/2999323309 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8a580022b0 con 0x7f8a680b7c20 2021-02-10T11:24:07.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.848 7f8a5f7fe700 1 -- 172.21.2.101:0/2999323309 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8a5800a000 con 0x7f8a680b7c20 2021-02-10T11:24:07.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.848 7f8a5f7fe700 1 --2- 172.21.2.101:0/2999323309 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a4c018730 0x7f8a4c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:07.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.848 7f8a5f7fe700 1 -- 172.21.2.101:0/2999323309 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8a580327c0 con 0x7f8a680b7c20 2021-02-10T11:24:07.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.848 7f8a6605d700 1 --2- 172.21.2.101:0/2999323309 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a4c018730 0x7f8a4c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:07.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.848 7f8a6605d700 1 --2- 172.21.2.101:0/2999323309 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a4c018730 0x7f8a4c01abc0 crc :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:07.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.879 7f8a6cc8f700 1 -- 172.21.2.101:0/2999323309 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a68105b30 con 0x7f8a680b7c20 2021-02-10T11:24:07.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:07.881 7f8a5f7fe700 1 -- 172.21.2.101:0/2999323309 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8a580009b0 con 0x7f8a680b7c20 2021-02-10T11:24:08.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.007 7f8a6cc8f700 1 -- 172.21.2.101:0/2999323309 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839298", "target": ["mgr", ""]}) v1 -- 0x7f8a6809b9c0 con 0x7f8a4c018730 2021-02-10T11:24:08.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.032 7f8a5f7fe700 1 -- 172.21.2.101:0/2999323309 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f8a50001a00 con 0x7f8a4c018730 2021-02-10T11:24:08.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.040 7f8a6cc8f700 1 -- 172.21.2.101:0/2999323309 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a4c018730 msgr2=0x7f8a4c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:08.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.040 7f8a6cc8f700 1 --2- 172.21.2.101:0/2999323309 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a4c018730 0x7f8a4c01abc0 crc :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:08.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.040 7f8a6cc8f700 1 -- 172.21.2.101:0/2999323309 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a680b7c20 msgr2=0x7f8a680b8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:08.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.040 7f8a6cc8f700 1 --2- 172.21.2.101:0/2999323309 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a680b7c20 0x7f8a680b8410 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f8a58006f20 tx=0x7f8a5800b990).stop 2021-02-10T11:24:08.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.040 7f8a6cc8f700 1 -- 172.21.2.101:0/2999323309 shutdown_connections 2021-02-10T11:24:08.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.040 7f8a6cc8f700 1 --2- 172.21.2.101:0/2999323309 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a4c018730 0x7f8a4c01abc0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:08.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.040 7f8a6cc8f700 1 --2- 172.21.2.101:0/2999323309 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a680b8a30 0x7f8a680c1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:08.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.040 7f8a6cc8f700 1 --2- 172.21.2.101:0/2999323309 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a680c25a0 0x7f8a680c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:08.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.040 7f8a6cc8f700 1 --2- 172.21.2.101:0/2999323309 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a680b7c20 0x7f8a680b8410 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:08.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.040 7f8a6cc8f700 1 -- 172.21.2.101:0/2999323309 >> 172.21.2.101:0/2999323309 conn(0x7f8a68094e10 msgr2=0x7f8a680a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:08.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.040 7f8a6cc8f700 1 -- 172.21.2.101:0/2999323309 shutdown_connections 2021-02-10T11:24:08.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.040 7f8a6cc8f700 1 -- 172.21.2.101:0/2999323309 wait complete. 2021-02-10T11:24:08.057 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839298 2021-02-10T11:24:08.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.464 7fd83fce8700 1 Processor -- start 2021-02-10T11:24:08.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.464 7fd83fce8700 1 -- start start 2021-02-10T11:24:08.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.464 7fd83fce8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8381b0870 0x7fd8380f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:08.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.464 7fd83fce8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd8380efe00 0x7fd8380f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:08.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.464 7fd83fce8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8381b0c70 con 0x7fd8381b0470 2021-02-10T11:24:08.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.464 7fd83fce8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8380f0b50 con 0x7fd8380efe00 2021-02-10T11:24:08.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.464 7fd83fce8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380f0820 con 0x7fd8381b0870 2021-02-10T11:24:08.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.466 7fd83e305700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd8380efe00 0x7fd8380f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:08.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.466 7fd83d303700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8381b0870 0x7fd8380f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:08.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.466 7fd83db04700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fd8381b0470 0x7fd8380ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:41230/0 (socket says 172.21.2.101:41230) 2021-02-10T11:24:08.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.466 7fd83db04700 1 -- 172.21.2.101:0/3877127006 learned_addr learned my addr 172.21.2.101:0/3877127006 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:24:08.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.466 7fd83d303700 1 -- 172.21.2.101:0/3877127006 >> v2:172.21.2.108:3300/0 conn(0x7fd8380efe00 msgr2=0x7fd8380f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:24:08.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.466 7fd83d303700 1 --2- 172.21.2.101:0/3877127006 >> v2:172.21.2.108:3300/0 conn(0x7fd8380efe00 0x7fd8380f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:08.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.466 7fd83d303700 1 -- 172.21.2.101:0/3877127006 >> v1:172.21.2.108:6789/0 conn(0x7fd8381b0470 legacy=0x7fd8380ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:08.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.466 7fd83d303700 1 -- 172.21.2.101:0/3877127006 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8380e52e0 con 0x7fd8381b0870 2021-02-10T11:24:08.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.466 7fd83cb02700 1 -- 172.21.2.101:0/3877127006 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 643937024 0 0) 0x7fd82c001190 con 0x7fd8381b0470 2021-02-10T11:24:08.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.467 7fd83e305700 1 --2- 172.21.2.101:0/3877127006 >> v2:172.21.2.108:3300/0 conn(0x7fd8380efe00 0x7fd8380f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:24:08.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.467 7fd83d303700 1 --2- 172.21.2.101:0/3877127006 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8381b0870 0x7fd8380f20e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fd834006650 tx=0x7fd834009a80).ready entity=mon.2 client_cookie=fd7bd3d123334535 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:08.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.467 7fd83cb02700 1 -- 172.21.2.101:0/3877127006 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd83400c0c0 con 0x7fd8381b0870 2021-02-10T11:24:08.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.467 7fd83cb02700 1 -- 172.21.2.101:0/3877127006 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83400c3c0 con 0x7fd8381b0870 2021-02-10T11:24:08.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.467 7fd83cb02700 1 -- 172.21.2.101:0/3877127006 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd83400c8e0 con 0x7fd8381b0870 2021-02-10T11:24:08.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.467 7fd83fce8700 1 -- 172.21.2.101:0/3877127006 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8381b0870 msgr2=0x7fd8380f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:08.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.467 7fd83fce8700 1 --2- 172.21.2.101:0/3877127006 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8381b0870 0x7fd8380f20e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fd834006650 tx=0x7fd834009a80).stop 2021-02-10T11:24:08.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.468 7fd83fce8700 1 -- 172.21.2.101:0/3877127006 shutdown_connections 2021-02-10T11:24:08.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.468 7fd83fce8700 1 --2- 172.21.2.101:0/3877127006 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8381b0870 0x7fd8380f20e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:08.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.468 7fd83fce8700 1 --2- 172.21.2.101:0/3877127006 >> v2:172.21.2.108:3300/0 conn(0x7fd8380efe00 0x7fd8380f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:08.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.468 7fd83fce8700 1 -- 172.21.2.101:0/3877127006 >> 172.21.2.101:0/3877127006 conn(0x7fd8380ce3e0 msgr2=0x7fd8380e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:08.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.468 7fd83fce8700 1 -- 172.21.2.101:0/3877127006 shutdown_connections 2021-02-10T11:24:08.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.468 7fd83fce8700 1 -- 172.21.2.101:0/3877127006 wait complete. 2021-02-10T11:24:08.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.483 7fd83fce8700 1 Processor -- start 2021-02-10T11:24:08.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.483 7fd83fce8700 1 -- start start 2021-02-10T11:24:08.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.483 7fd83fce8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd838190920 0x7fd8380b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:08.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.483 7fd83fce8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8380b8200 0x7fd8380b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:08.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.483 7fd83fce8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd8380c1d80 0x7fd8380c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:08.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.483 7fd83fce8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd838190da0 con 0x7fd8380b8200 2021-02-10T11:24:08.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.483 7fd83fce8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380b8c20 con 0x7fd8380c1d80 2021-02-10T11:24:08.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.483 7fd83fce8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380b8e10 con 0x7fd838190920 2021-02-10T11:24:08.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.483 7fd83d303700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8380b8200 0x7fd8380b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:08.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.483 7fd83d303700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8380b8200 0x7fd8380b8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47330/0 (socket says 172.21.2.101:47330) 2021-02-10T11:24:08.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.483 7fd83d303700 1 -- 172.21.2.101:0/1153785489 learned_addr learned my addr 172.21.2.101:0/1153785489 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:08.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.484 7fd83d303700 1 -- 172.21.2.101:0/1153785489 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd838190920 msgr2=0x7fd8380b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:08.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.484 7fd83d303700 1 --2- 172.21.2.101:0/1153785489 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd838190920 0x7fd8380b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:08.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.484 7fd83d303700 1 -- 172.21.2.101:0/1153785489 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd8380c1d80 msgr2=0x7fd8380c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:08.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.484 7fd83d303700 1 --2- 172.21.2.101:0/1153785489 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd8380c1d80 0x7fd8380c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:08.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.484 7fd83d303700 1 -- 172.21.2.101:0/1153785489 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8380c27d0 con 0x7fd8380b8200 2021-02-10T11:24:08.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.484 7fd83d303700 1 --2- 172.21.2.101:0/1153785489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8380b8200 0x7fd8380b8600 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7fd834006690 tx=0x7fd83400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:08.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.484 7fd82a7fc700 1 -- 172.21.2.101:0/1153785489 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd83400bd50 con 0x7fd8380b8200 2021-02-10T11:24:08.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.484 7fd82a7fc700 1 -- 172.21.2.101:0/1153785489 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8340093a0 con 0x7fd8380b8200 2021-02-10T11:24:08.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.484 7fd82a7fc700 1 -- 172.21.2.101:0/1153785489 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd834005560 con 0x7fd8380b8200 2021-02-10T11:24:08.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.484 7fd83fce8700 1 -- 172.21.2.101:0/1153785489 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8380c27d0 con 0x7fd8380b8200 2021-02-10T11:24:08.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.485 7fd83fce8700 1 -- 172.21.2.101:0/1153785489 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8380c27d0 con 0x7fd8380b8200 2021-02-10T11:24:08.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.485 7fd82a7fc700 1 -- 172.21.2.101:0/1153785489 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd83400a000 con 0x7fd8380b8200 2021-02-10T11:24:08.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.485 7fd82a7fc700 1 --2- 172.21.2.101:0/1153785489 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8200186c0 0x7fd82001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:08.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.485 7fd82a7fc700 1 -- 172.21.2.101:0/1153785489 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd8340323a0 con 0x7fd8380b8200 2021-02-10T11:24:08.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.485 7fd83db04700 1 --2- 172.21.2.101:0/1153785489 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8200186c0 0x7fd82001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:08.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.486 7fd83db04700 1 --2- 172.21.2.101:0/1153785489 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8200186c0 0x7fd82001abc0 crc :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:08.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.514 7fd83fce8700 1 -- 172.21.2.101:0/1153785489 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd838105ae0 con 0x7fd8380b8200 2021-02-10T11:24:08.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.517 7fd82a7fc700 1 -- 172.21.2.101:0/1153785489 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd83401b3f0 con 0x7fd8380b8200 2021-02-10T11:24:08.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.646 7fd83fce8700 1 -- 172.21.2.101:0/1153785489 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839298", "target": ["mgr", ""]}) v1 -- 0x7fd838105ae0 con 0x7fd8200186c0 2021-02-10T11:24:08.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.648 7fd82a7fc700 1 -- 172.21.2.101:0/1153785489 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fd82c000c90 con 0x7fd8200186c0 2021-02-10T11:24:08.657 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839298/80a01625-69c7-4964-8248-790d3e1ea2dd 2021-02-10T11:24:08.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.650 7fd83fce8700 1 -- 172.21.2.101:0/1153785489 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8200186c0 msgr2=0x7fd82001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:08.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.650 7fd83fce8700 1 --2- 172.21.2.101:0/1153785489 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8200186c0 0x7fd82001abc0 crc :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:08.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.650 7fd83fce8700 1 -- 172.21.2.101:0/1153785489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8380b8200 msgr2=0x7fd8380b8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:08.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.650 7fd83fce8700 1 --2- 172.21.2.101:0/1153785489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8380b8200 0x7fd8380b8600 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7fd834006690 tx=0x7fd83400b960).stop 2021-02-10T11:24:08.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.650 7fd83fce8700 1 -- 172.21.2.101:0/1153785489 shutdown_connections 2021-02-10T11:24:08.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.650 7fd83fce8700 1 --2- 172.21.2.101:0/1153785489 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd8380c1d80 0x7fd8380c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:08.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.650 7fd83fce8700 1 --2- 172.21.2.101:0/1153785489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8380b8200 0x7fd8380b8600 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:08.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.650 7fd83fce8700 1 --2- 172.21.2.101:0/1153785489 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8200186c0 0x7fd82001abc0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:08.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.650 7fd83fce8700 1 --2- 172.21.2.101:0/1153785489 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd838190920 0x7fd8380b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:08.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.650 7fd83fce8700 1 -- 172.21.2.101:0/1153785489 >> 172.21.2.101:0/1153785489 conn(0x7fd838094e10 msgr2=0x7fd8380a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:08.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.650 7fd83fce8700 1 -- 172.21.2.101:0/1153785489 shutdown_connections 2021-02-10T11:24:08.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:08.651 7fd83fce8700 1 -- 172.21.2.101:0/1153785489 wait complete. 2021-02-10T11:24:08.667 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839298 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839298/80a01625-69c7-4964-8248-790d3e1ea2dd 2021-02-10T11:24:08.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:08.667 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:08.667 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:09.003 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:09.003 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:09.004 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839298/80a01625-69c7-4964-8248-790d3e1ea2dd/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:09.380 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:09.381 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:09.381 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0224213 s, 46.8 MB/s 2021-02-10T11:24:09.382 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:09.382 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:09.382 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:09.718 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:09.719 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:09.719 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839298/80a01625-69c7-4964-8248-790d3e1ea2dd/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:10.096 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:10.096 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:10.096 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246394 s, 42.6 MB/s 2021-02-10T11:24:10.097 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:10.097 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:10.097 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:10.432 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:10.433 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:10.433 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839298/80a01625-69c7-4964-8248-790d3e1ea2dd/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:10.810 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:10.810 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:10.810 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.025072 s, 41.8 MB/s 2021-02-10T11:24:10.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:10.812 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:10.812 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:11.147 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:11.148 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:11.148 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839298/80a01625-69c7-4964-8248-790d3e1ea2dd/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:11.522 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:11.522 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:11.523 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235949 s, 44.4 MB/s 2021-02-10T11:24:11.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:11.524 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:11.524 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:11.859 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:11.859 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:11.860 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839298/80a01625-69c7-4964-8248-790d3e1ea2dd/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:12.233 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:12.233 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:12.233 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0222804 s, 47.1 MB/s 2021-02-10T11:24:12.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:12.234 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:12.234 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:12.570 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:12.570 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:12.571 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839298/80a01625-69c7-4964-8248-790d3e1ea2dd/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:12.948 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:12.949 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:12.949 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0254559 s, 41.2 MB/s 2021-02-10T11:24:12.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:12.950 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:12.950 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:13.286 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:13.286 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:13.287 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839298/80a01625-69c7-4964-8248-790d3e1ea2dd/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:13.661 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:13.661 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:13.662 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0228557 s, 45.9 MB/s 2021-02-10T11:24:13.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:13.663 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:13.663 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:13.999 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:13.999 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:14.000 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839298/80a01625-69c7-4964-8248-790d3e1ea2dd/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:14.371 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:14.372 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:14.372 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239967 s, 43.7 MB/s 2021-02-10T11:24:14.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:14.373 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:14.373 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:14.708 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:14.708 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:14.708 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839298/80a01625-69c7-4964-8248-790d3e1ea2dd/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:15.081 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:15.082 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:15.082 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0228486 s, 45.9 MB/s 2021-02-10T11:24:15.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:15.083 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:15.083 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:15.419 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:15.419 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:15.420 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839298/80a01625-69c7-4964-8248-790d3e1ea2dd/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:15.816 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:15.816 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:15.816 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236005 s, 44.4 MB/s 2021-02-10T11:24:15.817 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839299 2021-02-10T11:24:16.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.234 7f8e2ee25700 1 Processor -- start 2021-02-10T11:24:16.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.234 7f8e2ee25700 1 -- start start 2021-02-10T11:24:16.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.234 7f8e2ee25700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8e281b0870 0x7f8e280f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:16.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.234 7f8e2ee25700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e281b0c70 con 0x7f8e281b0470 2021-02-10T11:24:16.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.234 7f8e2ee25700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e280f0b50 con 0x7f8e280efe00 2021-02-10T11:24:16.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.234 7f8e2ee25700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e280e5ea0 con 0x7f8e281b0870 2021-02-10T11:24:16.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.234 7f8e2cc41700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8e281b0470 0x7f8e280ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47832/0 (socket says 172.21.2.101:47832) 2021-02-10T11:24:16.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.234 7f8e27fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8e281b0870 0x7f8e280f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:16.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.234 7f8e2cc41700 1 -- 172.21.2.101:0/3447712544 learned_addr learned my addr 172.21.2.101:0/3447712544 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:24:16.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.235 7f8e27fff700 1 -- 172.21.2.101:0/3447712544 >> v1:172.21.2.108:6789/0 conn(0x7f8e280efe00 legacy=0x7f8e280f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:24:16.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.235 7f8e27fff700 1 -- 172.21.2.101:0/3447712544 >> v1:172.21.2.101:6789/0 conn(0x7f8e281b0470 legacy=0x7f8e280ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:16.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.235 7f8e277fe700 1 -- 172.21.2.101:0/3447712544 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 58534371 0 0) 0x7f8e100009b0 con 0x7f8e281b0470 2021-02-10T11:24:16.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.235 7f8e27fff700 1 -- 172.21.2.101:0/3447712544 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e280e52e0 con 0x7f8e281b0870 2021-02-10T11:24:16.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.235 7f8e27fff700 1 --2- 172.21.2.101:0/3447712544 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e281b0870 0x7f8e280f20e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f8e18006650 tx=0x7f8e18009a80).ready entity=mon.0 client_cookie=5b8c697a5091916a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:16.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.235 7f8e277fe700 1 -- 172.21.2.101:0/3447712544 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8e1800c0c0 con 0x7f8e281b0870 2021-02-10T11:24:16.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.235 7f8e277fe700 1 -- 172.21.2.101:0/3447712544 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1800c340 con 0x7f8e281b0870 2021-02-10T11:24:16.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.235 7f8e277fe700 1 -- 172.21.2.101:0/3447712544 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8e1800c860 con 0x7f8e281b0870 2021-02-10T11:24:16.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.235 7f8e2ee25700 1 -- 172.21.2.101:0/3447712544 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e281b0870 msgr2=0x7f8e280f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:16.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.235 7f8e2ee25700 1 --2- 172.21.2.101:0/3447712544 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e281b0870 0x7f8e280f20e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f8e18006650 tx=0x7f8e18009a80).stop 2021-02-10T11:24:16.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.236 7f8e2ee25700 1 -- 172.21.2.101:0/3447712544 shutdown_connections 2021-02-10T11:24:16.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.236 7f8e2ee25700 1 --2- 172.21.2.101:0/3447712544 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e281b0870 0x7f8e280f20e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:16.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.236 7f8e2ee25700 1 -- 172.21.2.101:0/3447712544 >> 172.21.2.101:0/3447712544 conn(0x7f8e280ce3e0 msgr2=0x7f8e280e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:16.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.236 7f8e2ee25700 1 -- 172.21.2.101:0/3447712544 shutdown_connections 2021-02-10T11:24:16.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.236 7f8e2ee25700 1 -- 172.21.2.101:0/3447712544 wait complete. 2021-02-10T11:24:16.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.251 7f8e2ee25700 1 Processor -- start 2021-02-10T11:24:16.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.251 7f8e2ee25700 1 -- start start 2021-02-10T11:24:16.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.251 7f8e2ee25700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8e2818ca80 0x7f8e2818d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:16.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.251 7f8e2ee25700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8e2818d6c0 0x7f8e2818dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:16.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.251 7f8e2ee25700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e2818e6b0 0x7f8e2818eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:16.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.251 7f8e2ee25700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2819d290 con 0x7f8e2818e6b0 2021-02-10T11:24:16.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.251 7f8e2ee25700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2818e130 con 0x7f8e2818ca80 2021-02-10T11:24:16.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.251 7f8e2ee25700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2818c5e0 con 0x7f8e2818d6c0 2021-02-10T11:24:16.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.251 7f8e2d442700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e2818e6b0 0x7f8e2818eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:16.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.251 7f8e2d442700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e2818e6b0 0x7f8e2818eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47344/0 (socket says 172.21.2.101:47344) 2021-02-10T11:24:16.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.251 7f8e2d442700 1 -- 172.21.2.101:0/3436263951 learned_addr learned my addr 172.21.2.101:0/3436263951 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:16.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.252 7f8e2d442700 1 -- 172.21.2.101:0/3436263951 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8e2818d6c0 msgr2=0x7f8e2818dac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:24:16.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.252 7f8e2d442700 1 --2- 172.21.2.101:0/3436263951 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8e2818d6c0 0x7f8e2818dac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:16.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.252 7f8e2d442700 1 -- 172.21.2.101:0/3436263951 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8e2818ca80 msgr2=0x7f8e2818d0a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:24:16.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.252 7f8e2d442700 1 --2- 172.21.2.101:0/3436263951 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8e2818ca80 0x7f8e2818d0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:16.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.252 7f8e2d442700 1 -- 172.21.2.101:0/3436263951 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2818f0d0 con 0x7f8e2818e6b0 2021-02-10T11:24:16.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.252 7f8e2d442700 1 --2- 172.21.2.101:0/3436263951 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e2818e6b0 0x7f8e2818eab0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f8e2000b8e0 tx=0x7f8e2000bc80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:16.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.252 7f8e257fa700 1 -- 172.21.2.101:0/3436263951 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8e20000f90 con 0x7f8e2818e6b0 2021-02-10T11:24:16.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.252 7f8e257fa700 1 -- 172.21.2.101:0/3436263951 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2000c2f0 con 0x7f8e2818e6b0 2021-02-10T11:24:16.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.252 7f8e257fa700 1 -- 172.21.2.101:0/3436263951 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8e2000c810 con 0x7f8e2818e6b0 2021-02-10T11:24:16.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.252 7f8e2ee25700 1 -- 172.21.2.101:0/3436263951 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e2818f0d0 con 0x7f8e2818e6b0 2021-02-10T11:24:16.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.252 7f8e2ee25700 1 -- 172.21.2.101:0/3436263951 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e2818f660 con 0x7f8e2818e6b0 2021-02-10T11:24:16.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.253 7f8e257fa700 1 -- 172.21.2.101:0/3436263951 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8e20001220 con 0x7f8e2818e6b0 2021-02-10T11:24:16.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.253 7f8e257fa700 1 --2- 172.21.2.101:0/3436263951 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8e0c0186c0 0x7f8e0c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:16.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.253 7f8e257fa700 1 -- 172.21.2.101:0/3436263951 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8e200328e0 con 0x7f8e2818e6b0 2021-02-10T11:24:16.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.253 7f8e2cc41700 1 --2- 172.21.2.101:0/3436263951 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8e0c0186c0 0x7f8e0c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:16.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.253 7f8e2cc41700 1 --2- 172.21.2.101:0/3436263951 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8e0c0186c0 0x7f8e0c01abc0 crc :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:16.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.284 7f8e2ee25700 1 -- 172.21.2.101:0/3436263951 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e280edce0 con 0x7f8e2818e6b0 2021-02-10T11:24:16.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.286 7f8e257fa700 1 -- 172.21.2.101:0/3436263951 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8e20000f40 con 0x7f8e2818e6b0 2021-02-10T11:24:16.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.412 7f8e2ee25700 1 -- 172.21.2.101:0/3436263951 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839299", "target": ["mgr", ""]}) v1 -- 0x7f8e2818f660 con 0x7f8e0c0186c0 2021-02-10T11:24:16.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.439 7f8e257fa700 1 -- 172.21.2.101:0/3436263951 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f8e10001540 con 0x7f8e0c0186c0 2021-02-10T11:24:16.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.445 7f8e2ee25700 1 -- 172.21.2.101:0/3436263951 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8e0c0186c0 msgr2=0x7f8e0c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:16.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.445 7f8e2ee25700 1 --2- 172.21.2.101:0/3436263951 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8e0c0186c0 0x7f8e0c01abc0 crc :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:16.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.445 7f8e2ee25700 1 -- 172.21.2.101:0/3436263951 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e2818e6b0 msgr2=0x7f8e2818eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:16.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.445 7f8e2ee25700 1 --2- 172.21.2.101:0/3436263951 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e2818e6b0 0x7f8e2818eab0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f8e2000b8e0 tx=0x7f8e2000bc80).stop 2021-02-10T11:24:16.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.445 7f8e2ee25700 1 -- 172.21.2.101:0/3436263951 shutdown_connections 2021-02-10T11:24:16.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.445 7f8e2ee25700 1 --2- 172.21.2.101:0/3436263951 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8e0c0186c0 0x7f8e0c01abc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:16.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.445 7f8e2ee25700 1 --2- 172.21.2.101:0/3436263951 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8e2818d6c0 0x7f8e2818dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:16.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.445 7f8e2ee25700 1 --2- 172.21.2.101:0/3436263951 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e2818e6b0 0x7f8e2818eab0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:16.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.445 7f8e2ee25700 1 --2- 172.21.2.101:0/3436263951 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8e2818ca80 0x7f8e2818d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:16.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.445 7f8e2ee25700 1 -- 172.21.2.101:0/3436263951 >> 172.21.2.101:0/3436263951 conn(0x7f8e280bf4f0 msgr2=0x7f8e280be2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:16.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.445 7f8e2ee25700 1 -- 172.21.2.101:0/3436263951 shutdown_connections 2021-02-10T11:24:16.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.445 7f8e2ee25700 1 -- 172.21.2.101:0/3436263951 wait complete. 2021-02-10T11:24:16.462 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839299 2021-02-10T11:24:16.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.869 7fafbc141700 1 Processor -- start 2021-02-10T11:24:16.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.870 7fafbc141700 1 -- start start 2021-02-10T11:24:16.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.870 7fafbc141700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fafb41b0470 0x7fafb41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:16.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.870 7fafbc141700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fafb40f20e0 0x7fafb40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:16.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.870 7fafbc141700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb40f0b50 con 0x7fafb40ef610 2021-02-10T11:24:16.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.870 7fafbc141700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb40f0820 con 0x7fafb41b0470 2021-02-10T11:24:16.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.870 7fafbc141700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb40f24e0 con 0x7fafb40f20e0 2021-02-10T11:24:16.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.870 7fafb9f5d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fafb41b0470 0x7fafb41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:16.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.870 7fafb975c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fafb40ef610 0x7fafb40efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47846/0 (socket says 172.21.2.101:47846) 2021-02-10T11:24:16.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.870 7fafb9f5d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fafb41b0470 0x7fafb41b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47352/0 (socket says 172.21.2.101:47352) 2021-02-10T11:24:16.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.870 7fafb975c700 1 -- 172.21.2.101:0/159213854 learned_addr learned my addr 172.21.2.101:0/159213854 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:24:16.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.871 7fafb9f5d700 1 -- 172.21.2.101:0/159213854 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fafb40f20e0 msgr2=0x7fafb40f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:24:16.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.871 7fafb9f5d700 1 --2- 172.21.2.101:0/159213854 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fafb40f20e0 0x7fafb40f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:16.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.871 7fafb9f5d700 1 -- 172.21.2.101:0/159213854 >> v1:172.21.2.101:6789/0 conn(0x7fafb40ef610 legacy=0x7fafb40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:16.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.871 7fafb8f5b700 1 -- 172.21.2.101:0/159213854 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1969818172 0 0) 0x7fafa40009b0 con 0x7fafb40ef610 2021-02-10T11:24:16.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.871 7fafb9f5d700 1 -- 172.21.2.101:0/159213854 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb40e52e0 con 0x7fafb41b0470 2021-02-10T11:24:16.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.871 7fafb9f5d700 1 --2- 172.21.2.101:0/159213854 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fafb41b0470 0x7fafb41b0870 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fafb0006650 tx=0x7fafb0009a80).ready entity=mon.0 client_cookie=7b82685f6f1cd10a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:16.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.871 7fafb8f5b700 1 -- 172.21.2.101:0/159213854 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fafb000c0c0 con 0x7fafb41b0470 2021-02-10T11:24:16.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.871 7fafb8f5b700 1 -- 172.21.2.101:0/159213854 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb000c340 con 0x7fafb41b0470 2021-02-10T11:24:16.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.871 7fafb8f5b700 1 -- 172.21.2.101:0/159213854 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fafb000c860 con 0x7fafb41b0470 2021-02-10T11:24:16.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.871 7fafbc141700 1 -- 172.21.2.101:0/159213854 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fafb41b0470 msgr2=0x7fafb41b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:16.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.871 7fafbc141700 1 --2- 172.21.2.101:0/159213854 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fafb41b0470 0x7fafb41b0870 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fafb0006650 tx=0x7fafb0009a80).stop 2021-02-10T11:24:16.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.872 7fafbc141700 1 -- 172.21.2.101:0/159213854 shutdown_connections 2021-02-10T11:24:16.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.872 7fafbc141700 1 --2- 172.21.2.101:0/159213854 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fafb40f20e0 0x7fafb40f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:16.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.872 7fafbc141700 1 --2- 172.21.2.101:0/159213854 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fafb41b0470 0x7fafb41b0870 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:16.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.872 7fafbc141700 1 -- 172.21.2.101:0/159213854 >> 172.21.2.101:0/159213854 conn(0x7fafb40ce3e0 msgr2=0x7fafb40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:16.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.872 7fafbc141700 1 -- 172.21.2.101:0/159213854 shutdown_connections 2021-02-10T11:24:16.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.872 7fafbc141700 1 -- 172.21.2.101:0/159213854 wait complete. 2021-02-10T11:24:16.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.887 7fafbc141700 1 Processor -- start 2021-02-10T11:24:16.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.887 7fafbc141700 1 -- start start 2021-02-10T11:24:16.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.887 7fafbc141700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fafb4190920 0x7fafb408f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:16.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.888 7fafbc141700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fafb4190d20 0x7fafb408fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:16.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.888 7fafbc141700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fafb4193270 0x7fafb4193670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:16.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.888 7fafbc141700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb4090470 con 0x7fafb4190920 2021-02-10T11:24:16.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.888 7fafbc141700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb4193c90 con 0x7fafb4193270 2021-02-10T11:24:16.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.888 7fafbc141700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb4193e70 con 0x7fafb4190d20 2021-02-10T11:24:16.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.888 7fafb9f5d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fafb4190920 0x7fafb408f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:16.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.888 7fafb9f5d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fafb4190920 0x7fafb408f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47356/0 (socket says 172.21.2.101:47356) 2021-02-10T11:24:16.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.888 7fafb9f5d700 1 -- 172.21.2.101:0/769699611 learned_addr learned my addr 172.21.2.101:0/769699611 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:16.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.888 7fafba75e700 1 --2- 172.21.2.101:0/769699611 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fafb4193270 0x7fafb4193670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:16.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.888 7fafb975c700 1 --2- 172.21.2.101:0/769699611 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fafb4190d20 0x7fafb408fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:16.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.888 7fafb9f5d700 1 -- 172.21.2.101:0/769699611 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fafb4190d20 msgr2=0x7fafb408fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:16.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.888 7fafb9f5d700 1 --2- 172.21.2.101:0/769699611 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fafb4190d20 0x7fafb408fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:16.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.888 7fafb9f5d700 1 -- 172.21.2.101:0/769699611 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fafb4193270 msgr2=0x7fafb4193670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:16.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.888 7fafb9f5d700 1 --2- 172.21.2.101:0/769699611 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fafb4193270 0x7fafb4193670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:16.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.888 7fafb9f5d700 1 -- 172.21.2.101:0/769699611 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb4194090 con 0x7fafb4190920 2021-02-10T11:24:16.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.888 7fafb975c700 1 --2- 172.21.2.101:0/769699611 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fafb4190d20 0x7fafb408fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:24:16.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.888 7fafb9f5d700 1 --2- 172.21.2.101:0/769699611 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fafb4190920 0x7fafb408f600 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7fafb0009650 tx=0x7fafb000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:16.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.888 7fafaa7fc700 1 -- 172.21.2.101:0/769699611 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fafb00093a0 con 0x7fafb4190920 2021-02-10T11:24:16.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.888 7fafaa7fc700 1 -- 172.21.2.101:0/769699611 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb000bd50 con 0x7fafb4190920 2021-02-10T11:24:16.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.888 7fafbc141700 1 -- 172.21.2.101:0/769699611 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb4194090 con 0x7fafb4190920 2021-02-10T11:24:16.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.888 7fafaa7fc700 1 -- 172.21.2.101:0/769699611 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fafb00052e0 con 0x7fafb4190920 2021-02-10T11:24:16.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.889 7fafbc141700 1 -- 172.21.2.101:0/769699611 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb4194090 con 0x7fafb4190920 2021-02-10T11:24:16.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.889 7fafaa7fc700 1 -- 172.21.2.101:0/769699611 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fafb0006030 con 0x7fafb4190920 2021-02-10T11:24:16.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.889 7fafaa7fc700 1 --2- 172.21.2.101:0/769699611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faf9c018730 0x7faf9c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:16.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.889 7fafaa7fc700 1 -- 172.21.2.101:0/769699611 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fafb0032600 con 0x7fafb4190920 2021-02-10T11:24:16.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.889 7fafb975c700 1 --2- 172.21.2.101:0/769699611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faf9c018730 0x7faf9c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:16.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.890 7fafb975c700 1 --2- 172.21.2.101:0/769699611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faf9c018730 0x7faf9c01ac10 crc :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:16.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.920 7fafbc141700 1 -- 172.21.2.101:0/769699611 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb4105af0 con 0x7fafb4190920 2021-02-10T11:24:16.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:16.923 7fafaa7fc700 1 -- 172.21.2.101:0/769699611 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fafb00009c0 con 0x7fafb4190920 2021-02-10T11:24:17.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:17.050 7fafbc141700 1 -- 172.21.2.101:0/769699611 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839299", "target": ["mgr", ""]}) v1 -- 0x7fafb4105af0 con 0x7faf9c018730 2021-02-10T11:24:17.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:17.053 7fafaa7fc700 1 -- 172.21.2.101:0/769699611 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fafa4000960 con 0x7faf9c018730 2021-02-10T11:24:17.066 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839299/aed9c96e-538c-4e75-89e4-3cdc44fd2267 2021-02-10T11:24:17.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:17.059 7fafbc141700 1 -- 172.21.2.101:0/769699611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faf9c018730 msgr2=0x7faf9c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:17.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:17.059 7fafbc141700 1 --2- 172.21.2.101:0/769699611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faf9c018730 0x7faf9c01ac10 crc :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:17.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:17.059 7fafbc141700 1 -- 172.21.2.101:0/769699611 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fafb4190920 msgr2=0x7fafb408f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:17.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:17.059 7fafbc141700 1 --2- 172.21.2.101:0/769699611 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fafb4190920 0x7fafb408f600 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7fafb0009650 tx=0x7fafb000b960).stop 2021-02-10T11:24:17.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:17.059 7fafbc141700 1 -- 172.21.2.101:0/769699611 shutdown_connections 2021-02-10T11:24:17.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:17.059 7fafbc141700 1 --2- 172.21.2.101:0/769699611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faf9c018730 0x7faf9c01ac10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:17.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:17.059 7fafbc141700 1 --2- 172.21.2.101:0/769699611 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fafb4190d20 0x7fafb408fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:17.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:17.059 7fafbc141700 1 --2- 172.21.2.101:0/769699611 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fafb4193270 0x7fafb4193670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:17.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:17.059 7fafbc141700 1 --2- 172.21.2.101:0/769699611 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fafb4190920 0x7fafb408f600 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:17.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:17.059 7fafbc141700 1 -- 172.21.2.101:0/769699611 >> 172.21.2.101:0/769699611 conn(0x7fafb40bf4e0 msgr2=0x7fafb40be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:17.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:17.059 7fafbc141700 1 -- 172.21.2.101:0/769699611 shutdown_connections 2021-02-10T11:24:17.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:17.059 7fafbc141700 1 -- 172.21.2.101:0/769699611 wait complete. 2021-02-10T11:24:17.076 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839299 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839299/aed9c96e-538c-4e75-89e4-3cdc44fd2267 2021-02-10T11:24:17.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:17.076 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:17.076 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:17.412 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:17.413 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:17.413 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839299/aed9c96e-538c-4e75-89e4-3cdc44fd2267/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:17.792 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:17.793 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:17.793 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023575 s, 44.5 MB/s 2021-02-10T11:24:17.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:17.794 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:17.794 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:18.129 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:18.130 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:18.130 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839299/aed9c96e-538c-4e75-89e4-3cdc44fd2267/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:18.506 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:18.507 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:18.507 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024864 s, 42.2 MB/s 2021-02-10T11:24:18.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:18.508 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:18.508 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:18.843 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:18.843 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:18.844 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839299/aed9c96e-538c-4e75-89e4-3cdc44fd2267/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:19.216 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:19.217 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:19.217 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249804 s, 42.0 MB/s 2021-02-10T11:24:19.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:19.218 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:19.218 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:19.552 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:19.552 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:19.553 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839299/aed9c96e-538c-4e75-89e4-3cdc44fd2267/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:19.931 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:19.931 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:19.931 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0259094 s, 40.5 MB/s 2021-02-10T11:24:19.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:19.933 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:19.933 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:20.267 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:20.268 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:20.268 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839299/aed9c96e-538c-4e75-89e4-3cdc44fd2267/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:20.644 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:20.644 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:20.644 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236567 s, 44.3 MB/s 2021-02-10T11:24:20.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:20.645 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:20.645 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:20.980 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:20.981 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:20.981 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839299/aed9c96e-538c-4e75-89e4-3cdc44fd2267/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:21.356 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:21.356 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:21.356 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233867 s, 44.8 MB/s 2021-02-10T11:24:21.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:21.357 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:21.357 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:21.465 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:24:21.468 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:24:21.693 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:21.694 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:21.694 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839299/aed9c96e-538c-4e75-89e4-3cdc44fd2267/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:22.069 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:22.070 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:22.070 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239265 s, 43.8 MB/s 2021-02-10T11:24:22.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:22.071 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:22.071 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:22.407 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:22.407 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:22.408 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839299/aed9c96e-538c-4e75-89e4-3cdc44fd2267/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:22.784 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:22.785 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:22.785 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246656 s, 42.5 MB/s 2021-02-10T11:24:22.786 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:22.786 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:22.786 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:23.123 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:23.124 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:23.124 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839299/aed9c96e-538c-4e75-89e4-3cdc44fd2267/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:23.500 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:23.500 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:23.500 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252762 s, 41.5 MB/s 2021-02-10T11:24:23.501 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:23.501 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:23.502 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:23.837 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:23.837 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:23.837 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839299/aed9c96e-538c-4e75-89e4-3cdc44fd2267/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:24.209 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:24.210 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:24.210 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024862 s, 42.2 MB/s 2021-02-10T11:24:24.211 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839300 2021-02-10T11:24:24.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.617 7f4756308700 1 Processor -- start 2021-02-10T11:24:24.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.617 7f4756308700 1 -- start start 2021-02-10T11:24:24.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.617 7f4756308700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f47501b0470 0x7f47501b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:24.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.617 7f4756308700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47500f20e0 0x7f47500f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:24.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.617 7f4756308700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47500f0b50 con 0x7f47500ef610 2021-02-10T11:24:24.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.617 7f4756308700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47500f0820 con 0x7f47501b0470 2021-02-10T11:24:24.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.617 7f4756308700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47500f24e0 con 0x7f47500f20e0 2021-02-10T11:24:24.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.618 7f474ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f47501b0470 0x7f47501b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:24.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.618 7f474ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f47501b0470 0x7f47501b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47364/0 (socket says 172.21.2.101:47364) 2021-02-10T11:24:24.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.618 7f474ffff700 1 -- 172.21.2.101:0/471926086 learned_addr learned my addr 172.21.2.101:0/471926086 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:24.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.618 7f474ffff700 1 -- 172.21.2.101:0/471926086 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47500f20e0 msgr2=0x7f47500f0200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:24:24.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.618 7f474ffff700 1 --2- 172.21.2.101:0/471926086 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47500f20e0 0x7f47500f0200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:24.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.618 7f474ffff700 1 -- 172.21.2.101:0/471926086 >> v1:172.21.2.108:6789/0 conn(0x7f47500ef610 legacy=0x7f47500efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:24:24.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.618 7f474ffff700 1 -- 172.21.2.101:0/471926086 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47500e52e0 con 0x7f47501b0470 2021-02-10T11:24:24.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.618 7f474ffff700 1 --2- 172.21.2.101:0/471926086 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47501b0470 0x7f47501b0870 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f4740006700 tx=0x7f4740009b80).ready entity=mon.0 client_cookie=59bf29a34392166b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:24.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.618 7f474effd700 1 -- 172.21.2.101:0/471926086 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f474000c0c0 con 0x7f47501b0470 2021-02-10T11:24:24.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.618 7f474effd700 1 -- 172.21.2.101:0/471926086 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474000c340 con 0x7f47501b0470 2021-02-10T11:24:24.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.618 7f474effd700 1 -- 172.21.2.101:0/471926086 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f474000c860 con 0x7f47501b0470 2021-02-10T11:24:24.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.619 7f4756308700 1 -- 172.21.2.101:0/471926086 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47501b0470 msgr2=0x7f47501b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:24.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.619 7f4756308700 1 --2- 172.21.2.101:0/471926086 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47501b0470 0x7f47501b0870 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f4740006700 tx=0x7f4740009b80).stop 2021-02-10T11:24:24.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.619 7f4756308700 1 -- 172.21.2.101:0/471926086 shutdown_connections 2021-02-10T11:24:24.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.619 7f4756308700 1 --2- 172.21.2.101:0/471926086 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47500f20e0 0x7f47500f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:24.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.619 7f4756308700 1 --2- 172.21.2.101:0/471926086 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47501b0470 0x7f47501b0870 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:24.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.619 7f4756308700 1 -- 172.21.2.101:0/471926086 >> 172.21.2.101:0/471926086 conn(0x7f47500ce3e0 msgr2=0x7f47500e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:24.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.619 7f4756308700 1 -- 172.21.2.101:0/471926086 shutdown_connections 2021-02-10T11:24:24.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.619 7f4756308700 1 -- 172.21.2.101:0/471926086 wait complete. 2021-02-10T11:24:24.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.634 7f4756308700 1 Processor -- start 2021-02-10T11:24:24.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.634 7f4756308700 1 -- start start 2021-02-10T11:24:24.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.634 7f4756308700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47500c36a0 0x7f47500c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:24.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.634 7f4756308700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f47500c3190 0x7f47500b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:24.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.634 7f4756308700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47500b2370 0x7f47500b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:24.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.634 7f4756308700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47500b27c0 con 0x7f47500b2370 2021-02-10T11:24:24.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.634 7f4756308700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47500b3360 con 0x7f47500c3190 2021-02-10T11:24:24.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.634 7f4756308700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47500b3540 con 0x7f47500c36a0 2021-02-10T11:24:24.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.634 7f4754925700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47500b2370 0x7f47500b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:24.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.634 7f4754925700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47500b2370 0x7f47500b2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47368/0 (socket says 172.21.2.101:47368) 2021-02-10T11:24:24.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.634 7f4754925700 1 -- 172.21.2.101:0/1997570248 learned_addr learned my addr 172.21.2.101:0/1997570248 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:24.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.635 7f4754925700 1 -- 172.21.2.101:0/1997570248 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47500c36a0 msgr2=0x7f47500c2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:24.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.635 7f4754925700 1 --2- 172.21.2.101:0/1997570248 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47500c36a0 0x7f47500c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:24.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.635 7f4754925700 1 -- 172.21.2.101:0/1997570248 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f47500c3190 msgr2=0x7f47500b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:24.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.635 7f4754925700 1 --2- 172.21.2.101:0/1997570248 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f47500c3190 0x7f47500b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:24.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.635 7f4754925700 1 -- 172.21.2.101:0/1997570248 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47500b3760 con 0x7f47500b2370 2021-02-10T11:24:24.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.635 7f4754925700 1 --2- 172.21.2.101:0/1997570248 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47500b2370 0x7f47500b2d40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f4748009960 tx=0x7f474800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:24.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.635 7f474cff9700 1 -- 172.21.2.101:0/1997570248 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4748000d10 con 0x7f47500b2370 2021-02-10T11:24:24.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.635 7f474cff9700 1 -- 172.21.2.101:0/1997570248 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4748003030 con 0x7f47500b2370 2021-02-10T11:24:24.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.635 7f4756308700 1 -- 172.21.2.101:0/1997570248 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47500b3760 con 0x7f47500b2370 2021-02-10T11:24:24.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.635 7f4756308700 1 -- 172.21.2.101:0/1997570248 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47500b3cf0 con 0x7f47500b2370 2021-02-10T11:24:24.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.635 7f474cff9700 1 -- 172.21.2.101:0/1997570248 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4748001590 con 0x7f47500b2370 2021-02-10T11:24:24.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.636 7f474cff9700 1 -- 172.21.2.101:0/1997570248 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4748001310 con 0x7f47500b2370 2021-02-10T11:24:24.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.636 7f474cff9700 1 --2- 172.21.2.101:0/1997570248 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f47300186c0 0x7f473001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:24.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.636 7f474cff9700 1 -- 172.21.2.101:0/1997570248 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f47480328c0 con 0x7f47500b2370 2021-02-10T11:24:24.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.636 7f474ffff700 1 --2- 172.21.2.101:0/1997570248 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f47300186c0 0x7f473001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:24.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.636 7f474ffff700 1 --2- 172.21.2.101:0/1997570248 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f47300186c0 0x7f473001abc0 crc :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:24.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.667 7f4756308700 1 -- 172.21.2.101:0/1997570248 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47500edd10 con 0x7f47500b2370 2021-02-10T11:24:24.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.670 7f474cff9700 1 -- 172.21.2.101:0/1997570248 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4748000f40 con 0x7f47500b2370 2021-02-10T11:24:24.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.796 7f4756308700 1 -- 172.21.2.101:0/1997570248 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839300", "target": ["mgr", ""]}) v1 -- 0x7f47500b27c0 con 0x7f47300186c0 2021-02-10T11:24:24.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.824 7f474cff9700 1 -- 172.21.2.101:0/1997570248 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f4740000990 con 0x7f47300186c0 2021-02-10T11:24:24.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.828 7f4756308700 1 -- 172.21.2.101:0/1997570248 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f47300186c0 msgr2=0x7f473001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:24.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.828 7f4756308700 1 --2- 172.21.2.101:0/1997570248 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f47300186c0 0x7f473001abc0 crc :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:24.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.828 7f4756308700 1 -- 172.21.2.101:0/1997570248 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47500b2370 msgr2=0x7f47500b2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:24.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.828 7f4756308700 1 --2- 172.21.2.101:0/1997570248 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47500b2370 0x7f47500b2d40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f4748009960 tx=0x7f474800bcf0).stop 2021-02-10T11:24:24.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.828 7f4756308700 1 -- 172.21.2.101:0/1997570248 shutdown_connections 2021-02-10T11:24:24.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.828 7f4756308700 1 --2- 172.21.2.101:0/1997570248 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47500b2370 0x7f47500b2d40 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:24.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.828 7f4756308700 1 --2- 172.21.2.101:0/1997570248 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f47500c3190 0x7f47500b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:24.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.828 7f4756308700 1 --2- 172.21.2.101:0/1997570248 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f47300186c0 0x7f473001abc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:24.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.828 7f4756308700 1 --2- 172.21.2.101:0/1997570248 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47500c36a0 0x7f47500c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:24.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.828 7f4756308700 1 -- 172.21.2.101:0/1997570248 >> 172.21.2.101:0/1997570248 conn(0x7f475009f630 msgr2=0x7f47500a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:24.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.829 7f4756308700 1 -- 172.21.2.101:0/1997570248 shutdown_connections 2021-02-10T11:24:24.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:24.829 7f4756308700 1 -- 172.21.2.101:0/1997570248 wait complete. 2021-02-10T11:24:24.846 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839300 2021-02-10T11:24:25.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.252 7f14d899c700 1 Processor -- start 2021-02-10T11:24:25.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.252 7f14d899c700 1 -- start start 2021-02-10T11:24:25.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.253 7f14d899c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14d41b0870 0x7f14d40f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:25.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.253 7f14d899c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d41b0c70 con 0x7f14d40efe00 2021-02-10T11:24:25.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.253 7f14d899c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d40f0b50 con 0x7f14d41b0470 2021-02-10T11:24:25.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.253 7f14d899c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d40e5ea0 con 0x7f14d41b0870 2021-02-10T11:24:25.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.253 7f14d2e1d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f14d40efe00 0x7f14d40f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47874/0 (socket says 172.21.2.101:47874) 2021-02-10T11:24:25.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.253 7f14d2e1d700 1 -- 172.21.2.101:0/2952106104 learned_addr learned my addr 172.21.2.101:0/2952106104 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:24:25.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.254 7f14d1e1b700 1 --2- 172.21.2.101:0/2952106104 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14d41b0870 0x7f14d40f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:25.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.254 7f14d161a700 1 -- 172.21.2.101:0/2952106104 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901709134 0 0) 0x7f14cc001500 con 0x7f14d40efe00 2021-02-10T11:24:25.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.254 7f14d161a700 1 -- 172.21.2.101:0/2952106104 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14bc002fe0 con 0x7f14d40efe00 2021-02-10T11:24:25.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.254 7f14d161a700 1 -- 172.21.2.101:0/2952106104 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2790819291 0 0) 0x7f14c4001080 con 0x7f14d41b0470 2021-02-10T11:24:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.254 7f14d161a700 1 -- 172.21.2.101:0/2952106104 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14bc003210 con 0x7f14d41b0470 2021-02-10T11:24:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.254 7f14d1e1b700 1 -- 172.21.2.101:0/2952106104 >> v1:172.21.2.108:6789/0 conn(0x7f14d41b0470 legacy=0x7f14d40ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.254 7f14d161a700 1 -- 172.21.2.101:0/2952106104 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 933441145 0 0) 0x7f14cc001500 con 0x7f14d40efe00 2021-02-10T11:24:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.254 7f14d1e1b700 1 -- 172.21.2.101:0/2952106104 >> v1:172.21.2.101:6789/0 conn(0x7f14d40efe00 legacy=0x7f14d40f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.254 7f14d1e1b700 1 -- 172.21.2.101:0/2952106104 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d40e52e0 con 0x7f14d41b0870 2021-02-10T11:24:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.254 7f14d1e1b700 1 --2- 172.21.2.101:0/2952106104 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14d41b0870 0x7f14d40f20e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f14c8006650 tx=0x7f14c8009a80).ready entity=mon.2 client_cookie=9507fdb82b26e399 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.255 7f14d161a700 1 -- 172.21.2.101:0/2952106104 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f14c80009b0 con 0x7f14d41b0870 2021-02-10T11:24:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.255 7f14d161a700 1 -- 172.21.2.101:0/2952106104 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c800c0c0 con 0x7f14d41b0870 2021-02-10T11:24:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.255 7f14d161a700 1 -- 172.21.2.101:0/2952106104 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f14c800c5e0 con 0x7f14d41b0870 2021-02-10T11:24:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.255 7f14d899c700 1 -- 172.21.2.101:0/2952106104 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14d41b0870 msgr2=0x7f14d40f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.255 7f14d899c700 1 --2- 172.21.2.101:0/2952106104 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14d41b0870 0x7f14d40f20e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f14c8006650 tx=0x7f14c8009a80).stop 2021-02-10T11:24:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.256 7f14d899c700 1 -- 172.21.2.101:0/2952106104 shutdown_connections 2021-02-10T11:24:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.256 7f14d899c700 1 --2- 172.21.2.101:0/2952106104 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14d41b0870 0x7f14d40f20e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:25.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.256 7f14d899c700 1 -- 172.21.2.101:0/2952106104 >> 172.21.2.101:0/2952106104 conn(0x7f14d40ce3e0 msgr2=0x7f14d40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:25.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.256 7f14d899c700 1 -- 172.21.2.101:0/2952106104 shutdown_connections 2021-02-10T11:24:25.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.256 7f14d899c700 1 -- 172.21.2.101:0/2952106104 wait complete. 2021-02-10T11:24:25.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.270 7f14d899c700 1 Processor -- start 2021-02-10T11:24:25.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.270 7f14d899c700 1 -- start start 2021-02-10T11:24:25.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.271 7f14d899c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14d4190920 0x7f14d40b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:25.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.271 7f14d899c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14d40b8200 0x7f14d40b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.271 7f14d899c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14d40c1d80 0x7f14d40c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.271 7f14d899c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d4190d20 con 0x7f14d4190920 2021-02-10T11:24:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.271 7f14d899c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d40b8c20 con 0x7f14d40c1d80 2021-02-10T11:24:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.271 7f14d899c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d40c27a0 con 0x7f14d40b8200 2021-02-10T11:24:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.271 7f14d261c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14d4190920 0x7f14d40b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.271 7f14d261c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14d4190920 0x7f14d40b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47382/0 (socket says 172.21.2.101:47382) 2021-02-10T11:24:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.271 7f14d261c700 1 -- 172.21.2.101:0/1288550972 learned_addr learned my addr 172.21.2.101:0/1288550972 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.271 7f14d1e1b700 1 --2- 172.21.2.101:0/1288550972 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14d40b8200 0x7f14d40b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.271 7f14d2e1d700 1 --2- 172.21.2.101:0/1288550972 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14d40c1d80 0x7f14d40c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.272 7f14d261c700 1 -- 172.21.2.101:0/1288550972 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14d40b8200 msgr2=0x7f14d40b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.272 7f14d261c700 1 --2- 172.21.2.101:0/1288550972 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14d40b8200 0x7f14d40b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.272 7f14d261c700 1 -- 172.21.2.101:0/1288550972 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14d40c1d80 msgr2=0x7f14d40c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.272 7f14d261c700 1 --2- 172.21.2.101:0/1288550972 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14d40c1d80 0x7f14d40c2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.272 7f14d261c700 1 -- 172.21.2.101:0/1288550972 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d40c29c0 con 0x7f14d4190920 2021-02-10T11:24:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.272 7f14d2e1d700 1 --2- 172.21.2.101:0/1288550972 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14d40c1d80 0x7f14d40c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:24:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.272 7f14d261c700 1 --2- 172.21.2.101:0/1288550972 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14d4190920 0x7f14d40b7be0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f14c400b5b0 tx=0x7f14c400b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.272 7f14c2ffd700 1 -- 172.21.2.101:0/1288550972 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f14c4006240 con 0x7f14d4190920 2021-02-10T11:24:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.272 7f14c2ffd700 1 -- 172.21.2.101:0/1288550972 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c40054a0 con 0x7f14d4190920 2021-02-10T11:24:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.272 7f14c2ffd700 1 -- 172.21.2.101:0/1288550972 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f14c4005900 con 0x7f14d4190920 2021-02-10T11:24:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.272 7f14d899c700 1 -- 172.21.2.101:0/1288550972 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d40c29c0 con 0x7f14d4190920 2021-02-10T11:24:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.272 7f14d899c700 1 -- 172.21.2.101:0/1288550972 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d40c29c0 con 0x7f14d4190920 2021-02-10T11:24:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.273 7f14c2ffd700 1 -- 172.21.2.101:0/1288550972 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f14c4006030 con 0x7f14d4190920 2021-02-10T11:24:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.273 7f14c2ffd700 1 --2- 172.21.2.101:0/1288550972 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14b80186c0 0x7f14b801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.273 7f14c2ffd700 1 -- 172.21.2.101:0/1288550972 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f14c40326f0 con 0x7f14d4190920 2021-02-10T11:24:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.273 7f14d1e1b700 1 --2- 172.21.2.101:0/1288550972 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14b80186c0 0x7f14b801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.273 7f14d1e1b700 1 --2- 172.21.2.101:0/1288550972 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14b80186c0 0x7f14b801abc0 crc :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:25.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.303 7f14d899c700 1 -- 172.21.2.101:0/1288550972 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d4105ae0 con 0x7f14d4190920 2021-02-10T11:24:25.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.305 7f14c2ffd700 1 -- 172.21.2.101:0/1288550972 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f14c4005dd0 con 0x7f14d4190920 2021-02-10T11:24:25.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.433 7f14d899c700 1 -- 172.21.2.101:0/1288550972 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839300", "target": ["mgr", ""]}) v1 -- 0x7f14d40b8c20 con 0x7f14b80186c0 2021-02-10T11:24:25.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.436 7f14c2ffd700 1 -- 172.21.2.101:0/1288550972 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f14c8000c90 con 0x7f14b80186c0 2021-02-10T11:24:25.445 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839300/adcab26b-87ff-47b8-a48d-bc794fc85854 2021-02-10T11:24:25.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.438 7f14d899c700 1 -- 172.21.2.101:0/1288550972 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14b80186c0 msgr2=0x7f14b801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:25.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.438 7f14d899c700 1 --2- 172.21.2.101:0/1288550972 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14b80186c0 0x7f14b801abc0 crc :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:25.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.438 7f14d899c700 1 -- 172.21.2.101:0/1288550972 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14d4190920 msgr2=0x7f14d40b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:25.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.438 7f14d899c700 1 --2- 172.21.2.101:0/1288550972 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14d4190920 0x7f14d40b7be0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f14c400b5b0 tx=0x7f14c400b950).stop 2021-02-10T11:24:25.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.438 7f14d899c700 1 -- 172.21.2.101:0/1288550972 shutdown_connections 2021-02-10T11:24:25.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.438 7f14d899c700 1 --2- 172.21.2.101:0/1288550972 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14b80186c0 0x7f14b801abc0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:25.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.438 7f14d899c700 1 --2- 172.21.2.101:0/1288550972 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14d40b8200 0x7f14d40b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:25.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.438 7f14d899c700 1 --2- 172.21.2.101:0/1288550972 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14d40c1d80 0x7f14d40c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:25.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.438 7f14d899c700 1 --2- 172.21.2.101:0/1288550972 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14d4190920 0x7f14d40b7be0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:25.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.438 7f14d899c700 1 -- 172.21.2.101:0/1288550972 >> 172.21.2.101:0/1288550972 conn(0x7f14d4094e10 msgr2=0x7f14d40a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:25.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.438 7f14d899c700 1 -- 172.21.2.101:0/1288550972 shutdown_connections 2021-02-10T11:24:25.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:25.438 7f14d899c700 1 -- 172.21.2.101:0/1288550972 wait complete. 2021-02-10T11:24:25.455 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839300 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839300/adcab26b-87ff-47b8-a48d-bc794fc85854 2021-02-10T11:24:25.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:25.455 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:25.455 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:25.790 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:25.835 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:25.835 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839300/adcab26b-87ff-47b8-a48d-bc794fc85854/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:26.217 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:26.218 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:26.218 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023465 s, 44.7 MB/s 2021-02-10T11:24:26.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:26.219 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:26.219 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:26.554 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:26.555 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:26.555 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839300/adcab26b-87ff-47b8-a48d-bc794fc85854/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:26.932 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:26.933 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:26.933 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024676 s, 42.5 MB/s 2021-02-10T11:24:26.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:26.934 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:26.934 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:27.270 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:27.271 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:27.271 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839300/adcab26b-87ff-47b8-a48d-bc794fc85854/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:27.646 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:27.647 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:27.647 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244466 s, 42.9 MB/s 2021-02-10T11:24:27.648 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:27.648 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:27.648 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:27.982 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:27.982 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:27.983 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839300/adcab26b-87ff-47b8-a48d-bc794fc85854/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:28.358 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:28.358 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:28.358 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238217 s, 44.0 MB/s 2021-02-10T11:24:28.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:28.359 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:28.359 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:28.694 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:28.694 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:28.695 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839300/adcab26b-87ff-47b8-a48d-bc794fc85854/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:29.067 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:29.067 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:29.067 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247948 s, 42.3 MB/s 2021-02-10T11:24:29.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:29.068 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:29.069 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:29.403 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:29.403 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:29.403 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839300/adcab26b-87ff-47b8-a48d-bc794fc85854/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:29.777 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:29.777 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:29.777 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233542 s, 44.9 MB/s 2021-02-10T11:24:29.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:29.779 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:29.779 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:30.114 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:30.114 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:30.114 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839300/adcab26b-87ff-47b8-a48d-bc794fc85854/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:30.492 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:30.492 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:30.492 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251996 s, 41.6 MB/s 2021-02-10T11:24:30.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:30.493 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:30.493 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:30.830 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:30.830 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:30.830 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839300/adcab26b-87ff-47b8-a48d-bc794fc85854/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:31.207 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:31.208 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:31.208 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233365 s, 44.9 MB/s 2021-02-10T11:24:31.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:31.209 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:31.209 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:31.544 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:31.544 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:31.545 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839300/adcab26b-87ff-47b8-a48d-bc794fc85854/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:31.918 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:31.918 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:31.918 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0226657 s, 46.3 MB/s 2021-02-10T11:24:31.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:31.919 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:31.919 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:32.255 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:32.255 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:32.255 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839300/adcab26b-87ff-47b8-a48d-bc794fc85854/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:32.629 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:32.630 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:32.630 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0227805 s, 46.0 MB/s 2021-02-10T11:24:32.631 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839301 2021-02-10T11:24:33.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.037 7fab44cb0700 1 Processor -- start 2021-02-10T11:24:33.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.037 7fab44cb0700 1 -- start start 2021-02-10T11:24:33.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.038 7fab44cb0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fab401b0870 0x7fab400f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:33.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.038 7fab44cb0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab400efe00 0x7fab400f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:33.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.038 7fab44cb0700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab401b0c70 con 0x7fab401b0470 2021-02-10T11:24:33.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.038 7fab44cb0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab400f0b50 con 0x7fab401b0870 2021-02-10T11:24:33.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.038 7fab44cb0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab400f0820 con 0x7fab400efe00 2021-02-10T11:24:33.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.038 7fab3e05d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fab401b0870 0x7fab400f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:33.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.038 7fab3e05d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fab401b0870 0x7fab400f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47392/0 (socket says 172.21.2.101:47392) 2021-02-10T11:24:33.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.038 7fab3e05d700 1 -- 172.21.2.101:0/3245338109 learned_addr learned my addr 172.21.2.101:0/3245338109 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:33.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.038 7fab3e05d700 1 -- 172.21.2.101:0/3245338109 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab400efe00 msgr2=0x7fab400f0200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:24:33.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.038 7fab3e05d700 1 --2- 172.21.2.101:0/3245338109 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab400efe00 0x7fab400f0200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:33.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.038 7fab3e05d700 1 -- 172.21.2.101:0/3245338109 >> v1:172.21.2.101:6789/0 conn(0x7fab401b0470 legacy=0x7fab400ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:33.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.038 7fab3e05d700 1 -- 172.21.2.101:0/3245338109 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab400e52e0 con 0x7fab401b0870 2021-02-10T11:24:33.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.038 7fab3e05d700 1 --2- 172.21.2.101:0/3245338109 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab401b0870 0x7fab400f20e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7fab30006700 tx=0x7fab30009b80).ready entity=mon.0 client_cookie=73b078ac3ec4e647 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:33.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.039 7fab3d85c700 1 -- 172.21.2.101:0/3245338109 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fab300009b0 con 0x7fab401b0870 2021-02-10T11:24:33.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.039 7fab3d85c700 1 -- 172.21.2.101:0/3245338109 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3000c0c0 con 0x7fab401b0870 2021-02-10T11:24:33.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.039 7fab3d85c700 1 -- 172.21.2.101:0/3245338109 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fab3000c5e0 con 0x7fab401b0870 2021-02-10T11:24:33.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.039 7fab44cb0700 1 -- 172.21.2.101:0/3245338109 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab401b0870 msgr2=0x7fab400f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:33.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.039 7fab44cb0700 1 --2- 172.21.2.101:0/3245338109 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab401b0870 0x7fab400f20e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7fab30006700 tx=0x7fab30009b80).stop 2021-02-10T11:24:33.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.039 7fab44cb0700 1 -- 172.21.2.101:0/3245338109 shutdown_connections 2021-02-10T11:24:33.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.039 7fab44cb0700 1 --2- 172.21.2.101:0/3245338109 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab400efe00 0x7fab400f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:33.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.039 7fab44cb0700 1 --2- 172.21.2.101:0/3245338109 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab401b0870 0x7fab400f20e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:33.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.039 7fab44cb0700 1 -- 172.21.2.101:0/3245338109 >> 172.21.2.101:0/3245338109 conn(0x7fab400ce3e0 msgr2=0x7fab400e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:33.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.039 7fab44cb0700 1 -- 172.21.2.101:0/3245338109 shutdown_connections 2021-02-10T11:24:33.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.039 7fab44cb0700 1 -- 172.21.2.101:0/3245338109 wait complete. 2021-02-10T11:24:33.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.054 7fab44cb0700 1 Processor -- start 2021-02-10T11:24:33.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.054 7fab44cb0700 1 -- start start 2021-02-10T11:24:33.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.054 7fab44cb0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab400b2500 0x7fab400b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:33.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.054 7fab44cb0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab400b3080 0x7fab400b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:33.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.054 7fab44cb0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fab400b3aa0 0x7fab400f0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:33.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.054 7fab44cb0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab400effb0 con 0x7fab400b2500 2021-02-10T11:24:33.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.054 7fab44cb0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab400f0b50 con 0x7fab400b3aa0 2021-02-10T11:24:33.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.054 7fab44cb0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab400f0d30 con 0x7fab400b3080 2021-02-10T11:24:33.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.054 7fab3e85e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab400b2500 0x7fab400b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:33.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.054 7fab3e85e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab400b2500 0x7fab400b2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47398/0 (socket says 172.21.2.101:47398) 2021-02-10T11:24:33.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.054 7fab3e85e700 1 -- 172.21.2.101:0/2581407639 learned_addr learned my addr 172.21.2.101:0/2581407639 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:33.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.055 7fab3e85e700 1 -- 172.21.2.101:0/2581407639 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab400b3080 msgr2=0x7fab400b3480 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:24:33.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.055 7fab3e85e700 1 --2- 172.21.2.101:0/2581407639 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab400b3080 0x7fab400b3480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:33.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.055 7fab3e85e700 1 -- 172.21.2.101:0/2581407639 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fab400b3aa0 msgr2=0x7fab400f0530 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:24:33.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.055 7fab3e85e700 1 --2- 172.21.2.101:0/2581407639 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fab400b3aa0 0x7fab400f0530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:33.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.055 7fab3e85e700 1 -- 172.21.2.101:0/2581407639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab400f0f50 con 0x7fab400b2500 2021-02-10T11:24:33.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.055 7fab3e85e700 1 --2- 172.21.2.101:0/2581407639 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab400b2500 0x7fab400b2900 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7fab28006f00 tx=0x7fab2800b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:33.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.055 7fab377fe700 1 -- 172.21.2.101:0/2581407639 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fab28007070 con 0x7fab400b2500 2021-02-10T11:24:33.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.055 7fab377fe700 1 -- 172.21.2.101:0/2581407639 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab28007330 con 0x7fab400b2500 2021-02-10T11:24:33.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.055 7fab377fe700 1 -- 172.21.2.101:0/2581407639 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fab28007850 con 0x7fab400b2500 2021-02-10T11:24:33.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.055 7fab44cb0700 1 -- 172.21.2.101:0/2581407639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab400f0f50 con 0x7fab400b2500 2021-02-10T11:24:33.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.055 7fab44cb0700 1 -- 172.21.2.101:0/2581407639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab400f14e0 con 0x7fab400b2500 2021-02-10T11:24:33.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.056 7fab377fe700 1 -- 172.21.2.101:0/2581407639 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fab28007000 con 0x7fab400b2500 2021-02-10T11:24:33.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.056 7fab377fe700 1 --2- 172.21.2.101:0/2581407639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab240186c0 0x7fab2401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:33.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.056 7fab377fe700 1 -- 172.21.2.101:0/2581407639 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fab28032610 con 0x7fab400b2500 2021-02-10T11:24:33.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.056 7fab3e05d700 1 --2- 172.21.2.101:0/2581407639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab240186c0 0x7fab2401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:33.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.056 7fab3e05d700 1 --2- 172.21.2.101:0/2581407639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab240186c0 0x7fab2401abc0 crc :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:33.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.087 7fab44cb0700 1 -- 172.21.2.101:0/2581407639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab400b3ef0 con 0x7fab400b2500 2021-02-10T11:24:33.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.090 7fab377fe700 1 -- 172.21.2.101:0/2581407639 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fab28030dc0 con 0x7fab400b2500 2021-02-10T11:24:33.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.216 7fab44cb0700 1 -- 172.21.2.101:0/2581407639 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839301", "target": ["mgr", ""]}) v1 -- 0x7fab400effb0 con 0x7fab240186c0 2021-02-10T11:24:33.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.244 7fab377fe700 1 -- 172.21.2.101:0/2581407639 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fab30009100 con 0x7fab240186c0 2021-02-10T11:24:33.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.249 7fab44cb0700 1 -- 172.21.2.101:0/2581407639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab240186c0 msgr2=0x7fab2401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:33.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.249 7fab44cb0700 1 --2- 172.21.2.101:0/2581407639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab240186c0 0x7fab2401abc0 crc :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:33.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.249 7fab44cb0700 1 -- 172.21.2.101:0/2581407639 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab400b2500 msgr2=0x7fab400b2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:33.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.249 7fab44cb0700 1 --2- 172.21.2.101:0/2581407639 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab400b2500 0x7fab400b2900 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7fab28006f00 tx=0x7fab2800b940).stop 2021-02-10T11:24:33.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.249 7fab44cb0700 1 -- 172.21.2.101:0/2581407639 shutdown_connections 2021-02-10T11:24:33.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.249 7fab44cb0700 1 --2- 172.21.2.101:0/2581407639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab240186c0 0x7fab2401abc0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:33.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.249 7fab44cb0700 1 --2- 172.21.2.101:0/2581407639 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab400b3080 0x7fab400b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:33.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.249 7fab44cb0700 1 --2- 172.21.2.101:0/2581407639 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fab400b3aa0 0x7fab400f0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:33.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.249 7fab44cb0700 1 --2- 172.21.2.101:0/2581407639 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab400b2500 0x7fab400b2900 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:33.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.249 7fab44cb0700 1 -- 172.21.2.101:0/2581407639 >> 172.21.2.101:0/2581407639 conn(0x7fab400c3c70 msgr2=0x7fab40094c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:33.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.250 7fab44cb0700 1 -- 172.21.2.101:0/2581407639 shutdown_connections 2021-02-10T11:24:33.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.250 7fab44cb0700 1 -- 172.21.2.101:0/2581407639 wait complete. 2021-02-10T11:24:33.266 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839301 2021-02-10T11:24:33.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.672 7f6146ef5700 1 Processor -- start 2021-02-10T11:24:33.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.672 7f6146ef5700 1 -- start start 2021-02-10T11:24:33.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.672 7f6146ef5700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f61401b0470 0x7f61401b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:33.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.672 7f6146ef5700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f61400f20e0 0x7f61400f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:33.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.672 7f6146ef5700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61400f0b50 con 0x7f61400ef610 2021-02-10T11:24:33.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.673 7f6146ef5700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61400f0820 con 0x7f61401b0470 2021-02-10T11:24:33.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.673 7f6146ef5700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61400f24e0 con 0x7f61400f20e0 2021-02-10T11:24:33.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.673 7f6144d11700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f61401b0470 0x7f61401b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.673 7f6145512700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f61400f20e0 0x7f61400f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.673 7f6144d11700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f61401b0470 0x7f61401b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47404/0 (socket says 172.21.2.101:47404) 2021-02-10T11:24:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.673 7f6144d11700 1 -- 172.21.2.101:0/1837306284 learned_addr learned my addr 172.21.2.101:0/1837306284 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.673 7f6144d11700 1 -- 172.21.2.101:0/1837306284 >> v2:172.21.2.108:3300/0 conn(0x7f61400f20e0 msgr2=0x7f61400f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:24:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.673 7f6144d11700 1 --2- 172.21.2.101:0/1837306284 >> v2:172.21.2.108:3300/0 conn(0x7f61400f20e0 0x7f61400f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.673 7f6144d11700 1 -- 172.21.2.101:0/1837306284 >> v1:172.21.2.108:6789/0 conn(0x7f61400ef610 legacy=0x7f61400efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.673 7f6144d11700 1 -- 172.21.2.101:0/1837306284 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61400e52e0 con 0x7f61401b0470 2021-02-10T11:24:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.674 7f6144d11700 1 --2- 172.21.2.101:0/1837306284 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f61401b0470 0x7f61401b0870 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f6128006700 tx=0x7f6128009b80).ready entity=mon.0 client_cookie=1b1dff25b03c9074 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.674 7f613f7fe700 1 -- 172.21.2.101:0/1837306284 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f612800c0c0 con 0x7f61401b0470 2021-02-10T11:24:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.674 7f613f7fe700 1 -- 172.21.2.101:0/1837306284 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f612800c440 con 0x7f61401b0470 2021-02-10T11:24:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.674 7f613f7fe700 1 -- 172.21.2.101:0/1837306284 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f612800c960 con 0x7f61401b0470 2021-02-10T11:24:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.674 7f6146ef5700 1 -- 172.21.2.101:0/1837306284 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f61401b0470 msgr2=0x7f61401b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.674 7f6146ef5700 1 --2- 172.21.2.101:0/1837306284 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f61401b0470 0x7f61401b0870 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f6128006700 tx=0x7f6128009b80).stop 2021-02-10T11:24:33.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.675 7f6146ef5700 1 -- 172.21.2.101:0/1837306284 shutdown_connections 2021-02-10T11:24:33.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.675 7f6146ef5700 1 --2- 172.21.2.101:0/1837306284 >> v2:172.21.2.108:3300/0 conn(0x7f61400f20e0 0x7f61400f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:33.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.675 7f6146ef5700 1 --2- 172.21.2.101:0/1837306284 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f61401b0470 0x7f61401b0870 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:33.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.675 7f6146ef5700 1 -- 172.21.2.101:0/1837306284 >> 172.21.2.101:0/1837306284 conn(0x7f61400ce3e0 msgr2=0x7f61400e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:33.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.675 7f6146ef5700 1 -- 172.21.2.101:0/1837306284 shutdown_connections 2021-02-10T11:24:33.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.675 7f6146ef5700 1 -- 172.21.2.101:0/1837306284 wait complete. 2021-02-10T11:24:33.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.689 7f6146ef5700 1 Processor -- start 2021-02-10T11:24:33.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.690 7f6146ef5700 1 -- start start 2021-02-10T11:24:33.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.690 7f6146ef5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f61400a0420 0x7f61400a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:33.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.690 7f6146ef5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6140189d00 0x7f614018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:33.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.690 7f6146ef5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f614018a720 0x7f61400b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:33.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.690 7f6146ef5700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6140189780 con 0x7f614018a720 2021-02-10T11:24:33.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.690 7f6146ef5700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614018ab20 con 0x7f61400a0420 2021-02-10T11:24:33.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.690 7f6146ef5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61400a1140 con 0x7f6140189d00 2021-02-10T11:24:33.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.690 7f6145512700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f614018a720 0x7f61400b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:33.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.690 7f6145512700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f614018a720 0x7f61400b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47410/0 (socket says 172.21.2.101:47410) 2021-02-10T11:24:33.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.690 7f613ffff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6140189d00 0x7f614018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:33.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.690 7f6145512700 1 -- 172.21.2.101:0/462116690 learned_addr learned my addr 172.21.2.101:0/462116690 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:33.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.690 7f6144d11700 1 --2- 172.21.2.101:0/462116690 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f61400a0420 0x7f61400a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:33.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.691 7f613ffff700 1 -- 172.21.2.101:0/462116690 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f61400a0420 msgr2=0x7f61400a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:33.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.691 7f613ffff700 1 --2- 172.21.2.101:0/462116690 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f61400a0420 0x7f61400a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:33.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.691 7f613ffff700 1 -- 172.21.2.101:0/462116690 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f614018a720 msgr2=0x7f61400b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:33.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.691 7f613ffff700 1 --2- 172.21.2.101:0/462116690 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f614018a720 0x7f61400b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:33.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.691 7f613ffff700 1 -- 172.21.2.101:0/462116690 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61400b8200 con 0x7f6140189d00 2021-02-10T11:24:33.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.691 7f6144d11700 1 --2- 172.21.2.101:0/462116690 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f61400a0420 0x7f61400a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:24:33.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.691 7f6145512700 1 --2- 172.21.2.101:0/462116690 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f614018a720 0x7f61400b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:24:33.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.691 7f613ffff700 1 --2- 172.21.2.101:0/462116690 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6140189d00 0x7f614018a100 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f613000b8d0 tx=0x7f613000bc70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:33.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.691 7f613d7fa700 1 -- 172.21.2.101:0/462116690 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f613000c2f0 con 0x7f6140189d00 2021-02-10T11:24:33.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.691 7f613d7fa700 1 -- 172.21.2.101:0/462116690 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613000c2f0 con 0x7f6140189d00 2021-02-10T11:24:33.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.691 7f6146ef5700 1 -- 172.21.2.101:0/462116690 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61400b8200 con 0x7f6140189d00 2021-02-10T11:24:33.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.691 7f613d7fa700 1 -- 172.21.2.101:0/462116690 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f613000c5b0 con 0x7f6140189d00 2021-02-10T11:24:33.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.691 7f6146ef5700 1 -- 172.21.2.101:0/462116690 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61400b8200 con 0x7f6140189d00 2021-02-10T11:24:33.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.692 7f613d7fa700 1 -- 172.21.2.101:0/462116690 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6130003020 con 0x7f6140189d00 2021-02-10T11:24:33.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.692 7f613d7fa700 1 --2- 172.21.2.101:0/462116690 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61240186c0 0x7f612401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:33.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.692 7f613d7fa700 1 -- 172.21.2.101:0/462116690 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f61300326e0 con 0x7f6140189d00 2021-02-10T11:24:33.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.692 7f6144d11700 1 --2- 172.21.2.101:0/462116690 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61240186c0 0x7f612401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:33.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.692 7f6144d11700 1 --2- 172.21.2.101:0/462116690 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61240186c0 0x7f612401abc0 crc :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:33.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.722 7f6146ef5700 1 -- 172.21.2.101:0/462116690 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6140105ae0 con 0x7f6140189d00 2021-02-10T11:24:33.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.725 7f613d7fa700 1 -- 172.21.2.101:0/462116690 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f613000bdb0 con 0x7f6140189d00 2021-02-10T11:24:33.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.859 7f6146ef5700 1 -- 172.21.2.101:0/462116690 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839301", "target": ["mgr", ""]}) v1 -- 0x7f6140189780 con 0x7f61240186c0 2021-02-10T11:24:33.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.863 7f613d7fa700 1 -- 172.21.2.101:0/462116690 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f6128006e80 con 0x7f61240186c0 2021-02-10T11:24:33.875 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839301/ca62275b-fa49-467d-8ccb-0bf186524280 2021-02-10T11:24:33.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.867 7f6146ef5700 1 -- 172.21.2.101:0/462116690 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61240186c0 msgr2=0x7f612401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:33.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.867 7f6146ef5700 1 --2- 172.21.2.101:0/462116690 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61240186c0 0x7f612401abc0 crc :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:33.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.867 7f6146ef5700 1 -- 172.21.2.101:0/462116690 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6140189d00 msgr2=0x7f614018a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:33.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.867 7f6146ef5700 1 --2- 172.21.2.101:0/462116690 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6140189d00 0x7f614018a100 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f613000b8d0 tx=0x7f613000bc70).stop 2021-02-10T11:24:33.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.868 7f6146ef5700 1 -- 172.21.2.101:0/462116690 shutdown_connections 2021-02-10T11:24:33.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.868 7f6146ef5700 1 --2- 172.21.2.101:0/462116690 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61240186c0 0x7f612401abc0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:33.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.868 7f6146ef5700 1 --2- 172.21.2.101:0/462116690 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6140189d00 0x7f614018a100 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:33.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.868 7f6146ef5700 1 --2- 172.21.2.101:0/462116690 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f614018a720 0x7f61400b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:33.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.868 7f6146ef5700 1 --2- 172.21.2.101:0/462116690 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f61400a0420 0x7f61400a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:33.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.868 7f6146ef5700 1 -- 172.21.2.101:0/462116690 >> 172.21.2.101:0/462116690 conn(0x7f6140094e10 msgr2=0x7f61400a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:33.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.868 7f6146ef5700 1 -- 172.21.2.101:0/462116690 shutdown_connections 2021-02-10T11:24:33.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:33.868 7f6146ef5700 1 -- 172.21.2.101:0/462116690 wait complete. 2021-02-10T11:24:33.885 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839301 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839301/ca62275b-fa49-467d-8ccb-0bf186524280 2021-02-10T11:24:33.885 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:33.885 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:33.885 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:34.220 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:34.220 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:34.221 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839301/ca62275b-fa49-467d-8ccb-0bf186524280/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:34.600 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:34.600 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:34.600 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247071 s, 42.4 MB/s 2021-02-10T11:24:34.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:34.601 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:34.601 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:34.936 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:34.936 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:34.937 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839301/ca62275b-fa49-467d-8ccb-0bf186524280/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:35.313 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:35.313 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:35.313 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236507 s, 44.3 MB/s 2021-02-10T11:24:35.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:35.314 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:35.314 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:35.650 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:35.650 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:35.651 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839301/ca62275b-fa49-467d-8ccb-0bf186524280/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:36.028 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:36.028 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:36.029 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237784 s, 44.1 MB/s 2021-02-10T11:24:36.029 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:36.030 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:36.030 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:36.370 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:36.371 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:36.371 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839301/ca62275b-fa49-467d-8ccb-0bf186524280/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:36.747 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:36.748 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:36.748 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240329 s, 43.6 MB/s 2021-02-10T11:24:36.749 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:36.749 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:36.749 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:37.086 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:37.086 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:37.087 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839301/ca62275b-fa49-467d-8ccb-0bf186524280/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:37.464 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:37.464 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:37.464 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0255138 s, 41.1 MB/s 2021-02-10T11:24:37.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:37.465 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:37.465 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:37.800 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:37.801 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:37.801 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839301/ca62275b-fa49-467d-8ccb-0bf186524280/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:38.177 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:38.177 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:38.177 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246554 s, 42.5 MB/s 2021-02-10T11:24:38.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:38.178 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:38.178 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:38.514 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:38.515 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:38.515 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839301/ca62275b-fa49-467d-8ccb-0bf186524280/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:38.885 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:38.886 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:38.886 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235591 s, 44.5 MB/s 2021-02-10T11:24:38.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:38.887 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:38.887 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:39.223 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:39.224 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:39.224 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839301/ca62275b-fa49-467d-8ccb-0bf186524280/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:39.598 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:39.598 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:39.598 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0228376 s, 45.9 MB/s 2021-02-10T11:24:39.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:39.599 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:39.599 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:39.935 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:39.935 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:39.936 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839301/ca62275b-fa49-467d-8ccb-0bf186524280/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:40.311 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:40.312 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:40.312 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0227799 s, 46.0 MB/s 2021-02-10T11:24:40.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:40.313 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:40.313 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:40.647 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:40.647 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:40.648 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839301/ca62275b-fa49-467d-8ccb-0bf186524280/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:41.022 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:41.023 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:41.023 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242344 s, 43.3 MB/s 2021-02-10T11:24:41.024 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839302 2021-02-10T11:24:41.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.430 7ff95cdec700 1 Processor -- start 2021-02-10T11:24:41.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.430 7ff95cdec700 1 -- start start 2021-02-10T11:24:41.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.431 7ff95cdec700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff9581b0470 0x7ff9581b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:41.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.431 7ff95cdec700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff9580ef610 0x7ff9580efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:41.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.431 7ff95cdec700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9580f0030 0x7ff9580f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:41.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.431 7ff95cdec700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9580e5ea0 con 0x7ff9580ef610 2021-02-10T11:24:41.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.431 7ff95cdec700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9580f0b50 con 0x7ff9581b0470 2021-02-10T11:24:41.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.431 7ff95cdec700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9580e52e0 con 0x7ff9580f0030 2021-02-10T11:24:41.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.431 7ff956312700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff9580ef610 0x7ff9580efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:41.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.431 7ff956312700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff9580ef610 0x7ff9580efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47416/0 (socket says 172.21.2.101:47416) 2021-02-10T11:24:41.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.431 7ff956312700 1 -- 172.21.2.101:0/1872718966 learned_addr learned my addr 172.21.2.101:0/1872718966 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:41.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.431 7ff956312700 1 -- 172.21.2.101:0/1872718966 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9580f0030 msgr2=0x7ff9580f0430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:24:41.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.431 7ff956312700 1 --2- 172.21.2.101:0/1872718966 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9580f0030 0x7ff9580f0430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:41.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.431 7ff956312700 1 -- 172.21.2.101:0/1872718966 >> v2:172.21.2.108:3300/0 conn(0x7ff9581b0470 msgr2=0x7ff9581b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:24:41.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.431 7ff956312700 1 --2- 172.21.2.101:0/1872718966 >> v2:172.21.2.108:3300/0 conn(0x7ff9581b0470 0x7ff9581b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:41.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.431 7ff956312700 1 -- 172.21.2.101:0/1872718966 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9580f10d0 con 0x7ff9580ef610 2021-02-10T11:24:41.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.431 7ff956312700 1 --2- 172.21.2.101:0/1872718966 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9580ef610 0x7ff9580efa10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7ff940006700 tx=0x7ff940009b80).ready entity=mon.0 client_cookie=68c88b26e92ee200 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:41.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.431 7ff955b11700 1 -- 172.21.2.101:0/1872718966 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff94000c0c0 con 0x7ff9580ef610 2021-02-10T11:24:41.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.431 7ff955b11700 1 -- 172.21.2.101:0/1872718966 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94000c340 con 0x7ff9580ef610 2021-02-10T11:24:41.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.431 7ff955b11700 1 -- 172.21.2.101:0/1872718966 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff94000c860 con 0x7ff9580ef610 2021-02-10T11:24:41.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.431 7ff95cdec700 1 -- 172.21.2.101:0/1872718966 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9580ef610 msgr2=0x7ff9580efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:41.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.431 7ff95cdec700 1 --2- 172.21.2.101:0/1872718966 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9580ef610 0x7ff9580efa10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7ff940006700 tx=0x7ff940009b80).stop 2021-02-10T11:24:41.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.432 7ff95cdec700 1 -- 172.21.2.101:0/1872718966 shutdown_connections 2021-02-10T11:24:41.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.432 7ff95cdec700 1 --2- 172.21.2.101:0/1872718966 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9580f0030 0x7ff9580f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:41.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.432 7ff95cdec700 1 --2- 172.21.2.101:0/1872718966 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9580ef610 0x7ff9580efa10 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:41.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.432 7ff95cdec700 1 --2- 172.21.2.101:0/1872718966 >> v2:172.21.2.108:3300/0 conn(0x7ff9581b0470 0x7ff9581b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:41.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.432 7ff95cdec700 1 -- 172.21.2.101:0/1872718966 >> 172.21.2.101:0/1872718966 conn(0x7ff9580ce3e0 msgr2=0x7ff9580e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:41.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.432 7ff95cdec700 1 -- 172.21.2.101:0/1872718966 shutdown_connections 2021-02-10T11:24:41.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.432 7ff95cdec700 1 -- 172.21.2.101:0/1872718966 wait complete. 2021-02-10T11:24:41.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.446 7ff95cdec700 1 Processor -- start 2021-02-10T11:24:41.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.447 7ff95cdec700 1 -- start start 2021-02-10T11:24:41.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.447 7ff95cdec700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff958190920 0x7ff95808f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:41.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.447 7ff95cdec700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff958190d20 0x7ff95808fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:41.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.447 7ff95cdec700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff958193270 0x7ff958193670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:41.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.447 7ff95cdec700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff958090470 con 0x7ff958193270 2021-02-10T11:24:41.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.447 7ff95cdec700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff958193c90 con 0x7ff958190d20 2021-02-10T11:24:41.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.447 7ff95cdec700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff958193e70 con 0x7ff958190920 2021-02-10T11:24:41.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.447 7ff957314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff958193270 0x7ff958193670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:41.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.447 7ff957314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff958193270 0x7ff958193670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47420/0 (socket says 172.21.2.101:47420) 2021-02-10T11:24:41.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.447 7ff957314700 1 -- 172.21.2.101:0/2424780168 learned_addr learned my addr 172.21.2.101:0/2424780168 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:41.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.447 7ff957314700 1 -- 172.21.2.101:0/2424780168 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff958190920 msgr2=0x7ff95808f600 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:24:41.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.447 7ff957314700 1 --2- 172.21.2.101:0/2424780168 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff958190920 0x7ff95808f600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:41.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.447 7ff957314700 1 -- 172.21.2.101:0/2424780168 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff958190d20 msgr2=0x7ff95808fc20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:24:41.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.447 7ff957314700 1 --2- 172.21.2.101:0/2424780168 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff958190d20 0x7ff95808fc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:41.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.447 7ff957314700 1 -- 172.21.2.101:0/2424780168 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff958194090 con 0x7ff958193270 2021-02-10T11:24:41.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.448 7ff957314700 1 --2- 172.21.2.101:0/2424780168 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff958193270 0x7ff958193670 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7ff950009960 tx=0x7ff95000bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:41.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.448 7ff94f7fe700 1 -- 172.21.2.101:0/2424780168 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff950003030 con 0x7ff958193270 2021-02-10T11:24:41.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.448 7ff94f7fe700 1 -- 172.21.2.101:0/2424780168 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff950001310 con 0x7ff958193270 2021-02-10T11:24:41.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.448 7ff94f7fe700 1 -- 172.21.2.101:0/2424780168 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff950001830 con 0x7ff958193270 2021-02-10T11:24:41.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.448 7ff95cdec700 1 -- 172.21.2.101:0/2424780168 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff958194090 con 0x7ff958193270 2021-02-10T11:24:41.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.448 7ff95cdec700 1 -- 172.21.2.101:0/2424780168 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9581920b0 con 0x7ff958193270 2021-02-10T11:24:41.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.449 7ff94f7fe700 1 -- 172.21.2.101:0/2424780168 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff9500014f0 con 0x7ff958193270 2021-02-10T11:24:41.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.449 7ff94f7fe700 1 --2- 172.21.2.101:0/2424780168 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff93c0186c0 0x7ff93c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:41.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.449 7ff94f7fe700 1 -- 172.21.2.101:0/2424780168 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff950032910 con 0x7ff958193270 2021-02-10T11:24:41.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.449 7ff956b13700 1 --2- 172.21.2.101:0/2424780168 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff93c0186c0 0x7ff93c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:41.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.449 7ff956b13700 1 --2- 172.21.2.101:0/2424780168 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff93c0186c0 0x7ff93c01abc0 crc :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:41.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.480 7ff95cdec700 1 -- 172.21.2.101:0/2424780168 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff958105af0 con 0x7ff958193270 2021-02-10T11:24:41.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.483 7ff94f7fe700 1 -- 172.21.2.101:0/2424780168 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff950028f10 con 0x7ff958193270 2021-02-10T11:24:41.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.608 7ff95cdec700 1 -- 172.21.2.101:0/2424780168 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839302", "target": ["mgr", ""]}) v1 -- 0x7ff9581920b0 con 0x7ff93c0186c0 2021-02-10T11:24:41.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.633 7ff94f7fe700 1 -- 172.21.2.101:0/2424780168 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7ff9480009b0 con 0x7ff93c0186c0 2021-02-10T11:24:41.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.641 7ff95cdec700 1 -- 172.21.2.101:0/2424780168 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff93c0186c0 msgr2=0x7ff93c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:41.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.641 7ff95cdec700 1 --2- 172.21.2.101:0/2424780168 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff93c0186c0 0x7ff93c01abc0 crc :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:41.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.641 7ff95cdec700 1 -- 172.21.2.101:0/2424780168 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff958193270 msgr2=0x7ff958193670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:41.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.641 7ff95cdec700 1 --2- 172.21.2.101:0/2424780168 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff958193270 0x7ff958193670 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7ff950009960 tx=0x7ff95000bcf0).stop 2021-02-10T11:24:41.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.641 7ff95cdec700 1 -- 172.21.2.101:0/2424780168 shutdown_connections 2021-02-10T11:24:41.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.641 7ff95cdec700 1 --2- 172.21.2.101:0/2424780168 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff958193270 0x7ff958193670 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:41.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.641 7ff95cdec700 1 --2- 172.21.2.101:0/2424780168 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff958190d20 0x7ff95808fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:41.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.641 7ff95cdec700 1 --2- 172.21.2.101:0/2424780168 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff93c0186c0 0x7ff93c01abc0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:41.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.641 7ff95cdec700 1 --2- 172.21.2.101:0/2424780168 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff958190920 0x7ff95808f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:41.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.641 7ff95cdec700 1 -- 172.21.2.101:0/2424780168 >> 172.21.2.101:0/2424780168 conn(0x7ff9580bf4e0 msgr2=0x7ff9580be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:41.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.641 7ff95cdec700 1 -- 172.21.2.101:0/2424780168 shutdown_connections 2021-02-10T11:24:41.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:41.641 7ff95cdec700 1 -- 172.21.2.101:0/2424780168 wait complete. 2021-02-10T11:24:41.658 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839302 2021-02-10T11:24:42.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.064 7fef0a7f1700 1 Processor -- start 2021-02-10T11:24:42.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.064 7fef0a7f1700 1 -- start start 2021-02-10T11:24:42.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.065 7fef0a7f1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fef041b0470 0x7fef041b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:42.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.065 7fef0a7f1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fef040f20e0 0x7fef040f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:42.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.065 7fef0a7f1700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef040f0b50 con 0x7fef040ef610 2021-02-10T11:24:42.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.065 7fef0a7f1700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef040f0820 con 0x7fef041b0470 2021-02-10T11:24:42.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.065 7fef0a7f1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef040f24e0 con 0x7fef040f20e0 2021-02-10T11:24:42.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.065 7fef03fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fef041b0470 0x7fef041b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:42.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.065 7fef037fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fef040ef610 0x7fef040efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47922/0 (socket says 172.21.2.101:47922) 2021-02-10T11:24:42.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.065 7fef037fe700 1 -- 172.21.2.101:0/1465090971 learned_addr learned my addr 172.21.2.101:0/1465090971 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:24:42.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.065 7fef08e0e700 1 --2- 172.21.2.101:0/1465090971 >> v2:172.21.2.108:3300/0 conn(0x7fef040f20e0 0x7fef040f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:42.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.066 7fef03fff700 1 -- 172.21.2.101:0/1465090971 >> v2:172.21.2.108:3300/0 conn(0x7fef040f20e0 msgr2=0x7fef040f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:24:42.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.066 7fef03fff700 1 --2- 172.21.2.101:0/1465090971 >> v2:172.21.2.108:3300/0 conn(0x7fef040f20e0 0x7fef040f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:42.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.066 7fef03fff700 1 -- 172.21.2.101:0/1465090971 >> v1:172.21.2.101:6789/0 conn(0x7fef040ef610 legacy=0x7fef040efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:42.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.066 7fef03fff700 1 -- 172.21.2.101:0/1465090971 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef040e52e0 con 0x7fef041b0470 2021-02-10T11:24:42.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.066 7fef08e0e700 1 --2- 172.21.2.101:0/1465090971 >> v2:172.21.2.108:3300/0 conn(0x7fef040f20e0 0x7fef040f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:24:42.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.066 7fef03fff700 1 --2- 172.21.2.101:0/1465090971 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef041b0470 0x7fef041b0870 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7feef4006650 tx=0x7feef4009a80).ready entity=mon.0 client_cookie=f43a20a7d7873537 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:42.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.066 7fef02ffd700 1 -- 172.21.2.101:0/1465090971 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feef4006390 con 0x7fef041b0470 2021-02-10T11:24:42.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.066 7fef02ffd700 1 -- 172.21.2.101:0/1465090971 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef40009b0 con 0x7fef041b0470 2021-02-10T11:24:42.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.066 7fef02ffd700 1 -- 172.21.2.101:0/1465090971 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feef400c2b0 con 0x7fef041b0470 2021-02-10T11:24:42.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.066 7fef0a7f1700 1 -- 172.21.2.101:0/1465090971 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef041b0470 msgr2=0x7fef041b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:42.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.066 7fef0a7f1700 1 --2- 172.21.2.101:0/1465090971 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef041b0470 0x7fef041b0870 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7feef4006650 tx=0x7feef4009a80).stop 2021-02-10T11:24:42.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.067 7fef0a7f1700 1 -- 172.21.2.101:0/1465090971 shutdown_connections 2021-02-10T11:24:42.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.067 7fef0a7f1700 1 --2- 172.21.2.101:0/1465090971 >> v2:172.21.2.108:3300/0 conn(0x7fef040f20e0 0x7fef040f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:42.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.067 7fef0a7f1700 1 --2- 172.21.2.101:0/1465090971 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef041b0470 0x7fef041b0870 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:42.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.067 7fef0a7f1700 1 -- 172.21.2.101:0/1465090971 >> 172.21.2.101:0/1465090971 conn(0x7fef040ce3e0 msgr2=0x7fef040e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:42.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.067 7fef0a7f1700 1 -- 172.21.2.101:0/1465090971 shutdown_connections 2021-02-10T11:24:42.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.067 7fef0a7f1700 1 -- 172.21.2.101:0/1465090971 wait complete. 2021-02-10T11:24:42.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.081 7fef0a7f1700 1 Processor -- start 2021-02-10T11:24:42.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.081 7fef0a7f1700 1 -- start start 2021-02-10T11:24:42.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.082 7fef0a7f1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef040a0420 0x7fef040a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:42.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.082 7fef0a7f1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef04189d00 0x7fef0418a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:42.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.082 7fef0a7f1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef0418a720 0x7fef040b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:42.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.082 7fef0a7f1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef04189800 con 0x7fef040a0420 2021-02-10T11:24:42.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.082 7fef0a7f1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0418ab20 con 0x7fef0418a720 2021-02-10T11:24:42.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.082 7fef0a7f1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef040a1140 con 0x7fef04189d00 2021-02-10T11:24:42.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.082 7fef03fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef040a0420 0x7fef040a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:42.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.082 7fef03fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef040a0420 0x7fef040a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47434/0 (socket says 172.21.2.101:47434) 2021-02-10T11:24:42.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.082 7fef03fff700 1 -- 172.21.2.101:0/2406666692 learned_addr learned my addr 172.21.2.101:0/2406666692 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:42.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.082 7fef08e0e700 1 --2- 172.21.2.101:0/2406666692 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef0418a720 0x7fef040b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:42.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.082 7fef037fe700 1 --2- 172.21.2.101:0/2406666692 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef04189d00 0x7fef0418a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:42.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.083 7fef03fff700 1 -- 172.21.2.101:0/2406666692 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef04189d00 msgr2=0x7fef0418a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:42.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.083 7fef03fff700 1 --2- 172.21.2.101:0/2406666692 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef04189d00 0x7fef0418a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:42.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.083 7fef03fff700 1 -- 172.21.2.101:0/2406666692 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef0418a720 msgr2=0x7fef040b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:42.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.083 7fef03fff700 1 --2- 172.21.2.101:0/2406666692 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef0418a720 0x7fef040b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:42.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.083 7fef03fff700 1 -- 172.21.2.101:0/2406666692 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef040b8200 con 0x7fef040a0420 2021-02-10T11:24:42.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.083 7fef08e0e700 1 --2- 172.21.2.101:0/2406666692 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef0418a720 0x7fef040b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:24:42.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.083 7fef03fff700 1 --2- 172.21.2.101:0/2406666692 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef040a0420 0x7fef040a0820 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7feef4006f70 tx=0x7feef400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:42.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.083 7fef00ff9700 1 -- 172.21.2.101:0/2406666692 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feef400bd50 con 0x7fef040a0420 2021-02-10T11:24:42.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.083 7fef00ff9700 1 -- 172.21.2.101:0/2406666692 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef4006030 con 0x7fef040a0420 2021-02-10T11:24:42.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.083 7fef00ff9700 1 -- 172.21.2.101:0/2406666692 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feef4005560 con 0x7fef040a0420 2021-02-10T11:24:42.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.083 7fef0a7f1700 1 -- 172.21.2.101:0/2406666692 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef040b8200 con 0x7fef040a0420 2021-02-10T11:24:42.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.083 7fef0a7f1700 1 -- 172.21.2.101:0/2406666692 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef040b8790 con 0x7fef040a0420 2021-02-10T11:24:42.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.084 7fef00ff9700 1 -- 172.21.2.101:0/2406666692 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7feef40052e0 con 0x7fef040a0420 2021-02-10T11:24:42.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.084 7fef00ff9700 1 --2- 172.21.2.101:0/2406666692 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feee40186c0 0x7feee401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:42.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.084 7fef00ff9700 1 -- 172.21.2.101:0/2406666692 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7feef4032580 con 0x7fef040a0420 2021-02-10T11:24:42.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.084 7fef037fe700 1 --2- 172.21.2.101:0/2406666692 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feee40186c0 0x7feee401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:42.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.084 7fef037fe700 1 --2- 172.21.2.101:0/2406666692 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feee40186c0 0x7feee401abc0 crc :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:42.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.115 7fef0a7f1700 1 -- 172.21.2.101:0/2406666692 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef04105ae0 con 0x7fef040a0420 2021-02-10T11:24:42.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.117 7fef00ff9700 1 -- 172.21.2.101:0/2406666692 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7feef40009b0 con 0x7fef040a0420 2021-02-10T11:24:42.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.244 7fef0a7f1700 1 -- 172.21.2.101:0/2406666692 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839302", "target": ["mgr", ""]}) v1 -- 0x7fef040b9020 con 0x7feee40186c0 2021-02-10T11:24:42.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.247 7fef00ff9700 1 -- 172.21.2.101:0/2406666692 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7feef8001a00 con 0x7feee40186c0 2021-02-10T11:24:42.260 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839302/e6b0e368-5bdf-44e9-b55e-aaa58e574d62 2021-02-10T11:24:42.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.252 7fef0a7f1700 1 -- 172.21.2.101:0/2406666692 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feee40186c0 msgr2=0x7feee401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:42.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.252 7fef0a7f1700 1 --2- 172.21.2.101:0/2406666692 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feee40186c0 0x7feee401abc0 crc :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:42.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.252 7fef0a7f1700 1 -- 172.21.2.101:0/2406666692 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef040a0420 msgr2=0x7fef040a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:42.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.252 7fef0a7f1700 1 --2- 172.21.2.101:0/2406666692 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef040a0420 0x7fef040a0820 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7feef4006f70 tx=0x7feef400b960).stop 2021-02-10T11:24:42.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.253 7fef0a7f1700 1 -- 172.21.2.101:0/2406666692 shutdown_connections 2021-02-10T11:24:42.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.253 7fef0a7f1700 1 --2- 172.21.2.101:0/2406666692 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feee40186c0 0x7feee401abc0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:42.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.253 7fef0a7f1700 1 --2- 172.21.2.101:0/2406666692 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef04189d00 0x7fef0418a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:42.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.253 7fef0a7f1700 1 --2- 172.21.2.101:0/2406666692 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef0418a720 0x7fef040b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:42.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.253 7fef0a7f1700 1 --2- 172.21.2.101:0/2406666692 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef040a0420 0x7fef040a0820 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:42.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.253 7fef0a7f1700 1 -- 172.21.2.101:0/2406666692 >> 172.21.2.101:0/2406666692 conn(0x7fef04094e10 msgr2=0x7fef040a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:42.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.253 7fef0a7f1700 1 -- 172.21.2.101:0/2406666692 shutdown_connections 2021-02-10T11:24:42.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:42.253 7fef0a7f1700 1 -- 172.21.2.101:0/2406666692 wait complete. 2021-02-10T11:24:42.270 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839302 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839302/e6b0e368-5bdf-44e9-b55e-aaa58e574d62 2021-02-10T11:24:42.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:42.270 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:42.270 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:42.606 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:42.606 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:42.606 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839302/e6b0e368-5bdf-44e9-b55e-aaa58e574d62/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:42.987 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:42.987 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:42.987 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245645 s, 42.7 MB/s 2021-02-10T11:24:42.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:42.988 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:42.988 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:43.324 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:43.324 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:43.325 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839302/e6b0e368-5bdf-44e9-b55e-aaa58e574d62/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:43.702 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:43.702 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:43.702 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241533 s, 43.4 MB/s 2021-02-10T11:24:43.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:43.703 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:43.703 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:44.039 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:44.040 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:44.040 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839302/e6b0e368-5bdf-44e9-b55e-aaa58e574d62/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:44.413 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:44.414 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:44.414 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0250672 s, 41.8 MB/s 2021-02-10T11:24:44.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:44.415 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:44.415 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:44.749 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:44.750 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:44.750 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839302/e6b0e368-5bdf-44e9-b55e-aaa58e574d62/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:45.124 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:45.125 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:45.125 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024082 s, 43.5 MB/s 2021-02-10T11:24:45.126 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:45.126 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:45.126 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:45.461 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:45.462 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:45.462 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839302/e6b0e368-5bdf-44e9-b55e-aaa58e574d62/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:45.839 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:45.839 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:45.840 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0259609 s, 40.4 MB/s 2021-02-10T11:24:45.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:45.841 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:45.841 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:46.176 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:46.176 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:46.177 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839302/e6b0e368-5bdf-44e9-b55e-aaa58e574d62/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:46.556 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:46.556 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:46.556 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241212 s, 43.5 MB/s 2021-02-10T11:24:46.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:46.557 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:46.557 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:46.893 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:46.894 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:46.894 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839302/e6b0e368-5bdf-44e9-b55e-aaa58e574d62/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:47.271 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:47.271 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:47.271 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240212 s, 43.7 MB/s 2021-02-10T11:24:47.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:47.272 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:47.272 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:47.607 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:47.608 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:47.608 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839302/e6b0e368-5bdf-44e9-b55e-aaa58e574d62/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:47.982 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:47.982 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:47.982 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233777 s, 44.9 MB/s 2021-02-10T11:24:47.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:47.983 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:47.983 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:48.319 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:48.319 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:48.319 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839302/e6b0e368-5bdf-44e9-b55e-aaa58e574d62/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:48.694 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:48.695 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:48.695 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.025478 s, 41.2 MB/s 2021-02-10T11:24:48.695 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:48.696 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:48.696 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:49.031 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:49.032 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:49.032 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839302/e6b0e368-5bdf-44e9-b55e-aaa58e574d62/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:49.402 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:49.403 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:49.403 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0232695 s, 45.1 MB/s 2021-02-10T11:24:49.404 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839303 2021-02-10T11:24:49.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.812 7f7faadaf700 1 Processor -- start 2021-02-10T11:24:49.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.812 7f7faadaf700 1 -- start start 2021-02-10T11:24:49.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.812 7f7faadaf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fa41b0470 0x7f7fa41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:49.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.812 7f7faadaf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7fa40ef610 0x7f7fa40efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:49.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.813 7f7faadaf700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa40f0b50 con 0x7f7fa40f0030 2021-02-10T11:24:49.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.813 7f7faadaf700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa40e52e0 con 0x7f7fa40ef610 2021-02-10T11:24:49.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.813 7f7faadaf700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa40e5490 con 0x7f7fa41b0470 2021-02-10T11:24:49.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.813 7f7fa3fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7fa40ef610 0x7f7fa40efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:49.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.813 7f7fa3fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7fa40ef610 0x7f7fa40efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47440/0 (socket says 172.21.2.101:47440) 2021-02-10T11:24:49.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.813 7f7fa3fff700 1 -- 172.21.2.101:0/4258878869 learned_addr learned my addr 172.21.2.101:0/4258878869 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:49.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.813 7f7fa93cc700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f7fa40f0030 0x7f7fa40f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47934/0 (socket says 172.21.2.101:47934) 2021-02-10T11:24:49.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.813 7f7fa8bcb700 1 --2- 172.21.2.101:0/4258878869 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fa41b0470 0x7f7fa41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:49.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.813 7f7fa3fff700 1 -- 172.21.2.101:0/4258878869 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fa41b0470 msgr2=0x7f7fa41b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:24:49.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.813 7f7fa3fff700 1 --2- 172.21.2.101:0/4258878869 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fa41b0470 0x7f7fa41b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:49.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.813 7f7fa3fff700 1 -- 172.21.2.101:0/4258878869 >> v1:172.21.2.101:6789/0 conn(0x7f7fa40f0030 legacy=0x7f7fa40f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:49.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.813 7f7fa3fff700 1 -- 172.21.2.101:0/4258878869 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa40f0430 con 0x7f7fa40ef610 2021-02-10T11:24:49.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.813 7f7fa8bcb700 1 --2- 172.21.2.101:0/4258878869 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fa41b0470 0x7f7fa41b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:24:49.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.813 7f7fa3fff700 1 --2- 172.21.2.101:0/4258878869 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fa40ef610 0x7f7fa40efa10 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f7f94006700 tx=0x7f7f94009b80).ready entity=mon.0 client_cookie=7eb076d56bd36469 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:49.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.814 7f7fa37fe700 1 -- 172.21.2.101:0/4258878869 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7f9400c0c0 con 0x7f7fa40ef610 2021-02-10T11:24:49.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.814 7f7fa37fe700 1 -- 172.21.2.101:0/4258878869 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9400c440 con 0x7f7fa40ef610 2021-02-10T11:24:49.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.814 7f7fa37fe700 1 -- 172.21.2.101:0/4258878869 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7f9400c960 con 0x7f7fa40ef610 2021-02-10T11:24:49.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.814 7f7faadaf700 1 -- 172.21.2.101:0/4258878869 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fa40ef610 msgr2=0x7f7fa40efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:49.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.814 7f7faadaf700 1 --2- 172.21.2.101:0/4258878869 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fa40ef610 0x7f7fa40efa10 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f7f94006700 tx=0x7f7f94009b80).stop 2021-02-10T11:24:49.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.814 7f7faadaf700 1 -- 172.21.2.101:0/4258878869 shutdown_connections 2021-02-10T11:24:49.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.814 7f7faadaf700 1 --2- 172.21.2.101:0/4258878869 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fa40ef610 0x7f7fa40efa10 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:49.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.814 7f7faadaf700 1 --2- 172.21.2.101:0/4258878869 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fa41b0470 0x7f7fa41b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:49.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.814 7f7faadaf700 1 -- 172.21.2.101:0/4258878869 >> 172.21.2.101:0/4258878869 conn(0x7f7fa40ce3e0 msgr2=0x7f7fa40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:49.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.814 7f7faadaf700 1 -- 172.21.2.101:0/4258878869 shutdown_connections 2021-02-10T11:24:49.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.814 7f7faadaf700 1 -- 172.21.2.101:0/4258878869 wait complete. 2021-02-10T11:24:49.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.829 7f7faadaf700 1 Processor -- start 2021-02-10T11:24:49.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.829 7f7faadaf700 1 -- start start 2021-02-10T11:24:49.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.829 7f7faadaf700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7fa40a2400 0x7f7fa40a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:49.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.829 7f7faadaf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fa40add60 0x7f7fa40a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:49.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.829 7f7faadaf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fa40a1e20 0x7f7fa408c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:49.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.829 7f7faadaf700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa40a1300 con 0x7f7fa40a1e20 2021-02-10T11:24:49.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.829 7f7faadaf700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa40a2800 con 0x7f7fa40a2400 2021-02-10T11:24:49.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.829 7f7faadaf700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa408cca0 con 0x7f7fa40add60 2021-02-10T11:24:49.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.830 7f7fa93cc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fa40a1e20 0x7f7fa408c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:49.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.830 7f7fa93cc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fa40a1e20 0x7f7fa408c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47442/0 (socket says 172.21.2.101:47442) 2021-02-10T11:24:49.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.830 7f7fa93cc700 1 -- 172.21.2.101:0/4105756873 learned_addr learned my addr 172.21.2.101:0/4105756873 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:49.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.830 7f7fa93cc700 1 -- 172.21.2.101:0/4105756873 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fa40add60 msgr2=0x7f7fa40a1800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:24:49.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.830 7f7fa93cc700 1 --2- 172.21.2.101:0/4105756873 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fa40add60 0x7f7fa40a1800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:49.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.830 7f7fa93cc700 1 -- 172.21.2.101:0/4105756873 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7fa40a2400 msgr2=0x7f7fa40a0a80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:24:49.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.830 7f7fa93cc700 1 --2- 172.21.2.101:0/4105756873 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7fa40a2400 0x7f7fa40a0a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:49.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.830 7f7fa93cc700 1 -- 172.21.2.101:0/4105756873 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa408cec0 con 0x7f7fa40a1e20 2021-02-10T11:24:49.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.830 7f7fa93cc700 1 --2- 172.21.2.101:0/4105756873 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fa40a1e20 0x7f7fa408c680 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f7f9c007f90 tx=0x7f7f9c00ac50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:49.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.830 7f7fa17fa700 1 -- 172.21.2.101:0/4105756873 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7f9c001270 con 0x7f7fa40a1e20 2021-02-10T11:24:49.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.830 7f7fa17fa700 1 -- 172.21.2.101:0/4105756873 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9c0070b0 con 0x7f7fa40a1e20 2021-02-10T11:24:49.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.830 7f7fa17fa700 1 -- 172.21.2.101:0/4105756873 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7f9c007540 con 0x7f7fa40a1e20 2021-02-10T11:24:49.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.830 7f7faadaf700 1 -- 172.21.2.101:0/4105756873 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa408cec0 con 0x7f7fa40a1e20 2021-02-10T11:24:49.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.830 7f7faadaf700 1 -- 172.21.2.101:0/4105756873 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa408cec0 con 0x7f7fa40a1e20 2021-02-10T11:24:49.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.831 7f7fa17fa700 1 -- 172.21.2.101:0/4105756873 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7f9c007290 con 0x7f7fa40a1e20 2021-02-10T11:24:49.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.831 7f7fa17fa700 1 --2- 172.21.2.101:0/4105756873 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7f880186c0 0x7f7f8801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:49.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.831 7f7fa17fa700 1 -- 172.21.2.101:0/4105756873 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7f9c0328a0 con 0x7f7fa40a1e20 2021-02-10T11:24:49.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.831 7f7fa8bcb700 1 --2- 172.21.2.101:0/4105756873 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7f880186c0 0x7f7f8801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:49.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.831 7f7fa8bcb700 1 --2- 172.21.2.101:0/4105756873 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7f880186c0 0x7f7f8801abc0 crc :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:49.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.862 7f7faadaf700 1 -- 172.21.2.101:0/4105756873 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa40a2800 con 0x7f7fa40a1e20 2021-02-10T11:24:49.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.865 7f7fa17fa700 1 -- 172.21.2.101:0/4105756873 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7f9c007d00 con 0x7f7fa40a1e20 2021-02-10T11:24:50.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:49.992 7f7faadaf700 1 -- 172.21.2.101:0/4105756873 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839303", "target": ["mgr", ""]}) v1 -- 0x7f7fa40a2800 con 0x7f7f880186c0 2021-02-10T11:24:50.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.020 7f7fa17fa700 1 -- 172.21.2.101:0/4105756873 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f7f8c0009b0 con 0x7f7f880186c0 2021-02-10T11:24:50.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.024 7f7faadaf700 1 -- 172.21.2.101:0/4105756873 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7f880186c0 msgr2=0x7f7f8801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:50.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.024 7f7faadaf700 1 --2- 172.21.2.101:0/4105756873 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7f880186c0 0x7f7f8801abc0 crc :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:50.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.024 7f7faadaf700 1 -- 172.21.2.101:0/4105756873 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fa40a1e20 msgr2=0x7f7fa408c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:50.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.024 7f7faadaf700 1 --2- 172.21.2.101:0/4105756873 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fa40a1e20 0x7f7fa408c680 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f7f9c007f90 tx=0x7f7f9c00ac50).stop 2021-02-10T11:24:50.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.025 7f7faadaf700 1 -- 172.21.2.101:0/4105756873 shutdown_connections 2021-02-10T11:24:50.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.025 7f7faadaf700 1 --2- 172.21.2.101:0/4105756873 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7f880186c0 0x7f7f8801abc0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:50.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.025 7f7faadaf700 1 --2- 172.21.2.101:0/4105756873 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fa40add60 0x7f7fa40a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:50.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.025 7f7faadaf700 1 --2- 172.21.2.101:0/4105756873 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fa40a1e20 0x7f7fa408c680 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:50.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.025 7f7faadaf700 1 --2- 172.21.2.101:0/4105756873 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7fa40a2400 0x7f7fa40a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:50.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.025 7f7faadaf700 1 -- 172.21.2.101:0/4105756873 >> 172.21.2.101:0/4105756873 conn(0x7f7fa40bbf60 msgr2=0x7f7fa4105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:50.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.025 7f7faadaf700 1 -- 172.21.2.101:0/4105756873 shutdown_connections 2021-02-10T11:24:50.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.025 7f7faadaf700 1 -- 172.21.2.101:0/4105756873 wait complete. 2021-02-10T11:24:50.041 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839303 2021-02-10T11:24:50.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.449 7f5c34ad4700 1 Processor -- start 2021-02-10T11:24:50.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.449 7f5c34ad4700 1 -- start start 2021-02-10T11:24:50.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.450 7f5c34ad4700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f5c301b0470 0x7f5c301b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:50.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.450 7f5c34ad4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5c300f20e0 0x7f5c300f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:50.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.450 7f5c34ad4700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c300f0b50 con 0x7f5c300ef610 2021-02-10T11:24:50.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.450 7f5c34ad4700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c300f0820 con 0x7f5c300f20e0 2021-02-10T11:24:50.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.450 7f5c34ad4700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c300f24e0 con 0x7f5c301b0470 2021-02-10T11:24:50.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.450 7f5c2f05f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5c300f20e0 0x7f5c300f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:50.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.450 7f5c2f05f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5c300f20e0 0x7f5c300f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47454/0 (socket says 172.21.2.101:47454) 2021-02-10T11:24:50.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.450 7f5c2f05f700 1 -- 172.21.2.101:0/3055683462 learned_addr learned my addr 172.21.2.101:0/3055683462 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:50.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.450 7f5c2f05f700 1 -- 172.21.2.101:0/3055683462 >> v2:172.21.2.108:3300/0 conn(0x7f5c301b0470 msgr2=0x7f5c301b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:24:50.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.450 7f5c2f05f700 1 --2- 172.21.2.101:0/3055683462 >> v2:172.21.2.108:3300/0 conn(0x7f5c301b0470 0x7f5c301b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:50.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.450 7f5c2f05f700 1 -- 172.21.2.101:0/3055683462 >> v1:172.21.2.101:6789/0 conn(0x7f5c300ef610 legacy=0x7f5c300efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:50.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.450 7f5c2f05f700 1 -- 172.21.2.101:0/3055683462 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c300e52e0 con 0x7f5c300f20e0 2021-02-10T11:24:50.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.450 7f5c2f05f700 1 --2- 172.21.2.101:0/3055683462 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c300f20e0 0x7f5c300f0200 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f5c28006f00 tx=0x7f5c28009f10).ready entity=mon.0 client_cookie=6617cd13b0d4f9c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:50.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.450 7f5c2d85c700 1 -- 172.21.2.101:0/3055683462 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5c2800b3b0 con 0x7f5c300f20e0 2021-02-10T11:24:50.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.450 7f5c2d85c700 1 -- 172.21.2.101:0/3055683462 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2800b730 con 0x7f5c300f20e0 2021-02-10T11:24:50.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.450 7f5c2d85c700 1 -- 172.21.2.101:0/3055683462 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5c2800bc10 con 0x7f5c300f20e0 2021-02-10T11:24:50.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.451 7f5c34ad4700 1 -- 172.21.2.101:0/3055683462 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c300f20e0 msgr2=0x7f5c300f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:50.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.451 7f5c34ad4700 1 --2- 172.21.2.101:0/3055683462 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c300f20e0 0x7f5c300f0200 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f5c28006f00 tx=0x7f5c28009f10).stop 2021-02-10T11:24:50.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.451 7f5c34ad4700 1 -- 172.21.2.101:0/3055683462 shutdown_connections 2021-02-10T11:24:50.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.451 7f5c34ad4700 1 --2- 172.21.2.101:0/3055683462 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c300f20e0 0x7f5c300f0200 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:50.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.451 7f5c34ad4700 1 --2- 172.21.2.101:0/3055683462 >> v2:172.21.2.108:3300/0 conn(0x7f5c301b0470 0x7f5c301b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:50.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.451 7f5c34ad4700 1 -- 172.21.2.101:0/3055683462 >> 172.21.2.101:0/3055683462 conn(0x7f5c300ce3e0 msgr2=0x7f5c300e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:50.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.451 7f5c34ad4700 1 -- 172.21.2.101:0/3055683462 shutdown_connections 2021-02-10T11:24:50.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.451 7f5c34ad4700 1 -- 172.21.2.101:0/3055683462 wait complete. 2021-02-10T11:24:50.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.466 7f5c34ad4700 1 Processor -- start 2021-02-10T11:24:50.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.466 7f5c34ad4700 1 -- start start 2021-02-10T11:24:50.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.466 7f5c34ad4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5c300b5780 0x7f5c3009ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:50.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.466 7f5c34ad4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c300b5b80 0x7f5c3009f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:50.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.466 7f5c34ad4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c300c2700 0x7f5c300c2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:50.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.466 7f5c34ad4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3009fda0 con 0x7f5c300b5b80 2021-02-10T11:24:50.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.466 7f5c34ad4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3009f9f0 con 0x7f5c300b5780 2021-02-10T11:24:50.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.466 7f5c34ad4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c300a02a0 con 0x7f5c300c2700 2021-02-10T11:24:50.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.466 7f5c2e05d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c300b5b80 0x7f5c3009f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:50.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.466 7f5c2e05d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c300b5b80 0x7f5c3009f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47458/0 (socket says 172.21.2.101:47458) 2021-02-10T11:24:50.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.466 7f5c2e05d700 1 -- 172.21.2.101:0/3738467143 learned_addr learned my addr 172.21.2.101:0/3738467143 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:50.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.467 7f5c2e05d700 1 -- 172.21.2.101:0/3738467143 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c300c2700 msgr2=0x7f5c300c2b00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:24:50.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.467 7f5c2e05d700 1 --2- 172.21.2.101:0/3738467143 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c300c2700 0x7f5c300c2b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:50.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.467 7f5c2e05d700 1 -- 172.21.2.101:0/3738467143 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5c300b5780 msgr2=0x7f5c3009ed70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:24:50.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.467 7f5c2e05d700 1 --2- 172.21.2.101:0/3738467143 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5c300b5780 0x7f5c3009ed70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:50.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.467 7f5c2e05d700 1 -- 172.21.2.101:0/3738467143 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c300c3120 con 0x7f5c300b5b80 2021-02-10T11:24:50.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.467 7f5c2e05d700 1 --2- 172.21.2.101:0/3738467143 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c300b5b80 0x7f5c3009f390 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f5c20006ed0 tx=0x7f5c2000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:50.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.467 7f5c277fe700 1 -- 172.21.2.101:0/3738467143 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5c20006240 con 0x7f5c300b5b80 2021-02-10T11:24:50.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.467 7f5c277fe700 1 -- 172.21.2.101:0/3738467143 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c200009b0 con 0x7f5c300b5b80 2021-02-10T11:24:50.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.467 7f5c34ad4700 1 -- 172.21.2.101:0/3738467143 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c300c3120 con 0x7f5c300b5b80 2021-02-10T11:24:50.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.467 7f5c34ad4700 1 -- 172.21.2.101:0/3738467143 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c300eebd0 con 0x7f5c300b5b80 2021-02-10T11:24:50.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.467 7f5c277fe700 1 -- 172.21.2.101:0/3738467143 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5c20005730 con 0x7f5c300b5b80 2021-02-10T11:24:50.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.467 7f5c277fe700 1 -- 172.21.2.101:0/3738467143 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5c20006030 con 0x7f5c300b5b80 2021-02-10T11:24:50.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.468 7f5c277fe700 1 --2- 172.21.2.101:0/3738467143 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5c14018730 0x7f5c1401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:50.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.468 7f5c277fe700 1 -- 172.21.2.101:0/3738467143 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5c200327b0 con 0x7f5c300b5b80 2021-02-10T11:24:50.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.468 7f5c2e85e700 1 --2- 172.21.2.101:0/3738467143 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5c14018730 0x7f5c1401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:50.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.468 7f5c2e85e700 1 --2- 172.21.2.101:0/3738467143 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5c14018730 0x7f5c1401ac10 crc :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:50.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.500 7f5c34ad4700 1 -- 172.21.2.101:0/3738467143 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c300edcc0 con 0x7f5c300b5b80 2021-02-10T11:24:50.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.502 7f5c277fe700 1 -- 172.21.2.101:0/3738467143 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5c2001f4f0 con 0x7f5c300b5b80 2021-02-10T11:24:50.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.630 7f5c34ad4700 1 -- 172.21.2.101:0/3738467143 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839303", "target": ["mgr", ""]}) v1 -- 0x7f5c300eebd0 con 0x7f5c14018730 2021-02-10T11:24:50.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.633 7f5c277fe700 1 -- 172.21.2.101:0/3738467143 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f5c18006970 con 0x7f5c14018730 2021-02-10T11:24:50.645 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839303/96cc5ab4-c197-4a3f-9507-fb5b8e3c6384 2021-02-10T11:24:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.638 7f5c34ad4700 1 -- 172.21.2.101:0/3738467143 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5c14018730 msgr2=0x7f5c1401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:50.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.638 7f5c34ad4700 1 --2- 172.21.2.101:0/3738467143 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5c14018730 0x7f5c1401ac10 crc :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:50.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.638 7f5c34ad4700 1 -- 172.21.2.101:0/3738467143 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c300b5b80 msgr2=0x7f5c3009f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:50.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.638 7f5c34ad4700 1 --2- 172.21.2.101:0/3738467143 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c300b5b80 0x7f5c3009f390 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f5c20006ed0 tx=0x7f5c2000b960).stop 2021-02-10T11:24:50.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.638 7f5c34ad4700 1 -- 172.21.2.101:0/3738467143 shutdown_connections 2021-02-10T11:24:50.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.638 7f5c34ad4700 1 --2- 172.21.2.101:0/3738467143 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5c14018730 0x7f5c1401ac10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:50.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.638 7f5c34ad4700 1 --2- 172.21.2.101:0/3738467143 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c300c2700 0x7f5c300c2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:50.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.638 7f5c34ad4700 1 --2- 172.21.2.101:0/3738467143 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c300b5b80 0x7f5c3009f390 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:50.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.638 7f5c34ad4700 1 --2- 172.21.2.101:0/3738467143 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5c300b5780 0x7f5c3009ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:24:50.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.638 7f5c34ad4700 1 -- 172.21.2.101:0/3738467143 >> 172.21.2.101:0/3738467143 conn(0x7f5c300bbf60 msgr2=0x7f5c300edef0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:50.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.639 7f5c34ad4700 1 -- 172.21.2.101:0/3738467143 shutdown_connections 2021-02-10T11:24:50.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:50.639 7f5c34ad4700 1 -- 172.21.2.101:0/3738467143 wait complete. 2021-02-10T11:24:50.655 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839303 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839303/96cc5ab4-c197-4a3f-9507-fb5b8e3c6384 2021-02-10T11:24:50.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:50.655 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:50.655 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:50.990 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:50.991 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:50.991 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839303/96cc5ab4-c197-4a3f-9507-fb5b8e3c6384/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:51.372 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:51.373 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:51.373 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248419 s, 42.2 MB/s 2021-02-10T11:24:51.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:51.374 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:51.374 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:51.709 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:51.709 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:51.709 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839303/96cc5ab4-c197-4a3f-9507-fb5b8e3c6384/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:51.837 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:24:51.840 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:24:52.085 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:52.086 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:52.086 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0254291 s, 41.2 MB/s 2021-02-10T11:24:52.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:52.087 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:52.087 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:52.422 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:52.423 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:52.423 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839303/96cc5ab4-c197-4a3f-9507-fb5b8e3c6384/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:52.799 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:52.799 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:52.799 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0250073 s, 41.9 MB/s 2021-02-10T11:24:52.800 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:52.800 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:52.801 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:53.136 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:53.137 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:53.137 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839303/96cc5ab4-c197-4a3f-9507-fb5b8e3c6384/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:53.511 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:53.511 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:53.511 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239712 s, 43.7 MB/s 2021-02-10T11:24:53.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:53.512 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:53.512 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:53.848 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:53.849 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:53.849 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839303/96cc5ab4-c197-4a3f-9507-fb5b8e3c6384/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:54.224 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:54.224 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:54.224 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0263962 s, 39.7 MB/s 2021-02-10T11:24:54.225 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:54.225 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:54.225 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:54.560 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:54.561 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:54.561 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839303/96cc5ab4-c197-4a3f-9507-fb5b8e3c6384/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:54.935 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:54.936 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:54.936 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237107 s, 44.2 MB/s 2021-02-10T11:24:54.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:54.937 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:54.937 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:55.273 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:55.273 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:55.273 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839303/96cc5ab4-c197-4a3f-9507-fb5b8e3c6384/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:55.650 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:55.650 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:55.650 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253351 s, 41.4 MB/s 2021-02-10T11:24:55.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:55.651 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:55.651 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:55.987 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:55.987 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:55.987 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839303/96cc5ab4-c197-4a3f-9507-fb5b8e3c6384/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:56.362 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:56.362 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:56.363 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237178 s, 44.2 MB/s 2021-02-10T11:24:56.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:56.364 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:56.364 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:56.699 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:56.772 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:56.772 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839303/96cc5ab4-c197-4a3f-9507-fb5b8e3c6384/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:57.146 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:57.146 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:57.146 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234587 s, 44.7 MB/s 2021-02-10T11:24:57.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:57.147 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:57.147 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:57.483 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:57.483 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:57.483 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839303/96cc5ab4-c197-4a3f-9507-fb5b8e3c6384/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:57.860 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:57.860 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:57.860 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247184 s, 42.4 MB/s 2021-02-10T11:24:57.861 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839304 2021-02-10T11:24:58.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.269 7f7b9d5dd700 1 Processor -- start 2021-02-10T11:24:58.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.269 7f7b9d5dd700 1 -- start start 2021-02-10T11:24:58.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.270 7f7b9d5dd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7b981b0470 0x7f7b981b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:58.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.270 7f7b9d5dd700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7b980ef610 0x7f7b980efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:58.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.270 7f7b9d5dd700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b980f0b50 con 0x7f7b980f0030 2021-02-10T11:24:58.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.270 7f7b9d5dd700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b980e52e0 con 0x7f7b980ef610 2021-02-10T11:24:58.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.270 7f7b9d5dd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980e5490 con 0x7f7b981b0470 2021-02-10T11:24:58.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.271 7f7b972bf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7b981b0470 0x7f7b981b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:58.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.271 7f7b972bf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7b981b0470 0x7f7b981b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:40196/0 (socket says 172.21.2.101:40196) 2021-02-10T11:24:58.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.271 7f7b972bf700 1 -- 172.21.2.101:0/973224961 learned_addr learned my addr 172.21.2.101:0/973224961 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:58.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.271 7f7b96abe700 1 --2- 172.21.2.101:0/973224961 >> v2:172.21.2.108:3300/0 conn(0x7f7b980ef610 0x7f7b980efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:58.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.271 7f7b972bf700 1 -- 172.21.2.101:0/973224961 >> v2:172.21.2.108:3300/0 conn(0x7f7b980ef610 msgr2=0x7f7b980efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:24:58.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.271 7f7b972bf700 1 --2- 172.21.2.101:0/973224961 >> v2:172.21.2.108:3300/0 conn(0x7f7b980ef610 0x7f7b980efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:58.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.271 7f7b972bf700 1 -- 172.21.2.101:0/973224961 >> v1:172.21.2.108:6789/0 conn(0x7f7b980f0030 legacy=0x7f7b980f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:58.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.271 7f7b972bf700 1 -- 172.21.2.101:0/973224961 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b980f0430 con 0x7f7b981b0470 2021-02-10T11:24:58.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.271 7f7b972bf700 1 --2- 172.21.2.101:0/973224961 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7b981b0470 0x7f7b981b0870 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f7b80006700 tx=0x7f7b80009b80).ready entity=mon.2 client_cookie=582aea9bf2770621 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:58.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.272 7f7b962bd700 1 -- 172.21.2.101:0/973224961 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7b8000c0c0 con 0x7f7b981b0470 2021-02-10T11:24:58.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.272 7f7b962bd700 1 -- 172.21.2.101:0/973224961 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8000c340 con 0x7f7b981b0470 2021-02-10T11:24:58.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.272 7f7b962bd700 1 -- 172.21.2.101:0/973224961 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7b8000c860 con 0x7f7b981b0470 2021-02-10T11:24:58.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.272 7f7b9d5dd700 1 -- 172.21.2.101:0/973224961 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7b981b0470 msgr2=0x7f7b981b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:58.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.272 7f7b9d5dd700 1 --2- 172.21.2.101:0/973224961 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7b981b0470 0x7f7b981b0870 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f7b80006700 tx=0x7f7b80009b80).stop 2021-02-10T11:24:58.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.272 7f7b9d5dd700 1 -- 172.21.2.101:0/973224961 shutdown_connections 2021-02-10T11:24:58.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.272 7f7b9d5dd700 1 --2- 172.21.2.101:0/973224961 >> v2:172.21.2.108:3300/0 conn(0x7f7b980ef610 0x7f7b980efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:58.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.272 7f7b9d5dd700 1 --2- 172.21.2.101:0/973224961 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7b981b0470 0x7f7b981b0870 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:58.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.272 7f7b9d5dd700 1 -- 172.21.2.101:0/973224961 >> 172.21.2.101:0/973224961 conn(0x7f7b980ce3e0 msgr2=0x7f7b980e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:58.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.272 7f7b9d5dd700 1 -- 172.21.2.101:0/973224961 shutdown_connections 2021-02-10T11:24:58.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.272 7f7b9d5dd700 1 -- 172.21.2.101:0/973224961 wait complete. 2021-02-10T11:24:58.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.287 7f7b9d5dd700 1 Processor -- start 2021-02-10T11:24:58.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.287 7f7b9d5dd700 1 -- start start 2021-02-10T11:24:58.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.287 7f7b9d5dd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b980a0420 0x7f7b980a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:58.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.287 7f7b9d5dd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7b98189d00 0x7f7b9818a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:58.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.287 7f7b9d5dd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7b9818a720 0x7f7b980b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:58.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.287 7f7b9d5dd700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b98189780 con 0x7f7b980a0420 2021-02-10T11:24:58.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.287 7f7b9d5dd700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9818ab20 con 0x7f7b98189d00 2021-02-10T11:24:58.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.287 7f7b9d5dd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980a1140 con 0x7f7b9818a720 2021-02-10T11:24:58.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.287 7f7b972bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b980a0420 0x7f7b980a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:58.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.287 7f7b972bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b980a0420 0x7f7b980a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47466/0 (socket says 172.21.2.101:47466) 2021-02-10T11:24:58.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.287 7f7b972bf700 1 -- 172.21.2.101:0/1483957844 learned_addr learned my addr 172.21.2.101:0/1483957844 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:58.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.288 7f7b97ac0700 1 --2- 172.21.2.101:0/1483957844 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7b9818a720 0x7f7b980b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:58.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.288 7f7b96abe700 1 --2- 172.21.2.101:0/1483957844 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7b98189d00 0x7f7b9818a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:58.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.288 7f7b972bf700 1 -- 172.21.2.101:0/1483957844 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7b9818a720 msgr2=0x7f7b980b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:58.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.288 7f7b972bf700 1 --2- 172.21.2.101:0/1483957844 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7b9818a720 0x7f7b980b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:58.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.288 7f7b972bf700 1 -- 172.21.2.101:0/1483957844 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7b98189d00 msgr2=0x7f7b9818a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:58.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.288 7f7b972bf700 1 --2- 172.21.2.101:0/1483957844 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7b98189d00 0x7f7b9818a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:58.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.288 7f7b972bf700 1 -- 172.21.2.101:0/1483957844 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b980b8200 con 0x7f7b980a0420 2021-02-10T11:24:58.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.288 7f7b972bf700 1 --2- 172.21.2.101:0/1483957844 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b980a0420 0x7f7b980a0820 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f7b8000b650 tx=0x7f7b8000b9f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:58.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.288 7f7b8ffff700 1 -- 172.21.2.101:0/1483957844 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7b8000bde0 con 0x7f7b980a0420 2021-02-10T11:24:58.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.288 7f7b8ffff700 1 -- 172.21.2.101:0/1483957844 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b80006460 con 0x7f7b980a0420 2021-02-10T11:24:58.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.288 7f7b8ffff700 1 -- 172.21.2.101:0/1483957844 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7b80002220 con 0x7f7b980a0420 2021-02-10T11:24:58.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.289 7f7b9d5dd700 1 -- 172.21.2.101:0/1483957844 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b980b8200 con 0x7f7b980a0420 2021-02-10T11:24:58.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.289 7f7b9d5dd700 1 -- 172.21.2.101:0/1483957844 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b980b8790 con 0x7f7b980a0420 2021-02-10T11:24:58.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.289 7f7b8ffff700 1 -- 172.21.2.101:0/1483957844 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7b800052e0 con 0x7f7b980a0420 2021-02-10T11:24:58.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.289 7f7b8ffff700 1 --2- 172.21.2.101:0/1483957844 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7b7c0186c0 0x7f7b7c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:58.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.289 7f7b8ffff700 1 -- 172.21.2.101:0/1483957844 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7b800325a0 con 0x7f7b980a0420 2021-02-10T11:24:58.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.289 7f7b96abe700 1 --2- 172.21.2.101:0/1483957844 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7b7c0186c0 0x7f7b7c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:58.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.290 7f7b96abe700 1 --2- 172.21.2.101:0/1483957844 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7b7c0186c0 0x7f7b7c01abc0 crc :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:58.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.320 7f7b9d5dd700 1 -- 172.21.2.101:0/1483957844 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b98105ae0 con 0x7f7b980a0420 2021-02-10T11:24:58.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.322 7f7b8ffff700 1 -- 172.21.2.101:0/1483957844 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7b8002d270 con 0x7f7b980a0420 2021-02-10T11:24:58.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.449 7f7b9d5dd700 1 -- 172.21.2.101:0/1483957844 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839304", "target": ["mgr", ""]}) v1 -- 0x7f7b980b8790 con 0x7f7b7c0186c0 2021-02-10T11:24:58.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.475 7f7b8ffff700 1 -- 172.21.2.101:0/1483957844 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f7b880009b0 con 0x7f7b7c0186c0 2021-02-10T11:24:58.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.481 7f7b9d5dd700 1 -- 172.21.2.101:0/1483957844 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7b7c0186c0 msgr2=0x7f7b7c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:58.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.481 7f7b9d5dd700 1 --2- 172.21.2.101:0/1483957844 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7b7c0186c0 0x7f7b7c01abc0 crc :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:58.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.481 7f7b9d5dd700 1 -- 172.21.2.101:0/1483957844 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b980a0420 msgr2=0x7f7b980a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:58.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.481 7f7b9d5dd700 1 --2- 172.21.2.101:0/1483957844 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b980a0420 0x7f7b980a0820 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f7b8000b650 tx=0x7f7b8000b9f0).stop 2021-02-10T11:24:58.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.481 7f7b9d5dd700 1 -- 172.21.2.101:0/1483957844 shutdown_connections 2021-02-10T11:24:58.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.481 7f7b9d5dd700 1 --2- 172.21.2.101:0/1483957844 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7b7c0186c0 0x7f7b7c01abc0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:58.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.481 7f7b9d5dd700 1 --2- 172.21.2.101:0/1483957844 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7b9818a720 0x7f7b980b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:58.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.481 7f7b9d5dd700 1 --2- 172.21.2.101:0/1483957844 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7b98189d00 0x7f7b9818a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:58.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.481 7f7b9d5dd700 1 --2- 172.21.2.101:0/1483957844 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b980a0420 0x7f7b980a0820 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:58.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.481 7f7b9d5dd700 1 -- 172.21.2.101:0/1483957844 >> 172.21.2.101:0/1483957844 conn(0x7f7b98094e10 msgr2=0x7f7b980a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:58.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.482 7f7b9d5dd700 1 -- 172.21.2.101:0/1483957844 shutdown_connections 2021-02-10T11:24:58.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.482 7f7b9d5dd700 1 -- 172.21.2.101:0/1483957844 wait complete. 2021-02-10T11:24:58.498 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839304 2021-02-10T11:24:58.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.904 7f778266f700 1 Processor -- start 2021-02-10T11:24:58.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.904 7f778266f700 1 -- start start 2021-02-10T11:24:58.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.905 7f778266f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f777c1b0870 0x7f777c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:58.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.905 7f778266f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f777c0efe00 0x7f777c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:58.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.905 7f778266f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f777c1b0c70 con 0x7f777c1b0470 2021-02-10T11:24:58.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.905 7f778266f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f777c0e5ea0 con 0x7f777c0efe00 2021-02-10T11:24:58.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.905 7f778266f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f777c0f0b50 con 0x7f777c1b0870 2021-02-10T11:24:58.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.905 7f7780c8c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f777c0efe00 0x7f777c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:58.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.905 7f7780c8c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f777c0efe00 0x7f777c0f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47478/0 (socket says 172.21.2.101:47478) 2021-02-10T11:24:58.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.905 7f7780c8c700 1 -- 172.21.2.101:0/3498339024 learned_addr learned my addr 172.21.2.101:0/3498339024 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:58.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.905 7f777b7fe700 1 --2- 172.21.2.101:0/3498339024 >> v2:172.21.2.108:3300/0 conn(0x7f777c1b0870 0x7f777c0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:58.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.905 7f7780c8c700 1 -- 172.21.2.101:0/3498339024 >> v2:172.21.2.108:3300/0 conn(0x7f777c1b0870 msgr2=0x7f777c0f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:24:58.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.906 7f7780c8c700 1 --2- 172.21.2.101:0/3498339024 >> v2:172.21.2.108:3300/0 conn(0x7f777c1b0870 0x7f777c0f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:58.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.906 7f7780c8c700 1 -- 172.21.2.101:0/3498339024 >> v1:172.21.2.108:6789/0 conn(0x7f777c1b0470 legacy=0x7f777c0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:58.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.906 7f7780c8c700 1 -- 172.21.2.101:0/3498339024 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777c0f0820 con 0x7f777c0efe00 2021-02-10T11:24:58.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.906 7f7780c8c700 1 --2- 172.21.2.101:0/3498339024 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f777c0efe00 0x7f777c0f0200 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f7774006f00 tx=0x7f7774009f10).ready entity=mon.0 client_cookie=ce6d72a3ad5031fe server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:58.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.906 7f777affd700 1 -- 172.21.2.101:0/3498339024 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f777400b3b0 con 0x7f777c0efe00 2021-02-10T11:24:58.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.906 7f777affd700 1 -- 172.21.2.101:0/3498339024 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777400b630 con 0x7f777c0efe00 2021-02-10T11:24:58.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.906 7f777affd700 1 -- 172.21.2.101:0/3498339024 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f777400bb50 con 0x7f777c0efe00 2021-02-10T11:24:58.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.906 7f778266f700 1 -- 172.21.2.101:0/3498339024 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f777c0efe00 msgr2=0x7f777c0f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:58.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.906 7f778266f700 1 --2- 172.21.2.101:0/3498339024 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f777c0efe00 0x7f777c0f0200 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f7774006f00 tx=0x7f7774009f10).stop 2021-02-10T11:24:58.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.906 7f778266f700 1 -- 172.21.2.101:0/3498339024 shutdown_connections 2021-02-10T11:24:58.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.906 7f778266f700 1 --2- 172.21.2.101:0/3498339024 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f777c0efe00 0x7f777c0f0200 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:58.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.906 7f778266f700 1 --2- 172.21.2.101:0/3498339024 >> v2:172.21.2.108:3300/0 conn(0x7f777c1b0870 0x7f777c0f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:58.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.906 7f778266f700 1 -- 172.21.2.101:0/3498339024 >> 172.21.2.101:0/3498339024 conn(0x7f777c0ce3e0 msgr2=0x7f777c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:58.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.906 7f778266f700 1 -- 172.21.2.101:0/3498339024 shutdown_connections 2021-02-10T11:24:58.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.906 7f778266f700 1 -- 172.21.2.101:0/3498339024 wait complete. 2021-02-10T11:24:58.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.921 7f778266f700 1 Processor -- start 2021-02-10T11:24:58.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.921 7f778266f700 1 -- start start 2021-02-10T11:24:58.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.921 7f778266f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f777c190920 0x7f777c08f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:58.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.921 7f778266f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f777c190d20 0x7f777c08fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:58.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.921 7f778266f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f777c193270 0x7f777c193670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:58.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.921 7f778266f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c090470 con 0x7f777c190920 2021-02-10T11:24:58.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.921 7f778266f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c193c90 con 0x7f777c193270 2021-02-10T11:24:58.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.921 7f778266f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777c193e70 con 0x7f777c190d20 2021-02-10T11:24:58.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.922 7f777bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f777c190920 0x7f777c08f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:58.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.922 7f777bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f777c190920 0x7f777c08f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47480/0 (socket says 172.21.2.101:47480) 2021-02-10T11:24:58.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.922 7f777bfff700 1 -- 172.21.2.101:0/92123285 learned_addr learned my addr 172.21.2.101:0/92123285 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:24:58.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.922 7f777b7fe700 1 --2- 172.21.2.101:0/92123285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f777c190d20 0x7f777c08fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:58.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.922 7f7780c8c700 1 --2- 172.21.2.101:0/92123285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f777c193270 0x7f777c193670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:58.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.922 7f777bfff700 1 -- 172.21.2.101:0/92123285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f777c190d20 msgr2=0x7f777c08fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:58.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.922 7f777bfff700 1 --2- 172.21.2.101:0/92123285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f777c190d20 0x7f777c08fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:58.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.922 7f777bfff700 1 -- 172.21.2.101:0/92123285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f777c193270 msgr2=0x7f777c193670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:58.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.922 7f777bfff700 1 --2- 172.21.2.101:0/92123285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f777c193270 0x7f777c193670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:58.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.922 7f777bfff700 1 -- 172.21.2.101:0/92123285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777c194090 con 0x7f777c190920 2021-02-10T11:24:58.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.922 7f777bfff700 1 --2- 172.21.2.101:0/92123285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f777c190920 0x7f777c08f600 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f776c00b5b0 tx=0x7f776c00b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:58.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.923 7f7778ff9700 1 -- 172.21.2.101:0/92123285 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f776c00bd40 con 0x7f777c190920 2021-02-10T11:24:58.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.923 7f7778ff9700 1 -- 172.21.2.101:0/92123285 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776c006240 con 0x7f777c190920 2021-02-10T11:24:58.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.923 7f7778ff9700 1 -- 172.21.2.101:0/92123285 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f776c005720 con 0x7f777c190920 2021-02-10T11:24:58.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.923 7f778266f700 1 -- 172.21.2.101:0/92123285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f777c194090 con 0x7f777c190920 2021-02-10T11:24:58.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.923 7f778266f700 1 -- 172.21.2.101:0/92123285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f777c1920b0 con 0x7f777c190920 2021-02-10T11:24:58.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.923 7f7778ff9700 1 -- 172.21.2.101:0/92123285 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f776c0054a0 con 0x7f777c190920 2021-02-10T11:24:58.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.923 7f7778ff9700 1 --2- 172.21.2.101:0/92123285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f775c0186c0 0x7f775c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:24:58.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.923 7f7778ff9700 1 -- 172.21.2.101:0/92123285 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f776c032580 con 0x7f777c190920 2021-02-10T11:24:58.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.923 7f777b7fe700 1 --2- 172.21.2.101:0/92123285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f775c0186c0 0x7f775c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:24:58.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.924 7f777b7fe700 1 --2- 172.21.2.101:0/92123285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f775c0186c0 0x7f775c01abc0 crc :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:24:58.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.955 7f778266f700 1 -- 172.21.2.101:0/92123285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f777c105af0 con 0x7f777c190920 2021-02-10T11:24:58.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:58.957 7f7778ff9700 1 -- 172.21.2.101:0/92123285 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f776c000940 con 0x7f777c190920 2021-02-10T11:24:59.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:59.085 7f778266f700 1 -- 172.21.2.101:0/92123285 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839304", "target": ["mgr", ""]}) v1 -- 0x7f777c090470 con 0x7f775c0186c0 2021-02-10T11:24:59.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:59.088 7f7778ff9700 1 -- 172.21.2.101:0/92123285 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f7770000960 con 0x7f775c0186c0 2021-02-10T11:24:59.096 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839304/cdc0afac-edef-47e7-93ef-45395b055dd9 2021-02-10T11:24:59.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:59.089 7f778266f700 1 -- 172.21.2.101:0/92123285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f775c0186c0 msgr2=0x7f775c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:59.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:59.089 7f778266f700 1 --2- 172.21.2.101:0/92123285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f775c0186c0 0x7f775c01abc0 crc :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:59.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:59.089 7f778266f700 1 -- 172.21.2.101:0/92123285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f777c190920 msgr2=0x7f777c08f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:24:59.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:59.089 7f778266f700 1 --2- 172.21.2.101:0/92123285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f777c190920 0x7f777c08f600 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f776c00b5b0 tx=0x7f776c00b950).stop 2021-02-10T11:24:59.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:59.089 7f778266f700 1 -- 172.21.2.101:0/92123285 shutdown_connections 2021-02-10T11:24:59.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:59.089 7f778266f700 1 --2- 172.21.2.101:0/92123285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f775c0186c0 0x7f775c01abc0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:59.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:59.089 7f778266f700 1 --2- 172.21.2.101:0/92123285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f777c190d20 0x7f777c08fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:59.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:59.089 7f778266f700 1 --2- 172.21.2.101:0/92123285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f777c193270 0x7f777c193670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:59.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:59.089 7f778266f700 1 --2- 172.21.2.101:0/92123285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f777c190920 0x7f777c08f600 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:24:59.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:59.089 7f778266f700 1 -- 172.21.2.101:0/92123285 >> 172.21.2.101:0/92123285 conn(0x7f777c0bf4e0 msgr2=0x7f777c0be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:24:59.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:59.089 7f778266f700 1 -- 172.21.2.101:0/92123285 shutdown_connections 2021-02-10T11:24:59.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:24:59.089 7f778266f700 1 -- 172.21.2.101:0/92123285 wait complete. 2021-02-10T11:24:59.105 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839304 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839304/cdc0afac-edef-47e7-93ef-45395b055dd9 2021-02-10T11:24:59.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:59.105 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:59.105 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:24:59.442 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:24:59.442 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:24:59.442 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839304/cdc0afac-edef-47e7-93ef-45395b055dd9/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:24:59.822 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:24:59.822 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:24:59.822 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243278 s, 43.1 MB/s 2021-02-10T11:24:59.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:24:59.823 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:24:59.823 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:00.159 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:00.159 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:00.160 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839304/cdc0afac-edef-47e7-93ef-45395b055dd9/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:00.534 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:00.534 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:00.535 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0223644 s, 46.9 MB/s 2021-02-10T11:25:00.535 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:00.536 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:00.536 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:00.872 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:00.872 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:00.872 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839304/cdc0afac-edef-47e7-93ef-45395b055dd9/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:01.249 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:01.250 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:01.250 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246816 s, 42.5 MB/s 2021-02-10T11:25:01.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:01.252 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:01.252 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:01.589 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:01.589 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:01.589 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839304/cdc0afac-edef-47e7-93ef-45395b055dd9/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:01.964 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:01.964 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:01.964 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243213 s, 43.1 MB/s 2021-02-10T11:25:01.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:01.965 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:01.965 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:02.300 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:02.300 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:02.300 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839304/cdc0afac-edef-47e7-93ef-45395b055dd9/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:02.675 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:02.676 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:02.676 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0254346 s, 41.2 MB/s 2021-02-10T11:25:02.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:02.677 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:02.677 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:03.011 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:03.012 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:03.012 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839304/cdc0afac-edef-47e7-93ef-45395b055dd9/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:03.387 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:03.387 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:03.388 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0232193 s, 45.2 MB/s 2021-02-10T11:25:03.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:03.389 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:03.389 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:03.724 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:03.724 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:03.724 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839304/cdc0afac-edef-47e7-93ef-45395b055dd9/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:04.098 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:04.098 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:04.098 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234036 s, 44.8 MB/s 2021-02-10T11:25:04.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:04.099 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:04.099 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:04.435 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:04.436 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:04.436 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839304/cdc0afac-edef-47e7-93ef-45395b055dd9/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:04.816 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:04.817 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:04.817 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0287897 s, 36.4 MB/s 2021-02-10T11:25:04.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:04.818 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:04.818 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:05.152 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:05.152 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:05.153 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839304/cdc0afac-edef-47e7-93ef-45395b055dd9/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:05.528 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:05.528 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:05.528 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235469 s, 44.5 MB/s 2021-02-10T11:25:05.529 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:05.530 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:05.530 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:05.865 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:05.865 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:05.865 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839304/cdc0afac-edef-47e7-93ef-45395b055dd9/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:06.241 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:06.241 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:06.241 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253662 s, 41.3 MB/s 2021-02-10T11:25:06.242 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839305 2021-02-10T11:25:06.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.648 7fe5b83e0700 1 Processor -- start 2021-02-10T11:25:06.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.649 7fe5b83e0700 1 -- start start 2021-02-10T11:25:06.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.649 7fe5b83e0700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe5b01b0470 0x7fe5b01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:06.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.649 7fe5b83e0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5b00f20e0 0x7fe5b00f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:06.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.649 7fe5b83e0700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5b00f0b50 con 0x7fe5b00ef610 2021-02-10T11:25:06.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.649 7fe5b83e0700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5b00f0820 con 0x7fe5b01b0470 2021-02-10T11:25:06.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.649 7fe5b83e0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b00f24e0 con 0x7fe5b00f20e0 2021-02-10T11:25:06.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.649 7fe5b59fb700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe5b00ef610 0x7fe5b00efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:47982/0 (socket says 172.21.2.101:47982) 2021-02-10T11:25:06.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.649 7fe5b59fb700 1 -- 172.21.2.101:0/2162225947 learned_addr learned my addr 172.21.2.101:0/2162225947 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:25:06.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.650 7fe5b51fa700 1 -- 172.21.2.101:0/2162225947 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3098689192 0 0) 0x7fe5a4000ce0 con 0x7fe5b00ef610 2021-02-10T11:25:06.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.650 7fe5b51fa700 1 -- 172.21.2.101:0/2162225947 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe59c002fe0 con 0x7fe5b00ef610 2021-02-10T11:25:06.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.650 7fe5b69fd700 1 --2- 172.21.2.101:0/2162225947 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5b00f20e0 0x7fe5b00f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:06.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.650 7fe5b61fc700 1 --2- 172.21.2.101:0/2162225947 >> v2:172.21.2.108:3300/0 conn(0x7fe5b01b0470 0x7fe5b01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:06.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.650 7fe5b51fa700 1 -- 172.21.2.101:0/2162225947 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1343739222 0 0) 0x7fe5a4001180 con 0x7fe5b00ef610 2021-02-10T11:25:06.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.650 7fe5b51fa700 1 -- 172.21.2.101:0/2162225947 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5b00f20e0 msgr2=0x7fe5b00f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:25:06.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.650 7fe5b51fa700 1 --2- 172.21.2.101:0/2162225947 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5b00f20e0 0x7fe5b00f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:06.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.650 7fe5b51fa700 1 -- 172.21.2.101:0/2162225947 >> v2:172.21.2.108:3300/0 conn(0x7fe5b01b0470 msgr2=0x7fe5b01b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:25:06.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.650 7fe5b51fa700 1 --2- 172.21.2.101:0/2162225947 >> v2:172.21.2.108:3300/0 conn(0x7fe5b01b0470 0x7fe5b01b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:06.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.650 7fe5b51fa700 1 -- 172.21.2.101:0/2162225947 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b00e52e0 con 0x7fe5b00ef610 2021-02-10T11:25:06.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.650 7fe5b51fa700 1 -- 172.21.2.101:0/2162225947 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fe5a4001680 con 0x7fe5b00ef610 2021-02-10T11:25:06.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.650 7fe5b51fa700 1 -- 172.21.2.101:0/2162225947 >> v1:172.21.2.101:6789/0 conn(0x7fe5b00ef610 legacy=0x7fe5b00efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:06.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.650 7fe5b51fa700 1 --2- 172.21.2.101:0/2162225947 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe59c003af0 0x7fe59c005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:06.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.650 7fe5b51fa700 1 --2- 172.21.2.101:0/2162225947 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe59c006840 0x7fe59c008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:06.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.650 7fe5b51fa700 1 --2- 172.21.2.101:0/2162225947 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe59c009640 0x7fe59c00bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:06.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.650 7fe5b51fa700 1 -- 172.21.2.101:0/2162225947 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c050390 con 0x7fe59c006840 2021-02-10T11:25:06.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.650 7fe5b51fa700 1 -- 172.21.2.101:0/2162225947 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c050570 con 0x7fe59c003af0 2021-02-10T11:25:06.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.650 7fe5b51fa700 1 -- 172.21.2.101:0/2162225947 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c050790 con 0x7fe59c009640 2021-02-10T11:25:06.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.650 7fe5b61fc700 1 --2- 172.21.2.101:0/2162225947 >> v2:172.21.2.108:3300/0 conn(0x7fe5b01b0470 0x7fe5b01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:25:06.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.650 7fe5b59fb700 1 --2- 172.21.2.101:0/2162225947 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe59c006840 0x7fe59c008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:06.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.651 7fe5b69fd700 1 --2- 172.21.2.101:0/2162225947 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe59c009640 0x7fe59c00bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:06.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.651 7fe5b61fc700 1 --2- 172.21.2.101:0/2162225947 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe59c003af0 0x7fe59c005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:06.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.651 7fe5b59fb700 1 -- 172.21.2.101:0/2162225947 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe59c009640 msgr2=0x7fe59c00bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:25:06.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.651 7fe5b59fb700 1 --2- 172.21.2.101:0/2162225947 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe59c009640 0x7fe59c00bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:06.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.651 7fe5b59fb700 1 -- 172.21.2.101:0/2162225947 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe59c003af0 msgr2=0x7fe59c005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:25:06.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.651 7fe5b59fb700 1 --2- 172.21.2.101:0/2162225947 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe59c003af0 0x7fe59c005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:06.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.651 7fe5b59fb700 1 -- 172.21.2.101:0/2162225947 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c050a30 con 0x7fe59c006840 2021-02-10T11:25:06.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.651 7fe5b61fc700 1 --2- 172.21.2.101:0/2162225947 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe59c003af0 0x7fe59c005fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:25:06.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.651 7fe5b59fb700 1 --2- 172.21.2.101:0/2162225947 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe59c006840 0x7fe59c008d40 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7fe5a40014e0 tx=0x7fe5a4007d20).ready entity=mon.0 client_cookie=b395ac01804b4255 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:06.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.651 7fe5b51fa700 1 -- 172.21.2.101:0/2162225947 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5a40070c0 con 0x7fe59c006840 2021-02-10T11:25:06.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.651 7fe5b51fa700 1 -- 172.21.2.101:0/2162225947 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a40043a0 con 0x7fe59c006840 2021-02-10T11:25:06.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.651 7fe5b51fa700 1 -- 172.21.2.101:0/2162225947 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5a4004900 con 0x7fe59c006840 2021-02-10T11:25:06.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.651 7fe5b83e0700 1 -- 172.21.2.101:0/2162225947 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe59c006840 msgr2=0x7fe59c008d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:06.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.651 7fe5b83e0700 1 --2- 172.21.2.101:0/2162225947 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe59c006840 0x7fe59c008d40 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7fe5a40014e0 tx=0x7fe5a4007d20).stop 2021-02-10T11:25:06.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.651 7fe5b69fd700 1 -- 172.21.2.101:0/2162225947 reap_dead start 2021-02-10T11:25:06.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.652 7fe5b83e0700 1 -- 172.21.2.101:0/2162225947 shutdown_connections 2021-02-10T11:25:06.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.652 7fe5b83e0700 1 -- 172.21.2.101:0/2162225947 >> 172.21.2.101:0/2162225947 conn(0x7fe5b00ce3e0 msgr2=0x7fe5b00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:06.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.652 7fe5b83e0700 1 -- 172.21.2.101:0/2162225947 shutdown_connections 2021-02-10T11:25:06.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.652 7fe5b83e0700 1 -- 172.21.2.101:0/2162225947 wait complete. 2021-02-10T11:25:06.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.666 7fe5b83e0700 1 Processor -- start 2021-02-10T11:25:06.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.666 7fe5b83e0700 1 -- start start 2021-02-10T11:25:06.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.667 7fe5b83e0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5b00924b0 0x7fe5b01a3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:06.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.667 7fe5b83e0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5b01a3bb0 0x7fe5b00bf920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:06.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.667 7fe5b83e0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5b00bff40 0x7fe5b00c0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:06.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.667 7fe5b83e0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b009a0a0 con 0x7fe5b00bff40 2021-02-10T11:25:06.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.667 7fe5b83e0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b00c0960 con 0x7fe5b00924b0 2021-02-10T11:25:06.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.667 7fe5b83e0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b0098420 con 0x7fe5b01a3bb0 2021-02-10T11:25:06.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.667 7fe5b69fd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5b00bff40 0x7fe5b00c0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:06.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.667 7fe5b69fd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5b00bff40 0x7fe5b00c0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47502/0 (socket says 172.21.2.101:47502) 2021-02-10T11:25:06.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.667 7fe5b69fd700 1 -- 172.21.2.101:0/4278784330 learned_addr learned my addr 172.21.2.101:0/4278784330 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:25:06.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.667 7fe5b59fb700 1 --2- 172.21.2.101:0/4278784330 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5b01a3bb0 0x7fe5b00bf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:06.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.667 7fe5b61fc700 1 --2- 172.21.2.101:0/4278784330 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5b00924b0 0x7fe5b01a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:06.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.667 7fe5b69fd700 1 -- 172.21.2.101:0/4278784330 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5b01a3bb0 msgr2=0x7fe5b00bf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:06.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.667 7fe5b69fd700 1 --2- 172.21.2.101:0/4278784330 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5b01a3bb0 0x7fe5b00bf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:06.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.667 7fe5b69fd700 1 -- 172.21.2.101:0/4278784330 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5b00924b0 msgr2=0x7fe5b01a3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:06.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.667 7fe5b69fd700 1 --2- 172.21.2.101:0/4278784330 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5b00924b0 0x7fe5b01a3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:06.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.668 7fe5b69fd700 1 -- 172.21.2.101:0/4278784330 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b0098640 con 0x7fe5b00bff40 2021-02-10T11:25:06.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.668 7fe5b59fb700 1 --2- 172.21.2.101:0/4278784330 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5b01a3bb0 0x7fe5b00bf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:25:06.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.668 7fe5b61fc700 1 --2- 172.21.2.101:0/4278784330 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5b00924b0 0x7fe5b01a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:25:06.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.668 7fe5b69fd700 1 --2- 172.21.2.101:0/4278784330 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5b00bff40 0x7fe5b00c0340 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7fe5a80099e0 tx=0x7fe5a800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:06.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.668 7fe5a2ffd700 1 -- 172.21.2.101:0/4278784330 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5a8006030 con 0x7fe5b00bff40 2021-02-10T11:25:06.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.668 7fe5a2ffd700 1 -- 172.21.2.101:0/4278784330 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a80062b0 con 0x7fe5b00bff40 2021-02-10T11:25:06.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.668 7fe5b83e0700 1 -- 172.21.2.101:0/4278784330 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b0098640 con 0x7fe5b00bff40 2021-02-10T11:25:06.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.668 7fe5b83e0700 1 -- 172.21.2.101:0/4278784330 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b0098bd0 con 0x7fe5b00bff40 2021-02-10T11:25:06.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.668 7fe5a2ffd700 1 -- 172.21.2.101:0/4278784330 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5a80056b0 con 0x7fe5b00bff40 2021-02-10T11:25:06.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.669 7fe5a2ffd700 1 -- 172.21.2.101:0/4278784330 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe5a80056b0 con 0x7fe5b00bff40 2021-02-10T11:25:06.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.669 7fe5a2ffd700 1 --2- 172.21.2.101:0/4278784330 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe598018700 0x7fe59801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:06.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.669 7fe5a2ffd700 1 -- 172.21.2.101:0/4278784330 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe5a8032b60 con 0x7fe5b00bff40 2021-02-10T11:25:06.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.669 7fe5b61fc700 1 --2- 172.21.2.101:0/4278784330 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe598018700 0x7fe59801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:06.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.669 7fe5b61fc700 1 --2- 172.21.2.101:0/4278784330 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe598018700 0x7fe59801abc0 crc :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:06.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.699 7fe5b83e0700 1 -- 172.21.2.101:0/4278784330 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b0105b40 con 0x7fe5b00bff40 2021-02-10T11:25:06.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.702 7fe5a2ffd700 1 -- 172.21.2.101:0/4278784330 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe5a8006bc0 con 0x7fe5b00bff40 2021-02-10T11:25:06.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.827 7fe5b83e0700 1 -- 172.21.2.101:0/4278784330 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839305", "target": ["mgr", ""]}) v1 -- 0x7fe5b0098bd0 con 0x7fe598018700 2021-02-10T11:25:06.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.851 7fe5a2ffd700 1 -- 172.21.2.101:0/4278784330 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fe5ac000940 con 0x7fe598018700 2021-02-10T11:25:06.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.860 7fe5b83e0700 1 -- 172.21.2.101:0/4278784330 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe598018700 msgr2=0x7fe59801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:06.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.860 7fe5b83e0700 1 --2- 172.21.2.101:0/4278784330 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe598018700 0x7fe59801abc0 crc :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:06.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.860 7fe5b83e0700 1 -- 172.21.2.101:0/4278784330 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5b00bff40 msgr2=0x7fe5b00c0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:06.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.860 7fe5b83e0700 1 --2- 172.21.2.101:0/4278784330 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5b00bff40 0x7fe5b00c0340 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7fe5a80099e0 tx=0x7fe5a800bcf0).stop 2021-02-10T11:25:06.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.860 7fe5b83e0700 1 -- 172.21.2.101:0/4278784330 shutdown_connections 2021-02-10T11:25:06.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.860 7fe5b83e0700 1 --2- 172.21.2.101:0/4278784330 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe598018700 0x7fe59801abc0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:06.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.860 7fe5b83e0700 1 --2- 172.21.2.101:0/4278784330 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5b01a3bb0 0x7fe5b00bf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:06.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.860 7fe5b83e0700 1 --2- 172.21.2.101:0/4278784330 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5b00bff40 0x7fe5b00c0340 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:06.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.860 7fe5b83e0700 1 --2- 172.21.2.101:0/4278784330 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5b00924b0 0x7fe5b01a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:06.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.860 7fe5b83e0700 1 -- 172.21.2.101:0/4278784330 >> 172.21.2.101:0/4278784330 conn(0x7fe5b00b50c0 msgr2=0x7fe5b0092d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:06.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.860 7fe5b83e0700 1 -- 172.21.2.101:0/4278784330 shutdown_connections 2021-02-10T11:25:06.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:06.860 7fe5b83e0700 1 -- 172.21.2.101:0/4278784330 wait complete. 2021-02-10T11:25:06.876 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839305 2021-02-10T11:25:07.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.282 7ff6ac628700 1 Processor -- start 2021-02-10T11:25:07.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.282 7ff6ac628700 1 -- start start 2021-02-10T11:25:07.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.283 7ff6ac628700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff6a41b0470 0x7ff6a41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:07.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.283 7ff6ac628700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff6a40f20e0 0x7ff6a40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:07.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.283 7ff6ac628700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a40f0b50 con 0x7ff6a40ef610 2021-02-10T11:25:07.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.283 7ff6ac628700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a40f0820 con 0x7ff6a40f20e0 2021-02-10T11:25:07.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.283 7ff6ac628700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a40f24e0 con 0x7ff6a41b0470 2021-02-10T11:25:07.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.283 7ff6a9c43700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff6a40ef610 0x7ff6a40efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48000/0 (socket says 172.21.2.101:48000) 2021-02-10T11:25:07.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.283 7ff6aac45700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff6a40f20e0 0x7ff6a40f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:07.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.283 7ff6a9c43700 1 -- 172.21.2.101:0/585935900 learned_addr learned my addr 172.21.2.101:0/585935900 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:25:07.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.283 7ff6aa444700 1 --2- 172.21.2.101:0/585935900 >> v2:172.21.2.108:3300/0 conn(0x7ff6a41b0470 0x7ff6a41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:07.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.283 7ff6aac45700 1 -- 172.21.2.101:0/585935900 >> v2:172.21.2.108:3300/0 conn(0x7ff6a41b0470 msgr2=0x7ff6a41b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:25:07.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.284 7ff6aac45700 1 --2- 172.21.2.101:0/585935900 >> v2:172.21.2.108:3300/0 conn(0x7ff6a41b0470 0x7ff6a41b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:07.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.284 7ff6aac45700 1 -- 172.21.2.101:0/585935900 >> v1:172.21.2.101:6789/0 conn(0x7ff6a40ef610 legacy=0x7ff6a40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:07.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.284 7ff6aac45700 1 -- 172.21.2.101:0/585935900 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a40e52e0 con 0x7ff6a40f20e0 2021-02-10T11:25:07.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.284 7ff6aa444700 1 --2- 172.21.2.101:0/585935900 >> v2:172.21.2.108:3300/0 conn(0x7ff6a41b0470 0x7ff6a41b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:25:07.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.284 7ff6aac45700 1 --2- 172.21.2.101:0/585935900 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6a40f20e0 0x7ff6a40f0200 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7ff69c006e30 tx=0x7ff69c009e60).ready entity=mon.0 client_cookie=3e6cb742ad3505b3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:07.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.284 7ff6a9442700 1 -- 172.21.2.101:0/585935900 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff69c006a50 con 0x7ff6a40f20e0 2021-02-10T11:25:07.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.284 7ff6a9442700 1 -- 172.21.2.101:0/585935900 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c00b300 con 0x7ff6a40f20e0 2021-02-10T11:25:07.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.284 7ff6a9442700 1 -- 172.21.2.101:0/585935900 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff69c00b7e0 con 0x7ff6a40f20e0 2021-02-10T11:25:07.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.284 7ff6ac628700 1 -- 172.21.2.101:0/585935900 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6a40f20e0 msgr2=0x7ff6a40f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:07.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.284 7ff6ac628700 1 --2- 172.21.2.101:0/585935900 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6a40f20e0 0x7ff6a40f0200 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7ff69c006e30 tx=0x7ff69c009e60).stop 2021-02-10T11:25:07.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.285 7ff6ac628700 1 -- 172.21.2.101:0/585935900 shutdown_connections 2021-02-10T11:25:07.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.285 7ff6ac628700 1 --2- 172.21.2.101:0/585935900 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6a40f20e0 0x7ff6a40f0200 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:07.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.285 7ff6ac628700 1 --2- 172.21.2.101:0/585935900 >> v2:172.21.2.108:3300/0 conn(0x7ff6a41b0470 0x7ff6a41b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:07.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.285 7ff6ac628700 1 -- 172.21.2.101:0/585935900 >> 172.21.2.101:0/585935900 conn(0x7ff6a40ce3e0 msgr2=0x7ff6a40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:07.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.285 7ff6ac628700 1 -- 172.21.2.101:0/585935900 shutdown_connections 2021-02-10T11:25:07.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.285 7ff6ac628700 1 -- 172.21.2.101:0/585935900 wait complete. 2021-02-10T11:25:07.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.300 7ff6ac628700 1 Processor -- start 2021-02-10T11:25:07.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.300 7ff6ac628700 1 -- start start 2021-02-10T11:25:07.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.300 7ff6ac628700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6a4190920 0x7ff6a40b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:07.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.300 7ff6ac628700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6a40b8200 0x7ff6a40b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:07.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.300 7ff6ac628700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6a40c1d80 0x7ff6a40c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:07.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.300 7ff6ac628700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a4190d20 con 0x7ff6a4190920 2021-02-10T11:25:07.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.300 7ff6ac628700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a40b8c20 con 0x7ff6a40c1d80 2021-02-10T11:25:07.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.300 7ff6ac628700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a40c27a0 con 0x7ff6a40b8200 2021-02-10T11:25:07.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.300 7ff6aa444700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6a4190920 0x7ff6a40b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:07.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.300 7ff6aa444700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6a4190920 0x7ff6a40b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47512/0 (socket says 172.21.2.101:47512) 2021-02-10T11:25:07.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.300 7ff6aa444700 1 -- 172.21.2.101:0/1456162648 learned_addr learned my addr 172.21.2.101:0/1456162648 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:25:07.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.301 7ff6aa444700 1 -- 172.21.2.101:0/1456162648 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6a40b8200 msgr2=0x7ff6a40b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:07.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.301 7ff6aa444700 1 --2- 172.21.2.101:0/1456162648 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6a40b8200 0x7ff6a40b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:07.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.301 7ff6aa444700 1 -- 172.21.2.101:0/1456162648 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6a40c1d80 msgr2=0x7ff6a40c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:07.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.301 7ff6aa444700 1 --2- 172.21.2.101:0/1456162648 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6a40c1d80 0x7ff6a40c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:07.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.301 7ff6aa444700 1 -- 172.21.2.101:0/1456162648 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a40c29c0 con 0x7ff6a4190920 2021-02-10T11:25:07.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.301 7ff6aa444700 1 --2- 172.21.2.101:0/1456162648 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6a4190920 0x7ff6a40b7be0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7ff6a0006f20 tx=0x7ff6a000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:07.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.301 7ff69affd700 1 -- 172.21.2.101:0/1456162648 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff6a00009b0 con 0x7ff6a4190920 2021-02-10T11:25:07.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.301 7ff69affd700 1 -- 172.21.2.101:0/1456162648 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a000bd80 con 0x7ff6a4190920 2021-02-10T11:25:07.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.301 7ff69affd700 1 -- 172.21.2.101:0/1456162648 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff6a0002220 con 0x7ff6a4190920 2021-02-10T11:25:07.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.301 7ff6ac628700 1 -- 172.21.2.101:0/1456162648 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a40c29c0 con 0x7ff6a4190920 2021-02-10T11:25:07.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.301 7ff6ac628700 1 -- 172.21.2.101:0/1456162648 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a40c29c0 con 0x7ff6a4190920 2021-02-10T11:25:07.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.302 7ff69affd700 1 -- 172.21.2.101:0/1456162648 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff6a0000c90 con 0x7ff6a4190920 2021-02-10T11:25:07.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.302 7ff69affd700 1 --2- 172.21.2.101:0/1456162648 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff68c0186c0 0x7ff68c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:07.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.302 7ff69affd700 1 -- 172.21.2.101:0/1456162648 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff6a00327e0 con 0x7ff6a4190920 2021-02-10T11:25:07.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.302 7ff6a9c43700 1 --2- 172.21.2.101:0/1456162648 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff68c0186c0 0x7ff68c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:07.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.302 7ff6a9c43700 1 --2- 172.21.2.101:0/1456162648 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff68c0186c0 0x7ff68c01abc0 crc :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:07.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.333 7ff6ac628700 1 -- 172.21.2.101:0/1456162648 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a4105ae0 con 0x7ff6a4190920 2021-02-10T11:25:07.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.335 7ff69affd700 1 -- 172.21.2.101:0/1456162648 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff6a0006bb0 con 0x7ff6a4190920 2021-02-10T11:25:07.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.465 7ff6ac628700 1 -- 172.21.2.101:0/1456162648 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839305", "target": ["mgr", ""]}) v1 -- 0x7ff6a40b8c20 con 0x7ff68c0186c0 2021-02-10T11:25:07.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.468 7ff69affd700 1 -- 172.21.2.101:0/1456162648 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7ff694001590 con 0x7ff68c0186c0 2021-02-10T11:25:07.480 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839305/b904e2ef-2ea3-407d-925e-820ea4e987af 2021-02-10T11:25:07.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.474 7ff6ac628700 1 -- 172.21.2.101:0/1456162648 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff68c0186c0 msgr2=0x7ff68c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:07.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.474 7ff6ac628700 1 --2- 172.21.2.101:0/1456162648 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff68c0186c0 0x7ff68c01abc0 crc :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:07.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.474 7ff6ac628700 1 -- 172.21.2.101:0/1456162648 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6a4190920 msgr2=0x7ff6a40b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:07.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.474 7ff6ac628700 1 --2- 172.21.2.101:0/1456162648 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6a4190920 0x7ff6a40b7be0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7ff6a0006f20 tx=0x7ff6a000b990).stop 2021-02-10T11:25:07.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.474 7ff6ac628700 1 -- 172.21.2.101:0/1456162648 shutdown_connections 2021-02-10T11:25:07.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.474 7ff6ac628700 1 --2- 172.21.2.101:0/1456162648 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff68c0186c0 0x7ff68c01abc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:07.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.474 7ff6ac628700 1 --2- 172.21.2.101:0/1456162648 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6a40b8200 0x7ff6a40b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:07.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.474 7ff6ac628700 1 --2- 172.21.2.101:0/1456162648 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6a40c1d80 0x7ff6a40c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:07.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.474 7ff6ac628700 1 --2- 172.21.2.101:0/1456162648 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6a4190920 0x7ff6a40b7be0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:07.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.474 7ff6ac628700 1 -- 172.21.2.101:0/1456162648 >> 172.21.2.101:0/1456162648 conn(0x7ff6a4094e10 msgr2=0x7ff6a40a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:07.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.474 7ff6ac628700 1 -- 172.21.2.101:0/1456162648 shutdown_connections 2021-02-10T11:25:07.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:07.474 7ff6ac628700 1 -- 172.21.2.101:0/1456162648 wait complete. 2021-02-10T11:25:07.489 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839305 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839305/b904e2ef-2ea3-407d-925e-820ea4e987af 2021-02-10T11:25:07.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:07.490 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:07.490 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:07.826 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:07.826 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:07.826 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839305/b904e2ef-2ea3-407d-925e-820ea4e987af/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:08.208 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:08.209 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:08.209 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.025638 s, 40.9 MB/s 2021-02-10T11:25:08.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:08.210 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:08.210 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:08.544 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:08.544 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:08.544 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839305/b904e2ef-2ea3-407d-925e-820ea4e987af/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:08.918 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:08.918 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:08.918 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0261241 s, 40.1 MB/s 2021-02-10T11:25:08.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:08.919 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:08.919 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:09.254 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:09.254 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:09.254 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839305/b904e2ef-2ea3-407d-925e-820ea4e987af/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:09.630 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:09.630 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:09.630 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247021 s, 42.4 MB/s 2021-02-10T11:25:09.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:09.631 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:09.631 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:09.968 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:09.968 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:09.968 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839305/b904e2ef-2ea3-407d-925e-820ea4e987af/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:10.344 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:10.344 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:10.344 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237309 s, 44.2 MB/s 2021-02-10T11:25:10.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:10.345 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:10.345 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:10.681 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:10.681 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:10.681 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839305/b904e2ef-2ea3-407d-925e-820ea4e987af/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:11.056 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:11.056 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:11.057 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023242 s, 45.1 MB/s 2021-02-10T11:25:11.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:11.058 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:11.058 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:11.393 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:11.393 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:11.393 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839305/b904e2ef-2ea3-407d-925e-820ea4e987af/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:11.769 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:11.769 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:11.769 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.025119 s, 41.7 MB/s 2021-02-10T11:25:11.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:11.771 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:11.771 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:12.107 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:12.107 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:12.107 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839305/b904e2ef-2ea3-407d-925e-820ea4e987af/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:12.480 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:12.480 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:12.480 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0227174 s, 46.2 MB/s 2021-02-10T11:25:12.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:12.481 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:12.482 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:12.818 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:12.818 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:12.818 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839305/b904e2ef-2ea3-407d-925e-820ea4e987af/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:13.193 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:13.194 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:13.194 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245379 s, 42.7 MB/s 2021-02-10T11:25:13.195 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:13.195 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:13.195 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:13.531 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:13.531 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:13.531 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839305/b904e2ef-2ea3-407d-925e-820ea4e987af/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:13.905 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:13.906 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:13.906 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.025291 s, 41.5 MB/s 2021-02-10T11:25:13.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:13.907 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:13.907 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:14.241 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:14.242 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:14.242 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839305/b904e2ef-2ea3-407d-925e-820ea4e987af/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:14.617 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:14.617 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:14.617 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238409 s, 44.0 MB/s 2021-02-10T11:25:14.618 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839306 2021-02-10T11:25:15.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.025 7f5ef44fc700 1 Processor -- start 2021-02-10T11:25:15.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.025 7f5ef44fc700 1 -- start start 2021-02-10T11:25:15.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.026 7f5ef44fc700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f5eec1b0470 0x7f5eec1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:15.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.026 7f5ef44fc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eec0ef610 0x7f5eec0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:15.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.026 7f5ef44fc700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5eec0f0030 0x7f5eec0f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:15.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.026 7f5ef44fc700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eec0e5ea0 con 0x7f5eec0f0030 2021-02-10T11:25:15.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.026 7f5ef44fc700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eec0f0b50 con 0x7f5eec1b0470 2021-02-10T11:25:15.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.026 7f5ef44fc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0e52e0 con 0x7f5eec0ef610 2021-02-10T11:25:15.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.026 7f5ef2b19700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5eec0f0030 0x7f5eec0f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:15.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.026 7f5ef2b19700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5eec0f0030 0x7f5eec0f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47518/0 (socket says 172.21.2.101:47518) 2021-02-10T11:25:15.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.026 7f5ef2b19700 1 -- 172.21.2.101:0/2912738866 learned_addr learned my addr 172.21.2.101:0/2912738866 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:25:15.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.026 7f5ef1b17700 1 --2- 172.21.2.101:0/2912738866 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eec0ef610 0x7f5eec0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:15.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.026 7f5ef2318700 1 --2- 172.21.2.101:0/2912738866 >> v2:172.21.2.108:3300/0 conn(0x7f5eec1b0470 0x7f5eec1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:15.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.026 7f5ef2b19700 1 -- 172.21.2.101:0/2912738866 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eec0ef610 msgr2=0x7f5eec0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:25:15.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.026 7f5ef2b19700 1 --2- 172.21.2.101:0/2912738866 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eec0ef610 0x7f5eec0efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:15.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.026 7f5ef2b19700 1 -- 172.21.2.101:0/2912738866 >> v2:172.21.2.108:3300/0 conn(0x7f5eec1b0470 msgr2=0x7f5eec1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:25:15.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.026 7f5ef2b19700 1 --2- 172.21.2.101:0/2912738866 >> v2:172.21.2.108:3300/0 conn(0x7f5eec1b0470 0x7f5eec1b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:15.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.026 7f5ef2b19700 1 -- 172.21.2.101:0/2912738866 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec0f10d0 con 0x7f5eec0f0030 2021-02-10T11:25:15.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.027 7f5ef2b19700 1 --2- 172.21.2.101:0/2912738866 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eec0f0030 0x7f5eec0f0430 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f5ee4006f00 tx=0x7f5ee4009e90).ready entity=mon.0 client_cookie=2ad670771d8a16a3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:15.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.027 7f5ef1316700 1 -- 172.21.2.101:0/2912738866 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5ee400b300 con 0x7f5eec0f0030 2021-02-10T11:25:15.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.027 7f5ef1316700 1 -- 172.21.2.101:0/2912738866 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee400b580 con 0x7f5eec0f0030 2021-02-10T11:25:15.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.027 7f5ef1316700 1 -- 172.21.2.101:0/2912738866 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5ee400b9d0 con 0x7f5eec0f0030 2021-02-10T11:25:15.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.027 7f5ef44fc700 1 -- 172.21.2.101:0/2912738866 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eec0f0030 msgr2=0x7f5eec0f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:15.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.027 7f5ef44fc700 1 --2- 172.21.2.101:0/2912738866 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eec0f0030 0x7f5eec0f0430 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f5ee4006f00 tx=0x7f5ee4009e90).stop 2021-02-10T11:25:15.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.028 7f5ef44fc700 1 -- 172.21.2.101:0/2912738866 shutdown_connections 2021-02-10T11:25:15.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.028 7f5ef44fc700 1 --2- 172.21.2.101:0/2912738866 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eec0ef610 0x7f5eec0efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:15.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.028 7f5ef44fc700 1 --2- 172.21.2.101:0/2912738866 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eec0f0030 0x7f5eec0f0430 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:15.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.028 7f5ef44fc700 1 --2- 172.21.2.101:0/2912738866 >> v2:172.21.2.108:3300/0 conn(0x7f5eec1b0470 0x7f5eec1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:15.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.028 7f5ef44fc700 1 -- 172.21.2.101:0/2912738866 >> 172.21.2.101:0/2912738866 conn(0x7f5eec0ce3e0 msgr2=0x7f5eec0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:15.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.028 7f5ef44fc700 1 -- 172.21.2.101:0/2912738866 shutdown_connections 2021-02-10T11:25:15.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.028 7f5ef44fc700 1 -- 172.21.2.101:0/2912738866 wait complete. 2021-02-10T11:25:15.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.042 7f5ef44fc700 1 Processor -- start 2021-02-10T11:25:15.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.042 7f5ef44fc700 1 -- start start 2021-02-10T11:25:15.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.043 7f5ef44fc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eec0a0910 0x7f5eec105d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:15.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.043 7f5ef44fc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eec1a7a10 0x7f5eec1a7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:15.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.043 7f5ef44fc700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5eec093770 0x7f5eec093b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:15.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.043 7f5ef44fc700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec106390 con 0x7f5eec0a0910 2021-02-10T11:25:15.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.043 7f5ef44fc700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec094190 con 0x7f5eec093770 2021-02-10T11:25:15.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.043 7f5ef44fc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0943b0 con 0x7f5eec1a7a10 2021-02-10T11:25:15.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.043 7f5ef2318700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eec0a0910 0x7f5eec105d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:15.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.043 7f5ef1b17700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eec1a7a10 0x7f5eec1a7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:15.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.043 7f5ef2318700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eec0a0910 0x7f5eec105d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47524/0 (socket says 172.21.2.101:47524) 2021-02-10T11:25:15.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.043 7f5ef1b17700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eec1a7a10 0x7f5eec1a7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:40254/0 (socket says 172.21.2.101:40254) 2021-02-10T11:25:15.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.043 7f5ef2318700 1 -- 172.21.2.101:0/3775002439 learned_addr learned my addr 172.21.2.101:0/3775002439 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:25:15.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.043 7f5ef2b19700 1 --2- 172.21.2.101:0/3775002439 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5eec093770 0x7f5eec093b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:15.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.043 7f5ef2318700 1 -- 172.21.2.101:0/3775002439 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eec1a7a10 msgr2=0x7f5eec1a7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:15.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.043 7f5ef2318700 1 --2- 172.21.2.101:0/3775002439 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eec1a7a10 0x7f5eec1a7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:15.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.043 7f5ef2318700 1 -- 172.21.2.101:0/3775002439 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5eec093770 msgr2=0x7f5eec093b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:15.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.043 7f5ef2318700 1 --2- 172.21.2.101:0/3775002439 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5eec093770 0x7f5eec093b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:15.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.043 7f5ef2318700 1 -- 172.21.2.101:0/3775002439 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec106c30 con 0x7f5eec0a0910 2021-02-10T11:25:15.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.043 7f5ef2318700 1 --2- 172.21.2.101:0/3775002439 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eec0a0910 0x7f5eec105d70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f5ee8006f60 tx=0x7f5ee800b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:15.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.044 7f5edeffd700 1 -- 172.21.2.101:0/3775002439 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5ee80009b0 con 0x7f5eec0a0910 2021-02-10T11:25:15.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.044 7f5edeffd700 1 -- 172.21.2.101:0/3775002439 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee800bdf0 con 0x7f5eec0a0910 2021-02-10T11:25:15.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.044 7f5edeffd700 1 -- 172.21.2.101:0/3775002439 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5ee8002030 con 0x7f5eec0a0910 2021-02-10T11:25:15.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.044 7f5ef44fc700 1 -- 172.21.2.101:0/3775002439 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eec106c30 con 0x7f5eec0a0910 2021-02-10T11:25:15.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.044 7f5ef44fc700 1 -- 172.21.2.101:0/3775002439 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eec1071c0 con 0x7f5eec0a0910 2021-02-10T11:25:15.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.044 7f5edeffd700 1 -- 172.21.2.101:0/3775002439 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5ee8000c90 con 0x7f5eec0a0910 2021-02-10T11:25:15.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.044 7f5edeffd700 1 --2- 172.21.2.101:0/3775002439 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5ed40186c0 0x7f5ed401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:15.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.044 7f5edeffd700 1 -- 172.21.2.101:0/3775002439 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5ee8032550 con 0x7f5eec0a0910 2021-02-10T11:25:15.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.045 7f5ef1b17700 1 --2- 172.21.2.101:0/3775002439 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5ed40186c0 0x7f5ed401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:15.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.045 7f5ef1b17700 1 --2- 172.21.2.101:0/3775002439 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5ed40186c0 0x7f5ed401abc0 crc :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:15.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.076 7f5ef44fc700 1 -- 172.21.2.101:0/3775002439 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eec105b40 con 0x7f5eec0a0910 2021-02-10T11:25:15.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.078 7f5edeffd700 1 -- 172.21.2.101:0/3775002439 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5ee8000940 con 0x7f5eec0a0910 2021-02-10T11:25:15.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.207 7f5ef44fc700 1 -- 172.21.2.101:0/3775002439 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839306", "target": ["mgr", ""]}) v1 -- 0x7f5eec106390 con 0x7f5ed40186c0 2021-02-10T11:25:15.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.234 7f5edeffd700 1 -- 172.21.2.101:0/3775002439 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f5ee0006a70 con 0x7f5ed40186c0 2021-02-10T11:25:15.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.240 7f5ef44fc700 1 -- 172.21.2.101:0/3775002439 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5ed40186c0 msgr2=0x7f5ed401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:15.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.240 7f5ef44fc700 1 --2- 172.21.2.101:0/3775002439 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5ed40186c0 0x7f5ed401abc0 crc :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:15.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.240 7f5ef44fc700 1 -- 172.21.2.101:0/3775002439 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eec0a0910 msgr2=0x7f5eec105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:15.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.240 7f5ef44fc700 1 --2- 172.21.2.101:0/3775002439 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eec0a0910 0x7f5eec105d70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f5ee8006f60 tx=0x7f5ee800b970).stop 2021-02-10T11:25:15.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.240 7f5ef44fc700 1 -- 172.21.2.101:0/3775002439 shutdown_connections 2021-02-10T11:25:15.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.240 7f5ef44fc700 1 --2- 172.21.2.101:0/3775002439 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5ed40186c0 0x7f5ed401abc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:15.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.240 7f5ef44fc700 1 --2- 172.21.2.101:0/3775002439 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eec1a7a10 0x7f5eec1a7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:15.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.240 7f5ef44fc700 1 --2- 172.21.2.101:0/3775002439 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5eec093770 0x7f5eec093b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:15.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.240 7f5ef44fc700 1 --2- 172.21.2.101:0/3775002439 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eec0a0910 0x7f5eec105d70 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:15.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.240 7f5ef44fc700 1 -- 172.21.2.101:0/3775002439 >> 172.21.2.101:0/3775002439 conn(0x7f5eec095ef0 msgr2=0x7f5eec094e40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:15.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.240 7f5ef44fc700 1 -- 172.21.2.101:0/3775002439 shutdown_connections 2021-02-10T11:25:15.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.240 7f5ef44fc700 1 -- 172.21.2.101:0/3775002439 wait complete. 2021-02-10T11:25:15.256 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839306 2021-02-10T11:25:15.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.662 7f81ba7b0700 1 Processor -- start 2021-02-10T11:25:15.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.662 7f81ba7b0700 1 -- start start 2021-02-10T11:25:15.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.663 7f81ba7b0700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f81b41b0870 0x7f81b40f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:15.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.663 7f81ba7b0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f81b40efe00 0x7f81b40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:15.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.663 7f81ba7b0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b41b0c70 con 0x7f81b41b0470 2021-02-10T11:25:15.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.663 7f81ba7b0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b40e5ea0 con 0x7f81b40efe00 2021-02-10T11:25:15.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.663 7f81ba7b0700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b40f0b50 con 0x7f81b41b0870 2021-02-10T11:25:15.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.663 7f81b8dcd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f81b40efe00 0x7f81b40f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:15.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.663 7f81b8dcd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f81b40efe00 0x7f81b40f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47530/0 (socket says 172.21.2.101:47530) 2021-02-10T11:25:15.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.663 7f81b8dcd700 1 -- 172.21.2.101:0/2769255436 learned_addr learned my addr 172.21.2.101:0/2769255436 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:25:15.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.663 7f81b8dcd700 1 -- 172.21.2.101:0/2769255436 >> v2:172.21.2.108:3300/0 conn(0x7f81b41b0870 msgr2=0x7f81b40f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:25:15.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.663 7f81b8dcd700 1 --2- 172.21.2.101:0/2769255436 >> v2:172.21.2.108:3300/0 conn(0x7f81b41b0870 0x7f81b40f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:15.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.663 7f81b8dcd700 1 -- 172.21.2.101:0/2769255436 >> v1:172.21.2.108:6789/0 conn(0x7f81b41b0470 legacy=0x7f81b40ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:25:15.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.663 7f81b8dcd700 1 -- 172.21.2.101:0/2769255436 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b40f0820 con 0x7f81b40efe00 2021-02-10T11:25:15.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.663 7f81b8dcd700 1 --2- 172.21.2.101:0/2769255436 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81b40efe00 0x7f81b40f0200 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f81ac006f00 tx=0x7f81ac009f10).ready entity=mon.0 client_cookie=d920d0f60e6db851 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:15.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.664 7f81b2ffd700 1 -- 172.21.2.101:0/2769255436 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81ac00b3b0 con 0x7f81b40efe00 2021-02-10T11:25:15.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.664 7f81b2ffd700 1 -- 172.21.2.101:0/2769255436 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ac00b630 con 0x7f81b40efe00 2021-02-10T11:25:15.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.664 7f81b2ffd700 1 -- 172.21.2.101:0/2769255436 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81ac00bb50 con 0x7f81b40efe00 2021-02-10T11:25:15.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.664 7f81ba7b0700 1 -- 172.21.2.101:0/2769255436 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81b40efe00 msgr2=0x7f81b40f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:15.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.664 7f81ba7b0700 1 --2- 172.21.2.101:0/2769255436 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81b40efe00 0x7f81b40f0200 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f81ac006f00 tx=0x7f81ac009f10).stop 2021-02-10T11:25:15.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.664 7f81ba7b0700 1 -- 172.21.2.101:0/2769255436 shutdown_connections 2021-02-10T11:25:15.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.664 7f81ba7b0700 1 --2- 172.21.2.101:0/2769255436 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81b40efe00 0x7f81b40f0200 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:15.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.664 7f81ba7b0700 1 --2- 172.21.2.101:0/2769255436 >> v2:172.21.2.108:3300/0 conn(0x7f81b41b0870 0x7f81b40f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:15.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.664 7f81ba7b0700 1 -- 172.21.2.101:0/2769255436 >> 172.21.2.101:0/2769255436 conn(0x7f81b40ce3e0 msgr2=0x7f81b40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:15.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.664 7f81ba7b0700 1 -- 172.21.2.101:0/2769255436 shutdown_connections 2021-02-10T11:25:15.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.664 7f81ba7b0700 1 -- 172.21.2.101:0/2769255436 wait complete. 2021-02-10T11:25:15.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.679 7f81ba7b0700 1 Processor -- start 2021-02-10T11:25:15.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.679 7f81ba7b0700 1 -- start start 2021-02-10T11:25:15.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.679 7f81ba7b0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81b4094fb0 0x7f81b409e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:15.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.679 7f81ba7b0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81b40c2be0 0x7f81b40c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:15.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.679 7f81ba7b0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81b40c3600 0x7f81b40a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:15.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.679 7f81ba7b0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b409ebf0 con 0x7f81b40c2be0 2021-02-10T11:25:15.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.679 7f81ba7b0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b40aa2a0 con 0x7f81b40c3600 2021-02-10T11:25:15.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.679 7f81ba7b0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b40aa4c0 con 0x7f81b4094fb0 2021-02-10T11:25:15.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.680 7f81b37fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81b40c2be0 0x7f81b40c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:15.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.680 7f81b37fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81b40c2be0 0x7f81b40c2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47536/0 (socket says 172.21.2.101:47536) 2021-02-10T11:25:15.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.680 7f81b37fe700 1 -- 172.21.2.101:0/4159044317 learned_addr learned my addr 172.21.2.101:0/4159044317 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:25:15.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.680 7f81b37fe700 1 -- 172.21.2.101:0/4159044317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81b4094fb0 msgr2=0x7f81b409e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:15.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.680 7f81b37fe700 1 --2- 172.21.2.101:0/4159044317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81b4094fb0 0x7f81b409e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:15.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.680 7f81b37fe700 1 -- 172.21.2.101:0/4159044317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81b40c3600 msgr2=0x7f81b40a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:15.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.680 7f81b37fe700 1 --2- 172.21.2.101:0/4159044317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81b40c3600 0x7f81b40a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:15.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.680 7f81b37fe700 1 -- 172.21.2.101:0/4159044317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b40aa6e0 con 0x7f81b40c2be0 2021-02-10T11:25:15.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.680 7f81b37fe700 1 --2- 172.21.2.101:0/4159044317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81b40c2be0 0x7f81b40c2fe0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f81a8006f00 tx=0x7f81a800b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:15.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.681 7f81b0ff9700 1 -- 172.21.2.101:0/4159044317 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81a800bd30 con 0x7f81b40c2be0 2021-02-10T11:25:15.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.681 7f81b0ff9700 1 -- 172.21.2.101:0/4159044317 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a8000c90 con 0x7f81b40c2be0 2021-02-10T11:25:15.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.681 7f81b0ff9700 1 -- 172.21.2.101:0/4159044317 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81a80022a0 con 0x7f81b40c2be0 2021-02-10T11:25:15.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.681 7f81ba7b0700 1 -- 172.21.2.101:0/4159044317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b40aa6e0 con 0x7f81b40c2be0 2021-02-10T11:25:15.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.681 7f81ba7b0700 1 -- 172.21.2.101:0/4159044317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b40aac70 con 0x7f81b40c2be0 2021-02-10T11:25:15.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.681 7f81b0ff9700 1 -- 172.21.2.101:0/4159044317 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f81a8005310 con 0x7f81b40c2be0 2021-02-10T11:25:15.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.681 7f81b0ff9700 1 --2- 172.21.2.101:0/4159044317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81940186c0 0x7f819401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:15.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.681 7f81b0ff9700 1 -- 172.21.2.101:0/4159044317 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f81a8032430 con 0x7f81b40c2be0 2021-02-10T11:25:15.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.681 7f81b3fff700 1 --2- 172.21.2.101:0/4159044317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81940186c0 0x7f819401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:15.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.682 7f81b3fff700 1 --2- 172.21.2.101:0/4159044317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81940186c0 0x7f819401abc0 crc :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:15.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.713 7f81ba7b0700 1 -- 172.21.2.101:0/4159044317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b40a9750 con 0x7f81b40c2be0 2021-02-10T11:25:15.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.715 7f81b0ff9700 1 -- 172.21.2.101:0/4159044317 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f81a8000940 con 0x7f81b40c2be0 2021-02-10T11:25:15.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.843 7f81ba7b0700 1 -- 172.21.2.101:0/4159044317 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839306", "target": ["mgr", ""]}) v1 -- 0x7f81b40aa2a0 con 0x7f81940186c0 2021-02-10T11:25:15.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.847 7f81b0ff9700 1 -- 172.21.2.101:0/4159044317 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f81a4006e40 con 0x7f81940186c0 2021-02-10T11:25:15.859 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839306/d9460793-c481-40d7-81de-79bf22f18441 2021-02-10T11:25:15.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.852 7f81ba7b0700 1 -- 172.21.2.101:0/4159044317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81940186c0 msgr2=0x7f819401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:15.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.852 7f81ba7b0700 1 --2- 172.21.2.101:0/4159044317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81940186c0 0x7f819401abc0 crc :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:15.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.852 7f81ba7b0700 1 -- 172.21.2.101:0/4159044317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81b40c2be0 msgr2=0x7f81b40c2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:15.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.852 7f81ba7b0700 1 --2- 172.21.2.101:0/4159044317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81b40c2be0 0x7f81b40c2fe0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f81a8006f00 tx=0x7f81a800b940).stop 2021-02-10T11:25:15.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.852 7f81ba7b0700 1 -- 172.21.2.101:0/4159044317 shutdown_connections 2021-02-10T11:25:15.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.852 7f81ba7b0700 1 --2- 172.21.2.101:0/4159044317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81b40c3600 0x7f81b40a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:15.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.852 7f81ba7b0700 1 --2- 172.21.2.101:0/4159044317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81b40c2be0 0x7f81b40c2fe0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:15.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.852 7f81ba7b0700 1 --2- 172.21.2.101:0/4159044317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81940186c0 0x7f819401abc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:15.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.852 7f81ba7b0700 1 --2- 172.21.2.101:0/4159044317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81b4094fb0 0x7f81b409e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:15.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.852 7f81ba7b0700 1 -- 172.21.2.101:0/4159044317 >> 172.21.2.101:0/4159044317 conn(0x7f81b409f630 msgr2=0x7f81b40a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:15.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.853 7f81ba7b0700 1 -- 172.21.2.101:0/4159044317 shutdown_connections 2021-02-10T11:25:15.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:15.853 7f81ba7b0700 1 -- 172.21.2.101:0/4159044317 wait complete. 2021-02-10T11:25:15.868 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839306 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839306/d9460793-c481-40d7-81de-79bf22f18441 2021-02-10T11:25:15.869 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:15.869 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:15.869 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:16.204 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:16.204 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:16.205 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839306/d9460793-c481-40d7-81de-79bf22f18441/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:16.588 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:16.588 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:16.588 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251667 s, 41.7 MB/s 2021-02-10T11:25:16.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:16.589 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:16.590 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:16.925 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:16.926 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:16.926 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839306/d9460793-c481-40d7-81de-79bf22f18441/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:17.300 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:17.301 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:17.301 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0232385 s, 45.1 MB/s 2021-02-10T11:25:17.302 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:17.302 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:17.303 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:17.644 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:17.645 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:17.645 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839306/d9460793-c481-40d7-81de-79bf22f18441/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:18.022 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:18.022 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:18.022 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243649 s, 43.0 MB/s 2021-02-10T11:25:18.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:18.023 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:18.023 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:18.359 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:18.359 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:18.360 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839306/d9460793-c481-40d7-81de-79bf22f18441/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:18.734 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:18.734 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:18.734 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237587 s, 44.1 MB/s 2021-02-10T11:25:18.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:18.735 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:18.736 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:19.071 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:19.072 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:19.072 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839306/d9460793-c481-40d7-81de-79bf22f18441/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:19.447 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:19.447 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:19.448 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0261247 s, 40.1 MB/s 2021-02-10T11:25:19.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:19.449 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:19.449 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:19.784 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:19.784 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:19.784 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839306/d9460793-c481-40d7-81de-79bf22f18441/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:20.157 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:20.158 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:20.158 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0223735 s, 46.9 MB/s 2021-02-10T11:25:20.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:20.159 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:20.159 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:20.494 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:20.495 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:20.495 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839306/d9460793-c481-40d7-81de-79bf22f18441/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:20.871 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:20.871 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:20.871 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234064 s, 44.8 MB/s 2021-02-10T11:25:20.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:20.872 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:20.872 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:21.208 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:21.208 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:21.208 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839306/d9460793-c481-40d7-81de-79bf22f18441/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:21.584 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:21.584 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:21.584 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231565 s, 45.3 MB/s 2021-02-10T11:25:21.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:21.585 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:21.585 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:21.921 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:21.921 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:21.921 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839306/d9460793-c481-40d7-81de-79bf22f18441/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:22.202 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:25:22.204 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:25:22.298 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:22.298 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:22.299 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0255048 s, 41.1 MB/s 2021-02-10T11:25:22.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:22.300 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:22.300 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:22.657 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:22.657 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:22.658 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839306/d9460793-c481-40d7-81de-79bf22f18441/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:23.040 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:23.040 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:23.040 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023808 s, 44.0 MB/s 2021-02-10T11:25:23.041 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839307 2021-02-10T11:25:23.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.448 7fe471164700 1 Processor -- start 2021-02-10T11:25:23.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.448 7fe471164700 1 -- start start 2021-02-10T11:25:23.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.448 7fe471164700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe46c0f05f0 0x7fe46c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:23.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.448 7fe471164700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46c1b0c70 con 0x7fe46c1b0870 2021-02-10T11:25:23.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.448 7fe471164700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46c0f0b50 con 0x7fe46c1b0470 2021-02-10T11:25:23.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.448 7fe471164700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe46c0e5ea0 con 0x7fe46c0f05f0 2021-02-10T11:25:23.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.448 7fe46a59c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe46c1b0870 0x7fe46c0efe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48040/0 (socket says 172.21.2.101:48040) 2021-02-10T11:25:23.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.448 7fe46a59c700 1 -- 172.21.2.101:0/1287090904 learned_addr learned my addr 172.21.2.101:0/1287090904 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:25:23.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.449 7fe46b59e700 1 --2- 172.21.2.101:0/1287090904 >> v2:172.21.2.108:3300/0 conn(0x7fe46c0f05f0 0x7fe46c0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:23.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.449 7fe469d9b700 1 -- 172.21.2.101:0/1287090904 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559337044 0 0) 0x7fe4540010b0 con 0x7fe46c1b0870 2021-02-10T11:25:23.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.449 7fe469d9b700 1 -- 172.21.2.101:0/1287090904 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe458002fe0 con 0x7fe46c1b0870 2021-02-10T11:25:23.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.449 7fe46b59e700 1 -- 172.21.2.101:0/1287090904 >> v1:172.21.2.108:6789/0 conn(0x7fe46c1b0470 legacy=0x7fe46c0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:23.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.449 7fe46b59e700 1 -- 172.21.2.101:0/1287090904 >> v1:172.21.2.101:6789/0 conn(0x7fe46c1b0870 legacy=0x7fe46c0efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:23.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.449 7fe46b59e700 1 -- 172.21.2.101:0/1287090904 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c0e52e0 con 0x7fe46c0f05f0 2021-02-10T11:25:23.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.449 7fe46b59e700 1 --2- 172.21.2.101:0/1287090904 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe46c0f05f0 0x7fe46c0f20e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fe464006e50 tx=0x7fe464009e70).ready entity=mon.1 client_cookie=8e14830f91012e43 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:23.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.449 7fe469d9b700 1 -- 172.21.2.101:0/1287090904 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe46400b510 con 0x7fe46c0f05f0 2021-02-10T11:25:23.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.449 7fe469d9b700 1 -- 172.21.2.101:0/1287090904 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe464006790 con 0x7fe46c0f05f0 2021-02-10T11:25:23.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.449 7fe469d9b700 1 -- 172.21.2.101:0/1287090904 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe464000f90 con 0x7fe46c0f05f0 2021-02-10T11:25:23.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.449 7fe471164700 1 -- 172.21.2.101:0/1287090904 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe46c0f05f0 msgr2=0x7fe46c0f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:23.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.449 7fe471164700 1 --2- 172.21.2.101:0/1287090904 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe46c0f05f0 0x7fe46c0f20e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fe464006e50 tx=0x7fe464009e70).stop 2021-02-10T11:25:23.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.450 7fe471164700 1 -- 172.21.2.101:0/1287090904 shutdown_connections 2021-02-10T11:25:23.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.450 7fe471164700 1 --2- 172.21.2.101:0/1287090904 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe46c0f05f0 0x7fe46c0f20e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:23.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.450 7fe471164700 1 -- 172.21.2.101:0/1287090904 >> 172.21.2.101:0/1287090904 conn(0x7fe46c0ce3e0 msgr2=0x7fe46c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:23.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.450 7fe471164700 1 -- 172.21.2.101:0/1287090904 shutdown_connections 2021-02-10T11:25:23.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.450 7fe471164700 1 -- 172.21.2.101:0/1287090904 wait complete. 2021-02-10T11:25:23.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.465 7fe471164700 1 Processor -- start 2021-02-10T11:25:23.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.465 7fe471164700 1 -- start start 2021-02-10T11:25:23.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.465 7fe471164700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe46c1a0570 0x7fe46c0a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:23.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.465 7fe471164700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe46c0f1fd0 0x7fe46c0f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:23.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.465 7fe471164700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe46c0a5450 0x7fe46c0a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:23.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.465 7fe471164700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c0a4bb0 con 0x7fe46c0f1fd0 2021-02-10T11:25:23.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.465 7fe471164700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c0b35e0 con 0x7fe46c1a0570 2021-02-10T11:25:23.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.465 7fe471164700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c0a1f00 con 0x7fe46c0a5450 2021-02-10T11:25:23.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.465 7fe46a59c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe46c0f1fd0 0x7fe46c0f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:23.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.465 7fe46a59c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe46c0f1fd0 0x7fe46c0f23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47552/0 (socket says 172.21.2.101:47552) 2021-02-10T11:25:23.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.465 7fe46a59c700 1 -- 172.21.2.101:0/1883636825 learned_addr learned my addr 172.21.2.101:0/1883636825 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:25:23.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.466 7fe46b59e700 1 --2- 172.21.2.101:0/1883636825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe46c0a5450 0x7fe46c0a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:23.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.466 7fe46ad9d700 1 --2- 172.21.2.101:0/1883636825 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe46c1a0570 0x7fe46c0a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:23.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.466 7fe46a59c700 1 -- 172.21.2.101:0/1883636825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe46c0a5450 msgr2=0x7fe46c0a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:23.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.466 7fe46a59c700 1 --2- 172.21.2.101:0/1883636825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe46c0a5450 0x7fe46c0a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:23.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.466 7fe46a59c700 1 -- 172.21.2.101:0/1883636825 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe46c1a0570 msgr2=0x7fe46c0a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:23.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.466 7fe46a59c700 1 --2- 172.21.2.101:0/1883636825 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe46c1a0570 0x7fe46c0a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:23.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.466 7fe46a59c700 1 -- 172.21.2.101:0/1883636825 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c0a5e70 con 0x7fe46c0f1fd0 2021-02-10T11:25:23.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.466 7fe46a59c700 1 --2- 172.21.2.101:0/1883636825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe46c0f1fd0 0x7fe46c0f23d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7fe454001c60 tx=0x7fe45400a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:23.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.466 7fe4637fe700 1 -- 172.21.2.101:0/1883636825 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe45400acc0 con 0x7fe46c0f1fd0 2021-02-10T11:25:23.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.466 7fe4637fe700 1 -- 172.21.2.101:0/1883636825 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe454004350 con 0x7fe46c0f1fd0 2021-02-10T11:25:23.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.467 7fe471164700 1 -- 172.21.2.101:0/1883636825 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe46c0a5e70 con 0x7fe46c0f1fd0 2021-02-10T11:25:23.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.467 7fe4637fe700 1 -- 172.21.2.101:0/1883636825 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe4540047b0 con 0x7fe46c0f1fd0 2021-02-10T11:25:23.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.467 7fe471164700 1 -- 172.21.2.101:0/1883636825 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe46c0a5e70 con 0x7fe46c0f1fd0 2021-02-10T11:25:23.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.467 7fe4637fe700 1 -- 172.21.2.101:0/1883636825 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe454004530 con 0x7fe46c0f1fd0 2021-02-10T11:25:23.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.467 7fe4637fe700 1 --2- 172.21.2.101:0/1883636825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe4500186c0 0x7fe45001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:23.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.467 7fe4637fe700 1 -- 172.21.2.101:0/1883636825 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe454032670 con 0x7fe46c0f1fd0 2021-02-10T11:25:23.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.468 7fe46ad9d700 1 --2- 172.21.2.101:0/1883636825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe4500186c0 0x7fe45001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:23.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.468 7fe46ad9d700 1 --2- 172.21.2.101:0/1883636825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe4500186c0 0x7fe45001abc0 crc :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:23.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.498 7fe471164700 1 -- 172.21.2.101:0/1883636825 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46c0a4bb0 con 0x7fe46c0f1fd0 2021-02-10T11:25:23.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.501 7fe4637fe700 1 -- 172.21.2.101:0/1883636825 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe454004140 con 0x7fe46c0f1fd0 2021-02-10T11:25:23.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.626 7fe471164700 1 -- 172.21.2.101:0/1883636825 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839307", "target": ["mgr", ""]}) v1 -- 0x7fe46c0b35e0 con 0x7fe4500186c0 2021-02-10T11:25:23.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.653 7fe4637fe700 1 -- 172.21.2.101:0/1883636825 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fe45c006550 con 0x7fe4500186c0 2021-02-10T11:25:23.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.658 7fe471164700 1 -- 172.21.2.101:0/1883636825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe4500186c0 msgr2=0x7fe45001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:23.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.658 7fe471164700 1 --2- 172.21.2.101:0/1883636825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe4500186c0 0x7fe45001abc0 crc :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:23.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.659 7fe471164700 1 -- 172.21.2.101:0/1883636825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe46c0f1fd0 msgr2=0x7fe46c0f23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:23.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.659 7fe471164700 1 --2- 172.21.2.101:0/1883636825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe46c0f1fd0 0x7fe46c0f23d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7fe454001c60 tx=0x7fe45400a8d0).stop 2021-02-10T11:25:23.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.659 7fe471164700 1 -- 172.21.2.101:0/1883636825 shutdown_connections 2021-02-10T11:25:23.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.659 7fe471164700 1 --2- 172.21.2.101:0/1883636825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe4500186c0 0x7fe45001abc0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:23.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.659 7fe471164700 1 --2- 172.21.2.101:0/1883636825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe46c0a5450 0x7fe46c0a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:23.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.659 7fe471164700 1 --2- 172.21.2.101:0/1883636825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe46c0f1fd0 0x7fe46c0f23d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:23.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.659 7fe471164700 1 --2- 172.21.2.101:0/1883636825 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe46c1a0570 0x7fe46c0a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:23.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.659 7fe471164700 1 -- 172.21.2.101:0/1883636825 >> 172.21.2.101:0/1883636825 conn(0x7fe46c0958b0 msgr2=0x7fe46c0bf040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:23.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.659 7fe471164700 1 -- 172.21.2.101:0/1883636825 shutdown_connections 2021-02-10T11:25:23.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:23.659 7fe471164700 1 -- 172.21.2.101:0/1883636825 wait complete. 2021-02-10T11:25:23.675 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839307 2021-02-10T11:25:24.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.082 7f6fc97b7700 1 Processor -- start 2021-02-10T11:25:24.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.082 7f6fc97b7700 1 -- start start 2021-02-10T11:25:24.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.083 7f6fc97b7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6fc41b0470 0x7f6fc41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:24.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.083 7f6fc97b7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6fc40ef610 0x7f6fc40efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:24.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.083 7f6fc97b7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6fc40f0030 0x7f6fc40f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:24.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.083 7f6fc97b7700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc40e5ea0 con 0x7f6fc41b0470 2021-02-10T11:25:24.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.083 7f6fc97b7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc40f0b50 con 0x7f6fc40f0030 2021-02-10T11:25:24.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.083 7f6fc97b7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40e52e0 con 0x7f6fc40ef610 2021-02-10T11:25:24.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.083 7f6fc3574700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6fc41b0470 0x7f6fc41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:24.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.083 7f6fc3574700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6fc41b0470 0x7f6fc41b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47558/0 (socket says 172.21.2.101:47558) 2021-02-10T11:25:24.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.083 7f6fc2d73700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6fc40ef610 0x7f6fc40efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:24.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.083 7f6fc3574700 1 -- 172.21.2.101:0/4077492396 learned_addr learned my addr 172.21.2.101:0/4077492396 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:25:24.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.083 7f6fc3d75700 1 --2- 172.21.2.101:0/4077492396 >> v2:172.21.2.108:3300/0 conn(0x7f6fc40f0030 0x7f6fc40f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:24.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.084 7f6fc3d75700 1 -- 172.21.2.101:0/4077492396 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6fc40ef610 msgr2=0x7f6fc40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:25:24.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.084 7f6fc3d75700 1 --2- 172.21.2.101:0/4077492396 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6fc40ef610 0x7f6fc40efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:24.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.084 7f6fc3d75700 1 -- 172.21.2.101:0/4077492396 >> v2:172.21.2.101:3300/0 conn(0x7f6fc41b0470 msgr2=0x7f6fc41b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:25:24.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.084 7f6fc3d75700 1 --2- 172.21.2.101:0/4077492396 >> v2:172.21.2.101:3300/0 conn(0x7f6fc41b0470 0x7f6fc41b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:24.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.084 7f6fc3d75700 1 -- 172.21.2.101:0/4077492396 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc40f10d0 con 0x7f6fc40f0030 2021-02-10T11:25:24.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.084 7f6fc2d73700 1 --2- 172.21.2.101:0/4077492396 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6fc40ef610 0x7f6fc40efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:25:24.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.084 7f6fc3d75700 1 --2- 172.21.2.101:0/4077492396 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6fc40f0030 0x7f6fc40f0430 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f6fbc006e50 tx=0x7f6fbc009e70).ready entity=mon.1 client_cookie=94848ce2426b04dc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:24.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.084 7f6fc2572700 1 -- 172.21.2.101:0/4077492396 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6fbc000f90 con 0x7f6fc40f0030 2021-02-10T11:25:24.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.084 7f6fc2572700 1 -- 172.21.2.101:0/4077492396 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fbc0066f0 con 0x7f6fc40f0030 2021-02-10T11:25:24.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.084 7f6fc2572700 1 -- 172.21.2.101:0/4077492396 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6fbc00b700 con 0x7f6fc40f0030 2021-02-10T11:25:24.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.084 7f6fc97b7700 1 -- 172.21.2.101:0/4077492396 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6fc40f0030 msgr2=0x7f6fc40f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:24.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.084 7f6fc97b7700 1 --2- 172.21.2.101:0/4077492396 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6fc40f0030 0x7f6fc40f0430 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f6fbc006e50 tx=0x7f6fbc009e70).stop 2021-02-10T11:25:24.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.085 7f6fc97b7700 1 -- 172.21.2.101:0/4077492396 shutdown_connections 2021-02-10T11:25:24.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.085 7f6fc97b7700 1 --2- 172.21.2.101:0/4077492396 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6fc40ef610 0x7f6fc40efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:24.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.085 7f6fc97b7700 1 --2- 172.21.2.101:0/4077492396 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6fc40f0030 0x7f6fc40f0430 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:24.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.085 7f6fc97b7700 1 --2- 172.21.2.101:0/4077492396 >> v2:172.21.2.101:3300/0 conn(0x7f6fc41b0470 0x7f6fc41b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:24.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.085 7f6fc97b7700 1 -- 172.21.2.101:0/4077492396 >> 172.21.2.101:0/4077492396 conn(0x7f6fc40ce3e0 msgr2=0x7f6fc40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:24.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.085 7f6fc97b7700 1 -- 172.21.2.101:0/4077492396 shutdown_connections 2021-02-10T11:25:24.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.085 7f6fc97b7700 1 -- 172.21.2.101:0/4077492396 wait complete. 2021-02-10T11:25:24.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.100 7f6fc97b7700 1 Processor -- start 2021-02-10T11:25:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.100 7f6fc97b7700 1 -- start start 2021-02-10T11:25:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.100 7f6fc97b7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6fc40b7db0 0x7f6fc40b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.100 7f6fc97b7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6fc40b8880 0x7f6fc41a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.100 7f6fc97b7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6fc40b8c80 0x7f6fc41a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.100 7f6fc97b7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40b79b0 con 0x7f6fc40b8c80 2021-02-10T11:25:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.100 7f6fc97b7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc41a8090 con 0x7f6fc40b7db0 2021-02-10T11:25:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.100 7f6fc97b7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc41a82b0 con 0x7f6fc40b8880 2021-02-10T11:25:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.100 7f6fc3d75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6fc40b8c80 0x7f6fc41a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.100 7f6fc3d75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6fc40b8c80 0x7f6fc41a7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47566/0 (socket says 172.21.2.101:47566) 2021-02-10T11:25:24.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.100 7f6fc3d75700 1 -- 172.21.2.101:0/629853055 learned_addr learned my addr 172.21.2.101:0/629853055 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:25:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.100 7f6fc3574700 1 --2- 172.21.2.101:0/629853055 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6fc40b7db0 0x7f6fc40b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.101 7f6fc2d73700 1 --2- 172.21.2.101:0/629853055 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6fc40b8880 0x7f6fc41a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.101 7f6fc3574700 1 -- 172.21.2.101:0/629853055 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6fc40b8880 msgr2=0x7f6fc41a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.101 7f6fc3574700 1 --2- 172.21.2.101:0/629853055 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6fc40b8880 0x7f6fc41a7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.101 7f6fc3574700 1 -- 172.21.2.101:0/629853055 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6fc40b8c80 msgr2=0x7f6fc41a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.101 7f6fc3574700 1 --2- 172.21.2.101:0/629853055 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6fc40b8c80 0x7f6fc41a7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.101 7f6fc3574700 1 -- 172.21.2.101:0/629853055 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc4190c10 con 0x7f6fc40b7db0 2021-02-10T11:25:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.101 7f6fc3d75700 1 --2- 172.21.2.101:0/629853055 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6fc40b8c80 0x7f6fc41a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:25:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.101 7f6fc3574700 1 --2- 172.21.2.101:0/629853055 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6fc40b7db0 0x7f6fc40b8260 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f6fac006ed0 tx=0x7f6fac00b8c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.101 7f6fbbfff700 1 -- 172.21.2.101:0/629853055 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6fac006340 con 0x7f6fc40b7db0 2021-02-10T11:25:24.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.101 7f6fbbfff700 1 -- 172.21.2.101:0/629853055 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fac00bcb0 con 0x7f6fc40b7db0 2021-02-10T11:25:24.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.101 7f6fc97b7700 1 -- 172.21.2.101:0/629853055 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc4190c10 con 0x7f6fc40b7db0 2021-02-10T11:25:24.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.101 7f6fbbfff700 1 -- 172.21.2.101:0/629853055 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6fac0022b0 con 0x7f6fc40b7db0 2021-02-10T11:25:24.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.101 7f6fc97b7700 1 -- 172.21.2.101:0/629853055 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc4190c10 con 0x7f6fc40b7db0 2021-02-10T11:25:24.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.102 7f6fbbfff700 1 -- 172.21.2.101:0/629853055 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6fac0052e0 con 0x7f6fc40b7db0 2021-02-10T11:25:24.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.102 7f6fbbfff700 1 --2- 172.21.2.101:0/629853055 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6fa80186c0 0x7f6fa801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:24.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.102 7f6fbbfff700 1 -- 172.21.2.101:0/629853055 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6fac0326d0 con 0x7f6fc40b7db0 2021-02-10T11:25:24.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.102 7f6fc2d73700 1 --2- 172.21.2.101:0/629853055 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6fa80186c0 0x7f6fa801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:24.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.103 7f6fc2d73700 1 --2- 172.21.2.101:0/629853055 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6fa80186c0 0x7f6fa801abc0 crc :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:24.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.133 7f6fc97b7700 1 -- 172.21.2.101:0/629853055 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc4105b30 con 0x7f6fc40b7db0 2021-02-10T11:25:24.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.135 7f6fbbfff700 1 -- 172.21.2.101:0/629853055 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6fac006340 con 0x7f6fc40b7db0 2021-02-10T11:25:24.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.263 7f6fc97b7700 1 -- 172.21.2.101:0/629853055 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839307", "target": ["mgr", ""]}) v1 -- 0x7f6fc40f0b50 con 0x7f6fa80186c0 2021-02-10T11:25:24.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.266 7f6fbbfff700 1 -- 172.21.2.101:0/629853055 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f6fb4006a30 con 0x7f6fa80186c0 2021-02-10T11:25:24.274 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839307/c001e26c-0924-43ac-bcbf-a3d4f283907f 2021-02-10T11:25:24.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.267 7f6fc97b7700 1 -- 172.21.2.101:0/629853055 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6fa80186c0 msgr2=0x7f6fa801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:24.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.267 7f6fc97b7700 1 --2- 172.21.2.101:0/629853055 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6fa80186c0 0x7f6fa801abc0 crc :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:24.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.267 7f6fc97b7700 1 -- 172.21.2.101:0/629853055 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6fc40b7db0 msgr2=0x7f6fc40b8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:24.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.267 7f6fc97b7700 1 --2- 172.21.2.101:0/629853055 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6fc40b7db0 0x7f6fc40b8260 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f6fac006ed0 tx=0x7f6fac00b8c0).stop 2021-02-10T11:25:24.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.268 7f6fc97b7700 1 -- 172.21.2.101:0/629853055 shutdown_connections 2021-02-10T11:25:24.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.268 7f6fc97b7700 1 --2- 172.21.2.101:0/629853055 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6fa80186c0 0x7f6fa801abc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:24.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.268 7f6fc97b7700 1 --2- 172.21.2.101:0/629853055 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6fc40b8880 0x7f6fc41a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:24.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.268 7f6fc97b7700 1 --2- 172.21.2.101:0/629853055 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6fc40b8c80 0x7f6fc41a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:24.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.268 7f6fc97b7700 1 --2- 172.21.2.101:0/629853055 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6fc40b7db0 0x7f6fc40b8260 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:24.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.268 7f6fc97b7700 1 -- 172.21.2.101:0/629853055 >> 172.21.2.101:0/629853055 conn(0x7f6fc4094e10 msgr2=0x7f6fc40a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:24.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.268 7f6fc97b7700 1 -- 172.21.2.101:0/629853055 shutdown_connections 2021-02-10T11:25:24.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:24.268 7f6fc97b7700 1 -- 172.21.2.101:0/629853055 wait complete. 2021-02-10T11:25:24.284 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839307 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839307/c001e26c-0924-43ac-bcbf-a3d4f283907f 2021-02-10T11:25:24.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:24.285 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:24.285 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:24.620 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:24.620 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:24.620 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839307/c001e26c-0924-43ac-bcbf-a3d4f283907f/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:24.999 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:24.999 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:24.999 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239441 s, 43.8 MB/s 2021-02-10T11:25:25.000 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:25.000 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:25.001 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:25.335 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:25.335 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:25.335 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839307/c001e26c-0924-43ac-bcbf-a3d4f283907f/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:25.710 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:25.710 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:25.710 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0229978 s, 45.6 MB/s 2021-02-10T11:25:25.711 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:25.711 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:25.711 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:26.047 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:26.047 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:26.047 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839307/c001e26c-0924-43ac-bcbf-a3d4f283907f/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:26.422 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:26.422 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:26.422 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240323 s, 43.6 MB/s 2021-02-10T11:25:26.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:26.423 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:26.423 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:26.759 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:26.759 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:26.759 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839307/c001e26c-0924-43ac-bcbf-a3d4f283907f/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:27.136 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:27.136 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:27.136 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245192 s, 42.8 MB/s 2021-02-10T11:25:27.137 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:27.137 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:27.138 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:27.473 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:27.473 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:27.473 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839307/c001e26c-0924-43ac-bcbf-a3d4f283907f/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:27.850 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:27.850 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:27.851 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0266518 s, 39.3 MB/s 2021-02-10T11:25:27.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:27.852 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:27.852 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:28.187 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:28.187 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:28.187 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839307/c001e26c-0924-43ac-bcbf-a3d4f283907f/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:28.561 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:28.562 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:28.562 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234112 s, 44.8 MB/s 2021-02-10T11:25:28.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:28.563 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:28.563 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:28.898 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:28.898 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:28.898 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839307/c001e26c-0924-43ac-bcbf-a3d4f283907f/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:29.270 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:29.271 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:29.271 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248858 s, 42.1 MB/s 2021-02-10T11:25:29.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:29.272 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:29.272 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:29.606 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:29.607 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:29.607 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839307/c001e26c-0924-43ac-bcbf-a3d4f283907f/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:29.985 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:29.985 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:29.985 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0258249 s, 40.6 MB/s 2021-02-10T11:25:29.986 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:29.986 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:29.986 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:30.321 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:30.321 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:30.321 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839307/c001e26c-0924-43ac-bcbf-a3d4f283907f/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:30.697 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:30.697 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:30.697 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244084 s, 43.0 MB/s 2021-02-10T11:25:30.698 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:30.698 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:30.698 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:31.033 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:31.034 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:31.034 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839307/c001e26c-0924-43ac-bcbf-a3d4f283907f/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:31.411 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:31.411 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:31.411 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246395 s, 42.6 MB/s 2021-02-10T11:25:31.412 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839308 2021-02-10T11:25:31.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.818 7fac518d1700 1 Processor -- start 2021-02-10T11:25:31.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.818 7fac518d1700 1 -- start start 2021-02-10T11:25:31.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.819 7fac518d1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fac4c1b0470 0x7fac4c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:31.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.819 7fac518d1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fac4c0ef610 0x7fac4c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:31.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.819 7fac518d1700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac4c0f0b50 con 0x7fac4c0f0030 2021-02-10T11:25:31.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.819 7fac518d1700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac4c0e52e0 con 0x7fac4c0ef610 2021-02-10T11:25:31.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.819 7fac518d1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac4c0e5490 con 0x7fac4c1b0470 2021-02-10T11:25:31.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.819 7fac4ad73700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fac4c0ef610 0x7fac4c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:31.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.819 7fac4bd75700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fac4c0f0030 0x7fac4c0f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48070/0 (socket says 172.21.2.101:48070) 2021-02-10T11:25:31.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.819 7fac4bd75700 1 -- 172.21.2.101:0/59801841 learned_addr learned my addr 172.21.2.101:0/59801841 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:25:31.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.819 7fac4ad73700 1 -- 172.21.2.101:0/59801841 >> v2:172.21.2.108:3300/0 conn(0x7fac4c1b0470 msgr2=0x7fac4c1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:25:31.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.819 7fac4ad73700 1 --2- 172.21.2.101:0/59801841 >> v2:172.21.2.108:3300/0 conn(0x7fac4c1b0470 0x7fac4c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:31.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.819 7fac4ad73700 1 -- 172.21.2.101:0/59801841 >> v1:172.21.2.101:6789/0 conn(0x7fac4c0f0030 legacy=0x7fac4c0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:31.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.820 7fac4ad73700 1 -- 172.21.2.101:0/59801841 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c0f0430 con 0x7fac4c0ef610 2021-02-10T11:25:31.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.819 7fac4a572700 1 -- 172.21.2.101:0/59801841 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2661635872 0 0) 0x7fac3c000c10 con 0x7fac4c0f0030 2021-02-10T11:25:31.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.820 7fac4ad73700 1 --2- 172.21.2.101:0/59801841 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fac4c0ef610 0x7fac4c0efa10 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7fac440069d0 tx=0x7fac44009dc0).ready entity=mon.0 client_cookie=500b523a6e5b0d1d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:31.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.820 7fac4a572700 1 -- 172.21.2.101:0/59801841 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fac4400b510 con 0x7fac4c0ef610 2021-02-10T11:25:31.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.820 7fac4a572700 1 -- 172.21.2.101:0/59801841 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac44006ac0 con 0x7fac4c0ef610 2021-02-10T11:25:31.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.820 7fac4a572700 1 -- 172.21.2.101:0/59801841 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fac4400bb10 con 0x7fac4c0ef610 2021-02-10T11:25:31.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.820 7fac518d1700 1 -- 172.21.2.101:0/59801841 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fac4c0ef610 msgr2=0x7fac4c0efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:31.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.820 7fac518d1700 1 --2- 172.21.2.101:0/59801841 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fac4c0ef610 0x7fac4c0efa10 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7fac440069d0 tx=0x7fac44009dc0).stop 2021-02-10T11:25:31.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.820 7fac518d1700 1 -- 172.21.2.101:0/59801841 shutdown_connections 2021-02-10T11:25:31.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.820 7fac518d1700 1 --2- 172.21.2.101:0/59801841 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fac4c0ef610 0x7fac4c0efa10 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:31.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.820 7fac518d1700 1 --2- 172.21.2.101:0/59801841 >> v2:172.21.2.108:3300/0 conn(0x7fac4c1b0470 0x7fac4c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:31.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.820 7fac518d1700 1 -- 172.21.2.101:0/59801841 >> 172.21.2.101:0/59801841 conn(0x7fac4c0ce3e0 msgr2=0x7fac4c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:31.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.820 7fac518d1700 1 -- 172.21.2.101:0/59801841 shutdown_connections 2021-02-10T11:25:31.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.820 7fac518d1700 1 -- 172.21.2.101:0/59801841 wait complete. 2021-02-10T11:25:31.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.835 7fac518d1700 1 Processor -- start 2021-02-10T11:25:31.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.835 7fac518d1700 1 -- start start 2021-02-10T11:25:31.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.836 7fac518d1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fac4c0bf910 0x7fac4c0bfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:31.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.836 7fac518d1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fac4c0c0330 0x7fac4c0c0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:31.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.836 7fac518d1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fac4c0c13d0 0x7fac4c0c17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:31.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.836 7fac518d1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c109930 con 0x7fac4c0c13d0 2021-02-10T11:25:31.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.836 7fac518d1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c109b50 con 0x7fac4c0bf910 2021-02-10T11:25:31.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.836 7fac518d1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c0c1df0 con 0x7fac4c0c0330 2021-02-10T11:25:31.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.836 7fac4bd75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fac4c0c13d0 0x7fac4c0c17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:31.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.836 7fac4bd75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fac4c0c13d0 0x7fac4c0c17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47582/0 (socket says 172.21.2.101:47582) 2021-02-10T11:25:31.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.836 7fac4bd75700 1 -- 172.21.2.101:0/1247807890 learned_addr learned my addr 172.21.2.101:0/1247807890 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:25:31.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.836 7fac4bd75700 1 -- 172.21.2.101:0/1247807890 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fac4c0c0330 msgr2=0x7fac4c0c0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:31.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.836 7fac4bd75700 1 --2- 172.21.2.101:0/1247807890 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fac4c0c0330 0x7fac4c0c0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:31.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.836 7fac4bd75700 1 -- 172.21.2.101:0/1247807890 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fac4c0bf910 msgr2=0x7fac4c0bfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:31.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.836 7fac4bd75700 1 --2- 172.21.2.101:0/1247807890 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fac4c0bf910 0x7fac4c0bfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:31.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.836 7fac4bd75700 1 -- 172.21.2.101:0/1247807890 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c0c1fd0 con 0x7fac4c0c13d0 2021-02-10T11:25:31.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.836 7fac4bd75700 1 --2- 172.21.2.101:0/1247807890 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fac4c0c13d0 0x7fac4c0c17d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7fac3c00a590 tx=0x7fac3c00a930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:31.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.837 7fac33fff700 1 -- 172.21.2.101:0/1247807890 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fac3c00ad20 con 0x7fac4c0c13d0 2021-02-10T11:25:31.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.837 7fac33fff700 1 -- 172.21.2.101:0/1247807890 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3c0043a0 con 0x7fac4c0c13d0 2021-02-10T11:25:31.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.837 7fac518d1700 1 -- 172.21.2.101:0/1247807890 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac4c0c1fd0 con 0x7fac4c0c13d0 2021-02-10T11:25:31.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.837 7fac518d1700 1 -- 172.21.2.101:0/1247807890 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4c0c2560 con 0x7fac4c0c13d0 2021-02-10T11:25:31.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.837 7fac33fff700 1 -- 172.21.2.101:0/1247807890 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fac3c0048c0 con 0x7fac4c0c13d0 2021-02-10T11:25:31.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.837 7fac33fff700 1 -- 172.21.2.101:0/1247807890 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fac3c004890 con 0x7fac4c0c13d0 2021-02-10T11:25:31.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.837 7fac33fff700 1 --2- 172.21.2.101:0/1247807890 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fac2c0186c0 0x7fac2c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:31.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.837 7fac33fff700 1 -- 172.21.2.101:0/1247807890 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fac3c0325b0 con 0x7fac4c0c13d0 2021-02-10T11:25:31.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.838 7fac4b574700 1 --2- 172.21.2.101:0/1247807890 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fac2c0186c0 0x7fac2c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:31.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.838 7fac4b574700 1 --2- 172.21.2.101:0/1247807890 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fac2c0186c0 0x7fac2c01abc0 crc :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:31.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.869 7fac518d1700 1 -- 172.21.2.101:0/1247807890 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac4c0f0b50 con 0x7fac4c0c13d0 2021-02-10T11:25:31.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.872 7fac33fff700 1 -- 172.21.2.101:0/1247807890 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fac3c030d60 con 0x7fac4c0c13d0 2021-02-10T11:25:32.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:31.998 7fac518d1700 1 -- 172.21.2.101:0/1247807890 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839308", "target": ["mgr", ""]}) v1 -- 0x7fac4c0c2560 con 0x7fac2c0186c0 2021-02-10T11:25:32.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.024 7fac33fff700 1 -- 172.21.2.101:0/1247807890 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fac400009b0 con 0x7fac2c0186c0 2021-02-10T11:25:32.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.031 7fac518d1700 1 -- 172.21.2.101:0/1247807890 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fac2c0186c0 msgr2=0x7fac2c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:32.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.031 7fac518d1700 1 --2- 172.21.2.101:0/1247807890 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fac2c0186c0 0x7fac2c01abc0 crc :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:32.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.031 7fac518d1700 1 -- 172.21.2.101:0/1247807890 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fac4c0c13d0 msgr2=0x7fac4c0c17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:32.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.031 7fac518d1700 1 --2- 172.21.2.101:0/1247807890 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fac4c0c13d0 0x7fac4c0c17d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7fac3c00a590 tx=0x7fac3c00a930).stop 2021-02-10T11:25:32.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.031 7fac518d1700 1 -- 172.21.2.101:0/1247807890 shutdown_connections 2021-02-10T11:25:32.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.031 7fac518d1700 1 --2- 172.21.2.101:0/1247807890 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fac2c0186c0 0x7fac2c01abc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:32.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.031 7fac518d1700 1 --2- 172.21.2.101:0/1247807890 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fac4c0c0330 0x7fac4c0c0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:32.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.031 7fac518d1700 1 --2- 172.21.2.101:0/1247807890 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fac4c0c13d0 0x7fac4c0c17d0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:32.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.031 7fac518d1700 1 --2- 172.21.2.101:0/1247807890 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fac4c0bf910 0x7fac4c0bfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:32.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.031 7fac518d1700 1 -- 172.21.2.101:0/1247807890 >> 172.21.2.101:0/1247807890 conn(0x7fac4c0bf4e0 msgr2=0x7fac4c191190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:32.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.032 7fac518d1700 1 -- 172.21.2.101:0/1247807890 shutdown_connections 2021-02-10T11:25:32.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.032 7fac518d1700 1 -- 172.21.2.101:0/1247807890 wait complete. 2021-02-10T11:25:32.047 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839308 2021-02-10T11:25:32.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.454 7fdcc4470700 1 Processor -- start 2021-02-10T11:25:32.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.454 7fdcc4470700 1 -- start start 2021-02-10T11:25:32.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.455 7fdcc4470700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdcbc1b0470 0x7fdcbc1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:32.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.455 7fdcc4470700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdcbc0f20e0 0x7fdcbc0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:32.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.455 7fdcc4470700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcbc0f0b50 con 0x7fdcbc0ef610 2021-02-10T11:25:32.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.455 7fdcc4470700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcbc0f0820 con 0x7fdcbc1b0470 2021-02-10T11:25:32.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.455 7fdcc4470700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc0f24e0 con 0x7fdcbc0f20e0 2021-02-10T11:25:32.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.455 7fdcc228c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdcbc1b0470 0x7fdcbc1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:32.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.455 7fdcc228c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdcbc1b0470 0x7fdcbc1b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47586/0 (socket says 172.21.2.101:47586) 2021-02-10T11:25:32.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.455 7fdcc228c700 1 -- 172.21.2.101:0/1153096974 learned_addr learned my addr 172.21.2.101:0/1153096974 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:25:32.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.455 7fdcc2a8d700 1 --2- 172.21.2.101:0/1153096974 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdcbc0f20e0 0x7fdcbc0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:32.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.455 7fdcc228c700 1 -- 172.21.2.101:0/1153096974 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdcbc0f20e0 msgr2=0x7fdcbc0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:25:32.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.455 7fdcc228c700 1 --2- 172.21.2.101:0/1153096974 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdcbc0f20e0 0x7fdcbc0f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:32.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.455 7fdcc228c700 1 -- 172.21.2.101:0/1153096974 >> v1:172.21.2.108:6789/0 conn(0x7fdcbc0ef610 legacy=0x7fdcbc0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:32.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.455 7fdcc228c700 1 -- 172.21.2.101:0/1153096974 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcbc0e52e0 con 0x7fdcbc1b0470 2021-02-10T11:25:32.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.456 7fdcc228c700 1 --2- 172.21.2.101:0/1153096974 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdcbc1b0470 0x7fdcbc1b0870 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7fdcb8006700 tx=0x7fdcb8009b80).ready entity=mon.0 client_cookie=26b26c77fd156dae server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:32.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.456 7fdcc128a700 1 -- 172.21.2.101:0/1153096974 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdcb800c0c0 con 0x7fdcbc1b0470 2021-02-10T11:25:32.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.456 7fdcc128a700 1 -- 172.21.2.101:0/1153096974 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb800c340 con 0x7fdcbc1b0470 2021-02-10T11:25:32.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.456 7fdcc128a700 1 -- 172.21.2.101:0/1153096974 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdcb800c860 con 0x7fdcbc1b0470 2021-02-10T11:25:32.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.456 7fdcc4470700 1 -- 172.21.2.101:0/1153096974 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdcbc1b0470 msgr2=0x7fdcbc1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:32.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.456 7fdcc4470700 1 --2- 172.21.2.101:0/1153096974 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdcbc1b0470 0x7fdcbc1b0870 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7fdcb8006700 tx=0x7fdcb8009b80).stop 2021-02-10T11:25:32.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.457 7fdcc4470700 1 -- 172.21.2.101:0/1153096974 shutdown_connections 2021-02-10T11:25:32.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.457 7fdcc4470700 1 --2- 172.21.2.101:0/1153096974 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdcbc0f20e0 0x7fdcbc0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:32.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.457 7fdcc4470700 1 --2- 172.21.2.101:0/1153096974 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdcbc1b0470 0x7fdcbc1b0870 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:32.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.457 7fdcc4470700 1 -- 172.21.2.101:0/1153096974 >> 172.21.2.101:0/1153096974 conn(0x7fdcbc0ce3e0 msgr2=0x7fdcbc0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:32.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.457 7fdcc4470700 1 -- 172.21.2.101:0/1153096974 shutdown_connections 2021-02-10T11:25:32.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.457 7fdcc4470700 1 -- 172.21.2.101:0/1153096974 wait complete. 2021-02-10T11:25:32.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.471 7fdcc4470700 1 Processor -- start 2021-02-10T11:25:32.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.471 7fdcc4470700 1 -- start start 2021-02-10T11:25:32.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.472 7fdcc4470700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdcbc0a0420 0x7fdcbc0a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:32.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.472 7fdcc4470700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdcbc189d00 0x7fdcbc18a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:32.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.472 7fdcc4470700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdcbc18a720 0x7fdcbc0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:32.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.472 7fdcc4470700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc189800 con 0x7fdcbc18a720 2021-02-10T11:25:32.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.472 7fdcc4470700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc18ab20 con 0x7fdcbc189d00 2021-02-10T11:25:32.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.472 7fdcc4470700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc0a1140 con 0x7fdcbc0a0420 2021-02-10T11:25:32.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.472 7fdcc2a8d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdcbc18a720 0x7fdcbc0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:32.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.472 7fdcc2a8d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdcbc18a720 0x7fdcbc0b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47592/0 (socket says 172.21.2.101:47592) 2021-02-10T11:25:32.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.472 7fdcc2a8d700 1 -- 172.21.2.101:0/2780071956 learned_addr learned my addr 172.21.2.101:0/2780071956 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:25:32.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.472 7fdcc228c700 1 --2- 172.21.2.101:0/2780071956 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdcbc0a0420 0x7fdcbc0a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:32.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.472 7fdcc1a8b700 1 --2- 172.21.2.101:0/2780071956 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdcbc189d00 0x7fdcbc18a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:32.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.472 7fdcc2a8d700 1 -- 172.21.2.101:0/2780071956 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdcbc0a0420 msgr2=0x7fdcbc0a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:32.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.472 7fdcc2a8d700 1 --2- 172.21.2.101:0/2780071956 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdcbc0a0420 0x7fdcbc0a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:32.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.472 7fdcc2a8d700 1 -- 172.21.2.101:0/2780071956 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdcbc189d00 msgr2=0x7fdcbc18a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:32.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.473 7fdcc2a8d700 1 --2- 172.21.2.101:0/2780071956 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdcbc189d00 0x7fdcbc18a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:32.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.473 7fdcc2a8d700 1 -- 172.21.2.101:0/2780071956 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcbc0b8200 con 0x7fdcbc18a720 2021-02-10T11:25:32.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.473 7fdcc2a8d700 1 --2- 172.21.2.101:0/2780071956 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdcbc18a720 0x7fdcbc0b7be0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7fdcb40099a0 tx=0x7fdcb400bc70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:32.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.473 7fdcaeffd700 1 -- 172.21.2.101:0/2780071956 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdcb400bdb0 con 0x7fdcbc18a720 2021-02-10T11:25:32.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.473 7fdcaeffd700 1 -- 172.21.2.101:0/2780071956 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb4003030 con 0x7fdcbc18a720 2021-02-10T11:25:32.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.473 7fdcc4470700 1 -- 172.21.2.101:0/2780071956 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcbc0b8200 con 0x7fdcbc18a720 2021-02-10T11:25:32.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.473 7fdcc4470700 1 -- 172.21.2.101:0/2780071956 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcbc0b8790 con 0x7fdcbc18a720 2021-02-10T11:25:32.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.473 7fdcaeffd700 1 -- 172.21.2.101:0/2780071956 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdcb40013e0 con 0x7fdcbc18a720 2021-02-10T11:25:32.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.474 7fdcaeffd700 1 -- 172.21.2.101:0/2780071956 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdcb40011f0 con 0x7fdcbc18a720 2021-02-10T11:25:32.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.474 7fdcaeffd700 1 --2- 172.21.2.101:0/2780071956 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdca4018730 0x7fdca401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:32.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.474 7fdcaeffd700 1 -- 172.21.2.101:0/2780071956 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fdcb4032b10 con 0x7fdcbc18a720 2021-02-10T11:25:32.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.474 7fdcc228c700 1 --2- 172.21.2.101:0/2780071956 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdca4018730 0x7fdca401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:32.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.474 7fdcc228c700 1 --2- 172.21.2.101:0/2780071956 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdca4018730 0x7fdca401ac10 crc :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:32.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.505 7fdcc4470700 1 -- 172.21.2.101:0/2780071956 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcbc105ae0 con 0x7fdcbc18a720 2021-02-10T11:25:32.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.507 7fdcaeffd700 1 -- 172.21.2.101:0/2780071956 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdcb4003010 con 0x7fdcbc18a720 2021-02-10T11:25:32.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.636 7fdcc4470700 1 -- 172.21.2.101:0/2780071956 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839308", "target": ["mgr", ""]}) v1 -- 0x7fdcbc0b9020 con 0x7fdca4018730 2021-02-10T11:25:32.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.639 7fdcaeffd700 1 -- 172.21.2.101:0/2780071956 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fdcb8006430 con 0x7fdca4018730 2021-02-10T11:25:32.651 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839308/fa75cb60-e966-415c-8c70-57cabd34618e 2021-02-10T11:25:32.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.644 7fdcc4470700 1 -- 172.21.2.101:0/2780071956 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdca4018730 msgr2=0x7fdca401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:32.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.644 7fdcc4470700 1 --2- 172.21.2.101:0/2780071956 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdca4018730 0x7fdca401ac10 crc :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:32.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.644 7fdcc4470700 1 -- 172.21.2.101:0/2780071956 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdcbc18a720 msgr2=0x7fdcbc0b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:32.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.644 7fdcc4470700 1 --2- 172.21.2.101:0/2780071956 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdcbc18a720 0x7fdcbc0b7be0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7fdcb40099a0 tx=0x7fdcb400bc70).stop 2021-02-10T11:25:32.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.644 7fdcc4470700 1 -- 172.21.2.101:0/2780071956 shutdown_connections 2021-02-10T11:25:32.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.644 7fdcc4470700 1 --2- 172.21.2.101:0/2780071956 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdcbc18a720 0x7fdcbc0b7be0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:32.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.644 7fdcc4470700 1 --2- 172.21.2.101:0/2780071956 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdcbc189d00 0x7fdcbc18a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:32.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.644 7fdcc4470700 1 --2- 172.21.2.101:0/2780071956 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdca4018730 0x7fdca401ac10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:32.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.644 7fdcc4470700 1 --2- 172.21.2.101:0/2780071956 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdcbc0a0420 0x7fdcbc0a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:32.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.644 7fdcc4470700 1 -- 172.21.2.101:0/2780071956 >> 172.21.2.101:0/2780071956 conn(0x7fdcbc094e10 msgr2=0x7fdcbc0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:32.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.644 7fdcc4470700 1 -- 172.21.2.101:0/2780071956 shutdown_connections 2021-02-10T11:25:32.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:32.644 7fdcc4470700 1 -- 172.21.2.101:0/2780071956 wait complete. 2021-02-10T11:25:32.661 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839308 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839308/fa75cb60-e966-415c-8c70-57cabd34618e 2021-02-10T11:25:32.661 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:32.661 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:32.661 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:32.997 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:32.997 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:32.998 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839308/fa75cb60-e966-415c-8c70-57cabd34618e/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:33.380 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:33.380 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:33.380 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247855 s, 42.3 MB/s 2021-02-10T11:25:33.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:33.381 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:33.381 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:33.716 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:33.716 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:33.716 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839308/fa75cb60-e966-415c-8c70-57cabd34618e/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:34.088 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:34.088 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:34.088 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235211 s, 44.6 MB/s 2021-02-10T11:25:34.089 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:34.089 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:34.089 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:34.424 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:34.425 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:34.425 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839308/fa75cb60-e966-415c-8c70-57cabd34618e/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:34.801 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:34.801 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:34.801 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247161 s, 42.4 MB/s 2021-02-10T11:25:34.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:34.803 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:34.803 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:35.138 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:35.138 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:35.139 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839308/fa75cb60-e966-415c-8c70-57cabd34618e/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:35.514 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:35.514 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:35.514 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245044 s, 42.8 MB/s 2021-02-10T11:25:35.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:35.515 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:35.515 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:35.850 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:35.850 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:35.851 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839308/fa75cb60-e966-415c-8c70-57cabd34618e/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:36.225 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:36.225 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:36.225 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0229527 s, 45.7 MB/s 2021-02-10T11:25:36.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:36.226 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:36.227 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:36.563 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:36.563 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:36.563 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839308/fa75cb60-e966-415c-8c70-57cabd34618e/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:36.941 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:36.941 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:36.941 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0254499 s, 41.2 MB/s 2021-02-10T11:25:36.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:36.942 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:36.942 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:37.278 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:37.278 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:37.278 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839308/fa75cb60-e966-415c-8c70-57cabd34618e/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:37.657 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:37.658 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:37.658 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0264715 s, 39.6 MB/s 2021-02-10T11:25:37.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:37.659 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:37.659 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:37.994 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:37.994 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:37.994 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839308/fa75cb60-e966-415c-8c70-57cabd34618e/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:38.370 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:38.370 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:38.371 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244124 s, 43.0 MB/s 2021-02-10T11:25:38.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:38.372 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:38.372 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:38.707 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:38.707 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:38.708 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839308/fa75cb60-e966-415c-8c70-57cabd34618e/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:39.082 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:39.082 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:39.082 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245328 s, 42.7 MB/s 2021-02-10T11:25:39.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:39.083 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:39.084 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:39.420 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:39.420 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:39.420 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839308/fa75cb60-e966-415c-8c70-57cabd34618e/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:39.797 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:39.797 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:39.797 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251874 s, 41.6 MB/s 2021-02-10T11:25:39.798 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839309 2021-02-10T11:25:40.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.204 7fc0f8070700 1 Processor -- start 2021-02-10T11:25:40.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.204 7fc0f8070700 1 -- start start 2021-02-10T11:25:40.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.205 7fc0f8070700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0f01b0470 0x7fc0f01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:40.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.205 7fc0f8070700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc0f00ef610 0x7fc0f00efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:40.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.205 7fc0f8070700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc0f00f0030 0x7fc0f00f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:40.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.205 7fc0f8070700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0f00e5ea0 con 0x7fc0f00ef610 2021-02-10T11:25:40.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.205 7fc0f8070700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0f00f0b50 con 0x7fc0f00f0030 2021-02-10T11:25:40.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.205 7fc0f8070700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f00e52e0 con 0x7fc0f01b0470 2021-02-10T11:25:40.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.205 7fc0f568b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc0f00ef610 0x7fc0f00efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:40.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.205 7fc0f568b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc0f00ef610 0x7fc0f00efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47598/0 (socket says 172.21.2.101:47598) 2021-02-10T11:25:40.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.205 7fc0f568b700 1 -- 172.21.2.101:0/3409783226 learned_addr learned my addr 172.21.2.101:0/3409783226 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:25:40.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.205 7fc0f568b700 1 -- 172.21.2.101:0/3409783226 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0f01b0470 msgr2=0x7fc0f01b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:25:40.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.205 7fc0f568b700 1 --2- 172.21.2.101:0/3409783226 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0f01b0470 0x7fc0f01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:40.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.205 7fc0f568b700 1 -- 172.21.2.101:0/3409783226 >> v2:172.21.2.108:3300/0 conn(0x7fc0f00f0030 msgr2=0x7fc0f00f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:25:40.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.205 7fc0f568b700 1 --2- 172.21.2.101:0/3409783226 >> v2:172.21.2.108:3300/0 conn(0x7fc0f00f0030 0x7fc0f00f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:40.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.205 7fc0f568b700 1 -- 172.21.2.101:0/3409783226 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f00f10d0 con 0x7fc0f00ef610 2021-02-10T11:25:40.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.205 7fc0f568b700 1 --2- 172.21.2.101:0/3409783226 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0f00ef610 0x7fc0f00efa10 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7fc0e0006700 tx=0x7fc0e0009b00).ready entity=mon.0 client_cookie=33a5275f70e3f861 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:40.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.206 7fc0f4e8a700 1 -- 172.21.2.101:0/3409783226 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc0e000c0c0 con 0x7fc0f00ef610 2021-02-10T11:25:40.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.206 7fc0f4e8a700 1 -- 172.21.2.101:0/3409783226 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e000c340 con 0x7fc0f00ef610 2021-02-10T11:25:40.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.206 7fc0f4e8a700 1 -- 172.21.2.101:0/3409783226 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc0e000c7a0 con 0x7fc0f00ef610 2021-02-10T11:25:40.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.206 7fc0f8070700 1 -- 172.21.2.101:0/3409783226 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0f00ef610 msgr2=0x7fc0f00efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:40.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.206 7fc0f8070700 1 --2- 172.21.2.101:0/3409783226 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0f00ef610 0x7fc0f00efa10 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7fc0e0006700 tx=0x7fc0e0009b00).stop 2021-02-10T11:25:40.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.206 7fc0f8070700 1 -- 172.21.2.101:0/3409783226 shutdown_connections 2021-02-10T11:25:40.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.206 7fc0f8070700 1 --2- 172.21.2.101:0/3409783226 >> v2:172.21.2.108:3300/0 conn(0x7fc0f00f0030 0x7fc0f00f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:40.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.206 7fc0f8070700 1 --2- 172.21.2.101:0/3409783226 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0f00ef610 0x7fc0f00efa10 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:40.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.206 7fc0f8070700 1 --2- 172.21.2.101:0/3409783226 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0f01b0470 0x7fc0f01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:40.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.206 7fc0f8070700 1 -- 172.21.2.101:0/3409783226 >> 172.21.2.101:0/3409783226 conn(0x7fc0f00ce3e0 msgr2=0x7fc0f00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:40.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.207 7fc0f8070700 1 -- 172.21.2.101:0/3409783226 shutdown_connections 2021-02-10T11:25:40.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.207 7fc0f8070700 1 -- 172.21.2.101:0/3409783226 wait complete. 2021-02-10T11:25:40.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.221 7fc0f8070700 1 Processor -- start 2021-02-10T11:25:40.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.221 7fc0f8070700 1 -- start start 2021-02-10T11:25:40.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.221 7fc0f8070700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0f0190920 0x7fc0f008f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:40.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.221 7fc0f8070700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0f0190d20 0x7fc0f008fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:40.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.221 7fc0f8070700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0f0193270 0x7fc0f0193670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:40.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.221 7fc0f8070700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f0090470 con 0x7fc0f0193270 2021-02-10T11:25:40.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.221 7fc0f8070700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f0193c90 con 0x7fc0f0190d20 2021-02-10T11:25:40.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.221 7fc0f8070700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f0193e70 con 0x7fc0f0190920 2021-02-10T11:25:40.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.222 7fc0f668d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0f0193270 0x7fc0f0193670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:40.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.222 7fc0f668d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0f0193270 0x7fc0f0193670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47608/0 (socket says 172.21.2.101:47608) 2021-02-10T11:25:40.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.222 7fc0f668d700 1 -- 172.21.2.101:0/1352774359 learned_addr learned my addr 172.21.2.101:0/1352774359 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:25:40.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.222 7fc0f5e8c700 1 --2- 172.21.2.101:0/1352774359 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0f0190920 0x7fc0f008f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:40.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.222 7fc0f568b700 1 --2- 172.21.2.101:0/1352774359 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0f0190d20 0x7fc0f008fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:40.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.222 7fc0f668d700 1 -- 172.21.2.101:0/1352774359 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0f0190920 msgr2=0x7fc0f008f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:40.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.222 7fc0f668d700 1 --2- 172.21.2.101:0/1352774359 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0f0190920 0x7fc0f008f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:40.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.222 7fc0f668d700 1 -- 172.21.2.101:0/1352774359 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0f0190d20 msgr2=0x7fc0f008fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:40.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.222 7fc0f668d700 1 --2- 172.21.2.101:0/1352774359 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0f0190d20 0x7fc0f008fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:40.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.222 7fc0f668d700 1 -- 172.21.2.101:0/1352774359 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f0194090 con 0x7fc0f0193270 2021-02-10T11:25:40.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.223 7fc0f668d700 1 --2- 172.21.2.101:0/1352774359 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0f0193270 0x7fc0f0193670 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7fc0e800b8c0 tx=0x7fc0e800bc60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:40.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.223 7fc0e67fc700 1 -- 172.21.2.101:0/1352774359 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc0e8005af0 con 0x7fc0f0193270 2021-02-10T11:25:40.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.223 7fc0e67fc700 1 -- 172.21.2.101:0/1352774359 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e8005df0 con 0x7fc0f0193270 2021-02-10T11:25:40.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.223 7fc0e67fc700 1 -- 172.21.2.101:0/1352774359 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc0e8006220 con 0x7fc0f0193270 2021-02-10T11:25:40.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.223 7fc0f8070700 1 -- 172.21.2.101:0/1352774359 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f0194090 con 0x7fc0f0193270 2021-02-10T11:25:40.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.223 7fc0f8070700 1 -- 172.21.2.101:0/1352774359 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f0194090 con 0x7fc0f0193270 2021-02-10T11:25:40.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.223 7fc0e67fc700 1 -- 172.21.2.101:0/1352774359 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc0e8005870 con 0x7fc0f0193270 2021-02-10T11:25:40.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.224 7fc0e67fc700 1 --2- 172.21.2.101:0/1352774359 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0d80186c0 0x7fc0d801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:40.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.224 7fc0e67fc700 1 -- 172.21.2.101:0/1352774359 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc0e8032920 con 0x7fc0f0193270 2021-02-10T11:25:40.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.224 7fc0f5e8c700 1 --2- 172.21.2.101:0/1352774359 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0d80186c0 0x7fc0d801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:40.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.224 7fc0f5e8c700 1 --2- 172.21.2.101:0/1352774359 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0d80186c0 0x7fc0d801abc0 crc :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:40.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.255 7fc0f8070700 1 -- 172.21.2.101:0/1352774359 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f0105af0 con 0x7fc0f0193270 2021-02-10T11:25:40.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.257 7fc0e67fc700 1 -- 172.21.2.101:0/1352774359 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc0e800bda0 con 0x7fc0f0193270 2021-02-10T11:25:40.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.385 7fc0f8070700 1 -- 172.21.2.101:0/1352774359 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839309", "target": ["mgr", ""]}) v1 -- 0x7fc0f0105af0 con 0x7fc0d80186c0 2021-02-10T11:25:40.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.411 7fc0e67fc700 1 -- 172.21.2.101:0/1352774359 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fc0ec006550 con 0x7fc0d80186c0 2021-02-10T11:25:40.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.418 7fc0f8070700 1 -- 172.21.2.101:0/1352774359 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0d80186c0 msgr2=0x7fc0d801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:40.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.418 7fc0f8070700 1 --2- 172.21.2.101:0/1352774359 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0d80186c0 0x7fc0d801abc0 crc :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:40.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.418 7fc0f8070700 1 -- 172.21.2.101:0/1352774359 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0f0193270 msgr2=0x7fc0f0193670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:40.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.418 7fc0f8070700 1 --2- 172.21.2.101:0/1352774359 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0f0193270 0x7fc0f0193670 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7fc0e800b8c0 tx=0x7fc0e800bc60).stop 2021-02-10T11:25:40.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.418 7fc0f8070700 1 -- 172.21.2.101:0/1352774359 shutdown_connections 2021-02-10T11:25:40.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.418 7fc0f8070700 1 --2- 172.21.2.101:0/1352774359 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0f0193270 0x7fc0f0193670 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:40.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.418 7fc0f8070700 1 --2- 172.21.2.101:0/1352774359 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0f0190d20 0x7fc0f008fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:40.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.418 7fc0f8070700 1 --2- 172.21.2.101:0/1352774359 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0d80186c0 0x7fc0d801abc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:40.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.418 7fc0f8070700 1 --2- 172.21.2.101:0/1352774359 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0f0190920 0x7fc0f008f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:40.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.418 7fc0f8070700 1 -- 172.21.2.101:0/1352774359 >> 172.21.2.101:0/1352774359 conn(0x7fc0f00bf4e0 msgr2=0x7fc0f00be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:40.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.418 7fc0f8070700 1 -- 172.21.2.101:0/1352774359 shutdown_connections 2021-02-10T11:25:40.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.418 7fc0f8070700 1 -- 172.21.2.101:0/1352774359 wait complete. 2021-02-10T11:25:40.435 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839309 2021-02-10T11:25:40.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.842 7fc22b0d5700 1 Processor -- start 2021-02-10T11:25:40.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.842 7fc22b0d5700 1 -- start start 2021-02-10T11:25:40.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.843 7fc22b0d5700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc2241b0470 0x7fc2241b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:40.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.843 7fc22b0d5700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc2240ef610 0x7fc2240efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:40.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.843 7fc22b0d5700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2240f0b50 con 0x7fc2240f0030 2021-02-10T11:25:40.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.843 7fc22b0d5700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2240e52e0 con 0x7fc2241b0470 2021-02-10T11:25:40.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.843 7fc22b0d5700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2240e5490 con 0x7fc2240ef610 2021-02-10T11:25:40.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.843 7fc228ef1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc2241b0470 0x7fc2241b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:40.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.843 7fc228ef1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc2241b0470 0x7fc2241b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47614/0 (socket says 172.21.2.101:47614) 2021-02-10T11:25:40.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.843 7fc228ef1700 1 -- 172.21.2.101:0/785588724 learned_addr learned my addr 172.21.2.101:0/785588724 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:25:40.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.843 7fc223fff700 1 --2- 172.21.2.101:0/785588724 >> v2:172.21.2.108:3300/0 conn(0x7fc2240ef610 0x7fc2240efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:40.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.844 7fc228ef1700 1 -- 172.21.2.101:0/785588724 >> v2:172.21.2.108:3300/0 conn(0x7fc2240ef610 msgr2=0x7fc2240efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:25:40.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.844 7fc228ef1700 1 --2- 172.21.2.101:0/785588724 >> v2:172.21.2.108:3300/0 conn(0x7fc2240ef610 0x7fc2240efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:40.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.844 7fc228ef1700 1 -- 172.21.2.101:0/785588724 >> v1:172.21.2.108:6789/0 conn(0x7fc2240f0030 legacy=0x7fc2240f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:40.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.844 7fc228ef1700 1 -- 172.21.2.101:0/785588724 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2240f0430 con 0x7fc2241b0470 2021-02-10T11:25:40.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.844 7fc228ef1700 1 --2- 172.21.2.101:0/785588724 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2241b0470 0x7fc2241b0870 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7fc20c006700 tx=0x7fc20c009b80).ready entity=mon.0 client_cookie=88caa3edbc784302 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:40.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.844 7fc2237fe700 1 -- 172.21.2.101:0/785588724 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc20c00c0c0 con 0x7fc2241b0470 2021-02-10T11:25:40.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.844 7fc2237fe700 1 -- 172.21.2.101:0/785588724 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20c00c400 con 0x7fc2241b0470 2021-02-10T11:25:40.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.844 7fc2237fe700 1 -- 172.21.2.101:0/785588724 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc20c00c8e0 con 0x7fc2241b0470 2021-02-10T11:25:40.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.844 7fc22b0d5700 1 -- 172.21.2.101:0/785588724 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2241b0470 msgr2=0x7fc2241b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:40.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.844 7fc22b0d5700 1 --2- 172.21.2.101:0/785588724 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2241b0470 0x7fc2241b0870 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7fc20c006700 tx=0x7fc20c009b80).stop 2021-02-10T11:25:40.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.845 7fc22b0d5700 1 -- 172.21.2.101:0/785588724 shutdown_connections 2021-02-10T11:25:40.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.845 7fc22b0d5700 1 --2- 172.21.2.101:0/785588724 >> v2:172.21.2.108:3300/0 conn(0x7fc2240ef610 0x7fc2240efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:40.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.845 7fc22b0d5700 1 --2- 172.21.2.101:0/785588724 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2241b0470 0x7fc2241b0870 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:40.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.845 7fc22b0d5700 1 -- 172.21.2.101:0/785588724 >> 172.21.2.101:0/785588724 conn(0x7fc2240ce3e0 msgr2=0x7fc2240e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:40.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.845 7fc22b0d5700 1 -- 172.21.2.101:0/785588724 shutdown_connections 2021-02-10T11:25:40.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.845 7fc22b0d5700 1 -- 172.21.2.101:0/785588724 wait complete. 2021-02-10T11:25:40.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.860 7fc22b0d5700 1 Processor -- start 2021-02-10T11:25:40.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.860 7fc22b0d5700 1 -- start start 2021-02-10T11:25:40.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.860 7fc22b0d5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc22408ece0 0x7fc2240a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:40.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.860 7fc22b0d5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2240a6860 0x7fc2240a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:40.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.860 7fc22b0d5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2240a7280 0x7fc2240a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:40.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.860 7fc22b0d5700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2240a5cc0 con 0x7fc2240a7280 2021-02-10T11:25:40.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.860 7fc22b0d5700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22408f0e0 con 0x7fc22408ece0 2021-02-10T11:25:40.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.860 7fc22b0d5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22408f2c0 con 0x7fc2240a6860 2021-02-10T11:25:40.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.860 7fc2296f2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2240a7280 0x7fc2240a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:40.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.860 7fc2296f2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2240a7280 0x7fc2240a4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47618/0 (socket says 172.21.2.101:47618) 2021-02-10T11:25:40.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.860 7fc2296f2700 1 -- 172.21.2.101:0/761823724 learned_addr learned my addr 172.21.2.101:0/761823724 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:25:40.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.861 7fc2296f2700 1 -- 172.21.2.101:0/761823724 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2240a6860 msgr2=0x7fc2240a6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:40.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.861 7fc2296f2700 1 --2- 172.21.2.101:0/761823724 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2240a6860 0x7fc2240a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:40.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.861 7fc2296f2700 1 -- 172.21.2.101:0/761823724 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc22408ece0 msgr2=0x7fc2240a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:40.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.861 7fc2296f2700 1 --2- 172.21.2.101:0/761823724 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc22408ece0 0x7fc2240a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:40.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.861 7fc2296f2700 1 -- 172.21.2.101:0/761823724 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2240a4bb0 con 0x7fc2240a7280 2021-02-10T11:25:40.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.861 7fc2296f2700 1 --2- 172.21.2.101:0/761823724 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2240a7280 0x7fc2240a4590 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7fc21c009940 tx=0x7fc21c00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:40.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.861 7fc2217fa700 1 -- 172.21.2.101:0/761823724 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc21c000ce0 con 0x7fc2240a7280 2021-02-10T11:25:40.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.861 7fc2217fa700 1 -- 172.21.2.101:0/761823724 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21c00c2f0 con 0x7fc2240a7280 2021-02-10T11:25:40.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.861 7fc2217fa700 1 -- 172.21.2.101:0/761823724 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc21c00c750 con 0x7fc2240a7280 2021-02-10T11:25:40.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.861 7fc22b0d5700 1 -- 172.21.2.101:0/761823724 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2240a4bb0 con 0x7fc2240a7280 2021-02-10T11:25:40.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.861 7fc22b0d5700 1 -- 172.21.2.101:0/761823724 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2240a5140 con 0x7fc2240a7280 2021-02-10T11:25:40.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.861 7fc2217fa700 1 -- 172.21.2.101:0/761823724 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc21c001220 con 0x7fc2240a7280 2021-02-10T11:25:40.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.862 7fc2217fa700 1 --2- 172.21.2.101:0/761823724 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc2080186c0 0x7fc20801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:40.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.862 7fc2217fa700 1 -- 172.21.2.101:0/761823724 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc21c0328f0 con 0x7fc2240a7280 2021-02-10T11:25:40.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.862 7fc228ef1700 1 --2- 172.21.2.101:0/761823724 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc2080186c0 0x7fc20801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:40.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.862 7fc228ef1700 1 --2- 172.21.2.101:0/761823724 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc2080186c0 0x7fc20801abc0 crc :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:40.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.893 7fc22b0d5700 1 -- 172.21.2.101:0/761823724 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22419d1a0 con 0x7fc2240a7280 2021-02-10T11:25:40.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:40.895 7fc2217fa700 1 -- 172.21.2.101:0/761823724 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc21c000cc0 con 0x7fc2240a7280 2021-02-10T11:25:41.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:41.023 7fc22b0d5700 1 -- 172.21.2.101:0/761823724 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839309", "target": ["mgr", ""]}) v1 -- 0x7fc2240a5cc0 con 0x7fc2080186c0 2021-02-10T11:25:41.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:41.025 7fc2217fa700 1 -- 172.21.2.101:0/761823724 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fc20c006060 con 0x7fc2080186c0 2021-02-10T11:25:41.034 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839309/cdc1145f-37a3-4c83-9baf-b57026711e9c 2021-02-10T11:25:41.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:41.027 7fc22b0d5700 1 -- 172.21.2.101:0/761823724 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc2080186c0 msgr2=0x7fc20801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:41.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:41.027 7fc22b0d5700 1 --2- 172.21.2.101:0/761823724 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc2080186c0 0x7fc20801abc0 crc :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:41.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:41.027 7fc22b0d5700 1 -- 172.21.2.101:0/761823724 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2240a7280 msgr2=0x7fc2240a4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:41.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:41.027 7fc22b0d5700 1 --2- 172.21.2.101:0/761823724 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2240a7280 0x7fc2240a4590 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7fc21c009940 tx=0x7fc21c00bcf0).stop 2021-02-10T11:25:41.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:41.027 7fc22b0d5700 1 -- 172.21.2.101:0/761823724 shutdown_connections 2021-02-10T11:25:41.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:41.027 7fc22b0d5700 1 --2- 172.21.2.101:0/761823724 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc2080186c0 0x7fc20801abc0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:41.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:41.027 7fc22b0d5700 1 --2- 172.21.2.101:0/761823724 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2240a6860 0x7fc2240a6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:41.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:41.027 7fc22b0d5700 1 --2- 172.21.2.101:0/761823724 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2240a7280 0x7fc2240a4590 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:41.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:41.027 7fc22b0d5700 1 --2- 172.21.2.101:0/761823724 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc22408ece0 0x7fc2240a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:41.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:41.027 7fc22b0d5700 1 -- 172.21.2.101:0/761823724 >> 172.21.2.101:0/761823724 conn(0x7fc22408ffe0 msgr2=0x7fc2240bf050 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:41.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:41.027 7fc22b0d5700 1 -- 172.21.2.101:0/761823724 shutdown_connections 2021-02-10T11:25:41.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:41.028 7fc22b0d5700 1 -- 172.21.2.101:0/761823724 wait complete. 2021-02-10T11:25:41.044 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839309 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839309/cdc1145f-37a3-4c83-9baf-b57026711e9c 2021-02-10T11:25:41.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:41.044 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:41.044 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:41.379 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:41.379 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:41.379 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839309/cdc1145f-37a3-4c83-9baf-b57026711e9c/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:41.760 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:41.761 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:41.761 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241852 s, 43.4 MB/s 2021-02-10T11:25:41.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:41.762 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:41.762 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:42.098 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:42.098 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:42.099 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839309/cdc1145f-37a3-4c83-9baf-b57026711e9c/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:42.474 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:42.474 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:42.474 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240349 s, 43.6 MB/s 2021-02-10T11:25:42.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:42.475 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:42.475 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:42.812 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:42.812 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:42.812 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839309/cdc1145f-37a3-4c83-9baf-b57026711e9c/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:43.188 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:43.188 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:43.189 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244342 s, 42.9 MB/s 2021-02-10T11:25:43.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:43.190 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:43.190 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:43.525 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:43.525 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:43.525 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839309/cdc1145f-37a3-4c83-9baf-b57026711e9c/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:43.897 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:43.897 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:43.897 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240309 s, 43.6 MB/s 2021-02-10T11:25:43.898 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:43.898 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:43.898 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:44.233 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:44.233 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:44.233 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839309/cdc1145f-37a3-4c83-9baf-b57026711e9c/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:44.608 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:44.608 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:44.608 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242243 s, 43.3 MB/s 2021-02-10T11:25:44.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:44.609 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:44.610 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:44.944 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:44.944 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:44.944 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839309/cdc1145f-37a3-4c83-9baf-b57026711e9c/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:45.320 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:45.320 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:45.320 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252705 s, 41.5 MB/s 2021-02-10T11:25:45.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:45.321 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:45.322 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:45.656 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:45.656 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:45.656 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839309/cdc1145f-37a3-4c83-9baf-b57026711e9c/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:46.032 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:46.032 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:46.032 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.02465 s, 42.5 MB/s 2021-02-10T11:25:46.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:46.033 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:46.033 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:46.368 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:46.368 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:46.369 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839309/cdc1145f-37a3-4c83-9baf-b57026711e9c/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:46.743 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:46.743 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:46.743 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0226953 s, 46.2 MB/s 2021-02-10T11:25:46.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:46.744 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:46.744 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:47.080 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:47.080 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:47.081 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839309/cdc1145f-37a3-4c83-9baf-b57026711e9c/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:47.454 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:47.454 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:47.454 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231729 s, 45.3 MB/s 2021-02-10T11:25:47.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:47.455 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:47.455 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:47.791 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:47.791 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:47.792 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839309/cdc1145f-37a3-4c83-9baf-b57026711e9c/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:48.169 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:48.169 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:48.169 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247745 s, 42.3 MB/s 2021-02-10T11:25:48.170 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839310 2021-02-10T11:25:48.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.577 7fb82d235700 1 Processor -- start 2021-02-10T11:25:48.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.577 7fb82d235700 1 -- start start 2021-02-10T11:25:48.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.577 7fb82d235700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb8281b0870 0x7fb8280f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:48.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.577 7fb82d235700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8281b0c70 con 0x7fb8280efe00 2021-02-10T11:25:48.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.577 7fb82d235700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8280f0b50 con 0x7fb8281b0470 2021-02-10T11:25:48.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.577 7fb82d235700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8280e5ea0 con 0x7fb8281b0870 2021-02-10T11:25:48.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.577 7fb82759e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb8280efe00 0x7fb8280f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48122/0 (socket says 172.21.2.101:48122) 2021-02-10T11:25:48.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.577 7fb82759e700 1 -- 172.21.2.101:0/1392613646 learned_addr learned my addr 172.21.2.101:0/1392613646 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:25:48.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.578 7fb82659c700 1 --2- 172.21.2.101:0/1392613646 >> v2:172.21.2.108:3300/0 conn(0x7fb8281b0870 0x7fb8280f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:48.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.578 7fb825d9b700 1 -- 172.21.2.101:0/1392613646 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525352443 0 0) 0x7fb820000f90 con 0x7fb8280efe00 2021-02-10T11:25:48.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.578 7fb825d9b700 1 -- 172.21.2.101:0/1392613646 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb814002fe0 con 0x7fb8280efe00 2021-02-10T11:25:48.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.578 7fb825d9b700 1 -- 172.21.2.101:0/1392613646 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 520256994 0 0) 0x7fb820001460 con 0x7fb8280efe00 2021-02-10T11:25:48.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.578 7fb82659c700 1 -- 172.21.2.101:0/1392613646 >> v1:172.21.2.108:6789/0 conn(0x7fb8281b0470 legacy=0x7fb8280ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:48.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.578 7fb82659c700 1 -- 172.21.2.101:0/1392613646 >> v1:172.21.2.101:6789/0 conn(0x7fb8280efe00 legacy=0x7fb8280f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:48.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.578 7fb82659c700 1 -- 172.21.2.101:0/1392613646 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8280e52e0 con 0x7fb8281b0870 2021-02-10T11:25:48.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.578 7fb82659c700 1 --2- 172.21.2.101:0/1392613646 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb8281b0870 0x7fb8280f20e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fb810006650 tx=0x7fb810009a80).ready entity=mon.1 client_cookie=a9326e59b70e6d1d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:48.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.579 7fb825d9b700 1 -- 172.21.2.101:0/1392613646 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb81000c0c0 con 0x7fb8281b0870 2021-02-10T11:25:48.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.579 7fb825d9b700 1 -- 172.21.2.101:0/1392613646 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81000c340 con 0x7fb8281b0870 2021-02-10T11:25:48.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.579 7fb825d9b700 1 -- 172.21.2.101:0/1392613646 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb81000c860 con 0x7fb8281b0870 2021-02-10T11:25:48.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.579 7fb82d235700 1 -- 172.21.2.101:0/1392613646 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb8281b0870 msgr2=0x7fb8280f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:48.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.579 7fb82d235700 1 --2- 172.21.2.101:0/1392613646 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb8281b0870 0x7fb8280f20e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fb810006650 tx=0x7fb810009a80).stop 2021-02-10T11:25:48.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.579 7fb82d235700 1 -- 172.21.2.101:0/1392613646 shutdown_connections 2021-02-10T11:25:48.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.579 7fb82d235700 1 --2- 172.21.2.101:0/1392613646 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb8281b0870 0x7fb8280f20e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:48.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.579 7fb82d235700 1 -- 172.21.2.101:0/1392613646 >> 172.21.2.101:0/1392613646 conn(0x7fb8280ce3e0 msgr2=0x7fb8280e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:48.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.580 7fb82d235700 1 -- 172.21.2.101:0/1392613646 shutdown_connections 2021-02-10T11:25:48.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.580 7fb82d235700 1 -- 172.21.2.101:0/1392613646 wait complete. 2021-02-10T11:25:48.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.594 7fb82d235700 1 Processor -- start 2021-02-10T11:25:48.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.594 7fb82d235700 1 -- start start 2021-02-10T11:25:48.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.594 7fb82d235700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb828190920 0x7fb82808f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:48.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.594 7fb82d235700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb828190d20 0x7fb82808fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:48.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.594 7fb82d235700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb828193270 0x7fb828193670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:48.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.594 7fb82d235700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb828090470 con 0x7fb828190920 2021-02-10T11:25:48.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.594 7fb82d235700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb828193c90 con 0x7fb828193270 2021-02-10T11:25:48.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.594 7fb82d235700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb828193e70 con 0x7fb828190d20 2021-02-10T11:25:48.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.594 7fb826d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb828190920 0x7fb82808f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:48.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.594 7fb826d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb828190920 0x7fb82808f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47630/0 (socket says 172.21.2.101:47630) 2021-02-10T11:25:48.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.594 7fb826d9d700 1 -- 172.21.2.101:0/2984557082 learned_addr learned my addr 172.21.2.101:0/2984557082 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:25:48.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.595 7fb826d9d700 1 -- 172.21.2.101:0/2984557082 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb828190d20 msgr2=0x7fb82808fc20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:25:48.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.595 7fb826d9d700 1 --2- 172.21.2.101:0/2984557082 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb828190d20 0x7fb82808fc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:48.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.595 7fb826d9d700 1 -- 172.21.2.101:0/2984557082 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb828193270 msgr2=0x7fb828193670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:25:48.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.595 7fb826d9d700 1 --2- 172.21.2.101:0/2984557082 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb828193270 0x7fb828193670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:48.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.595 7fb826d9d700 1 -- 172.21.2.101:0/2984557082 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb828194090 con 0x7fb828190920 2021-02-10T11:25:48.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.595 7fb826d9d700 1 --2- 172.21.2.101:0/2984557082 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb828190920 0x7fb82808f600 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7fb81800b5b0 tx=0x7fb81800b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:48.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.595 7fb81f7fe700 1 -- 172.21.2.101:0/2984557082 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb81800bd40 con 0x7fb828190920 2021-02-10T11:25:48.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.595 7fb81f7fe700 1 -- 172.21.2.101:0/2984557082 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb818002080 con 0x7fb828190920 2021-02-10T11:25:48.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.595 7fb81f7fe700 1 -- 172.21.2.101:0/2984557082 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb818002560 con 0x7fb828190920 2021-02-10T11:25:48.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.595 7fb82d235700 1 -- 172.21.2.101:0/2984557082 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb828194090 con 0x7fb828190920 2021-02-10T11:25:48.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.595 7fb82d235700 1 -- 172.21.2.101:0/2984557082 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8281920b0 con 0x7fb828190920 2021-02-10T11:25:48.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.596 7fb81f7fe700 1 -- 172.21.2.101:0/2984557082 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb818002260 con 0x7fb828190920 2021-02-10T11:25:48.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.596 7fb81f7fe700 1 --2- 172.21.2.101:0/2984557082 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb80c0186c0 0x7fb80c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:48.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.596 7fb81f7fe700 1 -- 172.21.2.101:0/2984557082 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb8180324a0 con 0x7fb828190920 2021-02-10T11:25:48.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.596 7fb82659c700 1 --2- 172.21.2.101:0/2984557082 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb80c0186c0 0x7fb80c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:48.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.596 7fb82659c700 1 --2- 172.21.2.101:0/2984557082 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb80c0186c0 0x7fb80c01abc0 crc :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:48.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.627 7fb82d235700 1 -- 172.21.2.101:0/2984557082 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb828105af0 con 0x7fb828190920 2021-02-10T11:25:48.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.629 7fb81f7fe700 1 -- 172.21.2.101:0/2984557082 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb818030dd0 con 0x7fb828190920 2021-02-10T11:25:48.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.759 7fb82d235700 1 -- 172.21.2.101:0/2984557082 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839310", "target": ["mgr", ""]}) v1 -- 0x7fb828090470 con 0x7fb80c0186c0 2021-02-10T11:25:48.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.781 7fb81f7fe700 1 -- 172.21.2.101:0/2984557082 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fb810000d10 con 0x7fb80c0186c0 2021-02-10T11:25:48.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.792 7fb82d235700 1 -- 172.21.2.101:0/2984557082 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb80c0186c0 msgr2=0x7fb80c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:48.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.792 7fb82d235700 1 --2- 172.21.2.101:0/2984557082 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb80c0186c0 0x7fb80c01abc0 crc :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:48.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.792 7fb82d235700 1 -- 172.21.2.101:0/2984557082 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb828190920 msgr2=0x7fb82808f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:48.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.792 7fb82d235700 1 --2- 172.21.2.101:0/2984557082 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb828190920 0x7fb82808f600 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7fb81800b5b0 tx=0x7fb81800b950).stop 2021-02-10T11:25:48.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.792 7fb82d235700 1 -- 172.21.2.101:0/2984557082 shutdown_connections 2021-02-10T11:25:48.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.792 7fb82d235700 1 --2- 172.21.2.101:0/2984557082 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb80c0186c0 0x7fb80c01abc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:48.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.792 7fb82d235700 1 --2- 172.21.2.101:0/2984557082 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb828190d20 0x7fb82808fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:48.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.792 7fb82d235700 1 --2- 172.21.2.101:0/2984557082 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb828193270 0x7fb828193670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:48.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.792 7fb82d235700 1 --2- 172.21.2.101:0/2984557082 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb828190920 0x7fb82808f600 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:48.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.792 7fb82d235700 1 -- 172.21.2.101:0/2984557082 >> 172.21.2.101:0/2984557082 conn(0x7fb8280bf4e0 msgr2=0x7fb8280be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:48.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.793 7fb82d235700 1 -- 172.21.2.101:0/2984557082 shutdown_connections 2021-02-10T11:25:48.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:48.793 7fb82d235700 1 -- 172.21.2.101:0/2984557082 wait complete. 2021-02-10T11:25:48.809 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839310 2021-02-10T11:25:49.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.217 7f4c022cc700 1 Processor -- start 2021-02-10T11:25:49.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.217 7f4c022cc700 1 -- start start 2021-02-10T11:25:49.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.218 7f4c022cc700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4bfc1b0870 0x7f4bfc0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:49.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.218 7f4c022cc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bfc0efe00 0x7f4bfc0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:49.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.218 7f4c022cc700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bfc1b0c70 con 0x7f4bfc1b0470 2021-02-10T11:25:49.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.218 7f4c022cc700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bfc0f0b50 con 0x7f4bfc1b0870 2021-02-10T11:25:49.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.218 7f4c022cc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc0f0820 con 0x7f4bfc0efe00 2021-02-10T11:25:49.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.218 7f4c008e9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bfc0efe00 0x7f4bfc0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:49.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.218 7f4c008e9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bfc0efe00 0x7f4bfc0f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:40374/0 (socket says 172.21.2.101:40374) 2021-02-10T11:25:49.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.218 7f4bfb7fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4bfc1b0870 0x7f4bfc0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:49.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.219 7f4c008e9700 1 -- 172.21.2.101:0/973388516 learned_addr learned my addr 172.21.2.101:0/973388516 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:25:49.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.219 7f4c008e9700 1 -- 172.21.2.101:0/973388516 >> v2:172.21.2.108:3300/0 conn(0x7f4bfc1b0870 msgr2=0x7f4bfc0f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:25:49.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.219 7f4c008e9700 1 --2- 172.21.2.101:0/973388516 >> v2:172.21.2.108:3300/0 conn(0x7f4bfc1b0870 0x7f4bfc0f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:49.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.219 7f4c008e9700 1 -- 172.21.2.101:0/973388516 >> v1:172.21.2.108:6789/0 conn(0x7f4bfc1b0470 legacy=0x7f4bfc0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:49.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.219 7f4c008e9700 1 -- 172.21.2.101:0/973388516 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc0e52e0 con 0x7f4bfc0efe00 2021-02-10T11:25:49.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.219 7f4c008e9700 1 --2- 172.21.2.101:0/973388516 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bfc0efe00 0x7f4bfc0f0200 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f4bf4006aa0 tx=0x7f4bf4009f20).ready entity=mon.2 client_cookie=f71843a857a697cd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:49.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.219 7f4bfaffd700 1 -- 172.21.2.101:0/973388516 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4bf400b5c0 con 0x7f4bfc0efe00 2021-02-10T11:25:49.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.219 7f4bfaffd700 1 -- 172.21.2.101:0/973388516 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf4003030 con 0x7f4bfc0efe00 2021-02-10T11:25:49.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.219 7f4bfaffd700 1 -- 172.21.2.101:0/973388516 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4bf400b9f0 con 0x7f4bfc0efe00 2021-02-10T11:25:49.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.219 7f4c022cc700 1 -- 172.21.2.101:0/973388516 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bfc0efe00 msgr2=0x7f4bfc0f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:49.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.219 7f4c022cc700 1 --2- 172.21.2.101:0/973388516 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bfc0efe00 0x7f4bfc0f0200 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f4bf4006aa0 tx=0x7f4bf4009f20).stop 2021-02-10T11:25:49.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.220 7f4c022cc700 1 -- 172.21.2.101:0/973388516 shutdown_connections 2021-02-10T11:25:49.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.220 7f4c022cc700 1 --2- 172.21.2.101:0/973388516 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bfc0efe00 0x7f4bfc0f0200 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:49.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.220 7f4c022cc700 1 --2- 172.21.2.101:0/973388516 >> v2:172.21.2.108:3300/0 conn(0x7f4bfc1b0870 0x7f4bfc0f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:49.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.220 7f4c022cc700 1 -- 172.21.2.101:0/973388516 >> 172.21.2.101:0/973388516 conn(0x7f4bfc0ce3e0 msgr2=0x7f4bfc0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:49.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.220 7f4c022cc700 1 -- 172.21.2.101:0/973388516 shutdown_connections 2021-02-10T11:25:49.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.220 7f4c022cc700 1 -- 172.21.2.101:0/973388516 wait complete. 2021-02-10T11:25:49.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.235 7f4c022cc700 1 Processor -- start 2021-02-10T11:25:49.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.235 7f4c022cc700 1 -- start start 2021-02-10T11:25:49.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.235 7f4c022cc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bfc09f140 0x7f4bfc0c1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:49.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.235 7f4c022cc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bfc0c1f60 0x7f4bfc09e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:49.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.235 7f4c022cc700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4bfc09e920 0x7f4bfc0c2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:49.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.235 7f4c022cc700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc0c23e0 con 0x7f4bfc0c1f60 2021-02-10T11:25:49.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.235 7f4c022cc700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc09ed20 con 0x7f4bfc09e920 2021-02-10T11:25:49.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.235 7f4c022cc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc105d10 con 0x7f4bfc09f140 2021-02-10T11:25:49.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.236 7f4bfb7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bfc0c1f60 0x7f4bfc09e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:49.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.236 7f4bfb7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bfc0c1f60 0x7f4bfc09e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47648/0 (socket says 172.21.2.101:47648) 2021-02-10T11:25:49.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.236 7f4bfb7fe700 1 -- 172.21.2.101:0/2728117503 learned_addr learned my addr 172.21.2.101:0/2728117503 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:25:49.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.236 7f4bfbfff700 1 --2- 172.21.2.101:0/2728117503 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bfc09f140 0x7f4bfc0c1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:49.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.236 7f4c008e9700 1 --2- 172.21.2.101:0/2728117503 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4bfc09e920 0x7f4bfc0c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:49.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.236 7f4bfb7fe700 1 -- 172.21.2.101:0/2728117503 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bfc09f140 msgr2=0x7f4bfc0c1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:49.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.236 7f4bfb7fe700 1 --2- 172.21.2.101:0/2728117503 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bfc09f140 0x7f4bfc0c1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:49.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.236 7f4bfb7fe700 1 -- 172.21.2.101:0/2728117503 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4bfc09e920 msgr2=0x7f4bfc0c2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:49.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.236 7f4bfb7fe700 1 --2- 172.21.2.101:0/2728117503 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4bfc09e920 0x7f4bfc0c2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:49.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.236 7f4bfb7fe700 1 -- 172.21.2.101:0/2728117503 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc1a4910 con 0x7f4bfc0c1f60 2021-02-10T11:25:49.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.236 7f4c008e9700 1 --2- 172.21.2.101:0/2728117503 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4bfc09e920 0x7f4bfc0c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:25:49.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.236 7f4bfb7fe700 1 --2- 172.21.2.101:0/2728117503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bfc0c1f60 0x7f4bfc09e300 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f4bf0006f60 tx=0x7f4bf000b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:49.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.236 7f4bf8ff9700 1 -- 172.21.2.101:0/2728117503 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4bf00009b0 con 0x7f4bfc0c1f60 2021-02-10T11:25:49.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.237 7f4bf8ff9700 1 -- 172.21.2.101:0/2728117503 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf000bdf0 con 0x7f4bfc0c1f60 2021-02-10T11:25:49.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.237 7f4bf8ff9700 1 -- 172.21.2.101:0/2728117503 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4bf0002030 con 0x7f4bfc0c1f60 2021-02-10T11:25:49.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.237 7f4c022cc700 1 -- 172.21.2.101:0/2728117503 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bfc1a4910 con 0x7f4bfc0c1f60 2021-02-10T11:25:49.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.237 7f4c022cc700 1 -- 172.21.2.101:0/2728117503 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bfc0c3270 con 0x7f4bfc0c1f60 2021-02-10T11:25:49.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.237 7f4bf8ff9700 1 -- 172.21.2.101:0/2728117503 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4bf0006b00 con 0x7f4bfc0c1f60 2021-02-10T11:25:49.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.237 7f4bf8ff9700 1 --2- 172.21.2.101:0/2728117503 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bdc0186d0 0x7f4bdc01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:49.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.237 7f4bf8ff9700 1 -- 172.21.2.101:0/2728117503 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4bf0032820 con 0x7f4bfc0c1f60 2021-02-10T11:25:49.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.237 7f4bfbfff700 1 --2- 172.21.2.101:0/2728117503 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bdc0186d0 0x7f4bdc01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:49.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.237 7f4bfbfff700 1 --2- 172.21.2.101:0/2728117503 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bdc0186d0 0x7f4bdc01abd0 crc :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:49.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.267 7f4c022cc700 1 -- 172.21.2.101:0/2728117503 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bfc0f0b50 con 0x7f4bfc0c1f60 2021-02-10T11:25:49.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.270 7f4bf8ff9700 1 -- 172.21.2.101:0/2728117503 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4bf0000940 con 0x7f4bfc0c1f60 2021-02-10T11:25:49.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.400 7f4c022cc700 1 -- 172.21.2.101:0/2728117503 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839310", "target": ["mgr", ""]}) v1 -- 0x7f4bfc0c23e0 con 0x7f4bdc0186d0 2021-02-10T11:25:49.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.404 7f4bf8ff9700 1 -- 172.21.2.101:0/2728117503 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f4bec006b60 con 0x7f4bdc0186d0 2021-02-10T11:25:49.415 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839310/ef0d76b1-456e-46f1-8d1b-b636c3733c23 2021-02-10T11:25:49.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.408 7f4c022cc700 1 -- 172.21.2.101:0/2728117503 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bdc0186d0 msgr2=0x7f4bdc01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:49.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.408 7f4c022cc700 1 --2- 172.21.2.101:0/2728117503 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bdc0186d0 0x7f4bdc01abd0 crc :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:49.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.408 7f4c022cc700 1 -- 172.21.2.101:0/2728117503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bfc0c1f60 msgr2=0x7f4bfc09e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:49.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.409 7f4c022cc700 1 --2- 172.21.2.101:0/2728117503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bfc0c1f60 0x7f4bfc09e300 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f4bf0006f60 tx=0x7f4bf000b970).stop 2021-02-10T11:25:49.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.409 7f4c022cc700 1 -- 172.21.2.101:0/2728117503 shutdown_connections 2021-02-10T11:25:49.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.409 7f4c022cc700 1 --2- 172.21.2.101:0/2728117503 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4bfc09e920 0x7f4bfc0c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:49.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.409 7f4c022cc700 1 --2- 172.21.2.101:0/2728117503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bfc0c1f60 0x7f4bfc09e300 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:49.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.409 7f4c022cc700 1 --2- 172.21.2.101:0/2728117503 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bdc0186d0 0x7f4bdc01abd0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:49.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.409 7f4c022cc700 1 --2- 172.21.2.101:0/2728117503 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bfc09f140 0x7f4bfc0c1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:49.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.409 7f4c022cc700 1 -- 172.21.2.101:0/2728117503 >> 172.21.2.101:0/2728117503 conn(0x7f4bfc09fc10 msgr2=0x7f4bfc0b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:49.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.409 7f4c022cc700 1 -- 172.21.2.101:0/2728117503 shutdown_connections 2021-02-10T11:25:49.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:49.409 7f4c022cc700 1 -- 172.21.2.101:0/2728117503 wait complete. 2021-02-10T11:25:49.425 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839310 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839310/ef0d76b1-456e-46f1-8d1b-b636c3733c23 2021-02-10T11:25:49.426 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:49.426 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:49.426 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:49.763 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:49.764 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:49.764 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839310/ef0d76b1-456e-46f1-8d1b-b636c3733c23/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:50.144 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:50.144 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:50.144 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234882 s, 44.6 MB/s 2021-02-10T11:25:50.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:50.146 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:50.146 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:50.481 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:50.481 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:50.482 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839310/ef0d76b1-456e-46f1-8d1b-b636c3733c23/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:50.857 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:50.857 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:50.857 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234806 s, 44.7 MB/s 2021-02-10T11:25:50.858 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:50.858 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:50.858 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:51.193 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:51.194 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:51.194 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839310/ef0d76b1-456e-46f1-8d1b-b636c3733c23/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:51.568 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:51.568 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:51.569 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.02287 s, 45.8 MB/s 2021-02-10T11:25:51.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:51.570 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:51.570 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:51.905 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:51.905 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:51.906 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839310/ef0d76b1-456e-46f1-8d1b-b636c3733c23/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:52.283 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:52.284 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:52.284 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0268531 s, 39.0 MB/s 2021-02-10T11:25:52.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:52.285 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:52.285 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:52.564 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:25:52.567 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:25:52.620 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:52.620 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:52.620 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839310/ef0d76b1-456e-46f1-8d1b-b636c3733c23/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:52.939 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:52.939 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:52.939 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242207 s, 43.3 MB/s 2021-02-10T11:25:52.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:52.940 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:52.941 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:53.317 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:53.317 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:53.317 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839310/ef0d76b1-456e-46f1-8d1b-b636c3733c23/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:53.696 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:53.697 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:53.697 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248541 s, 42.2 MB/s 2021-02-10T11:25:53.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:53.698 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:53.698 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:54.034 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:54.035 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:54.035 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839310/ef0d76b1-456e-46f1-8d1b-b636c3733c23/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:54.408 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:54.408 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:54.408 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024125 s, 43.5 MB/s 2021-02-10T11:25:54.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:54.409 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:54.409 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:54.744 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:54.744 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:54.745 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839310/ef0d76b1-456e-46f1-8d1b-b636c3733c23/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:55.121 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:55.121 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:55.121 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023866 s, 43.9 MB/s 2021-02-10T11:25:55.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:55.122 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:55.122 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:55.458 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:55.458 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:55.458 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839310/ef0d76b1-456e-46f1-8d1b-b636c3733c23/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:55.835 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:55.835 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:55.835 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245948 s, 42.6 MB/s 2021-02-10T11:25:55.836 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:55.836 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:55.836 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:56.172 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:56.172 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:56.173 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839310/ef0d76b1-456e-46f1-8d1b-b636c3733c23/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:56.548 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:56.549 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:56.549 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235961 s, 44.4 MB/s 2021-02-10T11:25:56.550 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839311 2021-02-10T11:25:56.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.957 7fa1c5298700 1 Processor -- start 2021-02-10T11:25:56.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.957 7fa1c5298700 1 -- start start 2021-02-10T11:25:56.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.957 7fa1c5298700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa1c01b0470 0x7fa1c01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:56.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.957 7fa1c5298700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa1c00f20e0 0x7fa1c00f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:56.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.957 7fa1c5298700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1c00f0b50 con 0x7fa1c00ef610 2021-02-10T11:25:56.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.957 7fa1c5298700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c00f0820 con 0x7fa1c00f20e0 2021-02-10T11:25:56.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.957 7fa1c5298700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c00f24e0 con 0x7fa1c01b0470 2021-02-10T11:25:56.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.957 7fa1bf87e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa1c00f20e0 0x7fa1c00f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:56.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.957 7fa1be87c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa1c00ef610 0x7fa1c00efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48146/0 (socket says 172.21.2.101:48146) 2021-02-10T11:25:56.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.957 7fa1be87c700 1 -- 172.21.2.101:0/2045250192 learned_addr learned my addr 172.21.2.101:0/2045250192 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:25:56.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.958 7fa1bf87e700 1 -- 172.21.2.101:0/2045250192 >> v2:172.21.2.108:3300/0 conn(0x7fa1c01b0470 msgr2=0x7fa1c01b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:25:56.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.958 7fa1bf87e700 1 --2- 172.21.2.101:0/2045250192 >> v2:172.21.2.108:3300/0 conn(0x7fa1c01b0470 0x7fa1c01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:56.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.958 7fa1bf87e700 1 -- 172.21.2.101:0/2045250192 >> v1:172.21.2.101:6789/0 conn(0x7fa1c00ef610 legacy=0x7fa1c00efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:56.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.958 7fa1bf87e700 1 -- 172.21.2.101:0/2045250192 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c00e52e0 con 0x7fa1c00f20e0 2021-02-10T11:25:56.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.958 7fa1bf87e700 1 --2- 172.21.2.101:0/2045250192 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1c00f20e0 0x7fa1c00f0200 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7fa1b8006ab0 tx=0x7fa1b8009b40).ready entity=mon.0 client_cookie=a54838b8bbe9858f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:56.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.958 7fa1be07b700 1 -- 172.21.2.101:0/2045250192 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa1b8000c10 con 0x7fa1c00f20e0 2021-02-10T11:25:56.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.958 7fa1be07b700 1 -- 172.21.2.101:0/2045250192 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b80022e0 con 0x7fa1c00f20e0 2021-02-10T11:25:56.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.958 7fa1be07b700 1 -- 172.21.2.101:0/2045250192 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa1b8002730 con 0x7fa1c00f20e0 2021-02-10T11:25:56.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.958 7fa1c5298700 1 -- 172.21.2.101:0/2045250192 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1c00f20e0 msgr2=0x7fa1c00f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:56.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.958 7fa1c5298700 1 --2- 172.21.2.101:0/2045250192 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1c00f20e0 0x7fa1c00f0200 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7fa1b8006ab0 tx=0x7fa1b8009b40).stop 2021-02-10T11:25:56.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.959 7fa1c5298700 1 -- 172.21.2.101:0/2045250192 shutdown_connections 2021-02-10T11:25:56.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.959 7fa1c5298700 1 --2- 172.21.2.101:0/2045250192 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1c00f20e0 0x7fa1c00f0200 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:56.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.959 7fa1c5298700 1 --2- 172.21.2.101:0/2045250192 >> v2:172.21.2.108:3300/0 conn(0x7fa1c01b0470 0x7fa1c01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:56.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.959 7fa1c5298700 1 -- 172.21.2.101:0/2045250192 >> 172.21.2.101:0/2045250192 conn(0x7fa1c00ce3e0 msgr2=0x7fa1c00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:56.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.959 7fa1c5298700 1 -- 172.21.2.101:0/2045250192 shutdown_connections 2021-02-10T11:25:56.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.959 7fa1c5298700 1 -- 172.21.2.101:0/2045250192 wait complete. 2021-02-10T11:25:56.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.974 7fa1c5298700 1 Processor -- start 2021-02-10T11:25:56.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.974 7fa1c5298700 1 -- start start 2021-02-10T11:25:56.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.974 7fa1c5298700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1c00a6c90 0x7fa1c01924c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:56.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.974 7fa1c5298700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1c0192ae0 0x7fa1c0192ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:56.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.974 7fa1c5298700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1c00919e0 0x7fa1c0091de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:56.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.974 7fa1c5298700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c019d2e0 con 0x7fa1c0192ae0 2021-02-10T11:25:56.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.974 7fa1c5298700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c00914d0 con 0x7fa1c00919e0 2021-02-10T11:25:56.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.974 7fa1c5298700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c00a6820 con 0x7fa1c00a6c90 2021-02-10T11:25:56.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.974 7fa1be87c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1c0192ae0 0x7fa1c0192ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:56.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.974 7fa1be87c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1c0192ae0 0x7fa1c0192ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47658/0 (socket says 172.21.2.101:47658) 2021-02-10T11:25:56.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.974 7fa1bf07d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1c00a6c90 0x7fa1c01924c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:56.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.975 7fa1be87c700 1 -- 172.21.2.101:0/3687528059 learned_addr learned my addr 172.21.2.101:0/3687528059 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:25:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.975 7fa1bf87e700 1 --2- 172.21.2.101:0/3687528059 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1c00919e0 0x7fa1c0091de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.975 7fa1be87c700 1 -- 172.21.2.101:0/3687528059 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1c00a6c90 msgr2=0x7fa1c01924c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.975 7fa1be87c700 1 --2- 172.21.2.101:0/3687528059 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1c00a6c90 0x7fa1c01924c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.975 7fa1be87c700 1 -- 172.21.2.101:0/3687528059 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1c00919e0 msgr2=0x7fa1c0091de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.975 7fa1be87c700 1 --2- 172.21.2.101:0/3687528059 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1c00919e0 0x7fa1c0091de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.975 7fa1be87c700 1 -- 172.21.2.101:0/3687528059 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c0092400 con 0x7fa1c0192ae0 2021-02-10T11:25:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.975 7fa1bf07d700 1 --2- 172.21.2.101:0/3687528059 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1c00a6c90 0x7fa1c01924c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:25:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.975 7fa1be87c700 1 --2- 172.21.2.101:0/3687528059 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1c0192ae0 0x7fa1c0192ee0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fa1a800a920 tx=0x7fa1a800acc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.976 7fa1b7fff700 1 -- 172.21.2.101:0/3687528059 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa1a80070d0 con 0x7fa1c0192ae0 2021-02-10T11:25:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.976 7fa1b7fff700 1 -- 172.21.2.101:0/3687528059 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a8007390 con 0x7fa1c0192ae0 2021-02-10T11:25:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.976 7fa1b7fff700 1 -- 172.21.2.101:0/3687528059 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa1a8004740 con 0x7fa1c0192ae0 2021-02-10T11:25:56.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.976 7fa1c5298700 1 -- 172.21.2.101:0/3687528059 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c0092400 con 0x7fa1c0192ae0 2021-02-10T11:25:56.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.976 7fa1c5298700 1 -- 172.21.2.101:0/3687528059 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c0092990 con 0x7fa1c0192ae0 2021-02-10T11:25:56.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.976 7fa1b7fff700 1 -- 172.21.2.101:0/3687528059 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa1a80044c0 con 0x7fa1c0192ae0 2021-02-10T11:25:56.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.976 7fa1b7fff700 1 --2- 172.21.2.101:0/3687528059 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1a40186c0 0x7fa1a401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:56.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.976 7fa1b7fff700 1 -- 172.21.2.101:0/3687528059 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa1a8032930 con 0x7fa1c0192ae0 2021-02-10T11:25:56.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.977 7fa1bf07d700 1 --2- 172.21.2.101:0/3687528059 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1a40186c0 0x7fa1a401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:56.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:56.977 7fa1bf07d700 1 --2- 172.21.2.101:0/3687528059 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1a40186c0 0x7fa1a401abc0 crc :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:57.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.007 7fa1c5298700 1 -- 172.21.2.101:0/3687528059 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c00e52e0 con 0x7fa1c0192ae0 2021-02-10T11:25:57.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.010 7fa1b7fff700 1 -- 172.21.2.101:0/3687528059 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa1a80011a0 con 0x7fa1c0192ae0 2021-02-10T11:25:57.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.134 7fa1c5298700 1 -- 172.21.2.101:0/3687528059 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839311", "target": ["mgr", ""]}) v1 -- 0x7fa1c0092990 con 0x7fa1a40186c0 2021-02-10T11:25:57.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.160 7fa1b7fff700 1 -- 172.21.2.101:0/3687528059 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fa1b00068d0 con 0x7fa1a40186c0 2021-02-10T11:25:57.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.167 7fa1c5298700 1 -- 172.21.2.101:0/3687528059 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1a40186c0 msgr2=0x7fa1a401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:57.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.167 7fa1c5298700 1 --2- 172.21.2.101:0/3687528059 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1a40186c0 0x7fa1a401abc0 crc :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:57.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.167 7fa1c5298700 1 -- 172.21.2.101:0/3687528059 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1c0192ae0 msgr2=0x7fa1c0192ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:57.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.167 7fa1c5298700 1 --2- 172.21.2.101:0/3687528059 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1c0192ae0 0x7fa1c0192ee0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fa1a800a920 tx=0x7fa1a800acc0).stop 2021-02-10T11:25:57.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.167 7fa1c5298700 1 -- 172.21.2.101:0/3687528059 shutdown_connections 2021-02-10T11:25:57.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.167 7fa1c5298700 1 --2- 172.21.2.101:0/3687528059 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1c00919e0 0x7fa1c0091de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:57.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.167 7fa1c5298700 1 --2- 172.21.2.101:0/3687528059 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1c0192ae0 0x7fa1c0192ee0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:57.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.167 7fa1c5298700 1 --2- 172.21.2.101:0/3687528059 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1a40186c0 0x7fa1a401abc0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:57.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.167 7fa1c5298700 1 --2- 172.21.2.101:0/3687528059 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1c00a6c90 0x7fa1c01924c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:57.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.167 7fa1c5298700 1 -- 172.21.2.101:0/3687528059 >> 172.21.2.101:0/3687528059 conn(0x7fa1c00ac550 msgr2=0x7fa1c00be1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:57.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.168 7fa1c5298700 1 -- 172.21.2.101:0/3687528059 shutdown_connections 2021-02-10T11:25:57.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.168 7fa1c5298700 1 -- 172.21.2.101:0/3687528059 wait complete. 2021-02-10T11:25:57.185 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839311 2021-02-10T11:25:57.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.594 7f48895c4700 1 Processor -- start 2021-02-10T11:25:57.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.594 7f48895c4700 1 -- start start 2021-02-10T11:25:57.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.594 7f48895c4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48841b0470 0x7f48841b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:57.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.594 7f48895c4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f48840ef610 0x7f48840efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:57.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.594 7f48895c4700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48840f0b50 con 0x7f48840f0030 2021-02-10T11:25:57.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.594 7f48895c4700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48840e52e0 con 0x7f48840ef610 2021-02-10T11:25:57.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.594 7f48895c4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48840e5490 con 0x7f48841b0470 2021-02-10T11:25:57.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.594 7f4882abe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f48840ef610 0x7f48840efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:57.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.594 7f4883ac0700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f48840f0030 0x7f48840f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48158/0 (socket says 172.21.2.101:48158) 2021-02-10T11:25:57.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.594 7f4883ac0700 1 -- 172.21.2.101:0/2190687956 learned_addr learned my addr 172.21.2.101:0/2190687956 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:25:57.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.595 7f4882abe700 1 -- 172.21.2.101:0/2190687956 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48841b0470 msgr2=0x7f48841b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:25:57.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.595 7f4882abe700 1 --2- 172.21.2.101:0/2190687956 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48841b0470 0x7f48841b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:57.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.595 7f4882abe700 1 -- 172.21.2.101:0/2190687956 >> v1:172.21.2.101:6789/0 conn(0x7f48840f0030 legacy=0x7f48840f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:57.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.595 7f4882abe700 1 -- 172.21.2.101:0/2190687956 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48840f0430 con 0x7f48840ef610 2021-02-10T11:25:57.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.595 7f4882abe700 1 --2- 172.21.2.101:0/2190687956 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48840ef610 0x7f48840efa10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f486c006650 tx=0x7f486c009a80).ready entity=mon.0 client_cookie=e9b3780ae4521609 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:57.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.595 7f48822bd700 1 -- 172.21.2.101:0/2190687956 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f486c00c0c0 con 0x7f48840ef610 2021-02-10T11:25:57.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.595 7f48822bd700 1 -- 172.21.2.101:0/2190687956 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486c00c340 con 0x7f48840ef610 2021-02-10T11:25:57.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.595 7f48822bd700 1 -- 172.21.2.101:0/2190687956 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f486c00c860 con 0x7f48840ef610 2021-02-10T11:25:57.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.595 7f48895c4700 1 -- 172.21.2.101:0/2190687956 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48840ef610 msgr2=0x7f48840efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:57.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.595 7f48895c4700 1 --2- 172.21.2.101:0/2190687956 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48840ef610 0x7f48840efa10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f486c006650 tx=0x7f486c009a80).stop 2021-02-10T11:25:57.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.596 7f48895c4700 1 -- 172.21.2.101:0/2190687956 shutdown_connections 2021-02-10T11:25:57.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.596 7f48895c4700 1 --2- 172.21.2.101:0/2190687956 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48840ef610 0x7f48840efa10 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:57.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.596 7f48895c4700 1 --2- 172.21.2.101:0/2190687956 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48841b0470 0x7f48841b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:25:57.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.596 7f48895c4700 1 -- 172.21.2.101:0/2190687956 >> 172.21.2.101:0/2190687956 conn(0x7f48840ce3e0 msgr2=0x7f48840e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:57.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.596 7f48895c4700 1 -- 172.21.2.101:0/2190687956 shutdown_connections 2021-02-10T11:25:57.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.596 7f48895c4700 1 -- 172.21.2.101:0/2190687956 wait complete. 2021-02-10T11:25:57.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.611 7f48895c4700 1 Processor -- start 2021-02-10T11:25:57.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.611 7f48895c4700 1 -- start start 2021-02-10T11:25:57.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.611 7f48895c4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48840c36a0 0x7f48840c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:57.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.611 7f48895c4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f48840c3190 0x7f48840b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:57.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.611 7f48895c4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48840b2370 0x7f48840b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:57.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.611 7f48895c4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48840b27c0 con 0x7f48840c36a0 2021-02-10T11:25:57.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.612 7f48895c4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48840b3360 con 0x7f48840c3190 2021-02-10T11:25:57.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.612 7f48895c4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48840b3540 con 0x7f48840b2370 2021-02-10T11:25:57.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.612 7f48832bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48840c36a0 0x7f48840c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:57.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.612 7f48832bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48840c36a0 0x7f48840c2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47670/0 (socket says 172.21.2.101:47670) 2021-02-10T11:25:57.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.612 7f48832bf700 1 -- 172.21.2.101:0/1048141551 learned_addr learned my addr 172.21.2.101:0/1048141551 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:25:57.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.612 7f4882abe700 1 --2- 172.21.2.101:0/1048141551 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f48840c3190 0x7f48840b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:57.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.612 7f4883ac0700 1 --2- 172.21.2.101:0/1048141551 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48840b2370 0x7f48840b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:57.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.612 7f48832bf700 1 -- 172.21.2.101:0/1048141551 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48840b2370 msgr2=0x7f48840b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:57.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.612 7f48832bf700 1 --2- 172.21.2.101:0/1048141551 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48840b2370 0x7f48840b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:57.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.612 7f48832bf700 1 -- 172.21.2.101:0/1048141551 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f48840c3190 msgr2=0x7f48840b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:57.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.612 7f48832bf700 1 --2- 172.21.2.101:0/1048141551 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f48840c3190 0x7f48840b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:57.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.612 7f48832bf700 1 -- 172.21.2.101:0/1048141551 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48840b3760 con 0x7f48840c36a0 2021-02-10T11:25:57.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.612 7f48832bf700 1 --2- 172.21.2.101:0/1048141551 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48840c36a0 0x7f48840c2b70 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f4874006f00 tx=0x7f487400b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:57.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.613 7f487bfff700 1 -- 172.21.2.101:0/1048141551 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f487400bd30 con 0x7f48840c36a0 2021-02-10T11:25:57.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.613 7f487bfff700 1 -- 172.21.2.101:0/1048141551 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4874000c90 con 0x7f48840c36a0 2021-02-10T11:25:57.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.613 7f487bfff700 1 -- 172.21.2.101:0/1048141551 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f48740022a0 con 0x7f48840c36a0 2021-02-10T11:25:57.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.613 7f48895c4700 1 -- 172.21.2.101:0/1048141551 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48840b3760 con 0x7f48840c36a0 2021-02-10T11:25:57.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.613 7f48895c4700 1 -- 172.21.2.101:0/1048141551 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48840b3cf0 con 0x7f48840c36a0 2021-02-10T11:25:57.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.613 7f487bfff700 1 -- 172.21.2.101:0/1048141551 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4874005310 con 0x7f48840c36a0 2021-02-10T11:25:57.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.613 7f487bfff700 1 --2- 172.21.2.101:0/1048141551 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f48680186c0 0x7f486801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:25:57.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.613 7f487bfff700 1 -- 172.21.2.101:0/1048141551 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4874032560 con 0x7f48840c36a0 2021-02-10T11:25:57.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.613 7f4882abe700 1 --2- 172.21.2.101:0/1048141551 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f48680186c0 0x7f486801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:25:57.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.614 7f4882abe700 1 --2- 172.21.2.101:0/1048141551 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f48680186c0 0x7f486801abc0 crc :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:25:57.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.644 7f48895c4700 1 -- 172.21.2.101:0/1048141551 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48840edd10 con 0x7f48840c36a0 2021-02-10T11:25:57.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.647 7f487bfff700 1 -- 172.21.2.101:0/1048141551 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4874000960 con 0x7f48840c36a0 2021-02-10T11:25:57.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.776 7f48895c4700 1 -- 172.21.2.101:0/1048141551 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839311", "target": ["mgr", ""]}) v1 -- 0x7f48840b3cf0 con 0x7f48680186c0 2021-02-10T11:25:57.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.778 7f487bfff700 1 -- 172.21.2.101:0/1048141551 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f486c006ae0 con 0x7f48680186c0 2021-02-10T11:25:57.787 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839311/39408af9-d187-41fe-89eb-048189a6754a 2021-02-10T11:25:57.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.780 7f48895c4700 1 -- 172.21.2.101:0/1048141551 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f48680186c0 msgr2=0x7f486801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:57.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.780 7f48895c4700 1 --2- 172.21.2.101:0/1048141551 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f48680186c0 0x7f486801abc0 crc :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:57.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.780 7f48895c4700 1 -- 172.21.2.101:0/1048141551 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48840c36a0 msgr2=0x7f48840c2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:25:57.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.780 7f48895c4700 1 --2- 172.21.2.101:0/1048141551 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48840c36a0 0x7f48840c2b70 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f4874006f00 tx=0x7f487400b940).stop 2021-02-10T11:25:57.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.780 7f48895c4700 1 -- 172.21.2.101:0/1048141551 shutdown_connections 2021-02-10T11:25:57.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.780 7f48895c4700 1 --2- 172.21.2.101:0/1048141551 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f48680186c0 0x7f486801abc0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:57.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.780 7f48895c4700 1 --2- 172.21.2.101:0/1048141551 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48840b2370 0x7f48840b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:57.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.780 7f48895c4700 1 --2- 172.21.2.101:0/1048141551 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f48840c3190 0x7f48840b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:57.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.780 7f48895c4700 1 --2- 172.21.2.101:0/1048141551 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48840c36a0 0x7f48840c2b70 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:25:57.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.780 7f48895c4700 1 -- 172.21.2.101:0/1048141551 >> 172.21.2.101:0/1048141551 conn(0x7f488409f630 msgr2=0x7f48840a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:25:57.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.781 7f48895c4700 1 -- 172.21.2.101:0/1048141551 shutdown_connections 2021-02-10T11:25:57.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:25:57.781 7f48895c4700 1 -- 172.21.2.101:0/1048141551 wait complete. 2021-02-10T11:25:57.797 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839311 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839311/39408af9-d187-41fe-89eb-048189a6754a 2021-02-10T11:25:57.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:57.797 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:57.797 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:58.133 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:58.133 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:58.134 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839311/39408af9-d187-41fe-89eb-048189a6754a/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:58.514 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:58.514 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:58.514 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235425 s, 44.5 MB/s 2021-02-10T11:25:58.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:58.515 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:58.515 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:58.850 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:58.850 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:58.850 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839311/39408af9-d187-41fe-89eb-048189a6754a/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:59.226 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:59.226 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:59.226 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0269079 s, 39.0 MB/s 2021-02-10T11:25:59.227 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:59.227 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:59.227 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:25:59.562 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:25:59.563 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:25:59.563 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839311/39408af9-d187-41fe-89eb-048189a6754a/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:25:59.939 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:25:59.939 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:25:59.939 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243401 s, 43.1 MB/s 2021-02-10T11:25:59.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:25:59.940 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:25:59.940 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:00.276 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:00.276 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:00.276 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839311/39408af9-d187-41fe-89eb-048189a6754a/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:00.653 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:00.654 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:00.654 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247091 s, 42.4 MB/s 2021-02-10T11:26:00.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:00.655 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:00.655 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:00.990 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:00.990 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:00.990 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839311/39408af9-d187-41fe-89eb-048189a6754a/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:01.366 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:01.366 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:01.366 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238199 s, 44.0 MB/s 2021-02-10T11:26:01.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:01.367 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:01.367 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:01.702 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:01.702 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:01.703 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839311/39408af9-d187-41fe-89eb-048189a6754a/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:02.078 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:02.078 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:02.078 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235839 s, 44.5 MB/s 2021-02-10T11:26:02.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:02.079 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:02.079 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:02.417 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:02.417 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:02.417 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839311/39408af9-d187-41fe-89eb-048189a6754a/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:02.792 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:02.793 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:02.793 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233714 s, 44.9 MB/s 2021-02-10T11:26:02.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:02.794 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:02.794 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:03.129 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:03.129 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:03.130 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839311/39408af9-d187-41fe-89eb-048189a6754a/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:03.506 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:03.506 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:03.506 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243004 s, 43.2 MB/s 2021-02-10T11:26:03.507 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:03.507 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:03.508 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:03.842 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:03.842 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:03.842 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839311/39408af9-d187-41fe-89eb-048189a6754a/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:04.217 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:04.217 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:04.217 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247532 s, 42.4 MB/s 2021-02-10T11:26:04.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:04.218 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:04.218 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:04.553 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:04.553 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:04.553 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839311/39408af9-d187-41fe-89eb-048189a6754a/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:04.930 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:04.930 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:04.930 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238451 s, 44.0 MB/s 2021-02-10T11:26:04.931 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839312 2021-02-10T11:26:05.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.338 7f2d075f5700 1 Processor -- start 2021-02-10T11:26:05.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.338 7f2d075f5700 1 -- start start 2021-02-10T11:26:05.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.339 7f2d075f5700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2d001b0470 0x7f2d001b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:05.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.339 7f2d075f5700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2d000ef610 0x7f2d000efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:05.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.339 7f2d075f5700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d000f0b50 con 0x7f2d000f0030 2021-02-10T11:26:05.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.339 7f2d075f5700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d000e52e0 con 0x7f2d001b0470 2021-02-10T11:26:05.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.339 7f2d075f5700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d000e5490 con 0x7f2d000ef610 2021-02-10T11:26:05.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.339 7f2d05411700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2d001b0470 0x7f2d001b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:05.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.339 7f2d05c12700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f2d000f0030 0x7f2d000f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48172/0 (socket says 172.21.2.101:48172) 2021-02-10T11:26:05.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.339 7f2d05411700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2d001b0470 0x7f2d001b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47678/0 (socket says 172.21.2.101:47678) 2021-02-10T11:26:05.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.339 7f2d05c12700 1 -- 172.21.2.101:0/2244866923 learned_addr learned my addr 172.21.2.101:0/2244866923 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:26:05.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.339 7f2d04c10700 1 --2- 172.21.2.101:0/2244866923 >> v2:172.21.2.108:3300/0 conn(0x7f2d000ef610 0x7f2d000efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:05.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.339 7f2d05411700 1 -- 172.21.2.101:0/2244866923 >> v2:172.21.2.108:3300/0 conn(0x7f2d000ef610 msgr2=0x7f2d000efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:26:05.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.339 7f2d05411700 1 --2- 172.21.2.101:0/2244866923 >> v2:172.21.2.108:3300/0 conn(0x7f2d000ef610 0x7f2d000efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:05.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.339 7f2d05411700 1 -- 172.21.2.101:0/2244866923 >> v1:172.21.2.101:6789/0 conn(0x7f2d000f0030 legacy=0x7f2d000f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:05.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.339 7f2d05411700 1 -- 172.21.2.101:0/2244866923 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d000f0430 con 0x7f2d001b0470 2021-02-10T11:26:05.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.339 7f2d05411700 1 --2- 172.21.2.101:0/2244866923 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d001b0470 0x7f2d001b0870 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f2cf4006650 tx=0x7f2cf4009a80).ready entity=mon.0 client_cookie=a5eaad8f0ca90fb7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:05.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.340 7f2cf3fff700 1 -- 172.21.2.101:0/2244866923 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2cf40009b0 con 0x7f2d001b0470 2021-02-10T11:26:05.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.340 7f2cf3fff700 1 -- 172.21.2.101:0/2244866923 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf400c0c0 con 0x7f2d001b0470 2021-02-10T11:26:05.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.340 7f2cf3fff700 1 -- 172.21.2.101:0/2244866923 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2cf400c5e0 con 0x7f2d001b0470 2021-02-10T11:26:05.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.340 7f2d075f5700 1 -- 172.21.2.101:0/2244866923 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d001b0470 msgr2=0x7f2d001b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:05.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.340 7f2d075f5700 1 --2- 172.21.2.101:0/2244866923 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d001b0470 0x7f2d001b0870 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f2cf4006650 tx=0x7f2cf4009a80).stop 2021-02-10T11:26:05.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.341 7f2d075f5700 1 -- 172.21.2.101:0/2244866923 shutdown_connections 2021-02-10T11:26:05.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.341 7f2d075f5700 1 --2- 172.21.2.101:0/2244866923 >> v2:172.21.2.108:3300/0 conn(0x7f2d000ef610 0x7f2d000efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:05.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.341 7f2d075f5700 1 --2- 172.21.2.101:0/2244866923 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d001b0470 0x7f2d001b0870 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:05.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.341 7f2d075f5700 1 -- 172.21.2.101:0/2244866923 >> 172.21.2.101:0/2244866923 conn(0x7f2d000ce3e0 msgr2=0x7f2d000e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:05.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.341 7f2d075f5700 1 -- 172.21.2.101:0/2244866923 shutdown_connections 2021-02-10T11:26:05.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.341 7f2d075f5700 1 -- 172.21.2.101:0/2244866923 wait complete. 2021-02-10T11:26:05.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.355 7f2d075f5700 1 Processor -- start 2021-02-10T11:26:05.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.355 7f2d075f5700 1 -- start start 2021-02-10T11:26:05.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.356 7f2d075f5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d0008ece0 0x7f2d000a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:05.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.356 7f2d075f5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d000a6860 0x7f2d000a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:05.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.356 7f2d075f5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d000a7280 0x7f2d000a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:05.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.356 7f2d075f5700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d000a5cc0 con 0x7f2d000a6860 2021-02-10T11:26:05.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.356 7f2d075f5700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0008f0e0 con 0x7f2d000a7280 2021-02-10T11:26:05.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.356 7f2d075f5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0008f2c0 con 0x7f2d0008ece0 2021-02-10T11:26:05.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.356 7f2d04c10700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d000a6860 0x7f2d000a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:05.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.356 7f2d04c10700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d000a6860 0x7f2d000a6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47686/0 (socket says 172.21.2.101:47686) 2021-02-10T11:26:05.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.356 7f2d04c10700 1 -- 172.21.2.101:0/2324295917 learned_addr learned my addr 172.21.2.101:0/2324295917 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:26:05.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.356 7f2d05c12700 1 --2- 172.21.2.101:0/2324295917 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d000a7280 0x7f2d000a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:05.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.356 7f2d05411700 1 --2- 172.21.2.101:0/2324295917 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d0008ece0 0x7f2d000a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:05.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.357 7f2d04c10700 1 -- 172.21.2.101:0/2324295917 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d0008ece0 msgr2=0x7f2d000a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:05.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.357 7f2d04c10700 1 --2- 172.21.2.101:0/2324295917 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d0008ece0 0x7f2d000a6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:05.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.357 7f2d04c10700 1 -- 172.21.2.101:0/2324295917 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d000a7280 msgr2=0x7f2d000a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:05.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.357 7f2d04c10700 1 --2- 172.21.2.101:0/2324295917 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d000a7280 0x7f2d000a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:05.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.357 7f2d04c10700 1 -- 172.21.2.101:0/2324295917 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d000a4bb0 con 0x7f2d000a6860 2021-02-10T11:26:05.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.357 7f2d04c10700 1 --2- 172.21.2.101:0/2324295917 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d000a6860 0x7f2d000a6c60 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f2cfc006f60 tx=0x7f2cfc00b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:05.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.357 7f2cf1ffb700 1 -- 172.21.2.101:0/2324295917 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2cfc00bd60 con 0x7f2d000a6860 2021-02-10T11:26:05.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.357 7f2cf1ffb700 1 -- 172.21.2.101:0/2324295917 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cfc000c90 con 0x7f2d000a6860 2021-02-10T11:26:05.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.357 7f2cf1ffb700 1 -- 172.21.2.101:0/2324295917 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2cfc002220 con 0x7f2d000a6860 2021-02-10T11:26:05.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.357 7f2d075f5700 1 -- 172.21.2.101:0/2324295917 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d000a4bb0 con 0x7f2d000a6860 2021-02-10T11:26:05.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.357 7f2d075f5700 1 -- 172.21.2.101:0/2324295917 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d000a5140 con 0x7f2d000a6860 2021-02-10T11:26:05.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.358 7f2cf1ffb700 1 -- 172.21.2.101:0/2324295917 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2cfc006b00 con 0x7f2d000a6860 2021-02-10T11:26:05.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.358 7f2cf1ffb700 1 --2- 172.21.2.101:0/2324295917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2ce00186c0 0x7f2ce001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:05.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.358 7f2cf1ffb700 1 -- 172.21.2.101:0/2324295917 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2cfc032800 con 0x7f2d000a6860 2021-02-10T11:26:05.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.358 7f2d05411700 1 --2- 172.21.2.101:0/2324295917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2ce00186c0 0x7f2ce001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:05.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.358 7f2d05411700 1 --2- 172.21.2.101:0/2324295917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2ce00186c0 0x7f2ce001abc0 crc :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:05.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.388 7f2d075f5700 1 -- 172.21.2.101:0/2324295917 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d00105ae0 con 0x7f2d000a6860 2021-02-10T11:26:05.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.391 7f2cf1ffb700 1 -- 172.21.2.101:0/2324295917 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2cfc000940 con 0x7f2d000a6860 2021-02-10T11:26:05.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.518 7f2d075f5700 1 -- 172.21.2.101:0/2324295917 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839312", "target": ["mgr", ""]}) v1 -- 0x7f2d000f0b50 con 0x7f2ce00186c0 2021-02-10T11:26:05.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.545 7f2cf1ffb700 1 -- 172.21.2.101:0/2324295917 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f2cf4000c90 con 0x7f2ce00186c0 2021-02-10T11:26:05.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.550 7f2d075f5700 1 -- 172.21.2.101:0/2324295917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2ce00186c0 msgr2=0x7f2ce001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:05.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.550 7f2d075f5700 1 --2- 172.21.2.101:0/2324295917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2ce00186c0 0x7f2ce001abc0 crc :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:05.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.550 7f2d075f5700 1 -- 172.21.2.101:0/2324295917 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d000a6860 msgr2=0x7f2d000a6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:05.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.550 7f2d075f5700 1 --2- 172.21.2.101:0/2324295917 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d000a6860 0x7f2d000a6c60 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f2cfc006f60 tx=0x7f2cfc00b970).stop 2021-02-10T11:26:05.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.551 7f2d075f5700 1 -- 172.21.2.101:0/2324295917 shutdown_connections 2021-02-10T11:26:05.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.551 7f2d075f5700 1 --2- 172.21.2.101:0/2324295917 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d000a7280 0x7f2d000a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:05.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.551 7f2d075f5700 1 --2- 172.21.2.101:0/2324295917 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d000a6860 0x7f2d000a6c60 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:05.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.551 7f2d075f5700 1 --2- 172.21.2.101:0/2324295917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2ce00186c0 0x7f2ce001abc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:05.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.551 7f2d075f5700 1 --2- 172.21.2.101:0/2324295917 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d0008ece0 0x7f2d000a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:05.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.551 7f2d075f5700 1 -- 172.21.2.101:0/2324295917 >> 172.21.2.101:0/2324295917 conn(0x7f2d0008ffe0 msgr2=0x7f2d000bf050 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:05.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.551 7f2d075f5700 1 -- 172.21.2.101:0/2324295917 shutdown_connections 2021-02-10T11:26:05.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.551 7f2d075f5700 1 -- 172.21.2.101:0/2324295917 wait complete. 2021-02-10T11:26:05.567 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839312 2021-02-10T11:26:05.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.974 7fe8ca96e700 1 Processor -- start 2021-02-10T11:26:05.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.974 7fe8ca96e700 1 -- start start 2021-02-10T11:26:05.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.974 7fe8ca96e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8c41b0470 0x7fe8c41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:05.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.974 7fe8ca96e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c40f0b50 con 0x7fe8c40f20e0 2021-02-10T11:26:06.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.974 7fe8ca96e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c40f24e0 con 0x7fe8c40ef610 2021-02-10T11:26:06.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.974 7fe8ca96e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c40e5ea0 con 0x7fe8c41b0470 2021-02-10T11:26:06.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.975 7fe8c8f8b700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe8c40f20e0 0x7fe8c40f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48184/0 (socket says 172.21.2.101:48184) 2021-02-10T11:26:06.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.975 7fe8c8f8b700 1 -- 172.21.2.101:0/1438520350 learned_addr learned my addr 172.21.2.101:0/1438520350 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:26:06.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.975 7fe8c3fff700 1 --2- 172.21.2.101:0/1438520350 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8c41b0470 0x7fe8c41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:06.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.975 7fe8c2ffd700 1 -- 172.21.2.101:0/1438520350 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3555234656 0 0) 0x7fe8bc000d30 con 0x7fe8c40f20e0 2021-02-10T11:26:06.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.975 7fe8c2ffd700 1 -- 172.21.2.101:0/1438520350 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8ac002fe0 con 0x7fe8c40f20e0 2021-02-10T11:26:06.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.975 7fe8c2ffd700 1 -- 172.21.2.101:0/1438520350 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3959331040 0 0) 0x7fe8b8001080 con 0x7fe8c40ef610 2021-02-10T11:26:06.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.975 7fe8c2ffd700 1 -- 172.21.2.101:0/1438520350 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8ac004370 con 0x7fe8c40ef610 2021-02-10T11:26:06.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.975 7fe8c3fff700 1 -- 172.21.2.101:0/1438520350 >> v1:172.21.2.108:6789/0 conn(0x7fe8c40ef610 legacy=0x7fe8c40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:06.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.975 7fe8c3fff700 1 -- 172.21.2.101:0/1438520350 >> v1:172.21.2.101:6789/0 conn(0x7fe8c40f20e0 legacy=0x7fe8c40f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:06.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.975 7fe8c3fff700 1 -- 172.21.2.101:0/1438520350 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c40e52e0 con 0x7fe8c41b0470 2021-02-10T11:26:06.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.976 7fe8c3fff700 1 --2- 172.21.2.101:0/1438520350 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8c41b0470 0x7fe8c41b0870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fe8b4006650 tx=0x7fe8b4009a80).ready entity=mon.2 client_cookie=4c4663b0915d775b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:06.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.976 7fe8c2ffd700 1 -- 172.21.2.101:0/1438520350 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe8b40009b0 con 0x7fe8c41b0470 2021-02-10T11:26:06.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.976 7fe8c2ffd700 1 -- 172.21.2.101:0/1438520350 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b400c0c0 con 0x7fe8c41b0470 2021-02-10T11:26:06.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.976 7fe8c2ffd700 1 -- 172.21.2.101:0/1438520350 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe8b400c510 con 0x7fe8c41b0470 2021-02-10T11:26:06.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.976 7fe8ca96e700 1 -- 172.21.2.101:0/1438520350 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8c41b0470 msgr2=0x7fe8c41b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:06.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.976 7fe8ca96e700 1 --2- 172.21.2.101:0/1438520350 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8c41b0470 0x7fe8c41b0870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fe8b4006650 tx=0x7fe8b4009a80).stop 2021-02-10T11:26:06.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.976 7fe8ca96e700 1 -- 172.21.2.101:0/1438520350 shutdown_connections 2021-02-10T11:26:06.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.976 7fe8ca96e700 1 --2- 172.21.2.101:0/1438520350 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8c41b0470 0x7fe8c41b0870 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:06.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.976 7fe8ca96e700 1 -- 172.21.2.101:0/1438520350 >> 172.21.2.101:0/1438520350 conn(0x7fe8c40ce3e0 msgr2=0x7fe8c40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:06.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.976 7fe8ca96e700 1 -- 172.21.2.101:0/1438520350 shutdown_connections 2021-02-10T11:26:06.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.976 7fe8ca96e700 1 -- 172.21.2.101:0/1438520350 wait complete. 2021-02-10T11:26:06.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.991 7fe8ca96e700 1 Processor -- start 2021-02-10T11:26:06.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.991 7fe8ca96e700 1 -- start start 2021-02-10T11:26:06.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.992 7fe8ca96e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8c40a0420 0x7fe8c40a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:06.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.992 7fe8ca96e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe8c4189d00 0x7fe8c418a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:06.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.992 7fe8ca96e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8c418a720 0x7fe8c40b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:06.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.992 7fe8ca96e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c4189780 con 0x7fe8c40a0420 2021-02-10T11:26:06.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.992 7fe8ca96e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c418ab20 con 0x7fe8c4189d00 2021-02-10T11:26:06.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.992 7fe8ca96e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c40a1140 con 0x7fe8c418a720 2021-02-10T11:26:06.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.992 7fe8c3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8c40a0420 0x7fe8c40a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:06.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.992 7fe8c3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8c40a0420 0x7fe8c40a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47698/0 (socket says 172.21.2.101:47698) 2021-02-10T11:26:06.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.992 7fe8c3fff700 1 -- 172.21.2.101:0/1648913872 learned_addr learned my addr 172.21.2.101:0/1648913872 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:26:06.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.992 7fe8c8f8b700 1 --2- 172.21.2.101:0/1648913872 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8c418a720 0x7fe8c40b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:06.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.992 7fe8c37fe700 1 --2- 172.21.2.101:0/1648913872 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe8c4189d00 0x7fe8c418a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:06.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.993 7fe8c3fff700 1 -- 172.21.2.101:0/1648913872 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8c418a720 msgr2=0x7fe8c40b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:06.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.993 7fe8c3fff700 1 --2- 172.21.2.101:0/1648913872 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8c418a720 0x7fe8c40b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:06.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.993 7fe8c3fff700 1 -- 172.21.2.101:0/1648913872 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe8c4189d00 msgr2=0x7fe8c418a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:06.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.993 7fe8c3fff700 1 --2- 172.21.2.101:0/1648913872 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe8c4189d00 0x7fe8c418a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:06.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.993 7fe8c3fff700 1 -- 172.21.2.101:0/1648913872 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c40b8200 con 0x7fe8c40a0420 2021-02-10T11:26:06.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.993 7fe8c3fff700 1 --2- 172.21.2.101:0/1648913872 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8c40a0420 0x7fe8c40a0820 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7fe8b4009630 tx=0x7fe8b400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:06.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.993 7fe8c0ff9700 1 -- 172.21.2.101:0/1648913872 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe8b40093a0 con 0x7fe8c40a0420 2021-02-10T11:26:06.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.993 7fe8c0ff9700 1 -- 172.21.2.101:0/1648913872 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b400bd70 con 0x7fe8c40a0420 2021-02-10T11:26:06.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.993 7fe8c0ff9700 1 -- 172.21.2.101:0/1648913872 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe8b4005560 con 0x7fe8c40a0420 2021-02-10T11:26:06.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.993 7fe8ca96e700 1 -- 172.21.2.101:0/1648913872 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c40b8200 con 0x7fe8c40a0420 2021-02-10T11:26:06.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.993 7fe8ca96e700 1 -- 172.21.2.101:0/1648913872 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c40b8790 con 0x7fe8c40a0420 2021-02-10T11:26:06.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.994 7fe8c0ff9700 1 -- 172.21.2.101:0/1648913872 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe8b400a000 con 0x7fe8c40a0420 2021-02-10T11:26:06.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.994 7fe8c0ff9700 1 --2- 172.21.2.101:0/1648913872 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8a40186c0 0x7fe8a401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:06.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.994 7fe8c0ff9700 1 -- 172.21.2.101:0/1648913872 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe8b4032510 con 0x7fe8c40a0420 2021-02-10T11:26:06.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.994 7fe8c37fe700 1 --2- 172.21.2.101:0/1648913872 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8a40186c0 0x7fe8a401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:06.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:05.994 7fe8c37fe700 1 --2- 172.21.2.101:0/1648913872 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8a40186c0 0x7fe8a401abc0 crc :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:06.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:06.024 7fe8ca96e700 1 -- 172.21.2.101:0/1648913872 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c40a1360 con 0x7fe8c40a0420 2021-02-10T11:26:06.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:06.027 7fe8c0ff9700 1 -- 172.21.2.101:0/1648913872 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe8b40142f0 con 0x7fe8c40a0420 2021-02-10T11:26:06.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:06.155 7fe8ca96e700 1 -- 172.21.2.101:0/1648913872 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839312", "target": ["mgr", ""]}) v1 -- 0x7fe8c40b9020 con 0x7fe8a40186c0 2021-02-10T11:26:06.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:06.158 7fe8c0ff9700 1 -- 172.21.2.101:0/1648913872 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fe8b8006d50 con 0x7fe8a40186c0 2021-02-10T11:26:06.171 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839312/daa110ad-c3e7-4989-8b56-4e489d341ce6 2021-02-10T11:26:06.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:06.163 7fe8ca96e700 1 -- 172.21.2.101:0/1648913872 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8a40186c0 msgr2=0x7fe8a401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:06.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:06.163 7fe8ca96e700 1 --2- 172.21.2.101:0/1648913872 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8a40186c0 0x7fe8a401abc0 crc :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:06.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:06.163 7fe8ca96e700 1 -- 172.21.2.101:0/1648913872 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8c40a0420 msgr2=0x7fe8c40a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:06.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:06.163 7fe8ca96e700 1 --2- 172.21.2.101:0/1648913872 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8c40a0420 0x7fe8c40a0820 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7fe8b4009630 tx=0x7fe8b400b980).stop 2021-02-10T11:26:06.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:06.164 7fe8ca96e700 1 -- 172.21.2.101:0/1648913872 shutdown_connections 2021-02-10T11:26:06.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:06.164 7fe8ca96e700 1 --2- 172.21.2.101:0/1648913872 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8a40186c0 0x7fe8a401abc0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:06.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:06.164 7fe8ca96e700 1 --2- 172.21.2.101:0/1648913872 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8c418a720 0x7fe8c40b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:06.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:06.164 7fe8ca96e700 1 --2- 172.21.2.101:0/1648913872 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe8c4189d00 0x7fe8c418a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:06.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:06.164 7fe8ca96e700 1 --2- 172.21.2.101:0/1648913872 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8c40a0420 0x7fe8c40a0820 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:06.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:06.164 7fe8ca96e700 1 -- 172.21.2.101:0/1648913872 >> 172.21.2.101:0/1648913872 conn(0x7fe8c4094e10 msgr2=0x7fe8c40a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:06.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:06.164 7fe8ca96e700 1 -- 172.21.2.101:0/1648913872 shutdown_connections 2021-02-10T11:26:06.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:06.164 7fe8ca96e700 1 -- 172.21.2.101:0/1648913872 wait complete. 2021-02-10T11:26:06.180 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839312 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839312/daa110ad-c3e7-4989-8b56-4e489d341ce6 2021-02-10T11:26:06.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:06.181 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:06.181 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:06.516 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:06.516 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:06.517 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839312/daa110ad-c3e7-4989-8b56-4e489d341ce6/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:06.897 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:06.897 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:06.898 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0229027 s, 45.8 MB/s 2021-02-10T11:26:06.898 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:06.899 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:06.899 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:07.233 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:07.234 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:07.234 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839312/daa110ad-c3e7-4989-8b56-4e489d341ce6/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:07.608 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:07.608 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:07.609 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0227973 s, 46.0 MB/s 2021-02-10T11:26:07.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:07.610 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:07.610 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:07.945 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:07.945 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:07.945 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839312/daa110ad-c3e7-4989-8b56-4e489d341ce6/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:08.322 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:08.322 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:08.322 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234361 s, 44.7 MB/s 2021-02-10T11:26:08.323 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:08.323 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:08.323 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:08.659 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:08.659 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:08.660 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839312/daa110ad-c3e7-4989-8b56-4e489d341ce6/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:09.033 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:09.033 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:09.033 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243478 s, 43.1 MB/s 2021-02-10T11:26:09.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:09.034 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:09.034 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:09.369 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:09.369 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:09.369 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839312/daa110ad-c3e7-4989-8b56-4e489d341ce6/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:09.742 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:09.743 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:09.743 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235467 s, 44.5 MB/s 2021-02-10T11:26:09.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:09.744 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:09.744 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:10.080 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:10.081 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:10.081 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839312/daa110ad-c3e7-4989-8b56-4e489d341ce6/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:10.458 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:10.459 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:10.459 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249957 s, 42.0 MB/s 2021-02-10T11:26:10.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:10.460 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:10.460 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:10.794 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:10.795 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:10.795 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839312/daa110ad-c3e7-4989-8b56-4e489d341ce6/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:11.176 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:11.176 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:11.176 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0266279 s, 39.4 MB/s 2021-02-10T11:26:11.176 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:11.176 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:11.177 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:11.511 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:11.512 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:11.512 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839312/daa110ad-c3e7-4989-8b56-4e489d341ce6/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:11.888 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:11.889 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:11.889 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0261071 s, 40.2 MB/s 2021-02-10T11:26:11.890 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:11.890 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:11.890 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:12.225 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:12.225 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:12.226 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839312/daa110ad-c3e7-4989-8b56-4e489d341ce6/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:12.602 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:12.602 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:12.602 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246749 s, 42.5 MB/s 2021-02-10T11:26:12.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:12.603 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:12.603 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:12.938 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:12.939 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:12.939 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839312/daa110ad-c3e7-4989-8b56-4e489d341ce6/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:13.316 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:13.316 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:13.316 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247727 s, 42.3 MB/s 2021-02-10T11:26:13.317 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839313 2021-02-10T11:26:13.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.724 7f5fdce8b700 1 Processor -- start 2021-02-10T11:26:13.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.724 7f5fdce8b700 1 -- start start 2021-02-10T11:26:13.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.725 7f5fdce8b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5fd81b0470 0x7f5fd81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:13.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.725 7f5fdce8b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5fd80f20e0 0x7f5fd80f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:13.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.725 7f5fdce8b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd80f0b50 con 0x7f5fd80ef610 2021-02-10T11:26:13.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.725 7f5fdce8b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd80f0820 con 0x7f5fd80f20e0 2021-02-10T11:26:13.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.725 7f5fdce8b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd80f24e0 con 0x7f5fd81b0470 2021-02-10T11:26:13.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.725 7f5fd7314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5fd80f20e0 0x7f5fd80f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:13.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.725 7f5fd7314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5fd80f20e0 0x7f5fd80f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47706/0 (socket says 172.21.2.101:47706) 2021-02-10T11:26:13.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.725 7f5fd7314700 1 -- 172.21.2.101:0/2120322171 learned_addr learned my addr 172.21.2.101:0/2120322171 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:26:13.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.725 7f5fd6b13700 1 --2- 172.21.2.101:0/2120322171 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5fd81b0470 0x7f5fd81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:13.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.725 7f5fd7314700 1 -- 172.21.2.101:0/2120322171 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5fd81b0470 msgr2=0x7f5fd81b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:26:13.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.725 7f5fd7314700 1 --2- 172.21.2.101:0/2120322171 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5fd81b0470 0x7f5fd81b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:13.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.725 7f5fd7314700 1 -- 172.21.2.101:0/2120322171 >> v1:172.21.2.101:6789/0 conn(0x7f5fd80ef610 legacy=0x7f5fd80efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:13.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.725 7f5fd7314700 1 -- 172.21.2.101:0/2120322171 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd80e52e0 con 0x7f5fd80f20e0 2021-02-10T11:26:13.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.725 7f5fd7314700 1 --2- 172.21.2.101:0/2120322171 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5fd80f20e0 0x7f5fd80f0200 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f5fd0006f00 tx=0x7f5fd0009f10).ready entity=mon.0 client_cookie=1342258dea56da74 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:13.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.725 7f5fd5b11700 1 -- 172.21.2.101:0/2120322171 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5fd000b3b0 con 0x7f5fd80f20e0 2021-02-10T11:26:13.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.725 7f5fd5b11700 1 -- 172.21.2.101:0/2120322171 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd000b630 con 0x7f5fd80f20e0 2021-02-10T11:26:13.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.725 7f5fd5b11700 1 -- 172.21.2.101:0/2120322171 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5fd000bb50 con 0x7f5fd80f20e0 2021-02-10T11:26:13.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.725 7f5fdce8b700 1 -- 172.21.2.101:0/2120322171 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5fd80f20e0 msgr2=0x7f5fd80f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:13.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.725 7f5fdce8b700 1 --2- 172.21.2.101:0/2120322171 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5fd80f20e0 0x7f5fd80f0200 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f5fd0006f00 tx=0x7f5fd0009f10).stop 2021-02-10T11:26:13.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.726 7f5fdce8b700 1 -- 172.21.2.101:0/2120322171 shutdown_connections 2021-02-10T11:26:13.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.726 7f5fdce8b700 1 --2- 172.21.2.101:0/2120322171 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5fd80f20e0 0x7f5fd80f0200 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:13.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.726 7f5fdce8b700 1 --2- 172.21.2.101:0/2120322171 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5fd81b0470 0x7f5fd81b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:13.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.726 7f5fdce8b700 1 -- 172.21.2.101:0/2120322171 >> 172.21.2.101:0/2120322171 conn(0x7f5fd80ce3e0 msgr2=0x7f5fd80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:13.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.726 7f5fdce8b700 1 -- 172.21.2.101:0/2120322171 shutdown_connections 2021-02-10T11:26:13.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.726 7f5fdce8b700 1 -- 172.21.2.101:0/2120322171 wait complete. 2021-02-10T11:26:13.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.744 7f5fdce8b700 1 Processor -- start 2021-02-10T11:26:13.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.744 7f5fdce8b700 1 -- start start 2021-02-10T11:26:13.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.744 7f5fdce8b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5fd80a6e70 0x7f5fd80965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:13.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.744 7f5fdce8b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5fd8096c10 0x7f5fd8097010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:13.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.744 7f5fdce8b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5fd80c0ef0 0x7f5fd80c12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:13.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.744 7f5fdce8b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd80a7270 con 0x7f5fd80c0ef0 2021-02-10T11:26:13.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.744 7f5fdce8b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd8190c50 con 0x7f5fd8096c10 2021-02-10T11:26:13.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.744 7f5fdce8b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd80c1910 con 0x7f5fd80a6e70 2021-02-10T11:26:13.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.744 7f5fd6b13700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5fd80a6e70 0x7f5fd80965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:13.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.744 7f5fd7314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5fd80c0ef0 0x7f5fd80c12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:13.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.744 7f5fd6b13700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5fd80a6e70 0x7f5fd80965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:40442/0 (socket says 172.21.2.101:40442) 2021-02-10T11:26:13.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.744 7f5fd6b13700 1 -- 172.21.2.101:0/980216152 learned_addr learned my addr 172.21.2.101:0/980216152 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:26:13.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.744 7f5fd6312700 1 --2- 172.21.2.101:0/980216152 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5fd8096c10 0x7f5fd8097010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:13.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.745 7f5fd7314700 1 -- 172.21.2.101:0/980216152 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5fd80a6e70 msgr2=0x7f5fd80965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:13.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.745 7f5fd7314700 1 --2- 172.21.2.101:0/980216152 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5fd80a6e70 0x7f5fd80965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:13.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.745 7f5fd7314700 1 -- 172.21.2.101:0/980216152 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5fd8096c10 msgr2=0x7f5fd8097010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:13.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.745 7f5fd7314700 1 --2- 172.21.2.101:0/980216152 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5fd8096c10 0x7f5fd8097010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:13.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.745 7f5fd7314700 1 -- 172.21.2.101:0/980216152 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd80c1b30 con 0x7f5fd80c0ef0 2021-02-10T11:26:13.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.745 7f5fd7314700 1 --2- 172.21.2.101:0/980216152 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5fd80c0ef0 0x7f5fd80c12f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f5fd0009310 tx=0x7f5fd000bc70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:13.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.745 7f5fc77fe700 1 -- 172.21.2.101:0/980216152 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5fd000bdb0 con 0x7f5fd80c0ef0 2021-02-10T11:26:13.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.745 7f5fc77fe700 1 -- 172.21.2.101:0/980216152 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd0006c70 con 0x7f5fd80c0ef0 2021-02-10T11:26:13.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.745 7f5fdce8b700 1 -- 172.21.2.101:0/980216152 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd80c1b30 con 0x7f5fd80c0ef0 2021-02-10T11:26:13.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.745 7f5fc77fe700 1 -- 172.21.2.101:0/980216152 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5fd0005aa0 con 0x7f5fd80c0ef0 2021-02-10T11:26:13.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.745 7f5fdce8b700 1 -- 172.21.2.101:0/980216152 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd80c1b30 con 0x7f5fd80c0ef0 2021-02-10T11:26:13.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.746 7f5fc77fe700 1 -- 172.21.2.101:0/980216152 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5fd0005820 con 0x7f5fd80c0ef0 2021-02-10T11:26:13.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.746 7f5fc77fe700 1 --2- 172.21.2.101:0/980216152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5fbc018730 0x7f5fbc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:13.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.746 7f5fc77fe700 1 -- 172.21.2.101:0/980216152 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5fd0032950 con 0x7f5fd80c0ef0 2021-02-10T11:26:13.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.746 7f5fd6b13700 1 --2- 172.21.2.101:0/980216152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5fbc018730 0x7f5fbc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:13.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.746 7f5fd6b13700 1 --2- 172.21.2.101:0/980216152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5fbc018730 0x7f5fbc01abc0 crc :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:13.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.774 7f5fdce8b700 1 -- 172.21.2.101:0/980216152 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd8105b30 con 0x7f5fd80c0ef0 2021-02-10T11:26:13.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.777 7f5fc77fe700 1 -- 172.21.2.101:0/980216152 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5fd000bdb0 con 0x7f5fd80c0ef0 2021-02-10T11:26:13.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.906 7f5fdce8b700 1 -- 172.21.2.101:0/980216152 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839313", "target": ["mgr", ""]}) v1 -- 0x7f5fd8105b30 con 0x7f5fbc018730 2021-02-10T11:26:13.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.931 7f5fc77fe700 1 -- 172.21.2.101:0/980216152 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f5fc80009b0 con 0x7f5fbc018730 2021-02-10T11:26:13.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.939 7f5fdce8b700 1 -- 172.21.2.101:0/980216152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5fbc018730 msgr2=0x7f5fbc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:13.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.939 7f5fdce8b700 1 --2- 172.21.2.101:0/980216152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5fbc018730 0x7f5fbc01abc0 crc :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:13.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.939 7f5fdce8b700 1 -- 172.21.2.101:0/980216152 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5fd80c0ef0 msgr2=0x7f5fd80c12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:13.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.939 7f5fdce8b700 1 --2- 172.21.2.101:0/980216152 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5fd80c0ef0 0x7f5fd80c12f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f5fd0009310 tx=0x7f5fd000bc70).stop 2021-02-10T11:26:13.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.939 7f5fdce8b700 1 -- 172.21.2.101:0/980216152 shutdown_connections 2021-02-10T11:26:13.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.939 7f5fdce8b700 1 --2- 172.21.2.101:0/980216152 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5fd80c0ef0 0x7f5fd80c12f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:13.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.939 7f5fdce8b700 1 --2- 172.21.2.101:0/980216152 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5fd8096c10 0x7f5fd8097010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:13.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.939 7f5fdce8b700 1 --2- 172.21.2.101:0/980216152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5fbc018730 0x7f5fbc01abc0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:13.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.939 7f5fdce8b700 1 --2- 172.21.2.101:0/980216152 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5fd80a6e70 0x7f5fd80965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:13.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.939 7f5fdce8b700 1 -- 172.21.2.101:0/980216152 >> 172.21.2.101:0/980216152 conn(0x7f5fd80c40e0 msgr2=0x7f5fd80957c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:13.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.939 7f5fdce8b700 1 -- 172.21.2.101:0/980216152 shutdown_connections 2021-02-10T11:26:13.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:13.939 7f5fdce8b700 1 -- 172.21.2.101:0/980216152 wait complete. 2021-02-10T11:26:13.956 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839313 2021-02-10T11:26:14.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.362 7f6911b00700 1 Processor -- start 2021-02-10T11:26:14.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.362 7f6911b00700 1 -- start start 2021-02-10T11:26:14.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.363 7f6911b00700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f690c1b0470 0x7f690c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:14.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.363 7f6911b00700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f690c0f0b50 con 0x7f690c0ef610 2021-02-10T11:26:14.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.363 7f6911b00700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f690c0f24e0 con 0x7f690c0f20e0 2021-02-10T11:26:14.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.363 7f6911b00700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f690c0e5ea0 con 0x7f690c1b0470 2021-02-10T11:26:14.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.363 7f690affd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f690c0ef610 0x7f690c0efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48212/0 (socket says 172.21.2.101:48212) 2021-02-10T11:26:14.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.363 7f690affd700 1 -- 172.21.2.101:0/1221898542 learned_addr learned my addr 172.21.2.101:0/1221898542 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:26:14.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.363 7f690b7fe700 1 --2- 172.21.2.101:0/1221898542 >> v2:172.21.2.108:3300/0 conn(0x7f690c1b0470 0x7f690c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:14.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.363 7f690a7fc700 1 -- 172.21.2.101:0/1221898542 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2533810898 0 0) 0x7f68f40010b0 con 0x7f690c0ef610 2021-02-10T11:26:14.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.363 7f690a7fc700 1 -- 172.21.2.101:0/1221898542 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68f8002fe0 con 0x7f690c0ef610 2021-02-10T11:26:14.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.364 7f690b7fe700 1 -- 172.21.2.101:0/1221898542 >> v1:172.21.2.108:6789/0 conn(0x7f690c0f20e0 legacy=0x7f690c0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:14.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.364 7f690b7fe700 1 -- 172.21.2.101:0/1221898542 >> v1:172.21.2.101:6789/0 conn(0x7f690c0ef610 legacy=0x7f690c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:14.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.364 7f690b7fe700 1 -- 172.21.2.101:0/1221898542 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c0e52e0 con 0x7f690c1b0470 2021-02-10T11:26:14.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.364 7f690b7fe700 1 --2- 172.21.2.101:0/1221898542 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f690c1b0470 0x7f690c1b0870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f68fc006650 tx=0x7f68fc009a80).ready entity=mon.1 client_cookie=9068f042f14a1c0a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:14.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.365 7f690a7fc700 1 -- 172.21.2.101:0/1221898542 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f68fc00c0c0 con 0x7f690c1b0470 2021-02-10T11:26:14.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.365 7f690a7fc700 1 -- 172.21.2.101:0/1221898542 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68fc00c340 con 0x7f690c1b0470 2021-02-10T11:26:14.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.365 7f690a7fc700 1 -- 172.21.2.101:0/1221898542 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f68fc00c860 con 0x7f690c1b0470 2021-02-10T11:26:14.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.365 7f6911b00700 1 -- 172.21.2.101:0/1221898542 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f690c1b0470 msgr2=0x7f690c1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:14.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.365 7f6911b00700 1 --2- 172.21.2.101:0/1221898542 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f690c1b0470 0x7f690c1b0870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f68fc006650 tx=0x7f68fc009a80).stop 2021-02-10T11:26:14.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.365 7f6911b00700 1 -- 172.21.2.101:0/1221898542 shutdown_connections 2021-02-10T11:26:14.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.365 7f6911b00700 1 --2- 172.21.2.101:0/1221898542 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f690c1b0470 0x7f690c1b0870 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:14.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.365 7f6911b00700 1 -- 172.21.2.101:0/1221898542 >> 172.21.2.101:0/1221898542 conn(0x7f690c0ce3e0 msgr2=0x7f690c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:14.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.365 7f6911b00700 1 -- 172.21.2.101:0/1221898542 shutdown_connections 2021-02-10T11:26:14.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.365 7f6911b00700 1 -- 172.21.2.101:0/1221898542 wait complete. 2021-02-10T11:26:14.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.380 7f6911b00700 1 Processor -- start 2021-02-10T11:26:14.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.380 7f6911b00700 1 -- start start 2021-02-10T11:26:14.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.380 7f6911b00700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f690c0a0960 0x7f690c0a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:14.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.380 7f6911b00700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f690c1924c0 0x7f690c1928c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:14.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.380 7f6911b00700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f690c192ee0 0x7f690c0a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:14.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.380 7f6911b00700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c0a6e10 con 0x7f690c192ee0 2021-02-10T11:26:14.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.380 7f6911b00700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c190630 con 0x7f690c1924c0 2021-02-10T11:26:14.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.380 7f6911b00700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690c1932e0 con 0x7f690c0a0960 2021-02-10T11:26:14.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.380 7f690bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f690c192ee0 0x7f690c0a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:14.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.381 7f690bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f690c192ee0 0x7f690c0a29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47724/0 (socket says 172.21.2.101:47724) 2021-02-10T11:26:14.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.381 7f690bfff700 1 -- 172.21.2.101:0/2064801500 learned_addr learned my addr 172.21.2.101:0/2064801500 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:26:14.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.381 7f690bfff700 1 -- 172.21.2.101:0/2064801500 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f690c0a0960 msgr2=0x7f690c0a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:14.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.381 7f690bfff700 1 --2- 172.21.2.101:0/2064801500 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f690c0a0960 0x7f690c0a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:14.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.381 7f690bfff700 1 -- 172.21.2.101:0/2064801500 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f690c1924c0 msgr2=0x7f690c1928c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:14.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.381 7f690bfff700 1 --2- 172.21.2.101:0/2064801500 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f690c1924c0 0x7f690c1928c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:14.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.381 7f690bfff700 1 -- 172.21.2.101:0/2064801500 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c193500 con 0x7f690c192ee0 2021-02-10T11:26:14.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.381 7f690bfff700 1 --2- 172.21.2.101:0/2064801500 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f690c192ee0 0x7f690c0a29f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f6904009940 tx=0x7f690400bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:14.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.381 7f6903fff700 1 -- 172.21.2.101:0/2064801500 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6904000ce0 con 0x7f690c192ee0 2021-02-10T11:26:14.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.381 7f6903fff700 1 -- 172.21.2.101:0/2064801500 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690400c2f0 con 0x7f690c192ee0 2021-02-10T11:26:14.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.381 7f6903fff700 1 -- 172.21.2.101:0/2064801500 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f690400c810 con 0x7f690c192ee0 2021-02-10T11:26:14.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.381 7f6911b00700 1 -- 172.21.2.101:0/2064801500 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690c193500 con 0x7f690c192ee0 2021-02-10T11:26:14.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.381 7f6911b00700 1 -- 172.21.2.101:0/2064801500 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690c0a3300 con 0x7f690c192ee0 2021-02-10T11:26:14.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.382 7f6903fff700 1 -- 172.21.2.101:0/2064801500 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6904001220 con 0x7f690c192ee0 2021-02-10T11:26:14.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.382 7f6903fff700 1 --2- 172.21.2.101:0/2064801500 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68f00186c0 0x7f68f001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:14.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.382 7f6903fff700 1 -- 172.21.2.101:0/2064801500 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6904032780 con 0x7f690c192ee0 2021-02-10T11:26:14.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.382 7f690b7fe700 1 --2- 172.21.2.101:0/2064801500 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68f00186c0 0x7f68f001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:14.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.382 7f690b7fe700 1 --2- 172.21.2.101:0/2064801500 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68f00186c0 0x7f68f001abc0 crc :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:14.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.413 7f6911b00700 1 -- 172.21.2.101:0/2064801500 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690c0a24c0 con 0x7f690c192ee0 2021-02-10T11:26:14.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.415 7f6903fff700 1 -- 172.21.2.101:0/2064801500 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6904029fc0 con 0x7f690c192ee0 2021-02-10T11:26:14.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.542 7f6911b00700 1 -- 172.21.2.101:0/2064801500 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839313", "target": ["mgr", ""]}) v1 -- 0x7f690c190630 con 0x7f68f00186c0 2021-02-10T11:26:14.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.545 7f6903fff700 1 -- 172.21.2.101:0/2064801500 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f68fc000940 con 0x7f68f00186c0 2021-02-10T11:26:14.554 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839313/4cd54b3a-de2d-4daf-80b1-abf75a64d216 2021-02-10T11:26:14.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.547 7f6911b00700 1 -- 172.21.2.101:0/2064801500 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68f00186c0 msgr2=0x7f68f001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:14.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.547 7f6911b00700 1 --2- 172.21.2.101:0/2064801500 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68f00186c0 0x7f68f001abc0 crc :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:14.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.547 7f6911b00700 1 -- 172.21.2.101:0/2064801500 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f690c192ee0 msgr2=0x7f690c0a29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:14.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.547 7f6911b00700 1 --2- 172.21.2.101:0/2064801500 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f690c192ee0 0x7f690c0a29f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f6904009940 tx=0x7f690400bcf0).stop 2021-02-10T11:26:14.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.547 7f6911b00700 1 -- 172.21.2.101:0/2064801500 shutdown_connections 2021-02-10T11:26:14.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.547 7f6911b00700 1 --2- 172.21.2.101:0/2064801500 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f690c192ee0 0x7f690c0a29f0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:14.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.547 7f6911b00700 1 --2- 172.21.2.101:0/2064801500 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f690c1924c0 0x7f690c1928c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:14.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.547 7f6911b00700 1 --2- 172.21.2.101:0/2064801500 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68f00186c0 0x7f68f001abc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:14.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.547 7f6911b00700 1 --2- 172.21.2.101:0/2064801500 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f690c0a0960 0x7f690c0a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:14.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.547 7f6911b00700 1 -- 172.21.2.101:0/2064801500 >> 172.21.2.101:0/2064801500 conn(0x7f690c0ac550 msgr2=0x7f690c0be1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:14.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.547 7f6911b00700 1 -- 172.21.2.101:0/2064801500 shutdown_connections 2021-02-10T11:26:14.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:14.547 7f6911b00700 1 -- 172.21.2.101:0/2064801500 wait complete. 2021-02-10T11:26:14.564 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839313 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839313/4cd54b3a-de2d-4daf-80b1-abf75a64d216 2021-02-10T11:26:14.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:14.564 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:14.564 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:14.899 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:14.899 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:14.900 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839313/4cd54b3a-de2d-4daf-80b1-abf75a64d216/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:15.281 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:15.281 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:15.281 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0230043 s, 45.6 MB/s 2021-02-10T11:26:15.282 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:15.282 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:15.282 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:15.619 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:15.619 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:15.619 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839313/4cd54b3a-de2d-4daf-80b1-abf75a64d216/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:15.994 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:15.994 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:15.994 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0232293 s, 45.1 MB/s 2021-02-10T11:26:15.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:15.995 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:15.995 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:16.330 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:16.330 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:16.331 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839313/4cd54b3a-de2d-4daf-80b1-abf75a64d216/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:16.708 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:16.708 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:16.708 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0256254 s, 40.9 MB/s 2021-02-10T11:26:16.709 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:16.709 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:16.709 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:17.045 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:17.046 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:17.046 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839313/4cd54b3a-de2d-4daf-80b1-abf75a64d216/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:17.420 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:17.420 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:17.420 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234127 s, 44.8 MB/s 2021-02-10T11:26:17.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:17.421 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:17.421 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:17.757 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:17.758 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:17.758 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839313/4cd54b3a-de2d-4daf-80b1-abf75a64d216/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:18.134 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:18.134 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:18.134 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0226845 s, 46.2 MB/s 2021-02-10T11:26:18.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:18.135 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:18.135 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:18.470 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:18.471 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:18.471 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839313/4cd54b3a-de2d-4daf-80b1-abf75a64d216/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:18.845 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:18.845 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:18.845 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251401 s, 41.7 MB/s 2021-02-10T11:26:18.846 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:18.846 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:18.846 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:19.183 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:19.184 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:19.184 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839313/4cd54b3a-de2d-4daf-80b1-abf75a64d216/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:19.560 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:19.560 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:19.561 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242011 s, 43.3 MB/s 2021-02-10T11:26:19.562 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:19.562 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:19.562 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:19.897 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:19.898 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:19.898 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839313/4cd54b3a-de2d-4daf-80b1-abf75a64d216/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:20.274 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:20.274 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:20.274 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247785 s, 42.3 MB/s 2021-02-10T11:26:20.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:20.276 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:20.276 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:20.610 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:20.610 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:20.610 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839313/4cd54b3a-de2d-4daf-80b1-abf75a64d216/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:20.986 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:20.986 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:20.986 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245787 s, 42.7 MB/s 2021-02-10T11:26:20.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:20.987 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:20.987 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:21.323 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:21.323 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:21.324 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839313/4cd54b3a-de2d-4daf-80b1-abf75a64d216/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:21.701 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:21.701 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:21.701 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242768 s, 43.2 MB/s 2021-02-10T11:26:21.702 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839314 2021-02-10T11:26:22.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.107 7f30427c8700 1 Processor -- start 2021-02-10T11:26:22.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.107 7f30427c8700 1 -- start start 2021-02-10T11:26:22.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.108 7f30427c8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f303c1b0470 0x7f303c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:22.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.108 7f30427c8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f303c0ef610 0x7f303c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:22.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.108 7f30427c8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f303c0f0030 0x7f303c0f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:22.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.108 7f30427c8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f303c0e5ea0 con 0x7f303c1b0470 2021-02-10T11:26:22.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.108 7f30427c8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f303c0f0b50 con 0x7f303c0ef610 2021-02-10T11:26:22.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.108 7f30427c8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303c0e52e0 con 0x7f303c0f0030 2021-02-10T11:26:22.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.108 7f303bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f303c1b0470 0x7f303c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:22.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.108 7f303bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f303c1b0470 0x7f303c1b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47730/0 (socket says 172.21.2.101:47730) 2021-02-10T11:26:22.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.108 7f303bfff700 1 -- 172.21.2.101:0/2905063814 learned_addr learned my addr 172.21.2.101:0/2905063814 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:26:22.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.108 7f303b7fe700 1 --2- 172.21.2.101:0/2905063814 >> v2:172.21.2.108:3300/0 conn(0x7f303c0ef610 0x7f303c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:22.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.108 7f3040de5700 1 --2- 172.21.2.101:0/2905063814 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f303c0f0030 0x7f303c0f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:22.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.108 7f303bfff700 1 -- 172.21.2.101:0/2905063814 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f303c0f0030 msgr2=0x7f303c0f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:26:22.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.108 7f303bfff700 1 --2- 172.21.2.101:0/2905063814 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f303c0f0030 0x7f303c0f0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:22.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.108 7f303bfff700 1 -- 172.21.2.101:0/2905063814 >> v2:172.21.2.108:3300/0 conn(0x7f303c0ef610 msgr2=0x7f303c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:26:22.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.108 7f303bfff700 1 --2- 172.21.2.101:0/2905063814 >> v2:172.21.2.108:3300/0 conn(0x7f303c0ef610 0x7f303c0efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:22.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.108 7f303bfff700 1 -- 172.21.2.101:0/2905063814 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303c0f10d0 con 0x7f303c1b0470 2021-02-10T11:26:22.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.109 7f303bfff700 1 --2- 172.21.2.101:0/2905063814 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f303c1b0470 0x7f303c1b0870 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f302c006700 tx=0x7f302c009b80).ready entity=mon.0 client_cookie=d0a9933d97606ecd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:22.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.109 7f303affd700 1 -- 172.21.2.101:0/2905063814 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f302c00c0c0 con 0x7f303c1b0470 2021-02-10T11:26:22.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.109 7f303affd700 1 -- 172.21.2.101:0/2905063814 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302c00c440 con 0x7f303c1b0470 2021-02-10T11:26:22.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.109 7f303affd700 1 -- 172.21.2.101:0/2905063814 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f302c00c960 con 0x7f303c1b0470 2021-02-10T11:26:22.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.109 7f30427c8700 1 -- 172.21.2.101:0/2905063814 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f303c1b0470 msgr2=0x7f303c1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:22.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.109 7f30427c8700 1 --2- 172.21.2.101:0/2905063814 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f303c1b0470 0x7f303c1b0870 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f302c006700 tx=0x7f302c009b80).stop 2021-02-10T11:26:22.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.110 7f30427c8700 1 -- 172.21.2.101:0/2905063814 shutdown_connections 2021-02-10T11:26:22.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.110 7f30427c8700 1 --2- 172.21.2.101:0/2905063814 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f303c0f0030 0x7f303c0f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:22.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.110 7f30427c8700 1 --2- 172.21.2.101:0/2905063814 >> v2:172.21.2.108:3300/0 conn(0x7f303c0ef610 0x7f303c0efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:22.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.110 7f30427c8700 1 --2- 172.21.2.101:0/2905063814 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f303c1b0470 0x7f303c1b0870 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:22.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.110 7f30427c8700 1 -- 172.21.2.101:0/2905063814 >> 172.21.2.101:0/2905063814 conn(0x7f303c0ce3e0 msgr2=0x7f303c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:22.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.110 7f30427c8700 1 -- 172.21.2.101:0/2905063814 shutdown_connections 2021-02-10T11:26:22.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.110 7f30427c8700 1 -- 172.21.2.101:0/2905063814 wait complete. 2021-02-10T11:26:22.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.125 7f30427c8700 1 Processor -- start 2021-02-10T11:26:22.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.125 7f30427c8700 1 -- start start 2021-02-10T11:26:22.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.125 7f30427c8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f303c09b930 0x7f303c0b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:22.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.125 7f30427c8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f303c0b8410 0x7f303c0b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:22.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.125 7f30427c8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f303c0c1f80 0x7f303c0c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:22.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.125 7f30427c8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303c0c1a00 con 0x7f303c0c1f80 2021-02-10T11:26:22.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.125 7f30427c8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303c0b8e30 con 0x7f303c09b930 2021-02-10T11:26:22.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.125 7f30427c8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303c0b9050 con 0x7f303c0b8410 2021-02-10T11:26:22.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.125 7f3040de5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f303c0c1f80 0x7f303c0c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:22.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.125 7f3040de5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f303c0c1f80 0x7f303c0c2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47738/0 (socket says 172.21.2.101:47738) 2021-02-10T11:26:22.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.125 7f3040de5700 1 -- 172.21.2.101:0/182357714 learned_addr learned my addr 172.21.2.101:0/182357714 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:26:22.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.125 7f303b7fe700 1 --2- 172.21.2.101:0/182357714 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f303c0b8410 0x7f303c0b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:22.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.125 7f303bfff700 1 --2- 172.21.2.101:0/182357714 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f303c09b930 0x7f303c0b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:22.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.126 7f3040de5700 1 -- 172.21.2.101:0/182357714 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f303c0b8410 msgr2=0x7f303c0b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:22.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.126 7f3040de5700 1 --2- 172.21.2.101:0/182357714 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f303c0b8410 0x7f303c0b8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:22.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.126 7f3040de5700 1 -- 172.21.2.101:0/182357714 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f303c09b930 msgr2=0x7f303c0b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:22.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.126 7f3040de5700 1 --2- 172.21.2.101:0/182357714 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f303c09b930 0x7f303c0b7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:22.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.126 7f3040de5700 1 -- 172.21.2.101:0/182357714 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303c0c29a0 con 0x7f303c0c1f80 2021-02-10T11:26:22.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.126 7f3040de5700 1 --2- 172.21.2.101:0/182357714 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f303c0c1f80 0x7f303c0c2380 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f303400b910 tx=0x7f303400bcb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:22.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.126 7f3038ff9700 1 -- 172.21.2.101:0/182357714 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3034006aa0 con 0x7f303c0c1f80 2021-02-10T11:26:22.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.126 7f3038ff9700 1 -- 172.21.2.101:0/182357714 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30340011f0 con 0x7f303c0c1f80 2021-02-10T11:26:22.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.126 7f3038ff9700 1 -- 172.21.2.101:0/182357714 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f30340016d0 con 0x7f303c0c1f80 2021-02-10T11:26:22.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.126 7f30427c8700 1 -- 172.21.2.101:0/182357714 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303c0c29a0 con 0x7f303c0c1f80 2021-02-10T11:26:22.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.126 7f30427c8700 1 -- 172.21.2.101:0/182357714 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f303c0c29a0 con 0x7f303c0c1f80 2021-02-10T11:26:22.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.127 7f3038ff9700 1 -- 172.21.2.101:0/182357714 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3034006aa0 con 0x7f303c0c1f80 2021-02-10T11:26:22.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.127 7f3038ff9700 1 --2- 172.21.2.101:0/182357714 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f301c0186c0 0x7f301c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:22.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.127 7f3038ff9700 1 -- 172.21.2.101:0/182357714 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3034032930 con 0x7f303c0c1f80 2021-02-10T11:26:22.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.127 7f303bfff700 1 --2- 172.21.2.101:0/182357714 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f301c0186c0 0x7f301c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:22.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.127 7f303bfff700 1 --2- 172.21.2.101:0/182357714 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f301c0186c0 0x7f301c01abc0 crc :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:22.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.157 7f30427c8700 1 -- 172.21.2.101:0/182357714 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f303c105b30 con 0x7f303c0c1f80 2021-02-10T11:26:22.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.160 7f3038ff9700 1 -- 172.21.2.101:0/182357714 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f303400bdf0 con 0x7f303c0c1f80 2021-02-10T11:26:22.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.287 7f30427c8700 1 -- 172.21.2.101:0/182357714 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839314", "target": ["mgr", ""]}) v1 -- 0x7f303c105b30 con 0x7f301c0186c0 2021-02-10T11:26:22.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.312 7f3038ff9700 1 -- 172.21.2.101:0/182357714 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f302c006430 con 0x7f301c0186c0 2021-02-10T11:26:22.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.320 7f30427c8700 1 -- 172.21.2.101:0/182357714 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f301c0186c0 msgr2=0x7f301c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:22.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.320 7f30427c8700 1 --2- 172.21.2.101:0/182357714 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f301c0186c0 0x7f301c01abc0 crc :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:22.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.320 7f30427c8700 1 -- 172.21.2.101:0/182357714 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f303c0c1f80 msgr2=0x7f303c0c2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:22.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.320 7f30427c8700 1 --2- 172.21.2.101:0/182357714 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f303c0c1f80 0x7f303c0c2380 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f303400b910 tx=0x7f303400bcb0).stop 2021-02-10T11:26:22.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.320 7f30427c8700 1 -- 172.21.2.101:0/182357714 shutdown_connections 2021-02-10T11:26:22.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.320 7f30427c8700 1 --2- 172.21.2.101:0/182357714 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f301c0186c0 0x7f301c01abc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:22.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.320 7f30427c8700 1 --2- 172.21.2.101:0/182357714 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f303c0b8410 0x7f303c0b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:22.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.320 7f30427c8700 1 --2- 172.21.2.101:0/182357714 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f303c0c1f80 0x7f303c0c2380 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:22.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.320 7f30427c8700 1 --2- 172.21.2.101:0/182357714 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f303c09b930 0x7f303c0b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:22.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.320 7f30427c8700 1 -- 172.21.2.101:0/182357714 >> 172.21.2.101:0/182357714 conn(0x7f303c094e10 msgr2=0x7f303c0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:22.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.320 7f30427c8700 1 -- 172.21.2.101:0/182357714 shutdown_connections 2021-02-10T11:26:22.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.320 7f30427c8700 1 -- 172.21.2.101:0/182357714 wait complete. 2021-02-10T11:26:22.337 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839314 2021-02-10T11:26:22.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.744 7f9b09c5c700 1 Processor -- start 2021-02-10T11:26:22.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.744 7f9b09c5c700 1 -- start start 2021-02-10T11:26:22.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.744 7f9b09c5c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b041b0870 0x7f9b040f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:22.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.744 7f9b09c5c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b041b0c70 con 0x7f9b040efe00 2021-02-10T11:26:22.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.744 7f9b09c5c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b040f0b50 con 0x7f9b041b0470 2021-02-10T11:26:22.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.744 7f9b09c5c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b040e5ea0 con 0x7f9b041b0870 2021-02-10T11:26:22.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.744 7f9b03fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f9b040efe00 0x7f9b040f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48242/0 (socket says 172.21.2.101:48242) 2021-02-10T11:26:22.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.744 7f9b03fff700 1 -- 172.21.2.101:0/2729266515 learned_addr learned my addr 172.21.2.101:0/2729266515 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:26:22.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b027fc700 1 -- 172.21.2.101:0/2729266515 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1005130428 0 0) 0x7f9afc001500 con 0x7f9b040efe00 2021-02-10T11:26:22.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b027fc700 1 -- 172.21.2.101:0/2729266515 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9af0002fe0 con 0x7f9b040efe00 2021-02-10T11:26:22.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b02ffd700 1 --2- 172.21.2.101:0/2729266515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b041b0870 0x7f9b040f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:22.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b027fc700 1 -- 172.21.2.101:0/2729266515 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 920901091 0 0) 0x7f9afc001460 con 0x7f9b040efe00 2021-02-10T11:26:22.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b027fc700 1 -- 172.21.2.101:0/2729266515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b041b0870 msgr2=0x7f9b040f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:26:22.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b027fc700 1 --2- 172.21.2.101:0/2729266515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b041b0870 0x7f9b040f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:22.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b027fc700 1 -- 172.21.2.101:0/2729266515 >> v1:172.21.2.108:6789/0 conn(0x7f9b041b0470 legacy=0x7f9b040ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:26:22.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b027fc700 1 -- 172.21.2.101:0/2729266515 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b040e52e0 con 0x7f9b040efe00 2021-02-10T11:26:22.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b027fc700 1 -- 172.21.2.101:0/2729266515 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f9afc001980 con 0x7f9b040efe00 2021-02-10T11:26:22.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b027fc700 1 -- 172.21.2.101:0/2729266515 >> v1:172.21.2.101:6789/0 conn(0x7f9b040efe00 legacy=0x7f9b040f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:22.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b027fc700 1 --2- 172.21.2.101:0/2729266515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9af0003a60 0x7f9af0005e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:22.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b027fc700 1 --2- 172.21.2.101:0/2729266515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9af0006730 0x7f9af0008c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:22.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b027fc700 1 --2- 172.21.2.101:0/2729266515 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9af00094f0 0x7f9af000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:22.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b027fc700 1 -- 172.21.2.101:0/2729266515 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af000c2c0 con 0x7f9af0006730 2021-02-10T11:26:22.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b027fc700 1 -- 172.21.2.101:0/2729266515 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af000c4e0 con 0x7f9af00094f0 2021-02-10T11:26:22.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b027fc700 1 -- 172.21.2.101:0/2729266515 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af000c700 con 0x7f9af0003a60 2021-02-10T11:26:22.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b02ffd700 1 --2- 172.21.2.101:0/2729266515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9af0006730 0x7f9af0008c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:22.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b037fe700 1 --2- 172.21.2.101:0/2729266515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9af0003a60 0x7f9af0005e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:22.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b03fff700 1 --2- 172.21.2.101:0/2729266515 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9af00094f0 0x7f9af000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:22.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b02ffd700 1 -- 172.21.2.101:0/2729266515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9af0003a60 msgr2=0x7f9af0005e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:26:22.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b02ffd700 1 --2- 172.21.2.101:0/2729266515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9af0003a60 0x7f9af0005e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:22.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b02ffd700 1 -- 172.21.2.101:0/2729266515 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9af00094f0 msgr2=0x7f9af000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:26:22.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b02ffd700 1 --2- 172.21.2.101:0/2729266515 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9af00094f0 0x7f9af000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:22.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b02ffd700 1 -- 172.21.2.101:0/2729266515 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af000c9a0 con 0x7f9af0006730 2021-02-10T11:26:22.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.745 7f9b037fe700 1 --2- 172.21.2.101:0/2729266515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9af0003a60 0x7f9af0005e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:26:22.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.746 7f9b02ffd700 1 --2- 172.21.2.101:0/2729266515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9af0006730 0x7f9af0008c50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7f9aec04d720 tx=0x7f9aec04dac0).ready entity=mon.0 client_cookie=6bab9253eb7afef8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:22.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.746 7f9b027fc700 1 -- 172.21.2.101:0/2729266515 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9aec0066d0 con 0x7f9af0006730 2021-02-10T11:26:22.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.746 7f9b027fc700 1 -- 172.21.2.101:0/2729266515 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aec002030 con 0x7f9af0006730 2021-02-10T11:26:22.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.746 7f9b027fc700 1 -- 172.21.2.101:0/2729266515 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9aec002510 con 0x7f9af0006730 2021-02-10T11:26:22.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.746 7f9b09c5c700 1 -- 172.21.2.101:0/2729266515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9af0006730 msgr2=0x7f9af0008c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:22.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.746 7f9b09c5c700 1 --2- 172.21.2.101:0/2729266515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9af0006730 0x7f9af0008c50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7f9aec04d720 tx=0x7f9aec04dac0).stop 2021-02-10T11:26:22.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.746 7f9b03fff700 1 -- 172.21.2.101:0/2729266515 reap_dead start 2021-02-10T11:26:22.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.746 7f9b09c5c700 1 -- 172.21.2.101:0/2729266515 shutdown_connections 2021-02-10T11:26:22.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.746 7f9b09c5c700 1 -- 172.21.2.101:0/2729266515 >> 172.21.2.101:0/2729266515 conn(0x7f9b040ce3e0 msgr2=0x7f9b040e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:22.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.746 7f9b09c5c700 1 -- 172.21.2.101:0/2729266515 shutdown_connections 2021-02-10T11:26:22.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.746 7f9b09c5c700 1 -- 172.21.2.101:0/2729266515 wait complete. 2021-02-10T11:26:22.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.761 7f9b09c5c700 1 Processor -- start 2021-02-10T11:26:22.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.761 7f9b09c5c700 1 -- start start 2021-02-10T11:26:22.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.762 7f9b09c5c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b0409b930 0x7f9b040b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:22.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.762 7f9b09c5c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b040b8410 0x7f9b040b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:22.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.762 7f9b09c5c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b040c1f80 0x7f9b040c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:22.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.762 7f9b09c5c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b040c1a00 con 0x7f9b040c1f80 2021-02-10T11:26:22.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.762 7f9b09c5c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b040b8e30 con 0x7f9b0409b930 2021-02-10T11:26:22.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.762 7f9b09c5c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b040b9050 con 0x7f9b040b8410 2021-02-10T11:26:22.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.762 7f9b03fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b040c1f80 0x7f9b040c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:22.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.762 7f9b03fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b040c1f80 0x7f9b040c2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47756/0 (socket says 172.21.2.101:47756) 2021-02-10T11:26:22.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.762 7f9b03fff700 1 -- 172.21.2.101:0/2337773587 learned_addr learned my addr 172.21.2.101:0/2337773587 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:26:22.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.762 7f9b03fff700 1 -- 172.21.2.101:0/2337773587 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b040b8410 msgr2=0x7f9b040b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:22.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.762 7f9b03fff700 1 --2- 172.21.2.101:0/2337773587 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b040b8410 0x7f9b040b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:22.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.762 7f9b03fff700 1 -- 172.21.2.101:0/2337773587 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b0409b930 msgr2=0x7f9b040b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:22.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.762 7f9b03fff700 1 --2- 172.21.2.101:0/2337773587 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b0409b930 0x7f9b040b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:22.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.762 7f9b03fff700 1 -- 172.21.2.101:0/2337773587 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b040c29a0 con 0x7f9b040c1f80 2021-02-10T11:26:22.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.762 7f9b03fff700 1 --2- 172.21.2.101:0/2337773587 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b040c1f80 0x7f9b040c2380 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f9afc00a8f0 tx=0x7f9afc00ac90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:22.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.763 7f9afbfff700 1 -- 172.21.2.101:0/2337773587 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9afc001770 con 0x7f9b040c1f80 2021-02-10T11:26:22.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.763 7f9b09c5c700 1 -- 172.21.2.101:0/2337773587 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b040c29a0 con 0x7f9b040c1f80 2021-02-10T11:26:22.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.763 7f9b09c5c700 1 -- 172.21.2.101:0/2337773587 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b040c29a0 con 0x7f9b040c1f80 2021-02-10T11:26:22.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.763 7f9afbfff700 1 -- 172.21.2.101:0/2337773587 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9afc001770 con 0x7f9b040c1f80 2021-02-10T11:26:22.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.763 7f9afbfff700 1 -- 172.21.2.101:0/2337773587 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9afc004930 con 0x7f9b040c1f80 2021-02-10T11:26:22.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.763 7f9afbfff700 1 -- 172.21.2.101:0/2337773587 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9afc00add0 con 0x7f9b040c1f80 2021-02-10T11:26:22.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.763 7f9afbfff700 1 --2- 172.21.2.101:0/2337773587 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ae8018730 0x7f9ae801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:22.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.763 7f9afbfff700 1 -- 172.21.2.101:0/2337773587 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9afc032b80 con 0x7f9b040c1f80 2021-02-10T11:26:22.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.763 7f9b037fe700 1 --2- 172.21.2.101:0/2337773587 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ae8018730 0x7f9ae801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:22.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.764 7f9b037fe700 1 --2- 172.21.2.101:0/2337773587 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ae8018730 0x7f9ae801ac10 crc :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:22.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.794 7f9b09c5c700 1 -- 172.21.2.101:0/2337773587 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b04105b30 con 0x7f9b040c1f80 2021-02-10T11:26:22.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.796 7f9afbfff700 1 -- 172.21.2.101:0/2337773587 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9afc001750 con 0x7f9b040c1f80 2021-02-10T11:26:22.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.924 7f9b09c5c700 1 -- 172.21.2.101:0/2337773587 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839314", "target": ["mgr", ""]}) v1 -- 0x7f9b040c1a00 con 0x7f9ae8018730 2021-02-10T11:26:22.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.928 7f9afbfff700 1 -- 172.21.2.101:0/2337773587 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f9af4005330 con 0x7f9ae8018730 2021-02-10T11:26:22.940 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839314/48cf629d-5709-4949-9bf6-d5e24f5cd9c7 2021-02-10T11:26:22.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.933 7f9b09c5c700 1 -- 172.21.2.101:0/2337773587 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ae8018730 msgr2=0x7f9ae801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:22.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.933 7f9b09c5c700 1 --2- 172.21.2.101:0/2337773587 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ae8018730 0x7f9ae801ac10 crc :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:22.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.933 7f9b09c5c700 1 -- 172.21.2.101:0/2337773587 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b040c1f80 msgr2=0x7f9b040c2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:22.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.933 7f9b09c5c700 1 --2- 172.21.2.101:0/2337773587 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b040c1f80 0x7f9b040c2380 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f9afc00a8f0 tx=0x7f9afc00ac90).stop 2021-02-10T11:26:22.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.933 7f9b09c5c700 1 -- 172.21.2.101:0/2337773587 shutdown_connections 2021-02-10T11:26:22.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.933 7f9b09c5c700 1 --2- 172.21.2.101:0/2337773587 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ae8018730 0x7f9ae801ac10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:22.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.933 7f9b09c5c700 1 --2- 172.21.2.101:0/2337773587 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b040b8410 0x7f9b040b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:22.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.933 7f9b09c5c700 1 --2- 172.21.2.101:0/2337773587 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b040c1f80 0x7f9b040c2380 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:22.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.933 7f9b09c5c700 1 --2- 172.21.2.101:0/2337773587 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b0409b930 0x7f9b040b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:22.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.933 7f9b09c5c700 1 -- 172.21.2.101:0/2337773587 >> 172.21.2.101:0/2337773587 conn(0x7f9b04094e10 msgr2=0x7f9b040a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:22.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.933 7f9b09c5c700 1 -- 172.21.2.101:0/2337773587 shutdown_connections 2021-02-10T11:26:22.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:22.933 7f9b09c5c700 1 -- 172.21.2.101:0/2337773587 wait complete. 2021-02-10T11:26:22.951 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839314 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839314/48cf629d-5709-4949-9bf6-d5e24f5cd9c7 2021-02-10T11:26:22.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:22.951 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:22.951 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:22.956 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:26:22.997 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:26:23.288 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:23.288 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:23.288 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839314/48cf629d-5709-4949-9bf6-d5e24f5cd9c7/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:23.672 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:23.673 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:23.673 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238612 s, 43.9 MB/s 2021-02-10T11:26:23.674 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:23.674 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:23.674 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:24.010 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:24.011 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:24.011 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839314/48cf629d-5709-4949-9bf6-d5e24f5cd9c7/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:24.386 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:24.386 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:24.386 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0266926 s, 39.3 MB/s 2021-02-10T11:26:24.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:24.388 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:24.388 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:24.726 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:24.727 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:24.727 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839314/48cf629d-5709-4949-9bf6-d5e24f5cd9c7/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:25.109 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:25.109 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:25.109 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242667 s, 43.2 MB/s 2021-02-10T11:26:25.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:25.110 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:25.110 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:25.446 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:25.447 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:25.447 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839314/48cf629d-5709-4949-9bf6-d5e24f5cd9c7/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:25.824 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:25.824 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:25.824 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240277 s, 43.6 MB/s 2021-02-10T11:26:25.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:25.825 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:25.825 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:26.160 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:26.161 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:26.161 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839314/48cf629d-5709-4949-9bf6-d5e24f5cd9c7/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:26.536 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:26.536 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:26.536 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238123 s, 44.0 MB/s 2021-02-10T11:26:26.537 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:26.537 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:26.537 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:26.873 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:26.874 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:26.874 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839314/48cf629d-5709-4949-9bf6-d5e24f5cd9c7/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:27.249 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:27.249 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:27.249 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0250258 s, 41.9 MB/s 2021-02-10T11:26:27.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:27.251 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:27.251 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:27.586 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:27.587 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:27.587 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839314/48cf629d-5709-4949-9bf6-d5e24f5cd9c7/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:27.964 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:27.965 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:27.965 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.025524 s, 41.1 MB/s 2021-02-10T11:26:27.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:27.966 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:27.966 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:28.302 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:28.303 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:28.303 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839314/48cf629d-5709-4949-9bf6-d5e24f5cd9c7/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:28.680 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:28.680 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:28.680 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0250061 s, 41.9 MB/s 2021-02-10T11:26:28.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:28.681 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:28.681 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:29.016 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:29.017 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:29.017 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839314/48cf629d-5709-4949-9bf6-d5e24f5cd9c7/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:29.389 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:29.389 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:29.389 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241032 s, 43.5 MB/s 2021-02-10T11:26:29.390 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:29.390 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:29.390 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:29.726 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:29.741 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:29.741 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839314/48cf629d-5709-4949-9bf6-d5e24f5cd9c7/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:30.115 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:30.116 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:30.116 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237303 s, 44.2 MB/s 2021-02-10T11:26:30.117 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839315 2021-02-10T11:26:30.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.524 7fe00fbf6700 1 Processor -- start 2021-02-10T11:26:30.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.524 7fe00fbf6700 1 -- start start 2021-02-10T11:26:30.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.524 7fe00fbf6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0081b0470 0x7fe0081b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:30.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.525 7fe00fbf6700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe0080ef610 0x7fe0080efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:30.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.525 7fe00fbf6700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0080f0b50 con 0x7fe0080f0030 2021-02-10T11:26:30.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.525 7fe00fbf6700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0080e52e0 con 0x7fe0080ef610 2021-02-10T11:26:30.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.525 7fe00fbf6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080e5490 con 0x7fe0081b0470 2021-02-10T11:26:30.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.525 7fe00e213700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe0080f0030 0x7fe0080f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48260/0 (socket says 172.21.2.101:48260) 2021-02-10T11:26:30.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.525 7fe00e213700 1 -- 172.21.2.101:0/2569110804 learned_addr learned my addr 172.21.2.101:0/2569110804 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:26:30.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.525 7fe00ca10700 1 -- 172.21.2.101:0/2569110804 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1613637912 0 0) 0x7fe000001500 con 0x7fe0080f0030 2021-02-10T11:26:30.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.525 7fe00ca10700 1 -- 172.21.2.101:0/2569110804 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdff4002fe0 con 0x7fe0080f0030 2021-02-10T11:26:30.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.525 7fe00da12700 1 --2- 172.21.2.101:0/2569110804 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0081b0470 0x7fe0081b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:30.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.525 7fe00d211700 1 --2- 172.21.2.101:0/2569110804 >> v2:172.21.2.108:3300/0 conn(0x7fe0080ef610 0x7fe0080efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:30.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00ca10700 1 -- 172.21.2.101:0/2569110804 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 846815675 0 0) 0x7fe000001460 con 0x7fe0080f0030 2021-02-10T11:26:30.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00ca10700 1 -- 172.21.2.101:0/2569110804 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0081b0470 msgr2=0x7fe0081b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:26:30.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00ca10700 1 --2- 172.21.2.101:0/2569110804 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0081b0470 0x7fe0081b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:30.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00ca10700 1 -- 172.21.2.101:0/2569110804 >> v2:172.21.2.108:3300/0 conn(0x7fe0080ef610 msgr2=0x7fe0080efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:26:30.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00ca10700 1 --2- 172.21.2.101:0/2569110804 >> v2:172.21.2.108:3300/0 conn(0x7fe0080ef610 0x7fe0080efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:30.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00ca10700 1 -- 172.21.2.101:0/2569110804 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0080f0430 con 0x7fe0080f0030 2021-02-10T11:26:30.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00ca10700 1 -- 172.21.2.101:0/2569110804 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fe000001980 con 0x7fe0080f0030 2021-02-10T11:26:30.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00ca10700 1 -- 172.21.2.101:0/2569110804 >> v1:172.21.2.101:6789/0 conn(0x7fe0080f0030 legacy=0x7fe0080f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:30.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00ca10700 1 --2- 172.21.2.101:0/2569110804 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdff4003ac0 0x7fdff4005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:30.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00ca10700 1 --2- 172.21.2.101:0/2569110804 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdff4006780 0x7fdff4008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:30.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00ca10700 1 --2- 172.21.2.101:0/2569110804 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdff4009510 0x7fdff400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:30.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00ca10700 1 -- 172.21.2.101:0/2569110804 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff40502d0 con 0x7fdff4003ac0 2021-02-10T11:26:30.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00ca10700 1 -- 172.21.2.101:0/2569110804 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff40504f0 con 0x7fdff4009510 2021-02-10T11:26:30.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00ca10700 1 -- 172.21.2.101:0/2569110804 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff4050710 con 0x7fdff4006780 2021-02-10T11:26:30.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00da12700 1 --2- 172.21.2.101:0/2569110804 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdff4003ac0 0x7fdff4005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:30.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00d211700 1 --2- 172.21.2.101:0/2569110804 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdff4006780 0x7fdff4008c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:30.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00e213700 1 --2- 172.21.2.101:0/2569110804 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdff4009510 0x7fdff400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:30.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00da12700 1 -- 172.21.2.101:0/2569110804 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdff4006780 msgr2=0x7fdff4008c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:26:30.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00da12700 1 --2- 172.21.2.101:0/2569110804 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdff4006780 0x7fdff4008c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:30.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00da12700 1 -- 172.21.2.101:0/2569110804 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdff4009510 msgr2=0x7fdff400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:26:30.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00da12700 1 --2- 172.21.2.101:0/2569110804 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdff4009510 0x7fdff400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:30.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00da12700 1 -- 172.21.2.101:0/2569110804 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff40509b0 con 0x7fdff4003ac0 2021-02-10T11:26:30.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00e213700 1 --2- 172.21.2.101:0/2569110804 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdff4009510 0x7fdff400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:26:30.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00da12700 1 --2- 172.21.2.101:0/2569110804 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdff4003ac0 0x7fdff4005f60 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7fdffc009310 tx=0x7fdffc009a10).ready entity=mon.0 client_cookie=9ca1f07324dd0819 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:30.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00ca10700 1 -- 172.21.2.101:0/2569110804 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdffc0009b0 con 0x7fdff4003ac0 2021-02-10T11:26:30.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00ca10700 1 -- 172.21.2.101:0/2569110804 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdffc006400 con 0x7fdff4003ac0 2021-02-10T11:26:30.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00ca10700 1 -- 172.21.2.101:0/2569110804 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdffc00c2b0 con 0x7fdff4003ac0 2021-02-10T11:26:30.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00fbf6700 1 -- 172.21.2.101:0/2569110804 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdff4003ac0 msgr2=0x7fdff4005f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:30.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00fbf6700 1 --2- 172.21.2.101:0/2569110804 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdff4003ac0 0x7fdff4005f60 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7fdffc009310 tx=0x7fdffc009a10).stop 2021-02-10T11:26:30.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.526 7fe00e213700 1 -- 172.21.2.101:0/2569110804 reap_dead start 2021-02-10T11:26:30.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.527 7fe00fbf6700 1 -- 172.21.2.101:0/2569110804 shutdown_connections 2021-02-10T11:26:30.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.527 7fe00fbf6700 1 -- 172.21.2.101:0/2569110804 >> 172.21.2.101:0/2569110804 conn(0x7fe0080ce3e0 msgr2=0x7fe0080e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:30.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.527 7fe00fbf6700 1 -- 172.21.2.101:0/2569110804 shutdown_connections 2021-02-10T11:26:30.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.527 7fe00fbf6700 1 -- 172.21.2.101:0/2569110804 wait complete. 2021-02-10T11:26:30.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.542 7fe00fbf6700 1 Processor -- start 2021-02-10T11:26:30.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.542 7fe00fbf6700 1 -- start start 2021-02-10T11:26:30.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.542 7fe00fbf6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe00809b930 0x7fe0080b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:30.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.542 7fe00fbf6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0080b8410 0x7fe0080b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:30.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.542 7fe00fbf6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0080c1f80 0x7fe0080c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:30.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.542 7fe00fbf6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080c1a00 con 0x7fe0080b8410 2021-02-10T11:26:30.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.542 7fe00fbf6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080b8e30 con 0x7fe00809b930 2021-02-10T11:26:30.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.542 7fe00fbf6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080b9050 con 0x7fe0080c1f80 2021-02-10T11:26:30.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.543 7fe00d211700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0080b8410 0x7fe0080b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:30.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.543 7fe00d211700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0080b8410 0x7fe0080b8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47774/0 (socket says 172.21.2.101:47774) 2021-02-10T11:26:30.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.543 7fe00d211700 1 -- 172.21.2.101:0/1072673062 learned_addr learned my addr 172.21.2.101:0/1072673062 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:26:30.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.543 7fe00d211700 1 -- 172.21.2.101:0/1072673062 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0080c1f80 msgr2=0x7fe0080c2380 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:26:30.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.543 7fe00d211700 1 --2- 172.21.2.101:0/1072673062 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0080c1f80 0x7fe0080c2380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:30.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.543 7fe00d211700 1 -- 172.21.2.101:0/1072673062 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe00809b930 msgr2=0x7fe0080b7c20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:26:30.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.543 7fe00d211700 1 --2- 172.21.2.101:0/1072673062 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe00809b930 0x7fe0080b7c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:30.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.543 7fe00d211700 1 -- 172.21.2.101:0/1072673062 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0080c29a0 con 0x7fe0080b8410 2021-02-10T11:26:30.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.543 7fe00d211700 1 --2- 172.21.2.101:0/1072673062 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0080b8410 0x7fe0080b8810 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7fe004006950 tx=0x7fe00400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:30.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.543 7fdffa7fc700 1 -- 172.21.2.101:0/1072673062 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe00400bd50 con 0x7fe0080b8410 2021-02-10T11:26:30.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.543 7fdffa7fc700 1 -- 172.21.2.101:0/1072673062 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe004006030 con 0x7fe0080b8410 2021-02-10T11:26:30.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.543 7fe00fbf6700 1 -- 172.21.2.101:0/1072673062 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0080c29a0 con 0x7fe0080b8410 2021-02-10T11:26:30.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.543 7fdffa7fc700 1 -- 172.21.2.101:0/1072673062 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe0040054d0 con 0x7fe0080b8410 2021-02-10T11:26:30.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.543 7fe00fbf6700 1 -- 172.21.2.101:0/1072673062 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0080c29a0 con 0x7fe0080b8410 2021-02-10T11:26:30.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.544 7fdffa7fc700 1 -- 172.21.2.101:0/1072673062 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe0040052e0 con 0x7fe0080b8410 2021-02-10T11:26:30.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.544 7fdffa7fc700 1 --2- 172.21.2.101:0/1072673062 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdff0018730 0x7fdff001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:30.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.544 7fdffa7fc700 1 -- 172.21.2.101:0/1072673062 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe004032670 con 0x7fe0080b8410 2021-02-10T11:26:30.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.544 7fe00da12700 1 --2- 172.21.2.101:0/1072673062 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdff0018730 0x7fdff001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:30.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.544 7fe00da12700 1 --2- 172.21.2.101:0/1072673062 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdff0018730 0x7fdff001ac10 crc :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:30.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.574 7fe00fbf6700 1 -- 172.21.2.101:0/1072673062 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe008105b30 con 0x7fe0080b8410 2021-02-10T11:26:30.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.577 7fdffa7fc700 1 -- 172.21.2.101:0/1072673062 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe0040065e0 con 0x7fe0080b8410 2021-02-10T11:26:30.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.704 7fe00fbf6700 1 -- 172.21.2.101:0/1072673062 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839315", "target": ["mgr", ""]}) v1 -- 0x7fe008105b30 con 0x7fdff0018730 2021-02-10T11:26:30.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.731 7fdffa7fc700 1 -- 172.21.2.101:0/1072673062 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fdffc0092d0 con 0x7fdff0018730 2021-02-10T11:26:30.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.737 7fe00fbf6700 1 -- 172.21.2.101:0/1072673062 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdff0018730 msgr2=0x7fdff001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:30.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.737 7fe00fbf6700 1 --2- 172.21.2.101:0/1072673062 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdff0018730 0x7fdff001ac10 crc :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:30.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.737 7fe00fbf6700 1 -- 172.21.2.101:0/1072673062 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0080b8410 msgr2=0x7fe0080b8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:30.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.737 7fe00fbf6700 1 --2- 172.21.2.101:0/1072673062 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0080b8410 0x7fe0080b8810 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7fe004006950 tx=0x7fe00400b960).stop 2021-02-10T11:26:30.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.737 7fe00fbf6700 1 -- 172.21.2.101:0/1072673062 shutdown_connections 2021-02-10T11:26:30.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.737 7fe00fbf6700 1 --2- 172.21.2.101:0/1072673062 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdff0018730 0x7fdff001ac10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:30.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.737 7fe00fbf6700 1 --2- 172.21.2.101:0/1072673062 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0080c1f80 0x7fe0080c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:30.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.737 7fe00fbf6700 1 --2- 172.21.2.101:0/1072673062 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0080b8410 0x7fe0080b8810 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:30.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.737 7fe00fbf6700 1 --2- 172.21.2.101:0/1072673062 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe00809b930 0x7fe0080b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:30.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.737 7fe00fbf6700 1 -- 172.21.2.101:0/1072673062 >> 172.21.2.101:0/1072673062 conn(0x7fe008094e10 msgr2=0x7fe0080a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:30.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.737 7fe00fbf6700 1 -- 172.21.2.101:0/1072673062 shutdown_connections 2021-02-10T11:26:30.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:30.737 7fe00fbf6700 1 -- 172.21.2.101:0/1072673062 wait complete. 2021-02-10T11:26:30.754 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839315 2021-02-10T11:26:31.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.161 7f142312f700 1 Processor -- start 2021-02-10T11:26:31.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.161 7f142312f700 1 -- start start 2021-02-10T11:26:31.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.161 7f142312f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f141c1b0470 0x7f141c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:31.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.161 7f142312f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f141c0f20e0 0x7f141c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:31.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.161 7f142312f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141c0f0b50 con 0x7f141c0ef610 2021-02-10T11:26:31.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.161 7f142312f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f141c0f0820 con 0x7f141c0f20e0 2021-02-10T11:26:31.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.161 7f142312f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f141c0f24e0 con 0x7f141c1b0470 2021-02-10T11:26:31.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.162 7f142174c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f141c0f20e0 0x7f141c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:31.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.162 7f141bfff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f141c0ef610 0x7f141c0efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48276/0 (socket says 172.21.2.101:48276) 2021-02-10T11:26:31.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.162 7f141bfff700 1 -- 172.21.2.101:0/959337443 learned_addr learned my addr 172.21.2.101:0/959337443 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:26:31.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.162 7f1420f4b700 1 --2- 172.21.2.101:0/959337443 >> v2:172.21.2.108:3300/0 conn(0x7f141c1b0470 0x7f141c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:31.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.162 7f142174c700 1 -- 172.21.2.101:0/959337443 >> v2:172.21.2.108:3300/0 conn(0x7f141c1b0470 msgr2=0x7f141c1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:26:31.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.162 7f142174c700 1 --2- 172.21.2.101:0/959337443 >> v2:172.21.2.108:3300/0 conn(0x7f141c1b0470 0x7f141c1b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:31.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.162 7f142174c700 1 -- 172.21.2.101:0/959337443 >> v1:172.21.2.101:6789/0 conn(0x7f141c0ef610 legacy=0x7f141c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:31.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.162 7f142174c700 1 -- 172.21.2.101:0/959337443 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c0e52e0 con 0x7f141c0f20e0 2021-02-10T11:26:31.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.162 7f1420f4b700 1 --2- 172.21.2.101:0/959337443 >> v2:172.21.2.108:3300/0 conn(0x7f141c1b0470 0x7f141c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:26:31.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.162 7f142174c700 1 --2- 172.21.2.101:0/959337443 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f141c0f20e0 0x7f141c0f0200 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f1414006e30 tx=0x7f1414009e60).ready entity=mon.0 client_cookie=413bf73f8adc405e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:31.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.163 7f141b7fe700 1 -- 172.21.2.101:0/959337443 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1414006a50 con 0x7f141c0f20e0 2021-02-10T11:26:31.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.163 7f141b7fe700 1 -- 172.21.2.101:0/959337443 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141400b300 con 0x7f141c0f20e0 2021-02-10T11:26:31.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.163 7f141b7fe700 1 -- 172.21.2.101:0/959337443 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f141400b7e0 con 0x7f141c0f20e0 2021-02-10T11:26:31.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.163 7f142312f700 1 -- 172.21.2.101:0/959337443 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f141c0f20e0 msgr2=0x7f141c0f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:31.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.163 7f142312f700 1 --2- 172.21.2.101:0/959337443 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f141c0f20e0 0x7f141c0f0200 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f1414006e30 tx=0x7f1414009e60).stop 2021-02-10T11:26:31.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.163 7f142312f700 1 -- 172.21.2.101:0/959337443 shutdown_connections 2021-02-10T11:26:31.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.163 7f142312f700 1 --2- 172.21.2.101:0/959337443 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f141c0f20e0 0x7f141c0f0200 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:31.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.163 7f142312f700 1 --2- 172.21.2.101:0/959337443 >> v2:172.21.2.108:3300/0 conn(0x7f141c1b0470 0x7f141c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:31.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.163 7f142312f700 1 -- 172.21.2.101:0/959337443 >> 172.21.2.101:0/959337443 conn(0x7f141c0ce3e0 msgr2=0x7f141c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:31.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.163 7f142312f700 1 -- 172.21.2.101:0/959337443 shutdown_connections 2021-02-10T11:26:31.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.163 7f142312f700 1 -- 172.21.2.101:0/959337443 wait complete. 2021-02-10T11:26:31.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.178 7f142312f700 1 Processor -- start 2021-02-10T11:26:31.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.178 7f142312f700 1 -- start start 2021-02-10T11:26:31.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.178 7f142312f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f141c1a42e0 0x7f141c09ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:31.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.178 7f142312f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f141c1a46e0 0x7f141c09f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:31.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.178 7f142312f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f141c08ad30 0x7f141c08b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:31.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.178 7f142312f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c08a7b0 con 0x7f141c08ad30 2021-02-10T11:26:31.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.178 7f142312f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c0a65f0 con 0x7f141c1a46e0 2021-02-10T11:26:31.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.178 7f142312f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141c08a430 con 0x7f141c1a42e0 2021-02-10T11:26:31.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.179 7f142174c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f141c08ad30 0x7f141c08b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:31.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.179 7f1420f4b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f141c1a42e0 0x7f141c09ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:31.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.179 7f141bfff700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f141c1a46e0 0x7f141c09f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:31.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.179 7f142174c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f141c08ad30 0x7f141c08b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47790/0 (socket says 172.21.2.101:47790) 2021-02-10T11:26:31.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.179 7f142174c700 1 -- 172.21.2.101:0/2481892176 learned_addr learned my addr 172.21.2.101:0/2481892176 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:26:31.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.179 7f1420f4b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f141c1a42e0 0x7f141c09ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:40518/0 (socket says 172.21.2.101:40518) 2021-02-10T11:26:31.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.179 7f141bfff700 1 -- 172.21.2.101:0/2481892176 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f141c1a42e0 msgr2=0x7f141c09ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:31.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.179 7f141bfff700 1 --2- 172.21.2.101:0/2481892176 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f141c1a42e0 0x7f141c09ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:31.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.179 7f141bfff700 1 -- 172.21.2.101:0/2481892176 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f141c08ad30 msgr2=0x7f141c08b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:31.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.179 7f141bfff700 1 --2- 172.21.2.101:0/2481892176 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f141c08ad30 0x7f141c08b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:31.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.179 7f141bfff700 1 -- 172.21.2.101:0/2481892176 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c09fa50 con 0x7f141c1a46e0 2021-02-10T11:26:31.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.179 7f142174c700 1 --2- 172.21.2.101:0/2481892176 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f141c08ad30 0x7f141c08b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:26:31.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.179 7f141bfff700 1 --2- 172.21.2.101:0/2481892176 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f141c1a46e0 0x7f141c09f310 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f140c00a4c0 tx=0x7f140c00a860).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:31.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.180 7f14197fa700 1 -- 172.21.2.101:0/2481892176 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f140c007160 con 0x7f141c1a46e0 2021-02-10T11:26:31.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.180 7f14197fa700 1 -- 172.21.2.101:0/2481892176 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c00ac90 con 0x7f141c1a46e0 2021-02-10T11:26:31.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.180 7f14197fa700 1 -- 172.21.2.101:0/2481892176 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f140c0043a0 con 0x7f141c1a46e0 2021-02-10T11:26:31.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.180 7f142312f700 1 -- 172.21.2.101:0/2481892176 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f141c09fa50 con 0x7f141c1a46e0 2021-02-10T11:26:31.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.180 7f142312f700 1 -- 172.21.2.101:0/2481892176 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f141c09fa50 con 0x7f141c1a46e0 2021-02-10T11:26:31.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.180 7f14197fa700 1 -- 172.21.2.101:0/2481892176 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f140c000d30 con 0x7f141c1a46e0 2021-02-10T11:26:31.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.180 7f14197fa700 1 --2- 172.21.2.101:0/2481892176 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14000186c0 0x7f140001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:31.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.180 7f14197fa700 1 -- 172.21.2.101:0/2481892176 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f140c0326d0 con 0x7f141c1a46e0 2021-02-10T11:26:31.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.180 7f1420f4b700 1 --2- 172.21.2.101:0/2481892176 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14000186c0 0x7f140001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:31.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.181 7f1420f4b700 1 --2- 172.21.2.101:0/2481892176 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14000186c0 0x7f140001abc0 crc :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:31.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.211 7f142312f700 1 -- 172.21.2.101:0/2481892176 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f141c105ae0 con 0x7f141c1a46e0 2021-02-10T11:26:31.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.213 7f14197fa700 1 -- 172.21.2.101:0/2481892176 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f140c000980 con 0x7f141c1a46e0 2021-02-10T11:26:31.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.341 7f142312f700 1 -- 172.21.2.101:0/2481892176 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839315", "target": ["mgr", ""]}) v1 -- 0x7f141c0a65f0 con 0x7f14000186c0 2021-02-10T11:26:31.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.344 7f14197fa700 1 -- 172.21.2.101:0/2481892176 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f1404006a30 con 0x7f14000186c0 2021-02-10T11:26:31.353 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839315/82d06aa7-bfb5-4229-b615-3b768b214b0f 2021-02-10T11:26:31.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.346 7f142312f700 1 -- 172.21.2.101:0/2481892176 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14000186c0 msgr2=0x7f140001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:31.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.346 7f142312f700 1 --2- 172.21.2.101:0/2481892176 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14000186c0 0x7f140001abc0 crc :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:31.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.346 7f142312f700 1 -- 172.21.2.101:0/2481892176 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f141c1a46e0 msgr2=0x7f141c09f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:31.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.346 7f142312f700 1 --2- 172.21.2.101:0/2481892176 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f141c1a46e0 0x7f141c09f310 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f140c00a4c0 tx=0x7f140c00a860).stop 2021-02-10T11:26:31.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.346 7f142312f700 1 -- 172.21.2.101:0/2481892176 shutdown_connections 2021-02-10T11:26:31.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.346 7f142312f700 1 --2- 172.21.2.101:0/2481892176 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f141c08ad30 0x7f141c08b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:31.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.346 7f142312f700 1 --2- 172.21.2.101:0/2481892176 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f141c1a46e0 0x7f141c09f310 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:31.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.346 7f142312f700 1 --2- 172.21.2.101:0/2481892176 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14000186c0 0x7f140001abc0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:31.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.346 7f142312f700 1 --2- 172.21.2.101:0/2481892176 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f141c1a42e0 0x7f141c09ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:31.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.346 7f142312f700 1 -- 172.21.2.101:0/2481892176 >> 172.21.2.101:0/2481892176 conn(0x7f141c095a70 msgr2=0x7f141c0bf050 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:31.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.346 7f142312f700 1 -- 172.21.2.101:0/2481892176 shutdown_connections 2021-02-10T11:26:31.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:31.346 7f142312f700 1 -- 172.21.2.101:0/2481892176 wait complete. 2021-02-10T11:26:31.363 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839315 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839315/82d06aa7-bfb5-4229-b615-3b768b214b0f 2021-02-10T11:26:31.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:31.363 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:31.363 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:31.699 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:31.699 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:31.699 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839315/82d06aa7-bfb5-4229-b615-3b768b214b0f/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:32.081 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:32.081 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:32.081 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247288 s, 42.4 MB/s 2021-02-10T11:26:32.082 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:32.082 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:32.082 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:32.419 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:32.419 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:32.420 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839315/82d06aa7-bfb5-4229-b615-3b768b214b0f/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:32.796 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:32.796 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:32.796 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241248 s, 43.5 MB/s 2021-02-10T11:26:32.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:32.797 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:32.798 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:33.133 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:33.134 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:33.134 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839315/82d06aa7-bfb5-4229-b615-3b768b214b0f/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:33.508 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:33.508 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:33.508 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023003 s, 45.6 MB/s 2021-02-10T11:26:33.509 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:33.509 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:33.509 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:33.845 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:33.845 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:33.846 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839315/82d06aa7-bfb5-4229-b615-3b768b214b0f/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:34.219 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:34.219 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:34.219 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238853 s, 43.9 MB/s 2021-02-10T11:26:34.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:34.220 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:34.220 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:34.555 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:34.555 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:34.555 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839315/82d06aa7-bfb5-4229-b615-3b768b214b0f/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:34.932 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:34.932 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:34.932 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241161 s, 43.5 MB/s 2021-02-10T11:26:34.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:34.934 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:34.934 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:35.269 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:35.270 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:35.270 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839315/82d06aa7-bfb5-4229-b615-3b768b214b0f/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:35.645 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:35.646 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:35.646 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245854 s, 42.7 MB/s 2021-02-10T11:26:35.647 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:35.647 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:35.647 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:35.981 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:35.982 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:35.982 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839315/82d06aa7-bfb5-4229-b615-3b768b214b0f/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:36.359 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:36.359 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:36.359 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239716 s, 43.7 MB/s 2021-02-10T11:26:36.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:36.360 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:36.361 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:36.699 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:36.700 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:36.700 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839315/82d06aa7-bfb5-4229-b615-3b768b214b0f/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:37.077 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:37.077 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:37.077 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243866 s, 43.0 MB/s 2021-02-10T11:26:37.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:37.078 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:37.078 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:37.414 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:37.414 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:37.415 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839315/82d06aa7-bfb5-4229-b615-3b768b214b0f/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:37.792 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:37.792 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:37.793 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0254638 s, 41.2 MB/s 2021-02-10T11:26:37.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:37.794 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:37.794 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:38.130 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:38.130 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:38.131 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839315/82d06aa7-bfb5-4229-b615-3b768b214b0f/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:38.505 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:38.505 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:38.505 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237306 s, 44.2 MB/s 2021-02-10T11:26:38.506 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839316 2021-02-10T11:26:38.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.911 7fdd602c8700 1 Processor -- start 2021-02-10T11:26:38.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.911 7fdd602c8700 1 -- start start 2021-02-10T11:26:38.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.912 7fdd602c8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdd581b0470 0x7fdd581b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:38.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.912 7fdd602c8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fdd580f20e0 0x7fdd580f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:38.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.912 7fdd602c8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd580f0b50 con 0x7fdd580ef610 2021-02-10T11:26:38.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.912 7fdd602c8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd580f0820 con 0x7fdd581b0470 2021-02-10T11:26:38.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.912 7fdd602c8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd580f24e0 con 0x7fdd580f20e0 2021-02-10T11:26:38.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.912 7fdd5e0e4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdd581b0470 0x7fdd581b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.912 7fdd5e0e4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdd581b0470 0x7fdd581b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47794/0 (socket says 172.21.2.101:47794) 2021-02-10T11:26:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.912 7fdd5e0e4700 1 -- 172.21.2.101:0/2887278340 learned_addr learned my addr 172.21.2.101:0/2887278340 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:26:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.912 7fdd5e0e4700 1 -- 172.21.2.101:0/2887278340 >> v2:172.21.2.108:3300/0 conn(0x7fdd580f20e0 msgr2=0x7fdd580f0200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:26:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.912 7fdd5e0e4700 1 --2- 172.21.2.101:0/2887278340 >> v2:172.21.2.108:3300/0 conn(0x7fdd580f20e0 0x7fdd580f0200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.912 7fdd5e0e4700 1 -- 172.21.2.101:0/2887278340 >> v1:172.21.2.108:6789/0 conn(0x7fdd580ef610 legacy=0x7fdd580efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:26:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.912 7fdd5e0e4700 1 -- 172.21.2.101:0/2887278340 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd580e52e0 con 0x7fdd581b0470 2021-02-10T11:26:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.912 7fdd5e0e4700 1 --2- 172.21.2.101:0/2887278340 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd581b0470 0x7fdd581b0870 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7fdd4c006700 tx=0x7fdd4c009b80).ready entity=mon.0 client_cookie=49ea9f986d7e7e00 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.912 7fdd5d0e2700 1 -- 172.21.2.101:0/2887278340 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd4c00c0c0 con 0x7fdd581b0470 2021-02-10T11:26:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.912 7fdd5d0e2700 1 -- 172.21.2.101:0/2887278340 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c00c340 con 0x7fdd581b0470 2021-02-10T11:26:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.913 7fdd5d0e2700 1 -- 172.21.2.101:0/2887278340 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd4c00c860 con 0x7fdd581b0470 2021-02-10T11:26:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.913 7fdd602c8700 1 -- 172.21.2.101:0/2887278340 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd581b0470 msgr2=0x7fdd581b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.913 7fdd602c8700 1 --2- 172.21.2.101:0/2887278340 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd581b0470 0x7fdd581b0870 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7fdd4c006700 tx=0x7fdd4c009b80).stop 2021-02-10T11:26:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.913 7fdd602c8700 1 -- 172.21.2.101:0/2887278340 shutdown_connections 2021-02-10T11:26:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.913 7fdd602c8700 1 --2- 172.21.2.101:0/2887278340 >> v2:172.21.2.108:3300/0 conn(0x7fdd580f20e0 0x7fdd580f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:38.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.913 7fdd602c8700 1 --2- 172.21.2.101:0/2887278340 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd581b0470 0x7fdd581b0870 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:38.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.913 7fdd602c8700 1 -- 172.21.2.101:0/2887278340 >> 172.21.2.101:0/2887278340 conn(0x7fdd580ce3e0 msgr2=0x7fdd580e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:38.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.913 7fdd602c8700 1 -- 172.21.2.101:0/2887278340 shutdown_connections 2021-02-10T11:26:38.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.913 7fdd602c8700 1 -- 172.21.2.101:0/2887278340 wait complete. 2021-02-10T11:26:38.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.927 7fdd602c8700 1 Processor -- start 2021-02-10T11:26:38.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.928 7fdd602c8700 1 -- start start 2021-02-10T11:26:38.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.928 7fdd602c8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd58094fb0 0x7fdd5809e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:38.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.928 7fdd602c8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd580c2be0 0x7fdd580c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:38.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.928 7fdd602c8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd580c3600 0x7fdd580a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:38.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.928 7fdd602c8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5809ebf0 con 0x7fdd58094fb0 2021-02-10T11:26:38.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.928 7fdd602c8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd580aa2a0 con 0x7fdd580c3600 2021-02-10T11:26:38.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.928 7fdd602c8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd580aa4c0 con 0x7fdd580c2be0 2021-02-10T11:26:38.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.928 7fdd5e0e4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd58094fb0 0x7fdd5809e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:38.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.928 7fdd5e0e4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd58094fb0 0x7fdd5809e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47800/0 (socket says 172.21.2.101:47800) 2021-02-10T11:26:38.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.928 7fdd5e0e4700 1 -- 172.21.2.101:0/522895378 learned_addr learned my addr 172.21.2.101:0/522895378 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:26:38.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.928 7fdd5e0e4700 1 -- 172.21.2.101:0/522895378 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd580c2be0 msgr2=0x7fdd580c2fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:26:38.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.928 7fdd5e0e4700 1 --2- 172.21.2.101:0/522895378 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd580c2be0 0x7fdd580c2fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:38.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.928 7fdd5e0e4700 1 -- 172.21.2.101:0/522895378 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd580c3600 msgr2=0x7fdd580a9c80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:26:38.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.928 7fdd5e0e4700 1 --2- 172.21.2.101:0/522895378 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd580c3600 0x7fdd580a9c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:38.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.930 7fdd5e0e4700 1 -- 172.21.2.101:0/522895378 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd580aa6e0 con 0x7fdd58094fb0 2021-02-10T11:26:38.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.930 7fdd5e0e4700 1 --2- 172.21.2.101:0/522895378 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd58094fb0 0x7fdd5809e5d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7fdd4c00b5e0 tx=0x7fdd4c00b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:38.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.930 7fdd4affd700 1 -- 172.21.2.101:0/522895378 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd4c00bd70 con 0x7fdd58094fb0 2021-02-10T11:26:38.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.930 7fdd4affd700 1 -- 172.21.2.101:0/522895378 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c002030 con 0x7fdd58094fb0 2021-02-10T11:26:38.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.930 7fdd4affd700 1 -- 172.21.2.101:0/522895378 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd4c002550 con 0x7fdd58094fb0 2021-02-10T11:26:38.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.930 7fdd602c8700 1 -- 172.21.2.101:0/522895378 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd580aa6e0 con 0x7fdd58094fb0 2021-02-10T11:26:38.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.930 7fdd602c8700 1 -- 172.21.2.101:0/522895378 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd580aac70 con 0x7fdd58094fb0 2021-02-10T11:26:38.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.930 7fdd4affd700 1 -- 172.21.2.101:0/522895378 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdd4c000c90 con 0x7fdd58094fb0 2021-02-10T11:26:38.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.930 7fdd4affd700 1 --2- 172.21.2.101:0/522895378 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd400186c0 0x7fdd4001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:38.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.930 7fdd4affd700 1 -- 172.21.2.101:0/522895378 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fdd4c032580 con 0x7fdd58094fb0 2021-02-10T11:26:38.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.930 7fdd5d8e3700 1 --2- 172.21.2.101:0/522895378 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd400186c0 0x7fdd4001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:38.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.930 7fdd5d8e3700 1 --2- 172.21.2.101:0/522895378 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd400186c0 0x7fdd4001abc0 crc :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:38.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.962 7fdd602c8700 1 -- 172.21.2.101:0/522895378 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd580a9750 con 0x7fdd58094fb0 2021-02-10T11:26:38.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:38.964 7fdd4affd700 1 -- 172.21.2.101:0/522895378 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdd4c000990 con 0x7fdd58094fb0 2021-02-10T11:26:39.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.092 7fdd602c8700 1 -- 172.21.2.101:0/522895378 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839316", "target": ["mgr", ""]}) v1 -- 0x7fdd580aa2a0 con 0x7fdd400186c0 2021-02-10T11:26:39.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.120 7fdd4affd700 1 -- 172.21.2.101:0/522895378 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fdd540009b0 con 0x7fdd400186c0 2021-02-10T11:26:39.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.125 7fdd602c8700 1 -- 172.21.2.101:0/522895378 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd400186c0 msgr2=0x7fdd4001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:39.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.125 7fdd602c8700 1 --2- 172.21.2.101:0/522895378 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd400186c0 0x7fdd4001abc0 crc :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:39.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.125 7fdd602c8700 1 -- 172.21.2.101:0/522895378 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd58094fb0 msgr2=0x7fdd5809e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:39.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.125 7fdd602c8700 1 --2- 172.21.2.101:0/522895378 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd58094fb0 0x7fdd5809e5d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7fdd4c00b5e0 tx=0x7fdd4c00b980).stop 2021-02-10T11:26:39.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.125 7fdd602c8700 1 -- 172.21.2.101:0/522895378 shutdown_connections 2021-02-10T11:26:39.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.125 7fdd602c8700 1 --2- 172.21.2.101:0/522895378 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd400186c0 0x7fdd4001abc0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:39.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.125 7fdd602c8700 1 --2- 172.21.2.101:0/522895378 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd580c2be0 0x7fdd580c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:39.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.125 7fdd602c8700 1 --2- 172.21.2.101:0/522895378 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd580c3600 0x7fdd580a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:39.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.125 7fdd602c8700 1 --2- 172.21.2.101:0/522895378 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd58094fb0 0x7fdd5809e5d0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:39.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.125 7fdd602c8700 1 -- 172.21.2.101:0/522895378 >> 172.21.2.101:0/522895378 conn(0x7fdd5809f630 msgr2=0x7fdd580a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:39.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.125 7fdd602c8700 1 -- 172.21.2.101:0/522895378 shutdown_connections 2021-02-10T11:26:39.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.125 7fdd602c8700 1 -- 172.21.2.101:0/522895378 wait complete. 2021-02-10T11:26:39.142 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839316 2021-02-10T11:26:39.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.548 7fd1c2a63700 1 Processor -- start 2021-02-10T11:26:39.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.548 7fd1c2a63700 1 -- start start 2021-02-10T11:26:39.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.549 7fd1c2a63700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd1bc1b0470 0x7fd1bc1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:39.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.549 7fd1c2a63700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd1bc0ef610 0x7fd1bc0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:39.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.549 7fd1c2a63700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1bc0f0b50 con 0x7fd1bc0f0030 2021-02-10T11:26:39.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.549 7fd1c2a63700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1bc0e52e0 con 0x7fd1bc0ef610 2021-02-10T11:26:39.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.549 7fd1c2a63700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1bc0e5490 con 0x7fd1bc1b0470 2021-02-10T11:26:39.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.549 7fd1c1080700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd1bc0f0030 0x7fd1bc0f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48302/0 (socket says 172.21.2.101:48302) 2021-02-10T11:26:39.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.549 7fd1c1080700 1 -- 172.21.2.101:0/1976124942 learned_addr learned my addr 172.21.2.101:0/1976124942 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:26:39.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.549 7fd1bbfff700 1 --2- 172.21.2.101:0/1976124942 >> v2:172.21.2.101:3300/0 conn(0x7fd1bc0ef610 0x7fd1bc0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:39.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.549 7fd1c087f700 1 --2- 172.21.2.101:0/1976124942 >> v2:172.21.2.108:3300/0 conn(0x7fd1bc1b0470 0x7fd1bc1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:39.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.549 7fd1bb7fe700 1 -- 172.21.2.101:0/1976124942 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486787456 0 0) 0x7fd1b4001500 con 0x7fd1bc0f0030 2021-02-10T11:26:39.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.549 7fd1bb7fe700 1 -- 172.21.2.101:0/1976124942 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1a8002fe0 con 0x7fd1bc0f0030 2021-02-10T11:26:39.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.549 7fd1bbfff700 1 -- 172.21.2.101:0/1976124942 >> v2:172.21.2.108:3300/0 conn(0x7fd1bc1b0470 msgr2=0x7fd1bc1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:26:39.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.549 7fd1bbfff700 1 --2- 172.21.2.101:0/1976124942 >> v2:172.21.2.108:3300/0 conn(0x7fd1bc1b0470 0x7fd1bc1b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:39.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.549 7fd1bbfff700 1 -- 172.21.2.101:0/1976124942 >> v1:172.21.2.101:6789/0 conn(0x7fd1bc0f0030 legacy=0x7fd1bc0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:39.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.549 7fd1bbfff700 1 -- 172.21.2.101:0/1976124942 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1bc0f0430 con 0x7fd1bc0ef610 2021-02-10T11:26:39.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.549 7fd1bbfff700 1 --2- 172.21.2.101:0/1976124942 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1bc0ef610 0x7fd1bc0efa10 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7fd1ac006650 tx=0x7fd1ac009a00).ready entity=mon.0 client_cookie=a7947d296cfb2395 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:39.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.550 7fd1bb7fe700 1 -- 172.21.2.101:0/1976124942 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd1ac0009b0 con 0x7fd1bc0ef610 2021-02-10T11:26:39.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.550 7fd1bb7fe700 1 -- 172.21.2.101:0/1976124942 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ac00c0c0 con 0x7fd1bc0ef610 2021-02-10T11:26:39.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.550 7fd1bb7fe700 1 -- 172.21.2.101:0/1976124942 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd1ac00c5a0 con 0x7fd1bc0ef610 2021-02-10T11:26:39.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.550 7fd1c2a63700 1 -- 172.21.2.101:0/1976124942 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1bc0ef610 msgr2=0x7fd1bc0efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:39.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.550 7fd1c2a63700 1 --2- 172.21.2.101:0/1976124942 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1bc0ef610 0x7fd1bc0efa10 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7fd1ac006650 tx=0x7fd1ac009a00).stop 2021-02-10T11:26:39.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.550 7fd1c2a63700 1 -- 172.21.2.101:0/1976124942 shutdown_connections 2021-02-10T11:26:39.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.550 7fd1c2a63700 1 --2- 172.21.2.101:0/1976124942 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1bc0ef610 0x7fd1bc0efa10 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:39.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.550 7fd1c2a63700 1 --2- 172.21.2.101:0/1976124942 >> v2:172.21.2.108:3300/0 conn(0x7fd1bc1b0470 0x7fd1bc1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:39.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.550 7fd1c2a63700 1 -- 172.21.2.101:0/1976124942 >> 172.21.2.101:0/1976124942 conn(0x7fd1bc0ce3e0 msgr2=0x7fd1bc0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:39.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.550 7fd1c2a63700 1 -- 172.21.2.101:0/1976124942 shutdown_connections 2021-02-10T11:26:39.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.550 7fd1c2a63700 1 -- 172.21.2.101:0/1976124942 wait complete. 2021-02-10T11:26:39.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.565 7fd1c2a63700 1 Processor -- start 2021-02-10T11:26:39.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.565 7fd1c2a63700 1 -- start start 2021-02-10T11:26:39.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.565 7fd1c2a63700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1bc0a0420 0x7fd1bc0a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:39.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.565 7fd1c2a63700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1bc189d00 0x7fd1bc18a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:39.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.565 7fd1c2a63700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1bc18a720 0x7fd1bc0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:39.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.565 7fd1c2a63700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc189800 con 0x7fd1bc18a720 2021-02-10T11:26:39.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.565 7fd1c2a63700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc18ab20 con 0x7fd1bc189d00 2021-02-10T11:26:39.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.565 7fd1c2a63700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc0a1140 con 0x7fd1bc0a0420 2021-02-10T11:26:39.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.566 7fd1c1080700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1bc18a720 0x7fd1bc0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:39.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.566 7fd1c1080700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1bc18a720 0x7fd1bc0b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47812/0 (socket says 172.21.2.101:47812) 2021-02-10T11:26:39.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.566 7fd1c1080700 1 -- 172.21.2.101:0/2594834768 learned_addr learned my addr 172.21.2.101:0/2594834768 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:26:39.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.566 7fd1c087f700 1 --2- 172.21.2.101:0/2594834768 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1bc0a0420 0x7fd1bc0a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:39.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.566 7fd1c1080700 1 -- 172.21.2.101:0/2594834768 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1bc0a0420 msgr2=0x7fd1bc0a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:39.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.566 7fd1bbfff700 1 --2- 172.21.2.101:0/2594834768 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1bc189d00 0x7fd1bc18a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:39.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.566 7fd1c1080700 1 --2- 172.21.2.101:0/2594834768 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1bc0a0420 0x7fd1bc0a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:39.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.566 7fd1c1080700 1 -- 172.21.2.101:0/2594834768 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1bc189d00 msgr2=0x7fd1bc18a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:39.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.566 7fd1c1080700 1 --2- 172.21.2.101:0/2594834768 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1bc189d00 0x7fd1bc18a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:39.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.566 7fd1c1080700 1 -- 172.21.2.101:0/2594834768 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1bc0b8200 con 0x7fd1bc18a720 2021-02-10T11:26:39.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.566 7fd1c087f700 1 --2- 172.21.2.101:0/2594834768 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1bc0a0420 0x7fd1bc0a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:26:39.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.566 7fd1bbfff700 1 --2- 172.21.2.101:0/2594834768 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1bc189d00 0x7fd1bc18a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:26:39.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.566 7fd1c1080700 1 --2- 172.21.2.101:0/2594834768 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1bc18a720 0x7fd1bc0b7be0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7fd1b400a8f0 tx=0x7fd1b400ac90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:39.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.567 7fd1b97fa700 1 -- 172.21.2.101:0/2594834768 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd1b4001270 con 0x7fd1bc18a720 2021-02-10T11:26:39.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.567 7fd1b97fa700 1 -- 172.21.2.101:0/2594834768 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b40070d0 con 0x7fd1bc18a720 2021-02-10T11:26:39.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.567 7fd1b97fa700 1 -- 172.21.2.101:0/2594834768 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd1b4007520 con 0x7fd1bc18a720 2021-02-10T11:26:39.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.567 7fd1c2a63700 1 -- 172.21.2.101:0/2594834768 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1bc0b8200 con 0x7fd1bc18a720 2021-02-10T11:26:39.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.567 7fd1c2a63700 1 -- 172.21.2.101:0/2594834768 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1bc0b8790 con 0x7fd1bc18a720 2021-02-10T11:26:39.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.567 7fd1b97fa700 1 -- 172.21.2.101:0/2594834768 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd1b4000d30 con 0x7fd1bc18a720 2021-02-10T11:26:39.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.567 7fd1b97fa700 1 --2- 172.21.2.101:0/2594834768 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1a00186c0 0x7fd1a001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:39.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.567 7fd1b97fa700 1 -- 172.21.2.101:0/2594834768 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd1b4032910 con 0x7fd1bc18a720 2021-02-10T11:26:39.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.567 7fd1c087f700 1 --2- 172.21.2.101:0/2594834768 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1a00186c0 0x7fd1a001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:39.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.568 7fd1c087f700 1 --2- 172.21.2.101:0/2594834768 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1a00186c0 0x7fd1a001abc0 crc :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:39.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.598 7fd1c2a63700 1 -- 172.21.2.101:0/2594834768 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1bc105ae0 con 0x7fd1bc18a720 2021-02-10T11:26:39.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.602 7fd1b97fa700 1 -- 172.21.2.101:0/2594834768 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd1b4025ee0 con 0x7fd1bc18a720 2021-02-10T11:26:39.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.729 7fd1c2a63700 1 -- 172.21.2.101:0/2594834768 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839316", "target": ["mgr", ""]}) v1 -- 0x7fd1bc0b8790 con 0x7fd1a00186c0 2021-02-10T11:26:39.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.732 7fd1b97fa700 1 -- 172.21.2.101:0/2594834768 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fd1a4006b30 con 0x7fd1a00186c0 2021-02-10T11:26:39.741 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839316/0d7e66a4-96c4-4841-900d-b3f8103fd390 2021-02-10T11:26:39.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.734 7fd1c2a63700 1 -- 172.21.2.101:0/2594834768 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1a00186c0 msgr2=0x7fd1a001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:39.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.734 7fd1c2a63700 1 --2- 172.21.2.101:0/2594834768 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1a00186c0 0x7fd1a001abc0 crc :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:39.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.734 7fd1c2a63700 1 -- 172.21.2.101:0/2594834768 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1bc18a720 msgr2=0x7fd1bc0b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:39.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.734 7fd1c2a63700 1 --2- 172.21.2.101:0/2594834768 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1bc18a720 0x7fd1bc0b7be0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7fd1b400a8f0 tx=0x7fd1b400ac90).stop 2021-02-10T11:26:39.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.734 7fd1c2a63700 1 -- 172.21.2.101:0/2594834768 shutdown_connections 2021-02-10T11:26:39.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.734 7fd1c2a63700 1 --2- 172.21.2.101:0/2594834768 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1bc18a720 0x7fd1bc0b7be0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:39.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.734 7fd1c2a63700 1 --2- 172.21.2.101:0/2594834768 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1bc189d00 0x7fd1bc18a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:39.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.734 7fd1c2a63700 1 --2- 172.21.2.101:0/2594834768 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1a00186c0 0x7fd1a001abc0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:39.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.734 7fd1c2a63700 1 --2- 172.21.2.101:0/2594834768 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1bc0a0420 0x7fd1bc0a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:39.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.734 7fd1c2a63700 1 -- 172.21.2.101:0/2594834768 >> 172.21.2.101:0/2594834768 conn(0x7fd1bc094e10 msgr2=0x7fd1bc0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:39.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.734 7fd1c2a63700 1 -- 172.21.2.101:0/2594834768 shutdown_connections 2021-02-10T11:26:39.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:39.734 7fd1c2a63700 1 -- 172.21.2.101:0/2594834768 wait complete. 2021-02-10T11:26:39.751 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839316 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839316/0d7e66a4-96c4-4841-900d-b3f8103fd390 2021-02-10T11:26:39.751 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:39.751 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:39.751 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:40.087 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:40.088 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:40.088 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839316/0d7e66a4-96c4-4841-900d-b3f8103fd390/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:40.470 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:40.470 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:40.470 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242044 s, 43.3 MB/s 2021-02-10T11:26:40.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:40.472 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:40.472 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:40.808 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:40.808 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:40.808 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839316/0d7e66a4-96c4-4841-900d-b3f8103fd390/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:41.186 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:41.186 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:41.186 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.025051 s, 41.9 MB/s 2021-02-10T11:26:41.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:41.187 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:41.187 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:41.523 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:41.523 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:41.523 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839316/0d7e66a4-96c4-4841-900d-b3f8103fd390/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:41.898 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:41.898 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:41.898 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233369 s, 44.9 MB/s 2021-02-10T11:26:41.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:41.899 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:41.899 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:42.234 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:42.235 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:42.235 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839316/0d7e66a4-96c4-4841-900d-b3f8103fd390/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:42.610 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:42.610 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:42.610 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0232529 s, 45.1 MB/s 2021-02-10T11:26:42.611 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:42.611 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:42.611 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:42.947 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:42.948 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:42.948 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839316/0d7e66a4-96c4-4841-900d-b3f8103fd390/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:43.324 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:43.324 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:43.324 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241779 s, 43.4 MB/s 2021-02-10T11:26:43.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:43.325 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:43.325 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:43.661 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:43.661 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:43.661 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839316/0d7e66a4-96c4-4841-900d-b3f8103fd390/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:44.032 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:44.032 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:44.033 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0229458 s, 45.7 MB/s 2021-02-10T11:26:44.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:44.034 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:44.034 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:44.368 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:44.368 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:44.368 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839316/0d7e66a4-96c4-4841-900d-b3f8103fd390/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:44.745 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:44.746 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:44.746 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024148 s, 43.4 MB/s 2021-02-10T11:26:44.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:44.747 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:44.747 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:45.082 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:45.082 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:45.082 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839316/0d7e66a4-96c4-4841-900d-b3f8103fd390/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:45.459 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:45.459 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:45.459 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242578 s, 43.2 MB/s 2021-02-10T11:26:45.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:45.460 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:45.460 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:45.795 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:45.796 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:45.796 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839316/0d7e66a4-96c4-4841-900d-b3f8103fd390/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:46.171 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:46.171 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:46.171 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238603 s, 43.9 MB/s 2021-02-10T11:26:46.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:46.172 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:46.172 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:46.508 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:46.509 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:46.509 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839316/0d7e66a4-96c4-4841-900d-b3f8103fd390/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:46.887 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:46.887 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:46.887 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0254849 s, 41.1 MB/s 2021-02-10T11:26:46.888 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839317 2021-02-10T11:26:47.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.293 7fcddafac700 1 Processor -- start 2021-02-10T11:26:47.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.293 7fcddafac700 1 -- start start 2021-02-10T11:26:47.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.294 7fcddafac700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcdd41b0870 0x7fcdd40f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:47.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.294 7fcddafac700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fcdd40efe00 0x7fcdd40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:47.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.294 7fcddafac700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdd41b0c70 con 0x7fcdd41b0470 2021-02-10T11:26:47.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.294 7fcddafac700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdd40e5ea0 con 0x7fcdd41b0870 2021-02-10T11:26:47.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.294 7fcddafac700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdd40f0b50 con 0x7fcdd40efe00 2021-02-10T11:26:47.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.294 7fcdd3fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcdd41b0870 0x7fcdd40f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:47.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.294 7fcdd3fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcdd41b0870 0x7fcdd40f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47820/0 (socket says 172.21.2.101:47820) 2021-02-10T11:26:47.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.294 7fcdd3fff700 1 -- 172.21.2.101:0/187495087 learned_addr learned my addr 172.21.2.101:0/187495087 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:26:47.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.294 7fcdd3fff700 1 -- 172.21.2.101:0/187495087 >> v2:172.21.2.108:3300/0 conn(0x7fcdd40efe00 msgr2=0x7fcdd40f0200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:26:47.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.294 7fcdd3fff700 1 --2- 172.21.2.101:0/187495087 >> v2:172.21.2.108:3300/0 conn(0x7fcdd40efe00 0x7fcdd40f0200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:47.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.294 7fcdd3fff700 1 -- 172.21.2.101:0/187495087 >> v1:172.21.2.108:6789/0 conn(0x7fcdd41b0470 legacy=0x7fcdd40ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:26:47.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.294 7fcdd3fff700 1 -- 172.21.2.101:0/187495087 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd40f0820 con 0x7fcdd41b0870 2021-02-10T11:26:47.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.294 7fcdd3fff700 1 --2- 172.21.2.101:0/187495087 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcdd41b0870 0x7fcdd40f20e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7fcdc4006700 tx=0x7fcdc4009b00).ready entity=mon.0 client_cookie=3ca344fc938029a2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:47.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.294 7fcdd37fe700 1 -- 172.21.2.101:0/187495087 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcdc400c0c0 con 0x7fcdd41b0870 2021-02-10T11:26:47.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.295 7fcdd37fe700 1 -- 172.21.2.101:0/187495087 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc400c340 con 0x7fcdd41b0870 2021-02-10T11:26:47.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.295 7fcdd37fe700 1 -- 172.21.2.101:0/187495087 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcdc400c820 con 0x7fcdd41b0870 2021-02-10T11:26:47.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.295 7fcddafac700 1 -- 172.21.2.101:0/187495087 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcdd41b0870 msgr2=0x7fcdd40f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:47.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.295 7fcddafac700 1 --2- 172.21.2.101:0/187495087 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcdd41b0870 0x7fcdd40f20e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7fcdc4006700 tx=0x7fcdc4009b00).stop 2021-02-10T11:26:47.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.295 7fcddafac700 1 -- 172.21.2.101:0/187495087 shutdown_connections 2021-02-10T11:26:47.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.295 7fcddafac700 1 --2- 172.21.2.101:0/187495087 >> v2:172.21.2.108:3300/0 conn(0x7fcdd40efe00 0x7fcdd40f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:47.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.295 7fcddafac700 1 --2- 172.21.2.101:0/187495087 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcdd41b0870 0x7fcdd40f20e0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:47.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.295 7fcddafac700 1 -- 172.21.2.101:0/187495087 >> 172.21.2.101:0/187495087 conn(0x7fcdd40ce3e0 msgr2=0x7fcdd40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:47.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.295 7fcddafac700 1 -- 172.21.2.101:0/187495087 shutdown_connections 2021-02-10T11:26:47.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.295 7fcddafac700 1 -- 172.21.2.101:0/187495087 wait complete. 2021-02-10T11:26:47.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.310 7fcddafac700 1 Processor -- start 2021-02-10T11:26:47.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.310 7fcddafac700 1 -- start start 2021-02-10T11:26:47.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.310 7fcddafac700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcdd40b2840 0x7fcdd41a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:47.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.310 7fcddafac700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcdd40b30d0 0x7fcdd40b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:47.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.310 7fcddafac700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcdd40b3af0 0x7fcdd40f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:47.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.310 7fcddafac700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd40b2c40 con 0x7fcdd40b3af0 2021-02-10T11:26:47.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.310 7fcddafac700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd41a4920 con 0x7fcdd40b2840 2021-02-10T11:26:47.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.310 7fcddafac700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd40f0ba0 con 0x7fcdd40b30d0 2021-02-10T11:26:47.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.310 7fcdd95c9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcdd40b3af0 0x7fcdd40f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:47.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.310 7fcdd95c9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcdd40b3af0 0x7fcdd40f0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47824/0 (socket says 172.21.2.101:47824) 2021-02-10T11:26:47.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.310 7fcdd95c9700 1 -- 172.21.2.101:0/1861901537 learned_addr learned my addr 172.21.2.101:0/1861901537 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:26:47.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.311 7fcdd95c9700 1 -- 172.21.2.101:0/1861901537 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcdd40b30d0 msgr2=0x7fcdd40b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:47.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.311 7fcdd95c9700 1 --2- 172.21.2.101:0/1861901537 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcdd40b30d0 0x7fcdd40b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:47.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.311 7fcdd95c9700 1 -- 172.21.2.101:0/1861901537 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcdd40b2840 msgr2=0x7fcdd41a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:47.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.311 7fcdd95c9700 1 --2- 172.21.2.101:0/1861901537 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcdd40b2840 0x7fcdd41a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:47.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.311 7fcdd95c9700 1 -- 172.21.2.101:0/1861901537 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd40f0dc0 con 0x7fcdd40b3af0 2021-02-10T11:26:47.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.311 7fcdd95c9700 1 --2- 172.21.2.101:0/1861901537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcdd40b3af0 0x7fcdd40f0580 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fcdcc009960 tx=0x7fcdcc00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:47.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.311 7fcdd17fa700 1 -- 172.21.2.101:0/1861901537 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcdcc003030 con 0x7fcdd40b3af0 2021-02-10T11:26:47.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.311 7fcdd17fa700 1 -- 172.21.2.101:0/1861901537 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc001310 con 0x7fcdd40b3af0 2021-02-10T11:26:47.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.311 7fcdd17fa700 1 -- 172.21.2.101:0/1861901537 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcdcc001770 con 0x7fcdd40b3af0 2021-02-10T11:26:47.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.311 7fcddafac700 1 -- 172.21.2.101:0/1861901537 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd40f0dc0 con 0x7fcdd40b3af0 2021-02-10T11:26:47.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.311 7fcddafac700 1 -- 172.21.2.101:0/1861901537 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd40f0dc0 con 0x7fcdd40b3af0 2021-02-10T11:26:47.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.312 7fcdd17fa700 1 -- 172.21.2.101:0/1861901537 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fcdcc00a030 con 0x7fcdd40b3af0 2021-02-10T11:26:47.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.312 7fcdd17fa700 1 --2- 172.21.2.101:0/1861901537 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcdb8018730 0x7fcdb801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:47.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.312 7fcdd17fa700 1 -- 172.21.2.101:0/1861901537 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fcdcc032a20 con 0x7fcdd40b3af0 2021-02-10T11:26:47.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.312 7fcdd8dc8700 1 --2- 172.21.2.101:0/1861901537 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcdb8018730 0x7fcdb801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:47.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.312 7fcdd8dc8700 1 --2- 172.21.2.101:0/1861901537 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcdb8018730 0x7fcdb801ac10 crc :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:47.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.343 7fcddafac700 1 -- 172.21.2.101:0/1861901537 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd40f0050 con 0x7fcdd40b3af0 2021-02-10T11:26:47.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.347 7fcdd17fa700 1 -- 172.21.2.101:0/1861901537 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fcdcc001310 con 0x7fcdd40b3af0 2021-02-10T11:26:47.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.473 7fcddafac700 1 -- 172.21.2.101:0/1861901537 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839317", "target": ["mgr", ""]}) v1 -- 0x7fcdd40f0050 con 0x7fcdb8018730 2021-02-10T11:26:47.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.496 7fcdd17fa700 1 -- 172.21.2.101:0/1861901537 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fcdbc006e40 con 0x7fcdb8018730 2021-02-10T11:26:47.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.506 7fcddafac700 1 -- 172.21.2.101:0/1861901537 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcdb8018730 msgr2=0x7fcdb801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:47.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.506 7fcddafac700 1 --2- 172.21.2.101:0/1861901537 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcdb8018730 0x7fcdb801ac10 crc :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:47.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.506 7fcddafac700 1 -- 172.21.2.101:0/1861901537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcdd40b3af0 msgr2=0x7fcdd40f0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:47.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.506 7fcddafac700 1 --2- 172.21.2.101:0/1861901537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcdd40b3af0 0x7fcdd40f0580 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fcdcc009960 tx=0x7fcdcc00bcf0).stop 2021-02-10T11:26:47.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.506 7fcddafac700 1 -- 172.21.2.101:0/1861901537 shutdown_connections 2021-02-10T11:26:47.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.506 7fcddafac700 1 --2- 172.21.2.101:0/1861901537 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcdb8018730 0x7fcdb801ac10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:47.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.506 7fcddafac700 1 --2- 172.21.2.101:0/1861901537 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcdd40b30d0 0x7fcdd40b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:47.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.506 7fcddafac700 1 --2- 172.21.2.101:0/1861901537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcdd40b3af0 0x7fcdd40f0580 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:47.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.506 7fcddafac700 1 --2- 172.21.2.101:0/1861901537 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcdd40b2840 0x7fcdd41a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:47.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.506 7fcddafac700 1 -- 172.21.2.101:0/1861901537 >> 172.21.2.101:0/1861901537 conn(0x7fcdd40c36f0 msgr2=0x7fcdd4094c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:47.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.506 7fcddafac700 1 -- 172.21.2.101:0/1861901537 shutdown_connections 2021-02-10T11:26:47.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.506 7fcddafac700 1 -- 172.21.2.101:0/1861901537 wait complete. 2021-02-10T11:26:47.524 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839317 2021-02-10T11:26:47.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.931 7f8670797700 1 Processor -- start 2021-02-10T11:26:47.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.931 7f8670797700 1 -- start start 2021-02-10T11:26:47.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.931 7f8670797700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f86681b0470 0x7f86681b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:47.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.931 7f8670797700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f86680ef610 0x7f86680efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:47.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.931 7f8670797700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86680f0b50 con 0x7f86680f0030 2021-02-10T11:26:47.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.931 7f8670797700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86680e52e0 con 0x7f86680ef610 2021-02-10T11:26:47.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.931 7f8670797700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86680e5490 con 0x7f86681b0470 2021-02-10T11:26:47.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.931 7f866ddb2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f86680ef610 0x7f86680efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:47.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.931 7f866edb4700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f86680f0030 0x7f86680f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48330/0 (socket says 172.21.2.101:48330) 2021-02-10T11:26:47.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.931 7f866edb4700 1 -- 172.21.2.101:0/2119121050 learned_addr learned my addr 172.21.2.101:0/2119121050 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:26:47.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.932 7f866ddb2700 1 -- 172.21.2.101:0/2119121050 >> v2:172.21.2.108:3300/0 conn(0x7f86681b0470 msgr2=0x7f86681b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:26:47.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.932 7f866ddb2700 1 --2- 172.21.2.101:0/2119121050 >> v2:172.21.2.108:3300/0 conn(0x7f86681b0470 0x7f86681b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:47.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.932 7f866ddb2700 1 -- 172.21.2.101:0/2119121050 >> v1:172.21.2.101:6789/0 conn(0x7f86680f0030 legacy=0x7f86680f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:47.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.932 7f866ddb2700 1 -- 172.21.2.101:0/2119121050 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86680f0430 con 0x7f86680ef610 2021-02-10T11:26:47.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.932 7f866ddb2700 1 --2- 172.21.2.101:0/2119121050 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86680ef610 0x7f86680efa10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f865c006650 tx=0x7f865c009a80).ready entity=mon.0 client_cookie=752cc5c94829092f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:47.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.932 7f866d5b1700 1 -- 172.21.2.101:0/2119121050 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f865c00c0c0 con 0x7f86680ef610 2021-02-10T11:26:47.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.932 7f866d5b1700 1 -- 172.21.2.101:0/2119121050 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865c00c340 con 0x7f86680ef610 2021-02-10T11:26:47.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.932 7f866d5b1700 1 -- 172.21.2.101:0/2119121050 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f865c00c860 con 0x7f86680ef610 2021-02-10T11:26:47.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.932 7f8670797700 1 -- 172.21.2.101:0/2119121050 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86680ef610 msgr2=0x7f86680efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:47.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.932 7f8670797700 1 --2- 172.21.2.101:0/2119121050 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86680ef610 0x7f86680efa10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f865c006650 tx=0x7f865c009a80).stop 2021-02-10T11:26:47.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.933 7f8670797700 1 -- 172.21.2.101:0/2119121050 shutdown_connections 2021-02-10T11:26:47.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.933 7f8670797700 1 --2- 172.21.2.101:0/2119121050 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86680ef610 0x7f86680efa10 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:47.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.933 7f8670797700 1 --2- 172.21.2.101:0/2119121050 >> v2:172.21.2.108:3300/0 conn(0x7f86681b0470 0x7f86681b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:47.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.933 7f8670797700 1 -- 172.21.2.101:0/2119121050 >> 172.21.2.101:0/2119121050 conn(0x7f86680ce3e0 msgr2=0x7f86680e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:47.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.933 7f8670797700 1 -- 172.21.2.101:0/2119121050 shutdown_connections 2021-02-10T11:26:47.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.933 7f8670797700 1 -- 172.21.2.101:0/2119121050 wait complete. 2021-02-10T11:26:47.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.948 7f8670797700 1 Processor -- start 2021-02-10T11:26:47.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.948 7f8670797700 1 -- start start 2021-02-10T11:26:47.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.948 7f8670797700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86680c36a0 0x7f86680c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:47.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.948 7f8670797700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86680c3190 0x7f86680b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:47.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.948 7f8670797700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86680b2370 0x7f86680b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:47.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.948 7f8670797700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86680b27c0 con 0x7f86680c3190 2021-02-10T11:26:47.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.948 7f8670797700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86680b3360 con 0x7f86680b2370 2021-02-10T11:26:47.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.948 7f8670797700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86680b3540 con 0x7f86680c36a0 2021-02-10T11:26:47.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.949 7f866ddb2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86680c3190 0x7f86680b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:47.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.949 7f866ddb2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86680c3190 0x7f86680b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47842/0 (socket says 172.21.2.101:47842) 2021-02-10T11:26:47.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.949 7f866ddb2700 1 -- 172.21.2.101:0/698892664 learned_addr learned my addr 172.21.2.101:0/698892664 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:26:47.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.949 7f866ddb2700 1 -- 172.21.2.101:0/698892664 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86680c36a0 msgr2=0x7f86680c2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:47.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.949 7f866ddb2700 1 --2- 172.21.2.101:0/698892664 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86680c36a0 0x7f86680c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:47.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.949 7f866ddb2700 1 -- 172.21.2.101:0/698892664 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86680b2370 msgr2=0x7f86680b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:47.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.949 7f866ddb2700 1 --2- 172.21.2.101:0/698892664 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86680b2370 0x7f86680b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:47.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.949 7f866ddb2700 1 -- 172.21.2.101:0/698892664 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86680b3760 con 0x7f86680c3190 2021-02-10T11:26:47.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.949 7f866ddb2700 1 --2- 172.21.2.101:0/698892664 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86680c3190 0x7f86680b1d50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f865c009630 tx=0x7f865c00b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:47.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.949 7f865affd700 1 -- 172.21.2.101:0/698892664 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f865c0093a0 con 0x7f86680c3190 2021-02-10T11:26:47.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.950 7f865affd700 1 -- 172.21.2.101:0/698892664 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865c00bd70 con 0x7f86680c3190 2021-02-10T11:26:47.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.950 7f865affd700 1 -- 172.21.2.101:0/698892664 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f865c005560 con 0x7f86680c3190 2021-02-10T11:26:47.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.950 7f8670797700 1 -- 172.21.2.101:0/698892664 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86680b3760 con 0x7f86680c3190 2021-02-10T11:26:47.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.950 7f8670797700 1 -- 172.21.2.101:0/698892664 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86680b3760 con 0x7f86680c3190 2021-02-10T11:26:47.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.950 7f865affd700 1 -- 172.21.2.101:0/698892664 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f865c008240 con 0x7f86680c3190 2021-02-10T11:26:47.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.950 7f865affd700 1 --2- 172.21.2.101:0/698892664 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86500186c0 0x7f865001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:47.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.950 7f865affd700 1 -- 172.21.2.101:0/698892664 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f865c0325b0 con 0x7f86680c3190 2021-02-10T11:26:47.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.952 7f866e5b3700 1 --2- 172.21.2.101:0/698892664 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86500186c0 0x7f865001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:47.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.952 7f866e5b3700 1 --2- 172.21.2.101:0/698892664 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86500186c0 0x7f865001abc0 crc :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:47.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.981 7f8670797700 1 -- 172.21.2.101:0/698892664 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86680edd10 con 0x7f86680c3190 2021-02-10T11:26:47.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:47.985 7f865affd700 1 -- 172.21.2.101:0/698892664 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f865c008030 con 0x7f86680c3190 2021-02-10T11:26:48.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:48.111 7f8670797700 1 -- 172.21.2.101:0/698892664 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839317", "target": ["mgr", ""]}) v1 -- 0x7f86680e52e0 con 0x7f86500186c0 2021-02-10T11:26:48.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:48.115 7f865affd700 1 -- 172.21.2.101:0/698892664 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f8664006e40 con 0x7f86500186c0 2021-02-10T11:26:48.128 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839317/c96b3e63-5a57-42e6-8c56-ad62f0fb26a8 2021-02-10T11:26:48.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:48.121 7f8670797700 1 -- 172.21.2.101:0/698892664 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86500186c0 msgr2=0x7f865001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:48.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:48.121 7f8670797700 1 --2- 172.21.2.101:0/698892664 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86500186c0 0x7f865001abc0 crc :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:48.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:48.121 7f8670797700 1 -- 172.21.2.101:0/698892664 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86680c3190 msgr2=0x7f86680b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:48.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:48.121 7f8670797700 1 --2- 172.21.2.101:0/698892664 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86680c3190 0x7f86680b1d50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f865c009630 tx=0x7f865c00b980).stop 2021-02-10T11:26:48.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:48.121 7f8670797700 1 -- 172.21.2.101:0/698892664 shutdown_connections 2021-02-10T11:26:48.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:48.121 7f8670797700 1 --2- 172.21.2.101:0/698892664 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86680b2370 0x7f86680b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:48.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:48.121 7f8670797700 1 --2- 172.21.2.101:0/698892664 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86680c3190 0x7f86680b1d50 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:48.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:48.121 7f8670797700 1 --2- 172.21.2.101:0/698892664 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86500186c0 0x7f865001abc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:48.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:48.121 7f8670797700 1 --2- 172.21.2.101:0/698892664 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86680c36a0 0x7f86680c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:48.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:48.121 7f8670797700 1 -- 172.21.2.101:0/698892664 >> 172.21.2.101:0/698892664 conn(0x7f866809f630 msgr2=0x7f86680a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:48.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:48.121 7f8670797700 1 -- 172.21.2.101:0/698892664 shutdown_connections 2021-02-10T11:26:48.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:48.121 7f8670797700 1 -- 172.21.2.101:0/698892664 wait complete. 2021-02-10T11:26:48.139 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839317 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839317/c96b3e63-5a57-42e6-8c56-ad62f0fb26a8 2021-02-10T11:26:48.139 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:48.139 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:48.139 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:48.474 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:48.474 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:48.475 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839317/c96b3e63-5a57-42e6-8c56-ad62f0fb26a8/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:48.854 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:48.854 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:48.854 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024831 s, 42.2 MB/s 2021-02-10T11:26:48.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:48.855 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:48.855 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:49.189 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:49.190 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:49.190 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839317/c96b3e63-5a57-42e6-8c56-ad62f0fb26a8/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:49.566 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:49.566 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:49.566 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253529 s, 41.4 MB/s 2021-02-10T11:26:49.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:49.567 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:49.568 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:49.903 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:49.903 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:49.904 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839317/c96b3e63-5a57-42e6-8c56-ad62f0fb26a8/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:50.281 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:50.281 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:50.281 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249755 s, 42.0 MB/s 2021-02-10T11:26:50.282 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:50.282 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:50.282 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:50.619 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:50.619 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:50.620 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839317/c96b3e63-5a57-42e6-8c56-ad62f0fb26a8/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:50.997 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:50.997 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:50.997 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0267487 s, 39.2 MB/s 2021-02-10T11:26:50.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:50.998 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:50.998 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:51.335 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:51.335 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:51.335 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839317/c96b3e63-5a57-42e6-8c56-ad62f0fb26a8/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:51.711 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:51.711 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:51.711 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242869 s, 43.2 MB/s 2021-02-10T11:26:51.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:51.712 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:51.712 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:52.046 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:52.047 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:52.047 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839317/c96b3e63-5a57-42e6-8c56-ad62f0fb26a8/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:52.422 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:52.422 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:52.422 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237127 s, 44.2 MB/s 2021-02-10T11:26:52.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:52.423 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:52.424 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:52.759 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:52.760 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:52.760 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839317/c96b3e63-5a57-42e6-8c56-ad62f0fb26a8/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:53.133 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:53.134 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:53.134 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.022914 s, 45.8 MB/s 2021-02-10T11:26:53.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:53.135 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:53.135 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:53.356 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:26:53.359 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:26:53.469 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:53.470 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:53.470 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839317/c96b3e63-5a57-42e6-8c56-ad62f0fb26a8/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:53.845 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:53.845 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:53.845 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246748 s, 42.5 MB/s 2021-02-10T11:26:53.846 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:53.846 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:53.846 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:54.181 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:54.182 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:54.182 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839317/c96b3e63-5a57-42e6-8c56-ad62f0fb26a8/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:54.559 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:54.559 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:54.559 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0250965 s, 41.8 MB/s 2021-02-10T11:26:54.560 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:54.560 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:54.560 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:54.894 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:54.895 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:54.895 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839317/c96b3e63-5a57-42e6-8c56-ad62f0fb26a8/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:55.271 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:55.272 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:55.272 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234641 s, 44.7 MB/s 2021-02-10T11:26:55.273 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839318 2021-02-10T11:26:55.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.691 7ff92e20a700 1 Processor -- start 2021-02-10T11:26:55.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.691 7ff92e20a700 1 -- start start 2021-02-10T11:26:55.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.691 7ff92e20a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff9281b0470 0x7ff9281b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:55.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.691 7ff92e20a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9280ef610 0x7ff9280efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:55.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.691 7ff92e20a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9280f0b50 con 0x7ff9280f0030 2021-02-10T11:26:55.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.691 7ff92e20a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9280e52e0 con 0x7ff9281b0470 2021-02-10T11:26:55.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.691 7ff92e20a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9280e54d0 con 0x7ff9280ef610 2021-02-10T11:26:55.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.691 7ff927fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff9281b0470 0x7ff9281b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:55.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.691 7ff927fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff9281b0470 0x7ff9281b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47846/0 (socket says 172.21.2.101:47846) 2021-02-10T11:26:55.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.691 7ff927fff700 1 -- 172.21.2.101:0/1236482868 learned_addr learned my addr 172.21.2.101:0/1236482868 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:26:55.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.693 7ff9277fe700 1 --2- 172.21.2.101:0/1236482868 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9280ef610 0x7ff9280efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:55.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.693 7ff927fff700 1 -- 172.21.2.101:0/1236482868 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9280ef610 msgr2=0x7ff9280efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:26:55.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.693 7ff927fff700 1 --2- 172.21.2.101:0/1236482868 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9280ef610 0x7ff9280efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:55.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.693 7ff927fff700 1 -- 172.21.2.101:0/1236482868 >> v1:172.21.2.108:6789/0 conn(0x7ff9280f0030 legacy=0x7ff9280f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:55.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.693 7ff927fff700 1 -- 172.21.2.101:0/1236482868 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9280f0430 con 0x7ff9281b0470 2021-02-10T11:26:55.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.693 7ff927fff700 1 --2- 172.21.2.101:0/1236482868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9281b0470 0x7ff9281b0870 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7ff910006700 tx=0x7ff910009b80).ready entity=mon.0 client_cookie=3007a1c749fe8ebc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:55.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.693 7ff926ffd700 1 -- 172.21.2.101:0/1236482868 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff91000c0c0 con 0x7ff9281b0470 2021-02-10T11:26:55.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.693 7ff926ffd700 1 -- 172.21.2.101:0/1236482868 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91000c340 con 0x7ff9281b0470 2021-02-10T11:26:55.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.693 7ff926ffd700 1 -- 172.21.2.101:0/1236482868 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff91000c860 con 0x7ff9281b0470 2021-02-10T11:26:55.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.693 7ff92e20a700 1 -- 172.21.2.101:0/1236482868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9281b0470 msgr2=0x7ff9281b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:55.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.693 7ff92e20a700 1 --2- 172.21.2.101:0/1236482868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9281b0470 0x7ff9281b0870 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7ff910006700 tx=0x7ff910009b80).stop 2021-02-10T11:26:55.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.693 7ff92e20a700 1 -- 172.21.2.101:0/1236482868 shutdown_connections 2021-02-10T11:26:55.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.693 7ff92e20a700 1 --2- 172.21.2.101:0/1236482868 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9280ef610 0x7ff9280efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:55.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.693 7ff92e20a700 1 --2- 172.21.2.101:0/1236482868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9281b0470 0x7ff9281b0870 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:55.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.693 7ff92e20a700 1 -- 172.21.2.101:0/1236482868 >> 172.21.2.101:0/1236482868 conn(0x7ff9280ce3e0 msgr2=0x7ff9280e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:55.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.693 7ff92e20a700 1 -- 172.21.2.101:0/1236482868 shutdown_connections 2021-02-10T11:26:55.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.693 7ff92e20a700 1 -- 172.21.2.101:0/1236482868 wait complete. 2021-02-10T11:26:55.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.708 7ff92e20a700 1 Processor -- start 2021-02-10T11:26:55.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.708 7ff92e20a700 1 -- start start 2021-02-10T11:26:55.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.708 7ff92e20a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9280a0420 0x7ff9280a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:55.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.708 7ff92e20a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff928189d00 0x7ff92818a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:55.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.708 7ff92e20a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff92818a720 0x7ff9280b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:55.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.708 7ff92e20a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff928189780 con 0x7ff9280a0420 2021-02-10T11:26:55.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.708 7ff92e20a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92818ab20 con 0x7ff928189d00 2021-02-10T11:26:55.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.708 7ff92e20a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9280a1140 con 0x7ff92818a720 2021-02-10T11:26:55.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.709 7ff927fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9280a0420 0x7ff9280a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:55.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.709 7ff927fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9280a0420 0x7ff9280a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47854/0 (socket says 172.21.2.101:47854) 2021-02-10T11:26:55.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.709 7ff927fff700 1 -- 172.21.2.101:0/1185669093 learned_addr learned my addr 172.21.2.101:0/1185669093 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:26:55.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.709 7ff9277fe700 1 --2- 172.21.2.101:0/1185669093 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff928189d00 0x7ff92818a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:55.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.709 7ff92c827700 1 --2- 172.21.2.101:0/1185669093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff92818a720 0x7ff9280b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:55.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.709 7ff927fff700 1 -- 172.21.2.101:0/1185669093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff92818a720 msgr2=0x7ff9280b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:55.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.709 7ff927fff700 1 --2- 172.21.2.101:0/1185669093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff92818a720 0x7ff9280b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:55.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.709 7ff927fff700 1 -- 172.21.2.101:0/1185669093 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff928189d00 msgr2=0x7ff92818a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:55.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.709 7ff927fff700 1 --2- 172.21.2.101:0/1185669093 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff928189d00 0x7ff92818a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:55.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.709 7ff927fff700 1 -- 172.21.2.101:0/1185669093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9280b8200 con 0x7ff9280a0420 2021-02-10T11:26:55.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.709 7ff927fff700 1 --2- 172.21.2.101:0/1185669093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9280a0420 0x7ff9280a0820 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7ff91000b650 tx=0x7ff91000b9f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:55.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.709 7ff924ff9700 1 -- 172.21.2.101:0/1185669093 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff91000bde0 con 0x7ff9280a0420 2021-02-10T11:26:55.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.709 7ff924ff9700 1 -- 172.21.2.101:0/1185669093 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff910006390 con 0x7ff9280a0420 2021-02-10T11:26:55.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.709 7ff92e20a700 1 -- 172.21.2.101:0/1185669093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9280b8200 con 0x7ff9280a0420 2021-02-10T11:26:55.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.709 7ff924ff9700 1 -- 172.21.2.101:0/1185669093 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff9100022b0 con 0x7ff9280a0420 2021-02-10T11:26:55.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.709 7ff92e20a700 1 -- 172.21.2.101:0/1185669093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9280b8200 con 0x7ff9280a0420 2021-02-10T11:26:55.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.710 7ff924ff9700 1 -- 172.21.2.101:0/1185669093 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff9100052e0 con 0x7ff9280a0420 2021-02-10T11:26:55.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.710 7ff924ff9700 1 --2- 172.21.2.101:0/1185669093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff90c0186c0 0x7ff90c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:55.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.710 7ff924ff9700 1 -- 172.21.2.101:0/1185669093 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff910032410 con 0x7ff9280a0420 2021-02-10T11:26:55.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.710 7ff9277fe700 1 --2- 172.21.2.101:0/1185669093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff90c0186c0 0x7ff90c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:55.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.711 7ff9277fe700 1 --2- 172.21.2.101:0/1185669093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff90c0186c0 0x7ff90c01abc0 crc :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:55.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.741 7ff92e20a700 1 -- 172.21.2.101:0/1185669093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff928105ae0 con 0x7ff9280a0420 2021-02-10T11:26:55.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.744 7ff924ff9700 1 -- 172.21.2.101:0/1185669093 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff910030d40 con 0x7ff9280a0420 2021-02-10T11:26:55.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.868 7ff92e20a700 1 -- 172.21.2.101:0/1185669093 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839318", "target": ["mgr", ""]}) v1 -- 0x7ff9280f0b50 con 0x7ff90c0186c0 2021-02-10T11:26:55.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.895 7ff924ff9700 1 -- 172.21.2.101:0/1185669093 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7ff918006a70 con 0x7ff90c0186c0 2021-02-10T11:26:55.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.902 7ff92e20a700 1 -- 172.21.2.101:0/1185669093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff90c0186c0 msgr2=0x7ff90c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:55.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.902 7ff92e20a700 1 --2- 172.21.2.101:0/1185669093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff90c0186c0 0x7ff90c01abc0 crc :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:55.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.902 7ff92e20a700 1 -- 172.21.2.101:0/1185669093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9280a0420 msgr2=0x7ff9280a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:55.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.902 7ff92e20a700 1 --2- 172.21.2.101:0/1185669093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9280a0420 0x7ff9280a0820 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7ff91000b650 tx=0x7ff91000b9f0).stop 2021-02-10T11:26:55.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.902 7ff92e20a700 1 -- 172.21.2.101:0/1185669093 shutdown_connections 2021-02-10T11:26:55.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.902 7ff92e20a700 1 --2- 172.21.2.101:0/1185669093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff90c0186c0 0x7ff90c01abc0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:55.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.902 7ff92e20a700 1 --2- 172.21.2.101:0/1185669093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff92818a720 0x7ff9280b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:55.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.902 7ff92e20a700 1 --2- 172.21.2.101:0/1185669093 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff928189d00 0x7ff92818a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:55.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.902 7ff92e20a700 1 --2- 172.21.2.101:0/1185669093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9280a0420 0x7ff9280a0820 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:55.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.902 7ff92e20a700 1 -- 172.21.2.101:0/1185669093 >> 172.21.2.101:0/1185669093 conn(0x7ff928094e10 msgr2=0x7ff9280a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:55.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.902 7ff92e20a700 1 -- 172.21.2.101:0/1185669093 shutdown_connections 2021-02-10T11:26:55.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:55.902 7ff92e20a700 1 -- 172.21.2.101:0/1185669093 wait complete. 2021-02-10T11:26:55.920 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839318 2021-02-10T11:26:56.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.326 7fd8e3f30700 1 Processor -- start 2021-02-10T11:26:56.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.327 7fd8e3f30700 1 -- start start 2021-02-10T11:26:56.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.327 7fd8e3f30700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8dc1b0470 0x7fd8dc1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:56.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.327 7fd8e3f30700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd8dc0ef610 0x7fd8dc0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:56.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.327 7fd8e3f30700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd8dc0f0030 0x7fd8dc0f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:56.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.327 7fd8e3f30700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8dc0e5ea0 con 0x7fd8dc0ef610 2021-02-10T11:26:56.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.327 7fd8e3f30700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8dc0f0b50 con 0x7fd8dc0f0030 2021-02-10T11:26:56.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.327 7fd8e3f30700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc0e52e0 con 0x7fd8dc1b0470 2021-02-10T11:26:56.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.327 7fd8e154b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd8dc0ef610 0x7fd8dc0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:56.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.327 7fd8e154b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd8dc0ef610 0x7fd8dc0efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47858/0 (socket says 172.21.2.101:47858) 2021-02-10T11:26:56.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.327 7fd8e154b700 1 -- 172.21.2.101:0/4130064918 learned_addr learned my addr 172.21.2.101:0/4130064918 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:26:56.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.329 7fd8e154b700 1 -- 172.21.2.101:0/4130064918 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8dc1b0470 msgr2=0x7fd8dc1b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:26:56.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.329 7fd8e154b700 1 --2- 172.21.2.101:0/4130064918 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8dc1b0470 0x7fd8dc1b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:56.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.329 7fd8e154b700 1 -- 172.21.2.101:0/4130064918 >> v2:172.21.2.108:3300/0 conn(0x7fd8dc0f0030 msgr2=0x7fd8dc0f0430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:26:56.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.329 7fd8e154b700 1 --2- 172.21.2.101:0/4130064918 >> v2:172.21.2.108:3300/0 conn(0x7fd8dc0f0030 0x7fd8dc0f0430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:56.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.329 7fd8e154b700 1 -- 172.21.2.101:0/4130064918 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8dc0f10d0 con 0x7fd8dc0ef610 2021-02-10T11:26:56.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.329 7fd8e154b700 1 --2- 172.21.2.101:0/4130064918 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8dc0ef610 0x7fd8dc0efa10 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fd8d0006700 tx=0x7fd8d0009b00).ready entity=mon.0 client_cookie=6f35e3adb91752f8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:56.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.329 7fd8e0d4a700 1 -- 172.21.2.101:0/4130064918 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8d000c0c0 con 0x7fd8dc0ef610 2021-02-10T11:26:56.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.329 7fd8e0d4a700 1 -- 172.21.2.101:0/4130064918 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d000c340 con 0x7fd8dc0ef610 2021-02-10T11:26:56.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.329 7fd8e0d4a700 1 -- 172.21.2.101:0/4130064918 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8d000c820 con 0x7fd8dc0ef610 2021-02-10T11:26:56.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.329 7fd8e3f30700 1 -- 172.21.2.101:0/4130064918 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8dc0ef610 msgr2=0x7fd8dc0efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:56.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.329 7fd8e3f30700 1 --2- 172.21.2.101:0/4130064918 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8dc0ef610 0x7fd8dc0efa10 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fd8d0006700 tx=0x7fd8d0009b00).stop 2021-02-10T11:26:56.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.330 7fd8e3f30700 1 -- 172.21.2.101:0/4130064918 shutdown_connections 2021-02-10T11:26:56.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.330 7fd8e3f30700 1 --2- 172.21.2.101:0/4130064918 >> v2:172.21.2.108:3300/0 conn(0x7fd8dc0f0030 0x7fd8dc0f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:56.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.330 7fd8e3f30700 1 --2- 172.21.2.101:0/4130064918 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8dc0ef610 0x7fd8dc0efa10 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:56.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.330 7fd8e3f30700 1 --2- 172.21.2.101:0/4130064918 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8dc1b0470 0x7fd8dc1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:56.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.330 7fd8e3f30700 1 -- 172.21.2.101:0/4130064918 >> 172.21.2.101:0/4130064918 conn(0x7fd8dc0ce3e0 msgr2=0x7fd8dc0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:56.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.330 7fd8e3f30700 1 -- 172.21.2.101:0/4130064918 shutdown_connections 2021-02-10T11:26:56.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.330 7fd8e3f30700 1 -- 172.21.2.101:0/4130064918 wait complete. 2021-02-10T11:26:56.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.344 7fd8e3f30700 1 Processor -- start 2021-02-10T11:26:56.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.344 7fd8e3f30700 1 -- start start 2021-02-10T11:26:56.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.344 7fd8e3f30700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8dc0b20b0 0x7fd8dc0ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:56.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.344 7fd8e3f30700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8dc0b24b0 0x7fd8dc191dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:56.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.344 7fd8e3f30700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd8dc0acf90 0x7fd8dc1923e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:56.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.344 7fd8e3f30700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc0b1b30 con 0x7fd8dc0b24b0 2021-02-10T11:26:56.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.344 7fd8e3f30700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc1a04e0 con 0x7fd8dc0acf90 2021-02-10T11:26:56.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.344 7fd8e3f30700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc1a06c0 con 0x7fd8dc0b20b0 2021-02-10T11:26:56.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.344 7fd8e154b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8dc0b24b0 0x7fd8dc191dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:56.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.344 7fd8e154b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8dc0b24b0 0x7fd8dc191dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47864/0 (socket says 172.21.2.101:47864) 2021-02-10T11:26:56.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.344 7fd8e154b700 1 -- 172.21.2.101:0/4213595561 learned_addr learned my addr 172.21.2.101:0/4213595561 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:26:56.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.345 7fd8e154b700 1 -- 172.21.2.101:0/4213595561 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8dc0b20b0 msgr2=0x7fd8dc0ac970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:26:56.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.345 7fd8e154b700 1 --2- 172.21.2.101:0/4213595561 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8dc0b20b0 0x7fd8dc0ac970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:56.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.345 7fd8e154b700 1 -- 172.21.2.101:0/4213595561 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd8dc0acf90 msgr2=0x7fd8dc1923e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:26:56.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.345 7fd8e154b700 1 --2- 172.21.2.101:0/4213595561 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd8dc0acf90 0x7fd8dc1923e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:56.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.345 7fd8e154b700 1 -- 172.21.2.101:0/4213595561 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8dc192be0 con 0x7fd8dc0b24b0 2021-02-10T11:26:56.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.345 7fd8e154b700 1 --2- 172.21.2.101:0/4213595561 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8dc0b24b0 0x7fd8dc191dc0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fd8d0006f20 tx=0x7fd8d000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:56.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.345 7fd8ce7fc700 1 -- 172.21.2.101:0/4213595561 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8d000bd80 con 0x7fd8dc0b24b0 2021-02-10T11:26:56.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.345 7fd8ce7fc700 1 -- 172.21.2.101:0/4213595561 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d0000c90 con 0x7fd8dc0b24b0 2021-02-10T11:26:56.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.345 7fd8e3f30700 1 -- 172.21.2.101:0/4213595561 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8dc192be0 con 0x7fd8dc0b24b0 2021-02-10T11:26:56.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.345 7fd8ce7fc700 1 -- 172.21.2.101:0/4213595561 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8d00022b0 con 0x7fd8dc0b24b0 2021-02-10T11:26:56.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.345 7fd8e3f30700 1 -- 172.21.2.101:0/4213595561 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8dc192be0 con 0x7fd8dc0b24b0 2021-02-10T11:26:56.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.346 7fd8ce7fc700 1 -- 172.21.2.101:0/4213595561 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd8d0009100 con 0x7fd8dc0b24b0 2021-02-10T11:26:56.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.346 7fd8ce7fc700 1 --2- 172.21.2.101:0/4213595561 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8c4018730 0x7fd8c401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:26:56.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.346 7fd8ce7fc700 1 -- 172.21.2.101:0/4213595561 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd8d0032650 con 0x7fd8dc0b24b0 2021-02-10T11:26:56.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.346 7fd8e1d4c700 1 --2- 172.21.2.101:0/4213595561 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8c4018730 0x7fd8c401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:26:56.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.346 7fd8e1d4c700 1 --2- 172.21.2.101:0/4213595561 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8c4018730 0x7fd8c401ac10 crc :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:26:56.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.377 7fd8e3f30700 1 -- 172.21.2.101:0/4213595561 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8dc105af0 con 0x7fd8dc0b24b0 2021-02-10T11:26:56.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.380 7fd8ce7fc700 1 -- 172.21.2.101:0/4213595561 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd8d0006030 con 0x7fd8dc0b24b0 2021-02-10T11:26:56.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.509 7fd8e3f30700 1 -- 172.21.2.101:0/4213595561 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839318", "target": ["mgr", ""]}) v1 -- 0x7fd8dc105af0 con 0x7fd8c4018730 2021-02-10T11:26:56.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.514 7fd8ce7fc700 1 -- 172.21.2.101:0/4213595561 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fd8d8006970 con 0x7fd8c4018730 2021-02-10T11:26:56.526 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839318/a2d46d50-1b0f-458b-8a19-4fb1f824c520 2021-02-10T11:26:56.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.519 7fd8e3f30700 1 -- 172.21.2.101:0/4213595561 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8c4018730 msgr2=0x7fd8c401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:56.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.519 7fd8e3f30700 1 --2- 172.21.2.101:0/4213595561 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8c4018730 0x7fd8c401ac10 crc :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:56.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.519 7fd8e3f30700 1 -- 172.21.2.101:0/4213595561 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8dc0b24b0 msgr2=0x7fd8dc191dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:26:56.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.519 7fd8e3f30700 1 --2- 172.21.2.101:0/4213595561 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8dc0b24b0 0x7fd8dc191dc0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fd8d0006f20 tx=0x7fd8d000b990).stop 2021-02-10T11:26:56.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.519 7fd8e3f30700 1 -- 172.21.2.101:0/4213595561 shutdown_connections 2021-02-10T11:26:56.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.519 7fd8e3f30700 1 --2- 172.21.2.101:0/4213595561 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd8dc0acf90 0x7fd8dc1923e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:56.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.519 7fd8e3f30700 1 --2- 172.21.2.101:0/4213595561 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8dc0b24b0 0x7fd8dc191dc0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:56.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.519 7fd8e3f30700 1 --2- 172.21.2.101:0/4213595561 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8c4018730 0x7fd8c401ac10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:26:56.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.519 7fd8e3f30700 1 --2- 172.21.2.101:0/4213595561 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8dc0b20b0 0x7fd8dc0ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:26:56.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.519 7fd8e3f30700 1 -- 172.21.2.101:0/4213595561 >> 172.21.2.101:0/4213595561 conn(0x7fd8dc0bf4e0 msgr2=0x7fd8dc0be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:26:56.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.519 7fd8e3f30700 1 -- 172.21.2.101:0/4213595561 shutdown_connections 2021-02-10T11:26:56.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:26:56.519 7fd8e3f30700 1 -- 172.21.2.101:0/4213595561 wait complete. 2021-02-10T11:26:56.537 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839318 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839318/a2d46d50-1b0f-458b-8a19-4fb1f824c520 2021-02-10T11:26:56.537 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:56.537 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:56.537 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:56.873 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:56.874 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:56.874 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839318/a2d46d50-1b0f-458b-8a19-4fb1f824c520/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:57.254 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:57.254 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:57.254 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231869 s, 45.2 MB/s 2021-02-10T11:26:57.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:57.255 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:57.255 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:57.590 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:57.590 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:57.591 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839318/a2d46d50-1b0f-458b-8a19-4fb1f824c520/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:57.968 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:57.968 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:57.968 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252125 s, 41.6 MB/s 2021-02-10T11:26:57.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:57.969 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:57.969 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:58.303 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:58.304 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:58.304 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839318/a2d46d50-1b0f-458b-8a19-4fb1f824c520/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:58.681 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:58.682 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:58.682 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251861 s, 41.6 MB/s 2021-02-10T11:26:58.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:58.683 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:58.683 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:59.019 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:59.019 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:59.020 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839318/a2d46d50-1b0f-458b-8a19-4fb1f824c520/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:26:59.392 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:26:59.393 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:26:59.393 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245959 s, 42.6 MB/s 2021-02-10T11:26:59.394 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:26:59.394 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:26:59.394 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:26:59.731 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:26:59.731 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:26:59.731 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839318/a2d46d50-1b0f-458b-8a19-4fb1f824c520/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:00.109 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:00.109 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:00.109 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248535 s, 42.2 MB/s 2021-02-10T11:27:00.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:00.110 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:00.110 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:00.447 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:00.447 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:00.447 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839318/a2d46d50-1b0f-458b-8a19-4fb1f824c520/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:00.824 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:00.824 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:00.825 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248548 s, 42.2 MB/s 2021-02-10T11:27:00.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:00.826 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:00.826 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:01.160 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:01.161 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:01.161 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839318/a2d46d50-1b0f-458b-8a19-4fb1f824c520/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:01.537 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:01.537 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:01.538 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243747 s, 43.0 MB/s 2021-02-10T11:27:01.538 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:01.539 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:01.539 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:01.874 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:01.874 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:01.874 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839318/a2d46d50-1b0f-458b-8a19-4fb1f824c520/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:02.250 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:02.250 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:02.250 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245979 s, 42.6 MB/s 2021-02-10T11:27:02.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:02.251 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:02.251 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:02.587 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:02.587 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:02.587 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839318/a2d46d50-1b0f-458b-8a19-4fb1f824c520/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:02.965 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:02.965 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:02.965 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248781 s, 42.1 MB/s 2021-02-10T11:27:02.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:02.966 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:02.966 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:03.301 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:03.302 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:03.302 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839318/a2d46d50-1b0f-458b-8a19-4fb1f824c520/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:03.675 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:03.675 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:03.675 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0218547 s, 48.0 MB/s 2021-02-10T11:27:03.676 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839319 2021-02-10T11:27:04.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.082 7f8321721700 1 Processor -- start 2021-02-10T11:27:04.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.082 7f8321721700 1 -- start start 2021-02-10T11:27:04.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.082 7f8321721700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f831c1b0470 0x7f831c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:04.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.082 7f8321721700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f831c0ef610 0x7f831c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:04.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.082 7f8321721700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f831c0f0030 0x7f831c0f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:04.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.082 7f8321721700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f831c0e5ea0 con 0x7f831c0ef610 2021-02-10T11:27:04.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.082 7f8321721700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f831c0f0b50 con 0x7f831c1b0470 2021-02-10T11:27:04.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.082 7f8321721700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831c0e52e0 con 0x7f831c0f0030 2021-02-10T11:27:04.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.083 7f831aabe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f831c0ef610 0x7f831c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:04.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.083 7f831aabe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f831c0ef610 0x7f831c0efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47870/0 (socket says 172.21.2.101:47870) 2021-02-10T11:27:04.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.083 7f831aabe700 1 -- 172.21.2.101:0/1403936823 learned_addr learned my addr 172.21.2.101:0/1403936823 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:27:04.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.083 7f831aabe700 1 -- 172.21.2.101:0/1403936823 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f831c0f0030 msgr2=0x7f831c0f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:27:04.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.083 7f831aabe700 1 --2- 172.21.2.101:0/1403936823 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f831c0f0030 0x7f831c0f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:04.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.083 7f831aabe700 1 -- 172.21.2.101:0/1403936823 >> v2:172.21.2.108:3300/0 conn(0x7f831c1b0470 msgr2=0x7f831c1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:27:04.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.083 7f831aabe700 1 --2- 172.21.2.101:0/1403936823 >> v2:172.21.2.108:3300/0 conn(0x7f831c1b0470 0x7f831c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:04.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.083 7f831aabe700 1 -- 172.21.2.101:0/1403936823 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c0f10d0 con 0x7f831c0ef610 2021-02-10T11:27:04.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.083 7f831aabe700 1 --2- 172.21.2.101:0/1403936823 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0ef610 0x7f831c0efa10 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7f8310006700 tx=0x7f8310009b80).ready entity=mon.0 client_cookie=370ee429090faaa0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:04.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.083 7f831a2bd700 1 -- 172.21.2.101:0/1403936823 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f831000c0c0 con 0x7f831c0ef610 2021-02-10T11:27:04.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.085 7f831a2bd700 1 -- 172.21.2.101:0/1403936823 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831000c440 con 0x7f831c0ef610 2021-02-10T11:27:04.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.085 7f831a2bd700 1 -- 172.21.2.101:0/1403936823 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f831000c8a0 con 0x7f831c0ef610 2021-02-10T11:27:04.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.085 7f8321721700 1 -- 172.21.2.101:0/1403936823 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0ef610 msgr2=0x7f831c0efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:04.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.085 7f8321721700 1 --2- 172.21.2.101:0/1403936823 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0ef610 0x7f831c0efa10 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7f8310006700 tx=0x7f8310009b80).stop 2021-02-10T11:27:04.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.085 7f8321721700 1 -- 172.21.2.101:0/1403936823 shutdown_connections 2021-02-10T11:27:04.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.085 7f8321721700 1 --2- 172.21.2.101:0/1403936823 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f831c0f0030 0x7f831c0f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:04.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.085 7f8321721700 1 --2- 172.21.2.101:0/1403936823 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0ef610 0x7f831c0efa10 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:04.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.085 7f8321721700 1 --2- 172.21.2.101:0/1403936823 >> v2:172.21.2.108:3300/0 conn(0x7f831c1b0470 0x7f831c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:04.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.085 7f8321721700 1 -- 172.21.2.101:0/1403936823 >> 172.21.2.101:0/1403936823 conn(0x7f831c0ce3e0 msgr2=0x7f831c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:04.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.086 7f8321721700 1 -- 172.21.2.101:0/1403936823 shutdown_connections 2021-02-10T11:27:04.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.086 7f8321721700 1 -- 172.21.2.101:0/1403936823 wait complete. 2021-02-10T11:27:04.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.099 7f8321721700 1 Processor -- start 2021-02-10T11:27:04.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.099 7f8321721700 1 -- start start 2021-02-10T11:27:04.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.099 7f8321721700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f831c0bf910 0x7f831c0bfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:04.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.099 7f8321721700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f831c0c0330 0x7f831c0c0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:04.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.099 7f8321721700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0c13d0 0x7f831c0c17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:04.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.099 7f8321721700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831c109930 con 0x7f831c0c13d0 2021-02-10T11:27:04.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.099 7f8321721700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831c109b50 con 0x7f831c0c0330 2021-02-10T11:27:04.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.099 7f8321721700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831c0c1df0 con 0x7f831c0bf910 2021-02-10T11:27:04.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.100 7f831bac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0c13d0 0x7f831c0c17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:04.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.100 7f831bac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0c13d0 0x7f831c0c17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47876/0 (socket says 172.21.2.101:47876) 2021-02-10T11:27:04.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.100 7f831bac0700 1 -- 172.21.2.101:0/2646099879 learned_addr learned my addr 172.21.2.101:0/2646099879 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:27:04.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.100 7f831b2bf700 1 --2- 172.21.2.101:0/2646099879 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f831c0bf910 0x7f831c0bfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:04.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.100 7f831aabe700 1 --2- 172.21.2.101:0/2646099879 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f831c0c0330 0x7f831c0c0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:04.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.100 7f831bac0700 1 -- 172.21.2.101:0/2646099879 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f831c0bf910 msgr2=0x7f831c0bfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:04.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.100 7f831bac0700 1 --2- 172.21.2.101:0/2646099879 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f831c0bf910 0x7f831c0bfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:04.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.100 7f831bac0700 1 -- 172.21.2.101:0/2646099879 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f831c0c0330 msgr2=0x7f831c0c0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:04.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.100 7f831bac0700 1 --2- 172.21.2.101:0/2646099879 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f831c0c0330 0x7f831c0c0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:04.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.100 7f831bac0700 1 -- 172.21.2.101:0/2646099879 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c0c1fd0 con 0x7f831c0c13d0 2021-02-10T11:27:04.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.100 7f831aabe700 1 --2- 172.21.2.101:0/2646099879 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f831c0c0330 0x7f831c0c0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:27:04.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.100 7f831bac0700 1 --2- 172.21.2.101:0/2646099879 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0c13d0 0x7f831c0c17d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f8314009980 tx=0x7f831400bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:04.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.101 7f8303fff700 1 -- 172.21.2.101:0/2646099879 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8314005a60 con 0x7f831c0c13d0 2021-02-10T11:27:04.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.101 7f8303fff700 1 -- 172.21.2.101:0/2646099879 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8314005ce0 con 0x7f831c0c13d0 2021-02-10T11:27:04.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.101 7f8303fff700 1 -- 172.21.2.101:0/2646099879 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8314006220 con 0x7f831c0c13d0 2021-02-10T11:27:04.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.101 7f8321721700 1 -- 172.21.2.101:0/2646099879 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f831c0c1fd0 con 0x7f831c0c13d0 2021-02-10T11:27:04.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.101 7f8321721700 1 -- 172.21.2.101:0/2646099879 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f831c0c2560 con 0x7f831c0c13d0 2021-02-10T11:27:04.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.101 7f8303fff700 1 -- 172.21.2.101:0/2646099879 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8314006030 con 0x7f831c0c13d0 2021-02-10T11:27:04.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.101 7f8303fff700 1 --2- 172.21.2.101:0/2646099879 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82fc0186c0 0x7f82fc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:04.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.101 7f8303fff700 1 -- 172.21.2.101:0/2646099879 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f83140328d0 con 0x7f831c0c13d0 2021-02-10T11:27:04.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.103 7f831b2bf700 1 --2- 172.21.2.101:0/2646099879 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82fc0186c0 0x7f82fc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:04.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.103 7f831b2bf700 1 --2- 172.21.2.101:0/2646099879 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82fc0186c0 0x7f82fc01abc0 crc :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.133 7f8321721700 1 -- 172.21.2.101:0/2646099879 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f831c0f0b50 con 0x7f831c0c13d0 2021-02-10T11:27:04.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.136 7f8303fff700 1 -- 172.21.2.101:0/2646099879 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8314005870 con 0x7f831c0c13d0 2021-02-10T11:27:04.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.262 7f8321721700 1 -- 172.21.2.101:0/2646099879 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839319", "target": ["mgr", ""]}) v1 -- 0x7f831c0c2560 con 0x7f82fc0186c0 2021-02-10T11:27:04.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.290 7f8303fff700 1 -- 172.21.2.101:0/2646099879 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f830c0009b0 con 0x7f82fc0186c0 2021-02-10T11:27:04.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.295 7f8321721700 1 -- 172.21.2.101:0/2646099879 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82fc0186c0 msgr2=0x7f82fc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:04.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.295 7f8321721700 1 --2- 172.21.2.101:0/2646099879 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82fc0186c0 0x7f82fc01abc0 crc :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:04.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.295 7f8321721700 1 -- 172.21.2.101:0/2646099879 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0c13d0 msgr2=0x7f831c0c17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:04.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.295 7f8321721700 1 --2- 172.21.2.101:0/2646099879 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0c13d0 0x7f831c0c17d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f8314009980 tx=0x7f831400bcd0).stop 2021-02-10T11:27:04.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.295 7f8321721700 1 -- 172.21.2.101:0/2646099879 shutdown_connections 2021-02-10T11:27:04.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.295 7f8321721700 1 --2- 172.21.2.101:0/2646099879 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0c13d0 0x7f831c0c17d0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:04.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.295 7f8321721700 1 --2- 172.21.2.101:0/2646099879 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f831c0c0330 0x7f831c0c0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:04.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.295 7f8321721700 1 --2- 172.21.2.101:0/2646099879 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82fc0186c0 0x7f82fc01abc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:04.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.295 7f8321721700 1 --2- 172.21.2.101:0/2646099879 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f831c0bf910 0x7f831c0bfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:04.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.295 7f8321721700 1 -- 172.21.2.101:0/2646099879 >> 172.21.2.101:0/2646099879 conn(0x7f831c0bf4e0 msgr2=0x7f831c191190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:04.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.296 7f8321721700 1 -- 172.21.2.101:0/2646099879 shutdown_connections 2021-02-10T11:27:04.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.296 7f8321721700 1 -- 172.21.2.101:0/2646099879 wait complete. 2021-02-10T11:27:04.313 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839319 2021-02-10T11:27:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.719 7f8d3b29d700 1 Processor -- start 2021-02-10T11:27:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.719 7f8d3b29d700 1 -- start start 2021-02-10T11:27:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.719 7f8d3b29d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8d341b0870 0x7f8d340f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.719 7f8d3b29d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d341b0c70 con 0x7f8d341b0470 2021-02-10T11:27:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.719 7f8d3b29d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d340f0b50 con 0x7f8d340efe00 2021-02-10T11:27:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.719 7f8d3b29d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d340e5ea0 con 0x7f8d341b0870 2021-02-10T11:27:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.720 7f8d390b9700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8d341b0470 0x7f8d340ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48376/0 (socket says 172.21.2.101:48376) 2021-02-10T11:27:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.720 7f8d390b9700 1 -- 172.21.2.101:0/1469910831 learned_addr learned my addr 172.21.2.101:0/1469910831 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:27:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.720 7f8d27fff700 1 -- 172.21.2.101:0/1469910831 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 437929073 0 0) 0x7f8d300010b0 con 0x7f8d341b0470 2021-02-10T11:27:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.720 7f8d27fff700 1 -- 172.21.2.101:0/1469910831 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d20002fe0 con 0x7f8d341b0470 2021-02-10T11:27:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.720 7f8d388b8700 1 --2- 172.21.2.101:0/1469910831 >> v2:172.21.2.108:3300/0 conn(0x7f8d341b0870 0x7f8d340f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.720 7f8d27fff700 1 -- 172.21.2.101:0/1469910831 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3092746456 0 0) 0x7f8d30001010 con 0x7f8d341b0470 2021-02-10T11:27:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.720 7f8d27fff700 1 -- 172.21.2.101:0/1469910831 >> v2:172.21.2.108:3300/0 conn(0x7f8d341b0870 msgr2=0x7f8d340f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:27:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.720 7f8d27fff700 1 --2- 172.21.2.101:0/1469910831 >> v2:172.21.2.108:3300/0 conn(0x7f8d341b0870 0x7f8d340f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.720 7f8d27fff700 1 -- 172.21.2.101:0/1469910831 >> v1:172.21.2.108:6789/0 conn(0x7f8d340efe00 legacy=0x7f8d340f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.720 7f8d27fff700 1 -- 172.21.2.101:0/1469910831 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d340e52e0 con 0x7f8d341b0470 2021-02-10T11:27:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.720 7f8d27fff700 1 -- 172.21.2.101:0/1469910831 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f8d30001530 con 0x7f8d341b0470 2021-02-10T11:27:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.720 7f8d27fff700 1 -- 172.21.2.101:0/1469910831 >> v1:172.21.2.101:6789/0 conn(0x7f8d341b0470 legacy=0x7f8d340ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.720 7f8d27fff700 1 --2- 172.21.2.101:0/1469910831 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d20003ac0 0x7f8d20005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.720 7f8d27fff700 1 --2- 172.21.2.101:0/1469910831 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d20006790 0x7f8d20008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.720 7f8d27fff700 1 --2- 172.21.2.101:0/1469910831 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d20009590 0x7f8d2000bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.720 7f8d27fff700 1 -- 172.21.2.101:0/1469910831 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2000c300 con 0x7f8d20009590 2021-02-10T11:27:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.720 7f8d27fff700 1 -- 172.21.2.101:0/1469910831 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2000c520 con 0x7f8d20006790 2021-02-10T11:27:04.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.721 7f8d27fff700 1 -- 172.21.2.101:0/1469910831 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d20050750 con 0x7f8d20003ac0 2021-02-10T11:27:04.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.721 7f8d390b9700 1 --2- 172.21.2.101:0/1469910831 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d20003ac0 0x7f8d20005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:04.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.721 7f8d398ba700 1 --2- 172.21.2.101:0/1469910831 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d20009590 0x7f8d2000bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:04.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.721 7f8d388b8700 1 --2- 172.21.2.101:0/1469910831 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d20006790 0x7f8d20008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:04.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.721 7f8d390b9700 1 -- 172.21.2.101:0/1469910831 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d20006790 msgr2=0x7f8d20008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:27:04.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.721 7f8d390b9700 1 --2- 172.21.2.101:0/1469910831 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d20006790 0x7f8d20008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:04.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.721 7f8d390b9700 1 -- 172.21.2.101:0/1469910831 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d20009590 msgr2=0x7f8d2000bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:27:04.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.721 7f8d390b9700 1 --2- 172.21.2.101:0/1469910831 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d20009590 0x7f8d2000bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:04.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.721 7f8d390b9700 1 -- 172.21.2.101:0/1469910831 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d200509b0 con 0x7f8d20003ac0 2021-02-10T11:27:04.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.721 7f8d398ba700 1 -- 172.21.2.101:0/1469910831 reap_dead start 2021-02-10T11:27:04.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.721 7f8d390b9700 1 --2- 172.21.2.101:0/1469910831 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d20003ac0 0x7f8d20005ef0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f8d30001380 tx=0x7f8d30007d60).ready entity=mon.2 client_cookie=48666e405807ef5c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:04.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.721 7f8d27fff700 1 -- 172.21.2.101:0/1469910831 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d3000a520 con 0x7f8d20003ac0 2021-02-10T11:27:04.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.721 7f8d27fff700 1 -- 172.21.2.101:0/1469910831 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3000a520 con 0x7f8d20003ac0 2021-02-10T11:27:04.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.721 7f8d3b29d700 1 -- 172.21.2.101:0/1469910831 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d20003ac0 msgr2=0x7f8d20005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:04.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.721 7f8d27fff700 1 -- 172.21.2.101:0/1469910831 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d3000a860 con 0x7f8d20003ac0 2021-02-10T11:27:04.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.721 7f8d3b29d700 1 --2- 172.21.2.101:0/1469910831 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d20003ac0 0x7f8d20005ef0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f8d30001380 tx=0x7f8d30007d60).stop 2021-02-10T11:27:04.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.722 7f8d3b29d700 1 -- 172.21.2.101:0/1469910831 shutdown_connections 2021-02-10T11:27:04.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.722 7f8d3b29d700 1 --2- 172.21.2.101:0/1469910831 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d20003ac0 0x7f8d20005ef0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:04.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.722 7f8d3b29d700 1 -- 172.21.2.101:0/1469910831 >> 172.21.2.101:0/1469910831 conn(0x7f8d340ce3e0 msgr2=0x7f8d340e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:04.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.722 7f8d3b29d700 1 -- 172.21.2.101:0/1469910831 shutdown_connections 2021-02-10T11:27:04.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.722 7f8d3b29d700 1 -- 172.21.2.101:0/1469910831 wait complete. 2021-02-10T11:27:04.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.737 7f8d3b29d700 1 Processor -- start 2021-02-10T11:27:04.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.737 7f8d3b29d700 1 -- start start 2021-02-10T11:27:04.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.737 7f8d3b29d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d340b7db0 0x7f8d340b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:04.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.737 7f8d3b29d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d340b8880 0x7f8d341a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:04.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.737 7f8d3b29d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d340b8c80 0x7f8d341a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:04.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.737 7f8d3b29d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d340b79b0 con 0x7f8d340b7db0 2021-02-10T11:27:04.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.737 7f8d3b29d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d341a8090 con 0x7f8d340b8c80 2021-02-10T11:27:04.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.737 7f8d3b29d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d341a82b0 con 0x7f8d340b8880 2021-02-10T11:27:04.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.737 7f8d390b9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d340b7db0 0x7f8d340b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:04.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.737 7f8d390b9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d340b7db0 0x7f8d340b8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47892/0 (socket says 172.21.2.101:47892) 2021-02-10T11:27:04.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.737 7f8d390b9700 1 -- 172.21.2.101:0/3268667910 learned_addr learned my addr 172.21.2.101:0/3268667910 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:27:04.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.738 7f8d390b9700 1 -- 172.21.2.101:0/3268667910 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d340b8880 msgr2=0x7f8d341a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:04.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.738 7f8d390b9700 1 --2- 172.21.2.101:0/3268667910 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d340b8880 0x7f8d341a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:04.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.738 7f8d390b9700 1 -- 172.21.2.101:0/3268667910 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d340b8c80 msgr2=0x7f8d341a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:04.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.738 7f8d390b9700 1 --2- 172.21.2.101:0/3268667910 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d340b8c80 0x7f8d341a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:04.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.738 7f8d390b9700 1 -- 172.21.2.101:0/3268667910 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d34190c10 con 0x7f8d340b7db0 2021-02-10T11:27:04.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.738 7f8d390b9700 1 --2- 172.21.2.101:0/3268667910 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d340b7db0 0x7f8d340b8260 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f8d3000a5a0 tx=0x7f8d3000a940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:04.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.738 7f8d25ffb700 1 -- 172.21.2.101:0/3268667910 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d300072d0 con 0x7f8d340b7db0 2021-02-10T11:27:04.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.738 7f8d25ffb700 1 -- 172.21.2.101:0/3268667910 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d30000d70 con 0x7f8d340b7db0 2021-02-10T11:27:04.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.738 7f8d25ffb700 1 -- 172.21.2.101:0/3268667910 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d30004350 con 0x7f8d340b7db0 2021-02-10T11:27:04.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.738 7f8d3b29d700 1 -- 172.21.2.101:0/3268667910 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d34190c10 con 0x7f8d340b7db0 2021-02-10T11:27:04.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.738 7f8d3b29d700 1 -- 172.21.2.101:0/3268667910 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d34190c10 con 0x7f8d340b7db0 2021-02-10T11:27:04.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.739 7f8d25ffb700 1 -- 172.21.2.101:0/3268667910 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8d30004140 con 0x7f8d340b7db0 2021-02-10T11:27:04.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.739 7f8d25ffb700 1 --2- 172.21.2.101:0/3268667910 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d14018730 0x7f8d1401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:04.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.739 7f8d25ffb700 1 -- 172.21.2.101:0/3268667910 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8d300327d0 con 0x7f8d340b7db0 2021-02-10T11:27:04.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.739 7f8d388b8700 1 --2- 172.21.2.101:0/3268667910 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d14018730 0x7f8d1401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:04.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.739 7f8d388b8700 1 --2- 172.21.2.101:0/3268667910 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d14018730 0x7f8d1401abc0 crc :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:04.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.769 7f8d3b29d700 1 -- 172.21.2.101:0/3268667910 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d34105b30 con 0x7f8d340b7db0 2021-02-10T11:27:04.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.773 7f8d25ffb700 1 -- 172.21.2.101:0/3268667910 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8d300072d0 con 0x7f8d340b7db0 2021-02-10T11:27:04.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.901 7f8d3b29d700 1 -- 172.21.2.101:0/3268667910 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839319", "target": ["mgr", ""]}) v1 -- 0x7f8d340f0b50 con 0x7f8d14018730 2021-02-10T11:27:04.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.905 7f8d25ffb700 1 -- 172.21.2.101:0/3268667910 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f8d28006e70 con 0x7f8d14018730 2021-02-10T11:27:04.918 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839319/77e4fd1a-bbde-4764-93d1-ad0104e2f765 2021-02-10T11:27:04.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.911 7f8d3b29d700 1 -- 172.21.2.101:0/3268667910 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d14018730 msgr2=0x7f8d1401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:04.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.911 7f8d3b29d700 1 --2- 172.21.2.101:0/3268667910 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d14018730 0x7f8d1401abc0 crc :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:04.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.911 7f8d3b29d700 1 -- 172.21.2.101:0/3268667910 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d340b7db0 msgr2=0x7f8d340b8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:04.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.911 7f8d3b29d700 1 --2- 172.21.2.101:0/3268667910 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d340b7db0 0x7f8d340b8260 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f8d3000a5a0 tx=0x7f8d3000a940).stop 2021-02-10T11:27:04.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.911 7f8d3b29d700 1 -- 172.21.2.101:0/3268667910 shutdown_connections 2021-02-10T11:27:04.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.911 7f8d3b29d700 1 --2- 172.21.2.101:0/3268667910 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d14018730 0x7f8d1401abc0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:04.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.911 7f8d3b29d700 1 --2- 172.21.2.101:0/3268667910 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d340b8880 0x7f8d341a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:04.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.911 7f8d3b29d700 1 --2- 172.21.2.101:0/3268667910 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d340b8c80 0x7f8d341a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:04.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.911 7f8d3b29d700 1 --2- 172.21.2.101:0/3268667910 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d340b7db0 0x7f8d340b8260 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:04.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.911 7f8d3b29d700 1 -- 172.21.2.101:0/3268667910 >> 172.21.2.101:0/3268667910 conn(0x7f8d34094e10 msgr2=0x7f8d340a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:04.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.911 7f8d3b29d700 1 -- 172.21.2.101:0/3268667910 shutdown_connections 2021-02-10T11:27:04.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:04.911 7f8d3b29d700 1 -- 172.21.2.101:0/3268667910 wait complete. 2021-02-10T11:27:04.928 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839319 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839319/77e4fd1a-bbde-4764-93d1-ad0104e2f765 2021-02-10T11:27:04.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:04.929 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:04.929 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:05.266 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:05.266 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:05.266 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839319/77e4fd1a-bbde-4764-93d1-ad0104e2f765/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:05.650 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:05.650 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:05.650 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253669 s, 41.3 MB/s 2021-02-10T11:27:05.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:05.651 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:05.652 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:05.987 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:05.987 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:05.987 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839319/77e4fd1a-bbde-4764-93d1-ad0104e2f765/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:06.366 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:06.366 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:06.366 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0263917 s, 39.7 MB/s 2021-02-10T11:27:06.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:06.367 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:06.367 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:06.703 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:06.703 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:06.703 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839319/77e4fd1a-bbde-4764-93d1-ad0104e2f765/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:07.079 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:07.080 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:07.080 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234069 s, 44.8 MB/s 2021-02-10T11:27:07.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:07.081 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:07.081 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:07.415 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:07.416 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:07.416 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839319/77e4fd1a-bbde-4764-93d1-ad0104e2f765/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:07.793 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:07.793 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:07.793 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.025975 s, 40.4 MB/s 2021-02-10T11:27:07.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:07.794 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:07.795 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:08.129 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:08.129 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:08.130 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839319/77e4fd1a-bbde-4764-93d1-ad0104e2f765/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:08.505 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:08.505 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:08.505 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0227029 s, 46.2 MB/s 2021-02-10T11:27:08.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:08.506 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:08.506 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:08.842 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:08.842 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:08.842 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839319/77e4fd1a-bbde-4764-93d1-ad0104e2f765/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:09.216 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:09.216 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:09.217 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0258796 s, 40.5 MB/s 2021-02-10T11:27:09.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:09.218 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:09.218 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:09.552 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:09.553 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:09.553 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839319/77e4fd1a-bbde-4764-93d1-ad0104e2f765/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:09.930 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:09.930 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:09.930 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246208 s, 42.6 MB/s 2021-02-10T11:27:09.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:09.931 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:09.931 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:10.268 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:10.268 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:10.268 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839319/77e4fd1a-bbde-4764-93d1-ad0104e2f765/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:10.644 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:10.645 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:10.645 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243978 s, 43.0 MB/s 2021-02-10T11:27:10.646 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:10.646 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:10.646 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:10.982 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:10.982 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:10.982 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839319/77e4fd1a-bbde-4764-93d1-ad0104e2f765/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:11.360 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:11.360 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:11.360 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249587 s, 42.0 MB/s 2021-02-10T11:27:11.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:11.361 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:11.361 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:11.698 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:11.698 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:11.698 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839319/77e4fd1a-bbde-4764-93d1-ad0104e2f765/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:12.073 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:12.074 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:12.074 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241962 s, 43.3 MB/s 2021-02-10T11:27:12.075 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839320 2021-02-10T11:27:12.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.480 7f6b71f92700 1 Processor -- start 2021-02-10T11:27:12.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.480 7f6b71f92700 1 -- start start 2021-02-10T11:27:12.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.481 7f6b71f92700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b6c1b0870 0x7f6b6c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:12.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.481 7f6b71f92700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6b6c0efe00 0x7f6b6c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:12.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.481 7f6b71f92700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b6c1b0c70 con 0x7f6b6c1b0470 2021-02-10T11:27:12.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.481 7f6b71f92700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b6c0f0b50 con 0x7f6b6c0efe00 2021-02-10T11:27:12.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.481 7f6b71f92700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c0f0820 con 0x7f6b6c1b0870 2021-02-10T11:27:12.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.481 7f6b6bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6b6c0efe00 0x7f6b6c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:12.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.481 7f6b6b7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f6b6c1b0470 0x7f6b6c0ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48392/0 (socket says 172.21.2.101:48392) 2021-02-10T11:27:12.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.481 7f6b6b7fe700 1 -- 172.21.2.101:0/3565962511 learned_addr learned my addr 172.21.2.101:0/3565962511 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:27:12.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.482 7f6b6affd700 1 --2- 172.21.2.101:0/3565962511 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b6c1b0870 0x7f6b6c0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:12.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.482 7f6b6bfff700 1 -- 172.21.2.101:0/3565962511 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b6c1b0870 msgr2=0x7f6b6c0f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:27:12.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.482 7f6b6bfff700 1 --2- 172.21.2.101:0/3565962511 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b6c1b0870 0x7f6b6c0f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:12.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.482 7f6b6bfff700 1 -- 172.21.2.101:0/3565962511 >> v1:172.21.2.101:6789/0 conn(0x7f6b6c1b0470 legacy=0x7f6b6c0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:12.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.482 7f6b6a7fc700 1 -- 172.21.2.101:0/3565962511 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697571797 0 0) 0x7f6b5c000ce0 con 0x7f6b6c1b0470 2021-02-10T11:27:12.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.482 7f6b6bfff700 1 -- 172.21.2.101:0/3565962511 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c0e52e0 con 0x7f6b6c0efe00 2021-02-10T11:27:12.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.482 7f6b6affd700 1 --2- 172.21.2.101:0/3565962511 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b6c1b0870 0x7f6b6c0f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:27:12.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.482 7f6b6bfff700 1 --2- 172.21.2.101:0/3565962511 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b6c0efe00 0x7f6b6c0f0200 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f6b64006ab0 tx=0x7f6b64009b40).ready entity=mon.0 client_cookie=5aebe4397dfac8fb server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:12.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.482 7f6b6a7fc700 1 -- 172.21.2.101:0/3565962511 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6b64000c10 con 0x7f6b6c0efe00 2021-02-10T11:27:12.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.482 7f6b6a7fc700 1 -- 172.21.2.101:0/3565962511 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b640009b0 con 0x7f6b6c0efe00 2021-02-10T11:27:12.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.482 7f6b6a7fc700 1 -- 172.21.2.101:0/3565962511 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6b640024d0 con 0x7f6b6c0efe00 2021-02-10T11:27:12.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.482 7f6b71f92700 1 -- 172.21.2.101:0/3565962511 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b6c0efe00 msgr2=0x7f6b6c0f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:12.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.482 7f6b71f92700 1 --2- 172.21.2.101:0/3565962511 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b6c0efe00 0x7f6b6c0f0200 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f6b64006ab0 tx=0x7f6b64009b40).stop 2021-02-10T11:27:12.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.483 7f6b71f92700 1 -- 172.21.2.101:0/3565962511 shutdown_connections 2021-02-10T11:27:12.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.483 7f6b71f92700 1 --2- 172.21.2.101:0/3565962511 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b6c1b0870 0x7f6b6c0f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:12.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.483 7f6b71f92700 1 --2- 172.21.2.101:0/3565962511 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b6c0efe00 0x7f6b6c0f0200 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:12.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.483 7f6b71f92700 1 -- 172.21.2.101:0/3565962511 >> 172.21.2.101:0/3565962511 conn(0x7f6b6c0ce3e0 msgr2=0x7f6b6c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:12.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.483 7f6b71f92700 1 -- 172.21.2.101:0/3565962511 shutdown_connections 2021-02-10T11:27:12.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.483 7f6b71f92700 1 -- 172.21.2.101:0/3565962511 wait complete. 2021-02-10T11:27:12.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.498 7f6b71f92700 1 Processor -- start 2021-02-10T11:27:12.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.498 7f6b71f92700 1 -- start start 2021-02-10T11:27:12.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.498 7f6b71f92700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b6c0a0420 0x7f6b6c0a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:12.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.498 7f6b71f92700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b6c189d00 0x7f6b6c18a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:12.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.498 7f6b71f92700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b6c18a720 0x7f6b6c0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:12.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.498 7f6b71f92700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c189780 con 0x7f6b6c0a0420 2021-02-10T11:27:12.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.498 7f6b71f92700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c18ab20 con 0x7f6b6c189d00 2021-02-10T11:27:12.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.498 7f6b71f92700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c0a1140 con 0x7f6b6c18a720 2021-02-10T11:27:12.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.498 7f6b6b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b6c0a0420 0x7f6b6c0a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:12.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.499 7f6b6b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b6c0a0420 0x7f6b6c0a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47904/0 (socket says 172.21.2.101:47904) 2021-02-10T11:27:12.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.499 7f6b6b7fe700 1 -- 172.21.2.101:0/1606116285 learned_addr learned my addr 172.21.2.101:0/1606116285 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:27:12.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.499 7f6b6b7fe700 1 -- 172.21.2.101:0/1606116285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b6c18a720 msgr2=0x7f6b6c0b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:12.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.499 7f6b6b7fe700 1 --2- 172.21.2.101:0/1606116285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b6c18a720 0x7f6b6c0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:12.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.499 7f6b6b7fe700 1 -- 172.21.2.101:0/1606116285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b6c189d00 msgr2=0x7f6b6c18a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:12.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.499 7f6b6b7fe700 1 --2- 172.21.2.101:0/1606116285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b6c189d00 0x7f6b6c18a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:12.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.499 7f6b6b7fe700 1 -- 172.21.2.101:0/1606116285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c0b8200 con 0x7f6b6c0a0420 2021-02-10T11:27:12.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.499 7f6b6b7fe700 1 --2- 172.21.2.101:0/1606116285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b6c0a0420 0x7f6b6c0a0820 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f6b5c006f00 tx=0x7f6b5c00b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:12.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.499 7f6b63fff700 1 -- 172.21.2.101:0/1606116285 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6b5c006020 con 0x7f6b6c0a0420 2021-02-10T11:27:12.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.499 7f6b63fff700 1 -- 172.21.2.101:0/1606116285 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c0063a0 con 0x7f6b6c0a0420 2021-02-10T11:27:12.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.499 7f6b63fff700 1 -- 172.21.2.101:0/1606116285 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6b5c004590 con 0x7f6b6c0a0420 2021-02-10T11:27:12.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.499 7f6b71f92700 1 -- 172.21.2.101:0/1606116285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6c0b8200 con 0x7f6b6c0a0420 2021-02-10T11:27:12.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.499 7f6b71f92700 1 -- 172.21.2.101:0/1606116285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6c0b8200 con 0x7f6b6c0a0420 2021-02-10T11:27:12.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.500 7f6b63fff700 1 -- 172.21.2.101:0/1606116285 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6b5c00a000 con 0x7f6b6c0a0420 2021-02-10T11:27:12.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.500 7f6b63fff700 1 --2- 172.21.2.101:0/1606116285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b500186c0 0x7f6b5001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:12.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.500 7f6b63fff700 1 -- 172.21.2.101:0/1606116285 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6b5c032540 con 0x7f6b6c0a0420 2021-02-10T11:27:12.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.500 7f6b6affd700 1 --2- 172.21.2.101:0/1606116285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b500186c0 0x7f6b5001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:12.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.500 7f6b6affd700 1 --2- 172.21.2.101:0/1606116285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b500186c0 0x7f6b5001abc0 crc :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:12.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.531 7f6b71f92700 1 -- 172.21.2.101:0/1606116285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b6c105ae0 con 0x7f6b6c0a0420 2021-02-10T11:27:12.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.534 7f6b63fff700 1 -- 172.21.2.101:0/1606116285 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6b5c006b90 con 0x7f6b6c0a0420 2021-02-10T11:27:12.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.662 7f6b71f92700 1 -- 172.21.2.101:0/1606116285 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839320", "target": ["mgr", ""]}) v1 -- 0x7f6b6c0f0b50 con 0x7f6b500186c0 2021-02-10T11:27:12.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.689 7f6b63fff700 1 -- 172.21.2.101:0/1606116285 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f6b54000d30 con 0x7f6b500186c0 2021-02-10T11:27:12.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.695 7f6b71f92700 1 -- 172.21.2.101:0/1606116285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b500186c0 msgr2=0x7f6b5001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:12.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.695 7f6b71f92700 1 --2- 172.21.2.101:0/1606116285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b500186c0 0x7f6b5001abc0 crc :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:12.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.695 7f6b71f92700 1 -- 172.21.2.101:0/1606116285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b6c0a0420 msgr2=0x7f6b6c0a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:12.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.695 7f6b71f92700 1 --2- 172.21.2.101:0/1606116285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b6c0a0420 0x7f6b6c0a0820 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f6b5c006f00 tx=0x7f6b5c00b940).stop 2021-02-10T11:27:12.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.695 7f6b71f92700 1 -- 172.21.2.101:0/1606116285 shutdown_connections 2021-02-10T11:27:12.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.695 7f6b71f92700 1 --2- 172.21.2.101:0/1606116285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b500186c0 0x7f6b5001abc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:12.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.695 7f6b71f92700 1 --2- 172.21.2.101:0/1606116285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b6c18a720 0x7f6b6c0b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:12.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.695 7f6b71f92700 1 --2- 172.21.2.101:0/1606116285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b6c189d00 0x7f6b6c18a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:12.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.695 7f6b71f92700 1 --2- 172.21.2.101:0/1606116285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b6c0a0420 0x7f6b6c0a0820 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:12.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.695 7f6b71f92700 1 -- 172.21.2.101:0/1606116285 >> 172.21.2.101:0/1606116285 conn(0x7f6b6c094e10 msgr2=0x7f6b6c0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:12.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.695 7f6b71f92700 1 -- 172.21.2.101:0/1606116285 shutdown_connections 2021-02-10T11:27:12.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:12.695 7f6b71f92700 1 -- 172.21.2.101:0/1606116285 wait complete. 2021-02-10T11:27:12.714 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839320 2021-02-10T11:27:13.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.120 7f58c7f9d700 1 Processor -- start 2021-02-10T11:27:13.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.120 7f58c7f9d700 1 -- start start 2021-02-10T11:27:13.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.120 7f58c7f9d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f58c00f05f0 0x7f58c00f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:13.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.120 7f58c7f9d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c01b0c70 con 0x7f58c01b0870 2021-02-10T11:27:13.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.120 7f58c7f9d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c00f0b50 con 0x7f58c01b0470 2021-02-10T11:27:13.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.120 7f58c7f9d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c00e5ea0 con 0x7f58c00f05f0 2021-02-10T11:27:13.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.120 7f58c65ba700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f58c00f05f0 0x7f58c00f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:13.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.121 7f58c65ba700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f58c00f05f0 0x7f58c00f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47916/0 (socket says 172.21.2.101:47916) 2021-02-10T11:27:13.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.121 7f58c65ba700 1 -- 172.21.2.101:0/2573615573 learned_addr learned my addr 172.21.2.101:0/2573615573 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:27:13.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.121 7f58c55b8700 1 --1- 172.21.2.101:0/2573615573 >> v1:172.21.2.101:6789/0 conn(0x7f58c01b0870 0x7f58c00efe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48406/0 (socket says 172.21.2.101:48406) 2021-02-10T11:27:13.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.121 7f58c65ba700 1 -- 172.21.2.101:0/2573615573 >> v1:172.21.2.108:6789/0 conn(0x7f58c01b0470 legacy=0x7f58c00ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:13.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.121 7f58c65ba700 1 -- 172.21.2.101:0/2573615573 >> v1:172.21.2.101:6789/0 conn(0x7f58c01b0870 legacy=0x7f58c00efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:13.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.121 7f58c4db7700 1 -- 172.21.2.101:0/2573615573 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1310877489 0 0) 0x7f58b40009b0 con 0x7f58c01b0870 2021-02-10T11:27:13.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.121 7f58c65ba700 1 -- 172.21.2.101:0/2573615573 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c00e52e0 con 0x7f58c00f05f0 2021-02-10T11:27:13.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.121 7f58c65ba700 1 --2- 172.21.2.101:0/2573615573 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f58c00f05f0 0x7f58c00f20e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f58b8006f00 tx=0x7f58b8009e90).ready entity=mon.0 client_cookie=df51507829b261e4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:13.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.121 7f58c4db7700 1 -- 172.21.2.101:0/2573615573 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f58b800b300 con 0x7f58c00f05f0 2021-02-10T11:27:13.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.121 7f58c4db7700 1 -- 172.21.2.101:0/2573615573 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b800b580 con 0x7f58c00f05f0 2021-02-10T11:27:13.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.121 7f58c4db7700 1 -- 172.21.2.101:0/2573615573 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f58b800b9d0 con 0x7f58c00f05f0 2021-02-10T11:27:13.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.121 7f58c7f9d700 1 -- 172.21.2.101:0/2573615573 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f58c00f05f0 msgr2=0x7f58c00f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:13.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.121 7f58c7f9d700 1 --2- 172.21.2.101:0/2573615573 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f58c00f05f0 0x7f58c00f20e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f58b8006f00 tx=0x7f58b8009e90).stop 2021-02-10T11:27:13.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.122 7f58c7f9d700 1 -- 172.21.2.101:0/2573615573 shutdown_connections 2021-02-10T11:27:13.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.122 7f58c7f9d700 1 --2- 172.21.2.101:0/2573615573 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f58c00f05f0 0x7f58c00f20e0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:13.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.122 7f58c7f9d700 1 -- 172.21.2.101:0/2573615573 >> 172.21.2.101:0/2573615573 conn(0x7f58c00ce3e0 msgr2=0x7f58c00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:13.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.122 7f58c7f9d700 1 -- 172.21.2.101:0/2573615573 shutdown_connections 2021-02-10T11:27:13.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.122 7f58c7f9d700 1 -- 172.21.2.101:0/2573615573 wait complete. 2021-02-10T11:27:13.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.137 7f58c7f9d700 1 Processor -- start 2021-02-10T11:27:13.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.137 7f58c7f9d700 1 -- start start 2021-02-10T11:27:13.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.137 7f58c7f9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f58c00bdbf0 0x7f58c01a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:13.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.137 7f58c7f9d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f58c00bdff0 0x7f58c01a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:13.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.137 7f58c7f9d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f58c00abcc0 0x7f58c00ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:13.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.137 7f58c7f9d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c00be590 con 0x7f58c00bdbf0 2021-02-10T11:27:13.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.137 7f58c7f9d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c01a8090 con 0x7f58c00abcc0 2021-02-10T11:27:13.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.137 7f58c7f9d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c01a82b0 con 0x7f58c00bdff0 2021-02-10T11:27:13.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.138 7f58c5db9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f58c00bdbf0 0x7f58c01a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:13.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.138 7f58c5db9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f58c00bdbf0 0x7f58c01a7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47920/0 (socket says 172.21.2.101:47920) 2021-02-10T11:27:13.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.138 7f58c5db9700 1 -- 172.21.2.101:0/4128778814 learned_addr learned my addr 172.21.2.101:0/4128778814 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:27:13.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.138 7f58c65ba700 1 --2- 172.21.2.101:0/4128778814 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f58c00abcc0 0x7f58c00ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:13.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.138 7f58c55b8700 1 --2- 172.21.2.101:0/4128778814 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f58c00bdff0 0x7f58c01a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:13.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.138 7f58c5db9700 1 -- 172.21.2.101:0/4128778814 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f58c00bdff0 msgr2=0x7f58c01a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:13.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.138 7f58c5db9700 1 --2- 172.21.2.101:0/4128778814 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f58c00bdff0 0x7f58c01a7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:13.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.138 7f58c5db9700 1 -- 172.21.2.101:0/4128778814 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f58c00abcc0 msgr2=0x7f58c00ac0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:13.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.138 7f58c5db9700 1 --2- 172.21.2.101:0/4128778814 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f58c00abcc0 0x7f58c00ac0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:13.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.138 7f58c5db9700 1 -- 172.21.2.101:0/4128778814 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c00ac750 con 0x7f58c00bdbf0 2021-02-10T11:27:13.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.138 7f58c5db9700 1 --2- 172.21.2.101:0/4128778814 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f58c00bdbf0 0x7f58c01a7450 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f58bc006ed0 tx=0x7f58bc00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:13.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.138 7f58b27fc700 1 -- 172.21.2.101:0/4128778814 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f58bc006240 con 0x7f58c00bdbf0 2021-02-10T11:27:13.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.138 7f58b27fc700 1 -- 172.21.2.101:0/4128778814 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58bc0054b0 con 0x7f58c00bdbf0 2021-02-10T11:27:13.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.138 7f58b27fc700 1 -- 172.21.2.101:0/4128778814 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f58bc005910 con 0x7f58c00bdbf0 2021-02-10T11:27:13.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.138 7f58c7f9d700 1 -- 172.21.2.101:0/4128778814 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c00ac750 con 0x7f58c00bdbf0 2021-02-10T11:27:13.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.139 7f58c7f9d700 1 -- 172.21.2.101:0/4128778814 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c00ac750 con 0x7f58c00bdbf0 2021-02-10T11:27:13.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.139 7f58b27fc700 1 -- 172.21.2.101:0/4128778814 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f58bc006030 con 0x7f58c00bdbf0 2021-02-10T11:27:13.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.139 7f58b27fc700 1 --2- 172.21.2.101:0/4128778814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f58a8018730 0x7f58a801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:13.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.139 7f58b27fc700 1 -- 172.21.2.101:0/4128778814 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f58bc014140 con 0x7f58c00bdbf0 2021-02-10T11:27:13.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.139 7f58c55b8700 1 --2- 172.21.2.101:0/4128778814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f58a8018730 0x7f58a801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:13.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.140 7f58c55b8700 1 --2- 172.21.2.101:0/4128778814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f58a8018730 0x7f58a801ac10 crc :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:13.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.170 7f58c7f9d700 1 -- 172.21.2.101:0/4128778814 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c01a8090 con 0x7f58c00bdbf0 2021-02-10T11:27:13.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.173 7f58b27fc700 1 -- 172.21.2.101:0/4128778814 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f58bc006030 con 0x7f58c00bdbf0 2021-02-10T11:27:13.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.303 7f58c7f9d700 1 -- 172.21.2.101:0/4128778814 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839320", "target": ["mgr", ""]}) v1 -- 0x7f58c01a8090 con 0x7f58a8018730 2021-02-10T11:27:13.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.306 7f58b27fc700 1 -- 172.21.2.101:0/4128778814 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f58b4000970 con 0x7f58a8018730 2021-02-10T11:27:13.316 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839320/c89e69d9-b69c-410b-86da-7314ef6bc233 2021-02-10T11:27:13.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.308 7f58c7f9d700 1 -- 172.21.2.101:0/4128778814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f58a8018730 msgr2=0x7f58a801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:13.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.308 7f58c7f9d700 1 --2- 172.21.2.101:0/4128778814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f58a8018730 0x7f58a801ac10 crc :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:13.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.308 7f58c7f9d700 1 -- 172.21.2.101:0/4128778814 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f58c00bdbf0 msgr2=0x7f58c01a7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:13.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.308 7f58c7f9d700 1 --2- 172.21.2.101:0/4128778814 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f58c00bdbf0 0x7f58c01a7450 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f58bc006ed0 tx=0x7f58bc00b960).stop 2021-02-10T11:27:13.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.308 7f58c7f9d700 1 -- 172.21.2.101:0/4128778814 shutdown_connections 2021-02-10T11:27:13.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.308 7f58c7f9d700 1 --2- 172.21.2.101:0/4128778814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f58a8018730 0x7f58a801ac10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:13.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.308 7f58c7f9d700 1 --2- 172.21.2.101:0/4128778814 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f58c00bdff0 0x7f58c01a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:13.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.308 7f58c7f9d700 1 --2- 172.21.2.101:0/4128778814 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f58c00abcc0 0x7f58c00ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:13.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.308 7f58c7f9d700 1 --2- 172.21.2.101:0/4128778814 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f58c00bdbf0 0x7f58c01a7450 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:13.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.308 7f58c7f9d700 1 -- 172.21.2.101:0/4128778814 >> 172.21.2.101:0/4128778814 conn(0x7f58c0095a70 msgr2=0x7f58c00bf040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:13.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.308 7f58c7f9d700 1 -- 172.21.2.101:0/4128778814 shutdown_connections 2021-02-10T11:27:13.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:13.308 7f58c7f9d700 1 -- 172.21.2.101:0/4128778814 wait complete. 2021-02-10T11:27:13.326 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839320 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839320/c89e69d9-b69c-410b-86da-7314ef6bc233 2021-02-10T11:27:13.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:13.326 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:13.326 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:13.661 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:13.662 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:13.662 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839320/c89e69d9-b69c-410b-86da-7314ef6bc233/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:14.038 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:14.038 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:14.038 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0218881 s, 47.9 MB/s 2021-02-10T11:27:14.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:14.039 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:14.039 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:14.374 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:14.375 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:14.375 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839320/c89e69d9-b69c-410b-86da-7314ef6bc233/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:14.751 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:14.751 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:14.751 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.02368 s, 44.3 MB/s 2021-02-10T11:27:14.752 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:14.752 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:14.752 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:15.088 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:15.088 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:15.088 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839320/c89e69d9-b69c-410b-86da-7314ef6bc233/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:15.464 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:15.464 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:15.464 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246791 s, 42.5 MB/s 2021-02-10T11:27:15.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:15.465 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:15.465 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:15.801 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:15.801 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:15.801 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839320/c89e69d9-b69c-410b-86da-7314ef6bc233/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:16.178 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:16.178 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:16.178 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252531 s, 41.5 MB/s 2021-02-10T11:27:16.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:16.180 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:16.180 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:16.542 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:16.542 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:16.542 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839320/c89e69d9-b69c-410b-86da-7314ef6bc233/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:16.920 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:16.920 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:16.920 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.025786 s, 40.7 MB/s 2021-02-10T11:27:16.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:16.921 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:16.922 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:17.258 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:17.258 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:17.258 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839320/c89e69d9-b69c-410b-86da-7314ef6bc233/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:17.631 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:17.632 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:17.632 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0224045 s, 46.8 MB/s 2021-02-10T11:27:17.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:17.633 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:17.633 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:17.968 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:17.968 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:17.968 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839320/c89e69d9-b69c-410b-86da-7314ef6bc233/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:18.344 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:18.344 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:18.344 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247951 s, 42.3 MB/s 2021-02-10T11:27:18.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:18.345 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:18.346 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:18.681 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:18.681 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:18.681 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839320/c89e69d9-b69c-410b-86da-7314ef6bc233/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:19.053 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:19.054 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:19.054 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235864 s, 44.5 MB/s 2021-02-10T11:27:19.055 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:19.056 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:19.056 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:19.391 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:19.392 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:19.392 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839320/c89e69d9-b69c-410b-86da-7314ef6bc233/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:19.768 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:19.768 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:19.768 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024082 s, 43.5 MB/s 2021-02-10T11:27:19.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:19.769 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:19.769 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:20.106 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:20.106 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:20.106 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839320/c89e69d9-b69c-410b-86da-7314ef6bc233/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:20.482 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:20.482 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:20.482 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243986 s, 43.0 MB/s 2021-02-10T11:27:20.483 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839321 2021-02-10T11:27:20.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.888 7ff42660e700 1 Processor -- start 2021-02-10T11:27:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.889 7ff42660e700 1 -- start start 2021-02-10T11:27:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.889 7ff42660e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff4201b0870 0x7ff4200f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.889 7ff42660e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4200efe00 0x7ff4200f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.889 7ff42660e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4201b0c70 con 0x7ff4201b0470 2021-02-10T11:27:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.889 7ff42660e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4200f0b50 con 0x7ff4201b0870 2021-02-10T11:27:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.889 7ff42660e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4200f0820 con 0x7ff4200efe00 2021-02-10T11:27:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.889 7ff41f7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff4201b0870 0x7ff4200f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.889 7ff41ffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff4201b0470 0x7ff4200ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48420/0 (socket says 172.21.2.101:48420) 2021-02-10T11:27:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.889 7ff41ffff700 1 -- 172.21.2.101:0/1925572717 learned_addr learned my addr 172.21.2.101:0/1925572717 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:27:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.889 7ff424c2b700 1 --2- 172.21.2.101:0/1925572717 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4200efe00 0x7ff4200f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.890 7ff41f7fe700 1 -- 172.21.2.101:0/1925572717 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4200efe00 msgr2=0x7ff4200f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:27:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.890 7ff41f7fe700 1 --2- 172.21.2.101:0/1925572717 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4200efe00 0x7ff4200f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.890 7ff41f7fe700 1 -- 172.21.2.101:0/1925572717 >> v1:172.21.2.101:6789/0 conn(0x7ff4201b0470 legacy=0x7ff4200ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:20.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.890 7ff41f7fe700 1 -- 172.21.2.101:0/1925572717 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4200e52e0 con 0x7ff4201b0870 2021-02-10T11:27:20.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.890 7ff41f7fe700 1 --2- 172.21.2.101:0/1925572717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4201b0870 0x7ff4200f20e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7ff414006650 tx=0x7ff414009a80).ready entity=mon.0 client_cookie=b0d80230a792361b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:20.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.890 7ff41effd700 1 -- 172.21.2.101:0/1925572717 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff41400c0c0 con 0x7ff4201b0870 2021-02-10T11:27:20.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.890 7ff41effd700 1 -- 172.21.2.101:0/1925572717 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41400c340 con 0x7ff4201b0870 2021-02-10T11:27:20.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.890 7ff41effd700 1 -- 172.21.2.101:0/1925572717 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff41400c7a0 con 0x7ff4201b0870 2021-02-10T11:27:20.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.890 7ff42660e700 1 -- 172.21.2.101:0/1925572717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4201b0870 msgr2=0x7ff4200f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:20.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.890 7ff42660e700 1 --2- 172.21.2.101:0/1925572717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4201b0870 0x7ff4200f20e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7ff414006650 tx=0x7ff414009a80).stop 2021-02-10T11:27:20.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.891 7ff42660e700 1 -- 172.21.2.101:0/1925572717 shutdown_connections 2021-02-10T11:27:20.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.891 7ff42660e700 1 --2- 172.21.2.101:0/1925572717 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4200efe00 0x7ff4200f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:20.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.891 7ff42660e700 1 --2- 172.21.2.101:0/1925572717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4201b0870 0x7ff4200f20e0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:20.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.891 7ff42660e700 1 -- 172.21.2.101:0/1925572717 >> 172.21.2.101:0/1925572717 conn(0x7ff4200ce3e0 msgr2=0x7ff4200e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:20.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.891 7ff42660e700 1 -- 172.21.2.101:0/1925572717 shutdown_connections 2021-02-10T11:27:20.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.891 7ff42660e700 1 -- 172.21.2.101:0/1925572717 wait complete. 2021-02-10T11:27:20.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.906 7ff42660e700 1 Processor -- start 2021-02-10T11:27:20.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.906 7ff42660e700 1 -- start start 2021-02-10T11:27:20.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.906 7ff42660e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff420190920 0x7ff4200b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:20.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.906 7ff42660e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4200b8200 0x7ff4200b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:20.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.906 7ff42660e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4200c1d80 0x7ff4200c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:20.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.906 7ff42660e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff420190da0 con 0x7ff4200c1d80 2021-02-10T11:27:20.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.906 7ff42660e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4200b8c20 con 0x7ff420190920 2021-02-10T11:27:20.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.906 7ff42660e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4200b8e10 con 0x7ff4200b8200 2021-02-10T11:27:20.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.906 7ff424c2b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4200c1d80 0x7ff4200c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:20.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.906 7ff424c2b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4200c1d80 0x7ff4200c2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47936/0 (socket says 172.21.2.101:47936) 2021-02-10T11:27:20.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.906 7ff424c2b700 1 -- 172.21.2.101:0/4234697255 learned_addr learned my addr 172.21.2.101:0/4234697255 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:27:20.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.907 7ff424c2b700 1 -- 172.21.2.101:0/4234697255 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4200b8200 msgr2=0x7ff4200b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:20.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.907 7ff424c2b700 1 --2- 172.21.2.101:0/4234697255 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4200b8200 0x7ff4200b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:20.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.907 7ff424c2b700 1 -- 172.21.2.101:0/4234697255 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff420190920 msgr2=0x7ff4200b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:20.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.907 7ff424c2b700 1 --2- 172.21.2.101:0/4234697255 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff420190920 0x7ff4200b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:20.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.907 7ff424c2b700 1 -- 172.21.2.101:0/4234697255 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4200c27d0 con 0x7ff4200c1d80 2021-02-10T11:27:20.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.907 7ff424c2b700 1 --2- 172.21.2.101:0/4234697255 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4200c1d80 0x7ff4200c2180 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7ff41800b970 tx=0x7ff41800bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:20.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.907 7ff41cff9700 1 -- 172.21.2.101:0/4234697255 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff418000f90 con 0x7ff4200c1d80 2021-02-10T11:27:20.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.907 7ff41cff9700 1 -- 172.21.2.101:0/4234697255 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff418006aa0 con 0x7ff4200c1d80 2021-02-10T11:27:20.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.907 7ff41cff9700 1 -- 172.21.2.101:0/4234697255 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff418001680 con 0x7ff4200c1d80 2021-02-10T11:27:20.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.907 7ff42660e700 1 -- 172.21.2.101:0/4234697255 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4200c27d0 con 0x7ff4200c1d80 2021-02-10T11:27:20.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.907 7ff42660e700 1 -- 172.21.2.101:0/4234697255 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4200c2d60 con 0x7ff4200c1d80 2021-02-10T11:27:20.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.908 7ff41cff9700 1 -- 172.21.2.101:0/4234697255 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff4180011f0 con 0x7ff4200c1d80 2021-02-10T11:27:20.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.908 7ff41cff9700 1 --2- 172.21.2.101:0/4234697255 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4000186d0 0x7ff40001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:20.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.908 7ff41cff9700 1 -- 172.21.2.101:0/4234697255 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff418032b40 con 0x7ff4200c1d80 2021-02-10T11:27:20.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.908 7ff41ffff700 1 --2- 172.21.2.101:0/4234697255 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4000186d0 0x7ff40001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:20.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.908 7ff41ffff700 1 --2- 172.21.2.101:0/4234697255 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4000186d0 0x7ff40001abd0 crc :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:20.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.940 7ff42660e700 1 -- 172.21.2.101:0/4234697255 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff420105ae0 con 0x7ff4200c1d80 2021-02-10T11:27:20.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:20.941 7ff41cff9700 1 -- 172.21.2.101:0/4234697255 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff418006aa0 con 0x7ff4200c1d80 2021-02-10T11:27:21.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.066 7ff42660e700 1 -- 172.21.2.101:0/4234697255 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839321", "target": ["mgr", ""]}) v1 -- 0x7ff4200c2d60 con 0x7ff4000186d0 2021-02-10T11:27:21.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.092 7ff41cff9700 1 -- 172.21.2.101:0/4234697255 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7ff410000e90 con 0x7ff4000186d0 2021-02-10T11:27:21.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.099 7ff42660e700 1 -- 172.21.2.101:0/4234697255 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4000186d0 msgr2=0x7ff40001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:21.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.099 7ff42660e700 1 --2- 172.21.2.101:0/4234697255 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4000186d0 0x7ff40001abd0 crc :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:21.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.099 7ff42660e700 1 -- 172.21.2.101:0/4234697255 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4200c1d80 msgr2=0x7ff4200c2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:21.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.099 7ff42660e700 1 --2- 172.21.2.101:0/4234697255 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4200c1d80 0x7ff4200c2180 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7ff41800b970 tx=0x7ff41800bd10).stop 2021-02-10T11:27:21.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.099 7ff42660e700 1 -- 172.21.2.101:0/4234697255 shutdown_connections 2021-02-10T11:27:21.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.099 7ff42660e700 1 --2- 172.21.2.101:0/4234697255 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4000186d0 0x7ff40001abd0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:21.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.099 7ff42660e700 1 --2- 172.21.2.101:0/4234697255 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4200b8200 0x7ff4200b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:21.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.099 7ff42660e700 1 --2- 172.21.2.101:0/4234697255 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4200c1d80 0x7ff4200c2180 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:21.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.099 7ff42660e700 1 --2- 172.21.2.101:0/4234697255 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff420190920 0x7ff4200b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:21.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.099 7ff42660e700 1 -- 172.21.2.101:0/4234697255 >> 172.21.2.101:0/4234697255 conn(0x7ff420094e10 msgr2=0x7ff4200a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:21.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.099 7ff42660e700 1 -- 172.21.2.101:0/4234697255 shutdown_connections 2021-02-10T11:27:21.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.100 7ff42660e700 1 -- 172.21.2.101:0/4234697255 wait complete. 2021-02-10T11:27:21.118 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839321 2021-02-10T11:27:21.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.524 7fa4b4114700 1 Processor -- start 2021-02-10T11:27:21.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.524 7fa4b4114700 1 -- start start 2021-02-10T11:27:21.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.525 7fa4b4114700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa4ac1b0470 0x7fa4ac1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:21.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.525 7fa4b4114700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa4ac0f20e0 0x7fa4ac0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:21.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.525 7fa4b4114700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4ac0f0b50 con 0x7fa4ac0ef610 2021-02-10T11:27:21.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.525 7fa4b4114700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4ac0f0820 con 0x7fa4ac1b0470 2021-02-10T11:27:21.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.525 7fa4b4114700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4ac0f24e0 con 0x7fa4ac0f20e0 2021-02-10T11:27:21.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.525 7fa4b1f30700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa4ac1b0470 0x7fa4ac1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:21.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.525 7fa4b172f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa4ac0ef610 0x7fa4ac0efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48434/0 (socket says 172.21.2.101:48434) 2021-02-10T11:27:21.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.525 7fa4b172f700 1 -- 172.21.2.101:0/3832583749 learned_addr learned my addr 172.21.2.101:0/3832583749 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:27:21.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.525 7fa4b1f30700 1 -- 172.21.2.101:0/3832583749 >> v2:172.21.2.108:3300/0 conn(0x7fa4ac0f20e0 msgr2=0x7fa4ac0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:27:21.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.525 7fa4b1f30700 1 --2- 172.21.2.101:0/3832583749 >> v2:172.21.2.108:3300/0 conn(0x7fa4ac0f20e0 0x7fa4ac0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:21.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.525 7fa4b1f30700 1 -- 172.21.2.101:0/3832583749 >> v1:172.21.2.101:6789/0 conn(0x7fa4ac0ef610 legacy=0x7fa4ac0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:21.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.525 7fa4b1f30700 1 -- 172.21.2.101:0/3832583749 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ac0e52e0 con 0x7fa4ac1b0470 2021-02-10T11:27:21.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.525 7fa4b1f30700 1 --2- 172.21.2.101:0/3832583749 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4ac1b0470 0x7fa4ac1b0870 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7fa4a0006650 tx=0x7fa4a0009a80).ready entity=mon.0 client_cookie=96fabf7ef47bbe1f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:21.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.526 7fa4b0f2e700 1 -- 172.21.2.101:0/3832583749 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa4a00009b0 con 0x7fa4ac1b0470 2021-02-10T11:27:21.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.526 7fa4b0f2e700 1 -- 172.21.2.101:0/3832583749 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a000c0c0 con 0x7fa4ac1b0470 2021-02-10T11:27:21.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.526 7fa4b0f2e700 1 -- 172.21.2.101:0/3832583749 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa4a000c5e0 con 0x7fa4ac1b0470 2021-02-10T11:27:21.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.526 7fa4b4114700 1 -- 172.21.2.101:0/3832583749 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4ac1b0470 msgr2=0x7fa4ac1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:21.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.526 7fa4b4114700 1 --2- 172.21.2.101:0/3832583749 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4ac1b0470 0x7fa4ac1b0870 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7fa4a0006650 tx=0x7fa4a0009a80).stop 2021-02-10T11:27:21.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.526 7fa4b4114700 1 -- 172.21.2.101:0/3832583749 shutdown_connections 2021-02-10T11:27:21.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.526 7fa4b4114700 1 --2- 172.21.2.101:0/3832583749 >> v2:172.21.2.108:3300/0 conn(0x7fa4ac0f20e0 0x7fa4ac0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:21.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.526 7fa4b4114700 1 --2- 172.21.2.101:0/3832583749 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4ac1b0470 0x7fa4ac1b0870 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:21.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.526 7fa4b4114700 1 -- 172.21.2.101:0/3832583749 >> 172.21.2.101:0/3832583749 conn(0x7fa4ac0ce3e0 msgr2=0x7fa4ac0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:21.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.527 7fa4b4114700 1 -- 172.21.2.101:0/3832583749 shutdown_connections 2021-02-10T11:27:21.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.527 7fa4b4114700 1 -- 172.21.2.101:0/3832583749 wait complete. 2021-02-10T11:27:21.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.541 7fa4b4114700 1 Processor -- start 2021-02-10T11:27:21.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.541 7fa4b4114700 1 -- start start 2021-02-10T11:27:21.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.541 7fa4b4114700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4ac0c36a0 0x7fa4ac0c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:21.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.541 7fa4b4114700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4ac0c3190 0x7fa4ac0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:21.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.541 7fa4b4114700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa4ac0b2370 0x7fa4ac0b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:21.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.541 7fa4b4114700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac0b27c0 con 0x7fa4ac0c36a0 2021-02-10T11:27:21.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.541 7fa4b4114700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac0b3360 con 0x7fa4ac0b2370 2021-02-10T11:27:21.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.541 7fa4b4114700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac0b3540 con 0x7fa4ac0c3190 2021-02-10T11:27:21.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.541 7fa4b1f30700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4ac0c36a0 0x7fa4ac0c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:21.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.541 7fa4b1f30700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4ac0c36a0 0x7fa4ac0c2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47944/0 (socket says 172.21.2.101:47944) 2021-02-10T11:27:21.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.542 7fa4b1f30700 1 -- 172.21.2.101:0/934862963 learned_addr learned my addr 172.21.2.101:0/934862963 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:27:21.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.542 7fa4b2731700 1 --2- 172.21.2.101:0/934862963 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa4ac0b2370 0x7fa4ac0b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:21.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.542 7fa4b1f30700 1 -- 172.21.2.101:0/934862963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4ac0c3190 msgr2=0x7fa4ac0b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:21.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.542 7fa4b172f700 1 --2- 172.21.2.101:0/934862963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4ac0c3190 0x7fa4ac0b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:21.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.542 7fa4b1f30700 1 --2- 172.21.2.101:0/934862963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4ac0c3190 0x7fa4ac0b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:21.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.542 7fa4b1f30700 1 -- 172.21.2.101:0/934862963 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa4ac0b2370 msgr2=0x7fa4ac0b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:21.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.542 7fa4b1f30700 1 --2- 172.21.2.101:0/934862963 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa4ac0b2370 0x7fa4ac0b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:21.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.542 7fa4b1f30700 1 -- 172.21.2.101:0/934862963 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ac0b3760 con 0x7fa4ac0c36a0 2021-02-10T11:27:21.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.542 7fa4b2731700 1 --2- 172.21.2.101:0/934862963 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa4ac0b2370 0x7fa4ac0b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:27:21.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.542 7fa4b172f700 1 --2- 172.21.2.101:0/934862963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4ac0c3190 0x7fa4ac0b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:27:21.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.542 7fa4b1f30700 1 --2- 172.21.2.101:0/934862963 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4ac0c36a0 0x7fa4ac0c2b70 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7fa4a0009630 tx=0x7fa4a000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:21.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.543 7fa49e7fc700 1 -- 172.21.2.101:0/934862963 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa4a00093a0 con 0x7fa4ac0c36a0 2021-02-10T11:27:21.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.543 7fa49e7fc700 1 -- 172.21.2.101:0/934862963 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a000bd70 con 0x7fa4ac0c36a0 2021-02-10T11:27:21.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.543 7fa49e7fc700 1 -- 172.21.2.101:0/934862963 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa4a00054d0 con 0x7fa4ac0c36a0 2021-02-10T11:27:21.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.543 7fa4b4114700 1 -- 172.21.2.101:0/934862963 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4ac0b3760 con 0x7fa4ac0c36a0 2021-02-10T11:27:21.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.543 7fa4b4114700 1 -- 172.21.2.101:0/934862963 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4ac0b3760 con 0x7fa4ac0c36a0 2021-02-10T11:27:21.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.543 7fa49e7fc700 1 -- 172.21.2.101:0/934862963 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa4a000a000 con 0x7fa4ac0c36a0 2021-02-10T11:27:21.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.543 7fa49e7fc700 1 --2- 172.21.2.101:0/934862963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa4940186c0 0x7fa49401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:21.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.543 7fa49e7fc700 1 -- 172.21.2.101:0/934862963 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa4a0032540 con 0x7fa4ac0c36a0 2021-02-10T11:27:21.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.544 7fa4b172f700 1 --2- 172.21.2.101:0/934862963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa4940186c0 0x7fa49401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:21.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.544 7fa4b172f700 1 --2- 172.21.2.101:0/934862963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa4940186c0 0x7fa49401abc0 crc :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:21.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.575 7fa4b4114700 1 -- 172.21.2.101:0/934862963 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4ac0edd10 con 0x7fa4ac0c36a0 2021-02-10T11:27:21.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.577 7fa49e7fc700 1 -- 172.21.2.101:0/934862963 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa4a000a000 con 0x7fa4ac0c36a0 2021-02-10T11:27:21.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.704 7fa4b4114700 1 -- 172.21.2.101:0/934862963 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839321", "target": ["mgr", ""]}) v1 -- 0x7fa4ac0edd10 con 0x7fa4940186c0 2021-02-10T11:27:21.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.708 7fa49e7fc700 1 -- 172.21.2.101:0/934862963 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fa4a8000d80 con 0x7fa4940186c0 2021-02-10T11:27:21.722 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839321/a7a33f2e-1190-4978-a157-75b431658d26 2021-02-10T11:27:21.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.714 7fa4b4114700 1 -- 172.21.2.101:0/934862963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa4940186c0 msgr2=0x7fa49401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:21.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.714 7fa4b4114700 1 --2- 172.21.2.101:0/934862963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa4940186c0 0x7fa49401abc0 crc :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:21.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.714 7fa4b4114700 1 -- 172.21.2.101:0/934862963 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4ac0c36a0 msgr2=0x7fa4ac0c2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:21.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.714 7fa4b4114700 1 --2- 172.21.2.101:0/934862963 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4ac0c36a0 0x7fa4ac0c2b70 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7fa4a0009630 tx=0x7fa4a000b980).stop 2021-02-10T11:27:21.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.714 7fa4b4114700 1 -- 172.21.2.101:0/934862963 shutdown_connections 2021-02-10T11:27:21.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.714 7fa4b4114700 1 --2- 172.21.2.101:0/934862963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa4940186c0 0x7fa49401abc0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:21.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.714 7fa4b4114700 1 --2- 172.21.2.101:0/934862963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4ac0c3190 0x7fa4ac0b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:21.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.714 7fa4b4114700 1 --2- 172.21.2.101:0/934862963 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa4ac0b2370 0x7fa4ac0b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:21.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.714 7fa4b4114700 1 --2- 172.21.2.101:0/934862963 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4ac0c36a0 0x7fa4ac0c2b70 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:21.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.714 7fa4b4114700 1 -- 172.21.2.101:0/934862963 >> 172.21.2.101:0/934862963 conn(0x7fa4ac09f630 msgr2=0x7fa4ac0a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:21.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.714 7fa4b4114700 1 -- 172.21.2.101:0/934862963 shutdown_connections 2021-02-10T11:27:21.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:21.714 7fa4b4114700 1 -- 172.21.2.101:0/934862963 wait complete. 2021-02-10T11:27:21.732 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839321 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839321/a7a33f2e-1190-4978-a157-75b431658d26 2021-02-10T11:27:21.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:21.732 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:21.732 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:22.069 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:22.069 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:22.069 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839321/a7a33f2e-1190-4978-a157-75b431658d26/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:22.453 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:22.453 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:22.453 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251872 s, 41.6 MB/s 2021-02-10T11:27:22.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:22.454 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:22.455 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:22.791 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:22.791 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:22.792 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839321/a7a33f2e-1190-4978-a157-75b431658d26/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:23.169 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:23.170 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:23.170 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0254261 s, 41.2 MB/s 2021-02-10T11:27:23.171 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:23.171 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:23.171 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:23.507 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:23.507 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:23.507 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839321/a7a33f2e-1190-4978-a157-75b431658d26/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:23.741 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:27:23.744 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:27:23.881 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:23.881 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:23.881 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.025675 s, 40.8 MB/s 2021-02-10T11:27:23.882 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:23.882 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:23.882 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:24.218 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:24.219 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:24.219 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839321/a7a33f2e-1190-4978-a157-75b431658d26/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:24.594 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:24.595 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:24.595 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231793 s, 45.2 MB/s 2021-02-10T11:27:24.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:24.596 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:24.596 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:24.932 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:24.932 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:24.932 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839321/a7a33f2e-1190-4978-a157-75b431658d26/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:25.307 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:25.308 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:25.308 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0228592 s, 45.9 MB/s 2021-02-10T11:27:25.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:25.309 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:25.309 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:25.644 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:25.644 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:25.644 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839321/a7a33f2e-1190-4978-a157-75b431658d26/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:26.018 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:26.018 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:26.018 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0224866 s, 46.6 MB/s 2021-02-10T11:27:26.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:26.019 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:26.019 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:26.354 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:26.354 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:26.355 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839321/a7a33f2e-1190-4978-a157-75b431658d26/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:26.731 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:26.731 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:26.731 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240453 s, 43.6 MB/s 2021-02-10T11:27:26.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:26.732 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:26.732 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:27.094 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:27.095 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:27.095 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839321/a7a33f2e-1190-4978-a157-75b431658d26/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:27.471 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:27.471 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:27.471 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023983 s, 43.7 MB/s 2021-02-10T11:27:27.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:27.472 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:27.472 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:27.809 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:27.809 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:27.809 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839321/a7a33f2e-1190-4978-a157-75b431658d26/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:28.185 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:28.185 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:28.185 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0226683 s, 46.3 MB/s 2021-02-10T11:27:28.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:28.186 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:28.186 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:28.522 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:28.522 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:28.522 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839321/a7a33f2e-1190-4978-a157-75b431658d26/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:28.895 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:28.895 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:28.895 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0250227 s, 41.9 MB/s 2021-02-10T11:27:28.896 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839322 2021-02-10T11:27:29.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.302 7f81e3d6c700 1 Processor -- start 2021-02-10T11:27:29.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.302 7f81e3d6c700 1 -- start start 2021-02-10T11:27:29.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.303 7f81e3d6c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f81dc1b0470 0x7f81dc1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:29.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.303 7f81e3d6c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81dc0f0b50 con 0x7f81dc0ef610 2021-02-10T11:27:29.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.303 7f81e3d6c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81dc0f24e0 con 0x7f81dc0f20e0 2021-02-10T11:27:29.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.303 7f81e3d6c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81dc0e5ea0 con 0x7f81dc1b0470 2021-02-10T11:27:29.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.303 7f81e1387700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f81dc0ef610 0x7f81dc0efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48444/0 (socket says 172.21.2.101:48444) 2021-02-10T11:27:29.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.303 7f81e1387700 1 -- 172.21.2.101:0/4221992955 learned_addr learned my addr 172.21.2.101:0/4221992955 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:27:29.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.303 7f81e1b88700 1 --2- 172.21.2.101:0/4221992955 >> v2:172.21.2.108:3300/0 conn(0x7f81dc1b0470 0x7f81dc1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:29.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.303 7f81e0b86700 1 -- 172.21.2.101:0/4221992955 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2477962878 0 0) 0x7f81d00009b0 con 0x7f81dc0ef610 2021-02-10T11:27:29.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.304 7f81e0b86700 1 -- 172.21.2.101:0/4221992955 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81c8002fe0 con 0x7f81dc0ef610 2021-02-10T11:27:29.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.304 7f81e1b88700 1 -- 172.21.2.101:0/4221992955 >> v1:172.21.2.108:6789/0 conn(0x7f81dc0f20e0 legacy=0x7f81dc0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:29.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.304 7f81e1b88700 1 -- 172.21.2.101:0/4221992955 >> v1:172.21.2.101:6789/0 conn(0x7f81dc0ef610 legacy=0x7f81dc0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:29.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.304 7f81e1b88700 1 -- 172.21.2.101:0/4221992955 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81dc0e52e0 con 0x7f81dc1b0470 2021-02-10T11:27:29.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.304 7f81e1b88700 1 --2- 172.21.2.101:0/4221992955 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81dc1b0470 0x7f81dc1b0870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f81d8006650 tx=0x7f81d8009a80).ready entity=mon.1 client_cookie=b2a2ddeffaa8bbcd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:29.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.304 7f81e0b86700 1 -- 172.21.2.101:0/4221992955 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81d80009b0 con 0x7f81dc1b0470 2021-02-10T11:27:29.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.304 7f81e0b86700 1 -- 172.21.2.101:0/4221992955 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d800c0c0 con 0x7f81dc1b0470 2021-02-10T11:27:29.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.304 7f81e0b86700 1 -- 172.21.2.101:0/4221992955 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81d800c5e0 con 0x7f81dc1b0470 2021-02-10T11:27:29.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.304 7f81e3d6c700 1 -- 172.21.2.101:0/4221992955 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81dc1b0470 msgr2=0x7f81dc1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:29.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.304 7f81e3d6c700 1 --2- 172.21.2.101:0/4221992955 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81dc1b0470 0x7f81dc1b0870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f81d8006650 tx=0x7f81d8009a80).stop 2021-02-10T11:27:29.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.305 7f81e3d6c700 1 -- 172.21.2.101:0/4221992955 shutdown_connections 2021-02-10T11:27:29.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.305 7f81e3d6c700 1 --2- 172.21.2.101:0/4221992955 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81dc1b0470 0x7f81dc1b0870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:29.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.305 7f81e3d6c700 1 -- 172.21.2.101:0/4221992955 >> 172.21.2.101:0/4221992955 conn(0x7f81dc0ce3e0 msgr2=0x7f81dc0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:29.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.305 7f81e3d6c700 1 -- 172.21.2.101:0/4221992955 shutdown_connections 2021-02-10T11:27:29.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.305 7f81e3d6c700 1 -- 172.21.2.101:0/4221992955 wait complete. 2021-02-10T11:27:29.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.320 7f81e3d6c700 1 Processor -- start 2021-02-10T11:27:29.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.320 7f81e3d6c700 1 -- start start 2021-02-10T11:27:29.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.320 7f81e3d6c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81dc0c36a0 0x7f81dc0c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:29.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.320 7f81e3d6c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81dc0c3190 0x7f81dc0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:29.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.320 7f81e3d6c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81dc0b2370 0x7f81dc0b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:29.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.320 7f81e3d6c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc0b27c0 con 0x7f81dc0c36a0 2021-02-10T11:27:29.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.320 7f81e3d6c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc0b3360 con 0x7f81dc0b2370 2021-02-10T11:27:29.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.320 7f81e3d6c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc0b3540 con 0x7f81dc0c3190 2021-02-10T11:27:29.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.320 7f81e1b88700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81dc0c36a0 0x7f81dc0c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:29.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.320 7f81e1b88700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81dc0c36a0 0x7f81dc0c2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47954/0 (socket says 172.21.2.101:47954) 2021-02-10T11:27:29.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.320 7f81e1b88700 1 -- 172.21.2.101:0/3819725226 learned_addr learned my addr 172.21.2.101:0/3819725226 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:27:29.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.321 7f81e1b88700 1 -- 172.21.2.101:0/3819725226 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81dc0c3190 msgr2=0x7f81dc0b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:29.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.321 7f81e1b88700 1 --2- 172.21.2.101:0/3819725226 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81dc0c3190 0x7f81dc0b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:29.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.321 7f81e1b88700 1 -- 172.21.2.101:0/3819725226 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81dc0b2370 msgr2=0x7f81dc0b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:29.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.321 7f81e1b88700 1 --2- 172.21.2.101:0/3819725226 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81dc0b2370 0x7f81dc0b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:29.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.321 7f81e1b88700 1 -- 172.21.2.101:0/3819725226 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81dc0b3760 con 0x7f81dc0c36a0 2021-02-10T11:27:29.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.321 7f81e1b88700 1 --2- 172.21.2.101:0/3819725226 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81dc0c36a0 0x7f81dc0c2b70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f81d80096a0 tx=0x7f81d800b9b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:29.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.321 7f81ce7fc700 1 -- 172.21.2.101:0/3819725226 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81d8000940 con 0x7f81dc0c36a0 2021-02-10T11:27:29.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.321 7f81ce7fc700 1 -- 172.21.2.101:0/3819725226 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d80093a0 con 0x7f81dc0c36a0 2021-02-10T11:27:29.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.321 7f81ce7fc700 1 -- 172.21.2.101:0/3819725226 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81d80022b0 con 0x7f81dc0c36a0 2021-02-10T11:27:29.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.321 7f81e3d6c700 1 -- 172.21.2.101:0/3819725226 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81dc0b3760 con 0x7f81dc0c36a0 2021-02-10T11:27:29.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.321 7f81e3d6c700 1 -- 172.21.2.101:0/3819725226 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81dc0b3760 con 0x7f81dc0c36a0 2021-02-10T11:27:29.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.322 7f81ce7fc700 1 -- 172.21.2.101:0/3819725226 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f81d80052e0 con 0x7f81dc0c36a0 2021-02-10T11:27:29.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.322 7f81ce7fc700 1 --2- 172.21.2.101:0/3819725226 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81c40186c0 0x7f81c401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:29.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.322 7f81ce7fc700 1 -- 172.21.2.101:0/3819725226 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f81d8032550 con 0x7f81dc0c36a0 2021-02-10T11:27:29.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.322 7f81e1387700 1 --2- 172.21.2.101:0/3819725226 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81c40186c0 0x7f81c401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:29.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.322 7f81e1387700 1 --2- 172.21.2.101:0/3819725226 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81c40186c0 0x7f81c401abc0 crc :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:29.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.353 7f81e3d6c700 1 -- 172.21.2.101:0/3819725226 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81dc0edd10 con 0x7f81dc0c36a0 2021-02-10T11:27:29.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.356 7f81ce7fc700 1 -- 172.21.2.101:0/3819725226 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f81d8000c90 con 0x7f81dc0c36a0 2021-02-10T11:27:29.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.481 7f81e3d6c700 1 -- 172.21.2.101:0/3819725226 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839322", "target": ["mgr", ""]}) v1 -- 0x7f81dc0edd10 con 0x7f81c40186c0 2021-02-10T11:27:29.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.507 7f81ce7fc700 1 -- 172.21.2.101:0/3819725226 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f81d0006e30 con 0x7f81c40186c0 2021-02-10T11:27:29.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.514 7f81e3d6c700 1 -- 172.21.2.101:0/3819725226 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81c40186c0 msgr2=0x7f81c401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:29.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.514 7f81e3d6c700 1 --2- 172.21.2.101:0/3819725226 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81c40186c0 0x7f81c401abc0 crc :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:29.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.514 7f81e3d6c700 1 -- 172.21.2.101:0/3819725226 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81dc0c36a0 msgr2=0x7f81dc0c2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:29.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.514 7f81e3d6c700 1 --2- 172.21.2.101:0/3819725226 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81dc0c36a0 0x7f81dc0c2b70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f81d80096a0 tx=0x7f81d800b9b0).stop 2021-02-10T11:27:29.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.514 7f81e3d6c700 1 -- 172.21.2.101:0/3819725226 shutdown_connections 2021-02-10T11:27:29.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.514 7f81e3d6c700 1 --2- 172.21.2.101:0/3819725226 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81c40186c0 0x7f81c401abc0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:29.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.514 7f81e3d6c700 1 --2- 172.21.2.101:0/3819725226 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81dc0c3190 0x7f81dc0b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:29.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.514 7f81e3d6c700 1 --2- 172.21.2.101:0/3819725226 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81dc0b2370 0x7f81dc0b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:29.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.514 7f81e3d6c700 1 --2- 172.21.2.101:0/3819725226 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81dc0c36a0 0x7f81dc0c2b70 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:29.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.514 7f81e3d6c700 1 -- 172.21.2.101:0/3819725226 >> 172.21.2.101:0/3819725226 conn(0x7f81dc09f630 msgr2=0x7f81dc0a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:29.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.515 7f81e3d6c700 1 -- 172.21.2.101:0/3819725226 shutdown_connections 2021-02-10T11:27:29.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.515 7f81e3d6c700 1 -- 172.21.2.101:0/3819725226 wait complete. 2021-02-10T11:27:29.533 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839322 2021-02-10T11:27:29.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.940 7fe4af97a700 1 Processor -- start 2021-02-10T11:27:29.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.940 7fe4af97a700 1 -- start start 2021-02-10T11:27:29.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.941 7fe4af97a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe4a81b0470 0x7fe4a81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:29.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.941 7fe4af97a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe4a80f20e0 0x7fe4a80f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:29.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.941 7fe4af97a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a80f0b50 con 0x7fe4a80ef610 2021-02-10T11:27:29.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.941 7fe4af97a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a80f0820 con 0x7fe4a81b0470 2021-02-10T11:27:29.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.941 7fe4af97a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a80f24e0 con 0x7fe4a80f20e0 2021-02-10T11:27:29.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.941 7fe4acf95700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe4a80ef610 0x7fe4a80efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48454/0 (socket says 172.21.2.101:48454) 2021-02-10T11:27:29.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.941 7fe4acf95700 1 -- 172.21.2.101:0/3464759959 learned_addr learned my addr 172.21.2.101:0/3464759959 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:27:29.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.941 7fe4ad796700 1 --2- 172.21.2.101:0/3464759959 >> v2:172.21.2.101:3300/0 conn(0x7fe4a81b0470 0x7fe4a81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:29.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.942 7fe49bfff700 1 -- 172.21.2.101:0/3464759959 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2585678770 0 0) 0x7fe49c0009b0 con 0x7fe4a80ef610 2021-02-10T11:27:29.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.942 7fe49bfff700 1 -- 172.21.2.101:0/3464759959 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe494002fe0 con 0x7fe4a80ef610 2021-02-10T11:27:29.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.942 7fe4adf97700 1 --2- 172.21.2.101:0/3464759959 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe4a80f20e0 0x7fe4a80f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:29.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.942 7fe4ad796700 1 -- 172.21.2.101:0/3464759959 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe4a80f20e0 msgr2=0x7fe4a80f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:27:29.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.942 7fe4ad796700 1 --2- 172.21.2.101:0/3464759959 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe4a80f20e0 0x7fe4a80f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:29.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.942 7fe4ad796700 1 -- 172.21.2.101:0/3464759959 >> v1:172.21.2.101:6789/0 conn(0x7fe4a80ef610 legacy=0x7fe4a80efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:29.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.942 7fe4ad796700 1 -- 172.21.2.101:0/3464759959 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a80e52e0 con 0x7fe4a81b0470 2021-02-10T11:27:29.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.942 7fe4adf97700 1 --2- 172.21.2.101:0/3464759959 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe4a80f20e0 0x7fe4a80f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:27:29.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.942 7fe4ad796700 1 --2- 172.21.2.101:0/3464759959 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe4a81b0470 0x7fe4a81b0870 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7fe4a4006650 tx=0x7fe4a4009a80).ready entity=mon.0 client_cookie=ac750bd3bee563b9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:29.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.942 7fe49bfff700 1 -- 172.21.2.101:0/3464759959 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe4a40009b0 con 0x7fe4a81b0470 2021-02-10T11:27:29.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.942 7fe49bfff700 1 -- 172.21.2.101:0/3464759959 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a400c0c0 con 0x7fe4a81b0470 2021-02-10T11:27:29.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.942 7fe49bfff700 1 -- 172.21.2.101:0/3464759959 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe4a400c5e0 con 0x7fe4a81b0470 2021-02-10T11:27:29.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.942 7fe4af97a700 1 -- 172.21.2.101:0/3464759959 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe4a81b0470 msgr2=0x7fe4a81b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:29.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.942 7fe4af97a700 1 --2- 172.21.2.101:0/3464759959 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe4a81b0470 0x7fe4a81b0870 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7fe4a4006650 tx=0x7fe4a4009a80).stop 2021-02-10T11:27:29.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.942 7fe4af97a700 1 -- 172.21.2.101:0/3464759959 shutdown_connections 2021-02-10T11:27:29.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.942 7fe4af97a700 1 --2- 172.21.2.101:0/3464759959 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe4a80f20e0 0x7fe4a80f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:29.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.942 7fe4af97a700 1 --2- 172.21.2.101:0/3464759959 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe4a81b0470 0x7fe4a81b0870 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:29.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.942 7fe4af97a700 1 -- 172.21.2.101:0/3464759959 >> 172.21.2.101:0/3464759959 conn(0x7fe4a80ce3e0 msgr2=0x7fe4a80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:29.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.943 7fe4af97a700 1 -- 172.21.2.101:0/3464759959 shutdown_connections 2021-02-10T11:27:29.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.943 7fe4af97a700 1 -- 172.21.2.101:0/3464759959 wait complete. 2021-02-10T11:27:29.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.957 7fe4af97a700 1 Processor -- start 2021-02-10T11:27:29.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.957 7fe4af97a700 1 -- start start 2021-02-10T11:27:29.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.958 7fe4af97a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe4a80a0420 0x7fe4a80a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:29.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.958 7fe4af97a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe4a8189d00 0x7fe4a818a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:29.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.958 7fe4af97a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe4a818a720 0x7fe4a80b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:29.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.958 7fe4af97a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a8189780 con 0x7fe4a818a720 2021-02-10T11:27:29.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.958 7fe4af97a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a818ab20 con 0x7fe4a80a0420 2021-02-10T11:27:29.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.958 7fe4af97a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a80a1140 con 0x7fe4a8189d00 2021-02-10T11:27:29.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.958 7fe4adf97700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe4a818a720 0x7fe4a80b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:29.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.958 7fe4adf97700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe4a818a720 0x7fe4a80b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47968/0 (socket says 172.21.2.101:47968) 2021-02-10T11:27:29.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.958 7fe4adf97700 1 -- 172.21.2.101:0/557323220 learned_addr learned my addr 172.21.2.101:0/557323220 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:27:29.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.958 7fe4adf97700 1 -- 172.21.2.101:0/557323220 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe4a8189d00 msgr2=0x7fe4a818a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:29.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.958 7fe4adf97700 1 --2- 172.21.2.101:0/557323220 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe4a8189d00 0x7fe4a818a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:29.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.958 7fe4adf97700 1 -- 172.21.2.101:0/557323220 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe4a80a0420 msgr2=0x7fe4a80a0820 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:27:29.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.958 7fe4adf97700 1 --2- 172.21.2.101:0/557323220 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe4a80a0420 0x7fe4a80a0820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:29.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.958 7fe4adf97700 1 -- 172.21.2.101:0/557323220 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a80b8200 con 0x7fe4a818a720 2021-02-10T11:27:29.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.958 7fe4adf97700 1 --2- 172.21.2.101:0/557323220 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe4a818a720 0x7fe4a80b7be0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fe4a0009980 tx=0x7fe4a000bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:29.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.959 7fe499ffb700 1 -- 172.21.2.101:0/557323220 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe4a000c2f0 con 0x7fe4a818a720 2021-02-10T11:27:29.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.959 7fe499ffb700 1 -- 172.21.2.101:0/557323220 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a0006790 con 0x7fe4a818a720 2021-02-10T11:27:29.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.959 7fe499ffb700 1 -- 172.21.2.101:0/557323220 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe4a000c760 con 0x7fe4a818a720 2021-02-10T11:27:29.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.959 7fe4af97a700 1 -- 172.21.2.101:0/557323220 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a80b8200 con 0x7fe4a818a720 2021-02-10T11:27:29.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.959 7fe4af97a700 1 -- 172.21.2.101:0/557323220 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a80b8790 con 0x7fe4a818a720 2021-02-10T11:27:29.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.959 7fe499ffb700 1 -- 172.21.2.101:0/557323220 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe4a00011f0 con 0x7fe4a818a720 2021-02-10T11:27:29.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.959 7fe499ffb700 1 --2- 172.21.2.101:0/557323220 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe4880186c0 0x7fe48801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:29.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.959 7fe499ffb700 1 -- 172.21.2.101:0/557323220 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe4a00328a0 con 0x7fe4a818a720 2021-02-10T11:27:29.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.960 7fe4ad796700 1 --2- 172.21.2.101:0/557323220 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe4880186c0 0x7fe48801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:29.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.960 7fe4ad796700 1 --2- 172.21.2.101:0/557323220 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe4880186c0 0x7fe48801abc0 crc :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:30.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.991 7fe4af97a700 1 -- 172.21.2.101:0/557323220 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a8105ae0 con 0x7fe4a818a720 2021-02-10T11:27:30.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:29.994 7fe499ffb700 1 -- 172.21.2.101:0/557323220 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe4a0000f90 con 0x7fe4a818a720 2021-02-10T11:27:30.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:30.122 7fe4af97a700 1 -- 172.21.2.101:0/557323220 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839322", "target": ["mgr", ""]}) v1 -- 0x7fe4a80b9020 con 0x7fe4880186c0 2021-02-10T11:27:30.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:30.126 7fe499ffb700 1 -- 172.21.2.101:0/557323220 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fe4a4000c90 con 0x7fe4880186c0 2021-02-10T11:27:30.139 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839322/31c2bb82-cfaa-4033-84b4-3b2da7ac1109 2021-02-10T11:27:30.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:30.130 7fe4af97a700 1 -- 172.21.2.101:0/557323220 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe4880186c0 msgr2=0x7fe48801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:30.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:30.130 7fe4af97a700 1 --2- 172.21.2.101:0/557323220 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe4880186c0 0x7fe48801abc0 crc :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:30.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:30.130 7fe4af97a700 1 -- 172.21.2.101:0/557323220 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe4a818a720 msgr2=0x7fe4a80b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:30.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:30.130 7fe4af97a700 1 --2- 172.21.2.101:0/557323220 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe4a818a720 0x7fe4a80b7be0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fe4a0009980 tx=0x7fe4a000bcd0).stop 2021-02-10T11:27:30.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:30.131 7fe4af97a700 1 -- 172.21.2.101:0/557323220 shutdown_connections 2021-02-10T11:27:30.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:30.131 7fe4af97a700 1 --2- 172.21.2.101:0/557323220 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe4880186c0 0x7fe48801abc0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:30.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:30.131 7fe4af97a700 1 --2- 172.21.2.101:0/557323220 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe4a8189d00 0x7fe4a818a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:30.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:30.131 7fe4af97a700 1 --2- 172.21.2.101:0/557323220 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe4a818a720 0x7fe4a80b7be0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:30.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:30.131 7fe4af97a700 1 --2- 172.21.2.101:0/557323220 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe4a80a0420 0x7fe4a80a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:30.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:30.131 7fe4af97a700 1 -- 172.21.2.101:0/557323220 >> 172.21.2.101:0/557323220 conn(0x7fe4a8094e10 msgr2=0x7fe4a80a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:30.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:30.131 7fe4af97a700 1 -- 172.21.2.101:0/557323220 shutdown_connections 2021-02-10T11:27:30.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:30.131 7fe4af97a700 1 -- 172.21.2.101:0/557323220 wait complete. 2021-02-10T11:27:30.149 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839322 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839322/31c2bb82-cfaa-4033-84b4-3b2da7ac1109 2021-02-10T11:27:30.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:30.150 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:30.150 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:30.487 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:30.487 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:30.488 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839322/31c2bb82-cfaa-4033-84b4-3b2da7ac1109/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:30.870 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:30.870 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:30.870 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246325 s, 42.6 MB/s 2021-02-10T11:27:30.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:30.871 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:30.871 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:31.207 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:31.207 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:31.207 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839322/31c2bb82-cfaa-4033-84b4-3b2da7ac1109/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:31.582 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:31.582 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:31.582 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244243 s, 42.9 MB/s 2021-02-10T11:27:31.583 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:31.583 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:31.583 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:31.918 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:31.924 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:31.924 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839322/31c2bb82-cfaa-4033-84b4-3b2da7ac1109/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:32.300 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:32.300 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:32.300 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234618 s, 44.7 MB/s 2021-02-10T11:27:32.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:32.301 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:32.301 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:32.637 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:32.637 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:32.637 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839322/31c2bb82-cfaa-4033-84b4-3b2da7ac1109/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:33.013 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:33.013 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:33.014 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0255452 s, 41.0 MB/s 2021-02-10T11:27:33.014 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:33.015 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:33.015 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:33.351 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:33.351 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:33.351 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839322/31c2bb82-cfaa-4033-84b4-3b2da7ac1109/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:33.727 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:33.728 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:33.728 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241073 s, 43.5 MB/s 2021-02-10T11:27:33.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:33.729 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:33.729 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:34.066 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:34.067 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:34.067 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839322/31c2bb82-cfaa-4033-84b4-3b2da7ac1109/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:34.441 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:34.441 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:34.441 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247936 s, 42.3 MB/s 2021-02-10T11:27:34.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:34.442 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:34.442 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:34.779 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:34.779 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:34.779 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839322/31c2bb82-cfaa-4033-84b4-3b2da7ac1109/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:35.156 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:35.156 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:35.156 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247423 s, 42.4 MB/s 2021-02-10T11:27:35.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:35.157 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:35.158 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:35.494 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:35.494 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:35.494 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839322/31c2bb82-cfaa-4033-84b4-3b2da7ac1109/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:35.869 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:35.869 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:35.869 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242814 s, 43.2 MB/s 2021-02-10T11:27:35.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:35.870 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:35.870 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:36.207 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:36.207 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:36.207 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839322/31c2bb82-cfaa-4033-84b4-3b2da7ac1109/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:36.583 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:36.583 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:36.583 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242697 s, 43.2 MB/s 2021-02-10T11:27:36.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:36.584 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:36.584 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:36.921 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:36.921 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:36.921 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839322/31c2bb82-cfaa-4033-84b4-3b2da7ac1109/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:37.300 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:37.300 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:37.300 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253294 s, 41.4 MB/s 2021-02-10T11:27:37.301 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839323 2021-02-10T11:27:37.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.706 7fc1aaf96700 1 Processor -- start 2021-02-10T11:27:37.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.707 7fc1aaf96700 1 -- start start 2021-02-10T11:27:37.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.707 7fc1aaf96700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc1a40f05f0 0x7fc1a40f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:37.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.707 7fc1aaf96700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a41b0c70 con 0x7fc1a41b0470 2021-02-10T11:27:37.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.707 7fc1aaf96700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a40f0b50 con 0x7fc1a41b0870 2021-02-10T11:27:37.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.707 7fc1aaf96700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40e5ea0 con 0x7fc1a40f05f0 2021-02-10T11:27:37.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.707 7fc1a8db2700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc1a41b0470 0x7fc1a40ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48466/0 (socket says 172.21.2.101:48466) 2021-02-10T11:27:37.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.707 7fc1a8db2700 1 -- 172.21.2.101:0/2120393950 learned_addr learned my addr 172.21.2.101:0/2120393950 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:27:37.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.708 7fc1a37fe700 1 -- 172.21.2.101:0/2120393950 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1515974902 0 0) 0x7fc18c0009b0 con 0x7fc1a41b0470 2021-02-10T11:27:37.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.708 7fc1a37fe700 1 -- 172.21.2.101:0/2120393950 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc190002fe0 con 0x7fc1a41b0470 2021-02-10T11:27:37.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.708 7fc1a95b3700 1 --2- 172.21.2.101:0/2120393950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc1a40f05f0 0x7fc1a40f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:37.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.708 7fc1a37fe700 1 -- 172.21.2.101:0/2120393950 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4258273312 0 0) 0x7fc18c001010 con 0x7fc1a41b0470 2021-02-10T11:27:37.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.708 7fc1a37fe700 1 -- 172.21.2.101:0/2120393950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc1a40f05f0 msgr2=0x7fc1a40f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:27:37.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.708 7fc1a37fe700 1 --2- 172.21.2.101:0/2120393950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc1a40f05f0 0x7fc1a40f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:37.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.708 7fc1a37fe700 1 -- 172.21.2.101:0/2120393950 >> v1:172.21.2.108:6789/0 conn(0x7fc1a41b0870 legacy=0x7fc1a40efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:27:37.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.708 7fc1a37fe700 1 -- 172.21.2.101:0/2120393950 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a40e52e0 con 0x7fc1a41b0470 2021-02-10T11:27:37.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.708 7fc1a37fe700 1 -- 172.21.2.101:0/2120393950 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fc18c001590 con 0x7fc1a41b0470 2021-02-10T11:27:37.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.708 7fc1a37fe700 1 -- 172.21.2.101:0/2120393950 >> v1:172.21.2.101:6789/0 conn(0x7fc1a41b0470 legacy=0x7fc1a40ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.708 7fc1a37fe700 1 --2- 172.21.2.101:0/2120393950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc190003b90 0x7fc190005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.708 7fc1a37fe700 1 --2- 172.21.2.101:0/2120393950 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc190006770 0x7fc190008c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.708 7fc1a37fe700 1 --2- 172.21.2.101:0/2120393950 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc190009500 0x7fc19000ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.708 7fc1a37fe700 1 -- 172.21.2.101:0/2120393950 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19000c2d0 con 0x7fc190006770 2021-02-10T11:27:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.708 7fc1a37fe700 1 -- 172.21.2.101:0/2120393950 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19000c4f0 con 0x7fc190009500 2021-02-10T11:27:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.708 7fc1a37fe700 1 -- 172.21.2.101:0/2120393950 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19000c710 con 0x7fc190003b90 2021-02-10T11:27:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.708 7fc1a3fff700 1 --2- 172.21.2.101:0/2120393950 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc190006770 0x7fc190008c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.708 7fc1a8db2700 1 --2- 172.21.2.101:0/2120393950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc190003b90 0x7fc190005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.708 7fc1a95b3700 1 --2- 172.21.2.101:0/2120393950 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc190009500 0x7fc19000ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.709 7fc1a3fff700 1 -- 172.21.2.101:0/2120393950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc190003b90 msgr2=0x7fc190005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:27:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.709 7fc1a3fff700 1 --2- 172.21.2.101:0/2120393950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc190003b90 0x7fc190005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.709 7fc1a3fff700 1 -- 172.21.2.101:0/2120393950 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc190009500 msgr2=0x7fc19000ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:27:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.709 7fc1a3fff700 1 --2- 172.21.2.101:0/2120393950 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc190009500 0x7fc19000ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.709 7fc1a3fff700 1 -- 172.21.2.101:0/2120393950 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19000c9b0 con 0x7fc190006770 2021-02-10T11:27:37.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.709 7fc1a3fff700 1 --2- 172.21.2.101:0/2120393950 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc190006770 0x7fc190008c60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7fc194006ad0 tx=0x7fc19404db50).ready entity=mon.0 client_cookie=557267c7be2eb60f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:37.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.709 7fc1a37fe700 1 -- 172.21.2.101:0/2120393950 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc1940066d0 con 0x7fc190006770 2021-02-10T11:27:37.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.709 7fc1a37fe700 1 -- 172.21.2.101:0/2120393950 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc194000e20 con 0x7fc190006770 2021-02-10T11:27:37.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.709 7fc1a37fe700 1 -- 172.21.2.101:0/2120393950 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc194002500 con 0x7fc190006770 2021-02-10T11:27:37.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.709 7fc1aaf96700 1 -- 172.21.2.101:0/2120393950 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc190006770 msgr2=0x7fc190008c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:37.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.709 7fc1aaf96700 1 --2- 172.21.2.101:0/2120393950 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc190006770 0x7fc190008c60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7fc194006ad0 tx=0x7fc19404db50).stop 2021-02-10T11:27:37.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.709 7fc1a95b3700 1 -- 172.21.2.101:0/2120393950 reap_dead start 2021-02-10T11:27:37.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.709 7fc1aaf96700 1 -- 172.21.2.101:0/2120393950 shutdown_connections 2021-02-10T11:27:37.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.709 7fc1aaf96700 1 -- 172.21.2.101:0/2120393950 >> 172.21.2.101:0/2120393950 conn(0x7fc1a40ce3e0 msgr2=0x7fc1a40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:37.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.709 7fc1aaf96700 1 -- 172.21.2.101:0/2120393950 shutdown_connections 2021-02-10T11:27:37.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.709 7fc1aaf96700 1 -- 172.21.2.101:0/2120393950 wait complete. 2021-02-10T11:27:37.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.724 7fc1aaf96700 1 Processor -- start 2021-02-10T11:27:37.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.724 7fc1aaf96700 1 -- start start 2021-02-10T11:27:37.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.724 7fc1aaf96700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1a409b930 0x7fc1a40b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:37.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.724 7fc1aaf96700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc1a40b8410 0x7fc1a40b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:37.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.724 7fc1aaf96700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc1a40c1f80 0x7fc1a40c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:37.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.724 7fc1aaf96700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40c1a00 con 0x7fc1a409b930 2021-02-10T11:27:37.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.724 7fc1aaf96700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40b8e30 con 0x7fc1a40c1f80 2021-02-10T11:27:37.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.724 7fc1aaf96700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40b9050 con 0x7fc1a40b8410 2021-02-10T11:27:37.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.725 7fc1a8db2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1a409b930 0x7fc1a40b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:37.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.725 7fc1a8db2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1a409b930 0x7fc1a40b7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47984/0 (socket says 172.21.2.101:47984) 2021-02-10T11:27:37.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.725 7fc1a8db2700 1 -- 172.21.2.101:0/556066270 learned_addr learned my addr 172.21.2.101:0/556066270 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:27:37.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.725 7fc1a8db2700 1 -- 172.21.2.101:0/556066270 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc1a40b8410 msgr2=0x7fc1a40b8810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:27:37.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.725 7fc1a8db2700 1 --2- 172.21.2.101:0/556066270 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc1a40b8410 0x7fc1a40b8810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:37.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.725 7fc1a8db2700 1 -- 172.21.2.101:0/556066270 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc1a40c1f80 msgr2=0x7fc1a40c2380 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:27:37.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.725 7fc1a8db2700 1 --2- 172.21.2.101:0/556066270 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc1a40c1f80 0x7fc1a40c2380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:37.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.725 7fc1a8db2700 1 -- 172.21.2.101:0/556066270 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a40c29a0 con 0x7fc1a409b930 2021-02-10T11:27:37.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.725 7fc1a8db2700 1 --2- 172.21.2.101:0/556066270 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1a409b930 0x7fc1a40b7c20 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7fc18c001620 tx=0x7fc18c00a920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:37.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.725 7fc1a17fa700 1 -- 172.21.2.101:0/556066270 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc18c00ad10 con 0x7fc1a409b930 2021-02-10T11:27:37.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.725 7fc1a17fa700 1 -- 172.21.2.101:0/556066270 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c004350 con 0x7fc1a409b930 2021-02-10T11:27:37.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.725 7fc1a17fa700 1 -- 172.21.2.101:0/556066270 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc18c004830 con 0x7fc1a409b930 2021-02-10T11:27:37.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.725 7fc1aaf96700 1 -- 172.21.2.101:0/556066270 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a40c29a0 con 0x7fc1a409b930 2021-02-10T11:27:37.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.725 7fc1aaf96700 1 -- 172.21.2.101:0/556066270 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a40c29a0 con 0x7fc1a409b930 2021-02-10T11:27:37.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.726 7fc1a17fa700 1 -- 172.21.2.101:0/556066270 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc18c001690 con 0x7fc1a409b930 2021-02-10T11:27:37.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.726 7fc1a17fa700 1 --2- 172.21.2.101:0/556066270 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1880186c0 0x7fc18801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:37.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.726 7fc1a17fa700 1 -- 172.21.2.101:0/556066270 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc18c0327b0 con 0x7fc1a409b930 2021-02-10T11:27:37.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.726 7fc1a3fff700 1 --2- 172.21.2.101:0/556066270 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1880186c0 0x7fc18801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:37.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.726 7fc1a3fff700 1 --2- 172.21.2.101:0/556066270 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1880186c0 0x7fc18801abc0 crc :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:37.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.757 7fc1aaf96700 1 -- 172.21.2.101:0/556066270 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a4105b30 con 0x7fc1a409b930 2021-02-10T11:27:37.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.759 7fc1a17fa700 1 -- 172.21.2.101:0/556066270 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc18c004140 con 0x7fc1a409b930 2021-02-10T11:27:37.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.886 7fc1aaf96700 1 -- 172.21.2.101:0/556066270 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839323", "target": ["mgr", ""]}) v1 -- 0x7fc1a40b8e30 con 0x7fc1880186c0 2021-02-10T11:27:37.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.913 7fc1a17fa700 1 -- 172.21.2.101:0/556066270 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fc194006540 con 0x7fc1880186c0 2021-02-10T11:27:37.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.919 7fc1aaf96700 1 -- 172.21.2.101:0/556066270 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1880186c0 msgr2=0x7fc18801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:37.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.919 7fc1aaf96700 1 --2- 172.21.2.101:0/556066270 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1880186c0 0x7fc18801abc0 crc :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:37.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.919 7fc1aaf96700 1 -- 172.21.2.101:0/556066270 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1a409b930 msgr2=0x7fc1a40b7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:37.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.919 7fc1aaf96700 1 --2- 172.21.2.101:0/556066270 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1a409b930 0x7fc1a40b7c20 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7fc18c001620 tx=0x7fc18c00a920).stop 2021-02-10T11:27:37.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.919 7fc1aaf96700 1 -- 172.21.2.101:0/556066270 shutdown_connections 2021-02-10T11:27:37.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.919 7fc1aaf96700 1 --2- 172.21.2.101:0/556066270 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1880186c0 0x7fc18801abc0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:37.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.919 7fc1aaf96700 1 --2- 172.21.2.101:0/556066270 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc1a40b8410 0x7fc1a40b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:37.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.919 7fc1aaf96700 1 --2- 172.21.2.101:0/556066270 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc1a40c1f80 0x7fc1a40c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:37.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.919 7fc1aaf96700 1 --2- 172.21.2.101:0/556066270 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1a409b930 0x7fc1a40b7c20 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:37.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.919 7fc1aaf96700 1 -- 172.21.2.101:0/556066270 >> 172.21.2.101:0/556066270 conn(0x7fc1a4094e10 msgr2=0x7fc1a40a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:37.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.919 7fc1aaf96700 1 -- 172.21.2.101:0/556066270 shutdown_connections 2021-02-10T11:27:37.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:37.919 7fc1aaf96700 1 -- 172.21.2.101:0/556066270 wait complete. 2021-02-10T11:27:37.937 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839323 2021-02-10T11:27:38.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.343 7f7be5ad2700 1 Processor -- start 2021-02-10T11:27:38.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.343 7f7be5ad2700 1 -- start start 2021-02-10T11:27:38.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.343 7f7be5ad2700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7be01b0470 0x7f7be01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:38.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.343 7f7be5ad2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7be00ef610 0x7f7be00efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:38.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.343 7f7be5ad2700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be00f0b50 con 0x7f7be00f0030 2021-02-10T11:27:38.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.343 7f7be5ad2700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7be00e52e0 con 0x7f7be01b0470 2021-02-10T11:27:38.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.343 7f7be5ad2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be00e54d0 con 0x7f7be00ef610 2021-02-10T11:27:38.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.344 7f7bdeffd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7be00ef610 0x7f7be00efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:38.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.344 7f7bdf7fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7be01b0470 0x7f7be01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:38.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.344 7f7bdeffd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7be00ef610 0x7f7be00efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:40724/0 (socket says 172.21.2.101:40724) 2021-02-10T11:27:38.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.344 7f7bdf7fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7be01b0470 0x7f7be01b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:58478/0 (socket says 172.21.2.101:58478) 2021-02-10T11:27:38.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.344 7f7bdeffd700 1 -- 172.21.2.101:0/2052898628 learned_addr learned my addr 172.21.2.101:0/2052898628 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:27:38.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.344 7f7bdeffd700 1 -- 172.21.2.101:0/2052898628 >> v2:172.21.2.108:3300/0 conn(0x7f7be01b0470 msgr2=0x7f7be01b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:27:38.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.344 7f7bdeffd700 1 --2- 172.21.2.101:0/2052898628 >> v2:172.21.2.108:3300/0 conn(0x7f7be01b0470 0x7f7be01b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:38.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.344 7f7bdeffd700 1 -- 172.21.2.101:0/2052898628 >> v1:172.21.2.108:6789/0 conn(0x7f7be00f0030 legacy=0x7f7be00f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:38.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.344 7f7bdeffd700 1 -- 172.21.2.101:0/2052898628 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be00f0430 con 0x7f7be00ef610 2021-02-10T11:27:38.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.344 7f7bdf7fe700 1 --2- 172.21.2.101:0/2052898628 >> v2:172.21.2.108:3300/0 conn(0x7f7be01b0470 0x7f7be01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:27:38.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.344 7f7bdeffd700 1 --2- 172.21.2.101:0/2052898628 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7be00ef610 0x7f7be00efa10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f7bc8006700 tx=0x7f7bc8009b80).ready entity=mon.2 client_cookie=7d884b740803f48a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:38.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.344 7f7bde7fc700 1 -- 172.21.2.101:0/2052898628 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7bc800c0c0 con 0x7f7be00ef610 2021-02-10T11:27:38.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.344 7f7bde7fc700 1 -- 172.21.2.101:0/2052898628 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc800c340 con 0x7f7be00ef610 2021-02-10T11:27:38.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.344 7f7bde7fc700 1 -- 172.21.2.101:0/2052898628 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7bc800c820 con 0x7f7be00ef610 2021-02-10T11:27:38.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.344 7f7be5ad2700 1 -- 172.21.2.101:0/2052898628 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7be00ef610 msgr2=0x7f7be00efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:38.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.344 7f7be5ad2700 1 --2- 172.21.2.101:0/2052898628 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7be00ef610 0x7f7be00efa10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f7bc8006700 tx=0x7f7bc8009b80).stop 2021-02-10T11:27:38.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.345 7f7be5ad2700 1 -- 172.21.2.101:0/2052898628 shutdown_connections 2021-02-10T11:27:38.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.345 7f7be5ad2700 1 --2- 172.21.2.101:0/2052898628 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7be00ef610 0x7f7be00efa10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:38.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.345 7f7be5ad2700 1 --2- 172.21.2.101:0/2052898628 >> v2:172.21.2.108:3300/0 conn(0x7f7be01b0470 0x7f7be01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:38.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.345 7f7be5ad2700 1 -- 172.21.2.101:0/2052898628 >> 172.21.2.101:0/2052898628 conn(0x7f7be00ce3e0 msgr2=0x7f7be00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:38.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.345 7f7be5ad2700 1 -- 172.21.2.101:0/2052898628 shutdown_connections 2021-02-10T11:27:38.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.345 7f7be5ad2700 1 -- 172.21.2.101:0/2052898628 wait complete. 2021-02-10T11:27:38.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.359 7f7be5ad2700 1 Processor -- start 2021-02-10T11:27:38.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.359 7f7be5ad2700 1 -- start start 2021-02-10T11:27:38.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.360 7f7be5ad2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7be0190920 0x7f7be00b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:38.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.360 7f7be5ad2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7be00b8200 0x7f7be00b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:38.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.360 7f7be5ad2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7be00c1d80 0x7f7be00c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:38.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.360 7f7be5ad2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be0190d20 con 0x7f7be00c1d80 2021-02-10T11:27:38.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.360 7f7be5ad2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be00b8c20 con 0x7f7be0190920 2021-02-10T11:27:38.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.360 7f7be5ad2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be00c27a0 con 0x7f7be00b8200 2021-02-10T11:27:38.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.360 7f7bdeffd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7be00b8200 0x7f7be00b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:38.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.360 7f7bdffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7be00c1d80 0x7f7be00c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:38.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.360 7f7bdffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7be00c1d80 0x7f7be00c2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:47996/0 (socket says 172.21.2.101:47996) 2021-02-10T11:27:38.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.360 7f7bdffff700 1 -- 172.21.2.101:0/1524820184 learned_addr learned my addr 172.21.2.101:0/1524820184 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:27:38.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.360 7f7bdf7fe700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7be0190920 0x7f7be00b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:38.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.361 7f7bdffff700 1 -- 172.21.2.101:0/1524820184 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7be00b8200 msgr2=0x7f7be00b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:38.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.361 7f7bdffff700 1 --2- 172.21.2.101:0/1524820184 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7be00b8200 0x7f7be00b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:38.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.361 7f7bdffff700 1 -- 172.21.2.101:0/1524820184 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7be0190920 msgr2=0x7f7be00b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:38.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.361 7f7bdffff700 1 --2- 172.21.2.101:0/1524820184 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7be0190920 0x7f7be00b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:38.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.361 7f7bdffff700 1 -- 172.21.2.101:0/1524820184 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be00c29c0 con 0x7f7be00c1d80 2021-02-10T11:27:38.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.361 7f7bdffff700 1 --2- 172.21.2.101:0/1524820184 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7be00c1d80 0x7f7be00c2180 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f7bd8009940 tx=0x7f7bd800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:38.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.361 7f7bd7fff700 1 -- 172.21.2.101:0/1524820184 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7bd8000ce0 con 0x7f7be00c1d80 2021-02-10T11:27:38.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.361 7f7bd7fff700 1 -- 172.21.2.101:0/1524820184 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd800c2f0 con 0x7f7be00c1d80 2021-02-10T11:27:38.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.361 7f7bd7fff700 1 -- 172.21.2.101:0/1524820184 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7bd800c750 con 0x7f7be00c1d80 2021-02-10T11:27:38.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.361 7f7be5ad2700 1 -- 172.21.2.101:0/1524820184 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be00c29c0 con 0x7f7be00c1d80 2021-02-10T11:27:38.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.361 7f7be5ad2700 1 -- 172.21.2.101:0/1524820184 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be00c2f50 con 0x7f7be00c1d80 2021-02-10T11:27:38.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.362 7f7bd7fff700 1 -- 172.21.2.101:0/1524820184 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7bd8001220 con 0x7f7be00c1d80 2021-02-10T11:27:38.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.362 7f7bd7fff700 1 --2- 172.21.2.101:0/1524820184 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7bc40186c0 0x7f7bc401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:38.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.362 7f7bd7fff700 1 -- 172.21.2.101:0/1524820184 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7bd8032920 con 0x7f7be00c1d80 2021-02-10T11:27:38.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.362 7f7bdf7fe700 1 --2- 172.21.2.101:0/1524820184 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7bc40186c0 0x7f7bc401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:38.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.362 7f7bdf7fe700 1 --2- 172.21.2.101:0/1524820184 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7bc40186c0 0x7f7bc401abc0 crc :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:38.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.393 7f7be5ad2700 1 -- 172.21.2.101:0/1524820184 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be0105ae0 con 0x7f7be00c1d80 2021-02-10T11:27:38.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.396 7f7bd7fff700 1 -- 172.21.2.101:0/1524820184 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7bd8002ef0 con 0x7f7be00c1d80 2021-02-10T11:27:38.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.526 7f7be5ad2700 1 -- 172.21.2.101:0/1524820184 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839323", "target": ["mgr", ""]}) v1 -- 0x7f7be0190d20 con 0x7f7bc40186c0 2021-02-10T11:27:38.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.529 7f7bd7fff700 1 -- 172.21.2.101:0/1524820184 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f7bd0006a30 con 0x7f7bc40186c0 2021-02-10T11:27:38.539 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839323/ad2c8199-0c50-41a6-bca8-4edd212ef271 2021-02-10T11:27:38.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.530 7f7be5ad2700 1 -- 172.21.2.101:0/1524820184 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7bc40186c0 msgr2=0x7f7bc401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:38.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.530 7f7be5ad2700 1 --2- 172.21.2.101:0/1524820184 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7bc40186c0 0x7f7bc401abc0 crc :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:38.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.530 7f7be5ad2700 1 -- 172.21.2.101:0/1524820184 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7be00c1d80 msgr2=0x7f7be00c2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:38.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.530 7f7be5ad2700 1 --2- 172.21.2.101:0/1524820184 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7be00c1d80 0x7f7be00c2180 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f7bd8009940 tx=0x7f7bd800bcf0).stop 2021-02-10T11:27:38.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.530 7f7be5ad2700 1 -- 172.21.2.101:0/1524820184 shutdown_connections 2021-02-10T11:27:38.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.530 7f7be5ad2700 1 --2- 172.21.2.101:0/1524820184 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7bc40186c0 0x7f7bc401abc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:38.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.530 7f7be5ad2700 1 --2- 172.21.2.101:0/1524820184 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7be00b8200 0x7f7be00b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:38.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.530 7f7be5ad2700 1 --2- 172.21.2.101:0/1524820184 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7be00c1d80 0x7f7be00c2180 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:38.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.530 7f7be5ad2700 1 --2- 172.21.2.101:0/1524820184 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7be0190920 0x7f7be00b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:38.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.530 7f7be5ad2700 1 -- 172.21.2.101:0/1524820184 >> 172.21.2.101:0/1524820184 conn(0x7f7be0094e10 msgr2=0x7f7be00a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:38.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.531 7f7be5ad2700 1 -- 172.21.2.101:0/1524820184 shutdown_connections 2021-02-10T11:27:38.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:38.531 7f7be5ad2700 1 -- 172.21.2.101:0/1524820184 wait complete. 2021-02-10T11:27:38.549 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839323 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839323/ad2c8199-0c50-41a6-bca8-4edd212ef271 2021-02-10T11:27:38.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:38.549 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:38.549 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:38.886 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:38.886 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:38.886 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839323/ad2c8199-0c50-41a6-bca8-4edd212ef271/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:39.267 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:39.267 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:39.267 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024019 s, 43.7 MB/s 2021-02-10T11:27:39.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:39.269 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:39.269 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:39.607 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:39.608 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:39.608 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839323/ad2c8199-0c50-41a6-bca8-4edd212ef271/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:39.986 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:39.986 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:39.986 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241141 s, 43.5 MB/s 2021-02-10T11:27:39.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:39.987 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:39.987 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:40.323 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:40.323 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:40.323 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839323/ad2c8199-0c50-41a6-bca8-4edd212ef271/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:40.701 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:40.702 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:40.702 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240627 s, 43.6 MB/s 2021-02-10T11:27:40.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:40.703 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:40.703 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:41.037 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:41.038 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:41.038 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839323/ad2c8199-0c50-41a6-bca8-4edd212ef271/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:41.413 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:41.414 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:41.414 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235815 s, 44.5 MB/s 2021-02-10T11:27:41.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:41.415 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:41.415 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:41.751 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:41.751 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:41.751 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839323/ad2c8199-0c50-41a6-bca8-4edd212ef271/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:42.125 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:42.125 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:42.125 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0225689 s, 46.5 MB/s 2021-02-10T11:27:42.126 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:42.126 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:42.126 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:42.460 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:42.461 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:42.461 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839323/ad2c8199-0c50-41a6-bca8-4edd212ef271/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:42.837 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:42.837 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:42.837 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0250242 s, 41.9 MB/s 2021-02-10T11:27:42.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:42.838 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:42.838 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:43.175 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:43.176 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:43.176 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839323/ad2c8199-0c50-41a6-bca8-4edd212ef271/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:43.551 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:43.551 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:43.551 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241857 s, 43.4 MB/s 2021-02-10T11:27:43.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:43.552 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:43.553 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:43.888 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:43.888 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:43.888 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839323/ad2c8199-0c50-41a6-bca8-4edd212ef271/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:44.263 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:44.263 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:44.263 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0258341 s, 40.6 MB/s 2021-02-10T11:27:44.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:44.264 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:44.264 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:44.600 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:44.600 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:44.600 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839323/ad2c8199-0c50-41a6-bca8-4edd212ef271/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:44.977 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:44.977 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:44.977 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235837 s, 44.5 MB/s 2021-02-10T11:27:44.978 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:44.978 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:44.978 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:45.315 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:45.315 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:45.315 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839323/ad2c8199-0c50-41a6-bca8-4edd212ef271/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:45.692 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:45.692 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:45.693 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245989 s, 42.6 MB/s 2021-02-10T11:27:45.693 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839324 2021-02-10T11:27:46.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.099 7fedeed46700 1 Processor -- start 2021-02-10T11:27:46.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.099 7fedeed46700 1 -- start start 2021-02-10T11:27:46.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.100 7fedeed46700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fede81b0470 0x7fede81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:46.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.100 7fedeed46700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede80f0b50 con 0x7fede80f20e0 2021-02-10T11:27:46.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.100 7fedeed46700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede80f24e0 con 0x7fede80ef610 2021-02-10T11:27:46.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.100 7fedeed46700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fede80e5ea0 con 0x7fede81b0470 2021-02-10T11:27:46.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.100 7feded363700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fede80f20e0 0x7fede80f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48504/0 (socket says 172.21.2.101:48504) 2021-02-10T11:27:46.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.100 7feded363700 1 -- 172.21.2.101:0/3626205367 learned_addr learned my addr 172.21.2.101:0/3626205367 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:27:46.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.100 7fedecb62700 1 --2- 172.21.2.101:0/3626205367 >> v2:172.21.2.108:3300/0 conn(0x7fede81b0470 0x7fede81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:46.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.100 7feddf7fe700 1 -- 172.21.2.101:0/3626205367 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559193080 0 0) 0x7fede0001500 con 0x7fede80f20e0 2021-02-10T11:27:46.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.100 7feddf7fe700 1 -- 172.21.2.101:0/3626205367 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedd0002fe0 con 0x7fede80f20e0 2021-02-10T11:27:46.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.100 7feddf7fe700 1 -- 172.21.2.101:0/3626205367 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 175260427 0 0) 0x7fede4000f30 con 0x7fede80ef610 2021-02-10T11:27:46.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.100 7feddf7fe700 1 -- 172.21.2.101:0/3626205367 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedd0004370 con 0x7fede80ef610 2021-02-10T11:27:46.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.101 7fedecb62700 1 -- 172.21.2.101:0/3626205367 >> v1:172.21.2.108:6789/0 conn(0x7fede80ef610 legacy=0x7fede80efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:46.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.101 7fedecb62700 1 -- 172.21.2.101:0/3626205367 >> v1:172.21.2.101:6789/0 conn(0x7fede80f20e0 legacy=0x7fede80f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:46.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.101 7fedecb62700 1 -- 172.21.2.101:0/3626205367 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede80d2fb0 con 0x7fede81b0470 2021-02-10T11:27:46.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.101 7fedecb62700 1 --2- 172.21.2.101:0/3626205367 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fede81b0470 0x7fede81b0870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fedd8006650 tx=0x7fedd8009a80).ready entity=mon.1 client_cookie=891affa2f2a0a64c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:46.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.101 7feddf7fe700 1 -- 172.21.2.101:0/3626205367 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fedd80009b0 con 0x7fede81b0470 2021-02-10T11:27:46.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.101 7feddf7fe700 1 -- 172.21.2.101:0/3626205367 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd800c0c0 con 0x7fede81b0470 2021-02-10T11:27:46.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.101 7feddf7fe700 1 -- 172.21.2.101:0/3626205367 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fedd800c5e0 con 0x7fede81b0470 2021-02-10T11:27:46.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.101 7fedeed46700 1 -- 172.21.2.101:0/3626205367 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fede81b0470 msgr2=0x7fede81b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:46.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.101 7fedeed46700 1 --2- 172.21.2.101:0/3626205367 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fede81b0470 0x7fede81b0870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fedd8006650 tx=0x7fedd8009a80).stop 2021-02-10T11:27:46.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.102 7fedeed46700 1 -- 172.21.2.101:0/3626205367 shutdown_connections 2021-02-10T11:27:46.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.102 7fedeed46700 1 --2- 172.21.2.101:0/3626205367 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fede81b0470 0x7fede81b0870 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:46.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.102 7fedeed46700 1 -- 172.21.2.101:0/3626205367 >> 172.21.2.101:0/3626205367 conn(0x7fede80ce3e0 msgr2=0x7fede80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:46.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.102 7fedeed46700 1 -- 172.21.2.101:0/3626205367 shutdown_connections 2021-02-10T11:27:46.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.102 7fedeed46700 1 -- 172.21.2.101:0/3626205367 wait complete. 2021-02-10T11:27:46.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.116 7fedeed46700 1 Processor -- start 2021-02-10T11:27:46.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.116 7fedeed46700 1 -- start start 2021-02-10T11:27:46.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.117 7fedeed46700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fede809e160 0x7fede809e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:46.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.117 7fedeed46700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fede809ee60 0x7fede80ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:46.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.117 7fedeed46700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fede809f260 0x7fede80aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:46.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.117 7fedeed46700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede80ae470 con 0x7fede809e160 2021-02-10T11:27:46.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.117 7fedeed46700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede809e560 con 0x7fede809f260 2021-02-10T11:27:46.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.117 7fedeed46700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede81933c0 con 0x7fede809ee60 2021-02-10T11:27:46.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.117 7fedecb62700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fede809e160 0x7fede809e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:46.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.117 7fedecb62700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fede809e160 0x7fede809e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48014/0 (socket says 172.21.2.101:48014) 2021-02-10T11:27:46.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.117 7fedecb62700 1 -- 172.21.2.101:0/3505876861 learned_addr learned my addr 172.21.2.101:0/3505876861 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:27:46.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.117 7feded363700 1 --2- 172.21.2.101:0/3505876861 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fede809f260 0x7fede80aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:46.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.117 7feddffff700 1 --2- 172.21.2.101:0/3505876861 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fede809ee60 0x7fede80ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:46.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.117 7fedecb62700 1 -- 172.21.2.101:0/3505876861 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fede809ee60 msgr2=0x7fede80ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:46.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.117 7fedecb62700 1 --2- 172.21.2.101:0/3505876861 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fede809ee60 0x7fede80ae970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:46.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.117 7fedecb62700 1 -- 172.21.2.101:0/3505876861 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fede809f260 msgr2=0x7fede80aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:46.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.117 7fedecb62700 1 --2- 172.21.2.101:0/3505876861 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fede809f260 0x7fede80aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:46.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.117 7fedecb62700 1 -- 172.21.2.101:0/3505876861 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede80af5b0 con 0x7fede809e160 2021-02-10T11:27:46.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.118 7fedecb62700 1 --2- 172.21.2.101:0/3505876861 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fede809e160 0x7fede809e840 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7fedd8009630 tx=0x7fedd800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:46.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.118 7feddd7fa700 1 -- 172.21.2.101:0/3505876861 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fedd80093a0 con 0x7fede809e160 2021-02-10T11:27:46.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.118 7feddd7fa700 1 -- 172.21.2.101:0/3505876861 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd800bd70 con 0x7fede809e160 2021-02-10T11:27:46.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.118 7fedeed46700 1 -- 172.21.2.101:0/3505876861 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede80af5b0 con 0x7fede809e160 2021-02-10T11:27:46.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.118 7feddd7fa700 1 -- 172.21.2.101:0/3505876861 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fedd80054d0 con 0x7fede809e160 2021-02-10T11:27:46.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.118 7fedeed46700 1 -- 172.21.2.101:0/3505876861 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede80af5b0 con 0x7fede809e160 2021-02-10T11:27:46.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.118 7feddd7fa700 1 -- 172.21.2.101:0/3505876861 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fedd800a000 con 0x7fede809e160 2021-02-10T11:27:46.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.118 7feddd7fa700 1 --2- 172.21.2.101:0/3505876861 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fedc8018730 0x7fedc801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:46.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.118 7feddd7fa700 1 -- 172.21.2.101:0/3505876861 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fedd8032630 con 0x7fede809e160 2021-02-10T11:27:46.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.118 7feddffff700 1 --2- 172.21.2.101:0/3505876861 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fedc8018730 0x7fedc801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:46.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.119 7feddffff700 1 --2- 172.21.2.101:0/3505876861 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fedc8018730 0x7fedc801abc0 crc :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:46.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.150 7fedeed46700 1 -- 172.21.2.101:0/3505876861 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede81057f0 con 0x7fede809e160 2021-02-10T11:27:46.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.152 7feddd7fa700 1 -- 172.21.2.101:0/3505876861 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fedd800a330 con 0x7fede809e160 2021-02-10T11:27:46.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.280 7fedeed46700 1 -- 172.21.2.101:0/3505876861 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839324", "target": ["mgr", ""]}) v1 -- 0x7fede81057f0 con 0x7fedc8018730 2021-02-10T11:27:46.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.307 7feddd7fa700 1 -- 172.21.2.101:0/3505876861 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fede4006a50 con 0x7fedc8018730 2021-02-10T11:27:46.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.313 7fedeed46700 1 -- 172.21.2.101:0/3505876861 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fedc8018730 msgr2=0x7fedc801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:46.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.313 7fedeed46700 1 --2- 172.21.2.101:0/3505876861 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fedc8018730 0x7fedc801abc0 crc :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:46.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.313 7fedeed46700 1 -- 172.21.2.101:0/3505876861 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fede809e160 msgr2=0x7fede809e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:46.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.313 7fedeed46700 1 --2- 172.21.2.101:0/3505876861 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fede809e160 0x7fede809e840 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7fedd8009630 tx=0x7fedd800b980).stop 2021-02-10T11:27:46.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.313 7fedeed46700 1 -- 172.21.2.101:0/3505876861 shutdown_connections 2021-02-10T11:27:46.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.313 7fedeed46700 1 --2- 172.21.2.101:0/3505876861 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fedc8018730 0x7fedc801abc0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:46.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.313 7fedeed46700 1 --2- 172.21.2.101:0/3505876861 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fede809ee60 0x7fede80ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:46.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.313 7fedeed46700 1 --2- 172.21.2.101:0/3505876861 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fede809f260 0x7fede80aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:46.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.313 7fedeed46700 1 --2- 172.21.2.101:0/3505876861 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fede809e160 0x7fede809e840 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:46.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.313 7fedeed46700 1 -- 172.21.2.101:0/3505876861 >> 172.21.2.101:0/3505876861 conn(0x7fede80abdd0 msgr2=0x7fede80a9900 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:46.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.313 7fedeed46700 1 -- 172.21.2.101:0/3505876861 shutdown_connections 2021-02-10T11:27:46.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.313 7fedeed46700 1 -- 172.21.2.101:0/3505876861 wait complete. 2021-02-10T11:27:46.332 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839324 2021-02-10T11:27:46.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.738 7f8de618c700 1 Processor -- start 2021-02-10T11:27:46.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.738 7f8de618c700 1 -- start start 2021-02-10T11:27:46.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.739 7f8de618c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8de01b0470 0x7f8de01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:46.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.739 7f8de618c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8de00ef610 0x7f8de00efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:46.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.739 7f8de618c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8de00f0b50 con 0x7f8de00f0030 2021-02-10T11:27:46.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.739 7f8de618c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8de00e52e0 con 0x7f8de01b0470 2021-02-10T11:27:46.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.739 7f8de618c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de00e54d0 con 0x7f8de00ef610 2021-02-10T11:27:46.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.739 7f8ddffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8de00f0030 0x7f8de00f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48516/0 (socket says 172.21.2.101:48516) 2021-02-10T11:27:46.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.739 7f8ddffff700 1 -- 172.21.2.101:0/2597584239 learned_addr learned my addr 172.21.2.101:0/2597584239 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:27:46.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.739 7f8dde7fc700 1 -- 172.21.2.101:0/2597584239 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 25234174 0 0) 0x7f8dd8001500 con 0x7f8de00f0030 2021-02-10T11:27:46.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.739 7f8dde7fc700 1 -- 172.21.2.101:0/2597584239 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dcc002fe0 con 0x7f8de00f0030 2021-02-10T11:27:46.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.739 7f8ddf7fe700 1 --2- 172.21.2.101:0/2597584239 >> v2:172.21.2.108:3300/0 conn(0x7f8de01b0470 0x7f8de01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:46.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.739 7f8ddeffd700 1 --2- 172.21.2.101:0/2597584239 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8de00ef610 0x7f8de00efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:46.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8dde7fc700 1 -- 172.21.2.101:0/2597584239 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4209233472 0 0) 0x7f8dd8001460 con 0x7f8de00f0030 2021-02-10T11:27:46.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8dde7fc700 1 -- 172.21.2.101:0/2597584239 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8de00ef610 msgr2=0x7f8de00efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:27:46.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8dde7fc700 1 --2- 172.21.2.101:0/2597584239 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8de00ef610 0x7f8de00efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:46.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8dde7fc700 1 -- 172.21.2.101:0/2597584239 >> v2:172.21.2.108:3300/0 conn(0x7f8de01b0470 msgr2=0x7f8de01b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:27:46.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8dde7fc700 1 --2- 172.21.2.101:0/2597584239 >> v2:172.21.2.108:3300/0 conn(0x7f8de01b0470 0x7f8de01b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:46.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8dde7fc700 1 -- 172.21.2.101:0/2597584239 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de00f0430 con 0x7f8de00f0030 2021-02-10T11:27:46.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8dde7fc700 1 -- 172.21.2.101:0/2597584239 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f8dd8001980 con 0x7f8de00f0030 2021-02-10T11:27:46.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8dde7fc700 1 -- 172.21.2.101:0/2597584239 >> v1:172.21.2.101:6789/0 conn(0x7f8de00f0030 legacy=0x7f8de00f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:46.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8dde7fc700 1 --2- 172.21.2.101:0/2597584239 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8dcc003b10 0x7f8dcc005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:46.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8dde7fc700 1 --2- 172.21.2.101:0/2597584239 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8dcc006820 0x7f8dcc008d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:46.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8dde7fc700 1 --2- 172.21.2.101:0/2597584239 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8dcc0095b0 0x7f8dcc00bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:46.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8dde7fc700 1 -- 172.21.2.101:0/2597584239 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc00c380 con 0x7f8dcc003b10 2021-02-10T11:27:46.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8dde7fc700 1 -- 172.21.2.101:0/2597584239 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc050570 con 0x7f8dcc0095b0 2021-02-10T11:27:46.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8dde7fc700 1 -- 172.21.2.101:0/2597584239 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc050790 con 0x7f8dcc006820 2021-02-10T11:27:46.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8ddf7fe700 1 --2- 172.21.2.101:0/2597584239 >> v2:172.21.2.108:3300/0 conn(0x7f8de01b0470 0x7f8de01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:27:46.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8ddf7fe700 1 --2- 172.21.2.101:0/2597584239 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8dcc003b10 0x7f8dcc005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:46.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8ddeffd700 1 --2- 172.21.2.101:0/2597584239 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8dcc006820 0x7f8dcc008d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:46.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8ddffff700 1 --2- 172.21.2.101:0/2597584239 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8dcc0095b0 0x7f8dcc00bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:46.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8ddf7fe700 1 -- 172.21.2.101:0/2597584239 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8dcc006820 msgr2=0x7f8dcc008d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:27:46.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8ddf7fe700 1 --2- 172.21.2.101:0/2597584239 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8dcc006820 0x7f8dcc008d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:46.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8ddf7fe700 1 -- 172.21.2.101:0/2597584239 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8dcc0095b0 msgr2=0x7f8dcc00bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:27:46.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8ddf7fe700 1 --2- 172.21.2.101:0/2597584239 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8dcc0095b0 0x7f8dcc00bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:46.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8ddf7fe700 1 -- 172.21.2.101:0/2597584239 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dcc050a30 con 0x7f8dcc003b10 2021-02-10T11:27:46.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8ddf7fe700 1 --2- 172.21.2.101:0/2597584239 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8dcc003b10 0x7f8dcc005f80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f8dc8009880 tx=0x7f8dc8009c20).ready entity=mon.0 client_cookie=f5edb6b83b3b093b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:46.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.740 7f8dde7fc700 1 -- 172.21.2.101:0/2597584239 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8dc8006ce0 con 0x7f8dcc003b10 2021-02-10T11:27:46.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.741 7f8dde7fc700 1 -- 172.21.2.101:0/2597584239 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc8000c90 con 0x7f8dcc003b10 2021-02-10T11:27:46.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.741 7f8dde7fc700 1 -- 172.21.2.101:0/2597584239 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8dc8002220 con 0x7f8dcc003b10 2021-02-10T11:27:46.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.741 7f8de618c700 1 -- 172.21.2.101:0/2597584239 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8dcc003b10 msgr2=0x7f8dcc005f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:46.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.741 7f8de618c700 1 --2- 172.21.2.101:0/2597584239 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8dcc003b10 0x7f8dcc005f80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f8dc8009880 tx=0x7f8dc8009c20).stop 2021-02-10T11:27:46.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.741 7f8ddffff700 1 -- 172.21.2.101:0/2597584239 reap_dead start 2021-02-10T11:27:46.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.741 7f8de618c700 1 -- 172.21.2.101:0/2597584239 shutdown_connections 2021-02-10T11:27:46.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.741 7f8de618c700 1 -- 172.21.2.101:0/2597584239 >> 172.21.2.101:0/2597584239 conn(0x7f8de00ce3e0 msgr2=0x7f8de00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:46.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.741 7f8de618c700 1 -- 172.21.2.101:0/2597584239 shutdown_connections 2021-02-10T11:27:46.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.741 7f8de618c700 1 -- 172.21.2.101:0/2597584239 wait complete. 2021-02-10T11:27:46.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.756 7f8de618c700 1 Processor -- start 2021-02-10T11:27:46.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.756 7f8de618c700 1 -- start start 2021-02-10T11:27:46.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.756 7f8de618c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8de009b930 0x7f8de00b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:46.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.756 7f8de618c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8de00b8410 0x7f8de00b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:46.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.756 7f8de618c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8de00c1f80 0x7f8de00c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:46.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.756 7f8de618c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de00c1a00 con 0x7f8de00c1f80 2021-02-10T11:27:46.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.756 7f8de618c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de00b8e30 con 0x7f8de00b8410 2021-02-10T11:27:46.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.756 7f8de618c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de00b9050 con 0x7f8de009b930 2021-02-10T11:27:46.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.757 7f8ddffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8de00c1f80 0x7f8de00c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:46.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.757 7f8ddffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8de00c1f80 0x7f8de00c2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48032/0 (socket says 172.21.2.101:48032) 2021-02-10T11:27:46.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.757 7f8ddffff700 1 -- 172.21.2.101:0/2705530228 learned_addr learned my addr 172.21.2.101:0/2705530228 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:27:46.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.757 7f8ddffff700 1 -- 172.21.2.101:0/2705530228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8de009b930 msgr2=0x7f8de00b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:46.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.757 7f8ddffff700 1 --2- 172.21.2.101:0/2705530228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8de009b930 0x7f8de00b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:46.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.757 7f8ddffff700 1 -- 172.21.2.101:0/2705530228 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8de00b8410 msgr2=0x7f8de00b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:46.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.757 7f8ddffff700 1 --2- 172.21.2.101:0/2705530228 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8de00b8410 0x7f8de00b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:46.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.757 7f8ddffff700 1 -- 172.21.2.101:0/2705530228 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de00c29a0 con 0x7f8de00c1f80 2021-02-10T11:27:46.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.757 7f8ddffff700 1 --2- 172.21.2.101:0/2705530228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8de00c1f80 0x7f8de00c2380 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f8dd800a8e0 tx=0x7f8dd800ac80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:46.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.757 7f8dd7fff700 1 -- 172.21.2.101:0/2705530228 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8dd8004930 con 0x7f8de00c1f80 2021-02-10T11:27:46.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.757 7f8dd7fff700 1 -- 172.21.2.101:0/2705530228 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd8000eb0 con 0x7f8de00c1f80 2021-02-10T11:27:46.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.757 7f8dd7fff700 1 -- 172.21.2.101:0/2705530228 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8dd8007020 con 0x7f8de00c1f80 2021-02-10T11:27:46.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.757 7f8de618c700 1 -- 172.21.2.101:0/2705530228 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de00c29a0 con 0x7f8de00c1f80 2021-02-10T11:27:46.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.757 7f8de618c700 1 -- 172.21.2.101:0/2705530228 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de00c29a0 con 0x7f8de00c1f80 2021-02-10T11:27:46.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.758 7f8dd7fff700 1 -- 172.21.2.101:0/2705530228 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8dd800adc0 con 0x7f8de00c1f80 2021-02-10T11:27:46.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.758 7f8dd7fff700 1 --2- 172.21.2.101:0/2705530228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8dc4018730 0x7f8dc401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:46.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.758 7f8dd7fff700 1 -- 172.21.2.101:0/2705530228 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8dd8032960 con 0x7f8de00c1f80 2021-02-10T11:27:46.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.758 7f8ddf7fe700 1 --2- 172.21.2.101:0/2705530228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8dc4018730 0x7f8dc401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:46.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.758 7f8ddf7fe700 1 --2- 172.21.2.101:0/2705530228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8dc4018730 0x7f8dc401ac10 crc :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:46.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.789 7f8de618c700 1 -- 172.21.2.101:0/2705530228 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de0105b30 con 0x7f8de00c1f80 2021-02-10T11:27:46.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.791 7f8dd7fff700 1 -- 172.21.2.101:0/2705530228 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8dd8000eb0 con 0x7f8de00c1f80 2021-02-10T11:27:46.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.919 7f8de618c700 1 -- 172.21.2.101:0/2705530228 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839324", "target": ["mgr", ""]}) v1 -- 0x7f8de0105b30 con 0x7f8dc4018730 2021-02-10T11:27:46.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.922 7f8dd7fff700 1 -- 172.21.2.101:0/2705530228 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f8dc80069e0 con 0x7f8dc4018730 2021-02-10T11:27:46.937 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839324/92f9fcf7-31f7-45a1-811f-0d5376cc7c38 2021-02-10T11:27:46.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.928 7f8de618c700 1 -- 172.21.2.101:0/2705530228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8dc4018730 msgr2=0x7f8dc401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:46.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.928 7f8de618c700 1 --2- 172.21.2.101:0/2705530228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8dc4018730 0x7f8dc401ac10 crc :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:46.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.928 7f8de618c700 1 -- 172.21.2.101:0/2705530228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8de00c1f80 msgr2=0x7f8de00c2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:46.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.928 7f8de618c700 1 --2- 172.21.2.101:0/2705530228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8de00c1f80 0x7f8de00c2380 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f8dd800a8e0 tx=0x7f8dd800ac80).stop 2021-02-10T11:27:46.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.928 7f8de618c700 1 -- 172.21.2.101:0/2705530228 shutdown_connections 2021-02-10T11:27:46.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.928 7f8de618c700 1 --2- 172.21.2.101:0/2705530228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8de00c1f80 0x7f8de00c2380 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:46.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.928 7f8de618c700 1 --2- 172.21.2.101:0/2705530228 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8de00b8410 0x7f8de00b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:46.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.928 7f8de618c700 1 --2- 172.21.2.101:0/2705530228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8dc4018730 0x7f8dc401ac10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:46.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.928 7f8de618c700 1 --2- 172.21.2.101:0/2705530228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8de009b930 0x7f8de00b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:46.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.928 7f8de618c700 1 -- 172.21.2.101:0/2705530228 >> 172.21.2.101:0/2705530228 conn(0x7f8de0094e10 msgr2=0x7f8de00a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:46.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.928 7f8de618c700 1 -- 172.21.2.101:0/2705530228 shutdown_connections 2021-02-10T11:27:46.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:46.928 7f8de618c700 1 -- 172.21.2.101:0/2705530228 wait complete. 2021-02-10T11:27:46.946 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839324 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839324/92f9fcf7-31f7-45a1-811f-0d5376cc7c38 2021-02-10T11:27:46.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:46.947 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:46.947 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:47.283 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:47.283 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:47.283 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839324/92f9fcf7-31f7-45a1-811f-0d5376cc7c38/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:47.664 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:47.664 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:47.664 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0232131 s, 45.2 MB/s 2021-02-10T11:27:47.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:47.665 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:47.665 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:48.002 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:48.002 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:48.003 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839324/92f9fcf7-31f7-45a1-811f-0d5376cc7c38/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:48.380 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:48.381 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:48.381 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251771 s, 41.6 MB/s 2021-02-10T11:27:48.382 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:48.382 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:48.382 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:48.719 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:48.719 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:48.720 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839324/92f9fcf7-31f7-45a1-811f-0d5376cc7c38/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:49.092 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:49.092 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:49.092 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237014 s, 44.2 MB/s 2021-02-10T11:27:49.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:49.093 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:49.093 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:49.431 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:49.431 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:49.432 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839324/92f9fcf7-31f7-45a1-811f-0d5376cc7c38/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:49.809 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:49.809 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:49.809 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024223 s, 43.3 MB/s 2021-02-10T11:27:49.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:49.811 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:49.811 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:50.148 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:50.149 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:50.149 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839324/92f9fcf7-31f7-45a1-811f-0d5376cc7c38/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:50.523 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:50.523 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:50.524 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0221545 s, 47.3 MB/s 2021-02-10T11:27:50.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:50.525 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:50.525 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:50.859 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:50.860 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:50.860 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839324/92f9fcf7-31f7-45a1-811f-0d5376cc7c38/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:51.237 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:51.237 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:51.237 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.025134 s, 41.7 MB/s 2021-02-10T11:27:51.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:51.238 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:51.238 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:51.573 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:51.573 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:51.574 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839324/92f9fcf7-31f7-45a1-811f-0d5376cc7c38/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:51.949 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:51.949 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:51.949 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.022967 s, 45.7 MB/s 2021-02-10T11:27:51.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:51.951 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:51.951 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:52.287 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:52.287 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:52.288 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839324/92f9fcf7-31f7-45a1-811f-0d5376cc7c38/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:52.663 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:52.663 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:52.663 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0230487 s, 45.5 MB/s 2021-02-10T11:27:52.664 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:52.664 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:52.665 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:53.000 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:53.000 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:53.001 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839324/92f9fcf7-31f7-45a1-811f-0d5376cc7c38/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:53.378 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:53.378 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:53.379 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0250881 s, 41.8 MB/s 2021-02-10T11:27:53.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:53.379 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:53.380 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:53.716 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:53.716 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:53.717 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839324/92f9fcf7-31f7-45a1-811f-0d5376cc7c38/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:54.091 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:54.091 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:54.091 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0262243 s, 40.0 MB/s 2021-02-10T11:27:54.092 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839325 2021-02-10T11:27:54.110 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:27:54.137 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:27:54.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.499 7f10c5ef1700 1 Processor -- start 2021-02-10T11:27:54.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.499 7f10c5ef1700 1 -- start start 2021-02-10T11:27:54.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.499 7f10c5ef1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f10c01b0470 0x7f10c01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:54.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.500 7f10c5ef1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10c00ef610 0x7f10c00efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:54.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.500 7f10c5ef1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f10c00f0030 0x7f10c00f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:54.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.500 7f10c5ef1700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10c00e5ea0 con 0x7f10c00f0030 2021-02-10T11:27:54.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.500 7f10c5ef1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10c00f0b50 con 0x7f10c01b0470 2021-02-10T11:27:54.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.500 7f10c5ef1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c00e52e0 con 0x7f10c00ef610 2021-02-10T11:27:54.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.500 7f10bffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f10c00f0030 0x7f10c00f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:54.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.500 7f10bffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f10c00f0030 0x7f10c00f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48038/0 (socket says 172.21.2.101:48038) 2021-02-10T11:27:54.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.500 7f10bffff700 1 -- 172.21.2.101:0/3070292077 learned_addr learned my addr 172.21.2.101:0/3070292077 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:27:54.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.500 7f10bffff700 1 -- 172.21.2.101:0/3070292077 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10c00ef610 msgr2=0x7f10c00efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:27:54.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.500 7f10bffff700 1 --2- 172.21.2.101:0/3070292077 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10c00ef610 0x7f10c00efa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:54.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.500 7f10bffff700 1 -- 172.21.2.101:0/3070292077 >> v2:172.21.2.108:3300/0 conn(0x7f10c01b0470 msgr2=0x7f10c01b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:27:54.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.500 7f10bffff700 1 --2- 172.21.2.101:0/3070292077 >> v2:172.21.2.108:3300/0 conn(0x7f10c01b0470 0x7f10c01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:54.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.500 7f10bffff700 1 -- 172.21.2.101:0/3070292077 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c00f10d0 con 0x7f10c00f0030 2021-02-10T11:27:54.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.500 7f10bffff700 1 --2- 172.21.2.101:0/3070292077 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f10c00f0030 0x7f10c00f0430 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7f10b8006f00 tx=0x7f10b8009e90).ready entity=mon.0 client_cookie=d562c603b9e4b487 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:54.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.500 7f10be7fc700 1 -- 172.21.2.101:0/3070292077 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f10b800b300 con 0x7f10c00f0030 2021-02-10T11:27:54.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.500 7f10be7fc700 1 -- 172.21.2.101:0/3070292077 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b800b580 con 0x7f10c00f0030 2021-02-10T11:27:54.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.500 7f10be7fc700 1 -- 172.21.2.101:0/3070292077 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f10b800b9d0 con 0x7f10c00f0030 2021-02-10T11:27:54.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.500 7f10c5ef1700 1 -- 172.21.2.101:0/3070292077 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f10c00f0030 msgr2=0x7f10c00f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:54.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.500 7f10c5ef1700 1 --2- 172.21.2.101:0/3070292077 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f10c00f0030 0x7f10c00f0430 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7f10b8006f00 tx=0x7f10b8009e90).stop 2021-02-10T11:27:54.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.501 7f10c5ef1700 1 -- 172.21.2.101:0/3070292077 shutdown_connections 2021-02-10T11:27:54.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.501 7f10c5ef1700 1 --2- 172.21.2.101:0/3070292077 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10c00ef610 0x7f10c00efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:54.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.501 7f10c5ef1700 1 --2- 172.21.2.101:0/3070292077 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f10c00f0030 0x7f10c00f0430 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:54.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.501 7f10c5ef1700 1 --2- 172.21.2.101:0/3070292077 >> v2:172.21.2.108:3300/0 conn(0x7f10c01b0470 0x7f10c01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:54.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.501 7f10c5ef1700 1 -- 172.21.2.101:0/3070292077 >> 172.21.2.101:0/3070292077 conn(0x7f10c00ce3e0 msgr2=0x7f10c00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:54.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.501 7f10c5ef1700 1 -- 172.21.2.101:0/3070292077 shutdown_connections 2021-02-10T11:27:54.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.501 7f10c5ef1700 1 -- 172.21.2.101:0/3070292077 wait complete. 2021-02-10T11:27:54.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.516 7f10c5ef1700 1 Processor -- start 2021-02-10T11:27:54.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.516 7f10c5ef1700 1 -- start start 2021-02-10T11:27:54.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.516 7f10c5ef1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10c0190920 0x7f10c008f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:54.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.516 7f10c5ef1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f10c0190d20 0x7f10c008fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:54.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.516 7f10c5ef1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f10c0193270 0x7f10c0193670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:54.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.516 7f10c5ef1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c0090470 con 0x7f10c0190d20 2021-02-10T11:27:54.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.516 7f10c5ef1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c0193c90 con 0x7f10c0193270 2021-02-10T11:27:54.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.516 7f10c5ef1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c0193e70 con 0x7f10c0190920 2021-02-10T11:27:54.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.516 7f10beffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f10c0190d20 0x7f10c008fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:54.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.516 7f10beffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f10c0190d20 0x7f10c008fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48044/0 (socket says 172.21.2.101:48044) 2021-02-10T11:27:54.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.516 7f10beffd700 1 -- 172.21.2.101:0/3881849003 learned_addr learned my addr 172.21.2.101:0/3881849003 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:27:54.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.516 7f10beffd700 1 -- 172.21.2.101:0/3881849003 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10c0190920 msgr2=0x7f10c008f600 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:27:54.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.516 7f10beffd700 1 --2- 172.21.2.101:0/3881849003 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10c0190920 0x7f10c008f600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:54.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.516 7f10beffd700 1 -- 172.21.2.101:0/3881849003 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f10c0193270 msgr2=0x7f10c0193670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:27:54.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.516 7f10beffd700 1 --2- 172.21.2.101:0/3881849003 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f10c0193270 0x7f10c0193670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:54.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.517 7f10beffd700 1 -- 172.21.2.101:0/3881849003 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c0194090 con 0x7f10c0190d20 2021-02-10T11:27:54.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.517 7f10beffd700 1 --2- 172.21.2.101:0/3881849003 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f10c0190d20 0x7f10c008fc20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7f10b000b5c0 tx=0x7f10b000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:54.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.517 7f10b7fff700 1 -- 172.21.2.101:0/3881849003 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f10b000bd50 con 0x7f10c0190d20 2021-02-10T11:27:54.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.517 7f10b7fff700 1 -- 172.21.2.101:0/3881849003 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b0006030 con 0x7f10c0190d20 2021-02-10T11:27:54.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.517 7f10c5ef1700 1 -- 172.21.2.101:0/3881849003 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c0194090 con 0x7f10c0190d20 2021-02-10T11:27:54.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.517 7f10c5ef1700 1 -- 172.21.2.101:0/3881849003 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c01920b0 con 0x7f10c0190d20 2021-02-10T11:27:54.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.517 7f10b7fff700 1 -- 172.21.2.101:0/3881849003 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f10b0006360 con 0x7f10c0190d20 2021-02-10T11:27:54.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.517 7f10b7fff700 1 -- 172.21.2.101:0/3881849003 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f10b0006210 con 0x7f10c0190d20 2021-02-10T11:27:54.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.517 7f10b7fff700 1 --2- 172.21.2.101:0/3881849003 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10a40186c0 0x7f10a401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:54.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.517 7f10b7fff700 1 -- 172.21.2.101:0/3881849003 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f10b00327c0 con 0x7f10c0190d20 2021-02-10T11:27:54.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.518 7f10bf7fe700 1 --2- 172.21.2.101:0/3881849003 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10a40186c0 0x7f10a401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:54.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.518 7f10bf7fe700 1 --2- 172.21.2.101:0/3881849003 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10a40186c0 0x7f10a401abc0 crc :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:54.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.549 7f10c5ef1700 1 -- 172.21.2.101:0/3881849003 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10c0105af0 con 0x7f10c0190d20 2021-02-10T11:27:54.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.552 7f10b7fff700 1 -- 172.21.2.101:0/3881849003 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f10b0000940 con 0x7f10c0190d20 2021-02-10T11:27:54.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.678 7f10c5ef1700 1 -- 172.21.2.101:0/3881849003 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839325", "target": ["mgr", ""]}) v1 -- 0x7f10c0090470 con 0x7f10a40186c0 2021-02-10T11:27:54.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.708 7f10b7fff700 1 -- 172.21.2.101:0/3881849003 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f10a8006960 con 0x7f10a40186c0 2021-02-10T11:27:54.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.711 7f10c5ef1700 1 -- 172.21.2.101:0/3881849003 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10a40186c0 msgr2=0x7f10a401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:54.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.711 7f10c5ef1700 1 --2- 172.21.2.101:0/3881849003 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10a40186c0 0x7f10a401abc0 crc :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:54.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.711 7f10c5ef1700 1 -- 172.21.2.101:0/3881849003 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f10c0190d20 msgr2=0x7f10c008fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:54.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.711 7f10c5ef1700 1 --2- 172.21.2.101:0/3881849003 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f10c0190d20 0x7f10c008fc20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7f10b000b5c0 tx=0x7f10b000b960).stop 2021-02-10T11:27:54.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.712 7f10c5ef1700 1 -- 172.21.2.101:0/3881849003 shutdown_connections 2021-02-10T11:27:54.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.712 7f10c5ef1700 1 --2- 172.21.2.101:0/3881849003 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f10c0193270 0x7f10c0193670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:54.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.712 7f10c5ef1700 1 --2- 172.21.2.101:0/3881849003 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f10c0190d20 0x7f10c008fc20 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:54.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.712 7f10c5ef1700 1 --2- 172.21.2.101:0/3881849003 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10a40186c0 0x7f10a401abc0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:54.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.712 7f10c5ef1700 1 --2- 172.21.2.101:0/3881849003 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10c0190920 0x7f10c008f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:54.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.712 7f10c5ef1700 1 -- 172.21.2.101:0/3881849003 >> 172.21.2.101:0/3881849003 conn(0x7f10c00bf4e0 msgr2=0x7f10c00be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:54.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.712 7f10c5ef1700 1 -- 172.21.2.101:0/3881849003 shutdown_connections 2021-02-10T11:27:54.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:54.712 7f10c5ef1700 1 -- 172.21.2.101:0/3881849003 wait complete. 2021-02-10T11:27:54.730 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839325 2021-02-10T11:27:55.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.136 7f8a65c37700 1 Processor -- start 2021-02-10T11:27:55.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.136 7f8a65c37700 1 -- start start 2021-02-10T11:27:55.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.137 7f8a65c37700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8a601b0470 0x7f8a601b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:55.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.137 7f8a65c37700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a600f0b50 con 0x7f8a600ef610 2021-02-10T11:27:55.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.137 7f8a65c37700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a600f24e0 con 0x7f8a600f20e0 2021-02-10T11:27:55.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.137 7f8a65c37700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a600e5ea0 con 0x7f8a601b0470 2021-02-10T11:27:55.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.137 7f8a5f7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8a601b0470 0x7f8a601b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:55.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.137 7f8a5effd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8a600ef610 0x7f8a600efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48546/0 (socket says 172.21.2.101:48546) 2021-02-10T11:27:55.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.137 7f8a5effd700 1 -- 172.21.2.101:0/1861442499 learned_addr learned my addr 172.21.2.101:0/1861442499 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:27:55.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.137 7f8a5f7fe700 1 -- 172.21.2.101:0/1861442499 >> v1:172.21.2.108:6789/0 conn(0x7f8a600f20e0 legacy=0x7f8a600f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:55.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.137 7f8a5f7fe700 1 -- 172.21.2.101:0/1861442499 >> v1:172.21.2.101:6789/0 conn(0x7f8a600ef610 legacy=0x7f8a600efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:55.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.137 7f8a5f7fe700 1 -- 172.21.2.101:0/1861442499 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a600e52e0 con 0x7f8a601b0470 2021-02-10T11:27:55.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.138 7f8a5f7fe700 1 --2- 172.21.2.101:0/1861442499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a601b0470 0x7f8a601b0870 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f8a48006650 tx=0x7f8a48009a80).ready entity=mon.0 client_cookie=c752db7484436a1f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:55.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.138 7f8a5e7fc700 1 -- 172.21.2.101:0/1861442499 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8a480009b0 con 0x7f8a601b0470 2021-02-10T11:27:55.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.138 7f8a5e7fc700 1 -- 172.21.2.101:0/1861442499 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4800c0c0 con 0x7f8a601b0470 2021-02-10T11:27:55.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.138 7f8a5e7fc700 1 -- 172.21.2.101:0/1861442499 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8a4800c5e0 con 0x7f8a601b0470 2021-02-10T11:27:55.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.138 7f8a65c37700 1 -- 172.21.2.101:0/1861442499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a601b0470 msgr2=0x7f8a601b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:55.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.138 7f8a65c37700 1 --2- 172.21.2.101:0/1861442499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a601b0470 0x7f8a601b0870 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f8a48006650 tx=0x7f8a48009a80).stop 2021-02-10T11:27:55.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.138 7f8a65c37700 1 -- 172.21.2.101:0/1861442499 shutdown_connections 2021-02-10T11:27:55.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.138 7f8a65c37700 1 --2- 172.21.2.101:0/1861442499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a601b0470 0x7f8a601b0870 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:55.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.138 7f8a65c37700 1 -- 172.21.2.101:0/1861442499 >> 172.21.2.101:0/1861442499 conn(0x7f8a600ce3e0 msgr2=0x7f8a600e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:55.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.138 7f8a65c37700 1 -- 172.21.2.101:0/1861442499 shutdown_connections 2021-02-10T11:27:55.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.138 7f8a65c37700 1 -- 172.21.2.101:0/1861442499 wait complete. 2021-02-10T11:27:55.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.153 7f8a65c37700 1 Processor -- start 2021-02-10T11:27:55.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.153 7f8a65c37700 1 -- start start 2021-02-10T11:27:55.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.153 7f8a65c37700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a60087510 0x7f8a600aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:55.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.153 7f8a65c37700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a601c68f0 0x7f8a601b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:55.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.153 7f8a65c37700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a6008c4d0 0x7f8a601b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:55.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.153 7f8a65c37700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a600a2d50 con 0x7f8a601c68f0 2021-02-10T11:27:55.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.153 7f8a65c37700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a600a2f70 con 0x7f8a60087510 2021-02-10T11:27:55.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.153 7f8a65c37700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a600a3190 con 0x7f8a6008c4d0 2021-02-10T11:27:55.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.153 7f8a5effd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a601c68f0 0x7f8a601b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:55.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.153 7f8a5effd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a601c68f0 0x7f8a601b1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48058/0 (socket says 172.21.2.101:48058) 2021-02-10T11:27:55.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.153 7f8a5effd700 1 -- 172.21.2.101:0/4064680333 learned_addr learned my addr 172.21.2.101:0/4064680333 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:27:55.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.154 7f8a5effd700 1 -- 172.21.2.101:0/4064680333 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a6008c4d0 msgr2=0x7f8a601b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:55.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.154 7f8a5effd700 1 --2- 172.21.2.101:0/4064680333 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a6008c4d0 0x7f8a601b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:55.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.154 7f8a5effd700 1 -- 172.21.2.101:0/4064680333 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a60087510 msgr2=0x7f8a600aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:55.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.154 7f8a5effd700 1 --2- 172.21.2.101:0/4064680333 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a60087510 0x7f8a600aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:55.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.154 7f8a5effd700 1 -- 172.21.2.101:0/4064680333 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a600a33b0 con 0x7f8a601c68f0 2021-02-10T11:27:55.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.154 7f8a5effd700 1 --2- 172.21.2.101:0/4064680333 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a601c68f0 0x7f8a601b1f60 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7f8a5000a920 tx=0x7f8a5000acc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:55.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.154 7f8a57fff700 1 -- 172.21.2.101:0/4064680333 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8a500070d0 con 0x7f8a601c68f0 2021-02-10T11:27:55.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.154 7f8a57fff700 1 -- 172.21.2.101:0/4064680333 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a50007390 con 0x7f8a601c68f0 2021-02-10T11:27:55.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.154 7f8a57fff700 1 -- 172.21.2.101:0/4064680333 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8a50004740 con 0x7f8a601c68f0 2021-02-10T11:27:55.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.154 7f8a65c37700 1 -- 172.21.2.101:0/4064680333 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a600a33b0 con 0x7f8a601c68f0 2021-02-10T11:27:55.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.154 7f8a65c37700 1 -- 172.21.2.101:0/4064680333 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a600a33b0 con 0x7f8a601c68f0 2021-02-10T11:27:55.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.155 7f8a57fff700 1 -- 172.21.2.101:0/4064680333 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8a500044c0 con 0x7f8a601c68f0 2021-02-10T11:27:55.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.155 7f8a57fff700 1 --2- 172.21.2.101:0/4064680333 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a44018730 0x7f8a4401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:27:55.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.155 7f8a57fff700 1 -- 172.21.2.101:0/4064680333 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8a500329c0 con 0x7f8a601c68f0 2021-02-10T11:27:55.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.155 7f8a5f7fe700 1 --2- 172.21.2.101:0/4064680333 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a44018730 0x7f8a4401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:27:55.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.155 7f8a5f7fe700 1 --2- 172.21.2.101:0/4064680333 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a44018730 0x7f8a4401ac10 crc :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:27:55.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.187 7f8a65c37700 1 -- 172.21.2.101:0/4064680333 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a6007e6b0 con 0x7f8a601c68f0 2021-02-10T11:27:55.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.189 7f8a57fff700 1 -- 172.21.2.101:0/4064680333 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8a50007370 con 0x7f8a601c68f0 2021-02-10T11:27:55.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.316 7f8a65c37700 1 -- 172.21.2.101:0/4064680333 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839325", "target": ["mgr", ""]}) v1 -- 0x7f8a6007e6b0 con 0x7f8a44018730 2021-02-10T11:27:55.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.319 7f8a57fff700 1 -- 172.21.2.101:0/4064680333 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f8a48006060 con 0x7f8a44018730 2021-02-10T11:27:55.333 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839325/5d0d8249-7032-49fe-b2cf-f226d2c9ce68 2021-02-10T11:27:55.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.324 7f8a65c37700 1 -- 172.21.2.101:0/4064680333 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a44018730 msgr2=0x7f8a4401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:55.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.324 7f8a65c37700 1 --2- 172.21.2.101:0/4064680333 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a44018730 0x7f8a4401ac10 crc :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:55.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.324 7f8a65c37700 1 -- 172.21.2.101:0/4064680333 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a601c68f0 msgr2=0x7f8a601b1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:27:55.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.324 7f8a65c37700 1 --2- 172.21.2.101:0/4064680333 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a601c68f0 0x7f8a601b1f60 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7f8a5000a920 tx=0x7f8a5000acc0).stop 2021-02-10T11:27:55.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.325 7f8a65c37700 1 -- 172.21.2.101:0/4064680333 shutdown_connections 2021-02-10T11:27:55.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.325 7f8a65c37700 1 --2- 172.21.2.101:0/4064680333 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a44018730 0x7f8a4401ac10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:55.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.325 7f8a65c37700 1 --2- 172.21.2.101:0/4064680333 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a6008c4d0 0x7f8a601b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:55.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.325 7f8a65c37700 1 --2- 172.21.2.101:0/4064680333 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a601c68f0 0x7f8a601b1f60 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:27:55.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.325 7f8a65c37700 1 --2- 172.21.2.101:0/4064680333 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a60087510 0x7f8a600aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:27:55.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.325 7f8a65c37700 1 -- 172.21.2.101:0/4064680333 >> 172.21.2.101:0/4064680333 conn(0x7f8a600886d0 msgr2=0x7f8a60081c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:27:55.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.325 7f8a65c37700 1 -- 172.21.2.101:0/4064680333 shutdown_connections 2021-02-10T11:27:55.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:27:55.325 7f8a65c37700 1 -- 172.21.2.101:0/4064680333 wait complete. 2021-02-10T11:27:55.344 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839325 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839325/5d0d8249-7032-49fe-b2cf-f226d2c9ce68 2021-02-10T11:27:55.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:55.344 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:55.344 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:55.680 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:55.680 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:55.681 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839325/5d0d8249-7032-49fe-b2cf-f226d2c9ce68/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:56.062 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:56.063 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:56.063 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244267 s, 42.9 MB/s 2021-02-10T11:27:56.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:56.064 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:56.064 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:56.401 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:56.401 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:56.401 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839325/5d0d8249-7032-49fe-b2cf-f226d2c9ce68/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:56.778 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:56.779 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:56.779 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0255623 s, 41.0 MB/s 2021-02-10T11:27:56.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:56.780 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:56.780 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:57.115 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:57.116 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:57.116 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839325/5d0d8249-7032-49fe-b2cf-f226d2c9ce68/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:57.492 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:57.492 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:57.492 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245188 s, 42.8 MB/s 2021-02-10T11:27:57.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:57.493 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:57.493 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:57.829 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:57.830 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:57.830 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839325/5d0d8249-7032-49fe-b2cf-f226d2c9ce68/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:58.209 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:58.209 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:58.209 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245348 s, 42.7 MB/s 2021-02-10T11:27:58.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:58.210 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:58.210 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:58.547 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:58.547 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:58.548 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839325/5d0d8249-7032-49fe-b2cf-f226d2c9ce68/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:58.919 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:58.919 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:58.919 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231971 s, 45.2 MB/s 2021-02-10T11:27:58.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:58.920 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:58.920 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:59.257 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:59.257 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:59.257 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839325/5d0d8249-7032-49fe-b2cf-f226d2c9ce68/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:27:59.634 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:27:59.634 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:27:59.634 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251743 s, 41.7 MB/s 2021-02-10T11:27:59.635 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:27:59.635 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:27:59.635 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:27:59.972 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:27:59.973 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:27:59.973 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839325/5d0d8249-7032-49fe-b2cf-f226d2c9ce68/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:00.349 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:00.349 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:00.350 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243351 s, 43.1 MB/s 2021-02-10T11:28:00.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:00.351 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:00.352 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:00.688 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:00.688 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:00.689 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839325/5d0d8249-7032-49fe-b2cf-f226d2c9ce68/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:01.063 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:01.063 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:01.063 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243842 s, 43.0 MB/s 2021-02-10T11:28:01.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:01.064 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:01.064 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:01.400 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:01.400 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:01.401 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839325/5d0d8249-7032-49fe-b2cf-f226d2c9ce68/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:01.776 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:01.776 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:01.776 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235535 s, 44.5 MB/s 2021-02-10T11:28:01.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:01.777 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:01.777 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:02.113 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:02.113 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:02.113 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839325/5d0d8249-7032-49fe-b2cf-f226d2c9ce68/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:02.491 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248636 s, 42.2 MB/s 2021-02-10T11:28:02.491 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839326 2021-02-10T11:28:02.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.897 7f409df12700 1 Processor -- start 2021-02-10T11:28:03.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.897 7f409df12700 1 -- start start 2021-02-10T11:28:03.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.897 7f409df12700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f40981b0470 0x7f40981b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:03.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.897 7f409df12700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40980f0b50 con 0x7f40980ef610 2021-02-10T11:28:03.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.897 7f409df12700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40980f24e0 con 0x7f40980f20e0 2021-02-10T11:28:03.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.897 7f409df12700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40980e5ea0 con 0x7f40981b0470 2021-02-10T11:28:03.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.897 7f4097fff700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f40980f20e0 0x7f40980f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:41968/0 (socket says 172.21.2.101:41968) 2021-02-10T11:28:03.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.897 7f40977fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f40981b0470 0x7f40981b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:03.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.897 7f4096ffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f40980ef610 0x7f40980efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48558/0 (socket says 172.21.2.101:48558) 2021-02-10T11:28:03.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.897 7f4097fff700 1 -- 172.21.2.101:0/2947760786 learned_addr learned my addr 172.21.2.101:0/2947760786 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:28:03.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.898 7f40967fc700 1 -- 172.21.2.101:0/2947760786 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102461370 0 0) 0x7f4090000f90 con 0x7f40980f20e0 2021-02-10T11:28:03.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.898 7f40977fe700 1 -- 172.21.2.101:0/2947760786 >> v1:172.21.2.108:6789/0 conn(0x7f40980f20e0 legacy=0x7f40980f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:03.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.898 7f40977fe700 1 -- 172.21.2.101:0/2947760786 >> v1:172.21.2.101:6789/0 conn(0x7f40980ef610 legacy=0x7f40980efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:03.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.898 7f40977fe700 1 -- 172.21.2.101:0/2947760786 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40980e52e0 con 0x7f40981b0470 2021-02-10T11:28:03.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.898 7f40977fe700 1 --2- 172.21.2.101:0/2947760786 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f40981b0470 0x7f40981b0870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f4080006650 tx=0x7f4080009a80).ready entity=mon.2 client_cookie=4145ff9f300b8a83 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:03.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.899 7f40967fc700 1 -- 172.21.2.101:0/2947760786 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f408000c0c0 con 0x7f40981b0470 2021-02-10T11:28:03.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.899 7f40967fc700 1 -- 172.21.2.101:0/2947760786 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408000c340 con 0x7f40981b0470 2021-02-10T11:28:03.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.899 7f40967fc700 1 -- 172.21.2.101:0/2947760786 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f408000c860 con 0x7f40981b0470 2021-02-10T11:28:03.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.899 7f409df12700 1 -- 172.21.2.101:0/2947760786 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f40981b0470 msgr2=0x7f40981b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:03.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.899 7f409df12700 1 --2- 172.21.2.101:0/2947760786 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f40981b0470 0x7f40981b0870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f4080006650 tx=0x7f4080009a80).stop 2021-02-10T11:28:03.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.899 7f409df12700 1 -- 172.21.2.101:0/2947760786 shutdown_connections 2021-02-10T11:28:03.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.899 7f409df12700 1 --2- 172.21.2.101:0/2947760786 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f40981b0470 0x7f40981b0870 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:03.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.899 7f409df12700 1 -- 172.21.2.101:0/2947760786 >> 172.21.2.101:0/2947760786 conn(0x7f40980ce3e0 msgr2=0x7f40980e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:03.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.899 7f409df12700 1 -- 172.21.2.101:0/2947760786 shutdown_connections 2021-02-10T11:28:03.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.899 7f409df12700 1 -- 172.21.2.101:0/2947760786 wait complete. 2021-02-10T11:28:03.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.914 7f409df12700 1 Processor -- start 2021-02-10T11:28:03.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.914 7f409df12700 1 -- start start 2021-02-10T11:28:03.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.914 7f409df12700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f409818ca80 0x7f409818d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:03.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.914 7f409df12700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f409818d6c0 0x7f409818dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:03.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.914 7f409df12700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f409818e6b0 0x7f409818eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:03.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.914 7f409df12700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409819d290 con 0x7f409818e6b0 2021-02-10T11:28:03.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.914 7f409df12700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409818e130 con 0x7f409818ca80 2021-02-10T11:28:03.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.914 7f409df12700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409818c5e0 con 0x7f409818d6c0 2021-02-10T11:28:03.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.915 7f4097fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f409818e6b0 0x7f409818eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:03.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.915 7f4097fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f409818e6b0 0x7f409818eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48072/0 (socket says 172.21.2.101:48072) 2021-02-10T11:28:03.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.915 7f4097fff700 1 -- 172.21.2.101:0/2416637572 learned_addr learned my addr 172.21.2.101:0/2416637572 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:28:03.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.915 7f40977fe700 1 --2- 172.21.2.101:0/2416637572 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f409818ca80 0x7f409818d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:03.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.915 7f4096ffd700 1 --2- 172.21.2.101:0/2416637572 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f409818d6c0 0x7f409818dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:03.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.915 7f4097fff700 1 -- 172.21.2.101:0/2416637572 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f409818d6c0 msgr2=0x7f409818dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:03.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.915 7f4097fff700 1 --2- 172.21.2.101:0/2416637572 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f409818d6c0 0x7f409818dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:03.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.915 7f4097fff700 1 -- 172.21.2.101:0/2416637572 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f409818ca80 msgr2=0x7f409818d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:03.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.915 7f4097fff700 1 --2- 172.21.2.101:0/2416637572 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f409818ca80 0x7f409818d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:03.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.915 7f4097fff700 1 -- 172.21.2.101:0/2416637572 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409818f0d0 con 0x7f409818e6b0 2021-02-10T11:28:03.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.915 7f4097fff700 1 --2- 172.21.2.101:0/2416637572 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f409818e6b0 0x7f409818eab0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f4090007fc0 tx=0x7f409000ac70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:03.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.916 7f408ffff700 1 -- 172.21.2.101:0/2416637572 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f40900070d0 con 0x7f409818e6b0 2021-02-10T11:28:03.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.916 7f408ffff700 1 -- 172.21.2.101:0/2416637572 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4090007310 con 0x7f409818e6b0 2021-02-10T11:28:03.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.916 7f409df12700 1 -- 172.21.2.101:0/2416637572 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409818f0d0 con 0x7f409818e6b0 2021-02-10T11:28:03.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.916 7f409df12700 1 -- 172.21.2.101:0/2416637572 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409818f660 con 0x7f409818e6b0 2021-02-10T11:28:03.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.916 7f408ffff700 1 -- 172.21.2.101:0/2416637572 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4090004b30 con 0x7f409818e6b0 2021-02-10T11:28:03.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.916 7f408ffff700 1 -- 172.21.2.101:0/2416637572 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f409000adb0 con 0x7f409818e6b0 2021-02-10T11:28:03.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.916 7f408ffff700 1 --2- 172.21.2.101:0/2416637572 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f407c0186c0 0x7f407c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:03.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.916 7f408ffff700 1 -- 172.21.2.101:0/2416637572 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f40900328f0 con 0x7f409818e6b0 2021-02-10T11:28:03.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.916 7f40977fe700 1 --2- 172.21.2.101:0/2416637572 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f407c0186c0 0x7f407c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:03.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.917 7f40977fe700 1 --2- 172.21.2.101:0/2416637572 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f407c0186c0 0x7f407c01abc0 crc :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:03.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.947 7f409df12700 1 -- 172.21.2.101:0/2416637572 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4070002fd0 con 0x7f409818e6b0 2021-02-10T11:28:03.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:02.950 7f408ffff700 1 -- 172.21.2.101:0/2416637572 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4090026840 con 0x7f409818e6b0 2021-02-10T11:28:03.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.079 7f409df12700 1 -- 172.21.2.101:0/2416637572 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839326", "target": ["mgr", ""]}) v1 -- 0x7f4070002f80 con 0x7f407c0186c0 2021-02-10T11:28:03.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.106 7f408ffff700 1 -- 172.21.2.101:0/2416637572 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f4080000c90 con 0x7f407c0186c0 2021-02-10T11:28:03.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.112 7f409df12700 1 -- 172.21.2.101:0/2416637572 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f407c0186c0 msgr2=0x7f407c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:03.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.112 7f409df12700 1 --2- 172.21.2.101:0/2416637572 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f407c0186c0 0x7f407c01abc0 crc :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:03.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.112 7f409df12700 1 -- 172.21.2.101:0/2416637572 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f409818e6b0 msgr2=0x7f409818eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:03.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.112 7f409df12700 1 --2- 172.21.2.101:0/2416637572 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f409818e6b0 0x7f409818eab0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f4090007fc0 tx=0x7f409000ac70).stop 2021-02-10T11:28:03.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.112 7f409df12700 1 -- 172.21.2.101:0/2416637572 shutdown_connections 2021-02-10T11:28:03.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.112 7f409df12700 1 --2- 172.21.2.101:0/2416637572 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f407c0186c0 0x7f407c01abc0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:03.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.112 7f409df12700 1 --2- 172.21.2.101:0/2416637572 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f409818d6c0 0x7f409818dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:03.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.112 7f409df12700 1 --2- 172.21.2.101:0/2416637572 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f409818e6b0 0x7f409818eab0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:03.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.112 7f409df12700 1 --2- 172.21.2.101:0/2416637572 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f409818ca80 0x7f409818d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:03.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.112 7f409df12700 1 -- 172.21.2.101:0/2416637572 >> 172.21.2.101:0/2416637572 conn(0x7f40980bf4f0 msgr2=0x7f40980be2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:03.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.113 7f409df12700 1 -- 172.21.2.101:0/2416637572 shutdown_connections 2021-02-10T11:28:03.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.113 7f409df12700 1 -- 172.21.2.101:0/2416637572 wait complete. 2021-02-10T11:28:03.434 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839326 2021-02-10T11:28:03.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.845 7fe08935f700 1 Processor -- start 2021-02-10T11:28:03.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.845 7fe08935f700 1 -- start start 2021-02-10T11:28:03.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.846 7fe08935f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe0841b0470 0x7fe0841b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:03.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.846 7fe08935f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0840ef610 0x7fe0840efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:03.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.846 7fe08935f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe0840f0030 0x7fe0840f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:03.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.846 7fe08935f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0840e5ea0 con 0x7fe0840f0030 2021-02-10T11:28:03.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.846 7fe08935f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0840f0b50 con 0x7fe0841b0470 2021-02-10T11:28:03.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.846 7fe08935f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0840e52e0 con 0x7fe0840ef610 2021-02-10T11:28:03.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.846 7fe08387e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe0840f0030 0x7fe0840f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:03.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.846 7fe08387e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe0840f0030 0x7fe0840f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48076/0 (socket says 172.21.2.101:48076) 2021-02-10T11:28:03.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.846 7fe08387e700 1 -- 172.21.2.101:0/2252324111 learned_addr learned my addr 172.21.2.101:0/2252324111 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:28:03.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.846 7fe08387e700 1 -- 172.21.2.101:0/2252324111 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0840ef610 msgr2=0x7fe0840efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:28:03.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.846 7fe08387e700 1 --2- 172.21.2.101:0/2252324111 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0840ef610 0x7fe0840efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:03.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.846 7fe08387e700 1 -- 172.21.2.101:0/2252324111 >> v2:172.21.2.108:3300/0 conn(0x7fe0841b0470 msgr2=0x7fe0841b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:28:03.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.846 7fe08387e700 1 --2- 172.21.2.101:0/2252324111 >> v2:172.21.2.108:3300/0 conn(0x7fe0841b0470 0x7fe0841b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:03.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.846 7fe08387e700 1 -- 172.21.2.101:0/2252324111 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0840f10d0 con 0x7fe0840f0030 2021-02-10T11:28:03.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.847 7fe08387e700 1 --2- 172.21.2.101:0/2252324111 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0840f0030 0x7fe0840f0430 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7fe07c006f00 tx=0x7fe07c009f10).ready entity=mon.0 client_cookie=22b500b727b080cc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:03.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.847 7fe08207b700 1 -- 172.21.2.101:0/2252324111 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe07c00b3b0 con 0x7fe0840f0030 2021-02-10T11:28:03.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.847 7fe08207b700 1 -- 172.21.2.101:0/2252324111 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c00b730 con 0x7fe0840f0030 2021-02-10T11:28:03.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.847 7fe08207b700 1 -- 172.21.2.101:0/2252324111 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe07c00bc50 con 0x7fe0840f0030 2021-02-10T11:28:03.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.847 7fe08935f700 1 -- 172.21.2.101:0/2252324111 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0840f0030 msgr2=0x7fe0840f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:03.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.847 7fe08935f700 1 --2- 172.21.2.101:0/2252324111 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0840f0030 0x7fe0840f0430 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7fe07c006f00 tx=0x7fe07c009f10).stop 2021-02-10T11:28:03.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.847 7fe08935f700 1 -- 172.21.2.101:0/2252324111 shutdown_connections 2021-02-10T11:28:03.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.847 7fe08935f700 1 --2- 172.21.2.101:0/2252324111 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0840ef610 0x7fe0840efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:03.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.847 7fe08935f700 1 --2- 172.21.2.101:0/2252324111 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0840f0030 0x7fe0840f0430 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:03.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.847 7fe08935f700 1 --2- 172.21.2.101:0/2252324111 >> v2:172.21.2.108:3300/0 conn(0x7fe0841b0470 0x7fe0841b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:03.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.847 7fe08935f700 1 -- 172.21.2.101:0/2252324111 >> 172.21.2.101:0/2252324111 conn(0x7fe0840ce3e0 msgr2=0x7fe0840e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:03.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.847 7fe08935f700 1 -- 172.21.2.101:0/2252324111 shutdown_connections 2021-02-10T11:28:03.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.847 7fe08935f700 1 -- 172.21.2.101:0/2252324111 wait complete. 2021-02-10T11:28:03.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.862 7fe08935f700 1 Processor -- start 2021-02-10T11:28:03.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.862 7fe08935f700 1 -- start start 2021-02-10T11:28:03.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.863 7fe08935f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe084193230 0x7fe0840a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:03.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.863 7fe08935f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe08408f5b0 0x7fe08408f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:03.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.863 7fe08935f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe08408e6a0 0x7fe08408eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:03.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.863 7fe08935f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08408ffd0 con 0x7fe084193230 2021-02-10T11:28:03.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.863 7fe08935f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe084090240 con 0x7fe08408e6a0 2021-02-10T11:28:03.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.863 7fe08935f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe08408f0c0 con 0x7fe08408f5b0 2021-02-10T11:28:03.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.863 7fe08307d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe084193230 0x7fe0840a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:03.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.863 7fe08307d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe084193230 0x7fe0840a6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48080/0 (socket says 172.21.2.101:48080) 2021-02-10T11:28:03.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.863 7fe08307d700 1 -- 172.21.2.101:0/4113609423 learned_addr learned my addr 172.21.2.101:0/4113609423 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:28:03.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.863 7fe08307d700 1 -- 172.21.2.101:0/4113609423 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe08408f5b0 msgr2=0x7fe08408f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:03.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.863 7fe08307d700 1 --2- 172.21.2.101:0/4113609423 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe08408f5b0 0x7fe08408f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:03.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.863 7fe08307d700 1 -- 172.21.2.101:0/4113609423 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe08408e6a0 msgr2=0x7fe08408eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:03.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.863 7fe08307d700 1 --2- 172.21.2.101:0/4113609423 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe08408e6a0 0x7fe08408eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:03.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.863 7fe08307d700 1 -- 172.21.2.101:0/4113609423 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08408f2a0 con 0x7fe084193230 2021-02-10T11:28:03.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.863 7fe08307d700 1 --2- 172.21.2.101:0/4113609423 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe084193230 0x7fe0840a6a80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7fe07400b530 tx=0x7fe07400b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:03.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.863 7fe07bfff700 1 -- 172.21.2.101:0/4113609423 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe07400bcc0 con 0x7fe084193230 2021-02-10T11:28:03.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.864 7fe07bfff700 1 -- 172.21.2.101:0/4113609423 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe074000c90 con 0x7fe084193230 2021-02-10T11:28:03.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.864 7fe08935f700 1 -- 172.21.2.101:0/4113609423 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe08408f2a0 con 0x7fe084193230 2021-02-10T11:28:03.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.864 7fe07bfff700 1 -- 172.21.2.101:0/4113609423 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe0740022a0 con 0x7fe084193230 2021-02-10T11:28:03.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.864 7fe08935f700 1 -- 172.21.2.101:0/4113609423 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0841924f0 con 0x7fe084193230 2021-02-10T11:28:03.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.864 7fe07bfff700 1 -- 172.21.2.101:0/4113609423 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe074005310 con 0x7fe084193230 2021-02-10T11:28:03.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.864 7fe07bfff700 1 --2- 172.21.2.101:0/4113609423 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0680186c0 0x7fe06801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:03.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.864 7fe07bfff700 1 -- 172.21.2.101:0/4113609423 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe0740323f0 con 0x7fe084193230 2021-02-10T11:28:03.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.864 7fe08287c700 1 --2- 172.21.2.101:0/4113609423 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0680186c0 0x7fe06801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:03.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.865 7fe08287c700 1 --2- 172.21.2.101:0/4113609423 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0680186c0 0x7fe06801abc0 crc :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:03.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.895 7fe08935f700 1 -- 172.21.2.101:0/4113609423 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0840f0b50 con 0x7fe084193230 2021-02-10T11:28:03.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:03.898 7fe07bfff700 1 -- 172.21.2.101:0/4113609423 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe074000960 con 0x7fe084193230 2021-02-10T11:28:04.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:04.027 7fe08935f700 1 -- 172.21.2.101:0/4113609423 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839326", "target": ["mgr", ""]}) v1 -- 0x7fe08408f2a0 con 0x7fe0680186c0 2021-02-10T11:28:04.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:04.031 7fe07bfff700 1 -- 172.21.2.101:0/4113609423 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fe06c006e40 con 0x7fe0680186c0 2021-02-10T11:28:04.041 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839326/4686cb6b-b04d-416d-ac07-79fc893ae5ca 2021-02-10T11:28:04.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:04.032 7fe08935f700 1 -- 172.21.2.101:0/4113609423 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0680186c0 msgr2=0x7fe06801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:04.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:04.032 7fe08935f700 1 --2- 172.21.2.101:0/4113609423 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0680186c0 0x7fe06801abc0 crc :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:04.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:04.032 7fe08935f700 1 -- 172.21.2.101:0/4113609423 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe084193230 msgr2=0x7fe0840a6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:04.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:04.032 7fe08935f700 1 --2- 172.21.2.101:0/4113609423 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe084193230 0x7fe0840a6a80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7fe07400b530 tx=0x7fe07400b8d0).stop 2021-02-10T11:28:04.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:04.032 7fe08935f700 1 -- 172.21.2.101:0/4113609423 shutdown_connections 2021-02-10T11:28:04.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:04.032 7fe08935f700 1 --2- 172.21.2.101:0/4113609423 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0680186c0 0x7fe06801abc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:04.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:04.032 7fe08935f700 1 --2- 172.21.2.101:0/4113609423 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe08408f5b0 0x7fe08408f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:04.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:04.032 7fe08935f700 1 --2- 172.21.2.101:0/4113609423 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe08408e6a0 0x7fe08408eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:04.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:04.032 7fe08935f700 1 --2- 172.21.2.101:0/4113609423 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe084193230 0x7fe0840a6a80 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:04.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:04.032 7fe08935f700 1 -- 172.21.2.101:0/4113609423 >> 172.21.2.101:0/4113609423 conn(0x7fe0840bf4e0 msgr2=0x7fe0840be160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:04.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:04.032 7fe08935f700 1 -- 172.21.2.101:0/4113609423 shutdown_connections 2021-02-10T11:28:04.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:04.033 7fe08935f700 1 -- 172.21.2.101:0/4113609423 wait complete. 2021-02-10T11:28:04.051 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839326 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839326/4686cb6b-b04d-416d-ac07-79fc893ae5ca 2021-02-10T11:28:04.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:04.051 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:04.051 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:04.387 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:04.388 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:04.388 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839326/4686cb6b-b04d-416d-ac07-79fc893ae5ca/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:04.769 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:04.769 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:04.770 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0230281 s, 45.5 MB/s 2021-02-10T11:28:04.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:04.771 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:04.771 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:05.107 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:05.107 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:05.108 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839326/4686cb6b-b04d-416d-ac07-79fc893ae5ca/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:05.484 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:05.484 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:05.484 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246151 s, 42.6 MB/s 2021-02-10T11:28:05.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:05.486 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:05.486 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:05.822 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:05.822 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:05.822 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839326/4686cb6b-b04d-416d-ac07-79fc893ae5ca/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:06.196 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:06.197 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:06.197 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0225353 s, 46.5 MB/s 2021-02-10T11:28:06.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:06.198 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:06.198 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:06.533 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:06.533 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:06.534 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839326/4686cb6b-b04d-416d-ac07-79fc893ae5ca/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:06.907 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:06.908 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:06.908 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0227097 s, 46.2 MB/s 2021-02-10T11:28:06.909 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:06.909 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:06.909 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:07.245 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:07.245 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:07.246 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839326/4686cb6b-b04d-416d-ac07-79fc893ae5ca/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:07.622 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:07.622 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:07.622 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234413 s, 44.7 MB/s 2021-02-10T11:28:07.623 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:07.623 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:07.623 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:07.958 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:07.959 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:07.959 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839326/4686cb6b-b04d-416d-ac07-79fc893ae5ca/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:08.333 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:08.334 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:08.334 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0223559 s, 46.9 MB/s 2021-02-10T11:28:08.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:08.335 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:08.335 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:08.686 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:08.687 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:08.687 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839326/4686cb6b-b04d-416d-ac07-79fc893ae5ca/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:09.060 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:09.060 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:09.060 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233141 s, 45.0 MB/s 2021-02-10T11:28:09.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:09.061 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:09.062 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:09.396 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:09.397 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:09.397 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839326/4686cb6b-b04d-416d-ac07-79fc893ae5ca/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:09.773 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:09.774 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:09.774 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0257162 s, 40.8 MB/s 2021-02-10T11:28:09.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:09.775 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:09.775 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:10.111 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:10.111 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:10.111 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839326/4686cb6b-b04d-416d-ac07-79fc893ae5ca/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:10.485 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:10.486 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:10.486 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235642 s, 44.5 MB/s 2021-02-10T11:28:10.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:10.487 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:10.487 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:10.822 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:10.823 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:10.823 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839326/4686cb6b-b04d-416d-ac07-79fc893ae5ca/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:11.200 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:11.200 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:11.200 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249456 s, 42.0 MB/s 2021-02-10T11:28:11.201 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839327 2021-02-10T11:28:11.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.605 7f68620b8700 1 Processor -- start 2021-02-10T11:28:11.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.605 7f68620b8700 1 -- start start 2021-02-10T11:28:11.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.606 7f68620b8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f685c1b0870 0x7f685c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:11.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.606 7f68620b8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f685c0efe00 0x7f685c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:11.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.606 7f68620b8700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685c1b0c70 con 0x7f685c1b0470 2021-02-10T11:28:11.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.606 7f68620b8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685c0f0b50 con 0x7f685c1b0870 2021-02-10T11:28:11.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.606 7f68620b8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0f0820 con 0x7f685c0efe00 2021-02-10T11:28:11.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.606 7f685b7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f685c1b0470 0x7f685c0ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48590/0 (socket says 172.21.2.101:48590) 2021-02-10T11:28:11.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.606 7f685b7fe700 1 -- 172.21.2.101:0/3985659327 learned_addr learned my addr 172.21.2.101:0/3985659327 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:28:11.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.606 7f685bfff700 1 --2- 172.21.2.101:0/3985659327 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f685c0efe00 0x7f685c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:11.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.606 7f685affd700 1 --2- 172.21.2.101:0/3985659327 >> v2:172.21.2.108:3300/0 conn(0x7f685c1b0870 0x7f685c0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:11.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.606 7f685a7fc700 1 -- 172.21.2.101:0/3985659327 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550753837 0 0) 0x7f684c0010b0 con 0x7f685c1b0470 2021-02-10T11:28:11.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.606 7f685a7fc700 1 -- 172.21.2.101:0/3985659327 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6848002fe0 con 0x7f685c1b0470 2021-02-10T11:28:11.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.607 7f685bfff700 1 -- 172.21.2.101:0/3985659327 >> v2:172.21.2.108:3300/0 conn(0x7f685c1b0870 msgr2=0x7f685c0f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:28:11.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.607 7f685bfff700 1 --2- 172.21.2.101:0/3985659327 >> v2:172.21.2.108:3300/0 conn(0x7f685c1b0870 0x7f685c0f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:11.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.607 7f685bfff700 1 -- 172.21.2.101:0/3985659327 >> v1:172.21.2.101:6789/0 conn(0x7f685c1b0470 legacy=0x7f685c0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:11.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.607 7f685bfff700 1 -- 172.21.2.101:0/3985659327 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c0e52e0 con 0x7f685c0efe00 2021-02-10T11:28:11.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.607 7f685affd700 1 --2- 172.21.2.101:0/3985659327 >> v2:172.21.2.108:3300/0 conn(0x7f685c1b0870 0x7f685c0f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:28:11.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.607 7f685bfff700 1 --2- 172.21.2.101:0/3985659327 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f685c0efe00 0x7f685c0f0200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f6854006e50 tx=0x7f6854009e70).ready entity=mon.2 client_cookie=f8ee93bf0a9543b0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:11.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.607 7f685a7fc700 1 -- 172.21.2.101:0/3985659327 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f685400b300 con 0x7f685c0efe00 2021-02-10T11:28:11.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.607 7f685a7fc700 1 -- 172.21.2.101:0/3985659327 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6854003030 con 0x7f685c0efe00 2021-02-10T11:28:11.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.607 7f685a7fc700 1 -- 172.21.2.101:0/3985659327 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6854000f90 con 0x7f685c0efe00 2021-02-10T11:28:11.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.607 7f68620b8700 1 -- 172.21.2.101:0/3985659327 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f685c0efe00 msgr2=0x7f685c0f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:11.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.607 7f68620b8700 1 --2- 172.21.2.101:0/3985659327 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f685c0efe00 0x7f685c0f0200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f6854006e50 tx=0x7f6854009e70).stop 2021-02-10T11:28:11.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.608 7f68620b8700 1 -- 172.21.2.101:0/3985659327 shutdown_connections 2021-02-10T11:28:11.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.608 7f68620b8700 1 --2- 172.21.2.101:0/3985659327 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f685c0efe00 0x7f685c0f0200 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:11.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.608 7f68620b8700 1 --2- 172.21.2.101:0/3985659327 >> v2:172.21.2.108:3300/0 conn(0x7f685c1b0870 0x7f685c0f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:11.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.608 7f68620b8700 1 -- 172.21.2.101:0/3985659327 >> 172.21.2.101:0/3985659327 conn(0x7f685c0ce3e0 msgr2=0x7f685c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:11.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.608 7f68620b8700 1 -- 172.21.2.101:0/3985659327 shutdown_connections 2021-02-10T11:28:11.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.608 7f68620b8700 1 -- 172.21.2.101:0/3985659327 wait complete. 2021-02-10T11:28:11.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.622 7f68620b8700 1 Processor -- start 2021-02-10T11:28:11.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.622 7f68620b8700 1 -- start start 2021-02-10T11:28:11.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.623 7f68620b8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f685c09a2e0 0x7f685c19fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:11.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.623 7f68620b8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f685c09a6e0 0x7f685c1062f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:11.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.623 7f68620b8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f685c106910 0x7f685c0bdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:11.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.623 7f68620b8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c18bde0 con 0x7f685c09a2e0 2021-02-10T11:28:11.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.623 7f68620b8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c099d60 con 0x7f685c106910 2021-02-10T11:28:11.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.623 7f68620b8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c1a04f0 con 0x7f685c09a6e0 2021-02-10T11:28:11.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.623 7f685b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f685c09a2e0 0x7f685c19fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:11.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.623 7f685b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f685c09a2e0 0x7f685c19fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48104/0 (socket says 172.21.2.101:48104) 2021-02-10T11:28:11.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.623 7f685b7fe700 1 -- 172.21.2.101:0/1295779742 learned_addr learned my addr 172.21.2.101:0/1295779742 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:28:11.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.623 7f685bfff700 1 --2- 172.21.2.101:0/1295779742 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f685c106910 0x7f685c0bdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:11.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.623 7f685affd700 1 --2- 172.21.2.101:0/1295779742 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f685c09a6e0 0x7f685c1062f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:11.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.624 7f685bfff700 1 -- 172.21.2.101:0/1295779742 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f685c09a6e0 msgr2=0x7f685c1062f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:11.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.624 7f685bfff700 1 --2- 172.21.2.101:0/1295779742 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f685c09a6e0 0x7f685c1062f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:11.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.624 7f685bfff700 1 -- 172.21.2.101:0/1295779742 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f685c09a2e0 msgr2=0x7f685c19fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:11.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.624 7f685bfff700 1 --2- 172.21.2.101:0/1295779742 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f685c09a2e0 0x7f685c19fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:11.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.624 7f685bfff700 1 -- 172.21.2.101:0/1295779742 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c1a06d0 con 0x7f685c106910 2021-02-10T11:28:11.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.624 7f685bfff700 1 --2- 172.21.2.101:0/1295779742 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f685c106910 0x7f685c0bdc70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f6854009970 tx=0x7f685400bc20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:11.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.624 7f6853fff700 1 -- 172.21.2.101:0/1295779742 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6854001070 con 0x7f685c106910 2021-02-10T11:28:11.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.624 7f6853fff700 1 -- 172.21.2.101:0/1295779742 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68540013b0 con 0x7f685c106910 2021-02-10T11:28:11.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.624 7f6853fff700 1 -- 172.21.2.101:0/1295779742 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f68540018d0 con 0x7f685c106910 2021-02-10T11:28:11.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.624 7f68620b8700 1 -- 172.21.2.101:0/1295779742 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685c1a06d0 con 0x7f685c106910 2021-02-10T11:28:11.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.624 7f68620b8700 1 -- 172.21.2.101:0/1295779742 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685c0be580 con 0x7f685c106910 2021-02-10T11:28:11.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.625 7f6853fff700 1 -- 172.21.2.101:0/1295779742 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6854000f90 con 0x7f685c106910 2021-02-10T11:28:11.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.625 7f6853fff700 1 --2- 172.21.2.101:0/1295779742 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6840018730 0x7f684001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:11.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.625 7f6853fff700 1 -- 172.21.2.101:0/1295779742 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6854032a60 con 0x7f685c106910 2021-02-10T11:28:11.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.625 7f685b7fe700 1 --2- 172.21.2.101:0/1295779742 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6840018730 0x7f684001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:11.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.625 7f685b7fe700 1 --2- 172.21.2.101:0/1295779742 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6840018730 0x7f684001abc0 crc :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:11.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.656 7f68620b8700 1 -- 172.21.2.101:0/1295779742 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685c1057f0 con 0x7f685c106910 2021-02-10T11:28:11.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.659 7f6853fff700 1 -- 172.21.2.101:0/1295779742 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f685400bd60 con 0x7f685c106910 2021-02-10T11:28:11.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.785 7f68620b8700 1 -- 172.21.2.101:0/1295779742 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839327", "target": ["mgr", ""]}) v1 -- 0x7f685c1057f0 con 0x7f6840018730 2021-02-10T11:28:11.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.812 7f6853fff700 1 -- 172.21.2.101:0/1295779742 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f684c001560 con 0x7f6840018730 2021-02-10T11:28:11.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.818 7f68620b8700 1 -- 172.21.2.101:0/1295779742 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6840018730 msgr2=0x7f684001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:11.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.818 7f68620b8700 1 --2- 172.21.2.101:0/1295779742 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6840018730 0x7f684001abc0 crc :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:11.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.818 7f68620b8700 1 -- 172.21.2.101:0/1295779742 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f685c106910 msgr2=0x7f685c0bdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:11.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.818 7f68620b8700 1 --2- 172.21.2.101:0/1295779742 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f685c106910 0x7f685c0bdc70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f6854009970 tx=0x7f685400bc20).stop 2021-02-10T11:28:11.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.818 7f68620b8700 1 -- 172.21.2.101:0/1295779742 shutdown_connections 2021-02-10T11:28:11.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.818 7f68620b8700 1 --2- 172.21.2.101:0/1295779742 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6840018730 0x7f684001abc0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:11.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.818 7f68620b8700 1 --2- 172.21.2.101:0/1295779742 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f685c09a6e0 0x7f685c1062f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:11.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.818 7f68620b8700 1 --2- 172.21.2.101:0/1295779742 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f685c106910 0x7f685c0bdc70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:11.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.818 7f68620b8700 1 --2- 172.21.2.101:0/1295779742 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f685c09a2e0 0x7f685c19fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:11.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.818 7f68620b8700 1 -- 172.21.2.101:0/1295779742 >> 172.21.2.101:0/1295779742 conn(0x7f685c09dfd0 msgr2=0x7f685c09abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:11.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.818 7f68620b8700 1 -- 172.21.2.101:0/1295779742 shutdown_connections 2021-02-10T11:28:11.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:11.818 7f68620b8700 1 -- 172.21.2.101:0/1295779742 wait complete. 2021-02-10T11:28:11.837 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839327 2021-02-10T11:28:12.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.242 7f53cc36a700 1 Processor -- start 2021-02-10T11:28:12.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.242 7f53cc36a700 1 -- start start 2021-02-10T11:28:12.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.242 7f53cc36a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f53c41b0870 0x7f53c40f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:12.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.242 7f53cc36a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f53c40efe00 0x7f53c40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:12.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.242 7f53cc36a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c41b0c70 con 0x7f53c41b0470 2021-02-10T11:28:12.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.242 7f53cc36a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c40e5ea0 con 0x7f53c41b0870 2021-02-10T11:28:12.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.242 7f53cc36a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c40f0b50 con 0x7f53c40efe00 2021-02-10T11:28:12.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.243 7f53c9985700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f53c41b0870 0x7f53c40f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:12.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.243 7f53c9985700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f53c41b0870 0x7f53c40f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48110/0 (socket says 172.21.2.101:48110) 2021-02-10T11:28:12.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.243 7f53c9985700 1 -- 172.21.2.101:0/3050836412 learned_addr learned my addr 172.21.2.101:0/3050836412 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:28:12.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.243 7f53c9985700 1 -- 172.21.2.101:0/3050836412 >> v2:172.21.2.108:3300/0 conn(0x7f53c40efe00 msgr2=0x7f53c40f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:28:12.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.243 7f53c9985700 1 --2- 172.21.2.101:0/3050836412 >> v2:172.21.2.108:3300/0 conn(0x7f53c40efe00 0x7f53c40f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:12.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.243 7f53c9985700 1 -- 172.21.2.101:0/3050836412 >> v1:172.21.2.108:6789/0 conn(0x7f53c41b0470 legacy=0x7f53c40ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:28:12.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.243 7f53c9985700 1 -- 172.21.2.101:0/3050836412 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c40f0820 con 0x7f53c41b0870 2021-02-10T11:28:12.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.243 7f53c9985700 1 --2- 172.21.2.101:0/3050836412 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53c41b0870 0x7f53c40f20e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f53b8006700 tx=0x7f53b8009b80).ready entity=mon.0 client_cookie=5e239ae16e8d90d8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:12.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.243 7f53c9184700 1 -- 172.21.2.101:0/3050836412 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f53b800c0c0 con 0x7f53c41b0870 2021-02-10T11:28:12.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.243 7f53c9184700 1 -- 172.21.2.101:0/3050836412 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b800c400 con 0x7f53c41b0870 2021-02-10T11:28:12.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.243 7f53c9184700 1 -- 172.21.2.101:0/3050836412 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f53b800c860 con 0x7f53c41b0870 2021-02-10T11:28:12.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.243 7f53cc36a700 1 -- 172.21.2.101:0/3050836412 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53c41b0870 msgr2=0x7f53c40f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:12.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.243 7f53cc36a700 1 --2- 172.21.2.101:0/3050836412 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53c41b0870 0x7f53c40f20e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f53b8006700 tx=0x7f53b8009b80).stop 2021-02-10T11:28:12.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.244 7f53cc36a700 1 -- 172.21.2.101:0/3050836412 shutdown_connections 2021-02-10T11:28:12.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.244 7f53cc36a700 1 --2- 172.21.2.101:0/3050836412 >> v2:172.21.2.108:3300/0 conn(0x7f53c40efe00 0x7f53c40f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:12.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.244 7f53cc36a700 1 --2- 172.21.2.101:0/3050836412 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53c41b0870 0x7f53c40f20e0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:12.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.244 7f53cc36a700 1 -- 172.21.2.101:0/3050836412 >> 172.21.2.101:0/3050836412 conn(0x7f53c40ce3e0 msgr2=0x7f53c40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:12.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.244 7f53cc36a700 1 -- 172.21.2.101:0/3050836412 shutdown_connections 2021-02-10T11:28:12.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.244 7f53cc36a700 1 -- 172.21.2.101:0/3050836412 wait complete. 2021-02-10T11:28:12.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.259 7f53cc36a700 1 Processor -- start 2021-02-10T11:28:12.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.259 7f53cc36a700 1 -- start start 2021-02-10T11:28:12.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.259 7f53cc36a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53c40c36a0 0x7f53c40c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:12.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.259 7f53cc36a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53c40c3190 0x7f53c40b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:12.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.259 7f53cc36a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53c40b2370 0x7f53c40b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:12.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.259 7f53cc36a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c40b27c0 con 0x7f53c40b2370 2021-02-10T11:28:12.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.259 7f53cc36a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c40b3360 con 0x7f53c40c36a0 2021-02-10T11:28:12.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.259 7f53cc36a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c40b3540 con 0x7f53c40c3190 2021-02-10T11:28:12.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.260 7f53ca987700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53c40b2370 0x7f53c40b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:12.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.260 7f53ca987700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53c40b2370 0x7f53c40b2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48120/0 (socket says 172.21.2.101:48120) 2021-02-10T11:28:12.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.260 7f53ca987700 1 -- 172.21.2.101:0/2236884773 learned_addr learned my addr 172.21.2.101:0/2236884773 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:28:12.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.260 7f53ca987700 1 -- 172.21.2.101:0/2236884773 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53c40c3190 msgr2=0x7f53c40b1d50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:28:12.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.260 7f53ca987700 1 --2- 172.21.2.101:0/2236884773 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53c40c3190 0x7f53c40b1d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:12.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.260 7f53ca987700 1 -- 172.21.2.101:0/2236884773 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53c40c36a0 msgr2=0x7f53c40c2b70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:28:12.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.260 7f53ca987700 1 --2- 172.21.2.101:0/2236884773 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53c40c36a0 0x7f53c40c2b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:12.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.260 7f53ca987700 1 -- 172.21.2.101:0/2236884773 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c40b3760 con 0x7f53c40b2370 2021-02-10T11:28:12.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.260 7f53ca987700 1 --2- 172.21.2.101:0/2236884773 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53c40b2370 0x7f53c40b2d40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f53bc009980 tx=0x7f53bc00bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:12.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.260 7f53b6ffd700 1 -- 172.21.2.101:0/2236884773 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f53bc005870 con 0x7f53c40b2370 2021-02-10T11:28:12.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.260 7f53b6ffd700 1 -- 172.21.2.101:0/2236884773 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc005b30 con 0x7f53c40b2370 2021-02-10T11:28:12.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.260 7f53cc36a700 1 -- 172.21.2.101:0/2236884773 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c40b3760 con 0x7f53c40b2370 2021-02-10T11:28:12.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.260 7f53b6ffd700 1 -- 172.21.2.101:0/2236884773 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f53bc006030 con 0x7f53c40b2370 2021-02-10T11:28:12.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.260 7f53cc36a700 1 -- 172.21.2.101:0/2236884773 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c40b3cf0 con 0x7f53c40b2370 2021-02-10T11:28:12.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.261 7f53b6ffd700 1 -- 172.21.2.101:0/2236884773 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f53bc006030 con 0x7f53c40b2370 2021-02-10T11:28:12.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.261 7f53b6ffd700 1 --2- 172.21.2.101:0/2236884773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f53ac018730 0x7f53ac01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:12.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.261 7f53b6ffd700 1 -- 172.21.2.101:0/2236884773 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f53bc032b50 con 0x7f53c40b2370 2021-02-10T11:28:12.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.261 7f53ca186700 1 --2- 172.21.2.101:0/2236884773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f53ac018730 0x7f53ac01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:12.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.261 7f53ca186700 1 --2- 172.21.2.101:0/2236884773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f53ac018730 0x7f53ac01ac10 crc :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:12.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.292 7f53cc36a700 1 -- 172.21.2.101:0/2236884773 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c40edd10 con 0x7f53c40b2370 2021-02-10T11:28:12.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.295 7f53b6ffd700 1 -- 172.21.2.101:0/2236884773 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f53bc005d90 con 0x7f53c40b2370 2021-02-10T11:28:12.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.425 7f53cc36a700 1 -- 172.21.2.101:0/2236884773 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839327", "target": ["mgr", ""]}) v1 -- 0x7f53c40b3cf0 con 0x7f53ac018730 2021-02-10T11:28:12.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.428 7f53b6ffd700 1 -- 172.21.2.101:0/2236884773 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f53c0006970 con 0x7f53ac018730 2021-02-10T11:28:12.442 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839327/170b2a17-45b3-48cc-87fc-4a6115c8f6dc 2021-02-10T11:28:12.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.433 7f53cc36a700 1 -- 172.21.2.101:0/2236884773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f53ac018730 msgr2=0x7f53ac01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:12.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.433 7f53cc36a700 1 --2- 172.21.2.101:0/2236884773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f53ac018730 0x7f53ac01ac10 crc :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:12.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.433 7f53cc36a700 1 -- 172.21.2.101:0/2236884773 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53c40b2370 msgr2=0x7f53c40b2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:12.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.433 7f53cc36a700 1 --2- 172.21.2.101:0/2236884773 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53c40b2370 0x7f53c40b2d40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f53bc009980 tx=0x7f53bc00bcd0).stop 2021-02-10T11:28:12.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.433 7f53cc36a700 1 -- 172.21.2.101:0/2236884773 shutdown_connections 2021-02-10T11:28:12.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.433 7f53cc36a700 1 --2- 172.21.2.101:0/2236884773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f53ac018730 0x7f53ac01ac10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:12.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.433 7f53cc36a700 1 --2- 172.21.2.101:0/2236884773 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53c40c3190 0x7f53c40b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:12.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.433 7f53cc36a700 1 --2- 172.21.2.101:0/2236884773 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53c40b2370 0x7f53c40b2d40 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:12.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.433 7f53cc36a700 1 --2- 172.21.2.101:0/2236884773 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53c40c36a0 0x7f53c40c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:12.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.433 7f53cc36a700 1 -- 172.21.2.101:0/2236884773 >> 172.21.2.101:0/2236884773 conn(0x7f53c409f630 msgr2=0x7f53c40a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:12.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.433 7f53cc36a700 1 -- 172.21.2.101:0/2236884773 shutdown_connections 2021-02-10T11:28:12.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:12.433 7f53cc36a700 1 -- 172.21.2.101:0/2236884773 wait complete. 2021-02-10T11:28:12.452 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839327 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839327/170b2a17-45b3-48cc-87fc-4a6115c8f6dc 2021-02-10T11:28:12.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:12.452 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:12.452 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:12.789 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:12.789 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:12.790 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839327/170b2a17-45b3-48cc-87fc-4a6115c8f6dc/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:13.172 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:13.172 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:13.172 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243126 s, 43.1 MB/s 2021-02-10T11:28:13.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:13.173 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:13.173 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:13.510 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:13.510 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:13.511 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839327/170b2a17-45b3-48cc-87fc-4a6115c8f6dc/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:13.884 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:13.884 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:13.884 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242737 s, 43.2 MB/s 2021-02-10T11:28:13.885 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:13.885 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:13.885 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:14.219 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:14.219 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:14.220 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839327/170b2a17-45b3-48cc-87fc-4a6115c8f6dc/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:14.596 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:14.596 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:14.596 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242129 s, 43.3 MB/s 2021-02-10T11:28:14.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:14.597 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:14.597 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:14.932 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:14.932 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:14.932 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839327/170b2a17-45b3-48cc-87fc-4a6115c8f6dc/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:15.310 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:15.310 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:15.310 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0256476 s, 40.9 MB/s 2021-02-10T11:28:15.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:15.311 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:15.311 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:15.647 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:15.648 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:15.648 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839327/170b2a17-45b3-48cc-87fc-4a6115c8f6dc/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:16.023 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:16.023 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:16.023 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0227247 s, 46.1 MB/s 2021-02-10T11:28:16.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:16.025 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:16.025 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:16.359 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:16.360 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:16.360 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839327/170b2a17-45b3-48cc-87fc-4a6115c8f6dc/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:16.733 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:16.734 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:16.734 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0226132 s, 46.4 MB/s 2021-02-10T11:28:16.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:16.735 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:16.735 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:17.071 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:17.071 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:17.071 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839327/170b2a17-45b3-48cc-87fc-4a6115c8f6dc/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:17.448 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:17.448 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:17.448 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0265202 s, 39.5 MB/s 2021-02-10T11:28:17.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:17.449 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:17.449 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:17.783 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:17.783 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:17.783 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839327/170b2a17-45b3-48cc-87fc-4a6115c8f6dc/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:18.159 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:18.159 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:18.159 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243004 s, 43.2 MB/s 2021-02-10T11:28:18.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:18.160 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:18.160 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:18.496 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:18.496 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:18.496 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839327/170b2a17-45b3-48cc-87fc-4a6115c8f6dc/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:18.870 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:18.871 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:18.871 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251847 s, 41.6 MB/s 2021-02-10T11:28:18.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:18.872 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:18.872 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:19.206 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:19.206 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:19.207 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839327/170b2a17-45b3-48cc-87fc-4a6115c8f6dc/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:19.583 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:19.583 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:19.584 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239211 s, 43.8 MB/s 2021-02-10T11:28:19.584 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839328 2021-02-10T11:28:20.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.989 7fd178d6d700 1 Processor -- start 2021-02-10T11:28:20.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.989 7fd178d6d700 1 -- start start 2021-02-10T11:28:20.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.990 7fd178d6d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd1741b0470 0x7fd1741b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:20.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.990 7fd178d6d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd1740f20e0 0x7fd1740f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:20.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.990 7fd178d6d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1740f0b50 con 0x7fd1740ef610 2021-02-10T11:28:20.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.990 7fd178d6d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1740f0820 con 0x7fd1740f20e0 2021-02-10T11:28:20.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.990 7fd178d6d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1740f24e0 con 0x7fd1741b0470 2021-02-10T11:28:20.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.990 7fd172312700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd1740ef610 0x7fd1740efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48620/0 (socket says 172.21.2.101:48620) 2021-02-10T11:28:20.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.990 7fd172312700 1 -- 172.21.2.101:0/1868949997 learned_addr learned my addr 172.21.2.101:0/1868949997 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:28:20.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.990 7fd173314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd1740f20e0 0x7fd1740f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:20.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.990 7fd171b11700 1 -- 172.21.2.101:0/1868949997 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2431034015 0 0) 0x7fd1680010b0 con 0x7fd1740ef610 2021-02-10T11:28:20.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.990 7fd171b11700 1 -- 172.21.2.101:0/1868949997 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd15c002fe0 con 0x7fd1740ef610 2021-02-10T11:28:20.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.991 7fd173314700 1 -- 172.21.2.101:0/1868949997 >> v2:172.21.2.108:3300/0 conn(0x7fd1741b0470 msgr2=0x7fd1741b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:28:20.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.991 7fd173314700 1 --2- 172.21.2.101:0/1868949997 >> v2:172.21.2.108:3300/0 conn(0x7fd1741b0470 0x7fd1741b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:20.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.991 7fd173314700 1 -- 172.21.2.101:0/1868949997 >> v1:172.21.2.101:6789/0 conn(0x7fd1740ef610 legacy=0x7fd1740efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:20.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.991 7fd173314700 1 -- 172.21.2.101:0/1868949997 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1740e52e0 con 0x7fd1740f20e0 2021-02-10T11:28:20.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.991 7fd173314700 1 --2- 172.21.2.101:0/1868949997 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1740f20e0 0x7fd1740f0200 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7fd16c006e30 tx=0x7fd16c009e60).ready entity=mon.0 client_cookie=1aba60ec9979a894 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:20.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.991 7fd171b11700 1 -- 172.21.2.101:0/1868949997 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd16c006a50 con 0x7fd1740f20e0 2021-02-10T11:28:20.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.991 7fd171b11700 1 -- 172.21.2.101:0/1868949997 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16c00b300 con 0x7fd1740f20e0 2021-02-10T11:28:20.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.991 7fd171b11700 1 -- 172.21.2.101:0/1868949997 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd16c00b790 con 0x7fd1740f20e0 2021-02-10T11:28:20.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.991 7fd178d6d700 1 -- 172.21.2.101:0/1868949997 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1740f20e0 msgr2=0x7fd1740f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:20.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.991 7fd178d6d700 1 --2- 172.21.2.101:0/1868949997 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1740f20e0 0x7fd1740f0200 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7fd16c006e30 tx=0x7fd16c009e60).stop 2021-02-10T11:28:20.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.992 7fd178d6d700 1 -- 172.21.2.101:0/1868949997 shutdown_connections 2021-02-10T11:28:20.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.992 7fd178d6d700 1 --2- 172.21.2.101:0/1868949997 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1740f20e0 0x7fd1740f0200 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:20.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.992 7fd178d6d700 1 --2- 172.21.2.101:0/1868949997 >> v2:172.21.2.108:3300/0 conn(0x7fd1741b0470 0x7fd1741b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:20.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.992 7fd178d6d700 1 -- 172.21.2.101:0/1868949997 >> 172.21.2.101:0/1868949997 conn(0x7fd1740ce3e0 msgr2=0x7fd1740e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:20.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.992 7fd178d6d700 1 -- 172.21.2.101:0/1868949997 shutdown_connections 2021-02-10T11:28:20.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:19.992 7fd178d6d700 1 -- 172.21.2.101:0/1868949997 wait complete. 2021-02-10T11:28:20.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.006 7fd178d6d700 1 Processor -- start 2021-02-10T11:28:20.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.006 7fd178d6d700 1 -- start start 2021-02-10T11:28:20.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.007 7fd178d6d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd17409f140 0x7fd1740c1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:20.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.007 7fd178d6d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1740c1f60 0x7fd17409e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:20.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.007 7fd178d6d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd17409e920 0x7fd1740c2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:20.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.007 7fd178d6d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1740c23e0 con 0x7fd17409f140 2021-02-10T11:28:20.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.007 7fd178d6d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17409ed20 con 0x7fd17409e920 2021-02-10T11:28:20.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.007 7fd178d6d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd174105d10 con 0x7fd1740c1f60 2021-02-10T11:28:20.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.007 7fd172b13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd17409f140 0x7fd1740c1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:20.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.007 7fd172b13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd17409f140 0x7fd1740c1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48130/0 (socket says 172.21.2.101:48130) 2021-02-10T11:28:20.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.007 7fd172b13700 1 -- 172.21.2.101:0/725705260 learned_addr learned my addr 172.21.2.101:0/725705260 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:28:20.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.007 7fd172312700 1 --2- 172.21.2.101:0/725705260 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1740c1f60 0x7fd17409e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:20.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.007 7fd173314700 1 --2- 172.21.2.101:0/725705260 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd17409e920 0x7fd1740c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:20.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.007 7fd172b13700 1 -- 172.21.2.101:0/725705260 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1740c1f60 msgr2=0x7fd17409e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:20.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.007 7fd172b13700 1 --2- 172.21.2.101:0/725705260 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1740c1f60 0x7fd17409e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:20.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.008 7fd172b13700 1 -- 172.21.2.101:0/725705260 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd17409e920 msgr2=0x7fd1740c2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:20.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.008 7fd172b13700 1 --2- 172.21.2.101:0/725705260 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd17409e920 0x7fd1740c2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:20.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.008 7fd172b13700 1 -- 172.21.2.101:0/725705260 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1741a4910 con 0x7fd17409f140 2021-02-10T11:28:20.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.008 7fd173314700 1 --2- 172.21.2.101:0/725705260 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd17409e920 0x7fd1740c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:28:20.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.008 7fd172b13700 1 --2- 172.21.2.101:0/725705260 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd17409f140 0x7fd1740c1940 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7fd164006f00 tx=0x7fd16400b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:20.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.008 7fd1637fe700 1 -- 172.21.2.101:0/725705260 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd164006350 con 0x7fd17409f140 2021-02-10T11:28:20.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.008 7fd1637fe700 1 -- 172.21.2.101:0/725705260 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16400be00 con 0x7fd17409f140 2021-02-10T11:28:20.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.008 7fd178d6d700 1 -- 172.21.2.101:0/725705260 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1741a4910 con 0x7fd17409f140 2021-02-10T11:28:20.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.008 7fd1637fe700 1 -- 172.21.2.101:0/725705260 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd164002020 con 0x7fd17409f140 2021-02-10T11:28:20.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.008 7fd178d6d700 1 -- 172.21.2.101:0/725705260 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1741a4910 con 0x7fd17409f140 2021-02-10T11:28:20.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.009 7fd1637fe700 1 -- 172.21.2.101:0/725705260 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd164005310 con 0x7fd17409f140 2021-02-10T11:28:20.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.009 7fd1637fe700 1 --2- 172.21.2.101:0/725705260 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1580186c0 0x7fd15801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:20.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.009 7fd1637fe700 1 -- 172.21.2.101:0/725705260 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd164032550 con 0x7fd17409f140 2021-02-10T11:28:20.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.009 7fd172312700 1 --2- 172.21.2.101:0/725705260 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1580186c0 0x7fd15801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:20.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.009 7fd172312700 1 --2- 172.21.2.101:0/725705260 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1580186c0 0x7fd15801abc0 crc :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:20.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.040 7fd178d6d700 1 -- 172.21.2.101:0/725705260 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1740f0b50 con 0x7fd17409f140 2021-02-10T11:28:20.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.042 7fd1637fe700 1 -- 172.21.2.101:0/725705260 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd164006b90 con 0x7fd17409f140 2021-02-10T11:28:20.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.169 7fd178d6d700 1 -- 172.21.2.101:0/725705260 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839328", "target": ["mgr", ""]}) v1 -- 0x7fd1740f0b50 con 0x7fd1580186c0 2021-02-10T11:28:20.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.196 7fd1637fe700 1 -- 172.21.2.101:0/725705260 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fd1680009b0 con 0x7fd1580186c0 2021-02-10T11:28:20.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.202 7fd178d6d700 1 -- 172.21.2.101:0/725705260 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1580186c0 msgr2=0x7fd15801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:20.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.202 7fd178d6d700 1 --2- 172.21.2.101:0/725705260 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1580186c0 0x7fd15801abc0 crc :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:20.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.202 7fd178d6d700 1 -- 172.21.2.101:0/725705260 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd17409f140 msgr2=0x7fd1740c1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:20.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.202 7fd178d6d700 1 --2- 172.21.2.101:0/725705260 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd17409f140 0x7fd1740c1940 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7fd164006f00 tx=0x7fd16400b940).stop 2021-02-10T11:28:20.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.202 7fd178d6d700 1 -- 172.21.2.101:0/725705260 shutdown_connections 2021-02-10T11:28:20.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.202 7fd178d6d700 1 --2- 172.21.2.101:0/725705260 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1580186c0 0x7fd15801abc0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:20.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.202 7fd178d6d700 1 --2- 172.21.2.101:0/725705260 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1740c1f60 0x7fd17409e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:20.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.202 7fd178d6d700 1 --2- 172.21.2.101:0/725705260 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd17409e920 0x7fd1740c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:20.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.202 7fd178d6d700 1 --2- 172.21.2.101:0/725705260 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd17409f140 0x7fd1740c1940 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:20.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.202 7fd178d6d700 1 -- 172.21.2.101:0/725705260 >> 172.21.2.101:0/725705260 conn(0x7fd17409fc10 msgr2=0x7fd1740b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:20.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.202 7fd178d6d700 1 -- 172.21.2.101:0/725705260 shutdown_connections 2021-02-10T11:28:20.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.203 7fd178d6d700 1 -- 172.21.2.101:0/725705260 wait complete. 2021-02-10T11:28:20.220 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839328 2021-02-10T11:28:20.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.625 7fa59d1c3700 1 Processor -- start 2021-02-10T11:28:20.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.625 7fa59d1c3700 1 -- start start 2021-02-10T11:28:20.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.626 7fa59d1c3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa5981b0470 0x7fa5981b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:20.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.626 7fa59d1c3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa5980f20e0 0x7fa5980f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:20.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.626 7fa59d1c3700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5980f0b50 con 0x7fa5980ef610 2021-02-10T11:28:20.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.626 7fa59d1c3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5980f0820 con 0x7fa5981b0470 2021-02-10T11:28:20.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.626 7fa59d1c3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5980f24e0 con 0x7fa5980f20e0 2021-02-10T11:28:20.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.626 7fa596d9d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa5981b0470 0x7fa5981b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:20.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.626 7fa596d9d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa5981b0470 0x7fa5981b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48138/0 (socket says 172.21.2.101:48138) 2021-02-10T11:28:20.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.626 7fa596d9d700 1 -- 172.21.2.101:0/2381797046 learned_addr learned my addr 172.21.2.101:0/2381797046 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:28:20.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.626 7fa596d9d700 1 -- 172.21.2.101:0/2381797046 >> v2:172.21.2.108:3300/0 conn(0x7fa5980f20e0 msgr2=0x7fa5980f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:28:20.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.626 7fa596d9d700 1 --2- 172.21.2.101:0/2381797046 >> v2:172.21.2.108:3300/0 conn(0x7fa5980f20e0 0x7fa5980f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:20.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.626 7fa596d9d700 1 -- 172.21.2.101:0/2381797046 >> v1:172.21.2.108:6789/0 conn(0x7fa5980ef610 legacy=0x7fa5980efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:28:20.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.626 7fa596d9d700 1 -- 172.21.2.101:0/2381797046 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5980e52e0 con 0x7fa5981b0470 2021-02-10T11:28:20.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.626 7fa596d9d700 1 --2- 172.21.2.101:0/2381797046 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5981b0470 0x7fa5981b0870 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7fa580006700 tx=0x7fa580009b80).ready entity=mon.0 client_cookie=8699e6efa9ae35e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:20.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.627 7fa595d9b700 1 -- 172.21.2.101:0/2381797046 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa58000c0c0 con 0x7fa5981b0470 2021-02-10T11:28:20.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.627 7fa595d9b700 1 -- 172.21.2.101:0/2381797046 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58000c440 con 0x7fa5981b0470 2021-02-10T11:28:20.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.627 7fa595d9b700 1 -- 172.21.2.101:0/2381797046 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa58000c920 con 0x7fa5981b0470 2021-02-10T11:28:20.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.627 7fa59d1c3700 1 -- 172.21.2.101:0/2381797046 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5981b0470 msgr2=0x7fa5981b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:20.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.627 7fa59d1c3700 1 --2- 172.21.2.101:0/2381797046 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5981b0470 0x7fa5981b0870 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7fa580006700 tx=0x7fa580009b80).stop 2021-02-10T11:28:20.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.627 7fa59d1c3700 1 -- 172.21.2.101:0/2381797046 shutdown_connections 2021-02-10T11:28:20.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.627 7fa59d1c3700 1 --2- 172.21.2.101:0/2381797046 >> v2:172.21.2.108:3300/0 conn(0x7fa5980f20e0 0x7fa5980f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:20.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.627 7fa59d1c3700 1 --2- 172.21.2.101:0/2381797046 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5981b0470 0x7fa5981b0870 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:20.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.627 7fa59d1c3700 1 -- 172.21.2.101:0/2381797046 >> 172.21.2.101:0/2381797046 conn(0x7fa5980ce3e0 msgr2=0x7fa5980e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:20.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.627 7fa59d1c3700 1 -- 172.21.2.101:0/2381797046 shutdown_connections 2021-02-10T11:28:20.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.627 7fa59d1c3700 1 -- 172.21.2.101:0/2381797046 wait complete. 2021-02-10T11:28:20.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.642 7fa59d1c3700 1 Processor -- start 2021-02-10T11:28:20.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.642 7fa59d1c3700 1 -- start start 2021-02-10T11:28:20.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.642 7fa59d1c3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5980c36a0 0x7fa5980c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:20.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.642 7fa59d1c3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5980c3190 0x7fa5980b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:20.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.642 7fa59d1c3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa5980b2370 0x7fa5980b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:20.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.642 7fa59d1c3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5980b27c0 con 0x7fa5980c36a0 2021-02-10T11:28:20.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.642 7fa59d1c3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5980b3360 con 0x7fa5980b2370 2021-02-10T11:28:20.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.642 7fa59d1c3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5980b3540 con 0x7fa5980c3190 2021-02-10T11:28:20.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.643 7fa596d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5980c36a0 0x7fa5980c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:20.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.643 7fa596d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5980c36a0 0x7fa5980c2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48148/0 (socket says 172.21.2.101:48148) 2021-02-10T11:28:20.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.643 7fa596d9d700 1 -- 172.21.2.101:0/1792714492 learned_addr learned my addr 172.21.2.101:0/1792714492 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:28:20.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.643 7fa59759e700 1 --2- 172.21.2.101:0/1792714492 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa5980b2370 0x7fa5980b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:20.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.643 7fa59659c700 1 --2- 172.21.2.101:0/1792714492 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5980c3190 0x7fa5980b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:20.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.643 7fa596d9d700 1 -- 172.21.2.101:0/1792714492 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5980c3190 msgr2=0x7fa5980b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:20.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.643 7fa596d9d700 1 --2- 172.21.2.101:0/1792714492 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5980c3190 0x7fa5980b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:20.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.643 7fa596d9d700 1 -- 172.21.2.101:0/1792714492 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa5980b2370 msgr2=0x7fa5980b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:20.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.643 7fa596d9d700 1 --2- 172.21.2.101:0/1792714492 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa5980b2370 0x7fa5980b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:20.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.643 7fa596d9d700 1 -- 172.21.2.101:0/1792714492 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5980b3760 con 0x7fa5980c36a0 2021-02-10T11:28:20.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.643 7fa59759e700 1 --2- 172.21.2.101:0/1792714492 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa5980b2370 0x7fa5980b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:28:20.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.643 7fa596d9d700 1 --2- 172.21.2.101:0/1792714492 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5980c36a0 0x7fa5980c2b70 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7fa58000b580 tx=0x7fa58000b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:20.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.643 7fa58f7fe700 1 -- 172.21.2.101:0/1792714492 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa58000bd10 con 0x7fa5980c36a0 2021-02-10T11:28:20.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.643 7fa58f7fe700 1 -- 172.21.2.101:0/1792714492 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa580002030 con 0x7fa5980c36a0 2021-02-10T11:28:20.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.643 7fa58f7fe700 1 -- 172.21.2.101:0/1792714492 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa580002550 con 0x7fa5980c36a0 2021-02-10T11:28:20.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.643 7fa59d1c3700 1 -- 172.21.2.101:0/1792714492 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5980b3760 con 0x7fa5980c36a0 2021-02-10T11:28:20.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.643 7fa59d1c3700 1 -- 172.21.2.101:0/1792714492 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5980b3760 con 0x7fa5980c36a0 2021-02-10T11:28:20.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.644 7fa58f7fe700 1 -- 172.21.2.101:0/1792714492 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa580002420 con 0x7fa5980c36a0 2021-02-10T11:28:20.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.644 7fa58f7fe700 1 --2- 172.21.2.101:0/1792714492 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa57c0186c0 0x7fa57c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:20.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.644 7fa58f7fe700 1 -- 172.21.2.101:0/1792714492 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa580032630 con 0x7fa5980c36a0 2021-02-10T11:28:20.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.644 7fa59659c700 1 --2- 172.21.2.101:0/1792714492 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa57c0186c0 0x7fa57c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:20.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.644 7fa59659c700 1 --2- 172.21.2.101:0/1792714492 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa57c0186c0 0x7fa57c01abc0 crc :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:20.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.676 7fa59d1c3700 1 -- 172.21.2.101:0/1792714492 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5980edd10 con 0x7fa5980c36a0 2021-02-10T11:28:20.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.679 7fa58f7fe700 1 -- 172.21.2.101:0/1792714492 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa580002030 con 0x7fa5980c36a0 2021-02-10T11:28:20.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.806 7fa59d1c3700 1 -- 172.21.2.101:0/1792714492 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839328", "target": ["mgr", ""]}) v1 -- 0x7fa5980edd10 con 0x7fa57c0186c0 2021-02-10T11:28:20.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.809 7fa58f7fe700 1 -- 172.21.2.101:0/1792714492 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fa588000960 con 0x7fa57c0186c0 2021-02-10T11:28:20.824 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839328/2c436e5e-b0f3-40b1-9421-64f3981e085e 2021-02-10T11:28:20.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.815 7fa59d1c3700 1 -- 172.21.2.101:0/1792714492 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa57c0186c0 msgr2=0x7fa57c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:20.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.815 7fa59d1c3700 1 --2- 172.21.2.101:0/1792714492 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa57c0186c0 0x7fa57c01abc0 crc :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:20.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.815 7fa59d1c3700 1 -- 172.21.2.101:0/1792714492 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5980c36a0 msgr2=0x7fa5980c2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:20.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.815 7fa59d1c3700 1 --2- 172.21.2.101:0/1792714492 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5980c36a0 0x7fa5980c2b70 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7fa58000b580 tx=0x7fa58000b920).stop 2021-02-10T11:28:20.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.815 7fa59d1c3700 1 -- 172.21.2.101:0/1792714492 shutdown_connections 2021-02-10T11:28:20.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.815 7fa59d1c3700 1 --2- 172.21.2.101:0/1792714492 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa57c0186c0 0x7fa57c01abc0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:20.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.815 7fa59d1c3700 1 --2- 172.21.2.101:0/1792714492 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5980c3190 0x7fa5980b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:20.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.815 7fa59d1c3700 1 --2- 172.21.2.101:0/1792714492 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa5980b2370 0x7fa5980b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:20.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.815 7fa59d1c3700 1 --2- 172.21.2.101:0/1792714492 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5980c36a0 0x7fa5980c2b70 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:20.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.815 7fa59d1c3700 1 -- 172.21.2.101:0/1792714492 >> 172.21.2.101:0/1792714492 conn(0x7fa59809f630 msgr2=0x7fa5980a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:20.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.815 7fa59d1c3700 1 -- 172.21.2.101:0/1792714492 shutdown_connections 2021-02-10T11:28:20.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:20.815 7fa59d1c3700 1 -- 172.21.2.101:0/1792714492 wait complete. 2021-02-10T11:28:20.833 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839328 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839328/2c436e5e-b0f3-40b1-9421-64f3981e085e 2021-02-10T11:28:20.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:20.834 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:20.834 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:21.170 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:21.171 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:21.171 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839328/2c436e5e-b0f3-40b1-9421-64f3981e085e/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243213 s, 43.1 MB/s 2021-02-10T11:28:21.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:21.554 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:21.554 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:21.889 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:21.890 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:21.890 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839328/2c436e5e-b0f3-40b1-9421-64f3981e085e/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:22.264 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:22.264 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:22.264 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0221596 s, 47.3 MB/s 2021-02-10T11:28:22.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:22.265 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:22.265 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:22.601 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:22.602 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:22.602 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839328/2c436e5e-b0f3-40b1-9421-64f3981e085e/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:22.976 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:22.976 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:22.976 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237414 s, 44.2 MB/s 2021-02-10T11:28:22.978 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:22.978 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:22.978 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:23.314 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:23.315 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:23.315 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839328/2c436e5e-b0f3-40b1-9421-64f3981e085e/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:23.691 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:23.691 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:23.691 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235783 s, 44.5 MB/s 2021-02-10T11:28:23.692 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:23.692 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:23.692 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:24.093 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:24.094 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:24.094 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839328/2c436e5e-b0f3-40b1-9421-64f3981e085e/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:24.479 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:24.479 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:24.480 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252288 s, 41.6 MB/s 2021-02-10T11:28:24.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:24.481 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:24.481 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:24.495 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:28:24.526 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:28:24.817 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:24.817 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:24.817 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839328/2c436e5e-b0f3-40b1-9421-64f3981e085e/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:25.192 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:25.192 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:25.192 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238804 s, 43.9 MB/s 2021-02-10T11:28:25.193 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:25.193 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:25.194 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:25.529 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:25.530 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:25.530 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839328/2c436e5e-b0f3-40b1-9421-64f3981e085e/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:25.906 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:25.906 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:25.906 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247349 s, 42.4 MB/s 2021-02-10T11:28:25.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:25.907 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:25.907 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:26.242 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:26.246 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:26.247 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839328/2c436e5e-b0f3-40b1-9421-64f3981e085e/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:26.621 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:26.621 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:26.621 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0225217 s, 46.6 MB/s 2021-02-10T11:28:26.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:26.622 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:26.622 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:26.959 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:26.960 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:26.960 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839328/2c436e5e-b0f3-40b1-9421-64f3981e085e/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:27.336 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:27.336 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:27.336 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0258559 s, 40.6 MB/s 2021-02-10T11:28:27.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:27.337 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:27.337 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:27.675 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:27.675 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:27.675 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839328/2c436e5e-b0f3-40b1-9421-64f3981e085e/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:28.050 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:28.050 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:28.050 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.02327 s, 45.1 MB/s 2021-02-10T11:28:28.051 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839329 2021-02-10T11:28:28.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.457 7f0f27017700 1 Processor -- start 2021-02-10T11:28:28.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.457 7f0f27017700 1 -- start start 2021-02-10T11:28:28.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.457 7f0f27017700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f201b0870 0x7f0f200f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:28.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.457 7f0f27017700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0f200efe00 0x7f0f200f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:28.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.457 7f0f27017700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f201b0c70 con 0x7f0f201b0470 2021-02-10T11:28:28.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.457 7f0f27017700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f200f0b50 con 0x7f0f200efe00 2021-02-10T11:28:28.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.457 7f0f27017700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f200f0820 con 0x7f0f201b0870 2021-02-10T11:28:28.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.457 7f0f25634700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0f200efe00 0x7f0f200f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:28.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.457 7f0f25634700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0f200efe00 0x7f0f200f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48156/0 (socket says 172.21.2.101:48156) 2021-02-10T11:28:28.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.457 7f0f25634700 1 -- 172.21.2.101:0/3518478999 learned_addr learned my addr 172.21.2.101:0/3518478999 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:28:28.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.458 7f0f25634700 1 -- 172.21.2.101:0/3518478999 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f201b0870 msgr2=0x7f0f200f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:28:28.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.458 7f0f1ffff700 1 --2- 172.21.2.101:0/3518478999 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f201b0870 0x7f0f200f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:28.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.458 7f0f25634700 1 --2- 172.21.2.101:0/3518478999 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f201b0870 0x7f0f200f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:28.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.458 7f0f25634700 1 -- 172.21.2.101:0/3518478999 >> v1:172.21.2.101:6789/0 conn(0x7f0f201b0470 legacy=0x7f0f200ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:28.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.458 7f0f25634700 1 -- 172.21.2.101:0/3518478999 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f200e52e0 con 0x7f0f200efe00 2021-02-10T11:28:28.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.458 7f0f1ffff700 1 --2- 172.21.2.101:0/3518478999 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f201b0870 0x7f0f200f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:28:28.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.458 7f0f25634700 1 --2- 172.21.2.101:0/3518478999 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f200efe00 0x7f0f200f0200 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f0f18006b80 tx=0x7f0f18009b90).ready entity=mon.0 client_cookie=c86ed80aa89b725 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:28.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.458 7f0f1f7fe700 1 -- 172.21.2.101:0/3518478999 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0f180022e0 con 0x7f0f200efe00 2021-02-10T11:28:28.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.458 7f0f1f7fe700 1 -- 172.21.2.101:0/3518478999 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f180009b0 con 0x7f0f200efe00 2021-02-10T11:28:28.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.458 7f0f1f7fe700 1 -- 172.21.2.101:0/3518478999 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0f180027e0 con 0x7f0f200efe00 2021-02-10T11:28:28.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.458 7f0f27017700 1 -- 172.21.2.101:0/3518478999 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f200efe00 msgr2=0x7f0f200f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:28.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.458 7f0f27017700 1 --2- 172.21.2.101:0/3518478999 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f200efe00 0x7f0f200f0200 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f0f18006b80 tx=0x7f0f18009b90).stop 2021-02-10T11:28:28.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.459 7f0f27017700 1 -- 172.21.2.101:0/3518478999 shutdown_connections 2021-02-10T11:28:28.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.459 7f0f27017700 1 --2- 172.21.2.101:0/3518478999 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f201b0870 0x7f0f200f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:28.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.459 7f0f27017700 1 --2- 172.21.2.101:0/3518478999 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f200efe00 0x7f0f200f0200 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:28.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.459 7f0f27017700 1 -- 172.21.2.101:0/3518478999 >> 172.21.2.101:0/3518478999 conn(0x7f0f200ce3e0 msgr2=0x7f0f200e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:28.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.459 7f0f27017700 1 -- 172.21.2.101:0/3518478999 shutdown_connections 2021-02-10T11:28:28.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.459 7f0f27017700 1 -- 172.21.2.101:0/3518478999 wait complete. 2021-02-10T11:28:28.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.474 7f0f27017700 1 Processor -- start 2021-02-10T11:28:28.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.474 7f0f27017700 1 -- start start 2021-02-10T11:28:28.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.474 7f0f27017700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f2019cc10 0x7f0f200a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:28.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.474 7f0f27017700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f2019d010 0x7f0f200a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:28.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.474 7f0f27017700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f20105d20 0x7f0f20106120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:28.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.474 7f0f27017700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f200a7460 con 0x7f0f20105d20 2021-02-10T11:28:28.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.474 7f0f27017700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f20106740 con 0x7f0f2019d010 2021-02-10T11:28:28.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.474 7f0f27017700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f20106960 con 0x7f0f2019cc10 2021-02-10T11:28:28.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.474 7f0f25634700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f20105d20 0x7f0f20106120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:28.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.474 7f0f25634700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f20105d20 0x7f0f20106120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48162/0 (socket says 172.21.2.101:48162) 2021-02-10T11:28:28.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.474 7f0f25634700 1 -- 172.21.2.101:0/3550717550 learned_addr learned my addr 172.21.2.101:0/3550717550 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:28:28.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.475 7f0f24e33700 1 --2- 172.21.2.101:0/3550717550 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f2019cc10 0x7f0f200a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:28.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.475 7f0f1ffff700 1 --2- 172.21.2.101:0/3550717550 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f2019d010 0x7f0f200a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:28.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.475 7f0f25634700 1 -- 172.21.2.101:0/3550717550 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f2019cc10 msgr2=0x7f0f200a51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:28.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.475 7f0f25634700 1 --2- 172.21.2.101:0/3550717550 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f2019cc10 0x7f0f200a51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:28.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.475 7f0f25634700 1 -- 172.21.2.101:0/3550717550 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f2019d010 msgr2=0x7f0f200a5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:28.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.475 7f0f25634700 1 --2- 172.21.2.101:0/3550717550 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f2019d010 0x7f0f200a5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:28.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.475 7f0f25634700 1 -- 172.21.2.101:0/3550717550 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f20106b80 con 0x7f0f20105d20 2021-02-10T11:28:28.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.475 7f0f1ffff700 1 --2- 172.21.2.101:0/3550717550 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f2019d010 0x7f0f200a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:28:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.475 7f0f25634700 1 --2- 172.21.2.101:0/3550717550 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f20105d20 0x7f0f20106120 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f0f1800b590 tx=0x7f0f1800b930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.475 7f0f1d7fa700 1 -- 172.21.2.101:0/3550717550 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0f18000b30 con 0x7f0f20105d20 2021-02-10T11:28:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.475 7f0f1d7fa700 1 -- 172.21.2.101:0/3550717550 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1800bd20 con 0x7f0f20105d20 2021-02-10T11:28:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.475 7f0f27017700 1 -- 172.21.2.101:0/3550717550 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f20106b80 con 0x7f0f20105d20 2021-02-10T11:28:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.475 7f0f1d7fa700 1 -- 172.21.2.101:0/3550717550 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0f18006430 con 0x7f0f20105d20 2021-02-10T11:28:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.475 7f0f27017700 1 -- 172.21.2.101:0/3550717550 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f200a61a0 con 0x7f0f20105d20 2021-02-10T11:28:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.476 7f0f1d7fa700 1 -- 172.21.2.101:0/3550717550 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0f18009fe0 con 0x7f0f20105d20 2021-02-10T11:28:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.476 7f0f1d7fa700 1 --2- 172.21.2.101:0/3550717550 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0f040186d0 0x7f0f0401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.476 7f0f1d7fa700 1 -- 172.21.2.101:0/3550717550 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0f18032780 con 0x7f0f20105d20 2021-02-10T11:28:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.476 7f0f24e33700 1 --2- 172.21.2.101:0/3550717550 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0f040186d0 0x7f0f0401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.476 7f0f24e33700 1 --2- 172.21.2.101:0/3550717550 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0f040186d0 0x7f0f0401abd0 crc :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:28.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.507 7f0f27017700 1 -- 172.21.2.101:0/3550717550 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f20105af0 con 0x7f0f20105d20 2021-02-10T11:28:28.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.510 7f0f1d7fa700 1 -- 172.21.2.101:0/3550717550 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0f18009480 con 0x7f0f20105d20 2021-02-10T11:28:28.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.636 7f0f27017700 1 -- 172.21.2.101:0/3550717550 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839329", "target": ["mgr", ""]}) v1 -- 0x7f0f200a61a0 con 0x7f0f040186d0 2021-02-10T11:28:28.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.663 7f0f1d7fa700 1 -- 172.21.2.101:0/3550717550 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f0f08006680 con 0x7f0f040186d0 2021-02-10T11:28:28.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.669 7f0f27017700 1 -- 172.21.2.101:0/3550717550 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0f040186d0 msgr2=0x7f0f0401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:28.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.669 7f0f27017700 1 --2- 172.21.2.101:0/3550717550 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0f040186d0 0x7f0f0401abd0 crc :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:28.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.669 7f0f27017700 1 -- 172.21.2.101:0/3550717550 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f20105d20 msgr2=0x7f0f20106120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:28.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.669 7f0f27017700 1 --2- 172.21.2.101:0/3550717550 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f20105d20 0x7f0f20106120 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f0f1800b590 tx=0x7f0f1800b930).stop 2021-02-10T11:28:28.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.669 7f0f27017700 1 -- 172.21.2.101:0/3550717550 shutdown_connections 2021-02-10T11:28:28.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.669 7f0f27017700 1 --2- 172.21.2.101:0/3550717550 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f20105d20 0x7f0f20106120 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:28.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.669 7f0f27017700 1 --2- 172.21.2.101:0/3550717550 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f2019d010 0x7f0f200a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:28.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.669 7f0f27017700 1 --2- 172.21.2.101:0/3550717550 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0f040186d0 0x7f0f0401abd0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:28.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.669 7f0f27017700 1 --2- 172.21.2.101:0/3550717550 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f2019cc10 0x7f0f200a51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:28.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.669 7f0f27017700 1 -- 172.21.2.101:0/3550717550 >> 172.21.2.101:0/3550717550 conn(0x7f0f200afe70 msgr2=0x7f0f200a1580 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:28.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.669 7f0f27017700 1 -- 172.21.2.101:0/3550717550 shutdown_connections 2021-02-10T11:28:28.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:28.669 7f0f27017700 1 -- 172.21.2.101:0/3550717550 wait complete. 2021-02-10T11:28:28.688 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839329 2021-02-10T11:28:29.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.094 7f024e735700 1 Processor -- start 2021-02-10T11:28:29.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.094 7f024e735700 1 -- start start 2021-02-10T11:28:29.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.094 7f024e735700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f02481b0870 0x7f02480f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:29.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.094 7f024e735700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f02480efe00 0x7f02480f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:29.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.094 7f024e735700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02481b0c70 con 0x7f02481b0470 2021-02-10T11:28:29.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.094 7f024e735700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02480e5ea0 con 0x7f02480efe00 2021-02-10T11:28:29.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.094 7f024e735700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02480f0b50 con 0x7f02481b0870 2021-02-10T11:28:29.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.095 7f024cd52700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f02480efe00 0x7f02480f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:29.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.095 7f024cd52700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f02480efe00 0x7f02480f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48170/0 (socket says 172.21.2.101:48170) 2021-02-10T11:28:29.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.095 7f024cd52700 1 -- 172.21.2.101:0/3090710115 learned_addr learned my addr 172.21.2.101:0/3090710115 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:28:29.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.095 7f024cd52700 1 -- 172.21.2.101:0/3090710115 >> v2:172.21.2.108:3300/0 conn(0x7f02481b0870 msgr2=0x7f02480f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:28:29.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.095 7f024cd52700 1 --2- 172.21.2.101:0/3090710115 >> v2:172.21.2.108:3300/0 conn(0x7f02481b0870 0x7f02480f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:29.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.095 7f024cd52700 1 -- 172.21.2.101:0/3090710115 >> v1:172.21.2.101:6789/0 conn(0x7f02481b0470 legacy=0x7f02480ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:29.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.095 7f0246ffd700 1 -- 172.21.2.101:0/3090710115 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3234111170 0 0) 0x7f0238000ce0 con 0x7f02481b0470 2021-02-10T11:28:29.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.095 7f024cd52700 1 -- 172.21.2.101:0/3090710115 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02480f0820 con 0x7f02480efe00 2021-02-10T11:28:29.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.095 7f024cd52700 1 --2- 172.21.2.101:0/3090710115 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02480efe00 0x7f02480f0200 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f0240006f00 tx=0x7f0240009e90).ready entity=mon.0 client_cookie=7fbfe0b11d739318 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:29.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.096 7f0246ffd700 1 -- 172.21.2.101:0/3090710115 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f024000b300 con 0x7f02480efe00 2021-02-10T11:28:29.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.096 7f0246ffd700 1 -- 172.21.2.101:0/3090710115 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024000b640 con 0x7f02480efe00 2021-02-10T11:28:29.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.096 7f0246ffd700 1 -- 172.21.2.101:0/3090710115 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f024000ba90 con 0x7f02480efe00 2021-02-10T11:28:29.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.096 7f024e735700 1 -- 172.21.2.101:0/3090710115 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02480efe00 msgr2=0x7f02480f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:29.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.096 7f024e735700 1 --2- 172.21.2.101:0/3090710115 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02480efe00 0x7f02480f0200 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f0240006f00 tx=0x7f0240009e90).stop 2021-02-10T11:28:29.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.096 7f024e735700 1 -- 172.21.2.101:0/3090710115 shutdown_connections 2021-02-10T11:28:29.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.096 7f024e735700 1 --2- 172.21.2.101:0/3090710115 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02480efe00 0x7f02480f0200 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:29.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.096 7f024e735700 1 --2- 172.21.2.101:0/3090710115 >> v2:172.21.2.108:3300/0 conn(0x7f02481b0870 0x7f02480f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:29.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.096 7f024e735700 1 -- 172.21.2.101:0/3090710115 >> 172.21.2.101:0/3090710115 conn(0x7f02480ce3e0 msgr2=0x7f02480e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:29.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.097 7f024e735700 1 -- 172.21.2.101:0/3090710115 shutdown_connections 2021-02-10T11:28:29.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.097 7f024e735700 1 -- 172.21.2.101:0/3090710115 wait complete. 2021-02-10T11:28:29.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.111 7f024e735700 1 Processor -- start 2021-02-10T11:28:29.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.111 7f024e735700 1 -- start start 2021-02-10T11:28:29.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.112 7f024e735700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02480bf910 0x7f02480bfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:29.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.112 7f024e735700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02480c0330 0x7f02480c0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:29.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.112 7f024e735700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02480c13d0 0x7f02480c17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:29.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.112 7f024e735700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0248109930 con 0x7f02480c0330 2021-02-10T11:28:29.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.112 7f024e735700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0248109b50 con 0x7f02480c13d0 2021-02-10T11:28:29.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.112 7f024e735700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02480c1df0 con 0x7f02480bf910 2021-02-10T11:28:29.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.112 7f02477fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02480c0330 0x7f02480c0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:29.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.112 7f02477fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02480c0330 0x7f02480c0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48176/0 (socket says 172.21.2.101:48176) 2021-02-10T11:28:29.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.112 7f02477fe700 1 -- 172.21.2.101:0/2124306359 learned_addr learned my addr 172.21.2.101:0/2124306359 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:28:29.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.112 7f02477fe700 1 -- 172.21.2.101:0/2124306359 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02480bf910 msgr2=0x7f02480bfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:29.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.112 7f02477fe700 1 --2- 172.21.2.101:0/2124306359 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02480bf910 0x7f02480bfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:29.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.112 7f02477fe700 1 -- 172.21.2.101:0/2124306359 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02480c13d0 msgr2=0x7f02480c17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:29.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.112 7f02477fe700 1 --2- 172.21.2.101:0/2124306359 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02480c13d0 0x7f02480c17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:29.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.112 7f02477fe700 1 -- 172.21.2.101:0/2124306359 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02480c1fd0 con 0x7f02480c0330 2021-02-10T11:28:29.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.112 7f02477fe700 1 --2- 172.21.2.101:0/2124306359 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02480c0330 0x7f02480c0730 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f023c00b530 tx=0x7f023c00b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:29.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.112 7f0244ff9700 1 -- 172.21.2.101:0/2124306359 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f023c00bcc0 con 0x7f02480c0330 2021-02-10T11:28:29.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.112 7f0244ff9700 1 -- 172.21.2.101:0/2124306359 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023c000c90 con 0x7f02480c0330 2021-02-10T11:28:29.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.112 7f0244ff9700 1 -- 172.21.2.101:0/2124306359 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f023c0022a0 con 0x7f02480c0330 2021-02-10T11:28:29.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.113 7f024e735700 1 -- 172.21.2.101:0/2124306359 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02480c1fd0 con 0x7f02480c0330 2021-02-10T11:28:29.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.113 7f024e735700 1 -- 172.21.2.101:0/2124306359 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02480c1fd0 con 0x7f02480c0330 2021-02-10T11:28:29.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.113 7f0244ff9700 1 -- 172.21.2.101:0/2124306359 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f023c005310 con 0x7f02480c0330 2021-02-10T11:28:29.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.113 7f0244ff9700 1 --2- 172.21.2.101:0/2124306359 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02280186c0 0x7f022801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:29.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.113 7f0244ff9700 1 -- 172.21.2.101:0/2124306359 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f023c032580 con 0x7f02480c0330 2021-02-10T11:28:29.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.113 7f0247fff700 1 --2- 172.21.2.101:0/2124306359 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02280186c0 0x7f022801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:29.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.114 7f0247fff700 1 --2- 172.21.2.101:0/2124306359 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02280186c0 0x7f022801abc0 crc :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:29.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.144 7f024e735700 1 -- 172.21.2.101:0/2124306359 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02480f0b50 con 0x7f02480c0330 2021-02-10T11:28:29.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.147 7f0244ff9700 1 -- 172.21.2.101:0/2124306359 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f023c02dee0 con 0x7f02480c0330 2021-02-10T11:28:29.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.277 7f024e735700 1 -- 172.21.2.101:0/2124306359 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839329", "target": ["mgr", ""]}) v1 -- 0x7f02480f0b50 con 0x7f02280186c0 2021-02-10T11:28:29.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.279 7f0244ff9700 1 -- 172.21.2.101:0/2124306359 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f0238006a00 con 0x7f02280186c0 2021-02-10T11:28:29.291 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839329/39a09809-4119-4602-8a18-3eb9773940db 2021-02-10T11:28:29.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.281 7f024e735700 1 -- 172.21.2.101:0/2124306359 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02280186c0 msgr2=0x7f022801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:29.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.281 7f024e735700 1 --2- 172.21.2.101:0/2124306359 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02280186c0 0x7f022801abc0 crc :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:29.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.281 7f024e735700 1 -- 172.21.2.101:0/2124306359 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02480c0330 msgr2=0x7f02480c0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:29.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.281 7f024e735700 1 --2- 172.21.2.101:0/2124306359 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02480c0330 0x7f02480c0730 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f023c00b530 tx=0x7f023c00b8d0).stop 2021-02-10T11:28:29.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.281 7f024e735700 1 -- 172.21.2.101:0/2124306359 shutdown_connections 2021-02-10T11:28:29.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.281 7f024e735700 1 --2- 172.21.2.101:0/2124306359 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02480c13d0 0x7f02480c17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:29.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.281 7f024e735700 1 --2- 172.21.2.101:0/2124306359 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02480c0330 0x7f02480c0730 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:29.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.281 7f024e735700 1 --2- 172.21.2.101:0/2124306359 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02280186c0 0x7f022801abc0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:29.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.281 7f024e735700 1 --2- 172.21.2.101:0/2124306359 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02480bf910 0x7f02480bfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:29.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.281 7f024e735700 1 -- 172.21.2.101:0/2124306359 >> 172.21.2.101:0/2124306359 conn(0x7f02480bf4e0 msgr2=0x7f0248191190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:29.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.282 7f024e735700 1 -- 172.21.2.101:0/2124306359 shutdown_connections 2021-02-10T11:28:29.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:29.282 7f024e735700 1 -- 172.21.2.101:0/2124306359 wait complete. 2021-02-10T11:28:29.300 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839329 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839329/39a09809-4119-4602-8a18-3eb9773940db 2021-02-10T11:28:29.300 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:29.301 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:29.301 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:29.637 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:29.637 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:29.638 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839329/39a09809-4119-4602-8a18-3eb9773940db/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:30.017 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:30.017 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:30.017 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0221644 s, 47.3 MB/s 2021-02-10T11:28:30.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:30.018 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:30.018 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:30.353 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:30.354 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:30.354 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839329/39a09809-4119-4602-8a18-3eb9773940db/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:30.729 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:30.729 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:30.729 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024137 s, 43.4 MB/s 2021-02-10T11:28:30.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:30.730 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:30.730 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:31.064 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:31.064 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:31.064 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839329/39a09809-4119-4602-8a18-3eb9773940db/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:31.444 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:31.444 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:31.444 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252286 s, 41.6 MB/s 2021-02-10T11:28:31.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:31.445 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:31.445 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:31.781 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:31.782 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:31.782 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839329/39a09809-4119-4602-8a18-3eb9773940db/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:32.159 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:32.159 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:32.159 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240281 s, 43.6 MB/s 2021-02-10T11:28:32.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:32.160 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:32.161 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:32.496 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:32.496 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:32.497 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839329/39a09809-4119-4602-8a18-3eb9773940db/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:32.873 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:32.873 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:32.873 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0250196 s, 41.9 MB/s 2021-02-10T11:28:32.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:32.875 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:32.875 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:33.209 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:33.209 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:33.209 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839329/39a09809-4119-4602-8a18-3eb9773940db/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:33.586 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:33.586 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:33.586 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247017 s, 42.4 MB/s 2021-02-10T11:28:33.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:33.587 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:33.587 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:33.922 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:33.923 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:33.923 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839329/39a09809-4119-4602-8a18-3eb9773940db/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:34.298 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:34.298 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:34.298 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0256434 s, 40.9 MB/s 2021-02-10T11:28:34.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:34.299 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:34.299 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:34.638 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:34.639 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:34.639 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839329/39a09809-4119-4602-8a18-3eb9773940db/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:35.027 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:35.027 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:35.027 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0228238 s, 45.9 MB/s 2021-02-10T11:28:35.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:35.028 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:35.028 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:35.363 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:35.364 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:35.364 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839329/39a09809-4119-4602-8a18-3eb9773940db/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:35.740 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:35.741 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:35.741 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252531 s, 41.5 MB/s 2021-02-10T11:28:35.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:35.742 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:35.742 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:36.077 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:36.078 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:36.078 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839329/39a09809-4119-4602-8a18-3eb9773940db/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:36.452 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:36.453 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:36.453 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0220517 s, 47.6 MB/s 2021-02-10T11:28:36.454 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839330 2021-02-10T11:28:36.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.860 7f83e0113700 1 Processor -- start 2021-02-10T11:28:36.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.860 7f83e0113700 1 -- start start 2021-02-10T11:28:36.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.860 7f83e0113700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83d81b0870 0x7f83d80f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:36.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.860 7f83e0113700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d81b0c70 con 0x7f83d80efe00 2021-02-10T11:28:36.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.860 7f83e0113700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d80f0b50 con 0x7f83d81b0470 2021-02-10T11:28:36.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.860 7f83e0113700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d80e5ea0 con 0x7f83d81b0870 2021-02-10T11:28:36.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.860 7f83de730700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f83d80efe00 0x7f83d80f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48678/0 (socket says 172.21.2.101:48678) 2021-02-10T11:28:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.860 7f83de730700 1 -- 172.21.2.101:0/3147216836 learned_addr learned my addr 172.21.2.101:0/3147216836 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:28:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.861 7f83dcf2d700 1 -- 172.21.2.101:0/3147216836 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665512998 0 0) 0x7f83d0001180 con 0x7f83d80efe00 2021-02-10T11:28:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.861 7f83dcf2d700 1 -- 172.21.2.101:0/3147216836 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83c4002fe0 con 0x7f83d80efe00 2021-02-10T11:28:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.861 7f83dd72e700 1 --2- 172.21.2.101:0/3147216836 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83d81b0870 0x7f83d80f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.861 7f83dcf2d700 1 -- 172.21.2.101:0/3147216836 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 406202125 0 0) 0x7f83d00010e0 con 0x7f83d80efe00 2021-02-10T11:28:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.861 7f83dcf2d700 1 -- 172.21.2.101:0/3147216836 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83d81b0870 msgr2=0x7f83d80f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:28:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.861 7f83dcf2d700 1 --2- 172.21.2.101:0/3147216836 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83d81b0870 0x7f83d80f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.861 7f83dcf2d700 1 -- 172.21.2.101:0/3147216836 >> v1:172.21.2.108:6789/0 conn(0x7f83d81b0470 legacy=0x7f83d80ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.861 7f83dcf2d700 1 -- 172.21.2.101:0/3147216836 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d80e52e0 con 0x7f83d80efe00 2021-02-10T11:28:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.861 7f83dcf2d700 1 -- 172.21.2.101:0/3147216836 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f83d0001600 con 0x7f83d80efe00 2021-02-10T11:28:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.861 7f83dcf2d700 1 -- 172.21.2.101:0/3147216836 >> v1:172.21.2.101:6789/0 conn(0x7f83d80efe00 legacy=0x7f83d80f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.861 7f83dcf2d700 1 --2- 172.21.2.101:0/3147216836 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83c4003b10 0x7f83c4006000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.861 7f83dcf2d700 1 --2- 172.21.2.101:0/3147216836 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83c40067f0 0x7f83c4008d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:36.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.861 7f83dcf2d700 1 --2- 172.21.2.101:0/3147216836 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83c4009590 0x7f83c400bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:36.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.861 7f83dcf2d700 1 -- 172.21.2.101:0/3147216836 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c400c360 con 0x7f83c40067f0 2021-02-10T11:28:36.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.861 7f83dcf2d700 1 -- 172.21.2.101:0/3147216836 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c400c580 con 0x7f83c4009590 2021-02-10T11:28:36.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.861 7f83dcf2d700 1 -- 172.21.2.101:0/3147216836 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c400c7a0 con 0x7f83c4003b10 2021-02-10T11:28:36.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.861 7f83dd72e700 1 --2- 172.21.2.101:0/3147216836 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83d81b0870 0x7f83d80f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:28:36.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.862 7f83dd72e700 1 --2- 172.21.2.101:0/3147216836 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83c40067f0 0x7f83c4008d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:36.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.862 7f83de730700 1 --2- 172.21.2.101:0/3147216836 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83c4009590 0x7f83c400bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:36.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.862 7f83ddf2f700 1 --2- 172.21.2.101:0/3147216836 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83c4003b10 0x7f83c4006000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:36.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.862 7f83dd72e700 1 -- 172.21.2.101:0/3147216836 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83c4003b10 msgr2=0x7f83c4006000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:28:36.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.862 7f83dd72e700 1 --2- 172.21.2.101:0/3147216836 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83c4003b10 0x7f83c4006000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:36.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.862 7f83dd72e700 1 -- 172.21.2.101:0/3147216836 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83c4009590 msgr2=0x7f83c400bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:28:36.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.862 7f83dd72e700 1 --2- 172.21.2.101:0/3147216836 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83c4009590 0x7f83c400bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:36.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.862 7f83dd72e700 1 -- 172.21.2.101:0/3147216836 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c400ca40 con 0x7f83c40067f0 2021-02-10T11:28:36.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.862 7f83de730700 1 --2- 172.21.2.101:0/3147216836 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83c4009590 0x7f83c400bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:28:36.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.862 7f83dd72e700 1 --2- 172.21.2.101:0/3147216836 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83c40067f0 0x7f83c4008d10 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f83d4006890 tx=0x7f83d404dab0).ready entity=mon.0 client_cookie=d81d475af3fd9f79 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:36.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.862 7f83dcf2d700 1 -- 172.21.2.101:0/3147216836 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f83d4000c90 con 0x7f83c40067f0 2021-02-10T11:28:36.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.862 7f83dcf2d700 1 -- 172.21.2.101:0/3147216836 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d40500c0 con 0x7f83c40067f0 2021-02-10T11:28:36.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.862 7f83dcf2d700 1 -- 172.21.2.101:0/3147216836 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f83d40505a0 con 0x7f83c40067f0 2021-02-10T11:28:36.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.862 7f83e0113700 1 -- 172.21.2.101:0/3147216836 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83c40067f0 msgr2=0x7f83c4008d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:36.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.862 7f83e0113700 1 --2- 172.21.2.101:0/3147216836 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83c40067f0 0x7f83c4008d10 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f83d4006890 tx=0x7f83d404dab0).stop 2021-02-10T11:28:36.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.862 7f83de730700 1 -- 172.21.2.101:0/3147216836 reap_dead start 2021-02-10T11:28:36.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.863 7f83e0113700 1 -- 172.21.2.101:0/3147216836 shutdown_connections 2021-02-10T11:28:36.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.863 7f83e0113700 1 -- 172.21.2.101:0/3147216836 >> 172.21.2.101:0/3147216836 conn(0x7f83d80ce3e0 msgr2=0x7f83d80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:36.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.863 7f83e0113700 1 -- 172.21.2.101:0/3147216836 shutdown_connections 2021-02-10T11:28:36.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.863 7f83e0113700 1 -- 172.21.2.101:0/3147216836 wait complete. 2021-02-10T11:28:36.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.878 7f83e0113700 1 Processor -- start 2021-02-10T11:28:36.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.878 7f83e0113700 1 -- start start 2021-02-10T11:28:36.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.878 7f83e0113700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83d809b930 0x7f83d80b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:36.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.879 7f83e0113700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83d80b8410 0x7f83d80b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:36.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.879 7f83e0113700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83d80c1f80 0x7f83d80c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:36.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.879 7f83e0113700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d80c1a00 con 0x7f83d80b8410 2021-02-10T11:28:36.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.879 7f83e0113700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d80b8e30 con 0x7f83d809b930 2021-02-10T11:28:36.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.879 7f83e0113700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d80b9050 con 0x7f83d80c1f80 2021-02-10T11:28:36.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.879 7f83dd72e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83d80b8410 0x7f83d80b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:36.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.879 7f83dd72e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83d80b8410 0x7f83d80b8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48194/0 (socket says 172.21.2.101:48194) 2021-02-10T11:28:36.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.879 7f83dd72e700 1 -- 172.21.2.101:0/2301575278 learned_addr learned my addr 172.21.2.101:0/2301575278 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:28:36.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.879 7f83dd72e700 1 -- 172.21.2.101:0/2301575278 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83d80c1f80 msgr2=0x7f83d80c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:36.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.879 7f83dd72e700 1 --2- 172.21.2.101:0/2301575278 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83d80c1f80 0x7f83d80c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:36.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.879 7f83dd72e700 1 -- 172.21.2.101:0/2301575278 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83d809b930 msgr2=0x7f83d80b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:36.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.879 7f83dd72e700 1 --2- 172.21.2.101:0/2301575278 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83d809b930 0x7f83d80b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:36.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.879 7f83dd72e700 1 -- 172.21.2.101:0/2301575278 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d80c29a0 con 0x7f83d80b8410 2021-02-10T11:28:36.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.879 7f83dd72e700 1 --2- 172.21.2.101:0/2301575278 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83d80b8410 0x7f83d80b8810 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f83d404f5e0 tx=0x7f83d404f980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:36.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.879 7f83ca7fc700 1 -- 172.21.2.101:0/2301575278 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f83d404fd70 con 0x7f83d80b8410 2021-02-10T11:28:36.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.879 7f83ca7fc700 1 -- 172.21.2.101:0/2301575278 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d4002030 con 0x7f83d80b8410 2021-02-10T11:28:36.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.879 7f83e0113700 1 -- 172.21.2.101:0/2301575278 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d80c29a0 con 0x7f83d80b8410 2021-02-10T11:28:36.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.879 7f83ca7fc700 1 -- 172.21.2.101:0/2301575278 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f83d4002590 con 0x7f83d80b8410 2021-02-10T11:28:36.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.880 7f83e0113700 1 -- 172.21.2.101:0/2301575278 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d80c29a0 con 0x7f83d80b8410 2021-02-10T11:28:36.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.880 7f83ca7fc700 1 -- 172.21.2.101:0/2301575278 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f83d4002210 con 0x7f83d80b8410 2021-02-10T11:28:36.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.880 7f83ca7fc700 1 --2- 172.21.2.101:0/2301575278 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83c0018730 0x7f83c001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:36.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.880 7f83ca7fc700 1 -- 172.21.2.101:0/2301575278 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f83d4076a30 con 0x7f83d80b8410 2021-02-10T11:28:36.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.880 7f83ddf2f700 1 --2- 172.21.2.101:0/2301575278 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83c0018730 0x7f83c001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:36.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.881 7f83ddf2f700 1 --2- 172.21.2.101:0/2301575278 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83c0018730 0x7f83c001ac10 crc :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:36.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.910 7f83e0113700 1 -- 172.21.2.101:0/2301575278 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d8105b30 con 0x7f83d80b8410 2021-02-10T11:28:36.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:36.913 7f83ca7fc700 1 -- 172.21.2.101:0/2301575278 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f83d4006930 con 0x7f83d80b8410 2021-02-10T11:28:37.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.040 7f83e0113700 1 -- 172.21.2.101:0/2301575278 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839330", "target": ["mgr", ""]}) v1 -- 0x7f83d8105b30 con 0x7f83c0018730 2021-02-10T11:28:37.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.067 7f83ca7fc700 1 -- 172.21.2.101:0/2301575278 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f83cc006740 con 0x7f83c0018730 2021-02-10T11:28:37.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.073 7f83e0113700 1 -- 172.21.2.101:0/2301575278 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83c0018730 msgr2=0x7f83c001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:37.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.073 7f83e0113700 1 --2- 172.21.2.101:0/2301575278 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83c0018730 0x7f83c001ac10 crc :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:37.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.073 7f83e0113700 1 -- 172.21.2.101:0/2301575278 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83d80b8410 msgr2=0x7f83d80b8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:37.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.073 7f83e0113700 1 --2- 172.21.2.101:0/2301575278 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83d80b8410 0x7f83d80b8810 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f83d404f5e0 tx=0x7f83d404f980).stop 2021-02-10T11:28:37.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.073 7f83e0113700 1 -- 172.21.2.101:0/2301575278 shutdown_connections 2021-02-10T11:28:37.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.073 7f83e0113700 1 --2- 172.21.2.101:0/2301575278 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83c0018730 0x7f83c001ac10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:37.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.073 7f83e0113700 1 --2- 172.21.2.101:0/2301575278 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83d80c1f80 0x7f83d80c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:37.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.073 7f83e0113700 1 --2- 172.21.2.101:0/2301575278 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83d80b8410 0x7f83d80b8810 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:37.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.073 7f83e0113700 1 --2- 172.21.2.101:0/2301575278 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83d809b930 0x7f83d80b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:37.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.073 7f83e0113700 1 -- 172.21.2.101:0/2301575278 >> 172.21.2.101:0/2301575278 conn(0x7f83d8094e10 msgr2=0x7f83d80a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:37.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.073 7f83e0113700 1 -- 172.21.2.101:0/2301575278 shutdown_connections 2021-02-10T11:28:37.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.074 7f83e0113700 1 -- 172.21.2.101:0/2301575278 wait complete. 2021-02-10T11:28:37.092 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839330 2021-02-10T11:28:37.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.497 7efd3fa68700 1 Processor -- start 2021-02-10T11:28:37.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.498 7efd3fa68700 1 -- start start 2021-02-10T11:28:37.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.498 7efd3fa68700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd381b0470 0x7efd381b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:37.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.498 7efd3fa68700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd380f0b50 con 0x7efd380ef610 2021-02-10T11:28:37.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.498 7efd3fa68700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd380f24e0 con 0x7efd380f20e0 2021-02-10T11:28:37.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.498 7efd3fa68700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd380e5ea0 con 0x7efd381b0470 2021-02-10T11:28:37.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.498 7efd3d083700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7efd380ef610 0x7efd380efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48694/0 (socket says 172.21.2.101:48694) 2021-02-10T11:28:37.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.498 7efd3d083700 1 -- 172.21.2.101:0/1712513477 learned_addr learned my addr 172.21.2.101:0/1712513477 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:28:37.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.498 7efd3d884700 1 --2- 172.21.2.101:0/1712513477 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd381b0470 0x7efd381b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:37.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.499 7efd3c882700 1 -- 172.21.2.101:0/1712513477 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1439597365 0 0) 0x7efd340010b0 con 0x7efd380ef610 2021-02-10T11:28:37.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.499 7efd3c882700 1 -- 172.21.2.101:0/1712513477 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd24002fe0 con 0x7efd380ef610 2021-02-10T11:28:37.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.499 7efd3d884700 1 -- 172.21.2.101:0/1712513477 >> v1:172.21.2.108:6789/0 conn(0x7efd380f20e0 legacy=0x7efd380f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:37.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.499 7efd3d884700 1 -- 172.21.2.101:0/1712513477 >> v1:172.21.2.101:6789/0 conn(0x7efd380ef610 legacy=0x7efd380efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:37.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.499 7efd3d884700 1 -- 172.21.2.101:0/1712513477 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd380e52e0 con 0x7efd381b0470 2021-02-10T11:28:37.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.499 7efd3d884700 1 --2- 172.21.2.101:0/1712513477 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd381b0470 0x7efd381b0870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7efd2c006650 tx=0x7efd2c009a80).ready entity=mon.2 client_cookie=ee3b4300b06e6625 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:37.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.499 7efd3c882700 1 -- 172.21.2.101:0/1712513477 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd2c00c0c0 con 0x7efd381b0470 2021-02-10T11:28:37.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.499 7efd3c882700 1 -- 172.21.2.101:0/1712513477 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2c00c340 con 0x7efd381b0470 2021-02-10T11:28:37.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.499 7efd3c882700 1 -- 172.21.2.101:0/1712513477 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd2c00c860 con 0x7efd381b0470 2021-02-10T11:28:37.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.499 7efd3fa68700 1 -- 172.21.2.101:0/1712513477 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd381b0470 msgr2=0x7efd381b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:37.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.499 7efd3fa68700 1 --2- 172.21.2.101:0/1712513477 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd381b0470 0x7efd381b0870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7efd2c006650 tx=0x7efd2c009a80).stop 2021-02-10T11:28:37.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.500 7efd3fa68700 1 -- 172.21.2.101:0/1712513477 shutdown_connections 2021-02-10T11:28:37.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.500 7efd3fa68700 1 --2- 172.21.2.101:0/1712513477 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd381b0470 0x7efd381b0870 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:37.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.500 7efd3fa68700 1 -- 172.21.2.101:0/1712513477 >> 172.21.2.101:0/1712513477 conn(0x7efd380ce3e0 msgr2=0x7efd380e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:37.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.500 7efd3fa68700 1 -- 172.21.2.101:0/1712513477 shutdown_connections 2021-02-10T11:28:37.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.500 7efd3fa68700 1 -- 172.21.2.101:0/1712513477 wait complete. 2021-02-10T11:28:37.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.515 7efd3fa68700 1 Processor -- start 2021-02-10T11:28:37.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.515 7efd3fa68700 1 -- start start 2021-02-10T11:28:37.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.515 7efd3fa68700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd380a6c90 0x7efd381924c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:37.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.515 7efd3fa68700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd38192ae0 0x7efd38192ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:37.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.515 7efd3fa68700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd380919e0 0x7efd38091de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:37.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.515 7efd3fa68700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3819d2e0 con 0x7efd38192ae0 2021-02-10T11:28:37.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.515 7efd3fa68700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd380914d0 con 0x7efd380919e0 2021-02-10T11:28:37.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.515 7efd3fa68700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd380a6820 con 0x7efd380a6c90 2021-02-10T11:28:37.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.515 7efd3d083700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd38192ae0 0x7efd38192ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:37.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.515 7efd3d083700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd38192ae0 0x7efd38192ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48208/0 (socket says 172.21.2.101:48208) 2021-02-10T11:28:37.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.515 7efd3d083700 1 -- 172.21.2.101:0/669926385 learned_addr learned my addr 172.21.2.101:0/669926385 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:28:37.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.516 7efd3d083700 1 -- 172.21.2.101:0/669926385 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd380a6c90 msgr2=0x7efd381924c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:37.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.516 7efd3d083700 1 --2- 172.21.2.101:0/669926385 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd380a6c90 0x7efd381924c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:37.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.516 7efd3d083700 1 -- 172.21.2.101:0/669926385 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd380919e0 msgr2=0x7efd38091de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:37.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.516 7efd3d083700 1 --2- 172.21.2.101:0/669926385 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd380919e0 0x7efd38091de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:37.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.516 7efd3d083700 1 -- 172.21.2.101:0/669926385 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd38092400 con 0x7efd38192ae0 2021-02-10T11:28:37.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.516 7efd3d083700 1 --2- 172.21.2.101:0/669926385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd38192ae0 0x7efd38192ee0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7efd34001c60 tx=0x7efd3400a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:37.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.516 7efd2a7fc700 1 -- 172.21.2.101:0/669926385 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd3400acc0 con 0x7efd38192ae0 2021-02-10T11:28:37.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.516 7efd2a7fc700 1 -- 172.21.2.101:0/669926385 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd34004350 con 0x7efd38192ae0 2021-02-10T11:28:37.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.516 7efd2a7fc700 1 -- 172.21.2.101:0/669926385 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd340047b0 con 0x7efd38192ae0 2021-02-10T11:28:37.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.516 7efd3fa68700 1 -- 172.21.2.101:0/669926385 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd38092400 con 0x7efd38192ae0 2021-02-10T11:28:37.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.516 7efd3fa68700 1 -- 172.21.2.101:0/669926385 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd38092400 con 0x7efd38192ae0 2021-02-10T11:28:37.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.516 7efd2a7fc700 1 -- 172.21.2.101:0/669926385 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7efd34004530 con 0x7efd38192ae0 2021-02-10T11:28:37.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.517 7efd2a7fc700 1 --2- 172.21.2.101:0/669926385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd200186c0 0x7efd2001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:37.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.517 7efd2a7fc700 1 -- 172.21.2.101:0/669926385 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7efd34032670 con 0x7efd38192ae0 2021-02-10T11:28:37.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.517 7efd3d884700 1 --2- 172.21.2.101:0/669926385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd200186c0 0x7efd2001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:37.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.517 7efd3d884700 1 --2- 172.21.2.101:0/669926385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd200186c0 0x7efd2001abc0 crc :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:37.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.548 7efd3fa68700 1 -- 172.21.2.101:0/669926385 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd380e52e0 con 0x7efd38192ae0 2021-02-10T11:28:37.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.551 7efd2a7fc700 1 -- 172.21.2.101:0/669926385 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7efd34004140 con 0x7efd38192ae0 2021-02-10T11:28:37.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.679 7efd3fa68700 1 -- 172.21.2.101:0/669926385 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839330", "target": ["mgr", ""]}) v1 -- 0x7efd380e52e0 con 0x7efd200186c0 2021-02-10T11:28:37.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.682 7efd2a7fc700 1 -- 172.21.2.101:0/669926385 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7efd2c000940 con 0x7efd200186c0 2021-02-10T11:28:37.697 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839330/21f1b4a9-38ae-43b1-a994-b466e25f3cc0 2021-02-10T11:28:37.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.688 7efd3fa68700 1 -- 172.21.2.101:0/669926385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd200186c0 msgr2=0x7efd2001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:37.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.688 7efd3fa68700 1 --2- 172.21.2.101:0/669926385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd200186c0 0x7efd2001abc0 crc :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:37.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.688 7efd3fa68700 1 -- 172.21.2.101:0/669926385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd38192ae0 msgr2=0x7efd38192ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:37.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.688 7efd3fa68700 1 --2- 172.21.2.101:0/669926385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd38192ae0 0x7efd38192ee0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7efd34001c60 tx=0x7efd3400a8d0).stop 2021-02-10T11:28:37.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.688 7efd3fa68700 1 -- 172.21.2.101:0/669926385 shutdown_connections 2021-02-10T11:28:37.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.688 7efd3fa68700 1 --2- 172.21.2.101:0/669926385 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd380919e0 0x7efd38091de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:37.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.688 7efd3fa68700 1 --2- 172.21.2.101:0/669926385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd38192ae0 0x7efd38192ee0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:37.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.688 7efd3fa68700 1 --2- 172.21.2.101:0/669926385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd200186c0 0x7efd2001abc0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:37.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.688 7efd3fa68700 1 --2- 172.21.2.101:0/669926385 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd380a6c90 0x7efd381924c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:37.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.688 7efd3fa68700 1 -- 172.21.2.101:0/669926385 >> 172.21.2.101:0/669926385 conn(0x7efd380ac550 msgr2=0x7efd380be1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:37.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.688 7efd3fa68700 1 -- 172.21.2.101:0/669926385 shutdown_connections 2021-02-10T11:28:37.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:37.688 7efd3fa68700 1 -- 172.21.2.101:0/669926385 wait complete. 2021-02-10T11:28:37.707 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839330 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839330/21f1b4a9-38ae-43b1-a994-b466e25f3cc0 2021-02-10T11:28:37.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:37.707 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:37.707 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:38.044 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:38.044 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:38.045 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839330/21f1b4a9-38ae-43b1-a994-b466e25f3cc0/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:38.426 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:38.426 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:38.426 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240382 s, 43.6 MB/s 2021-02-10T11:28:38.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:38.427 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:38.427 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:38.762 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:38.762 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:38.762 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839330/21f1b4a9-38ae-43b1-a994-b466e25f3cc0/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:39.134 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:39.134 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:39.134 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244975 s, 42.8 MB/s 2021-02-10T11:28:39.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:39.135 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:39.135 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:39.472 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:39.472 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:39.472 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839330/21f1b4a9-38ae-43b1-a994-b466e25f3cc0/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:39.849 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:39.849 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:39.849 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239521 s, 43.8 MB/s 2021-02-10T11:28:39.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:39.850 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:39.850 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:40.185 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:40.186 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:40.186 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839330/21f1b4a9-38ae-43b1-a994-b466e25f3cc0/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:40.563 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:40.563 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:40.563 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0255373 s, 41.1 MB/s 2021-02-10T11:28:40.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:40.564 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:40.564 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:40.899 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:40.900 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:40.900 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839330/21f1b4a9-38ae-43b1-a994-b466e25f3cc0/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:41.277 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:41.277 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:41.277 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238417 s, 44.0 MB/s 2021-02-10T11:28:41.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:41.278 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:41.278 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:41.614 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:41.614 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:41.615 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839330/21f1b4a9-38ae-43b1-a994-b466e25f3cc0/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:41.991 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:41.991 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:41.991 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251311 s, 41.7 MB/s 2021-02-10T11:28:41.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:41.992 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:41.992 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:42.327 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:42.327 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:42.327 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839330/21f1b4a9-38ae-43b1-a994-b466e25f3cc0/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:42.704 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:42.705 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:42.705 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242128 s, 43.3 MB/s 2021-02-10T11:28:42.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:42.706 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:42.706 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:43.043 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:43.043 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:43.044 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839330/21f1b4a9-38ae-43b1-a994-b466e25f3cc0/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:43.420 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:43.420 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:43.420 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0223537 s, 46.9 MB/s 2021-02-10T11:28:43.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:43.421 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:43.421 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:43.759 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:43.760 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:43.760 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839330/21f1b4a9-38ae-43b1-a994-b466e25f3cc0/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:44.134 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:44.134 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:44.134 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244945 s, 42.8 MB/s 2021-02-10T11:28:44.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:44.135 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:44.135 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:44.471 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:44.471 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:44.472 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839330/21f1b4a9-38ae-43b1-a994-b466e25f3cc0/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:44.850 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:44.850 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:44.850 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0254353 s, 41.2 MB/s 2021-02-10T11:28:44.851 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839331 2021-02-10T11:28:45.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.256 7fbdafba9700 1 Processor -- start 2021-02-10T11:28:45.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.256 7fbdafba9700 1 -- start start 2021-02-10T11:28:45.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.256 7fbdafba9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbda81b0870 0x7fbda80f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:45.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.257 7fbdafba9700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbda80efe00 0x7fbda80f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:45.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.257 7fbdafba9700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda81b0c70 con 0x7fbda81b0470 2021-02-10T11:28:45.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.257 7fbdafba9700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda80f0b50 con 0x7fbda80efe00 2021-02-10T11:28:45.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.257 7fbdafba9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda80f0820 con 0x7fbda81b0870 2021-02-10T11:28:45.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.257 7fbdae1c6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbda80efe00 0x7fbda80f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:45.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.257 7fbdae1c6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbda80efe00 0x7fbda80f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48216/0 (socket says 172.21.2.101:48216) 2021-02-10T11:28:45.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.257 7fbdae1c6700 1 -- 172.21.2.101:0/68123795 learned_addr learned my addr 172.21.2.101:0/68123795 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:28:45.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.257 7fbdad1c4700 1 --2- 172.21.2.101:0/68123795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbda81b0870 0x7fbda80f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:45.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.257 7fbdae1c6700 1 -- 172.21.2.101:0/68123795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbda81b0870 msgr2=0x7fbda80f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:28:45.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.257 7fbdae1c6700 1 --2- 172.21.2.101:0/68123795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbda81b0870 0x7fbda80f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:45.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.257 7fbdae1c6700 1 -- 172.21.2.101:0/68123795 >> v1:172.21.2.108:6789/0 conn(0x7fbda81b0470 legacy=0x7fbda80ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:45.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.257 7fbdae1c6700 1 -- 172.21.2.101:0/68123795 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda80e52e0 con 0x7fbda80efe00 2021-02-10T11:28:45.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.257 7fbdae1c6700 1 --2- 172.21.2.101:0/68123795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbda80efe00 0x7fbda80f0200 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7fbda0006f00 tx=0x7fbda0009f10).ready entity=mon.0 client_cookie=7948fff3ea159d2f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:45.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.257 7fbdac9c3700 1 -- 172.21.2.101:0/68123795 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbda000b3b0 con 0x7fbda80efe00 2021-02-10T11:28:45.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.258 7fbdac9c3700 1 -- 172.21.2.101:0/68123795 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda000b730 con 0x7fbda80efe00 2021-02-10T11:28:45.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.258 7fbdac9c3700 1 -- 172.21.2.101:0/68123795 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbda000bc10 con 0x7fbda80efe00 2021-02-10T11:28:45.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.258 7fbdafba9700 1 -- 172.21.2.101:0/68123795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbda80efe00 msgr2=0x7fbda80f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:45.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.258 7fbdafba9700 1 --2- 172.21.2.101:0/68123795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbda80efe00 0x7fbda80f0200 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7fbda0006f00 tx=0x7fbda0009f10).stop 2021-02-10T11:28:45.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.258 7fbdafba9700 1 -- 172.21.2.101:0/68123795 shutdown_connections 2021-02-10T11:28:45.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.258 7fbdafba9700 1 --2- 172.21.2.101:0/68123795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbda81b0870 0x7fbda80f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:45.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.258 7fbdafba9700 1 --2- 172.21.2.101:0/68123795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbda80efe00 0x7fbda80f0200 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:45.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.258 7fbdafba9700 1 -- 172.21.2.101:0/68123795 >> 172.21.2.101:0/68123795 conn(0x7fbda80ce3e0 msgr2=0x7fbda80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:45.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.259 7fbdafba9700 1 -- 172.21.2.101:0/68123795 shutdown_connections 2021-02-10T11:28:45.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.259 7fbdafba9700 1 -- 172.21.2.101:0/68123795 wait complete. 2021-02-10T11:28:45.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.273 7fbdafba9700 1 Processor -- start 2021-02-10T11:28:45.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.273 7fbdafba9700 1 -- start start 2021-02-10T11:28:45.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.273 7fbdafba9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbda80a0420 0x7fbda80a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:45.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.273 7fbdafba9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbda8189d00 0x7fbda818a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:45.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.273 7fbdafba9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbda818a720 0x7fbda80b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:45.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.273 7fbdafba9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda8189780 con 0x7fbda8189d00 2021-02-10T11:28:45.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.273 7fbdafba9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda818ab20 con 0x7fbda818a720 2021-02-10T11:28:45.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.273 7fbdafba9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda80a1140 con 0x7fbda80a0420 2021-02-10T11:28:45.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.273 7fbdad1c4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbda8189d00 0x7fbda818a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:45.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.273 7fbdad1c4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbda8189d00 0x7fbda818a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48220/0 (socket says 172.21.2.101:48220) 2021-02-10T11:28:45.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.273 7fbdad1c4700 1 -- 172.21.2.101:0/3203291598 learned_addr learned my addr 172.21.2.101:0/3203291598 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:28:45.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.274 7fbdad1c4700 1 -- 172.21.2.101:0/3203291598 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbda80a0420 msgr2=0x7fbda80a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:45.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.274 7fbdad9c5700 1 --2- 172.21.2.101:0/3203291598 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbda80a0420 0x7fbda80a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:45.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.274 7fbdad1c4700 1 --2- 172.21.2.101:0/3203291598 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbda80a0420 0x7fbda80a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:45.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.274 7fbdad1c4700 1 -- 172.21.2.101:0/3203291598 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbda818a720 msgr2=0x7fbda80b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:45.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.274 7fbdad1c4700 1 --2- 172.21.2.101:0/3203291598 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbda818a720 0x7fbda80b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:45.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.274 7fbdad1c4700 1 -- 172.21.2.101:0/3203291598 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda80b8200 con 0x7fbda8189d00 2021-02-10T11:28:45.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.274 7fbdad1c4700 1 --2- 172.21.2.101:0/3203291598 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbda8189d00 0x7fbda818a100 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7fbd9c006f00 tx=0x7fbd9c00b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:45.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.274 7fbd9a7fc700 1 -- 172.21.2.101:0/3203291598 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbd9c00bd30 con 0x7fbda8189d00 2021-02-10T11:28:45.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.274 7fbd9a7fc700 1 -- 172.21.2.101:0/3203291598 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c002030 con 0x7fbda8189d00 2021-02-10T11:28:45.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.274 7fbd9a7fc700 1 -- 172.21.2.101:0/3203291598 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbd9c002510 con 0x7fbda8189d00 2021-02-10T11:28:45.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.274 7fbdafba9700 1 -- 172.21.2.101:0/3203291598 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda80b8200 con 0x7fbda8189d00 2021-02-10T11:28:45.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.274 7fbdafba9700 1 -- 172.21.2.101:0/3203291598 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda80b8200 con 0x7fbda8189d00 2021-02-10T11:28:45.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.275 7fbd9a7fc700 1 -- 172.21.2.101:0/3203291598 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbd9c000c90 con 0x7fbda8189d00 2021-02-10T11:28:45.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.275 7fbd9a7fc700 1 --2- 172.21.2.101:0/3203291598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd900186c0 0x7fbd9001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:45.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.275 7fbd9a7fc700 1 -- 172.21.2.101:0/3203291598 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fbd9c0327d0 con 0x7fbda8189d00 2021-02-10T11:28:45.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.275 7fbdad9c5700 1 --2- 172.21.2.101:0/3203291598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd900186c0 0x7fbd9001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:45.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.275 7fbdad9c5700 1 --2- 172.21.2.101:0/3203291598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd900186c0 0x7fbd9001abc0 crc :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:45.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.306 7fbdafba9700 1 -- 172.21.2.101:0/3203291598 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda8105ae0 con 0x7fbda8189d00 2021-02-10T11:28:45.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.309 7fbd9a7fc700 1 -- 172.21.2.101:0/3203291598 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbd9c01f510 con 0x7fbda8189d00 2021-02-10T11:28:45.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.434 7fbdafba9700 1 -- 172.21.2.101:0/3203291598 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839331", "target": ["mgr", ""]}) v1 -- 0x7fbda8105ae0 con 0x7fbd900186c0 2021-02-10T11:28:45.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.465 7fbd9a7fc700 1 -- 172.21.2.101:0/3203291598 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fbda40067d0 con 0x7fbd900186c0 2021-02-10T11:28:45.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.467 7fbdafba9700 1 -- 172.21.2.101:0/3203291598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd900186c0 msgr2=0x7fbd9001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:45.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.467 7fbdafba9700 1 --2- 172.21.2.101:0/3203291598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd900186c0 0x7fbd9001abc0 crc :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:45.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.467 7fbdafba9700 1 -- 172.21.2.101:0/3203291598 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbda8189d00 msgr2=0x7fbda818a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:45.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.467 7fbdafba9700 1 --2- 172.21.2.101:0/3203291598 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbda8189d00 0x7fbda818a100 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7fbd9c006f00 tx=0x7fbd9c00b940).stop 2021-02-10T11:28:45.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.468 7fbdafba9700 1 -- 172.21.2.101:0/3203291598 shutdown_connections 2021-02-10T11:28:45.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.468 7fbdafba9700 1 --2- 172.21.2.101:0/3203291598 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbda818a720 0x7fbda80b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:45.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.468 7fbdafba9700 1 --2- 172.21.2.101:0/3203291598 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbda8189d00 0x7fbda818a100 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:45.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.468 7fbdafba9700 1 --2- 172.21.2.101:0/3203291598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd900186c0 0x7fbd9001abc0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:45.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.468 7fbdafba9700 1 --2- 172.21.2.101:0/3203291598 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbda80a0420 0x7fbda80a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:45.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.468 7fbdafba9700 1 -- 172.21.2.101:0/3203291598 >> 172.21.2.101:0/3203291598 conn(0x7fbda8094e10 msgr2=0x7fbda80a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:45.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.468 7fbdafba9700 1 -- 172.21.2.101:0/3203291598 shutdown_connections 2021-02-10T11:28:45.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.468 7fbdafba9700 1 -- 172.21.2.101:0/3203291598 wait complete. 2021-02-10T11:28:45.487 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839331 2021-02-10T11:28:45.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.891 7f53d0ac8700 1 Processor -- start 2021-02-10T11:28:45.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.891 7f53d0ac8700 1 -- start start 2021-02-10T11:28:45.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.892 7f53d0ac8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f53cc1b0870 0x7f53cc0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:45.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.892 7f53d0ac8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f53cc0efe00 0x7f53cc0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:45.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.892 7f53d0ac8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53cc1b0c70 con 0x7f53cc1b0470 2021-02-10T11:28:45.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.892 7f53d0ac8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53cc0e5ea0 con 0x7f53cc0efe00 2021-02-10T11:28:45.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.892 7f53d0ac8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53cc0f0b50 con 0x7f53cc1b0870 2021-02-10T11:28:45.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.892 7f53cb05f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f53cc0efe00 0x7f53cc0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:45.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.892 7f53cb05f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f53cc0efe00 0x7f53cc0f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48230/0 (socket says 172.21.2.101:48230) 2021-02-10T11:28:45.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.892 7f53cb05f700 1 -- 172.21.2.101:0/1901009341 learned_addr learned my addr 172.21.2.101:0/1901009341 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:28:45.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.892 7f53cb05f700 1 -- 172.21.2.101:0/1901009341 >> v2:172.21.2.108:3300/0 conn(0x7f53cc1b0870 msgr2=0x7f53cc0f20e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:28:45.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.892 7f53cb05f700 1 --2- 172.21.2.101:0/1901009341 >> v2:172.21.2.108:3300/0 conn(0x7f53cc1b0870 0x7f53cc0f20e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:45.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.892 7f53cb05f700 1 -- 172.21.2.101:0/1901009341 >> v1:172.21.2.108:6789/0 conn(0x7f53cc1b0470 legacy=0x7f53cc0ef610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:28:45.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.892 7f53cb05f700 1 -- 172.21.2.101:0/1901009341 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53cc0f0820 con 0x7f53cc0efe00 2021-02-10T11:28:45.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.892 7f53cb05f700 1 --2- 172.21.2.101:0/1901009341 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53cc0efe00 0x7f53cc0f0200 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f53c4006f00 tx=0x7f53c4009f10).ready entity=mon.0 client_cookie=79755d07aa89c125 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:45.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.892 7f53c985c700 1 -- 172.21.2.101:0/1901009341 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f53c400b3b0 con 0x7f53cc0efe00 2021-02-10T11:28:45.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.892 7f53c985c700 1 -- 172.21.2.101:0/1901009341 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c4000d30 con 0x7f53cc0efe00 2021-02-10T11:28:45.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.892 7f53c985c700 1 -- 172.21.2.101:0/1901009341 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f53c4000f90 con 0x7f53cc0efe00 2021-02-10T11:28:45.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.892 7f53d0ac8700 1 -- 172.21.2.101:0/1901009341 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53cc0efe00 msgr2=0x7f53cc0f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:45.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.892 7f53d0ac8700 1 --2- 172.21.2.101:0/1901009341 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53cc0efe00 0x7f53cc0f0200 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f53c4006f00 tx=0x7f53c4009f10).stop 2021-02-10T11:28:45.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.893 7f53d0ac8700 1 -- 172.21.2.101:0/1901009341 shutdown_connections 2021-02-10T11:28:45.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.893 7f53d0ac8700 1 --2- 172.21.2.101:0/1901009341 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53cc0efe00 0x7f53cc0f0200 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:45.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.893 7f53d0ac8700 1 --2- 172.21.2.101:0/1901009341 >> v2:172.21.2.108:3300/0 conn(0x7f53cc1b0870 0x7f53cc0f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:45.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.893 7f53d0ac8700 1 -- 172.21.2.101:0/1901009341 >> 172.21.2.101:0/1901009341 conn(0x7f53cc0ce3e0 msgr2=0x7f53cc0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:45.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.893 7f53d0ac8700 1 -- 172.21.2.101:0/1901009341 shutdown_connections 2021-02-10T11:28:45.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.893 7f53d0ac8700 1 -- 172.21.2.101:0/1901009341 wait complete. 2021-02-10T11:28:45.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.907 7f53d0ac8700 1 Processor -- start 2021-02-10T11:28:45.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.907 7f53d0ac8700 1 -- start start 2021-02-10T11:28:45.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.908 7f53d0ac8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53cc0b2840 0x7f53cc1a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:45.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.908 7f53d0ac8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53cc0b30d0 0x7f53cc0b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:45.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.908 7f53d0ac8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53cc0b3af0 0x7f53cc0f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:45.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.908 7f53d0ac8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc0b2c40 con 0x7f53cc0b3af0 2021-02-10T11:28:45.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.908 7f53d0ac8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc1a4920 con 0x7f53cc0b2840 2021-02-10T11:28:45.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.908 7f53d0ac8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc0f0ba0 con 0x7f53cc0b30d0 2021-02-10T11:28:45.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.908 7f53cb05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53cc0b3af0 0x7f53cc0f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:45.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.908 7f53cb05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53cc0b3af0 0x7f53cc0f0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48236/0 (socket says 172.21.2.101:48236) 2021-02-10T11:28:45.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.908 7f53cb05f700 1 -- 172.21.2.101:0/2294459267 learned_addr learned my addr 172.21.2.101:0/2294459267 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:28:45.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.908 7f53cb05f700 1 -- 172.21.2.101:0/2294459267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53cc0b30d0 msgr2=0x7f53cc0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:45.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.908 7f53cb05f700 1 --2- 172.21.2.101:0/2294459267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53cc0b30d0 0x7f53cc0b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:45.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.908 7f53cb05f700 1 -- 172.21.2.101:0/2294459267 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53cc0b2840 msgr2=0x7f53cc1a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:45.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.908 7f53cb05f700 1 --2- 172.21.2.101:0/2294459267 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53cc0b2840 0x7f53cc1a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:45.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.908 7f53cb05f700 1 -- 172.21.2.101:0/2294459267 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53cc0f0dc0 con 0x7f53cc0b3af0 2021-02-10T11:28:45.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.909 7f53cb05f700 1 --2- 172.21.2.101:0/2294459267 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53cc0b3af0 0x7f53cc0f0580 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7f53c4009690 tx=0x7f53c400bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:45.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.909 7f53c37fe700 1 -- 172.21.2.101:0/2294459267 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f53c4003030 con 0x7f53cc0b3af0 2021-02-10T11:28:45.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.909 7f53c37fe700 1 -- 172.21.2.101:0/2294459267 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c4000f90 con 0x7f53cc0b3af0 2021-02-10T11:28:45.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.909 7f53d0ac8700 1 -- 172.21.2.101:0/2294459267 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53cc0f0dc0 con 0x7f53cc0b3af0 2021-02-10T11:28:45.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.909 7f53d0ac8700 1 -- 172.21.2.101:0/2294459267 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53cc0f1350 con 0x7f53cc0b3af0 2021-02-10T11:28:45.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.909 7f53c37fe700 1 -- 172.21.2.101:0/2294459267 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f53c4001480 con 0x7f53cc0b3af0 2021-02-10T11:28:45.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.909 7f53c37fe700 1 -- 172.21.2.101:0/2294459267 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f53c40096d0 con 0x7f53cc0b3af0 2021-02-10T11:28:45.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.910 7f53c37fe700 1 --2- 172.21.2.101:0/2294459267 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f53b00186c0 0x7f53b001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:45.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.910 7f53c37fe700 1 -- 172.21.2.101:0/2294459267 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f53c4032b40 con 0x7f53cc0b3af0 2021-02-10T11:28:45.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.910 7f53ca85e700 1 --2- 172.21.2.101:0/2294459267 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f53b00186c0 0x7f53b001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:45.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.910 7f53ca85e700 1 --2- 172.21.2.101:0/2294459267 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f53b00186c0 0x7f53b001abc0 crc :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:45.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.942 7f53d0ac8700 1 -- 172.21.2.101:0/2294459267 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53cc0f18f0 con 0x7f53cc0b3af0 2021-02-10T11:28:45.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:45.945 7f53c37fe700 1 -- 172.21.2.101:0/2294459267 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f53c4001640 con 0x7f53cc0b3af0 2021-02-10T11:28:46.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:46.073 7f53d0ac8700 1 -- 172.21.2.101:0/2294459267 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839331", "target": ["mgr", ""]}) v1 -- 0x7f53cc0f1350 con 0x7f53b00186c0 2021-02-10T11:28:46.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:46.077 7f53c37fe700 1 -- 172.21.2.101:0/2294459267 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f53bc006e40 con 0x7f53b00186c0 2021-02-10T11:28:46.092 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839331/77152b2e-9041-41b3-a578-84d742367487 2021-02-10T11:28:46.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:46.082 7f53d0ac8700 1 -- 172.21.2.101:0/2294459267 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f53b00186c0 msgr2=0x7f53b001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:46.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:46.082 7f53d0ac8700 1 --2- 172.21.2.101:0/2294459267 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f53b00186c0 0x7f53b001abc0 crc :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:46.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:46.082 7f53d0ac8700 1 -- 172.21.2.101:0/2294459267 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53cc0b3af0 msgr2=0x7f53cc0f0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:46.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:46.082 7f53d0ac8700 1 --2- 172.21.2.101:0/2294459267 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53cc0b3af0 0x7f53cc0f0580 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7f53c4009690 tx=0x7f53c400bd10).stop 2021-02-10T11:28:46.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:46.082 7f53d0ac8700 1 -- 172.21.2.101:0/2294459267 shutdown_connections 2021-02-10T11:28:46.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:46.082 7f53d0ac8700 1 --2- 172.21.2.101:0/2294459267 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f53b00186c0 0x7f53b001abc0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:46.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:46.082 7f53d0ac8700 1 --2- 172.21.2.101:0/2294459267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53cc0b30d0 0x7f53cc0b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:46.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:46.082 7f53d0ac8700 1 --2- 172.21.2.101:0/2294459267 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53cc0b3af0 0x7f53cc0f0580 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:46.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:46.082 7f53d0ac8700 1 --2- 172.21.2.101:0/2294459267 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53cc0b2840 0x7f53cc1a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:46.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:46.082 7f53d0ac8700 1 -- 172.21.2.101:0/2294459267 >> 172.21.2.101:0/2294459267 conn(0x7f53cc0c36f0 msgr2=0x7f53cc094c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:46.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:46.082 7f53d0ac8700 1 -- 172.21.2.101:0/2294459267 shutdown_connections 2021-02-10T11:28:46.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:46.082 7f53d0ac8700 1 -- 172.21.2.101:0/2294459267 wait complete. 2021-02-10T11:28:46.101 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839331 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839331/77152b2e-9041-41b3-a578-84d742367487 2021-02-10T11:28:46.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:46.101 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:46.102 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:46.438 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:46.439 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:46.439 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839331/77152b2e-9041-41b3-a578-84d742367487/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:46.823 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:46.823 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:46.823 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246483 s, 42.5 MB/s 2021-02-10T11:28:46.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:46.824 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:46.825 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:47.161 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:47.161 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:47.161 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839331/77152b2e-9041-41b3-a578-84d742367487/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:47.535 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:47.535 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:47.535 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237993 s, 44.1 MB/s 2021-02-10T11:28:47.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:47.536 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:47.536 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:47.871 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:47.872 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:47.872 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839331/77152b2e-9041-41b3-a578-84d742367487/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:48.249 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:48.249 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:48.249 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241578 s, 43.4 MB/s 2021-02-10T11:28:48.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:48.250 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:48.251 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:48.585 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:48.586 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:48.586 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839331/77152b2e-9041-41b3-a578-84d742367487/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:48.958 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:48.958 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:48.958 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231143 s, 45.4 MB/s 2021-02-10T11:28:48.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:48.959 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:48.959 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:49.294 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:49.294 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:49.295 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839331/77152b2e-9041-41b3-a578-84d742367487/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:49.671 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:49.672 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:49.672 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247709 s, 42.3 MB/s 2021-02-10T11:28:49.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:49.673 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:49.674 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:50.009 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:50.010 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:50.010 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839331/77152b2e-9041-41b3-a578-84d742367487/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:50.385 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:50.385 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:50.385 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0228711 s, 45.8 MB/s 2021-02-10T11:28:50.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:50.387 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:50.387 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:50.723 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:50.723 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:50.723 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839331/77152b2e-9041-41b3-a578-84d742367487/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:51.099 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:51.099 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:51.099 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241467 s, 43.4 MB/s 2021-02-10T11:28:51.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:51.100 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:51.101 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:51.437 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:51.437 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:51.437 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839331/77152b2e-9041-41b3-a578-84d742367487/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:51.815 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:51.815 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:51.815 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245108 s, 42.8 MB/s 2021-02-10T11:28:51.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:51.816 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:51.816 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:52.151 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:52.152 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:52.152 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839331/77152b2e-9041-41b3-a578-84d742367487/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:52.529 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:52.529 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:52.529 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241462 s, 43.4 MB/s 2021-02-10T11:28:52.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:52.530 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:52.530 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:52.864 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:52.865 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:52.865 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839331/77152b2e-9041-41b3-a578-84d742367487/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:53.242 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:53.243 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:53.243 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251918 s, 41.6 MB/s 2021-02-10T11:28:53.244 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839332 2021-02-10T11:28:53.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.649 7f7a75352700 1 Processor -- start 2021-02-10T11:28:53.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.649 7f7a75352700 1 -- start start 2021-02-10T11:28:53.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.649 7f7a75352700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7a700f05f0 0x7f7a700f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:53.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.649 7f7a75352700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a701b0c70 con 0x7f7a701b0870 2021-02-10T11:28:53.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.649 7f7a75352700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a700f0b50 con 0x7f7a701b0470 2021-02-10T11:28:53.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.649 7f7a75352700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a700e5ea0 con 0x7f7a700f05f0 2021-02-10T11:28:53.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.649 7f7a6f87e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7a700f05f0 0x7f7a700f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:53.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.649 7f7a6f87e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7a700f05f0 0x7f7a700f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48244/0 (socket says 172.21.2.101:48244) 2021-02-10T11:28:53.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.649 7f7a6f87e700 1 -- 172.21.2.101:0/2198125830 learned_addr learned my addr 172.21.2.101:0/2198125830 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:28:53.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.650 7f7a6f87e700 1 -- 172.21.2.101:0/2198125830 >> v1:172.21.2.108:6789/0 conn(0x7f7a701b0470 legacy=0x7f7a700ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:28:53.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.650 7f7a6f87e700 1 -- 172.21.2.101:0/2198125830 >> v1:172.21.2.101:6789/0 conn(0x7f7a701b0870 legacy=0x7f7a700efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:53.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.650 7f7a6f87e700 1 -- 172.21.2.101:0/2198125830 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a700e52e0 con 0x7f7a700f05f0 2021-02-10T11:28:53.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.650 7f7a6f87e700 1 --2- 172.21.2.101:0/2198125830 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a700f05f0 0x7f7a700f20e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f7a68006f00 tx=0x7f7a68009e90).ready entity=mon.0 client_cookie=29fd427ebc3ca26f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:53.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.650 7f7a6e07b700 1 -- 172.21.2.101:0/2198125830 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7a6800b300 con 0x7f7a700f05f0 2021-02-10T11:28:53.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.650 7f7a6e07b700 1 -- 172.21.2.101:0/2198125830 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6800b580 con 0x7f7a700f05f0 2021-02-10T11:28:53.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.650 7f7a6e07b700 1 -- 172.21.2.101:0/2198125830 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7a6800b9d0 con 0x7f7a700f05f0 2021-02-10T11:28:53.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.650 7f7a75352700 1 -- 172.21.2.101:0/2198125830 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a700f05f0 msgr2=0x7f7a700f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:53.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.650 7f7a75352700 1 --2- 172.21.2.101:0/2198125830 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a700f05f0 0x7f7a700f20e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f7a68006f00 tx=0x7f7a68009e90).stop 2021-02-10T11:28:53.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.651 7f7a75352700 1 -- 172.21.2.101:0/2198125830 shutdown_connections 2021-02-10T11:28:53.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.651 7f7a75352700 1 --2- 172.21.2.101:0/2198125830 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a700f05f0 0x7f7a700f20e0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:53.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.651 7f7a75352700 1 -- 172.21.2.101:0/2198125830 >> 172.21.2.101:0/2198125830 conn(0x7f7a700ce3e0 msgr2=0x7f7a700e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:53.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.651 7f7a75352700 1 -- 172.21.2.101:0/2198125830 shutdown_connections 2021-02-10T11:28:53.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.651 7f7a75352700 1 -- 172.21.2.101:0/2198125830 wait complete. 2021-02-10T11:28:53.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.666 7f7a75352700 1 Processor -- start 2021-02-10T11:28:53.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.666 7f7a75352700 1 -- start start 2021-02-10T11:28:53.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.666 7f7a75352700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a700a3b50 0x7f7a701b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.666 7f7a75352700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a701c25b0 0x7f7a701b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.666 7f7a75352700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a701b4350 0x7f7a701b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.666 7f7a75352700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a701b3190 con 0x7f7a701c25b0 2021-02-10T11:28:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.666 7f7a75352700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a70087750 con 0x7f7a701b4350 2021-02-10T11:28:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.666 7f7a75352700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a700a6bf0 con 0x7f7a700a3b50 2021-02-10T11:28:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.666 7f7a6e87c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a701c25b0 0x7f7a701b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.666 7f7a6e87c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a701c25b0 0x7f7a701b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48248/0 (socket says 172.21.2.101:48248) 2021-02-10T11:28:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.666 7f7a6e87c700 1 -- 172.21.2.101:0/1531451302 learned_addr learned my addr 172.21.2.101:0/1531451302 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:28:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.667 7f7a6f07d700 1 --2- 172.21.2.101:0/1531451302 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a700a3b50 0x7f7a701b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.667 7f7a6f87e700 1 --2- 172.21.2.101:0/1531451302 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a701b4350 0x7f7a701b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.667 7f7a6e87c700 1 -- 172.21.2.101:0/1531451302 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a700a3b50 msgr2=0x7f7a701b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.667 7f7a6e87c700 1 --2- 172.21.2.101:0/1531451302 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a700a3b50 0x7f7a701b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.667 7f7a6e87c700 1 -- 172.21.2.101:0/1531451302 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a701b4350 msgr2=0x7f7a701b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.667 7f7a6e87c700 1 --2- 172.21.2.101:0/1531451302 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a701b4350 0x7f7a701b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.667 7f7a6e87c700 1 -- 172.21.2.101:0/1531451302 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a701b1ec0 con 0x7f7a701c25b0 2021-02-10T11:28:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.667 7f7a6f07d700 1 --2- 172.21.2.101:0/1531451302 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a700a3b50 0x7f7a701b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:28:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.667 7f7a6e87c700 1 --2- 172.21.2.101:0/1531451302 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a701c25b0 0x7f7a701b3d30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f7a60006ed0 tx=0x7f7a6000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.667 7f7a67fff700 1 -- 172.21.2.101:0/1531451302 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7a60006240 con 0x7f7a701c25b0 2021-02-10T11:28:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.667 7f7a67fff700 1 -- 172.21.2.101:0/1531451302 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a600009b0 con 0x7f7a701c25b0 2021-02-10T11:28:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.667 7f7a75352700 1 -- 172.21.2.101:0/1531451302 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a701b1ec0 con 0x7f7a701c25b0 2021-02-10T11:28:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.667 7f7a67fff700 1 -- 172.21.2.101:0/1531451302 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7a600056a0 con 0x7f7a701c25b0 2021-02-10T11:28:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.667 7f7a75352700 1 -- 172.21.2.101:0/1531451302 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a701b2450 con 0x7f7a701c25b0 2021-02-10T11:28:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.668 7f7a67fff700 1 -- 172.21.2.101:0/1531451302 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7a600054b0 con 0x7f7a701c25b0 2021-02-10T11:28:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.668 7f7a67fff700 1 --2- 172.21.2.101:0/1531451302 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a540186c0 0x7f7a5401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.668 7f7a67fff700 1 -- 172.21.2.101:0/1531451302 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7a60032400 con 0x7f7a701c25b0 2021-02-10T11:28:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.668 7f7a6f07d700 1 --2- 172.21.2.101:0/1531451302 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a540186c0 0x7f7a5401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:53.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.668 7f7a6f07d700 1 --2- 172.21.2.101:0/1531451302 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a540186c0 0x7f7a5401abc0 crc :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:53.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.699 7f7a75352700 1 -- 172.21.2.101:0/1531451302 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a7007e6b0 con 0x7f7a701c25b0 2021-02-10T11:28:53.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.702 7f7a67fff700 1 -- 172.21.2.101:0/1531451302 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7a60030d30 con 0x7f7a701c25b0 2021-02-10T11:28:53.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.833 7f7a75352700 1 -- 172.21.2.101:0/1531451302 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839332", "target": ["mgr", ""]}) v1 -- 0x7f7a700ce3e0 con 0x7f7a540186c0 2021-02-10T11:28:53.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.857 7f7a67fff700 1 -- 172.21.2.101:0/1531451302 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f7a58006930 con 0x7f7a540186c0 2021-02-10T11:28:53.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.865 7f7a75352700 1 -- 172.21.2.101:0/1531451302 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a540186c0 msgr2=0x7f7a5401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:53.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.865 7f7a75352700 1 --2- 172.21.2.101:0/1531451302 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a540186c0 0x7f7a5401abc0 crc :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:53.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.866 7f7a75352700 1 -- 172.21.2.101:0/1531451302 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a701c25b0 msgr2=0x7f7a701b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:53.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.866 7f7a75352700 1 --2- 172.21.2.101:0/1531451302 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a701c25b0 0x7f7a701b3d30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f7a60006ed0 tx=0x7f7a6000b960).stop 2021-02-10T11:28:53.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.866 7f7a75352700 1 -- 172.21.2.101:0/1531451302 shutdown_connections 2021-02-10T11:28:53.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.866 7f7a75352700 1 --2- 172.21.2.101:0/1531451302 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a701b4350 0x7f7a701b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:53.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.866 7f7a75352700 1 --2- 172.21.2.101:0/1531451302 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a701c25b0 0x7f7a701b3d30 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:53.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.866 7f7a75352700 1 --2- 172.21.2.101:0/1531451302 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a540186c0 0x7f7a5401abc0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:53.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.866 7f7a75352700 1 --2- 172.21.2.101:0/1531451302 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a700a3b50 0x7f7a701b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:53.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.866 7f7a75352700 1 -- 172.21.2.101:0/1531451302 >> 172.21.2.101:0/1531451302 conn(0x7f7a700886d0 msgr2=0x7f7a70081c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:53.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.866 7f7a75352700 1 -- 172.21.2.101:0/1531451302 shutdown_connections 2021-02-10T11:28:53.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:53.866 7f7a75352700 1 -- 172.21.2.101:0/1531451302 wait complete. 2021-02-10T11:28:53.885 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839332 2021-02-10T11:28:54.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.290 7f5b682fa700 1 Processor -- start 2021-02-10T11:28:54.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.290 7f5b682fa700 1 -- start start 2021-02-10T11:28:54.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.291 7f5b682fa700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b601b0470 0x7f5b601b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:54.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.291 7f5b682fa700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b600f0b50 con 0x7f5b600ef610 2021-02-10T11:28:54.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.291 7f5b682fa700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b600f24e0 con 0x7f5b600f20e0 2021-02-10T11:28:54.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.291 7f5b682fa700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b600e5ea0 con 0x7f5b601b0470 2021-02-10T11:28:54.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.291 7f5b65915700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f5b600ef610 0x7f5b600efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48748/0 (socket says 172.21.2.101:48748) 2021-02-10T11:28:54.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.291 7f5b65915700 1 -- 172.21.2.101:0/1452303021 learned_addr learned my addr 172.21.2.101:0/1452303021 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:28:54.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.291 7f5b66116700 1 --2- 172.21.2.101:0/1452303021 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b601b0470 0x7f5b601b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:54.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.291 7f5b66116700 1 -- 172.21.2.101:0/1452303021 >> v1:172.21.2.108:6789/0 conn(0x7f5b600f20e0 legacy=0x7f5b600f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:54.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.292 7f5b66116700 1 -- 172.21.2.101:0/1452303021 >> v1:172.21.2.101:6789/0 conn(0x7f5b600ef610 legacy=0x7f5b600efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:54.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.292 7f5b66116700 1 -- 172.21.2.101:0/1452303021 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b600e52e0 con 0x7f5b601b0470 2021-02-10T11:28:54.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.292 7f5b66116700 1 --2- 172.21.2.101:0/1452303021 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b601b0470 0x7f5b601b0870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f5b5c006650 tx=0x7f5b5c009a80).ready entity=mon.2 client_cookie=6840600513c003a0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:54.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.292 7f5b65114700 1 -- 172.21.2.101:0/1452303021 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5b5c00c0c0 con 0x7f5b601b0470 2021-02-10T11:28:54.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.292 7f5b65114700 1 -- 172.21.2.101:0/1452303021 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5c00c340 con 0x7f5b601b0470 2021-02-10T11:28:54.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.292 7f5b65114700 1 -- 172.21.2.101:0/1452303021 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5b5c00c860 con 0x7f5b601b0470 2021-02-10T11:28:54.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.292 7f5b682fa700 1 -- 172.21.2.101:0/1452303021 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b601b0470 msgr2=0x7f5b601b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:54.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.292 7f5b682fa700 1 --2- 172.21.2.101:0/1452303021 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b601b0470 0x7f5b601b0870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f5b5c006650 tx=0x7f5b5c009a80).stop 2021-02-10T11:28:54.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.292 7f5b682fa700 1 -- 172.21.2.101:0/1452303021 shutdown_connections 2021-02-10T11:28:54.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.292 7f5b682fa700 1 --2- 172.21.2.101:0/1452303021 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b601b0470 0x7f5b601b0870 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:54.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.292 7f5b682fa700 1 -- 172.21.2.101:0/1452303021 >> 172.21.2.101:0/1452303021 conn(0x7f5b600ce3e0 msgr2=0x7f5b600e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:54.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.292 7f5b682fa700 1 -- 172.21.2.101:0/1452303021 shutdown_connections 2021-02-10T11:28:54.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.292 7f5b682fa700 1 -- 172.21.2.101:0/1452303021 wait complete. 2021-02-10T11:28:54.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.307 7f5b682fa700 1 Processor -- start 2021-02-10T11:28:54.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.307 7f5b682fa700 1 -- start start 2021-02-10T11:28:54.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.307 7f5b682fa700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5b600a4ec0 0x7f5b600a3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:54.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.307 7f5b682fa700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b600a4520 0x7f5b60092450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:54.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.307 7f5b682fa700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b600a4920 0x7f5b60092a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:54.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.307 7f5b682fa700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b60091ed0 con 0x7f5b600a4520 2021-02-10T11:28:54.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.307 7f5b682fa700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b600a5440 con 0x7f5b600a4ec0 2021-02-10T11:28:54.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.307 7f5b682fa700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6008aaa0 con 0x7f5b600a4920 2021-02-10T11:28:54.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.308 7f5b65915700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b600a4520 0x7f5b60092450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:54.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.308 7f5b65915700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b600a4520 0x7f5b60092450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48256/0 (socket says 172.21.2.101:48256) 2021-02-10T11:28:54.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.308 7f5b65915700 1 -- 172.21.2.101:0/1136127519 learned_addr learned my addr 172.21.2.101:0/1136127519 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:28:54.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.308 7f5b65915700 1 -- 172.21.2.101:0/1136127519 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b600a4920 msgr2=0x7f5b60092a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:54.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.308 7f5b65915700 1 --2- 172.21.2.101:0/1136127519 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b600a4920 0x7f5b60092a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:54.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.308 7f5b65915700 1 -- 172.21.2.101:0/1136127519 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5b600a4ec0 msgr2=0x7f5b600a3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:54.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.308 7f5b65915700 1 --2- 172.21.2.101:0/1136127519 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5b600a4ec0 0x7f5b600a3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:54.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.308 7f5b65915700 1 -- 172.21.2.101:0/1136127519 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b60091ac0 con 0x7f5b600a4520 2021-02-10T11:28:54.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.308 7f5b65915700 1 --2- 172.21.2.101:0/1136127519 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b600a4520 0x7f5b60092450 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f5b50001c60 tx=0x7f5b5000a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:54.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.308 7f5b56ffd700 1 -- 172.21.2.101:0/1136127519 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5b5000acc0 con 0x7f5b600a4520 2021-02-10T11:28:54.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.308 7f5b56ffd700 1 -- 172.21.2.101:0/1136127519 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b50004140 con 0x7f5b600a4520 2021-02-10T11:28:54.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.308 7f5b682fa700 1 -- 172.21.2.101:0/1136127519 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b60091ac0 con 0x7f5b600a4520 2021-02-10T11:28:54.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.308 7f5b56ffd700 1 -- 172.21.2.101:0/1136127519 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5b50004620 con 0x7f5b600a4520 2021-02-10T11:28:54.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.309 7f5b682fa700 1 -- 172.21.2.101:0/1136127519 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b60093380 con 0x7f5b600a4520 2021-02-10T11:28:54.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.309 7f5b56ffd700 1 -- 172.21.2.101:0/1136127519 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5b50000dd0 con 0x7f5b600a4520 2021-02-10T11:28:54.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.309 7f5b56ffd700 1 --2- 172.21.2.101:0/1136127519 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b480186c0 0x7f5b4801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:28:54.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.309 7f5b56ffd700 1 -- 172.21.2.101:0/1136127519 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5b50032590 con 0x7f5b600a4520 2021-02-10T11:28:54.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.309 7f5b66116700 1 --2- 172.21.2.101:0/1136127519 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b480186c0 0x7f5b4801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:28:54.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.309 7f5b66116700 1 --2- 172.21.2.101:0/1136127519 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b480186c0 0x7f5b4801abc0 crc :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:28:54.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.341 7f5b682fa700 1 -- 172.21.2.101:0/1136127519 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b6007e6b0 con 0x7f5b600a4520 2021-02-10T11:28:54.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.344 7f5b56ffd700 1 -- 172.21.2.101:0/1136127519 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5b500009b0 con 0x7f5b600a4520 2021-02-10T11:28:54.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.471 7f5b682fa700 1 -- 172.21.2.101:0/1136127519 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839332", "target": ["mgr", ""]}) v1 -- 0x7f5b60091ed0 con 0x7f5b480186c0 2021-02-10T11:28:54.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.474 7f5b56ffd700 1 -- 172.21.2.101:0/1136127519 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f5b5c006060 con 0x7f5b480186c0 2021-02-10T11:28:54.488 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839332/ed5d2eac-0bbc-454f-9efd-7c20c259cd00 2021-02-10T11:28:54.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.479 7f5b682fa700 1 -- 172.21.2.101:0/1136127519 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b480186c0 msgr2=0x7f5b4801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:54.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.479 7f5b682fa700 1 --2- 172.21.2.101:0/1136127519 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b480186c0 0x7f5b4801abc0 crc :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:54.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.479 7f5b682fa700 1 -- 172.21.2.101:0/1136127519 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b600a4520 msgr2=0x7f5b60092450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:28:54.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.479 7f5b682fa700 1 --2- 172.21.2.101:0/1136127519 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b600a4520 0x7f5b60092450 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f5b50001c60 tx=0x7f5b5000a8d0).stop 2021-02-10T11:28:54.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.479 7f5b682fa700 1 -- 172.21.2.101:0/1136127519 shutdown_connections 2021-02-10T11:28:54.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.479 7f5b682fa700 1 --2- 172.21.2.101:0/1136127519 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b480186c0 0x7f5b4801abc0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:54.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.479 7f5b682fa700 1 --2- 172.21.2.101:0/1136127519 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b600a4920 0x7f5b60092a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:54.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.479 7f5b682fa700 1 --2- 172.21.2.101:0/1136127519 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b600a4520 0x7f5b60092450 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:28:54.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.479 7f5b682fa700 1 --2- 172.21.2.101:0/1136127519 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5b600a4ec0 0x7f5b600a3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:28:54.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.479 7f5b682fa700 1 -- 172.21.2.101:0/1136127519 >> 172.21.2.101:0/1136127519 conn(0x7f5b60080ea0 msgr2=0x7f5b600897c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:28:54.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.479 7f5b682fa700 1 -- 172.21.2.101:0/1136127519 shutdown_connections 2021-02-10T11:28:54.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:28:54.480 7f5b682fa700 1 -- 172.21.2.101:0/1136127519 wait complete. 2021-02-10T11:28:54.499 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839332 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839332/ed5d2eac-0bbc-454f-9efd-7c20c259cd00 2021-02-10T11:28:54.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:54.499 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:54.499 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:54.838 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:54.839 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:54.844 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839332/ed5d2eac-0bbc-454f-9efd-7c20c259cd00/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:54.889 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:28:54.915 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:28:55.234 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:55.234 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:55.234 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0228758 s, 45.8 MB/s 2021-02-10T11:28:55.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:55.235 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:55.235 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:55.619 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:55.619 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:55.619 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839332/ed5d2eac-0bbc-454f-9efd-7c20c259cd00/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:55.995 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:55.996 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:55.996 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252729 s, 41.5 MB/s 2021-02-10T11:28:55.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:55.997 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:55.997 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:56.333 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:56.334 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:56.334 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839332/ed5d2eac-0bbc-454f-9efd-7c20c259cd00/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:56.711 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:56.712 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:56.712 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241904 s, 43.3 MB/s 2021-02-10T11:28:56.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:56.713 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:56.713 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:57.049 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:57.049 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:57.049 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839332/ed5d2eac-0bbc-454f-9efd-7c20c259cd00/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:57.425 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:57.426 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:57.426 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234569 s, 44.7 MB/s 2021-02-10T11:28:57.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:57.427 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:57.427 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:57.761 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:57.762 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:57.762 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839332/ed5d2eac-0bbc-454f-9efd-7c20c259cd00/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:58.140 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:58.140 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:58.140 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0263722 s, 39.8 MB/s 2021-02-10T11:28:58.142 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:58.142 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:58.142 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:58.478 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:58.478 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:58.479 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839332/ed5d2eac-0bbc-454f-9efd-7c20c259cd00/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:58.850 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:58.850 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:58.850 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245729 s, 42.7 MB/s 2021-02-10T11:28:58.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:58.851 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:58.851 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:59.187 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:59.187 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:59.187 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839332/ed5d2eac-0bbc-454f-9efd-7c20c259cd00/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:28:59.564 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:28:59.565 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:28:59.565 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0250557 s, 41.8 MB/s 2021-02-10T11:28:59.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:28:59.566 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:28:59.566 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:28:59.902 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:28:59.902 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:28:59.902 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839332/ed5d2eac-0bbc-454f-9efd-7c20c259cd00/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:00.278 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:00.278 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:00.278 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249512 s, 42.0 MB/s 2021-02-10T11:29:00.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:00.279 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:00.279 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:00.616 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:00.617 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:00.617 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839332/ed5d2eac-0bbc-454f-9efd-7c20c259cd00/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:00.995 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:00.996 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:00.996 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0271196 s, 38.7 MB/s 2021-02-10T11:29:00.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:00.997 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:00.997 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:01.332 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:01.333 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:01.333 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839332/ed5d2eac-0bbc-454f-9efd-7c20c259cd00/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:01.710 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:01.710 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:01.710 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0232871 s, 45.0 MB/s 2021-02-10T11:29:01.712 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839333 2021-02-10T11:29:02.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.117 7fdab39f9700 1 Processor -- start 2021-02-10T11:29:02.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.118 7fdab39f9700 1 -- start start 2021-02-10T11:29:02.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.118 7fdab39f9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdaac1b0470 0x7fdaac1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:02.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.118 7fdab39f9700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaac0f0b50 con 0x7fdaac0ef610 2021-02-10T11:29:02.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.118 7fdab39f9700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaac0f24e0 con 0x7fdaac0f20e0 2021-02-10T11:29:02.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.118 7fdab39f9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac0e5ea0 con 0x7fdaac1b0470 2021-02-10T11:29:02.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.118 7fdab1815700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdaac1b0470 0x7fdaac1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:02.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.118 7fdab1014700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fdaac0ef610 0x7fdaac0efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48758/0 (socket says 172.21.2.101:48758) 2021-02-10T11:29:02.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.118 7fdab1014700 1 -- 172.21.2.101:0/1773638554 learned_addr learned my addr 172.21.2.101:0/1773638554 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:29:02.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.119 7fdab1815700 1 -- 172.21.2.101:0/1773638554 >> v1:172.21.2.108:6789/0 conn(0x7fdaac0f20e0 legacy=0x7fdaac0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:02.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.119 7fdab1815700 1 -- 172.21.2.101:0/1773638554 >> v1:172.21.2.101:6789/0 conn(0x7fdaac0ef610 legacy=0x7fdaac0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:02.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.119 7fdab0813700 1 -- 172.21.2.101:0/1773638554 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 367058496 0 0) 0x7fdaa00010b0 con 0x7fdaac0ef610 2021-02-10T11:29:02.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.119 7fdab1815700 1 -- 172.21.2.101:0/1773638554 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaac0e52e0 con 0x7fdaac1b0470 2021-02-10T11:29:02.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.119 7fdab1815700 1 --2- 172.21.2.101:0/1773638554 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdaac1b0470 0x7fdaac1b0870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fdaa8006650 tx=0x7fdaa8009a80).ready entity=mon.2 client_cookie=ba6c5e6c57a79d47 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:02.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.119 7fdab0813700 1 -- 172.21.2.101:0/1773638554 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdaa800c0c0 con 0x7fdaac1b0470 2021-02-10T11:29:02.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.119 7fdab0813700 1 -- 172.21.2.101:0/1773638554 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa800c0c0 con 0x7fdaac1b0470 2021-02-10T11:29:02.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.119 7fdab39f9700 1 -- 172.21.2.101:0/1773638554 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdaac1b0470 msgr2=0x7fdaac1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:02.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.119 7fdab39f9700 1 --2- 172.21.2.101:0/1773638554 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdaac1b0470 0x7fdaac1b0870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fdaa8006650 tx=0x7fdaa8009a80).stop 2021-02-10T11:29:02.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.119 7fdab39f9700 1 -- 172.21.2.101:0/1773638554 shutdown_connections 2021-02-10T11:29:02.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.119 7fdab39f9700 1 --2- 172.21.2.101:0/1773638554 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdaac1b0470 0x7fdaac1b0870 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:02.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.119 7fdab39f9700 1 -- 172.21.2.101:0/1773638554 >> 172.21.2.101:0/1773638554 conn(0x7fdaac0ce3e0 msgr2=0x7fdaac0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:02.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.120 7fdab39f9700 1 -- 172.21.2.101:0/1773638554 shutdown_connections 2021-02-10T11:29:02.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.120 7fdab39f9700 1 -- 172.21.2.101:0/1773638554 wait complete. 2021-02-10T11:29:02.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.134 7fdab39f9700 1 Processor -- start 2021-02-10T11:29:02.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.134 7fdab39f9700 1 -- start start 2021-02-10T11:29:02.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.134 7fdab39f9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdaac089bb0 0x7fdaac0894a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:02.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.134 7fdab39f9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdaac085d90 0x7fdaac084d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:02.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.134 7fdab39f9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdaac085320 0x7fdaac0bf760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:02.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.134 7fdab39f9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac085810 con 0x7fdaac085320 2021-02-10T11:29:02.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.134 7fdab39f9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac0b42a0 con 0x7fdaac089bb0 2021-02-10T11:29:02.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.134 7fdab39f9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac0bfd80 con 0x7fdaac085d90 2021-02-10T11:29:02.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.134 7fdab2016700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdaac085320 0x7fdaac0bf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:02.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.134 7fdab2016700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdaac085320 0x7fdaac0bf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48270/0 (socket says 172.21.2.101:48270) 2021-02-10T11:29:02.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.135 7fdab2016700 1 -- 172.21.2.101:0/1831640037 learned_addr learned my addr 172.21.2.101:0/1831640037 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:29:02.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.135 7fdab1014700 1 --2- 172.21.2.101:0/1831640037 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdaac085d90 0x7fdaac084d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:02.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.135 7fdab1815700 1 --2- 172.21.2.101:0/1831640037 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdaac089bb0 0x7fdaac0894a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:02.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.135 7fdab2016700 1 -- 172.21.2.101:0/1831640037 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdaac085d90 msgr2=0x7fdaac084d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:02.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.135 7fdab2016700 1 --2- 172.21.2.101:0/1831640037 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdaac085d90 0x7fdaac084d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:02.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.135 7fdab2016700 1 -- 172.21.2.101:0/1831640037 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdaac089bb0 msgr2=0x7fdaac0894a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:02.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.135 7fdab2016700 1 --2- 172.21.2.101:0/1831640037 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdaac089bb0 0x7fdaac0894a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:02.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.135 7fdab2016700 1 -- 172.21.2.101:0/1831640037 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaac0bffa0 con 0x7fdaac085320 2021-02-10T11:29:02.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.135 7fdab1014700 1 --2- 172.21.2.101:0/1831640037 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdaac085d90 0x7fdaac084d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:29:02.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.135 7fdab2016700 1 --2- 172.21.2.101:0/1831640037 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdaac085320 0x7fdaac0bf760 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7fdaa4009940 tx=0x7fdaa400bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:02.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.135 7fda9e7fc700 1 -- 172.21.2.101:0/1831640037 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdaa4000ce0 con 0x7fdaac085320 2021-02-10T11:29:02.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.135 7fda9e7fc700 1 -- 172.21.2.101:0/1831640037 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa4000f90 con 0x7fdaac085320 2021-02-10T11:29:02.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.135 7fdab39f9700 1 -- 172.21.2.101:0/1831640037 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaac0bffa0 con 0x7fdaac085320 2021-02-10T11:29:02.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.135 7fda9e7fc700 1 -- 172.21.2.101:0/1831640037 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdaa400c570 con 0x7fdaac085320 2021-02-10T11:29:02.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.136 7fdab39f9700 1 -- 172.21.2.101:0/1831640037 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaac0bffa0 con 0x7fdaac085320 2021-02-10T11:29:02.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.136 7fda9e7fc700 1 -- 172.21.2.101:0/1831640037 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdaa4001220 con 0x7fdaac085320 2021-02-10T11:29:02.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.136 7fda9e7fc700 1 --2- 172.21.2.101:0/1831640037 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fda940186c0 0x7fda9401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:02.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.136 7fda9e7fc700 1 -- 172.21.2.101:0/1831640037 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fdaa4032900 con 0x7fdaac085320 2021-02-10T11:29:02.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.136 7fdab1815700 1 --2- 172.21.2.101:0/1831640037 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fda940186c0 0x7fda9401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:02.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.137 7fdab1815700 1 --2- 172.21.2.101:0/1831640037 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fda940186c0 0x7fda9401abc0 crc :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:02.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.168 7fdab39f9700 1 -- 172.21.2.101:0/1831640037 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaac0c0530 con 0x7fdaac085320 2021-02-10T11:29:02.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.171 7fda9e7fc700 1 -- 172.21.2.101:0/1831640037 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdaa400be30 con 0x7fdaac085320 2021-02-10T11:29:02.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.295 7fdab39f9700 1 -- 172.21.2.101:0/1831640037 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839333", "target": ["mgr", ""]}) v1 -- 0x7fdaac0b42a0 con 0x7fda940186c0 2021-02-10T11:29:02.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.324 7fda9e7fc700 1 -- 172.21.2.101:0/1831640037 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fdaa8009180 con 0x7fda940186c0 2021-02-10T11:29:02.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.328 7fdab39f9700 1 -- 172.21.2.101:0/1831640037 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fda940186c0 msgr2=0x7fda9401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:02.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.328 7fdab39f9700 1 --2- 172.21.2.101:0/1831640037 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fda940186c0 0x7fda9401abc0 crc :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:02.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.328 7fdab39f9700 1 -- 172.21.2.101:0/1831640037 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdaac085320 msgr2=0x7fdaac0bf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:02.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.328 7fdab39f9700 1 --2- 172.21.2.101:0/1831640037 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdaac085320 0x7fdaac0bf760 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7fdaa4009940 tx=0x7fdaa400bcf0).stop 2021-02-10T11:29:02.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.328 7fdab39f9700 1 -- 172.21.2.101:0/1831640037 shutdown_connections 2021-02-10T11:29:02.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.328 7fdab39f9700 1 --2- 172.21.2.101:0/1831640037 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fda940186c0 0x7fda9401abc0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:02.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.328 7fdab39f9700 1 --2- 172.21.2.101:0/1831640037 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdaac085d90 0x7fdaac084d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:02.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.328 7fdab39f9700 1 --2- 172.21.2.101:0/1831640037 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdaac085320 0x7fdaac0bf760 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:02.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.328 7fdab39f9700 1 --2- 172.21.2.101:0/1831640037 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdaac089bb0 0x7fdaac0894a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:02.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.328 7fdab39f9700 1 -- 172.21.2.101:0/1831640037 >> 172.21.2.101:0/1831640037 conn(0x7fdaac088d10 msgr2=0x7fdaac0b7740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:02.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.329 7fdab39f9700 1 -- 172.21.2.101:0/1831640037 shutdown_connections 2021-02-10T11:29:02.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.329 7fdab39f9700 1 -- 172.21.2.101:0/1831640037 wait complete. 2021-02-10T11:29:02.348 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839333 2021-02-10T11:29:02.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.754 7f79c5319700 1 Processor -- start 2021-02-10T11:29:02.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.754 7f79c5319700 1 -- start start 2021-02-10T11:29:02.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.754 7f79c5319700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f79c01b0470 0x7f79c01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:02.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.754 7f79c5319700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f79c00ef610 0x7f79c00efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:02.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.754 7f79c5319700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c00f0b50 con 0x7f79c00f0030 2021-02-10T11:29:02.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.754 7f79c5319700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79c00e52e0 con 0x7f79c01b0470 2021-02-10T11:29:02.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.754 7f79c5319700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c00e54d0 con 0x7f79c00ef610 2021-02-10T11:29:02.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.755 7f79be87c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f79c00ef610 0x7f79c00efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:02.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.755 7f79bf07d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f79c01b0470 0x7f79c01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:02.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.755 7f79bf87e700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f79c00f0030 0x7f79c00f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:42180/0 (socket says 172.21.2.101:42180) 2021-02-10T11:29:02.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.755 7f79bf87e700 1 -- 172.21.2.101:0/753374963 learned_addr learned my addr 172.21.2.101:0/753374963 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:29:02.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.756 7f79be87c700 1 -- 172.21.2.101:0/753374963 >> v2:172.21.2.108:3300/0 conn(0x7f79c01b0470 msgr2=0x7f79c01b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:29:02.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.756 7f79be87c700 1 --2- 172.21.2.101:0/753374963 >> v2:172.21.2.108:3300/0 conn(0x7f79c01b0470 0x7f79c01b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:02.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.756 7f79be07b700 1 -- 172.21.2.101:0/753374963 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4233405627 0 0) 0x7f79b8001510 con 0x7f79c00f0030 2021-02-10T11:29:02.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.756 7f79be87c700 1 -- 172.21.2.101:0/753374963 >> v1:172.21.2.108:6789/0 conn(0x7f79c00f0030 legacy=0x7f79c00f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:02.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.756 7f79be87c700 1 -- 172.21.2.101:0/753374963 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c00f0430 con 0x7f79c00ef610 2021-02-10T11:29:02.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.756 7f79bf07d700 1 --2- 172.21.2.101:0/753374963 >> v2:172.21.2.108:3300/0 conn(0x7f79c01b0470 0x7f79c01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:29:02.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.757 7f79be87c700 1 --2- 172.21.2.101:0/753374963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f79c00ef610 0x7f79c00efa10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f79b0006650 tx=0x7f79b0009a80).ready entity=mon.2 client_cookie=c666d16376e1cd43 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:02.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.757 7f79be07b700 1 -- 172.21.2.101:0/753374963 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f79b00009b0 con 0x7f79c00ef610 2021-02-10T11:29:02.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.757 7f79be07b700 1 -- 172.21.2.101:0/753374963 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b000c0c0 con 0x7f79c00ef610 2021-02-10T11:29:02.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.757 7f79be07b700 1 -- 172.21.2.101:0/753374963 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f79b000c5e0 con 0x7f79c00ef610 2021-02-10T11:29:02.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.757 7f79c5319700 1 -- 172.21.2.101:0/753374963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f79c00ef610 msgr2=0x7f79c00efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:02.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.757 7f79c5319700 1 --2- 172.21.2.101:0/753374963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f79c00ef610 0x7f79c00efa10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f79b0006650 tx=0x7f79b0009a80).stop 2021-02-10T11:29:02.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.758 7f79c5319700 1 -- 172.21.2.101:0/753374963 shutdown_connections 2021-02-10T11:29:02.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.758 7f79c5319700 1 --2- 172.21.2.101:0/753374963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f79c00ef610 0x7f79c00efa10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:02.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.758 7f79c5319700 1 --2- 172.21.2.101:0/753374963 >> v2:172.21.2.108:3300/0 conn(0x7f79c01b0470 0x7f79c01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:02.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.758 7f79c5319700 1 -- 172.21.2.101:0/753374963 >> 172.21.2.101:0/753374963 conn(0x7f79c00ce3e0 msgr2=0x7f79c00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:02.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.758 7f79c5319700 1 -- 172.21.2.101:0/753374963 shutdown_connections 2021-02-10T11:29:02.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.758 7f79c5319700 1 -- 172.21.2.101:0/753374963 wait complete. 2021-02-10T11:29:02.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.772 7f79c5319700 1 Processor -- start 2021-02-10T11:29:02.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.772 7f79c5319700 1 -- start start 2021-02-10T11:29:02.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.772 7f79c5319700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f79c0190920 0x7f79c00b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:02.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.772 7f79c5319700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f79c00b8200 0x7f79c00b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:02.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.772 7f79c5319700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f79c00c1d80 0x7f79c00c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:02.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.772 7f79c5319700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c0190da0 con 0x7f79c00c1d80 2021-02-10T11:29:02.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.772 7f79c5319700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c00b8c20 con 0x7f79c00b8200 2021-02-10T11:29:02.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.772 7f79c5319700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c00b8e10 con 0x7f79c0190920 2021-02-10T11:29:02.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.773 7f79bf87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f79c00c1d80 0x7f79c00c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:02.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.773 7f79bf87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f79c00c1d80 0x7f79c00c2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48280/0 (socket says 172.21.2.101:48280) 2021-02-10T11:29:02.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.773 7f79bf87e700 1 -- 172.21.2.101:0/300810057 learned_addr learned my addr 172.21.2.101:0/300810057 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:29:02.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.773 7f79be87c700 1 --2- 172.21.2.101:0/300810057 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f79c00b8200 0x7f79c00b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:02.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.773 7f79bf07d700 1 --2- 172.21.2.101:0/300810057 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f79c0190920 0x7f79c00b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:02.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.773 7f79bf87e700 1 -- 172.21.2.101:0/300810057 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f79c0190920 msgr2=0x7f79c00b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:02.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.773 7f79bf87e700 1 --2- 172.21.2.101:0/300810057 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f79c0190920 0x7f79c00b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:02.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.773 7f79bf87e700 1 -- 172.21.2.101:0/300810057 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f79c00b8200 msgr2=0x7f79c00b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:02.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.773 7f79bf87e700 1 --2- 172.21.2.101:0/300810057 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f79c00b8200 0x7f79c00b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:02.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.773 7f79bf87e700 1 -- 172.21.2.101:0/300810057 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c00c27d0 con 0x7f79c00c1d80 2021-02-10T11:29:02.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.773 7f79bf87e700 1 --2- 172.21.2.101:0/300810057 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f79c00c1d80 0x7f79c00c2180 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7f79b800a900 tx=0x7f79b800aca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:02.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.774 7f79b7fff700 1 -- 172.21.2.101:0/300810057 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f79b8007020 con 0x7f79c00c1d80 2021-02-10T11:29:02.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.774 7f79b7fff700 1 -- 172.21.2.101:0/300810057 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b8007870 con 0x7f79c00c1d80 2021-02-10T11:29:02.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.774 7f79c5319700 1 -- 172.21.2.101:0/300810057 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c00c27d0 con 0x7f79c00c1d80 2021-02-10T11:29:02.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.774 7f79c5319700 1 -- 172.21.2.101:0/300810057 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c00c27d0 con 0x7f79c00c1d80 2021-02-10T11:29:02.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.774 7f79b7fff700 1 -- 172.21.2.101:0/300810057 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f79b80072e0 con 0x7f79c00c1d80 2021-02-10T11:29:02.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.774 7f79b7fff700 1 -- 172.21.2.101:0/300810057 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f79b800ade0 con 0x7f79c00c1d80 2021-02-10T11:29:02.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.774 7f79b7fff700 1 --2- 172.21.2.101:0/300810057 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79a40186c0 0x7f79a401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:02.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.774 7f79b7fff700 1 -- 172.21.2.101:0/300810057 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f79b80328b0 con 0x7f79c00c1d80 2021-02-10T11:29:02.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.775 7f79bf07d700 1 --2- 172.21.2.101:0/300810057 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79a40186c0 0x7f79a401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:02.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.775 7f79bf07d700 1 --2- 172.21.2.101:0/300810057 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79a40186c0 0x7f79a401abc0 crc :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:02.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.804 7f79c5319700 1 -- 172.21.2.101:0/300810057 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c0105ae0 con 0x7f79c00c1d80 2021-02-10T11:29:02.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.807 7f79b7fff700 1 -- 172.21.2.101:0/300810057 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f79b800ade0 con 0x7f79c00c1d80 2021-02-10T11:29:02.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.934 7f79c5319700 1 -- 172.21.2.101:0/300810057 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839333", "target": ["mgr", ""]}) v1 -- 0x7f79c00f0b50 con 0x7f79a40186c0 2021-02-10T11:29:02.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.937 7f79b7fff700 1 -- 172.21.2.101:0/300810057 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f79a8006a30 con 0x7f79a40186c0 2021-02-10T11:29:02.952 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839333/17051c0c-5fa6-429e-ada6-d2e6eb6228a2 2021-02-10T11:29:02.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.942 7f79c5319700 1 -- 172.21.2.101:0/300810057 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79a40186c0 msgr2=0x7f79a401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:02.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.943 7f79c5319700 1 --2- 172.21.2.101:0/300810057 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79a40186c0 0x7f79a401abc0 crc :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:02.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.943 7f79c5319700 1 -- 172.21.2.101:0/300810057 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f79c00c1d80 msgr2=0x7f79c00c2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:02.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.943 7f79c5319700 1 --2- 172.21.2.101:0/300810057 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f79c00c1d80 0x7f79c00c2180 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7f79b800a900 tx=0x7f79b800aca0).stop 2021-02-10T11:29:02.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.943 7f79c5319700 1 -- 172.21.2.101:0/300810057 shutdown_connections 2021-02-10T11:29:02.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.943 7f79c5319700 1 --2- 172.21.2.101:0/300810057 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f79c00c1d80 0x7f79c00c2180 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:02.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.943 7f79c5319700 1 --2- 172.21.2.101:0/300810057 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f79c00b8200 0x7f79c00b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:02.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.943 7f79c5319700 1 --2- 172.21.2.101:0/300810057 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79a40186c0 0x7f79a401abc0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:02.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.943 7f79c5319700 1 --2- 172.21.2.101:0/300810057 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f79c0190920 0x7f79c00b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:02.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.943 7f79c5319700 1 -- 172.21.2.101:0/300810057 >> 172.21.2.101:0/300810057 conn(0x7f79c0094e10 msgr2=0x7f79c00a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:02.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.943 7f79c5319700 1 -- 172.21.2.101:0/300810057 shutdown_connections 2021-02-10T11:29:02.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:02.943 7f79c5319700 1 -- 172.21.2.101:0/300810057 wait complete. 2021-02-10T11:29:02.962 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839333 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839333/17051c0c-5fa6-429e-ada6-d2e6eb6228a2 2021-02-10T11:29:02.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:02.962 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:02.962 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:03.299 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:03.300 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:03.300 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839333/17051c0c-5fa6-429e-ada6-d2e6eb6228a2/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:03.682 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:03.682 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:03.682 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024305 s, 43.1 MB/s 2021-02-10T11:29:03.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:03.683 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:03.683 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:04.019 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:04.020 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:04.020 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839333/17051c0c-5fa6-429e-ada6-d2e6eb6228a2/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:04.393 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:04.394 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:04.394 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244185 s, 42.9 MB/s 2021-02-10T11:29:04.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:04.395 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:04.395 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:04.730 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:04.731 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:04.731 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839333/17051c0c-5fa6-429e-ada6-d2e6eb6228a2/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:05.103 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:05.103 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:05.103 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241081 s, 43.5 MB/s 2021-02-10T11:29:05.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:05.104 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:05.104 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:05.439 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:05.628 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:05.628 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839333/17051c0c-5fa6-429e-ada6-d2e6eb6228a2/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:06.009 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:06.009 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:06.009 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0261281 s, 40.1 MB/s 2021-02-10T11:29:06.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:06.010 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:06.010 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:06.347 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:06.348 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:06.348 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839333/17051c0c-5fa6-429e-ada6-d2e6eb6228a2/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:06.723 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:06.724 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:06.724 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238244 s, 44.0 MB/s 2021-02-10T11:29:06.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:06.725 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:06.725 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:07.061 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:07.062 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:07.062 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839333/17051c0c-5fa6-429e-ada6-d2e6eb6228a2/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:07.436 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:07.437 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:07.437 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231312 s, 45.3 MB/s 2021-02-10T11:29:07.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:07.438 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:07.438 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:07.773 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:07.774 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:07.774 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839333/17051c0c-5fa6-429e-ada6-d2e6eb6228a2/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:08.152 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:08.152 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:08.152 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024283 s, 43.2 MB/s 2021-02-10T11:29:08.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:08.153 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:08.153 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:08.489 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:08.490 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:08.490 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839333/17051c0c-5fa6-429e-ada6-d2e6eb6228a2/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:08.862 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:08.862 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:08.862 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023618 s, 44.4 MB/s 2021-02-10T11:29:08.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:08.863 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:08.863 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:09.198 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:09.199 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:09.199 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839333/17051c0c-5fa6-429e-ada6-d2e6eb6228a2/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:09.577 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:09.577 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:09.577 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251811 s, 41.6 MB/s 2021-02-10T11:29:09.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:09.578 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:09.579 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:09.915 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:09.915 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:09.915 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839333/17051c0c-5fa6-429e-ada6-d2e6eb6228a2/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:10.290 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:10.290 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:10.290 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0232724 s, 45.1 MB/s 2021-02-10T11:29:10.291 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839334 2021-02-10T11:29:10.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.697 7f9272fcd700 1 Processor -- start 2021-02-10T11:29:10.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.697 7f9272fcd700 1 -- start start 2021-02-10T11:29:10.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.698 7f9272fcd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f926c1b0470 0x7f926c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:10.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.698 7f9272fcd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f926c0f20e0 0x7f926c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:10.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.698 7f9272fcd700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f926c0f0b50 con 0x7f926c0ef610 2021-02-10T11:29:10.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.698 7f9272fcd700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f926c0f0820 con 0x7f926c0f20e0 2021-02-10T11:29:10.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.698 7f9272fcd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926c0f24e0 con 0x7f926c1b0470 2021-02-10T11:29:10.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.698 7f92715ea700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f926c0f20e0 0x7f926c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:10.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.698 7f92715ea700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f926c0f20e0 0x7f926c0f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48292/0 (socket says 172.21.2.101:48292) 2021-02-10T11:29:10.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.698 7f92715ea700 1 -- 172.21.2.101:0/1305686882 learned_addr learned my addr 172.21.2.101:0/1305686882 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:29:10.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.698 7f92715ea700 1 -- 172.21.2.101:0/1305686882 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f926c1b0470 msgr2=0x7f926c1b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:29:10.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.698 7f92715ea700 1 --2- 172.21.2.101:0/1305686882 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f926c1b0470 0x7f926c1b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:10.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.698 7f92715ea700 1 -- 172.21.2.101:0/1305686882 >> v1:172.21.2.101:6789/0 conn(0x7f926c0ef610 legacy=0x7f926c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:10.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.698 7f92715ea700 1 -- 172.21.2.101:0/1305686882 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c0e52e0 con 0x7f926c0f20e0 2021-02-10T11:29:10.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.698 7f92715ea700 1 --2- 172.21.2.101:0/1305686882 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f926c0f20e0 0x7f926c0f0200 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f9264006f00 tx=0x7f9264009f10).ready entity=mon.0 client_cookie=ea913c6b9fea3c51 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:10.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.698 7f926b7fe700 1 -- 172.21.2.101:0/1305686882 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f926400b3b0 con 0x7f926c0f20e0 2021-02-10T11:29:10.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.698 7f926b7fe700 1 -- 172.21.2.101:0/1305686882 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926400b630 con 0x7f926c0f20e0 2021-02-10T11:29:10.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.698 7f926b7fe700 1 -- 172.21.2.101:0/1305686882 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f926400bb50 con 0x7f926c0f20e0 2021-02-10T11:29:10.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.698 7f9272fcd700 1 -- 172.21.2.101:0/1305686882 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f926c0f20e0 msgr2=0x7f926c0f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:10.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.698 7f9272fcd700 1 --2- 172.21.2.101:0/1305686882 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f926c0f20e0 0x7f926c0f0200 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f9264006f00 tx=0x7f9264009f10).stop 2021-02-10T11:29:10.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.699 7f9272fcd700 1 -- 172.21.2.101:0/1305686882 shutdown_connections 2021-02-10T11:29:10.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.699 7f9272fcd700 1 --2- 172.21.2.101:0/1305686882 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f926c0f20e0 0x7f926c0f0200 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:10.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.699 7f9272fcd700 1 --2- 172.21.2.101:0/1305686882 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f926c1b0470 0x7f926c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:10.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.699 7f9272fcd700 1 -- 172.21.2.101:0/1305686882 >> 172.21.2.101:0/1305686882 conn(0x7f926c0ce3e0 msgr2=0x7f926c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:10.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.699 7f9272fcd700 1 -- 172.21.2.101:0/1305686882 shutdown_connections 2021-02-10T11:29:10.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.699 7f9272fcd700 1 -- 172.21.2.101:0/1305686882 wait complete. 2021-02-10T11:29:10.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.713 7f9272fcd700 1 Processor -- start 2021-02-10T11:29:10.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.713 7f9272fcd700 1 -- start start 2021-02-10T11:29:10.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.714 7f9272fcd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f926c0ee230 0x7f926c0bfc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:10.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.714 7f9272fcd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f926c0ee630 0x7f926c0eea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:10.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.714 7f9272fcd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f926c0c09e0 0x7f926c0c0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:10.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.714 7f9272fcd700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c0c1f60 con 0x7f926c0ee230 2021-02-10T11:29:10.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.714 7f9272fcd700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c18bdf0 con 0x7f926c0ee630 2021-02-10T11:29:10.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.714 7f9272fcd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926c0c0460 con 0x7f926c0c09e0 2021-02-10T11:29:10.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.714 7f9270de9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f926c0ee230 0x7f926c0bfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:10.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.714 7f9270de9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f926c0ee230 0x7f926c0bfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48296/0 (socket says 172.21.2.101:48296) 2021-02-10T11:29:10.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.714 7f9270de9700 1 -- 172.21.2.101:0/1247610317 learned_addr learned my addr 172.21.2.101:0/1247610317 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:29:10.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.714 7f9270de9700 1 -- 172.21.2.101:0/1247610317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f926c0c09e0 msgr2=0x7f926c0c0de0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:29:10.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.714 7f9270de9700 1 --2- 172.21.2.101:0/1247610317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f926c0c09e0 0x7f926c0c0de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:10.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.714 7f9270de9700 1 -- 172.21.2.101:0/1247610317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f926c0ee630 msgr2=0x7f926c0eea30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:29:10.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.714 7f9270de9700 1 --2- 172.21.2.101:0/1247610317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f926c0ee630 0x7f926c0eea30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:10.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.714 7f9270de9700 1 -- 172.21.2.101:0/1247610317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c0c1400 con 0x7f926c0ee230 2021-02-10T11:29:10.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.714 7f9270de9700 1 --2- 172.21.2.101:0/1247610317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f926c0ee230 0x7f926c0bfc80 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f925400b5c0 tx=0x7f925400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:10.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.714 7f92697fa700 1 -- 172.21.2.101:0/1247610317 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f925400bd50 con 0x7f926c0ee230 2021-02-10T11:29:10.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.715 7f92697fa700 1 -- 172.21.2.101:0/1247610317 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9254006030 con 0x7f926c0ee230 2021-02-10T11:29:10.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.715 7f92697fa700 1 -- 172.21.2.101:0/1247610317 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f92540054d0 con 0x7f926c0ee230 2021-02-10T11:29:10.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.715 7f9272fcd700 1 -- 172.21.2.101:0/1247610317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f926c0c1400 con 0x7f926c0ee230 2021-02-10T11:29:10.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.715 7f9272fcd700 1 -- 172.21.2.101:0/1247610317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926c0c1400 con 0x7f926c0ee230 2021-02-10T11:29:10.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.715 7f92697fa700 1 -- 172.21.2.101:0/1247610317 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9254006210 con 0x7f926c0ee230 2021-02-10T11:29:10.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.715 7f92697fa700 1 --2- 172.21.2.101:0/1247610317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9250018730 0x7f925001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:10.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.715 7f92697fa700 1 -- 172.21.2.101:0/1247610317 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9254032630 con 0x7f926c0ee230 2021-02-10T11:29:10.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.715 7f926bfff700 1 --2- 172.21.2.101:0/1247610317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9250018730 0x7f925001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:10.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.716 7f926bfff700 1 --2- 172.21.2.101:0/1247610317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9250018730 0x7f925001ac10 crc :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:10.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.748 7f9272fcd700 1 -- 172.21.2.101:0/1247610317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f926c0eda20 con 0x7f926c0ee230 2021-02-10T11:29:10.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.750 7f92697fa700 1 -- 172.21.2.101:0/1247610317 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9254000c40 con 0x7f926c0ee230 2021-02-10T11:29:10.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.875 7f9272fcd700 1 -- 172.21.2.101:0/1247610317 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839334", "target": ["mgr", ""]}) v1 -- 0x7f926c0eda20 con 0x7f9250018730 2021-02-10T11:29:10.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.902 7f92697fa700 1 -- 172.21.2.101:0/1247610317 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f925c006db0 con 0x7f9250018730 2021-02-10T11:29:10.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.908 7f9272fcd700 1 -- 172.21.2.101:0/1247610317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9250018730 msgr2=0x7f925001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:10.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.908 7f9272fcd700 1 --2- 172.21.2.101:0/1247610317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9250018730 0x7f925001ac10 crc :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:10.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.908 7f9272fcd700 1 -- 172.21.2.101:0/1247610317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f926c0ee230 msgr2=0x7f926c0bfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:10.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.908 7f9272fcd700 1 --2- 172.21.2.101:0/1247610317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f926c0ee230 0x7f926c0bfc80 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f925400b5c0 tx=0x7f925400b960).stop 2021-02-10T11:29:10.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.908 7f9272fcd700 1 -- 172.21.2.101:0/1247610317 shutdown_connections 2021-02-10T11:29:10.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.908 7f9272fcd700 1 --2- 172.21.2.101:0/1247610317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9250018730 0x7f925001ac10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:10.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.908 7f9272fcd700 1 --2- 172.21.2.101:0/1247610317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f926c0c09e0 0x7f926c0c0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:10.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.908 7f9272fcd700 1 --2- 172.21.2.101:0/1247610317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f926c0ee630 0x7f926c0eea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:10.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.908 7f9272fcd700 1 --2- 172.21.2.101:0/1247610317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f926c0ee230 0x7f926c0bfc80 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:10.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.908 7f9272fcd700 1 -- 172.21.2.101:0/1247610317 >> 172.21.2.101:0/1247610317 conn(0x7f926c09dfd0 msgr2=0x7f926c09abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:10.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.908 7f9272fcd700 1 -- 172.21.2.101:0/1247610317 shutdown_connections 2021-02-10T11:29:10.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:10.908 7f9272fcd700 1 -- 172.21.2.101:0/1247610317 wait complete. 2021-02-10T11:29:10.927 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839334 2021-02-10T11:29:11.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.333 7f8c5f59b700 1 Processor -- start 2021-02-10T11:29:11.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.333 7f8c5f59b700 1 -- start start 2021-02-10T11:29:11.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.334 7f8c5f59b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8c581b0870 0x7f8c580f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:11.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.334 7f8c5f59b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c580efe00 0x7f8c580f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:11.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.334 7f8c5f59b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c581b0c70 con 0x7f8c581b0470 2021-02-10T11:29:11.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.334 7f8c5f59b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c580f0b50 con 0x7f8c581b0870 2021-02-10T11:29:11.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.334 7f8c5f59b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c580f0820 con 0x7f8c580efe00 2021-02-10T11:29:11.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.334 7f8c5d3b7700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8c581b0470 0x7f8c580ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48796/0 (socket says 172.21.2.101:48796) 2021-02-10T11:29:11.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.334 7f8c5d3b7700 1 -- 172.21.2.101:0/1236021480 learned_addr learned my addr 172.21.2.101:0/1236021480 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:29:11.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.334 7f8c4bfff700 1 -- 172.21.2.101:0/1236021480 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2595014056 0 0) 0x7f8c4c001180 con 0x7f8c581b0470 2021-02-10T11:29:11.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.334 7f8c4bfff700 1 -- 172.21.2.101:0/1236021480 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c44002fe0 con 0x7f8c581b0470 2021-02-10T11:29:11.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.334 7f8c5dbb8700 1 --2- 172.21.2.101:0/1236021480 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c580efe00 0x7f8c580f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:11.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.334 7f8c5cbb6700 1 --2- 172.21.2.101:0/1236021480 >> v2:172.21.2.108:3300/0 conn(0x7f8c581b0870 0x7f8c580f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:11.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c4bfff700 1 -- 172.21.2.101:0/1236021480 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3949268940 0 0) 0x7f8c4c001180 con 0x7f8c581b0470 2021-02-10T11:29:11.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c4bfff700 1 -- 172.21.2.101:0/1236021480 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c580efe00 msgr2=0x7f8c580f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:29:11.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c4bfff700 1 --2- 172.21.2.101:0/1236021480 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c580efe00 0x7f8c580f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:11.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c4bfff700 1 -- 172.21.2.101:0/1236021480 >> v2:172.21.2.108:3300/0 conn(0x7f8c581b0870 msgr2=0x7f8c580f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:29:11.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c4bfff700 1 --2- 172.21.2.101:0/1236021480 >> v2:172.21.2.108:3300/0 conn(0x7f8c581b0870 0x7f8c580f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:11.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c4bfff700 1 -- 172.21.2.101:0/1236021480 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c580e52e0 con 0x7f8c581b0470 2021-02-10T11:29:11.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c4bfff700 1 -- 172.21.2.101:0/1236021480 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f8c4c001680 con 0x7f8c581b0470 2021-02-10T11:29:11.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c4bfff700 1 -- 172.21.2.101:0/1236021480 >> v1:172.21.2.101:6789/0 conn(0x7f8c581b0470 legacy=0x7f8c580ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:11.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c4bfff700 1 --2- 172.21.2.101:0/1236021480 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c44003a60 0x7f8c44005eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:11.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c4bfff700 1 --2- 172.21.2.101:0/1236021480 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c44006750 0x7f8c44008c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:11.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c4bfff700 1 --2- 172.21.2.101:0/1236021480 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c440094f0 0x7f8c4400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:11.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c4bfff700 1 -- 172.21.2.101:0/1236021480 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4400c2c0 con 0x7f8c44006750 2021-02-10T11:29:11.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c4bfff700 1 -- 172.21.2.101:0/1236021480 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c440504b0 con 0x7f8c440094f0 2021-02-10T11:29:11.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c4bfff700 1 -- 172.21.2.101:0/1236021480 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c440506d0 con 0x7f8c44003a60 2021-02-10T11:29:11.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c5cbb6700 1 --2- 172.21.2.101:0/1236021480 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c44006750 0x7f8c44008c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:11.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c5d3b7700 1 --2- 172.21.2.101:0/1236021480 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c44003a60 0x7f8c44005eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:11.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c5cbb6700 1 -- 172.21.2.101:0/1236021480 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c44003a60 msgr2=0x7f8c44005eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:29:11.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c5cbb6700 1 --2- 172.21.2.101:0/1236021480 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c44003a60 0x7f8c44005eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:11.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c5cbb6700 1 -- 172.21.2.101:0/1236021480 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c440094f0 msgr2=0x7f8c4400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:29:11.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c5dbb8700 1 --2- 172.21.2.101:0/1236021480 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c440094f0 0x7f8c4400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:11.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c5cbb6700 1 --2- 172.21.2.101:0/1236021480 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c440094f0 0x7f8c4400ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:11.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c5cbb6700 1 -- 172.21.2.101:0/1236021480 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c44050970 con 0x7f8c44006750 2021-02-10T11:29:11.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c5d3b7700 1 --2- 172.21.2.101:0/1236021480 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c44003a60 0x7f8c44005eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:29:11.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c5cbb6700 1 --2- 172.21.2.101:0/1236021480 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c44006750 0x7f8c44008c50 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f8c54006360 tx=0x7f8c54009c30).ready entity=mon.0 client_cookie=d3e1a4cfb6937df4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:11.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c4bfff700 1 -- 172.21.2.101:0/1236021480 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c540009b0 con 0x7f8c44006750 2021-02-10T11:29:11.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c4bfff700 1 -- 172.21.2.101:0/1236021480 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c54000c90 con 0x7f8c44006750 2021-02-10T11:29:11.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c4bfff700 1 -- 172.21.2.101:0/1236021480 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c54002220 con 0x7f8c44006750 2021-02-10T11:29:11.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c5f59b700 1 -- 172.21.2.101:0/1236021480 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c44006750 msgr2=0x7f8c44008c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:11.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c5f59b700 1 --2- 172.21.2.101:0/1236021480 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c44006750 0x7f8c44008c50 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f8c54006360 tx=0x7f8c54009c30).stop 2021-02-10T11:29:11.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.335 7f8c5dbb8700 1 -- 172.21.2.101:0/1236021480 reap_dead start 2021-02-10T11:29:11.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.336 7f8c5f59b700 1 -- 172.21.2.101:0/1236021480 shutdown_connections 2021-02-10T11:29:11.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.336 7f8c5f59b700 1 -- 172.21.2.101:0/1236021480 >> 172.21.2.101:0/1236021480 conn(0x7f8c580ce3e0 msgr2=0x7f8c580e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:11.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.336 7f8c5f59b700 1 -- 172.21.2.101:0/1236021480 shutdown_connections 2021-02-10T11:29:11.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.336 7f8c5f59b700 1 -- 172.21.2.101:0/1236021480 wait complete. 2021-02-10T11:29:11.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.351 7f8c5f59b700 1 Processor -- start 2021-02-10T11:29:11.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.351 7f8c5f59b700 1 -- start start 2021-02-10T11:29:11.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.351 7f8c5f59b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c5809b930 0x7f8c580b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:11.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.351 7f8c5f59b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c580b8410 0x7f8c580b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:11.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.351 7f8c5f59b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c580c1f80 0x7f8c580c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:11.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.351 7f8c5f59b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c580c1a00 con 0x7f8c580c1f80 2021-02-10T11:29:11.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.351 7f8c5f59b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c580b8e30 con 0x7f8c5809b930 2021-02-10T11:29:11.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.351 7f8c5f59b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c580b9050 con 0x7f8c580b8410 2021-02-10T11:29:11.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.351 7f8c5dbb8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c580c1f80 0x7f8c580c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:11.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.352 7f8c5dbb8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c580c1f80 0x7f8c580c2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48314/0 (socket says 172.21.2.101:48314) 2021-02-10T11:29:11.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.352 7f8c5dbb8700 1 -- 172.21.2.101:0/2365989868 learned_addr learned my addr 172.21.2.101:0/2365989868 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:29:11.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.352 7f8c5cbb6700 1 --2- 172.21.2.101:0/2365989868 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c580b8410 0x7f8c580b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:11.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.352 7f8c5d3b7700 1 --2- 172.21.2.101:0/2365989868 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c5809b930 0x7f8c580b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:11.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.352 7f8c5dbb8700 1 -- 172.21.2.101:0/2365989868 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c580b8410 msgr2=0x7f8c580b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:11.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.352 7f8c5dbb8700 1 --2- 172.21.2.101:0/2365989868 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c580b8410 0x7f8c580b8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:11.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.352 7f8c5dbb8700 1 -- 172.21.2.101:0/2365989868 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c5809b930 msgr2=0x7f8c580b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:11.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.352 7f8c5dbb8700 1 --2- 172.21.2.101:0/2365989868 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c5809b930 0x7f8c580b7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:11.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.352 7f8c5dbb8700 1 -- 172.21.2.101:0/2365989868 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c580c29a0 con 0x7f8c580c1f80 2021-02-10T11:29:11.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.352 7f8c5d3b7700 1 --2- 172.21.2.101:0/2365989868 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c5809b930 0x7f8c580b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:29:11.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.352 7f8c5dbb8700 1 --2- 172.21.2.101:0/2365989868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c580c1f80 0x7f8c580c2380 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f8c5000b940 tx=0x7f8c5000bce0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:11.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.352 7f8c49ffb700 1 -- 172.21.2.101:0/2365989868 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c5000c2f0 con 0x7f8c580c1f80 2021-02-10T11:29:11.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.352 7f8c49ffb700 1 -- 172.21.2.101:0/2365989868 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5000c630 con 0x7f8c580c1f80 2021-02-10T11:29:11.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.352 7f8c49ffb700 1 -- 172.21.2.101:0/2365989868 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c5000ca90 con 0x7f8c580c1f80 2021-02-10T11:29:11.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.352 7f8c5f59b700 1 -- 172.21.2.101:0/2365989868 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c580c29a0 con 0x7f8c580c1f80 2021-02-10T11:29:11.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.352 7f8c5f59b700 1 -- 172.21.2.101:0/2365989868 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c580c29a0 con 0x7f8c580c1f80 2021-02-10T11:29:11.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.353 7f8c49ffb700 1 -- 172.21.2.101:0/2365989868 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8c5000c2f0 con 0x7f8c580c1f80 2021-02-10T11:29:11.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.353 7f8c49ffb700 1 --2- 172.21.2.101:0/2365989868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c380186c0 0x7f8c3801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:11.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.353 7f8c49ffb700 1 -- 172.21.2.101:0/2365989868 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8c500329f0 con 0x7f8c580c1f80 2021-02-10T11:29:11.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.353 7f8c5d3b7700 1 --2- 172.21.2.101:0/2365989868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c380186c0 0x7f8c3801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:11.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.353 7f8c5d3b7700 1 --2- 172.21.2.101:0/2365989868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c380186c0 0x7f8c3801abc0 crc :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:11.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.384 7f8c5f59b700 1 -- 172.21.2.101:0/2365989868 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c58105b30 con 0x7f8c580c1f80 2021-02-10T11:29:11.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.386 7f8c49ffb700 1 -- 172.21.2.101:0/2365989868 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8c5000be20 con 0x7f8c580c1f80 2021-02-10T11:29:11.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.513 7f8c5f59b700 1 -- 172.21.2.101:0/2365989868 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839334", "target": ["mgr", ""]}) v1 -- 0x7f8c58105b30 con 0x7f8c380186c0 2021-02-10T11:29:11.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.516 7f8c49ffb700 1 -- 172.21.2.101:0/2365989868 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f8c4c001c40 con 0x7f8c380186c0 2021-02-10T11:29:11.527 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839334/05285056-70e2-466b-82f9-3f991022c260 2021-02-10T11:29:11.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.518 7f8c5f59b700 1 -- 172.21.2.101:0/2365989868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c380186c0 msgr2=0x7f8c3801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:11.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.518 7f8c5f59b700 1 --2- 172.21.2.101:0/2365989868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c380186c0 0x7f8c3801abc0 crc :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:11.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.518 7f8c5f59b700 1 -- 172.21.2.101:0/2365989868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c580c1f80 msgr2=0x7f8c580c2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:11.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.518 7f8c5f59b700 1 --2- 172.21.2.101:0/2365989868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c580c1f80 0x7f8c580c2380 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f8c5000b940 tx=0x7f8c5000bce0).stop 2021-02-10T11:29:11.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.518 7f8c5f59b700 1 -- 172.21.2.101:0/2365989868 shutdown_connections 2021-02-10T11:29:11.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.518 7f8c5f59b700 1 --2- 172.21.2.101:0/2365989868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c380186c0 0x7f8c3801abc0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:11.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.518 7f8c5f59b700 1 --2- 172.21.2.101:0/2365989868 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c580b8410 0x7f8c580b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:11.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.518 7f8c5f59b700 1 --2- 172.21.2.101:0/2365989868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c580c1f80 0x7f8c580c2380 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:11.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.518 7f8c5f59b700 1 --2- 172.21.2.101:0/2365989868 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c5809b930 0x7f8c580b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:11.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.518 7f8c5f59b700 1 -- 172.21.2.101:0/2365989868 >> 172.21.2.101:0/2365989868 conn(0x7f8c58094e10 msgr2=0x7f8c580a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:11.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.518 7f8c5f59b700 1 -- 172.21.2.101:0/2365989868 shutdown_connections 2021-02-10T11:29:11.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:11.518 7f8c5f59b700 1 -- 172.21.2.101:0/2365989868 wait complete. 2021-02-10T11:29:11.537 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839334 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839334/05285056-70e2-466b-82f9-3f991022c260 2021-02-10T11:29:11.537 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:11.537 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:11.537 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:11.874 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:11.875 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:11.875 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839334/05285056-70e2-466b-82f9-3f991022c260/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:12.258 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:12.258 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:12.259 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252357 s, 41.6 MB/s 2021-02-10T11:29:12.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:12.260 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:12.260 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:12.596 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:12.597 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:12.597 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839334/05285056-70e2-466b-82f9-3f991022c260/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:12.971 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:12.971 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:12.971 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0240901 s, 43.5 MB/s 2021-02-10T11:29:12.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:12.972 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:12.973 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:13.309 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:13.309 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:13.309 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839334/05285056-70e2-466b-82f9-3f991022c260/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:13.686 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:13.686 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:13.686 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0226303 s, 46.3 MB/s 2021-02-10T11:29:13.687 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:13.687 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:13.687 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:14.023 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:14.024 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:14.024 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839334/05285056-70e2-466b-82f9-3f991022c260/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:14.398 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:14.398 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:14.398 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253542 s, 41.4 MB/s 2021-02-10T11:29:14.399 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:14.399 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:14.400 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:14.736 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:14.736 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:14.737 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839334/05285056-70e2-466b-82f9-3f991022c260/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:15.112 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:15.112 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:15.112 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0230945 s, 45.4 MB/s 2021-02-10T11:29:15.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:15.114 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:15.114 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:15.449 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:15.450 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:15.450 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839334/05285056-70e2-466b-82f9-3f991022c260/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:15.823 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:15.823 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:15.823 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0220761 s, 47.5 MB/s 2021-02-10T11:29:15.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:15.825 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:15.825 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:16.161 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:16.162 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:16.162 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839334/05285056-70e2-466b-82f9-3f991022c260/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:16.539 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:16.539 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:16.539 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239805 s, 43.7 MB/s 2021-02-10T11:29:16.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:16.540 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:16.540 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:16.876 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:16.877 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:16.877 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839334/05285056-70e2-466b-82f9-3f991022c260/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:17.255 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:17.255 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:17.255 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252557 s, 41.5 MB/s 2021-02-10T11:29:17.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:17.256 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:17.257 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:17.593 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:17.594 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:17.594 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839334/05285056-70e2-466b-82f9-3f991022c260/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:17.972 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:17.972 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:17.972 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251902 s, 41.6 MB/s 2021-02-10T11:29:17.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:17.973 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:17.973 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:18.310 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:18.311 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:18.311 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839334/05285056-70e2-466b-82f9-3f991022c260/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:18.686 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:18.687 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:18.687 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024832 s, 42.2 MB/s 2021-02-10T11:29:18.688 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839335 2021-02-10T11:29:19.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.092 7f67662c1700 1 Processor -- start 2021-02-10T11:29:19.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.093 7f67662c1700 1 -- start start 2021-02-10T11:29:19.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.093 7f67662c1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f67601b0470 0x7f67601b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:19.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.093 7f67662c1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67600ef610 0x7f67600efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:19.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.093 7f67662c1700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67600f0b50 con 0x7f67600f0030 2021-02-10T11:29:19.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.093 7f67662c1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67600e52e0 con 0x7f67601b0470 2021-02-10T11:29:19.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.093 7f67662c1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67600e54d0 con 0x7f67600ef610 2021-02-10T11:29:19.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.093 7f67648de700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f67600f0030 0x7f67600f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:42228/0 (socket says 172.21.2.101:42228) 2021-02-10T11:29:19.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.093 7f67648de700 1 -- 172.21.2.101:0/46269658 learned_addr learned my addr 172.21.2.101:0/46269658 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:29:19.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.093 7f675ffff700 1 --2- 172.21.2.101:0/46269658 >> v2:172.21.2.108:3300/0 conn(0x7f67601b0470 0x7f67601b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:19.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.093 7f675f7fe700 1 --2- 172.21.2.101:0/46269658 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67600ef610 0x7f67600efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:19.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.094 7f675ffff700 1 -- 172.21.2.101:0/46269658 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67600ef610 msgr2=0x7f67600efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:29:19.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.094 7f675ffff700 1 --2- 172.21.2.101:0/46269658 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67600ef610 0x7f67600efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:19.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.094 7f675ffff700 1 -- 172.21.2.101:0/46269658 >> v1:172.21.2.108:6789/0 conn(0x7f67600f0030 legacy=0x7f67600f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:19.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.094 7f675ffff700 1 -- 172.21.2.101:0/46269658 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67600f0430 con 0x7f67601b0470 2021-02-10T11:29:19.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.094 7f675f7fe700 1 --2- 172.21.2.101:0/46269658 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67600ef610 0x7f67600efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:29:19.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.094 7f675ffff700 1 --2- 172.21.2.101:0/46269658 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f67601b0470 0x7f67601b0870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f6750006650 tx=0x7f6750009a80).ready entity=mon.1 client_cookie=ce205517a0e70337 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:19.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.094 7f675effd700 1 -- 172.21.2.101:0/46269658 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f675000c0c0 con 0x7f67601b0470 2021-02-10T11:29:19.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.094 7f675effd700 1 -- 172.21.2.101:0/46269658 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675000c340 con 0x7f67601b0470 2021-02-10T11:29:19.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.094 7f675effd700 1 -- 172.21.2.101:0/46269658 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f675000c860 con 0x7f67601b0470 2021-02-10T11:29:19.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.094 7f67662c1700 1 -- 172.21.2.101:0/46269658 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f67601b0470 msgr2=0x7f67601b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:19.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.094 7f67662c1700 1 --2- 172.21.2.101:0/46269658 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f67601b0470 0x7f67601b0870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f6750006650 tx=0x7f6750009a80).stop 2021-02-10T11:29:19.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.095 7f67662c1700 1 -- 172.21.2.101:0/46269658 shutdown_connections 2021-02-10T11:29:19.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.095 7f67662c1700 1 --2- 172.21.2.101:0/46269658 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67600ef610 0x7f67600efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:19.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.095 7f67662c1700 1 --2- 172.21.2.101:0/46269658 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f67601b0470 0x7f67601b0870 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:19.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.095 7f67662c1700 1 -- 172.21.2.101:0/46269658 >> 172.21.2.101:0/46269658 conn(0x7f67600ce3e0 msgr2=0x7f67600e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:19.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.095 7f67662c1700 1 -- 172.21.2.101:0/46269658 shutdown_connections 2021-02-10T11:29:19.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.095 7f67662c1700 1 -- 172.21.2.101:0/46269658 wait complete. 2021-02-10T11:29:19.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.109 7f67662c1700 1 Processor -- start 2021-02-10T11:29:19.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.109 7f67662c1700 1 -- start start 2021-02-10T11:29:19.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.110 7f67662c1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67600a0420 0x7f67600a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:19.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.110 7f67662c1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6760189d00 0x7f676018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:19.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.110 7f67662c1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f676018a720 0x7f67600b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:19.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.110 7f67662c1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6760189800 con 0x7f676018a720 2021-02-10T11:29:19.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.110 7f67662c1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676018ab20 con 0x7f6760189d00 2021-02-10T11:29:19.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.110 7f67662c1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67600a1140 con 0x7f67600a0420 2021-02-10T11:29:19.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.110 7f67648de700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f676018a720 0x7f67600b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:19.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.110 7f67648de700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f676018a720 0x7f67600b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48328/0 (socket says 172.21.2.101:48328) 2021-02-10T11:29:19.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.110 7f67648de700 1 -- 172.21.2.101:0/563648934 learned_addr learned my addr 172.21.2.101:0/563648934 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:29:19.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.110 7f67648de700 1 -- 172.21.2.101:0/563648934 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67600a0420 msgr2=0x7f67600a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:19.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.110 7f67648de700 1 --2- 172.21.2.101:0/563648934 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67600a0420 0x7f67600a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:19.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.110 7f67648de700 1 -- 172.21.2.101:0/563648934 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6760189d00 msgr2=0x7f676018a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:19.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.110 7f67648de700 1 --2- 172.21.2.101:0/563648934 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6760189d00 0x7f676018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:19.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.110 7f67648de700 1 -- 172.21.2.101:0/563648934 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67600b8200 con 0x7f676018a720 2021-02-10T11:29:19.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.111 7f67648de700 1 --2- 172.21.2.101:0/563648934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f676018a720 0x7f67600b7be0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f6758007fc0 tx=0x7f675800ac80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:19.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.111 7f675cff9700 1 -- 172.21.2.101:0/563648934 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6758007020 con 0x7f676018a720 2021-02-10T11:29:19.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.111 7f675cff9700 1 -- 172.21.2.101:0/563648934 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67580072e0 con 0x7f676018a720 2021-02-10T11:29:19.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.111 7f67662c1700 1 -- 172.21.2.101:0/563648934 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67600b8200 con 0x7f676018a720 2021-02-10T11:29:19.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.111 7f675cff9700 1 -- 172.21.2.101:0/563648934 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f67580046a0 con 0x7f676018a720 2021-02-10T11:29:19.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.111 7f67662c1700 1 -- 172.21.2.101:0/563648934 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67600b8790 con 0x7f676018a720 2021-02-10T11:29:19.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.111 7f675cff9700 1 -- 172.21.2.101:0/563648934 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f67580046a0 con 0x7f676018a720 2021-02-10T11:29:19.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.111 7f675cff9700 1 --2- 172.21.2.101:0/563648934 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6740018730 0x7f674001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:19.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.111 7f675cff9700 1 -- 172.21.2.101:0/563648934 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f67580329c0 con 0x7f676018a720 2021-02-10T11:29:19.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.112 7f675ffff700 1 --2- 172.21.2.101:0/563648934 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6740018730 0x7f674001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:19.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.112 7f675ffff700 1 --2- 172.21.2.101:0/563648934 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6740018730 0x7f674001abc0 crc :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:19.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.143 7f67662c1700 1 -- 172.21.2.101:0/563648934 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6760105ae0 con 0x7f676018a720 2021-02-10T11:29:19.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.145 7f675cff9700 1 -- 172.21.2.101:0/563648934 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f67580011f0 con 0x7f676018a720 2021-02-10T11:29:19.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.271 7f67662c1700 1 -- 172.21.2.101:0/563648934 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839335", "target": ["mgr", ""]}) v1 -- 0x7f67600b9020 con 0x7f6740018730 2021-02-10T11:29:19.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.296 7f675cff9700 1 -- 172.21.2.101:0/563648934 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f6750000940 con 0x7f6740018730 2021-02-10T11:29:19.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.304 7f67662c1700 1 -- 172.21.2.101:0/563648934 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6740018730 msgr2=0x7f674001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:19.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.304 7f67662c1700 1 --2- 172.21.2.101:0/563648934 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6740018730 0x7f674001abc0 crc :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:19.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.304 7f67662c1700 1 -- 172.21.2.101:0/563648934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f676018a720 msgr2=0x7f67600b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:19.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.304 7f67662c1700 1 --2- 172.21.2.101:0/563648934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f676018a720 0x7f67600b7be0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f6758007fc0 tx=0x7f675800ac80).stop 2021-02-10T11:29:19.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.304 7f67662c1700 1 -- 172.21.2.101:0/563648934 shutdown_connections 2021-02-10T11:29:19.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.304 7f67662c1700 1 --2- 172.21.2.101:0/563648934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f676018a720 0x7f67600b7be0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:19.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.304 7f67662c1700 1 --2- 172.21.2.101:0/563648934 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6760189d00 0x7f676018a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:19.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.304 7f67662c1700 1 --2- 172.21.2.101:0/563648934 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6740018730 0x7f674001abc0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:19.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.304 7f67662c1700 1 --2- 172.21.2.101:0/563648934 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67600a0420 0x7f67600a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:19.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.304 7f67662c1700 1 -- 172.21.2.101:0/563648934 >> 172.21.2.101:0/563648934 conn(0x7f6760094e10 msgr2=0x7f67600a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:19.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.305 7f67662c1700 1 -- 172.21.2.101:0/563648934 shutdown_connections 2021-02-10T11:29:19.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.305 7f67662c1700 1 -- 172.21.2.101:0/563648934 wait complete. 2021-02-10T11:29:19.324 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839335 2021-02-10T11:29:19.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.731 7f136593a700 1 Processor -- start 2021-02-10T11:29:19.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.731 7f136593a700 1 -- start start 2021-02-10T11:29:19.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.731 7f136593a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13601b0470 0x7f13601b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:19.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.731 7f136593a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13600f0b50 con 0x7f13600ef610 2021-02-10T11:29:19.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.731 7f136593a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13600f24e0 con 0x7f13600f20e0 2021-02-10T11:29:19.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.731 7f136593a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13600e5ea0 con 0x7f13601b0470 2021-02-10T11:29:19.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.731 7f135ed73700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f13600ef610 0x7f13600efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48830/0 (socket says 172.21.2.101:48830) 2021-02-10T11:29:19.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.731 7f135ed73700 1 -- 172.21.2.101:0/1264984861 learned_addr learned my addr 172.21.2.101:0/1264984861 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:29:19.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.731 7f135e572700 1 -- 172.21.2.101:0/1264984861 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 426541109 0 0) 0x7f13540010b0 con 0x7f13600ef610 2021-02-10T11:29:19.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.731 7f135e572700 1 -- 172.21.2.101:0/1264984861 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1348002fe0 con 0x7f13600ef610 2021-02-10T11:29:19.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135e572700 1 -- 172.21.2.101:0/1264984861 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2496395700 0 0) 0x7f1354001010 con 0x7f13600ef610 2021-02-10T11:29:19.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135e572700 1 -- 172.21.2.101:0/1264984861 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13601b0470 msgr2=0x7f13601b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:29:19.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135e572700 1 --2- 172.21.2.101:0/1264984861 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13601b0470 0x7f13601b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:19.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135e572700 1 -- 172.21.2.101:0/1264984861 >> v1:172.21.2.108:6789/0 conn(0x7f13600f20e0 legacy=0x7f13600f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:29:19.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135e572700 1 -- 172.21.2.101:0/1264984861 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13600e52e0 con 0x7f13600ef610 2021-02-10T11:29:19.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135e572700 1 -- 172.21.2.101:0/1264984861 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f1354001530 con 0x7f13600ef610 2021-02-10T11:29:19.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135e572700 1 -- 172.21.2.101:0/1264984861 >> v1:172.21.2.101:6789/0 conn(0x7f13600ef610 legacy=0x7f13600efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:19.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135e572700 1 --2- 172.21.2.101:0/1264984861 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1348003b10 0x7f1348005f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:19.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135e572700 1 --2- 172.21.2.101:0/1264984861 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f13480067a0 0x7f1348008ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:19.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135e572700 1 --2- 172.21.2.101:0/1264984861 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13480095a0 0x7f134800ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:19.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135e572700 1 -- 172.21.2.101:0/1264984861 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134800c2c0 con 0x7f1348003b10 2021-02-10T11:29:19.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135e572700 1 -- 172.21.2.101:0/1264984861 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134800c4e0 con 0x7f13480067a0 2021-02-10T11:29:19.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135e572700 1 -- 172.21.2.101:0/1264984861 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134800c700 con 0x7f13480095a0 2021-02-10T11:29:19.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135f574700 1 --2- 172.21.2.101:0/1264984861 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1348003b10 0x7f1348005f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:19.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135ed73700 1 --2- 172.21.2.101:0/1264984861 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f13480067a0 0x7f1348008ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:19.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135fd75700 1 --2- 172.21.2.101:0/1264984861 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13480095a0 0x7f134800ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:19.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135f574700 1 -- 172.21.2.101:0/1264984861 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13480095a0 msgr2=0x7f134800ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:29:19.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135f574700 1 --2- 172.21.2.101:0/1264984861 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13480095a0 0x7f134800ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:19.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135f574700 1 -- 172.21.2.101:0/1264984861 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f13480067a0 msgr2=0x7f1348008ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:29:19.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135f574700 1 --2- 172.21.2.101:0/1264984861 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f13480067a0 0x7f1348008ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:19.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135f574700 1 -- 172.21.2.101:0/1264984861 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134800c9a0 con 0x7f1348003b10 2021-02-10T11:29:19.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135fd75700 1 --2- 172.21.2.101:0/1264984861 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13480095a0 0x7f134800ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:29:19.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.732 7f135f574700 1 --2- 172.21.2.101:0/1264984861 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1348003b10 0x7f1348005f20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7f1350006ab0 tx=0x7f135004fbc0).ready entity=mon.0 client_cookie=91b994b016bd6be5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:19.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.733 7f135e572700 1 -- 172.21.2.101:0/1264984861 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1350002080 con 0x7f1348003b10 2021-02-10T11:29:19.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.733 7f135e572700 1 -- 172.21.2.101:0/1264984861 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13500063f0 con 0x7f1348003b10 2021-02-10T11:29:19.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.733 7f135e572700 1 -- 172.21.2.101:0/1264984861 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1350002380 con 0x7f1348003b10 2021-02-10T11:29:19.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.733 7f136593a700 1 -- 172.21.2.101:0/1264984861 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1348003b10 msgr2=0x7f1348005f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:19.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.733 7f136593a700 1 --2- 172.21.2.101:0/1264984861 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1348003b10 0x7f1348005f20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7f1350006ab0 tx=0x7f135004fbc0).stop 2021-02-10T11:29:19.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.733 7f135fd75700 1 -- 172.21.2.101:0/1264984861 reap_dead start 2021-02-10T11:29:19.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.733 7f136593a700 1 -- 172.21.2.101:0/1264984861 shutdown_connections 2021-02-10T11:29:19.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.733 7f136593a700 1 -- 172.21.2.101:0/1264984861 >> 172.21.2.101:0/1264984861 conn(0x7f13600ce3e0 msgr2=0x7f13600e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:19.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.733 7f136593a700 1 -- 172.21.2.101:0/1264984861 shutdown_connections 2021-02-10T11:29:19.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.733 7f136593a700 1 -- 172.21.2.101:0/1264984861 wait complete. 2021-02-10T11:29:19.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.748 7f136593a700 1 Processor -- start 2021-02-10T11:29:19.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.748 7f136593a700 1 -- start start 2021-02-10T11:29:19.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.748 7f136593a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f136010a540 0x7f13600a86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:19.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.748 7f136593a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1360190a70 0x7f13600a8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:19.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.748 7f136593a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13600a0ac0 0x7f13600a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:19.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.748 7f136593a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13600a9430 con 0x7f136010a540 2021-02-10T11:29:19.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.748 7f136593a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1360190630 con 0x7f1360190a70 2021-02-10T11:29:19.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.748 7f136593a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13600a9fd0 con 0x7f13600a0ac0 2021-02-10T11:29:19.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.749 7f135f574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f136010a540 0x7f13600a86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:19.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.749 7f135f574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f136010a540 0x7f13600a86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48348/0 (socket says 172.21.2.101:48348) 2021-02-10T11:29:19.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.749 7f135f574700 1 -- 172.21.2.101:0/4153474729 learned_addr learned my addr 172.21.2.101:0/4153474729 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:29:19.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.749 7f135fd75700 1 --2- 172.21.2.101:0/4153474729 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13600a0ac0 0x7f13600a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:19.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.749 7f135ed73700 1 --2- 172.21.2.101:0/4153474729 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1360190a70 0x7f13600a8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:19.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.749 7f135f574700 1 -- 172.21.2.101:0/4153474729 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13600a0ac0 msgr2=0x7f13600a99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:19.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.749 7f135f574700 1 --2- 172.21.2.101:0/4153474729 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13600a0ac0 0x7f13600a99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:19.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.749 7f135f574700 1 -- 172.21.2.101:0/4153474729 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1360190a70 msgr2=0x7f13600a8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:19.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.749 7f135f574700 1 --2- 172.21.2.101:0/4153474729 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1360190a70 0x7f13600a8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:19.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.749 7f135f574700 1 -- 172.21.2.101:0/4153474729 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13600aa1f0 con 0x7f136010a540 2021-02-10T11:29:19.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.749 7f135f574700 1 --2- 172.21.2.101:0/4153474729 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f136010a540 0x7f13600a86f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f13500067b0 tx=0x7f1350050930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:19.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.749 7f1347fff700 1 -- 172.21.2.101:0/4153474729 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f13500009b0 con 0x7f136010a540 2021-02-10T11:29:19.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.749 7f1347fff700 1 -- 172.21.2.101:0/4153474729 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1350000de0 con 0x7f136010a540 2021-02-10T11:29:19.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.749 7f1347fff700 1 -- 172.21.2.101:0/4153474729 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1350002220 con 0x7f136010a540 2021-02-10T11:29:19.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.749 7f136593a700 1 -- 172.21.2.101:0/4153474729 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13600aa1f0 con 0x7f136010a540 2021-02-10T11:29:19.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.749 7f136593a700 1 -- 172.21.2.101:0/4153474729 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13601a7440 con 0x7f136010a540 2021-02-10T11:29:19.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.750 7f1347fff700 1 -- 172.21.2.101:0/4153474729 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f135004e050 con 0x7f136010a540 2021-02-10T11:29:19.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.750 7f1347fff700 1 --2- 172.21.2.101:0/4153474729 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13400186c0 0x7f134001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:19.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.750 7f1347fff700 1 -- 172.21.2.101:0/4153474729 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f13500765b0 con 0x7f136010a540 2021-02-10T11:29:19.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.750 7f135ed73700 1 --2- 172.21.2.101:0/4153474729 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13400186c0 0x7f134001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:19.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.750 7f135ed73700 1 --2- 172.21.2.101:0/4153474729 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13400186c0 0x7f134001abc0 crc :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:19.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.781 7f136593a700 1 -- 172.21.2.101:0/4153474729 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1360105b30 con 0x7f136010a540 2021-02-10T11:29:19.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.783 7f1347fff700 1 -- 172.21.2.101:0/4153474729 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f135006e670 con 0x7f136010a540 2021-02-10T11:29:19.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.913 7f136593a700 1 -- 172.21.2.101:0/4153474729 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839335", "target": ["mgr", ""]}) v1 -- 0x7f13600a9430 con 0x7f13400186c0 2021-02-10T11:29:19.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.917 7f1347fff700 1 -- 172.21.2.101:0/4153474729 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f13540014b0 con 0x7f13400186c0 2021-02-10T11:29:19.931 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839335/ecfa6e6b-db61-41b3-8459-8943a9546bc7 2021-02-10T11:29:19.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.922 7f136593a700 1 -- 172.21.2.101:0/4153474729 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13400186c0 msgr2=0x7f134001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:19.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.922 7f136593a700 1 --2- 172.21.2.101:0/4153474729 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13400186c0 0x7f134001abc0 crc :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:19.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.922 7f136593a700 1 -- 172.21.2.101:0/4153474729 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f136010a540 msgr2=0x7f13600a86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:19.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.922 7f136593a700 1 --2- 172.21.2.101:0/4153474729 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f136010a540 0x7f13600a86f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f13500067b0 tx=0x7f1350050930).stop 2021-02-10T11:29:19.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.922 7f136593a700 1 -- 172.21.2.101:0/4153474729 shutdown_connections 2021-02-10T11:29:19.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.922 7f136593a700 1 --2- 172.21.2.101:0/4153474729 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13400186c0 0x7f134001abc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:19.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.922 7f136593a700 1 --2- 172.21.2.101:0/4153474729 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13600a0ac0 0x7f13600a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:19.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.922 7f136593a700 1 --2- 172.21.2.101:0/4153474729 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1360190a70 0x7f13600a8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:19.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.922 7f136593a700 1 --2- 172.21.2.101:0/4153474729 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f136010a540 0x7f13600a86f0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:19.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.922 7f136593a700 1 -- 172.21.2.101:0/4153474729 >> 172.21.2.101:0/4153474729 conn(0x7f13600c40e0 msgr2=0x7f1360094a30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:19.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.923 7f136593a700 1 -- 172.21.2.101:0/4153474729 shutdown_connections 2021-02-10T11:29:19.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:19.923 7f136593a700 1 -- 172.21.2.101:0/4153474729 wait complete. 2021-02-10T11:29:19.941 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839335 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839335/ecfa6e6b-db61-41b3-8459-8943a9546bc7 2021-02-10T11:29:19.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:19.942 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:19.942 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:20.279 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:20.279 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:20.279 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839335/ecfa6e6b-db61-41b3-8459-8943a9546bc7/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:20.661 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:20.661 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:20.661 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245189 s, 42.8 MB/s 2021-02-10T11:29:20.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:20.662 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:20.663 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:20.999 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:21.000 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:21.000 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839335/ecfa6e6b-db61-41b3-8459-8943a9546bc7/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:21.376 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:21.376 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:21.376 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244558 s, 42.9 MB/s 2021-02-10T11:29:21.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:21.377 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:21.377 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:21.713 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:21.713 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:21.714 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839335/ecfa6e6b-db61-41b3-8459-8943a9546bc7/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:22.115 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:22.116 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:22.116 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248125 s, 42.3 MB/s 2021-02-10T11:29:22.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:22.118 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:22.118 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:22.461 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:22.462 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:22.462 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839335/ecfa6e6b-db61-41b3-8459-8943a9546bc7/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:22.842 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:22.842 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:22.843 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239556 s, 43.8 MB/s 2021-02-10T11:29:22.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:22.844 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:22.844 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:23.182 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:23.183 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:23.183 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839335/ecfa6e6b-db61-41b3-8459-8943a9546bc7/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:23.561 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:23.562 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:23.562 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0226342 s, 46.3 MB/s 2021-02-10T11:29:23.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:23.563 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:23.563 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:23.900 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:23.901 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:23.901 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839335/ecfa6e6b-db61-41b3-8459-8943a9546bc7/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:24.273 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:24.273 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:24.273 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0222018 s, 47.2 MB/s 2021-02-10T11:29:24.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:24.274 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:24.274 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:24.610 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:24.611 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:24.611 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839335/ecfa6e6b-db61-41b3-8459-8943a9546bc7/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:24.987 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:24.987 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:24.988 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.02585 s, 40.6 MB/s 2021-02-10T11:29:24.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:24.989 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:24.989 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:25.299 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:29:25.303 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:29:25.323 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:25.323 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:25.324 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839335/ecfa6e6b-db61-41b3-8459-8943a9546bc7/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:25.674 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:25.675 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:25.675 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248309 s, 42.2 MB/s 2021-02-10T11:29:25.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:25.676 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:25.676 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:26.011 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:26.012 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:26.012 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839335/ecfa6e6b-db61-41b3-8459-8943a9546bc7/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:26.388 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:26.388 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:26.388 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238072 s, 44.0 MB/s 2021-02-10T11:29:26.389 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:26.389 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:26.389 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:26.724 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:26.725 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:26.725 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839335/ecfa6e6b-db61-41b3-8459-8943a9546bc7/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:27.099 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:27.099 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:27.099 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236726 s, 44.3 MB/s 2021-02-10T11:29:27.100 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839336 2021-02-10T11:29:27.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.501 7fe04fb8f700 1 Processor -- start 2021-02-10T11:29:27.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.501 7fe04fb8f700 1 -- start start 2021-02-10T11:29:27.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.502 7fe04fb8f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe0481b0470 0x7fe0481b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:27.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.502 7fe04fb8f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0480f0b50 con 0x7fe0480f20e0 2021-02-10T11:29:27.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.502 7fe04fb8f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0480f24e0 con 0x7fe0480ef610 2021-02-10T11:29:27.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.502 7fe04fb8f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0480e5ea0 con 0x7fe0481b0470 2021-02-10T11:29:27.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.502 7fe04e1ac700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe0480f20e0 0x7fe0480f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48848/0 (socket says 172.21.2.101:48848) 2021-02-10T11:29:27.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.502 7fe04e1ac700 1 -- 172.21.2.101:0/1261268230 learned_addr learned my addr 172.21.2.101:0/1261268230 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:29:27.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.502 7fe04d9ab700 1 --2- 172.21.2.101:0/1261268230 >> v2:172.21.2.108:3300/0 conn(0x7fe0481b0470 0x7fe0481b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:27.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.502 7fe04c9a9700 1 -- 172.21.2.101:0/1261268230 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4137563186 0 0) 0x7fe040001500 con 0x7fe0480f20e0 2021-02-10T11:29:27.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.502 7fe04c9a9700 1 -- 172.21.2.101:0/1261268230 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe034002fe0 con 0x7fe0480f20e0 2021-02-10T11:29:27.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.502 7fe04c9a9700 1 -- 172.21.2.101:0/1261268230 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693577178 0 0) 0x7fe03c000ff0 con 0x7fe0480ef610 2021-02-10T11:29:27.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.502 7fe04c9a9700 1 -- 172.21.2.101:0/1261268230 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0340042c0 con 0x7fe0480ef610 2021-02-10T11:29:27.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.502 7fe04d9ab700 1 -- 172.21.2.101:0/1261268230 >> v1:172.21.2.108:6789/0 conn(0x7fe0480ef610 legacy=0x7fe0480efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:27.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.502 7fe04d9ab700 1 -- 172.21.2.101:0/1261268230 >> v1:172.21.2.101:6789/0 conn(0x7fe0480f20e0 legacy=0x7fe0480f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:27.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.502 7fe04d9ab700 1 -- 172.21.2.101:0/1261268230 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0480e52e0 con 0x7fe0481b0470 2021-02-10T11:29:27.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.503 7fe04d9ab700 1 --2- 172.21.2.101:0/1261268230 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe0481b0470 0x7fe0481b0870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fe044006650 tx=0x7fe044009a00).ready entity=mon.1 client_cookie=69dc685ce31f76b7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:27.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.503 7fe04c9a9700 1 -- 172.21.2.101:0/1261268230 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe0440009b0 con 0x7fe0481b0470 2021-02-10T11:29:27.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.503 7fe04c9a9700 1 -- 172.21.2.101:0/1261268230 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04400c0c0 con 0x7fe0481b0470 2021-02-10T11:29:27.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.503 7fe04c9a9700 1 -- 172.21.2.101:0/1261268230 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe04400c5a0 con 0x7fe0481b0470 2021-02-10T11:29:27.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.503 7fe04fb8f700 1 -- 172.21.2.101:0/1261268230 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe0481b0470 msgr2=0x7fe0481b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:27.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.503 7fe04fb8f700 1 --2- 172.21.2.101:0/1261268230 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe0481b0470 0x7fe0481b0870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fe044006650 tx=0x7fe044009a00).stop 2021-02-10T11:29:27.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.504 7fe04fb8f700 1 -- 172.21.2.101:0/1261268230 shutdown_connections 2021-02-10T11:29:27.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.504 7fe04fb8f700 1 --2- 172.21.2.101:0/1261268230 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe0481b0470 0x7fe0481b0870 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:27.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.504 7fe04fb8f700 1 -- 172.21.2.101:0/1261268230 >> 172.21.2.101:0/1261268230 conn(0x7fe0480ce3e0 msgr2=0x7fe0480e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:27.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.504 7fe04fb8f700 1 -- 172.21.2.101:0/1261268230 shutdown_connections 2021-02-10T11:29:27.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.504 7fe04fb8f700 1 -- 172.21.2.101:0/1261268230 wait complete. 2021-02-10T11:29:27.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.518 7fe04fb8f700 1 Processor -- start 2021-02-10T11:29:27.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.519 7fe04fb8f700 1 -- start start 2021-02-10T11:29:27.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.519 7fe04fb8f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0480a0420 0x7fe0480a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:27.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.519 7fe04fb8f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe048189d00 0x7fe04818a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:27.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.519 7fe04fb8f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe04818a720 0x7fe0480b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:27.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.519 7fe04fb8f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe048189780 con 0x7fe0480a0420 2021-02-10T11:29:27.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.519 7fe04fb8f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04818ab20 con 0x7fe04818a720 2021-02-10T11:29:27.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.519 7fe04fb8f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0480a1140 con 0x7fe048189d00 2021-02-10T11:29:27.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.519 7fe04d9ab700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0480a0420 0x7fe0480a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:27.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.519 7fe04d9ab700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0480a0420 0x7fe0480a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48358/0 (socket says 172.21.2.101:48358) 2021-02-10T11:29:27.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.519 7fe04d9ab700 1 -- 172.21.2.101:0/2178132575 learned_addr learned my addr 172.21.2.101:0/2178132575 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:29:27.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.519 7fe04e1ac700 1 --2- 172.21.2.101:0/2178132575 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe04818a720 0x7fe0480b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:27.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.519 7fe04d1aa700 1 --2- 172.21.2.101:0/2178132575 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe048189d00 0x7fe04818a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:27.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.519 7fe04d9ab700 1 -- 172.21.2.101:0/2178132575 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe048189d00 msgr2=0x7fe04818a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:27.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.519 7fe04d9ab700 1 --2- 172.21.2.101:0/2178132575 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe048189d00 0x7fe04818a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:27.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.519 7fe04d9ab700 1 -- 172.21.2.101:0/2178132575 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe04818a720 msgr2=0x7fe0480b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:27.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.519 7fe04d9ab700 1 --2- 172.21.2.101:0/2178132575 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe04818a720 0x7fe0480b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:27.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.519 7fe04d9ab700 1 -- 172.21.2.101:0/2178132575 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0480b8200 con 0x7fe0480a0420 2021-02-10T11:29:27.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.520 7fe04d1aa700 1 --2- 172.21.2.101:0/2178132575 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe048189d00 0x7fe04818a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:29:27.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.520 7fe04d9ab700 1 --2- 172.21.2.101:0/2178132575 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0480a0420 0x7fe0480a0820 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7fe04400b640 tx=0x7fe04400b9e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:27.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.520 7fe03a7fc700 1 -- 172.21.2.101:0/2178132575 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe04400bdd0 con 0x7fe0480a0420 2021-02-10T11:29:27.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.520 7fe03a7fc700 1 -- 172.21.2.101:0/2178132575 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe044000c90 con 0x7fe0480a0420 2021-02-10T11:29:27.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.520 7fe04fb8f700 1 -- 172.21.2.101:0/2178132575 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0480b8200 con 0x7fe0480a0420 2021-02-10T11:29:27.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.520 7fe03a7fc700 1 -- 172.21.2.101:0/2178132575 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe0440022b0 con 0x7fe0480a0420 2021-02-10T11:29:27.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.520 7fe04fb8f700 1 -- 172.21.2.101:0/2178132575 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0480b8200 con 0x7fe0480a0420 2021-02-10T11:29:27.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.521 7fe03a7fc700 1 -- 172.21.2.101:0/2178132575 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe044006a10 con 0x7fe0480a0420 2021-02-10T11:29:27.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.521 7fe03a7fc700 1 --2- 172.21.2.101:0/2178132575 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe030018730 0x7fe03001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:27.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.521 7fe03a7fc700 1 -- 172.21.2.101:0/2178132575 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe044032800 con 0x7fe0480a0420 2021-02-10T11:29:27.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.521 7fe04d1aa700 1 --2- 172.21.2.101:0/2178132575 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe030018730 0x7fe03001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:27.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.521 7fe04d1aa700 1 --2- 172.21.2.101:0/2178132575 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe030018730 0x7fe03001ac10 crc :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:27.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.551 7fe04fb8f700 1 -- 172.21.2.101:0/2178132575 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe048105ae0 con 0x7fe0480a0420 2021-02-10T11:29:27.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.553 7fe03a7fc700 1 -- 172.21.2.101:0/2178132575 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe044000c90 con 0x7fe0480a0420 2021-02-10T11:29:27.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.681 7fe04fb8f700 1 -- 172.21.2.101:0/2178132575 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839336", "target": ["mgr", ""]}) v1 -- 0x7fe048105ae0 con 0x7fe030018730 2021-02-10T11:29:27.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.707 7fe03a7fc700 1 -- 172.21.2.101:0/2178132575 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fe03c006930 con 0x7fe030018730 2021-02-10T11:29:27.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.714 7fe04fb8f700 1 -- 172.21.2.101:0/2178132575 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe030018730 msgr2=0x7fe03001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:27.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.714 7fe04fb8f700 1 --2- 172.21.2.101:0/2178132575 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe030018730 0x7fe03001ac10 crc :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:27.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.714 7fe04fb8f700 1 -- 172.21.2.101:0/2178132575 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0480a0420 msgr2=0x7fe0480a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:27.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.714 7fe04fb8f700 1 --2- 172.21.2.101:0/2178132575 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0480a0420 0x7fe0480a0820 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7fe04400b640 tx=0x7fe04400b9e0).stop 2021-02-10T11:29:27.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.714 7fe04fb8f700 1 -- 172.21.2.101:0/2178132575 shutdown_connections 2021-02-10T11:29:27.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.714 7fe04fb8f700 1 --2- 172.21.2.101:0/2178132575 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe030018730 0x7fe03001ac10 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:27.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.714 7fe04fb8f700 1 --2- 172.21.2.101:0/2178132575 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe048189d00 0x7fe04818a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:27.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.714 7fe04fb8f700 1 --2- 172.21.2.101:0/2178132575 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe04818a720 0x7fe0480b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:27.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.714 7fe04fb8f700 1 --2- 172.21.2.101:0/2178132575 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0480a0420 0x7fe0480a0820 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:27.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.714 7fe04fb8f700 1 -- 172.21.2.101:0/2178132575 >> 172.21.2.101:0/2178132575 conn(0x7fe048094e10 msgr2=0x7fe0480a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:27.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.714 7fe04fb8f700 1 -- 172.21.2.101:0/2178132575 shutdown_connections 2021-02-10T11:29:27.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:27.714 7fe04fb8f700 1 -- 172.21.2.101:0/2178132575 wait complete. 2021-02-10T11:29:27.733 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839336 2021-02-10T11:29:28.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.134 7fc398988700 1 Processor -- start 2021-02-10T11:29:28.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.134 7fc398988700 1 -- start start 2021-02-10T11:29:28.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.135 7fc398988700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3941b0870 0x7fc3940f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:28.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.135 7fc398988700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc3940efe00 0x7fc3940f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:28.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.135 7fc398988700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3941b0c70 con 0x7fc3941b0470 2021-02-10T11:29:28.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.135 7fc398988700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3940f0b50 con 0x7fc3940efe00 2021-02-10T11:29:28.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.135 7fc398988700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3940f0820 con 0x7fc3941b0870 2021-02-10T11:29:28.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.135 7fc39261c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc3941b0470 0x7fc3940ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48858/0 (socket says 172.21.2.101:48858) 2021-02-10T11:29:28.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.135 7fc39261c700 1 -- 172.21.2.101:0/3159054654 learned_addr learned my addr 172.21.2.101:0/3159054654 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:29:28.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.135 7fc392e1d700 1 --2- 172.21.2.101:0/3159054654 >> v2:172.21.2.108:3300/0 conn(0x7fc3940efe00 0x7fc3940f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:28.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.135 7fc391e1b700 1 --2- 172.21.2.101:0/3159054654 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3941b0870 0x7fc3940f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:28.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.135 7fc39161a700 1 -- 172.21.2.101:0/3159054654 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4036662284 0 0) 0x7fc37c0010b0 con 0x7fc3941b0470 2021-02-10T11:29:28.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.135 7fc39161a700 1 -- 172.21.2.101:0/3159054654 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc380002fe0 con 0x7fc3941b0470 2021-02-10T11:29:28.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.136 7fc392e1d700 1 -- 172.21.2.101:0/3159054654 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3941b0870 msgr2=0x7fc3940f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:29:28.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.136 7fc392e1d700 1 --2- 172.21.2.101:0/3159054654 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3941b0870 0x7fc3940f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:28.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.136 7fc392e1d700 1 -- 172.21.2.101:0/3159054654 >> v1:172.21.2.101:6789/0 conn(0x7fc3941b0470 legacy=0x7fc3940ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:28.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.136 7fc392e1d700 1 -- 172.21.2.101:0/3159054654 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3940e52e0 con 0x7fc3940efe00 2021-02-10T11:29:28.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.136 7fc391e1b700 1 --2- 172.21.2.101:0/3159054654 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3941b0870 0x7fc3940f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:29:28.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.136 7fc392e1d700 1 --2- 172.21.2.101:0/3159054654 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3940efe00 0x7fc3940f0200 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fc38c006e50 tx=0x7fc38c009e70).ready entity=mon.1 client_cookie=d79b4a3049ddb12b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:28.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.136 7fc39161a700 1 -- 172.21.2.101:0/3159054654 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc38c006aa0 con 0x7fc3940efe00 2021-02-10T11:29:28.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.136 7fc39161a700 1 -- 172.21.2.101:0/3159054654 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38c006790 con 0x7fc3940efe00 2021-02-10T11:29:28.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.136 7fc39161a700 1 -- 172.21.2.101:0/3159054654 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc38c00b700 con 0x7fc3940efe00 2021-02-10T11:29:28.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.136 7fc398988700 1 -- 172.21.2.101:0/3159054654 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3940efe00 msgr2=0x7fc3940f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:28.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.136 7fc398988700 1 --2- 172.21.2.101:0/3159054654 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3940efe00 0x7fc3940f0200 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fc38c006e50 tx=0x7fc38c009e70).stop 2021-02-10T11:29:28.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.137 7fc398988700 1 -- 172.21.2.101:0/3159054654 shutdown_connections 2021-02-10T11:29:28.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.137 7fc398988700 1 --2- 172.21.2.101:0/3159054654 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3941b0870 0x7fc3940f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:28.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.137 7fc398988700 1 --2- 172.21.2.101:0/3159054654 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3940efe00 0x7fc3940f0200 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:28.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.137 7fc398988700 1 -- 172.21.2.101:0/3159054654 >> 172.21.2.101:0/3159054654 conn(0x7fc3940ce3e0 msgr2=0x7fc3940e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:28.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.137 7fc398988700 1 -- 172.21.2.101:0/3159054654 shutdown_connections 2021-02-10T11:29:28.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.137 7fc398988700 1 -- 172.21.2.101:0/3159054654 wait complete. 2021-02-10T11:29:28.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.152 7fc398988700 1 Processor -- start 2021-02-10T11:29:28.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.152 7fc398988700 1 -- start start 2021-02-10T11:29:28.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.152 7fc398988700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3940a0420 0x7fc3940a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:28.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.152 7fc398988700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc394189d00 0x7fc39418a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:28.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.152 7fc398988700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc39418a720 0x7fc3940b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:28.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.152 7fc398988700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc394189800 con 0x7fc394189d00 2021-02-10T11:29:28.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.152 7fc398988700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39418ab20 con 0x7fc39418a720 2021-02-10T11:29:28.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.152 7fc398988700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3940a1140 con 0x7fc3940a0420 2021-02-10T11:29:28.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.153 7fc391e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc394189d00 0x7fc39418a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:28.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.153 7fc391e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc394189d00 0x7fc39418a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48372/0 (socket says 172.21.2.101:48372) 2021-02-10T11:29:28.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.153 7fc391e1b700 1 -- 172.21.2.101:0/2482874570 learned_addr learned my addr 172.21.2.101:0/2482874570 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:29:28.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.153 7fc391e1b700 1 -- 172.21.2.101:0/2482874570 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3940a0420 msgr2=0x7fc3940a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:28.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.153 7fc391e1b700 1 --2- 172.21.2.101:0/2482874570 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3940a0420 0x7fc3940a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:28.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.153 7fc391e1b700 1 -- 172.21.2.101:0/2482874570 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc39418a720 msgr2=0x7fc3940b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:28.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.153 7fc391e1b700 1 --2- 172.21.2.101:0/2482874570 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc39418a720 0x7fc3940b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:28.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.153 7fc391e1b700 1 -- 172.21.2.101:0/2482874570 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3940b8200 con 0x7fc394189d00 2021-02-10T11:29:28.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.153 7fc391e1b700 1 --2- 172.21.2.101:0/2482874570 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc394189d00 0x7fc39418a100 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7fc38400b580 tx=0x7fc38400b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:28.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.153 7fc38affd700 1 -- 172.21.2.101:0/2482874570 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc38400bd10 con 0x7fc394189d00 2021-02-10T11:29:28.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.153 7fc38affd700 1 -- 172.21.2.101:0/2482874570 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc384000c90 con 0x7fc394189d00 2021-02-10T11:29:28.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.153 7fc38affd700 1 -- 172.21.2.101:0/2482874570 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc384002220 con 0x7fc394189d00 2021-02-10T11:29:28.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.153 7fc398988700 1 -- 172.21.2.101:0/2482874570 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3940b8200 con 0x7fc394189d00 2021-02-10T11:29:28.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.153 7fc398988700 1 -- 172.21.2.101:0/2482874570 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3940b8790 con 0x7fc394189d00 2021-02-10T11:29:28.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.154 7fc38affd700 1 -- 172.21.2.101:0/2482874570 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc38400a000 con 0x7fc394189d00 2021-02-10T11:29:28.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.154 7fc38affd700 1 --2- 172.21.2.101:0/2482874570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3780186c0 0x7fc37801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:28.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.154 7fc38affd700 1 -- 172.21.2.101:0/2482874570 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc384032470 con 0x7fc394189d00 2021-02-10T11:29:28.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.154 7fc39261c700 1 --2- 172.21.2.101:0/2482874570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3780186c0 0x7fc37801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:28.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.154 7fc39261c700 1 --2- 172.21.2.101:0/2482874570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3780186c0 0x7fc37801abc0 crc :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:28.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.185 7fc398988700 1 -- 172.21.2.101:0/2482874570 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc394105ae0 con 0x7fc394189d00 2021-02-10T11:29:28.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.187 7fc38affd700 1 -- 172.21.2.101:0/2482874570 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc384030da0 con 0x7fc394189d00 2021-02-10T11:29:28.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.315 7fc398988700 1 -- 172.21.2.101:0/2482874570 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839336", "target": ["mgr", ""]}) v1 -- 0x7fc3940b9020 con 0x7fc3780186c0 2021-02-10T11:29:28.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.318 7fc38affd700 1 -- 172.21.2.101:0/2482874570 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fc37c001590 con 0x7fc3780186c0 2021-02-10T11:29:28.329 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839336/3cdc4307-1bc3-4abf-9341-566870fe950e 2021-02-10T11:29:28.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.319 7fc398988700 1 -- 172.21.2.101:0/2482874570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3780186c0 msgr2=0x7fc37801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:28.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.319 7fc398988700 1 --2- 172.21.2.101:0/2482874570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3780186c0 0x7fc37801abc0 crc :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:28.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.319 7fc398988700 1 -- 172.21.2.101:0/2482874570 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc394189d00 msgr2=0x7fc39418a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:28.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.319 7fc398988700 1 --2- 172.21.2.101:0/2482874570 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc394189d00 0x7fc39418a100 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7fc38400b580 tx=0x7fc38400b920).stop 2021-02-10T11:29:28.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.319 7fc398988700 1 -- 172.21.2.101:0/2482874570 shutdown_connections 2021-02-10T11:29:28.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.319 7fc398988700 1 --2- 172.21.2.101:0/2482874570 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc39418a720 0x7fc3940b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:28.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.319 7fc398988700 1 --2- 172.21.2.101:0/2482874570 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc394189d00 0x7fc39418a100 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:28.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.319 7fc398988700 1 --2- 172.21.2.101:0/2482874570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3780186c0 0x7fc37801abc0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:28.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.319 7fc398988700 1 --2- 172.21.2.101:0/2482874570 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3940a0420 0x7fc3940a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:28.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.319 7fc398988700 1 -- 172.21.2.101:0/2482874570 >> 172.21.2.101:0/2482874570 conn(0x7fc394094e10 msgr2=0x7fc3940a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:28.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.319 7fc398988700 1 -- 172.21.2.101:0/2482874570 shutdown_connections 2021-02-10T11:29:28.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:28.319 7fc398988700 1 -- 172.21.2.101:0/2482874570 wait complete. 2021-02-10T11:29:28.338 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839336 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839336/3cdc4307-1bc3-4abf-9341-566870fe950e 2021-02-10T11:29:28.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:28.339 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:28.339 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:28.673 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:28.674 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:28.674 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839336/3cdc4307-1bc3-4abf-9341-566870fe950e/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:29.052 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:29.052 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:29.052 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0250098 s, 41.9 MB/s 2021-02-10T11:29:29.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:29.053 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:29.053 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:29.387 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:29.387 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:29.387 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839336/3cdc4307-1bc3-4abf-9341-566870fe950e/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:29.763 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:29.763 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:29.764 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0256824 s, 40.8 MB/s 2021-02-10T11:29:29.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:29.765 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:29.765 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:30.098 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:30.099 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:30.099 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839336/3cdc4307-1bc3-4abf-9341-566870fe950e/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:30.474 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:30.474 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:30.474 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237354 s, 44.2 MB/s 2021-02-10T11:29:30.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:30.475 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:30.475 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:30.808 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:30.808 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:30.808 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839336/3cdc4307-1bc3-4abf-9341-566870fe950e/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:31.183 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:31.183 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:31.183 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242796 s, 43.2 MB/s 2021-02-10T11:29:31.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:31.184 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:31.184 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:31.519 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:31.520 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:31.520 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839336/3cdc4307-1bc3-4abf-9341-566870fe950e/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:31.894 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:31.895 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:31.895 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0242364 s, 43.3 MB/s 2021-02-10T11:29:31.896 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:31.896 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:31.896 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:32.229 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:32.229 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:32.230 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839336/3cdc4307-1bc3-4abf-9341-566870fe950e/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:32.605 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:32.605 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:32.605 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0255575 s, 41.0 MB/s 2021-02-10T11:29:32.606 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:32.606 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:32.606 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:32.939 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:32.940 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:32.940 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839336/3cdc4307-1bc3-4abf-9341-566870fe950e/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:33.314 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:33.314 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:33.314 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024414 s, 42.9 MB/s 2021-02-10T11:29:33.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:33.316 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:33.316 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:33.649 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:33.650 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:33.650 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839336/3cdc4307-1bc3-4abf-9341-566870fe950e/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:34.021 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:34.021 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:34.021 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244508 s, 42.9 MB/s 2021-02-10T11:29:34.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:34.022 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:34.022 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:34.356 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:34.357 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:34.357 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839336/3cdc4307-1bc3-4abf-9341-566870fe950e/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:34.730 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:34.730 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:34.730 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0230681 s, 45.5 MB/s 2021-02-10T11:29:34.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:34.731 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:34.731 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:35.065 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:35.065 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:35.065 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839336/3cdc4307-1bc3-4abf-9341-566870fe950e/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:35.441 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:35.441 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:35.441 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251811 s, 41.6 MB/s 2021-02-10T11:29:35.443 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839337 2021-02-10T11:29:35.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.844 7f2af9c08700 1 Processor -- start 2021-02-10T11:29:35.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.844 7f2af9c08700 1 -- start start 2021-02-10T11:29:35.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.845 7f2af9c08700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2af41b0870 0x7f2af40f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:35.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.845 7f2af9c08700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2af40efe00 0x7f2af40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:35.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.845 7f2af9c08700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af41b0c70 con 0x7f2af41b0470 2021-02-10T11:29:35.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.845 7f2af9c08700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af40e5ea0 con 0x7f2af41b0870 2021-02-10T11:29:35.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.845 7f2af9c08700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af40f0b50 con 0x7f2af40efe00 2021-02-10T11:29:35.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.845 7f2af2ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2af41b0870 0x7f2af40f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:35.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.845 7f2af2ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2af41b0870 0x7f2af40f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48378/0 (socket says 172.21.2.101:48378) 2021-02-10T11:29:35.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.845 7f2af2ffd700 1 -- 172.21.2.101:0/2778023990 learned_addr learned my addr 172.21.2.101:0/2778023990 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:29:35.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.845 7f2af2ffd700 1 -- 172.21.2.101:0/2778023990 >> v2:172.21.2.108:3300/0 conn(0x7f2af40efe00 msgr2=0x7f2af40f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:29:35.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.845 7f2af2ffd700 1 --2- 172.21.2.101:0/2778023990 >> v2:172.21.2.108:3300/0 conn(0x7f2af40efe00 0x7f2af40f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:35.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.845 7f2af2ffd700 1 -- 172.21.2.101:0/2778023990 >> v1:172.21.2.108:6789/0 conn(0x7f2af41b0470 legacy=0x7f2af40ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:29:35.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.845 7f2af2ffd700 1 -- 172.21.2.101:0/2778023990 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af40f0820 con 0x7f2af41b0870 2021-02-10T11:29:35.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.846 7f2af2ffd700 1 --2- 172.21.2.101:0/2778023990 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2af41b0870 0x7f2af40f20e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f2ae4006700 tx=0x7f2ae4009b00).ready entity=mon.0 client_cookie=5986b603d0160b96 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:35.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.846 7f2af27fc700 1 -- 172.21.2.101:0/2778023990 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2ae400c0c0 con 0x7f2af41b0870 2021-02-10T11:29:35.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.846 7f2af27fc700 1 -- 172.21.2.101:0/2778023990 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae400c340 con 0x7f2af41b0870 2021-02-10T11:29:35.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.846 7f2af27fc700 1 -- 172.21.2.101:0/2778023990 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2ae400c820 con 0x7f2af41b0870 2021-02-10T11:29:35.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.846 7f2af9c08700 1 -- 172.21.2.101:0/2778023990 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2af41b0870 msgr2=0x7f2af40f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:35.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.846 7f2af9c08700 1 --2- 172.21.2.101:0/2778023990 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2af41b0870 0x7f2af40f20e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f2ae4006700 tx=0x7f2ae4009b00).stop 2021-02-10T11:29:35.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.846 7f2af9c08700 1 -- 172.21.2.101:0/2778023990 shutdown_connections 2021-02-10T11:29:35.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.846 7f2af9c08700 1 --2- 172.21.2.101:0/2778023990 >> v2:172.21.2.108:3300/0 conn(0x7f2af40efe00 0x7f2af40f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:35.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.846 7f2af9c08700 1 --2- 172.21.2.101:0/2778023990 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2af41b0870 0x7f2af40f20e0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:35.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.846 7f2af9c08700 1 -- 172.21.2.101:0/2778023990 >> 172.21.2.101:0/2778023990 conn(0x7f2af40ce3e0 msgr2=0x7f2af40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:35.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.846 7f2af9c08700 1 -- 172.21.2.101:0/2778023990 shutdown_connections 2021-02-10T11:29:35.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.846 7f2af9c08700 1 -- 172.21.2.101:0/2778023990 wait complete. 2021-02-10T11:29:35.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.861 7f2af9c08700 1 Processor -- start 2021-02-10T11:29:35.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.861 7f2af9c08700 1 -- start start 2021-02-10T11:29:35.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.861 7f2af9c08700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2af40b2840 0x7f2af41a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:35.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.861 7f2af9c08700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2af40b30d0 0x7f2af40b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:35.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.861 7f2af9c08700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2af40b3af0 0x7f2af40f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:35.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.861 7f2af9c08700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af40b2c40 con 0x7f2af40b3af0 2021-02-10T11:29:35.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.861 7f2af9c08700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af41a4920 con 0x7f2af40b2840 2021-02-10T11:29:35.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.861 7f2af9c08700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af40f0ba0 con 0x7f2af40b30d0 2021-02-10T11:29:35.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.861 7f2af3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2af40b3af0 0x7f2af40f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:35.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.861 7f2af3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2af40b3af0 0x7f2af40f0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48386/0 (socket says 172.21.2.101:48386) 2021-02-10T11:29:35.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.861 7f2af3fff700 1 -- 172.21.2.101:0/3640847937 learned_addr learned my addr 172.21.2.101:0/3640847937 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:29:35.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.861 7f2af3fff700 1 -- 172.21.2.101:0/3640847937 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2af40b30d0 msgr2=0x7f2af40b34d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:29:35.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.861 7f2af3fff700 1 --2- 172.21.2.101:0/3640847937 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2af40b30d0 0x7f2af40b34d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:35.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.861 7f2af3fff700 1 -- 172.21.2.101:0/3640847937 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2af40b2840 msgr2=0x7f2af41a4300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:29:35.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.861 7f2af3fff700 1 --2- 172.21.2.101:0/3640847937 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2af40b2840 0x7f2af41a4300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:35.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.861 7f2af3fff700 1 -- 172.21.2.101:0/3640847937 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af40f0dc0 con 0x7f2af40b3af0 2021-02-10T11:29:35.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.862 7f2af3fff700 1 --2- 172.21.2.101:0/3640847937 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2af40b3af0 0x7f2af40f0580 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7f2aec009980 tx=0x7f2aec00bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:35.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.862 7f2aebfff700 1 -- 172.21.2.101:0/3640847937 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2aec005870 con 0x7f2af40b3af0 2021-02-10T11:29:35.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.862 7f2aebfff700 1 -- 172.21.2.101:0/3640847937 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aec005b30 con 0x7f2af40b3af0 2021-02-10T11:29:35.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.862 7f2aebfff700 1 -- 172.21.2.101:0/3640847937 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2aec006030 con 0x7f2af40b3af0 2021-02-10T11:29:35.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.862 7f2af9c08700 1 -- 172.21.2.101:0/3640847937 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af40f0dc0 con 0x7f2af40b3af0 2021-02-10T11:29:35.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.862 7f2af9c08700 1 -- 172.21.2.101:0/3640847937 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af40f0dc0 con 0x7f2af40b3af0 2021-02-10T11:29:35.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.862 7f2aebfff700 1 -- 172.21.2.101:0/3640847937 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2aec005d10 con 0x7f2af40b3af0 2021-02-10T11:29:35.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.862 7f2aebfff700 1 --2- 172.21.2.101:0/3640847937 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2ad8018730 0x7f2ad801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:35.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.862 7f2aebfff700 1 -- 172.21.2.101:0/3640847937 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2aec0329f0 con 0x7f2af40b3af0 2021-02-10T11:29:35.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.862 7f2af37fe700 1 --2- 172.21.2.101:0/3640847937 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2ad8018730 0x7f2ad801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:35.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.863 7f2af37fe700 1 --2- 172.21.2.101:0/3640847937 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2ad8018730 0x7f2ad801ac10 crc :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:35.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.895 7f2af9c08700 1 -- 172.21.2.101:0/3640847937 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af40f0050 con 0x7f2af40b3af0 2021-02-10T11:29:35.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:35.897 7f2aebfff700 1 -- 172.21.2.101:0/3640847937 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2aec00be10 con 0x7f2af40b3af0 2021-02-10T11:29:36.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.022 7f2af9c08700 1 -- 172.21.2.101:0/3640847937 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839337", "target": ["mgr", ""]}) v1 -- 0x7f2af40f0050 con 0x7f2ad8018730 2021-02-10T11:29:36.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.049 7f2aebfff700 1 -- 172.21.2.101:0/3640847937 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f2adc0069b0 con 0x7f2ad8018730 2021-02-10T11:29:36.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.055 7f2af9c08700 1 -- 172.21.2.101:0/3640847937 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2ad8018730 msgr2=0x7f2ad801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:36.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.055 7f2af9c08700 1 --2- 172.21.2.101:0/3640847937 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2ad8018730 0x7f2ad801ac10 crc :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:36.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.055 7f2af9c08700 1 -- 172.21.2.101:0/3640847937 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2af40b3af0 msgr2=0x7f2af40f0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:36.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.055 7f2af9c08700 1 --2- 172.21.2.101:0/3640847937 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2af40b3af0 0x7f2af40f0580 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7f2aec009980 tx=0x7f2aec00bcd0).stop 2021-02-10T11:29:36.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.055 7f2af9c08700 1 -- 172.21.2.101:0/3640847937 shutdown_connections 2021-02-10T11:29:36.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.055 7f2af9c08700 1 --2- 172.21.2.101:0/3640847937 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2ad8018730 0x7f2ad801ac10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:36.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.055 7f2af9c08700 1 --2- 172.21.2.101:0/3640847937 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2af40b30d0 0x7f2af40b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:36.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.055 7f2af9c08700 1 --2- 172.21.2.101:0/3640847937 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2af40b3af0 0x7f2af40f0580 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:36.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.055 7f2af9c08700 1 --2- 172.21.2.101:0/3640847937 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2af40b2840 0x7f2af41a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:36.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.055 7f2af9c08700 1 -- 172.21.2.101:0/3640847937 >> 172.21.2.101:0/3640847937 conn(0x7f2af40c36f0 msgr2=0x7f2af4094c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:36.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.056 7f2af9c08700 1 -- 172.21.2.101:0/3640847937 shutdown_connections 2021-02-10T11:29:36.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.056 7f2af9c08700 1 -- 172.21.2.101:0/3640847937 wait complete. 2021-02-10T11:29:36.074 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839337 2021-02-10T11:29:36.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.477 7f888b55c700 1 Processor -- start 2021-02-10T11:29:36.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.477 7f888b55c700 1 -- start start 2021-02-10T11:29:36.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.477 7f888b55c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f88841b0870 0x7f88840f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:36.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.477 7f888b55c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88841b0c70 con 0x7f88841b0470 2021-02-10T11:29:36.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.477 7f888b55c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88840f0b50 con 0x7f88840efe00 2021-02-10T11:29:36.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.477 7f888b55c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88840e5ea0 con 0x7f88841b0870 2021-02-10T11:29:36.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.477 7f8888b77700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f88841b0870 0x7f88840f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:36.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.477 7f8889378700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f88841b0470 0x7f88840ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48886/0 (socket says 172.21.2.101:48886) 2021-02-10T11:29:36.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.477 7f8889378700 1 -- 172.21.2.101:0/2383544938 learned_addr learned my addr 172.21.2.101:0/2383544938 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:29:36.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.477 7f8888b77700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f88841b0870 0x7f88840f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48392/0 (socket says 172.21.2.101:48392) 2021-02-10T11:29:36.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.478 7f8888b77700 1 -- 172.21.2.101:0/2383544938 >> v1:172.21.2.108:6789/0 conn(0x7f88840efe00 legacy=0x7f88840f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:29:36.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.478 7f8888b77700 1 -- 172.21.2.101:0/2383544938 >> v1:172.21.2.101:6789/0 conn(0x7f88841b0470 legacy=0x7f88840ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:36.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.478 7f8888b77700 1 -- 172.21.2.101:0/2383544938 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88840e52e0 con 0x7f88841b0870 2021-02-10T11:29:36.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.478 7f8888b77700 1 --2- 172.21.2.101:0/2383544938 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88841b0870 0x7f88840f20e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7f8874006650 tx=0x7f8874009a80).ready entity=mon.0 client_cookie=f0461509d109fb7d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:36.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.478 7f887bfff700 1 -- 172.21.2.101:0/2383544938 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f88740009b0 con 0x7f88841b0870 2021-02-10T11:29:36.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.478 7f887bfff700 1 -- 172.21.2.101:0/2383544938 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887400c0c0 con 0x7f88841b0870 2021-02-10T11:29:36.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.478 7f887bfff700 1 -- 172.21.2.101:0/2383544938 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f887400c5e0 con 0x7f88841b0870 2021-02-10T11:29:36.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.478 7f888b55c700 1 -- 172.21.2.101:0/2383544938 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88841b0870 msgr2=0x7f88840f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:36.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.478 7f888b55c700 1 --2- 172.21.2.101:0/2383544938 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88841b0870 0x7f88840f20e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7f8874006650 tx=0x7f8874009a80).stop 2021-02-10T11:29:36.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.479 7f888b55c700 1 -- 172.21.2.101:0/2383544938 shutdown_connections 2021-02-10T11:29:36.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.479 7f888b55c700 1 --2- 172.21.2.101:0/2383544938 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88841b0870 0x7f88840f20e0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:36.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.479 7f888b55c700 1 -- 172.21.2.101:0/2383544938 >> 172.21.2.101:0/2383544938 conn(0x7f88840ce3e0 msgr2=0x7f88840e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:36.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.479 7f888b55c700 1 -- 172.21.2.101:0/2383544938 shutdown_connections 2021-02-10T11:29:36.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.479 7f888b55c700 1 -- 172.21.2.101:0/2383544938 wait complete. 2021-02-10T11:29:36.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.493 7f888b55c700 1 Processor -- start 2021-02-10T11:29:36.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.493 7f888b55c700 1 -- start start 2021-02-10T11:29:36.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.494 7f888b55c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8884086cf0 0x7f8884095580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:36.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.494 7f888b55c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8884095ba0 0x7f88840a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:36.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.494 7f888b55c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88840870f0 0x7f88840a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:36.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.494 7f888b55c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8884095080 con 0x7f88840870f0 2021-02-10T11:29:36.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.494 7f888b55c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8884196ec0 con 0x7f8884086cf0 2021-02-10T11:29:36.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.494 7f888b55c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88840a3390 con 0x7f8884095ba0 2021-02-10T11:29:36.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.494 7f8889b79700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88840870f0 0x7f88840a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:36.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.494 7f8889b79700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88840870f0 0x7f88840a59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48400/0 (socket says 172.21.2.101:48400) 2021-02-10T11:29:36.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.494 7f8889b79700 1 -- 172.21.2.101:0/617996999 learned_addr learned my addr 172.21.2.101:0/617996999 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:29:36.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.494 7f8889b79700 1 -- 172.21.2.101:0/617996999 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8884095ba0 msgr2=0x7f88840a53b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:29:36.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.494 7f8889b79700 1 --2- 172.21.2.101:0/617996999 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8884095ba0 0x7f88840a53b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:36.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.494 7f8889b79700 1 -- 172.21.2.101:0/617996999 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8884086cf0 msgr2=0x7f8884095580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:29:36.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.494 7f8889b79700 1 --2- 172.21.2.101:0/617996999 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8884086cf0 0x7f8884095580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:36.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.494 7f8889b79700 1 -- 172.21.2.101:0/617996999 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88840a5ff0 con 0x7f88840870f0 2021-02-10T11:29:36.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.494 7f8889b79700 1 --2- 172.21.2.101:0/617996999 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88840870f0 0x7f88840a59d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7f887c009940 tx=0x7f887c00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:36.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.495 7f8879ffb700 1 -- 172.21.2.101:0/617996999 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f887c000f90 con 0x7f88840870f0 2021-02-10T11:29:36.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.495 7f8879ffb700 1 -- 172.21.2.101:0/617996999 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887c00c2f0 con 0x7f88840870f0 2021-02-10T11:29:36.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.495 7f888b55c700 1 -- 172.21.2.101:0/617996999 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88840a5ff0 con 0x7f88840870f0 2021-02-10T11:29:36.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.495 7f888b55c700 1 -- 172.21.2.101:0/617996999 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88840a6580 con 0x7f88840870f0 2021-02-10T11:29:36.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.495 7f8879ffb700 1 -- 172.21.2.101:0/617996999 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f887c00c7d0 con 0x7f88840870f0 2021-02-10T11:29:36.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.495 7f8879ffb700 1 -- 172.21.2.101:0/617996999 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f887c001220 con 0x7f88840870f0 2021-02-10T11:29:36.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.495 7f8879ffb700 1 --2- 172.21.2.101:0/617996999 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f88640186c0 0x7f886401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:36.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.495 7f8879ffb700 1 -- 172.21.2.101:0/617996999 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f887c032930 con 0x7f88840870f0 2021-02-10T11:29:36.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.495 7f8889378700 1 --2- 172.21.2.101:0/617996999 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f88640186c0 0x7f886401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:36.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.496 7f8889378700 1 --2- 172.21.2.101:0/617996999 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f88640186c0 0x7f886401abc0 crc :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:36.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.527 7f888b55c700 1 -- 172.21.2.101:0/617996999 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f888407e6b0 con 0x7f88840870f0 2021-02-10T11:29:36.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.530 7f8879ffb700 1 -- 172.21.2.101:0/617996999 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f887c000f40 con 0x7f88840870f0 2021-02-10T11:29:36.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.661 7f888b55c700 1 -- 172.21.2.101:0/617996999 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839337", "target": ["mgr", ""]}) v1 -- 0x7f8884196ec0 con 0x7f88640186c0 2021-02-10T11:29:36.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.664 7f8879ffb700 1 -- 172.21.2.101:0/617996999 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f8880000ed0 con 0x7f88640186c0 2021-02-10T11:29:36.675 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839337/c493175b-7770-4f6a-b6b0-0370fa6812cf 2021-02-10T11:29:36.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.665 7f888b55c700 1 -- 172.21.2.101:0/617996999 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f88640186c0 msgr2=0x7f886401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:36.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.665 7f888b55c700 1 --2- 172.21.2.101:0/617996999 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f88640186c0 0x7f886401abc0 crc :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:36.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.665 7f888b55c700 1 -- 172.21.2.101:0/617996999 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88840870f0 msgr2=0x7f88840a59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:36.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.665 7f888b55c700 1 --2- 172.21.2.101:0/617996999 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88840870f0 0x7f88840a59d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7f887c009940 tx=0x7f887c00bcf0).stop 2021-02-10T11:29:36.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.665 7f888b55c700 1 -- 172.21.2.101:0/617996999 shutdown_connections 2021-02-10T11:29:36.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.665 7f888b55c700 1 --2- 172.21.2.101:0/617996999 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f88640186c0 0x7f886401abc0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:36.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.665 7f888b55c700 1 --2- 172.21.2.101:0/617996999 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8884095ba0 0x7f88840a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:36.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.665 7f888b55c700 1 --2- 172.21.2.101:0/617996999 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88840870f0 0x7f88840a59d0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:36.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.665 7f888b55c700 1 --2- 172.21.2.101:0/617996999 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8884086cf0 0x7f8884095580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:36.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.665 7f888b55c700 1 -- 172.21.2.101:0/617996999 >> 172.21.2.101:0/617996999 conn(0x7f8884096170 msgr2=0x7f8884099640 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:36.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.665 7f888b55c700 1 -- 172.21.2.101:0/617996999 shutdown_connections 2021-02-10T11:29:36.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:36.666 7f888b55c700 1 -- 172.21.2.101:0/617996999 wait complete. 2021-02-10T11:29:36.684 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839337 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839337/c493175b-7770-4f6a-b6b0-0370fa6812cf 2021-02-10T11:29:36.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:36.684 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:36.685 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:37.019 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:37.020 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:37.020 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839337/c493175b-7770-4f6a-b6b0-0370fa6812cf/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:37.401 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:37.402 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:37.402 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246722 s, 42.5 MB/s 2021-02-10T11:29:37.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:37.403 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:37.403 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:37.736 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:37.736 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:37.737 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839337/c493175b-7770-4f6a-b6b0-0370fa6812cf/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:38.112 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:38.112 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:38.112 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249756 s, 42.0 MB/s 2021-02-10T11:29:38.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:38.113 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:38.113 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:38.446 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:38.447 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:38.447 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839337/c493175b-7770-4f6a-b6b0-0370fa6812cf/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:38.816 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:38.816 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:38.816 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0229239 s, 45.7 MB/s 2021-02-10T11:29:38.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:38.817 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:38.817 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:39.152 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:39.152 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:39.152 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839337/c493175b-7770-4f6a-b6b0-0370fa6812cf/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:39.525 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:39.525 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:39.525 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237199 s, 44.2 MB/s 2021-02-10T11:29:39.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:39.527 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:39.527 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:39.860 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:39.861 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:39.861 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839337/c493175b-7770-4f6a-b6b0-0370fa6812cf/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:40.237 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:40.237 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:40.237 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0255338 s, 41.1 MB/s 2021-02-10T11:29:40.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:40.238 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:40.238 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:40.573 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:40.574 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:40.574 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839337/c493175b-7770-4f6a-b6b0-0370fa6812cf/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:40.951 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:40.951 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:40.951 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0258677 s, 40.5 MB/s 2021-02-10T11:29:40.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:40.952 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:40.952 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:41.286 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:41.286 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:41.287 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839337/c493175b-7770-4f6a-b6b0-0370fa6812cf/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:41.662 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:41.662 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:41.662 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239186 s, 43.8 MB/s 2021-02-10T11:29:41.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:41.663 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:41.663 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:41.996 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:41.997 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:41.997 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839337/c493175b-7770-4f6a-b6b0-0370fa6812cf/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:42.370 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:42.370 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:42.370 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023546 s, 44.5 MB/s 2021-02-10T11:29:42.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:42.371 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:42.371 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:42.705 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:42.705 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:42.706 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839337/c493175b-7770-4f6a-b6b0-0370fa6812cf/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:43.082 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:43.083 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:43.083 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243512 s, 43.1 MB/s 2021-02-10T11:29:43.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:43.084 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:43.084 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:43.417 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:43.418 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:43.418 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839337/c493175b-7770-4f6a-b6b0-0370fa6812cf/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:43.793 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:43.793 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:43.794 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235413 s, 44.5 MB/s 2021-02-10T11:29:43.795 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000839338 2021-02-10T11:29:44.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.197 7f84330af700 1 Processor -- start 2021-02-10T11:29:44.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.197 7f84330af700 1 -- start start 2021-02-10T11:29:44.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.198 7f84330af700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f842c1b0870 0x7f842c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:44.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.198 7f84330af700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f842c0efe00 0x7f842c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:44.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.198 7f84330af700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f842c1b0c70 con 0x7f842c1b0470 2021-02-10T11:29:44.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.198 7f84330af700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f842c0f0b50 con 0x7f842c0efe00 2021-02-10T11:29:44.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.198 7f84330af700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842c0f0820 con 0x7f842c1b0870 2021-02-10T11:29:44.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.198 7f84316cc700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f842c0efe00 0x7f842c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:44.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.198 7f84316cc700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f842c0efe00 0x7f842c0f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48412/0 (socket says 172.21.2.101:48412) 2021-02-10T11:29:44.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.198 7f84316cc700 1 -- 172.21.2.101:0/394133533 learned_addr learned my addr 172.21.2.101:0/394133533 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:29:44.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.198 7f84316cc700 1 -- 172.21.2.101:0/394133533 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f842c1b0870 msgr2=0x7f842c0f20e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:29:44.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.198 7f84316cc700 1 --2- 172.21.2.101:0/394133533 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f842c1b0870 0x7f842c0f20e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:44.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.198 7f84316cc700 1 -- 172.21.2.101:0/394133533 >> v1:172.21.2.108:6789/0 conn(0x7f842c1b0470 legacy=0x7f842c0ef610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:29:44.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.198 7f84316cc700 1 -- 172.21.2.101:0/394133533 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c0e52e0 con 0x7f842c0efe00 2021-02-10T11:29:44.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.198 7f84316cc700 1 --2- 172.21.2.101:0/394133533 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f842c0efe00 0x7f842c0f0200 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f8424006f00 tx=0x7f8424009f10).ready entity=mon.0 client_cookie=18192e3f59f2373a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:44.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.199 7f842b7fe700 1 -- 172.21.2.101:0/394133533 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f842400b3b0 con 0x7f842c0efe00 2021-02-10T11:29:44.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.199 7f842b7fe700 1 -- 172.21.2.101:0/394133533 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842400b730 con 0x7f842c0efe00 2021-02-10T11:29:44.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.199 7f842b7fe700 1 -- 172.21.2.101:0/394133533 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f842400bb90 con 0x7f842c0efe00 2021-02-10T11:29:44.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.199 7f84330af700 1 -- 172.21.2.101:0/394133533 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f842c0efe00 msgr2=0x7f842c0f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:44.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.199 7f84330af700 1 --2- 172.21.2.101:0/394133533 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f842c0efe00 0x7f842c0f0200 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f8424006f00 tx=0x7f8424009f10).stop 2021-02-10T11:29:44.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.199 7f84330af700 1 -- 172.21.2.101:0/394133533 shutdown_connections 2021-02-10T11:29:44.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.199 7f84330af700 1 --2- 172.21.2.101:0/394133533 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f842c1b0870 0x7f842c0f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:44.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.199 7f84330af700 1 --2- 172.21.2.101:0/394133533 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f842c0efe00 0x7f842c0f0200 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:44.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.199 7f84330af700 1 -- 172.21.2.101:0/394133533 >> 172.21.2.101:0/394133533 conn(0x7f842c0ce3e0 msgr2=0x7f842c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:44.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.200 7f84330af700 1 -- 172.21.2.101:0/394133533 shutdown_connections 2021-02-10T11:29:44.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.200 7f84330af700 1 -- 172.21.2.101:0/394133533 wait complete. 2021-02-10T11:29:44.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.214 7f84330af700 1 Processor -- start 2021-02-10T11:29:44.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.214 7f84330af700 1 -- start start 2021-02-10T11:29:44.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.214 7f84330af700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f842c0afa70 0x7f842c0afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:44.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.214 7f84330af700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f842c0b0490 0x7f842c0a78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:44.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.214 7f84330af700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f842c0a7f10 0x7f842c0a8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:44.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.214 7f84330af700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c0a8a40 con 0x7f842c0a7f10 2021-02-10T11:29:44.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.214 7f84330af700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c0a8fc0 con 0x7f842c0afa70 2021-02-10T11:29:44.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.214 7f84330af700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842c0a91a0 con 0x7f842c0b0490 2021-02-10T11:29:44.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.215 7f84316cc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f842c0a7f10 0x7f842c0a8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:44.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.215 7f84316cc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f842c0a7f10 0x7f842c0a8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48418/0 (socket says 172.21.2.101:48418) 2021-02-10T11:29:44.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.215 7f84316cc700 1 -- 172.21.2.101:0/3461437097 learned_addr learned my addr 172.21.2.101:0/3461437097 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:29:44.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.215 7f8430ecb700 1 --2- 172.21.2.101:0/3461437097 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f842c0afa70 0x7f842c0afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:44.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.215 7f842bfff700 1 --2- 172.21.2.101:0/3461437097 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f842c0b0490 0x7f842c0a78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:44.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.215 7f84316cc700 1 -- 172.21.2.101:0/3461437097 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f842c0b0490 msgr2=0x7f842c0a78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:44.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.215 7f84316cc700 1 --2- 172.21.2.101:0/3461437097 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f842c0b0490 0x7f842c0a78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:44.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.215 7f84316cc700 1 -- 172.21.2.101:0/3461437097 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f842c0afa70 msgr2=0x7f842c0afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:44.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.215 7f84316cc700 1 --2- 172.21.2.101:0/3461437097 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f842c0afa70 0x7f842c0afe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:44.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.215 7f84316cc700 1 -- 172.21.2.101:0/3461437097 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c0a93c0 con 0x7f842c0a7f10 2021-02-10T11:29:44.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.215 7f84316cc700 1 --2- 172.21.2.101:0/3461437097 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f842c0a7f10 0x7f842c0a8310 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f8424009390 tx=0x7f842400bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:44.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.215 7f84297fa700 1 -- 172.21.2.101:0/3461437097 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8424009450 con 0x7f842c0a7f10 2021-02-10T11:29:44.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.215 7f84297fa700 1 -- 172.21.2.101:0/3461437097 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84240068e0 con 0x7f842c0a7f10 2021-02-10T11:29:44.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.215 7f84297fa700 1 -- 172.21.2.101:0/3461437097 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f842400c4e0 con 0x7f842c0a7f10 2021-02-10T11:29:44.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.215 7f84330af700 1 -- 172.21.2.101:0/3461437097 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f842c0a93c0 con 0x7f842c0a7f10 2021-02-10T11:29:44.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.215 7f84330af700 1 -- 172.21.2.101:0/3461437097 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842c0a9950 con 0x7f842c0a7f10 2021-02-10T11:29:44.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.216 7f84297fa700 1 -- 172.21.2.101:0/3461437097 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8424001270 con 0x7f842c0a7f10 2021-02-10T11:29:44.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.216 7f84297fa700 1 --2- 172.21.2.101:0/3461437097 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84100186c0 0x7f841001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:44.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.216 7f84297fa700 1 -- 172.21.2.101:0/3461437097 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8424032900 con 0x7f842c0a7f10 2021-02-10T11:29:44.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.216 7f8430ecb700 1 --2- 172.21.2.101:0/3461437097 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84100186c0 0x7f841001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:44.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.216 7f8430ecb700 1 --2- 172.21.2.101:0/3461437097 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84100186c0 0x7f841001abc0 crc :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:44.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.247 7f84330af700 1 -- 172.21.2.101:0/3461437097 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f842c0e52e0 con 0x7f842c0a7f10 2021-02-10T11:29:44.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.250 7f84297fa700 1 -- 172.21.2.101:0/3461437097 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8424020ed0 con 0x7f842c0a7f10 2021-02-10T11:29:44.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.375 7f84330af700 1 -- 172.21.2.101:0/3461437097 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839338", "target": ["mgr", ""]}) v1 -- 0x7f842c0e52e0 con 0x7f84100186c0 2021-02-10T11:29:44.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.400 7f84297fa700 1 -- 172.21.2.101:0/3461437097 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f84140009b0 con 0x7f84100186c0 2021-02-10T11:29:44.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.408 7f84330af700 1 -- 172.21.2.101:0/3461437097 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84100186c0 msgr2=0x7f841001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:44.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.408 7f84330af700 1 --2- 172.21.2.101:0/3461437097 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84100186c0 0x7f841001abc0 crc :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:44.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.408 7f84330af700 1 -- 172.21.2.101:0/3461437097 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f842c0a7f10 msgr2=0x7f842c0a8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:44.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.408 7f84330af700 1 --2- 172.21.2.101:0/3461437097 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f842c0a7f10 0x7f842c0a8310 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f8424009390 tx=0x7f842400bd40).stop 2021-02-10T11:29:44.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.409 7f84330af700 1 -- 172.21.2.101:0/3461437097 shutdown_connections 2021-02-10T11:29:44.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.409 7f84330af700 1 --2- 172.21.2.101:0/3461437097 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84100186c0 0x7f841001abc0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:44.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.409 7f84330af700 1 --2- 172.21.2.101:0/3461437097 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f842c0b0490 0x7f842c0a78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:44.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.409 7f84330af700 1 --2- 172.21.2.101:0/3461437097 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f842c0a7f10 0x7f842c0a8310 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:44.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.409 7f84330af700 1 --2- 172.21.2.101:0/3461437097 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f842c0afa70 0x7f842c0afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:44.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.409 7f84330af700 1 -- 172.21.2.101:0/3461437097 >> 172.21.2.101:0/3461437097 conn(0x7f842c0ac550 msgr2=0x7f842c0acb70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:44.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.409 7f84330af700 1 -- 172.21.2.101:0/3461437097 shutdown_connections 2021-02-10T11:29:44.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.409 7f84330af700 1 -- 172.21.2.101:0/3461437097 wait complete. 2021-02-10T11:29:44.428 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000839338 2021-02-10T11:29:44.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.828 7fb9ea06a700 1 Processor -- start 2021-02-10T11:29:44.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.828 7fb9ea06a700 1 -- start start 2021-02-10T11:29:44.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.828 7fb9ea06a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9e41b0470 0x7fb9e41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:44.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.828 7fb9ea06a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb9e40ef610 0x7fb9e40efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:44.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.828 7fb9ea06a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e40f0b50 con 0x7fb9e40f0030 2021-02-10T11:29:44.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.828 7fb9ea06a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e40e52e0 con 0x7fb9e40ef610 2021-02-10T11:29:44.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.828 7fb9ea06a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e40e5490 con 0x7fb9e41b0470 2021-02-10T11:29:44.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.828 7fb9e3fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb9e40f0030 0x7fb9e40f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48920/0 (socket says 172.21.2.101:48920) 2021-02-10T11:29:44.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.828 7fb9e3fff700 1 -- 172.21.2.101:0/442158700 learned_addr learned my addr 172.21.2.101:0/442158700 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:29:44.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.829 7fb9e37fe700 1 --2- 172.21.2.101:0/442158700 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9e41b0470 0x7fb9e41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:44.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.829 7fb9e27fc700 1 -- 172.21.2.101:0/442158700 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247164023 0 0) 0x7fb9dc001500 con 0x7fb9e40f0030 2021-02-10T11:29:44.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.829 7fb9e27fc700 1 -- 172.21.2.101:0/442158700 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9d0002fe0 con 0x7fb9e40f0030 2021-02-10T11:29:44.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.829 7fb9e2ffd700 1 --2- 172.21.2.101:0/442158700 >> v2:172.21.2.108:3300/0 conn(0x7fb9e40ef610 0x7fb9e40efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:44.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.829 7fb9e27fc700 1 -- 172.21.2.101:0/442158700 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1201252405 0 0) 0x7fb9dc001460 con 0x7fb9e40f0030 2021-02-10T11:29:44.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.829 7fb9e27fc700 1 -- 172.21.2.101:0/442158700 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9e41b0470 msgr2=0x7fb9e41b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:29:44.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.829 7fb9e27fc700 1 --2- 172.21.2.101:0/442158700 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9e41b0470 0x7fb9e41b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:44.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.829 7fb9e27fc700 1 -- 172.21.2.101:0/442158700 >> v2:172.21.2.108:3300/0 conn(0x7fb9e40ef610 msgr2=0x7fb9e40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:29:44.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.829 7fb9e27fc700 1 --2- 172.21.2.101:0/442158700 >> v2:172.21.2.108:3300/0 conn(0x7fb9e40ef610 0x7fb9e40efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:44.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.829 7fb9e27fc700 1 -- 172.21.2.101:0/442158700 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e40f0430 con 0x7fb9e40f0030 2021-02-10T11:29:44.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.829 7fb9e27fc700 1 -- 172.21.2.101:0/442158700 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fb9dc001980 con 0x7fb9e40f0030 2021-02-10T11:29:44.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.829 7fb9e27fc700 1 -- 172.21.2.101:0/442158700 >> v1:172.21.2.101:6789/0 conn(0x7fb9e40f0030 legacy=0x7fb9e40f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:44.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.829 7fb9e27fc700 1 --2- 172.21.2.101:0/442158700 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9d0003a40 0x7fb9d0005ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:44.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.829 7fb9e27fc700 1 --2- 172.21.2.101:0/442158700 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9d004a780 0x7fb9d004cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:44.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.829 7fb9e27fc700 1 --2- 172.21.2.101:0/442158700 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9d004d5a0 0x7fb9d004fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:44.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.829 7fb9e27fc700 1 -- 172.21.2.101:0/442158700 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d00502e0 con 0x7fb9d004a780 2021-02-10T11:29:44.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.829 7fb9e27fc700 1 -- 172.21.2.101:0/442158700 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d0050500 con 0x7fb9d0003a40 2021-02-10T11:29:44.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.829 7fb9e27fc700 1 -- 172.21.2.101:0/442158700 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d0050720 con 0x7fb9d004d5a0 2021-02-10T11:29:44.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.829 7fb9e2ffd700 1 --2- 172.21.2.101:0/442158700 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9d004a780 0x7fb9d004cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:44.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.830 7fb9e3fff700 1 --2- 172.21.2.101:0/442158700 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9d004d5a0 0x7fb9d004fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:44.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.830 7fb9e37fe700 1 --2- 172.21.2.101:0/442158700 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9d0003a40 0x7fb9d0005ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:44.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.830 7fb9e2ffd700 1 -- 172.21.2.101:0/442158700 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9d004d5a0 msgr2=0x7fb9d004fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:29:44.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.830 7fb9e2ffd700 1 --2- 172.21.2.101:0/442158700 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9d004d5a0 0x7fb9d004fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:44.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.830 7fb9e2ffd700 1 -- 172.21.2.101:0/442158700 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9d0003a40 msgr2=0x7fb9d0005ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:29:44.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.830 7fb9e2ffd700 1 --2- 172.21.2.101:0/442158700 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9d0003a40 0x7fb9d0005ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:44.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.830 7fb9e2ffd700 1 -- 172.21.2.101:0/442158700 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d00509c0 con 0x7fb9d004a780 2021-02-10T11:29:44.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.830 7fb9e2ffd700 1 --2- 172.21.2.101:0/442158700 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9d004a780 0x7fb9d004cca0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7fb9cc00c8f0 tx=0x7fb9cc00cc90).ready entity=mon.0 client_cookie=12a3d03bb6e93a1e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:44.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.830 7fb9e27fc700 1 -- 172.21.2.101:0/442158700 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb9cc00cdd0 con 0x7fb9d004a780 2021-02-10T11:29:44.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.830 7fb9e27fc700 1 -- 172.21.2.101:0/442158700 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9cc009310 con 0x7fb9d004a780 2021-02-10T11:29:44.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.830 7fb9e27fc700 1 -- 172.21.2.101:0/442158700 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb9cc0097f0 con 0x7fb9d004a780 2021-02-10T11:29:44.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.830 7fb9ea06a700 1 -- 172.21.2.101:0/442158700 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9d004a780 msgr2=0x7fb9d004cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:44.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.830 7fb9ea06a700 1 --2- 172.21.2.101:0/442158700 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9d004a780 0x7fb9d004cca0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7fb9cc00c8f0 tx=0x7fb9cc00cc90).stop 2021-02-10T11:29:44.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.830 7fb9e3fff700 1 -- 172.21.2.101:0/442158700 reap_dead start 2021-02-10T11:29:44.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.830 7fb9ea06a700 1 -- 172.21.2.101:0/442158700 shutdown_connections 2021-02-10T11:29:44.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.830 7fb9ea06a700 1 -- 172.21.2.101:0/442158700 >> 172.21.2.101:0/442158700 conn(0x7fb9e40ce3e0 msgr2=0x7fb9e40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:44.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.830 7fb9ea06a700 1 -- 172.21.2.101:0/442158700 shutdown_connections 2021-02-10T11:29:44.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.830 7fb9ea06a700 1 -- 172.21.2.101:0/442158700 wait complete. 2021-02-10T11:29:44.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.845 7fb9ea06a700 1 Processor -- start 2021-02-10T11:29:44.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.845 7fb9ea06a700 1 -- start start 2021-02-10T11:29:44.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.845 7fb9ea06a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9e409e560 0x7fb9e40a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:44.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.845 7fb9ea06a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9e409e960 0x7fb9e40a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:44.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.845 7fb9ea06a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9e40a0d70 0x7fb9e40a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:44.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.845 7fb9ea06a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e40c38f0 con 0x7fb9e40a0d70 2021-02-10T11:29:44.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.845 7fb9ea06a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e40a1170 con 0x7fb9e409e960 2021-02-10T11:29:44.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.845 7fb9ea06a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e40a1390 con 0x7fb9e409e560 2021-02-10T11:29:44.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.845 7fb9e3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9e40a0d70 0x7fb9e40a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:44.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.845 7fb9e3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9e40a0d70 0x7fb9e40a1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48436/0 (socket says 172.21.2.101:48436) 2021-02-10T11:29:44.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.845 7fb9e3fff700 1 -- 172.21.2.101:0/2473832130 learned_addr learned my addr 172.21.2.101:0/2473832130 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:29:44.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.846 7fb9e37fe700 1 --2- 172.21.2.101:0/2473832130 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9e409e560 0x7fb9e40a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:44.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.846 7fb9e3fff700 1 -- 172.21.2.101:0/2473832130 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9e409e560 msgr2=0x7fb9e40a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:44.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.846 7fb9e3fff700 1 --2- 172.21.2.101:0/2473832130 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9e409e560 0x7fb9e40a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:44.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.846 7fb9e3fff700 1 -- 172.21.2.101:0/2473832130 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9e409e960 msgr2=0x7fb9e40a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:44.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.846 7fb9e3fff700 1 --2- 172.21.2.101:0/2473832130 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9e409e960 0x7fb9e40a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:44.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.846 7fb9e3fff700 1 -- 172.21.2.101:0/2473832130 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e40a2110 con 0x7fb9e40a0d70 2021-02-10T11:29:44.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.846 7fb9e37fe700 1 --2- 172.21.2.101:0/2473832130 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9e409e560 0x7fb9e40a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:29:44.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.846 7fb9e3fff700 1 --2- 172.21.2.101:0/2473832130 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9e40a0d70 0x7fb9e40a1af0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7fb9dc00a8f0 tx=0x7fb9dc00ac90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:44.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.846 7fb9dbfff700 1 -- 172.21.2.101:0/2473832130 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb9dc001e70 con 0x7fb9e40a0d70 2021-02-10T11:29:44.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.846 7fb9dbfff700 1 -- 172.21.2.101:0/2473832130 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9dc004930 con 0x7fb9e40a0d70 2021-02-10T11:29:44.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.846 7fb9ea06a700 1 -- 172.21.2.101:0/2473832130 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e40a2110 con 0x7fb9e40a0d70 2021-02-10T11:29:44.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.846 7fb9ea06a700 1 -- 172.21.2.101:0/2473832130 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e40a26a0 con 0x7fb9e40a0d70 2021-02-10T11:29:44.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.846 7fb9dbfff700 1 -- 172.21.2.101:0/2473832130 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb9dc007020 con 0x7fb9e40a0d70 2021-02-10T11:29:44.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.847 7fb9dbfff700 1 -- 172.21.2.101:0/2473832130 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb9dc001d00 con 0x7fb9e40a0d70 2021-02-10T11:29:44.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.847 7fb9dbfff700 1 --2- 172.21.2.101:0/2473832130 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9c80186c0 0x7fb9c801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:29:44.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.847 7fb9dbfff700 1 -- 172.21.2.101:0/2473832130 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb9dc032b40 con 0x7fb9e40a0d70 2021-02-10T11:29:44.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.847 7fb9e37fe700 1 --2- 172.21.2.101:0/2473832130 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9c80186c0 0x7fb9c801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:29:44.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.847 7fb9e37fe700 1 --2- 172.21.2.101:0/2473832130 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9c80186c0 0x7fb9c801abc0 crc :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:29:44.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.878 7fb9ea06a700 1 -- 172.21.2.101:0/2473832130 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e4105b30 con 0x7fb9e40a0d70 2021-02-10T11:29:44.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:44.881 7fb9dbfff700 1 -- 172.21.2.101:0/2473832130 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb9dc004930 con 0x7fb9e40a0d70 2021-02-10T11:29:45.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:45.010 7fb9ea06a700 1 -- 172.21.2.101:0/2473832130 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839338", "target": ["mgr", ""]}) v1 -- 0x7fb9e40a26a0 con 0x7fb9c80186c0 2021-02-10T11:29:45.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:45.013 7fb9dbfff700 1 -- 172.21.2.101:0/2473832130 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7fb9d4006b10 con 0x7fb9c80186c0 2021-02-10T11:29:45.024 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000839338/5f09834b-d00c-4ce7-b65f-6a692ef6d011 2021-02-10T11:29:45.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:45.014 7fb9ea06a700 1 -- 172.21.2.101:0/2473832130 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9c80186c0 msgr2=0x7fb9c801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:45.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:45.014 7fb9ea06a700 1 --2- 172.21.2.101:0/2473832130 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9c80186c0 0x7fb9c801abc0 crc :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:45.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:45.014 7fb9ea06a700 1 -- 172.21.2.101:0/2473832130 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9e40a0d70 msgr2=0x7fb9e40a1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:29:45.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:45.014 7fb9ea06a700 1 --2- 172.21.2.101:0/2473832130 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9e40a0d70 0x7fb9e40a1af0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7fb9dc00a8f0 tx=0x7fb9dc00ac90).stop 2021-02-10T11:29:45.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:45.015 7fb9ea06a700 1 -- 172.21.2.101:0/2473832130 shutdown_connections 2021-02-10T11:29:45.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:45.015 7fb9ea06a700 1 --2- 172.21.2.101:0/2473832130 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9e40a0d70 0x7fb9e40a1af0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:45.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:45.015 7fb9ea06a700 1 --2- 172.21.2.101:0/2473832130 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9e409e960 0x7fb9e40a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:29:45.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:45.015 7fb9ea06a700 1 --2- 172.21.2.101:0/2473832130 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9c80186c0 0x7fb9c801abc0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:45.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:45.015 7fb9ea06a700 1 --2- 172.21.2.101:0/2473832130 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9e409e560 0x7fb9e40a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:29:45.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:45.015 7fb9ea06a700 1 -- 172.21.2.101:0/2473832130 >> 172.21.2.101:0/2473832130 conn(0x7fb9e409fc10 msgr2=0x7fb9e4094ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:29:45.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:45.015 7fb9ea06a700 1 -- 172.21.2.101:0/2473832130 shutdown_connections 2021-02-10T11:29:45.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:29:45.015 7fb9ea06a700 1 -- 172.21.2.101:0/2473832130 wait complete. 2021-02-10T11:29:45.033 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000839338 with 10 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000839338/5f09834b-d00c-4ce7-b65f-6a692ef6d011 2021-02-10T11:29:45.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:45.034 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:45.034 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:45.368 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:45.369 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:45.369 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839338/5f09834b-d00c-4ce7-b65f-6a692ef6d011/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:45.751 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:45.751 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:45.752 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246265 s, 42.6 MB/s 2021-02-10T11:29:45.752 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:45.753 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:45.753 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:46.087 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:46.088 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:46.088 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839338/5f09834b-d00c-4ce7-b65f-6a692ef6d011/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:46.463 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:46.463 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:46.463 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238507 s, 44.0 MB/s 2021-02-10T11:29:46.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:46.464 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:46.464 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:46.798 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:46.799 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:46.799 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839338/5f09834b-d00c-4ce7-b65f-6a692ef6d011/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:47.175 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:47.497 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:47.497 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249586 s, 42.0 MB/s 2021-02-10T11:29:47.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:47.499 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:47.499 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:47.836 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:47.836 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:47.836 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839338/5f09834b-d00c-4ce7-b65f-6a692ef6d011/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:48.211 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:48.211 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:48.211 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252216 s, 41.6 MB/s 2021-02-10T11:29:48.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:48.213 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:48.213 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:48.546 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:48.547 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:48.547 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839338/5f09834b-d00c-4ce7-b65f-6a692ef6d011/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:48.916 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:48.916 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:48.916 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235154 s, 44.6 MB/s 2021-02-10T11:29:48.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:48.917 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:48.917 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:49.252 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:49.252 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:49.252 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839338/5f09834b-d00c-4ce7-b65f-6a692ef6d011/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:49.622 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:49.622 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:49.623 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234881 s, 44.6 MB/s 2021-02-10T11:29:49.623 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:49.624 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:49.624 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:49.958 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:49.959 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:49.959 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839338/5f09834b-d00c-4ce7-b65f-6a692ef6d011/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:50.333 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:50.333 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:50.333 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231237 s, 45.3 MB/s 2021-02-10T11:29:50.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:50.334 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:50.334 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:50.669 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:50.670 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:50.670 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839338/5f09834b-d00c-4ce7-b65f-6a692ef6d011/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:51.049 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:51.049 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:51.049 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0259236 s, 40.4 MB/s 2021-02-10T11:29:51.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:51.050 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:51.050 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:51.384 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:51.385 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:51.385 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839338/5f09834b-d00c-4ce7-b65f-6a692ef6d011/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:51.759 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:51.759 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:51.759 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.02323 s, 45.1 MB/s 2021-02-10T11:29:51.760 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:51.760 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:51.760 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:52.095 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:52.096 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:52.096 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000839338/5f09834b-d00c-4ce7-b65f-6a692ef6d011/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:29:52.476 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:29:52.477 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:29:52.477 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0270717 s, 38.7 MB/s 2021-02-10T11:29:52.478 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:52.478 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:52.478 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:52.842 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:29:52.842 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:29:52.842 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:29:52.842 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:29:52.842 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:53.256 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[34129]: fuse finished with error 0 and tester_r 0 2021-02-10T11:29:53.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:29:53.258 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:29:53.258 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:29:53.603 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:29:53.603 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:29:53.603 INFO:teuthology.orchestra.run:waiting for 900 2021-02-10T11:29:55.691 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:29:55.694 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:29:59.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:29:59.705 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:30:00.046 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839239 2021-02-10T11:30:00.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.448 7fea5538c700 1 Processor -- start 2021-02-10T11:30:00.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.448 7fea5538c700 1 -- start start 2021-02-10T11:30:00.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.448 7fea5538c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fea501b0470 0x7fea501b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:00.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.448 7fea5538c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea500ef610 0x7fea500efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:00.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.448 7fea5538c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fea500f0030 0x7fea500f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:00.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.448 7fea5538c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea500e5ea0 con 0x7fea500f0030 2021-02-10T11:30:00.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.448 7fea5538c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea500f0b50 con 0x7fea501b0470 2021-02-10T11:30:00.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.448 7fea5538c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea500e52e0 con 0x7fea500ef610 2021-02-10T11:30:00.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.448 7fea4f87e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fea500f0030 0x7fea500f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:00.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.449 7fea4f87e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fea500f0030 0x7fea500f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48442/0 (socket says 172.21.2.101:48442) 2021-02-10T11:30:00.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.449 7fea4f87e700 1 -- 172.21.2.101:0/2062326820 learned_addr learned my addr 172.21.2.101:0/2062326820 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:00.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.449 7fea4f87e700 1 -- 172.21.2.101:0/2062326820 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea500ef610 msgr2=0x7fea500efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:30:00.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.449 7fea4f87e700 1 --2- 172.21.2.101:0/2062326820 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea500ef610 0x7fea500efa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:00.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.449 7fea4f87e700 1 -- 172.21.2.101:0/2062326820 >> v2:172.21.2.108:3300/0 conn(0x7fea501b0470 msgr2=0x7fea501b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:30:00.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.449 7fea4f87e700 1 --2- 172.21.2.101:0/2062326820 >> v2:172.21.2.108:3300/0 conn(0x7fea501b0470 0x7fea501b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:00.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.449 7fea4f87e700 1 -- 172.21.2.101:0/2062326820 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea500f10d0 con 0x7fea500f0030 2021-02-10T11:30:00.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.449 7fea4f87e700 1 --2- 172.21.2.101:0/2062326820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea500f0030 0x7fea500f0430 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7fea48006f00 tx=0x7fea48009f10).ready entity=mon.0 client_cookie=7fa6d8b64721d370 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:00.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.449 7fea4e07b700 1 -- 172.21.2.101:0/2062326820 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fea4800b3b0 con 0x7fea500f0030 2021-02-10T11:30:00.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.449 7fea4e07b700 1 -- 172.21.2.101:0/2062326820 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4800b630 con 0x7fea500f0030 2021-02-10T11:30:00.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.449 7fea4e07b700 1 -- 172.21.2.101:0/2062326820 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fea4800bb50 con 0x7fea500f0030 2021-02-10T11:30:00.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.449 7fea5538c700 1 -- 172.21.2.101:0/2062326820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea500f0030 msgr2=0x7fea500f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:00.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.449 7fea5538c700 1 --2- 172.21.2.101:0/2062326820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea500f0030 0x7fea500f0430 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7fea48006f00 tx=0x7fea48009f10).stop 2021-02-10T11:30:00.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.450 7fea5538c700 1 -- 172.21.2.101:0/2062326820 shutdown_connections 2021-02-10T11:30:00.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.450 7fea5538c700 1 --2- 172.21.2.101:0/2062326820 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea500ef610 0x7fea500efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:00.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.450 7fea5538c700 1 --2- 172.21.2.101:0/2062326820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea500f0030 0x7fea500f0430 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:00.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.450 7fea5538c700 1 --2- 172.21.2.101:0/2062326820 >> v2:172.21.2.108:3300/0 conn(0x7fea501b0470 0x7fea501b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:00.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.450 7fea5538c700 1 -- 172.21.2.101:0/2062326820 >> 172.21.2.101:0/2062326820 conn(0x7fea500ce3e0 msgr2=0x7fea500e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:00.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.450 7fea5538c700 1 -- 172.21.2.101:0/2062326820 shutdown_connections 2021-02-10T11:30:00.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.450 7fea5538c700 1 -- 172.21.2.101:0/2062326820 wait complete. 2021-02-10T11:30:00.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.464 7fea5538c700 1 Processor -- start 2021-02-10T11:30:00.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.464 7fea5538c700 1 -- start start 2021-02-10T11:30:00.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.465 7fea5538c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea500bf910 0x7fea500bfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:00.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.465 7fea5538c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea500c0330 0x7fea500c0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:00.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.465 7fea5538c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea500c13d0 0x7fea500c17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:00.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.465 7fea5538c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea50109930 con 0x7fea500c13d0 2021-02-10T11:30:00.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.465 7fea5538c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea50109b50 con 0x7fea500c0330 2021-02-10T11:30:00.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.465 7fea5538c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea500c1df0 con 0x7fea500bf910 2021-02-10T11:30:00.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.465 7fea4f87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea500c13d0 0x7fea500c17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:00.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.465 7fea4f87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea500c13d0 0x7fea500c17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48448/0 (socket says 172.21.2.101:48448) 2021-02-10T11:30:00.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.465 7fea4f87e700 1 -- 172.21.2.101:0/1640519373 learned_addr learned my addr 172.21.2.101:0/1640519373 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:00.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.465 7fea4f87e700 1 -- 172.21.2.101:0/1640519373 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea500bf910 msgr2=0x7fea500bfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:00.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.465 7fea4f87e700 1 --2- 172.21.2.101:0/1640519373 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea500bf910 0x7fea500bfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:00.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.465 7fea4f87e700 1 -- 172.21.2.101:0/1640519373 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea500c0330 msgr2=0x7fea500c0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:00.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.465 7fea4f87e700 1 --2- 172.21.2.101:0/1640519373 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea500c0330 0x7fea500c0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:00.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.465 7fea4f87e700 1 -- 172.21.2.101:0/1640519373 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea500c1fd0 con 0x7fea500c13d0 2021-02-10T11:30:00.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.466 7fea4f87e700 1 --2- 172.21.2.101:0/1640519373 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea500c13d0 0x7fea500c17d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7fea4800b9b0 tx=0x7fea4800bd50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:00.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.466 7fea47fff700 1 -- 172.21.2.101:0/1640519373 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fea48003030 con 0x7fea500c13d0 2021-02-10T11:30:00.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.466 7fea47fff700 1 -- 172.21.2.101:0/1640519373 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4800c2f0 con 0x7fea500c13d0 2021-02-10T11:30:00.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.466 7fea47fff700 1 -- 172.21.2.101:0/1640519373 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fea4800c810 con 0x7fea500c13d0 2021-02-10T11:30:00.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.466 7fea5538c700 1 -- 172.21.2.101:0/1640519373 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea500c1fd0 con 0x7fea500c13d0 2021-02-10T11:30:00.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.466 7fea5538c700 1 -- 172.21.2.101:0/1640519373 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea500c2560 con 0x7fea500c13d0 2021-02-10T11:30:00.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.467 7fea47fff700 1 -- 172.21.2.101:0/1640519373 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fea48001270 con 0x7fea500c13d0 2021-02-10T11:30:00.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.467 7fea47fff700 1 --2- 172.21.2.101:0/1640519373 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea340186c0 0x7fea3401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:00.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.467 7fea47fff700 1 -- 172.21.2.101:0/1640519373 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fea480328f0 con 0x7fea500c13d0 2021-02-10T11:30:00.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.467 7fea4f07d700 1 --2- 172.21.2.101:0/1640519373 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea340186c0 0x7fea3401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:00.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.467 7fea4f07d700 1 --2- 172.21.2.101:0/1640519373 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea340186c0 0x7fea3401abc0 crc :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:00.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.498 7fea5538c700 1 -- 172.21.2.101:0/1640519373 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea500f0b50 con 0x7fea500c13d0 2021-02-10T11:30:00.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.501 7fea47fff700 1 -- 172.21.2.101:0/1640519373 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fea48009450 con 0x7fea500c13d0 2021-02-10T11:30:00.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.627 7fea5538c700 1 -- 172.21.2.101:0/1640519373 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839239", "target": ["mgr", ""]}) v1 -- 0x7fea500c2560 con 0x7fea340186c0 2021-02-10T11:30:00.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.639 7fea47fff700 1 -- 172.21.2.101:0/1640519373 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fea40006e40 con 0x7fea340186c0 2021-02-10T11:30:00.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.644 7fea5538c700 1 -- 172.21.2.101:0/1640519373 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea340186c0 msgr2=0x7fea3401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:00.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.644 7fea5538c700 1 --2- 172.21.2.101:0/1640519373 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea340186c0 0x7fea3401abc0 crc :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:00.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.644 7fea5538c700 1 -- 172.21.2.101:0/1640519373 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea500c13d0 msgr2=0x7fea500c17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:00.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.644 7fea5538c700 1 --2- 172.21.2.101:0/1640519373 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea500c13d0 0x7fea500c17d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7fea4800b9b0 tx=0x7fea4800bd50).stop 2021-02-10T11:30:00.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.644 7fea5538c700 1 -- 172.21.2.101:0/1640519373 shutdown_connections 2021-02-10T11:30:00.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.644 7fea5538c700 1 --2- 172.21.2.101:0/1640519373 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea500c13d0 0x7fea500c17d0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:00.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.644 7fea5538c700 1 --2- 172.21.2.101:0/1640519373 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea500c0330 0x7fea500c0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:00.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.644 7fea5538c700 1 --2- 172.21.2.101:0/1640519373 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea340186c0 0x7fea3401abc0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:00.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.644 7fea5538c700 1 --2- 172.21.2.101:0/1640519373 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea500bf910 0x7fea500bfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:00.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.644 7fea5538c700 1 -- 172.21.2.101:0/1640519373 >> 172.21.2.101:0/1640519373 conn(0x7fea500bf4e0 msgr2=0x7fea50191190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:00.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.645 7fea5538c700 1 -- 172.21.2.101:0/1640519373 shutdown_connections 2021-02-10T11:30:00.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:00.645 7fea5538c700 1 -- 172.21.2.101:0/1640519373 wait complete. 2021-02-10T11:30:00.664 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839240 2021-02-10T11:30:01.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.067 7ff1e56be700 1 Processor -- start 2021-02-10T11:30:01.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.067 7ff1e56be700 1 -- start start 2021-02-10T11:30:01.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.068 7ff1e56be700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff1e01b0470 0x7ff1e01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:01.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.068 7ff1e56be700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e00f0b50 con 0x7ff1e00ef610 2021-02-10T11:30:01.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.068 7ff1e56be700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e00f24e0 con 0x7ff1e00f20e0 2021-02-10T11:30:01.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.068 7ff1e56be700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e00e5ea0 con 0x7ff1e01b0470 2021-02-10T11:30:01.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.068 7ff1df2bf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff1e01b0470 0x7ff1e01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:01.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.068 7ff1deabe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff1e00ef610 0x7ff1e00efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48946/0 (socket says 172.21.2.101:48946) 2021-02-10T11:30:01.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.068 7ff1deabe700 1 -- 172.21.2.101:0/1859534056 learned_addr learned my addr 172.21.2.101:0/1859534056 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:01.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.069 7ff1df2bf700 1 -- 172.21.2.101:0/1859534056 >> v1:172.21.2.108:6789/0 conn(0x7ff1e00f20e0 legacy=0x7ff1e00f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:01.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.069 7ff1df2bf700 1 -- 172.21.2.101:0/1859534056 >> v1:172.21.2.101:6789/0 conn(0x7ff1e00ef610 legacy=0x7ff1e00efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:01.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.069 7ff1df2bf700 1 -- 172.21.2.101:0/1859534056 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e00e52e0 con 0x7ff1e01b0470 2021-02-10T11:30:01.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.069 7ff1df2bf700 1 --2- 172.21.2.101:0/1859534056 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1e01b0470 0x7ff1e01b0870 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7ff1c8006650 tx=0x7ff1c8009a80).ready entity=mon.0 client_cookie=b465b913331f405f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:01.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.069 7ff1de2bd700 1 -- 172.21.2.101:0/1859534056 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1c80009b0 con 0x7ff1e01b0470 2021-02-10T11:30:01.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.069 7ff1de2bd700 1 -- 172.21.2.101:0/1859534056 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c800c0c0 con 0x7ff1e01b0470 2021-02-10T11:30:01.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.069 7ff1de2bd700 1 -- 172.21.2.101:0/1859534056 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1c800c5e0 con 0x7ff1e01b0470 2021-02-10T11:30:01.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.069 7ff1e56be700 1 -- 172.21.2.101:0/1859534056 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1e01b0470 msgr2=0x7ff1e01b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:01.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.069 7ff1e56be700 1 --2- 172.21.2.101:0/1859534056 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1e01b0470 0x7ff1e01b0870 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7ff1c8006650 tx=0x7ff1c8009a80).stop 2021-02-10T11:30:01.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.070 7ff1e56be700 1 -- 172.21.2.101:0/1859534056 shutdown_connections 2021-02-10T11:30:01.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.070 7ff1e56be700 1 --2- 172.21.2.101:0/1859534056 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1e01b0470 0x7ff1e01b0870 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:01.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.070 7ff1e56be700 1 -- 172.21.2.101:0/1859534056 >> 172.21.2.101:0/1859534056 conn(0x7ff1e00ce3e0 msgr2=0x7ff1e00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:01.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.070 7ff1e56be700 1 -- 172.21.2.101:0/1859534056 shutdown_connections 2021-02-10T11:30:01.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.070 7ff1e56be700 1 -- 172.21.2.101:0/1859534056 wait complete. 2021-02-10T11:30:01.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.084 7ff1e56be700 1 Processor -- start 2021-02-10T11:30:01.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.084 7ff1e56be700 1 -- start start 2021-02-10T11:30:01.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.084 7ff1e56be700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1e00a3b50 0x7ff1e01b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:01.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.084 7ff1e56be700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1e01c25b0 0x7ff1e01b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:01.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.084 7ff1e56be700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1e01b4350 0x7ff1e01b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:01.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.084 7ff1e56be700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e01b3190 con 0x7ff1e01c25b0 2021-02-10T11:30:01.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.084 7ff1e56be700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e0087750 con 0x7ff1e01b4350 2021-02-10T11:30:01.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.084 7ff1e56be700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e00a6bf0 con 0x7ff1e00a3b50 2021-02-10T11:30:01.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.085 7ff1deabe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1e01c25b0 0x7ff1e01b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:01.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.085 7ff1deabe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1e01c25b0 0x7ff1e01b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48456/0 (socket says 172.21.2.101:48456) 2021-02-10T11:30:01.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.085 7ff1deabe700 1 -- 172.21.2.101:0/2014430834 learned_addr learned my addr 172.21.2.101:0/2014430834 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:01.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.085 7ff1deabe700 1 -- 172.21.2.101:0/2014430834 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1e00a3b50 msgr2=0x7ff1e01b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:01.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.085 7ff1deabe700 1 --2- 172.21.2.101:0/2014430834 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1e00a3b50 0x7ff1e01b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:01.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.085 7ff1deabe700 1 -- 172.21.2.101:0/2014430834 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1e01b4350 msgr2=0x7ff1e01b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:01.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.085 7ff1deabe700 1 --2- 172.21.2.101:0/2014430834 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1e01b4350 0x7ff1e01b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:01.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.085 7ff1deabe700 1 -- 172.21.2.101:0/2014430834 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e01b1ec0 con 0x7ff1e01c25b0 2021-02-10T11:30:01.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.085 7ff1deabe700 1 --2- 172.21.2.101:0/2014430834 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1e01c25b0 0x7ff1e01b3d30 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7ff1d000a4c0 tx=0x7ff1d000a860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:01.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.085 7ff1d7fff700 1 -- 172.21.2.101:0/2014430834 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1d000ac50 con 0x7ff1e01c25b0 2021-02-10T11:30:01.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.085 7ff1d7fff700 1 -- 172.21.2.101:0/2014430834 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d0004140 con 0x7ff1e01c25b0 2021-02-10T11:30:01.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.085 7ff1e56be700 1 -- 172.21.2.101:0/2014430834 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e01b1ec0 con 0x7ff1e01c25b0 2021-02-10T11:30:01.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.086 7ff1d7fff700 1 -- 172.21.2.101:0/2014430834 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1d0004660 con 0x7ff1e01c25b0 2021-02-10T11:30:01.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.086 7ff1e56be700 1 -- 172.21.2.101:0/2014430834 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e01b1ec0 con 0x7ff1e01c25b0 2021-02-10T11:30:01.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.086 7ff1d7fff700 1 -- 172.21.2.101:0/2014430834 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff1d0001050 con 0x7ff1e01c25b0 2021-02-10T11:30:01.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.086 7ff1d7fff700 1 --2- 172.21.2.101:0/2014430834 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1c40186d0 0x7ff1c401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:01.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.086 7ff1d7fff700 1 -- 172.21.2.101:0/2014430834 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff1d00325d0 con 0x7ff1e01c25b0 2021-02-10T11:30:01.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.086 7ff1df2bf700 1 --2- 172.21.2.101:0/2014430834 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1c40186d0 0x7ff1c401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:01.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.086 7ff1df2bf700 1 --2- 172.21.2.101:0/2014430834 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1c40186d0 0x7ff1c401abd0 crc :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:01.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.118 7ff1e56be700 1 -- 172.21.2.101:0/2014430834 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e007e6b0 con 0x7ff1e01c25b0 2021-02-10T11:30:01.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.120 7ff1d7fff700 1 -- 172.21.2.101:0/2014430834 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff1d0000ce0 con 0x7ff1e01c25b0 2021-02-10T11:30:01.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.250 7ff1e56be700 1 -- 172.21.2.101:0/2014430834 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839240", "target": ["mgr", ""]}) v1 -- 0x7ff1e01b3190 con 0x7ff1c40186d0 2021-02-10T11:30:01.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.261 7ff1d7fff700 1 -- 172.21.2.101:0/2014430834 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7ff1c8000940 con 0x7ff1c40186d0 2021-02-10T11:30:01.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.268 7ff1e56be700 1 -- 172.21.2.101:0/2014430834 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1c40186d0 msgr2=0x7ff1c401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:01.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.268 7ff1e56be700 1 --2- 172.21.2.101:0/2014430834 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1c40186d0 0x7ff1c401abd0 crc :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:01.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.268 7ff1e56be700 1 -- 172.21.2.101:0/2014430834 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1e01c25b0 msgr2=0x7ff1e01b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:01.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.268 7ff1e56be700 1 --2- 172.21.2.101:0/2014430834 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1e01c25b0 0x7ff1e01b3d30 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7ff1d000a4c0 tx=0x7ff1d000a860).stop 2021-02-10T11:30:01.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.268 7ff1e56be700 1 -- 172.21.2.101:0/2014430834 shutdown_connections 2021-02-10T11:30:01.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.268 7ff1e56be700 1 --2- 172.21.2.101:0/2014430834 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1e01b4350 0x7ff1e01b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:01.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.268 7ff1e56be700 1 --2- 172.21.2.101:0/2014430834 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1e01c25b0 0x7ff1e01b3d30 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:01.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.268 7ff1e56be700 1 --2- 172.21.2.101:0/2014430834 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1c40186d0 0x7ff1c401abd0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:01.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.268 7ff1e56be700 1 --2- 172.21.2.101:0/2014430834 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1e00a3b50 0x7ff1e01b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:01.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.268 7ff1e56be700 1 -- 172.21.2.101:0/2014430834 >> 172.21.2.101:0/2014430834 conn(0x7ff1e00886d0 msgr2=0x7ff1e0081c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:01.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.268 7ff1e56be700 1 -- 172.21.2.101:0/2014430834 shutdown_connections 2021-02-10T11:30:01.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.268 7ff1e56be700 1 -- 172.21.2.101:0/2014430834 wait complete. 2021-02-10T11:30:01.288 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839241 2021-02-10T11:30:01.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.619 7f81c7118700 1 Processor -- start 2021-02-10T11:30:01.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.619 7f81c7118700 1 -- start start 2021-02-10T11:30:01.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.619 7f81c7118700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81c01b0470 0x7f81c01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:01.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.619 7f81c7118700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f81c00ef610 0x7f81c00efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:01.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.619 7f81c7118700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c00f0b50 con 0x7f81c00f0030 2021-02-10T11:30:01.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.619 7f81c7118700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81c00e52e0 con 0x7f81c00ef610 2021-02-10T11:30:01.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.619 7f81c7118700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c00e5490 con 0x7f81c01b0470 2021-02-10T11:30:01.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.619 7f81bffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f81c00ef610 0x7f81c00efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:01.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.620 7f81bffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f81c00ef610 0x7f81c00efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48464/0 (socket says 172.21.2.101:48464) 2021-02-10T11:30:01.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.620 7f81bffff700 1 -- 172.21.2.101:0/4179187838 learned_addr learned my addr 172.21.2.101:0/4179187838 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:01.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.620 7f81c5735700 1 --1- 172.21.2.101:0/4179187838 >> v1:172.21.2.101:6789/0 conn(0x7f81c00f0030 0x7f81c00f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48962/0 (socket says 172.21.2.101:48962) 2021-02-10T11:30:01.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.620 7f81bffff700 1 -- 172.21.2.101:0/4179187838 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81c01b0470 msgr2=0x7f81c01b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:30:01.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.620 7f81bffff700 1 --2- 172.21.2.101:0/4179187838 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81c01b0470 0x7f81c01b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:01.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.620 7f81bffff700 1 -- 172.21.2.101:0/4179187838 >> v1:172.21.2.101:6789/0 conn(0x7f81c00f0030 legacy=0x7f81c00f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:01.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.620 7f81bffff700 1 -- 172.21.2.101:0/4179187838 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c00f0430 con 0x7f81c00ef610 2021-02-10T11:30:01.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.620 7f81bf7fe700 1 -- 172.21.2.101:0/4179187838 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2019897689 0 0) 0x7f81b8000c10 con 0x7f81c00f0030 2021-02-10T11:30:01.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.620 7f81bffff700 1 --2- 172.21.2.101:0/4179187838 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81c00ef610 0x7f81c00efa10 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7f81b0006a80 tx=0x7f81b0009f00).ready entity=mon.0 client_cookie=ed3294c272c24d2a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:01.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.620 7f81bf7fe700 1 -- 172.21.2.101:0/4179187838 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81b000b5c0 con 0x7f81c00ef610 2021-02-10T11:30:01.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.620 7f81bf7fe700 1 -- 172.21.2.101:0/4179187838 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b000b8c0 con 0x7f81c00ef610 2021-02-10T11:30:01.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.620 7f81bf7fe700 1 -- 172.21.2.101:0/4179187838 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81b000bda0 con 0x7f81c00ef610 2021-02-10T11:30:01.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.620 7f81c7118700 1 -- 172.21.2.101:0/4179187838 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81c00ef610 msgr2=0x7f81c00efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:01.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.620 7f81c7118700 1 --2- 172.21.2.101:0/4179187838 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81c00ef610 0x7f81c00efa10 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7f81b0006a80 tx=0x7f81b0009f00).stop 2021-02-10T11:30:01.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.621 7f81c7118700 1 -- 172.21.2.101:0/4179187838 shutdown_connections 2021-02-10T11:30:01.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.621 7f81c7118700 1 --2- 172.21.2.101:0/4179187838 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81c00ef610 0x7f81c00efa10 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:01.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.621 7f81c7118700 1 --2- 172.21.2.101:0/4179187838 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81c01b0470 0x7f81c01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:01.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.621 7f81c7118700 1 -- 172.21.2.101:0/4179187838 >> 172.21.2.101:0/4179187838 conn(0x7f81c00ce3e0 msgr2=0x7f81c00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:01.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.621 7f81c7118700 1 -- 172.21.2.101:0/4179187838 shutdown_connections 2021-02-10T11:30:01.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.621 7f81c7118700 1 -- 172.21.2.101:0/4179187838 wait complete. 2021-02-10T11:30:01.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.635 7f81c7118700 1 Processor -- start 2021-02-10T11:30:01.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.635 7f81c7118700 1 -- start start 2021-02-10T11:30:01.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.636 7f81c7118700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81c00b3570 0x7f81c0106cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:01.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.636 7f81c7118700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81c0107310 0x7f81c00aa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:01.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.636 7f81c7118700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81c00aa810 0x7f81c00a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:01.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.636 7f81c7118700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c00ab240 con 0x7f81c00b3570 2021-02-10T11:30:01.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.636 7f81c7118700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c00b3970 con 0x7f81c0107310 2021-02-10T11:30:01.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.636 7f81c7118700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c00ab7c0 con 0x7f81c00aa810 2021-02-10T11:30:01.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.636 7f81c4f34700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81c00b3570 0x7f81c0106cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:01.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.636 7f81c4f34700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81c00b3570 0x7f81c0106cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48474/0 (socket says 172.21.2.101:48474) 2021-02-10T11:30:01.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.636 7f81c4f34700 1 -- 172.21.2.101:0/1146532476 learned_addr learned my addr 172.21.2.101:0/1146532476 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:01.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.636 7f81c4f34700 1 -- 172.21.2.101:0/1146532476 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81c00aa810 msgr2=0x7f81c00a4300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:30:01.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.636 7f81c4f34700 1 --2- 172.21.2.101:0/1146532476 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81c00aa810 0x7f81c00a4300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:01.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.636 7f81c4f34700 1 -- 172.21.2.101:0/1146532476 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81c0107310 msgr2=0x7f81c00aa1f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:30:01.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.636 7f81c4f34700 1 --2- 172.21.2.101:0/1146532476 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81c0107310 0x7f81c00aa1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:01.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.636 7f81c4f34700 1 -- 172.21.2.101:0/1146532476 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c0107710 con 0x7f81c00b3570 2021-02-10T11:30:01.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.636 7f81c4f34700 1 --2- 172.21.2.101:0/1146532476 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81c00b3570 0x7f81c0106cf0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7f81a800b5c0 tx=0x7f81a800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:01.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.637 7f81bd7fa700 1 -- 172.21.2.101:0/1146532476 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81a800bd50 con 0x7f81c00b3570 2021-02-10T11:30:01.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.637 7f81bd7fa700 1 -- 172.21.2.101:0/1146532476 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a8006030 con 0x7f81c00b3570 2021-02-10T11:30:01.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.637 7f81bd7fa700 1 -- 172.21.2.101:0/1146532476 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81a80054d0 con 0x7f81c00b3570 2021-02-10T11:30:01.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.637 7f81c7118700 1 -- 172.21.2.101:0/1146532476 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c0107710 con 0x7f81c00b3570 2021-02-10T11:30:01.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.637 7f81c7118700 1 -- 172.21.2.101:0/1146532476 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c00aaf00 con 0x7f81c00b3570 2021-02-10T11:30:01.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.637 7f81bd7fa700 1 -- 172.21.2.101:0/1146532476 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f81a8006210 con 0x7f81c00b3570 2021-02-10T11:30:01.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.637 7f81bd7fa700 1 --2- 172.21.2.101:0/1146532476 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81a40186c0 0x7f81a401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:01.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.637 7f81bd7fa700 1 -- 172.21.2.101:0/1146532476 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f81a8032580 con 0x7f81c00b3570 2021-02-10T11:30:01.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.638 7f81bffff700 1 --2- 172.21.2.101:0/1146532476 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81a40186c0 0x7f81a401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:01.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.638 7f81bffff700 1 --2- 172.21.2.101:0/1146532476 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81a40186c0 0x7f81a401abc0 crc :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:01.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.669 7f81c7118700 1 -- 172.21.2.101:0/1146532476 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c0105af0 con 0x7f81c00b3570 2021-02-10T11:30:01.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.671 7f81bd7fa700 1 -- 172.21.2.101:0/1146532476 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f81a8030db0 con 0x7f81c00b3570 2021-02-10T11:30:01.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.800 7f81c7118700 1 -- 172.21.2.101:0/1146532476 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839241", "target": ["mgr", ""]}) v1 -- 0x7f81c00aaf00 con 0x7f81a40186c0 2021-02-10T11:30:01.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.810 7f81bd7fa700 1 -- 172.21.2.101:0/1146532476 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f81b0006b90 con 0x7f81a40186c0 2021-02-10T11:30:01.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.817 7f81c7118700 1 -- 172.21.2.101:0/1146532476 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81a40186c0 msgr2=0x7f81a401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:01.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.817 7f81c7118700 1 --2- 172.21.2.101:0/1146532476 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81a40186c0 0x7f81a401abc0 crc :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:01.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.817 7f81c7118700 1 -- 172.21.2.101:0/1146532476 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81c00b3570 msgr2=0x7f81c0106cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:01.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.817 7f81c7118700 1 --2- 172.21.2.101:0/1146532476 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81c00b3570 0x7f81c0106cf0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7f81a800b5c0 tx=0x7f81a800b960).stop 2021-02-10T11:30:01.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.818 7f81c7118700 1 -- 172.21.2.101:0/1146532476 shutdown_connections 2021-02-10T11:30:01.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.818 7f81c7118700 1 --2- 172.21.2.101:0/1146532476 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81a40186c0 0x7f81a401abc0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:01.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.818 7f81c7118700 1 --2- 172.21.2.101:0/1146532476 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81c00aa810 0x7f81c00a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:01.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.818 7f81c7118700 1 --2- 172.21.2.101:0/1146532476 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81c0107310 0x7f81c00aa1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:01.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.818 7f81c7118700 1 --2- 172.21.2.101:0/1146532476 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81c00b3570 0x7f81c0106cf0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:01.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.818 7f81c7118700 1 -- 172.21.2.101:0/1146532476 >> 172.21.2.101:0/1146532476 conn(0x7f81c00c40c0 msgr2=0x7f81c00a3520 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:01.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.818 7f81c7118700 1 -- 172.21.2.101:0/1146532476 shutdown_connections 2021-02-10T11:30:01.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:01.818 7f81c7118700 1 -- 172.21.2.101:0/1146532476 wait complete. 2021-02-10T11:30:01.837 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839242 2021-02-10T11:30:02.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.244 7f8bff7c1700 1 Processor -- start 2021-02-10T11:30:02.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.244 7f8bff7c1700 1 -- start start 2021-02-10T11:30:02.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.244 7f8bff7c1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8bf81b0870 0x7f8bf80f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:02.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.244 7f8bff7c1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8bf80efe00 0x7f8bf80f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:02.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.244 7f8bff7c1700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf81b0c70 con 0x7f8bf81b0470 2021-02-10T11:30:02.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.244 7f8bff7c1700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf80e5ea0 con 0x7f8bf81b0870 2021-02-10T11:30:02.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.244 7f8bff7c1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf80f0b50 con 0x7f8bf80efe00 2021-02-10T11:30:02.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.244 7f8bfcddc700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8bf81b0870 0x7f8bf80f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:02.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.244 7f8bfd5dd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8bf81b0470 0x7f8bf80ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48972/0 (socket says 172.21.2.101:48972) 2021-02-10T11:30:02.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.244 7f8bfd5dd700 1 -- 172.21.2.101:0/1318103015 learned_addr learned my addr 172.21.2.101:0/1318103015 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:02.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.244 7f8bfddde700 1 --2- 172.21.2.101:0/1318103015 >> v2:172.21.2.108:3300/0 conn(0x7f8bf80efe00 0x7f8bf80f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:02.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.245 7f8bfcddc700 1 -- 172.21.2.101:0/1318103015 >> v2:172.21.2.108:3300/0 conn(0x7f8bf80efe00 msgr2=0x7f8bf80f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:02.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.245 7f8bfcddc700 1 --2- 172.21.2.101:0/1318103015 >> v2:172.21.2.108:3300/0 conn(0x7f8bf80efe00 0x7f8bf80f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:02.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.245 7f8bfcddc700 1 -- 172.21.2.101:0/1318103015 >> v1:172.21.2.101:6789/0 conn(0x7f8bf81b0470 legacy=0x7f8bf80ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:02.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.245 7f8bfcddc700 1 -- 172.21.2.101:0/1318103015 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf80f0820 con 0x7f8bf81b0870 2021-02-10T11:30:02.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.245 7f8bfddde700 1 --2- 172.21.2.101:0/1318103015 >> v2:172.21.2.108:3300/0 conn(0x7f8bf80efe00 0x7f8bf80f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:02.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.245 7f8bfcddc700 1 --2- 172.21.2.101:0/1318103015 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8bf81b0870 0x7f8bf80f20e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f8bf00069d0 tx=0x7f8bf0009e40).ready entity=mon.0 client_cookie=a94fcb2fc60cc517 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:02.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.245 7f8beffff700 1 -- 172.21.2.101:0/1318103015 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8bf000b510 con 0x7f8bf81b0870 2021-02-10T11:30:02.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.245 7f8beffff700 1 -- 172.21.2.101:0/1318103015 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf000b810 con 0x7f8bf81b0870 2021-02-10T11:30:02.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.245 7f8beffff700 1 -- 172.21.2.101:0/1318103015 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8bf000bca0 con 0x7f8bf81b0870 2021-02-10T11:30:02.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.245 7f8bff7c1700 1 -- 172.21.2.101:0/1318103015 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8bf81b0870 msgr2=0x7f8bf80f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:02.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.245 7f8bff7c1700 1 --2- 172.21.2.101:0/1318103015 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8bf81b0870 0x7f8bf80f20e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f8bf00069d0 tx=0x7f8bf0009e40).stop 2021-02-10T11:30:02.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.246 7f8bff7c1700 1 -- 172.21.2.101:0/1318103015 shutdown_connections 2021-02-10T11:30:02.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.246 7f8bff7c1700 1 --2- 172.21.2.101:0/1318103015 >> v2:172.21.2.108:3300/0 conn(0x7f8bf80efe00 0x7f8bf80f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:02.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.246 7f8bff7c1700 1 --2- 172.21.2.101:0/1318103015 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8bf81b0870 0x7f8bf80f20e0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:02.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.246 7f8bff7c1700 1 -- 172.21.2.101:0/1318103015 >> 172.21.2.101:0/1318103015 conn(0x7f8bf80ce3e0 msgr2=0x7f8bf80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:02.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.246 7f8bff7c1700 1 -- 172.21.2.101:0/1318103015 shutdown_connections 2021-02-10T11:30:02.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.246 7f8bff7c1700 1 -- 172.21.2.101:0/1318103015 wait complete. 2021-02-10T11:30:02.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.260 7f8bff7c1700 1 Processor -- start 2021-02-10T11:30:02.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.261 7f8bff7c1700 1 -- start start 2021-02-10T11:30:02.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.261 7f8bff7c1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8bf80a0420 0x7f8bf80a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:02.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.261 7f8bff7c1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8bf8189d00 0x7f8bf818a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:02.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.261 7f8bff7c1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8bf818a720 0x7f8bf80b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:02.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.261 7f8bff7c1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf8189800 con 0x7f8bf818a720 2021-02-10T11:30:02.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.261 7f8bff7c1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf818ab20 con 0x7f8bf80a0420 2021-02-10T11:30:02.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.261 7f8bff7c1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf80a1140 con 0x7f8bf8189d00 2021-02-10T11:30:02.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.261 7f8bfd5dd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8bf80a0420 0x7f8bf80a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:02.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.261 7f8bfddde700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8bf818a720 0x7f8bf80b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:02.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.261 7f8bfd5dd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8bf80a0420 0x7f8bf80a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:58970/0 (socket says 172.21.2.101:58970) 2021-02-10T11:30:02.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.261 7f8bfd5dd700 1 -- 172.21.2.101:0/3820945704 learned_addr learned my addr 172.21.2.101:0/3820945704 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:02.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.261 7f8bfcddc700 1 --2- 172.21.2.101:0/3820945704 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8bf8189d00 0x7f8bf818a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:02.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.262 7f8bfddde700 1 -- 172.21.2.101:0/3820945704 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8bf8189d00 msgr2=0x7f8bf818a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:02.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.262 7f8bfddde700 1 --2- 172.21.2.101:0/3820945704 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8bf8189d00 0x7f8bf818a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:02.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.262 7f8bfddde700 1 -- 172.21.2.101:0/3820945704 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8bf80a0420 msgr2=0x7f8bf80a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:02.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.262 7f8bfddde700 1 --2- 172.21.2.101:0/3820945704 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8bf80a0420 0x7f8bf80a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:02.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.262 7f8bfddde700 1 -- 172.21.2.101:0/3820945704 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf80b8200 con 0x7f8bf818a720 2021-02-10T11:30:02.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.262 7f8bfcddc700 1 --2- 172.21.2.101:0/3820945704 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8bf8189d00 0x7f8bf818a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:30:02.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.262 7f8bfddde700 1 --2- 172.21.2.101:0/3820945704 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8bf818a720 0x7f8bf80b7be0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f8bf4006ed0 tx=0x7f8bf400b8c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:02.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.262 7f8bedffb700 1 -- 172.21.2.101:0/3820945704 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8bf40066c0 con 0x7f8bf818a720 2021-02-10T11:30:02.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.262 7f8bedffb700 1 -- 172.21.2.101:0/3820945704 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf4006430 con 0x7f8bf818a720 2021-02-10T11:30:02.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.262 7f8bff7c1700 1 -- 172.21.2.101:0/3820945704 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf80b8200 con 0x7f8bf818a720 2021-02-10T11:30:02.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.262 7f8bff7c1700 1 -- 172.21.2.101:0/3820945704 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf80b8790 con 0x7f8bf818a720 2021-02-10T11:30:02.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.262 7f8bedffb700 1 -- 172.21.2.101:0/3820945704 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8bf40022e0 con 0x7f8bf818a720 2021-02-10T11:30:02.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.263 7f8bedffb700 1 -- 172.21.2.101:0/3820945704 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8bf4002050 con 0x7f8bf818a720 2021-02-10T11:30:02.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.263 7f8bedffb700 1 --2- 172.21.2.101:0/3820945704 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8bd80186c0 0x7f8bd801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:02.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.263 7f8bedffb700 1 -- 172.21.2.101:0/3820945704 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8bf40323f0 con 0x7f8bf818a720 2021-02-10T11:30:02.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.263 7f8bfd5dd700 1 --2- 172.21.2.101:0/3820945704 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8bd80186c0 0x7f8bd801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:02.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.263 7f8bfd5dd700 1 --2- 172.21.2.101:0/3820945704 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8bd80186c0 0x7f8bd801abc0 crc :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:02.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.294 7f8bff7c1700 1 -- 172.21.2.101:0/3820945704 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf8105ae0 con 0x7f8bf818a720 2021-02-10T11:30:02.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.297 7f8bedffb700 1 -- 172.21.2.101:0/3820945704 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8bf4000bc0 con 0x7f8bf818a720 2021-02-10T11:30:02.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.427 7f8bff7c1700 1 -- 172.21.2.101:0/3820945704 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839242", "target": ["mgr", ""]}) v1 -- 0x7f8bf80b8790 con 0x7f8bd80186c0 2021-02-10T11:30:02.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.437 7f8bedffb700 1 -- 172.21.2.101:0/3820945704 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f8be8001570 con 0x7f8bd80186c0 2021-02-10T11:30:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.445 7f8bff7c1700 1 -- 172.21.2.101:0/3820945704 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8bd80186c0 msgr2=0x7f8bd801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.445 7f8bff7c1700 1 --2- 172.21.2.101:0/3820945704 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8bd80186c0 0x7f8bd801abc0 crc :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.445 7f8bff7c1700 1 -- 172.21.2.101:0/3820945704 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8bf818a720 msgr2=0x7f8bf80b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.445 7f8bff7c1700 1 --2- 172.21.2.101:0/3820945704 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8bf818a720 0x7f8bf80b7be0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f8bf4006ed0 tx=0x7f8bf400b8c0).stop 2021-02-10T11:30:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.445 7f8bff7c1700 1 -- 172.21.2.101:0/3820945704 shutdown_connections 2021-02-10T11:30:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.445 7f8bff7c1700 1 --2- 172.21.2.101:0/3820945704 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8bd80186c0 0x7f8bd801abc0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.445 7f8bff7c1700 1 --2- 172.21.2.101:0/3820945704 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8bf8189d00 0x7f8bf818a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.445 7f8bff7c1700 1 --2- 172.21.2.101:0/3820945704 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8bf818a720 0x7f8bf80b7be0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.445 7f8bff7c1700 1 --2- 172.21.2.101:0/3820945704 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8bf80a0420 0x7f8bf80a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.445 7f8bff7c1700 1 -- 172.21.2.101:0/3820945704 >> 172.21.2.101:0/3820945704 conn(0x7f8bf8094e10 msgr2=0x7f8bf80a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:02.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.445 7f8bff7c1700 1 -- 172.21.2.101:0/3820945704 shutdown_connections 2021-02-10T11:30:02.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.445 7f8bff7c1700 1 -- 172.21.2.101:0/3820945704 wait complete. 2021-02-10T11:30:02.465 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839243 2021-02-10T11:30:02.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.869 7f8dfc273700 1 Processor -- start 2021-02-10T11:30:02.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.869 7f8dfc273700 1 -- start start 2021-02-10T11:30:02.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.869 7f8dfc273700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8df41b0470 0x7f8df41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:02.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.869 7f8dfc273700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df40f0b50 con 0x7f8df40ef610 2021-02-10T11:30:02.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.869 7f8dfc273700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df40f24e0 con 0x7f8df40f20e0 2021-02-10T11:30:02.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.869 7f8dfc273700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df40e5ea0 con 0x7f8df41b0470 2021-02-10T11:30:02.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.870 7f8dfa08f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8df41b0470 0x7f8df41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:02.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.870 7f8df988e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8df40ef610 0x7f8df40efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:48986/0 (socket says 172.21.2.101:48986) 2021-02-10T11:30:02.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.870 7f8dfa890700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f8df40f20e0 0x7f8df40f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:42396/0 (socket says 172.21.2.101:42396) 2021-02-10T11:30:02.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.870 7f8df988e700 1 -- 172.21.2.101:0/2220406915 learned_addr learned my addr 172.21.2.101:0/2220406915 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:02.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.870 7f8dfa08f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8df41b0470 0x7f8df41b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48492/0 (socket says 172.21.2.101:48492) 2021-02-10T11:30:02.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.870 7f8dfa08f700 1 -- 172.21.2.101:0/2220406915 >> v1:172.21.2.108:6789/0 conn(0x7f8df40f20e0 legacy=0x7f8df40f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:02.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.870 7f8dfa08f700 1 -- 172.21.2.101:0/2220406915 >> v1:172.21.2.101:6789/0 conn(0x7f8df40ef610 legacy=0x7f8df40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:02.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.870 7f8dfa08f700 1 -- 172.21.2.101:0/2220406915 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df40e52e0 con 0x7f8df41b0470 2021-02-10T11:30:02.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.870 7f8dfa08f700 1 --2- 172.21.2.101:0/2220406915 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8df41b0470 0x7f8df41b0870 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7f8de4006650 tx=0x7f8de4009a00).ready entity=mon.0 client_cookie=8e1021a151e1ba73 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:02.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.870 7f8df908d700 1 -- 172.21.2.101:0/2220406915 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8de4009df0 con 0x7f8df41b0470 2021-02-10T11:30:02.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.871 7f8df908d700 1 -- 172.21.2.101:0/2220406915 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de40009b0 con 0x7f8df41b0470 2021-02-10T11:30:02.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.871 7f8df908d700 1 -- 172.21.2.101:0/2220406915 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8de400c2b0 con 0x7f8df41b0470 2021-02-10T11:30:02.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.871 7f8dfc273700 1 -- 172.21.2.101:0/2220406915 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8df41b0470 msgr2=0x7f8df41b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:02.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.871 7f8dfc273700 1 --2- 172.21.2.101:0/2220406915 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8df41b0470 0x7f8df41b0870 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7f8de4006650 tx=0x7f8de4009a00).stop 2021-02-10T11:30:02.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.871 7f8dfc273700 1 -- 172.21.2.101:0/2220406915 shutdown_connections 2021-02-10T11:30:02.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.871 7f8dfc273700 1 --2- 172.21.2.101:0/2220406915 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8df41b0470 0x7f8df41b0870 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:02.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.871 7f8dfc273700 1 -- 172.21.2.101:0/2220406915 >> 172.21.2.101:0/2220406915 conn(0x7f8df40ce3e0 msgr2=0x7f8df40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:02.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.872 7f8dfc273700 1 -- 172.21.2.101:0/2220406915 shutdown_connections 2021-02-10T11:30:02.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.872 7f8dfc273700 1 -- 172.21.2.101:0/2220406915 wait complete. 2021-02-10T11:30:02.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.886 7f8dfc273700 1 Processor -- start 2021-02-10T11:30:02.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.886 7f8dfc273700 1 -- start start 2021-02-10T11:30:02.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.887 7f8dfc273700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8df41a7a30 0x7f8df41a7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:02.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.887 7f8dfc273700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8df40ee0f0 0x7f8df40ee4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:02.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.887 7f8dfc273700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8df40eeb10 0x7f8df40eef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:02.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.887 7f8dfc273700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df4093300 con 0x7f8df40eeb10 2021-02-10T11:30:02.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.887 7f8dfc273700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df40ef530 con 0x7f8df41a7a30 2021-02-10T11:30:02.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.887 7f8dfc273700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df40c1e50 con 0x7f8df40ee0f0 2021-02-10T11:30:02.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.887 7f8dfa890700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8df40eeb10 0x7f8df40eef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:02.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.887 7f8dfa08f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8df41a7a30 0x7f8df41a7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:02.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.887 7f8dfa890700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8df40eeb10 0x7f8df40eef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48498/0 (socket says 172.21.2.101:48498) 2021-02-10T11:30:02.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.887 7f8dfa890700 1 -- 172.21.2.101:0/3987021446 learned_addr learned my addr 172.21.2.101:0/3987021446 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:02.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.887 7f8df988e700 1 --2- 172.21.2.101:0/3987021446 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8df40ee0f0 0x7f8df40ee4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:02.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.887 7f8dfa08f700 1 -- 172.21.2.101:0/3987021446 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8df40ee0f0 msgr2=0x7f8df40ee4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:02.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.887 7f8dfa08f700 1 --2- 172.21.2.101:0/3987021446 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8df40ee0f0 0x7f8df40ee4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:02.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.887 7f8dfa08f700 1 -- 172.21.2.101:0/3987021446 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8df40eeb10 msgr2=0x7f8df40eef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:02.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.887 7f8dfa08f700 1 --2- 172.21.2.101:0/3987021446 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8df40eeb10 0x7f8df40eef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:02.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.887 7f8dfa08f700 1 -- 172.21.2.101:0/3987021446 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df40c2070 con 0x7f8df41a7a30 2021-02-10T11:30:02.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.887 7f8dfa890700 1 --2- 172.21.2.101:0/3987021446 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8df40eeb10 0x7f8df40eef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:02.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.887 7f8dfa08f700 1 --2- 172.21.2.101:0/3987021446 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8df41a7a30 0x7f8df41a7e30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f8de4006620 tx=0x7f8de400b860).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:02.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.888 7f8deaffd700 1 -- 172.21.2.101:0/3987021446 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8de4000c90 con 0x7f8df41a7a30 2021-02-10T11:30:02.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.888 7f8deaffd700 1 -- 172.21.2.101:0/3987021446 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de400bc50 con 0x7f8df41a7a30 2021-02-10T11:30:02.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.888 7f8deaffd700 1 -- 172.21.2.101:0/3987021446 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8de40022b0 con 0x7f8df41a7a30 2021-02-10T11:30:02.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.888 7f8dfc273700 1 -- 172.21.2.101:0/3987021446 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df40c2070 con 0x7f8df41a7a30 2021-02-10T11:30:02.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.888 7f8dfc273700 1 -- 172.21.2.101:0/3987021446 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df40c2600 con 0x7f8df41a7a30 2021-02-10T11:30:02.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.888 7f8deaffd700 1 -- 172.21.2.101:0/3987021446 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8de40052e0 con 0x7f8df41a7a30 2021-02-10T11:30:02.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.888 7f8deaffd700 1 --2- 172.21.2.101:0/3987021446 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ddc018730 0x7f8ddc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:02.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.888 7f8deaffd700 1 -- 172.21.2.101:0/3987021446 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8de40326b0 con 0x7f8df41a7a30 2021-02-10T11:30:02.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.888 7f8df988e700 1 --2- 172.21.2.101:0/3987021446 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ddc018730 0x7f8ddc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:02.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.889 7f8df988e700 1 --2- 172.21.2.101:0/3987021446 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ddc018730 0x7f8ddc01abc0 crc :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:02.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.919 7f8dfc273700 1 -- 172.21.2.101:0/3987021446 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df40e52e0 con 0x7f8df41a7a30 2021-02-10T11:30:02.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:02.922 7f8deaffd700 1 -- 172.21.2.101:0/3987021446 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8de40009a0 con 0x7f8df41a7a30 2021-02-10T11:30:03.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.050 7f8dfc273700 1 -- 172.21.2.101:0/3987021446 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839243", "target": ["mgr", ""]}) v1 -- 0x7f8df40e52e0 con 0x7f8ddc018730 2021-02-10T11:30:03.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.061 7f8deaffd700 1 -- 172.21.2.101:0/3987021446 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f8df0001a00 con 0x7f8ddc018730 2021-02-10T11:30:03.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.067 7f8dfc273700 1 -- 172.21.2.101:0/3987021446 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ddc018730 msgr2=0x7f8ddc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:03.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.067 7f8dfc273700 1 --2- 172.21.2.101:0/3987021446 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ddc018730 0x7f8ddc01abc0 crc :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:03.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.067 7f8dfc273700 1 -- 172.21.2.101:0/3987021446 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8df41a7a30 msgr2=0x7f8df41a7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:03.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.067 7f8dfc273700 1 --2- 172.21.2.101:0/3987021446 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8df41a7a30 0x7f8df41a7e30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f8de4006620 tx=0x7f8de400b860).stop 2021-02-10T11:30:03.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.067 7f8dfc273700 1 -- 172.21.2.101:0/3987021446 shutdown_connections 2021-02-10T11:30:03.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.067 7f8dfc273700 1 --2- 172.21.2.101:0/3987021446 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ddc018730 0x7f8ddc01abc0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:03.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.067 7f8dfc273700 1 --2- 172.21.2.101:0/3987021446 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8df40ee0f0 0x7f8df40ee4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:03.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.067 7f8dfc273700 1 --2- 172.21.2.101:0/3987021446 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8df40eeb10 0x7f8df40eef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:03.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.067 7f8dfc273700 1 --2- 172.21.2.101:0/3987021446 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8df41a7a30 0x7f8df41a7e30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:03.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.067 7f8dfc273700 1 -- 172.21.2.101:0/3987021446 >> 172.21.2.101:0/3987021446 conn(0x7f8df40a2190 msgr2=0x7f8df40a2690 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:03.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.067 7f8dfc273700 1 -- 172.21.2.101:0/3987021446 shutdown_connections 2021-02-10T11:30:03.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.067 7f8dfc273700 1 -- 172.21.2.101:0/3987021446 wait complete. 2021-02-10T11:30:03.087 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839244 2021-02-10T11:30:03.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.493 7f7ac68e7700 1 Processor -- start 2021-02-10T11:30:03.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.493 7f7ac68e7700 1 -- start start 2021-02-10T11:30:03.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.493 7f7ac68e7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ac01b0470 0x7f7ac01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:03.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.493 7f7ac68e7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7ac00ef610 0x7f7ac00efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:03.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.493 7f7ac68e7700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac00f0b50 con 0x7f7ac00f0030 2021-02-10T11:30:03.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.493 7f7ac68e7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ac00e52e0 con 0x7f7ac00ef610 2021-02-10T11:30:03.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.493 7f7ac68e7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac00e5490 con 0x7f7ac01b0470 2021-02-10T11:30:03.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.493 7f7abffff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ac01b0470 0x7f7ac01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:03.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.493 7f7abffff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ac01b0470 0x7f7ac01b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:41234/0 (socket says 172.21.2.101:41234) 2021-02-10T11:30:03.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.493 7f7abffff700 1 -- 172.21.2.101:0/1517365891 learned_addr learned my addr 172.21.2.101:0/1517365891 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:03.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.493 7f7abf7fe700 1 --2- 172.21.2.101:0/1517365891 >> v2:172.21.2.108:3300/0 conn(0x7f7ac00ef610 0x7f7ac00efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:03.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.494 7f7abffff700 1 -- 172.21.2.101:0/1517365891 >> v2:172.21.2.108:3300/0 conn(0x7f7ac00ef610 msgr2=0x7f7ac00efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:03.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.494 7f7abffff700 1 --2- 172.21.2.101:0/1517365891 >> v2:172.21.2.108:3300/0 conn(0x7f7ac00ef610 0x7f7ac00efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:03.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.494 7f7abffff700 1 -- 172.21.2.101:0/1517365891 >> v1:172.21.2.108:6789/0 conn(0x7f7ac00f0030 legacy=0x7f7ac00f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:03.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.494 7f7abffff700 1 -- 172.21.2.101:0/1517365891 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac00f0430 con 0x7f7ac01b0470 2021-02-10T11:30:03.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.494 7f7abf7fe700 1 --2- 172.21.2.101:0/1517365891 >> v2:172.21.2.108:3300/0 conn(0x7f7ac00ef610 0x7f7ac00efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:03.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.494 7f7abffff700 1 --2- 172.21.2.101:0/1517365891 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ac01b0470 0x7f7ac01b0870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f7ab0006700 tx=0x7f7ab0009b80).ready entity=mon.2 client_cookie=9656afa950f4e940 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:03.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.494 7f7abeffd700 1 -- 172.21.2.101:0/1517365891 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ab0006340 con 0x7f7ac01b0470 2021-02-10T11:30:03.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.494 7f7abeffd700 1 -- 172.21.2.101:0/1517365891 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab000c0c0 con 0x7f7ac01b0470 2021-02-10T11:30:03.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.494 7f7abeffd700 1 -- 172.21.2.101:0/1517365891 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ab000c5e0 con 0x7f7ac01b0470 2021-02-10T11:30:03.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.494 7f7ac68e7700 1 -- 172.21.2.101:0/1517365891 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ac01b0470 msgr2=0x7f7ac01b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:03.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.494 7f7ac68e7700 1 --2- 172.21.2.101:0/1517365891 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ac01b0470 0x7f7ac01b0870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f7ab0006700 tx=0x7f7ab0009b80).stop 2021-02-10T11:30:03.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.495 7f7ac68e7700 1 -- 172.21.2.101:0/1517365891 shutdown_connections 2021-02-10T11:30:03.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.495 7f7ac68e7700 1 --2- 172.21.2.101:0/1517365891 >> v2:172.21.2.108:3300/0 conn(0x7f7ac00ef610 0x7f7ac00efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:03.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.495 7f7ac68e7700 1 --2- 172.21.2.101:0/1517365891 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ac01b0470 0x7f7ac01b0870 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:03.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.495 7f7ac68e7700 1 -- 172.21.2.101:0/1517365891 >> 172.21.2.101:0/1517365891 conn(0x7f7ac00ce3e0 msgr2=0x7f7ac00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:03.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.495 7f7ac68e7700 1 -- 172.21.2.101:0/1517365891 shutdown_connections 2021-02-10T11:30:03.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.495 7f7ac68e7700 1 -- 172.21.2.101:0/1517365891 wait complete. 2021-02-10T11:30:03.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.510 7f7ac68e7700 1 Processor -- start 2021-02-10T11:30:03.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.510 7f7ac68e7700 1 -- start start 2021-02-10T11:30:03.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.510 7f7ac68e7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ac0190920 0x7f7ac00b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:03.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.510 7f7ac68e7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ac00b8200 0x7f7ac00b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:03.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.510 7f7ac68e7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ac00c1d80 0x7f7ac00c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:03.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.510 7f7ac68e7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac0190da0 con 0x7f7ac00b8200 2021-02-10T11:30:03.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.510 7f7ac68e7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac00b8c20 con 0x7f7ac00c1d80 2021-02-10T11:30:03.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.510 7f7ac68e7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac00b8e10 con 0x7f7ac0190920 2021-02-10T11:30:03.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.510 7f7abf7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ac00b8200 0x7f7ac00b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:03.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.510 7f7abf7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ac00b8200 0x7f7ac00b8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48508/0 (socket says 172.21.2.101:48508) 2021-02-10T11:30:03.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.510 7f7abf7fe700 1 -- 172.21.2.101:0/3639815703 learned_addr learned my addr 172.21.2.101:0/3639815703 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:03.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.511 7f7abffff700 1 --2- 172.21.2.101:0/3639815703 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ac0190920 0x7f7ac00b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:03.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.511 7f7ac4f04700 1 --2- 172.21.2.101:0/3639815703 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ac00c1d80 0x7f7ac00c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:03.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.511 7f7abf7fe700 1 -- 172.21.2.101:0/3639815703 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ac0190920 msgr2=0x7f7ac00b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:03.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.511 7f7abf7fe700 1 --2- 172.21.2.101:0/3639815703 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ac0190920 0x7f7ac00b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:03.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.511 7f7abf7fe700 1 -- 172.21.2.101:0/3639815703 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ac00c1d80 msgr2=0x7f7ac00c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:03.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.511 7f7abf7fe700 1 --2- 172.21.2.101:0/3639815703 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ac00c1d80 0x7f7ac00c2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:03.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.511 7f7abf7fe700 1 -- 172.21.2.101:0/3639815703 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac00c27d0 con 0x7f7ac00b8200 2021-02-10T11:30:03.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.511 7f7abffff700 1 --2- 172.21.2.101:0/3639815703 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ac0190920 0x7f7ac00b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:30:03.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.511 7f7abf7fe700 1 --2- 172.21.2.101:0/3639815703 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ac00b8200 0x7f7ac00b8600 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f7ab4006f40 tx=0x7f7ab400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:03.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.511 7f7abcff9700 1 -- 172.21.2.101:0/3639815703 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ab400bd80 con 0x7f7ac00b8200 2021-02-10T11:30:03.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.511 7f7abcff9700 1 -- 172.21.2.101:0/3639815703 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab4000c90 con 0x7f7ac00b8200 2021-02-10T11:30:03.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.511 7f7abcff9700 1 -- 172.21.2.101:0/3639815703 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ab4002220 con 0x7f7ac00b8200 2021-02-10T11:30:03.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.511 7f7ac68e7700 1 -- 172.21.2.101:0/3639815703 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac00c27d0 con 0x7f7ac00b8200 2021-02-10T11:30:03.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.511 7f7ac68e7700 1 -- 172.21.2.101:0/3639815703 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac00c2d60 con 0x7f7ac00b8200 2021-02-10T11:30:03.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.512 7f7abcff9700 1 -- 172.21.2.101:0/3639815703 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7ab400a000 con 0x7f7ac00b8200 2021-02-10T11:30:03.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.512 7f7abcff9700 1 --2- 172.21.2.101:0/3639815703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7aa00186c0 0x7f7aa001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:03.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.512 7f7abcff9700 1 -- 172.21.2.101:0/3639815703 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7ab4032640 con 0x7f7ac00b8200 2021-02-10T11:30:03.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.512 7f7abffff700 1 --2- 172.21.2.101:0/3639815703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7aa00186c0 0x7f7aa001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:03.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.512 7f7abffff700 1 --2- 172.21.2.101:0/3639815703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7aa00186c0 0x7f7aa001abc0 crc :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:03.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.542 7f7ac68e7700 1 -- 172.21.2.101:0/3639815703 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac0105ae0 con 0x7f7ac00b8200 2021-02-10T11:30:03.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.544 7f7abcff9700 1 -- 172.21.2.101:0/3639815703 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7ab4000960 con 0x7f7ac00b8200 2021-02-10T11:30:03.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.670 7f7ac68e7700 1 -- 172.21.2.101:0/3639815703 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839244", "target": ["mgr", ""]}) v1 -- 0x7f7ac00c2d60 con 0x7f7aa00186c0 2021-02-10T11:30:03.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.679 7f7abcff9700 1 -- 172.21.2.101:0/3639815703 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f7ab00009b0 con 0x7f7aa00186c0 2021-02-10T11:30:03.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.687 7f7ac68e7700 1 -- 172.21.2.101:0/3639815703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7aa00186c0 msgr2=0x7f7aa001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:03.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.687 7f7ac68e7700 1 --2- 172.21.2.101:0/3639815703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7aa00186c0 0x7f7aa001abc0 crc :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:03.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.687 7f7ac68e7700 1 -- 172.21.2.101:0/3639815703 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ac00b8200 msgr2=0x7f7ac00b8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:03.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.687 7f7ac68e7700 1 --2- 172.21.2.101:0/3639815703 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ac00b8200 0x7f7ac00b8600 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f7ab4006f40 tx=0x7f7ab400b990).stop 2021-02-10T11:30:03.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.687 7f7ac68e7700 1 -- 172.21.2.101:0/3639815703 shutdown_connections 2021-02-10T11:30:03.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.687 7f7ac68e7700 1 --2- 172.21.2.101:0/3639815703 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ac00c1d80 0x7f7ac00c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:03.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.687 7f7ac68e7700 1 --2- 172.21.2.101:0/3639815703 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ac00b8200 0x7f7ac00b8600 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:03.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.687 7f7ac68e7700 1 --2- 172.21.2.101:0/3639815703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7aa00186c0 0x7f7aa001abc0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:03.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.687 7f7ac68e7700 1 --2- 172.21.2.101:0/3639815703 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ac0190920 0x7f7ac00b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:03.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.687 7f7ac68e7700 1 -- 172.21.2.101:0/3639815703 >> 172.21.2.101:0/3639815703 conn(0x7f7ac0094e10 msgr2=0x7f7ac00a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:03.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.687 7f7ac68e7700 1 -- 172.21.2.101:0/3639815703 shutdown_connections 2021-02-10T11:30:03.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:03.687 7f7ac68e7700 1 -- 172.21.2.101:0/3639815703 wait complete. 2021-02-10T11:30:03.707 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839245 2021-02-10T11:30:04.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.110 7f9205e2c700 1 Processor -- start 2021-02-10T11:30:04.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.110 7f9205e2c700 1 -- start start 2021-02-10T11:30:04.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.110 7f9205e2c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f92001b0870 0x7f92000f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:04.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.110 7f9205e2c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f92000efe00 0x7f92000f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:04.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.110 7f9205e2c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92001b0c70 con 0x7f92001b0470 2021-02-10T11:30:04.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.110 7f9205e2c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92000e5ea0 con 0x7f92000efe00 2021-02-10T11:30:04.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.110 7f9205e2c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92000f0b50 con 0x7f92001b0870 2021-02-10T11:30:04.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.110 7f91fffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f92000efe00 0x7f92000f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:04.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.110 7f91fffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f92000efe00 0x7f92000f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48518/0 (socket says 172.21.2.101:48518) 2021-02-10T11:30:04.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.110 7f91fffff700 1 -- 172.21.2.101:0/2422531134 learned_addr learned my addr 172.21.2.101:0/2422531134 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:04.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.110 7f91ff7fe700 1 --1- 172.21.2.101:0/2422531134 >> v1:172.21.2.101:6789/0 conn(0x7f92001b0470 0x7f92000ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49008/0 (socket says 172.21.2.101:49008) 2021-02-10T11:30:04.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.111 7f91feffd700 1 --2- 172.21.2.101:0/2422531134 >> v2:172.21.2.108:3300/0 conn(0x7f92001b0870 0x7f92000f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:04.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.111 7f91fffff700 1 -- 172.21.2.101:0/2422531134 >> v2:172.21.2.108:3300/0 conn(0x7f92001b0870 msgr2=0x7f92000f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:04.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.111 7f91fffff700 1 --2- 172.21.2.101:0/2422531134 >> v2:172.21.2.108:3300/0 conn(0x7f92001b0870 0x7f92000f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:04.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.111 7f91fffff700 1 -- 172.21.2.101:0/2422531134 >> v1:172.21.2.101:6789/0 conn(0x7f92001b0470 legacy=0x7f92000ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:04.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.111 7f91fffff700 1 -- 172.21.2.101:0/2422531134 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92000f0820 con 0x7f92000efe00 2021-02-10T11:30:04.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.111 7f91fffff700 1 --2- 172.21.2.101:0/2422531134 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f92000efe00 0x7f92000f0200 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7f91f8006f00 tx=0x7f91f8009f10).ready entity=mon.0 client_cookie=8e7971df5462a6cb server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:04.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.111 7f91fe7fc700 1 -- 172.21.2.101:0/2422531134 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f91f800b3b0 con 0x7f92000efe00 2021-02-10T11:30:04.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.111 7f91fe7fc700 1 -- 172.21.2.101:0/2422531134 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f800b3b0 con 0x7f92000efe00 2021-02-10T11:30:04.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.111 7f91fe7fc700 1 -- 172.21.2.101:0/2422531134 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f91f800b630 con 0x7f92000efe00 2021-02-10T11:30:04.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.111 7f9205e2c700 1 -- 172.21.2.101:0/2422531134 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f92000efe00 msgr2=0x7f92000f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:04.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.111 7f9205e2c700 1 --2- 172.21.2.101:0/2422531134 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f92000efe00 0x7f92000f0200 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7f91f8006f00 tx=0x7f91f8009f10).stop 2021-02-10T11:30:04.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.112 7f9205e2c700 1 -- 172.21.2.101:0/2422531134 shutdown_connections 2021-02-10T11:30:04.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.112 7f9205e2c700 1 --2- 172.21.2.101:0/2422531134 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f92000efe00 0x7f92000f0200 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:04.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.112 7f9205e2c700 1 --2- 172.21.2.101:0/2422531134 >> v2:172.21.2.108:3300/0 conn(0x7f92001b0870 0x7f92000f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:04.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.112 7f9205e2c700 1 -- 172.21.2.101:0/2422531134 >> 172.21.2.101:0/2422531134 conn(0x7f92000ce3e0 msgr2=0x7f92000e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:04.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.112 7f9205e2c700 1 -- 172.21.2.101:0/2422531134 shutdown_connections 2021-02-10T11:30:04.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.112 7f9205e2c700 1 -- 172.21.2.101:0/2422531134 wait complete. 2021-02-10T11:30:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.126 7f9205e2c700 1 Processor -- start 2021-02-10T11:30:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.127 7f9205e2c700 1 -- start start 2021-02-10T11:30:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.127 7f9205e2c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f920008ece0 0x7f92000a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.127 7f9205e2c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f92000a6860 0x7f92000a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.127 7f9205e2c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f92000a7280 0x7f92000a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.127 7f9205e2c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92000a5cc0 con 0x7f920008ece0 2021-02-10T11:30:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.127 7f9205e2c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920008f0e0 con 0x7f92000a7280 2021-02-10T11:30:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.127 7f9205e2c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920008f2c0 con 0x7f92000a6860 2021-02-10T11:30:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.127 7f91ff7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f920008ece0 0x7f92000a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.127 7f91ff7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f920008ece0 0x7f92000a6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48520/0 (socket says 172.21.2.101:48520) 2021-02-10T11:30:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.127 7f91ff7fe700 1 -- 172.21.2.101:0/3124792081 learned_addr learned my addr 172.21.2.101:0/3124792081 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.127 7f91fffff700 1 --2- 172.21.2.101:0/3124792081 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f92000a7280 0x7f92000a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.127 7f91feffd700 1 --2- 172.21.2.101:0/3124792081 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f92000a6860 0x7f92000a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.128 7f91ff7fe700 1 -- 172.21.2.101:0/3124792081 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f92000a6860 msgr2=0x7f92000a6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.128 7f91ff7fe700 1 --2- 172.21.2.101:0/3124792081 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f92000a6860 0x7f92000a6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.128 7f91ff7fe700 1 -- 172.21.2.101:0/3124792081 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f92000a7280 msgr2=0x7f92000a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.128 7f91ff7fe700 1 --2- 172.21.2.101:0/3124792081 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f92000a7280 0x7f92000a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.128 7f91ff7fe700 1 -- 172.21.2.101:0/3124792081 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92000a4bb0 con 0x7f920008ece0 2021-02-10T11:30:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.128 7f91ff7fe700 1 --2- 172.21.2.101:0/3124792081 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f920008ece0 0x7f92000a6240 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f91e800a4c0 tx=0x7f91e800a860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.128 7f91f7fff700 1 -- 172.21.2.101:0/3124792081 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f91e800ac50 con 0x7f920008ece0 2021-02-10T11:30:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.128 7f91f7fff700 1 -- 172.21.2.101:0/3124792081 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e8004140 con 0x7f920008ece0 2021-02-10T11:30:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.128 7f91f7fff700 1 -- 172.21.2.101:0/3124792081 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f91e8004660 con 0x7f920008ece0 2021-02-10T11:30:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.128 7f9205e2c700 1 -- 172.21.2.101:0/3124792081 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92000a4bb0 con 0x7f920008ece0 2021-02-10T11:30:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.128 7f9205e2c700 1 -- 172.21.2.101:0/3124792081 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92000a4bb0 con 0x7f920008ece0 2021-02-10T11:30:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.129 7f91f7fff700 1 -- 172.21.2.101:0/3124792081 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f91e8000da0 con 0x7f920008ece0 2021-02-10T11:30:04.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.129 7f91f7fff700 1 --2- 172.21.2.101:0/3124792081 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f91e40186d0 0x7f91e401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:04.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.129 7f91f7fff700 1 -- 172.21.2.101:0/3124792081 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f91e8032680 con 0x7f920008ece0 2021-02-10T11:30:04.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.129 7f91feffd700 1 --2- 172.21.2.101:0/3124792081 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f91e40186d0 0x7f91e401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:04.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.129 7f91feffd700 1 --2- 172.21.2.101:0/3124792081 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f91e40186d0 0x7f91e401abd0 crc :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:04.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.160 7f9205e2c700 1 -- 172.21.2.101:0/3124792081 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9200105ae0 con 0x7f920008ece0 2021-02-10T11:30:04.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.163 7f91f7fff700 1 -- 172.21.2.101:0/3124792081 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f91e80009b0 con 0x7f920008ece0 2021-02-10T11:30:04.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.295 7f9205e2c700 1 -- 172.21.2.101:0/3124792081 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839245", "target": ["mgr", ""]}) v1 -- 0x7f92000f0b50 con 0x7f91e40186d0 2021-02-10T11:30:04.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.305 7f91f7fff700 1 -- 172.21.2.101:0/3124792081 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f91f00009b0 con 0x7f91e40186d0 2021-02-10T11:30:04.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.311 7f9205e2c700 1 -- 172.21.2.101:0/3124792081 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f91e40186d0 msgr2=0x7f91e401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:04.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.311 7f9205e2c700 1 --2- 172.21.2.101:0/3124792081 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f91e40186d0 0x7f91e401abd0 crc :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:04.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.311 7f9205e2c700 1 -- 172.21.2.101:0/3124792081 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f920008ece0 msgr2=0x7f92000a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:04.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.311 7f9205e2c700 1 --2- 172.21.2.101:0/3124792081 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f920008ece0 0x7f92000a6240 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f91e800a4c0 tx=0x7f91e800a860).stop 2021-02-10T11:30:04.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.311 7f9205e2c700 1 -- 172.21.2.101:0/3124792081 shutdown_connections 2021-02-10T11:30:04.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.311 7f9205e2c700 1 --2- 172.21.2.101:0/3124792081 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f91e40186d0 0x7f91e401abd0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:04.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.311 7f9205e2c700 1 --2- 172.21.2.101:0/3124792081 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f92000a6860 0x7f92000a6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:04.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.311 7f9205e2c700 1 --2- 172.21.2.101:0/3124792081 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f92000a7280 0x7f92000a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:04.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.311 7f9205e2c700 1 --2- 172.21.2.101:0/3124792081 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f920008ece0 0x7f92000a6240 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:04.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.311 7f9205e2c700 1 -- 172.21.2.101:0/3124792081 >> 172.21.2.101:0/3124792081 conn(0x7f920008ffe0 msgr2=0x7f92000bf050 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:04.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.311 7f9205e2c700 1 -- 172.21.2.101:0/3124792081 shutdown_connections 2021-02-10T11:30:04.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.311 7f9205e2c700 1 -- 172.21.2.101:0/3124792081 wait complete. 2021-02-10T11:30:04.331 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839246 2021-02-10T11:30:04.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.733 7fa8ca331700 1 Processor -- start 2021-02-10T11:30:04.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.733 7fa8ca331700 1 -- start start 2021-02-10T11:30:04.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.733 7fa8ca331700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa8c41b0470 0x7fa8c41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:04.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.733 7fa8ca331700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa8c40f20e0 0x7fa8c40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:04.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.733 7fa8ca331700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c40f0b50 con 0x7fa8c40ef610 2021-02-10T11:30:04.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.733 7fa8ca331700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c40f0820 con 0x7fa8c40f20e0 2021-02-10T11:30:04.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.733 7fa8ca331700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c40f24e0 con 0x7fa8c41b0470 2021-02-10T11:30:04.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.734 7fa8c894e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa8c40f20e0 0x7fa8c40f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:04.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.734 7fa8c37fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa8c40ef610 0x7fa8c40efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49022/0 (socket says 172.21.2.101:49022) 2021-02-10T11:30:04.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.734 7fa8c37fe700 1 -- 172.21.2.101:0/3262192264 learned_addr learned my addr 172.21.2.101:0/3262192264 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:04.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.734 7fa8c894e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa8c40f20e0 0x7fa8c40f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48528/0 (socket says 172.21.2.101:48528) 2021-02-10T11:30:04.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.734 7fa8c3fff700 1 --2- 172.21.2.101:0/3262192264 >> v2:172.21.2.108:3300/0 conn(0x7fa8c41b0470 0x7fa8c41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:04.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.734 7fa8c894e700 1 -- 172.21.2.101:0/3262192264 >> v2:172.21.2.108:3300/0 conn(0x7fa8c41b0470 msgr2=0x7fa8c41b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:04.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.734 7fa8c894e700 1 --2- 172.21.2.101:0/3262192264 >> v2:172.21.2.108:3300/0 conn(0x7fa8c41b0470 0x7fa8c41b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:04.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.734 7fa8c894e700 1 -- 172.21.2.101:0/3262192264 >> v1:172.21.2.101:6789/0 conn(0x7fa8c40ef610 legacy=0x7fa8c40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:04.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.734 7fa8c2ffd700 1 -- 172.21.2.101:0/3262192264 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567848575 0 0) 0x7fa8b8001180 con 0x7fa8c40ef610 2021-02-10T11:30:04.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.734 7fa8c894e700 1 -- 172.21.2.101:0/3262192264 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c40e52e0 con 0x7fa8c40f20e0 2021-02-10T11:30:04.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.734 7fa8c894e700 1 --2- 172.21.2.101:0/3262192264 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8c40f20e0 0x7fa8c40f0200 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7fa8bc006e30 tx=0x7fa8bc009de0).ready entity=mon.0 client_cookie=16fd675dd1255468 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:04.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.735 7fa8c2ffd700 1 -- 172.21.2.101:0/3262192264 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa8bc00b300 con 0x7fa8c40f20e0 2021-02-10T11:30:04.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.735 7fa8c2ffd700 1 -- 172.21.2.101:0/3262192264 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8bc00b580 con 0x7fa8c40f20e0 2021-02-10T11:30:04.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.735 7fa8c2ffd700 1 -- 172.21.2.101:0/3262192264 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa8bc00baa0 con 0x7fa8c40f20e0 2021-02-10T11:30:04.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.735 7fa8ca331700 1 -- 172.21.2.101:0/3262192264 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8c40f20e0 msgr2=0x7fa8c40f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:04.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.735 7fa8ca331700 1 --2- 172.21.2.101:0/3262192264 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8c40f20e0 0x7fa8c40f0200 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7fa8bc006e30 tx=0x7fa8bc009de0).stop 2021-02-10T11:30:04.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.735 7fa8ca331700 1 -- 172.21.2.101:0/3262192264 shutdown_connections 2021-02-10T11:30:04.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.735 7fa8ca331700 1 --2- 172.21.2.101:0/3262192264 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8c40f20e0 0x7fa8c40f0200 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:04.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.735 7fa8ca331700 1 --2- 172.21.2.101:0/3262192264 >> v2:172.21.2.108:3300/0 conn(0x7fa8c41b0470 0x7fa8c41b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:04.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.735 7fa8ca331700 1 -- 172.21.2.101:0/3262192264 >> 172.21.2.101:0/3262192264 conn(0x7fa8c40ce3e0 msgr2=0x7fa8c40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:04.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.735 7fa8ca331700 1 -- 172.21.2.101:0/3262192264 shutdown_connections 2021-02-10T11:30:04.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.735 7fa8ca331700 1 -- 172.21.2.101:0/3262192264 wait complete. 2021-02-10T11:30:04.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.750 7fa8ca331700 1 Processor -- start 2021-02-10T11:30:04.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.750 7fa8ca331700 1 -- start start 2021-02-10T11:30:04.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.750 7fa8ca331700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8c40a0420 0x7fa8c40a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:04.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.750 7fa8ca331700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8c4189d00 0x7fa8c418a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:04.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.750 7fa8ca331700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8c418a720 0x7fa8c40b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:04.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.750 7fa8ca331700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c4189800 con 0x7fa8c4189d00 2021-02-10T11:30:04.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.750 7fa8ca331700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c418ab20 con 0x7fa8c40a0420 2021-02-10T11:30:04.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.750 7fa8ca331700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c40a1140 con 0x7fa8c418a720 2021-02-10T11:30:04.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.750 7fa8c37fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8c4189d00 0x7fa8c418a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:04.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.751 7fa8c37fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8c4189d00 0x7fa8c418a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48530/0 (socket says 172.21.2.101:48530) 2021-02-10T11:30:04.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.751 7fa8c37fe700 1 -- 172.21.2.101:0/1392307962 learned_addr learned my addr 172.21.2.101:0/1392307962 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:04.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.751 7fa8c894e700 1 --2- 172.21.2.101:0/1392307962 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8c418a720 0x7fa8c40b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:04.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.751 7fa8c3fff700 1 --2- 172.21.2.101:0/1392307962 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8c40a0420 0x7fa8c40a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:04.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.751 7fa8c894e700 1 -- 172.21.2.101:0/1392307962 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8c40a0420 msgr2=0x7fa8c40a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:04.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.751 7fa8c894e700 1 --2- 172.21.2.101:0/1392307962 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8c40a0420 0x7fa8c40a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:04.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.751 7fa8c894e700 1 -- 172.21.2.101:0/1392307962 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8c4189d00 msgr2=0x7fa8c418a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:04.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.751 7fa8c894e700 1 --2- 172.21.2.101:0/1392307962 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8c4189d00 0x7fa8c418a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:04.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.751 7fa8c894e700 1 -- 172.21.2.101:0/1392307962 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c40b8200 con 0x7fa8c418a720 2021-02-10T11:30:04.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.751 7fa8c37fe700 1 --2- 172.21.2.101:0/1392307962 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8c4189d00 0x7fa8c418a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:04.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.751 7fa8c894e700 1 --2- 172.21.2.101:0/1392307962 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8c418a720 0x7fa8c40b7be0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fa8bc0098f0 tx=0x7fa8bc00bc40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:04.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.751 7fa8c0ff9700 1 -- 172.21.2.101:0/1392307962 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa8bc00bd80 con 0x7fa8c418a720 2021-02-10T11:30:04.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.751 7fa8c0ff9700 1 -- 172.21.2.101:0/1392307962 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8bc005820 con 0x7fa8c418a720 2021-02-10T11:30:04.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.751 7fa8ca331700 1 -- 172.21.2.101:0/1392307962 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c40b8200 con 0x7fa8c418a720 2021-02-10T11:30:04.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.751 7fa8c0ff9700 1 -- 172.21.2.101:0/1392307962 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa8bc005cb0 con 0x7fa8c418a720 2021-02-10T11:30:04.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.751 7fa8ca331700 1 -- 172.21.2.101:0/1392307962 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c40b8200 con 0x7fa8c418a720 2021-02-10T11:30:04.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.752 7fa8c0ff9700 1 -- 172.21.2.101:0/1392307962 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa8bc005a00 con 0x7fa8c418a720 2021-02-10T11:30:04.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.752 7fa8c0ff9700 1 --2- 172.21.2.101:0/1392307962 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8a4018730 0x7fa8a401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:04.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.752 7fa8c0ff9700 1 -- 172.21.2.101:0/1392307962 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa8bc032950 con 0x7fa8c418a720 2021-02-10T11:30:04.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.752 7fa8c3fff700 1 --2- 172.21.2.101:0/1392307962 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8a4018730 0x7fa8a401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:04.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.752 7fa8c3fff700 1 --2- 172.21.2.101:0/1392307962 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8a4018730 0x7fa8a401ac10 crc :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:04.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.783 7fa8ca331700 1 -- 172.21.2.101:0/1392307962 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c418ab20 con 0x7fa8c418a720 2021-02-10T11:30:04.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.786 7fa8c0ff9700 1 -- 172.21.2.101:0/1392307962 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa8bc00bd80 con 0x7fa8c418a720 2021-02-10T11:30:04.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.915 7fa8ca331700 1 -- 172.21.2.101:0/1392307962 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839246", "target": ["mgr", ""]}) v1 -- 0x7fa8c418ab20 con 0x7fa8a4018730 2021-02-10T11:30:04.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.926 7fa8c0ff9700 1 -- 172.21.2.101:0/1392307962 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fa8b4006a70 con 0x7fa8a4018730 2021-02-10T11:30:04.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.932 7fa8ca331700 1 -- 172.21.2.101:0/1392307962 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8a4018730 msgr2=0x7fa8a401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:04.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.932 7fa8ca331700 1 --2- 172.21.2.101:0/1392307962 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8a4018730 0x7fa8a401ac10 crc :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:04.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.932 7fa8ca331700 1 -- 172.21.2.101:0/1392307962 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8c418a720 msgr2=0x7fa8c40b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:04.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.932 7fa8ca331700 1 --2- 172.21.2.101:0/1392307962 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8c418a720 0x7fa8c40b7be0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fa8bc0098f0 tx=0x7fa8bc00bc40).stop 2021-02-10T11:30:04.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.932 7fa8ca331700 1 -- 172.21.2.101:0/1392307962 shutdown_connections 2021-02-10T11:30:04.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.932 7fa8ca331700 1 --2- 172.21.2.101:0/1392307962 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8a4018730 0x7fa8a401ac10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:04.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.932 7fa8ca331700 1 --2- 172.21.2.101:0/1392307962 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8c418a720 0x7fa8c40b7be0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:04.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.932 7fa8ca331700 1 --2- 172.21.2.101:0/1392307962 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8c4189d00 0x7fa8c418a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:04.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.932 7fa8ca331700 1 --2- 172.21.2.101:0/1392307962 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8c40a0420 0x7fa8c40a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:04.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.932 7fa8ca331700 1 -- 172.21.2.101:0/1392307962 >> 172.21.2.101:0/1392307962 conn(0x7fa8c4094e10 msgr2=0x7fa8c40a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:04.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.932 7fa8ca331700 1 -- 172.21.2.101:0/1392307962 shutdown_connections 2021-02-10T11:30:04.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:04.932 7fa8ca331700 1 -- 172.21.2.101:0/1392307962 wait complete. 2021-02-10T11:30:04.952 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839247 2021-02-10T11:30:05.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.357 7f79371df700 1 Processor -- start 2021-02-10T11:30:05.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.357 7f79371df700 1 -- start start 2021-02-10T11:30:05.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.357 7f79371df700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f79301b0470 0x7f79301b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:05.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.357 7f79371df700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f79300f20e0 0x7f79300f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:05.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.357 7f79371df700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79300f0b50 con 0x7f79300ef610 2021-02-10T11:30:05.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.357 7f79371df700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79300f0820 con 0x7f79300f20e0 2021-02-10T11:30:05.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.357 7f79371df700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79300f24e0 con 0x7f79301b0470 2021-02-10T11:30:05.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.358 7f792ffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f79300ef610 0x7f79300efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49034/0 (socket says 172.21.2.101:49034) 2021-02-10T11:30:05.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.358 7f792ffff700 1 -- 172.21.2.101:0/2761709623 learned_addr learned my addr 172.21.2.101:0/2761709623 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:05.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.358 7f79357fc700 1 --2- 172.21.2.101:0/2761709623 >> v2:172.21.2.101:3300/0 conn(0x7f79300f20e0 0x7f79300f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:05.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.358 7f7934ffb700 1 --2- 172.21.2.101:0/2761709623 >> v2:172.21.2.108:3300/0 conn(0x7f79301b0470 0x7f79301b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:05.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.358 7f79357fc700 1 -- 172.21.2.101:0/2761709623 >> v2:172.21.2.108:3300/0 conn(0x7f79301b0470 msgr2=0x7f79301b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:05.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.358 7f79357fc700 1 --2- 172.21.2.101:0/2761709623 >> v2:172.21.2.108:3300/0 conn(0x7f79301b0470 0x7f79301b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:05.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.358 7f79357fc700 1 -- 172.21.2.101:0/2761709623 >> v1:172.21.2.101:6789/0 conn(0x7f79300ef610 legacy=0x7f79300efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:05.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.358 7f792f7fe700 1 -- 172.21.2.101:0/2761709623 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2128862938 0 0) 0x7f79240010b0 con 0x7f79300ef610 2021-02-10T11:30:05.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.358 7f79357fc700 1 -- 172.21.2.101:0/2761709623 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79300e52e0 con 0x7f79300f20e0 2021-02-10T11:30:05.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.358 7f79357fc700 1 --2- 172.21.2.101:0/2761709623 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f79300f20e0 0x7f79300f0200 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f7920006ab0 tx=0x7f7920009ac0).ready entity=mon.0 client_cookie=33c8694885cf40df server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:05.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.358 7f792f7fe700 1 -- 172.21.2.101:0/2761709623 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f79200066d0 con 0x7f79300f20e0 2021-02-10T11:30:05.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.358 7f792f7fe700 1 -- 172.21.2.101:0/2761709623 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79200022e0 con 0x7f79300f20e0 2021-02-10T11:30:05.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.358 7f792f7fe700 1 -- 172.21.2.101:0/2761709623 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f79200027c0 con 0x7f79300f20e0 2021-02-10T11:30:05.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.358 7f79371df700 1 -- 172.21.2.101:0/2761709623 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f79300f20e0 msgr2=0x7f79300f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:05.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.358 7f79371df700 1 --2- 172.21.2.101:0/2761709623 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f79300f20e0 0x7f79300f0200 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f7920006ab0 tx=0x7f7920009ac0).stop 2021-02-10T11:30:05.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.359 7f79371df700 1 -- 172.21.2.101:0/2761709623 shutdown_connections 2021-02-10T11:30:05.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.359 7f79371df700 1 --2- 172.21.2.101:0/2761709623 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f79300f20e0 0x7f79300f0200 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:05.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.359 7f79371df700 1 --2- 172.21.2.101:0/2761709623 >> v2:172.21.2.108:3300/0 conn(0x7f79301b0470 0x7f79301b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:05.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.359 7f79371df700 1 -- 172.21.2.101:0/2761709623 >> 172.21.2.101:0/2761709623 conn(0x7f79300ce3e0 msgr2=0x7f79300e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:05.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.359 7f79371df700 1 -- 172.21.2.101:0/2761709623 shutdown_connections 2021-02-10T11:30:05.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.359 7f79371df700 1 -- 172.21.2.101:0/2761709623 wait complete. 2021-02-10T11:30:05.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.374 7f79371df700 1 Processor -- start 2021-02-10T11:30:05.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.374 7f79371df700 1 -- start start 2021-02-10T11:30:05.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.374 7f79371df700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f79300c28d0 0x7f79300c2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:05.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.374 7f79371df700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f793008e690 0x7f793008ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:05.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.374 7f79371df700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f79300bc380 0x7f79300bc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:05.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.374 7f79371df700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793008f340 con 0x7f79300bc380 2021-02-10T11:30:05.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.374 7f79371df700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793008f0b0 con 0x7f79300c28d0 2021-02-10T11:30:05.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.374 7f79371df700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79300c3400 con 0x7f793008e690 2021-02-10T11:30:05.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.375 7f79357fc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f79300bc380 0x7f79300bc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:05.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.375 7f7934ffb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f79300c28d0 0x7f79300c2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:05.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.375 7f79357fc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f79300bc380 0x7f79300bc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48548/0 (socket says 172.21.2.101:48548) 2021-02-10T11:30:05.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.375 7f79357fc700 1 -- 172.21.2.101:0/395953482 learned_addr learned my addr 172.21.2.101:0/395953482 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:05.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.375 7f792ffff700 1 --2- 172.21.2.101:0/395953482 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f793008e690 0x7f793008ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:05.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.375 7f7934ffb700 1 -- 172.21.2.101:0/395953482 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f793008e690 msgr2=0x7f793008ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:05.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.375 7f7934ffb700 1 --2- 172.21.2.101:0/395953482 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f793008e690 0x7f793008ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:05.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.375 7f7934ffb700 1 -- 172.21.2.101:0/395953482 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f79300bc380 msgr2=0x7f79300bc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:05.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.375 7f7934ffb700 1 --2- 172.21.2.101:0/395953482 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f79300bc380 0x7f79300bc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:05.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.375 7f7934ffb700 1 -- 172.21.2.101:0/395953482 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79300bcda0 con 0x7f79300c28d0 2021-02-10T11:30:05.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.375 7f7934ffb700 1 --2- 172.21.2.101:0/395953482 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f79300c28d0 0x7f79300c2cd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f792800fc90 tx=0x7f792800fd80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:05.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.375 7f792d7fa700 1 -- 172.21.2.101:0/395953482 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f792800d030 con 0x7f79300c28d0 2021-02-10T11:30:05.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.375 7f792d7fa700 1 -- 172.21.2.101:0/395953482 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792800d230 con 0x7f79300c28d0 2021-02-10T11:30:05.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.375 7f792d7fa700 1 -- 172.21.2.101:0/395953482 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7928006430 con 0x7f79300c28d0 2021-02-10T11:30:05.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.375 7f79371df700 1 -- 172.21.2.101:0/395953482 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79300bcda0 con 0x7f79300c28d0 2021-02-10T11:30:05.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.375 7f79371df700 1 -- 172.21.2.101:0/395953482 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79300bcda0 con 0x7f79300c28d0 2021-02-10T11:30:05.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.376 7f792d7fa700 1 -- 172.21.2.101:0/395953482 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7928006240 con 0x7f79300c28d0 2021-02-10T11:30:05.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.376 7f792d7fa700 1 --2- 172.21.2.101:0/395953482 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79100186c0 0x7f791001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:05.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.376 7f792d7fa700 1 -- 172.21.2.101:0/395953482 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7928036c70 con 0x7f79300c28d0 2021-02-10T11:30:05.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.376 7f792ffff700 1 --2- 172.21.2.101:0/395953482 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79100186c0 0x7f791001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:05.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.376 7f792ffff700 1 --2- 172.21.2.101:0/395953482 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79100186c0 0x7f791001abc0 crc :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:05.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.406 7f79371df700 1 -- 172.21.2.101:0/395953482 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7930105ae0 con 0x7f79300c28d0 2021-02-10T11:30:05.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.409 7f792d7fa700 1 -- 172.21.2.101:0/395953482 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7928006030 con 0x7f79300c28d0 2021-02-10T11:30:05.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.537 7f79371df700 1 -- 172.21.2.101:0/395953482 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839247", "target": ["mgr", ""]}) v1 -- 0x7f793008f0b0 con 0x7f79100186c0 2021-02-10T11:30:05.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.548 7f792d7fa700 1 -- 172.21.2.101:0/395953482 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f79240009b0 con 0x7f79100186c0 2021-02-10T11:30:05.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.554 7f79371df700 1 -- 172.21.2.101:0/395953482 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79100186c0 msgr2=0x7f791001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:05.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.554 7f79371df700 1 --2- 172.21.2.101:0/395953482 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79100186c0 0x7f791001abc0 crc :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:05.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.554 7f79371df700 1 -- 172.21.2.101:0/395953482 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f79300c28d0 msgr2=0x7f79300c2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:05.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.554 7f79371df700 1 --2- 172.21.2.101:0/395953482 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f79300c28d0 0x7f79300c2cd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f792800fc90 tx=0x7f792800fd80).stop 2021-02-10T11:30:05.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.554 7f79371df700 1 -- 172.21.2.101:0/395953482 shutdown_connections 2021-02-10T11:30:05.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.554 7f79371df700 1 --2- 172.21.2.101:0/395953482 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79100186c0 0x7f791001abc0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:05.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.554 7f79371df700 1 --2- 172.21.2.101:0/395953482 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f793008e690 0x7f793008ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:05.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.554 7f79371df700 1 --2- 172.21.2.101:0/395953482 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f79300bc380 0x7f79300bc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:05.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.554 7f79371df700 1 --2- 172.21.2.101:0/395953482 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f79300c28d0 0x7f79300c2cd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:05.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.554 7f79371df700 1 -- 172.21.2.101:0/395953482 >> 172.21.2.101:0/395953482 conn(0x7f79300bbf60 msgr2=0x7f7930105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:05.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.555 7f79371df700 1 -- 172.21.2.101:0/395953482 shutdown_connections 2021-02-10T11:30:05.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.555 7f79371df700 1 -- 172.21.2.101:0/395953482 wait complete. 2021-02-10T11:30:05.575 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839248 2021-02-10T11:30:06.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.980 7fc1f7833700 1 Processor -- start 2021-02-10T11:30:06.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.980 7fc1f7833700 1 -- start start 2021-02-10T11:30:06.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.981 7fc1f7833700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc1f01b0870 0x7fc1f00f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:06.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.981 7fc1f7833700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc1f00efe00 0x7fc1f00f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:06.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.981 7fc1f7833700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f01b0c70 con 0x7fc1f01b0470 2021-02-10T11:30:06.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.981 7fc1f7833700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f00e5ea0 con 0x7fc1f01b0870 2021-02-10T11:30:06.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.981 7fc1f7833700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f00f0b50 con 0x7fc1f00efe00 2021-02-10T11:30:06.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.981 7fc1f4e4e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc1f01b0870 0x7fc1f00f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:06.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.981 7fc1f564f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc1f01b0470 0x7fc1f00ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49046/0 (socket says 172.21.2.101:49046) 2021-02-10T11:30:06.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.981 7fc1f564f700 1 -- 172.21.2.101:0/3739675391 learned_addr learned my addr 172.21.2.101:0/3739675391 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:06.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.981 7fc1f5e50700 1 --2- 172.21.2.101:0/3739675391 >> v2:172.21.2.108:3300/0 conn(0x7fc1f00efe00 0x7fc1f00f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:06.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.981 7fc1f4e4e700 1 -- 172.21.2.101:0/3739675391 >> v2:172.21.2.108:3300/0 conn(0x7fc1f00efe00 msgr2=0x7fc1f00f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:06.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.981 7fc1f4e4e700 1 --2- 172.21.2.101:0/3739675391 >> v2:172.21.2.108:3300/0 conn(0x7fc1f00efe00 0x7fc1f00f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:06.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.981 7fc1f4e4e700 1 -- 172.21.2.101:0/3739675391 >> v1:172.21.2.101:6789/0 conn(0x7fc1f01b0470 legacy=0x7fc1f00ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:06.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.982 7fc1e7fff700 1 -- 172.21.2.101:0/3739675391 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1273672917 0 0) 0x7fc1ec000ce0 con 0x7fc1f01b0470 2021-02-10T11:30:06.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.982 7fc1f4e4e700 1 -- 172.21.2.101:0/3739675391 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f00f0820 con 0x7fc1f01b0870 2021-02-10T11:30:06.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.982 7fc1f5e50700 1 --2- 172.21.2.101:0/3739675391 >> v2:172.21.2.108:3300/0 conn(0x7fc1f00efe00 0x7fc1f00f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:06.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.982 7fc1f4e4e700 1 --2- 172.21.2.101:0/3739675391 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1f01b0870 0x7fc1f00f20e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7fc1e0006650 tx=0x7fc1e0009a80).ready entity=mon.0 client_cookie=a961325832de505e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:06.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.982 7fc1e7fff700 1 -- 172.21.2.101:0/3739675391 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc1e00009b0 con 0x7fc1f01b0870 2021-02-10T11:30:06.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.982 7fc1e7fff700 1 -- 172.21.2.101:0/3739675391 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e000c0c0 con 0x7fc1f01b0870 2021-02-10T11:30:06.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.982 7fc1e7fff700 1 -- 172.21.2.101:0/3739675391 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc1e000c510 con 0x7fc1f01b0870 2021-02-10T11:30:06.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.982 7fc1f7833700 1 -- 172.21.2.101:0/3739675391 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1f01b0870 msgr2=0x7fc1f00f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:06.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.982 7fc1f7833700 1 --2- 172.21.2.101:0/3739675391 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1f01b0870 0x7fc1f00f20e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7fc1e0006650 tx=0x7fc1e0009a80).stop 2021-02-10T11:30:06.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.983 7fc1f7833700 1 -- 172.21.2.101:0/3739675391 shutdown_connections 2021-02-10T11:30:06.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.983 7fc1f7833700 1 --2- 172.21.2.101:0/3739675391 >> v2:172.21.2.108:3300/0 conn(0x7fc1f00efe00 0x7fc1f00f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:06.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.983 7fc1f7833700 1 --2- 172.21.2.101:0/3739675391 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1f01b0870 0x7fc1f00f20e0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:06.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.983 7fc1f7833700 1 -- 172.21.2.101:0/3739675391 >> 172.21.2.101:0/3739675391 conn(0x7fc1f00ce3e0 msgr2=0x7fc1f00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:06.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.983 7fc1f7833700 1 -- 172.21.2.101:0/3739675391 shutdown_connections 2021-02-10T11:30:06.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.983 7fc1f7833700 1 -- 172.21.2.101:0/3739675391 wait complete. 2021-02-10T11:30:06.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.997 7fc1f7833700 1 Processor -- start 2021-02-10T11:30:06.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.997 7fc1f7833700 1 -- start start 2021-02-10T11:30:06.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.998 7fc1f7833700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc1f00a0420 0x7fc1f00a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:06.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.998 7fc1f7833700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1f0189d00 0x7fc1f018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:06.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.998 7fc1f7833700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc1f018a720 0x7fc1f00b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:06.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.998 7fc1f7833700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f0189800 con 0x7fc1f0189d00 2021-02-10T11:30:06.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.998 7fc1f7833700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f018ab20 con 0x7fc1f018a720 2021-02-10T11:30:06.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.998 7fc1f7833700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f00a1140 con 0x7fc1f00a0420 2021-02-10T11:30:06.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.998 7fc1f4e4e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1f0189d00 0x7fc1f018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:06.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.998 7fc1f564f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc1f00a0420 0x7fc1f00a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:06.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.998 7fc1f4e4e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1f0189d00 0x7fc1f018a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48558/0 (socket says 172.21.2.101:48558) 2021-02-10T11:30:06.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.998 7fc1f4e4e700 1 -- 172.21.2.101:0/831746256 learned_addr learned my addr 172.21.2.101:0/831746256 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:06.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.998 7fc1f5e50700 1 --2- 172.21.2.101:0/831746256 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc1f018a720 0x7fc1f00b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:06.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.998 7fc1f4e4e700 1 -- 172.21.2.101:0/831746256 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc1f00a0420 msgr2=0x7fc1f00a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:06.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.998 7fc1f4e4e700 1 --2- 172.21.2.101:0/831746256 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc1f00a0420 0x7fc1f00a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:06.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.998 7fc1f4e4e700 1 -- 172.21.2.101:0/831746256 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc1f018a720 msgr2=0x7fc1f00b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:06.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.998 7fc1f4e4e700 1 --2- 172.21.2.101:0/831746256 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc1f018a720 0x7fc1f00b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:06.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.998 7fc1f4e4e700 1 -- 172.21.2.101:0/831746256 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f00b8200 con 0x7fc1f0189d00 2021-02-10T11:30:06.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.998 7fc1f5e50700 1 --2- 172.21.2.101:0/831746256 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc1f018a720 0x7fc1f00b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:06.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.999 7fc1f4e4e700 1 --2- 172.21.2.101:0/831746256 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1f0189d00 0x7fc1f018a100 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7fc1e0009650 tx=0x7fc1e000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:06.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.999 7fc1e5ffb700 1 -- 172.21.2.101:0/831746256 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc1e00093a0 con 0x7fc1f0189d00 2021-02-10T11:30:06.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.999 7fc1e5ffb700 1 -- 172.21.2.101:0/831746256 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e000bde0 con 0x7fc1f0189d00 2021-02-10T11:30:06.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.999 7fc1f7833700 1 -- 172.21.2.101:0/831746256 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f00b8200 con 0x7fc1f0189d00 2021-02-10T11:30:06.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.999 7fc1e5ffb700 1 -- 172.21.2.101:0/831746256 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc1e00052e0 con 0x7fc1f0189d00 2021-02-10T11:30:06.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.999 7fc1f7833700 1 -- 172.21.2.101:0/831746256 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f00b8200 con 0x7fc1f0189d00 2021-02-10T11:30:06.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.999 7fc1e5ffb700 1 -- 172.21.2.101:0/831746256 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc1e0006030 con 0x7fc1f0189d00 2021-02-10T11:30:06.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.999 7fc1e5ffb700 1 --2- 172.21.2.101:0/831746256 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1d0018730 0x7fc1d001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:06.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:05.999 7fc1e5ffb700 1 -- 172.21.2.101:0/831746256 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc1e0032660 con 0x7fc1f0189d00 2021-02-10T11:30:06.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.000 7fc1f564f700 1 --2- 172.21.2.101:0/831746256 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1d0018730 0x7fc1d001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:06.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.000 7fc1f564f700 1 --2- 172.21.2.101:0/831746256 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1d0018730 0x7fc1d001ac10 crc :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:06.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.029 7fc1f7833700 1 -- 172.21.2.101:0/831746256 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f018ab20 con 0x7fc1f0189d00 2021-02-10T11:30:06.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.032 7fc1e5ffb700 1 -- 172.21.2.101:0/831746256 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc1e00009c0 con 0x7fc1f0189d00 2021-02-10T11:30:06.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.159 7fc1f7833700 1 -- 172.21.2.101:0/831746256 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839248", "target": ["mgr", ""]}) v1 -- 0x7fc1f018ab20 con 0x7fc1d0018730 2021-02-10T11:30:06.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.169 7fc1e5ffb700 1 -- 172.21.2.101:0/831746256 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fc1ec001200 con 0x7fc1d0018730 2021-02-10T11:30:06.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.176 7fc1f7833700 1 -- 172.21.2.101:0/831746256 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1d0018730 msgr2=0x7fc1d001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:06.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.176 7fc1f7833700 1 --2- 172.21.2.101:0/831746256 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1d0018730 0x7fc1d001ac10 crc :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:06.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.176 7fc1f7833700 1 -- 172.21.2.101:0/831746256 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1f0189d00 msgr2=0x7fc1f018a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:06.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.176 7fc1f7833700 1 --2- 172.21.2.101:0/831746256 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1f0189d00 0x7fc1f018a100 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7fc1e0009650 tx=0x7fc1e000b960).stop 2021-02-10T11:30:06.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.176 7fc1f7833700 1 -- 172.21.2.101:0/831746256 shutdown_connections 2021-02-10T11:30:06.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.176 7fc1f7833700 1 --2- 172.21.2.101:0/831746256 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc1f018a720 0x7fc1f00b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:06.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.176 7fc1f7833700 1 --2- 172.21.2.101:0/831746256 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1f0189d00 0x7fc1f018a100 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:06.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.176 7fc1f7833700 1 --2- 172.21.2.101:0/831746256 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1d0018730 0x7fc1d001ac10 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:06.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.176 7fc1f7833700 1 --2- 172.21.2.101:0/831746256 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc1f00a0420 0x7fc1f00a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:06.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.176 7fc1f7833700 1 -- 172.21.2.101:0/831746256 >> 172.21.2.101:0/831746256 conn(0x7fc1f0094e10 msgr2=0x7fc1f00a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:06.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.176 7fc1f7833700 1 -- 172.21.2.101:0/831746256 shutdown_connections 2021-02-10T11:30:06.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.176 7fc1f7833700 1 -- 172.21.2.101:0/831746256 wait complete. 2021-02-10T11:30:06.196 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839249 2021-02-10T11:30:06.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.597 7fa146ec6700 1 Processor -- start 2021-02-10T11:30:06.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.597 7fa146ec6700 1 -- start start 2021-02-10T11:30:06.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.598 7fa146ec6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1401b0470 0x7fa1401b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:06.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.598 7fa146ec6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa1400ef610 0x7fa1400efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:06.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.598 7fa146ec6700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa1400f0030 0x7fa1400f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:06.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.598 7fa146ec6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1400e5ea0 con 0x7fa1400ef610 2021-02-10T11:30:06.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.598 7fa146ec6700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1400f0b50 con 0x7fa1400f0030 2021-02-10T11:30:06.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.598 7fa146ec6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1400e52e0 con 0x7fa1401b0470 2021-02-10T11:30:06.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.598 7fa137fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa1400ef610 0x7fa1400efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:06.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.598 7fa137fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa1400ef610 0x7fa1400efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48566/0 (socket says 172.21.2.101:48566) 2021-02-10T11:30:06.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.598 7fa137fff700 1 -- 172.21.2.101:0/785946277 learned_addr learned my addr 172.21.2.101:0/785946277 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:06.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.598 7fa144ce2700 1 --2- 172.21.2.101:0/785946277 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1401b0470 0x7fa1401b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:06.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.598 7fa1454e3700 1 --2- 172.21.2.101:0/785946277 >> v2:172.21.2.108:3300/0 conn(0x7fa1400f0030 0x7fa1400f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:06.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.598 7fa137fff700 1 -- 172.21.2.101:0/785946277 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1401b0470 msgr2=0x7fa1401b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:06.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.598 7fa137fff700 1 --2- 172.21.2.101:0/785946277 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1401b0470 0x7fa1401b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:06.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.598 7fa137fff700 1 -- 172.21.2.101:0/785946277 >> v2:172.21.2.108:3300/0 conn(0x7fa1400f0030 msgr2=0x7fa1400f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:06.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.598 7fa137fff700 1 --2- 172.21.2.101:0/785946277 >> v2:172.21.2.108:3300/0 conn(0x7fa1400f0030 0x7fa1400f0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:06.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.598 7fa137fff700 1 -- 172.21.2.101:0/785946277 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1400f10d0 con 0x7fa1400ef610 2021-02-10T11:30:06.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.598 7fa1454e3700 1 --2- 172.21.2.101:0/785946277 >> v2:172.21.2.108:3300/0 conn(0x7fa1400f0030 0x7fa1400f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:30:06.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.598 7fa137fff700 1 --2- 172.21.2.101:0/785946277 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1400ef610 0x7fa1400efa10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7fa13c006700 tx=0x7fa13c009b80).ready entity=mon.0 client_cookie=369c7eeede31b967 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:06.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.599 7fa1377fe700 1 -- 172.21.2.101:0/785946277 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa13c00c0c0 con 0x7fa1400ef610 2021-02-10T11:30:06.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.599 7fa1377fe700 1 -- 172.21.2.101:0/785946277 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c00c400 con 0x7fa1400ef610 2021-02-10T11:30:06.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.599 7fa1377fe700 1 -- 172.21.2.101:0/785946277 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa13c00c8e0 con 0x7fa1400ef610 2021-02-10T11:30:06.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.599 7fa146ec6700 1 -- 172.21.2.101:0/785946277 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1400ef610 msgr2=0x7fa1400efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:06.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.599 7fa146ec6700 1 --2- 172.21.2.101:0/785946277 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1400ef610 0x7fa1400efa10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7fa13c006700 tx=0x7fa13c009b80).stop 2021-02-10T11:30:06.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.599 7fa146ec6700 1 -- 172.21.2.101:0/785946277 shutdown_connections 2021-02-10T11:30:06.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.599 7fa146ec6700 1 --2- 172.21.2.101:0/785946277 >> v2:172.21.2.108:3300/0 conn(0x7fa1400f0030 0x7fa1400f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:06.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.599 7fa146ec6700 1 --2- 172.21.2.101:0/785946277 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1400ef610 0x7fa1400efa10 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:06.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.599 7fa146ec6700 1 --2- 172.21.2.101:0/785946277 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1401b0470 0x7fa1401b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:06.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.599 7fa146ec6700 1 -- 172.21.2.101:0/785946277 >> 172.21.2.101:0/785946277 conn(0x7fa1400ce3e0 msgr2=0x7fa1400e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:06.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.599 7fa146ec6700 1 -- 172.21.2.101:0/785946277 shutdown_connections 2021-02-10T11:30:06.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.599 7fa146ec6700 1 -- 172.21.2.101:0/785946277 wait complete. 2021-02-10T11:30:06.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.614 7fa146ec6700 1 Processor -- start 2021-02-10T11:30:06.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.614 7fa146ec6700 1 -- start start 2021-02-10T11:30:06.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.614 7fa146ec6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1400b7c20 0x7fa1400b8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:06.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.615 7fa146ec6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1400b8a30 0x7fa1400c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:06.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.615 7fa146ec6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1400c25a0 0x7fa1400c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:06.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.615 7fa146ec6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1400c1a80 con 0x7fa1400c25a0 2021-02-10T11:30:06.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.615 7fa146ec6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1400b8020 con 0x7fa1400b7c20 2021-02-10T11:30:06.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.615 7fa146ec6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14009b9c0 con 0x7fa1400b8a30 2021-02-10T11:30:06.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.615 7fa1454e3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1400c25a0 0x7fa1400c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:06.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.615 7fa1454e3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1400c25a0 0x7fa1400c29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48570/0 (socket says 172.21.2.101:48570) 2021-02-10T11:30:06.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.615 7fa1454e3700 1 -- 172.21.2.101:0/1130632974 learned_addr learned my addr 172.21.2.101:0/1130632974 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:06.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.615 7fa137fff700 1 --2- 172.21.2.101:0/1130632974 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1400b8a30 0x7fa1400c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:06.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.615 7fa144ce2700 1 --2- 172.21.2.101:0/1130632974 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1400b7c20 0x7fa1400b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:06.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.615 7fa1454e3700 1 -- 172.21.2.101:0/1130632974 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1400b8a30 msgr2=0x7fa1400c1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:06.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.615 7fa1454e3700 1 --2- 172.21.2.101:0/1130632974 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1400b8a30 0x7fa1400c1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:06.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.615 7fa1454e3700 1 -- 172.21.2.101:0/1130632974 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1400b7c20 msgr2=0x7fa1400b8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:06.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.615 7fa1454e3700 1 --2- 172.21.2.101:0/1130632974 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1400b7c20 0x7fa1400b8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:06.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.615 7fa1454e3700 1 -- 172.21.2.101:0/1130632974 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1400b8e30 con 0x7fa1400c25a0 2021-02-10T11:30:06.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.615 7fa1454e3700 1 --2- 172.21.2.101:0/1130632974 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1400c25a0 0x7fa1400c29a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7fa138009920 tx=0x7fa13800bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:06.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.615 7fa1357fa700 1 -- 172.21.2.101:0/1130632974 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa138006240 con 0x7fa1400c25a0 2021-02-10T11:30:06.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.615 7fa1357fa700 1 -- 172.21.2.101:0/1130632974 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa138006500 con 0x7fa1400c25a0 2021-02-10T11:30:06.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.615 7fa146ec6700 1 -- 172.21.2.101:0/1130632974 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1400b8e30 con 0x7fa1400c25a0 2021-02-10T11:30:06.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.615 7fa146ec6700 1 -- 172.21.2.101:0/1130632974 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1400c32b0 con 0x7fa1400c25a0 2021-02-10T11:30:06.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.615 7fa1357fa700 1 -- 172.21.2.101:0/1130632974 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa1380058a0 con 0x7fa1400c25a0 2021-02-10T11:30:06.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.616 7fa1357fa700 1 -- 172.21.2.101:0/1130632974 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa138006030 con 0x7fa1400c25a0 2021-02-10T11:30:06.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.616 7fa1357fa700 1 --2- 172.21.2.101:0/1130632974 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1200186d0 0x7fa12001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:06.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.616 7fa1357fa700 1 -- 172.21.2.101:0/1130632974 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa138032ba0 con 0x7fa1400c25a0 2021-02-10T11:30:06.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.616 7fa144ce2700 1 --2- 172.21.2.101:0/1130632974 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1200186d0 0x7fa12001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:06.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.616 7fa144ce2700 1 --2- 172.21.2.101:0/1130632974 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1200186d0 0x7fa12001abd0 crc :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:06.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.648 7fa146ec6700 1 -- 172.21.2.101:0/1130632974 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa140105b30 con 0x7fa1400c25a0 2021-02-10T11:30:06.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.650 7fa1357fa700 1 -- 172.21.2.101:0/1130632974 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa138006ce0 con 0x7fa1400c25a0 2021-02-10T11:30:06.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.777 7fa146ec6700 1 -- 172.21.2.101:0/1130632974 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839249", "target": ["mgr", ""]}) v1 -- 0x7fa1400c1a00 con 0x7fa1200186d0 2021-02-10T11:30:06.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.788 7fa1357fa700 1 -- 172.21.2.101:0/1130632974 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fa1300009b0 con 0x7fa1200186d0 2021-02-10T11:30:06.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.794 7fa146ec6700 1 -- 172.21.2.101:0/1130632974 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1200186d0 msgr2=0x7fa12001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:06.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.794 7fa146ec6700 1 --2- 172.21.2.101:0/1130632974 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1200186d0 0x7fa12001abd0 crc :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:06.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.794 7fa146ec6700 1 -- 172.21.2.101:0/1130632974 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1400c25a0 msgr2=0x7fa1400c29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:06.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.794 7fa146ec6700 1 --2- 172.21.2.101:0/1130632974 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1400c25a0 0x7fa1400c29a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7fa138009920 tx=0x7fa13800bd40).stop 2021-02-10T11:30:06.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.795 7fa146ec6700 1 -- 172.21.2.101:0/1130632974 shutdown_connections 2021-02-10T11:30:06.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.795 7fa146ec6700 1 --2- 172.21.2.101:0/1130632974 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1200186d0 0x7fa12001abd0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:06.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.795 7fa146ec6700 1 --2- 172.21.2.101:0/1130632974 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1400b8a30 0x7fa1400c1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:06.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.795 7fa146ec6700 1 --2- 172.21.2.101:0/1130632974 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1400c25a0 0x7fa1400c29a0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:06.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.795 7fa146ec6700 1 --2- 172.21.2.101:0/1130632974 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1400b7c20 0x7fa1400b8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:06.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.795 7fa146ec6700 1 -- 172.21.2.101:0/1130632974 >> 172.21.2.101:0/1130632974 conn(0x7fa140094e10 msgr2=0x7fa1400a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:06.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.795 7fa146ec6700 1 -- 172.21.2.101:0/1130632974 shutdown_connections 2021-02-10T11:30:06.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:06.795 7fa146ec6700 1 -- 172.21.2.101:0/1130632974 wait complete. 2021-02-10T11:30:06.815 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839250 2021-02-10T11:30:07.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.222 7fc7c9588700 1 Processor -- start 2021-02-10T11:30:07.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.222 7fc7c9588700 1 -- start start 2021-02-10T11:30:07.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.222 7fc7c9588700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc7c41b0870 0x7fc7c40f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:07.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.222 7fc7c9588700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc7c40efe00 0x7fc7c40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:07.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.222 7fc7c9588700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c41b0c70 con 0x7fc7c41b0470 2021-02-10T11:30:07.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.222 7fc7c9588700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c40e5ea0 con 0x7fc7c41b0870 2021-02-10T11:30:07.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.222 7fc7c9588700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c40f0b50 con 0x7fc7c40efe00 2021-02-10T11:30:07.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.222 7fc7bbfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc7c41b0870 0x7fc7c40f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:07.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.222 7fc7bbfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc7c41b0870 0x7fc7c40f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48578/0 (socket says 172.21.2.101:48578) 2021-02-10T11:30:07.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.223 7fc7bbfff700 1 -- 172.21.2.101:0/3419188143 learned_addr learned my addr 172.21.2.101:0/3419188143 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:07.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.223 7fc7c32bf700 1 --1- 172.21.2.101:0/3419188143 >> v1:172.21.2.101:6789/0 conn(0x7fc7c41b0470 0x7fc7c40ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49072/0 (socket says 172.21.2.101:49072) 2021-02-10T11:30:07.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.223 7fc7c3ac0700 1 --2- 172.21.2.101:0/3419188143 >> v2:172.21.2.108:3300/0 conn(0x7fc7c40efe00 0x7fc7c40f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:07.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.223 7fc7bbfff700 1 -- 172.21.2.101:0/3419188143 >> v2:172.21.2.108:3300/0 conn(0x7fc7c40efe00 msgr2=0x7fc7c40f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:07.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.223 7fc7bbfff700 1 --2- 172.21.2.101:0/3419188143 >> v2:172.21.2.108:3300/0 conn(0x7fc7c40efe00 0x7fc7c40f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:07.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.223 7fc7bbfff700 1 -- 172.21.2.101:0/3419188143 >> v1:172.21.2.101:6789/0 conn(0x7fc7c41b0470 legacy=0x7fc7c40ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:07.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.223 7fc7bbfff700 1 -- 172.21.2.101:0/3419188143 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c40f0820 con 0x7fc7c41b0870 2021-02-10T11:30:07.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.223 7fc7c2abe700 1 -- 172.21.2.101:0/3419188143 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2692157821 0 0) 0x7fc7b4000ce0 con 0x7fc7c41b0470 2021-02-10T11:30:07.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.223 7fc7bbfff700 1 --2- 172.21.2.101:0/3419188143 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c41b0870 0x7fc7c40f20e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7fc7ac006700 tx=0x7fc7ac009b00).ready entity=mon.0 client_cookie=41c6c3a2b991fe8a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:07.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.223 7fc7c2abe700 1 -- 172.21.2.101:0/3419188143 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc7ac00c0c0 con 0x7fc7c41b0870 2021-02-10T11:30:07.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.223 7fc7c2abe700 1 -- 172.21.2.101:0/3419188143 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7ac00c400 con 0x7fc7c41b0870 2021-02-10T11:30:07.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.223 7fc7c2abe700 1 -- 172.21.2.101:0/3419188143 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc7ac00c8e0 con 0x7fc7c41b0870 2021-02-10T11:30:07.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.223 7fc7c9588700 1 -- 172.21.2.101:0/3419188143 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c41b0870 msgr2=0x7fc7c40f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:07.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.223 7fc7c9588700 1 --2- 172.21.2.101:0/3419188143 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c41b0870 0x7fc7c40f20e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7fc7ac006700 tx=0x7fc7ac009b00).stop 2021-02-10T11:30:07.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.224 7fc7c9588700 1 -- 172.21.2.101:0/3419188143 shutdown_connections 2021-02-10T11:30:07.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.224 7fc7c9588700 1 --2- 172.21.2.101:0/3419188143 >> v2:172.21.2.108:3300/0 conn(0x7fc7c40efe00 0x7fc7c40f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:07.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.224 7fc7c9588700 1 --2- 172.21.2.101:0/3419188143 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c41b0870 0x7fc7c40f20e0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:07.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.224 7fc7c9588700 1 -- 172.21.2.101:0/3419188143 >> 172.21.2.101:0/3419188143 conn(0x7fc7c40ce3e0 msgr2=0x7fc7c40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:07.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.224 7fc7c9588700 1 -- 172.21.2.101:0/3419188143 shutdown_connections 2021-02-10T11:30:07.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.224 7fc7c9588700 1 -- 172.21.2.101:0/3419188143 wait complete. 2021-02-10T11:30:07.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.239 7fc7c9588700 1 Processor -- start 2021-02-10T11:30:07.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.239 7fc7c9588700 1 -- start start 2021-02-10T11:30:07.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.239 7fc7c9588700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c410a1e0 0x7fc7c40ab870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:07.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.239 7fc7c9588700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7c40abe90 0x7fc7c40ac290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:07.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.239 7fc7c9588700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c40ad200 0x7fc7c40ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:07.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.239 7fc7c9588700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c410a710 con 0x7fc7c40ad200 2021-02-10T11:30:07.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.239 7fc7c9588700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c40ac8b0 con 0x7fc7c40abe90 2021-02-10T11:30:07.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.239 7fc7c9588700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c40adc20 con 0x7fc7c410a1e0 2021-02-10T11:30:07.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.239 7fc7c3ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c40ad200 0x7fc7c40ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:07.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.240 7fc7bbfff700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7c40abe90 0x7fc7c40ac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:07.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.240 7fc7c3ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c40ad200 0x7fc7c40ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48586/0 (socket says 172.21.2.101:48586) 2021-02-10T11:30:07.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.240 7fc7bbfff700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7c40abe90 0x7fc7c40ac290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:59068/0 (socket says 172.21.2.101:59068) 2021-02-10T11:30:07.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.240 7fc7c3ac0700 1 -- 172.21.2.101:0/1939865813 learned_addr learned my addr 172.21.2.101:0/1939865813 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:07.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.240 7fc7c32bf700 1 --2- 172.21.2.101:0/1939865813 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c410a1e0 0x7fc7c40ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:07.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.240 7fc7c3ac0700 1 -- 172.21.2.101:0/1939865813 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c410a1e0 msgr2=0x7fc7c40ab870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:07.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.240 7fc7c3ac0700 1 --2- 172.21.2.101:0/1939865813 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c410a1e0 0x7fc7c40ab870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:07.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.240 7fc7c3ac0700 1 -- 172.21.2.101:0/1939865813 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7c40abe90 msgr2=0x7fc7c40ac290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:07.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.240 7fc7c3ac0700 1 --2- 172.21.2.101:0/1939865813 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7c40abe90 0x7fc7c40ac290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:07.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.240 7fc7c3ac0700 1 -- 172.21.2.101:0/1939865813 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c40ade40 con 0x7fc7c40ad200 2021-02-10T11:30:07.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.240 7fc7c3ac0700 1 --2- 172.21.2.101:0/1939865813 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c40ad200 0x7fc7c40ad600 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7fc7bc00b970 tx=0x7fc7bc00bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:07.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.240 7fc7c0aba700 1 -- 172.21.2.101:0/1939865813 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc7bc000f90 con 0x7fc7c40ad200 2021-02-10T11:30:07.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.240 7fc7c0aba700 1 -- 172.21.2.101:0/1939865813 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7bc003030 con 0x7fc7c40ad200 2021-02-10T11:30:07.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.240 7fc7c0aba700 1 -- 172.21.2.101:0/1939865813 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc7bc0015f0 con 0x7fc7c40ad200 2021-02-10T11:30:07.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.240 7fc7c9588700 1 -- 172.21.2.101:0/1939865813 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c40ade40 con 0x7fc7c40ad200 2021-02-10T11:30:07.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.240 7fc7c9588700 1 -- 172.21.2.101:0/1939865813 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c40ade40 con 0x7fc7c40ad200 2021-02-10T11:30:07.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.241 7fc7c0aba700 1 -- 172.21.2.101:0/1939865813 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc7bc000df0 con 0x7fc7c40ad200 2021-02-10T11:30:07.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.241 7fc7c0aba700 1 --2- 172.21.2.101:0/1939865813 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7a80186c0 0x7fc7a801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:07.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.241 7fc7c0aba700 1 -- 172.21.2.101:0/1939865813 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc7bc0328d0 con 0x7fc7c40ad200 2021-02-10T11:30:07.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.241 7fc7c32bf700 1 --2- 172.21.2.101:0/1939865813 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7a80186c0 0x7fc7a801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:07.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.242 7fc7c32bf700 1 --2- 172.21.2.101:0/1939865813 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7a80186c0 0x7fc7a801abc0 crc :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:07.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.270 7fc7c9588700 1 -- 172.21.2.101:0/1939865813 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c4105ae0 con 0x7fc7c40ad200 2021-02-10T11:30:07.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.273 7fc7c0aba700 1 -- 172.21.2.101:0/1939865813 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc7bc0066f0 con 0x7fc7c40ad200 2021-02-10T11:30:07.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.402 7fc7c9588700 1 -- 172.21.2.101:0/1939865813 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839250", "target": ["mgr", ""]}) v1 -- 0x7fc7c40ac8b0 con 0x7fc7a80186c0 2021-02-10T11:30:07.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.412 7fc7c0aba700 1 -- 172.21.2.101:0/1939865813 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fc7b4006820 con 0x7fc7a80186c0 2021-02-10T11:30:07.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.419 7fc7c9588700 1 -- 172.21.2.101:0/1939865813 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7a80186c0 msgr2=0x7fc7a801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:07.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.419 7fc7c9588700 1 --2- 172.21.2.101:0/1939865813 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7a80186c0 0x7fc7a801abc0 crc :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:07.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.419 7fc7c9588700 1 -- 172.21.2.101:0/1939865813 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c40ad200 msgr2=0x7fc7c40ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:07.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.419 7fc7c9588700 1 --2- 172.21.2.101:0/1939865813 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c40ad200 0x7fc7c40ad600 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7fc7bc00b970 tx=0x7fc7bc00bd10).stop 2021-02-10T11:30:07.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.419 7fc7c9588700 1 -- 172.21.2.101:0/1939865813 shutdown_connections 2021-02-10T11:30:07.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.419 7fc7c9588700 1 --2- 172.21.2.101:0/1939865813 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c40ad200 0x7fc7c40ad600 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:07.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.419 7fc7c9588700 1 --2- 172.21.2.101:0/1939865813 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7c40abe90 0x7fc7c40ac290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:07.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.419 7fc7c9588700 1 --2- 172.21.2.101:0/1939865813 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7a80186c0 0x7fc7a801abc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:07.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.419 7fc7c9588700 1 --2- 172.21.2.101:0/1939865813 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c410a1e0 0x7fc7c40ab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:07.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.419 7fc7c9588700 1 -- 172.21.2.101:0/1939865813 >> 172.21.2.101:0/1939865813 conn(0x7fc7c40bbf60 msgr2=0x7fc7c4105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:07.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.419 7fc7c9588700 1 -- 172.21.2.101:0/1939865813 shutdown_connections 2021-02-10T11:30:07.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.419 7fc7c9588700 1 -- 172.21.2.101:0/1939865813 wait complete. 2021-02-10T11:30:07.439 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839251 2021-02-10T11:30:07.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.844 7f0c67a30700 1 Processor -- start 2021-02-10T11:30:07.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.845 7f0c67a30700 1 -- start start 2021-02-10T11:30:07.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.845 7f0c67a30700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0c601b0470 0x7f0c601b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:07.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.845 7f0c67a30700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c600ef610 0x7f0c600efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:07.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.845 7f0c67a30700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0c600f0030 0x7f0c600f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:07.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.845 7f0c67a30700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c600e5ea0 con 0x7f0c600f0030 2021-02-10T11:30:07.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.845 7f0c67a30700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c600f0b50 con 0x7f0c601b0470 2021-02-10T11:30:07.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.845 7f0c67a30700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c600e52e0 con 0x7f0c600ef610 2021-02-10T11:30:07.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.845 7f0c6604d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0c600f0030 0x7f0c600f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:07.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.845 7f0c6584c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0c601b0470 0x7f0c601b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:07.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.845 7f0c6504b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c600ef610 0x7f0c600efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:07.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.845 7f0c6604d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0c600f0030 0x7f0c600f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48590/0 (socket says 172.21.2.101:48590) 2021-02-10T11:30:07.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.845 7f0c6504b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c600ef610 0x7f0c600efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:41326/0 (socket says 172.21.2.101:41326) 2021-02-10T11:30:07.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.845 7f0c6584c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0c601b0470 0x7f0c601b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:59076/0 (socket says 172.21.2.101:59076) 2021-02-10T11:30:07.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.845 7f0c6604d700 1 -- 172.21.2.101:0/3900924773 learned_addr learned my addr 172.21.2.101:0/3900924773 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:07.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.846 7f0c6584c700 1 -- 172.21.2.101:0/3900924773 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c600ef610 msgr2=0x7f0c600efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:07.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.846 7f0c6584c700 1 --2- 172.21.2.101:0/3900924773 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c600ef610 0x7f0c600efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:07.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.846 7f0c6584c700 1 -- 172.21.2.101:0/3900924773 >> v2:172.21.2.101:3300/0 conn(0x7f0c600f0030 msgr2=0x7f0c600f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:07.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.846 7f0c6584c700 1 --2- 172.21.2.101:0/3900924773 >> v2:172.21.2.101:3300/0 conn(0x7f0c600f0030 0x7f0c600f0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:07.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.846 7f0c6584c700 1 -- 172.21.2.101:0/3900924773 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c600f10d0 con 0x7f0c601b0470 2021-02-10T11:30:07.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.846 7f0c6584c700 1 --2- 172.21.2.101:0/3900924773 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c601b0470 0x7f0c601b0870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f0c5c006650 tx=0x7f0c5c009a80).ready entity=mon.1 client_cookie=dde90d7ae7474aab server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:07.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.846 7f0c6484a700 1 -- 172.21.2.101:0/3900924773 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0c5c00c0c0 con 0x7f0c601b0470 2021-02-10T11:30:07.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.846 7f0c6484a700 1 -- 172.21.2.101:0/3900924773 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5c00c340 con 0x7f0c601b0470 2021-02-10T11:30:07.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.846 7f0c6484a700 1 -- 172.21.2.101:0/3900924773 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0c5c00c820 con 0x7f0c601b0470 2021-02-10T11:30:07.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.846 7f0c67a30700 1 -- 172.21.2.101:0/3900924773 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c601b0470 msgr2=0x7f0c601b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:07.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.846 7f0c67a30700 1 --2- 172.21.2.101:0/3900924773 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c601b0470 0x7f0c601b0870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f0c5c006650 tx=0x7f0c5c009a80).stop 2021-02-10T11:30:07.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.847 7f0c67a30700 1 -- 172.21.2.101:0/3900924773 shutdown_connections 2021-02-10T11:30:07.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.847 7f0c67a30700 1 --2- 172.21.2.101:0/3900924773 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c600ef610 0x7f0c600efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:07.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.847 7f0c67a30700 1 --2- 172.21.2.101:0/3900924773 >> v2:172.21.2.101:3300/0 conn(0x7f0c600f0030 0x7f0c600f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:07.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.847 7f0c67a30700 1 --2- 172.21.2.101:0/3900924773 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c601b0470 0x7f0c601b0870 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:07.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.847 7f0c67a30700 1 -- 172.21.2.101:0/3900924773 >> 172.21.2.101:0/3900924773 conn(0x7f0c600ce3e0 msgr2=0x7f0c600e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:07.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.847 7f0c67a30700 1 -- 172.21.2.101:0/3900924773 shutdown_connections 2021-02-10T11:30:07.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.847 7f0c67a30700 1 -- 172.21.2.101:0/3900924773 wait complete. 2021-02-10T11:30:07.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.862 7f0c67a30700 1 Processor -- start 2021-02-10T11:30:07.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.862 7f0c67a30700 1 -- start start 2021-02-10T11:30:07.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.862 7f0c67a30700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c6009b930 0x7f0c600b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:07.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.862 7f0c67a30700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c600b8410 0x7f0c600b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:07.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.862 7f0c67a30700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c600c1f80 0x7f0c600c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:07.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.862 7f0c67a30700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c600c1a00 con 0x7f0c600b8410 2021-02-10T11:30:07.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.862 7f0c67a30700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c600b8e30 con 0x7f0c6009b930 2021-02-10T11:30:07.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.862 7f0c67a30700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c600b9050 con 0x7f0c600c1f80 2021-02-10T11:30:07.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.863 7f0c6504b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c600b8410 0x7f0c600b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:07.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.863 7f0c6504b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c600b8410 0x7f0c600b8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48600/0 (socket says 172.21.2.101:48600) 2021-02-10T11:30:07.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.863 7f0c6504b700 1 -- 172.21.2.101:0/1443956759 learned_addr learned my addr 172.21.2.101:0/1443956759 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:07.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.863 7f0c6584c700 1 --2- 172.21.2.101:0/1443956759 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c6009b930 0x7f0c600b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:07.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.863 7f0c6604d700 1 --2- 172.21.2.101:0/1443956759 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c600c1f80 0x7f0c600c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:07.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.863 7f0c6504b700 1 -- 172.21.2.101:0/1443956759 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c600c1f80 msgr2=0x7f0c600c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:07.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.863 7f0c6504b700 1 --2- 172.21.2.101:0/1443956759 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c600c1f80 0x7f0c600c2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:07.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.863 7f0c6504b700 1 -- 172.21.2.101:0/1443956759 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c6009b930 msgr2=0x7f0c600b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:07.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.863 7f0c6504b700 1 --2- 172.21.2.101:0/1443956759 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c6009b930 0x7f0c600b7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:07.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.863 7f0c6504b700 1 -- 172.21.2.101:0/1443956759 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c600c29a0 con 0x7f0c600b8410 2021-02-10T11:30:07.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.863 7f0c6504b700 1 --2- 172.21.2.101:0/1443956759 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c600b8410 0x7f0c600b8810 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f0c54006f60 tx=0x7f0c5400b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:07.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.863 7f0c527fc700 1 -- 172.21.2.101:0/1443956759 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0c5400bd60 con 0x7f0c600b8410 2021-02-10T11:30:07.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.863 7f0c527fc700 1 -- 172.21.2.101:0/1443956759 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c54000c90 con 0x7f0c600b8410 2021-02-10T11:30:07.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.863 7f0c527fc700 1 -- 172.21.2.101:0/1443956759 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0c54002220 con 0x7f0c600b8410 2021-02-10T11:30:07.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.863 7f0c67a30700 1 -- 172.21.2.101:0/1443956759 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c600c29a0 con 0x7f0c600b8410 2021-02-10T11:30:07.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.864 7f0c67a30700 1 -- 172.21.2.101:0/1443956759 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c600c29a0 con 0x7f0c600b8410 2021-02-10T11:30:07.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.864 7f0c527fc700 1 -- 172.21.2.101:0/1443956759 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0c540052e0 con 0x7f0c600b8410 2021-02-10T11:30:07.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.864 7f0c527fc700 1 --2- 172.21.2.101:0/1443956759 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c480186c0 0x7f0c4801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:07.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.864 7f0c527fc700 1 -- 172.21.2.101:0/1443956759 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0c54032780 con 0x7f0c600b8410 2021-02-10T11:30:07.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.864 7f0c6584c700 1 --2- 172.21.2.101:0/1443956759 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c480186c0 0x7f0c4801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:07.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.865 7f0c6584c700 1 --2- 172.21.2.101:0/1443956759 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c480186c0 0x7f0c4801abc0 crc :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:07.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.894 7f0c67a30700 1 -- 172.21.2.101:0/1443956759 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c60105b30 con 0x7f0c600b8410 2021-02-10T11:30:07.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:07.897 7f0c527fc700 1 -- 172.21.2.101:0/1443956759 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0c54000c90 con 0x7f0c600b8410 2021-02-10T11:30:08.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.023 7f0c67a30700 1 -- 172.21.2.101:0/1443956759 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839251", "target": ["mgr", ""]}) v1 -- 0x7f0c60105b30 con 0x7f0c480186c0 2021-02-10T11:30:08.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.034 7f0c527fc700 1 -- 172.21.2.101:0/1443956759 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f0c5c006800 con 0x7f0c480186c0 2021-02-10T11:30:08.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.041 7f0c67a30700 1 -- 172.21.2.101:0/1443956759 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c480186c0 msgr2=0x7f0c4801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:08.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.041 7f0c67a30700 1 --2- 172.21.2.101:0/1443956759 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c480186c0 0x7f0c4801abc0 crc :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:08.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.041 7f0c67a30700 1 -- 172.21.2.101:0/1443956759 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c600b8410 msgr2=0x7f0c600b8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:08.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.041 7f0c67a30700 1 --2- 172.21.2.101:0/1443956759 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c600b8410 0x7f0c600b8810 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f0c54006f60 tx=0x7f0c5400b970).stop 2021-02-10T11:30:08.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.041 7f0c67a30700 1 -- 172.21.2.101:0/1443956759 shutdown_connections 2021-02-10T11:30:08.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.041 7f0c67a30700 1 --2- 172.21.2.101:0/1443956759 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c480186c0 0x7f0c4801abc0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:08.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.041 7f0c67a30700 1 --2- 172.21.2.101:0/1443956759 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c600c1f80 0x7f0c600c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:08.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.041 7f0c67a30700 1 --2- 172.21.2.101:0/1443956759 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c600b8410 0x7f0c600b8810 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:08.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.041 7f0c67a30700 1 --2- 172.21.2.101:0/1443956759 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c6009b930 0x7f0c600b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:08.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.041 7f0c67a30700 1 -- 172.21.2.101:0/1443956759 >> 172.21.2.101:0/1443956759 conn(0x7f0c60094e10 msgr2=0x7f0c600a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:08.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.041 7f0c67a30700 1 -- 172.21.2.101:0/1443956759 shutdown_connections 2021-02-10T11:30:08.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.041 7f0c67a30700 1 -- 172.21.2.101:0/1443956759 wait complete. 2021-02-10T11:30:08.061 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839252 2021-02-10T11:30:08.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.466 7fb4c9e43700 1 Processor -- start 2021-02-10T11:30:08.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.466 7fb4c9e43700 1 -- start start 2021-02-10T11:30:08.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.466 7fb4c9e43700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb4c41b0470 0x7fb4c41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:08.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.466 7fb4c9e43700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb4c40f20e0 0x7fb4c40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:08.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.466 7fb4c9e43700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c40f0b50 con 0x7fb4c40ef610 2021-02-10T11:30:08.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.466 7fb4c9e43700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4c40f0820 con 0x7fb4c40f20e0 2021-02-10T11:30:08.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.466 7fb4c9e43700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4c40f24e0 con 0x7fb4c41b0470 2021-02-10T11:30:08.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.466 7fb4c37fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb4c41b0470 0x7fb4c41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:08.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.466 7fb4c3fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb4c40f20e0 0x7fb4c40f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:08.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.466 7fb4c37fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb4c41b0470 0x7fb4c41b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:59090/0 (socket says 172.21.2.101:59090) 2021-02-10T11:30:08.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.466 7fb4c3fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb4c40f20e0 0x7fb4c40f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48608/0 (socket says 172.21.2.101:48608) 2021-02-10T11:30:08.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.466 7fb4c2ffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb4c40ef610 0x7fb4c40efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49100/0 (socket says 172.21.2.101:49100) 2021-02-10T11:30:08.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.466 7fb4c37fe700 1 -- 172.21.2.101:0/2432125107 learned_addr learned my addr 172.21.2.101:0/2432125107 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:08.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.467 7fb4c3fff700 1 -- 172.21.2.101:0/2432125107 >> v2:172.21.2.108:3300/0 conn(0x7fb4c41b0470 msgr2=0x7fb4c41b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:08.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.467 7fb4c3fff700 1 --2- 172.21.2.101:0/2432125107 >> v2:172.21.2.108:3300/0 conn(0x7fb4c41b0470 0x7fb4c41b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:08.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.467 7fb4c3fff700 1 -- 172.21.2.101:0/2432125107 >> v1:172.21.2.101:6789/0 conn(0x7fb4c40ef610 legacy=0x7fb4c40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:08.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.467 7fb4c27fc700 1 -- 172.21.2.101:0/2432125107 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 110021519 0 0) 0x7fb4b80010b0 con 0x7fb4c40ef610 2021-02-10T11:30:08.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.467 7fb4c3fff700 1 -- 172.21.2.101:0/2432125107 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c40e52e0 con 0x7fb4c40f20e0 2021-02-10T11:30:08.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.467 7fb4c3fff700 1 --2- 172.21.2.101:0/2432125107 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4c40f20e0 0x7fb4c40f0200 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7fb4b4006ab0 tx=0x7fb4b4009ac0).ready entity=mon.0 client_cookie=727e86b2578204c9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:08.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.467 7fb4c27fc700 1 -- 172.21.2.101:0/2432125107 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb4b40066d0 con 0x7fb4c40f20e0 2021-02-10T11:30:08.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.467 7fb4c27fc700 1 -- 172.21.2.101:0/2432125107 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b40022e0 con 0x7fb4c40f20e0 2021-02-10T11:30:08.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.467 7fb4c27fc700 1 -- 172.21.2.101:0/2432125107 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb4b40027c0 con 0x7fb4c40f20e0 2021-02-10T11:30:08.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.467 7fb4c9e43700 1 -- 172.21.2.101:0/2432125107 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4c40f20e0 msgr2=0x7fb4c40f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:08.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.467 7fb4c9e43700 1 --2- 172.21.2.101:0/2432125107 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4c40f20e0 0x7fb4c40f0200 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7fb4b4006ab0 tx=0x7fb4b4009ac0).stop 2021-02-10T11:30:08.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.468 7fb4c9e43700 1 -- 172.21.2.101:0/2432125107 shutdown_connections 2021-02-10T11:30:08.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.468 7fb4c9e43700 1 --2- 172.21.2.101:0/2432125107 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4c40f20e0 0x7fb4c40f0200 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:08.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.468 7fb4c9e43700 1 --2- 172.21.2.101:0/2432125107 >> v2:172.21.2.108:3300/0 conn(0x7fb4c41b0470 0x7fb4c41b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:08.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.468 7fb4c9e43700 1 -- 172.21.2.101:0/2432125107 >> 172.21.2.101:0/2432125107 conn(0x7fb4c40ce3e0 msgr2=0x7fb4c40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:08.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.468 7fb4c9e43700 1 -- 172.21.2.101:0/2432125107 shutdown_connections 2021-02-10T11:30:08.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.468 7fb4c9e43700 1 -- 172.21.2.101:0/2432125107 wait complete. 2021-02-10T11:30:08.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.483 7fb4c9e43700 1 Processor -- start 2021-02-10T11:30:08.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.483 7fb4c9e43700 1 -- start start 2021-02-10T11:30:08.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.483 7fb4c9e43700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4c419e260 0x7fb4c408e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:08.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.483 7fb4c9e43700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4c4091900 0x7fb4c4091d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:08.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.483 7fb4c9e43700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4c40b9c40 0x7fb4c40ba040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:08.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.483 7fb4c9e43700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c4092440 con 0x7fb4c4091900 2021-02-10T11:30:08.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.483 7fb4c9e43700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40ba660 con 0x7fb4c40b9c40 2021-02-10T11:30:08.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.483 7fb4c9e43700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40ba880 con 0x7fb4c419e260 2021-02-10T11:30:08.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.484 7fb4c37fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4c419e260 0x7fb4c408e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:08.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.484 7fb4c2ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4c4091900 0x7fb4c4091d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:08.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.484 7fb4c37fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4c419e260 0x7fb4c408e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:41342/0 (socket says 172.21.2.101:41342) 2021-02-10T11:30:08.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.484 7fb4c37fe700 1 -- 172.21.2.101:0/937863790 learned_addr learned my addr 172.21.2.101:0/937863790 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:08.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.484 7fb4c3fff700 1 --2- 172.21.2.101:0/937863790 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4c40b9c40 0x7fb4c40ba040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:08.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.484 7fb4c37fe700 1 -- 172.21.2.101:0/937863790 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4c40b9c40 msgr2=0x7fb4c40ba040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:08.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.484 7fb4c37fe700 1 --2- 172.21.2.101:0/937863790 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4c40b9c40 0x7fb4c40ba040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:08.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.484 7fb4c37fe700 1 -- 172.21.2.101:0/937863790 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4c4091900 msgr2=0x7fb4c4091d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:08.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.484 7fb4c37fe700 1 --2- 172.21.2.101:0/937863790 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4c4091900 0x7fb4c4091d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:08.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.484 7fb4c37fe700 1 -- 172.21.2.101:0/937863790 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c40baaa0 con 0x7fb4c419e260 2021-02-10T11:30:08.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.484 7fb4c37fe700 1 --2- 172.21.2.101:0/937863790 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4c419e260 0x7fb4c408e550 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fb4bc00b910 tx=0x7fb4bc00bcb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:08.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.484 7fb4abfff700 1 -- 172.21.2.101:0/937863790 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb4bc000d30 con 0x7fb4c419e260 2021-02-10T11:30:08.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.485 7fb4abfff700 1 -- 172.21.2.101:0/937863790 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4bc00c2f0 con 0x7fb4c419e260 2021-02-10T11:30:08.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.485 7fb4c9e43700 1 -- 172.21.2.101:0/937863790 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c40baaa0 con 0x7fb4c419e260 2021-02-10T11:30:08.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.485 7fb4abfff700 1 -- 172.21.2.101:0/937863790 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb4bc00c810 con 0x7fb4c419e260 2021-02-10T11:30:08.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.485 7fb4c9e43700 1 -- 172.21.2.101:0/937863790 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c408f450 con 0x7fb4c419e260 2021-02-10T11:30:08.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.485 7fb4abfff700 1 -- 172.21.2.101:0/937863790 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb4bc001010 con 0x7fb4c419e260 2021-02-10T11:30:08.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.485 7fb4abfff700 1 --2- 172.21.2.101:0/937863790 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4a40186c0 0x7fb4a401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:08.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.485 7fb4abfff700 1 -- 172.21.2.101:0/937863790 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb4bc032760 con 0x7fb4c419e260 2021-02-10T11:30:08.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.485 7fb4c2ffd700 1 --2- 172.21.2.101:0/937863790 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4a40186c0 0x7fb4a401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:08.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.486 7fb4c2ffd700 1 --2- 172.21.2.101:0/937863790 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4a40186c0 0x7fb4a401abc0 crc :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:08.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.515 7fb4c9e43700 1 -- 172.21.2.101:0/937863790 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c40f0b50 con 0x7fb4c419e260 2021-02-10T11:30:08.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.518 7fb4abfff700 1 -- 172.21.2.101:0/937863790 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb4bc006b90 con 0x7fb4c419e260 2021-02-10T11:30:08.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.643 7fb4c9e43700 1 -- 172.21.2.101:0/937863790 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839252", "target": ["mgr", ""]}) v1 -- 0x7fb4c408f450 con 0x7fb4a40186c0 2021-02-10T11:30:08.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.653 7fb4abfff700 1 -- 172.21.2.101:0/937863790 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fb4b80009b0 con 0x7fb4a40186c0 2021-02-10T11:30:08.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.660 7fb4c9e43700 1 -- 172.21.2.101:0/937863790 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4a40186c0 msgr2=0x7fb4a401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:08.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.660 7fb4c9e43700 1 --2- 172.21.2.101:0/937863790 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4a40186c0 0x7fb4a401abc0 crc :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:08.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.660 7fb4c9e43700 1 -- 172.21.2.101:0/937863790 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4c419e260 msgr2=0x7fb4c408e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:08.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.660 7fb4c9e43700 1 --2- 172.21.2.101:0/937863790 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4c419e260 0x7fb4c408e550 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fb4bc00b910 tx=0x7fb4bc00bcb0).stop 2021-02-10T11:30:08.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.660 7fb4c9e43700 1 -- 172.21.2.101:0/937863790 shutdown_connections 2021-02-10T11:30:08.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.660 7fb4c9e43700 1 --2- 172.21.2.101:0/937863790 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4c40b9c40 0x7fb4c40ba040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:08.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.660 7fb4c9e43700 1 --2- 172.21.2.101:0/937863790 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4c4091900 0x7fb4c4091d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:08.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.660 7fb4c9e43700 1 --2- 172.21.2.101:0/937863790 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4a40186c0 0x7fb4a401abc0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:08.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.660 7fb4c9e43700 1 --2- 172.21.2.101:0/937863790 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4c419e260 0x7fb4c408e550 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:08.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.660 7fb4c9e43700 1 -- 172.21.2.101:0/937863790 >> 172.21.2.101:0/937863790 conn(0x7fb4c40bd060 msgr2=0x7fb4c409a020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:08.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.660 7fb4c9e43700 1 -- 172.21.2.101:0/937863790 shutdown_connections 2021-02-10T11:30:08.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:08.660 7fb4c9e43700 1 -- 172.21.2.101:0/937863790 wait complete. 2021-02-10T11:30:08.680 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839253 2021-02-10T11:30:09.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.085 7f00c5db4700 1 Processor -- start 2021-02-10T11:30:09.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.085 7f00c5db4700 1 -- start start 2021-02-10T11:30:09.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.085 7f00c5db4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00c01b0470 0x7f00c01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:09.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.085 7f00c5db4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f00c00ef610 0x7f00c00efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:09.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.085 7f00c5db4700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00c00f0b50 con 0x7f00c00f0030 2021-02-10T11:30:09.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.085 7f00c5db4700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c00e52e0 con 0x7f00c00ef610 2021-02-10T11:30:09.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.085 7f00c5db4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c00e5490 con 0x7f00c01b0470 2021-02-10T11:30:09.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.086 7f00beffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f00c00ef610 0x7f00c00efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:09.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.086 7f00beffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f00c00ef610 0x7f00c00efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48618/0 (socket says 172.21.2.101:48618) 2021-02-10T11:30:09.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.086 7f00beffd700 1 -- 172.21.2.101:0/2169828243 learned_addr learned my addr 172.21.2.101:0/2169828243 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:09.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.086 7f00bf7fe700 1 --2- 172.21.2.101:0/2169828243 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00c01b0470 0x7f00c01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:09.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.086 7f00beffd700 1 -- 172.21.2.101:0/2169828243 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00c01b0470 msgr2=0x7f00c01b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:09.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.086 7f00beffd700 1 --2- 172.21.2.101:0/2169828243 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00c01b0470 0x7f00c01b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:09.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.086 7f00beffd700 1 -- 172.21.2.101:0/2169828243 >> v1:172.21.2.108:6789/0 conn(0x7f00c00f0030 legacy=0x7f00c00f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:09.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.086 7f00beffd700 1 -- 172.21.2.101:0/2169828243 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c00f0430 con 0x7f00c00ef610 2021-02-10T11:30:09.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.086 7f00beffd700 1 --2- 172.21.2.101:0/2169828243 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00c00ef610 0x7f00c00efa10 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7f00a8006a80 tx=0x7f00a8009f00).ready entity=mon.0 client_cookie=efcf4aac6528b75 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:09.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.086 7f00be7fc700 1 -- 172.21.2.101:0/2169828243 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f00a800b5c0 con 0x7f00c00ef610 2021-02-10T11:30:09.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.086 7f00be7fc700 1 -- 172.21.2.101:0/2169828243 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a800b840 con 0x7f00c00ef610 2021-02-10T11:30:09.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.086 7f00be7fc700 1 -- 172.21.2.101:0/2169828243 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f00a800bd60 con 0x7f00c00ef610 2021-02-10T11:30:09.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.086 7f00c5db4700 1 -- 172.21.2.101:0/2169828243 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00c00ef610 msgr2=0x7f00c00efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:09.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.086 7f00c5db4700 1 --2- 172.21.2.101:0/2169828243 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00c00ef610 0x7f00c00efa10 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7f00a8006a80 tx=0x7f00a8009f00).stop 2021-02-10T11:30:09.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.087 7f00c5db4700 1 -- 172.21.2.101:0/2169828243 shutdown_connections 2021-02-10T11:30:09.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.087 7f00c5db4700 1 --2- 172.21.2.101:0/2169828243 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00c00ef610 0x7f00c00efa10 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:09.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.087 7f00c5db4700 1 --2- 172.21.2.101:0/2169828243 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00c01b0470 0x7f00c01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:09.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.087 7f00c5db4700 1 -- 172.21.2.101:0/2169828243 >> 172.21.2.101:0/2169828243 conn(0x7f00c00ce3e0 msgr2=0x7f00c00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:09.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.087 7f00c5db4700 1 -- 172.21.2.101:0/2169828243 shutdown_connections 2021-02-10T11:30:09.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.087 7f00c5db4700 1 -- 172.21.2.101:0/2169828243 wait complete. 2021-02-10T11:30:09.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.102 7f00c5db4700 1 Processor -- start 2021-02-10T11:30:09.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.102 7f00c5db4700 1 -- start start 2021-02-10T11:30:09.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.102 7f00c5db4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00c0105d10 0x7f00c009f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:09.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.102 7f00c5db4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00c00961f0 0x7f00c00965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:09.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.102 7f00c5db4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f00c0106890 0x7f00c0106c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:09.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.102 7f00c5db4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c0106110 con 0x7f00c00961f0 2021-02-10T11:30:09.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.102 7f00c5db4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c0106380 con 0x7f00c0106890 2021-02-10T11:30:09.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.102 7f00c5db4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c00a6620 con 0x7f00c0105d10 2021-02-10T11:30:09.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.102 7f00beffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00c00961f0 0x7f00c00965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:09.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.102 7f00beffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00c00961f0 0x7f00c00965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48626/0 (socket says 172.21.2.101:48626) 2021-02-10T11:30:09.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.102 7f00beffd700 1 -- 172.21.2.101:0/2370488292 learned_addr learned my addr 172.21.2.101:0/2370488292 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:09.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.103 7f00bf7fe700 1 --2- 172.21.2.101:0/2370488292 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00c0105d10 0x7f00c009f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:09.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.103 7f00bffff700 1 --2- 172.21.2.101:0/2370488292 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f00c0106890 0x7f00c0106c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:09.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.103 7f00beffd700 1 -- 172.21.2.101:0/2370488292 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00c0105d10 msgr2=0x7f00c009f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:09.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.103 7f00beffd700 1 --2- 172.21.2.101:0/2370488292 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00c0105d10 0x7f00c009f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:09.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.103 7f00beffd700 1 -- 172.21.2.101:0/2370488292 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f00c0106890 msgr2=0x7f00c0106c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:09.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.103 7f00beffd700 1 --2- 172.21.2.101:0/2370488292 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f00c0106890 0x7f00c0106c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:09.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.103 7f00beffd700 1 -- 172.21.2.101:0/2370488292 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c01072b0 con 0x7f00c00961f0 2021-02-10T11:30:09.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.103 7f00beffd700 1 --2- 172.21.2.101:0/2370488292 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00c00961f0 0x7f00c00965f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f00a800b9c0 tx=0x7f00a800bd60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:09.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.103 7f00b7fff700 1 -- 172.21.2.101:0/2370488292 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f00a8006240 con 0x7f00c00961f0 2021-02-10T11:30:09.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.103 7f00b7fff700 1 -- 172.21.2.101:0/2370488292 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a80064c0 con 0x7f00c00961f0 2021-02-10T11:30:09.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.103 7f00b7fff700 1 -- 172.21.2.101:0/2370488292 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f00a8005850 con 0x7f00c00961f0 2021-02-10T11:30:09.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.103 7f00c5db4700 1 -- 172.21.2.101:0/2370488292 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c01072b0 con 0x7f00c00961f0 2021-02-10T11:30:09.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.103 7f00c5db4700 1 -- 172.21.2.101:0/2370488292 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c01072b0 con 0x7f00c00961f0 2021-02-10T11:30:09.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.104 7f00b7fff700 1 -- 172.21.2.101:0/2370488292 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f00a8005660 con 0x7f00c00961f0 2021-02-10T11:30:09.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.104 7f00b7fff700 1 --2- 172.21.2.101:0/2370488292 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f00980186c0 0x7f009801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:09.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.104 7f00b7fff700 1 -- 172.21.2.101:0/2370488292 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f00a8032ba0 con 0x7f00c00961f0 2021-02-10T11:30:09.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.104 7f00bf7fe700 1 --2- 172.21.2.101:0/2370488292 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f00980186c0 0x7f009801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:09.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.104 7f00bf7fe700 1 --2- 172.21.2.101:0/2370488292 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f00980186c0 0x7f009801abc0 crc :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:09.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.134 7f00c5db4700 1 -- 172.21.2.101:0/2370488292 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c0105ae0 con 0x7f00c00961f0 2021-02-10T11:30:09.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.137 7f00b7fff700 1 -- 172.21.2.101:0/2370488292 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f00a8007ee0 con 0x7f00c00961f0 2021-02-10T11:30:09.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.268 7f00c5db4700 1 -- 172.21.2.101:0/2370488292 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839253", "target": ["mgr", ""]}) v1 -- 0x7f00c00f0b50 con 0x7f00980186c0 2021-02-10T11:30:09.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.278 7f00b7fff700 1 -- 172.21.2.101:0/2370488292 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f00b0006a70 con 0x7f00980186c0 2021-02-10T11:30:09.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.285 7f00c5db4700 1 -- 172.21.2.101:0/2370488292 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f00980186c0 msgr2=0x7f009801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:09.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.285 7f00c5db4700 1 --2- 172.21.2.101:0/2370488292 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f00980186c0 0x7f009801abc0 crc :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:09.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.285 7f00c5db4700 1 -- 172.21.2.101:0/2370488292 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00c00961f0 msgr2=0x7f00c00965f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:09.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.285 7f00c5db4700 1 --2- 172.21.2.101:0/2370488292 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00c00961f0 0x7f00c00965f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f00a800b9c0 tx=0x7f00a800bd60).stop 2021-02-10T11:30:09.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.286 7f00c5db4700 1 -- 172.21.2.101:0/2370488292 shutdown_connections 2021-02-10T11:30:09.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.286 7f00c5db4700 1 --2- 172.21.2.101:0/2370488292 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f00c0106890 0x7f00c0106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:09.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.286 7f00c5db4700 1 --2- 172.21.2.101:0/2370488292 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00c00961f0 0x7f00c00965f0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:09.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.286 7f00c5db4700 1 --2- 172.21.2.101:0/2370488292 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f00980186c0 0x7f009801abc0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:09.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.286 7f00c5db4700 1 --2- 172.21.2.101:0/2370488292 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00c0105d10 0x7f00c009f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:09.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.286 7f00c5db4700 1 -- 172.21.2.101:0/2370488292 >> 172.21.2.101:0/2370488292 conn(0x7f00c0095a70 msgr2=0x7f00c00bf050 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:09.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.286 7f00c5db4700 1 -- 172.21.2.101:0/2370488292 shutdown_connections 2021-02-10T11:30:09.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.286 7f00c5db4700 1 -- 172.21.2.101:0/2370488292 wait complete. 2021-02-10T11:30:09.305 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839254 2021-02-10T11:30:09.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.707 7fbb1de27700 1 Processor -- start 2021-02-10T11:30:09.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.707 7fbb1de27700 1 -- start start 2021-02-10T11:30:09.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.707 7fbb1de27700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb181b0870 0x7fbb180f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:09.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.707 7fbb1de27700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbb180efe00 0x7fbb180f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:09.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.707 7fbb1de27700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb181b0c70 con 0x7fbb181b0470 2021-02-10T11:30:09.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.707 7fbb1de27700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb180f0b50 con 0x7fbb180efe00 2021-02-10T11:30:09.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.707 7fbb1de27700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180f0820 con 0x7fbb181b0870 2021-02-10T11:30:09.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.708 7fbb177fe700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fbb181b0470 0x7fbb180ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:42534/0 (socket says 172.21.2.101:42534) 2021-02-10T11:30:09.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.708 7fbb177fe700 1 -- 172.21.2.101:0/1887042527 learned_addr learned my addr 172.21.2.101:0/1887042527 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:09.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.708 7fbb16ffd700 1 --2- 172.21.2.101:0/1887042527 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb181b0870 0x7fbb180f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:09.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.708 7fbb17fff700 1 --2- 172.21.2.101:0/1887042527 >> v2:172.21.2.108:3300/0 conn(0x7fbb180efe00 0x7fbb180f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:09.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.708 7fbb16ffd700 1 -- 172.21.2.101:0/1887042527 >> v2:172.21.2.108:3300/0 conn(0x7fbb180efe00 msgr2=0x7fbb180f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:09.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.708 7fbb16ffd700 1 --2- 172.21.2.101:0/1887042527 >> v2:172.21.2.108:3300/0 conn(0x7fbb180efe00 0x7fbb180f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:09.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.708 7fbb16ffd700 1 -- 172.21.2.101:0/1887042527 >> v1:172.21.2.108:6789/0 conn(0x7fbb181b0470 legacy=0x7fbb180ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:09.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.708 7fbb16ffd700 1 -- 172.21.2.101:0/1887042527 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb180e52e0 con 0x7fbb181b0870 2021-02-10T11:30:09.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.708 7fbb16ffd700 1 --2- 172.21.2.101:0/1887042527 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb181b0870 0x7fbb180f20e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fbb08006650 tx=0x7fbb08009a80).ready entity=mon.2 client_cookie=b005919b3b21298c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:09.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.708 7fbb167fc700 1 -- 172.21.2.101:0/1887042527 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbb0800c0c0 con 0x7fbb181b0870 2021-02-10T11:30:09.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.708 7fbb167fc700 1 -- 172.21.2.101:0/1887042527 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0800c340 con 0x7fbb181b0870 2021-02-10T11:30:09.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.708 7fbb167fc700 1 -- 172.21.2.101:0/1887042527 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbb0800c820 con 0x7fbb181b0870 2021-02-10T11:30:09.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.708 7fbb1de27700 1 -- 172.21.2.101:0/1887042527 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb181b0870 msgr2=0x7fbb180f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:09.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.708 7fbb1de27700 1 --2- 172.21.2.101:0/1887042527 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb181b0870 0x7fbb180f20e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fbb08006650 tx=0x7fbb08009a80).stop 2021-02-10T11:30:09.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.709 7fbb1de27700 1 -- 172.21.2.101:0/1887042527 shutdown_connections 2021-02-10T11:30:09.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.709 7fbb1de27700 1 --2- 172.21.2.101:0/1887042527 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb181b0870 0x7fbb180f20e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:09.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.709 7fbb1de27700 1 --2- 172.21.2.101:0/1887042527 >> v2:172.21.2.108:3300/0 conn(0x7fbb180efe00 0x7fbb180f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:09.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.709 7fbb1de27700 1 -- 172.21.2.101:0/1887042527 >> 172.21.2.101:0/1887042527 conn(0x7fbb180ce3e0 msgr2=0x7fbb180e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:09.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.709 7fbb1de27700 1 -- 172.21.2.101:0/1887042527 shutdown_connections 2021-02-10T11:30:09.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.709 7fbb1de27700 1 -- 172.21.2.101:0/1887042527 wait complete. 2021-02-10T11:30:09.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.723 7fbb1de27700 1 Processor -- start 2021-02-10T11:30:09.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.724 7fbb1de27700 1 -- start start 2021-02-10T11:30:09.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.724 7fbb1de27700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb18190920 0x7fbb180b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:09.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.724 7fbb1de27700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb180b8200 0x7fbb180b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:09.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.724 7fbb1de27700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb180c1d80 0x7fbb180c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:09.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.724 7fbb1de27700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb18190d20 con 0x7fbb18190920 2021-02-10T11:30:09.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.724 7fbb1de27700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180b8c20 con 0x7fbb180c1d80 2021-02-10T11:30:09.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.724 7fbb1de27700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180c27a0 con 0x7fbb180b8200 2021-02-10T11:30:09.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.724 7fbb177fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb18190920 0x7fbb180b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:09.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.724 7fbb16ffd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb180b8200 0x7fbb180b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:09.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.724 7fbb17fff700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb180c1d80 0x7fbb180c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.724 7fbb177fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb18190920 0x7fbb180b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48638/0 (socket says 172.21.2.101:48638) 2021-02-10T11:30:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.724 7fbb177fe700 1 -- 172.21.2.101:0/114915008 learned_addr learned my addr 172.21.2.101:0/114915008 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.724 7fbb16ffd700 1 -- 172.21.2.101:0/114915008 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb180c1d80 msgr2=0x7fbb180c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.724 7fbb16ffd700 1 --2- 172.21.2.101:0/114915008 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb180c1d80 0x7fbb180c2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.724 7fbb16ffd700 1 -- 172.21.2.101:0/114915008 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb18190920 msgr2=0x7fbb180b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.725 7fbb16ffd700 1 --2- 172.21.2.101:0/114915008 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb18190920 0x7fbb180b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.725 7fbb16ffd700 1 -- 172.21.2.101:0/114915008 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb180c29c0 con 0x7fbb180b8200 2021-02-10T11:30:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.725 7fbb177fe700 1 --2- 172.21.2.101:0/114915008 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb18190920 0x7fbb180b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.725 7fbb16ffd700 1 --2- 172.21.2.101:0/114915008 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb180b8200 0x7fbb180b8600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fbb0800b560 tx=0x7fbb0800b900).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.725 7fbb0ffff700 1 -- 172.21.2.101:0/114915008 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbb0800bcf0 con 0x7fbb180b8200 2021-02-10T11:30:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.725 7fbb0ffff700 1 -- 172.21.2.101:0/114915008 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb08006740 con 0x7fbb180b8200 2021-02-10T11:30:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.725 7fbb1de27700 1 -- 172.21.2.101:0/114915008 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb180c29c0 con 0x7fbb180b8200 2021-02-10T11:30:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.725 7fbb1de27700 1 -- 172.21.2.101:0/114915008 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb180c2f50 con 0x7fbb180b8200 2021-02-10T11:30:09.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.725 7fbb0ffff700 1 -- 172.21.2.101:0/114915008 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbb08002030 con 0x7fbb180b8200 2021-02-10T11:30:09.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.726 7fbb0ffff700 1 -- 172.21.2.101:0/114915008 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbb08009fe0 con 0x7fbb180b8200 2021-02-10T11:30:09.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.726 7fbb0ffff700 1 --2- 172.21.2.101:0/114915008 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbafc0186c0 0x7fbafc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:09.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.726 7fbb0ffff700 1 -- 172.21.2.101:0/114915008 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fbb08032500 con 0x7fbb180b8200 2021-02-10T11:30:09.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.726 7fbb177fe700 1 --2- 172.21.2.101:0/114915008 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbafc0186c0 0x7fbafc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:09.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.726 7fbb177fe700 1 --2- 172.21.2.101:0/114915008 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbafc0186c0 0x7fbafc01abc0 crc :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:09.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.757 7fbb1de27700 1 -- 172.21.2.101:0/114915008 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb18105ae0 con 0x7fbb180b8200 2021-02-10T11:30:09.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.760 7fbb0ffff700 1 -- 172.21.2.101:0/114915008 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbb080093a0 con 0x7fbb180b8200 2021-02-10T11:30:09.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.886 7fbb1de27700 1 -- 172.21.2.101:0/114915008 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839254", "target": ["mgr", ""]}) v1 -- 0x7fbb180c2f50 con 0x7fbafc0186c0 2021-02-10T11:30:09.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.896 7fbb0ffff700 1 -- 172.21.2.101:0/114915008 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fbb00001160 con 0x7fbafc0186c0 2021-02-10T11:30:09.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.903 7fbb1de27700 1 -- 172.21.2.101:0/114915008 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbafc0186c0 msgr2=0x7fbafc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:09.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.903 7fbb1de27700 1 --2- 172.21.2.101:0/114915008 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbafc0186c0 0x7fbafc01abc0 crc :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:09.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.903 7fbb1de27700 1 -- 172.21.2.101:0/114915008 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb180b8200 msgr2=0x7fbb180b8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:09.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.903 7fbb1de27700 1 --2- 172.21.2.101:0/114915008 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb180b8200 0x7fbb180b8600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fbb0800b560 tx=0x7fbb0800b900).stop 2021-02-10T11:30:09.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.903 7fbb1de27700 1 -- 172.21.2.101:0/114915008 shutdown_connections 2021-02-10T11:30:09.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.903 7fbb1de27700 1 --2- 172.21.2.101:0/114915008 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbafc0186c0 0x7fbafc01abc0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:09.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.903 7fbb1de27700 1 --2- 172.21.2.101:0/114915008 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb180b8200 0x7fbb180b8600 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:09.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.903 7fbb1de27700 1 --2- 172.21.2.101:0/114915008 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb180c1d80 0x7fbb180c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:09.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.903 7fbb1de27700 1 --2- 172.21.2.101:0/114915008 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb18190920 0x7fbb180b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:09.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.903 7fbb1de27700 1 -- 172.21.2.101:0/114915008 >> 172.21.2.101:0/114915008 conn(0x7fbb18094e10 msgr2=0x7fbb180a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:09.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.903 7fbb1de27700 1 -- 172.21.2.101:0/114915008 shutdown_connections 2021-02-10T11:30:09.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:09.903 7fbb1de27700 1 -- 172.21.2.101:0/114915008 wait complete. 2021-02-10T11:30:09.924 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839255 2021-02-10T11:30:10.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.328 7f45d7c96700 1 Processor -- start 2021-02-10T11:30:10.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.328 7f45d7c96700 1 -- start start 2021-02-10T11:30:10.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.328 7f45d7c96700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f45d01b0470 0x7f45d01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:10.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.329 7f45d7c96700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45d00ef610 0x7f45d00efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:10.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.329 7f45d7c96700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d00f0b50 con 0x7f45d00f0030 2021-02-10T11:30:10.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.329 7f45d7c96700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d00e52e0 con 0x7f45d01b0470 2021-02-10T11:30:10.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.329 7f45d7c96700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d00e54d0 con 0x7f45d00ef610 2021-02-10T11:30:10.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.329 7f45d5ab2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f45d01b0470 0x7f45d01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:10.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.329 7f45d5ab2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f45d01b0470 0x7f45d01b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48644/0 (socket says 172.21.2.101:48644) 2021-02-10T11:30:10.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.329 7f45d5ab2700 1 -- 172.21.2.101:0/2218609412 learned_addr learned my addr 172.21.2.101:0/2218609412 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:10.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.329 7f45d52b1700 1 --2- 172.21.2.101:0/2218609412 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45d00ef610 0x7f45d00efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:10.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.329 7f45d5ab2700 1 -- 172.21.2.101:0/2218609412 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45d00ef610 msgr2=0x7f45d00efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:10.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.329 7f45d5ab2700 1 --2- 172.21.2.101:0/2218609412 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45d00ef610 0x7f45d00efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:10.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.329 7f45d5ab2700 1 -- 172.21.2.101:0/2218609412 >> v1:172.21.2.108:6789/0 conn(0x7f45d00f0030 legacy=0x7f45d00f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:10.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.329 7f45d5ab2700 1 -- 172.21.2.101:0/2218609412 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d00f0430 con 0x7f45d01b0470 2021-02-10T11:30:10.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.329 7f45d52b1700 1 --2- 172.21.2.101:0/2218609412 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45d00ef610 0x7f45d00efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:30:10.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.329 7f45d5ab2700 1 --2- 172.21.2.101:0/2218609412 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45d01b0470 0x7f45d01b0870 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f45c8006a80 tx=0x7f45c8009f00).ready entity=mon.0 client_cookie=13b302cf4c2f5457 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:10.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.329 7f45d4ab0700 1 -- 172.21.2.101:0/2218609412 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f45c800b5c0 con 0x7f45d01b0470 2021-02-10T11:30:10.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.329 7f45d4ab0700 1 -- 172.21.2.101:0/2218609412 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c800b800 con 0x7f45d01b0470 2021-02-10T11:30:10.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.329 7f45d4ab0700 1 -- 172.21.2.101:0/2218609412 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f45c800bd20 con 0x7f45d01b0470 2021-02-10T11:30:10.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.329 7f45d7c96700 1 -- 172.21.2.101:0/2218609412 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45d01b0470 msgr2=0x7f45d01b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:10.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.329 7f45d7c96700 1 --2- 172.21.2.101:0/2218609412 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45d01b0470 0x7f45d01b0870 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f45c8006a80 tx=0x7f45c8009f00).stop 2021-02-10T11:30:10.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.330 7f45d7c96700 1 -- 172.21.2.101:0/2218609412 shutdown_connections 2021-02-10T11:30:10.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.330 7f45d7c96700 1 --2- 172.21.2.101:0/2218609412 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45d00ef610 0x7f45d00efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:10.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.330 7f45d7c96700 1 --2- 172.21.2.101:0/2218609412 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45d01b0470 0x7f45d01b0870 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:10.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.330 7f45d7c96700 1 -- 172.21.2.101:0/2218609412 >> 172.21.2.101:0/2218609412 conn(0x7f45d00ce3e0 msgr2=0x7f45d00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:10.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.330 7f45d7c96700 1 -- 172.21.2.101:0/2218609412 shutdown_connections 2021-02-10T11:30:10.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.330 7f45d7c96700 1 -- 172.21.2.101:0/2218609412 wait complete. 2021-02-10T11:30:10.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.345 7f45d7c96700 1 Processor -- start 2021-02-10T11:30:10.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.345 7f45d7c96700 1 -- start start 2021-02-10T11:30:10.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.345 7f45d7c96700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45d0193230 0x7f45d00a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:10.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.345 7f45d7c96700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45d008f5b0 0x7f45d008f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:10.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.345 7f45d7c96700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f45d008e6a0 0x7f45d008eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:10.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.345 7f45d7c96700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d008ffd0 con 0x7f45d0193230 2021-02-10T11:30:10.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.345 7f45d7c96700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d0090240 con 0x7f45d008e6a0 2021-02-10T11:30:10.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.345 7f45d7c96700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d008f0c0 con 0x7f45d008f5b0 2021-02-10T11:30:10.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.346 7f45d5ab2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45d0193230 0x7f45d00a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:10.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.346 7f45d5ab2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45d0193230 0x7f45d00a6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48650/0 (socket says 172.21.2.101:48650) 2021-02-10T11:30:10.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.346 7f45d5ab2700 1 -- 172.21.2.101:0/622675692 learned_addr learned my addr 172.21.2.101:0/622675692 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:10.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.346 7f45d62b3700 1 --2- 172.21.2.101:0/622675692 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f45d008e6a0 0x7f45d008eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:10.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.346 7f45d52b1700 1 --2- 172.21.2.101:0/622675692 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45d008f5b0 0x7f45d008f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:10.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.346 7f45d5ab2700 1 -- 172.21.2.101:0/622675692 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45d008f5b0 msgr2=0x7f45d008f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:10.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.346 7f45d5ab2700 1 --2- 172.21.2.101:0/622675692 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45d008f5b0 0x7f45d008f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:10.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.346 7f45d5ab2700 1 -- 172.21.2.101:0/622675692 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f45d008e6a0 msgr2=0x7f45d008eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:10.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.346 7f45d5ab2700 1 --2- 172.21.2.101:0/622675692 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f45d008e6a0 0x7f45d008eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:10.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.346 7f45d5ab2700 1 -- 172.21.2.101:0/622675692 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d008f2a0 con 0x7f45d0193230 2021-02-10T11:30:10.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.346 7f45d52b1700 1 --2- 172.21.2.101:0/622675692 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45d008f5b0 0x7f45d008f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:30:10.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.346 7f45d5ab2700 1 --2- 172.21.2.101:0/622675692 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45d0193230 0x7f45d00a6a80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7f45c800b970 tx=0x7f45c800bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:10.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.346 7f45c67fc700 1 -- 172.21.2.101:0/622675692 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f45c8000d30 con 0x7f45d0193230 2021-02-10T11:30:10.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.346 7f45c67fc700 1 -- 172.21.2.101:0/622675692 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c800c2f0 con 0x7f45d0193230 2021-02-10T11:30:10.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.346 7f45c67fc700 1 -- 172.21.2.101:0/622675692 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f45c800c750 con 0x7f45d0193230 2021-02-10T11:30:10.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.346 7f45d7c96700 1 -- 172.21.2.101:0/622675692 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d008f2a0 con 0x7f45d0193230 2021-02-10T11:30:10.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.346 7f45d7c96700 1 -- 172.21.2.101:0/622675692 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d008f2a0 con 0x7f45d0193230 2021-02-10T11:30:10.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.347 7f45c67fc700 1 -- 172.21.2.101:0/622675692 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f45c8001010 con 0x7f45d0193230 2021-02-10T11:30:10.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.347 7f45c67fc700 1 --2- 172.21.2.101:0/622675692 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45b80186c0 0x7f45b801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:10.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.347 7f45c67fc700 1 -- 172.21.2.101:0/622675692 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f45c80329f0 con 0x7f45d0193230 2021-02-10T11:30:10.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.347 7f45d52b1700 1 --2- 172.21.2.101:0/622675692 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45b80186c0 0x7f45b801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:10.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.347 7f45d52b1700 1 --2- 172.21.2.101:0/622675692 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45b80186c0 0x7f45b801abc0 crc :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:10.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.377 7f45d7c96700 1 -- 172.21.2.101:0/622675692 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d00f0b50 con 0x7f45d0193230 2021-02-10T11:30:10.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.380 7f45c67fc700 1 -- 172.21.2.101:0/622675692 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f45c800c2f0 con 0x7f45d0193230 2021-02-10T11:30:10.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.509 7f45d7c96700 1 -- 172.21.2.101:0/622675692 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839255", "target": ["mgr", ""]}) v1 -- 0x7f45d00f0b50 con 0x7f45b80186c0 2021-02-10T11:30:10.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.520 7f45c67fc700 1 -- 172.21.2.101:0/622675692 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f45c0006990 con 0x7f45b80186c0 2021-02-10T11:30:10.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.527 7f45d7c96700 1 -- 172.21.2.101:0/622675692 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45b80186c0 msgr2=0x7f45b801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:10.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.527 7f45d7c96700 1 --2- 172.21.2.101:0/622675692 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45b80186c0 0x7f45b801abc0 crc :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:10.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.527 7f45d7c96700 1 -- 172.21.2.101:0/622675692 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45d0193230 msgr2=0x7f45d00a6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:10.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.527 7f45d7c96700 1 --2- 172.21.2.101:0/622675692 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45d0193230 0x7f45d00a6a80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7f45c800b970 tx=0x7f45c800bd10).stop 2021-02-10T11:30:10.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.527 7f45d7c96700 1 -- 172.21.2.101:0/622675692 shutdown_connections 2021-02-10T11:30:10.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.527 7f45d7c96700 1 --2- 172.21.2.101:0/622675692 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45b80186c0 0x7f45b801abc0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:10.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.527 7f45d7c96700 1 --2- 172.21.2.101:0/622675692 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45d008f5b0 0x7f45d008f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:10.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.527 7f45d7c96700 1 --2- 172.21.2.101:0/622675692 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f45d008e6a0 0x7f45d008eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:10.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.527 7f45d7c96700 1 --2- 172.21.2.101:0/622675692 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45d0193230 0x7f45d00a6a80 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:10.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.527 7f45d7c96700 1 -- 172.21.2.101:0/622675692 >> 172.21.2.101:0/622675692 conn(0x7f45d00bf4e0 msgr2=0x7f45d00be160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:10.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.527 7f45d7c96700 1 -- 172.21.2.101:0/622675692 shutdown_connections 2021-02-10T11:30:10.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.527 7f45d7c96700 1 -- 172.21.2.101:0/622675692 wait complete. 2021-02-10T11:30:10.547 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839256 2021-02-10T11:30:10.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.952 7fab16303700 1 Processor -- start 2021-02-10T11:30:10.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.952 7fab16303700 1 -- start start 2021-02-10T11:30:10.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.952 7fab16303700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fab101b0470 0x7fab101b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:10.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.953 7fab16303700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab100f20e0 0x7fab100f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:10.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.953 7fab16303700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab100f0b50 con 0x7fab100ef610 2021-02-10T11:30:10.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.953 7fab16303700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab100f0820 con 0x7fab101b0470 2021-02-10T11:30:10.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.953 7fab16303700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab100f24e0 con 0x7fab100f20e0 2021-02-10T11:30:10.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.953 7fab0ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fab101b0470 0x7fab101b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:10.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.953 7fab0ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fab101b0470 0x7fab101b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48658/0 (socket says 172.21.2.101:48658) 2021-02-10T11:30:10.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.953 7fab0ffff700 1 -- 172.21.2.101:0/3827448410 learned_addr learned my addr 172.21.2.101:0/3827448410 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:10.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.953 7fab14920700 1 --2- 172.21.2.101:0/3827448410 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab100f20e0 0x7fab100f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:10.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.953 7fab0ffff700 1 -- 172.21.2.101:0/3827448410 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab100f20e0 msgr2=0x7fab100f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:10.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.953 7fab0ffff700 1 --2- 172.21.2.101:0/3827448410 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab100f20e0 0x7fab100f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:10.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.953 7fab0ffff700 1 -- 172.21.2.101:0/3827448410 >> v1:172.21.2.108:6789/0 conn(0x7fab100ef610 legacy=0x7fab100efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:10.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.953 7fab0ffff700 1 -- 172.21.2.101:0/3827448410 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab100e52e0 con 0x7fab101b0470 2021-02-10T11:30:10.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.953 7fab0ffff700 1 --2- 172.21.2.101:0/3827448410 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab101b0470 0x7fab101b0870 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7fab00006700 tx=0x7fab00009b80).ready entity=mon.0 client_cookie=1082d5d5b93e62cd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:10.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.953 7fab0effd700 1 -- 172.21.2.101:0/3827448410 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fab0000c0c0 con 0x7fab101b0470 2021-02-10T11:30:10.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.954 7fab0effd700 1 -- 172.21.2.101:0/3827448410 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0000c440 con 0x7fab101b0470 2021-02-10T11:30:10.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.954 7fab0effd700 1 -- 172.21.2.101:0/3827448410 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fab0000c960 con 0x7fab101b0470 2021-02-10T11:30:10.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.954 7fab16303700 1 -- 172.21.2.101:0/3827448410 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab101b0470 msgr2=0x7fab101b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:10.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.954 7fab16303700 1 --2- 172.21.2.101:0/3827448410 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab101b0470 0x7fab101b0870 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7fab00006700 tx=0x7fab00009b80).stop 2021-02-10T11:30:10.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.954 7fab16303700 1 -- 172.21.2.101:0/3827448410 shutdown_connections 2021-02-10T11:30:10.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.954 7fab16303700 1 --2- 172.21.2.101:0/3827448410 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab100f20e0 0x7fab100f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:10.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.954 7fab16303700 1 --2- 172.21.2.101:0/3827448410 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab101b0470 0x7fab101b0870 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:10.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.954 7fab16303700 1 -- 172.21.2.101:0/3827448410 >> 172.21.2.101:0/3827448410 conn(0x7fab100ce3e0 msgr2=0x7fab100e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:10.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.954 7fab16303700 1 -- 172.21.2.101:0/3827448410 shutdown_connections 2021-02-10T11:30:10.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.954 7fab16303700 1 -- 172.21.2.101:0/3827448410 wait complete. 2021-02-10T11:30:10.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.969 7fab16303700 1 Processor -- start 2021-02-10T11:30:10.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.969 7fab16303700 1 -- start start 2021-02-10T11:30:10.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.969 7fab16303700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab100a0420 0x7fab100a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:10.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.969 7fab16303700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fab10189d00 0x7fab1018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:10.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.969 7fab16303700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab1018a720 0x7fab100b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:10.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.969 7fab16303700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab10189780 con 0x7fab100a0420 2021-02-10T11:30:10.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.969 7fab16303700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1018ab20 con 0x7fab10189d00 2021-02-10T11:30:10.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.969 7fab16303700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab100a1140 con 0x7fab1018a720 2021-02-10T11:30:10.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.969 7fab0ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab100a0420 0x7fab100a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:10.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.970 7fab0ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab100a0420 0x7fab100a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48664/0 (socket says 172.21.2.101:48664) 2021-02-10T11:30:10.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.970 7fab0ffff700 1 -- 172.21.2.101:0/3159295701 learned_addr learned my addr 172.21.2.101:0/3159295701 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:10.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.970 7fab0f7fe700 1 --2- 172.21.2.101:0/3159295701 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fab10189d00 0x7fab1018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:10.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.970 7fab14920700 1 --2- 172.21.2.101:0/3159295701 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab1018a720 0x7fab100b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:10.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.970 7fab0ffff700 1 -- 172.21.2.101:0/3159295701 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab1018a720 msgr2=0x7fab100b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:10.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.970 7fab0ffff700 1 --2- 172.21.2.101:0/3159295701 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab1018a720 0x7fab100b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:10.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.970 7fab0ffff700 1 -- 172.21.2.101:0/3159295701 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fab10189d00 msgr2=0x7fab1018a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:10.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.970 7fab0ffff700 1 --2- 172.21.2.101:0/3159295701 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fab10189d00 0x7fab1018a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:10.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.970 7fab0ffff700 1 -- 172.21.2.101:0/3159295701 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab100b8200 con 0x7fab100a0420 2021-02-10T11:30:10.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.970 7fab0ffff700 1 --2- 172.21.2.101:0/3159295701 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab100a0420 0x7fab100a0820 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7fab0000b5e0 tx=0x7fab0000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:10.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.970 7fab0cff9700 1 -- 172.21.2.101:0/3159295701 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fab0000bd70 con 0x7fab100a0420 2021-02-10T11:30:10.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.970 7fab0cff9700 1 -- 172.21.2.101:0/3159295701 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab00002030 con 0x7fab100a0420 2021-02-10T11:30:10.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.970 7fab16303700 1 -- 172.21.2.101:0/3159295701 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab100b8200 con 0x7fab100a0420 2021-02-10T11:30:10.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.970 7fab0cff9700 1 -- 172.21.2.101:0/3159295701 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fab000024d0 con 0x7fab100a0420 2021-02-10T11:30:10.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.970 7fab16303700 1 -- 172.21.2.101:0/3159295701 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab100b8790 con 0x7fab100a0420 2021-02-10T11:30:10.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.971 7fab0cff9700 1 -- 172.21.2.101:0/3159295701 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fab00000c90 con 0x7fab100a0420 2021-02-10T11:30:10.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.971 7fab0cff9700 1 --2- 172.21.2.101:0/3159295701 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faaf00186d0 0x7faaf001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:10.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.971 7fab0cff9700 1 -- 172.21.2.101:0/3159295701 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fab00032690 con 0x7fab100a0420 2021-02-10T11:30:10.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.971 7fab0f7fe700 1 --2- 172.21.2.101:0/3159295701 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faaf00186d0 0x7faaf001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:10.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:10.971 7fab0f7fe700 1 --2- 172.21.2.101:0/3159295701 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faaf00186d0 0x7faaf001abd0 crc :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:11.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.002 7fab16303700 1 -- 172.21.2.101:0/3159295701 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab100a1360 con 0x7fab100a0420 2021-02-10T11:30:11.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.005 7fab0cff9700 1 -- 172.21.2.101:0/3159295701 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fab000009b0 con 0x7fab100a0420 2021-02-10T11:30:11.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.132 7fab16303700 1 -- 172.21.2.101:0/3159295701 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839256", "target": ["mgr", ""]}) v1 -- 0x7fab100b9020 con 0x7faaf00186d0 2021-02-10T11:30:11.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.142 7fab0cff9700 1 -- 172.21.2.101:0/3159295701 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fab040054b0 con 0x7faaf00186d0 2021-02-10T11:30:11.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.149 7fab16303700 1 -- 172.21.2.101:0/3159295701 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faaf00186d0 msgr2=0x7faaf001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:11.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.149 7fab16303700 1 --2- 172.21.2.101:0/3159295701 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faaf00186d0 0x7faaf001abd0 crc :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:11.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.149 7fab16303700 1 -- 172.21.2.101:0/3159295701 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab100a0420 msgr2=0x7fab100a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:11.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.149 7fab16303700 1 --2- 172.21.2.101:0/3159295701 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab100a0420 0x7fab100a0820 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7fab0000b5e0 tx=0x7fab0000b980).stop 2021-02-10T11:30:11.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.149 7fab16303700 1 -- 172.21.2.101:0/3159295701 shutdown_connections 2021-02-10T11:30:11.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.149 7fab16303700 1 --2- 172.21.2.101:0/3159295701 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faaf00186d0 0x7faaf001abd0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:11.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.149 7fab16303700 1 --2- 172.21.2.101:0/3159295701 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab1018a720 0x7fab100b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:11.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.149 7fab16303700 1 --2- 172.21.2.101:0/3159295701 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fab10189d00 0x7fab1018a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:11.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.149 7fab16303700 1 --2- 172.21.2.101:0/3159295701 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab100a0420 0x7fab100a0820 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:11.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.149 7fab16303700 1 -- 172.21.2.101:0/3159295701 >> 172.21.2.101:0/3159295701 conn(0x7fab10094e10 msgr2=0x7fab100a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:11.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.149 7fab16303700 1 -- 172.21.2.101:0/3159295701 shutdown_connections 2021-02-10T11:30:11.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.149 7fab16303700 1 -- 172.21.2.101:0/3159295701 wait complete. 2021-02-10T11:30:11.169 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839257 2021-02-10T11:30:11.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.573 7f7740cf8700 1 Processor -- start 2021-02-10T11:30:11.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.573 7f7740cf8700 1 -- start start 2021-02-10T11:30:11.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.574 7f7740cf8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f773c1b0470 0x7f773c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:11.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.574 7f7740cf8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f773c0ef610 0x7f773c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:11.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.574 7f7740cf8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f773c0f0b50 con 0x7f773c0f0030 2021-02-10T11:30:11.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.574 7f7740cf8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f773c0e52e0 con 0x7f773c1b0470 2021-02-10T11:30:11.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.574 7f7740cf8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f773c0e5490 con 0x7f773c0ef610 2021-02-10T11:30:11.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.574 7f773ab13700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f773c1b0470 0x7f773c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:11.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.574 7f773ab13700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f773c1b0470 0x7f773c1b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48672/0 (socket says 172.21.2.101:48672) 2021-02-10T11:30:11.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.574 7f773ab13700 1 -- 172.21.2.101:0/3376310061 learned_addr learned my addr 172.21.2.101:0/3376310061 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:11.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.574 7f773a312700 1 --2- 172.21.2.101:0/3376310061 >> v2:172.21.2.108:3300/0 conn(0x7f773c0ef610 0x7f773c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:11.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.575 7f773ab13700 1 -- 172.21.2.101:0/3376310061 >> v2:172.21.2.108:3300/0 conn(0x7f773c0ef610 msgr2=0x7f773c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:11.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.575 7f773ab13700 1 --2- 172.21.2.101:0/3376310061 >> v2:172.21.2.108:3300/0 conn(0x7f773c0ef610 0x7f773c0efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:11.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.575 7f773ab13700 1 -- 172.21.2.101:0/3376310061 >> v1:172.21.2.108:6789/0 conn(0x7f773c0f0030 legacy=0x7f773c0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:11.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.575 7f773ab13700 1 -- 172.21.2.101:0/3376310061 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773c0f0430 con 0x7f773c1b0470 2021-02-10T11:30:11.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.575 7f773ab13700 1 --2- 172.21.2.101:0/3376310061 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f773c1b0470 0x7f773c1b0870 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f7724006700 tx=0x7f7724009b80).ready entity=mon.0 client_cookie=e8c1ffb08deefb68 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:11.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.575 7f7739b11700 1 -- 172.21.2.101:0/3376310061 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f772400c0c0 con 0x7f773c1b0470 2021-02-10T11:30:11.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.575 7f7739b11700 1 -- 172.21.2.101:0/3376310061 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772400c440 con 0x7f773c1b0470 2021-02-10T11:30:11.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.575 7f7739b11700 1 -- 172.21.2.101:0/3376310061 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f772400c8a0 con 0x7f773c1b0470 2021-02-10T11:30:11.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.575 7f7740cf8700 1 -- 172.21.2.101:0/3376310061 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f773c1b0470 msgr2=0x7f773c1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:11.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.575 7f7740cf8700 1 --2- 172.21.2.101:0/3376310061 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f773c1b0470 0x7f773c1b0870 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f7724006700 tx=0x7f7724009b80).stop 2021-02-10T11:30:11.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.576 7f7740cf8700 1 -- 172.21.2.101:0/3376310061 shutdown_connections 2021-02-10T11:30:11.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.576 7f7740cf8700 1 --2- 172.21.2.101:0/3376310061 >> v2:172.21.2.108:3300/0 conn(0x7f773c0ef610 0x7f773c0efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:11.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.576 7f7740cf8700 1 --2- 172.21.2.101:0/3376310061 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f773c1b0470 0x7f773c1b0870 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:11.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.576 7f7740cf8700 1 -- 172.21.2.101:0/3376310061 >> 172.21.2.101:0/3376310061 conn(0x7f773c0ce3e0 msgr2=0x7f773c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:11.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.576 7f7740cf8700 1 -- 172.21.2.101:0/3376310061 shutdown_connections 2021-02-10T11:30:11.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.576 7f7740cf8700 1 -- 172.21.2.101:0/3376310061 wait complete. 2021-02-10T11:30:11.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.591 7f7740cf8700 1 Processor -- start 2021-02-10T11:30:11.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.591 7f7740cf8700 1 -- start start 2021-02-10T11:30:11.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.591 7f7740cf8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f773c0a0420 0x7f773c0a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:11.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.591 7f7740cf8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f773c189d00 0x7f773c18a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:11.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.591 7f7740cf8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f773c18a720 0x7f773c0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:11.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.591 7f7740cf8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c189780 con 0x7f773c0a0420 2021-02-10T11:30:11.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.591 7f7740cf8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c18ab20 con 0x7f773c189d00 2021-02-10T11:30:11.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.591 7f7740cf8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773c0a1140 con 0x7f773c18a720 2021-02-10T11:30:11.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.591 7f773ab13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f773c0a0420 0x7f773c0a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:11.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.591 7f773ab13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f773c0a0420 0x7f773c0a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48678/0 (socket says 172.21.2.101:48678) 2021-02-10T11:30:11.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.591 7f773ab13700 1 -- 172.21.2.101:0/965972514 learned_addr learned my addr 172.21.2.101:0/965972514 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:11.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.591 7f773b314700 1 --2- 172.21.2.101:0/965972514 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f773c18a720 0x7f773c0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:11.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.591 7f773a312700 1 --2- 172.21.2.101:0/965972514 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f773c189d00 0x7f773c18a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:11.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.592 7f773ab13700 1 -- 172.21.2.101:0/965972514 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f773c18a720 msgr2=0x7f773c0b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:11.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.592 7f773ab13700 1 --2- 172.21.2.101:0/965972514 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f773c18a720 0x7f773c0b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:11.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.592 7f773ab13700 1 -- 172.21.2.101:0/965972514 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f773c189d00 msgr2=0x7f773c18a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:11.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.592 7f773ab13700 1 --2- 172.21.2.101:0/965972514 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f773c189d00 0x7f773c18a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:11.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.592 7f773ab13700 1 -- 172.21.2.101:0/965972514 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773c0b8200 con 0x7f773c0a0420 2021-02-10T11:30:11.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.592 7f773ab13700 1 --2- 172.21.2.101:0/965972514 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f773c0a0420 0x7f773c0a0820 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f772400b5e0 tx=0x7f772400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:11.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.592 7f77337fe700 1 -- 172.21.2.101:0/965972514 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f772400bd70 con 0x7f773c0a0420 2021-02-10T11:30:11.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.592 7f77337fe700 1 -- 172.21.2.101:0/965972514 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7724002030 con 0x7f773c0a0420 2021-02-10T11:30:11.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.592 7f7740cf8700 1 -- 172.21.2.101:0/965972514 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f773c0b8200 con 0x7f773c0a0420 2021-02-10T11:30:11.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.592 7f77337fe700 1 -- 172.21.2.101:0/965972514 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7724002550 con 0x7f773c0a0420 2021-02-10T11:30:11.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.592 7f7740cf8700 1 -- 172.21.2.101:0/965972514 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773c0b8790 con 0x7f773c0a0420 2021-02-10T11:30:11.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.593 7f77337fe700 1 -- 172.21.2.101:0/965972514 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7724002210 con 0x7f773c0a0420 2021-02-10T11:30:11.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.593 7f77337fe700 1 --2- 172.21.2.101:0/965972514 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77200186c0 0x7f772001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:11.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.593 7f77337fe700 1 -- 172.21.2.101:0/965972514 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f77240323e0 con 0x7f773c0a0420 2021-02-10T11:30:11.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.593 7f773a312700 1 --2- 172.21.2.101:0/965972514 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77200186c0 0x7f772001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:11.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.593 7f773a312700 1 --2- 172.21.2.101:0/965972514 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77200186c0 0x7f772001abc0 crc :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:11.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.622 7f7740cf8700 1 -- 172.21.2.101:0/965972514 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f773c105ae0 con 0x7f773c0a0420 2021-02-10T11:30:11.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.624 7f77337fe700 1 -- 172.21.2.101:0/965972514 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f77240009b0 con 0x7f773c0a0420 2021-02-10T11:30:11.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.751 7f7740cf8700 1 -- 172.21.2.101:0/965972514 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839257", "target": ["mgr", ""]}) v1 -- 0x7f773c0b8790 con 0x7f77200186c0 2021-02-10T11:30:11.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.762 7f77337fe700 1 -- 172.21.2.101:0/965972514 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f772c006a70 con 0x7f77200186c0 2021-02-10T11:30:11.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.768 7f7740cf8700 1 -- 172.21.2.101:0/965972514 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77200186c0 msgr2=0x7f772001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:11.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.768 7f7740cf8700 1 --2- 172.21.2.101:0/965972514 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77200186c0 0x7f772001abc0 crc :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:11.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.768 7f7740cf8700 1 -- 172.21.2.101:0/965972514 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f773c0a0420 msgr2=0x7f773c0a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:11.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.768 7f7740cf8700 1 --2- 172.21.2.101:0/965972514 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f773c0a0420 0x7f773c0a0820 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f772400b5e0 tx=0x7f772400b980).stop 2021-02-10T11:30:11.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.768 7f7740cf8700 1 -- 172.21.2.101:0/965972514 shutdown_connections 2021-02-10T11:30:11.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.768 7f7740cf8700 1 --2- 172.21.2.101:0/965972514 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77200186c0 0x7f772001abc0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:11.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.768 7f7740cf8700 1 --2- 172.21.2.101:0/965972514 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f773c18a720 0x7f773c0b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:11.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.768 7f7740cf8700 1 --2- 172.21.2.101:0/965972514 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f773c189d00 0x7f773c18a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:11.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.768 7f7740cf8700 1 --2- 172.21.2.101:0/965972514 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f773c0a0420 0x7f773c0a0820 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:11.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.768 7f7740cf8700 1 -- 172.21.2.101:0/965972514 >> 172.21.2.101:0/965972514 conn(0x7f773c094e10 msgr2=0x7f773c0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:11.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.768 7f7740cf8700 1 -- 172.21.2.101:0/965972514 shutdown_connections 2021-02-10T11:30:11.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:11.768 7f7740cf8700 1 -- 172.21.2.101:0/965972514 wait complete. 2021-02-10T11:30:11.788 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839258 2021-02-10T11:30:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.192 7f6df5767700 1 Processor -- start 2021-02-10T11:30:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.192 7f6df5767700 1 -- start start 2021-02-10T11:30:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.192 7f6df5767700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6df01b0870 0x7f6df00f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.192 7f6df5767700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df01b0c70 con 0x7f6df00efe00 2021-02-10T11:30:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.192 7f6df5767700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df00f0b50 con 0x7f6df01b0470 2021-02-10T11:30:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.192 7f6df5767700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df00e5ea0 con 0x7f6df01b0870 2021-02-10T11:30:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.192 7f6defd75700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f6df00efe00 0x7f6df00f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49184/0 (socket says 172.21.2.101:49184) 2021-02-10T11:30:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.192 7f6defd75700 1 -- 172.21.2.101:0/2535322100 learned_addr learned my addr 172.21.2.101:0/2535322100 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:12.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.193 7f6deed73700 1 --2- 172.21.2.101:0/2535322100 >> v2:172.21.2.108:3300/0 conn(0x7f6df01b0870 0x7f6df00f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:12.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.193 7f6dee572700 1 -- 172.21.2.101:0/2535322100 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799571034 0 0) 0x7f6de8001500 con 0x7f6df00efe00 2021-02-10T11:30:12.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.193 7f6dee572700 1 -- 172.21.2.101:0/2535322100 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ddc002fe0 con 0x7f6df00efe00 2021-02-10T11:30:12.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.193 7f6deed73700 1 -- 172.21.2.101:0/2535322100 >> v1:172.21.2.108:6789/0 conn(0x7f6df01b0470 legacy=0x7f6df00ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:12.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.193 7f6deed73700 1 -- 172.21.2.101:0/2535322100 >> v1:172.21.2.101:6789/0 conn(0x7f6df00efe00 legacy=0x7f6df00f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:12.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.193 7f6deed73700 1 -- 172.21.2.101:0/2535322100 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df00e52e0 con 0x7f6df01b0870 2021-02-10T11:30:12.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.193 7f6deed73700 1 --2- 172.21.2.101:0/2535322100 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6df01b0870 0x7f6df00f20e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f6de0006650 tx=0x7f6de0009a80).ready entity=mon.1 client_cookie=53d1f336aa268dd4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:12.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.193 7f6dee572700 1 -- 172.21.2.101:0/2535322100 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6de000c0c0 con 0x7f6df01b0870 2021-02-10T11:30:12.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.193 7f6dee572700 1 -- 172.21.2.101:0/2535322100 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de000c340 con 0x7f6df01b0870 2021-02-10T11:30:12.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.193 7f6dee572700 1 -- 172.21.2.101:0/2535322100 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6de000c860 con 0x7f6df01b0870 2021-02-10T11:30:12.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.193 7f6df5767700 1 -- 172.21.2.101:0/2535322100 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6df01b0870 msgr2=0x7f6df00f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:12.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.193 7f6df5767700 1 --2- 172.21.2.101:0/2535322100 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6df01b0870 0x7f6df00f20e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f6de0006650 tx=0x7f6de0009a80).stop 2021-02-10T11:30:12.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.194 7f6df5767700 1 -- 172.21.2.101:0/2535322100 shutdown_connections 2021-02-10T11:30:12.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.194 7f6df5767700 1 --2- 172.21.2.101:0/2535322100 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6df01b0870 0x7f6df00f20e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:12.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.194 7f6df5767700 1 -- 172.21.2.101:0/2535322100 >> 172.21.2.101:0/2535322100 conn(0x7f6df00ce3e0 msgr2=0x7f6df00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:12.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.194 7f6df5767700 1 -- 172.21.2.101:0/2535322100 shutdown_connections 2021-02-10T11:30:12.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.194 7f6df5767700 1 -- 172.21.2.101:0/2535322100 wait complete. 2021-02-10T11:30:12.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.209 7f6df5767700 1 Processor -- start 2021-02-10T11:30:12.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.209 7f6df5767700 1 -- start start 2021-02-10T11:30:12.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.209 7f6df5767700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6df00b2840 0x7f6df01a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:12.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.209 7f6df5767700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6df00b30d0 0x7f6df00b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:12.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.209 7f6df5767700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6df00b3af0 0x7f6df00f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:12.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.209 7f6df5767700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df00b2c40 con 0x7f6df00b2840 2021-02-10T11:30:12.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.209 7f6df5767700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df01a4920 con 0x7f6df00b30d0 2021-02-10T11:30:12.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.209 7f6df5767700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df00f0ba0 con 0x7f6df00b3af0 2021-02-10T11:30:12.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.209 7f6def574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6df00b2840 0x7f6df01a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:12.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.210 7f6def574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6df00b2840 0x7f6df01a4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48696/0 (socket says 172.21.2.101:48696) 2021-02-10T11:30:12.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.210 7f6def574700 1 -- 172.21.2.101:0/3709729914 learned_addr learned my addr 172.21.2.101:0/3709729914 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:12.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.210 7f6defd75700 1 --2- 172.21.2.101:0/3709729914 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6df00b3af0 0x7f6df00f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:12.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.210 7f6deed73700 1 --2- 172.21.2.101:0/3709729914 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6df00b30d0 0x7f6df00b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:12.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.210 7f6deed73700 1 -- 172.21.2.101:0/3709729914 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6df00b3af0 msgr2=0x7f6df00f0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:12.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.210 7f6deed73700 1 --2- 172.21.2.101:0/3709729914 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6df00b3af0 0x7f6df00f0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:12.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.210 7f6deed73700 1 -- 172.21.2.101:0/3709729914 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6df00b2840 msgr2=0x7f6df01a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:12.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.210 7f6deed73700 1 --2- 172.21.2.101:0/3709729914 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6df00b2840 0x7f6df01a4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:12.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.210 7f6deed73700 1 -- 172.21.2.101:0/3709729914 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df00f0dc0 con 0x7f6df00b30d0 2021-02-10T11:30:12.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.210 7f6def574700 1 --2- 172.21.2.101:0/3709729914 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6df00b2840 0x7f6df01a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:12.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.210 7f6deed73700 1 --2- 172.21.2.101:0/3709729914 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6df00b30d0 0x7f6df00b34d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f6de0006f20 tx=0x7f6de000b8c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:12.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.210 7f6de7fff700 1 -- 172.21.2.101:0/3709729914 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6de000bcb0 con 0x7f6df00b30d0 2021-02-10T11:30:12.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.210 7f6de7fff700 1 -- 172.21.2.101:0/3709729914 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de00052e0 con 0x7f6df00b30d0 2021-02-10T11:30:12.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.210 7f6df5767700 1 -- 172.21.2.101:0/3709729914 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df00f0dc0 con 0x7f6df00b30d0 2021-02-10T11:30:12.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.210 7f6de7fff700 1 -- 172.21.2.101:0/3709729914 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6de0005800 con 0x7f6df00b30d0 2021-02-10T11:30:12.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.210 7f6df5767700 1 -- 172.21.2.101:0/3709729914 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df00f0dc0 con 0x7f6df00b30d0 2021-02-10T11:30:12.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.211 7f6de7fff700 1 -- 172.21.2.101:0/3709729914 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6de000a000 con 0x7f6df00b30d0 2021-02-10T11:30:12.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.211 7f6de7fff700 1 --2- 172.21.2.101:0/3709729914 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6dd40186c0 0x7f6dd401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:12.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.211 7f6de7fff700 1 -- 172.21.2.101:0/3709729914 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6de0032340 con 0x7f6df00b30d0 2021-02-10T11:30:12.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.211 7f6def574700 1 --2- 172.21.2.101:0/3709729914 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6dd40186c0 0x7f6dd401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:12.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.211 7f6def574700 1 --2- 172.21.2.101:0/3709729914 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6dd40186c0 0x7f6dd401abc0 crc :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:12.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.241 7f6df5767700 1 -- 172.21.2.101:0/3709729914 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df00f0050 con 0x7f6df00b30d0 2021-02-10T11:30:12.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.244 7f6de7fff700 1 -- 172.21.2.101:0/3709729914 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6de0025ad0 con 0x7f6df00b30d0 2021-02-10T11:30:12.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.374 7f6df5767700 1 -- 172.21.2.101:0/3709729914 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839258", "target": ["mgr", ""]}) v1 -- 0x7f6df00f0050 con 0x7f6dd40186c0 2021-02-10T11:30:12.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.384 7f6de7fff700 1 -- 172.21.2.101:0/3709729914 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f6dd8006a80 con 0x7f6dd40186c0 2021-02-10T11:30:12.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.390 7f6df5767700 1 -- 172.21.2.101:0/3709729914 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6dd40186c0 msgr2=0x7f6dd401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:12.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.390 7f6df5767700 1 --2- 172.21.2.101:0/3709729914 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6dd40186c0 0x7f6dd401abc0 crc :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:12.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.390 7f6df5767700 1 -- 172.21.2.101:0/3709729914 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6df00b30d0 msgr2=0x7f6df00b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:12.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.390 7f6df5767700 1 --2- 172.21.2.101:0/3709729914 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6df00b30d0 0x7f6df00b34d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f6de0006f20 tx=0x7f6de000b8c0).stop 2021-02-10T11:30:12.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.390 7f6df5767700 1 -- 172.21.2.101:0/3709729914 shutdown_connections 2021-02-10T11:30:12.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.390 7f6df5767700 1 --2- 172.21.2.101:0/3709729914 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6dd40186c0 0x7f6dd401abc0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:12.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.390 7f6df5767700 1 --2- 172.21.2.101:0/3709729914 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6df00b3af0 0x7f6df00f0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:12.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.390 7f6df5767700 1 --2- 172.21.2.101:0/3709729914 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6df00b30d0 0x7f6df00b34d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:12.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.390 7f6df5767700 1 --2- 172.21.2.101:0/3709729914 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6df00b2840 0x7f6df01a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:12.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.391 7f6df5767700 1 -- 172.21.2.101:0/3709729914 >> 172.21.2.101:0/3709729914 conn(0x7f6df00c36f0 msgr2=0x7f6df0094c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:12.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.391 7f6df5767700 1 -- 172.21.2.101:0/3709729914 shutdown_connections 2021-02-10T11:30:12.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.391 7f6df5767700 1 -- 172.21.2.101:0/3709729914 wait complete. 2021-02-10T11:30:12.411 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839259 2021-02-10T11:30:12.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.814 7ff64fd71700 1 Processor -- start 2021-02-10T11:30:12.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.815 7ff64fd71700 1 -- start start 2021-02-10T11:30:12.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.815 7ff64fd71700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff6481b0470 0x7ff6481b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:12.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.815 7ff64fd71700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6480f0b50 con 0x7ff6480ef610 2021-02-10T11:30:12.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.815 7ff64fd71700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6480f24e0 con 0x7ff6480f20e0 2021-02-10T11:30:12.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.815 7ff64fd71700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6480e5ea0 con 0x7ff6481b0470 2021-02-10T11:30:12.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.815 7ff64db8d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff6481b0470 0x7ff6481b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:12.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.815 7ff64db8d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff6481b0470 0x7ff6481b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48700/0 (socket says 172.21.2.101:48700) 2021-02-10T11:30:12.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.815 7ff64d38c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff6480ef610 0x7ff6480efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49194/0 (socket says 172.21.2.101:49194) 2021-02-10T11:30:12.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.815 7ff64db8d700 1 -- 172.21.2.101:0/1728666318 learned_addr learned my addr 172.21.2.101:0/1728666318 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:12.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.816 7ff64db8d700 1 -- 172.21.2.101:0/1728666318 >> v1:172.21.2.108:6789/0 conn(0x7ff6480f20e0 legacy=0x7ff6480f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:12.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.816 7ff64db8d700 1 -- 172.21.2.101:0/1728666318 >> v1:172.21.2.101:6789/0 conn(0x7ff6480ef610 legacy=0x7ff6480efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:12.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.816 7ff64db8d700 1 -- 172.21.2.101:0/1728666318 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6480e52e0 con 0x7ff6481b0470 2021-02-10T11:30:12.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.816 7ff64db8d700 1 --2- 172.21.2.101:0/1728666318 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6481b0470 0x7ff6481b0870 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7ff638006700 tx=0x7ff638009b80).ready entity=mon.0 client_cookie=d1626ae5e329ffc0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:12.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.816 7ff64cb8b700 1 -- 172.21.2.101:0/1728666318 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff63800c0c0 con 0x7ff6481b0470 2021-02-10T11:30:12.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.816 7ff64cb8b700 1 -- 172.21.2.101:0/1728666318 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63800c340 con 0x7ff6481b0470 2021-02-10T11:30:12.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.816 7ff64cb8b700 1 -- 172.21.2.101:0/1728666318 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff63800c860 con 0x7ff6481b0470 2021-02-10T11:30:12.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.816 7ff64fd71700 1 -- 172.21.2.101:0/1728666318 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6481b0470 msgr2=0x7ff6481b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:12.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.816 7ff64fd71700 1 --2- 172.21.2.101:0/1728666318 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6481b0470 0x7ff6481b0870 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7ff638006700 tx=0x7ff638009b80).stop 2021-02-10T11:30:12.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.817 7ff64fd71700 1 -- 172.21.2.101:0/1728666318 shutdown_connections 2021-02-10T11:30:12.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.817 7ff64fd71700 1 --2- 172.21.2.101:0/1728666318 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6481b0470 0x7ff6481b0870 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:12.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.817 7ff64fd71700 1 -- 172.21.2.101:0/1728666318 >> 172.21.2.101:0/1728666318 conn(0x7ff6480ce3e0 msgr2=0x7ff6480e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:12.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.817 7ff64fd71700 1 -- 172.21.2.101:0/1728666318 shutdown_connections 2021-02-10T11:30:12.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.817 7ff64fd71700 1 -- 172.21.2.101:0/1728666318 wait complete. 2021-02-10T11:30:12.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.831 7ff64fd71700 1 Processor -- start 2021-02-10T11:30:12.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.831 7ff64fd71700 1 -- start start 2021-02-10T11:30:12.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.832 7ff64fd71700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff648087510 0x7ff6480aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:12.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.832 7ff64fd71700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6481c68f0 0x7ff6481b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:12.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.832 7ff64fd71700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff64808c4d0 0x7ff6481b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:12.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.832 7ff64fd71700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6480a2d50 con 0x7ff648087510 2021-02-10T11:30:12.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.832 7ff64fd71700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6480a2f70 con 0x7ff6481c68f0 2021-02-10T11:30:12.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.832 7ff64fd71700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6480a3190 con 0x7ff64808c4d0 2021-02-10T11:30:12.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.832 7ff64db8d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff648087510 0x7ff6480aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:12.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.832 7ff64db8d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff648087510 0x7ff6480aedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48704/0 (socket says 172.21.2.101:48704) 2021-02-10T11:30:12.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.832 7ff64db8d700 1 -- 172.21.2.101:0/3964280450 learned_addr learned my addr 172.21.2.101:0/3964280450 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:12.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.832 7ff64e38e700 1 --2- 172.21.2.101:0/3964280450 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff64808c4d0 0x7ff6481b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:12.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.832 7ff64d38c700 1 --2- 172.21.2.101:0/3964280450 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6481c68f0 0x7ff6481b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:12.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.832 7ff64db8d700 1 -- 172.21.2.101:0/3964280450 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff64808c4d0 msgr2=0x7ff6481b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:12.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.833 7ff64db8d700 1 --2- 172.21.2.101:0/3964280450 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff64808c4d0 0x7ff6481b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:12.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.833 7ff64db8d700 1 -- 172.21.2.101:0/3964280450 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6481c68f0 msgr2=0x7ff6481b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:12.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.833 7ff64db8d700 1 --2- 172.21.2.101:0/3964280450 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6481c68f0 0x7ff6481b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:12.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.833 7ff64db8d700 1 -- 172.21.2.101:0/3964280450 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6480a33b0 con 0x7ff648087510 2021-02-10T11:30:12.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.833 7ff64db8d700 1 --2- 172.21.2.101:0/3964280450 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff648087510 0x7ff6480aedc0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7ff63800b580 tx=0x7ff63800b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:12.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.833 7ff63e7fc700 1 -- 172.21.2.101:0/3964280450 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff63800bd10 con 0x7ff648087510 2021-02-10T11:30:12.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.833 7ff63e7fc700 1 -- 172.21.2.101:0/3964280450 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff638002030 con 0x7ff648087510 2021-02-10T11:30:12.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.833 7ff63e7fc700 1 -- 172.21.2.101:0/3964280450 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff638002400 con 0x7ff648087510 2021-02-10T11:30:12.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.833 7ff64fd71700 1 -- 172.21.2.101:0/3964280450 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6480a33b0 con 0x7ff648087510 2021-02-10T11:30:12.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.833 7ff64fd71700 1 -- 172.21.2.101:0/3964280450 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6480a33b0 con 0x7ff648087510 2021-02-10T11:30:12.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.834 7ff63e7fc700 1 -- 172.21.2.101:0/3964280450 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff638000c90 con 0x7ff648087510 2021-02-10T11:30:12.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.834 7ff63e7fc700 1 --2- 172.21.2.101:0/3964280450 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6300186c0 0x7ff63001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:12.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.834 7ff63e7fc700 1 -- 172.21.2.101:0/3964280450 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff638032430 con 0x7ff648087510 2021-02-10T11:30:12.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.834 7ff64d38c700 1 --2- 172.21.2.101:0/3964280450 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6300186c0 0x7ff63001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:12.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.834 7ff64d38c700 1 --2- 172.21.2.101:0/3964280450 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6300186c0 0x7ff63001abc0 crc :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:12.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.865 7ff64fd71700 1 -- 172.21.2.101:0/3964280450 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff64807e6b0 con 0x7ff648087510 2021-02-10T11:30:12.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.867 7ff63e7fc700 1 -- 172.21.2.101:0/3964280450 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff638030d60 con 0x7ff648087510 2021-02-10T11:30:13.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:12.993 7ff64fd71700 1 -- 172.21.2.101:0/3964280450 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839259", "target": ["mgr", ""]}) v1 -- 0x7ff64807e6b0 con 0x7ff6300186c0 2021-02-10T11:30:13.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.004 7ff63e7fc700 1 -- 172.21.2.101:0/3964280450 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7ff640007770 con 0x7ff6300186c0 2021-02-10T11:30:13.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.010 7ff64fd71700 1 -- 172.21.2.101:0/3964280450 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6300186c0 msgr2=0x7ff63001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:13.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.010 7ff64fd71700 1 --2- 172.21.2.101:0/3964280450 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6300186c0 0x7ff63001abc0 crc :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:13.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.010 7ff64fd71700 1 -- 172.21.2.101:0/3964280450 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff648087510 msgr2=0x7ff6480aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:13.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.010 7ff64fd71700 1 --2- 172.21.2.101:0/3964280450 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff648087510 0x7ff6480aedc0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7ff63800b580 tx=0x7ff63800b920).stop 2021-02-10T11:30:13.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.010 7ff64fd71700 1 -- 172.21.2.101:0/3964280450 shutdown_connections 2021-02-10T11:30:13.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.010 7ff64fd71700 1 --2- 172.21.2.101:0/3964280450 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6300186c0 0x7ff63001abc0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:13.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.010 7ff64fd71700 1 --2- 172.21.2.101:0/3964280450 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff64808c4d0 0x7ff6481b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:13.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.010 7ff64fd71700 1 --2- 172.21.2.101:0/3964280450 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6481c68f0 0x7ff6481b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:13.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.010 7ff64fd71700 1 --2- 172.21.2.101:0/3964280450 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff648087510 0x7ff6480aedc0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:13.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.010 7ff64fd71700 1 -- 172.21.2.101:0/3964280450 >> 172.21.2.101:0/3964280450 conn(0x7ff6480886d0 msgr2=0x7ff648081c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:13.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.010 7ff64fd71700 1 -- 172.21.2.101:0/3964280450 shutdown_connections 2021-02-10T11:30:13.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.010 7ff64fd71700 1 -- 172.21.2.101:0/3964280450 wait complete. 2021-02-10T11:30:13.030 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839260 2021-02-10T11:30:13.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.434 7f6ef5050700 1 Processor -- start 2021-02-10T11:30:13.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.434 7f6ef5050700 1 -- start start 2021-02-10T11:30:13.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.435 7f6ef5050700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6ef01b0870 0x7f6ef00f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:13.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.435 7f6ef5050700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef01b0c70 con 0x7f6ef01b0470 2021-02-10T11:30:13.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.435 7f6ef5050700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef00f0b50 con 0x7f6ef00efe00 2021-02-10T11:30:13.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.435 7f6ef5050700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ef00e5ea0 con 0x7f6ef01b0870 2021-02-10T11:30:13.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.435 7f6eee59c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6ef01b0870 0x7f6ef00f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:13.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.435 7f6eeed9d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f6ef01b0470 0x7f6ef00ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49206/0 (socket says 172.21.2.101:49206) 2021-02-10T11:30:13.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.435 7f6eeed9d700 1 -- 172.21.2.101:0/1717976376 learned_addr learned my addr 172.21.2.101:0/1717976376 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:13.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.435 7f6eee59c700 1 --2- 172.21.2.101:0/1717976376 >> v2:172.21.2.101:3300/0 conn(0x7f6ef01b0870 0x7f6ef00f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48714/0 (socket says 172.21.2.101:48714) 2021-02-10T11:30:13.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.436 7f6eee59c700 1 -- 172.21.2.101:0/1717976376 >> v1:172.21.2.108:6789/0 conn(0x7f6ef00efe00 legacy=0x7f6ef00f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:13.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.436 7f6eee59c700 1 -- 172.21.2.101:0/1717976376 >> v1:172.21.2.101:6789/0 conn(0x7f6ef01b0470 legacy=0x7f6ef00ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:13.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.436 7f6eee59c700 1 -- 172.21.2.101:0/1717976376 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef00e52e0 con 0x7f6ef01b0870 2021-02-10T11:30:13.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.436 7f6eedd9b700 1 -- 172.21.2.101:0/1717976376 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112078002 0 0) 0x7f6ee0001180 con 0x7f6ef01b0470 2021-02-10T11:30:13.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.436 7f6eee59c700 1 --2- 172.21.2.101:0/1717976376 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ef01b0870 0x7f6ef00f20e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7f6ee4006650 tx=0x7f6ee4009a80).ready entity=mon.0 client_cookie=316fd8900626ffa2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:13.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.436 7f6eedd9b700 1 -- 172.21.2.101:0/1717976376 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ee40009b0 con 0x7f6ef01b0870 2021-02-10T11:30:13.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.436 7f6eedd9b700 1 -- 172.21.2.101:0/1717976376 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee400c0c0 con 0x7f6ef01b0870 2021-02-10T11:30:13.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.436 7f6eedd9b700 1 -- 172.21.2.101:0/1717976376 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ee400c510 con 0x7f6ef01b0870 2021-02-10T11:30:13.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.436 7f6ef5050700 1 -- 172.21.2.101:0/1717976376 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ef01b0870 msgr2=0x7f6ef00f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:13.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.436 7f6ef5050700 1 --2- 172.21.2.101:0/1717976376 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ef01b0870 0x7f6ef00f20e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7f6ee4006650 tx=0x7f6ee4009a80).stop 2021-02-10T11:30:13.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.437 7f6ef5050700 1 -- 172.21.2.101:0/1717976376 shutdown_connections 2021-02-10T11:30:13.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.437 7f6ef5050700 1 --2- 172.21.2.101:0/1717976376 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ef01b0870 0x7f6ef00f20e0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:13.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.437 7f6ef5050700 1 -- 172.21.2.101:0/1717976376 >> 172.21.2.101:0/1717976376 conn(0x7f6ef00ce3e0 msgr2=0x7f6ef00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:13.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.437 7f6ef5050700 1 -- 172.21.2.101:0/1717976376 shutdown_connections 2021-02-10T11:30:13.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.437 7f6ef5050700 1 -- 172.21.2.101:0/1717976376 wait complete. 2021-02-10T11:30:13.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.452 7f6ef5050700 1 Processor -- start 2021-02-10T11:30:13.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.452 7f6ef5050700 1 -- start start 2021-02-10T11:30:13.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.452 7f6ef5050700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ef0097a10 0x7f6ef0097e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:13.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.452 7f6ef5050700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6ef008a430 0x7f6ef008a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:13.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.452 7f6ef5050700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ef008ae50 0x7f6ef008ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:13.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.452 7f6ef5050700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef008b480 con 0x7f6ef0097a10 2021-02-10T11:30:13.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.452 7f6ef5050700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef008c020 con 0x7f6ef008a430 2021-02-10T11:30:13.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.452 7f6ef5050700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef008c200 con 0x7f6ef008ae50 2021-02-10T11:30:13.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.452 7f6eef59e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ef008ae50 0x7f6ef008ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:13.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.452 7f6eeed9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ef0097a10 0x7f6ef0097e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:13.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.452 7f6eef59e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ef008ae50 0x7f6ef008ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:41452/0 (socket says 172.21.2.101:41452) 2021-02-10T11:30:13.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.452 7f6eef59e700 1 -- 172.21.2.101:0/3912067302 learned_addr learned my addr 172.21.2.101:0/3912067302 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:13.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.452 7f6eee59c700 1 --2- 172.21.2.101:0/3912067302 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6ef008a430 0x7f6ef008a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:13.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.453 7f6eee59c700 1 -- 172.21.2.101:0/3912067302 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ef008ae50 msgr2=0x7f6ef008ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:13.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.453 7f6eee59c700 1 --2- 172.21.2.101:0/3912067302 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ef008ae50 0x7f6ef008ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:13.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.453 7f6eee59c700 1 -- 172.21.2.101:0/3912067302 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ef0097a10 msgr2=0x7f6ef0097e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:13.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.453 7f6eee59c700 1 --2- 172.21.2.101:0/3912067302 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ef0097a10 0x7f6ef0097e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:13.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.453 7f6eee59c700 1 -- 172.21.2.101:0/3912067302 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef008c420 con 0x7f6ef008a430 2021-02-10T11:30:13.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.453 7f6eef59e700 1 --2- 172.21.2.101:0/3912067302 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ef008ae50 0x7f6ef008ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:13.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.453 7f6eee59c700 1 --2- 172.21.2.101:0/3912067302 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6ef008a430 0x7f6ef008a830 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f6ee400b530 tx=0x7f6ee400b8d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:13.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.453 7f6edf7fe700 1 -- 172.21.2.101:0/3912067302 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ee4000a10 con 0x7f6ef008a430 2021-02-10T11:30:13.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.453 7f6ef5050700 1 -- 172.21.2.101:0/3912067302 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef008c420 con 0x7f6ef008a430 2021-02-10T11:30:13.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.453 7f6edf7fe700 1 -- 172.21.2.101:0/3912067302 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee4006740 con 0x7f6ef008a430 2021-02-10T11:30:13.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.453 7f6edf7fe700 1 -- 172.21.2.101:0/3912067302 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ee400bcc0 con 0x7f6ef008a430 2021-02-10T11:30:13.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.453 7f6ef5050700 1 -- 172.21.2.101:0/3912067302 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef008c420 con 0x7f6ef008a430 2021-02-10T11:30:13.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.454 7f6edf7fe700 1 -- 172.21.2.101:0/3912067302 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6ee4009280 con 0x7f6ef008a430 2021-02-10T11:30:13.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.454 7f6edf7fe700 1 --2- 172.21.2.101:0/3912067302 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ed4018730 0x7f6ed401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:13.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.454 7f6edf7fe700 1 -- 172.21.2.101:0/3912067302 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6ee4032770 con 0x7f6ef008a430 2021-02-10T11:30:13.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.454 7f6eeed9d700 1 --2- 172.21.2.101:0/3912067302 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ed4018730 0x7f6ed401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:13.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.454 7f6eeed9d700 1 --2- 172.21.2.101:0/3912067302 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ed4018730 0x7f6ed401abc0 crc :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:13.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.484 7f6ef5050700 1 -- 172.21.2.101:0/3912067302 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef00e52e0 con 0x7f6ef008a430 2021-02-10T11:30:13.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.486 7f6edf7fe700 1 -- 172.21.2.101:0/3912067302 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6ee4006740 con 0x7f6ef008a430 2021-02-10T11:30:13.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.615 7f6ef5050700 1 -- 172.21.2.101:0/3912067302 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839260", "target": ["mgr", ""]}) v1 -- 0x7f6ef00e52e0 con 0x7f6ed4018730 2021-02-10T11:30:13.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.624 7f6edf7fe700 1 -- 172.21.2.101:0/3912067302 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f6ee0001070 con 0x7f6ed4018730 2021-02-10T11:30:13.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.632 7f6ef5050700 1 -- 172.21.2.101:0/3912067302 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ed4018730 msgr2=0x7f6ed401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:13.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.632 7f6ef5050700 1 --2- 172.21.2.101:0/3912067302 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ed4018730 0x7f6ed401abc0 crc :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:13.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.632 7f6ef5050700 1 -- 172.21.2.101:0/3912067302 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6ef008a430 msgr2=0x7f6ef008a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:13.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.632 7f6ef5050700 1 --2- 172.21.2.101:0/3912067302 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6ef008a430 0x7f6ef008a830 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f6ee400b530 tx=0x7f6ee400b8d0).stop 2021-02-10T11:30:13.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.632 7f6ef5050700 1 -- 172.21.2.101:0/3912067302 shutdown_connections 2021-02-10T11:30:13.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.632 7f6ef5050700 1 --2- 172.21.2.101:0/3912067302 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ed4018730 0x7f6ed401abc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:13.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.632 7f6ef5050700 1 --2- 172.21.2.101:0/3912067302 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ef008ae50 0x7f6ef008ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:13.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.632 7f6ef5050700 1 --2- 172.21.2.101:0/3912067302 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6ef008a430 0x7f6ef008a830 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:13.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.632 7f6ef5050700 1 --2- 172.21.2.101:0/3912067302 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ef0097a10 0x7f6ef0097e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:13.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.632 7f6ef5050700 1 -- 172.21.2.101:0/3912067302 >> 172.21.2.101:0/3912067302 conn(0x7f6ef00a73f0 msgr2=0x7f6ef009a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:13.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.632 7f6ef5050700 1 -- 172.21.2.101:0/3912067302 shutdown_connections 2021-02-10T11:30:13.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:13.632 7f6ef5050700 1 -- 172.21.2.101:0/3912067302 wait complete. 2021-02-10T11:30:13.652 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839261 2021-02-10T11:30:14.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.055 7f511cc15700 1 Processor -- start 2021-02-10T11:30:14.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.055 7f511cc15700 1 -- start start 2021-02-10T11:30:14.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.056 7f511cc15700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f51181b0470 0x7f51181b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:14.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.056 7f511cc15700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51180f20e0 0x7f51180f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:14.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.056 7f511cc15700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51180f0b50 con 0x7f51180ef610 2021-02-10T11:30:14.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.056 7f511cc15700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51180f0820 con 0x7f51181b0470 2021-02-10T11:30:14.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.056 7f511cc15700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51180f24e0 con 0x7f51180f20e0 2021-02-10T11:30:14.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.056 7f511605d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f51180ef610 0x7f51180efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49220/0 (socket says 172.21.2.101:49220) 2021-02-10T11:30:14.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.056 7f511605d700 1 -- 172.21.2.101:0/605135665 learned_addr learned my addr 172.21.2.101:0/605135665 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:14.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.056 7f511705f700 1 --2- 172.21.2.101:0/605135665 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51180f20e0 0x7f51180f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:14.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.056 7f511685e700 1 --2- 172.21.2.101:0/605135665 >> v2:172.21.2.108:3300/0 conn(0x7f51181b0470 0x7f51181b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:14.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.057 7f511585c700 1 -- 172.21.2.101:0/605135665 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 841794895 0 0) 0x7f5100001180 con 0x7f51180ef610 2021-02-10T11:30:14.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.057 7f511585c700 1 -- 172.21.2.101:0/605135665 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5104002fe0 con 0x7f51180ef610 2021-02-10T11:30:14.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.057 7f511705f700 1 -- 172.21.2.101:0/605135665 >> v2:172.21.2.108:3300/0 conn(0x7f51181b0470 msgr2=0x7f51181b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:14.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.057 7f511705f700 1 --2- 172.21.2.101:0/605135665 >> v2:172.21.2.108:3300/0 conn(0x7f51181b0470 0x7f51181b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:14.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.057 7f511705f700 1 -- 172.21.2.101:0/605135665 >> v1:172.21.2.101:6789/0 conn(0x7f51180ef610 legacy=0x7f51180efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:14.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.057 7f511705f700 1 -- 172.21.2.101:0/605135665 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51180e52e0 con 0x7f51180f20e0 2021-02-10T11:30:14.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.057 7f511705f700 1 --2- 172.21.2.101:0/605135665 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51180f20e0 0x7f51180f0200 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f5108006ad0 tx=0x7f5108009ad0).ready entity=mon.2 client_cookie=5fe8215a822c9e48 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:14.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.057 7f511585c700 1 -- 172.21.2.101:0/605135665 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f51080022e0 con 0x7f51180f20e0 2021-02-10T11:30:14.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.057 7f511585c700 1 -- 172.21.2.101:0/605135665 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5108006410 con 0x7f51180f20e0 2021-02-10T11:30:14.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.057 7f511585c700 1 -- 172.21.2.101:0/605135665 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5108002560 con 0x7f51180f20e0 2021-02-10T11:30:14.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.057 7f511cc15700 1 -- 172.21.2.101:0/605135665 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51180f20e0 msgr2=0x7f51180f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:14.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.057 7f511cc15700 1 --2- 172.21.2.101:0/605135665 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51180f20e0 0x7f51180f0200 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f5108006ad0 tx=0x7f5108009ad0).stop 2021-02-10T11:30:14.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.058 7f511cc15700 1 -- 172.21.2.101:0/605135665 shutdown_connections 2021-02-10T11:30:14.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.058 7f511cc15700 1 --2- 172.21.2.101:0/605135665 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51180f20e0 0x7f51180f0200 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:14.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.058 7f511cc15700 1 --2- 172.21.2.101:0/605135665 >> v2:172.21.2.108:3300/0 conn(0x7f51181b0470 0x7f51181b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:14.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.058 7f511cc15700 1 -- 172.21.2.101:0/605135665 >> 172.21.2.101:0/605135665 conn(0x7f51180ce3e0 msgr2=0x7f51180e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:14.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.058 7f511cc15700 1 -- 172.21.2.101:0/605135665 shutdown_connections 2021-02-10T11:30:14.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.058 7f511cc15700 1 -- 172.21.2.101:0/605135665 wait complete. 2021-02-10T11:30:14.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.073 7f511cc15700 1 Processor -- start 2021-02-10T11:30:14.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.073 7f511cc15700 1 -- start start 2021-02-10T11:30:14.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.073 7f511cc15700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51180a0420 0x7f51180a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:14.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.073 7f511cc15700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5118189d00 0x7f511818a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:14.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.073 7f511cc15700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f511818a720 0x7f51180b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:14.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.073 7f511cc15700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5118189800 con 0x7f5118189d00 2021-02-10T11:30:14.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.073 7f511cc15700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511818ab20 con 0x7f511818a720 2021-02-10T11:30:14.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.073 7f511cc15700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51180a1140 con 0x7f51180a0420 2021-02-10T11:30:14.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.073 7f511605d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5118189d00 0x7f511818a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:14.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.073 7f511605d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5118189d00 0x7f511818a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48732/0 (socket says 172.21.2.101:48732) 2021-02-10T11:30:14.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.073 7f511605d700 1 -- 172.21.2.101:0/3319056626 learned_addr learned my addr 172.21.2.101:0/3319056626 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:14.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.074 7f511705f700 1 --2- 172.21.2.101:0/3319056626 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f511818a720 0x7f51180b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:14.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.074 7f511685e700 1 --2- 172.21.2.101:0/3319056626 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51180a0420 0x7f51180a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:14.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.074 7f511605d700 1 -- 172.21.2.101:0/3319056626 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51180a0420 msgr2=0x7f51180a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:14.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.074 7f511605d700 1 --2- 172.21.2.101:0/3319056626 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51180a0420 0x7f51180a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:14.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.074 7f511605d700 1 -- 172.21.2.101:0/3319056626 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f511818a720 msgr2=0x7f51180b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:14.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.074 7f511605d700 1 --2- 172.21.2.101:0/3319056626 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f511818a720 0x7f51180b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:14.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.074 7f511605d700 1 -- 172.21.2.101:0/3319056626 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51180b8200 con 0x7f5118189d00 2021-02-10T11:30:14.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.074 7f511605d700 1 --2- 172.21.2.101:0/3319056626 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5118189d00 0x7f511818a100 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7f510000a5d0 tx=0x7f510000a970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:14.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.074 7f510f7fe700 1 -- 172.21.2.101:0/3319056626 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5100000ce0 con 0x7f5118189d00 2021-02-10T11:30:14.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.074 7f510f7fe700 1 -- 172.21.2.101:0/3319056626 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51000043a0 con 0x7f5118189d00 2021-02-10T11:30:14.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.074 7f511cc15700 1 -- 172.21.2.101:0/3319056626 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51180b8200 con 0x7f5118189d00 2021-02-10T11:30:14.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.074 7f511cc15700 1 -- 172.21.2.101:0/3319056626 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51180b8200 con 0x7f5118189d00 2021-02-10T11:30:14.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.074 7f510f7fe700 1 -- 172.21.2.101:0/3319056626 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f51000047f0 con 0x7f5118189d00 2021-02-10T11:30:14.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.075 7f510f7fe700 1 -- 172.21.2.101:0/3319056626 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5100004cc0 con 0x7f5118189d00 2021-02-10T11:30:14.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.075 7f510f7fe700 1 --2- 172.21.2.101:0/3319056626 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50fc0186c0 0x7f50fc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:14.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.075 7f510f7fe700 1 -- 172.21.2.101:0/3319056626 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5100032800 con 0x7f5118189d00 2021-02-10T11:30:14.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.075 7f511685e700 1 --2- 172.21.2.101:0/3319056626 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50fc0186c0 0x7f50fc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:14.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.075 7f511685e700 1 --2- 172.21.2.101:0/3319056626 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50fc0186c0 0x7f50fc01abc0 crc :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:14.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.105 7f511cc15700 1 -- 172.21.2.101:0/3319056626 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511818ab20 con 0x7f5118189d00 2021-02-10T11:30:14.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.108 7f510f7fe700 1 -- 172.21.2.101:0/3319056626 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5100004ab0 con 0x7f5118189d00 2021-02-10T11:30:14.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.236 7f511cc15700 1 -- 172.21.2.101:0/3319056626 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839261", "target": ["mgr", ""]}) v1 -- 0x7f511818ab20 con 0x7f50fc0186c0 2021-02-10T11:30:14.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.245 7f510f7fe700 1 -- 172.21.2.101:0/3319056626 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f5110006bf0 con 0x7f50fc0186c0 2021-02-10T11:30:14.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.253 7f511cc15700 1 -- 172.21.2.101:0/3319056626 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50fc0186c0 msgr2=0x7f50fc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:14.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.253 7f511cc15700 1 --2- 172.21.2.101:0/3319056626 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50fc0186c0 0x7f50fc01abc0 crc :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:14.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.253 7f511cc15700 1 -- 172.21.2.101:0/3319056626 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5118189d00 msgr2=0x7f511818a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:14.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.253 7f511cc15700 1 --2- 172.21.2.101:0/3319056626 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5118189d00 0x7f511818a100 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7f510000a5d0 tx=0x7f510000a970).stop 2021-02-10T11:30:14.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.253 7f511cc15700 1 -- 172.21.2.101:0/3319056626 shutdown_connections 2021-02-10T11:30:14.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.253 7f511cc15700 1 --2- 172.21.2.101:0/3319056626 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f511818a720 0x7f51180b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:14.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.253 7f511cc15700 1 --2- 172.21.2.101:0/3319056626 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5118189d00 0x7f511818a100 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:14.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.253 7f511cc15700 1 --2- 172.21.2.101:0/3319056626 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50fc0186c0 0x7f50fc01abc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:14.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.253 7f511cc15700 1 --2- 172.21.2.101:0/3319056626 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51180a0420 0x7f51180a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:14.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.253 7f511cc15700 1 -- 172.21.2.101:0/3319056626 >> 172.21.2.101:0/3319056626 conn(0x7f5118094e10 msgr2=0x7f51180a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:14.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.253 7f511cc15700 1 -- 172.21.2.101:0/3319056626 shutdown_connections 2021-02-10T11:30:14.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.253 7f511cc15700 1 -- 172.21.2.101:0/3319056626 wait complete. 2021-02-10T11:30:14.273 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839262 2021-02-10T11:30:14.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.678 7f453762e700 1 Processor -- start 2021-02-10T11:30:14.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.678 7f453762e700 1 -- start start 2021-02-10T11:30:14.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.678 7f453762e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f45301b0870 0x7f45300f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:14.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.678 7f453762e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f45300efe00 0x7f45300f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:14.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.678 7f453762e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45301b0c70 con 0x7f45301b0470 2021-02-10T11:30:14.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.678 7f453762e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45300e5ea0 con 0x7f45300efe00 2021-02-10T11:30:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.678 7f453762e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45300f0b50 con 0x7f45301b0870 2021-02-10T11:30:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.679 7f453544a700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f45301b0470 0x7f45300ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49232/0 (socket says 172.21.2.101:49232) 2021-02-10T11:30:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.679 7f453544a700 1 -- 172.21.2.101:0/3606031863 learned_addr learned my addr 172.21.2.101:0/3606031863 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.679 7f4535c4b700 1 --2- 172.21.2.101:0/3606031863 >> v2:172.21.2.101:3300/0 conn(0x7f45300efe00 0x7f45300f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.679 7f4534c49700 1 --2- 172.21.2.101:0/3606031863 >> v2:172.21.2.108:3300/0 conn(0x7f45301b0870 0x7f45300f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.679 7f4527fff700 1 -- 172.21.2.101:0/3606031863 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3203923666 0 0) 0x7f4520000ce0 con 0x7f45301b0470 2021-02-10T11:30:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.679 7f4527fff700 1 -- 172.21.2.101:0/3606031863 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f451c002fe0 con 0x7f45301b0470 2021-02-10T11:30:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.679 7f4535c4b700 1 -- 172.21.2.101:0/3606031863 >> v2:172.21.2.108:3300/0 conn(0x7f45301b0870 msgr2=0x7f45300f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.679 7f4535c4b700 1 --2- 172.21.2.101:0/3606031863 >> v2:172.21.2.108:3300/0 conn(0x7f45301b0870 0x7f45300f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.679 7f4535c4b700 1 -- 172.21.2.101:0/3606031863 >> v1:172.21.2.101:6789/0 conn(0x7f45301b0470 legacy=0x7f45300ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.679 7f4535c4b700 1 -- 172.21.2.101:0/3606031863 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45300f0820 con 0x7f45300efe00 2021-02-10T11:30:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.679 7f4535c4b700 1 --2- 172.21.2.101:0/3606031863 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45300efe00 0x7f45300f0200 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7f4528006e30 tx=0x7f4528009e60).ready entity=mon.0 client_cookie=60f389d4a5ef409e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:14.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.679 7f4527fff700 1 -- 172.21.2.101:0/3606031863 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4528006a50 con 0x7f45300efe00 2021-02-10T11:30:14.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.679 7f4527fff700 1 -- 172.21.2.101:0/3606031863 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452800b300 con 0x7f45300efe00 2021-02-10T11:30:14.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.679 7f4527fff700 1 -- 172.21.2.101:0/3606031863 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f452800b7e0 con 0x7f45300efe00 2021-02-10T11:30:14.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.680 7f453762e700 1 -- 172.21.2.101:0/3606031863 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45300efe00 msgr2=0x7f45300f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:14.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.680 7f453762e700 1 --2- 172.21.2.101:0/3606031863 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45300efe00 0x7f45300f0200 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7f4528006e30 tx=0x7f4528009e60).stop 2021-02-10T11:30:14.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.680 7f453762e700 1 -- 172.21.2.101:0/3606031863 shutdown_connections 2021-02-10T11:30:14.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.680 7f453762e700 1 --2- 172.21.2.101:0/3606031863 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45300efe00 0x7f45300f0200 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:14.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.680 7f453762e700 1 --2- 172.21.2.101:0/3606031863 >> v2:172.21.2.108:3300/0 conn(0x7f45301b0870 0x7f45300f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:14.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.680 7f453762e700 1 -- 172.21.2.101:0/3606031863 >> 172.21.2.101:0/3606031863 conn(0x7f45300ce3e0 msgr2=0x7f45300e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:14.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.680 7f453762e700 1 -- 172.21.2.101:0/3606031863 shutdown_connections 2021-02-10T11:30:14.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.680 7f453762e700 1 -- 172.21.2.101:0/3606031863 wait complete. 2021-02-10T11:30:14.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.695 7f453762e700 1 Processor -- start 2021-02-10T11:30:14.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.695 7f453762e700 1 -- start start 2021-02-10T11:30:14.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.695 7f453762e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4530190920 0x7f45300b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:14.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.695 7f453762e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45300b8200 0x7f45300b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:14.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.695 7f453762e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45300c1d80 0x7f45300c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:14.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.695 7f453762e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4530190da0 con 0x7f45300b8200 2021-02-10T11:30:14.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.695 7f453762e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45300b8c20 con 0x7f4530190920 2021-02-10T11:30:14.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.695 7f453762e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45300b8e10 con 0x7f45300c1d80 2021-02-10T11:30:14.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.695 7f4534c49700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45300b8200 0x7f45300b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:14.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.695 7f4534c49700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45300b8200 0x7f45300b8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48742/0 (socket says 172.21.2.101:48742) 2021-02-10T11:30:14.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.695 7f4534c49700 1 -- 172.21.2.101:0/759331086 learned_addr learned my addr 172.21.2.101:0/759331086 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:14.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.695 7f453544a700 1 --2- 172.21.2.101:0/759331086 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4530190920 0x7f45300b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:14.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.696 7f4535c4b700 1 --2- 172.21.2.101:0/759331086 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45300c1d80 0x7f45300c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:14.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.696 7f4534c49700 1 -- 172.21.2.101:0/759331086 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45300c1d80 msgr2=0x7f45300c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:14.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.696 7f4534c49700 1 --2- 172.21.2.101:0/759331086 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45300c1d80 0x7f45300c2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:14.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.696 7f4534c49700 1 -- 172.21.2.101:0/759331086 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4530190920 msgr2=0x7f45300b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:14.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.696 7f4534c49700 1 --2- 172.21.2.101:0/759331086 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4530190920 0x7f45300b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:14.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.696 7f4534c49700 1 -- 172.21.2.101:0/759331086 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45300c27d0 con 0x7f45300b8200 2021-02-10T11:30:14.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.696 7f453544a700 1 --2- 172.21.2.101:0/759331086 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4530190920 0x7f45300b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:30:14.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.696 7f4534c49700 1 --2- 172.21.2.101:0/759331086 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45300b8200 0x7f45300b8600 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f452c006f60 tx=0x7f452c00b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:14.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.696 7f4525ffb700 1 -- 172.21.2.101:0/759331086 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f452c00bd60 con 0x7f45300b8200 2021-02-10T11:30:14.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.696 7f4525ffb700 1 -- 172.21.2.101:0/759331086 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452c000c90 con 0x7f45300b8200 2021-02-10T11:30:14.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.696 7f4525ffb700 1 -- 172.21.2.101:0/759331086 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f452c002220 con 0x7f45300b8200 2021-02-10T11:30:14.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.696 7f453762e700 1 -- 172.21.2.101:0/759331086 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45300c27d0 con 0x7f45300b8200 2021-02-10T11:30:14.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.696 7f453762e700 1 -- 172.21.2.101:0/759331086 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45300c27d0 con 0x7f45300b8200 2021-02-10T11:30:14.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.697 7f4525ffb700 1 -- 172.21.2.101:0/759331086 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f452c0052e0 con 0x7f45300b8200 2021-02-10T11:30:14.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.697 7f4525ffb700 1 --2- 172.21.2.101:0/759331086 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45100186d0 0x7f451001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:14.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.697 7f4525ffb700 1 -- 172.21.2.101:0/759331086 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f452c0327d0 con 0x7f45300b8200 2021-02-10T11:30:14.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.697 7f453544a700 1 --2- 172.21.2.101:0/759331086 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45100186d0 0x7f451001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:14.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.697 7f453544a700 1 --2- 172.21.2.101:0/759331086 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45100186d0 0x7f451001abd0 crc :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:14.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.727 7f453762e700 1 -- 172.21.2.101:0/759331086 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4530105ae0 con 0x7f45300b8200 2021-02-10T11:30:14.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.730 7f4525ffb700 1 -- 172.21.2.101:0/759331086 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f452c000c90 con 0x7f45300b8200 2021-02-10T11:30:14.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.855 7f453762e700 1 -- 172.21.2.101:0/759331086 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839262", "target": ["mgr", ""]}) v1 -- 0x7f45300f0b50 con 0x7f45100186d0 2021-02-10T11:30:14.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.865 7f4525ffb700 1 -- 172.21.2.101:0/759331086 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f45200079b0 con 0x7f45100186d0 2021-02-10T11:30:14.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.872 7f453762e700 1 -- 172.21.2.101:0/759331086 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45100186d0 msgr2=0x7f451001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:14.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.872 7f453762e700 1 --2- 172.21.2.101:0/759331086 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45100186d0 0x7f451001abd0 crc :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:14.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.872 7f453762e700 1 -- 172.21.2.101:0/759331086 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45300b8200 msgr2=0x7f45300b8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:14.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.872 7f453762e700 1 --2- 172.21.2.101:0/759331086 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45300b8200 0x7f45300b8600 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f452c006f60 tx=0x7f452c00b970).stop 2021-02-10T11:30:14.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.872 7f453762e700 1 -- 172.21.2.101:0/759331086 shutdown_connections 2021-02-10T11:30:14.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.872 7f453762e700 1 --2- 172.21.2.101:0/759331086 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45100186d0 0x7f451001abd0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:14.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.872 7f453762e700 1 --2- 172.21.2.101:0/759331086 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45300c1d80 0x7f45300c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:14.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.872 7f453762e700 1 --2- 172.21.2.101:0/759331086 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45300b8200 0x7f45300b8600 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:14.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.872 7f453762e700 1 --2- 172.21.2.101:0/759331086 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4530190920 0x7f45300b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:14.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.872 7f453762e700 1 -- 172.21.2.101:0/759331086 >> 172.21.2.101:0/759331086 conn(0x7f4530094e10 msgr2=0x7f45300a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:14.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.872 7f453762e700 1 -- 172.21.2.101:0/759331086 shutdown_connections 2021-02-10T11:30:14.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:14.872 7f453762e700 1 -- 172.21.2.101:0/759331086 wait complete. 2021-02-10T11:30:14.891 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839263 2021-02-10T11:30:15.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.295 7f8277d1c700 1 Processor -- start 2021-02-10T11:30:15.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.295 7f8277d1c700 1 -- start start 2021-02-10T11:30:15.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.295 7f8277d1c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f82701b0470 0x7f82701b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:15.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.295 7f8277d1c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82700f0b50 con 0x7f82700ef610 2021-02-10T11:30:15.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.295 7f8277d1c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82700f24e0 con 0x7f82700f20e0 2021-02-10T11:30:15.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.295 7f8277d1c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82700e5ea0 con 0x7f82701b0470 2021-02-10T11:30:15.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.296 7f8275337700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f82700ef610 0x7f82700efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49244/0 (socket says 172.21.2.101:49244) 2021-02-10T11:30:15.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.296 7f8275337700 1 -- 172.21.2.101:0/1950502910 learned_addr learned my addr 172.21.2.101:0/1950502910 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:15.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.296 7f8275b38700 1 --2- 172.21.2.101:0/1950502910 >> v2:172.21.2.108:3300/0 conn(0x7f82701b0470 0x7f82701b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:15.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.296 7f8275b38700 1 -- 172.21.2.101:0/1950502910 >> v1:172.21.2.108:6789/0 conn(0x7f82700f20e0 legacy=0x7f82700f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:15.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.296 7f8275b38700 1 -- 172.21.2.101:0/1950502910 >> v1:172.21.2.101:6789/0 conn(0x7f82700ef610 legacy=0x7f82700efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:15.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.296 7f8275b38700 1 -- 172.21.2.101:0/1950502910 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82700e52e0 con 0x7f82701b0470 2021-02-10T11:30:15.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.296 7f8275b38700 1 --2- 172.21.2.101:0/1950502910 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82701b0470 0x7f82701b0870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f826c006650 tx=0x7f826c009a00).ready entity=mon.1 client_cookie=5aa984fbf40ec870 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:15.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.296 7f8274b36700 1 -- 172.21.2.101:0/1950502910 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f826c0009b0 con 0x7f82701b0470 2021-02-10T11:30:15.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.296 7f8274b36700 1 -- 172.21.2.101:0/1950502910 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826c00c0c0 con 0x7f82701b0470 2021-02-10T11:30:15.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.296 7f8274b36700 1 -- 172.21.2.101:0/1950502910 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f826c00c5a0 con 0x7f82701b0470 2021-02-10T11:30:15.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.296 7f8277d1c700 1 -- 172.21.2.101:0/1950502910 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82701b0470 msgr2=0x7f82701b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:15.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.296 7f8277d1c700 1 --2- 172.21.2.101:0/1950502910 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82701b0470 0x7f82701b0870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f826c006650 tx=0x7f826c009a00).stop 2021-02-10T11:30:15.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.297 7f8277d1c700 1 -- 172.21.2.101:0/1950502910 shutdown_connections 2021-02-10T11:30:15.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.297 7f8277d1c700 1 --2- 172.21.2.101:0/1950502910 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82701b0470 0x7f82701b0870 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:15.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.297 7f8277d1c700 1 -- 172.21.2.101:0/1950502910 >> 172.21.2.101:0/1950502910 conn(0x7f82700ce3e0 msgr2=0x7f82700e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:15.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.297 7f8277d1c700 1 -- 172.21.2.101:0/1950502910 shutdown_connections 2021-02-10T11:30:15.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.297 7f8277d1c700 1 -- 172.21.2.101:0/1950502910 wait complete. 2021-02-10T11:30:15.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.312 7f8277d1c700 1 Processor -- start 2021-02-10T11:30:15.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.312 7f8277d1c700 1 -- start start 2021-02-10T11:30:15.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.312 7f8277d1c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8270082170 0x7f82700894a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:15.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.312 7f8277d1c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82701b01a0 0x7f82701b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:15.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.312 7f8277d1c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82701b1460 0x7f82701b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:15.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.312 7f8277d1c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82701bf2a0 con 0x7f82701b1460 2021-02-10T11:30:15.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.312 7f8277d1c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82701b0ee0 con 0x7f82701b01a0 2021-02-10T11:30:15.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.312 7f8277d1c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82701b60c0 con 0x7f8270082170 2021-02-10T11:30:15.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.313 7f8276339700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82701b1460 0x7f82701b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:15.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.313 7f8276339700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82701b1460 0x7f82701b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48756/0 (socket says 172.21.2.101:48756) 2021-02-10T11:30:15.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.313 7f8276339700 1 -- 172.21.2.101:0/374575667 learned_addr learned my addr 172.21.2.101:0/374575667 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:15.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.313 7f8275337700 1 --2- 172.21.2.101:0/374575667 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82701b01a0 0x7f82701b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:15.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.313 7f8275b38700 1 --2- 172.21.2.101:0/374575667 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8270082170 0x7f82700894a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:15.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.313 7f8275337700 1 -- 172.21.2.101:0/374575667 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8270082170 msgr2=0x7f82700894a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:15.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.313 7f8275337700 1 --2- 172.21.2.101:0/374575667 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8270082170 0x7f82700894a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:15.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.313 7f8275337700 1 -- 172.21.2.101:0/374575667 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82701b1460 msgr2=0x7f82701b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:15.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.313 7f8275337700 1 --2- 172.21.2.101:0/374575667 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82701b1460 0x7f82701b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:15.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.313 7f8275337700 1 -- 172.21.2.101:0/374575667 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82701b0bc0 con 0x7f82701b01a0 2021-02-10T11:30:15.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.313 7f8275337700 1 --2- 172.21.2.101:0/374575667 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82701b01a0 0x7f82701b05a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f826400a4c0 tx=0x7f826400a860).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:15.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.313 7f82627fc700 1 -- 172.21.2.101:0/374575667 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8264000e20 con 0x7f82701b01a0 2021-02-10T11:30:15.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.313 7f82627fc700 1 -- 172.21.2.101:0/374575667 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826400ac50 con 0x7f82701b01a0 2021-02-10T11:30:15.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.313 7f8277d1c700 1 -- 172.21.2.101:0/374575667 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82701b0bc0 con 0x7f82701b01a0 2021-02-10T11:30:15.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.313 7f82627fc700 1 -- 172.21.2.101:0/374575667 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f82640045d0 con 0x7f82701b01a0 2021-02-10T11:30:15.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.313 7f8277d1c700 1 -- 172.21.2.101:0/374575667 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82701b0bc0 con 0x7f82701b01a0 2021-02-10T11:30:15.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.314 7f82627fc700 1 -- 172.21.2.101:0/374575667 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8264000d30 con 0x7f82701b01a0 2021-02-10T11:30:15.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.314 7f82627fc700 1 --2- 172.21.2.101:0/374575667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82580186c0 0x7f825801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:15.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.314 7f82627fc700 1 -- 172.21.2.101:0/374575667 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f82640326e0 con 0x7f82701b01a0 2021-02-10T11:30:15.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.314 7f8275b38700 1 --2- 172.21.2.101:0/374575667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82580186c0 0x7f825801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:15.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.314 7f8275b38700 1 --2- 172.21.2.101:0/374575667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82580186c0 0x7f825801abc0 crc :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:15.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.345 7f8277d1c700 1 -- 172.21.2.101:0/374575667 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82701bf2a0 con 0x7f82701b01a0 2021-02-10T11:30:15.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.348 7f82627fc700 1 -- 172.21.2.101:0/374575667 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f826401f420 con 0x7f82701b01a0 2021-02-10T11:30:15.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.478 7f8277d1c700 1 -- 172.21.2.101:0/374575667 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839263", "target": ["mgr", ""]}) v1 -- 0x7f82701bf2a0 con 0x7f82580186c0 2021-02-10T11:30:15.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.488 7f82627fc700 1 -- 172.21.2.101:0/374575667 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f826c0009b0 con 0x7f82580186c0 2021-02-10T11:30:15.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.495 7f8277d1c700 1 -- 172.21.2.101:0/374575667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82580186c0 msgr2=0x7f825801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:15.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.495 7f8277d1c700 1 --2- 172.21.2.101:0/374575667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82580186c0 0x7f825801abc0 crc :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:15.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.495 7f8277d1c700 1 -- 172.21.2.101:0/374575667 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82701b01a0 msgr2=0x7f82701b05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:15.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.495 7f8277d1c700 1 --2- 172.21.2.101:0/374575667 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82701b01a0 0x7f82701b05a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f826400a4c0 tx=0x7f826400a860).stop 2021-02-10T11:30:15.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.495 7f8277d1c700 1 -- 172.21.2.101:0/374575667 shutdown_connections 2021-02-10T11:30:15.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.495 7f8277d1c700 1 --2- 172.21.2.101:0/374575667 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82701b1460 0x7f82701b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:15.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.495 7f8277d1c700 1 --2- 172.21.2.101:0/374575667 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82701b01a0 0x7f82701b05a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:15.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.495 7f8277d1c700 1 --2- 172.21.2.101:0/374575667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82580186c0 0x7f825801abc0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:15.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.495 7f8277d1c700 1 --2- 172.21.2.101:0/374575667 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8270082170 0x7f82700894a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:15.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.495 7f8277d1c700 1 -- 172.21.2.101:0/374575667 >> 172.21.2.101:0/374575667 conn(0x7f8270088d10 msgr2=0x7f82700afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:15.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.495 7f8277d1c700 1 -- 172.21.2.101:0/374575667 shutdown_connections 2021-02-10T11:30:15.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.495 7f8277d1c700 1 -- 172.21.2.101:0/374575667 wait complete. 2021-02-10T11:30:15.515 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839264 2021-02-10T11:30:15.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.920 7f02d8a96700 1 Processor -- start 2021-02-10T11:30:15.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.920 7f02d8a96700 1 -- start start 2021-02-10T11:30:15.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.921 7f02d8a96700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f02d41b0470 0x7f02d41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:15.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.921 7f02d8a96700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f02d40ef610 0x7f02d40efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:15.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.921 7f02d8a96700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02d40f0030 0x7f02d40f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:15.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.921 7f02d8a96700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02d40e5ea0 con 0x7f02d40ef610 2021-02-10T11:30:15.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.921 7f02d8a96700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02d40f0b50 con 0x7f02d41b0470 2021-02-10T11:30:15.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.921 7f02d8a96700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d40e52e0 con 0x7f02d40f0030 2021-02-10T11:30:15.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.921 7f02ca05d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f02d40ef610 0x7f02d40efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:15.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.921 7f02d305f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02d40f0030 0x7f02d40f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:15.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.921 7f02ca05d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f02d40ef610 0x7f02d40efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48766/0 (socket says 172.21.2.101:48766) 2021-02-10T11:30:15.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.921 7f02ca05d700 1 -- 172.21.2.101:0/3593296447 learned_addr learned my addr 172.21.2.101:0/3593296447 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:15.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.921 7f02d285e700 1 --2- 172.21.2.101:0/3593296447 >> v2:172.21.2.108:3300/0 conn(0x7f02d41b0470 0x7f02d41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:15.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.921 7f02d305f700 1 -- 172.21.2.101:0/3593296447 >> v2:172.21.2.108:3300/0 conn(0x7f02d41b0470 msgr2=0x7f02d41b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:15.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.921 7f02d305f700 1 --2- 172.21.2.101:0/3593296447 >> v2:172.21.2.108:3300/0 conn(0x7f02d41b0470 0x7f02d41b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:15.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.921 7f02d305f700 1 -- 172.21.2.101:0/3593296447 >> v2:172.21.2.101:3300/0 conn(0x7f02d40ef610 msgr2=0x7f02d40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:15.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.921 7f02d305f700 1 --2- 172.21.2.101:0/3593296447 >> v2:172.21.2.101:3300/0 conn(0x7f02d40ef610 0x7f02d40efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:15.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.921 7f02d305f700 1 -- 172.21.2.101:0/3593296447 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d40f10d0 con 0x7f02d40f0030 2021-02-10T11:30:15.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.921 7f02d285e700 1 --2- 172.21.2.101:0/3593296447 >> v2:172.21.2.108:3300/0 conn(0x7f02d41b0470 0x7f02d41b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:30:15.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.921 7f02d305f700 1 --2- 172.21.2.101:0/3593296447 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02d40f0030 0x7f02d40f0430 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f02cc006e50 tx=0x7f02cc009e70).ready entity=mon.2 client_cookie=22aae0c5f66af218 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:15.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.922 7f02d205d700 1 -- 172.21.2.101:0/3593296447 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f02cc000f90 con 0x7f02d40f0030 2021-02-10T11:30:15.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.922 7f02d205d700 1 -- 172.21.2.101:0/3593296447 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02cc0066f0 con 0x7f02d40f0030 2021-02-10T11:30:15.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.922 7f02d205d700 1 -- 172.21.2.101:0/3593296447 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f02cc00b790 con 0x7f02d40f0030 2021-02-10T11:30:15.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.922 7f02d8a96700 1 -- 172.21.2.101:0/3593296447 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02d40f0030 msgr2=0x7f02d40f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:15.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.922 7f02d8a96700 1 --2- 172.21.2.101:0/3593296447 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02d40f0030 0x7f02d40f0430 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f02cc006e50 tx=0x7f02cc009e70).stop 2021-02-10T11:30:15.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.922 7f02d8a96700 1 -- 172.21.2.101:0/3593296447 shutdown_connections 2021-02-10T11:30:15.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.922 7f02d8a96700 1 --2- 172.21.2.101:0/3593296447 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02d40f0030 0x7f02d40f0430 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:15.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.922 7f02d8a96700 1 --2- 172.21.2.101:0/3593296447 >> v2:172.21.2.101:3300/0 conn(0x7f02d40ef610 0x7f02d40efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:15.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.922 7f02d8a96700 1 --2- 172.21.2.101:0/3593296447 >> v2:172.21.2.108:3300/0 conn(0x7f02d41b0470 0x7f02d41b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:15.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.922 7f02d8a96700 1 -- 172.21.2.101:0/3593296447 >> 172.21.2.101:0/3593296447 conn(0x7f02d40ce3e0 msgr2=0x7f02d40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:15.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.922 7f02d8a96700 1 -- 172.21.2.101:0/3593296447 shutdown_connections 2021-02-10T11:30:15.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.922 7f02d8a96700 1 -- 172.21.2.101:0/3593296447 wait complete. 2021-02-10T11:30:15.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.937 7f02d8a96700 1 Processor -- start 2021-02-10T11:30:15.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.937 7f02d8a96700 1 -- start start 2021-02-10T11:30:15.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.937 7f02d8a96700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02d40b7c20 0x7f02d40b8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:15.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.937 7f02d8a96700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02d40b8a30 0x7f02d40c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:15.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.937 7f02d8a96700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02d40c25a0 0x7f02d40c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:15.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.937 7f02d8a96700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d40c1a80 con 0x7f02d40b8a30 2021-02-10T11:30:15.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.938 7f02d8a96700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d40b8020 con 0x7f02d40b7c20 2021-02-10T11:30:15.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.938 7f02d8a96700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d409b9c0 con 0x7f02d40c25a0 2021-02-10T11:30:15.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.938 7f02ca05d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02d40b8a30 0x7f02d40c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:15.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.938 7f02ca05d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02d40b8a30 0x7f02d40c1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48768/0 (socket says 172.21.2.101:48768) 2021-02-10T11:30:15.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.938 7f02ca05d700 1 -- 172.21.2.101:0/2718810068 learned_addr learned my addr 172.21.2.101:0/2718810068 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:15.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.938 7f02d305f700 1 --2- 172.21.2.101:0/2718810068 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02d40c25a0 0x7f02d40c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:15.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.938 7f02d285e700 1 --2- 172.21.2.101:0/2718810068 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02d40b7c20 0x7f02d40b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:15.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.938 7f02d305f700 1 -- 172.21.2.101:0/2718810068 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02d40b7c20 msgr2=0x7f02d40b8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:15.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.938 7f02d305f700 1 --2- 172.21.2.101:0/2718810068 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02d40b7c20 0x7f02d40b8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:15.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.938 7f02d305f700 1 -- 172.21.2.101:0/2718810068 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02d40b8a30 msgr2=0x7f02d40c1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:15.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.938 7f02d305f700 1 --2- 172.21.2.101:0/2718810068 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02d40b8a30 0x7f02d40c1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:15.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.938 7f02d305f700 1 -- 172.21.2.101:0/2718810068 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d40b8e30 con 0x7f02d40c25a0 2021-02-10T11:30:15.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.938 7f02d305f700 1 --2- 172.21.2.101:0/2718810068 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02d40c25a0 0x7f02d40c29a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f02cc00b8b0 tx=0x7f02cc00bc50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:15.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.938 7f02cbfff700 1 -- 172.21.2.101:0/2718810068 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f02cc00bd90 con 0x7f02d40c25a0 2021-02-10T11:30:15.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.938 7f02cbfff700 1 -- 172.21.2.101:0/2718810068 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02cc0066f0 con 0x7f02d40c25a0 2021-02-10T11:30:15.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.938 7f02d8a96700 1 -- 172.21.2.101:0/2718810068 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d40b8e30 con 0x7f02d40c25a0 2021-02-10T11:30:15.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.938 7f02cbfff700 1 -- 172.21.2.101:0/2718810068 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f02cc00c5a0 con 0x7f02d40c25a0 2021-02-10T11:30:15.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.938 7f02d8a96700 1 -- 172.21.2.101:0/2718810068 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d40b8e30 con 0x7f02d40c25a0 2021-02-10T11:30:15.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.939 7f02cbfff700 1 -- 172.21.2.101:0/2718810068 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f02cc0056b0 con 0x7f02d40c25a0 2021-02-10T11:30:15.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.939 7f02cbfff700 1 --2- 172.21.2.101:0/2718810068 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02b8018730 0x7f02b801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:15.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.939 7f02cbfff700 1 -- 172.21.2.101:0/2718810068 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f02cc032a70 con 0x7f02d40c25a0 2021-02-10T11:30:15.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.939 7f02d285e700 1 --2- 172.21.2.101:0/2718810068 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02b8018730 0x7f02b801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:15.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.940 7f02d285e700 1 --2- 172.21.2.101:0/2718810068 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02b8018730 0x7f02b801ac10 crc :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:15.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.969 7f02d8a96700 1 -- 172.21.2.101:0/2718810068 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d4105b30 con 0x7f02d40c25a0 2021-02-10T11:30:15.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:15.972 7f02cbfff700 1 -- 172.21.2.101:0/2718810068 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f02cc009680 con 0x7f02d40c25a0 2021-02-10T11:30:16.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.097 7f02d8a96700 1 -- 172.21.2.101:0/2718810068 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839264", "target": ["mgr", ""]}) v1 -- 0x7f02d40f0b50 con 0x7f02b8018730 2021-02-10T11:30:16.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.109 7f02cbfff700 1 -- 172.21.2.101:0/2718810068 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f02c4006540 con 0x7f02b8018730 2021-02-10T11:30:16.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.114 7f02d8a96700 1 -- 172.21.2.101:0/2718810068 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02b8018730 msgr2=0x7f02b801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:16.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.114 7f02d8a96700 1 --2- 172.21.2.101:0/2718810068 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02b8018730 0x7f02b801ac10 crc :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:16.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.114 7f02d8a96700 1 -- 172.21.2.101:0/2718810068 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02d40c25a0 msgr2=0x7f02d40c29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:16.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.114 7f02d8a96700 1 --2- 172.21.2.101:0/2718810068 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02d40c25a0 0x7f02d40c29a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f02cc00b8b0 tx=0x7f02cc00bc50).stop 2021-02-10T11:30:16.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.114 7f02d8a96700 1 -- 172.21.2.101:0/2718810068 shutdown_connections 2021-02-10T11:30:16.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.114 7f02d8a96700 1 --2- 172.21.2.101:0/2718810068 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02b8018730 0x7f02b801ac10 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:16.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.114 7f02d8a96700 1 --2- 172.21.2.101:0/2718810068 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02d40c25a0 0x7f02d40c29a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:16.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.114 7f02d8a96700 1 --2- 172.21.2.101:0/2718810068 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02d40b8a30 0x7f02d40c1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:16.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.114 7f02d8a96700 1 --2- 172.21.2.101:0/2718810068 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02d40b7c20 0x7f02d40b8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:16.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.114 7f02d8a96700 1 -- 172.21.2.101:0/2718810068 >> 172.21.2.101:0/2718810068 conn(0x7f02d4094e10 msgr2=0x7f02d40a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:16.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.115 7f02d8a96700 1 -- 172.21.2.101:0/2718810068 shutdown_connections 2021-02-10T11:30:16.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.115 7f02d8a96700 1 -- 172.21.2.101:0/2718810068 wait complete. 2021-02-10T11:30:16.135 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839265 2021-02-10T11:30:16.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.538 7fb39346a700 1 Processor -- start 2021-02-10T11:30:16.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.538 7fb39346a700 1 -- start start 2021-02-10T11:30:16.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.538 7fb39346a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb38c1b0470 0x7fb38c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:16.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.538 7fb39346a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb38c0ef610 0x7fb38c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:16.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.538 7fb39346a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38c0f0b50 con 0x7fb38c0f0030 2021-02-10T11:30:16.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.538 7fb39346a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb38c0e52e0 con 0x7fb38c1b0470 2021-02-10T11:30:16.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.538 7fb39346a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0e54d0 con 0x7fb38c0ef610 2021-02-10T11:30:16.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.538 7fb391286700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb38c1b0470 0x7fb38c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:16.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.538 7fb391a87700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb38c0f0030 0x7fb38c0f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49270/0 (socket says 172.21.2.101:49270) 2021-02-10T11:30:16.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.538 7fb391a87700 1 -- 172.21.2.101:0/1803547515 learned_addr learned my addr 172.21.2.101:0/1803547515 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:16.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.538 7fb390a85700 1 --2- 172.21.2.101:0/1803547515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb38c0ef610 0x7fb38c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:16.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.539 7fb391286700 1 -- 172.21.2.101:0/1803547515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb38c0ef610 msgr2=0x7fb38c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:16.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.539 7fb391286700 1 --2- 172.21.2.101:0/1803547515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb38c0ef610 0x7fb38c0efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:16.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.539 7fb391286700 1 -- 172.21.2.101:0/1803547515 >> v1:172.21.2.101:6789/0 conn(0x7fb38c0f0030 legacy=0x7fb38c0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:16.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.539 7fb391286700 1 -- 172.21.2.101:0/1803547515 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38c0f0430 con 0x7fb38c1b0470 2021-02-10T11:30:16.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.539 7fb391286700 1 --2- 172.21.2.101:0/1803547515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb38c1b0470 0x7fb38c1b0870 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7fb380006650 tx=0x7fb380009a80).ready entity=mon.0 client_cookie=6b8ab8cd12ac08d0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:16.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.539 7fb37ffff700 1 -- 172.21.2.101:0/1803547515 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb38000c0c0 con 0x7fb38c1b0470 2021-02-10T11:30:16.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.539 7fb37ffff700 1 -- 172.21.2.101:0/1803547515 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38000c340 con 0x7fb38c1b0470 2021-02-10T11:30:16.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.539 7fb37ffff700 1 -- 172.21.2.101:0/1803547515 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb38000c860 con 0x7fb38c1b0470 2021-02-10T11:30:16.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.539 7fb39346a700 1 -- 172.21.2.101:0/1803547515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb38c1b0470 msgr2=0x7fb38c1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:16.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.539 7fb39346a700 1 --2- 172.21.2.101:0/1803547515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb38c1b0470 0x7fb38c1b0870 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7fb380006650 tx=0x7fb380009a80).stop 2021-02-10T11:30:16.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.540 7fb39346a700 1 -- 172.21.2.101:0/1803547515 shutdown_connections 2021-02-10T11:30:16.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.540 7fb39346a700 1 --2- 172.21.2.101:0/1803547515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb38c0ef610 0x7fb38c0efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:16.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.540 7fb39346a700 1 --2- 172.21.2.101:0/1803547515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb38c1b0470 0x7fb38c1b0870 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:16.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.540 7fb39346a700 1 -- 172.21.2.101:0/1803547515 >> 172.21.2.101:0/1803547515 conn(0x7fb38c0ce3e0 msgr2=0x7fb38c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:16.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.540 7fb39346a700 1 -- 172.21.2.101:0/1803547515 shutdown_connections 2021-02-10T11:30:16.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.540 7fb39346a700 1 -- 172.21.2.101:0/1803547515 wait complete. 2021-02-10T11:30:16.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.555 7fb39346a700 1 Processor -- start 2021-02-10T11:30:16.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.555 7fb39346a700 1 -- start start 2021-02-10T11:30:16.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.555 7fb39346a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb38c190920 0x7fb38c0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:16.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.555 7fb39346a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb38c0b8200 0x7fb38c0b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:16.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.555 7fb39346a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb38c0c1d80 0x7fb38c0c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:16.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.555 7fb39346a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c190da0 con 0x7fb38c0c1d80 2021-02-10T11:30:16.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.555 7fb39346a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0b8c20 con 0x7fb38c190920 2021-02-10T11:30:16.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.555 7fb39346a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0b8e10 con 0x7fb38c0b8200 2021-02-10T11:30:16.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.555 7fb391a87700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb38c0c1d80 0x7fb38c0c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:16.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.556 7fb391a87700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb38c0c1d80 0x7fb38c0c2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48784/0 (socket says 172.21.2.101:48784) 2021-02-10T11:30:16.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.556 7fb391a87700 1 -- 172.21.2.101:0/3408383005 learned_addr learned my addr 172.21.2.101:0/3408383005 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:16.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.556 7fb390a85700 1 --2- 172.21.2.101:0/3408383005 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb38c0b8200 0x7fb38c0b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:16.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.556 7fb391286700 1 --2- 172.21.2.101:0/3408383005 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb38c190920 0x7fb38c0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:16.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.556 7fb391a87700 1 -- 172.21.2.101:0/3408383005 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb38c0b8200 msgr2=0x7fb38c0b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:16.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.556 7fb391a87700 1 --2- 172.21.2.101:0/3408383005 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb38c0b8200 0x7fb38c0b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:16.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.556 7fb391a87700 1 -- 172.21.2.101:0/3408383005 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb38c190920 msgr2=0x7fb38c0b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:16.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.556 7fb391a87700 1 --2- 172.21.2.101:0/3408383005 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb38c190920 0x7fb38c0b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:16.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.556 7fb391a87700 1 -- 172.21.2.101:0/3408383005 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38c0c27d0 con 0x7fb38c0c1d80 2021-02-10T11:30:16.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.556 7fb390a85700 1 --2- 172.21.2.101:0/3408383005 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb38c0b8200 0x7fb38c0b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:30:16.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.556 7fb391a87700 1 --2- 172.21.2.101:0/3408383005 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb38c0c1d80 0x7fb38c0c2180 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7fb38400a8f0 tx=0x7fb38400ac90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:16.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.556 7fb37dffb700 1 -- 172.21.2.101:0/3408383005 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb384000df0 con 0x7fb38c0c1d80 2021-02-10T11:30:16.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.556 7fb37dffb700 1 -- 172.21.2.101:0/3408383005 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3840070d0 con 0x7fb38c0c1d80 2021-02-10T11:30:16.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.556 7fb37dffb700 1 -- 172.21.2.101:0/3408383005 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb384004930 con 0x7fb38c0c1d80 2021-02-10T11:30:16.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.556 7fb39346a700 1 -- 172.21.2.101:0/3408383005 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb38c0c27d0 con 0x7fb38c0c1d80 2021-02-10T11:30:16.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.556 7fb39346a700 1 -- 172.21.2.101:0/3408383005 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38c0c2d60 con 0x7fb38c0c1d80 2021-02-10T11:30:16.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.557 7fb37dffb700 1 -- 172.21.2.101:0/3408383005 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb384000df0 con 0x7fb38c0c1d80 2021-02-10T11:30:16.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.557 7fb37dffb700 1 --2- 172.21.2.101:0/3408383005 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb36c0186c0 0x7fb36c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:16.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.557 7fb37dffb700 1 -- 172.21.2.101:0/3408383005 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb3840328b0 con 0x7fb38c0c1d80 2021-02-10T11:30:16.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.557 7fb391286700 1 --2- 172.21.2.101:0/3408383005 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb36c0186c0 0x7fb36c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:16.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.558 7fb391286700 1 --2- 172.21.2.101:0/3408383005 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb36c0186c0 0x7fb36c01abc0 crc :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:16.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.588 7fb39346a700 1 -- 172.21.2.101:0/3408383005 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb38c105ae0 con 0x7fb38c0c1d80 2021-02-10T11:30:16.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.590 7fb37dffb700 1 -- 172.21.2.101:0/3408383005 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb384001270 con 0x7fb38c0c1d80 2021-02-10T11:30:16.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.717 7fb39346a700 1 -- 172.21.2.101:0/3408383005 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839265", "target": ["mgr", ""]}) v1 -- 0x7fb38c0c2d60 con 0x7fb36c0186c0 2021-02-10T11:30:16.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.728 7fb37dffb700 1 -- 172.21.2.101:0/3408383005 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fb380000c90 con 0x7fb36c0186c0 2021-02-10T11:30:16.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.734 7fb39346a700 1 -- 172.21.2.101:0/3408383005 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb36c0186c0 msgr2=0x7fb36c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:16.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.734 7fb39346a700 1 --2- 172.21.2.101:0/3408383005 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb36c0186c0 0x7fb36c01abc0 crc :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:16.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.734 7fb39346a700 1 -- 172.21.2.101:0/3408383005 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb38c0c1d80 msgr2=0x7fb38c0c2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:16.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.734 7fb39346a700 1 --2- 172.21.2.101:0/3408383005 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb38c0c1d80 0x7fb38c0c2180 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7fb38400a8f0 tx=0x7fb38400ac90).stop 2021-02-10T11:30:16.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.735 7fb39346a700 1 -- 172.21.2.101:0/3408383005 shutdown_connections 2021-02-10T11:30:16.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.735 7fb39346a700 1 --2- 172.21.2.101:0/3408383005 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb36c0186c0 0x7fb36c01abc0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:16.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.735 7fb39346a700 1 --2- 172.21.2.101:0/3408383005 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb38c0b8200 0x7fb38c0b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:16.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.735 7fb39346a700 1 --2- 172.21.2.101:0/3408383005 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb38c0c1d80 0x7fb38c0c2180 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:16.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.735 7fb39346a700 1 --2- 172.21.2.101:0/3408383005 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb38c190920 0x7fb38c0b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:16.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.735 7fb39346a700 1 -- 172.21.2.101:0/3408383005 >> 172.21.2.101:0/3408383005 conn(0x7fb38c094e10 msgr2=0x7fb38c0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:16.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.735 7fb39346a700 1 -- 172.21.2.101:0/3408383005 shutdown_connections 2021-02-10T11:30:16.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:16.735 7fb39346a700 1 -- 172.21.2.101:0/3408383005 wait complete. 2021-02-10T11:30:16.754 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839266 2021-02-10T11:30:17.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.158 7f792d8bf700 1 Processor -- start 2021-02-10T11:30:17.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.158 7f792d8bf700 1 -- start start 2021-02-10T11:30:17.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.159 7f792d8bf700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f79281b0470 0x7f79281b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:17.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.159 7f792d8bf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f79280ef610 0x7f79280efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:17.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.159 7f792d8bf700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79280f0b50 con 0x7f79280f0030 2021-02-10T11:30:17.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.159 7f792d8bf700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79280e52e0 con 0x7f79281b0470 2021-02-10T11:30:17.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.159 7f792d8bf700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79280e54d0 con 0x7f79280ef610 2021-02-10T11:30:17.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.159 7f7927d75700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f79280f0030 0x7f79280f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:42692/0 (socket says 172.21.2.101:42692) 2021-02-10T11:30:17.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.159 7f7927d75700 1 -- 172.21.2.101:0/1467164784 learned_addr learned my addr 172.21.2.101:0/1467164784 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:17.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.159 7f7927574700 1 --2- 172.21.2.101:0/1467164784 >> v2:172.21.2.108:3300/0 conn(0x7f79281b0470 0x7f79281b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:17.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.159 7f7926d73700 1 --2- 172.21.2.101:0/1467164784 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f79280ef610 0x7f79280efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:17.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.160 7f7926572700 1 -- 172.21.2.101:0/1467164784 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152384876 0 0) 0x7f7920001440 con 0x7f79280f0030 2021-02-10T11:30:17.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.160 7f7926d73700 1 -- 172.21.2.101:0/1467164784 >> v2:172.21.2.108:3300/0 conn(0x7f79281b0470 msgr2=0x7f79281b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:17.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.160 7f7926d73700 1 --2- 172.21.2.101:0/1467164784 >> v2:172.21.2.108:3300/0 conn(0x7f79281b0470 0x7f79281b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:17.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.160 7f7926d73700 1 -- 172.21.2.101:0/1467164784 >> v1:172.21.2.108:6789/0 conn(0x7f79280f0030 legacy=0x7f79280f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:17.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.160 7f7926d73700 1 -- 172.21.2.101:0/1467164784 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79280f0430 con 0x7f79280ef610 2021-02-10T11:30:17.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.160 7f7927574700 1 --2- 172.21.2.101:0/1467164784 >> v2:172.21.2.108:3300/0 conn(0x7f79281b0470 0x7f79281b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:17.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.160 7f7926d73700 1 --2- 172.21.2.101:0/1467164784 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f79280ef610 0x7f79280efa10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f791c006650 tx=0x7f791c009a80).ready entity=mon.2 client_cookie=85f5da1e87b0fd45 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:17.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.160 7f7926572700 1 -- 172.21.2.101:0/1467164784 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f791c0009b0 con 0x7f79280ef610 2021-02-10T11:30:17.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.160 7f7926572700 1 -- 172.21.2.101:0/1467164784 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791c00c0c0 con 0x7f79280ef610 2021-02-10T11:30:17.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.160 7f7926572700 1 -- 172.21.2.101:0/1467164784 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f791c00c5e0 con 0x7f79280ef610 2021-02-10T11:30:17.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.160 7f792d8bf700 1 -- 172.21.2.101:0/1467164784 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f79280ef610 msgr2=0x7f79280efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:17.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.160 7f792d8bf700 1 --2- 172.21.2.101:0/1467164784 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f79280ef610 0x7f79280efa10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f791c006650 tx=0x7f791c009a80).stop 2021-02-10T11:30:17.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.161 7f792d8bf700 1 -- 172.21.2.101:0/1467164784 shutdown_connections 2021-02-10T11:30:17.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.161 7f792d8bf700 1 --2- 172.21.2.101:0/1467164784 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f79280ef610 0x7f79280efa10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:17.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.161 7f792d8bf700 1 --2- 172.21.2.101:0/1467164784 >> v2:172.21.2.108:3300/0 conn(0x7f79281b0470 0x7f79281b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:17.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.161 7f792d8bf700 1 -- 172.21.2.101:0/1467164784 >> 172.21.2.101:0/1467164784 conn(0x7f79280ce3e0 msgr2=0x7f79280e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:17.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.161 7f792d8bf700 1 -- 172.21.2.101:0/1467164784 shutdown_connections 2021-02-10T11:30:17.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.161 7f792d8bf700 1 -- 172.21.2.101:0/1467164784 wait complete. 2021-02-10T11:30:17.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.176 7f792d8bf700 1 Processor -- start 2021-02-10T11:30:17.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.176 7f792d8bf700 1 -- start start 2021-02-10T11:30:17.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.176 7f792d8bf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f79280b39b0 0x7f79280b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:17.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.176 7f792d8bf700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f792819ba90 0x7f79280b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:17.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.176 7f792d8bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f792819be90 0x7f79280b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:17.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.177 7f792d8bf700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79280c15d0 con 0x7f792819be90 2021-02-10T11:30:17.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.177 7f792d8bf700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79280b4ac0 con 0x7f792819ba90 2021-02-10T11:30:17.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.177 7f792d8bf700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79280c1b50 con 0x7f79280b39b0 2021-02-10T11:30:17.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.177 7f7927d75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f792819be90 0x7f79280b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:17.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.177 7f7927d75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f792819be90 0x7f79280b44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48798/0 (socket says 172.21.2.101:48798) 2021-02-10T11:30:17.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.177 7f7927d75700 1 -- 172.21.2.101:0/1131892316 learned_addr learned my addr 172.21.2.101:0/1131892316 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:17.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.177 7f7927574700 1 --2- 172.21.2.101:0/1131892316 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f79280b39b0 0x7f79280b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:17.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.177 7f7926d73700 1 --2- 172.21.2.101:0/1131892316 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f792819ba90 0x7f79280b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:17.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.177 7f7927d75700 1 -- 172.21.2.101:0/1131892316 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f79280b39b0 msgr2=0x7f79280b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:17.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.177 7f7927d75700 1 --2- 172.21.2.101:0/1131892316 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f79280b39b0 0x7f79280b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:17.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.177 7f7927d75700 1 -- 172.21.2.101:0/1131892316 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f792819ba90 msgr2=0x7f79280b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:17.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.177 7f7927d75700 1 --2- 172.21.2.101:0/1131892316 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f792819ba90 0x7f79280b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:17.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.177 7f7927d75700 1 -- 172.21.2.101:0/1131892316 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79280c1d70 con 0x7f792819be90 2021-02-10T11:30:17.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.177 7f7927d75700 1 --2- 172.21.2.101:0/1131892316 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f792819be90 0x7f79280b44a0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7f7920001b30 tx=0x7f792000aca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:17.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.178 7f790ffff700 1 -- 172.21.2.101:0/1131892316 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f79200070d0 con 0x7f792819be90 2021-02-10T11:30:17.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.178 7f790ffff700 1 -- 172.21.2.101:0/1131892316 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79200073d0 con 0x7f792819be90 2021-02-10T11:30:17.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.178 7f792d8bf700 1 -- 172.21.2.101:0/1131892316 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79280c1d70 con 0x7f792819be90 2021-02-10T11:30:17.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.178 7f790ffff700 1 -- 172.21.2.101:0/1131892316 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f79200048b0 con 0x7f792819be90 2021-02-10T11:30:17.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.178 7f792d8bf700 1 -- 172.21.2.101:0/1131892316 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79280c1d70 con 0x7f792819be90 2021-02-10T11:30:17.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.178 7f790ffff700 1 -- 172.21.2.101:0/1131892316 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7920001270 con 0x7f792819be90 2021-02-10T11:30:17.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.179 7f790ffff700 1 --2- 172.21.2.101:0/1131892316 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79080186d0 0x7f790801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:17.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.179 7f790ffff700 1 -- 172.21.2.101:0/1131892316 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7920032b50 con 0x7f792819be90 2021-02-10T11:30:17.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.179 7f7927574700 1 --2- 172.21.2.101:0/1131892316 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79080186d0 0x7f790801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:17.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.179 7f7927574700 1 --2- 172.21.2.101:0/1131892316 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79080186d0 0x7f790801abd0 crc :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:17.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.208 7f792d8bf700 1 -- 172.21.2.101:0/1131892316 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7928105ae0 con 0x7f792819be90 2021-02-10T11:30:17.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.211 7f790ffff700 1 -- 172.21.2.101:0/1131892316 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f79200073d0 con 0x7f792819be90 2021-02-10T11:30:17.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.342 7f792d8bf700 1 -- 172.21.2.101:0/1131892316 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839266", "target": ["mgr", ""]}) v1 -- 0x7f7928105ae0 con 0x7f79080186d0 2021-02-10T11:30:17.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.352 7f790ffff700 1 -- 172.21.2.101:0/1131892316 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f7918006a30 con 0x7f79080186d0 2021-02-10T11:30:17.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.359 7f792d8bf700 1 -- 172.21.2.101:0/1131892316 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79080186d0 msgr2=0x7f790801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:17.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.359 7f792d8bf700 1 --2- 172.21.2.101:0/1131892316 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79080186d0 0x7f790801abd0 crc :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:17.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.359 7f792d8bf700 1 -- 172.21.2.101:0/1131892316 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f792819be90 msgr2=0x7f79280b44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:17.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.359 7f792d8bf700 1 --2- 172.21.2.101:0/1131892316 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f792819be90 0x7f79280b44a0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7f7920001b30 tx=0x7f792000aca0).stop 2021-02-10T11:30:17.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.359 7f792d8bf700 1 -- 172.21.2.101:0/1131892316 shutdown_connections 2021-02-10T11:30:17.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.359 7f792d8bf700 1 --2- 172.21.2.101:0/1131892316 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f792819be90 0x7f79280b44a0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:17.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.359 7f792d8bf700 1 --2- 172.21.2.101:0/1131892316 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f792819ba90 0x7f79280b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:17.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.359 7f792d8bf700 1 --2- 172.21.2.101:0/1131892316 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79080186d0 0x7f790801abd0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:17.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.359 7f792d8bf700 1 --2- 172.21.2.101:0/1131892316 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f79280b39b0 0x7f79280b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:17.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.359 7f792d8bf700 1 -- 172.21.2.101:0/1131892316 >> 172.21.2.101:0/1131892316 conn(0x7f7928094e10 msgr2=0x7f79280a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:17.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.359 7f792d8bf700 1 -- 172.21.2.101:0/1131892316 shutdown_connections 2021-02-10T11:30:17.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.359 7f792d8bf700 1 -- 172.21.2.101:0/1131892316 wait complete. 2021-02-10T11:30:17.379 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839267 2021-02-10T11:30:17.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.782 7f1a0b658700 1 Processor -- start 2021-02-10T11:30:17.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.782 7f1a0b658700 1 -- start start 2021-02-10T11:30:17.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.782 7f1a0b658700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1a041b0870 0x7f1a040f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:17.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.782 7f1a0b658700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a040efe00 0x7f1a040f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:17.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.782 7f1a0b658700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a041b0c70 con 0x7f1a041b0470 2021-02-10T11:30:17.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.782 7f1a0b658700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a040f0b50 con 0x7f1a041b0870 2021-02-10T11:30:17.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.782 7f1a0b658700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a040f0820 con 0x7f1a040efe00 2021-02-10T11:30:17.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.783 7f1a09474700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f1a041b0470 0x7f1a040ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49296/0 (socket says 172.21.2.101:49296) 2021-02-10T11:30:17.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.783 7f1a09474700 1 -- 172.21.2.101:0/328451541 learned_addr learned my addr 172.21.2.101:0/328451541 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:17.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.783 7f1a08c73700 1 --2- 172.21.2.101:0/328451541 >> v2:172.21.2.108:3300/0 conn(0x7f1a041b0870 0x7f1a040f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:17.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.783 7f1a09c75700 1 --2- 172.21.2.101:0/328451541 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a040efe00 0x7f1a040f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:17.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.783 7f19f7fff700 1 -- 172.21.2.101:0/328451541 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1900697111 0 0) 0x7f19f8000ce0 con 0x7f1a041b0470 2021-02-10T11:30:17.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.783 7f19f7fff700 1 -- 172.21.2.101:0/328451541 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19f0002fe0 con 0x7f1a041b0470 2021-02-10T11:30:17.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.783 7f1a09c75700 1 -- 172.21.2.101:0/328451541 >> v2:172.21.2.108:3300/0 conn(0x7f1a041b0870 msgr2=0x7f1a040f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:17.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.783 7f1a09c75700 1 --2- 172.21.2.101:0/328451541 >> v2:172.21.2.108:3300/0 conn(0x7f1a041b0870 0x7f1a040f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:17.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.783 7f1a09c75700 1 -- 172.21.2.101:0/328451541 >> v1:172.21.2.101:6789/0 conn(0x7f1a041b0470 legacy=0x7f1a040ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:17.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.783 7f1a09c75700 1 -- 172.21.2.101:0/328451541 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a040e52e0 con 0x7f1a040efe00 2021-02-10T11:30:17.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.783 7f1a09c75700 1 --2- 172.21.2.101:0/328451541 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a040efe00 0x7f1a040f0200 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f19fc006e50 tx=0x7f19fc009e70).ready entity=mon.2 client_cookie=779c77df2493c80b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:17.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.783 7f19f7fff700 1 -- 172.21.2.101:0/328451541 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19fc006a20 con 0x7f1a040efe00 2021-02-10T11:30:17.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.784 7f19f7fff700 1 -- 172.21.2.101:0/328451541 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19fc006790 con 0x7f1a040efe00 2021-02-10T11:30:17.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.784 7f19f7fff700 1 -- 172.21.2.101:0/328451541 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19fc000f90 con 0x7f1a040efe00 2021-02-10T11:30:17.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.784 7f1a0b658700 1 -- 172.21.2.101:0/328451541 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a040efe00 msgr2=0x7f1a040f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:17.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.784 7f1a0b658700 1 --2- 172.21.2.101:0/328451541 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a040efe00 0x7f1a040f0200 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f19fc006e50 tx=0x7f19fc009e70).stop 2021-02-10T11:30:17.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.784 7f1a0b658700 1 -- 172.21.2.101:0/328451541 shutdown_connections 2021-02-10T11:30:17.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.784 7f1a0b658700 1 --2- 172.21.2.101:0/328451541 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a040efe00 0x7f1a040f0200 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:17.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.784 7f1a0b658700 1 --2- 172.21.2.101:0/328451541 >> v2:172.21.2.108:3300/0 conn(0x7f1a041b0870 0x7f1a040f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:17.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.784 7f1a0b658700 1 -- 172.21.2.101:0/328451541 >> 172.21.2.101:0/328451541 conn(0x7f1a040ce3e0 msgr2=0x7f1a040e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:17.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.785 7f1a0b658700 1 -- 172.21.2.101:0/328451541 shutdown_connections 2021-02-10T11:30:17.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.785 7f1a0b658700 1 -- 172.21.2.101:0/328451541 wait complete. 2021-02-10T11:30:17.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.799 7f1a0b658700 1 Processor -- start 2021-02-10T11:30:17.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.799 7f1a0b658700 1 -- start start 2021-02-10T11:30:17.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.800 7f1a0b658700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a0409f140 0x7f1a040c1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:17.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.800 7f1a0b658700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a040c1f60 0x7f1a0409e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:17.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.800 7f1a0b658700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a0409e920 0x7f1a040c2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:17.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.800 7f1a0b658700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a040c23e0 con 0x7f1a0409f140 2021-02-10T11:30:17.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.800 7f1a0b658700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0409ed20 con 0x7f1a040c1f60 2021-02-10T11:30:17.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.800 7f1a0b658700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a04105d10 con 0x7f1a0409e920 2021-02-10T11:30:17.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.800 7f1a09474700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a0409f140 0x7f1a040c1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:17.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.800 7f1a09474700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a0409f140 0x7f1a040c1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48808/0 (socket says 172.21.2.101:48808) 2021-02-10T11:30:17.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.800 7f1a09474700 1 -- 172.21.2.101:0/1988003800 learned_addr learned my addr 172.21.2.101:0/1988003800 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:17.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.800 7f1a09c75700 1 --2- 172.21.2.101:0/1988003800 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a0409e920 0x7f1a040c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:17.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.800 7f1a08c73700 1 --2- 172.21.2.101:0/1988003800 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a040c1f60 0x7f1a0409e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:17.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.800 7f1a09c75700 1 -- 172.21.2.101:0/1988003800 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a040c1f60 msgr2=0x7f1a0409e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:17.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.800 7f1a09c75700 1 --2- 172.21.2.101:0/1988003800 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a040c1f60 0x7f1a0409e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:17.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.800 7f1a09c75700 1 -- 172.21.2.101:0/1988003800 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a0409f140 msgr2=0x7f1a040c1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:17.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.800 7f1a09c75700 1 --2- 172.21.2.101:0/1988003800 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a0409f140 0x7f1a040c1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:17.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.800 7f1a09c75700 1 -- 172.21.2.101:0/1988003800 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a041a4910 con 0x7f1a0409e920 2021-02-10T11:30:17.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.800 7f1a09c75700 1 --2- 172.21.2.101:0/1988003800 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a0409e920 0x7f1a040c2960 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f19fc009930 tx=0x7f19fc00bbf0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:17.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.801 7f19f5ffb700 1 -- 172.21.2.101:0/1988003800 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19fc0093d0 con 0x7f1a0409e920 2021-02-10T11:30:17.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.801 7f19f5ffb700 1 -- 172.21.2.101:0/1988003800 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19fc002080 con 0x7f1a0409e920 2021-02-10T11:30:17.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.801 7f19f5ffb700 1 -- 172.21.2.101:0/1988003800 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19fc0024d0 con 0x7f1a0409e920 2021-02-10T11:30:17.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.801 7f1a0b658700 1 -- 172.21.2.101:0/1988003800 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a041a4910 con 0x7f1a0409e920 2021-02-10T11:30:17.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.801 7f1a0b658700 1 -- 172.21.2.101:0/1988003800 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a040c3270 con 0x7f1a0409e920 2021-02-10T11:30:17.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.801 7f19f5ffb700 1 -- 172.21.2.101:0/1988003800 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f19fc009300 con 0x7f1a0409e920 2021-02-10T11:30:17.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.801 7f19f5ffb700 1 --2- 172.21.2.101:0/1988003800 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19e40186c0 0x7f19e401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:17.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.801 7f19f5ffb700 1 -- 172.21.2.101:0/1988003800 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f19fc032830 con 0x7f1a0409e920 2021-02-10T11:30:17.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.801 7f1a09474700 1 --2- 172.21.2.101:0/1988003800 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19e40186c0 0x7f19e401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:17.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.802 7f1a09474700 1 --2- 172.21.2.101:0/1988003800 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19e40186c0 0x7f19e401abc0 crc :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:17.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.831 7f1a0b658700 1 -- 172.21.2.101:0/1988003800 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a040f0b50 con 0x7f1a0409e920 2021-02-10T11:30:17.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.835 7f19f5ffb700 1 -- 172.21.2.101:0/1988003800 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f19fc002790 con 0x7f1a0409e920 2021-02-10T11:30:17.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.966 7f1a0b658700 1 -- 172.21.2.101:0/1988003800 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839267", "target": ["mgr", ""]}) v1 -- 0x7f1a040c23e0 con 0x7f19e40186c0 2021-02-10T11:30:17.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.976 7f19f5ffb700 1 -- 172.21.2.101:0/1988003800 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f19f8000ef0 con 0x7f19e40186c0 2021-02-10T11:30:17.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.983 7f1a0b658700 1 -- 172.21.2.101:0/1988003800 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19e40186c0 msgr2=0x7f19e401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:17.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.983 7f1a0b658700 1 --2- 172.21.2.101:0/1988003800 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19e40186c0 0x7f19e401abc0 crc :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:17.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.983 7f1a0b658700 1 -- 172.21.2.101:0/1988003800 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a0409e920 msgr2=0x7f1a040c2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:17.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.983 7f1a0b658700 1 --2- 172.21.2.101:0/1988003800 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a0409e920 0x7f1a040c2960 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f19fc009930 tx=0x7f19fc00bbf0).stop 2021-02-10T11:30:17.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.983 7f1a0b658700 1 -- 172.21.2.101:0/1988003800 shutdown_connections 2021-02-10T11:30:17.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.983 7f1a0b658700 1 --2- 172.21.2.101:0/1988003800 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19e40186c0 0x7f19e401abc0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:17.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.983 7f1a0b658700 1 --2- 172.21.2.101:0/1988003800 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a0409e920 0x7f1a040c2960 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:17.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.983 7f1a0b658700 1 --2- 172.21.2.101:0/1988003800 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a040c1f60 0x7f1a0409e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:17.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.983 7f1a0b658700 1 --2- 172.21.2.101:0/1988003800 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a0409f140 0x7f1a040c1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:17.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.983 7f1a0b658700 1 -- 172.21.2.101:0/1988003800 >> 172.21.2.101:0/1988003800 conn(0x7f1a0409fc10 msgr2=0x7f1a040b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:17.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.983 7f1a0b658700 1 -- 172.21.2.101:0/1988003800 shutdown_connections 2021-02-10T11:30:17.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:17.983 7f1a0b658700 1 -- 172.21.2.101:0/1988003800 wait complete. 2021-02-10T11:30:18.003 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839268 2021-02-10T11:30:18.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.405 7f4d1d012700 1 Processor -- start 2021-02-10T11:30:18.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.405 7f4d1d012700 1 -- start start 2021-02-10T11:30:18.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.406 7f4d1d012700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d181b0470 0x7f4d181b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:18.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.406 7f4d1d012700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4d180ef610 0x7f4d180efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:18.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.406 7f4d1d012700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d180f0b50 con 0x7f4d180f0030 2021-02-10T11:30:18.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.406 7f4d1d012700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d180e52e0 con 0x7f4d180ef610 2021-02-10T11:30:18.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.406 7f4d1d012700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d180e5490 con 0x7f4d181b0470 2021-02-10T11:30:18.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.406 7f4d1759e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f4d180f0030 0x7f4d180f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49314/0 (socket says 172.21.2.101:49314) 2021-02-10T11:30:18.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.406 7f4d1759e700 1 -- 172.21.2.101:0/2114184255 learned_addr learned my addr 172.21.2.101:0/2114184255 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:18.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.406 7f4d1659c700 1 --2- 172.21.2.101:0/2114184255 >> v2:172.21.2.108:3300/0 conn(0x7f4d180ef610 0x7f4d180efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:18.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.406 7f4d16d9d700 1 --2- 172.21.2.101:0/2114184255 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d181b0470 0x7f4d181b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:18.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.406 7f4d15d9b700 1 -- 172.21.2.101:0/2114184255 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 553169432 0 0) 0x7f4d08001180 con 0x7f4d180f0030 2021-02-10T11:30:18.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.406 7f4d15d9b700 1 -- 172.21.2.101:0/2114184255 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d04002fe0 con 0x7f4d180f0030 2021-02-10T11:30:18.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.406 7f4d15d9b700 1 -- 172.21.2.101:0/2114184255 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1981635888 0 0) 0x7f4d080010e0 con 0x7f4d180f0030 2021-02-10T11:30:18.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.406 7f4d15d9b700 1 -- 172.21.2.101:0/2114184255 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d181b0470 msgr2=0x7f4d181b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:18.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.406 7f4d15d9b700 1 --2- 172.21.2.101:0/2114184255 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d181b0470 0x7f4d181b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:18.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.406 7f4d15d9b700 1 -- 172.21.2.101:0/2114184255 >> v2:172.21.2.108:3300/0 conn(0x7f4d180ef610 msgr2=0x7f4d180efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:18.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.406 7f4d15d9b700 1 --2- 172.21.2.101:0/2114184255 >> v2:172.21.2.108:3300/0 conn(0x7f4d180ef610 0x7f4d180efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:18.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.406 7f4d15d9b700 1 -- 172.21.2.101:0/2114184255 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d180f0430 con 0x7f4d180f0030 2021-02-10T11:30:18.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.406 7f4d15d9b700 1 -- 172.21.2.101:0/2114184255 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f4d08001600 con 0x7f4d180f0030 2021-02-10T11:30:18.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.406 7f4d15d9b700 1 -- 172.21.2.101:0/2114184255 >> v1:172.21.2.101:6789/0 conn(0x7f4d180f0030 legacy=0x7f4d180f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:18.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.406 7f4d15d9b700 1 --2- 172.21.2.101:0/2114184255 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d04003a60 0x7f4d04005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:18.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.407 7f4d15d9b700 1 --2- 172.21.2.101:0/2114184255 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d0404a7a0 0x7f4d0404cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:18.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.407 7f4d15d9b700 1 --2- 172.21.2.101:0/2114184255 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d0404d5a0 0x7f4d0404fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:18.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.407 7f4d15d9b700 1 -- 172.21.2.101:0/2114184255 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d040502e0 con 0x7f4d0404a7a0 2021-02-10T11:30:18.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.407 7f4d15d9b700 1 -- 172.21.2.101:0/2114184255 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d04050500 con 0x7f4d04003a60 2021-02-10T11:30:18.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.407 7f4d15d9b700 1 -- 172.21.2.101:0/2114184255 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d04050720 con 0x7f4d0404d5a0 2021-02-10T11:30:18.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.407 7f4d1659c700 1 --2- 172.21.2.101:0/2114184255 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d0404a7a0 0x7f4d0404cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:18.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.407 7f4d16d9d700 1 --2- 172.21.2.101:0/2114184255 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d04003a60 0x7f4d04005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:18.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.407 7f4d1759e700 1 --2- 172.21.2.101:0/2114184255 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d0404d5a0 0x7f4d0404fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:18.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.407 7f4d16d9d700 1 -- 172.21.2.101:0/2114184255 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d0404d5a0 msgr2=0x7f4d0404fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:18.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.407 7f4d16d9d700 1 --2- 172.21.2.101:0/2114184255 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d0404d5a0 0x7f4d0404fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:18.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.407 7f4d16d9d700 1 -- 172.21.2.101:0/2114184255 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d0404a7a0 msgr2=0x7f4d0404cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:18.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.407 7f4d16d9d700 1 --2- 172.21.2.101:0/2114184255 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d0404a7a0 0x7f4d0404cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:18.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.407 7f4d16d9d700 1 -- 172.21.2.101:0/2114184255 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d040509c0 con 0x7f4d04003a60 2021-02-10T11:30:18.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.407 7f4d16d9d700 1 --2- 172.21.2.101:0/2114184255 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d04003a60 0x7f4d04005ef0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f4d00006ac0 tx=0x7f4d00009ab0).ready entity=mon.1 client_cookie=e1d9d2afbfca6f08 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:18.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.407 7f4d15d9b700 1 -- 172.21.2.101:0/2114184255 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d0000c0c0 con 0x7f4d04003a60 2021-02-10T11:30:18.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.407 7f4d15d9b700 1 -- 172.21.2.101:0/2114184255 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0000c0c0 con 0x7f4d04003a60 2021-02-10T11:30:18.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.407 7f4d15d9b700 1 -- 172.21.2.101:0/2114184255 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d0000c400 con 0x7f4d04003a60 2021-02-10T11:30:18.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.407 7f4d1d012700 1 -- 172.21.2.101:0/2114184255 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d04003a60 msgr2=0x7f4d04005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:18.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.407 7f4d1d012700 1 --2- 172.21.2.101:0/2114184255 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d04003a60 0x7f4d04005ef0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f4d00006ac0 tx=0x7f4d00009ab0).stop 2021-02-10T11:30:18.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.407 7f4d1759e700 1 -- 172.21.2.101:0/2114184255 reap_dead start 2021-02-10T11:30:18.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.408 7f4d1d012700 1 -- 172.21.2.101:0/2114184255 shutdown_connections 2021-02-10T11:30:18.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.408 7f4d1d012700 1 -- 172.21.2.101:0/2114184255 >> 172.21.2.101:0/2114184255 conn(0x7f4d180ce3e0 msgr2=0x7f4d180e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:18.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.408 7f4d1d012700 1 -- 172.21.2.101:0/2114184255 shutdown_connections 2021-02-10T11:30:18.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.408 7f4d1d012700 1 -- 172.21.2.101:0/2114184255 wait complete. 2021-02-10T11:30:18.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.423 7f4d1d012700 1 Processor -- start 2021-02-10T11:30:18.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.423 7f4d1d012700 1 -- start start 2021-02-10T11:30:18.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.424 7f4d1d012700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d1809b930 0x7f4d180b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:18.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.424 7f4d1d012700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d180b8410 0x7f4d180b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:18.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.424 7f4d1d012700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d180c1f80 0x7f4d180c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:18.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.424 7f4d1d012700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d180c1a00 con 0x7f4d180c1f80 2021-02-10T11:30:18.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.424 7f4d1d012700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d180b8e30 con 0x7f4d180b8410 2021-02-10T11:30:18.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.424 7f4d1d012700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d180b9050 con 0x7f4d1809b930 2021-02-10T11:30:18.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.424 7f4d1759e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d180c1f80 0x7f4d180c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:18.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.424 7f4d1759e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d180c1f80 0x7f4d180c2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48832/0 (socket says 172.21.2.101:48832) 2021-02-10T11:30:18.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.424 7f4d1759e700 1 -- 172.21.2.101:0/3436976244 learned_addr learned my addr 172.21.2.101:0/3436976244 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:18.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.424 7f4d1659c700 1 --2- 172.21.2.101:0/3436976244 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d180b8410 0x7f4d180b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:18.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.424 7f4d16d9d700 1 --2- 172.21.2.101:0/3436976244 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d1809b930 0x7f4d180b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:18.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.424 7f4d1759e700 1 -- 172.21.2.101:0/3436976244 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d1809b930 msgr2=0x7f4d180b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:18.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.424 7f4d1759e700 1 --2- 172.21.2.101:0/3436976244 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d1809b930 0x7f4d180b7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:18.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.424 7f4d1759e700 1 -- 172.21.2.101:0/3436976244 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d180b8410 msgr2=0x7f4d180b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:18.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.424 7f4d1759e700 1 --2- 172.21.2.101:0/3436976244 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d180b8410 0x7f4d180b8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:18.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.424 7f4d1759e700 1 -- 172.21.2.101:0/3436976244 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d180c29a0 con 0x7f4d180c1f80 2021-02-10T11:30:18.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.425 7f4d1759e700 1 --2- 172.21.2.101:0/3436976244 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d180c1f80 0x7f4d180c2380 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f4d0800a570 tx=0x7f4d0800a910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:18.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.425 7f4d0f7fe700 1 -- 172.21.2.101:0/3436976244 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d08000b00 con 0x7f4d180c1f80 2021-02-10T11:30:18.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.425 7f4d0f7fe700 1 -- 172.21.2.101:0/3436976244 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d080070d0 con 0x7f4d180c1f80 2021-02-10T11:30:18.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.425 7f4d0f7fe700 1 -- 172.21.2.101:0/3436976244 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d080045b0 con 0x7f4d180c1f80 2021-02-10T11:30:18.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.425 7f4d1d012700 1 -- 172.21.2.101:0/3436976244 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d180c29a0 con 0x7f4d180c1f80 2021-02-10T11:30:18.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.425 7f4d1d012700 1 -- 172.21.2.101:0/3436976244 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d180c2f30 con 0x7f4d180c1f80 2021-02-10T11:30:18.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.426 7f4d0f7fe700 1 -- 172.21.2.101:0/3436976244 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4d080043a0 con 0x7f4d180c1f80 2021-02-10T11:30:18.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.426 7f4d0f7fe700 1 --2- 172.21.2.101:0/3436976244 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cfc0186d0 0x7f4cfc01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:18.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.426 7f4d0f7fe700 1 -- 172.21.2.101:0/3436976244 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4d08032620 con 0x7f4d180c1f80 2021-02-10T11:30:18.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.426 7f4d16d9d700 1 --2- 172.21.2.101:0/3436976244 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cfc0186d0 0x7f4cfc01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:18.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.426 7f4d16d9d700 1 --2- 172.21.2.101:0/3436976244 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cfc0186d0 0x7f4cfc01abd0 crc :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:18.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.455 7f4d1d012700 1 -- 172.21.2.101:0/3436976244 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d18105b30 con 0x7f4d180c1f80 2021-02-10T11:30:18.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.458 7f4d0f7fe700 1 -- 172.21.2.101:0/3436976244 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4d08000b00 con 0x7f4d180c1f80 2021-02-10T11:30:18.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.588 7f4d1d012700 1 -- 172.21.2.101:0/3436976244 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839268", "target": ["mgr", ""]}) v1 -- 0x7f4d180c2f30 con 0x7f4cfc0186d0 2021-02-10T11:30:18.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.598 7f4d0f7fe700 1 -- 172.21.2.101:0/3436976244 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f4d00000c90 con 0x7f4cfc0186d0 2021-02-10T11:30:18.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.605 7f4d1d012700 1 -- 172.21.2.101:0/3436976244 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cfc0186d0 msgr2=0x7f4cfc01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:18.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.605 7f4d1d012700 1 --2- 172.21.2.101:0/3436976244 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cfc0186d0 0x7f4cfc01abd0 crc :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:18.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.605 7f4d1d012700 1 -- 172.21.2.101:0/3436976244 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d180c1f80 msgr2=0x7f4d180c2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:18.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.605 7f4d1d012700 1 --2- 172.21.2.101:0/3436976244 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d180c1f80 0x7f4d180c2380 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f4d0800a570 tx=0x7f4d0800a910).stop 2021-02-10T11:30:18.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.606 7f4d1d012700 1 -- 172.21.2.101:0/3436976244 shutdown_connections 2021-02-10T11:30:18.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.606 7f4d1d012700 1 --2- 172.21.2.101:0/3436976244 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d180c1f80 0x7f4d180c2380 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:18.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.606 7f4d1d012700 1 --2- 172.21.2.101:0/3436976244 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d180b8410 0x7f4d180b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:18.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.606 7f4d1d012700 1 --2- 172.21.2.101:0/3436976244 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cfc0186d0 0x7f4cfc01abd0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:18.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.606 7f4d1d012700 1 --2- 172.21.2.101:0/3436976244 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d1809b930 0x7f4d180b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:18.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.606 7f4d1d012700 1 -- 172.21.2.101:0/3436976244 >> 172.21.2.101:0/3436976244 conn(0x7f4d18094e10 msgr2=0x7f4d180a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:18.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.606 7f4d1d012700 1 -- 172.21.2.101:0/3436976244 shutdown_connections 2021-02-10T11:30:18.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:18.606 7f4d1d012700 1 -- 172.21.2.101:0/3436976244 wait complete. 2021-02-10T11:30:18.624 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839269 2021-02-10T11:30:19.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.031 7f437e779700 1 Processor -- start 2021-02-10T11:30:19.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.031 7f437e779700 1 -- start start 2021-02-10T11:30:19.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.032 7f437e779700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43781b0870 0x7f43780f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:19.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.032 7f437e779700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f43780efe00 0x7f43780f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:19.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.032 7f437e779700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43781b0c70 con 0x7f43781b0470 2021-02-10T11:30:19.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.032 7f437e779700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43780f0b50 con 0x7f43780efe00 2021-02-10T11:30:19.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.032 7f437e779700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43780f0820 con 0x7f43781b0870 2021-02-10T11:30:19.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.032 7f43777fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43781b0870 0x7f43780f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:19.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.032 7f437cd96700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f43780efe00 0x7f43780f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:19.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.032 7f437cd96700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f43780efe00 0x7f43780f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:59326/0 (socket says 172.21.2.101:59326) 2021-02-10T11:30:19.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.032 7f43777fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43781b0870 0x7f43780f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:41568/0 (socket says 172.21.2.101:41568) 2021-02-10T11:30:19.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.032 7f437cd96700 1 -- 172.21.2.101:0/3961497554 learned_addr learned my addr 172.21.2.101:0/3961497554 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:19.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.032 7f4377fff700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f43781b0470 0x7f43780ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:42740/0 (socket says 172.21.2.101:42740) 2021-02-10T11:30:19.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.033 7f437cd96700 1 -- 172.21.2.101:0/3961497554 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43781b0870 msgr2=0x7f43780f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:19.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.033 7f437cd96700 1 --2- 172.21.2.101:0/3961497554 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43781b0870 0x7f43780f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:19.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.033 7f437cd96700 1 -- 172.21.2.101:0/3961497554 >> v1:172.21.2.108:6789/0 conn(0x7f43781b0470 legacy=0x7f43780ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:19.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.033 7f437cd96700 1 -- 172.21.2.101:0/3961497554 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43780e52e0 con 0x7f43780efe00 2021-02-10T11:30:19.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.033 7f43777fe700 1 --2- 172.21.2.101:0/3961497554 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43781b0870 0x7f43780f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:19.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.033 7f437cd96700 1 --2- 172.21.2.101:0/3961497554 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f43780efe00 0x7f43780f0200 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f4370006aa0 tx=0x7f4370009f20).ready entity=mon.1 client_cookie=aef946a2a3365324 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:19.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.034 7f4376ffd700 1 -- 172.21.2.101:0/3961497554 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f437000b5c0 con 0x7f43780efe00 2021-02-10T11:30:19.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.034 7f4376ffd700 1 -- 172.21.2.101:0/3961497554 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4370003030 con 0x7f43780efe00 2021-02-10T11:30:19.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.034 7f4376ffd700 1 -- 172.21.2.101:0/3961497554 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f437000ba30 con 0x7f43780efe00 2021-02-10T11:30:19.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.034 7f437e779700 1 -- 172.21.2.101:0/3961497554 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f43780efe00 msgr2=0x7f43780f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:19.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.034 7f437e779700 1 --2- 172.21.2.101:0/3961497554 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f43780efe00 0x7f43780f0200 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f4370006aa0 tx=0x7f4370009f20).stop 2021-02-10T11:30:19.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.034 7f437e779700 1 -- 172.21.2.101:0/3961497554 shutdown_connections 2021-02-10T11:30:19.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.034 7f437e779700 1 --2- 172.21.2.101:0/3961497554 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43781b0870 0x7f43780f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:19.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.034 7f437e779700 1 --2- 172.21.2.101:0/3961497554 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f43780efe00 0x7f43780f0200 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:19.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.034 7f437e779700 1 -- 172.21.2.101:0/3961497554 >> 172.21.2.101:0/3961497554 conn(0x7f43780ce3e0 msgr2=0x7f43780e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:19.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.035 7f437e779700 1 -- 172.21.2.101:0/3961497554 shutdown_connections 2021-02-10T11:30:19.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.035 7f437e779700 1 -- 172.21.2.101:0/3961497554 wait complete. 2021-02-10T11:30:19.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.049 7f437e779700 1 Processor -- start 2021-02-10T11:30:19.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.049 7f437e779700 1 -- start start 2021-02-10T11:30:19.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.049 7f437e779700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f43780a0420 0x7f43780a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:19.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.050 7f437e779700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4378189d00 0x7f437818a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:19.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.050 7f437e779700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f437818a720 0x7f43780b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:19.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.050 7f437e779700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4378189800 con 0x7f4378189d00 2021-02-10T11:30:19.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.050 7f437e779700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437818ab20 con 0x7f43780a0420 2021-02-10T11:30:19.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.050 7f437e779700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43780a1140 con 0x7f437818a720 2021-02-10T11:30:19.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.050 7f43777fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4378189d00 0x7f437818a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:19.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.050 7f437cd96700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f437818a720 0x7f43780b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:19.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.050 7f43777fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4378189d00 0x7f437818a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48844/0 (socket says 172.21.2.101:48844) 2021-02-10T11:30:19.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.050 7f43777fe700 1 -- 172.21.2.101:0/1379940256 learned_addr learned my addr 172.21.2.101:0/1379940256 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:19.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.050 7f4377fff700 1 --2- 172.21.2.101:0/1379940256 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f43780a0420 0x7f43780a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:19.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.050 7f437cd96700 1 -- 172.21.2.101:0/1379940256 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f43780a0420 msgr2=0x7f43780a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:19.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.050 7f437cd96700 1 --2- 172.21.2.101:0/1379940256 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f43780a0420 0x7f43780a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:19.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.050 7f437cd96700 1 -- 172.21.2.101:0/1379940256 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4378189d00 msgr2=0x7f437818a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:19.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.050 7f437cd96700 1 --2- 172.21.2.101:0/1379940256 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4378189d00 0x7f437818a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:19.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.050 7f437cd96700 1 -- 172.21.2.101:0/1379940256 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43780b8200 con 0x7f437818a720 2021-02-10T11:30:19.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.050 7f437cd96700 1 --2- 172.21.2.101:0/1379940256 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f437818a720 0x7f43780b7be0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f4370009410 tx=0x7f437000bc10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:19.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.051 7f4374ff9700 1 -- 172.21.2.101:0/1379940256 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4370009450 con 0x7f437818a720 2021-02-10T11:30:19.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.051 7f4374ff9700 1 -- 172.21.2.101:0/1379940256 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4370002100 con 0x7f437818a720 2021-02-10T11:30:19.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.051 7f437e779700 1 -- 172.21.2.101:0/1379940256 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43780b8200 con 0x7f437818a720 2021-02-10T11:30:19.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.051 7f437e779700 1 -- 172.21.2.101:0/1379940256 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43780b8200 con 0x7f437818a720 2021-02-10T11:30:19.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.051 7f4374ff9700 1 -- 172.21.2.101:0/1379940256 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4370006020 con 0x7f437818a720 2021-02-10T11:30:19.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.051 7f4374ff9700 1 -- 172.21.2.101:0/1379940256 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f43700065b0 con 0x7f437818a720 2021-02-10T11:30:19.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.051 7f4374ff9700 1 --2- 172.21.2.101:0/1379940256 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f43580186c0 0x7f435801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:19.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.051 7f4374ff9700 1 -- 172.21.2.101:0/1379940256 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4370032a40 con 0x7f437818a720 2021-02-10T11:30:19.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.051 7f4377fff700 1 --2- 172.21.2.101:0/1379940256 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f43580186c0 0x7f435801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:19.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.052 7f4377fff700 1 --2- 172.21.2.101:0/1379940256 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f43580186c0 0x7f435801abc0 crc :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:19.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.081 7f437e779700 1 -- 172.21.2.101:0/1379940256 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f437818ab20 con 0x7f437818a720 2021-02-10T11:30:19.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.085 7f4374ff9700 1 -- 172.21.2.101:0/1379940256 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f43700063a0 con 0x7f437818a720 2021-02-10T11:30:19.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.215 7f437e779700 1 -- 172.21.2.101:0/1379940256 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839269", "target": ["mgr", ""]}) v1 -- 0x7f437818ab20 con 0x7f43580186c0 2021-02-10T11:30:19.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.224 7f4374ff9700 1 -- 172.21.2.101:0/1379940256 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f4368007da0 con 0x7f43580186c0 2021-02-10T11:30:19.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.232 7f437e779700 1 -- 172.21.2.101:0/1379940256 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f43580186c0 msgr2=0x7f435801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:19.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.232 7f437e779700 1 --2- 172.21.2.101:0/1379940256 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f43580186c0 0x7f435801abc0 crc :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:19.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.232 7f437e779700 1 -- 172.21.2.101:0/1379940256 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f437818a720 msgr2=0x7f43780b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:19.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.232 7f437e779700 1 --2- 172.21.2.101:0/1379940256 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f437818a720 0x7f43780b7be0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f4370009410 tx=0x7f437000bc10).stop 2021-02-10T11:30:19.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.232 7f437e779700 1 -- 172.21.2.101:0/1379940256 shutdown_connections 2021-02-10T11:30:19.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.232 7f437e779700 1 --2- 172.21.2.101:0/1379940256 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f43580186c0 0x7f435801abc0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:19.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.232 7f437e779700 1 --2- 172.21.2.101:0/1379940256 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f437818a720 0x7f43780b7be0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:19.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.232 7f437e779700 1 --2- 172.21.2.101:0/1379940256 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4378189d00 0x7f437818a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:19.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.232 7f437e779700 1 --2- 172.21.2.101:0/1379940256 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f43780a0420 0x7f43780a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:19.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.232 7f437e779700 1 -- 172.21.2.101:0/1379940256 >> 172.21.2.101:0/1379940256 conn(0x7f4378094e10 msgr2=0x7f43780a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:19.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.233 7f437e779700 1 -- 172.21.2.101:0/1379940256 shutdown_connections 2021-02-10T11:30:19.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.233 7f437e779700 1 -- 172.21.2.101:0/1379940256 wait complete. 2021-02-10T11:30:19.251 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839270 2021-02-10T11:30:19.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.656 7fe5d9f38700 1 Processor -- start 2021-02-10T11:30:19.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.656 7fe5d9f38700 1 -- start start 2021-02-10T11:30:19.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.657 7fe5d9f38700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe5d41b0870 0x7fe5d40f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:19.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.657 7fe5d9f38700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d41b0c70 con 0x7fe5d40efe00 2021-02-10T11:30:19.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.657 7fe5d9f38700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d40f0b50 con 0x7fe5d41b0470 2021-02-10T11:30:19.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.657 7fe5d9f38700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d40e5ea0 con 0x7fe5d41b0870 2021-02-10T11:30:19.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.657 7fe5caffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe5d41b0870 0x7fe5d40f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:19.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.657 7fe5caffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe5d41b0870 0x7fe5d40f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48850/0 (socket says 172.21.2.101:48850) 2021-02-10T11:30:19.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.657 7fe5d3fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe5d40efe00 0x7fe5d40f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49344/0 (socket says 172.21.2.101:49344) 2021-02-10T11:30:19.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.657 7fe5caffd700 1 -- 172.21.2.101:0/2046754686 learned_addr learned my addr 172.21.2.101:0/2046754686 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:19.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.658 7fe5caffd700 1 -- 172.21.2.101:0/2046754686 >> v1:172.21.2.108:6789/0 conn(0x7fe5d41b0470 legacy=0x7fe5d40ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:19.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.658 7fe5caffd700 1 -- 172.21.2.101:0/2046754686 >> v1:172.21.2.101:6789/0 conn(0x7fe5d40efe00 legacy=0x7fe5d40f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:19.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.658 7fe5caffd700 1 -- 172.21.2.101:0/2046754686 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d40e52e0 con 0x7fe5d41b0870 2021-02-10T11:30:19.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.658 7fe5d2ffd700 1 -- 172.21.2.101:0/2046754686 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3375291776 0 0) 0x7fe5cc001480 con 0x7fe5d40efe00 2021-02-10T11:30:19.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.658 7fe5caffd700 1 --2- 172.21.2.101:0/2046754686 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5d41b0870 0x7fe5d40f20e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7fe5bc006700 tx=0x7fe5bc009b00).ready entity=mon.0 client_cookie=f3fbf5a075b818 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:19.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.658 7fe5d2ffd700 1 -- 172.21.2.101:0/2046754686 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5bc00c0c0 con 0x7fe5d41b0870 2021-02-10T11:30:19.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.658 7fe5d2ffd700 1 -- 172.21.2.101:0/2046754686 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc00c400 con 0x7fe5d41b0870 2021-02-10T11:30:19.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.658 7fe5d2ffd700 1 -- 172.21.2.101:0/2046754686 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5bc00c860 con 0x7fe5d41b0870 2021-02-10T11:30:19.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.658 7fe5d9f38700 1 -- 172.21.2.101:0/2046754686 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5d41b0870 msgr2=0x7fe5d40f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:19.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.658 7fe5d9f38700 1 --2- 172.21.2.101:0/2046754686 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5d41b0870 0x7fe5d40f20e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7fe5bc006700 tx=0x7fe5bc009b00).stop 2021-02-10T11:30:19.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.658 7fe5d9f38700 1 -- 172.21.2.101:0/2046754686 shutdown_connections 2021-02-10T11:30:19.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.658 7fe5d9f38700 1 --2- 172.21.2.101:0/2046754686 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5d41b0870 0x7fe5d40f20e0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:19.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.658 7fe5d9f38700 1 -- 172.21.2.101:0/2046754686 >> 172.21.2.101:0/2046754686 conn(0x7fe5d40ce3e0 msgr2=0x7fe5d40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:19.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.659 7fe5d9f38700 1 -- 172.21.2.101:0/2046754686 shutdown_connections 2021-02-10T11:30:19.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.659 7fe5d9f38700 1 -- 172.21.2.101:0/2046754686 wait complete. 2021-02-10T11:30:19.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.673 7fe5d9f38700 1 Processor -- start 2021-02-10T11:30:19.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.673 7fe5d9f38700 1 -- start start 2021-02-10T11:30:19.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.674 7fe5d9f38700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5d40bdbf0 0x7fe5d41a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:19.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.674 7fe5d9f38700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5d40bdff0 0x7fe5d41a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:19.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.674 7fe5d9f38700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5d40abcc0 0x7fe5d40ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:19.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.674 7fe5d9f38700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d40be590 con 0x7fe5d40bdff0 2021-02-10T11:30:19.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.674 7fe5d9f38700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d41a8090 con 0x7fe5d40bdbf0 2021-02-10T11:30:19.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.674 7fe5d9f38700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d41a82b0 con 0x7fe5d40abcc0 2021-02-10T11:30:19.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.674 7fe5caffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5d40bdff0 0x7fe5d41a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:19.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.674 7fe5caffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5d40bdff0 0x7fe5d41a7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48856/0 (socket says 172.21.2.101:48856) 2021-02-10T11:30:19.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.674 7fe5caffd700 1 -- 172.21.2.101:0/1508409916 learned_addr learned my addr 172.21.2.101:0/1508409916 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:19.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.674 7fe5d3fff700 1 --2- 172.21.2.101:0/1508409916 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5d40abcc0 0x7fe5d40ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:19.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.674 7fe5d37fe700 1 --2- 172.21.2.101:0/1508409916 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5d40bdbf0 0x7fe5d41a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:19.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.675 7fe5caffd700 1 -- 172.21.2.101:0/1508409916 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5d40abcc0 msgr2=0x7fe5d40ac0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:19.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.675 7fe5caffd700 1 --2- 172.21.2.101:0/1508409916 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5d40abcc0 0x7fe5d40ac0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:19.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.675 7fe5caffd700 1 -- 172.21.2.101:0/1508409916 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5d40bdbf0 msgr2=0x7fe5d41a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:19.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.675 7fe5caffd700 1 --2- 172.21.2.101:0/1508409916 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5d40bdbf0 0x7fe5d41a7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:19.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.675 7fe5caffd700 1 -- 172.21.2.101:0/1508409916 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d40ac750 con 0x7fe5d40bdff0 2021-02-10T11:30:19.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.675 7fe5caffd700 1 --2- 172.21.2.101:0/1508409916 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5d40bdff0 0x7fe5d41a7a70 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7fe5bc00b660 tx=0x7fe5bc00ba00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:19.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.675 7fe5d0ff9700 1 -- 172.21.2.101:0/1508409916 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5bc009440 con 0x7fe5d40bdff0 2021-02-10T11:30:19.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.675 7fe5d0ff9700 1 -- 172.21.2.101:0/1508409916 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc000c90 con 0x7fe5d40bdff0 2021-02-10T11:30:19.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.675 7fe5d0ff9700 1 -- 172.21.2.101:0/1508409916 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5bc002030 con 0x7fe5d40bdff0 2021-02-10T11:30:19.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.675 7fe5d9f38700 1 -- 172.21.2.101:0/1508409916 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d40ac750 con 0x7fe5d40bdff0 2021-02-10T11:30:19.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.675 7fe5d9f38700 1 -- 172.21.2.101:0/1508409916 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d40ac750 con 0x7fe5d40bdff0 2021-02-10T11:30:19.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.676 7fe5d0ff9700 1 -- 172.21.2.101:0/1508409916 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe5bc0093e0 con 0x7fe5d40bdff0 2021-02-10T11:30:19.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.676 7fe5d0ff9700 1 --2- 172.21.2.101:0/1508409916 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5ac0186c0 0x7fe5ac01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:19.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.676 7fe5d0ff9700 1 -- 172.21.2.101:0/1508409916 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe5bc032800 con 0x7fe5d40bdff0 2021-02-10T11:30:19.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.676 7fe5d37fe700 1 --2- 172.21.2.101:0/1508409916 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5ac0186c0 0x7fe5ac01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:19.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.676 7fe5d37fe700 1 --2- 172.21.2.101:0/1508409916 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5ac0186c0 0x7fe5ac01abc0 crc :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:19.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.705 7fe5d9f38700 1 -- 172.21.2.101:0/1508409916 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d41a8090 con 0x7fe5d40bdff0 2021-02-10T11:30:19.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.708 7fe5d0ff9700 1 -- 172.21.2.101:0/1508409916 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe5bc0090c0 con 0x7fe5d40bdff0 2021-02-10T11:30:19.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.836 7fe5d9f38700 1 -- 172.21.2.101:0/1508409916 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839270", "target": ["mgr", ""]}) v1 -- 0x7fe5d41a8090 con 0x7fe5ac0186c0 2021-02-10T11:30:19.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.847 7fe5d0ff9700 1 -- 172.21.2.101:0/1508409916 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fe5c4006550 con 0x7fe5ac0186c0 2021-02-10T11:30:19.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.853 7fe5d9f38700 1 -- 172.21.2.101:0/1508409916 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5ac0186c0 msgr2=0x7fe5ac01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:19.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.853 7fe5d9f38700 1 --2- 172.21.2.101:0/1508409916 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5ac0186c0 0x7fe5ac01abc0 crc :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:19.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.853 7fe5d9f38700 1 -- 172.21.2.101:0/1508409916 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5d40bdff0 msgr2=0x7fe5d41a7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:19.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.853 7fe5d9f38700 1 --2- 172.21.2.101:0/1508409916 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5d40bdff0 0x7fe5d41a7a70 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7fe5bc00b660 tx=0x7fe5bc00ba00).stop 2021-02-10T11:30:19.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.853 7fe5d9f38700 1 -- 172.21.2.101:0/1508409916 shutdown_connections 2021-02-10T11:30:19.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.853 7fe5d9f38700 1 --2- 172.21.2.101:0/1508409916 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5ac0186c0 0x7fe5ac01abc0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:19.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.853 7fe5d9f38700 1 --2- 172.21.2.101:0/1508409916 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5d40abcc0 0x7fe5d40ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:19.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.853 7fe5d9f38700 1 --2- 172.21.2.101:0/1508409916 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5d40bdff0 0x7fe5d41a7a70 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:19.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.853 7fe5d9f38700 1 --2- 172.21.2.101:0/1508409916 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5d40bdbf0 0x7fe5d41a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:19.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.853 7fe5d9f38700 1 -- 172.21.2.101:0/1508409916 >> 172.21.2.101:0/1508409916 conn(0x7fe5d4095a70 msgr2=0x7fe5d40bf040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:19.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.853 7fe5d9f38700 1 -- 172.21.2.101:0/1508409916 shutdown_connections 2021-02-10T11:30:19.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:19.853 7fe5d9f38700 1 -- 172.21.2.101:0/1508409916 wait complete. 2021-02-10T11:30:19.872 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839271 2021-02-10T11:30:20.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.279 7fa76ad42700 1 Processor -- start 2021-02-10T11:30:20.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.279 7fa76ad42700 1 -- start start 2021-02-10T11:30:20.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.280 7fa76ad42700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa7641b0470 0x7fa7641b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:20.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.280 7fa76ad42700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa7640f20e0 0x7fa7640f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:20.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.280 7fa76ad42700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7640f0b50 con 0x7fa7640ef610 2021-02-10T11:30:20.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.280 7fa76ad42700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7640f0820 con 0x7fa7640f20e0 2021-02-10T11:30:20.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.280 7fa76ad42700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7640f24e0 con 0x7fa7641b0470 2021-02-10T11:30:20.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.280 7fa76935f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa7640f20e0 0x7fa7640f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:20.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.280 7fa76935f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa7640f20e0 0x7fa7640f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48862/0 (socket says 172.21.2.101:48862) 2021-02-10T11:30:20.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.280 7fa76935f700 1 -- 172.21.2.101:0/1452561630 learned_addr learned my addr 172.21.2.101:0/1452561630 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:20.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.280 7fa768b5e700 1 --2- 172.21.2.101:0/1452561630 >> v2:172.21.2.108:3300/0 conn(0x7fa7641b0470 0x7fa7641b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:20.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.280 7fa76935f700 1 -- 172.21.2.101:0/1452561630 >> v2:172.21.2.108:3300/0 conn(0x7fa7641b0470 msgr2=0x7fa7641b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:20.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.280 7fa76935f700 1 --2- 172.21.2.101:0/1452561630 >> v2:172.21.2.108:3300/0 conn(0x7fa7641b0470 0x7fa7641b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:20.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.280 7fa76935f700 1 -- 172.21.2.101:0/1452561630 >> v1:172.21.2.101:6789/0 conn(0x7fa7640ef610 legacy=0x7fa7640efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:20.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.280 7fa76935f700 1 -- 172.21.2.101:0/1452561630 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7640e52e0 con 0x7fa7640f20e0 2021-02-10T11:30:20.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.280 7fa76935f700 1 --2- 172.21.2.101:0/1452561630 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7640f20e0 0x7fa7640f0200 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7fa75c006f00 tx=0x7fa75c009f10).ready entity=mon.0 client_cookie=b4085f08530f97d1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:20.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.280 7fa7637fe700 1 -- 172.21.2.101:0/1452561630 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa75c00b3b0 con 0x7fa7640f20e0 2021-02-10T11:30:20.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.280 7fa7637fe700 1 -- 172.21.2.101:0/1452561630 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75c00b730 con 0x7fa7640f20e0 2021-02-10T11:30:20.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.280 7fa7637fe700 1 -- 172.21.2.101:0/1452561630 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa75c00bc10 con 0x7fa7640f20e0 2021-02-10T11:30:20.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.281 7fa76ad42700 1 -- 172.21.2.101:0/1452561630 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7640f20e0 msgr2=0x7fa7640f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:20.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.281 7fa76ad42700 1 --2- 172.21.2.101:0/1452561630 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7640f20e0 0x7fa7640f0200 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7fa75c006f00 tx=0x7fa75c009f10).stop 2021-02-10T11:30:20.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.281 7fa76ad42700 1 -- 172.21.2.101:0/1452561630 shutdown_connections 2021-02-10T11:30:20.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.281 7fa76ad42700 1 --2- 172.21.2.101:0/1452561630 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7640f20e0 0x7fa7640f0200 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:20.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.281 7fa76ad42700 1 --2- 172.21.2.101:0/1452561630 >> v2:172.21.2.108:3300/0 conn(0x7fa7641b0470 0x7fa7641b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:20.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.281 7fa76ad42700 1 -- 172.21.2.101:0/1452561630 >> 172.21.2.101:0/1452561630 conn(0x7fa7640ce3e0 msgr2=0x7fa7640e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:20.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.281 7fa76ad42700 1 -- 172.21.2.101:0/1452561630 shutdown_connections 2021-02-10T11:30:20.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.281 7fa76ad42700 1 -- 172.21.2.101:0/1452561630 wait complete. 2021-02-10T11:30:20.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.296 7fa76ad42700 1 Processor -- start 2021-02-10T11:30:20.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.296 7fa76ad42700 1 -- start start 2021-02-10T11:30:20.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.296 7fa76ad42700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7640a0420 0x7fa7640a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:20.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.296 7fa76ad42700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa764189d00 0x7fa76418a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:20.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.296 7fa76ad42700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa76418a720 0x7fa7640b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:20.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.296 7fa76ad42700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa764189780 con 0x7fa76418a720 2021-02-10T11:30:20.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.296 7fa76ad42700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76418ab20 con 0x7fa7640a0420 2021-02-10T11:30:20.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.296 7fa76ad42700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7640a1140 con 0x7fa764189d00 2021-02-10T11:30:20.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.297 7fa76935f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa76418a720 0x7fa7640b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:20.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.297 7fa763fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa764189d00 0x7fa76418a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:20.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.297 7fa76935f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa76418a720 0x7fa7640b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48872/0 (socket says 172.21.2.101:48872) 2021-02-10T11:30:20.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.297 7fa76935f700 1 -- 172.21.2.101:0/1859330281 learned_addr learned my addr 172.21.2.101:0/1859330281 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:20.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.297 7fa768b5e700 1 --2- 172.21.2.101:0/1859330281 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7640a0420 0x7fa7640a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:20.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.297 7fa76935f700 1 -- 172.21.2.101:0/1859330281 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa764189d00 msgr2=0x7fa76418a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:20.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.297 7fa76935f700 1 --2- 172.21.2.101:0/1859330281 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa764189d00 0x7fa76418a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:20.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.297 7fa76935f700 1 -- 172.21.2.101:0/1859330281 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7640a0420 msgr2=0x7fa7640a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:20.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.297 7fa76935f700 1 --2- 172.21.2.101:0/1859330281 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7640a0420 0x7fa7640a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:20.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.297 7fa76935f700 1 -- 172.21.2.101:0/1859330281 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7640b8200 con 0x7fa76418a720 2021-02-10T11:30:20.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.297 7fa76935f700 1 --2- 172.21.2.101:0/1859330281 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa76418a720 0x7fa7640b7be0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7fa75c009390 tx=0x7fa75c00bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:20.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.297 7fa7617fa700 1 -- 172.21.2.101:0/1859330281 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa75c009450 con 0x7fa76418a720 2021-02-10T11:30:20.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.297 7fa7617fa700 1 -- 172.21.2.101:0/1859330281 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75c0068e0 con 0x7fa76418a720 2021-02-10T11:30:20.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.297 7fa7617fa700 1 -- 172.21.2.101:0/1859330281 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa75c00c4e0 con 0x7fa76418a720 2021-02-10T11:30:20.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.297 7fa76ad42700 1 -- 172.21.2.101:0/1859330281 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7640b8200 con 0x7fa76418a720 2021-02-10T11:30:20.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.297 7fa76ad42700 1 -- 172.21.2.101:0/1859330281 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7640b8790 con 0x7fa76418a720 2021-02-10T11:30:20.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.298 7fa7617fa700 1 -- 172.21.2.101:0/1859330281 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa75c001270 con 0x7fa76418a720 2021-02-10T11:30:20.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.298 7fa7617fa700 1 --2- 172.21.2.101:0/1859330281 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7480186c0 0x7fa74801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:20.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.298 7fa7617fa700 1 -- 172.21.2.101:0/1859330281 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa75c032b40 con 0x7fa76418a720 2021-02-10T11:30:20.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.298 7fa768b5e700 1 --2- 172.21.2.101:0/1859330281 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7480186c0 0x7fa74801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:20.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.298 7fa768b5e700 1 --2- 172.21.2.101:0/1859330281 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7480186c0 0x7fa74801abc0 crc :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:20.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.328 7fa76ad42700 1 -- 172.21.2.101:0/1859330281 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa764105ae0 con 0x7fa76418a720 2021-02-10T11:30:20.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.331 7fa7617fa700 1 -- 172.21.2.101:0/1859330281 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa75c000cc0 con 0x7fa76418a720 2021-02-10T11:30:20.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.458 7fa76ad42700 1 -- 172.21.2.101:0/1859330281 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839271", "target": ["mgr", ""]}) v1 -- 0x7fa7640b9020 con 0x7fa7480186c0 2021-02-10T11:30:20.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.468 7fa7617fa700 1 -- 172.21.2.101:0/1859330281 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fa74c0009b0 con 0x7fa7480186c0 2021-02-10T11:30:20.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.474 7fa76ad42700 1 -- 172.21.2.101:0/1859330281 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7480186c0 msgr2=0x7fa74801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.474 7fa76ad42700 1 --2- 172.21.2.101:0/1859330281 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7480186c0 0x7fa74801abc0 crc :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.474 7fa76ad42700 1 -- 172.21.2.101:0/1859330281 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa76418a720 msgr2=0x7fa7640b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.474 7fa76ad42700 1 --2- 172.21.2.101:0/1859330281 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa76418a720 0x7fa7640b7be0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7fa75c009390 tx=0x7fa75c00bd40).stop 2021-02-10T11:30:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.475 7fa76ad42700 1 -- 172.21.2.101:0/1859330281 shutdown_connections 2021-02-10T11:30:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.475 7fa76ad42700 1 --2- 172.21.2.101:0/1859330281 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7480186c0 0x7fa74801abc0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.475 7fa76ad42700 1 --2- 172.21.2.101:0/1859330281 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa764189d00 0x7fa76418a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.475 7fa76ad42700 1 --2- 172.21.2.101:0/1859330281 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa76418a720 0x7fa7640b7be0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.475 7fa76ad42700 1 --2- 172.21.2.101:0/1859330281 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7640a0420 0x7fa7640a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.475 7fa76ad42700 1 -- 172.21.2.101:0/1859330281 >> 172.21.2.101:0/1859330281 conn(0x7fa764094e10 msgr2=0x7fa7640a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.475 7fa76ad42700 1 -- 172.21.2.101:0/1859330281 shutdown_connections 2021-02-10T11:30:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.475 7fa76ad42700 1 -- 172.21.2.101:0/1859330281 wait complete. 2021-02-10T11:30:20.493 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839272 2021-02-10T11:30:20.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.897 7faae9b60700 1 Processor -- start 2021-02-10T11:30:20.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.897 7faae9b60700 1 -- start start 2021-02-10T11:30:20.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.897 7faae9b60700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7faae41b0470 0x7faae41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:20.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.897 7faae9b60700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faae40ef610 0x7faae40efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:20.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.897 7faae9b60700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faae40f0b50 con 0x7faae40f0030 2021-02-10T11:30:20.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.897 7faae9b60700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae40e52e0 con 0x7faae41b0470 2021-02-10T11:30:20.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.897 7faae9b60700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae40e54d0 con 0x7faae40ef610 2021-02-10T11:30:20.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.897 7faae3fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7faae40f0030 0x7faae40f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49370/0 (socket says 172.21.2.101:49370) 2021-02-10T11:30:20.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.898 7faae3fff700 1 -- 172.21.2.101:0/1685901609 learned_addr learned my addr 172.21.2.101:0/1685901609 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:20.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.898 7faae37fe700 1 --2- 172.21.2.101:0/1685901609 >> v2:172.21.2.101:3300/0 conn(0x7faae41b0470 0x7faae41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:20.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.898 7faae2ffd700 1 --2- 172.21.2.101:0/1685901609 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faae40ef610 0x7faae40efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:20.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.898 7faae37fe700 1 -- 172.21.2.101:0/1685901609 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faae40ef610 msgr2=0x7faae40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:20.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.898 7faae37fe700 1 --2- 172.21.2.101:0/1685901609 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faae40ef610 0x7faae40efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:20.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.898 7faae37fe700 1 -- 172.21.2.101:0/1685901609 >> v1:172.21.2.101:6789/0 conn(0x7faae40f0030 legacy=0x7faae40f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:20.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.898 7faae27fc700 1 -- 172.21.2.101:0/1685901609 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243196824 0 0) 0x7faadc000f90 con 0x7faae40f0030 2021-02-10T11:30:20.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.898 7faae37fe700 1 -- 172.21.2.101:0/1685901609 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae40f0430 con 0x7faae41b0470 2021-02-10T11:30:20.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.898 7faae37fe700 1 --2- 172.21.2.101:0/1685901609 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faae41b0470 0x7faae41b0870 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7faad0006650 tx=0x7faad0009a00).ready entity=mon.0 client_cookie=c05e5882e293b308 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:20.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.898 7faae27fc700 1 -- 172.21.2.101:0/1685901609 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faad00009b0 con 0x7faae41b0470 2021-02-10T11:30:20.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.899 7faae27fc700 1 -- 172.21.2.101:0/1685901609 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad000c0c0 con 0x7faae41b0470 2021-02-10T11:30:20.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.899 7faae27fc700 1 -- 172.21.2.101:0/1685901609 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faad000c5a0 con 0x7faae41b0470 2021-02-10T11:30:20.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.899 7faae9b60700 1 -- 172.21.2.101:0/1685901609 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faae41b0470 msgr2=0x7faae41b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:20.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.899 7faae9b60700 1 --2- 172.21.2.101:0/1685901609 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faae41b0470 0x7faae41b0870 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7faad0006650 tx=0x7faad0009a00).stop 2021-02-10T11:30:20.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.899 7faae9b60700 1 -- 172.21.2.101:0/1685901609 shutdown_connections 2021-02-10T11:30:20.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.899 7faae9b60700 1 --2- 172.21.2.101:0/1685901609 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faae40ef610 0x7faae40efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:20.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.899 7faae9b60700 1 --2- 172.21.2.101:0/1685901609 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faae41b0470 0x7faae41b0870 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:20.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.899 7faae9b60700 1 -- 172.21.2.101:0/1685901609 >> 172.21.2.101:0/1685901609 conn(0x7faae40ce3e0 msgr2=0x7faae40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:20.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.899 7faae9b60700 1 -- 172.21.2.101:0/1685901609 shutdown_connections 2021-02-10T11:30:20.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.899 7faae9b60700 1 -- 172.21.2.101:0/1685901609 wait complete. 2021-02-10T11:30:20.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.914 7faae9b60700 1 Processor -- start 2021-02-10T11:30:20.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.914 7faae9b60700 1 -- start start 2021-02-10T11:30:20.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.915 7faae9b60700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faae40c28d0 0x7faae40c2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:20.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.915 7faae9b60700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faae408e690 0x7faae408ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:20.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.915 7faae9b60700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faae40bc380 0x7faae40bc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:20.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.915 7faae9b60700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae408f340 con 0x7faae408e690 2021-02-10T11:30:20.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.915 7faae9b60700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae408f0b0 con 0x7faae40bc380 2021-02-10T11:30:20.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.915 7faae9b60700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae40c3400 con 0x7faae40c28d0 2021-02-10T11:30:20.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.915 7faae2ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faae408e690 0x7faae408ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:20.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.915 7faae2ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faae408e690 0x7faae408ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48882/0 (socket says 172.21.2.101:48882) 2021-02-10T11:30:20.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.915 7faae2ffd700 1 -- 172.21.2.101:0/2412676614 learned_addr learned my addr 172.21.2.101:0/2412676614 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:20.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.915 7faae2ffd700 1 -- 172.21.2.101:0/2412676614 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faae40c28d0 msgr2=0x7faae40c2cd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:30:20.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.915 7faae2ffd700 1 --2- 172.21.2.101:0/2412676614 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faae40c28d0 0x7faae40c2cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:20.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.915 7faae2ffd700 1 -- 172.21.2.101:0/2412676614 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faae40bc380 msgr2=0x7faae40bc780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:30:20.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.915 7faae2ffd700 1 --2- 172.21.2.101:0/2412676614 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faae40bc380 0x7faae40bc780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:20.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.915 7faae2ffd700 1 -- 172.21.2.101:0/2412676614 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae40bcda0 con 0x7faae408e690 2021-02-10T11:30:20.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.915 7faae2ffd700 1 --2- 172.21.2.101:0/2412676614 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faae408e690 0x7faae408ea90 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7faad4006f60 tx=0x7faad400b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:20.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.915 7faadbfff700 1 -- 172.21.2.101:0/2412676614 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faad400bd60 con 0x7faae408e690 2021-02-10T11:30:20.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.915 7faadbfff700 1 -- 172.21.2.101:0/2412676614 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad4000c90 con 0x7faae408e690 2021-02-10T11:30:20.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.915 7faae9b60700 1 -- 172.21.2.101:0/2412676614 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae40bcda0 con 0x7faae408e690 2021-02-10T11:30:20.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.915 7faadbfff700 1 -- 172.21.2.101:0/2412676614 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faad4002220 con 0x7faae408e690 2021-02-10T11:30:20.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.915 7faae9b60700 1 -- 172.21.2.101:0/2412676614 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae40bcda0 con 0x7faae408e690 2021-02-10T11:30:20.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.916 7faadbfff700 1 -- 172.21.2.101:0/2412676614 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7faad40052e0 con 0x7faae408e690 2021-02-10T11:30:20.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.916 7faadbfff700 1 --2- 172.21.2.101:0/2412676614 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faac0018730 0x7faac001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:20.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.916 7faadbfff700 1 -- 172.21.2.101:0/2412676614 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7faad4032640 con 0x7faae408e690 2021-02-10T11:30:20.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.916 7faae37fe700 1 --2- 172.21.2.101:0/2412676614 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faac0018730 0x7faac001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:20.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.916 7faae37fe700 1 --2- 172.21.2.101:0/2412676614 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faac0018730 0x7faac001ac10 crc :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:20.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.947 7faae9b60700 1 -- 172.21.2.101:0/2412676614 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae4105ae0 con 0x7faae408e690 2021-02-10T11:30:20.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:20.949 7faadbfff700 1 -- 172.21.2.101:0/2412676614 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7faad4000c90 con 0x7faae408e690 2021-02-10T11:30:21.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.077 7faae9b60700 1 -- 172.21.2.101:0/2412676614 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839272", "target": ["mgr", ""]}) v1 -- 0x7faae408f0b0 con 0x7faac0018730 2021-02-10T11:30:21.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.088 7faadbfff700 1 -- 172.21.2.101:0/2412676614 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7faad00009b0 con 0x7faac0018730 2021-02-10T11:30:21.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.094 7faae9b60700 1 -- 172.21.2.101:0/2412676614 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faac0018730 msgr2=0x7faac001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:21.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.094 7faae9b60700 1 --2- 172.21.2.101:0/2412676614 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faac0018730 0x7faac001ac10 crc :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:21.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.094 7faae9b60700 1 -- 172.21.2.101:0/2412676614 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faae408e690 msgr2=0x7faae408ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:21.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.094 7faae9b60700 1 --2- 172.21.2.101:0/2412676614 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faae408e690 0x7faae408ea90 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7faad4006f60 tx=0x7faad400b970).stop 2021-02-10T11:30:21.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.094 7faae9b60700 1 -- 172.21.2.101:0/2412676614 shutdown_connections 2021-02-10T11:30:21.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.094 7faae9b60700 1 --2- 172.21.2.101:0/2412676614 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faae40bc380 0x7faae40bc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:21.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.094 7faae9b60700 1 --2- 172.21.2.101:0/2412676614 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faae408e690 0x7faae408ea90 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:21.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.094 7faae9b60700 1 --2- 172.21.2.101:0/2412676614 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faac0018730 0x7faac001ac10 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:21.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.094 7faae9b60700 1 --2- 172.21.2.101:0/2412676614 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faae40c28d0 0x7faae40c2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:21.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.094 7faae9b60700 1 -- 172.21.2.101:0/2412676614 >> 172.21.2.101:0/2412676614 conn(0x7faae40bbf60 msgr2=0x7faae4105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:21.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.094 7faae9b60700 1 -- 172.21.2.101:0/2412676614 shutdown_connections 2021-02-10T11:30:21.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.094 7faae9b60700 1 -- 172.21.2.101:0/2412676614 wait complete. 2021-02-10T11:30:21.113 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839273 2021-02-10T11:30:21.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.518 7f0a873bf700 1 Processor -- start 2021-02-10T11:30:21.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.518 7f0a873bf700 1 -- start start 2021-02-10T11:30:21.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.519 7f0a873bf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0a801b0870 0x7f0a800f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:21.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.519 7f0a873bf700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a801b0c70 con 0x7f0a800efe00 2021-02-10T11:30:21.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.519 7f0a873bf700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a800f0b50 con 0x7f0a801b0470 2021-02-10T11:30:21.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.519 7f0a873bf700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a800e5ea0 con 0x7f0a801b0870 2021-02-10T11:30:21.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.519 7f0a851db700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f0a801b0470 0x7f0a800ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:42794/0 (socket says 172.21.2.101:42794) 2021-02-10T11:30:21.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.519 7f0a849da700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0a801b0870 0x7f0a800f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:21.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.519 7f0a851db700 1 -- 172.21.2.101:0/3818788913 learned_addr learned my addr 172.21.2.101:0/3818788913 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:21.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.519 7f0a73fff700 1 -- 172.21.2.101:0/3818788913 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 185554747 0 0) 0x7f0a7c0009b0 con 0x7f0a801b0470 2021-02-10T11:30:21.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.519 7f0a73fff700 1 -- 172.21.2.101:0/3818788913 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a6c002fe0 con 0x7f0a801b0470 2021-02-10T11:30:21.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.519 7f0a849da700 1 -- 172.21.2.101:0/3818788913 >> v1:172.21.2.108:6789/0 conn(0x7f0a801b0470 legacy=0x7f0a800ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:21.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.519 7f0a73fff700 1 -- 172.21.2.101:0/3818788913 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1365537634 0 0) 0x7f0a78000f90 con 0x7f0a800efe00 2021-02-10T11:30:21.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.519 7f0a849da700 1 -- 172.21.2.101:0/3818788913 >> v1:172.21.2.101:6789/0 conn(0x7f0a800efe00 legacy=0x7f0a800f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:21.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.519 7f0a849da700 1 -- 172.21.2.101:0/3818788913 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a800e52e0 con 0x7f0a801b0870 2021-02-10T11:30:21.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.520 7f0a849da700 1 --2- 172.21.2.101:0/3818788913 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a801b0870 0x7f0a800f20e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f0a74006650 tx=0x7f0a74009a00).ready entity=mon.0 client_cookie=51dbe695b0908247 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:21.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.520 7f0a73fff700 1 -- 172.21.2.101:0/3818788913 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0a74009df0 con 0x7f0a801b0870 2021-02-10T11:30:21.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.520 7f0a73fff700 1 -- 172.21.2.101:0/3818788913 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a740009b0 con 0x7f0a801b0870 2021-02-10T11:30:21.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.520 7f0a73fff700 1 -- 172.21.2.101:0/3818788913 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0a7400c2b0 con 0x7f0a801b0870 2021-02-10T11:30:21.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.520 7f0a873bf700 1 -- 172.21.2.101:0/3818788913 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a801b0870 msgr2=0x7f0a800f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:21.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.520 7f0a873bf700 1 --2- 172.21.2.101:0/3818788913 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a801b0870 0x7f0a800f20e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f0a74006650 tx=0x7f0a74009a00).stop 2021-02-10T11:30:21.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.520 7f0a873bf700 1 -- 172.21.2.101:0/3818788913 shutdown_connections 2021-02-10T11:30:21.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.520 7f0a873bf700 1 --2- 172.21.2.101:0/3818788913 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a801b0870 0x7f0a800f20e0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:21.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.520 7f0a873bf700 1 -- 172.21.2.101:0/3818788913 >> 172.21.2.101:0/3818788913 conn(0x7f0a800ce3e0 msgr2=0x7f0a800e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:21.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.521 7f0a873bf700 1 -- 172.21.2.101:0/3818788913 shutdown_connections 2021-02-10T11:30:21.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.521 7f0a873bf700 1 -- 172.21.2.101:0/3818788913 wait complete. 2021-02-10T11:30:21.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.535 7f0a873bf700 1 Processor -- start 2021-02-10T11:30:21.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.535 7f0a873bf700 1 -- start start 2021-02-10T11:30:21.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.535 7f0a873bf700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0a800b20b0 0x7f0a800ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:21.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.535 7f0a873bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a800b24b0 0x7f0a80191dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:21.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.535 7f0a873bf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a800acf90 0x7f0a801923e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:21.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.535 7f0a873bf700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a800b1b30 con 0x7f0a800b24b0 2021-02-10T11:30:21.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.535 7f0a873bf700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a801a04e0 con 0x7f0a800b20b0 2021-02-10T11:30:21.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.535 7f0a873bf700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a801a06c0 con 0x7f0a800acf90 2021-02-10T11:30:21.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.536 7f0a849da700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a800b24b0 0x7f0a80191dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:21.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.536 7f0a849da700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a800b24b0 0x7f0a80191dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48892/0 (socket says 172.21.2.101:48892) 2021-02-10T11:30:21.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.536 7f0a849da700 1 -- 172.21.2.101:0/2527787399 learned_addr learned my addr 172.21.2.101:0/2527787399 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:21.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.536 7f0a851db700 1 --2- 172.21.2.101:0/2527787399 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0a800b20b0 0x7f0a800ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:21.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.536 7f0a859dc700 1 --2- 172.21.2.101:0/2527787399 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a800acf90 0x7f0a801923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:21.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.536 7f0a849da700 1 -- 172.21.2.101:0/2527787399 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a800acf90 msgr2=0x7f0a801923e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:21.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.536 7f0a849da700 1 --2- 172.21.2.101:0/2527787399 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a800acf90 0x7f0a801923e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:21.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.536 7f0a849da700 1 -- 172.21.2.101:0/2527787399 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0a800b20b0 msgr2=0x7f0a800ac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:21.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.536 7f0a849da700 1 --2- 172.21.2.101:0/2527787399 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0a800b20b0 0x7f0a800ac970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:21.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.536 7f0a849da700 1 -- 172.21.2.101:0/2527787399 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a80192be0 con 0x7f0a800b24b0 2021-02-10T11:30:21.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.536 7f0a849da700 1 --2- 172.21.2.101:0/2527787399 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a800b24b0 0x7f0a80191dc0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f0a740096a0 tx=0x7f0a7400b9b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:21.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.536 7f0a71ffb700 1 -- 172.21.2.101:0/2527787399 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0a740093a0 con 0x7f0a800b24b0 2021-02-10T11:30:21.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.536 7f0a71ffb700 1 -- 172.21.2.101:0/2527787399 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a74002030 con 0x7f0a800b24b0 2021-02-10T11:30:21.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.536 7f0a873bf700 1 -- 172.21.2.101:0/2527787399 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a80192be0 con 0x7f0a800b24b0 2021-02-10T11:30:21.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.536 7f0a873bf700 1 -- 172.21.2.101:0/2527787399 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a80192be0 con 0x7f0a800b24b0 2021-02-10T11:30:21.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.536 7f0a71ffb700 1 -- 172.21.2.101:0/2527787399 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0a74002550 con 0x7f0a800b24b0 2021-02-10T11:30:21.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.537 7f0a71ffb700 1 -- 172.21.2.101:0/2527787399 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0a74000c90 con 0x7f0a800b24b0 2021-02-10T11:30:21.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.537 7f0a71ffb700 1 --2- 172.21.2.101:0/2527787399 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a60018730 0x7f0a6001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:21.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.537 7f0a71ffb700 1 -- 172.21.2.101:0/2527787399 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0a740327f0 con 0x7f0a800b24b0 2021-02-10T11:30:21.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.537 7f0a851db700 1 --2- 172.21.2.101:0/2527787399 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a60018730 0x7f0a6001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:21.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.538 7f0a851db700 1 --2- 172.21.2.101:0/2527787399 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a60018730 0x7f0a6001ac10 crc :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:21.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.568 7f0a873bf700 1 -- 172.21.2.101:0/2527787399 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a80105af0 con 0x7f0a800b24b0 2021-02-10T11:30:21.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.571 7f0a71ffb700 1 -- 172.21.2.101:0/2527787399 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0a7401f530 con 0x7f0a800b24b0 2021-02-10T11:30:21.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.699 7f0a873bf700 1 -- 172.21.2.101:0/2527787399 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839273", "target": ["mgr", ""]}) v1 -- 0x7f0a801a04e0 con 0x7f0a60018730 2021-02-10T11:30:21.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.709 7f0a71ffb700 1 -- 172.21.2.101:0/2527787399 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f0a7c000ec0 con 0x7f0a60018730 2021-02-10T11:30:21.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.716 7f0a873bf700 1 -- 172.21.2.101:0/2527787399 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a60018730 msgr2=0x7f0a6001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:21.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.716 7f0a873bf700 1 --2- 172.21.2.101:0/2527787399 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a60018730 0x7f0a6001ac10 crc :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:21.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.716 7f0a873bf700 1 -- 172.21.2.101:0/2527787399 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a800b24b0 msgr2=0x7f0a80191dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:21.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.716 7f0a873bf700 1 --2- 172.21.2.101:0/2527787399 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a800b24b0 0x7f0a80191dc0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f0a740096a0 tx=0x7f0a7400b9b0).stop 2021-02-10T11:30:21.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.716 7f0a873bf700 1 -- 172.21.2.101:0/2527787399 shutdown_connections 2021-02-10T11:30:21.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.716 7f0a873bf700 1 --2- 172.21.2.101:0/2527787399 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a60018730 0x7f0a6001ac10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:21.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.716 7f0a873bf700 1 --2- 172.21.2.101:0/2527787399 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a800acf90 0x7f0a801923e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:21.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.716 7f0a873bf700 1 --2- 172.21.2.101:0/2527787399 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a800b24b0 0x7f0a80191dc0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:21.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.716 7f0a873bf700 1 --2- 172.21.2.101:0/2527787399 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0a800b20b0 0x7f0a800ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:21.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.716 7f0a873bf700 1 -- 172.21.2.101:0/2527787399 >> 172.21.2.101:0/2527787399 conn(0x7f0a800bf4e0 msgr2=0x7f0a800be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:21.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.716 7f0a873bf700 1 -- 172.21.2.101:0/2527787399 shutdown_connections 2021-02-10T11:30:21.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:21.716 7f0a873bf700 1 -- 172.21.2.101:0/2527787399 wait complete. 2021-02-10T11:30:21.735 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839274 2021-02-10T11:30:22.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.139 7f1b7f9b7700 1 Processor -- start 2021-02-10T11:30:22.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.139 7f1b7f9b7700 1 -- start start 2021-02-10T11:30:22.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.140 7f1b7f9b7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b781b0870 0x7f1b780f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:22.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.140 7f1b7f9b7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1b780efe00 0x7f1b780f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:22.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.140 7f1b7f9b7700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b781b0c70 con 0x7f1b781b0470 2021-02-10T11:30:22.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.140 7f1b7f9b7700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b780f0b50 con 0x7f1b780efe00 2021-02-10T11:30:22.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.140 7f1b7f9b7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b780f0820 con 0x7f1b781b0870 2021-02-10T11:30:22.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.140 7f1b7dfd4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1b780efe00 0x7f1b780f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:22.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.140 7f1b7d7d3700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f1b781b0470 0x7f1b780ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:42804/0 (socket says 172.21.2.101:42804) 2021-02-10T11:30:22.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.140 7f1b7d7d3700 1 -- 172.21.2.101:0/3536448653 learned_addr learned my addr 172.21.2.101:0/3536448653 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:22.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.140 7f1b7cfd2700 1 --2- 172.21.2.101:0/3536448653 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b781b0870 0x7f1b780f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:22.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.140 7f1b7dfd4700 1 -- 172.21.2.101:0/3536448653 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b781b0870 msgr2=0x7f1b780f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:22.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.140 7f1b7dfd4700 1 --2- 172.21.2.101:0/3536448653 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b781b0870 0x7f1b780f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:22.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.140 7f1b7dfd4700 1 -- 172.21.2.101:0/3536448653 >> v1:172.21.2.108:6789/0 conn(0x7f1b781b0470 legacy=0x7f1b780ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:22.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.140 7f1b6ffff700 1 -- 172.21.2.101:0/3536448653 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3780711873 0 0) 0x7f1b740010b0 con 0x7f1b781b0470 2021-02-10T11:30:22.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.141 7f1b7dfd4700 1 -- 172.21.2.101:0/3536448653 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b780e52e0 con 0x7f1b780efe00 2021-02-10T11:30:22.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.141 7f1b7dfd4700 1 --2- 172.21.2.101:0/3536448653 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b780efe00 0x7f1b780f0200 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f1b70006e30 tx=0x7f1b70009de0).ready entity=mon.0 client_cookie=b73cd0b996660b44 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:22.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.141 7f1b6ffff700 1 -- 172.21.2.101:0/3536448653 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1b7000b300 con 0x7f1b780efe00 2021-02-10T11:30:22.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.141 7f1b6ffff700 1 -- 172.21.2.101:0/3536448653 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7000b580 con 0x7f1b780efe00 2021-02-10T11:30:22.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.141 7f1b6ffff700 1 -- 172.21.2.101:0/3536448653 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1b7000baa0 con 0x7f1b780efe00 2021-02-10T11:30:22.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.141 7f1b7f9b7700 1 -- 172.21.2.101:0/3536448653 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b780efe00 msgr2=0x7f1b780f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:22.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.141 7f1b7f9b7700 1 --2- 172.21.2.101:0/3536448653 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b780efe00 0x7f1b780f0200 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f1b70006e30 tx=0x7f1b70009de0).stop 2021-02-10T11:30:22.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.142 7f1b7f9b7700 1 -- 172.21.2.101:0/3536448653 shutdown_connections 2021-02-10T11:30:22.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.142 7f1b7f9b7700 1 --2- 172.21.2.101:0/3536448653 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b781b0870 0x7f1b780f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:22.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.142 7f1b7f9b7700 1 --2- 172.21.2.101:0/3536448653 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b780efe00 0x7f1b780f0200 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:22.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.142 7f1b7f9b7700 1 -- 172.21.2.101:0/3536448653 >> 172.21.2.101:0/3536448653 conn(0x7f1b780ce3e0 msgr2=0x7f1b780e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:22.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.142 7f1b7f9b7700 1 -- 172.21.2.101:0/3536448653 shutdown_connections 2021-02-10T11:30:22.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.142 7f1b7f9b7700 1 -- 172.21.2.101:0/3536448653 wait complete. 2021-02-10T11:30:22.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.156 7f1b7f9b7700 1 Processor -- start 2021-02-10T11:30:22.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.157 7f1b7f9b7700 1 -- start start 2021-02-10T11:30:22.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.157 7f1b7f9b7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1b7809f140 0x7f1b780c1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:22.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.157 7f1b7f9b7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b780c1f60 0x7f1b7809e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:22.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.157 7f1b7f9b7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b7809e920 0x7f1b780c2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.157 7f1b7f9b7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b780c23e0 con 0x7f1b780c1f60 2021-02-10T11:30:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.157 7f1b7f9b7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7809ed20 con 0x7f1b7809f140 2021-02-10T11:30:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.157 7f1b7f9b7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b78105d10 con 0x7f1b7809e920 2021-02-10T11:30:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.157 7f1b7cfd2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b780c1f60 0x7f1b7809e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.157 7f1b7cfd2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b780c1f60 0x7f1b7809e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48904/0 (socket says 172.21.2.101:48904) 2021-02-10T11:30:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.157 7f1b7cfd2700 1 -- 172.21.2.101:0/2786965090 learned_addr learned my addr 172.21.2.101:0/2786965090 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.157 7f1b7dfd4700 1 --2- 172.21.2.101:0/2786965090 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b7809e920 0x7f1b780c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.157 7f1b7d7d3700 1 --2- 172.21.2.101:0/2786965090 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1b7809f140 0x7f1b780c1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.158 7f1b7cfd2700 1 -- 172.21.2.101:0/2786965090 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b7809e920 msgr2=0x7f1b780c2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.158 7f1b7cfd2700 1 --2- 172.21.2.101:0/2786965090 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b7809e920 0x7f1b780c2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.158 7f1b7cfd2700 1 -- 172.21.2.101:0/2786965090 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1b7809f140 msgr2=0x7f1b780c1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.158 7f1b7cfd2700 1 --2- 172.21.2.101:0/2786965090 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1b7809f140 0x7f1b780c1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.158 7f1b7cfd2700 1 -- 172.21.2.101:0/2786965090 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b781a4910 con 0x7f1b780c1f60 2021-02-10T11:30:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.158 7f1b7d7d3700 1 --2- 172.21.2.101:0/2786965090 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1b7809f140 0x7f1b780c1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:30:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.158 7f1b7cfd2700 1 --2- 172.21.2.101:0/2786965090 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b780c1f60 0x7f1b7809e300 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f1b68006f60 tx=0x7f1b6800b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.158 7f1b6dffb700 1 -- 172.21.2.101:0/2786965090 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1b6800bd60 con 0x7f1b780c1f60 2021-02-10T11:30:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.158 7f1b6dffb700 1 -- 172.21.2.101:0/2786965090 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b68000c90 con 0x7f1b780c1f60 2021-02-10T11:30:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.158 7f1b7f9b7700 1 -- 172.21.2.101:0/2786965090 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b781a4910 con 0x7f1b780c1f60 2021-02-10T11:30:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.158 7f1b7f9b7700 1 -- 172.21.2.101:0/2786965090 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b780c3270 con 0x7f1b780c1f60 2021-02-10T11:30:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.158 7f1b6dffb700 1 -- 172.21.2.101:0/2786965090 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1b68002220 con 0x7f1b780c1f60 2021-02-10T11:30:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.159 7f1b6dffb700 1 -- 172.21.2.101:0/2786965090 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1b68006b00 con 0x7f1b780c1f60 2021-02-10T11:30:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.159 7f1b6dffb700 1 --2- 172.21.2.101:0/2786965090 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b58018730 0x7f1b5801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.159 7f1b6dffb700 1 -- 172.21.2.101:0/2786965090 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1b68032790 con 0x7f1b780c1f60 2021-02-10T11:30:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.159 7f1b7d7d3700 1 --2- 172.21.2.101:0/2786965090 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b58018730 0x7f1b5801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:22.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.159 7f1b7d7d3700 1 --2- 172.21.2.101:0/2786965090 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b58018730 0x7f1b5801ac10 crc :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:22.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.189 7f1b7f9b7700 1 -- 172.21.2.101:0/2786965090 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b780f0b50 con 0x7f1b780c1f60 2021-02-10T11:30:22.202 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.192 7f1b6dffb700 1 -- 172.21.2.101:0/2786965090 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1b68000940 con 0x7f1b780c1f60 2021-02-10T11:30:22.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.324 7f1b7f9b7700 1 -- 172.21.2.101:0/2786965090 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839274", "target": ["mgr", ""]}) v1 -- 0x7f1b780c3270 con 0x7f1b58018730 2021-02-10T11:30:22.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.334 7f1b6dffb700 1 -- 172.21.2.101:0/2786965090 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f1b740009b0 con 0x7f1b58018730 2021-02-10T11:30:22.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.341 7f1b7f9b7700 1 -- 172.21.2.101:0/2786965090 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b58018730 msgr2=0x7f1b5801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:22.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.341 7f1b7f9b7700 1 --2- 172.21.2.101:0/2786965090 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b58018730 0x7f1b5801ac10 crc :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:22.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.341 7f1b7f9b7700 1 -- 172.21.2.101:0/2786965090 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b780c1f60 msgr2=0x7f1b7809e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:22.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.341 7f1b7f9b7700 1 --2- 172.21.2.101:0/2786965090 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b780c1f60 0x7f1b7809e300 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f1b68006f60 tx=0x7f1b6800b970).stop 2021-02-10T11:30:22.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.341 7f1b7f9b7700 1 -- 172.21.2.101:0/2786965090 shutdown_connections 2021-02-10T11:30:22.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.341 7f1b7f9b7700 1 --2- 172.21.2.101:0/2786965090 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b58018730 0x7f1b5801ac10 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:22.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.341 7f1b7f9b7700 1 --2- 172.21.2.101:0/2786965090 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b7809e920 0x7f1b780c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:22.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.341 7f1b7f9b7700 1 --2- 172.21.2.101:0/2786965090 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b780c1f60 0x7f1b7809e300 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:22.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.341 7f1b7f9b7700 1 --2- 172.21.2.101:0/2786965090 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1b7809f140 0x7f1b780c1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:22.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.341 7f1b7f9b7700 1 -- 172.21.2.101:0/2786965090 >> 172.21.2.101:0/2786965090 conn(0x7f1b7809fc10 msgr2=0x7f1b780b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:22.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.341 7f1b7f9b7700 1 -- 172.21.2.101:0/2786965090 shutdown_connections 2021-02-10T11:30:22.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.341 7f1b7f9b7700 1 -- 172.21.2.101:0/2786965090 wait complete. 2021-02-10T11:30:22.359 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839275 2021-02-10T11:30:22.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.763 7fdae7a9e700 1 Processor -- start 2021-02-10T11:30:22.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.763 7fdae7a9e700 1 -- start start 2021-02-10T11:30:22.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.764 7fdae7a9e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdae01b0870 0x7fdae00f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:22.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.764 7fdae7a9e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae01b0c70 con 0x7fdae00efe00 2021-02-10T11:30:22.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.764 7fdae7a9e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae00f0b50 con 0x7fdae01b0470 2021-02-10T11:30:22.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.764 7fdae7a9e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdae00e5ea0 con 0x7fdae01b0870 2021-02-10T11:30:22.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.764 7fdae58ba700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fdae01b0470 0x7fdae00ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:42814/0 (socket says 172.21.2.101:42814) 2021-02-10T11:30:22.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.764 7fdae58ba700 1 -- 172.21.2.101:0/2726532616 learned_addr learned my addr 172.21.2.101:0/2726532616 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:22.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.764 7fdae50b9700 1 --2- 172.21.2.101:0/2726532616 >> v2:172.21.2.101:3300/0 conn(0x7fdae01b0870 0x7fdae00f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:22.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.764 7fdae60bb700 1 --1- 172.21.2.101:0/2726532616 >> v1:172.21.2.101:6789/0 conn(0x7fdae00efe00 0x7fdae00f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49408/0 (socket says 172.21.2.101:49408) 2021-02-10T11:30:22.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.764 7fdae50b9700 1 -- 172.21.2.101:0/2726532616 >> v1:172.21.2.108:6789/0 conn(0x7fdae01b0470 legacy=0x7fdae00ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:22.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.765 7fdae50b9700 1 -- 172.21.2.101:0/2726532616 >> v1:172.21.2.101:6789/0 conn(0x7fdae00efe00 legacy=0x7fdae00f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:22.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.764 7fdae48b8700 1 -- 172.21.2.101:0/2726532616 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1480418235 0 0) 0x7fdad8001480 con 0x7fdae00efe00 2021-02-10T11:30:22.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.765 7fdae50b9700 1 -- 172.21.2.101:0/2726532616 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae00e52e0 con 0x7fdae01b0870 2021-02-10T11:30:22.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.765 7fdae50b9700 1 --2- 172.21.2.101:0/2726532616 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdae01b0870 0x7fdae00f20e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7fdad4006650 tx=0x7fdad4009a00).ready entity=mon.0 client_cookie=4ac32c3dfc0fd735 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:22.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.765 7fdae48b8700 1 -- 172.21.2.101:0/2726532616 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdad40009b0 con 0x7fdae01b0870 2021-02-10T11:30:22.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.765 7fdae48b8700 1 -- 172.21.2.101:0/2726532616 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad400c0c0 con 0x7fdae01b0870 2021-02-10T11:30:22.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.765 7fdae48b8700 1 -- 172.21.2.101:0/2726532616 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdad400c5a0 con 0x7fdae01b0870 2021-02-10T11:30:22.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.765 7fdae7a9e700 1 -- 172.21.2.101:0/2726532616 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdae01b0870 msgr2=0x7fdae00f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:22.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.765 7fdae7a9e700 1 --2- 172.21.2.101:0/2726532616 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdae01b0870 0x7fdae00f20e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7fdad4006650 tx=0x7fdad4009a00).stop 2021-02-10T11:30:22.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.766 7fdae7a9e700 1 -- 172.21.2.101:0/2726532616 shutdown_connections 2021-02-10T11:30:22.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.766 7fdae7a9e700 1 --2- 172.21.2.101:0/2726532616 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdae01b0870 0x7fdae00f20e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:22.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.766 7fdae7a9e700 1 -- 172.21.2.101:0/2726532616 >> 172.21.2.101:0/2726532616 conn(0x7fdae00ce3e0 msgr2=0x7fdae00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:22.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.766 7fdae7a9e700 1 -- 172.21.2.101:0/2726532616 shutdown_connections 2021-02-10T11:30:22.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.766 7fdae7a9e700 1 -- 172.21.2.101:0/2726532616 wait complete. 2021-02-10T11:30:22.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.780 7fdae7a9e700 1 Processor -- start 2021-02-10T11:30:22.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.780 7fdae7a9e700 1 -- start start 2021-02-10T11:30:22.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.781 7fdae7a9e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdae00a1dd0 0x7fdae0190920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:22.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.781 7fdae7a9e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdae008fd30 0x7fdae00effc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:22.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.781 7fdae7a9e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdae00f0e90 0x7fdae00f1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:22.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.781 7fdae7a9e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae00f06f0 con 0x7fdae008fd30 2021-02-10T11:30:22.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.781 7fdae7a9e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae0090160 con 0x7fdae00f0e90 2021-02-10T11:30:22.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.781 7fdae7a9e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae00f18b0 con 0x7fdae00a1dd0 2021-02-10T11:30:22.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.781 7fdae50b9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdae008fd30 0x7fdae00effc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:22.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.781 7fdae50b9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdae008fd30 0x7fdae00effc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48918/0 (socket says 172.21.2.101:48918) 2021-02-10T11:30:22.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.781 7fdae50b9700 1 -- 172.21.2.101:0/3266603613 learned_addr learned my addr 172.21.2.101:0/3266603613 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:22.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.782 7fdae58ba700 1 --2- 172.21.2.101:0/3266603613 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdae00a1dd0 0x7fdae0190920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:22.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.782 7fdae50b9700 1 -- 172.21.2.101:0/3266603613 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdae00a1dd0 msgr2=0x7fdae0190920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:22.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.782 7fdae50b9700 1 --2- 172.21.2.101:0/3266603613 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdae00a1dd0 0x7fdae0190920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:22.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.782 7fdae50b9700 1 -- 172.21.2.101:0/3266603613 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdae00f0e90 msgr2=0x7fdae00f1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:22.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.782 7fdae50b9700 1 --2- 172.21.2.101:0/3266603613 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdae00f0e90 0x7fdae00f1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:22.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.782 7fdae50b9700 1 -- 172.21.2.101:0/3266603613 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae00f1ad0 con 0x7fdae008fd30 2021-02-10T11:30:22.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.782 7fdae58ba700 1 --2- 172.21.2.101:0/3266603613 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdae00a1dd0 0x7fdae0190920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:30:22.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.782 7fdae50b9700 1 --2- 172.21.2.101:0/3266603613 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdae008fd30 0x7fdae00effc0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7fdad400b640 tx=0x7fdad400b9e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:22.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.782 7fdad27fc700 1 -- 172.21.2.101:0/3266603613 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdad400bdd0 con 0x7fdae008fd30 2021-02-10T11:30:22.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.782 7fdad27fc700 1 -- 172.21.2.101:0/3266603613 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad4000c90 con 0x7fdae008fd30 2021-02-10T11:30:22.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.782 7fdad27fc700 1 -- 172.21.2.101:0/3266603613 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdad40022b0 con 0x7fdae008fd30 2021-02-10T11:30:22.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.782 7fdae7a9e700 1 -- 172.21.2.101:0/3266603613 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae00f1ad0 con 0x7fdae008fd30 2021-02-10T11:30:22.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.782 7fdae7a9e700 1 -- 172.21.2.101:0/3266603613 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae00f1ad0 con 0x7fdae008fd30 2021-02-10T11:30:22.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.783 7fdad27fc700 1 -- 172.21.2.101:0/3266603613 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdad40052e0 con 0x7fdae008fd30 2021-02-10T11:30:22.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.783 7fdad27fc700 1 --2- 172.21.2.101:0/3266603613 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdac80186c0 0x7fdac801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:22.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.783 7fdad27fc700 1 -- 172.21.2.101:0/3266603613 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fdad40325a0 con 0x7fdae008fd30 2021-02-10T11:30:22.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.783 7fdae58ba700 1 --2- 172.21.2.101:0/3266603613 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdac80186c0 0x7fdac801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:22.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.783 7fdae58ba700 1 --2- 172.21.2.101:0/3266603613 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdac80186c0 0x7fdac801abc0 crc :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:22.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.813 7fdae7a9e700 1 -- 172.21.2.101:0/3266603613 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdae00e52e0 con 0x7fdae008fd30 2021-02-10T11:30:22.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.816 7fdad27fc700 1 -- 172.21.2.101:0/3266603613 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdad4030d50 con 0x7fdae008fd30 2021-02-10T11:30:22.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.943 7fdae7a9e700 1 -- 172.21.2.101:0/3266603613 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839275", "target": ["mgr", ""]}) v1 -- 0x7fdae00e52e0 con 0x7fdac80186c0 2021-02-10T11:30:22.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.954 7fdad27fc700 1 -- 172.21.2.101:0/3266603613 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fdadc006e70 con 0x7fdac80186c0 2021-02-10T11:30:22.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.960 7fdae7a9e700 1 -- 172.21.2.101:0/3266603613 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdac80186c0 msgr2=0x7fdac801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:22.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.960 7fdae7a9e700 1 --2- 172.21.2.101:0/3266603613 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdac80186c0 0x7fdac801abc0 crc :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:22.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.960 7fdae7a9e700 1 -- 172.21.2.101:0/3266603613 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdae008fd30 msgr2=0x7fdae00effc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:22.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.960 7fdae7a9e700 1 --2- 172.21.2.101:0/3266603613 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdae008fd30 0x7fdae00effc0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7fdad400b640 tx=0x7fdad400b9e0).stop 2021-02-10T11:30:22.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.960 7fdae7a9e700 1 -- 172.21.2.101:0/3266603613 shutdown_connections 2021-02-10T11:30:22.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.960 7fdae7a9e700 1 --2- 172.21.2.101:0/3266603613 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdae00f0e90 0x7fdae00f1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:22.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.960 7fdae7a9e700 1 --2- 172.21.2.101:0/3266603613 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdae008fd30 0x7fdae00effc0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:22.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.960 7fdae7a9e700 1 --2- 172.21.2.101:0/3266603613 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdac80186c0 0x7fdac801abc0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:22.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.960 7fdae7a9e700 1 --2- 172.21.2.101:0/3266603613 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdae00a1dd0 0x7fdae0190920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:22.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.960 7fdae7a9e700 1 -- 172.21.2.101:0/3266603613 >> 172.21.2.101:0/3266603613 conn(0x7fdae00958b0 msgr2=0x7fdae00bf040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:22.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.960 7fdae7a9e700 1 -- 172.21.2.101:0/3266603613 shutdown_connections 2021-02-10T11:30:22.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:22.960 7fdae7a9e700 1 -- 172.21.2.101:0/3266603613 wait complete. 2021-02-10T11:30:22.979 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839276 2021-02-10T11:30:23.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.383 7fb287514700 1 Processor -- start 2021-02-10T11:30:23.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.383 7fb287514700 1 -- start start 2021-02-10T11:30:23.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.383 7fb287514700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb2801b0870 0x7fb2800f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:23.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.383 7fb287514700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb2800efe00 0x7fb2800f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:23.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.383 7fb287514700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2801b0c70 con 0x7fb2801b0470 2021-02-10T11:30:23.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.383 7fb287514700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2800e5ea0 con 0x7fb2801b0870 2021-02-10T11:30:23.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.383 7fb287514700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2800f0b50 con 0x7fb2800efe00 2021-02-10T11:30:23.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.384 7fb285330700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb2801b0470 0x7fb2800ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49418/0 (socket says 172.21.2.101:49418) 2021-02-10T11:30:23.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.384 7fb284b2f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb2801b0870 0x7fb2800f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:23.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.384 7fb285330700 1 -- 172.21.2.101:0/2837096889 learned_addr learned my addr 172.21.2.101:0/2837096889 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:23.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.384 7fb285b31700 1 --2- 172.21.2.101:0/2837096889 >> v2:172.21.2.108:3300/0 conn(0x7fb2800efe00 0x7fb2800f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:23.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.384 7fb284b2f700 1 -- 172.21.2.101:0/2837096889 >> v2:172.21.2.108:3300/0 conn(0x7fb2800efe00 msgr2=0x7fb2800f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:23.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.384 7fb284b2f700 1 --2- 172.21.2.101:0/2837096889 >> v2:172.21.2.108:3300/0 conn(0x7fb2800efe00 0x7fb2800f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:23.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.384 7fb284b2f700 1 -- 172.21.2.101:0/2837096889 >> v1:172.21.2.101:6789/0 conn(0x7fb2801b0470 legacy=0x7fb2800ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:23.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.384 7fb284b2f700 1 -- 172.21.2.101:0/2837096889 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2800f0820 con 0x7fb2801b0870 2021-02-10T11:30:23.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.384 7fb285b31700 1 --2- 172.21.2.101:0/2837096889 >> v2:172.21.2.108:3300/0 conn(0x7fb2800efe00 0x7fb2800f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:30:23.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.384 7fb284b2f700 1 --2- 172.21.2.101:0/2837096889 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2801b0870 0x7fb2800f20e0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7fb270006650 tx=0x7fb270009a80).ready entity=mon.0 client_cookie=d1b76d0ffbd81e3b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:23.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.384 7fb277fff700 1 -- 172.21.2.101:0/2837096889 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb2700009b0 con 0x7fb2801b0870 2021-02-10T11:30:23.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.384 7fb277fff700 1 -- 172.21.2.101:0/2837096889 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27000c0c0 con 0x7fb2801b0870 2021-02-10T11:30:23.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.384 7fb277fff700 1 -- 172.21.2.101:0/2837096889 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb27000c5e0 con 0x7fb2801b0870 2021-02-10T11:30:23.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.384 7fb287514700 1 -- 172.21.2.101:0/2837096889 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2801b0870 msgr2=0x7fb2800f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:23.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.384 7fb287514700 1 --2- 172.21.2.101:0/2837096889 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2801b0870 0x7fb2800f20e0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7fb270006650 tx=0x7fb270009a80).stop 2021-02-10T11:30:23.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.385 7fb287514700 1 -- 172.21.2.101:0/2837096889 shutdown_connections 2021-02-10T11:30:23.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.385 7fb287514700 1 --2- 172.21.2.101:0/2837096889 >> v2:172.21.2.108:3300/0 conn(0x7fb2800efe00 0x7fb2800f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:23.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.385 7fb287514700 1 --2- 172.21.2.101:0/2837096889 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2801b0870 0x7fb2800f20e0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:23.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.385 7fb287514700 1 -- 172.21.2.101:0/2837096889 >> 172.21.2.101:0/2837096889 conn(0x7fb2800ce3e0 msgr2=0x7fb2800e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:23.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.385 7fb287514700 1 -- 172.21.2.101:0/2837096889 shutdown_connections 2021-02-10T11:30:23.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.385 7fb287514700 1 -- 172.21.2.101:0/2837096889 wait complete. 2021-02-10T11:30:23.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.400 7fb287514700 1 Processor -- start 2021-02-10T11:30:23.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.400 7fb287514700 1 -- start start 2021-02-10T11:30:23.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.400 7fb287514700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2800a0420 0x7fb2800a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:23.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.400 7fb287514700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb280189d00 0x7fb28018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:23.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.400 7fb287514700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb28018a720 0x7fb2800b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:23.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.400 7fb287514700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb280189780 con 0x7fb28018a720 2021-02-10T11:30:23.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.400 7fb287514700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28018ab20 con 0x7fb280189d00 2021-02-10T11:30:23.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.400 7fb287514700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2800a1140 con 0x7fb2800a0420 2021-02-10T11:30:23.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.400 7fb285b31700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb28018a720 0x7fb2800b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:23.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.400 7fb285b31700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb28018a720 0x7fb2800b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48932/0 (socket says 172.21.2.101:48932) 2021-02-10T11:30:23.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.400 7fb285b31700 1 -- 172.21.2.101:0/3435357252 learned_addr learned my addr 172.21.2.101:0/3435357252 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:23.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.400 7fb285330700 1 --2- 172.21.2.101:0/3435357252 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2800a0420 0x7fb2800a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:23.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.400 7fb284b2f700 1 --2- 172.21.2.101:0/3435357252 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb280189d00 0x7fb28018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:23.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.401 7fb285330700 1 -- 172.21.2.101:0/3435357252 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb280189d00 msgr2=0x7fb28018a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:23.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.401 7fb285330700 1 --2- 172.21.2.101:0/3435357252 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb280189d00 0x7fb28018a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:23.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.401 7fb285330700 1 -- 172.21.2.101:0/3435357252 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb28018a720 msgr2=0x7fb2800b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:23.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.401 7fb285330700 1 --2- 172.21.2.101:0/3435357252 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb28018a720 0x7fb2800b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:23.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.401 7fb285330700 1 -- 172.21.2.101:0/3435357252 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2800b8200 con 0x7fb2800a0420 2021-02-10T11:30:23.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.401 7fb285b31700 1 --2- 172.21.2.101:0/3435357252 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb28018a720 0x7fb2800b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:23.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.401 7fb285330700 1 --2- 172.21.2.101:0/3435357252 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2800a0420 0x7fb2800a0820 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fb2780079f0 tx=0x7fb27800ac20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:23.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.401 7fb275ffb700 1 -- 172.21.2.101:0/3435357252 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb278007020 con 0x7fb2800a0420 2021-02-10T11:30:23.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.401 7fb275ffb700 1 -- 172.21.2.101:0/3435357252 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb278001500 con 0x7fb2800a0420 2021-02-10T11:30:23.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.401 7fb275ffb700 1 -- 172.21.2.101:0/3435357252 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb2780075c0 con 0x7fb2800a0420 2021-02-10T11:30:23.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.401 7fb287514700 1 -- 172.21.2.101:0/3435357252 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2800b8200 con 0x7fb2800a0420 2021-02-10T11:30:23.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.401 7fb287514700 1 -- 172.21.2.101:0/3435357252 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2800b8200 con 0x7fb2800a0420 2021-02-10T11:30:23.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.402 7fb275ffb700 1 -- 172.21.2.101:0/3435357252 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb27800ad60 con 0x7fb2800a0420 2021-02-10T11:30:23.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.402 7fb275ffb700 1 --2- 172.21.2.101:0/3435357252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2600186c0 0x7fb26001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:23.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.402 7fb275ffb700 1 -- 172.21.2.101:0/3435357252 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb278032830 con 0x7fb2800a0420 2021-02-10T11:30:23.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.402 7fb284b2f700 1 --2- 172.21.2.101:0/3435357252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2600186c0 0x7fb26001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:23.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.402 7fb284b2f700 1 --2- 172.21.2.101:0/3435357252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2600186c0 0x7fb26001abc0 crc :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:23.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.433 7fb287514700 1 -- 172.21.2.101:0/3435357252 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb280105ae0 con 0x7fb2800a0420 2021-02-10T11:30:23.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.436 7fb275ffb700 1 -- 172.21.2.101:0/3435357252 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb278001010 con 0x7fb2800a0420 2021-02-10T11:30:23.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.567 7fb287514700 1 -- 172.21.2.101:0/3435357252 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839276", "target": ["mgr", ""]}) v1 -- 0x7fb280105ae0 con 0x7fb2600186c0 2021-02-10T11:30:23.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.577 7fb275ffb700 1 -- 172.21.2.101:0/3435357252 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fb270000c90 con 0x7fb2600186c0 2021-02-10T11:30:23.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.584 7fb287514700 1 -- 172.21.2.101:0/3435357252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2600186c0 msgr2=0x7fb26001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:23.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.584 7fb287514700 1 --2- 172.21.2.101:0/3435357252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2600186c0 0x7fb26001abc0 crc :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:23.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.584 7fb287514700 1 -- 172.21.2.101:0/3435357252 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2800a0420 msgr2=0x7fb2800a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:23.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.584 7fb287514700 1 --2- 172.21.2.101:0/3435357252 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2800a0420 0x7fb2800a0820 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fb2780079f0 tx=0x7fb27800ac20).stop 2021-02-10T11:30:23.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.584 7fb287514700 1 -- 172.21.2.101:0/3435357252 shutdown_connections 2021-02-10T11:30:23.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.584 7fb287514700 1 --2- 172.21.2.101:0/3435357252 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb28018a720 0x7fb2800b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:23.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.584 7fb287514700 1 --2- 172.21.2.101:0/3435357252 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb280189d00 0x7fb28018a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:23.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.584 7fb287514700 1 --2- 172.21.2.101:0/3435357252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2600186c0 0x7fb26001abc0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:23.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.584 7fb287514700 1 --2- 172.21.2.101:0/3435357252 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2800a0420 0x7fb2800a0820 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:23.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.584 7fb287514700 1 -- 172.21.2.101:0/3435357252 >> 172.21.2.101:0/3435357252 conn(0x7fb280094e10 msgr2=0x7fb2800a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:23.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.584 7fb287514700 1 -- 172.21.2.101:0/3435357252 shutdown_connections 2021-02-10T11:30:23.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:23.584 7fb287514700 1 -- 172.21.2.101:0/3435357252 wait complete. 2021-02-10T11:30:23.602 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839277 2021-02-10T11:30:24.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.011 7f236313b700 1 Processor -- start 2021-02-10T11:30:24.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.012 7f236313b700 1 -- start start 2021-02-10T11:30:24.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.012 7f236313b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f235c1b0870 0x7f235c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:24.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.012 7f236313b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f235c0efe00 0x7f235c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:24.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.012 7f236313b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f235c1b0c70 con 0x7f235c1b0470 2021-02-10T11:30:24.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.012 7f236313b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f235c0f0b50 con 0x7f235c1b0870 2021-02-10T11:30:24.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.012 7f236313b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235c0f0820 con 0x7f235c0efe00 2021-02-10T11:30:24.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.012 7f235bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f235c1b0870 0x7f235c0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:24.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.012 7f2360f57700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f235c1b0470 0x7f235c0ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49434/0 (socket says 172.21.2.101:49434) 2021-02-10T11:30:24.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.012 7f2360f57700 1 -- 172.21.2.101:0/1269732035 learned_addr learned my addr 172.21.2.101:0/1269732035 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:24.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.012 7f2361758700 1 --2- 172.21.2.101:0/1269732035 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f235c0efe00 0x7f235c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:24.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.013 7f235bfff700 1 -- 172.21.2.101:0/1269732035 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f235c0efe00 msgr2=0x7f235c0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:24.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.013 7f235bfff700 1 --2- 172.21.2.101:0/1269732035 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f235c0efe00 0x7f235c0f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:24.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.013 7f235bfff700 1 -- 172.21.2.101:0/1269732035 >> v1:172.21.2.101:6789/0 conn(0x7f235c1b0470 legacy=0x7f235c0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:24.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.013 7f235bfff700 1 -- 172.21.2.101:0/1269732035 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c0e52e0 con 0x7f235c1b0870 2021-02-10T11:30:24.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.013 7f235b7fe700 1 -- 172.21.2.101:0/1269732035 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318072114 0 0) 0x7f23440009b0 con 0x7f235c1b0470 2021-02-10T11:30:24.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.013 7f235bfff700 1 --2- 172.21.2.101:0/1269732035 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f235c1b0870 0x7f235c0f20e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f234c006650 tx=0x7f234c009a80).ready entity=mon.0 client_cookie=4e5143c5392cbce0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:24.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.013 7f235b7fe700 1 -- 172.21.2.101:0/1269732035 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f234c0009b0 con 0x7f235c1b0870 2021-02-10T11:30:24.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.013 7f235b7fe700 1 -- 172.21.2.101:0/1269732035 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234c00c0c0 con 0x7f235c1b0870 2021-02-10T11:30:24.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.013 7f235b7fe700 1 -- 172.21.2.101:0/1269732035 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f234c00c520 con 0x7f235c1b0870 2021-02-10T11:30:24.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.013 7f236313b700 1 -- 172.21.2.101:0/1269732035 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f235c1b0870 msgr2=0x7f235c0f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:24.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.013 7f236313b700 1 --2- 172.21.2.101:0/1269732035 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f235c1b0870 0x7f235c0f20e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f234c006650 tx=0x7f234c009a80).stop 2021-02-10T11:30:24.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.013 7f236313b700 1 -- 172.21.2.101:0/1269732035 shutdown_connections 2021-02-10T11:30:24.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.013 7f236313b700 1 --2- 172.21.2.101:0/1269732035 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f235c0efe00 0x7f235c0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:24.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.013 7f236313b700 1 --2- 172.21.2.101:0/1269732035 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f235c1b0870 0x7f235c0f20e0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:24.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.013 7f236313b700 1 -- 172.21.2.101:0/1269732035 >> 172.21.2.101:0/1269732035 conn(0x7f235c0ce3e0 msgr2=0x7f235c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:24.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.014 7f236313b700 1 -- 172.21.2.101:0/1269732035 shutdown_connections 2021-02-10T11:30:24.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.014 7f236313b700 1 -- 172.21.2.101:0/1269732035 wait complete. 2021-02-10T11:30:24.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.028 7f236313b700 1 Processor -- start 2021-02-10T11:30:24.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.028 7f236313b700 1 -- start start 2021-02-10T11:30:24.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f236313b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f235c0a0420 0x7f235c0a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:24.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f236313b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f235c189d00 0x7f235c18a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:24.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f236313b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f235c18a720 0x7f235c0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:24.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f236313b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c189780 con 0x7f235c18a720 2021-02-10T11:30:24.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f236313b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c18ab20 con 0x7f235c0a0420 2021-02-10T11:30:24.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f236313b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235c0a1140 con 0x7f235c189d00 2021-02-10T11:30:24.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f2361758700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f235c18a720 0x7f235c0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:24.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f2361758700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f235c18a720 0x7f235c0b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48946/0 (socket says 172.21.2.101:48946) 2021-02-10T11:30:24.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f2361758700 1 -- 172.21.2.101:0/1924831604 learned_addr learned my addr 172.21.2.101:0/1924831604 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:24.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f235bfff700 1 --2- 172.21.2.101:0/1924831604 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f235c189d00 0x7f235c18a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:24.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f2361758700 1 -- 172.21.2.101:0/1924831604 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f235c189d00 msgr2=0x7f235c18a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:24.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f2361758700 1 --2- 172.21.2.101:0/1924831604 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f235c189d00 0x7f235c18a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:24.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f2360f57700 1 --2- 172.21.2.101:0/1924831604 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f235c0a0420 0x7f235c0a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:24.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f2361758700 1 -- 172.21.2.101:0/1924831604 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f235c0a0420 msgr2=0x7f235c0a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:24.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f2361758700 1 --2- 172.21.2.101:0/1924831604 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f235c0a0420 0x7f235c0a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:24.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f2361758700 1 -- 172.21.2.101:0/1924831604 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c0b8200 con 0x7f235c18a720 2021-02-10T11:30:24.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f2360f57700 1 --2- 172.21.2.101:0/1924831604 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f235c0a0420 0x7f235c0a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:30:24.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f2361758700 1 --2- 172.21.2.101:0/1924831604 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f235c18a720 0x7f235c0b7be0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7f235400b910 tx=0x7f235400bcb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:24.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f23597fa700 1 -- 172.21.2.101:0/1924831604 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2354006aa0 con 0x7f235c18a720 2021-02-10T11:30:24.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f23597fa700 1 -- 172.21.2.101:0/1924831604 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2354003030 con 0x7f235c18a720 2021-02-10T11:30:24.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f236313b700 1 -- 172.21.2.101:0/1924831604 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235c0b8200 con 0x7f235c18a720 2021-02-10T11:30:24.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f23597fa700 1 -- 172.21.2.101:0/1924831604 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2354001470 con 0x7f235c18a720 2021-02-10T11:30:24.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.029 7f236313b700 1 -- 172.21.2.101:0/1924831604 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235c0b8200 con 0x7f235c18a720 2021-02-10T11:30:24.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.030 7f23597fa700 1 -- 172.21.2.101:0/1924831604 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2354000ec0 con 0x7f235c18a720 2021-02-10T11:30:24.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.030 7f23597fa700 1 --2- 172.21.2.101:0/1924831604 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2340018730 0x7f234001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:24.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.030 7f23597fa700 1 -- 172.21.2.101:0/1924831604 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f23540329e0 con 0x7f235c18a720 2021-02-10T11:30:24.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.030 7f2360f57700 1 --2- 172.21.2.101:0/1924831604 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2340018730 0x7f234001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:24.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.030 7f2360f57700 1 --2- 172.21.2.101:0/1924831604 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2340018730 0x7f234001ac10 crc :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:24.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.061 7f236313b700 1 -- 172.21.2.101:0/1924831604 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235c105ae0 con 0x7f235c18a720 2021-02-10T11:30:24.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.064 7f23597fa700 1 -- 172.21.2.101:0/1924831604 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f235400bdf0 con 0x7f235c18a720 2021-02-10T11:30:24.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.190 7f236313b700 1 -- 172.21.2.101:0/1924831604 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839277", "target": ["mgr", ""]}) v1 -- 0x7f235c0f0b50 con 0x7f2340018730 2021-02-10T11:30:24.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.201 7f23597fa700 1 -- 172.21.2.101:0/1924831604 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f2344000e20 con 0x7f2340018730 2021-02-10T11:30:24.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.207 7f236313b700 1 -- 172.21.2.101:0/1924831604 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2340018730 msgr2=0x7f234001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:24.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.207 7f236313b700 1 --2- 172.21.2.101:0/1924831604 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2340018730 0x7f234001ac10 crc :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:24.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.207 7f236313b700 1 -- 172.21.2.101:0/1924831604 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f235c18a720 msgr2=0x7f235c0b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:24.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.207 7f236313b700 1 --2- 172.21.2.101:0/1924831604 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f235c18a720 0x7f235c0b7be0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7f235400b910 tx=0x7f235400bcb0).stop 2021-02-10T11:30:24.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.208 7f236313b700 1 -- 172.21.2.101:0/1924831604 shutdown_connections 2021-02-10T11:30:24.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.208 7f236313b700 1 --2- 172.21.2.101:0/1924831604 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2340018730 0x7f234001ac10 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:24.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.208 7f236313b700 1 --2- 172.21.2.101:0/1924831604 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f235c189d00 0x7f235c18a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:24.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.208 7f236313b700 1 --2- 172.21.2.101:0/1924831604 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f235c18a720 0x7f235c0b7be0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:24.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.208 7f236313b700 1 --2- 172.21.2.101:0/1924831604 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f235c0a0420 0x7f235c0a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:24.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.208 7f236313b700 1 -- 172.21.2.101:0/1924831604 >> 172.21.2.101:0/1924831604 conn(0x7f235c094e10 msgr2=0x7f235c0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:24.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.208 7f236313b700 1 -- 172.21.2.101:0/1924831604 shutdown_connections 2021-02-10T11:30:24.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.208 7f236313b700 1 -- 172.21.2.101:0/1924831604 wait complete. 2021-02-10T11:30:24.227 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839278 2021-02-10T11:30:24.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.632 7fc17374b700 1 Processor -- start 2021-02-10T11:30:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.632 7fc17374b700 1 -- start start 2021-02-10T11:30:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.633 7fc17374b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc16c1b0470 0x7fc16c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.633 7fc17374b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc16c0ef610 0x7fc16c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.633 7fc17374b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc16c0f0b50 con 0x7fc16c0f0030 2021-02-10T11:30:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.633 7fc17374b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc16c0e52e0 con 0x7fc16c1b0470 2021-02-10T11:30:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.633 7fc17374b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0e54d0 con 0x7fc16c0ef610 2021-02-10T11:30:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.633 7fc171d68700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc16c0f0030 0x7fc16c0f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49446/0 (socket says 172.21.2.101:49446) 2021-02-10T11:30:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.633 7fc171d68700 1 -- 172.21.2.101:0/3044081130 learned_addr learned my addr 172.21.2.101:0/3044081130 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.633 7fc171567700 1 --2- 172.21.2.101:0/3044081130 >> v2:172.21.2.108:3300/0 conn(0x7fc16c1b0470 0x7fc16c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.633 7fc170d66700 1 --2- 172.21.2.101:0/3044081130 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc16c0ef610 0x7fc16c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.633 7fc15ffff700 1 -- 172.21.2.101:0/3044081130 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 658820477 0 0) 0x7fc164001500 con 0x7fc16c0f0030 2021-02-10T11:30:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.633 7fc15ffff700 1 -- 172.21.2.101:0/3044081130 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc158002fe0 con 0x7fc16c0f0030 2021-02-10T11:30:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.633 7fc170d66700 1 -- 172.21.2.101:0/3044081130 >> v2:172.21.2.108:3300/0 conn(0x7fc16c1b0470 msgr2=0x7fc16c1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:24.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.633 7fc170d66700 1 --2- 172.21.2.101:0/3044081130 >> v2:172.21.2.108:3300/0 conn(0x7fc16c1b0470 0x7fc16c1b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:24.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.633 7fc170d66700 1 -- 172.21.2.101:0/3044081130 >> v1:172.21.2.101:6789/0 conn(0x7fc16c0f0030 legacy=0x7fc16c0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:24.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.633 7fc170d66700 1 -- 172.21.2.101:0/3044081130 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c0f0430 con 0x7fc16c0ef610 2021-02-10T11:30:24.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.633 7fc171567700 1 --2- 172.21.2.101:0/3044081130 >> v2:172.21.2.108:3300/0 conn(0x7fc16c1b0470 0x7fc16c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:24.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.634 7fc170d66700 1 --2- 172.21.2.101:0/3044081130 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc16c0ef610 0x7fc16c0efa10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fc160006650 tx=0x7fc160009a80).ready entity=mon.2 client_cookie=c2128730948c3d14 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:24.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.634 7fc15ffff700 1 -- 172.21.2.101:0/3044081130 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc16000c0c0 con 0x7fc16c0ef610 2021-02-10T11:30:24.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.634 7fc15ffff700 1 -- 172.21.2.101:0/3044081130 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16000c340 con 0x7fc16c0ef610 2021-02-10T11:30:24.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.634 7fc15ffff700 1 -- 172.21.2.101:0/3044081130 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc16000c860 con 0x7fc16c0ef610 2021-02-10T11:30:24.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.634 7fc17374b700 1 -- 172.21.2.101:0/3044081130 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc16c0ef610 msgr2=0x7fc16c0efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:24.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.634 7fc17374b700 1 --2- 172.21.2.101:0/3044081130 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc16c0ef610 0x7fc16c0efa10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fc160006650 tx=0x7fc160009a80).stop 2021-02-10T11:30:24.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.635 7fc17374b700 1 -- 172.21.2.101:0/3044081130 shutdown_connections 2021-02-10T11:30:24.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.635 7fc17374b700 1 --2- 172.21.2.101:0/3044081130 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc16c0ef610 0x7fc16c0efa10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:24.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.635 7fc17374b700 1 --2- 172.21.2.101:0/3044081130 >> v2:172.21.2.108:3300/0 conn(0x7fc16c1b0470 0x7fc16c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:24.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.635 7fc17374b700 1 -- 172.21.2.101:0/3044081130 >> 172.21.2.101:0/3044081130 conn(0x7fc16c0ce3e0 msgr2=0x7fc16c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:24.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.635 7fc17374b700 1 -- 172.21.2.101:0/3044081130 shutdown_connections 2021-02-10T11:30:24.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.635 7fc17374b700 1 -- 172.21.2.101:0/3044081130 wait complete. 2021-02-10T11:30:24.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.650 7fc17374b700 1 Processor -- start 2021-02-10T11:30:24.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.650 7fc17374b700 1 -- start start 2021-02-10T11:30:24.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.650 7fc17374b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc16c09f140 0x7fc16c0c1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:24.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.650 7fc17374b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc16c0c1f60 0x7fc16c09e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:24.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.650 7fc17374b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc16c09e920 0x7fc16c0c2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:24.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.650 7fc17374b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0c23e0 con 0x7fc16c0c1f60 2021-02-10T11:30:24.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.650 7fc17374b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c09ed20 con 0x7fc16c09e920 2021-02-10T11:30:24.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.650 7fc17374b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c105d10 con 0x7fc16c09f140 2021-02-10T11:30:24.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.650 7fc170d66700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc16c0c1f60 0x7fc16c09e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:24.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.650 7fc170d66700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc16c0c1f60 0x7fc16c09e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48958/0 (socket says 172.21.2.101:48958) 2021-02-10T11:30:24.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.650 7fc170d66700 1 -- 172.21.2.101:0/3768092634 learned_addr learned my addr 172.21.2.101:0/3768092634 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:24.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.651 7fc171d68700 1 --2- 172.21.2.101:0/3768092634 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc16c09e920 0x7fc16c0c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:24.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.651 7fc171567700 1 --2- 172.21.2.101:0/3768092634 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc16c09f140 0x7fc16c0c1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:24.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.651 7fc170d66700 1 -- 172.21.2.101:0/3768092634 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc16c09f140 msgr2=0x7fc16c0c1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:24.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.651 7fc170d66700 1 --2- 172.21.2.101:0/3768092634 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc16c09f140 0x7fc16c0c1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:24.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.651 7fc170d66700 1 -- 172.21.2.101:0/3768092634 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc16c09e920 msgr2=0x7fc16c0c2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:24.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.651 7fc170d66700 1 --2- 172.21.2.101:0/3768092634 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc16c09e920 0x7fc16c0c2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:24.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.651 7fc170d66700 1 -- 172.21.2.101:0/3768092634 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c1a4910 con 0x7fc16c0c1f60 2021-02-10T11:30:24.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.651 7fc171d68700 1 --2- 172.21.2.101:0/3768092634 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc16c09e920 0x7fc16c0c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:30:24.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.651 7fc170d66700 1 --2- 172.21.2.101:0/3768092634 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc16c0c1f60 0x7fc16c09e300 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7fc1600093a0 tx=0x7fc160002670).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:24.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.651 7fc15dffb700 1 -- 172.21.2.101:0/3768092634 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc160006070 con 0x7fc16c0c1f60 2021-02-10T11:30:24.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.651 7fc15dffb700 1 -- 172.21.2.101:0/3768092634 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc160006ae0 con 0x7fc16c0c1f60 2021-02-10T11:30:24.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.651 7fc15dffb700 1 -- 172.21.2.101:0/3768092634 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc160007030 con 0x7fc16c0c1f60 2021-02-10T11:30:24.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.651 7fc17374b700 1 -- 172.21.2.101:0/3768092634 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16c1a4910 con 0x7fc16c0c1f60 2021-02-10T11:30:24.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.651 7fc17374b700 1 -- 172.21.2.101:0/3768092634 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc16c1a4910 con 0x7fc16c0c1f60 2021-02-10T11:30:24.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.652 7fc15dffb700 1 -- 172.21.2.101:0/3768092634 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc160007000 con 0x7fc16c0c1f60 2021-02-10T11:30:24.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.652 7fc15dffb700 1 --2- 172.21.2.101:0/3768092634 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc14c018730 0x7fc14c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:24.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.652 7fc15dffb700 1 -- 172.21.2.101:0/3768092634 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc160032850 con 0x7fc16c0c1f60 2021-02-10T11:30:24.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.652 7fc171567700 1 --2- 172.21.2.101:0/3768092634 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc14c018730 0x7fc14c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:24.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.652 7fc171567700 1 --2- 172.21.2.101:0/3768092634 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc14c018730 0x7fc14c01abc0 crc :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:24.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.682 7fc17374b700 1 -- 172.21.2.101:0/3768092634 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc16c0f0b50 con 0x7fc16c0c1f60 2021-02-10T11:30:24.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.685 7fc15dffb700 1 -- 172.21.2.101:0/3768092634 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc160002a60 con 0x7fc16c0c1f60 2021-02-10T11:30:24.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.810 7fc17374b700 1 -- 172.21.2.101:0/3768092634 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839278", "target": ["mgr", ""]}) v1 -- 0x7fc16c0f0b50 con 0x7fc14c018730 2021-02-10T11:30:24.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.821 7fc15dffb700 1 -- 172.21.2.101:0/3768092634 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fc1680009b0 con 0x7fc14c018730 2021-02-10T11:30:24.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.827 7fc17374b700 1 -- 172.21.2.101:0/3768092634 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc14c018730 msgr2=0x7fc14c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:24.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.827 7fc17374b700 1 --2- 172.21.2.101:0/3768092634 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc14c018730 0x7fc14c01abc0 crc :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:24.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.827 7fc17374b700 1 -- 172.21.2.101:0/3768092634 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc16c0c1f60 msgr2=0x7fc16c09e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:24.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.827 7fc17374b700 1 --2- 172.21.2.101:0/3768092634 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc16c0c1f60 0x7fc16c09e300 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7fc1600093a0 tx=0x7fc160002670).stop 2021-02-10T11:30:24.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.828 7fc17374b700 1 -- 172.21.2.101:0/3768092634 shutdown_connections 2021-02-10T11:30:24.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.828 7fc17374b700 1 --2- 172.21.2.101:0/3768092634 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc16c09e920 0x7fc16c0c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:24.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.828 7fc17374b700 1 --2- 172.21.2.101:0/3768092634 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc16c0c1f60 0x7fc16c09e300 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:24.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.828 7fc17374b700 1 --2- 172.21.2.101:0/3768092634 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc14c018730 0x7fc14c01abc0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:24.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.828 7fc17374b700 1 --2- 172.21.2.101:0/3768092634 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc16c09f140 0x7fc16c0c1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:24.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.828 7fc17374b700 1 -- 172.21.2.101:0/3768092634 >> 172.21.2.101:0/3768092634 conn(0x7fc16c09fc10 msgr2=0x7fc16c0b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:24.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.828 7fc17374b700 1 -- 172.21.2.101:0/3768092634 shutdown_connections 2021-02-10T11:30:24.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:24.828 7fc17374b700 1 -- 172.21.2.101:0/3768092634 wait complete. 2021-02-10T11:30:24.846 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839279 2021-02-10T11:30:25.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.253 7fbb68ad7700 1 Processor -- start 2021-02-10T11:30:25.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.253 7fbb68ad7700 1 -- start start 2021-02-10T11:30:25.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.253 7fbb68ad7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb641b0470 0x7fbb641b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:25.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.253 7fbb68ad7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbb640f20e0 0x7fbb640f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:25.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.253 7fbb68ad7700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb640f0b50 con 0x7fbb640ef610 2021-02-10T11:30:25.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.253 7fbb68ad7700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb640f0820 con 0x7fbb640f20e0 2021-02-10T11:30:25.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.253 7fbb68ad7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb640f24e0 con 0x7fbb641b0470 2021-02-10T11:30:25.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.253 7fbb6305f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbb640f20e0 0x7fbb640f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:25.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.254 7fbb6305f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbb640f20e0 0x7fbb640f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48964/0 (socket says 172.21.2.101:48964) 2021-02-10T11:30:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.254 7fbb6305f700 1 -- 172.21.2.101:0/10838182 learned_addr learned my addr 172.21.2.101:0/10838182 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.254 7fbb6285e700 1 --2- 172.21.2.101:0/10838182 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb641b0470 0x7fbb641b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.254 7fbb6305f700 1 -- 172.21.2.101:0/10838182 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb641b0470 msgr2=0x7fbb641b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.254 7fbb6305f700 1 --2- 172.21.2.101:0/10838182 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb641b0470 0x7fbb641b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.254 7fbb6305f700 1 -- 172.21.2.101:0/10838182 >> v1:172.21.2.108:6789/0 conn(0x7fbb640ef610 legacy=0x7fbb640efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.254 7fbb6305f700 1 -- 172.21.2.101:0/10838182 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb640e52e0 con 0x7fbb640f20e0 2021-02-10T11:30:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.254 7fbb6305f700 1 --2- 172.21.2.101:0/10838182 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb640f20e0 0x7fbb640f0200 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7fbb5c006f00 tx=0x7fbb5c009f10).ready entity=mon.0 client_cookie=9b586ddfa7cbcda4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.254 7fbb6205d700 1 -- 172.21.2.101:0/10838182 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbb5c003030 con 0x7fbb640f20e0 2021-02-10T11:30:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.254 7fbb6205d700 1 -- 172.21.2.101:0/10838182 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5c00b3b0 con 0x7fbb640f20e0 2021-02-10T11:30:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.254 7fbb6205d700 1 -- 172.21.2.101:0/10838182 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbb5c00b8d0 con 0x7fbb640f20e0 2021-02-10T11:30:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.254 7fbb68ad7700 1 -- 172.21.2.101:0/10838182 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb640f20e0 msgr2=0x7fbb640f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.254 7fbb68ad7700 1 --2- 172.21.2.101:0/10838182 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb640f20e0 0x7fbb640f0200 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7fbb5c006f00 tx=0x7fbb5c009f10).stop 2021-02-10T11:30:25.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.255 7fbb68ad7700 1 -- 172.21.2.101:0/10838182 shutdown_connections 2021-02-10T11:30:25.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.255 7fbb68ad7700 1 --2- 172.21.2.101:0/10838182 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb640f20e0 0x7fbb640f0200 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:25.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.255 7fbb68ad7700 1 --2- 172.21.2.101:0/10838182 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb641b0470 0x7fbb641b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:25.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.255 7fbb68ad7700 1 -- 172.21.2.101:0/10838182 >> 172.21.2.101:0/10838182 conn(0x7fbb640ce3e0 msgr2=0x7fbb640e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:25.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.255 7fbb68ad7700 1 -- 172.21.2.101:0/10838182 shutdown_connections 2021-02-10T11:30:25.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.255 7fbb68ad7700 1 -- 172.21.2.101:0/10838182 wait complete. 2021-02-10T11:30:25.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.269 7fbb68ad7700 1 Processor -- start 2021-02-10T11:30:25.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.269 7fbb68ad7700 1 -- start start 2021-02-10T11:30:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.270 7fbb68ad7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb640b2eb0 0x7fbb640b2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.270 7fbb68ad7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb640b2950 0x7fbb640b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.270 7fbb68ad7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb640b39a0 0x7fbb6409d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.270 7fbb68ad7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6409cdc0 con 0x7fbb640b2950 2021-02-10T11:30:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.270 7fbb68ad7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6409d960 con 0x7fbb640b39a0 2021-02-10T11:30:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.270 7fbb68ad7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6409db80 con 0x7fbb640b2eb0 2021-02-10T11:30:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.270 7fbb5bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb640b2950 0x7fbb640b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.270 7fbb5bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb640b2950 0x7fbb640b3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48972/0 (socket says 172.21.2.101:48972) 2021-02-10T11:30:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.270 7fbb5bfff700 1 -- 172.21.2.101:0/481233478 learned_addr learned my addr 172.21.2.101:0/481233478 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.270 7fbb6285e700 1 --2- 172.21.2.101:0/481233478 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb640b2eb0 0x7fbb640b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.270 7fbb6305f700 1 --2- 172.21.2.101:0/481233478 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb640b39a0 0x7fbb6409d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.270 7fbb5bfff700 1 -- 172.21.2.101:0/481233478 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb640b2eb0 msgr2=0x7fbb640b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.270 7fbb5bfff700 1 --2- 172.21.2.101:0/481233478 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb640b2eb0 0x7fbb640b2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:25.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.270 7fbb5bfff700 1 -- 172.21.2.101:0/481233478 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb640b39a0 msgr2=0x7fbb6409d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.270 7fbb5bfff700 1 --2- 172.21.2.101:0/481233478 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb640b39a0 0x7fbb6409d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.270 7fbb5bfff700 1 -- 172.21.2.101:0/481233478 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6409dda0 con 0x7fbb640b2950 2021-02-10T11:30:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.271 7fbb5bfff700 1 --2- 172.21.2.101:0/481233478 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb640b2950 0x7fbb640b3380 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7fbb5400b5b0 tx=0x7fbb5400b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.271 7fbb5b7fe700 1 -- 172.21.2.101:0/481233478 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbb5400bd40 con 0x7fbb640b2950 2021-02-10T11:30:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.271 7fbb5b7fe700 1 -- 172.21.2.101:0/481233478 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb54000e20 con 0x7fbb640b2950 2021-02-10T11:30:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.271 7fbb68ad7700 1 -- 172.21.2.101:0/481233478 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6409dda0 con 0x7fbb640b2950 2021-02-10T11:30:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.271 7fbb68ad7700 1 -- 172.21.2.101:0/481233478 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6409e330 con 0x7fbb640b2950 2021-02-10T11:30:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.271 7fbb5b7fe700 1 -- 172.21.2.101:0/481233478 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbb540054a0 con 0x7fbb640b2950 2021-02-10T11:30:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.271 7fbb5b7fe700 1 -- 172.21.2.101:0/481233478 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbb54006030 con 0x7fbb640b2950 2021-02-10T11:30:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.271 7fbb5b7fe700 1 --2- 172.21.2.101:0/481233478 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbb48018730 0x7fbb4801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.271 7fbb5b7fe700 1 -- 172.21.2.101:0/481233478 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fbb54032820 con 0x7fbb640b2950 2021-02-10T11:30:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.272 7fbb6285e700 1 --2- 172.21.2.101:0/481233478 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbb48018730 0x7fbb4801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:25.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.272 7fbb6285e700 1 --2- 172.21.2.101:0/481233478 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbb48018730 0x7fbb4801ac10 crc :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:25.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.303 7fbb68ad7700 1 -- 172.21.2.101:0/481233478 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb64105b30 con 0x7fbb640b2950 2021-02-10T11:30:25.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.306 7fbb5b7fe700 1 -- 172.21.2.101:0/481233478 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbb540009b0 con 0x7fbb640b2950 2021-02-10T11:30:25.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.434 7fbb68ad7700 1 -- 172.21.2.101:0/481233478 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839279", "target": ["mgr", ""]}) v1 -- 0x7fbb6409e330 con 0x7fbb48018730 2021-02-10T11:30:25.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.444 7fbb5b7fe700 1 -- 172.21.2.101:0/481233478 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fbb4c0009b0 con 0x7fbb48018730 2021-02-10T11:30:25.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.451 7fbb68ad7700 1 -- 172.21.2.101:0/481233478 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbb48018730 msgr2=0x7fbb4801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:25.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.451 7fbb68ad7700 1 --2- 172.21.2.101:0/481233478 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbb48018730 0x7fbb4801ac10 crc :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:25.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.451 7fbb68ad7700 1 -- 172.21.2.101:0/481233478 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb640b2950 msgr2=0x7fbb640b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:25.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.451 7fbb68ad7700 1 --2- 172.21.2.101:0/481233478 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb640b2950 0x7fbb640b3380 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7fbb5400b5b0 tx=0x7fbb5400b950).stop 2021-02-10T11:30:25.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.451 7fbb68ad7700 1 -- 172.21.2.101:0/481233478 shutdown_connections 2021-02-10T11:30:25.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.451 7fbb68ad7700 1 --2- 172.21.2.101:0/481233478 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb640b39a0 0x7fbb6409d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:25.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.451 7fbb68ad7700 1 --2- 172.21.2.101:0/481233478 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb640b2950 0x7fbb640b3380 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:25.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.451 7fbb68ad7700 1 --2- 172.21.2.101:0/481233478 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbb48018730 0x7fbb4801ac10 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:25.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.451 7fbb68ad7700 1 --2- 172.21.2.101:0/481233478 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb640b2eb0 0x7fbb640b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:25.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.451 7fbb68ad7700 1 -- 172.21.2.101:0/481233478 >> 172.21.2.101:0/481233478 conn(0x7fbb6409fc90 msgr2=0x7fbb64094ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:25.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.451 7fbb68ad7700 1 -- 172.21.2.101:0/481233478 shutdown_connections 2021-02-10T11:30:25.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.451 7fbb68ad7700 1 -- 172.21.2.101:0/481233478 wait complete. 2021-02-10T11:30:25.470 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839280 2021-02-10T11:30:25.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.877 7fda7c725700 1 Processor -- start 2021-02-10T11:30:25.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.877 7fda7c725700 1 -- start start 2021-02-10T11:30:25.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.878 7fda7c725700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fda741b0870 0x7fda740f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:25.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.878 7fda7c725700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda741b0c70 con 0x7fda740efe00 2021-02-10T11:30:25.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.878 7fda7c725700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda740f0b50 con 0x7fda741b0470 2021-02-10T11:30:25.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.878 7fda7c725700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda740e5ea0 con 0x7fda741b0870 2021-02-10T11:30:25.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.878 7fda79d40700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fda741b0870 0x7fda740f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:25.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.878 7fda79d40700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fda741b0870 0x7fda740f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48976/0 (socket says 172.21.2.101:48976) 2021-02-10T11:30:25.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.878 7fda79d40700 1 -- 172.21.2.101:0/2835665657 learned_addr learned my addr 172.21.2.101:0/2835665657 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:25.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.878 7fda79d40700 1 -- 172.21.2.101:0/2835665657 >> v1:172.21.2.108:6789/0 conn(0x7fda741b0470 legacy=0x7fda740ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:25.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.878 7fda79d40700 1 -- 172.21.2.101:0/2835665657 >> v1:172.21.2.101:6789/0 conn(0x7fda740efe00 legacy=0x7fda740f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:25.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.878 7fda79d40700 1 -- 172.21.2.101:0/2835665657 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda740e52e0 con 0x7fda741b0870 2021-02-10T11:30:25.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.878 7fda79d40700 1 --2- 172.21.2.101:0/2835665657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fda741b0870 0x7fda740f20e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7fda70006700 tx=0x7fda70009b80).ready entity=mon.0 client_cookie=56c4296475e3c426 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:25.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.879 7fda7953f700 1 -- 172.21.2.101:0/2835665657 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fda7000c0c0 con 0x7fda741b0870 2021-02-10T11:30:25.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.879 7fda7953f700 1 -- 172.21.2.101:0/2835665657 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7000c340 con 0x7fda741b0870 2021-02-10T11:30:25.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.879 7fda7953f700 1 -- 172.21.2.101:0/2835665657 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fda7000c860 con 0x7fda741b0870 2021-02-10T11:30:25.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.879 7fda7c725700 1 -- 172.21.2.101:0/2835665657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fda741b0870 msgr2=0x7fda740f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:25.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.879 7fda7c725700 1 --2- 172.21.2.101:0/2835665657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fda741b0870 0x7fda740f20e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7fda70006700 tx=0x7fda70009b80).stop 2021-02-10T11:30:25.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.879 7fda7c725700 1 -- 172.21.2.101:0/2835665657 shutdown_connections 2021-02-10T11:30:25.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.879 7fda7c725700 1 --2- 172.21.2.101:0/2835665657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fda741b0870 0x7fda740f20e0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:25.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.879 7fda7c725700 1 -- 172.21.2.101:0/2835665657 >> 172.21.2.101:0/2835665657 conn(0x7fda740ce3e0 msgr2=0x7fda740e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:25.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.879 7fda7c725700 1 -- 172.21.2.101:0/2835665657 shutdown_connections 2021-02-10T11:30:25.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.880 7fda7c725700 1 -- 172.21.2.101:0/2835665657 wait complete. 2021-02-10T11:30:25.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.894 7fda7c725700 1 Processor -- start 2021-02-10T11:30:25.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.894 7fda7c725700 1 -- start start 2021-02-10T11:30:25.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.895 7fda7c725700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fda740abfc0 0x7fda740a9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:25.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.895 7fda7c725700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fda740a9eb0 0x7fda740aa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:25.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.895 7fda7c725700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fda740aa8d0 0x7fda740a7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:25.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.895 7fda7c725700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda740ac3c0 con 0x7fda740aa8d0 2021-02-10T11:30:25.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.895 7fda7c725700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda740ac5b0 con 0x7fda740a9eb0 2021-02-10T11:30:25.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.895 7fda7c725700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda740a8140 con 0x7fda740abfc0 2021-02-10T11:30:25.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.895 7fda7ad42700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fda740aa8d0 0x7fda740a7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:25.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.895 7fda7ad42700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fda740aa8d0 0x7fda740a7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48982/0 (socket says 172.21.2.101:48982) 2021-02-10T11:30:25.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.895 7fda7ad42700 1 -- 172.21.2.101:0/672044825 learned_addr learned my addr 172.21.2.101:0/672044825 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:25.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.895 7fda7a541700 1 --2- 172.21.2.101:0/672044825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fda740abfc0 0x7fda740a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:25.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.895 7fda79d40700 1 --2- 172.21.2.101:0/672044825 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fda740a9eb0 0x7fda740aa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:25.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.895 7fda7ad42700 1 -- 172.21.2.101:0/672044825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fda740abfc0 msgr2=0x7fda740a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:25.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.895 7fda7ad42700 1 --2- 172.21.2.101:0/672044825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fda740abfc0 0x7fda740a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:25.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.895 7fda7ad42700 1 -- 172.21.2.101:0/672044825 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fda740a9eb0 msgr2=0x7fda740aa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.895 7fda7ad42700 1 --2- 172.21.2.101:0/672044825 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fda740a9eb0 0x7fda740aa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.895 7fda7ad42700 1 -- 172.21.2.101:0/672044825 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda740a8360 con 0x7fda740aa8d0 2021-02-10T11:30:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.895 7fda7ad42700 1 --2- 172.21.2.101:0/672044825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fda740aa8d0 0x7fda740a7b20 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7fda6c009980 tx=0x7fda6c00bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.896 7fda6affd700 1 -- 172.21.2.101:0/672044825 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fda6c005870 con 0x7fda740aa8d0 2021-02-10T11:30:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.896 7fda6affd700 1 -- 172.21.2.101:0/672044825 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6c005a70 con 0x7fda740aa8d0 2021-02-10T11:30:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.896 7fda6affd700 1 -- 172.21.2.101:0/672044825 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fda6c006030 con 0x7fda740aa8d0 2021-02-10T11:30:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.896 7fda7c725700 1 -- 172.21.2.101:0/672044825 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda740a8360 con 0x7fda740aa8d0 2021-02-10T11:30:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.896 7fda7c725700 1 -- 172.21.2.101:0/672044825 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda740a8360 con 0x7fda740aa8d0 2021-02-10T11:30:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.896 7fda6affd700 1 -- 172.21.2.101:0/672044825 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fda6c005c50 con 0x7fda740aa8d0 2021-02-10T11:30:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.896 7fda6affd700 1 --2- 172.21.2.101:0/672044825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fda5c0186c0 0x7fda5c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.896 7fda6affd700 1 -- 172.21.2.101:0/672044825 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fda6c032940 con 0x7fda740aa8d0 2021-02-10T11:30:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.896 7fda7a541700 1 --2- 172.21.2.101:0/672044825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fda5c0186c0 0x7fda5c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.897 7fda7a541700 1 --2- 172.21.2.101:0/672044825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fda5c0186c0 0x7fda5c01abc0 crc :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:25.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.926 7fda7c725700 1 -- 172.21.2.101:0/672044825 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda7407e6b0 con 0x7fda740aa8d0 2021-02-10T11:30:25.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:25.929 7fda6affd700 1 -- 172.21.2.101:0/672044825 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fda6c00be10 con 0x7fda740aa8d0 2021-02-10T11:30:26.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.056 7fda7c725700 1 -- 172.21.2.101:0/672044825 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839280", "target": ["mgr", ""]}) v1 -- 0x7fda7407e6b0 con 0x7fda5c0186c0 2021-02-10T11:30:26.071 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:30:26.074 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:30:26.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.066 7fda6affd700 1 -- 172.21.2.101:0/672044825 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fda64005540 con 0x7fda5c0186c0 2021-02-10T11:30:26.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.074 7fda7c725700 1 -- 172.21.2.101:0/672044825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fda5c0186c0 msgr2=0x7fda5c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:26.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.074 7fda7c725700 1 --2- 172.21.2.101:0/672044825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fda5c0186c0 0x7fda5c01abc0 crc :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:26.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.074 7fda7c725700 1 -- 172.21.2.101:0/672044825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fda740aa8d0 msgr2=0x7fda740a7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:26.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.074 7fda7c725700 1 --2- 172.21.2.101:0/672044825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fda740aa8d0 0x7fda740a7b20 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7fda6c009980 tx=0x7fda6c00bcd0).stop 2021-02-10T11:30:26.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.074 7fda7c725700 1 -- 172.21.2.101:0/672044825 shutdown_connections 2021-02-10T11:30:26.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.074 7fda7c725700 1 --2- 172.21.2.101:0/672044825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fda740aa8d0 0x7fda740a7b20 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:26.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.074 7fda7c725700 1 --2- 172.21.2.101:0/672044825 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fda740a9eb0 0x7fda740aa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:26.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.074 7fda7c725700 1 --2- 172.21.2.101:0/672044825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fda5c0186c0 0x7fda5c01abc0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:26.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.074 7fda7c725700 1 --2- 172.21.2.101:0/672044825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fda740abfc0 0x7fda740a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:26.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.074 7fda7c725700 1 -- 172.21.2.101:0/672044825 >> 172.21.2.101:0/672044825 conn(0x7fda740a6930 msgr2=0x7fda740c1230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:26.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.074 7fda7c725700 1 -- 172.21.2.101:0/672044825 shutdown_connections 2021-02-10T11:30:26.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.074 7fda7c725700 1 -- 172.21.2.101:0/672044825 wait complete. 2021-02-10T11:30:26.095 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839281 2021-02-10T11:30:26.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.481 7f1312e95700 1 Processor -- start 2021-02-10T11:30:26.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.481 7f1312e95700 1 -- start start 2021-02-10T11:30:26.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.481 7f1312e95700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f130c1b0470 0x7f130c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:26.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.481 7f1312e95700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f130c0ef610 0x7f130c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:26.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.481 7f1312e95700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f130c0f0b50 con 0x7f130c0f0030 2021-02-10T11:30:26.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.481 7f1312e95700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f130c0e52e0 con 0x7f130c0ef610 2021-02-10T11:30:26.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.481 7f1312e95700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f130c0e5490 con 0x7f130c1b0470 2021-02-10T11:30:26.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.481 7f130bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f130c0ef610 0x7f130c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:26.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.481 7f130bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f130c0ef610 0x7f130c0efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48994/0 (socket says 172.21.2.101:48994) 2021-02-10T11:30:26.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.481 7f130bfff700 1 -- 172.21.2.101:0/2273255457 learned_addr learned my addr 172.21.2.101:0/2273255457 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:26.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.482 7f130bfff700 1 -- 172.21.2.101:0/2273255457 >> v2:172.21.2.108:3300/0 conn(0x7f130c1b0470 msgr2=0x7f130c1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:26.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.482 7f130bfff700 1 --2- 172.21.2.101:0/2273255457 >> v2:172.21.2.108:3300/0 conn(0x7f130c1b0470 0x7f130c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:26.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.482 7f130bfff700 1 -- 172.21.2.101:0/2273255457 >> v1:172.21.2.108:6789/0 conn(0x7f130c0f0030 legacy=0x7f130c0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:26.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.482 7f130bfff700 1 -- 172.21.2.101:0/2273255457 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130c0f0430 con 0x7f130c0ef610 2021-02-10T11:30:26.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.482 7f130bfff700 1 --2- 172.21.2.101:0/2273255457 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f130c0ef610 0x7f130c0efa10 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7f12fc006a80 tx=0x7f12fc009f00).ready entity=mon.0 client_cookie=b534787cbad59232 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:26.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.482 7f130b7fe700 1 -- 172.21.2.101:0/2273255457 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f12fc00b5c0 con 0x7f130c0ef610 2021-02-10T11:30:26.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.482 7f130b7fe700 1 -- 172.21.2.101:0/2273255457 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12fc00b8c0 con 0x7f130c0ef610 2021-02-10T11:30:26.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.482 7f130b7fe700 1 -- 172.21.2.101:0/2273255457 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f12fc00bda0 con 0x7f130c0ef610 2021-02-10T11:30:26.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.482 7f1312e95700 1 -- 172.21.2.101:0/2273255457 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f130c0ef610 msgr2=0x7f130c0efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:26.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.482 7f1312e95700 1 --2- 172.21.2.101:0/2273255457 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f130c0ef610 0x7f130c0efa10 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7f12fc006a80 tx=0x7f12fc009f00).stop 2021-02-10T11:30:26.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.484 7f1312e95700 1 -- 172.21.2.101:0/2273255457 shutdown_connections 2021-02-10T11:30:26.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.484 7f1312e95700 1 --2- 172.21.2.101:0/2273255457 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f130c0ef610 0x7f130c0efa10 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:26.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.484 7f1312e95700 1 --2- 172.21.2.101:0/2273255457 >> v2:172.21.2.108:3300/0 conn(0x7f130c1b0470 0x7f130c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:26.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.484 7f1312e95700 1 -- 172.21.2.101:0/2273255457 >> 172.21.2.101:0/2273255457 conn(0x7f130c0ce3e0 msgr2=0x7f130c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:26.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.484 7f1312e95700 1 -- 172.21.2.101:0/2273255457 shutdown_connections 2021-02-10T11:30:26.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.484 7f1312e95700 1 -- 172.21.2.101:0/2273255457 wait complete. 2021-02-10T11:30:26.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.498 7f1312e95700 1 Processor -- start 2021-02-10T11:30:26.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.498 7f1312e95700 1 -- start start 2021-02-10T11:30:26.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.499 7f1312e95700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f130c0a6c90 0x7f130c1924c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:26.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.499 7f1312e95700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f130c192ae0 0x7f130c192ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:26.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.499 7f1312e95700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f130c0919e0 0x7f130c091de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:26.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.499 7f1312e95700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c19d2e0 con 0x7f130c0919e0 2021-02-10T11:30:26.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.499 7f1312e95700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c0914d0 con 0x7f130c0a6c90 2021-02-10T11:30:26.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.499 7f1312e95700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130c0a6820 con 0x7f130c192ae0 2021-02-10T11:30:26.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.499 7f13114b2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f130c0919e0 0x7f130c091de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:26.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.499 7f13114b2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f130c0919e0 0x7f130c091de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:48998/0 (socket says 172.21.2.101:48998) 2021-02-10T11:30:26.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.499 7f13114b2700 1 -- 172.21.2.101:0/536406137 learned_addr learned my addr 172.21.2.101:0/536406137 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:26.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.499 7f130bfff700 1 --2- 172.21.2.101:0/536406137 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f130c192ae0 0x7f130c192ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:26.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.499 7f1310cb1700 1 --2- 172.21.2.101:0/536406137 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f130c0a6c90 0x7f130c1924c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:26.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.499 7f13114b2700 1 -- 172.21.2.101:0/536406137 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f130c192ae0 msgr2=0x7f130c192ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:26.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.499 7f13114b2700 1 --2- 172.21.2.101:0/536406137 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f130c192ae0 0x7f130c192ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:26.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.499 7f13114b2700 1 -- 172.21.2.101:0/536406137 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f130c0a6c90 msgr2=0x7f130c1924c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:26.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.499 7f13114b2700 1 --2- 172.21.2.101:0/536406137 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f130c0a6c90 0x7f130c1924c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:26.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.499 7f13114b2700 1 -- 172.21.2.101:0/536406137 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130c092400 con 0x7f130c0919e0 2021-02-10T11:30:26.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.501 7f13114b2700 1 --2- 172.21.2.101:0/536406137 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f130c0919e0 0x7f130c091de0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7f1304006ed0 tx=0x7f130400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:26.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.501 7f13097fa700 1 -- 172.21.2.101:0/536406137 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f130400bd50 con 0x7f130c0919e0 2021-02-10T11:30:26.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.501 7f13097fa700 1 -- 172.21.2.101:0/536406137 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13040020d0 con 0x7f130c0919e0 2021-02-10T11:30:26.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.501 7f1312e95700 1 -- 172.21.2.101:0/536406137 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130c092400 con 0x7f130c0919e0 2021-02-10T11:30:26.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.501 7f13097fa700 1 -- 172.21.2.101:0/536406137 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f13040024a0 con 0x7f130c0919e0 2021-02-10T11:30:26.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.501 7f1312e95700 1 -- 172.21.2.101:0/536406137 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130c092990 con 0x7f130c0919e0 2021-02-10T11:30:26.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.501 7f13097fa700 1 -- 172.21.2.101:0/536406137 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f13040022b0 con 0x7f130c0919e0 2021-02-10T11:30:26.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.501 7f13097fa700 1 --2- 172.21.2.101:0/536406137 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f12f0018730 0x7f12f001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:26.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.501 7f13097fa700 1 -- 172.21.2.101:0/536406137 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1304032670 con 0x7f130c0919e0 2021-02-10T11:30:26.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.501 7f1310cb1700 1 --2- 172.21.2.101:0/536406137 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f12f0018730 0x7f12f001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:26.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.501 7f1310cb1700 1 --2- 172.21.2.101:0/536406137 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f12f0018730 0x7f12f001abc0 crc :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:26.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.532 7f1312e95700 1 -- 172.21.2.101:0/536406137 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f130c0e52e0 con 0x7f130c0919e0 2021-02-10T11:30:26.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.534 7f13097fa700 1 -- 172.21.2.101:0/536406137 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f13040009b0 con 0x7f130c0919e0 2021-02-10T11:30:26.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.661 7f1312e95700 1 -- 172.21.2.101:0/536406137 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839281", "target": ["mgr", ""]}) v1 -- 0x7f130c092990 con 0x7f12f0018730 2021-02-10T11:30:26.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.671 7f13097fa700 1 -- 172.21.2.101:0/536406137 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f12f4006550 con 0x7f12f0018730 2021-02-10T11:30:26.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.678 7f1312e95700 1 -- 172.21.2.101:0/536406137 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f12f0018730 msgr2=0x7f12f001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:26.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.678 7f1312e95700 1 --2- 172.21.2.101:0/536406137 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f12f0018730 0x7f12f001abc0 crc :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:26.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.678 7f1312e95700 1 -- 172.21.2.101:0/536406137 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f130c0919e0 msgr2=0x7f130c091de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:26.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.678 7f1312e95700 1 --2- 172.21.2.101:0/536406137 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f130c0919e0 0x7f130c091de0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7f1304006ed0 tx=0x7f130400b960).stop 2021-02-10T11:30:26.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.678 7f1312e95700 1 -- 172.21.2.101:0/536406137 shutdown_connections 2021-02-10T11:30:26.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.678 7f1312e95700 1 --2- 172.21.2.101:0/536406137 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f12f0018730 0x7f12f001abc0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:26.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.678 7f1312e95700 1 --2- 172.21.2.101:0/536406137 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f130c192ae0 0x7f130c192ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:26.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.678 7f1312e95700 1 --2- 172.21.2.101:0/536406137 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f130c0919e0 0x7f130c091de0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:26.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.678 7f1312e95700 1 --2- 172.21.2.101:0/536406137 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f130c0a6c90 0x7f130c1924c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:26.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.678 7f1312e95700 1 -- 172.21.2.101:0/536406137 >> 172.21.2.101:0/536406137 conn(0x7f130c0ac550 msgr2=0x7f130c0be1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:26.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.678 7f1312e95700 1 -- 172.21.2.101:0/536406137 shutdown_connections 2021-02-10T11:30:26.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:26.678 7f1312e95700 1 -- 172.21.2.101:0/536406137 wait complete. 2021-02-10T11:30:26.698 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839282 2021-02-10T11:30:27.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.099 7f8697722700 1 Processor -- start 2021-02-10T11:30:27.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.099 7f8697722700 1 -- start start 2021-02-10T11:30:27.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.100 7f8697722700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f86901b0470 0x7f86901b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:27.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.100 7f8697722700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86900ef610 0x7f86900efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:27.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.100 7f8697722700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86900f0b50 con 0x7f86900f0030 2021-02-10T11:30:27.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.100 7f8697722700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86900e52e0 con 0x7f86901b0470 2021-02-10T11:30:27.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.100 7f8697722700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86900e54d0 con 0x7f86900ef610 2021-02-10T11:30:27.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.100 7f8694d3d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86900ef610 0x7f86900efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:27.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.100 7f869553e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f86901b0470 0x7f86901b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:27.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.100 7f8694d3d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86900ef610 0x7f86900efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:41736/0 (socket says 172.21.2.101:41736) 2021-02-10T11:30:27.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.100 7f869553e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f86901b0470 0x7f86901b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:59490/0 (socket says 172.21.2.101:59490) 2021-02-10T11:30:27.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.100 7f8694d3d700 1 -- 172.21.2.101:0/352301118 learned_addr learned my addr 172.21.2.101:0/352301118 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:27.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.100 7f8695d3f700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f86900f0030 0x7f86900f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:42910/0 (socket says 172.21.2.101:42910) 2021-02-10T11:30:27.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.101 7f8694d3d700 1 -- 172.21.2.101:0/352301118 >> v2:172.21.2.108:3300/0 conn(0x7f86901b0470 msgr2=0x7f86901b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:27.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.101 7f8694d3d700 1 --2- 172.21.2.101:0/352301118 >> v2:172.21.2.108:3300/0 conn(0x7f86901b0470 0x7f86901b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:27.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.101 7f8694d3d700 1 -- 172.21.2.101:0/352301118 >> v1:172.21.2.108:6789/0 conn(0x7f86900f0030 legacy=0x7f86900f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:27.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.101 7f8694d3d700 1 -- 172.21.2.101:0/352301118 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86900f0430 con 0x7f86900ef610 2021-02-10T11:30:27.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.101 7f8694d3d700 1 --2- 172.21.2.101:0/352301118 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86900ef610 0x7f86900efa10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f8680006700 tx=0x7f8680009b80).ready entity=mon.2 client_cookie=c9c03ae0afb545d4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:27.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.101 7f8687fff700 1 -- 172.21.2.101:0/352301118 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f868000c0c0 con 0x7f86900ef610 2021-02-10T11:30:27.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.101 7f8687fff700 1 -- 172.21.2.101:0/352301118 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868000c340 con 0x7f86900ef610 2021-02-10T11:30:27.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.101 7f8687fff700 1 -- 172.21.2.101:0/352301118 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f868000c860 con 0x7f86900ef610 2021-02-10T11:30:27.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.101 7f8697722700 1 -- 172.21.2.101:0/352301118 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86900ef610 msgr2=0x7f86900efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:27.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.101 7f8697722700 1 --2- 172.21.2.101:0/352301118 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86900ef610 0x7f86900efa10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f8680006700 tx=0x7f8680009b80).stop 2021-02-10T11:30:27.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.103 7f8697722700 1 -- 172.21.2.101:0/352301118 shutdown_connections 2021-02-10T11:30:27.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.103 7f8697722700 1 --2- 172.21.2.101:0/352301118 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86900ef610 0x7f86900efa10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:27.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.103 7f8697722700 1 --2- 172.21.2.101:0/352301118 >> v2:172.21.2.108:3300/0 conn(0x7f86901b0470 0x7f86901b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:27.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.103 7f8697722700 1 -- 172.21.2.101:0/352301118 >> 172.21.2.101:0/352301118 conn(0x7f86900ce3e0 msgr2=0x7f86900e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:27.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.103 7f8697722700 1 -- 172.21.2.101:0/352301118 shutdown_connections 2021-02-10T11:30:27.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.103 7f8697722700 1 -- 172.21.2.101:0/352301118 wait complete. 2021-02-10T11:30:27.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.117 7f8697722700 1 Processor -- start 2021-02-10T11:30:27.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.117 7f8697722700 1 -- start start 2021-02-10T11:30:27.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.118 7f8697722700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8690190920 0x7f86900b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:27.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.118 7f8697722700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86900b8200 0x7f86900b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:27.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.118 7f8697722700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86900c1d80 0x7f86900c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:27.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.118 7f8697722700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8690190da0 con 0x7f86900c1d80 2021-02-10T11:30:27.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.118 7f8697722700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86900b8c20 con 0x7f86900b8200 2021-02-10T11:30:27.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.118 7f8697722700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86900b8e10 con 0x7f8690190920 2021-02-10T11:30:27.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.118 7f8695d3f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86900c1d80 0x7f86900c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:27.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.118 7f8695d3f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86900c1d80 0x7f86900c2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49012/0 (socket says 172.21.2.101:49012) 2021-02-10T11:30:27.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.118 7f8695d3f700 1 -- 172.21.2.101:0/2528608793 learned_addr learned my addr 172.21.2.101:0/2528608793 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:27.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.118 7f869553e700 1 --2- 172.21.2.101:0/2528608793 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8690190920 0x7f86900b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:27.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.118 7f8694d3d700 1 --2- 172.21.2.101:0/2528608793 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86900b8200 0x7f86900b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:27.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.118 7f869553e700 1 -- 172.21.2.101:0/2528608793 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86900b8200 msgr2=0x7f86900b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:27.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.118 7f869553e700 1 --2- 172.21.2.101:0/2528608793 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86900b8200 0x7f86900b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:27.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.118 7f869553e700 1 -- 172.21.2.101:0/2528608793 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86900c1d80 msgr2=0x7f86900c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:27.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.118 7f869553e700 1 --2- 172.21.2.101:0/2528608793 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86900c1d80 0x7f86900c2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:27.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.118 7f869553e700 1 -- 172.21.2.101:0/2528608793 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86900c27d0 con 0x7f8690190920 2021-02-10T11:30:27.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.118 7f8695d3f700 1 --2- 172.21.2.101:0/2528608793 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86900c1d80 0x7f86900c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:27.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.119 7f869553e700 1 --2- 172.21.2.101:0/2528608793 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8690190920 0x7f86900b7be0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f868c009540 tx=0x7f868c00b8e0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:27.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.119 7f8685ffb700 1 -- 172.21.2.101:0/2528608793 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f868c00bcd0 con 0x7f8690190920 2021-02-10T11:30:27.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.119 7f8685ffb700 1 -- 172.21.2.101:0/2528608793 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c006740 con 0x7f8690190920 2021-02-10T11:30:27.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.119 7f8685ffb700 1 -- 172.21.2.101:0/2528608793 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f868c002030 con 0x7f8690190920 2021-02-10T11:30:27.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.119 7f8697722700 1 -- 172.21.2.101:0/2528608793 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86900c27d0 con 0x7f8690190920 2021-02-10T11:30:27.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.119 7f8697722700 1 -- 172.21.2.101:0/2528608793 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86900c27d0 con 0x7f8690190920 2021-02-10T11:30:27.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.120 7f8685ffb700 1 -- 172.21.2.101:0/2528608793 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f868c000c90 con 0x7f8690190920 2021-02-10T11:30:27.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.120 7f8685ffb700 1 --2- 172.21.2.101:0/2528608793 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86700186c0 0x7f867001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:27.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.120 7f8685ffb700 1 -- 172.21.2.101:0/2528608793 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f868c0322c0 con 0x7f8690190920 2021-02-10T11:30:27.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.120 7f8694d3d700 1 --2- 172.21.2.101:0/2528608793 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86700186c0 0x7f867001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:27.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.120 7f8694d3d700 1 --2- 172.21.2.101:0/2528608793 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86700186c0 0x7f867001abc0 crc :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:27.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.149 7f8697722700 1 -- 172.21.2.101:0/2528608793 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8690105ae0 con 0x7f8690190920 2021-02-10T11:30:27.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.154 7f8685ffb700 1 -- 172.21.2.101:0/2528608793 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f868c006ae0 con 0x7f8690190920 2021-02-10T11:30:27.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.286 7f8697722700 1 -- 172.21.2.101:0/2528608793 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839282", "target": ["mgr", ""]}) v1 -- 0x7f86900f0b50 con 0x7f86700186c0 2021-02-10T11:30:27.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.297 7f8685ffb700 1 -- 172.21.2.101:0/2528608793 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f8680006430 con 0x7f86700186c0 2021-02-10T11:30:27.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.303 7f8697722700 1 -- 172.21.2.101:0/2528608793 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86700186c0 msgr2=0x7f867001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:27.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.303 7f8697722700 1 --2- 172.21.2.101:0/2528608793 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86700186c0 0x7f867001abc0 crc :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:27.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.303 7f8697722700 1 -- 172.21.2.101:0/2528608793 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8690190920 msgr2=0x7f86900b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:27.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.303 7f8697722700 1 --2- 172.21.2.101:0/2528608793 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8690190920 0x7f86900b7be0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f868c009540 tx=0x7f868c00b8e0).stop 2021-02-10T11:30:27.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.303 7f8697722700 1 -- 172.21.2.101:0/2528608793 shutdown_connections 2021-02-10T11:30:27.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.303 7f8697722700 1 --2- 172.21.2.101:0/2528608793 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86900c1d80 0x7f86900c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:27.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.303 7f8697722700 1 --2- 172.21.2.101:0/2528608793 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86900b8200 0x7f86900b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:27.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.303 7f8697722700 1 --2- 172.21.2.101:0/2528608793 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86700186c0 0x7f867001abc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:27.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.303 7f8697722700 1 --2- 172.21.2.101:0/2528608793 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8690190920 0x7f86900b7be0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:27.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.303 7f8697722700 1 -- 172.21.2.101:0/2528608793 >> 172.21.2.101:0/2528608793 conn(0x7f8690094e10 msgr2=0x7f86900a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:27.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.303 7f8697722700 1 -- 172.21.2.101:0/2528608793 shutdown_connections 2021-02-10T11:30:27.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.303 7f8697722700 1 -- 172.21.2.101:0/2528608793 wait complete. 2021-02-10T11:30:27.323 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839283 2021-02-10T11:30:27.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.723 7fcd1d58e700 1 Processor -- start 2021-02-10T11:30:27.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.723 7fcd1d58e700 1 -- start start 2021-02-10T11:30:27.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.723 7fcd1d58e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fcd181b0470 0x7fcd181b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:27.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.723 7fcd1d58e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcd180ef610 0x7fcd180efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:27.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.723 7fcd1d58e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd180f0030 0x7fcd180f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:27.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.723 7fcd1d58e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd180e5ea0 con 0x7fcd180ef610 2021-02-10T11:30:27.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.723 7fcd1d58e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd180f0b50 con 0x7fcd181b0470 2021-02-10T11:30:27.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.723 7fcd1d58e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd180e52e0 con 0x7fcd180f0030 2021-02-10T11:30:27.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.723 7fcd16abe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcd180ef610 0x7fcd180efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:27.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.724 7fcd16abe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcd180ef610 0x7fcd180efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49016/0 (socket says 172.21.2.101:49016) 2021-02-10T11:30:27.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.724 7fcd16abe700 1 -- 172.21.2.101:0/3343662203 learned_addr learned my addr 172.21.2.101:0/3343662203 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:27.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.724 7fcd17ac0700 1 --2- 172.21.2.101:0/3343662203 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd180f0030 0x7fcd180f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:27.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.724 7fcd172bf700 1 --2- 172.21.2.101:0/3343662203 >> v2:172.21.2.108:3300/0 conn(0x7fcd181b0470 0x7fcd181b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:27.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.724 7fcd16abe700 1 -- 172.21.2.101:0/3343662203 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd180f0030 msgr2=0x7fcd180f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:27.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.724 7fcd16abe700 1 --2- 172.21.2.101:0/3343662203 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd180f0030 0x7fcd180f0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:27.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.724 7fcd16abe700 1 -- 172.21.2.101:0/3343662203 >> v2:172.21.2.108:3300/0 conn(0x7fcd181b0470 msgr2=0x7fcd181b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:27.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.724 7fcd16abe700 1 --2- 172.21.2.101:0/3343662203 >> v2:172.21.2.108:3300/0 conn(0x7fcd181b0470 0x7fcd181b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:27.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.724 7fcd16abe700 1 -- 172.21.2.101:0/3343662203 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd180f10d0 con 0x7fcd180ef610 2021-02-10T11:30:27.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.724 7fcd16abe700 1 --2- 172.21.2.101:0/3343662203 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd180ef610 0x7fcd180efa10 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7fcd10006a80 tx=0x7fcd10009f00).ready entity=mon.0 client_cookie=2f312465c9d56b31 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:27.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.724 7fcd162bd700 1 -- 172.21.2.101:0/3343662203 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd1000b5c0 con 0x7fcd180ef610 2021-02-10T11:30:27.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.724 7fcd162bd700 1 -- 172.21.2.101:0/3343662203 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1000b8c0 con 0x7fcd180ef610 2021-02-10T11:30:27.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.724 7fcd162bd700 1 -- 172.21.2.101:0/3343662203 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd1000bda0 con 0x7fcd180ef610 2021-02-10T11:30:27.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.724 7fcd1d58e700 1 -- 172.21.2.101:0/3343662203 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd180ef610 msgr2=0x7fcd180efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:27.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.724 7fcd1d58e700 1 --2- 172.21.2.101:0/3343662203 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd180ef610 0x7fcd180efa10 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7fcd10006a80 tx=0x7fcd10009f00).stop 2021-02-10T11:30:27.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.726 7fcd1d58e700 1 -- 172.21.2.101:0/3343662203 shutdown_connections 2021-02-10T11:30:27.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.726 7fcd1d58e700 1 --2- 172.21.2.101:0/3343662203 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd180f0030 0x7fcd180f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:27.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.726 7fcd1d58e700 1 --2- 172.21.2.101:0/3343662203 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd180ef610 0x7fcd180efa10 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:27.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.726 7fcd1d58e700 1 --2- 172.21.2.101:0/3343662203 >> v2:172.21.2.108:3300/0 conn(0x7fcd181b0470 0x7fcd181b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:27.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.726 7fcd1d58e700 1 -- 172.21.2.101:0/3343662203 >> 172.21.2.101:0/3343662203 conn(0x7fcd180ce3e0 msgr2=0x7fcd180e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:27.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.726 7fcd1d58e700 1 -- 172.21.2.101:0/3343662203 shutdown_connections 2021-02-10T11:30:27.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.726 7fcd1d58e700 1 -- 172.21.2.101:0/3343662203 wait complete. 2021-02-10T11:30:27.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.739 7fcd1d58e700 1 Processor -- start 2021-02-10T11:30:27.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.739 7fcd1d58e700 1 -- start start 2021-02-10T11:30:27.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.740 7fcd1d58e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd1809e560 0x7fcd180a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:27.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.740 7fcd1d58e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd1809e960 0x7fcd180a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:27.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.740 7fcd1d58e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd180a0d70 0x7fcd180a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:27.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.740 7fcd1d58e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd180c38f0 con 0x7fcd180a0d70 2021-02-10T11:30:27.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.740 7fcd1d58e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd180a1170 con 0x7fcd1809e560 2021-02-10T11:30:27.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.740 7fcd1d58e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd180a1390 con 0x7fcd1809e960 2021-02-10T11:30:27.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.740 7fcd17ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd180a0d70 0x7fcd180a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:27.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.740 7fcd17ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd180a0d70 0x7fcd180a1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49022/0 (socket says 172.21.2.101:49022) 2021-02-10T11:30:27.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.740 7fcd17ac0700 1 -- 172.21.2.101:0/1354407395 learned_addr learned my addr 172.21.2.101:0/1354407395 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:27.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.740 7fcd16abe700 1 --2- 172.21.2.101:0/1354407395 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd1809e960 0x7fcd180a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:27.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.740 7fcd172bf700 1 --2- 172.21.2.101:0/1354407395 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd1809e560 0x7fcd180a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:27.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.741 7fcd17ac0700 1 -- 172.21.2.101:0/1354407395 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd1809e960 msgr2=0x7fcd180a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:27.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.741 7fcd17ac0700 1 --2- 172.21.2.101:0/1354407395 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd1809e960 0x7fcd180a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:27.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.741 7fcd17ac0700 1 -- 172.21.2.101:0/1354407395 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd1809e560 msgr2=0x7fcd180a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:27.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.741 7fcd17ac0700 1 --2- 172.21.2.101:0/1354407395 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd1809e560 0x7fcd180a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:27.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.741 7fcd17ac0700 1 -- 172.21.2.101:0/1354407395 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd180a2110 con 0x7fcd180a0d70 2021-02-10T11:30:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.741 7fcd16abe700 1 --2- 172.21.2.101:0/1354407395 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd1809e960 0x7fcd180a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.741 7fcd17ac0700 1 --2- 172.21.2.101:0/1354407395 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd180a0d70 0x7fcd180a1af0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7fcd08006f20 tx=0x7fcd0800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.741 7fccfffff700 1 -- 172.21.2.101:0/1354407395 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd08000c10 con 0x7fcd180a0d70 2021-02-10T11:30:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.741 7fccfffff700 1 -- 172.21.2.101:0/1354407395 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0800bd80 con 0x7fcd180a0d70 2021-02-10T11:30:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.741 7fccfffff700 1 -- 172.21.2.101:0/1354407395 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd080022b0 con 0x7fcd180a0d70 2021-02-10T11:30:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.741 7fcd1d58e700 1 -- 172.21.2.101:0/1354407395 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd180a2110 con 0x7fcd180a0d70 2021-02-10T11:30:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.741 7fcd1d58e700 1 -- 172.21.2.101:0/1354407395 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd180a2110 con 0x7fcd180a0d70 2021-02-10T11:30:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.743 7fccfffff700 1 -- 172.21.2.101:0/1354407395 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fcd08002020 con 0x7fcd180a0d70 2021-02-10T11:30:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.743 7fccfffff700 1 --2- 172.21.2.101:0/1354407395 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fccf80186c0 0x7fccf801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.743 7fccfffff700 1 -- 172.21.2.101:0/1354407395 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fcd08032570 con 0x7fcd180a0d70 2021-02-10T11:30:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.743 7fcd172bf700 1 --2- 172.21.2.101:0/1354407395 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fccf80186c0 0x7fccf801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:27.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.743 7fcd172bf700 1 --2- 172.21.2.101:0/1354407395 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fccf80186c0 0x7fccf801abc0 crc :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:27.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.774 7fcd1d58e700 1 -- 172.21.2.101:0/1354407395 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd18105b30 con 0x7fcd180a0d70 2021-02-10T11:30:27.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.776 7fccfffff700 1 -- 172.21.2.101:0/1354407395 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fcd0801cb80 con 0x7fcd180a0d70 2021-02-10T11:30:27.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.903 7fcd1d58e700 1 -- 172.21.2.101:0/1354407395 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839283", "target": ["mgr", ""]}) v1 -- 0x7fcd180f0b50 con 0x7fccf80186c0 2021-02-10T11:30:27.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.914 7fccfffff700 1 -- 172.21.2.101:0/1354407395 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fcd0c006a70 con 0x7fccf80186c0 2021-02-10T11:30:27.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.920 7fcd1d58e700 1 -- 172.21.2.101:0/1354407395 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fccf80186c0 msgr2=0x7fccf801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:27.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.920 7fcd1d58e700 1 --2- 172.21.2.101:0/1354407395 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fccf80186c0 0x7fccf801abc0 crc :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:27.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.920 7fcd1d58e700 1 -- 172.21.2.101:0/1354407395 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd180a0d70 msgr2=0x7fcd180a1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:27.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.920 7fcd1d58e700 1 --2- 172.21.2.101:0/1354407395 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd180a0d70 0x7fcd180a1af0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7fcd08006f20 tx=0x7fcd0800b990).stop 2021-02-10T11:30:27.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.920 7fcd1d58e700 1 -- 172.21.2.101:0/1354407395 shutdown_connections 2021-02-10T11:30:27.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.920 7fcd1d58e700 1 --2- 172.21.2.101:0/1354407395 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fccf80186c0 0x7fccf801abc0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:27.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.920 7fcd1d58e700 1 --2- 172.21.2.101:0/1354407395 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd1809e960 0x7fcd180a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:27.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.920 7fcd1d58e700 1 --2- 172.21.2.101:0/1354407395 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd180a0d70 0x7fcd180a1af0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:27.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.920 7fcd1d58e700 1 --2- 172.21.2.101:0/1354407395 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd1809e560 0x7fcd180a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:27.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.920 7fcd1d58e700 1 -- 172.21.2.101:0/1354407395 >> 172.21.2.101:0/1354407395 conn(0x7fcd1809fc10 msgr2=0x7fcd18094ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:27.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.920 7fcd1d58e700 1 -- 172.21.2.101:0/1354407395 shutdown_connections 2021-02-10T11:30:27.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:27.920 7fcd1d58e700 1 -- 172.21.2.101:0/1354407395 wait complete. 2021-02-10T11:30:27.940 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839284 2021-02-10T11:30:28.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.345 7f326a517700 1 Processor -- start 2021-02-10T11:30:28.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.345 7f326a517700 1 -- start start 2021-02-10T11:30:28.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.345 7f326a517700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f32640f05f0 0x7f32640f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:28.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.345 7f326a517700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32641b0c70 con 0x7f32641b0870 2021-02-10T11:30:28.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.345 7f326a517700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32640f0b50 con 0x7f32641b0470 2021-02-10T11:30:28.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.346 7f326a517700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32640e5ea0 con 0x7f32640f05f0 2021-02-10T11:30:28.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.346 7f3268b34700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f32640f05f0 0x7f32640f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:28.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.346 7f32637fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f32641b0870 0x7f32640efe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49524/0 (socket says 172.21.2.101:49524) 2021-02-10T11:30:28.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.346 7f32637fe700 1 -- 172.21.2.101:0/3231673593 learned_addr learned my addr 172.21.2.101:0/3231673593 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:28.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.346 7f3262ffd700 1 -- 172.21.2.101:0/3231673593 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709769338 0 0) 0x7f3258001180 con 0x7f32641b0870 2021-02-10T11:30:28.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.346 7f3268b34700 1 -- 172.21.2.101:0/3231673593 >> v1:172.21.2.108:6789/0 conn(0x7f32641b0470 legacy=0x7f32640ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:28.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.346 7f3268b34700 1 -- 172.21.2.101:0/3231673593 >> v1:172.21.2.101:6789/0 conn(0x7f32641b0870 legacy=0x7f32640efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:28.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.346 7f3268b34700 1 -- 172.21.2.101:0/3231673593 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32640e52e0 con 0x7f32640f05f0 2021-02-10T11:30:28.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.346 7f3268b34700 1 --2- 172.21.2.101:0/3231673593 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32640f05f0 0x7f32640f20e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f325c006e30 tx=0x7f325c009de0).ready entity=mon.0 client_cookie=ae9f21aa683cbb18 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:28.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.346 7f3262ffd700 1 -- 172.21.2.101:0/3231673593 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f325c00b300 con 0x7f32640f05f0 2021-02-10T11:30:28.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.346 7f3262ffd700 1 -- 172.21.2.101:0/3231673593 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325c00b580 con 0x7f32640f05f0 2021-02-10T11:30:28.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.346 7f3262ffd700 1 -- 172.21.2.101:0/3231673593 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f325c00baa0 con 0x7f32640f05f0 2021-02-10T11:30:28.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.346 7f326a517700 1 -- 172.21.2.101:0/3231673593 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32640f05f0 msgr2=0x7f32640f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:28.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.346 7f326a517700 1 --2- 172.21.2.101:0/3231673593 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32640f05f0 0x7f32640f20e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f325c006e30 tx=0x7f325c009de0).stop 2021-02-10T11:30:28.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.347 7f326a517700 1 -- 172.21.2.101:0/3231673593 shutdown_connections 2021-02-10T11:30:28.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.347 7f326a517700 1 --2- 172.21.2.101:0/3231673593 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32640f05f0 0x7f32640f20e0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:28.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.347 7f326a517700 1 -- 172.21.2.101:0/3231673593 >> 172.21.2.101:0/3231673593 conn(0x7f32640ce3e0 msgr2=0x7f32640e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:28.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.347 7f326a517700 1 -- 172.21.2.101:0/3231673593 shutdown_connections 2021-02-10T11:30:28.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.347 7f326a517700 1 -- 172.21.2.101:0/3231673593 wait complete. 2021-02-10T11:30:28.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.362 7f326a517700 1 Processor -- start 2021-02-10T11:30:28.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.362 7f326a517700 1 -- start start 2021-02-10T11:30:28.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f326a517700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32640a1dd0 0x7f3264190920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:28.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f326a517700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f326408fd30 0x7f32640effc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:28.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f326a517700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f32640f0e90 0x7f32640f1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:28.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f326a517700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32640f06f0 con 0x7f32640a1dd0 2021-02-10T11:30:28.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f326a517700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3264090160 con 0x7f32640f0e90 2021-02-10T11:30:28.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f326a517700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32640f18b0 con 0x7f326408fd30 2021-02-10T11:30:28.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f3263fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32640a1dd0 0x7f3264190920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:28.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f3263fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32640a1dd0 0x7f3264190920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49034/0 (socket says 172.21.2.101:49034) 2021-02-10T11:30:28.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f3263fff700 1 -- 172.21.2.101:0/4153907135 learned_addr learned my addr 172.21.2.101:0/4153907135 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:28.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f3268b34700 1 --2- 172.21.2.101:0/4153907135 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f32640f0e90 0x7f32640f1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:28.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f32637fe700 1 --2- 172.21.2.101:0/4153907135 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f326408fd30 0x7f32640effc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:28.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f3263fff700 1 -- 172.21.2.101:0/4153907135 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f326408fd30 msgr2=0x7f32640effc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:28.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f3263fff700 1 --2- 172.21.2.101:0/4153907135 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f326408fd30 0x7f32640effc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:28.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f3263fff700 1 -- 172.21.2.101:0/4153907135 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f32640f0e90 msgr2=0x7f32640f1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:28.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f3263fff700 1 --2- 172.21.2.101:0/4153907135 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f32640f0e90 0x7f32640f1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:28.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f3263fff700 1 -- 172.21.2.101:0/4153907135 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32640f1ad0 con 0x7f32640a1dd0 2021-02-10T11:30:28.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f3268b34700 1 --2- 172.21.2.101:0/4153907135 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f32640f0e90 0x7f32640f1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:30:28.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f3263fff700 1 --2- 172.21.2.101:0/4153907135 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32640a1dd0 0x7f3264190920 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7f3254006f20 tx=0x7f325400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:28.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f3260ff9700 1 -- 172.21.2.101:0/4153907135 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f325400bd80 con 0x7f32640a1dd0 2021-02-10T11:30:28.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f3260ff9700 1 -- 172.21.2.101:0/4153907135 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3254006030 con 0x7f32640a1dd0 2021-02-10T11:30:28.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f326a517700 1 -- 172.21.2.101:0/4153907135 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32640f1ad0 con 0x7f32640a1dd0 2021-02-10T11:30:28.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f326a517700 1 -- 172.21.2.101:0/4153907135 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32640f1ad0 con 0x7f32640a1dd0 2021-02-10T11:30:28.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.363 7f3260ff9700 1 -- 172.21.2.101:0/4153907135 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3254005520 con 0x7f32640a1dd0 2021-02-10T11:30:28.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.364 7f3260ff9700 1 -- 172.21.2.101:0/4153907135 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3254006210 con 0x7f32640a1dd0 2021-02-10T11:30:28.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.364 7f3260ff9700 1 --2- 172.21.2.101:0/4153907135 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3244018730 0x7f324401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:28.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.364 7f3260ff9700 1 -- 172.21.2.101:0/4153907135 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f32540327b0 con 0x7f32640a1dd0 2021-02-10T11:30:28.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.364 7f32637fe700 1 --2- 172.21.2.101:0/4153907135 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3244018730 0x7f324401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:28.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.364 7f32637fe700 1 --2- 172.21.2.101:0/4153907135 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3244018730 0x7f324401ac10 crc :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:28.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.394 7f326a517700 1 -- 172.21.2.101:0/4153907135 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3264090130 con 0x7f32640a1dd0 2021-02-10T11:30:28.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.397 7f3260ff9700 1 -- 172.21.2.101:0/4153907135 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3254006bb0 con 0x7f32640a1dd0 2021-02-10T11:30:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.524 7f326a517700 1 -- 172.21.2.101:0/4153907135 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839284", "target": ["mgr", ""]}) v1 -- 0x7f3264090130 con 0x7f3244018730 2021-02-10T11:30:28.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.535 7f3260ff9700 1 -- 172.21.2.101:0/4153907135 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f3258001180 con 0x7f3244018730 2021-02-10T11:30:28.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.541 7f326a517700 1 -- 172.21.2.101:0/4153907135 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3244018730 msgr2=0x7f324401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:28.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.541 7f326a517700 1 --2- 172.21.2.101:0/4153907135 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3244018730 0x7f324401ac10 crc :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:28.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.541 7f326a517700 1 -- 172.21.2.101:0/4153907135 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32640a1dd0 msgr2=0x7f3264190920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:28.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.541 7f326a517700 1 --2- 172.21.2.101:0/4153907135 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32640a1dd0 0x7f3264190920 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7f3254006f20 tx=0x7f325400b990).stop 2021-02-10T11:30:28.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.541 7f326a517700 1 -- 172.21.2.101:0/4153907135 shutdown_connections 2021-02-10T11:30:28.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.541 7f326a517700 1 --2- 172.21.2.101:0/4153907135 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3244018730 0x7f324401ac10 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:28.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.541 7f326a517700 1 --2- 172.21.2.101:0/4153907135 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f326408fd30 0x7f32640effc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:28.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.541 7f326a517700 1 --2- 172.21.2.101:0/4153907135 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f32640f0e90 0x7f32640f1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:28.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.541 7f326a517700 1 --2- 172.21.2.101:0/4153907135 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32640a1dd0 0x7f3264190920 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:28.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.541 7f326a517700 1 -- 172.21.2.101:0/4153907135 >> 172.21.2.101:0/4153907135 conn(0x7f32640958b0 msgr2=0x7f32640bf040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:28.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.541 7f326a517700 1 -- 172.21.2.101:0/4153907135 shutdown_connections 2021-02-10T11:30:28.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.541 7f326a517700 1 -- 172.21.2.101:0/4153907135 wait complete. 2021-02-10T11:30:28.561 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839285 2021-02-10T11:30:28.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.967 7fa7f2b80700 1 Processor -- start 2021-02-10T11:30:28.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.967 7fa7f2b80700 1 -- start start 2021-02-10T11:30:28.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.968 7fa7f2b80700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa7ec1b0870 0x7fa7ec0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:28.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.968 7fa7f2b80700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec0efe00 0x7fa7ec0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:28.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.968 7fa7f2b80700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7ec1b0c70 con 0x7fa7ec1b0470 2021-02-10T11:30:28.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.968 7fa7f2b80700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7ec0f0b50 con 0x7fa7ec1b0870 2021-02-10T11:30:28.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.968 7fa7f2b80700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec0f0820 con 0x7fa7ec0efe00 2021-02-10T11:30:28.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.968 7fa7f119d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec0efe00 0x7fa7ec0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:28.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.968 7fa7f119d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec0efe00 0x7fa7ec0f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:41778/0 (socket says 172.21.2.101:41778) 2021-02-10T11:30:28.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.968 7fa7f119d700 1 -- 172.21.2.101:0/2800531572 learned_addr learned my addr 172.21.2.101:0/2800531572 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:28.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.968 7fa7ebfff700 1 --2- 172.21.2.101:0/2800531572 >> v2:172.21.2.108:3300/0 conn(0x7fa7ec1b0870 0x7fa7ec0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:28.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.968 7fa7f119d700 1 -- 172.21.2.101:0/2800531572 >> v2:172.21.2.108:3300/0 conn(0x7fa7ec1b0870 msgr2=0x7fa7ec0f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:28.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.968 7fa7f119d700 1 --2- 172.21.2.101:0/2800531572 >> v2:172.21.2.108:3300/0 conn(0x7fa7ec1b0870 0x7fa7ec0f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:28.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.968 7fa7f119d700 1 -- 172.21.2.101:0/2800531572 >> v1:172.21.2.108:6789/0 conn(0x7fa7ec1b0470 legacy=0x7fa7ec0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:28.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.968 7fa7f119d700 1 -- 172.21.2.101:0/2800531572 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec0e52e0 con 0x7fa7ec0efe00 2021-02-10T11:30:28.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.969 7fa7f119d700 1 --2- 172.21.2.101:0/2800531572 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec0efe00 0x7fa7ec0f0200 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fa7e4006720 tx=0x7fa7e4009ba0).ready entity=mon.2 client_cookie=f347c34d8bb2e895 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:28.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.969 7fa7eb7fe700 1 -- 172.21.2.101:0/2800531572 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa7e40022e0 con 0x7fa7ec0efe00 2021-02-10T11:30:28.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.969 7fa7eb7fe700 1 -- 172.21.2.101:0/2800531572 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e4002620 con 0x7fa7ec0efe00 2021-02-10T11:30:28.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.969 7fa7eb7fe700 1 -- 172.21.2.101:0/2800531572 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa7e4002a80 con 0x7fa7ec0efe00 2021-02-10T11:30:28.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.969 7fa7f2b80700 1 -- 172.21.2.101:0/2800531572 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec0efe00 msgr2=0x7fa7ec0f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:28.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.969 7fa7f2b80700 1 --2- 172.21.2.101:0/2800531572 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec0efe00 0x7fa7ec0f0200 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fa7e4006720 tx=0x7fa7e4009ba0).stop 2021-02-10T11:30:28.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.971 7fa7f2b80700 1 -- 172.21.2.101:0/2800531572 shutdown_connections 2021-02-10T11:30:28.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.971 7fa7f2b80700 1 --2- 172.21.2.101:0/2800531572 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec0efe00 0x7fa7ec0f0200 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:28.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.971 7fa7f2b80700 1 --2- 172.21.2.101:0/2800531572 >> v2:172.21.2.108:3300/0 conn(0x7fa7ec1b0870 0x7fa7ec0f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:28.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.971 7fa7f2b80700 1 -- 172.21.2.101:0/2800531572 >> 172.21.2.101:0/2800531572 conn(0x7fa7ec0ce3e0 msgr2=0x7fa7ec0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:28.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.971 7fa7f2b80700 1 -- 172.21.2.101:0/2800531572 shutdown_connections 2021-02-10T11:30:28.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.971 7fa7f2b80700 1 -- 172.21.2.101:0/2800531572 wait complete. 2021-02-10T11:30:28.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.984 7fa7f2b80700 1 Processor -- start 2021-02-10T11:30:28.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.984 7fa7f2b80700 1 -- start start 2021-02-10T11:30:28.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.985 7fa7f2b80700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7ec0a96e0 0x7fa7ec19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:28.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.985 7fa7f2b80700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec19f630 0x7fa7ec0ba3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:28.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.985 7fa7f2b80700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7ec0baa00 0x7fa7ec0bae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:28.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.985 7fa7f2b80700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec0a9e00 con 0x7fa7ec0baa00 2021-02-10T11:30:28.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.985 7fa7f2b80700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec0a9b20 con 0x7fa7ec0a96e0 2021-02-10T11:30:28.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.985 7fa7f2b80700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec0b6a40 con 0x7fa7ec19f630 2021-02-10T11:30:28.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.985 7fa7f119d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7ec0baa00 0x7fa7ec0bae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:28.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.985 7fa7f119d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7ec0baa00 0x7fa7ec0bae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49048/0 (socket says 172.21.2.101:49048) 2021-02-10T11:30:28.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.985 7fa7f119d700 1 -- 172.21.2.101:0/1575351740 learned_addr learned my addr 172.21.2.101:0/1575351740 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:28.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.985 7fa7ebfff700 1 --2- 172.21.2.101:0/1575351740 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec19f630 0x7fa7ec0ba3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:28.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.985 7fa7f119d700 1 -- 172.21.2.101:0/1575351740 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec19f630 msgr2=0x7fa7ec0ba3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:28.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.985 7fa7f119d700 1 --2- 172.21.2.101:0/1575351740 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec19f630 0x7fa7ec0ba3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:28.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.985 7fa7f119d700 1 -- 172.21.2.101:0/1575351740 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7ec0a96e0 msgr2=0x7fa7ec19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:29.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.985 7fa7f099c700 1 --2- 172.21.2.101:0/1575351740 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7ec0a96e0 0x7fa7ec19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:29.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.986 7fa7f119d700 1 --2- 172.21.2.101:0/1575351740 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7ec0a96e0 0x7fa7ec19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:29.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.986 7fa7f119d700 1 -- 172.21.2.101:0/1575351740 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec0aa300 con 0x7fa7ec0baa00 2021-02-10T11:30:29.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.986 7fa7ebfff700 1 --2- 172.21.2.101:0/1575351740 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec19f630 0x7fa7ec0ba3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:30:29.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.986 7fa7f119d700 1 --2- 172.21.2.101:0/1575351740 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7ec0baa00 0x7fa7ec0bae00 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7fa7e40095c0 tx=0x7fa7e400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:29.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.986 7fa7f099c700 1 --2- 172.21.2.101:0/1575351740 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7ec0a96e0 0x7fa7ec19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:30:29.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.986 7fa7e97fa700 1 -- 172.21.2.101:0/1575351740 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa7e400bd80 con 0x7fa7ec0baa00 2021-02-10T11:30:29.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.986 7fa7e97fa700 1 -- 172.21.2.101:0/1575351740 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e4006240 con 0x7fa7ec0baa00 2021-02-10T11:30:29.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.986 7fa7e97fa700 1 -- 172.21.2.101:0/1575351740 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa7e40055b0 con 0x7fa7ec0baa00 2021-02-10T11:30:29.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.986 7fa7f2b80700 1 -- 172.21.2.101:0/1575351740 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7ec0aa300 con 0x7fa7ec0baa00 2021-02-10T11:30:29.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.986 7fa7f2b80700 1 -- 172.21.2.101:0/1575351740 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ec0aa300 con 0x7fa7ec0baa00 2021-02-10T11:30:29.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.987 7fa7e97fa700 1 -- 172.21.2.101:0/1575351740 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa7e4005330 con 0x7fa7ec0baa00 2021-02-10T11:30:29.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.987 7fa7e97fa700 1 --2- 172.21.2.101:0/1575351740 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7d00186c0 0x7fa7d001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:29.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.987 7fa7e97fa700 1 -- 172.21.2.101:0/1575351740 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa7e4032410 con 0x7fa7ec0baa00 2021-02-10T11:30:29.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.987 7fa7f099c700 1 --2- 172.21.2.101:0/1575351740 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7d00186c0 0x7fa7d001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:29.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:28.987 7fa7f099c700 1 --2- 172.21.2.101:0/1575351740 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7d00186c0 0x7fa7d001abc0 crc :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:29.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.017 7fa7f2b80700 1 -- 172.21.2.101:0/1575351740 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7ec105ae0 con 0x7fa7ec0baa00 2021-02-10T11:30:29.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.020 7fa7e97fa700 1 -- 172.21.2.101:0/1575351740 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa7e4025d30 con 0x7fa7ec0baa00 2021-02-10T11:30:29.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.145 7fa7f2b80700 1 -- 172.21.2.101:0/1575351740 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839285", "target": ["mgr", ""]}) v1 -- 0x7fa7ec0f0b50 con 0x7fa7d00186c0 2021-02-10T11:30:29.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.157 7fa7e97fa700 1 -- 172.21.2.101:0/1575351740 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fa7d4000960 con 0x7fa7d00186c0 2021-02-10T11:30:29.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.163 7fa7f2b80700 1 -- 172.21.2.101:0/1575351740 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7d00186c0 msgr2=0x7fa7d001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:29.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.163 7fa7f2b80700 1 --2- 172.21.2.101:0/1575351740 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7d00186c0 0x7fa7d001abc0 crc :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:29.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.163 7fa7f2b80700 1 -- 172.21.2.101:0/1575351740 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7ec0baa00 msgr2=0x7fa7ec0bae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:29.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.163 7fa7f2b80700 1 --2- 172.21.2.101:0/1575351740 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7ec0baa00 0x7fa7ec0bae00 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7fa7e40095c0 tx=0x7fa7e400b990).stop 2021-02-10T11:30:29.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.163 7fa7f2b80700 1 -- 172.21.2.101:0/1575351740 shutdown_connections 2021-02-10T11:30:29.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.163 7fa7f2b80700 1 --2- 172.21.2.101:0/1575351740 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7d00186c0 0x7fa7d001abc0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:29.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.163 7fa7f2b80700 1 --2- 172.21.2.101:0/1575351740 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec19f630 0x7fa7ec0ba3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:29.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.163 7fa7f2b80700 1 --2- 172.21.2.101:0/1575351740 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7ec0baa00 0x7fa7ec0bae00 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:29.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.163 7fa7f2b80700 1 --2- 172.21.2.101:0/1575351740 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7ec0a96e0 0x7fa7ec19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:29.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.163 7fa7f2b80700 1 -- 172.21.2.101:0/1575351740 >> 172.21.2.101:0/1575351740 conn(0x7fa7ec093b50 msgr2=0x7fa7ec0bb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:29.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.163 7fa7f2b80700 1 -- 172.21.2.101:0/1575351740 shutdown_connections 2021-02-10T11:30:29.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.163 7fa7f2b80700 1 -- 172.21.2.101:0/1575351740 wait complete. 2021-02-10T11:30:29.182 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839286 2021-02-10T11:30:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.587 7f5320d07700 1 Processor -- start 2021-02-10T11:30:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.587 7f5320d07700 1 -- start start 2021-02-10T11:30:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.587 7f5320d07700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f531c1b0870 0x7f531c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.587 7f5320d07700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f531c0efe00 0x7f531c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.587 7f5320d07700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f531c1b0c70 con 0x7f531c1b0470 2021-02-10T11:30:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.587 7f5320d07700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f531c0e5ea0 con 0x7f531c1b0870 2021-02-10T11:30:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.587 7f5320d07700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f531c0f0b50 con 0x7f531c0efe00 2021-02-10T11:30:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.588 7f531a312700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f531c1b0870 0x7f531c0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.588 7f531a312700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f531c1b0870 0x7f531c0f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49058/0 (socket says 172.21.2.101:49058) 2021-02-10T11:30:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.588 7f531a312700 1 -- 172.21.2.101:0/731220771 learned_addr learned my addr 172.21.2.101:0/731220771 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.588 7f531ab13700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f531c1b0470 0x7f531c0ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49550/0 (socket says 172.21.2.101:49550) 2021-02-10T11:30:29.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.588 7f531b314700 1 --2- 172.21.2.101:0/731220771 >> v2:172.21.2.108:3300/0 conn(0x7f531c0efe00 0x7f531c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:29.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.588 7f531a312700 1 -- 172.21.2.101:0/731220771 >> v2:172.21.2.108:3300/0 conn(0x7f531c0efe00 msgr2=0x7f531c0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:29.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.588 7f531a312700 1 --2- 172.21.2.101:0/731220771 >> v2:172.21.2.108:3300/0 conn(0x7f531c0efe00 0x7f531c0f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:29.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.588 7f531a312700 1 -- 172.21.2.101:0/731220771 >> v1:172.21.2.101:6789/0 conn(0x7f531c1b0470 legacy=0x7f531c0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:29.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.588 7f5319b11700 1 -- 172.21.2.101:0/731220771 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1099082168 0 0) 0x7f530c000ce0 con 0x7f531c1b0470 2021-02-10T11:30:29.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.588 7f531a312700 1 -- 172.21.2.101:0/731220771 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531c0f0820 con 0x7f531c1b0870 2021-02-10T11:30:29.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.588 7f531b314700 1 --2- 172.21.2.101:0/731220771 >> v2:172.21.2.108:3300/0 conn(0x7f531c0efe00 0x7f531c0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:29.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.588 7f531a312700 1 --2- 172.21.2.101:0/731220771 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f531c1b0870 0x7f531c0f20e0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7f5304006700 tx=0x7f5304009b80).ready entity=mon.0 client_cookie=a151f7de938b347b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:29.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.588 7f5319b11700 1 -- 172.21.2.101:0/731220771 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f530400c0c0 con 0x7f531c1b0870 2021-02-10T11:30:29.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.588 7f5319b11700 1 -- 172.21.2.101:0/731220771 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530400c440 con 0x7f531c1b0870 2021-02-10T11:30:29.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.588 7f5319b11700 1 -- 172.21.2.101:0/731220771 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f530400c920 con 0x7f531c1b0870 2021-02-10T11:30:29.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.589 7f5320d07700 1 -- 172.21.2.101:0/731220771 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f531c1b0870 msgr2=0x7f531c0f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:29.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.589 7f5320d07700 1 --2- 172.21.2.101:0/731220771 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f531c1b0870 0x7f531c0f20e0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7f5304006700 tx=0x7f5304009b80).stop 2021-02-10T11:30:29.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.589 7f5320d07700 1 -- 172.21.2.101:0/731220771 shutdown_connections 2021-02-10T11:30:29.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.589 7f5320d07700 1 --2- 172.21.2.101:0/731220771 >> v2:172.21.2.108:3300/0 conn(0x7f531c0efe00 0x7f531c0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:29.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.589 7f5320d07700 1 --2- 172.21.2.101:0/731220771 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f531c1b0870 0x7f531c0f20e0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:29.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.589 7f5320d07700 1 -- 172.21.2.101:0/731220771 >> 172.21.2.101:0/731220771 conn(0x7f531c0ce3e0 msgr2=0x7f531c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:29.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.589 7f5320d07700 1 -- 172.21.2.101:0/731220771 shutdown_connections 2021-02-10T11:30:29.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.589 7f5320d07700 1 -- 172.21.2.101:0/731220771 wait complete. 2021-02-10T11:30:29.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.604 7f5320d07700 1 Processor -- start 2021-02-10T11:30:29.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.604 7f5320d07700 1 -- start start 2021-02-10T11:30:29.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.604 7f5320d07700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f531c0a0420 0x7f531c0a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:29.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.604 7f5320d07700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f531c189d00 0x7f531c18a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:29.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.604 7f5320d07700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f531c18a720 0x7f531c0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:29.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.604 7f5320d07700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c189800 con 0x7f531c0a0420 2021-02-10T11:30:29.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.604 7f5320d07700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c18ab20 con 0x7f531c189d00 2021-02-10T11:30:29.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.604 7f5320d07700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531c0a1140 con 0x7f531c18a720 2021-02-10T11:30:29.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.604 7f531ab13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f531c0a0420 0x7f531c0a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:29.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.604 7f531ab13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f531c0a0420 0x7f531c0a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49062/0 (socket says 172.21.2.101:49062) 2021-02-10T11:30:29.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.604 7f531ab13700 1 -- 172.21.2.101:0/82746733 learned_addr learned my addr 172.21.2.101:0/82746733 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:29.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.605 7f531ab13700 1 -- 172.21.2.101:0/82746733 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f531c18a720 msgr2=0x7f531c0b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:29.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.605 7f531ab13700 1 --2- 172.21.2.101:0/82746733 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f531c18a720 0x7f531c0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:29.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.605 7f531ab13700 1 -- 172.21.2.101:0/82746733 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f531c189d00 msgr2=0x7f531c18a100 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:30:29.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.605 7f531ab13700 1 --2- 172.21.2.101:0/82746733 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f531c189d00 0x7f531c18a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:29.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.605 7f531ab13700 1 -- 172.21.2.101:0/82746733 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531c0b8200 con 0x7f531c0a0420 2021-02-10T11:30:29.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.605 7f531ab13700 1 --2- 172.21.2.101:0/82746733 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f531c0a0420 0x7f531c0a0820 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7f530c006f00 tx=0x7f530c00b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:29.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.605 7f53137fe700 1 -- 172.21.2.101:0/82746733 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f530c006020 con 0x7f531c0a0420 2021-02-10T11:30:29.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.605 7f53137fe700 1 -- 172.21.2.101:0/82746733 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530c0062a0 con 0x7f531c0a0420 2021-02-10T11:30:29.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.605 7f5320d07700 1 -- 172.21.2.101:0/82746733 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531c0b8200 con 0x7f531c0a0420 2021-02-10T11:30:29.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.605 7f5320d07700 1 -- 172.21.2.101:0/82746733 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531c0b8200 con 0x7f531c0a0420 2021-02-10T11:30:29.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.605 7f53137fe700 1 -- 172.21.2.101:0/82746733 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f530c004620 con 0x7f531c0a0420 2021-02-10T11:30:29.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.606 7f53137fe700 1 -- 172.21.2.101:0/82746733 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f530c00a030 con 0x7f531c0a0420 2021-02-10T11:30:29.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.606 7f53137fe700 1 --2- 172.21.2.101:0/82746733 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5300018730 0x7f530001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:29.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.606 7f53137fe700 1 -- 172.21.2.101:0/82746733 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f530c0327b0 con 0x7f531c0a0420 2021-02-10T11:30:29.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.606 7f531a312700 1 --2- 172.21.2.101:0/82746733 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5300018730 0x7f530001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:29.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.606 7f531a312700 1 --2- 172.21.2.101:0/82746733 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5300018730 0x7f530001ac10 crc :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:29.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.637 7f5320d07700 1 -- 172.21.2.101:0/82746733 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531c105ae0 con 0x7f531c0a0420 2021-02-10T11:30:29.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.639 7f53137fe700 1 -- 172.21.2.101:0/82746733 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f530c006b90 con 0x7f531c0a0420 2021-02-10T11:30:29.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.767 7f5320d07700 1 -- 172.21.2.101:0/82746733 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839286", "target": ["mgr", ""]}) v1 -- 0x7f531c18ab20 con 0x7f5300018730 2021-02-10T11:30:29.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.777 7f53137fe700 1 -- 172.21.2.101:0/82746733 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f53040095d0 con 0x7f5300018730 2021-02-10T11:30:29.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.784 7f5320d07700 1 -- 172.21.2.101:0/82746733 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5300018730 msgr2=0x7f530001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:29.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.784 7f5320d07700 1 --2- 172.21.2.101:0/82746733 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5300018730 0x7f530001ac10 crc :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:29.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.784 7f5320d07700 1 -- 172.21.2.101:0/82746733 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f531c0a0420 msgr2=0x7f531c0a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:29.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.784 7f5320d07700 1 --2- 172.21.2.101:0/82746733 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f531c0a0420 0x7f531c0a0820 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7f530c006f00 tx=0x7f530c00b940).stop 2021-02-10T11:30:29.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.785 7f5320d07700 1 -- 172.21.2.101:0/82746733 shutdown_connections 2021-02-10T11:30:29.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.785 7f5320d07700 1 --2- 172.21.2.101:0/82746733 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5300018730 0x7f530001ac10 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:29.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.785 7f5320d07700 1 --2- 172.21.2.101:0/82746733 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f531c18a720 0x7f531c0b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:29.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.785 7f5320d07700 1 --2- 172.21.2.101:0/82746733 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f531c189d00 0x7f531c18a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:29.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.785 7f5320d07700 1 --2- 172.21.2.101:0/82746733 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f531c0a0420 0x7f531c0a0820 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:29.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.785 7f5320d07700 1 -- 172.21.2.101:0/82746733 >> 172.21.2.101:0/82746733 conn(0x7f531c094e10 msgr2=0x7f531c0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:29.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.785 7f5320d07700 1 -- 172.21.2.101:0/82746733 shutdown_connections 2021-02-10T11:30:29.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:29.785 7f5320d07700 1 -- 172.21.2.101:0/82746733 wait complete. 2021-02-10T11:30:29.804 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839287 2021-02-10T11:30:30.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.205 7f2abdb65700 1 Processor -- start 2021-02-10T11:30:30.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.205 7f2abdb65700 1 -- start start 2021-02-10T11:30:30.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.205 7f2abdb65700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2ab81b0470 0x7f2ab81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:30.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.205 7f2abdb65700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2ab80ef610 0x7f2ab80efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:30.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.205 7f2abdb65700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab80f0b50 con 0x7f2ab80f0030 2021-02-10T11:30:30.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.205 7f2abdb65700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab80e52e0 con 0x7f2ab80ef610 2021-02-10T11:30:30.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.205 7f2abdb65700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab80e5490 con 0x7f2ab81b0470 2021-02-10T11:30:30.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.206 7f2ab6ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2ab80ef610 0x7f2ab80efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:30.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.206 7f2ab6ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2ab80ef610 0x7f2ab80efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49072/0 (socket says 172.21.2.101:49072) 2021-02-10T11:30:30.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.206 7f2ab6ffd700 1 -- 172.21.2.101:0/2464756694 learned_addr learned my addr 172.21.2.101:0/2464756694 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:30.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.206 7f2ab77fe700 1 --2- 172.21.2.101:0/2464756694 >> v2:172.21.2.108:3300/0 conn(0x7f2ab81b0470 0x7f2ab81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:30.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.206 7f2ab6ffd700 1 -- 172.21.2.101:0/2464756694 >> v2:172.21.2.108:3300/0 conn(0x7f2ab81b0470 msgr2=0x7f2ab81b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:30.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.206 7f2ab6ffd700 1 --2- 172.21.2.101:0/2464756694 >> v2:172.21.2.108:3300/0 conn(0x7f2ab81b0470 0x7f2ab81b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:30.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.206 7f2ab6ffd700 1 -- 172.21.2.101:0/2464756694 >> v1:172.21.2.108:6789/0 conn(0x7f2ab80f0030 legacy=0x7f2ab80f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:30.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.206 7f2ab6ffd700 1 -- 172.21.2.101:0/2464756694 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab80f0430 con 0x7f2ab80ef610 2021-02-10T11:30:30.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.206 7f2ab6ffd700 1 --2- 172.21.2.101:0/2464756694 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ab80ef610 0x7f2ab80efa10 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7f2aac006700 tx=0x7f2aac009b00).ready entity=mon.0 client_cookie=9a9ad177e9f35ed1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:30.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.206 7f2ab67fc700 1 -- 172.21.2.101:0/2464756694 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2aac00c0c0 con 0x7f2ab80ef610 2021-02-10T11:30:30.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.206 7f2ab67fc700 1 -- 172.21.2.101:0/2464756694 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aac00c340 con 0x7f2ab80ef610 2021-02-10T11:30:30.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.206 7f2ab67fc700 1 -- 172.21.2.101:0/2464756694 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2aac00c820 con 0x7f2ab80ef610 2021-02-10T11:30:30.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.206 7f2abdb65700 1 -- 172.21.2.101:0/2464756694 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ab80ef610 msgr2=0x7f2ab80efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:30.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.206 7f2abdb65700 1 --2- 172.21.2.101:0/2464756694 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ab80ef610 0x7f2ab80efa10 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7f2aac006700 tx=0x7f2aac009b00).stop 2021-02-10T11:30:30.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.207 7f2abdb65700 1 -- 172.21.2.101:0/2464756694 shutdown_connections 2021-02-10T11:30:30.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.207 7f2abdb65700 1 --2- 172.21.2.101:0/2464756694 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ab80ef610 0x7f2ab80efa10 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:30.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.207 7f2abdb65700 1 --2- 172.21.2.101:0/2464756694 >> v2:172.21.2.108:3300/0 conn(0x7f2ab81b0470 0x7f2ab81b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:30.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.207 7f2abdb65700 1 -- 172.21.2.101:0/2464756694 >> 172.21.2.101:0/2464756694 conn(0x7f2ab80ce3e0 msgr2=0x7f2ab80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:30.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.207 7f2abdb65700 1 -- 172.21.2.101:0/2464756694 shutdown_connections 2021-02-10T11:30:30.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.207 7f2abdb65700 1 -- 172.21.2.101:0/2464756694 wait complete. 2021-02-10T11:30:30.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.223 7f2abdb65700 1 Processor -- start 2021-02-10T11:30:30.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.223 7f2abdb65700 1 -- start start 2021-02-10T11:30:30.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.224 7f2abdb65700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ab8106ad0 0x7f2ab8106ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:30.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.224 7f2abdb65700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ab80b00f0 0x7f2ab80b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:30.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.224 7f2abdb65700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ab80b0b10 0x7f2ab80b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:30.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.224 7f2abdb65700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab80b1bd0 con 0x7f2ab8106ad0 2021-02-10T11:30:30.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.224 7f2abdb65700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab80b1df0 con 0x7f2ab80b00f0 2021-02-10T11:30:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.224 7f2abdb65700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab80b2010 con 0x7f2ab80b0b10 2021-02-10T11:30:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.224 7f2ab77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ab8106ad0 0x7f2ab8106ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.224 7f2ab77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ab8106ad0 0x7f2ab8106ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49076/0 (socket says 172.21.2.101:49076) 2021-02-10T11:30:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.224 7f2ab77fe700 1 -- 172.21.2.101:0/1364551817 learned_addr learned my addr 172.21.2.101:0/1364551817 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.224 7f2ab6ffd700 1 --2- 172.21.2.101:0/1364551817 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ab80b00f0 0x7f2ab80b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.224 7f2ab7fff700 1 --2- 172.21.2.101:0/1364551817 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ab80b0b10 0x7f2ab80b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.225 7f2ab77fe700 1 -- 172.21.2.101:0/1364551817 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ab80b0b10 msgr2=0x7f2ab80b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.225 7f2ab77fe700 1 --2- 172.21.2.101:0/1364551817 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ab80b0b10 0x7f2ab80b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.225 7f2ab77fe700 1 -- 172.21.2.101:0/1364551817 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ab80b00f0 msgr2=0x7f2ab80b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.225 7f2ab77fe700 1 --2- 172.21.2.101:0/1364551817 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ab80b00f0 0x7f2ab80b04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.225 7f2ab77fe700 1 -- 172.21.2.101:0/1364551817 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab80b2230 con 0x7f2ab8106ad0 2021-02-10T11:30:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.225 7f2ab77fe700 1 --2- 172.21.2.101:0/1364551817 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ab8106ad0 0x7f2ab8106ed0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f2ab00099e0 tx=0x7f2ab000bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.225 7f2a9ffff700 1 -- 172.21.2.101:0/1364551817 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2ab000c2f0 con 0x7f2ab8106ad0 2021-02-10T11:30:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.225 7f2a9ffff700 1 -- 172.21.2.101:0/1364551817 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab000c630 con 0x7f2ab8106ad0 2021-02-10T11:30:30.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.225 7f2a9ffff700 1 -- 172.21.2.101:0/1364551817 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2ab000cb10 con 0x7f2ab8106ad0 2021-02-10T11:30:30.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.225 7f2abdb65700 1 -- 172.21.2.101:0/1364551817 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab80b2230 con 0x7f2ab8106ad0 2021-02-10T11:30:30.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.225 7f2abdb65700 1 -- 172.21.2.101:0/1364551817 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab80b2230 con 0x7f2ab8106ad0 2021-02-10T11:30:30.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.226 7f2a9ffff700 1 -- 172.21.2.101:0/1364551817 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2ab0001010 con 0x7f2ab8106ad0 2021-02-10T11:30:30.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.226 7f2a9ffff700 1 --2- 172.21.2.101:0/1364551817 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a98018730 0x7f2a9801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:30.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.226 7f2a9ffff700 1 -- 172.21.2.101:0/1364551817 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2ab00329c0 con 0x7f2ab8106ad0 2021-02-10T11:30:30.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.226 7f2ab6ffd700 1 --2- 172.21.2.101:0/1364551817 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a98018730 0x7f2a9801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:30.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.226 7f2ab6ffd700 1 --2- 172.21.2.101:0/1364551817 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a98018730 0x7f2a9801ac10 crc :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:30.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.255 7f2abdb65700 1 -- 172.21.2.101:0/1364551817 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab8105ae0 con 0x7f2ab8106ad0 2021-02-10T11:30:30.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.259 7f2a9ffff700 1 -- 172.21.2.101:0/1364551817 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2ab000be30 con 0x7f2ab8106ad0 2021-02-10T11:30:30.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.389 7f2abdb65700 1 -- 172.21.2.101:0/1364551817 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839287", "target": ["mgr", ""]}) v1 -- 0x7f2ab80f0b50 con 0x7f2a98018730 2021-02-10T11:30:30.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.400 7f2a9ffff700 1 -- 172.21.2.101:0/1364551817 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f2aac006500 con 0x7f2a98018730 2021-02-10T11:30:30.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.405 7f2abdb65700 1 -- 172.21.2.101:0/1364551817 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a98018730 msgr2=0x7f2a9801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:30.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.405 7f2abdb65700 1 --2- 172.21.2.101:0/1364551817 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a98018730 0x7f2a9801ac10 crc :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:30.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.405 7f2abdb65700 1 -- 172.21.2.101:0/1364551817 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ab8106ad0 msgr2=0x7f2ab8106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:30.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.405 7f2abdb65700 1 --2- 172.21.2.101:0/1364551817 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ab8106ad0 0x7f2ab8106ed0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f2ab00099e0 tx=0x7f2ab000bcf0).stop 2021-02-10T11:30:30.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.406 7f2abdb65700 1 -- 172.21.2.101:0/1364551817 shutdown_connections 2021-02-10T11:30:30.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.406 7f2abdb65700 1 --2- 172.21.2.101:0/1364551817 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a98018730 0x7f2a9801ac10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:30.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.406 7f2abdb65700 1 --2- 172.21.2.101:0/1364551817 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ab80b0b10 0x7f2ab80b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:30.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.406 7f2abdb65700 1 --2- 172.21.2.101:0/1364551817 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ab80b00f0 0x7f2ab80b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:30.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.406 7f2abdb65700 1 --2- 172.21.2.101:0/1364551817 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ab8106ad0 0x7f2ab8106ed0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:30.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.406 7f2abdb65700 1 -- 172.21.2.101:0/1364551817 >> 172.21.2.101:0/1364551817 conn(0x7f2ab80bd060 msgr2=0x7f2ab809a020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:30.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.406 7f2abdb65700 1 -- 172.21.2.101:0/1364551817 shutdown_connections 2021-02-10T11:30:30.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.406 7f2abdb65700 1 -- 172.21.2.101:0/1364551817 wait complete. 2021-02-10T11:30:30.426 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839288 2021-02-10T11:30:30.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.829 7f7b5b891700 1 Processor -- start 2021-02-10T11:30:30.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.830 7f7b5b891700 1 -- start start 2021-02-10T11:30:30.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.830 7f7b5b891700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7b541b0470 0x7f7b541b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:30.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.830 7f7b5b891700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7b540f20e0 0x7f7b540f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:30.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.830 7f7b5b891700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b540f0b50 con 0x7f7b540ef610 2021-02-10T11:30:30.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.830 7f7b5b891700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b540f0820 con 0x7f7b540f20e0 2021-02-10T11:30:30.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.830 7f7b5b891700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b540f24e0 con 0x7f7b541b0470 2021-02-10T11:30:30.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.830 7f7b59eae700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7b540f20e0 0x7f7b540f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:30.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.830 7f7b59eae700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7b540f20e0 0x7f7b540f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49084/0 (socket says 172.21.2.101:49084) 2021-02-10T11:30:30.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.830 7f7b59eae700 1 -- 172.21.2.101:0/4124805268 learned_addr learned my addr 172.21.2.101:0/4124805268 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:30.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.830 7f7b596ad700 1 --2- 172.21.2.101:0/4124805268 >> v2:172.21.2.108:3300/0 conn(0x7f7b541b0470 0x7f7b541b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:30.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.830 7f7b59eae700 1 -- 172.21.2.101:0/4124805268 >> v2:172.21.2.108:3300/0 conn(0x7f7b541b0470 msgr2=0x7f7b541b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:30.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.830 7f7b59eae700 1 --2- 172.21.2.101:0/4124805268 >> v2:172.21.2.108:3300/0 conn(0x7f7b541b0470 0x7f7b541b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:30.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.830 7f7b59eae700 1 -- 172.21.2.101:0/4124805268 >> v1:172.21.2.101:6789/0 conn(0x7f7b540ef610 legacy=0x7f7b540efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:30.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.830 7f7b59eae700 1 -- 172.21.2.101:0/4124805268 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b540e52e0 con 0x7f7b540f20e0 2021-02-10T11:30:30.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.831 7f7b59eae700 1 --2- 172.21.2.101:0/4124805268 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b540f20e0 0x7f7b540f0200 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f7b4c006f00 tx=0x7f7b4c009e90).ready entity=mon.0 client_cookie=fa630d21905dc327 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:30.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.831 7f7b47fff700 1 -- 172.21.2.101:0/4124805268 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7b4c00b300 con 0x7f7b540f20e0 2021-02-10T11:30:30.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.831 7f7b47fff700 1 -- 172.21.2.101:0/4124805268 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4c00b580 con 0x7f7b540f20e0 2021-02-10T11:30:30.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.831 7f7b47fff700 1 -- 172.21.2.101:0/4124805268 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7b4c00b9d0 con 0x7f7b540f20e0 2021-02-10T11:30:30.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.831 7f7b5b891700 1 -- 172.21.2.101:0/4124805268 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b540f20e0 msgr2=0x7f7b540f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:30.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.831 7f7b5b891700 1 --2- 172.21.2.101:0/4124805268 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b540f20e0 0x7f7b540f0200 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f7b4c006f00 tx=0x7f7b4c009e90).stop 2021-02-10T11:30:30.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.831 7f7b5b891700 1 -- 172.21.2.101:0/4124805268 shutdown_connections 2021-02-10T11:30:30.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.831 7f7b5b891700 1 --2- 172.21.2.101:0/4124805268 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b540f20e0 0x7f7b540f0200 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:30.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.831 7f7b5b891700 1 --2- 172.21.2.101:0/4124805268 >> v2:172.21.2.108:3300/0 conn(0x7f7b541b0470 0x7f7b541b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:30.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.831 7f7b5b891700 1 -- 172.21.2.101:0/4124805268 >> 172.21.2.101:0/4124805268 conn(0x7f7b540ce3e0 msgr2=0x7f7b540e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:30.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.832 7f7b5b891700 1 -- 172.21.2.101:0/4124805268 shutdown_connections 2021-02-10T11:30:30.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.832 7f7b5b891700 1 -- 172.21.2.101:0/4124805268 wait complete. 2021-02-10T11:30:30.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.846 7f7b5b891700 1 Processor -- start 2021-02-10T11:30:30.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.846 7f7b5b891700 1 -- start start 2021-02-10T11:30:30.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.847 7f7b5b891700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b540c35c0 0x7f7b540a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:30.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.847 7f7b5b891700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7b540c39c0 0x7f7b540a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:30.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.847 7f7b5b891700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7b540a59c0 0x7f7b54108d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:30.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.847 7f7b5b891700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b541087f0 con 0x7f7b540c35c0 2021-02-10T11:30:30.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.847 7f7b5b891700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b54108550 con 0x7f7b540a59c0 2021-02-10T11:30:30.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.847 7f7b5b891700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b54109390 con 0x7f7b540c39c0 2021-02-10T11:30:30.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.847 7f7b596ad700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b540c35c0 0x7f7b540a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:30.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.847 7f7b596ad700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b540c35c0 0x7f7b540a4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49090/0 (socket says 172.21.2.101:49090) 2021-02-10T11:30:30.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.847 7f7b596ad700 1 -- 172.21.2.101:0/493219280 learned_addr learned my addr 172.21.2.101:0/493219280 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:30.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.847 7f7b58eac700 1 --2- 172.21.2.101:0/493219280 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7b540c39c0 0x7f7b540a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:30.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.847 7f7b59eae700 1 --2- 172.21.2.101:0/493219280 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7b540a59c0 0x7f7b54108d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:30.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.847 7f7b596ad700 1 -- 172.21.2.101:0/493219280 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7b540c39c0 msgr2=0x7f7b540a53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:30.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.847 7f7b596ad700 1 --2- 172.21.2.101:0/493219280 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7b540c39c0 0x7f7b540a53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:30.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.847 7f7b596ad700 1 -- 172.21.2.101:0/493219280 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7b540a59c0 msgr2=0x7f7b54108d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:30.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.847 7f7b596ad700 1 --2- 172.21.2.101:0/493219280 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7b540a59c0 0x7f7b54108d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:30.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.847 7f7b596ad700 1 -- 172.21.2.101:0/493219280 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b541095b0 con 0x7f7b540c35c0 2021-02-10T11:30:30.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.847 7f7b596ad700 1 --2- 172.21.2.101:0/493219280 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b540c35c0 0x7f7b540a4d80 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f7b480067e0 tx=0x7f7b4800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:30.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.848 7f7b45ffb700 1 -- 172.21.2.101:0/493219280 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7b48006900 con 0x7f7b540c35c0 2021-02-10T11:30:30.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.848 7f7b45ffb700 1 -- 172.21.2.101:0/493219280 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4800bd80 con 0x7f7b540c35c0 2021-02-10T11:30:30.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.848 7f7b45ffb700 1 -- 172.21.2.101:0/493219280 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7b48002030 con 0x7f7b540c35c0 2021-02-10T11:30:30.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.848 7f7b5b891700 1 -- 172.21.2.101:0/493219280 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b541095b0 con 0x7f7b540c35c0 2021-02-10T11:30:30.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.848 7f7b5b891700 1 -- 172.21.2.101:0/493219280 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b541095b0 con 0x7f7b540c35c0 2021-02-10T11:30:30.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.848 7f7b45ffb700 1 -- 172.21.2.101:0/493219280 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7b480068b0 con 0x7f7b540c35c0 2021-02-10T11:30:30.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.848 7f7b45ffb700 1 --2- 172.21.2.101:0/493219280 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7b340186d0 0x7f7b3401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:30.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.848 7f7b45ffb700 1 -- 172.21.2.101:0/493219280 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7b48032660 con 0x7f7b540c35c0 2021-02-10T11:30:30.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.848 7f7b58eac700 1 --2- 172.21.2.101:0/493219280 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7b340186d0 0x7f7b3401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:30.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.849 7f7b58eac700 1 --2- 172.21.2.101:0/493219280 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7b340186d0 0x7f7b3401abd0 crc :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:30.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.880 7f7b5b891700 1 -- 172.21.2.101:0/493219280 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b541057f0 con 0x7f7b540c35c0 2021-02-10T11:30:30.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:30.883 7f7b45ffb700 1 -- 172.21.2.101:0/493219280 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7b48000940 con 0x7f7b540c35c0 2021-02-10T11:30:31.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.009 7f7b5b891700 1 -- 172.21.2.101:0/493219280 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839288", "target": ["mgr", ""]}) v1 -- 0x7f7b54108550 con 0x7f7b340186d0 2021-02-10T11:30:31.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.019 7f7b45ffb700 1 -- 172.21.2.101:0/493219280 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f7b500054b0 con 0x7f7b340186d0 2021-02-10T11:30:31.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.026 7f7b5b891700 1 -- 172.21.2.101:0/493219280 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7b340186d0 msgr2=0x7f7b3401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:31.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.026 7f7b5b891700 1 --2- 172.21.2.101:0/493219280 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7b340186d0 0x7f7b3401abd0 crc :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:31.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.026 7f7b5b891700 1 -- 172.21.2.101:0/493219280 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b540c35c0 msgr2=0x7f7b540a4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:31.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.026 7f7b5b891700 1 --2- 172.21.2.101:0/493219280 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b540c35c0 0x7f7b540a4d80 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f7b480067e0 tx=0x7f7b4800b990).stop 2021-02-10T11:30:31.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.026 7f7b5b891700 1 -- 172.21.2.101:0/493219280 shutdown_connections 2021-02-10T11:30:31.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.026 7f7b5b891700 1 --2- 172.21.2.101:0/493219280 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7b340186d0 0x7f7b3401abd0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:31.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.026 7f7b5b891700 1 --2- 172.21.2.101:0/493219280 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7b540c39c0 0x7f7b540a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:31.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.026 7f7b5b891700 1 --2- 172.21.2.101:0/493219280 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7b540a59c0 0x7f7b54108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:31.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.026 7f7b5b891700 1 --2- 172.21.2.101:0/493219280 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b540c35c0 0x7f7b540a4d80 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:31.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.026 7f7b5b891700 1 -- 172.21.2.101:0/493219280 >> 172.21.2.101:0/493219280 conn(0x7f7b5409c0c0 msgr2=0x7f7b54105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:31.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.026 7f7b5b891700 1 -- 172.21.2.101:0/493219280 shutdown_connections 2021-02-10T11:30:31.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.026 7f7b5b891700 1 -- 172.21.2.101:0/493219280 wait complete. 2021-02-10T11:30:31.046 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839289 2021-02-10T11:30:31.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.450 7f7ffade9700 1 Processor -- start 2021-02-10T11:30:31.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.450 7f7ffade9700 1 -- start start 2021-02-10T11:30:31.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.450 7f7ffade9700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7ff41b0470 0x7f7ff41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:31.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.450 7f7ffade9700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff40f0b50 con 0x7f7ff40f20e0 2021-02-10T11:30:31.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.450 7f7ffade9700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff40f24e0 con 0x7f7ff40ef610 2021-02-10T11:30:31.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.450 7f7ffade9700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff40e5ea0 con 0x7f7ff41b0470 2021-02-10T11:30:31.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.451 7f7ff9406700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f7ff40f20e0 0x7f7ff40f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49594/0 (socket says 172.21.2.101:49594) 2021-02-10T11:30:31.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.451 7f7ff9406700 1 -- 172.21.2.101:0/1124751406 learned_addr learned my addr 172.21.2.101:0/1124751406 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:31.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.451 7f7ff8c05700 1 --2- 172.21.2.101:0/1124751406 >> v2:172.21.2.108:3300/0 conn(0x7f7ff41b0470 0x7f7ff41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:31.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.451 7f7ff37fe700 1 -- 172.21.2.101:0/1124751406 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2744582019 0 0) 0x7f7fec001180 con 0x7f7ff40f20e0 2021-02-10T11:30:31.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.451 7f7ff37fe700 1 -- 172.21.2.101:0/1124751406 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fe0002fe0 con 0x7f7ff40f20e0 2021-02-10T11:30:31.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.451 7f7ff37fe700 1 -- 172.21.2.101:0/1124751406 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 964428123 0 0) 0x7f7fec0010e0 con 0x7f7ff40f20e0 2021-02-10T11:30:31.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.451 7f7ff37fe700 1 -- 172.21.2.101:0/1124751406 >> v2:172.21.2.108:3300/0 conn(0x7f7ff41b0470 msgr2=0x7f7ff41b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:31.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.451 7f7ff37fe700 1 --2- 172.21.2.101:0/1124751406 >> v2:172.21.2.108:3300/0 conn(0x7f7ff41b0470 0x7f7ff41b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:31.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.451 7f7ff37fe700 1 -- 172.21.2.101:0/1124751406 >> v1:172.21.2.108:6789/0 conn(0x7f7ff40ef610 legacy=0x7f7ff40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:31.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.451 7f7ff37fe700 1 -- 172.21.2.101:0/1124751406 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff40e52e0 con 0x7f7ff40f20e0 2021-02-10T11:30:31.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.451 7f7ff37fe700 1 -- 172.21.2.101:0/1124751406 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f7fec001600 con 0x7f7ff40f20e0 2021-02-10T11:30:31.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.451 7f7ff37fe700 1 -- 172.21.2.101:0/1124751406 >> v1:172.21.2.101:6789/0 conn(0x7f7ff40f20e0 legacy=0x7f7ff40f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:31.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.451 7f7ff37fe700 1 --2- 172.21.2.101:0/1124751406 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fe0003af0 0x7f7fe0005f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:31.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.451 7f7ff37fe700 1 --2- 172.21.2.101:0/1124751406 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fe0006780 0x7f7fe0008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:31.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.451 7f7ff37fe700 1 --2- 172.21.2.101:0/1124751406 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7fe0009580 0x7f7fe000baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:31.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.451 7f7ff37fe700 1 -- 172.21.2.101:0/1124751406 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe000c2f0 con 0x7f7fe0003af0 2021-02-10T11:30:31.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.451 7f7ff37fe700 1 -- 172.21.2.101:0/1124751406 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe000c510 con 0x7f7fe0009580 2021-02-10T11:30:31.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.451 7f7ff37fe700 1 -- 172.21.2.101:0/1124751406 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe000c730 con 0x7f7fe0006780 2021-02-10T11:30:31.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.451 7f7ff8c05700 1 --2- 172.21.2.101:0/1124751406 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fe0003af0 0x7f7fe0005f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:31.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.452 7f7ff9406700 1 --2- 172.21.2.101:0/1124751406 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7fe0009580 0x7f7fe000baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:31.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.452 7f7ff3fff700 1 --2- 172.21.2.101:0/1124751406 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fe0006780 0x7f7fe0008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:31.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.452 7f7ff8c05700 1 -- 172.21.2.101:0/1124751406 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fe0006780 msgr2=0x7f7fe0008c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:31.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.452 7f7ff8c05700 1 --2- 172.21.2.101:0/1124751406 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fe0006780 0x7f7fe0008c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:31.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.452 7f7ff8c05700 1 -- 172.21.2.101:0/1124751406 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7fe0009580 msgr2=0x7f7fe000baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:31.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.452 7f7ff8c05700 1 --2- 172.21.2.101:0/1124751406 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7fe0009580 0x7f7fe000baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:31.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.452 7f7ff8c05700 1 -- 172.21.2.101:0/1124751406 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe00509a0 con 0x7f7fe0003af0 2021-02-10T11:30:31.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.452 7f7ff3fff700 1 --2- 172.21.2.101:0/1124751406 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fe0006780 0x7f7fe0008c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:30:31.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.452 7f7ff9406700 1 -- 172.21.2.101:0/1124751406 reap_dead start 2021-02-10T11:30:31.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.452 7f7ff8c05700 1 --2- 172.21.2.101:0/1124751406 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fe0003af0 0x7f7fe0005f00 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7f7fdc00c8f0 tx=0x7f7fdc00cc90).ready entity=mon.0 client_cookie=a115223d60a3cf0d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:31.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.452 7f7ff37fe700 1 -- 172.21.2.101:0/1124751406 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7fdc0009b0 con 0x7f7fe0003af0 2021-02-10T11:30:31.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.452 7f7ff37fe700 1 -- 172.21.2.101:0/1124751406 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fdc009310 con 0x7f7fe0003af0 2021-02-10T11:30:31.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.452 7f7ff37fe700 1 -- 172.21.2.101:0/1124751406 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7fdc0097f0 con 0x7f7fe0003af0 2021-02-10T11:30:31.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.452 7f7ffade9700 1 -- 172.21.2.101:0/1124751406 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fe0003af0 msgr2=0x7f7fe0005f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:31.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.452 7f7ffade9700 1 --2- 172.21.2.101:0/1124751406 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fe0003af0 0x7f7fe0005f00 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7f7fdc00c8f0 tx=0x7f7fdc00cc90).stop 2021-02-10T11:30:31.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.453 7f7ffade9700 1 -- 172.21.2.101:0/1124751406 shutdown_connections 2021-02-10T11:30:31.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.453 7f7ffade9700 1 --2- 172.21.2.101:0/1124751406 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fe0003af0 0x7f7fe0005f00 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:31.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.453 7f7ffade9700 1 -- 172.21.2.101:0/1124751406 >> 172.21.2.101:0/1124751406 conn(0x7f7ff40ce3e0 msgr2=0x7f7ff40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:31.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.453 7f7ffade9700 1 -- 172.21.2.101:0/1124751406 shutdown_connections 2021-02-10T11:30:31.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.453 7f7ffade9700 1 -- 172.21.2.101:0/1124751406 wait complete. 2021-02-10T11:30:31.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.468 7f7ffade9700 1 Processor -- start 2021-02-10T11:30:31.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.468 7f7ffade9700 1 -- start start 2021-02-10T11:30:31.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.468 7f7ffade9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ff409b930 0x7f7ff40b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:31.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.468 7f7ffade9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ff40b8410 0x7f7ff40b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:31.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.468 7f7ffade9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ff40c1f80 0x7f7ff40c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:31.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.468 7f7ffade9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff40c1a00 con 0x7f7ff40c1f80 2021-02-10T11:30:31.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.468 7f7ffade9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff40b8e30 con 0x7f7ff409b930 2021-02-10T11:30:31.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.468 7f7ffade9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff40b9050 con 0x7f7ff40b8410 2021-02-10T11:30:31.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.468 7f7ff9406700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ff40c1f80 0x7f7ff40c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:31.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.468 7f7ff9406700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ff40c1f80 0x7f7ff40c2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49110/0 (socket says 172.21.2.101:49110) 2021-02-10T11:30:31.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.468 7f7ff9406700 1 -- 172.21.2.101:0/1496962406 learned_addr learned my addr 172.21.2.101:0/1496962406 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:31.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.468 7f7ff3fff700 1 --2- 172.21.2.101:0/1496962406 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ff40b8410 0x7f7ff40b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:31.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.468 7f7ff8c05700 1 --2- 172.21.2.101:0/1496962406 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ff409b930 0x7f7ff40b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:31.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.469 7f7ff9406700 1 -- 172.21.2.101:0/1496962406 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ff40b8410 msgr2=0x7f7ff40b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:31.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.469 7f7ff9406700 1 --2- 172.21.2.101:0/1496962406 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ff40b8410 0x7f7ff40b8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:31.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.469 7f7ff9406700 1 -- 172.21.2.101:0/1496962406 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ff409b930 msgr2=0x7f7ff40b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:31.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.469 7f7ff9406700 1 --2- 172.21.2.101:0/1496962406 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ff409b930 0x7f7ff40b7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:31.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.469 7f7ff9406700 1 -- 172.21.2.101:0/1496962406 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff40c29a0 con 0x7f7ff40c1f80 2021-02-10T11:30:31.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.469 7f7ff9406700 1 --2- 172.21.2.101:0/1496962406 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ff40c1f80 0x7f7ff40c2380 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7f7fec007fa0 tx=0x7f7fec00a920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:31.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.469 7f7ff17fa700 1 -- 172.21.2.101:0/1496962406 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7fec007500 con 0x7f7ff40c1f80 2021-02-10T11:30:31.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.469 7f7ff17fa700 1 -- 172.21.2.101:0/1496962406 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fec00ad10 con 0x7f7ff40c1f80 2021-02-10T11:30:31.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.469 7f7ffade9700 1 -- 172.21.2.101:0/1496962406 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff40c29a0 con 0x7f7ff40c1f80 2021-02-10T11:30:31.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.469 7f7ff17fa700 1 -- 172.21.2.101:0/1496962406 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7fec0045b0 con 0x7f7ff40c1f80 2021-02-10T11:30:31.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.469 7f7ffade9700 1 -- 172.21.2.101:0/1496962406 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff40c2f30 con 0x7f7ff40c1f80 2021-02-10T11:30:31.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.470 7f7ff17fa700 1 -- 172.21.2.101:0/1496962406 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7fec0043a0 con 0x7f7ff40c1f80 2021-02-10T11:30:31.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.470 7f7ff17fa700 1 --2- 172.21.2.101:0/1496962406 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7fd80186d0 0x7f7fd801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:31.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.470 7f7ff17fa700 1 -- 172.21.2.101:0/1496962406 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7fec032790 con 0x7f7ff40c1f80 2021-02-10T11:30:31.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.470 7f7ff8c05700 1 --2- 172.21.2.101:0/1496962406 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7fd80186d0 0x7f7fd801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:31.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.470 7f7ff8c05700 1 --2- 172.21.2.101:0/1496962406 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7fd80186d0 0x7f7fd801abd0 crc :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:31.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.500 7f7ffade9700 1 -- 172.21.2.101:0/1496962406 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff4105b30 con 0x7f7ff40c1f80 2021-02-10T11:30:31.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.502 7f7ff17fa700 1 -- 172.21.2.101:0/1496962406 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7fec001c30 con 0x7f7ff40c1f80 2021-02-10T11:30:31.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.629 7f7ffade9700 1 -- 172.21.2.101:0/1496962406 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839289", "target": ["mgr", ""]}) v1 -- 0x7f7ff40c2f30 con 0x7f7fd80186d0 2021-02-10T11:30:31.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.639 7f7ff17fa700 1 -- 172.21.2.101:0/1496962406 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f7fdc006040 con 0x7f7fd80186d0 2021-02-10T11:30:31.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.646 7f7ffade9700 1 -- 172.21.2.101:0/1496962406 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7fd80186d0 msgr2=0x7f7fd801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:31.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.646 7f7ffade9700 1 --2- 172.21.2.101:0/1496962406 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7fd80186d0 0x7f7fd801abd0 crc :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:31.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.646 7f7ffade9700 1 -- 172.21.2.101:0/1496962406 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ff40c1f80 msgr2=0x7f7ff40c2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:31.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.646 7f7ffade9700 1 --2- 172.21.2.101:0/1496962406 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ff40c1f80 0x7f7ff40c2380 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7f7fec007fa0 tx=0x7f7fec00a920).stop 2021-02-10T11:30:31.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.646 7f7ffade9700 1 -- 172.21.2.101:0/1496962406 shutdown_connections 2021-02-10T11:30:31.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.646 7f7ffade9700 1 --2- 172.21.2.101:0/1496962406 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7fd80186d0 0x7f7fd801abd0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:31.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.646 7f7ffade9700 1 --2- 172.21.2.101:0/1496962406 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ff40b8410 0x7f7ff40b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:31.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.646 7f7ffade9700 1 --2- 172.21.2.101:0/1496962406 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ff40c1f80 0x7f7ff40c2380 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:31.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.646 7f7ffade9700 1 --2- 172.21.2.101:0/1496962406 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ff409b930 0x7f7ff40b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:31.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.646 7f7ffade9700 1 -- 172.21.2.101:0/1496962406 >> 172.21.2.101:0/1496962406 conn(0x7f7ff4094e10 msgr2=0x7f7ff40a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:31.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.646 7f7ffade9700 1 -- 172.21.2.101:0/1496962406 shutdown_connections 2021-02-10T11:30:31.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:31.646 7f7ffade9700 1 -- 172.21.2.101:0/1496962406 wait complete. 2021-02-10T11:30:31.666 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839290 2021-02-10T11:30:32.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.071 7fc51b69a700 1 Processor -- start 2021-02-10T11:30:32.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.071 7fc51b69a700 1 -- start start 2021-02-10T11:30:32.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.072 7fc51b69a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5141b0870 0x7fc5140f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:32.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.072 7fc51b69a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc5140efe00 0x7fc5140f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:32.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.072 7fc51b69a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5141b0c70 con 0x7fc5141b0470 2021-02-10T11:30:32.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.072 7fc51b69a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5140f0b50 con 0x7fc5140efe00 2021-02-10T11:30:32.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.072 7fc51b69a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5140f0820 con 0x7fc5141b0870 2021-02-10T11:30:32.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.072 7fc5194b6700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc5141b0470 0x7fc5140ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49610/0 (socket says 172.21.2.101:49610) 2021-02-10T11:30:32.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.072 7fc5194b6700 1 -- 172.21.2.101:0/2990813851 learned_addr learned my addr 172.21.2.101:0/2990813851 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:32.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.072 7fc519cb7700 1 --2- 172.21.2.101:0/2990813851 >> v2:172.21.2.101:3300/0 conn(0x7fc5140efe00 0x7fc5140f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:32.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.072 7fc518cb5700 1 --2- 172.21.2.101:0/2990813851 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5141b0870 0x7fc5140f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:32.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.072 7fc513fff700 1 -- 172.21.2.101:0/2990813851 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736290509 0 0) 0x7fc504001060 con 0x7fc5141b0470 2021-02-10T11:30:32.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.072 7fc513fff700 1 -- 172.21.2.101:0/2990813851 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4fc002fe0 con 0x7fc5141b0470 2021-02-10T11:30:32.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.072 7fc518cb5700 1 -- 172.21.2.101:0/2990813851 >> v2:172.21.2.101:3300/0 conn(0x7fc5140efe00 msgr2=0x7fc5140f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:32.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.072 7fc518cb5700 1 --2- 172.21.2.101:0/2990813851 >> v2:172.21.2.101:3300/0 conn(0x7fc5140efe00 0x7fc5140f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:32.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.072 7fc518cb5700 1 -- 172.21.2.101:0/2990813851 >> v1:172.21.2.101:6789/0 conn(0x7fc5141b0470 legacy=0x7fc5140ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:32.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.072 7fc518cb5700 1 -- 172.21.2.101:0/2990813851 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5140e52e0 con 0x7fc5141b0870 2021-02-10T11:30:32.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.072 7fc519cb7700 1 --2- 172.21.2.101:0/2990813851 >> v2:172.21.2.101:3300/0 conn(0x7fc5140efe00 0x7fc5140f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:32.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.072 7fc518cb5700 1 --2- 172.21.2.101:0/2990813851 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5141b0870 0x7fc5140f20e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fc508006650 tx=0x7fc508009a80).ready entity=mon.2 client_cookie=e5bf0028264e415c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:32.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.073 7fc513fff700 1 -- 172.21.2.101:0/2990813851 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc50800c0c0 con 0x7fc5141b0870 2021-02-10T11:30:32.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.073 7fc513fff700 1 -- 172.21.2.101:0/2990813851 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50800c340 con 0x7fc5141b0870 2021-02-10T11:30:32.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.073 7fc513fff700 1 -- 172.21.2.101:0/2990813851 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc50800c860 con 0x7fc5141b0870 2021-02-10T11:30:32.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.073 7fc51b69a700 1 -- 172.21.2.101:0/2990813851 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5141b0870 msgr2=0x7fc5140f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:32.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.073 7fc51b69a700 1 --2- 172.21.2.101:0/2990813851 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5141b0870 0x7fc5140f20e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fc508006650 tx=0x7fc508009a80).stop 2021-02-10T11:30:32.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.073 7fc51b69a700 1 -- 172.21.2.101:0/2990813851 shutdown_connections 2021-02-10T11:30:32.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.073 7fc51b69a700 1 --2- 172.21.2.101:0/2990813851 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5141b0870 0x7fc5140f20e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:32.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.073 7fc51b69a700 1 --2- 172.21.2.101:0/2990813851 >> v2:172.21.2.101:3300/0 conn(0x7fc5140efe00 0x7fc5140f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:32.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.073 7fc51b69a700 1 -- 172.21.2.101:0/2990813851 >> 172.21.2.101:0/2990813851 conn(0x7fc5140ce3e0 msgr2=0x7fc5140e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:32.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.074 7fc51b69a700 1 -- 172.21.2.101:0/2990813851 shutdown_connections 2021-02-10T11:30:32.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.074 7fc51b69a700 1 -- 172.21.2.101:0/2990813851 wait complete. 2021-02-10T11:30:32.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.088 7fc51b69a700 1 Processor -- start 2021-02-10T11:30:32.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.088 7fc51b69a700 1 -- start start 2021-02-10T11:30:32.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.089 7fc51b69a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5140a0420 0x7fc5140a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:32.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.089 7fc51b69a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc514189d00 0x7fc51418a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:32.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.089 7fc51b69a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc51418a720 0x7fc5140b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:32.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.089 7fc51b69a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc514189800 con 0x7fc5140a0420 2021-02-10T11:30:32.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.089 7fc51b69a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51418ab20 con 0x7fc514189d00 2021-02-10T11:30:32.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.089 7fc51b69a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5140a1140 con 0x7fc51418a720 2021-02-10T11:30:32.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.089 7fc5194b6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5140a0420 0x7fc5140a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:32.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.089 7fc5194b6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5140a0420 0x7fc5140a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49122/0 (socket says 172.21.2.101:49122) 2021-02-10T11:30:32.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.089 7fc5194b6700 1 -- 172.21.2.101:0/951247715 learned_addr learned my addr 172.21.2.101:0/951247715 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:32.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.089 7fc519cb7700 1 --2- 172.21.2.101:0/951247715 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc51418a720 0x7fc5140b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:32.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.089 7fc518cb5700 1 --2- 172.21.2.101:0/951247715 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc514189d00 0x7fc51418a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:32.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.089 7fc5194b6700 1 -- 172.21.2.101:0/951247715 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc51418a720 msgr2=0x7fc5140b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:32.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.090 7fc5194b6700 1 --2- 172.21.2.101:0/951247715 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc51418a720 0x7fc5140b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:32.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.090 7fc5194b6700 1 -- 172.21.2.101:0/951247715 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc514189d00 msgr2=0x7fc51418a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:32.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.090 7fc5194b6700 1 --2- 172.21.2.101:0/951247715 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc514189d00 0x7fc51418a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:32.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.090 7fc5194b6700 1 -- 172.21.2.101:0/951247715 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5140b8200 con 0x7fc5140a0420 2021-02-10T11:30:32.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.090 7fc519cb7700 1 --2- 172.21.2.101:0/951247715 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc51418a720 0x7fc5140b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:32.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.090 7fc5194b6700 1 --2- 172.21.2.101:0/951247715 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5140a0420 0x7fc5140a0820 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7fc504001c10 tx=0x7fc50400aca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:32.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.090 7fc511ffb700 1 -- 172.21.2.101:0/951247715 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc5040070d0 con 0x7fc5140a0420 2021-02-10T11:30:32.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.090 7fc511ffb700 1 -- 172.21.2.101:0/951247715 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5040072d0 con 0x7fc5140a0420 2021-02-10T11:30:32.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.090 7fc51b69a700 1 -- 172.21.2.101:0/951247715 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5140b8200 con 0x7fc5140a0420 2021-02-10T11:30:32.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.090 7fc51b69a700 1 -- 172.21.2.101:0/951247715 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5140b8200 con 0x7fc5140a0420 2021-02-10T11:30:32.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.090 7fc511ffb700 1 -- 172.21.2.101:0/951247715 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc504004720 con 0x7fc5140a0420 2021-02-10T11:30:32.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.091 7fc511ffb700 1 -- 172.21.2.101:0/951247715 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc504004720 con 0x7fc5140a0420 2021-02-10T11:30:32.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.091 7fc511ffb700 1 --2- 172.21.2.101:0/951247715 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4f40186c0 0x7fc4f401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:32.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.091 7fc511ffb700 1 -- 172.21.2.101:0/951247715 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc504032900 con 0x7fc5140a0420 2021-02-10T11:30:32.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.091 7fc518cb5700 1 --2- 172.21.2.101:0/951247715 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4f40186c0 0x7fc4f401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:32.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.091 7fc518cb5700 1 --2- 172.21.2.101:0/951247715 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4f40186c0 0x7fc4f401abc0 crc :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:32.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.120 7fc51b69a700 1 -- 172.21.2.101:0/951247715 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc514105ae0 con 0x7fc5140a0420 2021-02-10T11:30:32.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.123 7fc511ffb700 1 -- 172.21.2.101:0/951247715 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc50402f2b0 con 0x7fc5140a0420 2021-02-10T11:30:32.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.253 7fc51b69a700 1 -- 172.21.2.101:0/951247715 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839290", "target": ["mgr", ""]}) v1 -- 0x7fc51418ab20 con 0x7fc4f40186c0 2021-02-10T11:30:32.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.262 7fc511ffb700 1 -- 172.21.2.101:0/951247715 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fc508000c90 con 0x7fc4f40186c0 2021-02-10T11:30:32.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.271 7fc51b69a700 1 -- 172.21.2.101:0/951247715 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4f40186c0 msgr2=0x7fc4f401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:32.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.271 7fc51b69a700 1 --2- 172.21.2.101:0/951247715 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4f40186c0 0x7fc4f401abc0 crc :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:32.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.271 7fc51b69a700 1 -- 172.21.2.101:0/951247715 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5140a0420 msgr2=0x7fc5140a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:32.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.271 7fc51b69a700 1 --2- 172.21.2.101:0/951247715 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5140a0420 0x7fc5140a0820 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7fc504001c10 tx=0x7fc50400aca0).stop 2021-02-10T11:30:32.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.271 7fc51b69a700 1 -- 172.21.2.101:0/951247715 shutdown_connections 2021-02-10T11:30:32.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.271 7fc51b69a700 1 --2- 172.21.2.101:0/951247715 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4f40186c0 0x7fc4f401abc0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:32.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.271 7fc51b69a700 1 --2- 172.21.2.101:0/951247715 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc51418a720 0x7fc5140b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:32.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.271 7fc51b69a700 1 --2- 172.21.2.101:0/951247715 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc514189d00 0x7fc51418a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:32.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.271 7fc51b69a700 1 --2- 172.21.2.101:0/951247715 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5140a0420 0x7fc5140a0820 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:32.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.271 7fc51b69a700 1 -- 172.21.2.101:0/951247715 >> 172.21.2.101:0/951247715 conn(0x7fc514094e10 msgr2=0x7fc5140a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:32.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.271 7fc51b69a700 1 -- 172.21.2.101:0/951247715 shutdown_connections 2021-02-10T11:30:32.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.271 7fc51b69a700 1 -- 172.21.2.101:0/951247715 wait complete. 2021-02-10T11:30:32.291 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839291 2021-02-10T11:30:32.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.695 7f38aeef5700 1 Processor -- start 2021-02-10T11:30:32.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.695 7f38aeef5700 1 -- start start 2021-02-10T11:30:32.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.695 7f38aeef5700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f38a81b0470 0x7f38a81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:32.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.695 7f38aeef5700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f38a80ef610 0x7f38a80efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:32.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.695 7f38aeef5700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a80f0b50 con 0x7f38a80f0030 2021-02-10T11:30:32.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.695 7f38aeef5700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a80e52e0 con 0x7f38a81b0470 2021-02-10T11:30:32.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.695 7f38aeef5700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a80e5490 con 0x7f38a80ef610 2021-02-10T11:30:32.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.695 7f38acd11700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f38a81b0470 0x7f38a81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:32.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.695 7f38acd11700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f38a81b0470 0x7f38a81b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49130/0 (socket says 172.21.2.101:49130) 2021-02-10T11:30:32.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.695 7f38acd11700 1 -- 172.21.2.101:0/3788225961 learned_addr learned my addr 172.21.2.101:0/3788225961 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:32.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.695 7f38a7fff700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f38a80ef610 0x7f38a80efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:32.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.696 7f38acd11700 1 -- 172.21.2.101:0/3788225961 >> v2:172.21.2.108:3300/0 conn(0x7f38a80ef610 msgr2=0x7f38a80efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:32.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.696 7f38acd11700 1 --2- 172.21.2.101:0/3788225961 >> v2:172.21.2.108:3300/0 conn(0x7f38a80ef610 0x7f38a80efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:32.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.696 7f38acd11700 1 -- 172.21.2.101:0/3788225961 >> v1:172.21.2.108:6789/0 conn(0x7f38a80f0030 legacy=0x7f38a80f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:32.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.696 7f38acd11700 1 -- 172.21.2.101:0/3788225961 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a80f0430 con 0x7f38a81b0470 2021-02-10T11:30:32.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.696 7f38acd11700 1 --2- 172.21.2.101:0/3788225961 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38a81b0470 0x7f38a81b0870 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7f38a0006a80 tx=0x7f38a0009f00).ready entity=mon.0 client_cookie=cf157b7fda6b97e8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:32.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.696 7f38a77fe700 1 -- 172.21.2.101:0/3788225961 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f38a000b5c0 con 0x7f38a81b0470 2021-02-10T11:30:32.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.696 7f38a77fe700 1 -- 172.21.2.101:0/3788225961 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a000b8c0 con 0x7f38a81b0470 2021-02-10T11:30:32.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.696 7f38a77fe700 1 -- 172.21.2.101:0/3788225961 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f38a000bde0 con 0x7f38a81b0470 2021-02-10T11:30:32.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.696 7f38aeef5700 1 -- 172.21.2.101:0/3788225961 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38a81b0470 msgr2=0x7f38a81b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:32.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.696 7f38aeef5700 1 --2- 172.21.2.101:0/3788225961 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38a81b0470 0x7f38a81b0870 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7f38a0006a80 tx=0x7f38a0009f00).stop 2021-02-10T11:30:32.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.697 7f38aeef5700 1 -- 172.21.2.101:0/3788225961 shutdown_connections 2021-02-10T11:30:32.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.697 7f38aeef5700 1 --2- 172.21.2.101:0/3788225961 >> v2:172.21.2.108:3300/0 conn(0x7f38a80ef610 0x7f38a80efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:32.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.697 7f38aeef5700 1 --2- 172.21.2.101:0/3788225961 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38a81b0470 0x7f38a81b0870 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:32.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.697 7f38aeef5700 1 -- 172.21.2.101:0/3788225961 >> 172.21.2.101:0/3788225961 conn(0x7f38a80ce3e0 msgr2=0x7f38a80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:32.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.697 7f38aeef5700 1 -- 172.21.2.101:0/3788225961 shutdown_connections 2021-02-10T11:30:32.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.697 7f38aeef5700 1 -- 172.21.2.101:0/3788225961 wait complete. 2021-02-10T11:30:32.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.712 7f38aeef5700 1 Processor -- start 2021-02-10T11:30:32.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.712 7f38aeef5700 1 -- start start 2021-02-10T11:30:32.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.713 7f38aeef5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38a8105d10 0x7f38a809ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:32.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.713 7f38aeef5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38a8106110 0x7f38a80b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:32.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.713 7f38aeef5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38a809e0a0 0x7f38a80b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:32.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.713 7f38aeef5700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a809e640 con 0x7f38a8106110 2021-02-10T11:30:32.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.713 7f38aeef5700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a81a1620 con 0x7f38a8105d10 2021-02-10T11:30:32.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.713 7f38aeef5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a80b3a70 con 0x7f38a809e0a0 2021-02-10T11:30:32.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.713 7f38a7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38a8106110 0x7f38a80b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:32.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.713 7f38a7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38a8106110 0x7f38a80b2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49136/0 (socket says 172.21.2.101:49136) 2021-02-10T11:30:32.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.713 7f38a7fff700 1 -- 172.21.2.101:0/1592854151 learned_addr learned my addr 172.21.2.101:0/1592854151 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:32.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.713 7f38acd11700 1 --2- 172.21.2.101:0/1592854151 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38a8105d10 0x7f38a809ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:32.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.713 7f38ad512700 1 --2- 172.21.2.101:0/1592854151 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38a809e0a0 0x7f38a80b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:32.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.714 7f38a7fff700 1 -- 172.21.2.101:0/1592854151 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38a809e0a0 msgr2=0x7f38a80b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:32.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.714 7f38a7fff700 1 --2- 172.21.2.101:0/1592854151 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38a809e0a0 0x7f38a80b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:32.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.714 7f38a7fff700 1 -- 172.21.2.101:0/1592854151 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38a8105d10 msgr2=0x7f38a809ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:32.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.714 7f38a7fff700 1 --2- 172.21.2.101:0/1592854151 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38a8105d10 0x7f38a809ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:32.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.714 7f38a7fff700 1 -- 172.21.2.101:0/1592854151 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a80b3660 con 0x7f38a8106110 2021-02-10T11:30:32.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.714 7f38ad512700 1 --2- 172.21.2.101:0/1592854151 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38a809e0a0 0x7f38a80b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:30:32.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.714 7f38a7fff700 1 --2- 172.21.2.101:0/1592854151 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38a8106110 0x7f38a80b2a20 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7f389c006be0 tx=0x7f389c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:32.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.714 7f38a57fa700 1 -- 172.21.2.101:0/1592854151 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f389c0064d0 con 0x7f38a8106110 2021-02-10T11:30:32.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.714 7f38a57fa700 1 -- 172.21.2.101:0/1592854151 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c00bd80 con 0x7f38a8106110 2021-02-10T11:30:32.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.714 7f38aeef5700 1 -- 172.21.2.101:0/1592854151 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a80b3660 con 0x7f38a8106110 2021-02-10T11:30:32.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.714 7f38a57fa700 1 -- 172.21.2.101:0/1592854151 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f389c002030 con 0x7f38a8106110 2021-02-10T11:30:32.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.714 7f38aeef5700 1 -- 172.21.2.101:0/1592854151 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a80b3660 con 0x7f38a8106110 2021-02-10T11:30:32.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.715 7f38a57fa700 1 -- 172.21.2.101:0/1592854151 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f389c006c90 con 0x7f38a8106110 2021-02-10T11:30:32.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.715 7f38a57fa700 1 --2- 172.21.2.101:0/1592854151 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3888018730 0x7f388801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:32.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.715 7f38a57fa700 1 -- 172.21.2.101:0/1592854151 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f389c0325f0 con 0x7f38a8106110 2021-02-10T11:30:32.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.715 7f38acd11700 1 --2- 172.21.2.101:0/1592854151 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3888018730 0x7f388801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:32.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.715 7f38acd11700 1 --2- 172.21.2.101:0/1592854151 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3888018730 0x7f388801ac10 crc :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:32.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.744 7f38aeef5700 1 -- 172.21.2.101:0/1592854151 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a8105ae0 con 0x7f38a8106110 2021-02-10T11:30:32.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.747 7f38a57fa700 1 -- 172.21.2.101:0/1592854151 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f389c006820 con 0x7f38a8106110 2021-02-10T11:30:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.873 7f38aeef5700 1 -- 172.21.2.101:0/1592854151 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839291", "target": ["mgr", ""]}) v1 -- 0x7f38a81a1620 con 0x7f3888018730 2021-02-10T11:30:32.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.882 7f38a57fa700 1 -- 172.21.2.101:0/1592854151 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f38a0000d30 con 0x7f3888018730 2021-02-10T11:30:32.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.890 7f38aeef5700 1 -- 172.21.2.101:0/1592854151 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3888018730 msgr2=0x7f388801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:32.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.890 7f38aeef5700 1 --2- 172.21.2.101:0/1592854151 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3888018730 0x7f388801ac10 crc :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:32.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.890 7f38aeef5700 1 -- 172.21.2.101:0/1592854151 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38a8106110 msgr2=0x7f38a80b2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:32.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.890 7f38aeef5700 1 --2- 172.21.2.101:0/1592854151 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38a8106110 0x7f38a80b2a20 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7f389c006be0 tx=0x7f389c00b990).stop 2021-02-10T11:30:32.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.890 7f38aeef5700 1 -- 172.21.2.101:0/1592854151 shutdown_connections 2021-02-10T11:30:32.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.890 7f38aeef5700 1 --2- 172.21.2.101:0/1592854151 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3888018730 0x7f388801ac10 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:32.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.890 7f38aeef5700 1 --2- 172.21.2.101:0/1592854151 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38a809e0a0 0x7f38a80b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:32.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.890 7f38aeef5700 1 --2- 172.21.2.101:0/1592854151 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38a8106110 0x7f38a80b2a20 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:32.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.890 7f38aeef5700 1 --2- 172.21.2.101:0/1592854151 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38a8105d10 0x7f38a809ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:32.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.890 7f38aeef5700 1 -- 172.21.2.101:0/1592854151 >> 172.21.2.101:0/1592854151 conn(0x7f38a80c3c70 msgr2=0x7f38a80bd640 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:32.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.891 7f38aeef5700 1 -- 172.21.2.101:0/1592854151 shutdown_connections 2021-02-10T11:30:32.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:32.891 7f38aeef5700 1 -- 172.21.2.101:0/1592854151 wait complete. 2021-02-10T11:30:32.910 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839292 2021-02-10T11:30:33.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.313 7f3bf6f12700 1 Processor -- start 2021-02-10T11:30:33.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.313 7f3bf6f12700 1 -- start start 2021-02-10T11:30:33.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.313 7f3bf6f12700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3bf00f05f0 0x7f3bf00f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:33.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.313 7f3bf6f12700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf01b0c70 con 0x7f3bf01b0470 2021-02-10T11:30:33.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.313 7f3bf6f12700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf00f0b50 con 0x7f3bf01b0870 2021-02-10T11:30:33.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.313 7f3bf6f12700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf00e5ea0 con 0x7f3bf00f05f0 2021-02-10T11:30:33.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.313 7f3bf4d2e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f3bf01b0470 0x7f3bf00ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49638/0 (socket says 172.21.2.101:49638) 2021-02-10T11:30:33.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.313 7f3bf4d2e700 1 -- 172.21.2.101:0/1475639295 learned_addr learned my addr 172.21.2.101:0/1475639295 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:33.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.314 7f3bf552f700 1 --2- 172.21.2.101:0/1475639295 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3bf00f05f0 0x7f3bf00f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:33.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.314 7f3bef7fe700 1 -- 172.21.2.101:0/1475639295 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 363224885 0 0) 0x7f3bd8000ce0 con 0x7f3bf01b0470 2021-02-10T11:30:33.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.314 7f3bef7fe700 1 -- 172.21.2.101:0/1475639295 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bdc002fe0 con 0x7f3bf01b0470 2021-02-10T11:30:33.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.314 7f3bf552f700 1 -- 172.21.2.101:0/1475639295 >> v1:172.21.2.108:6789/0 conn(0x7f3bf01b0870 legacy=0x7f3bf00efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:33.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.314 7f3bf552f700 1 -- 172.21.2.101:0/1475639295 >> v1:172.21.2.101:6789/0 conn(0x7f3bf01b0470 legacy=0x7f3bf00ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:33.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.314 7f3bf552f700 1 -- 172.21.2.101:0/1475639295 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf00e52e0 con 0x7f3bf00f05f0 2021-02-10T11:30:33.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.314 7f3bf552f700 1 --2- 172.21.2.101:0/1475639295 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3bf00f05f0 0x7f3bf00f20e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f3be8006ad0 tx=0x7f3be8009ad0).ready entity=mon.2 client_cookie=60cc0d8665e92644 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:33.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.314 7f3bef7fe700 1 -- 172.21.2.101:0/1475639295 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3be80066a0 con 0x7f3bf00f05f0 2021-02-10T11:30:33.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.314 7f3bef7fe700 1 -- 172.21.2.101:0/1475639295 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be8006410 con 0x7f3bf00f05f0 2021-02-10T11:30:33.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.314 7f3bef7fe700 1 -- 172.21.2.101:0/1475639295 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3be8000c10 con 0x7f3bf00f05f0 2021-02-10T11:30:33.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.314 7f3bf6f12700 1 -- 172.21.2.101:0/1475639295 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3bf00f05f0 msgr2=0x7f3bf00f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:33.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.314 7f3bf6f12700 1 --2- 172.21.2.101:0/1475639295 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3bf00f05f0 0x7f3bf00f20e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f3be8006ad0 tx=0x7f3be8009ad0).stop 2021-02-10T11:30:33.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.315 7f3bf6f12700 1 -- 172.21.2.101:0/1475639295 shutdown_connections 2021-02-10T11:30:33.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.315 7f3bf6f12700 1 --2- 172.21.2.101:0/1475639295 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3bf00f05f0 0x7f3bf00f20e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:33.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.315 7f3bf6f12700 1 -- 172.21.2.101:0/1475639295 >> 172.21.2.101:0/1475639295 conn(0x7f3bf00ce3e0 msgr2=0x7f3bf00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:33.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.315 7f3bf6f12700 1 -- 172.21.2.101:0/1475639295 shutdown_connections 2021-02-10T11:30:33.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.315 7f3bf6f12700 1 -- 172.21.2.101:0/1475639295 wait complete. 2021-02-10T11:30:33.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.330 7f3bf6f12700 1 Processor -- start 2021-02-10T11:30:33.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.330 7f3bf6f12700 1 -- start start 2021-02-10T11:30:33.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.330 7f3bf6f12700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3bf00a9690 0x7f3bf00a9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:33.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.330 7f3bf6f12700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3bf00aa0b0 0x7f3bf00aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:33.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.330 7f3bf6f12700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3bf00f1800 0x7f3bf00f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:33.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.330 7f3bf6f12700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf0092350 con 0x7f3bf00f1800 2021-02-10T11:30:33.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.330 7f3bf6f12700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf00a6540 con 0x7f3bf00aa0b0 2021-02-10T11:30:33.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.330 7f3bf6f12700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf00a6720 con 0x7f3bf00a9690 2021-02-10T11:30:33.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.331 7f3bf552f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3bf00f1800 0x7f3bf00f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:33.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.331 7f3bf552f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3bf00f1800 0x7f3bf00f1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49152/0 (socket says 172.21.2.101:49152) 2021-02-10T11:30:33.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.331 7f3bf552f700 1 -- 172.21.2.101:0/925385161 learned_addr learned my addr 172.21.2.101:0/925385161 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:33.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.331 7f3beffff700 1 --2- 172.21.2.101:0/925385161 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3bf00aa0b0 0x7f3bf00aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:33.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.331 7f3bf4d2e700 1 --2- 172.21.2.101:0/925385161 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3bf00a9690 0x7f3bf00a9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:33.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.331 7f3bf552f700 1 -- 172.21.2.101:0/925385161 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3bf00a9690 msgr2=0x7f3bf00a9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:33.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.331 7f3bf552f700 1 --2- 172.21.2.101:0/925385161 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3bf00a9690 0x7f3bf00a9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:33.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.331 7f3bf552f700 1 -- 172.21.2.101:0/925385161 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3bf00aa0b0 msgr2=0x7f3bf00aa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:33.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.331 7f3bf552f700 1 --2- 172.21.2.101:0/925385161 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3bf00aa0b0 0x7f3bf00aa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:33.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.331 7f3bf552f700 1 -- 172.21.2.101:0/925385161 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf00a69e0 con 0x7f3bf00f1800 2021-02-10T11:30:33.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.331 7f3bf552f700 1 --2- 172.21.2.101:0/925385161 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3bf00f1800 0x7f3bf00f1c00 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f3be800b610 tx=0x7f3be800b9b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:33.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.331 7f3bed7fa700 1 -- 172.21.2.101:0/925385161 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3be8002030 con 0x7f3bf00f1800 2021-02-10T11:30:33.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.331 7f3bed7fa700 1 -- 172.21.2.101:0/925385161 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be8002330 con 0x7f3bf00f1800 2021-02-10T11:30:33.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.331 7f3bed7fa700 1 -- 172.21.2.101:0/925385161 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3be8002780 con 0x7f3bf00f1800 2021-02-10T11:30:33.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.331 7f3bf6f12700 1 -- 172.21.2.101:0/925385161 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf00a69e0 con 0x7f3bf00f1800 2021-02-10T11:30:33.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.331 7f3bf6f12700 1 -- 172.21.2.101:0/925385161 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf00a69e0 con 0x7f3bf00f1800 2021-02-10T11:30:33.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.332 7f3bed7fa700 1 -- 172.21.2.101:0/925385161 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3be8000a70 con 0x7f3bf00f1800 2021-02-10T11:30:33.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.332 7f3bed7fa700 1 --2- 172.21.2.101:0/925385161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3bd40186c0 0x7f3bd401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:33.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.332 7f3bed7fa700 1 -- 172.21.2.101:0/925385161 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3be80325f0 con 0x7f3bf00f1800 2021-02-10T11:30:33.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.332 7f3bf4d2e700 1 --2- 172.21.2.101:0/925385161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3bd40186c0 0x7f3bd401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:33.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.332 7f3bf4d2e700 1 --2- 172.21.2.101:0/925385161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3bd40186c0 0x7f3bd401abc0 crc :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:33.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.362 7f3bf6f12700 1 -- 172.21.2.101:0/925385161 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf00edd10 con 0x7f3bf00f1800 2021-02-10T11:30:33.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.365 7f3bed7fa700 1 -- 172.21.2.101:0/925385161 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3be802df50 con 0x7f3bf00f1800 2021-02-10T11:30:33.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.497 7f3bf6f12700 1 -- 172.21.2.101:0/925385161 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839292", "target": ["mgr", ""]}) v1 -- 0x7f3bf00e52e0 con 0x7f3bd40186c0 2021-02-10T11:30:33.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.506 7f3bed7fa700 1 -- 172.21.2.101:0/925385161 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f3bd8000ce0 con 0x7f3bd40186c0 2021-02-10T11:30:33.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.513 7f3bf6f12700 1 -- 172.21.2.101:0/925385161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3bd40186c0 msgr2=0x7f3bd401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:33.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.513 7f3bf6f12700 1 --2- 172.21.2.101:0/925385161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3bd40186c0 0x7f3bd401abc0 crc :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:33.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.513 7f3bf6f12700 1 -- 172.21.2.101:0/925385161 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3bf00f1800 msgr2=0x7f3bf00f1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:33.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.513 7f3bf6f12700 1 --2- 172.21.2.101:0/925385161 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3bf00f1800 0x7f3bf00f1c00 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f3be800b610 tx=0x7f3be800b9b0).stop 2021-02-10T11:30:33.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.514 7f3bf6f12700 1 -- 172.21.2.101:0/925385161 shutdown_connections 2021-02-10T11:30:33.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.514 7f3bf6f12700 1 --2- 172.21.2.101:0/925385161 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3bf00f1800 0x7f3bf00f1c00 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:33.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.514 7f3bf6f12700 1 --2- 172.21.2.101:0/925385161 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3bf00aa0b0 0x7f3bf00aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:33.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.514 7f3bf6f12700 1 --2- 172.21.2.101:0/925385161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3bd40186c0 0x7f3bd401abc0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:33.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.514 7f3bf6f12700 1 --2- 172.21.2.101:0/925385161 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3bf00a9690 0x7f3bf00a9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:33.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.514 7f3bf6f12700 1 -- 172.21.2.101:0/925385161 >> 172.21.2.101:0/925385161 conn(0x7f3bf00c40e0 msgr2=0x7f3bf0094eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:33.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.514 7f3bf6f12700 1 -- 172.21.2.101:0/925385161 shutdown_connections 2021-02-10T11:30:33.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.514 7f3bf6f12700 1 -- 172.21.2.101:0/925385161 wait complete. 2021-02-10T11:30:33.534 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839293 2021-02-10T11:30:33.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.937 7f1771562700 1 Processor -- start 2021-02-10T11:30:33.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.937 7f1771562700 1 -- start start 2021-02-10T11:30:33.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.938 7f1771562700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f176c0f05f0 0x7f176c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:33.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.938 7f1771562700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f176c1b0c70 con 0x7f176c1b0470 2021-02-10T11:30:33.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.938 7f1771562700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f176c0f0b50 con 0x7f176c1b0870 2021-02-10T11:30:33.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.938 7f1771562700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f176c0e5ea0 con 0x7f176c0f05f0 2021-02-10T11:30:33.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.938 7f176b2bf700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f176c1b0470 0x7f176c0ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49650/0 (socket says 172.21.2.101:49650) 2021-02-10T11:30:33.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.938 7f176b2bf700 1 -- 172.21.2.101:0/3976867546 learned_addr learned my addr 172.21.2.101:0/3976867546 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:33.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.938 7f176bac0700 1 --2- 172.21.2.101:0/3976867546 >> v2:172.21.2.108:3300/0 conn(0x7f176c0f05f0 0x7f176c0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:33.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.939 7f176a2bd700 1 -- 172.21.2.101:0/3976867546 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306088979 0 0) 0x7f17540010b0 con 0x7f176c1b0470 2021-02-10T11:30:33.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.939 7f176a2bd700 1 -- 172.21.2.101:0/3976867546 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1758002fe0 con 0x7f176c1b0470 2021-02-10T11:30:33.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.939 7f176bac0700 1 -- 172.21.2.101:0/3976867546 >> v1:172.21.2.108:6789/0 conn(0x7f176c1b0870 legacy=0x7f176c0efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:33.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.939 7f176bac0700 1 -- 172.21.2.101:0/3976867546 >> v1:172.21.2.101:6789/0 conn(0x7f176c1b0470 legacy=0x7f176c0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:33.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.939 7f176bac0700 1 -- 172.21.2.101:0/3976867546 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c0e52e0 con 0x7f176c0f05f0 2021-02-10T11:30:33.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.939 7f176bac0700 1 --2- 172.21.2.101:0/3976867546 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f176c0f05f0 0x7f176c0f20e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f1764006e50 tx=0x7f1764009e70).ready entity=mon.1 client_cookie=1d39a94a744c6eaa server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:33.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.939 7f176a2bd700 1 -- 172.21.2.101:0/3976867546 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1764006a20 con 0x7f176c0f05f0 2021-02-10T11:30:33.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.939 7f176a2bd700 1 -- 172.21.2.101:0/3976867546 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1764006790 con 0x7f176c0f05f0 2021-02-10T11:30:33.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.939 7f176a2bd700 1 -- 172.21.2.101:0/3976867546 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f176400b700 con 0x7f176c0f05f0 2021-02-10T11:30:33.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.939 7f1771562700 1 -- 172.21.2.101:0/3976867546 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f176c0f05f0 msgr2=0x7f176c0f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:33.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.939 7f1771562700 1 --2- 172.21.2.101:0/3976867546 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f176c0f05f0 0x7f176c0f20e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f1764006e50 tx=0x7f1764009e70).stop 2021-02-10T11:30:33.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.940 7f1771562700 1 -- 172.21.2.101:0/3976867546 shutdown_connections 2021-02-10T11:30:33.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.940 7f1771562700 1 --2- 172.21.2.101:0/3976867546 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f176c0f05f0 0x7f176c0f20e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:33.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.940 7f1771562700 1 -- 172.21.2.101:0/3976867546 >> 172.21.2.101:0/3976867546 conn(0x7f176c0ce3e0 msgr2=0x7f176c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:33.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.940 7f1771562700 1 -- 172.21.2.101:0/3976867546 shutdown_connections 2021-02-10T11:30:33.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.940 7f1771562700 1 -- 172.21.2.101:0/3976867546 wait complete. 2021-02-10T11:30:33.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.954 7f1771562700 1 Processor -- start 2021-02-10T11:30:33.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.955 7f1771562700 1 -- start start 2021-02-10T11:30:33.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.955 7f1771562700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f176c0c36a0 0x7f176c0c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:33.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.955 7f1771562700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f176c0c3190 0x7f176c0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:33.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.955 7f1771562700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f176c0b2370 0x7f176c0b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:33.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.955 7f1771562700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0b27c0 con 0x7f176c0c36a0 2021-02-10T11:30:33.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.955 7f1771562700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0b3360 con 0x7f176c0b2370 2021-02-10T11:30:33.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.955 7f1771562700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0b3540 con 0x7f176c0c3190 2021-02-10T11:30:33.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.955 7f176b2bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f176c0c36a0 0x7f176c0c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:33.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.955 7f176b2bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f176c0c36a0 0x7f176c0c2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49160/0 (socket says 172.21.2.101:49160) 2021-02-10T11:30:33.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.955 7f176b2bf700 1 -- 172.21.2.101:0/2433236142 learned_addr learned my addr 172.21.2.101:0/2433236142 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:33.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.955 7f176b2bf700 1 -- 172.21.2.101:0/2433236142 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f176c0c3190 msgr2=0x7f176c0b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:33.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.955 7f176b2bf700 1 --2- 172.21.2.101:0/2433236142 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f176c0c3190 0x7f176c0b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:33.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.955 7f176b2bf700 1 -- 172.21.2.101:0/2433236142 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f176c0b2370 msgr2=0x7f176c0b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:33.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.956 7f176b2bf700 1 --2- 172.21.2.101:0/2433236142 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f176c0b2370 0x7f176c0b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:33.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.956 7f176b2bf700 1 -- 172.21.2.101:0/2433236142 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c0b3760 con 0x7f176c0c36a0 2021-02-10T11:30:33.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.956 7f176b2bf700 1 --2- 172.21.2.101:0/2433236142 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f176c0c36a0 0x7f176c0c2b70 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7f1754001c60 tx=0x7f175400a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:33.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.956 7f1763fff700 1 -- 172.21.2.101:0/2433236142 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f175400acc0 con 0x7f176c0c36a0 2021-02-10T11:30:33.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.956 7f1763fff700 1 -- 172.21.2.101:0/2433236142 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1754004350 con 0x7f176c0c36a0 2021-02-10T11:30:33.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.956 7f1763fff700 1 -- 172.21.2.101:0/2433236142 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f17540047b0 con 0x7f176c0c36a0 2021-02-10T11:30:33.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.956 7f1771562700 1 -- 172.21.2.101:0/2433236142 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176c0b3760 con 0x7f176c0c36a0 2021-02-10T11:30:33.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.956 7f1771562700 1 -- 172.21.2.101:0/2433236142 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176c0b3760 con 0x7f176c0c36a0 2021-02-10T11:30:33.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.956 7f1763fff700 1 -- 172.21.2.101:0/2433236142 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1754004530 con 0x7f176c0c36a0 2021-02-10T11:30:33.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.956 7f1763fff700 1 --2- 172.21.2.101:0/2433236142 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f17500186c0 0x7f175001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:33.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.956 7f1763fff700 1 -- 172.21.2.101:0/2433236142 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1754032400 con 0x7f176c0c36a0 2021-02-10T11:30:33.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.957 7f176aabe700 1 --2- 172.21.2.101:0/2433236142 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f17500186c0 0x7f175001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:33.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.957 7f176aabe700 1 --2- 172.21.2.101:0/2433236142 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f17500186c0 0x7f175001abc0 crc :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:33.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.987 7f1771562700 1 -- 172.21.2.101:0/2433236142 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f176c0edd10 con 0x7f176c0c36a0 2021-02-10T11:30:34.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:33.990 7f1763fff700 1 -- 172.21.2.101:0/2433236142 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f175402cf30 con 0x7f176c0c36a0 2021-02-10T11:30:34.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.116 7f1771562700 1 -- 172.21.2.101:0/2433236142 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839293", "target": ["mgr", ""]}) v1 -- 0x7f176c0edd10 con 0x7f17500186c0 2021-02-10T11:30:34.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.126 7f1763fff700 1 -- 172.21.2.101:0/2433236142 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f175c000de0 con 0x7f17500186c0 2021-02-10T11:30:34.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.133 7f1771562700 1 -- 172.21.2.101:0/2433236142 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f17500186c0 msgr2=0x7f175001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:34.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.133 7f1771562700 1 --2- 172.21.2.101:0/2433236142 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f17500186c0 0x7f175001abc0 crc :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:34.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.133 7f1771562700 1 -- 172.21.2.101:0/2433236142 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f176c0c36a0 msgr2=0x7f176c0c2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:34.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.133 7f1771562700 1 --2- 172.21.2.101:0/2433236142 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f176c0c36a0 0x7f176c0c2b70 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7f1754001c60 tx=0x7f175400a8d0).stop 2021-02-10T11:30:34.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.133 7f1771562700 1 -- 172.21.2.101:0/2433236142 shutdown_connections 2021-02-10T11:30:34.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.133 7f1771562700 1 --2- 172.21.2.101:0/2433236142 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f17500186c0 0x7f175001abc0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:34.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.133 7f1771562700 1 --2- 172.21.2.101:0/2433236142 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f176c0c3190 0x7f176c0b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:34.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.133 7f1771562700 1 --2- 172.21.2.101:0/2433236142 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f176c0b2370 0x7f176c0b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:34.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.133 7f1771562700 1 --2- 172.21.2.101:0/2433236142 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f176c0c36a0 0x7f176c0c2b70 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:34.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.133 7f1771562700 1 -- 172.21.2.101:0/2433236142 >> 172.21.2.101:0/2433236142 conn(0x7f176c09f630 msgr2=0x7f176c0a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:34.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.134 7f1771562700 1 -- 172.21.2.101:0/2433236142 shutdown_connections 2021-02-10T11:30:34.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.134 7f1771562700 1 -- 172.21.2.101:0/2433236142 wait complete. 2021-02-10T11:30:34.154 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839294 2021-02-10T11:30:34.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.558 7fc71b0e8700 1 Processor -- start 2021-02-10T11:30:34.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.558 7fc71b0e8700 1 -- start start 2021-02-10T11:30:34.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.559 7fc71b0e8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc7141b0870 0x7fc7140f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:34.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.559 7fc71b0e8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7140efe00 0x7fc7140f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:34.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.559 7fc71b0e8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7141b0c70 con 0x7fc7141b0470 2021-02-10T11:30:34.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.559 7fc71b0e8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7140f0b50 con 0x7fc7141b0870 2021-02-10T11:30:34.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.559 7fc71b0e8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7140f0820 con 0x7fc7140efe00 2021-02-10T11:30:34.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.559 7fc718f04700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fc7141b0470 0x7fc7140ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:43072/0 (socket says 172.21.2.101:43072) 2021-02-10T11:30:34.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.559 7fc718f04700 1 -- 172.21.2.101:0/1839767062 learned_addr learned my addr 172.21.2.101:0/1839767062 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:34.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.559 7fc713fff700 1 --2- 172.21.2.101:0/1839767062 >> v2:172.21.2.108:3300/0 conn(0x7fc7141b0870 0x7fc7140f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:34.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.559 7fc719705700 1 --2- 172.21.2.101:0/1839767062 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7140efe00 0x7fc7140f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:34.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.560 7fc7137fe700 1 -- 172.21.2.101:0/1839767062 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010476778 0 0) 0x7fc6fc000c60 con 0x7fc7141b0470 2021-02-10T11:30:34.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.560 7fc7137fe700 1 -- 172.21.2.101:0/1839767062 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc700002fe0 con 0x7fc7141b0470 2021-02-10T11:30:34.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.560 7fc713fff700 1 -- 172.21.2.101:0/1839767062 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7140efe00 msgr2=0x7fc7140f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:34.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.560 7fc713fff700 1 --2- 172.21.2.101:0/1839767062 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7140efe00 0x7fc7140f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:34.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.560 7fc713fff700 1 -- 172.21.2.101:0/1839767062 >> v1:172.21.2.108:6789/0 conn(0x7fc7141b0470 legacy=0x7fc7140ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:34.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.560 7fc713fff700 1 -- 172.21.2.101:0/1839767062 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7140e52e0 con 0x7fc7141b0870 2021-02-10T11:30:34.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.560 7fc713fff700 1 --2- 172.21.2.101:0/1839767062 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7141b0870 0x7fc7140f20e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fc7040069d0 tx=0x7fc704009e40).ready entity=mon.1 client_cookie=e5351e8ef001bf95 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:34.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.560 7fc7137fe700 1 -- 172.21.2.101:0/1839767062 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc70400b510 con 0x7fc7141b0870 2021-02-10T11:30:34.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.560 7fc7137fe700 1 -- 172.21.2.101:0/1839767062 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70400b790 con 0x7fc7141b0870 2021-02-10T11:30:34.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.560 7fc7137fe700 1 -- 172.21.2.101:0/1839767062 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc70400bcb0 con 0x7fc7141b0870 2021-02-10T11:30:34.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.560 7fc71b0e8700 1 -- 172.21.2.101:0/1839767062 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7141b0870 msgr2=0x7fc7140f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:34.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.560 7fc71b0e8700 1 --2- 172.21.2.101:0/1839767062 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7141b0870 0x7fc7140f20e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fc7040069d0 tx=0x7fc704009e40).stop 2021-02-10T11:30:34.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.561 7fc71b0e8700 1 -- 172.21.2.101:0/1839767062 shutdown_connections 2021-02-10T11:30:34.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.561 7fc71b0e8700 1 --2- 172.21.2.101:0/1839767062 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7140efe00 0x7fc7140f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:34.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.561 7fc71b0e8700 1 --2- 172.21.2.101:0/1839767062 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7141b0870 0x7fc7140f20e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:34.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.561 7fc71b0e8700 1 -- 172.21.2.101:0/1839767062 >> 172.21.2.101:0/1839767062 conn(0x7fc7140ce3e0 msgr2=0x7fc7140e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:34.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.561 7fc71b0e8700 1 -- 172.21.2.101:0/1839767062 shutdown_connections 2021-02-10T11:30:34.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.561 7fc71b0e8700 1 -- 172.21.2.101:0/1839767062 wait complete. 2021-02-10T11:30:34.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.576 7fc71b0e8700 1 Processor -- start 2021-02-10T11:30:34.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.576 7fc71b0e8700 1 -- start start 2021-02-10T11:30:34.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.576 7fc71b0e8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc714105d10 0x7fc71409f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:34.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.576 7fc71b0e8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7140961f0 0x7fc7140965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:34.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.576 7fc71b0e8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc714106890 0x7fc714106c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:34.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.576 7fc71b0e8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc714106110 con 0x7fc714105d10 2021-02-10T11:30:34.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.576 7fc71b0e8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc714106380 con 0x7fc714106890 2021-02-10T11:30:34.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.576 7fc71b0e8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7140a6620 con 0x7fc7140961f0 2021-02-10T11:30:34.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.577 7fc718f04700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc714105d10 0x7fc71409f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:34.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.577 7fc718f04700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc714105d10 0x7fc71409f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49176/0 (socket says 172.21.2.101:49176) 2021-02-10T11:30:34.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.577 7fc718f04700 1 -- 172.21.2.101:0/3813867987 learned_addr learned my addr 172.21.2.101:0/3813867987 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:34.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.577 7fc719705700 1 --2- 172.21.2.101:0/3813867987 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc714106890 0x7fc714106c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:34.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.577 7fc713fff700 1 --2- 172.21.2.101:0/3813867987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7140961f0 0x7fc7140965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:34.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.577 7fc718f04700 1 -- 172.21.2.101:0/3813867987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7140961f0 msgr2=0x7fc7140965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:34.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.577 7fc718f04700 1 --2- 172.21.2.101:0/3813867987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7140961f0 0x7fc7140965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:34.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.577 7fc718f04700 1 -- 172.21.2.101:0/3813867987 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc714106890 msgr2=0x7fc714106c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:34.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.577 7fc718f04700 1 --2- 172.21.2.101:0/3813867987 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc714106890 0x7fc714106c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:34.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.577 7fc718f04700 1 -- 172.21.2.101:0/3813867987 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7141072b0 con 0x7fc714105d10 2021-02-10T11:30:34.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.577 7fc718f04700 1 --2- 172.21.2.101:0/3813867987 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc714105d10 0x7fc71409f880 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7fc6fc001420 tx=0x7fc6fc00a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:34.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.578 7fc7117fa700 1 -- 172.21.2.101:0/3813867987 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc6fc000c60 con 0x7fc714105d10 2021-02-10T11:30:34.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.578 7fc7117fa700 1 -- 172.21.2.101:0/3813867987 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6fc007030 con 0x7fc714105d10 2021-02-10T11:30:34.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.578 7fc7117fa700 1 -- 172.21.2.101:0/3813867987 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc6fc004320 con 0x7fc714105d10 2021-02-10T11:30:34.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.578 7fc71b0e8700 1 -- 172.21.2.101:0/3813867987 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7141072b0 con 0x7fc714105d10 2021-02-10T11:30:34.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.578 7fc71b0e8700 1 -- 172.21.2.101:0/3813867987 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7141072b0 con 0x7fc714105d10 2021-02-10T11:30:34.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.578 7fc7117fa700 1 -- 172.21.2.101:0/3813867987 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc6fc004320 con 0x7fc714105d10 2021-02-10T11:30:34.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.578 7fc7117fa700 1 --2- 172.21.2.101:0/3813867987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6f80186c0 0x7fc6f801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:34.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.578 7fc7117fa700 1 -- 172.21.2.101:0/3813867987 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc6fc032640 con 0x7fc714105d10 2021-02-10T11:30:34.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.578 7fc713fff700 1 --2- 172.21.2.101:0/3813867987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6f80186c0 0x7fc6f801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:34.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.579 7fc713fff700 1 --2- 172.21.2.101:0/3813867987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6f80186c0 0x7fc6f801abc0 crc :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:34.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.609 7fc71b0e8700 1 -- 172.21.2.101:0/3813867987 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc714105ae0 con 0x7fc714105d10 2021-02-10T11:30:34.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.611 7fc7117fa700 1 -- 172.21.2.101:0/3813867987 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc6fc000e70 con 0x7fc714105d10 2021-02-10T11:30:34.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.738 7fc71b0e8700 1 -- 172.21.2.101:0/3813867987 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839294", "target": ["mgr", ""]}) v1 -- 0x7fc7140f0b50 con 0x7fc6f80186c0 2021-02-10T11:30:34.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.748 7fc7117fa700 1 -- 172.21.2.101:0/3813867987 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fc7040068d0 con 0x7fc6f80186c0 2021-02-10T11:30:34.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.755 7fc71b0e8700 1 -- 172.21.2.101:0/3813867987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6f80186c0 msgr2=0x7fc6f801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:34.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.755 7fc71b0e8700 1 --2- 172.21.2.101:0/3813867987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6f80186c0 0x7fc6f801abc0 crc :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:34.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.755 7fc71b0e8700 1 -- 172.21.2.101:0/3813867987 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc714105d10 msgr2=0x7fc71409f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:34.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.755 7fc71b0e8700 1 --2- 172.21.2.101:0/3813867987 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc714105d10 0x7fc71409f880 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7fc6fc001420 tx=0x7fc6fc00a8d0).stop 2021-02-10T11:30:34.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.755 7fc71b0e8700 1 -- 172.21.2.101:0/3813867987 shutdown_connections 2021-02-10T11:30:34.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.755 7fc71b0e8700 1 --2- 172.21.2.101:0/3813867987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6f80186c0 0x7fc6f801abc0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:34.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.755 7fc71b0e8700 1 --2- 172.21.2.101:0/3813867987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7140961f0 0x7fc7140965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:34.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.755 7fc71b0e8700 1 --2- 172.21.2.101:0/3813867987 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc714106890 0x7fc714106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:34.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.755 7fc71b0e8700 1 --2- 172.21.2.101:0/3813867987 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc714105d10 0x7fc71409f880 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:34.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.755 7fc71b0e8700 1 -- 172.21.2.101:0/3813867987 >> 172.21.2.101:0/3813867987 conn(0x7fc714095a70 msgr2=0x7fc7140bf050 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:34.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.756 7fc71b0e8700 1 -- 172.21.2.101:0/3813867987 shutdown_connections 2021-02-10T11:30:34.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:34.756 7fc71b0e8700 1 -- 172.21.2.101:0/3813867987 wait complete. 2021-02-10T11:30:34.775 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839295 2021-02-10T11:30:35.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.180 7f0eb5ebc700 1 Processor -- start 2021-02-10T11:30:35.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.180 7f0eb5ebc700 1 -- start start 2021-02-10T11:30:35.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.181 7f0eb5ebc700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0eb01b0470 0x7f0eb01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:35.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.181 7f0eb5ebc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0eb00ef610 0x7f0eb00efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:35.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.181 7f0eb5ebc700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb00f0b50 con 0x7f0eb00f0030 2021-02-10T11:30:35.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.181 7f0eb5ebc700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb00e52e0 con 0x7f0eb01b0470 2021-02-10T11:30:35.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.181 7f0eb5ebc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb00e54d0 con 0x7f0eb00ef610 2021-02-10T11:30:35.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.181 7f0eaf7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0eb01b0470 0x7f0eb01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:35.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.181 7f0eaf7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0eb01b0470 0x7f0eb01b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49182/0 (socket says 172.21.2.101:49182) 2021-02-10T11:30:35.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.181 7f0eaf7fe700 1 -- 172.21.2.101:0/2880604209 learned_addr learned my addr 172.21.2.101:0/2880604209 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:35.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.181 7f0ea6ffd700 1 --2- 172.21.2.101:0/2880604209 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0eb00ef610 0x7f0eb00efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:35.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.181 7f0eaf7fe700 1 -- 172.21.2.101:0/2880604209 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0eb00ef610 msgr2=0x7f0eb00efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:35.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.181 7f0eaf7fe700 1 --2- 172.21.2.101:0/2880604209 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0eb00ef610 0x7f0eb00efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:35.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.181 7f0eaf7fe700 1 -- 172.21.2.101:0/2880604209 >> v1:172.21.2.108:6789/0 conn(0x7f0eb00f0030 legacy=0x7f0eb00f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:35.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.181 7f0eaf7fe700 1 -- 172.21.2.101:0/2880604209 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb00f0430 con 0x7f0eb01b0470 2021-02-10T11:30:35.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.181 7f0eaf7fe700 1 --2- 172.21.2.101:0/2880604209 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0eb01b0470 0x7f0eb01b0870 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7f0ea0006700 tx=0x7f0ea0009b80).ready entity=mon.0 client_cookie=dabfeb3bf0dd2437 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:35.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.181 7f0eaeffd700 1 -- 172.21.2.101:0/2880604209 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0ea000c0c0 con 0x7f0eb01b0470 2021-02-10T11:30:35.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.181 7f0eaeffd700 1 -- 172.21.2.101:0/2880604209 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea000c440 con 0x7f0eb01b0470 2021-02-10T11:30:35.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.181 7f0eaeffd700 1 -- 172.21.2.101:0/2880604209 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0ea000c8a0 con 0x7f0eb01b0470 2021-02-10T11:30:35.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.181 7f0eb5ebc700 1 -- 172.21.2.101:0/2880604209 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0eb01b0470 msgr2=0x7f0eb01b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:35.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.181 7f0eb5ebc700 1 --2- 172.21.2.101:0/2880604209 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0eb01b0470 0x7f0eb01b0870 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7f0ea0006700 tx=0x7f0ea0009b80).stop 2021-02-10T11:30:35.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.182 7f0eb5ebc700 1 -- 172.21.2.101:0/2880604209 shutdown_connections 2021-02-10T11:30:35.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.182 7f0eb5ebc700 1 --2- 172.21.2.101:0/2880604209 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0eb00ef610 0x7f0eb00efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:35.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.182 7f0eb5ebc700 1 --2- 172.21.2.101:0/2880604209 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0eb01b0470 0x7f0eb01b0870 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:35.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.182 7f0eb5ebc700 1 -- 172.21.2.101:0/2880604209 >> 172.21.2.101:0/2880604209 conn(0x7f0eb00ce3e0 msgr2=0x7f0eb00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:35.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.182 7f0eb5ebc700 1 -- 172.21.2.101:0/2880604209 shutdown_connections 2021-02-10T11:30:35.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.182 7f0eb5ebc700 1 -- 172.21.2.101:0/2880604209 wait complete. 2021-02-10T11:30:35.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.197 7f0eb5ebc700 1 Processor -- start 2021-02-10T11:30:35.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.197 7f0eb5ebc700 1 -- start start 2021-02-10T11:30:35.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.197 7f0eb5ebc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0eb01a42e0 0x7f0eb009ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:35.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.197 7f0eb5ebc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0eb01a46e0 0x7f0eb009f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:35.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.197 7f0eb5ebc700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0eb008ad30 0x7f0eb008b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:35.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.197 7f0eb5ebc700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb008a7b0 con 0x7f0eb01a46e0 2021-02-10T11:30:35.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.197 7f0eb5ebc700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb00a65f0 con 0x7f0eb008ad30 2021-02-10T11:30:35.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.197 7f0eb5ebc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb008a430 con 0x7f0eb01a42e0 2021-02-10T11:30:35.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.197 7f0ea6ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0eb01a46e0 0x7f0eb009f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:35.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.197 7f0ea6ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0eb01a46e0 0x7f0eb009f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49188/0 (socket says 172.21.2.101:49188) 2021-02-10T11:30:35.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.197 7f0ea6ffd700 1 -- 172.21.2.101:0/1888366822 learned_addr learned my addr 172.21.2.101:0/1888366822 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:35.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.197 7f0eaffff700 1 --2- 172.21.2.101:0/1888366822 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0eb008ad30 0x7f0eb008b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:35.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.198 7f0eaf7fe700 1 --2- 172.21.2.101:0/1888366822 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0eb01a42e0 0x7f0eb009ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:35.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.198 7f0ea6ffd700 1 -- 172.21.2.101:0/1888366822 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0eb01a42e0 msgr2=0x7f0eb009ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:35.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.198 7f0ea6ffd700 1 --2- 172.21.2.101:0/1888366822 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0eb01a42e0 0x7f0eb009ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:35.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.198 7f0ea6ffd700 1 -- 172.21.2.101:0/1888366822 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0eb008ad30 msgr2=0x7f0eb008b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:35.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.198 7f0ea6ffd700 1 --2- 172.21.2.101:0/1888366822 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0eb008ad30 0x7f0eb008b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:35.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.198 7f0ea6ffd700 1 -- 172.21.2.101:0/1888366822 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb009fa50 con 0x7f0eb01a46e0 2021-02-10T11:30:35.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.198 7f0ea6ffd700 1 --2- 172.21.2.101:0/1888366822 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0eb01a46e0 0x7f0eb009f310 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7f0e980099e0 tx=0x7f0e9800bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:35.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.198 7f0eacff9700 1 -- 172.21.2.101:0/1888366822 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e9800c2f0 con 0x7f0eb01a46e0 2021-02-10T11:30:35.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.198 7f0eacff9700 1 -- 172.21.2.101:0/1888366822 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9800c5f0 con 0x7f0eb01a46e0 2021-02-10T11:30:35.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.198 7f0eb5ebc700 1 -- 172.21.2.101:0/1888366822 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb009fa50 con 0x7f0eb01a46e0 2021-02-10T11:30:35.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.198 7f0eacff9700 1 -- 172.21.2.101:0/1888366822 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e9800cad0 con 0x7f0eb01a46e0 2021-02-10T11:30:35.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.198 7f0eb5ebc700 1 -- 172.21.2.101:0/1888366822 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb009fa50 con 0x7f0eb01a46e0 2021-02-10T11:30:35.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.199 7f0eacff9700 1 -- 172.21.2.101:0/1888366822 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0e98005660 con 0x7f0eb01a46e0 2021-02-10T11:30:35.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.199 7f0eacff9700 1 --2- 172.21.2.101:0/1888366822 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e94018730 0x7f0e9401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:35.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.199 7f0eacff9700 1 -- 172.21.2.101:0/1888366822 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0e980329c0 con 0x7f0eb01a46e0 2021-02-10T11:30:35.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.199 7f0eaf7fe700 1 --2- 172.21.2.101:0/1888366822 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e94018730 0x7f0e9401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:35.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.199 7f0eaf7fe700 1 --2- 172.21.2.101:0/1888366822 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e94018730 0x7f0e9401ac10 crc :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:35.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.229 7f0eb5ebc700 1 -- 172.21.2.101:0/1888366822 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb0105ae0 con 0x7f0eb01a46e0 2021-02-10T11:30:35.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.232 7f0eacff9700 1 -- 172.21.2.101:0/1888366822 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0e9801f880 con 0x7f0eb01a46e0 2021-02-10T11:30:35.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.361 7f0eb5ebc700 1 -- 172.21.2.101:0/1888366822 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839295", "target": ["mgr", ""]}) v1 -- 0x7f0eb00f0b50 con 0x7f0e94018730 2021-02-10T11:30:35.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.372 7f0eacff9700 1 -- 172.21.2.101:0/1888366822 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f0ea0006430 con 0x7f0e94018730 2021-02-10T11:30:35.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.378 7f0eb5ebc700 1 -- 172.21.2.101:0/1888366822 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e94018730 msgr2=0x7f0e9401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:35.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.378 7f0eb5ebc700 1 --2- 172.21.2.101:0/1888366822 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e94018730 0x7f0e9401ac10 crc :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:35.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.378 7f0eb5ebc700 1 -- 172.21.2.101:0/1888366822 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0eb01a46e0 msgr2=0x7f0eb009f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:35.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.378 7f0eb5ebc700 1 --2- 172.21.2.101:0/1888366822 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0eb01a46e0 0x7f0eb009f310 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7f0e980099e0 tx=0x7f0e9800bd40).stop 2021-02-10T11:30:35.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.378 7f0eb5ebc700 1 -- 172.21.2.101:0/1888366822 shutdown_connections 2021-02-10T11:30:35.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.378 7f0eb5ebc700 1 --2- 172.21.2.101:0/1888366822 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0eb008ad30 0x7f0eb008b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:35.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.378 7f0eb5ebc700 1 --2- 172.21.2.101:0/1888366822 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0eb01a46e0 0x7f0eb009f310 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:35.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.378 7f0eb5ebc700 1 --2- 172.21.2.101:0/1888366822 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e94018730 0x7f0e9401ac10 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:35.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.378 7f0eb5ebc700 1 --2- 172.21.2.101:0/1888366822 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0eb01a42e0 0x7f0eb009ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:35.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.378 7f0eb5ebc700 1 -- 172.21.2.101:0/1888366822 >> 172.21.2.101:0/1888366822 conn(0x7f0eb0095a70 msgr2=0x7f0eb00bf050 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:35.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.378 7f0eb5ebc700 1 -- 172.21.2.101:0/1888366822 shutdown_connections 2021-02-10T11:30:35.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.378 7f0eb5ebc700 1 -- 172.21.2.101:0/1888366822 wait complete. 2021-02-10T11:30:35.397 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839296 2021-02-10T11:30:35.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.800 7fdb06a50700 1 Processor -- start 2021-02-10T11:30:35.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.800 7fdb06a50700 1 -- start start 2021-02-10T11:30:35.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.800 7fdb06a50700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdb000f05f0 0x7fdb000f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:35.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.800 7fdb06a50700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb001b0c70 con 0x7fdb001b0470 2021-02-10T11:30:35.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.800 7fdb06a50700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb000f0b50 con 0x7fdb001b0870 2021-02-10T11:30:35.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.800 7fdb06a50700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb000e5ea0 con 0x7fdb000f05f0 2021-02-10T11:30:35.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.801 7fdb0506d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdb000f05f0 0x7fdb000f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:35.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.801 7fdb0506d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdb000f05f0 0x7fdb000f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49196/0 (socket says 172.21.2.101:49196) 2021-02-10T11:30:35.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.801 7fdb0506d700 1 -- 172.21.2.101:0/1197467919 learned_addr learned my addr 172.21.2.101:0/1197467919 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:35.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.801 7fdb0506d700 1 -- 172.21.2.101:0/1197467919 >> v1:172.21.2.108:6789/0 conn(0x7fdb001b0870 legacy=0x7fdb000efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:35.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.801 7fdb0506d700 1 -- 172.21.2.101:0/1197467919 >> v1:172.21.2.101:6789/0 conn(0x7fdb001b0470 legacy=0x7fdb000ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:35.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.801 7fdb0506d700 1 -- 172.21.2.101:0/1197467919 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb000e52e0 con 0x7fdb000f05f0 2021-02-10T11:30:35.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.801 7fdb0506d700 1 --2- 172.21.2.101:0/1197467919 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb000f05f0 0x7fdb000f20e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7fdaf8006f00 tx=0x7fdaf8009e90).ready entity=mon.0 client_cookie=afa4fbf9b2a443fa server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:35.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.801 7fdaff7fe700 1 -- 172.21.2.101:0/1197467919 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdaf800b300 con 0x7fdb000f05f0 2021-02-10T11:30:35.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.801 7fdaff7fe700 1 -- 172.21.2.101:0/1197467919 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf800b580 con 0x7fdb000f05f0 2021-02-10T11:30:35.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.801 7fdaff7fe700 1 -- 172.21.2.101:0/1197467919 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdaf800b9d0 con 0x7fdb000f05f0 2021-02-10T11:30:35.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.801 7fdb06a50700 1 -- 172.21.2.101:0/1197467919 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb000f05f0 msgr2=0x7fdb000f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:35.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.801 7fdb06a50700 1 --2- 172.21.2.101:0/1197467919 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb000f05f0 0x7fdb000f20e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7fdaf8006f00 tx=0x7fdaf8009e90).stop 2021-02-10T11:30:35.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.802 7fdb06a50700 1 -- 172.21.2.101:0/1197467919 shutdown_connections 2021-02-10T11:30:35.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.802 7fdb06a50700 1 --2- 172.21.2.101:0/1197467919 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb000f05f0 0x7fdb000f20e0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:35.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.802 7fdb06a50700 1 -- 172.21.2.101:0/1197467919 >> 172.21.2.101:0/1197467919 conn(0x7fdb000ce3e0 msgr2=0x7fdb000e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:35.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.802 7fdb06a50700 1 -- 172.21.2.101:0/1197467919 shutdown_connections 2021-02-10T11:30:35.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.802 7fdb06a50700 1 -- 172.21.2.101:0/1197467919 wait complete. 2021-02-10T11:30:35.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.817 7fdb06a50700 1 Processor -- start 2021-02-10T11:30:35.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.817 7fdb06a50700 1 -- start start 2021-02-10T11:30:35.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.818 7fdb06a50700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb00087510 0x7fdb000aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:35.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.818 7fdb06a50700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb001c68f0 0x7fdb001b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:35.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.818 7fdb06a50700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdb0008c4d0 0x7fdb001b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:35.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.818 7fdb06a50700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb000a2d50 con 0x7fdb00087510 2021-02-10T11:30:35.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.818 7fdb06a50700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb000a2f70 con 0x7fdb0008c4d0 2021-02-10T11:30:35.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.818 7fdb06a50700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb000a3190 con 0x7fdb001c68f0 2021-02-10T11:30:35.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.818 7fdb0486c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb00087510 0x7fdb000aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:35.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.818 7fdb0486c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb00087510 0x7fdb000aedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49198/0 (socket says 172.21.2.101:49198) 2021-02-10T11:30:35.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.818 7fdb0486c700 1 -- 172.21.2.101:0/377391146 learned_addr learned my addr 172.21.2.101:0/377391146 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:35.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.818 7fdb0506d700 1 --2- 172.21.2.101:0/377391146 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdb0008c4d0 0x7fdb001b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:35.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.818 7fdafffff700 1 --2- 172.21.2.101:0/377391146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb001c68f0 0x7fdb001b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:35.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.818 7fdb0486c700 1 -- 172.21.2.101:0/377391146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb001c68f0 msgr2=0x7fdb001b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:35.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.818 7fdb0486c700 1 --2- 172.21.2.101:0/377391146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb001c68f0 0x7fdb001b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:35.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.818 7fdb0486c700 1 -- 172.21.2.101:0/377391146 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdb0008c4d0 msgr2=0x7fdb001b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:35.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.818 7fdb0486c700 1 --2- 172.21.2.101:0/377391146 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdb0008c4d0 0x7fdb001b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:35.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.818 7fdb0486c700 1 -- 172.21.2.101:0/377391146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb000a33b0 con 0x7fdb00087510 2021-02-10T11:30:35.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.818 7fdb0486c700 1 --2- 172.21.2.101:0/377391146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb00087510 0x7fdb000aedc0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7fdae8006f00 tx=0x7fdae800b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:35.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.819 7fdafd7fa700 1 -- 172.21.2.101:0/377391146 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdae8007070 con 0x7fdb00087510 2021-02-10T11:30:35.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.819 7fdafd7fa700 1 -- 172.21.2.101:0/377391146 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae8007330 con 0x7fdb00087510 2021-02-10T11:30:35.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.819 7fdafd7fa700 1 -- 172.21.2.101:0/377391146 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdae8007850 con 0x7fdb00087510 2021-02-10T11:30:35.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.819 7fdb06a50700 1 -- 172.21.2.101:0/377391146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb000a33b0 con 0x7fdb00087510 2021-02-10T11:30:35.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.819 7fdb06a50700 1 -- 172.21.2.101:0/377391146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb000a33b0 con 0x7fdb00087510 2021-02-10T11:30:35.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.819 7fdafd7fa700 1 -- 172.21.2.101:0/377391146 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdae8007000 con 0x7fdb00087510 2021-02-10T11:30:35.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.819 7fdafd7fa700 1 --2- 172.21.2.101:0/377391146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdae40186c0 0x7fdae401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:35.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.819 7fdafd7fa700 1 -- 172.21.2.101:0/377391146 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fdae80326c0 con 0x7fdb00087510 2021-02-10T11:30:35.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.820 7fdafffff700 1 --2- 172.21.2.101:0/377391146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdae40186c0 0x7fdae401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:35.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.820 7fdafffff700 1 --2- 172.21.2.101:0/377391146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdae40186c0 0x7fdae401abc0 crc :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:35.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.849 7fdb06a50700 1 -- 172.21.2.101:0/377391146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0007e6b0 con 0x7fdb00087510 2021-02-10T11:30:35.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.852 7fdafd7fa700 1 -- 172.21.2.101:0/377391146 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdae8007210 con 0x7fdb00087510 2021-02-10T11:30:35.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.979 7fdb06a50700 1 -- 172.21.2.101:0/377391146 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839296", "target": ["mgr", ""]}) v1 -- 0x7fdb0007e6b0 con 0x7fdae40186c0 2021-02-10T11:30:35.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.988 7fdafd7fa700 1 -- 172.21.2.101:0/377391146 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fdaf0000960 con 0x7fdae40186c0 2021-02-10T11:30:36.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.996 7fdb06a50700 1 -- 172.21.2.101:0/377391146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdae40186c0 msgr2=0x7fdae401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:36.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.996 7fdb06a50700 1 --2- 172.21.2.101:0/377391146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdae40186c0 0x7fdae401abc0 crc :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:36.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.996 7fdb06a50700 1 -- 172.21.2.101:0/377391146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb00087510 msgr2=0x7fdb000aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:36.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.996 7fdb06a50700 1 --2- 172.21.2.101:0/377391146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb00087510 0x7fdb000aedc0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7fdae8006f00 tx=0x7fdae800b940).stop 2021-02-10T11:30:36.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.996 7fdb06a50700 1 -- 172.21.2.101:0/377391146 shutdown_connections 2021-02-10T11:30:36.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.996 7fdb06a50700 1 --2- 172.21.2.101:0/377391146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdae40186c0 0x7fdae401abc0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:36.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.996 7fdb06a50700 1 --2- 172.21.2.101:0/377391146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb001c68f0 0x7fdb001b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:36.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.996 7fdb06a50700 1 --2- 172.21.2.101:0/377391146 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdb0008c4d0 0x7fdb001b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:36.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.996 7fdb06a50700 1 --2- 172.21.2.101:0/377391146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb00087510 0x7fdb000aedc0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:36.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.996 7fdb06a50700 1 -- 172.21.2.101:0/377391146 >> 172.21.2.101:0/377391146 conn(0x7fdb000886d0 msgr2=0x7fdb00081c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:36.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.996 7fdb06a50700 1 -- 172.21.2.101:0/377391146 shutdown_connections 2021-02-10T11:30:36.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:35.996 7fdb06a50700 1 -- 172.21.2.101:0/377391146 wait complete. 2021-02-10T11:30:36.016 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839297 2021-02-10T11:30:36.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.420 7f8c04daa700 1 Processor -- start 2021-02-10T11:30:36.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.420 7f8c04daa700 1 -- start start 2021-02-10T11:30:36.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.421 7f8c04daa700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c001b0470 0x7f8c001b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:36.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.421 7f8c04daa700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c000f0b50 con 0x7f8c000ef610 2021-02-10T11:30:36.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.421 7f8c04daa700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c000f24e0 con 0x7f8c000f20e0 2021-02-10T11:30:36.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.421 7f8c04daa700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c000e5ea0 con 0x7f8c001b0470 2021-02-10T11:30:36.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.421 7f8bfe312700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8c000ef610 0x7f8c000efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49700/0 (socket says 172.21.2.101:49700) 2021-02-10T11:30:36.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.421 7f8bfe312700 1 -- 172.21.2.101:0/377645546 learned_addr learned my addr 172.21.2.101:0/377645546 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:36.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.421 7f8bfeb13700 1 --2- 172.21.2.101:0/377645546 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c001b0470 0x7f8c001b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:36.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.421 7f8bfdb11700 1 -- 172.21.2.101:0/377645546 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853802531 0 0) 0x7f8bf0000c10 con 0x7f8c000f20e0 2021-02-10T11:30:36.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.421 7f8bfdb11700 1 -- 172.21.2.101:0/377645546 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8be8002fe0 con 0x7f8c000f20e0 2021-02-10T11:30:36.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.421 7f8bfdb11700 1 -- 172.21.2.101:0/377645546 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2198473883 0 0) 0x7f8bf8001060 con 0x7f8c000ef610 2021-02-10T11:30:36.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.421 7f8bfdb11700 1 -- 172.21.2.101:0/377645546 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8be8003020 con 0x7f8c000ef610 2021-02-10T11:30:36.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.421 7f8bfeb13700 1 -- 172.21.2.101:0/377645546 >> v1:172.21.2.108:6789/0 conn(0x7f8c000f20e0 legacy=0x7f8c000f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:36.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.421 7f8bfeb13700 1 -- 172.21.2.101:0/377645546 >> v1:172.21.2.101:6789/0 conn(0x7f8c000ef610 legacy=0x7f8c000efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:36.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.421 7f8bfeb13700 1 -- 172.21.2.101:0/377645546 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c000e52e0 con 0x7f8c001b0470 2021-02-10T11:30:36.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.422 7f8bfeb13700 1 --2- 172.21.2.101:0/377645546 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c001b0470 0x7f8c001b0870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f8bf4006650 tx=0x7f8bf4009a00).ready entity=mon.2 client_cookie=2efaa381561f3b6c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:36.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.422 7f8bfdb11700 1 -- 172.21.2.101:0/377645546 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8bf40009b0 con 0x7f8c001b0470 2021-02-10T11:30:36.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.422 7f8bfdb11700 1 -- 172.21.2.101:0/377645546 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf400c0c0 con 0x7f8c001b0470 2021-02-10T11:30:36.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.422 7f8bfdb11700 1 -- 172.21.2.101:0/377645546 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8bf400c5a0 con 0x7f8c001b0470 2021-02-10T11:30:36.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.422 7f8c04daa700 1 -- 172.21.2.101:0/377645546 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c001b0470 msgr2=0x7f8c001b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:36.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.422 7f8c04daa700 1 --2- 172.21.2.101:0/377645546 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c001b0470 0x7f8c001b0870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f8bf4006650 tx=0x7f8bf4009a00).stop 2021-02-10T11:30:36.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.423 7f8c04daa700 1 -- 172.21.2.101:0/377645546 shutdown_connections 2021-02-10T11:30:36.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.423 7f8c04daa700 1 --2- 172.21.2.101:0/377645546 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c001b0470 0x7f8c001b0870 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:36.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.423 7f8c04daa700 1 -- 172.21.2.101:0/377645546 >> 172.21.2.101:0/377645546 conn(0x7f8c000ce3e0 msgr2=0x7f8c000e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:36.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.423 7f8c04daa700 1 -- 172.21.2.101:0/377645546 shutdown_connections 2021-02-10T11:30:36.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.423 7f8c04daa700 1 -- 172.21.2.101:0/377645546 wait complete. 2021-02-10T11:30:36.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.437 7f8c04daa700 1 Processor -- start 2021-02-10T11:30:36.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.437 7f8c04daa700 1 -- start start 2021-02-10T11:30:36.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.438 7f8c04daa700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c000a0420 0x7f8c000a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:36.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.438 7f8c04daa700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c00189d00 0x7f8c0018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:36.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.438 7f8c04daa700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c0018a720 0x7f8c000b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:36.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.438 7f8c04daa700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c00189780 con 0x7f8c000a0420 2021-02-10T11:30:36.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.438 7f8c04daa700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0018ab20 con 0x7f8c00189d00 2021-02-10T11:30:36.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.438 7f8c04daa700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c000a1140 con 0x7f8c0018a720 2021-02-10T11:30:36.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.438 7f8bfeb13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c000a0420 0x7f8c000a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:36.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.438 7f8bfeb13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c000a0420 0x7f8c000a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49214/0 (socket says 172.21.2.101:49214) 2021-02-10T11:30:36.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.438 7f8bfeb13700 1 -- 172.21.2.101:0/2572461650 learned_addr learned my addr 172.21.2.101:0/2572461650 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:36.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.438 7f8bfeb13700 1 -- 172.21.2.101:0/2572461650 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c0018a720 msgr2=0x7f8c000b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:36.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.438 7f8bfeb13700 1 --2- 172.21.2.101:0/2572461650 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c0018a720 0x7f8c000b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:36.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.438 7f8bfeb13700 1 -- 172.21.2.101:0/2572461650 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c00189d00 msgr2=0x7f8c0018a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:36.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.438 7f8bfeb13700 1 --2- 172.21.2.101:0/2572461650 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c00189d00 0x7f8c0018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:36.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.438 7f8bfeb13700 1 -- 172.21.2.101:0/2572461650 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c000b8200 con 0x7f8c000a0420 2021-02-10T11:30:36.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.438 7f8bfeb13700 1 --2- 172.21.2.101:0/2572461650 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c000a0420 0x7f8c000a0820 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7f8bf400b640 tx=0x7f8bf400b9e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:36.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.439 7f8bef7fe700 1 -- 172.21.2.101:0/2572461650 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8bf400bdd0 con 0x7f8c000a0420 2021-02-10T11:30:36.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.439 7f8bef7fe700 1 -- 172.21.2.101:0/2572461650 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf4000c90 con 0x7f8c000a0420 2021-02-10T11:30:36.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.439 7f8bef7fe700 1 -- 172.21.2.101:0/2572461650 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8bf40022b0 con 0x7f8c000a0420 2021-02-10T11:30:36.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.439 7f8c04daa700 1 -- 172.21.2.101:0/2572461650 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c000b8200 con 0x7f8c000a0420 2021-02-10T11:30:36.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.439 7f8c04daa700 1 -- 172.21.2.101:0/2572461650 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c000b8790 con 0x7f8c000a0420 2021-02-10T11:30:36.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.439 7f8bef7fe700 1 -- 172.21.2.101:0/2572461650 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8bf40052e0 con 0x7f8c000a0420 2021-02-10T11:30:36.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.439 7f8bef7fe700 1 --2- 172.21.2.101:0/2572461650 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8be40186c0 0x7f8be401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:36.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.439 7f8bef7fe700 1 -- 172.21.2.101:0/2572461650 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8bf40325b0 con 0x7f8c000a0420 2021-02-10T11:30:36.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.440 7f8bfe312700 1 --2- 172.21.2.101:0/2572461650 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8be40186c0 0x7f8be401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:36.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.440 7f8bfe312700 1 --2- 172.21.2.101:0/2572461650 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8be40186c0 0x7f8be401abc0 crc :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:36.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.470 7f8c04daa700 1 -- 172.21.2.101:0/2572461650 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c000a1360 con 0x7f8c000a0420 2021-02-10T11:30:36.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.473 7f8bef7fe700 1 -- 172.21.2.101:0/2572461650 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8bf4025fe0 con 0x7f8c000a0420 2021-02-10T11:30:36.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.599 7f8c04daa700 1 -- 172.21.2.101:0/2572461650 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839297", "target": ["mgr", ""]}) v1 -- 0x7f8c000b9020 con 0x7f8be40186c0 2021-02-10T11:30:36.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.610 7f8bef7fe700 1 -- 172.21.2.101:0/2572461650 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f8bf8004720 con 0x7f8be40186c0 2021-02-10T11:30:36.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.617 7f8c04daa700 1 -- 172.21.2.101:0/2572461650 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8be40186c0 msgr2=0x7f8be401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:36.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.617 7f8c04daa700 1 --2- 172.21.2.101:0/2572461650 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8be40186c0 0x7f8be401abc0 crc :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:36.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.617 7f8c04daa700 1 -- 172.21.2.101:0/2572461650 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c000a0420 msgr2=0x7f8c000a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:36.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.617 7f8c04daa700 1 --2- 172.21.2.101:0/2572461650 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c000a0420 0x7f8c000a0820 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7f8bf400b640 tx=0x7f8bf400b9e0).stop 2021-02-10T11:30:36.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.617 7f8c04daa700 1 -- 172.21.2.101:0/2572461650 shutdown_connections 2021-02-10T11:30:36.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.617 7f8c04daa700 1 --2- 172.21.2.101:0/2572461650 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8be40186c0 0x7f8be401abc0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:36.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.617 7f8c04daa700 1 --2- 172.21.2.101:0/2572461650 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c0018a720 0x7f8c000b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:36.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.617 7f8c04daa700 1 --2- 172.21.2.101:0/2572461650 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c00189d00 0x7f8c0018a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:36.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.617 7f8c04daa700 1 --2- 172.21.2.101:0/2572461650 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c000a0420 0x7f8c000a0820 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:36.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.617 7f8c04daa700 1 -- 172.21.2.101:0/2572461650 >> 172.21.2.101:0/2572461650 conn(0x7f8c00094e10 msgr2=0x7f8c000a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:36.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.617 7f8c04daa700 1 -- 172.21.2.101:0/2572461650 shutdown_connections 2021-02-10T11:30:36.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:36.617 7f8c04daa700 1 -- 172.21.2.101:0/2572461650 wait complete. 2021-02-10T11:30:36.637 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839298 2021-02-10T11:30:37.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.039 7efd25589700 1 Processor -- start 2021-02-10T11:30:37.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.039 7efd25589700 1 -- start start 2021-02-10T11:30:37.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.040 7efd25589700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd200f05f0 0x7efd200f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:37.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.040 7efd25589700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd201b0c70 con 0x7efd201b0470 2021-02-10T11:30:37.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.040 7efd25589700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd200f0b50 con 0x7efd201b0870 2021-02-10T11:30:37.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.040 7efd25589700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd200e5ea0 con 0x7efd200f05f0 2021-02-10T11:30:37.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.040 7efd1f2bf700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7efd201b0470 0x7efd200ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49712/0 (socket says 172.21.2.101:49712) 2021-02-10T11:30:37.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.040 7efd1f2bf700 1 -- 172.21.2.101:0/2108835961 learned_addr learned my addr 172.21.2.101:0/2108835961 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:37.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.040 7efd1fac0700 1 --2- 172.21.2.101:0/2108835961 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd200f05f0 0x7efd200f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:37.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.040 7efd1e2bd700 1 -- 172.21.2.101:0/2108835961 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1635357656 0 0) 0x7efd10001180 con 0x7efd201b0470 2021-02-10T11:30:37.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.040 7efd1e2bd700 1 -- 172.21.2.101:0/2108835961 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd0c002fe0 con 0x7efd201b0470 2021-02-10T11:30:37.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.041 7efd1e2bd700 1 -- 172.21.2.101:0/2108835961 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814770907 0 0) 0x7efd08000fe0 con 0x7efd201b0870 2021-02-10T11:30:37.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.041 7efd1fac0700 1 -- 172.21.2.101:0/2108835961 >> v1:172.21.2.108:6789/0 conn(0x7efd201b0870 legacy=0x7efd200efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:37.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.041 7efd1fac0700 1 -- 172.21.2.101:0/2108835961 >> v1:172.21.2.101:6789/0 conn(0x7efd201b0470 legacy=0x7efd200ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:37.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.041 7efd1fac0700 1 -- 172.21.2.101:0/2108835961 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd200d3330 con 0x7efd200f05f0 2021-02-10T11:30:37.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.041 7efd1fac0700 1 --2- 172.21.2.101:0/2108835961 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd200f05f0 0x7efd200f20e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7efd18006ad0 tx=0x7efd18009ad0).ready entity=mon.2 client_cookie=492593ec5f90197e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:37.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.041 7efd1e2bd700 1 -- 172.21.2.101:0/2108835961 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd180022e0 con 0x7efd200f05f0 2021-02-10T11:30:37.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.041 7efd1e2bd700 1 -- 172.21.2.101:0/2108835961 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd18006410 con 0x7efd200f05f0 2021-02-10T11:30:37.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.041 7efd1e2bd700 1 -- 172.21.2.101:0/2108835961 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd18002560 con 0x7efd200f05f0 2021-02-10T11:30:37.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.041 7efd25589700 1 -- 172.21.2.101:0/2108835961 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd200f05f0 msgr2=0x7efd200f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:37.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.041 7efd25589700 1 --2- 172.21.2.101:0/2108835961 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd200f05f0 0x7efd200f20e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7efd18006ad0 tx=0x7efd18009ad0).stop 2021-02-10T11:30:37.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.042 7efd25589700 1 -- 172.21.2.101:0/2108835961 shutdown_connections 2021-02-10T11:30:37.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.042 7efd25589700 1 --2- 172.21.2.101:0/2108835961 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd200f05f0 0x7efd200f20e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:37.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.042 7efd25589700 1 -- 172.21.2.101:0/2108835961 >> 172.21.2.101:0/2108835961 conn(0x7efd200ce3e0 msgr2=0x7efd200e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:37.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.042 7efd25589700 1 -- 172.21.2.101:0/2108835961 shutdown_connections 2021-02-10T11:30:37.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.042 7efd25589700 1 -- 172.21.2.101:0/2108835961 wait complete. 2021-02-10T11:30:37.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.057 7efd25589700 1 Processor -- start 2021-02-10T11:30:37.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.057 7efd25589700 1 -- start start 2021-02-10T11:30:37.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.057 7efd25589700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd20190920 0x7efd2008f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:37.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.057 7efd25589700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd20190d20 0x7efd2008fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:37.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.057 7efd25589700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd20193270 0x7efd20193670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:37.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.057 7efd25589700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd20090470 con 0x7efd20193270 2021-02-10T11:30:37.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.057 7efd25589700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd20193c90 con 0x7efd20190d20 2021-02-10T11:30:37.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.057 7efd25589700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd20193e70 con 0x7efd20190920 2021-02-10T11:30:37.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.058 7efd1fac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd20193270 0x7efd20193670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:37.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.058 7efd1fac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd20193270 0x7efd20193670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49224/0 (socket says 172.21.2.101:49224) 2021-02-10T11:30:37.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.058 7efd1fac0700 1 -- 172.21.2.101:0/2650522227 learned_addr learned my addr 172.21.2.101:0/2650522227 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:37.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.058 7efd1eabe700 1 --2- 172.21.2.101:0/2650522227 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd20190d20 0x7efd2008fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:37.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.058 7efd1f2bf700 1 --2- 172.21.2.101:0/2650522227 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd20190920 0x7efd2008f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:37.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.058 7efd1fac0700 1 -- 172.21.2.101:0/2650522227 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd20190920 msgr2=0x7efd2008f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:37.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.058 7efd1fac0700 1 --2- 172.21.2.101:0/2650522227 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd20190920 0x7efd2008f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:37.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.058 7efd1fac0700 1 -- 172.21.2.101:0/2650522227 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd20190d20 msgr2=0x7efd2008fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:37.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.058 7efd1fac0700 1 --2- 172.21.2.101:0/2650522227 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd20190d20 0x7efd2008fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:37.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.058 7efd1fac0700 1 -- 172.21.2.101:0/2650522227 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd20194090 con 0x7efd20193270 2021-02-10T11:30:37.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.058 7efd1fac0700 1 --2- 172.21.2.101:0/2650522227 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd20193270 0x7efd20193670 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7efd18009670 tx=0x7efd1800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:37.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.059 7efd17fff700 1 -- 172.21.2.101:0/2650522227 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd1800bd80 con 0x7efd20193270 2021-02-10T11:30:37.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.059 7efd17fff700 1 -- 172.21.2.101:0/2650522227 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd18002080 con 0x7efd20193270 2021-02-10T11:30:37.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.059 7efd17fff700 1 -- 172.21.2.101:0/2650522227 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd18002560 con 0x7efd20193270 2021-02-10T11:30:37.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.059 7efd25589700 1 -- 172.21.2.101:0/2650522227 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd20194090 con 0x7efd20193270 2021-02-10T11:30:37.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.059 7efd25589700 1 -- 172.21.2.101:0/2650522227 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd201920b0 con 0x7efd20193270 2021-02-10T11:30:37.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.059 7efd17fff700 1 -- 172.21.2.101:0/2650522227 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7efd18002260 con 0x7efd20193270 2021-02-10T11:30:37.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.059 7efd17fff700 1 --2- 172.21.2.101:0/2650522227 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd040186c0 0x7efd0401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:37.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.059 7efd17fff700 1 -- 172.21.2.101:0/2650522227 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7efd18032580 con 0x7efd20193270 2021-02-10T11:30:37.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.059 7efd1f2bf700 1 --2- 172.21.2.101:0/2650522227 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd040186c0 0x7efd0401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:37.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.060 7efd1f2bf700 1 --2- 172.21.2.101:0/2650522227 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd040186c0 0x7efd0401abc0 crc :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:37.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.089 7efd25589700 1 -- 172.21.2.101:0/2650522227 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd20105af0 con 0x7efd20193270 2021-02-10T11:30:37.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.092 7efd17fff700 1 -- 172.21.2.101:0/2650522227 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7efd180093e0 con 0x7efd20193270 2021-02-10T11:30:37.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.221 7efd25589700 1 -- 172.21.2.101:0/2650522227 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839298", "target": ["mgr", ""]}) v1 -- 0x7efd201920b0 con 0x7efd040186c0 2021-02-10T11:30:37.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.231 7efd17fff700 1 -- 172.21.2.101:0/2650522227 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7efd10000ce0 con 0x7efd040186c0 2021-02-10T11:30:37.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.238 7efd25589700 1 -- 172.21.2.101:0/2650522227 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd040186c0 msgr2=0x7efd0401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:37.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.238 7efd25589700 1 --2- 172.21.2.101:0/2650522227 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd040186c0 0x7efd0401abc0 crc :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:37.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.238 7efd25589700 1 -- 172.21.2.101:0/2650522227 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd20193270 msgr2=0x7efd20193670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:37.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.238 7efd25589700 1 --2- 172.21.2.101:0/2650522227 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd20193270 0x7efd20193670 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7efd18009670 tx=0x7efd1800b990).stop 2021-02-10T11:30:37.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.238 7efd25589700 1 -- 172.21.2.101:0/2650522227 shutdown_connections 2021-02-10T11:30:37.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.238 7efd25589700 1 --2- 172.21.2.101:0/2650522227 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd20193270 0x7efd20193670 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:37.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.238 7efd25589700 1 --2- 172.21.2.101:0/2650522227 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd20190d20 0x7efd2008fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:37.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.238 7efd25589700 1 --2- 172.21.2.101:0/2650522227 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd040186c0 0x7efd0401abc0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:37.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.238 7efd25589700 1 --2- 172.21.2.101:0/2650522227 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd20190920 0x7efd2008f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:37.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.238 7efd25589700 1 -- 172.21.2.101:0/2650522227 >> 172.21.2.101:0/2650522227 conn(0x7efd200bf4e0 msgr2=0x7efd200be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:37.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.239 7efd25589700 1 -- 172.21.2.101:0/2650522227 shutdown_connections 2021-02-10T11:30:37.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.239 7efd25589700 1 -- 172.21.2.101:0/2650522227 wait complete. 2021-02-10T11:30:37.258 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839299 2021-02-10T11:30:37.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.662 7ffb39734700 1 Processor -- start 2021-02-10T11:30:37.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.662 7ffb39734700 1 -- start start 2021-02-10T11:30:37.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.663 7ffb39734700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffb341b0870 0x7ffb340f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:37.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.663 7ffb39734700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ffb340efe00 0x7ffb340f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:37.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.663 7ffb39734700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb341b0c70 con 0x7ffb341b0470 2021-02-10T11:30:37.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.663 7ffb39734700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb340f0b50 con 0x7ffb340efe00 2021-02-10T11:30:37.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.663 7ffb39734700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb340f0820 con 0x7ffb341b0870 2021-02-10T11:30:37.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.663 7ffb32d48700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffb341b0870 0x7ffb340f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:37.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.663 7ffb32d48700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffb341b0870 0x7ffb340f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:41962/0 (socket says 172.21.2.101:41962) 2021-02-10T11:30:37.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.663 7ffb32d48700 1 -- 172.21.2.101:0/1107468189 learned_addr learned my addr 172.21.2.101:0/1107468189 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:37.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.663 7ffb33d4a700 1 --2- 172.21.2.101:0/1107468189 >> v2:172.21.2.108:3300/0 conn(0x7ffb340efe00 0x7ffb340f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:37.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.664 7ffb32d48700 1 -- 172.21.2.101:0/1107468189 >> v2:172.21.2.108:3300/0 conn(0x7ffb340efe00 msgr2=0x7ffb340f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:37.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.664 7ffb32d48700 1 --2- 172.21.2.101:0/1107468189 >> v2:172.21.2.108:3300/0 conn(0x7ffb340efe00 0x7ffb340f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:37.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.664 7ffb32d48700 1 -- 172.21.2.101:0/1107468189 >> v1:172.21.2.108:6789/0 conn(0x7ffb341b0470 legacy=0x7ffb340ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:37.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.664 7ffb32d48700 1 -- 172.21.2.101:0/1107468189 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb340e52e0 con 0x7ffb341b0870 2021-02-10T11:30:37.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.664 7ffb33d4a700 1 --2- 172.21.2.101:0/1107468189 >> v2:172.21.2.108:3300/0 conn(0x7ffb340efe00 0x7ffb340f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:37.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.664 7ffb32d48700 1 --2- 172.21.2.101:0/1107468189 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffb341b0870 0x7ffb340f20e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7ffb24006700 tx=0x7ffb24009b80).ready entity=mon.2 client_cookie=f1f056e36bf35291 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:37.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.664 7ffb32547700 1 -- 172.21.2.101:0/1107468189 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffb2400c0c0 con 0x7ffb341b0870 2021-02-10T11:30:37.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.664 7ffb32547700 1 -- 172.21.2.101:0/1107468189 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2400c340 con 0x7ffb341b0870 2021-02-10T11:30:37.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.664 7ffb32547700 1 -- 172.21.2.101:0/1107468189 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffb2400c860 con 0x7ffb341b0870 2021-02-10T11:30:37.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.664 7ffb39734700 1 -- 172.21.2.101:0/1107468189 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffb341b0870 msgr2=0x7ffb340f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:37.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.664 7ffb39734700 1 --2- 172.21.2.101:0/1107468189 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffb341b0870 0x7ffb340f20e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7ffb24006700 tx=0x7ffb24009b80).stop 2021-02-10T11:30:37.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.665 7ffb39734700 1 -- 172.21.2.101:0/1107468189 shutdown_connections 2021-02-10T11:30:37.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.665 7ffb39734700 1 --2- 172.21.2.101:0/1107468189 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffb341b0870 0x7ffb340f20e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:37.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.665 7ffb39734700 1 --2- 172.21.2.101:0/1107468189 >> v2:172.21.2.108:3300/0 conn(0x7ffb340efe00 0x7ffb340f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:37.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.665 7ffb39734700 1 -- 172.21.2.101:0/1107468189 >> 172.21.2.101:0/1107468189 conn(0x7ffb340ce3e0 msgr2=0x7ffb340e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:37.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.665 7ffb39734700 1 -- 172.21.2.101:0/1107468189 shutdown_connections 2021-02-10T11:30:37.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.665 7ffb39734700 1 -- 172.21.2.101:0/1107468189 wait complete. 2021-02-10T11:30:37.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.680 7ffb39734700 1 Processor -- start 2021-02-10T11:30:37.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.680 7ffb39734700 1 -- start start 2021-02-10T11:30:37.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.680 7ffb39734700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffb34190920 0x7ffb340b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:37.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.680 7ffb39734700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffb340b8200 0x7ffb340b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:37.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.680 7ffb39734700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffb340c1d80 0x7ffb340c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:37.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.681 7ffb39734700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb34190d20 con 0x7ffb34190920 2021-02-10T11:30:37.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.681 7ffb39734700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb340b8c20 con 0x7ffb340c1d80 2021-02-10T11:30:37.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.681 7ffb39734700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb340c27a0 con 0x7ffb340b8200 2021-02-10T11:30:37.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.681 7ffb33549700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffb34190920 0x7ffb340b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:37.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.681 7ffb33549700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffb34190920 0x7ffb340b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49234/0 (socket says 172.21.2.101:49234) 2021-02-10T11:30:37.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.681 7ffb33549700 1 -- 172.21.2.101:0/459651127 learned_addr learned my addr 172.21.2.101:0/459651127 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:37.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.681 7ffb32d48700 1 --2- 172.21.2.101:0/459651127 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffb340b8200 0x7ffb340b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:37.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.681 7ffb33d4a700 1 --2- 172.21.2.101:0/459651127 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffb340c1d80 0x7ffb340c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:37.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.681 7ffb33549700 1 -- 172.21.2.101:0/459651127 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffb340b8200 msgr2=0x7ffb340b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:37.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.681 7ffb33549700 1 --2- 172.21.2.101:0/459651127 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffb340b8200 0x7ffb340b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:37.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.681 7ffb33549700 1 -- 172.21.2.101:0/459651127 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffb340c1d80 msgr2=0x7ffb340c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:37.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.681 7ffb33549700 1 --2- 172.21.2.101:0/459651127 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffb340c1d80 0x7ffb340c2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:37.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.681 7ffb33549700 1 -- 172.21.2.101:0/459651127 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb340c29c0 con 0x7ffb34190920 2021-02-10T11:30:37.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.681 7ffb33549700 1 --2- 172.21.2.101:0/459651127 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffb34190920 0x7ffb340b7be0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7ffb1c00b5b0 tx=0x7ffb1c00b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:37.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.681 7ffb2bfff700 1 -- 172.21.2.101:0/459651127 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffb1c00bd40 con 0x7ffb34190920 2021-02-10T11:30:37.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.681 7ffb2bfff700 1 -- 172.21.2.101:0/459651127 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1c006240 con 0x7ffb34190920 2021-02-10T11:30:37.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.681 7ffb39734700 1 -- 172.21.2.101:0/459651127 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb340c29c0 con 0x7ffb34190920 2021-02-10T11:30:37.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.681 7ffb2bfff700 1 -- 172.21.2.101:0/459651127 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffb1c005690 con 0x7ffb34190920 2021-02-10T11:30:37.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.681 7ffb39734700 1 -- 172.21.2.101:0/459651127 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb340c29c0 con 0x7ffb34190920 2021-02-10T11:30:37.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.682 7ffb2bfff700 1 -- 172.21.2.101:0/459651127 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ffb1c0054a0 con 0x7ffb34190920 2021-02-10T11:30:37.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.682 7ffb2bfff700 1 --2- 172.21.2.101:0/459651127 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffb18018730 0x7ffb1801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:37.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.682 7ffb2bfff700 1 -- 172.21.2.101:0/459651127 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ffb1c032650 con 0x7ffb34190920 2021-02-10T11:30:37.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.682 7ffb32d48700 1 --2- 172.21.2.101:0/459651127 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffb18018730 0x7ffb1801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:37.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.682 7ffb32d48700 1 --2- 172.21.2.101:0/459651127 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffb18018730 0x7ffb1801ac10 crc :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:37.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.712 7ffb39734700 1 -- 172.21.2.101:0/459651127 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb34105ae0 con 0x7ffb34190920 2021-02-10T11:30:37.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.714 7ffb2bfff700 1 -- 172.21.2.101:0/459651127 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ffb1c000c60 con 0x7ffb34190920 2021-02-10T11:30:37.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.840 7ffb39734700 1 -- 172.21.2.101:0/459651127 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839299", "target": ["mgr", ""]}) v1 -- 0x7ffb340b8c20 con 0x7ffb18018730 2021-02-10T11:30:37.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.850 7ffb2bfff700 1 -- 172.21.2.101:0/459651127 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7ffb24006430 con 0x7ffb18018730 2021-02-10T11:30:37.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.857 7ffb39734700 1 -- 172.21.2.101:0/459651127 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffb18018730 msgr2=0x7ffb1801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:37.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.857 7ffb39734700 1 --2- 172.21.2.101:0/459651127 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffb18018730 0x7ffb1801ac10 crc :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:37.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.857 7ffb39734700 1 -- 172.21.2.101:0/459651127 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffb34190920 msgr2=0x7ffb340b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:37.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.857 7ffb39734700 1 --2- 172.21.2.101:0/459651127 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffb34190920 0x7ffb340b7be0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7ffb1c00b5b0 tx=0x7ffb1c00b950).stop 2021-02-10T11:30:37.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.857 7ffb39734700 1 -- 172.21.2.101:0/459651127 shutdown_connections 2021-02-10T11:30:37.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.857 7ffb39734700 1 --2- 172.21.2.101:0/459651127 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffb18018730 0x7ffb1801ac10 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:37.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.857 7ffb39734700 1 --2- 172.21.2.101:0/459651127 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffb340b8200 0x7ffb340b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:37.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.857 7ffb39734700 1 --2- 172.21.2.101:0/459651127 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffb340c1d80 0x7ffb340c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:37.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.857 7ffb39734700 1 --2- 172.21.2.101:0/459651127 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffb34190920 0x7ffb340b7be0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:37.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.857 7ffb39734700 1 -- 172.21.2.101:0/459651127 >> 172.21.2.101:0/459651127 conn(0x7ffb34094e10 msgr2=0x7ffb340a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:37.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.858 7ffb39734700 1 -- 172.21.2.101:0/459651127 shutdown_connections 2021-02-10T11:30:37.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:37.858 7ffb39734700 1 -- 172.21.2.101:0/459651127 wait complete. 2021-02-10T11:30:37.877 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839300 2021-02-10T11:30:38.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.284 7f46613cc700 1 Processor -- start 2021-02-10T11:30:38.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.284 7f46613cc700 1 -- start start 2021-02-10T11:30:38.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.284 7f46613cc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f465c1b0870 0x7f465c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:38.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.284 7f46613cc700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f465c0efe00 0x7f465c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:38.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.284 7f46613cc700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f465c1b0c70 con 0x7f465c1b0470 2021-02-10T11:30:38.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.284 7f46613cc700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f465c0f0b50 con 0x7f465c0efe00 2021-02-10T11:30:38.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.284 7f46613cc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465c0f0820 con 0x7f465c1b0870 2021-02-10T11:30:38.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.284 7f465b87e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f465c0efe00 0x7f465c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:38.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.285 7f465287c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f465c1b0870 0x7f465c0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:38.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.285 7f465b07d700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f465c1b0470 0x7f465c0ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:43146/0 (socket says 172.21.2.101:43146) 2021-02-10T11:30:38.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.285 7f465b07d700 1 -- 172.21.2.101:0/139805999 learned_addr learned my addr 172.21.2.101:0/139805999 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:38.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.285 7f465a87c700 1 -- 172.21.2.101:0/139805999 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3114385553 0 0) 0x7f464c0010c0 con 0x7f465c1b0470 2021-02-10T11:30:38.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.285 7f465287c700 1 -- 172.21.2.101:0/139805999 >> v2:172.21.2.108:3300/0 conn(0x7f465c0efe00 msgr2=0x7f465c0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:38.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.285 7f465287c700 1 --2- 172.21.2.101:0/139805999 >> v2:172.21.2.108:3300/0 conn(0x7f465c0efe00 0x7f465c0f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:38.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.285 7f465287c700 1 -- 172.21.2.101:0/139805999 >> v1:172.21.2.108:6789/0 conn(0x7f465c1b0470 legacy=0x7f465c0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:38.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.285 7f465287c700 1 -- 172.21.2.101:0/139805999 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c0e52e0 con 0x7f465c1b0870 2021-02-10T11:30:38.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.285 7f465287c700 1 --2- 172.21.2.101:0/139805999 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f465c1b0870 0x7f465c0f20e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f4644006650 tx=0x7f4644009a80).ready entity=mon.2 client_cookie=b37fa66cb512a7d2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:38.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.285 7f465a87c700 1 -- 172.21.2.101:0/139805999 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f46440009b0 con 0x7f465c1b0870 2021-02-10T11:30:38.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.285 7f465a87c700 1 -- 172.21.2.101:0/139805999 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f464400c0c0 con 0x7f465c1b0870 2021-02-10T11:30:38.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.285 7f465a87c700 1 -- 172.21.2.101:0/139805999 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f464400c520 con 0x7f465c1b0870 2021-02-10T11:30:38.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.285 7f46613cc700 1 -- 172.21.2.101:0/139805999 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f465c1b0870 msgr2=0x7f465c0f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:38.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.285 7f46613cc700 1 --2- 172.21.2.101:0/139805999 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f465c1b0870 0x7f465c0f20e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f4644006650 tx=0x7f4644009a80).stop 2021-02-10T11:30:38.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.286 7f46613cc700 1 -- 172.21.2.101:0/139805999 shutdown_connections 2021-02-10T11:30:38.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.286 7f46613cc700 1 --2- 172.21.2.101:0/139805999 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f465c1b0870 0x7f465c0f20e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:38.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.286 7f46613cc700 1 --2- 172.21.2.101:0/139805999 >> v2:172.21.2.108:3300/0 conn(0x7f465c0efe00 0x7f465c0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:38.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.286 7f46613cc700 1 -- 172.21.2.101:0/139805999 >> 172.21.2.101:0/139805999 conn(0x7f465c0ce3e0 msgr2=0x7f465c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:38.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.286 7f46613cc700 1 -- 172.21.2.101:0/139805999 shutdown_connections 2021-02-10T11:30:38.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.286 7f46613cc700 1 -- 172.21.2.101:0/139805999 wait complete. 2021-02-10T11:30:38.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.301 7f46613cc700 1 Processor -- start 2021-02-10T11:30:38.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.301 7f46613cc700 1 -- start start 2021-02-10T11:30:38.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.301 7f46613cc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f465c105d10 0x7f465c09ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:38.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.301 7f46613cc700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f465c106110 0x7f465c0b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:38.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.301 7f46613cc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f465c09e0a0 0x7f465c0b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:38.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.301 7f46613cc700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465c09e640 con 0x7f465c09e0a0 2021-02-10T11:30:38.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.301 7f46613cc700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465c1a1620 con 0x7f465c106110 2021-02-10T11:30:38.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.301 7f46613cc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465c0b3a70 con 0x7f465c105d10 2021-02-10T11:30:38.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.301 7f465b87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f465c09e0a0 0x7f465c0b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:38.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.301 7f465b87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f465c09e0a0 0x7f465c0b3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49252/0 (socket says 172.21.2.101:49252) 2021-02-10T11:30:38.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.301 7f465b87e700 1 -- 172.21.2.101:0/2450637221 learned_addr learned my addr 172.21.2.101:0/2450637221 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:38.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.301 7f465b07d700 1 --2- 172.21.2.101:0/2450637221 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f465c105d10 0x7f465c09ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:38.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.301 7f465287c700 1 --2- 172.21.2.101:0/2450637221 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f465c106110 0x7f465c0b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:38.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.302 7f465287c700 1 -- 172.21.2.101:0/2450637221 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f465c105d10 msgr2=0x7f465c09ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:38.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.302 7f465287c700 1 --2- 172.21.2.101:0/2450637221 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f465c105d10 0x7f465c09ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:38.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.302 7f465287c700 1 -- 172.21.2.101:0/2450637221 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f465c09e0a0 msgr2=0x7f465c0b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:38.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.302 7f465287c700 1 --2- 172.21.2.101:0/2450637221 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f465c09e0a0 0x7f465c0b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:38.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.302 7f465287c700 1 -- 172.21.2.101:0/2450637221 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c0b3660 con 0x7f465c106110 2021-02-10T11:30:38.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.302 7f465287c700 1 --2- 172.21.2.101:0/2450637221 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f465c106110 0x7f465c0b2a20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f464400b530 tx=0x7f464400b8d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:38.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.302 7f4658878700 1 -- 172.21.2.101:0/2450637221 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f46440093a0 con 0x7f465c106110 2021-02-10T11:30:38.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.302 7f4658878700 1 -- 172.21.2.101:0/2450637221 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f464400bcc0 con 0x7f465c106110 2021-02-10T11:30:38.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.302 7f46613cc700 1 -- 172.21.2.101:0/2450637221 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f465c0b3660 con 0x7f465c106110 2021-02-10T11:30:38.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.302 7f46613cc700 1 -- 172.21.2.101:0/2450637221 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465c0b3660 con 0x7f465c106110 2021-02-10T11:30:38.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.302 7f4658878700 1 -- 172.21.2.101:0/2450637221 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f46440022b0 con 0x7f465c106110 2021-02-10T11:30:38.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.302 7f4658878700 1 -- 172.21.2.101:0/2450637221 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f46440052e0 con 0x7f465c106110 2021-02-10T11:30:38.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.303 7f4658878700 1 --2- 172.21.2.101:0/2450637221 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4640018730 0x7f464001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:38.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.303 7f4658878700 1 -- 172.21.2.101:0/2450637221 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f46440326d0 con 0x7f465c106110 2021-02-10T11:30:38.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.303 7f465b07d700 1 --2- 172.21.2.101:0/2450637221 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4640018730 0x7f464001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:38.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.303 7f465b07d700 1 --2- 172.21.2.101:0/2450637221 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4640018730 0x7f464001abc0 crc :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:38.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.333 7f46613cc700 1 -- 172.21.2.101:0/2450637221 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f465c1a1620 con 0x7f465c106110 2021-02-10T11:30:38.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.336 7f4658878700 1 -- 172.21.2.101:0/2450637221 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f46440052e0 con 0x7f465c106110 2021-02-10T11:30:38.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.465 7f46613cc700 1 -- 172.21.2.101:0/2450637221 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839300", "target": ["mgr", ""]}) v1 -- 0x7f465c1a1620 con 0x7f4640018730 2021-02-10T11:30:38.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.475 7f4658878700 1 -- 172.21.2.101:0/2450637221 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f464c000c10 con 0x7f4640018730 2021-02-10T11:30:38.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.482 7f46613cc700 1 -- 172.21.2.101:0/2450637221 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4640018730 msgr2=0x7f464001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:38.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.482 7f46613cc700 1 --2- 172.21.2.101:0/2450637221 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4640018730 0x7f464001abc0 crc :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:38.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.482 7f46613cc700 1 -- 172.21.2.101:0/2450637221 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f465c106110 msgr2=0x7f465c0b2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:38.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.482 7f46613cc700 1 --2- 172.21.2.101:0/2450637221 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f465c106110 0x7f465c0b2a20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f464400b530 tx=0x7f464400b8d0).stop 2021-02-10T11:30:38.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.482 7f46613cc700 1 -- 172.21.2.101:0/2450637221 shutdown_connections 2021-02-10T11:30:38.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.482 7f46613cc700 1 --2- 172.21.2.101:0/2450637221 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f465c09e0a0 0x7f465c0b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:38.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.482 7f46613cc700 1 --2- 172.21.2.101:0/2450637221 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f465c106110 0x7f465c0b2a20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:38.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.482 7f46613cc700 1 --2- 172.21.2.101:0/2450637221 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4640018730 0x7f464001abc0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:38.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.482 7f46613cc700 1 --2- 172.21.2.101:0/2450637221 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f465c105d10 0x7f465c09ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:38.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.482 7f46613cc700 1 -- 172.21.2.101:0/2450637221 >> 172.21.2.101:0/2450637221 conn(0x7f465c0c3c70 msgr2=0x7f465c0bd640 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:38.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.483 7f46613cc700 1 -- 172.21.2.101:0/2450637221 shutdown_connections 2021-02-10T11:30:38.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.483 7f46613cc700 1 -- 172.21.2.101:0/2450637221 wait complete. 2021-02-10T11:30:38.502 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839301 2021-02-10T11:30:38.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.905 7f66d84a7700 1 Processor -- start 2021-02-10T11:30:38.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.905 7f66d84a7700 1 -- start start 2021-02-10T11:30:38.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.905 7f66d84a7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66d01b0470 0x7f66d01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:38.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.905 7f66d84a7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f66d00ef610 0x7f66d00efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:38.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.905 7f66d84a7700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66d00f0b50 con 0x7f66d00f0030 2021-02-10T11:30:38.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.905 7f66d84a7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66d00e52e0 con 0x7f66d00ef610 2021-02-10T11:30:38.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.905 7f66d84a7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d00e5490 con 0x7f66d01b0470 2021-02-10T11:30:38.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.905 7f66d6ac4700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f66d00f0030 0x7f66d00f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49754/0 (socket says 172.21.2.101:49754) 2021-02-10T11:30:38.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.905 7f66d6ac4700 1 -- 172.21.2.101:0/1450895950 learned_addr learned my addr 172.21.2.101:0/1450895950 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:38.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.905 7f66d62c3700 1 --2- 172.21.2.101:0/1450895950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66d01b0470 0x7f66d01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:38.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.906 7f66d5ac2700 1 --2- 172.21.2.101:0/1450895950 >> v2:172.21.2.108:3300/0 conn(0x7f66d00ef610 0x7f66d00efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:38.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.906 7f66d52c1700 1 -- 172.21.2.101:0/1450895950 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3151752227 0 0) 0x7f66cc000c10 con 0x7f66d00f0030 2021-02-10T11:30:38.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.906 7f66d52c1700 1 -- 172.21.2.101:0/1450895950 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66bc002fe0 con 0x7f66d00f0030 2021-02-10T11:30:38.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.906 7f66d62c3700 1 -- 172.21.2.101:0/1450895950 >> v2:172.21.2.108:3300/0 conn(0x7f66d00ef610 msgr2=0x7f66d00efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:38.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.906 7f66d62c3700 1 --2- 172.21.2.101:0/1450895950 >> v2:172.21.2.108:3300/0 conn(0x7f66d00ef610 0x7f66d00efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:38.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.906 7f66d62c3700 1 -- 172.21.2.101:0/1450895950 >> v1:172.21.2.101:6789/0 conn(0x7f66d00f0030 legacy=0x7f66d00f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:38.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.906 7f66d62c3700 1 -- 172.21.2.101:0/1450895950 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d00f0430 con 0x7f66d01b0470 2021-02-10T11:30:38.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.906 7f66d62c3700 1 --2- 172.21.2.101:0/1450895950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66d01b0470 0x7f66d01b0870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f66c80069d0 tx=0x7f66c8009e40).ready entity=mon.2 client_cookie=e18e6cf414f01408 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:38.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.906 7f66d52c1700 1 -- 172.21.2.101:0/1450895950 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f66c800b510 con 0x7f66d01b0470 2021-02-10T11:30:38.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.906 7f66d52c1700 1 -- 172.21.2.101:0/1450895950 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c800b890 con 0x7f66d01b0470 2021-02-10T11:30:38.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.906 7f66d84a7700 1 -- 172.21.2.101:0/1450895950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66d01b0470 msgr2=0x7f66d01b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:38.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.906 7f66d84a7700 1 --2- 172.21.2.101:0/1450895950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66d01b0470 0x7f66d01b0870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f66c80069d0 tx=0x7f66c8009e40).stop 2021-02-10T11:30:38.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.907 7f66d84a7700 1 -- 172.21.2.101:0/1450895950 shutdown_connections 2021-02-10T11:30:38.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.907 7f66d84a7700 1 --2- 172.21.2.101:0/1450895950 >> v2:172.21.2.108:3300/0 conn(0x7f66d00ef610 0x7f66d00efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:38.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.907 7f66d84a7700 1 --2- 172.21.2.101:0/1450895950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66d01b0470 0x7f66d01b0870 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:38.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.907 7f66d84a7700 1 -- 172.21.2.101:0/1450895950 >> 172.21.2.101:0/1450895950 conn(0x7f66d00ce3e0 msgr2=0x7f66d00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:38.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.907 7f66d84a7700 1 -- 172.21.2.101:0/1450895950 shutdown_connections 2021-02-10T11:30:38.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.907 7f66d84a7700 1 -- 172.21.2.101:0/1450895950 wait complete. 2021-02-10T11:30:38.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.922 7f66d84a7700 1 Processor -- start 2021-02-10T11:30:38.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.922 7f66d84a7700 1 -- start start 2021-02-10T11:30:38.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.922 7f66d84a7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66d00a0420 0x7f66d00a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:38.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.922 7f66d84a7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f66d0189d00 0x7f66d018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:38.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.922 7f66d84a7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66d018a720 0x7f66d00b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:38.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.922 7f66d84a7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d0189800 con 0x7f66d00a0420 2021-02-10T11:30:38.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.922 7f66d84a7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d018ab20 con 0x7f66d0189d00 2021-02-10T11:30:38.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.922 7f66d84a7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d00a1140 con 0x7f66d018a720 2021-02-10T11:30:38.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.922 7f66d62c3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66d00a0420 0x7f66d00a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:38.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.922 7f66d62c3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66d00a0420 0x7f66d00a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49262/0 (socket says 172.21.2.101:49262) 2021-02-10T11:30:38.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.922 7f66d62c3700 1 -- 172.21.2.101:0/986898189 learned_addr learned my addr 172.21.2.101:0/986898189 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:38.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.922 7f66d62c3700 1 -- 172.21.2.101:0/986898189 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66d018a720 msgr2=0x7f66d00b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.922 7f66d62c3700 1 --2- 172.21.2.101:0/986898189 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66d018a720 0x7f66d00b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.922 7f66d62c3700 1 -- 172.21.2.101:0/986898189 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f66d0189d00 msgr2=0x7f66d018a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.922 7f66d62c3700 1 --2- 172.21.2.101:0/986898189 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f66d0189d00 0x7f66d018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.922 7f66d62c3700 1 -- 172.21.2.101:0/986898189 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d00b8200 con 0x7f66d00a0420 2021-02-10T11:30:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.923 7f66d62c3700 1 --2- 172.21.2.101:0/986898189 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66d00a0420 0x7f66d00a0820 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7f66c8009f40 tx=0x7f66c800bd50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.923 7f66c6ffd700 1 -- 172.21.2.101:0/986898189 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f66c8006240 con 0x7f66d00a0420 2021-02-10T11:30:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.923 7f66c6ffd700 1 -- 172.21.2.101:0/986898189 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c80065c0 con 0x7f66d00a0420 2021-02-10T11:30:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.923 7f66d84a7700 1 -- 172.21.2.101:0/986898189 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d00b8200 con 0x7f66d00a0420 2021-02-10T11:30:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.923 7f66c6ffd700 1 -- 172.21.2.101:0/986898189 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f66c80058e0 con 0x7f66d00a0420 2021-02-10T11:30:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.923 7f66d84a7700 1 -- 172.21.2.101:0/986898189 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d00b8200 con 0x7f66d00a0420 2021-02-10T11:30:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.923 7f66c6ffd700 1 -- 172.21.2.101:0/986898189 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f66c8005660 con 0x7f66d00a0420 2021-02-10T11:30:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.923 7f66c6ffd700 1 --2- 172.21.2.101:0/986898189 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66b80186d0 0x7f66b801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:38.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.923 7f66c6ffd700 1 -- 172.21.2.101:0/986898189 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f66c8032b70 con 0x7f66d00a0420 2021-02-10T11:30:38.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.924 7f66d5ac2700 1 --2- 172.21.2.101:0/986898189 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66b80186d0 0x7f66b801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:38.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.924 7f66d5ac2700 1 --2- 172.21.2.101:0/986898189 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66b80186d0 0x7f66b801abd0 crc :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:38.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.955 7f66d84a7700 1 -- 172.21.2.101:0/986898189 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d0105ae0 con 0x7f66d00a0420 2021-02-10T11:30:38.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:38.957 7f66c6ffd700 1 -- 172.21.2.101:0/986898189 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f66c8000cc0 con 0x7f66d00a0420 2021-02-10T11:30:39.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.083 7f66d84a7700 1 -- 172.21.2.101:0/986898189 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839301", "target": ["mgr", ""]}) v1 -- 0x7f66d018ab20 con 0x7f66b80186d0 2021-02-10T11:30:39.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.093 7f66c6ffd700 1 -- 172.21.2.101:0/986898189 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f66c00052e0 con 0x7f66b80186d0 2021-02-10T11:30:39.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.100 7f66d84a7700 1 -- 172.21.2.101:0/986898189 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66b80186d0 msgr2=0x7f66b801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:39.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.100 7f66d84a7700 1 --2- 172.21.2.101:0/986898189 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66b80186d0 0x7f66b801abd0 crc :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:39.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.100 7f66d84a7700 1 -- 172.21.2.101:0/986898189 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66d00a0420 msgr2=0x7f66d00a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:39.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.100 7f66d84a7700 1 --2- 172.21.2.101:0/986898189 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66d00a0420 0x7f66d00a0820 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7f66c8009f40 tx=0x7f66c800bd50).stop 2021-02-10T11:30:39.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.100 7f66d84a7700 1 -- 172.21.2.101:0/986898189 shutdown_connections 2021-02-10T11:30:39.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.100 7f66d84a7700 1 --2- 172.21.2.101:0/986898189 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66b80186d0 0x7f66b801abd0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:39.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.100 7f66d84a7700 1 --2- 172.21.2.101:0/986898189 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66d018a720 0x7f66d00b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:39.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.100 7f66d84a7700 1 --2- 172.21.2.101:0/986898189 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f66d0189d00 0x7f66d018a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:39.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.100 7f66d84a7700 1 --2- 172.21.2.101:0/986898189 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66d00a0420 0x7f66d00a0820 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:39.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.100 7f66d84a7700 1 -- 172.21.2.101:0/986898189 >> 172.21.2.101:0/986898189 conn(0x7f66d0094e10 msgr2=0x7f66d00a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:39.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.100 7f66d84a7700 1 -- 172.21.2.101:0/986898189 shutdown_connections 2021-02-10T11:30:39.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.100 7f66d84a7700 1 -- 172.21.2.101:0/986898189 wait complete. 2021-02-10T11:30:39.120 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839302 2021-02-10T11:30:39.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.524 7f7f09a69700 1 Processor -- start 2021-02-10T11:30:39.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.524 7f7f09a69700 1 -- start start 2021-02-10T11:30:39.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.525 7f7f09a69700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7f040f05f0 0x7f7f040f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:39.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.525 7f7f09a69700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f041b0c70 con 0x7f7f041b0470 2021-02-10T11:30:39.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.525 7f7f09a69700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f040f0b50 con 0x7f7f041b0870 2021-02-10T11:30:39.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.525 7f7f09a69700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f040e5ea0 con 0x7f7f040f05f0 2021-02-10T11:30:39.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.525 7f7f03fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7f040f05f0 0x7f7f040f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:39.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.525 7f7f03fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7f040f05f0 0x7f7f040f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49272/0 (socket says 172.21.2.101:49272) 2021-02-10T11:30:39.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.525 7f7f037fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f7f041b0470 0x7f7f040ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49764/0 (socket says 172.21.2.101:49764) 2021-02-10T11:30:39.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.525 7f7f03fff700 1 -- 172.21.2.101:0/4167385650 learned_addr learned my addr 172.21.2.101:0/4167385650 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:39.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.525 7f7f03fff700 1 -- 172.21.2.101:0/4167385650 >> v1:172.21.2.108:6789/0 conn(0x7f7f041b0870 legacy=0x7f7f040efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:39.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.525 7f7f03fff700 1 -- 172.21.2.101:0/4167385650 >> v1:172.21.2.101:6789/0 conn(0x7f7f041b0470 legacy=0x7f7f040ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:39.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.525 7f7f03fff700 1 -- 172.21.2.101:0/4167385650 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f040e52e0 con 0x7f7f040f05f0 2021-02-10T11:30:39.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.525 7f7f03fff700 1 --2- 172.21.2.101:0/4167385650 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f040f05f0 0x7f7f040f20e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7f7ef4006b80 tx=0x7f7ef4009b90).ready entity=mon.0 client_cookie=fcd306f21191ef12 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:39.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.526 7f7f027fc700 1 -- 172.21.2.101:0/4167385650 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ef40022e0 con 0x7f7f040f05f0 2021-02-10T11:30:39.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.526 7f7f027fc700 1 -- 172.21.2.101:0/4167385650 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef4002560 con 0x7f7f040f05f0 2021-02-10T11:30:39.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.526 7f7f027fc700 1 -- 172.21.2.101:0/4167385650 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ef4002a40 con 0x7f7f040f05f0 2021-02-10T11:30:39.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.526 7f7f09a69700 1 -- 172.21.2.101:0/4167385650 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f040f05f0 msgr2=0x7f7f040f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:39.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.526 7f7f09a69700 1 --2- 172.21.2.101:0/4167385650 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f040f05f0 0x7f7f040f20e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7f7ef4006b80 tx=0x7f7ef4009b90).stop 2021-02-10T11:30:39.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.526 7f7f09a69700 1 -- 172.21.2.101:0/4167385650 shutdown_connections 2021-02-10T11:30:39.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.526 7f7f09a69700 1 --2- 172.21.2.101:0/4167385650 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f040f05f0 0x7f7f040f20e0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:39.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.526 7f7f09a69700 1 -- 172.21.2.101:0/4167385650 >> 172.21.2.101:0/4167385650 conn(0x7f7f040ce3e0 msgr2=0x7f7f040e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:39.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.526 7f7f09a69700 1 -- 172.21.2.101:0/4167385650 shutdown_connections 2021-02-10T11:30:39.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.526 7f7f09a69700 1 -- 172.21.2.101:0/4167385650 wait complete. 2021-02-10T11:30:39.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.540 7f7f09a69700 1 Processor -- start 2021-02-10T11:30:39.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.541 7f7f09a69700 1 -- start start 2021-02-10T11:30:39.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.541 7f7f09a69700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7f04081860 0x7f7f040a9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:39.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.541 7f7f09a69700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7f040aa210 0x7f7f040c19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:39.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.541 7f7f09a69700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f040aa610 0x7f7f040c1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:39.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.541 7f7f09a69700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f040c2f20 con 0x7f7f040aa610 2021-02-10T11:30:39.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.541 7f7f09a69700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f040c1420 con 0x7f7f04081860 2021-02-10T11:30:39.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.541 7f7f09a69700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f041b56f0 con 0x7f7f040aa210 2021-02-10T11:30:39.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.541 7f7f03fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f040aa610 0x7f7f040c1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:39.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.541 7f7f03fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f040aa610 0x7f7f040c1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49276/0 (socket says 172.21.2.101:49276) 2021-02-10T11:30:39.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.541 7f7f03fff700 1 -- 172.21.2.101:0/2970647285 learned_addr learned my addr 172.21.2.101:0/2970647285 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:39.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.541 7f7f02ffd700 1 --2- 172.21.2.101:0/2970647285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7f040aa210 0x7f7f040c19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:39.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.541 7f7f037fe700 1 --2- 172.21.2.101:0/2970647285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7f04081860 0x7f7f040a9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:39.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.542 7f7f03fff700 1 -- 172.21.2.101:0/2970647285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7f040aa210 msgr2=0x7f7f040c19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:39.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.542 7f7f03fff700 1 --2- 172.21.2.101:0/2970647285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7f040aa210 0x7f7f040c19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:39.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.542 7f7f03fff700 1 -- 172.21.2.101:0/2970647285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7f04081860 msgr2=0x7f7f040a9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:39.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.542 7f7f03fff700 1 --2- 172.21.2.101:0/2970647285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7f04081860 0x7f7f040a9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:39.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.542 7f7f03fff700 1 -- 172.21.2.101:0/2970647285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f040c25e0 con 0x7f7f040aa610 2021-02-10T11:30:39.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.542 7f7f03fff700 1 --2- 172.21.2.101:0/2970647285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f040aa610 0x7f7f040c1fc0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7f7ef400b630 tx=0x7f7ef400b9d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:39.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.542 7f7eebfff700 1 -- 172.21.2.101:0/2970647285 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ef4000b50 con 0x7f7f040aa610 2021-02-10T11:30:39.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.542 7f7eebfff700 1 -- 172.21.2.101:0/2970647285 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef4002030 con 0x7f7f040aa610 2021-02-10T11:30:39.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.542 7f7eebfff700 1 -- 172.21.2.101:0/2970647285 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ef4002550 con 0x7f7f040aa610 2021-02-10T11:30:39.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.542 7f7f09a69700 1 -- 172.21.2.101:0/2970647285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f040c25e0 con 0x7f7f040aa610 2021-02-10T11:30:39.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.542 7f7f09a69700 1 -- 172.21.2.101:0/2970647285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f040c2b70 con 0x7f7f040aa610 2021-02-10T11:30:39.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.542 7f7eebfff700 1 -- 172.21.2.101:0/2970647285 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7ef4002210 con 0x7f7f040aa610 2021-02-10T11:30:39.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.543 7f7eebfff700 1 --2- 172.21.2.101:0/2970647285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ee4018730 0x7f7ee401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:39.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.543 7f7eebfff700 1 -- 172.21.2.101:0/2970647285 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7ef40326b0 con 0x7f7f040aa610 2021-02-10T11:30:39.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.543 7f7f037fe700 1 --2- 172.21.2.101:0/2970647285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ee4018730 0x7f7ee401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:39.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.543 7f7f037fe700 1 --2- 172.21.2.101:0/2970647285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ee4018730 0x7f7ee401ac10 crc :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:39.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.575 7f7f09a69700 1 -- 172.21.2.101:0/2970647285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f0407e6b0 con 0x7f7f040aa610 2021-02-10T11:30:39.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.577 7f7eebfff700 1 -- 172.21.2.101:0/2970647285 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7ef4000940 con 0x7f7f040aa610 2021-02-10T11:30:39.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.705 7f7f09a69700 1 -- 172.21.2.101:0/2970647285 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839302", "target": ["mgr", ""]}) v1 -- 0x7f7f040ce3e0 con 0x7f7ee4018730 2021-02-10T11:30:39.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.715 7f7eebfff700 1 -- 172.21.2.101:0/2970647285 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f7efc007b40 con 0x7f7ee4018730 2021-02-10T11:30:39.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.722 7f7f09a69700 1 -- 172.21.2.101:0/2970647285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ee4018730 msgr2=0x7f7ee401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:39.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.722 7f7f09a69700 1 --2- 172.21.2.101:0/2970647285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ee4018730 0x7f7ee401ac10 crc :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:39.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.722 7f7f09a69700 1 -- 172.21.2.101:0/2970647285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f040aa610 msgr2=0x7f7f040c1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:39.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.722 7f7f09a69700 1 --2- 172.21.2.101:0/2970647285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f040aa610 0x7f7f040c1fc0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7f7ef400b630 tx=0x7f7ef400b9d0).stop 2021-02-10T11:30:39.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.722 7f7f09a69700 1 -- 172.21.2.101:0/2970647285 shutdown_connections 2021-02-10T11:30:39.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.722 7f7f09a69700 1 --2- 172.21.2.101:0/2970647285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ee4018730 0x7f7ee401ac10 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:39.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.722 7f7f09a69700 1 --2- 172.21.2.101:0/2970647285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7f040aa210 0x7f7f040c19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:39.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.722 7f7f09a69700 1 --2- 172.21.2.101:0/2970647285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f040aa610 0x7f7f040c1fc0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:39.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.722 7f7f09a69700 1 --2- 172.21.2.101:0/2970647285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7f04081860 0x7f7f040a9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:39.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.722 7f7f09a69700 1 -- 172.21.2.101:0/2970647285 >> 172.21.2.101:0/2970647285 conn(0x7f7f04088220 msgr2=0x7f7f04081e30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:39.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.723 7f7f09a69700 1 -- 172.21.2.101:0/2970647285 shutdown_connections 2021-02-10T11:30:39.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:39.723 7f7f09a69700 1 -- 172.21.2.101:0/2970647285 wait complete. 2021-02-10T11:30:39.743 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839303 2021-02-10T11:30:40.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.146 7f4df33f2700 1 Processor -- start 2021-02-10T11:30:40.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.147 7f4df33f2700 1 -- start start 2021-02-10T11:30:40.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.147 7f4df33f2700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4dec1b0870 0x7f4dec0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:40.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.147 7f4df33f2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dec0efe00 0x7f4dec0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:40.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.147 7f4df33f2700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dec1b0c70 con 0x7f4dec1b0470 2021-02-10T11:30:40.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.147 7f4df33f2700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dec0f0b50 con 0x7f4dec1b0870 2021-02-10T11:30:40.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.147 7f4df33f2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec0f0820 con 0x7f4dec0efe00 2021-02-10T11:30:40.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.147 7f4df1a0f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dec0efe00 0x7f4dec0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:40.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.147 7f4df1a0f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dec0efe00 0x7f4dec0f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:42016/0 (socket says 172.21.2.101:42016) 2021-02-10T11:30:40.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.147 7f4df1a0f700 1 -- 172.21.2.101:0/2901855621 learned_addr learned my addr 172.21.2.101:0/2901855621 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:40.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.147 7f4df0a0d700 1 --2- 172.21.2.101:0/2901855621 >> v2:172.21.2.108:3300/0 conn(0x7f4dec1b0870 0x7f4dec0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:40.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.148 7f4df1a0f700 1 -- 172.21.2.101:0/2901855621 >> v2:172.21.2.108:3300/0 conn(0x7f4dec1b0870 msgr2=0x7f4dec0f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:40.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.148 7f4df1a0f700 1 --2- 172.21.2.101:0/2901855621 >> v2:172.21.2.108:3300/0 conn(0x7f4dec1b0870 0x7f4dec0f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:40.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.148 7f4df1a0f700 1 -- 172.21.2.101:0/2901855621 >> v1:172.21.2.108:6789/0 conn(0x7f4dec1b0470 legacy=0x7f4dec0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:40.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.148 7f4df1a0f700 1 -- 172.21.2.101:0/2901855621 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dec0e52e0 con 0x7f4dec0efe00 2021-02-10T11:30:40.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.148 7f4df1a0f700 1 --2- 172.21.2.101:0/2901855621 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dec0efe00 0x7f4dec0f0200 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f4de4006aa0 tx=0x7f4de4009f20).ready entity=mon.2 client_cookie=55c719a1adacad6a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:40.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.148 7f4de3fff700 1 -- 172.21.2.101:0/2901855621 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4de400b5c0 con 0x7f4dec0efe00 2021-02-10T11:30:40.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.148 7f4de3fff700 1 -- 172.21.2.101:0/2901855621 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de4003030 con 0x7f4dec0efe00 2021-02-10T11:30:40.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.148 7f4de3fff700 1 -- 172.21.2.101:0/2901855621 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4de400bb40 con 0x7f4dec0efe00 2021-02-10T11:30:40.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.148 7f4df33f2700 1 -- 172.21.2.101:0/2901855621 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dec0efe00 msgr2=0x7f4dec0f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:40.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.148 7f4df33f2700 1 --2- 172.21.2.101:0/2901855621 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dec0efe00 0x7f4dec0f0200 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f4de4006aa0 tx=0x7f4de4009f20).stop 2021-02-10T11:30:40.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.149 7f4df33f2700 1 -- 172.21.2.101:0/2901855621 shutdown_connections 2021-02-10T11:30:40.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.149 7f4df33f2700 1 --2- 172.21.2.101:0/2901855621 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dec0efe00 0x7f4dec0f0200 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:40.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.149 7f4df33f2700 1 --2- 172.21.2.101:0/2901855621 >> v2:172.21.2.108:3300/0 conn(0x7f4dec1b0870 0x7f4dec0f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:40.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.149 7f4df33f2700 1 -- 172.21.2.101:0/2901855621 >> 172.21.2.101:0/2901855621 conn(0x7f4dec0ce3e0 msgr2=0x7f4dec0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:40.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.149 7f4df33f2700 1 -- 172.21.2.101:0/2901855621 shutdown_connections 2021-02-10T11:30:40.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.149 7f4df33f2700 1 -- 172.21.2.101:0/2901855621 wait complete. 2021-02-10T11:30:40.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.164 7f4df33f2700 1 Processor -- start 2021-02-10T11:30:40.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.164 7f4df33f2700 1 -- start start 2021-02-10T11:30:40.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.164 7f4df33f2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dec1a48b0 0x7f4dec0bf930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:40.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.164 7f4df33f2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dec0a73c0 0x7f4dec0bff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:40.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.164 7f4df33f2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4dec0a5260 0x7f4dec0a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:40.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.164 7f4df33f2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec0a6ec0 con 0x7f4dec1a48b0 2021-02-10T11:30:40.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.164 7f4df33f2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec0bc010 con 0x7f4dec0a5260 2021-02-10T11:30:40.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.164 7f4df33f2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec0c0570 con 0x7f4dec0a73c0 2021-02-10T11:30:40.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.164 7f4df120e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dec1a48b0 0x7f4dec0bf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:40.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.164 7f4df0a0d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dec0a73c0 0x7f4dec0bff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:40.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.164 7f4df120e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dec1a48b0 0x7f4dec0bf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49292/0 (socket says 172.21.2.101:49292) 2021-02-10T11:30:40.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.164 7f4df1a0f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4dec0a5260 0x7f4dec0a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:40.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.164 7f4df120e700 1 -- 172.21.2.101:0/434558921 learned_addr learned my addr 172.21.2.101:0/434558921 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:40.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.165 7f4df120e700 1 -- 172.21.2.101:0/434558921 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dec0a73c0 msgr2=0x7f4dec0bff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:40.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.165 7f4df120e700 1 --2- 172.21.2.101:0/434558921 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dec0a73c0 0x7f4dec0bff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:40.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.165 7f4df120e700 1 -- 172.21.2.101:0/434558921 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4dec0a5260 msgr2=0x7f4dec0a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:40.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.165 7f4df120e700 1 --2- 172.21.2.101:0/434558921 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4dec0a5260 0x7f4dec0a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:40.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.165 7f4df120e700 1 -- 172.21.2.101:0/434558921 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dec0a5c80 con 0x7f4dec1a48b0 2021-02-10T11:30:40.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.165 7f4df120e700 1 --2- 172.21.2.101:0/434558921 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dec1a48b0 0x7f4dec0bf930 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7f4de800b5b0 tx=0x7f4de800b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:40.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.165 7f4de1ffb700 1 -- 172.21.2.101:0/434558921 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4de800bd40 con 0x7f4dec1a48b0 2021-02-10T11:30:40.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.165 7f4de1ffb700 1 -- 172.21.2.101:0/434558921 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de8006240 con 0x7f4dec1a48b0 2021-02-10T11:30:40.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.165 7f4df33f2700 1 -- 172.21.2.101:0/434558921 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dec0a5c80 con 0x7f4dec1a48b0 2021-02-10T11:30:40.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.165 7f4de1ffb700 1 -- 172.21.2.101:0/434558921 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4de8005720 con 0x7f4dec1a48b0 2021-02-10T11:30:40.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.165 7f4df33f2700 1 -- 172.21.2.101:0/434558921 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dec0a6210 con 0x7f4dec1a48b0 2021-02-10T11:30:40.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.166 7f4de1ffb700 1 -- 172.21.2.101:0/434558921 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4de80054a0 con 0x7f4dec1a48b0 2021-02-10T11:30:40.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.166 7f4de1ffb700 1 --2- 172.21.2.101:0/434558921 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4dcc0186d0 0x7f4dcc01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:40.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.166 7f4de1ffb700 1 -- 172.21.2.101:0/434558921 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4de8032820 con 0x7f4dec1a48b0 2021-02-10T11:30:40.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.166 7f4df0a0d700 1 --2- 172.21.2.101:0/434558921 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4dcc0186d0 0x7f4dcc01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:40.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.166 7f4df0a0d700 1 --2- 172.21.2.101:0/434558921 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4dcc0186d0 0x7f4dcc01abd0 crc :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:40.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.197 7f4df33f2700 1 -- 172.21.2.101:0/434558921 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dec105ae0 con 0x7f4dec1a48b0 2021-02-10T11:30:40.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.199 7f4de1ffb700 1 -- 172.21.2.101:0/434558921 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4de8000940 con 0x7f4dec1a48b0 2021-02-10T11:30:40.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.329 7f4df33f2700 1 -- 172.21.2.101:0/434558921 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839303", "target": ["mgr", ""]}) v1 -- 0x7f4dec0f0b50 con 0x7f4dcc0186d0 2021-02-10T11:30:40.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.339 7f4de1ffb700 1 -- 172.21.2.101:0/434558921 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f4ddc006a70 con 0x7f4dcc0186d0 2021-02-10T11:30:40.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.346 7f4df33f2700 1 -- 172.21.2.101:0/434558921 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4dcc0186d0 msgr2=0x7f4dcc01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:40.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.346 7f4df33f2700 1 --2- 172.21.2.101:0/434558921 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4dcc0186d0 0x7f4dcc01abd0 crc :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:40.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.346 7f4df33f2700 1 -- 172.21.2.101:0/434558921 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dec1a48b0 msgr2=0x7f4dec0bf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:40.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.346 7f4df33f2700 1 --2- 172.21.2.101:0/434558921 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dec1a48b0 0x7f4dec0bf930 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7f4de800b5b0 tx=0x7f4de800b950).stop 2021-02-10T11:30:40.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.346 7f4df33f2700 1 -- 172.21.2.101:0/434558921 shutdown_connections 2021-02-10T11:30:40.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.346 7f4df33f2700 1 --2- 172.21.2.101:0/434558921 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4dcc0186d0 0x7f4dcc01abd0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:40.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.346 7f4df33f2700 1 --2- 172.21.2.101:0/434558921 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4dec0a73c0 0x7f4dec0bff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:40.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.346 7f4df33f2700 1 --2- 172.21.2.101:0/434558921 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4dec0a5260 0x7f4dec0a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:40.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.346 7f4df33f2700 1 --2- 172.21.2.101:0/434558921 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4dec1a48b0 0x7f4dec0bf930 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:40.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.346 7f4df33f2700 1 -- 172.21.2.101:0/434558921 >> 172.21.2.101:0/434558921 conn(0x7f4dec109e80 msgr2=0x7f4dec0af060 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:40.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.346 7f4df33f2700 1 -- 172.21.2.101:0/434558921 shutdown_connections 2021-02-10T11:30:40.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.346 7f4df33f2700 1 -- 172.21.2.101:0/434558921 wait complete. 2021-02-10T11:30:40.366 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839304 2021-02-10T11:30:40.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.769 7f89a6665700 1 Processor -- start 2021-02-10T11:30:40.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.769 7f89a6665700 1 -- start start 2021-02-10T11:30:40.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.769 7f89a6665700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89a01b0470 0x7f89a01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:40.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.770 7f89a6665700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f89a00f20e0 0x7f89a00f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:40.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.770 7f89a6665700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a00f0b50 con 0x7f89a00ef610 2021-02-10T11:30:40.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.770 7f89a6665700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a00f0820 con 0x7f89a00f20e0 2021-02-10T11:30:40.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.770 7f89a6665700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a00f24e0 con 0x7f89a01b0470 2021-02-10T11:30:40.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.770 7f89977fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f89a00ef610 0x7f89a00efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49794/0 (socket says 172.21.2.101:49794) 2021-02-10T11:30:40.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.770 7f89977fe700 1 -- 172.21.2.101:0/4178157422 learned_addr learned my addr 172.21.2.101:0/4178157422 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:40.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.770 7f899ffff700 1 --2- 172.21.2.101:0/4178157422 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89a01b0470 0x7f89a01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:40.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.770 7f89a4c82700 1 --2- 172.21.2.101:0/4178157422 >> v2:172.21.2.108:3300/0 conn(0x7f89a00f20e0 0x7f89a00f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:40.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.770 7f899f7fe700 1 -- 172.21.2.101:0/4178157422 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2477827144 0 0) 0x7f8988000ce0 con 0x7f89a00ef610 2021-02-10T11:30:40.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.770 7f899f7fe700 1 -- 172.21.2.101:0/4178157422 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f898c002fe0 con 0x7f89a00ef610 2021-02-10T11:30:40.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.770 7f899ffff700 1 -- 172.21.2.101:0/4178157422 >> v2:172.21.2.108:3300/0 conn(0x7f89a00f20e0 msgr2=0x7f89a00f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:40.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.770 7f899ffff700 1 --2- 172.21.2.101:0/4178157422 >> v2:172.21.2.108:3300/0 conn(0x7f89a00f20e0 0x7f89a00f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:40.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.770 7f899ffff700 1 -- 172.21.2.101:0/4178157422 >> v1:172.21.2.101:6789/0 conn(0x7f89a00ef610 legacy=0x7f89a00efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:40.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.770 7f899f7fe700 1 -- 172.21.2.101:0/4178157422 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3750650865 0 0) 0x7f89880010e0 con 0x7f89a00ef610 2021-02-10T11:30:40.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.770 7f899ffff700 1 -- 172.21.2.101:0/4178157422 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a00e52e0 con 0x7f89a01b0470 2021-02-10T11:30:40.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.770 7f89a4c82700 1 --2- 172.21.2.101:0/4178157422 >> v2:172.21.2.108:3300/0 conn(0x7f89a00f20e0 0x7f89a00f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:40.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.770 7f899ffff700 1 --2- 172.21.2.101:0/4178157422 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89a01b0470 0x7f89a01b0870 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f8990006650 tx=0x7f8990009a80).ready entity=mon.2 client_cookie=58a25820b207be0e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:40.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.771 7f899f7fe700 1 -- 172.21.2.101:0/4178157422 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f89900009b0 con 0x7f89a01b0470 2021-02-10T11:30:40.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.771 7f899f7fe700 1 -- 172.21.2.101:0/4178157422 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899000c0c0 con 0x7f89a01b0470 2021-02-10T11:30:40.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.771 7f899f7fe700 1 -- 172.21.2.101:0/4178157422 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f899000c5e0 con 0x7f89a01b0470 2021-02-10T11:30:40.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.771 7f89a6665700 1 -- 172.21.2.101:0/4178157422 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89a01b0470 msgr2=0x7f89a01b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:40.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.771 7f89a6665700 1 --2- 172.21.2.101:0/4178157422 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89a01b0470 0x7f89a01b0870 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f8990006650 tx=0x7f8990009a80).stop 2021-02-10T11:30:40.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.771 7f89a6665700 1 -- 172.21.2.101:0/4178157422 shutdown_connections 2021-02-10T11:30:40.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.771 7f89a6665700 1 --2- 172.21.2.101:0/4178157422 >> v2:172.21.2.108:3300/0 conn(0x7f89a00f20e0 0x7f89a00f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:40.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.771 7f89a6665700 1 --2- 172.21.2.101:0/4178157422 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89a01b0470 0x7f89a01b0870 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:40.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.771 7f89a6665700 1 -- 172.21.2.101:0/4178157422 >> 172.21.2.101:0/4178157422 conn(0x7f89a00ce3e0 msgr2=0x7f89a00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:40.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.771 7f89a6665700 1 -- 172.21.2.101:0/4178157422 shutdown_connections 2021-02-10T11:30:40.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.771 7f89a6665700 1 -- 172.21.2.101:0/4178157422 wait complete. 2021-02-10T11:30:40.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.786 7f89a6665700 1 Processor -- start 2021-02-10T11:30:40.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.786 7f89a6665700 1 -- start start 2021-02-10T11:30:40.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.786 7f89a6665700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89a00a62e0 0x7f89a00c2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:40.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.786 7f89a6665700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89a00ae960 0x7f89a00aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:40.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.786 7f89a6665700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89a00a78f0 0x7f89a00a7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:40.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.786 7f89a6665700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a00a8310 con 0x7f89a00a62e0 2021-02-10T11:30:40.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.786 7f89a6665700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a00c1b80 con 0x7f89a00a78f0 2021-02-10T11:30:40.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.786 7f89a6665700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a00c2720 con 0x7f89a00ae960 2021-02-10T11:30:40.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.787 7f899ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89a00a62e0 0x7f89a00c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:40.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.787 7f899ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89a00a62e0 0x7f89a00c2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49302/0 (socket says 172.21.2.101:49302) 2021-02-10T11:30:40.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.787 7f899ffff700 1 -- 172.21.2.101:0/3742295537 learned_addr learned my addr 172.21.2.101:0/3742295537 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:40.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.787 7f89977fe700 1 --2- 172.21.2.101:0/3742295537 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89a00ae960 0x7f89a00aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:40.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.787 7f89a4c82700 1 --2- 172.21.2.101:0/3742295537 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89a00a78f0 0x7f89a00a7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:40.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.787 7f899ffff700 1 -- 172.21.2.101:0/3742295537 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89a00ae960 msgr2=0x7f89a00aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:40.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.787 7f899ffff700 1 --2- 172.21.2.101:0/3742295537 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89a00ae960 0x7f89a00aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:40.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.787 7f899ffff700 1 -- 172.21.2.101:0/3742295537 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89a00a78f0 msgr2=0x7f89a00a7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:40.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.787 7f899ffff700 1 --2- 172.21.2.101:0/3742295537 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89a00a78f0 0x7f89a00a7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:40.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.787 7f899ffff700 1 -- 172.21.2.101:0/3742295537 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a00a66e0 con 0x7f89a00a62e0 2021-02-10T11:30:40.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.787 7f89a4c82700 1 --2- 172.21.2.101:0/3742295537 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89a00a78f0 0x7f89a00a7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:30:40.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.787 7f899ffff700 1 --2- 172.21.2.101:0/3742295537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89a00a62e0 0x7f89a00c2100 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7f8990009630 tx=0x7f899000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:40.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.787 7f899d7fa700 1 -- 172.21.2.101:0/3742295537 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f89900093a0 con 0x7f89a00a62e0 2021-02-10T11:30:40.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.787 7f899d7fa700 1 -- 172.21.2.101:0/3742295537 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899000bd70 con 0x7f89a00a62e0 2021-02-10T11:30:40.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.787 7f899d7fa700 1 -- 172.21.2.101:0/3742295537 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8990005560 con 0x7f89a00a62e0 2021-02-10T11:30:40.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.787 7f89a6665700 1 -- 172.21.2.101:0/3742295537 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a00a66e0 con 0x7f89a00a62e0 2021-02-10T11:30:40.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.787 7f89a6665700 1 -- 172.21.2.101:0/3742295537 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a00a66e0 con 0x7f89a00a62e0 2021-02-10T11:30:40.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.788 7f899d7fa700 1 -- 172.21.2.101:0/3742295537 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f899000a000 con 0x7f89a00a62e0 2021-02-10T11:30:40.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.788 7f899d7fa700 1 --2- 172.21.2.101:0/3742295537 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f89840186c0 0x7f898401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:40.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.788 7f899d7fa700 1 -- 172.21.2.101:0/3742295537 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f89900327b0 con 0x7f89a00a62e0 2021-02-10T11:30:40.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.788 7f89977fe700 1 --2- 172.21.2.101:0/3742295537 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f89840186c0 0x7f898401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:40.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.788 7f89977fe700 1 --2- 172.21.2.101:0/3742295537 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f89840186c0 0x7f898401abc0 crc :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:40.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.818 7f89a6665700 1 -- 172.21.2.101:0/3742295537 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a0105b30 con 0x7f89a00a62e0 2021-02-10T11:30:40.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.821 7f899d7fa700 1 -- 172.21.2.101:0/3742295537 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f899001f4f0 con 0x7f89a00a62e0 2021-02-10T11:30:40.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.948 7f89a6665700 1 -- 172.21.2.101:0/3742295537 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839304", "target": ["mgr", ""]}) v1 -- 0x7f89a00f0b50 con 0x7f89840186c0 2021-02-10T11:30:40.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.958 7f899d7fa700 1 -- 172.21.2.101:0/3742295537 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f89880078a0 con 0x7f89840186c0 2021-02-10T11:30:40.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.964 7f89a6665700 1 -- 172.21.2.101:0/3742295537 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f89840186c0 msgr2=0x7f898401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.964 7f89a6665700 1 --2- 172.21.2.101:0/3742295537 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f89840186c0 0x7f898401abc0 crc :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.965 7f89a6665700 1 -- 172.21.2.101:0/3742295537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89a00a62e0 msgr2=0x7f89a00c2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.965 7f89a6665700 1 --2- 172.21.2.101:0/3742295537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89a00a62e0 0x7f89a00c2100 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7f8990009630 tx=0x7f899000b980).stop 2021-02-10T11:30:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.965 7f89a6665700 1 -- 172.21.2.101:0/3742295537 shutdown_connections 2021-02-10T11:30:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.965 7f89a6665700 1 --2- 172.21.2.101:0/3742295537 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f89840186c0 0x7f898401abc0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.965 7f89a6665700 1 --2- 172.21.2.101:0/3742295537 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89a00ae960 0x7f89a00aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.965 7f89a6665700 1 --2- 172.21.2.101:0/3742295537 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89a00a78f0 0x7f89a00a7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.965 7f89a6665700 1 --2- 172.21.2.101:0/3742295537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89a00a62e0 0x7f89a00c2100 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.965 7f89a6665700 1 -- 172.21.2.101:0/3742295537 >> 172.21.2.101:0/3742295537 conn(0x7f89a00af720 msgr2=0x7f89a00c3280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.965 7f89a6665700 1 -- 172.21.2.101:0/3742295537 shutdown_connections 2021-02-10T11:30:40.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:40.965 7f89a6665700 1 -- 172.21.2.101:0/3742295537 wait complete. 2021-02-10T11:30:40.985 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839305 2021-02-10T11:30:41.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.389 7f7e6f804700 1 Processor -- start 2021-02-10T11:30:41.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.389 7f7e6f804700 1 -- start start 2021-02-10T11:30:41.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.390 7f7e6f804700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e681b0470 0x7f7e681b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:41.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.390 7f7e6f804700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e680f0b50 con 0x7f7e680ef610 2021-02-10T11:30:41.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.390 7f7e6f804700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e680f24e0 con 0x7f7e680f20e0 2021-02-10T11:30:41.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.390 7f7e6f804700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e680e5ea0 con 0x7f7e681b0470 2021-02-10T11:30:41.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.390 7f7e6ce1f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f7e680ef610 0x7f7e680efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49804/0 (socket says 172.21.2.101:49804) 2021-02-10T11:30:41.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.390 7f7e6ce1f700 1 -- 172.21.2.101:0/1396203172 learned_addr learned my addr 172.21.2.101:0/1396203172 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:41.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.390 7f7e6d620700 1 --2- 172.21.2.101:0/1396203172 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e681b0470 0x7f7e681b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:41.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.390 7f7e5ffff700 1 -- 172.21.2.101:0/1396203172 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2021344837 0 0) 0x7f7e580010b0 con 0x7f7e680ef610 2021-02-10T11:30:41.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.390 7f7e5ffff700 1 -- 172.21.2.101:0/1396203172 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e54002fe0 con 0x7f7e680ef610 2021-02-10T11:30:41.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.390 7f7e5ffff700 1 -- 172.21.2.101:0/1396203172 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2943613030 0 0) 0x7f7e60001400 con 0x7f7e680f20e0 2021-02-10T11:30:41.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.390 7f7e6d620700 1 -- 172.21.2.101:0/1396203172 >> v1:172.21.2.108:6789/0 conn(0x7f7e680f20e0 legacy=0x7f7e680f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:41.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.390 7f7e6d620700 1 -- 172.21.2.101:0/1396203172 >> v1:172.21.2.101:6789/0 conn(0x7f7e680ef610 legacy=0x7f7e680efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:41.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.390 7f7e6d620700 1 -- 172.21.2.101:0/1396203172 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e680e52e0 con 0x7f7e681b0470 2021-02-10T11:30:41.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.391 7f7e6d620700 1 --2- 172.21.2.101:0/1396203172 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e681b0470 0x7f7e681b0870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f7e64006650 tx=0x7f7e64009a80).ready entity=mon.2 client_cookie=f34194b4e01166fa server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:41.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.391 7f7e5ffff700 1 -- 172.21.2.101:0/1396203172 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7e6400c0c0 con 0x7f7e681b0470 2021-02-10T11:30:41.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.391 7f7e5ffff700 1 -- 172.21.2.101:0/1396203172 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6400c340 con 0x7f7e681b0470 2021-02-10T11:30:41.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.391 7f7e5ffff700 1 -- 172.21.2.101:0/1396203172 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7e6400c820 con 0x7f7e681b0470 2021-02-10T11:30:41.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.391 7f7e6f804700 1 -- 172.21.2.101:0/1396203172 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e681b0470 msgr2=0x7f7e681b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:41.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.391 7f7e6f804700 1 --2- 172.21.2.101:0/1396203172 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e681b0470 0x7f7e681b0870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f7e64006650 tx=0x7f7e64009a80).stop 2021-02-10T11:30:41.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.391 7f7e6f804700 1 -- 172.21.2.101:0/1396203172 shutdown_connections 2021-02-10T11:30:41.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.391 7f7e6f804700 1 --2- 172.21.2.101:0/1396203172 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e681b0470 0x7f7e681b0870 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:41.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.391 7f7e6f804700 1 -- 172.21.2.101:0/1396203172 >> 172.21.2.101:0/1396203172 conn(0x7f7e680ce3e0 msgr2=0x7f7e680e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:41.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.392 7f7e6f804700 1 -- 172.21.2.101:0/1396203172 shutdown_connections 2021-02-10T11:30:41.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.392 7f7e6f804700 1 -- 172.21.2.101:0/1396203172 wait complete. 2021-02-10T11:30:41.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.406 7f7e6f804700 1 Processor -- start 2021-02-10T11:30:41.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.406 7f7e6f804700 1 -- start start 2021-02-10T11:30:41.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.407 7f7e6f804700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7e68190920 0x7f7e6808f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:41.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.407 7f7e6f804700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e68190d20 0x7f7e6808fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:41.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.407 7f7e6f804700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e68193270 0x7f7e68193670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:41.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.407 7f7e6f804700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e68090470 con 0x7f7e68193270 2021-02-10T11:30:41.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.407 7f7e6f804700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e68193c90 con 0x7f7e68190920 2021-02-10T11:30:41.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.407 7f7e6f804700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e68193e70 con 0x7f7e68190d20 2021-02-10T11:30:41.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.407 7f7e6de21700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e68193270 0x7f7e68193670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:41.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.407 7f7e6de21700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e68193270 0x7f7e68193670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49316/0 (socket says 172.21.2.101:49316) 2021-02-10T11:30:41.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.407 7f7e6de21700 1 -- 172.21.2.101:0/2255529745 learned_addr learned my addr 172.21.2.101:0/2255529745 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:41.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.407 7f7e6ce1f700 1 --2- 172.21.2.101:0/2255529745 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e68190d20 0x7f7e6808fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:41.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.407 7f7e6d620700 1 --2- 172.21.2.101:0/2255529745 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7e68190920 0x7f7e6808f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:41.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.407 7f7e6de21700 1 -- 172.21.2.101:0/2255529745 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e68190d20 msgr2=0x7f7e6808fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:41.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.407 7f7e6de21700 1 --2- 172.21.2.101:0/2255529745 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e68190d20 0x7f7e6808fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:41.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.407 7f7e6de21700 1 -- 172.21.2.101:0/2255529745 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7e68190920 msgr2=0x7f7e6808f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:41.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.407 7f7e6de21700 1 --2- 172.21.2.101:0/2255529745 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7e68190920 0x7f7e6808f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:41.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.407 7f7e6de21700 1 -- 172.21.2.101:0/2255529745 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e68194090 con 0x7f7e68193270 2021-02-10T11:30:41.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.407 7f7e6de21700 1 --2- 172.21.2.101:0/2255529745 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e68193270 0x7f7e68193670 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f7e6000b8e0 tx=0x7f7e6000bc80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:41.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.408 7f7e5dffb700 1 -- 172.21.2.101:0/2255529745 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7e60000f40 con 0x7f7e68193270 2021-02-10T11:30:41.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.408 7f7e5dffb700 1 -- 172.21.2.101:0/2255529745 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6000c2f0 con 0x7f7e68193270 2021-02-10T11:30:41.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.408 7f7e5dffb700 1 -- 172.21.2.101:0/2255529745 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7e6000c810 con 0x7f7e68193270 2021-02-10T11:30:41.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.408 7f7e6f804700 1 -- 172.21.2.101:0/2255529745 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e68194090 con 0x7f7e68193270 2021-02-10T11:30:41.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.408 7f7e6f804700 1 -- 172.21.2.101:0/2255529745 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e68194090 con 0x7f7e68193270 2021-02-10T11:30:41.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.408 7f7e5dffb700 1 -- 172.21.2.101:0/2255529745 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7e60003080 con 0x7f7e68193270 2021-02-10T11:30:41.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.408 7f7e5dffb700 1 --2- 172.21.2.101:0/2255529745 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7e480186d0 0x7f7e4801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:41.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.408 7f7e5dffb700 1 -- 172.21.2.101:0/2255529745 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7e60032a30 con 0x7f7e68193270 2021-02-10T11:30:41.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.408 7f7e6d620700 1 --2- 172.21.2.101:0/2255529745 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7e480186d0 0x7f7e4801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:41.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.409 7f7e6d620700 1 --2- 172.21.2.101:0/2255529745 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7e480186d0 0x7f7e4801abd0 crc :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:41.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.439 7f7e6f804700 1 -- 172.21.2.101:0/2255529745 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e68193c90 con 0x7f7e68193270 2021-02-10T11:30:41.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.442 7f7e5dffb700 1 -- 172.21.2.101:0/2255529745 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7e6000bdc0 con 0x7f7e68193270 2021-02-10T11:30:41.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.568 7f7e6f804700 1 -- 172.21.2.101:0/2255529745 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839305", "target": ["mgr", ""]}) v1 -- 0x7f7e68193c90 con 0x7f7e480186d0 2021-02-10T11:30:41.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.578 7f7e5dffb700 1 -- 172.21.2.101:0/2255529745 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f7e64009080 con 0x7f7e480186d0 2021-02-10T11:30:41.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.585 7f7e6f804700 1 -- 172.21.2.101:0/2255529745 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7e480186d0 msgr2=0x7f7e4801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:41.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.585 7f7e6f804700 1 --2- 172.21.2.101:0/2255529745 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7e480186d0 0x7f7e4801abd0 crc :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:41.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.585 7f7e6f804700 1 -- 172.21.2.101:0/2255529745 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e68193270 msgr2=0x7f7e68193670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:41.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.585 7f7e6f804700 1 --2- 172.21.2.101:0/2255529745 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e68193270 0x7f7e68193670 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f7e6000b8e0 tx=0x7f7e6000bc80).stop 2021-02-10T11:30:41.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.585 7f7e6f804700 1 -- 172.21.2.101:0/2255529745 shutdown_connections 2021-02-10T11:30:41.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.585 7f7e6f804700 1 --2- 172.21.2.101:0/2255529745 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7e480186d0 0x7f7e4801abd0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:41.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.585 7f7e6f804700 1 --2- 172.21.2.101:0/2255529745 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e68190d20 0x7f7e6808fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:41.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.585 7f7e6f804700 1 --2- 172.21.2.101:0/2255529745 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e68193270 0x7f7e68193670 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:41.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.585 7f7e6f804700 1 --2- 172.21.2.101:0/2255529745 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7e68190920 0x7f7e6808f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:41.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.585 7f7e6f804700 1 -- 172.21.2.101:0/2255529745 >> 172.21.2.101:0/2255529745 conn(0x7f7e680bf4e0 msgr2=0x7f7e680be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:41.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.585 7f7e6f804700 1 -- 172.21.2.101:0/2255529745 shutdown_connections 2021-02-10T11:30:41.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:41.585 7f7e6f804700 1 -- 172.21.2.101:0/2255529745 wait complete. 2021-02-10T11:30:41.605 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839306 2021-02-10T11:30:42.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.007 7fac2c735700 1 Processor -- start 2021-02-10T11:30:42.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.007 7fac2c735700 1 -- start start 2021-02-10T11:30:42.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.008 7fac2c735700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fac241b0870 0x7fac240f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:42.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.008 7fac2c735700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac241b0c70 con 0x7fac240efe00 2021-02-10T11:30:42.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.008 7fac2c735700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac240f0b50 con 0x7fac241b0470 2021-02-10T11:30:42.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.008 7fac2c735700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac240e5ea0 con 0x7fac241b0870 2021-02-10T11:30:42.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.008 7fac2ad52700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fac240efe00 0x7fac240f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49818/0 (socket says 172.21.2.101:49818) 2021-02-10T11:30:42.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.008 7fac2ad52700 1 -- 172.21.2.101:0/2271806007 learned_addr learned my addr 172.21.2.101:0/2271806007 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:42.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.008 7fac29d50700 1 --2- 172.21.2.101:0/2271806007 >> v2:172.21.2.108:3300/0 conn(0x7fac241b0870 0x7fac240f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:42.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.008 7fac2954f700 1 -- 172.21.2.101:0/2271806007 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3168725494 0 0) 0x7fac1c001500 con 0x7fac240efe00 2021-02-10T11:30:42.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.008 7fac2954f700 1 -- 172.21.2.101:0/2271806007 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac10002fe0 con 0x7fac240efe00 2021-02-10T11:30:42.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.009 7fac29d50700 1 -- 172.21.2.101:0/2271806007 >> v1:172.21.2.108:6789/0 conn(0x7fac241b0470 legacy=0x7fac240ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:42.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.009 7fac29d50700 1 -- 172.21.2.101:0/2271806007 >> v1:172.21.2.101:6789/0 conn(0x7fac240efe00 legacy=0x7fac240f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:42.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.009 7fac29d50700 1 -- 172.21.2.101:0/2271806007 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac240e52e0 con 0x7fac241b0870 2021-02-10T11:30:42.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.009 7fac29d50700 1 --2- 172.21.2.101:0/2271806007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fac241b0870 0x7fac240f20e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fac20006650 tx=0x7fac20009a80).ready entity=mon.1 client_cookie=f5142ab029d66b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:42.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.009 7fac2954f700 1 -- 172.21.2.101:0/2271806007 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fac2000c0c0 con 0x7fac241b0870 2021-02-10T11:30:42.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.009 7fac2954f700 1 -- 172.21.2.101:0/2271806007 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2000c340 con 0x7fac241b0870 2021-02-10T11:30:42.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.009 7fac2954f700 1 -- 172.21.2.101:0/2271806007 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fac2000c860 con 0x7fac241b0870 2021-02-10T11:30:42.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.009 7fac2c735700 1 -- 172.21.2.101:0/2271806007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fac241b0870 msgr2=0x7fac240f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:42.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.009 7fac2c735700 1 --2- 172.21.2.101:0/2271806007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fac241b0870 0x7fac240f20e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fac20006650 tx=0x7fac20009a80).stop 2021-02-10T11:30:42.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.010 7fac2c735700 1 -- 172.21.2.101:0/2271806007 shutdown_connections 2021-02-10T11:30:42.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.010 7fac2c735700 1 --2- 172.21.2.101:0/2271806007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fac241b0870 0x7fac240f20e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:42.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.010 7fac2c735700 1 -- 172.21.2.101:0/2271806007 >> 172.21.2.101:0/2271806007 conn(0x7fac240ce3e0 msgr2=0x7fac240e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:42.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.010 7fac2c735700 1 -- 172.21.2.101:0/2271806007 shutdown_connections 2021-02-10T11:30:42.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.010 7fac2c735700 1 -- 172.21.2.101:0/2271806007 wait complete. 2021-02-10T11:30:42.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.025 7fac2c735700 1 Processor -- start 2021-02-10T11:30:42.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.025 7fac2c735700 1 -- start start 2021-02-10T11:30:42.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.025 7fac2c735700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fac240a6c90 0x7fac241924c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:42.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.025 7fac2c735700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fac24192ae0 0x7fac24192ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:42.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.025 7fac2c735700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fac240919e0 0x7fac24091de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:42.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.025 7fac2c735700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2419d2e0 con 0x7fac240a6c90 2021-02-10T11:30:42.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.025 7fac2c735700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac240914d0 con 0x7fac24192ae0 2021-02-10T11:30:42.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.025 7fac2c735700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac240a6820 con 0x7fac240919e0 2021-02-10T11:30:42.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.025 7fac2a551700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fac240a6c90 0x7fac241924c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:42.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.026 7fac2a551700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fac240a6c90 0x7fac241924c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49326/0 (socket says 172.21.2.101:49326) 2021-02-10T11:30:42.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.026 7fac2a551700 1 -- 172.21.2.101:0/1180615817 learned_addr learned my addr 172.21.2.101:0/1180615817 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:42.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.026 7fac2ad52700 1 --2- 172.21.2.101:0/1180615817 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fac240919e0 0x7fac24091de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:42.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.026 7fac29d50700 1 --2- 172.21.2.101:0/1180615817 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fac24192ae0 0x7fac24192ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:42.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.026 7fac2a551700 1 -- 172.21.2.101:0/1180615817 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fac240919e0 msgr2=0x7fac24091de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:42.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.026 7fac2a551700 1 --2- 172.21.2.101:0/1180615817 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fac240919e0 0x7fac24091de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:42.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.026 7fac2a551700 1 -- 172.21.2.101:0/1180615817 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fac24192ae0 msgr2=0x7fac24192ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:42.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.026 7fac2a551700 1 --2- 172.21.2.101:0/1180615817 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fac24192ae0 0x7fac24192ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:42.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.026 7fac2a551700 1 -- 172.21.2.101:0/1180615817 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac24092400 con 0x7fac240a6c90 2021-02-10T11:30:42.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.026 7fac2a551700 1 --2- 172.21.2.101:0/1180615817 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fac240a6c90 0x7fac241924c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7fac18006ed0 tx=0x7fac1800b8e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:42.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.026 7fac16ffd700 1 -- 172.21.2.101:0/1180615817 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fac18006240 con 0x7fac240a6c90 2021-02-10T11:30:42.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.026 7fac16ffd700 1 -- 172.21.2.101:0/1180615817 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac180054b0 con 0x7fac240a6c90 2021-02-10T11:30:42.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.026 7fac16ffd700 1 -- 172.21.2.101:0/1180615817 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fac180059d0 con 0x7fac240a6c90 2021-02-10T11:30:42.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.026 7fac2c735700 1 -- 172.21.2.101:0/1180615817 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac24092400 con 0x7fac240a6c90 2021-02-10T11:30:42.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.026 7fac2c735700 1 -- 172.21.2.101:0/1180615817 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac24092990 con 0x7fac240a6c90 2021-02-10T11:30:42.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.027 7fac16ffd700 1 -- 172.21.2.101:0/1180615817 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fac18006030 con 0x7fac240a6c90 2021-02-10T11:30:42.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.027 7fac16ffd700 1 --2- 172.21.2.101:0/1180615817 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fac0c0186c0 0x7fac0c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:42.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.027 7fac16ffd700 1 -- 172.21.2.101:0/1180615817 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fac18032590 con 0x7fac240a6c90 2021-02-10T11:30:42.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.027 7fac29d50700 1 --2- 172.21.2.101:0/1180615817 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fac0c0186c0 0x7fac0c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:42.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.027 7fac29d50700 1 --2- 172.21.2.101:0/1180615817 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fac0c0186c0 0x7fac0c01abc0 crc :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:42.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.057 7fac2c735700 1 -- 172.21.2.101:0/1180615817 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac240e52e0 con 0x7fac240a6c90 2021-02-10T11:30:42.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.060 7fac16ffd700 1 -- 172.21.2.101:0/1180615817 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fac18030d40 con 0x7fac240a6c90 2021-02-10T11:30:42.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.187 7fac2c735700 1 -- 172.21.2.101:0/1180615817 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839306", "target": ["mgr", ""]}) v1 -- 0x7fac24092990 con 0x7fac0c0186c0 2021-02-10T11:30:42.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.197 7fac16ffd700 1 -- 172.21.2.101:0/1180615817 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fac20006740 con 0x7fac0c0186c0 2021-02-10T11:30:42.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.204 7fac2c735700 1 -- 172.21.2.101:0/1180615817 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fac0c0186c0 msgr2=0x7fac0c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:42.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.204 7fac2c735700 1 --2- 172.21.2.101:0/1180615817 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fac0c0186c0 0x7fac0c01abc0 crc :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:42.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.204 7fac2c735700 1 -- 172.21.2.101:0/1180615817 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fac240a6c90 msgr2=0x7fac241924c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:42.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.204 7fac2c735700 1 --2- 172.21.2.101:0/1180615817 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fac240a6c90 0x7fac241924c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7fac18006ed0 tx=0x7fac1800b8e0).stop 2021-02-10T11:30:42.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.204 7fac2c735700 1 -- 172.21.2.101:0/1180615817 shutdown_connections 2021-02-10T11:30:42.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.204 7fac2c735700 1 --2- 172.21.2.101:0/1180615817 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fac0c0186c0 0x7fac0c01abc0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:42.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.204 7fac2c735700 1 --2- 172.21.2.101:0/1180615817 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fac240919e0 0x7fac24091de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:42.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.204 7fac2c735700 1 --2- 172.21.2.101:0/1180615817 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fac24192ae0 0x7fac24192ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:42.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.204 7fac2c735700 1 --2- 172.21.2.101:0/1180615817 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fac240a6c90 0x7fac241924c0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:42.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.204 7fac2c735700 1 -- 172.21.2.101:0/1180615817 >> 172.21.2.101:0/1180615817 conn(0x7fac240ac550 msgr2=0x7fac240be1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:42.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.204 7fac2c735700 1 -- 172.21.2.101:0/1180615817 shutdown_connections 2021-02-10T11:30:42.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.204 7fac2c735700 1 -- 172.21.2.101:0/1180615817 wait complete. 2021-02-10T11:30:42.224 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839307 2021-02-10T11:30:42.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.625 7fc9b421d700 1 Processor -- start 2021-02-10T11:30:42.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.625 7fc9b421d700 1 -- start start 2021-02-10T11:30:42.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.626 7fc9b421d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc9ac1b0470 0x7fc9ac1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:42.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.626 7fc9b421d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc9ac0f20e0 0x7fc9ac0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:42.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.626 7fc9b421d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ac0f0b50 con 0x7fc9ac0ef610 2021-02-10T11:30:42.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.626 7fc9b421d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ac0f0820 con 0x7fc9ac1b0470 2021-02-10T11:30:42.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.626 7fc9b421d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ac0f24e0 con 0x7fc9ac0f20e0 2021-02-10T11:30:42.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.626 7fc9b2039700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc9ac1b0470 0x7fc9ac1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:42.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.626 7fc9b1838700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc9ac0ef610 0x7fc9ac0efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49828/0 (socket says 172.21.2.101:49828) 2021-02-10T11:30:42.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.626 7fc9b1838700 1 -- 172.21.2.101:0/3215857976 learned_addr learned my addr 172.21.2.101:0/3215857976 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:42.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.626 7fc9b283a700 1 --2- 172.21.2.101:0/3215857976 >> v2:172.21.2.108:3300/0 conn(0x7fc9ac0f20e0 0x7fc9ac0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:42.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.626 7fc9b2039700 1 -- 172.21.2.101:0/3215857976 >> v2:172.21.2.108:3300/0 conn(0x7fc9ac0f20e0 msgr2=0x7fc9ac0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:42.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.626 7fc9b2039700 1 --2- 172.21.2.101:0/3215857976 >> v2:172.21.2.108:3300/0 conn(0x7fc9ac0f20e0 0x7fc9ac0f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:42.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.626 7fc9b1037700 1 -- 172.21.2.101:0/3215857976 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 16515751 0 0) 0x7fc9a80010b0 con 0x7fc9ac0ef610 2021-02-10T11:30:42.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.626 7fc9b2039700 1 -- 172.21.2.101:0/3215857976 >> v1:172.21.2.101:6789/0 conn(0x7fc9ac0ef610 legacy=0x7fc9ac0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:42.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.626 7fc9b2039700 1 -- 172.21.2.101:0/3215857976 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac0e52e0 con 0x7fc9ac1b0470 2021-02-10T11:30:42.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.627 7fc9b2039700 1 --2- 172.21.2.101:0/3215857976 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9ac1b0470 0x7fc9ac1b0870 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7fc99c006650 tx=0x7fc99c009a80).ready entity=mon.0 client_cookie=bb969f924d34e575 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:42.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.627 7fc9b1037700 1 -- 172.21.2.101:0/3215857976 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc99c0009b0 con 0x7fc9ac1b0470 2021-02-10T11:30:42.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.627 7fc9b1037700 1 -- 172.21.2.101:0/3215857976 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99c00c0c0 con 0x7fc9ac1b0470 2021-02-10T11:30:42.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.627 7fc9b1037700 1 -- 172.21.2.101:0/3215857976 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc99c00c520 con 0x7fc9ac1b0470 2021-02-10T11:30:42.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.627 7fc9b421d700 1 -- 172.21.2.101:0/3215857976 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9ac1b0470 msgr2=0x7fc9ac1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:42.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.627 7fc9b421d700 1 --2- 172.21.2.101:0/3215857976 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9ac1b0470 0x7fc9ac1b0870 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7fc99c006650 tx=0x7fc99c009a80).stop 2021-02-10T11:30:42.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.628 7fc9b421d700 1 -- 172.21.2.101:0/3215857976 shutdown_connections 2021-02-10T11:30:42.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.628 7fc9b421d700 1 --2- 172.21.2.101:0/3215857976 >> v2:172.21.2.108:3300/0 conn(0x7fc9ac0f20e0 0x7fc9ac0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:42.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.628 7fc9b421d700 1 --2- 172.21.2.101:0/3215857976 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9ac1b0470 0x7fc9ac1b0870 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:42.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.628 7fc9b421d700 1 -- 172.21.2.101:0/3215857976 >> 172.21.2.101:0/3215857976 conn(0x7fc9ac0ce3e0 msgr2=0x7fc9ac0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:42.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.628 7fc9b421d700 1 -- 172.21.2.101:0/3215857976 shutdown_connections 2021-02-10T11:30:42.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.628 7fc9b421d700 1 -- 172.21.2.101:0/3215857976 wait complete. 2021-02-10T11:30:42.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.643 7fc9b421d700 1 Processor -- start 2021-02-10T11:30:42.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.643 7fc9b421d700 1 -- start start 2021-02-10T11:30:42.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.643 7fc9b421d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9ac0a0420 0x7fc9ac0a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:42.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.643 7fc9b421d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9ac189d00 0x7fc9ac18a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:42.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.643 7fc9b421d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9ac18a720 0x7fc9ac0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:42.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.643 7fc9b421d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac189780 con 0x7fc9ac189d00 2021-02-10T11:30:42.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.643 7fc9b421d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac18ab20 con 0x7fc9ac0a0420 2021-02-10T11:30:42.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.643 7fc9b421d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac0a1140 con 0x7fc9ac18a720 2021-02-10T11:30:42.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.643 7fc9b1838700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9ac189d00 0x7fc9ac18a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:42.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.643 7fc9b2039700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9ac0a0420 0x7fc9ac0a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:42.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.643 7fc9b1838700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9ac189d00 0x7fc9ac18a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49340/0 (socket says 172.21.2.101:49340) 2021-02-10T11:30:42.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.643 7fc9b1838700 1 -- 172.21.2.101:0/3816698825 learned_addr learned my addr 172.21.2.101:0/3816698825 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:42.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.643 7fc9b2039700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9ac0a0420 0x7fc9ac0a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:59828/0 (socket says 172.21.2.101:59828) 2021-02-10T11:30:42.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.643 7fc9b283a700 1 --2- 172.21.2.101:0/3816698825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9ac18a720 0x7fc9ac0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:42.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.644 7fc9b2039700 1 -- 172.21.2.101:0/3816698825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9ac18a720 msgr2=0x7fc9ac0b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:42.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.644 7fc9b2039700 1 --2- 172.21.2.101:0/3816698825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9ac18a720 0x7fc9ac0b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:42.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.644 7fc9b2039700 1 -- 172.21.2.101:0/3816698825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9ac189d00 msgr2=0x7fc9ac18a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:42.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.644 7fc9b2039700 1 --2- 172.21.2.101:0/3816698825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9ac189d00 0x7fc9ac18a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:42.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.644 7fc9b2039700 1 -- 172.21.2.101:0/3816698825 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac0b8200 con 0x7fc9ac0a0420 2021-02-10T11:30:42.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.644 7fc9b2039700 1 --2- 172.21.2.101:0/3816698825 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9ac0a0420 0x7fc9ac0a0820 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fc99c00b530 tx=0x7fc99c00b8d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:42.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.644 7fc9a2ffd700 1 -- 172.21.2.101:0/3816698825 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc99c009280 con 0x7fc9ac0a0420 2021-02-10T11:30:42.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.644 7fc9a2ffd700 1 -- 172.21.2.101:0/3816698825 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99c006740 con 0x7fc9ac0a0420 2021-02-10T11:30:42.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.644 7fc9a2ffd700 1 -- 172.21.2.101:0/3816698825 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc99c00bcc0 con 0x7fc9ac0a0420 2021-02-10T11:30:42.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.644 7fc9b421d700 1 -- 172.21.2.101:0/3816698825 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ac0b8200 con 0x7fc9ac0a0420 2021-02-10T11:30:42.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.644 7fc9b421d700 1 -- 172.21.2.101:0/3816698825 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ac0b8200 con 0x7fc9ac0a0420 2021-02-10T11:30:42.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.645 7fc9a2ffd700 1 -- 172.21.2.101:0/3816698825 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc99c000c90 con 0x7fc9ac0a0420 2021-02-10T11:30:42.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.645 7fc9a2ffd700 1 --2- 172.21.2.101:0/3816698825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc994018730 0x7fc99401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:42.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.645 7fc9a2ffd700 1 -- 172.21.2.101:0/3816698825 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc99c0325b0 con 0x7fc9ac0a0420 2021-02-10T11:30:42.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.645 7fc9b1838700 1 --2- 172.21.2.101:0/3816698825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc994018730 0x7fc99401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:42.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.645 7fc9b1838700 1 --2- 172.21.2.101:0/3816698825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc994018730 0x7fc99401ac10 crc :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:42.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.675 7fc9b421d700 1 -- 172.21.2.101:0/3816698825 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ac105ae0 con 0x7fc9ac0a0420 2021-02-10T11:30:42.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.677 7fc9a2ffd700 1 -- 172.21.2.101:0/3816698825 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc99c006740 con 0x7fc9ac0a0420 2021-02-10T11:30:42.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.805 7fc9b421d700 1 -- 172.21.2.101:0/3816698825 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839307", "target": ["mgr", ""]}) v1 -- 0x7fc9ac105ae0 con 0x7fc994018730 2021-02-10T11:30:42.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.815 7fc9a2ffd700 1 -- 172.21.2.101:0/3816698825 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fc9a80009b0 con 0x7fc994018730 2021-02-10T11:30:42.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.822 7fc9b421d700 1 -- 172.21.2.101:0/3816698825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc994018730 msgr2=0x7fc99401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:42.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.822 7fc9b421d700 1 --2- 172.21.2.101:0/3816698825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc994018730 0x7fc99401ac10 crc :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:42.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.822 7fc9b421d700 1 -- 172.21.2.101:0/3816698825 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9ac0a0420 msgr2=0x7fc9ac0a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:42.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.822 7fc9b421d700 1 --2- 172.21.2.101:0/3816698825 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9ac0a0420 0x7fc9ac0a0820 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fc99c00b530 tx=0x7fc99c00b8d0).stop 2021-02-10T11:30:42.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.822 7fc9b421d700 1 -- 172.21.2.101:0/3816698825 shutdown_connections 2021-02-10T11:30:42.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.822 7fc9b421d700 1 --2- 172.21.2.101:0/3816698825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc994018730 0x7fc99401ac10 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:42.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.822 7fc9b421d700 1 --2- 172.21.2.101:0/3816698825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9ac18a720 0x7fc9ac0b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:42.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.822 7fc9b421d700 1 --2- 172.21.2.101:0/3816698825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9ac189d00 0x7fc9ac18a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:42.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.822 7fc9b421d700 1 --2- 172.21.2.101:0/3816698825 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9ac0a0420 0x7fc9ac0a0820 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:42.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.822 7fc9b421d700 1 -- 172.21.2.101:0/3816698825 >> 172.21.2.101:0/3816698825 conn(0x7fc9ac094e10 msgr2=0x7fc9ac0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:42.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.823 7fc9b421d700 1 -- 172.21.2.101:0/3816698825 shutdown_connections 2021-02-10T11:30:42.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:42.823 7fc9b421d700 1 -- 172.21.2.101:0/3816698825 wait complete. 2021-02-10T11:30:42.843 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839308 2021-02-10T11:30:43.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.250 7f3249a21700 1 Processor -- start 2021-02-10T11:30:43.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.250 7f3249a21700 1 -- start start 2021-02-10T11:30:43.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.250 7f3249a21700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f32440f05f0 0x7f32440f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:43.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.250 7f3249a21700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32441b0c70 con 0x7f32441b0870 2021-02-10T11:30:43.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.250 7f3249a21700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32440f0b50 con 0x7f32441b0470 2021-02-10T11:30:43.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.250 7f3249a21700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32440e5ea0 con 0x7f32440f05f0 2021-02-10T11:30:43.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.250 7f3242ffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f32441b0870 0x7f32440efe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49842/0 (socket says 172.21.2.101:49842) 2021-02-10T11:30:43.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.250 7f3242ffd700 1 -- 172.21.2.101:0/3408739903 learned_addr learned my addr 172.21.2.101:0/3408739903 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:43.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.250 7f3243fff700 1 --2- 172.21.2.101:0/3408739903 >> v2:172.21.2.101:3300/0 conn(0x7f32440f05f0 0x7f32440f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:43.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.251 7f32427fc700 1 -- 172.21.2.101:0/3408739903 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822208846 0 0) 0x7f323c001500 con 0x7f32441b0870 2021-02-10T11:30:43.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.251 7f32427fc700 1 -- 172.21.2.101:0/3408739903 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f322c002fe0 con 0x7f32441b0870 2021-02-10T11:30:43.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.251 7f3243fff700 1 -- 172.21.2.101:0/3408739903 >> v1:172.21.2.108:6789/0 conn(0x7f32441b0470 legacy=0x7f32440ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:43.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.251 7f3243fff700 1 -- 172.21.2.101:0/3408739903 >> v1:172.21.2.101:6789/0 conn(0x7f32441b0870 legacy=0x7f32440efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:43.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.251 7f3243fff700 1 -- 172.21.2.101:0/3408739903 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32440e52e0 con 0x7f32440f05f0 2021-02-10T11:30:43.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.251 7f3243fff700 1 --2- 172.21.2.101:0/3408739903 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32440f05f0 0x7f32440f20e0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7f3234006ab0 tx=0x7f3234009b40).ready entity=mon.0 client_cookie=987fb5856ecfadd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:43.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.251 7f32427fc700 1 -- 172.21.2.101:0/3408739903 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f32340022e0 con 0x7f32440f05f0 2021-02-10T11:30:43.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.251 7f32427fc700 1 -- 172.21.2.101:0/3408739903 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3234002560 con 0x7f32440f05f0 2021-02-10T11:30:43.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.251 7f32427fc700 1 -- 172.21.2.101:0/3408739903 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3234002930 con 0x7f32440f05f0 2021-02-10T11:30:43.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.251 7f3249a21700 1 -- 172.21.2.101:0/3408739903 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32440f05f0 msgr2=0x7f32440f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:43.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.251 7f3249a21700 1 --2- 172.21.2.101:0/3408739903 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32440f05f0 0x7f32440f20e0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7f3234006ab0 tx=0x7f3234009b40).stop 2021-02-10T11:30:43.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.252 7f3249a21700 1 -- 172.21.2.101:0/3408739903 shutdown_connections 2021-02-10T11:30:43.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.252 7f3249a21700 1 --2- 172.21.2.101:0/3408739903 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32440f05f0 0x7f32440f20e0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:43.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.252 7f3249a21700 1 -- 172.21.2.101:0/3408739903 >> 172.21.2.101:0/3408739903 conn(0x7f32440ce3e0 msgr2=0x7f32440e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:43.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.252 7f3249a21700 1 -- 172.21.2.101:0/3408739903 shutdown_connections 2021-02-10T11:30:43.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.252 7f3249a21700 1 -- 172.21.2.101:0/3408739903 wait complete. 2021-02-10T11:30:43.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.266 7f3249a21700 1 Processor -- start 2021-02-10T11:30:43.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.267 7f3249a21700 1 -- start start 2021-02-10T11:30:43.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.267 7f3249a21700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f32440b2840 0x7f32441a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:43.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.267 7f3249a21700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32440b30d0 0x7f32440b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:43.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.267 7f3249a21700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32440b3af0 0x7f32440f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:43.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.267 7f3249a21700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32440b2c40 con 0x7f32440b3af0 2021-02-10T11:30:43.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.267 7f3249a21700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32441a4920 con 0x7f32440b2840 2021-02-10T11:30:43.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.267 7f3249a21700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32440f0ba0 con 0x7f32440b30d0 2021-02-10T11:30:43.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.267 7f3243fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32440b3af0 0x7f32440f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:43.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.267 7f3243fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32440b3af0 0x7f32440f0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49358/0 (socket says 172.21.2.101:49358) 2021-02-10T11:30:43.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.267 7f3243fff700 1 -- 172.21.2.101:0/3046345881 learned_addr learned my addr 172.21.2.101:0/3046345881 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:43.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.267 7f3242ffd700 1 --2- 172.21.2.101:0/3046345881 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32440b30d0 0x7f32440b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:43.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.267 7f32437fe700 1 --2- 172.21.2.101:0/3046345881 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f32440b2840 0x7f32441a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:43.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.268 7f3243fff700 1 -- 172.21.2.101:0/3046345881 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32440b30d0 msgr2=0x7f32440b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:43.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.268 7f3243fff700 1 --2- 172.21.2.101:0/3046345881 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32440b30d0 0x7f32440b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:43.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.268 7f3243fff700 1 -- 172.21.2.101:0/3046345881 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f32440b2840 msgr2=0x7f32441a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:43.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.268 7f3243fff700 1 --2- 172.21.2.101:0/3046345881 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f32440b2840 0x7f32441a4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:43.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.268 7f3243fff700 1 -- 172.21.2.101:0/3046345881 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32440f0dc0 con 0x7f32440b3af0 2021-02-10T11:30:43.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.268 7f3243fff700 1 --2- 172.21.2.101:0/3046345881 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32440b3af0 0x7f32440f0580 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7f323400b5f0 tx=0x7f323400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:43.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.268 7f322bfff700 1 -- 172.21.2.101:0/3046345881 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f323400bd80 con 0x7f32440b3af0 2021-02-10T11:30:43.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.268 7f322bfff700 1 -- 172.21.2.101:0/3046345881 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3234006020 con 0x7f32440b3af0 2021-02-10T11:30:43.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.268 7f3249a21700 1 -- 172.21.2.101:0/3046345881 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32440f0dc0 con 0x7f32440b3af0 2021-02-10T11:30:43.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.268 7f3249a21700 1 -- 172.21.2.101:0/3046345881 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32440f1350 con 0x7f32440b3af0 2021-02-10T11:30:43.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.268 7f322bfff700 1 -- 172.21.2.101:0/3046345881 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f32340054a0 con 0x7f32440b3af0 2021-02-10T11:30:43.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.269 7f322bfff700 1 -- 172.21.2.101:0/3046345881 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3234006200 con 0x7f32440b3af0 2021-02-10T11:30:43.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.269 7f322bfff700 1 --2- 172.21.2.101:0/3046345881 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3224018730 0x7f322401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:43.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.269 7f322bfff700 1 -- 172.21.2.101:0/3046345881 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3234032800 con 0x7f32440b3af0 2021-02-10T11:30:43.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.269 7f32437fe700 1 --2- 172.21.2.101:0/3046345881 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3224018730 0x7f322401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:43.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.269 7f32437fe700 1 --2- 172.21.2.101:0/3046345881 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3224018730 0x7f322401ac10 crc :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:43.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.300 7f3249a21700 1 -- 172.21.2.101:0/3046345881 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32440f0050 con 0x7f32440b3af0 2021-02-10T11:30:43.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.302 7f322bfff700 1 -- 172.21.2.101:0/3046345881 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f32340093e0 con 0x7f32440b3af0 2021-02-10T11:30:43.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.431 7f3249a21700 1 -- 172.21.2.101:0/3046345881 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839308", "target": ["mgr", ""]}) v1 -- 0x7f32440f1350 con 0x7f3224018730 2021-02-10T11:30:43.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.442 7f322bfff700 1 -- 172.21.2.101:0/3046345881 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f3238006550 con 0x7f3224018730 2021-02-10T11:30:43.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.448 7f3249a21700 1 -- 172.21.2.101:0/3046345881 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3224018730 msgr2=0x7f322401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:43.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.448 7f3249a21700 1 --2- 172.21.2.101:0/3046345881 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3224018730 0x7f322401ac10 crc :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:43.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.448 7f3249a21700 1 -- 172.21.2.101:0/3046345881 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32440b3af0 msgr2=0x7f32440f0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:43.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.448 7f3249a21700 1 --2- 172.21.2.101:0/3046345881 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32440b3af0 0x7f32440f0580 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7f323400b5f0 tx=0x7f323400b990).stop 2021-02-10T11:30:43.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.449 7f3249a21700 1 -- 172.21.2.101:0/3046345881 shutdown_connections 2021-02-10T11:30:43.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.449 7f3249a21700 1 --2- 172.21.2.101:0/3046345881 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3224018730 0x7f322401ac10 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:43.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.449 7f3249a21700 1 --2- 172.21.2.101:0/3046345881 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32440b30d0 0x7f32440b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:43.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.449 7f3249a21700 1 --2- 172.21.2.101:0/3046345881 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32440b3af0 0x7f32440f0580 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:43.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.449 7f3249a21700 1 --2- 172.21.2.101:0/3046345881 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f32440b2840 0x7f32441a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:43.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.449 7f3249a21700 1 -- 172.21.2.101:0/3046345881 >> 172.21.2.101:0/3046345881 conn(0x7f32440c36f0 msgr2=0x7f3244094c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:43.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.449 7f3249a21700 1 -- 172.21.2.101:0/3046345881 shutdown_connections 2021-02-10T11:30:43.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.449 7f3249a21700 1 -- 172.21.2.101:0/3046345881 wait complete. 2021-02-10T11:30:43.469 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839309 2021-02-10T11:30:43.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.874 7f9d89f50700 1 Processor -- start 2021-02-10T11:30:43.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.874 7f9d89f50700 1 -- start start 2021-02-10T11:30:43.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.874 7f9d89f50700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f9d841b0470 0x7f9d841b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:43.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.874 7f9d89f50700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d840f0b50 con 0x7f9d840ef610 2021-02-10T11:30:43.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.874 7f9d89f50700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d840f24e0 con 0x7f9d840f20e0 2021-02-10T11:30:43.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.874 7f9d89f50700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d840e5ea0 con 0x7f9d841b0470 2021-02-10T11:30:43.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.874 7f9d82ffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f9d840ef610 0x7f9d840efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49856/0 (socket says 172.21.2.101:49856) 2021-02-10T11:30:43.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.874 7f9d82ffd700 1 -- 172.21.2.101:0/465374174 learned_addr learned my addr 172.21.2.101:0/465374174 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:43.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.875 7f9d837fe700 1 --2- 172.21.2.101:0/465374174 >> v2:172.21.2.108:3300/0 conn(0x7f9d841b0470 0x7f9d841b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:43.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.875 7f9d827fc700 1 -- 172.21.2.101:0/465374174 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1977225467 0 0) 0x7f9d740010b0 con 0x7f9d840ef610 2021-02-10T11:30:43.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.875 7f9d827fc700 1 -- 172.21.2.101:0/465374174 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d70002fe0 con 0x7f9d840ef610 2021-02-10T11:30:43.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.875 7f9d837fe700 1 -- 172.21.2.101:0/465374174 >> v1:172.21.2.108:6789/0 conn(0x7f9d840f20e0 legacy=0x7f9d840f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:43.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.875 7f9d837fe700 1 -- 172.21.2.101:0/465374174 >> v1:172.21.2.101:6789/0 conn(0x7f9d840ef610 legacy=0x7f9d840efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:43.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.875 7f9d837fe700 1 -- 172.21.2.101:0/465374174 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d840e52e0 con 0x7f9d841b0470 2021-02-10T11:30:43.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.875 7f9d837fe700 1 --2- 172.21.2.101:0/465374174 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d841b0470 0x7f9d841b0870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f9d6c006650 tx=0x7f9d6c009a80).ready entity=mon.1 client_cookie=2066d69f7d8e0b7e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:43.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.875 7f9d827fc700 1 -- 172.21.2.101:0/465374174 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d6c00c0c0 con 0x7f9d841b0470 2021-02-10T11:30:43.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.875 7f9d827fc700 1 -- 172.21.2.101:0/465374174 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6c00c340 con 0x7f9d841b0470 2021-02-10T11:30:43.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.875 7f9d827fc700 1 -- 172.21.2.101:0/465374174 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d6c00c860 con 0x7f9d841b0470 2021-02-10T11:30:43.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.875 7f9d89f50700 1 -- 172.21.2.101:0/465374174 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d841b0470 msgr2=0x7f9d841b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:43.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.875 7f9d89f50700 1 --2- 172.21.2.101:0/465374174 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d841b0470 0x7f9d841b0870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f9d6c006650 tx=0x7f9d6c009a80).stop 2021-02-10T11:30:43.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.876 7f9d89f50700 1 -- 172.21.2.101:0/465374174 shutdown_connections 2021-02-10T11:30:43.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.876 7f9d89f50700 1 --2- 172.21.2.101:0/465374174 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d841b0470 0x7f9d841b0870 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:43.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.876 7f9d89f50700 1 -- 172.21.2.101:0/465374174 >> 172.21.2.101:0/465374174 conn(0x7f9d840ce3e0 msgr2=0x7f9d840e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:43.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.876 7f9d89f50700 1 -- 172.21.2.101:0/465374174 shutdown_connections 2021-02-10T11:30:43.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.876 7f9d89f50700 1 -- 172.21.2.101:0/465374174 wait complete. 2021-02-10T11:30:43.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.891 7f9d89f50700 1 Processor -- start 2021-02-10T11:30:43.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.891 7f9d89f50700 1 -- start start 2021-02-10T11:30:43.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.891 7f9d89f50700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d840c36a0 0x7f9d840c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:43.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.892 7f9d89f50700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d840c3190 0x7f9d840b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:43.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.892 7f9d89f50700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d840b2370 0x7f9d840b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:43.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.892 7f9d89f50700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d840b27c0 con 0x7f9d840c3190 2021-02-10T11:30:43.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.892 7f9d89f50700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d840b3360 con 0x7f9d840b2370 2021-02-10T11:30:43.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.892 7f9d89f50700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d840b3540 con 0x7f9d840c36a0 2021-02-10T11:30:43.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.892 7f9d82ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d840c3190 0x7f9d840b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:43.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.892 7f9d82ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d840c3190 0x7f9d840b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49366/0 (socket says 172.21.2.101:49366) 2021-02-10T11:30:43.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.892 7f9d82ffd700 1 -- 172.21.2.101:0/1999118597 learned_addr learned my addr 172.21.2.101:0/1999118597 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:43.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.892 7f9d82ffd700 1 -- 172.21.2.101:0/1999118597 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d840c36a0 msgr2=0x7f9d840c2b70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:30:43.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.892 7f9d82ffd700 1 --2- 172.21.2.101:0/1999118597 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d840c36a0 0x7f9d840c2b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:43.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.892 7f9d82ffd700 1 -- 172.21.2.101:0/1999118597 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d840b2370 msgr2=0x7f9d840b2d40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:30:43.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.892 7f9d82ffd700 1 --2- 172.21.2.101:0/1999118597 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d840b2370 0x7f9d840b2d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:43.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.892 7f9d82ffd700 1 -- 172.21.2.101:0/1999118597 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d840b3760 con 0x7f9d840c3190 2021-02-10T11:30:43.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.892 7f9d82ffd700 1 --2- 172.21.2.101:0/1999118597 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d840c3190 0x7f9d840b1d50 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7f9d74001c60 tx=0x7f9d7400a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:43.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.892 7f9d7bfff700 1 -- 172.21.2.101:0/1999118597 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d7400acc0 con 0x7f9d840c3190 2021-02-10T11:30:43.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.892 7f9d7bfff700 1 -- 172.21.2.101:0/1999118597 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d74004140 con 0x7f9d840c3190 2021-02-10T11:30:43.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.892 7f9d7bfff700 1 -- 172.21.2.101:0/1999118597 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d74004620 con 0x7f9d840c3190 2021-02-10T11:30:43.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.893 7f9d89f50700 1 -- 172.21.2.101:0/1999118597 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d840b3760 con 0x7f9d840c3190 2021-02-10T11:30:43.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.893 7f9d89f50700 1 -- 172.21.2.101:0/1999118597 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d840b3760 con 0x7f9d840c3190 2021-02-10T11:30:43.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.893 7f9d7bfff700 1 -- 172.21.2.101:0/1999118597 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9d74004320 con 0x7f9d840c3190 2021-02-10T11:30:43.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.893 7f9d7bfff700 1 --2- 172.21.2.101:0/1999118597 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d680186c0 0x7f9d6801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:43.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.893 7f9d7bfff700 1 -- 172.21.2.101:0/1999118597 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9d74032390 con 0x7f9d840c3190 2021-02-10T11:30:43.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.893 7f9d837fe700 1 --2- 172.21.2.101:0/1999118597 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d680186c0 0x7f9d6801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:43.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.894 7f9d837fe700 1 --2- 172.21.2.101:0/1999118597 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d680186c0 0x7f9d6801abc0 crc :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:43.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.924 7f9d89f50700 1 -- 172.21.2.101:0/1999118597 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d840edd10 con 0x7f9d840c3190 2021-02-10T11:30:43.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:43.926 7f9d7bfff700 1 -- 172.21.2.101:0/1999118597 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9d74025fa0 con 0x7f9d840c3190 2021-02-10T11:30:44.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.054 7f9d89f50700 1 -- 172.21.2.101:0/1999118597 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839309", "target": ["mgr", ""]}) v1 -- 0x7f9d840e52e0 con 0x7f9d680186c0 2021-02-10T11:30:44.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.064 7f9d7bfff700 1 -- 172.21.2.101:0/1999118597 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f9d6c000d10 con 0x7f9d680186c0 2021-02-10T11:30:44.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.071 7f9d89f50700 1 -- 172.21.2.101:0/1999118597 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d680186c0 msgr2=0x7f9d6801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:44.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.071 7f9d89f50700 1 --2- 172.21.2.101:0/1999118597 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d680186c0 0x7f9d6801abc0 crc :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:44.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.071 7f9d89f50700 1 -- 172.21.2.101:0/1999118597 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d840c3190 msgr2=0x7f9d840b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:44.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.071 7f9d89f50700 1 --2- 172.21.2.101:0/1999118597 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d840c3190 0x7f9d840b1d50 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7f9d74001c60 tx=0x7f9d7400a8d0).stop 2021-02-10T11:30:44.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.071 7f9d89f50700 1 -- 172.21.2.101:0/1999118597 shutdown_connections 2021-02-10T11:30:44.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.071 7f9d89f50700 1 --2- 172.21.2.101:0/1999118597 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d840b2370 0x7f9d840b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:44.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.071 7f9d89f50700 1 --2- 172.21.2.101:0/1999118597 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d840c3190 0x7f9d840b1d50 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:44.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.071 7f9d89f50700 1 --2- 172.21.2.101:0/1999118597 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d680186c0 0x7f9d6801abc0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:44.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.071 7f9d89f50700 1 --2- 172.21.2.101:0/1999118597 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d840c36a0 0x7f9d840c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:44.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.071 7f9d89f50700 1 -- 172.21.2.101:0/1999118597 >> 172.21.2.101:0/1999118597 conn(0x7f9d8409f630 msgr2=0x7f9d840a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:44.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.071 7f9d89f50700 1 -- 172.21.2.101:0/1999118597 shutdown_connections 2021-02-10T11:30:44.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.071 7f9d89f50700 1 -- 172.21.2.101:0/1999118597 wait complete. 2021-02-10T11:30:44.090 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839310 2021-02-10T11:30:44.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.495 7fa0c9fee700 1 Processor -- start 2021-02-10T11:30:44.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.496 7fa0c9fee700 1 -- start start 2021-02-10T11:30:44.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.496 7fa0c9fee700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa0c41b0870 0x7fa0c40f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:44.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.496 7fa0c9fee700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c41b0c70 con 0x7fa0c41b0470 2021-02-10T11:30:44.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.496 7fa0c9fee700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c40f0b50 con 0x7fa0c40efe00 2021-02-10T11:30:44.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.496 7fa0c9fee700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c40e5ea0 con 0x7fa0c41b0870 2021-02-10T11:30:44.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.496 7fa0c2ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa0c41b0870 0x7fa0c40f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:44.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.496 7fa0c37fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa0c41b0470 0x7fa0c40ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49868/0 (socket says 172.21.2.101:49868) 2021-02-10T11:30:44.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.496 7fa0c37fe700 1 -- 172.21.2.101:0/4246903962 learned_addr learned my addr 172.21.2.101:0/4246903962 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:44.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.496 7fa0c2ffd700 1 -- 172.21.2.101:0/4246903962 >> v1:172.21.2.108:6789/0 conn(0x7fa0c40efe00 legacy=0x7fa0c40f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:44.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.496 7fa0c2ffd700 1 -- 172.21.2.101:0/4246903962 >> v1:172.21.2.101:6789/0 conn(0x7fa0c41b0470 legacy=0x7fa0c40ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:44.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.496 7fa0c2ffd700 1 -- 172.21.2.101:0/4246903962 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c40e52e0 con 0x7fa0c41b0870 2021-02-10T11:30:44.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.497 7fa0c2ffd700 1 --2- 172.21.2.101:0/4246903962 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0c41b0870 0x7fa0c40f20e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7fa0bc0069d0 tx=0x7fa0bc009e40).ready entity=mon.0 client_cookie=318813eff38a497b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:44.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.497 7fa0c27fc700 1 -- 172.21.2.101:0/4246903962 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa0bc00b510 con 0x7fa0c41b0870 2021-02-10T11:30:44.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.497 7fa0c27fc700 1 -- 172.21.2.101:0/4246903962 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc00b790 con 0x7fa0c41b0870 2021-02-10T11:30:44.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.497 7fa0c27fc700 1 -- 172.21.2.101:0/4246903962 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa0bc00bcb0 con 0x7fa0c41b0870 2021-02-10T11:30:44.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.497 7fa0c9fee700 1 -- 172.21.2.101:0/4246903962 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0c41b0870 msgr2=0x7fa0c40f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:44.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.497 7fa0c9fee700 1 --2- 172.21.2.101:0/4246903962 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0c41b0870 0x7fa0c40f20e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7fa0bc0069d0 tx=0x7fa0bc009e40).stop 2021-02-10T11:30:44.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.497 7fa0c9fee700 1 -- 172.21.2.101:0/4246903962 shutdown_connections 2021-02-10T11:30:44.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.497 7fa0c9fee700 1 --2- 172.21.2.101:0/4246903962 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0c41b0870 0x7fa0c40f20e0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:44.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.497 7fa0c9fee700 1 -- 172.21.2.101:0/4246903962 >> 172.21.2.101:0/4246903962 conn(0x7fa0c40ce3e0 msgr2=0x7fa0c40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:44.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.497 7fa0c9fee700 1 -- 172.21.2.101:0/4246903962 shutdown_connections 2021-02-10T11:30:44.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.498 7fa0c9fee700 1 -- 172.21.2.101:0/4246903962 wait complete. 2021-02-10T11:30:44.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.512 7fa0c9fee700 1 Processor -- start 2021-02-10T11:30:44.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.512 7fa0c9fee700 1 -- start start 2021-02-10T11:30:44.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.512 7fa0c9fee700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa0c4087510 0x7fa0c40aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:44.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.512 7fa0c9fee700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa0c41c68f0 0x7fa0c41b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:44.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.512 7fa0c9fee700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0c408c4d0 0x7fa0c41b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:44.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.512 7fa0c9fee700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40a2d50 con 0x7fa0c408c4d0 2021-02-10T11:30:44.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.512 7fa0c9fee700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40a2f70 con 0x7fa0c41c68f0 2021-02-10T11:30:44.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.512 7fa0c9fee700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40a3190 con 0x7fa0c4087510 2021-02-10T11:30:44.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.513 7fa0c2ffd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa0c41c68f0 0x7fa0c41b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:44.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.513 7fa0c3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0c408c4d0 0x7fa0c41b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:44.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.513 7fa0c37fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa0c4087510 0x7fa0c40aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:44.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.513 7fa0c3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0c408c4d0 0x7fa0c41b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49382/0 (socket says 172.21.2.101:49382) 2021-02-10T11:30:44.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.513 7fa0c3fff700 1 -- 172.21.2.101:0/583164618 learned_addr learned my addr 172.21.2.101:0/583164618 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:44.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.513 7fa0c3fff700 1 -- 172.21.2.101:0/583164618 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa0c4087510 msgr2=0x7fa0c40aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:44.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.513 7fa0c3fff700 1 --2- 172.21.2.101:0/583164618 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa0c4087510 0x7fa0c40aedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:44.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.513 7fa0c3fff700 1 -- 172.21.2.101:0/583164618 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa0c41c68f0 msgr2=0x7fa0c41b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:44.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.513 7fa0c3fff700 1 --2- 172.21.2.101:0/583164618 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa0c41c68f0 0x7fa0c41b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:44.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.513 7fa0c3fff700 1 -- 172.21.2.101:0/583164618 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c40a33b0 con 0x7fa0c408c4d0 2021-02-10T11:30:44.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.513 7fa0c2ffd700 1 --2- 172.21.2.101:0/583164618 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa0c41c68f0 0x7fa0c41b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:44.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.513 7fa0c3fff700 1 --2- 172.21.2.101:0/583164618 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0c408c4d0 0x7fa0c41b18a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7fa0b4006ed0 tx=0x7fa0b400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:44.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.513 7fa0abfff700 1 -- 172.21.2.101:0/583164618 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa0b400bd50 con 0x7fa0c408c4d0 2021-02-10T11:30:44.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.513 7fa0abfff700 1 -- 172.21.2.101:0/583164618 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b4006020 con 0x7fa0c408c4d0 2021-02-10T11:30:44.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.514 7fa0c9fee700 1 -- 172.21.2.101:0/583164618 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c40a33b0 con 0x7fa0c408c4d0 2021-02-10T11:30:44.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.514 7fa0c9fee700 1 -- 172.21.2.101:0/583164618 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c40a33b0 con 0x7fa0c408c4d0 2021-02-10T11:30:44.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.514 7fa0abfff700 1 -- 172.21.2.101:0/583164618 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa0b40054b0 con 0x7fa0c408c4d0 2021-02-10T11:30:44.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.514 7fa0abfff700 1 -- 172.21.2.101:0/583164618 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa0b4006200 con 0x7fa0c408c4d0 2021-02-10T11:30:44.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.514 7fa0abfff700 1 --2- 172.21.2.101:0/583164618 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa0a40186c0 0x7fa0a401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:44.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.514 7fa0abfff700 1 -- 172.21.2.101:0/583164618 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa0b40327c0 con 0x7fa0c408c4d0 2021-02-10T11:30:44.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.515 7fa0c37fe700 1 --2- 172.21.2.101:0/583164618 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa0a40186c0 0x7fa0a401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:44.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.515 7fa0c37fe700 1 --2- 172.21.2.101:0/583164618 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa0a40186c0 0x7fa0a401abc0 crc :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:44.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.544 7fa0c9fee700 1 -- 172.21.2.101:0/583164618 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c407e6b0 con 0x7fa0c408c4d0 2021-02-10T11:30:44.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.547 7fa0abfff700 1 -- 172.21.2.101:0/583164618 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa0b4006230 con 0x7fa0c408c4d0 2021-02-10T11:30:44.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.673 7fa0c9fee700 1 -- 172.21.2.101:0/583164618 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839310", "target": ["mgr", ""]}) v1 -- 0x7fa0c407e6b0 con 0x7fa0a40186c0 2021-02-10T11:30:44.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.684 7fa0abfff700 1 -- 172.21.2.101:0/583164618 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fa0b8001070 con 0x7fa0a40186c0 2021-02-10T11:30:44.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.691 7fa0c9fee700 1 -- 172.21.2.101:0/583164618 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa0a40186c0 msgr2=0x7fa0a401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:44.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.691 7fa0c9fee700 1 --2- 172.21.2.101:0/583164618 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa0a40186c0 0x7fa0a401abc0 crc :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:44.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.691 7fa0c9fee700 1 -- 172.21.2.101:0/583164618 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0c408c4d0 msgr2=0x7fa0c41b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:44.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.691 7fa0c9fee700 1 --2- 172.21.2.101:0/583164618 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0c408c4d0 0x7fa0c41b18a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7fa0b4006ed0 tx=0x7fa0b400b960).stop 2021-02-10T11:30:44.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.691 7fa0c9fee700 1 -- 172.21.2.101:0/583164618 shutdown_connections 2021-02-10T11:30:44.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.691 7fa0c9fee700 1 --2- 172.21.2.101:0/583164618 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0c408c4d0 0x7fa0c41b18a0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:44.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.691 7fa0c9fee700 1 --2- 172.21.2.101:0/583164618 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa0c41c68f0 0x7fa0c41b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:44.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.691 7fa0c9fee700 1 --2- 172.21.2.101:0/583164618 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa0a40186c0 0x7fa0a401abc0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:44.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.691 7fa0c9fee700 1 --2- 172.21.2.101:0/583164618 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa0c4087510 0x7fa0c40aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:44.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.691 7fa0c9fee700 1 -- 172.21.2.101:0/583164618 >> 172.21.2.101:0/583164618 conn(0x7fa0c40886d0 msgr2=0x7fa0c4081c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:44.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.691 7fa0c9fee700 1 -- 172.21.2.101:0/583164618 shutdown_connections 2021-02-10T11:30:44.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:44.691 7fa0c9fee700 1 -- 172.21.2.101:0/583164618 wait complete. 2021-02-10T11:30:44.711 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839311 2021-02-10T11:30:45.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.112 7fd956060700 1 Processor -- start 2021-02-10T11:30:45.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.112 7fd956060700 1 -- start start 2021-02-10T11:30:45.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.113 7fd956060700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9501b0870 0x7fd9500f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:45.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.113 7fd956060700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd9500efe00 0x7fd9500f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:45.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.113 7fd956060700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9501b0c70 con 0x7fd9501b0470 2021-02-10T11:30:45.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.113 7fd956060700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9500f0b50 con 0x7fd9500efe00 2021-02-10T11:30:45.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.113 7fd956060700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9500d2fb0 con 0x7fd9501b0870 2021-02-10T11:30:45.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.113 7fd94f7fe700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fd9501b0470 0x7fd9500ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:43290/0 (socket says 172.21.2.101:43290) 2021-02-10T11:30:45.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.113 7fd94f7fe700 1 -- 172.21.2.101:0/4108328318 learned_addr learned my addr 172.21.2.101:0/4108328318 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:45.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.113 7fd94ffff700 1 --2- 172.21.2.101:0/4108328318 >> v2:172.21.2.101:3300/0 conn(0x7fd9500efe00 0x7fd9500f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:45.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.113 7fd94effd700 1 --2- 172.21.2.101:0/4108328318 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9501b0870 0x7fd9500f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:45.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.113 7fd94ffff700 1 -- 172.21.2.101:0/4108328318 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9501b0870 msgr2=0x7fd9500f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:45.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.113 7fd94ffff700 1 --2- 172.21.2.101:0/4108328318 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9501b0870 0x7fd9500f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:45.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.113 7fd94ffff700 1 -- 172.21.2.101:0/4108328318 >> v1:172.21.2.108:6789/0 conn(0x7fd9501b0470 legacy=0x7fd9500ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:45.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.113 7fd94ffff700 1 -- 172.21.2.101:0/4108328318 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9500f0820 con 0x7fd9500efe00 2021-02-10T11:30:45.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.113 7fd94e7fc700 1 -- 172.21.2.101:0/4108328318 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 675391469 0 0) 0x7fd948000fe0 con 0x7fd9501b0470 2021-02-10T11:30:45.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.114 7fd94ffff700 1 --2- 172.21.2.101:0/4108328318 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9500efe00 0x7fd9500f0200 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7fd940006ab0 tx=0x7fd940009b40).ready entity=mon.0 client_cookie=173391ff989e4bce server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:45.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.114 7fd94e7fc700 1 -- 172.21.2.101:0/4108328318 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd9400022e0 con 0x7fd9500efe00 2021-02-10T11:30:45.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.114 7fd94e7fc700 1 -- 172.21.2.101:0/4108328318 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd940002660 con 0x7fd9500efe00 2021-02-10T11:30:45.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.114 7fd94e7fc700 1 -- 172.21.2.101:0/4108328318 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd940002a70 con 0x7fd9500efe00 2021-02-10T11:30:45.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.114 7fd956060700 1 -- 172.21.2.101:0/4108328318 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9500efe00 msgr2=0x7fd9500f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:45.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.114 7fd956060700 1 --2- 172.21.2.101:0/4108328318 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9500efe00 0x7fd9500f0200 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7fd940006ab0 tx=0x7fd940009b40).stop 2021-02-10T11:30:45.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.114 7fd956060700 1 -- 172.21.2.101:0/4108328318 shutdown_connections 2021-02-10T11:30:45.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.114 7fd956060700 1 --2- 172.21.2.101:0/4108328318 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9501b0870 0x7fd9500f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:45.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.114 7fd956060700 1 --2- 172.21.2.101:0/4108328318 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9500efe00 0x7fd9500f0200 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:45.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.114 7fd956060700 1 -- 172.21.2.101:0/4108328318 >> 172.21.2.101:0/4108328318 conn(0x7fd9500ce3e0 msgr2=0x7fd9500e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:45.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.115 7fd956060700 1 -- 172.21.2.101:0/4108328318 shutdown_connections 2021-02-10T11:30:45.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.115 7fd956060700 1 -- 172.21.2.101:0/4108328318 wait complete. 2021-02-10T11:30:45.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.129 7fd956060700 1 Processor -- start 2021-02-10T11:30:45.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.129 7fd956060700 1 -- start start 2021-02-10T11:30:45.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.130 7fd956060700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9500a0420 0x7fd9500a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:45.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.130 7fd956060700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd950189d00 0x7fd95018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:45.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.130 7fd956060700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd95018a720 0x7fd9500b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:45.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.130 7fd956060700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd950189800 con 0x7fd95018a720 2021-02-10T11:30:45.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.130 7fd956060700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95018ab20 con 0x7fd950189d00 2021-02-10T11:30:45.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.130 7fd956060700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9500a1140 con 0x7fd9500a0420 2021-02-10T11:30:45.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.130 7fd94ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd95018a720 0x7fd9500b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:45.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.130 7fd94ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd95018a720 0x7fd9500b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49392/0 (socket says 172.21.2.101:49392) 2021-02-10T11:30:45.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.130 7fd94ffff700 1 -- 172.21.2.101:0/153092167 learned_addr learned my addr 172.21.2.101:0/153092167 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:45.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.130 7fd94effd700 1 --2- 172.21.2.101:0/153092167 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd950189d00 0x7fd95018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:45.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.130 7fd94f7fe700 1 --2- 172.21.2.101:0/153092167 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9500a0420 0x7fd9500a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:45.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.130 7fd94ffff700 1 -- 172.21.2.101:0/153092167 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9500a0420 msgr2=0x7fd9500a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:45.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.130 7fd94ffff700 1 --2- 172.21.2.101:0/153092167 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9500a0420 0x7fd9500a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:45.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.130 7fd94ffff700 1 -- 172.21.2.101:0/153092167 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd950189d00 msgr2=0x7fd95018a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:45.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.130 7fd94ffff700 1 --2- 172.21.2.101:0/153092167 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd950189d00 0x7fd95018a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:45.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.130 7fd94ffff700 1 -- 172.21.2.101:0/153092167 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9500b8200 con 0x7fd95018a720 2021-02-10T11:30:45.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.130 7fd94f7fe700 1 --2- 172.21.2.101:0/153092167 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9500a0420 0x7fd9500a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:30:45.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.130 7fd94ffff700 1 --2- 172.21.2.101:0/153092167 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd95018a720 0x7fd9500b7be0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7fd94000b5e0 tx=0x7fd94000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:45.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.131 7fd937fff700 1 -- 172.21.2.101:0/153092167 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd940009070 con 0x7fd95018a720 2021-02-10T11:30:45.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.131 7fd937fff700 1 -- 172.21.2.101:0/153092167 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd940002280 con 0x7fd95018a720 2021-02-10T11:30:45.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.131 7fd937fff700 1 -- 172.21.2.101:0/153092167 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd940002760 con 0x7fd95018a720 2021-02-10T11:30:45.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.131 7fd956060700 1 -- 172.21.2.101:0/153092167 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9500b8200 con 0x7fd95018a720 2021-02-10T11:30:45.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.131 7fd956060700 1 -- 172.21.2.101:0/153092167 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9500b8200 con 0x7fd95018a720 2021-02-10T11:30:45.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.131 7fd937fff700 1 -- 172.21.2.101:0/153092167 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd940002460 con 0x7fd95018a720 2021-02-10T11:30:45.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.131 7fd937fff700 1 --2- 172.21.2.101:0/153092167 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd9300186c0 0x7fd93001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:45.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.131 7fd937fff700 1 -- 172.21.2.101:0/153092167 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd9400325e0 con 0x7fd95018a720 2021-02-10T11:30:45.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.131 7fd94f7fe700 1 --2- 172.21.2.101:0/153092167 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd9300186c0 0x7fd93001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:45.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.132 7fd94f7fe700 1 --2- 172.21.2.101:0/153092167 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd9300186c0 0x7fd93001abc0 crc :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:45.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.163 7fd956060700 1 -- 172.21.2.101:0/153092167 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd95018ab20 con 0x7fd95018a720 2021-02-10T11:30:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.165 7fd937fff700 1 -- 172.21.2.101:0/153092167 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd940021f20 con 0x7fd95018a720 2021-02-10T11:30:45.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.294 7fd956060700 1 -- 172.21.2.101:0/153092167 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839311", "target": ["mgr", ""]}) v1 -- 0x7fd95018ab20 con 0x7fd9300186c0 2021-02-10T11:30:45.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.304 7fd937fff700 1 -- 172.21.2.101:0/153092167 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fd948007b50 con 0x7fd9300186c0 2021-02-10T11:30:45.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.311 7fd956060700 1 -- 172.21.2.101:0/153092167 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd9300186c0 msgr2=0x7fd93001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:45.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.311 7fd956060700 1 --2- 172.21.2.101:0/153092167 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd9300186c0 0x7fd93001abc0 crc :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:45.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.311 7fd956060700 1 -- 172.21.2.101:0/153092167 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd95018a720 msgr2=0x7fd9500b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:45.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.311 7fd956060700 1 --2- 172.21.2.101:0/153092167 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd95018a720 0x7fd9500b7be0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7fd94000b5e0 tx=0x7fd94000b980).stop 2021-02-10T11:30:45.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.311 7fd956060700 1 -- 172.21.2.101:0/153092167 shutdown_connections 2021-02-10T11:30:45.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.311 7fd956060700 1 --2- 172.21.2.101:0/153092167 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd95018a720 0x7fd9500b7be0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:45.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.311 7fd956060700 1 --2- 172.21.2.101:0/153092167 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd950189d00 0x7fd95018a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:45.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.311 7fd956060700 1 --2- 172.21.2.101:0/153092167 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd9300186c0 0x7fd93001abc0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:45.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.311 7fd956060700 1 --2- 172.21.2.101:0/153092167 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9500a0420 0x7fd9500a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:45.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.311 7fd956060700 1 -- 172.21.2.101:0/153092167 >> 172.21.2.101:0/153092167 conn(0x7fd950094e10 msgr2=0x7fd9500a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:45.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.311 7fd956060700 1 -- 172.21.2.101:0/153092167 shutdown_connections 2021-02-10T11:30:45.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.311 7fd956060700 1 -- 172.21.2.101:0/153092167 wait complete. 2021-02-10T11:30:45.330 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839312 2021-02-10T11:30:45.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.733 7fe5f487d700 1 Processor -- start 2021-02-10T11:30:45.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.733 7fe5f487d700 1 -- start start 2021-02-10T11:30:45.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.734 7fe5f487d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe5f01b0470 0x7fe5f01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:45.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.734 7fe5f487d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe5f00ef610 0x7fe5f00efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:45.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.734 7fe5f487d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f00f0b50 con 0x7fe5f00f0030 2021-02-10T11:30:45.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.734 7fe5f487d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f00e52e0 con 0x7fe5f00ef610 2021-02-10T11:30:45.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.734 7fe5f487d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f00e5490 con 0x7fe5f01b0470 2021-02-10T11:30:45.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.734 7fe5ede1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe5f00ef610 0x7fe5f00efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:45.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.734 7fe5ede1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe5f00ef610 0x7fe5f00efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49402/0 (socket says 172.21.2.101:49402) 2021-02-10T11:30:45.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.734 7fe5ede1b700 1 -- 172.21.2.101:0/2445503405 learned_addr learned my addr 172.21.2.101:0/2445503405 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:45.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.734 7fe5ee61c700 1 --2- 172.21.2.101:0/2445503405 >> v2:172.21.2.108:3300/0 conn(0x7fe5f01b0470 0x7fe5f01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:45.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.734 7fe5ede1b700 1 -- 172.21.2.101:0/2445503405 >> v2:172.21.2.108:3300/0 conn(0x7fe5f01b0470 msgr2=0x7fe5f01b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:45.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.734 7fe5ede1b700 1 --2- 172.21.2.101:0/2445503405 >> v2:172.21.2.108:3300/0 conn(0x7fe5f01b0470 0x7fe5f01b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:45.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.734 7fe5ede1b700 1 -- 172.21.2.101:0/2445503405 >> v1:172.21.2.108:6789/0 conn(0x7fe5f00f0030 legacy=0x7fe5f00f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:45.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.734 7fe5ede1b700 1 -- 172.21.2.101:0/2445503405 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f00f0430 con 0x7fe5f00ef610 2021-02-10T11:30:45.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.734 7fe5ee61c700 1 --2- 172.21.2.101:0/2445503405 >> v2:172.21.2.108:3300/0 conn(0x7fe5f01b0470 0x7fe5f01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:30:45.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.734 7fe5ede1b700 1 --2- 172.21.2.101:0/2445503405 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5f00ef610 0x7fe5f00efa10 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7fe5e4006700 tx=0x7fe5e4009b80).ready entity=mon.0 client_cookie=1696314362a020d5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:45.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.734 7fe5ed61a700 1 -- 172.21.2.101:0/2445503405 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5e400c0c0 con 0x7fe5f00ef610 2021-02-10T11:30:45.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.735 7fe5ed61a700 1 -- 172.21.2.101:0/2445503405 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e400c340 con 0x7fe5f00ef610 2021-02-10T11:30:45.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.735 7fe5ed61a700 1 -- 172.21.2.101:0/2445503405 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5e400c860 con 0x7fe5f00ef610 2021-02-10T11:30:45.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.735 7fe5f487d700 1 -- 172.21.2.101:0/2445503405 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5f00ef610 msgr2=0x7fe5f00efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:45.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.735 7fe5f487d700 1 --2- 172.21.2.101:0/2445503405 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5f00ef610 0x7fe5f00efa10 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7fe5e4006700 tx=0x7fe5e4009b80).stop 2021-02-10T11:30:45.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.735 7fe5f487d700 1 -- 172.21.2.101:0/2445503405 shutdown_connections 2021-02-10T11:30:45.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.735 7fe5f487d700 1 --2- 172.21.2.101:0/2445503405 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5f00ef610 0x7fe5f00efa10 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:45.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.735 7fe5f487d700 1 --2- 172.21.2.101:0/2445503405 >> v2:172.21.2.108:3300/0 conn(0x7fe5f01b0470 0x7fe5f01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:45.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.735 7fe5f487d700 1 -- 172.21.2.101:0/2445503405 >> 172.21.2.101:0/2445503405 conn(0x7fe5f00ce3e0 msgr2=0x7fe5f00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:45.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.735 7fe5f487d700 1 -- 172.21.2.101:0/2445503405 shutdown_connections 2021-02-10T11:30:45.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.735 7fe5f487d700 1 -- 172.21.2.101:0/2445503405 wait complete. 2021-02-10T11:30:45.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.750 7fe5f487d700 1 Processor -- start 2021-02-10T11:30:45.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.750 7fe5f487d700 1 -- start start 2021-02-10T11:30:45.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.750 7fe5f487d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5f00a0420 0x7fe5f00a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:45.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.750 7fe5f487d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5f0189d00 0x7fe5f018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:45.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.751 7fe5f487d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5f018a720 0x7fe5f00b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:45.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.751 7fe5f487d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f0189780 con 0x7fe5f0189d00 2021-02-10T11:30:45.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.751 7fe5f487d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f018ab20 con 0x7fe5f018a720 2021-02-10T11:30:45.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.751 7fe5f487d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f00a1140 con 0x7fe5f00a0420 2021-02-10T11:30:45.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.751 7fe5ede1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5f0189d00 0x7fe5f018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:45.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.751 7fe5ede1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5f0189d00 0x7fe5f018a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49408/0 (socket says 172.21.2.101:49408) 2021-02-10T11:30:45.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.751 7fe5ede1b700 1 -- 172.21.2.101:0/2141599266 learned_addr learned my addr 172.21.2.101:0/2141599266 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:45.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.751 7fe5ee61c700 1 --2- 172.21.2.101:0/2141599266 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5f00a0420 0x7fe5f00a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:45.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.751 7fe5eee1d700 1 --2- 172.21.2.101:0/2141599266 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5f018a720 0x7fe5f00b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:45.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.751 7fe5ede1b700 1 -- 172.21.2.101:0/2141599266 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5f00a0420 msgr2=0x7fe5f00a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:45.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.751 7fe5ede1b700 1 --2- 172.21.2.101:0/2141599266 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5f00a0420 0x7fe5f00a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:45.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.751 7fe5ede1b700 1 -- 172.21.2.101:0/2141599266 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5f018a720 msgr2=0x7fe5f00b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:45.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.751 7fe5ede1b700 1 --2- 172.21.2.101:0/2141599266 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5f018a720 0x7fe5f00b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:45.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.751 7fe5ede1b700 1 -- 172.21.2.101:0/2141599266 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f00b8200 con 0x7fe5f0189d00 2021-02-10T11:30:45.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.751 7fe5ede1b700 1 --2- 172.21.2.101:0/2141599266 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5f0189d00 0x7fe5f018a100 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7fe5e4009760 tx=0x7fe5e400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:45.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.751 7fe5deffd700 1 -- 172.21.2.101:0/2141599266 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5e400bd80 con 0x7fe5f0189d00 2021-02-10T11:30:45.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.751 7fe5deffd700 1 -- 172.21.2.101:0/2141599266 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e4006030 con 0x7fe5f0189d00 2021-02-10T11:30:45.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.752 7fe5f487d700 1 -- 172.21.2.101:0/2141599266 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f00b8200 con 0x7fe5f0189d00 2021-02-10T11:30:45.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.752 7fe5deffd700 1 -- 172.21.2.101:0/2141599266 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5e4005560 con 0x7fe5f0189d00 2021-02-10T11:30:45.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.752 7fe5f487d700 1 -- 172.21.2.101:0/2141599266 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f00b8200 con 0x7fe5f0189d00 2021-02-10T11:30:45.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.752 7fe5deffd700 1 -- 172.21.2.101:0/2141599266 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe5e40052e0 con 0x7fe5f0189d00 2021-02-10T11:30:45.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.752 7fe5deffd700 1 --2- 172.21.2.101:0/2141599266 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5d4018730 0x7fe5d401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:45.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.752 7fe5deffd700 1 -- 172.21.2.101:0/2141599266 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe5e4032670 con 0x7fe5f0189d00 2021-02-10T11:30:45.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.752 7fe5ee61c700 1 --2- 172.21.2.101:0/2141599266 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5d4018730 0x7fe5d401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:45.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.753 7fe5ee61c700 1 --2- 172.21.2.101:0/2141599266 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5d4018730 0x7fe5d401ac10 crc :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:45.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.783 7fe5f487d700 1 -- 172.21.2.101:0/2141599266 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f0105ae0 con 0x7fe5f0189d00 2021-02-10T11:30:45.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.786 7fe5deffd700 1 -- 172.21.2.101:0/2141599266 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe5e4006030 con 0x7fe5f0189d00 2021-02-10T11:30:45.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.910 7fe5f487d700 1 -- 172.21.2.101:0/2141599266 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839312", "target": ["mgr", ""]}) v1 -- 0x7fe5f0189780 con 0x7fe5d4018730 2021-02-10T11:30:45.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.921 7fe5deffd700 1 -- 172.21.2.101:0/2141599266 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fe5e8000d30 con 0x7fe5d4018730 2021-02-10T11:30:45.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.926 7fe5f487d700 1 -- 172.21.2.101:0/2141599266 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5d4018730 msgr2=0x7fe5d401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:45.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.926 7fe5f487d700 1 --2- 172.21.2.101:0/2141599266 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5d4018730 0x7fe5d401ac10 crc :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:45.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.926 7fe5f487d700 1 -- 172.21.2.101:0/2141599266 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5f0189d00 msgr2=0x7fe5f018a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:45.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.926 7fe5f487d700 1 --2- 172.21.2.101:0/2141599266 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5f0189d00 0x7fe5f018a100 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7fe5e4009760 tx=0x7fe5e400b990).stop 2021-02-10T11:30:45.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.926 7fe5f487d700 1 -- 172.21.2.101:0/2141599266 shutdown_connections 2021-02-10T11:30:45.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.926 7fe5f487d700 1 --2- 172.21.2.101:0/2141599266 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5f018a720 0x7fe5f00b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:45.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.926 7fe5f487d700 1 --2- 172.21.2.101:0/2141599266 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5f0189d00 0x7fe5f018a100 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:45.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.926 7fe5f487d700 1 --2- 172.21.2.101:0/2141599266 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5d4018730 0x7fe5d401ac10 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:45.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.926 7fe5f487d700 1 --2- 172.21.2.101:0/2141599266 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5f00a0420 0x7fe5f00a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:45.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.926 7fe5f487d700 1 -- 172.21.2.101:0/2141599266 >> 172.21.2.101:0/2141599266 conn(0x7fe5f0094e10 msgr2=0x7fe5f00a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:45.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.926 7fe5f487d700 1 -- 172.21.2.101:0/2141599266 shutdown_connections 2021-02-10T11:30:45.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:45.926 7fe5f487d700 1 -- 172.21.2.101:0/2141599266 wait complete. 2021-02-10T11:30:45.946 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839313 2021-02-10T11:30:46.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.352 7f8b882ff700 1 Processor -- start 2021-02-10T11:30:46.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.352 7f8b882ff700 1 -- start start 2021-02-10T11:30:46.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.353 7f8b882ff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8b800f05f0 0x7f8b800f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:46.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.353 7f8b882ff700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b801b0c70 con 0x7f8b801b0470 2021-02-10T11:30:46.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.353 7f8b882ff700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b800f0b50 con 0x7f8b801b0870 2021-02-10T11:30:46.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.353 7f8b882ff700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b800e5ea0 con 0x7f8b800f05f0 2021-02-10T11:30:46.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.353 7f8b8611b700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8b801b0470 0x7f8b800ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49908/0 (socket says 172.21.2.101:49908) 2021-02-10T11:30:46.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.353 7f8b8611b700 1 -- 172.21.2.101:0/3585352653 learned_addr learned my addr 172.21.2.101:0/3585352653 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:46.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.353 7f8b8691c700 1 --2- 172.21.2.101:0/3585352653 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8b800f05f0 0x7f8b800f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:46.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.353 7f8b8691c700 1 -- 172.21.2.101:0/3585352653 >> v1:172.21.2.108:6789/0 conn(0x7f8b801b0870 legacy=0x7f8b800efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:46.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.353 7f8b8691c700 1 -- 172.21.2.101:0/3585352653 >> v1:172.21.2.101:6789/0 conn(0x7f8b801b0470 legacy=0x7f8b800ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:46.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.353 7f8b8691c700 1 -- 172.21.2.101:0/3585352653 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b800e52e0 con 0x7f8b800f05f0 2021-02-10T11:30:46.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.353 7f8b85119700 1 -- 172.21.2.101:0/3585352653 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419571775 0 0) 0x7f8b7c0010b0 con 0x7f8b801b0470 2021-02-10T11:30:46.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.354 7f8b8691c700 1 --2- 172.21.2.101:0/3585352653 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8b800f05f0 0x7f8b800f20e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f8b78006e50 tx=0x7f8b78009df0).ready entity=mon.2 client_cookie=875565d4d1cd8349 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:46.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.354 7f8b85119700 1 -- 172.21.2.101:0/3585352653 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8b7800b510 con 0x7f8b800f05f0 2021-02-10T11:30:46.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.354 7f8b85119700 1 -- 172.21.2.101:0/3585352653 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b78006790 con 0x7f8b800f05f0 2021-02-10T11:30:46.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.354 7f8b85119700 1 -- 172.21.2.101:0/3585352653 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8b7800b750 con 0x7f8b800f05f0 2021-02-10T11:30:46.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.354 7f8b882ff700 1 -- 172.21.2.101:0/3585352653 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8b800f05f0 msgr2=0x7f8b800f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:46.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.354 7f8b882ff700 1 --2- 172.21.2.101:0/3585352653 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8b800f05f0 0x7f8b800f20e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f8b78006e50 tx=0x7f8b78009df0).stop 2021-02-10T11:30:46.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.354 7f8b882ff700 1 -- 172.21.2.101:0/3585352653 shutdown_connections 2021-02-10T11:30:46.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.354 7f8b882ff700 1 --2- 172.21.2.101:0/3585352653 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8b800f05f0 0x7f8b800f20e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:46.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.354 7f8b882ff700 1 -- 172.21.2.101:0/3585352653 >> 172.21.2.101:0/3585352653 conn(0x7f8b800ce3e0 msgr2=0x7f8b800e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:46.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.355 7f8b882ff700 1 -- 172.21.2.101:0/3585352653 shutdown_connections 2021-02-10T11:30:46.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.355 7f8b882ff700 1 -- 172.21.2.101:0/3585352653 wait complete. 2021-02-10T11:30:46.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.369 7f8b882ff700 1 Processor -- start 2021-02-10T11:30:46.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.369 7f8b882ff700 1 -- start start 2021-02-10T11:30:46.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.370 7f8b882ff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8b801a0570 0x7f8b800a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:46.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.370 7f8b882ff700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8b800f1fd0 0x7f8b800f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:46.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.370 7f8b882ff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8b800a5450 0x7f8b800a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:46.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.370 7f8b882ff700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b800a4bb0 con 0x7f8b800a5450 2021-02-10T11:30:46.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.370 7f8b882ff700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b800b35e0 con 0x7f8b800f1fd0 2021-02-10T11:30:46.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.370 7f8b882ff700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b800a1f00 con 0x7f8b801a0570 2021-02-10T11:30:46.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.370 7f8b8691c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8b800a5450 0x7f8b800a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:46.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.370 7f8b8691c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8b800a5450 0x7f8b800a5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49420/0 (socket says 172.21.2.101:49420) 2021-02-10T11:30:46.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.370 7f8b8691c700 1 -- 172.21.2.101:0/41339570 learned_addr learned my addr 172.21.2.101:0/41339570 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:46.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.370 7f8b8691c700 1 -- 172.21.2.101:0/41339570 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8b801a0570 msgr2=0x7f8b800a4590 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:30:46.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.370 7f8b8691c700 1 --2- 172.21.2.101:0/41339570 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8b801a0570 0x7f8b800a4590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:46.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.370 7f8b8691c700 1 -- 172.21.2.101:0/41339570 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8b800f1fd0 msgr2=0x7f8b800f23d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:30:46.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.370 7f8b8691c700 1 --2- 172.21.2.101:0/41339570 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8b800f1fd0 0x7f8b800f23d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:46.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.370 7f8b8691c700 1 -- 172.21.2.101:0/41339570 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b800a5e70 con 0x7f8b800a5450 2021-02-10T11:30:46.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.370 7f8b8691c700 1 --2- 172.21.2.101:0/41339570 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8b800a5450 0x7f8b800a5850 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0x7f8b7800b970 tx=0x7f8b7800bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:46.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.370 7f8b72ffd700 1 -- 172.21.2.101:0/41339570 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8b780097b0 con 0x7f8b800a5450 2021-02-10T11:30:46.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.370 7f8b72ffd700 1 -- 172.21.2.101:0/41339570 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b78009a10 con 0x7f8b800a5450 2021-02-10T11:30:46.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.371 7f8b882ff700 1 -- 172.21.2.101:0/41339570 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b800a5e70 con 0x7f8b800a5450 2021-02-10T11:30:46.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.371 7f8b882ff700 1 -- 172.21.2.101:0/41339570 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b800a5e70 con 0x7f8b800a5450 2021-02-10T11:30:46.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.371 7f8b72ffd700 1 -- 172.21.2.101:0/41339570 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8b780084d0 con 0x7f8b800a5450 2021-02-10T11:30:46.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.371 7f8b72ffd700 1 -- 172.21.2.101:0/41339570 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8b780082e0 con 0x7f8b800a5450 2021-02-10T11:30:46.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.371 7f8b72ffd700 1 --2- 172.21.2.101:0/41339570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8b68018730 0x7f8b6801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:46.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.371 7f8b72ffd700 1 -- 172.21.2.101:0/41339570 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8b78032b90 con 0x7f8b800a5450 2021-02-10T11:30:46.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.371 7f8b8611b700 1 --2- 172.21.2.101:0/41339570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8b68018730 0x7f8b6801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:46.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.371 7f8b8611b700 1 --2- 172.21.2.101:0/41339570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8b68018730 0x7f8b6801ac10 crc :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:46.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.402 7f8b882ff700 1 -- 172.21.2.101:0/41339570 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b800a4bb0 con 0x7f8b800a5450 2021-02-10T11:30:46.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.404 7f8b72ffd700 1 -- 172.21.2.101:0/41339570 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8b78032da0 con 0x7f8b800a5450 2021-02-10T11:30:46.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.534 7f8b882ff700 1 -- 172.21.2.101:0/41339570 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839313", "target": ["mgr", ""]}) v1 -- 0x7f8b800a4bb0 con 0x7f8b68018730 2021-02-10T11:30:46.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.544 7f8b72ffd700 1 -- 172.21.2.101:0/41339570 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f8b7c000e20 con 0x7f8b68018730 2021-02-10T11:30:46.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.551 7f8b882ff700 1 -- 172.21.2.101:0/41339570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8b68018730 msgr2=0x7f8b6801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:46.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.551 7f8b882ff700 1 --2- 172.21.2.101:0/41339570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8b68018730 0x7f8b6801ac10 crc :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:46.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.551 7f8b882ff700 1 -- 172.21.2.101:0/41339570 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8b800a5450 msgr2=0x7f8b800a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:46.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.551 7f8b882ff700 1 --2- 172.21.2.101:0/41339570 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8b800a5450 0x7f8b800a5850 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0x7f8b7800b970 tx=0x7f8b7800bd10).stop 2021-02-10T11:30:46.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.551 7f8b882ff700 1 -- 172.21.2.101:0/41339570 shutdown_connections 2021-02-10T11:30:46.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.551 7f8b882ff700 1 --2- 172.21.2.101:0/41339570 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8b800a5450 0x7f8b800a5850 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:46.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.551 7f8b882ff700 1 --2- 172.21.2.101:0/41339570 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8b800f1fd0 0x7f8b800f23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:46.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.551 7f8b882ff700 1 --2- 172.21.2.101:0/41339570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8b68018730 0x7f8b6801ac10 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:46.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.551 7f8b882ff700 1 --2- 172.21.2.101:0/41339570 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8b801a0570 0x7f8b800a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:46.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.551 7f8b882ff700 1 -- 172.21.2.101:0/41339570 >> 172.21.2.101:0/41339570 conn(0x7f8b800958b0 msgr2=0x7f8b800bf040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:46.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.551 7f8b882ff700 1 -- 172.21.2.101:0/41339570 shutdown_connections 2021-02-10T11:30:46.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.551 7f8b882ff700 1 -- 172.21.2.101:0/41339570 wait complete. 2021-02-10T11:30:46.570 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839314 2021-02-10T11:30:47.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.974 7fe588d87700 1 Processor -- start 2021-02-10T11:30:47.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.974 7fe588d87700 1 -- start start 2021-02-10T11:30:47.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.975 7fe588d87700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe5841b0470 0x7fe5841b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:47.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.975 7fe588d87700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5840f20e0 0x7fe5840f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:47.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.975 7fe588d87700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5840f0b50 con 0x7fe5840ef610 2021-02-10T11:30:47.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.975 7fe588d87700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5840f0820 con 0x7fe5841b0470 2021-02-10T11:30:47.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.975 7fe588d87700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840f24e0 con 0x7fe5840f20e0 2021-02-10T11:30:47.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.975 7fe583314700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5840f20e0 0x7fe5840f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:47.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.975 7fe582312700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fe5840ef610 0x7fe5840efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:43334/0 (socket says 172.21.2.101:43334) 2021-02-10T11:30:47.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.975 7fe582b13700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe5841b0470 0x7fe5841b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:47.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.975 7fe582312700 1 -- 172.21.2.101:0/1262223308 learned_addr learned my addr 172.21.2.101:0/1262223308 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:47.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.976 7fe583314700 1 -- 172.21.2.101:0/1262223308 >> v2:172.21.2.108:3300/0 conn(0x7fe5841b0470 msgr2=0x7fe5841b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:47.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.976 7fe583314700 1 --2- 172.21.2.101:0/1262223308 >> v2:172.21.2.108:3300/0 conn(0x7fe5841b0470 0x7fe5841b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:47.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.976 7fe583314700 1 -- 172.21.2.101:0/1262223308 >> v1:172.21.2.108:6789/0 conn(0x7fe5840ef610 legacy=0x7fe5840efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:47.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.976 7fe583314700 1 -- 172.21.2.101:0/1262223308 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5840e52e0 con 0x7fe5840f20e0 2021-02-10T11:30:47.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.976 7fe583314700 1 --2- 172.21.2.101:0/1262223308 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5840f20e0 0x7fe5840f0200 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fe574006ad0 tx=0x7fe574009ad0).ready entity=mon.2 client_cookie=302d3e5913fc4571 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:47.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.976 7fe581b11700 1 -- 172.21.2.101:0/1262223308 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5740022e0 con 0x7fe5840f20e0 2021-02-10T11:30:47.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.976 7fe581b11700 1 -- 172.21.2.101:0/1262223308 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe574006410 con 0x7fe5840f20e0 2021-02-10T11:30:47.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.976 7fe581b11700 1 -- 172.21.2.101:0/1262223308 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe574002620 con 0x7fe5840f20e0 2021-02-10T11:30:47.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.976 7fe588d87700 1 -- 172.21.2.101:0/1262223308 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5840f20e0 msgr2=0x7fe5840f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:47.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.976 7fe588d87700 1 --2- 172.21.2.101:0/1262223308 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5840f20e0 0x7fe5840f0200 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fe574006ad0 tx=0x7fe574009ad0).stop 2021-02-10T11:30:47.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.977 7fe588d87700 1 -- 172.21.2.101:0/1262223308 shutdown_connections 2021-02-10T11:30:47.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.977 7fe588d87700 1 --2- 172.21.2.101:0/1262223308 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5840f20e0 0x7fe5840f0200 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:47.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.977 7fe588d87700 1 --2- 172.21.2.101:0/1262223308 >> v2:172.21.2.108:3300/0 conn(0x7fe5841b0470 0x7fe5841b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:47.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.977 7fe588d87700 1 -- 172.21.2.101:0/1262223308 >> 172.21.2.101:0/1262223308 conn(0x7fe5840ce3e0 msgr2=0x7fe5840e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:47.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.977 7fe588d87700 1 -- 172.21.2.101:0/1262223308 shutdown_connections 2021-02-10T11:30:47.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.977 7fe588d87700 1 -- 172.21.2.101:0/1262223308 wait complete. 2021-02-10T11:30:47.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.991 7fe588d87700 1 Processor -- start 2021-02-10T11:30:47.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.992 7fe588d87700 1 -- start start 2021-02-10T11:30:47.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.992 7fe588d87700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe584105d10 0x7fe58409eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:47.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.992 7fe588d87700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe584106110 0x7fe58409f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:47.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.992 7fe588d87700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5840b2a00 0x7fe5840b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:47.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.992 7fe588d87700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58409e560 con 0x7fe584105d10 2021-02-10T11:30:47.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.992 7fe588d87700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58409c090 con 0x7fe584106110 2021-02-10T11:30:47.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.992 7fe588d87700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840b3420 con 0x7fe5840b2a00 2021-02-10T11:30:47.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.992 7fe582b13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe584105d10 0x7fe58409eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:47.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.992 7fe582b13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe584105d10 0x7fe58409eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49434/0 (socket says 172.21.2.101:49434) 2021-02-10T11:30:47.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.992 7fe582b13700 1 -- 172.21.2.101:0/3747243331 learned_addr learned my addr 172.21.2.101:0/3747243331 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:47.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.992 7fe583314700 1 --2- 172.21.2.101:0/3747243331 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5840b2a00 0x7fe5840b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:47.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.992 7fe582312700 1 --2- 172.21.2.101:0/3747243331 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe584106110 0x7fe58409f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.993 7fe582b13700 1 -- 172.21.2.101:0/3747243331 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5840b2a00 msgr2=0x7fe5840b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.993 7fe582b13700 1 --2- 172.21.2.101:0/3747243331 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5840b2a00 0x7fe5840b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.993 7fe582b13700 1 -- 172.21.2.101:0/3747243331 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe584106110 msgr2=0x7fe58409f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.993 7fe582b13700 1 --2- 172.21.2.101:0/3747243331 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe584106110 0x7fe58409f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.993 7fe582b13700 1 -- 172.21.2.101:0/3747243331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5840b3640 con 0x7fe584105d10 2021-02-10T11:30:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.993 7fe582312700 1 --2- 172.21.2.101:0/3747243331 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe584106110 0x7fe58409f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:30:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.993 7fe582b13700 1 --2- 172.21.2.101:0/3747243331 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe584105d10 0x7fe58409eae0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7fe57c0099e0 tx=0x7fe57c00bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.993 7fe5737fe700 1 -- 172.21.2.101:0/3747243331 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe57c00c2f0 con 0x7fe584105d10 2021-02-10T11:30:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.993 7fe5737fe700 1 -- 172.21.2.101:0/3747243331 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c00c5f0 con 0x7fe584105d10 2021-02-10T11:30:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.993 7fe588d87700 1 -- 172.21.2.101:0/3747243331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5840b3640 con 0x7fe584105d10 2021-02-10T11:30:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.993 7fe5737fe700 1 -- 172.21.2.101:0/3747243331 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe57c00ca50 con 0x7fe584105d10 2021-02-10T11:30:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.993 7fe588d87700 1 -- 172.21.2.101:0/3747243331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5840b3bd0 con 0x7fe584105d10 2021-02-10T11:30:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.994 7fe5737fe700 1 -- 172.21.2.101:0/3747243331 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe57c00c2f0 con 0x7fe584105d10 2021-02-10T11:30:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.994 7fe5737fe700 1 --2- 172.21.2.101:0/3747243331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5680186d0 0x7fe56801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.994 7fe5737fe700 1 -- 172.21.2.101:0/3747243331 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe57c0329b0 con 0x7fe584105d10 2021-02-10T11:30:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.994 7fe582312700 1 --2- 172.21.2.101:0/3747243331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5680186d0 0x7fe56801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:46.994 7fe582312700 1 --2- 172.21.2.101:0/3747243331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5680186d0 0x7fe56801abd0 crc :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:47.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.024 7fe588d87700 1 -- 172.21.2.101:0/3747243331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe584105ae0 con 0x7fe584105d10 2021-02-10T11:30:47.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.026 7fe5737fe700 1 -- 172.21.2.101:0/3747243331 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe57c000cc0 con 0x7fe584105d10 2021-02-10T11:30:47.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.152 7fe588d87700 1 -- 172.21.2.101:0/3747243331 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839314", "target": ["mgr", ""]}) v1 -- 0x7fe5841a16c0 con 0x7fe5680186d0 2021-02-10T11:30:47.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.162 7fe5737fe700 1 -- 172.21.2.101:0/3747243331 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fe578007930 con 0x7fe5680186d0 2021-02-10T11:30:47.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.169 7fe588d87700 1 -- 172.21.2.101:0/3747243331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5680186d0 msgr2=0x7fe56801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:47.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.169 7fe588d87700 1 --2- 172.21.2.101:0/3747243331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5680186d0 0x7fe56801abd0 crc :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:47.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.169 7fe588d87700 1 -- 172.21.2.101:0/3747243331 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe584105d10 msgr2=0x7fe58409eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:47.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.169 7fe588d87700 1 --2- 172.21.2.101:0/3747243331 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe584105d10 0x7fe58409eae0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7fe57c0099e0 tx=0x7fe57c00bd40).stop 2021-02-10T11:30:47.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.169 7fe588d87700 1 -- 172.21.2.101:0/3747243331 shutdown_connections 2021-02-10T11:30:47.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.169 7fe588d87700 1 --2- 172.21.2.101:0/3747243331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5680186d0 0x7fe56801abd0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:47.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.169 7fe588d87700 1 --2- 172.21.2.101:0/3747243331 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5840b2a00 0x7fe5840b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:47.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.169 7fe588d87700 1 --2- 172.21.2.101:0/3747243331 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe584106110 0x7fe58409f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:47.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.169 7fe588d87700 1 --2- 172.21.2.101:0/3747243331 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe584105d10 0x7fe58409eae0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:47.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.169 7fe588d87700 1 -- 172.21.2.101:0/3747243331 >> 172.21.2.101:0/3747243331 conn(0x7fe5840c36f0 msgr2=0x7fe5840bd640 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:47.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.169 7fe588d87700 1 -- 172.21.2.101:0/3747243331 shutdown_connections 2021-02-10T11:30:47.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.169 7fe588d87700 1 -- 172.21.2.101:0/3747243331 wait complete. 2021-02-10T11:30:47.189 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839315 2021-02-10T11:30:47.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.591 7fe980308700 1 Processor -- start 2021-02-10T11:30:47.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.591 7fe980308700 1 -- start start 2021-02-10T11:30:47.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.591 7fe980308700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe9781b0470 0x7fe9781b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:47.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.591 7fe980308700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9780f0b50 con 0x7fe9780ef610 2021-02-10T11:30:47.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.591 7fe980308700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9780f24e0 con 0x7fe9780f20e0 2021-02-10T11:30:47.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.591 7fe980308700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9780e5ea0 con 0x7fe9781b0470 2021-02-10T11:30:47.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.592 7fe97e124700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe9781b0470 0x7fe9781b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:47.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.592 7fe97d923700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe9780ef610 0x7fe9780efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49936/0 (socket says 172.21.2.101:49936) 2021-02-10T11:30:47.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.592 7fe97d923700 1 -- 172.21.2.101:0/3083001556 learned_addr learned my addr 172.21.2.101:0/3083001556 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:47.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.592 7fe97e124700 1 -- 172.21.2.101:0/3083001556 >> v1:172.21.2.108:6789/0 conn(0x7fe9780f20e0 legacy=0x7fe9780f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:47.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.592 7fe97e124700 1 -- 172.21.2.101:0/3083001556 >> v1:172.21.2.101:6789/0 conn(0x7fe9780ef610 legacy=0x7fe9780efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:47.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.592 7fe97e124700 1 -- 172.21.2.101:0/3083001556 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9780e52e0 con 0x7fe9781b0470 2021-02-10T11:30:47.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.592 7fe97e124700 1 --2- 172.21.2.101:0/3083001556 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9781b0470 0x7fe9781b0870 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7fe96c006650 tx=0x7fe96c009a80).ready entity=mon.0 client_cookie=f160472c0a43e072 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:47.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.592 7fe97d122700 1 -- 172.21.2.101:0/3083001556 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe96c00c0c0 con 0x7fe9781b0470 2021-02-10T11:30:47.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.592 7fe97d122700 1 -- 172.21.2.101:0/3083001556 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96c00c340 con 0x7fe9781b0470 2021-02-10T11:30:47.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.592 7fe97d122700 1 -- 172.21.2.101:0/3083001556 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe96c00c820 con 0x7fe9781b0470 2021-02-10T11:30:47.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.592 7fe980308700 1 -- 172.21.2.101:0/3083001556 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9781b0470 msgr2=0x7fe9781b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:47.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.592 7fe980308700 1 --2- 172.21.2.101:0/3083001556 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9781b0470 0x7fe9781b0870 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7fe96c006650 tx=0x7fe96c009a80).stop 2021-02-10T11:30:47.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.593 7fe980308700 1 -- 172.21.2.101:0/3083001556 shutdown_connections 2021-02-10T11:30:47.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.593 7fe980308700 1 --2- 172.21.2.101:0/3083001556 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9781b0470 0x7fe9781b0870 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:47.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.593 7fe980308700 1 -- 172.21.2.101:0/3083001556 >> 172.21.2.101:0/3083001556 conn(0x7fe9780ce3e0 msgr2=0x7fe9780e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:47.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.593 7fe980308700 1 -- 172.21.2.101:0/3083001556 shutdown_connections 2021-02-10T11:30:47.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.593 7fe980308700 1 -- 172.21.2.101:0/3083001556 wait complete. 2021-02-10T11:30:47.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.608 7fe980308700 1 Processor -- start 2021-02-10T11:30:47.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.608 7fe980308700 1 -- start start 2021-02-10T11:30:47.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.608 7fe980308700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe978087510 0x7fe9780aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:47.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.608 7fe980308700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9781c68f0 0x7fe9781b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:47.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.608 7fe980308700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe97808c4d0 0x7fe9781b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:47.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.608 7fe980308700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9780a2d50 con 0x7fe978087510 2021-02-10T11:30:47.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.608 7fe980308700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9780a2f70 con 0x7fe97808c4d0 2021-02-10T11:30:47.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.608 7fe980308700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9780a3190 con 0x7fe9781c68f0 2021-02-10T11:30:47.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.608 7fe97e124700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe978087510 0x7fe9780aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:47.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.608 7fe97e124700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe978087510 0x7fe9780aedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49448/0 (socket says 172.21.2.101:49448) 2021-02-10T11:30:47.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.608 7fe97e124700 1 -- 172.21.2.101:0/3827573524 learned_addr learned my addr 172.21.2.101:0/3827573524 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:47.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.609 7fe97d923700 1 --2- 172.21.2.101:0/3827573524 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9781c68f0 0x7fe9781b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:47.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.609 7fe97e925700 1 --2- 172.21.2.101:0/3827573524 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe97808c4d0 0x7fe9781b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:47.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.609 7fe97e925700 1 -- 172.21.2.101:0/3827573524 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9781c68f0 msgr2=0x7fe9781b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:47.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.609 7fe97e925700 1 --2- 172.21.2.101:0/3827573524 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9781c68f0 0x7fe9781b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:47.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.609 7fe97e925700 1 -- 172.21.2.101:0/3827573524 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe978087510 msgr2=0x7fe9780aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:47.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.609 7fe97e925700 1 --2- 172.21.2.101:0/3827573524 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe978087510 0x7fe9780aedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:47.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.609 7fe97e925700 1 -- 172.21.2.101:0/3827573524 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9780a33b0 con 0x7fe97808c4d0 2021-02-10T11:30:47.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.609 7fe97e124700 1 --2- 172.21.2.101:0/3827573524 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe978087510 0x7fe9780aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:47.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.609 7fe97e925700 1 --2- 172.21.2.101:0/3827573524 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe97808c4d0 0x7fe9781b18a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fe974006e80 tx=0x7fe97400b8b0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:47.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.609 7fe96affd700 1 -- 172.21.2.101:0/3827573524 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe974000960 con 0x7fe97808c4d0 2021-02-10T11:30:47.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.609 7fe96affd700 1 -- 172.21.2.101:0/3827573524 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe97400bca0 con 0x7fe97808c4d0 2021-02-10T11:30:47.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.609 7fe980308700 1 -- 172.21.2.101:0/3827573524 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9780a33b0 con 0x7fe97808c4d0 2021-02-10T11:30:47.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.609 7fe980308700 1 -- 172.21.2.101:0/3827573524 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9780a33b0 con 0x7fe97808c4d0 2021-02-10T11:30:47.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.609 7fe96affd700 1 -- 172.21.2.101:0/3827573524 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe974002080 con 0x7fe97808c4d0 2021-02-10T11:30:47.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.610 7fe96affd700 1 -- 172.21.2.101:0/3827573524 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe974000c90 con 0x7fe97808c4d0 2021-02-10T11:30:47.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.610 7fe96affd700 1 --2- 172.21.2.101:0/3827573524 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe960018730 0x7fe96001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:47.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.610 7fe96affd700 1 -- 172.21.2.101:0/3827573524 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe974032580 con 0x7fe97808c4d0 2021-02-10T11:30:47.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.610 7fe97e124700 1 --2- 172.21.2.101:0/3827573524 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe960018730 0x7fe96001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:47.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.610 7fe97e124700 1 --2- 172.21.2.101:0/3827573524 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe960018730 0x7fe96001ac10 crc :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:47.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.641 7fe980308700 1 -- 172.21.2.101:0/3827573524 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe97807e6b0 con 0x7fe97808c4d0 2021-02-10T11:30:47.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.645 7fe96affd700 1 -- 172.21.2.101:0/3827573524 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe974006b50 con 0x7fe97808c4d0 2021-02-10T11:30:47.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.773 7fe980308700 1 -- 172.21.2.101:0/3827573524 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839315", "target": ["mgr", ""]}) v1 -- 0x7fe9780a2d50 con 0x7fe960018730 2021-02-10T11:30:47.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.783 7fe96affd700 1 -- 172.21.2.101:0/3827573524 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fe96c000960 con 0x7fe960018730 2021-02-10T11:30:47.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.790 7fe980308700 1 -- 172.21.2.101:0/3827573524 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe960018730 msgr2=0x7fe96001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:47.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.790 7fe980308700 1 --2- 172.21.2.101:0/3827573524 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe960018730 0x7fe96001ac10 crc :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:47.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.790 7fe980308700 1 -- 172.21.2.101:0/3827573524 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe97808c4d0 msgr2=0x7fe9781b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:47.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.790 7fe980308700 1 --2- 172.21.2.101:0/3827573524 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe97808c4d0 0x7fe9781b18a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fe974006e80 tx=0x7fe97400b8b0).stop 2021-02-10T11:30:47.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.790 7fe980308700 1 -- 172.21.2.101:0/3827573524 shutdown_connections 2021-02-10T11:30:47.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.790 7fe980308700 1 --2- 172.21.2.101:0/3827573524 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe960018730 0x7fe96001ac10 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:47.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.790 7fe980308700 1 --2- 172.21.2.101:0/3827573524 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9781c68f0 0x7fe9781b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:47.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.790 7fe980308700 1 --2- 172.21.2.101:0/3827573524 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe97808c4d0 0x7fe9781b18a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:47.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.790 7fe980308700 1 --2- 172.21.2.101:0/3827573524 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe978087510 0x7fe9780aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:47.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.790 7fe980308700 1 -- 172.21.2.101:0/3827573524 >> 172.21.2.101:0/3827573524 conn(0x7fe9780886d0 msgr2=0x7fe978081c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:47.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.790 7fe980308700 1 -- 172.21.2.101:0/3827573524 shutdown_connections 2021-02-10T11:30:47.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:47.790 7fe980308700 1 -- 172.21.2.101:0/3827573524 wait complete. 2021-02-10T11:30:47.810 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839316 2021-02-10T11:30:48.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.217 7f03a3a10700 1 Processor -- start 2021-02-10T11:30:48.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.217 7f03a3a10700 1 -- start start 2021-02-10T11:30:48.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.218 7f03a3a10700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f039c1b0870 0x7f039c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:48.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.218 7f03a3a10700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f039c0efe00 0x7f039c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:48.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.218 7f03a3a10700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f039c1b0c70 con 0x7f039c1b0470 2021-02-10T11:30:48.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.218 7f03a3a10700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f039c0f0b50 con 0x7f039c0efe00 2021-02-10T11:30:48.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.218 7f03a3a10700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c0f0820 con 0x7f039c1b0870 2021-02-10T11:30:48.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.218 7f03a202d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f039c0efe00 0x7f039c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:48.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.218 7f03a202d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f039c0efe00 0x7f039c0f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49460/0 (socket says 172.21.2.101:49460) 2021-02-10T11:30:48.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.218 7f03a102b700 1 --2- 172.21.2.101:0/2113194144 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f039c1b0870 0x7f039c0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:48.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.218 7f03a202d700 1 -- 172.21.2.101:0/2113194144 learned_addr learned my addr 172.21.2.101:0/2113194144 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:48.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.218 7f03a082a700 1 -- 172.21.2.101:0/2113194144 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423521658 0 0) 0x7f0398000c60 con 0x7f039c1b0470 2021-02-10T11:30:48.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.218 7f03a202d700 1 -- 172.21.2.101:0/2113194144 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f039c1b0870 msgr2=0x7f039c0f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:48.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.218 7f03a202d700 1 --2- 172.21.2.101:0/2113194144 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f039c1b0870 0x7f039c0f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:48.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.218 7f03a202d700 1 -- 172.21.2.101:0/2113194144 >> v1:172.21.2.108:6789/0 conn(0x7f039c1b0470 legacy=0x7f039c0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:48.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.218 7f03a202d700 1 -- 172.21.2.101:0/2113194144 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c0d2fb0 con 0x7f039c0efe00 2021-02-10T11:30:48.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.218 7f03a102b700 1 --2- 172.21.2.101:0/2113194144 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f039c1b0870 0x7f039c0f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:48.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.218 7f03a202d700 1 --2- 172.21.2.101:0/2113194144 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f039c0efe00 0x7f039c0f0200 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7f0394006f00 tx=0x7f0394009f10).ready entity=mon.0 client_cookie=fcad88dcd7367c34 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:48.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.218 7f03a082a700 1 -- 172.21.2.101:0/2113194144 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f039400b3b0 con 0x7f039c0efe00 2021-02-10T11:30:48.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.218 7f03a082a700 1 -- 172.21.2.101:0/2113194144 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039400b730 con 0x7f039c0efe00 2021-02-10T11:30:48.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.218 7f03a082a700 1 -- 172.21.2.101:0/2113194144 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f039400bc10 con 0x7f039c0efe00 2021-02-10T11:30:48.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.218 7f03a3a10700 1 -- 172.21.2.101:0/2113194144 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f039c0efe00 msgr2=0x7f039c0f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:48.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.218 7f03a3a10700 1 --2- 172.21.2.101:0/2113194144 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f039c0efe00 0x7f039c0f0200 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7f0394006f00 tx=0x7f0394009f10).stop 2021-02-10T11:30:48.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.219 7f03a3a10700 1 -- 172.21.2.101:0/2113194144 shutdown_connections 2021-02-10T11:30:48.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.219 7f03a3a10700 1 --2- 172.21.2.101:0/2113194144 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f039c1b0870 0x7f039c0f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:48.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.219 7f03a3a10700 1 --2- 172.21.2.101:0/2113194144 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f039c0efe00 0x7f039c0f0200 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:48.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.219 7f03a3a10700 1 -- 172.21.2.101:0/2113194144 >> 172.21.2.101:0/2113194144 conn(0x7f039c0ce3e0 msgr2=0x7f039c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:48.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.219 7f03a3a10700 1 -- 172.21.2.101:0/2113194144 shutdown_connections 2021-02-10T11:30:48.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.219 7f03a3a10700 1 -- 172.21.2.101:0/2113194144 wait complete. 2021-02-10T11:30:48.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.235 7f03a3a10700 1 Processor -- start 2021-02-10T11:30:48.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.235 7f03a3a10700 1 -- start start 2021-02-10T11:30:48.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.235 7f03a3a10700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f039c190920 0x7f039c0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:48.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.235 7f03a3a10700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f039c0b8200 0x7f039c0b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:48.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.235 7f03a3a10700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f039c0c1d80 0x7f039c0c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:48.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.235 7f03a3a10700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c190da0 con 0x7f039c190920 2021-02-10T11:30:48.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.235 7f03a3a10700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c0b8c20 con 0x7f039c0b8200 2021-02-10T11:30:48.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.235 7f03a3a10700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c0b8e10 con 0x7f039c0c1d80 2021-02-10T11:30:48.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.235 7f03a182c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f039c190920 0x7f039c0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:48.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.235 7f03a202d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f039c0c1d80 0x7f039c0c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:48.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.235 7f03a182c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f039c190920 0x7f039c0b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49462/0 (socket says 172.21.2.101:49462) 2021-02-10T11:30:48.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.235 7f03a182c700 1 -- 172.21.2.101:0/1130793593 learned_addr learned my addr 172.21.2.101:0/1130793593 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:48.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.235 7f03a102b700 1 --2- 172.21.2.101:0/1130793593 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f039c0b8200 0x7f039c0b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:48.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.236 7f03a182c700 1 -- 172.21.2.101:0/1130793593 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f039c0c1d80 msgr2=0x7f039c0c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:48.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.236 7f03a182c700 1 --2- 172.21.2.101:0/1130793593 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f039c0c1d80 0x7f039c0c2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:48.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.236 7f03a182c700 1 -- 172.21.2.101:0/1130793593 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f039c0b8200 msgr2=0x7f039c0b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:48.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.236 7f03a182c700 1 --2- 172.21.2.101:0/1130793593 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f039c0b8200 0x7f039c0b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:48.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.236 7f03a182c700 1 -- 172.21.2.101:0/1130793593 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c0c27d0 con 0x7f039c190920 2021-02-10T11:30:48.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.236 7f03a102b700 1 --2- 172.21.2.101:0/1130793593 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f039c0b8200 0x7f039c0b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:30:48.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.236 7f03a182c700 1 --2- 172.21.2.101:0/1130793593 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f039c190920 0x7f039c0b7be0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f0398006f20 tx=0x7f039800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:48.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.236 7f03927fc700 1 -- 172.21.2.101:0/1130793593 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f039800bd80 con 0x7f039c190920 2021-02-10T11:30:48.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.236 7f03927fc700 1 -- 172.21.2.101:0/1130793593 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0398001080 con 0x7f039c190920 2021-02-10T11:30:48.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.236 7f03a3a10700 1 -- 172.21.2.101:0/1130793593 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f039c0c27d0 con 0x7f039c190920 2021-02-10T11:30:48.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.236 7f03927fc700 1 -- 172.21.2.101:0/1130793593 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0398001550 con 0x7f039c190920 2021-02-10T11:30:48.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.236 7f03a3a10700 1 -- 172.21.2.101:0/1130793593 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f039c0c27d0 con 0x7f039c190920 2021-02-10T11:30:48.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.236 7f03927fc700 1 -- 172.21.2.101:0/1130793593 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0398001260 con 0x7f039c190920 2021-02-10T11:30:48.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.236 7f03927fc700 1 --2- 172.21.2.101:0/1130793593 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0384018730 0x7f038401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:48.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.236 7f03927fc700 1 -- 172.21.2.101:0/1130793593 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0398032660 con 0x7f039c190920 2021-02-10T11:30:48.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.236 7f03a102b700 1 --2- 172.21.2.101:0/1130793593 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0384018730 0x7f038401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:48.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.237 7f03a102b700 1 --2- 172.21.2.101:0/1130793593 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0384018730 0x7f038401ac10 crc :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:48.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.266 7f03a3a10700 1 -- 172.21.2.101:0/1130793593 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039c105ae0 con 0x7f039c190920 2021-02-10T11:30:48.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.269 7f03927fc700 1 -- 172.21.2.101:0/1130793593 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0398006bf0 con 0x7f039c190920 2021-02-10T11:30:48.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.399 7f03a3a10700 1 -- 172.21.2.101:0/1130793593 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839316", "target": ["mgr", ""]}) v1 -- 0x7f039c105ae0 con 0x7f0384018730 2021-02-10T11:30:48.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.410 7f03927fc700 1 -- 172.21.2.101:0/1130793593 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f038c0009b0 con 0x7f0384018730 2021-02-10T11:30:48.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.416 7f03a3a10700 1 -- 172.21.2.101:0/1130793593 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0384018730 msgr2=0x7f038401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:48.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.417 7f03a3a10700 1 --2- 172.21.2.101:0/1130793593 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0384018730 0x7f038401ac10 crc :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:48.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.417 7f03a3a10700 1 -- 172.21.2.101:0/1130793593 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f039c190920 msgr2=0x7f039c0b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:48.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.417 7f03a3a10700 1 --2- 172.21.2.101:0/1130793593 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f039c190920 0x7f039c0b7be0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f0398006f20 tx=0x7f039800b990).stop 2021-02-10T11:30:48.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.417 7f03a3a10700 1 -- 172.21.2.101:0/1130793593 shutdown_connections 2021-02-10T11:30:48.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.417 7f03a3a10700 1 --2- 172.21.2.101:0/1130793593 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0384018730 0x7f038401ac10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:48.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.417 7f03a3a10700 1 --2- 172.21.2.101:0/1130793593 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f039c0c1d80 0x7f039c0c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:48.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.417 7f03a3a10700 1 --2- 172.21.2.101:0/1130793593 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f039c0b8200 0x7f039c0b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:48.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.417 7f03a3a10700 1 --2- 172.21.2.101:0/1130793593 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f039c190920 0x7f039c0b7be0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:48.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.417 7f03a3a10700 1 -- 172.21.2.101:0/1130793593 >> 172.21.2.101:0/1130793593 conn(0x7f039c094e10 msgr2=0x7f039c0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:48.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.417 7f03a3a10700 1 -- 172.21.2.101:0/1130793593 shutdown_connections 2021-02-10T11:30:48.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.417 7f03a3a10700 1 -- 172.21.2.101:0/1130793593 wait complete. 2021-02-10T11:30:48.436 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839317 2021-02-10T11:30:48.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.839 7f30f5a9a700 1 Processor -- start 2021-02-10T11:30:48.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.839 7f30f5a9a700 1 -- start start 2021-02-10T11:30:48.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.840 7f30f5a9a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30f01b0470 0x7f30f01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:48.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.840 7f30f5a9a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f00f0b50 con 0x7f30f00ef610 2021-02-10T11:30:48.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.840 7f30f5a9a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f00f24e0 con 0x7f30f00f20e0 2021-02-10T11:30:48.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.840 7f30f5a9a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f00e5ea0 con 0x7f30f01b0470 2021-02-10T11:30:48.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.840 7f30eeffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f30f00ef610 0x7f30f00efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:49964/0 (socket says 172.21.2.101:49964) 2021-02-10T11:30:48.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.840 7f30eeffd700 1 -- 172.21.2.101:0/566003787 learned_addr learned my addr 172.21.2.101:0/566003787 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:48.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.840 7f30ef7fe700 1 --2- 172.21.2.101:0/566003787 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30f01b0470 0x7f30f01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:48.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.840 7f30ee7fc700 1 -- 172.21.2.101:0/566003787 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153846509 0 0) 0x7f30e00010b0 con 0x7f30f00ef610 2021-02-10T11:30:48.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.841 7f30ee7fc700 1 -- 172.21.2.101:0/566003787 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30dc002fe0 con 0x7f30f00ef610 2021-02-10T11:30:48.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.841 7f30ee7fc700 1 -- 172.21.2.101:0/566003787 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3328838173 0 0) 0x7f30e8000f90 con 0x7f30f00f20e0 2021-02-10T11:30:48.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.841 7f30ee7fc700 1 -- 172.21.2.101:0/566003787 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30dc003210 con 0x7f30f00f20e0 2021-02-10T11:30:48.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.841 7f30ef7fe700 1 -- 172.21.2.101:0/566003787 >> v1:172.21.2.108:6789/0 conn(0x7f30f00f20e0 legacy=0x7f30f00f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:48.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.841 7f30ef7fe700 1 -- 172.21.2.101:0/566003787 >> v1:172.21.2.101:6789/0 conn(0x7f30f00ef610 legacy=0x7f30f00efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:48.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.841 7f30ef7fe700 1 -- 172.21.2.101:0/566003787 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f00e52e0 con 0x7f30f01b0470 2021-02-10T11:30:48.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.841 7f30ef7fe700 1 --2- 172.21.2.101:0/566003787 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30f01b0470 0x7f30f01b0870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f30d8006650 tx=0x7f30d8009a80).ready entity=mon.2 client_cookie=29769844ac33bf21 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:48.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.841 7f30ee7fc700 1 -- 172.21.2.101:0/566003787 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f30d80009b0 con 0x7f30f01b0470 2021-02-10T11:30:48.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.841 7f30ee7fc700 1 -- 172.21.2.101:0/566003787 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d800c0c0 con 0x7f30f01b0470 2021-02-10T11:30:48.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.841 7f30ee7fc700 1 -- 172.21.2.101:0/566003787 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f30d800c510 con 0x7f30f01b0470 2021-02-10T11:30:48.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.841 7f30f5a9a700 1 -- 172.21.2.101:0/566003787 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30f01b0470 msgr2=0x7f30f01b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:48.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.841 7f30f5a9a700 1 --2- 172.21.2.101:0/566003787 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30f01b0470 0x7f30f01b0870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f30d8006650 tx=0x7f30d8009a80).stop 2021-02-10T11:30:48.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.842 7f30f5a9a700 1 -- 172.21.2.101:0/566003787 shutdown_connections 2021-02-10T11:30:48.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.842 7f30f5a9a700 1 --2- 172.21.2.101:0/566003787 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30f01b0470 0x7f30f01b0870 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:48.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.842 7f30f5a9a700 1 -- 172.21.2.101:0/566003787 >> 172.21.2.101:0/566003787 conn(0x7f30f00ce3e0 msgr2=0x7f30f00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:48.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.842 7f30f5a9a700 1 -- 172.21.2.101:0/566003787 shutdown_connections 2021-02-10T11:30:48.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.842 7f30f5a9a700 1 -- 172.21.2.101:0/566003787 wait complete. 2021-02-10T11:30:48.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.857 7f30f5a9a700 1 Processor -- start 2021-02-10T11:30:48.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.857 7f30f5a9a700 1 -- start start 2021-02-10T11:30:48.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.857 7f30f5a9a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f30f008ece0 0x7f30f00a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:48.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.857 7f30f5a9a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30f00a6860 0x7f30f00a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:48.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.857 7f30f5a9a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30f00a7280 0x7f30f00a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:48.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.857 7f30f5a9a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f00a5cc0 con 0x7f30f00a6860 2021-02-10T11:30:48.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.857 7f30f5a9a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f008f0e0 con 0x7f30f008ece0 2021-02-10T11:30:48.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.857 7f30f5a9a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f008f2c0 con 0x7f30f00a7280 2021-02-10T11:30:48.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.858 7f30eeffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30f00a6860 0x7f30f00a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:48.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.858 7f30eeffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30f00a6860 0x7f30f00a6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49478/0 (socket says 172.21.2.101:49478) 2021-02-10T11:30:48.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.858 7f30eeffd700 1 -- 172.21.2.101:0/3075301006 learned_addr learned my addr 172.21.2.101:0/3075301006 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:48.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.858 7f30eeffd700 1 -- 172.21.2.101:0/3075301006 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30f00a7280 msgr2=0x7f30f00a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:48.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.858 7f30eeffd700 1 --2- 172.21.2.101:0/3075301006 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30f00a7280 0x7f30f00a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:48.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.858 7f30eeffd700 1 -- 172.21.2.101:0/3075301006 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f30f008ece0 msgr2=0x7f30f00a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:48.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.858 7f30eeffd700 1 --2- 172.21.2.101:0/3075301006 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f30f008ece0 0x7f30f00a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:48.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.858 7f30eeffd700 1 -- 172.21.2.101:0/3075301006 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f00a4bb0 con 0x7f30f00a6860 2021-02-10T11:30:48.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.858 7f30eeffd700 1 --2- 172.21.2.101:0/3075301006 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30f00a6860 0x7f30f00a6c60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7f30e0001c60 tx=0x7f30e000a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:48.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.858 7f30e7fff700 1 -- 172.21.2.101:0/3075301006 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f30e000acc0 con 0x7f30f00a6860 2021-02-10T11:30:48.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.858 7f30e7fff700 1 -- 172.21.2.101:0/3075301006 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e0004140 con 0x7f30f00a6860 2021-02-10T11:30:48.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.858 7f30e7fff700 1 -- 172.21.2.101:0/3075301006 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f30e0004620 con 0x7f30f00a6860 2021-02-10T11:30:48.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.858 7f30f5a9a700 1 -- 172.21.2.101:0/3075301006 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f00a4bb0 con 0x7f30f00a6860 2021-02-10T11:30:48.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.858 7f30f5a9a700 1 -- 172.21.2.101:0/3075301006 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f00a5140 con 0x7f30f00a6860 2021-02-10T11:30:48.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.859 7f30e7fff700 1 -- 172.21.2.101:0/3075301006 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f30e0004320 con 0x7f30f00a6860 2021-02-10T11:30:48.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.859 7f30e7fff700 1 --2- 172.21.2.101:0/3075301006 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30d40186c0 0x7f30d401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:48.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.859 7f30e7fff700 1 -- 172.21.2.101:0/3075301006 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f30e00323f0 con 0x7f30f00a6860 2021-02-10T11:30:48.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.859 7f30ef7fe700 1 --2- 172.21.2.101:0/3075301006 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30d40186c0 0x7f30d401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:48.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.860 7f30ef7fe700 1 --2- 172.21.2.101:0/3075301006 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30d40186c0 0x7f30d401abc0 crc :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:48.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.889 7f30f5a9a700 1 -- 172.21.2.101:0/3075301006 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f0105ae0 con 0x7f30f00a6860 2021-02-10T11:30:48.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:48.892 7f30e7fff700 1 -- 172.21.2.101:0/3075301006 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f30e0000940 con 0x7f30f00a6860 2021-02-10T11:30:49.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.019 7f30f5a9a700 1 -- 172.21.2.101:0/3075301006 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839317", "target": ["mgr", ""]}) v1 -- 0x7f30f00f0b50 con 0x7f30d40186c0 2021-02-10T11:30:49.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.029 7f30e7fff700 1 -- 172.21.2.101:0/3075301006 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f30d8000940 con 0x7f30d40186c0 2021-02-10T11:30:49.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.036 7f30f5a9a700 1 -- 172.21.2.101:0/3075301006 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30d40186c0 msgr2=0x7f30d401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:49.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.036 7f30f5a9a700 1 --2- 172.21.2.101:0/3075301006 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30d40186c0 0x7f30d401abc0 crc :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:49.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.036 7f30f5a9a700 1 -- 172.21.2.101:0/3075301006 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30f00a6860 msgr2=0x7f30f00a6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:49.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.036 7f30f5a9a700 1 --2- 172.21.2.101:0/3075301006 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30f00a6860 0x7f30f00a6c60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7f30e0001c60 tx=0x7f30e000a8d0).stop 2021-02-10T11:30:49.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.036 7f30f5a9a700 1 -- 172.21.2.101:0/3075301006 shutdown_connections 2021-02-10T11:30:49.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.036 7f30f5a9a700 1 --2- 172.21.2.101:0/3075301006 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30d40186c0 0x7f30d401abc0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:49.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.036 7f30f5a9a700 1 --2- 172.21.2.101:0/3075301006 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30f00a7280 0x7f30f00a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:49.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.036 7f30f5a9a700 1 --2- 172.21.2.101:0/3075301006 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30f00a6860 0x7f30f00a6c60 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:49.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.036 7f30f5a9a700 1 --2- 172.21.2.101:0/3075301006 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f30f008ece0 0x7f30f00a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:49.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.036 7f30f5a9a700 1 -- 172.21.2.101:0/3075301006 >> 172.21.2.101:0/3075301006 conn(0x7f30f008ffe0 msgr2=0x7f30f00bf050 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:49.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.036 7f30f5a9a700 1 -- 172.21.2.101:0/3075301006 shutdown_connections 2021-02-10T11:30:49.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.036 7f30f5a9a700 1 -- 172.21.2.101:0/3075301006 wait complete. 2021-02-10T11:30:49.056 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839318 2021-02-10T11:30:49.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.459 7f67bdd11700 1 Processor -- start 2021-02-10T11:30:49.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.460 7f67bdd11700 1 -- start start 2021-02-10T11:30:49.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.460 7f67bdd11700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f67b81b0870 0x7f67b80f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:49.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.460 7f67bdd11700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67b80efe00 0x7f67b80f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:49.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.460 7f67bdd11700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b81b0c70 con 0x7f67b81b0470 2021-02-10T11:30:49.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.460 7f67bdd11700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b80f0b50 con 0x7f67b81b0870 2021-02-10T11:30:49.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.460 7f67bdd11700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b80f0820 con 0x7f67b80efe00 2021-02-10T11:30:49.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.460 7f67b7fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67b80efe00 0x7f67b80f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:49.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.460 7f67b7fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67b80efe00 0x7f67b80f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:42214/0 (socket says 172.21.2.101:42214) 2021-02-10T11:30:49.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.460 7f67b7fff700 1 -- 172.21.2.101:0/3768542219 learned_addr learned my addr 172.21.2.101:0/3768542219 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:49.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.460 7f67b6ffd700 1 --2- 172.21.2.101:0/3768542219 >> v2:172.21.2.108:3300/0 conn(0x7f67b81b0870 0x7f67b80f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:49.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.461 7f67b7fff700 1 -- 172.21.2.101:0/3768542219 >> v2:172.21.2.108:3300/0 conn(0x7f67b81b0870 msgr2=0x7f67b80f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:49.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.461 7f67b7fff700 1 --2- 172.21.2.101:0/3768542219 >> v2:172.21.2.108:3300/0 conn(0x7f67b81b0870 0x7f67b80f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:49.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.461 7f67b7fff700 1 -- 172.21.2.101:0/3768542219 >> v1:172.21.2.108:6789/0 conn(0x7f67b81b0470 legacy=0x7f67b80ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:49.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.461 7f67b7fff700 1 -- 172.21.2.101:0/3768542219 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b80e52e0 con 0x7f67b80efe00 2021-02-10T11:30:49.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.461 7f67b6ffd700 1 --2- 172.21.2.101:0/3768542219 >> v2:172.21.2.108:3300/0 conn(0x7f67b81b0870 0x7f67b80f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:49.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.461 7f67b7fff700 1 --2- 172.21.2.101:0/3768542219 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67b80efe00 0x7f67b80f0200 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f67a8006720 tx=0x7f67a8009ba0).ready entity=mon.2 client_cookie=22ebeb94d1bfbd8b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:49.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.461 7f67b67fc700 1 -- 172.21.2.101:0/3768542219 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f67a8006370 con 0x7f67b80efe00 2021-02-10T11:30:49.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.461 7f67b67fc700 1 -- 172.21.2.101:0/3768542219 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a80022e0 con 0x7f67b80efe00 2021-02-10T11:30:49.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.461 7f67b67fc700 1 -- 172.21.2.101:0/3768542219 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f67a8002800 con 0x7f67b80efe00 2021-02-10T11:30:49.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.461 7f67bdd11700 1 -- 172.21.2.101:0/3768542219 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67b80efe00 msgr2=0x7f67b80f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:49.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.461 7f67bdd11700 1 --2- 172.21.2.101:0/3768542219 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67b80efe00 0x7f67b80f0200 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f67a8006720 tx=0x7f67a8009ba0).stop 2021-02-10T11:30:49.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.462 7f67bdd11700 1 -- 172.21.2.101:0/3768542219 shutdown_connections 2021-02-10T11:30:49.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.462 7f67bdd11700 1 --2- 172.21.2.101:0/3768542219 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67b80efe00 0x7f67b80f0200 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:49.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.462 7f67bdd11700 1 --2- 172.21.2.101:0/3768542219 >> v2:172.21.2.108:3300/0 conn(0x7f67b81b0870 0x7f67b80f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:49.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.462 7f67bdd11700 1 -- 172.21.2.101:0/3768542219 >> 172.21.2.101:0/3768542219 conn(0x7f67b80ce3e0 msgr2=0x7f67b80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:49.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.462 7f67bdd11700 1 -- 172.21.2.101:0/3768542219 shutdown_connections 2021-02-10T11:30:49.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.462 7f67bdd11700 1 -- 172.21.2.101:0/3768542219 wait complete. 2021-02-10T11:30:49.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.477 7f67bdd11700 1 Processor -- start 2021-02-10T11:30:49.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.477 7f67bdd11700 1 -- start start 2021-02-10T11:30:49.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.477 7f67bdd11700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f67b80a0420 0x7f67b80a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:49.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.477 7f67bdd11700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67b8189d00 0x7f67b818a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:49.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.477 7f67bdd11700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67b818a720 0x7f67b80b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:49.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.477 7f67bdd11700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b8189780 con 0x7f67b818a720 2021-02-10T11:30:49.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.477 7f67bdd11700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b818ab20 con 0x7f67b80a0420 2021-02-10T11:30:49.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.477 7f67bdd11700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b80a1140 con 0x7f67b8189d00 2021-02-10T11:30:49.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.477 7f67b7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67b818a720 0x7f67b80b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:49.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.477 7f67b7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67b818a720 0x7f67b80b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49488/0 (socket says 172.21.2.101:49488) 2021-02-10T11:30:49.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.477 7f67b7fff700 1 -- 172.21.2.101:0/2848131332 learned_addr learned my addr 172.21.2.101:0/2848131332 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:49.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.478 7f67b6ffd700 1 --2- 172.21.2.101:0/2848131332 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67b8189d00 0x7f67b818a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:49.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.478 7f67b77fe700 1 --2- 172.21.2.101:0/2848131332 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f67b80a0420 0x7f67b80a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:49.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.478 7f67b7fff700 1 -- 172.21.2.101:0/2848131332 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67b8189d00 msgr2=0x7f67b818a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:49.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.478 7f67b7fff700 1 --2- 172.21.2.101:0/2848131332 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67b8189d00 0x7f67b818a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:49.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.478 7f67b7fff700 1 -- 172.21.2.101:0/2848131332 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f67b80a0420 msgr2=0x7f67b80a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:49.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.478 7f67b7fff700 1 --2- 172.21.2.101:0/2848131332 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f67b80a0420 0x7f67b80a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:49.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.478 7f67b7fff700 1 -- 172.21.2.101:0/2848131332 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b80b8200 con 0x7f67b818a720 2021-02-10T11:30:49.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.478 7f67b6ffd700 1 --2- 172.21.2.101:0/2848131332 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67b8189d00 0x7f67b818a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:30:49.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.478 7f67b7fff700 1 --2- 172.21.2.101:0/2848131332 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67b818a720 0x7f67b80b7be0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7f67a80095c0 tx=0x7f67a800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:49.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.478 7f679ffff700 1 -- 172.21.2.101:0/2848131332 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f67a800bd80 con 0x7f67b818a720 2021-02-10T11:30:49.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.478 7f679ffff700 1 -- 172.21.2.101:0/2848131332 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a8006240 con 0x7f67b818a720 2021-02-10T11:30:49.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.478 7f679ffff700 1 -- 172.21.2.101:0/2848131332 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f67a8005520 con 0x7f67b818a720 2021-02-10T11:30:49.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.478 7f67bdd11700 1 -- 172.21.2.101:0/2848131332 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b80b8200 con 0x7f67b818a720 2021-02-10T11:30:49.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.478 7f67bdd11700 1 -- 172.21.2.101:0/2848131332 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b80b8790 con 0x7f67b818a720 2021-02-10T11:30:49.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.479 7f679ffff700 1 -- 172.21.2.101:0/2848131332 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f67a8005330 con 0x7f67b818a720 2021-02-10T11:30:49.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.479 7f679ffff700 1 --2- 172.21.2.101:0/2848131332 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f67980186c0 0x7f679801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:49.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.479 7f679ffff700 1 -- 172.21.2.101:0/2848131332 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f67a8032560 con 0x7f67b818a720 2021-02-10T11:30:49.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.479 7f67b77fe700 1 --2- 172.21.2.101:0/2848131332 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f67980186c0 0x7f679801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:49.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.479 7f67b77fe700 1 --2- 172.21.2.101:0/2848131332 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f67980186c0 0x7f679801abc0 crc :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:49.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.510 7f67bdd11700 1 -- 172.21.2.101:0/2848131332 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b8105ae0 con 0x7f67b818a720 2021-02-10T11:30:49.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.512 7f679ffff700 1 -- 172.21.2.101:0/2848131332 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f67a80090b0 con 0x7f67b818a720 2021-02-10T11:30:49.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.641 7f67bdd11700 1 -- 172.21.2.101:0/2848131332 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839318", "target": ["mgr", ""]}) v1 -- 0x7f67b80b9020 con 0x7f67980186c0 2021-02-10T11:30:49.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.651 7f679ffff700 1 -- 172.21.2.101:0/2848131332 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f67ac000960 con 0x7f67980186c0 2021-02-10T11:30:49.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.658 7f67bdd11700 1 -- 172.21.2.101:0/2848131332 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f67980186c0 msgr2=0x7f679801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:49.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.658 7f67bdd11700 1 --2- 172.21.2.101:0/2848131332 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f67980186c0 0x7f679801abc0 crc :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:49.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.658 7f67bdd11700 1 -- 172.21.2.101:0/2848131332 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67b818a720 msgr2=0x7f67b80b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:49.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.658 7f67bdd11700 1 --2- 172.21.2.101:0/2848131332 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67b818a720 0x7f67b80b7be0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7f67a80095c0 tx=0x7f67a800b990).stop 2021-02-10T11:30:49.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.658 7f67bdd11700 1 -- 172.21.2.101:0/2848131332 shutdown_connections 2021-02-10T11:30:49.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.658 7f67bdd11700 1 --2- 172.21.2.101:0/2848131332 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f67980186c0 0x7f679801abc0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:49.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.658 7f67bdd11700 1 --2- 172.21.2.101:0/2848131332 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67b8189d00 0x7f67b818a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:49.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.658 7f67bdd11700 1 --2- 172.21.2.101:0/2848131332 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67b818a720 0x7f67b80b7be0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:49.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.658 7f67bdd11700 1 --2- 172.21.2.101:0/2848131332 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f67b80a0420 0x7f67b80a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:49.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.658 7f67bdd11700 1 -- 172.21.2.101:0/2848131332 >> 172.21.2.101:0/2848131332 conn(0x7f67b8094e10 msgr2=0x7f67b80a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:49.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.658 7f67bdd11700 1 -- 172.21.2.101:0/2848131332 shutdown_connections 2021-02-10T11:30:49.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:49.658 7f67bdd11700 1 -- 172.21.2.101:0/2848131332 wait complete. 2021-02-10T11:30:49.677 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839319 2021-02-10T11:30:50.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.081 7f9328b0c700 1 Processor -- start 2021-02-10T11:30:50.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.081 7f9328b0c700 1 -- start start 2021-02-10T11:30:50.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.082 7f9328b0c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f93241b0870 0x7f93240f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:50.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.082 7f9328b0c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f93240efe00 0x7f93240f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:50.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.082 7f9328b0c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93241b0c70 con 0x7f93241b0470 2021-02-10T11:30:50.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.082 7f9328b0c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93240e5ea0 con 0x7f93241b0870 2021-02-10T11:30:50.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.082 7f9328b0c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93240f0b50 con 0x7f93240efe00 2021-02-10T11:30:50.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.082 7f932205d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f93241b0870 0x7f93240f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:50.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.082 7f932205d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f93241b0870 0x7f93240f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49496/0 (socket says 172.21.2.101:49496) 2021-02-10T11:30:50.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.082 7f932305f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f93240efe00 0x7f93240f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:50.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.082 7f932205d700 1 -- 172.21.2.101:0/318677281 learned_addr learned my addr 172.21.2.101:0/318677281 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:50.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.082 7f932285e700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f93241b0470 0x7f93240ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:43398/0 (socket says 172.21.2.101:43398) 2021-02-10T11:30:50.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.082 7f932305f700 1 -- 172.21.2.101:0/318677281 >> v2:172.21.2.101:3300/0 conn(0x7f93241b0870 msgr2=0x7f93240f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:50.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.082 7f932305f700 1 --2- 172.21.2.101:0/318677281 >> v2:172.21.2.101:3300/0 conn(0x7f93241b0870 0x7f93240f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:50.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.082 7f932305f700 1 -- 172.21.2.101:0/318677281 >> v1:172.21.2.108:6789/0 conn(0x7f93241b0470 legacy=0x7f93240ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:50.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.082 7f932305f700 1 -- 172.21.2.101:0/318677281 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93240f0820 con 0x7f93240efe00 2021-02-10T11:30:50.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.083 7f932305f700 1 --2- 172.21.2.101:0/318677281 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93240efe00 0x7f93240f0200 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f9314006ad0 tx=0x7f9314009ad0).ready entity=mon.1 client_cookie=752ed699575c5551 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:50.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.083 7f932185c700 1 -- 172.21.2.101:0/318677281 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f93140066a0 con 0x7f93240efe00 2021-02-10T11:30:50.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.083 7f932185c700 1 -- 172.21.2.101:0/318677281 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9314006410 con 0x7f93240efe00 2021-02-10T11:30:50.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.083 7f932185c700 1 -- 172.21.2.101:0/318677281 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f93140024d0 con 0x7f93240efe00 2021-02-10T11:30:50.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.083 7f9328b0c700 1 -- 172.21.2.101:0/318677281 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93240efe00 msgr2=0x7f93240f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:50.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.083 7f9328b0c700 1 --2- 172.21.2.101:0/318677281 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93240efe00 0x7f93240f0200 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f9314006ad0 tx=0x7f9314009ad0).stop 2021-02-10T11:30:50.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.084 7f9328b0c700 1 -- 172.21.2.101:0/318677281 shutdown_connections 2021-02-10T11:30:50.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.084 7f9328b0c700 1 --2- 172.21.2.101:0/318677281 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93240efe00 0x7f93240f0200 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:50.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.084 7f9328b0c700 1 --2- 172.21.2.101:0/318677281 >> v2:172.21.2.101:3300/0 conn(0x7f93241b0870 0x7f93240f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:50.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.084 7f9328b0c700 1 -- 172.21.2.101:0/318677281 >> 172.21.2.101:0/318677281 conn(0x7f93240ce3e0 msgr2=0x7f93240e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:50.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.084 7f9328b0c700 1 -- 172.21.2.101:0/318677281 shutdown_connections 2021-02-10T11:30:50.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.084 7f9328b0c700 1 -- 172.21.2.101:0/318677281 wait complete. 2021-02-10T11:30:50.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.098 7f9328b0c700 1 Processor -- start 2021-02-10T11:30:50.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.099 7f9328b0c700 1 -- start start 2021-02-10T11:30:50.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.099 7f9328b0c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9324105d10 0x7f932409eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:50.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.099 7f9328b0c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9324106110 0x7f932409f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:50.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.099 7f9328b0c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93240b2a00 0x7f93240b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:50.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.099 7f9328b0c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932409e560 con 0x7f9324106110 2021-02-10T11:30:50.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.099 7f9328b0c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932409c090 con 0x7f9324105d10 2021-02-10T11:30:50.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.099 7f9328b0c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93240b3420 con 0x7f93240b2a00 2021-02-10T11:30:50.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.099 7f932205d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9324106110 0x7f932409f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:50.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.099 7f932205d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9324106110 0x7f932409f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49502/0 (socket says 172.21.2.101:49502) 2021-02-10T11:30:50.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.099 7f932205d700 1 -- 172.21.2.101:0/2260463218 learned_addr learned my addr 172.21.2.101:0/2260463218 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:50.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.099 7f932305f700 1 --2- 172.21.2.101:0/2260463218 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93240b2a00 0x7f93240b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:50.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.099 7f932285e700 1 --2- 172.21.2.101:0/2260463218 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9324105d10 0x7f932409eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:50.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.100 7f932205d700 1 -- 172.21.2.101:0/2260463218 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93240b2a00 msgr2=0x7f93240b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:50.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.100 7f932205d700 1 --2- 172.21.2.101:0/2260463218 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93240b2a00 0x7f93240b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:50.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.100 7f932205d700 1 -- 172.21.2.101:0/2260463218 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9324105d10 msgr2=0x7f932409eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:50.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.100 7f932205d700 1 --2- 172.21.2.101:0/2260463218 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9324105d10 0x7f932409eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:50.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.100 7f932205d700 1 -- 172.21.2.101:0/2260463218 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93240b3640 con 0x7f9324106110 2021-02-10T11:30:50.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.100 7f932205d700 1 --2- 172.21.2.101:0/2260463218 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9324106110 0x7f932409f100 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7f931000b5e0 tx=0x7f931000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:50.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.100 7f931b7fe700 1 -- 172.21.2.101:0/2260463218 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f931000bd70 con 0x7f9324106110 2021-02-10T11:30:50.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.100 7f931b7fe700 1 -- 172.21.2.101:0/2260463218 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9310002030 con 0x7f9324106110 2021-02-10T11:30:50.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.100 7f931b7fe700 1 -- 172.21.2.101:0/2260463218 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9310002550 con 0x7f9324106110 2021-02-10T11:30:50.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.100 7f9328b0c700 1 -- 172.21.2.101:0/2260463218 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93240b3640 con 0x7f9324106110 2021-02-10T11:30:50.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.100 7f9328b0c700 1 -- 172.21.2.101:0/2260463218 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93240b3bd0 con 0x7f9324106110 2021-02-10T11:30:50.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.100 7f931b7fe700 1 -- 172.21.2.101:0/2260463218 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9310000c90 con 0x7f9324106110 2021-02-10T11:30:50.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.101 7f931b7fe700 1 --2- 172.21.2.101:0/2260463218 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93000186c0 0x7f930001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:50.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.101 7f931b7fe700 1 -- 172.21.2.101:0/2260463218 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9310032410 con 0x7f9324106110 2021-02-10T11:30:50.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.101 7f932285e700 1 --2- 172.21.2.101:0/2260463218 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93000186c0 0x7f930001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:50.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.101 7f932285e700 1 --2- 172.21.2.101:0/2260463218 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93000186c0 0x7f930001abc0 crc :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:50.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.131 7f9328b0c700 1 -- 172.21.2.101:0/2260463218 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9324105ae0 con 0x7f9324106110 2021-02-10T11:30:50.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.133 7f931b7fe700 1 -- 172.21.2.101:0/2260463218 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9310021010 con 0x7f9324106110 2021-02-10T11:30:50.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.263 7f9328b0c700 1 -- 172.21.2.101:0/2260463218 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839319", "target": ["mgr", ""]}) v1 -- 0x7f93241a16c0 con 0x7f93000186c0 2021-02-10T11:30:50.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.272 7f931b7fe700 1 -- 172.21.2.101:0/2260463218 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f931c00c050 con 0x7f93000186c0 2021-02-10T11:30:50.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.280 7f9328b0c700 1 -- 172.21.2.101:0/2260463218 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93000186c0 msgr2=0x7f930001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:50.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.280 7f9328b0c700 1 --2- 172.21.2.101:0/2260463218 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93000186c0 0x7f930001abc0 crc :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:50.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.280 7f9328b0c700 1 -- 172.21.2.101:0/2260463218 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9324106110 msgr2=0x7f932409f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:50.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.280 7f9328b0c700 1 --2- 172.21.2.101:0/2260463218 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9324106110 0x7f932409f100 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7f931000b5e0 tx=0x7f931000b980).stop 2021-02-10T11:30:50.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.280 7f9328b0c700 1 -- 172.21.2.101:0/2260463218 shutdown_connections 2021-02-10T11:30:50.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.280 7f9328b0c700 1 --2- 172.21.2.101:0/2260463218 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93000186c0 0x7f930001abc0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:50.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.280 7f9328b0c700 1 --2- 172.21.2.101:0/2260463218 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93240b2a00 0x7f93240b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:50.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.280 7f9328b0c700 1 --2- 172.21.2.101:0/2260463218 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9324106110 0x7f932409f100 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:50.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.280 7f9328b0c700 1 --2- 172.21.2.101:0/2260463218 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9324105d10 0x7f932409eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:50.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.280 7f9328b0c700 1 -- 172.21.2.101:0/2260463218 >> 172.21.2.101:0/2260463218 conn(0x7f93240c36f0 msgr2=0x7f93240bd640 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:50.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.280 7f9328b0c700 1 -- 172.21.2.101:0/2260463218 shutdown_connections 2021-02-10T11:30:50.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.280 7f9328b0c700 1 -- 172.21.2.101:0/2260463218 wait complete. 2021-02-10T11:30:50.300 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839320 2021-02-10T11:30:50.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.703 7f7a61a8d700 1 Processor -- start 2021-02-10T11:30:50.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.703 7f7a61a8d700 1 -- start start 2021-02-10T11:30:50.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.703 7f7a61a8d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7a5c1b0470 0x7f7a5c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:50.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.703 7f7a61a8d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5c0f0b50 con 0x7f7a5c0f20e0 2021-02-10T11:30:50.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.703 7f7a61a8d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5c0f24e0 con 0x7f7a5c0ef610 2021-02-10T11:30:50.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.703 7f7a61a8d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a5c0e5ea0 con 0x7f7a5c1b0470 2021-02-10T11:30:50.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.703 7f7a5bfff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f7a5c0f20e0 0x7f7a5c0f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50006/0 (socket says 172.21.2.101:50006) 2021-02-10T11:30:50.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.703 7f7a5bfff700 1 -- 172.21.2.101:0/3120537584 learned_addr learned my addr 172.21.2.101:0/3120537584 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:50.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.703 7f7a5b7fe700 1 --2- 172.21.2.101:0/3120537584 >> v2:172.21.2.108:3300/0 conn(0x7f7a5c1b0470 0x7f7a5c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:50.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.704 7f7a5a7fc700 1 -- 172.21.2.101:0/3120537584 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899469256 0 0) 0x7f7a4c001180 con 0x7f7a5c0f20e0 2021-02-10T11:30:50.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.704 7f7a5a7fc700 1 -- 172.21.2.101:0/3120537584 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a44002fe0 con 0x7f7a5c0f20e0 2021-02-10T11:30:50.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.704 7f7a5b7fe700 1 -- 172.21.2.101:0/3120537584 >> v1:172.21.2.108:6789/0 conn(0x7f7a5c0ef610 legacy=0x7f7a5c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:50.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.704 7f7a5b7fe700 1 -- 172.21.2.101:0/3120537584 >> v1:172.21.2.101:6789/0 conn(0x7f7a5c0f20e0 legacy=0x7f7a5c0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:50.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.704 7f7a5b7fe700 1 -- 172.21.2.101:0/3120537584 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c0e52e0 con 0x7f7a5c1b0470 2021-02-10T11:30:50.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.704 7f7a5b7fe700 1 --2- 172.21.2.101:0/3120537584 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a5c1b0470 0x7f7a5c1b0870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f7a50006650 tx=0x7f7a50009a00).ready entity=mon.1 client_cookie=44c0660ad3d62ba7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:50.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.704 7f7a5a7fc700 1 -- 172.21.2.101:0/3120537584 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7a500009b0 con 0x7f7a5c1b0470 2021-02-10T11:30:50.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.704 7f7a5a7fc700 1 -- 172.21.2.101:0/3120537584 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5000c0c0 con 0x7f7a5c1b0470 2021-02-10T11:30:50.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.704 7f7a5a7fc700 1 -- 172.21.2.101:0/3120537584 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7a5000c5a0 con 0x7f7a5c1b0470 2021-02-10T11:30:50.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.704 7f7a61a8d700 1 -- 172.21.2.101:0/3120537584 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a5c1b0470 msgr2=0x7f7a5c1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:50.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.704 7f7a61a8d700 1 --2- 172.21.2.101:0/3120537584 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a5c1b0470 0x7f7a5c1b0870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f7a50006650 tx=0x7f7a50009a00).stop 2021-02-10T11:30:50.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.705 7f7a61a8d700 1 -- 172.21.2.101:0/3120537584 shutdown_connections 2021-02-10T11:30:50.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.705 7f7a61a8d700 1 --2- 172.21.2.101:0/3120537584 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a5c1b0470 0x7f7a5c1b0870 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:50.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.705 7f7a61a8d700 1 -- 172.21.2.101:0/3120537584 >> 172.21.2.101:0/3120537584 conn(0x7f7a5c0ce3e0 msgr2=0x7f7a5c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:50.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.705 7f7a61a8d700 1 -- 172.21.2.101:0/3120537584 shutdown_connections 2021-02-10T11:30:50.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.705 7f7a61a8d700 1 -- 172.21.2.101:0/3120537584 wait complete. 2021-02-10T11:30:50.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.720 7f7a61a8d700 1 Processor -- start 2021-02-10T11:30:50.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.720 7f7a61a8d700 1 -- start start 2021-02-10T11:30:50.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.720 7f7a61a8d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a5c0b2840 0x7f7a5c1a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:50.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.720 7f7a61a8d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a5c0b30d0 0x7f7a5c0b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:50.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.720 7f7a61a8d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a5c0b3af0 0x7f7a5c0f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:50.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.720 7f7a61a8d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c0b2c40 con 0x7f7a5c0b3af0 2021-02-10T11:30:50.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.720 7f7a61a8d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c1a4920 con 0x7f7a5c0b30d0 2021-02-10T11:30:50.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.720 7f7a61a8d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c0f0ba0 con 0x7f7a5c0b2840 2021-02-10T11:30:50.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.720 7f7a5bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a5c0b3af0 0x7f7a5c0f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:50.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.720 7f7a5bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a5c0b3af0 0x7f7a5c0f0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49518/0 (socket says 172.21.2.101:49518) 2021-02-10T11:30:50.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.720 7f7a5bfff700 1 -- 172.21.2.101:0/99640301 learned_addr learned my addr 172.21.2.101:0/99640301 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:50.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.721 7f7a5b7fe700 1 --2- 172.21.2.101:0/99640301 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a5c0b2840 0x7f7a5c1a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:50.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.721 7f7a5affd700 1 --2- 172.21.2.101:0/99640301 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a5c0b30d0 0x7f7a5c0b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:50.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.721 7f7a5bfff700 1 -- 172.21.2.101:0/99640301 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a5c0b2840 msgr2=0x7f7a5c1a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:50.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.721 7f7a5bfff700 1 --2- 172.21.2.101:0/99640301 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a5c0b2840 0x7f7a5c1a4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:50.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.721 7f7a5bfff700 1 -- 172.21.2.101:0/99640301 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a5c0b30d0 msgr2=0x7f7a5c0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:50.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.721 7f7a5bfff700 1 --2- 172.21.2.101:0/99640301 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a5c0b30d0 0x7f7a5c0b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:50.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.721 7f7a5bfff700 1 -- 172.21.2.101:0/99640301 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c0f0dc0 con 0x7f7a5c0b3af0 2021-02-10T11:30:50.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.721 7f7a5bfff700 1 --2- 172.21.2.101:0/99640301 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a5c0b3af0 0x7f7a5c0f0580 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f7a4c006f00 tx=0x7f7a4c00b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:50.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.721 7f7a43fff700 1 -- 172.21.2.101:0/99640301 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7a4c006b90 con 0x7f7a5c0b3af0 2021-02-10T11:30:50.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.721 7f7a43fff700 1 -- 172.21.2.101:0/99640301 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c006020 con 0x7f7a5c0b3af0 2021-02-10T11:30:50.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.721 7f7a43fff700 1 -- 172.21.2.101:0/99640301 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7a4c0043a0 con 0x7f7a5c0b3af0 2021-02-10T11:30:50.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.721 7f7a61a8d700 1 -- 172.21.2.101:0/99640301 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5c0f0dc0 con 0x7f7a5c0b3af0 2021-02-10T11:30:50.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.721 7f7a61a8d700 1 -- 172.21.2.101:0/99640301 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5c0f1350 con 0x7f7a5c0b3af0 2021-02-10T11:30:50.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.722 7f7a43fff700 1 -- 172.21.2.101:0/99640301 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7a4c006200 con 0x7f7a5c0b3af0 2021-02-10T11:30:50.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.722 7f7a43fff700 1 --2- 172.21.2.101:0/99640301 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a3c0186c0 0x7f7a3c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:50.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.722 7f7a43fff700 1 -- 172.21.2.101:0/99640301 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7a4c0324f0 con 0x7f7a5c0b3af0 2021-02-10T11:30:50.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.722 7f7a5b7fe700 1 --2- 172.21.2.101:0/99640301 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a3c0186c0 0x7f7a3c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:50.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.722 7f7a5b7fe700 1 --2- 172.21.2.101:0/99640301 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a3c0186c0 0x7f7a3c01abc0 crc :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:50.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.752 7f7a61a8d700 1 -- 172.21.2.101:0/99640301 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a5c0f18f0 con 0x7f7a5c0b3af0 2021-02-10T11:30:50.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.755 7f7a43fff700 1 -- 172.21.2.101:0/99640301 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7a4c000c10 con 0x7f7a5c0b3af0 2021-02-10T11:30:50.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.882 7f7a61a8d700 1 -- 172.21.2.101:0/99640301 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839320", "target": ["mgr", ""]}) v1 -- 0x7f7a5c0f1350 con 0x7f7a3c0186c0 2021-02-10T11:30:50.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.892 7f7a43fff700 1 -- 172.21.2.101:0/99640301 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f7a50006450 con 0x7f7a3c0186c0 2021-02-10T11:30:50.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.899 7f7a61a8d700 1 -- 172.21.2.101:0/99640301 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a3c0186c0 msgr2=0x7f7a3c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:50.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.899 7f7a61a8d700 1 --2- 172.21.2.101:0/99640301 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a3c0186c0 0x7f7a3c01abc0 crc :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:50.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.899 7f7a61a8d700 1 -- 172.21.2.101:0/99640301 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a5c0b3af0 msgr2=0x7f7a5c0f0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:50.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.899 7f7a61a8d700 1 --2- 172.21.2.101:0/99640301 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a5c0b3af0 0x7f7a5c0f0580 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f7a4c006f00 tx=0x7f7a4c00b940).stop 2021-02-10T11:30:50.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.899 7f7a61a8d700 1 -- 172.21.2.101:0/99640301 shutdown_connections 2021-02-10T11:30:50.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.899 7f7a61a8d700 1 --2- 172.21.2.101:0/99640301 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a5c0b3af0 0x7f7a5c0f0580 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:50.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.899 7f7a61a8d700 1 --2- 172.21.2.101:0/99640301 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a5c0b30d0 0x7f7a5c0b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:50.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.899 7f7a61a8d700 1 --2- 172.21.2.101:0/99640301 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a3c0186c0 0x7f7a3c01abc0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:50.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.899 7f7a61a8d700 1 --2- 172.21.2.101:0/99640301 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a5c0b2840 0x7f7a5c1a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:50.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.899 7f7a61a8d700 1 -- 172.21.2.101:0/99640301 >> 172.21.2.101:0/99640301 conn(0x7f7a5c0c36f0 msgr2=0x7f7a5c094c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:50.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.900 7f7a61a8d700 1 -- 172.21.2.101:0/99640301 shutdown_connections 2021-02-10T11:30:50.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:50.900 7f7a61a8d700 1 -- 172.21.2.101:0/99640301 wait complete. 2021-02-10T11:30:50.920 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839321 2021-02-10T11:30:51.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.323 7f058c60d700 1 Processor -- start 2021-02-10T11:30:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.323 7f058c60d700 1 -- start start 2021-02-10T11:30:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.324 7f058c60d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f05841b0470 0x7f05841b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.324 7f058c60d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f05840f20e0 0x7f05840f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.324 7f058c60d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05840f0b50 con 0x7f05840ef610 2021-02-10T11:30:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.324 7f058c60d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05840f0820 con 0x7f05841b0470 2021-02-10T11:30:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.324 7f058c60d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05840f24e0 con 0x7f05840f20e0 2021-02-10T11:30:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.324 7f058a429700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f05841b0470 0x7f05841b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.324 7f058a429700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f05841b0470 0x7f05841b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49522/0 (socket says 172.21.2.101:49522) 2021-02-10T11:30:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.324 7f058a429700 1 -- 172.21.2.101:0/736547445 learned_addr learned my addr 172.21.2.101:0/736547445 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.324 7f058ac2a700 1 --2- 172.21.2.101:0/736547445 >> v2:172.21.2.108:3300/0 conn(0x7f05840f20e0 0x7f05840f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.324 7f0589c28700 1 --1- 172.21.2.101:0/736547445 >> v1:172.21.2.101:6789/0 conn(0x7f05840ef610 0x7f05840efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50020/0 (socket says 172.21.2.101:50020) 2021-02-10T11:30:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.324 7f058ac2a700 1 -- 172.21.2.101:0/736547445 >> v2:172.21.2.101:3300/0 conn(0x7f05841b0470 msgr2=0x7f05841b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.324 7f058ac2a700 1 --2- 172.21.2.101:0/736547445 >> v2:172.21.2.101:3300/0 conn(0x7f05841b0470 0x7f05841b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.324 7f058ac2a700 1 -- 172.21.2.101:0/736547445 >> v1:172.21.2.101:6789/0 conn(0x7f05840ef610 legacy=0x7f05840efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.324 7f058ac2a700 1 -- 172.21.2.101:0/736547445 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05840e52e0 con 0x7f05840f20e0 2021-02-10T11:30:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.324 7f058a429700 1 --2- 172.21.2.101:0/736547445 >> v2:172.21.2.101:3300/0 conn(0x7f05841b0470 0x7f05841b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.324 7f058ac2a700 1 --2- 172.21.2.101:0/736547445 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f05840f20e0 0x7f05840f0200 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f057c006e50 tx=0x7f057c009e70).ready entity=mon.1 client_cookie=fec604aaf9a4b1e6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.324 7f0589427700 1 -- 172.21.2.101:0/736547445 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f057c000f90 con 0x7f05840f20e0 2021-02-10T11:30:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.324 7f0589427700 1 -- 172.21.2.101:0/736547445 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057c003030 con 0x7f05840f20e0 2021-02-10T11:30:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.324 7f0589427700 1 -- 172.21.2.101:0/736547445 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f057c00b700 con 0x7f05840f20e0 2021-02-10T11:30:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.324 7f058c60d700 1 -- 172.21.2.101:0/736547445 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f05840f20e0 msgr2=0x7f05840f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.324 7f058c60d700 1 --2- 172.21.2.101:0/736547445 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f05840f20e0 0x7f05840f0200 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f057c006e50 tx=0x7f057c009e70).stop 2021-02-10T11:30:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.325 7f058c60d700 1 -- 172.21.2.101:0/736547445 shutdown_connections 2021-02-10T11:30:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.325 7f058c60d700 1 --2- 172.21.2.101:0/736547445 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f05840f20e0 0x7f05840f0200 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:51.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.325 7f058c60d700 1 --2- 172.21.2.101:0/736547445 >> v2:172.21.2.101:3300/0 conn(0x7f05841b0470 0x7f05841b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:51.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.325 7f058c60d700 1 -- 172.21.2.101:0/736547445 >> 172.21.2.101:0/736547445 conn(0x7f05840ce3e0 msgr2=0x7f05840e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:51.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.325 7f058c60d700 1 -- 172.21.2.101:0/736547445 shutdown_connections 2021-02-10T11:30:51.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.325 7f058c60d700 1 -- 172.21.2.101:0/736547445 wait complete. 2021-02-10T11:30:51.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.340 7f058c60d700 1 Processor -- start 2021-02-10T11:30:51.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.340 7f058c60d700 1 -- start start 2021-02-10T11:30:51.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.340 7f058c60d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05841a0470 0x7f058408a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:51.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.340 7f058c60d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f058408aa50 0x7f0584099180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:51.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.340 7f058c60d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05840997a0 0x7f0584099ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:51.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.340 7f058c60d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0584095610 con 0x7f05841a0470 2021-02-10T11:30:51.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.340 7f058c60d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0584095800 con 0x7f058408aa50 2021-02-10T11:30:51.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.340 7f058c60d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058409a1c0 con 0x7f05840997a0 2021-02-10T11:30:51.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.340 7f058a429700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05841a0470 0x7f058408a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:51.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.340 7f058a429700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05841a0470 0x7f058408a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49528/0 (socket says 172.21.2.101:49528) 2021-02-10T11:30:51.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.340 7f058a429700 1 -- 172.21.2.101:0/2532946177 learned_addr learned my addr 172.21.2.101:0/2532946177 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:51.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.341 7f058ac2a700 1 --2- 172.21.2.101:0/2532946177 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05840997a0 0x7f0584099ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:51.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.341 7f0589c28700 1 --2- 172.21.2.101:0/2532946177 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f058408aa50 0x7f0584099180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.341 7f058a429700 1 -- 172.21.2.101:0/2532946177 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05840997a0 msgr2=0x7f0584099ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.341 7f058a429700 1 --2- 172.21.2.101:0/2532946177 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05840997a0 0x7f0584099ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.341 7f058a429700 1 -- 172.21.2.101:0/2532946177 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f058408aa50 msgr2=0x7f0584099180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.341 7f058a429700 1 --2- 172.21.2.101:0/2532946177 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f058408aa50 0x7f0584099180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.341 7f058a429700 1 -- 172.21.2.101:0/2532946177 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058409a3b0 con 0x7f05841a0470 2021-02-10T11:30:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.341 7f058a429700 1 --2- 172.21.2.101:0/2532946177 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05841a0470 0x7f058408a430 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x7f0580006ee0 tx=0x7f058000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.341 7f057affd700 1 -- 172.21.2.101:0/2532946177 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f058000bd50 con 0x7f05841a0470 2021-02-10T11:30:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.341 7f057affd700 1 -- 172.21.2.101:0/2532946177 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05800052e0 con 0x7f05841a0470 2021-02-10T11:30:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.341 7f058c60d700 1 -- 172.21.2.101:0/2532946177 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f058409a3b0 con 0x7f05841a0470 2021-02-10T11:30:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.341 7f057affd700 1 -- 172.21.2.101:0/2532946177 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0580005800 con 0x7f05841a0470 2021-02-10T11:30:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.341 7f058c60d700 1 -- 172.21.2.101:0/2532946177 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058409a3b0 con 0x7f05841a0470 2021-02-10T11:30:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.341 7f057affd700 1 -- 172.21.2.101:0/2532946177 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f05800054c0 con 0x7f05841a0470 2021-02-10T11:30:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.341 7f057affd700 1 --2- 172.21.2.101:0/2532946177 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f056c018730 0x7f056c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.341 7f057affd700 1 -- 172.21.2.101:0/2532946177 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0580032630 con 0x7f05841a0470 2021-02-10T11:30:51.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.341 7f0589c28700 1 --2- 172.21.2.101:0/2532946177 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f056c018730 0x7f056c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:51.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.342 7f0589c28700 1 --2- 172.21.2.101:0/2532946177 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f056c018730 0x7f056c01ac10 crc :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:51.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.373 7f058c60d700 1 -- 172.21.2.101:0/2532946177 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0584105ae0 con 0x7f05841a0470 2021-02-10T11:30:51.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.375 7f057affd700 1 -- 172.21.2.101:0/2532946177 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f058001f4f0 con 0x7f05841a0470 2021-02-10T11:30:51.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.501 7f058c60d700 1 -- 172.21.2.101:0/2532946177 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839321", "target": ["mgr", ""]}) v1 -- 0x7f0584105ae0 con 0x7f056c018730 2021-02-10T11:30:51.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.512 7f057affd700 1 -- 172.21.2.101:0/2532946177 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f0574000e20 con 0x7f056c018730 2021-02-10T11:30:51.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.519 7f058c60d700 1 -- 172.21.2.101:0/2532946177 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f056c018730 msgr2=0x7f056c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:51.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.519 7f058c60d700 1 --2- 172.21.2.101:0/2532946177 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f056c018730 0x7f056c01ac10 crc :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:51.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.519 7f058c60d700 1 -- 172.21.2.101:0/2532946177 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05841a0470 msgr2=0x7f058408a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:51.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.519 7f058c60d700 1 --2- 172.21.2.101:0/2532946177 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05841a0470 0x7f058408a430 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x7f0580006ee0 tx=0x7f058000b960).stop 2021-02-10T11:30:51.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.519 7f058c60d700 1 -- 172.21.2.101:0/2532946177 shutdown_connections 2021-02-10T11:30:51.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.519 7f058c60d700 1 --2- 172.21.2.101:0/2532946177 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f056c018730 0x7f056c01ac10 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:51.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.519 7f058c60d700 1 --2- 172.21.2.101:0/2532946177 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05840997a0 0x7f0584099ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:51.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.519 7f058c60d700 1 --2- 172.21.2.101:0/2532946177 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f058408aa50 0x7f0584099180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:51.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.519 7f058c60d700 1 --2- 172.21.2.101:0/2532946177 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05841a0470 0x7f058408a430 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:51.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.519 7f058c60d700 1 -- 172.21.2.101:0/2532946177 >> 172.21.2.101:0/2532946177 conn(0x7f05840bbf60 msgr2=0x7f0584105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:51.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.519 7f058c60d700 1 -- 172.21.2.101:0/2532946177 shutdown_connections 2021-02-10T11:30:51.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.519 7f058c60d700 1 -- 172.21.2.101:0/2532946177 wait complete. 2021-02-10T11:30:51.538 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839322 2021-02-10T11:30:51.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.942 7ff20ec50700 1 Processor -- start 2021-02-10T11:30:51.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.942 7ff20ec50700 1 -- start start 2021-02-10T11:30:51.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.943 7ff20ec50700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff2081b0470 0x7ff2081b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:51.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.943 7ff20ec50700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff2080ef610 0x7ff2080efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:51.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.943 7ff20ec50700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2080f0030 0x7ff2080f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:51.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.943 7ff20ec50700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2080e5ea0 con 0x7ff2080ef610 2021-02-10T11:30:51.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.943 7ff20ec50700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2080f0b50 con 0x7ff2081b0470 2021-02-10T11:30:51.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.943 7ff20ec50700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2080e52e0 con 0x7ff2080f0030 2021-02-10T11:30:51.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.943 7ff207fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff2080ef610 0x7ff2080efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:51.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.943 7ff207fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff2080ef610 0x7ff2080efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49538/0 (socket says 172.21.2.101:49538) 2021-02-10T11:30:51.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.943 7ff207fff700 1 -- 172.21.2.101:0/433204546 learned_addr learned my addr 172.21.2.101:0/433204546 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:51.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.943 7ff20ca6c700 1 --2- 172.21.2.101:0/433204546 >> v2:172.21.2.108:3300/0 conn(0x7ff2081b0470 0x7ff2081b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:51.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.943 7ff20d26d700 1 --2- 172.21.2.101:0/433204546 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2080f0030 0x7ff2080f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:51.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.943 7ff207fff700 1 -- 172.21.2.101:0/433204546 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2080f0030 msgr2=0x7ff2080f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:51.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.943 7ff207fff700 1 --2- 172.21.2.101:0/433204546 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2080f0030 0x7ff2080f0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:51.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.943 7ff207fff700 1 -- 172.21.2.101:0/433204546 >> v2:172.21.2.108:3300/0 conn(0x7ff2081b0470 msgr2=0x7ff2081b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:51.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.943 7ff207fff700 1 --2- 172.21.2.101:0/433204546 >> v2:172.21.2.108:3300/0 conn(0x7ff2081b0470 0x7ff2081b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:51.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.943 7ff207fff700 1 -- 172.21.2.101:0/433204546 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2080f10d0 con 0x7ff2080ef610 2021-02-10T11:30:51.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.943 7ff207fff700 1 --2- 172.21.2.101:0/433204546 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2080ef610 0x7ff2080efa10 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7ff1f8006a80 tx=0x7ff1f8009f00).ready entity=mon.0 client_cookie=4b62b6bf4e0de422 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:51.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.944 7ff2077fe700 1 -- 172.21.2.101:0/433204546 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1f800b3b0 con 0x7ff2080ef610 2021-02-10T11:30:51.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.944 7ff2077fe700 1 -- 172.21.2.101:0/433204546 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f800b6b0 con 0x7ff2080ef610 2021-02-10T11:30:51.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.944 7ff2077fe700 1 -- 172.21.2.101:0/433204546 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1f800bb10 con 0x7ff2080ef610 2021-02-10T11:30:51.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.944 7ff20ec50700 1 -- 172.21.2.101:0/433204546 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2080ef610 msgr2=0x7ff2080efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:51.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.944 7ff20ec50700 1 --2- 172.21.2.101:0/433204546 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2080ef610 0x7ff2080efa10 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7ff1f8006a80 tx=0x7ff1f8009f00).stop 2021-02-10T11:30:51.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.944 7ff20ec50700 1 -- 172.21.2.101:0/433204546 shutdown_connections 2021-02-10T11:30:51.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.944 7ff20ec50700 1 --2- 172.21.2.101:0/433204546 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2080f0030 0x7ff2080f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:51.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.944 7ff20ec50700 1 --2- 172.21.2.101:0/433204546 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2080ef610 0x7ff2080efa10 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:51.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.944 7ff20ec50700 1 --2- 172.21.2.101:0/433204546 >> v2:172.21.2.108:3300/0 conn(0x7ff2081b0470 0x7ff2081b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:51.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.944 7ff20ec50700 1 -- 172.21.2.101:0/433204546 >> 172.21.2.101:0/433204546 conn(0x7ff2080ce3e0 msgr2=0x7ff2080e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:51.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.944 7ff20ec50700 1 -- 172.21.2.101:0/433204546 shutdown_connections 2021-02-10T11:30:51.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.945 7ff20ec50700 1 -- 172.21.2.101:0/433204546 wait complete. 2021-02-10T11:30:51.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.959 7ff20ec50700 1 Processor -- start 2021-02-10T11:30:51.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.959 7ff20ec50700 1 -- start start 2021-02-10T11:30:51.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.960 7ff20ec50700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff20808ec50 0x7ff20808f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:51.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.960 7ff20ec50700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff20808f670 0x7ff20808fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:51.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.960 7ff20ec50700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2081065f0 0x7ff2081069f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:51.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.960 7ff20ec50700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20808e6d0 con 0x7ff2081065f0 2021-02-10T11:30:51.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.960 7ff20ec50700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20808e380 con 0x7ff20808f670 2021-02-10T11:30:51.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.960 7ff20ec50700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff208107010 con 0x7ff20808ec50 2021-02-10T11:30:51.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.960 7ff20d26d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2081065f0 0x7ff2081069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:51.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.960 7ff20d26d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2081065f0 0x7ff2081069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49544/0 (socket says 172.21.2.101:49544) 2021-02-10T11:30:51.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.960 7ff20d26d700 1 -- 172.21.2.101:0/1575853366 learned_addr learned my addr 172.21.2.101:0/1575853366 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:51.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.960 7ff207fff700 1 --2- 172.21.2.101:0/1575853366 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff20808f670 0x7ff20808fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:51.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.960 7ff20ca6c700 1 --2- 172.21.2.101:0/1575853366 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff20808ec50 0x7ff20808f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:51.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.960 7ff20d26d700 1 -- 172.21.2.101:0/1575853366 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff20808ec50 msgr2=0x7ff20808f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:51.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.960 7ff20d26d700 1 --2- 172.21.2.101:0/1575853366 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff20808ec50 0x7ff20808f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:51.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.960 7ff20d26d700 1 -- 172.21.2.101:0/1575853366 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff20808f670 msgr2=0x7ff20808fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:51.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.960 7ff20d26d700 1 --2- 172.21.2.101:0/1575853366 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff20808f670 0x7ff20808fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:51.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.960 7ff20d26d700 1 -- 172.21.2.101:0/1575853366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff208107230 con 0x7ff2081065f0 2021-02-10T11:30:51.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.960 7ff20d26d700 1 --2- 172.21.2.101:0/1575853366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2081065f0 0x7ff2081069f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7ff200006f20 tx=0x7ff20000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:51.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.960 7ff2057fa700 1 -- 172.21.2.101:0/1575853366 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff20000bd80 con 0x7ff2081065f0 2021-02-10T11:30:51.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.961 7ff2057fa700 1 -- 172.21.2.101:0/1575853366 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2000067e0 con 0x7ff2081065f0 2021-02-10T11:30:51.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.961 7ff20ec50700 1 -- 172.21.2.101:0/1575853366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff208107230 con 0x7ff2081065f0 2021-02-10T11:30:51.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.961 7ff2057fa700 1 -- 172.21.2.101:0/1575853366 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff200005330 con 0x7ff2081065f0 2021-02-10T11:30:51.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.961 7ff20ec50700 1 -- 172.21.2.101:0/1575853366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2080bc820 con 0x7ff2081065f0 2021-02-10T11:30:51.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.961 7ff2057fa700 1 -- 172.21.2.101:0/1575853366 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff200005330 con 0x7ff2081065f0 2021-02-10T11:30:51.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.961 7ff2057fa700 1 --2- 172.21.2.101:0/1575853366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1ec018730 0x7ff1ec01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:51.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.961 7ff2057fa700 1 -- 172.21.2.101:0/1575853366 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff200032640 con 0x7ff2081065f0 2021-02-10T11:30:51.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.961 7ff20ca6c700 1 --2- 172.21.2.101:0/1575853366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1ec018730 0x7ff1ec01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:51.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.962 7ff20ca6c700 1 --2- 172.21.2.101:0/1575853366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1ec018730 0x7ff1ec01ac10 crc :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:52.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.992 7ff20ec50700 1 -- 172.21.2.101:0/1575853366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2081074d0 con 0x7ff2081065f0 2021-02-10T11:30:52.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:51.995 7ff2057fa700 1 -- 172.21.2.101:0/1575853366 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff2000067c0 con 0x7ff2081065f0 2021-02-10T11:30:52.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.121 7ff20ec50700 1 -- 172.21.2.101:0/1575853366 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839322", "target": ["mgr", ""]}) v1 -- 0x7ff2080bc820 con 0x7ff1ec018730 2021-02-10T11:30:52.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.131 7ff2057fa700 1 -- 172.21.2.101:0/1575853366 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7ff1f0006a70 con 0x7ff1ec018730 2021-02-10T11:30:52.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.138 7ff20ec50700 1 -- 172.21.2.101:0/1575853366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1ec018730 msgr2=0x7ff1ec01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:52.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.138 7ff20ec50700 1 --2- 172.21.2.101:0/1575853366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1ec018730 0x7ff1ec01ac10 crc :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:52.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.138 7ff20ec50700 1 -- 172.21.2.101:0/1575853366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2081065f0 msgr2=0x7ff2081069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:52.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.138 7ff20ec50700 1 --2- 172.21.2.101:0/1575853366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2081065f0 0x7ff2081069f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7ff200006f20 tx=0x7ff20000b990).stop 2021-02-10T11:30:52.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.138 7ff20ec50700 1 -- 172.21.2.101:0/1575853366 shutdown_connections 2021-02-10T11:30:52.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.138 7ff20ec50700 1 --2- 172.21.2.101:0/1575853366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2081065f0 0x7ff2081069f0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:52.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.138 7ff20ec50700 1 --2- 172.21.2.101:0/1575853366 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff20808f670 0x7ff20808fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:52.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.138 7ff20ec50700 1 --2- 172.21.2.101:0/1575853366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1ec018730 0x7ff1ec01ac10 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:52.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.138 7ff20ec50700 1 --2- 172.21.2.101:0/1575853366 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff20808ec50 0x7ff20808f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:52.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.138 7ff20ec50700 1 -- 172.21.2.101:0/1575853366 >> 172.21.2.101:0/1575853366 conn(0x7ff2080b42f0 msgr2=0x7ff20810a140 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:52.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.138 7ff20ec50700 1 -- 172.21.2.101:0/1575853366 shutdown_connections 2021-02-10T11:30:52.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.138 7ff20ec50700 1 -- 172.21.2.101:0/1575853366 wait complete. 2021-02-10T11:30:52.158 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839323 2021-02-10T11:30:52.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.571 7fd171399700 1 Processor -- start 2021-02-10T11:30:52.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.571 7fd171399700 1 -- start start 2021-02-10T11:30:52.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.571 7fd171399700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd16c1b0870 0x7fd16c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:52.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.571 7fd171399700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd16c0efe00 0x7fd16c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:52.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.571 7fd171399700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd16c1b0c70 con 0x7fd16c1b0470 2021-02-10T11:30:52.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.571 7fd171399700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd16c0f0b50 con 0x7fd16c0efe00 2021-02-10T11:30:52.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.572 7fd171399700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c0f0820 con 0x7fd16c1b0870 2021-02-10T11:30:52.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.572 7fd16287c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd16c1b0870 0x7fd16c0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:52.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.572 7fd16b87e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd16c0efe00 0x7fd16c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:52.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.572 7fd16287c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd16c1b0870 0x7fd16c0f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:42280/0 (socket says 172.21.2.101:42280) 2021-02-10T11:30:52.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.572 7fd16287c700 1 -- 172.21.2.101:0/4077304088 learned_addr learned my addr 172.21.2.101:0/4077304088 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:52.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.572 7fd16287c700 1 -- 172.21.2.101:0/4077304088 >> v2:172.21.2.108:3300/0 conn(0x7fd16c0efe00 msgr2=0x7fd16c0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:52.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.572 7fd16287c700 1 --2- 172.21.2.101:0/4077304088 >> v2:172.21.2.108:3300/0 conn(0x7fd16c0efe00 0x7fd16c0f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:52.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.572 7fd16287c700 1 -- 172.21.2.101:0/4077304088 >> v1:172.21.2.108:6789/0 conn(0x7fd16c1b0470 legacy=0x7fd16c0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:52.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.572 7fd16287c700 1 -- 172.21.2.101:0/4077304088 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16c0e52e0 con 0x7fd16c1b0870 2021-02-10T11:30:52.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.572 7fd16287c700 1 --2- 172.21.2.101:0/4077304088 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd16c1b0870 0x7fd16c0f20e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fd154006700 tx=0x7fd154009b80).ready entity=mon.2 client_cookie=13c6d890f84d22da server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:52.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.572 7fd16a87c700 1 -- 172.21.2.101:0/4077304088 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd15400c0c0 con 0x7fd16c1b0870 2021-02-10T11:30:52.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.573 7fd16a87c700 1 -- 172.21.2.101:0/4077304088 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15400c440 con 0x7fd16c1b0870 2021-02-10T11:30:52.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.573 7fd16a87c700 1 -- 172.21.2.101:0/4077304088 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd15400c920 con 0x7fd16c1b0870 2021-02-10T11:30:52.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.573 7fd171399700 1 -- 172.21.2.101:0/4077304088 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd16c1b0870 msgr2=0x7fd16c0f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:52.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.573 7fd171399700 1 --2- 172.21.2.101:0/4077304088 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd16c1b0870 0x7fd16c0f20e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fd154006700 tx=0x7fd154009b80).stop 2021-02-10T11:30:52.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.573 7fd171399700 1 -- 172.21.2.101:0/4077304088 shutdown_connections 2021-02-10T11:30:52.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.573 7fd171399700 1 --2- 172.21.2.101:0/4077304088 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd16c1b0870 0x7fd16c0f20e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:52.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.573 7fd171399700 1 --2- 172.21.2.101:0/4077304088 >> v2:172.21.2.108:3300/0 conn(0x7fd16c0efe00 0x7fd16c0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:52.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.573 7fd171399700 1 -- 172.21.2.101:0/4077304088 >> 172.21.2.101:0/4077304088 conn(0x7fd16c0ce3e0 msgr2=0x7fd16c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:52.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.573 7fd171399700 1 -- 172.21.2.101:0/4077304088 shutdown_connections 2021-02-10T11:30:52.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.573 7fd171399700 1 -- 172.21.2.101:0/4077304088 wait complete. 2021-02-10T11:30:52.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.588 7fd171399700 1 Processor -- start 2021-02-10T11:30:52.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.588 7fd171399700 1 -- start start 2021-02-10T11:30:52.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.588 7fd171399700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd16c0a0420 0x7fd16c0a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:52.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.588 7fd171399700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd16c189d00 0x7fd16c18a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:52.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.588 7fd171399700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd16c18a720 0x7fd16c0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:52.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.588 7fd171399700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c189780 con 0x7fd16c18a720 2021-02-10T11:30:52.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.588 7fd171399700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c18ab20 con 0x7fd16c189d00 2021-02-10T11:30:52.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.588 7fd171399700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c0a1140 con 0x7fd16c0a0420 2021-02-10T11:30:52.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.589 7fd16b87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd16c18a720 0x7fd16c0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:52.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.589 7fd16b87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd16c18a720 0x7fd16c0b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49552/0 (socket says 172.21.2.101:49552) 2021-02-10T11:30:52.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.589 7fd16b87e700 1 -- 172.21.2.101:0/71086541 learned_addr learned my addr 172.21.2.101:0/71086541 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:52.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.589 7fd16b07d700 1 --2- 172.21.2.101:0/71086541 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd16c0a0420 0x7fd16c0a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:52.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.589 7fd16287c700 1 --2- 172.21.2.101:0/71086541 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd16c189d00 0x7fd16c18a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:52.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.589 7fd16b87e700 1 -- 172.21.2.101:0/71086541 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd16c0a0420 msgr2=0x7fd16c0a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:52.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.589 7fd16b87e700 1 --2- 172.21.2.101:0/71086541 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd16c0a0420 0x7fd16c0a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:52.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.589 7fd16b87e700 1 -- 172.21.2.101:0/71086541 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd16c189d00 msgr2=0x7fd16c18a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:52.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.589 7fd16b87e700 1 --2- 172.21.2.101:0/71086541 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd16c189d00 0x7fd16c18a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:52.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.589 7fd16b87e700 1 -- 172.21.2.101:0/71086541 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16c0b8200 con 0x7fd16c18a720 2021-02-10T11:30:52.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.589 7fd16287c700 1 --2- 172.21.2.101:0/71086541 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd16c189d00 0x7fd16c18a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:52.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.589 7fd16b87e700 1 --2- 172.21.2.101:0/71086541 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd16c18a720 0x7fd16c0b7be0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7fd16400b970 tx=0x7fd16400bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:52.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.589 7fd168878700 1 -- 172.21.2.101:0/71086541 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd164000f90 con 0x7fd16c18a720 2021-02-10T11:30:52.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.589 7fd168878700 1 -- 172.21.2.101:0/71086541 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd164006aa0 con 0x7fd16c18a720 2021-02-10T11:30:52.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.589 7fd168878700 1 -- 172.21.2.101:0/71086541 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd1640015f0 con 0x7fd16c18a720 2021-02-10T11:30:52.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.589 7fd171399700 1 -- 172.21.2.101:0/71086541 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd16c0b8200 con 0x7fd16c18a720 2021-02-10T11:30:52.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.590 7fd171399700 1 -- 172.21.2.101:0/71086541 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd16c0b8200 con 0x7fd16c18a720 2021-02-10T11:30:52.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.590 7fd168878700 1 -- 172.21.2.101:0/71086541 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd164000e70 con 0x7fd16c18a720 2021-02-10T11:30:52.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.590 7fd168878700 1 --2- 172.21.2.101:0/71086541 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd150018730 0x7fd15001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:52.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.590 7fd168878700 1 -- 172.21.2.101:0/71086541 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd1640329c0 con 0x7fd16c18a720 2021-02-10T11:30:52.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.590 7fd16b07d700 1 --2- 172.21.2.101:0/71086541 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd150018730 0x7fd15001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:52.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.591 7fd16b07d700 1 --2- 172.21.2.101:0/71086541 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd150018730 0x7fd15001ac10 crc :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:52.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.620 7fd171399700 1 -- 172.21.2.101:0/71086541 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd16c105ae0 con 0x7fd16c18a720 2021-02-10T11:30:52.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.623 7fd168878700 1 -- 172.21.2.101:0/71086541 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd1640066f0 con 0x7fd16c18a720 2021-02-10T11:30:52.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.751 7fd171399700 1 -- 172.21.2.101:0/71086541 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839323", "target": ["mgr", ""]}) v1 -- 0x7fd16c105ae0 con 0x7fd150018730 2021-02-10T11:30:52.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.761 7fd168878700 1 -- 172.21.2.101:0/71086541 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fd15c006550 con 0x7fd150018730 2021-02-10T11:30:52.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.768 7fd171399700 1 -- 172.21.2.101:0/71086541 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd150018730 msgr2=0x7fd15001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:52.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.768 7fd171399700 1 --2- 172.21.2.101:0/71086541 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd150018730 0x7fd15001ac10 crc :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:52.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.768 7fd171399700 1 -- 172.21.2.101:0/71086541 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd16c18a720 msgr2=0x7fd16c0b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:52.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.768 7fd171399700 1 --2- 172.21.2.101:0/71086541 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd16c18a720 0x7fd16c0b7be0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7fd16400b970 tx=0x7fd16400bd10).stop 2021-02-10T11:30:52.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.768 7fd171399700 1 -- 172.21.2.101:0/71086541 shutdown_connections 2021-02-10T11:30:52.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.768 7fd171399700 1 --2- 172.21.2.101:0/71086541 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd16c18a720 0x7fd16c0b7be0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:52.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.768 7fd171399700 1 --2- 172.21.2.101:0/71086541 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd16c189d00 0x7fd16c18a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:52.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.768 7fd171399700 1 --2- 172.21.2.101:0/71086541 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd150018730 0x7fd15001ac10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:52.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.768 7fd171399700 1 --2- 172.21.2.101:0/71086541 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd16c0a0420 0x7fd16c0a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:52.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.768 7fd171399700 1 -- 172.21.2.101:0/71086541 >> 172.21.2.101:0/71086541 conn(0x7fd16c094e10 msgr2=0x7fd16c0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:52.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.768 7fd171399700 1 -- 172.21.2.101:0/71086541 shutdown_connections 2021-02-10T11:30:52.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:52.768 7fd171399700 1 -- 172.21.2.101:0/71086541 wait complete. 2021-02-10T11:30:52.788 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839324 2021-02-10T11:30:53.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.204 7fd6e2ef1700 1 Processor -- start 2021-02-10T11:30:53.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.204 7fd6e2ef1700 1 -- start start 2021-02-10T11:30:53.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.204 7fd6e2ef1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6dc0f05f0 0x7fd6dc0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:53.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.205 7fd6e2ef1700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6dc1b0c70 con 0x7fd6dc1b0870 2021-02-10T11:30:53.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.205 7fd6e2ef1700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6dc0f0b50 con 0x7fd6dc1b0470 2021-02-10T11:30:53.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.205 7fd6e2ef1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc0e5ea0 con 0x7fd6dc0f05f0 2021-02-10T11:30:53.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.205 7fd6dbfff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd6dc1b0870 0x7fd6dc0efe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50052/0 (socket says 172.21.2.101:50052) 2021-02-10T11:30:53.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.205 7fd6dbfff700 1 -- 172.21.2.101:0/736083265 learned_addr learned my addr 172.21.2.101:0/736083265 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:53.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.205 7fd6e150e700 1 --2- 172.21.2.101:0/736083265 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6dc0f05f0 0x7fd6dc0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:53.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.205 7fd6db7fe700 1 -- 172.21.2.101:0/736083265 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795112012 0 0) 0x7fd6cc001060 con 0x7fd6dc1b0870 2021-02-10T11:30:53.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.205 7fd6db7fe700 1 -- 172.21.2.101:0/736083265 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6c8002fe0 con 0x7fd6dc1b0870 2021-02-10T11:30:53.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.205 7fd6db7fe700 1 -- 172.21.2.101:0/736083265 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729020040 0 0) 0x7fd6c4001080 con 0x7fd6dc1b0470 2021-02-10T11:30:53.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.205 7fd6db7fe700 1 -- 172.21.2.101:0/736083265 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6c8004370 con 0x7fd6dc1b0470 2021-02-10T11:30:53.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.205 7fd6e150e700 1 -- 172.21.2.101:0/736083265 >> v1:172.21.2.108:6789/0 conn(0x7fd6dc1b0470 legacy=0x7fd6dc0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:53.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.205 7fd6e150e700 1 -- 172.21.2.101:0/736083265 >> v1:172.21.2.101:6789/0 conn(0x7fd6dc1b0870 legacy=0x7fd6dc0efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:53.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.205 7fd6e150e700 1 -- 172.21.2.101:0/736083265 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6dc0e52e0 con 0x7fd6dc0f05f0 2021-02-10T11:30:53.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.206 7fd6e150e700 1 --2- 172.21.2.101:0/736083265 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6dc0f05f0 0x7fd6dc0f20e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fd6d4006ad0 tx=0x7fd6d4009ad0).ready entity=mon.2 client_cookie=749a63deb5bdbb50 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:53.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.206 7fd6db7fe700 1 -- 172.21.2.101:0/736083265 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6d40022e0 con 0x7fd6dc0f05f0 2021-02-10T11:30:53.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.206 7fd6db7fe700 1 -- 172.21.2.101:0/736083265 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d4006410 con 0x7fd6dc0f05f0 2021-02-10T11:30:53.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.206 7fd6db7fe700 1 -- 172.21.2.101:0/736083265 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6d40022e0 con 0x7fd6dc0f05f0 2021-02-10T11:30:53.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.206 7fd6e2ef1700 1 -- 172.21.2.101:0/736083265 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6dc0f05f0 msgr2=0x7fd6dc0f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:53.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.206 7fd6e2ef1700 1 --2- 172.21.2.101:0/736083265 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6dc0f05f0 0x7fd6dc0f20e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fd6d4006ad0 tx=0x7fd6d4009ad0).stop 2021-02-10T11:30:53.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.207 7fd6e2ef1700 1 -- 172.21.2.101:0/736083265 shutdown_connections 2021-02-10T11:30:53.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.207 7fd6e2ef1700 1 --2- 172.21.2.101:0/736083265 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6dc0f05f0 0x7fd6dc0f20e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:53.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.207 7fd6e2ef1700 1 -- 172.21.2.101:0/736083265 >> 172.21.2.101:0/736083265 conn(0x7fd6dc0ce3e0 msgr2=0x7fd6dc0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:53.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.207 7fd6e2ef1700 1 -- 172.21.2.101:0/736083265 shutdown_connections 2021-02-10T11:30:53.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.207 7fd6e2ef1700 1 -- 172.21.2.101:0/736083265 wait complete. 2021-02-10T11:30:53.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.225 7fd6e2ef1700 1 Processor -- start 2021-02-10T11:30:53.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.225 7fd6e2ef1700 1 -- start start 2021-02-10T11:30:53.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.226 7fd6e2ef1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6dc0a0420 0x7fd6dc0a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:53.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.226 7fd6e2ef1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6dc189d00 0x7fd6dc18a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:53.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.226 7fd6e2ef1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6dc18a720 0x7fd6dc0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:53.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.226 7fd6e2ef1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc189780 con 0x7fd6dc18a720 2021-02-10T11:30:53.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.226 7fd6e2ef1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc18ab20 con 0x7fd6dc0a0420 2021-02-10T11:30:53.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.226 7fd6e2ef1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc0a1140 con 0x7fd6dc189d00 2021-02-10T11:30:53.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.226 7fd6e150e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6dc18a720 0x7fd6dc0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:53.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.226 7fd6e150e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6dc18a720 0x7fd6dc0b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49568/0 (socket says 172.21.2.101:49568) 2021-02-10T11:30:53.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.226 7fd6e150e700 1 -- 172.21.2.101:0/2842108215 learned_addr learned my addr 172.21.2.101:0/2842108215 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:53.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.226 7fd6dbfff700 1 --2- 172.21.2.101:0/2842108215 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6dc189d00 0x7fd6dc18a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:53.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.226 7fd6e0d0d700 1 --2- 172.21.2.101:0/2842108215 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6dc0a0420 0x7fd6dc0a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:53.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.226 7fd6e150e700 1 -- 172.21.2.101:0/2842108215 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6dc189d00 msgr2=0x7fd6dc18a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:53.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.226 7fd6e150e700 1 --2- 172.21.2.101:0/2842108215 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6dc189d00 0x7fd6dc18a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:53.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.226 7fd6e150e700 1 -- 172.21.2.101:0/2842108215 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6dc0a0420 msgr2=0x7fd6dc0a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:53.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.226 7fd6e150e700 1 --2- 172.21.2.101:0/2842108215 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6dc0a0420 0x7fd6dc0a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:53.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.226 7fd6e150e700 1 -- 172.21.2.101:0/2842108215 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6dc0b8200 con 0x7fd6dc18a720 2021-02-10T11:30:53.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.226 7fd6e150e700 1 --2- 172.21.2.101:0/2842108215 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6dc18a720 0x7fd6dc0b7be0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7fd6d4009670 tx=0x7fd6d400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:53.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.226 7fd6d97fa700 1 -- 172.21.2.101:0/2842108215 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6d4009020 con 0x7fd6dc18a720 2021-02-10T11:30:53.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.226 7fd6d97fa700 1 -- 172.21.2.101:0/2842108215 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d400bd80 con 0x7fd6dc18a720 2021-02-10T11:30:53.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.226 7fd6e2ef1700 1 -- 172.21.2.101:0/2842108215 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6dc0b8200 con 0x7fd6dc18a720 2021-02-10T11:30:53.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.226 7fd6d97fa700 1 -- 172.21.2.101:0/2842108215 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6d4002270 con 0x7fd6dc18a720 2021-02-10T11:30:53.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.226 7fd6e2ef1700 1 -- 172.21.2.101:0/2842108215 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6dc0b8200 con 0x7fd6dc18a720 2021-02-10T11:30:53.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.227 7fd6d97fa700 1 -- 172.21.2.101:0/2842108215 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd6d4002020 con 0x7fd6dc18a720 2021-02-10T11:30:53.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.227 7fd6d97fa700 1 --2- 172.21.2.101:0/2842108215 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6c0018730 0x7fd6c001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:53.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.227 7fd6d97fa700 1 -- 172.21.2.101:0/2842108215 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd6d40325f0 con 0x7fd6dc18a720 2021-02-10T11:30:53.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.227 7fd6e0d0d700 1 --2- 172.21.2.101:0/2842108215 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6c0018730 0x7fd6c001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:53.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.227 7fd6e0d0d700 1 --2- 172.21.2.101:0/2842108215 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6c0018730 0x7fd6c001abc0 crc :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:53.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.254 7fd6e2ef1700 1 -- 172.21.2.101:0/2842108215 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6dc105ae0 con 0x7fd6dc18a720 2021-02-10T11:30:53.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.256 7fd6d97fa700 1 -- 172.21.2.101:0/2842108215 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd6d4000dc0 con 0x7fd6dc18a720 2021-02-10T11:30:53.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.386 7fd6e2ef1700 1 -- 172.21.2.101:0/2842108215 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839324", "target": ["mgr", ""]}) v1 -- 0x7fd6dc189780 con 0x7fd6c0018730 2021-02-10T11:30:53.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.396 7fd6d97fa700 1 -- 172.21.2.101:0/2842108215 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fd6c4006550 con 0x7fd6c0018730 2021-02-10T11:30:53.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.403 7fd6e2ef1700 1 -- 172.21.2.101:0/2842108215 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6c0018730 msgr2=0x7fd6c001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:53.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.403 7fd6e2ef1700 1 --2- 172.21.2.101:0/2842108215 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6c0018730 0x7fd6c001abc0 crc :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:53.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.403 7fd6e2ef1700 1 -- 172.21.2.101:0/2842108215 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6dc18a720 msgr2=0x7fd6dc0b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:53.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.403 7fd6e2ef1700 1 --2- 172.21.2.101:0/2842108215 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6dc18a720 0x7fd6dc0b7be0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7fd6d4009670 tx=0x7fd6d400b990).stop 2021-02-10T11:30:53.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.403 7fd6e2ef1700 1 -- 172.21.2.101:0/2842108215 shutdown_connections 2021-02-10T11:30:53.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.403 7fd6e2ef1700 1 --2- 172.21.2.101:0/2842108215 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6c0018730 0x7fd6c001abc0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:53.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.403 7fd6e2ef1700 1 --2- 172.21.2.101:0/2842108215 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6dc189d00 0x7fd6dc18a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:53.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.403 7fd6e2ef1700 1 --2- 172.21.2.101:0/2842108215 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6dc18a720 0x7fd6dc0b7be0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:53.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.403 7fd6e2ef1700 1 --2- 172.21.2.101:0/2842108215 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6dc0a0420 0x7fd6dc0a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:53.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.403 7fd6e2ef1700 1 -- 172.21.2.101:0/2842108215 >> 172.21.2.101:0/2842108215 conn(0x7fd6dc094e10 msgr2=0x7fd6dc0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:53.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.403 7fd6e2ef1700 1 -- 172.21.2.101:0/2842108215 shutdown_connections 2021-02-10T11:30:53.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.403 7fd6e2ef1700 1 -- 172.21.2.101:0/2842108215 wait complete. 2021-02-10T11:30:53.424 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839325 2021-02-10T11:30:53.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.832 7f51cad4f700 1 Processor -- start 2021-02-10T11:30:53.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.833 7f51cad4f700 1 -- start start 2021-02-10T11:30:53.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.833 7f51cad4f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f51c41b0470 0x7f51c41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:53.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.833 7f51cad4f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f51c40f20e0 0x7f51c40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:53.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.833 7f51cad4f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c40f0b50 con 0x7f51c40ef610 2021-02-10T11:30:53.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.833 7f51cad4f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c40f0820 con 0x7f51c41b0470 2021-02-10T11:30:53.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.833 7f51cad4f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c40f24e0 con 0x7f51c40f20e0 2021-02-10T11:30:53.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.833 7f51c3fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f51c40ef610 0x7f51c40efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50068/0 (socket says 172.21.2.101:50068) 2021-02-10T11:30:53.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.833 7f51c8b6b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f51c41b0470 0x7f51c41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:53.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.833 7f51c3fff700 1 -- 172.21.2.101:0/3942171426 learned_addr learned my addr 172.21.2.101:0/3942171426 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:53.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.833 7f51c936c700 1 --2- 172.21.2.101:0/3942171426 >> v2:172.21.2.108:3300/0 conn(0x7f51c40f20e0 0x7f51c40f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:53.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.833 7f51c37fe700 1 -- 172.21.2.101:0/3942171426 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1101445111 0 0) 0x7f51b4000d30 con 0x7f51c40ef610 2021-02-10T11:30:53.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.833 7f51c37fe700 1 -- 172.21.2.101:0/3942171426 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51b0002fe0 con 0x7f51c40ef610 2021-02-10T11:30:53.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.834 7f51c8b6b700 1 -- 172.21.2.101:0/3942171426 >> v2:172.21.2.108:3300/0 conn(0x7f51c40f20e0 msgr2=0x7f51c40f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:53.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.834 7f51c8b6b700 1 --2- 172.21.2.101:0/3942171426 >> v2:172.21.2.108:3300/0 conn(0x7f51c40f20e0 0x7f51c40f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:53.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.834 7f51c8b6b700 1 -- 172.21.2.101:0/3942171426 >> v1:172.21.2.101:6789/0 conn(0x7f51c40ef610 legacy=0x7f51c40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:53.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.834 7f51c8b6b700 1 -- 172.21.2.101:0/3942171426 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c40e52e0 con 0x7f51c41b0470 2021-02-10T11:30:53.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.834 7f51c8b6b700 1 --2- 172.21.2.101:0/3942171426 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51c41b0470 0x7f51c41b0870 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f51ac006650 tx=0x7f51ac009a80).ready entity=mon.0 client_cookie=2b28425b9398b084 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:53.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.834 7f51c37fe700 1 -- 172.21.2.101:0/3942171426 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f51ac0009b0 con 0x7f51c41b0470 2021-02-10T11:30:53.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.834 7f51c37fe700 1 -- 172.21.2.101:0/3942171426 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ac00c0c0 con 0x7f51c41b0470 2021-02-10T11:30:53.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.834 7f51c37fe700 1 -- 172.21.2.101:0/3942171426 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f51ac00c5e0 con 0x7f51c41b0470 2021-02-10T11:30:53.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.834 7f51cad4f700 1 -- 172.21.2.101:0/3942171426 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51c41b0470 msgr2=0x7f51c41b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:53.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.834 7f51cad4f700 1 --2- 172.21.2.101:0/3942171426 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51c41b0470 0x7f51c41b0870 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f51ac006650 tx=0x7f51ac009a80).stop 2021-02-10T11:30:53.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.834 7f51cad4f700 1 -- 172.21.2.101:0/3942171426 shutdown_connections 2021-02-10T11:30:53.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.834 7f51cad4f700 1 --2- 172.21.2.101:0/3942171426 >> v2:172.21.2.108:3300/0 conn(0x7f51c40f20e0 0x7f51c40f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:53.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.834 7f51cad4f700 1 --2- 172.21.2.101:0/3942171426 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51c41b0470 0x7f51c41b0870 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:53.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.834 7f51cad4f700 1 -- 172.21.2.101:0/3942171426 >> 172.21.2.101:0/3942171426 conn(0x7f51c40ce3e0 msgr2=0x7f51c40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:53.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.835 7f51cad4f700 1 -- 172.21.2.101:0/3942171426 shutdown_connections 2021-02-10T11:30:53.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.835 7f51cad4f700 1 -- 172.21.2.101:0/3942171426 wait complete. 2021-02-10T11:30:53.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.849 7f51cad4f700 1 Processor -- start 2021-02-10T11:30:53.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.849 7f51cad4f700 1 -- start start 2021-02-10T11:30:53.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.850 7f51cad4f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f51c40a0420 0x7f51c40a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:53.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.850 7f51cad4f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51c4189d00 0x7f51c418a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:53.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.850 7f51cad4f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51c418a720 0x7f51c40b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:53.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.850 7f51cad4f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c4189780 con 0x7f51c418a720 2021-02-10T11:30:53.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.850 7f51cad4f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c418ab20 con 0x7f51c40a0420 2021-02-10T11:30:53.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.850 7f51cad4f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c40a1140 con 0x7f51c4189d00 2021-02-10T11:30:53.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.850 7f51c936c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51c418a720 0x7f51c40b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:53.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.850 7f51c3fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51c4189d00 0x7f51c418a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:53.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.850 7f51c936c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51c418a720 0x7f51c40b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49580/0 (socket says 172.21.2.101:49580) 2021-02-10T11:30:53.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.850 7f51c936c700 1 -- 172.21.2.101:0/4251100535 learned_addr learned my addr 172.21.2.101:0/4251100535 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:53.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.850 7f51c8b6b700 1 --2- 172.21.2.101:0/4251100535 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f51c40a0420 0x7f51c40a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:53.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.850 7f51c936c700 1 -- 172.21.2.101:0/4251100535 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51c4189d00 msgr2=0x7f51c418a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:53.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.850 7f51c936c700 1 --2- 172.21.2.101:0/4251100535 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51c4189d00 0x7f51c418a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:53.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.850 7f51c936c700 1 -- 172.21.2.101:0/4251100535 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f51c40a0420 msgr2=0x7f51c40a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:53.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.850 7f51c936c700 1 --2- 172.21.2.101:0/4251100535 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f51c40a0420 0x7f51c40a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:53.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.850 7f51c936c700 1 -- 172.21.2.101:0/4251100535 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c40b8200 con 0x7f51c418a720 2021-02-10T11:30:53.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.851 7f51c936c700 1 --2- 172.21.2.101:0/4251100535 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51c418a720 0x7f51c40b7be0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7f51bc006f20 tx=0x7f51bc00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:53.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.851 7f51c17fa700 1 -- 172.21.2.101:0/4251100535 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f51bc000c10 con 0x7f51c418a720 2021-02-10T11:30:53.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.851 7f51c17fa700 1 -- 172.21.2.101:0/4251100535 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51bc00bd80 con 0x7f51c418a720 2021-02-10T11:30:53.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.851 7f51c17fa700 1 -- 172.21.2.101:0/4251100535 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f51bc0022b0 con 0x7f51c418a720 2021-02-10T11:30:53.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.851 7f51cad4f700 1 -- 172.21.2.101:0/4251100535 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c40b8200 con 0x7f51c418a720 2021-02-10T11:30:53.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.851 7f51cad4f700 1 -- 172.21.2.101:0/4251100535 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c40b8200 con 0x7f51c418a720 2021-02-10T11:30:53.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.851 7f51c17fa700 1 -- 172.21.2.101:0/4251100535 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f51bc005330 con 0x7f51c418a720 2021-02-10T11:30:53.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.851 7f51c17fa700 1 --2- 172.21.2.101:0/4251100535 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51a80186d0 0x7f51a801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:53.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.851 7f51c17fa700 1 -- 172.21.2.101:0/4251100535 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f51bc032650 con 0x7f51c418a720 2021-02-10T11:30:53.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.851 7f51c8b6b700 1 --2- 172.21.2.101:0/4251100535 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51a80186d0 0x7f51a801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:53.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.852 7f51c8b6b700 1 --2- 172.21.2.101:0/4251100535 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51a80186d0 0x7f51a801abd0 crc :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:53.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.882 7f51cad4f700 1 -- 172.21.2.101:0/4251100535 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c4105ae0 con 0x7f51c418a720 2021-02-10T11:30:53.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:53.885 7f51c17fa700 1 -- 172.21.2.101:0/4251100535 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f51bc006370 con 0x7f51c418a720 2021-02-10T11:30:54.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.011 7f51cad4f700 1 -- 172.21.2.101:0/4251100535 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839325", "target": ["mgr", ""]}) v1 -- 0x7f51c4189780 con 0x7f51a80186d0 2021-02-10T11:30:54.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.022 7f51c17fa700 1 -- 172.21.2.101:0/4251100535 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f51ac009180 con 0x7f51a80186d0 2021-02-10T11:30:54.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.028 7f51cad4f700 1 -- 172.21.2.101:0/4251100535 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51a80186d0 msgr2=0x7f51a801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:54.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.028 7f51cad4f700 1 --2- 172.21.2.101:0/4251100535 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51a80186d0 0x7f51a801abd0 crc :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:54.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.028 7f51cad4f700 1 -- 172.21.2.101:0/4251100535 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51c418a720 msgr2=0x7f51c40b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:54.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.028 7f51cad4f700 1 --2- 172.21.2.101:0/4251100535 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51c418a720 0x7f51c40b7be0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7f51bc006f20 tx=0x7f51bc00b990).stop 2021-02-10T11:30:54.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.028 7f51cad4f700 1 -- 172.21.2.101:0/4251100535 shutdown_connections 2021-02-10T11:30:54.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.028 7f51cad4f700 1 --2- 172.21.2.101:0/4251100535 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51a80186d0 0x7f51a801abd0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:54.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.028 7f51cad4f700 1 --2- 172.21.2.101:0/4251100535 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51c4189d00 0x7f51c418a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:54.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.028 7f51cad4f700 1 --2- 172.21.2.101:0/4251100535 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51c418a720 0x7f51c40b7be0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:54.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.028 7f51cad4f700 1 --2- 172.21.2.101:0/4251100535 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f51c40a0420 0x7f51c40a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:54.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.029 7f51cad4f700 1 -- 172.21.2.101:0/4251100535 >> 172.21.2.101:0/4251100535 conn(0x7f51c4094e10 msgr2=0x7f51c40a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:54.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.029 7f51cad4f700 1 -- 172.21.2.101:0/4251100535 shutdown_connections 2021-02-10T11:30:54.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.029 7f51cad4f700 1 -- 172.21.2.101:0/4251100535 wait complete. 2021-02-10T11:30:54.048 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839326 2021-02-10T11:30:54.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.454 7efe3ca91700 1 Processor -- start 2021-02-10T11:30:54.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.454 7efe3ca91700 1 -- start start 2021-02-10T11:30:54.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.455 7efe3ca91700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7efe381b0870 0x7efe380f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:54.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.455 7efe3ca91700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe381b0c70 con 0x7efe380efe00 2021-02-10T11:30:54.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.455 7efe3ca91700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe380f0b50 con 0x7efe381b0470 2021-02-10T11:30:54.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.455 7efe3ca91700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe380e5ea0 con 0x7efe381b0870 2021-02-10T11:30:54.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.455 7efe3705f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7efe380efe00 0x7efe380f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50084/0 (socket says 172.21.2.101:50084) 2021-02-10T11:30:54.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.455 7efe3705f700 1 -- 172.21.2.101:0/828576648 learned_addr learned my addr 172.21.2.101:0/828576648 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:54.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.455 7efe3605d700 1 --2- 172.21.2.101:0/828576648 >> v2:172.21.2.108:3300/0 conn(0x7efe381b0870 0x7efe380f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:54.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.456 7efe3585c700 1 -- 172.21.2.101:0/828576648 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2030255670 0 0) 0x7efe28001180 con 0x7efe380efe00 2021-02-10T11:30:54.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.456 7efe3585c700 1 -- 172.21.2.101:0/828576648 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe20002fe0 con 0x7efe380efe00 2021-02-10T11:30:54.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.456 7efe3585c700 1 -- 172.21.2.101:0/828576648 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2258257120 0 0) 0x7efe30000fe0 con 0x7efe381b0470 2021-02-10T11:30:54.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.456 7efe3585c700 1 -- 172.21.2.101:0/828576648 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe20002fe0 con 0x7efe381b0470 2021-02-10T11:30:54.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.456 7efe3605d700 1 -- 172.21.2.101:0/828576648 >> v1:172.21.2.108:6789/0 conn(0x7efe381b0470 legacy=0x7efe380ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:54.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.456 7efe3605d700 1 -- 172.21.2.101:0/828576648 >> v1:172.21.2.101:6789/0 conn(0x7efe380efe00 legacy=0x7efe380f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:54.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.456 7efe3605d700 1 -- 172.21.2.101:0/828576648 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe380e52e0 con 0x7efe381b0870 2021-02-10T11:30:54.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.456 7efe3605d700 1 --2- 172.21.2.101:0/828576648 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efe381b0870 0x7efe380f20e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7efe2c006650 tx=0x7efe2c009a80).ready entity=mon.1 client_cookie=3cecde90525d0910 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:54.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.456 7efe3585c700 1 -- 172.21.2.101:0/828576648 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efe2c0009b0 con 0x7efe381b0870 2021-02-10T11:30:54.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.456 7efe3585c700 1 -- 172.21.2.101:0/828576648 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c00c0c0 con 0x7efe381b0870 2021-02-10T11:30:54.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.456 7efe3585c700 1 -- 172.21.2.101:0/828576648 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efe2c00c520 con 0x7efe381b0870 2021-02-10T11:30:54.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.456 7efe3ca91700 1 -- 172.21.2.101:0/828576648 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efe381b0870 msgr2=0x7efe380f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:54.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.456 7efe3ca91700 1 --2- 172.21.2.101:0/828576648 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efe381b0870 0x7efe380f20e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7efe2c006650 tx=0x7efe2c009a80).stop 2021-02-10T11:30:54.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.457 7efe3ca91700 1 -- 172.21.2.101:0/828576648 shutdown_connections 2021-02-10T11:30:54.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.457 7efe3ca91700 1 --2- 172.21.2.101:0/828576648 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efe381b0870 0x7efe380f20e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:54.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.457 7efe3ca91700 1 -- 172.21.2.101:0/828576648 >> 172.21.2.101:0/828576648 conn(0x7efe380ce3e0 msgr2=0x7efe380e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:54.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.457 7efe3ca91700 1 -- 172.21.2.101:0/828576648 shutdown_connections 2021-02-10T11:30:54.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.457 7efe3ca91700 1 -- 172.21.2.101:0/828576648 wait complete. 2021-02-10T11:30:54.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.472 7efe3ca91700 1 Processor -- start 2021-02-10T11:30:54.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.472 7efe3ca91700 1 -- start start 2021-02-10T11:30:54.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.472 7efe3ca91700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efe380b3890 0x7efe380c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:54.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.472 7efe3ca91700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe380b3c90 0x7efe380c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.472 7efe3ca91700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efe380aa1f0 0x7efe380aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.472 7efe3ca91700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe380ab240 con 0x7efe380b3c90 2021-02-10T11:30:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.472 7efe3ca91700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe380ab7c0 con 0x7efe380aa1f0 2021-02-10T11:30:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.472 7efe3ca91700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe380ab9a0 con 0x7efe380b3890 2021-02-10T11:30:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.472 7efe3605d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe380b3c90 0x7efe380c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.472 7efe3605d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe380b3c90 0x7efe380c34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49594/0 (socket says 172.21.2.101:49594) 2021-02-10T11:30:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.472 7efe3605d700 1 -- 172.21.2.101:0/2558668596 learned_addr learned my addr 172.21.2.101:0/2558668596 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.472 7efe3605d700 1 -- 172.21.2.101:0/2558668596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efe380b3890 msgr2=0x7efe380c2ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:30:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.472 7efe3605d700 1 --2- 172.21.2.101:0/2558668596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efe380b3890 0x7efe380c2ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.472 7efe3605d700 1 -- 172.21.2.101:0/2558668596 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efe380aa1f0 msgr2=0x7efe380aa5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:30:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.472 7efe3605d700 1 --2- 172.21.2.101:0/2558668596 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efe380aa1f0 0x7efe380aa5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.472 7efe3605d700 1 -- 172.21.2.101:0/2558668596 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe380aac10 con 0x7efe380b3c90 2021-02-10T11:30:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.473 7efe3605d700 1 --2- 172.21.2.101:0/2558668596 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe380b3c90 0x7efe380c34f0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7efe2c009630 tx=0x7efe2c00b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:54.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.473 7efe277fe700 1 -- 172.21.2.101:0/2558668596 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efe2c0093a0 con 0x7efe380b3c90 2021-02-10T11:30:54.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.473 7efe277fe700 1 -- 172.21.2.101:0/2558668596 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c00bd70 con 0x7efe380b3c90 2021-02-10T11:30:54.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.473 7efe3ca91700 1 -- 172.21.2.101:0/2558668596 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe380aac10 con 0x7efe380b3c90 2021-02-10T11:30:54.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.473 7efe277fe700 1 -- 172.21.2.101:0/2558668596 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efe2c005560 con 0x7efe380b3c90 2021-02-10T11:30:54.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.473 7efe3ca91700 1 -- 172.21.2.101:0/2558668596 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe380aac10 con 0x7efe380b3c90 2021-02-10T11:30:54.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.473 7efe277fe700 1 -- 172.21.2.101:0/2558668596 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7efe2c00a000 con 0x7efe380b3c90 2021-02-10T11:30:54.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.473 7efe277fe700 1 --2- 172.21.2.101:0/2558668596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efe1c018730 0x7efe1c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:54.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.473 7efe277fe700 1 -- 172.21.2.101:0/2558668596 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7efe2c032620 con 0x7efe380b3c90 2021-02-10T11:30:54.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.473 7efe3685e700 1 --2- 172.21.2.101:0/2558668596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efe1c018730 0x7efe1c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:54.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.474 7efe3685e700 1 --2- 172.21.2.101:0/2558668596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efe1c018730 0x7efe1c01ac10 crc :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:54.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.504 7efe3ca91700 1 -- 172.21.2.101:0/2558668596 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe38105af0 con 0x7efe380b3c90 2021-02-10T11:30:54.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.507 7efe277fe700 1 -- 172.21.2.101:0/2558668596 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7efe2c01f4e0 con 0x7efe380b3c90 2021-02-10T11:30:54.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.631 7efe3ca91700 1 -- 172.21.2.101:0/2558668596 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839326", "target": ["mgr", ""]}) v1 -- 0x7efe380f0b50 con 0x7efe1c018730 2021-02-10T11:30:54.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.641 7efe277fe700 1 -- 172.21.2.101:0/2558668596 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7efe30006cf0 con 0x7efe1c018730 2021-02-10T11:30:54.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.648 7efe3ca91700 1 -- 172.21.2.101:0/2558668596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efe1c018730 msgr2=0x7efe1c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:54.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.648 7efe3ca91700 1 --2- 172.21.2.101:0/2558668596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efe1c018730 0x7efe1c01ac10 crc :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:54.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.648 7efe3ca91700 1 -- 172.21.2.101:0/2558668596 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe380b3c90 msgr2=0x7efe380c34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:54.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.648 7efe3ca91700 1 --2- 172.21.2.101:0/2558668596 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe380b3c90 0x7efe380c34f0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7efe2c009630 tx=0x7efe2c00b980).stop 2021-02-10T11:30:54.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.648 7efe3ca91700 1 -- 172.21.2.101:0/2558668596 shutdown_connections 2021-02-10T11:30:54.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.648 7efe3ca91700 1 --2- 172.21.2.101:0/2558668596 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efe380aa1f0 0x7efe380aa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:54.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.648 7efe3ca91700 1 --2- 172.21.2.101:0/2558668596 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe380b3c90 0x7efe380c34f0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:54.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.648 7efe3ca91700 1 --2- 172.21.2.101:0/2558668596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efe1c018730 0x7efe1c01ac10 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:54.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.648 7efe3ca91700 1 --2- 172.21.2.101:0/2558668596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efe380b3890 0x7efe380c2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:54.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.648 7efe3ca91700 1 -- 172.21.2.101:0/2558668596 >> 172.21.2.101:0/2558668596 conn(0x7efe380c40c0 msgr2=0x7efe380a3520 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:54.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.648 7efe3ca91700 1 -- 172.21.2.101:0/2558668596 shutdown_connections 2021-02-10T11:30:54.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:54.648 7efe3ca91700 1 -- 172.21.2.101:0/2558668596 wait complete. 2021-02-10T11:30:54.668 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839327 2021-02-10T11:30:55.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.075 7f1a1d962700 1 Processor -- start 2021-02-10T11:30:55.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.075 7f1a1d962700 1 -- start start 2021-02-10T11:30:55.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.075 7f1a1d962700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1a181b0470 0x7f1a181b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:55.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.076 7f1a1d962700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1a180f20e0 0x7f1a180f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:55.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.076 7f1a1d962700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a180f0b50 con 0x7f1a180ef610 2021-02-10T11:30:55.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.076 7f1a1d962700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a180f0820 con 0x7f1a180f20e0 2021-02-10T11:30:55.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.076 7f1a1d962700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a180f24e0 con 0x7f1a181b0470 2021-02-10T11:30:55.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.076 7f1a17d75700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1a180f20e0 0x7f1a180f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:55.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.076 7f1a17574700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1a181b0470 0x7f1a181b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:55.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.076 7f1a17574700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1a181b0470 0x7f1a181b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:60086/0 (socket says 172.21.2.101:60086) 2021-02-10T11:30:55.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.076 7f1a17d75700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1a180f20e0 0x7f1a180f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49600/0 (socket says 172.21.2.101:49600) 2021-02-10T11:30:55.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.076 7f1a17d75700 1 -- 172.21.2.101:0/2962841917 learned_addr learned my addr 172.21.2.101:0/2962841917 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:55.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.076 7f1a16d73700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f1a180ef610 0x7f1a180efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:43504/0 (socket says 172.21.2.101:43504) 2021-02-10T11:30:55.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.076 7f1a17d75700 1 -- 172.21.2.101:0/2962841917 >> v2:172.21.2.108:3300/0 conn(0x7f1a181b0470 msgr2=0x7f1a181b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:55.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.076 7f1a17d75700 1 --2- 172.21.2.101:0/2962841917 >> v2:172.21.2.108:3300/0 conn(0x7f1a181b0470 0x7f1a181b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:55.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.076 7f1a17d75700 1 -- 172.21.2.101:0/2962841917 >> v1:172.21.2.108:6789/0 conn(0x7f1a180ef610 legacy=0x7f1a180efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:55.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.076 7f1a17d75700 1 -- 172.21.2.101:0/2962841917 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a180e52e0 con 0x7f1a180f20e0 2021-02-10T11:30:55.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.077 7f1a17d75700 1 --2- 172.21.2.101:0/2962841917 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a180f20e0 0x7f1a180f0200 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7f1a10006f00 tx=0x7f1a10009f10).ready entity=mon.0 client_cookie=5f1c2755e645c904 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:55.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.077 7f1a16572700 1 -- 172.21.2.101:0/2962841917 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1a1000b3b0 con 0x7f1a180f20e0 2021-02-10T11:30:55.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.077 7f1a16572700 1 -- 172.21.2.101:0/2962841917 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1000b730 con 0x7f1a180f20e0 2021-02-10T11:30:55.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.077 7f1a16572700 1 -- 172.21.2.101:0/2962841917 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1a1000bc10 con 0x7f1a180f20e0 2021-02-10T11:30:55.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.077 7f1a1d962700 1 -- 172.21.2.101:0/2962841917 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a180f20e0 msgr2=0x7f1a180f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:55.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.077 7f1a1d962700 1 --2- 172.21.2.101:0/2962841917 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a180f20e0 0x7f1a180f0200 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7f1a10006f00 tx=0x7f1a10009f10).stop 2021-02-10T11:30:55.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.077 7f1a1d962700 1 -- 172.21.2.101:0/2962841917 shutdown_connections 2021-02-10T11:30:55.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.077 7f1a1d962700 1 --2- 172.21.2.101:0/2962841917 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a180f20e0 0x7f1a180f0200 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:55.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.077 7f1a1d962700 1 --2- 172.21.2.101:0/2962841917 >> v2:172.21.2.108:3300/0 conn(0x7f1a181b0470 0x7f1a181b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:55.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.077 7f1a1d962700 1 -- 172.21.2.101:0/2962841917 >> 172.21.2.101:0/2962841917 conn(0x7f1a180ce3e0 msgr2=0x7f1a180e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:55.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.078 7f1a1d962700 1 -- 172.21.2.101:0/2962841917 shutdown_connections 2021-02-10T11:30:55.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.078 7f1a1d962700 1 -- 172.21.2.101:0/2962841917 wait complete. 2021-02-10T11:30:55.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.092 7f1a1d962700 1 Processor -- start 2021-02-10T11:30:55.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.092 7f1a1d962700 1 -- start start 2021-02-10T11:30:55.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.093 7f1a1d962700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a180a0420 0x7f1a180a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:55.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.093 7f1a1d962700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a18189d00 0x7f1a1818a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:55.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.093 7f1a1d962700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a1818a720 0x7f1a180b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:55.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.093 7f1a1d962700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a18189800 con 0x7f1a1818a720 2021-02-10T11:30:55.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.093 7f1a1d962700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1818ab20 con 0x7f1a180a0420 2021-02-10T11:30:55.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.093 7f1a1d962700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a180a1140 con 0x7f1a18189d00 2021-02-10T11:30:55.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.093 7f1a17d75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a1818a720 0x7f1a180b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:55.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.093 7f1a17d75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a1818a720 0x7f1a180b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49602/0 (socket says 172.21.2.101:49602) 2021-02-10T11:30:55.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.093 7f1a17d75700 1 -- 172.21.2.101:0/516407611 learned_addr learned my addr 172.21.2.101:0/516407611 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:55.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.093 7f1a17574700 1 --2- 172.21.2.101:0/516407611 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a180a0420 0x7f1a180a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:55.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.093 7f1a16d73700 1 --2- 172.21.2.101:0/516407611 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a18189d00 0x7f1a1818a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:55.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.093 7f1a17d75700 1 -- 172.21.2.101:0/516407611 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a18189d00 msgr2=0x7f1a1818a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:55.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.093 7f1a17d75700 1 --2- 172.21.2.101:0/516407611 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a18189d00 0x7f1a1818a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:55.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.093 7f1a17d75700 1 -- 172.21.2.101:0/516407611 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a180a0420 msgr2=0x7f1a180a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:55.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.093 7f1a17d75700 1 --2- 172.21.2.101:0/516407611 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a180a0420 0x7f1a180a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:55.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.093 7f1a17d75700 1 -- 172.21.2.101:0/516407611 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a180b8200 con 0x7f1a1818a720 2021-02-10T11:30:55.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.093 7f1a17d75700 1 --2- 172.21.2.101:0/516407611 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a1818a720 0x7f1a180b7be0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0x7f1a10009390 tx=0x7f1a1000bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:55.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.094 7f1a0ffff700 1 -- 172.21.2.101:0/516407611 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1a10003030 con 0x7f1a1818a720 2021-02-10T11:30:55.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.094 7f1a0ffff700 1 -- 172.21.2.101:0/516407611 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a10006c80 con 0x7f1a1818a720 2021-02-10T11:30:55.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.094 7f1a0ffff700 1 -- 172.21.2.101:0/516407611 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1a10001480 con 0x7f1a1818a720 2021-02-10T11:30:55.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.094 7f1a1d962700 1 -- 172.21.2.101:0/516407611 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a180b8200 con 0x7f1a1818a720 2021-02-10T11:30:55.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.094 7f1a1d962700 1 -- 172.21.2.101:0/516407611 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a180b8790 con 0x7f1a1818a720 2021-02-10T11:30:55.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.094 7f1a0ffff700 1 -- 172.21.2.101:0/516407611 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1a10001270 con 0x7f1a1818a720 2021-02-10T11:30:55.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.094 7f1a0ffff700 1 --2- 172.21.2.101:0/516407611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19fc018730 0x7f19fc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:55.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.094 7f1a0ffff700 1 -- 172.21.2.101:0/516407611 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1a100329a0 con 0x7f1a1818a720 2021-02-10T11:30:55.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.094 7f1a17574700 1 --2- 172.21.2.101:0/516407611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19fc018730 0x7f19fc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:55.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.095 7f1a17574700 1 --2- 172.21.2.101:0/516407611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19fc018730 0x7f19fc01abc0 crc :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:55.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.124 7f1a1d962700 1 -- 172.21.2.101:0/516407611 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a1818ab20 con 0x7f1a1818a720 2021-02-10T11:30:55.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.126 7f1a0ffff700 1 -- 172.21.2.101:0/516407611 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1a10003020 con 0x7f1a1818a720 2021-02-10T11:30:55.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.259 7f1a1d962700 1 -- 172.21.2.101:0/516407611 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839327", "target": ["mgr", ""]}) v1 -- 0x7f1a1818ab20 con 0x7f19fc018730 2021-02-10T11:30:55.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.269 7f1a0ffff700 1 -- 172.21.2.101:0/516407611 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f1a00006a70 con 0x7f19fc018730 2021-02-10T11:30:55.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.276 7f1a1d962700 1 -- 172.21.2.101:0/516407611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19fc018730 msgr2=0x7f19fc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:55.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.276 7f1a1d962700 1 --2- 172.21.2.101:0/516407611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19fc018730 0x7f19fc01abc0 crc :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:55.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.276 7f1a1d962700 1 -- 172.21.2.101:0/516407611 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a1818a720 msgr2=0x7f1a180b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:55.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.276 7f1a1d962700 1 --2- 172.21.2.101:0/516407611 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a1818a720 0x7f1a180b7be0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0x7f1a10009390 tx=0x7f1a1000bd40).stop 2021-02-10T11:30:55.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.276 7f1a1d962700 1 -- 172.21.2.101:0/516407611 shutdown_connections 2021-02-10T11:30:55.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.276 7f1a1d962700 1 --2- 172.21.2.101:0/516407611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19fc018730 0x7f19fc01abc0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:55.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.276 7f1a1d962700 1 --2- 172.21.2.101:0/516407611 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a18189d00 0x7f1a1818a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:55.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.276 7f1a1d962700 1 --2- 172.21.2.101:0/516407611 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a1818a720 0x7f1a180b7be0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:55.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.276 7f1a1d962700 1 --2- 172.21.2.101:0/516407611 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a180a0420 0x7f1a180a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:55.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.276 7f1a1d962700 1 -- 172.21.2.101:0/516407611 >> 172.21.2.101:0/516407611 conn(0x7f1a18094e10 msgr2=0x7f1a180a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:55.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.277 7f1a1d962700 1 -- 172.21.2.101:0/516407611 shutdown_connections 2021-02-10T11:30:55.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.277 7f1a1d962700 1 -- 172.21.2.101:0/516407611 wait complete. 2021-02-10T11:30:55.296 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839328 2021-02-10T11:30:55.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.699 7f95611d9700 1 Processor -- start 2021-02-10T11:30:55.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.699 7f95611d9700 1 -- start start 2021-02-10T11:30:55.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.700 7f95611d9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f955c1b0470 0x7f955c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:55.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.700 7f95611d9700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f955c0f20e0 0x7f955c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:55.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.700 7f95611d9700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955c0f0b50 con 0x7f955c0ef610 2021-02-10T11:30:55.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.700 7f95611d9700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f955c0f0820 con 0x7f955c0f20e0 2021-02-10T11:30:55.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.700 7f95611d9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955c0f24e0 con 0x7f955c1b0470 2021-02-10T11:30:55.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.700 7f955b59e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f955c0f20e0 0x7f955c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:55.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.700 7f955ad9d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f955c1b0470 0x7f955c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:55.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.700 7f955b59e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f955c0f20e0 0x7f955c0f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49614/0 (socket says 172.21.2.101:49614) 2021-02-10T11:30:55.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.700 7f955ad9d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f955c1b0470 0x7f955c1b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:42342/0 (socket says 172.21.2.101:42342) 2021-02-10T11:30:55.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.700 7f955b59e700 1 -- 172.21.2.101:0/579275121 learned_addr learned my addr 172.21.2.101:0/579275121 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:55.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.700 7f955b59e700 1 -- 172.21.2.101:0/579275121 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f955c1b0470 msgr2=0x7f955c1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:55.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.700 7f955b59e700 1 --2- 172.21.2.101:0/579275121 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f955c1b0470 0x7f955c1b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:55.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.700 7f955b59e700 1 -- 172.21.2.101:0/579275121 >> v1:172.21.2.108:6789/0 conn(0x7f955c0ef610 legacy=0x7f955c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:55.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.701 7f955b59e700 1 -- 172.21.2.101:0/579275121 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c0e52e0 con 0x7f955c0f20e0 2021-02-10T11:30:55.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.701 7f955b59e700 1 --2- 172.21.2.101:0/579275121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f955c0f20e0 0x7f955c0f0200 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7f9554006f00 tx=0x7f9554009f10).ready entity=mon.0 client_cookie=834f2c4f7894604 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:55.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.701 7f9559d9b700 1 -- 172.21.2.101:0/579275121 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f955400b3b0 con 0x7f955c0f20e0 2021-02-10T11:30:55.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.701 7f9559d9b700 1 -- 172.21.2.101:0/579275121 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955400b730 con 0x7f955c0f20e0 2021-02-10T11:30:55.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.701 7f9559d9b700 1 -- 172.21.2.101:0/579275121 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f955400bc10 con 0x7f955c0f20e0 2021-02-10T11:30:55.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.701 7f95611d9700 1 -- 172.21.2.101:0/579275121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f955c0f20e0 msgr2=0x7f955c0f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:55.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.701 7f95611d9700 1 --2- 172.21.2.101:0/579275121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f955c0f20e0 0x7f955c0f0200 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7f9554006f00 tx=0x7f9554009f10).stop 2021-02-10T11:30:55.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.702 7f95611d9700 1 -- 172.21.2.101:0/579275121 shutdown_connections 2021-02-10T11:30:55.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.702 7f95611d9700 1 --2- 172.21.2.101:0/579275121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f955c0f20e0 0x7f955c0f0200 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:55.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.702 7f95611d9700 1 --2- 172.21.2.101:0/579275121 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f955c1b0470 0x7f955c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:55.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.702 7f95611d9700 1 -- 172.21.2.101:0/579275121 >> 172.21.2.101:0/579275121 conn(0x7f955c0ce3e0 msgr2=0x7f955c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:55.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.702 7f95611d9700 1 -- 172.21.2.101:0/579275121 shutdown_connections 2021-02-10T11:30:55.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.702 7f95611d9700 1 -- 172.21.2.101:0/579275121 wait complete. 2021-02-10T11:30:55.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.717 7f95611d9700 1 Processor -- start 2021-02-10T11:30:55.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.717 7f95611d9700 1 -- start start 2021-02-10T11:30:55.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.717 7f95611d9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f955c0b7c20 0x7f955c0b8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:55.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.717 7f95611d9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f955c0b8a30 0x7f955c0c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:55.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.717 7f95611d9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f955c0c25a0 0x7f955c0c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:55.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.717 7f95611d9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c0c1a80 con 0x7f955c0b8a30 2021-02-10T11:30:55.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.717 7f95611d9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c0b8020 con 0x7f955c0c25a0 2021-02-10T11:30:55.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.717 7f95611d9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955c09b9c0 con 0x7f955c0b7c20 2021-02-10T11:30:55.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.717 7f955a59c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f955c0b8a30 0x7f955c0c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:55.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.717 7f955a59c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f955c0b8a30 0x7f955c0c1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49616/0 (socket says 172.21.2.101:49616) 2021-02-10T11:30:55.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.717 7f955a59c700 1 -- 172.21.2.101:0/3526863847 learned_addr learned my addr 172.21.2.101:0/3526863847 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:55.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.718 7f955a59c700 1 -- 172.21.2.101:0/3526863847 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f955c0b7c20 msgr2=0x7f955c0b8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:55.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.718 7f955a59c700 1 --2- 172.21.2.101:0/3526863847 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f955c0b7c20 0x7f955c0b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:55.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.718 7f955a59c700 1 -- 172.21.2.101:0/3526863847 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f955c0c25a0 msgr2=0x7f955c0c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:55.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.718 7f955a59c700 1 --2- 172.21.2.101:0/3526863847 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f955c0c25a0 0x7f955c0c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:55.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.718 7f955a59c700 1 -- 172.21.2.101:0/3526863847 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c0b8e30 con 0x7f955c0b8a30 2021-02-10T11:30:55.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.718 7f955a59c700 1 --2- 172.21.2.101:0/3526863847 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f955c0b8a30 0x7f955c0c1f80 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x7f954c006ed0 tx=0x7f954c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:55.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.718 7f95537fe700 1 -- 172.21.2.101:0/3526863847 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f954c006240 con 0x7f955c0b8a30 2021-02-10T11:30:55.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.718 7f95537fe700 1 -- 172.21.2.101:0/3526863847 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c0009b0 con 0x7f955c0b8a30 2021-02-10T11:30:55.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.718 7f95537fe700 1 -- 172.21.2.101:0/3526863847 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f954c005730 con 0x7f955c0b8a30 2021-02-10T11:30:55.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.718 7f95611d9700 1 -- 172.21.2.101:0/3526863847 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f955c0b8e30 con 0x7f955c0b8a30 2021-02-10T11:30:55.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.718 7f95611d9700 1 -- 172.21.2.101:0/3526863847 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955c0b8e30 con 0x7f955c0b8a30 2021-02-10T11:30:55.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.719 7f95537fe700 1 -- 172.21.2.101:0/3526863847 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f954c006030 con 0x7f955c0b8a30 2021-02-10T11:30:55.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.719 7f95537fe700 1 --2- 172.21.2.101:0/3526863847 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95400186c0 0x7f954001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:55.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.719 7f95537fe700 1 -- 172.21.2.101:0/3526863847 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f954c0327c0 con 0x7f955c0b8a30 2021-02-10T11:30:55.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.719 7f955ad9d700 1 --2- 172.21.2.101:0/3526863847 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95400186c0 0x7f954001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:55.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.719 7f955ad9d700 1 --2- 172.21.2.101:0/3526863847 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95400186c0 0x7f954001abc0 crc :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:55.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.749 7f95611d9700 1 -- 172.21.2.101:0/3526863847 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f955c105b30 con 0x7f955c0b8a30 2021-02-10T11:30:55.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.751 7f95537fe700 1 -- 172.21.2.101:0/3526863847 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f954c000de0 con 0x7f955c0b8a30 2021-02-10T11:30:55.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.877 7f95611d9700 1 -- 172.21.2.101:0/3526863847 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839328", "target": ["mgr", ""]}) v1 -- 0x7f955c105b30 con 0x7f95400186c0 2021-02-10T11:30:55.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.887 7f95537fe700 1 -- 172.21.2.101:0/3526863847 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f9544006e70 con 0x7f95400186c0 2021-02-10T11:30:55.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.894 7f95611d9700 1 -- 172.21.2.101:0/3526863847 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95400186c0 msgr2=0x7f954001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:55.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.894 7f95611d9700 1 --2- 172.21.2.101:0/3526863847 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95400186c0 0x7f954001abc0 crc :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:55.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.894 7f95611d9700 1 -- 172.21.2.101:0/3526863847 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f955c0b8a30 msgr2=0x7f955c0c1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:55.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.894 7f95611d9700 1 --2- 172.21.2.101:0/3526863847 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f955c0b8a30 0x7f955c0c1f80 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x7f954c006ed0 tx=0x7f954c00b960).stop 2021-02-10T11:30:55.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.894 7f95611d9700 1 -- 172.21.2.101:0/3526863847 shutdown_connections 2021-02-10T11:30:55.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.894 7f95611d9700 1 --2- 172.21.2.101:0/3526863847 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f955c0c25a0 0x7f955c0c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:55.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.894 7f95611d9700 1 --2- 172.21.2.101:0/3526863847 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f955c0b8a30 0x7f955c0c1f80 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:55.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.894 7f95611d9700 1 --2- 172.21.2.101:0/3526863847 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95400186c0 0x7f954001abc0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:55.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.894 7f95611d9700 1 --2- 172.21.2.101:0/3526863847 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f955c0b7c20 0x7f955c0b8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:55.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.894 7f95611d9700 1 -- 172.21.2.101:0/3526863847 >> 172.21.2.101:0/3526863847 conn(0x7f955c094e10 msgr2=0x7f955c0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:55.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.894 7f95611d9700 1 -- 172.21.2.101:0/3526863847 shutdown_connections 2021-02-10T11:30:55.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:55.895 7f95611d9700 1 -- 172.21.2.101:0/3526863847 wait complete. 2021-02-10T11:30:55.914 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839329 2021-02-10T11:30:56.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.318 7ff306adb700 1 Processor -- start 2021-02-10T11:30:56.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.318 7ff306adb700 1 -- start start 2021-02-10T11:30:56.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.318 7ff306adb700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff3001b0870 0x7ff3000f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:56.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.318 7ff306adb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3000efe00 0x7ff3000f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:56.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.318 7ff306adb700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3001b0c70 con 0x7ff3001b0470 2021-02-10T11:30:56.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.318 7ff306adb700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3000f0b50 con 0x7ff3001b0870 2021-02-10T11:30:56.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.318 7ff306adb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3000f0820 con 0x7ff3000efe00 2021-02-10T11:30:56.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.319 7ff3048f7700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff3001b0470 0x7ff3000ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50118/0 (socket says 172.21.2.101:50118) 2021-02-10T11:30:56.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.319 7ff3048f7700 1 -- 172.21.2.101:0/266963368 learned_addr learned my addr 172.21.2.101:0/266963368 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:56.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.319 7ff3050f8700 1 --2- 172.21.2.101:0/266963368 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3000efe00 0x7ff3000f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:56.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.319 7ff2fffff700 1 --2- 172.21.2.101:0/266963368 >> v2:172.21.2.108:3300/0 conn(0x7ff3001b0870 0x7ff3000f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:56.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.319 7ff2ff7fe700 1 -- 172.21.2.101:0/266963368 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4039267536 0 0) 0x7ff2e8001180 con 0x7ff3001b0470 2021-02-10T11:30:56.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.319 7ff2ff7fe700 1 -- 172.21.2.101:0/266963368 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2ec002fe0 con 0x7ff3001b0470 2021-02-10T11:30:56.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.319 7ff2fffff700 1 -- 172.21.2.101:0/266963368 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3000efe00 msgr2=0x7ff3000f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:56.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.319 7ff2fffff700 1 --2- 172.21.2.101:0/266963368 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3000efe00 0x7ff3000f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:56.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.319 7ff2fffff700 1 -- 172.21.2.101:0/266963368 >> v1:172.21.2.101:6789/0 conn(0x7ff3001b0470 legacy=0x7ff3000ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:56.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.319 7ff2fffff700 1 -- 172.21.2.101:0/266963368 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3000e52e0 con 0x7ff3001b0870 2021-02-10T11:30:56.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.319 7ff2fffff700 1 --2- 172.21.2.101:0/266963368 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff3001b0870 0x7ff3000f20e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7ff2f00069d0 tx=0x7ff2f0009e40).ready entity=mon.1 client_cookie=4ac8ef40f084bf19 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:56.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.319 7ff2ff7fe700 1 -- 172.21.2.101:0/266963368 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff2f000b510 con 0x7ff3001b0870 2021-02-10T11:30:56.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.319 7ff2ff7fe700 1 -- 172.21.2.101:0/266963368 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f000b890 con 0x7ff3001b0870 2021-02-10T11:30:56.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.319 7ff2ff7fe700 1 -- 172.21.2.101:0/266963368 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff2f000bdb0 con 0x7ff3001b0870 2021-02-10T11:30:56.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.320 7ff306adb700 1 -- 172.21.2.101:0/266963368 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff3001b0870 msgr2=0x7ff3000f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:56.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.320 7ff306adb700 1 --2- 172.21.2.101:0/266963368 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff3001b0870 0x7ff3000f20e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7ff2f00069d0 tx=0x7ff2f0009e40).stop 2021-02-10T11:30:56.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.320 7ff306adb700 1 -- 172.21.2.101:0/266963368 shutdown_connections 2021-02-10T11:30:56.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.320 7ff306adb700 1 --2- 172.21.2.101:0/266963368 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3000efe00 0x7ff3000f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:56.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.320 7ff306adb700 1 --2- 172.21.2.101:0/266963368 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff3001b0870 0x7ff3000f20e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:56.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.320 7ff306adb700 1 -- 172.21.2.101:0/266963368 >> 172.21.2.101:0/266963368 conn(0x7ff3000ce3e0 msgr2=0x7ff3000e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:56.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.320 7ff306adb700 1 -- 172.21.2.101:0/266963368 shutdown_connections 2021-02-10T11:30:56.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.320 7ff306adb700 1 -- 172.21.2.101:0/266963368 wait complete. 2021-02-10T11:30:56.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.335 7ff306adb700 1 Processor -- start 2021-02-10T11:30:56.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.335 7ff306adb700 1 -- start start 2021-02-10T11:30:56.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.336 7ff306adb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3000a4840 0x7ff30010a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:56.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.336 7ff306adb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff30008d6c0 0x7ff3000a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:56.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.336 7ff306adb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff3000a5ab0 0x7ff3000a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:56.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.336 7ff306adb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30010a7f0 con 0x7ff30008d6c0 2021-02-10T11:30:56.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.336 7ff306adb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3000a4c40 con 0x7ff3000a5ab0 2021-02-10T11:30:56.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.336 7ff306adb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3001906b0 con 0x7ff3000a4840 2021-02-10T11:30:56.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.336 7ff2fffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff30008d6c0 0x7ff3000a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:56.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.336 7ff2fffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff30008d6c0 0x7ff3000a5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49632/0 (socket says 172.21.2.101:49632) 2021-02-10T11:30:56.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.336 7ff2fffff700 1 -- 172.21.2.101:0/3437835657 learned_addr learned my addr 172.21.2.101:0/3437835657 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:56.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.336 7ff3048f7700 1 --2- 172.21.2.101:0/3437835657 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3000a4840 0x7ff30010a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:56.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.336 7ff3050f8700 1 --2- 172.21.2.101:0/3437835657 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff3000a5ab0 0x7ff3000a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:56.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.336 7ff2fffff700 1 -- 172.21.2.101:0/3437835657 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3000a4840 msgr2=0x7ff30010a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:56.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.336 7ff2fffff700 1 --2- 172.21.2.101:0/3437835657 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3000a4840 0x7ff30010a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:56.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.336 7ff2fffff700 1 -- 172.21.2.101:0/3437835657 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff3000a5ab0 msgr2=0x7ff3000a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:56.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.336 7ff2fffff700 1 --2- 172.21.2.101:0/3437835657 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff3000a5ab0 0x7ff3000a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:56.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.336 7ff2fffff700 1 -- 172.21.2.101:0/3437835657 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff300190b00 con 0x7ff30008d6c0 2021-02-10T11:30:56.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.336 7ff2fffff700 1 --2- 172.21.2.101:0/3437835657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff30008d6c0 0x7ff3000a5490 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7ff2f0009f40 tx=0x7ff2f000bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:56.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.336 7ff2fd7fa700 1 -- 172.21.2.101:0/3437835657 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff2f000c2f0 con 0x7ff30008d6c0 2021-02-10T11:30:56.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.336 7ff2fd7fa700 1 -- 172.21.2.101:0/3437835657 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f000c5f0 con 0x7ff30008d6c0 2021-02-10T11:30:56.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.336 7ff306adb700 1 -- 172.21.2.101:0/3437835657 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff300190b00 con 0x7ff30008d6c0 2021-02-10T11:30:56.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.336 7ff2fd7fa700 1 -- 172.21.2.101:0/3437835657 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff2f000cad0 con 0x7ff30008d6c0 2021-02-10T11:30:56.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.336 7ff306adb700 1 -- 172.21.2.101:0/3437835657 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff300190b00 con 0x7ff30008d6c0 2021-02-10T11:30:56.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.337 7ff2fd7fa700 1 -- 172.21.2.101:0/3437835657 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff2f000c2f0 con 0x7ff30008d6c0 2021-02-10T11:30:56.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.337 7ff2fd7fa700 1 --2- 172.21.2.101:0/3437835657 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2e4018730 0x7ff2e401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:56.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.337 7ff2fd7fa700 1 -- 172.21.2.101:0/3437835657 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff2f00329c0 con 0x7ff30008d6c0 2021-02-10T11:30:56.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.337 7ff3048f7700 1 --2- 172.21.2.101:0/3437835657 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2e4018730 0x7ff2e401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:56.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.337 7ff3048f7700 1 --2- 172.21.2.101:0/3437835657 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2e4018730 0x7ff2e401ac10 crc :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:56.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.367 7ff306adb700 1 -- 172.21.2.101:0/3437835657 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3000a4f60 con 0x7ff30008d6c0 2021-02-10T11:30:56.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.370 7ff2fd7fa700 1 -- 172.21.2.101:0/3437835657 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff2f000be10 con 0x7ff30008d6c0 2021-02-10T11:30:56.436 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:30:56.439 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:30:56.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.499 7ff306adb700 1 -- 172.21.2.101:0/3437835657 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839329", "target": ["mgr", ""]}) v1 -- 0x7ff3000a4c40 con 0x7ff2e4018730 2021-02-10T11:30:56.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.510 7ff2fd7fa700 1 -- 172.21.2.101:0/3437835657 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7ff2e8007ce0 con 0x7ff2e4018730 2021-02-10T11:30:56.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.516 7ff306adb700 1 -- 172.21.2.101:0/3437835657 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2e4018730 msgr2=0x7ff2e401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:56.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.516 7ff306adb700 1 --2- 172.21.2.101:0/3437835657 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2e4018730 0x7ff2e401ac10 crc :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:56.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.516 7ff306adb700 1 -- 172.21.2.101:0/3437835657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff30008d6c0 msgr2=0x7ff3000a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:56.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.516 7ff306adb700 1 --2- 172.21.2.101:0/3437835657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff30008d6c0 0x7ff3000a5490 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7ff2f0009f40 tx=0x7ff2f000bcd0).stop 2021-02-10T11:30:56.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.516 7ff306adb700 1 -- 172.21.2.101:0/3437835657 shutdown_connections 2021-02-10T11:30:56.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.516 7ff306adb700 1 --2- 172.21.2.101:0/3437835657 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff3000a5ab0 0x7ff3000a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:56.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.516 7ff306adb700 1 --2- 172.21.2.101:0/3437835657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff30008d6c0 0x7ff3000a5490 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:56.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.516 7ff306adb700 1 --2- 172.21.2.101:0/3437835657 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2e4018730 0x7ff2e401ac10 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:56.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.516 7ff306adb700 1 --2- 172.21.2.101:0/3437835657 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3000a4840 0x7ff30010a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:56.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.516 7ff306adb700 1 -- 172.21.2.101:0/3437835657 >> 172.21.2.101:0/3437835657 conn(0x7ff3000bbf60 msgr2=0x7ff300105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:56.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.516 7ff306adb700 1 -- 172.21.2.101:0/3437835657 shutdown_connections 2021-02-10T11:30:56.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.516 7ff306adb700 1 -- 172.21.2.101:0/3437835657 wait complete. 2021-02-10T11:30:56.536 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839330 2021-02-10T11:30:56.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.840 7fee5215d700 1 Processor -- start 2021-02-10T11:30:56.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.840 7fee5215d700 1 -- start start 2021-02-10T11:30:56.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.840 7fee5215d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fee4c1b0870 0x7fee4c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:56.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.840 7fee5215d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fee4c0efe00 0x7fee4c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:56.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.840 7fee5215d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee4c1b0c70 con 0x7fee4c1b0470 2021-02-10T11:30:56.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.840 7fee5215d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee4c0f0b50 con 0x7fee4c1b0870 2021-02-10T11:30:56.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.840 7fee5215d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c0f0820 con 0x7fee4c0efe00 2021-02-10T11:30:56.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.841 7fee4affd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fee4c1b0870 0x7fee4c0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:56.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.841 7fee4b7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fee4c1b0470 0x7fee4c0ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50132/0 (socket says 172.21.2.101:50132) 2021-02-10T11:30:56.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.841 7fee4b7fe700 1 -- 172.21.2.101:0/2922539381 learned_addr learned my addr 172.21.2.101:0/2922539381 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:56.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.841 7fee4affd700 1 -- 172.21.2.101:0/2922539381 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fee4c0efe00 msgr2=0x7fee4c0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:56.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.841 7fee4affd700 1 --2- 172.21.2.101:0/2922539381 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fee4c0efe00 0x7fee4c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:56.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.841 7fee4affd700 1 -- 172.21.2.101:0/2922539381 >> v1:172.21.2.101:6789/0 conn(0x7fee4c1b0470 legacy=0x7fee4c0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:56.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.841 7fee4affd700 1 -- 172.21.2.101:0/2922539381 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c0e52e0 con 0x7fee4c1b0870 2021-02-10T11:30:56.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.841 7fee4affd700 1 --2- 172.21.2.101:0/2922539381 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fee4c1b0870 0x7fee4c0f20e0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0x7fee34006650 tx=0x7fee34009a80).ready entity=mon.0 client_cookie=92cd42abf875abd8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:56.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.841 7fee4a7fc700 1 -- 172.21.2.101:0/2922539381 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fee3400c0c0 con 0x7fee4c1b0870 2021-02-10T11:30:56.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.841 7fee4a7fc700 1 -- 172.21.2.101:0/2922539381 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3400c340 con 0x7fee4c1b0870 2021-02-10T11:30:56.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.841 7fee4a7fc700 1 -- 172.21.2.101:0/2922539381 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fee3400c860 con 0x7fee4c1b0870 2021-02-10T11:30:56.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.841 7fee5215d700 1 -- 172.21.2.101:0/2922539381 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fee4c1b0870 msgr2=0x7fee4c0f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:56.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.841 7fee5215d700 1 --2- 172.21.2.101:0/2922539381 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fee4c1b0870 0x7fee4c0f20e0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0x7fee34006650 tx=0x7fee34009a80).stop 2021-02-10T11:30:56.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.842 7fee5215d700 1 -- 172.21.2.101:0/2922539381 shutdown_connections 2021-02-10T11:30:56.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.842 7fee5215d700 1 --2- 172.21.2.101:0/2922539381 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fee4c0efe00 0x7fee4c0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:56.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.842 7fee5215d700 1 --2- 172.21.2.101:0/2922539381 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fee4c1b0870 0x7fee4c0f20e0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:56.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.842 7fee5215d700 1 -- 172.21.2.101:0/2922539381 >> 172.21.2.101:0/2922539381 conn(0x7fee4c0ce3e0 msgr2=0x7fee4c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:56.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.842 7fee5215d700 1 -- 172.21.2.101:0/2922539381 shutdown_connections 2021-02-10T11:30:56.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.842 7fee5215d700 1 -- 172.21.2.101:0/2922539381 wait complete. 2021-02-10T11:30:56.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.856 7fee5215d700 1 Processor -- start 2021-02-10T11:30:56.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.857 7fee5215d700 1 -- start start 2021-02-10T11:30:56.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.857 7fee5215d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fee4c094fb0 0x7fee4c09e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:56.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.857 7fee5215d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fee4c0c2be0 0x7fee4c0c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:56.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.857 7fee5215d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fee4c0c3600 0x7fee4c0a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:56.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.857 7fee5215d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c09ebf0 con 0x7fee4c0c2be0 2021-02-10T11:30:56.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.857 7fee5215d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c0aa2a0 con 0x7fee4c0c3600 2021-02-10T11:30:56.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.857 7fee5215d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c0aa4c0 con 0x7fee4c094fb0 2021-02-10T11:30:56.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.857 7fee4affd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fee4c0c2be0 0x7fee4c0c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:56.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.857 7fee4affd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fee4c0c2be0 0x7fee4c0c2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49646/0 (socket says 172.21.2.101:49646) 2021-02-10T11:30:56.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.857 7fee4affd700 1 -- 172.21.2.101:0/3260938490 learned_addr learned my addr 172.21.2.101:0/3260938490 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:56.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.858 7fee4affd700 1 -- 172.21.2.101:0/3260938490 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fee4c094fb0 msgr2=0x7fee4c09e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:56.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.858 7fee4affd700 1 --2- 172.21.2.101:0/3260938490 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fee4c094fb0 0x7fee4c09e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:56.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.858 7fee4affd700 1 -- 172.21.2.101:0/3260938490 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fee4c0c3600 msgr2=0x7fee4c0a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:56.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.858 7fee4affd700 1 --2- 172.21.2.101:0/3260938490 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fee4c0c3600 0x7fee4c0a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:56.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.858 7fee4affd700 1 -- 172.21.2.101:0/3260938490 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c0aa6e0 con 0x7fee4c0c2be0 2021-02-10T11:30:56.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.858 7fee4affd700 1 --2- 172.21.2.101:0/3260938490 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fee4c0c2be0 0x7fee4c0c2fe0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7fee34009630 tx=0x7fee3400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:56.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.858 7fee43fff700 1 -- 172.21.2.101:0/3260938490 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fee340093a0 con 0x7fee4c0c2be0 2021-02-10T11:30:56.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.858 7fee43fff700 1 -- 172.21.2.101:0/3260938490 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3400bd70 con 0x7fee4c0c2be0 2021-02-10T11:30:56.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.858 7fee43fff700 1 -- 172.21.2.101:0/3260938490 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fee34005560 con 0x7fee4c0c2be0 2021-02-10T11:30:56.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.858 7fee5215d700 1 -- 172.21.2.101:0/3260938490 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee4c0aa6e0 con 0x7fee4c0c2be0 2021-02-10T11:30:56.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.858 7fee5215d700 1 -- 172.21.2.101:0/3260938490 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4c0aac70 con 0x7fee4c0c2be0 2021-02-10T11:30:56.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.858 7fee43fff700 1 -- 172.21.2.101:0/3260938490 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fee34007020 con 0x7fee4c0c2be0 2021-02-10T11:30:56.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.859 7fee43fff700 1 --2- 172.21.2.101:0/3260938490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fee300186c0 0x7fee3001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:56.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.859 7fee43fff700 1 -- 172.21.2.101:0/3260938490 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fee34032560 con 0x7fee4c0c2be0 2021-02-10T11:30:56.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.859 7fee4b7fe700 1 --2- 172.21.2.101:0/3260938490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fee300186c0 0x7fee3001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:56.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.859 7fee4b7fe700 1 --2- 172.21.2.101:0/3260938490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fee300186c0 0x7fee3001abc0 crc :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:56.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.890 7fee5215d700 1 -- 172.21.2.101:0/3260938490 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4c0a9750 con 0x7fee4c0c2be0 2021-02-10T11:30:56.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:56.893 7fee43fff700 1 -- 172.21.2.101:0/3260938490 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fee34009340 con 0x7fee4c0c2be0 2021-02-10T11:30:57.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.018 7fee5215d700 1 -- 172.21.2.101:0/3260938490 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839330", "target": ["mgr", ""]}) v1 -- 0x7fee4c0aa2a0 con 0x7fee300186c0 2021-02-10T11:30:57.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.028 7fee43fff700 1 -- 172.21.2.101:0/3260938490 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fee3c001590 con 0x7fee300186c0 2021-02-10T11:30:57.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.035 7fee5215d700 1 -- 172.21.2.101:0/3260938490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fee300186c0 msgr2=0x7fee3001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:57.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.035 7fee5215d700 1 --2- 172.21.2.101:0/3260938490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fee300186c0 0x7fee3001abc0 crc :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:57.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.035 7fee5215d700 1 -- 172.21.2.101:0/3260938490 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fee4c0c2be0 msgr2=0x7fee4c0c2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:57.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.035 7fee5215d700 1 --2- 172.21.2.101:0/3260938490 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fee4c0c2be0 0x7fee4c0c2fe0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7fee34009630 tx=0x7fee3400b980).stop 2021-02-10T11:30:57.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.035 7fee5215d700 1 -- 172.21.2.101:0/3260938490 shutdown_connections 2021-02-10T11:30:57.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.035 7fee5215d700 1 --2- 172.21.2.101:0/3260938490 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fee4c0c3600 0x7fee4c0a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:57.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.035 7fee5215d700 1 --2- 172.21.2.101:0/3260938490 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fee4c0c2be0 0x7fee4c0c2fe0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:57.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.035 7fee5215d700 1 --2- 172.21.2.101:0/3260938490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fee300186c0 0x7fee3001abc0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:57.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.035 7fee5215d700 1 --2- 172.21.2.101:0/3260938490 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fee4c094fb0 0x7fee4c09e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:57.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.035 7fee5215d700 1 -- 172.21.2.101:0/3260938490 >> 172.21.2.101:0/3260938490 conn(0x7fee4c09f630 msgr2=0x7fee4c0a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:57.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.035 7fee5215d700 1 -- 172.21.2.101:0/3260938490 shutdown_connections 2021-02-10T11:30:57.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.035 7fee5215d700 1 -- 172.21.2.101:0/3260938490 wait complete. 2021-02-10T11:30:57.055 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839331 2021-02-10T11:30:57.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.460 7ff82e0cb700 1 Processor -- start 2021-02-10T11:30:57.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.460 7ff82e0cb700 1 -- start start 2021-02-10T11:30:57.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.461 7ff82e0cb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8281b0470 0x7ff8281b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:57.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.461 7ff82e0cb700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8280f0b50 con 0x7ff8280f20e0 2021-02-10T11:30:57.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.461 7ff82e0cb700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8280f24e0 con 0x7ff8280ef610 2021-02-10T11:30:57.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.461 7ff82e0cb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8280e5ea0 con 0x7ff8281b0470 2021-02-10T11:30:57.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.461 7ff827fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff8280f20e0 0x7ff8280f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50146/0 (socket says 172.21.2.101:50146) 2021-02-10T11:30:57.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.461 7ff827fff700 1 -- 172.21.2.101:0/668326708 learned_addr learned my addr 172.21.2.101:0/668326708 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:57.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.461 7ff8267fc700 1 -- 172.21.2.101:0/668326708 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 954458310 0 0) 0x7ff820001500 con 0x7ff8280f20e0 2021-02-10T11:30:57.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.461 7ff8267fc700 1 -- 172.21.2.101:0/668326708 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff814002fe0 con 0x7ff8280f20e0 2021-02-10T11:30:57.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.462 7ff8277fe700 1 --2- 172.21.2.101:0/668326708 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8281b0470 0x7ff8281b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:57.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.462 7ff8267fc700 1 -- 172.21.2.101:0/668326708 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3414735058 0 0) 0x7ff820001500 con 0x7ff8280f20e0 2021-02-10T11:30:57.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.462 7ff8267fc700 1 -- 172.21.2.101:0/668326708 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8281b0470 msgr2=0x7ff8281b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:57.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.462 7ff8267fc700 1 --2- 172.21.2.101:0/668326708 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8281b0470 0x7ff8281b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:57.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.462 7ff8267fc700 1 -- 172.21.2.101:0/668326708 >> v1:172.21.2.108:6789/0 conn(0x7ff8280ef610 legacy=0x7ff8280efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:57.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.462 7ff8267fc700 1 -- 172.21.2.101:0/668326708 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8280e52e0 con 0x7ff8280f20e0 2021-02-10T11:30:57.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.462 7ff8267fc700 1 -- 172.21.2.101:0/668326708 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7ff820001a00 con 0x7ff8280f20e0 2021-02-10T11:30:57.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.462 7ff8267fc700 1 -- 172.21.2.101:0/668326708 >> v1:172.21.2.101:6789/0 conn(0x7ff8280f20e0 legacy=0x7ff8280f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:57.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.462 7ff8267fc700 1 --2- 172.21.2.101:0/668326708 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff814003a60 0x7ff814005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:57.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.462 7ff8267fc700 1 --2- 172.21.2.101:0/668326708 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff814006770 0x7ff814008c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:57.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.462 7ff8267fc700 1 --2- 172.21.2.101:0/668326708 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff814009500 0x7ff81400ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:57.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.462 7ff8267fc700 1 -- 172.21.2.101:0/668326708 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81400c2d0 con 0x7ff814003a60 2021-02-10T11:30:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.462 7ff8267fc700 1 -- 172.21.2.101:0/668326708 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81400c4f0 con 0x7ff814009500 2021-02-10T11:30:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.462 7ff8267fc700 1 -- 172.21.2.101:0/668326708 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81400c710 con 0x7ff814006770 2021-02-10T11:30:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.462 7ff8277fe700 1 --2- 172.21.2.101:0/668326708 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff814003a60 0x7ff814005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.462 7ff826ffd700 1 --2- 172.21.2.101:0/668326708 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff814006770 0x7ff814008c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.462 7ff827fff700 1 --2- 172.21.2.101:0/668326708 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff814009500 0x7ff81400ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.462 7ff8277fe700 1 -- 172.21.2.101:0/668326708 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff814006770 msgr2=0x7ff814008c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.462 7ff8277fe700 1 --2- 172.21.2.101:0/668326708 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff814006770 0x7ff814008c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.462 7ff8277fe700 1 -- 172.21.2.101:0/668326708 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff814009500 msgr2=0x7ff81400ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.462 7ff8277fe700 1 --2- 172.21.2.101:0/668326708 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff814009500 0x7ff81400ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.462 7ff8277fe700 1 -- 172.21.2.101:0/668326708 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff814050980 con 0x7ff814003a60 2021-02-10T11:30:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.463 7ff827fff700 1 --2- 172.21.2.101:0/668326708 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff814009500 0x7ff81400ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:30:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.463 7ff8277fe700 1 --2- 172.21.2.101:0/668326708 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff814003a60 0x7ff814005ef0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7ff810009310 tx=0x7ff810009a90).ready entity=mon.0 client_cookie=875886ff6cae1abb server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.463 7ff8267fc700 1 -- 172.21.2.101:0/668326708 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff81000c0c0 con 0x7ff814003a60 2021-02-10T11:30:57.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.463 7ff8267fc700 1 -- 172.21.2.101:0/668326708 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff810006400 con 0x7ff814003a60 2021-02-10T11:30:57.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.463 7ff8267fc700 1 -- 172.21.2.101:0/668326708 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff81000c400 con 0x7ff814003a60 2021-02-10T11:30:57.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.463 7ff82e0cb700 1 -- 172.21.2.101:0/668326708 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff814003a60 msgr2=0x7ff814005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:57.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.463 7ff82e0cb700 1 --2- 172.21.2.101:0/668326708 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff814003a60 0x7ff814005ef0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7ff810009310 tx=0x7ff810009a90).stop 2021-02-10T11:30:57.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.463 7ff827fff700 1 -- 172.21.2.101:0/668326708 reap_dead start 2021-02-10T11:30:57.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.463 7ff82e0cb700 1 -- 172.21.2.101:0/668326708 shutdown_connections 2021-02-10T11:30:57.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.463 7ff82e0cb700 1 -- 172.21.2.101:0/668326708 >> 172.21.2.101:0/668326708 conn(0x7ff8280ce3e0 msgr2=0x7ff8280e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:57.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.463 7ff82e0cb700 1 -- 172.21.2.101:0/668326708 shutdown_connections 2021-02-10T11:30:57.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.463 7ff82e0cb700 1 -- 172.21.2.101:0/668326708 wait complete. 2021-02-10T11:30:57.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.478 7ff82e0cb700 1 Processor -- start 2021-02-10T11:30:57.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.478 7ff82e0cb700 1 -- start start 2021-02-10T11:30:57.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.479 7ff82e0cb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8280b7db0 0x7ff8280b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:57.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.479 7ff82e0cb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff8280b8880 0x7ff8281a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:57.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.479 7ff82e0cb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8280b8c80 0x7ff8281a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:57.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.479 7ff82e0cb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8280b79b0 con 0x7ff8280b8c80 2021-02-10T11:30:57.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.479 7ff82e0cb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8281a8090 con 0x7ff8280b8880 2021-02-10T11:30:57.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.479 7ff82e0cb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8281a82b0 con 0x7ff8280b7db0 2021-02-10T11:30:57.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.479 7ff827fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8280b8c80 0x7ff8281a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:57.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.479 7ff827fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8280b8c80 0x7ff8281a7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49664/0 (socket says 172.21.2.101:49664) 2021-02-10T11:30:57.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.479 7ff827fff700 1 -- 172.21.2.101:0/3665790464 learned_addr learned my addr 172.21.2.101:0/3665790464 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:57.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.479 7ff827fff700 1 -- 172.21.2.101:0/3665790464 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8280b7db0 msgr2=0x7ff8280b8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:57.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.479 7ff827fff700 1 --2- 172.21.2.101:0/3665790464 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8280b7db0 0x7ff8280b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:57.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.479 7ff827fff700 1 -- 172.21.2.101:0/3665790464 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff8280b8880 msgr2=0x7ff8281a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:57.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.479 7ff827fff700 1 --2- 172.21.2.101:0/3665790464 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff8280b8880 0x7ff8281a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:57.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.479 7ff827fff700 1 -- 172.21.2.101:0/3665790464 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff828190c10 con 0x7ff8280b8c80 2021-02-10T11:30:57.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.479 7ff827fff700 1 --2- 172.21.2.101:0/3665790464 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8280b8c80 0x7ff8281a7a70 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7ff82000b8f0 tx=0x7ff82000bc90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:57.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.480 7ff81ffff700 1 -- 172.21.2.101:0/3665790464 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff820007c00 con 0x7ff8280b8c80 2021-02-10T11:30:57.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.480 7ff81ffff700 1 -- 172.21.2.101:0/3665790464 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff820006080 con 0x7ff8280b8c80 2021-02-10T11:30:57.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.480 7ff81ffff700 1 -- 172.21.2.101:0/3665790464 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff820004720 con 0x7ff8280b8c80 2021-02-10T11:30:57.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.480 7ff82e0cb700 1 -- 172.21.2.101:0/3665790464 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff828190c10 con 0x7ff8280b8c80 2021-02-10T11:30:57.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.480 7ff82e0cb700 1 -- 172.21.2.101:0/3665790464 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8280c1eb0 con 0x7ff8280b8c80 2021-02-10T11:30:57.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.480 7ff81ffff700 1 -- 172.21.2.101:0/3665790464 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff820000d80 con 0x7ff8280b8c80 2021-02-10T11:30:57.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.481 7ff81ffff700 1 --2- 172.21.2.101:0/3665790464 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff80c0186c0 0x7ff80c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:57.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.481 7ff81ffff700 1 -- 172.21.2.101:0/3665790464 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff8200328d0 con 0x7ff8280b8c80 2021-02-10T11:30:57.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.481 7ff8277fe700 1 --2- 172.21.2.101:0/3665790464 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff80c0186c0 0x7ff80c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:57.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.481 7ff8277fe700 1 --2- 172.21.2.101:0/3665790464 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff80c0186c0 0x7ff80c01abc0 crc :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:57.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.511 7ff82e0cb700 1 -- 172.21.2.101:0/3665790464 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff828105b30 con 0x7ff8280b8c80 2021-02-10T11:30:57.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.513 7ff81ffff700 1 -- 172.21.2.101:0/3665790464 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff820030840 con 0x7ff8280b8c80 2021-02-10T11:30:57.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.639 7ff82e0cb700 1 -- 172.21.2.101:0/3665790464 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839331", "target": ["mgr", ""]}) v1 -- 0x7ff8280c1eb0 con 0x7ff80c0186c0 2021-02-10T11:30:57.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.651 7ff81ffff700 1 -- 172.21.2.101:0/3665790464 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7ff810006370 con 0x7ff80c0186c0 2021-02-10T11:30:57.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.656 7ff82e0cb700 1 -- 172.21.2.101:0/3665790464 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff80c0186c0 msgr2=0x7ff80c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:57.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.656 7ff82e0cb700 1 --2- 172.21.2.101:0/3665790464 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff80c0186c0 0x7ff80c01abc0 crc :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:57.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.656 7ff82e0cb700 1 -- 172.21.2.101:0/3665790464 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8280b8c80 msgr2=0x7ff8281a7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:57.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.656 7ff82e0cb700 1 --2- 172.21.2.101:0/3665790464 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8280b8c80 0x7ff8281a7a70 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7ff82000b8f0 tx=0x7ff82000bc90).stop 2021-02-10T11:30:57.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.656 7ff82e0cb700 1 -- 172.21.2.101:0/3665790464 shutdown_connections 2021-02-10T11:30:57.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.656 7ff82e0cb700 1 --2- 172.21.2.101:0/3665790464 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8280b8c80 0x7ff8281a7a70 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:57.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.656 7ff82e0cb700 1 --2- 172.21.2.101:0/3665790464 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff8280b8880 0x7ff8281a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:57.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.656 7ff82e0cb700 1 --2- 172.21.2.101:0/3665790464 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff80c0186c0 0x7ff80c01abc0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:57.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.656 7ff82e0cb700 1 --2- 172.21.2.101:0/3665790464 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8280b7db0 0x7ff8280b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:57.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.656 7ff82e0cb700 1 -- 172.21.2.101:0/3665790464 >> 172.21.2.101:0/3665790464 conn(0x7ff828094e10 msgr2=0x7ff8280a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:57.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.657 7ff82e0cb700 1 -- 172.21.2.101:0/3665790464 shutdown_connections 2021-02-10T11:30:57.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:57.657 7ff82e0cb700 1 -- 172.21.2.101:0/3665790464 wait complete. 2021-02-10T11:30:57.677 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839332 2021-02-10T11:30:58.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.083 7fe108b68700 1 Processor -- start 2021-02-10T11:30:58.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.083 7fe108b68700 1 -- start start 2021-02-10T11:30:58.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.083 7fe108b68700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1041b0470 0x7fe1041b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:58.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.083 7fe108b68700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe1040f20e0 0x7fe1040f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:58.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.083 7fe108b68700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1040f0b50 con 0x7fe1040ef610 2021-02-10T11:30:58.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.083 7fe108b68700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1040f0820 con 0x7fe1040f20e0 2021-02-10T11:30:58.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.083 7fe108b68700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1040f24e0 con 0x7fe1041b0470 2021-02-10T11:30:58.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.084 7fe10305f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe1040f20e0 0x7fe1040f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:58.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.084 7fe10305f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe1040f20e0 0x7fe1040f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49672/0 (socket says 172.21.2.101:49672) 2021-02-10T11:30:58.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.084 7fe10305f700 1 -- 172.21.2.101:0/1805818273 learned_addr learned my addr 172.21.2.101:0/1805818273 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:58.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.084 7fe10285e700 1 --2- 172.21.2.101:0/1805818273 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1041b0470 0x7fe1041b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:58.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.084 7fe10305f700 1 -- 172.21.2.101:0/1805818273 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1041b0470 msgr2=0x7fe1041b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:58.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.084 7fe10305f700 1 --2- 172.21.2.101:0/1805818273 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1041b0470 0x7fe1041b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:58.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.084 7fe10305f700 1 -- 172.21.2.101:0/1805818273 >> v1:172.21.2.108:6789/0 conn(0x7fe1040ef610 legacy=0x7fe1040efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:58.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.084 7fe10305f700 1 -- 172.21.2.101:0/1805818273 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1040e52e0 con 0x7fe1040f20e0 2021-02-10T11:30:58.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.084 7fe10305f700 1 --2- 172.21.2.101:0/1805818273 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1040f20e0 0x7fe1040f0200 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7fe0fc006f00 tx=0x7fe0fc009f10).ready entity=mon.0 client_cookie=fbbd735f36d10c77 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:58.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.084 7fe10185c700 1 -- 172.21.2.101:0/1805818273 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe0fc00b3b0 con 0x7fe1040f20e0 2021-02-10T11:30:58.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.085 7fe10185c700 1 -- 172.21.2.101:0/1805818273 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0fc00b730 con 0x7fe1040f20e0 2021-02-10T11:30:58.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.085 7fe10185c700 1 -- 172.21.2.101:0/1805818273 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe0fc00bc10 con 0x7fe1040f20e0 2021-02-10T11:30:58.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.085 7fe108b68700 1 -- 172.21.2.101:0/1805818273 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1040f20e0 msgr2=0x7fe1040f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:58.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.085 7fe108b68700 1 --2- 172.21.2.101:0/1805818273 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1040f20e0 0x7fe1040f0200 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7fe0fc006f00 tx=0x7fe0fc009f10).stop 2021-02-10T11:30:58.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.085 7fe108b68700 1 -- 172.21.2.101:0/1805818273 shutdown_connections 2021-02-10T11:30:58.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.085 7fe108b68700 1 --2- 172.21.2.101:0/1805818273 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1040f20e0 0x7fe1040f0200 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:58.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.085 7fe108b68700 1 --2- 172.21.2.101:0/1805818273 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1041b0470 0x7fe1041b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:58.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.085 7fe108b68700 1 -- 172.21.2.101:0/1805818273 >> 172.21.2.101:0/1805818273 conn(0x7fe1040ce3e0 msgr2=0x7fe1040e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:58.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.085 7fe108b68700 1 -- 172.21.2.101:0/1805818273 shutdown_connections 2021-02-10T11:30:58.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.085 7fe108b68700 1 -- 172.21.2.101:0/1805818273 wait complete. 2021-02-10T11:30:58.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.100 7fe108b68700 1 Processor -- start 2021-02-10T11:30:58.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.100 7fe108b68700 1 -- start start 2021-02-10T11:30:58.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.100 7fe108b68700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe1040bf920 0x7fe1040bfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:58.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.100 7fe108b68700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1040b6570 0x7fe1040b6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:58.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.100 7fe108b68700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1041a7430 0x7fe1041a7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:58.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.100 7fe108b68700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1040c0440 con 0x7fe1041a7430 2021-02-10T11:30:58.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.100 7fe108b68700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1040924b0 con 0x7fe1040bf920 2021-02-10T11:30:58.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.100 7fe108b68700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe104092970 con 0x7fe1040b6570 2021-02-10T11:30:58.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.101 7fe10305f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1041a7430 0x7fe1041a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:58.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.101 7fe10305f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1041a7430 0x7fe1041a7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49674/0 (socket says 172.21.2.101:49674) 2021-02-10T11:30:58.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.101 7fe10305f700 1 -- 172.21.2.101:0/3188224726 learned_addr learned my addr 172.21.2.101:0/3188224726 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:58.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.101 7fe10205d700 1 --2- 172.21.2.101:0/3188224726 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1040b6570 0x7fe1040b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:58.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.101 7fe10305f700 1 -- 172.21.2.101:0/3188224726 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1040b6570 msgr2=0x7fe1040b6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:58.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.101 7fe10305f700 1 --2- 172.21.2.101:0/3188224726 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1040b6570 0x7fe1040b6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:58.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.101 7fe10305f700 1 -- 172.21.2.101:0/3188224726 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe1040bf920 msgr2=0x7fe1040bfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:58.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.101 7fe10305f700 1 --2- 172.21.2.101:0/3188224726 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe1040bf920 0x7fe1040bfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:58.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.101 7fe10305f700 1 -- 172.21.2.101:0/3188224726 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1041a7e50 con 0x7fe1041a7430 2021-02-10T11:30:58.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.101 7fe10305f700 1 --2- 172.21.2.101:0/3188224726 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1041a7430 0x7fe1041a7830 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7fe0fc009390 tx=0x7fe0fc00bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:58.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.101 7fe0f37fe700 1 -- 172.21.2.101:0/3188224726 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe0fc003030 con 0x7fe1041a7430 2021-02-10T11:30:58.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.101 7fe0f37fe700 1 -- 172.21.2.101:0/3188224726 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0fc0068e0 con 0x7fe1041a7430 2021-02-10T11:30:58.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.101 7fe0f37fe700 1 -- 172.21.2.101:0/3188224726 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe0fc00c570 con 0x7fe1041a7430 2021-02-10T11:30:58.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.101 7fe108b68700 1 -- 172.21.2.101:0/3188224726 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1041a7e50 con 0x7fe1041a7430 2021-02-10T11:30:58.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.101 7fe108b68700 1 -- 172.21.2.101:0/3188224726 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1040c09c0 con 0x7fe1041a7430 2021-02-10T11:30:58.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.102 7fe0f37fe700 1 -- 172.21.2.101:0/3188224726 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe0fc005820 con 0x7fe1041a7430 2021-02-10T11:30:58.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.102 7fe0f37fe700 1 --2- 172.21.2.101:0/3188224726 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0e80186d0 0x7fe0e801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:58.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.102 7fe0f37fe700 1 -- 172.21.2.101:0/3188224726 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe0fc0329a0 con 0x7fe1041a7430 2021-02-10T11:30:58.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.102 7fe10285e700 1 --2- 172.21.2.101:0/3188224726 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0e80186d0 0x7fe0e801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:58.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.102 7fe10285e700 1 --2- 172.21.2.101:0/3188224726 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0e80186d0 0x7fe0e801abd0 crc :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:58.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.132 7fe108b68700 1 -- 172.21.2.101:0/3188224726 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe104105b40 con 0x7fe1041a7430 2021-02-10T11:30:58.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.135 7fe0f37fe700 1 -- 172.21.2.101:0/3188224726 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe0fc006c80 con 0x7fe1041a7430 2021-02-10T11:30:58.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.265 7fe108b68700 1 -- 172.21.2.101:0/3188224726 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839332", "target": ["mgr", ""]}) v1 -- 0x7fe1040b6f90 con 0x7fe0e80186d0 2021-02-10T11:30:58.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.276 7fe0f37fe700 1 -- 172.21.2.101:0/3188224726 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fe0f8006e70 con 0x7fe0e80186d0 2021-02-10T11:30:58.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.283 7fe108b68700 1 -- 172.21.2.101:0/3188224726 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0e80186d0 msgr2=0x7fe0e801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:58.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.283 7fe108b68700 1 --2- 172.21.2.101:0/3188224726 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0e80186d0 0x7fe0e801abd0 crc :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:58.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.283 7fe108b68700 1 -- 172.21.2.101:0/3188224726 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1041a7430 msgr2=0x7fe1041a7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:58.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.283 7fe108b68700 1 --2- 172.21.2.101:0/3188224726 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1041a7430 0x7fe1041a7830 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7fe0fc009390 tx=0x7fe0fc00bd40).stop 2021-02-10T11:30:58.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.283 7fe108b68700 1 -- 172.21.2.101:0/3188224726 shutdown_connections 2021-02-10T11:30:58.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.283 7fe108b68700 1 --2- 172.21.2.101:0/3188224726 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0e80186d0 0x7fe0e801abd0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:58.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.283 7fe108b68700 1 --2- 172.21.2.101:0/3188224726 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1040b6570 0x7fe1040b6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:58.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.283 7fe108b68700 1 --2- 172.21.2.101:0/3188224726 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1041a7430 0x7fe1041a7830 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:58.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.283 7fe108b68700 1 --2- 172.21.2.101:0/3188224726 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe1040bf920 0x7fe1040bfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:30:58.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.283 7fe108b68700 1 -- 172.21.2.101:0/3188224726 >> 172.21.2.101:0/3188224726 conn(0x7fe1040b50c0 msgr2=0x7fe104092d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:58.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.283 7fe108b68700 1 -- 172.21.2.101:0/3188224726 shutdown_connections 2021-02-10T11:30:58.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.283 7fe108b68700 1 -- 172.21.2.101:0/3188224726 wait complete. 2021-02-10T11:30:58.302 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839333 2021-02-10T11:30:58.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.705 7f25e48bc700 1 Processor -- start 2021-02-10T11:30:58.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.705 7f25e48bc700 1 -- start start 2021-02-10T11:30:58.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.705 7f25e48bc700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f25e01b0870 0x7f25e00f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:58.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.705 7f25e48bc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25e00efe00 0x7f25e00f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:58.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.705 7f25e48bc700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e01b0c70 con 0x7f25e01b0470 2021-02-10T11:30:58.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.705 7f25e48bc700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e00f0b50 con 0x7f25e01b0870 2021-02-10T11:30:58.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.705 7f25e48bc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e00f0820 con 0x7f25e00efe00 2021-02-10T11:30:58.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.705 7f25de61c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f25e01b0470 0x7f25e00ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50176/0 (socket says 172.21.2.101:50176) 2021-02-10T11:30:58.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.705 7f25de61c700 1 -- 172.21.2.101:0/3786380223 learned_addr learned my addr 172.21.2.101:0/3786380223 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:58.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.706 7f25dde1b700 1 --2- 172.21.2.101:0/3786380223 >> v2:172.21.2.108:3300/0 conn(0x7f25e01b0870 0x7f25e00f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:58.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.706 7f25dee1d700 1 --2- 172.21.2.101:0/3786380223 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25e00efe00 0x7f25e00f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:58.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.706 7f25dd61a700 1 -- 172.21.2.101:0/3786380223 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 495391279 0 0) 0x7f25d00009b0 con 0x7f25e01b0470 2021-02-10T11:30:58.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.706 7f25dd61a700 1 -- 172.21.2.101:0/3786380223 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25cc002fe0 con 0x7f25e01b0470 2021-02-10T11:30:58.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.706 7f25dde1b700 1 -- 172.21.2.101:0/3786380223 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25e00efe00 msgr2=0x7f25e00f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:58.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.706 7f25dde1b700 1 --2- 172.21.2.101:0/3786380223 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25e00efe00 0x7f25e00f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:58.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.706 7f25dde1b700 1 -- 172.21.2.101:0/3786380223 >> v1:172.21.2.101:6789/0 conn(0x7f25e01b0470 legacy=0x7f25e00ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:58.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.706 7f25dde1b700 1 -- 172.21.2.101:0/3786380223 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e00e52e0 con 0x7f25e01b0870 2021-02-10T11:30:58.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.706 7f25dde1b700 1 --2- 172.21.2.101:0/3786380223 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25e01b0870 0x7f25e00f20e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f25c8006650 tx=0x7f25c8009a80).ready entity=mon.1 client_cookie=55cfdebcad127841 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:58.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.707 7f25dd61a700 1 -- 172.21.2.101:0/3786380223 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f25c80009b0 con 0x7f25e01b0870 2021-02-10T11:30:58.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.707 7f25dd61a700 1 -- 172.21.2.101:0/3786380223 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c800c0c0 con 0x7f25e01b0870 2021-02-10T11:30:58.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.707 7f25dd61a700 1 -- 172.21.2.101:0/3786380223 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f25c800c5e0 con 0x7f25e01b0870 2021-02-10T11:30:58.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.707 7f25e48bc700 1 -- 172.21.2.101:0/3786380223 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25e01b0870 msgr2=0x7f25e00f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:58.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.707 7f25e48bc700 1 --2- 172.21.2.101:0/3786380223 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25e01b0870 0x7f25e00f20e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f25c8006650 tx=0x7f25c8009a80).stop 2021-02-10T11:30:58.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.708 7f25e48bc700 1 -- 172.21.2.101:0/3786380223 shutdown_connections 2021-02-10T11:30:58.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.708 7f25e48bc700 1 --2- 172.21.2.101:0/3786380223 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25e00efe00 0x7f25e00f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:58.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.708 7f25e48bc700 1 --2- 172.21.2.101:0/3786380223 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25e01b0870 0x7f25e00f20e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:58.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.708 7f25e48bc700 1 -- 172.21.2.101:0/3786380223 >> 172.21.2.101:0/3786380223 conn(0x7f25e00ce3e0 msgr2=0x7f25e00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:58.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.708 7f25e48bc700 1 -- 172.21.2.101:0/3786380223 shutdown_connections 2021-02-10T11:30:58.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.708 7f25e48bc700 1 -- 172.21.2.101:0/3786380223 wait complete. 2021-02-10T11:30:58.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.723 7f25e48bc700 1 Processor -- start 2021-02-10T11:30:58.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.723 7f25e48bc700 1 -- start start 2021-02-10T11:30:58.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.723 7f25e48bc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25e00a0420 0x7f25e00a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:58.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.723 7f25e48bc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25e0189d00 0x7f25e018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:58.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.723 7f25e48bc700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25e018a720 0x7f25e00b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:58.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.723 7f25e48bc700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e0189800 con 0x7f25e00a0420 2021-02-10T11:30:58.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.723 7f25e48bc700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e018ab20 con 0x7f25e018a720 2021-02-10T11:30:58.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.723 7f25e48bc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e00a1140 con 0x7f25e0189d00 2021-02-10T11:30:58.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.723 7f25de61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25e00a0420 0x7f25e00a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:58.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.723 7f25de61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25e00a0420 0x7f25e00a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49690/0 (socket says 172.21.2.101:49690) 2021-02-10T11:30:58.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.723 7f25de61c700 1 -- 172.21.2.101:0/2857002534 learned_addr learned my addr 172.21.2.101:0/2857002534 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:58.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.723 7f25dee1d700 1 --2- 172.21.2.101:0/2857002534 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25e018a720 0x7f25e00b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:58.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.723 7f25dde1b700 1 --2- 172.21.2.101:0/2857002534 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25e0189d00 0x7f25e018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:58.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.724 7f25dee1d700 1 -- 172.21.2.101:0/2857002534 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25e0189d00 msgr2=0x7f25e018a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:58.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.724 7f25dee1d700 1 --2- 172.21.2.101:0/2857002534 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25e0189d00 0x7f25e018a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:58.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.724 7f25dee1d700 1 -- 172.21.2.101:0/2857002534 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25e00a0420 msgr2=0x7f25e00a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:58.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.724 7f25dee1d700 1 --2- 172.21.2.101:0/2857002534 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25e00a0420 0x7f25e00a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:58.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.724 7f25dee1d700 1 -- 172.21.2.101:0/2857002534 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e00b8200 con 0x7f25e018a720 2021-02-10T11:30:58.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.724 7f25dee1d700 1 --2- 172.21.2.101:0/2857002534 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25e018a720 0x7f25e00b7be0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f25d800b820 tx=0x7f25d800bbc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:58.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.724 7f25d6ffd700 1 -- 172.21.2.101:0/2857002534 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f25d8000f90 con 0x7f25e018a720 2021-02-10T11:30:58.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.724 7f25d6ffd700 1 -- 172.21.2.101:0/2857002534 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d80011f0 con 0x7f25e018a720 2021-02-10T11:30:58.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.724 7f25d6ffd700 1 -- 172.21.2.101:0/2857002534 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f25d8001730 con 0x7f25e018a720 2021-02-10T11:30:58.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.724 7f25e48bc700 1 -- 172.21.2.101:0/2857002534 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e00b8200 con 0x7f25e018a720 2021-02-10T11:30:58.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.724 7f25e48bc700 1 -- 172.21.2.101:0/2857002534 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e00b8200 con 0x7f25e018a720 2021-02-10T11:30:58.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.725 7f25d6ffd700 1 -- 172.21.2.101:0/2857002534 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f25d80056b0 con 0x7f25e018a720 2021-02-10T11:30:58.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.725 7f25d6ffd700 1 --2- 172.21.2.101:0/2857002534 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25c4018730 0x7f25c401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:58.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.725 7f25d6ffd700 1 -- 172.21.2.101:0/2857002534 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f25d8032a90 con 0x7f25e018a720 2021-02-10T11:30:58.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.725 7f25de61c700 1 --2- 172.21.2.101:0/2857002534 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25c4018730 0x7f25c401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:58.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.725 7f25de61c700 1 --2- 172.21.2.101:0/2857002534 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25c4018730 0x7f25c401ac10 crc :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:58.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.754 7f25e48bc700 1 -- 172.21.2.101:0/2857002534 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e018ab20 con 0x7f25e018a720 2021-02-10T11:30:58.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.757 7f25d6ffd700 1 -- 172.21.2.101:0/2857002534 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f25d80066f0 con 0x7f25e018a720 2021-02-10T11:30:58.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.888 7f25e48bc700 1 -- 172.21.2.101:0/2857002534 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839333", "target": ["mgr", ""]}) v1 -- 0x7f25e018ab20 con 0x7f25c4018730 2021-02-10T11:30:58.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.898 7f25d6ffd700 1 -- 172.21.2.101:0/2857002534 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f25d0001560 con 0x7f25c4018730 2021-02-10T11:30:58.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.905 7f25e48bc700 1 -- 172.21.2.101:0/2857002534 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25c4018730 msgr2=0x7f25c401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:58.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.905 7f25e48bc700 1 --2- 172.21.2.101:0/2857002534 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25c4018730 0x7f25c401ac10 crc :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:58.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.905 7f25e48bc700 1 -- 172.21.2.101:0/2857002534 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25e018a720 msgr2=0x7f25e00b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:58.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.905 7f25e48bc700 1 --2- 172.21.2.101:0/2857002534 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25e018a720 0x7f25e00b7be0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f25d800b820 tx=0x7f25d800bbc0).stop 2021-02-10T11:30:58.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.905 7f25e48bc700 1 -- 172.21.2.101:0/2857002534 shutdown_connections 2021-02-10T11:30:58.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.905 7f25e48bc700 1 --2- 172.21.2.101:0/2857002534 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25c4018730 0x7f25c401ac10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:58.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.905 7f25e48bc700 1 --2- 172.21.2.101:0/2857002534 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25e0189d00 0x7f25e018a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:58.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.905 7f25e48bc700 1 --2- 172.21.2.101:0/2857002534 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25e018a720 0x7f25e00b7be0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:58.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.905 7f25e48bc700 1 --2- 172.21.2.101:0/2857002534 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25e00a0420 0x7f25e00a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:58.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.905 7f25e48bc700 1 -- 172.21.2.101:0/2857002534 >> 172.21.2.101:0/2857002534 conn(0x7f25e0094e10 msgr2=0x7f25e00a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:58.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.905 7f25e48bc700 1 -- 172.21.2.101:0/2857002534 shutdown_connections 2021-02-10T11:30:58.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:58.905 7f25e48bc700 1 -- 172.21.2.101:0/2857002534 wait complete. 2021-02-10T11:30:58.925 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839334 2021-02-10T11:30:59.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.329 7f0a3786f700 1 Processor -- start 2021-02-10T11:30:59.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.330 7f0a3786f700 1 -- start start 2021-02-10T11:30:59.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.330 7f0a3786f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0a301b0470 0x7f0a301b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:59.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.330 7f0a3786f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0a300ef610 0x7f0a300efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:59.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.330 7f0a3786f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a300f0b50 con 0x7f0a300f0030 2021-02-10T11:30:59.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.330 7f0a3786f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a300e52e0 con 0x7f0a301b0470 2021-02-10T11:30:59.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.330 7f0a3786f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a300e5490 con 0x7f0a300ef610 2021-02-10T11:30:59.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.330 7f0a3568b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0a301b0470 0x7f0a301b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:59.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.330 7f0a3568b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0a301b0470 0x7f0a301b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49696/0 (socket says 172.21.2.101:49696) 2021-02-10T11:30:59.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.330 7f0a3568b700 1 -- 172.21.2.101:0/1758563369 learned_addr learned my addr 172.21.2.101:0/1758563369 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:59.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.330 7f0a34e8a700 1 --2- 172.21.2.101:0/1758563369 >> v2:172.21.2.108:3300/0 conn(0x7f0a300ef610 0x7f0a300efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:59.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.330 7f0a3568b700 1 -- 172.21.2.101:0/1758563369 >> v2:172.21.2.108:3300/0 conn(0x7f0a300ef610 msgr2=0x7f0a300efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:59.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.330 7f0a3568b700 1 --2- 172.21.2.101:0/1758563369 >> v2:172.21.2.108:3300/0 conn(0x7f0a300ef610 0x7f0a300efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:59.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.331 7f0a3568b700 1 -- 172.21.2.101:0/1758563369 >> v1:172.21.2.108:6789/0 conn(0x7f0a300f0030 legacy=0x7f0a300f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:59.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.331 7f0a3568b700 1 -- 172.21.2.101:0/1758563369 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a300f0430 con 0x7f0a301b0470 2021-02-10T11:30:59.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.331 7f0a3568b700 1 --2- 172.21.2.101:0/1758563369 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a301b0470 0x7f0a301b0870 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7f0a2c006700 tx=0x7f0a2c009b80).ready entity=mon.0 client_cookie=8402b6480c11dbc4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:59.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.331 7f0a23fff700 1 -- 172.21.2.101:0/1758563369 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0a2c00c0c0 con 0x7f0a301b0470 2021-02-10T11:30:59.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.331 7f0a23fff700 1 -- 172.21.2.101:0/1758563369 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2c00c300 con 0x7f0a301b0470 2021-02-10T11:30:59.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.331 7f0a23fff700 1 -- 172.21.2.101:0/1758563369 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0a2c00c840 con 0x7f0a301b0470 2021-02-10T11:30:59.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.331 7f0a3786f700 1 -- 172.21.2.101:0/1758563369 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a301b0470 msgr2=0x7f0a301b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:59.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.331 7f0a3786f700 1 --2- 172.21.2.101:0/1758563369 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a301b0470 0x7f0a301b0870 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7f0a2c006700 tx=0x7f0a2c009b80).stop 2021-02-10T11:30:59.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.332 7f0a3786f700 1 -- 172.21.2.101:0/1758563369 shutdown_connections 2021-02-10T11:30:59.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.332 7f0a3786f700 1 --2- 172.21.2.101:0/1758563369 >> v2:172.21.2.108:3300/0 conn(0x7f0a300ef610 0x7f0a300efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:59.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.332 7f0a3786f700 1 --2- 172.21.2.101:0/1758563369 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a301b0470 0x7f0a301b0870 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:59.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.332 7f0a3786f700 1 -- 172.21.2.101:0/1758563369 >> 172.21.2.101:0/1758563369 conn(0x7f0a300ce3e0 msgr2=0x7f0a300e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:59.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.332 7f0a3786f700 1 -- 172.21.2.101:0/1758563369 shutdown_connections 2021-02-10T11:30:59.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.332 7f0a3786f700 1 -- 172.21.2.101:0/1758563369 wait complete. 2021-02-10T11:30:59.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.347 7f0a3786f700 1 Processor -- start 2021-02-10T11:30:59.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.347 7f0a3786f700 1 -- start start 2021-02-10T11:30:59.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.347 7f0a3786f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a300a0420 0x7f0a300a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:59.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.347 7f0a3786f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a30189d00 0x7f0a3018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:59.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.347 7f0a3786f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0a3018a720 0x7f0a300b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:59.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.347 7f0a3786f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a30189780 con 0x7f0a300a0420 2021-02-10T11:30:59.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.347 7f0a3786f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3018ab20 con 0x7f0a3018a720 2021-02-10T11:30:59.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.347 7f0a3786f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a300a1140 con 0x7f0a30189d00 2021-02-10T11:30:59.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.347 7f0a3568b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a300a0420 0x7f0a300a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:59.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.347 7f0a3568b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a300a0420 0x7f0a300a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49700/0 (socket says 172.21.2.101:49700) 2021-02-10T11:30:59.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.347 7f0a3568b700 1 -- 172.21.2.101:0/3974632899 learned_addr learned my addr 172.21.2.101:0/3974632899 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:59.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.347 7f0a34e8a700 1 --2- 172.21.2.101:0/3974632899 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a30189d00 0x7f0a3018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:59.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.347 7f0a35e8c700 1 --2- 172.21.2.101:0/3974632899 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0a3018a720 0x7f0a300b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:59.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.348 7f0a3568b700 1 -- 172.21.2.101:0/3974632899 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a30189d00 msgr2=0x7f0a3018a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:59.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.348 7f0a3568b700 1 --2- 172.21.2.101:0/3974632899 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a30189d00 0x7f0a3018a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:59.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.348 7f0a3568b700 1 -- 172.21.2.101:0/3974632899 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0a3018a720 msgr2=0x7f0a300b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:59.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.348 7f0a3568b700 1 --2- 172.21.2.101:0/3974632899 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0a3018a720 0x7f0a300b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:59.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.348 7f0a3568b700 1 -- 172.21.2.101:0/3974632899 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a300b8200 con 0x7f0a300a0420 2021-02-10T11:30:59.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.348 7f0a34e8a700 1 --2- 172.21.2.101:0/3974632899 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a30189d00 0x7f0a3018a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:30:59.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.348 7f0a3568b700 1 --2- 172.21.2.101:0/3974632899 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a300a0420 0x7f0a300a0820 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7f0a2c006730 tx=0x7f0a2c00b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:59.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.348 7f0a21ffb700 1 -- 172.21.2.101:0/3974632899 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0a2c000c90 con 0x7f0a300a0420 2021-02-10T11:30:59.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.348 7f0a21ffb700 1 -- 172.21.2.101:0/3974632899 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2c002030 con 0x7f0a300a0420 2021-02-10T11:30:59.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.348 7f0a21ffb700 1 -- 172.21.2.101:0/3974632899 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0a2c002550 con 0x7f0a300a0420 2021-02-10T11:30:59.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.348 7f0a3786f700 1 -- 172.21.2.101:0/3974632899 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a300b8200 con 0x7f0a300a0420 2021-02-10T11:30:59.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.348 7f0a3786f700 1 -- 172.21.2.101:0/3974632899 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a300b8790 con 0x7f0a300a0420 2021-02-10T11:30:59.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.348 7f0a21ffb700 1 -- 172.21.2.101:0/3974632899 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0a2c000c90 con 0x7f0a300a0420 2021-02-10T11:30:59.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.348 7f0a21ffb700 1 --2- 172.21.2.101:0/3974632899 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a100186c0 0x7f0a1001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:59.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.348 7f0a21ffb700 1 -- 172.21.2.101:0/3974632899 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0a2c0323e0 con 0x7f0a300a0420 2021-02-10T11:30:59.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.349 7f0a34e8a700 1 --2- 172.21.2.101:0/3974632899 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a100186c0 0x7f0a1001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:59.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.349 7f0a34e8a700 1 --2- 172.21.2.101:0/3974632899 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a100186c0 0x7f0a1001abc0 crc :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:59.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.379 7f0a3786f700 1 -- 172.21.2.101:0/3974632899 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a300a1360 con 0x7f0a300a0420 2021-02-10T11:30:59.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.381 7f0a21ffb700 1 -- 172.21.2.101:0/3974632899 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0a2c025240 con 0x7f0a300a0420 2021-02-10T11:30:59.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.507 7f0a3786f700 1 -- 172.21.2.101:0/3974632899 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839334", "target": ["mgr", ""]}) v1 -- 0x7f0a300b9020 con 0x7f0a100186c0 2021-02-10T11:30:59.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.517 7f0a21ffb700 1 -- 172.21.2.101:0/3974632899 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f0a240009b0 con 0x7f0a100186c0 2021-02-10T11:30:59.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.524 7f0a3786f700 1 -- 172.21.2.101:0/3974632899 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a100186c0 msgr2=0x7f0a1001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:59.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.524 7f0a3786f700 1 --2- 172.21.2.101:0/3974632899 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a100186c0 0x7f0a1001abc0 crc :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:59.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.524 7f0a3786f700 1 -- 172.21.2.101:0/3974632899 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a300a0420 msgr2=0x7f0a300a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:59.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.524 7f0a3786f700 1 --2- 172.21.2.101:0/3974632899 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a300a0420 0x7f0a300a0820 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7f0a2c006730 tx=0x7f0a2c00b940).stop 2021-02-10T11:30:59.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.524 7f0a3786f700 1 -- 172.21.2.101:0/3974632899 shutdown_connections 2021-02-10T11:30:59.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.524 7f0a3786f700 1 --2- 172.21.2.101:0/3974632899 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a100186c0 0x7f0a1001abc0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:59.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.524 7f0a3786f700 1 --2- 172.21.2.101:0/3974632899 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a30189d00 0x7f0a3018a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:59.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.524 7f0a3786f700 1 --2- 172.21.2.101:0/3974632899 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0a3018a720 0x7f0a300b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:59.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.524 7f0a3786f700 1 --2- 172.21.2.101:0/3974632899 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a300a0420 0x7f0a300a0820 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:59.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.524 7f0a3786f700 1 -- 172.21.2.101:0/3974632899 >> 172.21.2.101:0/3974632899 conn(0x7f0a30094e10 msgr2=0x7f0a300a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:59.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.524 7f0a3786f700 1 -- 172.21.2.101:0/3974632899 shutdown_connections 2021-02-10T11:30:59.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.524 7f0a3786f700 1 -- 172.21.2.101:0/3974632899 wait complete. 2021-02-10T11:30:59.544 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839335 2021-02-10T11:30:59.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.947 7fd4786c6700 1 Processor -- start 2021-02-10T11:30:59.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.947 7fd4786c6700 1 -- start start 2021-02-10T11:30:59.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.948 7fd4786c6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd4701b0870 0x7fd4700f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:59.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.948 7fd4786c6700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd4700efe00 0x7fd4700f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:59.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.948 7fd4786c6700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4701b0c70 con 0x7fd4701b0470 2021-02-10T11:30:59.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.948 7fd4786c6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4700e5ea0 con 0x7fd4701b0870 2021-02-10T11:30:59.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.948 7fd4786c6700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4700f0b50 con 0x7fd4700efe00 2021-02-10T11:30:59.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.948 7fd475ce1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd4701b0870 0x7fd4700f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:59.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.948 7fd4764e2700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd4701b0470 0x7fd4700ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50202/0 (socket says 172.21.2.101:50202) 2021-02-10T11:30:59.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.948 7fd4764e2700 1 -- 172.21.2.101:0/3247332021 learned_addr learned my addr 172.21.2.101:0/3247332021 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:30:59.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.948 7fd476ce3700 1 --2- 172.21.2.101:0/3247332021 >> v2:172.21.2.108:3300/0 conn(0x7fd4700efe00 0x7fd4700f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:59.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.948 7fd476ce3700 1 -- 172.21.2.101:0/3247332021 >> v2:172.21.2.101:3300/0 conn(0x7fd4701b0870 msgr2=0x7fd4700f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:30:59.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.948 7fd476ce3700 1 --2- 172.21.2.101:0/3247332021 >> v2:172.21.2.101:3300/0 conn(0x7fd4701b0870 0x7fd4700f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:59.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.948 7fd476ce3700 1 -- 172.21.2.101:0/3247332021 >> v1:172.21.2.101:6789/0 conn(0x7fd4701b0470 legacy=0x7fd4700ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:59.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.948 7fd476ce3700 1 -- 172.21.2.101:0/3247332021 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4700f0820 con 0x7fd4700efe00 2021-02-10T11:30:59.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.948 7fd475ce1700 1 --2- 172.21.2.101:0/3247332021 >> v2:172.21.2.101:3300/0 conn(0x7fd4701b0870 0x7fd4700f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:30:59.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.948 7fd476ce3700 1 --2- 172.21.2.101:0/3247332021 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4700efe00 0x7fd4700f0200 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fd468006e50 tx=0x7fd468009e70).ready entity=mon.1 client_cookie=25c7390fddcdefd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:59.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.949 7fd4754e0700 1 -- 172.21.2.101:0/3247332021 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd468006a20 con 0x7fd4700efe00 2021-02-10T11:30:59.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.949 7fd4754e0700 1 -- 172.21.2.101:0/3247332021 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd468006790 con 0x7fd4700efe00 2021-02-10T11:30:59.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.949 7fd4754e0700 1 -- 172.21.2.101:0/3247332021 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd46800b700 con 0x7fd4700efe00 2021-02-10T11:30:59.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.949 7fd4786c6700 1 -- 172.21.2.101:0/3247332021 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4700efe00 msgr2=0x7fd4700f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:59.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.949 7fd4786c6700 1 --2- 172.21.2.101:0/3247332021 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4700efe00 0x7fd4700f0200 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fd468006e50 tx=0x7fd468009e70).stop 2021-02-10T11:30:59.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.950 7fd4786c6700 1 -- 172.21.2.101:0/3247332021 shutdown_connections 2021-02-10T11:30:59.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.950 7fd4786c6700 1 --2- 172.21.2.101:0/3247332021 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4700efe00 0x7fd4700f0200 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:59.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.950 7fd4786c6700 1 --2- 172.21.2.101:0/3247332021 >> v2:172.21.2.101:3300/0 conn(0x7fd4701b0870 0x7fd4700f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:59.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.950 7fd4786c6700 1 -- 172.21.2.101:0/3247332021 >> 172.21.2.101:0/3247332021 conn(0x7fd4700ce3e0 msgr2=0x7fd4700e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:30:59.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.950 7fd4786c6700 1 -- 172.21.2.101:0/3247332021 shutdown_connections 2021-02-10T11:30:59.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.950 7fd4786c6700 1 -- 172.21.2.101:0/3247332021 wait complete. 2021-02-10T11:30:59.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.964 7fd4786c6700 1 Processor -- start 2021-02-10T11:30:59.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.964 7fd4786c6700 1 -- start start 2021-02-10T11:30:59.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.965 7fd4786c6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4700a0420 0x7fd4700a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:59.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.965 7fd4786c6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd470189d00 0x7fd47018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:59.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.965 7fd4786c6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd47018a720 0x7fd4700b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:59.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.965 7fd4786c6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd470189780 con 0x7fd470189d00 2021-02-10T11:30:59.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.965 7fd4786c6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47018ab20 con 0x7fd4700a0420 2021-02-10T11:30:59.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.965 7fd4786c6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4700a1140 con 0x7fd47018a720 2021-02-10T11:30:59.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.965 7fd475ce1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd470189d00 0x7fd47018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:59.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.965 7fd475ce1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd470189d00 0x7fd47018a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49714/0 (socket says 172.21.2.101:49714) 2021-02-10T11:30:59.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.965 7fd475ce1700 1 -- 172.21.2.101:0/909414007 learned_addr learned my addr 172.21.2.101:0/909414007 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:30:59.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.965 7fd4764e2700 1 --2- 172.21.2.101:0/909414007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4700a0420 0x7fd4700a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:59.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.965 7fd476ce3700 1 --2- 172.21.2.101:0/909414007 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd47018a720 0x7fd4700b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:59.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.965 7fd475ce1700 1 -- 172.21.2.101:0/909414007 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd47018a720 msgr2=0x7fd4700b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:59.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.965 7fd475ce1700 1 --2- 172.21.2.101:0/909414007 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd47018a720 0x7fd4700b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:59.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.965 7fd475ce1700 1 -- 172.21.2.101:0/909414007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4700a0420 msgr2=0x7fd4700a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:30:59.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.965 7fd475ce1700 1 --2- 172.21.2.101:0/909414007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4700a0420 0x7fd4700a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:30:59.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.965 7fd475ce1700 1 -- 172.21.2.101:0/909414007 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4700b8200 con 0x7fd470189d00 2021-02-10T11:30:59.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.965 7fd475ce1700 1 --2- 172.21.2.101:0/909414007 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd470189d00 0x7fd47018a100 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7fd46c00b580 tx=0x7fd46c00b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:30:59.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.966 7fd462ffd700 1 -- 172.21.2.101:0/909414007 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd46c00bd10 con 0x7fd470189d00 2021-02-10T11:30:59.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.966 7fd462ffd700 1 -- 172.21.2.101:0/909414007 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46c000c90 con 0x7fd470189d00 2021-02-10T11:30:59.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.966 7fd462ffd700 1 -- 172.21.2.101:0/909414007 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd46c002220 con 0x7fd470189d00 2021-02-10T11:30:59.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.966 7fd4786c6700 1 -- 172.21.2.101:0/909414007 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4700b8200 con 0x7fd470189d00 2021-02-10T11:30:59.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.966 7fd4786c6700 1 -- 172.21.2.101:0/909414007 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4700b8790 con 0x7fd470189d00 2021-02-10T11:30:59.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.966 7fd462ffd700 1 -- 172.21.2.101:0/909414007 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd46c006b00 con 0x7fd470189d00 2021-02-10T11:30:59.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.966 7fd462ffd700 1 --2- 172.21.2.101:0/909414007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4580186c0 0x7fd45801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:30:59.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.966 7fd462ffd700 1 -- 172.21.2.101:0/909414007 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd46c032780 con 0x7fd470189d00 2021-02-10T11:30:59.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.966 7fd4764e2700 1 --2- 172.21.2.101:0/909414007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4580186c0 0x7fd45801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:30:59.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.967 7fd4764e2700 1 --2- 172.21.2.101:0/909414007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4580186c0 0x7fd45801abc0 crc :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:00.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.997 7fd4786c6700 1 -- 172.21.2.101:0/909414007 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd470105ae0 con 0x7fd470189d00 2021-02-10T11:31:00.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:30:59.999 7fd462ffd700 1 -- 172.21.2.101:0/909414007 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd46c00bd10 con 0x7fd470189d00 2021-02-10T11:31:00.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.126 7fd4786c6700 1 -- 172.21.2.101:0/909414007 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839335", "target": ["mgr", ""]}) v1 -- 0x7fd4700f0b50 con 0x7fd4580186c0 2021-02-10T11:31:00.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.136 7fd462ffd700 1 -- 172.21.2.101:0/909414007 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fd464000df0 con 0x7fd4580186c0 2021-02-10T11:31:00.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.144 7fd4786c6700 1 -- 172.21.2.101:0/909414007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4580186c0 msgr2=0x7fd45801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:00.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.144 7fd4786c6700 1 --2- 172.21.2.101:0/909414007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4580186c0 0x7fd45801abc0 crc :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:00.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.144 7fd4786c6700 1 -- 172.21.2.101:0/909414007 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd470189d00 msgr2=0x7fd47018a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:00.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.144 7fd4786c6700 1 --2- 172.21.2.101:0/909414007 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd470189d00 0x7fd47018a100 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7fd46c00b580 tx=0x7fd46c00b920).stop 2021-02-10T11:31:00.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.144 7fd4786c6700 1 -- 172.21.2.101:0/909414007 shutdown_connections 2021-02-10T11:31:00.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.144 7fd4786c6700 1 --2- 172.21.2.101:0/909414007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4580186c0 0x7fd45801abc0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:00.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.144 7fd4786c6700 1 --2- 172.21.2.101:0/909414007 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd47018a720 0x7fd4700b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:00.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.144 7fd4786c6700 1 --2- 172.21.2.101:0/909414007 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd470189d00 0x7fd47018a100 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:00.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.144 7fd4786c6700 1 --2- 172.21.2.101:0/909414007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4700a0420 0x7fd4700a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:00.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.144 7fd4786c6700 1 -- 172.21.2.101:0/909414007 >> 172.21.2.101:0/909414007 conn(0x7fd470094e10 msgr2=0x7fd4700a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:00.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.144 7fd4786c6700 1 -- 172.21.2.101:0/909414007 shutdown_connections 2021-02-10T11:31:00.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.144 7fd4786c6700 1 -- 172.21.2.101:0/909414007 wait complete. 2021-02-10T11:31:00.163 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839336 2021-02-10T11:31:00.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.567 7f7d041ed700 1 Processor -- start 2021-02-10T11:31:00.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.567 7f7d041ed700 1 -- start start 2021-02-10T11:31:00.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.568 7f7d041ed700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7cfc1b0470 0x7f7cfc1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:00.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.568 7f7d041ed700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cfc0f0b50 con 0x7f7cfc0ef610 2021-02-10T11:31:00.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.568 7f7d041ed700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cfc0f24e0 con 0x7f7cfc0f20e0 2021-02-10T11:31:00.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.568 7f7d041ed700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cfc0e5ea0 con 0x7f7cfc1b0470 2021-02-10T11:31:00.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.568 7f7d02009700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7cfc1b0470 0x7f7cfc1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:00.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.568 7f7d02009700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7cfc1b0470 0x7f7cfc1b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49722/0 (socket says 172.21.2.101:49722) 2021-02-10T11:31:00.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.568 7f7d02009700 1 -- 172.21.2.101:0/3205606634 learned_addr learned my addr 172.21.2.101:0/3205606634 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:00.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.568 7f7d02009700 1 -- 172.21.2.101:0/3205606634 >> v1:172.21.2.108:6789/0 conn(0x7f7cfc0f20e0 legacy=0x7f7cfc0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:00.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.568 7f7d02009700 1 -- 172.21.2.101:0/3205606634 >> v1:172.21.2.101:6789/0 conn(0x7f7cfc0ef610 legacy=0x7f7cfc0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:00.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.568 7f7d02009700 1 -- 172.21.2.101:0/3205606634 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc0e52e0 con 0x7f7cfc1b0470 2021-02-10T11:31:00.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.568 7f7d02009700 1 --2- 172.21.2.101:0/3205606634 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7cfc1b0470 0x7f7cfc1b0870 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 rx=0x7f7cf8006a80 tx=0x7f7cf8009e80).ready entity=mon.0 client_cookie=2e678123e101734e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:00.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.568 7f7d01007700 1 -- 172.21.2.101:0/3205606634 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7cf800b510 con 0x7f7cfc1b0470 2021-02-10T11:31:00.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.569 7f7d01007700 1 -- 172.21.2.101:0/3205606634 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf800b7d0 con 0x7f7cfc1b0470 2021-02-10T11:31:00.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.569 7f7d01007700 1 -- 172.21.2.101:0/3205606634 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7cf800bcb0 con 0x7f7cfc1b0470 2021-02-10T11:31:00.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.569 7f7d041ed700 1 -- 172.21.2.101:0/3205606634 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7cfc1b0470 msgr2=0x7f7cfc1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:00.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.569 7f7d041ed700 1 --2- 172.21.2.101:0/3205606634 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7cfc1b0470 0x7f7cfc1b0870 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 rx=0x7f7cf8006a80 tx=0x7f7cf8009e80).stop 2021-02-10T11:31:00.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.569 7f7d041ed700 1 -- 172.21.2.101:0/3205606634 shutdown_connections 2021-02-10T11:31:00.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.569 7f7d041ed700 1 --2- 172.21.2.101:0/3205606634 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7cfc1b0470 0x7f7cfc1b0870 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:00.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.569 7f7d041ed700 1 -- 172.21.2.101:0/3205606634 >> 172.21.2.101:0/3205606634 conn(0x7f7cfc0ce3e0 msgr2=0x7f7cfc0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:00.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.569 7f7d041ed700 1 -- 172.21.2.101:0/3205606634 shutdown_connections 2021-02-10T11:31:00.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.569 7f7d041ed700 1 -- 172.21.2.101:0/3205606634 wait complete. 2021-02-10T11:31:00.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.584 7f7d041ed700 1 Processor -- start 2021-02-10T11:31:00.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.584 7f7d041ed700 1 -- start start 2021-02-10T11:31:00.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.584 7f7d041ed700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7cfc082170 0x7f7cfc0894a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:00.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.584 7f7d041ed700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7cfc1b01a0 0x7f7cfc1b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:00.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.584 7f7d041ed700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7cfc1b1460 0x7f7cfc1b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:00.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.584 7f7d041ed700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc1bf2a0 con 0x7f7cfc082170 2021-02-10T11:31:00.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.584 7f7d041ed700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc1b0ee0 con 0x7f7cfc1b01a0 2021-02-10T11:31:00.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.584 7f7d041ed700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc1b60c0 con 0x7f7cfc1b1460 2021-02-10T11:31:00.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.584 7f7d02009700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7cfc082170 0x7f7cfc0894a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:00.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.584 7f7d02009700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7cfc082170 0x7f7cfc0894a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49728/0 (socket says 172.21.2.101:49728) 2021-02-10T11:31:00.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.584 7f7d02009700 1 -- 172.21.2.101:0/386603167 learned_addr learned my addr 172.21.2.101:0/386603167 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:00.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.584 7f7d01808700 1 --2- 172.21.2.101:0/386603167 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7cfc1b01a0 0x7f7cfc1b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:00.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.584 7f7d0280a700 1 --2- 172.21.2.101:0/386603167 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7cfc1b1460 0x7f7cfc1b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:00.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.585 7f7d02009700 1 -- 172.21.2.101:0/386603167 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7cfc1b1460 msgr2=0x7f7cfc1b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:00.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.585 7f7d02009700 1 --2- 172.21.2.101:0/386603167 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7cfc1b1460 0x7f7cfc1b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:00.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.585 7f7d02009700 1 -- 172.21.2.101:0/386603167 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7cfc1b01a0 msgr2=0x7f7cfc1b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:00.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.585 7f7d02009700 1 --2- 172.21.2.101:0/386603167 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7cfc1b01a0 0x7f7cfc1b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:00.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.585 7f7d02009700 1 -- 172.21.2.101:0/386603167 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc1b0bc0 con 0x7f7cfc082170 2021-02-10T11:31:00.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.585 7f7d01808700 1 --2- 172.21.2.101:0/386603167 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7cfc1b01a0 0x7f7cfc1b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:31:00.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.585 7f7d02009700 1 --2- 172.21.2.101:0/386603167 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7cfc082170 0x7f7cfc0894a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7f7cf800b970 tx=0x7f7cf800bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:00.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.585 7f7cf2ffd700 1 -- 172.21.2.101:0/386603167 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7cf8000d30 con 0x7f7cfc082170 2021-02-10T11:31:00.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.585 7f7cf2ffd700 1 -- 172.21.2.101:0/386603167 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf8006240 con 0x7f7cfc082170 2021-02-10T11:31:00.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.585 7f7cf2ffd700 1 -- 172.21.2.101:0/386603167 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7cf8005660 con 0x7f7cfc082170 2021-02-10T11:31:00.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.585 7f7d041ed700 1 -- 172.21.2.101:0/386603167 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cfc1b0bc0 con 0x7f7cfc082170 2021-02-10T11:31:00.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.585 7f7d041ed700 1 -- 172.21.2.101:0/386603167 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cfc1b1e80 con 0x7f7cfc082170 2021-02-10T11:31:00.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.586 7f7cf2ffd700 1 -- 172.21.2.101:0/386603167 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7cf8006420 con 0x7f7cfc082170 2021-02-10T11:31:00.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.586 7f7cf2ffd700 1 --2- 172.21.2.101:0/386603167 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ce4018730 0x7f7ce401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:00.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.586 7f7cf2ffd700 1 -- 172.21.2.101:0/386603167 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7cf8032a40 con 0x7f7cfc082170 2021-02-10T11:31:00.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.586 7f7d01808700 1 --2- 172.21.2.101:0/386603167 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ce4018730 0x7f7ce401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:00.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.586 7f7d01808700 1 --2- 172.21.2.101:0/386603167 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ce4018730 0x7f7ce401abc0 crc :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:00.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.617 7f7d041ed700 1 -- 172.21.2.101:0/386603167 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cfc1bf2a0 con 0x7f7cfc082170 2021-02-10T11:31:00.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.619 7f7cf2ffd700 1 -- 172.21.2.101:0/386603167 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7cf8000d30 con 0x7f7cfc082170 2021-02-10T11:31:00.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.748 7f7d041ed700 1 -- 172.21.2.101:0/386603167 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839336", "target": ["mgr", ""]}) v1 -- 0x7f7cfc1b1e80 con 0x7f7ce4018730 2021-02-10T11:31:00.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.759 7f7cf2ffd700 1 -- 172.21.2.101:0/386603167 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f7cec006be0 con 0x7f7ce4018730 2021-02-10T11:31:00.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.765 7f7d041ed700 1 -- 172.21.2.101:0/386603167 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ce4018730 msgr2=0x7f7ce401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:00.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.765 7f7d041ed700 1 --2- 172.21.2.101:0/386603167 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ce4018730 0x7f7ce401abc0 crc :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:00.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.765 7f7d041ed700 1 -- 172.21.2.101:0/386603167 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7cfc082170 msgr2=0x7f7cfc0894a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:00.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.765 7f7d041ed700 1 --2- 172.21.2.101:0/386603167 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7cfc082170 0x7f7cfc0894a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7f7cf800b970 tx=0x7f7cf800bd10).stop 2021-02-10T11:31:00.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.765 7f7d041ed700 1 -- 172.21.2.101:0/386603167 shutdown_connections 2021-02-10T11:31:00.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.765 7f7d041ed700 1 --2- 172.21.2.101:0/386603167 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ce4018730 0x7f7ce401abc0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:00.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.765 7f7d041ed700 1 --2- 172.21.2.101:0/386603167 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7cfc1b1460 0x7f7cfc1b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:00.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.765 7f7d041ed700 1 --2- 172.21.2.101:0/386603167 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7cfc1b01a0 0x7f7cfc1b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:00.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.765 7f7d041ed700 1 --2- 172.21.2.101:0/386603167 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7cfc082170 0x7f7cfc0894a0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:00.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.765 7f7d041ed700 1 -- 172.21.2.101:0/386603167 >> 172.21.2.101:0/386603167 conn(0x7f7cfc088d10 msgr2=0x7f7cfc0afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:00.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.765 7f7d041ed700 1 -- 172.21.2.101:0/386603167 shutdown_connections 2021-02-10T11:31:00.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:00.765 7f7d041ed700 1 -- 172.21.2.101:0/386603167 wait complete. 2021-02-10T11:31:00.784 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839337 2021-02-10T11:31:01.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.188 7fc841da8700 1 Processor -- start 2021-02-10T11:31:01.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.188 7fc841da8700 1 -- start start 2021-02-10T11:31:01.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.188 7fc841da8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc83c1b0470 0x7fc83c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:01.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.188 7fc841da8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc83c0ef610 0x7fc83c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:01.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.188 7fc841da8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc83c0f0b50 con 0x7fc83c0f0030 2021-02-10T11:31:01.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.188 7fc841da8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc83c0e52e0 con 0x7fc83c0ef610 2021-02-10T11:31:01.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.188 7fc841da8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc83c0e5490 con 0x7fc83c1b0470 2021-02-10T11:31:01.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.188 7fc83affd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc83c0ef610 0x7fc83c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:01.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.188 7fc83affd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc83c0ef610 0x7fc83c0efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49736/0 (socket says 172.21.2.101:49736) 2021-02-10T11:31:01.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.188 7fc83affd700 1 -- 172.21.2.101:0/1795536338 learned_addr learned my addr 172.21.2.101:0/1795536338 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:01.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.189 7fc83affd700 1 -- 172.21.2.101:0/1795536338 >> v2:172.21.2.108:3300/0 conn(0x7fc83c1b0470 msgr2=0x7fc83c1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:01.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.189 7fc83b7fe700 1 --2- 172.21.2.101:0/1795536338 >> v2:172.21.2.108:3300/0 conn(0x7fc83c1b0470 0x7fc83c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:01.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.189 7fc83affd700 1 --2- 172.21.2.101:0/1795536338 >> v2:172.21.2.108:3300/0 conn(0x7fc83c1b0470 0x7fc83c1b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:01.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.189 7fc83affd700 1 -- 172.21.2.101:0/1795536338 >> v1:172.21.2.108:6789/0 conn(0x7fc83c0f0030 legacy=0x7fc83c0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:01.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.189 7fc83affd700 1 -- 172.21.2.101:0/1795536338 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83c0f0430 con 0x7fc83c0ef610 2021-02-10T11:31:01.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.189 7fc83b7fe700 1 --2- 172.21.2.101:0/1795536338 >> v2:172.21.2.108:3300/0 conn(0x7fc83c1b0470 0x7fc83c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:31:01.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.189 7fc83affd700 1 --2- 172.21.2.101:0/1795536338 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc83c0ef610 0x7fc83c0efa10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7fc834006a80 tx=0x7fc834009f00).ready entity=mon.0 client_cookie=54865842ae0129ab server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:01.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.189 7fc83a7fc700 1 -- 172.21.2.101:0/1795536338 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc83400b5c0 con 0x7fc83c0ef610 2021-02-10T11:31:01.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.189 7fc83a7fc700 1 -- 172.21.2.101:0/1795536338 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83400b800 con 0x7fc83c0ef610 2021-02-10T11:31:01.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.189 7fc83a7fc700 1 -- 172.21.2.101:0/1795536338 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc83400bd20 con 0x7fc83c0ef610 2021-02-10T11:31:01.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.189 7fc841da8700 1 -- 172.21.2.101:0/1795536338 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc83c0ef610 msgr2=0x7fc83c0efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:01.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.189 7fc841da8700 1 --2- 172.21.2.101:0/1795536338 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc83c0ef610 0x7fc83c0efa10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7fc834006a80 tx=0x7fc834009f00).stop 2021-02-10T11:31:01.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.190 7fc841da8700 1 -- 172.21.2.101:0/1795536338 shutdown_connections 2021-02-10T11:31:01.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.190 7fc841da8700 1 --2- 172.21.2.101:0/1795536338 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc83c0ef610 0x7fc83c0efa10 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:01.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.190 7fc841da8700 1 --2- 172.21.2.101:0/1795536338 >> v2:172.21.2.108:3300/0 conn(0x7fc83c1b0470 0x7fc83c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:01.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.190 7fc841da8700 1 -- 172.21.2.101:0/1795536338 >> 172.21.2.101:0/1795536338 conn(0x7fc83c0ce3e0 msgr2=0x7fc83c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:01.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.190 7fc841da8700 1 -- 172.21.2.101:0/1795536338 shutdown_connections 2021-02-10T11:31:01.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.190 7fc841da8700 1 -- 172.21.2.101:0/1795536338 wait complete. 2021-02-10T11:31:01.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.205 7fc841da8700 1 Processor -- start 2021-02-10T11:31:01.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.205 7fc841da8700 1 -- start start 2021-02-10T11:31:01.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.205 7fc841da8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc83c190920 0x7fc83c08f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:01.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.205 7fc841da8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc83c190d20 0x7fc83c08fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:01.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.205 7fc841da8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc83c193270 0x7fc83c193670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:01.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.205 7fc841da8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83c090470 con 0x7fc83c190d20 2021-02-10T11:31:01.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.205 7fc841da8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83c193c90 con 0x7fc83c193270 2021-02-10T11:31:01.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.205 7fc841da8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc83c193e70 con 0x7fc83c190920 2021-02-10T11:31:01.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.206 7fc83affd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc83c190d20 0x7fc83c08fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:01.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.206 7fc83affd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc83c190d20 0x7fc83c08fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49742/0 (socket says 172.21.2.101:49742) 2021-02-10T11:31:01.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.206 7fc83affd700 1 -- 172.21.2.101:0/3455098416 learned_addr learned my addr 172.21.2.101:0/3455098416 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:01.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.206 7fc83bfff700 1 --2- 172.21.2.101:0/3455098416 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc83c193270 0x7fc83c193670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:01.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.206 7fc83b7fe700 1 --2- 172.21.2.101:0/3455098416 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc83c190920 0x7fc83c08f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:01.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.206 7fc83affd700 1 -- 172.21.2.101:0/3455098416 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc83c190920 msgr2=0x7fc83c08f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:01.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.206 7fc83affd700 1 --2- 172.21.2.101:0/3455098416 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc83c190920 0x7fc83c08f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:01.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.206 7fc83affd700 1 -- 172.21.2.101:0/3455098416 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc83c193270 msgr2=0x7fc83c193670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:01.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.206 7fc83affd700 1 --2- 172.21.2.101:0/3455098416 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc83c193270 0x7fc83c193670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:01.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.206 7fc83affd700 1 -- 172.21.2.101:0/3455098416 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83c194090 con 0x7fc83c190d20 2021-02-10T11:31:01.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.206 7fc83affd700 1 --2- 172.21.2.101:0/3455098416 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc83c190d20 0x7fc83c08fc20 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0x7fc83400b970 tx=0x7fc83400bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:01.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.206 7fc823fff700 1 -- 172.21.2.101:0/3455098416 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc834000d30 con 0x7fc83c190d20 2021-02-10T11:31:01.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.206 7fc823fff700 1 -- 172.21.2.101:0/3455098416 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83400c2f0 con 0x7fc83c190d20 2021-02-10T11:31:01.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.206 7fc823fff700 1 -- 172.21.2.101:0/3455098416 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc83400c750 con 0x7fc83c190d20 2021-02-10T11:31:01.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.206 7fc841da8700 1 -- 172.21.2.101:0/3455098416 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc83c194090 con 0x7fc83c190d20 2021-02-10T11:31:01.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.206 7fc841da8700 1 -- 172.21.2.101:0/3455098416 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc83c1920b0 con 0x7fc83c190d20 2021-02-10T11:31:01.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.207 7fc823fff700 1 -- 172.21.2.101:0/3455098416 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc834001010 con 0x7fc83c190d20 2021-02-10T11:31:01.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.207 7fc823fff700 1 --2- 172.21.2.101:0/3455098416 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc81c0186c0 0x7fc81c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:01.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.207 7fc823fff700 1 -- 172.21.2.101:0/3455098416 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc834032b10 con 0x7fc83c190d20 2021-02-10T11:31:01.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.207 7fc83b7fe700 1 --2- 172.21.2.101:0/3455098416 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc81c0186c0 0x7fc81c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:01.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.207 7fc83b7fe700 1 --2- 172.21.2.101:0/3455098416 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc81c0186c0 0x7fc81c01abc0 crc :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:01.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.237 7fc841da8700 1 -- 172.21.2.101:0/3455098416 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc83c105af0 con 0x7fc83c190d20 2021-02-10T11:31:01.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.239 7fc823fff700 1 -- 172.21.2.101:0/3455098416 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc834000d30 con 0x7fc83c190d20 2021-02-10T11:31:01.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.370 7fc841da8700 1 -- 172.21.2.101:0/3455098416 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839337", "target": ["mgr", ""]}) v1 -- 0x7fc83c1920b0 con 0x7fc81c0186c0 2021-02-10T11:31:01.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.382 7fc823fff700 1 -- 172.21.2.101:0/3455098416 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fc830006540 con 0x7fc81c0186c0 2021-02-10T11:31:01.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.386 7fc841da8700 1 -- 172.21.2.101:0/3455098416 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc81c0186c0 msgr2=0x7fc81c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:01.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.386 7fc841da8700 1 --2- 172.21.2.101:0/3455098416 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc81c0186c0 0x7fc81c01abc0 crc :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:01.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.386 7fc841da8700 1 -- 172.21.2.101:0/3455098416 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc83c190d20 msgr2=0x7fc83c08fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:01.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.386 7fc841da8700 1 --2- 172.21.2.101:0/3455098416 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc83c190d20 0x7fc83c08fc20 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0x7fc83400b970 tx=0x7fc83400bd10).stop 2021-02-10T11:31:01.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.387 7fc841da8700 1 -- 172.21.2.101:0/3455098416 shutdown_connections 2021-02-10T11:31:01.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.387 7fc841da8700 1 --2- 172.21.2.101:0/3455098416 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc83c193270 0x7fc83c193670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:01.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.387 7fc841da8700 1 --2- 172.21.2.101:0/3455098416 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc83c190d20 0x7fc83c08fc20 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:01.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.387 7fc841da8700 1 --2- 172.21.2.101:0/3455098416 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc81c0186c0 0x7fc81c01abc0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:01.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.387 7fc841da8700 1 --2- 172.21.2.101:0/3455098416 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc83c190920 0x7fc83c08f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:01.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.387 7fc841da8700 1 -- 172.21.2.101:0/3455098416 >> 172.21.2.101:0/3455098416 conn(0x7fc83c0bf4e0 msgr2=0x7fc83c0be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:01.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.387 7fc841da8700 1 -- 172.21.2.101:0/3455098416 shutdown_connections 2021-02-10T11:31:01.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.387 7fc841da8700 1 -- 172.21.2.101:0/3455098416 wait complete. 2021-02-10T11:31:01.407 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000839338 2021-02-10T11:31:01.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.809 7fbf25f5c700 1 Processor -- start 2021-02-10T11:31:01.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.809 7fbf25f5c700 1 -- start start 2021-02-10T11:31:01.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.809 7fbf25f5c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbf201b0470 0x7fbf201b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:01.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.809 7fbf25f5c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbf200f20e0 0x7fbf200f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:01.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.809 7fbf25f5c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf200f0b50 con 0x7fbf200ef610 2021-02-10T11:31:01.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.809 7fbf25f5c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf200f0820 con 0x7fbf200f20e0 2021-02-10T11:31:01.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.809 7fbf25f5c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf200f24e0 con 0x7fbf201b0470 2021-02-10T11:31:01.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.810 7fbf1ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbf200f20e0 0x7fbf200f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:01.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.810 7fbf1f7fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbf201b0470 0x7fbf201b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:01.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.810 7fbf1effd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fbf200ef610 0x7fbf200efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50246/0 (socket says 172.21.2.101:50246) 2021-02-10T11:31:01.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.810 7fbf1ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbf200f20e0 0x7fbf200f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49752/0 (socket says 172.21.2.101:49752) 2021-02-10T11:31:01.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.810 7fbf1ffff700 1 -- 172.21.2.101:0/3525548673 learned_addr learned my addr 172.21.2.101:0/3525548673 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:01.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.810 7fbf1ffff700 1 -- 172.21.2.101:0/3525548673 >> v2:172.21.2.108:3300/0 conn(0x7fbf201b0470 msgr2=0x7fbf201b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:01.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.810 7fbf1ffff700 1 --2- 172.21.2.101:0/3525548673 >> v2:172.21.2.108:3300/0 conn(0x7fbf201b0470 0x7fbf201b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:01.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.810 7fbf1ffff700 1 -- 172.21.2.101:0/3525548673 >> v1:172.21.2.101:6789/0 conn(0x7fbf200ef610 legacy=0x7fbf200efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:01.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.810 7fbf1ffff700 1 -- 172.21.2.101:0/3525548673 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf200e52e0 con 0x7fbf200f20e0 2021-02-10T11:31:01.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.810 7fbf1ffff700 1 --2- 172.21.2.101:0/3525548673 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf200f20e0 0x7fbf200f0200 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7fbf18006f00 tx=0x7fbf18009f10).ready entity=mon.0 client_cookie=ebc50b8dfbe2c506 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:01.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.810 7fbf1e7fc700 1 -- 172.21.2.101:0/3525548673 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbf1800b3b0 con 0x7fbf200f20e0 2021-02-10T11:31:01.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.810 7fbf1e7fc700 1 -- 172.21.2.101:0/3525548673 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1800b730 con 0x7fbf200f20e0 2021-02-10T11:31:01.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.810 7fbf1e7fc700 1 -- 172.21.2.101:0/3525548673 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbf1800bc10 con 0x7fbf200f20e0 2021-02-10T11:31:01.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.810 7fbf25f5c700 1 -- 172.21.2.101:0/3525548673 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf200f20e0 msgr2=0x7fbf200f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:01.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.810 7fbf25f5c700 1 --2- 172.21.2.101:0/3525548673 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf200f20e0 0x7fbf200f0200 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7fbf18006f00 tx=0x7fbf18009f10).stop 2021-02-10T11:31:01.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.811 7fbf25f5c700 1 -- 172.21.2.101:0/3525548673 shutdown_connections 2021-02-10T11:31:01.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.811 7fbf25f5c700 1 --2- 172.21.2.101:0/3525548673 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf200f20e0 0x7fbf200f0200 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:01.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.811 7fbf25f5c700 1 --2- 172.21.2.101:0/3525548673 >> v2:172.21.2.108:3300/0 conn(0x7fbf201b0470 0x7fbf201b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:01.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.811 7fbf25f5c700 1 -- 172.21.2.101:0/3525548673 >> 172.21.2.101:0/3525548673 conn(0x7fbf200ce3e0 msgr2=0x7fbf200e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:01.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.811 7fbf25f5c700 1 -- 172.21.2.101:0/3525548673 shutdown_connections 2021-02-10T11:31:01.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.811 7fbf25f5c700 1 -- 172.21.2.101:0/3525548673 wait complete. 2021-02-10T11:31:01.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.826 7fbf25f5c700 1 Processor -- start 2021-02-10T11:31:01.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.826 7fbf25f5c700 1 -- start start 2021-02-10T11:31:01.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.826 7fbf25f5c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf200a0420 0x7fbf200a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:01.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.826 7fbf25f5c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf20189d00 0x7fbf2018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:01.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.826 7fbf25f5c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbf2018a720 0x7fbf200b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:01.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.826 7fbf25f5c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf20189800 con 0x7fbf200a0420 2021-02-10T11:31:01.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.826 7fbf25f5c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2018ab20 con 0x7fbf2018a720 2021-02-10T11:31:01.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.826 7fbf25f5c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf200a1140 con 0x7fbf20189d00 2021-02-10T11:31:01.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.826 7fbf1f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf200a0420 0x7fbf200a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:01.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.827 7fbf1f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf200a0420 0x7fbf200a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49758/0 (socket says 172.21.2.101:49758) 2021-02-10T11:31:01.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.827 7fbf1f7fe700 1 -- 172.21.2.101:0/3575084176 learned_addr learned my addr 172.21.2.101:0/3575084176 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:01.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.827 7fbf1effd700 1 --2- 172.21.2.101:0/3575084176 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf20189d00 0x7fbf2018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:01.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.827 7fbf1ffff700 1 --2- 172.21.2.101:0/3575084176 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbf2018a720 0x7fbf200b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:01.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.827 7fbf1f7fe700 1 -- 172.21.2.101:0/3575084176 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf20189d00 msgr2=0x7fbf2018a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:01.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.827 7fbf1f7fe700 1 --2- 172.21.2.101:0/3575084176 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf20189d00 0x7fbf2018a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:01.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.827 7fbf1f7fe700 1 -- 172.21.2.101:0/3575084176 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbf2018a720 msgr2=0x7fbf200b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:01.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.827 7fbf1f7fe700 1 --2- 172.21.2.101:0/3575084176 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbf2018a720 0x7fbf200b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:01.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.827 7fbf1f7fe700 1 -- 172.21.2.101:0/3575084176 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf200b8200 con 0x7fbf200a0420 2021-02-10T11:31:01.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.827 7fbf1f7fe700 1 --2- 172.21.2.101:0/3575084176 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf200a0420 0x7fbf200a0820 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0x7fbf08006f60 tx=0x7fbf0800b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:01.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.827 7fbf17fff700 1 -- 172.21.2.101:0/3575084176 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbf0800bd60 con 0x7fbf200a0420 2021-02-10T11:31:01.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.827 7fbf17fff700 1 -- 172.21.2.101:0/3575084176 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf08000c90 con 0x7fbf200a0420 2021-02-10T11:31:01.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.827 7fbf25f5c700 1 -- 172.21.2.101:0/3575084176 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf200b8200 con 0x7fbf200a0420 2021-02-10T11:31:01.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.827 7fbf25f5c700 1 -- 172.21.2.101:0/3575084176 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf200b8790 con 0x7fbf200a0420 2021-02-10T11:31:01.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.827 7fbf17fff700 1 -- 172.21.2.101:0/3575084176 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbf080022b0 con 0x7fbf200a0420 2021-02-10T11:31:01.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.828 7fbf17fff700 1 -- 172.21.2.101:0/3575084176 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbf08006b00 con 0x7fbf200a0420 2021-02-10T11:31:01.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.828 7fbf17fff700 1 --2- 172.21.2.101:0/3575084176 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf040186c0 0x7fbf0401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:01.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.828 7fbf17fff700 1 -- 172.21.2.101:0/3575084176 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fbf08032520 con 0x7fbf200a0420 2021-02-10T11:31:01.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.828 7fbf1effd700 1 --2- 172.21.2.101:0/3575084176 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf040186c0 0x7fbf0401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:01.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.828 7fbf1effd700 1 --2- 172.21.2.101:0/3575084176 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf040186c0 0x7fbf0401abc0 crc :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:01.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.859 7fbf25f5c700 1 -- 172.21.2.101:0/3575084176 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef8002fd0 con 0x7fbf200a0420 2021-02-10T11:31:01.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.861 7fbf17fff700 1 -- 172.21.2.101:0/3575084176 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbf0802f030 con 0x7fbf200a0420 2021-02-10T11:31:02.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:01.989 7fbf25f5c700 1 -- 172.21.2.101:0/3575084176 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000839338", "target": ["mgr", ""]}) v1 -- 0x7fbef8002f80 con 0x7fbf040186c0 2021-02-10T11:31:02.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.000 7fbf17fff700 1 -- 172.21.2.101:0/3575084176 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fbf10000e20 con 0x7fbf040186c0 2021-02-10T11:31:02.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.006 7fbf25f5c700 1 -- 172.21.2.101:0/3575084176 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf040186c0 msgr2=0x7fbf0401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:02.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.006 7fbf25f5c700 1 --2- 172.21.2.101:0/3575084176 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf040186c0 0x7fbf0401abc0 crc :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:02.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.006 7fbf25f5c700 1 -- 172.21.2.101:0/3575084176 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf200a0420 msgr2=0x7fbf200a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:02.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.006 7fbf25f5c700 1 --2- 172.21.2.101:0/3575084176 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf200a0420 0x7fbf200a0820 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0x7fbf08006f60 tx=0x7fbf0800b970).stop 2021-02-10T11:31:02.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.006 7fbf25f5c700 1 -- 172.21.2.101:0/3575084176 shutdown_connections 2021-02-10T11:31:02.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.006 7fbf25f5c700 1 --2- 172.21.2.101:0/3575084176 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf040186c0 0x7fbf0401abc0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:02.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.006 7fbf25f5c700 1 --2- 172.21.2.101:0/3575084176 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf20189d00 0x7fbf2018a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:02.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.006 7fbf25f5c700 1 --2- 172.21.2.101:0/3575084176 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbf2018a720 0x7fbf200b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:02.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.006 7fbf25f5c700 1 --2- 172.21.2.101:0/3575084176 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf200a0420 0x7fbf200a0820 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:02.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.006 7fbf25f5c700 1 -- 172.21.2.101:0/3575084176 >> 172.21.2.101:0/3575084176 conn(0x7fbf20094e10 msgr2=0x7fbf200a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:02.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.007 7fbf25f5c700 1 -- 172.21.2.101:0/3575084176 shutdown_connections 2021-02-10T11:31:02.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.007 7fbf25f5c700 1 -- 172.21.2.101:0/3575084176 wait complete. 2021-02-10T11:31:02.026 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:31:02.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.427 7f35ecc93700 1 Processor -- start 2021-02-10T11:31:02.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.427 7f35ecc93700 1 -- start start 2021-02-10T11:31:02.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.428 7f35ecc93700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35e80e5470 0x7f35e807e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:02.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.428 7f35ecc93700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f35e807f300 0x7f35e807f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:02.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.428 7f35ecc93700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e807ed80 con 0x7f35e80e5070 2021-02-10T11:31:02.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.428 7f35ecc93700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e81b4010 con 0x7f35e807f300 2021-02-10T11:31:02.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.428 7f35ecc93700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e80d1790 con 0x7f35e80e5470 2021-02-10T11:31:02.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.428 7f35e705f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f35e807f300 0x7f35e807f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:02.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.428 7f35e705f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f35e807f300 0x7f35e807f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49766/0 (socket says 172.21.2.101:49766) 2021-02-10T11:31:02.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.428 7f35e705f700 1 -- 172.21.2.101:0/2191693871 learned_addr learned my addr 172.21.2.101:0/2191693871 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:02.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.428 7f35e705f700 1 -- 172.21.2.101:0/2191693871 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35e80e5470 msgr2=0x7f35e807e6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:31:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.428 7f35e705f700 1 --2- 172.21.2.101:0/2191693871 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35e80e5470 0x7f35e807e6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.428 7f35e705f700 1 -- 172.21.2.101:0/2191693871 >> v1:172.21.2.108:6789/0 conn(0x7f35e80e5070 legacy=0x7f35e81b3820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:31:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.428 7f35e705f700 1 -- 172.21.2.101:0/2191693871 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e80d1980 con 0x7f35e807f300 2021-02-10T11:31:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.428 7f35e705f700 1 --2- 172.21.2.101:0/2191693871 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35e807f300 0x7f35e807f700 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7f35e0006f00 tx=0x7f35e0009f10).ready entity=mon.0 client_cookie=b4cff5d4aeebd53b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.428 7f35e585c700 1 -- 172.21.2.101:0/2191693871 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f35e000b3b0 con 0x7f35e807f300 2021-02-10T11:31:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.428 7f35e585c700 1 -- 172.21.2.101:0/2191693871 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e000b630 con 0x7f35e807f300 2021-02-10T11:31:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.428 7f35e585c700 1 -- 172.21.2.101:0/2191693871 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f35e000bb50 con 0x7f35e807f300 2021-02-10T11:31:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.428 7f35ecc93700 1 -- 172.21.2.101:0/2191693871 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35e807f300 msgr2=0x7f35e807f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.428 7f35ecc93700 1 --2- 172.21.2.101:0/2191693871 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35e807f300 0x7f35e807f700 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7f35e0006f00 tx=0x7f35e0009f10).stop 2021-02-10T11:31:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.429 7f35ecc93700 1 -- 172.21.2.101:0/2191693871 shutdown_connections 2021-02-10T11:31:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.429 7f35ecc93700 1 --2- 172.21.2.101:0/2191693871 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35e80e5470 0x7f35e807e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.429 7f35ecc93700 1 --2- 172.21.2.101:0/2191693871 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35e807f300 0x7f35e807f700 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.429 7f35ecc93700 1 -- 172.21.2.101:0/2191693871 >> 172.21.2.101:0/2191693871 conn(0x7f35e80ce690 msgr2=0x7f35e80e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.429 7f35ecc93700 1 -- 172.21.2.101:0/2191693871 shutdown_connections 2021-02-10T11:31:02.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.429 7f35ecc93700 1 -- 172.21.2.101:0/2191693871 wait complete. 2021-02-10T11:31:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.444 7f35ecc93700 1 Processor -- start 2021-02-10T11:31:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.444 7f35ecc93700 1 -- start start 2021-02-10T11:31:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.445 7f35ecc93700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35e80add70 0x7f35e8085290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.445 7f35ecc93700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f35e80858b0 0x7f35e8198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.445 7f35ecc93700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35e80ae170 0x7f35e8199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.445 7f35ecc93700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e81ab000 con 0x7f35e80add70 2021-02-10T11:31:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.445 7f35ecc93700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e81a5610 con 0x7f35e80858b0 2021-02-10T11:31:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.445 7f35ecc93700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e81985a0 con 0x7f35e80ae170 2021-02-10T11:31:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.445 7f35e685e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35e80add70 0x7f35e8085290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.445 7f35e685e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35e80add70 0x7f35e8085290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49774/0 (socket says 172.21.2.101:49774) 2021-02-10T11:31:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.445 7f35e685e700 1 -- 172.21.2.101:0/960165418 learned_addr learned my addr 172.21.2.101:0/960165418 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.445 7f35e705f700 1 --2- 172.21.2.101:0/960165418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35e80ae170 0x7f35e8199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.445 7f35e605d700 1 --2- 172.21.2.101:0/960165418 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f35e80858b0 0x7f35e8198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.445 7f35e685e700 1 -- 172.21.2.101:0/960165418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35e80ae170 msgr2=0x7f35e8199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.445 7f35e685e700 1 --2- 172.21.2.101:0/960165418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35e80ae170 0x7f35e8199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.445 7f35e685e700 1 -- 172.21.2.101:0/960165418 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f35e80858b0 msgr2=0x7f35e8198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.445 7f35e685e700 1 --2- 172.21.2.101:0/960165418 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f35e80858b0 0x7f35e8198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.445 7f35e685e700 1 -- 172.21.2.101:0/960165418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e81a7bf0 con 0x7f35e80add70 2021-02-10T11:31:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.445 7f35e605d700 1 --2- 172.21.2.101:0/960165418 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f35e80858b0 0x7f35e8198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:31:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.445 7f35e685e700 1 --2- 172.21.2.101:0/960165418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35e80add70 0x7f35e8085290 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7f35d800b530 tx=0x7f35d800b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.445 7f35d77fe700 1 -- 172.21.2.101:0/960165418 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f35d800bcc0 con 0x7f35e80add70 2021-02-10T11:31:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.445 7f35d77fe700 1 -- 172.21.2.101:0/960165418 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d80009b0 con 0x7f35e80add70 2021-02-10T11:31:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.445 7f35ecc93700 1 -- 172.21.2.101:0/960165418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e81a7bf0 con 0x7f35e80add70 2021-02-10T11:31:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.445 7f35ecc93700 1 -- 172.21.2.101:0/960165418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e81a7bf0 con 0x7f35e80add70 2021-02-10T11:31:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.446 7f35d77fe700 1 -- 172.21.2.101:0/960165418 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f35d80062a0 con 0x7f35e80add70 2021-02-10T11:31:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.446 7f35d77fe700 1 -- 172.21.2.101:0/960165418 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f35d8006270 con 0x7f35e80add70 2021-02-10T11:31:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.446 7f35d77fe700 1 --2- 172.21.2.101:0/960165418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f35cc018730 0x7f35cc01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.446 7f35d77fe700 1 -- 172.21.2.101:0/960165418 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f35d8032800 con 0x7f35e80add70 2021-02-10T11:31:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.446 7f35e605d700 1 --2- 172.21.2.101:0/960165418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f35cc018730 0x7f35cc01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.446 7f35e605d700 1 --2- 172.21.2.101:0/960165418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f35cc018730 0x7f35cc01ac10 crc :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:02.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.477 7f35ecc93700 1 -- 172.21.2.101:0/960165418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e807ed80 con 0x7f35e80add70 2021-02-10T11:31:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.480 7f35d77fe700 1 -- 172.21.2.101:0/960165418 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f35d800bcc0 con 0x7f35e80add70 2021-02-10T11:31:02.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.605 7f35ecc93700 1 -- 172.21.2.101:0/960165418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35e81ab000 con 0x7f35e80add70 2021-02-10T11:31:02.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.605 7f35d77fe700 1 -- 172.21.2.101:0/960165418 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+13993 (secure 0 0 0) 0x7f35d8006270 con 0x7f35e80add70 2021-02-10T11:31:02.617 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:31:02.617 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":29,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:14:58.132054","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:14:54.652789","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2021-02-10 11:14:56.125641","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:31:02.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.607 7f35ecc93700 1 -- 172.21.2.101:0/960165418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f35cc018730 msgr2=0x7f35cc01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:02.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.607 7f35ecc93700 1 --2- 172.21.2.101:0/960165418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f35cc018730 0x7f35cc01ac10 crc :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:02.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.607 7f35ecc93700 1 -- 172.21.2.101:0/960165418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35e80add70 msgr2=0x7f35e8085290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:02.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.607 7f35ecc93700 1 --2- 172.21.2.101:0/960165418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35e80add70 0x7f35e8085290 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7f35d800b530 tx=0x7f35d800b8d0).stop 2021-02-10T11:31:02.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.607 7f35ecc93700 1 -- 172.21.2.101:0/960165418 shutdown_connections 2021-02-10T11:31:02.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.607 7f35ecc93700 1 --2- 172.21.2.101:0/960165418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f35cc018730 0x7f35cc01ac10 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:02.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.607 7f35ecc93700 1 --2- 172.21.2.101:0/960165418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35e80ae170 0x7f35e8199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:02.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.607 7f35ecc93700 1 --2- 172.21.2.101:0/960165418 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f35e80858b0 0x7f35e8198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:02.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.607 7f35ecc93700 1 --2- 172.21.2.101:0/960165418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35e80add70 0x7f35e8085290 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:02.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.607 7f35ecc93700 1 -- 172.21.2.101:0/960165418 >> 172.21.2.101:0/960165418 conn(0x7f35e819b160 msgr2=0x7f35e8187eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:02.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.607 7f35ecc93700 1 -- 172.21.2.101:0/960165418 shutdown_connections 2021-02-10T11:31:02.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:02.607 7f35ecc93700 1 -- 172.21.2.101:0/960165418 wait complete. 2021-02-10T11:31:02.627 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:14:54.652789', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:14:56.125641', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:31:02.627 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:31:03.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.029 7f69d7903700 1 Processor -- start 2021-02-10T11:31:03.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.029 7f69d7903700 1 -- start start 2021-02-10T11:31:03.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.030 7f69d7903700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f69d01b0870 0x7f69d00f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:03.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.030 7f69d7903700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f69d00efe00 0x7f69d00f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:03.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.030 7f69d7903700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69d01b0c70 con 0x7f69d01b0470 2021-02-10T11:31:03.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.030 7f69d7903700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69d00e5ea0 con 0x7f69d01b0870 2021-02-10T11:31:03.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.030 7f69d7903700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69d00f0b50 con 0x7f69d00efe00 2021-02-10T11:31:03.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.030 7f69d4f1e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f69d01b0870 0x7f69d00f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:03.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.030 7f69d4f1e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f69d01b0870 0x7f69d00f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49778/0 (socket says 172.21.2.101:49778) 2021-02-10T11:31:03.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.030 7f69d4f1e700 1 -- 172.21.2.101:0/453677275 learned_addr learned my addr 172.21.2.101:0/453677275 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:03.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.030 7f69d5f20700 1 --2- 172.21.2.101:0/453677275 >> v2:172.21.2.108:3300/0 conn(0x7f69d00efe00 0x7f69d00f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:03.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.030 7f69d4f1e700 1 -- 172.21.2.101:0/453677275 >> v2:172.21.2.108:3300/0 conn(0x7f69d00efe00 msgr2=0x7f69d00f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:03.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.030 7f69d4f1e700 1 --2- 172.21.2.101:0/453677275 >> v2:172.21.2.108:3300/0 conn(0x7f69d00efe00 0x7f69d00f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:03.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.030 7f69d4f1e700 1 -- 172.21.2.101:0/453677275 >> v1:172.21.2.108:6789/0 conn(0x7f69d01b0470 legacy=0x7f69d00ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:03.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.030 7f69d4f1e700 1 -- 172.21.2.101:0/453677275 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d00f0820 con 0x7f69d01b0870 2021-02-10T11:31:03.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.030 7f69d4f1e700 1 --2- 172.21.2.101:0/453677275 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f69d01b0870 0x7f69d00f20e0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7f69cc006700 tx=0x7f69cc009b80).ready entity=mon.0 client_cookie=ff08e470c57bb367 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:03.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.031 7f69c3fff700 1 -- 172.21.2.101:0/453677275 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f69cc00c0c0 con 0x7f69d01b0870 2021-02-10T11:31:03.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.031 7f69c3fff700 1 -- 172.21.2.101:0/453677275 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69cc00c340 con 0x7f69d01b0870 2021-02-10T11:31:03.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.031 7f69c3fff700 1 -- 172.21.2.101:0/453677275 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f69cc00c860 con 0x7f69d01b0870 2021-02-10T11:31:03.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.031 7f69d7903700 1 -- 172.21.2.101:0/453677275 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f69d01b0870 msgr2=0x7f69d00f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:03.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.031 7f69d7903700 1 --2- 172.21.2.101:0/453677275 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f69d01b0870 0x7f69d00f20e0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7f69cc006700 tx=0x7f69cc009b80).stop 2021-02-10T11:31:03.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.031 7f69d7903700 1 -- 172.21.2.101:0/453677275 shutdown_connections 2021-02-10T11:31:03.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.031 7f69d7903700 1 --2- 172.21.2.101:0/453677275 >> v2:172.21.2.108:3300/0 conn(0x7f69d00efe00 0x7f69d00f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:03.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.031 7f69d7903700 1 --2- 172.21.2.101:0/453677275 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f69d01b0870 0x7f69d00f20e0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:03.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.031 7f69d7903700 1 -- 172.21.2.101:0/453677275 >> 172.21.2.101:0/453677275 conn(0x7f69d00ce3e0 msgr2=0x7f69d00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:03.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.032 7f69d7903700 1 -- 172.21.2.101:0/453677275 shutdown_connections 2021-02-10T11:31:03.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.032 7f69d7903700 1 -- 172.21.2.101:0/453677275 wait complete. 2021-02-10T11:31:03.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.046 7f69d7903700 1 Processor -- start 2021-02-10T11:31:03.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.046 7f69d7903700 1 -- start start 2021-02-10T11:31:03.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.047 7f69d7903700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f69d009f140 0x7f69d00c1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:03.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.047 7f69d7903700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f69d00c1f60 0x7f69d009e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:03.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.047 7f69d7903700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f69d009e920 0x7f69d00c2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:03.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.047 7f69d7903700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d00c23e0 con 0x7f69d00c1f60 2021-02-10T11:31:03.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.047 7f69d7903700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d009ed20 con 0x7f69d009f140 2021-02-10T11:31:03.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.047 7f69d7903700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d0105d10 con 0x7f69d009e920 2021-02-10T11:31:03.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.047 7f69d4f1e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f69d00c1f60 0x7f69d009e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:03.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.047 7f69d571f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f69d009f140 0x7f69d00c1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:03.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.047 7f69d4f1e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f69d00c1f60 0x7f69d009e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49788/0 (socket says 172.21.2.101:49788) 2021-02-10T11:31:03.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.047 7f69d4f1e700 1 -- 172.21.2.101:0/4279691110 learned_addr learned my addr 172.21.2.101:0/4279691110 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:03.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.047 7f69d5f20700 1 --2- 172.21.2.101:0/4279691110 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f69d009e920 0x7f69d00c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:03.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.047 7f69d4f1e700 1 -- 172.21.2.101:0/4279691110 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f69d009e920 msgr2=0x7f69d00c2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:03.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.047 7f69d4f1e700 1 --2- 172.21.2.101:0/4279691110 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f69d009e920 0x7f69d00c2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:03.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.047 7f69d4f1e700 1 -- 172.21.2.101:0/4279691110 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f69d009f140 msgr2=0x7f69d00c1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:03.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.047 7f69d4f1e700 1 --2- 172.21.2.101:0/4279691110 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f69d009f140 0x7f69d00c1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:03.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.047 7f69d4f1e700 1 -- 172.21.2.101:0/4279691110 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d01a4910 con 0x7f69d00c1f60 2021-02-10T11:31:03.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.047 7f69d4f1e700 1 --2- 172.21.2.101:0/4279691110 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f69d00c1f60 0x7f69d009e300 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7f69cc00b650 tx=0x7f69cc00b9f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:03.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.047 7f69c1ffb700 1 -- 172.21.2.101:0/4279691110 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f69cc00bde0 con 0x7f69d00c1f60 2021-02-10T11:31:03.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.047 7f69c1ffb700 1 -- 172.21.2.101:0/4279691110 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69cc006390 con 0x7f69d00c1f60 2021-02-10T11:31:03.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.047 7f69c1ffb700 1 -- 172.21.2.101:0/4279691110 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f69cc002220 con 0x7f69d00c1f60 2021-02-10T11:31:03.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.047 7f69d7903700 1 -- 172.21.2.101:0/4279691110 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d01a4910 con 0x7f69d00c1f60 2021-02-10T11:31:03.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.048 7f69d7903700 1 -- 172.21.2.101:0/4279691110 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d00c3270 con 0x7f69d00c1f60 2021-02-10T11:31:03.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.048 7f69c1ffb700 1 -- 172.21.2.101:0/4279691110 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f69cc000c90 con 0x7f69d00c1f60 2021-02-10T11:31:03.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.048 7f69c1ffb700 1 --2- 172.21.2.101:0/4279691110 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f69b00186d0 0x7f69b001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:03.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.048 7f69c1ffb700 1 -- 172.21.2.101:0/4279691110 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f69cc0325f0 con 0x7f69d00c1f60 2021-02-10T11:31:03.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.048 7f69d571f700 1 --2- 172.21.2.101:0/4279691110 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f69b00186d0 0x7f69b001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:03.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.049 7f69d571f700 1 --2- 172.21.2.101:0/4279691110 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f69b00186d0 0x7f69b001abd0 crc :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:03.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.079 7f69d7903700 1 -- 172.21.2.101:0/4279691110 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d00f0b50 con 0x7f69d00c1f60 2021-02-10T11:31:03.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.082 7f69c1ffb700 1 -- 172.21.2.101:0/4279691110 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f69cc0090b0 con 0x7f69d00c1f60 2021-02-10T11:31:03.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.209 7f69d7903700 1 -- 172.21.2.101:0/4279691110 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f69d00f0b50 con 0x7f69d00c1f60 2021-02-10T11:31:03.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.209 7f69c1ffb700 1 -- 172.21.2.101:0/4279691110 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f69cc00bde0 con 0x7f69d00c1f60 2021-02-10T11:31:03.220 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:31:03.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.211 7f69d7903700 1 -- 172.21.2.101:0/4279691110 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f69b00186d0 msgr2=0x7f69b001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:03.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.211 7f69d7903700 1 --2- 172.21.2.101:0/4279691110 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f69b00186d0 0x7f69b001abd0 crc :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:03.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.211 7f69d7903700 1 -- 172.21.2.101:0/4279691110 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f69d00c1f60 msgr2=0x7f69d009e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:03.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.211 7f69d7903700 1 --2- 172.21.2.101:0/4279691110 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f69d00c1f60 0x7f69d009e300 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7f69cc00b650 tx=0x7f69cc00b9f0).stop 2021-02-10T11:31:03.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.211 7f69d7903700 1 -- 172.21.2.101:0/4279691110 shutdown_connections 2021-02-10T11:31:03.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.211 7f69d7903700 1 --2- 172.21.2.101:0/4279691110 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f69b00186d0 0x7f69b001abd0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:03.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.211 7f69d7903700 1 --2- 172.21.2.101:0/4279691110 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f69d009e920 0x7f69d00c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:03.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.211 7f69d7903700 1 --2- 172.21.2.101:0/4279691110 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f69d00c1f60 0x7f69d009e300 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:03.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.211 7f69d7903700 1 --2- 172.21.2.101:0/4279691110 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f69d009f140 0x7f69d00c1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:03.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.211 7f69d7903700 1 -- 172.21.2.101:0/4279691110 >> 172.21.2.101:0/4279691110 conn(0x7f69d009fc10 msgr2=0x7f69d00b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:03.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.212 7f69d7903700 1 -- 172.21.2.101:0/4279691110 shutdown_connections 2021-02-10T11:31:03.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.212 7f69d7903700 1 -- 172.21.2.101:0/4279691110 wait complete. 2021-02-10T11:31:03.231 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:31:03.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.632 7fb9eeb45700 1 Processor -- start 2021-02-10T11:31:03.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.632 7fb9eeb45700 1 -- start start 2021-02-10T11:31:03.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.633 7fb9eeb45700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb9e81b0470 0x7fb9e81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:03.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.633 7fb9eeb45700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9e80ef610 0x7fb9e80efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:03.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.633 7fb9eeb45700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb9e80f0030 0x7fb9e80f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:03.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.633 7fb9eeb45700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e80e5ea0 con 0x7fb9e80f0030 2021-02-10T11:31:03.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.633 7fb9eeb45700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e80f0b50 con 0x7fb9e81b0470 2021-02-10T11:31:03.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.633 7fb9eeb45700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e80e52e0 con 0x7fb9e80ef610 2021-02-10T11:31:03.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.633 7fb9ed162700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb9e80f0030 0x7fb9e80f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:03.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.633 7fb9ed162700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb9e80f0030 0x7fb9e80f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49796/0 (socket says 172.21.2.101:49796) 2021-02-10T11:31:03.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.633 7fb9ed162700 1 -- 172.21.2.101:0/1592387183 learned_addr learned my addr 172.21.2.101:0/1592387183 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:03.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.633 7fb9ed162700 1 -- 172.21.2.101:0/1592387183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9e80ef610 msgr2=0x7fb9e80efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:31:03.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.633 7fb9ed162700 1 --2- 172.21.2.101:0/1592387183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9e80ef610 0x7fb9e80efa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:03.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.633 7fb9ed162700 1 -- 172.21.2.101:0/1592387183 >> v2:172.21.2.108:3300/0 conn(0x7fb9e81b0470 msgr2=0x7fb9e81b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:31:03.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.633 7fb9ed162700 1 --2- 172.21.2.101:0/1592387183 >> v2:172.21.2.108:3300/0 conn(0x7fb9e81b0470 0x7fb9e81b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:03.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.633 7fb9ed162700 1 -- 172.21.2.101:0/1592387183 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e80f10d0 con 0x7fb9e80f0030 2021-02-10T11:31:03.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.633 7fb9ed162700 1 --2- 172.21.2.101:0/1592387183 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9e80f0030 0x7fb9e80f0430 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7fb9e0006f00 tx=0x7fb9e0009f10).ready entity=mon.0 client_cookie=368ff4546fa49931 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:03.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.633 7fb9e77fe700 1 -- 172.21.2.101:0/1592387183 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb9e000b3b0 con 0x7fb9e80f0030 2021-02-10T11:31:03.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.633 7fb9e77fe700 1 -- 172.21.2.101:0/1592387183 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e000b630 con 0x7fb9e80f0030 2021-02-10T11:31:03.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.633 7fb9e77fe700 1 -- 172.21.2.101:0/1592387183 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb9e000bb10 con 0x7fb9e80f0030 2021-02-10T11:31:03.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.633 7fb9eeb45700 1 -- 172.21.2.101:0/1592387183 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9e80f0030 msgr2=0x7fb9e80f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:03.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.633 7fb9eeb45700 1 --2- 172.21.2.101:0/1592387183 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9e80f0030 0x7fb9e80f0430 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7fb9e0006f00 tx=0x7fb9e0009f10).stop 2021-02-10T11:31:03.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.634 7fb9eeb45700 1 -- 172.21.2.101:0/1592387183 shutdown_connections 2021-02-10T11:31:03.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.634 7fb9eeb45700 1 --2- 172.21.2.101:0/1592387183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9e80ef610 0x7fb9e80efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:03.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.634 7fb9eeb45700 1 --2- 172.21.2.101:0/1592387183 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9e80f0030 0x7fb9e80f0430 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:03.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.634 7fb9eeb45700 1 --2- 172.21.2.101:0/1592387183 >> v2:172.21.2.108:3300/0 conn(0x7fb9e81b0470 0x7fb9e81b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:03.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.634 7fb9eeb45700 1 -- 172.21.2.101:0/1592387183 >> 172.21.2.101:0/1592387183 conn(0x7fb9e80ce3e0 msgr2=0x7fb9e80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:03.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.634 7fb9eeb45700 1 -- 172.21.2.101:0/1592387183 shutdown_connections 2021-02-10T11:31:03.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.634 7fb9eeb45700 1 -- 172.21.2.101:0/1592387183 wait complete. 2021-02-10T11:31:03.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.648 7fb9eeb45700 1 Processor -- start 2021-02-10T11:31:03.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.648 7fb9eeb45700 1 -- start start 2021-02-10T11:31:03.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.649 7fb9eeb45700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9e80bf910 0x7fb9e80bfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:03.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.649 7fb9eeb45700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9e80c0330 0x7fb9e80c0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:03.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.649 7fb9eeb45700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9e80c13d0 0x7fb9e80c17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:03.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.649 7fb9eeb45700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e8109930 con 0x7fb9e80c0330 2021-02-10T11:31:03.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.649 7fb9eeb45700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e8109b50 con 0x7fb9e80c13d0 2021-02-10T11:31:03.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.649 7fb9eeb45700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e80c1df0 con 0x7fb9e80bf910 2021-02-10T11:31:03.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.649 7fb9e7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9e80c0330 0x7fb9e80c0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:03.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.649 7fb9e7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9e80c0330 0x7fb9e80c0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49800/0 (socket says 172.21.2.101:49800) 2021-02-10T11:31:03.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.649 7fb9e7fff700 1 -- 172.21.2.101:0/2535574934 learned_addr learned my addr 172.21.2.101:0/2535574934 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:03.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.649 7fb9ec961700 1 --2- 172.21.2.101:0/2535574934 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9e80bf910 0x7fb9e80bfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:03.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.649 7fb9e7fff700 1 -- 172.21.2.101:0/2535574934 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9e80bf910 msgr2=0x7fb9e80bfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:03.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.649 7fb9e7fff700 1 --2- 172.21.2.101:0/2535574934 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9e80bf910 0x7fb9e80bfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:03.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.649 7fb9e7fff700 1 -- 172.21.2.101:0/2535574934 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9e80c13d0 msgr2=0x7fb9e80c17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:03.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.649 7fb9e7fff700 1 --2- 172.21.2.101:0/2535574934 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9e80c13d0 0x7fb9e80c17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:03.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.649 7fb9e7fff700 1 -- 172.21.2.101:0/2535574934 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e80c1fd0 con 0x7fb9e80c0330 2021-02-10T11:31:03.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.649 7fb9ec961700 1 --2- 172.21.2.101:0/2535574934 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9e80bf910 0x7fb9e80bfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:31:03.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.650 7fb9e7fff700 1 --2- 172.21.2.101:0/2535574934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9e80c0330 0x7fb9e80c0730 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7fb9d800b5c0 tx=0x7fb9d800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:03.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.650 7fb9e57fa700 1 -- 172.21.2.101:0/2535574934 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb9d800bd50 con 0x7fb9e80c0330 2021-02-10T11:31:03.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.650 7fb9e57fa700 1 -- 172.21.2.101:0/2535574934 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d8006030 con 0x7fb9e80c0330 2021-02-10T11:31:03.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.650 7fb9eeb45700 1 -- 172.21.2.101:0/2535574934 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e80c1fd0 con 0x7fb9e80c0330 2021-02-10T11:31:03.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.650 7fb9eeb45700 1 -- 172.21.2.101:0/2535574934 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e80c2560 con 0x7fb9e80c0330 2021-02-10T11:31:03.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.650 7fb9e57fa700 1 -- 172.21.2.101:0/2535574934 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb9d80054d0 con 0x7fb9e80c0330 2021-02-10T11:31:03.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.650 7fb9e57fa700 1 -- 172.21.2.101:0/2535574934 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb9d8006210 con 0x7fb9e80c0330 2021-02-10T11:31:03.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.650 7fb9e57fa700 1 --2- 172.21.2.101:0/2535574934 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9cc0186d0 0x7fb9cc01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:03.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.650 7fb9e57fa700 1 -- 172.21.2.101:0/2535574934 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb9d8032830 con 0x7fb9e80c0330 2021-02-10T11:31:03.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.651 7fb9ec961700 1 --2- 172.21.2.101:0/2535574934 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9cc0186d0 0x7fb9cc01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:03.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.651 7fb9ec961700 1 --2- 172.21.2.101:0/2535574934 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9cc0186d0 0x7fb9cc01abd0 crc :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:03.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.683 7fb9eeb45700 1 -- 172.21.2.101:0/2535574934 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e80f0b50 con 0x7fb9e80c0330 2021-02-10T11:31:03.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.685 7fb9e57fa700 1 -- 172.21.2.101:0/2535574934 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb9d8000940 con 0x7fb9e80c0330 2021-02-10T11:31:03.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.818 7fb9eeb45700 1 -- 172.21.2.101:0/2535574934 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7fb9e80f0b50 con 0x7fb9e80c0330 2021-02-10T11:31:03.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.818 7fb9e57fa700 1 -- 172.21.2.101:0/2535574934 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v29) v1 ==== 104+0+10 (secure 0 0 0) 0x7fb9d800bd50 con 0x7fb9e80c0330 2021-02-10T11:31:03.829 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:31:03.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.820 7fb9eeb45700 1 -- 172.21.2.101:0/2535574934 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9cc0186d0 msgr2=0x7fb9cc01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:03.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.820 7fb9eeb45700 1 --2- 172.21.2.101:0/2535574934 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9cc0186d0 0x7fb9cc01abd0 crc :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:03.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.820 7fb9eeb45700 1 -- 172.21.2.101:0/2535574934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9e80c0330 msgr2=0x7fb9e80c0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:03.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.820 7fb9eeb45700 1 --2- 172.21.2.101:0/2535574934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9e80c0330 0x7fb9e80c0730 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7fb9d800b5c0 tx=0x7fb9d800b960).stop 2021-02-10T11:31:03.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.821 7fb9eeb45700 1 -- 172.21.2.101:0/2535574934 shutdown_connections 2021-02-10T11:31:03.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.821 7fb9eeb45700 1 --2- 172.21.2.101:0/2535574934 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9e80c13d0 0x7fb9e80c17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:03.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.821 7fb9eeb45700 1 --2- 172.21.2.101:0/2535574934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9e80c0330 0x7fb9e80c0730 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:03.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.821 7fb9eeb45700 1 --2- 172.21.2.101:0/2535574934 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9cc0186d0 0x7fb9cc01abd0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:03.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.821 7fb9eeb45700 1 --2- 172.21.2.101:0/2535574934 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9e80bf910 0x7fb9e80bfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:03.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.821 7fb9eeb45700 1 -- 172.21.2.101:0/2535574934 >> 172.21.2.101:0/2535574934 conn(0x7fb9e80bf4e0 msgr2=0x7fb9e8191190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:03.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.821 7fb9eeb45700 1 -- 172.21.2.101:0/2535574934 shutdown_connections 2021-02-10T11:31:03.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:03.821 7fb9eeb45700 1 -- 172.21.2.101:0/2535574934 wait complete. 2021-02-10T11:31:03.840 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:31:04.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.241 7fc4a4794700 1 Processor -- start 2021-02-10T11:31:04.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.241 7fc4a4794700 1 -- start start 2021-02-10T11:31:04.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.242 7fc4a4794700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc49c1b0470 0x7fc49c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:04.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.242 7fc4a4794700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc49c0f20e0 0x7fc49c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:04.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.242 7fc4a4794700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc49c0f0b50 con 0x7fc49c0ef610 2021-02-10T11:31:04.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.242 7fc4a4794700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49c0f0820 con 0x7fc49c1b0470 2021-02-10T11:31:04.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.242 7fc4a4794700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49c0f24e0 con 0x7fc49c0f20e0 2021-02-10T11:31:04.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.242 7fc4a25b0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc49c1b0470 0x7fc49c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:04.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.242 7fc4a25b0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc49c1b0470 0x7fc49c1b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49806/0 (socket says 172.21.2.101:49806) 2021-02-10T11:31:04.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.242 7fc4a1daf700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc49c0ef610 0x7fc49c0efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50300/0 (socket says 172.21.2.101:50300) 2021-02-10T11:31:04.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.242 7fc4a25b0700 1 -- 172.21.2.101:0/4098998271 learned_addr learned my addr 172.21.2.101:0/4098998271 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:04.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.242 7fc4a25b0700 1 -- 172.21.2.101:0/4098998271 >> v2:172.21.2.108:3300/0 conn(0x7fc49c0f20e0 msgr2=0x7fc49c0f0200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:31:04.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.242 7fc4a25b0700 1 --2- 172.21.2.101:0/4098998271 >> v2:172.21.2.108:3300/0 conn(0x7fc49c0f20e0 0x7fc49c0f0200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:04.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.242 7fc4a25b0700 1 -- 172.21.2.101:0/4098998271 >> v1:172.21.2.101:6789/0 conn(0x7fc49c0ef610 legacy=0x7fc49c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:04.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.242 7fc4a25b0700 1 -- 172.21.2.101:0/4098998271 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c0e52e0 con 0x7fc49c1b0470 2021-02-10T11:31:04.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.242 7fc4a25b0700 1 --2- 172.21.2.101:0/4098998271 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc49c1b0470 0x7fc49c1b0870 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7fc48c006700 tx=0x7fc48c009b80).ready entity=mon.0 client_cookie=ecd50e7d7c042dd8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:04.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.242 7fc4a15ae700 1 -- 172.21.2.101:0/4098998271 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc48c00c0c0 con 0x7fc49c1b0470 2021-02-10T11:31:04.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.242 7fc4a15ae700 1 -- 172.21.2.101:0/4098998271 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48c00c340 con 0x7fc49c1b0470 2021-02-10T11:31:04.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.243 7fc4a15ae700 1 -- 172.21.2.101:0/4098998271 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc48c00c860 con 0x7fc49c1b0470 2021-02-10T11:31:04.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.243 7fc4a4794700 1 -- 172.21.2.101:0/4098998271 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc49c1b0470 msgr2=0x7fc49c1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:04.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.243 7fc4a4794700 1 --2- 172.21.2.101:0/4098998271 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc49c1b0470 0x7fc49c1b0870 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7fc48c006700 tx=0x7fc48c009b80).stop 2021-02-10T11:31:04.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.243 7fc4a4794700 1 -- 172.21.2.101:0/4098998271 shutdown_connections 2021-02-10T11:31:04.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.243 7fc4a4794700 1 --2- 172.21.2.101:0/4098998271 >> v2:172.21.2.108:3300/0 conn(0x7fc49c0f20e0 0x7fc49c0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:04.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.243 7fc4a4794700 1 --2- 172.21.2.101:0/4098998271 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc49c1b0470 0x7fc49c1b0870 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:04.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.243 7fc4a4794700 1 -- 172.21.2.101:0/4098998271 >> 172.21.2.101:0/4098998271 conn(0x7fc49c0ce3e0 msgr2=0x7fc49c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:04.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.243 7fc4a4794700 1 -- 172.21.2.101:0/4098998271 shutdown_connections 2021-02-10T11:31:04.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.243 7fc4a4794700 1 -- 172.21.2.101:0/4098998271 wait complete. 2021-02-10T11:31:04.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.258 7fc4a4794700 1 Processor -- start 2021-02-10T11:31:04.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.258 7fc4a4794700 1 -- start start 2021-02-10T11:31:04.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.258 7fc4a4794700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc49c18ca80 0x7fc49c18d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:04.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.258 7fc4a4794700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc49c18d6c0 0x7fc49c18dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:04.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.258 7fc4a4794700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc49c18e6b0 0x7fc49c18eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:04.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.258 7fc4a4794700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c19d290 con 0x7fc49c18d6c0 2021-02-10T11:31:04.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.258 7fc4a4794700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c18e130 con 0x7fc49c18e6b0 2021-02-10T11:31:04.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.258 7fc4a4794700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c18c5e0 con 0x7fc49c18ca80 2021-02-10T11:31:04.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.258 7fc4a1daf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc49c18d6c0 0x7fc49c18dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:04.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.258 7fc4a1daf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc49c18d6c0 0x7fc49c18dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49814/0 (socket says 172.21.2.101:49814) 2021-02-10T11:31:04.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.258 7fc4a1daf700 1 -- 172.21.2.101:0/2080961388 learned_addr learned my addr 172.21.2.101:0/2080961388 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:04.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.258 7fc4a25b0700 1 --2- 172.21.2.101:0/2080961388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc49c18ca80 0x7fc49c18d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:04.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.259 7fc4a2db1700 1 --2- 172.21.2.101:0/2080961388 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc49c18e6b0 0x7fc49c18eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:04.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.259 7fc4a1daf700 1 -- 172.21.2.101:0/2080961388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc49c18ca80 msgr2=0x7fc49c18d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:04.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.259 7fc4a1daf700 1 --2- 172.21.2.101:0/2080961388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc49c18ca80 0x7fc49c18d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:04.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.259 7fc4a1daf700 1 -- 172.21.2.101:0/2080961388 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc49c18e6b0 msgr2=0x7fc49c18eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:04.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.259 7fc4a1daf700 1 --2- 172.21.2.101:0/2080961388 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc49c18e6b0 0x7fc49c18eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:04.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.259 7fc4a1daf700 1 -- 172.21.2.101:0/2080961388 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c18f0d0 con 0x7fc49c18d6c0 2021-02-10T11:31:04.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.259 7fc4a2db1700 1 --2- 172.21.2.101:0/2080961388 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc49c18e6b0 0x7fc49c18eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:31:04.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.259 7fc4a1daf700 1 --2- 172.21.2.101:0/2080961388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc49c18d6c0 0x7fc49c18dac0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0x7fc494009fc0 tx=0x7fc49400bc40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:04.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.259 7fc492ffd700 1 -- 172.21.2.101:0/2080961388 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc4940062b0 con 0x7fc49c18d6c0 2021-02-10T11:31:04.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.259 7fc492ffd700 1 -- 172.21.2.101:0/2080961388 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc494004720 con 0x7fc49c18d6c0 2021-02-10T11:31:04.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.259 7fc492ffd700 1 -- 172.21.2.101:0/2080961388 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc494004b70 con 0x7fc49c18d6c0 2021-02-10T11:31:04.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.259 7fc4a4794700 1 -- 172.21.2.101:0/2080961388 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49c18f0d0 con 0x7fc49c18d6c0 2021-02-10T11:31:04.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.259 7fc4a4794700 1 -- 172.21.2.101:0/2080961388 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49c18f660 con 0x7fc49c18d6c0 2021-02-10T11:31:04.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.260 7fc492ffd700 1 -- 172.21.2.101:0/2080961388 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc494006530 con 0x7fc49c18d6c0 2021-02-10T11:31:04.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.260 7fc492ffd700 1 --2- 172.21.2.101:0/2080961388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4840186d0 0x7fc48401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:04.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.260 7fc492ffd700 1 -- 172.21.2.101:0/2080961388 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc494032a50 con 0x7fc49c18d6c0 2021-02-10T11:31:04.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.260 7fc4a25b0700 1 --2- 172.21.2.101:0/2080961388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4840186d0 0x7fc48401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:04.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.260 7fc4a25b0700 1 --2- 172.21.2.101:0/2080961388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4840186d0 0x7fc48401abd0 crc :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:04.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.291 7fc4a4794700 1 -- 172.21.2.101:0/2080961388 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc49c19d290 con 0x7fc49c18d6c0 2021-02-10T11:31:04.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.294 7fc492ffd700 1 -- 172.21.2.101:0/2080961388 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc494004980 con 0x7fc49c18d6c0 2021-02-10T11:31:04.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.421 7fc4a4794700 1 -- 172.21.2.101:0/2080961388 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7fc49c19d290 con 0x7fc49c18d6c0 2021-02-10T11:31:04.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.422 7fc492ffd700 1 -- 172.21.2.101:0/2080961388 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v29) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc494006490 con 0x7fc49c18d6c0 2021-02-10T11:31:04.433 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:31:04.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.424 7fc4a4794700 1 -- 172.21.2.101:0/2080961388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4840186d0 msgr2=0x7fc48401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:04.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.424 7fc4a4794700 1 --2- 172.21.2.101:0/2080961388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4840186d0 0x7fc48401abd0 crc :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:04.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.424 7fc4a4794700 1 -- 172.21.2.101:0/2080961388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc49c18d6c0 msgr2=0x7fc49c18dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:04.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.424 7fc4a4794700 1 --2- 172.21.2.101:0/2080961388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc49c18d6c0 0x7fc49c18dac0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0x7fc494009fc0 tx=0x7fc49400bc40).stop 2021-02-10T11:31:04.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.424 7fc4a4794700 1 -- 172.21.2.101:0/2080961388 shutdown_connections 2021-02-10T11:31:04.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.424 7fc4a4794700 1 --2- 172.21.2.101:0/2080961388 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc49c18e6b0 0x7fc49c18eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:04.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.424 7fc4a4794700 1 --2- 172.21.2.101:0/2080961388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc49c18d6c0 0x7fc49c18dac0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:04.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.424 7fc4a4794700 1 --2- 172.21.2.101:0/2080961388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4840186d0 0x7fc48401abd0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:04.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.424 7fc4a4794700 1 --2- 172.21.2.101:0/2080961388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc49c18ca80 0x7fc49c18d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:04.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.424 7fc4a4794700 1 -- 172.21.2.101:0/2080961388 >> 172.21.2.101:0/2080961388 conn(0x7fc49c0bf4f0 msgr2=0x7fc49c0be2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:04.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.424 7fc4a4794700 1 -- 172.21.2.101:0/2080961388 shutdown_connections 2021-02-10T11:31:04.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.424 7fc4a4794700 1 -- 172.21.2.101:0/2080961388 wait complete. 2021-02-10T11:31:04.443 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:31:04.444 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:31:04.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.841 7fa759590700 1 Processor -- start 2021-02-10T11:31:04.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.841 7fa759590700 1 -- start start 2021-02-10T11:31:04.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.841 7fa759590700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7540e5070 0x7fa7541b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:04.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.841 7fa759590700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa7540e5470 0x7fa75407d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:04.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.841 7fa759590700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa75407ed80 con 0x7fa7541b3e40 2021-02-10T11:31:04.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.841 7fa759590700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa754080310 con 0x7fa7540e5470 2021-02-10T11:31:04.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.841 7fa759590700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa754080500 con 0x7fa7540e5070 2021-02-10T11:31:04.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.842 7fa752abe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa7540e5470 0x7fa75407d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:04.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.842 7fa7532bf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7540e5070 0x7fa7541b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:04.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.842 7fa753ac0700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa7541b3e40 0x7fa75407dcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50316/0 (socket says 172.21.2.101:50316) 2021-02-10T11:31:04.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.842 7fa752abe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa7540e5470 0x7fa75407d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49820/0 (socket says 172.21.2.101:49820) 2021-02-10T11:31:04.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.842 7fa752abe700 1 -- 172.21.2.101:0/2966615603 learned_addr learned my addr 172.21.2.101:0/2966615603 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:04.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.842 7fa752abe700 1 -- 172.21.2.101:0/2966615603 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7540e5070 msgr2=0x7fa7541b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:04.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.842 7fa752abe700 1 --2- 172.21.2.101:0/2966615603 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7540e5070 0x7fa7541b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:04.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.842 7fa752abe700 1 -- 172.21.2.101:0/2966615603 >> v1:172.21.2.101:6789/0 conn(0x7fa7541b3e40 legacy=0x7fa75407dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:04.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.842 7fa752abe700 1 -- 172.21.2.101:0/2966615603 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7540806f0 con 0x7fa7540e5470 2021-02-10T11:31:04.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.842 7fa752abe700 1 --2- 172.21.2.101:0/2966615603 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7540e5470 0x7fa75407d690 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x7fa744006700 tx=0x7fa744009b80).ready entity=mon.0 client_cookie=276203b60a3be04e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:04.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.842 7fa7522bd700 1 -- 172.21.2.101:0/2966615603 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa74400c0c0 con 0x7fa7540e5470 2021-02-10T11:31:04.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.842 7fa7522bd700 1 -- 172.21.2.101:0/2966615603 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74400c340 con 0x7fa7540e5470 2021-02-10T11:31:04.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.842 7fa7522bd700 1 -- 172.21.2.101:0/2966615603 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa74400c860 con 0x7fa7540e5470 2021-02-10T11:31:04.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.842 7fa759590700 1 -- 172.21.2.101:0/2966615603 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7540e5470 msgr2=0x7fa75407d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:04.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.842 7fa759590700 1 --2- 172.21.2.101:0/2966615603 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7540e5470 0x7fa75407d690 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x7fa744006700 tx=0x7fa744009b80).stop 2021-02-10T11:31:04.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.843 7fa759590700 1 -- 172.21.2.101:0/2966615603 shutdown_connections 2021-02-10T11:31:04.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.843 7fa759590700 1 --2- 172.21.2.101:0/2966615603 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7540e5470 0x7fa75407d690 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:04.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.843 7fa759590700 1 --2- 172.21.2.101:0/2966615603 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7540e5070 0x7fa7541b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:04.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.843 7fa759590700 1 -- 172.21.2.101:0/2966615603 >> 172.21.2.101:0/2966615603 conn(0x7fa7540ce690 msgr2=0x7fa7540e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:04.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.843 7fa759590700 1 -- 172.21.2.101:0/2966615603 shutdown_connections 2021-02-10T11:31:04.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.843 7fa759590700 1 -- 172.21.2.101:0/2966615603 wait complete. 2021-02-10T11:31:04.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.858 7fa759590700 1 Processor -- start 2021-02-10T11:31:04.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.858 7fa759590700 1 -- start start 2021-02-10T11:31:04.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.858 7fa759590700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa754190ca0 0x7fa754088740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:04.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.858 7fa759590700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7541a5920 0x7fa7541a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:04.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.858 7fa759590700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa754080d40 0x7fa754081140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:04.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.858 7fa759590700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa754088d60 con 0x7fa754080d40 2021-02-10T11:31:04.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.858 7fa759590700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7541910f0 con 0x7fa754190ca0 2021-02-10T11:31:04.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.858 7fa759590700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7540b1400 con 0x7fa7541a5920 2021-02-10T11:31:04.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.859 7fa753ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa754080d40 0x7fa754081140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:04.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.859 7fa753ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa754080d40 0x7fa754081140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49828/0 (socket says 172.21.2.101:49828) 2021-02-10T11:31:04.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.859 7fa753ac0700 1 -- 172.21.2.101:0/2147474741 learned_addr learned my addr 172.21.2.101:0/2147474741 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:04.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.859 7fa753ac0700 1 -- 172.21.2.101:0/2147474741 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7541a5920 msgr2=0x7fa7541a5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:04.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.859 7fa753ac0700 1 --2- 172.21.2.101:0/2147474741 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7541a5920 0x7fa7541a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:04.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.859 7fa753ac0700 1 -- 172.21.2.101:0/2147474741 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa754190ca0 msgr2=0x7fa754088740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:04.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.859 7fa753ac0700 1 --2- 172.21.2.101:0/2147474741 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa754190ca0 0x7fa754088740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:04.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.859 7fa753ac0700 1 -- 172.21.2.101:0/2147474741 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7540b1620 con 0x7fa754080d40 2021-02-10T11:31:04.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.859 7fa753ac0700 1 --2- 172.21.2.101:0/2147474741 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa754080d40 0x7fa754081140 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0x7fa74c007f90 tx=0x7fa74c00abd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:04.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.859 7fa74bfff700 1 -- 172.21.2.101:0/2147474741 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa74c000ec0 con 0x7fa754080d40 2021-02-10T11:31:04.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.859 7fa74bfff700 1 -- 172.21.2.101:0/2147474741 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74c0070d0 con 0x7fa754080d40 2021-02-10T11:31:04.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.859 7fa759590700 1 -- 172.21.2.101:0/2147474741 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7540b1620 con 0x7fa754080d40 2021-02-10T11:31:04.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.859 7fa74bfff700 1 -- 172.21.2.101:0/2147474741 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa74c0074a0 con 0x7fa754080d40 2021-02-10T11:31:04.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.860 7fa759590700 1 -- 172.21.2.101:0/2147474741 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7540b1620 con 0x7fa754080d40 2021-02-10T11:31:04.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.860 7fa74bfff700 1 -- 172.21.2.101:0/2147474741 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa74c0072b0 con 0x7fa754080d40 2021-02-10T11:31:04.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.860 7fa74bfff700 1 --2- 172.21.2.101:0/2147474741 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa738018730 0x7fa73801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:04.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.860 7fa74bfff700 1 -- 172.21.2.101:0/2147474741 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa74c032960 con 0x7fa754080d40 2021-02-10T11:31:04.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.860 7fa7532bf700 1 --2- 172.21.2.101:0/2147474741 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa738018730 0x7fa73801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:04.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.860 7fa7532bf700 1 --2- 172.21.2.101:0/2147474741 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa738018730 0x7fa73801ac10 crc :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:04.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.891 7fa759590700 1 -- 172.21.2.101:0/2147474741 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7540817f0 con 0x7fa754080d40 2021-02-10T11:31:04.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:04.893 7fa74bfff700 1 -- 172.21.2.101:0/2147474741 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa74c007d00 con 0x7fa754080d40 2021-02-10T11:31:05.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:05.022 7fa759590700 1 -- 172.21.2.101:0/2147474741 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7540817f0 con 0x7fa754080d40 2021-02-10T11:31:05.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:05.023 7fa74bfff700 1 -- 172.21.2.101:0/2147474741 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+2098 (secure 0 0 0) 0x7fa74c006fd0 con 0x7fa754080d40 2021-02-10T11:31:05.034 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:31:05.034 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/3226463341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3226463341},{"type":"v1","addr":"172.21.2.101:6835","nonce":3226463341}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:14:57.130312","modified":"2021-02-10 11:14:59.506936","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4373":{"gid":4373,"name":"b","rank":0,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.2.108:6833/3230549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3230549207},{"type":"v1","addr":"172.21.2.108:6833","nonce":3230549207}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-02-10T11:31:05.034 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 14 2021-02-10T11:31:05.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:05.025 7fa759590700 1 -- 172.21.2.101:0/2147474741 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa738018730 msgr2=0x7fa73801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:05.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:05.025 7fa759590700 1 --2- 172.21.2.101:0/2147474741 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa738018730 0x7fa73801ac10 crc :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:05.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:05.025 7fa759590700 1 -- 172.21.2.101:0/2147474741 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa754080d40 msgr2=0x7fa754081140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:05.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:05.025 7fa759590700 1 --2- 172.21.2.101:0/2147474741 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa754080d40 0x7fa754081140 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0x7fa74c007f90 tx=0x7fa74c00abd0).stop 2021-02-10T11:31:05.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:05.025 7fa759590700 1 -- 172.21.2.101:0/2147474741 shutdown_connections 2021-02-10T11:31:05.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:05.025 7fa759590700 1 --2- 172.21.2.101:0/2147474741 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa738018730 0x7fa73801ac10 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:05.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:05.025 7fa759590700 1 --2- 172.21.2.101:0/2147474741 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7541a5920 0x7fa7541a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:05.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:05.025 7fa759590700 1 --2- 172.21.2.101:0/2147474741 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa754080d40 0x7fa754081140 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:05.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:05.025 7fa759590700 1 --2- 172.21.2.101:0/2147474741 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa754190ca0 0x7fa754088740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:05.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:05.025 7fa759590700 1 -- 172.21.2.101:0/2147474741 >> 172.21.2.101:0/2147474741 conn(0x7fa7541afc80 msgr2=0x7fa754089400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:05.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:05.025 7fa759590700 1 -- 172.21.2.101:0/2147474741 shutdown_connections 2021-02-10T11:31:05.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:05.025 7fa759590700 1 -- 172.21.2.101:0/2147474741 wait complete. 2021-02-10T11:31:05.246 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:14:57.130312', 'modified': '2021-02-10 11:14:59.506936', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4373}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4373': {'gid': 4373, 'name': 'b', 'rank': 0, 'incarnation': 10, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.2.108:6833/3230549207', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 3230549207}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 3230549207}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:31:05.246 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:31:05.247 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:31:05.643 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:31:05.643 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:31:05.643 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:31:05.644 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 4373, 2021-02-10T11:31:05.644 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:31:05.644 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:31:05.644 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 968.52303892299994, 2021-02-10T11:31:05.644 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 13, 2021-02-10T11:31:05.644 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 29, 2021-02-10T11:31:05.644 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 29, 2021-02-10T11:31:05.644 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 986.94812456900002 2021-02-10T11:31:05.644 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:31:05.647 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 4373, "want_state": "up:active", "state": "up:active", "rank_uptime": 968.52303892299994, "mdsmap_epoch": 13, "osdmap_epoch": 29, "osdmap_epoch_barrier": 29, "uptime": 986.94812456900002 } 2021-02-10T11:31:05.648 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:31:05.648 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-02-10T11:31:05.648 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-02-10T11:31:05.648 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:05.648 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:31:05.985 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:31:06.329 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:31:06.329 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:31:06.673 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:31:06.674 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:31:06.674 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:31:06.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:06.675 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:31:07.010 INFO:teuthology.orchestra.run.gibba001.stdout:44 2021-02-10T11:31:07.010 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [44] 2021-02-10T11:31:07.011 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:31:07.013 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:31:07.356 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:31:07.357 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:31:07.422 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[107694]: starting ceph client 2021-02-10T11:31:07.423 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:2021-02-10 11:31:07.411 7f4de01d4f80 -1 init, newargv = 0x55a0bd2d1110 newargc=9 2021-02-10T11:31:07.428 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[107694]: starting fuse 2021-02-10T11:31:07.698 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:31:07.699 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:31:07.699 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:31:07.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:07.700 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:31:08.035 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:31:08.035 INFO:teuthology.orchestra.run.gibba001.stdout:44 2021-02-10T11:31:08.035 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2021-02-10T11:31:08.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:08.036 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:31:08.036 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:31:08.036 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:31:08.036 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:31:08.036 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:31:08.036 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:31:08.036 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:31:08.036 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-02-10T11:31:08.036 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:31:08.036 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:31:08.036 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:31:08.037 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:31:08.037 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:31:08.037 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:31:08.037 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:31:08.037 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:31:08.037 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:31:08.037 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:31:08.037 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:31:08.037 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:31:08.037 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:31:08.037 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.0")) 2021-02-10T11:31:08.037 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:31:08.400 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.0.107694.asok 2021-02-10T11:31:08.403 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.107694.asok 2021-02-10T11:31:08.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:08.404 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.107694.asok status 2021-02-10T11:31:08.795 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:31:08.795 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:31:08.795 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:31:08.795 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:31:08.795 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "0", 2021-02-10T11:31:08.795 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:31:08.795 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-02-10T11:31:08.796 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "107694", 2021-02-10T11:31:08.796 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:31:08.796 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:08.796 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:31:08.796 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:31:08.796 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 7209, 2021-02-10T11:31:08.797 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:31:08.797 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:31:08.797 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:31:08.797 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 7209 2021-02-10T11:31:08.797 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:08.797 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:31:08.797 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:08.797 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:31:08.797 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 3492299951 2021-02-10T11:31:08.797 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:08.798 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:08.798 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:31:08.798 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:08.798 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:31:08.798 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 3492299951 2021-02-10T11:31:08.798 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:08.798 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.7209 172.21.2.101:0/3492299951", 2021-02-10T11:31:08.798 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/3492299951", 2021-02-10T11:31:08.798 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:31:08.798 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 13, 2021-02-10T11:31:08.798 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 29, 2021-02-10T11:31:08.798 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:31:08.798 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:31:08.799 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:31:08.800 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo getfattr --only-values -n ceph.dir.entries /home/ubuntu/cephtest/mnt.0/./volumes/_deleting 2021-02-10T11:31:09.147 INFO:teuthology.orchestra.run.gibba001.stderr:getfattr: Removing leading '/' from absolute path names 2021-02-10T11:31:09.148 INFO:teuthology.orchestra.run.gibba001.stdout:0 2021-02-10T11:31:09.149 DEBUG:tasks.cephfs.mount:Directory ./volumes/_deleting seen empty from 0 after 0s 2021-02-10T11:31:09.149 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:31:09.149 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:31:09.495 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:31:09.841 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:31:09.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:09.841 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:31:09.841 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:31:10.176 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:31:10.177 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:31:10.177 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:31:10.177 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:31:10.177 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:31:10.528 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[107694]: fuse finished with error 0 and tester_r 0 2021-02-10T11:31:10.529 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:10.529 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:31:10.529 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:31:10.864 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:31:10.864 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:31:10.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:31:10.865 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:31:11.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:11.199 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:31:11.200 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:31:11.533 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:31:11.534 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:31:11.534 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:31:11.534 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:31:11.534 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:31:11.884 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[34770]: fuse finished with error 0 and tester_r 0 2021-02-10T11:31:11.885 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:11.885 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:31:11.885 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:31:12.219 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:31:12.219 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:31:12.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:31:12.220 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:31:12.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:12.554 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:31:12.554 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:31:12.887 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:31:12.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:31:12.888 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:31:12.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:31:12.888 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:31:13.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:13.223 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:31:13.223 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:31:13.557 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:31:13.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:31:13.558 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:31:13.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:31:13.558 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:31:13.893 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-02-10T11:31:14.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.296 7fe1a01d6700 1 Processor -- start 2021-02-10T11:31:14.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.296 7fe1a01d6700 1 -- start start 2021-02-10T11:31:14.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.296 7fe1a01d6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe1980cff00 0x7fe1981b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:14.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.296 7fe1a01d6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1980d1bf0 0x7fe1980ef620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:14.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.296 7fe1a01d6700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1980f0b20 con 0x7fe1980d17f0 2021-02-10T11:31:14.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.296 7fe1a01d6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1981b3ff0 con 0x7fe1980cff00 2021-02-10T11:31:14.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.296 7fe1a01d6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1980f20b0 con 0x7fe1980d1bf0 2021-02-10T11:31:14.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.296 7fe19d7f1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe1980cff00 0x7fe1981b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:14.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.296 7fe19dff2700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe1980d17f0 0x7fe1981b31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50340/0 (socket says 172.21.2.101:50340) 2021-02-10T11:31:14.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.296 7fe19d7f1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe1980cff00 0x7fe1981b39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49846/0 (socket says 172.21.2.101:49846) 2021-02-10T11:31:14.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.296 7fe19dff2700 1 -- 172.21.2.101:0/1268773653 learned_addr learned my addr 172.21.2.101:0/1268773653 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:14.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.297 7fe19e7f3700 1 --2- 172.21.2.101:0/1268773653 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1980d1bf0 0x7fe1980ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:14.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.297 7fe19d7f1700 1 -- 172.21.2.101:0/1268773653 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1980d1bf0 msgr2=0x7fe1980ef620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:14.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.297 7fe19d7f1700 1 --2- 172.21.2.101:0/1268773653 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1980d1bf0 0x7fe1980ef620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:14.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.297 7fe19d7f1700 1 -- 172.21.2.101:0/1268773653 >> v1:172.21.2.101:6789/0 conn(0x7fe1980d17f0 legacy=0x7fe1981b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:14.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.297 7fe19d7f1700 1 -- 172.21.2.101:0/1268773653 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1980f22d0 con 0x7fe1980cff00 2021-02-10T11:31:14.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.297 7fe19e7f3700 1 --2- 172.21.2.101:0/1268773653 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1980d1bf0 0x7fe1980ef620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:31:14.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.297 7fe19d7f1700 1 --2- 172.21.2.101:0/1268773653 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1980cff00 0x7fe1981b39d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7fe18c006650 tx=0x7fe18c009a80).ready entity=mon.0 client_cookie=3611778d113a7084 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:14.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.297 7fe19cff0700 1 -- 172.21.2.101:0/1268773653 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe18c00c0c0 con 0x7fe1980cff00 2021-02-10T11:31:14.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.297 7fe19cff0700 1 -- 172.21.2.101:0/1268773653 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18c00c340 con 0x7fe1980cff00 2021-02-10T11:31:14.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.297 7fe19cff0700 1 -- 172.21.2.101:0/1268773653 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe18c00c860 con 0x7fe1980cff00 2021-02-10T11:31:14.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.297 7fe1a01d6700 1 -- 172.21.2.101:0/1268773653 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1980cff00 msgr2=0x7fe1981b39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:14.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.297 7fe1a01d6700 1 --2- 172.21.2.101:0/1268773653 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1980cff00 0x7fe1981b39d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7fe18c006650 tx=0x7fe18c009a80).stop 2021-02-10T11:31:14.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.298 7fe1a01d6700 1 -- 172.21.2.101:0/1268773653 shutdown_connections 2021-02-10T11:31:14.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.298 7fe1a01d6700 1 --2- 172.21.2.101:0/1268773653 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1980d1bf0 0x7fe1980ef620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:14.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.298 7fe1a01d6700 1 --2- 172.21.2.101:0/1268773653 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1980cff00 0x7fe1981b39d0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:14.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.298 7fe1a01d6700 1 -- 172.21.2.101:0/1268773653 >> 172.21.2.101:0/1268773653 conn(0x7fe1980fde40 msgr2=0x7fe1980ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:14.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.298 7fe1a01d6700 1 -- 172.21.2.101:0/1268773653 shutdown_connections 2021-02-10T11:31:14.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.299 7fe1a01d6700 1 -- 172.21.2.101:0/1268773653 wait complete. 2021-02-10T11:31:14.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.313 7fe1a01d6700 1 Processor -- start 2021-02-10T11:31:14.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.313 7fe1a01d6700 1 -- start start 2021-02-10T11:31:14.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.314 7fe1a01d6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe1981a7a20 0x7fe1981a7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:14.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.314 7fe1a01d6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1980aeb20 0x7fe1980aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:14.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.314 7fe1a01d6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1980af540 0x7fe1980af940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:14.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.314 7fe1a01d6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1980ae5a0 con 0x7fe1980aeb20 2021-02-10T11:31:14.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.314 7fe1a01d6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1980a43d0 con 0x7fe1981a7a20 2021-02-10T11:31:14.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.314 7fe1a01d6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1980a45f0 con 0x7fe1980af540 2021-02-10T11:31:14.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.314 7fe19d7f1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1980aeb20 0x7fe1980aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:14.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.314 7fe19d7f1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1980aeb20 0x7fe1980aef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49852/0 (socket says 172.21.2.101:49852) 2021-02-10T11:31:14.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.314 7fe19d7f1700 1 -- 172.21.2.101:0/192431031 learned_addr learned my addr 172.21.2.101:0/192431031 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:14.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.314 7fe19d7f1700 1 -- 172.21.2.101:0/192431031 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1980af540 msgr2=0x7fe1980af940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:31:14.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.314 7fe19d7f1700 1 --2- 172.21.2.101:0/192431031 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1980af540 0x7fe1980af940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:14.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.314 7fe19d7f1700 1 -- 172.21.2.101:0/192431031 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe1981a7a20 msgr2=0x7fe1981a7e20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:31:14.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.314 7fe19d7f1700 1 --2- 172.21.2.101:0/192431031 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe1981a7a20 0x7fe1981a7e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:14.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.314 7fe19d7f1700 1 -- 172.21.2.101:0/192431031 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1980a4810 con 0x7fe1980aeb20 2021-02-10T11:31:14.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.314 7fe19d7f1700 1 --2- 172.21.2.101:0/192431031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1980aeb20 0x7fe1980aef20 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7fe18c009630 tx=0x7fe18c00b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:14.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.314 7fe18a7fc700 1 -- 172.21.2.101:0/192431031 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe18c0093a0 con 0x7fe1980aeb20 2021-02-10T11:31:14.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.315 7fe18a7fc700 1 -- 172.21.2.101:0/192431031 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18c00bd70 con 0x7fe1980aeb20 2021-02-10T11:31:14.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.315 7fe18a7fc700 1 -- 172.21.2.101:0/192431031 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe18c005560 con 0x7fe1980aeb20 2021-02-10T11:31:14.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.315 7fe1a01d6700 1 -- 172.21.2.101:0/192431031 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1980a4810 con 0x7fe1980aeb20 2021-02-10T11:31:14.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.315 7fe1a01d6700 1 -- 172.21.2.101:0/192431031 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1980a4810 con 0x7fe1980aeb20 2021-02-10T11:31:14.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.315 7fe18a7fc700 1 -- 172.21.2.101:0/192431031 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe18c000c90 con 0x7fe1980aeb20 2021-02-10T11:31:14.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.315 7fe18a7fc700 1 --2- 172.21.2.101:0/192431031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe1800186c0 0x7fe18001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:14.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.315 7fe18a7fc700 1 -- 172.21.2.101:0/192431031 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe18c032560 con 0x7fe1980aeb20 2021-02-10T11:31:14.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.316 7fe19dff2700 1 --2- 172.21.2.101:0/192431031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe1800186c0 0x7fe18001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:14.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.316 7fe19dff2700 1 --2- 172.21.2.101:0/192431031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe1800186c0 0x7fe18001abc0 crc :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:14.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.346 7fe1a01d6700 1 -- 172.21.2.101:0/192431031 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe198103690 con 0x7fe1980aeb20 2021-02-10T11:31:14.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.349 7fe18a7fc700 1 -- 172.21.2.101:0/192431031 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe18c025550 con 0x7fe1980aeb20 2021-02-10T11:31:14.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.501 7fe1a01d6700 1 -- 172.21.2.101:0/192431031 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe1980f0b20 con 0x7fe1980aeb20 2021-02-10T11:31:14.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.505 7fe18a7fc700 1 -- 172.21.2.101:0/192431031 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe18c025550 con 0x7fe1980aeb20 2021-02-10T11:31:14.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.510 7fe1a01d6700 1 -- 172.21.2.101:0/192431031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe1800186c0 msgr2=0x7fe18001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:14.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.510 7fe1a01d6700 1 --2- 172.21.2.101:0/192431031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe1800186c0 0x7fe18001abc0 crc :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:14.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.510 7fe1a01d6700 1 -- 172.21.2.101:0/192431031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1980aeb20 msgr2=0x7fe1980aef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:14.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.510 7fe1a01d6700 1 --2- 172.21.2.101:0/192431031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1980aeb20 0x7fe1980aef20 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7fe18c009630 tx=0x7fe18c00b980).stop 2021-02-10T11:31:14.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.510 7fe1a01d6700 1 -- 172.21.2.101:0/192431031 shutdown_connections 2021-02-10T11:31:14.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.510 7fe1a01d6700 1 --2- 172.21.2.101:0/192431031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe1800186c0 0x7fe18001abc0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:14.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.510 7fe1a01d6700 1 --2- 172.21.2.101:0/192431031 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1980af540 0x7fe1980af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:14.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.510 7fe1a01d6700 1 --2- 172.21.2.101:0/192431031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1980aeb20 0x7fe1980aef20 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:14.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.510 7fe1a01d6700 1 --2- 172.21.2.101:0/192431031 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe1981a7a20 0x7fe1981a7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:14.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.510 7fe1a01d6700 1 -- 172.21.2.101:0/192431031 >> 172.21.2.101:0/192431031 conn(0x7fe1980b5de0 msgr2=0x7fe1980a6510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:14.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.510 7fe1a01d6700 1 -- 172.21.2.101:0/192431031 shutdown_connections 2021-02-10T11:31:14.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.510 7fe1a01d6700 1 -- 172.21.2.101:0/192431031 wait complete. 2021-02-10T11:31:14.529 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_async_subvolume_rm' 2021-02-10T11:31:14.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.928 7f82ebc6b700 1 Processor -- start 2021-02-10T11:31:14.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.929 7f82ebc6b700 1 -- start start 2021-02-10T11:31:14.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.929 7f82ebc6b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f82e40e5470 0x7f82e407e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:14.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.929 7f82ebc6b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e407ed80 con 0x7f82e40e5070 2021-02-10T11:31:14.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.929 7f82ebc6b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e41b4010 con 0x7f82e407f300 2021-02-10T11:31:14.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.929 7f82ebc6b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e40d1790 con 0x7f82e40e5470 2021-02-10T11:31:14.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.929 7f82e9a87700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f82e40e5070 0x7f82e41b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50354/0 (socket says 172.21.2.101:50354) 2021-02-10T11:31:14.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.929 7f82e9a87700 1 -- 172.21.2.101:0/2550255333 learned_addr learned my addr 172.21.2.101:0/2550255333 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:14.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.930 7f82e8a85700 1 -- 172.21.2.101:0/2550255333 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859624727 0 0) 0x7f82d80010b0 con 0x7f82e40e5070 2021-02-10T11:31:14.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.930 7f82e8a85700 1 -- 172.21.2.101:0/2550255333 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82d0002fe0 con 0x7f82e40e5070 2021-02-10T11:31:14.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.930 7f82e9286700 1 --2- 172.21.2.101:0/2550255333 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f82e40e5470 0x7f82e407e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:14.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.930 7f82e8a85700 1 -- 172.21.2.101:0/2550255333 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3904981435 0 0) 0x7f82d80010b0 con 0x7f82e40e5070 2021-02-10T11:31:14.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.930 7f82e8a85700 1 -- 172.21.2.101:0/2550255333 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f82e40e5470 msgr2=0x7f82e407e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:14.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.930 7f82e8a85700 1 --2- 172.21.2.101:0/2550255333 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f82e40e5470 0x7f82e407e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:14.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.930 7f82e8a85700 1 -- 172.21.2.101:0/2550255333 >> v1:172.21.2.108:6789/0 conn(0x7f82e407f300 legacy=0x7f82e407f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:14.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.930 7f82e8a85700 1 -- 172.21.2.101:0/2550255333 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e40d1980 con 0x7f82e40e5070 2021-02-10T11:31:14.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.930 7f82e8a85700 1 -- 172.21.2.101:0/2550255333 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f82d80015b0 con 0x7f82e40e5070 2021-02-10T11:31:14.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.930 7f82e8a85700 1 -- 172.21.2.101:0/2550255333 >> v1:172.21.2.101:6789/0 conn(0x7f82e40e5070 legacy=0x7f82e41b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:14.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.930 7f82e8a85700 1 --2- 172.21.2.101:0/2550255333 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82d0003a60 0x7f82d0005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:14.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.930 7f82e8a85700 1 --2- 172.21.2.101:0/2550255333 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f82d0006780 0x7f82d0008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:14.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.930 7f82e8a85700 1 --2- 172.21.2.101:0/2550255333 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82d0009510 0x7f82d000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:14.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.930 7f82e8a85700 1 -- 172.21.2.101:0/2550255333 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d000c2c0 con 0x7f82d0009510 2021-02-10T11:31:14.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.930 7f82e8a85700 1 -- 172.21.2.101:0/2550255333 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d000c4e0 con 0x7f82d0003a60 2021-02-10T11:31:14.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.930 7f82e8a85700 1 -- 172.21.2.101:0/2550255333 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d000c700 con 0x7f82d0006780 2021-02-10T11:31:14.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.930 7f82ea288700 1 --2- 172.21.2.101:0/2550255333 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82d0009510 0x7f82d000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:14.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.930 7f82e9a87700 1 --2- 172.21.2.101:0/2550255333 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82d0003a60 0x7f82d0005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:14.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.930 7f82e9286700 1 --2- 172.21.2.101:0/2550255333 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f82d0006780 0x7f82d0008c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:14.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.931 7f82ea288700 1 -- 172.21.2.101:0/2550255333 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f82d0006780 msgr2=0x7f82d0008c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:14.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.931 7f82ea288700 1 --2- 172.21.2.101:0/2550255333 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f82d0006780 0x7f82d0008c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:14.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.931 7f82ea288700 1 -- 172.21.2.101:0/2550255333 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82d0003a60 msgr2=0x7f82d0005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:14.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.931 7f82ea288700 1 --2- 172.21.2.101:0/2550255333 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82d0003a60 0x7f82d0005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:14.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.931 7f82ea288700 1 -- 172.21.2.101:0/2550255333 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d000c9a0 con 0x7f82d0009510 2021-02-10T11:31:14.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.931 7f82ea288700 1 --2- 172.21.2.101:0/2550255333 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82d0009510 0x7f82d000ba10 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7f82dc006e30 tx=0x7f82dc04de60).ready entity=mon.0 client_cookie=ef0e74917fd8714 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:14.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.931 7f82e8a85700 1 -- 172.21.2.101:0/2550255333 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f82dc006a50 con 0x7f82d0009510 2021-02-10T11:31:14.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.931 7f82e8a85700 1 -- 172.21.2.101:0/2550255333 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82dc001400 con 0x7f82d0009510 2021-02-10T11:31:14.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.931 7f82e8a85700 1 -- 172.21.2.101:0/2550255333 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f82dc001940 con 0x7f82d0009510 2021-02-10T11:31:14.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.931 7f82ebc6b700 1 -- 172.21.2.101:0/2550255333 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82d0009510 msgr2=0x7f82d000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:14.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.931 7f82ebc6b700 1 --2- 172.21.2.101:0/2550255333 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82d0009510 0x7f82d000ba10 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7f82dc006e30 tx=0x7f82dc04de60).stop 2021-02-10T11:31:14.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.931 7f82ea288700 1 -- 172.21.2.101:0/2550255333 reap_dead start 2021-02-10T11:31:14.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.931 7f82ebc6b700 1 -- 172.21.2.101:0/2550255333 shutdown_connections 2021-02-10T11:31:14.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.931 7f82ebc6b700 1 -- 172.21.2.101:0/2550255333 >> 172.21.2.101:0/2550255333 conn(0x7f82e40ce690 msgr2=0x7f82e40e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:14.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.932 7f82ebc6b700 1 -- 172.21.2.101:0/2550255333 shutdown_connections 2021-02-10T11:31:14.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.932 7f82ebc6b700 1 -- 172.21.2.101:0/2550255333 wait complete. 2021-02-10T11:31:14.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.946 7f82ebc6b700 1 Processor -- start 2021-02-10T11:31:14.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.946 7f82ebc6b700 1 -- start start 2021-02-10T11:31:14.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.947 7f82ebc6b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82e41ad6e0 0x7f82e40a3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:14.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.947 7f82ebc6b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82e408da10 0x7f82e41ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:14.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.947 7f82ebc6b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f82e408de10 0x7f82e41ae920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:14.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.947 7f82ebc6b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e41add80 con 0x7f82e41ad6e0 2021-02-10T11:31:14.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.947 7f82ebc6b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e41adae0 con 0x7f82e408da10 2021-02-10T11:31:14.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.947 7f82ebc6b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e40a3830 con 0x7f82e408de10 2021-02-10T11:31:14.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.947 7f82e9a87700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82e41ad6e0 0x7f82e40a3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:14.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.947 7f82e9a87700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82e41ad6e0 0x7f82e40a3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49872/0 (socket says 172.21.2.101:49872) 2021-02-10T11:31:14.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.947 7f82e9a87700 1 -- 172.21.2.101:0/2800857244 learned_addr learned my addr 172.21.2.101:0/2800857244 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:14.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.947 7f82e9a87700 1 -- 172.21.2.101:0/2800857244 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f82e408de10 msgr2=0x7f82e41ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:14.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.947 7f82e9a87700 1 --2- 172.21.2.101:0/2800857244 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f82e408de10 0x7f82e41ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:14.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.947 7f82e9a87700 1 -- 172.21.2.101:0/2800857244 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82e408da10 msgr2=0x7f82e41ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:14.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.947 7f82e9a87700 1 --2- 172.21.2.101:0/2800857244 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82e408da10 0x7f82e41ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:14.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.947 7f82e9a87700 1 -- 172.21.2.101:0/2800857244 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e41aef40 con 0x7f82e41ad6e0 2021-02-10T11:31:14.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.947 7f82e9a87700 1 --2- 172.21.2.101:0/2800857244 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82e41ad6e0 0x7f82e40a3210 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 rx=0x7f82d8001660 tx=0x7f82d800a930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:14.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.948 7f82d67fc700 1 -- 172.21.2.101:0/2800857244 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f82d800ad20 con 0x7f82e41ad6e0 2021-02-10T11:31:14.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.948 7f82d67fc700 1 -- 172.21.2.101:0/2800857244 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d80016a0 con 0x7f82e41ad6e0 2021-02-10T11:31:14.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.948 7f82d67fc700 1 -- 172.21.2.101:0/2800857244 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f82d80043c0 con 0x7f82e41ad6e0 2021-02-10T11:31:14.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.948 7f82ebc6b700 1 -- 172.21.2.101:0/2800857244 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e41aef40 con 0x7f82e41ad6e0 2021-02-10T11:31:14.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.948 7f82ebc6b700 1 -- 172.21.2.101:0/2800857244 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e41aef40 con 0x7f82e41ad6e0 2021-02-10T11:31:14.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.948 7f82d67fc700 1 -- 172.21.2.101:0/2800857244 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f82d8004140 con 0x7f82e41ad6e0 2021-02-10T11:31:14.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.948 7f82d67fc700 1 --2- 172.21.2.101:0/2800857244 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82cc0186c0 0x7f82cc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:14.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.948 7f82d67fc700 1 -- 172.21.2.101:0/2800857244 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f82d80327a0 con 0x7f82e41ad6e0 2021-02-10T11:31:14.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.948 7f82e9286700 1 --2- 172.21.2.101:0/2800857244 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82cc0186c0 0x7f82cc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:14.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.949 7f82e9286700 1 --2- 172.21.2.101:0/2800857244 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82cc0186c0 0x7f82cc01abc0 crc :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:14.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.979 7f82ebc6b700 1 -- 172.21.2.101:0/2800857244 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e407ed80 con 0x7f82e41ad6e0 2021-02-10T11:31:14.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:14.981 7f82d67fc700 1 -- 172.21.2.101:0/2800857244 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f82d8001180 con 0x7f82e41ad6e0 2021-02-10T11:31:15.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.114 7f82ebc6b700 1 -- 172.21.2.101:0/2800857244 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_async_subvolume_rm"]} v 0) v1 -- 0x7f82e407ed80 con 0x7f82e41ad6e0 2021-02-10T11:31:15.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.427 7f82d67fc700 1 -- 172.21.2.101:0/2800857244 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_async_subvolume_rm"]}]=0 v209) v1 ==== 140+0+0 (secure 0 0 0) 0x7f82d8004140 con 0x7f82e41ad6e0 2021-02-10T11:31:15.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.432 7f82ebc6b700 1 -- 172.21.2.101:0/2800857244 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82cc0186c0 msgr2=0x7f82cc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:15.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.432 7f82ebc6b700 1 --2- 172.21.2.101:0/2800857244 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82cc0186c0 0x7f82cc01abc0 crc :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:15.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.432 7f82ebc6b700 1 -- 172.21.2.101:0/2800857244 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82e41ad6e0 msgr2=0x7f82e40a3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:15.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.432 7f82ebc6b700 1 --2- 172.21.2.101:0/2800857244 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82e41ad6e0 0x7f82e40a3210 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 rx=0x7f82d8001660 tx=0x7f82d800a930).stop 2021-02-10T11:31:15.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.432 7f82ebc6b700 1 -- 172.21.2.101:0/2800857244 shutdown_connections 2021-02-10T11:31:15.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.432 7f82ebc6b700 1 --2- 172.21.2.101:0/2800857244 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82cc0186c0 0x7f82cc01abc0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:15.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.432 7f82ebc6b700 1 --2- 172.21.2.101:0/2800857244 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f82e408de10 0x7f82e41ae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:15.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.432 7f82ebc6b700 1 --2- 172.21.2.101:0/2800857244 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82e408da10 0x7f82e41ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:15.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.432 7f82ebc6b700 1 --2- 172.21.2.101:0/2800857244 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82e41ad6e0 0x7f82e40a3210 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:15.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.432 7f82ebc6b700 1 -- 172.21.2.101:0/2800857244 >> 172.21.2.101:0/2800857244 conn(0x7f82e41a31b0 msgr2=0x7f82e418c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:15.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.433 7f82ebc6b700 1 -- 172.21.2.101:0/2800857244 shutdown_connections 2021-02-10T11:31:15.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.433 7f82ebc6b700 1 -- 172.21.2.101:0/2800857244 wait complete. 2021-02-10T11:31:15.452 INFO:tasks.cephfs_test_runner:test_async_subvolume_rm (tasks.cephfs.test_volumes.TestVolumes) ... ok 2021-02-10T11:31:15.452 INFO:tasks.cephfs_test_runner:Starting test: test_authorize_allow_existing_id_option (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:31:15.452 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_authorize_allow_existing_id_option' 2021-02-10T11:31:15.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.850 7fa7f2498700 1 Processor -- start 2021-02-10T11:31:15.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.850 7fa7f2498700 1 -- start start 2021-02-10T11:31:15.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.851 7fa7f2498700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec0e5070 0x7fa7ec1b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:15.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.851 7fa7f2498700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa7ec0e5470 0x7fa7ec07d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:15.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.851 7fa7f2498700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa7ec1b3e40 0x7fa7ec07dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:15.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.851 7fa7f2498700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7ec07ed80 con 0x7fa7ec0e5470 2021-02-10T11:31:15.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.851 7fa7f2498700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7ec07e2d0 con 0x7fa7ec1b3e40 2021-02-10T11:31:15.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.851 7fa7f2498700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec07e4c0 con 0x7fa7ec0e5070 2021-02-10T11:31:15.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.851 7fa7eb7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa7ec0e5470 0x7fa7ec07d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:15.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.851 7fa7eb7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa7ec0e5470 0x7fa7ec07d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49878/0 (socket says 172.21.2.101:49878) 2021-02-10T11:31:15.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.851 7fa7eb7fe700 1 -- 172.21.2.101:0/1091726979 learned_addr learned my addr 172.21.2.101:0/1091726979 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:15.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.851 7fa7ebfff700 1 --2- 172.21.2.101:0/1091726979 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec0e5070 0x7fa7ec1b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:15.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.851 7fa7f0ab5700 1 --2- 172.21.2.101:0/1091726979 >> v2:172.21.2.108:3300/0 conn(0x7fa7ec1b3e40 0x7fa7ec07dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:15.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.851 7fa7eb7fe700 1 -- 172.21.2.101:0/1091726979 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec0e5070 msgr2=0x7fa7ec1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:15.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.851 7fa7eb7fe700 1 --2- 172.21.2.101:0/1091726979 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec0e5070 0x7fa7ec1b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:15.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.851 7fa7eb7fe700 1 -- 172.21.2.101:0/1091726979 >> v2:172.21.2.108:3300/0 conn(0x7fa7ec1b3e40 msgr2=0x7fa7ec07dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:15.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.851 7fa7eb7fe700 1 --2- 172.21.2.101:0/1091726979 >> v2:172.21.2.108:3300/0 conn(0x7fa7ec1b3e40 0x7fa7ec07dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:15.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.851 7fa7eb7fe700 1 -- 172.21.2.101:0/1091726979 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec07e6e0 con 0x7fa7ec0e5470 2021-02-10T11:31:15.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.851 7fa7f0ab5700 1 --2- 172.21.2.101:0/1091726979 >> v2:172.21.2.108:3300/0 conn(0x7fa7ec1b3e40 0x7fa7ec07dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:31:15.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.851 7fa7eb7fe700 1 --2- 172.21.2.101:0/1091726979 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7ec0e5470 0x7fa7ec07d690 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7fa7d4006700 tx=0x7fa7d4009b80).ready entity=mon.0 client_cookie=b4c52b03a6040020 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:15.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.852 7fa7eaffd700 1 -- 172.21.2.101:0/1091726979 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa7d40009b0 con 0x7fa7ec0e5470 2021-02-10T11:31:15.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.852 7fa7eaffd700 1 -- 172.21.2.101:0/1091726979 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d400c0c0 con 0x7fa7ec0e5470 2021-02-10T11:31:15.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.852 7fa7eaffd700 1 -- 172.21.2.101:0/1091726979 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa7d400c560 con 0x7fa7ec0e5470 2021-02-10T11:31:15.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.852 7fa7f2498700 1 -- 172.21.2.101:0/1091726979 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7ec0e5470 msgr2=0x7fa7ec07d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:15.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.852 7fa7f2498700 1 --2- 172.21.2.101:0/1091726979 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7ec0e5470 0x7fa7ec07d690 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7fa7d4006700 tx=0x7fa7d4009b80).stop 2021-02-10T11:31:15.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.852 7fa7f2498700 1 -- 172.21.2.101:0/1091726979 shutdown_connections 2021-02-10T11:31:15.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.852 7fa7f2498700 1 --2- 172.21.2.101:0/1091726979 >> v2:172.21.2.108:3300/0 conn(0x7fa7ec1b3e40 0x7fa7ec07dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:15.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.852 7fa7f2498700 1 --2- 172.21.2.101:0/1091726979 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7ec0e5470 0x7fa7ec07d690 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:15.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.852 7fa7f2498700 1 --2- 172.21.2.101:0/1091726979 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec0e5070 0x7fa7ec1b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:15.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.852 7fa7f2498700 1 -- 172.21.2.101:0/1091726979 >> 172.21.2.101:0/1091726979 conn(0x7fa7ec0ce690 msgr2=0x7fa7ec0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:15.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.852 7fa7f2498700 1 -- 172.21.2.101:0/1091726979 shutdown_connections 2021-02-10T11:31:15.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.852 7fa7f2498700 1 -- 172.21.2.101:0/1091726979 wait complete. 2021-02-10T11:31:15.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.867 7fa7f2498700 1 Processor -- start 2021-02-10T11:31:15.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.867 7fa7f2498700 1 -- start start 2021-02-10T11:31:15.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.867 7fa7f2498700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7ec1918e0 0x7fa7ec189c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:15.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.867 7fa7f2498700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7ec18a2a0 0x7fa7ec18a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:15.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.867 7fa7f2498700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec18acc0 0x7fa7ec190070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:15.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.867 7fa7f2498700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec1a2f00 con 0x7fa7ec18a2a0 2021-02-10T11:31:15.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.867 7fa7f2498700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec191ce0 con 0x7fa7ec1918e0 2021-02-10T11:31:15.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.867 7fa7f2498700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec191f00 con 0x7fa7ec18acc0 2021-02-10T11:31:15.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.868 7fa7eb7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7ec18a2a0 0x7fa7ec18a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:15.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.868 7fa7eb7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7ec18a2a0 0x7fa7ec18a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49884/0 (socket says 172.21.2.101:49884) 2021-02-10T11:31:15.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.868 7fa7eb7fe700 1 -- 172.21.2.101:0/3064605243 learned_addr learned my addr 172.21.2.101:0/3064605243 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:15.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.868 7fa7ebfff700 1 --2- 172.21.2.101:0/3064605243 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7ec1918e0 0x7fa7ec189c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:15.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.868 7fa7f0ab5700 1 --2- 172.21.2.101:0/3064605243 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec18acc0 0x7fa7ec190070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:15.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.868 7fa7eb7fe700 1 -- 172.21.2.101:0/3064605243 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec18acc0 msgr2=0x7fa7ec190070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:15.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.868 7fa7eb7fe700 1 --2- 172.21.2.101:0/3064605243 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec18acc0 0x7fa7ec190070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:15.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.868 7fa7eb7fe700 1 -- 172.21.2.101:0/3064605243 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7ec1918e0 msgr2=0x7fa7ec189c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:15.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.868 7fa7eb7fe700 1 --2- 172.21.2.101:0/3064605243 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7ec1918e0 0x7fa7ec189c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:15.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.868 7fa7eb7fe700 1 -- 172.21.2.101:0/3064605243 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec190690 con 0x7fa7ec18a2a0 2021-02-10T11:31:15.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.868 7fa7eb7fe700 1 --2- 172.21.2.101:0/3064605243 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7ec18a2a0 0x7fa7ec18a6a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7fa7d4007ce0 tx=0x7fa7d4002620).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:15.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.868 7fa7e8ff9700 1 -- 172.21.2.101:0/3064605243 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa7d40052e0 con 0x7fa7ec18a2a0 2021-02-10T11:31:15.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.868 7fa7e8ff9700 1 -- 172.21.2.101:0/3064605243 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d40055e0 con 0x7fa7ec18a2a0 2021-02-10T11:31:15.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.868 7fa7f2498700 1 -- 172.21.2.101:0/3064605243 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7ec190690 con 0x7fa7ec18a2a0 2021-02-10T11:31:15.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.868 7fa7e8ff9700 1 -- 172.21.2.101:0/3064605243 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa7d4005b00 con 0x7fa7ec18a2a0 2021-02-10T11:31:15.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.868 7fa7f2498700 1 -- 172.21.2.101:0/3064605243 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ec190c20 con 0x7fa7ec18a2a0 2021-02-10T11:31:15.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.869 7fa7e8ff9700 1 -- 172.21.2.101:0/3064605243 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa7d40052e0 con 0x7fa7ec18a2a0 2021-02-10T11:31:15.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.869 7fa7e8ff9700 1 --2- 172.21.2.101:0/3064605243 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7d0018730 0x7fa7d001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:15.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.869 7fa7e8ff9700 1 -- 172.21.2.101:0/3064605243 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa7d4032820 con 0x7fa7ec18a2a0 2021-02-10T11:31:15.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.869 7fa7ebfff700 1 --2- 172.21.2.101:0/3064605243 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7d0018730 0x7fa7d001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:15.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.869 7fa7ebfff700 1 --2- 172.21.2.101:0/3064605243 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7d0018730 0x7fa7d001abc0 crc :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:15.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.901 7fa7f2498700 1 -- 172.21.2.101:0/3064605243 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7ec07ed80 con 0x7fa7ec18a2a0 2021-02-10T11:31:15.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:15.904 7fa7e8ff9700 1 -- 172.21.2.101:0/3064605243 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa7d4002a10 con 0x7fa7ec18a2a0 2021-02-10T11:31:16.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.035 7fa7f2498700 1 -- 172.21.2.101:0/3064605243 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_authorize_allow_existing_id_option"]} v 0) v1 -- 0x7fa7ec07ed80 con 0x7fa7ec18a2a0 2021-02-10T11:31:16.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.432 7fa7e8ff9700 1 -- 172.21.2.101:0/3064605243 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_authorize_allow_existing_id_option"]}]=0 v210) v1 ==== 159+0+0 (secure 0 0 0) 0x7fa7d40052e0 con 0x7fa7ec18a2a0 2021-02-10T11:31:16.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.454 7fa7f2498700 1 -- 172.21.2.101:0/3064605243 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7d0018730 msgr2=0x7fa7d001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:16.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.454 7fa7f2498700 1 --2- 172.21.2.101:0/3064605243 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7d0018730 0x7fa7d001abc0 crc :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:16.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.454 7fa7f2498700 1 -- 172.21.2.101:0/3064605243 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7ec18a2a0 msgr2=0x7fa7ec18a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:16.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.454 7fa7f2498700 1 --2- 172.21.2.101:0/3064605243 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7ec18a2a0 0x7fa7ec18a6a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7fa7d4007ce0 tx=0x7fa7d4002620).stop 2021-02-10T11:31:16.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.454 7fa7f2498700 1 -- 172.21.2.101:0/3064605243 shutdown_connections 2021-02-10T11:31:16.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.454 7fa7f2498700 1 --2- 172.21.2.101:0/3064605243 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7d0018730 0x7fa7d001abc0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:16.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.454 7fa7f2498700 1 --2- 172.21.2.101:0/3064605243 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7ec18acc0 0x7fa7ec190070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:16.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.454 7fa7f2498700 1 --2- 172.21.2.101:0/3064605243 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7ec18a2a0 0x7fa7ec18a6a0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:16.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.454 7fa7f2498700 1 --2- 172.21.2.101:0/3064605243 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7ec1918e0 0x7fa7ec189c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:16.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.454 7fa7f2498700 1 -- 172.21.2.101:0/3064605243 >> 172.21.2.101:0/3064605243 conn(0x7fa7ec1a31d0 msgr2=0x7fa7ec08b830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:16.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.454 7fa7f2498700 1 -- 172.21.2.101:0/3064605243 shutdown_connections 2021-02-10T11:31:16.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.454 7fa7f2498700 1 -- 172.21.2.101:0/3064605243 wait complete. 2021-02-10T11:31:16.474 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:31:16.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.875 7f7065869700 1 Processor -- start 2021-02-10T11:31:16.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.876 7f7065869700 1 -- start start 2021-02-10T11:31:16.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.876 7f7065869700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70601b0470 0x7f70601b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:16.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.876 7f7065869700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f70600f20e0 0x7f70600f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:16.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.876 7f7065869700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70600f0b50 con 0x7f70600ef610 2021-02-10T11:31:16.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.876 7f7065869700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70600f0820 con 0x7f70600f20e0 2021-02-10T11:31:16.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.876 7f7065869700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70600f24e0 con 0x7f70601b0470 2021-02-10T11:31:16.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.876 7f705ed73700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f70600ef610 0x7f70600efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50384/0 (socket says 172.21.2.101:50384) 2021-02-10T11:31:16.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.876 7f705ed73700 1 -- 172.21.2.101:0/234339926 learned_addr learned my addr 172.21.2.101:0/234339926 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:16.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.877 7f705e572700 1 -- 172.21.2.101:0/234339926 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4126901987 0 0) 0x7f7054001180 con 0x7f70600ef610 2021-02-10T11:31:16.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.877 7f705e572700 1 -- 172.21.2.101:0/234339926 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7048002fe0 con 0x7f70600ef610 2021-02-10T11:31:16.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.877 7f705f574700 1 --2- 172.21.2.101:0/234339926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70601b0470 0x7f70601b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:16.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.877 7f705fd75700 1 --2- 172.21.2.101:0/234339926 >> v2:172.21.2.108:3300/0 conn(0x7f70600f20e0 0x7f70600f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:16.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.877 7f705e572700 1 -- 172.21.2.101:0/234339926 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2437944016 0 0) 0x7f7054001180 con 0x7f70600ef610 2021-02-10T11:31:16.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.877 7f705e572700 1 -- 172.21.2.101:0/234339926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70601b0470 msgr2=0x7f70601b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:16.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.877 7f705e572700 1 --2- 172.21.2.101:0/234339926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70601b0470 0x7f70601b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:16.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.877 7f705e572700 1 -- 172.21.2.101:0/234339926 >> v2:172.21.2.108:3300/0 conn(0x7f70600f20e0 msgr2=0x7f70600f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:16.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.877 7f705e572700 1 --2- 172.21.2.101:0/234339926 >> v2:172.21.2.108:3300/0 conn(0x7f70600f20e0 0x7f70600f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:16.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.877 7f705e572700 1 -- 172.21.2.101:0/234339926 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70600e52e0 con 0x7f70600ef610 2021-02-10T11:31:16.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.877 7f705e572700 1 -- 172.21.2.101:0/234339926 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f7054001680 con 0x7f70600ef610 2021-02-10T11:31:16.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.877 7f705e572700 1 -- 172.21.2.101:0/234339926 >> v1:172.21.2.101:6789/0 conn(0x7f70600ef610 legacy=0x7f70600efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:16.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.877 7f705e572700 1 --2- 172.21.2.101:0/234339926 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7048003a60 0x7f7048005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:16.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.878 7f705e572700 1 --2- 172.21.2.101:0/234339926 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f704804a7a0 0x7f704804cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:16.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.878 7f705e572700 1 --2- 172.21.2.101:0/234339926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f704804d5a0 0x7f704804fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:16.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.878 7f705e572700 1 -- 172.21.2.101:0/234339926 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70480502e0 con 0x7f704804a7a0 2021-02-10T11:31:16.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.878 7f705e572700 1 -- 172.21.2.101:0/234339926 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7048050500 con 0x7f7048003a60 2021-02-10T11:31:16.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.878 7f705e572700 1 -- 172.21.2.101:0/234339926 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7048050720 con 0x7f704804d5a0 2021-02-10T11:31:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.878 7f705ed73700 1 --2- 172.21.2.101:0/234339926 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f704804a7a0 0x7f704804cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.878 7f705f574700 1 --2- 172.21.2.101:0/234339926 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7048003a60 0x7f7048005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.878 7f705fd75700 1 --2- 172.21.2.101:0/234339926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f704804d5a0 0x7f704804fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.878 7f705ed73700 1 -- 172.21.2.101:0/234339926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f704804d5a0 msgr2=0x7f704804fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.878 7f705ed73700 1 --2- 172.21.2.101:0/234339926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f704804d5a0 0x7f704804fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.878 7f705ed73700 1 -- 172.21.2.101:0/234339926 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7048003a60 msgr2=0x7f7048005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.878 7f705ed73700 1 --2- 172.21.2.101:0/234339926 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7048003a60 0x7f7048005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.878 7f705ed73700 1 -- 172.21.2.101:0/234339926 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70480509c0 con 0x7f704804a7a0 2021-02-10T11:31:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.878 7f705fd75700 1 --2- 172.21.2.101:0/234339926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f704804d5a0 0x7f704804fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:31:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.878 7f705ed73700 1 --2- 172.21.2.101:0/234339926 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f704804a7a0 0x7f704804cca0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7f70540076f0 tx=0x7f7054001c60).ready entity=mon.0 client_cookie=ee837ddf3b6f4640 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.878 7f705e572700 1 -- 172.21.2.101:0/234339926 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f70540070c0 con 0x7f704804a7a0 2021-02-10T11:31:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.878 7f705e572700 1 -- 172.21.2.101:0/234339926 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7054007340 con 0x7f704804a7a0 2021-02-10T11:31:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.878 7f705e572700 1 -- 172.21.2.101:0/234339926 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7054004620 con 0x7f704804a7a0 2021-02-10T11:31:16.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.878 7f7065869700 1 -- 172.21.2.101:0/234339926 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f704804a7a0 msgr2=0x7f704804cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:16.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.878 7f7065869700 1 --2- 172.21.2.101:0/234339926 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f704804a7a0 0x7f704804cca0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7f70540076f0 tx=0x7f7054001c60).stop 2021-02-10T11:31:16.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.879 7f705fd75700 1 -- 172.21.2.101:0/234339926 reap_dead start 2021-02-10T11:31:16.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.879 7f7065869700 1 -- 172.21.2.101:0/234339926 shutdown_connections 2021-02-10T11:31:16.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.879 7f7065869700 1 -- 172.21.2.101:0/234339926 >> 172.21.2.101:0/234339926 conn(0x7f70600ce3e0 msgr2=0x7f70600e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:16.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.879 7f7065869700 1 -- 172.21.2.101:0/234339926 shutdown_connections 2021-02-10T11:31:16.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.879 7f7065869700 1 -- 172.21.2.101:0/234339926 wait complete. 2021-02-10T11:31:16.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.893 7f7065869700 1 Processor -- start 2021-02-10T11:31:16.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.894 7f7065869700 1 -- start start 2021-02-10T11:31:16.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.894 7f7065869700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f706009b930 0x7f70600b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:16.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.894 7f7065869700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70600b8410 0x7f70600b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:16.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.894 7f7065869700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70600c1f80 0x7f70600c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:16.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.894 7f7065869700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70600c1a00 con 0x7f70600b8410 2021-02-10T11:31:16.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.894 7f7065869700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70600b8e30 con 0x7f706009b930 2021-02-10T11:31:16.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.894 7f7065869700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70600b9050 con 0x7f70600c1f80 2021-02-10T11:31:16.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.894 7f705ed73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70600b8410 0x7f70600b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:16.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.894 7f705ed73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70600b8410 0x7f70600b8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49900/0 (socket says 172.21.2.101:49900) 2021-02-10T11:31:16.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.894 7f705ed73700 1 -- 172.21.2.101:0/3768314791 learned_addr learned my addr 172.21.2.101:0/3768314791 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:16.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.894 7f705ed73700 1 -- 172.21.2.101:0/3768314791 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70600c1f80 msgr2=0x7f70600c2380 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:31:16.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.894 7f705ed73700 1 --2- 172.21.2.101:0/3768314791 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70600c1f80 0x7f70600c2380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:16.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.894 7f705ed73700 1 -- 172.21.2.101:0/3768314791 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f706009b930 msgr2=0x7f70600b7c20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:31:16.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.894 7f705ed73700 1 --2- 172.21.2.101:0/3768314791 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f706009b930 0x7f70600b7c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:16.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.894 7f705ed73700 1 -- 172.21.2.101:0/3768314791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70600c29a0 con 0x7f70600b8410 2021-02-10T11:31:16.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.895 7f705ed73700 1 --2- 172.21.2.101:0/3768314791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70600b8410 0x7f70600b8810 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7f705400a580 tx=0x7f705400a920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:16.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.895 7f7047fff700 1 -- 172.21.2.101:0/3768314791 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f705400ad10 con 0x7f70600b8410 2021-02-10T11:31:16.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.895 7f7047fff700 1 -- 172.21.2.101:0/3768314791 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70540070d0 con 0x7f70600b8410 2021-02-10T11:31:16.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.895 7f7047fff700 1 -- 172.21.2.101:0/3768314791 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7054004830 con 0x7f70600b8410 2021-02-10T11:31:16.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.895 7f7065869700 1 -- 172.21.2.101:0/3768314791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70600c29a0 con 0x7f70600b8410 2021-02-10T11:31:16.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.895 7f7065869700 1 -- 172.21.2.101:0/3768314791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70600c2f30 con 0x7f70600b8410 2021-02-10T11:31:16.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.896 7f7047fff700 1 -- 172.21.2.101:0/3768314791 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f70540047c0 con 0x7f70600b8410 2021-02-10T11:31:16.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.896 7f7047fff700 1 --2- 172.21.2.101:0/3768314791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70400186c0 0x7f704001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:16.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.896 7f7047fff700 1 -- 172.21.2.101:0/3768314791 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7054032550 con 0x7f70600b8410 2021-02-10T11:31:16.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.896 7f705f574700 1 --2- 172.21.2.101:0/3768314791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70400186c0 0x7f704001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:16.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.896 7f705f574700 1 --2- 172.21.2.101:0/3768314791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70400186c0 0x7f704001abc0 crc :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:16.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.926 7f7065869700 1 -- 172.21.2.101:0/3768314791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7060105b30 con 0x7f70600b8410 2021-02-10T11:31:16.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:16.928 7f7047fff700 1 -- 172.21.2.101:0/3768314791 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f70540045b0 con 0x7f70600b8410 2021-02-10T11:31:17.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:17.077 7f7065869700 1 -- 172.21.2.101:0/3768314791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f70600f0b50 con 0x7f70600b8410 2021-02-10T11:31:17.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:17.081 7f7047fff700 1 -- 172.21.2.101:0/3768314791 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v4)=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7f70540047c0 con 0x7f70600b8410 2021-02-10T11:31:17.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:17.086 7f7065869700 1 -- 172.21.2.101:0/3768314791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70400186c0 msgr2=0x7f704001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:17.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:17.086 7f7065869700 1 --2- 172.21.2.101:0/3768314791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70400186c0 0x7f704001abc0 crc :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:17.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:17.086 7f7065869700 1 -- 172.21.2.101:0/3768314791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70600b8410 msgr2=0x7f70600b8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:17.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:17.086 7f7065869700 1 --2- 172.21.2.101:0/3768314791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70600b8410 0x7f70600b8810 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7f705400a580 tx=0x7f705400a920).stop 2021-02-10T11:31:17.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:17.086 7f7065869700 1 -- 172.21.2.101:0/3768314791 shutdown_connections 2021-02-10T11:31:17.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:17.086 7f7065869700 1 --2- 172.21.2.101:0/3768314791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70400186c0 0x7f704001abc0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:17.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:17.086 7f7065869700 1 --2- 172.21.2.101:0/3768314791 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70600c1f80 0x7f70600c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:17.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:17.086 7f7065869700 1 --2- 172.21.2.101:0/3768314791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70600b8410 0x7f70600b8810 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:17.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:17.086 7f7065869700 1 --2- 172.21.2.101:0/3768314791 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f706009b930 0x7f70600b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:17.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:17.086 7f7065869700 1 -- 172.21.2.101:0/3768314791 >> 172.21.2.101:0/3768314791 conn(0x7f7060094e10 msgr2=0x7f70600a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:17.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:17.086 7f7065869700 1 -- 172.21.2.101:0/3768314791 shutdown_connections 2021-02-10T11:31:17.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:17.086 7f7065869700 1 -- 172.21.2.101:0/3768314791 wait complete. 2021-02-10T11:31:17.106 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:31:17.106 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:31:17.452 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:31:17.796 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:31:17.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:17.796 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:31:17.797 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:31:18.130 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2021-02-10T11:31:18.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:31:18.131 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-02-10T11:31:18.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:18.132 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:31:18.132 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:31:18.466 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:31:18.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:31:18.466 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:31:18.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:18.466 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:31:18.467 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:31:18.801 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:31:18.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:31:18.802 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:31:18.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:18.803 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:31:18.803 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:31:19.137 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:31:19.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:31:19.138 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:31:19.138 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json-pretty 2021-02-10T11:31:19.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.536 7f972973f700 1 Processor -- start 2021-02-10T11:31:19.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.536 7f972973f700 1 -- start start 2021-02-10T11:31:19.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.536 7f972973f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97240cff00 0x7f97241b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:19.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.536 7f972973f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f97240d1bf0 0x7f97240ef620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:19.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.536 7f972973f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97240f0b20 con 0x7f97240d17f0 2021-02-10T11:31:19.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.536 7f972973f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97241b3ff0 con 0x7f97240d1bf0 2021-02-10T11:31:19.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.536 7f972973f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97240f20b0 con 0x7f97240cff00 2021-02-10T11:31:19.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.538 7f9723d4a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f97240d1bf0 0x7f97240ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:19.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.538 7f9722d48700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97240cff00 0x7f97241b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:19.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.538 7f9723549700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f97240d17f0 0x7f97241b31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:43812/0 (socket says 172.21.2.101:43812) 2021-02-10T11:31:19.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.538 7f9723549700 1 -- 172.21.2.101:0/322704374 learned_addr learned my addr 172.21.2.101:0/322704374 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:19.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.538 7f9723d4a700 1 -- 172.21.2.101:0/322704374 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97240cff00 msgr2=0x7f97241b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:19.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.538 7f9723d4a700 1 --2- 172.21.2.101:0/322704374 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97240cff00 0x7f97241b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:19.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.538 7f9723d4a700 1 -- 172.21.2.101:0/322704374 >> v1:172.21.2.108:6789/0 conn(0x7f97240d17f0 legacy=0x7f97241b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:19.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.538 7f9722547700 1 -- 172.21.2.101:0/322704374 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2183466027 0 0) 0x7f97140010b0 con 0x7f97240d17f0 2021-02-10T11:31:19.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.538 7f9723d4a700 1 -- 172.21.2.101:0/322704374 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97240f22a0 con 0x7f97240d1bf0 2021-02-10T11:31:19.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.538 7f9722d48700 1 --2- 172.21.2.101:0/322704374 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97240cff00 0x7f97241b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:31:19.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.538 7f9723d4a700 1 --2- 172.21.2.101:0/322704374 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97240d1bf0 0x7f97240ef620 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f971c006e50 tx=0x7f971c009e70).ready entity=mon.1 client_cookie=cb98364d3dcc7a95 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:19.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.539 7f9722547700 1 -- 172.21.2.101:0/322704374 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f971c006a20 con 0x7f97240d1bf0 2021-02-10T11:31:19.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.539 7f9722547700 1 -- 172.21.2.101:0/322704374 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971c006790 con 0x7f97240d1bf0 2021-02-10T11:31:19.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.539 7f9722547700 1 -- 172.21.2.101:0/322704374 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f971c00b700 con 0x7f97240d1bf0 2021-02-10T11:31:19.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.539 7f972973f700 1 -- 172.21.2.101:0/322704374 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97240d1bf0 msgr2=0x7f97240ef620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:19.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.539 7f972973f700 1 --2- 172.21.2.101:0/322704374 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97240d1bf0 0x7f97240ef620 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f971c006e50 tx=0x7f971c009e70).stop 2021-02-10T11:31:19.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.539 7f972973f700 1 -- 172.21.2.101:0/322704374 shutdown_connections 2021-02-10T11:31:19.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.539 7f972973f700 1 --2- 172.21.2.101:0/322704374 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97240cff00 0x7f97241b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:19.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.539 7f972973f700 1 --2- 172.21.2.101:0/322704374 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97240d1bf0 0x7f97240ef620 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:19.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.539 7f972973f700 1 -- 172.21.2.101:0/322704374 >> 172.21.2.101:0/322704374 conn(0x7f97240fde40 msgr2=0x7f97240ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:19.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.539 7f972973f700 1 -- 172.21.2.101:0/322704374 shutdown_connections 2021-02-10T11:31:19.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.539 7f972973f700 1 -- 172.21.2.101:0/322704374 wait complete. 2021-02-10T11:31:19.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.554 7f972973f700 1 Processor -- start 2021-02-10T11:31:19.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.554 7f972973f700 1 -- start start 2021-02-10T11:31:19.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.554 7f972973f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f97241a48f0 0x7f97240b85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:19.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.554 7f972973f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97240b9320 0x7f972409bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:19.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.554 7f972973f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9724107bf0 0x7f9724107ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:19.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.554 7f972973f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97240b8da0 con 0x7f97241a48f0 2021-02-10T11:31:19.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.554 7f972973f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97240c26a0 con 0x7f97240b9320 2021-02-10T11:31:19.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.554 7f972973f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97240c2880 con 0x7f9724107bf0 2021-02-10T11:31:19.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.554 7f9723549700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f97241a48f0 0x7f97240b85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:19.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.554 7f9723549700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f97241a48f0 0x7f97240b85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49912/0 (socket says 172.21.2.101:49912) 2021-02-10T11:31:19.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.554 7f9723549700 1 -- 172.21.2.101:0/2518928022 learned_addr learned my addr 172.21.2.101:0/2518928022 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:19.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.555 7f9723549700 1 -- 172.21.2.101:0/2518928022 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9724107bf0 msgr2=0x7f9724107ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:31:19.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.555 7f9723549700 1 --2- 172.21.2.101:0/2518928022 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9724107bf0 0x7f9724107ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:19.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.555 7f9723549700 1 -- 172.21.2.101:0/2518928022 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97240b9320 msgr2=0x7f972409bbb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:31:19.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.555 7f9723549700 1 --2- 172.21.2.101:0/2518928022 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97240b9320 0x7f972409bbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:19.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.555 7f9723549700 1 -- 172.21.2.101:0/2518928022 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97240a6620 con 0x7f97241a48f0 2021-02-10T11:31:19.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.555 7f9723549700 1 --2- 172.21.2.101:0/2518928022 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f97241a48f0 0x7f97240b85b0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7f9714001c10 tx=0x7f971400a8f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:19.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.555 7f971bfff700 1 -- 172.21.2.101:0/2518928022 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f97140009b0 con 0x7f97241a48f0 2021-02-10T11:31:19.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.555 7f971bfff700 1 -- 172.21.2.101:0/2518928022 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97140012c0 con 0x7f97241a48f0 2021-02-10T11:31:19.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.555 7f972973f700 1 -- 172.21.2.101:0/2518928022 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97240a6620 con 0x7f97241a48f0 2021-02-10T11:31:19.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.555 7f972973f700 1 -- 172.21.2.101:0/2518928022 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97240a6620 con 0x7f97241a48f0 2021-02-10T11:31:19.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.555 7f971bfff700 1 -- 172.21.2.101:0/2518928022 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9714004350 con 0x7f97241a48f0 2021-02-10T11:31:19.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.556 7f971bfff700 1 -- 172.21.2.101:0/2518928022 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9714004350 con 0x7f97241a48f0 2021-02-10T11:31:19.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.556 7f971bfff700 1 --2- 172.21.2.101:0/2518928022 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97080186c0 0x7f970801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:19.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.556 7f971bfff700 1 -- 172.21.2.101:0/2518928022 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9714032520 con 0x7f97241a48f0 2021-02-10T11:31:19.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.556 7f9722d48700 1 --2- 172.21.2.101:0/2518928022 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97080186c0 0x7f970801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:19.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.556 7f9722d48700 1 --2- 172.21.2.101:0/2518928022 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97080186c0 0x7f970801abc0 crc :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:19.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.586 7f972973f700 1 -- 172.21.2.101:0/2518928022 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97241036a0 con 0x7f97241a48f0 2021-02-10T11:31:19.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.589 7f971bfff700 1 -- 172.21.2.101:0/2518928022 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9714026150 con 0x7f97241a48f0 2021-02-10T11:31:19.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.713 7f972973f700 1 -- 172.21.2.101:0/2518928022 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f97240f0b20 con 0x7f97241a48f0 2021-02-10T11:31:19.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.714 7f971bfff700 1 -- 172.21.2.101:0/2518928022 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v29) v1 ==== 81+0+30360 (secure 0 0 0) 0x7f9714026150 con 0x7f97241a48f0 2021-02-10T11:31:19.726 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:31:19.726 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:31:19.726 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 29, 2021-02-10T11:31:19.726 INFO:teuthology.orchestra.run.gibba001.stdout: "fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:31:19.726 INFO:teuthology.orchestra.run.gibba001.stdout: "created": "2021-02-10 11:12:36.788366", 2021-02-10T11:31:19.726 INFO:teuthology.orchestra.run.gibba001.stdout: "modified": "2021-02-10 11:14:58.132054", 2021-02-10T11:31:19.726 INFO:teuthology.orchestra.run.gibba001.stdout: "last_up_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:31:19.726 INFO:teuthology.orchestra.run.gibba001.stdout: "last_in_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:31:19.726 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-02-10T11:31:19.726 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_num": 5799936, 2021-02-10T11:31:19.727 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_set": [ 2021-02-10T11:31:19.727 INFO:teuthology.orchestra.run.gibba001.stdout: "pglog_hardlimit", 2021-02-10T11:31:19.727 INFO:teuthology.orchestra.run.gibba001.stdout: "purged_snapdirs", 2021-02-10T11:31:19.727 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_deletes", 2021-02-10T11:31:19.727 INFO:teuthology.orchestra.run.gibba001.stdout: "sortbitwise" 2021-02-10T11:31:19.727 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:31:19.727 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_version": 4, 2021-02-10T11:31:19.727 INFO:teuthology.orchestra.run.gibba001.stdout: "full_ratio": 0.94999998807907104, 2021-02-10T11:31:19.727 INFO:teuthology.orchestra.run.gibba001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-02-10T11:31:19.727 INFO:teuthology.orchestra.run.gibba001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-02-10T11:31:19.727 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_snapshot": "", 2021-02-10T11:31:19.727 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_max": 5, 2021-02-10T11:31:19.727 INFO:teuthology.orchestra.run.gibba001.stdout: "max_osd": 8, 2021-02-10T11:31:19.728 INFO:teuthology.orchestra.run.gibba001.stdout: "require_min_compat_client": "jewel", 2021-02-10T11:31:19.728 INFO:teuthology.orchestra.run.gibba001.stdout: "min_compat_client": "jewel", 2021-02-10T11:31:19.728 INFO:teuthology.orchestra.run.gibba001.stdout: "require_osd_release": "nautilus", 2021-02-10T11:31:19.728 INFO:teuthology.orchestra.run.gibba001.stdout: "pools": [ 2021-02-10T11:31:19.728 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.728 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 1, 2021-02-10T11:31:19.728 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "rbd", 2021-02-10T11:31:19.728 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:13:08.037472", 2021-02-10T11:31:19.728 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:31:19.728 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:31:19.728 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:31:19.729 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:31:19.729 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:31:19.729 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:31:19.729 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:31:19.729 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:31:19.729 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:31:19.729 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:31:19.729 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:31:19.729 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:31:19.730 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:31:19.730 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:31:19.730 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:31:19.730 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:31:19.730 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:31:19.730 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:31:19.730 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:31:19.730 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:31:19.730 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.731 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "16", 2021-02-10T11:31:19.731 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:31:19.731 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:31:19.731 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:31:19.731 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:31:19.731 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:31:19.731 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:31:19.731 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:31:19.731 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:31:19.732 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:31:19.732 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:31:19.732 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:31:19.732 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:31:19.732 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:31:19.732 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:31:19.732 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:31:19.732 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:31:19.732 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:31:19.732 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:31:19.732 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:31:19.732 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:31:19.733 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:31:19.733 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:31:19.733 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:31:19.733 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:31:19.733 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:31:19.733 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:31:19.733 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.733 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:31:19.733 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:31:19.733 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:31:19.733 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:31:19.734 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:31:19.734 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:31:19.734 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:31:19.734 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:31:19.734 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:31:19.734 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:31:19.734 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:31:19.734 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:31:19.734 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:31:19.734 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd": {} 2021-02-10T11:31:19.735 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.735 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.735 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.735 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 4, 2021-02-10T11:31:19.735 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_metadata", 2021-02-10T11:31:19.735 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:14:54.652789", 2021-02-10T11:31:19.735 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:31:19.735 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:31:19.735 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:31:19.735 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:31:19.735 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:31:19.735 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:31:19.735 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:31:19.736 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:31:19.736 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:31:19.736 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:31:19.736 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:31:19.736 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:31:19.736 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:31:19.736 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:31:19.736 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:31:19.736 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:31:19.736 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:31:19.736 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:31:19.736 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:31:19.736 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:31:19.737 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.737 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "28", 2021-02-10T11:31:19.737 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:31:19.737 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:31:19.737 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:31:19.737 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:31:19.737 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:31:19.737 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:31:19.737 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:31:19.737 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:31:19.737 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:31:19.738 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:31:19.738 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:31:19.738 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:31:19.738 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:31:19.738 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:31:19.738 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:31:19.738 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:31:19.738 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:31:19.738 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:31:19.738 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:31:19.738 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:31:19.738 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:31:19.738 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:31:19.739 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:31:19.739 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:31:19.739 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:31:19.739 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:31:19.739 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.739 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:31:19.739 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:31:19.739 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:31:19.739 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:31:19.739 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:31:19.739 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:31:19.739 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:31:19.739 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:31:19.740 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:31:19.740 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:31:19.740 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:31:19.740 INFO:teuthology.orchestra.run.gibba001.stdout: "options": { 2021-02-10T11:31:19.740 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_bias": 4, 2021-02-10T11:31:19.740 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_min": 16, 2021-02-10T11:31:19.740 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_priority": 5 2021-02-10T11:31:19.740 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.740 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:31:19.740 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:31:19.740 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": "cephfs" 2021-02-10T11:31:19.740 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.740 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.741 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.741 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.741 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 5, 2021-02-10T11:31:19.741 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_data", 2021-02-10T11:31:19.741 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:14:56.125641", 2021-02-10T11:31:19.741 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:31:19.741 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:31:19.741 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:31:19.741 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:31:19.741 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:31:19.741 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:31:19.741 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:31:19.741 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:31:19.741 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:31:19.742 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:31:19.742 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:31:19.742 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:31:19.742 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:31:19.742 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:31:19.742 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:31:19.742 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:31:19.742 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:31:19.742 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:31:19.742 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:31:19.742 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:31:19.742 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.743 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "29", 2021-02-10T11:31:19.743 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:31:19.743 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:31:19.743 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:31:19.743 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:31:19.743 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:31:19.743 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:31:19.743 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:31:19.743 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:31:19.743 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:31:19.743 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:31:19.743 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:31:19.744 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:31:19.744 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:31:19.744 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:31:19.744 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:31:19.744 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:31:19.744 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:31:19.744 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:31:19.744 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:31:19.744 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:31:19.744 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:31:19.744 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:31:19.744 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:31:19.744 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:31:19.745 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:31:19.745 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:31:19.745 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.745 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:31:19.745 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:31:19.745 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:31:19.745 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:31:19.745 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:31:19.745 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:31:19.745 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:31:19.745 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:31:19.745 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:31:19.746 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:31:19.746 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:31:19.746 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:31:19.746 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:31:19.746 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:31:19.746 INFO:teuthology.orchestra.run.gibba001.stdout: "data": "cephfs" 2021-02-10T11:31:19.746 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.746 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.746 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.746 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:31:19.746 INFO:teuthology.orchestra.run.gibba001.stdout: "osds": [ 2021-02-10T11:31:19.746 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.747 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:31:19.747 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5", 2021-02-10T11:31:19.747 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:31:19.747 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:31:19.747 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:31:19.747 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:31:19.747 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:31:19.747 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:31:19.748 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:31:19.748 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 27, 2021-02-10T11:31:19.748 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:31:19.748 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:31:19.748 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:31:19.748 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.748 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.748 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.748 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6802", 2021-02-10T11:31:19.748 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:31:19.748 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.748 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.748 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.749 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6805", 2021-02-10T11:31:19.749 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:31:19.749 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.749 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.749 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.749 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:31:19.749 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.749 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.749 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.749 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6807", 2021-02-10T11:31:19.749 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:31:19.749 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.749 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.749 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.750 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6809", 2021-02-10T11:31:19.750 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:31:19.750 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.750 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.750 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.750 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:31:19.750 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.750 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.750 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.750 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6817", 2021-02-10T11:31:19.750 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:31:19.750 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.750 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.751 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.751 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6820", 2021-02-10T11:31:19.751 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:31:19.751 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.751 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.751 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.751 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:31:19.751 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.751 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.751 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.751 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6812", 2021-02-10T11:31:19.751 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:31:19.751 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.752 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.752 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.752 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6814", 2021-02-10T11:31:19.752 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:31:19.752 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.752 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.752 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.752 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6805/25093", 2021-02-10T11:31:19.752 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6809/25093", 2021-02-10T11:31:19.752 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6820/25093", 2021-02-10T11:31:19.752 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6814/25093", 2021-02-10T11:31:19.752 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:31:19.752 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:31:19.752 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:31:19.753 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.753 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.753 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.753 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:31:19.753 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "ddc04ce3-b313-4707-8270-585391e38ecc", 2021-02-10T11:31:19.753 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:31:19.753 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:31:19.753 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:31:19.753 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:31:19.753 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:31:19.753 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:31:19.753 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:31:19.753 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 26, 2021-02-10T11:31:19.753 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:31:19.754 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:31:19.754 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:31:19.754 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.754 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.754 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.754 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6803", 2021-02-10T11:31:19.754 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:31:19.754 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.754 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.754 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.754 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6804", 2021-02-10T11:31:19.754 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:31:19.754 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.755 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.755 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.755 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:31:19.755 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.755 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.755 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.755 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6806", 2021-02-10T11:31:19.755 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:31:19.755 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.755 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.755 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.755 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6808", 2021-02-10T11:31:19.755 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:31:19.756 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.756 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.756 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.756 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:31:19.756 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.756 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.756 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.756 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6818", 2021-02-10T11:31:19.756 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:31:19.756 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.756 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.756 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.756 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6821", 2021-02-10T11:31:19.756 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:31:19.757 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.757 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.757 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.757 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:31:19.757 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.757 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.757 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.757 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6810", 2021-02-10T11:31:19.757 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:31:19.757 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.757 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.757 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.757 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6815", 2021-02-10T11:31:19.758 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:31:19.758 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.758 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.758 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.758 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6804/25092", 2021-02-10T11:31:19.758 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6808/25092", 2021-02-10T11:31:19.758 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6821/25092", 2021-02-10T11:31:19.758 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6815/25092", 2021-02-10T11:31:19.758 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:31:19.758 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:31:19.758 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:31:19.758 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.759 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.759 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.759 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:31:19.759 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3", 2021-02-10T11:31:19.759 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:31:19.759 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:31:19.759 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:31:19.759 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:31:19.759 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:31:19.759 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:31:19.759 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:31:19.759 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 27, 2021-02-10T11:31:19.760 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:31:19.760 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:31:19.760 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:31:19.760 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.760 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.760 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.760 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6811", 2021-02-10T11:31:19.760 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:31:19.760 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.760 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.760 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.760 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6813", 2021-02-10T11:31:19.760 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:31:19.761 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.761 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.761 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.761 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:31:19.761 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.761 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.761 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.761 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6816", 2021-02-10T11:31:19.761 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:31:19.761 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.761 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.761 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.762 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6819", 2021-02-10T11:31:19.762 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:31:19.762 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.762 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.762 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.762 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:31:19.762 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.762 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.762 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.762 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6825", 2021-02-10T11:31:19.762 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:31:19.762 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.762 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.763 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.763 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6827", 2021-02-10T11:31:19.763 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:31:19.763 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.763 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.763 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.763 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:31:19.763 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.763 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.763 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.763 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6822", 2021-02-10T11:31:19.763 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:31:19.763 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.763 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.764 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.764 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6823", 2021-02-10T11:31:19.764 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:31:19.764 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.764 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.764 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.764 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6813/25094", 2021-02-10T11:31:19.764 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6819/25094", 2021-02-10T11:31:19.764 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6827/25094", 2021-02-10T11:31:19.764 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6823/25094", 2021-02-10T11:31:19.764 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:31:19.764 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:31:19.764 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:31:19.765 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.765 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.765 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.765 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:31:19.765 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "0d9ef9b1-98aa-4d37-aa16-6305c7d46d52", 2021-02-10T11:31:19.765 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:31:19.765 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:31:19.765 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:31:19.765 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:31:19.765 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:31:19.765 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:31:19.765 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:31:19.765 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 27, 2021-02-10T11:31:19.765 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:31:19.766 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:31:19.766 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:31:19.766 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.766 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.766 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.766 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6824", 2021-02-10T11:31:19.766 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:31:19.766 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.766 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.766 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.766 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6826", 2021-02-10T11:31:19.766 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:31:19.766 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.767 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.767 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.767 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:31:19.767 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.767 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.767 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.767 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6828", 2021-02-10T11:31:19.767 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:31:19.767 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.767 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.767 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.767 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6829", 2021-02-10T11:31:19.767 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:31:19.768 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.768 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.768 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.768 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:31:19.768 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.768 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.768 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.768 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6832", 2021-02-10T11:31:19.768 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:31:19.768 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.768 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.768 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.768 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6833", 2021-02-10T11:31:19.768 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:31:19.769 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.769 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.769 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.769 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:31:19.769 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.769 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.769 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.769 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6830", 2021-02-10T11:31:19.769 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:31:19.769 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.769 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.769 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.769 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6831", 2021-02-10T11:31:19.770 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:31:19.770 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.770 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.770 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.770 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6826/25095", 2021-02-10T11:31:19.770 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6829/25095", 2021-02-10T11:31:19.770 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6833/25095", 2021-02-10T11:31:19.770 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6831/25095", 2021-02-10T11:31:19.770 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:31:19.770 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:31:19.770 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:31:19.770 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.771 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.771 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.771 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:31:19.771 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "36742c21-bfeb-4e11-bce9-8e745378b605", 2021-02-10T11:31:19.771 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:31:19.771 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:31:19.771 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:31:19.771 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:31:19.771 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:31:19.771 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:31:19.771 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:31:19.771 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 27, 2021-02-10T11:31:19.771 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:31:19.772 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:31:19.772 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:31:19.772 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.772 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.772 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.772 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6816", 2021-02-10T11:31:19.772 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:31:19.772 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.772 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.772 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.772 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6818", 2021-02-10T11:31:19.772 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:31:19.772 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.772 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.773 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.773 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:31:19.773 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.773 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.773 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.773 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6820", 2021-02-10T11:31:19.773 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:31:19.773 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.773 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.773 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.773 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6822", 2021-02-10T11:31:19.773 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:31:19.773 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.774 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.774 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.774 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:31:19.774 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.774 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.774 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.774 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6828", 2021-02-10T11:31:19.774 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:31:19.774 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.774 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.774 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.774 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6830", 2021-02-10T11:31:19.775 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:31:19.775 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.775 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.775 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.775 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:31:19.775 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.775 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.775 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.775 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6824", 2021-02-10T11:31:19.775 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:31:19.776 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.776 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.776 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.776 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6826", 2021-02-10T11:31:19.776 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:31:19.776 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.776 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.776 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.776 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6818/36380", 2021-02-10T11:31:19.776 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6822/36380", 2021-02-10T11:31:19.776 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6830/36380", 2021-02-10T11:31:19.776 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6826/36380", 2021-02-10T11:31:19.776 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:31:19.777 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:31:19.777 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:31:19.777 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.777 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.777 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.777 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:31:19.777 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "88e694e8-fe98-4d1b-88c8-8c9907ece60b", 2021-02-10T11:31:19.777 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:31:19.777 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:31:19.777 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:31:19.777 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:31:19.777 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:31:19.777 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:31:19.778 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:31:19.778 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 27, 2021-02-10T11:31:19.778 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:31:19.778 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:31:19.778 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:31:19.778 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.778 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.778 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.778 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6808", 2021-02-10T11:31:19.778 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:31:19.778 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.778 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.778 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.779 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6809", 2021-02-10T11:31:19.779 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:31:19.779 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.779 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.779 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.779 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:31:19.779 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.779 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.779 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.779 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6810", 2021-02-10T11:31:19.779 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:31:19.779 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.779 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.780 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.780 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6811", 2021-02-10T11:31:19.780 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:31:19.780 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.780 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.780 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.780 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:31:19.780 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.780 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.780 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.780 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6814", 2021-02-10T11:31:19.780 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:31:19.780 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.781 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.781 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.781 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6815", 2021-02-10T11:31:19.781 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:31:19.781 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.781 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.781 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.781 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:31:19.781 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.781 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.781 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.781 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6812", 2021-02-10T11:31:19.781 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:31:19.781 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.782 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.782 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.782 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6813", 2021-02-10T11:31:19.782 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:31:19.782 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.782 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.782 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.782 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6809/36383", 2021-02-10T11:31:19.782 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6811/36383", 2021-02-10T11:31:19.782 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6815/36383", 2021-02-10T11:31:19.782 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6813/36383", 2021-02-10T11:31:19.782 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:31:19.782 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:31:19.783 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:31:19.783 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.783 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.783 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.783 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:31:19.783 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "87a13c6c-cf85-4aaf-916b-e73913484a49", 2021-02-10T11:31:19.783 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:31:19.783 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:31:19.783 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:31:19.783 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:31:19.783 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:31:19.783 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:31:19.783 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:31:19.784 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 27, 2021-02-10T11:31:19.784 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:31:19.784 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:31:19.784 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:31:19.784 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.784 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.784 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.784 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6800", 2021-02-10T11:31:19.784 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:31:19.784 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.784 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.784 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.785 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6801", 2021-02-10T11:31:19.785 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:31:19.785 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.785 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.785 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.785 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:31:19.785 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.785 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.785 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.785 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6802", 2021-02-10T11:31:19.785 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:31:19.785 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.785 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.786 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.786 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6803", 2021-02-10T11:31:19.786 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:31:19.786 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.786 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.786 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.786 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:31:19.786 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.786 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.786 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.786 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6806", 2021-02-10T11:31:19.786 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:31:19.786 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.787 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.787 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.787 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6807", 2021-02-10T11:31:19.787 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:31:19.787 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.787 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.787 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.787 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:31:19.787 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.787 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.787 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.787 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6804", 2021-02-10T11:31:19.787 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:31:19.788 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.788 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.788 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.788 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6805", 2021-02-10T11:31:19.788 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:31:19.788 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.788 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.788 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.788 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6801/36381", 2021-02-10T11:31:19.788 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6803/36381", 2021-02-10T11:31:19.788 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6807/36381", 2021-02-10T11:31:19.788 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6805/36381", 2021-02-10T11:31:19.788 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:31:19.789 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:31:19.789 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:31:19.789 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.789 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.789 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.789 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:31:19.789 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "7b90823f-854c-4592-a95b-a7fa98f93254", 2021-02-10T11:31:19.789 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:31:19.789 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:31:19.789 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:31:19.789 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:31:19.789 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:31:19.789 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:31:19.789 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:31:19.790 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 27, 2021-02-10T11:31:19.790 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:31:19.790 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:31:19.790 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:31:19.790 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.790 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.790 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.790 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6817", 2021-02-10T11:31:19.790 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:31:19.790 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.790 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.790 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.790 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6819", 2021-02-10T11:31:19.791 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:31:19.791 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.791 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.791 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.791 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:31:19.791 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.791 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.791 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.791 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6821", 2021-02-10T11:31:19.791 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:31:19.791 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.791 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.791 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.791 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6823", 2021-02-10T11:31:19.792 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:31:19.792 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.792 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.792 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.792 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:31:19.792 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.792 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.792 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.792 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6829", 2021-02-10T11:31:19.792 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:31:19.792 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.792 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.792 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.793 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6831", 2021-02-10T11:31:19.793 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:31:19.793 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.793 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.793 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.793 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:31:19.793 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:31:19.793 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.793 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:31:19.793 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6825", 2021-02-10T11:31:19.793 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:31:19.793 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.793 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.794 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:19.794 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6827", 2021-02-10T11:31:19.794 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:31:19.794 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.794 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.794 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.794 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6819/36382", 2021-02-10T11:31:19.794 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6823/36382", 2021-02-10T11:31:19.794 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6831/36382", 2021-02-10T11:31:19.794 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6827/36382", 2021-02-10T11:31:19.794 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:31:19.794 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:31:19.794 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:31:19.795 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:19.795 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.795 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:31:19.795 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_xinfo": [ 2021-02-10T11:31:19.795 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.795 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:31:19.795 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:31:19.795 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:31:19.795 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:31:19.795 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:31:19.795 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:31:19.795 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.796 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.796 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:31:19.796 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:31:19.796 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:31:19.796 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:31:19.796 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:31:19.796 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:31:19.796 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.796 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.796 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:31:19.796 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:31:19.796 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:31:19.796 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:31:19.797 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:31:19.797 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:31:19.797 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.797 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.797 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:31:19.797 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:31:19.797 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:31:19.797 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:31:19.797 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:31:19.797 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:31:19.797 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.797 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.797 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:31:19.798 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:31:19.798 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:31:19.798 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:31:19.798 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:31:19.798 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:31:19.798 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.798 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.798 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:31:19.798 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:31:19.798 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:31:19.798 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:31:19.799 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:31:19.799 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:31:19.799 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.799 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.799 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:31:19.799 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:31:19.799 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:31:19.799 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:31:19.799 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:31:19.799 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:31:19.799 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.799 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:19.799 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:31:19.799 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:31:19.800 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:31:19.800 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:31:19.800 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:31:19.800 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:31:19.800 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.800 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:31:19.800 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap": [], 2021-02-10T11:31:19.800 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap_items": [], 2021-02-10T11:31:19.800 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_temp": [], 2021-02-10T11:31:19.800 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_temp": [], 2021-02-10T11:31:19.800 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklist": {}, 2021-02-10T11:31:19.800 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profiles": { 2021-02-10T11:31:19.800 INFO:teuthology.orchestra.run.gibba001.stdout: "default": { 2021-02-10T11:31:19.801 INFO:teuthology.orchestra.run.gibba001.stdout: "crush-failure-domain": "osd", 2021-02-10T11:31:19.801 INFO:teuthology.orchestra.run.gibba001.stdout: "k": "2", 2021-02-10T11:31:19.801 INFO:teuthology.orchestra.run.gibba001.stdout: "m": "1", 2021-02-10T11:31:19.801 INFO:teuthology.orchestra.run.gibba001.stdout: "plugin": "jerasure", 2021-02-10T11:31:19.801 INFO:teuthology.orchestra.run.gibba001.stdout: "ruleset-failure-domain": "osd", 2021-02-10T11:31:19.801 INFO:teuthology.orchestra.run.gibba001.stdout: "technique": "reed_sol_van" 2021-02-10T11:31:19.801 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:19.801 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:19.801 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps_queue": [], 2021-02-10T11:31:19.801 INFO:teuthology.orchestra.run.gibba001.stdout: "new_removed_snaps": [], 2021-02-10T11:31:19.801 INFO:teuthology.orchestra.run.gibba001.stdout: "new_purged_snaps": [], 2021-02-10T11:31:19.801 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_node_flags": {}, 2021-02-10T11:31:19.801 INFO:teuthology.orchestra.run.gibba001.stdout: "device_class_flags": {} 2021-02-10T11:31:19.802 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:31:19.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.716 7f972973f700 1 -- 172.21.2.101:0/2518928022 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97080186c0 msgr2=0x7f970801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:19.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.716 7f972973f700 1 --2- 172.21.2.101:0/2518928022 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97080186c0 0x7f970801abc0 crc :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:19.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.716 7f972973f700 1 -- 172.21.2.101:0/2518928022 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f97241a48f0 msgr2=0x7f97240b85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:19.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.716 7f972973f700 1 --2- 172.21.2.101:0/2518928022 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f97241a48f0 0x7f97240b85b0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7f9714001c10 tx=0x7f971400a8f0).stop 2021-02-10T11:31:19.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.716 7f972973f700 1 -- 172.21.2.101:0/2518928022 shutdown_connections 2021-02-10T11:31:19.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.716 7f972973f700 1 --2- 172.21.2.101:0/2518928022 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97080186c0 0x7f970801abc0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:19.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.716 7f972973f700 1 --2- 172.21.2.101:0/2518928022 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9724107bf0 0x7f9724107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:19.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.716 7f972973f700 1 --2- 172.21.2.101:0/2518928022 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97240b9320 0x7f972409bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:19.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.716 7f972973f700 1 --2- 172.21.2.101:0/2518928022 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f97241a48f0 0x7f97240b85b0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:19.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.716 7f972973f700 1 -- 172.21.2.101:0/2518928022 >> 172.21.2.101:0/2518928022 conn(0x7f972409ca80 msgr2=0x7f97240c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:19.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.716 7f972973f700 1 -- 172.21.2.101:0/2518928022 shutdown_connections 2021-02-10T11:31:19.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:19.716 7f972973f700 1 -- 172.21.2.101:0/2518928022 wait complete. 2021-02-10T11:31:19.805 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:31:20.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.203 7fc598ac7700 1 Processor -- start 2021-02-10T11:31:20.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.203 7fc598ac7700 1 -- start start 2021-02-10T11:31:20.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.204 7fc598ac7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc5940e5070 0x7fc5941b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:20.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.204 7fc598ac7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5940e5470 0x7fc594080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:20.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.204 7fc598ac7700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc59407ed80 con 0x7fc5941b3e40 2021-02-10T11:31:20.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.204 7fc598ac7700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5940d1790 con 0x7fc5940e5070 2021-02-10T11:31:20.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.204 7fc598ac7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5940d19b0 con 0x7fc5940e5470 2021-02-10T11:31:20.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.204 7fc59285e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc5940e5070 0x7fc5941b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:20.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.204 7fc59285e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc5940e5070 0x7fc5941b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49920/0 (socket says 172.21.2.101:49920) 2021-02-10T11:31:20.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.204 7fc59285e700 1 -- 172.21.2.101:0/1429887344 learned_addr learned my addr 172.21.2.101:0/1429887344 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:20.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.204 7fc59305f700 1 --2- 172.21.2.101:0/1429887344 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5940e5470 0x7fc594080310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:20.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.204 7fc59285e700 1 -- 172.21.2.101:0/1429887344 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5940e5470 msgr2=0x7fc594080310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:20.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.204 7fc59285e700 1 --2- 172.21.2.101:0/1429887344 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5940e5470 0x7fc594080310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:20.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.204 7fc59285e700 1 -- 172.21.2.101:0/1429887344 >> v1:172.21.2.101:6789/0 conn(0x7fc5941b3e40 legacy=0x7fc59407d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:20.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.204 7fc59285e700 1 -- 172.21.2.101:0/1429887344 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5940d1bd0 con 0x7fc5940e5070 2021-02-10T11:31:20.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.205 7fc59285e700 1 --2- 172.21.2.101:0/1429887344 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5940e5070 0x7fc5941b3820 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7fc584006700 tx=0x7fc584009b00).ready entity=mon.0 client_cookie=f0e890efa5683cfa server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:20.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.205 7fc59185c700 1 -- 172.21.2.101:0/1429887344 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc58400c0c0 con 0x7fc5940e5070 2021-02-10T11:31:20.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.205 7fc59185c700 1 -- 172.21.2.101:0/1429887344 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58400c400 con 0x7fc5940e5070 2021-02-10T11:31:20.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.205 7fc59185c700 1 -- 172.21.2.101:0/1429887344 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc58400c8e0 con 0x7fc5940e5070 2021-02-10T11:31:20.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.205 7fc598ac7700 1 -- 172.21.2.101:0/1429887344 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5940e5070 msgr2=0x7fc5941b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:20.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.205 7fc598ac7700 1 --2- 172.21.2.101:0/1429887344 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5940e5070 0x7fc5941b3820 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7fc584006700 tx=0x7fc584009b00).stop 2021-02-10T11:31:20.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.205 7fc598ac7700 1 -- 172.21.2.101:0/1429887344 shutdown_connections 2021-02-10T11:31:20.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.205 7fc598ac7700 1 --2- 172.21.2.101:0/1429887344 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5940e5470 0x7fc594080310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:20.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.205 7fc598ac7700 1 --2- 172.21.2.101:0/1429887344 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5940e5070 0x7fc5941b3820 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:20.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.205 7fc598ac7700 1 -- 172.21.2.101:0/1429887344 >> 172.21.2.101:0/1429887344 conn(0x7fc5940ce690 msgr2=0x7fc5940e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:20.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.206 7fc598ac7700 1 -- 172.21.2.101:0/1429887344 shutdown_connections 2021-02-10T11:31:20.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.206 7fc598ac7700 1 -- 172.21.2.101:0/1429887344 wait complete. 2021-02-10T11:31:20.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.220 7fc598ac7700 1 Processor -- start 2021-02-10T11:31:20.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.220 7fc598ac7700 1 -- start start 2021-02-10T11:31:20.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.221 7fc598ac7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc594191bb0 0x7fc5940a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:20.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.221 7fc598ac7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5940a29a0 0x7fc5940a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:20.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.221 7fc598ac7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc5940a33c0 0x7fc5940a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:20.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.221 7fc598ac7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5940adf90 con 0x7fc594191bb0 2021-02-10T11:31:20.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.221 7fc598ac7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5940a1e00 con 0x7fc5940a33c0 2021-02-10T11:31:20.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.221 7fc598ac7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5940a09f0 con 0x7fc5940a29a0 2021-02-10T11:31:20.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.221 7fc59285e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc594191bb0 0x7fc5940a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:20.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.221 7fc59205d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5940a29a0 0x7fc5940a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:20.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.221 7fc59285e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc594191bb0 0x7fc5940a2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49930/0 (socket says 172.21.2.101:49930) 2021-02-10T11:31:20.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.221 7fc59285e700 1 -- 172.21.2.101:0/2323567695 learned_addr learned my addr 172.21.2.101:0/2323567695 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:20.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.221 7fc59305f700 1 --2- 172.21.2.101:0/2323567695 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc5940a33c0 0x7fc5940a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:20.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.221 7fc59285e700 1 -- 172.21.2.101:0/2323567695 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5940a29a0 msgr2=0x7fc5940a2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:20.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.221 7fc59285e700 1 --2- 172.21.2.101:0/2323567695 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5940a29a0 0x7fc5940a2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:20.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.221 7fc59285e700 1 -- 172.21.2.101:0/2323567695 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc5940a33c0 msgr2=0x7fc5940a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:20.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.221 7fc59285e700 1 --2- 172.21.2.101:0/2323567695 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc5940a33c0 0x7fc5940a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:20.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.221 7fc59285e700 1 -- 172.21.2.101:0/2323567695 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5940a0bd0 con 0x7fc594191bb0 2021-02-10T11:31:20.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.221 7fc59285e700 1 --2- 172.21.2.101:0/2323567695 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc594191bb0 0x7fc5940a2380 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7fc58400b660 tx=0x7fc58400ba00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:20.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.221 7fc58b7fe700 1 -- 172.21.2.101:0/2323567695 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc584009440 con 0x7fc594191bb0 2021-02-10T11:31:20.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.222 7fc58b7fe700 1 -- 172.21.2.101:0/2323567695 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc584006390 con 0x7fc594191bb0 2021-02-10T11:31:20.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.222 7fc598ac7700 1 -- 172.21.2.101:0/2323567695 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5940a0bd0 con 0x7fc594191bb0 2021-02-10T11:31:20.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.222 7fc58b7fe700 1 -- 172.21.2.101:0/2323567695 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc584002030 con 0x7fc594191bb0 2021-02-10T11:31:20.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.222 7fc598ac7700 1 -- 172.21.2.101:0/2323567695 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5940a0bd0 con 0x7fc594191bb0 2021-02-10T11:31:20.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.222 7fc58b7fe700 1 -- 172.21.2.101:0/2323567695 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc5840093e0 con 0x7fc594191bb0 2021-02-10T11:31:20.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.222 7fc58b7fe700 1 --2- 172.21.2.101:0/2323567695 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5780186d0 0x7fc57801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:20.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.222 7fc58b7fe700 1 -- 172.21.2.101:0/2323567695 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc584032620 con 0x7fc594191bb0 2021-02-10T11:31:20.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.222 7fc59205d700 1 --2- 172.21.2.101:0/2323567695 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5780186d0 0x7fc57801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:20.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.223 7fc59205d700 1 --2- 172.21.2.101:0/2323567695 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5780186d0 0x7fc57801abd0 crc :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:20.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.254 7fc598ac7700 1 -- 172.21.2.101:0/2323567695 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5940a1e00 con 0x7fc594191bb0 2021-02-10T11:31:20.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.256 7fc58b7fe700 1 -- 172.21.2.101:0/2323567695 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc5840090c0 con 0x7fc594191bb0 2021-02-10T11:31:20.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.386 7fc598ac7700 1 -- 172.21.2.101:0/2323567695 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5940a1e00 con 0x7fc594191bb0 2021-02-10T11:31:20.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.386 7fc58b7fe700 1 -- 172.21.2.101:0/2323567695 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+2098 (secure 0 0 0) 0x7fc58400bdf0 con 0x7fc594191bb0 2021-02-10T11:31:20.397 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:31:20.398 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/3226463341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3226463341},{"type":"v1","addr":"172.21.2.101:6835","nonce":3226463341}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:14:57.130312","modified":"2021-02-10 11:14:59.506936","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4373":{"gid":4373,"name":"b","rank":0,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.2.108:6833/3230549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3230549207},{"type":"v1","addr":"172.21.2.108:6833","nonce":3230549207}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-02-10T11:31:20.398 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 14 2021-02-10T11:31:20.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.388 7fc598ac7700 1 -- 172.21.2.101:0/2323567695 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5780186d0 msgr2=0x7fc57801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:20.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.388 7fc598ac7700 1 --2- 172.21.2.101:0/2323567695 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5780186d0 0x7fc57801abd0 crc :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:20.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.388 7fc598ac7700 1 -- 172.21.2.101:0/2323567695 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc594191bb0 msgr2=0x7fc5940a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:20.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.388 7fc598ac7700 1 --2- 172.21.2.101:0/2323567695 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc594191bb0 0x7fc5940a2380 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7fc58400b660 tx=0x7fc58400ba00).stop 2021-02-10T11:31:20.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.388 7fc598ac7700 1 -- 172.21.2.101:0/2323567695 shutdown_connections 2021-02-10T11:31:20.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.388 7fc598ac7700 1 --2- 172.21.2.101:0/2323567695 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5780186d0 0x7fc57801abd0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:20.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.388 7fc598ac7700 1 --2- 172.21.2.101:0/2323567695 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5940a29a0 0x7fc5940a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:20.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.388 7fc598ac7700 1 --2- 172.21.2.101:0/2323567695 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc5940a33c0 0x7fc5940a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:20.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.388 7fc598ac7700 1 --2- 172.21.2.101:0/2323567695 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc594191bb0 0x7fc5940a2380 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:20.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.388 7fc598ac7700 1 -- 172.21.2.101:0/2323567695 >> 172.21.2.101:0/2323567695 conn(0x7fc5941a31d0 msgr2=0x7fc59408b830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:20.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.388 7fc598ac7700 1 -- 172.21.2.101:0/2323567695 shutdown_connections 2021-02-10T11:31:20.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.388 7fc598ac7700 1 -- 172.21.2.101:0/2323567695 wait complete. 2021-02-10T11:31:20.407 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs fail cephfs 2021-02-10T11:31:20.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.806 7f5a64f30700 1 Processor -- start 2021-02-10T11:31:20.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.806 7f5a64f30700 1 -- start start 2021-02-10T11:31:20.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.807 7f5a64f30700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f5a600e5070 0x7f5a601b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:20.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.807 7f5a64f30700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a601b3e40 0x7f5a6007d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:20.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.807 7f5a64f30700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a6007ed80 con 0x7f5a600e5470 2021-02-10T11:31:20.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.807 7f5a64f30700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a60080310 con 0x7f5a600e5070 2021-02-10T11:31:20.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.807 7f5a64f30700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a60080500 con 0x7f5a601b3e40 2021-02-10T11:31:20.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.807 7f5a5e312700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a601b3e40 0x7f5a6007d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:20.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.807 7f5a5e312700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a601b3e40 0x7f5a6007d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:42668/0 (socket says 172.21.2.101:42668) 2021-02-10T11:31:20.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.807 7f5a5e312700 1 -- 172.21.2.101:0/2152448770 learned_addr learned my addr 172.21.2.101:0/2152448770 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:20.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.807 7f5a5eb13700 1 --2- 172.21.2.101:0/2152448770 >> v2:172.21.2.108:3300/0 conn(0x7f5a600e5070 0x7f5a601b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:20.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.808 7f5a5e312700 1 -- 172.21.2.101:0/2152448770 >> v2:172.21.2.108:3300/0 conn(0x7f5a600e5070 msgr2=0x7f5a601b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:20.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.808 7f5a5e312700 1 --2- 172.21.2.101:0/2152448770 >> v2:172.21.2.108:3300/0 conn(0x7f5a600e5070 0x7f5a601b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:20.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.808 7f5a5e312700 1 -- 172.21.2.101:0/2152448770 >> v1:172.21.2.108:6789/0 conn(0x7f5a600e5470 legacy=0x7f5a6007dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:20.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.808 7f5a5e312700 1 -- 172.21.2.101:0/2152448770 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a600806f0 con 0x7f5a601b3e40 2021-02-10T11:31:20.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.808 7f5a5e312700 1 --2- 172.21.2.101:0/2152448770 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a601b3e40 0x7f5a6007d690 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f5a48006700 tx=0x7f5a48009b80).ready entity=mon.2 client_cookie=163951ce416ea9a4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:20.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.808 7f5a5db11700 1 -- 172.21.2.101:0/2152448770 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5a4800c0c0 con 0x7f5a601b3e40 2021-02-10T11:31:20.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.808 7f5a5db11700 1 -- 172.21.2.101:0/2152448770 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4800c340 con 0x7f5a601b3e40 2021-02-10T11:31:20.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.808 7f5a5db11700 1 -- 172.21.2.101:0/2152448770 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5a480009b0 con 0x7f5a601b3e40 2021-02-10T11:31:20.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.808 7f5a64f30700 1 -- 172.21.2.101:0/2152448770 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a601b3e40 msgr2=0x7f5a6007d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:20.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.808 7f5a64f30700 1 --2- 172.21.2.101:0/2152448770 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a601b3e40 0x7f5a6007d690 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f5a48006700 tx=0x7f5a48009b80).stop 2021-02-10T11:31:20.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.809 7f5a64f30700 1 -- 172.21.2.101:0/2152448770 shutdown_connections 2021-02-10T11:31:20.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.809 7f5a64f30700 1 --2- 172.21.2.101:0/2152448770 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a601b3e40 0x7f5a6007d690 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:20.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.809 7f5a64f30700 1 --2- 172.21.2.101:0/2152448770 >> v2:172.21.2.108:3300/0 conn(0x7f5a600e5070 0x7f5a601b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:20.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.809 7f5a64f30700 1 -- 172.21.2.101:0/2152448770 >> 172.21.2.101:0/2152448770 conn(0x7f5a600ce690 msgr2=0x7f5a600e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:20.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.809 7f5a64f30700 1 -- 172.21.2.101:0/2152448770 shutdown_connections 2021-02-10T11:31:20.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.809 7f5a64f30700 1 -- 172.21.2.101:0/2152448770 wait complete. 2021-02-10T11:31:20.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.824 7f5a64f30700 1 Processor -- start 2021-02-10T11:31:20.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.824 7f5a64f30700 1 -- start start 2021-02-10T11:31:20.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.824 7f5a64f30700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a6019d5c0 0x7f5a60088fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:20.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.824 7f5a64f30700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5a600895f0 0x7f5a600b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:20.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.824 7f5a64f30700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a600b0700 0x7f5a600b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:20.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.824 7f5a64f30700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a600b0d30 con 0x7f5a6019d5c0 2021-02-10T11:31:20.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.824 7f5a64f30700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a600a6d50 con 0x7f5a600895f0 2021-02-10T11:31:20.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.824 7f5a64f30700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a600875a0 con 0x7f5a600b0700 2021-02-10T11:31:20.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.824 7f5a5eb13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a6019d5c0 0x7f5a60088fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:20.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.824 7f5a5eb13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a6019d5c0 0x7f5a60088fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49940/0 (socket says 172.21.2.101:49940) 2021-02-10T11:31:20.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.824 7f5a5eb13700 1 -- 172.21.2.101:0/209029624 learned_addr learned my addr 172.21.2.101:0/209029624 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:20.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.825 7f5a5eb13700 1 -- 172.21.2.101:0/209029624 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a600b0700 msgr2=0x7f5a600b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:20.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.825 7f5a5eb13700 1 --2- 172.21.2.101:0/209029624 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a600b0700 0x7f5a600b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:20.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.825 7f5a5eb13700 1 -- 172.21.2.101:0/209029624 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5a600895f0 msgr2=0x7f5a600b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:20.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.825 7f5a5eb13700 1 --2- 172.21.2.101:0/209029624 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5a600895f0 0x7f5a600b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:20.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.825 7f5a5eb13700 1 -- 172.21.2.101:0/209029624 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a600b18d0 con 0x7f5a6019d5c0 2021-02-10T11:31:20.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.825 7f5a5eb13700 1 --2- 172.21.2.101:0/209029624 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a6019d5c0 0x7f5a60088fd0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0x7f5a50006f60 tx=0x7f5a5000b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:20.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.825 7f5a577fe700 1 -- 172.21.2.101:0/209029624 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5a500009b0 con 0x7f5a6019d5c0 2021-02-10T11:31:20.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.825 7f5a577fe700 1 -- 172.21.2.101:0/209029624 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5000bd60 con 0x7f5a6019d5c0 2021-02-10T11:31:20.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.825 7f5a64f30700 1 -- 172.21.2.101:0/209029624 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a600b18d0 con 0x7f5a6019d5c0 2021-02-10T11:31:20.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.825 7f5a64f30700 1 -- 172.21.2.101:0/209029624 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a600b1e60 con 0x7f5a6019d5c0 2021-02-10T11:31:20.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.825 7f5a577fe700 1 -- 172.21.2.101:0/209029624 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5a50002220 con 0x7f5a6019d5c0 2021-02-10T11:31:20.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.825 7f5a577fe700 1 -- 172.21.2.101:0/209029624 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5a50006b00 con 0x7f5a6019d5c0 2021-02-10T11:31:20.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.826 7f5a577fe700 1 --2- 172.21.2.101:0/209029624 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5a44018730 0x7f5a4401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:20.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.826 7f5a577fe700 1 -- 172.21.2.101:0/209029624 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5a50032790 con 0x7f5a6019d5c0 2021-02-10T11:31:20.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.826 7f5a5e312700 1 --2- 172.21.2.101:0/209029624 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5a44018730 0x7f5a4401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:20.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.826 7f5a5e312700 1 --2- 172.21.2.101:0/209029624 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5a44018730 0x7f5a4401ac10 crc :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:20.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.857 7f5a64f30700 1 -- 172.21.2.101:0/209029624 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a60107180 con 0x7f5a6019d5c0 2021-02-10T11:31:20.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.860 7f5a577fe700 1 -- 172.21.2.101:0/209029624 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5a50000940 con 0x7f5a6019d5c0 2021-02-10T11:31:20.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:20.987 7f5a64f30700 1 -- 172.21.2.101:0/209029624 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5a6007ed80 con 0x7f5a6019d5c0 2021-02-10T11:31:21.784 INFO:tasks.ceph.mon.a.gibba001.stderr:2021-02-10 11:31:21.773 7f974d04d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-02-10T11:31:21.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:21.780 7f5a577fe700 1 -- 172.21.2.101:0/209029624 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v15) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5a50006b00 con 0x7f5a6019d5c0 2021-02-10T11:31:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-02-10T11:31:21.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:21.808 7f5a64f30700 1 -- 172.21.2.101:0/209029624 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5a44018730 msgr2=0x7f5a4401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:21.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:21.808 7f5a64f30700 1 --2- 172.21.2.101:0/209029624 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5a44018730 0x7f5a4401ac10 crc :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:21.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:21.808 7f5a64f30700 1 -- 172.21.2.101:0/209029624 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a6019d5c0 msgr2=0x7f5a60088fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:21.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:21.808 7f5a64f30700 1 --2- 172.21.2.101:0/209029624 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a6019d5c0 0x7f5a60088fd0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0x7f5a50006f60 tx=0x7f5a5000b970).stop 2021-02-10T11:31:21.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:21.808 7f5a64f30700 1 -- 172.21.2.101:0/209029624 shutdown_connections 2021-02-10T11:31:21.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:21.808 7f5a64f30700 1 --2- 172.21.2.101:0/209029624 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5a44018730 0x7f5a4401ac10 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:21.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:21.808 7f5a64f30700 1 --2- 172.21.2.101:0/209029624 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a600b0700 0x7f5a600b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:21.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:21.808 7f5a64f30700 1 --2- 172.21.2.101:0/209029624 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5a600895f0 0x7f5a600b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:21.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:21.808 7f5a64f30700 1 --2- 172.21.2.101:0/209029624 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a6019d5c0 0x7f5a60088fd0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:21.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:21.808 7f5a64f30700 1 -- 172.21.2.101:0/209029624 >> 172.21.2.101:0/209029624 conn(0x7f5a600c0d40 msgr2=0x7f5a601073b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:21.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:21.808 7f5a64f30700 1 -- 172.21.2.101:0/209029624 shutdown_connections 2021-02-10T11:31:21.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:21.808 7f5a64f30700 1 -- 172.21.2.101:0/209029624 wait complete. 2021-02-10T11:31:21.828 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:31:21.842 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:31:22.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.226 7f99ada16700 1 Processor -- start 2021-02-10T11:31:22.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.226 7f99ada16700 1 -- start start 2021-02-10T11:31:22.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.227 7f99ada16700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f99a80e5470 0x7f99a807e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:22.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.227 7f99ada16700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f99a807f300 0x7f99a807f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:22.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.227 7f99ada16700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a807ed80 con 0x7f99a80e5070 2021-02-10T11:31:22.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.227 7f99ada16700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a81b4010 con 0x7f99a807f300 2021-02-10T11:31:22.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.227 7f99ada16700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a80d1790 con 0x7f99a80e5470 2021-02-10T11:31:22.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.227 7f99a7fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f99a807f300 0x7f99a807f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:22.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.227 7f99a7fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f99a807f300 0x7f99a807f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49952/0 (socket says 172.21.2.101:49952) 2021-02-10T11:31:22.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.227 7f99a7fff700 1 -- 172.21.2.101:0/879383613 learned_addr learned my addr 172.21.2.101:0/879383613 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:22.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.227 7f99a6ffd700 1 --2- 172.21.2.101:0/879383613 >> v2:172.21.2.108:3300/0 conn(0x7f99a80e5470 0x7f99a807e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:22.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.227 7f99a7fff700 1 -- 172.21.2.101:0/879383613 >> v2:172.21.2.108:3300/0 conn(0x7f99a80e5470 msgr2=0x7f99a807e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:22.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.227 7f99a7fff700 1 --2- 172.21.2.101:0/879383613 >> v2:172.21.2.108:3300/0 conn(0x7f99a80e5470 0x7f99a807e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:22.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.227 7f99a7fff700 1 -- 172.21.2.101:0/879383613 >> v1:172.21.2.108:6789/0 conn(0x7f99a80e5070 legacy=0x7f99a81b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:22.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.227 7f99a7fff700 1 -- 172.21.2.101:0/879383613 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a80d19b0 con 0x7f99a807f300 2021-02-10T11:31:22.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.227 7f99a7fff700 1 --2- 172.21.2.101:0/879383613 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99a807f300 0x7f99a807f700 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f99a0006f00 tx=0x7f99a0009f10).ready entity=mon.0 client_cookie=ae1f272fc6116206 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:22.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.228 7f99a67fc700 1 -- 172.21.2.101:0/879383613 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f99a000b3b0 con 0x7f99a807f300 2021-02-10T11:31:22.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.228 7f99a67fc700 1 -- 172.21.2.101:0/879383613 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a000b730 con 0x7f99a807f300 2021-02-10T11:31:22.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.228 7f99a67fc700 1 -- 172.21.2.101:0/879383613 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f99a000bc50 con 0x7f99a807f300 2021-02-10T11:31:22.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.228 7f99ada16700 1 -- 172.21.2.101:0/879383613 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99a807f300 msgr2=0x7f99a807f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:22.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.228 7f99ada16700 1 --2- 172.21.2.101:0/879383613 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99a807f300 0x7f99a807f700 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f99a0006f00 tx=0x7f99a0009f10).stop 2021-02-10T11:31:22.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.228 7f99ada16700 1 -- 172.21.2.101:0/879383613 shutdown_connections 2021-02-10T11:31:22.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.228 7f99ada16700 1 --2- 172.21.2.101:0/879383613 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99a807f300 0x7f99a807f700 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:22.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.228 7f99ada16700 1 --2- 172.21.2.101:0/879383613 >> v2:172.21.2.108:3300/0 conn(0x7f99a80e5470 0x7f99a807e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:22.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.228 7f99ada16700 1 -- 172.21.2.101:0/879383613 >> 172.21.2.101:0/879383613 conn(0x7f99a80ce690 msgr2=0x7f99a80e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:22.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.228 7f99ada16700 1 -- 172.21.2.101:0/879383613 shutdown_connections 2021-02-10T11:31:22.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.228 7f99ada16700 1 -- 172.21.2.101:0/879383613 wait complete. 2021-02-10T11:31:22.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.243 7f99ada16700 1 Processor -- start 2021-02-10T11:31:22.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.243 7f99ada16700 1 -- start start 2021-02-10T11:31:22.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.243 7f99ada16700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99a808d400 0x7f99a8081230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:22.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.243 7f99ada16700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99a8081850 0x7f99a80acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:22.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.243 7f99ada16700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99a8081c50 0x7f99a80ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:22.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.243 7f99ada16700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a8082110 con 0x7f99a8081850 2021-02-10T11:31:22.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.243 7f99ada16700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a808d800 con 0x7f99a8081c50 2021-02-10T11:31:22.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.243 7f99ada16700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a80adbc0 con 0x7f99a808d400 2021-02-10T11:31:22.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.243 7f99a6ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99a8081850 0x7f99a80acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:22.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.243 7f99a6ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99a8081850 0x7f99a80acf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49956/0 (socket says 172.21.2.101:49956) 2021-02-10T11:31:22.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.243 7f99a6ffd700 1 -- 172.21.2.101:0/4127976545 learned_addr learned my addr 172.21.2.101:0/4127976545 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:22.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.244 7f99a6ffd700 1 -- 172.21.2.101:0/4127976545 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99a808d400 msgr2=0x7f99a8081230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:22.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.244 7f99a6ffd700 1 --2- 172.21.2.101:0/4127976545 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99a808d400 0x7f99a8081230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:22.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.244 7f99a6ffd700 1 -- 172.21.2.101:0/4127976545 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99a8081c50 msgr2=0x7f99a80ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:22.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.244 7f99a6ffd700 1 --2- 172.21.2.101:0/4127976545 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99a8081c50 0x7f99a80ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:22.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.244 7f99a6ffd700 1 -- 172.21.2.101:0/4127976545 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a8187fc0 con 0x7f99a8081850 2021-02-10T11:31:22.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.244 7f99a6ffd700 1 --2- 172.21.2.101:0/4127976545 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99a8081850 0x7f99a80acf80 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7f9998006f00 tx=0x7f999800b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:22.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.244 7f999ffff700 1 -- 172.21.2.101:0/4127976545 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f999800bd30 con 0x7f99a8081850 2021-02-10T11:31:22.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.244 7f999ffff700 1 -- 172.21.2.101:0/4127976545 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9998002030 con 0x7f99a8081850 2021-02-10T11:31:22.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.244 7f999ffff700 1 -- 172.21.2.101:0/4127976545 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9998002510 con 0x7f99a8081850 2021-02-10T11:31:22.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.244 7f99ada16700 1 -- 172.21.2.101:0/4127976545 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a8187fc0 con 0x7f99a8081850 2021-02-10T11:31:22.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.244 7f99ada16700 1 -- 172.21.2.101:0/4127976545 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a8188550 con 0x7f99a8081850 2021-02-10T11:31:22.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.245 7f999ffff700 1 -- 172.21.2.101:0/4127976545 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9998000c90 con 0x7f99a8081850 2021-02-10T11:31:22.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.245 7f999ffff700 1 --2- 172.21.2.101:0/4127976545 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f998c0186c0 0x7f998c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:22.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.245 7f999ffff700 1 -- 172.21.2.101:0/4127976545 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f9998032610 con 0x7f99a8081850 2021-02-10T11:31:22.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.245 7f99a77fe700 1 --2- 172.21.2.101:0/4127976545 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f998c0186c0 0x7f998c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:22.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.245 7f99a77fe700 1 --2- 172.21.2.101:0/4127976545 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f998c0186c0 0x7f998c01abc0 crc :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:22.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.277 7f99ada16700 1 -- 172.21.2.101:0/4127976545 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a808d800 con 0x7f99a8081850 2021-02-10T11:31:22.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.279 7f999ffff700 1 -- 172.21.2.101:0/4127976545 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9998030d70 con 0x7f99a8081850 2021-02-10T11:31:22.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.407 7f99ada16700 1 -- 172.21.2.101:0/4127976545 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99a808d800 con 0x7f99a8081850 2021-02-10T11:31:22.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.408 7f999ffff700 1 -- 172.21.2.101:0/4127976545 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+1753 (secure 0 0 0) 0x7f9998000c90 con 0x7f99a8081850 2021-02-10T11:31:22.419 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:31:22.419 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/3226463341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3226463341},{"type":"v1","addr":"172.21.2.101:6835","nonce":3226463341}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":15,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:14:57.130312","modified":"2021-02-10 11:31:21.774320","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-02-10T11:31:22.420 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 15 2021-02-10T11:31:22.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.410 7f99ada16700 1 -- 172.21.2.101:0/4127976545 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f998c0186c0 msgr2=0x7f998c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:22.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.410 7f99ada16700 1 --2- 172.21.2.101:0/4127976545 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f998c0186c0 0x7f998c01abc0 crc :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:22.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.410 7f99ada16700 1 -- 172.21.2.101:0/4127976545 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99a8081850 msgr2=0x7f99a80acf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:22.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.410 7f99ada16700 1 --2- 172.21.2.101:0/4127976545 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99a8081850 0x7f99a80acf80 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7f9998006f00 tx=0x7f999800b940).stop 2021-02-10T11:31:22.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.410 7f99ada16700 1 -- 172.21.2.101:0/4127976545 shutdown_connections 2021-02-10T11:31:22.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.410 7f99ada16700 1 --2- 172.21.2.101:0/4127976545 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99a8081c50 0x7f99a80ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:22.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.410 7f99ada16700 1 --2- 172.21.2.101:0/4127976545 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99a8081850 0x7f99a80acf80 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:22.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.410 7f99ada16700 1 --2- 172.21.2.101:0/4127976545 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f998c0186c0 0x7f998c01abc0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:22.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.410 7f99ada16700 1 --2- 172.21.2.101:0/4127976545 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99a808d400 0x7f99a8081230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:22.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.410 7f99ada16700 1 -- 172.21.2.101:0/4127976545 >> 172.21.2.101:0/4127976545 conn(0x7f99a81ac620 msgr2=0x7f99a80896e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:22.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.410 7f99ada16700 1 -- 172.21.2.101:0/4127976545 shutdown_connections 2021-02-10T11:31:22.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.410 7f99ada16700 1 -- 172.21.2.101:0/4127976545 wait complete. 2021-02-10T11:31:22.429 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-02-10T11:31:22.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.832 7efcc742f700 1 Processor -- start 2021-02-10T11:31:22.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.832 7efcc742f700 1 -- start start 2021-02-10T11:31:22.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.833 7efcc742f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcc00d17f0 0x7efcc01b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:22.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.833 7efcc742f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcc00f0b20 con 0x7efcc00cff00 2021-02-10T11:31:22.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.833 7efcc742f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcc01b3ff0 con 0x7efcc00d1bf0 2021-02-10T11:31:22.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.833 7efcc742f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc00e5eb0 con 0x7efcc00d17f0 2021-02-10T11:31:22.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.833 7efcc5a4c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7efcc00cff00 0x7efcc00ef620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50456/0 (socket says 172.21.2.101:50456) 2021-02-10T11:31:22.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.833 7efcc5a4c700 1 -- 172.21.2.101:0/1234116359 learned_addr learned my addr 172.21.2.101:0/1234116359 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:22.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.833 7efcc524b700 1 --2- 172.21.2.101:0/1234116359 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcc00d17f0 0x7efcc01b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:22.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.833 7efcb3fff700 1 -- 172.21.2.101:0/1234116359 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1155697912 0 0) 0x7efcb8001500 con 0x7efcc00cff00 2021-02-10T11:31:22.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.833 7efcc524b700 1 -- 172.21.2.101:0/1234116359 >> v1:172.21.2.108:6789/0 conn(0x7efcc00d1bf0 legacy=0x7efcc01b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:22.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.833 7efcc524b700 1 -- 172.21.2.101:0/1234116359 >> v1:172.21.2.101:6789/0 conn(0x7efcc00cff00 legacy=0x7efcc00ef620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:22.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.833 7efcc524b700 1 -- 172.21.2.101:0/1234116359 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc00f20b0 con 0x7efcc00d17f0 2021-02-10T11:31:22.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.834 7efcc524b700 1 --2- 172.21.2.101:0/1234116359 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcc00d17f0 0x7efcc01b31e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7efcb4006650 tx=0x7efcb4009a80).ready entity=mon.2 client_cookie=d57b9c9ef6e40e6e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:22.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.834 7efcb3fff700 1 -- 172.21.2.101:0/1234116359 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efcb40009b0 con 0x7efcc00d17f0 2021-02-10T11:31:22.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.834 7efcb3fff700 1 -- 172.21.2.101:0/1234116359 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb400c0c0 con 0x7efcc00d17f0 2021-02-10T11:31:22.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.834 7efcb3fff700 1 -- 172.21.2.101:0/1234116359 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efcb400c510 con 0x7efcc00d17f0 2021-02-10T11:31:22.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.834 7efcc742f700 1 -- 172.21.2.101:0/1234116359 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcc00d17f0 msgr2=0x7efcc01b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:22.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.834 7efcc742f700 1 --2- 172.21.2.101:0/1234116359 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcc00d17f0 0x7efcc01b31e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7efcb4006650 tx=0x7efcb4009a80).stop 2021-02-10T11:31:22.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.834 7efcc742f700 1 -- 172.21.2.101:0/1234116359 shutdown_connections 2021-02-10T11:31:22.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.834 7efcc742f700 1 --2- 172.21.2.101:0/1234116359 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcc00d17f0 0x7efcc01b31e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:22.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.834 7efcc742f700 1 -- 172.21.2.101:0/1234116359 >> 172.21.2.101:0/1234116359 conn(0x7efcc00fde40 msgr2=0x7efcc00ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:22.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.835 7efcc742f700 1 -- 172.21.2.101:0/1234116359 shutdown_connections 2021-02-10T11:31:22.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.835 7efcc742f700 1 -- 172.21.2.101:0/1234116359 wait complete. 2021-02-10T11:31:22.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.849 7efcc742f700 1 Processor -- start 2021-02-10T11:31:22.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.849 7efcc742f700 1 -- start start 2021-02-10T11:31:22.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.849 7efcc742f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcc008dea0 0x7efcc009c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:22.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.849 7efcc742f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efcc00b4330 0x7efcc00b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:22.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.849 7efcc742f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcc01ad3f0 0x7efcc01ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:22.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.849 7efcc742f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc009cc10 con 0x7efcc008dea0 2021-02-10T11:31:22.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.849 7efcc742f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc008e2a0 con 0x7efcc00b4330 2021-02-10T11:31:22.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.849 7efcc742f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc01ade10 con 0x7efcc01ad3f0 2021-02-10T11:31:22.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.850 7efcc524b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcc008dea0 0x7efcc009c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:22.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.850 7efcc524b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcc008dea0 0x7efcc009c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49966/0 (socket says 172.21.2.101:49966) 2021-02-10T11:31:22.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.850 7efcc524b700 1 -- 172.21.2.101:0/3820629045 learned_addr learned my addr 172.21.2.101:0/3820629045 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:22.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.850 7efcc524b700 1 -- 172.21.2.101:0/3820629045 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcc01ad3f0 msgr2=0x7efcc01ad7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:31:22.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.850 7efcc524b700 1 --2- 172.21.2.101:0/3820629045 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcc01ad3f0 0x7efcc01ad7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:22.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.850 7efcc524b700 1 -- 172.21.2.101:0/3820629045 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efcc00b4330 msgr2=0x7efcc00b4730 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:31:22.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.850 7efcc524b700 1 --2- 172.21.2.101:0/3820629045 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efcc00b4330 0x7efcc00b4730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:22.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.850 7efcc524b700 1 -- 172.21.2.101:0/3820629045 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc01ae030 con 0x7efcc008dea0 2021-02-10T11:31:22.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.850 7efcc524b700 1 --2- 172.21.2.101:0/3820629045 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcc008dea0 0x7efcc009c5f0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7efcb4009630 tx=0x7efcb400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:22.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.850 7efcb1ffb700 1 -- 172.21.2.101:0/3820629045 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efcb40093a0 con 0x7efcc008dea0 2021-02-10T11:31:22.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.850 7efcb1ffb700 1 -- 172.21.2.101:0/3820629045 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb400bd70 con 0x7efcc008dea0 2021-02-10T11:31:22.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.850 7efcb1ffb700 1 -- 172.21.2.101:0/3820629045 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efcb4005560 con 0x7efcc008dea0 2021-02-10T11:31:22.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.850 7efcc742f700 1 -- 172.21.2.101:0/3820629045 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc01ae030 con 0x7efcc008dea0 2021-02-10T11:31:22.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.850 7efcc742f700 1 -- 172.21.2.101:0/3820629045 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc008a6f0 con 0x7efcc008dea0 2021-02-10T11:31:22.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.851 7efcb1ffb700 1 -- 172.21.2.101:0/3820629045 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7efcb400a000 con 0x7efcc008dea0 2021-02-10T11:31:22.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.851 7efcb1ffb700 1 --2- 172.21.2.101:0/3820629045 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efca0018700 0x7efca001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:22.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.851 7efcb1ffb700 1 -- 172.21.2.101:0/3820629045 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5659+0+0 (secure 0 0 0) 0x7efcb40326d0 con 0x7efcc008dea0 2021-02-10T11:31:22.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.851 7efcc4a4a700 1 --2- 172.21.2.101:0/3820629045 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efca0018700 0x7efca001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:22.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.851 7efcc4a4a700 1 --2- 172.21.2.101:0/3820629045 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efca0018700 0x7efca001abc0 crc :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:22.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.883 7efcc742f700 1 -- 172.21.2.101:0/3820629045 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc0103760 con 0x7efcc008dea0 2021-02-10T11:31:22.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:22.885 7efcb1ffb700 1 -- 172.21.2.101:0/3820629045 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7efcb4009340 con 0x7efcc008dea0 2021-02-10T11:31:23.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:23.013 7efcc742f700 1 -- 172.21.2.101:0/3820629045 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7efcc00f0b20 con 0x7efcc008dea0 2021-02-10T11:31:23.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:23.788 7efcb1ffb700 1 -- 172.21.2.101:0/3820629045 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v16) v1 ==== 104+0+0 (secure 0 0 0) 0x7efcb400a000 con 0x7efcc008dea0 2021-02-10T11:31:23.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:23.834 7efcc742f700 1 -- 172.21.2.101:0/3820629045 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efca0018700 msgr2=0x7efca001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:23.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:23.834 7efcc742f700 1 --2- 172.21.2.101:0/3820629045 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efca0018700 0x7efca001abc0 crc :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:23.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:23.834 7efcc742f700 1 -- 172.21.2.101:0/3820629045 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcc008dea0 msgr2=0x7efcc009c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:23.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:23.834 7efcc742f700 1 --2- 172.21.2.101:0/3820629045 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcc008dea0 0x7efcc009c5f0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7efcb4009630 tx=0x7efcb400b980).stop 2021-02-10T11:31:23.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:23.834 7efcc742f700 1 -- 172.21.2.101:0/3820629045 shutdown_connections 2021-02-10T11:31:23.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:23.834 7efcc742f700 1 --2- 172.21.2.101:0/3820629045 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efca0018700 0x7efca001abc0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:23.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:23.834 7efcc742f700 1 --2- 172.21.2.101:0/3820629045 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcc01ad3f0 0x7efcc01ad7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:23.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:23.834 7efcc742f700 1 --2- 172.21.2.101:0/3820629045 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efcc00b4330 0x7efcc00b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:23.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:23.834 7efcc742f700 1 --2- 172.21.2.101:0/3820629045 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcc008dea0 0x7efcc009c5f0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:23.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:23.834 7efcc742f700 1 -- 172.21.2.101:0/3820629045 >> 172.21.2.101:0/3820629045 conn(0x7efcc01a30f0 msgr2=0x7efcc018c240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:23.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:23.835 7efcc742f700 1 -- 172.21.2.101:0/3820629045 shutdown_connections 2021-02-10T11:31:23.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:23.835 7efcc742f700 1 -- 172.21.2.101:0/3820629045 wait complete. 2021-02-10T11:31:23.853 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-02-10T11:31:24.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.268 7f7ac0e84700 1 Processor -- start 2021-02-10T11:31:24.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.268 7f7ac0e84700 1 -- start start 2021-02-10T11:31:24.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.269 7f7ac0e84700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7abc0d1870 0x7f7abc1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:24.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.269 7f7ac0e84700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7abc1b0ae0 0x7f7abc108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:24.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.269 7f7ac0e84700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7abc0fe1c0 con 0x7f7abc0d1c70 2021-02-10T11:31:24.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.269 7f7ac0e84700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7abc109a60 con 0x7f7abc0d1870 2021-02-10T11:31:24.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.269 7f7ac0e84700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7abc109750 con 0x7f7abc1b0ae0 2021-02-10T11:31:24.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.269 7f7abab13700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7abc0d1870 0x7f7abc1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:24.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.269 7f7abb314700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f7abc0d1c70 0x7f7abc108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50468/0 (socket says 172.21.2.101:50468) 2021-02-10T11:31:24.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.269 7f7abb314700 1 -- 172.21.2.101:0/1545160242 learned_addr learned my addr 172.21.2.101:0/1545160242 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:24.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.270 7f7abab13700 1 -- 172.21.2.101:0/1545160242 >> v2:172.21.2.108:3300/0 conn(0x7f7abc1b0ae0 msgr2=0x7f7abc108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:24.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.270 7f7abab13700 1 --2- 172.21.2.101:0/1545160242 >> v2:172.21.2.108:3300/0 conn(0x7f7abc1b0ae0 0x7f7abc108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:24.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.270 7f7abab13700 1 -- 172.21.2.101:0/1545160242 >> v1:172.21.2.101:6789/0 conn(0x7f7abc0d1c70 legacy=0x7f7abc108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:24.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.270 7f7abab13700 1 -- 172.21.2.101:0/1545160242 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7abc088020 con 0x7f7abc0d1870 2021-02-10T11:31:24.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.270 7f7abab13700 1 --2- 172.21.2.101:0/1545160242 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7abc0d1870 0x7f7abc1b04c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f7aac006650 tx=0x7f7aac009a80).ready entity=mon.0 client_cookie=60bcdd271bc48dbf server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:24.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.270 7f7ab9b11700 1 -- 172.21.2.101:0/1545160242 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7aac0009b0 con 0x7f7abc0d1870 2021-02-10T11:31:24.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.270 7f7ab9b11700 1 -- 172.21.2.101:0/1545160242 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aac00c0c0 con 0x7f7abc0d1870 2021-02-10T11:31:24.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.270 7f7ab9b11700 1 -- 172.21.2.101:0/1545160242 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7aac00c5e0 con 0x7f7abc0d1870 2021-02-10T11:31:24.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.270 7f7ac0e84700 1 -- 172.21.2.101:0/1545160242 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7abc0d1870 msgr2=0x7f7abc1b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:24.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.270 7f7ac0e84700 1 --2- 172.21.2.101:0/1545160242 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7abc0d1870 0x7f7abc1b04c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f7aac006650 tx=0x7f7aac009a80).stop 2021-02-10T11:31:24.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.271 7f7ac0e84700 1 -- 172.21.2.101:0/1545160242 shutdown_connections 2021-02-10T11:31:24.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.271 7f7ac0e84700 1 --2- 172.21.2.101:0/1545160242 >> v2:172.21.2.108:3300/0 conn(0x7f7abc1b0ae0 0x7f7abc108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:24.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.271 7f7ac0e84700 1 --2- 172.21.2.101:0/1545160242 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7abc0d1870 0x7f7abc1b04c0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:24.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.271 7f7ac0e84700 1 -- 172.21.2.101:0/1545160242 >> 172.21.2.101:0/1545160242 conn(0x7f7abc0ce420 msgr2=0x7f7abc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:24.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.271 7f7ac0e84700 1 -- 172.21.2.101:0/1545160242 shutdown_connections 2021-02-10T11:31:24.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.271 7f7ac0e84700 1 -- 172.21.2.101:0/1545160242 wait complete. 2021-02-10T11:31:24.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.286 7f7ac0e84700 1 Processor -- start 2021-02-10T11:31:24.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.286 7f7ac0e84700 1 -- start start 2021-02-10T11:31:24.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.286 7f7ac0e84700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7abc0ac5e0 0x7f7abc189e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:24.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.286 7f7ac0e84700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7abc18a4a0 0x7f7abc1aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:24.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.286 7f7ac0e84700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7abc1abc40 0x7f7abc1ab440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:24.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.286 7f7ac0e84700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc1ac040 con 0x7f7abc0ac5e0 2021-02-10T11:31:24.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.286 7f7ac0e84700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc18a9a0 con 0x7f7abc1abc40 2021-02-10T11:31:24.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.286 7f7ac0e84700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc18af20 con 0x7f7abc18a4a0 2021-02-10T11:31:24.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.286 7f7abab13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7abc0ac5e0 0x7f7abc189e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:24.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.286 7f7abab13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7abc0ac5e0 0x7f7abc189e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49980/0 (socket says 172.21.2.101:49980) 2021-02-10T11:31:24.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.286 7f7abab13700 1 -- 172.21.2.101:0/325066676 learned_addr learned my addr 172.21.2.101:0/325066676 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:24.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.287 7f7abab13700 1 -- 172.21.2.101:0/325066676 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7abc18a4a0 msgr2=0x7f7abc1aae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:24.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.287 7f7abab13700 1 --2- 172.21.2.101:0/325066676 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7abc18a4a0 0x7f7abc1aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:24.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.287 7f7abab13700 1 -- 172.21.2.101:0/325066676 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7abc1abc40 msgr2=0x7f7abc1ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:24.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.287 7f7abab13700 1 --2- 172.21.2.101:0/325066676 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7abc1abc40 0x7f7abc1ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:24.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.287 7f7abab13700 1 -- 172.21.2.101:0/325066676 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7abc18b100 con 0x7f7abc0ac5e0 2021-02-10T11:31:24.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.287 7f7abab13700 1 --2- 172.21.2.101:0/325066676 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7abc0ac5e0 0x7f7abc189e80 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7f7aac009630 tx=0x7f7aac00b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:24.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.287 7f7aab7fe700 1 -- 172.21.2.101:0/325066676 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7aac0093a0 con 0x7f7abc0ac5e0 2021-02-10T11:31:24.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.287 7f7aab7fe700 1 -- 172.21.2.101:0/325066676 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aac00bd70 con 0x7f7abc0ac5e0 2021-02-10T11:31:24.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.287 7f7aab7fe700 1 -- 172.21.2.101:0/325066676 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7aac005560 con 0x7f7abc0ac5e0 2021-02-10T11:31:24.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.287 7f7ac0e84700 1 -- 172.21.2.101:0/325066676 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7abc18b100 con 0x7f7abc0ac5e0 2021-02-10T11:31:24.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.287 7f7ac0e84700 1 -- 172.21.2.101:0/325066676 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7abc18b690 con 0x7f7abc0ac5e0 2021-02-10T11:31:24.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.288 7f7aab7fe700 1 -- 172.21.2.101:0/325066676 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7aac007020 con 0x7f7abc0ac5e0 2021-02-10T11:31:24.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.288 7f7aab7fe700 1 --2- 172.21.2.101:0/325066676 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7aa00186c0 0x7f7aa001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:24.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.288 7f7aab7fe700 1 -- 172.21.2.101:0/325066676 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f7aac032440 con 0x7f7abc0ac5e0 2021-02-10T11:31:24.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.288 7f7aba312700 1 --2- 172.21.2.101:0/325066676 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7aa00186c0 0x7f7aa001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:24.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.289 7f7aba312700 1 --2- 172.21.2.101:0/325066676 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7aa00186c0 0x7f7aa001abc0 crc :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:24.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.319 7f7ac0e84700 1 -- 172.21.2.101:0/325066676 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7abc1ac040 con 0x7f7abc0ac5e0 2021-02-10T11:31:24.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.321 7f7aab7fe700 1 -- 172.21.2.101:0/325066676 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7aac009340 con 0x7f7abc0ac5e0 2021-02-10T11:31:24.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.448 7f7ac0e84700 1 -- 172.21.2.101:0/325066676 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7abc1ac040 con 0x7f7abc0ac5e0 2021-02-10T11:31:24.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.792 7f7aab7fe700 1 -- 172.21.2.101:0/325066676 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v31) v1 ==== 185+0+0 (secure 0 0 0) 0x7f7aac01f510 con 0x7f7abc0ac5e0 2021-02-10T11:31:24.826 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' removed 2021-02-10T11:31:24.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.817 7f7ac0e84700 1 -- 172.21.2.101:0/325066676 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7aa00186c0 msgr2=0x7f7aa001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:24.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.817 7f7ac0e84700 1 --2- 172.21.2.101:0/325066676 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7aa00186c0 0x7f7aa001abc0 crc :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:24.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.817 7f7ac0e84700 1 -- 172.21.2.101:0/325066676 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7abc0ac5e0 msgr2=0x7f7abc189e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:24.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.817 7f7ac0e84700 1 --2- 172.21.2.101:0/325066676 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7abc0ac5e0 0x7f7abc189e80 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7f7aac009630 tx=0x7f7aac00b980).stop 2021-02-10T11:31:24.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.817 7f7ac0e84700 1 -- 172.21.2.101:0/325066676 shutdown_connections 2021-02-10T11:31:24.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.817 7f7ac0e84700 1 --2- 172.21.2.101:0/325066676 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7aa00186c0 0x7f7aa001abc0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:24.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.817 7f7ac0e84700 1 --2- 172.21.2.101:0/325066676 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7abc18a4a0 0x7f7abc1aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:24.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.817 7f7ac0e84700 1 --2- 172.21.2.101:0/325066676 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7abc1abc40 0x7f7abc1ab440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:24.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.817 7f7ac0e84700 1 --2- 172.21.2.101:0/325066676 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7abc0ac5e0 0x7f7abc189e80 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:24.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.817 7f7ac0e84700 1 -- 172.21.2.101:0/325066676 >> 172.21.2.101:0/325066676 conn(0x7f7abc19f840 msgr2=0x7f7abc08f700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:24.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.817 7f7ac0e84700 1 -- 172.21.2.101:0/325066676 shutdown_connections 2021-02-10T11:31:24.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:24.817 7f7ac0e84700 1 -- 172.21.2.101:0/325066676 wait complete. 2021-02-10T11:31:24.836 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-02-10T11:31:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.240 7fdfe1d67700 1 Processor -- start 2021-02-10T11:31:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.240 7fdfe1d67700 1 -- start start 2021-02-10T11:31:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.240 7fdfe1d67700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdfdc10aff0 0x7fdfdc0cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.240 7fdfe1d67700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfdc0fe1c0 con 0x7fdfdc0d1870 2021-02-10T11:31:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.240 7fdfe1d67700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfdc1b0cb0 con 0x7fdfdc0d1c70 2021-02-10T11:31:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.240 7fdfe1d67700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfdc109a60 con 0x7fdfdc10aff0 2021-02-10T11:31:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.240 7fdfdb7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fdfdc0d1870 0x7fdfdc1b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50480/0 (socket says 172.21.2.101:50480) 2021-02-10T11:31:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.240 7fdfdbfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdfdc10aff0 0x7fdfdc0cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.240 7fdfdb7fe700 1 -- 172.21.2.101:0/60129060 learned_addr learned my addr 172.21.2.101:0/60129060 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.241 7fdfdbfff700 1 -- 172.21.2.101:0/60129060 >> v1:172.21.2.108:6789/0 conn(0x7fdfdc0d1c70 legacy=0x7fdfdc0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.241 7fdfdbfff700 1 -- 172.21.2.101:0/60129060 >> v1:172.21.2.101:6789/0 conn(0x7fdfdc0d1870 legacy=0x7fdfdc1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.241 7fdfda7fc700 1 -- 172.21.2.101:0/60129060 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4095452773 0 0) 0x7fdfd4001500 con 0x7fdfdc0d1870 2021-02-10T11:31:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.241 7fdfdbfff700 1 -- 172.21.2.101:0/60129060 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfdc10b3f0 con 0x7fdfdc10aff0 2021-02-10T11:31:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.241 7fdfdbfff700 1 --2- 172.21.2.101:0/60129060 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfdc10aff0 0x7fdfdc0cf750 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7fdfcc006ab0 tx=0x7fdfcc009ac0).ready entity=mon.0 client_cookie=fdd8e2c75cc304a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.241 7fdfda7fc700 1 -- 172.21.2.101:0/60129060 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdfcc0066d0 con 0x7fdfdc10aff0 2021-02-10T11:31:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.241 7fdfda7fc700 1 -- 172.21.2.101:0/60129060 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfcc0022e0 con 0x7fdfdc10aff0 2021-02-10T11:31:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.241 7fdfda7fc700 1 -- 172.21.2.101:0/60129060 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdfcc0027c0 con 0x7fdfdc10aff0 2021-02-10T11:31:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.241 7fdfe1d67700 1 -- 172.21.2.101:0/60129060 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfdc10aff0 msgr2=0x7fdfdc0cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.241 7fdfe1d67700 1 --2- 172.21.2.101:0/60129060 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfdc10aff0 0x7fdfdc0cf750 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7fdfcc006ab0 tx=0x7fdfcc009ac0).stop 2021-02-10T11:31:25.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.242 7fdfe1d67700 1 -- 172.21.2.101:0/60129060 shutdown_connections 2021-02-10T11:31:25.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.242 7fdfe1d67700 1 --2- 172.21.2.101:0/60129060 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfdc10aff0 0x7fdfdc0cf750 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:25.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.242 7fdfe1d67700 1 -- 172.21.2.101:0/60129060 >> 172.21.2.101:0/60129060 conn(0x7fdfdc0ce420 msgr2=0x7fdfdc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:25.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.242 7fdfe1d67700 1 -- 172.21.2.101:0/60129060 shutdown_connections 2021-02-10T11:31:25.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.242 7fdfe1d67700 1 -- 172.21.2.101:0/60129060 wait complete. 2021-02-10T11:31:25.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.257 7fdfe1d67700 1 Processor -- start 2021-02-10T11:31:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.257 7fdfe1d67700 1 -- start start 2021-02-10T11:31:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.257 7fdfe1d67700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdfdc1a9930 0x7fdfdc1a9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.257 7fdfe1d67700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdfdc1abd60 0x7fdfdc192a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.257 7fdfe1d67700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfdc0b03e0 0x7fdfdc0b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.257 7fdfe1d67700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc1932c0 con 0x7fdfdc0b03e0 2021-02-10T11:31:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.257 7fdfe1d67700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc193840 con 0x7fdfdc1abd60 2021-02-10T11:31:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.257 7fdfe1d67700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc193a20 con 0x7fdfdc1a9930 2021-02-10T11:31:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.257 7fdfdbfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfdc0b03e0 0x7fdfdc0b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.257 7fdfdb7fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdfdc1a9930 0x7fdfdc1a9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.257 7fdfdbfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfdc0b03e0 0x7fdfdc0b07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:49996/0 (socket says 172.21.2.101:49996) 2021-02-10T11:31:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.257 7fdfdbfff700 1 -- 172.21.2.101:0/2097142066 learned_addr learned my addr 172.21.2.101:0/2097142066 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:25.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.257 7fdfdaffd700 1 --2- 172.21.2.101:0/2097142066 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdfdc1abd60 0x7fdfdc192a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:25.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.258 7fdfdbfff700 1 -- 172.21.2.101:0/2097142066 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdfdc1a9930 msgr2=0x7fdfdc1a9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:25.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.258 7fdfdbfff700 1 --2- 172.21.2.101:0/2097142066 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdfdc1a9930 0x7fdfdc1a9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:25.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.258 7fdfdbfff700 1 -- 172.21.2.101:0/2097142066 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdfdc1abd60 msgr2=0x7fdfdc192a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:25.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.258 7fdfdbfff700 1 --2- 172.21.2.101:0/2097142066 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdfdc1abd60 0x7fdfdc192a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:25.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.258 7fdfdbfff700 1 -- 172.21.2.101:0/2097142066 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfdc193c40 con 0x7fdfdc0b03e0 2021-02-10T11:31:25.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.258 7fdfdbfff700 1 --2- 172.21.2.101:0/2097142066 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfdc0b03e0 0x7fdfdc0b07e0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7fdfcc000e10 tx=0x7fdfcc002880).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:25.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.258 7fdfc3fff700 1 -- 172.21.2.101:0/2097142066 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdfcc00c2c0 con 0x7fdfdc0b03e0 2021-02-10T11:31:25.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.258 7fdfc3fff700 1 -- 172.21.2.101:0/2097142066 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfcc00c580 con 0x7fdfdc0b03e0 2021-02-10T11:31:25.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.258 7fdfc3fff700 1 -- 172.21.2.101:0/2097142066 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdfcc00c9d0 con 0x7fdfdc0b03e0 2021-02-10T11:31:25.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.258 7fdfe1d67700 1 -- 172.21.2.101:0/2097142066 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfdc193c40 con 0x7fdfdc0b03e0 2021-02-10T11:31:25.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.258 7fdfe1d67700 1 -- 172.21.2.101:0/2097142066 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfdc1941d0 con 0x7fdfdc0b03e0 2021-02-10T11:31:25.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.259 7fdfc3fff700 1 -- 172.21.2.101:0/2097142066 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdfcc00c760 con 0x7fdfdc0b03e0 2021-02-10T11:31:25.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.259 7fdfc3fff700 1 --2- 172.21.2.101:0/2097142066 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdfbc0186c0 0x7fdfbc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:25.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.259 7fdfc3fff700 1 -- 172.21.2.101:0/2097142066 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5221+0+0 (secure 0 0 0) 0x7fdfcc032520 con 0x7fdfdc0b03e0 2021-02-10T11:31:25.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.259 7fdfdb7fe700 1 --2- 172.21.2.101:0/2097142066 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdfbc0186c0 0x7fdfbc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:25.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.259 7fdfdb7fe700 1 --2- 172.21.2.101:0/2097142066 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdfbc0186c0 0x7fdfbc01abc0 crc :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:25.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.289 7fdfe1d67700 1 -- 172.21.2.101:0/2097142066 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfdc0828e0 con 0x7fdfdc0b03e0 2021-02-10T11:31:25.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.292 7fdfc3fff700 1 -- 172.21.2.101:0/2097142066 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdfcc01f540 con 0x7fdfdc0b03e0 2021-02-10T11:31:25.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.419 7fdfe1d67700 1 -- 172.21.2.101:0/2097142066 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdfdc0828e0 con 0x7fdfdc0b03e0 2021-02-10T11:31:25.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.794 7fdfc3fff700 1 -- 172.21.2.101:0/2097142066 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v32) v1 ==== 173+0+0 (secure 0 0 0) 0x7fdfcc00c580 con 0x7fdfdc0b03e0 2021-02-10T11:31:25.847 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' removed 2021-02-10T11:31:25.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.838 7fdfe1d67700 1 -- 172.21.2.101:0/2097142066 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdfbc0186c0 msgr2=0x7fdfbc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:25.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.838 7fdfe1d67700 1 --2- 172.21.2.101:0/2097142066 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdfbc0186c0 0x7fdfbc01abc0 crc :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:25.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.838 7fdfe1d67700 1 -- 172.21.2.101:0/2097142066 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfdc0b03e0 msgr2=0x7fdfdc0b07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:25.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.838 7fdfe1d67700 1 --2- 172.21.2.101:0/2097142066 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfdc0b03e0 0x7fdfdc0b07e0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7fdfcc000e10 tx=0x7fdfcc002880).stop 2021-02-10T11:31:25.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.838 7fdfe1d67700 1 -- 172.21.2.101:0/2097142066 shutdown_connections 2021-02-10T11:31:25.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.838 7fdfe1d67700 1 --2- 172.21.2.101:0/2097142066 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfdc0b03e0 0x7fdfdc0b07e0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:25.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.838 7fdfe1d67700 1 --2- 172.21.2.101:0/2097142066 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdfdc1abd60 0x7fdfdc192a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:25.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.838 7fdfe1d67700 1 --2- 172.21.2.101:0/2097142066 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdfbc0186c0 0x7fdfbc01abc0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:25.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.838 7fdfe1d67700 1 --2- 172.21.2.101:0/2097142066 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdfdc1a9930 0x7fdfdc1a9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:25.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.838 7fdfe1d67700 1 -- 172.21.2.101:0/2097142066 >> 172.21.2.101:0/2097142066 conn(0x7fdfdc19f820 msgr2=0x7fdfdc082b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:25.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.838 7fdfe1d67700 1 -- 172.21.2.101:0/2097142066 shutdown_connections 2021-02-10T11:31:25.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:25.838 7fdfe1d67700 1 -- 172.21.2.101:0/2097142066 wait complete. 2021-02-10T11:31:25.857 INFO:tasks.ceph.mds.a:Restarting daemon 2021-02-10T11:31:25.857 INFO:tasks.ceph.mds.a:Stopping old one... 2021-02-10T11:31:25.858 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-02-10T11:31:25.858 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:31:25.858 INFO:tasks.ceph.mds.b:Restarting daemon 2021-02-10T11:31:25.858 INFO:tasks.ceph.mds.b:Stopping old one... 2021-02-10T11:31:25.858 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-02-10T11:31:25.858 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:31:25.863 INFO:tasks.ceph.mds.a:Stopped 2021-02-10T11:31:25.864 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-02-10T11:31:25.864 INFO:tasks.ceph.mds.b:Stopped 2021-02-10T11:31:25.865 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-02-10T11:31:25.906 INFO:tasks.ceph.mds.a:Started 2021-02-10T11:31:25.907 DEBUG:teuthology.parallel:result is None 2021-02-10T11:31:25.907 INFO:tasks.ceph.mds.b:Started 2021-02-10T11:31:25.907 DEBUG:teuthology.parallel:result is None 2021-02-10T11:31:25.907 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd blacklist clear 2021-02-10T11:31:26.312 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:31:26.314 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:31:26.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.308 7f82f6e84700 1 Processor -- start 2021-02-10T11:31:26.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.308 7f82f6e84700 1 -- start start 2021-02-10T11:31:26.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.308 7f82f6e84700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f82f00e5470 0x7f82f007e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:26.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.308 7f82f6e84700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f00e5f70 con 0x7f82f007f300 2021-02-10T11:31:26.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.308 7f82f6e84700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f007ed80 con 0x7f82f00e5070 2021-02-10T11:31:26.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.308 7f82f6e84700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f01b4010 con 0x7f82f00e5470 2021-02-10T11:31:26.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.308 7f82f54a1700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f82f007f300 0x7f82f007f700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50508/0 (socket says 172.21.2.101:50508) 2021-02-10T11:31:26.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.308 7f82f54a1700 1 -- 172.21.2.101:0/3536524754 learned_addr learned my addr 172.21.2.101:0/3536524754 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:26.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.308 7f82effff700 1 --2- 172.21.2.101:0/3536524754 >> v2:172.21.2.101:3300/0 conn(0x7f82f00e5470 0x7f82f007e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:26.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.308 7f82ef7fe700 1 -- 172.21.2.101:0/3536524754 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936571538 0 0) 0x7f82e0000c10 con 0x7f82f007f300 2021-02-10T11:31:26.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.309 7f82ef7fe700 1 -- 172.21.2.101:0/3536524754 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82d8002fe0 con 0x7f82f007f300 2021-02-10T11:31:26.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.309 7f82effff700 1 -- 172.21.2.101:0/3536524754 >> v1:172.21.2.108:6789/0 conn(0x7f82f00e5070 legacy=0x7f82f01b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:26.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.309 7f82effff700 1 -- 172.21.2.101:0/3536524754 >> v1:172.21.2.101:6789/0 conn(0x7f82f007f300 legacy=0x7f82f007f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:26.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.309 7f82effff700 1 -- 172.21.2.101:0/3536524754 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f00d1790 con 0x7f82f00e5470 2021-02-10T11:31:26.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.309 7f82effff700 1 --2- 172.21.2.101:0/3536524754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82f00e5470 0x7f82f007e6a0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7f82e4006650 tx=0x7f82e4009a80).ready entity=mon.0 client_cookie=a0f2b49f3898453e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:26.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.309 7f82ef7fe700 1 -- 172.21.2.101:0/3536524754 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f82e40009b0 con 0x7f82f00e5470 2021-02-10T11:31:26.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.309 7f82ef7fe700 1 -- 172.21.2.101:0/3536524754 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e400c0c0 con 0x7f82f00e5470 2021-02-10T11:31:26.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.309 7f82ef7fe700 1 -- 172.21.2.101:0/3536524754 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f82e400c510 con 0x7f82f00e5470 2021-02-10T11:31:26.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.309 7f82f6e84700 1 -- 172.21.2.101:0/3536524754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82f00e5470 msgr2=0x7f82f007e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:26.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.309 7f82f6e84700 1 --2- 172.21.2.101:0/3536524754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82f00e5470 0x7f82f007e6a0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7f82e4006650 tx=0x7f82e4009a80).stop 2021-02-10T11:31:26.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.309 7f82f6e84700 1 -- 172.21.2.101:0/3536524754 shutdown_connections 2021-02-10T11:31:26.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.309 7f82f6e84700 1 --2- 172.21.2.101:0/3536524754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82f00e5470 0x7f82f007e6a0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:26.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.309 7f82f6e84700 1 -- 172.21.2.101:0/3536524754 >> 172.21.2.101:0/3536524754 conn(0x7f82f00ce690 msgr2=0x7f82f00e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:26.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.310 7f82f6e84700 1 -- 172.21.2.101:0/3536524754 shutdown_connections 2021-02-10T11:31:26.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.310 7f82f6e84700 1 -- 172.21.2.101:0/3536524754 wait complete. 2021-02-10T11:31:26.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.324 7f82f6e84700 1 Processor -- start 2021-02-10T11:31:26.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.324 7f82f6e84700 1 -- start start 2021-02-10T11:31:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.325 7f82f6e84700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f82f00a1b40 0x7f82f0194850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.325 7f82f6e84700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82f01a2f80 0x7f82f01a3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.325 7f82f6e84700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82f00be290 0x7f82f00be690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.325 7f82f6e84700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f0194e70 con 0x7f82f01a2f80 2021-02-10T11:31:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.325 7f82f6e84700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f0195060 con 0x7f82f00be290 2021-02-10T11:31:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.325 7f82f6e84700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f0191bc0 con 0x7f82f00a1b40 2021-02-10T11:31:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.325 7f82effff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82f01a2f80 0x7f82f01a3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.325 7f82effff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82f01a2f80 0x7f82f01a3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50020/0 (socket says 172.21.2.101:50020) 2021-02-10T11:31:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.325 7f82effff700 1 -- 172.21.2.101:0/1852850596 learned_addr learned my addr 172.21.2.101:0/1852850596 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.325 7f82f54a1700 1 --2- 172.21.2.101:0/1852850596 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82f00be290 0x7f82f00be690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.325 7f82f4ca0700 1 --2- 172.21.2.101:0/1852850596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f82f00a1b40 0x7f82f0194850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.326 7f82effff700 1 -- 172.21.2.101:0/1852850596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f82f00a1b40 msgr2=0x7f82f0194850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.326 7f82effff700 1 --2- 172.21.2.101:0/1852850596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f82f00a1b40 0x7f82f0194850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.326 7f82effff700 1 -- 172.21.2.101:0/1852850596 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82f00be290 msgr2=0x7f82f00be690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.326 7f82effff700 1 --2- 172.21.2.101:0/1852850596 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82f00be290 0x7f82f00be690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:26.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.326 7f82effff700 1 -- 172.21.2.101:0/1852850596 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f0191de0 con 0x7f82f01a2f80 2021-02-10T11:31:26.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.326 7f82f4ca0700 1 --2- 172.21.2.101:0/1852850596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f82f00a1b40 0x7f82f0194850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:31:26.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.326 7f82effff700 1 --2- 172.21.2.101:0/1852850596 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82f01a2f80 0x7f82f01a3380 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7f82e4009650 tx=0x7f82e400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:26.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.326 7f82ed7fa700 1 -- 172.21.2.101:0/1852850596 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f82e400bd50 con 0x7f82f01a2f80 2021-02-10T11:31:26.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.326 7f82ed7fa700 1 -- 172.21.2.101:0/1852850596 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e4006030 con 0x7f82f01a2f80 2021-02-10T11:31:26.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.326 7f82f6e84700 1 -- 172.21.2.101:0/1852850596 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f0191de0 con 0x7f82f01a2f80 2021-02-10T11:31:26.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.326 7f82f6e84700 1 -- 172.21.2.101:0/1852850596 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f0191de0 con 0x7f82f01a2f80 2021-02-10T11:31:26.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.326 7f82ed7fa700 1 -- 172.21.2.101:0/1852850596 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f82e4005560 con 0x7f82f01a2f80 2021-02-10T11:31:26.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.326 7f82ed7fa700 1 -- 172.21.2.101:0/1852850596 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f82e40052e0 con 0x7f82f01a2f80 2021-02-10T11:31:26.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.326 7f82ed7fa700 1 --2- 172.21.2.101:0/1852850596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82d00186d0 0x7f82d001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:26.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.327 7f82ed7fa700 1 -- 172.21.2.101:0/1852850596 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4839+0+0 (secure 0 0 0) 0x7f82e40324b0 con 0x7f82f01a2f80 2021-02-10T11:31:26.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.327 7f82f4ca0700 1 --2- 172.21.2.101:0/1852850596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82d00186d0 0x7f82d001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:26.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.327 7f82f4ca0700 1 --2- 172.21.2.101:0/1852850596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82d00186d0 0x7f82d001abd0 crc :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:26.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.358 7f82f6e84700 1 -- 172.21.2.101:0/1852850596 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f007ed80 con 0x7f82f01a2f80 2021-02-10T11:31:26.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.360 7f82ed7fa700 1 -- 172.21.2.101:0/1852850596 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f82e4000940 con 0x7f82f01a2f80 2021-02-10T11:31:26.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.499 7f82f6e84700 1 -- 172.21.2.101:0/1852850596 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f82f007ed80 con 0x7f82f01a2f80 2021-02-10T11:31:26.797 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:31:26.800 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:31:26.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.842 7f82ed7fa700 1 -- 172.21.2.101:0/1852850596 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v33) v1 ==== 97+0+0 (secure 0 0 0) 0x7f82e40052e0 con 0x7f82f01a2f80 2021-02-10T11:31:26.875 INFO:teuthology.orchestra.run.gibba001.stderr: removed all blacklist entries 2021-02-10T11:31:26.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.866 7f82f6e84700 1 -- 172.21.2.101:0/1852850596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82d00186d0 msgr2=0x7f82d001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:26.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.866 7f82f6e84700 1 --2- 172.21.2.101:0/1852850596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82d00186d0 0x7f82d001abd0 crc :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:26.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.866 7f82f6e84700 1 -- 172.21.2.101:0/1852850596 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82f01a2f80 msgr2=0x7f82f01a3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:26.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.866 7f82f6e84700 1 --2- 172.21.2.101:0/1852850596 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82f01a2f80 0x7f82f01a3380 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7f82e4009650 tx=0x7f82e400b960).stop 2021-02-10T11:31:26.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.866 7f82f6e84700 1 -- 172.21.2.101:0/1852850596 shutdown_connections 2021-02-10T11:31:26.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.866 7f82f6e84700 1 --2- 172.21.2.101:0/1852850596 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f82f00be290 0x7f82f00be690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:26.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.866 7f82f6e84700 1 --2- 172.21.2.101:0/1852850596 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82f01a2f80 0x7f82f01a3380 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:26.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.866 7f82f6e84700 1 --2- 172.21.2.101:0/1852850596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f82d00186d0 0x7f82d001abd0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:26.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.866 7f82f6e84700 1 --2- 172.21.2.101:0/1852850596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f82f00a1b40 0x7f82f0194850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:26.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.866 7f82f6e84700 1 -- 172.21.2.101:0/1852850596 >> 172.21.2.101:0/1852850596 conn(0x7f82f00a7430 msgr2=0x7f82f00898f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:26.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.866 7f82f6e84700 1 -- 172.21.2.101:0/1852850596 shutdown_connections 2021-02-10T11:31:26.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:26.866 7f82f6e84700 1 -- 172.21.2.101:0/1852850596 wait complete. 2021-02-10T11:31:26.886 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2021-02-10T11:31:26.886 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:31:27.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.199 7f3326bb6700 1 Processor -- start 2021-02-10T11:31:27.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.199 7f3326bb6700 1 -- start start 2021-02-10T11:31:27.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.200 7f3326bb6700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f33200e5470 0x7f332007e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:27.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.200 7f3326bb6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f332007f300 0x7f332007f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:27.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.200 7f3326bb6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f332007ed80 con 0x7f33200e5070 2021-02-10T11:31:27.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.200 7f3326bb6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33201b4010 con 0x7f332007f300 2021-02-10T11:31:27.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.200 7f3326bb6700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33200d1790 con 0x7f33200e5470 2021-02-10T11:31:27.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.200 7f33251d3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f332007f300 0x7f332007f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:27.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.200 7f33251d3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f332007f300 0x7f332007f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50028/0 (socket says 172.21.2.101:50028) 2021-02-10T11:31:27.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.200 7f33251d3700 1 -- 172.21.2.101:0/1542938057 learned_addr learned my addr 172.21.2.101:0/1542938057 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:27.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.200 7f33251d3700 1 -- 172.21.2.101:0/1542938057 >> v2:172.21.2.108:3300/0 conn(0x7f33200e5470 msgr2=0x7f332007e6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:31:27.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.200 7f33251d3700 1 --2- 172.21.2.101:0/1542938057 >> v2:172.21.2.108:3300/0 conn(0x7f33200e5470 0x7f332007e6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:27.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.200 7f33251d3700 1 -- 172.21.2.101:0/1542938057 >> v1:172.21.2.108:6789/0 conn(0x7f33200e5070 legacy=0x7f33201b3820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:31:27.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.200 7f33251d3700 1 -- 172.21.2.101:0/1542938057 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33200d19b0 con 0x7f332007f300 2021-02-10T11:31:27.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.200 7f33251d3700 1 --2- 172.21.2.101:0/1542938057 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f332007f300 0x7f332007f700 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f3318006b80 tx=0x7f3318009b90).ready entity=mon.0 client_cookie=3ee9829dad575f81 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:27.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.201 7f331f7fe700 1 -- 172.21.2.101:0/1542938057 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f33180022e0 con 0x7f332007f300 2021-02-10T11:31:27.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.201 7f331f7fe700 1 -- 172.21.2.101:0/1542938057 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3318002620 con 0x7f332007f300 2021-02-10T11:31:27.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.201 7f331f7fe700 1 -- 172.21.2.101:0/1542938057 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3318002b00 con 0x7f332007f300 2021-02-10T11:31:27.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.201 7f3326bb6700 1 -- 172.21.2.101:0/1542938057 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f332007f300 msgr2=0x7f332007f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:27.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.201 7f3326bb6700 1 --2- 172.21.2.101:0/1542938057 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f332007f300 0x7f332007f700 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f3318006b80 tx=0x7f3318009b90).stop 2021-02-10T11:31:27.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.201 7f3326bb6700 1 -- 172.21.2.101:0/1542938057 shutdown_connections 2021-02-10T11:31:27.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.201 7f3326bb6700 1 --2- 172.21.2.101:0/1542938057 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f332007f300 0x7f332007f700 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:27.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.201 7f3326bb6700 1 --2- 172.21.2.101:0/1542938057 >> v2:172.21.2.108:3300/0 conn(0x7f33200e5470 0x7f332007e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:27.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.201 7f3326bb6700 1 -- 172.21.2.101:0/1542938057 >> 172.21.2.101:0/1542938057 conn(0x7f33200ce690 msgr2=0x7f33200e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:27.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.201 7f3326bb6700 1 -- 172.21.2.101:0/1542938057 shutdown_connections 2021-02-10T11:31:27.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.201 7f3326bb6700 1 -- 172.21.2.101:0/1542938057 wait complete. 2021-02-10T11:31:27.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.216 7f3326bb6700 1 Processor -- start 2021-02-10T11:31:27.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.216 7f3326bb6700 1 -- start start 2021-02-10T11:31:27.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.216 7f3326bb6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f33201aad40 0x7f3320192110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:27.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.216 7f3326bb6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33201a1aa0 0x7f33200b7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:27.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.216 7f3326bb6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3320192730 0x7f33200b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:27.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.216 7f3326bb6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3320192d70 con 0x7f33201a1aa0 2021-02-10T11:31:27.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.216 7f3326bb6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33201a1590 con 0x7f33201aad40 2021-02-10T11:31:27.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.216 7f3326bb6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33201a11b0 con 0x7f3320192730 2021-02-10T11:31:27.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.216 7f331ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33201a1aa0 0x7f33200b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:27.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.216 7f331ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33201a1aa0 0x7f33200b7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50034/0 (socket says 172.21.2.101:50034) 2021-02-10T11:31:27.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.216 7f331ffff700 1 -- 172.21.2.101:0/231904682 learned_addr learned my addr 172.21.2.101:0/231904682 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:27.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.217 7f331ffff700 1 -- 172.21.2.101:0/231904682 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3320192730 msgr2=0x7f33200b7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:27.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.217 7f331ffff700 1 --2- 172.21.2.101:0/231904682 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3320192730 0x7f33200b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:27.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.217 7f331ffff700 1 -- 172.21.2.101:0/231904682 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f33201aad40 msgr2=0x7f3320192110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:27.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.217 7f331ffff700 1 --2- 172.21.2.101:0/231904682 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f33201aad40 0x7f3320192110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:27.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.217 7f331ffff700 1 -- 172.21.2.101:0/231904682 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33200b8170 con 0x7f33201a1aa0 2021-02-10T11:31:27.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.217 7f331ffff700 1 --2- 172.21.2.101:0/231904682 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33201a1aa0 0x7f33200b7530 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7f3310009960 tx=0x7f331000bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:27.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.217 7f331d7fa700 1 -- 172.21.2.101:0/231904682 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3310000fc0 con 0x7f33201a1aa0 2021-02-10T11:31:27.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.217 7f331d7fa700 1 -- 172.21.2.101:0/231904682 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3310003020 con 0x7f33201a1aa0 2021-02-10T11:31:27.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.217 7f331d7fa700 1 -- 172.21.2.101:0/231904682 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3310005ad0 con 0x7f33201a1aa0 2021-02-10T11:31:27.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.217 7f3326bb6700 1 -- 172.21.2.101:0/231904682 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33200b8170 con 0x7f33201a1aa0 2021-02-10T11:31:27.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.217 7f3326bb6700 1 -- 172.21.2.101:0/231904682 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33201935e0 con 0x7f33201a1aa0 2021-02-10T11:31:27.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.218 7f331d7fa700 1 -- 172.21.2.101:0/231904682 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3310005850 con 0x7f33201a1aa0 2021-02-10T11:31:27.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.218 7f331d7fa700 1 --2- 172.21.2.101:0/231904682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f33040186c0 0x7f330401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:27.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.218 7f331d7fa700 1 -- 172.21.2.101:0/231904682 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f3310032590 con 0x7f33201a1aa0 2021-02-10T11:31:27.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.218 7f33249d2700 1 --2- 172.21.2.101:0/231904682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f33040186c0 0x7f330401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:27.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.218 7f33249d2700 1 --2- 172.21.2.101:0/231904682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f33040186c0 0x7f330401abc0 crc :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:27.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.249 7f3326bb6700 1 -- 172.21.2.101:0/231904682 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3320083da0 con 0x7f33201a1aa0 2021-02-10T11:31:27.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.252 7f331d7fa700 1 -- 172.21.2.101:0/231904682 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3310005a60 con 0x7f33201a1aa0 2021-02-10T11:31:27.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.384 7f3326bb6700 1 -- 172.21.2.101:0/231904682 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f332007ed80 con 0x7f33201a1aa0 2021-02-10T11:31:27.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.384 7f331d7fa700 1 -- 172.21.2.101:0/231904682 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+4513 (secure 0 0 0) 0x7f3310005850 con 0x7f33201a1aa0 2021-02-10T11:31:27.396 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:31:27.396 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:31:27.397 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:31:27.397 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:27.397 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:31:27.397 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:31:27.397 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:31:27.397 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:31:27.398 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:31:27.398 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:31:27.398 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:31:27.398 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:27.398 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:27.398 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:27.398 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:31:27.399 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:31:27.399 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:31:27.399 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:31:27.399 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:31:27.399 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:31:27.399 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:31:27.399 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:27.400 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:27.400 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:27.400 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:31:27.400 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:31:27.400 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:31:27.400 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:31:27.400 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:31:27.400 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:31:27.401 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:27.401 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:27.401 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:27.401 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:31:27.401 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:31:27.401 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:31:27.401 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:31:27.401 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:31:27.401 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:31:27.402 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:27.402 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:27.402 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:27.402 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:31:27.402 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:31:27.402 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:31:27.402 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:31:27.402 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:31:27.403 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:31:27.403 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:27.403 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:27.403 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:27.403 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:31:27.403 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:31:27.403 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:31:27.403 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:31:27.403 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:31:27.404 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:31:27.404 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:27.404 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:27.404 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:27.404 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:31:27.404 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:31:27.404 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:31:27.404 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:31:27.404 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:31:27.405 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:31:27.405 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:27.405 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:27.405 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:27.405 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:31:27.405 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:31:27.405 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:31:27.405 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:31:27.405 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:31:27.406 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:31:27.406 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:27.406 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:27.406 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:27.406 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:31:27.406 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:31:27.406 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:31:27.406 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:31:27.407 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:31:27.407 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:31:27.407 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:27.407 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:27.407 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:27.407 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:31:27.407 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:31:27.407 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:31:27.407 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:31:27.408 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:31:27.408 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:31:27.408 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:27.408 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:27.408 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:27.408 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:31:27.408 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:31:27.408 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:31:27.408 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:31:27.409 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:31:27.409 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:31:27.409 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:27.409 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:27.409 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:27.409 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:31:27.409 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:31:27.409 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:31:27.409 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:31:27.409 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:31:27.410 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:31:27.410 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:27.410 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:27.410 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:27.410 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:31:27.410 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:31:27.410 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:31:27.410 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:31:27.410 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:31:27.411 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:31:27.411 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:27.411 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:27.411 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:27.411 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:31:27.411 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:31:27.411 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:31:27.411 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:31:27.411 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:31:27.412 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:31:27.412 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:27.412 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:27.412 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:27.412 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:31:27.412 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:31:27.412 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:31:27.412 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:31:27.412 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:31:27.413 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:31:27.413 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:31:27.413 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:27.413 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:27.413 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:27.413 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:31:27.413 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:31:27.413 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:31:27.413 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:31:27.414 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:31:27.414 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:31:27.414 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:27.414 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:27.414 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:27.415 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:31:27.415 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:31:27.415 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:31:27.415 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:31:27.415 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:31:27.415 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:31:27.415 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:27.415 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:27.415 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:31:27.415 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:31:27.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.386 7f3326bb6700 1 -- 172.21.2.101:0/231904682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f33040186c0 msgr2=0x7f330401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:27.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.386 7f3326bb6700 1 --2- 172.21.2.101:0/231904682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f33040186c0 0x7f330401abc0 crc :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:27.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.386 7f3326bb6700 1 -- 172.21.2.101:0/231904682 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33201a1aa0 msgr2=0x7f33200b7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:27.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.386 7f3326bb6700 1 --2- 172.21.2.101:0/231904682 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33201a1aa0 0x7f33200b7530 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7f3310009960 tx=0x7f331000bcf0).stop 2021-02-10T11:31:27.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.387 7f3326bb6700 1 -- 172.21.2.101:0/231904682 shutdown_connections 2021-02-10T11:31:27.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.387 7f3326bb6700 1 --2- 172.21.2.101:0/231904682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f33040186c0 0x7f330401abc0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:27.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.387 7f3326bb6700 1 --2- 172.21.2.101:0/231904682 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3320192730 0x7f33200b7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:27.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.387 7f3326bb6700 1 --2- 172.21.2.101:0/231904682 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33201a1aa0 0x7f33200b7530 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:27.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.387 7f3326bb6700 1 --2- 172.21.2.101:0/231904682 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f33201aad40 0x7f3320192110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:27.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.387 7f3326bb6700 1 -- 172.21.2.101:0/231904682 >> 172.21.2.101:0/231904682 conn(0x7f33201a2470 msgr2=0x7f3320194f50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:27.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.387 7f3326bb6700 1 -- 172.21.2.101:0/231904682 shutdown_connections 2021-02-10T11:31:27.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.387 7f3326bb6700 1 -- 172.21.2.101:0/231904682 wait complete. 2021-02-10T11:31:27.419 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:31:27.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.856 7fe3e1bea700 1 Processor -- start 2021-02-10T11:31:27.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.856 7fe3e1bea700 1 -- start start 2021-02-10T11:31:27.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.857 7fe3e1bea700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe3dc07e6a0 0x7fe3dc0d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:27.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.857 7fe3e1bea700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3dc07ed80 con 0x7fe3dc0e5470 2021-02-10T11:31:27.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.857 7fe3e1bea700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3dc07eaa0 con 0x7fe3dc0e5070 2021-02-10T11:31:27.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.857 7fe3e1bea700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3dc1b4080 con 0x7fe3dc07e6a0 2021-02-10T11:31:27.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.857 7fe3daffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe3dc0e5470 0x7fe3dc07f300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50532/0 (socket says 172.21.2.101:50532) 2021-02-10T11:31:27.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.857 7fe3daffd700 1 -- 172.21.2.101:0/3395175305 learned_addr learned my addr 172.21.2.101:0/3395175305 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:27.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.857 7fe3da7fc700 1 -- 172.21.2.101:0/3395175305 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101225641 0 0) 0x7fe3cc0010b0 con 0x7fe3dc0e5470 2021-02-10T11:31:27.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.857 7fe3da7fc700 1 -- 172.21.2.101:0/3395175305 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3c8002fe0 con 0x7fe3dc0e5470 2021-02-10T11:31:27.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.857 7fe3dbfff700 1 --2- 172.21.2.101:0/3395175305 >> v2:172.21.2.108:3300/0 conn(0x7fe3dc07e6a0 0x7fe3dc0d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:27.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.857 7fe3da7fc700 1 -- 172.21.2.101:0/3395175305 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3627860753 0 0) 0x7fe3cc0010b0 con 0x7fe3dc0e5470 2021-02-10T11:31:27.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3da7fc700 1 -- 172.21.2.101:0/3395175305 >> v2:172.21.2.108:3300/0 conn(0x7fe3dc07e6a0 msgr2=0x7fe3dc0d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:27.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3da7fc700 1 --2- 172.21.2.101:0/3395175305 >> v2:172.21.2.108:3300/0 conn(0x7fe3dc07e6a0 0x7fe3dc0d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:27.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3da7fc700 1 -- 172.21.2.101:0/3395175305 >> v1:172.21.2.108:6789/0 conn(0x7fe3dc0e5070 legacy=0x7fe3dc1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:27.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3da7fc700 1 -- 172.21.2.101:0/3395175305 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3dc0d1db0 con 0x7fe3dc0e5470 2021-02-10T11:31:27.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3da7fc700 1 -- 172.21.2.101:0/3395175305 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fe3cc0015b0 con 0x7fe3dc0e5470 2021-02-10T11:31:27.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3da7fc700 1 -- 172.21.2.101:0/3395175305 >> v1:172.21.2.101:6789/0 conn(0x7fe3dc0e5470 legacy=0x7fe3dc07f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:27.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3da7fc700 1 --2- 172.21.2.101:0/3395175305 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe3c8003aa0 0x7fe3c8005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:27.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3da7fc700 1 --2- 172.21.2.101:0/3395175305 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3c8006780 0x7fe3c8008ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:27.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3da7fc700 1 --2- 172.21.2.101:0/3395175305 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3c80095a0 0x7fe3c800baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:27.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3da7fc700 1 -- 172.21.2.101:0/3395175305 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c800c2f0 con 0x7fe3c80095a0 2021-02-10T11:31:27.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3da7fc700 1 -- 172.21.2.101:0/3395175305 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c800c510 con 0x7fe3c8003aa0 2021-02-10T11:31:27.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3da7fc700 1 -- 172.21.2.101:0/3395175305 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c800c730 con 0x7fe3c8006780 2021-02-10T11:31:27.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3dbfff700 1 --2- 172.21.2.101:0/3395175305 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3c80095a0 0x7fe3c800baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:27.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3daffd700 1 --2- 172.21.2.101:0/3395175305 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3c8006780 0x7fe3c8008ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:27.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3db7fe700 1 --2- 172.21.2.101:0/3395175305 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe3c8003aa0 0x7fe3c8005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:27.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3dbfff700 1 -- 172.21.2.101:0/3395175305 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3c8006780 msgr2=0x7fe3c8008ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:27.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3dbfff700 1 --2- 172.21.2.101:0/3395175305 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3c8006780 0x7fe3c8008ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:27.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3dbfff700 1 -- 172.21.2.101:0/3395175305 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe3c8003aa0 msgr2=0x7fe3c8005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:27.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3dbfff700 1 --2- 172.21.2.101:0/3395175305 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe3c8003aa0 0x7fe3c8005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:27.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3dbfff700 1 -- 172.21.2.101:0/3395175305 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c800c9d0 con 0x7fe3c80095a0 2021-02-10T11:31:27.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3daffd700 1 --2- 172.21.2.101:0/3395175305 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3c8006780 0x7fe3c8008ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:31:27.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3dbfff700 1 -- 172.21.2.101:0/3395175305 reap_dead start 2021-02-10T11:31:27.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3dbfff700 1 --2- 172.21.2.101:0/3395175305 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3c80095a0 0x7fe3c800baa0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7fe3d404dc80 tx=0x7fe3d404f300).ready entity=mon.0 client_cookie=4aa4818841d75c79 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:27.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3da7fc700 1 -- 172.21.2.101:0/3395175305 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe3d4006cb0 con 0x7fe3c80095a0 2021-02-10T11:31:27.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3da7fc700 1 -- 172.21.2.101:0/3395175305 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d4003030 con 0x7fe3c80095a0 2021-02-10T11:31:27.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3da7fc700 1 -- 172.21.2.101:0/3395175305 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe3d404f8e0 con 0x7fe3c80095a0 2021-02-10T11:31:27.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3e1bea700 1 -- 172.21.2.101:0/3395175305 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3c80095a0 msgr2=0x7fe3c800baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:27.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.858 7fe3e1bea700 1 --2- 172.21.2.101:0/3395175305 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3c80095a0 0x7fe3c800baa0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7fe3d404dc80 tx=0x7fe3d404f300).stop 2021-02-10T11:31:27.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.859 7fe3e1bea700 1 -- 172.21.2.101:0/3395175305 shutdown_connections 2021-02-10T11:31:27.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.859 7fe3e1bea700 1 --2- 172.21.2.101:0/3395175305 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3c80095a0 0x7fe3c800baa0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:27.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.859 7fe3e1bea700 1 -- 172.21.2.101:0/3395175305 >> 172.21.2.101:0/3395175305 conn(0x7fe3dc0ce690 msgr2=0x7fe3dc0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:27.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.859 7fe3e1bea700 1 -- 172.21.2.101:0/3395175305 shutdown_connections 2021-02-10T11:31:27.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.859 7fe3e1bea700 1 -- 172.21.2.101:0/3395175305 wait complete. 2021-02-10T11:31:27.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.874 7fe3e1bea700 1 Processor -- start 2021-02-10T11:31:27.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.874 7fe3e1bea700 1 -- start start 2021-02-10T11:31:27.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.874 7fe3e1bea700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3dc194000 0x7fe3dc0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:27.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.874 7fe3e1bea700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3dc0b1330 0x7fe3dc087050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:27.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.874 7fe3e1bea700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe3dc087670 0x7fe3dc087a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:27.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.874 7fe3e1bea700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc09c0e0 con 0x7fe3dc194000 2021-02-10T11:31:27.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.874 7fe3e1bea700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc194440 con 0x7fe3dc087670 2021-02-10T11:31:27.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.874 7fe3e1bea700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc09c5e0 con 0x7fe3dc0b1330 2021-02-10T11:31:27.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.874 7fe3db7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3dc194000 0x7fe3dc0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:27.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.875 7fe3db7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3dc194000 0x7fe3dc0b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50050/0 (socket says 172.21.2.101:50050) 2021-02-10T11:31:27.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.875 7fe3db7fe700 1 -- 172.21.2.101:0/3623002464 learned_addr learned my addr 172.21.2.101:0/3623002464 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:27.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.875 7fe3db7fe700 1 -- 172.21.2.101:0/3623002464 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3dc0b1330 msgr2=0x7fe3dc087050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:27.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.875 7fe3db7fe700 1 --2- 172.21.2.101:0/3623002464 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3dc0b1330 0x7fe3dc087050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:27.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.875 7fe3db7fe700 1 -- 172.21.2.101:0/3623002464 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe3dc087670 msgr2=0x7fe3dc087a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:27.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.875 7fe3db7fe700 1 --2- 172.21.2.101:0/3623002464 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe3dc087670 0x7fe3dc087a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:27.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.875 7fe3db7fe700 1 -- 172.21.2.101:0/3623002464 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3dc09c7d0 con 0x7fe3dc194000 2021-02-10T11:31:27.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.875 7fe3db7fe700 1 --2- 172.21.2.101:0/3623002464 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3dc194000 0x7fe3dc0b0d10 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7fe3c4006990 tx=0x7fe3c400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:27.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.875 7fe3d3fff700 1 -- 172.21.2.101:0/3623002464 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe3c4006a80 con 0x7fe3dc194000 2021-02-10T11:31:27.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.875 7fe3d3fff700 1 -- 172.21.2.101:0/3623002464 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c4002030 con 0x7fe3dc194000 2021-02-10T11:31:27.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.875 7fe3d3fff700 1 -- 172.21.2.101:0/3623002464 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe3c4002550 con 0x7fe3dc194000 2021-02-10T11:31:27.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.875 7fe3e1bea700 1 -- 172.21.2.101:0/3623002464 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3dc09c7d0 con 0x7fe3dc194000 2021-02-10T11:31:27.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.875 7fe3e1bea700 1 -- 172.21.2.101:0/3623002464 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3dc09cd60 con 0x7fe3dc194000 2021-02-10T11:31:27.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.876 7fe3d3fff700 1 -- 172.21.2.101:0/3623002464 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe3c4006a80 con 0x7fe3dc194000 2021-02-10T11:31:27.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.876 7fe3d3fff700 1 --2- 172.21.2.101:0/3623002464 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3c00186c0 0x7fe3c001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:27.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.876 7fe3d3fff700 1 -- 172.21.2.101:0/3623002464 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fe3c4032180 con 0x7fe3dc194000 2021-02-10T11:31:27.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.876 7fe3daffd700 1 --2- 172.21.2.101:0/3623002464 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3c00186c0 0x7fe3c001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:27.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.876 7fe3daffd700 1 --2- 172.21.2.101:0/3623002464 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3c00186c0 0x7fe3c001abc0 crc :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:27.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.907 7fe3e1bea700 1 -- 172.21.2.101:0/3623002464 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3dc084b20 con 0x7fe3dc194000 2021-02-10T11:31:27.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:27.909 7fe3d3fff700 1 -- 172.21.2.101:0/3623002464 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe3c4030de0 con 0x7fe3dc194000 2021-02-10T11:31:28.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.036 7fe3e1bea700 1 -- 172.21.2.101:0/3623002464 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3dc07ed80 con 0x7fe3dc194000 2021-02-10T11:31:28.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.037 7fe3d3fff700 1 -- 172.21.2.101:0/3623002464 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+11144 (secure 0 0 0) 0x7fe3c4006a80 con 0x7fe3dc194000 2021-02-10T11:31:28.048 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:31:28.048 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":33,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:31:26.838143","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:31:28.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.038 7fe3e1bea700 1 -- 172.21.2.101:0/3623002464 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3c00186c0 msgr2=0x7fe3c001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:28.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.038 7fe3e1bea700 1 --2- 172.21.2.101:0/3623002464 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3c00186c0 0x7fe3c001abc0 crc :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:28.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.038 7fe3e1bea700 1 -- 172.21.2.101:0/3623002464 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3dc194000 msgr2=0x7fe3dc0b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:28.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.038 7fe3e1bea700 1 --2- 172.21.2.101:0/3623002464 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3dc194000 0x7fe3dc0b0d10 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7fe3c4006990 tx=0x7fe3c400b990).stop 2021-02-10T11:31:28.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.039 7fe3e1bea700 1 -- 172.21.2.101:0/3623002464 shutdown_connections 2021-02-10T11:31:28.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.039 7fe3e1bea700 1 --2- 172.21.2.101:0/3623002464 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3c00186c0 0x7fe3c001abc0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:28.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.039 7fe3e1bea700 1 --2- 172.21.2.101:0/3623002464 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3dc0b1330 0x7fe3dc087050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:28.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.039 7fe3e1bea700 1 --2- 172.21.2.101:0/3623002464 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe3dc087670 0x7fe3dc087a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:28.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.039 7fe3e1bea700 1 --2- 172.21.2.101:0/3623002464 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3dc194000 0x7fe3dc0b0d10 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:28.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.039 7fe3e1bea700 1 -- 172.21.2.101:0/3623002464 >> 172.21.2.101:0/3623002464 conn(0x7fe3dc092c20 msgr2=0x7fe3dc195070 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:28.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.039 7fe3e1bea700 1 -- 172.21.2.101:0/3623002464 shutdown_connections 2021-02-10T11:31:28.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.039 7fe3e1bea700 1 -- 172.21.2.101:0/3623002464 wait complete. 2021-02-10T11:31:28.058 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-10T11:31:28.059 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:31:28.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.462 7fe7fa188700 1 Processor -- start 2021-02-10T11:31:28.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.462 7fe7fa188700 1 -- start start 2021-02-10T11:31:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.462 7fe7fa188700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7f41b0470 0x7fe7f41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.462 7fe7fa188700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe7f40ef610 0x7fe7f40efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.462 7fe7fa188700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7f40f0b50 con 0x7fe7f40f0030 2021-02-10T11:31:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.462 7fe7fa188700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f40e52e0 con 0x7fe7f40ef610 2021-02-10T11:31:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.462 7fe7fa188700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f40e5490 con 0x7fe7f41b0470 2021-02-10T11:31:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.463 7fe7f2ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe7f40ef610 0x7fe7f40efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.463 7fe7f2ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe7f40ef610 0x7fe7f40efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50058/0 (socket says 172.21.2.101:50058) 2021-02-10T11:31:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.463 7fe7f2ffd700 1 -- 172.21.2.101:0/1705521905 learned_addr learned my addr 172.21.2.101:0/1705521905 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.463 7fe7f37fe700 1 --2- 172.21.2.101:0/1705521905 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7f41b0470 0x7fe7f41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.463 7fe7f2ffd700 1 -- 172.21.2.101:0/1705521905 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7f41b0470 msgr2=0x7fe7f41b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.463 7fe7f2ffd700 1 --2- 172.21.2.101:0/1705521905 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7f41b0470 0x7fe7f41b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:28.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.463 7fe7f2ffd700 1 -- 172.21.2.101:0/1705521905 >> v1:172.21.2.108:6789/0 conn(0x7fe7f40f0030 legacy=0x7fe7f40f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:28.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.463 7fe7f2ffd700 1 -- 172.21.2.101:0/1705521905 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f40f0430 con 0x7fe7f40ef610 2021-02-10T11:31:28.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.463 7fe7f37fe700 1 --2- 172.21.2.101:0/1705521905 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7f41b0470 0x7fe7f41b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:31:28.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.463 7fe7f2ffd700 1 --2- 172.21.2.101:0/1705521905 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7f40ef610 0x7fe7f40efa10 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7fe7e4006700 tx=0x7fe7e4009b80).ready entity=mon.0 client_cookie=a7a33343f86b56cd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:28.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.463 7fe7f27fc700 1 -- 172.21.2.101:0/1705521905 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe7e4000c90 con 0x7fe7f40ef610 2021-02-10T11:31:28.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.463 7fe7f27fc700 1 -- 172.21.2.101:0/1705521905 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e400c0c0 con 0x7fe7f40ef610 2021-02-10T11:31:28.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.463 7fe7f27fc700 1 -- 172.21.2.101:0/1705521905 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe7e40009b0 con 0x7fe7f40ef610 2021-02-10T11:31:28.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.464 7fe7fa188700 1 -- 172.21.2.101:0/1705521905 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7f40ef610 msgr2=0x7fe7f40efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:28.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.464 7fe7fa188700 1 --2- 172.21.2.101:0/1705521905 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7f40ef610 0x7fe7f40efa10 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7fe7e4006700 tx=0x7fe7e4009b80).stop 2021-02-10T11:31:28.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.464 7fe7fa188700 1 -- 172.21.2.101:0/1705521905 shutdown_connections 2021-02-10T11:31:28.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.464 7fe7fa188700 1 --2- 172.21.2.101:0/1705521905 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7f40ef610 0x7fe7f40efa10 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:28.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.464 7fe7fa188700 1 --2- 172.21.2.101:0/1705521905 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7f41b0470 0x7fe7f41b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:28.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.464 7fe7fa188700 1 -- 172.21.2.101:0/1705521905 >> 172.21.2.101:0/1705521905 conn(0x7fe7f40ce3e0 msgr2=0x7fe7f40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:28.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.464 7fe7fa188700 1 -- 172.21.2.101:0/1705521905 shutdown_connections 2021-02-10T11:31:28.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.464 7fe7fa188700 1 -- 172.21.2.101:0/1705521905 wait complete. 2021-02-10T11:31:28.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.479 7fe7fa188700 1 Processor -- start 2021-02-10T11:31:28.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.479 7fe7fa188700 1 -- start start 2021-02-10T11:31:28.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.479 7fe7fa188700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe7f4190920 0x7fe7f408f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:28.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.479 7fe7fa188700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7f4190d20 0x7fe7f408fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:28.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.479 7fe7fa188700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7f4193270 0x7fe7f4193670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:28.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.479 7fe7fa188700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f4090470 con 0x7fe7f4190d20 2021-02-10T11:31:28.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.479 7fe7fa188700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f4193c90 con 0x7fe7f4190920 2021-02-10T11:31:28.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.479 7fe7fa188700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f4193e70 con 0x7fe7f4193270 2021-02-10T11:31:28.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.480 7fe7f2ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7f4190d20 0x7fe7f408fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:28.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.480 7fe7f2ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7f4190d20 0x7fe7f408fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50064/0 (socket says 172.21.2.101:50064) 2021-02-10T11:31:28.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.480 7fe7f2ffd700 1 -- 172.21.2.101:0/19228428 learned_addr learned my addr 172.21.2.101:0/19228428 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:28.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.480 7fe7f3fff700 1 --2- 172.21.2.101:0/19228428 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7f4193270 0x7fe7f4193670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:28.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.480 7fe7f37fe700 1 --2- 172.21.2.101:0/19228428 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe7f4190920 0x7fe7f408f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:28.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.480 7fe7f2ffd700 1 -- 172.21.2.101:0/19228428 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7f4193270 msgr2=0x7fe7f4193670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.480 7fe7f2ffd700 1 --2- 172.21.2.101:0/19228428 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7f4193270 0x7fe7f4193670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.480 7fe7f2ffd700 1 -- 172.21.2.101:0/19228428 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe7f4190920 msgr2=0x7fe7f408f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.480 7fe7f2ffd700 1 --2- 172.21.2.101:0/19228428 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe7f4190920 0x7fe7f408f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.480 7fe7f2ffd700 1 -- 172.21.2.101:0/19228428 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f4194090 con 0x7fe7f4190d20 2021-02-10T11:31:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.480 7fe7f2ffd700 1 --2- 172.21.2.101:0/19228428 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7f4190d20 0x7fe7f408fc20 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7fe7e4006f00 tx=0x7fe7e400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.480 7fe7ebfff700 1 -- 172.21.2.101:0/19228428 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe7e400bd80 con 0x7fe7f4190d20 2021-02-10T11:31:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.481 7fe7ebfff700 1 -- 172.21.2.101:0/19228428 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e4002030 con 0x7fe7f4190d20 2021-02-10T11:31:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.481 7fe7ebfff700 1 -- 172.21.2.101:0/19228428 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe7e4002510 con 0x7fe7f4190d20 2021-02-10T11:31:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.481 7fe7fa188700 1 -- 172.21.2.101:0/19228428 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f4194090 con 0x7fe7f4190d20 2021-02-10T11:31:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.481 7fe7fa188700 1 -- 172.21.2.101:0/19228428 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f41920b0 con 0x7fe7f4190d20 2021-02-10T11:31:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.481 7fe7ebfff700 1 -- 172.21.2.101:0/19228428 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe7e4000c90 con 0x7fe7f4190d20 2021-02-10T11:31:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.481 7fe7ebfff700 1 --2- 172.21.2.101:0/19228428 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7d80186c0 0x7fe7d801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.481 7fe7ebfff700 1 -- 172.21.2.101:0/19228428 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fe7e4032110 con 0x7fe7f4190d20 2021-02-10T11:31:28.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.481 7fe7f37fe700 1 --2- 172.21.2.101:0/19228428 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7d80186c0 0x7fe7d801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:28.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.482 7fe7f37fe700 1 --2- 172.21.2.101:0/19228428 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7d80186c0 0x7fe7d801abc0 crc :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:28.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.512 7fe7fa188700 1 -- 172.21.2.101:0/19228428 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f4105af0 con 0x7fe7f4190d20 2021-02-10T11:31:28.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.515 7fe7ebfff700 1 -- 172.21.2.101:0/19228428 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe7e4030d70 con 0x7fe7f4190d20 2021-02-10T11:31:28.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.641 7fe7fa188700 1 -- 172.21.2.101:0/19228428 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fe7f40f0b50 con 0x7fe7f4190d20 2021-02-10T11:31:28.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.642 7fe7ebfff700 1 -- 172.21.2.101:0/19228428 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v33) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe7e4000c90 con 0x7fe7f4190d20 2021-02-10T11:31:28.653 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:31:28.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.644 7fe7fa188700 1 -- 172.21.2.101:0/19228428 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7d80186c0 msgr2=0x7fe7d801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:28.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.644 7fe7fa188700 1 --2- 172.21.2.101:0/19228428 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7d80186c0 0x7fe7d801abc0 crc :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:28.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.644 7fe7fa188700 1 -- 172.21.2.101:0/19228428 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7f4190d20 msgr2=0x7fe7f408fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:28.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.644 7fe7fa188700 1 --2- 172.21.2.101:0/19228428 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7f4190d20 0x7fe7f408fc20 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7fe7e4006f00 tx=0x7fe7e400b990).stop 2021-02-10T11:31:28.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.644 7fe7fa188700 1 -- 172.21.2.101:0/19228428 shutdown_connections 2021-02-10T11:31:28.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.644 7fe7fa188700 1 --2- 172.21.2.101:0/19228428 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7d80186c0 0x7fe7d801abc0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:28.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.644 7fe7fa188700 1 --2- 172.21.2.101:0/19228428 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7f4193270 0x7fe7f4193670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:28.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.644 7fe7fa188700 1 --2- 172.21.2.101:0/19228428 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7f4190d20 0x7fe7f408fc20 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:28.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.644 7fe7fa188700 1 --2- 172.21.2.101:0/19228428 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe7f4190920 0x7fe7f408f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:28.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.644 7fe7fa188700 1 -- 172.21.2.101:0/19228428 >> 172.21.2.101:0/19228428 conn(0x7fe7f40bf4e0 msgr2=0x7fe7f40be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:28.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.644 7fe7fa188700 1 -- 172.21.2.101:0/19228428 shutdown_connections 2021-02-10T11:31:28.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:28.644 7fe7fa188700 1 -- 172.21.2.101:0/19228428 wait complete. 2021-02-10T11:31:28.668 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --format=json-pretty osd lspools 2021-02-10T11:31:29.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.065 7f79a0b40700 1 Processor -- start 2021-02-10T11:31:29.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.065 7f79a0b40700 1 -- start start 2021-02-10T11:31:29.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.066 7f79a0b40700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f799c0e5070 0x7f799c1b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:29.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.066 7f79a0b40700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f799c1b3e40 0x7f799c07d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:29.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.066 7f79a0b40700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f799c07ed80 con 0x7f799c0e5470 2021-02-10T11:31:29.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.066 7f79a0b40700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f799c080310 con 0x7f799c0e5070 2021-02-10T11:31:29.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.066 7f79a0b40700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799c080500 con 0x7f799c1b3e40 2021-02-10T11:31:29.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.066 7f799a85e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f799c0e5070 0x7f799c1b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:29.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.066 7f799a85e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f799c0e5070 0x7f799c1b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50070/0 (socket says 172.21.2.101:50070) 2021-02-10T11:31:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.066 7f799a85e700 1 -- 172.21.2.101:0/2576971818 learned_addr learned my addr 172.21.2.101:0/2576971818 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.066 7f799a85e700 1 -- 172.21.2.101:0/2576971818 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f799c1b3e40 msgr2=0x7f799c07d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.066 7f799a85e700 1 --2- 172.21.2.101:0/2576971818 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f799c1b3e40 0x7f799c07d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.066 7f799a85e700 1 -- 172.21.2.101:0/2576971818 >> v1:172.21.2.108:6789/0 conn(0x7f799c0e5470 legacy=0x7f799c07dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.066 7f799a85e700 1 -- 172.21.2.101:0/2576971818 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c0806f0 con 0x7f799c0e5070 2021-02-10T11:31:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.067 7f799a85e700 1 --2- 172.21.2.101:0/2576971818 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f799c0e5070 0x7f799c1b3820 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f798c006700 tx=0x7f798c009b80).ready entity=mon.0 client_cookie=41a17116158a3c47 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.067 7f799985c700 1 -- 172.21.2.101:0/2576971818 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f798c00c0c0 con 0x7f799c0e5070 2021-02-10T11:31:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.067 7f799985c700 1 -- 172.21.2.101:0/2576971818 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798c00c440 con 0x7f799c0e5070 2021-02-10T11:31:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.067 7f799985c700 1 -- 172.21.2.101:0/2576971818 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f798c00c960 con 0x7f799c0e5070 2021-02-10T11:31:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.067 7f79a0b40700 1 -- 172.21.2.101:0/2576971818 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f799c0e5070 msgr2=0x7f799c1b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.067 7f79a0b40700 1 --2- 172.21.2.101:0/2576971818 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f799c0e5070 0x7f799c1b3820 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f798c006700 tx=0x7f798c009b80).stop 2021-02-10T11:31:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.068 7f79a0b40700 1 -- 172.21.2.101:0/2576971818 shutdown_connections 2021-02-10T11:31:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.068 7f79a0b40700 1 --2- 172.21.2.101:0/2576971818 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f799c1b3e40 0x7f799c07d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.068 7f79a0b40700 1 --2- 172.21.2.101:0/2576971818 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f799c0e5070 0x7f799c1b3820 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.068 7f79a0b40700 1 -- 172.21.2.101:0/2576971818 >> 172.21.2.101:0/2576971818 conn(0x7f799c0ce690 msgr2=0x7f799c0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.068 7f79a0b40700 1 -- 172.21.2.101:0/2576971818 shutdown_connections 2021-02-10T11:31:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.068 7f79a0b40700 1 -- 172.21.2.101:0/2576971818 wait complete. 2021-02-10T11:31:29.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.082 7f79a0b40700 1 Processor -- start 2021-02-10T11:31:29.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.083 7f79a0b40700 1 -- start start 2021-02-10T11:31:29.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.083 7f79a0b40700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f799c1a2a90 0x7f799c1aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:29.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.083 7f79a0b40700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f799c1aa700 0x7f799c1a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:29.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.083 7f79a0b40700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f799c0a55e0 0x7f799c1a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.083 7f79a0b40700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c1a34a0 con 0x7f799c1aa700 2021-02-10T11:31:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.083 7f79a0b40700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c1a2e90 con 0x7f799c1a2a90 2021-02-10T11:31:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.083 7f79a0b40700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799c1a30b0 con 0x7f799c0a55e0 2021-02-10T11:31:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.083 7f799a05d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f799c1aa700 0x7f799c1a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.083 7f799a05d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f799c1aa700 0x7f799c1a3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50078/0 (socket says 172.21.2.101:50078) 2021-02-10T11:31:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.083 7f799a05d700 1 -- 172.21.2.101:0/3255052605 learned_addr learned my addr 172.21.2.101:0/3255052605 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.084 7f799a05d700 1 -- 172.21.2.101:0/3255052605 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f799c0a55e0 msgr2=0x7f799c1a4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.084 7f799a05d700 1 --2- 172.21.2.101:0/3255052605 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f799c0a55e0 0x7f799c1a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.084 7f799a05d700 1 -- 172.21.2.101:0/3255052605 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f799c1a2a90 msgr2=0x7f799c1aa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.084 7f799a05d700 1 --2- 172.21.2.101:0/3255052605 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f799c1a2a90 0x7f799c1aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.084 7f799a05d700 1 -- 172.21.2.101:0/3255052605 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c1a4660 con 0x7f799c1aa700 2021-02-10T11:31:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.084 7f799a05d700 1 --2- 172.21.2.101:0/3255052605 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f799c1aa700 0x7f799c1a3a20 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7f7990006f00 tx=0x7f799000b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.084 7f798b7fe700 1 -- 172.21.2.101:0/3255052605 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f799000bd30 con 0x7f799c1aa700 2021-02-10T11:31:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.084 7f798b7fe700 1 -- 172.21.2.101:0/3255052605 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7990000c90 con 0x7f799c1aa700 2021-02-10T11:31:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.084 7f798b7fe700 1 -- 172.21.2.101:0/3255052605 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f79900022a0 con 0x7f799c1aa700 2021-02-10T11:31:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.084 7f79a0b40700 1 -- 172.21.2.101:0/3255052605 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f799c1a4660 con 0x7f799c1aa700 2021-02-10T11:31:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.084 7f79a0b40700 1 -- 172.21.2.101:0/3255052605 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f799c1a4660 con 0x7f799c1aa700 2021-02-10T11:31:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.085 7f798b7fe700 1 -- 172.21.2.101:0/3255052605 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7990005310 con 0x7f799c1aa700 2021-02-10T11:31:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.085 7f798b7fe700 1 --2- 172.21.2.101:0/3255052605 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79800186c0 0x7f798001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.085 7f798b7fe700 1 -- 172.21.2.101:0/3255052605 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f7990032450 con 0x7f799c1aa700 2021-02-10T11:31:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.085 7f799a85e700 1 --2- 172.21.2.101:0/3255052605 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79800186c0 0x7f798001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.086 7f799a85e700 1 --2- 172.21.2.101:0/3255052605 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79800186c0 0x7f798001abc0 crc :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:29.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.116 7f79a0b40700 1 -- 172.21.2.101:0/3255052605 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f799c083da0 con 0x7f799c1aa700 2021-02-10T11:31:29.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.119 7f798b7fe700 1 -- 172.21.2.101:0/3255052605 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7990006b90 con 0x7f799c1aa700 2021-02-10T11:31:29.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.242 7f79a0b40700 1 -- 172.21.2.101:0/3255052605 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f799c07ed80 con 0x7f799c1aa700 2021-02-10T11:31:29.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.243 7f798b7fe700 1 -- 172.21.2.101:0/3255052605 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v33) v1 ==== 84+0+64 (secure 0 0 0) 0x7f799000bd30 con 0x7f799c1aa700 2021-02-10T11:31:29.254 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:31:29.254 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:31:29.254 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:29.254 INFO:teuthology.orchestra.run.gibba001.stdout: "poolnum": 1, 2021-02-10T11:31:29.254 INFO:teuthology.orchestra.run.gibba001.stdout: "poolname": "rbd" 2021-02-10T11:31:29.255 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:29.255 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:31:29.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.245 7f79a0b40700 1 -- 172.21.2.101:0/3255052605 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79800186c0 msgr2=0x7f798001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:29.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.245 7f79a0b40700 1 --2- 172.21.2.101:0/3255052605 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79800186c0 0x7f798001abc0 crc :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:29.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.245 7f79a0b40700 1 -- 172.21.2.101:0/3255052605 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f799c1aa700 msgr2=0x7f799c1a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:29.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.245 7f79a0b40700 1 --2- 172.21.2.101:0/3255052605 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f799c1aa700 0x7f799c1a3a20 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7f7990006f00 tx=0x7f799000b940).stop 2021-02-10T11:31:29.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.245 7f79a0b40700 1 -- 172.21.2.101:0/3255052605 shutdown_connections 2021-02-10T11:31:29.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.245 7f79a0b40700 1 --2- 172.21.2.101:0/3255052605 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79800186c0 0x7f798001abc0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:29.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.245 7f79a0b40700 1 --2- 172.21.2.101:0/3255052605 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f799c0a55e0 0x7f799c1a4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:29.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.245 7f79a0b40700 1 --2- 172.21.2.101:0/3255052605 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f799c1aa700 0x7f799c1a3a20 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:29.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.245 7f79a0b40700 1 --2- 172.21.2.101:0/3255052605 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f799c1a2a90 0x7f799c1aa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:29.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.245 7f79a0b40700 1 -- 172.21.2.101:0/3255052605 >> 172.21.2.101:0/3255052605 conn(0x7f799c08fe10 msgr2=0x7f799c19c020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:29.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.245 7f79a0b40700 1 -- 172.21.2.101:0/3255052605 shutdown_connections 2021-02-10T11:31:29.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.245 7f79a0b40700 1 -- 172.21.2.101:0/3255052605 wait complete. 2021-02-10T11:31:29.264 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-02-10T11:31:29.264 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-02-10T11:31:29.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.667 7f7712dc1700 1 Processor -- start 2021-02-10T11:31:29.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.667 7f7712dc1700 1 -- start start 2021-02-10T11:31:29.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.668 7f7712dc1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f770c1b0470 0x7f770c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:29.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.668 7f7712dc1700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f770c0f0b50 con 0x7f770c0f20e0 2021-02-10T11:31:29.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.668 7f7712dc1700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f770c0f24e0 con 0x7f770c0ef610 2021-02-10T11:31:29.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.668 7f7712dc1700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f770c0e5ea0 con 0x7f770c1b0470 2021-02-10T11:31:29.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.668 7f77113de700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f770c0f20e0 0x7f770c0f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50578/0 (socket says 172.21.2.101:50578) 2021-02-10T11:31:29.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.668 7f77113de700 1 -- 172.21.2.101:0/3943744932 learned_addr learned my addr 172.21.2.101:0/3943744932 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:29.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.668 7f7710bdd700 1 --2- 172.21.2.101:0/3943744932 >> v2:172.21.2.101:3300/0 conn(0x7f770c1b0470 0x7f770c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:29.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.668 7f770b7fe700 1 -- 172.21.2.101:0/3943744932 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2527922381 0 0) 0x7f7704001180 con 0x7f770c0f20e0 2021-02-10T11:31:29.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.668 7f7710bdd700 1 -- 172.21.2.101:0/3943744932 >> v1:172.21.2.108:6789/0 conn(0x7f770c0ef610 legacy=0x7f770c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:29.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.668 7f7710bdd700 1 -- 172.21.2.101:0/3943744932 >> v1:172.21.2.101:6789/0 conn(0x7f770c0f20e0 legacy=0x7f770c0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:29.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.668 7f7710bdd700 1 -- 172.21.2.101:0/3943744932 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770c0e52e0 con 0x7f770c1b0470 2021-02-10T11:31:29.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.669 7f7710bdd700 1 --2- 172.21.2.101:0/3943744932 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f770c1b0470 0x7f770c1b0870 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7f76f4006650 tx=0x7f76f4009a80).ready entity=mon.0 client_cookie=587e030a0ffe6d8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:29.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.669 7f770b7fe700 1 -- 172.21.2.101:0/3943744932 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f76f40009b0 con 0x7f770c1b0470 2021-02-10T11:31:29.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.669 7f770b7fe700 1 -- 172.21.2.101:0/3943744932 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f400c0c0 con 0x7f770c1b0470 2021-02-10T11:31:29.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.669 7f770b7fe700 1 -- 172.21.2.101:0/3943744932 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f76f400c5e0 con 0x7f770c1b0470 2021-02-10T11:31:29.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.669 7f7712dc1700 1 -- 172.21.2.101:0/3943744932 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f770c1b0470 msgr2=0x7f770c1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:29.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.669 7f7712dc1700 1 --2- 172.21.2.101:0/3943744932 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f770c1b0470 0x7f770c1b0870 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7f76f4006650 tx=0x7f76f4009a80).stop 2021-02-10T11:31:29.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.670 7f7712dc1700 1 -- 172.21.2.101:0/3943744932 shutdown_connections 2021-02-10T11:31:29.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.670 7f7712dc1700 1 --2- 172.21.2.101:0/3943744932 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f770c1b0470 0x7f770c1b0870 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:29.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.670 7f7712dc1700 1 -- 172.21.2.101:0/3943744932 >> 172.21.2.101:0/3943744932 conn(0x7f770c0ce3e0 msgr2=0x7f770c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:29.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.670 7f7712dc1700 1 -- 172.21.2.101:0/3943744932 shutdown_connections 2021-02-10T11:31:29.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.670 7f7712dc1700 1 -- 172.21.2.101:0/3943744932 wait complete. 2021-02-10T11:31:29.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.684 7f7712dc1700 1 Processor -- start 2021-02-10T11:31:29.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.684 7f7712dc1700 1 -- start start 2021-02-10T11:31:29.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.685 7f7712dc1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f770c19c0f0 0x7f770c1908a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:29.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.685 7f7712dc1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f770c0a4590 0x7f770c0f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:29.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.685 7f7712dc1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f770c0f25f0 0x7f770c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:29.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.685 7f7712dc1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c0a4b80 con 0x7f770c19c0f0 2021-02-10T11:31:29.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.685 7f7712dc1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c0f29f0 con 0x7f770c0f25f0 2021-02-10T11:31:29.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.685 7f7712dc1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f770c0a20c0 con 0x7f770c0a4590 2021-02-10T11:31:29.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.685 7f7710bdd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f770c19c0f0 0x7f770c1908a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:29.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.685 7f7710bdd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f770c19c0f0 0x7f770c1908a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50088/0 (socket says 172.21.2.101:50088) 2021-02-10T11:31:29.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.685 7f7710bdd700 1 -- 172.21.2.101:0/3623300693 learned_addr learned my addr 172.21.2.101:0/3623300693 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:29.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.686 7f7710bdd700 1 -- 172.21.2.101:0/3623300693 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f770c0a4590 msgr2=0x7f770c0f1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:29.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.686 7f7710bdd700 1 --2- 172.21.2.101:0/3623300693 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f770c0a4590 0x7f770c0f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:29.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.686 7f7710bdd700 1 -- 172.21.2.101:0/3623300693 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f770c0f25f0 msgr2=0x7f770c0a5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:29.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.686 7f7710bdd700 1 --2- 172.21.2.101:0/3623300693 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f770c0f25f0 0x7f770c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:29.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.686 7f7710bdd700 1 -- 172.21.2.101:0/3623300693 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770c0a5720 con 0x7f770c19c0f0 2021-02-10T11:31:29.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.686 7f7710bdd700 1 --2- 172.21.2.101:0/3623300693 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f770c19c0f0 0x7f770c1908a0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7f76f4009630 tx=0x7f76f400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:29.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.686 7f77097fa700 1 -- 172.21.2.101:0/3623300693 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f76f40093a0 con 0x7f770c19c0f0 2021-02-10T11:31:29.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.686 7f77097fa700 1 -- 172.21.2.101:0/3623300693 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f400bd70 con 0x7f770c19c0f0 2021-02-10T11:31:29.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.686 7f77097fa700 1 -- 172.21.2.101:0/3623300693 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f76f4005560 con 0x7f770c19c0f0 2021-02-10T11:31:29.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.686 7f7712dc1700 1 -- 172.21.2.101:0/3623300693 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f770c0a5720 con 0x7f770c19c0f0 2021-02-10T11:31:29.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.686 7f7712dc1700 1 -- 172.21.2.101:0/3623300693 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770c0a5720 con 0x7f770c19c0f0 2021-02-10T11:31:29.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.687 7f77097fa700 1 -- 172.21.2.101:0/3623300693 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f76f400a000 con 0x7f770c19c0f0 2021-02-10T11:31:29.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.687 7f77097fa700 1 --2- 172.21.2.101:0/3623300693 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76f00186d0 0x7f76f001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:29.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.687 7f77097fa700 1 -- 172.21.2.101:0/3623300693 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f76f4032480 con 0x7f770c19c0f0 2021-02-10T11:31:29.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.687 7f770bfff700 1 --2- 172.21.2.101:0/3623300693 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76f00186d0 0x7f76f001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:29.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.687 7f770bfff700 1 --2- 172.21.2.101:0/3623300693 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76f00186d0 0x7f76f001abd0 crc :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:29.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.717 7f7712dc1700 1 -- 172.21.2.101:0/3623300693 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f770c0f29f0 con 0x7f770c19c0f0 2021-02-10T11:31:29.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.720 7f77097fa700 1 -- 172.21.2.101:0/3623300693 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f76f401f4f0 con 0x7f770c19c0f0 2021-02-10T11:31:29.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:29.847 7f7712dc1700 1 -- 172.21.2.101:0/3623300693 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"} v 0) v1 -- 0x7f770c0f29f0 con 0x7f770c19c0f0 2021-02-10T11:31:30.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.309 7f77097fa700 1 -- 172.21.2.101:0/3623300693 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"}]=0 pool 'cephfs_metadata' created v34) v1 ==== 133+0+0 (secure 0 0 0) 0x7f76f400a000 con 0x7f770c19c0f0 2021-02-10T11:31:30.325 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' created 2021-02-10T11:31:30.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.316 7f7712dc1700 1 -- 172.21.2.101:0/3623300693 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76f00186d0 msgr2=0x7f76f001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:30.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.316 7f7712dc1700 1 --2- 172.21.2.101:0/3623300693 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76f00186d0 0x7f76f001abd0 crc :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:30.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.316 7f7712dc1700 1 -- 172.21.2.101:0/3623300693 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f770c19c0f0 msgr2=0x7f770c1908a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:30.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.316 7f7712dc1700 1 --2- 172.21.2.101:0/3623300693 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f770c19c0f0 0x7f770c1908a0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7f76f4009630 tx=0x7f76f400b980).stop 2021-02-10T11:31:30.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.316 7f7712dc1700 1 -- 172.21.2.101:0/3623300693 shutdown_connections 2021-02-10T11:31:30.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.316 7f7712dc1700 1 --2- 172.21.2.101:0/3623300693 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76f00186d0 0x7f76f001abd0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:30.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.316 7f7712dc1700 1 --2- 172.21.2.101:0/3623300693 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f770c0a4590 0x7f770c0f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:30.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.316 7f7712dc1700 1 --2- 172.21.2.101:0/3623300693 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f770c0f25f0 0x7f770c0a5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:30.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.316 7f7712dc1700 1 --2- 172.21.2.101:0/3623300693 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f770c19c0f0 0x7f770c1908a0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:30.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.316 7f7712dc1700 1 -- 172.21.2.101:0/3623300693 >> 172.21.2.101:0/3623300693 conn(0x7f770c0958c0 msgr2=0x7f770c0bf040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:30.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.316 7f7712dc1700 1 -- 172.21.2.101:0/3623300693 shutdown_connections 2021-02-10T11:31:30.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.316 7f7712dc1700 1 -- 172.21.2.101:0/3623300693 wait complete. 2021-02-10T11:31:30.335 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_data 8 2021-02-10T11:31:30.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.736 7fc4195de700 1 Processor -- start 2021-02-10T11:31:30.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.736 7fc4195de700 1 -- start start 2021-02-10T11:31:30.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.736 7fc4195de700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc4141b0470 0x7fc4141b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:30.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.736 7fc4195de700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc4140f20e0 0x7fc4140f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:30.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.736 7fc4195de700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4140f0b50 con 0x7fc4140ef610 2021-02-10T11:31:30.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.736 7fc4195de700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4140f0820 con 0x7fc4141b0470 2021-02-10T11:31:30.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.736 7fc4195de700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4140f24e0 con 0x7fc4140f20e0 2021-02-10T11:31:30.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.737 7fc4132bf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc4141b0470 0x7fc4141b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:30.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.737 7fc412abe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc4140ef610 0x7fc4140efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50590/0 (socket says 172.21.2.101:50590) 2021-02-10T11:31:30.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.737 7fc412abe700 1 -- 172.21.2.101:0/484236672 learned_addr learned my addr 172.21.2.101:0/484236672 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:30.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.737 7fc4132bf700 1 -- 172.21.2.101:0/484236672 >> v2:172.21.2.108:3300/0 conn(0x7fc4140f20e0 msgr2=0x7fc4140f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:30.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.737 7fc4132bf700 1 --2- 172.21.2.101:0/484236672 >> v2:172.21.2.108:3300/0 conn(0x7fc4140f20e0 0x7fc4140f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:30.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.737 7fc4132bf700 1 -- 172.21.2.101:0/484236672 >> v1:172.21.2.101:6789/0 conn(0x7fc4140ef610 legacy=0x7fc4140efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:30.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.737 7fc4132bf700 1 -- 172.21.2.101:0/484236672 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4140e52e0 con 0x7fc4141b0470 2021-02-10T11:31:30.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.737 7fc4122bd700 1 -- 172.21.2.101:0/484236672 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 887336837 0 0) 0x7fc4040010b0 con 0x7fc4140ef610 2021-02-10T11:31:30.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.737 7fc4132bf700 1 --2- 172.21.2.101:0/484236672 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc4141b0470 0x7fc4141b0870 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7fc3fc006650 tx=0x7fc3fc009a00).ready entity=mon.0 client_cookie=1662da640e54a585 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:30.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.737 7fc4122bd700 1 -- 172.21.2.101:0/484236672 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc3fc0009b0 con 0x7fc4141b0470 2021-02-10T11:31:30.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.737 7fc4122bd700 1 -- 172.21.2.101:0/484236672 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3fc00c0c0 con 0x7fc4141b0470 2021-02-10T11:31:30.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.737 7fc4122bd700 1 -- 172.21.2.101:0/484236672 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc3fc00c5a0 con 0x7fc4141b0470 2021-02-10T11:31:30.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.737 7fc4195de700 1 -- 172.21.2.101:0/484236672 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc4141b0470 msgr2=0x7fc4141b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:30.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.737 7fc4195de700 1 --2- 172.21.2.101:0/484236672 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc4141b0470 0x7fc4141b0870 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7fc3fc006650 tx=0x7fc3fc009a00).stop 2021-02-10T11:31:30.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.738 7fc4195de700 1 -- 172.21.2.101:0/484236672 shutdown_connections 2021-02-10T11:31:30.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.738 7fc4195de700 1 --2- 172.21.2.101:0/484236672 >> v2:172.21.2.108:3300/0 conn(0x7fc4140f20e0 0x7fc4140f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:30.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.738 7fc4195de700 1 --2- 172.21.2.101:0/484236672 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc4141b0470 0x7fc4141b0870 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:30.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.738 7fc4195de700 1 -- 172.21.2.101:0/484236672 >> 172.21.2.101:0/484236672 conn(0x7fc4140ce3e0 msgr2=0x7fc4140e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:30.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.738 7fc4195de700 1 -- 172.21.2.101:0/484236672 shutdown_connections 2021-02-10T11:31:30.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.738 7fc4195de700 1 -- 172.21.2.101:0/484236672 wait complete. 2021-02-10T11:31:30.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.753 7fc4195de700 1 Processor -- start 2021-02-10T11:31:30.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.753 7fc4195de700 1 -- start start 2021-02-10T11:31:30.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.753 7fc4195de700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc4141a42f0 0x7fc4141a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:30.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.753 7fc4195de700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc414196170 0x7fc414196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:30.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.753 7fc4195de700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc41408f5b0 0x7fc41408f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:30.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.753 7fc4195de700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc414195bf0 con 0x7fc414196170 2021-02-10T11:31:30.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.753 7fc4195de700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4140c4130 con 0x7fc41408f5b0 2021-02-10T11:31:30.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.753 7fc4195de700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4140a6e70 con 0x7fc4141a42f0 2021-02-10T11:31:30.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.754 7fc412abe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc414196170 0x7fc414196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:30.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.754 7fc412abe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc414196170 0x7fc414196570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50100/0 (socket says 172.21.2.101:50100) 2021-02-10T11:31:30.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.754 7fc412abe700 1 -- 172.21.2.101:0/67814730 learned_addr learned my addr 172.21.2.101:0/67814730 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:30.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.754 7fc412abe700 1 -- 172.21.2.101:0/67814730 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc4141a42f0 msgr2=0x7fc4141a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:30.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.754 7fc412abe700 1 --2- 172.21.2.101:0/67814730 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc4141a42f0 0x7fc4141a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:30.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.754 7fc412abe700 1 -- 172.21.2.101:0/67814730 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc41408f5b0 msgr2=0x7fc41408f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:30.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.754 7fc412abe700 1 --2- 172.21.2.101:0/67814730 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc41408f5b0 0x7fc41408f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:30.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.754 7fc412abe700 1 -- 172.21.2.101:0/67814730 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4140a6a80 con 0x7fc414196170 2021-02-10T11:31:30.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.754 7fc412abe700 1 --2- 172.21.2.101:0/67814730 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc414196170 0x7fc414196570 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x7fc404001c10 tx=0x7fc40400a8f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:30.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.754 7fc40bfff700 1 -- 172.21.2.101:0/67814730 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc4040009b0 con 0x7fc414196170 2021-02-10T11:31:30.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.754 7fc40bfff700 1 -- 172.21.2.101:0/67814730 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4040012c0 con 0x7fc414196170 2021-02-10T11:31:30.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.754 7fc40bfff700 1 -- 172.21.2.101:0/67814730 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc404004350 con 0x7fc414196170 2021-02-10T11:31:30.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.754 7fc4195de700 1 -- 172.21.2.101:0/67814730 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4140a6a80 con 0x7fc414196170 2021-02-10T11:31:30.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.754 7fc4195de700 1 -- 172.21.2.101:0/67814730 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4141924c0 con 0x7fc414196170 2021-02-10T11:31:30.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.755 7fc40bfff700 1 -- 172.21.2.101:0/67814730 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc404004140 con 0x7fc414196170 2021-02-10T11:31:30.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.755 7fc40bfff700 1 --2- 172.21.2.101:0/67814730 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3f80186c0 0x7fc3f801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:30.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.755 7fc40bfff700 1 -- 172.21.2.101:0/67814730 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5107+0+0 (secure 0 0 0) 0x7fc404032430 con 0x7fc414196170 2021-02-10T11:31:30.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.755 7fc4132bf700 1 --2- 172.21.2.101:0/67814730 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3f80186c0 0x7fc3f801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:30.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.756 7fc4132bf700 1 --2- 172.21.2.101:0/67814730 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3f80186c0 0x7fc3f801abc0 crc :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:30.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.786 7fc4195de700 1 -- 172.21.2.101:0/67814730 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4140f0b50 con 0x7fc414196170 2021-02-10T11:31:30.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.789 7fc40bfff700 1 -- 172.21.2.101:0/67814730 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc4040009b0 con 0x7fc414196170 2021-02-10T11:31:30.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:30.920 7fc4195de700 1 -- 172.21.2.101:0/67814730 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"} v 0) v1 -- 0x7fc4140f0b50 con 0x7fc414196170 2021-02-10T11:31:31.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.316 7fc40bfff700 1 -- 172.21.2.101:0/67814730 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"}]=0 pool 'cephfs_data' created v36) v1 ==== 125+0+0 (secure 0 0 0) 0x7fc404004140 con 0x7fc414196170 2021-02-10T11:31:31.346 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' created 2021-02-10T11:31:31.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.337 7fc4195de700 1 -- 172.21.2.101:0/67814730 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3f80186c0 msgr2=0x7fc3f801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:31.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.337 7fc4195de700 1 --2- 172.21.2.101:0/67814730 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3f80186c0 0x7fc3f801abc0 crc :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:31.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.337 7fc4195de700 1 -- 172.21.2.101:0/67814730 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc414196170 msgr2=0x7fc414196570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:31.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.337 7fc4195de700 1 --2- 172.21.2.101:0/67814730 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc414196170 0x7fc414196570 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x7fc404001c10 tx=0x7fc40400a8f0).stop 2021-02-10T11:31:31.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.337 7fc4195de700 1 -- 172.21.2.101:0/67814730 shutdown_connections 2021-02-10T11:31:31.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.337 7fc4195de700 1 --2- 172.21.2.101:0/67814730 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc41408f5b0 0x7fc41408f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:31.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.337 7fc4195de700 1 --2- 172.21.2.101:0/67814730 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc414196170 0x7fc414196570 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:31.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.337 7fc4195de700 1 --2- 172.21.2.101:0/67814730 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3f80186c0 0x7fc3f801abc0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:31.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.337 7fc4195de700 1 --2- 172.21.2.101:0/67814730 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc4141a42f0 0x7fc4141a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:31.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.337 7fc4195de700 1 -- 172.21.2.101:0/67814730 >> 172.21.2.101:0/67814730 conn(0x7fc4140bf4e0 msgr2=0x7fc4140be160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:31.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.337 7fc4195de700 1 -- 172.21.2.101:0/67814730 shutdown_connections 2021-02-10T11:31:31.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.338 7fc4195de700 1 -- 172.21.2.101:0/67814730 wait complete. 2021-02-10T11:31:31.357 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-02-10T11:31:31.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.759 7f0e124a0700 1 Processor -- start 2021-02-10T11:31:31.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.759 7f0e124a0700 1 -- start start 2021-02-10T11:31:31.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.759 7f0e124a0700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0e0c0d1860 0x7f0e0c1b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:31.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.759 7f0e124a0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0e0c1b0ad0 0x7f0e0c108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:31.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.759 7f0e124a0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e0c0fe1b0 con 0x7f0e0c0d1c60 2021-02-10T11:31:31.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.759 7f0e124a0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e0c109a50 con 0x7f0e0c1b0ad0 2021-02-10T11:31:31.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.759 7f0e124a0700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e0c109740 con 0x7f0e0c0d1860 2021-02-10T11:31:31.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.760 7f0e0b7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0e0c1b0ad0 0x7f0e0c108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:31.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.760 7f0e0b7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0e0c1b0ad0 0x7f0e0c108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50110/0 (socket says 172.21.2.101:50110) 2021-02-10T11:31:31.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.760 7f0e0b7fe700 1 -- 172.21.2.101:0/1737310159 learned_addr learned my addr 172.21.2.101:0/1737310159 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:31.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.760 7f0e0b7fe700 1 -- 172.21.2.101:0/1737310159 >> v2:172.21.2.108:3300/0 conn(0x7f0e0c0d1860 msgr2=0x7f0e0c1b04b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:31:31.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.760 7f0e0b7fe700 1 --2- 172.21.2.101:0/1737310159 >> v2:172.21.2.108:3300/0 conn(0x7f0e0c0d1860 0x7f0e0c1b04b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:31.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.760 7f0e0b7fe700 1 -- 172.21.2.101:0/1737310159 >> v1:172.21.2.108:6789/0 conn(0x7f0e0c0d1c60 legacy=0x7f0e0c108f50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:31:31.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.760 7f0e0b7fe700 1 -- 172.21.2.101:0/1737310159 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c088020 con 0x7f0e0c1b0ad0 2021-02-10T11:31:31.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.760 7f0e0b7fe700 1 --2- 172.21.2.101:0/1737310159 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e0c1b0ad0 0x7f0e0c108930 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7f0e00006700 tx=0x7f0e00009b00).ready entity=mon.0 client_cookie=16c9a100c3ac6651 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:31.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.760 7f0e0affd700 1 -- 172.21.2.101:0/1737310159 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e0000c0c0 con 0x7f0e0c1b0ad0 2021-02-10T11:31:31.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.760 7f0e0affd700 1 -- 172.21.2.101:0/1737310159 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0000c400 con 0x7f0e0c1b0ad0 2021-02-10T11:31:31.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.760 7f0e0affd700 1 -- 172.21.2.101:0/1737310159 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e0000c920 con 0x7f0e0c1b0ad0 2021-02-10T11:31:31.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.760 7f0e124a0700 1 -- 172.21.2.101:0/1737310159 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e0c1b0ad0 msgr2=0x7f0e0c108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:31.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.760 7f0e124a0700 1 --2- 172.21.2.101:0/1737310159 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e0c1b0ad0 0x7f0e0c108930 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7f0e00006700 tx=0x7f0e00009b00).stop 2021-02-10T11:31:31.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.761 7f0e124a0700 1 -- 172.21.2.101:0/1737310159 shutdown_connections 2021-02-10T11:31:31.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.761 7f0e124a0700 1 --2- 172.21.2.101:0/1737310159 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e0c1b0ad0 0x7f0e0c108930 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:31.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.761 7f0e124a0700 1 --2- 172.21.2.101:0/1737310159 >> v2:172.21.2.108:3300/0 conn(0x7f0e0c0d1860 0x7f0e0c1b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:31.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.761 7f0e124a0700 1 -- 172.21.2.101:0/1737310159 >> 172.21.2.101:0/1737310159 conn(0x7f0e0c0ce410 msgr2=0x7f0e0c0d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:31.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.761 7f0e124a0700 1 -- 172.21.2.101:0/1737310159 shutdown_connections 2021-02-10T11:31:31.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.761 7f0e124a0700 1 -- 172.21.2.101:0/1737310159 wait complete. 2021-02-10T11:31:31.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.776 7f0e124a0700 1 Processor -- start 2021-02-10T11:31:31.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.776 7f0e124a0700 1 -- start start 2021-02-10T11:31:31.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.776 7f0e124a0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e0c0b04e0 0x7f0e0c0b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:31.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.776 7f0e124a0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e0c18ec70 0x7f0e0c18f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:31.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.776 7f0e124a0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e0c18f690 0x7f0e0c1b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:31.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.776 7f0e124a0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c19efc0 con 0x7f0e0c18ec70 2021-02-10T11:31:31.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.776 7f0e124a0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c1afb50 con 0x7f0e0c0b04e0 2021-02-10T11:31:31.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.776 7f0e124a0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c18fa90 con 0x7f0e0c18f690 2021-02-10T11:31:31.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.777 7f0e0b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e0c18ec70 0x7f0e0c18f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:31.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.777 7f0e0b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e0c18ec70 0x7f0e0c18f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50116/0 (socket says 172.21.2.101:50116) 2021-02-10T11:31:31.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.777 7f0e0b7fe700 1 -- 172.21.2.101:0/1311799155 learned_addr learned my addr 172.21.2.101:0/1311799155 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:31.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.777 7f0e0b7fe700 1 -- 172.21.2.101:0/1311799155 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e0c18f690 msgr2=0x7f0e0c1b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:31.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.777 7f0e0b7fe700 1 --2- 172.21.2.101:0/1311799155 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e0c18f690 0x7f0e0c1b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:31.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.777 7f0e0b7fe700 1 -- 172.21.2.101:0/1311799155 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e0c0b04e0 msgr2=0x7f0e0c0b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:31.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.777 7f0e0b7fe700 1 --2- 172.21.2.101:0/1311799155 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e0c0b04e0 0x7f0e0c0b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:31.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.777 7f0e0b7fe700 1 -- 172.21.2.101:0/1311799155 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c18fe40 con 0x7f0e0c18ec70 2021-02-10T11:31:31.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.777 7f0e0b7fe700 1 --2- 172.21.2.101:0/1311799155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e0c18ec70 0x7f0e0c18f070 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f0e0000b660 tx=0x7f0e0000ba00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:31.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.777 7f0e08ff9700 1 -- 172.21.2.101:0/1311799155 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e00009440 con 0x7f0e0c18ec70 2021-02-10T11:31:31.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.778 7f0e08ff9700 1 -- 172.21.2.101:0/1311799155 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e000090c0 con 0x7f0e0c18ec70 2021-02-10T11:31:31.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.778 7f0e124a0700 1 -- 172.21.2.101:0/1311799155 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e0c18fe40 con 0x7f0e0c18ec70 2021-02-10T11:31:31.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.778 7f0e08ff9700 1 -- 172.21.2.101:0/1311799155 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e00005370 con 0x7f0e0c18ec70 2021-02-10T11:31:31.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.778 7f0e124a0700 1 -- 172.21.2.101:0/1311799155 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e0c1b09e0 con 0x7f0e0c18ec70 2021-02-10T11:31:31.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.778 7f0e08ff9700 1 -- 172.21.2.101:0/1311799155 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0e000062c0 con 0x7f0e0c18ec70 2021-02-10T11:31:31.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.778 7f0e08ff9700 1 --2- 172.21.2.101:0/1311799155 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0dec0186c0 0x7f0dec01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:31.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.778 7f0e08ff9700 1 -- 172.21.2.101:0/1311799155 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5457+0+0 (secure 0 0 0) 0x7f0e00032570 con 0x7f0e0c18ec70 2021-02-10T11:31:31.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.779 7f0e0bfff700 1 --2- 172.21.2.101:0/1311799155 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0dec0186c0 0x7f0dec01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:31.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.779 7f0e0bfff700 1 --2- 172.21.2.101:0/1311799155 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0dec0186c0 0x7f0dec01abc0 crc :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:31.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.810 7f0e124a0700 1 -- 172.21.2.101:0/1311799155 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e0c19efc0 con 0x7f0e0c18ec70 2021-02-10T11:31:31.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.812 7f0e08ff9700 1 -- 172.21.2.101:0/1311799155 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0e000093e0 con 0x7f0e0c18ec70 2021-02-10T11:31:31.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.940 7f0e124a0700 1 -- 172.21.2.101:0/1311799155 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"} v 0) v1 -- 0x7f0e0c19efc0 con 0x7f0e0c18ec70 2021-02-10T11:31:31.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.944 7f0e08ff9700 1 -- 172.21.2.101:0/1311799155 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"}]=0 new fs with metadata pool 6 and data pool 7 v19) v1 ==== 187+0+0 (secure 0 0 0) 0x7f0e0001f540 con 0x7f0e0c18ec70 2021-02-10T11:31:31.957 INFO:teuthology.orchestra.run.gibba001.stderr:new fs with metadata pool 6 and data pool 7 2021-02-10T11:31:31.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.949 7f0e124a0700 1 -- 172.21.2.101:0/1311799155 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0dec0186c0 msgr2=0x7f0dec01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:31.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.949 7f0e124a0700 1 --2- 172.21.2.101:0/1311799155 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0dec0186c0 0x7f0dec01abc0 crc :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:31.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.949 7f0e124a0700 1 -- 172.21.2.101:0/1311799155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e0c18ec70 msgr2=0x7f0e0c18f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:31.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.949 7f0e124a0700 1 --2- 172.21.2.101:0/1311799155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e0c18ec70 0x7f0e0c18f070 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f0e0000b660 tx=0x7f0e0000ba00).stop 2021-02-10T11:31:31.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.949 7f0e124a0700 1 -- 172.21.2.101:0/1311799155 shutdown_connections 2021-02-10T11:31:31.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.949 7f0e124a0700 1 --2- 172.21.2.101:0/1311799155 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0dec0186c0 0x7f0dec01abc0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:31.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.949 7f0e124a0700 1 --2- 172.21.2.101:0/1311799155 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e0c18f690 0x7f0e0c1b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:31.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.949 7f0e124a0700 1 --2- 172.21.2.101:0/1311799155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e0c18ec70 0x7f0e0c18f070 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:31.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.949 7f0e124a0700 1 --2- 172.21.2.101:0/1311799155 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e0c0b04e0 0x7f0e0c0b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:31.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.949 7f0e124a0700 1 -- 172.21.2.101:0/1311799155 >> 172.21.2.101:0/1311799155 conn(0x7f0e0c19f830 msgr2=0x7f0e0c0858d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:31.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.949 7f0e124a0700 1 -- 172.21.2.101:0/1311799155 shutdown_connections 2021-02-10T11:31:31.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:31.950 7f0e124a0700 1 -- 172.21.2.101:0/1311799155 wait complete. 2021-02-10T11:31:31.969 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:31:32.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.366 7f2295682700 1 Processor -- start 2021-02-10T11:31:32.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.366 7f2295682700 1 -- start start 2021-02-10T11:31:32.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.367 7f2295682700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22900e5470 0x7f229007e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:32.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.367 7f2295682700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f229007f300 0x7f229007f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:32.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.367 7f2295682700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f229007ed80 con 0x7f22900e5070 2021-02-10T11:31:32.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.367 7f2295682700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22901b4010 con 0x7f229007f300 2021-02-10T11:31:32.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.367 7f2295682700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22900d1790 con 0x7f22900e5470 2021-02-10T11:31:32.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.367 7f228f2bf700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f22900e5070 0x7f22901b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50616/0 (socket says 172.21.2.101:50616) 2021-02-10T11:31:32.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.367 7f228f2bf700 1 -- 172.21.2.101:0/2394647758 learned_addr learned my addr 172.21.2.101:0/2394647758 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:32.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.367 7f228eabe700 1 --2- 172.21.2.101:0/2394647758 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22900e5470 0x7f229007e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:32.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.367 7f228fac0700 1 --2- 172.21.2.101:0/2394647758 >> v2:172.21.2.108:3300/0 conn(0x7f229007f300 0x7f229007f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:32.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.367 7f228e2bd700 1 -- 172.21.2.101:0/2394647758 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1800555257 0 0) 0x7f2278000ce0 con 0x7f22900e5070 2021-02-10T11:31:32.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.367 7f228e2bd700 1 -- 172.21.2.101:0/2394647758 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f227c002fe0 con 0x7f22900e5070 2021-02-10T11:31:32.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.368 7f228fac0700 1 -- 172.21.2.101:0/2394647758 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22900e5470 msgr2=0x7f229007e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:32.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.368 7f228fac0700 1 --2- 172.21.2.101:0/2394647758 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22900e5470 0x7f229007e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:32.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.368 7f228fac0700 1 -- 172.21.2.101:0/2394647758 >> v1:172.21.2.101:6789/0 conn(0x7f22900e5070 legacy=0x7f22901b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:32.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.368 7f228fac0700 1 -- 172.21.2.101:0/2394647758 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22900d1980 con 0x7f229007f300 2021-02-10T11:31:32.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.368 7f228fac0700 1 --2- 172.21.2.101:0/2394647758 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f229007f300 0x7f229007f700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f2288006e50 tx=0x7f2288009e70).ready entity=mon.1 client_cookie=f97a19709853be0e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:32.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.368 7f228e2bd700 1 -- 172.21.2.101:0/2394647758 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f228800b510 con 0x7f229007f300 2021-02-10T11:31:32.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.368 7f228e2bd700 1 -- 172.21.2.101:0/2394647758 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2288006790 con 0x7f229007f300 2021-02-10T11:31:32.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.368 7f228e2bd700 1 -- 172.21.2.101:0/2394647758 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f228800b940 con 0x7f229007f300 2021-02-10T11:31:32.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.368 7f2295682700 1 -- 172.21.2.101:0/2394647758 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f229007f300 msgr2=0x7f229007f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:32.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.368 7f2295682700 1 --2- 172.21.2.101:0/2394647758 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f229007f300 0x7f229007f700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f2288006e50 tx=0x7f2288009e70).stop 2021-02-10T11:31:32.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.369 7f2295682700 1 -- 172.21.2.101:0/2394647758 shutdown_connections 2021-02-10T11:31:32.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.369 7f2295682700 1 --2- 172.21.2.101:0/2394647758 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22900e5470 0x7f229007e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:32.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.369 7f2295682700 1 --2- 172.21.2.101:0/2394647758 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f229007f300 0x7f229007f700 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:32.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.369 7f2295682700 1 -- 172.21.2.101:0/2394647758 >> 172.21.2.101:0/2394647758 conn(0x7f22900ce690 msgr2=0x7f22900e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:32.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.369 7f2295682700 1 -- 172.21.2.101:0/2394647758 shutdown_connections 2021-02-10T11:31:32.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.369 7f2295682700 1 -- 172.21.2.101:0/2394647758 wait complete. 2021-02-10T11:31:32.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.383 7f2295682700 1 Processor -- start 2021-02-10T11:31:32.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.384 7f2295682700 1 -- start start 2021-02-10T11:31:32.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.384 7f2295682700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2290191bb0 0x7f22900a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:32.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.384 7f2295682700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22900a29a0 0x7f22900a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:32.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.384 7f2295682700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22900a33c0 0x7f22900a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:32.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.384 7f2295682700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22900adf90 con 0x7f22900a29a0 2021-02-10T11:31:32.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.384 7f2295682700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22900a1e00 con 0x7f22900a33c0 2021-02-10T11:31:32.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.384 7f2295682700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22900a09f0 con 0x7f2290191bb0 2021-02-10T11:31:32.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.384 7f228eabe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22900a29a0 0x7f22900a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:32.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.384 7f228f2bf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2290191bb0 0x7f22900a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:32.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.384 7f228eabe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22900a29a0 0x7f22900a2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50130/0 (socket says 172.21.2.101:50130) 2021-02-10T11:31:32.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.384 7f228eabe700 1 -- 172.21.2.101:0/319394062 learned_addr learned my addr 172.21.2.101:0/319394062 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:32.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.384 7f228fac0700 1 --2- 172.21.2.101:0/319394062 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22900a33c0 0x7f22900a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:32.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.385 7f228f2bf700 1 -- 172.21.2.101:0/319394062 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22900a33c0 msgr2=0x7f22900a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:32.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.385 7f228f2bf700 1 --2- 172.21.2.101:0/319394062 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22900a33c0 0x7f22900a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:32.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.385 7f228f2bf700 1 -- 172.21.2.101:0/319394062 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22900a29a0 msgr2=0x7f22900a2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:32.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.385 7f228f2bf700 1 --2- 172.21.2.101:0/319394062 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22900a29a0 0x7f22900a2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:32.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.385 7f228f2bf700 1 -- 172.21.2.101:0/319394062 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22900a0bd0 con 0x7f2290191bb0 2021-02-10T11:31:32.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.385 7f228f2bf700 1 --2- 172.21.2.101:0/319394062 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2290191bb0 0x7f22900a2380 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f2278001430 tx=0x7f227800a870).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:32.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.385 7f2287fff700 1 -- 172.21.2.101:0/319394062 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2278000ce0 con 0x7f2290191bb0 2021-02-10T11:31:32.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.385 7f2287fff700 1 -- 172.21.2.101:0/319394062 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22780070d0 con 0x7f2290191bb0 2021-02-10T11:31:32.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.385 7f2295682700 1 -- 172.21.2.101:0/319394062 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22900a0bd0 con 0x7f2290191bb0 2021-02-10T11:31:32.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.385 7f2295682700 1 -- 172.21.2.101:0/319394062 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22900a1160 con 0x7f2290191bb0 2021-02-10T11:31:32.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.385 7f2287fff700 1 -- 172.21.2.101:0/319394062 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f22780043a0 con 0x7f2290191bb0 2021-02-10T11:31:32.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.386 7f2287fff700 1 -- 172.21.2.101:0/319394062 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f22780043a0 con 0x7f2290191bb0 2021-02-10T11:31:32.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.386 7f2287fff700 1 --2- 172.21.2.101:0/319394062 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22740186d0 0x7f227401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:32.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.386 7f2287fff700 1 -- 172.21.2.101:0/319394062 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2278032580 con 0x7f2290191bb0 2021-02-10T11:31:32.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.386 7f228eabe700 1 --2- 172.21.2.101:0/319394062 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22740186d0 0x7f227401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:32.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.387 7f228eabe700 1 --2- 172.21.2.101:0/319394062 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22740186d0 0x7f227401abd0 crc :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:32.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.417 7f2295682700 1 -- 172.21.2.101:0/319394062 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f229007ed80 con 0x7f2290191bb0 2021-02-10T11:31:32.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.420 7f2287fff700 1 -- 172.21.2.101:0/319394062 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2278000c90 con 0x7f2290191bb0 2021-02-10T11:31:32.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.547 7f2295682700 1 -- 172.21.2.101:0/319394062 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22900adf90 con 0x7f2290191bb0 2021-02-10T11:31:32.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.549 7f2287fff700 1 -- 172.21.2.101:0/319394062 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+14006 (secure 0 0 0) 0x7f22780043a0 con 0x7f2290191bb0 2021-02-10T11:31:32.562 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:31:32.562 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":37,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:31:31.941531","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:31:29.865604","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2021-02-10 11:31:30.939202","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:31:32.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.552 7f2295682700 1 -- 172.21.2.101:0/319394062 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22740186d0 msgr2=0x7f227401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:32.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.552 7f2295682700 1 --2- 172.21.2.101:0/319394062 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22740186d0 0x7f227401abd0 crc :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:32.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.552 7f2295682700 1 -- 172.21.2.101:0/319394062 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2290191bb0 msgr2=0x7f22900a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:32.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.552 7f2295682700 1 --2- 172.21.2.101:0/319394062 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2290191bb0 0x7f22900a2380 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f2278001430 tx=0x7f227800a870).stop 2021-02-10T11:31:32.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.552 7f2295682700 1 -- 172.21.2.101:0/319394062 shutdown_connections 2021-02-10T11:31:32.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.552 7f2295682700 1 --2- 172.21.2.101:0/319394062 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22900a33c0 0x7f22900a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:32.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.552 7f2295682700 1 --2- 172.21.2.101:0/319394062 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22900a29a0 0x7f22900a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:32.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.552 7f2295682700 1 --2- 172.21.2.101:0/319394062 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22740186d0 0x7f227401abd0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:32.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.552 7f2295682700 1 --2- 172.21.2.101:0/319394062 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2290191bb0 0x7f22900a2380 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:32.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.552 7f2295682700 1 -- 172.21.2.101:0/319394062 >> 172.21.2.101:0/319394062 conn(0x7f22901a31d0 msgr2=0x7f229008b830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:32.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.553 7f2295682700 1 -- 172.21.2.101:0/319394062 shutdown_connections 2021-02-10T11:31:32.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.553 7f2295682700 1 -- 172.21.2.101:0/319394062 wait complete. 2021-02-10T11:31:32.572 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:31:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.971 7fda250a0700 1 Processor -- start 2021-02-10T11:31:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.971 7fda250a0700 1 -- start start 2021-02-10T11:31:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.971 7fda250a0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fda200e5470 0x7fda2007e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.971 7fda250a0700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda2007ed80 con 0x7fda200e5070 2021-02-10T11:31:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.971 7fda250a0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda201b4010 con 0x7fda2007f300 2021-02-10T11:31:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.971 7fda250a0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda200d1790 con 0x7fda200e5470 2021-02-10T11:31:32.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.971 7fda1ed9d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fda200e5070 0x7fda201b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50630/0 (socket says 172.21.2.101:50630) 2021-02-10T11:31:32.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.971 7fda1ed9d700 1 -- 172.21.2.101:0/190536274 learned_addr learned my addr 172.21.2.101:0/190536274 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:32.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.971 7fda1e59c700 1 --2- 172.21.2.101:0/190536274 >> v2:172.21.2.101:3300/0 conn(0x7fda200e5470 0x7fda2007e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:32.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.972 7fda1e59c700 1 -- 172.21.2.101:0/190536274 >> v1:172.21.2.108:6789/0 conn(0x7fda2007f300 legacy=0x7fda2007f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:32.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.972 7fda1e59c700 1 -- 172.21.2.101:0/190536274 >> v1:172.21.2.101:6789/0 conn(0x7fda200e5070 legacy=0x7fda201b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:32.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.972 7fda1e59c700 1 -- 172.21.2.101:0/190536274 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda200d19b0 con 0x7fda200e5470 2021-02-10T11:31:32.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.972 7fda1e59c700 1 --2- 172.21.2.101:0/190536274 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fda200e5470 0x7fda2007e6a0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7fda08006650 tx=0x7fda08009a80).ready entity=mon.0 client_cookie=1491021225fa25f3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:32.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.972 7fda1dd9b700 1 -- 172.21.2.101:0/190536274 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fda0800c0c0 con 0x7fda200e5470 2021-02-10T11:31:32.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.972 7fda1dd9b700 1 -- 172.21.2.101:0/190536274 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0800c340 con 0x7fda200e5470 2021-02-10T11:31:32.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.972 7fda1dd9b700 1 -- 172.21.2.101:0/190536274 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fda0800c860 con 0x7fda200e5470 2021-02-10T11:31:32.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.972 7fda250a0700 1 -- 172.21.2.101:0/190536274 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fda200e5470 msgr2=0x7fda2007e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:32.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.972 7fda250a0700 1 --2- 172.21.2.101:0/190536274 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fda200e5470 0x7fda2007e6a0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7fda08006650 tx=0x7fda08009a80).stop 2021-02-10T11:31:32.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.973 7fda250a0700 1 -- 172.21.2.101:0/190536274 shutdown_connections 2021-02-10T11:31:33.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.973 7fda250a0700 1 --2- 172.21.2.101:0/190536274 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fda200e5470 0x7fda2007e6a0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:33.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.973 7fda250a0700 1 -- 172.21.2.101:0/190536274 >> 172.21.2.101:0/190536274 conn(0x7fda200ce690 msgr2=0x7fda200e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:33.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.973 7fda250a0700 1 -- 172.21.2.101:0/190536274 shutdown_connections 2021-02-10T11:31:33.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.973 7fda250a0700 1 -- 172.21.2.101:0/190536274 wait complete. 2021-02-10T11:31:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.988 7fda250a0700 1 Processor -- start 2021-02-10T11:31:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.988 7fda250a0700 1 -- start start 2021-02-10T11:31:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.988 7fda250a0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fda20092dc0 0x7fda200931c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.988 7fda250a0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fda200937e0 0x7fda200ab250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.988 7fda250a0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fda200ab870 0x7fda200942e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.988 7fda250a0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda20093de0 con 0x7fda20092dc0 2021-02-10T11:31:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.988 7fda250a0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda201c2500 con 0x7fda200ab870 2021-02-10T11:31:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.988 7fda250a0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda201c2720 con 0x7fda200937e0 2021-02-10T11:31:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.988 7fda1ed9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fda20092dc0 0x7fda200931c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.988 7fda1e59c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fda200937e0 0x7fda200ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.988 7fda1e59c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fda200937e0 0x7fda200ab250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:42874/0 (socket says 172.21.2.101:42874) 2021-02-10T11:31:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.988 7fda1ed9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fda20092dc0 0x7fda200931c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50144/0 (socket says 172.21.2.101:50144) 2021-02-10T11:31:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.988 7fda1e59c700 1 -- 172.21.2.101:0/3569014617 learned_addr learned my addr 172.21.2.101:0/3569014617 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.989 7fda1f59e700 1 --2- 172.21.2.101:0/3569014617 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fda200ab870 0x7fda200942e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.989 7fda1ed9d700 1 -- 172.21.2.101:0/3569014617 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fda200937e0 msgr2=0x7fda200ab250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.989 7fda1ed9d700 1 --2- 172.21.2.101:0/3569014617 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fda200937e0 0x7fda200ab250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:33.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.989 7fda1ed9d700 1 -- 172.21.2.101:0/3569014617 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fda200ab870 msgr2=0x7fda200942e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:33.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.989 7fda1ed9d700 1 --2- 172.21.2.101:0/3569014617 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fda200ab870 0x7fda200942e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:33.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.989 7fda1ed9d700 1 -- 172.21.2.101:0/3569014617 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda200abce0 con 0x7fda20092dc0 2021-02-10T11:31:33.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.989 7fda1e59c700 1 --2- 172.21.2.101:0/3569014617 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fda200937e0 0x7fda200ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:31:33.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.989 7fda1ed9d700 1 --2- 172.21.2.101:0/3569014617 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fda20092dc0 0x7fda200931c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7fda10001410 tx=0x7fda1000a8a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:33.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.989 7fda177fe700 1 -- 172.21.2.101:0/3569014617 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fda10000ef0 con 0x7fda20092dc0 2021-02-10T11:31:33.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.989 7fda177fe700 1 -- 172.21.2.101:0/3569014617 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda10007160 con 0x7fda20092dc0 2021-02-10T11:31:33.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.989 7fda250a0700 1 -- 172.21.2.101:0/3569014617 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda200abce0 con 0x7fda20092dc0 2021-02-10T11:31:33.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.989 7fda250a0700 1 -- 172.21.2.101:0/3569014617 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda20094bf0 con 0x7fda20092dc0 2021-02-10T11:31:33.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.989 7fda177fe700 1 -- 172.21.2.101:0/3569014617 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fda100043a0 con 0x7fda20092dc0 2021-02-10T11:31:33.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.990 7fda177fe700 1 -- 172.21.2.101:0/3569014617 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fda100043a0 con 0x7fda20092dc0 2021-02-10T11:31:33.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.990 7fda177fe700 1 --2- 172.21.2.101:0/3569014617 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fda04018730 0x7fda0401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:33.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.990 7fda177fe700 1 -- 172.21.2.101:0/3569014617 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fda10032730 con 0x7fda20092dc0 2021-02-10T11:31:33.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.990 7fda1e59c700 1 --2- 172.21.2.101:0/3569014617 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fda04018730 0x7fda0401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:33.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:32.990 7fda1e59c700 1 --2- 172.21.2.101:0/3569014617 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fda04018730 0x7fda0401ac10 crc :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:33.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.021 7fda250a0700 1 -- 172.21.2.101:0/3569014617 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda200ce690 con 0x7fda20092dc0 2021-02-10T11:31:33.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.024 7fda177fe700 1 -- 172.21.2.101:0/3569014617 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fda10000ef0 con 0x7fda20092dc0 2021-02-10T11:31:33.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.147 7fda250a0700 1 -- 172.21.2.101:0/3569014617 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda200ce690 con 0x7fda20092dc0 2021-02-10T11:31:33.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.148 7fda177fe700 1 -- 172.21.2.101:0/3569014617 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+13993 (secure 0 0 0) 0x7fda10007140 con 0x7fda20092dc0 2021-02-10T11:31:33.159 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:31:33.159 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":38,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:31:32.943916","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:31:29.865604","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2021-02-10 11:31:30.939202","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:31:33.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.149 7fda250a0700 1 -- 172.21.2.101:0/3569014617 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fda04018730 msgr2=0x7fda0401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:33.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.149 7fda250a0700 1 --2- 172.21.2.101:0/3569014617 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fda04018730 0x7fda0401ac10 crc :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:33.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.149 7fda250a0700 1 -- 172.21.2.101:0/3569014617 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fda20092dc0 msgr2=0x7fda200931c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:33.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.149 7fda250a0700 1 --2- 172.21.2.101:0/3569014617 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fda20092dc0 0x7fda200931c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7fda10001410 tx=0x7fda1000a8a0).stop 2021-02-10T11:31:33.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.150 7fda250a0700 1 -- 172.21.2.101:0/3569014617 shutdown_connections 2021-02-10T11:31:33.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.150 7fda250a0700 1 --2- 172.21.2.101:0/3569014617 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fda04018730 0x7fda0401ac10 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:33.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.150 7fda250a0700 1 --2- 172.21.2.101:0/3569014617 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fda200937e0 0x7fda200ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:33.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.150 7fda250a0700 1 --2- 172.21.2.101:0/3569014617 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fda200ab870 0x7fda200942e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:33.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.150 7fda250a0700 1 --2- 172.21.2.101:0/3569014617 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fda20092dc0 0x7fda200931c0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:33.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.150 7fda250a0700 1 -- 172.21.2.101:0/3569014617 >> 172.21.2.101:0/3569014617 conn(0x7fda20106df0 msgr2=0x7fda200962d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:33.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.150 7fda250a0700 1 -- 172.21.2.101:0/3569014617 shutdown_connections 2021-02-10T11:31:33.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.150 7fda250a0700 1 -- 172.21.2.101:0/3569014617 wait complete. 2021-02-10T11:31:33.169 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-02-10T11:31:33.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.573 7fc383ddf700 1 Processor -- start 2021-02-10T11:31:33.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.573 7fc383ddf700 1 -- start start 2021-02-10T11:31:33.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.574 7fc383ddf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc37c1b0470 0x7fc37c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:33.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.574 7fc383ddf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc37c0ef610 0x7fc37c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:33.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.574 7fc383ddf700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37c0f0b50 con 0x7fc37c0f0030 2021-02-10T11:31:33.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.574 7fc383ddf700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc37c0e52e0 con 0x7fc37c0ef610 2021-02-10T11:31:33.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.574 7fc383ddf700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c0e5490 con 0x7fc37c1b0470 2021-02-10T11:31:33.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.574 7fc3813fa700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc37c0ef610 0x7fc37c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:33.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.574 7fc3813fa700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc37c0ef610 0x7fc37c0efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50150/0 (socket says 172.21.2.101:50150) 2021-02-10T11:31:33.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.574 7fc3813fa700 1 -- 172.21.2.101:0/2231783452 learned_addr learned my addr 172.21.2.101:0/2231783452 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:33.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.574 7fc3813fa700 1 -- 172.21.2.101:0/2231783452 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc37c1b0470 msgr2=0x7fc37c1b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:31:33.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.574 7fc3813fa700 1 --2- 172.21.2.101:0/2231783452 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc37c1b0470 0x7fc37c1b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:33.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.574 7fc3813fa700 1 -- 172.21.2.101:0/2231783452 >> v1:172.21.2.108:6789/0 conn(0x7fc37c0f0030 legacy=0x7fc37c0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:33.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.575 7fc3813fa700 1 -- 172.21.2.101:0/2231783452 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37c0f0430 con 0x7fc37c0ef610 2021-02-10T11:31:33.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.575 7fc3813fa700 1 --2- 172.21.2.101:0/2231783452 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc37c0ef610 0x7fc37c0efa10 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7fc36c006700 tx=0x7fc36c009b00).ready entity=mon.0 client_cookie=7f79a85a9a53064f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:33.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.575 7fc380bf9700 1 -- 172.21.2.101:0/2231783452 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc36c00c0c0 con 0x7fc37c0ef610 2021-02-10T11:31:33.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.575 7fc380bf9700 1 -- 172.21.2.101:0/2231783452 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36c00c400 con 0x7fc37c0ef610 2021-02-10T11:31:33.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.575 7fc380bf9700 1 -- 172.21.2.101:0/2231783452 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc36c00c920 con 0x7fc37c0ef610 2021-02-10T11:31:33.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.575 7fc383ddf700 1 -- 172.21.2.101:0/2231783452 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc37c0ef610 msgr2=0x7fc37c0efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:33.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.575 7fc383ddf700 1 --2- 172.21.2.101:0/2231783452 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc37c0ef610 0x7fc37c0efa10 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7fc36c006700 tx=0x7fc36c009b00).stop 2021-02-10T11:31:33.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.576 7fc383ddf700 1 -- 172.21.2.101:0/2231783452 shutdown_connections 2021-02-10T11:31:33.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.576 7fc383ddf700 1 --2- 172.21.2.101:0/2231783452 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc37c0ef610 0x7fc37c0efa10 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:33.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.576 7fc383ddf700 1 --2- 172.21.2.101:0/2231783452 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc37c1b0470 0x7fc37c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:33.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.576 7fc383ddf700 1 -- 172.21.2.101:0/2231783452 >> 172.21.2.101:0/2231783452 conn(0x7fc37c0ce3e0 msgr2=0x7fc37c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:33.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.576 7fc383ddf700 1 -- 172.21.2.101:0/2231783452 shutdown_connections 2021-02-10T11:31:33.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.576 7fc383ddf700 1 -- 172.21.2.101:0/2231783452 wait complete. 2021-02-10T11:31:33.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.590 7fc383ddf700 1 Processor -- start 2021-02-10T11:31:33.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.591 7fc383ddf700 1 -- start start 2021-02-10T11:31:33.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.591 7fc383ddf700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc37c0a6c90 0x7fc37c1924c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:33.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.591 7fc383ddf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc37c192ae0 0x7fc37c192ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:33.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.591 7fc383ddf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc37c0919e0 0x7fc37c091de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:33.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.591 7fc383ddf700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c19d2e0 con 0x7fc37c0919e0 2021-02-10T11:31:33.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.591 7fc383ddf700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c0914d0 con 0x7fc37c0a6c90 2021-02-10T11:31:33.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.591 7fc383ddf700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c0a6820 con 0x7fc37c192ae0 2021-02-10T11:31:33.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.591 7fc3823fc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc37c0919e0 0x7fc37c091de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:33.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.591 7fc3823fc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc37c0919e0 0x7fc37c091de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50156/0 (socket says 172.21.2.101:50156) 2021-02-10T11:31:33.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.591 7fc3823fc700 1 -- 172.21.2.101:0/2161109280 learned_addr learned my addr 172.21.2.101:0/2161109280 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:33.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.592 7fc3823fc700 1 -- 172.21.2.101:0/2161109280 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc37c192ae0 msgr2=0x7fc37c192ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:33.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.592 7fc3823fc700 1 --2- 172.21.2.101:0/2161109280 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc37c192ae0 0x7fc37c192ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:33.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.592 7fc3823fc700 1 -- 172.21.2.101:0/2161109280 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc37c0a6c90 msgr2=0x7fc37c1924c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:33.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.592 7fc3823fc700 1 --2- 172.21.2.101:0/2161109280 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc37c0a6c90 0x7fc37c1924c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:33.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.592 7fc3823fc700 1 -- 172.21.2.101:0/2161109280 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37c092400 con 0x7fc37c0919e0 2021-02-10T11:31:33.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.592 7fc3823fc700 1 --2- 172.21.2.101:0/2161109280 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc37c0919e0 0x7fc37c091de0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7fc374009940 tx=0x7fc37400bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:33.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.592 7fc3727fc700 1 -- 172.21.2.101:0/2161109280 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc374000d30 con 0x7fc37c0919e0 2021-02-10T11:31:33.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.592 7fc3727fc700 1 -- 172.21.2.101:0/2161109280 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc37400c2f0 con 0x7fc37c0919e0 2021-02-10T11:31:33.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.592 7fc3727fc700 1 -- 172.21.2.101:0/2161109280 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc37400c810 con 0x7fc37c0919e0 2021-02-10T11:31:33.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.592 7fc383ddf700 1 -- 172.21.2.101:0/2161109280 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc37c092400 con 0x7fc37c0919e0 2021-02-10T11:31:33.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.592 7fc383ddf700 1 -- 172.21.2.101:0/2161109280 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc37c092400 con 0x7fc37c0919e0 2021-02-10T11:31:33.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.593 7fc3727fc700 1 -- 172.21.2.101:0/2161109280 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc374001220 con 0x7fc37c0919e0 2021-02-10T11:31:33.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.593 7fc3727fc700 1 --2- 172.21.2.101:0/2161109280 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3640186c0 0x7fc36401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:33.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.593 7fc3727fc700 1 -- 172.21.2.101:0/2161109280 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc3740328d0 con 0x7fc37c0919e0 2021-02-10T11:31:33.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.593 7fc381bfb700 1 --2- 172.21.2.101:0/2161109280 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3640186c0 0x7fc36401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:33.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.593 7fc381bfb700 1 --2- 172.21.2.101:0/2161109280 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3640186c0 0x7fc36401abc0 crc :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:33.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.623 7fc383ddf700 1 -- 172.21.2.101:0/2161109280 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc37c0e52e0 con 0x7fc37c0919e0 2021-02-10T11:31:33.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.626 7fc3727fc700 1 -- 172.21.2.101:0/2161109280 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc37400be30 con 0x7fc37c0919e0 2021-02-10T11:31:33.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.762 7fc383ddf700 1 -- 172.21.2.101:0/2161109280 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"} v 0) v1 -- 0x7fc37c0e52e0 con 0x7fc37c0919e0 2021-02-10T11:31:33.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.953 7fc3727fc700 1 -- 172.21.2.101:0/2161109280 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"}]=0 v22) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc374001220 con 0x7fc37c0919e0 2021-02-10T11:31:33.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.980 7fc383ddf700 1 -- 172.21.2.101:0/2161109280 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3640186c0 msgr2=0x7fc36401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:33.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.980 7fc383ddf700 1 --2- 172.21.2.101:0/2161109280 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3640186c0 0x7fc36401abc0 crc :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:33.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.980 7fc383ddf700 1 -- 172.21.2.101:0/2161109280 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc37c0919e0 msgr2=0x7fc37c091de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:33.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.980 7fc383ddf700 1 --2- 172.21.2.101:0/2161109280 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc37c0919e0 0x7fc37c091de0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7fc374009940 tx=0x7fc37400bcf0).stop 2021-02-10T11:31:33.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.980 7fc383ddf700 1 -- 172.21.2.101:0/2161109280 shutdown_connections 2021-02-10T11:31:33.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.980 7fc383ddf700 1 --2- 172.21.2.101:0/2161109280 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3640186c0 0x7fc36401abc0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:33.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.980 7fc383ddf700 1 --2- 172.21.2.101:0/2161109280 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc37c192ae0 0x7fc37c192ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:33.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.980 7fc383ddf700 1 --2- 172.21.2.101:0/2161109280 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc37c0919e0 0x7fc37c091de0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:33.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.980 7fc383ddf700 1 --2- 172.21.2.101:0/2161109280 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc37c0a6c90 0x7fc37c1924c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:33.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.980 7fc383ddf700 1 -- 172.21.2.101:0/2161109280 >> 172.21.2.101:0/2161109280 conn(0x7fc37c0ac550 msgr2=0x7fc37c0be1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:33.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.980 7fc383ddf700 1 -- 172.21.2.101:0/2161109280 shutdown_connections 2021-02-10T11:31:33.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:33.980 7fc383ddf700 1 -- 172.21.2.101:0/2161109280 wait complete. 2021-02-10T11:31:33.999 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:31:34.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.398 7f1916e86700 1 Processor -- start 2021-02-10T11:31:34.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.398 7f1916e86700 1 -- start start 2021-02-10T11:31:34.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.398 7f1916e86700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f19100e5070 0x7f19101b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:34.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.398 7f1916e86700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f19101b3e40 0x7f191007d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:34.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.398 7f1916e86700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f191007ed80 con 0x7f19100e5470 2021-02-10T11:31:34.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.398 7f1916e86700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1910080310 con 0x7f19101b3e40 2021-02-10T11:31:34.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.398 7f1916e86700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1910080530 con 0x7f19100e5070 2021-02-10T11:31:34.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.398 7f19154a3700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f19100e5470 0x7f191007dcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50656/0 (socket says 172.21.2.101:50656) 2021-02-10T11:31:34.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.398 7f19154a3700 1 -- 172.21.2.101:0/1855379562 learned_addr learned my addr 172.21.2.101:0/1855379562 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:34.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.398 7f190ffff700 1 --2- 172.21.2.101:0/1855379562 >> v2:172.21.2.101:3300/0 conn(0x7f19101b3e40 0x7f191007d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:34.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.399 7f190ffff700 1 -- 172.21.2.101:0/1855379562 >> v2:172.21.2.108:3300/0 conn(0x7f19100e5070 msgr2=0x7f19101b3820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:31:34.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.399 7f190ffff700 1 --2- 172.21.2.101:0/1855379562 >> v2:172.21.2.108:3300/0 conn(0x7f19100e5070 0x7f19101b3820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:34.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.399 7f190ffff700 1 -- 172.21.2.101:0/1855379562 >> v1:172.21.2.101:6789/0 conn(0x7f19100e5470 legacy=0x7f191007dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:34.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.399 7f190ffff700 1 -- 172.21.2.101:0/1855379562 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1910080750 con 0x7f19101b3e40 2021-02-10T11:31:34.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.399 7f190ffff700 1 --2- 172.21.2.101:0/1855379562 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19101b3e40 0x7f191007d690 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 rx=0x7f19000069d0 tx=0x7f1900009e40).ready entity=mon.0 client_cookie=7d567b82fc3585ad server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:34.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.399 7f190f7fe700 1 -- 172.21.2.101:0/1855379562 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f190000b510 con 0x7f19101b3e40 2021-02-10T11:31:34.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.399 7f190f7fe700 1 -- 172.21.2.101:0/1855379562 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190000b790 con 0x7f19101b3e40 2021-02-10T11:31:34.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.399 7f190f7fe700 1 -- 172.21.2.101:0/1855379562 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f190000bcb0 con 0x7f19101b3e40 2021-02-10T11:31:34.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.399 7f1916e86700 1 -- 172.21.2.101:0/1855379562 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19101b3e40 msgr2=0x7f191007d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:34.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.399 7f1916e86700 1 --2- 172.21.2.101:0/1855379562 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19101b3e40 0x7f191007d690 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 rx=0x7f19000069d0 tx=0x7f1900009e40).stop 2021-02-10T11:31:34.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.400 7f1916e86700 1 -- 172.21.2.101:0/1855379562 shutdown_connections 2021-02-10T11:31:34.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.400 7f1916e86700 1 --2- 172.21.2.101:0/1855379562 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19101b3e40 0x7f191007d690 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:34.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.400 7f1916e86700 1 --2- 172.21.2.101:0/1855379562 >> v2:172.21.2.108:3300/0 conn(0x7f19100e5070 0x7f19101b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:34.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.400 7f1916e86700 1 -- 172.21.2.101:0/1855379562 >> 172.21.2.101:0/1855379562 conn(0x7f19100ce690 msgr2=0x7f19100e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:34.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.400 7f1916e86700 1 -- 172.21.2.101:0/1855379562 shutdown_connections 2021-02-10T11:31:34.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.400 7f1916e86700 1 -- 172.21.2.101:0/1855379562 wait complete. 2021-02-10T11:31:34.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.414 7f1916e86700 1 Processor -- start 2021-02-10T11:31:34.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.414 7f1916e86700 1 -- start start 2021-02-10T11:31:34.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.415 7f1916e86700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19101b19d0 0x7f191008cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:34.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.415 7f1916e86700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19101b0d00 0x7f19101b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:34.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.415 7f1916e86700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19101a2b50 0x7f19101a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:34.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.415 7f1916e86700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19101b1720 con 0x7f19101b0d00 2021-02-10T11:31:34.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.415 7f1916e86700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191008c470 con 0x7f19101b19d0 2021-02-10T11:31:34.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.415 7f1916e86700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19101b1f50 con 0x7f19101a2b50 2021-02-10T11:31:34.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.415 7f190ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19101b0d00 0x7f19101b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:34.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.415 7f190ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19101b0d00 0x7f19101b1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50168/0 (socket says 172.21.2.101:50168) 2021-02-10T11:31:34.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.415 7f190ffff700 1 -- 172.21.2.101:0/298498553 learned_addr learned my addr 172.21.2.101:0/298498553 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:34.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.415 7f190ffff700 1 -- 172.21.2.101:0/298498553 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19101a2b50 msgr2=0x7f19101a2f50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:31:34.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.415 7f190ffff700 1 --2- 172.21.2.101:0/298498553 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19101a2b50 0x7f19101a2f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:34.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.415 7f190ffff700 1 -- 172.21.2.101:0/298498553 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19101b19d0 msgr2=0x7f191008cd90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:31:34.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.415 7f190ffff700 1 --2- 172.21.2.101:0/298498553 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19101b19d0 0x7f191008cd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:34.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.415 7f190ffff700 1 -- 172.21.2.101:0/298498553 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19101a3570 con 0x7f19101b0d00 2021-02-10T11:31:34.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.416 7f190ffff700 1 --2- 172.21.2.101:0/298498553 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19101b0d00 0x7f19101b1100 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7f1900009f40 tx=0x7f190000bd50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:34.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.416 7f190d7fa700 1 -- 172.21.2.101:0/298498553 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1900006240 con 0x7f19101b0d00 2021-02-10T11:31:34.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.416 7f190d7fa700 1 -- 172.21.2.101:0/298498553 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19000065c0 con 0x7f19101b0d00 2021-02-10T11:31:34.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.416 7f1916e86700 1 -- 172.21.2.101:0/298498553 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19101a3570 con 0x7f19101b0d00 2021-02-10T11:31:34.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.416 7f1916e86700 1 -- 172.21.2.101:0/298498553 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19100b3330 con 0x7f19101b0d00 2021-02-10T11:31:34.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.416 7f190d7fa700 1 -- 172.21.2.101:0/298498553 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19000058e0 con 0x7f19101b0d00 2021-02-10T11:31:34.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.416 7f190d7fa700 1 -- 172.21.2.101:0/298498553 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1900006ca0 con 0x7f19101b0d00 2021-02-10T11:31:34.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.416 7f190d7fa700 1 --2- 172.21.2.101:0/298498553 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18f40186c0 0x7f18f401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:34.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.416 7f190d7fa700 1 -- 172.21.2.101:0/298498553 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1900032b30 con 0x7f19101b0d00 2021-02-10T11:31:34.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.416 7f1914ca2700 1 --2- 172.21.2.101:0/298498553 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18f40186c0 0x7f18f401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:34.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.417 7f1914ca2700 1 --2- 172.21.2.101:0/298498553 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18f40186c0 0x7f18f401abc0 crc :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:34.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.448 7f1916e86700 1 -- 172.21.2.101:0/298498553 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1910083dd0 con 0x7f19101b0d00 2021-02-10T11:31:34.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.450 7f190d7fa700 1 -- 172.21.2.101:0/298498553 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1900000d30 con 0x7f19101b0d00 2021-02-10T11:31:34.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.579 7f1916e86700 1 -- 172.21.2.101:0/298498553 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1910083dd0 con 0x7f19101b0d00 2021-02-10T11:31:34.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.579 7f190d7fa700 1 -- 172.21.2.101:0/298498553 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+2096 (secure 0 0 0) 0x7f1900008f70 con 0x7f19101b0d00 2021-02-10T11:31:34.591 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:31:34.591 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/243293953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":243293953},{"type":"v1","addr":"172.21.2.101:6835","nonce":243293953}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:31:31.941616","modified":"2021-02-10 11:31:33.951741","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7299":{"gid":7299,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/1600862555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":1600862555},{"type":"v1","addr":"172.21.2.108:6833","nonce":1600862555}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-02-10T11:31:34.591 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 22 2021-02-10T11:31:34.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.581 7f1916e86700 1 -- 172.21.2.101:0/298498553 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18f40186c0 msgr2=0x7f18f401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:34.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.581 7f1916e86700 1 --2- 172.21.2.101:0/298498553 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18f40186c0 0x7f18f401abc0 crc :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:34.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.581 7f1916e86700 1 -- 172.21.2.101:0/298498553 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19101b0d00 msgr2=0x7f19101b1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:34.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.581 7f1916e86700 1 --2- 172.21.2.101:0/298498553 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19101b0d00 0x7f19101b1100 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7f1900009f40 tx=0x7f190000bd50).stop 2021-02-10T11:31:34.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.581 7f1916e86700 1 -- 172.21.2.101:0/298498553 shutdown_connections 2021-02-10T11:31:34.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.581 7f1916e86700 1 --2- 172.21.2.101:0/298498553 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18f40186c0 0x7f18f401abc0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:34.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.581 7f1916e86700 1 --2- 172.21.2.101:0/298498553 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19101a2b50 0x7f19101a2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:34.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.581 7f1916e86700 1 --2- 172.21.2.101:0/298498553 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19101b0d00 0x7f19101b1100 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:34.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.581 7f1916e86700 1 --2- 172.21.2.101:0/298498553 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19101b19d0 0x7f191008cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:34.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.581 7f1916e86700 1 -- 172.21.2.101:0/298498553 >> 172.21.2.101:0/298498553 conn(0x7f1910093720 msgr2=0x7f1910084b80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:34.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.582 7f1916e86700 1 -- 172.21.2.101:0/298498553 shutdown_connections 2021-02-10T11:31:34.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:34.582 7f1916e86700 1 -- 172.21.2.101:0/298498553 wait complete. 2021-02-10T11:31:34.601 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:31:35.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.001 7f081151c700 1 Processor -- start 2021-02-10T11:31:35.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.001 7f081151c700 1 -- start start 2021-02-10T11:31:35.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.001 7f081151c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f080c0e5470 0x7f080c07e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:35.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.001 7f081151c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f080c07f300 0x7f080c07f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:35.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.001 7f081151c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f080c07ed80 con 0x7f080c0e5070 2021-02-10T11:31:35.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.001 7f081151c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f080c1b4010 con 0x7f080c07f300 2021-02-10T11:31:35.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.001 7f081151c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080c0d1790 con 0x7f080c0e5470 2021-02-10T11:31:35.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.002 7f080b2bf700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f080c0e5070 0x7f080c1b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50668/0 (socket says 172.21.2.101:50668) 2021-02-10T11:31:35.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.002 7f080b2bf700 1 -- 172.21.2.101:0/3592580475 learned_addr learned my addr 172.21.2.101:0/3592580475 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:35.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.002 7f080bac0700 1 --2- 172.21.2.101:0/3592580475 >> v2:172.21.2.108:3300/0 conn(0x7f080c07f300 0x7f080c07f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:35.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.002 7f080aabe700 1 --2- 172.21.2.101:0/3592580475 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f080c0e5470 0x7f080c07e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:35.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.002 7f080a2bd700 1 -- 172.21.2.101:0/3592580475 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 40175362 0 0) 0x7f07fc0010b0 con 0x7f080c0e5070 2021-02-10T11:31:35.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.002 7f080a2bd700 1 -- 172.21.2.101:0/3592580475 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07f8002fe0 con 0x7f080c0e5070 2021-02-10T11:31:35.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.002 7f080aabe700 1 -- 172.21.2.101:0/3592580475 >> v2:172.21.2.108:3300/0 conn(0x7f080c07f300 msgr2=0x7f080c07f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:35.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.002 7f080aabe700 1 --2- 172.21.2.101:0/3592580475 >> v2:172.21.2.108:3300/0 conn(0x7f080c07f300 0x7f080c07f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:35.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.002 7f080aabe700 1 -- 172.21.2.101:0/3592580475 >> v1:172.21.2.101:6789/0 conn(0x7f080c0e5070 legacy=0x7f080c1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:35.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.002 7f080aabe700 1 -- 172.21.2.101:0/3592580475 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f080c0d1980 con 0x7f080c0e5470 2021-02-10T11:31:35.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.002 7f080aabe700 1 --2- 172.21.2.101:0/3592580475 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f080c0e5470 0x7f080c07e6a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f07f4006650 tx=0x7f07f4009a80).ready entity=mon.2 client_cookie=2a072d165225f8c4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:35.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.002 7f080a2bd700 1 -- 172.21.2.101:0/3592580475 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f07f40009b0 con 0x7f080c0e5470 2021-02-10T11:31:35.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.003 7f080a2bd700 1 -- 172.21.2.101:0/3592580475 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f400c0c0 con 0x7f080c0e5470 2021-02-10T11:31:35.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.003 7f080a2bd700 1 -- 172.21.2.101:0/3592580475 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f07f400c510 con 0x7f080c0e5470 2021-02-10T11:31:35.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.003 7f081151c700 1 -- 172.21.2.101:0/3592580475 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f080c0e5470 msgr2=0x7f080c07e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:35.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.003 7f081151c700 1 --2- 172.21.2.101:0/3592580475 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f080c0e5470 0x7f080c07e6a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f07f4006650 tx=0x7f07f4009a80).stop 2021-02-10T11:31:35.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.003 7f081151c700 1 -- 172.21.2.101:0/3592580475 shutdown_connections 2021-02-10T11:31:35.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.003 7f081151c700 1 --2- 172.21.2.101:0/3592580475 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f080c0e5470 0x7f080c07e6a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:35.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.003 7f081151c700 1 --2- 172.21.2.101:0/3592580475 >> v2:172.21.2.108:3300/0 conn(0x7f080c07f300 0x7f080c07f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:35.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.003 7f081151c700 1 -- 172.21.2.101:0/3592580475 >> 172.21.2.101:0/3592580475 conn(0x7f080c0ce690 msgr2=0x7f080c0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:35.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.003 7f081151c700 1 -- 172.21.2.101:0/3592580475 shutdown_connections 2021-02-10T11:31:35.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.003 7f081151c700 1 -- 172.21.2.101:0/3592580475 wait complete. 2021-02-10T11:31:35.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.018 7f081151c700 1 Processor -- start 2021-02-10T11:31:35.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.018 7f081151c700 1 -- start start 2021-02-10T11:31:35.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.018 7f081151c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f080c1918e0 0x7f080c189c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:35.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.018 7f081151c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f080c18a2a0 0x7f080c18a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:35.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.018 7f081151c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f080c18acc0 0x7f080c190070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:35.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.018 7f081151c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c1a2f00 con 0x7f080c1918e0 2021-02-10T11:31:35.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.018 7f081151c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c191ce0 con 0x7f080c18acc0 2021-02-10T11:31:35.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.018 7f081151c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080c191f00 con 0x7f080c18a2a0 2021-02-10T11:31:35.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.019 7f080b2bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f080c1918e0 0x7f080c189c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:35.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.019 7f080b2bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f080c1918e0 0x7f080c189c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50180/0 (socket says 172.21.2.101:50180) 2021-02-10T11:31:35.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.019 7f080b2bf700 1 -- 172.21.2.101:0/2222051292 learned_addr learned my addr 172.21.2.101:0/2222051292 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:35.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.019 7f080b2bf700 1 -- 172.21.2.101:0/2222051292 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f080c18a2a0 msgr2=0x7f080c18a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:35.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.019 7f080b2bf700 1 --2- 172.21.2.101:0/2222051292 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f080c18a2a0 0x7f080c18a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:35.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.019 7f080b2bf700 1 -- 172.21.2.101:0/2222051292 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f080c18acc0 msgr2=0x7f080c190070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:35.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.019 7f080b2bf700 1 --2- 172.21.2.101:0/2222051292 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f080c18acc0 0x7f080c190070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:35.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.019 7f080b2bf700 1 -- 172.21.2.101:0/2222051292 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f080c190690 con 0x7f080c1918e0 2021-02-10T11:31:35.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.019 7f080b2bf700 1 --2- 172.21.2.101:0/2222051292 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f080c1918e0 0x7f080c189c80 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7f07fc001c60 tx=0x7f07fc00a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:35.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.019 7f0803fff700 1 -- 172.21.2.101:0/2222051292 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f07fc00acc0 con 0x7f080c1918e0 2021-02-10T11:31:35.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.019 7f0803fff700 1 -- 172.21.2.101:0/2222051292 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07fc004350 con 0x7f080c1918e0 2021-02-10T11:31:35.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.019 7f0803fff700 1 -- 172.21.2.101:0/2222051292 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f07fc0047b0 con 0x7f080c1918e0 2021-02-10T11:31:35.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.019 7f081151c700 1 -- 172.21.2.101:0/2222051292 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f080c190690 con 0x7f080c1918e0 2021-02-10T11:31:35.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.019 7f081151c700 1 -- 172.21.2.101:0/2222051292 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f080c190690 con 0x7f080c1918e0 2021-02-10T11:31:35.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.020 7f0803fff700 1 -- 172.21.2.101:0/2222051292 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f07fc007f30 con 0x7f080c1918e0 2021-02-10T11:31:35.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.020 7f0803fff700 1 --2- 172.21.2.101:0/2222051292 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07f00186c0 0x7f07f001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:35.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.020 7f0803fff700 1 -- 172.21.2.101:0/2222051292 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f07fc032400 con 0x7f080c1918e0 2021-02-10T11:31:35.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.020 7f080aabe700 1 --2- 172.21.2.101:0/2222051292 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07f00186c0 0x7f07f001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:35.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.020 7f080aabe700 1 --2- 172.21.2.101:0/2222051292 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07f00186c0 0x7f07f001abc0 crc :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:35.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.051 7f081151c700 1 -- 172.21.2.101:0/2222051292 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f080c07ed80 con 0x7f080c1918e0 2021-02-10T11:31:35.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.054 7f0803fff700 1 -- 172.21.2.101:0/2222051292 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f07fc004140 con 0x7f080c1918e0 2021-02-10T11:31:35.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.180 7f081151c700 1 -- 172.21.2.101:0/2222051292 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f080c07ed80 con 0x7f080c1918e0 2021-02-10T11:31:35.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.181 7f0803fff700 1 -- 172.21.2.101:0/2222051292 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+13993 (secure 0 0 0) 0x7f07fc007f30 con 0x7f080c1918e0 2021-02-10T11:31:35.192 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:31:35.193 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":38,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:31:32.943916","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:31:29.865604","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2021-02-10 11:31:30.939202","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:31:35.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.183 7f081151c700 1 -- 172.21.2.101:0/2222051292 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07f00186c0 msgr2=0x7f07f001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:35.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.183 7f081151c700 1 --2- 172.21.2.101:0/2222051292 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07f00186c0 0x7f07f001abc0 crc :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:35.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.183 7f081151c700 1 -- 172.21.2.101:0/2222051292 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f080c1918e0 msgr2=0x7f080c189c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:35.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.183 7f081151c700 1 --2- 172.21.2.101:0/2222051292 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f080c1918e0 0x7f080c189c80 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7f07fc001c60 tx=0x7f07fc00a8d0).stop 2021-02-10T11:31:35.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.183 7f081151c700 1 -- 172.21.2.101:0/2222051292 shutdown_connections 2021-02-10T11:31:35.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.183 7f081151c700 1 --2- 172.21.2.101:0/2222051292 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07f00186c0 0x7f07f001abc0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:35.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.183 7f081151c700 1 --2- 172.21.2.101:0/2222051292 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f080c18a2a0 0x7f080c18a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:35.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.183 7f081151c700 1 --2- 172.21.2.101:0/2222051292 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f080c18acc0 0x7f080c190070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:35.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.183 7f081151c700 1 --2- 172.21.2.101:0/2222051292 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f080c1918e0 0x7f080c189c80 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:35.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.183 7f081151c700 1 -- 172.21.2.101:0/2222051292 >> 172.21.2.101:0/2222051292 conn(0x7f080c1a31d0 msgr2=0x7f080c08b830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:35.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.183 7f081151c700 1 -- 172.21.2.101:0/2222051292 shutdown_connections 2021-02-10T11:31:35.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.183 7f081151c700 1 -- 172.21.2.101:0/2222051292 wait complete. 2021-02-10T11:31:35.202 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:31:35.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.603 7f22b0c34700 1 Processor -- start 2021-02-10T11:31:35.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.603 7f22b0c34700 1 -- start start 2021-02-10T11:31:35.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.604 7f22b0c34700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f22ac1b0870 0x7f22ac0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:35.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.604 7f22b0c34700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ac1b0c70 con 0x7f22ac1b0470 2021-02-10T11:31:35.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.604 7f22b0c34700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ac0f0b50 con 0x7f22ac0efe00 2021-02-10T11:31:35.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.604 7f22b0c34700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ac0e5ea0 con 0x7f22ac1b0870 2021-02-10T11:31:35.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.604 7f22aa85e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f22ac1b0470 0x7f22ac0ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50680/0 (socket says 172.21.2.101:50680) 2021-02-10T11:31:35.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.604 7f22aa85e700 1 -- 172.21.2.101:0/3054424646 learned_addr learned my addr 172.21.2.101:0/3054424646 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:35.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.604 7f22aa05d700 1 --2- 172.21.2.101:0/3054424646 >> v2:172.21.2.101:3300/0 conn(0x7f22ac1b0870 0x7f22ac0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:35.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.604 7f22aa05d700 1 -- 172.21.2.101:0/3054424646 >> v1:172.21.2.108:6789/0 conn(0x7f22ac0efe00 legacy=0x7f22ac0f0200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:31:35.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.604 7f22aa05d700 1 -- 172.21.2.101:0/3054424646 >> v1:172.21.2.101:6789/0 conn(0x7f22ac1b0470 legacy=0x7f22ac0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:35.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.604 7f22aa05d700 1 -- 172.21.2.101:0/3054424646 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac0e52e0 con 0x7f22ac1b0870 2021-02-10T11:31:35.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.604 7f22aa05d700 1 --2- 172.21.2.101:0/3054424646 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22ac1b0870 0x7f22ac0f20e0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7f229c006650 tx=0x7f229c009a80).ready entity=mon.0 client_cookie=90f029d74ad7bf02 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:35.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.604 7f22a985c700 1 -- 172.21.2.101:0/3054424646 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f229c0009b0 con 0x7f22ac1b0870 2021-02-10T11:31:35.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.604 7f22a985c700 1 -- 172.21.2.101:0/3054424646 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229c00c0c0 con 0x7f22ac1b0870 2021-02-10T11:31:35.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.604 7f22a985c700 1 -- 172.21.2.101:0/3054424646 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f229c00c5e0 con 0x7f22ac1b0870 2021-02-10T11:31:35.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.604 7f22b0c34700 1 -- 172.21.2.101:0/3054424646 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22ac1b0870 msgr2=0x7f22ac0f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:35.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.604 7f22b0c34700 1 --2- 172.21.2.101:0/3054424646 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22ac1b0870 0x7f22ac0f20e0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7f229c006650 tx=0x7f229c009a80).stop 2021-02-10T11:31:35.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.605 7f22b0c34700 1 -- 172.21.2.101:0/3054424646 shutdown_connections 2021-02-10T11:31:35.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.605 7f22b0c34700 1 --2- 172.21.2.101:0/3054424646 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22ac1b0870 0x7f22ac0f20e0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:35.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.605 7f22b0c34700 1 -- 172.21.2.101:0/3054424646 >> 172.21.2.101:0/3054424646 conn(0x7f22ac0ce3e0 msgr2=0x7f22ac0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:35.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.605 7f22b0c34700 1 -- 172.21.2.101:0/3054424646 shutdown_connections 2021-02-10T11:31:35.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.605 7f22b0c34700 1 -- 172.21.2.101:0/3054424646 wait complete. 2021-02-10T11:31:35.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.620 7f22b0c34700 1 Processor -- start 2021-02-10T11:31:35.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.620 7f22b0c34700 1 -- start start 2021-02-10T11:31:35.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.621 7f22b0c34700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22ac09f300 0x7f22ac0a73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:35.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.621 7f22b0c34700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22ac0aa870 0x7f22ac0aac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:35.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.621 7f22b0c34700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22ac0ab290 0x7f22ac0a8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:35.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.621 7f22b0c34700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac081940 con 0x7f22ac0ab290 2021-02-10T11:31:35.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.621 7f22b0c34700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac0a7bd0 con 0x7f22ac09f300 2021-02-10T11:31:35.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.621 7f22b0c34700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac0ab690 con 0x7f22ac0aa870 2021-02-10T11:31:35.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.621 7f22aa05d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22ac0aa870 0x7f22ac0aac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:35.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.621 7f22aa05d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22ac0aa870 0x7f22ac0aac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:42924/0 (socket says 172.21.2.101:42924) 2021-02-10T11:31:35.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.621 7f22aa05d700 1 -- 172.21.2.101:0/965555388 learned_addr learned my addr 172.21.2.101:0/965555388 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:35.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.621 7f22ab05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22ac0ab290 0x7f22ac0a8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:35.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.621 7f22aa85e700 1 --2- 172.21.2.101:0/965555388 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22ac09f300 0x7f22ac0a73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:35.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.621 7f22ab05f700 1 -- 172.21.2.101:0/965555388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22ac0aa870 msgr2=0x7f22ac0aac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:35.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.621 7f22ab05f700 1 --2- 172.21.2.101:0/965555388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22ac0aa870 0x7f22ac0aac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:35.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.621 7f22ab05f700 1 -- 172.21.2.101:0/965555388 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22ac09f300 msgr2=0x7f22ac0a73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:35.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.621 7f22ab05f700 1 --2- 172.21.2.101:0/965555388 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22ac09f300 0x7f22ac0a73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:35.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.621 7f22ab05f700 1 -- 172.21.2.101:0/965555388 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac09f700 con 0x7f22ac0ab290 2021-02-10T11:31:35.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.621 7f22aa05d700 1 --2- 172.21.2.101:0/965555388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22ac0aa870 0x7f22ac0aac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:31:35.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.621 7f22ab05f700 1 --2- 172.21.2.101:0/965555388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22ac0ab290 0x7f22ac0a8150 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 rx=0x7f22a400b8a0 tx=0x7f22a400bc40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:35.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.622 7f22a37fe700 1 -- 172.21.2.101:0/965555388 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f22a4003030 con 0x7f22ac0ab290 2021-02-10T11:31:35.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.622 7f22a37fe700 1 -- 172.21.2.101:0/965555388 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a4000d30 con 0x7f22ac0ab290 2021-02-10T11:31:35.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.622 7f22a37fe700 1 -- 172.21.2.101:0/965555388 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f22a4001530 con 0x7f22ac0ab290 2021-02-10T11:31:35.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.622 7f22b0c34700 1 -- 172.21.2.101:0/965555388 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ac09f700 con 0x7f22ac0ab290 2021-02-10T11:31:35.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.622 7f22b0c34700 1 -- 172.21.2.101:0/965555388 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ac0a8a60 con 0x7f22ac0ab290 2021-02-10T11:31:35.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.622 7f22a37fe700 1 -- 172.21.2.101:0/965555388 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f22a4007000 con 0x7f22ac0ab290 2021-02-10T11:31:35.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.623 7f22a37fe700 1 --2- 172.21.2.101:0/965555388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22900186c0 0x7f229001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:35.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.623 7f22a37fe700 1 -- 172.21.2.101:0/965555388 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f22a4032880 con 0x7f22ac0ab290 2021-02-10T11:31:35.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.623 7f22aa85e700 1 --2- 172.21.2.101:0/965555388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22900186c0 0x7f229001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:35.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.623 7f22aa85e700 1 --2- 172.21.2.101:0/965555388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22900186c0 0x7f229001abc0 crc :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:35.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.653 7f22b0c34700 1 -- 172.21.2.101:0/965555388 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ac07ed20 con 0x7f22ac0ab290 2021-02-10T11:31:35.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.656 7f22a37fe700 1 -- 172.21.2.101:0/965555388 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f22a4000f40 con 0x7f22ac0ab290 2021-02-10T11:31:35.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.785 7f22b0c34700 1 -- 172.21.2.101:0/965555388 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f22ac0ce3e0 con 0x7f22ac0ab290 2021-02-10T11:31:35.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.788 7f22a37fe700 1 -- 172.21.2.101:0/965555388 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v17) v1 ==== 184+0+0 (secure 0 0 0) 0x7f22a40073b0 con 0x7f22ac0ab290 2021-02-10T11:31:35.800 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.0 2021-02-10T11:31:35.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.790 7f22b0c34700 1 -- 172.21.2.101:0/965555388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22900186c0 msgr2=0x7f229001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:35.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.790 7f22b0c34700 1 --2- 172.21.2.101:0/965555388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22900186c0 0x7f229001abc0 crc :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:35.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.790 7f22b0c34700 1 -- 172.21.2.101:0/965555388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22ac0ab290 msgr2=0x7f22ac0a8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:35.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.790 7f22b0c34700 1 --2- 172.21.2.101:0/965555388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22ac0ab290 0x7f22ac0a8150 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 rx=0x7f22a400b8a0 tx=0x7f22a400bc40).stop 2021-02-10T11:31:35.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.790 7f22b0c34700 1 -- 172.21.2.101:0/965555388 shutdown_connections 2021-02-10T11:31:35.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.790 7f22b0c34700 1 --2- 172.21.2.101:0/965555388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22900186c0 0x7f229001abc0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:35.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.790 7f22b0c34700 1 --2- 172.21.2.101:0/965555388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22ac0aa870 0x7f22ac0aac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:35.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.790 7f22b0c34700 1 --2- 172.21.2.101:0/965555388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22ac0ab290 0x7f22ac0a8150 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:35.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.790 7f22b0c34700 1 --2- 172.21.2.101:0/965555388 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22ac09f300 0x7f22ac0a73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:35.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.790 7f22b0c34700 1 -- 172.21.2.101:0/965555388 >> 172.21.2.101:0/965555388 conn(0x7f22ac09fab0 msgr2=0x7f22ac089220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:35.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.790 7f22b0c34700 1 -- 172.21.2.101:0/965555388 shutdown_connections 2021-02-10T11:31:35.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:35.790 7f22b0c34700 1 -- 172.21.2.101:0/965555388 wait complete. 2021-02-10T11:31:35.812 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:31:36.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.222 7f655dec8700 1 Processor -- start 2021-02-10T11:31:36.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.222 7f655dec8700 1 -- start start 2021-02-10T11:31:36.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.222 7f655dec8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f65581b0870 0x7f65580f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:36.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.222 7f655dec8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65580efe00 0x7f65580f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:36.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.222 7f655dec8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65581b0c70 con 0x7f65581b0470 2021-02-10T11:31:36.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.222 7f655dec8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65580f0b50 con 0x7f65581b0870 2021-02-10T11:31:36.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.222 7f655dec8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65580f0820 con 0x7f65580efe00 2021-02-10T11:31:36.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.223 7f6557fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65580efe00 0x7f65580f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:36.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.223 7f6556ffd700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f65581b0870 0x7f65580f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:36.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.223 7f65577fe700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f65581b0470 0x7f65580ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:44104/0 (socket says 172.21.2.101:44104) 2021-02-10T11:31:36.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.223 7f65577fe700 1 -- 172.21.2.101:0/1720124810 learned_addr learned my addr 172.21.2.101:0/1720124810 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:36.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.224 7f6556ffd700 1 -- 172.21.2.101:0/1720124810 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65580efe00 msgr2=0x7f65580f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:36.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.224 7f6556ffd700 1 --2- 172.21.2.101:0/1720124810 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65580efe00 0x7f65580f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:36.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.224 7f6556ffd700 1 -- 172.21.2.101:0/1720124810 >> v1:172.21.2.108:6789/0 conn(0x7f65581b0470 legacy=0x7f65580ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:36.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.224 7f6556ffd700 1 -- 172.21.2.101:0/1720124810 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65580e52e0 con 0x7f65581b0870 2021-02-10T11:31:36.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.224 7f6556ffd700 1 --2- 172.21.2.101:0/1720124810 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65581b0870 0x7f65580f20e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f6540006650 tx=0x7f6540009a80).ready entity=mon.1 client_cookie=14721bf5cd4b4dbc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:36.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.224 7f65567fc700 1 -- 172.21.2.101:0/1720124810 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f654000c0c0 con 0x7f65581b0870 2021-02-10T11:31:36.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.224 7f65567fc700 1 -- 172.21.2.101:0/1720124810 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654000c340 con 0x7f65581b0870 2021-02-10T11:31:36.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.224 7f65567fc700 1 -- 172.21.2.101:0/1720124810 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f654000c860 con 0x7f65581b0870 2021-02-10T11:31:36.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.224 7f655dec8700 1 -- 172.21.2.101:0/1720124810 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65581b0870 msgr2=0x7f65580f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:36.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.224 7f655dec8700 1 --2- 172.21.2.101:0/1720124810 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65581b0870 0x7f65580f20e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f6540006650 tx=0x7f6540009a80).stop 2021-02-10T11:31:36.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.225 7f655dec8700 1 -- 172.21.2.101:0/1720124810 shutdown_connections 2021-02-10T11:31:36.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.225 7f655dec8700 1 --2- 172.21.2.101:0/1720124810 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65580efe00 0x7f65580f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:36.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.225 7f655dec8700 1 --2- 172.21.2.101:0/1720124810 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65581b0870 0x7f65580f20e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:36.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.225 7f655dec8700 1 -- 172.21.2.101:0/1720124810 >> 172.21.2.101:0/1720124810 conn(0x7f65580ce3e0 msgr2=0x7f65580e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:36.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.225 7f655dec8700 1 -- 172.21.2.101:0/1720124810 shutdown_connections 2021-02-10T11:31:36.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.225 7f655dec8700 1 -- 172.21.2.101:0/1720124810 wait complete. 2021-02-10T11:31:36.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.240 7f655dec8700 1 Processor -- start 2021-02-10T11:31:36.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.240 7f655dec8700 1 -- start start 2021-02-10T11:31:36.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.240 7f655dec8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65580b65c0 0x7f65581a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:36.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.240 7f655dec8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65581a8030 0x7f65580b7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:36.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.240 7f655dec8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65580b69c0 0x7f65580b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:36.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.240 7f655dec8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65580b6ec0 con 0x7f65581a8030 2021-02-10T11:31:36.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.240 7f655dec8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65580b8080 con 0x7f65580b65c0 2021-02-10T11:31:36.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.240 7f655dec8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65580b8260 con 0x7f65580b69c0 2021-02-10T11:31:36.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.240 7f6556ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65581a8030 0x7f65580b7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:36.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.240 7f6556ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65581a8030 0x7f65580b7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50204/0 (socket says 172.21.2.101:50204) 2021-02-10T11:31:36.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.240 7f6556ffd700 1 -- 172.21.2.101:0/3932534449 learned_addr learned my addr 172.21.2.101:0/3932534449 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:36.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.241 7f6556ffd700 1 -- 172.21.2.101:0/3932534449 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65580b69c0 msgr2=0x7f65580b7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:36.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.241 7f6556ffd700 1 --2- 172.21.2.101:0/3932534449 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65580b69c0 0x7f65580b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:36.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.241 7f6556ffd700 1 -- 172.21.2.101:0/3932534449 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65580b65c0 msgr2=0x7f65581a7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:36.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.241 7f6556ffd700 1 --2- 172.21.2.101:0/3932534449 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65580b65c0 0x7f65581a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:36.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.241 7f6556ffd700 1 -- 172.21.2.101:0/3932534449 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65580b8480 con 0x7f65581a8030 2021-02-10T11:31:36.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.241 7f6556ffd700 1 --2- 172.21.2.101:0/3932534449 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65581a8030 0x7f65580b7440 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x7f6540009630 tx=0x7f654000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:36.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.241 7f654ffff700 1 -- 172.21.2.101:0/3932534449 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f65400093a0 con 0x7f65581a8030 2021-02-10T11:31:36.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.241 7f654ffff700 1 -- 172.21.2.101:0/3932534449 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654000bd70 con 0x7f65581a8030 2021-02-10T11:31:36.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.241 7f655dec8700 1 -- 172.21.2.101:0/3932534449 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65580b8480 con 0x7f65581a8030 2021-02-10T11:31:36.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.241 7f655dec8700 1 -- 172.21.2.101:0/3932534449 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65580bc420 con 0x7f65581a8030 2021-02-10T11:31:36.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.241 7f654ffff700 1 -- 172.21.2.101:0/3932534449 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6540005560 con 0x7f65581a8030 2021-02-10T11:31:36.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.242 7f654ffff700 1 -- 172.21.2.101:0/3932534449 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f654000a000 con 0x7f65581a8030 2021-02-10T11:31:36.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.242 7f654ffff700 1 --2- 172.21.2.101:0/3932534449 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f653c018730 0x7f653c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:36.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.242 7f654ffff700 1 -- 172.21.2.101:0/3932534449 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6540032790 con 0x7f65581a8030 2021-02-10T11:31:36.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.242 7f65577fe700 1 --2- 172.21.2.101:0/3932534449 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f653c018730 0x7f653c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:36.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.242 7f65577fe700 1 --2- 172.21.2.101:0/3932534449 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f653c018730 0x7f653c01ac10 crc :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:36.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.272 7f655dec8700 1 -- 172.21.2.101:0/3932534449 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65581059f0 con 0x7f65581a8030 2021-02-10T11:31:36.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.274 7f654ffff700 1 -- 172.21.2.101:0/3932534449 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6540009340 con 0x7f65581a8030 2021-02-10T11:31:36.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.405 7f655dec8700 1 -- 172.21.2.101:0/3932534449 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f65580f0b50 con 0x7f65581a8030 2021-02-10T11:31:36.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.409 7f654ffff700 1 -- 172.21.2.101:0/3932534449 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v18) v1 ==== 184+0+0 (secure 0 0 0) 0x7f654000a000 con 0x7f65581a8030 2021-02-10T11:31:36.423 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.1 2021-02-10T11:31:36.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.414 7f655dec8700 1 -- 172.21.2.101:0/3932534449 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f653c018730 msgr2=0x7f653c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:36.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.414 7f655dec8700 1 --2- 172.21.2.101:0/3932534449 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f653c018730 0x7f653c01ac10 crc :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:36.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.414 7f655dec8700 1 -- 172.21.2.101:0/3932534449 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65581a8030 msgr2=0x7f65580b7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:36.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.414 7f655dec8700 1 --2- 172.21.2.101:0/3932534449 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65581a8030 0x7f65580b7440 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x7f6540009630 tx=0x7f654000b980).stop 2021-02-10T11:31:36.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.414 7f655dec8700 1 -- 172.21.2.101:0/3932534449 shutdown_connections 2021-02-10T11:31:36.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.414 7f655dec8700 1 --2- 172.21.2.101:0/3932534449 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f653c018730 0x7f653c01ac10 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:36.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.414 7f655dec8700 1 --2- 172.21.2.101:0/3932534449 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65580b69c0 0x7f65580b7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:36.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.414 7f655dec8700 1 --2- 172.21.2.101:0/3932534449 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65581a8030 0x7f65580b7440 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:36.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.414 7f655dec8700 1 --2- 172.21.2.101:0/3932534449 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65580b65c0 0x7f65581a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:36.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.414 7f655dec8700 1 -- 172.21.2.101:0/3932534449 >> 172.21.2.101:0/3932534449 conn(0x7f65580b5850 msgr2=0x7f6558093a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:36.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.415 7f655dec8700 1 -- 172.21.2.101:0/3932534449 shutdown_connections 2021-02-10T11:31:36.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.415 7f655dec8700 1 -- 172.21.2.101:0/3932534449 wait complete. 2021-02-10T11:31:36.434 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:31:36.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.845 7f661a6f2700 1 Processor -- start 2021-02-10T11:31:36.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.845 7f661a6f2700 1 -- start start 2021-02-10T11:31:36.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.845 7f661a6f2700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f66140e5470 0x7f661407e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:36.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.845 7f661a6f2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f661407f300 0x7f661407f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:36.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.845 7f661a6f2700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661407ed80 con 0x7f66140e5070 2021-02-10T11:31:36.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.845 7f661a6f2700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66141b4010 con 0x7f66140e5470 2021-02-10T11:31:36.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.845 7f661a6f2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66140d1790 con 0x7f661407f300 2021-02-10T11:31:36.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.845 7f6613fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f66140e5070 0x7f66141b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50706/0 (socket says 172.21.2.101:50706) 2021-02-10T11:31:36.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.845 7f6613fff700 1 -- 172.21.2.101:0/1639698127 learned_addr learned my addr 172.21.2.101:0/1639698127 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:36.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.846 7f6612ffd700 1 -- 172.21.2.101:0/1639698127 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3400368099 0 0) 0x7f66040009b0 con 0x7f66140e5070 2021-02-10T11:31:36.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.846 7f6612ffd700 1 -- 172.21.2.101:0/1639698127 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65fc002fe0 con 0x7f66140e5070 2021-02-10T11:31:36.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.846 7f6618d0f700 1 --2- 172.21.2.101:0/1639698127 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f661407f300 0x7f661407f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:36.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.846 7f66137fe700 1 --2- 172.21.2.101:0/1639698127 >> v2:172.21.2.108:3300/0 conn(0x7f66140e5470 0x7f661407e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:36.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.846 7f6612ffd700 1 -- 172.21.2.101:0/1639698127 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 848156883 0 0) 0x7f66040010b0 con 0x7f66140e5070 2021-02-10T11:31:36.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.846 7f6612ffd700 1 -- 172.21.2.101:0/1639698127 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f661407f300 msgr2=0x7f661407f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:36.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.846 7f6612ffd700 1 --2- 172.21.2.101:0/1639698127 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f661407f300 0x7f661407f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:36.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.846 7f6612ffd700 1 -- 172.21.2.101:0/1639698127 >> v2:172.21.2.108:3300/0 conn(0x7f66140e5470 msgr2=0x7f661407e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.846 7f6612ffd700 1 --2- 172.21.2.101:0/1639698127 >> v2:172.21.2.108:3300/0 conn(0x7f66140e5470 0x7f661407e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.846 7f6612ffd700 1 -- 172.21.2.101:0/1639698127 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66140d19b0 con 0x7f66140e5070 2021-02-10T11:31:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.846 7f6612ffd700 1 -- 172.21.2.101:0/1639698127 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f66040015b0 con 0x7f66140e5070 2021-02-10T11:31:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.846 7f6612ffd700 1 -- 172.21.2.101:0/1639698127 >> v1:172.21.2.101:6789/0 conn(0x7f66140e5070 legacy=0x7f66141b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.846 7f6612ffd700 1 --2- 172.21.2.101:0/1639698127 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65fc003b90 0x7f65fc005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.846 7f6612ffd700 1 --2- 172.21.2.101:0/1639698127 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65fc006820 0x7f65fc008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.846 7f6612ffd700 1 --2- 172.21.2.101:0/1639698127 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65fc04d650 0x7f65fc04fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.846 7f6612ffd700 1 -- 172.21.2.101:0/1639698127 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc050390 con 0x7f65fc006820 2021-02-10T11:31:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.847 7f6612ffd700 1 -- 172.21.2.101:0/1639698127 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc0505b0 con 0x7f65fc003b90 2021-02-10T11:31:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.847 7f6612ffd700 1 -- 172.21.2.101:0/1639698127 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc0507d0 con 0x7f65fc04d650 2021-02-10T11:31:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.847 7f66137fe700 1 --2- 172.21.2.101:0/1639698127 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65fc006820 0x7f65fc008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.847 7f6618d0f700 1 --2- 172.21.2.101:0/1639698127 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f661407f300 0x7f661407f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:31:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.847 7f6613fff700 1 --2- 172.21.2.101:0/1639698127 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65fc003b90 0x7f65fc005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.847 7f66137fe700 1 -- 172.21.2.101:0/1639698127 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65fc04d650 msgr2=0x7f65fc04fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.847 7f66137fe700 1 --2- 172.21.2.101:0/1639698127 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65fc04d650 0x7f65fc04fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.847 7f66137fe700 1 -- 172.21.2.101:0/1639698127 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65fc003b90 msgr2=0x7f65fc005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.847 7f66137fe700 1 --2- 172.21.2.101:0/1639698127 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65fc003b90 0x7f65fc005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.847 7f66137fe700 1 -- 172.21.2.101:0/1639698127 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65fc050a70 con 0x7f65fc006820 2021-02-10T11:31:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.847 7f66137fe700 1 --2- 172.21.2.101:0/1639698127 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65fc006820 0x7f65fc008d40 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 rx=0x7f660800c8f0 tx=0x7f660800cc90).ready entity=mon.0 client_cookie=f6caf46231bcdb42 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.847 7f6612ffd700 1 -- 172.21.2.101:0/1639698127 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f660800cdd0 con 0x7f65fc006820 2021-02-10T11:31:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.847 7f6612ffd700 1 -- 172.21.2.101:0/1639698127 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6608009310 con 0x7f65fc006820 2021-02-10T11:31:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.847 7f6612ffd700 1 -- 172.21.2.101:0/1639698127 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6608009830 con 0x7f65fc006820 2021-02-10T11:31:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.847 7f661a6f2700 1 -- 172.21.2.101:0/1639698127 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65fc006820 msgr2=0x7f65fc008d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.847 7f661a6f2700 1 --2- 172.21.2.101:0/1639698127 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65fc006820 0x7f65fc008d40 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 rx=0x7f660800c8f0 tx=0x7f660800cc90).stop 2021-02-10T11:31:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.847 7f6618d0f700 1 -- 172.21.2.101:0/1639698127 reap_dead start 2021-02-10T11:31:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.848 7f661a6f2700 1 -- 172.21.2.101:0/1639698127 shutdown_connections 2021-02-10T11:31:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.848 7f661a6f2700 1 -- 172.21.2.101:0/1639698127 >> 172.21.2.101:0/1639698127 conn(0x7f66140ce690 msgr2=0x7f66140e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:36.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.848 7f661a6f2700 1 -- 172.21.2.101:0/1639698127 shutdown_connections 2021-02-10T11:31:36.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.848 7f661a6f2700 1 -- 172.21.2.101:0/1639698127 wait complete. 2021-02-10T11:31:36.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.863 7f661a6f2700 1 Processor -- start 2021-02-10T11:31:36.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.863 7f661a6f2700 1 -- start start 2021-02-10T11:31:36.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.863 7f661a6f2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f661419e8c0 0x7f661408cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.863 7f661a6f2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66141acd60 0x7f66141ad160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.863 7f661a6f2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66141adf40 0x7f66141ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.863 7f661a6f2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66141ada40 con 0x7f66141adf40 2021-02-10T11:31:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.863 7f661a6f2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661419ee80 con 0x7f661419e8c0 2021-02-10T11:31:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.863 7f661a6f2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66140b5640 con 0x7f66141acd60 2021-02-10T11:31:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.863 7f6618d0f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66141adf40 0x7f66141ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.863 7f6618d0f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66141adf40 0x7f66141ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50224/0 (socket says 172.21.2.101:50224) 2021-02-10T11:31:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.863 7f6618d0f700 1 -- 172.21.2.101:0/4243326771 learned_addr learned my addr 172.21.2.101:0/4243326771 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.864 7f66137fe700 1 --2- 172.21.2.101:0/4243326771 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66141acd60 0x7f66141ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.864 7f6613fff700 1 --2- 172.21.2.101:0/4243326771 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f661419e8c0 0x7f661408cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.864 7f6618d0f700 1 -- 172.21.2.101:0/4243326771 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66141acd60 msgr2=0x7f66141ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.864 7f6618d0f700 1 --2- 172.21.2.101:0/4243326771 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66141acd60 0x7f66141ad160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.864 7f6618d0f700 1 -- 172.21.2.101:0/4243326771 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f661419e8c0 msgr2=0x7f661408cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:36.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.864 7f6618d0f700 1 --2- 172.21.2.101:0/4243326771 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f661419e8c0 0x7f661408cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:36.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.864 7f6618d0f700 1 -- 172.21.2.101:0/4243326771 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661408d2b0 con 0x7f66141adf40 2021-02-10T11:31:36.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.864 7f6618d0f700 1 --2- 172.21.2.101:0/4243326771 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66141adf40 0x7f66141ae340 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 rx=0x7f660c0099a0 tx=0x7f660c00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:36.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.864 7f6610ff9700 1 -- 172.21.2.101:0/4243326771 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f660c0069c0 con 0x7f66141adf40 2021-02-10T11:31:36.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.864 7f6610ff9700 1 -- 172.21.2.101:0/4243326771 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660c002030 con 0x7f66141adf40 2021-02-10T11:31:36.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.864 7f6610ff9700 1 -- 172.21.2.101:0/4243326771 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f660c002480 con 0x7f66141adf40 2021-02-10T11:31:36.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.864 7f661a6f2700 1 -- 172.21.2.101:0/4243326771 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f661408d2b0 con 0x7f66141adf40 2021-02-10T11:31:36.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.864 7f661a6f2700 1 -- 172.21.2.101:0/4243326771 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f661408d2b0 con 0x7f66141adf40 2021-02-10T11:31:36.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.865 7f6610ff9700 1 -- 172.21.2.101:0/4243326771 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f660c002210 con 0x7f66141adf40 2021-02-10T11:31:36.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.865 7f6610ff9700 1 --2- 172.21.2.101:0/4243326771 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65f40186c0 0x7f65f401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:36.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.865 7f6610ff9700 1 -- 172.21.2.101:0/4243326771 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f660c0329f0 con 0x7f66141adf40 2021-02-10T11:31:36.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.865 7f6613fff700 1 --2- 172.21.2.101:0/4243326771 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65f40186c0 0x7f65f401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:36.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.865 7f6613fff700 1 --2- 172.21.2.101:0/4243326771 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65f40186c0 0x7f65f401abc0 crc :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:36.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.895 7f661a6f2700 1 -- 172.21.2.101:0/4243326771 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f661407ed80 con 0x7f66141adf40 2021-02-10T11:31:36.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:36.897 7f6610ff9700 1 -- 172.21.2.101:0/4243326771 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f660c00be30 con 0x7f66141adf40 2021-02-10T11:31:37.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.027 7f661a6f2700 1 -- 172.21.2.101:0/4243326771 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f661407ed80 con 0x7f66141adf40 2021-02-10T11:31:37.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.030 7f6610ff9700 1 -- 172.21.2.101:0/4243326771 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v19) v1 ==== 184+0+0 (secure 0 0 0) 0x7f660c002030 con 0x7f66141adf40 2021-02-10T11:31:37.045 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.2 2021-02-10T11:31:37.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.035 7f661a6f2700 1 -- 172.21.2.101:0/4243326771 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65f40186c0 msgr2=0x7f65f401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:37.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.035 7f661a6f2700 1 --2- 172.21.2.101:0/4243326771 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65f40186c0 0x7f65f401abc0 crc :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:37.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.036 7f661a6f2700 1 -- 172.21.2.101:0/4243326771 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66141adf40 msgr2=0x7f66141ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:37.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.036 7f661a6f2700 1 --2- 172.21.2.101:0/4243326771 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66141adf40 0x7f66141ae340 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 rx=0x7f660c0099a0 tx=0x7f660c00bcf0).stop 2021-02-10T11:31:37.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.036 7f661a6f2700 1 -- 172.21.2.101:0/4243326771 shutdown_connections 2021-02-10T11:31:37.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.036 7f661a6f2700 1 --2- 172.21.2.101:0/4243326771 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65f40186c0 0x7f65f401abc0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:37.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.036 7f661a6f2700 1 --2- 172.21.2.101:0/4243326771 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66141acd60 0x7f66141ad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:37.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.036 7f661a6f2700 1 --2- 172.21.2.101:0/4243326771 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66141adf40 0x7f66141ae340 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:37.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.036 7f661a6f2700 1 --2- 172.21.2.101:0/4243326771 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f661419e8c0 0x7f661408cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:37.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.036 7f661a6f2700 1 -- 172.21.2.101:0/4243326771 >> 172.21.2.101:0/4243326771 conn(0x7f6614086620 msgr2=0x7f66141909f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:37.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.036 7f661a6f2700 1 -- 172.21.2.101:0/4243326771 shutdown_connections 2021-02-10T11:31:37.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.036 7f661a6f2700 1 -- 172.21.2.101:0/4243326771 wait complete. 2021-02-10T11:31:37.055 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:31:37.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.457 7fb9cd939700 1 Processor -- start 2021-02-10T11:31:37.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.457 7fb9cd939700 1 -- start start 2021-02-10T11:31:37.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.457 7fb9cd939700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb9c81b0470 0x7fb9c81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:37.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.457 7fb9cd939700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c80f0b50 con 0x7fb9c80ef610 2021-02-10T11:31:37.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.457 7fb9cd939700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c80f24e0 con 0x7fb9c80f20e0 2021-02-10T11:31:37.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.457 7fb9cd939700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9c80e5ea0 con 0x7fb9c81b0470 2021-02-10T11:31:37.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.458 7fb9c6d73700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb9c80ef610 0x7fb9c80efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50726/0 (socket says 172.21.2.101:50726) 2021-02-10T11:31:37.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.458 7fb9c6d73700 1 -- 172.21.2.101:0/3936288061 learned_addr learned my addr 172.21.2.101:0/3936288061 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:37.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.458 7fb9c6572700 1 -- 172.21.2.101:0/3936288061 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2135853526 0 0) 0x7fb9b80010b0 con 0x7fb9c80ef610 2021-02-10T11:31:37.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.458 7fb9c6572700 1 -- 172.21.2.101:0/3936288061 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9b4002fe0 con 0x7fb9c80ef610 2021-02-10T11:31:37.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.458 7fb9c7574700 1 --2- 172.21.2.101:0/3936288061 >> v2:172.21.2.108:3300/0 conn(0x7fb9c81b0470 0x7fb9c81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:37.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.458 7fb9c6572700 1 -- 172.21.2.101:0/3936288061 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1089879934 0 0) 0x7fb9b80010b0 con 0x7fb9c80ef610 2021-02-10T11:31:37.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c6572700 1 -- 172.21.2.101:0/3936288061 >> v2:172.21.2.108:3300/0 conn(0x7fb9c81b0470 msgr2=0x7fb9c81b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:37.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c6572700 1 --2- 172.21.2.101:0/3936288061 >> v2:172.21.2.108:3300/0 conn(0x7fb9c81b0470 0x7fb9c81b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:37.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c6572700 1 -- 172.21.2.101:0/3936288061 >> v1:172.21.2.108:6789/0 conn(0x7fb9c80f20e0 legacy=0x7fb9c80f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:37.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c6572700 1 -- 172.21.2.101:0/3936288061 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c80e52e0 con 0x7fb9c80ef610 2021-02-10T11:31:37.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c6572700 1 -- 172.21.2.101:0/3936288061 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fb9b80015b0 con 0x7fb9c80ef610 2021-02-10T11:31:37.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c6572700 1 -- 172.21.2.101:0/3936288061 >> v1:172.21.2.101:6789/0 conn(0x7fb9c80ef610 legacy=0x7fb9c80efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c6572700 1 --2- 172.21.2.101:0/3936288061 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9b4003ac0 0x7fb9b4005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c6572700 1 --2- 172.21.2.101:0/3936288061 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9b4006780 0x7fb9b4008ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c6572700 1 --2- 172.21.2.101:0/3936288061 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9b40095a0 0x7fb9b400baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c6572700 1 -- 172.21.2.101:0/3936288061 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b400c2f0 con 0x7fb9b4006780 2021-02-10T11:31:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c6572700 1 -- 172.21.2.101:0/3936288061 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b400c510 con 0x7fb9b4003ac0 2021-02-10T11:31:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c6572700 1 -- 172.21.2.101:0/3936288061 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b400c730 con 0x7fb9b40095a0 2021-02-10T11:31:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c7574700 1 --2- 172.21.2.101:0/3936288061 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9b4003ac0 0x7fb9b4005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c6d73700 1 --2- 172.21.2.101:0/3936288061 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9b4006780 0x7fb9b4008ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c7d75700 1 --2- 172.21.2.101:0/3936288061 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9b40095a0 0x7fb9b400baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c6d73700 1 -- 172.21.2.101:0/3936288061 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9b40095a0 msgr2=0x7fb9b400baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c6d73700 1 --2- 172.21.2.101:0/3936288061 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9b40095a0 0x7fb9b400baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c6d73700 1 -- 172.21.2.101:0/3936288061 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9b4003ac0 msgr2=0x7fb9b4005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c6d73700 1 --2- 172.21.2.101:0/3936288061 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9b4003ac0 0x7fb9b4005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c6d73700 1 -- 172.21.2.101:0/3936288061 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b40509a0 con 0x7fb9b4006780 2021-02-10T11:31:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c7d75700 1 -- 172.21.2.101:0/3936288061 reap_dead start 2021-02-10T11:31:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c6d73700 1 --2- 172.21.2.101:0/3936288061 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9b4006780 0x7fb9b4008ca0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 rx=0x7fb9b8007990 tx=0x7fb9b8007d30).ready entity=mon.0 client_cookie=ae4e0e12d3c68a7a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.459 7fb9c6572700 1 -- 172.21.2.101:0/3936288061 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb9b80009b0 con 0x7fb9b4006780 2021-02-10T11:31:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.460 7fb9c6572700 1 -- 172.21.2.101:0/3936288061 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b800a310 con 0x7fb9b4006780 2021-02-10T11:31:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.460 7fb9c6572700 1 -- 172.21.2.101:0/3936288061 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb9b800a760 con 0x7fb9b4006780 2021-02-10T11:31:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.460 7fb9cd939700 1 -- 172.21.2.101:0/3936288061 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9b4006780 msgr2=0x7fb9b4008ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.460 7fb9cd939700 1 --2- 172.21.2.101:0/3936288061 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9b4006780 0x7fb9b4008ca0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 rx=0x7fb9b8007990 tx=0x7fb9b8007d30).stop 2021-02-10T11:31:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.460 7fb9cd939700 1 -- 172.21.2.101:0/3936288061 shutdown_connections 2021-02-10T11:31:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.460 7fb9cd939700 1 --2- 172.21.2.101:0/3936288061 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9b4006780 0x7fb9b4008ca0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.460 7fb9cd939700 1 -- 172.21.2.101:0/3936288061 >> 172.21.2.101:0/3936288061 conn(0x7fb9c80ce3e0 msgr2=0x7fb9c80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.460 7fb9cd939700 1 -- 172.21.2.101:0/3936288061 shutdown_connections 2021-02-10T11:31:37.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.460 7fb9cd939700 1 -- 172.21.2.101:0/3936288061 wait complete. 2021-02-10T11:31:37.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.475 7fb9cd939700 1 Processor -- start 2021-02-10T11:31:37.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.475 7fb9cd939700 1 -- start start 2021-02-10T11:31:37.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.475 7fb9cd939700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9c81a7960 0x7fb9c81a7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:37.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.475 7fb9cd939700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9c808c440 0x7fb9c8189710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:37.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.475 7fb9cd939700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9c8189d30 0x7fb9c818a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:37.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.475 7fb9cd939700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c818a850 con 0x7fb9c81a7960 2021-02-10T11:31:37.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.475 7fb9cd939700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c808c840 con 0x7fb9c8189d30 2021-02-10T11:31:37.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.475 7fb9cd939700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c808cbb0 con 0x7fb9c808c440 2021-02-10T11:31:37.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.475 7fb9c7574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9c81a7960 0x7fb9c81a7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:37.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.475 7fb9c7574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9c81a7960 0x7fb9c81a7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50242/0 (socket says 172.21.2.101:50242) 2021-02-10T11:31:37.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.475 7fb9c7574700 1 -- 172.21.2.101:0/157103391 learned_addr learned my addr 172.21.2.101:0/157103391 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:37.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.476 7fb9c7574700 1 -- 172.21.2.101:0/157103391 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9c808c440 msgr2=0x7fb9c8189710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:37.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.476 7fb9c7574700 1 --2- 172.21.2.101:0/157103391 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9c808c440 0x7fb9c8189710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:37.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.476 7fb9c7574700 1 -- 172.21.2.101:0/157103391 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9c8189d30 msgr2=0x7fb9c818a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:37.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.476 7fb9c7574700 1 --2- 172.21.2.101:0/157103391 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9c8189d30 0x7fb9c818a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:37.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.476 7fb9c7574700 1 -- 172.21.2.101:0/157103391 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c818ad50 con 0x7fb9c81a7960 2021-02-10T11:31:37.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.476 7fb9c7574700 1 --2- 172.21.2.101:0/157103391 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9c81a7960 0x7fb9c81a7d60 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 rx=0x7fb9b0006f60 tx=0x7fb9b000b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:37.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.476 7fb9bffff700 1 -- 172.21.2.101:0/157103391 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb9b000bd60 con 0x7fb9c81a7960 2021-02-10T11:31:37.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.476 7fb9bffff700 1 -- 172.21.2.101:0/157103391 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b0000c90 con 0x7fb9c81a7960 2021-02-10T11:31:37.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.476 7fb9cd939700 1 -- 172.21.2.101:0/157103391 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c818ad50 con 0x7fb9c81a7960 2021-02-10T11:31:37.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.476 7fb9cd939700 1 -- 172.21.2.101:0/157103391 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c818b2e0 con 0x7fb9c81a7960 2021-02-10T11:31:37.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.476 7fb9bffff700 1 -- 172.21.2.101:0/157103391 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb9b00022b0 con 0x7fb9c81a7960 2021-02-10T11:31:37.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.477 7fb9bffff700 1 -- 172.21.2.101:0/157103391 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb9b0006b00 con 0x7fb9c81a7960 2021-02-10T11:31:37.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.477 7fb9bffff700 1 --2- 172.21.2.101:0/157103391 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9ac0186c0 0x7fb9ac01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:37.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.477 7fb9bffff700 1 -- 172.21.2.101:0/157103391 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb9b0032550 con 0x7fb9c81a7960 2021-02-10T11:31:37.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.477 7fb9c6d73700 1 --2- 172.21.2.101:0/157103391 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9ac0186c0 0x7fb9ac01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:37.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.477 7fb9c6d73700 1 --2- 172.21.2.101:0/157103391 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9ac0186c0 0x7fb9ac01abc0 crc :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:37.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.508 7fb9cd939700 1 -- 172.21.2.101:0/157103391 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c8105a30 con 0x7fb9c81a7960 2021-02-10T11:31:37.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.510 7fb9bffff700 1 -- 172.21.2.101:0/157103391 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb9b0000960 con 0x7fb9c81a7960 2021-02-10T11:31:37.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.641 7fb9cd939700 1 -- 172.21.2.101:0/157103391 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fb9c80f0b50 con 0x7fb9c81a7960 2021-02-10T11:31:37.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.645 7fb9bffff700 1 -- 172.21.2.101:0/157103391 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v20) v1 ==== 184+0+0 (secure 0 0 0) 0x7fb9b001f4e0 con 0x7fb9c81a7960 2021-02-10T11:31:37.659 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.3 2021-02-10T11:31:37.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.650 7fb9cd939700 1 -- 172.21.2.101:0/157103391 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9ac0186c0 msgr2=0x7fb9ac01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:37.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.650 7fb9cd939700 1 --2- 172.21.2.101:0/157103391 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9ac0186c0 0x7fb9ac01abc0 crc :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:37.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.650 7fb9cd939700 1 -- 172.21.2.101:0/157103391 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9c81a7960 msgr2=0x7fb9c81a7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:37.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.650 7fb9cd939700 1 --2- 172.21.2.101:0/157103391 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9c81a7960 0x7fb9c81a7d60 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 rx=0x7fb9b0006f60 tx=0x7fb9b000b970).stop 2021-02-10T11:31:37.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.650 7fb9cd939700 1 -- 172.21.2.101:0/157103391 shutdown_connections 2021-02-10T11:31:37.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.650 7fb9cd939700 1 --2- 172.21.2.101:0/157103391 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9ac0186c0 0x7fb9ac01abc0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:37.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.650 7fb9cd939700 1 --2- 172.21.2.101:0/157103391 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9c808c440 0x7fb9c8189710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:37.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.650 7fb9cd939700 1 --2- 172.21.2.101:0/157103391 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9c8189d30 0x7fb9c818a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:37.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.650 7fb9cd939700 1 --2- 172.21.2.101:0/157103391 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9c81a7960 0x7fb9c81a7d60 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:37.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.650 7fb9cd939700 1 -- 172.21.2.101:0/157103391 >> 172.21.2.101:0/157103391 conn(0x7fb9c80b5850 msgr2=0x7fb9c8091a50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:37.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.650 7fb9cd939700 1 -- 172.21.2.101:0/157103391 shutdown_connections 2021-02-10T11:31:37.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:37.650 7fb9cd939700 1 -- 172.21.2.101:0/157103391 wait complete. 2021-02-10T11:31:37.669 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:31:38.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.068 7ff1f1096700 1 Processor -- start 2021-02-10T11:31:38.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.068 7ff1f1096700 1 -- start start 2021-02-10T11:31:38.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.068 7ff1f1096700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ec0e5070 0x7ff1ec1b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:38.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.068 7ff1f1096700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff1ec1b3e40 0x7ff1ec080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:38.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.068 7ff1f1096700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ec07ed80 con 0x7ff1ec0e5470 2021-02-10T11:31:38.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.068 7ff1f1096700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ec0d1790 con 0x7ff1ec1b3e40 2021-02-10T11:31:38.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.068 7ff1f1096700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec0d1980 con 0x7ff1ec0e5070 2021-02-10T11:31:38.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.069 7ff1eb59e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff1ec1b3e40 0x7ff1ec080310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:38.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.069 7ff1eb59e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff1ec1b3e40 0x7ff1ec080310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50250/0 (socket says 172.21.2.101:50250) 2021-02-10T11:31:38.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.069 7ff1eb59e700 1 -- 172.21.2.101:0/2070656499 learned_addr learned my addr 172.21.2.101:0/2070656499 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:38.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.069 7ff1eb59e700 1 -- 172.21.2.101:0/2070656499 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ec0e5070 msgr2=0x7ff1ec1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:38.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.069 7ff1eb59e700 1 --2- 172.21.2.101:0/2070656499 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ec0e5070 0x7ff1ec1b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:38.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.069 7ff1eb59e700 1 -- 172.21.2.101:0/2070656499 >> v1:172.21.2.108:6789/0 conn(0x7ff1ec0e5470 legacy=0x7ff1ec07d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:38.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.069 7ff1eb59e700 1 -- 172.21.2.101:0/2070656499 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec0d1ba0 con 0x7ff1ec1b3e40 2021-02-10T11:31:38.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.069 7ff1eb59e700 1 --2- 172.21.2.101:0/2070656499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec1b3e40 0x7ff1ec080310 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0x7ff1e4006f00 tx=0x7ff1e4009f10).ready entity=mon.0 client_cookie=6139bf2b2785ef67 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:38.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.069 7ff1ea59c700 1 -- 172.21.2.101:0/2070656499 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1e400b3b0 con 0x7ff1ec1b3e40 2021-02-10T11:31:38.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.069 7ff1ea59c700 1 -- 172.21.2.101:0/2070656499 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e400b730 con 0x7ff1ec1b3e40 2021-02-10T11:31:38.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.069 7ff1ea59c700 1 -- 172.21.2.101:0/2070656499 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1e400bc10 con 0x7ff1ec1b3e40 2021-02-10T11:31:38.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.069 7ff1f1096700 1 -- 172.21.2.101:0/2070656499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec1b3e40 msgr2=0x7ff1ec080310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:38.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.069 7ff1f1096700 1 --2- 172.21.2.101:0/2070656499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec1b3e40 0x7ff1ec080310 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0x7ff1e4006f00 tx=0x7ff1e4009f10).stop 2021-02-10T11:31:38.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.070 7ff1f1096700 1 -- 172.21.2.101:0/2070656499 shutdown_connections 2021-02-10T11:31:38.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.070 7ff1f1096700 1 --2- 172.21.2.101:0/2070656499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec1b3e40 0x7ff1ec080310 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:38.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.070 7ff1f1096700 1 --2- 172.21.2.101:0/2070656499 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ec0e5070 0x7ff1ec1b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:38.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.070 7ff1f1096700 1 -- 172.21.2.101:0/2070656499 >> 172.21.2.101:0/2070656499 conn(0x7ff1ec0ce690 msgr2=0x7ff1ec0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:38.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.070 7ff1f1096700 1 -- 172.21.2.101:0/2070656499 shutdown_connections 2021-02-10T11:31:38.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.070 7ff1f1096700 1 -- 172.21.2.101:0/2070656499 wait complete. 2021-02-10T11:31:38.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.085 7ff1f1096700 1 Processor -- start 2021-02-10T11:31:38.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.085 7ff1f1096700 1 -- start start 2021-02-10T11:31:38.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.086 7ff1f1096700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec0b4870 0x7ff1ec09dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:38.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.086 7ff1f1096700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1ec1aac10 0x7ff1ec09ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:38.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.086 7ff1f1096700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ec09e2d0 0x7ff1ec09d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:38.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.086 7ff1f1096700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec192410 con 0x7ff1ec0b4870 2021-02-10T11:31:38.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.086 7ff1f1096700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec192630 con 0x7ff1ec1aac10 2021-02-10T11:31:38.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.086 7ff1f1096700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec192850 con 0x7ff1ec09e2d0 2021-02-10T11:31:38.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.086 7ff1ead9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec0b4870 0x7ff1ec09dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:38.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.086 7ff1ead9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec0b4870 0x7ff1ec09dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50252/0 (socket says 172.21.2.101:50252) 2021-02-10T11:31:38.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.086 7ff1ead9d700 1 -- 172.21.2.101:0/1654584515 learned_addr learned my addr 172.21.2.101:0/1654584515 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:38.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.086 7ff1ead9d700 1 -- 172.21.2.101:0/1654584515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ec09e2d0 msgr2=0x7ff1ec09d4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:31:38.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.086 7ff1ead9d700 1 --2- 172.21.2.101:0/1654584515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ec09e2d0 0x7ff1ec09d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:38.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.086 7ff1ead9d700 1 -- 172.21.2.101:0/1654584515 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1ec1aac10 msgr2=0x7ff1ec09ce90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:31:38.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.086 7ff1ead9d700 1 --2- 172.21.2.101:0/1654584515 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1ec1aac10 0x7ff1ec09ce90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:38.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.086 7ff1ead9d700 1 -- 172.21.2.101:0/1654584515 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec192a70 con 0x7ff1ec0b4870 2021-02-10T11:31:38.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.086 7ff1ead9d700 1 --2- 172.21.2.101:0/1654584515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec0b4870 0x7ff1ec09dcb0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7ff1dc00b530 tx=0x7ff1dc00b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:38.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.086 7ff1e3fff700 1 -- 172.21.2.101:0/1654584515 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1dc00bcc0 con 0x7ff1ec0b4870 2021-02-10T11:31:38.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.086 7ff1e3fff700 1 -- 172.21.2.101:0/1654584515 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc002080 con 0x7ff1ec0b4870 2021-02-10T11:31:38.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.086 7ff1f1096700 1 -- 172.21.2.101:0/1654584515 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ec192a70 con 0x7ff1ec0b4870 2021-02-10T11:31:38.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.087 7ff1e3fff700 1 -- 172.21.2.101:0/1654584515 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1dc002560 con 0x7ff1ec0b4870 2021-02-10T11:31:38.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.087 7ff1f1096700 1 -- 172.21.2.101:0/1654584515 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec192a70 con 0x7ff1ec0b4870 2021-02-10T11:31:38.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.087 7ff1e3fff700 1 -- 172.21.2.101:0/1654584515 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff1dc002d20 con 0x7ff1ec0b4870 2021-02-10T11:31:38.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.087 7ff1e3fff700 1 --2- 172.21.2.101:0/1654584515 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1d0018700 0x7ff1d001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:38.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.087 7ff1e3fff700 1 -- 172.21.2.101:0/1654584515 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff1dc032800 con 0x7ff1ec0b4870 2021-02-10T11:31:38.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.087 7ff1e259c700 1 --2- 172.21.2.101:0/1654584515 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1d0018700 0x7ff1d001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:38.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.088 7ff1e259c700 1 --2- 172.21.2.101:0/1654584515 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1d0018700 0x7ff1d001abc0 crc :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:38.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.118 7ff1f1096700 1 -- 172.21.2.101:0/1654584515 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ec083da0 con 0x7ff1ec0b4870 2021-02-10T11:31:38.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.121 7ff1e3fff700 1 -- 172.21.2.101:0/1654584515 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff1dc000c90 con 0x7ff1ec0b4870 2021-02-10T11:31:38.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.249 7ff1f1096700 1 -- 172.21.2.101:0/1654584515 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1ec07ed80 con 0x7ff1ec0b4870 2021-02-10T11:31:38.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.249 7ff1e3fff700 1 -- 172.21.2.101:0/1654584515 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+2096 (secure 0 0 0) 0x7ff1dc002d20 con 0x7ff1ec0b4870 2021-02-10T11:31:38.261 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:31:38.261 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/243293953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":243293953},{"type":"v1","addr":"172.21.2.101:6835","nonce":243293953}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:31:31.941616","modified":"2021-02-10 11:31:33.951741","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7299":{"gid":7299,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/1600862555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":1600862555},{"type":"v1","addr":"172.21.2.108:6833","nonce":1600862555}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-02-10T11:31:38.261 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 22 2021-02-10T11:31:38.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.251 7ff1f1096700 1 -- 172.21.2.101:0/1654584515 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1d0018700 msgr2=0x7ff1d001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:38.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.251 7ff1f1096700 1 --2- 172.21.2.101:0/1654584515 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1d0018700 0x7ff1d001abc0 crc :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:38.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.251 7ff1f1096700 1 -- 172.21.2.101:0/1654584515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec0b4870 msgr2=0x7ff1ec09dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:38.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.251 7ff1f1096700 1 --2- 172.21.2.101:0/1654584515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec0b4870 0x7ff1ec09dcb0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7ff1dc00b530 tx=0x7ff1dc00b8d0).stop 2021-02-10T11:31:38.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.251 7ff1f1096700 1 -- 172.21.2.101:0/1654584515 shutdown_connections 2021-02-10T11:31:38.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.251 7ff1f1096700 1 --2- 172.21.2.101:0/1654584515 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1d0018700 0x7ff1d001abc0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:38.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.251 7ff1f1096700 1 --2- 172.21.2.101:0/1654584515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ec09e2d0 0x7ff1ec09d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:38.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.251 7ff1f1096700 1 --2- 172.21.2.101:0/1654584515 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1ec1aac10 0x7ff1ec09ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:38.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.251 7ff1f1096700 1 --2- 172.21.2.101:0/1654584515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec0b4870 0x7ff1ec09dcb0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:38.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.251 7ff1f1096700 1 -- 172.21.2.101:0/1654584515 >> 172.21.2.101:0/1654584515 conn(0x7ff1ec1a7c60 msgr2=0x7ff1ec09ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:38.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.252 7ff1f1096700 1 -- 172.21.2.101:0/1654584515 shutdown_connections 2021-02-10T11:31:38.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:38.252 7ff1f1096700 1 -- 172.21.2.101:0/1654584515 wait complete. 2021-02-10T11:31:38.473 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:31:31.941616', 'modified': '2021-02-10 11:31:33.951741', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7299': {'gid': 7299, 'name': 'b', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/1600862555', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 1600862555}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 1600862555}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:31:38.473 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:31:38.474 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:31:38.872 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:31:38.872 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:31:38.872 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:31:38.872 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 7299, 2021-02-10T11:31:38.872 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:31:38.872 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:31:38.872 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 6.9182362460000002, 2021-02-10T11:31:38.872 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 22, 2021-02-10T11:31:38.872 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 38, 2021-02-10T11:31:38.872 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 38, 2021-02-10T11:31:38.873 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 12.558254076000001 2021-02-10T11:31:38.873 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:31:38.876 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 7299, "want_state": "up:active", "state": "up:active", "rank_uptime": 6.9182362460000002, "mdsmap_epoch": 22, "osdmap_epoch": 38, "osdmap_epoch_barrier": 38, "uptime": 12.558254076000001 } 2021-02-10T11:31:38.876 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:31:39.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.277 7f7a3ba06700 1 Processor -- start 2021-02-10T11:31:39.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.277 7f7a3ba06700 1 -- start start 2021-02-10T11:31:39.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.277 7f7a3ba06700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7a340e5470 0x7f7a3407e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:39.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.277 7f7a3ba06700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7a3407f300 0x7f7a3407f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:39.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.277 7f7a3ba06700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a3407ed80 con 0x7f7a340e5070 2021-02-10T11:31:39.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.277 7f7a3ba06700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a341b4010 con 0x7f7a340e5470 2021-02-10T11:31:39.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.277 7f7a3ba06700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a340d1790 con 0x7f7a3407f300 2021-02-10T11:31:39.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.277 7f7a39822700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f7a340e5070 0x7f7a341b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50754/0 (socket says 172.21.2.101:50754) 2021-02-10T11:31:39.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.277 7f7a39822700 1 -- 172.21.2.101:0/1367960207 learned_addr learned my addr 172.21.2.101:0/1367960207 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:39.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.277 7f7a39021700 1 --2- 172.21.2.101:0/1367960207 >> v2:172.21.2.101:3300/0 conn(0x7f7a340e5470 0x7f7a3407e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:39.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.278 7f7a39021700 1 -- 172.21.2.101:0/1367960207 >> v2:172.21.2.108:3300/0 conn(0x7f7a3407f300 msgr2=0x7f7a3407f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:39.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.278 7f7a39021700 1 --2- 172.21.2.101:0/1367960207 >> v2:172.21.2.108:3300/0 conn(0x7f7a3407f300 0x7f7a3407f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:39.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.278 7f7a39021700 1 -- 172.21.2.101:0/1367960207 >> v1:172.21.2.101:6789/0 conn(0x7f7a340e5070 legacy=0x7f7a341b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:39.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.278 7f7a39021700 1 -- 172.21.2.101:0/1367960207 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a340d19b0 con 0x7f7a340e5470 2021-02-10T11:31:39.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.278 7f7a39021700 1 --2- 172.21.2.101:0/1367960207 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a340e5470 0x7f7a3407e6a0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7f7a30006650 tx=0x7f7a30009a00).ready entity=mon.0 client_cookie=539da34d56233945 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:39.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.278 7f7a38820700 1 -- 172.21.2.101:0/1367960207 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7a30009df0 con 0x7f7a340e5470 2021-02-10T11:31:39.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.278 7f7a38820700 1 -- 172.21.2.101:0/1367960207 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a300009b0 con 0x7f7a340e5470 2021-02-10T11:31:39.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.278 7f7a38820700 1 -- 172.21.2.101:0/1367960207 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7a3000c2b0 con 0x7f7a340e5470 2021-02-10T11:31:39.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.278 7f7a3ba06700 1 -- 172.21.2.101:0/1367960207 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a340e5470 msgr2=0x7f7a3407e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:39.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.278 7f7a3ba06700 1 --2- 172.21.2.101:0/1367960207 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a340e5470 0x7f7a3407e6a0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7f7a30006650 tx=0x7f7a30009a00).stop 2021-02-10T11:31:39.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.279 7f7a3ba06700 1 -- 172.21.2.101:0/1367960207 shutdown_connections 2021-02-10T11:31:39.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.279 7f7a3ba06700 1 --2- 172.21.2.101:0/1367960207 >> v2:172.21.2.108:3300/0 conn(0x7f7a3407f300 0x7f7a3407f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:39.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.279 7f7a3ba06700 1 --2- 172.21.2.101:0/1367960207 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a340e5470 0x7f7a3407e6a0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:39.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.279 7f7a3ba06700 1 -- 172.21.2.101:0/1367960207 >> 172.21.2.101:0/1367960207 conn(0x7f7a340ce690 msgr2=0x7f7a340e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:39.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.279 7f7a3ba06700 1 -- 172.21.2.101:0/1367960207 shutdown_connections 2021-02-10T11:31:39.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.279 7f7a3ba06700 1 -- 172.21.2.101:0/1367960207 wait complete. 2021-02-10T11:31:39.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.294 7f7a3ba06700 1 Processor -- start 2021-02-10T11:31:39.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.294 7f7a3ba06700 1 -- start start 2021-02-10T11:31:39.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.294 7f7a3ba06700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a341b19d0 0x7f7a3408cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:39.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.294 7f7a3ba06700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a341b0d00 0x7f7a341b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:39.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.294 7f7a3ba06700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a341a2b50 0x7f7a341a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:39.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.294 7f7a3ba06700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a341b1720 con 0x7f7a341b0d00 2021-02-10T11:31:39.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.294 7f7a3ba06700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3408c470 con 0x7f7a341a2b50 2021-02-10T11:31:39.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.294 7f7a3ba06700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a341b1f50 con 0x7f7a341b19d0 2021-02-10T11:31:39.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.294 7f7a39021700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a341b0d00 0x7f7a341b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:39.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.294 7f7a39021700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a341b0d00 0x7f7a341b1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50266/0 (socket says 172.21.2.101:50266) 2021-02-10T11:31:39.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.294 7f7a39021700 1 -- 172.21.2.101:0/1523280952 learned_addr learned my addr 172.21.2.101:0/1523280952 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:39.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.295 7f7a3a023700 1 --2- 172.21.2.101:0/1523280952 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a341a2b50 0x7f7a341a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:39.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.295 7f7a39822700 1 --2- 172.21.2.101:0/1523280952 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a341b19d0 0x7f7a3408cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:39.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.295 7f7a39021700 1 -- 172.21.2.101:0/1523280952 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a341b19d0 msgr2=0x7f7a3408cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:39.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.295 7f7a39021700 1 --2- 172.21.2.101:0/1523280952 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a341b19d0 0x7f7a3408cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:39.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.295 7f7a39021700 1 -- 172.21.2.101:0/1523280952 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a341a2b50 msgr2=0x7f7a341a2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:39.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.295 7f7a39021700 1 --2- 172.21.2.101:0/1523280952 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a341a2b50 0x7f7a341a2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:39.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.295 7f7a39021700 1 -- 172.21.2.101:0/1523280952 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a341a3570 con 0x7f7a341b0d00 2021-02-10T11:31:39.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.295 7f7a39822700 1 --2- 172.21.2.101:0/1523280952 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a341b19d0 0x7f7a3408cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:31:39.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.295 7f7a3a023700 1 --2- 172.21.2.101:0/1523280952 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a341a2b50 0x7f7a341a2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:31:39.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.295 7f7a39021700 1 --2- 172.21.2.101:0/1523280952 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a341b0d00 0x7f7a341b1100 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7f7a300096a0 tx=0x7f7a3000b9b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:39.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.295 7f7a267fc700 1 -- 172.21.2.101:0/1523280952 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7a300093a0 con 0x7f7a341b0d00 2021-02-10T11:31:39.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.295 7f7a267fc700 1 -- 172.21.2.101:0/1523280952 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3000bda0 con 0x7f7a341b0d00 2021-02-10T11:31:39.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.295 7f7a267fc700 1 -- 172.21.2.101:0/1523280952 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7a30002080 con 0x7f7a341b0d00 2021-02-10T11:31:39.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.295 7f7a3ba06700 1 -- 172.21.2.101:0/1523280952 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a341a3570 con 0x7f7a341b0d00 2021-02-10T11:31:39.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.295 7f7a3ba06700 1 -- 172.21.2.101:0/1523280952 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a341a3570 con 0x7f7a341b0d00 2021-02-10T11:31:39.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.296 7f7a267fc700 1 -- 172.21.2.101:0/1523280952 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7a30002020 con 0x7f7a341b0d00 2021-02-10T11:31:39.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.296 7f7a267fc700 1 --2- 172.21.2.101:0/1523280952 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a1c0186d0 0x7f7a1c01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:39.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.296 7f7a267fc700 1 -- 172.21.2.101:0/1523280952 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7a30032760 con 0x7f7a341b0d00 2021-02-10T11:31:39.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.296 7f7a39822700 1 --2- 172.21.2.101:0/1523280952 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a1c0186d0 0x7f7a1c01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:39.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.296 7f7a39822700 1 --2- 172.21.2.101:0/1523280952 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a1c0186d0 0x7f7a1c01abd0 crc :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:39.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.327 7f7a3ba06700 1 -- 172.21.2.101:0/1523280952 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a34083dd0 con 0x7f7a341b0d00 2021-02-10T11:31:39.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.330 7f7a267fc700 1 -- 172.21.2.101:0/1523280952 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7a30000c90 con 0x7f7a341b0d00 2021-02-10T11:31:39.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.453 7f7a3ba06700 1 -- 172.21.2.101:0/1523280952 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a3407ed80 con 0x7f7a341b0d00 2021-02-10T11:31:39.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.454 7f7a267fc700 1 -- 172.21.2.101:0/1523280952 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+13993 (secure 0 0 0) 0x7f7a30002000 con 0x7f7a341b0d00 2021-02-10T11:31:39.465 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:31:39.466 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":38,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:31:32.943916","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:31:29.865604","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2021-02-10 11:31:30.939202","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:31:39.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.456 7f7a3ba06700 1 -- 172.21.2.101:0/1523280952 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a1c0186d0 msgr2=0x7f7a1c01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:39.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.456 7f7a3ba06700 1 --2- 172.21.2.101:0/1523280952 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a1c0186d0 0x7f7a1c01abd0 crc :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:39.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.456 7f7a3ba06700 1 -- 172.21.2.101:0/1523280952 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a341b0d00 msgr2=0x7f7a341b1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:39.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.456 7f7a3ba06700 1 --2- 172.21.2.101:0/1523280952 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a341b0d00 0x7f7a341b1100 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7f7a300096a0 tx=0x7f7a3000b9b0).stop 2021-02-10T11:31:39.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.456 7f7a3ba06700 1 -- 172.21.2.101:0/1523280952 shutdown_connections 2021-02-10T11:31:39.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.456 7f7a3ba06700 1 --2- 172.21.2.101:0/1523280952 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a341a2b50 0x7f7a341a2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:39.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.456 7f7a3ba06700 1 --2- 172.21.2.101:0/1523280952 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a341b0d00 0x7f7a341b1100 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:39.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.456 7f7a3ba06700 1 --2- 172.21.2.101:0/1523280952 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a1c0186d0 0x7f7a1c01abd0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:39.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.456 7f7a3ba06700 1 --2- 172.21.2.101:0/1523280952 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a341b19d0 0x7f7a3408cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:39.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.456 7f7a3ba06700 1 -- 172.21.2.101:0/1523280952 >> 172.21.2.101:0/1523280952 conn(0x7f7a34093720 msgr2=0x7f7a34084b80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:39.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.456 7f7a3ba06700 1 -- 172.21.2.101:0/1523280952 shutdown_connections 2021-02-10T11:31:39.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.456 7f7a3ba06700 1 -- 172.21.2.101:0/1523280952 wait complete. 2021-02-10T11:31:39.476 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:31:29.865604', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:31:30.939202', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:31:39.476 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:31:39.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.879 7fe1258cb700 1 Processor -- start 2021-02-10T11:31:39.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.879 7fe1258cb700 1 -- start start 2021-02-10T11:31:39.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.880 7fe1258cb700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe1201b0470 0x7fe1201b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:39.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.880 7fe1258cb700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe1200ef610 0x7fe1200efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:39.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.880 7fe1258cb700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1200f0b50 con 0x7fe1200f0030 2021-02-10T11:31:39.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.880 7fe1258cb700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1200e52e0 con 0x7fe1200ef610 2021-02-10T11:31:39.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.880 7fe1258cb700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1200e5490 con 0x7fe1201b0470 2021-02-10T11:31:39.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.880 7fe11fd75700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe1200f0030 0x7fe1200f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50768/0 (socket says 172.21.2.101:50768) 2021-02-10T11:31:39.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.880 7fe11fd75700 1 -- 172.21.2.101:0/2427721652 learned_addr learned my addr 172.21.2.101:0/2427721652 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:39.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.880 7fe11ed73700 1 --2- 172.21.2.101:0/2427721652 >> v2:172.21.2.101:3300/0 conn(0x7fe1200ef610 0x7fe1200efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:39.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.880 7fe11e572700 1 -- 172.21.2.101:0/2427721652 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278878044 0 0) 0x7fe118000f90 con 0x7fe1200f0030 2021-02-10T11:31:39.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.880 7fe11e572700 1 -- 172.21.2.101:0/2427721652 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe108002fe0 con 0x7fe1200f0030 2021-02-10T11:31:39.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.880 7fe11ed73700 1 -- 172.21.2.101:0/2427721652 >> v2:172.21.2.108:3300/0 conn(0x7fe1201b0470 msgr2=0x7fe1201b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:39.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.880 7fe11ed73700 1 --2- 172.21.2.101:0/2427721652 >> v2:172.21.2.108:3300/0 conn(0x7fe1201b0470 0x7fe1201b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:39.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.880 7fe11ed73700 1 -- 172.21.2.101:0/2427721652 >> v1:172.21.2.101:6789/0 conn(0x7fe1200f0030 legacy=0x7fe1200f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:39.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.881 7fe11ed73700 1 -- 172.21.2.101:0/2427721652 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1200f0430 con 0x7fe1200ef610 2021-02-10T11:31:39.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.881 7fe11ed73700 1 --2- 172.21.2.101:0/2427721652 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1200ef610 0x7fe1200efa10 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7fe114006650 tx=0x7fe114009a80).ready entity=mon.0 client_cookie=bfe35c2f4fe7c640 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:39.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.881 7fe11e572700 1 -- 172.21.2.101:0/2427721652 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe1140009b0 con 0x7fe1200ef610 2021-02-10T11:31:39.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.881 7fe11e572700 1 -- 172.21.2.101:0/2427721652 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11400c0c0 con 0x7fe1200ef610 2021-02-10T11:31:39.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.881 7fe11e572700 1 -- 172.21.2.101:0/2427721652 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe11400c5e0 con 0x7fe1200ef610 2021-02-10T11:31:39.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.881 7fe1258cb700 1 -- 172.21.2.101:0/2427721652 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1200ef610 msgr2=0x7fe1200efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:39.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.881 7fe1258cb700 1 --2- 172.21.2.101:0/2427721652 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1200ef610 0x7fe1200efa10 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7fe114006650 tx=0x7fe114009a80).stop 2021-02-10T11:31:39.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.882 7fe1258cb700 1 -- 172.21.2.101:0/2427721652 shutdown_connections 2021-02-10T11:31:39.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.882 7fe1258cb700 1 --2- 172.21.2.101:0/2427721652 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe1200ef610 0x7fe1200efa10 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:39.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.882 7fe1258cb700 1 --2- 172.21.2.101:0/2427721652 >> v2:172.21.2.108:3300/0 conn(0x7fe1201b0470 0x7fe1201b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:39.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.882 7fe1258cb700 1 -- 172.21.2.101:0/2427721652 >> 172.21.2.101:0/2427721652 conn(0x7fe1200ce3e0 msgr2=0x7fe1200e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:39.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.882 7fe1258cb700 1 -- 172.21.2.101:0/2427721652 shutdown_connections 2021-02-10T11:31:39.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.882 7fe1258cb700 1 -- 172.21.2.101:0/2427721652 wait complete. 2021-02-10T11:31:39.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.896 7fe1258cb700 1 Processor -- start 2021-02-10T11:31:39.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.896 7fe1258cb700 1 -- start start 2021-02-10T11:31:39.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.897 7fe1258cb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1200a0420 0x7fe1200a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:39.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.897 7fe1258cb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe120189d00 0x7fe12018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:39.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.897 7fe1258cb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe12018a720 0x7fe1200b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:39.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.897 7fe1258cb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe120189780 con 0x7fe12018a720 2021-02-10T11:31:39.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.897 7fe1258cb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12018ab20 con 0x7fe120189d00 2021-02-10T11:31:39.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.897 7fe1258cb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1200a1140 con 0x7fe1200a0420 2021-02-10T11:31:39.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.897 7fe11fd75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe12018a720 0x7fe1200b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:39.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.897 7fe11fd75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe12018a720 0x7fe1200b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50282/0 (socket says 172.21.2.101:50282) 2021-02-10T11:31:39.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.897 7fe11fd75700 1 -- 172.21.2.101:0/3753070828 learned_addr learned my addr 172.21.2.101:0/3753070828 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.897 7fe11fd75700 1 -- 172.21.2.101:0/3753070828 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1200a0420 msgr2=0x7fe1200a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.897 7fe11fd75700 1 --2- 172.21.2.101:0/3753070828 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1200a0420 0x7fe1200a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.897 7fe11fd75700 1 -- 172.21.2.101:0/3753070828 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe120189d00 msgr2=0x7fe12018a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.897 7fe11fd75700 1 --2- 172.21.2.101:0/3753070828 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe120189d00 0x7fe12018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.897 7fe11fd75700 1 -- 172.21.2.101:0/3753070828 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1200b8200 con 0x7fe12018a720 2021-02-10T11:31:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.897 7fe11fd75700 1 --2- 172.21.2.101:0/3753070828 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe12018a720 0x7fe1200b7be0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7fe11800b8c0 tx=0x7fe11800bc60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.897 7fe107fff700 1 -- 172.21.2.101:0/3753070828 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe118006030 con 0x7fe12018a720 2021-02-10T11:31:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.898 7fe107fff700 1 -- 172.21.2.101:0/3753070828 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1180062f0 con 0x7fe12018a720 2021-02-10T11:31:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.898 7fe107fff700 1 -- 172.21.2.101:0/3753070828 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe118004720 con 0x7fe12018a720 2021-02-10T11:31:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.898 7fe1258cb700 1 -- 172.21.2.101:0/3753070828 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1200b8200 con 0x7fe12018a720 2021-02-10T11:31:39.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.898 7fe1258cb700 1 -- 172.21.2.101:0/3753070828 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1200b8790 con 0x7fe12018a720 2021-02-10T11:31:39.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.898 7fe107fff700 1 -- 172.21.2.101:0/3753070828 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe1180064d0 con 0x7fe12018a720 2021-02-10T11:31:39.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.898 7fe107fff700 1 --2- 172.21.2.101:0/3753070828 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe1000186c0 0x7fe10001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:39.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.898 7fe107fff700 1 -- 172.21.2.101:0/3753070828 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe1180327f0 con 0x7fe12018a720 2021-02-10T11:31:39.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.898 7fe11f574700 1 --2- 172.21.2.101:0/3753070828 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe1000186c0 0x7fe10001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:39.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.899 7fe11f574700 1 --2- 172.21.2.101:0/3753070828 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe1000186c0 0x7fe10001abc0 crc :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:39.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.930 7fe1258cb700 1 -- 172.21.2.101:0/3753070828 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe120105ae0 con 0x7fe12018a720 2021-02-10T11:31:39.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:39.932 7fe107fff700 1 -- 172.21.2.101:0/3753070828 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe118000f90 con 0x7fe12018a720 2021-02-10T11:31:40.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.060 7fe1258cb700 1 -- 172.21.2.101:0/3753070828 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fe1200f0b50 con 0x7fe12018a720 2021-02-10T11:31:40.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.061 7fe107fff700 1 -- 172.21.2.101:0/3753070828 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v38) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe1180064d0 con 0x7fe12018a720 2021-02-10T11:31:40.072 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:31:40.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.062 7fe1258cb700 1 -- 172.21.2.101:0/3753070828 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe1000186c0 msgr2=0x7fe10001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:40.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.062 7fe1258cb700 1 --2- 172.21.2.101:0/3753070828 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe1000186c0 0x7fe10001abc0 crc :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:40.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.062 7fe1258cb700 1 -- 172.21.2.101:0/3753070828 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe12018a720 msgr2=0x7fe1200b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:40.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.062 7fe1258cb700 1 --2- 172.21.2.101:0/3753070828 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe12018a720 0x7fe1200b7be0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7fe11800b8c0 tx=0x7fe11800bc60).stop 2021-02-10T11:31:40.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.063 7fe1258cb700 1 -- 172.21.2.101:0/3753070828 shutdown_connections 2021-02-10T11:31:40.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.063 7fe1258cb700 1 --2- 172.21.2.101:0/3753070828 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe12018a720 0x7fe1200b7be0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:40.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.063 7fe1258cb700 1 --2- 172.21.2.101:0/3753070828 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe120189d00 0x7fe12018a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:40.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.063 7fe1258cb700 1 --2- 172.21.2.101:0/3753070828 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe1000186c0 0x7fe10001abc0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:40.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.063 7fe1258cb700 1 --2- 172.21.2.101:0/3753070828 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1200a0420 0x7fe1200a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:40.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.063 7fe1258cb700 1 -- 172.21.2.101:0/3753070828 >> 172.21.2.101:0/3753070828 conn(0x7fe120094e10 msgr2=0x7fe1200a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:40.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.063 7fe1258cb700 1 -- 172.21.2.101:0/3753070828 shutdown_connections 2021-02-10T11:31:40.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.063 7fe1258cb700 1 -- 172.21.2.101:0/3753070828 wait complete. 2021-02-10T11:31:40.082 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:31:40.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.486 7f28ce994700 1 Processor -- start 2021-02-10T11:31:40.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.486 7f28ce994700 1 -- start start 2021-02-10T11:31:40.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.486 7f28ce994700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f28c81b0470 0x7f28c81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:40.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.486 7f28ce994700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28c80f20e0 0x7f28c80f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:40.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.486 7f28ce994700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c80f0b50 con 0x7f28c80ef610 2021-02-10T11:31:40.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.486 7f28ce994700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c80f0820 con 0x7f28c81b0470 2021-02-10T11:31:40.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.486 7f28ce994700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80f24e0 con 0x7f28c80f20e0 2021-02-10T11:31:40.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.486 7f28c7fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f28c81b0470 0x7f28c81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:40.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.486 7f28c77fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f28c80ef610 0x7f28c80efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50780/0 (socket says 172.21.2.101:50780) 2021-02-10T11:31:40.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.486 7f28c77fe700 1 -- 172.21.2.101:0/983083459 learned_addr learned my addr 172.21.2.101:0/983083459 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:40.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.487 7f28ccfb1700 1 --2- 172.21.2.101:0/983083459 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28c80f20e0 0x7f28c80f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:40.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.487 7f28c6ffd700 1 -- 172.21.2.101:0/983083459 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3387236520 0 0) 0x7f28bc0009b0 con 0x7f28c80ef610 2021-02-10T11:31:40.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.487 7f28c6ffd700 1 -- 172.21.2.101:0/983083459 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28b0002fe0 con 0x7f28c80ef610 2021-02-10T11:31:40.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.487 7f28c7fff700 1 -- 172.21.2.101:0/983083459 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28c80f20e0 msgr2=0x7f28c80f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:40.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.487 7f28c7fff700 1 --2- 172.21.2.101:0/983083459 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28c80f20e0 0x7f28c80f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:40.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.487 7f28c7fff700 1 -- 172.21.2.101:0/983083459 >> v1:172.21.2.101:6789/0 conn(0x7f28c80ef610 legacy=0x7f28c80efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:40.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.487 7f28c7fff700 1 -- 172.21.2.101:0/983083459 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c80e52e0 con 0x7f28c81b0470 2021-02-10T11:31:40.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.487 7f28ccfb1700 1 --2- 172.21.2.101:0/983083459 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28c80f20e0 0x7f28c80f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:31:40.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.487 7f28c7fff700 1 --2- 172.21.2.101:0/983083459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28c81b0470 0x7f28c81b0870 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 rx=0x7f28b8006650 tx=0x7f28b8009a80).ready entity=mon.0 client_cookie=206d8004e829f35c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:40.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.487 7f28c6ffd700 1 -- 172.21.2.101:0/983083459 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f28b8006390 con 0x7f28c81b0470 2021-02-10T11:31:40.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.487 7f28c6ffd700 1 -- 172.21.2.101:0/983083459 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b80009b0 con 0x7f28c81b0470 2021-02-10T11:31:40.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.487 7f28c6ffd700 1 -- 172.21.2.101:0/983083459 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f28b800c2b0 con 0x7f28c81b0470 2021-02-10T11:31:40.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.487 7f28ce994700 1 -- 172.21.2.101:0/983083459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28c81b0470 msgr2=0x7f28c81b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:40.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.487 7f28ce994700 1 --2- 172.21.2.101:0/983083459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28c81b0470 0x7f28c81b0870 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 rx=0x7f28b8006650 tx=0x7f28b8009a80).stop 2021-02-10T11:31:40.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.488 7f28ce994700 1 -- 172.21.2.101:0/983083459 shutdown_connections 2021-02-10T11:31:40.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.488 7f28ce994700 1 --2- 172.21.2.101:0/983083459 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28c80f20e0 0x7f28c80f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:40.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.488 7f28ce994700 1 --2- 172.21.2.101:0/983083459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28c81b0470 0x7f28c81b0870 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:40.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.488 7f28ce994700 1 -- 172.21.2.101:0/983083459 >> 172.21.2.101:0/983083459 conn(0x7f28c80ce3e0 msgr2=0x7f28c80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:40.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.488 7f28ce994700 1 -- 172.21.2.101:0/983083459 shutdown_connections 2021-02-10T11:31:40.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.488 7f28ce994700 1 -- 172.21.2.101:0/983083459 wait complete. 2021-02-10T11:31:40.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.503 7f28ce994700 1 Processor -- start 2021-02-10T11:31:40.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.503 7f28ce994700 1 -- start start 2021-02-10T11:31:40.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.503 7f28ce994700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28c8190920 0x7f28c80b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:40.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.503 7f28ce994700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28c80b8200 0x7f28c80b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:40.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.503 7f28ce994700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f28c80c1d80 0x7f28c80c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:40.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.503 7f28ce994700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c8190d20 con 0x7f28c80b8200 2021-02-10T11:31:40.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.503 7f28ce994700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80b8c20 con 0x7f28c80c1d80 2021-02-10T11:31:40.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.503 7f28ce994700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80c27a0 con 0x7f28c8190920 2021-02-10T11:31:40.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.503 7f28c77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28c80b8200 0x7f28c80b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:40.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.503 7f28c77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28c80b8200 0x7f28c80b8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50294/0 (socket says 172.21.2.101:50294) 2021-02-10T11:31:40.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.503 7f28c77fe700 1 -- 172.21.2.101:0/2060367859 learned_addr learned my addr 172.21.2.101:0/2060367859 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:40.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.504 7f28c77fe700 1 -- 172.21.2.101:0/2060367859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28c8190920 msgr2=0x7f28c80b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:40.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.504 7f28c77fe700 1 --2- 172.21.2.101:0/2060367859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28c8190920 0x7f28c80b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:40.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.504 7f28c77fe700 1 -- 172.21.2.101:0/2060367859 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f28c80c1d80 msgr2=0x7f28c80c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:40.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.504 7f28c77fe700 1 --2- 172.21.2.101:0/2060367859 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f28c80c1d80 0x7f28c80c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:40.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.504 7f28c77fe700 1 -- 172.21.2.101:0/2060367859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c80c29c0 con 0x7f28c80b8200 2021-02-10T11:31:40.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.504 7f28c77fe700 1 --2- 172.21.2.101:0/2060367859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28c80b8200 0x7f28c80b8600 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 rx=0x7f28bc001c60 tx=0x7f28bc00a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:40.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.504 7f28c4ff9700 1 -- 172.21.2.101:0/2060367859 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f28bc00acc0 con 0x7f28c80b8200 2021-02-10T11:31:40.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.504 7f28c4ff9700 1 -- 172.21.2.101:0/2060367859 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc004140 con 0x7f28c80b8200 2021-02-10T11:31:40.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.504 7f28ce994700 1 -- 172.21.2.101:0/2060367859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c80c29c0 con 0x7f28c80b8200 2021-02-10T11:31:40.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.504 7f28c4ff9700 1 -- 172.21.2.101:0/2060367859 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f28bc004660 con 0x7f28c80b8200 2021-02-10T11:31:40.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.504 7f28ce994700 1 -- 172.21.2.101:0/2060367859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c80c2f50 con 0x7f28c80b8200 2021-02-10T11:31:40.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.505 7f28c4ff9700 1 -- 172.21.2.101:0/2060367859 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f28bc004320 con 0x7f28c80b8200 2021-02-10T11:31:40.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.505 7f28c4ff9700 1 --2- 172.21.2.101:0/2060367859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28a80186c0 0x7f28a801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:40.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.505 7f28c4ff9700 1 -- 172.21.2.101:0/2060367859 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f28bc032660 con 0x7f28c80b8200 2021-02-10T11:31:40.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.505 7f28c7fff700 1 --2- 172.21.2.101:0/2060367859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28a80186c0 0x7f28a801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:40.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.505 7f28c7fff700 1 --2- 172.21.2.101:0/2060367859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28a80186c0 0x7f28a801abc0 crc :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:40.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.536 7f28ce994700 1 -- 172.21.2.101:0/2060367859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c8105ae0 con 0x7f28c80b8200 2021-02-10T11:31:40.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.538 7f28c4ff9700 1 -- 172.21.2.101:0/2060367859 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f28bc000960 con 0x7f28c80b8200 2021-02-10T11:31:40.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.667 7f28ce994700 1 -- 172.21.2.101:0/2060367859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f28c80f0b50 con 0x7f28c80b8200 2021-02-10T11:31:40.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.667 7f28c4ff9700 1 -- 172.21.2.101:0/2060367859 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v38) v1 ==== 104+0+10 (secure 0 0 0) 0x7f28bc00acc0 con 0x7f28c80b8200 2021-02-10T11:31:40.679 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:31:40.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.669 7f28ce994700 1 -- 172.21.2.101:0/2060367859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28a80186c0 msgr2=0x7f28a801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:40.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.669 7f28ce994700 1 --2- 172.21.2.101:0/2060367859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28a80186c0 0x7f28a801abc0 crc :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:40.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.669 7f28ce994700 1 -- 172.21.2.101:0/2060367859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28c80b8200 msgr2=0x7f28c80b8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:40.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.669 7f28ce994700 1 --2- 172.21.2.101:0/2060367859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28c80b8200 0x7f28c80b8600 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 rx=0x7f28bc001c60 tx=0x7f28bc00a8d0).stop 2021-02-10T11:31:40.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.669 7f28ce994700 1 -- 172.21.2.101:0/2060367859 shutdown_connections 2021-02-10T11:31:40.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.669 7f28ce994700 1 --2- 172.21.2.101:0/2060367859 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f28c80c1d80 0x7f28c80c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:40.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.669 7f28ce994700 1 --2- 172.21.2.101:0/2060367859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28c80b8200 0x7f28c80b8600 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:40.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.669 7f28ce994700 1 --2- 172.21.2.101:0/2060367859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28a80186c0 0x7f28a801abc0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:40.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.669 7f28ce994700 1 --2- 172.21.2.101:0/2060367859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28c8190920 0x7f28c80b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:40.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.669 7f28ce994700 1 -- 172.21.2.101:0/2060367859 >> 172.21.2.101:0/2060367859 conn(0x7f28c8094e10 msgr2=0x7f28c80a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:40.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.670 7f28ce994700 1 -- 172.21.2.101:0/2060367859 shutdown_connections 2021-02-10T11:31:40.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:40.670 7f28ce994700 1 -- 172.21.2.101:0/2060367859 wait complete. 2021-02-10T11:31:40.689 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:31:41.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.092 7fb286487700 1 Processor -- start 2021-02-10T11:31:41.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.092 7fb286487700 1 -- start start 2021-02-10T11:31:41.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.092 7fb286487700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2801b0870 0x7fb2800f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:41.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.092 7fb286487700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb2800efe00 0x7fb2800f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:41.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.092 7fb286487700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2801b0c70 con 0x7fb2801b0470 2021-02-10T11:31:41.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.092 7fb286487700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2800f0b50 con 0x7fb2800efe00 2021-02-10T11:31:41.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.092 7fb286487700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2800f0820 con 0x7fb2801b0870 2021-02-10T11:31:41.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.093 7fb27f7fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2801b0870 0x7fb2800f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:41.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.093 7fb27f7fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2801b0870 0x7fb2800f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:43032/0 (socket says 172.21.2.101:43032) 2021-02-10T11:31:41.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.093 7fb27f7fe700 1 -- 172.21.2.101:0/991287746 learned_addr learned my addr 172.21.2.101:0/991287746 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:41.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.093 7fb284aa4700 1 --2- 172.21.2.101:0/991287746 >> v2:172.21.2.108:3300/0 conn(0x7fb2800efe00 0x7fb2800f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:41.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.093 7fb27f7fe700 1 -- 172.21.2.101:0/991287746 >> v2:172.21.2.108:3300/0 conn(0x7fb2800efe00 msgr2=0x7fb2800f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:41.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.093 7fb27f7fe700 1 --2- 172.21.2.101:0/991287746 >> v2:172.21.2.108:3300/0 conn(0x7fb2800efe00 0x7fb2800f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:41.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.093 7fb27f7fe700 1 -- 172.21.2.101:0/991287746 >> v1:172.21.2.108:6789/0 conn(0x7fb2801b0470 legacy=0x7fb2800ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:41.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.093 7fb27f7fe700 1 -- 172.21.2.101:0/991287746 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2800e52e0 con 0x7fb2801b0870 2021-02-10T11:31:41.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.094 7fb27f7fe700 1 --2- 172.21.2.101:0/991287746 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2801b0870 0x7fb2800f20e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fb274006700 tx=0x7fb274009b80).ready entity=mon.2 client_cookie=ec02955e0b320b65 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:41.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.094 7fb27effd700 1 -- 172.21.2.101:0/991287746 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb27400c0c0 con 0x7fb2801b0870 2021-02-10T11:31:41.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.094 7fb27effd700 1 -- 172.21.2.101:0/991287746 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27400c400 con 0x7fb2801b0870 2021-02-10T11:31:41.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.094 7fb27effd700 1 -- 172.21.2.101:0/991287746 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb27400c860 con 0x7fb2801b0870 2021-02-10T11:31:41.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.094 7fb286487700 1 -- 172.21.2.101:0/991287746 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2801b0870 msgr2=0x7fb2800f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:41.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.094 7fb286487700 1 --2- 172.21.2.101:0/991287746 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2801b0870 0x7fb2800f20e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fb274006700 tx=0x7fb274009b80).stop 2021-02-10T11:31:41.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.095 7fb286487700 1 -- 172.21.2.101:0/991287746 shutdown_connections 2021-02-10T11:31:41.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.095 7fb286487700 1 --2- 172.21.2.101:0/991287746 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2801b0870 0x7fb2800f20e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:41.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.095 7fb286487700 1 --2- 172.21.2.101:0/991287746 >> v2:172.21.2.108:3300/0 conn(0x7fb2800efe00 0x7fb2800f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:41.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.095 7fb286487700 1 -- 172.21.2.101:0/991287746 >> 172.21.2.101:0/991287746 conn(0x7fb2800ce3e0 msgr2=0x7fb2800e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:41.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.095 7fb286487700 1 -- 172.21.2.101:0/991287746 shutdown_connections 2021-02-10T11:31:41.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.095 7fb286487700 1 -- 172.21.2.101:0/991287746 wait complete. 2021-02-10T11:31:41.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.109 7fb286487700 1 Processor -- start 2021-02-10T11:31:41.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.109 7fb286487700 1 -- start start 2021-02-10T11:31:41.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.109 7fb286487700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb280105d10 0x7fb28009eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:41.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.109 7fb286487700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb280106110 0x7fb28009f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:41.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.109 7fb286487700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2800b2a00 0x7fb2800b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:41.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.109 7fb286487700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28009e560 con 0x7fb280105d10 2021-02-10T11:31:41.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.109 7fb286487700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28009c090 con 0x7fb280106110 2021-02-10T11:31:41.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.109 7fb286487700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2800b3420 con 0x7fb2800b2a00 2021-02-10T11:31:41.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.110 7fb27ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb280105d10 0x7fb28009eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:41.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.110 7fb27ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb280105d10 0x7fb28009eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50306/0 (socket says 172.21.2.101:50306) 2021-02-10T11:31:41.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.110 7fb27ffff700 1 -- 172.21.2.101:0/2514029754 learned_addr learned my addr 172.21.2.101:0/2514029754 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:41.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.110 7fb27ffff700 1 -- 172.21.2.101:0/2514029754 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2800b2a00 msgr2=0x7fb2800b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:41.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.110 7fb27ffff700 1 --2- 172.21.2.101:0/2514029754 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2800b2a00 0x7fb2800b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:41.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.110 7fb27ffff700 1 -- 172.21.2.101:0/2514029754 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb280106110 msgr2=0x7fb28009f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:41.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.110 7fb27ffff700 1 --2- 172.21.2.101:0/2514029754 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb280106110 0x7fb28009f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:41.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.110 7fb27ffff700 1 -- 172.21.2.101:0/2514029754 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2800b3640 con 0x7fb280105d10 2021-02-10T11:31:41.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.110 7fb27ffff700 1 --2- 172.21.2.101:0/2514029754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb280105d10 0x7fb28009eae0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7fb270006ed0 tx=0x7fb27000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:41.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.110 7fb27cff9700 1 -- 172.21.2.101:0/2514029754 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb270006240 con 0x7fb280105d10 2021-02-10T11:31:41.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.111 7fb27cff9700 1 -- 172.21.2.101:0/2514029754 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2700054b0 con 0x7fb280105d10 2021-02-10T11:31:41.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.111 7fb286487700 1 -- 172.21.2.101:0/2514029754 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2800b3640 con 0x7fb280105d10 2021-02-10T11:31:41.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.111 7fb27cff9700 1 -- 172.21.2.101:0/2514029754 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb270005910 con 0x7fb280105d10 2021-02-10T11:31:41.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.111 7fb286487700 1 -- 172.21.2.101:0/2514029754 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2800b3bd0 con 0x7fb280105d10 2021-02-10T11:31:41.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.111 7fb27cff9700 1 -- 172.21.2.101:0/2514029754 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb270006030 con 0x7fb280105d10 2021-02-10T11:31:41.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.111 7fb27cff9700 1 --2- 172.21.2.101:0/2514029754 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2600186c0 0x7fb26001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:41.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.111 7fb27cff9700 1 -- 172.21.2.101:0/2514029754 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb2700323e0 con 0x7fb280105d10 2021-02-10T11:31:41.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.111 7fb27f7fe700 1 --2- 172.21.2.101:0/2514029754 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2600186c0 0x7fb26001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:41.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.112 7fb27f7fe700 1 --2- 172.21.2.101:0/2514029754 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2600186c0 0x7fb26001abc0 crc :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:41.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.142 7fb286487700 1 -- 172.21.2.101:0/2514029754 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb280105ae0 con 0x7fb280105d10 2021-02-10T11:31:41.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.144 7fb27cff9700 1 -- 172.21.2.101:0/2514029754 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb27002cee0 con 0x7fb280105d10 2021-02-10T11:31:41.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.272 7fb286487700 1 -- 172.21.2.101:0/2514029754 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7fb280105ae0 con 0x7fb280105d10 2021-02-10T11:31:41.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.273 7fb27cff9700 1 -- 172.21.2.101:0/2514029754 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v38) v1 ==== 100+0+10 (secure 0 0 0) 0x7fb27002cee0 con 0x7fb280105d10 2021-02-10T11:31:41.284 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:31:41.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.275 7fb286487700 1 -- 172.21.2.101:0/2514029754 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2600186c0 msgr2=0x7fb26001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:41.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.275 7fb286487700 1 --2- 172.21.2.101:0/2514029754 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2600186c0 0x7fb26001abc0 crc :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:41.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.275 7fb286487700 1 -- 172.21.2.101:0/2514029754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb280105d10 msgr2=0x7fb28009eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:41.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.275 7fb286487700 1 --2- 172.21.2.101:0/2514029754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb280105d10 0x7fb28009eae0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7fb270006ed0 tx=0x7fb27000b960).stop 2021-02-10T11:31:41.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.275 7fb286487700 1 -- 172.21.2.101:0/2514029754 shutdown_connections 2021-02-10T11:31:41.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.275 7fb286487700 1 --2- 172.21.2.101:0/2514029754 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2600186c0 0x7fb26001abc0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:41.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.275 7fb286487700 1 --2- 172.21.2.101:0/2514029754 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2800b2a00 0x7fb2800b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:41.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.275 7fb286487700 1 --2- 172.21.2.101:0/2514029754 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb280106110 0x7fb28009f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:41.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.275 7fb286487700 1 --2- 172.21.2.101:0/2514029754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb280105d10 0x7fb28009eae0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:41.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.275 7fb286487700 1 -- 172.21.2.101:0/2514029754 >> 172.21.2.101:0/2514029754 conn(0x7fb2800c36f0 msgr2=0x7fb2800bd640 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:41.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.275 7fb286487700 1 -- 172.21.2.101:0/2514029754 shutdown_connections 2021-02-10T11:31:41.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.275 7fb286487700 1 -- 172.21.2.101:0/2514029754 wait complete. 2021-02-10T11:31:41.295 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:31:41.295 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:31:41.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.693 7fabb46c1700 1 Processor -- start 2021-02-10T11:31:41.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.693 7fabb46c1700 1 -- start start 2021-02-10T11:31:41.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.694 7fabb46c1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fabac0d17f0 0x7fabac1b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:41.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.694 7fabb46c1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabac0cff00 0x7fabac0ef620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:41.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.694 7fabb46c1700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabac0f0b20 con 0x7fabac0d1bf0 2021-02-10T11:31:41.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.694 7fabb46c1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabac1b3ff0 con 0x7fabac0d17f0 2021-02-10T11:31:41.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.694 7fabb46c1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabac0f20b0 con 0x7fabac0cff00 2021-02-10T11:31:41.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.694 7fabb2cde700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabac0cff00 0x7fabac0ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:41.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.694 7fabb24dd700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fabac0d17f0 0x7fabac1b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:41.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.694 7fabb24dd700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fabac0d17f0 0x7fabac1b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:60798/0 (socket says 172.21.2.101:60798) 2021-02-10T11:31:41.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.694 7fabb2cde700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabac0cff00 0x7fabac0ef620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:43046/0 (socket says 172.21.2.101:43046) 2021-02-10T11:31:41.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.694 7fabb24dd700 1 -- 172.21.2.101:0/3372126521 learned_addr learned my addr 172.21.2.101:0/3372126521 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:41.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.695 7fabb14db700 1 -- 172.21.2.101:0/3372126521 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 244713328 0 0) 0x7fab9c000ff0 con 0x7fabac0d1bf0 2021-02-10T11:31:41.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.695 7fabb14db700 1 -- 172.21.2.101:0/3372126521 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab98002fe0 con 0x7fabac0d1bf0 2021-02-10T11:31:41.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.695 7fabb24dd700 1 -- 172.21.2.101:0/3372126521 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabac0cff00 msgr2=0x7fabac0ef620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:41.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.695 7fabb24dd700 1 --2- 172.21.2.101:0/3372126521 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabac0cff00 0x7fabac0ef620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:41.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.695 7fabb24dd700 1 -- 172.21.2.101:0/3372126521 >> v1:172.21.2.108:6789/0 conn(0x7fabac0d1bf0 legacy=0x7fabac1b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:41.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.695 7fabb24dd700 1 -- 172.21.2.101:0/3372126521 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac0f22d0 con 0x7fabac0d17f0 2021-02-10T11:31:41.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.695 7fabb24dd700 1 --2- 172.21.2.101:0/3372126521 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabac0d17f0 0x7fabac1b31e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7faba8006700 tx=0x7faba8009b80).ready entity=mon.1 client_cookie=d365ca0ab1d32fdd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:41.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.695 7fabb14db700 1 -- 172.21.2.101:0/3372126521 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faba800c0c0 con 0x7fabac0d17f0 2021-02-10T11:31:41.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.695 7fabb14db700 1 -- 172.21.2.101:0/3372126521 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba800c340 con 0x7fabac0d17f0 2021-02-10T11:31:41.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.695 7fabb46c1700 1 -- 172.21.2.101:0/3372126521 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabac0d17f0 msgr2=0x7fabac1b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:41.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.695 7fabb46c1700 1 --2- 172.21.2.101:0/3372126521 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabac0d17f0 0x7fabac1b31e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7faba8006700 tx=0x7faba8009b80).stop 2021-02-10T11:31:41.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.695 7fabb14db700 1 -- 172.21.2.101:0/3372126521 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faba800c860 con 0x7fabac0d17f0 2021-02-10T11:31:41.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.696 7fabb46c1700 1 -- 172.21.2.101:0/3372126521 shutdown_connections 2021-02-10T11:31:41.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.696 7fabb46c1700 1 --2- 172.21.2.101:0/3372126521 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabac0cff00 0x7fabac0ef620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:41.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.696 7fabb46c1700 1 --2- 172.21.2.101:0/3372126521 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabac0d17f0 0x7fabac1b31e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:41.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.696 7fabb46c1700 1 -- 172.21.2.101:0/3372126521 >> 172.21.2.101:0/3372126521 conn(0x7fabac0fde40 msgr2=0x7fabac0ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:41.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.696 7fabb46c1700 1 -- 172.21.2.101:0/3372126521 shutdown_connections 2021-02-10T11:31:41.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.696 7fabb46c1700 1 -- 172.21.2.101:0/3372126521 wait complete. 2021-02-10T11:31:41.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.711 7fabb46c1700 1 Processor -- start 2021-02-10T11:31:41.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.711 7fabb46c1700 1 -- start start 2021-02-10T11:31:41.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.711 7fabb46c1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabac1a7e80 0x7fabac0ad650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:41.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.711 7fabb46c1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabac0adc70 0x7fabac1b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:41.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.711 7fabb46c1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabac1b15c0 0x7fabac1a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:41.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.711 7fabb46c1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac1a7980 con 0x7fabac1b15c0 2021-02-10T11:31:41.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.711 7fabb46c1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac198920 con 0x7fabac1a7e80 2021-02-10T11:31:41.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.711 7fabb46c1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabac1b19c0 con 0x7fabac0adc70 2021-02-10T11:31:41.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.711 7fabb2cde700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabac1b15c0 0x7fabac1a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:41.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.711 7fabb2cde700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabac1b15c0 0x7fabac1a5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50322/0 (socket says 172.21.2.101:50322) 2021-02-10T11:31:41.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.711 7fabb2cde700 1 -- 172.21.2.101:0/1775041820 learned_addr learned my addr 172.21.2.101:0/1775041820 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:41.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.711 7fabb24dd700 1 --2- 172.21.2.101:0/1775041820 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabac1a7e80 0x7fabac0ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:41.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.712 7fabb1cdc700 1 --2- 172.21.2.101:0/1775041820 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabac0adc70 0x7fabac1b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:41.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.712 7fabb2cde700 1 -- 172.21.2.101:0/1775041820 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabac0adc70 msgr2=0x7fabac1b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:41.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.712 7fabb2cde700 1 --2- 172.21.2.101:0/1775041820 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabac0adc70 0x7fabac1b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:41.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.712 7fabb2cde700 1 -- 172.21.2.101:0/1775041820 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabac1a7e80 msgr2=0x7fabac0ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:41.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.712 7fabb2cde700 1 --2- 172.21.2.101:0/1775041820 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabac1a7e80 0x7fabac0ad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:41.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.712 7fabb2cde700 1 -- 172.21.2.101:0/1775041820 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac1b1be0 con 0x7fabac1b15c0 2021-02-10T11:31:41.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.712 7fabb2cde700 1 --2- 172.21.2.101:0/1775041820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabac1b15c0 0x7fabac1a5c80 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0x7faba400b970 tx=0x7faba400bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:41.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.712 7faba2ffd700 1 -- 172.21.2.101:0/1775041820 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faba4000f90 con 0x7fabac1b15c0 2021-02-10T11:31:41.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.712 7faba2ffd700 1 -- 172.21.2.101:0/1775041820 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba4003030 con 0x7fabac1b15c0 2021-02-10T11:31:41.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.712 7faba2ffd700 1 -- 172.21.2.101:0/1775041820 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faba4001400 con 0x7fabac1b15c0 2021-02-10T11:31:41.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.712 7fabb46c1700 1 -- 172.21.2.101:0/1775041820 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabac1b1be0 con 0x7fabac1b15c0 2021-02-10T11:31:41.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.712 7fabb46c1700 1 -- 172.21.2.101:0/1775041820 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabac1a6590 con 0x7fabac1b15c0 2021-02-10T11:31:41.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.713 7faba2ffd700 1 -- 172.21.2.101:0/1775041820 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7faba40011f0 con 0x7fabac1b15c0 2021-02-10T11:31:41.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.713 7faba2ffd700 1 --2- 172.21.2.101:0/1775041820 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab940186c0 0x7fab9401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:41.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.713 7faba2ffd700 1 -- 172.21.2.101:0/1775041820 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7faba4032930 con 0x7fabac1b15c0 2021-02-10T11:31:41.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.713 7fabb24dd700 1 --2- 172.21.2.101:0/1775041820 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab940186c0 0x7fab9401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:41.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.713 7fabb24dd700 1 --2- 172.21.2.101:0/1775041820 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab940186c0 0x7fab9401abc0 crc :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:41.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.744 7fabb46c1700 1 -- 172.21.2.101:0/1775041820 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabac1036a0 con 0x7fabac1b15c0 2021-02-10T11:31:41.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.747 7faba2ffd700 1 -- 172.21.2.101:0/1775041820 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7faba4001f00 con 0x7fabac1b15c0 2021-02-10T11:31:41.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.876 7fabb46c1700 1 -- 172.21.2.101:0/1775041820 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabac0f0b20 con 0x7fabac1b15c0 2021-02-10T11:31:41.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.876 7faba2ffd700 1 -- 172.21.2.101:0/1775041820 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+2096 (secure 0 0 0) 0x7faba40011f0 con 0x7fabac1b15c0 2021-02-10T11:31:41.887 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:31:41.888 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/243293953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":243293953},{"type":"v1","addr":"172.21.2.101:6835","nonce":243293953}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:31:31.941616","modified":"2021-02-10 11:31:33.951741","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7299":{"gid":7299,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/1600862555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":1600862555},{"type":"v1","addr":"172.21.2.108:6833","nonce":1600862555}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-02-10T11:31:41.888 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 22 2021-02-10T11:31:41.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.878 7fabb46c1700 1 -- 172.21.2.101:0/1775041820 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab940186c0 msgr2=0x7fab9401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:41.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.878 7fabb46c1700 1 --2- 172.21.2.101:0/1775041820 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab940186c0 0x7fab9401abc0 crc :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:41.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.878 7fabb46c1700 1 -- 172.21.2.101:0/1775041820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabac1b15c0 msgr2=0x7fabac1a5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:41.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.878 7fabb46c1700 1 --2- 172.21.2.101:0/1775041820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabac1b15c0 0x7fabac1a5c80 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0x7faba400b970 tx=0x7faba400bd10).stop 2021-02-10T11:31:41.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.878 7fabb46c1700 1 -- 172.21.2.101:0/1775041820 shutdown_connections 2021-02-10T11:31:41.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.878 7fabb46c1700 1 --2- 172.21.2.101:0/1775041820 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab940186c0 0x7fab9401abc0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:41.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.878 7fabb46c1700 1 --2- 172.21.2.101:0/1775041820 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabac0adc70 0x7fabac1b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:41.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.878 7fabb46c1700 1 --2- 172.21.2.101:0/1775041820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabac1b15c0 0x7fabac1a5c80 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:41.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.878 7fabb46c1700 1 --2- 172.21.2.101:0/1775041820 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabac1a7e80 0x7fabac0ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:41.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.878 7fabb46c1700 1 -- 172.21.2.101:0/1775041820 >> 172.21.2.101:0/1775041820 conn(0x7fabac108000 msgr2=0x7fabac08ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:41.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.878 7fabb46c1700 1 -- 172.21.2.101:0/1775041820 shutdown_connections 2021-02-10T11:31:41.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:41.878 7fabb46c1700 1 -- 172.21.2.101:0/1775041820 wait complete. 2021-02-10T11:31:42.099 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:31:31.941616', 'modified': '2021-02-10 11:31:33.951741', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7299': {'gid': 7299, 'name': 'b', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/1600862555', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 1600862555}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 1600862555}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:31:42.099 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:31:42.099 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:31:42.498 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:31:42.498 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:31:42.499 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:31:42.499 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 7299, 2021-02-10T11:31:42.499 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:31:42.499 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:31:42.499 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 10.543809595000001, 2021-02-10T11:31:42.499 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 22, 2021-02-10T11:31:42.499 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 38, 2021-02-10T11:31:42.499 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 38, 2021-02-10T11:31:42.499 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 16.183828278 2021-02-10T11:31:42.499 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:31:42.501 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 7299, "want_state": "up:active", "state": "up:active", "rank_uptime": 10.543809595000001, "mdsmap_epoch": 22, "osdmap_epoch": 38, "osdmap_epoch_barrier": 38, "uptime": 16.183828278 } 2021-02-10T11:31:42.502 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:31:42.502 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-02-10T11:31:42.502 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-02-10T11:31:42.502 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:42.502 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:31:42.836 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:31:43.181 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:31:43.181 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:31:43.522 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:31:43.523 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:31:43.523 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:31:43.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:43.524 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:31:43.857 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-02-10T11:31:43.857 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:31:43.860 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:31:44.204 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:31:44.204 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:31:44.270 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[110368]: starting ceph client 2021-02-10T11:31:44.270 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:2021-02-10 11:31:44.259 7fd7e6326f80 -1 init, newargv = 0x562a9bac4b00 newargc=9 2021-02-10T11:31:44.276 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[110368]: starting fuse 2021-02-10T11:31:44.547 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:31:44.547 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:31:44.548 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:31:44.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:44.548 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:31:44.883 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:31:44.884 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2021-02-10T11:31:44.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:44.884 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:31:44.884 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:31:44.884 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:31:44.884 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:31:44.884 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:31:44.884 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:31:44.884 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:31:44.884 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-02-10T11:31:44.885 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:31:44.885 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:31:44.885 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:31:44.885 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:31:44.885 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:31:44.885 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:31:44.885 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:31:44.885 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:31:44.885 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:31:44.885 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:31:44.885 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:31:44.885 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:31:44.885 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:31:44.885 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.0")) 2021-02-10T11:31:44.885 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:31:45.246 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.0.110368.asok 2021-02-10T11:31:45.249 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.110368.asok 2021-02-10T11:31:45.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:45.249 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.110368.asok status 2021-02-10T11:31:45.637 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:31:45.637 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:31:45.637 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:31:45.638 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:31:45.638 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "0", 2021-02-10T11:31:45.638 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:31:45.638 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-02-10T11:31:45.638 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "110368", 2021-02-10T11:31:45.638 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:31:45.638 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:45.638 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:31:45.638 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:31:45.638 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 7464, 2021-02-10T11:31:45.638 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:31:45.638 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:31:45.639 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:31:45.639 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 7464 2021-02-10T11:31:45.639 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:45.639 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:31:45.639 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:45.639 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:31:45.639 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 2180004229 2021-02-10T11:31:45.639 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:45.639 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:45.639 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:31:45.639 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:45.639 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:31:45.639 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 2180004229 2021-02-10T11:31:45.639 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:45.640 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.7464 172.21.2.101:0/2180004229", 2021-02-10T11:31:45.640 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/2180004229", 2021-02-10T11:31:45.640 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:31:45.640 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 22, 2021-02-10T11:31:45.640 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 38, 2021-02-10T11:31:45.640 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:31:45.640 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:31:45.640 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:31:45.641 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:45.642 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:31:45.642 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:31:45.975 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:31:45.976 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:31:45.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:45.976 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:31:46.323 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:31:46.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.720 7f6238266700 1 Processor -- start 2021-02-10T11:31:46.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.720 7f6238266700 1 -- start start 2021-02-10T11:31:46.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.721 7f6238266700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f62300d17f0 0x7f62301b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:46.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.721 7f6238266700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f62300d1bf0 0x7f62301b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:46.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.721 7f6238266700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62300f0b20 con 0x7f62301b3e20 2021-02-10T11:31:46.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.721 7f6238266700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62300e5eb0 con 0x7f62300d1bf0 2021-02-10T11:31:46.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.721 7f6238266700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62300f0630 con 0x7f62300d17f0 2021-02-10T11:31:46.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.721 7f6235881700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f62300d1bf0 0x7f62301b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:46.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.721 7f6235881700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f62300d1bf0 0x7f62301b3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50338/0 (socket says 172.21.2.101:50338) 2021-02-10T11:31:46.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.721 7f6235881700 1 -- 172.21.2.101:0/3790559447 learned_addr learned my addr 172.21.2.101:0/3790559447 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:46.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.721 7f6236883700 1 --1- 172.21.2.101:0/3790559447 >> v1:172.21.2.101:6789/0 conn(0x7f62301b3e20 0x7f62300ceef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50832/0 (socket says 172.21.2.101:50832) 2021-02-10T11:31:46.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.721 7f6235881700 1 -- 172.21.2.101:0/3790559447 >> v2:172.21.2.108:3300/0 conn(0x7f62300d17f0 msgr2=0x7f62301b31e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:31:46.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.721 7f6235881700 1 --2- 172.21.2.101:0/3790559447 >> v2:172.21.2.108:3300/0 conn(0x7f62300d17f0 0x7f62301b31e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:46.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.721 7f6235881700 1 -- 172.21.2.101:0/3790559447 >> v1:172.21.2.101:6789/0 conn(0x7f62301b3e20 legacy=0x7f62300ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:46.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.721 7f6235881700 1 -- 172.21.2.101:0/3790559447 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62300f0850 con 0x7f62300d1bf0 2021-02-10T11:31:46.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.721 7f6235881700 1 --2- 172.21.2.101:0/3790559447 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62300d1bf0 0x7f62301b3800 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7f6224006700 tx=0x7f6224009b00).ready entity=mon.0 client_cookie=35223887ded6404 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:46.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.722 7f6235080700 1 -- 172.21.2.101:0/3790559447 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f622400c0c0 con 0x7f62300d1bf0 2021-02-10T11:31:46.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.722 7f6235080700 1 -- 172.21.2.101:0/3790559447 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622400c400 con 0x7f62300d1bf0 2021-02-10T11:31:46.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.722 7f6235080700 1 -- 172.21.2.101:0/3790559447 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f622400c920 con 0x7f62300d1bf0 2021-02-10T11:31:46.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.722 7f6238266700 1 -- 172.21.2.101:0/3790559447 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62300d1bf0 msgr2=0x7f62301b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:46.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.722 7f6238266700 1 --2- 172.21.2.101:0/3790559447 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62300d1bf0 0x7f62301b3800 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7f6224006700 tx=0x7f6224009b00).stop 2021-02-10T11:31:46.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.722 7f6238266700 1 -- 172.21.2.101:0/3790559447 shutdown_connections 2021-02-10T11:31:46.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.722 7f6238266700 1 --2- 172.21.2.101:0/3790559447 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62300d1bf0 0x7f62301b3800 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:46.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.722 7f6238266700 1 --2- 172.21.2.101:0/3790559447 >> v2:172.21.2.108:3300/0 conn(0x7f62300d17f0 0x7f62301b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:46.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.722 7f6238266700 1 -- 172.21.2.101:0/3790559447 >> 172.21.2.101:0/3790559447 conn(0x7f62300fde40 msgr2=0x7f62300ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:46.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.722 7f6238266700 1 -- 172.21.2.101:0/3790559447 shutdown_connections 2021-02-10T11:31:46.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.722 7f6238266700 1 -- 172.21.2.101:0/3790559447 wait complete. 2021-02-10T11:31:46.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.737 7f6238266700 1 Processor -- start 2021-02-10T11:31:46.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.737 7f6238266700 1 -- start start 2021-02-10T11:31:46.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.737 7f6238266700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62301986f0 0x7f623019fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:46.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.737 7f6238266700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62301980d0 0x7f62300a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:46.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.737 7f6238266700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62300aa470 0x7f62301a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:46.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.737 7f6238266700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62301a05f0 con 0x7f62301986f0 2021-02-10T11:31:46.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.737 7f6238266700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623009dab0 con 0x7f62301980d0 2021-02-10T11:31:46.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.737 7f6238266700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623018eec0 con 0x7f62300aa470 2021-02-10T11:31:46.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.737 7f6236082700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62301986f0 0x7f623019fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:46.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.737 7f6236082700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62301986f0 0x7f623019fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50344/0 (socket says 172.21.2.101:50344) 2021-02-10T11:31:46.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.737 7f6236082700 1 -- 172.21.2.101:0/488425162 learned_addr learned my addr 172.21.2.101:0/488425162 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:46.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.738 7f6236082700 1 -- 172.21.2.101:0/488425162 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62300aa470 msgr2=0x7f62301a0af0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:31:46.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.738 7f6236082700 1 --2- 172.21.2.101:0/488425162 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62300aa470 0x7f62301a0af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:46.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.738 7f6236082700 1 -- 172.21.2.101:0/488425162 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62301980d0 msgr2=0x7f62300a9e50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:31:46.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.738 7f6236082700 1 --2- 172.21.2.101:0/488425162 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62301980d0 0x7f62300a9e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:46.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.738 7f6236082700 1 -- 172.21.2.101:0/488425162 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623018f0a0 con 0x7f62301986f0 2021-02-10T11:31:46.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.738 7f6236082700 1 --2- 172.21.2.101:0/488425162 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62301986f0 0x7f623019fea0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7f622c00b5c0 tx=0x7f622c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:46.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.738 7f6222ffd700 1 -- 172.21.2.101:0/488425162 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f622c00bd50 con 0x7f62301986f0 2021-02-10T11:31:46.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.738 7f6222ffd700 1 -- 172.21.2.101:0/488425162 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622c006030 con 0x7f62301986f0 2021-02-10T11:31:46.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.738 7f6238266700 1 -- 172.21.2.101:0/488425162 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623018f0a0 con 0x7f62301986f0 2021-02-10T11:31:46.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.738 7f6238266700 1 -- 172.21.2.101:0/488425162 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62301a1400 con 0x7f62301986f0 2021-02-10T11:31:46.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.738 7f6222ffd700 1 -- 172.21.2.101:0/488425162 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f622c0054d0 con 0x7f62301986f0 2021-02-10T11:31:46.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.739 7f6222ffd700 1 -- 172.21.2.101:0/488425162 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f622c006210 con 0x7f62301986f0 2021-02-10T11:31:46.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.739 7f6222ffd700 1 --2- 172.21.2.101:0/488425162 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6218018730 0x7f621801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:46.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.739 7f6222ffd700 1 -- 172.21.2.101:0/488425162 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f622c032800 con 0x7f62301986f0 2021-02-10T11:31:46.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.739 7f6235881700 1 --2- 172.21.2.101:0/488425162 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6218018730 0x7f621801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:46.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.739 7f6235881700 1 --2- 172.21.2.101:0/488425162 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6218018730 0x7f621801abc0 crc :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:46.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.771 7f6238266700 1 -- 172.21.2.101:0/488425162 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62300f0b20 con 0x7f62301986f0 2021-02-10T11:31:46.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.774 7f6222ffd700 1 -- 172.21.2.101:0/488425162 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f622c0009b0 con 0x7f62301986f0 2021-02-10T11:31:46.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.898 7f6238266700 1 -- 172.21.2.101:0/488425162 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62301a0570 con 0x7f62301986f0 2021-02-10T11:31:46.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.899 7f6222ffd700 1 -- 172.21.2.101:0/488425162 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+13993 (secure 0 0 0) 0x7f622c006030 con 0x7f62301986f0 2021-02-10T11:31:46.911 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:31:46.912 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":38,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:31:32.943916","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:31:29.865604","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2021-02-10 11:31:30.939202","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:31:46.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.901 7f6238266700 1 -- 172.21.2.101:0/488425162 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6218018730 msgr2=0x7f621801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:46.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.901 7f6238266700 1 --2- 172.21.2.101:0/488425162 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6218018730 0x7f621801abc0 crc :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:46.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.901 7f6238266700 1 -- 172.21.2.101:0/488425162 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62301986f0 msgr2=0x7f623019fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:46.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.901 7f6238266700 1 --2- 172.21.2.101:0/488425162 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62301986f0 0x7f623019fea0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7f622c00b5c0 tx=0x7f622c00b960).stop 2021-02-10T11:31:46.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.901 7f6238266700 1 -- 172.21.2.101:0/488425162 shutdown_connections 2021-02-10T11:31:46.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.901 7f6238266700 1 --2- 172.21.2.101:0/488425162 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6218018730 0x7f621801abc0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:46.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.901 7f6238266700 1 --2- 172.21.2.101:0/488425162 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62300aa470 0x7f62301a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:46.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.901 7f6238266700 1 --2- 172.21.2.101:0/488425162 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62301980d0 0x7f62300a9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:46.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.901 7f6238266700 1 --2- 172.21.2.101:0/488425162 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62301986f0 0x7f623019fea0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:46.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.901 7f6238266700 1 -- 172.21.2.101:0/488425162 >> 172.21.2.101:0/488425162 conn(0x7f62301a3780 msgr2=0x7f6230198c50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:46.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.901 7f6238266700 1 -- 172.21.2.101:0/488425162 shutdown_connections 2021-02-10T11:31:46.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:46.901 7f6238266700 1 -- 172.21.2.101:0/488425162 wait complete. 2021-02-10T11:31:46.920 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:31:29.865604', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:31:30.939202', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:31:46.921 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:31:47.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.326 7efd7c4c3700 1 Processor -- start 2021-02-10T11:31:47.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.326 7efd7c4c3700 1 -- start start 2021-02-10T11:31:47.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.326 7efd7c4c3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd741b0470 0x7efd741b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:47.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.326 7efd7c4c3700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd740f0b50 con 0x7efd740ef610 2021-02-10T11:31:47.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.326 7efd7c4c3700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd740f24e0 con 0x7efd740f20e0 2021-02-10T11:31:47.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.326 7efd7c4c3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd740e5ea0 con 0x7efd741b0470 2021-02-10T11:31:47.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.327 7efd79ade700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7efd740ef610 0x7efd740efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50846/0 (socket says 172.21.2.101:50846) 2021-02-10T11:31:47.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.327 7efd79ade700 1 -- 172.21.2.101:0/2774885206 learned_addr learned my addr 172.21.2.101:0/2774885206 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:47.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.327 7efd7a2df700 1 --2- 172.21.2.101:0/2774885206 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd741b0470 0x7efd741b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:47.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.327 7efd792dd700 1 -- 172.21.2.101:0/2774885206 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312640105 0 0) 0x7efd700010b0 con 0x7efd740ef610 2021-02-10T11:31:47.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.327 7efd792dd700 1 -- 172.21.2.101:0/2774885206 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd60002fe0 con 0x7efd740ef610 2021-02-10T11:31:47.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.327 7efd792dd700 1 -- 172.21.2.101:0/2774885206 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 96680551 0 0) 0x7efd700010b0 con 0x7efd740ef610 2021-02-10T11:31:47.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.327 7efd792dd700 1 -- 172.21.2.101:0/2774885206 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd741b0470 msgr2=0x7efd741b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:47.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.327 7efd792dd700 1 --2- 172.21.2.101:0/2774885206 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd741b0470 0x7efd741b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:47.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.327 7efd792dd700 1 -- 172.21.2.101:0/2774885206 >> v1:172.21.2.108:6789/0 conn(0x7efd740f20e0 legacy=0x7efd740f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:47.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.327 7efd792dd700 1 -- 172.21.2.101:0/2774885206 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd740e52e0 con 0x7efd740ef610 2021-02-10T11:31:47.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.327 7efd792dd700 1 -- 172.21.2.101:0/2774885206 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7efd700015b0 con 0x7efd740ef610 2021-02-10T11:31:47.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.327 7efd792dd700 1 -- 172.21.2.101:0/2774885206 >> v1:172.21.2.101:6789/0 conn(0x7efd740ef610 legacy=0x7efd740efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:47.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.327 7efd792dd700 1 --2- 172.21.2.101:0/2774885206 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd60003ac0 0x7efd60005ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:47.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.327 7efd792dd700 1 --2- 172.21.2.101:0/2774885206 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd60006770 0x7efd60008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:47.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.327 7efd792dd700 1 --2- 172.21.2.101:0/2774885206 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd60009510 0x7efd6000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:47.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.327 7efd792dd700 1 -- 172.21.2.101:0/2774885206 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6000c2e0 con 0x7efd60009510 2021-02-10T11:31:47.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.327 7efd792dd700 1 -- 172.21.2.101:0/2774885206 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6000c500 con 0x7efd60006770 2021-02-10T11:31:47.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.327 7efd792dd700 1 -- 172.21.2.101:0/2774885206 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd6000c720 con 0x7efd60003ac0 2021-02-10T11:31:47.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.328 7efd7a2df700 1 --2- 172.21.2.101:0/2774885206 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd60003ac0 0x7efd60005ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:47.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.328 7efd7aae0700 1 --2- 172.21.2.101:0/2774885206 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd60009510 0x7efd6000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:47.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.328 7efd79ade700 1 --2- 172.21.2.101:0/2774885206 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd60006770 0x7efd60008c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:47.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.328 7efd7aae0700 1 -- 172.21.2.101:0/2774885206 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd60003ac0 msgr2=0x7efd60005ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:47.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.328 7efd7aae0700 1 --2- 172.21.2.101:0/2774885206 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd60003ac0 0x7efd60005ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:47.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.328 7efd7aae0700 1 -- 172.21.2.101:0/2774885206 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd60006770 msgr2=0x7efd60008c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:47.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.328 7efd7aae0700 1 --2- 172.21.2.101:0/2774885206 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd60006770 0x7efd60008c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:47.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.328 7efd7aae0700 1 -- 172.21.2.101:0/2774885206 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd60050990 con 0x7efd60009510 2021-02-10T11:31:47.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.328 7efd7aae0700 1 --2- 172.21.2.101:0/2774885206 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd60009510 0x7efd6000ba30 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7efd6c006e40 tx=0x7efd6c009e30).ready entity=mon.0 client_cookie=5367fc35521fa41e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:47.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.328 7efd792dd700 1 -- 172.21.2.101:0/2774885206 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd6c003020 con 0x7efd60009510 2021-02-10T11:31:47.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.328 7efd792dd700 1 -- 172.21.2.101:0/2774885206 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6c005830 con 0x7efd60009510 2021-02-10T11:31:47.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.328 7efd792dd700 1 -- 172.21.2.101:0/2774885206 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd6c005d50 con 0x7efd60009510 2021-02-10T11:31:47.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.328 7efd7c4c3700 1 -- 172.21.2.101:0/2774885206 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd60009510 msgr2=0x7efd6000ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:47.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.328 7efd7c4c3700 1 --2- 172.21.2.101:0/2774885206 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd60009510 0x7efd6000ba30 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7efd6c006e40 tx=0x7efd6c009e30).stop 2021-02-10T11:31:47.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.328 7efd7aae0700 1 -- 172.21.2.101:0/2774885206 reap_dead start 2021-02-10T11:31:47.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.329 7efd7c4c3700 1 -- 172.21.2.101:0/2774885206 shutdown_connections 2021-02-10T11:31:47.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.329 7efd7c4c3700 1 -- 172.21.2.101:0/2774885206 >> 172.21.2.101:0/2774885206 conn(0x7efd740ce3e0 msgr2=0x7efd740e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:47.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.329 7efd7c4c3700 1 -- 172.21.2.101:0/2774885206 shutdown_connections 2021-02-10T11:31:47.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.329 7efd7c4c3700 1 -- 172.21.2.101:0/2774885206 wait complete. 2021-02-10T11:31:47.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.344 7efd7c4c3700 1 Processor -- start 2021-02-10T11:31:47.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.344 7efd7c4c3700 1 -- start start 2021-02-10T11:31:47.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.344 7efd7c4c3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd7409e560 0x7efd740a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:47.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.344 7efd7c4c3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd7409e960 0x7efd740a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:47.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.344 7efd7c4c3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd740a0d70 0x7efd740a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:47.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.344 7efd7c4c3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd740c38f0 con 0x7efd7409e960 2021-02-10T11:31:47.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.344 7efd7c4c3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd740a1170 con 0x7efd740a0d70 2021-02-10T11:31:47.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.344 7efd7c4c3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd740a1390 con 0x7efd7409e560 2021-02-10T11:31:47.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.344 7efd79ade700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd7409e960 0x7efd740a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:47.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.344 7efd79ade700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd7409e960 0x7efd740a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50366/0 (socket says 172.21.2.101:50366) 2021-02-10T11:31:47.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.344 7efd79ade700 1 -- 172.21.2.101:0/2729416553 learned_addr learned my addr 172.21.2.101:0/2729416553 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:47.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.345 7efd7aae0700 1 --2- 172.21.2.101:0/2729416553 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd740a0d70 0x7efd740a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:47.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.345 7efd7a2df700 1 --2- 172.21.2.101:0/2729416553 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd7409e560 0x7efd740a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:47.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.345 7efd79ade700 1 -- 172.21.2.101:0/2729416553 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd7409e560 msgr2=0x7efd740a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:47.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.345 7efd79ade700 1 --2- 172.21.2.101:0/2729416553 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd7409e560 0x7efd740a0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:47.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.345 7efd79ade700 1 -- 172.21.2.101:0/2729416553 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd740a0d70 msgr2=0x7efd740a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:47.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.345 7efd79ade700 1 --2- 172.21.2.101:0/2729416553 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd740a0d70 0x7efd740a1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:47.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.345 7efd79ade700 1 -- 172.21.2.101:0/2729416553 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd740a2110 con 0x7efd7409e960 2021-02-10T11:31:47.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.345 7efd79ade700 1 --2- 172.21.2.101:0/2729416553 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd7409e960 0x7efd740a0750 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7efd70001660 tx=0x7efd7000a930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:47.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.345 7efd66ffd700 1 -- 172.21.2.101:0/2729416553 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd7000ad20 con 0x7efd7409e960 2021-02-10T11:31:47.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.345 7efd66ffd700 1 -- 172.21.2.101:0/2729416553 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd700016a0 con 0x7efd7409e960 2021-02-10T11:31:47.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.345 7efd66ffd700 1 -- 172.21.2.101:0/2729416553 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd70004330 con 0x7efd7409e960 2021-02-10T11:31:47.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.345 7efd7c4c3700 1 -- 172.21.2.101:0/2729416553 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd740a2110 con 0x7efd7409e960 2021-02-10T11:31:47.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.345 7efd7c4c3700 1 -- 172.21.2.101:0/2729416553 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd740a2110 con 0x7efd7409e960 2021-02-10T11:31:47.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.345 7efd66ffd700 1 -- 172.21.2.101:0/2729416553 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7efd70004140 con 0x7efd7409e960 2021-02-10T11:31:47.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.345 7efd66ffd700 1 --2- 172.21.2.101:0/2729416553 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd5c018730 0x7efd5c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:47.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.346 7efd66ffd700 1 -- 172.21.2.101:0/2729416553 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7efd70032670 con 0x7efd7409e960 2021-02-10T11:31:47.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.346 7efd7a2df700 1 --2- 172.21.2.101:0/2729416553 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd5c018730 0x7efd5c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:47.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.346 7efd7a2df700 1 --2- 172.21.2.101:0/2729416553 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd5c018730 0x7efd5c01abc0 crc :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:47.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.376 7efd7c4c3700 1 -- 172.21.2.101:0/2729416553 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd74105b30 con 0x7efd7409e960 2021-02-10T11:31:47.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.379 7efd66ffd700 1 -- 172.21.2.101:0/2729416553 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7efd7000ad20 con 0x7efd7409e960 2021-02-10T11:31:47.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.507 7efd7c4c3700 1 -- 172.21.2.101:0/2729416553 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7efd740f0b50 con 0x7efd7409e960 2021-02-10T11:31:47.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.508 7efd66ffd700 1 -- 172.21.2.101:0/2729416553 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v38) v1 ==== 92+0+10 (secure 0 0 0) 0x7efd70004140 con 0x7efd7409e960 2021-02-10T11:31:47.519 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:31:47.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.510 7efd7c4c3700 1 -- 172.21.2.101:0/2729416553 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd5c018730 msgr2=0x7efd5c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:47.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.510 7efd7c4c3700 1 --2- 172.21.2.101:0/2729416553 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd5c018730 0x7efd5c01abc0 crc :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:47.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.510 7efd7c4c3700 1 -- 172.21.2.101:0/2729416553 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd7409e960 msgr2=0x7efd740a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:47.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.510 7efd7c4c3700 1 --2- 172.21.2.101:0/2729416553 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd7409e960 0x7efd740a0750 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7efd70001660 tx=0x7efd7000a930).stop 2021-02-10T11:31:47.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.510 7efd7c4c3700 1 -- 172.21.2.101:0/2729416553 shutdown_connections 2021-02-10T11:31:47.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.510 7efd7c4c3700 1 --2- 172.21.2.101:0/2729416553 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd740a0d70 0x7efd740a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:47.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.510 7efd7c4c3700 1 --2- 172.21.2.101:0/2729416553 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd7409e960 0x7efd740a0750 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:47.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.510 7efd7c4c3700 1 --2- 172.21.2.101:0/2729416553 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd5c018730 0x7efd5c01abc0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:47.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.510 7efd7c4c3700 1 --2- 172.21.2.101:0/2729416553 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd7409e560 0x7efd740a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:47.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.510 7efd7c4c3700 1 -- 172.21.2.101:0/2729416553 >> 172.21.2.101:0/2729416553 conn(0x7efd7409fc10 msgr2=0x7efd74094ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:47.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.510 7efd7c4c3700 1 -- 172.21.2.101:0/2729416553 shutdown_connections 2021-02-10T11:31:47.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.510 7efd7c4c3700 1 -- 172.21.2.101:0/2729416553 wait complete. 2021-02-10T11:31:47.529 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:31:47.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.931 7fbf44333700 1 Processor -- start 2021-02-10T11:31:47.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.931 7fbf44333700 1 -- start start 2021-02-10T11:31:47.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.931 7fbf44333700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbf3c1b0470 0x7fbf3c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:47.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.931 7fbf44333700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbf3c0ef610 0x7fbf3c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:47.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.931 7fbf44333700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf3c0f0b50 con 0x7fbf3c0f0030 2021-02-10T11:31:47.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.931 7fbf44333700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3c0e52e0 con 0x7fbf3c1b0470 2021-02-10T11:31:47.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.931 7fbf44333700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3c0e5490 con 0x7fbf3c0ef610 2021-02-10T11:31:47.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.932 7fbf4214f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbf3c1b0470 0x7fbf3c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:47.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.932 7fbf4214f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbf3c1b0470 0x7fbf3c1b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50370/0 (socket says 172.21.2.101:50370) 2021-02-10T11:31:47.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.932 7fbf4214f700 1 -- 172.21.2.101:0/1409480322 learned_addr learned my addr 172.21.2.101:0/1409480322 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:47.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.932 7fbf4214f700 1 -- 172.21.2.101:0/1409480322 >> v2:172.21.2.108:3300/0 conn(0x7fbf3c0ef610 msgr2=0x7fbf3c0efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:31:47.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.932 7fbf4214f700 1 --2- 172.21.2.101:0/1409480322 >> v2:172.21.2.108:3300/0 conn(0x7fbf3c0ef610 0x7fbf3c0efa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:47.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.932 7fbf4214f700 1 -- 172.21.2.101:0/1409480322 >> v1:172.21.2.108:6789/0 conn(0x7fbf3c0f0030 legacy=0x7fbf3c0f10d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:31:47.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.932 7fbf4214f700 1 -- 172.21.2.101:0/1409480322 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c0f0430 con 0x7fbf3c1b0470 2021-02-10T11:31:47.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.932 7fbf4214f700 1 --2- 172.21.2.101:0/1409480322 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf3c1b0470 0x7fbf3c1b0870 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7fbf30006700 tx=0x7fbf30009b00).ready entity=mon.0 client_cookie=41b069e4432f91e2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:47.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.932 7fbf4114d700 1 -- 172.21.2.101:0/1409480322 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbf3000c0c0 con 0x7fbf3c1b0470 2021-02-10T11:31:47.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.932 7fbf4114d700 1 -- 172.21.2.101:0/1409480322 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3000c400 con 0x7fbf3c1b0470 2021-02-10T11:31:47.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.932 7fbf4114d700 1 -- 172.21.2.101:0/1409480322 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbf3000c920 con 0x7fbf3c1b0470 2021-02-10T11:31:47.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.932 7fbf44333700 1 -- 172.21.2.101:0/1409480322 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf3c1b0470 msgr2=0x7fbf3c1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:47.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.932 7fbf44333700 1 --2- 172.21.2.101:0/1409480322 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf3c1b0470 0x7fbf3c1b0870 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7fbf30006700 tx=0x7fbf30009b00).stop 2021-02-10T11:31:47.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.933 7fbf44333700 1 -- 172.21.2.101:0/1409480322 shutdown_connections 2021-02-10T11:31:47.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.933 7fbf44333700 1 --2- 172.21.2.101:0/1409480322 >> v2:172.21.2.108:3300/0 conn(0x7fbf3c0ef610 0x7fbf3c0efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:47.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.933 7fbf44333700 1 --2- 172.21.2.101:0/1409480322 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf3c1b0470 0x7fbf3c1b0870 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:47.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.933 7fbf44333700 1 -- 172.21.2.101:0/1409480322 >> 172.21.2.101:0/1409480322 conn(0x7fbf3c0ce3e0 msgr2=0x7fbf3c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:47.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.933 7fbf44333700 1 -- 172.21.2.101:0/1409480322 shutdown_connections 2021-02-10T11:31:47.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.933 7fbf44333700 1 -- 172.21.2.101:0/1409480322 wait complete. 2021-02-10T11:31:47.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.947 7fbf44333700 1 Processor -- start 2021-02-10T11:31:47.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.948 7fbf44333700 1 -- start start 2021-02-10T11:31:47.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.948 7fbf44333700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbf3c0c36a0 0x7fbf3c0c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:47.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.948 7fbf44333700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf3c0c3190 0x7fbf3c0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:47.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.948 7fbf44333700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf3c0b2370 0x7fbf3c0b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:47.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.948 7fbf44333700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0b27c0 con 0x7fbf3c0b2370 2021-02-10T11:31:47.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.948 7fbf44333700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0b3360 con 0x7fbf3c0c36a0 2021-02-10T11:31:47.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.948 7fbf44333700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0b3540 con 0x7fbf3c0c3190 2021-02-10T11:31:47.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.948 7fbf42950700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf3c0b2370 0x7fbf3c0b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:47.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.948 7fbf42950700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf3c0b2370 0x7fbf3c0b2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50374/0 (socket says 172.21.2.101:50374) 2021-02-10T11:31:47.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.948 7fbf42950700 1 -- 172.21.2.101:0/2837166366 learned_addr learned my addr 172.21.2.101:0/2837166366 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:47.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.948 7fbf42950700 1 -- 172.21.2.101:0/2837166366 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf3c0c3190 msgr2=0x7fbf3c0b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:47.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.948 7fbf42950700 1 --2- 172.21.2.101:0/2837166366 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf3c0c3190 0x7fbf3c0b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:47.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.948 7fbf42950700 1 -- 172.21.2.101:0/2837166366 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbf3c0c36a0 msgr2=0x7fbf3c0c2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:47.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.948 7fbf42950700 1 --2- 172.21.2.101:0/2837166366 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbf3c0c36a0 0x7fbf3c0c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:47.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.949 7fbf42950700 1 -- 172.21.2.101:0/2837166366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c0b3760 con 0x7fbf3c0b2370 2021-02-10T11:31:47.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.949 7fbf42950700 1 --2- 172.21.2.101:0/2837166366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf3c0b2370 0x7fbf3c0b2d40 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7fbf34009940 tx=0x7fbf3400bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:47.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.949 7fbf2effd700 1 -- 172.21.2.101:0/2837166366 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbf34003030 con 0x7fbf3c0b2370 2021-02-10T11:31:47.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.949 7fbf2effd700 1 -- 172.21.2.101:0/2837166366 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf34001270 con 0x7fbf3c0b2370 2021-02-10T11:31:47.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.949 7fbf2effd700 1 -- 172.21.2.101:0/2837166366 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbf34001750 con 0x7fbf3c0b2370 2021-02-10T11:31:47.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.949 7fbf44333700 1 -- 172.21.2.101:0/2837166366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3c0b3760 con 0x7fbf3c0b2370 2021-02-10T11:31:47.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.949 7fbf44333700 1 -- 172.21.2.101:0/2837166366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3c0b3760 con 0x7fbf3c0b2370 2021-02-10T11:31:47.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.950 7fbf2effd700 1 -- 172.21.2.101:0/2837166366 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbf34001450 con 0x7fbf3c0b2370 2021-02-10T11:31:47.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.950 7fbf2effd700 1 --2- 172.21.2.101:0/2837166366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf240186c0 0x7fbf2401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:47.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.950 7fbf2effd700 1 -- 172.21.2.101:0/2837166366 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fbf340327a0 con 0x7fbf3c0b2370 2021-02-10T11:31:47.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.950 7fbf4214f700 1 --2- 172.21.2.101:0/2837166366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf240186c0 0x7fbf2401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:47.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.950 7fbf4214f700 1 --2- 172.21.2.101:0/2837166366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf240186c0 0x7fbf2401abc0 crc :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:47.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.981 7fbf44333700 1 -- 172.21.2.101:0/2837166366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3c0edd10 con 0x7fbf3c0b2370 2021-02-10T11:31:47.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:47.984 7fbf2effd700 1 -- 172.21.2.101:0/2837166366 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbf3400be30 con 0x7fbf3c0b2370 2021-02-10T11:31:48.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.113 7fbf44333700 1 -- 172.21.2.101:0/2837166366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7fbf3c0edd10 con 0x7fbf3c0b2370 2021-02-10T11:31:48.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.113 7fbf2effd700 1 -- 172.21.2.101:0/2837166366 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v38) v1 ==== 104+0+10 (secure 0 0 0) 0x7fbf34001450 con 0x7fbf3c0b2370 2021-02-10T11:31:48.125 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:31:48.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.115 7fbf44333700 1 -- 172.21.2.101:0/2837166366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf240186c0 msgr2=0x7fbf2401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:48.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.115 7fbf44333700 1 --2- 172.21.2.101:0/2837166366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf240186c0 0x7fbf2401abc0 crc :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:48.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.115 7fbf44333700 1 -- 172.21.2.101:0/2837166366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf3c0b2370 msgr2=0x7fbf3c0b2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:48.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.115 7fbf44333700 1 --2- 172.21.2.101:0/2837166366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf3c0b2370 0x7fbf3c0b2d40 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7fbf34009940 tx=0x7fbf3400bcf0).stop 2021-02-10T11:31:48.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.115 7fbf44333700 1 -- 172.21.2.101:0/2837166366 shutdown_connections 2021-02-10T11:31:48.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.115 7fbf44333700 1 --2- 172.21.2.101:0/2837166366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf240186c0 0x7fbf2401abc0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:48.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.115 7fbf44333700 1 --2- 172.21.2.101:0/2837166366 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf3c0c3190 0x7fbf3c0b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:48.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.115 7fbf44333700 1 --2- 172.21.2.101:0/2837166366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf3c0b2370 0x7fbf3c0b2d40 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:48.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.115 7fbf44333700 1 --2- 172.21.2.101:0/2837166366 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbf3c0c36a0 0x7fbf3c0c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:48.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.115 7fbf44333700 1 -- 172.21.2.101:0/2837166366 >> 172.21.2.101:0/2837166366 conn(0x7fbf3c09f630 msgr2=0x7fbf3c0a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:48.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.115 7fbf44333700 1 -- 172.21.2.101:0/2837166366 shutdown_connections 2021-02-10T11:31:48.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.116 7fbf44333700 1 -- 172.21.2.101:0/2837166366 wait complete. 2021-02-10T11:31:48.134 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:31:48.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.537 7fdac45f2700 1 Processor -- start 2021-02-10T11:31:48.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.537 7fdac45f2700 1 -- start start 2021-02-10T11:31:48.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.538 7fdac45f2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdabc1b0470 0x7fdabc1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:48.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.538 7fdac45f2700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdabc0f0b50 con 0x7fdabc0f20e0 2021-02-10T11:31:48.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.538 7fdac45f2700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdabc0f24e0 con 0x7fdabc0ef610 2021-02-10T11:31:48.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.538 7fdac45f2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc0e5ea0 con 0x7fdabc1b0470 2021-02-10T11:31:48.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.538 7fdac2c0f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fdabc0f20e0 0x7fdabc0f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50876/0 (socket says 172.21.2.101:50876) 2021-02-10T11:31:48.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.538 7fdac2c0f700 1 -- 172.21.2.101:0/2225974317 learned_addr learned my addr 172.21.2.101:0/2225974317 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:48.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.538 7fdac140c700 1 -- 172.21.2.101:0/2225974317 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2446967410 0 0) 0x7fdab4001500 con 0x7fdabc0f20e0 2021-02-10T11:31:48.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.538 7fdac140c700 1 -- 172.21.2.101:0/2225974317 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaa8002fe0 con 0x7fdabc0f20e0 2021-02-10T11:31:48.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.538 7fdac140c700 1 -- 172.21.2.101:0/2225974317 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2577017807 0 0) 0x7fdab4001460 con 0x7fdabc0f20e0 2021-02-10T11:31:48.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.538 7fdac140c700 1 -- 172.21.2.101:0/2225974317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdabc1b0470 msgr2=0x7fdabc1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:48.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.538 7fdac140c700 1 --2- 172.21.2.101:0/2225974317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdabc1b0470 0x7fdabc1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:48.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.538 7fdac140c700 1 -- 172.21.2.101:0/2225974317 >> v1:172.21.2.108:6789/0 conn(0x7fdabc0ef610 legacy=0x7fdabc0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:48.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.538 7fdac140c700 1 -- 172.21.2.101:0/2225974317 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc0e52e0 con 0x7fdabc0f20e0 2021-02-10T11:31:48.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.538 7fdac140c700 1 -- 172.21.2.101:0/2225974317 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fdab4001980 con 0x7fdabc0f20e0 2021-02-10T11:31:48.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.538 7fdac140c700 1 -- 172.21.2.101:0/2225974317 >> v1:172.21.2.101:6789/0 conn(0x7fdabc0f20e0 legacy=0x7fdabc0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:48.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.538 7fdac140c700 1 --2- 172.21.2.101:0/2225974317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdaa8003a70 0x7fdaa8005f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:48.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.538 7fdac140c700 1 --2- 172.21.2.101:0/2225974317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdaa80067a0 0x7fdaa8008ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:48.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.538 7fdac140c700 1 --2- 172.21.2.101:0/2225974317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdaa80095a0 0x7fdaa800ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:48.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.538 7fdac140c700 1 -- 172.21.2.101:0/2225974317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa800c2e0 con 0x7fdaa8003a70 2021-02-10T11:31:48.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.538 7fdac140c700 1 -- 172.21.2.101:0/2225974317 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa800c500 con 0x7fdaa80067a0 2021-02-10T11:31:48.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.538 7fdac140c700 1 -- 172.21.2.101:0/2225974317 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa800c720 con 0x7fdaa80095a0 2021-02-10T11:31:48.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.539 7fdac240e700 1 --2- 172.21.2.101:0/2225974317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdaa8003a70 0x7fdaa8005f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:48.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.539 7fdac2c0f700 1 --2- 172.21.2.101:0/2225974317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdaa80095a0 0x7fdaa800ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:48.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.539 7fdac1c0d700 1 --2- 172.21.2.101:0/2225974317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdaa80067a0 0x7fdaa8008ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:48.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.539 7fdac240e700 1 -- 172.21.2.101:0/2225974317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdaa80095a0 msgr2=0x7fdaa800ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:48.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.539 7fdac240e700 1 --2- 172.21.2.101:0/2225974317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdaa80095a0 0x7fdaa800ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:48.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.539 7fdac240e700 1 -- 172.21.2.101:0/2225974317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdaa80067a0 msgr2=0x7fdaa8008ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:48.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.539 7fdac240e700 1 --2- 172.21.2.101:0/2225974317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdaa80067a0 0x7fdaa8008ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:48.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.539 7fdac240e700 1 -- 172.21.2.101:0/2225974317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa800c9c0 con 0x7fdaa8003a70 2021-02-10T11:31:48.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.539 7fdac240e700 1 --2- 172.21.2.101:0/2225974317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdaa8003a70 0x7fdaa8005f00 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0x7fdab0006ab0 tx=0x7fdab0009b40).ready entity=mon.0 client_cookie=4f0409f36843d289 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:48.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.539 7fdac140c700 1 -- 172.21.2.101:0/2225974317 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdab0000c90 con 0x7fdaa8003a70 2021-02-10T11:31:48.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.539 7fdac140c700 1 -- 172.21.2.101:0/2225974317 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab0002030 con 0x7fdaa8003a70 2021-02-10T11:31:48.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.539 7fdac140c700 1 -- 172.21.2.101:0/2225974317 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdab0002480 con 0x7fdaa8003a70 2021-02-10T11:31:48.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.539 7fdac45f2700 1 -- 172.21.2.101:0/2225974317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdaa8003a70 msgr2=0x7fdaa8005f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:48.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.539 7fdac45f2700 1 --2- 172.21.2.101:0/2225974317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdaa8003a70 0x7fdaa8005f00 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0x7fdab0006ab0 tx=0x7fdab0009b40).stop 2021-02-10T11:31:48.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.539 7fdac2c0f700 1 -- 172.21.2.101:0/2225974317 reap_dead start 2021-02-10T11:31:48.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.540 7fdac45f2700 1 -- 172.21.2.101:0/2225974317 shutdown_connections 2021-02-10T11:31:48.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.540 7fdac45f2700 1 -- 172.21.2.101:0/2225974317 >> 172.21.2.101:0/2225974317 conn(0x7fdabc0ce3e0 msgr2=0x7fdabc0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:48.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.540 7fdac45f2700 1 -- 172.21.2.101:0/2225974317 shutdown_connections 2021-02-10T11:31:48.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.540 7fdac45f2700 1 -- 172.21.2.101:0/2225974317 wait complete. 2021-02-10T11:31:48.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.555 7fdac45f2700 1 Processor -- start 2021-02-10T11:31:48.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.555 7fdac45f2700 1 -- start start 2021-02-10T11:31:48.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.555 7fdac45f2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdabc0b7db0 0x7fdabc0b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:48.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.555 7fdac45f2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdabc0b8880 0x7fdabc1a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:48.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.555 7fdac45f2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdabc0b8c80 0x7fdabc1a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:48.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.555 7fdac45f2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc0b79b0 con 0x7fdabc0b7db0 2021-02-10T11:31:48.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.555 7fdac45f2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc1a8090 con 0x7fdabc0b8c80 2021-02-10T11:31:48.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.555 7fdac45f2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc1a82b0 con 0x7fdabc0b8880 2021-02-10T11:31:48.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.555 7fdac240e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdabc0b7db0 0x7fdabc0b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:48.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.555 7fdac240e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdabc0b7db0 0x7fdabc0b8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50394/0 (socket says 172.21.2.101:50394) 2021-02-10T11:31:48.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.555 7fdac240e700 1 -- 172.21.2.101:0/3041222366 learned_addr learned my addr 172.21.2.101:0/3041222366 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:48.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.556 7fdac240e700 1 -- 172.21.2.101:0/3041222366 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdabc0b8880 msgr2=0x7fdabc1a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:48.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.556 7fdac240e700 1 --2- 172.21.2.101:0/3041222366 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdabc0b8880 0x7fdabc1a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:48.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.556 7fdac240e700 1 -- 172.21.2.101:0/3041222366 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdabc0b8c80 msgr2=0x7fdabc1a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:48.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.556 7fdac240e700 1 --2- 172.21.2.101:0/3041222366 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdabc0b8c80 0x7fdabc1a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:48.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.556 7fdac240e700 1 -- 172.21.2.101:0/3041222366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc190c10 con 0x7fdabc0b7db0 2021-02-10T11:31:48.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.556 7fdac240e700 1 --2- 172.21.2.101:0/3041222366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdabc0b7db0 0x7fdabc0b8260 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7fdab0006f50 tx=0x7fdab000b9b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:48.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.556 7fdaaeffd700 1 -- 172.21.2.101:0/3041222366 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdab000bda0 con 0x7fdabc0b7db0 2021-02-10T11:31:48.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.556 7fdaaeffd700 1 -- 172.21.2.101:0/3041222366 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab0000c90 con 0x7fdabc0b7db0 2021-02-10T11:31:48.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.556 7fdaaeffd700 1 -- 172.21.2.101:0/3041222366 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdab00022a0 con 0x7fdabc0b7db0 2021-02-10T11:31:48.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.556 7fdac45f2700 1 -- 172.21.2.101:0/3041222366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdabc190c10 con 0x7fdabc0b7db0 2021-02-10T11:31:48.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.556 7fdac45f2700 1 -- 172.21.2.101:0/3041222366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdabc0c1eb0 con 0x7fdabc0b7db0 2021-02-10T11:31:48.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.557 7fdaaeffd700 1 -- 172.21.2.101:0/3041222366 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdab0005310 con 0x7fdabc0b7db0 2021-02-10T11:31:48.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.557 7fdaaeffd700 1 --2- 172.21.2.101:0/3041222366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdaa40186c0 0x7fdaa401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:48.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.557 7fdaaeffd700 1 -- 172.21.2.101:0/3041222366 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fdab0032500 con 0x7fdabc0b7db0 2021-02-10T11:31:48.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.557 7fdac1c0d700 1 --2- 172.21.2.101:0/3041222366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdaa40186c0 0x7fdaa401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:48.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.557 7fdac1c0d700 1 --2- 172.21.2.101:0/3041222366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdaa40186c0 0x7fdaa401abc0 crc :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:48.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.588 7fdac45f2700 1 -- 172.21.2.101:0/3041222366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdabc105b30 con 0x7fdabc0b7db0 2021-02-10T11:31:48.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.590 7fdaaeffd700 1 -- 172.21.2.101:0/3041222366 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdab0025b00 con 0x7fdabc0b7db0 2021-02-10T11:31:48.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.717 7fdac45f2700 1 -- 172.21.2.101:0/3041222366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7fdabc0f0b50 con 0x7fdabc0b7db0 2021-02-10T11:31:48.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.718 7fdaaeffd700 1 -- 172.21.2.101:0/3041222366 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v38) v1 ==== 100+0+10 (secure 0 0 0) 0x7fdab0025b00 con 0x7fdabc0b7db0 2021-02-10T11:31:48.729 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:31:48.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.720 7fdac45f2700 1 -- 172.21.2.101:0/3041222366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdaa40186c0 msgr2=0x7fdaa401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:48.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.720 7fdac45f2700 1 --2- 172.21.2.101:0/3041222366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdaa40186c0 0x7fdaa401abc0 crc :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:48.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.720 7fdac45f2700 1 -- 172.21.2.101:0/3041222366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdabc0b7db0 msgr2=0x7fdabc0b8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:48.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.720 7fdac45f2700 1 --2- 172.21.2.101:0/3041222366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdabc0b7db0 0x7fdabc0b8260 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7fdab0006f50 tx=0x7fdab000b9b0).stop 2021-02-10T11:31:48.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.720 7fdac45f2700 1 -- 172.21.2.101:0/3041222366 shutdown_connections 2021-02-10T11:31:48.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.720 7fdac45f2700 1 --2- 172.21.2.101:0/3041222366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdaa40186c0 0x7fdaa401abc0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:48.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.720 7fdac45f2700 1 --2- 172.21.2.101:0/3041222366 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdabc0b8880 0x7fdabc1a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:48.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.720 7fdac45f2700 1 --2- 172.21.2.101:0/3041222366 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdabc0b8c80 0x7fdabc1a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:48.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.720 7fdac45f2700 1 --2- 172.21.2.101:0/3041222366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdabc0b7db0 0x7fdabc0b8260 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:48.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.720 7fdac45f2700 1 -- 172.21.2.101:0/3041222366 >> 172.21.2.101:0/3041222366 conn(0x7fdabc094e10 msgr2=0x7fdabc0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:48.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.720 7fdac45f2700 1 -- 172.21.2.101:0/3041222366 shutdown_connections 2021-02-10T11:31:48.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:48.720 7fdac45f2700 1 -- 172.21.2.101:0/3041222366 wait complete. 2021-02-10T11:31:48.739 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:31:48.739 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:31:49.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.139 7fb330b74700 1 Processor -- start 2021-02-10T11:31:49.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.139 7fb330b74700 1 -- start start 2021-02-10T11:31:49.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.139 7fb330b74700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb32c0e5470 0x7fb32c07e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:49.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.139 7fb330b74700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb32c07f300 0x7fb32c07f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:49.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.139 7fb330b74700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb32c07ed80 con 0x7fb32c0e5070 2021-02-10T11:31:49.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.139 7fb330b74700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb32c1b4010 con 0x7fb32c0e5470 2021-02-10T11:31:49.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.139 7fb330b74700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c0d1790 con 0x7fb32c07f300 2021-02-10T11:31:49.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.139 7fb32a05d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb32c0e5470 0x7fb32c07e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:49.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.140 7fb32a05d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb32c0e5470 0x7fb32c07e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50402/0 (socket says 172.21.2.101:50402) 2021-02-10T11:31:49.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.140 7fb32a85e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb32c0e5070 0x7fb32c1b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50894/0 (socket says 172.21.2.101:50894) 2021-02-10T11:31:49.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.140 7fb32a05d700 1 -- 172.21.2.101:0/461455610 learned_addr learned my addr 172.21.2.101:0/461455610 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:49.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.140 7fb32a05d700 1 -- 172.21.2.101:0/461455610 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb32c07f300 msgr2=0x7fb32c07f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:49.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.140 7fb32a05d700 1 --2- 172.21.2.101:0/461455610 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb32c07f300 0x7fb32c07f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:49.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.140 7fb32a05d700 1 -- 172.21.2.101:0/461455610 >> v1:172.21.2.101:6789/0 conn(0x7fb32c0e5070 legacy=0x7fb32c1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:49.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.140 7fb32a05d700 1 -- 172.21.2.101:0/461455610 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c0d19b0 con 0x7fb32c0e5470 2021-02-10T11:31:49.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.140 7fb32a05d700 1 --2- 172.21.2.101:0/461455610 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb32c0e5470 0x7fb32c07e6a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0x7fb31c006700 tx=0x7fb31c009b80).ready entity=mon.0 client_cookie=fcd47cd8d152f734 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:49.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.140 7fb32985c700 1 -- 172.21.2.101:0/461455610 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb31c00c0c0 con 0x7fb32c0e5470 2021-02-10T11:31:49.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.140 7fb32985c700 1 -- 172.21.2.101:0/461455610 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31c00c340 con 0x7fb32c0e5470 2021-02-10T11:31:49.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.140 7fb32985c700 1 -- 172.21.2.101:0/461455610 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb31c00c7a0 con 0x7fb32c0e5470 2021-02-10T11:31:49.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.140 7fb330b74700 1 -- 172.21.2.101:0/461455610 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb32c0e5470 msgr2=0x7fb32c07e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:49.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.140 7fb330b74700 1 --2- 172.21.2.101:0/461455610 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb32c0e5470 0x7fb32c07e6a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0x7fb31c006700 tx=0x7fb31c009b80).stop 2021-02-10T11:31:49.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.141 7fb330b74700 1 -- 172.21.2.101:0/461455610 shutdown_connections 2021-02-10T11:31:49.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.141 7fb330b74700 1 --2- 172.21.2.101:0/461455610 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb32c07f300 0x7fb32c07f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:49.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.141 7fb330b74700 1 --2- 172.21.2.101:0/461455610 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb32c0e5470 0x7fb32c07e6a0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:49.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.141 7fb330b74700 1 -- 172.21.2.101:0/461455610 >> 172.21.2.101:0/461455610 conn(0x7fb32c0ce690 msgr2=0x7fb32c0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:49.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.141 7fb330b74700 1 -- 172.21.2.101:0/461455610 shutdown_connections 2021-02-10T11:31:49.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.141 7fb330b74700 1 -- 172.21.2.101:0/461455610 wait complete. 2021-02-10T11:31:49.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.156 7fb330b74700 1 Processor -- start 2021-02-10T11:31:49.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.156 7fb330b74700 1 -- start start 2021-02-10T11:31:49.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.156 7fb330b74700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb32c18c700 0x7fb32c096c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:49.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.156 7fb330b74700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb32c0ae400 0x7fb32c0ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:49.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.156 7fb330b74700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb32c192380 0x7fb32c192780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:49.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.156 7fb330b74700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c097290 con 0x7fb32c192380 2021-02-10T11:31:49.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.156 7fb330b74700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c192da0 con 0x7fb32c18c700 2021-02-10T11:31:49.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.156 7fb330b74700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c192f80 con 0x7fb32c0ae400 2021-02-10T11:31:49.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.156 7fb32b05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb32c192380 0x7fb32c192780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:49.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.156 7fb32b05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb32c192380 0x7fb32c192780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50408/0 (socket says 172.21.2.101:50408) 2021-02-10T11:31:49.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.156 7fb32b05f700 1 -- 172.21.2.101:0/2779447478 learned_addr learned my addr 172.21.2.101:0/2779447478 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:49.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.156 7fb32a85e700 1 --2- 172.21.2.101:0/2779447478 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb32c18c700 0x7fb32c096c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:49.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.157 7fb32a05d700 1 --2- 172.21.2.101:0/2779447478 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb32c0ae400 0x7fb32c0ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:49.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.157 7fb32b05f700 1 -- 172.21.2.101:0/2779447478 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb32c0ae400 msgr2=0x7fb32c0ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:49.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.157 7fb32b05f700 1 --2- 172.21.2.101:0/2779447478 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb32c0ae400 0x7fb32c0ae800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:49.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.157 7fb32b05f700 1 -- 172.21.2.101:0/2779447478 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb32c18c700 msgr2=0x7fb32c096c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:49.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.157 7fb32b05f700 1 --2- 172.21.2.101:0/2779447478 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb32c18c700 0x7fb32c096c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:49.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.157 7fb32b05f700 1 -- 172.21.2.101:0/2779447478 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c1931a0 con 0x7fb32c192380 2021-02-10T11:31:49.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.157 7fb32b05f700 1 --2- 172.21.2.101:0/2779447478 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb32c192380 0x7fb32c192780 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7fb324009980 tx=0x7fb32400bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:49.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.157 7fb3237fe700 1 -- 172.21.2.101:0/2779447478 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb324005870 con 0x7fb32c192380 2021-02-10T11:31:49.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.157 7fb3237fe700 1 -- 172.21.2.101:0/2779447478 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb324005b30 con 0x7fb32c192380 2021-02-10T11:31:49.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.157 7fb330b74700 1 -- 172.21.2.101:0/2779447478 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb32c1931a0 con 0x7fb32c192380 2021-02-10T11:31:49.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.157 7fb3237fe700 1 -- 172.21.2.101:0/2779447478 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb324006030 con 0x7fb32c192380 2021-02-10T11:31:49.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.157 7fb330b74700 1 -- 172.21.2.101:0/2779447478 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb32c07f510 con 0x7fb32c192380 2021-02-10T11:31:49.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.158 7fb3237fe700 1 -- 172.21.2.101:0/2779447478 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb324005d10 con 0x7fb32c192380 2021-02-10T11:31:49.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.158 7fb3237fe700 1 --2- 172.21.2.101:0/2779447478 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb3100186c0 0x7fb31001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:49.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.158 7fb3237fe700 1 -- 172.21.2.101:0/2779447478 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb324032950 con 0x7fb32c192380 2021-02-10T11:31:49.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.158 7fb32a85e700 1 --2- 172.21.2.101:0/2779447478 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb3100186c0 0x7fb31001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:49.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.158 7fb32a85e700 1 --2- 172.21.2.101:0/2779447478 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb3100186c0 0x7fb31001abc0 crc :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:49.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.189 7fb330b74700 1 -- 172.21.2.101:0/2779447478 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb32c107120 con 0x7fb32c192380 2021-02-10T11:31:49.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.192 7fb3237fe700 1 -- 172.21.2.101:0/2779447478 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb3240062b0 con 0x7fb32c192380 2021-02-10T11:31:49.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.319 7fb330b74700 1 -- 172.21.2.101:0/2779447478 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb32c07ed80 con 0x7fb32c192380 2021-02-10T11:31:49.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.320 7fb3237fe700 1 -- 172.21.2.101:0/2779447478 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+2096 (secure 0 0 0) 0x7fb324006f20 con 0x7fb32c192380 2021-02-10T11:31:49.331 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:31:49.331 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/243293953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":243293953},{"type":"v1","addr":"172.21.2.101:6835","nonce":243293953}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:31:31.941616","modified":"2021-02-10 11:31:33.951741","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7299":{"gid":7299,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/1600862555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":1600862555},{"type":"v1","addr":"172.21.2.108:6833","nonce":1600862555}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-02-10T11:31:49.331 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 22 2021-02-10T11:31:49.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.322 7fb330b74700 1 -- 172.21.2.101:0/2779447478 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb3100186c0 msgr2=0x7fb31001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:49.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.322 7fb330b74700 1 --2- 172.21.2.101:0/2779447478 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb3100186c0 0x7fb31001abc0 crc :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:49.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.322 7fb330b74700 1 -- 172.21.2.101:0/2779447478 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb32c192380 msgr2=0x7fb32c192780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:49.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.322 7fb330b74700 1 --2- 172.21.2.101:0/2779447478 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb32c192380 0x7fb32c192780 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7fb324009980 tx=0x7fb32400bcd0).stop 2021-02-10T11:31:49.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.322 7fb330b74700 1 -- 172.21.2.101:0/2779447478 shutdown_connections 2021-02-10T11:31:49.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.322 7fb330b74700 1 --2- 172.21.2.101:0/2779447478 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb3100186c0 0x7fb31001abc0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:49.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.322 7fb330b74700 1 --2- 172.21.2.101:0/2779447478 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb32c0ae400 0x7fb32c0ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:49.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.322 7fb330b74700 1 --2- 172.21.2.101:0/2779447478 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb32c192380 0x7fb32c192780 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:49.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.322 7fb330b74700 1 --2- 172.21.2.101:0/2779447478 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb32c18c700 0x7fb32c096c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:49.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.322 7fb330b74700 1 -- 172.21.2.101:0/2779447478 >> 172.21.2.101:0/2779447478 conn(0x7fb32c0891f0 msgr2=0x7fb32c1a0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:49.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.322 7fb330b74700 1 -- 172.21.2.101:0/2779447478 shutdown_connections 2021-02-10T11:31:49.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:49.322 7fb330b74700 1 -- 172.21.2.101:0/2779447478 wait complete. 2021-02-10T11:31:49.543 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:31:31.941616', 'modified': '2021-02-10 11:31:33.951741', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7299': {'gid': 7299, 'name': 'b', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/1600862555', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 1600862555}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 1600862555}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:31:49.543 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:31:49.543 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:31:49.940 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:31:49.940 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:31:49.940 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:31:49.940 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 7299, 2021-02-10T11:31:49.940 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:31:49.940 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:31:49.940 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 17.986260433000002, 2021-02-10T11:31:49.940 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 22, 2021-02-10T11:31:49.940 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 38, 2021-02-10T11:31:49.941 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 38, 2021-02-10T11:31:49.941 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 23.626278844000002 2021-02-10T11:31:49.941 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:31:49.944 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 7299, "want_state": "up:active", "state": "up:active", "rank_uptime": 17.986260433000002, "mdsmap_epoch": 22, "osdmap_epoch": 38, "osdmap_epoch_barrier": 38, "uptime": 23.626278844000002 } 2021-02-10T11:31:49.944 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:31:49.944 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-02-10T11:31:49.944 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:31:49.944 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:49.944 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:31:50.279 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:31:50.622 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:31:50.622 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:31:50.964 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:31:50.964 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:31:50.965 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:31:50.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:50.965 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:31:51.299 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:31:51.299 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2021-02-10T11:31:51.299 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:31:51.302 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:31:51.647 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:31:51.647 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:31:51.712 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[111008]: starting ceph client 2021-02-10T11:31:51.712 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:2021-02-10 11:31:51.701 7f6e9e90ef80 -1 init, newargv = 0x55d41fa38cb0 newargc=9 2021-02-10T11:31:51.718 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[111008]: starting fuse 2021-02-10T11:31:51.990 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:31:51.990 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:31:51.991 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:31:51.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:51.992 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:31:52.327 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:31:52.327 INFO:teuthology.orchestra.run.gibba001.stdout:44 2021-02-10T11:31:52.328 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2021-02-10T11:31:52.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:52.328 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:31:52.328 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:31:52.328 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:31:52.328 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:31:52.328 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:31:52.328 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:31:52.328 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:31:52.328 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-02-10T11:31:52.328 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:31:52.329 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:31:52.329 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:31:52.329 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:31:52.329 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:31:52.329 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:31:52.329 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:31:52.329 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:31:52.329 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:31:52.329 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:31:52.329 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:31:52.329 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:31:52.329 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:31:52.329 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.1")) 2021-02-10T11:31:52.329 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:31:52.690 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.1.111008.asok 2021-02-10T11:31:52.693 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.111008.asok 2021-02-10T11:31:52.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:52.694 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.111008.asok status 2021-02-10T11:31:53.082 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:31:53.082 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:31:53.082 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:31:53.082 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:31:53.082 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "1", 2021-02-10T11:31:53.083 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:31:53.083 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:31:53.083 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "111008", 2021-02-10T11:31:53.083 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:31:53.083 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:53.083 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:31:53.083 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:31:53.083 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 7509, 2021-02-10T11:31:53.083 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:31:53.083 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:31:53.083 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:31:53.083 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 7509 2021-02-10T11:31:53.084 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:53.084 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:31:53.084 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:53.084 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:31:53.084 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1051061368 2021-02-10T11:31:53.084 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:53.084 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:53.084 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:31:53.084 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:31:53.084 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:31:53.084 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1051061368 2021-02-10T11:31:53.084 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:31:53.084 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.7509 172.21.2.101:0/1051061368", 2021-02-10T11:31:53.085 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/1051061368", 2021-02-10T11:31:53.085 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:31:53.085 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 22, 2021-02-10T11:31:53.085 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 38, 2021-02-10T11:31:53.085 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:31:53.085 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:31:53.085 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:31:53.086 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:53.086 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:31:53.086 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:31:53.421 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:31:53.421 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:31:53.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:31:53.421 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:31:53.767 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2021-02-10T11:31:54.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.248 7f4d45c01700 1 Processor -- start 2021-02-10T11:31:54.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.248 7f4d45c01700 1 -- start start 2021-02-10T11:31:54.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.248 7f4d45c01700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d400d1c70 0x7f4d400cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:54.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.248 7f4d45c01700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4d400cf580 0x7f4d400cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:54.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.248 7f4d45c01700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d400fe1c0 con 0x7f4d400d1870 2021-02-10T11:31:54.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.248 7f4d45c01700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d401b0cb0 con 0x7f4d400cf580 2021-02-10T11:31:54.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.248 7f4d45c01700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d40109a60 con 0x7f4d400d1c70 2021-02-10T11:31:54.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.249 7f4d3f7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f4d400d1870 0x7f4d401b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50920/0 (socket says 172.21.2.101:50920) 2021-02-10T11:31:54.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.249 7f4d3f7fe700 1 -- 172.21.2.101:0/2718413415 learned_addr learned my addr 172.21.2.101:0/2718413415 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:54.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.249 7f4d3e7fc700 1 -- 172.21.2.101:0/2718413415 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2566151490 0 0) 0x7f4d280010b0 con 0x7f4d400d1870 2021-02-10T11:31:54.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.249 7f4d3e7fc700 1 -- 172.21.2.101:0/2718413415 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d2c002fe0 con 0x7f4d400d1870 2021-02-10T11:31:54.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.249 7f4d3ffff700 1 --2- 172.21.2.101:0/2718413415 >> v2:172.21.2.108:3300/0 conn(0x7f4d400cf580 0x7f4d400cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:54.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.249 7f4d3effd700 1 --2- 172.21.2.101:0/2718413415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d400d1c70 0x7f4d400cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:54.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.249 7f4d3e7fc700 1 -- 172.21.2.101:0/2718413415 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2681117098 0 0) 0x7f4d280010b0 con 0x7f4d400d1870 2021-02-10T11:31:54.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.249 7f4d3e7fc700 1 -- 172.21.2.101:0/2718413415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d400d1c70 msgr2=0x7f4d400cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:54.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.249 7f4d3e7fc700 1 --2- 172.21.2.101:0/2718413415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d400d1c70 0x7f4d400cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:54.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.249 7f4d3e7fc700 1 -- 172.21.2.101:0/2718413415 >> v2:172.21.2.108:3300/0 conn(0x7f4d400cf580 msgr2=0x7f4d400cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:54.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.250 7f4d3e7fc700 1 --2- 172.21.2.101:0/2718413415 >> v2:172.21.2.108:3300/0 conn(0x7f4d400cf580 0x7f4d400cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:54.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.250 7f4d3e7fc700 1 -- 172.21.2.101:0/2718413415 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d40088020 con 0x7f4d400d1870 2021-02-10T11:31:54.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.250 7f4d3e7fc700 1 -- 172.21.2.101:0/2718413415 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f4d280015b0 con 0x7f4d400d1870 2021-02-10T11:31:54.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.250 7f4d3e7fc700 1 -- 172.21.2.101:0/2718413415 >> v1:172.21.2.101:6789/0 conn(0x7f4d400d1870 legacy=0x7f4d401b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:54.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.250 7f4d3e7fc700 1 --2- 172.21.2.101:0/2718413415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d2c003a60 0x7f4d2c005f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:54.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.250 7f4d3e7fc700 1 --2- 172.21.2.101:0/2718413415 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d2c04a760 0x7f4d2c04cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:54.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.250 7f4d3e7fc700 1 --2- 172.21.2.101:0/2718413415 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d2c04d500 0x7f4d2c04fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:54.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.250 7f4d3e7fc700 1 -- 172.21.2.101:0/2718413415 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c0502d0 con 0x7f4d2c04d500 2021-02-10T11:31:54.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.250 7f4d3e7fc700 1 -- 172.21.2.101:0/2718413415 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c0504f0 con 0x7f4d2c04a760 2021-02-10T11:31:54.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.250 7f4d3e7fc700 1 -- 172.21.2.101:0/2718413415 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c050710 con 0x7f4d2c003a60 2021-02-10T11:31:54.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.250 7f4d3ffff700 1 --2- 172.21.2.101:0/2718413415 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d2c04d500 0x7f4d2c04fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:54.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.250 7f4d3f7fe700 1 --2- 172.21.2.101:0/2718413415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d2c003a60 0x7f4d2c005f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:54.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.250 7f4d3effd700 1 --2- 172.21.2.101:0/2718413415 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d2c04a760 0x7f4d2c04cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:54.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.250 7f4d3ffff700 1 -- 172.21.2.101:0/2718413415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d2c003a60 msgr2=0x7f4d2c005f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:54.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.250 7f4d3ffff700 1 --2- 172.21.2.101:0/2718413415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d2c003a60 0x7f4d2c005f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:54.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.250 7f4d3ffff700 1 -- 172.21.2.101:0/2718413415 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d2c04a760 msgr2=0x7f4d2c04cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:54.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.250 7f4d3ffff700 1 --2- 172.21.2.101:0/2718413415 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d2c04a760 0x7f4d2c04cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:54.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.250 7f4d3ffff700 1 -- 172.21.2.101:0/2718413415 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2c0509b0 con 0x7f4d2c04d500 2021-02-10T11:31:54.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.250 7f4d3ffff700 1 --2- 172.21.2.101:0/2718413415 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d2c04d500 0x7f4d2c04fa20 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 rx=0x7f4d38006a40 tx=0x7f4d3800c0c0).ready entity=mon.0 client_cookie=28918db7b5988a74 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:54.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.250 7f4d3e7fc700 1 -- 172.21.2.101:0/2718413415 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d3800c4b0 con 0x7f4d2c04d500 2021-02-10T11:31:54.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.251 7f4d3e7fc700 1 -- 172.21.2.101:0/2718413415 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3800c770 con 0x7f4d2c04d500 2021-02-10T11:31:54.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.251 7f4d3e7fc700 1 -- 172.21.2.101:0/2718413415 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d3800cc50 con 0x7f4d2c04d500 2021-02-10T11:31:54.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.251 7f4d45c01700 1 -- 172.21.2.101:0/2718413415 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d2c04d500 msgr2=0x7f4d2c04fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:54.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.251 7f4d45c01700 1 --2- 172.21.2.101:0/2718413415 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d2c04d500 0x7f4d2c04fa20 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 rx=0x7f4d38006a40 tx=0x7f4d3800c0c0).stop 2021-02-10T11:31:54.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.251 7f4d3ffff700 1 -- 172.21.2.101:0/2718413415 reap_dead start 2021-02-10T11:31:54.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.251 7f4d45c01700 1 -- 172.21.2.101:0/2718413415 shutdown_connections 2021-02-10T11:31:54.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.251 7f4d45c01700 1 -- 172.21.2.101:0/2718413415 >> 172.21.2.101:0/2718413415 conn(0x7f4d400ce420 msgr2=0x7f4d400d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:54.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.251 7f4d45c01700 1 -- 172.21.2.101:0/2718413415 shutdown_connections 2021-02-10T11:31:54.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.251 7f4d45c01700 1 -- 172.21.2.101:0/2718413415 wait complete. 2021-02-10T11:31:54.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.266 7f4d45c01700 1 Processor -- start 2021-02-10T11:31:54.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.266 7f4d45c01700 1 -- start start 2021-02-10T11:31:54.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.266 7f4d45c01700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d400b8070 0x7f4d400b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:54.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.266 7f4d45c01700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d400ba4a0 0x7f4d400ba8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:54.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.266 7f4d45c01700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d401a2fd0 0x7f4d401a33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:54.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.266 7f4d45c01700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d400baec0 con 0x7f4d400ba4a0 2021-02-10T11:31:54.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.266 7f4d45c01700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d401a39f0 con 0x7f4d400b8070 2021-02-10T11:31:54.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.266 7f4d45c01700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d401a3c10 con 0x7f4d401a2fd0 2021-02-10T11:31:54.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.267 7f4d3effd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d400ba4a0 0x7f4d400ba8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:54.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.267 7f4d3effd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d400ba4a0 0x7f4d400ba8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50436/0 (socket says 172.21.2.101:50436) 2021-02-10T11:31:54.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.267 7f4d3effd700 1 -- 172.21.2.101:0/1646569426 learned_addr learned my addr 172.21.2.101:0/1646569426 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:54.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.267 7f4d3f7fe700 1 --2- 172.21.2.101:0/1646569426 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d400b8070 0x7f4d400b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:54.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.267 7f4d3ffff700 1 --2- 172.21.2.101:0/1646569426 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d401a2fd0 0x7f4d401a33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:54.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.267 7f4d3effd700 1 -- 172.21.2.101:0/1646569426 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d401a2fd0 msgr2=0x7f4d401a33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:54.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.267 7f4d3effd700 1 --2- 172.21.2.101:0/1646569426 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d401a2fd0 0x7f4d401a33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:54.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.267 7f4d3effd700 1 -- 172.21.2.101:0/1646569426 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d400b8070 msgr2=0x7f4d400b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:54.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.267 7f4d3effd700 1 --2- 172.21.2.101:0/1646569426 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d400b8070 0x7f4d400b77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:54.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.267 7f4d3effd700 1 -- 172.21.2.101:0/1646569426 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d400ad700 con 0x7f4d400ba4a0 2021-02-10T11:31:54.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.267 7f4d3effd700 1 --2- 172.21.2.101:0/1646569426 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d400ba4a0 0x7f4d400ba8a0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7f4d30006f60 tx=0x7f4d3000b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:54.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.267 7f4d37fff700 1 -- 172.21.2.101:0/1646569426 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d3000bd60 con 0x7f4d400ba4a0 2021-02-10T11:31:54.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.267 7f4d37fff700 1 -- 172.21.2.101:0/1646569426 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d30000c90 con 0x7f4d400ba4a0 2021-02-10T11:31:54.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.267 7f4d45c01700 1 -- 172.21.2.101:0/1646569426 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d400ad700 con 0x7f4d400ba4a0 2021-02-10T11:31:54.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.267 7f4d45c01700 1 -- 172.21.2.101:0/1646569426 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d400adc90 con 0x7f4d400ba4a0 2021-02-10T11:31:54.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.268 7f4d37fff700 1 -- 172.21.2.101:0/1646569426 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d30002220 con 0x7f4d400ba4a0 2021-02-10T11:31:54.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.268 7f4d37fff700 1 -- 172.21.2.101:0/1646569426 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4d3000a000 con 0x7f4d400ba4a0 2021-02-10T11:31:54.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.268 7f4d37fff700 1 --2- 172.21.2.101:0/1646569426 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4d240186d0 0x7f4d2401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:54.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.268 7f4d37fff700 1 -- 172.21.2.101:0/1646569426 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4d300327e0 con 0x7f4d400ba4a0 2021-02-10T11:31:54.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.268 7f4d3f7fe700 1 --2- 172.21.2.101:0/1646569426 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4d240186d0 0x7f4d2401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:54.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.269 7f4d3f7fe700 1 --2- 172.21.2.101:0/1646569426 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4d240186d0 0x7f4d2401abd0 crc :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:54.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.298 7f4d45c01700 1 -- 172.21.2.101:0/1646569426 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d400828e0 con 0x7f4d400ba4a0 2021-02-10T11:31:54.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.301 7f4d37fff700 1 -- 172.21.2.101:0/1646569426 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4d30000ee0 con 0x7f4d400ba4a0 2021-02-10T11:31:54.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:54.439 7f4d45c01700 1 -- 172.21.2.101:0/1646569426 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4d400828e0 con 0x7f4d400ba4a0 2021-02-10T11:31:55.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.439 7f4d37fff700 1 -- 172.21.2.101:0/1646569426 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v23) v1 ==== 136+0+0 (secure 0 0 0) 0x7f4d30000940 con 0x7f4d400ba4a0 2021-02-10T11:31:55.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.459 7f4d45c01700 1 -- 172.21.2.101:0/1646569426 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4d240186d0 msgr2=0x7f4d2401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:55.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.459 7f4d45c01700 1 --2- 172.21.2.101:0/1646569426 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4d240186d0 0x7f4d2401abd0 crc :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:55.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.459 7f4d45c01700 1 -- 172.21.2.101:0/1646569426 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d400ba4a0 msgr2=0x7f4d400ba8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:55.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.459 7f4d45c01700 1 --2- 172.21.2.101:0/1646569426 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d400ba4a0 0x7f4d400ba8a0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7f4d30006f60 tx=0x7f4d3000b970).stop 2021-02-10T11:31:55.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.459 7f4d45c01700 1 -- 172.21.2.101:0/1646569426 shutdown_connections 2021-02-10T11:31:55.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.459 7f4d45c01700 1 --2- 172.21.2.101:0/1646569426 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4d240186d0 0x7f4d2401abd0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:55.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.459 7f4d45c01700 1 --2- 172.21.2.101:0/1646569426 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d401a2fd0 0x7f4d401a33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:55.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.459 7f4d45c01700 1 --2- 172.21.2.101:0/1646569426 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d400ba4a0 0x7f4d400ba8a0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:55.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.459 7f4d45c01700 1 --2- 172.21.2.101:0/1646569426 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d400b8070 0x7f4d400b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:55.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.459 7f4d45c01700 1 -- 172.21.2.101:0/1646569426 >> 172.21.2.101:0/1646569426 conn(0x7f4d400b58d0 msgr2=0x7f4d400b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:55.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.460 7f4d45c01700 1 -- 172.21.2.101:0/1646569426 shutdown_connections 2021-02-10T11:31:55.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.460 7f4d45c01700 1 -- 172.21.2.101:0/1646569426 wait complete. 2021-02-10T11:31:55.479 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs volume ls 2021-02-10T11:31:55.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.877 7f73e629b700 1 Processor -- start 2021-02-10T11:31:55.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.877 7f73e629b700 1 -- start start 2021-02-10T11:31:55.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.878 7f73e629b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f73e00e5070 0x7f73e01b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:55.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.878 7f73e629b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73e00e5470 0x7f73e0080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:55.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.878 7f73e629b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e007ed80 con 0x7f73e01b3e40 2021-02-10T11:31:55.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.878 7f73e629b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e00d1790 con 0x7f73e00e5070 2021-02-10T11:31:55.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.878 7f73e629b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e00d19b0 con 0x7f73e00e5470 2021-02-10T11:31:55.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.878 7f73dffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f73e00e5070 0x7f73e01b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:55.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.878 7f73dffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f73e00e5070 0x7f73e01b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50444/0 (socket says 172.21.2.101:50444) 2021-02-10T11:31:55.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.878 7f73dffff700 1 -- 172.21.2.101:0/394668098 learned_addr learned my addr 172.21.2.101:0/394668098 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:55.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.878 7f73dffff700 1 -- 172.21.2.101:0/394668098 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73e00e5470 msgr2=0x7f73e0080310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:55.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.878 7f73dffff700 1 --2- 172.21.2.101:0/394668098 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73e00e5470 0x7f73e0080310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:55.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.878 7f73dffff700 1 -- 172.21.2.101:0/394668098 >> v1:172.21.2.108:6789/0 conn(0x7f73e01b3e40 legacy=0x7f73e007d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:55.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.878 7f73dffff700 1 -- 172.21.2.101:0/394668098 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e00d1bd0 con 0x7f73e00e5070 2021-02-10T11:31:55.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.879 7f73dffff700 1 --2- 172.21.2.101:0/394668098 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73e00e5070 0x7f73e01b3820 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7f73d0006700 tx=0x7f73d0009b80).ready entity=mon.0 client_cookie=87238b70c0a3362e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:55.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.879 7f73deffd700 1 -- 172.21.2.101:0/394668098 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73d000c0c0 con 0x7f73e00e5070 2021-02-10T11:31:55.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.879 7f73deffd700 1 -- 172.21.2.101:0/394668098 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d000c440 con 0x7f73e00e5070 2021-02-10T11:31:55.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.879 7f73deffd700 1 -- 172.21.2.101:0/394668098 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73d000c960 con 0x7f73e00e5070 2021-02-10T11:31:55.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.879 7f73e629b700 1 -- 172.21.2.101:0/394668098 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73e00e5070 msgr2=0x7f73e01b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:55.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.879 7f73e629b700 1 --2- 172.21.2.101:0/394668098 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73e00e5070 0x7f73e01b3820 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7f73d0006700 tx=0x7f73d0009b80).stop 2021-02-10T11:31:55.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.880 7f73e629b700 1 -- 172.21.2.101:0/394668098 shutdown_connections 2021-02-10T11:31:55.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.880 7f73e629b700 1 --2- 172.21.2.101:0/394668098 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73e00e5470 0x7f73e0080310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:55.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.880 7f73e629b700 1 --2- 172.21.2.101:0/394668098 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73e00e5070 0x7f73e01b3820 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:55.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.880 7f73e629b700 1 -- 172.21.2.101:0/394668098 >> 172.21.2.101:0/394668098 conn(0x7f73e00ce690 msgr2=0x7f73e00e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:55.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.880 7f73e629b700 1 -- 172.21.2.101:0/394668098 shutdown_connections 2021-02-10T11:31:55.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.880 7f73e629b700 1 -- 172.21.2.101:0/394668098 wait complete. 2021-02-10T11:31:55.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.894 7f73e629b700 1 Processor -- start 2021-02-10T11:31:55.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.894 7f73e629b700 1 -- start start 2021-02-10T11:31:55.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.895 7f73e629b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73e00b07a0 0x7f73e019ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:55.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.895 7f73e629b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73e019f2b0 0x7f73e00a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:55.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.895 7f73e629b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73e00a1490 0x7f73e009fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:55.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.895 7f73e629b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e00b0ba0 con 0x7f73e00b07a0 2021-02-10T11:31:55.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.895 7f73e629b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e00a1890 con 0x7f73e019f2b0 2021-02-10T11:31:55.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.895 7f73e629b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e00a1ab0 con 0x7f73e00a1490 2021-02-10T11:31:55.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.895 7f73dffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73e00b07a0 0x7f73e019ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:55.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.895 7f73dffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73e00b07a0 0x7f73e019ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50446/0 (socket says 172.21.2.101:50446) 2021-02-10T11:31:55.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.895 7f73dffff700 1 -- 172.21.2.101:0/1012044915 learned_addr learned my addr 172.21.2.101:0/1012044915 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:55.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.895 7f73dffff700 1 -- 172.21.2.101:0/1012044915 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73e00a1490 msgr2=0x7f73e009fcf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:31:55.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.895 7f73dffff700 1 --2- 172.21.2.101:0/1012044915 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73e00a1490 0x7f73e009fcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:55.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.895 7f73dffff700 1 -- 172.21.2.101:0/1012044915 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73e019f2b0 msgr2=0x7f73e00a0e70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:31:55.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.895 7f73dffff700 1 --2- 172.21.2.101:0/1012044915 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73e019f2b0 0x7f73e00a0e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:55.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.895 7f73dffff700 1 -- 172.21.2.101:0/1012044915 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e00a0310 con 0x7f73e00b07a0 2021-02-10T11:31:55.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.895 7f73dffff700 1 --2- 172.21.2.101:0/1012044915 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73e00b07a0 0x7f73e019ec90 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7f73d000b5e0 tx=0x7f73d000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:55.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.895 7f73dcff9700 1 -- 172.21.2.101:0/1012044915 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73d000bd70 con 0x7f73e00b07a0 2021-02-10T11:31:55.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.895 7f73dcff9700 1 -- 172.21.2.101:0/1012044915 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d0002030 con 0x7f73e00b07a0 2021-02-10T11:31:55.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.895 7f73e629b700 1 -- 172.21.2.101:0/1012044915 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e00a0310 con 0x7f73e00b07a0 2021-02-10T11:31:55.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.895 7f73dcff9700 1 -- 172.21.2.101:0/1012044915 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73d0002550 con 0x7f73e00b07a0 2021-02-10T11:31:55.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.895 7f73e629b700 1 -- 172.21.2.101:0/1012044915 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e00a0310 con 0x7f73e00b07a0 2021-02-10T11:31:55.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.896 7f73dcff9700 1 -- 172.21.2.101:0/1012044915 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f73d0000c90 con 0x7f73e00b07a0 2021-02-10T11:31:55.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.896 7f73dcff9700 1 --2- 172.21.2.101:0/1012044915 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73c00186c0 0x7f73c001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:55.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.896 7f73dcff9700 1 -- 172.21.2.101:0/1012044915 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f73d0032860 con 0x7f73e00b07a0 2021-02-10T11:31:55.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.896 7f73df7fe700 1 --2- 172.21.2.101:0/1012044915 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73c00186c0 0x7f73c001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:55.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.896 7f73df7fe700 1 --2- 172.21.2.101:0/1012044915 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73c00186c0 0x7f73c001abc0 crc :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:55.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.928 7f73e629b700 1 -- 172.21.2.101:0/1012044915 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e00a0b50 con 0x7f73e00b07a0 2021-02-10T11:31:55.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:55.930 7f73dcff9700 1 -- 172.21.2.101:0/1012044915 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f73d001f5a0 con 0x7f73e00b07a0 2021-02-10T11:31:56.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.065 7f73e629b700 1 -- 172.21.2.101:0/1012044915 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs volume ls", "target": ["mgr", ""]}) v1 -- 0x7f73e00a0b50 con 0x7f73c00186c0 2021-02-10T11:31:56.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.065 7f73dcff9700 1 -- 172.21.2.101:0/1012044915 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7f73d4006970 con 0x7f73c00186c0 2021-02-10T11:31:56.076 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:31:56.077 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:31:56.077 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "cephfs" 2021-02-10T11:31:56.077 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:31:56.077 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:31:56.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.067 7f73e629b700 1 -- 172.21.2.101:0/1012044915 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73c00186c0 msgr2=0x7f73c001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:56.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.067 7f73e629b700 1 --2- 172.21.2.101:0/1012044915 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73c00186c0 0x7f73c001abc0 crc :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:56.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.067 7f73e629b700 1 -- 172.21.2.101:0/1012044915 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73e00b07a0 msgr2=0x7f73e019ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:56.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.067 7f73e629b700 1 --2- 172.21.2.101:0/1012044915 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73e00b07a0 0x7f73e019ec90 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7f73d000b5e0 tx=0x7f73d000b980).stop 2021-02-10T11:31:56.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.067 7f73e629b700 1 -- 172.21.2.101:0/1012044915 shutdown_connections 2021-02-10T11:31:56.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.067 7f73e629b700 1 --2- 172.21.2.101:0/1012044915 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73c00186c0 0x7f73c001abc0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:56.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.067 7f73e629b700 1 --2- 172.21.2.101:0/1012044915 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73e00a1490 0x7f73e009fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:56.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.067 7f73e629b700 1 --2- 172.21.2.101:0/1012044915 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73e019f2b0 0x7f73e00a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:56.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.067 7f73e629b700 1 --2- 172.21.2.101:0/1012044915 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73e00b07a0 0x7f73e019ec90 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:56.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.067 7f73e629b700 1 -- 172.21.2.101:0/1012044915 >> 172.21.2.101:0/1012044915 conn(0x7f73e00bde90 msgr2=0x7f73e00bd600 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:56.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.067 7f73e629b700 1 -- 172.21.2.101:0/1012044915 shutdown_connections 2021-02-10T11:31:56.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.067 7f73e629b700 1 -- 172.21.2.101:0/1012044915 wait complete. 2021-02-10T11:31:56.086 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:31:56.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.490 7f7e536bb700 1 Processor -- start 2021-02-10T11:31:56.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.490 7f7e536bb700 1 -- start start 2021-02-10T11:31:56.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.490 7f7e536bb700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7e4c1b0470 0x7f7e4c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:56.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.490 7f7e536bb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e4c0ef610 0x7f7e4c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:56.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.490 7f7e536bb700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7e4c0f0030 0x7f7e4c0f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:56.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.490 7f7e536bb700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e4c0e5ea0 con 0x7f7e4c0f0030 2021-02-10T11:31:56.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.490 7f7e536bb700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e4c0f0b50 con 0x7f7e4c1b0470 2021-02-10T11:31:56.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.490 7f7e536bb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c0e52e0 con 0x7f7e4c0ef610 2021-02-10T11:31:56.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.490 7f7e51cd8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7e4c0f0030 0x7f7e4c0f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:56.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.490 7f7e51cd8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7e4c0f0030 0x7f7e4c0f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50454/0 (socket says 172.21.2.101:50454) 2021-02-10T11:31:56.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.490 7f7e51cd8700 1 -- 172.21.2.101:0/1995068654 learned_addr learned my addr 172.21.2.101:0/1995068654 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:56.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.491 7f7e51cd8700 1 -- 172.21.2.101:0/1995068654 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e4c0ef610 msgr2=0x7f7e4c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:56.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.491 7f7e51cd8700 1 --2- 172.21.2.101:0/1995068654 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e4c0ef610 0x7f7e4c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:56.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.491 7f7e51cd8700 1 -- 172.21.2.101:0/1995068654 >> v2:172.21.2.108:3300/0 conn(0x7f7e4c1b0470 msgr2=0x7f7e4c1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:56.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.491 7f7e51cd8700 1 --2- 172.21.2.101:0/1995068654 >> v2:172.21.2.108:3300/0 conn(0x7f7e4c1b0470 0x7f7e4c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:56.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.491 7f7e51cd8700 1 -- 172.21.2.101:0/1995068654 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e4c0f10d0 con 0x7f7e4c0f0030 2021-02-10T11:31:56.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.491 7f7e51cd8700 1 --2- 172.21.2.101:0/1995068654 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e4c0f0030 0x7f7e4c0f0430 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7f7e44006f00 tx=0x7f7e44009e90).ready entity=mon.0 client_cookie=1ed3ea195a0d1925 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:56.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.491 7f7e3ffff700 1 -- 172.21.2.101:0/1995068654 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7e4400b300 con 0x7f7e4c0f0030 2021-02-10T11:31:56.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.491 7f7e3ffff700 1 -- 172.21.2.101:0/1995068654 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4400b580 con 0x7f7e4c0f0030 2021-02-10T11:31:56.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.491 7f7e3ffff700 1 -- 172.21.2.101:0/1995068654 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7e4400b9d0 con 0x7f7e4c0f0030 2021-02-10T11:31:56.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.491 7f7e536bb700 1 -- 172.21.2.101:0/1995068654 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e4c0f0030 msgr2=0x7f7e4c0f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:56.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.491 7f7e536bb700 1 --2- 172.21.2.101:0/1995068654 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e4c0f0030 0x7f7e4c0f0430 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7f7e44006f00 tx=0x7f7e44009e90).stop 2021-02-10T11:31:56.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.492 7f7e536bb700 1 -- 172.21.2.101:0/1995068654 shutdown_connections 2021-02-10T11:31:56.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.492 7f7e536bb700 1 --2- 172.21.2.101:0/1995068654 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e4c0ef610 0x7f7e4c0efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:56.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.492 7f7e536bb700 1 --2- 172.21.2.101:0/1995068654 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e4c0f0030 0x7f7e4c0f0430 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:56.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.492 7f7e536bb700 1 --2- 172.21.2.101:0/1995068654 >> v2:172.21.2.108:3300/0 conn(0x7f7e4c1b0470 0x7f7e4c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:56.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.492 7f7e536bb700 1 -- 172.21.2.101:0/1995068654 >> 172.21.2.101:0/1995068654 conn(0x7f7e4c0ce3e0 msgr2=0x7f7e4c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:56.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.492 7f7e536bb700 1 -- 172.21.2.101:0/1995068654 shutdown_connections 2021-02-10T11:31:56.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.492 7f7e536bb700 1 -- 172.21.2.101:0/1995068654 wait complete. 2021-02-10T11:31:56.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.506 7f7e536bb700 1 Processor -- start 2021-02-10T11:31:56.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.506 7f7e536bb700 1 -- start start 2021-02-10T11:31:56.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.507 7f7e536bb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7e4c190920 0x7f7e4c08f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:56.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.507 7f7e536bb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e4c190d20 0x7f7e4c08fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:56.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.507 7f7e536bb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e4c193270 0x7f7e4c193670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:56.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.507 7f7e536bb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c090470 con 0x7f7e4c193270 2021-02-10T11:31:56.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.507 7f7e536bb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c193c90 con 0x7f7e4c190920 2021-02-10T11:31:56.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.507 7f7e536bb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c193e70 con 0x7f7e4c190d20 2021-02-10T11:31:56.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.507 7f7e51cd8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e4c193270 0x7f7e4c193670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:56.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.507 7f7e51cd8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e4c193270 0x7f7e4c193670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50462/0 (socket says 172.21.2.101:50462) 2021-02-10T11:31:56.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.507 7f7e51cd8700 1 -- 172.21.2.101:0/3330377878 learned_addr learned my addr 172.21.2.101:0/3330377878 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:56.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.507 7f7e50cd6700 1 --2- 172.21.2.101:0/3330377878 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e4c190d20 0x7f7e4c08fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:56.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.507 7f7e51cd8700 1 -- 172.21.2.101:0/3330377878 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e4c190d20 msgr2=0x7f7e4c08fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:56.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.507 7f7e51cd8700 1 --2- 172.21.2.101:0/3330377878 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e4c190d20 0x7f7e4c08fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:56.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.507 7f7e51cd8700 1 -- 172.21.2.101:0/3330377878 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7e4c190920 msgr2=0x7f7e4c08f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:56.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.507 7f7e51cd8700 1 --2- 172.21.2.101:0/3330377878 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7e4c190920 0x7f7e4c08f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:56.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.507 7f7e51cd8700 1 -- 172.21.2.101:0/3330377878 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e4c194090 con 0x7f7e4c193270 2021-02-10T11:31:56.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.507 7f7e51cd8700 1 --2- 172.21.2.101:0/3330377878 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e4c193270 0x7f7e4c193670 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7f7e4400b910 tx=0x7f7e4400bcb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:56.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.508 7f7e3dffb700 1 -- 172.21.2.101:0/3330377878 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7e4400bdf0 con 0x7f7e4c193270 2021-02-10T11:31:56.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.508 7f7e3dffb700 1 -- 172.21.2.101:0/3330377878 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e44005820 con 0x7f7e4c193270 2021-02-10T11:31:56.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.508 7f7e3dffb700 1 -- 172.21.2.101:0/3330377878 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7e44005cc0 con 0x7f7e4c193270 2021-02-10T11:31:56.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.508 7f7e536bb700 1 -- 172.21.2.101:0/3330377878 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e4c194090 con 0x7f7e4c193270 2021-02-10T11:31:56.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.508 7f7e536bb700 1 -- 172.21.2.101:0/3330377878 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e4c1920b0 con 0x7f7e4c193270 2021-02-10T11:31:56.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.508 7f7e3dffb700 1 -- 172.21.2.101:0/3330377878 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7e44005a00 con 0x7f7e4c193270 2021-02-10T11:31:56.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.508 7f7e3dffb700 1 --2- 172.21.2.101:0/3330377878 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7e2c0186c0 0x7f7e2c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:56.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.508 7f7e3dffb700 1 -- 172.21.2.101:0/3330377878 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7e440328b0 con 0x7f7e4c193270 2021-02-10T11:31:56.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.509 7f7e514d7700 1 --2- 172.21.2.101:0/3330377878 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7e2c0186c0 0x7f7e2c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:56.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.509 7f7e514d7700 1 --2- 172.21.2.101:0/3330377878 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7e2c0186c0 0x7f7e2c01abc0 crc :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:56.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.540 7f7e536bb700 1 -- 172.21.2.101:0/3330377878 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e4c105af0 con 0x7f7e4c193270 2021-02-10T11:31:56.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.543 7f7e3dffb700 1 -- 172.21.2.101:0/3330377878 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7e44000f90 con 0x7f7e4c193270 2021-02-10T11:31:56.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.693 7f7e536bb700 1 -- 172.21.2.101:0/3330377878 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7e4c0f0b50 con 0x7f7e4c193270 2021-02-10T11:31:56.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.693 7f7e3dffb700 1 -- 172.21.2.101:0/3330377878 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v4)=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7e44006ef0 con 0x7f7e4c193270 2021-02-10T11:31:56.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.695 7f7e536bb700 1 -- 172.21.2.101:0/3330377878 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7e2c0186c0 msgr2=0x7f7e2c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:56.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.695 7f7e536bb700 1 --2- 172.21.2.101:0/3330377878 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7e2c0186c0 0x7f7e2c01abc0 crc :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:56.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.695 7f7e536bb700 1 -- 172.21.2.101:0/3330377878 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e4c193270 msgr2=0x7f7e4c193670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:56.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.695 7f7e536bb700 1 --2- 172.21.2.101:0/3330377878 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e4c193270 0x7f7e4c193670 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7f7e4400b910 tx=0x7f7e4400bcb0).stop 2021-02-10T11:31:56.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.695 7f7e536bb700 1 -- 172.21.2.101:0/3330377878 shutdown_connections 2021-02-10T11:31:56.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.695 7f7e536bb700 1 --2- 172.21.2.101:0/3330377878 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7e2c0186c0 0x7f7e2c01abc0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:56.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.695 7f7e536bb700 1 --2- 172.21.2.101:0/3330377878 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7e4c190d20 0x7f7e4c08fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:56.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.695 7f7e536bb700 1 --2- 172.21.2.101:0/3330377878 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7e4c193270 0x7f7e4c193670 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:56.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.695 7f7e536bb700 1 --2- 172.21.2.101:0/3330377878 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7e4c190920 0x7f7e4c08f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:56.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.695 7f7e536bb700 1 -- 172.21.2.101:0/3330377878 >> 172.21.2.101:0/3330377878 conn(0x7f7e4c0bf4e0 msgr2=0x7f7e4c0be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:56.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.696 7f7e536bb700 1 -- 172.21.2.101:0/3330377878 shutdown_connections 2021-02-10T11:31:56.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:56.696 7f7e536bb700 1 -- 172.21.2.101:0/3330377878 wait complete. 2021-02-10T11:31:56.715 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth get-or-create client.guest1 mds 'allow *' osd 'allow rw' mon 'allow *' 2021-02-10T11:31:57.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.124 7f439b3f8700 1 Processor -- start 2021-02-10T11:31:57.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.124 7f439b3f8700 1 -- start start 2021-02-10T11:31:57.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.125 7f439b3f8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f43940e5070 0x7f43941b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:57.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.125 7f439b3f8700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f439407ed80 con 0x7f43941b3e40 2021-02-10T11:31:57.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.125 7f439b3f8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f439407e670 con 0x7f43940e5470 2021-02-10T11:31:57.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.125 7f439b3f8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f439407e890 con 0x7f43940e5070 2021-02-10T11:31:57.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.125 7f4399214700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f43940e5070 0x7f43941b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:57.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.125 7f4398a13700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f43941b3e40 0x7f439407d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50960/0 (socket says 172.21.2.101:50960) 2021-02-10T11:31:57.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.125 7f4398a13700 1 -- 172.21.2.101:0/436921685 learned_addr learned my addr 172.21.2.101:0/436921685 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:57.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.125 7f4399214700 1 -- 172.21.2.101:0/436921685 >> v1:172.21.2.108:6789/0 conn(0x7f43940e5470 legacy=0x7f439407de80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:31:57.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.125 7f4399214700 1 -- 172.21.2.101:0/436921685 >> v1:172.21.2.101:6789/0 conn(0x7f43941b3e40 legacy=0x7f439407d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:57.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.125 7f4399214700 1 -- 172.21.2.101:0/436921685 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439407eab0 con 0x7f43940e5070 2021-02-10T11:31:57.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.125 7f4399214700 1 --2- 172.21.2.101:0/436921685 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43940e5070 0x7f43941b3820 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 rx=0x7f4384006650 tx=0x7f4384009a80).ready entity=mon.0 client_cookie=a1c15fa235d83df0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:57.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.126 7f438bfff700 1 -- 172.21.2.101:0/436921685 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f43840009b0 con 0x7f43940e5070 2021-02-10T11:31:57.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.126 7f438bfff700 1 -- 172.21.2.101:0/436921685 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f438400c0c0 con 0x7f43940e5070 2021-02-10T11:31:57.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.126 7f438bfff700 1 -- 172.21.2.101:0/436921685 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f438400c520 con 0x7f43940e5070 2021-02-10T11:31:57.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.126 7f439b3f8700 1 -- 172.21.2.101:0/436921685 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43940e5070 msgr2=0x7f43941b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:57.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.126 7f439b3f8700 1 --2- 172.21.2.101:0/436921685 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43940e5070 0x7f43941b3820 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 rx=0x7f4384006650 tx=0x7f4384009a80).stop 2021-02-10T11:31:57.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.126 7f439b3f8700 1 -- 172.21.2.101:0/436921685 shutdown_connections 2021-02-10T11:31:57.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.126 7f439b3f8700 1 --2- 172.21.2.101:0/436921685 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43940e5070 0x7f43941b3820 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:57.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.126 7f439b3f8700 1 -- 172.21.2.101:0/436921685 >> 172.21.2.101:0/436921685 conn(0x7f43940ce690 msgr2=0x7f43940e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:57.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.127 7f439b3f8700 1 -- 172.21.2.101:0/436921685 shutdown_connections 2021-02-10T11:31:57.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.127 7f439b3f8700 1 -- 172.21.2.101:0/436921685 wait complete. 2021-02-10T11:31:57.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.141 7f439b3f8700 1 Processor -- start 2021-02-10T11:31:57.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.141 7f439b3f8700 1 -- start start 2021-02-10T11:31:57.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.141 7f439b3f8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f43940b6910 0x7f43940bec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:57.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.141 7f439b3f8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43941c1ec0 0x7f43940fea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:57.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.141 7f439b3f8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43941c22c0 0x7f43940ff030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:57.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.141 7f439b3f8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43940ba980 con 0x7f43941c1ec0 2021-02-10T11:31:57.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.141 7f439b3f8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43940b6d40 con 0x7f43940b6910 2021-02-10T11:31:57.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.141 7f439b3f8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43940ff650 con 0x7f43941c22c0 2021-02-10T11:31:57.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.141 7f4398a13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43941c1ec0 0x7f43940fea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:57.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.141 7f4398a13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43941c1ec0 0x7f43940fea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50470/0 (socket says 172.21.2.101:50470) 2021-02-10T11:31:57.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.141 7f4398a13700 1 -- 172.21.2.101:0/2270343717 learned_addr learned my addr 172.21.2.101:0/2270343717 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:57.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.142 7f4398a13700 1 -- 172.21.2.101:0/2270343717 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43941c22c0 msgr2=0x7f43940ff030 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:31:57.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.142 7f4398a13700 1 --2- 172.21.2.101:0/2270343717 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43941c22c0 0x7f43940ff030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:57.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.142 7f4398a13700 1 -- 172.21.2.101:0/2270343717 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f43940b6910 msgr2=0x7f43940bec50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:31:57.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.142 7f4398a13700 1 --2- 172.21.2.101:0/2270343717 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f43940b6910 0x7f43940bec50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:57.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.142 7f4398a13700 1 -- 172.21.2.101:0/2270343717 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43940ff830 con 0x7f43941c1ec0 2021-02-10T11:31:57.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.142 7f4398a13700 1 --2- 172.21.2.101:0/2270343717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43941c1ec0 0x7f43940fea10 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7f4390001c60 tx=0x7f439000a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:57.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.142 7f4389ffb700 1 -- 172.21.2.101:0/2270343717 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f439000acc0 con 0x7f43941c1ec0 2021-02-10T11:31:57.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.142 7f4389ffb700 1 -- 172.21.2.101:0/2270343717 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4390004140 con 0x7f43941c1ec0 2021-02-10T11:31:57.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.142 7f4389ffb700 1 -- 172.21.2.101:0/2270343717 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4390004660 con 0x7f43941c1ec0 2021-02-10T11:31:57.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.142 7f439b3f8700 1 -- 172.21.2.101:0/2270343717 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43940ff830 con 0x7f43941c1ec0 2021-02-10T11:31:57.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.142 7f439b3f8700 1 -- 172.21.2.101:0/2270343717 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43940ff830 con 0x7f43941c1ec0 2021-02-10T11:31:57.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.143 7f4389ffb700 1 -- 172.21.2.101:0/2270343717 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4390000d60 con 0x7f43941c1ec0 2021-02-10T11:31:57.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.143 7f4389ffb700 1 --2- 172.21.2.101:0/2270343717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f43740186c0 0x7f437401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:57.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.143 7f4389ffb700 1 -- 172.21.2.101:0/2270343717 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f43900327b0 con 0x7f43941c1ec0 2021-02-10T11:31:57.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.143 7f4399214700 1 --2- 172.21.2.101:0/2270343717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f43740186c0 0x7f437401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:57.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.143 7f4399214700 1 --2- 172.21.2.101:0/2270343717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f43740186c0 0x7f437401abc0 crc :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:57.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.175 7f439b3f8700 1 -- 172.21.2.101:0/2270343717 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43940ba980 con 0x7f43941c1ec0 2021-02-10T11:31:57.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.177 7f4389ffb700 1 -- 172.21.2.101:0/2270343717 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f43900009b0 con 0x7f43941c1ec0 2021-02-10T11:31:57.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.308 7f439b3f8700 1 -- 172.21.2.101:0/2270343717 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.guest1", "caps": ["mds", "allow *", "osd", "allow rw", "mon", "allow *"]} v 0) v1 -- 0x7f43940ba980 con 0x7f43941c1ec0 2021-02-10T11:31:57.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.311 7f4389ffb700 1 -- 172.21.2.101:0/2270343717 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.guest1", "caps": ["mds", "allow *", "osd", "allow rw", "mon", "allow *"]}]=0 v21) v1 ==== 158+0+64 (secure 0 0 0) 0x7f4390000e70 con 0x7f43941c1ec0 2021-02-10T11:31:57.322 INFO:teuthology.orchestra.run.gibba001.stdout:[client.guest1] 2021-02-10T11:31:57.323 INFO:teuthology.orchestra.run.gibba001.stdout: key = AQAtxCNgnDB+EhAAzR4CBA86Hd4diHnxNQbfxA== 2021-02-10T11:31:57.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.313 7f439b3f8700 1 -- 172.21.2.101:0/2270343717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f43740186c0 msgr2=0x7f437401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:57.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.313 7f439b3f8700 1 --2- 172.21.2.101:0/2270343717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f43740186c0 0x7f437401abc0 crc :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:57.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.313 7f439b3f8700 1 -- 172.21.2.101:0/2270343717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43941c1ec0 msgr2=0x7f43940fea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:57.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.313 7f439b3f8700 1 --2- 172.21.2.101:0/2270343717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43941c1ec0 0x7f43940fea10 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7f4390001c60 tx=0x7f439000a8d0).stop 2021-02-10T11:31:57.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.313 7f439b3f8700 1 -- 172.21.2.101:0/2270343717 shutdown_connections 2021-02-10T11:31:57.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.313 7f439b3f8700 1 --2- 172.21.2.101:0/2270343717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f43740186c0 0x7f437401abc0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:57.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.313 7f439b3f8700 1 --2- 172.21.2.101:0/2270343717 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43941c22c0 0x7f43940ff030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:57.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.313 7f439b3f8700 1 --2- 172.21.2.101:0/2270343717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43941c1ec0 0x7f43940fea10 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:57.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.313 7f439b3f8700 1 --2- 172.21.2.101:0/2270343717 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f43940b6910 0x7f43940bec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:57.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.313 7f439b3f8700 1 -- 172.21.2.101:0/2270343717 >> 172.21.2.101:0/2270343717 conn(0x7f4394089b50 msgr2=0x7f43940fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:57.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.313 7f439b3f8700 1 -- 172.21.2.101:0/2270343717 shutdown_connections 2021-02-10T11:31:57.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.313 7f439b3f8700 1 -- 172.21.2.101:0/2270343717 wait complete. 2021-02-10T11:31:57.333 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolumegroup create cephfs group_0000000000422627 2021-02-10T11:31:57.385 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:31:57.389 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:31:57.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.736 7ff1b47fc700 1 Processor -- start 2021-02-10T11:31:57.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.736 7ff1b47fc700 1 -- start start 2021-02-10T11:31:57.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.736 7ff1b47fc700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff1ac1b0470 0x7ff1ac1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:57.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.736 7ff1b47fc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ac0ef610 0x7ff1ac0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:57.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.736 7ff1b47fc700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ac0f0b50 con 0x7ff1ac0f0030 2021-02-10T11:31:57.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.736 7ff1b47fc700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ac0e52e0 con 0x7ff1ac1b0470 2021-02-10T11:31:57.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.736 7ff1b47fc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac0e54d0 con 0x7ff1ac0ef610 2021-02-10T11:31:57.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.737 7ff1b2e19700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7ff1ac0f0030 0x7ff1ac0f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:44382/0 (socket says 172.21.2.101:44382) 2021-02-10T11:31:57.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.737 7ff1b1e17700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ac0ef610 0x7ff1ac0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:57.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.737 7ff1b2e19700 1 -- 172.21.2.101:0/4064101297 learned_addr learned my addr 172.21.2.101:0/4064101297 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:57.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.737 7ff1b2618700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff1ac1b0470 0x7ff1ac1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:57.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.737 7ff1b2618700 1 -- 172.21.2.101:0/4064101297 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ac0ef610 msgr2=0x7ff1ac0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:57.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.737 7ff1b2618700 1 --2- 172.21.2.101:0/4064101297 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ac0ef610 0x7ff1ac0efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:57.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.737 7ff1b2618700 1 -- 172.21.2.101:0/4064101297 >> v1:172.21.2.108:6789/0 conn(0x7ff1ac0f0030 legacy=0x7ff1ac0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:57.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.737 7ff1b2618700 1 -- 172.21.2.101:0/4064101297 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ac0f0430 con 0x7ff1ac1b0470 2021-02-10T11:31:57.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.737 7ff1b1e17700 1 --2- 172.21.2.101:0/4064101297 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ac0ef610 0x7ff1ac0efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:31:57.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.738 7ff1b2618700 1 --2- 172.21.2.101:0/4064101297 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1ac1b0470 0x7ff1ac1b0870 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7ff1a8006650 tx=0x7ff1a8009a80).ready entity=mon.1 client_cookie=437961ad1e130ed1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:57.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.738 7ff1b1616700 1 -- 172.21.2.101:0/4064101297 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1a8006390 con 0x7ff1ac1b0470 2021-02-10T11:31:57.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.738 7ff1b1616700 1 -- 172.21.2.101:0/4064101297 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a80009b0 con 0x7ff1ac1b0470 2021-02-10T11:31:57.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.738 7ff1b1616700 1 -- 172.21.2.101:0/4064101297 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1a800c2b0 con 0x7ff1ac1b0470 2021-02-10T11:31:57.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.738 7ff1b47fc700 1 -- 172.21.2.101:0/4064101297 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1ac1b0470 msgr2=0x7ff1ac1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:57.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.738 7ff1b47fc700 1 --2- 172.21.2.101:0/4064101297 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1ac1b0470 0x7ff1ac1b0870 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7ff1a8006650 tx=0x7ff1a8009a80).stop 2021-02-10T11:31:57.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.739 7ff1b47fc700 1 -- 172.21.2.101:0/4064101297 shutdown_connections 2021-02-10T11:31:57.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.739 7ff1b47fc700 1 --2- 172.21.2.101:0/4064101297 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ac0ef610 0x7ff1ac0efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:57.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.739 7ff1b47fc700 1 --2- 172.21.2.101:0/4064101297 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1ac1b0470 0x7ff1ac1b0870 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:57.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.739 7ff1b47fc700 1 -- 172.21.2.101:0/4064101297 >> 172.21.2.101:0/4064101297 conn(0x7ff1ac0ce3e0 msgr2=0x7ff1ac0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:57.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.739 7ff1b47fc700 1 -- 172.21.2.101:0/4064101297 shutdown_connections 2021-02-10T11:31:57.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.739 7ff1b47fc700 1 -- 172.21.2.101:0/4064101297 wait complete. 2021-02-10T11:31:57.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.753 7ff1b47fc700 1 Processor -- start 2021-02-10T11:31:57.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.754 7ff1b47fc700 1 -- start start 2021-02-10T11:31:57.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.754 7ff1b47fc700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1ac0a0420 0x7ff1ac0a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:57.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.754 7ff1b47fc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ac189d00 0x7ff1ac18a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:57.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.754 7ff1b47fc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ac18a720 0x7ff1ac0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:57.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.754 7ff1b47fc700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac189780 con 0x7ff1ac189d00 2021-02-10T11:31:57.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.754 7ff1b47fc700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac18ab20 con 0x7ff1ac0a0420 2021-02-10T11:31:57.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.754 7ff1b47fc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac0a1140 con 0x7ff1ac18a720 2021-02-10T11:31:57.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.754 7ff1b1e17700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ac189d00 0x7ff1ac18a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:57.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.754 7ff1b1e17700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ac189d00 0x7ff1ac18a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50482/0 (socket says 172.21.2.101:50482) 2021-02-10T11:31:57.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.754 7ff1b1e17700 1 -- 172.21.2.101:0/1074535071 learned_addr learned my addr 172.21.2.101:0/1074535071 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:57.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.754 7ff1b2e19700 1 --2- 172.21.2.101:0/1074535071 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ac18a720 0x7ff1ac0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:57.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.754 7ff1b2618700 1 --2- 172.21.2.101:0/1074535071 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1ac0a0420 0x7ff1ac0a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:57.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.755 7ff1b1e17700 1 -- 172.21.2.101:0/1074535071 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ac18a720 msgr2=0x7ff1ac0b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:57.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.755 7ff1b1e17700 1 --2- 172.21.2.101:0/1074535071 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ac18a720 0x7ff1ac0b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:57.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.755 7ff1b1e17700 1 -- 172.21.2.101:0/1074535071 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1ac0a0420 msgr2=0x7ff1ac0a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:57.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.755 7ff1b1e17700 1 --2- 172.21.2.101:0/1074535071 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1ac0a0420 0x7ff1ac0a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:57.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.755 7ff1b1e17700 1 -- 172.21.2.101:0/1074535071 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ac0b8200 con 0x7ff1ac189d00 2021-02-10T11:31:57.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.755 7ff1b2618700 1 --2- 172.21.2.101:0/1074535071 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1ac0a0420 0x7ff1ac0a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:31:57.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.755 7ff1b1e17700 1 --2- 172.21.2.101:0/1074535071 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ac189d00 0x7ff1ac18a100 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 rx=0x7ff19c006f40 tx=0x7ff19c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:57.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.755 7ff1a2ffd700 1 -- 172.21.2.101:0/1074535071 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff19c0009b0 con 0x7ff1ac189d00 2021-02-10T11:31:57.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.755 7ff1a2ffd700 1 -- 172.21.2.101:0/1074535071 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19c00bd80 con 0x7ff1ac189d00 2021-02-10T11:31:57.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.755 7ff1a2ffd700 1 -- 172.21.2.101:0/1074535071 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff19c0022b0 con 0x7ff1ac189d00 2021-02-10T11:31:57.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.755 7ff1b47fc700 1 -- 172.21.2.101:0/1074535071 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ac0b8200 con 0x7ff1ac189d00 2021-02-10T11:31:57.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.755 7ff1b47fc700 1 -- 172.21.2.101:0/1074535071 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ac0b8790 con 0x7ff1ac189d00 2021-02-10T11:31:57.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.756 7ff1a2ffd700 1 -- 172.21.2.101:0/1074535071 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff19c006b00 con 0x7ff1ac189d00 2021-02-10T11:31:57.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.756 7ff1a2ffd700 1 --2- 172.21.2.101:0/1074535071 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1940186c0 0x7ff19401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:57.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.756 7ff1a2ffd700 1 -- 172.21.2.101:0/1074535071 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff19c0325b0 con 0x7ff1ac189d00 2021-02-10T11:31:57.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.756 7ff1b2618700 1 --2- 172.21.2.101:0/1074535071 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1940186c0 0x7ff19401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:57.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.756 7ff1b2618700 1 --2- 172.21.2.101:0/1074535071 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1940186c0 0x7ff19401abc0 crc :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:57.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.786 7ff1b47fc700 1 -- 172.21.2.101:0/1074535071 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ac0a1360 con 0x7ff1ac189d00 2021-02-10T11:31:57.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.789 7ff1a2ffd700 1 -- 172.21.2.101:0/1074535071 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff19c0009b0 con 0x7ff1ac189d00 2021-02-10T11:31:57.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.915 7ff1b47fc700 1 -- 172.21.2.101:0/1074535071 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "target": ["mgr", ""], "group_name": "group_0000000000422627"}) v1 -- 0x7ff1ac0b9020 con 0x7ff1940186c0 2021-02-10T11:31:57.931 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:31:57.918 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:31:57.932 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:31:57.918 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:31:57.932 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:31:57.918 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:31:57.932 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:31:57.918 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:31:57.932 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:31:57.918 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:31:57.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.933 7ff1a2ffd700 1 -- 172.21.2.101:0/1074535071 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7ff1a8006540 con 0x7ff1940186c0 2021-02-10T11:31:57.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.948 7ff1b47fc700 1 -- 172.21.2.101:0/1074535071 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1940186c0 msgr2=0x7ff19401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:57.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.948 7ff1b47fc700 1 --2- 172.21.2.101:0/1074535071 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1940186c0 0x7ff19401abc0 crc :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:57.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.948 7ff1b47fc700 1 -- 172.21.2.101:0/1074535071 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ac189d00 msgr2=0x7ff1ac18a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:57.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.948 7ff1b47fc700 1 --2- 172.21.2.101:0/1074535071 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ac189d00 0x7ff1ac18a100 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 rx=0x7ff19c006f40 tx=0x7ff19c00b990).stop 2021-02-10T11:31:57.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.948 7ff1b47fc700 1 -- 172.21.2.101:0/1074535071 shutdown_connections 2021-02-10T11:31:57.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.948 7ff1b47fc700 1 --2- 172.21.2.101:0/1074535071 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1940186c0 0x7ff19401abc0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:57.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.948 7ff1b47fc700 1 --2- 172.21.2.101:0/1074535071 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ac18a720 0x7ff1ac0b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:57.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.948 7ff1b47fc700 1 --2- 172.21.2.101:0/1074535071 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ac189d00 0x7ff1ac18a100 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:57.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.948 7ff1b47fc700 1 --2- 172.21.2.101:0/1074535071 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1ac0a0420 0x7ff1ac0a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:57.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.948 7ff1b47fc700 1 -- 172.21.2.101:0/1074535071 >> 172.21.2.101:0/1074535071 conn(0x7ff1ac094e10 msgr2=0x7ff1ac0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:57.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.948 7ff1b47fc700 1 -- 172.21.2.101:0/1074535071 shutdown_connections 2021-02-10T11:31:57.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:57.948 7ff1b47fc700 1 -- 172.21.2.101:0/1074535071 wait complete. 2021-02-10T11:31:57.967 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000841945 --group_name group_0000000000422627 2021-02-10T11:31:58.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.409 7f0b178c2700 1 Processor -- start 2021-02-10T11:31:58.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.409 7f0b178c2700 1 -- start start 2021-02-10T11:31:58.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.410 7f0b178c2700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0b100d1860 0x7f0b101b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:58.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.410 7f0b178c2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0b101b0a00 0x7f0b100cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:58.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.410 7f0b178c2700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b100f0b00 con 0x7f0b100d1c60 2021-02-10T11:31:58.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.410 7f0b178c2700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b100e5ec0 con 0x7f0b101b0a00 2021-02-10T11:31:58.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.410 7f0b178c2700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b100f0640 con 0x7f0b100d1860 2021-02-10T11:31:58.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.410 7f0b15edf700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f0b100d1c60 0x7f0b100cf940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:50998/0 (socket says 172.21.2.101:50998) 2021-02-10T11:31:58.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.410 7f0b15edf700 1 -- 172.21.2.101:0/1689635841 learned_addr learned my addr 172.21.2.101:0/1689635841 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:58.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.410 7f0b14edd700 1 --2- 172.21.2.101:0/1689635841 >> v2:172.21.2.101:3300/0 conn(0x7f0b101b0a00 0x7f0b100cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:58.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.410 7f0b03fff700 1 -- 172.21.2.101:0/1689635841 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1776190750 0 0) 0x7f0b08001500 con 0x7f0b100d1c60 2021-02-10T11:31:58.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.410 7f0b03fff700 1 -- 172.21.2.101:0/1689635841 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0afc002fe0 con 0x7f0b100d1c60 2021-02-10T11:31:58.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.410 7f0b14edd700 1 -- 172.21.2.101:0/1689635841 >> v2:172.21.2.108:3300/0 conn(0x7f0b100d1860 msgr2=0x7f0b101b03e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:31:58.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.410 7f0b14edd700 1 --2- 172.21.2.101:0/1689635841 >> v2:172.21.2.108:3300/0 conn(0x7f0b100d1860 0x7f0b101b03e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:58.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.410 7f0b14edd700 1 -- 172.21.2.101:0/1689635841 >> v1:172.21.2.101:6789/0 conn(0x7f0b100d1c60 legacy=0x7f0b100cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:58.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.411 7f0b14edd700 1 -- 172.21.2.101:0/1689635841 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b100f0830 con 0x7f0b101b0a00 2021-02-10T11:31:58.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.411 7f0b14edd700 1 --2- 172.21.2.101:0/1689635841 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b101b0a00 0x7f0b100cf320 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7f0b04006650 tx=0x7f0b04009a80).ready entity=mon.0 client_cookie=628aea571213ef79 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:58.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.411 7f0b03fff700 1 -- 172.21.2.101:0/1689635841 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0b040009b0 con 0x7f0b101b0a00 2021-02-10T11:31:58.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.411 7f0b03fff700 1 -- 172.21.2.101:0/1689635841 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0400c0c0 con 0x7f0b101b0a00 2021-02-10T11:31:58.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.411 7f0b03fff700 1 -- 172.21.2.101:0/1689635841 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0b0400c5e0 con 0x7f0b101b0a00 2021-02-10T11:31:58.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.411 7f0b178c2700 1 -- 172.21.2.101:0/1689635841 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b101b0a00 msgr2=0x7f0b100cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:58.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.411 7f0b178c2700 1 --2- 172.21.2.101:0/1689635841 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b101b0a00 0x7f0b100cf320 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7f0b04006650 tx=0x7f0b04009a80).stop 2021-02-10T11:31:58.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.412 7f0b178c2700 1 -- 172.21.2.101:0/1689635841 shutdown_connections 2021-02-10T11:31:58.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.412 7f0b178c2700 1 --2- 172.21.2.101:0/1689635841 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b101b0a00 0x7f0b100cf320 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:58.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.412 7f0b178c2700 1 --2- 172.21.2.101:0/1689635841 >> v2:172.21.2.108:3300/0 conn(0x7f0b100d1860 0x7f0b101b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:58.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.412 7f0b178c2700 1 -- 172.21.2.101:0/1689635841 >> 172.21.2.101:0/1689635841 conn(0x7f0b100ce440 msgr2=0x7f0b100d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:58.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.412 7f0b178c2700 1 -- 172.21.2.101:0/1689635841 shutdown_connections 2021-02-10T11:31:58.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.412 7f0b178c2700 1 -- 172.21.2.101:0/1689635841 wait complete. 2021-02-10T11:31:58.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.426 7f0b178c2700 1 Processor -- start 2021-02-10T11:31:58.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.427 7f0b178c2700 1 -- start start 2021-02-10T11:31:58.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.427 7f0b178c2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b100b5a50 0x7f0b100a6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:58.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.427 7f0b178c2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b100b5040 0x7f0b100a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:58.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.427 7f0b178c2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b100b5e50 0x7f0b100902b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:58.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.427 7f0b178c2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b100917f0 con 0x7f0b100b5040 2021-02-10T11:31:58.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.427 7f0b178c2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1019d290 con 0x7f0b100b5e50 2021-02-10T11:31:58.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.427 7f0b178c2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b10091300 con 0x7f0b100b5a50 2021-02-10T11:31:58.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.427 7f0b14edd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b100b5040 0x7f0b100a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:58.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.427 7f0b14edd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b100b5040 0x7f0b100a6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50508/0 (socket says 172.21.2.101:50508) 2021-02-10T11:31:58.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.427 7f0b14edd700 1 -- 172.21.2.101:0/1544947821 learned_addr learned my addr 172.21.2.101:0/1544947821 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:58.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.427 7f0b14edd700 1 -- 172.21.2.101:0/1544947821 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b100b5a50 msgr2=0x7f0b100a6350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:31:58.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.428 7f0b14edd700 1 --2- 172.21.2.101:0/1544947821 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b100b5a50 0x7f0b100a6350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:58.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.428 7f0b14edd700 1 -- 172.21.2.101:0/1544947821 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b100b5e50 msgr2=0x7f0b100902b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:31:58.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.428 7f0b14edd700 1 --2- 172.21.2.101:0/1544947821 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b100b5e50 0x7f0b100902b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:58.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.428 7f0b14edd700 1 -- 172.21.2.101:0/1544947821 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b100908d0 con 0x7f0b100b5040 2021-02-10T11:31:58.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.428 7f0b14edd700 1 --2- 172.21.2.101:0/1544947821 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b100b5040 0x7f0b100a6970 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7f0b04009630 tx=0x7f0b0400b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:58.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.428 7f0b01ffb700 1 -- 172.21.2.101:0/1544947821 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0b040093a0 con 0x7f0b100b5040 2021-02-10T11:31:58.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.428 7f0b01ffb700 1 -- 172.21.2.101:0/1544947821 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0400bcf0 con 0x7f0b100b5040 2021-02-10T11:31:58.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.428 7f0b01ffb700 1 -- 172.21.2.101:0/1544947821 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0b04005560 con 0x7f0b100b5040 2021-02-10T11:31:58.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.428 7f0b178c2700 1 -- 172.21.2.101:0/1544947821 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b100908d0 con 0x7f0b100b5040 2021-02-10T11:31:58.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.428 7f0b178c2700 1 -- 172.21.2.101:0/1544947821 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b10090e60 con 0x7f0b100b5040 2021-02-10T11:31:58.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.429 7f0b01ffb700 1 -- 172.21.2.101:0/1544947821 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0b0400a000 con 0x7f0b100b5040 2021-02-10T11:31:58.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.429 7f0b01ffb700 1 --2- 172.21.2.101:0/1544947821 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0af00186c0 0x7f0af001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:58.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.429 7f0b01ffb700 1 -- 172.21.2.101:0/1544947821 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0b04032530 con 0x7f0b100b5040 2021-02-10T11:31:58.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.429 7f0b156de700 1 --2- 172.21.2.101:0/1544947821 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0af00186c0 0x7f0af001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:58.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.429 7f0b156de700 1 --2- 172.21.2.101:0/1544947821 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0af00186c0 0x7f0af001abc0 crc :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:58.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.460 7f0b178c2700 1 -- 172.21.2.101:0/1544947821 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b100ebc30 con 0x7f0b100b5040 2021-02-10T11:31:58.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.463 7f0b01ffb700 1 -- 172.21.2.101:0/1544947821 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0b04009340 con 0x7f0b100b5040 2021-02-10T11:31:58.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.589 7f0b178c2700 1 -- 172.21.2.101:0/1544947821 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000841945", "target": ["mgr", ""], "group_name": "group_0000000000422627"}) v1 -- 0x7f0b10090e60 con 0x7f0af00186c0 2021-02-10T11:31:58.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.615 7f0b01ffb700 1 -- 172.21.2.101:0/1544947821 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f0b0c0009b0 con 0x7f0af00186c0 2021-02-10T11:31:58.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.621 7f0b178c2700 1 -- 172.21.2.101:0/1544947821 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0af00186c0 msgr2=0x7f0af001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:58.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.621 7f0b178c2700 1 --2- 172.21.2.101:0/1544947821 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0af00186c0 0x7f0af001abc0 crc :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:58.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.621 7f0b178c2700 1 -- 172.21.2.101:0/1544947821 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b100b5040 msgr2=0x7f0b100a6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:58.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.621 7f0b178c2700 1 --2- 172.21.2.101:0/1544947821 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b100b5040 0x7f0b100a6970 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7f0b04009630 tx=0x7f0b0400b900).stop 2021-02-10T11:31:58.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.622 7f0b178c2700 1 -- 172.21.2.101:0/1544947821 shutdown_connections 2021-02-10T11:31:58.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.622 7f0b178c2700 1 --2- 172.21.2.101:0/1544947821 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b100b5e50 0x7f0b100902b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:58.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.622 7f0b178c2700 1 --2- 172.21.2.101:0/1544947821 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b100b5040 0x7f0b100a6970 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:58.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.622 7f0b178c2700 1 --2- 172.21.2.101:0/1544947821 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0af00186c0 0x7f0af001abc0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:58.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.622 7f0b178c2700 1 --2- 172.21.2.101:0/1544947821 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b100b5a50 0x7f0b100a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:58.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.622 7f0b178c2700 1 -- 172.21.2.101:0/1544947821 >> 172.21.2.101:0/1544947821 conn(0x7f0b100bc0e0 msgr2=0x7f0b100ba710 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:58.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.622 7f0b178c2700 1 -- 172.21.2.101:0/1544947821 shutdown_connections 2021-02-10T11:31:58.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:58.622 7f0b178c2700 1 -- 172.21.2.101:0/1544947821 wait complete. 2021-02-10T11:31:58.641 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume authorize cephfs subvolume_0000000000841945 guest1 --group_name group_0000000000422627 --tenant_id tenant1 --allow-existing-id 2021-02-10T11:31:59.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.053 7fe69599b700 1 Processor -- start 2021-02-10T11:31:59.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.053 7fe69599b700 1 -- start start 2021-02-10T11:31:59.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.054 7fe69599b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe6900eb220 0x7fe6900eb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:59.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.054 7fe69599b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe69010a550 con 0x7fe6900dce80 2021-02-10T11:31:59.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.054 7fe69599b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe690088180 con 0x7fe69010aad0 2021-02-10T11:31:59.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.054 7fe69599b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6900d1ce0 con 0x7fe6900eb220 2021-02-10T11:31:59.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.054 7fe68ed73700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe6900eb220 0x7fe6900eb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:59.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.054 7fe68f574700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe6900dce80 0x7fe6900d14f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51010/0 (socket says 172.21.2.101:51010) 2021-02-10T11:31:59.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.054 7fe68ed73700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe6900eb220 0x7fe6900eb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50516/0 (socket says 172.21.2.101:50516) 2021-02-10T11:31:59.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.054 7fe68f574700 1 -- 172.21.2.101:0/2814909287 learned_addr learned my addr 172.21.2.101:0/2814909287 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:59.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.054 7fe68ed73700 1 -- 172.21.2.101:0/2814909287 >> v1:172.21.2.108:6789/0 conn(0x7fe69010aad0 legacy=0x7fe69010aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:31:59.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.054 7fe68ed73700 1 -- 172.21.2.101:0/2814909287 >> v1:172.21.2.101:6789/0 conn(0x7fe6900dce80 legacy=0x7fe6900d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:59.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.054 7fe68ed73700 1 -- 172.21.2.101:0/2814909287 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6900d1f60 con 0x7fe6900eb220 2021-02-10T11:31:59.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.054 7fe68ed73700 1 --2- 172.21.2.101:0/2814909287 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6900eb220 0x7fe6900eb620 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 rx=0x7fe680006650 tx=0x7fe680009a80).ready entity=mon.0 client_cookie=50a29590018944ef server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:59.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.054 7fe68e572700 1 -- 172.21.2.101:0/2814909287 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe68000c0c0 con 0x7fe6900eb220 2021-02-10T11:31:59.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.054 7fe68e572700 1 -- 172.21.2.101:0/2814909287 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68000c340 con 0x7fe6900eb220 2021-02-10T11:31:59.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.054 7fe68e572700 1 -- 172.21.2.101:0/2814909287 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe68000c820 con 0x7fe6900eb220 2021-02-10T11:31:59.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.055 7fe69599b700 1 -- 172.21.2.101:0/2814909287 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6900eb220 msgr2=0x7fe6900eb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:59.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.055 7fe69599b700 1 --2- 172.21.2.101:0/2814909287 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6900eb220 0x7fe6900eb620 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 rx=0x7fe680006650 tx=0x7fe680009a80).stop 2021-02-10T11:31:59.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.055 7fe69599b700 1 -- 172.21.2.101:0/2814909287 shutdown_connections 2021-02-10T11:31:59.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.055 7fe69599b700 1 --2- 172.21.2.101:0/2814909287 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6900eb220 0x7fe6900eb620 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:59.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.055 7fe69599b700 1 -- 172.21.2.101:0/2814909287 >> 172.21.2.101:0/2814909287 conn(0x7fe6900ea630 msgr2=0x7fe6900d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:59.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.055 7fe69599b700 1 -- 172.21.2.101:0/2814909287 shutdown_connections 2021-02-10T11:31:59.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.055 7fe69599b700 1 -- 172.21.2.101:0/2814909287 wait complete. 2021-02-10T11:31:59.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.070 7fe69599b700 1 Processor -- start 2021-02-10T11:31:59.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.070 7fe69599b700 1 -- start start 2021-02-10T11:31:59.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.070 7fe69599b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6901c6a70 0x7fe690093860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:59.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.070 7fe69599b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe690093e80 0x7fe6900a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:59.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.070 7fe69599b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6900a3ee0 0x7fe6900a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:59.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.070 7fe69599b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6900a6080 con 0x7fe690093e80 2021-02-10T11:31:59.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.070 7fe69599b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6901079e0 con 0x7fe6900a3ee0 2021-02-10T11:31:59.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.070 7fe69599b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6900a4580 con 0x7fe6901c6a70 2021-02-10T11:31:59.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.070 7fe68ed73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe690093e80 0x7fe6900a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:59.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.070 7fe68ed73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe690093e80 0x7fe6900a38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50526/0 (socket says 172.21.2.101:50526) 2021-02-10T11:31:59.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.070 7fe68ed73700 1 -- 172.21.2.101:0/423855371 learned_addr learned my addr 172.21.2.101:0/423855371 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:59.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.071 7fe68ed73700 1 -- 172.21.2.101:0/423855371 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6901c6a70 msgr2=0x7fe690093860 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:31:59.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.071 7fe68ed73700 1 --2- 172.21.2.101:0/423855371 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6901c6a70 0x7fe690093860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:59.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.071 7fe68ed73700 1 -- 172.21.2.101:0/423855371 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6900a3ee0 msgr2=0x7fe6900a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:59.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.071 7fe68ed73700 1 --2- 172.21.2.101:0/423855371 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6900a3ee0 0x7fe6900a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:59.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.071 7fe68ed73700 1 -- 172.21.2.101:0/423855371 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6900a42e0 con 0x7fe690093e80 2021-02-10T11:31:59.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.071 7fe68ed73700 1 --2- 172.21.2.101:0/423855371 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe690093e80 0x7fe6900a38c0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7fe680009630 tx=0x7fe68000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:59.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.071 7fe687fff700 1 -- 172.21.2.101:0/423855371 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe6800093a0 con 0x7fe690093e80 2021-02-10T11:31:59.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.071 7fe687fff700 1 -- 172.21.2.101:0/423855371 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68000bd70 con 0x7fe690093e80 2021-02-10T11:31:59.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.071 7fe687fff700 1 -- 172.21.2.101:0/423855371 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe680005560 con 0x7fe690093e80 2021-02-10T11:31:59.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.071 7fe69599b700 1 -- 172.21.2.101:0/423855371 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6900a42e0 con 0x7fe690093e80 2021-02-10T11:31:59.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.071 7fe69599b700 1 -- 172.21.2.101:0/423855371 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6900a5410 con 0x7fe690093e80 2021-02-10T11:31:59.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.072 7fe687fff700 1 -- 172.21.2.101:0/423855371 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe68000a000 con 0x7fe690093e80 2021-02-10T11:31:59.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.072 7fe687fff700 1 --2- 172.21.2.101:0/423855371 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6740186c0 0x7fe67401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:59.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.072 7fe687fff700 1 -- 172.21.2.101:0/423855371 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe680032560 con 0x7fe690093e80 2021-02-10T11:31:59.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.072 7fe68f574700 1 --2- 172.21.2.101:0/423855371 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6740186c0 0x7fe67401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:59.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.072 7fe68f574700 1 --2- 172.21.2.101:0/423855371 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6740186c0 0x7fe67401abc0 crc :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:59.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.104 7fe69599b700 1 -- 172.21.2.101:0/423855371 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6901079e0 con 0x7fe690093e80 2021-02-10T11:31:59.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.106 7fe687fff700 1 -- 172.21.2.101:0/423855371 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe680009340 con 0x7fe690093e80 2021-02-10T11:31:59.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.232 7fe69599b700 1 -- 172.21.2.101:0/423855371 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"allow_existing_id": true, "vol_name": "cephfs", "sub_name": "subvolume_0000000000841945", "target": ["mgr", ""], "tenant_id": "tenant1", "group_name": "group_0000000000422627", "prefix": "fs subvolume authorize", "auth_id": "guest1"}) v1 -- 0x7fe6900a5410 con 0x7fe6740186c0 2021-02-10T11:31:59.249 INFO:tasks.ceph.mgr.y.gibba001.stderr:Creating meta for ID guest1 with tenant tenant1 2021-02-10T11:31:59.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.267 7fe687fff700 1 -- 172.21.2.101:0/423855371 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7fe678000960 con 0x7fe6740186c0 2021-02-10T11:31:59.307 INFO:teuthology.orchestra.run.gibba001.stdout:AQAtxCNgnDB+EhAAzR4CBA86Hd4diHnxNQbfxA== 2021-02-10T11:31:59.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.297 7fe69599b700 1 -- 172.21.2.101:0/423855371 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6740186c0 msgr2=0x7fe67401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:59.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.297 7fe69599b700 1 --2- 172.21.2.101:0/423855371 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6740186c0 0x7fe67401abc0 crc :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:59.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.297 7fe69599b700 1 -- 172.21.2.101:0/423855371 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe690093e80 msgr2=0x7fe6900a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:59.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.297 7fe69599b700 1 --2- 172.21.2.101:0/423855371 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe690093e80 0x7fe6900a38c0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7fe680009630 tx=0x7fe68000b980).stop 2021-02-10T11:31:59.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.297 7fe69599b700 1 -- 172.21.2.101:0/423855371 shutdown_connections 2021-02-10T11:31:59.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.297 7fe69599b700 1 --2- 172.21.2.101:0/423855371 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6900a3ee0 0x7fe6900a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:59.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.298 7fe69599b700 1 --2- 172.21.2.101:0/423855371 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe690093e80 0x7fe6900a38c0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:59.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.298 7fe69599b700 1 --2- 172.21.2.101:0/423855371 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6740186c0 0x7fe67401abc0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:59.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.298 7fe69599b700 1 --2- 172.21.2.101:0/423855371 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6901c6a70 0x7fe690093860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:31:59.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.298 7fe69599b700 1 -- 172.21.2.101:0/423855371 >> 172.21.2.101:0/423855371 conn(0x7fe6900a3080 msgr2=0x7fe6900bb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:59.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.298 7fe69599b700 1 -- 172.21.2.101:0/423855371 shutdown_connections 2021-02-10T11:31:59.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.298 7fe69599b700 1 -- 172.21.2.101:0/423855371 wait complete. 2021-02-10T11:31:59.317 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume deauthorize cephfs subvolume_0000000000841945 guest1 --group_name group_0000000000422627 2021-02-10T11:31:59.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.721 7fc86eccf700 1 Processor -- start 2021-02-10T11:31:59.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.721 7fc86eccf700 1 -- start start 2021-02-10T11:31:59.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.721 7fc86eccf700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc8680d1c60 0x7fc8680cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:59.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.721 7fc86eccf700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8680d0160 con 0x7fc8680d1860 2021-02-10T11:31:59.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.721 7fc86eccf700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8680fe1b0 con 0x7fc8680cf570 2021-02-10T11:31:59.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.721 7fc86eccf700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8681b0ca0 con 0x7fc8680d1c60 2021-02-10T11:31:59.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.722 7fc86caeb700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc8680d1860 0x7fc8681b04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51026/0 (socket says 172.21.2.101:51026) 2021-02-10T11:31:59.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.722 7fc86caeb700 1 -- 172.21.2.101:0/3906354350 learned_addr learned my addr 172.21.2.101:0/3906354350 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:31:59.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.722 7fc867fff700 1 --2- 172.21.2.101:0/3906354350 >> v2:172.21.2.108:3300/0 conn(0x7fc8680d1c60 0x7fc8680cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:59.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.722 7fc8677fe700 1 -- 172.21.2.101:0/3906354350 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1665191716 0 0) 0x7fc850000ce0 con 0x7fc8680d1860 2021-02-10T11:31:59.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.722 7fc8677fe700 1 -- 172.21.2.101:0/3906354350 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc854002fe0 con 0x7fc8680d1860 2021-02-10T11:31:59.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.722 7fc8677fe700 1 -- 172.21.2.101:0/3906354350 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3470364371 0 0) 0x7fc860000c10 con 0x7fc8680cf570 2021-02-10T11:31:59.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.722 7fc8677fe700 1 -- 172.21.2.101:0/3906354350 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc854003210 con 0x7fc8680cf570 2021-02-10T11:31:59.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.722 7fc867fff700 1 -- 172.21.2.101:0/3906354350 >> v1:172.21.2.108:6789/0 conn(0x7fc8680cf570 legacy=0x7fc8680cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:59.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.722 7fc867fff700 1 -- 172.21.2.101:0/3906354350 >> v1:172.21.2.101:6789/0 conn(0x7fc8680d1860 legacy=0x7fc8681b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:59.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.722 7fc867fff700 1 -- 172.21.2.101:0/3906354350 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc868109a50 con 0x7fc8680d1c60 2021-02-10T11:31:59.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.722 7fc867fff700 1 --2- 172.21.2.101:0/3906354350 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8680d1c60 0x7fc8680cef50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fc8580069d0 tx=0x7fc858009e40).ready entity=mon.1 client_cookie=2b994d028cb93a3e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:59.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.723 7fc8677fe700 1 -- 172.21.2.101:0/3906354350 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc85800b510 con 0x7fc8680d1c60 2021-02-10T11:31:59.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.723 7fc8677fe700 1 -- 172.21.2.101:0/3906354350 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85800b890 con 0x7fc8680d1c60 2021-02-10T11:31:59.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.723 7fc8677fe700 1 -- 172.21.2.101:0/3906354350 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc85800bd70 con 0x7fc8680d1c60 2021-02-10T11:31:59.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.723 7fc86eccf700 1 -- 172.21.2.101:0/3906354350 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8680d1c60 msgr2=0x7fc8680cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:59.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.723 7fc86eccf700 1 --2- 172.21.2.101:0/3906354350 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8680d1c60 0x7fc8680cef50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fc8580069d0 tx=0x7fc858009e40).stop 2021-02-10T11:31:59.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.723 7fc86eccf700 1 -- 172.21.2.101:0/3906354350 shutdown_connections 2021-02-10T11:31:59.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.723 7fc86eccf700 1 --2- 172.21.2.101:0/3906354350 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8680d1c60 0x7fc8680cef50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:59.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.724 7fc86eccf700 1 -- 172.21.2.101:0/3906354350 >> 172.21.2.101:0/3906354350 conn(0x7fc8680ce410 msgr2=0x7fc8680d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:59.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.724 7fc86eccf700 1 -- 172.21.2.101:0/3906354350 shutdown_connections 2021-02-10T11:31:59.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.724 7fc86eccf700 1 -- 172.21.2.101:0/3906354350 wait complete. 2021-02-10T11:31:59.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.738 7fc86eccf700 1 Processor -- start 2021-02-10T11:31:59.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.738 7fc86eccf700 1 -- start start 2021-02-10T11:31:59.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.739 7fc86eccf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc868080f10 0x7fc86808c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:59.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.739 7fc86eccf700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc868081310 0x7fc86808ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.739 7fc86eccf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc86819b890 0x7fc86819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.739 7fc86eccf700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86819cda0 con 0x7fc86819b890 2021-02-10T11:31:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.739 7fc86eccf700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86808d220 con 0x7fc868081310 2021-02-10T11:31:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.739 7fc86eccf700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc868099b10 con 0x7fc868080f10 2021-02-10T11:31:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.739 7fc86d2ec700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc86819b890 0x7fc86819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.739 7fc86d2ec700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc86819b890 0x7fc86819bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50538/0 (socket says 172.21.2.101:50538) 2021-02-10T11:31:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.739 7fc86d2ec700 1 -- 172.21.2.101:0/466644562 learned_addr learned my addr 172.21.2.101:0/466644562 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:31:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.739 7fc86caeb700 1 --2- 172.21.2.101:0/466644562 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc868080f10 0x7fc86808c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.739 7fc867fff700 1 --2- 172.21.2.101:0/466644562 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc868081310 0x7fc86808ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.739 7fc86d2ec700 1 -- 172.21.2.101:0/466644562 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc868080f10 msgr2=0x7fc86808c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.739 7fc86d2ec700 1 --2- 172.21.2.101:0/466644562 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc868080f10 0x7fc86808c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.739 7fc86d2ec700 1 -- 172.21.2.101:0/466644562 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc868081310 msgr2=0x7fc86808ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.739 7fc86d2ec700 1 --2- 172.21.2.101:0/466644562 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc868081310 0x7fc86808ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.739 7fc86d2ec700 1 -- 172.21.2.101:0/466644562 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86819c2b0 con 0x7fc86819b890 2021-02-10T11:31:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.739 7fc86d2ec700 1 --2- 172.21.2.101:0/466644562 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc86819b890 0x7fc86819bc90 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7fc86000b580 tx=0x7fc86000b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.740 7fc8657fa700 1 -- 172.21.2.101:0/466644562 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc86000bd10 con 0x7fc86819b890 2021-02-10T11:31:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.740 7fc8657fa700 1 -- 172.21.2.101:0/466644562 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc860006240 con 0x7fc86819b890 2021-02-10T11:31:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.740 7fc8657fa700 1 -- 172.21.2.101:0/466644562 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc8600057a0 con 0x7fc86819b890 2021-02-10T11:31:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.740 7fc86eccf700 1 -- 172.21.2.101:0/466644562 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86819c2b0 con 0x7fc86819b890 2021-02-10T11:31:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.740 7fc86eccf700 1 -- 172.21.2.101:0/466644562 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86819c840 con 0x7fc86819b890 2021-02-10T11:31:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.740 7fc8657fa700 1 -- 172.21.2.101:0/466644562 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc860005520 con 0x7fc86819b890 2021-02-10T11:31:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.741 7fc8657fa700 1 --2- 172.21.2.101:0/466644562 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc84c0186c0 0x7fc84c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:31:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.741 7fc8657fa700 1 -- 172.21.2.101:0/466644562 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc860032450 con 0x7fc86819b890 2021-02-10T11:31:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.741 7fc86caeb700 1 --2- 172.21.2.101:0/466644562 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc84c0186c0 0x7fc84c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:31:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.741 7fc86caeb700 1 --2- 172.21.2.101:0/466644562 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc84c0186c0 0x7fc84c01abc0 crc :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:31:59.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.771 7fc86eccf700 1 -- 172.21.2.101:0/466644562 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86807e3d0 con 0x7fc86819b890 2021-02-10T11:31:59.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.774 7fc8657fa700 1 -- 172.21.2.101:0/466644562 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc860030d80 con 0x7fc86819b890 2021-02-10T11:31:59.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.899 7fc86eccf700 1 -- 172.21.2.101:0/466644562 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume deauthorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000841945", "target": ["mgr", ""], "auth_id": "guest1", "group_name": "group_0000000000422627"}) v1 -- 0x7fc86808d220 con 0x7fc84c0186c0 2021-02-10T11:31:59.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.935 7fc8657fa700 1 -- 172.21.2.101:0/466644562 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fc850006860 con 0x7fc84c0186c0 2021-02-10T11:31:59.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.965 7fc86eccf700 1 -- 172.21.2.101:0/466644562 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc84c0186c0 msgr2=0x7fc84c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:59.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.965 7fc86eccf700 1 --2- 172.21.2.101:0/466644562 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc84c0186c0 0x7fc84c01abc0 crc :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:59.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.965 7fc86eccf700 1 -- 172.21.2.101:0/466644562 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc86819b890 msgr2=0x7fc86819bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:31:59.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.965 7fc86eccf700 1 --2- 172.21.2.101:0/466644562 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc86819b890 0x7fc86819bc90 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7fc86000b580 tx=0x7fc86000b920).stop 2021-02-10T11:31:59.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.965 7fc86eccf700 1 -- 172.21.2.101:0/466644562 shutdown_connections 2021-02-10T11:31:59.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.965 7fc86eccf700 1 --2- 172.21.2.101:0/466644562 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc86819b890 0x7fc86819bc90 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:59.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.965 7fc86eccf700 1 --2- 172.21.2.101:0/466644562 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc868081310 0x7fc86808ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:59.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.965 7fc86eccf700 1 --2- 172.21.2.101:0/466644562 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc84c0186c0 0x7fc84c01abc0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:59.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.965 7fc86eccf700 1 --2- 172.21.2.101:0/466644562 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc868080f10 0x7fc86808c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:31:59.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.965 7fc86eccf700 1 -- 172.21.2.101:0/466644562 >> 172.21.2.101:0/466644562 conn(0x7fc86819f270 msgr2=0x7fc868093d00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:31:59.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.966 7fc86eccf700 1 -- 172.21.2.101:0/466644562 shutdown_connections 2021-02-10T11:31:59.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:31:59.966 7fc86eccf700 1 -- 172.21.2.101:0/466644562 wait complete. 2021-02-10T11:31:59.985 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth rm client.guest1 2021-02-10T11:32:00.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.384 7f2432263700 1 Processor -- start 2021-02-10T11:32:00.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.384 7f2432263700 1 -- start start 2021-02-10T11:32:00.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.385 7f2432263700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f242c0e5470 0x7f242c07e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:00.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.385 7f2432263700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f242c07f300 0x7f242c07f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:00.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.385 7f2432263700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f242c07ed80 con 0x7f242c0e5070 2021-02-10T11:32:00.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.385 7f2432263700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f242c1b4010 con 0x7f242c07f300 2021-02-10T11:32:00.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.385 7f2432263700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242c0d1790 con 0x7f242c0e5470 2021-02-10T11:32:00.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.385 7f242bfff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f242c0e5070 0x7f242c1b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51038/0 (socket says 172.21.2.101:51038) 2021-02-10T11:32:00.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.385 7f242bfff700 1 -- 172.21.2.101:0/252358795 learned_addr learned my addr 172.21.2.101:0/252358795 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:00.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.385 7f2430880700 1 --2- 172.21.2.101:0/252358795 >> v2:172.21.2.101:3300/0 conn(0x7f242c07f300 0x7f242c07f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:00.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.385 7f242b7fe700 1 --2- 172.21.2.101:0/252358795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f242c0e5470 0x7f242c07e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:00.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.386 7f2430880700 1 -- 172.21.2.101:0/252358795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f242c0e5470 msgr2=0x7f242c07e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:00.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.386 7f2430880700 1 --2- 172.21.2.101:0/252358795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f242c0e5470 0x7f242c07e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:00.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.386 7f2430880700 1 -- 172.21.2.101:0/252358795 >> v1:172.21.2.101:6789/0 conn(0x7f242c0e5070 legacy=0x7f242c1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:00.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.386 7f242affd700 1 -- 172.21.2.101:0/252358795 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970217538 0 0) 0x7f241c001180 con 0x7f242c0e5070 2021-02-10T11:32:00.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.386 7f2430880700 1 -- 172.21.2.101:0/252358795 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c0d1980 con 0x7f242c07f300 2021-02-10T11:32:00.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.386 7f242b7fe700 1 --2- 172.21.2.101:0/252358795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f242c0e5470 0x7f242c07e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:32:00.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.386 7f2430880700 1 --2- 172.21.2.101:0/252358795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f242c07f300 0x7f242c07f700 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7f2424006e30 tx=0x7f2424009e60).ready entity=mon.0 client_cookie=22d6a998ead6b69d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:00.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.386 7f242affd700 1 -- 172.21.2.101:0/252358795 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f242400b300 con 0x7f242c07f300 2021-02-10T11:32:00.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.386 7f242affd700 1 -- 172.21.2.101:0/252358795 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2424000d30 con 0x7f242c07f300 2021-02-10T11:32:00.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.386 7f242affd700 1 -- 172.21.2.101:0/252358795 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f242400b830 con 0x7f242c07f300 2021-02-10T11:32:00.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.386 7f2432263700 1 -- 172.21.2.101:0/252358795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f242c07f300 msgr2=0x7f242c07f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:00.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.386 7f2432263700 1 --2- 172.21.2.101:0/252358795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f242c07f300 0x7f242c07f700 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7f2424006e30 tx=0x7f2424009e60).stop 2021-02-10T11:32:00.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.387 7f2432263700 1 -- 172.21.2.101:0/252358795 shutdown_connections 2021-02-10T11:32:00.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.387 7f2432263700 1 --2- 172.21.2.101:0/252358795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f242c0e5470 0x7f242c07e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:00.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.387 7f2432263700 1 --2- 172.21.2.101:0/252358795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f242c07f300 0x7f242c07f700 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:00.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.387 7f2432263700 1 -- 172.21.2.101:0/252358795 >> 172.21.2.101:0/252358795 conn(0x7f242c0ce690 msgr2=0x7f242c0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:00.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.387 7f2432263700 1 -- 172.21.2.101:0/252358795 shutdown_connections 2021-02-10T11:32:00.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.387 7f2432263700 1 -- 172.21.2.101:0/252358795 wait complete. 2021-02-10T11:32:00.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.401 7f2432263700 1 Processor -- start 2021-02-10T11:32:00.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.402 7f2432263700 1 -- start start 2021-02-10T11:32:00.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.402 7f2432263700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f242c08f500 0x7f242c19f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:00.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.402 7f2432263700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f242c08fd90 0x7f242c090190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:00.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.402 7f2432263700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f242c0907b0 0x7f242c094370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:00.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.402 7f2432263700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c08f900 con 0x7f242c08f500 2021-02-10T11:32:00.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.402 7f2432263700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c19ff20 con 0x7f242c0907b0 2021-02-10T11:32:00.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.402 7f2432263700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242c1a0140 con 0x7f242c08fd90 2021-02-10T11:32:00.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.402 7f242bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f242c08f500 0x7f242c19f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:00.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.402 7f242bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f242c08f500 0x7f242c19f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50550/0 (socket says 172.21.2.101:50550) 2021-02-10T11:32:00.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.402 7f242bfff700 1 -- 172.21.2.101:0/3414654962 learned_addr learned my addr 172.21.2.101:0/3414654962 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:00.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.402 7f242bfff700 1 -- 172.21.2.101:0/3414654962 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f242c08fd90 msgr2=0x7f242c090190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:00.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.402 7f242bfff700 1 --2- 172.21.2.101:0/3414654962 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f242c08fd90 0x7f242c090190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:00.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.402 7f242bfff700 1 -- 172.21.2.101:0/3414654962 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f242c0907b0 msgr2=0x7f242c094370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:00.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.402 7f242bfff700 1 --2- 172.21.2.101:0/3414654962 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f242c0907b0 0x7f242c094370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:00.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.402 7f242bfff700 1 -- 172.21.2.101:0/3414654962 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c094990 con 0x7f242c08f500 2021-02-10T11:32:00.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.403 7f242bfff700 1 --2- 172.21.2.101:0/3414654962 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f242c08f500 0x7f242c19f900 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7f241c00a570 tx=0x7f241c00a910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:00.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.403 7f2428ff9700 1 -- 172.21.2.101:0/3414654962 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f241c00ad00 con 0x7f242c08f500 2021-02-10T11:32:00.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.403 7f2428ff9700 1 -- 172.21.2.101:0/3414654962 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241c001180 con 0x7f242c08f500 2021-02-10T11:32:00.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.403 7f2432263700 1 -- 172.21.2.101:0/3414654962 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f242c094990 con 0x7f242c08f500 2021-02-10T11:32:00.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.403 7f2432263700 1 -- 172.21.2.101:0/3414654962 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242c094f20 con 0x7f242c08f500 2021-02-10T11:32:00.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.403 7f2428ff9700 1 -- 172.21.2.101:0/3414654962 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f241c004620 con 0x7f242c08f500 2021-02-10T11:32:00.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.404 7f2428ff9700 1 -- 172.21.2.101:0/3414654962 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f241c009030 con 0x7f242c08f500 2021-02-10T11:32:00.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.404 7f2428ff9700 1 --2- 172.21.2.101:0/3414654962 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f240c0186c0 0x7f240c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:00.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.404 7f2428ff9700 1 -- 172.21.2.101:0/3414654962 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f241c032510 con 0x7f242c08f500 2021-02-10T11:32:00.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.404 7f242b7fe700 1 --2- 172.21.2.101:0/3414654962 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f240c0186c0 0x7f240c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:00.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.404 7f242b7fe700 1 --2- 172.21.2.101:0/3414654962 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f240c0186c0 0x7f240c01abc0 crc :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:00.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.435 7f2432263700 1 -- 172.21.2.101:0/3414654962 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f242c093e40 con 0x7f242c08f500 2021-02-10T11:32:00.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.437 7f2428ff9700 1 -- 172.21.2.101:0/3414654962 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f241c0070d0 con 0x7f242c08f500 2021-02-10T11:32:00.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.569 7f2432263700 1 -- 172.21.2.101:0/3414654962 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth rm", "entity": "client.guest1"} v 0) v1 -- 0x7f242c093e40 con 0x7f242c08f500 2021-02-10T11:32:00.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.572 7f2428ff9700 1 -- 172.21.2.101:0/3414654962 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth rm", "entity": "client.guest1"}]=0 updated v24) v1 ==== 89+0+0 (secure 0 0 0) 0x7f241c009030 con 0x7f242c08f500 2021-02-10T11:32:00.587 INFO:teuthology.orchestra.run.gibba001.stderr:updated 2021-02-10T11:32:00.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.578 7f2432263700 1 -- 172.21.2.101:0/3414654962 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f240c0186c0 msgr2=0x7f240c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:00.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.578 7f2432263700 1 --2- 172.21.2.101:0/3414654962 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f240c0186c0 0x7f240c01abc0 crc :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:00.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.578 7f2432263700 1 -- 172.21.2.101:0/3414654962 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f242c08f500 msgr2=0x7f242c19f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:00.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.578 7f2432263700 1 --2- 172.21.2.101:0/3414654962 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f242c08f500 0x7f242c19f900 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7f241c00a570 tx=0x7f241c00a910).stop 2021-02-10T11:32:00.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.578 7f2432263700 1 -- 172.21.2.101:0/3414654962 shutdown_connections 2021-02-10T11:32:00.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.578 7f2432263700 1 --2- 172.21.2.101:0/3414654962 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f240c0186c0 0x7f240c01abc0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:00.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.578 7f2432263700 1 --2- 172.21.2.101:0/3414654962 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f242c08fd90 0x7f242c090190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:00.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.578 7f2432263700 1 --2- 172.21.2.101:0/3414654962 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f242c0907b0 0x7f242c094370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:00.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.578 7f2432263700 1 --2- 172.21.2.101:0/3414654962 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f242c08f500 0x7f242c19f900 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:00.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.578 7f2432263700 1 -- 172.21.2.101:0/3414654962 >> 172.21.2.101:0/3414654962 conn(0x7f242c0b5560 msgr2=0x7f242c0a6200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:00.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.578 7f2432263700 1 -- 172.21.2.101:0/3414654962 shutdown_connections 2021-02-10T11:32:00.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:00.578 7f2432263700 1 -- 172.21.2.101:0/3414654962 wait complete. 2021-02-10T11:32:00.597 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000841945 --group_name group_0000000000422627 2021-02-10T11:32:01.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.001 7fa95e027700 1 Processor -- start 2021-02-10T11:32:01.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.001 7fa95e027700 1 -- start start 2021-02-10T11:32:01.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.001 7fa95e027700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa9580d1860 0x7fa9581b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:01.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.001 7fa95e027700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa9581b0a00 0x7fa9580cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:01.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.001 7fa95e027700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9580f0b00 con 0x7fa9580d1c60 2021-02-10T11:32:01.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.001 7fa95e027700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9580e5ec0 con 0x7fa9580d1860 2021-02-10T11:32:01.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.001 7fa95e027700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9580f0640 con 0x7fa9581b0a00 2021-02-10T11:32:01.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.002 7fa956ffd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa9581b0a00 0x7fa9580cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:01.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.002 7fa9577fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa9580d1860 0x7fa9581b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:01.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.002 7fa957fff700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fa9580d1c60 0x7fa9580cf940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:44462/0 (socket says 172.21.2.101:44462) 2021-02-10T11:32:01.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.002 7fa957fff700 1 -- 172.21.2.101:0/758380660 learned_addr learned my addr 172.21.2.101:0/758380660 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:01.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.002 7fa956ffd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa9581b0a00 0x7fa9580cf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:43290/0 (socket says 172.21.2.101:43290) 2021-02-10T11:32:01.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.003 7fa956ffd700 1 -- 172.21.2.101:0/758380660 >> v2:172.21.2.108:3300/0 conn(0x7fa9580d1860 msgr2=0x7fa9581b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:01.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.003 7fa956ffd700 1 --2- 172.21.2.101:0/758380660 >> v2:172.21.2.108:3300/0 conn(0x7fa9580d1860 0x7fa9581b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:01.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.003 7fa956ffd700 1 -- 172.21.2.101:0/758380660 >> v1:172.21.2.108:6789/0 conn(0x7fa9580d1c60 legacy=0x7fa9580cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:01.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.003 7fa956ffd700 1 -- 172.21.2.101:0/758380660 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9580f0860 con 0x7fa9581b0a00 2021-02-10T11:32:01.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.003 7fa9577fe700 1 --2- 172.21.2.101:0/758380660 >> v2:172.21.2.108:3300/0 conn(0x7fa9580d1860 0x7fa9581b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:32:01.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.003 7fa956ffd700 1 --2- 172.21.2.101:0/758380660 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa9581b0a00 0x7fa9580cf320 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fa940006650 tx=0x7fa940009a80).ready entity=mon.2 client_cookie=fa1bd0c807e4847f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:01.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.004 7fa9567fc700 1 -- 172.21.2.101:0/758380660 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa94000c0c0 con 0x7fa9581b0a00 2021-02-10T11:32:01.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.004 7fa9567fc700 1 -- 172.21.2.101:0/758380660 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94000c340 con 0x7fa9581b0a00 2021-02-10T11:32:01.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.004 7fa9567fc700 1 -- 172.21.2.101:0/758380660 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa94000c860 con 0x7fa9581b0a00 2021-02-10T11:32:01.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.004 7fa95e027700 1 -- 172.21.2.101:0/758380660 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa9581b0a00 msgr2=0x7fa9580cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:01.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.004 7fa95e027700 1 --2- 172.21.2.101:0/758380660 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa9581b0a00 0x7fa9580cf320 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fa940006650 tx=0x7fa940009a80).stop 2021-02-10T11:32:01.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.005 7fa95e027700 1 -- 172.21.2.101:0/758380660 shutdown_connections 2021-02-10T11:32:01.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.005 7fa95e027700 1 --2- 172.21.2.101:0/758380660 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa9581b0a00 0x7fa9580cf320 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:01.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.005 7fa95e027700 1 --2- 172.21.2.101:0/758380660 >> v2:172.21.2.108:3300/0 conn(0x7fa9580d1860 0x7fa9581b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:01.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.005 7fa95e027700 1 -- 172.21.2.101:0/758380660 >> 172.21.2.101:0/758380660 conn(0x7fa9580ce440 msgr2=0x7fa9580d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:01.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.005 7fa95e027700 1 -- 172.21.2.101:0/758380660 shutdown_connections 2021-02-10T11:32:01.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.005 7fa95e027700 1 -- 172.21.2.101:0/758380660 wait complete. 2021-02-10T11:32:01.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.019 7fa95e027700 1 Processor -- start 2021-02-10T11:32:01.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.019 7fa95e027700 1 -- start start 2021-02-10T11:32:01.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.020 7fa95e027700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa9581a7070 0x7fa9581a7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:01.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.020 7fa95e027700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa95818e290 0x7fa95818e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:01.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.020 7fa95e027700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa95818ecb0 0x7fa958097830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:01.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.020 7fa95e027700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa958097330 con 0x7fa95818e290 2021-02-10T11:32:01.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.020 7fa95e027700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95818f0b0 con 0x7fa95818ecb0 2021-02-10T11:32:01.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.020 7fa95e027700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa958097e50 con 0x7fa9581a7070 2021-02-10T11:32:01.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.020 7fa956ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa95818e290 0x7fa95818e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:01.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.020 7fa956ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa95818e290 0x7fa95818e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50562/0 (socket says 172.21.2.101:50562) 2021-02-10T11:32:01.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.020 7fa956ffd700 1 -- 172.21.2.101:0/933234773 learned_addr learned my addr 172.21.2.101:0/933234773 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:01.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.020 7fa956ffd700 1 -- 172.21.2.101:0/933234773 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa9581a7070 msgr2=0x7fa9581a7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:01.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.020 7fa956ffd700 1 --2- 172.21.2.101:0/933234773 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa9581a7070 0x7fa9581a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:01.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.020 7fa956ffd700 1 -- 172.21.2.101:0/933234773 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa95818ecb0 msgr2=0x7fa958097830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:01.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.020 7fa956ffd700 1 --2- 172.21.2.101:0/933234773 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa95818ecb0 0x7fa958097830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:01.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.020 7fa956ffd700 1 -- 172.21.2.101:0/933234773 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa958098070 con 0x7fa95818e290 2021-02-10T11:32:01.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.020 7fa956ffd700 1 --2- 172.21.2.101:0/933234773 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa95818e290 0x7fa95818e690 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0x7fa940009630 tx=0x7fa94000b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:01.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.020 7fa94ffff700 1 -- 172.21.2.101:0/933234773 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa9400093a0 con 0x7fa95818e290 2021-02-10T11:32:01.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.021 7fa94ffff700 1 -- 172.21.2.101:0/933234773 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94000bcf0 con 0x7fa95818e290 2021-02-10T11:32:01.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.021 7fa94ffff700 1 -- 172.21.2.101:0/933234773 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa9400054d0 con 0x7fa95818e290 2021-02-10T11:32:01.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.021 7fa95e027700 1 -- 172.21.2.101:0/933234773 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa958098070 con 0x7fa95818e290 2021-02-10T11:32:01.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.021 7fa95e027700 1 -- 172.21.2.101:0/933234773 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa958098600 con 0x7fa95818e290 2021-02-10T11:32:01.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.021 7fa94ffff700 1 -- 172.21.2.101:0/933234773 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa94000a000 con 0x7fa95818e290 2021-02-10T11:32:01.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.021 7fa94ffff700 1 --2- 172.21.2.101:0/933234773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa93c0186c0 0x7fa93c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:01.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.021 7fa94ffff700 1 -- 172.21.2.101:0/933234773 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa9400327e0 con 0x7fa95818e290 2021-02-10T11:32:01.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.021 7fa9577fe700 1 --2- 172.21.2.101:0/933234773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa93c0186c0 0x7fa93c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:01.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.022 7fa9577fe700 1 --2- 172.21.2.101:0/933234773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa93c0186c0 0x7fa93c01abc0 crc :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:01.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.051 7fa95e027700 1 -- 172.21.2.101:0/933234773 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa958098ba0 con 0x7fa95818e290 2021-02-10T11:32:01.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.054 7fa94ffff700 1 -- 172.21.2.101:0/933234773 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa940009340 con 0x7fa95818e290 2021-02-10T11:32:01.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.181 7fa95e027700 1 -- 172.21.2.101:0/933234773 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000841945", "target": ["mgr", ""], "group_name": "group_0000000000422627"}) v1 -- 0x7fa958097e50 con 0x7fa93c0186c0 2021-02-10T11:32:01.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.189 7fa94ffff700 1 -- 172.21.2.101:0/933234773 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fa9480009b0 con 0x7fa93c0186c0 2021-02-10T11:32:01.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.197 7fa95e027700 1 -- 172.21.2.101:0/933234773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa93c0186c0 msgr2=0x7fa93c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:01.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.197 7fa95e027700 1 --2- 172.21.2.101:0/933234773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa93c0186c0 0x7fa93c01abc0 crc :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:01.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.197 7fa95e027700 1 -- 172.21.2.101:0/933234773 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa95818e290 msgr2=0x7fa95818e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:01.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.197 7fa95e027700 1 --2- 172.21.2.101:0/933234773 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa95818e290 0x7fa95818e690 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0x7fa940009630 tx=0x7fa94000b900).stop 2021-02-10T11:32:01.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.198 7fa95e027700 1 -- 172.21.2.101:0/933234773 shutdown_connections 2021-02-10T11:32:01.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.198 7fa95e027700 1 --2- 172.21.2.101:0/933234773 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa95818ecb0 0x7fa958097830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:01.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.198 7fa95e027700 1 --2- 172.21.2.101:0/933234773 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa95818e290 0x7fa95818e690 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:01.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.198 7fa95e027700 1 --2- 172.21.2.101:0/933234773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa93c0186c0 0x7fa93c01abc0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:01.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.198 7fa95e027700 1 --2- 172.21.2.101:0/933234773 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa9581a7070 0x7fa9581a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:01.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.198 7fa95e027700 1 -- 172.21.2.101:0/933234773 >> 172.21.2.101:0/933234773 conn(0x7fa9581980c0 msgr2=0x7fa9580ebe10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:01.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.198 7fa95e027700 1 -- 172.21.2.101:0/933234773 shutdown_connections 2021-02-10T11:32:01.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.198 7fa95e027700 1 -- 172.21.2.101:0/933234773 wait complete. 2021-02-10T11:32:01.218 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolumegroup rm cephfs group_0000000000422627 2021-02-10T11:32:01.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.619 7f3719dd8700 1 Processor -- start 2021-02-10T11:32:01.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.619 7f3719dd8700 1 -- start start 2021-02-10T11:32:01.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.620 7f3719dd8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37141b0470 0x7f37141b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:01.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.620 7f3719dd8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f37140ef610 0x7f37140efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:01.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.620 7f3719dd8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37140f0b50 con 0x7f37140f0030 2021-02-10T11:32:01.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.620 7f3719dd8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37140e52e0 con 0x7f37140ef610 2021-02-10T11:32:01.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.620 7f3719dd8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37140e5490 con 0x7f37141b0470 2021-02-10T11:32:01.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.620 7f3712ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f37140ef610 0x7f37140efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:01.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.620 7f3712ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f37140ef610 0x7f37140efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50570/0 (socket says 172.21.2.101:50570) 2021-02-10T11:32:01.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.620 7f3712ffd700 1 -- 172.21.2.101:0/658124173 learned_addr learned my addr 172.21.2.101:0/658124173 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:01.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.620 7f37137fe700 1 --2- 172.21.2.101:0/658124173 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37141b0470 0x7f37141b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:01.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.620 7f3712ffd700 1 -- 172.21.2.101:0/658124173 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37141b0470 msgr2=0x7f37141b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:01.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.620 7f3712ffd700 1 --2- 172.21.2.101:0/658124173 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37141b0470 0x7f37141b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:01.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.620 7f3712ffd700 1 -- 172.21.2.101:0/658124173 >> v1:172.21.2.108:6789/0 conn(0x7f37140f0030 legacy=0x7f37140f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:01.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.620 7f3712ffd700 1 -- 172.21.2.101:0/658124173 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37140f0430 con 0x7f37140ef610 2021-02-10T11:32:01.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.621 7f3712ffd700 1 --2- 172.21.2.101:0/658124173 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37140ef610 0x7f37140efa10 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7f36fc006700 tx=0x7f36fc009b80).ready entity=mon.0 client_cookie=4350d4dfd24d32c0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:01.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.621 7f37127fc700 1 -- 172.21.2.101:0/658124173 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f36fc00c0c0 con 0x7f37140ef610 2021-02-10T11:32:01.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.621 7f37127fc700 1 -- 172.21.2.101:0/658124173 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc00c340 con 0x7f37140ef610 2021-02-10T11:32:01.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.621 7f37127fc700 1 -- 172.21.2.101:0/658124173 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f36fc00c860 con 0x7f37140ef610 2021-02-10T11:32:01.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.621 7f3719dd8700 1 -- 172.21.2.101:0/658124173 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37140ef610 msgr2=0x7f37140efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:01.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.621 7f3719dd8700 1 --2- 172.21.2.101:0/658124173 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37140ef610 0x7f37140efa10 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7f36fc006700 tx=0x7f36fc009b80).stop 2021-02-10T11:32:01.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.622 7f3719dd8700 1 -- 172.21.2.101:0/658124173 shutdown_connections 2021-02-10T11:32:01.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.622 7f3719dd8700 1 --2- 172.21.2.101:0/658124173 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37140ef610 0x7f37140efa10 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:01.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.622 7f3719dd8700 1 --2- 172.21.2.101:0/658124173 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37141b0470 0x7f37141b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:01.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.622 7f3719dd8700 1 -- 172.21.2.101:0/658124173 >> 172.21.2.101:0/658124173 conn(0x7f37140ce3e0 msgr2=0x7f37140e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:01.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.622 7f3719dd8700 1 -- 172.21.2.101:0/658124173 shutdown_connections 2021-02-10T11:32:01.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.622 7f3719dd8700 1 -- 172.21.2.101:0/658124173 wait complete. 2021-02-10T11:32:01.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.636 7f3719dd8700 1 Processor -- start 2021-02-10T11:32:01.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.636 7f3719dd8700 1 -- start start 2021-02-10T11:32:01.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.637 7f3719dd8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3714190920 0x7f37140b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:01.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.637 7f3719dd8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37140b8200 0x7f37140b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:01.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.637 7f3719dd8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37140c1d80 0x7f37140c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:01.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.637 7f3719dd8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3714190da0 con 0x7f37140b8200 2021-02-10T11:32:01.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.637 7f3719dd8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37140b8c20 con 0x7f37140c1d80 2021-02-10T11:32:01.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.637 7f3719dd8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37140b8e10 con 0x7f3714190920 2021-02-10T11:32:01.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.637 7f3712ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37140b8200 0x7f37140b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:01.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.637 7f3712ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37140b8200 0x7f37140b8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50578/0 (socket says 172.21.2.101:50578) 2021-02-10T11:32:01.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.637 7f3712ffd700 1 -- 172.21.2.101:0/1705118912 learned_addr learned my addr 172.21.2.101:0/1705118912 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:01.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.637 7f3712ffd700 1 -- 172.21.2.101:0/1705118912 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3714190920 msgr2=0x7f37140b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:01.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.637 7f3712ffd700 1 --2- 172.21.2.101:0/1705118912 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3714190920 0x7f37140b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:01.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.637 7f3712ffd700 1 -- 172.21.2.101:0/1705118912 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37140c1d80 msgr2=0x7f37140c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:01.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.637 7f3712ffd700 1 --2- 172.21.2.101:0/1705118912 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37140c1d80 0x7f37140c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:01.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.637 7f3712ffd700 1 -- 172.21.2.101:0/1705118912 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37140c27d0 con 0x7f37140b8200 2021-02-10T11:32:01.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.638 7f3712ffd700 1 --2- 172.21.2.101:0/1705118912 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37140b8200 0x7f37140b8600 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7f36fc00b5e0 tx=0x7f36fc00b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:01.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.638 7f370bfff700 1 -- 172.21.2.101:0/1705118912 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f36fc00bd70 con 0x7f37140b8200 2021-02-10T11:32:01.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.638 7f370bfff700 1 -- 172.21.2.101:0/1705118912 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc002030 con 0x7f37140b8200 2021-02-10T11:32:01.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.638 7f3719dd8700 1 -- 172.21.2.101:0/1705118912 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37140c27d0 con 0x7f37140b8200 2021-02-10T11:32:01.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.638 7f3719dd8700 1 -- 172.21.2.101:0/1705118912 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37140c2d60 con 0x7f37140b8200 2021-02-10T11:32:01.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.638 7f370bfff700 1 -- 172.21.2.101:0/1705118912 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f36fc002550 con 0x7f37140b8200 2021-02-10T11:32:01.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.638 7f370bfff700 1 -- 172.21.2.101:0/1705118912 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f36fc002210 con 0x7f37140b8200 2021-02-10T11:32:01.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.638 7f370bfff700 1 --2- 172.21.2.101:0/1705118912 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36f80186c0 0x7f36f801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:01.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.639 7f370bfff700 1 -- 172.21.2.101:0/1705118912 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f36fc032550 con 0x7f37140b8200 2021-02-10T11:32:01.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.639 7f37137fe700 1 --2- 172.21.2.101:0/1705118912 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36f80186c0 0x7f36f801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:01.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.639 7f37137fe700 1 --2- 172.21.2.101:0/1705118912 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36f80186c0 0x7f36f801abc0 crc :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:01.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.670 7f3719dd8700 1 -- 172.21.2.101:0/1705118912 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3714105ae0 con 0x7f37140b8200 2021-02-10T11:32:01.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.672 7f370bfff700 1 -- 172.21.2.101:0/1705118912 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f36fc0009b0 con 0x7f37140b8200 2021-02-10T11:32:01.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.799 7f3719dd8700 1 -- 172.21.2.101:0/1705118912 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "target": ["mgr", ""], "group_name": "group_0000000000422627"}) v1 -- 0x7f37140c2d60 con 0x7f36f80186c0 2021-02-10T11:32:01.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.804 7f370bfff700 1 -- 172.21.2.101:0/1705118912 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f37040052e0 con 0x7f36f80186c0 2021-02-10T11:32:01.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.808 7f3719dd8700 1 -- 172.21.2.101:0/1705118912 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36f80186c0 msgr2=0x7f36f801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:01.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.808 7f3719dd8700 1 --2- 172.21.2.101:0/1705118912 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36f80186c0 0x7f36f801abc0 crc :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:01.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.808 7f3719dd8700 1 -- 172.21.2.101:0/1705118912 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37140b8200 msgr2=0x7f37140b8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:01.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.808 7f3719dd8700 1 --2- 172.21.2.101:0/1705118912 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37140b8200 0x7f37140b8600 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7f36fc00b5e0 tx=0x7f36fc00b980).stop 2021-02-10T11:32:01.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.808 7f3719dd8700 1 -- 172.21.2.101:0/1705118912 shutdown_connections 2021-02-10T11:32:01.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.808 7f3719dd8700 1 --2- 172.21.2.101:0/1705118912 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37140c1d80 0x7f37140c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:01.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.808 7f3719dd8700 1 --2- 172.21.2.101:0/1705118912 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37140b8200 0x7f37140b8600 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:01.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.808 7f3719dd8700 1 --2- 172.21.2.101:0/1705118912 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36f80186c0 0x7f36f801abc0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:01.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.808 7f3719dd8700 1 --2- 172.21.2.101:0/1705118912 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3714190920 0x7f37140b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:01.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.808 7f3719dd8700 1 -- 172.21.2.101:0/1705118912 >> 172.21.2.101:0/1705118912 conn(0x7f3714094e10 msgr2=0x7f37140a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:01.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.809 7f3719dd8700 1 -- 172.21.2.101:0/1705118912 shutdown_connections 2021-02-10T11:32:01.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:01.809 7f3719dd8700 1 -- 172.21.2.101:0/1705118912 wait complete. 2021-02-10T11:32:01.828 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:32:01.828 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:32:02.174 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:32:02.518 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:32:02.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:02.518 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:32:02.518 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:32:02.855 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:32:02.855 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:32:02.855 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:32:02.855 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:32:02.855 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:32:03.208 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[110368]: fuse finished with error 0 and tester_r 0 2021-02-10T11:32:03.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:03.209 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:32:03.209 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:32:03.544 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:32:03.545 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:32:03.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:32:03.545 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:32:03.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:03.880 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:32:03.880 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:32:04.216 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:32:04.216 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:32:04.217 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:32:04.217 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:32:04.217 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:32:04.569 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[111008]: fuse finished with error 0 and tester_r 0 2021-02-10T11:32:04.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:04.570 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:32:04.570 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:32:04.903 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:32:04.904 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:32:04.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:32:04.904 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:32:05.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:05.239 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:32:05.239 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:32:05.574 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:32:05.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:32:05.574 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:32:05.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:32:05.575 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:32:05.909 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:05.909 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:32:05.909 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:32:06.245 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:32:06.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:32:06.245 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:32:06.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:32:06.245 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:32:06.578 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-02-10T11:32:07.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.978 7f8400f2c700 1 Processor -- start 2021-02-10T11:32:07.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.978 7f8400f2c700 1 -- start start 2021-02-10T11:32:07.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.979 7f8400f2c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f83fc0d17f0 0x7f83fc1b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:07.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.979 7f8400f2c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83fc0d1bf0 0x7f83fc1b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:07.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.979 7f8400f2c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83fc0f0b20 con 0x7f83fc1b3e20 2021-02-10T11:32:07.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.979 7f8400f2c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83fc0e5eb0 con 0x7f83fc0d17f0 2021-02-10T11:32:07.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.979 7f8400f2c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc0f0630 con 0x7f83fc0d1bf0 2021-02-10T11:32:07.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.979 7f83fab13700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f83fc0d17f0 0x7f83fc1b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:07.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.979 7f83fab13700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f83fc0d17f0 0x7f83fc1b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50582/0 (socket says 172.21.2.101:50582) 2021-02-10T11:32:07.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.979 7f83fab13700 1 -- 172.21.2.101:0/2359061934 learned_addr learned my addr 172.21.2.101:0/2359061934 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:07.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.980 7f83fab13700 1 -- 172.21.2.101:0/2359061934 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83fc0d1bf0 msgr2=0x7f83fc1b3800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:32:07.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.980 7f83fab13700 1 --2- 172.21.2.101:0/2359061934 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83fc0d1bf0 0x7f83fc1b3800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:07.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.980 7f83fab13700 1 -- 172.21.2.101:0/2359061934 >> v1:172.21.2.108:6789/0 conn(0x7f83fc1b3e20 legacy=0x7f83fc0ceef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:32:07.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.980 7f83fab13700 1 -- 172.21.2.101:0/2359061934 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc0f0820 con 0x7f83fc0d17f0 2021-02-10T11:32:07.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.980 7f83fab13700 1 --2- 172.21.2.101:0/2359061934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83fc0d17f0 0x7f83fc1b31e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7f83ec006700 tx=0x7f83ec009b80).ready entity=mon.0 client_cookie=3ee15dd50029e25e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:07.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.980 7f83f9b11700 1 -- 172.21.2.101:0/2359061934 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f83ec00c0c0 con 0x7f83fc0d17f0 2021-02-10T11:32:07.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.980 7f83f9b11700 1 -- 172.21.2.101:0/2359061934 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec00c440 con 0x7f83fc0d17f0 2021-02-10T11:32:07.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.980 7f83f9b11700 1 -- 172.21.2.101:0/2359061934 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f83ec00c960 con 0x7f83fc0d17f0 2021-02-10T11:32:07.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.980 7f8400f2c700 1 -- 172.21.2.101:0/2359061934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83fc0d17f0 msgr2=0x7f83fc1b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:07.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.980 7f8400f2c700 1 --2- 172.21.2.101:0/2359061934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83fc0d17f0 0x7f83fc1b31e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7f83ec006700 tx=0x7f83ec009b80).stop 2021-02-10T11:32:07.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.981 7f8400f2c700 1 -- 172.21.2.101:0/2359061934 shutdown_connections 2021-02-10T11:32:07.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.981 7f8400f2c700 1 --2- 172.21.2.101:0/2359061934 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83fc0d1bf0 0x7f83fc1b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:07.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.981 7f8400f2c700 1 --2- 172.21.2.101:0/2359061934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83fc0d17f0 0x7f83fc1b31e0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:07.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.981 7f8400f2c700 1 -- 172.21.2.101:0/2359061934 >> 172.21.2.101:0/2359061934 conn(0x7f83fc0fde40 msgr2=0x7f83fc0ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:07.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.981 7f8400f2c700 1 -- 172.21.2.101:0/2359061934 shutdown_connections 2021-02-10T11:32:07.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.981 7f8400f2c700 1 -- 172.21.2.101:0/2359061934 wait complete. 2021-02-10T11:32:07.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.995 7f8400f2c700 1 Processor -- start 2021-02-10T11:32:07.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.995 7f8400f2c700 1 -- start start 2021-02-10T11:32:07.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.996 7f8400f2c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83fc0b7a00 0x7f83fc0b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:07.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.996 7f8400f2c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83fc08a400 0x7f83fc08a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:07.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.996 7f8400f2c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83fc08ae20 0x7f83fc092480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:07.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.996 7f8400f2c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc08b330 con 0x7f83fc08a400 2021-02-10T11:32:07.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.996 7f8400f2c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc092aa0 con 0x7f83fc08ae20 2021-02-10T11:32:07.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.996 7f8400f2c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc092c80 con 0x7f83fc0b7a00 2021-02-10T11:32:07.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.996 7f83fa312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83fc08a400 0x7f83fc08a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:07.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.996 7f83fa312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83fc08a400 0x7f83fc08a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50586/0 (socket says 172.21.2.101:50586) 2021-02-10T11:32:07.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.996 7f83fa312700 1 -- 172.21.2.101:0/872844013 learned_addr learned my addr 172.21.2.101:0/872844013 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:07.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.996 7f83fa312700 1 -- 172.21.2.101:0/872844013 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83fc0b7a00 msgr2=0x7f83fc0b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:07.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.996 7f83fa312700 1 --2- 172.21.2.101:0/872844013 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83fc0b7a00 0x7f83fc0b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:07.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.996 7f83fa312700 1 -- 172.21.2.101:0/872844013 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83fc08ae20 msgr2=0x7f83fc092480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:07.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.996 7f83fa312700 1 --2- 172.21.2.101:0/872844013 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83fc08ae20 0x7f83fc092480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:07.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.996 7f83fa312700 1 -- 172.21.2.101:0/872844013 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc092ea0 con 0x7f83fc08a400 2021-02-10T11:32:07.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.996 7f83fa312700 1 --2- 172.21.2.101:0/872844013 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83fc08a400 0x7f83fc08a800 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7f83f000b5c0 tx=0x7f83f000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:07.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.997 7f83eb7fe700 1 -- 172.21.2.101:0/872844013 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f83f000bd50 con 0x7f83fc08a400 2021-02-10T11:32:07.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.997 7f83eb7fe700 1 -- 172.21.2.101:0/872844013 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f0006030 con 0x7f83fc08a400 2021-02-10T11:32:07.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.997 7f83eb7fe700 1 -- 172.21.2.101:0/872844013 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f83f00054d0 con 0x7f83fc08a400 2021-02-10T11:32:07.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.997 7f8400f2c700 1 -- 172.21.2.101:0/872844013 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83fc092ea0 con 0x7f83fc08a400 2021-02-10T11:32:07.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.997 7f8400f2c700 1 -- 172.21.2.101:0/872844013 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83fc08bba0 con 0x7f83fc08a400 2021-02-10T11:32:07.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.997 7f83eb7fe700 1 -- 172.21.2.101:0/872844013 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f83f0006210 con 0x7f83fc08a400 2021-02-10T11:32:07.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.997 7f83eb7fe700 1 --2- 172.21.2.101:0/872844013 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83e00186c0 0x7f83e001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:07.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.997 7f83eb7fe700 1 -- 172.21.2.101:0/872844013 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f83f0032680 con 0x7f83fc08a400 2021-02-10T11:32:07.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.998 7f83fab13700 1 --2- 172.21.2.101:0/872844013 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83e00186c0 0x7f83e001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:07.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:06.998 7f83fab13700 1 --2- 172.21.2.101:0/872844013 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83e00186c0 0x7f83e001abc0 crc :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:07.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.029 7f8400f2c700 1 -- 172.21.2.101:0/872844013 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83fc1036f0 con 0x7f83fc08a400 2021-02-10T11:32:07.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.031 7f83eb7fe700 1 -- 172.21.2.101:0/872844013 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f83f0000940 con 0x7f83fc08a400 2021-02-10T11:32:07.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.179 7f8400f2c700 1 -- 172.21.2.101:0/872844013 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f83fc0f0b20 con 0x7f83fc08a400 2021-02-10T11:32:07.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.183 7f83eb7fe700 1 -- 172.21.2.101:0/872844013 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7f83f0006030 con 0x7f83fc08a400 2021-02-10T11:32:07.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.188 7f8400f2c700 1 -- 172.21.2.101:0/872844013 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83e00186c0 msgr2=0x7f83e001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:07.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.188 7f8400f2c700 1 --2- 172.21.2.101:0/872844013 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83e00186c0 0x7f83e001abc0 crc :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:07.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.188 7f8400f2c700 1 -- 172.21.2.101:0/872844013 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83fc08a400 msgr2=0x7f83fc08a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:07.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.188 7f8400f2c700 1 --2- 172.21.2.101:0/872844013 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83fc08a400 0x7f83fc08a800 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7f83f000b5c0 tx=0x7f83f000b960).stop 2021-02-10T11:32:07.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.188 7f8400f2c700 1 -- 172.21.2.101:0/872844013 shutdown_connections 2021-02-10T11:32:07.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.189 7f8400f2c700 1 --2- 172.21.2.101:0/872844013 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83fc08ae20 0x7f83fc092480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:07.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.189 7f8400f2c700 1 --2- 172.21.2.101:0/872844013 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83fc08a400 0x7f83fc08a800 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:07.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.189 7f8400f2c700 1 --2- 172.21.2.101:0/872844013 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83e00186c0 0x7f83e001abc0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:07.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.189 7f8400f2c700 1 --2- 172.21.2.101:0/872844013 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83fc0b7a00 0x7f83fc0b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:07.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.189 7f8400f2c700 1 -- 172.21.2.101:0/872844013 >> 172.21.2.101:0/872844013 conn(0x7f83fc08fa40 msgr2=0x7f83fc08de60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:07.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.189 7f8400f2c700 1 -- 172.21.2.101:0/872844013 shutdown_connections 2021-02-10T11:32:07.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.189 7f8400f2c700 1 -- 172.21.2.101:0/872844013 wait complete. 2021-02-10T11:32:07.208 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_authorize_allow_existing_id_option' 2021-02-10T11:32:07.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.606 7fc7c67a7700 1 Processor -- start 2021-02-10T11:32:07.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.606 7fc7c67a7700 1 -- start start 2021-02-10T11:32:07.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.607 7fc7c67a7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc7c00e5070 0x7fc7c01b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:07.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.607 7fc7c67a7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c00e5470 0x7fc7c0080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:07.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.607 7fc7c67a7700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c007ed80 con 0x7fc7c01b3e40 2021-02-10T11:32:07.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.607 7fc7c67a7700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c00d1790 con 0x7fc7c00e5070 2021-02-10T11:32:07.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.607 7fc7c67a7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c00d19b0 con 0x7fc7c00e5470 2021-02-10T11:32:07.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.607 7fc7bffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc7c00e5070 0x7fc7c01b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:07.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.607 7fc7bffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc7c00e5070 0x7fc7c01b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50594/0 (socket says 172.21.2.101:50594) 2021-02-10T11:32:07.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.607 7fc7bffff700 1 -- 172.21.2.101:0/3792865197 learned_addr learned my addr 172.21.2.101:0/3792865197 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:07.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.607 7fc7bffff700 1 -- 172.21.2.101:0/3792865197 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c00e5470 msgr2=0x7fc7c0080310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:32:07.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.607 7fc7bffff700 1 --2- 172.21.2.101:0/3792865197 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c00e5470 0x7fc7c0080310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:07.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.607 7fc7bffff700 1 -- 172.21.2.101:0/3792865197 >> v1:172.21.2.108:6789/0 conn(0x7fc7c01b3e40 legacy=0x7fc7c007d690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:32:07.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.607 7fc7bffff700 1 -- 172.21.2.101:0/3792865197 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c00d1bd0 con 0x7fc7c00e5070 2021-02-10T11:32:07.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.607 7fc7bffff700 1 --2- 172.21.2.101:0/3792865197 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c00e5070 0x7fc7c01b3820 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7fc7b0006a80 tx=0x7fc7b0009e80).ready entity=mon.0 client_cookie=541f6bb3e5339aeb server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:07.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.607 7fc7beffd700 1 -- 172.21.2.101:0/3792865197 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc7b000b510 con 0x7fc7c00e5070 2021-02-10T11:32:07.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.607 7fc7beffd700 1 -- 172.21.2.101:0/3792865197 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b000b7d0 con 0x7fc7c00e5070 2021-02-10T11:32:07.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.607 7fc7beffd700 1 -- 172.21.2.101:0/3792865197 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc7b000bcb0 con 0x7fc7c00e5070 2021-02-10T11:32:07.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.608 7fc7c67a7700 1 -- 172.21.2.101:0/3792865197 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c00e5070 msgr2=0x7fc7c01b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:07.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.608 7fc7c67a7700 1 --2- 172.21.2.101:0/3792865197 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c00e5070 0x7fc7c01b3820 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7fc7b0006a80 tx=0x7fc7b0009e80).stop 2021-02-10T11:32:07.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.608 7fc7c67a7700 1 -- 172.21.2.101:0/3792865197 shutdown_connections 2021-02-10T11:32:07.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.608 7fc7c67a7700 1 --2- 172.21.2.101:0/3792865197 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c00e5470 0x7fc7c0080310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:07.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.608 7fc7c67a7700 1 --2- 172.21.2.101:0/3792865197 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c00e5070 0x7fc7c01b3820 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:07.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.608 7fc7c67a7700 1 -- 172.21.2.101:0/3792865197 >> 172.21.2.101:0/3792865197 conn(0x7fc7c00ce690 msgr2=0x7fc7c00e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:07.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.608 7fc7c67a7700 1 -- 172.21.2.101:0/3792865197 shutdown_connections 2021-02-10T11:32:07.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.608 7fc7c67a7700 1 -- 172.21.2.101:0/3792865197 wait complete. 2021-02-10T11:32:07.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.623 7fc7c67a7700 1 Processor -- start 2021-02-10T11:32:07.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.623 7fc7c67a7700 1 -- start start 2021-02-10T11:32:07.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.623 7fc7c67a7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c01a2a90 0x7fc7c01aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:07.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.623 7fc7c67a7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7c01aa700 0x7fc7c01a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:07.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.623 7fc7c67a7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c00a55e0 0x7fc7c01a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:07.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.623 7fc7c67a7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c01a34a0 con 0x7fc7c00a55e0 2021-02-10T11:32:07.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.623 7fc7c67a7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c01a2e90 con 0x7fc7c01aa700 2021-02-10T11:32:07.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.623 7fc7c67a7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c01a30b0 con 0x7fc7c01a2a90 2021-02-10T11:32:07.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.623 7fc7c4dc4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c00a55e0 0x7fc7c01a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:07.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.623 7fc7c4dc4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c00a55e0 0x7fc7c01a4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50604/0 (socket says 172.21.2.101:50604) 2021-02-10T11:32:07.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.623 7fc7c4dc4700 1 -- 172.21.2.101:0/909913857 learned_addr learned my addr 172.21.2.101:0/909913857 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:07.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.624 7fc7c4dc4700 1 -- 172.21.2.101:0/909913857 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c01a2a90 msgr2=0x7fc7c01aa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:07.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.624 7fc7c4dc4700 1 --2- 172.21.2.101:0/909913857 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c01a2a90 0x7fc7c01aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:07.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.624 7fc7c4dc4700 1 -- 172.21.2.101:0/909913857 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7c01aa700 msgr2=0x7fc7c01a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:07.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.624 7fc7c4dc4700 1 --2- 172.21.2.101:0/909913857 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7c01aa700 0x7fc7c01a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:07.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.624 7fc7c4dc4700 1 -- 172.21.2.101:0/909913857 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c01a4660 con 0x7fc7c00a55e0 2021-02-10T11:32:07.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.624 7fc7c4dc4700 1 --2- 172.21.2.101:0/909913857 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c00a55e0 0x7fc7c01a4040 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7fc7b800b5c0 tx=0x7fc7b800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:07.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.624 7fc7bcff9700 1 -- 172.21.2.101:0/909913857 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc7b8000990 con 0x7fc7c00a55e0 2021-02-10T11:32:07.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.624 7fc7bcff9700 1 -- 172.21.2.101:0/909913857 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b8006030 con 0x7fc7c00a55e0 2021-02-10T11:32:07.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.624 7fc7bcff9700 1 -- 172.21.2.101:0/909913857 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc7b8006320 con 0x7fc7c00a55e0 2021-02-10T11:32:07.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.624 7fc7c67a7700 1 -- 172.21.2.101:0/909913857 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c01a4660 con 0x7fc7c00a55e0 2021-02-10T11:32:07.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.624 7fc7c67a7700 1 -- 172.21.2.101:0/909913857 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c01a4660 con 0x7fc7c00a55e0 2021-02-10T11:32:07.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.625 7fc7bcff9700 1 -- 172.21.2.101:0/909913857 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc7b80062a0 con 0x7fc7c00a55e0 2021-02-10T11:32:07.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.625 7fc7bcff9700 1 --2- 172.21.2.101:0/909913857 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7a00186d0 0x7fc7a001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:07.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.625 7fc7bcff9700 1 -- 172.21.2.101:0/909913857 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc7b8032800 con 0x7fc7c00a55e0 2021-02-10T11:32:07.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.625 7fc7bffff700 1 --2- 172.21.2.101:0/909913857 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7a00186d0 0x7fc7a001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:07.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.625 7fc7bffff700 1 --2- 172.21.2.101:0/909913857 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7a00186d0 0x7fc7a001abd0 crc :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:07.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.656 7fc7c67a7700 1 -- 172.21.2.101:0/909913857 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c0083da0 con 0x7fc7c00a55e0 2021-02-10T11:32:07.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.659 7fc7bcff9700 1 -- 172.21.2.101:0/909913857 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc7b8006b80 con 0x7fc7c00a55e0 2021-02-10T11:32:07.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:07.790 7fc7c67a7700 1 -- 172.21.2.101:0/909913857 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_authorize_allow_existing_id_option"]} v 0) v1 -- 0x7fc7c007ed80 con 0x7fc7c00a55e0 2021-02-10T11:32:08.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.445 7fc7bcff9700 1 -- 172.21.2.101:0/909913857 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_authorize_allow_existing_id_option"]}]=0 v246) v1 ==== 156+0+0 (secure 0 0 0) 0x7fc7b8006030 con 0x7fc7c00a55e0 2021-02-10T11:32:08.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.462 7fc7c67a7700 1 -- 172.21.2.101:0/909913857 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7a00186d0 msgr2=0x7fc7a001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:08.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.462 7fc7c67a7700 1 --2- 172.21.2.101:0/909913857 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7a00186d0 0x7fc7a001abd0 crc :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:08.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.462 7fc7c67a7700 1 -- 172.21.2.101:0/909913857 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c00a55e0 msgr2=0x7fc7c01a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:08.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.462 7fc7c67a7700 1 --2- 172.21.2.101:0/909913857 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c00a55e0 0x7fc7c01a4040 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7fc7b800b5c0 tx=0x7fc7b800b960).stop 2021-02-10T11:32:08.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.462 7fc7c67a7700 1 -- 172.21.2.101:0/909913857 shutdown_connections 2021-02-10T11:32:08.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.462 7fc7c67a7700 1 --2- 172.21.2.101:0/909913857 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c00a55e0 0x7fc7c01a4040 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:08.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.462 7fc7c67a7700 1 --2- 172.21.2.101:0/909913857 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7c01aa700 0x7fc7c01a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:08.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.462 7fc7c67a7700 1 --2- 172.21.2.101:0/909913857 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7a00186d0 0x7fc7a001abd0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:08.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.462 7fc7c67a7700 1 --2- 172.21.2.101:0/909913857 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c01a2a90 0x7fc7c01aa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:08.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.462 7fc7c67a7700 1 -- 172.21.2.101:0/909913857 >> 172.21.2.101:0/909913857 conn(0x7fc7c008fe10 msgr2=0x7fc7c019c020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:08.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.462 7fc7c67a7700 1 -- 172.21.2.101:0/909913857 shutdown_connections 2021-02-10T11:32:08.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.462 7fc7c67a7700 1 -- 172.21.2.101:0/909913857 wait complete. 2021-02-10T11:32:08.481 INFO:tasks.cephfs_test_runner:test_authorize_allow_existing_id_option (tasks.cephfs.test_volumes.TestVolumes) ... ok 2021-02-10T11:32:08.481 INFO:tasks.cephfs_test_runner:Starting test: test_authorize_auth_id_not_created_by_mgr_volumes (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:32:08.482 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_authorize_auth_id_not_created_by_mgr_volumes' 2021-02-10T11:32:08.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.881 7fdc76a5a700 1 Processor -- start 2021-02-10T11:32:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.881 7fdc76a5a700 1 -- start start 2021-02-10T11:32:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.881 7fdc76a5a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc700e5470 0x7fdc7007e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.881 7fdc76a5a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc7007ed80 con 0x7fdc700e5070 2021-02-10T11:32:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.881 7fdc76a5a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc701b4010 con 0x7fdc7007f300 2021-02-10T11:32:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.881 7fdc76a5a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc700d1790 con 0x7fdc700e5470 2021-02-10T11:32:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.882 7fdc74876700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fdc700e5070 0x7fdc701b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51102/0 (socket says 172.21.2.101:51102) 2021-02-10T11:32:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.882 7fdc74876700 1 -- 172.21.2.101:0/2855665086 learned_addr learned my addr 172.21.2.101:0/2855665086 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.882 7fdc6ffff700 1 --2- 172.21.2.101:0/2855665086 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc700e5470 0x7fdc7007e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.882 7fdc6f7fe700 1 -- 172.21.2.101:0/2855665086 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1954152837 0 0) 0x7fdc58000ce0 con 0x7fdc700e5070 2021-02-10T11:32:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.882 7fdc6f7fe700 1 -- 172.21.2.101:0/2855665086 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc5c002fe0 con 0x7fdc700e5070 2021-02-10T11:32:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.882 7fdc6f7fe700 1 -- 172.21.2.101:0/2855665086 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3083011159 0 0) 0x7fdc68001400 con 0x7fdc7007f300 2021-02-10T11:32:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.882 7fdc6f7fe700 1 -- 172.21.2.101:0/2855665086 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc5c003210 con 0x7fdc7007f300 2021-02-10T11:32:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.882 7fdc6ffff700 1 -- 172.21.2.101:0/2855665086 >> v1:172.21.2.108:6789/0 conn(0x7fdc7007f300 legacy=0x7fdc7007f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.882 7fdc6ffff700 1 -- 172.21.2.101:0/2855665086 >> v1:172.21.2.101:6789/0 conn(0x7fdc700e5070 legacy=0x7fdc701b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.882 7fdc6ffff700 1 -- 172.21.2.101:0/2855665086 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc700d1980 con 0x7fdc700e5470 2021-02-10T11:32:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.883 7fdc6ffff700 1 --2- 172.21.2.101:0/2855665086 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc700e5470 0x7fdc7007e6a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fdc60006650 tx=0x7fdc60009a80).ready entity=mon.2 client_cookie=631b3ab194e979b6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.883 7fdc6f7fe700 1 -- 172.21.2.101:0/2855665086 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc6000c0c0 con 0x7fdc700e5470 2021-02-10T11:32:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.883 7fdc6f7fe700 1 -- 172.21.2.101:0/2855665086 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6000c340 con 0x7fdc700e5470 2021-02-10T11:32:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.883 7fdc6f7fe700 1 -- 172.21.2.101:0/2855665086 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc6000c860 con 0x7fdc700e5470 2021-02-10T11:32:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.883 7fdc76a5a700 1 -- 172.21.2.101:0/2855665086 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc700e5470 msgr2=0x7fdc7007e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.883 7fdc76a5a700 1 --2- 172.21.2.101:0/2855665086 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc700e5470 0x7fdc7007e6a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fdc60006650 tx=0x7fdc60009a80).stop 2021-02-10T11:32:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.883 7fdc76a5a700 1 -- 172.21.2.101:0/2855665086 shutdown_connections 2021-02-10T11:32:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.883 7fdc76a5a700 1 --2- 172.21.2.101:0/2855665086 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc700e5470 0x7fdc7007e6a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.883 7fdc76a5a700 1 -- 172.21.2.101:0/2855665086 >> 172.21.2.101:0/2855665086 conn(0x7fdc700ce690 msgr2=0x7fdc700e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.883 7fdc76a5a700 1 -- 172.21.2.101:0/2855665086 shutdown_connections 2021-02-10T11:32:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.883 7fdc76a5a700 1 -- 172.21.2.101:0/2855665086 wait complete. 2021-02-10T11:32:08.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.898 7fdc76a5a700 1 Processor -- start 2021-02-10T11:32:08.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.898 7fdc76a5a700 1 -- start start 2021-02-10T11:32:08.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.898 7fdc76a5a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc7008d400 0x7fdc70081230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:08.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.898 7fdc76a5a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc70081850 0x7fdc700acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:08.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.898 7fdc76a5a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc70081c50 0x7fdc700ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:08.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.898 7fdc76a5a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc70082110 con 0x7fdc70081c50 2021-02-10T11:32:08.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.898 7fdc76a5a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7008d800 con 0x7fdc7008d400 2021-02-10T11:32:08.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.898 7fdc76a5a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc700adbc0 con 0x7fdc70081850 2021-02-10T11:32:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.899 7fdc75077700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc70081c50 0x7fdc700ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.899 7fdc75077700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc70081c50 0x7fdc700ad5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50612/0 (socket says 172.21.2.101:50612) 2021-02-10T11:32:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.899 7fdc75077700 1 -- 172.21.2.101:0/164236814 learned_addr learned my addr 172.21.2.101:0/164236814 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.899 7fdc74876700 1 --2- 172.21.2.101:0/164236814 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc7008d400 0x7fdc70081230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.899 7fdc6ffff700 1 --2- 172.21.2.101:0/164236814 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc70081850 0x7fdc700acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.899 7fdc75077700 1 -- 172.21.2.101:0/164236814 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc70081850 msgr2=0x7fdc700acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.899 7fdc75077700 1 --2- 172.21.2.101:0/164236814 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc70081850 0x7fdc700acf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.899 7fdc75077700 1 -- 172.21.2.101:0/164236814 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc7008d400 msgr2=0x7fdc70081230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.899 7fdc75077700 1 --2- 172.21.2.101:0/164236814 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc7008d400 0x7fdc70081230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.899 7fdc75077700 1 -- 172.21.2.101:0/164236814 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc70187fc0 con 0x7fdc70081c50 2021-02-10T11:32:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.899 7fdc75077700 1 --2- 172.21.2.101:0/164236814 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc70081c50 0x7fdc700ad5a0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7fdc68009940 tx=0x7fdc6800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.899 7fdc6d7fa700 1 -- 172.21.2.101:0/164236814 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc68000f40 con 0x7fdc70081c50 2021-02-10T11:32:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.900 7fdc6d7fa700 1 -- 172.21.2.101:0/164236814 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6800c2f0 con 0x7fdc70081c50 2021-02-10T11:32:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.900 7fdc6d7fa700 1 -- 172.21.2.101:0/164236814 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc6800c750 con 0x7fdc70081c50 2021-02-10T11:32:08.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.900 7fdc76a5a700 1 -- 172.21.2.101:0/164236814 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc70187fc0 con 0x7fdc70081c50 2021-02-10T11:32:08.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.900 7fdc76a5a700 1 -- 172.21.2.101:0/164236814 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc70188550 con 0x7fdc70081c50 2021-02-10T11:32:08.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.900 7fdc6d7fa700 1 -- 172.21.2.101:0/164236814 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdc68001250 con 0x7fdc70081c50 2021-02-10T11:32:08.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.900 7fdc6d7fa700 1 --2- 172.21.2.101:0/164236814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc540186c0 0x7fdc5401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:08.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.900 7fdc6d7fa700 1 -- 172.21.2.101:0/164236814 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fdc68032930 con 0x7fdc70081c50 2021-02-10T11:32:08.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.900 7fdc74876700 1 --2- 172.21.2.101:0/164236814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc540186c0 0x7fdc5401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:08.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.901 7fdc74876700 1 --2- 172.21.2.101:0/164236814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc540186c0 0x7fdc5401abc0 crc :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:08.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.931 7fdc76a5a700 1 -- 172.21.2.101:0/164236814 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc7007ed80 con 0x7fdc70081c50 2021-02-10T11:32:08.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:08.933 7fdc6d7fa700 1 -- 172.21.2.101:0/164236814 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdc68003080 con 0x7fdc70081c50 2021-02-10T11:32:09.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.065 7fdc76a5a700 1 -- 172.21.2.101:0/164236814 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_authorize_auth_id_not_created_by_mgr_volumes"]} v 0) v1 -- 0x7fdc70082090 con 0x7fdc70081c50 2021-02-10T11:32:09.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.448 7fdc6d7fa700 1 -- 172.21.2.101:0/164236814 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_authorize_auth_id_not_created_by_mgr_volumes"]}]=0 v247) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdc680015b0 con 0x7fdc70081c50 2021-02-10T11:32:09.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.483 7fdc76a5a700 1 -- 172.21.2.101:0/164236814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc540186c0 msgr2=0x7fdc5401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:09.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.483 7fdc76a5a700 1 --2- 172.21.2.101:0/164236814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc540186c0 0x7fdc5401abc0 crc :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:09.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.483 7fdc76a5a700 1 -- 172.21.2.101:0/164236814 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc70081c50 msgr2=0x7fdc700ad5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:09.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.483 7fdc76a5a700 1 --2- 172.21.2.101:0/164236814 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc70081c50 0x7fdc700ad5a0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7fdc68009940 tx=0x7fdc6800bcf0).stop 2021-02-10T11:32:09.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.484 7fdc76a5a700 1 -- 172.21.2.101:0/164236814 shutdown_connections 2021-02-10T11:32:09.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.484 7fdc76a5a700 1 --2- 172.21.2.101:0/164236814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc540186c0 0x7fdc5401abc0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:09.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.484 7fdc76a5a700 1 --2- 172.21.2.101:0/164236814 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc70081850 0x7fdc700acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:09.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.484 7fdc76a5a700 1 --2- 172.21.2.101:0/164236814 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc70081c50 0x7fdc700ad5a0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:09.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.484 7fdc76a5a700 1 --2- 172.21.2.101:0/164236814 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc7008d400 0x7fdc70081230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:09.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.484 7fdc76a5a700 1 -- 172.21.2.101:0/164236814 >> 172.21.2.101:0/164236814 conn(0x7fdc701ac620 msgr2=0x7fdc700896e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:09.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.484 7fdc76a5a700 1 -- 172.21.2.101:0/164236814 shutdown_connections 2021-02-10T11:32:09.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.484 7fdc76a5a700 1 -- 172.21.2.101:0/164236814 wait complete. 2021-02-10T11:32:09.503 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:32:09.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.905 7fb22f5e9700 1 Processor -- start 2021-02-10T11:32:09.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.905 7fb22f5e9700 1 -- start start 2021-02-10T11:32:09.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.905 7fb22f5e9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2281b0870 0x7fb2280f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:09.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.905 7fb22f5e9700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb2280efe00 0x7fb2280f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:09.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.905 7fb22f5e9700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2281b0c70 con 0x7fb2281b0470 2021-02-10T11:32:09.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.905 7fb22f5e9700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2280f0b50 con 0x7fb2280efe00 2021-02-10T11:32:09.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.905 7fb22f5e9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2280f0820 con 0x7fb2281b0870 2021-02-10T11:32:09.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.906 7fb22dc06700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb2280efe00 0x7fb2280f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:09.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.906 7fb22d405700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fb2281b0470 0x7fb2280ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:44522/0 (socket says 172.21.2.101:44522) 2021-02-10T11:32:09.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.906 7fb22cc04700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2281b0870 0x7fb2280f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:09.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.906 7fb22d405700 1 -- 172.21.2.101:0/1078054087 learned_addr learned my addr 172.21.2.101:0/1078054087 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:09.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.907 7fb22cc04700 1 -- 172.21.2.101:0/1078054087 >> v2:172.21.2.108:3300/0 conn(0x7fb2280efe00 msgr2=0x7fb2280f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:09.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.907 7fb22cc04700 1 --2- 172.21.2.101:0/1078054087 >> v2:172.21.2.108:3300/0 conn(0x7fb2280efe00 0x7fb2280f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:09.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.907 7fb22cc04700 1 -- 172.21.2.101:0/1078054087 >> v1:172.21.2.108:6789/0 conn(0x7fb2281b0470 legacy=0x7fb2280ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:09.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.907 7fb22cc04700 1 -- 172.21.2.101:0/1078054087 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2280e52e0 con 0x7fb2281b0870 2021-02-10T11:32:09.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.907 7fb22dc06700 1 --2- 172.21.2.101:0/1078054087 >> v2:172.21.2.108:3300/0 conn(0x7fb2280efe00 0x7fb2280f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:32:09.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.907 7fb22cc04700 1 --2- 172.21.2.101:0/1078054087 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2281b0870 0x7fb2280f20e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fb224006650 tx=0x7fb224009a80).ready entity=mon.2 client_cookie=befa9a4f716a1a91 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:09.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.908 7fb21bfff700 1 -- 172.21.2.101:0/1078054087 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb2240009b0 con 0x7fb2281b0870 2021-02-10T11:32:09.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.908 7fb21bfff700 1 -- 172.21.2.101:0/1078054087 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22400c0c0 con 0x7fb2281b0870 2021-02-10T11:32:09.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.908 7fb21bfff700 1 -- 172.21.2.101:0/1078054087 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb22400c510 con 0x7fb2281b0870 2021-02-10T11:32:09.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.908 7fb22f5e9700 1 -- 172.21.2.101:0/1078054087 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2281b0870 msgr2=0x7fb2280f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:09.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.908 7fb22f5e9700 1 --2- 172.21.2.101:0/1078054087 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2281b0870 0x7fb2280f20e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fb224006650 tx=0x7fb224009a80).stop 2021-02-10T11:32:09.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.909 7fb22f5e9700 1 -- 172.21.2.101:0/1078054087 shutdown_connections 2021-02-10T11:32:09.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.909 7fb22f5e9700 1 --2- 172.21.2.101:0/1078054087 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2281b0870 0x7fb2280f20e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:09.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.909 7fb22f5e9700 1 --2- 172.21.2.101:0/1078054087 >> v2:172.21.2.108:3300/0 conn(0x7fb2280efe00 0x7fb2280f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:09.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.909 7fb22f5e9700 1 -- 172.21.2.101:0/1078054087 >> 172.21.2.101:0/1078054087 conn(0x7fb2280ce3e0 msgr2=0x7fb2280e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:09.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.909 7fb22f5e9700 1 -- 172.21.2.101:0/1078054087 shutdown_connections 2021-02-10T11:32:09.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.909 7fb22f5e9700 1 -- 172.21.2.101:0/1078054087 wait complete. 2021-02-10T11:32:09.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.923 7fb22f5e9700 1 Processor -- start 2021-02-10T11:32:09.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.923 7fb22f5e9700 1 -- start start 2021-02-10T11:32:09.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.924 7fb22f5e9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb228190920 0x7fb2280b81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:09.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.924 7fb22f5e9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2280b87c0 0x7fb2280b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:09.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.924 7fb22f5e9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2280b8bc0 0x7fb2280b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:09.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.924 7fb22f5e9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb228190d20 con 0x7fb228190920 2021-02-10T11:32:09.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.924 7fb22f5e9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2280b5ad0 con 0x7fb2280b87c0 2021-02-10T11:32:09.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.924 7fb22f5e9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2280b7d00 con 0x7fb2280b8bc0 2021-02-10T11:32:09.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.924 7fb22d405700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb228190920 0x7fb2280b81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:09.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.924 7fb22d405700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb228190920 0x7fb2280b81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50624/0 (socket says 172.21.2.101:50624) 2021-02-10T11:32:09.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.924 7fb22d405700 1 -- 172.21.2.101:0/2391409061 learned_addr learned my addr 172.21.2.101:0/2391409061 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:09.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.924 7fb22d405700 1 -- 172.21.2.101:0/2391409061 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2280b8bc0 msgr2=0x7fb2280b76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:09.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.924 7fb22d405700 1 --2- 172.21.2.101:0/2391409061 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2280b8bc0 0x7fb2280b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:09.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.924 7fb22d405700 1 -- 172.21.2.101:0/2391409061 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2280b87c0 msgr2=0x7fb2280b70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:09.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.924 7fb22d405700 1 --2- 172.21.2.101:0/2391409061 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2280b87c0 0x7fb2280b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:09.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.924 7fb22d405700 1 -- 172.21.2.101:0/2391409061 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2280a4590 con 0x7fb228190920 2021-02-10T11:32:09.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.925 7fb22d405700 1 --2- 172.21.2.101:0/2391409061 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb228190920 0x7fb2280b81a0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7fb21c007650 tx=0x7fb21c00a900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:09.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.925 7fb219ffb700 1 -- 172.21.2.101:0/2391409061 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb21c00acf0 con 0x7fb228190920 2021-02-10T11:32:09.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.925 7fb219ffb700 1 -- 172.21.2.101:0/2391409061 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21c004140 con 0x7fb228190920 2021-02-10T11:32:09.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.925 7fb219ffb700 1 -- 172.21.2.101:0/2391409061 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb21c0045e0 con 0x7fb228190920 2021-02-10T11:32:09.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.925 7fb22f5e9700 1 -- 172.21.2.101:0/2391409061 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2280a4590 con 0x7fb228190920 2021-02-10T11:32:09.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.925 7fb22f5e9700 1 -- 172.21.2.101:0/2391409061 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2280a4b20 con 0x7fb228190920 2021-02-10T11:32:09.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.925 7fb219ffb700 1 -- 172.21.2.101:0/2391409061 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb21c004320 con 0x7fb228190920 2021-02-10T11:32:09.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.925 7fb219ffb700 1 --2- 172.21.2.101:0/2391409061 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2080186c0 0x7fb20801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:09.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.926 7fb219ffb700 1 -- 172.21.2.101:0/2391409061 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb21c0323b0 con 0x7fb228190920 2021-02-10T11:32:09.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.926 7fb22cc04700 1 --2- 172.21.2.101:0/2391409061 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2080186c0 0x7fb20801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:09.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.926 7fb22cc04700 1 --2- 172.21.2.101:0/2391409061 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2080186c0 0x7fb20801abc0 crc :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:09.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.955 7fb22f5e9700 1 -- 172.21.2.101:0/2391409061 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb228105ae0 con 0x7fb228190920 2021-02-10T11:32:09.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:09.957 7fb219ffb700 1 -- 172.21.2.101:0/2391409061 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb21c000e20 con 0x7fb228190920 2021-02-10T11:32:10.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:10.106 7fb22f5e9700 1 -- 172.21.2.101:0/2391409061 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb2280f0b50 con 0x7fb228190920 2021-02-10T11:32:10.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:10.110 7fb219ffb700 1 -- 172.21.2.101:0/2391409061 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb21c01f490 con 0x7fb228190920 2021-02-10T11:32:10.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:10.115 7fb22f5e9700 1 -- 172.21.2.101:0/2391409061 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2080186c0 msgr2=0x7fb20801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:10.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:10.115 7fb22f5e9700 1 --2- 172.21.2.101:0/2391409061 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2080186c0 0x7fb20801abc0 crc :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:10.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:10.115 7fb22f5e9700 1 -- 172.21.2.101:0/2391409061 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb228190920 msgr2=0x7fb2280b81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:10.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:10.115 7fb22f5e9700 1 --2- 172.21.2.101:0/2391409061 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb228190920 0x7fb2280b81a0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7fb21c007650 tx=0x7fb21c00a900).stop 2021-02-10T11:32:10.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:10.115 7fb22f5e9700 1 -- 172.21.2.101:0/2391409061 shutdown_connections 2021-02-10T11:32:10.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:10.115 7fb22f5e9700 1 --2- 172.21.2.101:0/2391409061 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2080186c0 0x7fb20801abc0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:10.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:10.115 7fb22f5e9700 1 --2- 172.21.2.101:0/2391409061 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2280b8bc0 0x7fb2280b76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:10.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:10.116 7fb22f5e9700 1 --2- 172.21.2.101:0/2391409061 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2280b87c0 0x7fb2280b70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:10.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:10.116 7fb22f5e9700 1 --2- 172.21.2.101:0/2391409061 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb228190920 0x7fb2280b81a0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:10.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:10.116 7fb22f5e9700 1 -- 172.21.2.101:0/2391409061 >> 172.21.2.101:0/2391409061 conn(0x7fb22808ffe0 msgr2=0x7fb2280bf050 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:10.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:10.116 7fb22f5e9700 1 -- 172.21.2.101:0/2391409061 shutdown_connections 2021-02-10T11:32:10.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:10.116 7fb22f5e9700 1 -- 172.21.2.101:0/2391409061 wait complete. 2021-02-10T11:32:10.135 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:32:10.135 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:32:10.480 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:32:10.825 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:32:10.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:10.826 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:32:10.826 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:32:11.161 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2021-02-10T11:32:11.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:32:11.162 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-02-10T11:32:11.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:11.162 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:32:11.162 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:32:11.497 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:32:11.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:32:11.498 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:32:11.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:11.498 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:32:11.498 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:32:11.833 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:32:11.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:32:11.833 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:32:11.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:11.833 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:32:11.833 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:32:12.167 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:32:12.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:32:12.167 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:32:12.167 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json-pretty 2021-02-10T11:32:12.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.567 7f6708ffe700 1 Processor -- start 2021-02-10T11:32:12.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.567 7f6708ffe700 1 -- start start 2021-02-10T11:32:12.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.568 7f6708ffe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67040e5070 0x7f67041b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:12.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.568 7f6708ffe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f67040e5470 0x7f670407d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:12.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.568 7f6708ffe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f67041b3e40 0x7f670407dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:12.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.568 7f6708ffe700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f670407ed80 con 0x7f67040e5470 2021-02-10T11:32:12.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.568 7f6708ffe700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f670407e2d0 con 0x7f67041b3e40 2021-02-10T11:32:12.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.568 7f6708ffe700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670407e4c0 con 0x7f67040e5070 2021-02-10T11:32:12.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.568 7f670259c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f67040e5470 0x7f670407d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:12.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.568 7f670259c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f67040e5470 0x7f670407d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50632/0 (socket says 172.21.2.101:50632) 2021-02-10T11:32:12.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.568 7f670259c700 1 -- 172.21.2.101:0/1336937817 learned_addr learned my addr 172.21.2.101:0/1336937817 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:12.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.568 7f670259c700 1 -- 172.21.2.101:0/1336937817 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67040e5070 msgr2=0x7f67041b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:12.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.568 7f670259c700 1 --2- 172.21.2.101:0/1336937817 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67040e5070 0x7f67041b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:12.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.568 7f670259c700 1 -- 172.21.2.101:0/1336937817 >> v2:172.21.2.108:3300/0 conn(0x7f67041b3e40 msgr2=0x7f670407dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:12.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.568 7f670259c700 1 --2- 172.21.2.101:0/1336937817 >> v2:172.21.2.108:3300/0 conn(0x7f67041b3e40 0x7f670407dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:12.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.568 7f670259c700 1 -- 172.21.2.101:0/1336937817 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670407e6e0 con 0x7f67040e5470 2021-02-10T11:32:12.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.569 7f670259c700 1 --2- 172.21.2.101:0/1336937817 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67040e5470 0x7f670407d690 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0x7f66ec006700 tx=0x7f66ec009b80).ready entity=mon.0 client_cookie=5bfa62abb6dfeeaa server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:12.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.569 7f6701d9b700 1 -- 172.21.2.101:0/1336937817 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f66ec000c90 con 0x7f67040e5470 2021-02-10T11:32:12.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.569 7f6701d9b700 1 -- 172.21.2.101:0/1336937817 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ec00c0c0 con 0x7f67040e5470 2021-02-10T11:32:12.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.569 7f6701d9b700 1 -- 172.21.2.101:0/1336937817 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f66ec00c520 con 0x7f67040e5470 2021-02-10T11:32:12.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.569 7f6708ffe700 1 -- 172.21.2.101:0/1336937817 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67040e5470 msgr2=0x7f670407d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:12.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.569 7f6708ffe700 1 --2- 172.21.2.101:0/1336937817 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67040e5470 0x7f670407d690 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0x7f66ec006700 tx=0x7f66ec009b80).stop 2021-02-10T11:32:12.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.570 7f6708ffe700 1 -- 172.21.2.101:0/1336937817 shutdown_connections 2021-02-10T11:32:12.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.570 7f6708ffe700 1 --2- 172.21.2.101:0/1336937817 >> v2:172.21.2.108:3300/0 conn(0x7f67041b3e40 0x7f670407dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:12.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.570 7f6708ffe700 1 --2- 172.21.2.101:0/1336937817 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67040e5470 0x7f670407d690 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:12.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.570 7f6708ffe700 1 --2- 172.21.2.101:0/1336937817 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67040e5070 0x7f67041b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:12.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.570 7f6708ffe700 1 -- 172.21.2.101:0/1336937817 >> 172.21.2.101:0/1336937817 conn(0x7f67040ce690 msgr2=0x7f67040e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:12.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.570 7f6708ffe700 1 -- 172.21.2.101:0/1336937817 shutdown_connections 2021-02-10T11:32:12.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.570 7f6708ffe700 1 -- 172.21.2.101:0/1336937817 wait complete. 2021-02-10T11:32:12.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.585 7f6708ffe700 1 Processor -- start 2021-02-10T11:32:12.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.585 7f6708ffe700 1 -- start start 2021-02-10T11:32:12.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.585 7f6708ffe700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6704192960 0x7f6704192d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:12.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.585 7f6708ffe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67040ad110 0x7f67040ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:12.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.585 7f6708ffe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67040adb30 0x7f670408e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:12.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.585 7f6708ffe700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67041a03e0 con 0x7f67040ad110 2021-02-10T11:32:12.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.585 7f6708ffe700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6704193380 con 0x7f6704192960 2021-02-10T11:32:12.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.585 7f6708ffe700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670408ea80 con 0x7f67040adb30 2021-02-10T11:32:12.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.585 7f670259c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67040ad110 0x7f67040ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:12.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.585 7f670259c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67040ad110 0x7f67040ad510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50638/0 (socket says 172.21.2.101:50638) 2021-02-10T11:32:12.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.585 7f670259c700 1 -- 172.21.2.101:0/4264703044 learned_addr learned my addr 172.21.2.101:0/4264703044 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:12.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.586 7f670259c700 1 -- 172.21.2.101:0/4264703044 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67040adb30 msgr2=0x7f670408e460 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:12.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.586 7f670259c700 1 --2- 172.21.2.101:0/4264703044 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67040adb30 0x7f670408e460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:12.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.586 7f670259c700 1 -- 172.21.2.101:0/4264703044 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6704192960 msgr2=0x7f6704192d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:12.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.586 7f670259c700 1 --2- 172.21.2.101:0/4264703044 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6704192960 0x7f6704192d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:12.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.586 7f670259c700 1 -- 172.21.2.101:0/4264703044 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670408eca0 con 0x7f67040ad110 2021-02-10T11:32:12.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.586 7f670259c700 1 --2- 172.21.2.101:0/4264703044 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67040ad110 0x7f67040ad510 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7f66ec006ba0 tx=0x7f66ec00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:12.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.586 7f66fb7fe700 1 -- 172.21.2.101:0/4264703044 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f66ec00bd80 con 0x7f67040ad110 2021-02-10T11:32:12.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.586 7f66fb7fe700 1 -- 172.21.2.101:0/4264703044 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ec002030 con 0x7f67040ad110 2021-02-10T11:32:12.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.586 7f6708ffe700 1 -- 172.21.2.101:0/4264703044 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670408eca0 con 0x7f67040ad110 2021-02-10T11:32:12.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.586 7f66fb7fe700 1 -- 172.21.2.101:0/4264703044 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f66ec002510 con 0x7f67040ad110 2021-02-10T11:32:12.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.586 7f6708ffe700 1 -- 172.21.2.101:0/4264703044 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670408eca0 con 0x7f67040ad110 2021-02-10T11:32:12.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.587 7f66fb7fe700 1 -- 172.21.2.101:0/4264703044 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f66ec000c90 con 0x7f67040ad110 2021-02-10T11:32:12.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.587 7f66fb7fe700 1 --2- 172.21.2.101:0/4264703044 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66e80186c0 0x7f66e801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:12.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.587 7f66fb7fe700 1 -- 172.21.2.101:0/4264703044 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f66ec032650 con 0x7f67040ad110 2021-02-10T11:32:12.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.587 7f6702d9d700 1 --2- 172.21.2.101:0/4264703044 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66e80186c0 0x7f66e801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:12.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.587 7f6702d9d700 1 --2- 172.21.2.101:0/4264703044 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66e80186c0 0x7f66e801abc0 crc :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:12.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.618 7f6708ffe700 1 -- 172.21.2.101:0/4264703044 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6704107120 con 0x7f67040ad110 2021-02-10T11:32:12.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.620 7f66fb7fe700 1 -- 172.21.2.101:0/4264703044 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f66ec01f510 con 0x7f67040ad110 2021-02-10T11:32:12.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.744 7f6708ffe700 1 -- 172.21.2.101:0/4264703044 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6704107120 con 0x7f67040ad110 2021-02-10T11:32:12.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.745 7f66fb7fe700 1 -- 172.21.2.101:0/4264703044 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v38) v1 ==== 81+0+30360 (secure 0 0 0) 0x7f66ec000c90 con 0x7f67040ad110 2021-02-10T11:32:12.756 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:32:12.756 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:32:12.756 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 38, 2021-02-10T11:32:12.756 INFO:teuthology.orchestra.run.gibba001.stdout: "fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:32:12.757 INFO:teuthology.orchestra.run.gibba001.stdout: "created": "2021-02-10 11:12:36.788366", 2021-02-10T11:32:12.757 INFO:teuthology.orchestra.run.gibba001.stdout: "modified": "2021-02-10 11:31:32.943916", 2021-02-10T11:32:12.757 INFO:teuthology.orchestra.run.gibba001.stdout: "last_up_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:32:12.757 INFO:teuthology.orchestra.run.gibba001.stdout: "last_in_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:32:12.757 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-02-10T11:32:12.757 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_num": 5799936, 2021-02-10T11:32:12.757 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_set": [ 2021-02-10T11:32:12.757 INFO:teuthology.orchestra.run.gibba001.stdout: "pglog_hardlimit", 2021-02-10T11:32:12.757 INFO:teuthology.orchestra.run.gibba001.stdout: "purged_snapdirs", 2021-02-10T11:32:12.757 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_deletes", 2021-02-10T11:32:12.757 INFO:teuthology.orchestra.run.gibba001.stdout: "sortbitwise" 2021-02-10T11:32:12.757 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:32:12.757 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_version": 4, 2021-02-10T11:32:12.757 INFO:teuthology.orchestra.run.gibba001.stdout: "full_ratio": 0.94999998807907104, 2021-02-10T11:32:12.758 INFO:teuthology.orchestra.run.gibba001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-02-10T11:32:12.758 INFO:teuthology.orchestra.run.gibba001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-02-10T11:32:12.758 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_snapshot": "", 2021-02-10T11:32:12.758 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_max": 7, 2021-02-10T11:32:12.758 INFO:teuthology.orchestra.run.gibba001.stdout: "max_osd": 8, 2021-02-10T11:32:12.758 INFO:teuthology.orchestra.run.gibba001.stdout: "require_min_compat_client": "jewel", 2021-02-10T11:32:12.758 INFO:teuthology.orchestra.run.gibba001.stdout: "min_compat_client": "jewel", 2021-02-10T11:32:12.758 INFO:teuthology.orchestra.run.gibba001.stdout: "require_osd_release": "nautilus", 2021-02-10T11:32:12.758 INFO:teuthology.orchestra.run.gibba001.stdout: "pools": [ 2021-02-10T11:32:12.758 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.758 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 1, 2021-02-10T11:32:12.759 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "rbd", 2021-02-10T11:32:12.759 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:13:08.037472", 2021-02-10T11:32:12.759 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:32:12.759 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:32:12.759 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:32:12.759 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:32:12.759 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:32:12.759 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:32:12.759 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:32:12.759 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:32:12.759 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:32:12.759 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:32:12.759 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:32:12.760 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:32:12.760 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:32:12.760 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:32:12.760 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:32:12.760 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:32:12.760 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:32:12.760 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:32:12.760 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:32:12.760 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:32:12.760 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.760 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "16", 2021-02-10T11:32:12.760 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:32:12.760 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:32:12.760 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:32:12.761 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:32:12.761 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:32:12.761 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:32:12.761 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:32:12.761 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:32:12.761 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:32:12.761 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:32:12.761 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:32:12.761 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:32:12.761 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:32:12.761 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:32:12.761 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:32:12.761 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:32:12.761 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:32:12.762 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:32:12.762 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:32:12.762 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:32:12.762 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:32:12.762 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:32:12.762 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:32:12.762 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:32:12.762 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:32:12.762 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:32:12.762 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.762 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:32:12.762 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:32:12.762 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:32:12.763 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:32:12.763 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:32:12.763 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:32:12.763 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:32:12.763 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:32:12.763 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:32:12.763 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:32:12.763 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:32:12.763 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:32:12.763 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:32:12.763 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd": {} 2021-02-10T11:32:12.763 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.763 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.763 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.764 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 6, 2021-02-10T11:32:12.764 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_metadata", 2021-02-10T11:32:12.764 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:31:29.865604", 2021-02-10T11:32:12.764 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:32:12.764 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:32:12.764 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:32:12.764 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:32:12.764 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:32:12.764 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:32:12.764 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:32:12.764 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:32:12.764 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:32:12.764 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:32:12.764 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:32:12.765 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:32:12.765 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:32:12.765 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:32:12.765 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:32:12.765 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:32:12.765 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:32:12.765 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:32:12.765 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:32:12.765 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:32:12.765 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.765 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "37", 2021-02-10T11:32:12.765 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:32:12.765 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:32:12.765 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:32:12.766 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:32:12.766 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:32:12.766 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:32:12.766 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:32:12.766 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:32:12.766 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:32:12.766 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:32:12.766 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:32:12.766 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:32:12.766 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:32:12.766 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:32:12.766 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:32:12.766 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:32:12.766 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:32:12.767 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:32:12.767 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:32:12.767 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:32:12.767 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:32:12.767 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:32:12.767 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:32:12.767 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:32:12.767 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:32:12.767 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:32:12.767 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.767 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:32:12.767 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:32:12.767 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:32:12.767 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:32:12.768 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:32:12.768 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:32:12.768 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:32:12.768 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:32:12.768 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:32:12.768 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:32:12.768 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:32:12.768 INFO:teuthology.orchestra.run.gibba001.stdout: "options": { 2021-02-10T11:32:12.768 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_bias": 4, 2021-02-10T11:32:12.768 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_min": 16, 2021-02-10T11:32:12.768 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_priority": 5 2021-02-10T11:32:12.768 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.768 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:32:12.769 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:32:12.769 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": "cephfs" 2021-02-10T11:32:12.769 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.769 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.769 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.769 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.769 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 7, 2021-02-10T11:32:12.769 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_data", 2021-02-10T11:32:12.769 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:31:30.939202", 2021-02-10T11:32:12.769 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:32:12.769 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:32:12.769 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:32:12.769 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:32:12.769 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:32:12.770 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:32:12.770 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:32:12.770 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:32:12.770 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:32:12.770 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:32:12.770 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:32:12.770 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:32:12.770 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:32:12.770 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:32:12.770 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:32:12.770 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:32:12.770 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:32:12.770 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:32:12.770 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:32:12.771 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:32:12.771 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.771 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "38", 2021-02-10T11:32:12.771 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:32:12.771 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:32:12.771 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:32:12.771 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:32:12.771 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:32:12.771 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:32:12.771 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:32:12.771 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:32:12.771 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:32:12.771 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:32:12.771 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:32:12.772 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:32:12.772 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:32:12.772 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:32:12.772 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:32:12.772 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:32:12.772 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:32:12.772 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:32:12.772 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:32:12.772 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:32:12.772 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:32:12.772 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:32:12.772 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:32:12.772 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:32:12.772 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:32:12.773 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:32:12.773 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.773 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:32:12.773 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:32:12.773 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:32:12.773 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:32:12.773 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:32:12.773 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:32:12.773 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:32:12.773 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:32:12.773 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:32:12.773 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:32:12.773 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:32:12.773 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:32:12.774 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:32:12.774 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:32:12.774 INFO:teuthology.orchestra.run.gibba001.stdout: "data": "cephfs" 2021-02-10T11:32:12.774 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.774 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.774 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.774 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:32:12.774 INFO:teuthology.orchestra.run.gibba001.stdout: "osds": [ 2021-02-10T11:32:12.774 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.774 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:32:12.774 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5", 2021-02-10T11:32:12.774 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:32:12.774 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:32:12.775 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:32:12.775 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:32:12.775 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:32:12.775 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:32:12.775 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:32:12.775 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 36, 2021-02-10T11:32:12.775 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:32:12.775 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:32:12.775 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:32:12.775 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.775 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.775 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.775 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6802", 2021-02-10T11:32:12.776 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:32:12.776 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.776 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.776 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.776 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6805", 2021-02-10T11:32:12.776 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:32:12.776 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.776 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.776 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.776 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:32:12.776 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.776 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.776 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.776 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6807", 2021-02-10T11:32:12.777 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:32:12.777 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.777 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.777 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.777 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6809", 2021-02-10T11:32:12.777 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:32:12.777 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.777 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.777 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.777 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:32:12.777 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.777 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.778 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.778 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6817", 2021-02-10T11:32:12.778 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:32:12.778 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.778 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.778 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.778 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6820", 2021-02-10T11:32:12.778 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:32:12.778 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.778 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.778 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.779 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:32:12.779 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.779 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.779 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.779 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6812", 2021-02-10T11:32:12.779 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:32:12.779 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.779 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.779 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.779 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6814", 2021-02-10T11:32:12.779 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:32:12.779 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.779 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.780 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.780 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6805/25093", 2021-02-10T11:32:12.780 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6809/25093", 2021-02-10T11:32:12.780 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6820/25093", 2021-02-10T11:32:12.780 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6814/25093", 2021-02-10T11:32:12.780 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:32:12.780 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:32:12.780 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:32:12.780 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.780 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.780 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.780 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:32:12.781 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "ddc04ce3-b313-4707-8270-585391e38ecc", 2021-02-10T11:32:12.781 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:32:12.781 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:32:12.781 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:32:12.781 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:32:12.781 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:32:12.781 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:32:12.781 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:32:12.781 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 35, 2021-02-10T11:32:12.781 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:32:12.781 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:32:12.781 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:32:12.782 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.782 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.782 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.782 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6803", 2021-02-10T11:32:12.782 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:32:12.782 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.782 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.782 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.782 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6804", 2021-02-10T11:32:12.782 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:32:12.782 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.782 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.783 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.783 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:32:12.783 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.783 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.783 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.783 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6806", 2021-02-10T11:32:12.783 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:32:12.783 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.783 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.783 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.783 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6808", 2021-02-10T11:32:12.783 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:32:12.784 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.784 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.784 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.784 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:32:12.784 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.784 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.784 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.784 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6818", 2021-02-10T11:32:12.784 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:32:12.784 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.784 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.784 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.784 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6821", 2021-02-10T11:32:12.784 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:32:12.785 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.785 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.785 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.785 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:32:12.785 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.785 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.785 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.785 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6810", 2021-02-10T11:32:12.785 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:32:12.785 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.785 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.785 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.785 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6815", 2021-02-10T11:32:12.786 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:32:12.786 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.786 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.786 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.786 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6804/25092", 2021-02-10T11:32:12.786 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6808/25092", 2021-02-10T11:32:12.786 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6821/25092", 2021-02-10T11:32:12.786 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6815/25092", 2021-02-10T11:32:12.786 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:32:12.786 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:32:12.786 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:32:12.786 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.786 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.787 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.787 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:32:12.787 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3", 2021-02-10T11:32:12.787 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:32:12.787 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:32:12.787 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:32:12.787 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:32:12.787 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:32:12.787 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:32:12.787 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:32:12.787 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 27, 2021-02-10T11:32:12.787 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:32:12.787 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:32:12.788 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:32:12.788 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.788 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.788 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.788 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6811", 2021-02-10T11:32:12.788 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:32:12.788 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.788 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.788 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.788 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6813", 2021-02-10T11:32:12.788 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:32:12.788 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.788 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.788 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.788 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:32:12.789 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.789 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.789 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.789 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6816", 2021-02-10T11:32:12.789 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:32:12.789 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.789 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.789 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.789 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6819", 2021-02-10T11:32:12.789 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:32:12.789 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.789 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.789 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.790 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:32:12.790 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.790 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.790 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.790 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6825", 2021-02-10T11:32:12.790 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:32:12.790 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.790 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.790 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.790 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6827", 2021-02-10T11:32:12.790 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:32:12.790 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.790 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.790 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.791 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:32:12.791 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.791 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.791 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.791 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6822", 2021-02-10T11:32:12.791 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:32:12.791 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.791 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.791 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.791 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6823", 2021-02-10T11:32:12.791 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:32:12.791 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.791 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.791 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.792 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6813/25094", 2021-02-10T11:32:12.792 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6819/25094", 2021-02-10T11:32:12.792 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6827/25094", 2021-02-10T11:32:12.792 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6823/25094", 2021-02-10T11:32:12.792 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:32:12.792 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:32:12.792 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:32:12.792 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.792 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.792 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.792 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:32:12.792 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "0d9ef9b1-98aa-4d37-aa16-6305c7d46d52", 2021-02-10T11:32:12.792 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:32:12.793 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:32:12.793 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:32:12.793 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:32:12.793 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:32:12.793 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:32:12.793 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:32:12.793 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 36, 2021-02-10T11:32:12.793 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:32:12.793 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:32:12.793 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:32:12.793 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.793 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.793 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.793 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6824", 2021-02-10T11:32:12.794 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:32:12.794 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.794 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.794 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.794 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6826", 2021-02-10T11:32:12.794 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:32:12.794 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.794 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.794 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.794 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:32:12.794 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.794 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.794 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.795 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6828", 2021-02-10T11:32:12.795 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:32:12.795 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.795 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.795 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.795 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6829", 2021-02-10T11:32:12.795 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:32:12.795 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.795 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.795 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.795 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:32:12.795 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.795 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.795 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.796 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6832", 2021-02-10T11:32:12.796 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:32:12.796 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.796 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.796 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.796 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6833", 2021-02-10T11:32:12.796 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:32:12.796 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.796 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.796 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.796 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:32:12.796 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.796 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.797 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.797 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6830", 2021-02-10T11:32:12.797 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:32:12.797 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.797 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.797 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.797 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6831", 2021-02-10T11:32:12.797 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:32:12.797 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.797 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.797 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.797 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6826/25095", 2021-02-10T11:32:12.797 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6829/25095", 2021-02-10T11:32:12.797 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6833/25095", 2021-02-10T11:32:12.798 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6831/25095", 2021-02-10T11:32:12.798 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:32:12.798 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:32:12.798 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:32:12.798 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.798 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.798 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.798 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:32:12.798 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "36742c21-bfeb-4e11-bce9-8e745378b605", 2021-02-10T11:32:12.798 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:32:12.798 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:32:12.798 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:32:12.798 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:32:12.799 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:32:12.799 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:32:12.799 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:32:12.799 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 36, 2021-02-10T11:32:12.799 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:32:12.799 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:32:12.799 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:32:12.799 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.799 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.799 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.799 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6816", 2021-02-10T11:32:12.799 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:32:12.799 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.800 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.800 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.800 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6818", 2021-02-10T11:32:12.800 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:32:12.800 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.800 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.800 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.800 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:32:12.800 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.800 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.800 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.800 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6820", 2021-02-10T11:32:12.800 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:32:12.800 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.801 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.801 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.801 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6822", 2021-02-10T11:32:12.801 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:32:12.801 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.801 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.801 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.801 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:32:12.801 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.801 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.801 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.801 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6828", 2021-02-10T11:32:12.801 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:32:12.801 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.802 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.802 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.802 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6830", 2021-02-10T11:32:12.802 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:32:12.802 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.802 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.802 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.802 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:32:12.802 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.802 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.802 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.802 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6824", 2021-02-10T11:32:12.802 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:32:12.802 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.803 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.803 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.803 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6826", 2021-02-10T11:32:12.803 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:32:12.803 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.803 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.803 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.803 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6818/36380", 2021-02-10T11:32:12.803 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6822/36380", 2021-02-10T11:32:12.803 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6830/36380", 2021-02-10T11:32:12.803 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6826/36380", 2021-02-10T11:32:12.803 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:32:12.803 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:32:12.803 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:32:12.804 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.804 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.804 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.804 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:32:12.804 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "88e694e8-fe98-4d1b-88c8-8c9907ece60b", 2021-02-10T11:32:12.804 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:32:12.804 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:32:12.804 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:32:12.804 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:32:12.804 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:32:12.804 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:32:12.804 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:32:12.804 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 36, 2021-02-10T11:32:12.805 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:32:12.805 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:32:12.805 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:32:12.805 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.805 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.805 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.805 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6808", 2021-02-10T11:32:12.805 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:32:12.805 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.805 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.805 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.805 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6809", 2021-02-10T11:32:12.805 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:32:12.805 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.806 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.806 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.806 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:32:12.806 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.806 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.806 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.806 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6810", 2021-02-10T11:32:12.806 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:32:12.806 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.806 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.806 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.806 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6811", 2021-02-10T11:32:12.806 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:32:12.807 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.807 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.807 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.807 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:32:12.807 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.807 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.807 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.807 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6814", 2021-02-10T11:32:12.807 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:32:12.807 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.807 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.807 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.808 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6815", 2021-02-10T11:32:12.808 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:32:12.808 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.808 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.808 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.808 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:32:12.808 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.808 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.808 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.808 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6812", 2021-02-10T11:32:12.808 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:32:12.808 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.808 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.809 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.809 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6813", 2021-02-10T11:32:12.809 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:32:12.809 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.809 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.809 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.809 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6809/36383", 2021-02-10T11:32:12.809 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6811/36383", 2021-02-10T11:32:12.809 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6815/36383", 2021-02-10T11:32:12.809 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6813/36383", 2021-02-10T11:32:12.809 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:32:12.809 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:32:12.809 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:32:12.810 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.810 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.810 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.810 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:32:12.810 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "87a13c6c-cf85-4aaf-916b-e73913484a49", 2021-02-10T11:32:12.810 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:32:12.810 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:32:12.810 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:32:12.810 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:32:12.810 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:32:12.810 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:32:12.810 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:32:12.810 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 36, 2021-02-10T11:32:12.811 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:32:12.811 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:32:12.811 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:32:12.811 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.811 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.811 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.811 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6800", 2021-02-10T11:32:12.811 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:32:12.811 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.811 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.811 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.811 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6801", 2021-02-10T11:32:12.811 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:32:12.811 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.812 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.812 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.812 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:32:12.812 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.812 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.812 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.812 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6802", 2021-02-10T11:32:12.812 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:32:12.812 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.812 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.812 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.812 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6803", 2021-02-10T11:32:12.812 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:32:12.812 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.813 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.813 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.813 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:32:12.813 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.813 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.813 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.813 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6806", 2021-02-10T11:32:12.813 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:32:12.813 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.813 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.813 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.813 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6807", 2021-02-10T11:32:12.813 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:32:12.814 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.814 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.814 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.814 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:32:12.814 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.814 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.814 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.814 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6804", 2021-02-10T11:32:12.814 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:32:12.814 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.814 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.814 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.814 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6805", 2021-02-10T11:32:12.815 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:32:12.815 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.815 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.815 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.815 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6801/36381", 2021-02-10T11:32:12.815 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6803/36381", 2021-02-10T11:32:12.815 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6807/36381", 2021-02-10T11:32:12.815 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6805/36381", 2021-02-10T11:32:12.815 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:32:12.815 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:32:12.815 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:32:12.815 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.815 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.815 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.816 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:32:12.816 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "7b90823f-854c-4592-a95b-a7fa98f93254", 2021-02-10T11:32:12.816 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:32:12.816 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:32:12.816 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:32:12.816 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:32:12.816 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:32:12.816 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:32:12.816 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:32:12.816 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 36, 2021-02-10T11:32:12.816 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:32:12.816 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:32:12.816 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:32:12.816 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.817 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.817 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.817 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6817", 2021-02-10T11:32:12.817 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:32:12.817 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.817 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.817 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.817 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6819", 2021-02-10T11:32:12.817 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:32:12.817 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.817 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.817 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.817 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:32:12.817 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.818 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.818 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.818 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6821", 2021-02-10T11:32:12.818 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:32:12.818 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.818 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.818 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.818 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6823", 2021-02-10T11:32:12.818 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:32:12.818 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.818 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.818 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.818 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:32:12.818 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.819 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.819 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.819 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6829", 2021-02-10T11:32:12.819 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:32:12.819 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.819 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.819 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.819 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6831", 2021-02-10T11:32:12.819 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:32:12.819 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.819 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.819 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.819 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:32:12.819 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:32:12.820 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.820 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:32:12.820 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6825", 2021-02-10T11:32:12.820 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:32:12.820 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.820 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.820 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:12.820 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6827", 2021-02-10T11:32:12.820 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:32:12.820 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.820 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.820 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.820 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6819/36382", 2021-02-10T11:32:12.820 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6823/36382", 2021-02-10T11:32:12.821 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6831/36382", 2021-02-10T11:32:12.821 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6827/36382", 2021-02-10T11:32:12.821 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:32:12.821 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:32:12.821 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:32:12.821 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:12.821 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.821 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:32:12.821 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_xinfo": [ 2021-02-10T11:32:12.821 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.821 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:32:12.821 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:32:12.821 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:32:12.821 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:32:12.822 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:32:12.822 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:32:12.822 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.822 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.822 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:32:12.822 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:32:12.822 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:32:12.822 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:32:12.822 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:32:12.822 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:32:12.822 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.822 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.822 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:32:12.823 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:32:12.823 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:32:12.823 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:32:12.823 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:32:12.823 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:32:12.823 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.823 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.823 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:32:12.823 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:32:12.823 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:32:12.823 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:32:12.823 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:32:12.823 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:32:12.824 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.824 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.824 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:32:12.824 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:32:12.824 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:32:12.824 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:32:12.824 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:32:12.824 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:32:12.824 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.824 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.824 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:32:12.824 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:32:12.824 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:32:12.824 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:32:12.825 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:32:12.825 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:32:12.825 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.825 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.825 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:32:12.825 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:32:12.825 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:32:12.825 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:32:12.825 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:32:12.825 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:32:12.825 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.825 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:12.825 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:32:12.825 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:32:12.826 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:32:12.826 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:32:12.826 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:32:12.826 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:32:12.826 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.826 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:32:12.826 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap": [], 2021-02-10T11:32:12.826 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap_items": [], 2021-02-10T11:32:12.826 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_temp": [], 2021-02-10T11:32:12.826 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_temp": [], 2021-02-10T11:32:12.826 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklist": {}, 2021-02-10T11:32:12.826 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profiles": { 2021-02-10T11:32:12.826 INFO:teuthology.orchestra.run.gibba001.stdout: "default": { 2021-02-10T11:32:12.827 INFO:teuthology.orchestra.run.gibba001.stdout: "crush-failure-domain": "osd", 2021-02-10T11:32:12.827 INFO:teuthology.orchestra.run.gibba001.stdout: "k": "2", 2021-02-10T11:32:12.827 INFO:teuthology.orchestra.run.gibba001.stdout: "m": "1", 2021-02-10T11:32:12.827 INFO:teuthology.orchestra.run.gibba001.stdout: "plugin": "jerasure", 2021-02-10T11:32:12.827 INFO:teuthology.orchestra.run.gibba001.stdout: "ruleset-failure-domain": "osd", 2021-02-10T11:32:12.827 INFO:teuthology.orchestra.run.gibba001.stdout: "technique": "reed_sol_van" 2021-02-10T11:32:12.827 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:12.827 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:12.827 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps_queue": [], 2021-02-10T11:32:12.827 INFO:teuthology.orchestra.run.gibba001.stdout: "new_removed_snaps": [], 2021-02-10T11:32:12.827 INFO:teuthology.orchestra.run.gibba001.stdout: "new_purged_snaps": [], 2021-02-10T11:32:12.827 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_node_flags": {}, 2021-02-10T11:32:12.827 INFO:teuthology.orchestra.run.gibba001.stdout: "device_class_flags": {} 2021-02-10T11:32:12.828 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:32:12.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.746 7f6708ffe700 1 -- 172.21.2.101:0/4264703044 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66e80186c0 msgr2=0x7f66e801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:12.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.746 7f6708ffe700 1 --2- 172.21.2.101:0/4264703044 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66e80186c0 0x7f66e801abc0 crc :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:12.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.746 7f6708ffe700 1 -- 172.21.2.101:0/4264703044 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67040ad110 msgr2=0x7f67040ad510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:12.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.746 7f6708ffe700 1 --2- 172.21.2.101:0/4264703044 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67040ad110 0x7f67040ad510 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7f66ec006ba0 tx=0x7f66ec00b990).stop 2021-02-10T11:32:12.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.747 7f6708ffe700 1 -- 172.21.2.101:0/4264703044 shutdown_connections 2021-02-10T11:32:12.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.747 7f6708ffe700 1 --2- 172.21.2.101:0/4264703044 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66e80186c0 0x7f66e801abc0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:12.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.747 7f6708ffe700 1 --2- 172.21.2.101:0/4264703044 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f67040adb30 0x7f670408e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:12.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.747 7f6708ffe700 1 --2- 172.21.2.101:0/4264703044 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67040ad110 0x7f67040ad510 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:12.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.747 7f6708ffe700 1 --2- 172.21.2.101:0/4264703044 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6704192960 0x7f6704192d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:12.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.747 7f6708ffe700 1 -- 172.21.2.101:0/4264703044 >> 172.21.2.101:0/4264703044 conn(0x7f67040891f0 msgr2=0x7f67041a0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:12.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.747 7f6708ffe700 1 -- 172.21.2.101:0/4264703044 shutdown_connections 2021-02-10T11:32:12.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:12.747 7f6708ffe700 1 -- 172.21.2.101:0/4264703044 wait complete. 2021-02-10T11:32:12.831 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:32:13.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.229 7f75c2379700 1 Processor -- start 2021-02-10T11:32:13.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.229 7f75c2379700 1 -- start start 2021-02-10T11:32:13.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.229 7f75c2379700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75bc0e5070 0x7f75bc1b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:13.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.229 7f75c2379700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f75bc1b3e40 0x7f75bc080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:13.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.229 7f75c2379700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75bc07ed80 con 0x7f75bc0e5470 2021-02-10T11:32:13.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.229 7f75c2379700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75bc0d1790 con 0x7f75bc1b3e40 2021-02-10T11:32:13.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.229 7f75c2379700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc0d1980 con 0x7f75bc0e5070 2021-02-10T11:32:13.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.229 7f75bb7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f75bc0e5470 0x7f75bc07d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51140/0 (socket says 172.21.2.101:51140) 2021-02-10T11:32:13.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.229 7f75bb7fe700 1 -- 172.21.2.101:0/2476216540 learned_addr learned my addr 172.21.2.101:0/2476216540 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:13.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.229 7f75c0996700 1 --2- 172.21.2.101:0/2476216540 >> v2:172.21.2.101:3300/0 conn(0x7f75bc1b3e40 0x7f75bc080310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:13.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.230 7f75c0996700 1 -- 172.21.2.101:0/2476216540 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75bc0e5070 msgr2=0x7f75bc1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:13.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.230 7f75c0996700 1 --2- 172.21.2.101:0/2476216540 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75bc0e5070 0x7f75bc1b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:13.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.230 7f75c0996700 1 -- 172.21.2.101:0/2476216540 >> v1:172.21.2.101:6789/0 conn(0x7f75bc0e5470 legacy=0x7f75bc07d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:13.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.230 7f75baffd700 1 -- 172.21.2.101:0/2476216540 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3397847321 0 0) 0x7f75b00010b0 con 0x7f75bc0e5470 2021-02-10T11:32:13.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.230 7f75c0996700 1 -- 172.21.2.101:0/2476216540 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75bc0d1ba0 con 0x7f75bc1b3e40 2021-02-10T11:32:13.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.230 7f75c0996700 1 --2- 172.21.2.101:0/2476216540 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75bc1b3e40 0x7f75bc080310 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7f75b4006e30 tx=0x7f75b4009de0).ready entity=mon.0 client_cookie=a5e3f9711c8da504 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:13.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.230 7f75baffd700 1 -- 172.21.2.101:0/2476216540 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f75b400b300 con 0x7f75bc1b3e40 2021-02-10T11:32:13.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.230 7f75baffd700 1 -- 172.21.2.101:0/2476216540 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b400b680 con 0x7f75bc1b3e40 2021-02-10T11:32:13.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.230 7f75baffd700 1 -- 172.21.2.101:0/2476216540 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f75b400bba0 con 0x7f75bc1b3e40 2021-02-10T11:32:13.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.230 7f75c2379700 1 -- 172.21.2.101:0/2476216540 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75bc1b3e40 msgr2=0x7f75bc080310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:13.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.230 7f75c2379700 1 --2- 172.21.2.101:0/2476216540 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75bc1b3e40 0x7f75bc080310 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7f75b4006e30 tx=0x7f75b4009de0).stop 2021-02-10T11:32:13.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.231 7f75c2379700 1 -- 172.21.2.101:0/2476216540 shutdown_connections 2021-02-10T11:32:13.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.231 7f75c2379700 1 --2- 172.21.2.101:0/2476216540 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75bc1b3e40 0x7f75bc080310 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:13.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.231 7f75c2379700 1 --2- 172.21.2.101:0/2476216540 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75bc0e5070 0x7f75bc1b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:13.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.231 7f75c2379700 1 -- 172.21.2.101:0/2476216540 >> 172.21.2.101:0/2476216540 conn(0x7f75bc0ce690 msgr2=0x7f75bc0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:13.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.231 7f75c2379700 1 -- 172.21.2.101:0/2476216540 shutdown_connections 2021-02-10T11:32:13.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.231 7f75c2379700 1 -- 172.21.2.101:0/2476216540 wait complete. 2021-02-10T11:32:13.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.245 7f75c2379700 1 Processor -- start 2021-02-10T11:32:13.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.246 7f75c2379700 1 -- start start 2021-02-10T11:32:13.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.246 7f75c2379700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75bc192960 0x7f75bc192d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:13.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.246 7f75c2379700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75bc0ad110 0x7f75bc0ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:13.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.246 7f75c2379700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75bc0adb30 0x7f75bc08e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:13.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.246 7f75c2379700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc1a03e0 con 0x7f75bc0adb30 2021-02-10T11:32:13.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.246 7f75c2379700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc193380 con 0x7f75bc0ad110 2021-02-10T11:32:13.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.246 7f75c2379700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc08ea80 con 0x7f75bc192960 2021-02-10T11:32:13.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.246 7f75c0996700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75bc0adb30 0x7f75bc08e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:13.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.246 7f75c0996700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75bc0adb30 0x7f75bc08e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50654/0 (socket says 172.21.2.101:50654) 2021-02-10T11:32:13.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.246 7f75c0996700 1 -- 172.21.2.101:0/2571161883 learned_addr learned my addr 172.21.2.101:0/2571161883 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:13.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.246 7f75c0996700 1 -- 172.21.2.101:0/2571161883 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75bc192960 msgr2=0x7f75bc192d60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:13.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.246 7f75c0996700 1 --2- 172.21.2.101:0/2571161883 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75bc192960 0x7f75bc192d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:13.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.246 7f75c0996700 1 -- 172.21.2.101:0/2571161883 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75bc0ad110 msgr2=0x7f75bc0ad510 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:13.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.246 7f75c0996700 1 --2- 172.21.2.101:0/2571161883 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75bc0ad110 0x7f75bc0ad510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:13.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.246 7f75c0996700 1 -- 172.21.2.101:0/2571161883 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75bc08eca0 con 0x7f75bc0adb30 2021-02-10T11:32:13.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.247 7f75c0996700 1 --2- 172.21.2.101:0/2571161883 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75bc0adb30 0x7f75bc08e460 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7f75b400b910 tx=0x7f75b400bcb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:13.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.247 7f75b8ff9700 1 -- 172.21.2.101:0/2571161883 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f75b400bdf0 con 0x7f75bc0adb30 2021-02-10T11:32:13.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.247 7f75b8ff9700 1 -- 172.21.2.101:0/2571161883 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b4005820 con 0x7f75bc0adb30 2021-02-10T11:32:13.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.247 7f75c2379700 1 -- 172.21.2.101:0/2571161883 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75bc08eca0 con 0x7f75bc0adb30 2021-02-10T11:32:13.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.247 7f75c2379700 1 -- 172.21.2.101:0/2571161883 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75bc08f230 con 0x7f75bc0adb30 2021-02-10T11:32:13.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.247 7f75b8ff9700 1 -- 172.21.2.101:0/2571161883 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f75b4005d40 con 0x7f75bc0adb30 2021-02-10T11:32:13.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.247 7f75b8ff9700 1 -- 172.21.2.101:0/2571161883 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f75b4005a00 con 0x7f75bc0adb30 2021-02-10T11:32:13.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.248 7f75b8ff9700 1 --2- 172.21.2.101:0/2571161883 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f759c018730 0x7f759c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:13.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.248 7f75b8ff9700 1 -- 172.21.2.101:0/2571161883 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f75b4032b70 con 0x7f75bc0adb30 2021-02-10T11:32:13.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.248 7f75bbfff700 1 --2- 172.21.2.101:0/2571161883 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f759c018730 0x7f759c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:13.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.248 7f75bbfff700 1 --2- 172.21.2.101:0/2571161883 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f759c018730 0x7f759c01ac10 crc :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:13.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.279 7f75c2379700 1 -- 172.21.2.101:0/2571161883 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75bc107120 con 0x7f75bc0adb30 2021-02-10T11:32:13.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.281 7f75b8ff9700 1 -- 172.21.2.101:0/2571161883 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f75b40093d0 con 0x7f75bc0adb30 2021-02-10T11:32:13.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.409 7f75c2379700 1 -- 172.21.2.101:0/2571161883 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75bc07ed80 con 0x7f75bc0adb30 2021-02-10T11:32:13.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.409 7f75b8ff9700 1 -- 172.21.2.101:0/2571161883 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2096 (secure 0 0 0) 0x7f75b4006dc0 con 0x7f75bc0adb30 2021-02-10T11:32:13.421 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 23 2021-02-10T11:32:13.421 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:32:13.421 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/243293953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":243293953},{"type":"v1","addr":"172.21.2.101:6835","nonce":243293953}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:31:31.941616","modified":"2021-02-10 11:31:33.951741","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7299":{"gid":7299,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/1600862555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":1600862555},{"type":"v1","addr":"172.21.2.108:6833","nonce":1600862555}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-02-10T11:32:13.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.411 7f75c2379700 1 -- 172.21.2.101:0/2571161883 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f759c018730 msgr2=0x7f759c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:13.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.411 7f75c2379700 1 --2- 172.21.2.101:0/2571161883 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f759c018730 0x7f759c01ac10 crc :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:13.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.411 7f75c2379700 1 -- 172.21.2.101:0/2571161883 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75bc0adb30 msgr2=0x7f75bc08e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:13.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.411 7f75c2379700 1 --2- 172.21.2.101:0/2571161883 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75bc0adb30 0x7f75bc08e460 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7f75b400b910 tx=0x7f75b400bcb0).stop 2021-02-10T11:32:13.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.411 7f75c2379700 1 -- 172.21.2.101:0/2571161883 shutdown_connections 2021-02-10T11:32:13.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.411 7f75c2379700 1 --2- 172.21.2.101:0/2571161883 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75bc0adb30 0x7f75bc08e460 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:13.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.411 7f75c2379700 1 --2- 172.21.2.101:0/2571161883 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75bc0ad110 0x7f75bc0ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:13.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.411 7f75c2379700 1 --2- 172.21.2.101:0/2571161883 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f759c018730 0x7f759c01ac10 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:13.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.411 7f75c2379700 1 --2- 172.21.2.101:0/2571161883 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75bc192960 0x7f75bc192d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:13.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.411 7f75c2379700 1 -- 172.21.2.101:0/2571161883 >> 172.21.2.101:0/2571161883 conn(0x7f75bc0891f0 msgr2=0x7f75bc1a0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:13.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.412 7f75c2379700 1 -- 172.21.2.101:0/2571161883 shutdown_connections 2021-02-10T11:32:13.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.412 7f75c2379700 1 -- 172.21.2.101:0/2571161883 wait complete. 2021-02-10T11:32:13.431 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs fail cephfs 2021-02-10T11:32:13.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.829 7f95a8da3700 1 Processor -- start 2021-02-10T11:32:13.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.829 7f95a8da3700 1 -- start start 2021-02-10T11:32:13.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.830 7f95a8da3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95a40e5070 0x7f95a41b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:13.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.830 7f95a8da3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f95a40e5470 0x7f95a407d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:13.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.830 7f95a8da3700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a407ed80 con 0x7f95a41b3e40 2021-02-10T11:32:13.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.830 7f95a8da3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95a4080310 con 0x7f95a40e5470 2021-02-10T11:32:13.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.830 7f95a8da3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a4080500 con 0x7f95a40e5070 2021-02-10T11:32:13.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.830 7f95a2b13700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95a40e5070 0x7f95a41b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:13.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.831 7f95a2312700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f95a40e5470 0x7f95a407d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:13.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.831 7f95a2b13700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95a40e5070 0x7f95a41b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:43390/0 (socket says 172.21.2.101:43390) 2021-02-10T11:32:13.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.831 7f95a2b13700 1 -- 172.21.2.101:0/3279479884 learned_addr learned my addr 172.21.2.101:0/3279479884 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:13.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.831 7f95a2312700 1 -- 172.21.2.101:0/3279479884 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95a40e5070 msgr2=0x7f95a41b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:13.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.831 7f95a2312700 1 --2- 172.21.2.101:0/3279479884 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95a40e5070 0x7f95a41b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:13.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.831 7f95a2312700 1 -- 172.21.2.101:0/3279479884 >> v1:172.21.2.108:6789/0 conn(0x7f95a41b3e40 legacy=0x7f95a407dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:13.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.831 7f95a1b11700 1 -- 172.21.2.101:0/3279479884 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220474995 0 0) 0x7f9594001080 con 0x7f95a41b3e40 2021-02-10T11:32:13.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.831 7f95a2312700 1 -- 172.21.2.101:0/3279479884 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a40806f0 con 0x7f95a40e5470 2021-02-10T11:32:13.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.831 7f95a2b13700 1 --2- 172.21.2.101:0/3279479884 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95a40e5070 0x7f95a41b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:32:13.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.831 7f95a2312700 1 --2- 172.21.2.101:0/3279479884 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f95a40e5470 0x7f95a407d690 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f959c0069d0 tx=0x7f959c009e40).ready entity=mon.1 client_cookie=c716a53ca31d15b1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:13.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.832 7f95a1b11700 1 -- 172.21.2.101:0/3279479884 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f959c00b510 con 0x7f95a40e5470 2021-02-10T11:32:13.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.832 7f95a1b11700 1 -- 172.21.2.101:0/3279479884 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c00b890 con 0x7f95a40e5470 2021-02-10T11:32:13.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.832 7f95a1b11700 1 -- 172.21.2.101:0/3279479884 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f959c00bdb0 con 0x7f95a40e5470 2021-02-10T11:32:13.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.832 7f95a8da3700 1 -- 172.21.2.101:0/3279479884 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f95a40e5470 msgr2=0x7f95a407d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:13.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.832 7f95a8da3700 1 --2- 172.21.2.101:0/3279479884 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f95a40e5470 0x7f95a407d690 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f959c0069d0 tx=0x7f959c009e40).stop 2021-02-10T11:32:13.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.832 7f95a8da3700 1 -- 172.21.2.101:0/3279479884 shutdown_connections 2021-02-10T11:32:13.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.832 7f95a8da3700 1 --2- 172.21.2.101:0/3279479884 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f95a40e5470 0x7f95a407d690 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:13.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.832 7f95a8da3700 1 --2- 172.21.2.101:0/3279479884 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95a40e5070 0x7f95a41b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:13.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.832 7f95a8da3700 1 -- 172.21.2.101:0/3279479884 >> 172.21.2.101:0/3279479884 conn(0x7f95a40ce690 msgr2=0x7f95a40e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:13.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.832 7f95a8da3700 1 -- 172.21.2.101:0/3279479884 shutdown_connections 2021-02-10T11:32:13.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.832 7f95a8da3700 1 -- 172.21.2.101:0/3279479884 wait complete. 2021-02-10T11:32:13.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.847 7f95a8da3700 1 Processor -- start 2021-02-10T11:32:13.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.847 7f95a8da3700 1 -- start start 2021-02-10T11:32:13.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.847 7f95a8da3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95a40b4ad0 0x7f95a40a78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:13.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.847 7f95a8da3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95a40a7ef0 0x7f95a41a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:13.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.847 7f95a8da3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f95a40a82f0 0x7f95a41a73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:13.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.847 7f95a8da3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a40aa3c0 con 0x7f95a40b4ad0 2021-02-10T11:32:13.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.847 7f95a8da3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a40a88c0 con 0x7f95a40a82f0 2021-02-10T11:32:13.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.847 7f95a8da3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a40b4ed0 con 0x7f95a40a7ef0 2021-02-10T11:32:13.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.847 7f95a2b13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95a40b4ad0 0x7f95a40a78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:13.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.847 7f95a2b13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95a40b4ad0 0x7f95a40a78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50668/0 (socket says 172.21.2.101:50668) 2021-02-10T11:32:13.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.847 7f95a2b13700 1 -- 172.21.2.101:0/627634399 learned_addr learned my addr 172.21.2.101:0/627634399 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:13.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.848 7f95a2b13700 1 -- 172.21.2.101:0/627634399 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95a40a7ef0 msgr2=0x7f95a41a6d90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:13.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.848 7f95a2b13700 1 --2- 172.21.2.101:0/627634399 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95a40a7ef0 0x7f95a41a6d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:13.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.848 7f95a2b13700 1 -- 172.21.2.101:0/627634399 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f95a40a82f0 msgr2=0x7f95a41a73b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:13.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.848 7f95a2b13700 1 --2- 172.21.2.101:0/627634399 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f95a40a82f0 0x7f95a41a73b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:13.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.848 7f95a2b13700 1 -- 172.21.2.101:0/627634399 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a41a79d0 con 0x7f95a40b4ad0 2021-02-10T11:32:13.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.848 7f95a2b13700 1 --2- 172.21.2.101:0/627634399 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95a40b4ad0 0x7f95a40a78d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7f9598006ee0 tx=0x7f959800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:13.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.848 7f95937fe700 1 -- 172.21.2.101:0/627634399 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f959800bd50 con 0x7f95a40b4ad0 2021-02-10T11:32:13.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.848 7f95937fe700 1 -- 172.21.2.101:0/627634399 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95980052e0 con 0x7f95a40b4ad0 2021-02-10T11:32:13.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.848 7f95a8da3700 1 -- 172.21.2.101:0/627634399 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a41a79d0 con 0x7f95a40b4ad0 2021-02-10T11:32:13.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.848 7f95937fe700 1 -- 172.21.2.101:0/627634399 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f95980057c0 con 0x7f95a40b4ad0 2021-02-10T11:32:13.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.848 7f95a8da3700 1 -- 172.21.2.101:0/627634399 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a41a79d0 con 0x7f95a40b4ad0 2021-02-10T11:32:13.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.849 7f95937fe700 1 -- 172.21.2.101:0/627634399 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f95980054c0 con 0x7f95a40b4ad0 2021-02-10T11:32:13.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.849 7f95937fe700 1 --2- 172.21.2.101:0/627634399 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9588018730 0x7f958801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:13.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.849 7f95937fe700 1 -- 172.21.2.101:0/627634399 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9598032610 con 0x7f95a40b4ad0 2021-02-10T11:32:13.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.849 7f95a2312700 1 --2- 172.21.2.101:0/627634399 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9588018730 0x7f958801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:13.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.849 7f95a2312700 1 --2- 172.21.2.101:0/627634399 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9588018730 0x7f958801abc0 crc :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:13.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.879 7f95a8da3700 1 -- 172.21.2.101:0/627634399 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a4084a90 con 0x7f95a40b4ad0 2021-02-10T11:32:13.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:13.882 7f95937fe700 1 -- 172.21.2.101:0/627634399 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9598000c90 con 0x7f95a40b4ad0 2021-02-10T11:32:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.009 7f95a8da3700 1 -- 172.21.2.101:0/627634399 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f95a4084a90 con 0x7f95a40b4ad0 2021-02-10T11:32:14.451 INFO:tasks.ceph.mon.a.gibba001.stderr:2021-02-10 11:32:14.440 7f974d04d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-02-10T11:32:14.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.446 7f95937fe700 1 -- 172.21.2.101:0/627634399 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7f959801f4d0 con 0x7f95a40b4ad0 2021-02-10T11:32:14.488 INFO:teuthology.orchestra.run.gibba001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-02-10T11:32:14.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.479 7f95a8da3700 1 -- 172.21.2.101:0/627634399 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9588018730 msgr2=0x7f958801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:14.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.479 7f95a8da3700 1 --2- 172.21.2.101:0/627634399 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9588018730 0x7f958801abc0 crc :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:14.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.479 7f95a8da3700 1 -- 172.21.2.101:0/627634399 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95a40b4ad0 msgr2=0x7f95a40a78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:14.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.479 7f95a8da3700 1 --2- 172.21.2.101:0/627634399 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95a40b4ad0 0x7f95a40a78d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7f9598006ee0 tx=0x7f959800b960).stop 2021-02-10T11:32:14.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.479 7f95a8da3700 1 -- 172.21.2.101:0/627634399 shutdown_connections 2021-02-10T11:32:14.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.479 7f95a8da3700 1 --2- 172.21.2.101:0/627634399 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9588018730 0x7f958801abc0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:14.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.479 7f95a8da3700 1 --2- 172.21.2.101:0/627634399 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95a40a7ef0 0x7f95a41a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:14.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.479 7f95a8da3700 1 --2- 172.21.2.101:0/627634399 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f95a40a82f0 0x7f95a41a73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:14.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.479 7f95a8da3700 1 --2- 172.21.2.101:0/627634399 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95a40b4ad0 0x7f95a40a78d0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:14.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.479 7f95a8da3700 1 -- 172.21.2.101:0/627634399 >> 172.21.2.101:0/627634399 conn(0x7f95a40b5560 msgr2=0x7f95a40a6200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:14.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.479 7f95a8da3700 1 -- 172.21.2.101:0/627634399 shutdown_connections 2021-02-10T11:32:14.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.479 7f95a8da3700 1 -- 172.21.2.101:0/627634399 wait complete. 2021-02-10T11:32:14.498 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:32:14.508 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:32:14.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.897 7f6212cb1700 1 Processor -- start 2021-02-10T11:32:14.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.897 7f6212cb1700 1 -- start start 2021-02-10T11:32:14.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.897 7f6212cb1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f620c0e5470 0x7f620c07e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:14.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.897 7f6212cb1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f620c07f300 0x7f620c07f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:14.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.897 7f6212cb1700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f620c07ed80 con 0x7f620c0e5070 2021-02-10T11:32:14.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.897 7f6212cb1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f620c1b4010 con 0x7f620c07f300 2021-02-10T11:32:14.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.897 7f6212cb1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620c0d1790 con 0x7f620c0e5470 2021-02-10T11:32:14.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.899 7f62112ce700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f620c07f300 0x7f620c07f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:14.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.899 7f620bfff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f620c0e5470 0x7f620c07e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:14.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.899 7f6210acd700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f620c0e5070 0x7f620c1b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:44576/0 (socket says 172.21.2.101:44576) 2021-02-10T11:32:14.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.899 7f6210acd700 1 -- 172.21.2.101:0/2752526863 learned_addr learned my addr 172.21.2.101:0/2752526863 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:14.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.899 7f620bfff700 1 -- 172.21.2.101:0/2752526863 >> v2:172.21.2.108:3300/0 conn(0x7f620c07f300 msgr2=0x7f620c07f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:14.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.899 7f620bfff700 1 --2- 172.21.2.101:0/2752526863 >> v2:172.21.2.108:3300/0 conn(0x7f620c07f300 0x7f620c07f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:14.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.899 7f620bfff700 1 -- 172.21.2.101:0/2752526863 >> v1:172.21.2.108:6789/0 conn(0x7f620c0e5070 legacy=0x7f620c1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:14.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.899 7f620bfff700 1 -- 172.21.2.101:0/2752526863 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620c0d1980 con 0x7f620c0e5470 2021-02-10T11:32:14.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.899 7f620bfff700 1 --2- 172.21.2.101:0/2752526863 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f620c0e5470 0x7f620c07e6a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f6200006650 tx=0x7f6200009a80).ready entity=mon.2 client_cookie=c919b0e5c6f5f19b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:14.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.900 7f620b7fe700 1 -- 172.21.2.101:0/2752526863 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f620000c0c0 con 0x7f620c0e5470 2021-02-10T11:32:14.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.900 7f620b7fe700 1 -- 172.21.2.101:0/2752526863 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620000c340 con 0x7f620c0e5470 2021-02-10T11:32:14.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.900 7f620b7fe700 1 -- 172.21.2.101:0/2752526863 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f620000c860 con 0x7f620c0e5470 2021-02-10T11:32:14.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.900 7f6212cb1700 1 -- 172.21.2.101:0/2752526863 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f620c0e5470 msgr2=0x7f620c07e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:14.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.900 7f6212cb1700 1 --2- 172.21.2.101:0/2752526863 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f620c0e5470 0x7f620c07e6a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f6200006650 tx=0x7f6200009a80).stop 2021-02-10T11:32:14.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.900 7f6212cb1700 1 -- 172.21.2.101:0/2752526863 shutdown_connections 2021-02-10T11:32:14.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.900 7f6212cb1700 1 --2- 172.21.2.101:0/2752526863 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f620c0e5470 0x7f620c07e6a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:14.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.900 7f6212cb1700 1 --2- 172.21.2.101:0/2752526863 >> v2:172.21.2.108:3300/0 conn(0x7f620c07f300 0x7f620c07f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:14.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.900 7f6212cb1700 1 -- 172.21.2.101:0/2752526863 >> 172.21.2.101:0/2752526863 conn(0x7f620c0ce690 msgr2=0x7f620c0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:14.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.901 7f6212cb1700 1 -- 172.21.2.101:0/2752526863 shutdown_connections 2021-02-10T11:32:14.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.901 7f6212cb1700 1 -- 172.21.2.101:0/2752526863 wait complete. 2021-02-10T11:32:14.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.915 7f6212cb1700 1 Processor -- start 2021-02-10T11:32:14.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.915 7f6212cb1700 1 -- start start 2021-02-10T11:32:14.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.916 7f6212cb1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f620c191bb0 0x7f620c0a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:14.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.916 7f6212cb1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f620c0a29a0 0x7f620c0a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:14.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.916 7f6212cb1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f620c0a33c0 0x7f620c0a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:14.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.916 7f6212cb1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c0adf90 con 0x7f620c191bb0 2021-02-10T11:32:14.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.916 7f6212cb1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c0a1e00 con 0x7f620c0a33c0 2021-02-10T11:32:14.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.916 7f6212cb1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620c0a09f0 con 0x7f620c0a29a0 2021-02-10T11:32:14.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.916 7f6210acd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f620c191bb0 0x7f620c0a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:14.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.916 7f6210acd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f620c191bb0 0x7f620c0a2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50678/0 (socket says 172.21.2.101:50678) 2021-02-10T11:32:14.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.916 7f6210acd700 1 -- 172.21.2.101:0/167318895 learned_addr learned my addr 172.21.2.101:0/167318895 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:14.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.916 7f620bfff700 1 --2- 172.21.2.101:0/167318895 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f620c0a29a0 0x7f620c0a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:14.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.916 7f62112ce700 1 --2- 172.21.2.101:0/167318895 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f620c0a33c0 0x7f620c0a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:14.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.916 7f6210acd700 1 -- 172.21.2.101:0/167318895 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f620c0a29a0 msgr2=0x7f620c0a2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:14.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.916 7f6210acd700 1 --2- 172.21.2.101:0/167318895 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f620c0a29a0 0x7f620c0a2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:14.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.916 7f6210acd700 1 -- 172.21.2.101:0/167318895 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f620c0a33c0 msgr2=0x7f620c0a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:14.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.916 7f6210acd700 1 --2- 172.21.2.101:0/167318895 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f620c0a33c0 0x7f620c0a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:14.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.916 7f6210acd700 1 -- 172.21.2.101:0/167318895 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620c0a0bd0 con 0x7f620c191bb0 2021-02-10T11:32:14.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.916 7f6210acd700 1 --2- 172.21.2.101:0/167318895 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f620c191bb0 0x7f620c0a2380 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0x7f61fc007fa0 tx=0x7f61fc00a950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:14.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.917 7f62097fa700 1 -- 172.21.2.101:0/167318895 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f61fc0070d0 con 0x7f620c191bb0 2021-02-10T11:32:14.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.917 7f62097fa700 1 -- 172.21.2.101:0/167318895 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61fc00ad40 con 0x7f620c191bb0 2021-02-10T11:32:14.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.917 7f62097fa700 1 -- 172.21.2.101:0/167318895 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f61fc0045a0 con 0x7f620c191bb0 2021-02-10T11:32:14.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.917 7f6212cb1700 1 -- 172.21.2.101:0/167318895 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f620c0a0bd0 con 0x7f620c191bb0 2021-02-10T11:32:14.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.917 7f6212cb1700 1 -- 172.21.2.101:0/167318895 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f620c0a1160 con 0x7f620c191bb0 2021-02-10T11:32:14.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.917 7f62097fa700 1 -- 172.21.2.101:0/167318895 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f61fc000c10 con 0x7f620c191bb0 2021-02-10T11:32:14.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.917 7f62097fa700 1 --2- 172.21.2.101:0/167318895 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61ec0186c0 0x7f61ec01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:14.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.917 7f62097fa700 1 -- 172.21.2.101:0/167318895 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f61fc0325c0 con 0x7f620c191bb0 2021-02-10T11:32:14.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.918 7f620bfff700 1 --2- 172.21.2.101:0/167318895 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61ec0186c0 0x7f61ec01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:14.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.918 7f620bfff700 1 --2- 172.21.2.101:0/167318895 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61ec0186c0 0x7f61ec01abc0 crc :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:14.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.947 7f6212cb1700 1 -- 172.21.2.101:0/167318895 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f620c0a1160 con 0x7f620c191bb0 2021-02-10T11:32:14.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:14.949 7f62097fa700 1 -- 172.21.2.101:0/167318895 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f61fc030d20 con 0x7f620c191bb0 2021-02-10T11:32:15.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.077 7f6212cb1700 1 -- 172.21.2.101:0/167318895 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f620c0a1160 con 0x7f620c191bb0 2021-02-10T11:32:15.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.077 7f62097fa700 1 -- 172.21.2.101:0/167318895 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+1751 (secure 0 0 0) 0x7f61fc000c10 con 0x7f620c191bb0 2021-02-10T11:32:15.088 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:32:15.089 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/243293953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":243293953},{"type":"v1","addr":"172.21.2.101:6835","nonce":243293953}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":18}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:31:31.941616","modified":"2021-02-10 11:32:14.441187","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-02-10T11:32:15.089 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 24 2021-02-10T11:32:15.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.079 7f6212cb1700 1 -- 172.21.2.101:0/167318895 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61ec0186c0 msgr2=0x7f61ec01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:15.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.079 7f6212cb1700 1 --2- 172.21.2.101:0/167318895 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61ec0186c0 0x7f61ec01abc0 crc :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:15.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.079 7f6212cb1700 1 -- 172.21.2.101:0/167318895 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f620c191bb0 msgr2=0x7f620c0a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:15.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.079 7f6212cb1700 1 --2- 172.21.2.101:0/167318895 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f620c191bb0 0x7f620c0a2380 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0x7f61fc007fa0 tx=0x7f61fc00a950).stop 2021-02-10T11:32:15.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.079 7f6212cb1700 1 -- 172.21.2.101:0/167318895 shutdown_connections 2021-02-10T11:32:15.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.079 7f6212cb1700 1 --2- 172.21.2.101:0/167318895 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61ec0186c0 0x7f61ec01abc0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:15.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.079 7f6212cb1700 1 --2- 172.21.2.101:0/167318895 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f620c0a29a0 0x7f620c0a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:15.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.079 7f6212cb1700 1 --2- 172.21.2.101:0/167318895 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f620c0a33c0 0x7f620c0a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:15.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.079 7f6212cb1700 1 --2- 172.21.2.101:0/167318895 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f620c191bb0 0x7f620c0a2380 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:15.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.079 7f6212cb1700 1 -- 172.21.2.101:0/167318895 >> 172.21.2.101:0/167318895 conn(0x7f620c1a31d0 msgr2=0x7f620c08b830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:15.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.079 7f6212cb1700 1 -- 172.21.2.101:0/167318895 shutdown_connections 2021-02-10T11:32:15.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.079 7f6212cb1700 1 -- 172.21.2.101:0/167318895 wait complete. 2021-02-10T11:32:15.099 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-02-10T11:32:15.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.504 7f2875c67700 1 Processor -- start 2021-02-10T11:32:15.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.504 7f2875c67700 1 -- start start 2021-02-10T11:32:15.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.505 7f2875c67700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f28700d17f0 0x7f28701b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:15.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.505 7f2875c67700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f28700d1bf0 0x7f28701b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:15.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.505 7f2875c67700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28700f0b20 con 0x7f28701b3e20 2021-02-10T11:32:15.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.505 7f2875c67700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28700e5eb0 con 0x7f28700d1bf0 2021-02-10T11:32:15.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.505 7f2875c67700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28700f0630 con 0x7f28700d17f0 2021-02-10T11:32:15.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.505 7f286ffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f28701b3e20 0x7f28700ceef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51178/0 (socket says 172.21.2.101:51178) 2021-02-10T11:32:15.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.505 7f286ffff700 1 -- 172.21.2.101:0/3703106242 learned_addr learned my addr 172.21.2.101:0/3703106242 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:15.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.505 7f286effd700 1 --2- 172.21.2.101:0/3703106242 >> v2:172.21.2.101:3300/0 conn(0x7f28700d1bf0 0x7f28701b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:15.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.505 7f286f7fe700 1 --2- 172.21.2.101:0/3703106242 >> v2:172.21.2.108:3300/0 conn(0x7f28700d17f0 0x7f28701b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:15.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.505 7f286effd700 1 -- 172.21.2.101:0/3703106242 >> v2:172.21.2.108:3300/0 conn(0x7f28700d17f0 msgr2=0x7f28701b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:15.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.505 7f286effd700 1 --2- 172.21.2.101:0/3703106242 >> v2:172.21.2.108:3300/0 conn(0x7f28700d17f0 0x7f28701b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:15.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.505 7f286effd700 1 -- 172.21.2.101:0/3703106242 >> v1:172.21.2.101:6789/0 conn(0x7f28701b3e20 legacy=0x7f28700ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:15.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.505 7f286e7fc700 1 -- 172.21.2.101:0/3703106242 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 592167795 0 0) 0x7f2868001500 con 0x7f28701b3e20 2021-02-10T11:32:15.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.505 7f286effd700 1 -- 172.21.2.101:0/3703106242 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28700f0850 con 0x7f28700d1bf0 2021-02-10T11:32:15.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.505 7f286effd700 1 --2- 172.21.2.101:0/3703106242 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28700d1bf0 0x7f28701b3800 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7f2860006650 tx=0x7f2860009a00).ready entity=mon.0 client_cookie=c1dec8d08b2c151f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:15.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.505 7f286e7fc700 1 -- 172.21.2.101:0/3703106242 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f28600009b0 con 0x7f28700d1bf0 2021-02-10T11:32:15.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.505 7f286e7fc700 1 -- 172.21.2.101:0/3703106242 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286000c0c0 con 0x7f28700d1bf0 2021-02-10T11:32:15.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.505 7f286e7fc700 1 -- 172.21.2.101:0/3703106242 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f286000c5a0 con 0x7f28700d1bf0 2021-02-10T11:32:15.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.505 7f2875c67700 1 -- 172.21.2.101:0/3703106242 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28700d1bf0 msgr2=0x7f28701b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:15.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.505 7f2875c67700 1 --2- 172.21.2.101:0/3703106242 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28700d1bf0 0x7f28701b3800 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7f2860006650 tx=0x7f2860009a00).stop 2021-02-10T11:32:15.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.506 7f2875c67700 1 -- 172.21.2.101:0/3703106242 shutdown_connections 2021-02-10T11:32:15.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.506 7f2875c67700 1 --2- 172.21.2.101:0/3703106242 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28700d1bf0 0x7f28701b3800 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:15.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.506 7f2875c67700 1 --2- 172.21.2.101:0/3703106242 >> v2:172.21.2.108:3300/0 conn(0x7f28700d17f0 0x7f28701b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:15.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.506 7f2875c67700 1 -- 172.21.2.101:0/3703106242 >> 172.21.2.101:0/3703106242 conn(0x7f28700fde40 msgr2=0x7f28700ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:15.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.506 7f2875c67700 1 -- 172.21.2.101:0/3703106242 shutdown_connections 2021-02-10T11:32:15.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.506 7f2875c67700 1 -- 172.21.2.101:0/3703106242 wait complete. 2021-02-10T11:32:15.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.521 7f2875c67700 1 Processor -- start 2021-02-10T11:32:15.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.521 7f2875c67700 1 -- start start 2021-02-10T11:32:15.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.521 7f2875c67700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28701a49b0 0x7f287008a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:15.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.521 7f2875c67700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f287009e080 0x7f28700c3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:15.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.521 7f2875c67700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f287009e480 0x7f28700c3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:15.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.521 7f2875c67700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28700c2b80 con 0x7f287009e480 2021-02-10T11:32:15.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.521 7f2875c67700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28700bee50 con 0x7f287009e080 2021-02-10T11:32:15.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.521 7f2875c67700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28700c13f0 con 0x7f28701a49b0 2021-02-10T11:32:15.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.522 7f286ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f287009e480 0x7f28700c3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:15.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.522 7f286ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f287009e480 0x7f28700c3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50692/0 (socket says 172.21.2.101:50692) 2021-02-10T11:32:15.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.522 7f286ffff700 1 -- 172.21.2.101:0/3859436453 learned_addr learned my addr 172.21.2.101:0/3859436453 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:15.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.522 7f286f7fe700 1 --2- 172.21.2.101:0/3859436453 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28701a49b0 0x7f287008a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:15.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.522 7f286effd700 1 --2- 172.21.2.101:0/3859436453 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f287009e080 0x7f28700c3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:15.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.522 7f286ffff700 1 -- 172.21.2.101:0/3859436453 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28701a49b0 msgr2=0x7f287008a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:15.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.522 7f286ffff700 1 --2- 172.21.2.101:0/3859436453 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28701a49b0 0x7f287008a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:15.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.522 7f286ffff700 1 -- 172.21.2.101:0/3859436453 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f287009e080 msgr2=0x7f28700c3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:15.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.522 7f286ffff700 1 --2- 172.21.2.101:0/3859436453 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f287009e080 0x7f28700c3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:15.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.522 7f286ffff700 1 -- 172.21.2.101:0/3859436453 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28700c1610 con 0x7f287009e480 2021-02-10T11:32:15.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.522 7f286ffff700 1 --2- 172.21.2.101:0/3859436453 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f287009e480 0x7f28700c3720 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7f2868001c10 tx=0x7f286800ac60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:15.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.522 7f2867fff700 1 -- 172.21.2.101:0/3859436453 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2868007020 con 0x7f287009e480 2021-02-10T11:32:15.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.522 7f2867fff700 1 -- 172.21.2.101:0/3859436453 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2868007320 con 0x7f287009e480 2021-02-10T11:32:15.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.522 7f2867fff700 1 -- 172.21.2.101:0/3859436453 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2868004910 con 0x7f287009e480 2021-02-10T11:32:15.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.522 7f2875c67700 1 -- 172.21.2.101:0/3859436453 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28700c1610 con 0x7f287009e480 2021-02-10T11:32:15.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.522 7f2875c67700 1 -- 172.21.2.101:0/3859436453 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28700c1ba0 con 0x7f287009e480 2021-02-10T11:32:15.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.523 7f2867fff700 1 -- 172.21.2.101:0/3859436453 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2868004720 con 0x7f287009e480 2021-02-10T11:32:15.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.523 7f2867fff700 1 --2- 172.21.2.101:0/3859436453 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28540186c0 0x7f285401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:15.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.523 7f2867fff700 1 -- 172.21.2.101:0/3859436453 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f2868032920 con 0x7f287009e480 2021-02-10T11:32:15.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.523 7f286f7fe700 1 --2- 172.21.2.101:0/3859436453 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28540186c0 0x7f285401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:15.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.523 7f286f7fe700 1 --2- 172.21.2.101:0/3859436453 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28540186c0 0x7f285401abc0 crc :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:15.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.554 7f2875c67700 1 -- 172.21.2.101:0/3859436453 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2870103690 con 0x7f287009e480 2021-02-10T11:32:15.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.557 7f2867fff700 1 -- 172.21.2.101:0/3859436453 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2868000f90 con 0x7f287009e480 2021-02-10T11:32:15.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:15.683 7f2875c67700 1 -- 172.21.2.101:0/3859436453 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f28700f0b20 con 0x7f287009e480 2021-02-10T11:32:16.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.451 7f2867fff700 1 -- 172.21.2.101:0/3859436453 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2868004720 con 0x7f287009e480 2021-02-10T11:32:16.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.454 7f2875c67700 1 -- 172.21.2.101:0/3859436453 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28540186c0 msgr2=0x7f285401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:16.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.454 7f2875c67700 1 --2- 172.21.2.101:0/3859436453 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28540186c0 0x7f285401abc0 crc :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:16.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.454 7f2875c67700 1 -- 172.21.2.101:0/3859436453 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f287009e480 msgr2=0x7f28700c3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:16.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.454 7f2875c67700 1 --2- 172.21.2.101:0/3859436453 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f287009e480 0x7f28700c3720 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7f2868001c10 tx=0x7f286800ac60).stop 2021-02-10T11:32:16.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.455 7f2875c67700 1 -- 172.21.2.101:0/3859436453 shutdown_connections 2021-02-10T11:32:16.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.455 7f2875c67700 1 --2- 172.21.2.101:0/3859436453 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f287009e480 0x7f28700c3720 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:16.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.455 7f2875c67700 1 --2- 172.21.2.101:0/3859436453 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f287009e080 0x7f28700c3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:16.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.455 7f2875c67700 1 --2- 172.21.2.101:0/3859436453 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28540186c0 0x7f285401abc0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:16.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.455 7f2875c67700 1 --2- 172.21.2.101:0/3859436453 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28701a49b0 0x7f287008a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:16.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.455 7f2875c67700 1 -- 172.21.2.101:0/3859436453 >> 172.21.2.101:0/3859436453 conn(0x7f28700a1690 msgr2=0x7f28700a94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:16.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.455 7f2875c67700 1 -- 172.21.2.101:0/3859436453 shutdown_connections 2021-02-10T11:32:16.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.455 7f2875c67700 1 -- 172.21.2.101:0/3859436453 wait complete. 2021-02-10T11:32:16.474 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-02-10T11:32:16.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.878 7f87cc8b6700 1 Processor -- start 2021-02-10T11:32:16.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.878 7f87cc8b6700 1 -- start start 2021-02-10T11:32:16.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.879 7f87cc8b6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f87c80d1870 0x7f87c81b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:16.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.879 7f87cc8b6700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f87c80cff70 0x7f87c8109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.879 7f87cc8b6700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c80fe1c0 con 0x7f87c81b0ae0 2021-02-10T11:32:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.879 7f87cc8b6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c8109ad0 con 0x7f87c80d1870 2021-02-10T11:32:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.879 7f87cc8b6700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c8109750 con 0x7f87c80cff70 2021-02-10T11:32:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.879 7f87c661c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f87c80d1870 0x7f87c81b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.879 7f87c5e1b700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f87c81b0ae0 0x7f87c8108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51190/0 (socket says 172.21.2.101:51190) 2021-02-10T11:32:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.879 7f87c5e1b700 1 -- 172.21.2.101:0/1817097767 learned_addr learned my addr 172.21.2.101:0/1817097767 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.880 7f87c661c700 1 -- 172.21.2.101:0/1817097767 >> v2:172.21.2.108:3300/0 conn(0x7f87c80cff70 msgr2=0x7f87c8109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.880 7f87c661c700 1 --2- 172.21.2.101:0/1817097767 >> v2:172.21.2.108:3300/0 conn(0x7f87c80cff70 0x7f87c8109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.880 7f87c661c700 1 -- 172.21.2.101:0/1817097767 >> v1:172.21.2.101:6789/0 conn(0x7f87c81b0ae0 legacy=0x7f87c8108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.880 7f87c661c700 1 -- 172.21.2.101:0/1817097767 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c8088020 con 0x7f87c80d1870 2021-02-10T11:32:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.880 7f87c661c700 1 --2- 172.21.2.101:0/1817097767 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87c80d1870 0x7f87c81b04c0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f87b8006650 tx=0x7f87b8009a80).ready entity=mon.0 client_cookie=4ca3b4f8ecef1030 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.880 7f87c561a700 1 -- 172.21.2.101:0/1817097767 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f87b80009b0 con 0x7f87c80d1870 2021-02-10T11:32:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.880 7f87c561a700 1 -- 172.21.2.101:0/1817097767 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b800c0c0 con 0x7f87c80d1870 2021-02-10T11:32:16.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.880 7f87c561a700 1 -- 172.21.2.101:0/1817097767 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f87b800c5e0 con 0x7f87c80d1870 2021-02-10T11:32:16.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.880 7f87cc8b6700 1 -- 172.21.2.101:0/1817097767 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87c80d1870 msgr2=0x7f87c81b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:16.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.880 7f87cc8b6700 1 --2- 172.21.2.101:0/1817097767 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87c80d1870 0x7f87c81b04c0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f87b8006650 tx=0x7f87b8009a80).stop 2021-02-10T11:32:16.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.881 7f87cc8b6700 1 -- 172.21.2.101:0/1817097767 shutdown_connections 2021-02-10T11:32:16.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.881 7f87cc8b6700 1 --2- 172.21.2.101:0/1817097767 >> v2:172.21.2.108:3300/0 conn(0x7f87c80cff70 0x7f87c8109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:16.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.881 7f87cc8b6700 1 --2- 172.21.2.101:0/1817097767 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87c80d1870 0x7f87c81b04c0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:16.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.881 7f87cc8b6700 1 -- 172.21.2.101:0/1817097767 >> 172.21.2.101:0/1817097767 conn(0x7f87c80ce420 msgr2=0x7f87c80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:16.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.881 7f87cc8b6700 1 -- 172.21.2.101:0/1817097767 shutdown_connections 2021-02-10T11:32:16.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.881 7f87cc8b6700 1 -- 172.21.2.101:0/1817097767 wait complete. 2021-02-10T11:32:16.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.896 7f87cc8b6700 1 Processor -- start 2021-02-10T11:32:16.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.896 7f87cc8b6700 1 -- start start 2021-02-10T11:32:16.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.896 7f87cc8b6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87c81a9930 0x7f87c81a9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:16.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.896 7f87cc8b6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87c81abd60 0x7f87c8192a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:16.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.896 7f87cc8b6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87c80b03e0 0x7f87c80b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:16.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.896 7f87cc8b6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c81932c0 con 0x7f87c81abd60 2021-02-10T11:32:16.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.896 7f87cc8b6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c8193840 con 0x7f87c80b03e0 2021-02-10T11:32:16.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.896 7f87cc8b6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c8193a20 con 0x7f87c81a9930 2021-02-10T11:32:16.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.896 7f87c5e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87c81abd60 0x7f87c8192a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:16.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.896 7f87c5e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87c81abd60 0x7f87c8192a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50700/0 (socket says 172.21.2.101:50700) 2021-02-10T11:32:16.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.896 7f87c5e1b700 1 -- 172.21.2.101:0/4162478494 learned_addr learned my addr 172.21.2.101:0/4162478494 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:16.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.897 7f87c5e1b700 1 -- 172.21.2.101:0/4162478494 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87c81a9930 msgr2=0x7f87c81a9080 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:16.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.897 7f87c5e1b700 1 --2- 172.21.2.101:0/4162478494 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87c81a9930 0x7f87c81a9080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:16.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.897 7f87c5e1b700 1 -- 172.21.2.101:0/4162478494 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87c80b03e0 msgr2=0x7f87c80b07e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:16.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.897 7f87c5e1b700 1 --2- 172.21.2.101:0/4162478494 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87c80b03e0 0x7f87c80b07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:16.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.897 7f87c5e1b700 1 -- 172.21.2.101:0/4162478494 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c8193c40 con 0x7f87c81abd60 2021-02-10T11:32:16.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.897 7f87c5e1b700 1 --2- 172.21.2.101:0/4162478494 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87c81abd60 0x7f87c8192a60 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7f87bc00a4c0 tx=0x7f87bc00a860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:16.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.897 7f87b6ffd700 1 -- 172.21.2.101:0/4162478494 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f87bc00ac50 con 0x7f87c81abd60 2021-02-10T11:32:16.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.897 7f87b6ffd700 1 -- 172.21.2.101:0/4162478494 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87bc004140 con 0x7f87c81abd60 2021-02-10T11:32:16.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.897 7f87b6ffd700 1 -- 172.21.2.101:0/4162478494 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f87bc004660 con 0x7f87c81abd60 2021-02-10T11:32:16.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.897 7f87cc8b6700 1 -- 172.21.2.101:0/4162478494 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c8193c40 con 0x7f87c81abd60 2021-02-10T11:32:16.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.897 7f87cc8b6700 1 -- 172.21.2.101:0/4162478494 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c81941d0 con 0x7f87c81abd60 2021-02-10T11:32:16.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.898 7f87b6ffd700 1 -- 172.21.2.101:0/4162478494 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f87bc004320 con 0x7f87c81abd60 2021-02-10T11:32:16.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.898 7f87b6ffd700 1 --2- 172.21.2.101:0/4162478494 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87ac0186c0 0x7f87ac01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:16.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.898 7f87b6ffd700 1 -- 172.21.2.101:0/4162478494 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f87bc0325a0 con 0x7f87c81abd60 2021-02-10T11:32:16.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.898 7f87c661c700 1 --2- 172.21.2.101:0/4162478494 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87ac0186c0 0x7f87ac01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:16.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.898 7f87c661c700 1 --2- 172.21.2.101:0/4162478494 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87ac0186c0 0x7f87ac01abc0 crc :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:16.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.929 7f87cc8b6700 1 -- 172.21.2.101:0/4162478494 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c80828e0 con 0x7f87c81abd60 2021-02-10T11:32:16.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:16.931 7f87b6ffd700 1 -- 172.21.2.101:0/4162478494 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f87bc030d00 con 0x7f87c81abd60 2021-02-10T11:32:17.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.058 7f87cc8b6700 1 -- 172.21.2.101:0/4162478494 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f87c80828e0 con 0x7f87c81abd60 2021-02-10T11:32:17.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.455 7f87b6ffd700 1 -- 172.21.2.101:0/4162478494 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v40) v1 ==== 185+0+0 (secure 0 0 0) 0x7f87bc004680 con 0x7f87c81abd60 2021-02-10T11:32:17.486 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' removed 2021-02-10T11:32:17.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.477 7f87cc8b6700 1 -- 172.21.2.101:0/4162478494 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87ac0186c0 msgr2=0x7f87ac01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:17.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.477 7f87cc8b6700 1 --2- 172.21.2.101:0/4162478494 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87ac0186c0 0x7f87ac01abc0 crc :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:17.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.477 7f87cc8b6700 1 -- 172.21.2.101:0/4162478494 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87c81abd60 msgr2=0x7f87c8192a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:17.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.477 7f87cc8b6700 1 --2- 172.21.2.101:0/4162478494 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87c81abd60 0x7f87c8192a60 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7f87bc00a4c0 tx=0x7f87bc00a860).stop 2021-02-10T11:32:17.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.477 7f87cc8b6700 1 -- 172.21.2.101:0/4162478494 shutdown_connections 2021-02-10T11:32:17.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.477 7f87cc8b6700 1 --2- 172.21.2.101:0/4162478494 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87c80b03e0 0x7f87c80b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:17.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.477 7f87cc8b6700 1 --2- 172.21.2.101:0/4162478494 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87c81abd60 0x7f87c8192a60 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:17.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.477 7f87cc8b6700 1 --2- 172.21.2.101:0/4162478494 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87ac0186c0 0x7f87ac01abc0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:17.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.477 7f87cc8b6700 1 --2- 172.21.2.101:0/4162478494 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87c81a9930 0x7f87c81a9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:17.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.477 7f87cc8b6700 1 -- 172.21.2.101:0/4162478494 >> 172.21.2.101:0/4162478494 conn(0x7f87c819f820 msgr2=0x7f87c8082b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:17.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.477 7f87cc8b6700 1 -- 172.21.2.101:0/4162478494 shutdown_connections 2021-02-10T11:32:17.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.477 7f87cc8b6700 1 -- 172.21.2.101:0/4162478494 wait complete. 2021-02-10T11:32:17.497 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-02-10T11:32:17.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.901 7f73b0007700 1 Processor -- start 2021-02-10T11:32:17.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.901 7f73b0007700 1 -- start start 2021-02-10T11:32:17.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.901 7f73b0007700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73a80d1860 0x7f73a81b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:17.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.901 7f73b0007700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f73a81b0ad0 0x7f73a8108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:17.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.901 7f73b0007700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73a8109a50 con 0x7f73a80d1c60 2021-02-10T11:32:17.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.901 7f73b0007700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a80e5ee0 con 0x7f73a81b0ad0 2021-02-10T11:32:17.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.901 7f73b0007700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a8109740 con 0x7f73a80d1860 2021-02-10T11:32:17.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.901 7f73ad622700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f73a81b0ad0 0x7f73a8108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:17.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.901 7f73ad622700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f73a81b0ad0 0x7f73a8108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50708/0 (socket says 172.21.2.101:50708) 2021-02-10T11:32:17.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.901 7f73ad622700 1 -- 172.21.2.101:0/242931725 learned_addr learned my addr 172.21.2.101:0/242931725 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:17.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.902 7f73ad622700 1 -- 172.21.2.101:0/242931725 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73a80d1860 msgr2=0x7f73a81b04b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:32:17.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.902 7f73ad622700 1 --2- 172.21.2.101:0/242931725 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73a80d1860 0x7f73a81b04b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:17.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.902 7f73ad622700 1 -- 172.21.2.101:0/242931725 >> v1:172.21.2.108:6789/0 conn(0x7f73a80d1c60 legacy=0x7f73a8108f50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:32:17.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.902 7f73ad622700 1 -- 172.21.2.101:0/242931725 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a8088020 con 0x7f73a81b0ad0 2021-02-10T11:32:17.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.902 7f73ad622700 1 --2- 172.21.2.101:0/242931725 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73a81b0ad0 0x7f73a8108930 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f73a4006700 tx=0x7f73a4009b00).ready entity=mon.0 client_cookie=3bbeecadf84e34e1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:17.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.902 7f73ace21700 1 -- 172.21.2.101:0/242931725 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73a400c0c0 con 0x7f73a81b0ad0 2021-02-10T11:32:17.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.902 7f73ace21700 1 -- 172.21.2.101:0/242931725 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a400c340 con 0x7f73a81b0ad0 2021-02-10T11:32:17.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.902 7f73ace21700 1 -- 172.21.2.101:0/242931725 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73a400c820 con 0x7f73a81b0ad0 2021-02-10T11:32:17.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.902 7f73b0007700 1 -- 172.21.2.101:0/242931725 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73a81b0ad0 msgr2=0x7f73a8108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:17.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.902 7f73b0007700 1 --2- 172.21.2.101:0/242931725 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73a81b0ad0 0x7f73a8108930 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f73a4006700 tx=0x7f73a4009b00).stop 2021-02-10T11:32:17.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.903 7f73b0007700 1 -- 172.21.2.101:0/242931725 shutdown_connections 2021-02-10T11:32:17.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.903 7f73b0007700 1 --2- 172.21.2.101:0/242931725 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73a81b0ad0 0x7f73a8108930 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:17.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.903 7f73b0007700 1 --2- 172.21.2.101:0/242931725 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73a80d1860 0x7f73a81b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:17.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.903 7f73b0007700 1 -- 172.21.2.101:0/242931725 >> 172.21.2.101:0/242931725 conn(0x7f73a80ce410 msgr2=0x7f73a80d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:17.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.903 7f73b0007700 1 -- 172.21.2.101:0/242931725 shutdown_connections 2021-02-10T11:32:17.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.903 7f73b0007700 1 -- 172.21.2.101:0/242931725 wait complete. 2021-02-10T11:32:17.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.917 7f73b0007700 1 Processor -- start 2021-02-10T11:32:17.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.917 7f73b0007700 1 -- start start 2021-02-10T11:32:17.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.918 7f73b0007700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73a81a9920 0x7f73a81a9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:17.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.918 7f73b0007700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73a81abd50 0x7f73a8192a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:17.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.918 7f73b0007700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73a80b03e0 0x7f73a80b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:17.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.918 7f73b0007700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a81932b0 con 0x7f73a81a9920 2021-02-10T11:32:17.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.918 7f73b0007700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a8193830 con 0x7f73a80b03e0 2021-02-10T11:32:17.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.918 7f73b0007700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a8193a10 con 0x7f73a81abd50 2021-02-10T11:32:17.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.918 7f73ade23700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73a81a9920 0x7f73a81a9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:17.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.918 7f73ade23700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73a81a9920 0x7f73a81a9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50714/0 (socket says 172.21.2.101:50714) 2021-02-10T11:32:17.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.918 7f73ade23700 1 -- 172.21.2.101:0/3879189296 learned_addr learned my addr 172.21.2.101:0/3879189296 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:17.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.918 7f73ade23700 1 -- 172.21.2.101:0/3879189296 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73a81abd50 msgr2=0x7f73a8192a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:17.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.918 7f73ade23700 1 --2- 172.21.2.101:0/3879189296 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73a81abd50 0x7f73a8192a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:17.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.918 7f73ade23700 1 -- 172.21.2.101:0/3879189296 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73a80b03e0 msgr2=0x7f73a80b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:17.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.918 7f73ade23700 1 --2- 172.21.2.101:0/3879189296 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73a80b03e0 0x7f73a80b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:17.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.918 7f73ade23700 1 -- 172.21.2.101:0/3879189296 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a8193c30 con 0x7f73a81a9920 2021-02-10T11:32:17.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.919 7f73ade23700 1 --2- 172.21.2.101:0/3879189296 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73a81a9920 0x7f73a81a9070 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f739c00b5c0 tx=0x7f739c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:17.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.919 7f739a7fc700 1 -- 172.21.2.101:0/3879189296 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f739c00bd50 con 0x7f73a81a9920 2021-02-10T11:32:17.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.919 7f739a7fc700 1 -- 172.21.2.101:0/3879189296 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739c006030 con 0x7f73a81a9920 2021-02-10T11:32:17.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.919 7f739a7fc700 1 -- 172.21.2.101:0/3879189296 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f739c0054d0 con 0x7f73a81a9920 2021-02-10T11:32:17.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.919 7f73b0007700 1 -- 172.21.2.101:0/3879189296 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a8193c30 con 0x7f73a81a9920 2021-02-10T11:32:17.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.919 7f73b0007700 1 -- 172.21.2.101:0/3879189296 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a8193c30 con 0x7f73a81a9920 2021-02-10T11:32:17.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.920 7f739a7fc700 1 -- 172.21.2.101:0/3879189296 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f739c006210 con 0x7f73a81a9920 2021-02-10T11:32:17.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.920 7f739a7fc700 1 --2- 172.21.2.101:0/3879189296 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73900186c0 0x7f739001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:17.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.920 7f739a7fc700 1 -- 172.21.2.101:0/3879189296 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f739c0323f0 con 0x7f73a81a9920 2021-02-10T11:32:17.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.920 7f73ad622700 1 --2- 172.21.2.101:0/3879189296 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73900186c0 0x7f739001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:17.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.920 7f73ad622700 1 --2- 172.21.2.101:0/3879189296 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73900186c0 0x7f739001abc0 crc :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:17.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.951 7f73b0007700 1 -- 172.21.2.101:0/3879189296 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a80828e0 con 0x7f73a81a9920 2021-02-10T11:32:17.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:17.953 7f739a7fc700 1 -- 172.21.2.101:0/3879189296 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f739c0288b0 con 0x7f73a81a9920 2021-02-10T11:32:18.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.082 7f73b0007700 1 -- 172.21.2.101:0/3879189296 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f73a80828e0 con 0x7f73a81a9920 2021-02-10T11:32:18.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.459 7f739a7fc700 1 -- 172.21.2.101:0/3879189296 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v41) v1 ==== 173+0+0 (secure 0 0 0) 0x7f739c0288b0 con 0x7f73a81a9920 2021-02-10T11:32:18.509 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' removed 2021-02-10T11:32:18.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.500 7f73b0007700 1 -- 172.21.2.101:0/3879189296 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73900186c0 msgr2=0x7f739001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:18.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.500 7f73b0007700 1 --2- 172.21.2.101:0/3879189296 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73900186c0 0x7f739001abc0 crc :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:18.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.500 7f73b0007700 1 -- 172.21.2.101:0/3879189296 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73a81a9920 msgr2=0x7f73a81a9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:18.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.500 7f73b0007700 1 --2- 172.21.2.101:0/3879189296 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73a81a9920 0x7f73a81a9070 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f739c00b5c0 tx=0x7f739c00b960).stop 2021-02-10T11:32:18.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.500 7f73b0007700 1 -- 172.21.2.101:0/3879189296 shutdown_connections 2021-02-10T11:32:18.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.500 7f73b0007700 1 --2- 172.21.2.101:0/3879189296 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73900186c0 0x7f739001abc0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:18.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.500 7f73b0007700 1 --2- 172.21.2.101:0/3879189296 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73a81abd50 0x7f73a8192a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:18.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.500 7f73b0007700 1 --2- 172.21.2.101:0/3879189296 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73a80b03e0 0x7f73a80b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:18.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.500 7f73b0007700 1 --2- 172.21.2.101:0/3879189296 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73a81a9920 0x7f73a81a9070 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:18.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.500 7f73b0007700 1 -- 172.21.2.101:0/3879189296 >> 172.21.2.101:0/3879189296 conn(0x7f73a819f810 msgr2=0x7f73a8082b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:18.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.500 7f73b0007700 1 -- 172.21.2.101:0/3879189296 shutdown_connections 2021-02-10T11:32:18.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.500 7f73b0007700 1 -- 172.21.2.101:0/3879189296 wait complete. 2021-02-10T11:32:18.520 INFO:tasks.ceph.mds.a:Restarting daemon 2021-02-10T11:32:18.520 INFO:tasks.ceph.mds.a:Stopping old one... 2021-02-10T11:32:18.520 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-02-10T11:32:18.521 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:32:18.521 INFO:tasks.ceph.mds.b:Restarting daemon 2021-02-10T11:32:18.521 INFO:tasks.ceph.mds.b:Stopping old one... 2021-02-10T11:32:18.522 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-02-10T11:32:18.522 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:32:18.527 INFO:tasks.ceph.mds.a:Stopped 2021-02-10T11:32:18.528 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-02-10T11:32:18.530 INFO:tasks.ceph.mds.b:Stopped 2021-02-10T11:32:18.530 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-02-10T11:32:18.570 INFO:tasks.ceph.mds.a:Started 2021-02-10T11:32:18.570 DEBUG:teuthology.parallel:result is None 2021-02-10T11:32:18.572 INFO:tasks.ceph.mds.b:Started 2021-02-10T11:32:18.573 DEBUG:teuthology.parallel:result is None 2021-02-10T11:32:18.573 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd blacklist clear 2021-02-10T11:32:18.977 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:32:18.980 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:32:19.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.975 7f4502e0f700 1 Processor -- start 2021-02-10T11:32:19.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.975 7f4502e0f700 1 -- start start 2021-02-10T11:32:19.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.976 7f4502e0f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc0e5470 0x7f44fc07e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:19.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.976 7f4502e0f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f44fc07f300 0x7f44fc07f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:19.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.976 7f4502e0f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44fc07ed80 con 0x7f44fc0e5070 2021-02-10T11:32:19.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.976 7f4502e0f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44fc1b4010 con 0x7f44fc07f300 2021-02-10T11:32:19.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.976 7f4502e0f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc0d1790 con 0x7f44fc0e5470 2021-02-10T11:32:19.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.976 7f4500c2b700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f44fc0e5070 0x7f44fc1b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51224/0 (socket says 172.21.2.101:51224) 2021-02-10T11:32:19.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.976 7f4500c2b700 1 -- 172.21.2.101:0/3918870081 learned_addr learned my addr 172.21.2.101:0/3918870081 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:19.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.976 7f44fbfff700 1 --2- 172.21.2.101:0/3918870081 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc0e5470 0x7f44fc07e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:19.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.976 7f450142c700 1 --2- 172.21.2.101:0/3918870081 >> v2:172.21.2.108:3300/0 conn(0x7f44fc07f300 0x7f44fc07f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:19.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.976 7f44fb7fe700 1 -- 172.21.2.101:0/3918870081 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1498301645 0 0) 0x7f44e4000ce0 con 0x7f44fc0e5070 2021-02-10T11:32:19.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.977 7f44fb7fe700 1 -- 172.21.2.101:0/3918870081 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44e8002fe0 con 0x7f44fc0e5070 2021-02-10T11:32:19.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.977 7f44fbfff700 1 -- 172.21.2.101:0/3918870081 >> v2:172.21.2.108:3300/0 conn(0x7f44fc07f300 msgr2=0x7f44fc07f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:19.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.977 7f44fbfff700 1 --2- 172.21.2.101:0/3918870081 >> v2:172.21.2.108:3300/0 conn(0x7f44fc07f300 0x7f44fc07f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:19.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.977 7f44fbfff700 1 -- 172.21.2.101:0/3918870081 >> v1:172.21.2.101:6789/0 conn(0x7f44fc0e5070 legacy=0x7f44fc1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:19.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.977 7f44fbfff700 1 -- 172.21.2.101:0/3918870081 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc0d1980 con 0x7f44fc0e5470 2021-02-10T11:32:19.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.977 7f44fbfff700 1 --2- 172.21.2.101:0/3918870081 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc0e5470 0x7f44fc07e6a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f44ec006650 tx=0x7f44ec009a80).ready entity=mon.2 client_cookie=14f756dea0f34ef9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:19.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.977 7f44fb7fe700 1 -- 172.21.2.101:0/3918870081 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f44ec0009b0 con 0x7f44fc0e5470 2021-02-10T11:32:19.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.977 7f44fb7fe700 1 -- 172.21.2.101:0/3918870081 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ec00c0c0 con 0x7f44fc0e5470 2021-02-10T11:32:19.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.977 7f44fb7fe700 1 -- 172.21.2.101:0/3918870081 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f44ec00c510 con 0x7f44fc0e5470 2021-02-10T11:32:19.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.977 7f4502e0f700 1 -- 172.21.2.101:0/3918870081 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc0e5470 msgr2=0x7f44fc07e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:19.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.977 7f4502e0f700 1 --2- 172.21.2.101:0/3918870081 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc0e5470 0x7f44fc07e6a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f44ec006650 tx=0x7f44ec009a80).stop 2021-02-10T11:32:19.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.978 7f4502e0f700 1 -- 172.21.2.101:0/3918870081 shutdown_connections 2021-02-10T11:32:19.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.978 7f4502e0f700 1 --2- 172.21.2.101:0/3918870081 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc0e5470 0x7f44fc07e6a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:19.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.978 7f4502e0f700 1 --2- 172.21.2.101:0/3918870081 >> v2:172.21.2.108:3300/0 conn(0x7f44fc07f300 0x7f44fc07f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:19.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.978 7f4502e0f700 1 -- 172.21.2.101:0/3918870081 >> 172.21.2.101:0/3918870081 conn(0x7f44fc0ce690 msgr2=0x7f44fc0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:19.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.978 7f4502e0f700 1 -- 172.21.2.101:0/3918870081 shutdown_connections 2021-02-10T11:32:19.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.978 7f4502e0f700 1 -- 172.21.2.101:0/3918870081 wait complete. 2021-02-10T11:32:19.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.993 7f4502e0f700 1 Processor -- start 2021-02-10T11:32:19.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.993 7f4502e0f700 1 -- start start 2021-02-10T11:32:19.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.993 7f4502e0f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc0b4ad0 0x7f44fc0a78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:19.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.993 7f4502e0f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44fc0a7ef0 0x7f44fc1a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:19.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.993 7f4502e0f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44fc0a82f0 0x7f44fc1a73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:19.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.993 7f4502e0f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc0aa3c0 con 0x7f44fc0a82f0 2021-02-10T11:32:19.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.993 7f4502e0f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc0a88c0 con 0x7f44fc0a7ef0 2021-02-10T11:32:19.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.993 7f4502e0f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc0b4ed0 con 0x7f44fc0b4ad0 2021-02-10T11:32:19.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.993 7f450142c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44fc0a82f0 0x7f44fc1a73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:19.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.993 7f450142c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44fc0a82f0 0x7f44fc1a73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50738/0 (socket says 172.21.2.101:50738) 2021-02-10T11:32:19.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.993 7f450142c700 1 -- 172.21.2.101:0/321345983 learned_addr learned my addr 172.21.2.101:0/321345983 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:19.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.994 7f44fbfff700 1 --2- 172.21.2.101:0/321345983 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44fc0a7ef0 0x7f44fc1a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:19.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.994 7f4500c2b700 1 --2- 172.21.2.101:0/321345983 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc0b4ad0 0x7f44fc0a78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:19.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.994 7f450142c700 1 -- 172.21.2.101:0/321345983 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc0b4ad0 msgr2=0x7f44fc0a78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:19.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.994 7f450142c700 1 --2- 172.21.2.101:0/321345983 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc0b4ad0 0x7f44fc0a78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:19.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.994 7f450142c700 1 -- 172.21.2.101:0/321345983 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44fc0a7ef0 msgr2=0x7f44fc1a6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:19.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.994 7f450142c700 1 --2- 172.21.2.101:0/321345983 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44fc0a7ef0 0x7f44fc1a6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:19.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.994 7f450142c700 1 -- 172.21.2.101:0/321345983 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc1a79d0 con 0x7f44fc0a82f0 2021-02-10T11:32:19.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.994 7f4500c2b700 1 --2- 172.21.2.101:0/321345983 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc0b4ad0 0x7f44fc0a78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:32:19.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.994 7f450142c700 1 --2- 172.21.2.101:0/321345983 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44fc0a82f0 0x7f44fc1a73b0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7f44f400b970 tx=0x7f44f400bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:19.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.994 7f44f97fa700 1 -- 172.21.2.101:0/321345983 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f44f4000f90 con 0x7f44fc0a82f0 2021-02-10T11:32:19.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.994 7f44f97fa700 1 -- 172.21.2.101:0/321345983 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f40066f0 con 0x7f44fc0a82f0 2021-02-10T11:32:19.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.994 7f4502e0f700 1 -- 172.21.2.101:0/321345983 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44fc1a79d0 con 0x7f44fc0a82f0 2021-02-10T11:32:19.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.994 7f4502e0f700 1 -- 172.21.2.101:0/321345983 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44fc0a9130 con 0x7f44fc0a82f0 2021-02-10T11:32:19.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.994 7f44f97fa700 1 -- 172.21.2.101:0/321345983 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f44f40015f0 con 0x7f44fc0a82f0 2021-02-10T11:32:19.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.995 7f44f97fa700 1 -- 172.21.2.101:0/321345983 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f44f40011f0 con 0x7f44fc0a82f0 2021-02-10T11:32:19.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.995 7f44f97fa700 1 --2- 172.21.2.101:0/321345983 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44e00186c0 0x7f44e001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:19.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.995 7f44f97fa700 1 -- 172.21.2.101:0/321345983 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4839+0+0 (secure 0 0 0) 0x7f44f40325e0 con 0x7f44fc0a82f0 2021-02-10T11:32:19.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.995 7f4500c2b700 1 --2- 172.21.2.101:0/321345983 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44e00186c0 0x7f44e001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:19.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:18.995 7f4500c2b700 1 --2- 172.21.2.101:0/321345983 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44e00186c0 0x7f44e001abc0 crc :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:19.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.024 7f4502e0f700 1 -- 172.21.2.101:0/321345983 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44fc084a90 con 0x7f44fc0a82f0 2021-02-10T11:32:19.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.027 7f44f97fa700 1 -- 172.21.2.101:0/321345983 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f44f4000f40 con 0x7f44fc0a82f0 2021-02-10T11:32:19.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.169 7f4502e0f700 1 -- 172.21.2.101:0/321345983 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f44fc07ed80 con 0x7f44fc0a82f0 2021-02-10T11:32:19.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.505 7f44f97fa700 1 -- 172.21.2.101:0/321345983 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v42) v1 ==== 97+0+0 (secure 0 0 0) 0x7f44f40011f0 con 0x7f44fc0a82f0 2021-02-10T11:32:19.546 INFO:teuthology.orchestra.run.gibba001.stderr: removed all blacklist entries 2021-02-10T11:32:19.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.536 7f4502e0f700 1 -- 172.21.2.101:0/321345983 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44e00186c0 msgr2=0x7f44e001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:19.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.536 7f4502e0f700 1 --2- 172.21.2.101:0/321345983 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44e00186c0 0x7f44e001abc0 crc :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:19.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.537 7f4502e0f700 1 -- 172.21.2.101:0/321345983 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44fc0a82f0 msgr2=0x7f44fc1a73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:19.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.537 7f4502e0f700 1 --2- 172.21.2.101:0/321345983 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44fc0a82f0 0x7f44fc1a73b0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7f44f400b970 tx=0x7f44f400bd10).stop 2021-02-10T11:32:19.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.537 7f4502e0f700 1 -- 172.21.2.101:0/321345983 shutdown_connections 2021-02-10T11:32:19.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.537 7f4502e0f700 1 --2- 172.21.2.101:0/321345983 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44fc0a82f0 0x7f44fc1a73b0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:19.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.537 7f4502e0f700 1 --2- 172.21.2.101:0/321345983 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44fc0a7ef0 0x7f44fc1a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:19.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.537 7f4502e0f700 1 --2- 172.21.2.101:0/321345983 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44e00186c0 0x7f44e001abc0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:19.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.537 7f4502e0f700 1 --2- 172.21.2.101:0/321345983 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44fc0b4ad0 0x7f44fc0a78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:19.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.537 7f4502e0f700 1 -- 172.21.2.101:0/321345983 >> 172.21.2.101:0/321345983 conn(0x7f44fc0b5560 msgr2=0x7f44fc0a6200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:19.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.537 7f4502e0f700 1 -- 172.21.2.101:0/321345983 shutdown_connections 2021-02-10T11:32:19.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.537 7f4502e0f700 1 -- 172.21.2.101:0/321345983 wait complete. 2021-02-10T11:32:19.556 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2021-02-10T11:32:19.557 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:32:19.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.959 7fc3d7a47700 1 Processor -- start 2021-02-10T11:32:19.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.959 7fc3d7a47700 1 -- start start 2021-02-10T11:32:19.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.960 7fc3d7a47700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3d00d17f0 0x7fc3d01b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:19.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.960 7fc3d7a47700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc3d00cff00 0x7fc3d00ef620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:19.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.960 7fc3d7a47700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d00f0b20 con 0x7fc3d00d1bf0 2021-02-10T11:32:19.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.960 7fc3d7a47700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d00e5eb0 con 0x7fc3d00cff00 2021-02-10T11:32:19.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.960 7fc3d7a47700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d01b3ff0 con 0x7fc3d00d17f0 2021-02-10T11:32:19.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.960 7fc3d5062700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc3d00d1bf0 0x7fc3d01b3800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51238/0 (socket says 172.21.2.101:51238) 2021-02-10T11:32:19.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.960 7fc3d5062700 1 -- 172.21.2.101:0/676085351 learned_addr learned my addr 172.21.2.101:0/676085351 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:19.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.961 7fc3d4861700 1 -- 172.21.2.101:0/676085351 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4210243390 0 0) 0x7fc3cc0010b0 con 0x7fc3d00d1bf0 2021-02-10T11:32:19.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.961 7fc3d4861700 1 -- 172.21.2.101:0/676085351 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3bc002fe0 con 0x7fc3d00d1bf0 2021-02-10T11:32:19.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.961 7fc3d6064700 1 --2- 172.21.2.101:0/676085351 >> v2:172.21.2.108:3300/0 conn(0x7fc3d00cff00 0x7fc3d00ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:19.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.961 7fc3d5863700 1 --2- 172.21.2.101:0/676085351 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3d00d17f0 0x7fc3d01b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:19.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.961 7fc3d4861700 1 -- 172.21.2.101:0/676085351 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 759000227 0 0) 0x7fc3cc0010b0 con 0x7fc3d00d1bf0 2021-02-10T11:32:19.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.961 7fc3d4861700 1 -- 172.21.2.101:0/676085351 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3d00d17f0 msgr2=0x7fc3d01b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:19.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.961 7fc3d4861700 1 --2- 172.21.2.101:0/676085351 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3d00d17f0 0x7fc3d01b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:19.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.961 7fc3d4861700 1 -- 172.21.2.101:0/676085351 >> v2:172.21.2.108:3300/0 conn(0x7fc3d00cff00 msgr2=0x7fc3d00ef620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:19.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.961 7fc3d4861700 1 --2- 172.21.2.101:0/676085351 >> v2:172.21.2.108:3300/0 conn(0x7fc3d00cff00 0x7fc3d00ef620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:19.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.961 7fc3d4861700 1 -- 172.21.2.101:0/676085351 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d00f20b0 con 0x7fc3d00d1bf0 2021-02-10T11:32:19.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.961 7fc3d4861700 1 -- 172.21.2.101:0/676085351 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fc3cc0015b0 con 0x7fc3d00d1bf0 2021-02-10T11:32:19.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.961 7fc3d4861700 1 -- 172.21.2.101:0/676085351 >> v1:172.21.2.101:6789/0 conn(0x7fc3d00d1bf0 legacy=0x7fc3d01b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:19.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.961 7fc3d4861700 1 --2- 172.21.2.101:0/676085351 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3bc003ac0 0x7fc3bc005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:19.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.961 7fc3d4861700 1 --2- 172.21.2.101:0/676085351 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3bc04a7d0 0x7fc3bc04cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:19.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.961 7fc3d4861700 1 --2- 172.21.2.101:0/676085351 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3bc04d590 0x7fc3bc04fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:19.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.961 7fc3d4861700 1 -- 172.21.2.101:0/676085351 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc0502d0 con 0x7fc3bc04a7d0 2021-02-10T11:32:19.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.961 7fc3d4861700 1 -- 172.21.2.101:0/676085351 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc0504f0 con 0x7fc3bc003ac0 2021-02-10T11:32:19.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.961 7fc3d4861700 1 -- 172.21.2.101:0/676085351 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc050710 con 0x7fc3bc04d590 2021-02-10T11:32:19.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.962 7fc3d5062700 1 --2- 172.21.2.101:0/676085351 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3bc04a7d0 0x7fc3bc04cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:19.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.962 7fc3d5863700 1 --2- 172.21.2.101:0/676085351 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3bc003ac0 0x7fc3bc005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:19.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.962 7fc3d6064700 1 --2- 172.21.2.101:0/676085351 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3bc04d590 0x7fc3bc04fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:19.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.962 7fc3d5062700 1 -- 172.21.2.101:0/676085351 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3bc04d590 msgr2=0x7fc3bc04fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:19.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.962 7fc3d5062700 1 --2- 172.21.2.101:0/676085351 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3bc04d590 0x7fc3bc04fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:19.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.962 7fc3d5062700 1 -- 172.21.2.101:0/676085351 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3bc003ac0 msgr2=0x7fc3bc005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:19.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.962 7fc3d5062700 1 --2- 172.21.2.101:0/676085351 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3bc003ac0 0x7fc3bc005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:19.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.962 7fc3d5062700 1 -- 172.21.2.101:0/676085351 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3bc0509b0 con 0x7fc3bc04a7d0 2021-02-10T11:32:19.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.962 7fc3d5062700 1 --2- 172.21.2.101:0/676085351 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3bc04a7d0 0x7fc3bc04cc90 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 rx=0x7fc3cc0017d0 tx=0x7fc3cc007d30).ready entity=mon.0 client_cookie=6791c782093ce40f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:19.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.962 7fc3d4861700 1 -- 172.21.2.101:0/676085351 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc3cc0013a0 con 0x7fc3bc04a7d0 2021-02-10T11:32:19.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.962 7fc3d4861700 1 -- 172.21.2.101:0/676085351 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3cc00a310 con 0x7fc3bc04a7d0 2021-02-10T11:32:19.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.962 7fc3d4861700 1 -- 172.21.2.101:0/676085351 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc3cc00a7f0 con 0x7fc3bc04a7d0 2021-02-10T11:32:19.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.962 7fc3d7a47700 1 -- 172.21.2.101:0/676085351 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3bc04a7d0 msgr2=0x7fc3bc04cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:19.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.962 7fc3d7a47700 1 --2- 172.21.2.101:0/676085351 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3bc04a7d0 0x7fc3bc04cc90 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 rx=0x7fc3cc0017d0 tx=0x7fc3cc007d30).stop 2021-02-10T11:32:19.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.962 7fc3d6064700 1 -- 172.21.2.101:0/676085351 reap_dead start 2021-02-10T11:32:19.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.963 7fc3d7a47700 1 -- 172.21.2.101:0/676085351 shutdown_connections 2021-02-10T11:32:19.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.963 7fc3d7a47700 1 -- 172.21.2.101:0/676085351 >> 172.21.2.101:0/676085351 conn(0x7fc3d00fde40 msgr2=0x7fc3d00ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:19.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.963 7fc3d7a47700 1 -- 172.21.2.101:0/676085351 shutdown_connections 2021-02-10T11:32:19.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.963 7fc3d7a47700 1 -- 172.21.2.101:0/676085351 wait complete. 2021-02-10T11:32:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.978 7fc3d7a47700 1 Processor -- start 2021-02-10T11:32:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.978 7fc3d7a47700 1 -- start start 2021-02-10T11:32:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.979 7fc3d7a47700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3d00bf760 0x7fc3d00bfb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.979 7fc3d7a47700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3d00c0180 0x7fc3d00c0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.979 7fc3d7a47700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3d01050d0 0x7fc3d01054d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.979 7fc3d7a47700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d00c0ba0 con 0x7fc3d00bf760 2021-02-10T11:32:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.979 7fc3d7a47700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d00bedd0 con 0x7fc3d01050d0 2021-02-10T11:32:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.979 7fc3d7a47700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d00beff0 con 0x7fc3d00c0180 2021-02-10T11:32:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.979 7fc3d5863700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3d00bf760 0x7fc3d00bfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.979 7fc3d5863700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3d00bf760 0x7fc3d00bfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50756/0 (socket says 172.21.2.101:50756) 2021-02-10T11:32:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.979 7fc3d5863700 1 -- 172.21.2.101:0/4166258295 learned_addr learned my addr 172.21.2.101:0/4166258295 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.979 7fc3d5863700 1 -- 172.21.2.101:0/4166258295 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3d00c0180 msgr2=0x7fc3d00c0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.979 7fc3d5863700 1 --2- 172.21.2.101:0/4166258295 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3d00c0180 0x7fc3d00c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.979 7fc3d5863700 1 -- 172.21.2.101:0/4166258295 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3d01050d0 msgr2=0x7fc3d01054d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.979 7fc3d5863700 1 --2- 172.21.2.101:0/4166258295 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3d01050d0 0x7fc3d01054d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.979 7fc3d5863700 1 -- 172.21.2.101:0/4166258295 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d0105af0 con 0x7fc3d00bf760 2021-02-10T11:32:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.980 7fc3d5863700 1 --2- 172.21.2.101:0/4166258295 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3d00bf760 0x7fc3d00bfb60 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7fc3c4006f60 tx=0x7fc3c400b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.980 7fc3c27fc700 1 -- 172.21.2.101:0/4166258295 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc3c400bd60 con 0x7fc3d00bf760 2021-02-10T11:32:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.980 7fc3c27fc700 1 -- 172.21.2.101:0/4166258295 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c4000c90 con 0x7fc3d00bf760 2021-02-10T11:32:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.980 7fc3c27fc700 1 -- 172.21.2.101:0/4166258295 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc3c40022b0 con 0x7fc3d00bf760 2021-02-10T11:32:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.980 7fc3d7a47700 1 -- 172.21.2.101:0/4166258295 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d0105af0 con 0x7fc3d00bf760 2021-02-10T11:32:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.980 7fc3d7a47700 1 -- 172.21.2.101:0/4166258295 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d0105af0 con 0x7fc3d00bf760 2021-02-10T11:32:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.981 7fc3c27fc700 1 -- 172.21.2.101:0/4166258295 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc3c4006b00 con 0x7fc3d00bf760 2021-02-10T11:32:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.981 7fc3c27fc700 1 --2- 172.21.2.101:0/4166258295 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3b80186d0 0x7fc3b801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.981 7fc3c27fc700 1 -- 172.21.2.101:0/4166258295 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fc3c4032340 con 0x7fc3d00bf760 2021-02-10T11:32:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.981 7fc3d5062700 1 --2- 172.21.2.101:0/4166258295 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3b80186d0 0x7fc3b801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:19.981 7fc3d5062700 1 --2- 172.21.2.101:0/4166258295 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3b80186d0 0x7fc3b801abd0 crc :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:20.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.010 7fc3d7a47700 1 -- 172.21.2.101:0/4166258295 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d0103750 con 0x7fc3d00bf760 2021-02-10T11:32:20.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.012 7fc3c27fc700 1 -- 172.21.2.101:0/4166258295 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc3c4000c90 con 0x7fc3d00bf760 2021-02-10T11:32:20.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.145 7fc3d7a47700 1 -- 172.21.2.101:0/4166258295 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc3d0103750 con 0x7fc3d00bf760 2021-02-10T11:32:20.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.146 7fc3c27fc700 1 -- 172.21.2.101:0/4166258295 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) v1 ==== 80+0+4513 (secure 0 0 0) 0x7fc3c400bd60 con 0x7fc3d00bf760 2021-02-10T11:32:20.157 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:32:20.157 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:32:20.158 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:32:20.158 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:20.158 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:32:20.158 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:32:20.158 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:32:20.158 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:32:20.158 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:32:20.158 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:32:20.158 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:32:20.158 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:20.158 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:20.158 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:20.158 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:32:20.159 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:32:20.159 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:32:20.159 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:32:20.159 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:32:20.159 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:32:20.159 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:32:20.159 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:20.160 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:20.160 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:20.160 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:32:20.160 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:32:20.160 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:32:20.160 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:32:20.160 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:32:20.160 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:32:20.160 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:20.160 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:20.160 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:20.160 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:32:20.160 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:32:20.161 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:32:20.161 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:32:20.161 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:32:20.161 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:32:20.161 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:20.161 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:20.161 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:20.162 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:32:20.162 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:32:20.162 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:32:20.162 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:32:20.162 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:32:20.162 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:32:20.162 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:20.162 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:20.162 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:20.162 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:32:20.162 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:32:20.162 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:32:20.163 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:32:20.163 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:32:20.163 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:32:20.163 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:20.163 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:20.163 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:20.163 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:32:20.163 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:32:20.163 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:32:20.163 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:32:20.163 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:32:20.163 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:32:20.164 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:20.164 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:20.164 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:20.164 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:32:20.164 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:32:20.164 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:32:20.164 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:32:20.164 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:32:20.164 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:32:20.164 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:20.165 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:20.165 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:20.165 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:32:20.165 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:32:20.165 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:32:20.165 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:32:20.165 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:32:20.165 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:32:20.165 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:20.165 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:20.165 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:20.165 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:32:20.165 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:32:20.166 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:32:20.166 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:32:20.166 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:32:20.166 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:32:20.166 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:20.166 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:20.166 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:20.166 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:32:20.166 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:32:20.166 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:32:20.166 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:32:20.166 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:32:20.166 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:32:20.167 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:20.167 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:20.167 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:20.167 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:32:20.167 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:32:20.167 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:32:20.167 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:32:20.167 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:32:20.167 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:32:20.167 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:20.167 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:20.167 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:20.167 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:32:20.167 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:32:20.168 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:32:20.168 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:32:20.168 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:32:20.168 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:32:20.168 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:20.168 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:20.168 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:20.168 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:32:20.168 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:32:20.168 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:32:20.168 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:32:20.169 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:32:20.169 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:32:20.169 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:20.169 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:20.169 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:20.169 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:32:20.169 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:32:20.169 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:32:20.169 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:32:20.169 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:32:20.169 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:32:20.170 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:32:20.170 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:20.170 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:20.170 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:20.170 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:32:20.170 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:32:20.170 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:32:20.170 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:32:20.170 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:32:20.170 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:32:20.171 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:20.171 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:20.171 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:20.171 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:32:20.171 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:32:20.171 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:32:20.171 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:32:20.171 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:32:20.171 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:32:20.171 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:20.171 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:20.171 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:32:20.171 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:32:20.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.148 7fc3d7a47700 1 -- 172.21.2.101:0/4166258295 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3b80186d0 msgr2=0x7fc3b801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:20.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.148 7fc3d7a47700 1 --2- 172.21.2.101:0/4166258295 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3b80186d0 0x7fc3b801abd0 crc :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:20.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.148 7fc3d7a47700 1 -- 172.21.2.101:0/4166258295 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3d00bf760 msgr2=0x7fc3d00bfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:20.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.148 7fc3d7a47700 1 --2- 172.21.2.101:0/4166258295 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3d00bf760 0x7fc3d00bfb60 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7fc3c4006f60 tx=0x7fc3c400b970).stop 2021-02-10T11:32:20.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.148 7fc3d7a47700 1 -- 172.21.2.101:0/4166258295 shutdown_connections 2021-02-10T11:32:20.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.148 7fc3d7a47700 1 --2- 172.21.2.101:0/4166258295 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3b80186d0 0x7fc3b801abd0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:20.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.148 7fc3d7a47700 1 --2- 172.21.2.101:0/4166258295 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3d00c0180 0x7fc3d00c0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:20.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.148 7fc3d7a47700 1 --2- 172.21.2.101:0/4166258295 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3d01050d0 0x7fc3d01054d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:20.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.148 7fc3d7a47700 1 --2- 172.21.2.101:0/4166258295 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3d00bf760 0x7fc3d00bfb60 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:20.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.148 7fc3d7a47700 1 -- 172.21.2.101:0/4166258295 >> 172.21.2.101:0/4166258295 conn(0x7fc3d00a16a0 msgr2=0x7fc3d00a09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:20.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.148 7fc3d7a47700 1 -- 172.21.2.101:0/4166258295 shutdown_connections 2021-02-10T11:32:20.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.148 7fc3d7a47700 1 -- 172.21.2.101:0/4166258295 wait complete. 2021-02-10T11:32:20.175 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:32:20.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.576 7f52d1d84700 1 Processor -- start 2021-02-10T11:32:20.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.576 7f52d1d84700 1 -- start start 2021-02-10T11:32:20.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.577 7f52d1d84700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f52cc0e5070 0x7f52cc1b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:20.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.577 7f52d1d84700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f52cc0e5470 0x7f52cc080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:20.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.577 7f52d1d84700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52cc07ed80 con 0x7f52cc1b3e40 2021-02-10T11:32:20.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.577 7f52d1d84700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52cc0d1790 con 0x7f52cc0e5470 2021-02-10T11:32:20.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.577 7f52d1d84700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52cc0d19b0 con 0x7f52cc0e5070 2021-02-10T11:32:20.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.577 7f52cbfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f52cc0e5470 0x7f52cc080310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:20.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.577 7f52caffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f52cc1b3e40 0x7f52cc07d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51258/0 (socket says 172.21.2.101:51258) 2021-02-10T11:32:20.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.577 7f52cbfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f52cc0e5470 0x7f52cc080310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50764/0 (socket says 172.21.2.101:50764) 2021-02-10T11:32:20.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.577 7f52caffd700 1 -- 172.21.2.101:0/3055466763 learned_addr learned my addr 172.21.2.101:0/3055466763 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:20.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.577 7f52cbfff700 1 -- 172.21.2.101:0/3055466763 >> v2:172.21.2.108:3300/0 conn(0x7f52cc0e5070 msgr2=0x7f52cc1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:20.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.577 7f52cbfff700 1 --2- 172.21.2.101:0/3055466763 >> v2:172.21.2.108:3300/0 conn(0x7f52cc0e5070 0x7f52cc1b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:20.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.577 7f52cbfff700 1 -- 172.21.2.101:0/3055466763 >> v1:172.21.2.101:6789/0 conn(0x7f52cc1b3e40 legacy=0x7f52cc07d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:20.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.577 7f52cbfff700 1 -- 172.21.2.101:0/3055466763 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc0d1bd0 con 0x7f52cc0e5470 2021-02-10T11:32:20.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.577 7f52cbfff700 1 --2- 172.21.2.101:0/3055466763 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52cc0e5470 0x7f52cc080310 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7f52c4006e30 tx=0x7f52c4009e60).ready entity=mon.0 client_cookie=f8db2d7017a08c1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:20.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.578 7f52ca7fc700 1 -- 172.21.2.101:0/3055466763 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f52c4006a50 con 0x7f52cc0e5470 2021-02-10T11:32:20.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.578 7f52ca7fc700 1 -- 172.21.2.101:0/3055466763 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c4006a50 con 0x7f52cc0e5470 2021-02-10T11:32:20.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.578 7f52ca7fc700 1 -- 172.21.2.101:0/3055466763 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f52c400b300 con 0x7f52cc0e5470 2021-02-10T11:32:20.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.578 7f52d1d84700 1 -- 172.21.2.101:0/3055466763 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52cc0e5470 msgr2=0x7f52cc080310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:20.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.578 7f52d1d84700 1 --2- 172.21.2.101:0/3055466763 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52cc0e5470 0x7f52cc080310 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7f52c4006e30 tx=0x7f52c4009e60).stop 2021-02-10T11:32:20.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.578 7f52d1d84700 1 -- 172.21.2.101:0/3055466763 shutdown_connections 2021-02-10T11:32:20.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.578 7f52d1d84700 1 --2- 172.21.2.101:0/3055466763 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52cc0e5470 0x7f52cc080310 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:20.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.578 7f52d1d84700 1 --2- 172.21.2.101:0/3055466763 >> v2:172.21.2.108:3300/0 conn(0x7f52cc0e5070 0x7f52cc1b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:20.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.578 7f52d1d84700 1 -- 172.21.2.101:0/3055466763 >> 172.21.2.101:0/3055466763 conn(0x7f52cc0ce690 msgr2=0x7f52cc0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:20.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.578 7f52d1d84700 1 -- 172.21.2.101:0/3055466763 shutdown_connections 2021-02-10T11:32:20.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.578 7f52d1d84700 1 -- 172.21.2.101:0/3055466763 wait complete. 2021-02-10T11:32:20.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.593 7f52d1d84700 1 Processor -- start 2021-02-10T11:32:20.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.593 7f52d1d84700 1 -- start start 2021-02-10T11:32:20.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.593 7f52d1d84700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f52cc192fc0 0x7f52cc192780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:20.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.593 7f52d1d84700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52cc0b1430 0x7f52cc0b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:20.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.594 7f52d1d84700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f52cc0b1e50 0x7f52cc08a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:20.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.594 7f52d1d84700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc0b2250 con 0x7f52cc0b1430 2021-02-10T11:32:20.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.594 7f52d1d84700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc08b5d0 con 0x7f52cc0b1e50 2021-02-10T11:32:20.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.594 7f52d1d84700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc08aa80 con 0x7f52cc192fc0 2021-02-10T11:32:20.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.594 7f52caffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52cc0b1430 0x7f52cc0b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:20.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.594 7f52caffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52cc0b1430 0x7f52cc0b1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50770/0 (socket says 172.21.2.101:50770) 2021-02-10T11:32:20.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.594 7f52caffd700 1 -- 172.21.2.101:0/1291006378 learned_addr learned my addr 172.21.2.101:0/1291006378 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:20.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.594 7f52caffd700 1 -- 172.21.2.101:0/1291006378 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f52cc192fc0 msgr2=0x7f52cc192780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:20.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.594 7f52caffd700 1 --2- 172.21.2.101:0/1291006378 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f52cc192fc0 0x7f52cc192780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:20.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.594 7f52caffd700 1 -- 172.21.2.101:0/1291006378 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f52cc0b1e50 msgr2=0x7f52cc08a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:20.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.594 7f52caffd700 1 --2- 172.21.2.101:0/1291006378 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f52cc0b1e50 0x7f52cc08a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:20.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.594 7f52caffd700 1 -- 172.21.2.101:0/1291006378 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc08ac60 con 0x7f52cc0b1430 2021-02-10T11:32:20.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.594 7f52caffd700 1 --2- 172.21.2.101:0/1291006378 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52cc0b1430 0x7f52cc0b1830 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7f52bc001c60 tx=0x7f52bc00a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:20.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.594 7f52c3fff700 1 -- 172.21.2.101:0/1291006378 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f52bc0010b0 con 0x7f52cc0b1430 2021-02-10T11:32:20.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.594 7f52c3fff700 1 -- 172.21.2.101:0/1291006378 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc004140 con 0x7f52cc0b1430 2021-02-10T11:32:20.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.594 7f52d1d84700 1 -- 172.21.2.101:0/1291006378 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52cc08ac60 con 0x7f52cc0b1430 2021-02-10T11:32:20.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.594 7f52c3fff700 1 -- 172.21.2.101:0/1291006378 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f52bc004660 con 0x7f52cc0b1430 2021-02-10T11:32:20.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.594 7f52d1d84700 1 -- 172.21.2.101:0/1291006378 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52cc08ac60 con 0x7f52cc0b1430 2021-02-10T11:32:20.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.595 7f52c3fff700 1 -- 172.21.2.101:0/1291006378 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f52bc004320 con 0x7f52cc0b1430 2021-02-10T11:32:20.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.595 7f52c3fff700 1 --2- 172.21.2.101:0/1291006378 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52b00186d0 0x7f52b001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:20.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.595 7f52c3fff700 1 -- 172.21.2.101:0/1291006378 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f52bc0322c0 con 0x7f52cc0b1430 2021-02-10T11:32:20.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.595 7f52cb7fe700 1 --2- 172.21.2.101:0/1291006378 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52b00186d0 0x7f52b001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:20.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.595 7f52cb7fe700 1 --2- 172.21.2.101:0/1291006378 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52b00186d0 0x7f52b001abd0 crc :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:20.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.626 7f52d1d84700 1 -- 172.21.2.101:0/1291006378 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52cc107120 con 0x7f52cc0b1430 2021-02-10T11:32:20.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.629 7f52c3fff700 1 -- 172.21.2.101:0/1291006378 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f52bc0009b0 con 0x7f52cc0b1430 2021-02-10T11:32:20.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.755 7f52d1d84700 1 -- 172.21.2.101:0/1291006378 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52cc07ed80 con 0x7f52cc0b1430 2021-02-10T11:32:20.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.756 7f52c3fff700 1 -- 172.21.2.101:0/1291006378 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+11144 (secure 0 0 0) 0x7f52bc004140 con 0x7f52cc0b1430 2021-02-10T11:32:20.766 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:32:20.766 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":42,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:32:19.503082","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:32:20.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.757 7f52d1d84700 1 -- 172.21.2.101:0/1291006378 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52b00186d0 msgr2=0x7f52b001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:20.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.757 7f52d1d84700 1 --2- 172.21.2.101:0/1291006378 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52b00186d0 0x7f52b001abd0 crc :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:20.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.757 7f52d1d84700 1 -- 172.21.2.101:0/1291006378 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52cc0b1430 msgr2=0x7f52cc0b1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:20.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.757 7f52d1d84700 1 --2- 172.21.2.101:0/1291006378 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52cc0b1430 0x7f52cc0b1830 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7f52bc001c60 tx=0x7f52bc00a8d0).stop 2021-02-10T11:32:20.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.758 7f52d1d84700 1 -- 172.21.2.101:0/1291006378 shutdown_connections 2021-02-10T11:32:20.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.758 7f52d1d84700 1 --2- 172.21.2.101:0/1291006378 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f52cc0b1e50 0x7f52cc08a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:20.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.758 7f52d1d84700 1 --2- 172.21.2.101:0/1291006378 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52cc0b1430 0x7f52cc0b1830 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:20.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.758 7f52d1d84700 1 --2- 172.21.2.101:0/1291006378 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52b00186d0 0x7f52b001abd0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:20.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.758 7f52d1d84700 1 --2- 172.21.2.101:0/1291006378 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f52cc192fc0 0x7f52cc192780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:20.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.758 7f52d1d84700 1 -- 172.21.2.101:0/1291006378 >> 172.21.2.101:0/1291006378 conn(0x7f52cc1ac620 msgr2=0x7f52cc19fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:20.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.758 7f52d1d84700 1 -- 172.21.2.101:0/1291006378 shutdown_connections 2021-02-10T11:32:20.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:20.758 7f52d1d84700 1 -- 172.21.2.101:0/1291006378 wait complete. 2021-02-10T11:32:20.776 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-10T11:32:20.776 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:32:21.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.179 7f9ef96c0700 1 Processor -- start 2021-02-10T11:32:21.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.179 7f9ef96c0700 1 -- start start 2021-02-10T11:32:21.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.179 7f9ef96c0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ef41b0470 0x7f9ef41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:21.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.179 7f9ef96c0700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f9ef40ef610 0x7f9ef40efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:21.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.179 7f9ef96c0700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef40f0b50 con 0x7f9ef40f0030 2021-02-10T11:32:21.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.179 7f9ef96c0700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef40e52e0 con 0x7f9ef40ef610 2021-02-10T11:32:21.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.179 7f9ef96c0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40e5490 con 0x7f9ef41b0470 2021-02-10T11:32:21.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.180 7f9ef3ac0700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f9ef40f0030 0x7f9ef40f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51270/0 (socket says 172.21.2.101:51270) 2021-02-10T11:32:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.180 7f9ef3ac0700 1 -- 172.21.2.101:0/4080718489 learned_addr learned my addr 172.21.2.101:0/4080718489 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.180 7f9ef32bf700 1 --2- 172.21.2.101:0/4080718489 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ef41b0470 0x7f9ef41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.180 7f9ef2abe700 1 --2- 172.21.2.101:0/4080718489 >> v2:172.21.2.108:3300/0 conn(0x7f9ef40ef610 0x7f9ef40efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.180 7f9ef22bd700 1 -- 172.21.2.101:0/4080718489 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 643190570 0 0) 0x7f9eec001500 con 0x7f9ef40f0030 2021-02-10T11:32:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.180 7f9ef22bd700 1 -- 172.21.2.101:0/4080718489 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9edc002fe0 con 0x7f9ef40f0030 2021-02-10T11:32:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.180 7f9ef22bd700 1 -- 172.21.2.101:0/4080718489 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 587841202 0 0) 0x7f9eec001270 con 0x7f9ef40f0030 2021-02-10T11:32:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.180 7f9ef22bd700 1 -- 172.21.2.101:0/4080718489 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ef41b0470 msgr2=0x7f9ef41b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.180 7f9ef22bd700 1 --2- 172.21.2.101:0/4080718489 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ef41b0470 0x7f9ef41b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.180 7f9ef22bd700 1 -- 172.21.2.101:0/4080718489 >> v2:172.21.2.108:3300/0 conn(0x7f9ef40ef610 msgr2=0x7f9ef40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.180 7f9ef22bd700 1 --2- 172.21.2.101:0/4080718489 >> v2:172.21.2.108:3300/0 conn(0x7f9ef40ef610 0x7f9ef40efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.180 7f9ef22bd700 1 -- 172.21.2.101:0/4080718489 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef40f0430 con 0x7f9ef40f0030 2021-02-10T11:32:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.180 7f9ef22bd700 1 -- 172.21.2.101:0/4080718489 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f9eec001b50 con 0x7f9ef40f0030 2021-02-10T11:32:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.180 7f9ef22bd700 1 -- 172.21.2.101:0/4080718489 >> v1:172.21.2.101:6789/0 conn(0x7f9ef40f0030 legacy=0x7f9ef40f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.181 7f9ef22bd700 1 --2- 172.21.2.101:0/4080718489 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9edc003a40 0x7f9edc005eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.181 7f9ef22bd700 1 --2- 172.21.2.101:0/4080718489 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9edc04a740 0x7f9edc04cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.181 7f9ef22bd700 1 --2- 172.21.2.101:0/4080718489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9edc04d500 0x7f9edc04fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.181 7f9ef22bd700 1 -- 172.21.2.101:0/4080718489 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc0502d0 con 0x7f9edc04d500 2021-02-10T11:32:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.181 7f9ef22bd700 1 -- 172.21.2.101:0/4080718489 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc0504f0 con 0x7f9edc04a740 2021-02-10T11:32:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.181 7f9ef22bd700 1 -- 172.21.2.101:0/4080718489 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc050710 con 0x7f9edc003a40 2021-02-10T11:32:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.181 7f9ef3ac0700 1 --2- 172.21.2.101:0/4080718489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9edc04d500 0x7f9edc04fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.181 7f9ef2abe700 1 --2- 172.21.2.101:0/4080718489 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9edc04a740 0x7f9edc04cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.181 7f9ef32bf700 1 --2- 172.21.2.101:0/4080718489 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9edc003a40 0x7f9edc005eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.181 7f9ef3ac0700 1 -- 172.21.2.101:0/4080718489 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9edc003a40 msgr2=0x7f9edc005eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.181 7f9ef3ac0700 1 --2- 172.21.2.101:0/4080718489 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9edc003a40 0x7f9edc005eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.181 7f9ef3ac0700 1 -- 172.21.2.101:0/4080718489 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9edc04a740 msgr2=0x7f9edc04cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.181 7f9ef3ac0700 1 --2- 172.21.2.101:0/4080718489 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9edc04a740 0x7f9edc04cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:21.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.181 7f9ef3ac0700 1 -- 172.21.2.101:0/4080718489 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9edc0509b0 con 0x7f9edc04d500 2021-02-10T11:32:21.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.181 7f9ef32bf700 1 --2- 172.21.2.101:0/4080718489 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9edc003a40 0x7f9edc005eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:32:21.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.181 7f9ef3ac0700 1 --2- 172.21.2.101:0/4080718489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9edc04d500 0x7f9edc04fa20 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7f9eec007cb0 tx=0x7f9eec00a310).ready entity=mon.0 client_cookie=32524dcea9b4c1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:21.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.181 7f9ef22bd700 1 -- 172.21.2.101:0/4080718489 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9eec001c10 con 0x7f9edc04d500 2021-02-10T11:32:21.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.181 7f9ef22bd700 1 -- 172.21.2.101:0/4080718489 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eec0070c0 con 0x7f9edc04d500 2021-02-10T11:32:21.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.181 7f9ef22bd700 1 -- 172.21.2.101:0/4080718489 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9eec007510 con 0x7f9edc04d500 2021-02-10T11:32:21.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.181 7f9ef96c0700 1 -- 172.21.2.101:0/4080718489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9edc04d500 msgr2=0x7f9edc04fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:21.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.181 7f9ef96c0700 1 --2- 172.21.2.101:0/4080718489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9edc04d500 0x7f9edc04fa20 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7f9eec007cb0 tx=0x7f9eec00a310).stop 2021-02-10T11:32:21.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.181 7f9ef3ac0700 1 -- 172.21.2.101:0/4080718489 reap_dead start 2021-02-10T11:32:21.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.182 7f9ef96c0700 1 -- 172.21.2.101:0/4080718489 shutdown_connections 2021-02-10T11:32:21.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.182 7f9ef96c0700 1 -- 172.21.2.101:0/4080718489 >> 172.21.2.101:0/4080718489 conn(0x7f9ef40ce3e0 msgr2=0x7f9ef40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:21.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.182 7f9ef96c0700 1 -- 172.21.2.101:0/4080718489 shutdown_connections 2021-02-10T11:32:21.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.182 7f9ef96c0700 1 -- 172.21.2.101:0/4080718489 wait complete. 2021-02-10T11:32:21.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.196 7f9ef96c0700 1 Processor -- start 2021-02-10T11:32:21.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.196 7f9ef96c0700 1 -- start start 2021-02-10T11:32:21.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.197 7f9ef96c0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9ef409b930 0x7f9ef40b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:21.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.197 7f9ef96c0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ef40b8410 0x7f9ef40b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:21.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.197 7f9ef96c0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ef40c1f80 0x7f9ef40c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:21.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.197 7f9ef96c0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40c1a00 con 0x7f9ef40b8410 2021-02-10T11:32:21.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.197 7f9ef96c0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40b8e30 con 0x7f9ef409b930 2021-02-10T11:32:21.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.197 7f9ef96c0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40b9050 con 0x7f9ef40c1f80 2021-02-10T11:32:21.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.197 7f9ef2abe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ef40b8410 0x7f9ef40b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:21.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.197 7f9ef2abe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ef40b8410 0x7f9ef40b8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50786/0 (socket says 172.21.2.101:50786) 2021-02-10T11:32:21.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.197 7f9ef2abe700 1 -- 172.21.2.101:0/2912880355 learned_addr learned my addr 172.21.2.101:0/2912880355 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:21.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.197 7f9ef2abe700 1 -- 172.21.2.101:0/2912880355 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ef40c1f80 msgr2=0x7f9ef40c2380 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:21.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.197 7f9ef2abe700 1 --2- 172.21.2.101:0/2912880355 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ef40c1f80 0x7f9ef40c2380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:21.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.197 7f9ef2abe700 1 -- 172.21.2.101:0/2912880355 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9ef409b930 msgr2=0x7f9ef40b7c20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:21.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.197 7f9ef2abe700 1 --2- 172.21.2.101:0/2912880355 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9ef409b930 0x7f9ef40b7c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:21.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.197 7f9ef2abe700 1 -- 172.21.2.101:0/2912880355 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef40c29a0 con 0x7f9ef40b8410 2021-02-10T11:32:21.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.198 7f9ef2abe700 1 --2- 172.21.2.101:0/2912880355 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ef40b8410 0x7f9ef40b8810 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7f9ee8006f60 tx=0x7f9ee800b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:21.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.198 7f9edbfff700 1 -- 172.21.2.101:0/2912880355 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9ee80009b0 con 0x7f9ef40b8410 2021-02-10T11:32:21.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.198 7f9edbfff700 1 -- 172.21.2.101:0/2912880355 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee800bdf0 con 0x7f9ef40b8410 2021-02-10T11:32:21.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.198 7f9edbfff700 1 -- 172.21.2.101:0/2912880355 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9ee8002030 con 0x7f9ef40b8410 2021-02-10T11:32:21.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.198 7f9ef96c0700 1 -- 172.21.2.101:0/2912880355 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef40c29a0 con 0x7f9ef40b8410 2021-02-10T11:32:21.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.198 7f9ef96c0700 1 -- 172.21.2.101:0/2912880355 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef40c2f30 con 0x7f9ef40b8410 2021-02-10T11:32:21.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.198 7f9edbfff700 1 -- 172.21.2.101:0/2912880355 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9ee8006b00 con 0x7f9ef40b8410 2021-02-10T11:32:21.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.199 7f9edbfff700 1 --2- 172.21.2.101:0/2912880355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ed40186c0 0x7f9ed401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:21.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.199 7f9edbfff700 1 -- 172.21.2.101:0/2912880355 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f9ee8032120 con 0x7f9ef40b8410 2021-02-10T11:32:21.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.199 7f9ef32bf700 1 --2- 172.21.2.101:0/2912880355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ed40186c0 0x7f9ed401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:21.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.199 7f9ef32bf700 1 --2- 172.21.2.101:0/2912880355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ed40186c0 0x7f9ed401abc0 crc :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:21.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.229 7f9ef96c0700 1 -- 172.21.2.101:0/2912880355 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef4105b30 con 0x7f9ef40b8410 2021-02-10T11:32:21.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.232 7f9edbfff700 1 -- 172.21.2.101:0/2912880355 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9ee802ef80 con 0x7f9ef40b8410 2021-02-10T11:32:21.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.359 7f9ef96c0700 1 -- 172.21.2.101:0/2912880355 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f9ef40f0b50 con 0x7f9ef40b8410 2021-02-10T11:32:21.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.360 7f9edbfff700 1 -- 172.21.2.101:0/2912880355 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v42) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9ee802ef80 con 0x7f9ef40b8410 2021-02-10T11:32:21.370 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:32:21.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.362 7f9ef96c0700 1 -- 172.21.2.101:0/2912880355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ed40186c0 msgr2=0x7f9ed401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:21.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.362 7f9ef96c0700 1 --2- 172.21.2.101:0/2912880355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ed40186c0 0x7f9ed401abc0 crc :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:21.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.362 7f9ef96c0700 1 -- 172.21.2.101:0/2912880355 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ef40b8410 msgr2=0x7f9ef40b8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:21.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.362 7f9ef96c0700 1 --2- 172.21.2.101:0/2912880355 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ef40b8410 0x7f9ef40b8810 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7f9ee8006f60 tx=0x7f9ee800b970).stop 2021-02-10T11:32:21.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.362 7f9ef96c0700 1 -- 172.21.2.101:0/2912880355 shutdown_connections 2021-02-10T11:32:21.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.362 7f9ef96c0700 1 --2- 172.21.2.101:0/2912880355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ed40186c0 0x7f9ed401abc0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:21.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.362 7f9ef96c0700 1 --2- 172.21.2.101:0/2912880355 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ef40c1f80 0x7f9ef40c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:21.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.362 7f9ef96c0700 1 --2- 172.21.2.101:0/2912880355 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ef40b8410 0x7f9ef40b8810 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:21.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.362 7f9ef96c0700 1 --2- 172.21.2.101:0/2912880355 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9ef409b930 0x7f9ef40b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:21.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.362 7f9ef96c0700 1 -- 172.21.2.101:0/2912880355 >> 172.21.2.101:0/2912880355 conn(0x7f9ef4094e10 msgr2=0x7f9ef40a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:21.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.362 7f9ef96c0700 1 -- 172.21.2.101:0/2912880355 shutdown_connections 2021-02-10T11:32:21.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.362 7f9ef96c0700 1 -- 172.21.2.101:0/2912880355 wait complete. 2021-02-10T11:32:21.381 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --format=json-pretty osd lspools 2021-02-10T11:32:21.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.781 7fbe2bbb3700 1 Processor -- start 2021-02-10T11:32:21.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.781 7fbe2bbb3700 1 -- start start 2021-02-10T11:32:21.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.782 7fbe2bbb3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbe240e5470 0x7fbe2407e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:21.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.782 7fbe2bbb3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe2407f300 0x7fbe2407f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:21.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.782 7fbe2bbb3700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe2407ed80 con 0x7fbe240e5070 2021-02-10T11:32:21.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.782 7fbe2bbb3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe241b4010 con 0x7fbe240e5470 2021-02-10T11:32:21.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.782 7fbe2bbb3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe240d1790 con 0x7fbe2407f300 2021-02-10T11:32:21.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.782 7fbe291ce700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbe240e5470 0x7fbe2407e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:21.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.782 7fbe299cf700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fbe240e5070 0x7fbe241b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:44698/0 (socket says 172.21.2.101:44698) 2021-02-10T11:32:21.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.782 7fbe299cf700 1 -- 172.21.2.101:0/1756579398 learned_addr learned my addr 172.21.2.101:0/1756579398 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:21.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.782 7fbe2a1d0700 1 --2- 172.21.2.101:0/1756579398 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe2407f300 0x7fbe2407f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:21.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.783 7fbe291ce700 1 -- 172.21.2.101:0/1756579398 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe2407f300 msgr2=0x7fbe2407f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:21.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.783 7fbe291ce700 1 --2- 172.21.2.101:0/1756579398 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe2407f300 0x7fbe2407f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:21.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.783 7fbe291ce700 1 -- 172.21.2.101:0/1756579398 >> v1:172.21.2.108:6789/0 conn(0x7fbe240e5070 legacy=0x7fbe241b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:21.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.783 7fbe289cd700 1 -- 172.21.2.101:0/1756579398 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1297485873 0 0) 0x7fbe20001190 con 0x7fbe240e5070 2021-02-10T11:32:21.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.783 7fbe291ce700 1 -- 172.21.2.101:0/1756579398 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe240d19b0 con 0x7fbe240e5470 2021-02-10T11:32:21.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.783 7fbe2a1d0700 1 --2- 172.21.2.101:0/1756579398 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe2407f300 0x7fbe2407f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:32:21.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.783 7fbe291ce700 1 --2- 172.21.2.101:0/1756579398 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbe240e5470 0x7fbe2407e6a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fbe14006650 tx=0x7fbe14009a80).ready entity=mon.1 client_cookie=92e48978e86b0a32 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:21.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.783 7fbe289cd700 1 -- 172.21.2.101:0/1756579398 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbe140009b0 con 0x7fbe240e5470 2021-02-10T11:32:21.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.784 7fbe289cd700 1 -- 172.21.2.101:0/1756579398 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1400c0c0 con 0x7fbe240e5470 2021-02-10T11:32:21.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.784 7fbe289cd700 1 -- 172.21.2.101:0/1756579398 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbe1400c5e0 con 0x7fbe240e5470 2021-02-10T11:32:21.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.784 7fbe2bbb3700 1 -- 172.21.2.101:0/1756579398 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbe240e5470 msgr2=0x7fbe2407e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:21.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.784 7fbe2bbb3700 1 --2- 172.21.2.101:0/1756579398 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbe240e5470 0x7fbe2407e6a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fbe14006650 tx=0x7fbe14009a80).stop 2021-02-10T11:32:21.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.784 7fbe2bbb3700 1 -- 172.21.2.101:0/1756579398 shutdown_connections 2021-02-10T11:32:21.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.784 7fbe2bbb3700 1 --2- 172.21.2.101:0/1756579398 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe2407f300 0x7fbe2407f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:21.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.784 7fbe2bbb3700 1 --2- 172.21.2.101:0/1756579398 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbe240e5470 0x7fbe2407e6a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:21.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.784 7fbe2bbb3700 1 -- 172.21.2.101:0/1756579398 >> 172.21.2.101:0/1756579398 conn(0x7fbe240ce690 msgr2=0x7fbe240e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:21.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.785 7fbe2bbb3700 1 -- 172.21.2.101:0/1756579398 shutdown_connections 2021-02-10T11:32:21.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.785 7fbe2bbb3700 1 -- 172.21.2.101:0/1756579398 wait complete. 2021-02-10T11:32:21.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.799 7fbe2bbb3700 1 Processor -- start 2021-02-10T11:32:21.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.799 7fbe2bbb3700 1 -- start start 2021-02-10T11:32:21.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.799 7fbe2bbb3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbe241918e0 0x7fbe24189c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:21.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.799 7fbe2bbb3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe2418a2a0 0x7fbe2418a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:21.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.799 7fbe2bbb3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe2418acc0 0x7fbe24190070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:21.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.799 7fbe2bbb3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe241a2f00 con 0x7fbe2418acc0 2021-02-10T11:32:21.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.799 7fbe2bbb3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe24191ce0 con 0x7fbe241918e0 2021-02-10T11:32:21.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.799 7fbe2bbb3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe24191f00 con 0x7fbe2418a2a0 2021-02-10T11:32:21.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.800 7fbe2a1d0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe2418acc0 0x7fbe24190070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.800 7fbe2a1d0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe2418acc0 0x7fbe24190070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50804/0 (socket says 172.21.2.101:50804) 2021-02-10T11:32:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.800 7fbe2a1d0700 1 -- 172.21.2.101:0/1021637130 learned_addr learned my addr 172.21.2.101:0/1021637130 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.800 7fbe2a1d0700 1 -- 172.21.2.101:0/1021637130 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe2418a2a0 msgr2=0x7fbe2418a6a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.800 7fbe2a1d0700 1 --2- 172.21.2.101:0/1021637130 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe2418a2a0 0x7fbe2418a6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.800 7fbe2a1d0700 1 -- 172.21.2.101:0/1021637130 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbe241918e0 msgr2=0x7fbe24189c80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.800 7fbe2a1d0700 1 --2- 172.21.2.101:0/1021637130 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbe241918e0 0x7fbe24189c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.800 7fbe2a1d0700 1 -- 172.21.2.101:0/1021637130 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe24190690 con 0x7fbe2418acc0 2021-02-10T11:32:21.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.800 7fbe2a1d0700 1 --2- 172.21.2.101:0/1021637130 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe2418acc0 0x7fbe24190070 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7fbe1c00b970 tx=0x7fbe1c00bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:21.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.800 7fbe1a7fc700 1 -- 172.21.2.101:0/1021637130 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbe1c000f90 con 0x7fbe2418acc0 2021-02-10T11:32:21.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.800 7fbe1a7fc700 1 -- 172.21.2.101:0/1021637130 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1c006aa0 con 0x7fbe2418acc0 2021-02-10T11:32:21.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.800 7fbe1a7fc700 1 -- 172.21.2.101:0/1021637130 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbe1c0015f0 con 0x7fbe2418acc0 2021-02-10T11:32:21.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.800 7fbe2bbb3700 1 -- 172.21.2.101:0/1021637130 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe24190690 con 0x7fbe2418acc0 2021-02-10T11:32:21.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.800 7fbe2bbb3700 1 -- 172.21.2.101:0/1021637130 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe24190690 con 0x7fbe2418acc0 2021-02-10T11:32:21.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.801 7fbe1a7fc700 1 -- 172.21.2.101:0/1021637130 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbe1c000e70 con 0x7fbe2418acc0 2021-02-10T11:32:21.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.801 7fbe1a7fc700 1 --2- 172.21.2.101:0/1021637130 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe0c018730 0x7fbe0c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:21.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.801 7fbe1a7fc700 1 -- 172.21.2.101:0/1021637130 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fbe1c032810 con 0x7fbe2418acc0 2021-02-10T11:32:21.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.801 7fbe299cf700 1 --2- 172.21.2.101:0/1021637130 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe0c018730 0x7fbe0c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:21.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.802 7fbe299cf700 1 --2- 172.21.2.101:0/1021637130 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe0c018730 0x7fbe0c01ac10 crc :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:21.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.832 7fbe2bbb3700 1 -- 172.21.2.101:0/1021637130 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe24189750 con 0x7fbe2418acc0 2021-02-10T11:32:21.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.834 7fbe1a7fc700 1 -- 172.21.2.101:0/1021637130 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbe1c0066f0 con 0x7fbe2418acc0 2021-02-10T11:32:21.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.958 7fbe2bbb3700 1 -- 172.21.2.101:0/1021637130 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbe24189750 con 0x7fbe2418acc0 2021-02-10T11:32:21.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.959 7fbe1a7fc700 1 -- 172.21.2.101:0/1021637130 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) v1 ==== 84+0+64 (secure 0 0 0) 0x7fbe1c006aa0 con 0x7fbe2418acc0 2021-02-10T11:32:21.969 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:32:21.970 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:32:21.970 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:21.970 INFO:teuthology.orchestra.run.gibba001.stdout: "poolnum": 1, 2021-02-10T11:32:21.970 INFO:teuthology.orchestra.run.gibba001.stdout: "poolname": "rbd" 2021-02-10T11:32:21.970 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:21.970 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:32:21.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.961 7fbe2bbb3700 1 -- 172.21.2.101:0/1021637130 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe0c018730 msgr2=0x7fbe0c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:21.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.961 7fbe2bbb3700 1 --2- 172.21.2.101:0/1021637130 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe0c018730 0x7fbe0c01ac10 crc :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:21.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.961 7fbe2bbb3700 1 -- 172.21.2.101:0/1021637130 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe2418acc0 msgr2=0x7fbe24190070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:21.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.961 7fbe2bbb3700 1 --2- 172.21.2.101:0/1021637130 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe2418acc0 0x7fbe24190070 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7fbe1c00b970 tx=0x7fbe1c00bd10).stop 2021-02-10T11:32:21.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.961 7fbe2bbb3700 1 -- 172.21.2.101:0/1021637130 shutdown_connections 2021-02-10T11:32:21.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.961 7fbe2bbb3700 1 --2- 172.21.2.101:0/1021637130 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe0c018730 0x7fbe0c01ac10 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:21.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.961 7fbe2bbb3700 1 --2- 172.21.2.101:0/1021637130 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe2418a2a0 0x7fbe2418a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:21.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.961 7fbe2bbb3700 1 --2- 172.21.2.101:0/1021637130 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe2418acc0 0x7fbe24190070 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:21.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.961 7fbe2bbb3700 1 --2- 172.21.2.101:0/1021637130 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbe241918e0 0x7fbe24189c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:21.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.961 7fbe2bbb3700 1 -- 172.21.2.101:0/1021637130 >> 172.21.2.101:0/1021637130 conn(0x7fbe241a31d0 msgr2=0x7fbe2408b830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:21.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.961 7fbe2bbb3700 1 -- 172.21.2.101:0/1021637130 shutdown_connections 2021-02-10T11:32:21.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:21.961 7fbe2bbb3700 1 -- 172.21.2.101:0/1021637130 wait complete. 2021-02-10T11:32:21.979 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-02-10T11:32:21.980 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-02-10T11:32:22.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.383 7f842dda3700 1 Processor -- start 2021-02-10T11:32:22.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.383 7f842dda3700 1 -- start start 2021-02-10T11:32:22.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.383 7f842dda3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f84281b0470 0x7f84281b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:22.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.383 7f842dda3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f84280f20e0 0x7f84280f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:22.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.383 7f842dda3700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84280f0b50 con 0x7f84280ef610 2021-02-10T11:32:22.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.383 7f842dda3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84280f0820 con 0x7f84280f20e0 2021-02-10T11:32:22.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.383 7f842dda3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84280f24e0 con 0x7f84281b0470 2021-02-10T11:32:22.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.384 7f8426ffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f84280ef610 0x7f84280efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51302/0 (socket says 172.21.2.101:51302) 2021-02-10T11:32:22.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.384 7f8426ffd700 1 -- 172.21.2.101:0/3658871778 learned_addr learned my addr 172.21.2.101:0/3658871778 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:22.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.384 7f8427fff700 1 --2- 172.21.2.101:0/3658871778 >> v2:172.21.2.101:3300/0 conn(0x7f84280f20e0 0x7f84280f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:22.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.384 7f8427fff700 1 -- 172.21.2.101:0/3658871778 >> v2:172.21.2.108:3300/0 conn(0x7f84281b0470 msgr2=0x7f84281b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:22.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.384 7f8427fff700 1 --2- 172.21.2.101:0/3658871778 >> v2:172.21.2.108:3300/0 conn(0x7f84281b0470 0x7f84281b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:22.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.384 7f8427fff700 1 -- 172.21.2.101:0/3658871778 >> v1:172.21.2.101:6789/0 conn(0x7f84280ef610 legacy=0x7f84280efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:22.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.384 7f8427fff700 1 -- 172.21.2.101:0/3658871778 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84280e52e0 con 0x7f84280f20e0 2021-02-10T11:32:22.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.385 7f8427fff700 1 --2- 172.21.2.101:0/3658871778 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84280f20e0 0x7f84280f0200 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 rx=0x7f8420006e30 tx=0x7f8420009e60).ready entity=mon.0 client_cookie=73d85b0eb324be69 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:22.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.385 7f84267fc700 1 -- 172.21.2.101:0/3658871778 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8420006a50 con 0x7f84280f20e0 2021-02-10T11:32:22.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.385 7f84267fc700 1 -- 172.21.2.101:0/3658871778 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842000b300 con 0x7f84280f20e0 2021-02-10T11:32:22.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.385 7f84267fc700 1 -- 172.21.2.101:0/3658871778 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f842000b7e0 con 0x7f84280f20e0 2021-02-10T11:32:22.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.385 7f842dda3700 1 -- 172.21.2.101:0/3658871778 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84280f20e0 msgr2=0x7f84280f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:22.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.385 7f842dda3700 1 --2- 172.21.2.101:0/3658871778 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84280f20e0 0x7f84280f0200 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 rx=0x7f8420006e30 tx=0x7f8420009e60).stop 2021-02-10T11:32:22.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.385 7f842dda3700 1 -- 172.21.2.101:0/3658871778 shutdown_connections 2021-02-10T11:32:22.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.386 7f842dda3700 1 --2- 172.21.2.101:0/3658871778 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84280f20e0 0x7f84280f0200 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:22.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.386 7f842dda3700 1 --2- 172.21.2.101:0/3658871778 >> v2:172.21.2.108:3300/0 conn(0x7f84281b0470 0x7f84281b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:22.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.386 7f842dda3700 1 -- 172.21.2.101:0/3658871778 >> 172.21.2.101:0/3658871778 conn(0x7f84280ce3e0 msgr2=0x7f84280e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:22.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.386 7f842dda3700 1 -- 172.21.2.101:0/3658871778 shutdown_connections 2021-02-10T11:32:22.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.386 7f842dda3700 1 -- 172.21.2.101:0/3658871778 wait complete. 2021-02-10T11:32:22.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.400 7f842dda3700 1 Processor -- start 2021-02-10T11:32:22.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.400 7f842dda3700 1 -- start start 2021-02-10T11:32:22.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.401 7f842dda3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f84280acce0 0x7f84280a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:22.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.401 7f842dda3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f84280a4010 0x7f84280a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:22.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.401 7f842dda3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84280eec20 0x7f84280ef020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:22.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.401 7f842dda3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84280a5330 con 0x7f84280eec20 2021-02-10T11:32:22.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.401 7f842dda3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84280ad0e0 con 0x7f84280acce0 2021-02-10T11:32:22.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.401 7f842dda3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84280ef640 con 0x7f84280a4010 2021-02-10T11:32:22.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.401 7f8427fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84280eec20 0x7f84280ef020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:22.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.401 7f8427fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84280eec20 0x7f84280ef020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50816/0 (socket says 172.21.2.101:50816) 2021-02-10T11:32:22.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.401 7f8427fff700 1 -- 172.21.2.101:0/3428068805 learned_addr learned my addr 172.21.2.101:0/3428068805 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:22.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.401 7f8427fff700 1 -- 172.21.2.101:0/3428068805 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f84280a4010 msgr2=0x7f84280a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:22.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.401 7f8427fff700 1 --2- 172.21.2.101:0/3428068805 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f84280a4010 0x7f84280a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:22.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.401 7f8427fff700 1 -- 172.21.2.101:0/3428068805 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f84280acce0 msgr2=0x7f84280a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:22.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.401 7f8427fff700 1 --2- 172.21.2.101:0/3428068805 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f84280acce0 0x7f84280a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:22.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.401 7f8427fff700 1 -- 172.21.2.101:0/3428068805 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84280ef860 con 0x7f84280eec20 2021-02-10T11:32:22.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.402 7f8427fff700 1 --2- 172.21.2.101:0/3428068805 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84280eec20 0x7f84280ef020 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7f842000b970 tx=0x7f842000bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:22.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.402 7f841ffff700 1 -- 172.21.2.101:0/3428068805 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8420009380 con 0x7f84280eec20 2021-02-10T11:32:22.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.402 7f841ffff700 1 -- 172.21.2.101:0/3428068805 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8420005820 con 0x7f84280eec20 2021-02-10T11:32:22.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.402 7f841ffff700 1 -- 172.21.2.101:0/3428068805 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8420005d00 con 0x7f84280eec20 2021-02-10T11:32:22.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.402 7f842dda3700 1 -- 172.21.2.101:0/3428068805 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84280ef860 con 0x7f84280eec20 2021-02-10T11:32:22.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.402 7f842dda3700 1 -- 172.21.2.101:0/3428068805 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84280ef860 con 0x7f84280eec20 2021-02-10T11:32:22.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.402 7f841ffff700 1 -- 172.21.2.101:0/3428068805 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8420005a00 con 0x7f84280eec20 2021-02-10T11:32:22.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.402 7f841ffff700 1 --2- 172.21.2.101:0/3428068805 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f840c018730 0x7f840c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:22.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.402 7f841ffff700 1 -- 172.21.2.101:0/3428068805 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f8420032680 con 0x7f84280eec20 2021-02-10T11:32:22.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.403 7f84277fe700 1 --2- 172.21.2.101:0/3428068805 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f840c018730 0x7f840c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:22.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.403 7f84277fe700 1 --2- 172.21.2.101:0/3428068805 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f840c018730 0x7f840c01ac10 crc :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:22.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.433 7f842dda3700 1 -- 172.21.2.101:0/3428068805 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84280edd10 con 0x7f84280eec20 2021-02-10T11:32:22.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.436 7f841ffff700 1 -- 172.21.2.101:0/3428068805 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8420009380 con 0x7f84280eec20 2021-02-10T11:32:22.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.563 7f842dda3700 1 -- 172.21.2.101:0/3428068805 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"} v 0) v1 -- 0x7f84280e52e0 con 0x7f84280eec20 2021-02-10T11:32:22.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.976 7f841ffff700 1 -- 172.21.2.101:0/3428068805 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"}]=0 pool 'cephfs_metadata' created v43) v1 ==== 133+0+0 (secure 0 0 0) 0x7f8420005820 con 0x7f84280eec20 2021-02-10T11:32:22.988 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' created 2021-02-10T11:32:22.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.980 7f842dda3700 1 -- 172.21.2.101:0/3428068805 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f840c018730 msgr2=0x7f840c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:22.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.980 7f842dda3700 1 --2- 172.21.2.101:0/3428068805 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f840c018730 0x7f840c01ac10 crc :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:22.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.980 7f842dda3700 1 -- 172.21.2.101:0/3428068805 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84280eec20 msgr2=0x7f84280ef020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:22.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.980 7f842dda3700 1 --2- 172.21.2.101:0/3428068805 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84280eec20 0x7f84280ef020 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7f842000b970 tx=0x7f842000bd10).stop 2021-02-10T11:32:22.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.980 7f842dda3700 1 -- 172.21.2.101:0/3428068805 shutdown_connections 2021-02-10T11:32:22.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.980 7f842dda3700 1 --2- 172.21.2.101:0/3428068805 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f840c018730 0x7f840c01ac10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:22.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.980 7f842dda3700 1 --2- 172.21.2.101:0/3428068805 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f84280a4010 0x7f84280a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:22.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.980 7f842dda3700 1 --2- 172.21.2.101:0/3428068805 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84280eec20 0x7f84280ef020 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:22.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.980 7f842dda3700 1 --2- 172.21.2.101:0/3428068805 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f84280acce0 0x7f84280a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:22.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.980 7f842dda3700 1 -- 172.21.2.101:0/3428068805 >> 172.21.2.101:0/3428068805 conn(0x7f84280c40e0 msgr2=0x7f84280f22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:22.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.980 7f842dda3700 1 -- 172.21.2.101:0/3428068805 shutdown_connections 2021-02-10T11:32:22.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:22.980 7f842dda3700 1 -- 172.21.2.101:0/3428068805 wait complete. 2021-02-10T11:32:22.999 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_data 8 2021-02-10T11:32:23.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.400 7fa20a8c9700 1 Processor -- start 2021-02-10T11:32:23.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.400 7fa20a8c9700 1 -- start start 2021-02-10T11:32:23.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.401 7fa20a8c9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2041b0470 0x7fa2041b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:23.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.401 7fa20a8c9700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa2040ef610 0x7fa2040efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:23.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.401 7fa20a8c9700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2040f0b50 con 0x7fa2040f0030 2021-02-10T11:32:23.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.401 7fa20a8c9700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2040e52e0 con 0x7fa2040ef610 2021-02-10T11:32:23.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.401 7fa20a8c9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2040e5490 con 0x7fa2041b0470 2021-02-10T11:32:23.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.401 7fa2037fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa2040ef610 0x7fa2040efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:23.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.401 7fa203fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2041b0470 0x7fa2041b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:23.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.401 7fa208ee6700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fa2040f0030 0x7fa2040f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:44724/0 (socket says 172.21.2.101:44724) 2021-02-10T11:32:23.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.401 7fa208ee6700 1 -- 172.21.2.101:0/1427409067 learned_addr learned my addr 172.21.2.101:0/1427409067 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:23.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.402 7fa202ffd700 1 -- 172.21.2.101:0/1427409067 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102798455 0 0) 0x7fa1fc001440 con 0x7fa2040f0030 2021-02-10T11:32:23.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.402 7fa202ffd700 1 -- 172.21.2.101:0/1427409067 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1ec002fe0 con 0x7fa2040f0030 2021-02-10T11:32:23.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.402 7fa203fff700 1 -- 172.21.2.101:0/1427409067 >> v2:172.21.2.108:3300/0 conn(0x7fa2040ef610 msgr2=0x7fa2040efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:23.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.402 7fa203fff700 1 --2- 172.21.2.101:0/1427409067 >> v2:172.21.2.108:3300/0 conn(0x7fa2040ef610 0x7fa2040efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:23.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.402 7fa203fff700 1 -- 172.21.2.101:0/1427409067 >> v1:172.21.2.108:6789/0 conn(0x7fa2040f0030 legacy=0x7fa2040f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:23.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.402 7fa203fff700 1 -- 172.21.2.101:0/1427409067 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2040f0430 con 0x7fa2041b0470 2021-02-10T11:32:23.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.402 7fa2037fe700 1 --2- 172.21.2.101:0/1427409067 >> v2:172.21.2.108:3300/0 conn(0x7fa2040ef610 0x7fa2040efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:32:23.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.402 7fa203fff700 1 --2- 172.21.2.101:0/1427409067 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2041b0470 0x7fa2041b0870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fa1f4006650 tx=0x7fa1f4009a80).ready entity=mon.2 client_cookie=85a77c49d705de62 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:23.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.403 7fa202ffd700 1 -- 172.21.2.101:0/1427409067 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa1f40009b0 con 0x7fa2041b0470 2021-02-10T11:32:23.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.403 7fa202ffd700 1 -- 172.21.2.101:0/1427409067 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f400c0c0 con 0x7fa2041b0470 2021-02-10T11:32:23.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.403 7fa202ffd700 1 -- 172.21.2.101:0/1427409067 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa1f400c5e0 con 0x7fa2041b0470 2021-02-10T11:32:23.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.403 7fa20a8c9700 1 -- 172.21.2.101:0/1427409067 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2041b0470 msgr2=0x7fa2041b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:23.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.403 7fa20a8c9700 1 --2- 172.21.2.101:0/1427409067 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2041b0470 0x7fa2041b0870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fa1f4006650 tx=0x7fa1f4009a80).stop 2021-02-10T11:32:23.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.403 7fa20a8c9700 1 -- 172.21.2.101:0/1427409067 shutdown_connections 2021-02-10T11:32:23.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.403 7fa20a8c9700 1 --2- 172.21.2.101:0/1427409067 >> v2:172.21.2.108:3300/0 conn(0x7fa2040ef610 0x7fa2040efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:23.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.403 7fa20a8c9700 1 --2- 172.21.2.101:0/1427409067 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2041b0470 0x7fa2041b0870 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:23.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.403 7fa20a8c9700 1 -- 172.21.2.101:0/1427409067 >> 172.21.2.101:0/1427409067 conn(0x7fa2040ce3e0 msgr2=0x7fa2040e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:23.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.403 7fa20a8c9700 1 -- 172.21.2.101:0/1427409067 shutdown_connections 2021-02-10T11:32:23.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.403 7fa20a8c9700 1 -- 172.21.2.101:0/1427409067 wait complete. 2021-02-10T11:32:23.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.418 7fa20a8c9700 1 Processor -- start 2021-02-10T11:32:23.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.418 7fa20a8c9700 1 -- start start 2021-02-10T11:32:23.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.418 7fa20a8c9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2040a0420 0x7fa2040a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:23.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.418 7fa20a8c9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa204189d00 0x7fa20418a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:23.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.418 7fa20a8c9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa20418a720 0x7fa2040b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:23.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.418 7fa20a8c9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa204189800 con 0x7fa204189d00 2021-02-10T11:32:23.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.418 7fa20a8c9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20418ab20 con 0x7fa20418a720 2021-02-10T11:32:23.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.418 7fa20a8c9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2040a1140 con 0x7fa2040a0420 2021-02-10T11:32:23.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.418 7fa2037fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa204189d00 0x7fa20418a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:23.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.418 7fa2037fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa204189d00 0x7fa20418a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50826/0 (socket says 172.21.2.101:50826) 2021-02-10T11:32:23.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.418 7fa2037fe700 1 -- 172.21.2.101:0/1769377582 learned_addr learned my addr 172.21.2.101:0/1769377582 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:23.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.419 7fa2037fe700 1 -- 172.21.2.101:0/1769377582 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2040a0420 msgr2=0x7fa2040a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:23.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.419 7fa2037fe700 1 --2- 172.21.2.101:0/1769377582 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2040a0420 0x7fa2040a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:23.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.419 7fa2037fe700 1 -- 172.21.2.101:0/1769377582 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa20418a720 msgr2=0x7fa2040b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:23.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.419 7fa2037fe700 1 --2- 172.21.2.101:0/1769377582 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa20418a720 0x7fa2040b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:23.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.419 7fa2037fe700 1 -- 172.21.2.101:0/1769377582 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2040b8200 con 0x7fa204189d00 2021-02-10T11:32:23.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.419 7fa2037fe700 1 --2- 172.21.2.101:0/1769377582 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa204189d00 0x7fa20418a100 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7fa1f8006f40 tx=0x7fa1f800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:23.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.419 7fa200ff9700 1 -- 172.21.2.101:0/1769377582 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa1f800bd80 con 0x7fa204189d00 2021-02-10T11:32:23.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.419 7fa200ff9700 1 -- 172.21.2.101:0/1769377582 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f8000c90 con 0x7fa204189d00 2021-02-10T11:32:23.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.419 7fa200ff9700 1 -- 172.21.2.101:0/1769377582 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa1f80022b0 con 0x7fa204189d00 2021-02-10T11:32:23.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.419 7fa20a8c9700 1 -- 172.21.2.101:0/1769377582 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2040b8200 con 0x7fa204189d00 2021-02-10T11:32:23.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.419 7fa20a8c9700 1 -- 172.21.2.101:0/1769377582 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2040b8790 con 0x7fa204189d00 2021-02-10T11:32:23.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.420 7fa200ff9700 1 -- 172.21.2.101:0/1769377582 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa1f800a000 con 0x7fa204189d00 2021-02-10T11:32:23.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.420 7fa200ff9700 1 --2- 172.21.2.101:0/1769377582 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1e40186c0 0x7fa1e401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:23.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.420 7fa200ff9700 1 -- 172.21.2.101:0/1769377582 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5107+0+0 (secure 0 0 0) 0x7fa1f8032350 con 0x7fa204189d00 2021-02-10T11:32:23.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.420 7fa203fff700 1 --2- 172.21.2.101:0/1769377582 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1e40186c0 0x7fa1e401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:23.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.420 7fa203fff700 1 --2- 172.21.2.101:0/1769377582 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1e40186c0 0x7fa1e401abc0 crc :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:23.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.450 7fa20a8c9700 1 -- 172.21.2.101:0/1769377582 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa204105ae0 con 0x7fa204189d00 2021-02-10T11:32:23.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.453 7fa200ff9700 1 -- 172.21.2.101:0/1769377582 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa1f80009b0 con 0x7fa204189d00 2021-02-10T11:32:23.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.580 7fa20a8c9700 1 -- 172.21.2.101:0/1769377582 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"} v 0) v1 -- 0x7fa2040f0b50 con 0x7fa204189d00 2021-02-10T11:32:23.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.985 7fa200ff9700 1 -- 172.21.2.101:0/1769377582 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"}]=0 pool 'cephfs_data' created v45) v1 ==== 125+0+0 (secure 0 0 0) 0x7fa1f800a000 con 0x7fa204189d00 2021-02-10T11:32:24.006 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' created 2021-02-10T11:32:24.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.998 7fa20a8c9700 1 -- 172.21.2.101:0/1769377582 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1e40186c0 msgr2=0x7fa1e401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:24.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.998 7fa20a8c9700 1 --2- 172.21.2.101:0/1769377582 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1e40186c0 0x7fa1e401abc0 crc :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:24.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.998 7fa20a8c9700 1 -- 172.21.2.101:0/1769377582 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa204189d00 msgr2=0x7fa20418a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:24.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.998 7fa20a8c9700 1 --2- 172.21.2.101:0/1769377582 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa204189d00 0x7fa20418a100 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7fa1f8006f40 tx=0x7fa1f800b990).stop 2021-02-10T11:32:24.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.998 7fa20a8c9700 1 -- 172.21.2.101:0/1769377582 shutdown_connections 2021-02-10T11:32:24.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.998 7fa20a8c9700 1 --2- 172.21.2.101:0/1769377582 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa20418a720 0x7fa2040b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:24.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.998 7fa20a8c9700 1 --2- 172.21.2.101:0/1769377582 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa204189d00 0x7fa20418a100 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:24.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.998 7fa20a8c9700 1 --2- 172.21.2.101:0/1769377582 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1e40186c0 0x7fa1e401abc0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:24.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.998 7fa20a8c9700 1 --2- 172.21.2.101:0/1769377582 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2040a0420 0x7fa2040a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:24.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.998 7fa20a8c9700 1 -- 172.21.2.101:0/1769377582 >> 172.21.2.101:0/1769377582 conn(0x7fa204094e10 msgr2=0x7fa2040a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:24.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.998 7fa20a8c9700 1 -- 172.21.2.101:0/1769377582 shutdown_connections 2021-02-10T11:32:24.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:23.998 7fa20a8c9700 1 -- 172.21.2.101:0/1769377582 wait complete. 2021-02-10T11:32:24.017 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-02-10T11:32:24.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.419 7faedf55e700 1 Processor -- start 2021-02-10T11:32:24.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.420 7faedf55e700 1 -- start start 2021-02-10T11:32:24.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.420 7faedf55e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7faed810aff0 0x7faed80cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:24.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.420 7faedf55e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed80fe1c0 con 0x7faed80d1c70 2021-02-10T11:32:24.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.420 7faedf55e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed81b0cb0 con 0x7faed80d1870 2021-02-10T11:32:24.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.420 7faedf55e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed8109a60 con 0x7faed810aff0 2021-02-10T11:32:24.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.420 7faeddb7b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7faed810aff0 0x7faed80cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:24.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.420 7faeddb7b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7faed810aff0 0x7faed80cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50836/0 (socket says 172.21.2.101:50836) 2021-02-10T11:32:24.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.420 7faedcb79700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7faed80d1c70 0x7faed80cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51328/0 (socket says 172.21.2.101:51328) 2021-02-10T11:32:24.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.420 7faeddb7b700 1 -- 172.21.2.101:0/3205486173 learned_addr learned my addr 172.21.2.101:0/3205486173 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:24.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.421 7faeddb7b700 1 -- 172.21.2.101:0/3205486173 >> v1:172.21.2.108:6789/0 conn(0x7faed80d1870 legacy=0x7faed81b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:32:24.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.421 7faeddb7b700 1 -- 172.21.2.101:0/3205486173 >> v1:172.21.2.101:6789/0 conn(0x7faed80d1c70 legacy=0x7faed80cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:24.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.421 7faeddb7b700 1 -- 172.21.2.101:0/3205486173 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed810b3f0 con 0x7faed810aff0 2021-02-10T11:32:24.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.421 7faeddb7b700 1 --2- 172.21.2.101:0/3205486173 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faed810aff0 0x7faed80cf750 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7faed0006f00 tx=0x7faed0009f10).ready entity=mon.0 client_cookie=79e2733d9c6fcf31 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:24.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.421 7faecbfff700 1 -- 172.21.2.101:0/3205486173 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faed000b3b0 con 0x7faed810aff0 2021-02-10T11:32:24.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.421 7faecbfff700 1 -- 172.21.2.101:0/3205486173 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed000b630 con 0x7faed810aff0 2021-02-10T11:32:24.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.421 7faecbfff700 1 -- 172.21.2.101:0/3205486173 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faed000bb50 con 0x7faed810aff0 2021-02-10T11:32:24.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.421 7faedf55e700 1 -- 172.21.2.101:0/3205486173 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faed810aff0 msgr2=0x7faed80cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:24.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.421 7faedf55e700 1 --2- 172.21.2.101:0/3205486173 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faed810aff0 0x7faed80cf750 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7faed0006f00 tx=0x7faed0009f10).stop 2021-02-10T11:32:24.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.422 7faedf55e700 1 -- 172.21.2.101:0/3205486173 shutdown_connections 2021-02-10T11:32:24.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.422 7faedf55e700 1 --2- 172.21.2.101:0/3205486173 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faed810aff0 0x7faed80cf750 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:24.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.422 7faedf55e700 1 -- 172.21.2.101:0/3205486173 >> 172.21.2.101:0/3205486173 conn(0x7faed80ce420 msgr2=0x7faed80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:24.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.422 7faedf55e700 1 -- 172.21.2.101:0/3205486173 shutdown_connections 2021-02-10T11:32:24.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.422 7faedf55e700 1 -- 172.21.2.101:0/3205486173 wait complete. 2021-02-10T11:32:24.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.436 7faedf55e700 1 Processor -- start 2021-02-10T11:32:24.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.436 7faedf55e700 1 -- start start 2021-02-10T11:32:24.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.437 7faedf55e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faed80935a0 0x7faed80939a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:24.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.437 7faedf55e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faed80a7030 0x7faed80a7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:24.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.437 7faedf55e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faed80a7a50 0x7faed80a7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:24.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.437 7faedf55e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed8094050 con 0x7faed80935a0 2021-02-10T11:32:24.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.437 7faedf55e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed80945d0 con 0x7faed80a7a50 2021-02-10T11:32:24.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.437 7faedf55e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed80947f0 con 0x7faed80a7030 2021-02-10T11:32:24.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.437 7faedd37a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faed80935a0 0x7faed80939a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:24.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.437 7faedd37a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faed80935a0 0x7faed80939a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50838/0 (socket says 172.21.2.101:50838) 2021-02-10T11:32:24.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.437 7faedd37a700 1 -- 172.21.2.101:0/2832919019 learned_addr learned my addr 172.21.2.101:0/2832919019 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:24.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.437 7faedd37a700 1 -- 172.21.2.101:0/2832919019 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faed80a7030 msgr2=0x7faed80a7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:24.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.437 7faedd37a700 1 --2- 172.21.2.101:0/2832919019 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faed80a7030 0x7faed80a7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:24.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.437 7faedd37a700 1 -- 172.21.2.101:0/2832919019 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faed80a7a50 msgr2=0x7faed80a7e50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:24.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.437 7faedd37a700 1 --2- 172.21.2.101:0/2832919019 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faed80a7a50 0x7faed80a7e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:24.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.437 7faedd37a700 1 -- 172.21.2.101:0/2832919019 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed8094a10 con 0x7faed80935a0 2021-02-10T11:32:24.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.438 7faedd37a700 1 --2- 172.21.2.101:0/2832919019 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faed80935a0 0x7faed80939a0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7faed400b530 tx=0x7faed400b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:24.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.438 7faec9ffb700 1 -- 172.21.2.101:0/2832919019 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faed400bcc0 con 0x7faed80935a0 2021-02-10T11:32:24.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.438 7faec9ffb700 1 -- 172.21.2.101:0/2832919019 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed4006020 con 0x7faed80935a0 2021-02-10T11:32:24.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.438 7faec9ffb700 1 -- 172.21.2.101:0/2832919019 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faed40054a0 con 0x7faed80935a0 2021-02-10T11:32:24.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.438 7faedf55e700 1 -- 172.21.2.101:0/2832919019 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed8094a10 con 0x7faed80935a0 2021-02-10T11:32:24.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.438 7faedf55e700 1 -- 172.21.2.101:0/2832919019 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed8094fa0 con 0x7faed80935a0 2021-02-10T11:32:24.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.439 7faec9ffb700 1 -- 172.21.2.101:0/2832919019 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7faed4006200 con 0x7faed80935a0 2021-02-10T11:32:24.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.439 7faec9ffb700 1 --2- 172.21.2.101:0/2832919019 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faeb80186c0 0x7faeb801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:24.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.439 7faec9ffb700 1 -- 172.21.2.101:0/2832919019 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5457+0+0 (secure 0 0 0) 0x7faed4032360 con 0x7faed80935a0 2021-02-10T11:32:24.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.439 7faedcb79700 1 --2- 172.21.2.101:0/2832919019 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faeb80186c0 0x7faeb801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:24.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.439 7faedcb79700 1 --2- 172.21.2.101:0/2832919019 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faeb80186c0 0x7faeb801abc0 crc :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:24.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.470 7faedf55e700 1 -- 172.21.2.101:0/2832919019 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed80806b0 con 0x7faed80935a0 2021-02-10T11:32:24.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.473 7faec9ffb700 1 -- 172.21.2.101:0/2832919019 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7faed4021470 con 0x7faed80935a0 2021-02-10T11:32:24.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.603 7faedf55e700 1 -- 172.21.2.101:0/2832919019 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"} v 0) v1 -- 0x7faed8088020 con 0x7faed80935a0 2021-02-10T11:32:24.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.608 7faec9ffb700 1 -- 172.21.2.101:0/2832919019 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"}]=0 new fs with metadata pool 8 and data pool 9 v28) v1 ==== 187+0+0 (secure 0 0 0) 0x7faed4006200 con 0x7faed80935a0 2021-02-10T11:32:24.620 INFO:teuthology.orchestra.run.gibba001.stderr:new fs with metadata pool 8 and data pool 9 2021-02-10T11:32:24.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.613 7faedf55e700 1 -- 172.21.2.101:0/2832919019 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faeb80186c0 msgr2=0x7faeb801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:24.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.613 7faedf55e700 1 --2- 172.21.2.101:0/2832919019 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faeb80186c0 0x7faeb801abc0 crc :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:24.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.613 7faedf55e700 1 -- 172.21.2.101:0/2832919019 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faed80935a0 msgr2=0x7faed80939a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:24.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.613 7faedf55e700 1 --2- 172.21.2.101:0/2832919019 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faed80935a0 0x7faed80939a0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7faed400b530 tx=0x7faed400b8d0).stop 2021-02-10T11:32:24.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.613 7faedf55e700 1 -- 172.21.2.101:0/2832919019 shutdown_connections 2021-02-10T11:32:24.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.613 7faedf55e700 1 --2- 172.21.2.101:0/2832919019 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faeb80186c0 0x7faeb801abc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:24.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.613 7faedf55e700 1 --2- 172.21.2.101:0/2832919019 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faed80a7030 0x7faed80a7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:24.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.613 7faedf55e700 1 --2- 172.21.2.101:0/2832919019 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faed80a7a50 0x7faed80a7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:24.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.613 7faedf55e700 1 --2- 172.21.2.101:0/2832919019 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faed80935a0 0x7faed80939a0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:24.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.613 7faedf55e700 1 -- 172.21.2.101:0/2832919019 >> 172.21.2.101:0/2832919019 conn(0x7faed80bbea0 msgr2=0x7faed80ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:24.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.613 7faedf55e700 1 -- 172.21.2.101:0/2832919019 shutdown_connections 2021-02-10T11:32:24.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:24.613 7faedf55e700 1 -- 172.21.2.101:0/2832919019 wait complete. 2021-02-10T11:32:24.631 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:32:25.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.030 7f64bf063700 1 Processor -- start 2021-02-10T11:32:25.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.030 7f64bf063700 1 -- start start 2021-02-10T11:32:25.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.030 7f64bf063700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f64b80e5070 0x7f64b81b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:25.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.030 7f64bf063700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b807ed80 con 0x7f64b80e5470 2021-02-10T11:32:25.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.030 7f64bf063700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b807e670 con 0x7f64b81b3e40 2021-02-10T11:32:25.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.030 7f64bf063700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64b807e890 con 0x7f64b80e5070 2021-02-10T11:32:25.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.031 7f64bd680700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f64b80e5470 0x7f64b807de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51344/0 (socket says 172.21.2.101:51344) 2021-02-10T11:32:25.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.031 7f64bd680700 1 -- 172.21.2.101:0/3540896114 learned_addr learned my addr 172.21.2.101:0/3540896114 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:25.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.031 7f64bce7f700 1 --2- 172.21.2.101:0/3540896114 >> v2:172.21.2.101:3300/0 conn(0x7f64b80e5070 0x7f64b81b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:25.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.031 7f64b77fe700 1 -- 172.21.2.101:0/3540896114 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404724477 0 0) 0x7f64b0001500 con 0x7f64b80e5470 2021-02-10T11:32:25.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.031 7f64bce7f700 1 -- 172.21.2.101:0/3540896114 >> v1:172.21.2.108:6789/0 conn(0x7f64b81b3e40 legacy=0x7f64b807d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:25.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.031 7f64bce7f700 1 -- 172.21.2.101:0/3540896114 >> v1:172.21.2.101:6789/0 conn(0x7f64b80e5470 legacy=0x7f64b807de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:25.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.031 7f64bce7f700 1 -- 172.21.2.101:0/3540896114 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b807eab0 con 0x7f64b80e5070 2021-02-10T11:32:25.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.032 7f64bce7f700 1 --2- 172.21.2.101:0/3540896114 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f64b80e5070 0x7f64b81b3820 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f64a0006650 tx=0x7f64a0009a80).ready entity=mon.0 client_cookie=9c335bd46ed8b4b1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:25.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.032 7f64b77fe700 1 -- 172.21.2.101:0/3540896114 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f64a000c0c0 con 0x7f64b80e5070 2021-02-10T11:32:25.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.032 7f64b77fe700 1 -- 172.21.2.101:0/3540896114 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a000c340 con 0x7f64b80e5070 2021-02-10T11:32:25.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.032 7f64b77fe700 1 -- 172.21.2.101:0/3540896114 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f64a000c860 con 0x7f64b80e5070 2021-02-10T11:32:25.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.032 7f64bf063700 1 -- 172.21.2.101:0/3540896114 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f64b80e5070 msgr2=0x7f64b81b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:25.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.032 7f64bf063700 1 --2- 172.21.2.101:0/3540896114 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f64b80e5070 0x7f64b81b3820 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f64a0006650 tx=0x7f64a0009a80).stop 2021-02-10T11:32:25.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.032 7f64bf063700 1 -- 172.21.2.101:0/3540896114 shutdown_connections 2021-02-10T11:32:25.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.032 7f64bf063700 1 --2- 172.21.2.101:0/3540896114 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f64b80e5070 0x7f64b81b3820 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:25.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.032 7f64bf063700 1 -- 172.21.2.101:0/3540896114 >> 172.21.2.101:0/3540896114 conn(0x7f64b80ce690 msgr2=0x7f64b80e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:25.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.032 7f64bf063700 1 -- 172.21.2.101:0/3540896114 shutdown_connections 2021-02-10T11:32:25.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.032 7f64bf063700 1 -- 172.21.2.101:0/3540896114 wait complete. 2021-02-10T11:32:25.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.047 7f64bf063700 1 Processor -- start 2021-02-10T11:32:25.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.047 7f64bf063700 1 -- start start 2021-02-10T11:32:25.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.047 7f64bf063700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f64b808c830 0x7f64b808d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:25.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.047 7f64bf063700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f64b81afe20 0x7f64b80b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:25.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.047 7f64bf063700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f64b81b0220 0x7f64b80b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:25.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.047 7f64bf063700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b808cc30 con 0x7f64b81b0220 2021-02-10T11:32:25.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.047 7f64bf063700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b80b3b70 con 0x7f64b81afe20 2021-02-10T11:32:25.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.047 7f64bf063700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b80b3d90 con 0x7f64b808c830 2021-02-10T11:32:25.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.048 7f64bd680700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f64b81b0220 0x7f64b80b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:25.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.048 7f64bd680700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f64b81b0220 0x7f64b80b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50852/0 (socket says 172.21.2.101:50852) 2021-02-10T11:32:25.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.048 7f64bd680700 1 -- 172.21.2.101:0/3777049646 learned_addr learned my addr 172.21.2.101:0/3777049646 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:25.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.048 7f64bd680700 1 -- 172.21.2.101:0/3777049646 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f64b808c830 msgr2=0x7f64b808d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:25.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.048 7f64bd680700 1 --2- 172.21.2.101:0/3777049646 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f64b808c830 0x7f64b808d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:25.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.048 7f64bd680700 1 -- 172.21.2.101:0/3777049646 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f64b81afe20 msgr2=0x7f64b80b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:25.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.048 7f64bd680700 1 --2- 172.21.2.101:0/3777049646 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f64b81afe20 0x7f64b80b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:25.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.048 7f64bd680700 1 -- 172.21.2.101:0/3777049646 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b81b0c00 con 0x7f64b81b0220 2021-02-10T11:32:25.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.048 7f64bd680700 1 --2- 172.21.2.101:0/3777049646 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f64b81b0220 0x7f64b80b3550 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0x7f64b0001c10 tx=0x7f64b000ac60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:25.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.048 7f64b57fa700 1 -- 172.21.2.101:0/3777049646 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f64b0007020 con 0x7f64b81b0220 2021-02-10T11:32:25.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.049 7f64b57fa700 1 -- 172.21.2.101:0/3777049646 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b00072a0 con 0x7f64b81b0220 2021-02-10T11:32:25.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.049 7f64b57fa700 1 -- 172.21.2.101:0/3777049646 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f64b0004720 con 0x7f64b81b0220 2021-02-10T11:32:25.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.049 7f64bf063700 1 -- 172.21.2.101:0/3777049646 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b81b0c00 con 0x7f64b81b0220 2021-02-10T11:32:25.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.049 7f64bf063700 1 -- 172.21.2.101:0/3777049646 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b81b0c00 con 0x7f64b81b0220 2021-02-10T11:32:25.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.049 7f64b57fa700 1 -- 172.21.2.101:0/3777049646 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f64b0007480 con 0x7f64b81b0220 2021-02-10T11:32:25.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.049 7f64b57fa700 1 --2- 172.21.2.101:0/3777049646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f649c0186c0 0x7f649c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:25.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.049 7f64b57fa700 1 -- 172.21.2.101:0/3777049646 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f64b00328c0 con 0x7f64b81b0220 2021-02-10T11:32:25.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.049 7f64bce7f700 1 --2- 172.21.2.101:0/3777049646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f649c0186c0 0x7f649c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:25.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.050 7f64bce7f700 1 --2- 172.21.2.101:0/3777049646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f649c0186c0 0x7f649c01abc0 crc :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:25.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.080 7f64bf063700 1 -- 172.21.2.101:0/3777049646 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b8083db0 con 0x7f64b81b0220 2021-02-10T11:32:25.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.083 7f64b57fa700 1 -- 172.21.2.101:0/3777049646 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f64b0000cc0 con 0x7f64b81b0220 2021-02-10T11:32:25.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.207 7f64bf063700 1 -- 172.21.2.101:0/3777049646 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64b8083db0 con 0x7f64b81b0220 2021-02-10T11:32:25.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.207 7f64b57fa700 1 -- 172.21.2.101:0/3777049646 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+14006 (secure 0 0 0) 0x7f64b0007480 con 0x7f64b81b0220 2021-02-10T11:32:25.218 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:32:25.219 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":46,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:32:24.604264","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:32:22.580828","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2021-02-10 11:32:23.597744","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:32:25.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.209 7f64bf063700 1 -- 172.21.2.101:0/3777049646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f649c0186c0 msgr2=0x7f649c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:25.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.209 7f64bf063700 1 --2- 172.21.2.101:0/3777049646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f649c0186c0 0x7f649c01abc0 crc :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:25.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.209 7f64bf063700 1 -- 172.21.2.101:0/3777049646 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f64b81b0220 msgr2=0x7f64b80b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:25.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.209 7f64bf063700 1 --2- 172.21.2.101:0/3777049646 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f64b81b0220 0x7f64b80b3550 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0x7f64b0001c10 tx=0x7f64b000ac60).stop 2021-02-10T11:32:25.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.209 7f64bf063700 1 -- 172.21.2.101:0/3777049646 shutdown_connections 2021-02-10T11:32:25.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.209 7f64bf063700 1 --2- 172.21.2.101:0/3777049646 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f64b81b0220 0x7f64b80b3550 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:25.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.209 7f64bf063700 1 --2- 172.21.2.101:0/3777049646 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f64b81afe20 0x7f64b80b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:25.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.209 7f64bf063700 1 --2- 172.21.2.101:0/3777049646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f649c0186c0 0x7f649c01abc0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:25.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.209 7f64bf063700 1 --2- 172.21.2.101:0/3777049646 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f64b808c830 0x7f64b808d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:25.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.209 7f64bf063700 1 -- 172.21.2.101:0/3777049646 >> 172.21.2.101:0/3777049646 conn(0x7f64b80937a0 msgr2=0x7f64b8197e80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:25.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.209 7f64bf063700 1 -- 172.21.2.101:0/3777049646 shutdown_connections 2021-02-10T11:32:25.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.209 7f64bf063700 1 -- 172.21.2.101:0/3777049646 wait complete. 2021-02-10T11:32:25.228 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:32:25.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.628 7f80dcf3c700 1 Processor -- start 2021-02-10T11:32:25.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.628 7f80dcf3c700 1 -- start start 2021-02-10T11:32:25.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.628 7f80dcf3c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f80d807e6a0 0x7f80d80d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:25.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.628 7f80dcf3c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d807ed80 con 0x7f80d80e5070 2021-02-10T11:32:25.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.628 7f80dcf3c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d807eaa0 con 0x7f80d80e5470 2021-02-10T11:32:25.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.628 7f80dcf3c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80d81b4080 con 0x7f80d807e6a0 2021-02-10T11:32:25.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.629 7f80d6b13700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f80d80e5070 0x7f80d81b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51354/0 (socket says 172.21.2.101:51354) 2021-02-10T11:32:25.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.629 7f80d6b13700 1 -- 172.21.2.101:0/1514414785 learned_addr learned my addr 172.21.2.101:0/1514414785 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:25.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.629 7f80d7314700 1 --2- 172.21.2.101:0/1514414785 >> v2:172.21.2.108:3300/0 conn(0x7f80d807e6a0 0x7f80d80d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:25.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.629 7f80d5b11700 1 -- 172.21.2.101:0/1514414785 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954654689 0 0) 0x7f80c8000ce0 con 0x7f80d80e5070 2021-02-10T11:32:25.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.629 7f80d5b11700 1 -- 172.21.2.101:0/1514414785 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80c4002fe0 con 0x7f80d80e5070 2021-02-10T11:32:25.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.629 7f80d5b11700 1 -- 172.21.2.101:0/1514414785 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2548506124 0 0) 0x7f80c0000ff0 con 0x7f80d80e5470 2021-02-10T11:32:25.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.629 7f80d5b11700 1 -- 172.21.2.101:0/1514414785 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80c4004370 con 0x7f80d80e5470 2021-02-10T11:32:25.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.629 7f80d7314700 1 -- 172.21.2.101:0/1514414785 >> v1:172.21.2.108:6789/0 conn(0x7f80d80e5470 legacy=0x7f80d807f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:25.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.629 7f80d7314700 1 -- 172.21.2.101:0/1514414785 >> v1:172.21.2.101:6789/0 conn(0x7f80d80e5070 legacy=0x7f80d81b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:25.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.629 7f80d7314700 1 -- 172.21.2.101:0/1514414785 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d80d1db0 con 0x7f80d807e6a0 2021-02-10T11:32:25.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.630 7f80d7314700 1 --2- 172.21.2.101:0/1514414785 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f80d807e6a0 0x7f80d80d1790 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f80d0006e50 tx=0x7f80d0009df0).ready entity=mon.1 client_cookie=c7f9000f39050ef2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:25.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.630 7f80d5b11700 1 -- 172.21.2.101:0/1514414785 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f80d00066f0 con 0x7f80d807e6a0 2021-02-10T11:32:25.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.630 7f80d5b11700 1 -- 172.21.2.101:0/1514414785 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d0003030 con 0x7f80d807e6a0 2021-02-10T11:32:25.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.630 7f80d5b11700 1 -- 172.21.2.101:0/1514414785 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f80d000b700 con 0x7f80d807e6a0 2021-02-10T11:32:25.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.630 7f80dcf3c700 1 -- 172.21.2.101:0/1514414785 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f80d807e6a0 msgr2=0x7f80d80d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:25.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.630 7f80dcf3c700 1 --2- 172.21.2.101:0/1514414785 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f80d807e6a0 0x7f80d80d1790 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f80d0006e50 tx=0x7f80d0009df0).stop 2021-02-10T11:32:25.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.631 7f80dcf3c700 1 -- 172.21.2.101:0/1514414785 shutdown_connections 2021-02-10T11:32:25.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.631 7f80dcf3c700 1 --2- 172.21.2.101:0/1514414785 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f80d807e6a0 0x7f80d80d1790 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:25.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.631 7f80dcf3c700 1 -- 172.21.2.101:0/1514414785 >> 172.21.2.101:0/1514414785 conn(0x7f80d80ce690 msgr2=0x7f80d80e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:25.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.631 7f80dcf3c700 1 -- 172.21.2.101:0/1514414785 shutdown_connections 2021-02-10T11:32:25.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.631 7f80dcf3c700 1 -- 172.21.2.101:0/1514414785 wait complete. 2021-02-10T11:32:25.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.645 7f80dcf3c700 1 Processor -- start 2021-02-10T11:32:25.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.645 7f80dcf3c700 1 -- start start 2021-02-10T11:32:25.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.646 7f80dcf3c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f80d808d400 0x7f80d8081230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:25.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.646 7f80dcf3c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80d8081850 0x7f80d80acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:25.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.646 7f80dcf3c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f80d8081c50 0x7f80d80ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:25.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.646 7f80dcf3c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d8082110 con 0x7f80d8081850 2021-02-10T11:32:25.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.646 7f80dcf3c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d808d800 con 0x7f80d808d400 2021-02-10T11:32:25.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.646 7f80dcf3c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d80adbc0 con 0x7f80d8081c50 2021-02-10T11:32:25.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.646 7f80d6312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80d8081850 0x7f80d80acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:25.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.646 7f80d6312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80d8081850 0x7f80d80acf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50866/0 (socket says 172.21.2.101:50866) 2021-02-10T11:32:25.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.646 7f80d6312700 1 -- 172.21.2.101:0/2722171154 learned_addr learned my addr 172.21.2.101:0/2722171154 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:25.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.646 7f80d7314700 1 --2- 172.21.2.101:0/2722171154 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f80d8081c50 0x7f80d80ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:25.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.646 7f80d6b13700 1 --2- 172.21.2.101:0/2722171154 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f80d808d400 0x7f80d8081230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:25.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.647 7f80d6312700 1 -- 172.21.2.101:0/2722171154 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f80d8081c50 msgr2=0x7f80d80ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:25.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.647 7f80d6312700 1 --2- 172.21.2.101:0/2722171154 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f80d8081c50 0x7f80d80ad5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:25.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.647 7f80d6312700 1 -- 172.21.2.101:0/2722171154 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f80d808d400 msgr2=0x7f80d8081230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:25.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.647 7f80d6312700 1 --2- 172.21.2.101:0/2722171154 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f80d808d400 0x7f80d8081230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:25.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.647 7f80d6312700 1 -- 172.21.2.101:0/2722171154 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d8187fc0 con 0x7f80d8081850 2021-02-10T11:32:25.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.647 7f80d7314700 1 --2- 172.21.2.101:0/2722171154 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f80d8081c50 0x7f80d80ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:32:25.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.647 7f80d6b13700 1 --2- 172.21.2.101:0/2722171154 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f80d808d400 0x7f80d8081230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:32:25.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.647 7f80d6312700 1 --2- 172.21.2.101:0/2722171154 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80d8081850 0x7f80d80acf80 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7f80c000a570 tx=0x7f80c000a910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:25.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.647 7f80cf7fe700 1 -- 172.21.2.101:0/2722171154 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f80c000ad00 con 0x7f80d8081850 2021-02-10T11:32:25.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.647 7f80cf7fe700 1 -- 172.21.2.101:0/2722171154 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c000c0d0 con 0x7f80d8081850 2021-02-10T11:32:25.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.647 7f80cf7fe700 1 -- 172.21.2.101:0/2722171154 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f80c000c4a0 con 0x7f80d8081850 2021-02-10T11:32:25.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.647 7f80dcf3c700 1 -- 172.21.2.101:0/2722171154 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d8187fc0 con 0x7f80d8081850 2021-02-10T11:32:25.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.647 7f80dcf3c700 1 -- 172.21.2.101:0/2722171154 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d8188550 con 0x7f80d8081850 2021-02-10T11:32:25.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.648 7f80cf7fe700 1 -- 172.21.2.101:0/2722171154 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f80c0008030 con 0x7f80d8081850 2021-02-10T11:32:25.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.648 7f80cf7fe700 1 --2- 172.21.2.101:0/2722171154 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f80bc0186c0 0x7f80bc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:25.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.648 7f80cf7fe700 1 -- 172.21.2.101:0/2722171154 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f80c0032420 con 0x7f80d8081850 2021-02-10T11:32:25.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.648 7f80d6b13700 1 --2- 172.21.2.101:0/2722171154 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f80bc0186c0 0x7f80bc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:25.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.648 7f80d6b13700 1 --2- 172.21.2.101:0/2722171154 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f80bc0186c0 0x7f80bc01abc0 crc :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:25.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.678 7f80dcf3c700 1 -- 172.21.2.101:0/2722171154 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d807ed80 con 0x7f80d8081850 2021-02-10T11:32:25.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.681 7f80cf7fe700 1 -- 172.21.2.101:0/2722171154 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f80c00009b0 con 0x7f80d8081850 2021-02-10T11:32:25.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.807 7f80dcf3c700 1 -- 172.21.2.101:0/2722171154 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80d8082090 con 0x7f80d8081850 2021-02-10T11:32:25.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.808 7f80cf7fe700 1 -- 172.21.2.101:0/2722171154 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+13993 (secure 0 0 0) 0x7f80c0008030 con 0x7f80d8081850 2021-02-10T11:32:25.820 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:32:25.821 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":47,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:32:25.606853","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:32:22.580828","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2021-02-10 11:32:23.597744","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:32:25.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.810 7f80dcf3c700 1 -- 172.21.2.101:0/2722171154 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f80bc0186c0 msgr2=0x7f80bc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:25.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.810 7f80dcf3c700 1 --2- 172.21.2.101:0/2722171154 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f80bc0186c0 0x7f80bc01abc0 crc :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:25.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.810 7f80dcf3c700 1 -- 172.21.2.101:0/2722171154 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80d8081850 msgr2=0x7f80d80acf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:25.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.810 7f80dcf3c700 1 --2- 172.21.2.101:0/2722171154 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80d8081850 0x7f80d80acf80 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7f80c000a570 tx=0x7f80c000a910).stop 2021-02-10T11:32:25.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.810 7f80dcf3c700 1 -- 172.21.2.101:0/2722171154 shutdown_connections 2021-02-10T11:32:25.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.810 7f80dcf3c700 1 --2- 172.21.2.101:0/2722171154 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f80bc0186c0 0x7f80bc01abc0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:25.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.810 7f80dcf3c700 1 --2- 172.21.2.101:0/2722171154 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f80d8081c50 0x7f80d80ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:25.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.810 7f80dcf3c700 1 --2- 172.21.2.101:0/2722171154 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80d8081850 0x7f80d80acf80 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:25.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.810 7f80dcf3c700 1 --2- 172.21.2.101:0/2722171154 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f80d808d400 0x7f80d8081230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:25.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.810 7f80dcf3c700 1 -- 172.21.2.101:0/2722171154 >> 172.21.2.101:0/2722171154 conn(0x7f80d81ac620 msgr2=0x7f80d80896e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:25.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.810 7f80dcf3c700 1 -- 172.21.2.101:0/2722171154 shutdown_connections 2021-02-10T11:32:25.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:25.810 7f80dcf3c700 1 -- 172.21.2.101:0/2722171154 wait complete. 2021-02-10T11:32:25.831 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-02-10T11:32:26.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.237 7f8641d21700 1 Processor -- start 2021-02-10T11:32:26.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.237 7f8641d21700 1 -- start start 2021-02-10T11:32:26.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.238 7f8641d21700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f863c1b0870 0x7f863c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:26.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.238 7f8641d21700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f863c0efe00 0x7f863c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:26.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.238 7f8641d21700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f863c1b0c70 con 0x7f863c1b0470 2021-02-10T11:32:26.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.238 7f8641d21700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f863c0e5ea0 con 0x7f863c0efe00 2021-02-10T11:32:26.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.238 7f8641d21700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f863c0f0b50 con 0x7f863c1b0870 2021-02-10T11:32:26.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.238 7f863bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f863c0efe00 0x7f863c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:26.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.238 7f863bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f863c0efe00 0x7f863c0f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50878/0 (socket says 172.21.2.101:50878) 2021-02-10T11:32:26.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.238 7f863bfff700 1 -- 172.21.2.101:0/3210252303 learned_addr learned my addr 172.21.2.101:0/3210252303 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:26.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.238 7f863bfff700 1 -- 172.21.2.101:0/3210252303 >> v2:172.21.2.108:3300/0 conn(0x7f863c1b0870 msgr2=0x7f863c0f20e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:32:26.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.238 7f863bfff700 1 --2- 172.21.2.101:0/3210252303 >> v2:172.21.2.108:3300/0 conn(0x7f863c1b0870 0x7f863c0f20e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:26.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.238 7f863bfff700 1 -- 172.21.2.101:0/3210252303 >> v1:172.21.2.108:6789/0 conn(0x7f863c1b0470 legacy=0x7f863c0ef610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:32:26.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.238 7f863bfff700 1 -- 172.21.2.101:0/3210252303 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f863c0f0820 con 0x7f863c0efe00 2021-02-10T11:32:26.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.238 7f863bfff700 1 --2- 172.21.2.101:0/3210252303 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f863c0efe00 0x7f863c0f0200 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7f862c006b80 tx=0x7f862c009b90).ready entity=mon.0 client_cookie=bd3b0982e8f19829 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:26.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.239 7f863a7fc700 1 -- 172.21.2.101:0/3210252303 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f862c0022e0 con 0x7f863c0efe00 2021-02-10T11:32:26.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.239 7f863a7fc700 1 -- 172.21.2.101:0/3210252303 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862c002560 con 0x7f863c0efe00 2021-02-10T11:32:26.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.239 7f863a7fc700 1 -- 172.21.2.101:0/3210252303 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f862c002a80 con 0x7f863c0efe00 2021-02-10T11:32:26.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.239 7f8641d21700 1 -- 172.21.2.101:0/3210252303 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f863c0efe00 msgr2=0x7f863c0f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:26.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.239 7f8641d21700 1 --2- 172.21.2.101:0/3210252303 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f863c0efe00 0x7f863c0f0200 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7f862c006b80 tx=0x7f862c009b90).stop 2021-02-10T11:32:26.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.239 7f8641d21700 1 -- 172.21.2.101:0/3210252303 shutdown_connections 2021-02-10T11:32:26.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.239 7f8641d21700 1 --2- 172.21.2.101:0/3210252303 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f863c0efe00 0x7f863c0f0200 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:26.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.239 7f8641d21700 1 --2- 172.21.2.101:0/3210252303 >> v2:172.21.2.108:3300/0 conn(0x7f863c1b0870 0x7f863c0f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:26.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.239 7f8641d21700 1 -- 172.21.2.101:0/3210252303 >> 172.21.2.101:0/3210252303 conn(0x7f863c0ce3e0 msgr2=0x7f863c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:26.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.239 7f8641d21700 1 -- 172.21.2.101:0/3210252303 shutdown_connections 2021-02-10T11:32:26.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.239 7f8641d21700 1 -- 172.21.2.101:0/3210252303 wait complete. 2021-02-10T11:32:26.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.254 7f8641d21700 1 Processor -- start 2021-02-10T11:32:26.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.254 7f8641d21700 1 -- start start 2021-02-10T11:32:26.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.255 7f8641d21700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f863c094fb0 0x7f863c09e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:26.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.255 7f8641d21700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f863c0c2be0 0x7f863c0c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:26.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.255 7f8641d21700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f863c0c3600 0x7f863c0a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:26.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.255 7f8641d21700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c09ebf0 con 0x7f863c094fb0 2021-02-10T11:32:26.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.255 7f8641d21700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c0aa2a0 con 0x7f863c0c2be0 2021-02-10T11:32:26.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.255 7f8641d21700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f863c0aa4c0 con 0x7f863c0c3600 2021-02-10T11:32:26.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.255 7f863b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f863c094fb0 0x7f863c09e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:26.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.255 7f863b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f863c094fb0 0x7f863c09e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50882/0 (socket says 172.21.2.101:50882) 2021-02-10T11:32:26.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.255 7f863b7fe700 1 -- 172.21.2.101:0/3154323769 learned_addr learned my addr 172.21.2.101:0/3154323769 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:26.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.255 7f863affd700 1 --2- 172.21.2.101:0/3154323769 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f863c0c2be0 0x7f863c0c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:26.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.255 7f863b7fe700 1 -- 172.21.2.101:0/3154323769 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f863c0c3600 msgr2=0x7f863c0a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:26.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.255 7f863b7fe700 1 --2- 172.21.2.101:0/3154323769 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f863c0c3600 0x7f863c0a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:26.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.255 7f863b7fe700 1 -- 172.21.2.101:0/3154323769 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f863c0c2be0 msgr2=0x7f863c0c2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:26.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.255 7f863b7fe700 1 --2- 172.21.2.101:0/3154323769 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f863c0c2be0 0x7f863c0c2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:26.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.255 7f863b7fe700 1 -- 172.21.2.101:0/3154323769 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f863c0aa6e0 con 0x7f863c094fb0 2021-02-10T11:32:26.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.255 7f863affd700 1 --2- 172.21.2.101:0/3154323769 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f863c0c2be0 0x7f863c0c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:32:26.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.255 7f863b7fe700 1 --2- 172.21.2.101:0/3154323769 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f863c094fb0 0x7f863c09e5d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7f863000b5b0 tx=0x7f863000b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:26.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.256 7f8623fff700 1 -- 172.21.2.101:0/3154323769 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f863000bd40 con 0x7f863c094fb0 2021-02-10T11:32:26.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.256 7f8623fff700 1 -- 172.21.2.101:0/3154323769 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8630006240 con 0x7f863c094fb0 2021-02-10T11:32:26.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.256 7f8623fff700 1 -- 172.21.2.101:0/3154323769 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8630005720 con 0x7f863c094fb0 2021-02-10T11:32:26.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.256 7f8641d21700 1 -- 172.21.2.101:0/3154323769 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f863c0aa6e0 con 0x7f863c094fb0 2021-02-10T11:32:26.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.256 7f8641d21700 1 -- 172.21.2.101:0/3154323769 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f863c0aac70 con 0x7f863c094fb0 2021-02-10T11:32:26.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.256 7f8623fff700 1 -- 172.21.2.101:0/3154323769 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f86300054a0 con 0x7f863c094fb0 2021-02-10T11:32:26.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.256 7f8623fff700 1 --2- 172.21.2.101:0/3154323769 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f861c0186c0 0x7f861c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:26.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.256 7f8623fff700 1 -- 172.21.2.101:0/3154323769 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8630032420 con 0x7f863c094fb0 2021-02-10T11:32:26.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.256 7f863affd700 1 --2- 172.21.2.101:0/3154323769 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f861c0186c0 0x7f861c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:26.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.257 7f863affd700 1 --2- 172.21.2.101:0/3154323769 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f861c0186c0 0x7f861c01abc0 crc :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:26.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.288 7f8641d21700 1 -- 172.21.2.101:0/3154323769 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f863c0a9750 con 0x7f863c094fb0 2021-02-10T11:32:26.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.290 7f8623fff700 1 -- 172.21.2.101:0/3154323769 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8630000c60 con 0x7f863c094fb0 2021-02-10T11:32:26.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.428 7f8641d21700 1 -- 172.21.2.101:0/3154323769 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"} v 0) v1 -- 0x7f863c0a9750 con 0x7f863c094fb0 2021-02-10T11:32:26.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.615 7f8623fff700 1 -- 172.21.2.101:0/3154323769 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"}]=0 v31) v1 ==== 118+0+0 (secure 0 0 0) 0x7f86300054a0 con 0x7f863c094fb0 2021-02-10T11:32:26.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.645 7f8641d21700 1 -- 172.21.2.101:0/3154323769 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f861c0186c0 msgr2=0x7f861c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:26.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.645 7f8641d21700 1 --2- 172.21.2.101:0/3154323769 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f861c0186c0 0x7f861c01abc0 crc :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:26.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.645 7f8641d21700 1 -- 172.21.2.101:0/3154323769 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f863c094fb0 msgr2=0x7f863c09e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:26.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.645 7f8641d21700 1 --2- 172.21.2.101:0/3154323769 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f863c094fb0 0x7f863c09e5d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7f863000b5b0 tx=0x7f863000b950).stop 2021-02-10T11:32:26.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.645 7f8641d21700 1 -- 172.21.2.101:0/3154323769 shutdown_connections 2021-02-10T11:32:26.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.645 7f8641d21700 1 --2- 172.21.2.101:0/3154323769 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f861c0186c0 0x7f861c01abc0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:26.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.645 7f8641d21700 1 --2- 172.21.2.101:0/3154323769 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f863c0c3600 0x7f863c0a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:26.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.645 7f8641d21700 1 --2- 172.21.2.101:0/3154323769 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f863c0c2be0 0x7f863c0c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:26.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.645 7f8641d21700 1 --2- 172.21.2.101:0/3154323769 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f863c094fb0 0x7f863c09e5d0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:26.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.645 7f8641d21700 1 -- 172.21.2.101:0/3154323769 >> 172.21.2.101:0/3154323769 conn(0x7f863c09f630 msgr2=0x7f863c0a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:26.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.646 7f8641d21700 1 -- 172.21.2.101:0/3154323769 shutdown_connections 2021-02-10T11:32:26.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:26.646 7f8641d21700 1 -- 172.21.2.101:0/3154323769 wait complete. 2021-02-10T11:32:26.664 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:32:27.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.066 7fe540067700 1 Processor -- start 2021-02-10T11:32:27.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.066 7fe540067700 1 -- start start 2021-02-10T11:32:27.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.066 7fe540067700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5380e5070 0x7fe5381b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:27.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.066 7fe540067700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe5381b3e40 0x7fe538080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:27.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.066 7fe540067700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe53807ed80 con 0x7fe5380e5470 2021-02-10T11:32:27.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.066 7fe540067700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5380d1790 con 0x7fe5381b3e40 2021-02-10T11:32:27.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.066 7fe540067700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5380d1980 con 0x7fe5380e5070 2021-02-10T11:32:27.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.067 7fe53e684700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe5381b3e40 0x7fe538080310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.067 7fe53e684700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe5381b3e40 0x7fe538080310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50888/0 (socket says 172.21.2.101:50888) 2021-02-10T11:32:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.067 7fe53e684700 1 -- 172.21.2.101:0/1764580123 learned_addr learned my addr 172.21.2.101:0/1764580123 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.067 7fe53e684700 1 -- 172.21.2.101:0/1764580123 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5380e5070 msgr2=0x7fe5381b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.067 7fe53e684700 1 --2- 172.21.2.101:0/1764580123 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5380e5070 0x7fe5381b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.067 7fe53e684700 1 -- 172.21.2.101:0/1764580123 >> v1:172.21.2.108:6789/0 conn(0x7fe5380e5470 legacy=0x7fe53807d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.067 7fe53e684700 1 -- 172.21.2.101:0/1764580123 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5380d1ba0 con 0x7fe5381b3e40 2021-02-10T11:32:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.067 7fe53e684700 1 --2- 172.21.2.101:0/1764580123 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5381b3e40 0x7fe538080310 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7fe530006b80 tx=0x7fe530009b10).ready entity=mon.0 client_cookie=379ad4fb4dbeff89 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.067 7fe53ce81700 1 -- 172.21.2.101:0/1764580123 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5300022e0 con 0x7fe5381b3e40 2021-02-10T11:32:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.067 7fe53ce81700 1 -- 172.21.2.101:0/1764580123 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe530002560 con 0x7fe5381b3e40 2021-02-10T11:32:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.067 7fe53ce81700 1 -- 172.21.2.101:0/1764580123 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe530002a40 con 0x7fe5381b3e40 2021-02-10T11:32:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.067 7fe540067700 1 -- 172.21.2.101:0/1764580123 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5381b3e40 msgr2=0x7fe538080310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:27.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.067 7fe540067700 1 --2- 172.21.2.101:0/1764580123 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5381b3e40 0x7fe538080310 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7fe530006b80 tx=0x7fe530009b10).stop 2021-02-10T11:32:27.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.068 7fe540067700 1 -- 172.21.2.101:0/1764580123 shutdown_connections 2021-02-10T11:32:27.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.068 7fe540067700 1 --2- 172.21.2.101:0/1764580123 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5381b3e40 0x7fe538080310 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:27.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.068 7fe540067700 1 --2- 172.21.2.101:0/1764580123 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5380e5070 0x7fe5381b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:27.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.068 7fe540067700 1 -- 172.21.2.101:0/1764580123 >> 172.21.2.101:0/1764580123 conn(0x7fe5380ce690 msgr2=0x7fe5380e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:27.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.068 7fe540067700 1 -- 172.21.2.101:0/1764580123 shutdown_connections 2021-02-10T11:32:27.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.068 7fe540067700 1 -- 172.21.2.101:0/1764580123 wait complete. 2021-02-10T11:32:27.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.083 7fe540067700 1 Processor -- start 2021-02-10T11:32:27.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.083 7fe540067700 1 -- start start 2021-02-10T11:32:27.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.083 7fe540067700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5381a2a90 0x7fe5381aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:27.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.083 7fe540067700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5381aa700 0x7fe5381a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:27.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.083 7fe540067700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5380a55e0 0x7fe5381a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:27.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.083 7fe540067700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5381a34a0 con 0x7fe5381a2a90 2021-02-10T11:32:27.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.083 7fe540067700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5381a2e90 con 0x7fe5380a55e0 2021-02-10T11:32:27.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.083 7fe540067700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5381a30b0 con 0x7fe5381aa700 2021-02-10T11:32:27.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.083 7fe53de83700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5381a2a90 0x7fe5381aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:27.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.083 7fe53de83700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5381a2a90 0x7fe5381aa0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50894/0 (socket says 172.21.2.101:50894) 2021-02-10T11:32:27.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.083 7fe53de83700 1 -- 172.21.2.101:0/2290737602 learned_addr learned my addr 172.21.2.101:0/2290737602 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:27.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.084 7fe53de83700 1 -- 172.21.2.101:0/2290737602 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5381aa700 msgr2=0x7fe5381a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:27.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.084 7fe53de83700 1 --2- 172.21.2.101:0/2290737602 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5381aa700 0x7fe5381a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:27.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.084 7fe53de83700 1 -- 172.21.2.101:0/2290737602 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5380a55e0 msgr2=0x7fe5381a4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:27.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.084 7fe53de83700 1 --2- 172.21.2.101:0/2290737602 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5380a55e0 0x7fe5381a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:27.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.084 7fe53de83700 1 -- 172.21.2.101:0/2290737602 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5381a4660 con 0x7fe5381a2a90 2021-02-10T11:32:27.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.084 7fe53de83700 1 --2- 172.21.2.101:0/2290737602 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5381a2a90 0x7fe5381aa0e0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0x7fe534009980 tx=0x7fe53400bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:27.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.084 7fe52a7fc700 1 -- 172.21.2.101:0/2290737602 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5340066d0 con 0x7fe5381a2a90 2021-02-10T11:32:27.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.084 7fe52a7fc700 1 -- 172.21.2.101:0/2290737602 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53400c2f0 con 0x7fe5381a2a90 2021-02-10T11:32:27.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.084 7fe52a7fc700 1 -- 172.21.2.101:0/2290737602 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe53400c7d0 con 0x7fe5381a2a90 2021-02-10T11:32:27.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.084 7fe540067700 1 -- 172.21.2.101:0/2290737602 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5381a4660 con 0x7fe5381a2a90 2021-02-10T11:32:27.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.084 7fe540067700 1 -- 172.21.2.101:0/2290737602 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5381a4660 con 0x7fe5381a2a90 2021-02-10T11:32:27.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.085 7fe52a7fc700 1 -- 172.21.2.101:0/2290737602 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe534001010 con 0x7fe5381a2a90 2021-02-10T11:32:27.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.085 7fe52a7fc700 1 --2- 172.21.2.101:0/2290737602 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5200186c0 0x7fe52001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:27.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.085 7fe52a7fc700 1 -- 172.21.2.101:0/2290737602 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe534032910 con 0x7fe5381a2a90 2021-02-10T11:32:27.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.085 7fe53d682700 1 --2- 172.21.2.101:0/2290737602 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5200186c0 0x7fe52001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:27.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.085 7fe53d682700 1 --2- 172.21.2.101:0/2290737602 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5200186c0 0x7fe52001abc0 crc :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:27.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.116 7fe540067700 1 -- 172.21.2.101:0/2290737602 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe538083da0 con 0x7fe5381a2a90 2021-02-10T11:32:27.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.119 7fe52a7fc700 1 -- 172.21.2.101:0/2290737602 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe53400be10 con 0x7fe5381a2a90 2021-02-10T11:32:27.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.246 7fe540067700 1 -- 172.21.2.101:0/2290737602 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe538083da0 con 0x7fe5381a2a90 2021-02-10T11:32:27.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.246 7fe52a7fc700 1 -- 172.21.2.101:0/2290737602 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+2099 (secure 0 0 0) 0x7fe534001010 con 0x7fe5381a2a90 2021-02-10T11:32:27.257 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:32:27.257 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7674,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1796077990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1796077990},{"type":"v1","addr":"172.21.2.101:6835","nonce":1796077990}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:32:24.604347","modified":"2021-02-10 11:32:26.612664","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5986},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5986":{"gid":5986,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/3346373345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3346373345},{"type":"v1","addr":"172.21.2.108:6833","nonce":3346373345}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-02-10T11:32:27.257 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 31 2021-02-10T11:32:27.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.248 7fe540067700 1 -- 172.21.2.101:0/2290737602 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5200186c0 msgr2=0x7fe52001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:27.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.248 7fe540067700 1 --2- 172.21.2.101:0/2290737602 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5200186c0 0x7fe52001abc0 crc :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:27.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.248 7fe540067700 1 -- 172.21.2.101:0/2290737602 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5381a2a90 msgr2=0x7fe5381aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:27.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.248 7fe540067700 1 --2- 172.21.2.101:0/2290737602 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5381a2a90 0x7fe5381aa0e0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0x7fe534009980 tx=0x7fe53400bcd0).stop 2021-02-10T11:32:27.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.248 7fe540067700 1 -- 172.21.2.101:0/2290737602 shutdown_connections 2021-02-10T11:32:27.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.248 7fe540067700 1 --2- 172.21.2.101:0/2290737602 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5200186c0 0x7fe52001abc0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:27.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.248 7fe540067700 1 --2- 172.21.2.101:0/2290737602 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5381aa700 0x7fe5381a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:27.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.248 7fe540067700 1 --2- 172.21.2.101:0/2290737602 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5380a55e0 0x7fe5381a4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:27.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.248 7fe540067700 1 --2- 172.21.2.101:0/2290737602 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5381a2a90 0x7fe5381aa0e0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:27.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.248 7fe540067700 1 -- 172.21.2.101:0/2290737602 >> 172.21.2.101:0/2290737602 conn(0x7fe53808fe10 msgr2=0x7fe53819c020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:27.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.249 7fe540067700 1 -- 172.21.2.101:0/2290737602 shutdown_connections 2021-02-10T11:32:27.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.249 7fe540067700 1 -- 172.21.2.101:0/2290737602 wait complete. 2021-02-10T11:32:27.267 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:32:27.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.668 7fb1f3012700 1 Processor -- start 2021-02-10T11:32:27.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.668 7fb1f3012700 1 -- start start 2021-02-10T11:32:27.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.669 7fb1f3012700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1ec0e5470 0x7fb1ec07e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:27.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.669 7fb1f3012700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb1ec07f300 0x7fb1ec07f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:27.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.669 7fb1f3012700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1ec07ed80 con 0x7fb1ec0e5070 2021-02-10T11:32:27.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.669 7fb1f3012700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1ec1b4010 con 0x7fb1ec0e5470 2021-02-10T11:32:27.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.669 7fb1f3012700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1ec0d1790 con 0x7fb1ec07f300 2021-02-10T11:32:27.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.669 7fb1ebfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1ec0e5470 0x7fb1ec07e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:27.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.669 7fb1f0e2e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb1ec0e5070 0x7fb1ec1b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51394/0 (socket says 172.21.2.101:51394) 2021-02-10T11:32:27.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.669 7fb1ebfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1ec0e5470 0x7fb1ec07e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50900/0 (socket says 172.21.2.101:50900) 2021-02-10T11:32:27.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.669 7fb1f0e2e700 1 -- 172.21.2.101:0/3407673651 learned_addr learned my addr 172.21.2.101:0/3407673651 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:27.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.669 7fb1f162f700 1 --2- 172.21.2.101:0/3407673651 >> v2:172.21.2.108:3300/0 conn(0x7fb1ec07f300 0x7fb1ec07f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:27.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.670 7fb1ebfff700 1 -- 172.21.2.101:0/3407673651 >> v2:172.21.2.108:3300/0 conn(0x7fb1ec07f300 msgr2=0x7fb1ec07f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:27.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.670 7fb1ebfff700 1 --2- 172.21.2.101:0/3407673651 >> v2:172.21.2.108:3300/0 conn(0x7fb1ec07f300 0x7fb1ec07f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:27.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.670 7fb1ebfff700 1 -- 172.21.2.101:0/3407673651 >> v1:172.21.2.101:6789/0 conn(0x7fb1ec0e5070 legacy=0x7fb1ec1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:27.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.670 7fb1ebfff700 1 -- 172.21.2.101:0/3407673651 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ec0d19b0 con 0x7fb1ec0e5470 2021-02-10T11:32:27.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.670 7fb1ebfff700 1 --2- 172.21.2.101:0/3407673651 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1ec0e5470 0x7fb1ec07e6a0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7fb1dc0069d0 tx=0x7fb1dc009e40).ready entity=mon.0 client_cookie=3b595bdb61d6a020 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:27.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.670 7fb1eb7fe700 1 -- 172.21.2.101:0/3407673651 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1dc00b300 con 0x7fb1ec0e5470 2021-02-10T11:32:27.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.670 7fb1eb7fe700 1 -- 172.21.2.101:0/3407673651 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1dc00b540 con 0x7fb1ec0e5470 2021-02-10T11:32:27.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.670 7fb1eb7fe700 1 -- 172.21.2.101:0/3407673651 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1dc00ba60 con 0x7fb1ec0e5470 2021-02-10T11:32:27.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.670 7fb1f3012700 1 -- 172.21.2.101:0/3407673651 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1ec0e5470 msgr2=0x7fb1ec07e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:27.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.670 7fb1f3012700 1 --2- 172.21.2.101:0/3407673651 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1ec0e5470 0x7fb1ec07e6a0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7fb1dc0069d0 tx=0x7fb1dc009e40).stop 2021-02-10T11:32:27.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.671 7fb1f3012700 1 -- 172.21.2.101:0/3407673651 shutdown_connections 2021-02-10T11:32:27.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.671 7fb1f3012700 1 --2- 172.21.2.101:0/3407673651 >> v2:172.21.2.108:3300/0 conn(0x7fb1ec07f300 0x7fb1ec07f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:27.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.671 7fb1f3012700 1 --2- 172.21.2.101:0/3407673651 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1ec0e5470 0x7fb1ec07e6a0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:27.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.671 7fb1f3012700 1 -- 172.21.2.101:0/3407673651 >> 172.21.2.101:0/3407673651 conn(0x7fb1ec0ce690 msgr2=0x7fb1ec0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:27.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.671 7fb1f3012700 1 -- 172.21.2.101:0/3407673651 shutdown_connections 2021-02-10T11:32:27.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.671 7fb1f3012700 1 -- 172.21.2.101:0/3407673651 wait complete. 2021-02-10T11:32:27.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.685 7fb1f3012700 1 Processor -- start 2021-02-10T11:32:27.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.686 7fb1f3012700 1 -- start start 2021-02-10T11:32:27.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.686 7fb1f3012700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1ec08d400 0x7fb1ec081230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:27.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.686 7fb1f3012700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1ec081850 0x7fb1ec0acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:27.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.686 7fb1f3012700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1ec081c50 0x7fb1ec0ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:27.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.686 7fb1f3012700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ec082110 con 0x7fb1ec08d400 2021-02-10T11:32:27.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.686 7fb1f3012700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ec08d800 con 0x7fb1ec081c50 2021-02-10T11:32:27.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.686 7fb1f3012700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ec0adbc0 con 0x7fb1ec081850 2021-02-10T11:32:27.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.686 7fb1f0e2e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1ec08d400 0x7fb1ec081230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:27.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.686 7fb1f162f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1ec081c50 0x7fb1ec0ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:27.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.686 7fb1f162f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1ec081c50 0x7fb1ec0ad5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:33160/0 (socket says 172.21.2.101:33160) 2021-02-10T11:32:27.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.686 7fb1f162f700 1 -- 172.21.2.101:0/4101038920 learned_addr learned my addr 172.21.2.101:0/4101038920 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:27.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.686 7fb1f0e2e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1ec08d400 0x7fb1ec081230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50908/0 (socket says 172.21.2.101:50908) 2021-02-10T11:32:27.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.686 7fb1ebfff700 1 --2- 172.21.2.101:0/4101038920 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1ec081850 0x7fb1ec0acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:27.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.687 7fb1f0e2e700 1 -- 172.21.2.101:0/4101038920 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1ec081850 msgr2=0x7fb1ec0acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:27.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.687 7fb1f0e2e700 1 --2- 172.21.2.101:0/4101038920 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1ec081850 0x7fb1ec0acf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:27.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.687 7fb1f0e2e700 1 -- 172.21.2.101:0/4101038920 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1ec081c50 msgr2=0x7fb1ec0ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:27.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.687 7fb1f0e2e700 1 --2- 172.21.2.101:0/4101038920 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1ec081c50 0x7fb1ec0ad5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:27.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.687 7fb1f0e2e700 1 -- 172.21.2.101:0/4101038920 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ec187fc0 con 0x7fb1ec08d400 2021-02-10T11:32:27.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.687 7fb1f0e2e700 1 --2- 172.21.2.101:0/4101038920 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1ec08d400 0x7fb1ec081230 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 rx=0x7fb1d400a4c0 tx=0x7fb1d400a860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:27.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.687 7fb1e97fa700 1 -- 172.21.2.101:0/4101038920 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1d4007480 con 0x7fb1ec08d400 2021-02-10T11:32:27.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.687 7fb1e97fa700 1 -- 172.21.2.101:0/4101038920 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d40070d0 con 0x7fb1ec08d400 2021-02-10T11:32:27.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.687 7fb1e97fa700 1 -- 172.21.2.101:0/4101038920 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1d40043a0 con 0x7fb1ec08d400 2021-02-10T11:32:27.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.687 7fb1f3012700 1 -- 172.21.2.101:0/4101038920 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1ec187fc0 con 0x7fb1ec08d400 2021-02-10T11:32:27.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.687 7fb1f3012700 1 -- 172.21.2.101:0/4101038920 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1ec187fc0 con 0x7fb1ec08d400 2021-02-10T11:32:27.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.688 7fb1e97fa700 1 -- 172.21.2.101:0/4101038920 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb1d40043a0 con 0x7fb1ec08d400 2021-02-10T11:32:27.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.688 7fb1e97fa700 1 --2- 172.21.2.101:0/4101038920 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1d00186c0 0x7fb1d001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:27.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.688 7fb1e97fa700 1 -- 172.21.2.101:0/4101038920 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb1d4032570 con 0x7fb1ec08d400 2021-02-10T11:32:27.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.688 7fb1ebfff700 1 --2- 172.21.2.101:0/4101038920 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1d00186c0 0x7fb1d001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:27.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.688 7fb1ebfff700 1 --2- 172.21.2.101:0/4101038920 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1d00186c0 0x7fb1d001abc0 crc :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:27.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.719 7fb1f3012700 1 -- 172.21.2.101:0/4101038920 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1ec08d800 con 0x7fb1ec08d400 2021-02-10T11:32:27.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.721 7fb1e97fa700 1 -- 172.21.2.101:0/4101038920 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb1d40070b0 con 0x7fb1ec08d400 2021-02-10T11:32:27.757 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:32:27.759 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:32:27.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.848 7fb1f3012700 1 -- 172.21.2.101:0/4101038920 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1ec08d800 con 0x7fb1ec08d400 2021-02-10T11:32:27.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.849 7fb1e97fa700 1 -- 172.21.2.101:0/4101038920 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+13993 (secure 0 0 0) 0x7fb1d40043a0 con 0x7fb1ec08d400 2021-02-10T11:32:27.860 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:32:27.860 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":47,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:32:25.606853","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:32:22.580828","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2021-02-10 11:32:23.597744","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:32:27.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.850 7fb1f3012700 1 -- 172.21.2.101:0/4101038920 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1d00186c0 msgr2=0x7fb1d001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:27.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.850 7fb1f3012700 1 --2- 172.21.2.101:0/4101038920 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1d00186c0 0x7fb1d001abc0 crc :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:27.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.850 7fb1f3012700 1 -- 172.21.2.101:0/4101038920 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1ec08d400 msgr2=0x7fb1ec081230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:27.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.850 7fb1f3012700 1 --2- 172.21.2.101:0/4101038920 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1ec08d400 0x7fb1ec081230 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 rx=0x7fb1d400a4c0 tx=0x7fb1d400a860).stop 2021-02-10T11:32:27.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.851 7fb1f3012700 1 -- 172.21.2.101:0/4101038920 shutdown_connections 2021-02-10T11:32:27.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.851 7fb1f3012700 1 --2- 172.21.2.101:0/4101038920 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1d00186c0 0x7fb1d001abc0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:27.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.851 7fb1f3012700 1 --2- 172.21.2.101:0/4101038920 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1ec081850 0x7fb1ec0acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:27.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.851 7fb1f3012700 1 --2- 172.21.2.101:0/4101038920 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1ec081c50 0x7fb1ec0ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:27.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.851 7fb1f3012700 1 --2- 172.21.2.101:0/4101038920 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1ec08d400 0x7fb1ec081230 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:27.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.851 7fb1f3012700 1 -- 172.21.2.101:0/4101038920 >> 172.21.2.101:0/4101038920 conn(0x7fb1ec1ac620 msgr2=0x7fb1ec0896e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:27.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.851 7fb1f3012700 1 -- 172.21.2.101:0/4101038920 shutdown_connections 2021-02-10T11:32:27.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:27.851 7fb1f3012700 1 -- 172.21.2.101:0/4101038920 wait complete. 2021-02-10T11:32:27.871 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:32:28.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.284 7f8168c24700 1 Processor -- start 2021-02-10T11:32:28.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.284 7f8168c24700 1 -- start start 2021-02-10T11:32:28.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.284 7f8168c24700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f81641b0470 0x7f81641b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:28.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.284 7f8168c24700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81640f0b50 con 0x7f81640ef610 2021-02-10T11:32:28.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.284 7f8168c24700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81640f24e0 con 0x7f81640f20e0 2021-02-10T11:32:28.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.284 7f8168c24700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81640e5ea0 con 0x7f81641b0470 2021-02-10T11:32:28.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.285 7f816285e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f81641b0470 0x7f81641b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:28.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.285 7f816205d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f81640ef610 0x7f81640efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51408/0 (socket says 172.21.2.101:51408) 2021-02-10T11:32:28.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.285 7f816205d700 1 -- 172.21.2.101:0/1015878265 learned_addr learned my addr 172.21.2.101:0/1015878265 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:28.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.285 7f816285e700 1 -- 172.21.2.101:0/1015878265 >> v1:172.21.2.108:6789/0 conn(0x7f81640f20e0 legacy=0x7f81640f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:28.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.285 7f816285e700 1 -- 172.21.2.101:0/1015878265 >> v1:172.21.2.101:6789/0 conn(0x7f81640ef610 legacy=0x7f81640efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:28.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.285 7f816285e700 1 -- 172.21.2.101:0/1015878265 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81640e52e0 con 0x7f81641b0470 2021-02-10T11:32:28.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.285 7f816285e700 1 --2- 172.21.2.101:0/1015878265 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81641b0470 0x7f81641b0870 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7f8154006650 tx=0x7f8154009a80).ready entity=mon.0 client_cookie=ab49477702c34a49 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:28.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.285 7f816185c700 1 -- 172.21.2.101:0/1015878265 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81540009b0 con 0x7f81641b0470 2021-02-10T11:32:28.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.285 7f816185c700 1 -- 172.21.2.101:0/1015878265 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815400c0c0 con 0x7f81641b0470 2021-02-10T11:32:28.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.285 7f816185c700 1 -- 172.21.2.101:0/1015878265 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f815400c5e0 con 0x7f81641b0470 2021-02-10T11:32:28.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.285 7f8168c24700 1 -- 172.21.2.101:0/1015878265 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81641b0470 msgr2=0x7f81641b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:28.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.285 7f8168c24700 1 --2- 172.21.2.101:0/1015878265 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81641b0470 0x7f81641b0870 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7f8154006650 tx=0x7f8154009a80).stop 2021-02-10T11:32:28.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.286 7f8168c24700 1 -- 172.21.2.101:0/1015878265 shutdown_connections 2021-02-10T11:32:28.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.286 7f8168c24700 1 --2- 172.21.2.101:0/1015878265 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81641b0470 0x7f81641b0870 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:28.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.286 7f8168c24700 1 -- 172.21.2.101:0/1015878265 >> 172.21.2.101:0/1015878265 conn(0x7f81640ce3e0 msgr2=0x7f81640e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:28.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.286 7f8168c24700 1 -- 172.21.2.101:0/1015878265 shutdown_connections 2021-02-10T11:32:28.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.286 7f8168c24700 1 -- 172.21.2.101:0/1015878265 wait complete. 2021-02-10T11:32:28.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.301 7f8168c24700 1 Processor -- start 2021-02-10T11:32:28.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.301 7f8168c24700 1 -- start start 2021-02-10T11:32:28.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.301 7f8168c24700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f816409f300 0x7f81640a73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:28.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.301 7f8168c24700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81640aa870 0x7f81640aac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:28.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.301 7f8168c24700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81640ab290 0x7f81640a8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:28.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.301 7f8168c24700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8164081940 con 0x7f81640ab290 2021-02-10T11:32:28.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.301 7f8168c24700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81640a7bd0 con 0x7f81640aa870 2021-02-10T11:32:28.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.301 7f8168c24700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81640ab690 con 0x7f816409f300 2021-02-10T11:32:28.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.301 7f816305f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81640ab290 0x7f81640a8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:28.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.302 7f816305f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81640ab290 0x7f81640a8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50920/0 (socket says 172.21.2.101:50920) 2021-02-10T11:32:28.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.302 7f816305f700 1 -- 172.21.2.101:0/435132623 learned_addr learned my addr 172.21.2.101:0/435132623 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:28.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.302 7f816305f700 1 -- 172.21.2.101:0/435132623 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f816409f300 msgr2=0x7f81640a73d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:28.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.302 7f816305f700 1 --2- 172.21.2.101:0/435132623 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f816409f300 0x7f81640a73d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:28.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.302 7f816305f700 1 -- 172.21.2.101:0/435132623 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81640aa870 msgr2=0x7f81640aac70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:28.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.302 7f816305f700 1 --2- 172.21.2.101:0/435132623 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81640aa870 0x7f81640aac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:28.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.302 7f816305f700 1 -- 172.21.2.101:0/435132623 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816409f700 con 0x7f81640ab290 2021-02-10T11:32:28.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.302 7f816305f700 1 --2- 172.21.2.101:0/435132623 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81640ab290 0x7f81640a8150 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7f815c009940 tx=0x7f815c00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:28.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.302 7f815b7fe700 1 -- 172.21.2.101:0/435132623 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f815c000ce0 con 0x7f81640ab290 2021-02-10T11:32:28.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.302 7f815b7fe700 1 -- 172.21.2.101:0/435132623 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815c00c2f0 con 0x7f81640ab290 2021-02-10T11:32:28.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.302 7f815b7fe700 1 -- 172.21.2.101:0/435132623 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f815c00c810 con 0x7f81640ab290 2021-02-10T11:32:28.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.302 7f8168c24700 1 -- 172.21.2.101:0/435132623 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816409f700 con 0x7f81640ab290 2021-02-10T11:32:28.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.302 7f8168c24700 1 -- 172.21.2.101:0/435132623 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81640a8a60 con 0x7f81640ab290 2021-02-10T11:32:28.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.303 7f815b7fe700 1 -- 172.21.2.101:0/435132623 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f815c001220 con 0x7f81640ab290 2021-02-10T11:32:28.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.303 7f815b7fe700 1 --2- 172.21.2.101:0/435132623 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81480186c0 0x7f814801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:28.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.303 7f815b7fe700 1 -- 172.21.2.101:0/435132623 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f815c0328f0 con 0x7f81640ab290 2021-02-10T11:32:28.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.303 7f816285e700 1 --2- 172.21.2.101:0/435132623 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81480186c0 0x7f814801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:28.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.303 7f816285e700 1 --2- 172.21.2.101:0/435132623 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81480186c0 0x7f814801abc0 crc :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:28.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.334 7f8168c24700 1 -- 172.21.2.101:0/435132623 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f816407ed20 con 0x7f81640ab290 2021-02-10T11:32:28.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.337 7f815b7fe700 1 -- 172.21.2.101:0/435132623 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f815c000cc0 con 0x7f81640ab290 2021-02-10T11:32:28.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.466 7f8168c24700 1 -- 172.21.2.101:0/435132623 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f81640ce3e0 con 0x7f81640ab290 2021-02-10T11:32:28.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.470 7f815b7fe700 1 -- 172.21.2.101:0/435132623 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v25) v1 ==== 184+0+0 (secure 0 0 0) 0x7f815c01fa20 con 0x7f81640ab290 2021-02-10T11:32:28.485 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.0 2021-02-10T11:32:28.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.475 7f8168c24700 1 -- 172.21.2.101:0/435132623 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81480186c0 msgr2=0x7f814801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:28.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.475 7f8168c24700 1 --2- 172.21.2.101:0/435132623 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81480186c0 0x7f814801abc0 crc :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:28.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.475 7f8168c24700 1 -- 172.21.2.101:0/435132623 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81640ab290 msgr2=0x7f81640a8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:28.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.475 7f8168c24700 1 --2- 172.21.2.101:0/435132623 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81640ab290 0x7f81640a8150 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7f815c009940 tx=0x7f815c00bcf0).stop 2021-02-10T11:32:28.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.475 7f8168c24700 1 -- 172.21.2.101:0/435132623 shutdown_connections 2021-02-10T11:32:28.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.475 7f8168c24700 1 --2- 172.21.2.101:0/435132623 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81640ab290 0x7f81640a8150 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:28.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.475 7f8168c24700 1 --2- 172.21.2.101:0/435132623 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81640aa870 0x7f81640aac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:28.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.475 7f8168c24700 1 --2- 172.21.2.101:0/435132623 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81480186c0 0x7f814801abc0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:28.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.475 7f8168c24700 1 --2- 172.21.2.101:0/435132623 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f816409f300 0x7f81640a73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:28.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.475 7f8168c24700 1 -- 172.21.2.101:0/435132623 >> 172.21.2.101:0/435132623 conn(0x7f816409fab0 msgr2=0x7f8164089220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:28.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.475 7f8168c24700 1 -- 172.21.2.101:0/435132623 shutdown_connections 2021-02-10T11:32:28.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.475 7f8168c24700 1 -- 172.21.2.101:0/435132623 wait complete. 2021-02-10T11:32:28.495 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:32:28.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.906 7f1542fd2700 1 Processor -- start 2021-02-10T11:32:28.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.906 7f1542fd2700 1 -- start start 2021-02-10T11:32:28.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.906 7f1542fd2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f153c1b0470 0x7f153c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:28.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.906 7f1542fd2700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f153c0ef610 0x7f153c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:28.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.906 7f1542fd2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f153c0f0030 0x7f153c0f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:28.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.906 7f1542fd2700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f153c0e5ea0 con 0x7f153c0f0030 2021-02-10T11:32:28.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.906 7f1542fd2700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f153c0f0b50 con 0x7f153c0ef610 2021-02-10T11:32:28.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.906 7f1542fd2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153c0e52e0 con 0x7f153c1b0470 2021-02-10T11:32:28.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.906 7f15415ef700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f153c0f0030 0x7f153c0f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:28.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.906 7f15415ef700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f153c0f0030 0x7f153c0f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50924/0 (socket says 172.21.2.101:50924) 2021-02-10T11:32:28.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.906 7f15415ef700 1 -- 172.21.2.101:0/594990522 learned_addr learned my addr 172.21.2.101:0/594990522 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:28.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.907 7f15415ef700 1 -- 172.21.2.101:0/594990522 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f153c1b0470 msgr2=0x7f153c1b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:32:28.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.907 7f15415ef700 1 --2- 172.21.2.101:0/594990522 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f153c1b0470 0x7f153c1b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:28.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.907 7f15415ef700 1 -- 172.21.2.101:0/594990522 >> v2:172.21.2.108:3300/0 conn(0x7f153c0ef610 msgr2=0x7f153c0efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:32:28.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.907 7f15415ef700 1 --2- 172.21.2.101:0/594990522 >> v2:172.21.2.108:3300/0 conn(0x7f153c0ef610 0x7f153c0efa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:28.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.907 7f15415ef700 1 -- 172.21.2.101:0/594990522 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c0f10d0 con 0x7f153c0f0030 2021-02-10T11:32:28.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.907 7f15415ef700 1 --2- 172.21.2.101:0/594990522 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f153c0f0030 0x7f153c0f0430 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7f1534006f00 tx=0x7f1534009f10).ready entity=mon.0 client_cookie=afce135b457c2c79 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:28.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.907 7f153b7fe700 1 -- 172.21.2.101:0/594990522 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f153400b3b0 con 0x7f153c0f0030 2021-02-10T11:32:28.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.907 7f153b7fe700 1 -- 172.21.2.101:0/594990522 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153400b630 con 0x7f153c0f0030 2021-02-10T11:32:28.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.907 7f153b7fe700 1 -- 172.21.2.101:0/594990522 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f153400bb50 con 0x7f153c0f0030 2021-02-10T11:32:28.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.907 7f1542fd2700 1 -- 172.21.2.101:0/594990522 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f153c0f0030 msgr2=0x7f153c0f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:28.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.907 7f1542fd2700 1 --2- 172.21.2.101:0/594990522 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f153c0f0030 0x7f153c0f0430 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7f1534006f00 tx=0x7f1534009f10).stop 2021-02-10T11:32:28.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.908 7f1542fd2700 1 -- 172.21.2.101:0/594990522 shutdown_connections 2021-02-10T11:32:28.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.908 7f1542fd2700 1 --2- 172.21.2.101:0/594990522 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f153c0f0030 0x7f153c0f0430 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:28.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.908 7f1542fd2700 1 --2- 172.21.2.101:0/594990522 >> v2:172.21.2.108:3300/0 conn(0x7f153c0ef610 0x7f153c0efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:28.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.908 7f1542fd2700 1 --2- 172.21.2.101:0/594990522 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f153c1b0470 0x7f153c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:28.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.908 7f1542fd2700 1 -- 172.21.2.101:0/594990522 >> 172.21.2.101:0/594990522 conn(0x7f153c0ce3e0 msgr2=0x7f153c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:28.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.908 7f1542fd2700 1 -- 172.21.2.101:0/594990522 shutdown_connections 2021-02-10T11:32:28.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.908 7f1542fd2700 1 -- 172.21.2.101:0/594990522 wait complete. 2021-02-10T11:32:28.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.923 7f1542fd2700 1 Processor -- start 2021-02-10T11:32:28.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.923 7f1542fd2700 1 -- start start 2021-02-10T11:32:28.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.923 7f1542fd2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f153c1b6de0 0x7f153c1b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:28.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.923 7f1542fd2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f153c18f3f0 0x7f153c18f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:28.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.923 7f1542fd2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f153c18fe10 0x7f153c18ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:28.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.923 7f1542fd2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c18d950 con 0x7f153c18fe10 2021-02-10T11:32:28.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.923 7f1542fd2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c1b71e0 con 0x7f153c18f3f0 2021-02-10T11:32:28.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.923 7f1542fd2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153c1b7400 con 0x7f153c1b6de0 2021-02-10T11:32:28.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.923 7f15415ef700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f153c18fe10 0x7f153c18ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:28.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.924 7f15415ef700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f153c18fe10 0x7f153c18ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50932/0 (socket says 172.21.2.101:50932) 2021-02-10T11:32:28.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.924 7f15415ef700 1 -- 172.21.2.101:0/1074442980 learned_addr learned my addr 172.21.2.101:0/1074442980 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:28.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.924 7f15415ef700 1 -- 172.21.2.101:0/1074442980 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f153c1b6de0 msgr2=0x7f153c1b63f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:28.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.924 7f15415ef700 1 --2- 172.21.2.101:0/1074442980 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f153c1b6de0 0x7f153c1b63f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:28.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.924 7f15415ef700 1 -- 172.21.2.101:0/1074442980 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f153c18f3f0 msgr2=0x7f153c18f7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:28.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.924 7f15415ef700 1 --2- 172.21.2.101:0/1074442980 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f153c18f3f0 0x7f153c18f7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:28.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.924 7f15415ef700 1 -- 172.21.2.101:0/1074442980 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c190210 con 0x7f153c18fe10 2021-02-10T11:32:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.924 7f15415ef700 1 --2- 172.21.2.101:0/1074442980 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f153c18fe10 0x7f153c18ddf0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7f1534009390 tx=0x7f153400bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.924 7f15397fa700 1 -- 172.21.2.101:0/1074442980 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1534003030 con 0x7f153c18fe10 2021-02-10T11:32:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.924 7f15397fa700 1 -- 172.21.2.101:0/1074442980 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15340068e0 con 0x7f153c18fe10 2021-02-10T11:32:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.924 7f1542fd2700 1 -- 172.21.2.101:0/1074442980 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f153c190210 con 0x7f153c18fe10 2021-02-10T11:32:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.924 7f1542fd2700 1 -- 172.21.2.101:0/1074442980 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f153c18e700 con 0x7f153c18fe10 2021-02-10T11:32:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.924 7f15397fa700 1 -- 172.21.2.101:0/1074442980 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f153400c4e0 con 0x7f153c18fe10 2021-02-10T11:32:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.925 7f15397fa700 1 -- 172.21.2.101:0/1074442980 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1534001270 con 0x7f153c18fe10 2021-02-10T11:32:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.925 7f15397fa700 1 --2- 172.21.2.101:0/1074442980 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15200186c0 0x7f152001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.925 7f15397fa700 1 -- 172.21.2.101:0/1074442980 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1534032b00 con 0x7f153c18fe10 2021-02-10T11:32:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.925 7f1540dee700 1 --2- 172.21.2.101:0/1074442980 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15200186c0 0x7f152001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.925 7f1540dee700 1 --2- 172.21.2.101:0/1074442980 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15200186c0 0x7f152001abc0 crc :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:28.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.956 7f1542fd2700 1 -- 172.21.2.101:0/1074442980 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f153c1059e0 con 0x7f153c18fe10 2021-02-10T11:32:28.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:28.959 7f15397fa700 1 -- 172.21.2.101:0/1074442980 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1534009450 con 0x7f153c18fe10 2021-02-10T11:32:29.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.091 7f1542fd2700 1 -- 172.21.2.101:0/1074442980 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f153c0f0b50 con 0x7f153c18fe10 2021-02-10T11:32:29.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.095 7f15397fa700 1 -- 172.21.2.101:0/1074442980 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v26) v1 ==== 184+0+0 (secure 0 0 0) 0x7f1534001270 con 0x7f153c18fe10 2021-02-10T11:32:29.109 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.1 2021-02-10T11:32:29.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.101 7f1542fd2700 1 -- 172.21.2.101:0/1074442980 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15200186c0 msgr2=0x7f152001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:29.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.101 7f1542fd2700 1 --2- 172.21.2.101:0/1074442980 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15200186c0 0x7f152001abc0 crc :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:29.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.101 7f1542fd2700 1 -- 172.21.2.101:0/1074442980 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f153c18fe10 msgr2=0x7f153c18ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:29.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.101 7f1542fd2700 1 --2- 172.21.2.101:0/1074442980 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f153c18fe10 0x7f153c18ddf0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7f1534009390 tx=0x7f153400bd40).stop 2021-02-10T11:32:29.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.101 7f1542fd2700 1 -- 172.21.2.101:0/1074442980 shutdown_connections 2021-02-10T11:32:29.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.101 7f1542fd2700 1 --2- 172.21.2.101:0/1074442980 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f153c18fe10 0x7f153c18ddf0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:29.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.101 7f1542fd2700 1 --2- 172.21.2.101:0/1074442980 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f153c18f3f0 0x7f153c18f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:29.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.101 7f1542fd2700 1 --2- 172.21.2.101:0/1074442980 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15200186c0 0x7f152001abc0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:29.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.101 7f1542fd2700 1 --2- 172.21.2.101:0/1074442980 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f153c1b6de0 0x7f153c1b63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:29.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.101 7f1542fd2700 1 -- 172.21.2.101:0/1074442980 >> 172.21.2.101:0/1074442980 conn(0x7f153c1a37a0 msgr2=0x7f153c18c230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:29.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.101 7f1542fd2700 1 -- 172.21.2.101:0/1074442980 shutdown_connections 2021-02-10T11:32:29.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.101 7f1542fd2700 1 -- 172.21.2.101:0/1074442980 wait complete. 2021-02-10T11:32:29.120 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:32:29.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.531 7f0920717700 1 Processor -- start 2021-02-10T11:32:29.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.531 7f0920717700 1 -- start start 2021-02-10T11:32:29.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.532 7f0920717700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09181b0870 0x7f09180f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:29.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.532 7f0920717700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f09180efe00 0x7f09180f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:29.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.532 7f0920717700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09181b0c70 con 0x7f09181b0470 2021-02-10T11:32:29.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.532 7f0920717700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09180f0b50 con 0x7f09180efe00 2021-02-10T11:32:29.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.532 7f0920717700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09180f0820 con 0x7f09181b0870 2021-02-10T11:32:29.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.533 7f091dd32700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09181b0870 0x7f09180f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:29.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.533 7f091e533700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f09181b0470 0x7f09180ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:44840/0 (socket says 172.21.2.101:44840) 2021-02-10T11:32:29.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.533 7f091ed34700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f09180efe00 0x7f09180f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:29.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.533 7f091e533700 1 -- 172.21.2.101:0/3094839055 learned_addr learned my addr 172.21.2.101:0/3094839055 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:29.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.533 7f091dd32700 1 -- 172.21.2.101:0/3094839055 >> v2:172.21.2.108:3300/0 conn(0x7f09180efe00 msgr2=0x7f09180f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:29.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.533 7f091dd32700 1 --2- 172.21.2.101:0/3094839055 >> v2:172.21.2.108:3300/0 conn(0x7f09180efe00 0x7f09180f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:29.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.533 7f091dd32700 1 -- 172.21.2.101:0/3094839055 >> v1:172.21.2.108:6789/0 conn(0x7f09181b0470 legacy=0x7f09180ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:29.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.533 7f091dd32700 1 -- 172.21.2.101:0/3094839055 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09180e52e0 con 0x7f09181b0870 2021-02-10T11:32:29.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.533 7f091ed34700 1 --2- 172.21.2.101:0/3094839055 >> v2:172.21.2.108:3300/0 conn(0x7f09180efe00 0x7f09180f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:32:29.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.533 7f091dd32700 1 --2- 172.21.2.101:0/3094839055 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09181b0870 0x7f09180f20e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f090c006650 tx=0x7f090c009a80).ready entity=mon.2 client_cookie=4b76a08984721840 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:29.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.533 7f091d531700 1 -- 172.21.2.101:0/3094839055 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f090c00c0c0 con 0x7f09181b0870 2021-02-10T11:32:29.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.535 7f091d531700 1 -- 172.21.2.101:0/3094839055 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090c00c340 con 0x7f09181b0870 2021-02-10T11:32:29.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.535 7f091d531700 1 -- 172.21.2.101:0/3094839055 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f090c00c820 con 0x7f09181b0870 2021-02-10T11:32:29.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.535 7f0920717700 1 -- 172.21.2.101:0/3094839055 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09181b0870 msgr2=0x7f09180f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:29.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.535 7f0920717700 1 --2- 172.21.2.101:0/3094839055 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09181b0870 0x7f09180f20e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f090c006650 tx=0x7f090c009a80).stop 2021-02-10T11:32:29.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.535 7f0920717700 1 -- 172.21.2.101:0/3094839055 shutdown_connections 2021-02-10T11:32:29.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.535 7f0920717700 1 --2- 172.21.2.101:0/3094839055 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09181b0870 0x7f09180f20e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:29.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.535 7f0920717700 1 --2- 172.21.2.101:0/3094839055 >> v2:172.21.2.108:3300/0 conn(0x7f09180efe00 0x7f09180f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:29.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.535 7f0920717700 1 -- 172.21.2.101:0/3094839055 >> 172.21.2.101:0/3094839055 conn(0x7f09180ce3e0 msgr2=0x7f09180e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:29.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.535 7f0920717700 1 -- 172.21.2.101:0/3094839055 shutdown_connections 2021-02-10T11:32:29.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.535 7f0920717700 1 -- 172.21.2.101:0/3094839055 wait complete. 2021-02-10T11:32:29.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.549 7f0920717700 1 Processor -- start 2021-02-10T11:32:29.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.549 7f0920717700 1 -- start start 2021-02-10T11:32:29.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.549 7f0920717700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09180b65c0 0x7f09181a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:29.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.549 7f0920717700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09181a8030 0x7f09180b7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:29.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.549 7f0920717700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09180b69c0 0x7f09180b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:29.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.549 7f0920717700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09180b6ec0 con 0x7f09180b65c0 2021-02-10T11:32:29.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.549 7f0920717700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09180b8080 con 0x7f09181a8030 2021-02-10T11:32:29.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.549 7f0920717700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09180b8260 con 0x7f09180b69c0 2021-02-10T11:32:29.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.549 7f091e533700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09180b65c0 0x7f09181a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:29.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.550 7f091e533700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09180b65c0 0x7f09181a7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50944/0 (socket says 172.21.2.101:50944) 2021-02-10T11:32:29.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.550 7f091e533700 1 -- 172.21.2.101:0/441011545 learned_addr learned my addr 172.21.2.101:0/441011545 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:29.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.550 7f091ed34700 1 --2- 172.21.2.101:0/441011545 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09180b69c0 0x7f09180b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:29.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.550 7f091dd32700 1 --2- 172.21.2.101:0/441011545 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09181a8030 0x7f09180b7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:29.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.550 7f091e533700 1 -- 172.21.2.101:0/441011545 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09180b69c0 msgr2=0x7f09180b7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:29.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.550 7f091e533700 1 --2- 172.21.2.101:0/441011545 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09180b69c0 0x7f09180b7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:29.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.550 7f091e533700 1 -- 172.21.2.101:0/441011545 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09181a8030 msgr2=0x7f09180b7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:29.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.550 7f091e533700 1 --2- 172.21.2.101:0/441011545 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09181a8030 0x7f09180b7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:29.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.550 7f091e533700 1 -- 172.21.2.101:0/441011545 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09180b8480 con 0x7f09180b65c0 2021-02-10T11:32:29.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.550 7f091dd32700 1 --2- 172.21.2.101:0/441011545 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09181a8030 0x7f09180b7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:32:29.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.550 7f091e533700 1 --2- 172.21.2.101:0/441011545 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09180b65c0 0x7f09181a7a10 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7f0914001c60 tx=0x7f091400a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:29.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.550 7f090affd700 1 -- 172.21.2.101:0/441011545 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f091400acc0 con 0x7f09180b65c0 2021-02-10T11:32:29.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.550 7f090affd700 1 -- 172.21.2.101:0/441011545 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0914004140 con 0x7f09180b65c0 2021-02-10T11:32:29.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.550 7f090affd700 1 -- 172.21.2.101:0/441011545 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f09140045a0 con 0x7f09180b65c0 2021-02-10T11:32:29.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.550 7f0920717700 1 -- 172.21.2.101:0/441011545 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09180b8480 con 0x7f09180b65c0 2021-02-10T11:32:29.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.550 7f0920717700 1 -- 172.21.2.101:0/441011545 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09180b8480 con 0x7f09180b65c0 2021-02-10T11:32:29.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.551 7f090affd700 1 -- 172.21.2.101:0/441011545 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0914004320 con 0x7f09180b65c0 2021-02-10T11:32:29.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.551 7f090affd700 1 --2- 172.21.2.101:0/441011545 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0900018730 0x7f090001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:29.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.551 7f090affd700 1 -- 172.21.2.101:0/441011545 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0914032600 con 0x7f09180b65c0 2021-02-10T11:32:29.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.551 7f091dd32700 1 --2- 172.21.2.101:0/441011545 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0900018730 0x7f090001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:29.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.552 7f091dd32700 1 --2- 172.21.2.101:0/441011545 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0900018730 0x7f090001abc0 crc :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:29.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.582 7f0920717700 1 -- 172.21.2.101:0/441011545 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09181059f0 con 0x7f09180b65c0 2021-02-10T11:32:29.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.584 7f090affd700 1 -- 172.21.2.101:0/441011545 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0914004350 con 0x7f09180b65c0 2021-02-10T11:32:29.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.716 7f0920717700 1 -- 172.21.2.101:0/441011545 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f09181059f0 con 0x7f09180b65c0 2021-02-10T11:32:29.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.719 7f090affd700 1 -- 172.21.2.101:0/441011545 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v27) v1 ==== 184+0+0 (secure 0 0 0) 0x7f0914000e20 con 0x7f09180b65c0 2021-02-10T11:32:29.734 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.2 2021-02-10T11:32:29.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.725 7f0920717700 1 -- 172.21.2.101:0/441011545 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0900018730 msgr2=0x7f090001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:29.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.725 7f0920717700 1 --2- 172.21.2.101:0/441011545 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0900018730 0x7f090001abc0 crc :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:29.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.725 7f0920717700 1 -- 172.21.2.101:0/441011545 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09180b65c0 msgr2=0x7f09181a7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:29.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.725 7f0920717700 1 --2- 172.21.2.101:0/441011545 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09180b65c0 0x7f09181a7a10 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7f0914001c60 tx=0x7f091400a8d0).stop 2021-02-10T11:32:29.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.725 7f0920717700 1 -- 172.21.2.101:0/441011545 shutdown_connections 2021-02-10T11:32:29.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.725 7f0920717700 1 --2- 172.21.2.101:0/441011545 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0900018730 0x7f090001abc0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:29.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.725 7f0920717700 1 --2- 172.21.2.101:0/441011545 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09180b69c0 0x7f09180b7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:29.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.725 7f0920717700 1 --2- 172.21.2.101:0/441011545 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09181a8030 0x7f09180b7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:29.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.725 7f0920717700 1 --2- 172.21.2.101:0/441011545 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09180b65c0 0x7f09181a7a10 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:29.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.725 7f0920717700 1 -- 172.21.2.101:0/441011545 >> 172.21.2.101:0/441011545 conn(0x7f09180b5850 msgr2=0x7f0918093a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:29.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.725 7f0920717700 1 -- 172.21.2.101:0/441011545 shutdown_connections 2021-02-10T11:32:29.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:29.725 7f0920717700 1 -- 172.21.2.101:0/441011545 wait complete. 2021-02-10T11:32:29.744 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:32:30.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.154 7f3abd13d700 1 Processor -- start 2021-02-10T11:32:30.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.154 7f3abd13d700 1 -- start start 2021-02-10T11:32:30.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.155 7f3abd13d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3ab81b0870 0x7f3ab80f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:30.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.155 7f3abd13d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ab80efe00 0x7f3ab80f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:30.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.155 7f3abd13d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab81b0c70 con 0x7f3ab81b0470 2021-02-10T11:32:30.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.155 7f3abd13d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab80f0b50 con 0x7f3ab81b0870 2021-02-10T11:32:30.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.155 7f3abd13d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80f0820 con 0x7f3ab80efe00 2021-02-10T11:32:30.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.155 7f3ab6d9d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f3ab81b0470 0x7f3ab80ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51444/0 (socket says 172.21.2.101:51444) 2021-02-10T11:32:30.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.155 7f3ab6d9d700 1 -- 172.21.2.101:0/3513746845 learned_addr learned my addr 172.21.2.101:0/3513746845 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:30.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.155 7f3ab5d9b700 1 -- 172.21.2.101:0/3513746845 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3218804314 0 0) 0x7f3aa8001180 con 0x7f3ab81b0470 2021-02-10T11:32:30.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.155 7f3ab5d9b700 1 -- 172.21.2.101:0/3513746845 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3aa4002fe0 con 0x7f3ab81b0470 2021-02-10T11:32:30.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab759e700 1 --2- 172.21.2.101:0/3513746845 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ab80efe00 0x7f3ab80f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:30.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab659c700 1 --2- 172.21.2.101:0/3513746845 >> v2:172.21.2.108:3300/0 conn(0x7f3ab81b0870 0x7f3ab80f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:30.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab5d9b700 1 -- 172.21.2.101:0/3513746845 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3067757186 0 0) 0x7f3aa8001180 con 0x7f3ab81b0470 2021-02-10T11:32:30.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab5d9b700 1 -- 172.21.2.101:0/3513746845 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ab80efe00 msgr2=0x7f3ab80f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:30.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab5d9b700 1 --2- 172.21.2.101:0/3513746845 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ab80efe00 0x7f3ab80f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:30.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab5d9b700 1 -- 172.21.2.101:0/3513746845 >> v2:172.21.2.108:3300/0 conn(0x7f3ab81b0870 msgr2=0x7f3ab80f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:30.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab5d9b700 1 --2- 172.21.2.101:0/3513746845 >> v2:172.21.2.108:3300/0 conn(0x7f3ab81b0870 0x7f3ab80f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:30.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab5d9b700 1 -- 172.21.2.101:0/3513746845 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab80e52e0 con 0x7f3ab81b0470 2021-02-10T11:32:30.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab5d9b700 1 -- 172.21.2.101:0/3513746845 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f3aa8001680 con 0x7f3ab81b0470 2021-02-10T11:32:30.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab5d9b700 1 -- 172.21.2.101:0/3513746845 >> v1:172.21.2.101:6789/0 conn(0x7f3ab81b0470 legacy=0x7f3ab80ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:30.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab5d9b700 1 --2- 172.21.2.101:0/3513746845 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3aa4003a60 0x7f3aa4005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:30.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab5d9b700 1 --2- 172.21.2.101:0/3513746845 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3aa4006780 0x7f3aa4008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:30.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab5d9b700 1 --2- 172.21.2.101:0/3513746845 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3aa4009580 0x7f3aa400ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:30.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab5d9b700 1 -- 172.21.2.101:0/3513746845 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa400c2c0 con 0x7f3aa4006780 2021-02-10T11:32:30.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab5d9b700 1 -- 172.21.2.101:0/3513746845 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa40504b0 con 0x7f3aa4003a60 2021-02-10T11:32:30.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab5d9b700 1 -- 172.21.2.101:0/3513746845 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa40506d0 con 0x7f3aa4009580 2021-02-10T11:32:30.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab659c700 1 --2- 172.21.2.101:0/3513746845 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3aa4006780 0x7f3aa4008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:30.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab6d9d700 1 --2- 172.21.2.101:0/3513746845 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3aa4003a60 0x7f3aa4005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:30.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab659c700 1 -- 172.21.2.101:0/3513746845 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3aa4009580 msgr2=0x7f3aa400ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:30.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab659c700 1 --2- 172.21.2.101:0/3513746845 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3aa4009580 0x7f3aa400ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:30.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab659c700 1 -- 172.21.2.101:0/3513746845 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3aa4003a60 msgr2=0x7f3aa4005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:30.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab659c700 1 --2- 172.21.2.101:0/3513746845 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3aa4003a60 0x7f3aa4005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:30.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab659c700 1 -- 172.21.2.101:0/3513746845 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa4050970 con 0x7f3aa4006780 2021-02-10T11:32:30.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab659c700 1 --2- 172.21.2.101:0/3513746845 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3aa4006780 0x7f3aa4008c80 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 rx=0x7f3aa0006360 tx=0x7f3aa0009c30).ready entity=mon.0 client_cookie=6fe433681e46cf14 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:30.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab5d9b700 1 -- 172.21.2.101:0/3513746845 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3aa00009b0 con 0x7f3aa4006780 2021-02-10T11:32:30.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab5d9b700 1 -- 172.21.2.101:0/3513746845 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa0000c90 con 0x7f3aa4006780 2021-02-10T11:32:30.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3ab5d9b700 1 -- 172.21.2.101:0/3513746845 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3aa0002220 con 0x7f3aa4006780 2021-02-10T11:32:30.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3abd13d700 1 -- 172.21.2.101:0/3513746845 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3aa4006780 msgr2=0x7f3aa4008c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:30.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.156 7f3abd13d700 1 --2- 172.21.2.101:0/3513746845 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3aa4006780 0x7f3aa4008c80 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 rx=0x7f3aa0006360 tx=0x7f3aa0009c30).stop 2021-02-10T11:32:30.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.157 7f3ab759e700 1 -- 172.21.2.101:0/3513746845 reap_dead start 2021-02-10T11:32:30.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.157 7f3abd13d700 1 -- 172.21.2.101:0/3513746845 shutdown_connections 2021-02-10T11:32:30.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.157 7f3abd13d700 1 -- 172.21.2.101:0/3513746845 >> 172.21.2.101:0/3513746845 conn(0x7f3ab80ce3e0 msgr2=0x7f3ab80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:30.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.157 7f3abd13d700 1 -- 172.21.2.101:0/3513746845 shutdown_connections 2021-02-10T11:32:30.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.157 7f3abd13d700 1 -- 172.21.2.101:0/3513746845 wait complete. 2021-02-10T11:32:30.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.172 7f3abd13d700 1 Processor -- start 2021-02-10T11:32:30.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.172 7f3abd13d700 1 -- start start 2021-02-10T11:32:30.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.172 7f3abd13d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ab81a7960 0x7f3ab81a7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:30.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.172 7f3abd13d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ab808c440 0x7f3ab8189710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:30.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.172 7f3abd13d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ab8189d30 0x7f3ab818a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:30.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.172 7f3abd13d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab818a850 con 0x7f3ab81a7960 2021-02-10T11:32:30.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.172 7f3abd13d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab808c840 con 0x7f3ab8189d30 2021-02-10T11:32:30.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.172 7f3abd13d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab808cbb0 con 0x7f3ab808c440 2021-02-10T11:32:30.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.173 7f3ab6d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ab81a7960 0x7f3ab81a7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:30.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.173 7f3ab6d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ab81a7960 0x7f3ab81a7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50966/0 (socket says 172.21.2.101:50966) 2021-02-10T11:32:30.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.173 7f3ab6d9d700 1 -- 172.21.2.101:0/1419895814 learned_addr learned my addr 172.21.2.101:0/1419895814 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:30.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.173 7f3ab659c700 1 --2- 172.21.2.101:0/1419895814 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ab808c440 0x7f3ab8189710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:30.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.173 7f3ab759e700 1 --2- 172.21.2.101:0/1419895814 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ab8189d30 0x7f3ab818a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:30.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.173 7f3ab6d9d700 1 -- 172.21.2.101:0/1419895814 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ab808c440 msgr2=0x7f3ab8189710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:30.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.173 7f3ab6d9d700 1 --2- 172.21.2.101:0/1419895814 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ab808c440 0x7f3ab8189710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:30.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.173 7f3ab6d9d700 1 -- 172.21.2.101:0/1419895814 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ab8189d30 msgr2=0x7f3ab818a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:30.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.173 7f3ab6d9d700 1 --2- 172.21.2.101:0/1419895814 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ab8189d30 0x7f3ab818a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:30.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.173 7f3ab6d9d700 1 -- 172.21.2.101:0/1419895814 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab818ad50 con 0x7f3ab81a7960 2021-02-10T11:32:30.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.173 7f3ab659c700 1 --2- 172.21.2.101:0/1419895814 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ab808c440 0x7f3ab8189710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:32:30.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.173 7f3ab6d9d700 1 --2- 172.21.2.101:0/1419895814 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ab81a7960 0x7f3ab81a7d60 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7f3aa8001e90 tx=0x7f3aa800a8f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:30.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.173 7f3aaf7fe700 1 -- 172.21.2.101:0/1419895814 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3aa8000ef0 con 0x7f3ab81a7960 2021-02-10T11:32:30.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.173 7f3aaf7fe700 1 -- 172.21.2.101:0/1419895814 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa80043a0 con 0x7f3ab81a7960 2021-02-10T11:32:30.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.173 7f3aaf7fe700 1 -- 172.21.2.101:0/1419895814 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3aa80048c0 con 0x7f3ab81a7960 2021-02-10T11:32:30.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.173 7f3abd13d700 1 -- 172.21.2.101:0/1419895814 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab818ad50 con 0x7f3ab81a7960 2021-02-10T11:32:30.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.173 7f3abd13d700 1 -- 172.21.2.101:0/1419895814 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab818ad50 con 0x7f3ab81a7960 2021-02-10T11:32:30.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.174 7f3aaf7fe700 1 -- 172.21.2.101:0/1419895814 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3aa8008000 con 0x7f3ab81a7960 2021-02-10T11:32:30.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.174 7f3aaf7fe700 1 --2- 172.21.2.101:0/1419895814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a9c0186d0 0x7f3a9c01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:30.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.174 7f3aaf7fe700 1 -- 172.21.2.101:0/1419895814 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3aa8032780 con 0x7f3ab81a7960 2021-02-10T11:32:30.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.174 7f3ab659c700 1 --2- 172.21.2.101:0/1419895814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a9c0186d0 0x7f3a9c01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:30.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.174 7f3ab659c700 1 --2- 172.21.2.101:0/1419895814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a9c0186d0 0x7f3a9c01abd0 crc :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:30.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.205 7f3abd13d700 1 -- 172.21.2.101:0/1419895814 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab8105a30 con 0x7f3ab81a7960 2021-02-10T11:32:30.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.207 7f3aaf7fe700 1 -- 172.21.2.101:0/1419895814 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3aa8000940 con 0x7f3ab81a7960 2021-02-10T11:32:30.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.337 7f3abd13d700 1 -- 172.21.2.101:0/1419895814 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f3ab8105a30 con 0x7f3ab81a7960 2021-02-10T11:32:30.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.340 7f3aaf7fe700 1 -- 172.21.2.101:0/1419895814 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v28) v1 ==== 184+0+0 (secure 0 0 0) 0x7f3aa8000940 con 0x7f3ab81a7960 2021-02-10T11:32:30.355 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.3 2021-02-10T11:32:30.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.346 7f3abd13d700 1 -- 172.21.2.101:0/1419895814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a9c0186d0 msgr2=0x7f3a9c01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:30.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.346 7f3abd13d700 1 --2- 172.21.2.101:0/1419895814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a9c0186d0 0x7f3a9c01abd0 crc :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:30.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.346 7f3abd13d700 1 -- 172.21.2.101:0/1419895814 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ab81a7960 msgr2=0x7f3ab81a7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:30.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.346 7f3abd13d700 1 --2- 172.21.2.101:0/1419895814 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ab81a7960 0x7f3ab81a7d60 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7f3aa8001e90 tx=0x7f3aa800a8f0).stop 2021-02-10T11:32:30.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.346 7f3abd13d700 1 -- 172.21.2.101:0/1419895814 shutdown_connections 2021-02-10T11:32:30.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.346 7f3abd13d700 1 --2- 172.21.2.101:0/1419895814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a9c0186d0 0x7f3a9c01abd0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:30.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.346 7f3abd13d700 1 --2- 172.21.2.101:0/1419895814 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ab808c440 0x7f3ab8189710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:30.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.346 7f3abd13d700 1 --2- 172.21.2.101:0/1419895814 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ab8189d30 0x7f3ab818a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:30.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.346 7f3abd13d700 1 --2- 172.21.2.101:0/1419895814 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ab81a7960 0x7f3ab81a7d60 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:30.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.346 7f3abd13d700 1 -- 172.21.2.101:0/1419895814 >> 172.21.2.101:0/1419895814 conn(0x7f3ab80b5850 msgr2=0x7f3ab8091a50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:30.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.346 7f3abd13d700 1 -- 172.21.2.101:0/1419895814 shutdown_connections 2021-02-10T11:32:30.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.346 7f3abd13d700 1 -- 172.21.2.101:0/1419895814 wait complete. 2021-02-10T11:32:30.365 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:32:30.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.761 7effd75cb700 1 Processor -- start 2021-02-10T11:32:30.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.762 7effd75cb700 1 -- start start 2021-02-10T11:32:30.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.762 7effd75cb700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7effd00e5070 0x7effd01b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:30.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.762 7effd75cb700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effd007ed80 con 0x7effd01b3e40 2021-02-10T11:32:30.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.762 7effd75cb700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effd007e670 con 0x7effd00e5470 2021-02-10T11:32:30.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.762 7effd75cb700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd007e890 con 0x7effd00e5070 2021-02-10T11:32:30.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.762 7effd4be6700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7effd01b3e40 0x7effd007d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51464/0 (socket says 172.21.2.101:51464) 2021-02-10T11:32:30.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.762 7effd4be6700 1 -- 172.21.2.101:0/3103228876 learned_addr learned my addr 172.21.2.101:0/3103228876 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:30.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.762 7effd53e7700 1 --2- 172.21.2.101:0/3103228876 >> v2:172.21.2.101:3300/0 conn(0x7effd00e5070 0x7effd01b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:30.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.763 7effd53e7700 1 -- 172.21.2.101:0/3103228876 >> v1:172.21.2.108:6789/0 conn(0x7effd00e5470 legacy=0x7effd007de80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:32:30.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.763 7effd53e7700 1 -- 172.21.2.101:0/3103228876 >> v1:172.21.2.101:6789/0 conn(0x7effd01b3e40 legacy=0x7effd007d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:30.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.763 7effd53e7700 1 -- 172.21.2.101:0/3103228876 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd007eab0 con 0x7effd00e5070 2021-02-10T11:32:30.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.763 7effd53e7700 1 --2- 172.21.2.101:0/3103228876 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7effd00e5070 0x7effd01b3820 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7effc4006650 tx=0x7effc4009a80).ready entity=mon.0 client_cookie=57ed1eaac13dbf38 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:30.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.763 7effc3fff700 1 -- 172.21.2.101:0/3103228876 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7effc400c0c0 con 0x7effd00e5070 2021-02-10T11:32:30.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.763 7effc3fff700 1 -- 172.21.2.101:0/3103228876 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc400c340 con 0x7effd00e5070 2021-02-10T11:32:30.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.763 7effc3fff700 1 -- 172.21.2.101:0/3103228876 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7effc400c860 con 0x7effd00e5070 2021-02-10T11:32:30.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.763 7effd75cb700 1 -- 172.21.2.101:0/3103228876 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7effd00e5070 msgr2=0x7effd01b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:30.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.763 7effd75cb700 1 --2- 172.21.2.101:0/3103228876 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7effd00e5070 0x7effd01b3820 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7effc4006650 tx=0x7effc4009a80).stop 2021-02-10T11:32:30.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.765 7effd75cb700 1 -- 172.21.2.101:0/3103228876 shutdown_connections 2021-02-10T11:32:30.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.765 7effd75cb700 1 --2- 172.21.2.101:0/3103228876 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7effd00e5070 0x7effd01b3820 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:30.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.765 7effd75cb700 1 -- 172.21.2.101:0/3103228876 >> 172.21.2.101:0/3103228876 conn(0x7effd00ce690 msgr2=0x7effd00e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:30.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.765 7effd75cb700 1 -- 172.21.2.101:0/3103228876 shutdown_connections 2021-02-10T11:32:30.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.765 7effd75cb700 1 -- 172.21.2.101:0/3103228876 wait complete. 2021-02-10T11:32:30.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.779 7effd75cb700 1 Processor -- start 2021-02-10T11:32:30.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.779 7effd75cb700 1 -- start start 2021-02-10T11:32:30.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.779 7effd75cb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7effd00befe0 0x7effd00bf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:30.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.779 7effd75cb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7effd0094dd0 0x7effd00951d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:30.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.779 7effd75cb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7effd00bfa70 0x7effd00bfe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:30.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.779 7effd75cb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd0095b30 con 0x7effd00befe0 2021-02-10T11:32:30.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.779 7effd75cb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd0095830 con 0x7effd0094dd0 2021-02-10T11:32:30.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.779 7effd75cb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd00a19e0 con 0x7effd00bfa70 2021-02-10T11:32:30.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.779 7effd53e7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7effd00befe0 0x7effd00bf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:30.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.779 7effd53e7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7effd00befe0 0x7effd00bf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50974/0 (socket says 172.21.2.101:50974) 2021-02-10T11:32:30.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.779 7effd53e7700 1 -- 172.21.2.101:0/346343770 learned_addr learned my addr 172.21.2.101:0/346343770 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:30.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.780 7effd53e7700 1 -- 172.21.2.101:0/346343770 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7effd00bfa70 msgr2=0x7effd00bfe70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:30.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.780 7effd53e7700 1 --2- 172.21.2.101:0/346343770 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7effd00bfa70 0x7effd00bfe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:30.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.780 7effd53e7700 1 -- 172.21.2.101:0/346343770 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7effd0094dd0 msgr2=0x7effd00951d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:30.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.780 7effd53e7700 1 --2- 172.21.2.101:0/346343770 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7effd0094dd0 0x7effd00951d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:30.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.780 7effd53e7700 1 -- 172.21.2.101:0/346343770 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd00c0490 con 0x7effd00befe0 2021-02-10T11:32:30.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.780 7effd53e7700 1 --2- 172.21.2.101:0/346343770 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7effd00befe0 0x7effd00bf3e0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7effc4009630 tx=0x7effc400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:30.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.780 7effc1ffb700 1 -- 172.21.2.101:0/346343770 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7effc40093a0 con 0x7effd00befe0 2021-02-10T11:32:30.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.780 7effc1ffb700 1 -- 172.21.2.101:0/346343770 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc400bd70 con 0x7effd00befe0 2021-02-10T11:32:30.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.780 7effc1ffb700 1 -- 172.21.2.101:0/346343770 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7effc4005560 con 0x7effd00befe0 2021-02-10T11:32:30.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.780 7effd75cb700 1 -- 172.21.2.101:0/346343770 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd00c0490 con 0x7effd00befe0 2021-02-10T11:32:30.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.780 7effd75cb700 1 -- 172.21.2.101:0/346343770 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd00c0a20 con 0x7effd00befe0 2021-02-10T11:32:30.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.781 7effc1ffb700 1 -- 172.21.2.101:0/346343770 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7effc400a000 con 0x7effd00befe0 2021-02-10T11:32:30.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.781 7effc1ffb700 1 --2- 172.21.2.101:0/346343770 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7effb00186c0 0x7effb001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:30.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.781 7effc1ffb700 1 -- 172.21.2.101:0/346343770 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7effc4032560 con 0x7effd00befe0 2021-02-10T11:32:30.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.781 7effd4be6700 1 --2- 172.21.2.101:0/346343770 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7effb00186c0 0x7effb001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:30.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.781 7effd4be6700 1 --2- 172.21.2.101:0/346343770 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7effb00186c0 0x7effb001abc0 crc :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:30.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.812 7effd75cb700 1 -- 172.21.2.101:0/346343770 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd0095ab0 con 0x7effd00befe0 2021-02-10T11:32:30.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.815 7effc1ffb700 1 -- 172.21.2.101:0/346343770 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7effc40093a0 con 0x7effd00befe0 2021-02-10T11:32:30.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.942 7effd75cb700 1 -- 172.21.2.101:0/346343770 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effd00ce690 con 0x7effd00befe0 2021-02-10T11:32:30.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.942 7effc1ffb700 1 -- 172.21.2.101:0/346343770 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+2099 (secure 0 0 0) 0x7effc400a000 con 0x7effd00befe0 2021-02-10T11:32:30.954 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 31 2021-02-10T11:32:30.954 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:32:30.954 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7674,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1796077990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1796077990},{"type":"v1","addr":"172.21.2.101:6835","nonce":1796077990}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:32:24.604347","modified":"2021-02-10 11:32:26.612664","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5986},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5986":{"gid":5986,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/3346373345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3346373345},{"type":"v1","addr":"172.21.2.108:6833","nonce":3346373345}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-02-10T11:32:30.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.945 7effd75cb700 1 -- 172.21.2.101:0/346343770 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7effb00186c0 msgr2=0x7effb001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:30.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.945 7effd75cb700 1 --2- 172.21.2.101:0/346343770 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7effb00186c0 0x7effb001abc0 crc :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:30.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.945 7effd75cb700 1 -- 172.21.2.101:0/346343770 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7effd00befe0 msgr2=0x7effd00bf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:30.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.945 7effd75cb700 1 --2- 172.21.2.101:0/346343770 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7effd00befe0 0x7effd00bf3e0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7effc4009630 tx=0x7effc400b980).stop 2021-02-10T11:32:30.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.945 7effd75cb700 1 -- 172.21.2.101:0/346343770 shutdown_connections 2021-02-10T11:32:30.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.945 7effd75cb700 1 --2- 172.21.2.101:0/346343770 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7effb00186c0 0x7effb001abc0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:30.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.945 7effd75cb700 1 --2- 172.21.2.101:0/346343770 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7effd00bfa70 0x7effd00bfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:30.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.945 7effd75cb700 1 --2- 172.21.2.101:0/346343770 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7effd0094dd0 0x7effd00951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:30.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.945 7effd75cb700 1 --2- 172.21.2.101:0/346343770 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7effd00befe0 0x7effd00bf3e0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:30.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.945 7effd75cb700 1 -- 172.21.2.101:0/346343770 >> 172.21.2.101:0/346343770 conn(0x7effd0106df0 msgr2=0x7effd00962d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:30.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.945 7effd75cb700 1 -- 172.21.2.101:0/346343770 shutdown_connections 2021-02-10T11:32:30.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:30.945 7effd75cb700 1 -- 172.21.2.101:0/346343770 wait complete. 2021-02-10T11:32:31.165 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:32:24.604347', 'modified': '2021-02-10 11:32:26.612664', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5986}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5986': {'gid': 5986, 'name': 'b', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/3346373345', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 3346373345}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 3346373345}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:32:31.165 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:32:31.165 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:32:31.564 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:32:31.564 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:32:31.564 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:32:31.564 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 5986, 2021-02-10T11:32:31.564 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:32:31.564 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:32:31.564 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 6.9462377640000001, 2021-02-10T11:32:31.564 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 31, 2021-02-10T11:32:31.564 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 47, 2021-02-10T11:32:31.564 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 47, 2021-02-10T11:32:31.564 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 12.585200037 2021-02-10T11:32:31.565 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:32:31.568 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 5986, "want_state": "up:active", "state": "up:active", "rank_uptime": 6.9462377640000001, "mdsmap_epoch": 31, "osdmap_epoch": 47, "osdmap_epoch_barrier": 47, "uptime": 12.585200037 } 2021-02-10T11:32:31.568 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:32:31.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.969 7f34939e9700 1 Processor -- start 2021-02-10T11:32:31.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.969 7f34939e9700 1 -- start start 2021-02-10T11:32:31.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.969 7f34939e9700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f348c0e5070 0x7f348c1b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:31.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.969 7f34939e9700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f348c0e5470 0x7f348c080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:31.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.969 7f34939e9700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348c07ed80 con 0x7f348c1b3e40 2021-02-10T11:32:31.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.969 7f34939e9700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348c0d1790 con 0x7f348c0e5470 2021-02-10T11:32:31.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.969 7f34939e9700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348c0d19b0 con 0x7f348c0e5070 2021-02-10T11:32:31.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.970 7f3491004700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f348c1b3e40 0x7f348c07d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51476/0 (socket says 172.21.2.101:51476) 2021-02-10T11:32:31.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.970 7f3492006700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f348c0e5470 0x7f348c080310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:31.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.970 7f3491004700 1 -- 172.21.2.101:0/4143660573 learned_addr learned my addr 172.21.2.101:0/4143660573 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:31.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.970 7f3490803700 1 -- 172.21.2.101:0/4143660573 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 27703656 0 0) 0x7f347c0010b0 con 0x7f348c1b3e40 2021-02-10T11:32:31.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.970 7f3490803700 1 -- 172.21.2.101:0/4143660573 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3478002fe0 con 0x7f348c1b3e40 2021-02-10T11:32:31.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.970 7f3492006700 1 -- 172.21.2.101:0/4143660573 >> v2:172.21.2.108:3300/0 conn(0x7f348c0e5070 msgr2=0x7f348c1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:31.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.970 7f3491805700 1 --2- 172.21.2.101:0/4143660573 >> v2:172.21.2.108:3300/0 conn(0x7f348c0e5070 0x7f348c1b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:31.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.970 7f3492006700 1 --2- 172.21.2.101:0/4143660573 >> v2:172.21.2.108:3300/0 conn(0x7f348c0e5070 0x7f348c1b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:31.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.970 7f3492006700 1 -- 172.21.2.101:0/4143660573 >> v1:172.21.2.101:6789/0 conn(0x7f348c1b3e40 legacy=0x7f348c07d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:31.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.970 7f3492006700 1 -- 172.21.2.101:0/4143660573 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c0d1bd0 con 0x7f348c0e5470 2021-02-10T11:32:31.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.970 7f3491805700 1 --2- 172.21.2.101:0/4143660573 >> v2:172.21.2.108:3300/0 conn(0x7f348c0e5070 0x7f348c1b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:32:31.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.970 7f3492006700 1 --2- 172.21.2.101:0/4143660573 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f348c0e5470 0x7f348c080310 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f3484006e30 tx=0x7f3484009e60).ready entity=mon.0 client_cookie=622cdd31d629f1c1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:31.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.971 7f3490803700 1 -- 172.21.2.101:0/4143660573 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3484006a50 con 0x7f348c0e5470 2021-02-10T11:32:31.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.971 7f3490803700 1 -- 172.21.2.101:0/4143660573 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348400b300 con 0x7f348c0e5470 2021-02-10T11:32:31.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.971 7f3490803700 1 -- 172.21.2.101:0/4143660573 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f348400b820 con 0x7f348c0e5470 2021-02-10T11:32:31.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.971 7f34939e9700 1 -- 172.21.2.101:0/4143660573 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f348c0e5470 msgr2=0x7f348c080310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:31.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.971 7f34939e9700 1 --2- 172.21.2.101:0/4143660573 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f348c0e5470 0x7f348c080310 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f3484006e30 tx=0x7f3484009e60).stop 2021-02-10T11:32:31.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.971 7f34939e9700 1 -- 172.21.2.101:0/4143660573 shutdown_connections 2021-02-10T11:32:31.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.971 7f34939e9700 1 --2- 172.21.2.101:0/4143660573 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f348c0e5470 0x7f348c080310 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:31.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.971 7f34939e9700 1 --2- 172.21.2.101:0/4143660573 >> v2:172.21.2.108:3300/0 conn(0x7f348c0e5070 0x7f348c1b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:31.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.971 7f34939e9700 1 -- 172.21.2.101:0/4143660573 >> 172.21.2.101:0/4143660573 conn(0x7f348c0ce690 msgr2=0x7f348c0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:31.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.972 7f34939e9700 1 -- 172.21.2.101:0/4143660573 shutdown_connections 2021-02-10T11:32:31.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.972 7f34939e9700 1 -- 172.21.2.101:0/4143660573 wait complete. 2021-02-10T11:32:32.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.986 7f34939e9700 1 Processor -- start 2021-02-10T11:32:32.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.986 7f34939e9700 1 -- start start 2021-02-10T11:32:32.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.987 7f34939e9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f348c08d400 0x7f348c081230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:32.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.987 7f34939e9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f348c081850 0x7f348c0acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:32.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.987 7f34939e9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f348c081c50 0x7f348c0ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:32.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.987 7f34939e9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c082110 con 0x7f348c081850 2021-02-10T11:32:32.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.987 7f34939e9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c08d800 con 0x7f348c08d400 2021-02-10T11:32:32.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.987 7f34939e9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c0adbc0 con 0x7f348c081c50 2021-02-10T11:32:32.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.987 7f3491004700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f348c081850 0x7f348c0acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:32.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.987 7f3491004700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f348c081850 0x7f348c0acf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:50990/0 (socket says 172.21.2.101:50990) 2021-02-10T11:32:32.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.987 7f3491004700 1 -- 172.21.2.101:0/3549005432 learned_addr learned my addr 172.21.2.101:0/3549005432 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:32.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.987 7f3491805700 1 --2- 172.21.2.101:0/3549005432 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f348c08d400 0x7f348c081230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:32.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.987 7f3492006700 1 --2- 172.21.2.101:0/3549005432 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f348c081c50 0x7f348c0ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:32.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.987 7f3491004700 1 -- 172.21.2.101:0/3549005432 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f348c081c50 msgr2=0x7f348c0ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:32.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.987 7f3491004700 1 --2- 172.21.2.101:0/3549005432 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f348c081c50 0x7f348c0ad5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:32.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.987 7f3491004700 1 -- 172.21.2.101:0/3549005432 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f348c08d400 msgr2=0x7f348c081230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:32.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.987 7f3491004700 1 --2- 172.21.2.101:0/3549005432 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f348c08d400 0x7f348c081230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:32.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.987 7f3491004700 1 -- 172.21.2.101:0/3549005432 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c187fc0 con 0x7f348c081850 2021-02-10T11:32:32.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.987 7f3492006700 1 --2- 172.21.2.101:0/3549005432 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f348c081c50 0x7f348c0ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:32:32.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.987 7f3491805700 1 --2- 172.21.2.101:0/3549005432 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f348c08d400 0x7f348c081230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:32:32.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.988 7f3491004700 1 --2- 172.21.2.101:0/3549005432 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f348c081850 0x7f348c0acf80 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f347c001c60 tx=0x7f347c00a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:32.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.988 7f34827fc700 1 -- 172.21.2.101:0/3549005432 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f347c000cc0 con 0x7f348c081850 2021-02-10T11:32:32.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.988 7f34827fc700 1 -- 172.21.2.101:0/3549005432 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347c00acc0 con 0x7f348c081850 2021-02-10T11:32:32.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.988 7f34827fc700 1 -- 172.21.2.101:0/3549005432 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f347c0045d0 con 0x7f348c081850 2021-02-10T11:32:32.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.988 7f34939e9700 1 -- 172.21.2.101:0/3549005432 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f348c187fc0 con 0x7f348c081850 2021-02-10T11:32:32.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.988 7f34939e9700 1 -- 172.21.2.101:0/3549005432 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348c187fc0 con 0x7f348c081850 2021-02-10T11:32:32.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.990 7f34827fc700 1 -- 172.21.2.101:0/3549005432 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f347c004140 con 0x7f348c081850 2021-02-10T11:32:32.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.990 7f34827fc700 1 --2- 172.21.2.101:0/3549005432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f34740186c0 0x7f347401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:32.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.990 7f34827fc700 1 -- 172.21.2.101:0/3549005432 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f347c032560 con 0x7f348c081850 2021-02-10T11:32:32.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.990 7f3491805700 1 --2- 172.21.2.101:0/3549005432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f34740186c0 0x7f347401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:32.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:31.990 7f3491805700 1 --2- 172.21.2.101:0/3549005432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f34740186c0 0x7f347401abc0 crc :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:32.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.020 7f34939e9700 1 -- 172.21.2.101:0/3549005432 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f348c07ed80 con 0x7f348c081850 2021-02-10T11:32:32.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.022 7f34827fc700 1 -- 172.21.2.101:0/3549005432 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f347c000cc0 con 0x7f348c081850 2021-02-10T11:32:32.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.146 7f34939e9700 1 -- 172.21.2.101:0/3549005432 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f348c07ed80 con 0x7f348c081850 2021-02-10T11:32:32.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.146 7f34827fc700 1 -- 172.21.2.101:0/3549005432 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+13993 (secure 0 0 0) 0x7f347c004140 con 0x7f348c081850 2021-02-10T11:32:32.158 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:32:32.158 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":47,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:32:25.606853","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:32:22.580828","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2021-02-10 11:32:23.597744","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:32:32.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.148 7f34939e9700 1 -- 172.21.2.101:0/3549005432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f34740186c0 msgr2=0x7f347401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:32.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.148 7f34939e9700 1 --2- 172.21.2.101:0/3549005432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f34740186c0 0x7f347401abc0 crc :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:32.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.148 7f34939e9700 1 -- 172.21.2.101:0/3549005432 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f348c081850 msgr2=0x7f348c0acf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:32.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.148 7f34939e9700 1 --2- 172.21.2.101:0/3549005432 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f348c081850 0x7f348c0acf80 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f347c001c60 tx=0x7f347c00a8d0).stop 2021-02-10T11:32:32.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.148 7f34939e9700 1 -- 172.21.2.101:0/3549005432 shutdown_connections 2021-02-10T11:32:32.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.148 7f34939e9700 1 --2- 172.21.2.101:0/3549005432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f34740186c0 0x7f347401abc0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:32.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.148 7f34939e9700 1 --2- 172.21.2.101:0/3549005432 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f348c081c50 0x7f348c0ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:32.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.148 7f34939e9700 1 --2- 172.21.2.101:0/3549005432 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f348c081850 0x7f348c0acf80 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:32.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.148 7f34939e9700 1 --2- 172.21.2.101:0/3549005432 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f348c08d400 0x7f348c081230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:32.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.148 7f34939e9700 1 -- 172.21.2.101:0/3549005432 >> 172.21.2.101:0/3549005432 conn(0x7f348c1ac620 msgr2=0x7f348c0896e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:32.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.149 7f34939e9700 1 -- 172.21.2.101:0/3549005432 shutdown_connections 2021-02-10T11:32:32.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.149 7f34939e9700 1 -- 172.21.2.101:0/3549005432 wait complete. 2021-02-10T11:32:32.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:32:22.580828', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:32:23.597744', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:32:32.169 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:32:32.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.570 7f95b16bb700 1 Processor -- start 2021-02-10T11:32:32.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.570 7f95b16bb700 1 -- start start 2021-02-10T11:32:32.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.570 7f95b16bb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95ac0f05f0 0x7f95ac0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:32.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.570 7f95b16bb700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac1b0c70 con 0x7f95ac1b0870 2021-02-10T11:32:32.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.570 7f95b16bb700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac0f0b50 con 0x7f95ac1b0470 2021-02-10T11:32:32.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.570 7f95b16bb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0e5ea0 con 0x7f95ac0f05f0 2021-02-10T11:32:32.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.571 7f95aaabe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f95ac1b0870 0x7f95ac0efe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51488/0 (socket says 172.21.2.101:51488) 2021-02-10T11:32:32.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.571 7f95aaabe700 1 -- 172.21.2.101:0/537568183 learned_addr learned my addr 172.21.2.101:0/537568183 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:32.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.571 7f95aa2bd700 1 -- 172.21.2.101:0/537568183 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2057366712 0 0) 0x7f95a0000ce0 con 0x7f95ac1b0870 2021-02-10T11:32:32.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.571 7f95aa2bd700 1 -- 172.21.2.101:0/537568183 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9594002fe0 con 0x7f95ac1b0870 2021-02-10T11:32:32.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.571 7f95abac0700 1 --2- 172.21.2.101:0/537568183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95ac0f05f0 0x7f95ac0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:32.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95aa2bd700 1 -- 172.21.2.101:0/537568183 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3525461096 0 0) 0x7f95a0001180 con 0x7f95ac1b0870 2021-02-10T11:32:32.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95aa2bd700 1 -- 172.21.2.101:0/537568183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95ac0f05f0 msgr2=0x7f95ac0f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:32.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95aa2bd700 1 --2- 172.21.2.101:0/537568183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95ac0f05f0 0x7f95ac0f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:32.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95aa2bd700 1 -- 172.21.2.101:0/537568183 >> v1:172.21.2.108:6789/0 conn(0x7f95ac1b0470 legacy=0x7f95ac0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:32.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95aa2bd700 1 -- 172.21.2.101:0/537568183 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac0e52e0 con 0x7f95ac1b0870 2021-02-10T11:32:32.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95aa2bd700 1 -- 172.21.2.101:0/537568183 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f95a0001680 con 0x7f95ac1b0870 2021-02-10T11:32:32.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95aa2bd700 1 -- 172.21.2.101:0/537568183 >> v1:172.21.2.101:6789/0 conn(0x7f95ac1b0870 legacy=0x7f95ac0efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:32.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95aa2bd700 1 --2- 172.21.2.101:0/537568183 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9594003af0 0x7f9594005f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:32.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95aa2bd700 1 --2- 172.21.2.101:0/537568183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95940067a0 0x7f9594008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:32.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95aa2bd700 1 --2- 172.21.2.101:0/537568183 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9594009510 0x7f959400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:32.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95aa2bd700 1 -- 172.21.2.101:0/537568183 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959400c2c0 con 0x7f9594003af0 2021-02-10T11:32:32.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95aa2bd700 1 -- 172.21.2.101:0/537568183 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959400c4e0 con 0x7f9594009510 2021-02-10T11:32:32.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95aa2bd700 1 -- 172.21.2.101:0/537568183 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f959400c700 con 0x7f95940067a0 2021-02-10T11:32:32.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95ab2bf700 1 --2- 172.21.2.101:0/537568183 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9594003af0 0x7f9594005f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:32.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95abac0700 1 --2- 172.21.2.101:0/537568183 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9594009510 0x7f959400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:32.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95aaabe700 1 --2- 172.21.2.101:0/537568183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95940067a0 0x7f9594008c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:32.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95ab2bf700 1 -- 172.21.2.101:0/537568183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95940067a0 msgr2=0x7f9594008c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:32.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95ab2bf700 1 --2- 172.21.2.101:0/537568183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95940067a0 0x7f9594008c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:32.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95ab2bf700 1 -- 172.21.2.101:0/537568183 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9594009510 msgr2=0x7f959400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:32.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95ab2bf700 1 --2- 172.21.2.101:0/537568183 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9594009510 0x7f959400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:32.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95ab2bf700 1 -- 172.21.2.101:0/537568183 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9594050970 con 0x7f9594003af0 2021-02-10T11:32:32.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95aaabe700 1 --2- 172.21.2.101:0/537568183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95940067a0 0x7f9594008c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:32:32.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.572 7f95ab2bf700 1 --2- 172.21.2.101:0/537568183 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9594003af0 0x7f9594005f00 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f959c006ab0 tx=0x7f959c009b40).ready entity=mon.0 client_cookie=523be5446ba4734 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:32.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.573 7f95aa2bd700 1 -- 172.21.2.101:0/537568183 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f959c00b2e0 con 0x7f9594003af0 2021-02-10T11:32:32.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.573 7f95aa2bd700 1 -- 172.21.2.101:0/537568183 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c00b560 con 0x7f9594003af0 2021-02-10T11:32:32.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.573 7f95aa2bd700 1 -- 172.21.2.101:0/537568183 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f959c00ba80 con 0x7f9594003af0 2021-02-10T11:32:32.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.573 7f95b16bb700 1 -- 172.21.2.101:0/537568183 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9594003af0 msgr2=0x7f9594005f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:32.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.573 7f95b16bb700 1 --2- 172.21.2.101:0/537568183 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9594003af0 0x7f9594005f00 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f959c006ab0 tx=0x7f959c009b40).stop 2021-02-10T11:32:32.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.573 7f95abac0700 1 -- 172.21.2.101:0/537568183 reap_dead start 2021-02-10T11:32:32.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.573 7f95b16bb700 1 -- 172.21.2.101:0/537568183 shutdown_connections 2021-02-10T11:32:32.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.573 7f95b16bb700 1 -- 172.21.2.101:0/537568183 >> 172.21.2.101:0/537568183 conn(0x7f95ac0ce3e0 msgr2=0x7f95ac0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:32.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.573 7f95b16bb700 1 -- 172.21.2.101:0/537568183 shutdown_connections 2021-02-10T11:32:32.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.573 7f95b16bb700 1 -- 172.21.2.101:0/537568183 wait complete. 2021-02-10T11:32:32.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.588 7f95b16bb700 1 Processor -- start 2021-02-10T11:32:32.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.588 7f95b16bb700 1 -- start start 2021-02-10T11:32:32.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.589 7f95b16bb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f95ac092c40 0x7f95ac0b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:32.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.589 7f95b16bb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95ac093040 0x7f95ac106a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:32.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.589 7f95b16bb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95ac107060 0x7f95ac107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:32.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.589 7f95b16bb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0c2120 con 0x7f95ac107060 2021-02-10T11:32:32.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.589 7f95b16bb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0b3c00 con 0x7f95ac092c40 2021-02-10T11:32:32.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.589 7f95b16bb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0b3780 con 0x7f95ac093040 2021-02-10T11:32:32.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.589 7f95abac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95ac107060 0x7f95ac107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:32.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.589 7f95abac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95ac107060 0x7f95ac107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51006/0 (socket says 172.21.2.101:51006) 2021-02-10T11:32:32.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.589 7f95abac0700 1 -- 172.21.2.101:0/1826859228 learned_addr learned my addr 172.21.2.101:0/1826859228 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:32.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.589 7f95abac0700 1 -- 172.21.2.101:0/1826859228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95ac093040 msgr2=0x7f95ac106a40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:32.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.589 7f95abac0700 1 --2- 172.21.2.101:0/1826859228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95ac093040 0x7f95ac106a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:32.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.589 7f95abac0700 1 -- 172.21.2.101:0/1826859228 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f95ac092c40 msgr2=0x7f95ac0b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:32.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.589 7f95abac0700 1 --2- 172.21.2.101:0/1826859228 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f95ac092c40 0x7f95ac0b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:32.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.589 7f95abac0700 1 -- 172.21.2.101:0/1826859228 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac0c0ba0 con 0x7f95ac107060 2021-02-10T11:32:32.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.589 7f95abac0700 1 --2- 172.21.2.101:0/1826859228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95ac107060 0x7f95ac107460 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0x7f95a4009a30 tx=0x7f95a400bd20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:32.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.590 7f9593fff700 1 -- 172.21.2.101:0/1826859228 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f95a4006bf0 con 0x7f95ac107060 2021-02-10T11:32:32.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.590 7f9593fff700 1 -- 172.21.2.101:0/1826859228 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a400be60 con 0x7f95ac107060 2021-02-10T11:32:32.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.590 7f9593fff700 1 -- 172.21.2.101:0/1826859228 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f95a40015f0 con 0x7f95ac107060 2021-02-10T11:32:32.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.590 7f95b16bb700 1 -- 172.21.2.101:0/1826859228 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ac0c0ba0 con 0x7f95ac107060 2021-02-10T11:32:32.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.590 7f95b16bb700 1 -- 172.21.2.101:0/1826859228 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac0c0ba0 con 0x7f95ac107060 2021-02-10T11:32:32.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.590 7f9593fff700 1 -- 172.21.2.101:0/1826859228 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f95a40069b0 con 0x7f95ac107060 2021-02-10T11:32:32.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.590 7f9593fff700 1 --2- 172.21.2.101:0/1826859228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f958c0186c0 0x7f958c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:32.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.590 7f9593fff700 1 -- 172.21.2.101:0/1826859228 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f95a4032900 con 0x7f95ac107060 2021-02-10T11:32:32.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.590 7f95ab2bf700 1 --2- 172.21.2.101:0/1826859228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f958c0186c0 0x7f958c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:32.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.591 7f95ab2bf700 1 --2- 172.21.2.101:0/1826859228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f958c0186c0 0x7f958c01abc0 crc :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:32.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.620 7f95b16bb700 1 -- 172.21.2.101:0/1826859228 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ac0b3780 con 0x7f95ac107060 2021-02-10T11:32:32.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.623 7f9593fff700 1 -- 172.21.2.101:0/1826859228 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f95a401f880 con 0x7f95ac107060 2021-02-10T11:32:32.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.752 7f95b16bb700 1 -- 172.21.2.101:0/1826859228 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f95ac0b3780 con 0x7f95ac107060 2021-02-10T11:32:32.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.752 7f9593fff700 1 -- 172.21.2.101:0/1826859228 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v47) v1 ==== 92+0+10 (secure 0 0 0) 0x7f95a40069b0 con 0x7f95ac107060 2021-02-10T11:32:32.764 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:32:32.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.754 7f95b16bb700 1 -- 172.21.2.101:0/1826859228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f958c0186c0 msgr2=0x7f958c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:32.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.754 7f95b16bb700 1 --2- 172.21.2.101:0/1826859228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f958c0186c0 0x7f958c01abc0 crc :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:32.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.754 7f95b16bb700 1 -- 172.21.2.101:0/1826859228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95ac107060 msgr2=0x7f95ac107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:32.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.754 7f95b16bb700 1 --2- 172.21.2.101:0/1826859228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95ac107060 0x7f95ac107460 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0x7f95a4009a30 tx=0x7f95a400bd20).stop 2021-02-10T11:32:32.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.754 7f95b16bb700 1 -- 172.21.2.101:0/1826859228 shutdown_connections 2021-02-10T11:32:32.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.754 7f95b16bb700 1 --2- 172.21.2.101:0/1826859228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f958c0186c0 0x7f958c01abc0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:32.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.754 7f95b16bb700 1 --2- 172.21.2.101:0/1826859228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95ac093040 0x7f95ac106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:32.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.754 7f95b16bb700 1 --2- 172.21.2.101:0/1826859228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95ac107060 0x7f95ac107460 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:32.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.754 7f95b16bb700 1 --2- 172.21.2.101:0/1826859228 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f95ac092c40 0x7f95ac0b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:32.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.754 7f95b16bb700 1 -- 172.21.2.101:0/1826859228 >> 172.21.2.101:0/1826859228 conn(0x7f95ac0c40e0 msgr2=0x7f95ac0bf620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:32.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.755 7f95b16bb700 1 -- 172.21.2.101:0/1826859228 shutdown_connections 2021-02-10T11:32:32.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:32.755 7f95b16bb700 1 -- 172.21.2.101:0/1826859228 wait complete. 2021-02-10T11:32:32.774 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:32:33.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.178 7fc8ac11f700 1 Processor -- start 2021-02-10T11:32:33.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.178 7fc8ac11f700 1 -- start start 2021-02-10T11:32:33.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.178 7fc8ac11f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc8a41b0470 0x7fc8a41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:33.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.178 7fc8ac11f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc8a40f20e0 0x7fc8a40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:33.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.178 7fc8ac11f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a40f0b50 con 0x7fc8a40ef610 2021-02-10T11:32:33.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.178 7fc8ac11f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a40f0820 con 0x7fc8a41b0470 2021-02-10T11:32:33.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.178 7fc8ac11f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a40f24e0 con 0x7fc8a40f20e0 2021-02-10T11:32:33.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.178 7fc8a9f3b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc8a41b0470 0x7fc8a41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:33.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.179 7fc8a9f3b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc8a41b0470 0x7fc8a41b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51014/0 (socket says 172.21.2.101:51014) 2021-02-10T11:32:33.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.179 7fc8a9f3b700 1 -- 172.21.2.101:0/1918162174 learned_addr learned my addr 172.21.2.101:0/1918162174 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:33.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.179 7fc8aa73c700 1 --2- 172.21.2.101:0/1918162174 >> v2:172.21.2.108:3300/0 conn(0x7fc8a40f20e0 0x7fc8a40f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:33.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.179 7fc8a9f3b700 1 -- 172.21.2.101:0/1918162174 >> v2:172.21.2.108:3300/0 conn(0x7fc8a40f20e0 msgr2=0x7fc8a40f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:33.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.179 7fc8a9f3b700 1 --2- 172.21.2.101:0/1918162174 >> v2:172.21.2.108:3300/0 conn(0x7fc8a40f20e0 0x7fc8a40f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:33.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.179 7fc8a9f3b700 1 -- 172.21.2.101:0/1918162174 >> v1:172.21.2.108:6789/0 conn(0x7fc8a40ef610 legacy=0x7fc8a40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:33.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.179 7fc8a9f3b700 1 -- 172.21.2.101:0/1918162174 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a40e52e0 con 0x7fc8a41b0470 2021-02-10T11:32:33.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.179 7fc8a9f3b700 1 --2- 172.21.2.101:0/1918162174 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8a41b0470 0x7fc8a41b0870 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7fc8a0006700 tx=0x7fc8a0009b80).ready entity=mon.0 client_cookie=24ec7cbb3030683a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:33.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.179 7fc8a8f39700 1 -- 172.21.2.101:0/1918162174 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc8a000c0c0 con 0x7fc8a41b0470 2021-02-10T11:32:33.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.179 7fc8a8f39700 1 -- 172.21.2.101:0/1918162174 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a000c440 con 0x7fc8a41b0470 2021-02-10T11:32:33.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.179 7fc8a8f39700 1 -- 172.21.2.101:0/1918162174 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc8a000c920 con 0x7fc8a41b0470 2021-02-10T11:32:33.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.179 7fc8ac11f700 1 -- 172.21.2.101:0/1918162174 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8a41b0470 msgr2=0x7fc8a41b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:33.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.179 7fc8ac11f700 1 --2- 172.21.2.101:0/1918162174 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8a41b0470 0x7fc8a41b0870 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7fc8a0006700 tx=0x7fc8a0009b80).stop 2021-02-10T11:32:33.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.180 7fc8ac11f700 1 -- 172.21.2.101:0/1918162174 shutdown_connections 2021-02-10T11:32:33.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.180 7fc8ac11f700 1 --2- 172.21.2.101:0/1918162174 >> v2:172.21.2.108:3300/0 conn(0x7fc8a40f20e0 0x7fc8a40f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:33.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.180 7fc8ac11f700 1 --2- 172.21.2.101:0/1918162174 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8a41b0470 0x7fc8a41b0870 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:33.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.180 7fc8ac11f700 1 -- 172.21.2.101:0/1918162174 >> 172.21.2.101:0/1918162174 conn(0x7fc8a40ce3e0 msgr2=0x7fc8a40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:33.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.180 7fc8ac11f700 1 -- 172.21.2.101:0/1918162174 shutdown_connections 2021-02-10T11:32:33.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.180 7fc8ac11f700 1 -- 172.21.2.101:0/1918162174 wait complete. 2021-02-10T11:32:33.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.195 7fc8ac11f700 1 Processor -- start 2021-02-10T11:32:33.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.195 7fc8ac11f700 1 -- start start 2021-02-10T11:32:33.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.195 7fc8ac11f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8a4190920 0x7fc8a40b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:33.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.195 7fc8ac11f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8a40b8200 0x7fc8a40b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:33.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.195 7fc8ac11f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8a40c1d80 0x7fc8a40c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:33.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.195 7fc8ac11f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a4190da0 con 0x7fc8a40b8200 2021-02-10T11:32:33.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.195 7fc8ac11f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a40b8c20 con 0x7fc8a40c1d80 2021-02-10T11:32:33.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.195 7fc8ac11f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a40b8e10 con 0x7fc8a4190920 2021-02-10T11:32:33.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.195 7fc8a973a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8a40b8200 0x7fc8a40b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:33.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.195 7fc8a973a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8a40b8200 0x7fc8a40b8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51016/0 (socket says 172.21.2.101:51016) 2021-02-10T11:32:33.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.195 7fc8a973a700 1 -- 172.21.2.101:0/2091570601 learned_addr learned my addr 172.21.2.101:0/2091570601 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:33.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.195 7fc8a9f3b700 1 --2- 172.21.2.101:0/2091570601 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8a4190920 0x7fc8a40b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:33.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.196 7fc8aa73c700 1 --2- 172.21.2.101:0/2091570601 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8a40c1d80 0x7fc8a40c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:33.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.196 7fc8a973a700 1 -- 172.21.2.101:0/2091570601 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8a4190920 msgr2=0x7fc8a40b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:33.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.196 7fc8a973a700 1 --2- 172.21.2.101:0/2091570601 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8a4190920 0x7fc8a40b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:33.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.196 7fc8a973a700 1 -- 172.21.2.101:0/2091570601 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8a40c1d80 msgr2=0x7fc8a40c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:33.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.196 7fc8a973a700 1 --2- 172.21.2.101:0/2091570601 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8a40c1d80 0x7fc8a40c2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:33.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.196 7fc8a973a700 1 -- 172.21.2.101:0/2091570601 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a40c27d0 con 0x7fc8a40b8200 2021-02-10T11:32:33.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.196 7fc8aa73c700 1 --2- 172.21.2.101:0/2091570601 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8a40c1d80 0x7fc8a40c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:32:33.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.196 7fc8a9f3b700 1 --2- 172.21.2.101:0/2091570601 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8a4190920 0x7fc8a40b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:32:33.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.196 7fc8a973a700 1 --2- 172.21.2.101:0/2091570601 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8a40b8200 0x7fc8a40b8600 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 rx=0x7fc894006ed0 tx=0x7fc89400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:33.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.196 7fc89a7fc700 1 -- 172.21.2.101:0/2091570601 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc894006020 con 0x7fc8a40b8200 2021-02-10T11:32:33.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.196 7fc89a7fc700 1 -- 172.21.2.101:0/2091570601 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8940009b0 con 0x7fc8a40b8200 2021-02-10T11:32:33.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.196 7fc89a7fc700 1 -- 172.21.2.101:0/2091570601 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc8940054b0 con 0x7fc8a40b8200 2021-02-10T11:32:33.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.196 7fc8ac11f700 1 -- 172.21.2.101:0/2091570601 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a40c27d0 con 0x7fc8a40b8200 2021-02-10T11:32:33.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.197 7fc8ac11f700 1 -- 172.21.2.101:0/2091570601 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a40c27d0 con 0x7fc8a40b8200 2021-02-10T11:32:33.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.197 7fc89a7fc700 1 -- 172.21.2.101:0/2091570601 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc894002050 con 0x7fc8a40b8200 2021-02-10T11:32:33.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.197 7fc89a7fc700 1 --2- 172.21.2.101:0/2091570601 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc88c0186d0 0x7fc88c01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:33.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.197 7fc89a7fc700 1 -- 172.21.2.101:0/2091570601 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc8940327c0 con 0x7fc8a40b8200 2021-02-10T11:32:33.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.197 7fc8a9f3b700 1 --2- 172.21.2.101:0/2091570601 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc88c0186d0 0x7fc88c01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:33.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.198 7fc8a9f3b700 1 --2- 172.21.2.101:0/2091570601 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc88c0186d0 0x7fc88c01abd0 crc :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:33.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.228 7fc8ac11f700 1 -- 172.21.2.101:0/2091570601 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a4105ae0 con 0x7fc8a40b8200 2021-02-10T11:32:33.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.231 7fc89a7fc700 1 -- 172.21.2.101:0/2091570601 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc89401f500 con 0x7fc8a40b8200 2021-02-10T11:32:33.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.360 7fc8ac11f700 1 -- 172.21.2.101:0/2091570601 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7fc8a40f0b50 con 0x7fc8a40b8200 2021-02-10T11:32:33.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.360 7fc89a7fc700 1 -- 172.21.2.101:0/2091570601 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v47) v1 ==== 104+0+10 (secure 0 0 0) 0x7fc894001fe0 con 0x7fc8a40b8200 2021-02-10T11:32:33.372 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:32:33.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.362 7fc8ac11f700 1 -- 172.21.2.101:0/2091570601 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc88c0186d0 msgr2=0x7fc88c01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:33.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.362 7fc8ac11f700 1 --2- 172.21.2.101:0/2091570601 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc88c0186d0 0x7fc88c01abd0 crc :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:33.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.362 7fc8ac11f700 1 -- 172.21.2.101:0/2091570601 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8a40b8200 msgr2=0x7fc8a40b8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:33.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.362 7fc8ac11f700 1 --2- 172.21.2.101:0/2091570601 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8a40b8200 0x7fc8a40b8600 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 rx=0x7fc894006ed0 tx=0x7fc89400b960).stop 2021-02-10T11:32:33.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.362 7fc8ac11f700 1 -- 172.21.2.101:0/2091570601 shutdown_connections 2021-02-10T11:32:33.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.362 7fc8ac11f700 1 --2- 172.21.2.101:0/2091570601 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8a40c1d80 0x7fc8a40c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:33.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.362 7fc8ac11f700 1 --2- 172.21.2.101:0/2091570601 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8a40b8200 0x7fc8a40b8600 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:33.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.362 7fc8ac11f700 1 --2- 172.21.2.101:0/2091570601 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc88c0186d0 0x7fc88c01abd0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:33.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.362 7fc8ac11f700 1 --2- 172.21.2.101:0/2091570601 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8a4190920 0x7fc8a40b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:33.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.362 7fc8ac11f700 1 -- 172.21.2.101:0/2091570601 >> 172.21.2.101:0/2091570601 conn(0x7fc8a4094e10 msgr2=0x7fc8a40a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:33.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.363 7fc8ac11f700 1 -- 172.21.2.101:0/2091570601 shutdown_connections 2021-02-10T11:32:33.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.363 7fc8ac11f700 1 -- 172.21.2.101:0/2091570601 wait complete. 2021-02-10T11:32:33.382 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:32:33.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.783 7fb5d951e700 1 Processor -- start 2021-02-10T11:32:33.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.783 7fb5d951e700 1 -- start start 2021-02-10T11:32:33.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.784 7fb5d951e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5d41b0870 0x7fb5d40f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:33.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.784 7fb5d951e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d41b0c70 con 0x7fb5d40efe00 2021-02-10T11:32:33.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.784 7fb5d951e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d40f0b50 con 0x7fb5d41b0470 2021-02-10T11:32:33.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.784 7fb5d951e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d40e5ea0 con 0x7fb5d41b0870 2021-02-10T11:32:33.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.784 7fb5d3ac0700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb5d40efe00 0x7fb5d40f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51518/0 (socket says 172.21.2.101:51518) 2021-02-10T11:32:33.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.784 7fb5d3ac0700 1 -- 172.21.2.101:0/3657165262 learned_addr learned my addr 172.21.2.101:0/3657165262 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:33.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.784 7fb5d22bd700 1 -- 172.21.2.101:0/3657165262 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3427551702 0 0) 0x7fb5cc000f90 con 0x7fb5d40efe00 2021-02-10T11:32:33.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.784 7fb5d22bd700 1 -- 172.21.2.101:0/3657165262 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5c0002fe0 con 0x7fb5d40efe00 2021-02-10T11:32:33.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.784 7fb5d2abe700 1 --2- 172.21.2.101:0/3657165262 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5d41b0870 0x7fb5d40f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.784 7fb5d22bd700 1 -- 172.21.2.101:0/3657165262 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2074613156 0 0) 0x7fb5cc001270 con 0x7fb5d40efe00 2021-02-10T11:32:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.784 7fb5d22bd700 1 -- 172.21.2.101:0/3657165262 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5d41b0870 msgr2=0x7fb5d40f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.784 7fb5d22bd700 1 --2- 172.21.2.101:0/3657165262 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5d41b0870 0x7fb5d40f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.784 7fb5d22bd700 1 -- 172.21.2.101:0/3657165262 >> v1:172.21.2.108:6789/0 conn(0x7fb5d41b0470 legacy=0x7fb5d40ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.784 7fb5d22bd700 1 -- 172.21.2.101:0/3657165262 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d40e52e0 con 0x7fb5d40efe00 2021-02-10T11:32:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.784 7fb5d22bd700 1 -- 172.21.2.101:0/3657165262 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fb5cc001a00 con 0x7fb5d40efe00 2021-02-10T11:32:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d22bd700 1 -- 172.21.2.101:0/3657165262 >> v1:172.21.2.101:6789/0 conn(0x7fb5d40efe00 legacy=0x7fb5d40f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d22bd700 1 --2- 172.21.2.101:0/3657165262 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5c0003aa0 0x7fb5c0005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d22bd700 1 --2- 172.21.2.101:0/3657165262 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb5c0006790 0x7fb5c0008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d22bd700 1 --2- 172.21.2.101:0/3657165262 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5c0009590 0x7fb5c000ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d22bd700 1 -- 172.21.2.101:0/3657165262 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c000c2d0 con 0x7fb5c0003aa0 2021-02-10T11:32:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d22bd700 1 -- 172.21.2.101:0/3657165262 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c000c4f0 con 0x7fb5c0006790 2021-02-10T11:32:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d22bd700 1 -- 172.21.2.101:0/3657165262 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c0050720 con 0x7fb5c0009590 2021-02-10T11:32:33.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d32bf700 1 --2- 172.21.2.101:0/3657165262 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5c0003aa0 0x7fb5c0005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:33.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d2abe700 1 --2- 172.21.2.101:0/3657165262 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb5c0006790 0x7fb5c0008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:33.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d3ac0700 1 --2- 172.21.2.101:0/3657165262 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5c0009590 0x7fb5c000ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:33.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d32bf700 1 -- 172.21.2.101:0/3657165262 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5c0009590 msgr2=0x7fb5c000ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:33.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d32bf700 1 --2- 172.21.2.101:0/3657165262 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5c0009590 0x7fb5c000ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:33.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d32bf700 1 -- 172.21.2.101:0/3657165262 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb5c0006790 msgr2=0x7fb5c0008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:33.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d32bf700 1 --2- 172.21.2.101:0/3657165262 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb5c0006790 0x7fb5c0008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:33.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d32bf700 1 -- 172.21.2.101:0/3657165262 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c0050980 con 0x7fb5c0003aa0 2021-02-10T11:32:33.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d3ac0700 1 --2- 172.21.2.101:0/3657165262 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5c0009590 0x7fb5c000ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:32:33.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d32bf700 1 --2- 172.21.2.101:0/3657165262 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5c0003aa0 0x7fb5c0005ef0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7fb5bc006ac0 tx=0x7fb5bc009af0).ready entity=mon.0 client_cookie=a075b04c9b3ecd65 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:33.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d22bd700 1 -- 172.21.2.101:0/3657165262 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb5bc007280 con 0x7fb5c0003aa0 2021-02-10T11:32:33.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d22bd700 1 -- 172.21.2.101:0/3657165262 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc007540 con 0x7fb5c0003aa0 2021-02-10T11:32:33.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d22bd700 1 -- 172.21.2.101:0/3657165262 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb5bc007a20 con 0x7fb5c0003aa0 2021-02-10T11:32:33.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d951e700 1 -- 172.21.2.101:0/3657165262 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5c0003aa0 msgr2=0x7fb5c0005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:33.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d951e700 1 --2- 172.21.2.101:0/3657165262 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5c0003aa0 0x7fb5c0005ef0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7fb5bc006ac0 tx=0x7fb5bc009af0).stop 2021-02-10T11:32:33.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.785 7fb5d3ac0700 1 -- 172.21.2.101:0/3657165262 reap_dead start 2021-02-10T11:32:33.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.786 7fb5d951e700 1 -- 172.21.2.101:0/3657165262 shutdown_connections 2021-02-10T11:32:33.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.786 7fb5d951e700 1 -- 172.21.2.101:0/3657165262 >> 172.21.2.101:0/3657165262 conn(0x7fb5d40ce3e0 msgr2=0x7fb5d40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:33.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.786 7fb5d951e700 1 -- 172.21.2.101:0/3657165262 shutdown_connections 2021-02-10T11:32:33.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.786 7fb5d951e700 1 -- 172.21.2.101:0/3657165262 wait complete. 2021-02-10T11:32:33.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.801 7fb5d951e700 1 Processor -- start 2021-02-10T11:32:33.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.801 7fb5d951e700 1 -- start start 2021-02-10T11:32:33.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.801 7fb5d951e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5d409e560 0x7fb5d40a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:33.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.801 7fb5d951e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5d409e960 0x7fb5d40a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:33.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.801 7fb5d951e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb5d40a0d70 0x7fb5d40a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:33.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.801 7fb5d951e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d40c38f0 con 0x7fb5d409e560 2021-02-10T11:32:33.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.801 7fb5d951e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d40a1170 con 0x7fb5d40a0d70 2021-02-10T11:32:33.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.801 7fb5d951e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d40a1390 con 0x7fb5d409e960 2021-02-10T11:32:33.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.801 7fb5d32bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5d409e560 0x7fb5d40a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:33.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.801 7fb5d32bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5d409e560 0x7fb5d40a0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51034/0 (socket says 172.21.2.101:51034) 2021-02-10T11:32:33.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.801 7fb5d32bf700 1 -- 172.21.2.101:0/1723000368 learned_addr learned my addr 172.21.2.101:0/1723000368 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:33.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.802 7fb5d3ac0700 1 --2- 172.21.2.101:0/1723000368 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb5d40a0d70 0x7fb5d40a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:33.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.802 7fb5d2abe700 1 --2- 172.21.2.101:0/1723000368 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5d409e960 0x7fb5d40a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:33.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.802 7fb5d32bf700 1 -- 172.21.2.101:0/1723000368 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5d409e960 msgr2=0x7fb5d40a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:33.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.802 7fb5d32bf700 1 --2- 172.21.2.101:0/1723000368 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5d409e960 0x7fb5d40a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:33.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.802 7fb5d32bf700 1 -- 172.21.2.101:0/1723000368 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb5d40a0d70 msgr2=0x7fb5d40a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:33.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.802 7fb5d32bf700 1 --2- 172.21.2.101:0/1723000368 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb5d40a0d70 0x7fb5d40a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:33.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.802 7fb5d32bf700 1 -- 172.21.2.101:0/1723000368 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d40a2110 con 0x7fb5d409e560 2021-02-10T11:32:33.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.802 7fb5d3ac0700 1 --2- 172.21.2.101:0/1723000368 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb5d40a0d70 0x7fb5d40a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:32:33.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.802 7fb5d2abe700 1 --2- 172.21.2.101:0/1723000368 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5d409e960 0x7fb5d40a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:32:33.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.802 7fb5d32bf700 1 --2- 172.21.2.101:0/1723000368 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5d409e560 0x7fb5d40a0130 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7fb5bc0069a0 tx=0x7fb5bc00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:33.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.802 7fb5cbfff700 1 -- 172.21.2.101:0/1723000368 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb5bc00bd80 con 0x7fb5d409e560 2021-02-10T11:32:33.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.802 7fb5cbfff700 1 -- 172.21.2.101:0/1723000368 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc002020 con 0x7fb5d409e560 2021-02-10T11:32:33.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.802 7fb5cbfff700 1 -- 172.21.2.101:0/1723000368 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb5bc002500 con 0x7fb5d409e560 2021-02-10T11:32:33.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.802 7fb5d951e700 1 -- 172.21.2.101:0/1723000368 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d40a2110 con 0x7fb5d409e560 2021-02-10T11:32:33.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.802 7fb5d951e700 1 -- 172.21.2.101:0/1723000368 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d40a2110 con 0x7fb5d409e560 2021-02-10T11:32:33.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.803 7fb5cbfff700 1 -- 172.21.2.101:0/1723000368 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb5bc002200 con 0x7fb5d409e560 2021-02-10T11:32:33.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.803 7fb5cbfff700 1 --2- 172.21.2.101:0/1723000368 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5b80186d0 0x7fb5b801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:33.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.803 7fb5cbfff700 1 -- 172.21.2.101:0/1723000368 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb5bc0326a0 con 0x7fb5d409e560 2021-02-10T11:32:33.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.803 7fb5d2abe700 1 --2- 172.21.2.101:0/1723000368 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5b80186d0 0x7fb5b801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:33.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.803 7fb5d2abe700 1 --2- 172.21.2.101:0/1723000368 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5b80186d0 0x7fb5b801abd0 crc :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:33.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.834 7fb5d951e700 1 -- 172.21.2.101:0/1723000368 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d4105b30 con 0x7fb5d409e560 2021-02-10T11:32:33.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.836 7fb5cbfff700 1 -- 172.21.2.101:0/1723000368 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb5bc002230 con 0x7fb5d409e560 2021-02-10T11:32:33.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.967 7fb5d951e700 1 -- 172.21.2.101:0/1723000368 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7fb5d40f0b50 con 0x7fb5d409e560 2021-02-10T11:32:33.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.967 7fb5cbfff700 1 -- 172.21.2.101:0/1723000368 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v47) v1 ==== 100+0+10 (secure 0 0 0) 0x7fb5bc002020 con 0x7fb5d409e560 2021-02-10T11:32:33.978 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:32:33.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.969 7fb5d951e700 1 -- 172.21.2.101:0/1723000368 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5b80186d0 msgr2=0x7fb5b801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:33.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.969 7fb5d951e700 1 --2- 172.21.2.101:0/1723000368 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5b80186d0 0x7fb5b801abd0 crc :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:33.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.969 7fb5d951e700 1 -- 172.21.2.101:0/1723000368 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5d409e560 msgr2=0x7fb5d40a0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:33.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.969 7fb5d951e700 1 --2- 172.21.2.101:0/1723000368 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5d409e560 0x7fb5d40a0130 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7fb5bc0069a0 tx=0x7fb5bc00b990).stop 2021-02-10T11:32:33.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.969 7fb5d951e700 1 -- 172.21.2.101:0/1723000368 shutdown_connections 2021-02-10T11:32:33.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.969 7fb5d951e700 1 --2- 172.21.2.101:0/1723000368 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5b80186d0 0x7fb5b801abd0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:33.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.969 7fb5d951e700 1 --2- 172.21.2.101:0/1723000368 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5d409e960 0x7fb5d40a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:33.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.969 7fb5d951e700 1 --2- 172.21.2.101:0/1723000368 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb5d40a0d70 0x7fb5d40a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:33.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.969 7fb5d951e700 1 --2- 172.21.2.101:0/1723000368 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5d409e560 0x7fb5d40a0130 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:33.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.969 7fb5d951e700 1 -- 172.21.2.101:0/1723000368 >> 172.21.2.101:0/1723000368 conn(0x7fb5d409fc10 msgr2=0x7fb5d4094ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:33.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.970 7fb5d951e700 1 -- 172.21.2.101:0/1723000368 shutdown_connections 2021-02-10T11:32:33.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:33.970 7fb5d951e700 1 -- 172.21.2.101:0/1723000368 wait complete. 2021-02-10T11:32:33.989 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:32:33.989 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:32:34.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.387 7f3d15c69700 1 Processor -- start 2021-02-10T11:32:34.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.387 7f3d15c69700 1 -- start start 2021-02-10T11:32:34.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.388 7f3d15c69700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d100e5470 0x7f3d1007e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:34.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.388 7f3d15c69700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d1007ed80 con 0x7f3d1007f300 2021-02-10T11:32:34.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.388 7f3d15c69700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d101b4010 con 0x7f3d100e5070 2021-02-10T11:32:34.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.388 7f3d15c69700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d100d1790 con 0x7f3d100e5470 2021-02-10T11:32:34.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.388 7f3d0ffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f3d1007f300 0x7f3d1007f700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51536/0 (socket says 172.21.2.101:51536) 2021-02-10T11:32:34.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.388 7f3d0ffff700 1 -- 172.21.2.101:0/3849884185 learned_addr learned my addr 172.21.2.101:0/3849884185 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:34.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.389 7f3d0e7fc700 1 -- 172.21.2.101:0/3849884185 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1208750059 0 0) 0x7f3d08001500 con 0x7f3d1007f300 2021-02-10T11:32:34.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.389 7f3d0e7fc700 1 -- 172.21.2.101:0/3849884185 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cf8002fe0 con 0x7f3d1007f300 2021-02-10T11:32:34.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.389 7f3d0effd700 1 --2- 172.21.2.101:0/3849884185 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d100e5470 0x7f3d1007e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:34.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.389 7f3d0e7fc700 1 -- 172.21.2.101:0/3849884185 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 685447563 0 0) 0x7f3d08001460 con 0x7f3d1007f300 2021-02-10T11:32:34.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.389 7f3d0e7fc700 1 -- 172.21.2.101:0/3849884185 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d100e5470 msgr2=0x7f3d1007e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:34.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.389 7f3d0e7fc700 1 --2- 172.21.2.101:0/3849884185 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d100e5470 0x7f3d1007e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:34.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.389 7f3d0e7fc700 1 -- 172.21.2.101:0/3849884185 >> v1:172.21.2.108:6789/0 conn(0x7f3d100e5070 legacy=0x7f3d101b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:34.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.389 7f3d0e7fc700 1 -- 172.21.2.101:0/3849884185 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d100d1980 con 0x7f3d1007f300 2021-02-10T11:32:34.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.389 7f3d0e7fc700 1 -- 172.21.2.101:0/3849884185 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f3d08001980 con 0x7f3d1007f300 2021-02-10T11:32:34.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.389 7f3d0effd700 1 --2- 172.21.2.101:0/3849884185 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d100e5470 0x7f3d1007e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:32:34.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.389 7f3d0e7fc700 1 -- 172.21.2.101:0/3849884185 >> v1:172.21.2.101:6789/0 conn(0x7f3d1007f300 legacy=0x7f3d1007f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:34.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.389 7f3d0e7fc700 1 --2- 172.21.2.101:0/3849884185 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3cf8003a60 0x7f3cf8005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:34.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.389 7f3d0e7fc700 1 --2- 172.21.2.101:0/3849884185 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3cf8006790 0x7f3cf8008c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:34.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.389 7f3d0e7fc700 1 --2- 172.21.2.101:0/3849884185 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3cf8009500 0x7f3cf800ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:34.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.389 7f3d0e7fc700 1 -- 172.21.2.101:0/3849884185 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf800c2d0 con 0x7f3cf8009500 2021-02-10T11:32:34.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.389 7f3d0e7fc700 1 -- 172.21.2.101:0/3849884185 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf800c4f0 con 0x7f3cf8003a60 2021-02-10T11:32:34.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.389 7f3d0e7fc700 1 -- 172.21.2.101:0/3849884185 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf80506b0 con 0x7f3cf8006790 2021-02-10T11:32:34.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.389 7f3d0ffff700 1 --2- 172.21.2.101:0/3849884185 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3cf8009500 0x7f3cf800ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:34.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.389 7f3d0effd700 1 --2- 172.21.2.101:0/3849884185 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3cf8006790 0x7f3cf8008c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:34.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.390 7f3d0f7fe700 1 --2- 172.21.2.101:0/3849884185 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3cf8003a60 0x7f3cf8005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:34.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.390 7f3d0ffff700 1 -- 172.21.2.101:0/3849884185 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3cf8006790 msgr2=0x7f3cf8008c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:34.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.390 7f3d0ffff700 1 --2- 172.21.2.101:0/3849884185 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3cf8006790 0x7f3cf8008c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:34.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.390 7f3d0ffff700 1 -- 172.21.2.101:0/3849884185 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3cf8003a60 msgr2=0x7f3cf8005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:34.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.390 7f3d0ffff700 1 --2- 172.21.2.101:0/3849884185 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3cf8003a60 0x7f3cf8005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:34.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.390 7f3d0ffff700 1 -- 172.21.2.101:0/3849884185 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf8050950 con 0x7f3cf8009500 2021-02-10T11:32:34.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.390 7f3d0f7fe700 1 --2- 172.21.2.101:0/3849884185 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3cf8003a60 0x7f3cf8005ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:32:34.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.390 7f3d0ffff700 1 --2- 172.21.2.101:0/3849884185 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3cf8009500 0x7f3cf800ba20 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7f3d08000ed0 tx=0x7f3d0800a310).ready entity=mon.0 client_cookie=1dca02707ba4a63a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:34.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.390 7f3d0e7fc700 1 -- 172.21.2.101:0/3849884185 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3d080070c0 con 0x7f3cf8009500 2021-02-10T11:32:34.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.390 7f3d0e7fc700 1 -- 172.21.2.101:0/3849884185 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d080073c0 con 0x7f3cf8009500 2021-02-10T11:32:34.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.390 7f3d0e7fc700 1 -- 172.21.2.101:0/3849884185 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3d0800ab90 con 0x7f3cf8009500 2021-02-10T11:32:34.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.390 7f3d15c69700 1 -- 172.21.2.101:0/3849884185 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3cf8009500 msgr2=0x7f3cf800ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:34.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.390 7f3d15c69700 1 --2- 172.21.2.101:0/3849884185 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3cf8009500 0x7f3cf800ba20 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7f3d08000ed0 tx=0x7f3d0800a310).stop 2021-02-10T11:32:34.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.390 7f3d0ffff700 1 -- 172.21.2.101:0/3849884185 reap_dead start 2021-02-10T11:32:34.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.391 7f3d15c69700 1 -- 172.21.2.101:0/3849884185 shutdown_connections 2021-02-10T11:32:34.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.391 7f3d15c69700 1 -- 172.21.2.101:0/3849884185 >> 172.21.2.101:0/3849884185 conn(0x7f3d100ce690 msgr2=0x7f3d100e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:34.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.391 7f3d15c69700 1 -- 172.21.2.101:0/3849884185 shutdown_connections 2021-02-10T11:32:34.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.391 7f3d15c69700 1 -- 172.21.2.101:0/3849884185 wait complete. 2021-02-10T11:32:34.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.406 7f3d15c69700 1 Processor -- start 2021-02-10T11:32:34.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.406 7f3d15c69700 1 -- start start 2021-02-10T11:32:34.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.406 7f3d15c69700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3d101ad6e0 0x7f3d100a3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:34.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.406 7f3d15c69700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3d1008da10 0x7f3d101ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:34.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.406 7f3d15c69700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d1008de10 0x7f3d101ae920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:34.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.406 7f3d15c69700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d101add80 con 0x7f3d1008da10 2021-02-10T11:32:34.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.406 7f3d15c69700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d101adae0 con 0x7f3d101ad6e0 2021-02-10T11:32:34.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.406 7f3d15c69700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d100a3830 con 0x7f3d1008de10 2021-02-10T11:32:34.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.406 7f3d0effd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3d1008da10 0x7f3d101ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:34.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.406 7f3d0effd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3d1008da10 0x7f3d101ae300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51054/0 (socket says 172.21.2.101:51054) 2021-02-10T11:32:34.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.406 7f3d0effd700 1 -- 172.21.2.101:0/3192998897 learned_addr learned my addr 172.21.2.101:0/3192998897 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:34.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.407 7f3d0effd700 1 -- 172.21.2.101:0/3192998897 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d1008de10 msgr2=0x7f3d101ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:34.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.407 7f3d0effd700 1 --2- 172.21.2.101:0/3192998897 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d1008de10 0x7f3d101ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:34.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.407 7f3d0effd700 1 -- 172.21.2.101:0/3192998897 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3d101ad6e0 msgr2=0x7f3d100a3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:34.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.407 7f3d0f7fe700 1 --2- 172.21.2.101:0/3192998897 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3d101ad6e0 0x7f3d100a3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:34.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.407 7f3d0effd700 1 --2- 172.21.2.101:0/3192998897 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3d101ad6e0 0x7f3d100a3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:34.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.407 7f3d0effd700 1 -- 172.21.2.101:0/3192998897 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d101aef40 con 0x7f3d1008da10 2021-02-10T11:32:34.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.407 7f3d0f7fe700 1 --2- 172.21.2.101:0/3192998897 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3d101ad6e0 0x7f3d100a3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:32:34.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.407 7f3d0effd700 1 --2- 172.21.2.101:0/3192998897 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3d1008da10 0x7f3d101ae300 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7f3d0400b590 tx=0x7f3d0400b930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:34.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.407 7f3cf7fff700 1 -- 172.21.2.101:0/3192998897 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3d04006be0 con 0x7f3d1008da10 2021-02-10T11:32:34.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.407 7f3cf7fff700 1 -- 172.21.2.101:0/3192998897 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0400bd20 con 0x7f3d1008da10 2021-02-10T11:32:34.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.407 7f3d15c69700 1 -- 172.21.2.101:0/3192998897 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d101aef40 con 0x7f3d1008da10 2021-02-10T11:32:34.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.407 7f3d15c69700 1 -- 172.21.2.101:0/3192998897 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d101af4d0 con 0x7f3d1008da10 2021-02-10T11:32:34.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.407 7f3cf7fff700 1 -- 172.21.2.101:0/3192998897 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3d040022b0 con 0x7f3d1008da10 2021-02-10T11:32:34.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.408 7f3cf7fff700 1 -- 172.21.2.101:0/3192998897 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3d04000c90 con 0x7f3d1008da10 2021-02-10T11:32:34.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.408 7f3cf7fff700 1 --2- 172.21.2.101:0/3192998897 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cf00186c0 0x7f3cf001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:34.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.408 7f3cf7fff700 1 -- 172.21.2.101:0/3192998897 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3d040325a0 con 0x7f3d1008da10 2021-02-10T11:32:34.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.408 7f3d0f7fe700 1 --2- 172.21.2.101:0/3192998897 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cf00186c0 0x7f3cf001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:34.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.408 7f3d0f7fe700 1 --2- 172.21.2.101:0/3192998897 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cf00186c0 0x7f3cf001abc0 crc :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:34.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.438 7f3d15c69700 1 -- 172.21.2.101:0/3192998897 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d1007ed80 con 0x7f3d1008da10 2021-02-10T11:32:34.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.441 7f3cf7fff700 1 -- 172.21.2.101:0/3192998897 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3d04000960 con 0x7f3d1008da10 2021-02-10T11:32:34.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.568 7f3d15c69700 1 -- 172.21.2.101:0/3192998897 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d1007ed80 con 0x7f3d1008da10 2021-02-10T11:32:34.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.569 7f3cf7fff700 1 -- 172.21.2.101:0/3192998897 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+2099 (secure 0 0 0) 0x7f3d04000c90 con 0x7f3d1008da10 2021-02-10T11:32:34.580 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:32:34.580 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7674,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1796077990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1796077990},{"type":"v1","addr":"172.21.2.101:6835","nonce":1796077990}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:32:24.604347","modified":"2021-02-10 11:32:26.612664","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5986},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5986":{"gid":5986,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/3346373345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3346373345},{"type":"v1","addr":"172.21.2.108:6833","nonce":3346373345}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-02-10T11:32:34.581 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 31 2021-02-10T11:32:34.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.570 7f3d15c69700 1 -- 172.21.2.101:0/3192998897 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cf00186c0 msgr2=0x7f3cf001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:34.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.570 7f3d15c69700 1 --2- 172.21.2.101:0/3192998897 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cf00186c0 0x7f3cf001abc0 crc :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:34.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.570 7f3d15c69700 1 -- 172.21.2.101:0/3192998897 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3d1008da10 msgr2=0x7f3d101ae300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:34.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.570 7f3d15c69700 1 --2- 172.21.2.101:0/3192998897 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3d1008da10 0x7f3d101ae300 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7f3d0400b590 tx=0x7f3d0400b930).stop 2021-02-10T11:32:34.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.571 7f3d15c69700 1 -- 172.21.2.101:0/3192998897 shutdown_connections 2021-02-10T11:32:34.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.571 7f3d15c69700 1 --2- 172.21.2.101:0/3192998897 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cf00186c0 0x7f3cf001abc0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:34.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.571 7f3d15c69700 1 --2- 172.21.2.101:0/3192998897 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d1008de10 0x7f3d101ae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:34.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.571 7f3d15c69700 1 --2- 172.21.2.101:0/3192998897 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3d1008da10 0x7f3d101ae300 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:34.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.571 7f3d15c69700 1 --2- 172.21.2.101:0/3192998897 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3d101ad6e0 0x7f3d100a3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:34.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.571 7f3d15c69700 1 -- 172.21.2.101:0/3192998897 >> 172.21.2.101:0/3192998897 conn(0x7f3d101a31b0 msgr2=0x7f3d1018c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:34.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.571 7f3d15c69700 1 -- 172.21.2.101:0/3192998897 shutdown_connections 2021-02-10T11:32:34.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:34.571 7f3d15c69700 1 -- 172.21.2.101:0/3192998897 wait complete. 2021-02-10T11:32:34.792 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:32:24.604347', 'modified': '2021-02-10 11:32:26.612664', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5986}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5986': {'gid': 5986, 'name': 'b', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/3346373345', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 3346373345}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 3346373345}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:32:34.792 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:32:34.793 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:32:35.192 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:32:35.192 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:32:35.192 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:32:35.192 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 5986, 2021-02-10T11:32:35.193 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:32:35.193 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:32:35.193 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 10.574515336999999, 2021-02-10T11:32:35.193 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 31, 2021-02-10T11:32:35.193 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 47, 2021-02-10T11:32:35.193 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 47, 2021-02-10T11:32:35.193 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 16.213478393999999 2021-02-10T11:32:35.193 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:32:35.196 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 5986, "want_state": "up:active", "state": "up:active", "rank_uptime": 10.574515336999999, "mdsmap_epoch": 31, "osdmap_epoch": 47, "osdmap_epoch_barrier": 47, "uptime": 16.213478393999999 } 2021-02-10T11:32:35.196 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:32:35.196 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-02-10T11:32:35.196 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-02-10T11:32:35.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:35.197 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:32:35.533 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:32:35.875 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:32:35.876 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:32:36.218 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:32:36.219 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:32:36.219 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:32:36.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:36.220 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:32:36.553 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-02-10T11:32:36.553 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:32:36.556 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:32:36.900 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:32:36.900 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:32:36.964 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:2021-02-10 11:32:36.953 7fd0b8b3af80 -1 init, newargv = 0x55c07b89a100 newargc=9 2021-02-10T11:32:36.964 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[114372]: starting ceph client 2021-02-10T11:32:36.970 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[114372]: starting fuse 2021-02-10T11:32:37.243 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:32:37.243 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:32:37.244 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:32:37.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:37.244 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:32:37.579 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:32:37.580 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2021-02-10T11:32:37.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:37.580 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:32:37.580 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:32:37.580 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:32:37.580 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:32:37.580 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:32:37.580 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:32:37.580 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:32:37.581 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-02-10T11:32:37.581 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:32:37.581 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:32:37.581 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:32:37.581 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:32:37.581 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:32:37.581 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:32:37.581 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:32:37.581 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:32:37.581 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:32:37.581 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:32:37.581 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:32:37.581 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:32:37.582 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:32:37.582 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.0")) 2021-02-10T11:32:37.582 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:32:37.942 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.0.114372.asok 2021-02-10T11:32:37.945 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.114372.asok 2021-02-10T11:32:37.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:37.946 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.114372.asok status 2021-02-10T11:32:38.335 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:32:38.335 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:32:38.336 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:32:38.336 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:32:38.336 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "0", 2021-02-10T11:32:38.336 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:32:38.336 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-02-10T11:32:38.336 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "114372", 2021-02-10T11:32:38.336 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:32:38.336 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:38.336 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:32:38.336 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:32:38.336 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 7836, 2021-02-10T11:32:38.336 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:32:38.337 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:32:38.337 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:32:38.337 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 7836 2021-02-10T11:32:38.337 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:38.337 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:32:38.337 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:38.337 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:32:38.337 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 2626556571 2021-02-10T11:32:38.337 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:38.337 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:38.337 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:32:38.337 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:38.337 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:32:38.337 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 2626556571 2021-02-10T11:32:38.338 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:38.338 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.7836 172.21.2.101:0/2626556571", 2021-02-10T11:32:38.338 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/2626556571", 2021-02-10T11:32:38.338 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:32:38.338 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 31, 2021-02-10T11:32:38.338 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 47, 2021-02-10T11:32:38.338 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:32:38.338 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:32:38.338 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:32:38.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:38.340 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:32:38.340 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:32:38.674 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:32:38.674 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:32:38.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:38.675 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:32:39.022 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:32:39.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.422 7f6cb35a5700 1 Processor -- start 2021-02-10T11:32:39.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.422 7f6cb35a5700 1 -- start start 2021-02-10T11:32:39.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.422 7f6cb35a5700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6cac07e6a0 0x7f6cac0d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:39.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.422 7f6cb35a5700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cac07ed80 con 0x7f6cac0e5070 2021-02-10T11:32:39.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.422 7f6cb35a5700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cac07eaa0 con 0x7f6cac0e5470 2021-02-10T11:32:39.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.422 7f6cb35a5700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cac1b4080 con 0x7f6cac07e6a0 2021-02-10T11:32:39.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.423 7f6cb13c1700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f6cac0e5070 0x7f6cac1b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51568/0 (socket says 172.21.2.101:51568) 2021-02-10T11:32:39.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.423 7f6cb13c1700 1 -- 172.21.2.101:0/3558602448 learned_addr learned my addr 172.21.2.101:0/3558602448 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:39.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.423 7f6c9ffff700 1 -- 172.21.2.101:0/3558602448 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 746821353 0 0) 0x7f6ca00010b0 con 0x7f6cac0e5070 2021-02-10T11:32:39.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.423 7f6c9ffff700 1 -- 172.21.2.101:0/3558602448 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c98002fe0 con 0x7f6cac0e5070 2021-02-10T11:32:39.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.423 7f6cb1bc2700 1 --2- 172.21.2.101:0/3558602448 >> v2:172.21.2.108:3300/0 conn(0x7f6cac07e6a0 0x7f6cac0d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:39.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.423 7f6c9ffff700 1 -- 172.21.2.101:0/3558602448 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3779763353 0 0) 0x7f6ca00010b0 con 0x7f6cac0e5070 2021-02-10T11:32:39.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.423 7f6c9ffff700 1 -- 172.21.2.101:0/3558602448 >> v2:172.21.2.108:3300/0 conn(0x7f6cac07e6a0 msgr2=0x7f6cac0d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:39.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.423 7f6c9ffff700 1 --2- 172.21.2.101:0/3558602448 >> v2:172.21.2.108:3300/0 conn(0x7f6cac07e6a0 0x7f6cac0d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:39.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.423 7f6c9ffff700 1 -- 172.21.2.101:0/3558602448 >> v1:172.21.2.108:6789/0 conn(0x7f6cac0e5470 legacy=0x7f6cac07f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:39.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.423 7f6c9ffff700 1 -- 172.21.2.101:0/3558602448 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac0d1db0 con 0x7f6cac0e5070 2021-02-10T11:32:39.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.423 7f6c9ffff700 1 -- 172.21.2.101:0/3558602448 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f6ca00015b0 con 0x7f6cac0e5070 2021-02-10T11:32:39.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.423 7f6c9ffff700 1 -- 172.21.2.101:0/3558602448 >> v1:172.21.2.101:6789/0 conn(0x7f6cac0e5070 legacy=0x7f6cac1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:39.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.424 7f6c9ffff700 1 --2- 172.21.2.101:0/3558602448 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c98003aa0 0x7f6c98005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:39.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.424 7f6c9ffff700 1 --2- 172.21.2.101:0/3558602448 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c98006790 0x7f6c98008cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:39.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.424 7f6c9ffff700 1 --2- 172.21.2.101:0/3558602448 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c98009590 0x7f6c9800bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:39.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.424 7f6c9ffff700 1 -- 172.21.2.101:0/3558602448 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9800c300 con 0x7f6c98006790 2021-02-10T11:32:39.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.424 7f6c9ffff700 1 -- 172.21.2.101:0/3558602448 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9800c520 con 0x7f6c98003aa0 2021-02-10T11:32:39.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.424 7f6c9ffff700 1 -- 172.21.2.101:0/3558602448 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c98050750 con 0x7f6c98009590 2021-02-10T11:32:39.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.424 7f6cb0bc0700 1 --2- 172.21.2.101:0/3558602448 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c98006790 0x7f6c98008cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:39.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.424 7f6cb13c1700 1 --2- 172.21.2.101:0/3558602448 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c98003aa0 0x7f6c98005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:39.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.424 7f6cb1bc2700 1 --2- 172.21.2.101:0/3558602448 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c98009590 0x7f6c9800bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:39.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.424 7f6cb0bc0700 1 -- 172.21.2.101:0/3558602448 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c98009590 msgr2=0x7f6c9800bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:39.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.424 7f6cb0bc0700 1 --2- 172.21.2.101:0/3558602448 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c98009590 0x7f6c9800bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:39.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.424 7f6cb0bc0700 1 -- 172.21.2.101:0/3558602448 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c98003aa0 msgr2=0x7f6c98005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:39.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.424 7f6cb0bc0700 1 --2- 172.21.2.101:0/3558602448 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c98003aa0 0x7f6c98005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:39.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.424 7f6cb0bc0700 1 -- 172.21.2.101:0/3558602448 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c98050980 con 0x7f6c98006790 2021-02-10T11:32:39.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.424 7f6cb1bc2700 1 -- 172.21.2.101:0/3558602448 reap_dead start 2021-02-10T11:32:39.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.424 7f6cb0bc0700 1 --2- 172.21.2.101:0/3558602448 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c98006790 0x7f6c98008cb0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0x7f6ca8006ac0 tx=0x7f6ca8009af0).ready entity=mon.0 client_cookie=72b7858a1263f5ce server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:39.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.424 7f6c9ffff700 1 -- 172.21.2.101:0/3558602448 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ca8007280 con 0x7f6c98006790 2021-02-10T11:32:39.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.424 7f6c9ffff700 1 -- 172.21.2.101:0/3558602448 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca8007600 con 0x7f6c98006790 2021-02-10T11:32:39.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.424 7f6c9ffff700 1 -- 172.21.2.101:0/3558602448 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ca8007ae0 con 0x7f6c98006790 2021-02-10T11:32:39.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.424 7f6cb35a5700 1 -- 172.21.2.101:0/3558602448 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c98006790 msgr2=0x7f6c98008cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:39.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.424 7f6cb35a5700 1 --2- 172.21.2.101:0/3558602448 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c98006790 0x7f6c98008cb0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0x7f6ca8006ac0 tx=0x7f6ca8009af0).stop 2021-02-10T11:32:39.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.425 7f6cb35a5700 1 -- 172.21.2.101:0/3558602448 shutdown_connections 2021-02-10T11:32:39.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.425 7f6cb35a5700 1 --2- 172.21.2.101:0/3558602448 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c98006790 0x7f6c98008cb0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:39.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.425 7f6cb35a5700 1 -- 172.21.2.101:0/3558602448 >> 172.21.2.101:0/3558602448 conn(0x7f6cac0ce690 msgr2=0x7f6cac0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:39.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.425 7f6cb35a5700 1 -- 172.21.2.101:0/3558602448 shutdown_connections 2021-02-10T11:32:39.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.425 7f6cb35a5700 1 -- 172.21.2.101:0/3558602448 wait complete. 2021-02-10T11:32:39.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.440 7f6cb35a5700 1 Processor -- start 2021-02-10T11:32:39.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.440 7f6cb35a5700 1 -- start start 2021-02-10T11:32:39.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.440 7f6cb35a5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6cac1ad6e0 0x7f6cac0a3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:39.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.440 7f6cb35a5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cac08da10 0x7f6cac1ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:39.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.440 7f6cb35a5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6cac08de10 0x7f6cac1ae920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:39.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.440 7f6cb35a5700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac1add80 con 0x7f6cac08da10 2021-02-10T11:32:39.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.440 7f6cb35a5700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac1adae0 con 0x7f6cac1ad6e0 2021-02-10T11:32:39.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.440 7f6cb35a5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac0a3830 con 0x7f6cac08de10 2021-02-10T11:32:39.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.440 7f6cb0bc0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cac08da10 0x7f6cac1ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:39.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.440 7f6cb0bc0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cac08da10 0x7f6cac1ae300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51084/0 (socket says 172.21.2.101:51084) 2021-02-10T11:32:39.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.440 7f6cb0bc0700 1 -- 172.21.2.101:0/1881057659 learned_addr learned my addr 172.21.2.101:0/1881057659 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:39.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.441 7f6cb0bc0700 1 -- 172.21.2.101:0/1881057659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6cac08de10 msgr2=0x7f6cac1ae920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:39.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.441 7f6cb0bc0700 1 --2- 172.21.2.101:0/1881057659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6cac08de10 0x7f6cac1ae920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:39.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.441 7f6cb0bc0700 1 -- 172.21.2.101:0/1881057659 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6cac1ad6e0 msgr2=0x7f6cac0a3210 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:39.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.441 7f6cb0bc0700 1 --2- 172.21.2.101:0/1881057659 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6cac1ad6e0 0x7f6cac0a3210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:39.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.441 7f6cb0bc0700 1 -- 172.21.2.101:0/1881057659 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac1aef40 con 0x7f6cac08da10 2021-02-10T11:32:39.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.441 7f6cb0bc0700 1 --2- 172.21.2.101:0/1881057659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cac08da10 0x7f6cac1ae300 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f6ca800b6b0 tx=0x7f6ca800d910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:39.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.441 7f6c9dffb700 1 -- 172.21.2.101:0/1881057659 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ca800b0d0 con 0x7f6cac08da10 2021-02-10T11:32:39.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.441 7f6c9dffb700 1 -- 172.21.2.101:0/1881057659 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca800b350 con 0x7f6cac08da10 2021-02-10T11:32:39.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.441 7f6c9dffb700 1 -- 172.21.2.101:0/1881057659 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ca8009890 con 0x7f6cac08da10 2021-02-10T11:32:39.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.441 7f6cb35a5700 1 -- 172.21.2.101:0/1881057659 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cac1aef40 con 0x7f6cac08da10 2021-02-10T11:32:39.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.441 7f6cb35a5700 1 -- 172.21.2.101:0/1881057659 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cac1aef40 con 0x7f6cac08da10 2021-02-10T11:32:39.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.442 7f6c9dffb700 1 -- 172.21.2.101:0/1881057659 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6ca80095f0 con 0x7f6cac08da10 2021-02-10T11:32:39.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.442 7f6c9dffb700 1 --2- 172.21.2.101:0/1881057659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c8c0186c0 0x7f6c8c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:39.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.442 7f6c9dffb700 1 -- 172.21.2.101:0/1881057659 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6ca8032550 con 0x7f6cac08da10 2021-02-10T11:32:39.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.442 7f6cb13c1700 1 --2- 172.21.2.101:0/1881057659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c8c0186c0 0x7f6c8c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:39.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.442 7f6cb13c1700 1 --2- 172.21.2.101:0/1881057659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c8c0186c0 0x7f6c8c01abc0 crc :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:39.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.472 7f6cb35a5700 1 -- 172.21.2.101:0/1881057659 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cac1adae0 con 0x7f6cac08da10 2021-02-10T11:32:39.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.475 7f6c9dffb700 1 -- 172.21.2.101:0/1881057659 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6ca80091a0 con 0x7f6cac08da10 2021-02-10T11:32:39.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.597 7f6cb35a5700 1 -- 172.21.2.101:0/1881057659 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6cac1adae0 con 0x7f6cac08da10 2021-02-10T11:32:39.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.598 7f6c9dffb700 1 -- 172.21.2.101:0/1881057659 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+13993 (secure 0 0 0) 0x7f6ca80095f0 con 0x7f6cac08da10 2021-02-10T11:32:39.609 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:32:39.609 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":47,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:32:25.606853","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:32:22.580828","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2021-02-10 11:32:23.597744","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:32:39.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.599 7f6cb35a5700 1 -- 172.21.2.101:0/1881057659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c8c0186c0 msgr2=0x7f6c8c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:39.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.599 7f6cb35a5700 1 --2- 172.21.2.101:0/1881057659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c8c0186c0 0x7f6c8c01abc0 crc :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:39.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.599 7f6cb35a5700 1 -- 172.21.2.101:0/1881057659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cac08da10 msgr2=0x7f6cac1ae300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:39.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.599 7f6cb35a5700 1 --2- 172.21.2.101:0/1881057659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cac08da10 0x7f6cac1ae300 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f6ca800b6b0 tx=0x7f6ca800d910).stop 2021-02-10T11:32:39.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.600 7f6cb35a5700 1 -- 172.21.2.101:0/1881057659 shutdown_connections 2021-02-10T11:32:39.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.600 7f6cb35a5700 1 --2- 172.21.2.101:0/1881057659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c8c0186c0 0x7f6c8c01abc0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:39.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.600 7f6cb35a5700 1 --2- 172.21.2.101:0/1881057659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6cac08de10 0x7f6cac1ae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:39.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.600 7f6cb35a5700 1 --2- 172.21.2.101:0/1881057659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cac08da10 0x7f6cac1ae300 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:39.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.600 7f6cb35a5700 1 --2- 172.21.2.101:0/1881057659 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6cac1ad6e0 0x7f6cac0a3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:39.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.600 7f6cb35a5700 1 -- 172.21.2.101:0/1881057659 >> 172.21.2.101:0/1881057659 conn(0x7f6cac1a31b0 msgr2=0x7f6cac18c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:39.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.600 7f6cb35a5700 1 -- 172.21.2.101:0/1881057659 shutdown_connections 2021-02-10T11:32:39.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:39.600 7f6cb35a5700 1 -- 172.21.2.101:0/1881057659 wait complete. 2021-02-10T11:32:39.619 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:32:22.580828', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:32:23.597744', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:32:39.619 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:32:40.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.021 7f8e8baa9700 1 Processor -- start 2021-02-10T11:32:40.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.021 7f8e8baa9700 1 -- start start 2021-02-10T11:32:40.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.021 7f8e8baa9700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8e841b0470 0x7f8e841b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:40.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.021 7f8e8baa9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8e840ef610 0x7f8e840efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:40.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.021 7f8e8baa9700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8e840f0030 0x7f8e840f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:40.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.021 7f8e8baa9700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e840e5ea0 con 0x7f8e840f0030 2021-02-10T11:32:40.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.021 7f8e8baa9700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e840f0b50 con 0x7f8e841b0470 2021-02-10T11:32:40.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.021 7f8e8baa9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e840e52e0 con 0x7f8e840ef610 2021-02-10T11:32:40.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.022 7f8e8a0c6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8e840f0030 0x7f8e840f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:40.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.022 7f8e8a0c6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8e840f0030 0x7f8e840f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51092/0 (socket says 172.21.2.101:51092) 2021-02-10T11:32:40.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.022 7f8e8a0c6700 1 -- 172.21.2.101:0/2643193326 learned_addr learned my addr 172.21.2.101:0/2643193326 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:40.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.022 7f8e8a0c6700 1 -- 172.21.2.101:0/2643193326 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8e840ef610 msgr2=0x7f8e840efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:40.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.022 7f8e8a0c6700 1 --2- 172.21.2.101:0/2643193326 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8e840ef610 0x7f8e840efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:40.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.022 7f8e8a0c6700 1 -- 172.21.2.101:0/2643193326 >> v2:172.21.2.108:3300/0 conn(0x7f8e841b0470 msgr2=0x7f8e841b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:40.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.022 7f8e8a0c6700 1 --2- 172.21.2.101:0/2643193326 >> v2:172.21.2.108:3300/0 conn(0x7f8e841b0470 0x7f8e841b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:40.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.022 7f8e8a0c6700 1 -- 172.21.2.101:0/2643193326 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e840f10d0 con 0x7f8e840f0030 2021-02-10T11:32:40.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.022 7f8e8a0c6700 1 --2- 172.21.2.101:0/2643193326 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e840f0030 0x7f8e840f0430 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f8e7c006f00 tx=0x7f8e7c009f10).ready entity=mon.0 client_cookie=4899e402bc2a814a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:40.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.022 7f8e888c3700 1 -- 172.21.2.101:0/2643193326 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8e7c003030 con 0x7f8e840f0030 2021-02-10T11:32:40.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.022 7f8e888c3700 1 -- 172.21.2.101:0/2643193326 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7c00b3b0 con 0x7f8e840f0030 2021-02-10T11:32:40.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.022 7f8e888c3700 1 -- 172.21.2.101:0/2643193326 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8e7c00b8d0 con 0x7f8e840f0030 2021-02-10T11:32:40.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.022 7f8e8baa9700 1 -- 172.21.2.101:0/2643193326 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e840f0030 msgr2=0x7f8e840f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:40.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.022 7f8e8baa9700 1 --2- 172.21.2.101:0/2643193326 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e840f0030 0x7f8e840f0430 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f8e7c006f00 tx=0x7f8e7c009f10).stop 2021-02-10T11:32:40.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.023 7f8e8baa9700 1 -- 172.21.2.101:0/2643193326 shutdown_connections 2021-02-10T11:32:40.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.023 7f8e8baa9700 1 --2- 172.21.2.101:0/2643193326 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8e840ef610 0x7f8e840efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:40.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.023 7f8e8baa9700 1 --2- 172.21.2.101:0/2643193326 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e840f0030 0x7f8e840f0430 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:40.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.023 7f8e8baa9700 1 --2- 172.21.2.101:0/2643193326 >> v2:172.21.2.108:3300/0 conn(0x7f8e841b0470 0x7f8e841b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:40.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.023 7f8e8baa9700 1 -- 172.21.2.101:0/2643193326 >> 172.21.2.101:0/2643193326 conn(0x7f8e840ce3e0 msgr2=0x7f8e840e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:40.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.023 7f8e8baa9700 1 -- 172.21.2.101:0/2643193326 shutdown_connections 2021-02-10T11:32:40.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.023 7f8e8baa9700 1 -- 172.21.2.101:0/2643193326 wait complete. 2021-02-10T11:32:40.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.038 7f8e8baa9700 1 Processor -- start 2021-02-10T11:32:40.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.038 7f8e8baa9700 1 -- start start 2021-02-10T11:32:40.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.038 7f8e8baa9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e84187e20 0x7f8e840b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:40.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.038 7f8e8baa9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8e840b5340 0x7f8e84188ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:40.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.038 7f8e8baa9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8e841890f0 0x7f8e841894f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:40.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.038 7f8e8baa9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e84188550 con 0x7f8e84187e20 2021-02-10T11:32:40.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.038 7f8e8baa9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e84188220 con 0x7f8e840b5340 2021-02-10T11:32:40.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.038 7f8e8baa9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e840b5740 con 0x7f8e841890f0 2021-02-10T11:32:40.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.039 7f8e898c5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e84187e20 0x7f8e840b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:40.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.039 7f8e898c5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e84187e20 0x7f8e840b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51094/0 (socket says 172.21.2.101:51094) 2021-02-10T11:32:40.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.039 7f8e898c5700 1 -- 172.21.2.101:0/343464844 learned_addr learned my addr 172.21.2.101:0/343464844 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:40.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.039 7f8e898c5700 1 -- 172.21.2.101:0/343464844 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8e841890f0 msgr2=0x7f8e841894f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:40.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.039 7f8e898c5700 1 --2- 172.21.2.101:0/343464844 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8e841890f0 0x7f8e841894f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:40.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.039 7f8e898c5700 1 -- 172.21.2.101:0/343464844 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8e840b5340 msgr2=0x7f8e84188ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:40.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.039 7f8e898c5700 1 --2- 172.21.2.101:0/343464844 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8e840b5340 0x7f8e84188ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:40.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.039 7f8e898c5700 1 -- 172.21.2.101:0/343464844 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e840c02c0 con 0x7f8e84187e20 2021-02-10T11:32:40.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.039 7f8e898c5700 1 --2- 172.21.2.101:0/343464844 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e84187e20 0x7f8e840b4d20 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 rx=0x7f8e7800b530 tx=0x7f8e7800b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:40.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.039 7f8e767fc700 1 -- 172.21.2.101:0/343464844 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8e7800bcc0 con 0x7f8e84187e20 2021-02-10T11:32:40.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.039 7f8e767fc700 1 -- 172.21.2.101:0/343464844 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e78000c90 con 0x7f8e84187e20 2021-02-10T11:32:40.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.039 7f8e8baa9700 1 -- 172.21.2.101:0/343464844 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e840c02c0 con 0x7f8e84187e20 2021-02-10T11:32:40.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.039 7f8e767fc700 1 -- 172.21.2.101:0/343464844 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8e780022a0 con 0x7f8e84187e20 2021-02-10T11:32:40.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.040 7f8e8baa9700 1 -- 172.21.2.101:0/343464844 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e840c02c0 con 0x7f8e84187e20 2021-02-10T11:32:40.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.040 7f8e767fc700 1 -- 172.21.2.101:0/343464844 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8e78005310 con 0x7f8e84187e20 2021-02-10T11:32:40.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.040 7f8e767fc700 1 --2- 172.21.2.101:0/343464844 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8e6c018700 0x7f8e6c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:40.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.040 7f8e767fc700 1 -- 172.21.2.101:0/343464844 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8e780325f0 con 0x7f8e84187e20 2021-02-10T11:32:40.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.040 7f8e890c4700 1 --2- 172.21.2.101:0/343464844 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8e6c018700 0x7f8e6c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:40.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.041 7f8e890c4700 1 --2- 172.21.2.101:0/343464844 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8e6c018700 0x7f8e6c01abc0 crc :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:40.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.071 7f8e8baa9700 1 -- 172.21.2.101:0/343464844 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e84105af0 con 0x7f8e84187e20 2021-02-10T11:32:40.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.074 7f8e767fc700 1 -- 172.21.2.101:0/343464844 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8e78006b90 con 0x7f8e84187e20 2021-02-10T11:32:40.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.201 7f8e8baa9700 1 -- 172.21.2.101:0/343464844 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f8e840f0b50 con 0x7f8e84187e20 2021-02-10T11:32:40.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.201 7f8e767fc700 1 -- 172.21.2.101:0/343464844 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v47) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8e78000c90 con 0x7f8e84187e20 2021-02-10T11:32:40.213 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:32:40.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.203 7f8e8baa9700 1 -- 172.21.2.101:0/343464844 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8e6c018700 msgr2=0x7f8e6c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:40.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.203 7f8e8baa9700 1 --2- 172.21.2.101:0/343464844 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8e6c018700 0x7f8e6c01abc0 crc :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:40.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.203 7f8e8baa9700 1 -- 172.21.2.101:0/343464844 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e84187e20 msgr2=0x7f8e840b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:40.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.203 7f8e8baa9700 1 --2- 172.21.2.101:0/343464844 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e84187e20 0x7f8e840b4d20 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 rx=0x7f8e7800b530 tx=0x7f8e7800b8d0).stop 2021-02-10T11:32:40.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.203 7f8e8baa9700 1 -- 172.21.2.101:0/343464844 shutdown_connections 2021-02-10T11:32:40.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.203 7f8e8baa9700 1 --2- 172.21.2.101:0/343464844 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8e6c018700 0x7f8e6c01abc0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:40.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.203 7f8e8baa9700 1 --2- 172.21.2.101:0/343464844 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8e841890f0 0x7f8e841894f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:40.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.203 7f8e8baa9700 1 --2- 172.21.2.101:0/343464844 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8e840b5340 0x7f8e84188ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:40.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.203 7f8e8baa9700 1 --2- 172.21.2.101:0/343464844 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e84187e20 0x7f8e840b4d20 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:40.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.203 7f8e8baa9700 1 -- 172.21.2.101:0/343464844 >> 172.21.2.101:0/343464844 conn(0x7f8e84091d80 msgr2=0x7f8e840ab8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:40.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.203 7f8e8baa9700 1 -- 172.21.2.101:0/343464844 shutdown_connections 2021-02-10T11:32:40.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.203 7f8e8baa9700 1 -- 172.21.2.101:0/343464844 wait complete. 2021-02-10T11:32:40.223 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:32:40.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.626 7fdf373e4700 1 Processor -- start 2021-02-10T11:32:40.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.626 7fdf373e4700 1 -- start start 2021-02-10T11:32:40.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.626 7fdf373e4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdf301b0870 0x7fdf300f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:40.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.626 7fdf373e4700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf301b0c70 con 0x7fdf300efe00 2021-02-10T11:32:40.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.626 7fdf373e4700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf300f0b50 con 0x7fdf301b0470 2021-02-10T11:32:40.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.626 7fdf373e4700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf300e5ea0 con 0x7fdf301b0870 2021-02-10T11:32:40.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.627 7fdf349ff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdf301b0870 0x7fdf300f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:40.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.627 7fdf35a01700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fdf300efe00 0x7fdf300f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51600/0 (socket says 172.21.2.101:51600) 2021-02-10T11:32:40.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.627 7fdf35a01700 1 -- 172.21.2.101:0/2290069142 learned_addr learned my addr 172.21.2.101:0/2290069142 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:40.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.627 7fdf349ff700 1 -- 172.21.2.101:0/2290069142 >> v1:172.21.2.108:6789/0 conn(0x7fdf301b0470 legacy=0x7fdf300ef610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:32:40.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.627 7fdf349ff700 1 -- 172.21.2.101:0/2290069142 >> v1:172.21.2.101:6789/0 conn(0x7fdf300efe00 legacy=0x7fdf300f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:40.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.627 7fdf23fff700 1 -- 172.21.2.101:0/2290069142 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299278068 0 0) 0x7fdf28001480 con 0x7fdf300efe00 2021-02-10T11:32:40.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.627 7fdf349ff700 1 -- 172.21.2.101:0/2290069142 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf300e52e0 con 0x7fdf301b0870 2021-02-10T11:32:40.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.627 7fdf349ff700 1 --2- 172.21.2.101:0/2290069142 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf301b0870 0x7fdf300f20e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0x7fdf2c006650 tx=0x7fdf2c009a00).ready entity=mon.0 client_cookie=cbb2fac17810e1de server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:40.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.628 7fdf23fff700 1 -- 172.21.2.101:0/2290069142 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdf2c0009b0 con 0x7fdf301b0870 2021-02-10T11:32:40.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.628 7fdf23fff700 1 -- 172.21.2.101:0/2290069142 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2c00c0c0 con 0x7fdf301b0870 2021-02-10T11:32:40.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.628 7fdf23fff700 1 -- 172.21.2.101:0/2290069142 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdf2c00c5a0 con 0x7fdf301b0870 2021-02-10T11:32:40.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.628 7fdf373e4700 1 -- 172.21.2.101:0/2290069142 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf301b0870 msgr2=0x7fdf300f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:40.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.628 7fdf373e4700 1 --2- 172.21.2.101:0/2290069142 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf301b0870 0x7fdf300f20e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0x7fdf2c006650 tx=0x7fdf2c009a00).stop 2021-02-10T11:32:40.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.628 7fdf373e4700 1 -- 172.21.2.101:0/2290069142 shutdown_connections 2021-02-10T11:32:40.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.628 7fdf373e4700 1 --2- 172.21.2.101:0/2290069142 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf301b0870 0x7fdf300f20e0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:40.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.628 7fdf373e4700 1 -- 172.21.2.101:0/2290069142 >> 172.21.2.101:0/2290069142 conn(0x7fdf300ce3e0 msgr2=0x7fdf300e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:40.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.629 7fdf373e4700 1 -- 172.21.2.101:0/2290069142 shutdown_connections 2021-02-10T11:32:40.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.629 7fdf373e4700 1 -- 172.21.2.101:0/2290069142 wait complete. 2021-02-10T11:32:40.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.643 7fdf373e4700 1 Processor -- start 2021-02-10T11:32:40.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.643 7fdf373e4700 1 -- start start 2021-02-10T11:32:40.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.644 7fdf373e4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf300ab620 0x7fdf300a78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:40.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.644 7fdf373e4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf300b3480 0x7fdf300a8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:40.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.644 7fdf373e4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf300a8b50 0x7fdf300a8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:40.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.644 7fdf373e4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf300a7fb0 con 0x7fdf300a8b50 2021-02-10T11:32:40.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.644 7fdf373e4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf301931a0 con 0x7fdf300ab620 2021-02-10T11:32:40.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.644 7fdf373e4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf300b3880 con 0x7fdf300b3480 2021-02-10T11:32:40.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.644 7fdf35a01700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf300a8b50 0x7fdf300a8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:40.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.644 7fdf35a01700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf300a8b50 0x7fdf300a8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51108/0 (socket says 172.21.2.101:51108) 2021-02-10T11:32:40.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.644 7fdf35a01700 1 -- 172.21.2.101:0/2912377313 learned_addr learned my addr 172.21.2.101:0/2912377313 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:40.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.644 7fdf35a01700 1 -- 172.21.2.101:0/2912377313 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf300b3480 msgr2=0x7fdf300a8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:40.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.644 7fdf35a01700 1 --2- 172.21.2.101:0/2912377313 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf300b3480 0x7fdf300a8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:40.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.644 7fdf35a01700 1 -- 172.21.2.101:0/2912377313 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf300ab620 msgr2=0x7fdf300a78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:40.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.644 7fdf35a01700 1 --2- 172.21.2.101:0/2912377313 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf300ab620 0x7fdf300a78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:40.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.644 7fdf35a01700 1 -- 172.21.2.101:0/2912377313 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf300f1ac0 con 0x7fdf300a8b50 2021-02-10T11:32:40.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.644 7fdf35a01700 1 --2- 172.21.2.101:0/2912377313 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf300a8b50 0x7fdf300a8f50 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0x7fdf28007f90 tx=0x7fdf2800ac50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:40.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.645 7fdf21ffb700 1 -- 172.21.2.101:0/2912377313 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdf28000ec0 con 0x7fdf300a8b50 2021-02-10T11:32:40.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.645 7fdf21ffb700 1 -- 172.21.2.101:0/2912377313 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf28007020 con 0x7fdf300a8b50 2021-02-10T11:32:40.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.645 7fdf21ffb700 1 -- 172.21.2.101:0/2912377313 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdf28007500 con 0x7fdf300a8b50 2021-02-10T11:32:40.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.645 7fdf373e4700 1 -- 172.21.2.101:0/2912377313 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf300f1ac0 con 0x7fdf300a8b50 2021-02-10T11:32:40.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.645 7fdf373e4700 1 -- 172.21.2.101:0/2912377313 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf300f1ac0 con 0x7fdf300a8b50 2021-02-10T11:32:40.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.645 7fdf21ffb700 1 -- 172.21.2.101:0/2912377313 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdf28007200 con 0x7fdf300a8b50 2021-02-10T11:32:40.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.645 7fdf21ffb700 1 --2- 172.21.2.101:0/2912377313 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf100186d0 0x7fdf1001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:40.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.645 7fdf21ffb700 1 -- 172.21.2.101:0/2912377313 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fdf28032b10 con 0x7fdf300a8b50 2021-02-10T11:32:40.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.645 7fdf35200700 1 --2- 172.21.2.101:0/2912377313 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf100186d0 0x7fdf1001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:40.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.646 7fdf35200700 1 --2- 172.21.2.101:0/2912377313 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf100186d0 0x7fdf1001abd0 crc :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:40.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.676 7fdf373e4700 1 -- 172.21.2.101:0/2912377313 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf300e52e0 con 0x7fdf300a8b50 2021-02-10T11:32:40.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.678 7fdf21ffb700 1 -- 172.21.2.101:0/2912377313 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdf28007d00 con 0x7fdf300a8b50 2021-02-10T11:32:40.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.807 7fdf373e4700 1 -- 172.21.2.101:0/2912377313 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7fdf300e52e0 con 0x7fdf300a8b50 2021-02-10T11:32:40.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.807 7fdf21ffb700 1 -- 172.21.2.101:0/2912377313 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v47) v1 ==== 104+0+10 (secure 0 0 0) 0x7fdf28007020 con 0x7fdf300a8b50 2021-02-10T11:32:40.819 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:32:40.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.809 7fdf373e4700 1 -- 172.21.2.101:0/2912377313 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf100186d0 msgr2=0x7fdf1001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:40.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.809 7fdf373e4700 1 --2- 172.21.2.101:0/2912377313 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf100186d0 0x7fdf1001abd0 crc :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:40.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.809 7fdf373e4700 1 -- 172.21.2.101:0/2912377313 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf300a8b50 msgr2=0x7fdf300a8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:40.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.809 7fdf373e4700 1 --2- 172.21.2.101:0/2912377313 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf300a8b50 0x7fdf300a8f50 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0x7fdf28007f90 tx=0x7fdf2800ac50).stop 2021-02-10T11:32:40.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.809 7fdf373e4700 1 -- 172.21.2.101:0/2912377313 shutdown_connections 2021-02-10T11:32:40.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.809 7fdf373e4700 1 --2- 172.21.2.101:0/2912377313 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf100186d0 0x7fdf1001abd0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:40.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.809 7fdf373e4700 1 --2- 172.21.2.101:0/2912377313 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf300b3480 0x7fdf300a8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:40.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.809 7fdf373e4700 1 --2- 172.21.2.101:0/2912377313 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf300a8b50 0x7fdf300a8f50 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:40.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.809 7fdf373e4700 1 --2- 172.21.2.101:0/2912377313 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf300ab620 0x7fdf300a78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:40.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.809 7fdf373e4700 1 -- 172.21.2.101:0/2912377313 >> 172.21.2.101:0/2912377313 conn(0x7fdf300b3be0 msgr2=0x7fdf30093840 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:40.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.810 7fdf373e4700 1 -- 172.21.2.101:0/2912377313 shutdown_connections 2021-02-10T11:32:40.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:40.810 7fdf373e4700 1 -- 172.21.2.101:0/2912377313 wait complete. 2021-02-10T11:32:40.829 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:32:41.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.233 7fc494010700 1 Processor -- start 2021-02-10T11:32:41.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.233 7fc494010700 1 -- start start 2021-02-10T11:32:41.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.234 7fc494010700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc48c0f05f0 0x7fc48c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:41.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.234 7fc494010700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc48c1b0c70 con 0x7fc48c1b0870 2021-02-10T11:32:41.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.234 7fc494010700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc48c0f0b50 con 0x7fc48c1b0470 2021-02-10T11:32:41.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.234 7fc494010700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c0e5ea0 con 0x7fc48c0f05f0 2021-02-10T11:32:41.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.234 7fc49162b700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc48c1b0870 0x7fc48c0efe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51610/0 (socket says 172.21.2.101:51610) 2021-02-10T11:32:41.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.234 7fc49162b700 1 -- 172.21.2.101:0/1094434754 learned_addr learned my addr 172.21.2.101:0/1094434754 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:41.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.234 7fc490e2a700 1 -- 172.21.2.101:0/1094434754 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 231599634 0 0) 0x7fc480001180 con 0x7fc48c1b0870 2021-02-10T11:32:41.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.234 7fc490e2a700 1 -- 172.21.2.101:0/1094434754 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc478002fe0 con 0x7fc48c1b0870 2021-02-10T11:32:41.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.234 7fc49262d700 1 --2- 172.21.2.101:0/1094434754 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc48c0f05f0 0x7fc48c0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:41.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.234 7fc490e2a700 1 -- 172.21.2.101:0/1094434754 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2241377843 0 0) 0x7fc480001180 con 0x7fc48c1b0870 2021-02-10T11:32:41.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.234 7fc490e2a700 1 -- 172.21.2.101:0/1094434754 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc48c0f05f0 msgr2=0x7fc48c0f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:41.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.234 7fc490e2a700 1 --2- 172.21.2.101:0/1094434754 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc48c0f05f0 0x7fc48c0f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:41.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.234 7fc490e2a700 1 -- 172.21.2.101:0/1094434754 >> v1:172.21.2.108:6789/0 conn(0x7fc48c1b0470 legacy=0x7fc48c0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:41.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.234 7fc490e2a700 1 -- 172.21.2.101:0/1094434754 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48c0e52e0 con 0x7fc48c1b0870 2021-02-10T11:32:41.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.234 7fc490e2a700 1 -- 172.21.2.101:0/1094434754 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fc480001680 con 0x7fc48c1b0870 2021-02-10T11:32:41.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.235 7fc490e2a700 1 -- 172.21.2.101:0/1094434754 >> v1:172.21.2.101:6789/0 conn(0x7fc48c1b0870 legacy=0x7fc48c0efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:41.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.235 7fc490e2a700 1 --2- 172.21.2.101:0/1094434754 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc478003a70 0x7fc478005ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:41.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.235 7fc490e2a700 1 --2- 172.21.2.101:0/1094434754 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc478006760 0x7fc478008c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:41.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.235 7fc490e2a700 1 --2- 172.21.2.101:0/1094434754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc478009500 0x7fc47800ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:41.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.235 7fc490e2a700 1 -- 172.21.2.101:0/1094434754 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47800c2d0 con 0x7fc478009500 2021-02-10T11:32:41.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.235 7fc490e2a700 1 -- 172.21.2.101:0/1094434754 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47800c4f0 con 0x7fc478006760 2021-02-10T11:32:41.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.235 7fc490e2a700 1 -- 172.21.2.101:0/1094434754 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47800c710 con 0x7fc478003a70 2021-02-10T11:32:41.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.235 7fc49262d700 1 --2- 172.21.2.101:0/1094434754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc478009500 0x7fc47800ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:41.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.235 7fc491e2c700 1 --2- 172.21.2.101:0/1094434754 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc478003a70 0x7fc478005ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:41.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.235 7fc49162b700 1 --2- 172.21.2.101:0/1094434754 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc478006760 0x7fc478008c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:41.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.235 7fc49262d700 1 -- 172.21.2.101:0/1094434754 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc478003a70 msgr2=0x7fc478005ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:41.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.235 7fc49262d700 1 --2- 172.21.2.101:0/1094434754 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc478003a70 0x7fc478005ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:41.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.235 7fc49262d700 1 -- 172.21.2.101:0/1094434754 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc478006760 msgr2=0x7fc478008c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:41.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.235 7fc49262d700 1 --2- 172.21.2.101:0/1094434754 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc478006760 0x7fc478008c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:41.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.235 7fc49262d700 1 -- 172.21.2.101:0/1094434754 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47800c9b0 con 0x7fc478009500 2021-02-10T11:32:41.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.235 7fc49262d700 1 --2- 172.21.2.101:0/1094434754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc478009500 0x7fc47800ba20 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7fc48404ae60 tx=0x7fc48404de00).ready entity=mon.0 client_cookie=c746bbc14c8965af server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:41.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.235 7fc490e2a700 1 -- 172.21.2.101:0/1094434754 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc48404aa30 con 0x7fc478009500 2021-02-10T11:32:41.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.235 7fc490e2a700 1 -- 172.21.2.101:0/1094434754 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc484003030 con 0x7fc478009500 2021-02-10T11:32:41.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.235 7fc490e2a700 1 -- 172.21.2.101:0/1094434754 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc48404f570 con 0x7fc478009500 2021-02-10T11:32:41.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.235 7fc494010700 1 -- 172.21.2.101:0/1094434754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc478009500 msgr2=0x7fc47800ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:41.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.235 7fc494010700 1 --2- 172.21.2.101:0/1094434754 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc478009500 0x7fc47800ba20 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7fc48404ae60 tx=0x7fc48404de00).stop 2021-02-10T11:32:41.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.235 7fc49262d700 1 -- 172.21.2.101:0/1094434754 reap_dead start 2021-02-10T11:32:41.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.236 7fc494010700 1 -- 172.21.2.101:0/1094434754 shutdown_connections 2021-02-10T11:32:41.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.236 7fc494010700 1 -- 172.21.2.101:0/1094434754 >> 172.21.2.101:0/1094434754 conn(0x7fc48c0ce3e0 msgr2=0x7fc48c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:41.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.236 7fc494010700 1 -- 172.21.2.101:0/1094434754 shutdown_connections 2021-02-10T11:32:41.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.236 7fc494010700 1 -- 172.21.2.101:0/1094434754 wait complete. 2021-02-10T11:32:41.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.251 7fc494010700 1 Processor -- start 2021-02-10T11:32:41.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.251 7fc494010700 1 -- start start 2021-02-10T11:32:41.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.251 7fc494010700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc48c0b7db0 0x7fc48c0b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:41.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.251 7fc494010700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc48c0b8880 0x7fc48c1a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:41.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.251 7fc494010700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc48c0b8c80 0x7fc48c1a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:41.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.251 7fc494010700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c0b79b0 con 0x7fc48c0b8c80 2021-02-10T11:32:41.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.251 7fc494010700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c1a8090 con 0x7fc48c0b8880 2021-02-10T11:32:41.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.251 7fc494010700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c1a82b0 con 0x7fc48c0b7db0 2021-02-10T11:32:41.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.251 7fc49262d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc48c0b8c80 0x7fc48c1a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:41.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.251 7fc49262d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc48c0b8c80 0x7fc48c1a7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51130/0 (socket says 172.21.2.101:51130) 2021-02-10T11:32:41.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.251 7fc49262d700 1 -- 172.21.2.101:0/776441416 learned_addr learned my addr 172.21.2.101:0/776441416 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:41.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.251 7fc49162b700 1 --2- 172.21.2.101:0/776441416 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc48c0b8880 0x7fc48c1a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:41.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.252 7fc491e2c700 1 --2- 172.21.2.101:0/776441416 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc48c0b7db0 0x7fc48c0b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:41.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.252 7fc49262d700 1 -- 172.21.2.101:0/776441416 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc48c0b7db0 msgr2=0x7fc48c0b8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:41.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.252 7fc49262d700 1 --2- 172.21.2.101:0/776441416 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc48c0b7db0 0x7fc48c0b8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:41.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.252 7fc49262d700 1 -- 172.21.2.101:0/776441416 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc48c0b8880 msgr2=0x7fc48c1a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:41.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.252 7fc49262d700 1 --2- 172.21.2.101:0/776441416 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc48c0b8880 0x7fc48c1a7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:41.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.252 7fc49262d700 1 -- 172.21.2.101:0/776441416 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48c190c10 con 0x7fc48c0b8c80 2021-02-10T11:32:41.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.252 7fc49262d700 1 --2- 172.21.2.101:0/776441416 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc48c0b8c80 0x7fc48c1a7a70 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0x7fc48404d880 tx=0x7fc48404fd20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:41.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.252 7fc47e7fc700 1 -- 172.21.2.101:0/776441416 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc48404b280 con 0x7fc48c0b8c80 2021-02-10T11:32:41.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.252 7fc47e7fc700 1 -- 172.21.2.101:0/776441416 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48404fe60 con 0x7fc48c0b8c80 2021-02-10T11:32:41.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.252 7fc47e7fc700 1 -- 172.21.2.101:0/776441416 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc48404b600 con 0x7fc48c0b8c80 2021-02-10T11:32:41.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.252 7fc494010700 1 -- 172.21.2.101:0/776441416 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48c190c10 con 0x7fc48c0b8c80 2021-02-10T11:32:41.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.252 7fc494010700 1 -- 172.21.2.101:0/776441416 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48c190c10 con 0x7fc48c0b8c80 2021-02-10T11:32:41.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.253 7fc47e7fc700 1 -- 172.21.2.101:0/776441416 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc48404b000 con 0x7fc48c0b8c80 2021-02-10T11:32:41.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.253 7fc47e7fc700 1 --2- 172.21.2.101:0/776441416 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc474018730 0x7fc47401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:41.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.253 7fc47e7fc700 1 -- 172.21.2.101:0/776441416 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc4840769c0 con 0x7fc48c0b8c80 2021-02-10T11:32:41.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.253 7fc491e2c700 1 --2- 172.21.2.101:0/776441416 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc474018730 0x7fc47401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:41.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.253 7fc491e2c700 1 --2- 172.21.2.101:0/776441416 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc474018730 0x7fc47401ac10 crc :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:41.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.284 7fc494010700 1 -- 172.21.2.101:0/776441416 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc48c105b30 con 0x7fc48c0b8c80 2021-02-10T11:32:41.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.286 7fc47e7fc700 1 -- 172.21.2.101:0/776441416 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc48404aa20 con 0x7fc48c0b8c80 2021-02-10T11:32:41.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.413 7fc494010700 1 -- 172.21.2.101:0/776441416 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7fc48c0f0b50 con 0x7fc48c0b8c80 2021-02-10T11:32:41.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.414 7fc47e7fc700 1 -- 172.21.2.101:0/776441416 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v47) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc48404b000 con 0x7fc48c0b8c80 2021-02-10T11:32:41.425 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:32:41.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.416 7fc494010700 1 -- 172.21.2.101:0/776441416 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc474018730 msgr2=0x7fc47401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:41.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.416 7fc494010700 1 --2- 172.21.2.101:0/776441416 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc474018730 0x7fc47401ac10 crc :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:41.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.416 7fc494010700 1 -- 172.21.2.101:0/776441416 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc48c0b8c80 msgr2=0x7fc48c1a7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:41.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.416 7fc494010700 1 --2- 172.21.2.101:0/776441416 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc48c0b8c80 0x7fc48c1a7a70 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0x7fc48404d880 tx=0x7fc48404fd20).stop 2021-02-10T11:32:41.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.416 7fc494010700 1 -- 172.21.2.101:0/776441416 shutdown_connections 2021-02-10T11:32:41.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.416 7fc494010700 1 --2- 172.21.2.101:0/776441416 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc48c0b8c80 0x7fc48c1a7a70 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:41.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.416 7fc494010700 1 --2- 172.21.2.101:0/776441416 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc48c0b8880 0x7fc48c1a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:41.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.416 7fc494010700 1 --2- 172.21.2.101:0/776441416 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc474018730 0x7fc47401ac10 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:41.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.416 7fc494010700 1 --2- 172.21.2.101:0/776441416 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc48c0b7db0 0x7fc48c0b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:41.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.416 7fc494010700 1 -- 172.21.2.101:0/776441416 >> 172.21.2.101:0/776441416 conn(0x7fc48c094e10 msgr2=0x7fc48c0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:41.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.416 7fc494010700 1 -- 172.21.2.101:0/776441416 shutdown_connections 2021-02-10T11:32:41.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.416 7fc494010700 1 -- 172.21.2.101:0/776441416 wait complete. 2021-02-10T11:32:41.436 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:32:41.436 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:32:41.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.835 7f28b564b700 1 Processor -- start 2021-02-10T11:32:41.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.835 7f28b564b700 1 -- start start 2021-02-10T11:32:41.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.836 7f28b564b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f28b00e5070 0x7f28b01b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:41.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.836 7f28b564b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28b00e5470 0x7f28b0080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:41.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.836 7f28b564b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28b007ed80 con 0x7f28b01b3e40 2021-02-10T11:32:41.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.836 7f28b564b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28b00d1790 con 0x7f28b00e5070 2021-02-10T11:32:41.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.836 7f28b564b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b00d19b0 con 0x7f28b00e5470 2021-02-10T11:32:41.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.836 7f28aeabe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f28b01b3e40 0x7f28b007d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51630/0 (socket says 172.21.2.101:51630) 2021-02-10T11:32:41.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.836 7f28aeabe700 1 -- 172.21.2.101:0/1148433029 learned_addr learned my addr 172.21.2.101:0/1148433029 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:41.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.836 7f28af2bf700 1 --2- 172.21.2.101:0/1148433029 >> v2:172.21.2.101:3300/0 conn(0x7f28b00e5070 0x7f28b01b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:41.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.836 7f28af2bf700 1 -- 172.21.2.101:0/1148433029 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28b00e5470 msgr2=0x7f28b0080310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:41.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.836 7f28af2bf700 1 --2- 172.21.2.101:0/1148433029 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28b00e5470 0x7f28b0080310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:41.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.836 7f28af2bf700 1 -- 172.21.2.101:0/1148433029 >> v1:172.21.2.101:6789/0 conn(0x7f28b01b3e40 legacy=0x7f28b007d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:41.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.836 7f28af2bf700 1 -- 172.21.2.101:0/1148433029 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b00d1bd0 con 0x7f28b00e5070 2021-02-10T11:32:41.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.836 7f28af2bf700 1 --2- 172.21.2.101:0/1148433029 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28b00e5070 0x7f28b01b3820 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f2898006650 tx=0x7f2898009a80).ready entity=mon.0 client_cookie=a581f962da3234ee server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:41.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.837 7f28ae2bd700 1 -- 172.21.2.101:0/1148433029 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f289800c0c0 con 0x7f28b00e5070 2021-02-10T11:32:41.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.837 7f28ae2bd700 1 -- 172.21.2.101:0/1148433029 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289800c340 con 0x7f28b00e5070 2021-02-10T11:32:41.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.837 7f28ae2bd700 1 -- 172.21.2.101:0/1148433029 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f289800c860 con 0x7f28b00e5070 2021-02-10T11:32:41.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.837 7f28b564b700 1 -- 172.21.2.101:0/1148433029 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28b00e5070 msgr2=0x7f28b01b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:41.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.837 7f28b564b700 1 --2- 172.21.2.101:0/1148433029 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28b00e5070 0x7f28b01b3820 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f2898006650 tx=0x7f2898009a80).stop 2021-02-10T11:32:41.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.837 7f28b564b700 1 -- 172.21.2.101:0/1148433029 shutdown_connections 2021-02-10T11:32:41.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.837 7f28b564b700 1 --2- 172.21.2.101:0/1148433029 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28b00e5470 0x7f28b0080310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:41.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.837 7f28b564b700 1 --2- 172.21.2.101:0/1148433029 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28b00e5070 0x7f28b01b3820 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:41.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.837 7f28b564b700 1 -- 172.21.2.101:0/1148433029 >> 172.21.2.101:0/1148433029 conn(0x7f28b00ce690 msgr2=0x7f28b00e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:41.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.837 7f28b564b700 1 -- 172.21.2.101:0/1148433029 shutdown_connections 2021-02-10T11:32:41.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.837 7f28b564b700 1 -- 172.21.2.101:0/1148433029 wait complete. 2021-02-10T11:32:41.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.852 7f28b564b700 1 Processor -- start 2021-02-10T11:32:41.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.852 7f28b564b700 1 -- start start 2021-02-10T11:32:41.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.852 7f28b564b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f28b01b19d0 0x7f28b008cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:41.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.853 7f28b564b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28b01b0d00 0x7f28b01b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:41.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.853 7f28b564b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28b01a2b50 0x7f28b01a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:41.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.853 7f28b564b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b01b1720 con 0x7f28b01a2b50 2021-02-10T11:32:41.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.853 7f28b564b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b008c470 con 0x7f28b01b19d0 2021-02-10T11:32:41.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.853 7f28b564b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b01b1f50 con 0x7f28b01b0d00 2021-02-10T11:32:41.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.853 7f28afac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28b01a2b50 0x7f28b01a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:41.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.853 7f28afac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28b01a2b50 0x7f28b01a2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51138/0 (socket says 172.21.2.101:51138) 2021-02-10T11:32:41.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.853 7f28afac0700 1 -- 172.21.2.101:0/3767713461 learned_addr learned my addr 172.21.2.101:0/3767713461 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:41.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.853 7f28afac0700 1 -- 172.21.2.101:0/3767713461 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28b01b0d00 msgr2=0x7f28b01b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:41.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.853 7f28afac0700 1 --2- 172.21.2.101:0/3767713461 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28b01b0d00 0x7f28b01b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:41.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.853 7f28afac0700 1 -- 172.21.2.101:0/3767713461 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f28b01b19d0 msgr2=0x7f28b008cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:41.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.853 7f28afac0700 1 --2- 172.21.2.101:0/3767713461 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f28b01b19d0 0x7f28b008cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:41.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.853 7f28afac0700 1 -- 172.21.2.101:0/3767713461 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b01a3570 con 0x7f28b01a2b50 2021-02-10T11:32:41.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.853 7f28afac0700 1 --2- 172.21.2.101:0/3767713461 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28b01a2b50 0x7f28b01a2f50 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7f28a8009980 tx=0x7f28a800bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:41.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.854 7f28a7fff700 1 -- 172.21.2.101:0/3767713461 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f28a8005870 con 0x7f28b01a2b50 2021-02-10T11:32:41.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.854 7f28a7fff700 1 -- 172.21.2.101:0/3767713461 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a8005b30 con 0x7f28b01a2b50 2021-02-10T11:32:41.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.854 7f28b564b700 1 -- 172.21.2.101:0/3767713461 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b01a3570 con 0x7f28b01a2b50 2021-02-10T11:32:41.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.854 7f28b564b700 1 -- 172.21.2.101:0/3767713461 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b00b3330 con 0x7f28b01a2b50 2021-02-10T11:32:41.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.854 7f28a7fff700 1 -- 172.21.2.101:0/3767713461 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f28a8006030 con 0x7f28b01a2b50 2021-02-10T11:32:41.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.854 7f28a7fff700 1 -- 172.21.2.101:0/3767713461 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f28a8005d10 con 0x7f28b01a2b50 2021-02-10T11:32:41.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.854 7f28a7fff700 1 --2- 172.21.2.101:0/3767713461 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2894018730 0x7f289401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:41.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.854 7f28a7fff700 1 -- 172.21.2.101:0/3767713461 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f28a8032bb0 con 0x7f28b01a2b50 2021-02-10T11:32:41.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.855 7f28af2bf700 1 --2- 172.21.2.101:0/3767713461 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2894018730 0x7f289401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:41.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.855 7f28af2bf700 1 --2- 172.21.2.101:0/3767713461 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2894018730 0x7f289401abc0 crc :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:41.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.886 7f28b564b700 1 -- 172.21.2.101:0/3767713461 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b0083dd0 con 0x7f28b01a2b50 2021-02-10T11:32:41.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:41.888 7f28a7fff700 1 -- 172.21.2.101:0/3767713461 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f28a8005d40 con 0x7f28b01a2b50 2021-02-10T11:32:42.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:42.015 7f28b564b700 1 -- 172.21.2.101:0/3767713461 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28b007ed80 con 0x7f28b01a2b50 2021-02-10T11:32:42.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:42.015 7f28a7fff700 1 -- 172.21.2.101:0/3767713461 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+2099 (secure 0 0 0) 0x7f28a8006f20 con 0x7f28b01a2b50 2021-02-10T11:32:42.027 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 31 2021-02-10T11:32:42.027 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:32:42.027 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7674,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1796077990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1796077990},{"type":"v1","addr":"172.21.2.101:6835","nonce":1796077990}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:32:24.604347","modified":"2021-02-10 11:32:26.612664","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5986},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5986":{"gid":5986,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/3346373345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3346373345},{"type":"v1","addr":"172.21.2.108:6833","nonce":3346373345}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-02-10T11:32:42.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:42.017 7f28b564b700 1 -- 172.21.2.101:0/3767713461 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2894018730 msgr2=0x7f289401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:42.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:42.017 7f28b564b700 1 --2- 172.21.2.101:0/3767713461 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2894018730 0x7f289401abc0 crc :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:42.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:42.017 7f28b564b700 1 -- 172.21.2.101:0/3767713461 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28b01a2b50 msgr2=0x7f28b01a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:42.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:42.017 7f28b564b700 1 --2- 172.21.2.101:0/3767713461 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28b01a2b50 0x7f28b01a2f50 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7f28a8009980 tx=0x7f28a800bcd0).stop 2021-02-10T11:32:42.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:42.017 7f28b564b700 1 -- 172.21.2.101:0/3767713461 shutdown_connections 2021-02-10T11:32:42.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:42.017 7f28b564b700 1 --2- 172.21.2.101:0/3767713461 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2894018730 0x7f289401abc0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:42.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:42.017 7f28b564b700 1 --2- 172.21.2.101:0/3767713461 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28b01b0d00 0x7f28b01b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:42.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:42.017 7f28b564b700 1 --2- 172.21.2.101:0/3767713461 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28b01a2b50 0x7f28b01a2f50 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:42.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:42.017 7f28b564b700 1 --2- 172.21.2.101:0/3767713461 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f28b01b19d0 0x7f28b008cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:42.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:42.017 7f28b564b700 1 -- 172.21.2.101:0/3767713461 >> 172.21.2.101:0/3767713461 conn(0x7f28b0093720 msgr2=0x7f28b0084b80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:42.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:42.018 7f28b564b700 1 -- 172.21.2.101:0/3767713461 shutdown_connections 2021-02-10T11:32:42.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:42.018 7f28b564b700 1 -- 172.21.2.101:0/3767713461 wait complete. 2021-02-10T11:32:42.239 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:32:24.604347', 'modified': '2021-02-10 11:32:26.612664', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5986}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5986': {'gid': 5986, 'name': 'b', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/3346373345', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 3346373345}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 3346373345}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:32:42.239 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:32:42.240 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:32:42.638 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:32:42.638 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:32:42.639 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:32:42.639 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 5986, 2021-02-10T11:32:42.639 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:32:42.639 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:32:42.639 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 18.020899639, 2021-02-10T11:32:42.639 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 31, 2021-02-10T11:32:42.639 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 47, 2021-02-10T11:32:42.639 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 47, 2021-02-10T11:32:42.639 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 23.659860843000001 2021-02-10T11:32:42.639 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:32:42.642 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 5986, "want_state": "up:active", "state": "up:active", "rank_uptime": 18.020899639, "mdsmap_epoch": 31, "osdmap_epoch": 47, "osdmap_epoch_barrier": 47, "uptime": 23.659860843000001 } 2021-02-10T11:32:42.642 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:32:42.643 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-02-10T11:32:42.643 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:32:42.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:42.643 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:32:42.979 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:32:43.323 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:32:43.323 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:32:43.666 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:32:43.666 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:32:43.667 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:32:43.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:43.667 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:32:44.000 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:32:44.000 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2021-02-10T11:32:44.000 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:32:44.003 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:32:44.347 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:32:44.348 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:32:44.414 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[115013]: starting ceph client 2021-02-10T11:32:44.414 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:2021-02-10 11:32:44.402 7f83c04f7f80 -1 init, newargv = 0x562a7c37db10 newargc=9 2021-02-10T11:32:44.419 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[115013]: starting fuse 2021-02-10T11:32:44.689 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:32:44.690 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:32:44.690 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:32:44.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:44.691 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:32:45.024 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:32:45.024 INFO:teuthology.orchestra.run.gibba001.stdout:44 2021-02-10T11:32:45.024 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2021-02-10T11:32:45.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:45.025 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:32:45.025 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:32:45.025 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:32:45.025 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:32:45.025 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:32:45.025 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:32:45.025 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:32:45.025 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-02-10T11:32:45.025 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:32:45.025 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:32:45.025 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:32:45.025 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:32:45.026 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:32:45.026 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:32:45.026 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:32:45.026 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:32:45.026 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:32:45.026 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:32:45.026 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:32:45.026 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:32:45.026 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:32:45.026 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.1")) 2021-02-10T11:32:45.026 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:32:45.388 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.1.115013.asok 2021-02-10T11:32:45.391 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.115013.asok 2021-02-10T11:32:45.391 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:45.391 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.115013.asok status 2021-02-10T11:32:45.779 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:32:45.779 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:32:45.779 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:32:45.779 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:32:45.780 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "1", 2021-02-10T11:32:45.780 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:32:45.780 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:32:45.780 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "115013", 2021-02-10T11:32:45.780 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:32:45.780 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:45.780 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:32:45.780 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:32:45.780 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 7881, 2021-02-10T11:32:45.780 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:32:45.780 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:32:45.780 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:32:45.780 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 7881 2021-02-10T11:32:45.781 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:45.781 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:32:45.781 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:45.781 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:32:45.781 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1569738620 2021-02-10T11:32:45.781 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:45.781 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:45.781 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:32:45.781 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:32:45.781 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:32:45.781 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1569738620 2021-02-10T11:32:45.781 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:32:45.781 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.7881 172.21.2.101:0/1569738620", 2021-02-10T11:32:45.781 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/1569738620", 2021-02-10T11:32:45.782 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:32:45.782 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 31, 2021-02-10T11:32:45.782 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 47, 2021-02-10T11:32:45.782 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:32:45.782 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:32:45.782 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:32:45.784 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:45.784 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:32:45.784 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:32:46.119 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:32:46.119 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:32:46.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:46.119 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:32:46.465 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2021-02-10T11:32:46.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.869 7f5eb67bb700 1 Processor -- start 2021-02-10T11:32:46.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.869 7f5eb67bb700 1 -- start start 2021-02-10T11:32:46.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.870 7f5eb67bb700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5eb00d1c70 0x7f5eb00cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:46.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.870 7f5eb67bb700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb00d0170 con 0x7f5eb00d1870 2021-02-10T11:32:46.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.870 7f5eb67bb700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb00fe1c0 con 0x7f5eb00cf580 2021-02-10T11:32:46.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.870 7f5eb67bb700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb01b0cb0 con 0x7f5eb00d1c70 2021-02-10T11:32:46.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.870 7f5eaf7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5eb00d1c70 0x7f5eb00cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:46.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.870 7f5eaf7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5eb00d1c70 0x7f5eb00cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51158/0 (socket says 172.21.2.101:51158) 2021-02-10T11:32:46.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.870 7f5eaf7fe700 1 -- 172.21.2.101:0/1570920001 learned_addr learned my addr 172.21.2.101:0/1570920001 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:46.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.871 7f5eaf7fe700 1 -- 172.21.2.101:0/1570920001 >> v1:172.21.2.108:6789/0 conn(0x7f5eb00cf580 legacy=0x7f5eb00cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:46.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.871 7f5eaf7fe700 1 -- 172.21.2.101:0/1570920001 >> v1:172.21.2.101:6789/0 conn(0x7f5eb00d1870 legacy=0x7f5eb01b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:46.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.871 7f5eaf7fe700 1 -- 172.21.2.101:0/1570920001 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb0109a60 con 0x7f5eb00d1c70 2021-02-10T11:32:46.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.871 7f5eaf7fe700 1 --2- 172.21.2.101:0/1570920001 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eb00d1c70 0x7f5eb00cef60 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 rx=0x7f5ea4006700 tx=0x7f5ea4009b80).ready entity=mon.0 client_cookie=5edb0c08027afa35 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:46.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.871 7f5eaeffd700 1 -- 172.21.2.101:0/1570920001 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5ea400c0c0 con 0x7f5eb00d1c70 2021-02-10T11:32:46.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.871 7f5eaeffd700 1 -- 172.21.2.101:0/1570920001 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea400c440 con 0x7f5eb00d1c70 2021-02-10T11:32:46.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.871 7f5eaeffd700 1 -- 172.21.2.101:0/1570920001 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5ea400c960 con 0x7f5eb00d1c70 2021-02-10T11:32:46.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.871 7f5eb67bb700 1 -- 172.21.2.101:0/1570920001 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eb00d1c70 msgr2=0x7f5eb00cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:46.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.871 7f5eb67bb700 1 --2- 172.21.2.101:0/1570920001 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eb00d1c70 0x7f5eb00cef60 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 rx=0x7f5ea4006700 tx=0x7f5ea4009b80).stop 2021-02-10T11:32:46.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.872 7f5eb67bb700 1 -- 172.21.2.101:0/1570920001 shutdown_connections 2021-02-10T11:32:46.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.872 7f5eb67bb700 1 --2- 172.21.2.101:0/1570920001 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eb00d1c70 0x7f5eb00cef60 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:46.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.872 7f5eb67bb700 1 -- 172.21.2.101:0/1570920001 >> 172.21.2.101:0/1570920001 conn(0x7f5eb00ce420 msgr2=0x7f5eb00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:46.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.872 7f5eb67bb700 1 -- 172.21.2.101:0/1570920001 shutdown_connections 2021-02-10T11:32:46.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.872 7f5eb67bb700 1 -- 172.21.2.101:0/1570920001 wait complete. 2021-02-10T11:32:46.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.887 7f5eb67bb700 1 Processor -- start 2021-02-10T11:32:46.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.887 7f5eb67bb700 1 -- start start 2021-02-10T11:32:46.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.887 7f5eb67bb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eb00a51e0 0x7f5eb00a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:46.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.887 7f5eb67bb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eb009e0d0 0x7f5eb009e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:46.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.887 7f5eb67bb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5eb009eaf0 0x7f5eb00a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:46.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.887 7f5eb67bb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00a5c00 con 0x7f5eb00a51e0 2021-02-10T11:32:46.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.887 7f5eb67bb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00a5e70 con 0x7f5eb009eaf0 2021-02-10T11:32:46.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.887 7f5eb67bb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb009eef0 con 0x7f5eb009e0d0 2021-02-10T11:32:46.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.887 7f5eaffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eb00a51e0 0x7f5eb00a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:46.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.887 7f5eaffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eb00a51e0 0x7f5eb00a55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51162/0 (socket says 172.21.2.101:51162) 2021-02-10T11:32:46.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.887 7f5eaffff700 1 -- 172.21.2.101:0/1594767738 learned_addr learned my addr 172.21.2.101:0/1594767738 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:46.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.888 7f5eaffff700 1 -- 172.21.2.101:0/1594767738 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eb009e0d0 msgr2=0x7f5eb009e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:46.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.888 7f5eaffff700 1 --2- 172.21.2.101:0/1594767738 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eb009e0d0 0x7f5eb009e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:46.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.888 7f5eaffff700 1 -- 172.21.2.101:0/1594767738 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5eb009eaf0 msgr2=0x7f5eb00a63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:46.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.888 7f5eaffff700 1 --2- 172.21.2.101:0/1594767738 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5eb009eaf0 0x7f5eb00a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:46.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.888 7f5eaffff700 1 -- 172.21.2.101:0/1594767738 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb009f0d0 con 0x7f5eb00a51e0 2021-02-10T11:32:46.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.888 7f5eaffff700 1 --2- 172.21.2.101:0/1594767738 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eb00a51e0 0x7f5eb00a55e0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7f5ea0006f00 tx=0x7f5ea000b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:46.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.888 7f5eacff9700 1 -- 172.21.2.101:0/1594767738 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5ea0000ef0 con 0x7f5eb00a51e0 2021-02-10T11:32:46.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.888 7f5eacff9700 1 -- 172.21.2.101:0/1594767738 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea0000c60 con 0x7f5eb00a51e0 2021-02-10T11:32:46.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.888 7f5eacff9700 1 -- 172.21.2.101:0/1594767738 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5ea0001310 con 0x7f5eb00a51e0 2021-02-10T11:32:46.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.888 7f5eb67bb700 1 -- 172.21.2.101:0/1594767738 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb009f0d0 con 0x7f5eb00a51e0 2021-02-10T11:32:46.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.888 7f5eb67bb700 1 -- 172.21.2.101:0/1594767738 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb00a6d30 con 0x7f5eb00a51e0 2021-02-10T11:32:46.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.889 7f5eacff9700 1 -- 172.21.2.101:0/1594767738 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5ea0002020 con 0x7f5eb00a51e0 2021-02-10T11:32:46.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.889 7f5eacff9700 1 --2- 172.21.2.101:0/1594767738 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5e900186c0 0x7f5e9001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:46.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.889 7f5eacff9700 1 -- 172.21.2.101:0/1594767738 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5ea00323c0 con 0x7f5eb00a51e0 2021-02-10T11:32:46.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.889 7f5eaf7fe700 1 --2- 172.21.2.101:0/1594767738 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5e900186c0 0x7f5e9001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:46.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.889 7f5eaf7fe700 1 --2- 172.21.2.101:0/1594767738 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5e900186c0 0x7f5e9001abc0 crc :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:46.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.920 7f5eb67bb700 1 -- 172.21.2.101:0/1594767738 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb00806b0 con 0x7f5eb00a51e0 2021-02-10T11:32:46.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:46.922 7f5eacff9700 1 -- 172.21.2.101:0/1594767738 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5ea0006b90 con 0x7f5eb00a51e0 2021-02-10T11:32:47.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.057 7f5eb67bb700 1 -- 172.21.2.101:0/1594767738 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5eb0088020 con 0x7f5eb00a51e0 2021-02-10T11:32:47.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.449 7f5eacff9700 1 -- 172.21.2.101:0/1594767738 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v32) v1 ==== 136+0+0 (secure 0 0 0) 0x7f5ea0002020 con 0x7f5eb00a51e0 2021-02-10T11:32:47.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.475 7f5eb67bb700 1 -- 172.21.2.101:0/1594767738 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5e900186c0 msgr2=0x7f5e9001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:47.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.475 7f5eb67bb700 1 --2- 172.21.2.101:0/1594767738 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5e900186c0 0x7f5e9001abc0 crc :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:47.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.475 7f5eb67bb700 1 -- 172.21.2.101:0/1594767738 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eb00a51e0 msgr2=0x7f5eb00a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:47.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.475 7f5eb67bb700 1 --2- 172.21.2.101:0/1594767738 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eb00a51e0 0x7f5eb00a55e0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7f5ea0006f00 tx=0x7f5ea000b940).stop 2021-02-10T11:32:47.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.476 7f5eb67bb700 1 -- 172.21.2.101:0/1594767738 shutdown_connections 2021-02-10T11:32:47.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.476 7f5eb67bb700 1 --2- 172.21.2.101:0/1594767738 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5e900186c0 0x7f5e9001abc0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:47.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.476 7f5eb67bb700 1 --2- 172.21.2.101:0/1594767738 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eb009e0d0 0x7f5eb009e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:47.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.476 7f5eb67bb700 1 --2- 172.21.2.101:0/1594767738 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5eb009eaf0 0x7f5eb00a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:47.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.476 7f5eb67bb700 1 --2- 172.21.2.101:0/1594767738 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eb00a51e0 0x7f5eb00a55e0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:47.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.476 7f5eb67bb700 1 -- 172.21.2.101:0/1594767738 >> 172.21.2.101:0/1594767738 conn(0x7f5eb00bbea0 msgr2=0x7f5eb00ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:47.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.476 7f5eb67bb700 1 -- 172.21.2.101:0/1594767738 shutdown_connections 2021-02-10T11:32:47.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.476 7f5eb67bb700 1 -- 172.21.2.101:0/1594767738 wait complete. 2021-02-10T11:32:47.495 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs volume ls 2021-02-10T11:32:47.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.893 7fa398a7b700 1 Processor -- start 2021-02-10T11:32:47.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.893 7fa398a7b700 1 -- start start 2021-02-10T11:32:47.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.894 7fa398a7b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa39407e6a0 0x7fa3940d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:47.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.894 7fa398a7b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa39407ed80 con 0x7fa3940e5070 2021-02-10T11:32:47.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.894 7fa398a7b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa39407eaa0 con 0x7fa3940e5470 2021-02-10T11:32:47.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.894 7fa398a7b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3941b4080 con 0x7fa39407e6a0 2021-02-10T11:32:47.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.894 7fa39285e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa3940e5070 0x7fa3941b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51664/0 (socket says 172.21.2.101:51664) 2021-02-10T11:32:47.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.894 7fa39285e700 1 -- 172.21.2.101:0/1470606712 learned_addr learned my addr 172.21.2.101:0/1470606712 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:47.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.894 7fa39305f700 1 --2- 172.21.2.101:0/1470606712 >> v2:172.21.2.108:3300/0 conn(0x7fa39407e6a0 0x7fa3940d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:47.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.894 7fa39205d700 1 -- 172.21.2.101:0/1470606712 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3251458686 0 0) 0x7fa384000ce0 con 0x7fa3940e5070 2021-02-10T11:32:47.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.895 7fa39205d700 1 -- 172.21.2.101:0/1470606712 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa380002fe0 con 0x7fa3940e5070 2021-02-10T11:32:47.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.895 7fa39305f700 1 -- 172.21.2.101:0/1470606712 >> v1:172.21.2.108:6789/0 conn(0x7fa3940e5470 legacy=0x7fa39407f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:47.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.895 7fa39305f700 1 -- 172.21.2.101:0/1470606712 >> v1:172.21.2.101:6789/0 conn(0x7fa3940e5070 legacy=0x7fa3941b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:47.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.895 7fa39205d700 1 -- 172.21.2.101:0/1470606712 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337417043 0 0) 0x7fa37c000ff0 con 0x7fa3940e5470 2021-02-10T11:32:47.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.895 7fa39305f700 1 -- 172.21.2.101:0/1470606712 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3940d1db0 con 0x7fa39407e6a0 2021-02-10T11:32:47.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.895 7fa39305f700 1 --2- 172.21.2.101:0/1470606712 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa39407e6a0 0x7fa3940d1790 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7fa38c006e50 tx=0x7fa38c009df0).ready entity=mon.1 client_cookie=cea59cb27b12494 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:47.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.895 7fa39205d700 1 -- 172.21.2.101:0/1470606712 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa38c00b510 con 0x7fa39407e6a0 2021-02-10T11:32:47.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.895 7fa39205d700 1 -- 172.21.2.101:0/1470606712 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38c003030 con 0x7fa39407e6a0 2021-02-10T11:32:47.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.895 7fa39205d700 1 -- 172.21.2.101:0/1470606712 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa38c00b7d0 con 0x7fa39407e6a0 2021-02-10T11:32:47.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.895 7fa398a7b700 1 -- 172.21.2.101:0/1470606712 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa39407e6a0 msgr2=0x7fa3940d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:47.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.895 7fa398a7b700 1 --2- 172.21.2.101:0/1470606712 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa39407e6a0 0x7fa3940d1790 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7fa38c006e50 tx=0x7fa38c009df0).stop 2021-02-10T11:32:47.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.896 7fa398a7b700 1 -- 172.21.2.101:0/1470606712 shutdown_connections 2021-02-10T11:32:47.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.896 7fa398a7b700 1 --2- 172.21.2.101:0/1470606712 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa39407e6a0 0x7fa3940d1790 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:47.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.896 7fa398a7b700 1 -- 172.21.2.101:0/1470606712 >> 172.21.2.101:0/1470606712 conn(0x7fa3940ce690 msgr2=0x7fa3940e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:47.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.896 7fa398a7b700 1 -- 172.21.2.101:0/1470606712 shutdown_connections 2021-02-10T11:32:47.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.896 7fa398a7b700 1 -- 172.21.2.101:0/1470606712 wait complete. 2021-02-10T11:32:47.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.910 7fa398a7b700 1 Processor -- start 2021-02-10T11:32:47.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.911 7fa398a7b700 1 -- start start 2021-02-10T11:32:47.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.911 7fa398a7b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa39419d0b0 0x7fa39419d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:47.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.911 7fa398a7b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3940977d0 0x7fa394097bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:47.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.911 7fa398a7b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3940981f0 0x7fa394095d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:47.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.911 7fa398a7b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3940aea20 con 0x7fa39419d0b0 2021-02-10T11:32:47.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.911 7fa398a7b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3940985f0 con 0x7fa3940977d0 2021-02-10T11:32:47.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.911 7fa398a7b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa394098810 con 0x7fa3940981f0 2021-02-10T11:32:47.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.911 7fa39285e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa39419d0b0 0x7fa39419d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:47.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.911 7fa39285e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa39419d0b0 0x7fa39419d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51178/0 (socket says 172.21.2.101:51178) 2021-02-10T11:32:47.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.911 7fa39285e700 1 -- 172.21.2.101:0/4100204510 learned_addr learned my addr 172.21.2.101:0/4100204510 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:47.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.911 7fa38a05d700 1 --2- 172.21.2.101:0/4100204510 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3940977d0 0x7fa394097bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:47.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.911 7fa39305f700 1 --2- 172.21.2.101:0/4100204510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3940981f0 0x7fa394095d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:47.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.912 7fa39285e700 1 -- 172.21.2.101:0/4100204510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3940981f0 msgr2=0x7fa394095d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:47.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.912 7fa39285e700 1 --2- 172.21.2.101:0/4100204510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3940981f0 0x7fa394095d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:47.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.912 7fa39285e700 1 -- 172.21.2.101:0/4100204510 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3940977d0 msgr2=0x7fa394097bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:47.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.912 7fa39285e700 1 --2- 172.21.2.101:0/4100204510 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3940977d0 0x7fa394097bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:47.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.912 7fa39285e700 1 -- 172.21.2.101:0/4100204510 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa394096340 con 0x7fa39419d0b0 2021-02-10T11:32:47.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.912 7fa39285e700 1 --2- 172.21.2.101:0/4100204510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa39419d0b0 0x7fa39419d4b0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 rx=0x7fa38400b5c0 tx=0x7fa38400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:47.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.912 7fa38bfff700 1 -- 172.21.2.101:0/4100204510 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa384006030 con 0x7fa39419d0b0 2021-02-10T11:32:47.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.912 7fa38bfff700 1 -- 172.21.2.101:0/4100204510 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3840043a0 con 0x7fa39419d0b0 2021-02-10T11:32:47.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.912 7fa38bfff700 1 -- 172.21.2.101:0/4100204510 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa3840048c0 con 0x7fa39419d0b0 2021-02-10T11:32:47.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.912 7fa398a7b700 1 -- 172.21.2.101:0/4100204510 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa394096340 con 0x7fa39419d0b0 2021-02-10T11:32:47.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.912 7fa398a7b700 1 -- 172.21.2.101:0/4100204510 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa394096340 con 0x7fa39419d0b0 2021-02-10T11:32:47.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.913 7fa38bfff700 1 -- 172.21.2.101:0/4100204510 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa384004580 con 0x7fa39419d0b0 2021-02-10T11:32:47.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.913 7fa38bfff700 1 --2- 172.21.2.101:0/4100204510 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa3780186c0 0x7fa37801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:47.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.913 7fa38bfff700 1 -- 172.21.2.101:0/4100204510 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa384032800 con 0x7fa39419d0b0 2021-02-10T11:32:47.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.913 7fa38a05d700 1 --2- 172.21.2.101:0/4100204510 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa3780186c0 0x7fa37801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:47.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.913 7fa38a05d700 1 --2- 172.21.2.101:0/4100204510 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa3780186c0 0x7fa37801abc0 crc :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:47.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.944 7fa398a7b700 1 -- 172.21.2.101:0/4100204510 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3941071c0 con 0x7fa39419d0b0 2021-02-10T11:32:47.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:47.947 7fa38bfff700 1 -- 172.21.2.101:0/4100204510 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa384006b80 con 0x7fa39419d0b0 2021-02-10T11:32:48.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.083 7fa398a7b700 1 -- 172.21.2.101:0/4100204510 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs volume ls", "target": ["mgr", ""]}) v1 -- 0x7fa3940985f0 con 0x7fa3780186c0 2021-02-10T11:32:48.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.084 7fa38bfff700 1 -- 172.21.2.101:0/4100204510 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7fa37c0054b0 con 0x7fa3780186c0 2021-02-10T11:32:48.095 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:32:48.095 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:32:48.095 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "cephfs" 2021-02-10T11:32:48.095 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:32:48.095 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:32:48.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.085 7fa398a7b700 1 -- 172.21.2.101:0/4100204510 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa3780186c0 msgr2=0x7fa37801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:48.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.085 7fa398a7b700 1 --2- 172.21.2.101:0/4100204510 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa3780186c0 0x7fa37801abc0 crc :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:48.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.085 7fa398a7b700 1 -- 172.21.2.101:0/4100204510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa39419d0b0 msgr2=0x7fa39419d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:48.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.085 7fa398a7b700 1 --2- 172.21.2.101:0/4100204510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa39419d0b0 0x7fa39419d4b0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 rx=0x7fa38400b5c0 tx=0x7fa38400b960).stop 2021-02-10T11:32:48.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.085 7fa398a7b700 1 -- 172.21.2.101:0/4100204510 shutdown_connections 2021-02-10T11:32:48.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.085 7fa398a7b700 1 --2- 172.21.2.101:0/4100204510 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa3780186c0 0x7fa37801abc0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:48.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.085 7fa398a7b700 1 --2- 172.21.2.101:0/4100204510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3940981f0 0x7fa394095d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:48.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.085 7fa398a7b700 1 --2- 172.21.2.101:0/4100204510 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3940977d0 0x7fa394097bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:48.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.085 7fa398a7b700 1 --2- 172.21.2.101:0/4100204510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa39419d0b0 0x7fa39419d4b0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:48.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.085 7fa398a7b700 1 -- 172.21.2.101:0/4100204510 >> 172.21.2.101:0/4100204510 conn(0x7fa3940a74a0 msgr2=0x7fa3940a0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:48.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.086 7fa398a7b700 1 -- 172.21.2.101:0/4100204510 shutdown_connections 2021-02-10T11:32:48.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.086 7fa398a7b700 1 -- 172.21.2.101:0/4100204510 wait complete. 2021-02-10T11:32:48.105 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:32:48.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.510 7f1cb5797700 1 Processor -- start 2021-02-10T11:32:48.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.510 7f1cb5797700 1 -- start start 2021-02-10T11:32:48.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.511 7f1cb5797700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1cb00f05f0 0x7f1cb00f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:48.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.511 7f1cb5797700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb01b0c70 con 0x7f1cb01b0470 2021-02-10T11:32:48.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.511 7f1cb5797700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb00f0b50 con 0x7f1cb01b0870 2021-02-10T11:32:48.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.511 7f1cb5797700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cb00e5ea0 con 0x7f1cb00f05f0 2021-02-10T11:32:48.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.511 7f1caf574700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f1cb01b0470 0x7f1cb00ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51678/0 (socket says 172.21.2.101:51678) 2021-02-10T11:32:48.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.511 7f1caf574700 1 -- 172.21.2.101:0/3054133690 learned_addr learned my addr 172.21.2.101:0/3054133690 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:48.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.511 7f1cafd75700 1 --2- 172.21.2.101:0/3054133690 >> v2:172.21.2.108:3300/0 conn(0x7f1cb00f05f0 0x7f1cb00f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:48.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.511 7f1cae572700 1 -- 172.21.2.101:0/3054133690 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1401059495 0 0) 0x7f1c980010b0 con 0x7f1cb01b0470 2021-02-10T11:32:48.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.511 7f1cae572700 1 -- 172.21.2.101:0/3054133690 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c9c002fe0 con 0x7f1cb01b0470 2021-02-10T11:32:48.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.511 7f1cae572700 1 -- 172.21.2.101:0/3054133690 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2479813244 0 0) 0x7f1ca0000ff0 con 0x7f1cb01b0870 2021-02-10T11:32:48.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.511 7f1cae572700 1 -- 172.21.2.101:0/3054133690 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c9c003210 con 0x7f1cb01b0870 2021-02-10T11:32:48.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.511 7f1cafd75700 1 -- 172.21.2.101:0/3054133690 >> v1:172.21.2.108:6789/0 conn(0x7f1cb01b0870 legacy=0x7f1cb00efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:48.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.511 7f1cafd75700 1 -- 172.21.2.101:0/3054133690 >> v1:172.21.2.101:6789/0 conn(0x7f1cb01b0470 legacy=0x7f1cb00ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:48.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.511 7f1cafd75700 1 -- 172.21.2.101:0/3054133690 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb00e52e0 con 0x7f1cb00f05f0 2021-02-10T11:32:48.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.512 7f1cafd75700 1 --2- 172.21.2.101:0/3054133690 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1cb00f05f0 0x7f1cb00f20e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f1ca8006e50 tx=0x7f1ca8009df0).ready entity=mon.1 client_cookie=2aaa4f4cfbd00252 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:48.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.512 7f1cae572700 1 -- 172.21.2.101:0/3054133690 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1ca800b510 con 0x7f1cb00f05f0 2021-02-10T11:32:48.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.512 7f1cae572700 1 -- 172.21.2.101:0/3054133690 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca8003030 con 0x7f1cb00f05f0 2021-02-10T11:32:48.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.512 7f1cae572700 1 -- 172.21.2.101:0/3054133690 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1ca800b7d0 con 0x7f1cb00f05f0 2021-02-10T11:32:48.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.512 7f1cb5797700 1 -- 172.21.2.101:0/3054133690 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1cb00f05f0 msgr2=0x7f1cb00f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:48.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.512 7f1cb5797700 1 --2- 172.21.2.101:0/3054133690 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1cb00f05f0 0x7f1cb00f20e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f1ca8006e50 tx=0x7f1ca8009df0).stop 2021-02-10T11:32:48.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.512 7f1cb5797700 1 -- 172.21.2.101:0/3054133690 shutdown_connections 2021-02-10T11:32:48.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.512 7f1cb5797700 1 --2- 172.21.2.101:0/3054133690 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1cb00f05f0 0x7f1cb00f20e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:48.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.512 7f1cb5797700 1 -- 172.21.2.101:0/3054133690 >> 172.21.2.101:0/3054133690 conn(0x7f1cb00ce3e0 msgr2=0x7f1cb00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:48.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.512 7f1cb5797700 1 -- 172.21.2.101:0/3054133690 shutdown_connections 2021-02-10T11:32:48.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.512 7f1cb5797700 1 -- 172.21.2.101:0/3054133690 wait complete. 2021-02-10T11:32:48.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.527 7f1cb5797700 1 Processor -- start 2021-02-10T11:32:48.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.527 7f1cb5797700 1 -- start start 2021-02-10T11:32:48.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.527 7f1cb5797700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1cb0190920 0x7f1cb00b81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:48.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.527 7f1cb5797700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1cb00b87c0 0x7f1cb00b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:48.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.527 7f1cb5797700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1cb00b8bc0 0x7f1cb00b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:48.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.527 7f1cb5797700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb0190d20 con 0x7f1cb00b87c0 2021-02-10T11:32:48.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.527 7f1cb5797700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00b5ad0 con 0x7f1cb00b8bc0 2021-02-10T11:32:48.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.527 7f1cb5797700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00b7d00 con 0x7f1cb0190920 2021-02-10T11:32:48.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.528 7f1caed73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1cb00b87c0 0x7f1cb00b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:48.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.528 7f1caed73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1cb00b87c0 0x7f1cb00b70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51188/0 (socket says 172.21.2.101:51188) 2021-02-10T11:32:48.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.528 7f1caed73700 1 -- 172.21.2.101:0/3323224912 learned_addr learned my addr 172.21.2.101:0/3323224912 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:48.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.528 7f1caf574700 1 --2- 172.21.2.101:0/3323224912 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1cb0190920 0x7f1cb00b81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:48.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.528 7f1cafd75700 1 --2- 172.21.2.101:0/3323224912 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1cb00b8bc0 0x7f1cb00b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:48.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.528 7f1caf574700 1 -- 172.21.2.101:0/3323224912 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1cb00b8bc0 msgr2=0x7f1cb00b76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:48.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.528 7f1caf574700 1 --2- 172.21.2.101:0/3323224912 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1cb00b8bc0 0x7f1cb00b76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:48.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.528 7f1caf574700 1 -- 172.21.2.101:0/3323224912 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1cb00b87c0 msgr2=0x7f1cb00b70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:48.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.528 7f1caf574700 1 --2- 172.21.2.101:0/3323224912 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1cb00b87c0 0x7f1cb00b70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:48.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.528 7f1caf574700 1 -- 172.21.2.101:0/3323224912 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb00a4590 con 0x7f1cb0190920 2021-02-10T11:32:48.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.528 7f1caed73700 1 --2- 172.21.2.101:0/3323224912 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1cb00b87c0 0x7f1cb00b70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:32:48.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.528 7f1caf574700 1 --2- 172.21.2.101:0/3323224912 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1cb0190920 0x7f1cb00b81a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f1c98001c60 tx=0x7f1c9800a870).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:48.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.528 7f1ca7fff700 1 -- 172.21.2.101:0/3323224912 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1c9800ac60 con 0x7f1cb0190920 2021-02-10T11:32:48.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.528 7f1ca7fff700 1 -- 172.21.2.101:0/3323224912 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c98000cc0 con 0x7f1cb0190920 2021-02-10T11:32:48.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.528 7f1cb5797700 1 -- 172.21.2.101:0/3323224912 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb00a4590 con 0x7f1cb0190920 2021-02-10T11:32:48.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.528 7f1ca7fff700 1 -- 172.21.2.101:0/3323224912 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1c98004540 con 0x7f1cb0190920 2021-02-10T11:32:48.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.528 7f1cb5797700 1 -- 172.21.2.101:0/3323224912 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb00a4590 con 0x7f1cb0190920 2021-02-10T11:32:48.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.529 7f1ca7fff700 1 -- 172.21.2.101:0/3323224912 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1c98004140 con 0x7f1cb0190920 2021-02-10T11:32:48.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.529 7f1ca7fff700 1 --2- 172.21.2.101:0/3323224912 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1c940186c0 0x7f1c9401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:48.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.529 7f1ca7fff700 1 -- 172.21.2.101:0/3323224912 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1c980326c0 con 0x7f1cb0190920 2021-02-10T11:32:48.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.529 7f1caed73700 1 --2- 172.21.2.101:0/3323224912 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1c940186c0 0x7f1c9401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:48.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.529 7f1caed73700 1 --2- 172.21.2.101:0/3323224912 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1c940186c0 0x7f1c9401abc0 crc :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:48.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.561 7f1cb5797700 1 -- 172.21.2.101:0/3323224912 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb0105ae0 con 0x7f1cb0190920 2021-02-10T11:32:48.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.563 7f1ca7fff700 1 -- 172.21.2.101:0/3323224912 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1c98000cc0 con 0x7f1cb0190920 2021-02-10T11:32:48.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.713 7f1cb5797700 1 -- 172.21.2.101:0/3323224912 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1cb0105ae0 con 0x7f1cb0190920 2021-02-10T11:32:48.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.716 7f1ca7fff700 1 -- 172.21.2.101:0/3323224912 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1c98000ed0 con 0x7f1cb0190920 2021-02-10T11:32:48.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.718 7f1cb5797700 1 -- 172.21.2.101:0/3323224912 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1c940186c0 msgr2=0x7f1c9401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:48.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.718 7f1cb5797700 1 --2- 172.21.2.101:0/3323224912 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1c940186c0 0x7f1c9401abc0 crc :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:48.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.718 7f1cb5797700 1 -- 172.21.2.101:0/3323224912 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1cb0190920 msgr2=0x7f1cb00b81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:48.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.718 7f1cb5797700 1 --2- 172.21.2.101:0/3323224912 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1cb0190920 0x7f1cb00b81a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f1c98001c60 tx=0x7f1c9800a870).stop 2021-02-10T11:32:48.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.718 7f1cb5797700 1 -- 172.21.2.101:0/3323224912 shutdown_connections 2021-02-10T11:32:48.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.718 7f1cb5797700 1 --2- 172.21.2.101:0/3323224912 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1cb00b8bc0 0x7f1cb00b76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:48.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.718 7f1cb5797700 1 --2- 172.21.2.101:0/3323224912 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1cb00b87c0 0x7f1cb00b70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:48.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.718 7f1cb5797700 1 --2- 172.21.2.101:0/3323224912 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1c940186c0 0x7f1c9401abc0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:48.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.718 7f1cb5797700 1 --2- 172.21.2.101:0/3323224912 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1cb0190920 0x7f1cb00b81a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:48.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.718 7f1cb5797700 1 -- 172.21.2.101:0/3323224912 >> 172.21.2.101:0/3323224912 conn(0x7f1cb008ffe0 msgr2=0x7f1cb00bf050 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:48.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.718 7f1cb5797700 1 -- 172.21.2.101:0/3323224912 shutdown_connections 2021-02-10T11:32:48.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:48.718 7f1cb5797700 1 -- 172.21.2.101:0/3323224912 wait complete. 2021-02-10T11:32:48.738 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth get-or-create client.guest1 mds 'allow *' osd 'allow rw' mon 'allow *' 2021-02-10T11:32:49.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.149 7f1f4ce7a700 1 Processor -- start 2021-02-10T11:32:49.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.149 7f1f4ce7a700 1 -- start start 2021-02-10T11:32:49.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.150 7f1f4ce7a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1f481b0470 0x7f1f481b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:49.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.150 7f1f4ce7a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f480ef610 0x7f1f480efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:49.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.150 7f1f4ce7a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f480f0b50 con 0x7f1f480f0030 2021-02-10T11:32:49.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.150 7f1f4ce7a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f480e52e0 con 0x7f1f481b0470 2021-02-10T11:32:49.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.150 7f1f4ce7a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f480e54d0 con 0x7f1f480ef610 2021-02-10T11:32:49.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.150 7f1f46b13700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1f481b0470 0x7f1f481b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:49.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.150 7f1f46b13700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1f481b0470 0x7f1f481b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:33450/0 (socket says 172.21.2.101:33450) 2021-02-10T11:32:49.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.150 7f1f46312700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f480ef610 0x7f1f480efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:49.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.150 7f1f46b13700 1 -- 172.21.2.101:0/2237359802 learned_addr learned my addr 172.21.2.101:0/2237359802 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:49.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.151 7f1f46312700 1 -- 172.21.2.101:0/2237359802 >> v2:172.21.2.108:3300/0 conn(0x7f1f481b0470 msgr2=0x7f1f481b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:49.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.151 7f1f46312700 1 --2- 172.21.2.101:0/2237359802 >> v2:172.21.2.108:3300/0 conn(0x7f1f481b0470 0x7f1f481b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:49.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.151 7f1f46312700 1 -- 172.21.2.101:0/2237359802 >> v1:172.21.2.108:6789/0 conn(0x7f1f480f0030 legacy=0x7f1f480f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:49.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.151 7f1f46312700 1 -- 172.21.2.101:0/2237359802 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f480f0430 con 0x7f1f480ef610 2021-02-10T11:32:49.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.151 7f1f46312700 1 --2- 172.21.2.101:0/2237359802 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f480ef610 0x7f1f480efa10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f1f38006650 tx=0x7f1f38009a80).ready entity=mon.2 client_cookie=7cfb0c3b97d605ca server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:49.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.151 7f1f45b11700 1 -- 172.21.2.101:0/2237359802 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f380009b0 con 0x7f1f480ef610 2021-02-10T11:32:49.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.151 7f1f45b11700 1 -- 172.21.2.101:0/2237359802 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3800c0c0 con 0x7f1f480ef610 2021-02-10T11:32:49.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.151 7f1f45b11700 1 -- 172.21.2.101:0/2237359802 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f3800c490 con 0x7f1f480ef610 2021-02-10T11:32:49.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.151 7f1f4ce7a700 1 -- 172.21.2.101:0/2237359802 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f480ef610 msgr2=0x7f1f480efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:49.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.151 7f1f4ce7a700 1 --2- 172.21.2.101:0/2237359802 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f480ef610 0x7f1f480efa10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f1f38006650 tx=0x7f1f38009a80).stop 2021-02-10T11:32:49.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.152 7f1f4ce7a700 1 -- 172.21.2.101:0/2237359802 shutdown_connections 2021-02-10T11:32:49.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.152 7f1f4ce7a700 1 --2- 172.21.2.101:0/2237359802 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f480ef610 0x7f1f480efa10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:49.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.152 7f1f4ce7a700 1 --2- 172.21.2.101:0/2237359802 >> v2:172.21.2.108:3300/0 conn(0x7f1f481b0470 0x7f1f481b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:49.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.152 7f1f4ce7a700 1 -- 172.21.2.101:0/2237359802 >> 172.21.2.101:0/2237359802 conn(0x7f1f480ce3e0 msgr2=0x7f1f480e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:49.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.152 7f1f4ce7a700 1 -- 172.21.2.101:0/2237359802 shutdown_connections 2021-02-10T11:32:49.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.152 7f1f4ce7a700 1 -- 172.21.2.101:0/2237359802 wait complete. 2021-02-10T11:32:49.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.166 7f1f4ce7a700 1 Processor -- start 2021-02-10T11:32:49.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.166 7f1f4ce7a700 1 -- start start 2021-02-10T11:32:49.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.167 7f1f4ce7a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f4818ed50 0x7f1f480ad800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:49.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.167 7f1f4ce7a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f480ade20 0x7f1f481b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:49.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.167 7f1f4ce7a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f4819c2d0 0x7f1f481b73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:49.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.167 7f1f4ce7a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4818e7d0 con 0x7f1f480ade20 2021-02-10T11:32:49.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.167 7f1f4ce7a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f481b7a10 con 0x7f1f4819c2d0 2021-02-10T11:32:49.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.167 7f1f4ce7a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f481b7bf0 con 0x7f1f4818ed50 2021-02-10T11:32:49.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.167 7f1f46312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f480ade20 0x7f1f481b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:49.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.167 7f1f46312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f480ade20 0x7f1f481b6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51206/0 (socket says 172.21.2.101:51206) 2021-02-10T11:32:49.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.167 7f1f46312700 1 -- 172.21.2.101:0/1170840523 learned_addr learned my addr 172.21.2.101:0/1170840523 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:49.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.167 7f1f46312700 1 -- 172.21.2.101:0/1170840523 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f4818ed50 msgr2=0x7f1f480ad800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:49.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.167 7f1f46312700 1 --2- 172.21.2.101:0/1170840523 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f4818ed50 0x7f1f480ad800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:49.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.167 7f1f46312700 1 -- 172.21.2.101:0/1170840523 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f4819c2d0 msgr2=0x7f1f481b73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:49.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.167 7f1f46312700 1 --2- 172.21.2.101:0/1170840523 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f4819c2d0 0x7f1f481b73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:49.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.167 7f1f46312700 1 -- 172.21.2.101:0/1170840523 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4818cee0 con 0x7f1f480ade20 2021-02-10T11:32:49.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.167 7f1f46312700 1 --2- 172.21.2.101:0/1170840523 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f480ade20 0x7f1f481b6dd0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7f1f38009630 tx=0x7f1f3800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:49.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.168 7f1f3f7fe700 1 -- 172.21.2.101:0/1170840523 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f380093a0 con 0x7f1f480ade20 2021-02-10T11:32:49.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.168 7f1f3f7fe700 1 -- 172.21.2.101:0/1170840523 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3800bd70 con 0x7f1f480ade20 2021-02-10T11:32:49.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.168 7f1f4ce7a700 1 -- 172.21.2.101:0/1170840523 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f4818cee0 con 0x7f1f480ade20 2021-02-10T11:32:49.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.168 7f1f4ce7a700 1 -- 172.21.2.101:0/1170840523 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f4818cee0 con 0x7f1f480ade20 2021-02-10T11:32:49.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.168 7f1f3f7fe700 1 -- 172.21.2.101:0/1170840523 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f38005560 con 0x7f1f480ade20 2021-02-10T11:32:49.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.168 7f1f3f7fe700 1 -- 172.21.2.101:0/1170840523 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1f380052e0 con 0x7f1f480ade20 2021-02-10T11:32:49.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.169 7f1f3f7fe700 1 --2- 172.21.2.101:0/1170840523 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f2c018730 0x7f1f2c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:49.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.169 7f1f3f7fe700 1 -- 172.21.2.101:0/1170840523 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1f38032680 con 0x7f1f480ade20 2021-02-10T11:32:49.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.169 7f1f46b13700 1 --2- 172.21.2.101:0/1170840523 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f2c018730 0x7f1f2c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:49.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.169 7f1f46b13700 1 --2- 172.21.2.101:0/1170840523 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f2c018730 0x7f1f2c01ac10 crc :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:49.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.200 7f1f4ce7a700 1 -- 172.21.2.101:0/1170840523 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f481b7bf0 con 0x7f1f480ade20 2021-02-10T11:32:49.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.202 7f1f3f7fe700 1 -- 172.21.2.101:0/1170840523 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1f380052e0 con 0x7f1f480ade20 2021-02-10T11:32:49.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.332 7f1f4ce7a700 1 -- 172.21.2.101:0/1170840523 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.guest1", "caps": ["mds", "allow *", "osd", "allow rw", "mon", "allow *"]} v 0) v1 -- 0x7f1f481b7bf0 con 0x7f1f480ade20 2021-02-10T11:32:49.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.336 7f1f3f7fe700 1 -- 172.21.2.101:0/1170840523 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.guest1", "caps": ["mds", "allow *", "osd", "allow rw", "mon", "allow *"]}]=0 v29) v1 ==== 158+0+64 (secure 0 0 0) 0x7f1f3800bd70 con 0x7f1f480ade20 2021-02-10T11:32:49.350 INFO:teuthology.orchestra.run.gibba001.stdout:[client.guest1] 2021-02-10T11:32:49.350 INFO:teuthology.orchestra.run.gibba001.stdout: key = AQBhxCNgE5TnExAA0E6CPWdaeWdkD5/cs4fXXg== 2021-02-10T11:32:49.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.341 7f1f4ce7a700 1 -- 172.21.2.101:0/1170840523 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f2c018730 msgr2=0x7f1f2c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:49.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.341 7f1f4ce7a700 1 --2- 172.21.2.101:0/1170840523 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f2c018730 0x7f1f2c01ac10 crc :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:49.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.341 7f1f4ce7a700 1 -- 172.21.2.101:0/1170840523 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f480ade20 msgr2=0x7f1f481b6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:49.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.341 7f1f4ce7a700 1 --2- 172.21.2.101:0/1170840523 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f480ade20 0x7f1f481b6dd0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7f1f38009630 tx=0x7f1f3800b980).stop 2021-02-10T11:32:49.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.342 7f1f4ce7a700 1 -- 172.21.2.101:0/1170840523 shutdown_connections 2021-02-10T11:32:49.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.342 7f1f4ce7a700 1 --2- 172.21.2.101:0/1170840523 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f4819c2d0 0x7f1f481b73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:49.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.342 7f1f4ce7a700 1 --2- 172.21.2.101:0/1170840523 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f480ade20 0x7f1f481b6dd0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:49.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.342 7f1f4ce7a700 1 --2- 172.21.2.101:0/1170840523 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f2c018730 0x7f1f2c01ac10 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:49.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.342 7f1f4ce7a700 1 --2- 172.21.2.101:0/1170840523 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f4818ed50 0x7f1f480ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:49.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.342 7f1f4ce7a700 1 -- 172.21.2.101:0/1170840523 >> 172.21.2.101:0/1170840523 conn(0x7f1f481a3790 msgr2=0x7f1f4819f670 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:49.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.342 7f1f4ce7a700 1 -- 172.21.2.101:0/1170840523 shutdown_connections 2021-02-10T11:32:49.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.342 7f1f4ce7a700 1 -- 172.21.2.101:0/1170840523 wait complete. 2021-02-10T11:32:49.361 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolumegroup create cephfs group_0000000000436021 2021-02-10T11:32:49.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.763 7f29a66b1700 1 Processor -- start 2021-02-10T11:32:49.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.763 7f29a66b1700 1 -- start start 2021-02-10T11:32:49.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.764 7f29a66b1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29a01b0870 0x7f29a00f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:49.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.764 7f29a66b1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f29a00efe00 0x7f29a00f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:49.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.764 7f29a66b1700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a01b0c70 con 0x7f29a01b0470 2021-02-10T11:32:49.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.764 7f29a66b1700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a00f0b50 con 0x7f29a00efe00 2021-02-10T11:32:49.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.764 7f29a66b1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a00f0820 con 0x7f29a01b0870 2021-02-10T11:32:49.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.764 7f299ffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f29a01b0470 0x7f29a00ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51704/0 (socket says 172.21.2.101:51704) 2021-02-10T11:32:49.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.764 7f299ffff700 1 -- 172.21.2.101:0/2064392243 learned_addr learned my addr 172.21.2.101:0/2064392243 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:49.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.764 7f29a4cce700 1 --2- 172.21.2.101:0/2064392243 >> v2:172.21.2.101:3300/0 conn(0x7f29a00efe00 0x7f29a00f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:49.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.764 7f299f7fe700 1 --2- 172.21.2.101:0/2064392243 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29a01b0870 0x7f29a00f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:49.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.764 7f299effd700 1 -- 172.21.2.101:0/2064392243 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519778275 0 0) 0x7f2998001500 con 0x7f29a01b0470 2021-02-10T11:32:49.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.764 7f299effd700 1 -- 172.21.2.101:0/2064392243 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2988002fe0 con 0x7f29a01b0470 2021-02-10T11:32:49.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.764 7f29a4cce700 1 -- 172.21.2.101:0/2064392243 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29a01b0870 msgr2=0x7f29a00f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:49.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.764 7f29a4cce700 1 --2- 172.21.2.101:0/2064392243 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29a01b0870 0x7f29a00f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:49.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.764 7f29a4cce700 1 -- 172.21.2.101:0/2064392243 >> v1:172.21.2.101:6789/0 conn(0x7f29a01b0470 legacy=0x7f29a00ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:49.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.764 7f29a4cce700 1 -- 172.21.2.101:0/2064392243 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a00e52e0 con 0x7f29a00efe00 2021-02-10T11:32:49.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.764 7f299f7fe700 1 --2- 172.21.2.101:0/2064392243 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29a01b0870 0x7f29a00f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:32:49.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.765 7f29a4cce700 1 --2- 172.21.2.101:0/2064392243 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29a00efe00 0x7f29a00f0200 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7f2990006ab0 tx=0x7f2990009b40).ready entity=mon.0 client_cookie=9b5f90d2d0be40e9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:49.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.765 7f299effd700 1 -- 172.21.2.101:0/2064392243 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f29900022e0 con 0x7f29a00efe00 2021-02-10T11:32:49.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.765 7f299effd700 1 -- 172.21.2.101:0/2064392243 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29900025e0 con 0x7f29a00efe00 2021-02-10T11:32:49.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.765 7f29a66b1700 1 -- 172.21.2.101:0/2064392243 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29a00efe00 msgr2=0x7f29a00f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:49.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.765 7f29a66b1700 1 --2- 172.21.2.101:0/2064392243 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29a00efe00 0x7f29a00f0200 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7f2990006ab0 tx=0x7f2990009b40).stop 2021-02-10T11:32:49.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.765 7f299effd700 1 -- 172.21.2.101:0/2064392243 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2990002b00 con 0x7f29a00efe00 2021-02-10T11:32:49.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.765 7f29a66b1700 1 -- 172.21.2.101:0/2064392243 shutdown_connections 2021-02-10T11:32:49.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.765 7f29a66b1700 1 --2- 172.21.2.101:0/2064392243 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29a01b0870 0x7f29a00f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:49.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.765 7f29a66b1700 1 --2- 172.21.2.101:0/2064392243 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29a00efe00 0x7f29a00f0200 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:49.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.765 7f29a66b1700 1 -- 172.21.2.101:0/2064392243 >> 172.21.2.101:0/2064392243 conn(0x7f29a00ce3e0 msgr2=0x7f29a00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:49.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.765 7f29a66b1700 1 -- 172.21.2.101:0/2064392243 shutdown_connections 2021-02-10T11:32:49.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.765 7f29a66b1700 1 -- 172.21.2.101:0/2064392243 wait complete. 2021-02-10T11:32:49.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.780 7f29a66b1700 1 Processor -- start 2021-02-10T11:32:49.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.780 7f29a66b1700 1 -- start start 2021-02-10T11:32:49.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.780 7f29a66b1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29a00a0420 0x7f29a00a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:49.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.780 7f29a66b1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29a0189d00 0x7f29a018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:49.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.780 7f29a66b1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29a018a720 0x7f29a00b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:49.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.780 7f29a66b1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a0189780 con 0x7f29a00a0420 2021-02-10T11:32:49.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.780 7f29a66b1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a018ab20 con 0x7f29a018a720 2021-02-10T11:32:49.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.780 7f29a66b1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a00a1140 con 0x7f29a0189d00 2021-02-10T11:32:49.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.781 7f299ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29a00a0420 0x7f29a00a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:49.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.781 7f299ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29a00a0420 0x7f29a00a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51216/0 (socket says 172.21.2.101:51216) 2021-02-10T11:32:49.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.781 7f299ffff700 1 -- 172.21.2.101:0/811280504 learned_addr learned my addr 172.21.2.101:0/811280504 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:49.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.781 7f299f7fe700 1 --2- 172.21.2.101:0/811280504 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29a0189d00 0x7f29a018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:49.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.781 7f29a4cce700 1 --2- 172.21.2.101:0/811280504 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29a018a720 0x7f29a00b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:49.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.781 7f299ffff700 1 -- 172.21.2.101:0/811280504 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29a0189d00 msgr2=0x7f29a018a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:49.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.781 7f299ffff700 1 --2- 172.21.2.101:0/811280504 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29a0189d00 0x7f29a018a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:49.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.781 7f299ffff700 1 -- 172.21.2.101:0/811280504 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29a018a720 msgr2=0x7f29a00b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:49.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.781 7f299ffff700 1 --2- 172.21.2.101:0/811280504 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29a018a720 0x7f29a00b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:49.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.781 7f299ffff700 1 -- 172.21.2.101:0/811280504 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a00b8200 con 0x7f29a00a0420 2021-02-10T11:32:49.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.781 7f299ffff700 1 --2- 172.21.2.101:0/811280504 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29a00a0420 0x7f29a00a0820 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7f299800b8c0 tx=0x7f299800bc60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:49.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.782 7f299cff9700 1 -- 172.21.2.101:0/811280504 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2998006030 con 0x7f29a00a0420 2021-02-10T11:32:49.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.782 7f299cff9700 1 -- 172.21.2.101:0/811280504 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29980063b0 con 0x7f29a00a0420 2021-02-10T11:32:49.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.782 7f299cff9700 1 -- 172.21.2.101:0/811280504 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2998004910 con 0x7f29a00a0420 2021-02-10T11:32:49.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.782 7f29a66b1700 1 -- 172.21.2.101:0/811280504 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a00b8200 con 0x7f29a00a0420 2021-02-10T11:32:49.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.782 7f29a66b1700 1 -- 172.21.2.101:0/811280504 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a00b8200 con 0x7f29a00a0420 2021-02-10T11:32:49.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.782 7f299cff9700 1 -- 172.21.2.101:0/811280504 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2998004720 con 0x7f29a00a0420 2021-02-10T11:32:49.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.782 7f299cff9700 1 --2- 172.21.2.101:0/811280504 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29800186c0 0x7f298001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:49.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.782 7f299cff9700 1 -- 172.21.2.101:0/811280504 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2998032760 con 0x7f29a00a0420 2021-02-10T11:32:49.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.782 7f299f7fe700 1 --2- 172.21.2.101:0/811280504 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29800186c0 0x7f298001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:49.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.783 7f299f7fe700 1 --2- 172.21.2.101:0/811280504 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29800186c0 0x7f298001abc0 crc :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:49.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.813 7f29a66b1700 1 -- 172.21.2.101:0/811280504 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a0105ae0 con 0x7f29a00a0420 2021-02-10T11:32:49.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.816 7f299cff9700 1 -- 172.21.2.101:0/811280504 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f299802df80 con 0x7f29a00a0420 2021-02-10T11:32:49.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.945 7f29a66b1700 1 -- 172.21.2.101:0/811280504 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "target": ["mgr", ""], "group_name": "group_0000000000436021"}) v1 -- 0x7f29a00f0b50 con 0x7f29800186c0 2021-02-10T11:32:49.959 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:32:49.948 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:32:49.959 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:32:49.948 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:32:49.959 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:32:49.948 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:32:49.959 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:32:49.948 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:32:49.959 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:32:49.948 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:32:49.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.963 7f299cff9700 1 -- 172.21.2.101:0/811280504 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f2994006a30 con 0x7f29800186c0 2021-02-10T11:32:49.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.977 7f29a66b1700 1 -- 172.21.2.101:0/811280504 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29800186c0 msgr2=0x7f298001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:49.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.977 7f29a66b1700 1 --2- 172.21.2.101:0/811280504 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29800186c0 0x7f298001abc0 crc :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:49.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.977 7f29a66b1700 1 -- 172.21.2.101:0/811280504 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29a00a0420 msgr2=0x7f29a00a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:49.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.977 7f29a66b1700 1 --2- 172.21.2.101:0/811280504 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29a00a0420 0x7f29a00a0820 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7f299800b8c0 tx=0x7f299800bc60).stop 2021-02-10T11:32:49.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.978 7f29a66b1700 1 -- 172.21.2.101:0/811280504 shutdown_connections 2021-02-10T11:32:49.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.978 7f29a66b1700 1 --2- 172.21.2.101:0/811280504 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29800186c0 0x7f298001abc0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:49.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.978 7f29a66b1700 1 --2- 172.21.2.101:0/811280504 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29a0189d00 0x7f29a018a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:49.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.978 7f29a66b1700 1 --2- 172.21.2.101:0/811280504 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29a018a720 0x7f29a00b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:49.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.978 7f29a66b1700 1 --2- 172.21.2.101:0/811280504 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29a00a0420 0x7f29a00a0820 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:49.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.978 7f29a66b1700 1 -- 172.21.2.101:0/811280504 >> 172.21.2.101:0/811280504 conn(0x7f29a0094e10 msgr2=0x7f29a00a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:49.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.978 7f29a66b1700 1 -- 172.21.2.101:0/811280504 shutdown_connections 2021-02-10T11:32:49.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:49.978 7f29a66b1700 1 -- 172.21.2.101:0/811280504 wait complete. 2021-02-10T11:32:49.997 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000415154 --group_name group_0000000000436021 2021-02-10T11:32:50.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.399 7fd97ef80700 1 Processor -- start 2021-02-10T11:32:50.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.399 7fd97ef80700 1 -- start start 2021-02-10T11:32:50.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.400 7fd97ef80700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd9780d1860 0x7fd9781b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:50.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.400 7fd97ef80700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9780f0b00 con 0x7fd9780f0640 2021-02-10T11:32:50.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.400 7fd97ef80700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9780d0300 con 0x7fd9781b0a00 2021-02-10T11:32:50.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.400 7fd97ef80700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9780d0520 con 0x7fd9780d1860 2021-02-10T11:32:50.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.400 7fd97d59d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd9780f0640 0x7fd9780cfb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51732/0 (socket says 172.21.2.101:51732) 2021-02-10T11:32:50.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.400 7fd97d59d700 1 -- 172.21.2.101:0/1743983418 learned_addr learned my addr 172.21.2.101:0/1743983418 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:50.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.400 7fd97cd9c700 1 --2- 172.21.2.101:0/1743983418 >> v2:172.21.2.108:3300/0 conn(0x7fd9780d1860 0x7fd9781b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:50.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.400 7fd9777fe700 1 -- 172.21.2.101:0/1743983418 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1167368003 0 0) 0x7fd970001500 con 0x7fd9780f0640 2021-02-10T11:32:50.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.400 7fd9777fe700 1 -- 172.21.2.101:0/1743983418 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd964002fe0 con 0x7fd9780f0640 2021-02-10T11:32:50.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.401 7fd9777fe700 1 -- 172.21.2.101:0/1743983418 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 613310991 0 0) 0x7fd970001500 con 0x7fd9780f0640 2021-02-10T11:32:50.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.401 7fd9777fe700 1 -- 172.21.2.101:0/1743983418 >> v2:172.21.2.108:3300/0 conn(0x7fd9780d1860 msgr2=0x7fd9781b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:50.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.401 7fd9777fe700 1 --2- 172.21.2.101:0/1743983418 >> v2:172.21.2.108:3300/0 conn(0x7fd9780d1860 0x7fd9781b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:50.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.401 7fd9777fe700 1 -- 172.21.2.101:0/1743983418 >> v1:172.21.2.108:6789/0 conn(0x7fd9781b0a00 legacy=0x7fd9780cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:50.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.401 7fd9777fe700 1 -- 172.21.2.101:0/1743983418 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9780d1c60 con 0x7fd9780f0640 2021-02-10T11:32:50.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.401 7fd9777fe700 1 -- 172.21.2.101:0/1743983418 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fd970001a00 con 0x7fd9780f0640 2021-02-10T11:32:50.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.401 7fd9777fe700 1 -- 172.21.2.101:0/1743983418 >> v1:172.21.2.101:6789/0 conn(0x7fd9780f0640 legacy=0x7fd9780cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:50.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.401 7fd9777fe700 1 --2- 172.21.2.101:0/1743983418 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd964003ae0 0x7fd964005f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:50.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.401 7fd9777fe700 1 --2- 172.21.2.101:0/1743983418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd964006830 0x7fd964008d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:50.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.401 7fd9777fe700 1 --2- 172.21.2.101:0/1743983418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd964009630 0x7fd96400bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:50.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.401 7fd9777fe700 1 -- 172.21.2.101:0/1743983418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96400c3a0 con 0x7fd964006830 2021-02-10T11:32:50.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.401 7fd9777fe700 1 -- 172.21.2.101:0/1743983418 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96400c5c0 con 0x7fd964003ae0 2021-02-10T11:32:50.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.401 7fd9777fe700 1 -- 172.21.2.101:0/1743983418 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd96400c7e0 con 0x7fd964009630 2021-02-10T11:32:50.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.401 7fd97cd9c700 1 --2- 172.21.2.101:0/1743983418 >> v2:172.21.2.108:3300/0 conn(0x7fd9780d1860 0x7fd9781b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:32:50.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.401 7fd977fff700 1 --2- 172.21.2.101:0/1743983418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd964006830 0x7fd964008d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:50.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.401 7fd977fff700 1 -- 172.21.2.101:0/1743983418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd964009630 msgr2=0x7fd96400bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:50.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.401 7fd977fff700 1 --2- 172.21.2.101:0/1743983418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd964009630 0x7fd96400bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:50.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.401 7fd977fff700 1 -- 172.21.2.101:0/1743983418 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd964003ae0 msgr2=0x7fd964005f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:50.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.401 7fd977fff700 1 --2- 172.21.2.101:0/1743983418 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd964003ae0 0x7fd964005f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:50.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.401 7fd977fff700 1 -- 172.21.2.101:0/1743983418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd964050a50 con 0x7fd964006830 2021-02-10T11:32:50.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.401 7fd97d59d700 1 -- 172.21.2.101:0/1743983418 reap_dead start 2021-02-10T11:32:50.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.401 7fd977fff700 1 --2- 172.21.2.101:0/1743983418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd964006830 0x7fd964008d30 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7fd968006ab0 tx=0x7fd968009ac0).ready entity=mon.0 client_cookie=480971c3a7f33d80 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:50.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.402 7fd9777fe700 1 -- 172.21.2.101:0/1743983418 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd9680066a0 con 0x7fd964006830 2021-02-10T11:32:50.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.402 7fd9777fe700 1 -- 172.21.2.101:0/1743983418 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9680066a0 con 0x7fd964006830 2021-02-10T11:32:50.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.402 7fd9777fe700 1 -- 172.21.2.101:0/1743983418 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd9680022e0 con 0x7fd964006830 2021-02-10T11:32:50.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.402 7fd97ef80700 1 -- 172.21.2.101:0/1743983418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd964006830 msgr2=0x7fd964008d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:50.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.402 7fd97ef80700 1 --2- 172.21.2.101:0/1743983418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd964006830 0x7fd964008d30 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7fd968006ab0 tx=0x7fd968009ac0).stop 2021-02-10T11:32:50.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.402 7fd97ef80700 1 -- 172.21.2.101:0/1743983418 shutdown_connections 2021-02-10T11:32:50.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.402 7fd97ef80700 1 --2- 172.21.2.101:0/1743983418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd964006830 0x7fd964008d30 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:50.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.402 7fd97ef80700 1 -- 172.21.2.101:0/1743983418 >> 172.21.2.101:0/1743983418 conn(0x7fd9780ce440 msgr2=0x7fd9780d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:50.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.402 7fd97ef80700 1 -- 172.21.2.101:0/1743983418 shutdown_connections 2021-02-10T11:32:50.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.402 7fd97ef80700 1 -- 172.21.2.101:0/1743983418 wait complete. 2021-02-10T11:32:50.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.417 7fd97ef80700 1 Processor -- start 2021-02-10T11:32:50.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.417 7fd97ef80700 1 -- start start 2021-02-10T11:32:50.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.417 7fd97ef80700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd978098b00 0x7fd978098f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:50.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.417 7fd97ef80700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd978099520 0x7fd978099d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:50.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.417 7fd97ef80700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd97809a330 0x7fd97809b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:50.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.417 7fd97ef80700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97809adf0 con 0x7fd97809a330 2021-02-10T11:32:50.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.417 7fd97ef80700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97819d2e0 con 0x7fd978098b00 2021-02-10T11:32:50.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.417 7fd97ef80700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97809a730 con 0x7fd978099520 2021-02-10T11:32:50.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.418 7fd97d59d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd97809a330 0x7fd97809b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:50.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.418 7fd97d59d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd97809a330 0x7fd97809b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51252/0 (socket says 172.21.2.101:51252) 2021-02-10T11:32:50.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.418 7fd97d59d700 1 -- 172.21.2.101:0/1894533271 learned_addr learned my addr 172.21.2.101:0/1894533271 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:50.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.418 7fd97d59d700 1 -- 172.21.2.101:0/1894533271 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd978099520 msgr2=0x7fd978099d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:50.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.418 7fd97d59d700 1 --2- 172.21.2.101:0/1894533271 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd978099520 0x7fd978099d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:50.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.418 7fd97d59d700 1 -- 172.21.2.101:0/1894533271 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd978098b00 msgr2=0x7fd978098f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:50.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.418 7fd97d59d700 1 --2- 172.21.2.101:0/1894533271 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd978098b00 0x7fd978098f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:50.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.418 7fd97d59d700 1 -- 172.21.2.101:0/1894533271 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97809a950 con 0x7fd97809a330 2021-02-10T11:32:50.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.418 7fd97d59d700 1 --2- 172.21.2.101:0/1894533271 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd97809a330 0x7fd97809b2f0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7fd97000a8f0 tx=0x7fd97000ac90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:50.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.418 7fd9757fa700 1 -- 172.21.2.101:0/1894533271 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd970007020 con 0x7fd97809a330 2021-02-10T11:32:50.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.418 7fd9757fa700 1 -- 172.21.2.101:0/1894533271 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd970000cc0 con 0x7fd97809a330 2021-02-10T11:32:50.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.418 7fd97ef80700 1 -- 172.21.2.101:0/1894533271 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97809a950 con 0x7fd97809a330 2021-02-10T11:32:50.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.418 7fd9757fa700 1 -- 172.21.2.101:0/1894533271 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd970004720 con 0x7fd97809a330 2021-02-10T11:32:50.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.418 7fd97ef80700 1 -- 172.21.2.101:0/1894533271 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97809bc00 con 0x7fd97809a330 2021-02-10T11:32:50.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.419 7fd9757fa700 1 -- 172.21.2.101:0/1894533271 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd9700072a0 con 0x7fd97809a330 2021-02-10T11:32:50.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.419 7fd9757fa700 1 --2- 172.21.2.101:0/1894533271 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd95c0186c0 0x7fd95c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:50.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.419 7fd9757fa700 1 -- 172.21.2.101:0/1894533271 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd970032890 con 0x7fd97809a330 2021-02-10T11:32:50.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.419 7fd97cd9c700 1 --2- 172.21.2.101:0/1894533271 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd95c0186c0 0x7fd95c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:50.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.419 7fd97cd9c700 1 --2- 172.21.2.101:0/1894533271 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd95c0186c0 0x7fd95c01abc0 crc :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:50.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.450 7fd97ef80700 1 -- 172.21.2.101:0/1894533271 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9780ebbe0 con 0x7fd97809a330 2021-02-10T11:32:50.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.452 7fd9757fa700 1 -- 172.21.2.101:0/1894533271 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd970001270 con 0x7fd97809a330 2021-02-10T11:32:50.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.582 7fd97ef80700 1 -- 172.21.2.101:0/1894533271 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000415154", "target": ["mgr", ""], "group_name": "group_0000000000436021"}) v1 -- 0x7fd97809bc00 con 0x7fd95c0186c0 2021-02-10T11:32:50.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.606 7fd9757fa700 1 -- 172.21.2.101:0/1894533271 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fd9600052e0 con 0x7fd95c0186c0 2021-02-10T11:32:50.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.615 7fd97ef80700 1 -- 172.21.2.101:0/1894533271 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd95c0186c0 msgr2=0x7fd95c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:50.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.615 7fd97ef80700 1 --2- 172.21.2.101:0/1894533271 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd95c0186c0 0x7fd95c01abc0 crc :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:50.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.615 7fd97ef80700 1 -- 172.21.2.101:0/1894533271 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd97809a330 msgr2=0x7fd97809b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:50.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.615 7fd97ef80700 1 --2- 172.21.2.101:0/1894533271 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd97809a330 0x7fd97809b2f0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7fd97000a8f0 tx=0x7fd97000ac90).stop 2021-02-10T11:32:50.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.615 7fd97ef80700 1 -- 172.21.2.101:0/1894533271 shutdown_connections 2021-02-10T11:32:50.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.615 7fd97ef80700 1 --2- 172.21.2.101:0/1894533271 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd95c0186c0 0x7fd95c01abc0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:50.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.615 7fd97ef80700 1 --2- 172.21.2.101:0/1894533271 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd978099520 0x7fd978099d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:50.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.615 7fd97ef80700 1 --2- 172.21.2.101:0/1894533271 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd97809a330 0x7fd97809b2f0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:50.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.615 7fd97ef80700 1 --2- 172.21.2.101:0/1894533271 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd978098b00 0x7fd978098f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:50.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.615 7fd97ef80700 1 -- 172.21.2.101:0/1894533271 >> 172.21.2.101:0/1894533271 conn(0x7fd9780b2a80 msgr2=0x7fd97809e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:50.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.616 7fd97ef80700 1 -- 172.21.2.101:0/1894533271 shutdown_connections 2021-02-10T11:32:50.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:50.616 7fd97ef80700 1 -- 172.21.2.101:0/1894533271 wait complete. 2021-02-10T11:32:50.634 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume authorize cephfs subvolume_0000000000415154 guest1 --group_name group_0000000000436021 --tenant_id tenant1 2021-02-10T11:32:51.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.045 7f8c63453700 1 Processor -- start 2021-02-10T11:32:51.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.045 7f8c63453700 1 -- start start 2021-02-10T11:32:51.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.046 7f8c63453700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c5c0e5010 0x7f8c5c1b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:51.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.046 7f8c63453700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8c5c0e5410 0x7f8c5c0f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:51.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.046 7f8c63453700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c5c0f5480 con 0x7f8c5c1b3de0 2021-02-10T11:32:51.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.046 7f8c63453700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c5c0e5e40 con 0x7f8c5c0e5410 2021-02-10T11:32:51.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.046 7f8c63453700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c0d1750 con 0x7f8c5c0e5010 2021-02-10T11:32:51.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.046 7f8c61a70700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8c5c0e5410 0x7f8c5c0f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:51.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.046 7f8c61a70700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8c5c0e5410 0x7f8c5c0f4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51258/0 (socket says 172.21.2.101:51258) 2021-02-10T11:32:51.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.046 7f8c61a70700 1 -- 172.21.2.101:0/1183629269 learned_addr learned my addr 172.21.2.101:0/1183629269 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:51.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.046 7f8c61a70700 1 -- 172.21.2.101:0/1183629269 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c5c0e5010 msgr2=0x7f8c5c1b37c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:32:51.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.046 7f8c61a70700 1 --2- 172.21.2.101:0/1183629269 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c5c0e5010 0x7f8c5c1b37c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:51.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.046 7f8c61a70700 1 -- 172.21.2.101:0/1183629269 >> v1:172.21.2.101:6789/0 conn(0x7f8c5c1b3de0 legacy=0x7f8c5c0f5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:51.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.046 7f8c61a70700 1 -- 172.21.2.101:0/1183629269 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5c0d1940 con 0x7f8c5c0e5410 2021-02-10T11:32:51.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.046 7f8c61a70700 1 --2- 172.21.2.101:0/1183629269 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c5c0e5410 0x7f8c5c0f4de0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7f8c54006f00 tx=0x7f8c54009f10).ready entity=mon.0 client_cookie=e2290fe46303392 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:51.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.047 7f8c4ffff700 1 -- 172.21.2.101:0/1183629269 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c54003030 con 0x7f8c5c0e5410 2021-02-10T11:32:51.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.047 7f8c4ffff700 1 -- 172.21.2.101:0/1183629269 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5400b3b0 con 0x7f8c5c0e5410 2021-02-10T11:32:51.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.047 7f8c4ffff700 1 -- 172.21.2.101:0/1183629269 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c5400b8d0 con 0x7f8c5c0e5410 2021-02-10T11:32:51.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.047 7f8c63453700 1 -- 172.21.2.101:0/1183629269 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c5c0e5410 msgr2=0x7f8c5c0f4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:51.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.047 7f8c63453700 1 --2- 172.21.2.101:0/1183629269 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c5c0e5410 0x7f8c5c0f4de0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7f8c54006f00 tx=0x7f8c54009f10).stop 2021-02-10T11:32:51.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.047 7f8c63453700 1 -- 172.21.2.101:0/1183629269 shutdown_connections 2021-02-10T11:32:51.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.047 7f8c63453700 1 --2- 172.21.2.101:0/1183629269 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c5c0e5410 0x7f8c5c0f4de0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:51.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.047 7f8c63453700 1 --2- 172.21.2.101:0/1183629269 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c5c0e5010 0x7f8c5c1b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:51.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.047 7f8c63453700 1 -- 172.21.2.101:0/1183629269 >> 172.21.2.101:0/1183629269 conn(0x7f8c5c0ce670 msgr2=0x7f8c5c0e44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:51.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.048 7f8c63453700 1 -- 172.21.2.101:0/1183629269 shutdown_connections 2021-02-10T11:32:51.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.048 7f8c63453700 1 -- 172.21.2.101:0/1183629269 wait complete. 2021-02-10T11:32:51.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.062 7f8c63453700 1 Processor -- start 2021-02-10T11:32:51.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.062 7f8c63453700 1 -- start start 2021-02-10T11:32:51.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.062 7f8c63453700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c5c187e70 0x7f8c5c09c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:51.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.062 7f8c63453700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c5c0b7990 0x7f8c5c0b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:51.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.062 7f8c63453700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c5c1b1600 0x7f8c5c1b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:51.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.062 7f8c63453700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c188460 con 0x7f8c5c0b7990 2021-02-10T11:32:51.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.062 7f8c63453700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c1b2020 con 0x7f8c5c187e70 2021-02-10T11:32:51.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.062 7f8c63453700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c18beb0 con 0x7f8c5c1b1600 2021-02-10T11:32:51.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.063 7f8c60a6e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c5c0b7990 0x7f8c5c0b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:51.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.063 7f8c60a6e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c5c0b7990 0x7f8c5c0b7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51264/0 (socket says 172.21.2.101:51264) 2021-02-10T11:32:51.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.063 7f8c60a6e700 1 -- 172.21.2.101:0/1874708990 learned_addr learned my addr 172.21.2.101:0/1874708990 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:51.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.063 7f8c60a6e700 1 -- 172.21.2.101:0/1874708990 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c5c1b1600 msgr2=0x7f8c5c1b1a00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:51.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.063 7f8c60a6e700 1 --2- 172.21.2.101:0/1874708990 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c5c1b1600 0x7f8c5c1b1a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:51.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.063 7f8c60a6e700 1 -- 172.21.2.101:0/1874708990 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c5c187e70 msgr2=0x7f8c5c09c760 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:51.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.063 7f8c60a6e700 1 --2- 172.21.2.101:0/1874708990 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c5c187e70 0x7f8c5c09c760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:51.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.063 7f8c60a6e700 1 -- 172.21.2.101:0/1874708990 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5c1889e0 con 0x7f8c5c0b7990 2021-02-10T11:32:51.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.063 7f8c60a6e700 1 --2- 172.21.2.101:0/1874708990 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c5c0b7990 0x7f8c5c0b7d90 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 rx=0x7f8c58006ed0 tx=0x7f8c5800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:51.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.063 7f8c4dffb700 1 -- 172.21.2.101:0/1874708990 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c58006020 con 0x7f8c5c0b7990 2021-02-10T11:32:51.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.063 7f8c4dffb700 1 -- 172.21.2.101:0/1874708990 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c58006360 con 0x7f8c5c0b7990 2021-02-10T11:32:51.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.063 7f8c4dffb700 1 -- 172.21.2.101:0/1874708990 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c58005730 con 0x7f8c5c0b7990 2021-02-10T11:32:51.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.063 7f8c63453700 1 -- 172.21.2.101:0/1874708990 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5c1889e0 con 0x7f8c5c0b7990 2021-02-10T11:32:51.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.063 7f8c63453700 1 -- 172.21.2.101:0/1874708990 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5c1889e0 con 0x7f8c5c0b7990 2021-02-10T11:32:51.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.064 7f8c4dffb700 1 -- 172.21.2.101:0/1874708990 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8c580054b0 con 0x7f8c5c0b7990 2021-02-10T11:32:51.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.064 7f8c4dffb700 1 --2- 172.21.2.101:0/1874708990 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c3c0186d0 0x7f8c3c01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:51.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.064 7f8c4dffb700 1 -- 172.21.2.101:0/1874708990 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8c58032800 con 0x7f8c5c0b7990 2021-02-10T11:32:51.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.064 7f8c6126f700 1 --2- 172.21.2.101:0/1874708990 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c3c0186d0 0x7f8c3c01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:51.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.064 7f8c6126f700 1 --2- 172.21.2.101:0/1874708990 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c3c0186d0 0x7f8c3c01abd0 crc :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:51.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.096 7f8c63453700 1 -- 172.21.2.101:0/1874708990 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c5c101700 con 0x7f8c5c0b7990 2021-02-10T11:32:51.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.099 7f8c4dffb700 1 -- 172.21.2.101:0/1874708990 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8c58006000 con 0x7f8c5c0b7990 2021-02-10T11:32:51.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.223 7f8c63453700 1 -- 172.21.2.101:0/1874708990 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000415154", "target": ["mgr", ""], "tenant_id": "tenant1", "auth_id": "guest1", "group_name": "group_0000000000436021"}) v1 -- 0x7f8c5c0f5480 con 0x7f8c3c0186d0 2021-02-10T11:32:51.240 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:32:51.229 7f2cc39e1700 -1 mgr.server reply reply (1) Operation not permitted auth ID: guest1 exists and not created by mgr plugin. Not allowed to modify 2021-02-10T11:32:51.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.229 7f8c4dffb700 1 -- 172.21.2.101:0/1874708990 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: -1 auth ID: guest1 exists and not created by mgr plugin. Not allowed to modify) v1 ==== 83+0+0 (crc 0 0 0) 0x7f8c50006970 con 0x7f8c3c0186d0 2021-02-10T11:32:51.241 INFO:teuthology.orchestra.run.gibba001.stderr:Error EPERM: auth ID: guest1 exists and not created by mgr plugin. Not allowed to modify 2021-02-10T11:32:51.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.231 7f8c63453700 1 -- 172.21.2.101:0/1874708990 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c3c0186d0 msgr2=0x7f8c3c01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:51.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.231 7f8c63453700 1 --2- 172.21.2.101:0/1874708990 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c3c0186d0 0x7f8c3c01abd0 crc :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:51.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.231 7f8c63453700 1 -- 172.21.2.101:0/1874708990 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c5c0b7990 msgr2=0x7f8c5c0b7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:51.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.231 7f8c63453700 1 --2- 172.21.2.101:0/1874708990 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c5c0b7990 0x7f8c5c0b7d90 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 rx=0x7f8c58006ed0 tx=0x7f8c5800b960).stop 2021-02-10T11:32:51.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.232 7f8c63453700 1 -- 172.21.2.101:0/1874708990 shutdown_connections 2021-02-10T11:32:51.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.232 7f8c63453700 1 --2- 172.21.2.101:0/1874708990 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c3c0186d0 0x7f8c3c01abd0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:51.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.232 7f8c63453700 1 --2- 172.21.2.101:0/1874708990 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c5c1b1600 0x7f8c5c1b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:51.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.232 7f8c63453700 1 --2- 172.21.2.101:0/1874708990 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c5c0b7990 0x7f8c5c0b7d90 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:51.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.232 7f8c63453700 1 --2- 172.21.2.101:0/1874708990 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c5c187e70 0x7f8c5c09c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:51.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.232 7f8c63453700 1 -- 172.21.2.101:0/1874708990 >> 172.21.2.101:0/1874708990 conn(0x7f8c5c19b640 msgr2=0x7f8c5c19bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:51.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.232 7f8c63453700 1 -- 172.21.2.101:0/1874708990 shutdown_connections 2021-02-10T11:32:51.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.232 7f8c63453700 1 -- 172.21.2.101:0/1874708990 wait complete. 2021-02-10T11:32:51.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:32:51.251 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth rm client.guest1 2021-02-10T11:32:51.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.649 7f3151d54700 1 Processor -- start 2021-02-10T11:32:51.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.649 7f3151d54700 1 -- start start 2021-02-10T11:32:51.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.650 7f3151d54700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f314c07e6a0 0x7f314c0d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:51.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.650 7f3151d54700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f314c07ed80 con 0x7f314c0e5070 2021-02-10T11:32:51.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.650 7f3151d54700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f314c07eaa0 con 0x7f314c0e5470 2021-02-10T11:32:51.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.650 7f3151d54700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f314c1b4080 con 0x7f314c07e6a0 2021-02-10T11:32:51.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.650 7f314bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f314c07e6a0 0x7f314c0d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:51.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.650 7f314bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f314c07e6a0 0x7f314c0d1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51274/0 (socket says 172.21.2.101:51274) 2021-02-10T11:32:51.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.650 7f314bfff700 1 -- 172.21.2.101:0/4170042503 learned_addr learned my addr 172.21.2.101:0/4170042503 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:51.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.650 7f314bfff700 1 -- 172.21.2.101:0/4170042503 >> v1:172.21.2.108:6789/0 conn(0x7f314c0e5470 legacy=0x7f314c07f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:51.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.650 7f314bfff700 1 -- 172.21.2.101:0/4170042503 >> v1:172.21.2.101:6789/0 conn(0x7f314c0e5070 legacy=0x7f314c1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:51.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.650 7f314bfff700 1 -- 172.21.2.101:0/4170042503 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314c0d1db0 con 0x7f314c07e6a0 2021-02-10T11:32:51.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.651 7f314bfff700 1 --2- 172.21.2.101:0/4170042503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f314c07e6a0 0x7f314c0d1790 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7f313c006b80 tx=0x7f313c009b90).ready entity=mon.0 client_cookie=485f63e6bbb324f6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:51.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.651 7f314a7fc700 1 -- 172.21.2.101:0/4170042503 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f313c0022e0 con 0x7f314c07e6a0 2021-02-10T11:32:51.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.651 7f314a7fc700 1 -- 172.21.2.101:0/4170042503 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c002560 con 0x7f314c07e6a0 2021-02-10T11:32:51.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.651 7f314a7fc700 1 -- 172.21.2.101:0/4170042503 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f313c002a80 con 0x7f314c07e6a0 2021-02-10T11:32:51.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.651 7f3151d54700 1 -- 172.21.2.101:0/4170042503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f314c07e6a0 msgr2=0x7f314c0d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:51.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.651 7f3151d54700 1 --2- 172.21.2.101:0/4170042503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f314c07e6a0 0x7f314c0d1790 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7f313c006b80 tx=0x7f313c009b90).stop 2021-02-10T11:32:51.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.651 7f3151d54700 1 -- 172.21.2.101:0/4170042503 shutdown_connections 2021-02-10T11:32:51.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.651 7f3151d54700 1 --2- 172.21.2.101:0/4170042503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f314c07e6a0 0x7f314c0d1790 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:51.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.651 7f3151d54700 1 -- 172.21.2.101:0/4170042503 >> 172.21.2.101:0/4170042503 conn(0x7f314c0ce690 msgr2=0x7f314c0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:51.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.651 7f3151d54700 1 -- 172.21.2.101:0/4170042503 shutdown_connections 2021-02-10T11:32:51.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.651 7f3151d54700 1 -- 172.21.2.101:0/4170042503 wait complete. 2021-02-10T11:32:51.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.666 7f3151d54700 1 Processor -- start 2021-02-10T11:32:51.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.666 7f3151d54700 1 -- start start 2021-02-10T11:32:51.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.666 7f3151d54700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f314c0a3d70 0x7f314c0a3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:51.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.666 7f3151d54700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f314c097f70 0x7f314c0a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:51.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.666 7f3151d54700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f314c098370 0x7f314c0a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:51.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.666 7f3151d54700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c098770 con 0x7f314c098370 2021-02-10T11:32:51.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.666 7f3151d54700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c0a4220 con 0x7f314c0a3d70 2021-02-10T11:32:51.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.666 7f3151d54700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f314c0a3670 con 0x7f314c097f70 2021-02-10T11:32:51.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.667 7f314bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f314c098370 0x7f314c0a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:51.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.667 7f314bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f314c098370 0x7f314c0a4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51280/0 (socket says 172.21.2.101:51280) 2021-02-10T11:32:51.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.667 7f314b7fe700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f314c0a3d70 0x7f314c0a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:51.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.667 7f314bfff700 1 -- 172.21.2.101:0/4249943041 learned_addr learned my addr 172.21.2.101:0/4249943041 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:51.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.667 7f314affd700 1 --2- 172.21.2.101:0/4249943041 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f314c097f70 0x7f314c0a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:51.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.667 7f314bfff700 1 -- 172.21.2.101:0/4249943041 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f314c097f70 msgr2=0x7f314c0a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:51.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.667 7f314bfff700 1 --2- 172.21.2.101:0/4249943041 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f314c097f70 0x7f314c0a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:51.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.667 7f314bfff700 1 -- 172.21.2.101:0/4249943041 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f314c0a3d70 msgr2=0x7f314c0a3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:51.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.667 7f314bfff700 1 --2- 172.21.2.101:0/4249943041 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f314c0a3d70 0x7f314c0a3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:51.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.667 7f314bfff700 1 -- 172.21.2.101:0/4249943041 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314c0a53e0 con 0x7f314c098370 2021-02-10T11:32:51.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.667 7f314bfff700 1 --2- 172.21.2.101:0/4249943041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f314c098370 0x7f314c0a4dc0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0x7f313c006f10 tx=0x7f313c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:51.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.667 7f3133fff700 1 -- 172.21.2.101:0/4249943041 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f313c009100 con 0x7f314c098370 2021-02-10T11:32:51.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.667 7f3133fff700 1 -- 172.21.2.101:0/4249943041 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c000b90 con 0x7f314c098370 2021-02-10T11:32:51.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.667 7f3133fff700 1 -- 172.21.2.101:0/4249943041 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f313c002080 con 0x7f314c098370 2021-02-10T11:32:51.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.667 7f3151d54700 1 -- 172.21.2.101:0/4249943041 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f314c0a53e0 con 0x7f314c098370 2021-02-10T11:32:51.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.668 7f3151d54700 1 -- 172.21.2.101:0/4249943041 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f314c0a53e0 con 0x7f314c098370 2021-02-10T11:32:51.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.668 7f3133fff700 1 -- 172.21.2.101:0/4249943041 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f313c0054a0 con 0x7f314c098370 2021-02-10T11:32:51.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.668 7f3133fff700 1 --2- 172.21.2.101:0/4249943041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f312c0186c0 0x7f312c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:51.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.668 7f3133fff700 1 -- 172.21.2.101:0/4249943041 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f313c0323b0 con 0x7f314c098370 2021-02-10T11:32:51.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.668 7f314b7fe700 1 --2- 172.21.2.101:0/4249943041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f312c0186c0 0x7f312c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:51.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.669 7f314b7fe700 1 --2- 172.21.2.101:0/4249943041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f312c0186c0 0x7f312c01abc0 crc :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:51.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.700 7f3151d54700 1 -- 172.21.2.101:0/4249943041 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f314c0897d0 con 0x7f314c098370 2021-02-10T11:32:51.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.703 7f3133fff700 1 -- 172.21.2.101:0/4249943041 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f313c009310 con 0x7f314c098370 2021-02-10T11:32:51.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.833 7f3151d54700 1 -- 172.21.2.101:0/4249943041 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth rm", "entity": "client.guest1"} v 0) v1 -- 0x7f314c0ce690 con 0x7f314c098370 2021-02-10T11:32:51.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.836 7f3133fff700 1 -- 172.21.2.101:0/4249943041 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth rm", "entity": "client.guest1"}]=0 updated v30) v1 ==== 89+0+0 (secure 0 0 0) 0x7f313c0054a0 con 0x7f314c098370 2021-02-10T11:32:51.851 INFO:teuthology.orchestra.run.gibba001.stderr:updated 2021-02-10T11:32:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.841 7f3151d54700 1 -- 172.21.2.101:0/4249943041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f312c0186c0 msgr2=0x7f312c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.841 7f3151d54700 1 --2- 172.21.2.101:0/4249943041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f312c0186c0 0x7f312c01abc0 crc :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.841 7f3151d54700 1 -- 172.21.2.101:0/4249943041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f314c098370 msgr2=0x7f314c0a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.841 7f3151d54700 1 --2- 172.21.2.101:0/4249943041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f314c098370 0x7f314c0a4dc0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0x7f313c006f10 tx=0x7f313c00b960).stop 2021-02-10T11:32:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.842 7f3151d54700 1 -- 172.21.2.101:0/4249943041 shutdown_connections 2021-02-10T11:32:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.842 7f3151d54700 1 --2- 172.21.2.101:0/4249943041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f312c0186c0 0x7f312c01abc0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.842 7f3151d54700 1 --2- 172.21.2.101:0/4249943041 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f314c097f70 0x7f314c0a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.842 7f3151d54700 1 --2- 172.21.2.101:0/4249943041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f314c098370 0x7f314c0a4dc0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.842 7f3151d54700 1 --2- 172.21.2.101:0/4249943041 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f314c0a3d70 0x7f314c0a3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.842 7f3151d54700 1 -- 172.21.2.101:0/4249943041 >> 172.21.2.101:0/4249943041 conn(0x7f314c089a50 msgr2=0x7f314c08edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.842 7f3151d54700 1 -- 172.21.2.101:0/4249943041 shutdown_connections 2021-02-10T11:32:51.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:51.842 7f3151d54700 1 -- 172.21.2.101:0/4249943041 wait complete. 2021-02-10T11:32:51.861 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000415154 --group_name group_0000000000436021 2021-02-10T11:32:52.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.264 7f762f56c700 1 Processor -- start 2021-02-10T11:32:52.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.264 7f762f56c700 1 -- start start 2021-02-10T11:32:52.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.264 7f762f56c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f76280d0330 0x7f76280ef630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:52.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.264 7f762f56c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f76280d1c60 0x7f76280efc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:52.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.264 7f762f56c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76280f0b00 con 0x7f76280d1860 2021-02-10T11:32:52.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.264 7f762f56c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76280e5ec0 con 0x7f76280d1c60 2021-02-10T11:32:52.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.265 7f762f56c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76281b0bd0 con 0x7f76280d0330 2021-02-10T11:32:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.265 7f762db89700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f76280d1c60 0x7f76280efc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.265 7f762db89700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f76280d1c60 0x7f76280efc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51288/0 (socket says 172.21.2.101:51288) 2021-02-10T11:32:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.265 7f762db89700 1 -- 172.21.2.101:0/3700489293 learned_addr learned my addr 172.21.2.101:0/3700489293 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.265 7f762d388700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f76280d1860 0x7f76281b03e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51778/0 (socket says 172.21.2.101:51778) 2021-02-10T11:32:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.265 7f762cb87700 1 --2- 172.21.2.101:0/3700489293 >> v2:172.21.2.108:3300/0 conn(0x7f76280d0330 0x7f76280ef630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.265 7f762db89700 1 -- 172.21.2.101:0/3700489293 >> v2:172.21.2.108:3300/0 conn(0x7f76280d0330 msgr2=0x7f76280ef630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.265 7f762db89700 1 --2- 172.21.2.101:0/3700489293 >> v2:172.21.2.108:3300/0 conn(0x7f76280d0330 0x7f76280ef630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.265 7f762db89700 1 -- 172.21.2.101:0/3700489293 >> v1:172.21.2.101:6789/0 conn(0x7f76280d1860 legacy=0x7f76281b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.265 7f762db89700 1 -- 172.21.2.101:0/3700489293 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76280f2090 con 0x7f76280d1c60 2021-02-10T11:32:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.265 7f762cb87700 1 --2- 172.21.2.101:0/3700489293 >> v2:172.21.2.108:3300/0 conn(0x7f76280d0330 0x7f76280ef630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:32:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.265 7f762db89700 1 --2- 172.21.2.101:0/3700489293 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f76280d1c60 0x7f76280efc50 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7f7620006f00 tx=0x7f7620009f10).ready entity=mon.0 client_cookie=97a0093ab8ff082 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.266 7f761bfff700 1 -- 172.21.2.101:0/3700489293 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f762000b3b0 con 0x7f76280d1c60 2021-02-10T11:32:52.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.266 7f761bfff700 1 -- 172.21.2.101:0/3700489293 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762000b730 con 0x7f76280d1c60 2021-02-10T11:32:52.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.266 7f761bfff700 1 -- 172.21.2.101:0/3700489293 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f762000bc50 con 0x7f76280d1c60 2021-02-10T11:32:52.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.266 7f762f56c700 1 -- 172.21.2.101:0/3700489293 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f76280d1c60 msgr2=0x7f76280efc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:52.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.266 7f762f56c700 1 --2- 172.21.2.101:0/3700489293 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f76280d1c60 0x7f76280efc50 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7f7620006f00 tx=0x7f7620009f10).stop 2021-02-10T11:32:52.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.267 7f762f56c700 1 -- 172.21.2.101:0/3700489293 shutdown_connections 2021-02-10T11:32:52.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.267 7f762f56c700 1 --2- 172.21.2.101:0/3700489293 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f76280d1c60 0x7f76280efc50 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:52.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.267 7f762f56c700 1 --2- 172.21.2.101:0/3700489293 >> v2:172.21.2.108:3300/0 conn(0x7f76280d0330 0x7f76280ef630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:52.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.267 7f762f56c700 1 -- 172.21.2.101:0/3700489293 >> 172.21.2.101:0/3700489293 conn(0x7f76280ce440 msgr2=0x7f76280d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:52.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.267 7f762f56c700 1 -- 172.21.2.101:0/3700489293 shutdown_connections 2021-02-10T11:32:52.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.267 7f762f56c700 1 -- 172.21.2.101:0/3700489293 wait complete. 2021-02-10T11:32:52.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.281 7f762f56c700 1 Processor -- start 2021-02-10T11:32:52.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.281 7f762f56c700 1 -- start start 2021-02-10T11:32:52.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.282 7f762f56c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7628099d50 0x7f762809a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:52.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.282 7f762f56c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f762809a770 0x7f762809ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:52.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.282 7f762f56c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f762808a460 0x7f762808a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:52.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.282 7f762f56c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76281a4910 con 0x7f762809a770 2021-02-10T11:32:52.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.282 7f762f56c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76281a4b30 con 0x7f7628099d50 2021-02-10T11:32:52.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.282 7f762f56c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762808ae80 con 0x7f762808a460 2021-02-10T11:32:52.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.282 7f762cb87700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f762809a770 0x7f762809ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:52.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.282 7f762cb87700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f762809a770 0x7f762809ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51290/0 (socket says 172.21.2.101:51290) 2021-02-10T11:32:52.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.282 7f762cb87700 1 -- 172.21.2.101:0/488119735 learned_addr learned my addr 172.21.2.101:0/488119735 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:52.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.282 7f762cb87700 1 -- 172.21.2.101:0/488119735 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f762808a460 msgr2=0x7f762808a860 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:52.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.282 7f762cb87700 1 --2- 172.21.2.101:0/488119735 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f762808a460 0x7f762808a860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:52.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.282 7f762cb87700 1 -- 172.21.2.101:0/488119735 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7628099d50 msgr2=0x7f762809a150 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:32:52.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.282 7f762cb87700 1 --2- 172.21.2.101:0/488119735 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7628099d50 0x7f762809a150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:52.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.282 7f762cb87700 1 -- 172.21.2.101:0/488119735 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762808b0a0 con 0x7f762809a770 2021-02-10T11:32:52.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.283 7f762cb87700 1 --2- 172.21.2.101:0/488119735 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f762809a770 0x7f762809ab70 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 rx=0x7f761c006f40 tx=0x7f761c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:52.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.283 7f7619ffb700 1 -- 172.21.2.101:0/488119735 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f761c00bd80 con 0x7f762809a770 2021-02-10T11:32:52.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.283 7f7619ffb700 1 -- 172.21.2.101:0/488119735 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761c000c90 con 0x7f762809a770 2021-02-10T11:32:52.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.283 7f7619ffb700 1 -- 172.21.2.101:0/488119735 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f761c002220 con 0x7f762809a770 2021-02-10T11:32:52.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.283 7f762f56c700 1 -- 172.21.2.101:0/488119735 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f762808b0a0 con 0x7f762809a770 2021-02-10T11:32:52.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.283 7f762f56c700 1 -- 172.21.2.101:0/488119735 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f762808b0a0 con 0x7f762809a770 2021-02-10T11:32:52.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.283 7f7619ffb700 1 -- 172.21.2.101:0/488119735 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f761c00a000 con 0x7f762809a770 2021-02-10T11:32:52.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.284 7f7619ffb700 1 --2- 172.21.2.101:0/488119735 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76080186c0 0x7f760801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:52.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.284 7f7619ffb700 1 -- 172.21.2.101:0/488119735 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f761c032790 con 0x7f762809a770 2021-02-10T11:32:52.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.284 7f762d388700 1 --2- 172.21.2.101:0/488119735 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76080186c0 0x7f760801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:52.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.284 7f762d388700 1 --2- 172.21.2.101:0/488119735 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76080186c0 0x7f760801abc0 crc :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:52.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.315 7f762f56c700 1 -- 172.21.2.101:0/488119735 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76280ebbe0 con 0x7f762809a770 2021-02-10T11:32:52.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.317 7f7619ffb700 1 -- 172.21.2.101:0/488119735 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f761c000c90 con 0x7f762809a770 2021-02-10T11:32:52.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.444 7f762f56c700 1 -- 172.21.2.101:0/488119735 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000415154", "target": ["mgr", ""], "group_name": "group_0000000000436021"}) v1 -- 0x7f76280ebbe0 con 0x7f76080186c0 2021-02-10T11:32:52.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.453 7f7619ffb700 1 -- 172.21.2.101:0/488119735 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f7624007b90 con 0x7f76080186c0 2021-02-10T11:32:52.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.461 7f762f56c700 1 -- 172.21.2.101:0/488119735 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76080186c0 msgr2=0x7f760801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:52.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.461 7f762f56c700 1 --2- 172.21.2.101:0/488119735 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76080186c0 0x7f760801abc0 crc :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:52.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.461 7f762f56c700 1 -- 172.21.2.101:0/488119735 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f762809a770 msgr2=0x7f762809ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:52.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.461 7f762f56c700 1 --2- 172.21.2.101:0/488119735 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f762809a770 0x7f762809ab70 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 rx=0x7f761c006f40 tx=0x7f761c00b990).stop 2021-02-10T11:32:52.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.461 7f762f56c700 1 -- 172.21.2.101:0/488119735 shutdown_connections 2021-02-10T11:32:52.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.461 7f762f56c700 1 --2- 172.21.2.101:0/488119735 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76080186c0 0x7f760801abc0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:52.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.461 7f762f56c700 1 --2- 172.21.2.101:0/488119735 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f762808a460 0x7f762808a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:52.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.461 7f762f56c700 1 --2- 172.21.2.101:0/488119735 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f762809a770 0x7f762809ab70 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:52.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.461 7f762f56c700 1 --2- 172.21.2.101:0/488119735 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7628099d50 0x7f762809a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:52.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.461 7f762f56c700 1 -- 172.21.2.101:0/488119735 >> 172.21.2.101:0/488119735 conn(0x7f76280b2a80 msgr2=0x7f762809e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:52.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.461 7f762f56c700 1 -- 172.21.2.101:0/488119735 shutdown_connections 2021-02-10T11:32:52.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.461 7f762f56c700 1 -- 172.21.2.101:0/488119735 wait complete. 2021-02-10T11:32:52.481 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolumegroup rm cephfs group_0000000000436021 2021-02-10T11:32:52.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.885 7fad2a89e700 1 Processor -- start 2021-02-10T11:32:52.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.885 7fad2a89e700 1 -- start start 2021-02-10T11:32:52.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.885 7fad2a89e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fad241b0470 0x7fad241b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:52.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.885 7fad2a89e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad240ef610 0x7fad240efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:52.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.885 7fad2a89e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad240f0b50 con 0x7fad240f0030 2021-02-10T11:32:52.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.886 7fad2a89e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad240e52e0 con 0x7fad241b0470 2021-02-10T11:32:52.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.886 7fad2a89e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad240e54d0 con 0x7fad240ef610 2021-02-10T11:32:52.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.886 7fad28ebb700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fad240f0030 0x7fad240f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51792/0 (socket says 172.21.2.101:51792) 2021-02-10T11:32:52.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.886 7fad28ebb700 1 -- 172.21.2.101:0/2053422933 learned_addr learned my addr 172.21.2.101:0/2053422933 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:52.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.886 7fad237fe700 1 --2- 172.21.2.101:0/2053422933 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad240ef610 0x7fad240efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:52.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.886 7fad23fff700 1 --2- 172.21.2.101:0/2053422933 >> v2:172.21.2.108:3300/0 conn(0x7fad241b0470 0x7fad241b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:52.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.886 7fad22ffd700 1 -- 172.21.2.101:0/2053422933 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964389886 0 0) 0x7fad1c000f90 con 0x7fad240f0030 2021-02-10T11:32:52.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.886 7fad22ffd700 1 -- 172.21.2.101:0/2053422933 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad0c002fe0 con 0x7fad240f0030 2021-02-10T11:32:52.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.886 7fad237fe700 1 -- 172.21.2.101:0/2053422933 >> v2:172.21.2.108:3300/0 conn(0x7fad241b0470 msgr2=0x7fad241b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:52.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.886 7fad237fe700 1 --2- 172.21.2.101:0/2053422933 >> v2:172.21.2.108:3300/0 conn(0x7fad241b0470 0x7fad241b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:52.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.886 7fad237fe700 1 -- 172.21.2.101:0/2053422933 >> v1:172.21.2.101:6789/0 conn(0x7fad240f0030 legacy=0x7fad240f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:52.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.886 7fad237fe700 1 -- 172.21.2.101:0/2053422933 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad240f0430 con 0x7fad240ef610 2021-02-10T11:32:52.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.887 7fad237fe700 1 --2- 172.21.2.101:0/2053422933 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad240ef610 0x7fad240efa10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fad18006650 tx=0x7fad18009a80).ready entity=mon.2 client_cookie=464a7dedeffa7f7f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:52.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.887 7fad22ffd700 1 -- 172.21.2.101:0/2053422933 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fad180009b0 con 0x7fad240ef610 2021-02-10T11:32:52.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.887 7fad22ffd700 1 -- 172.21.2.101:0/2053422933 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1800c0c0 con 0x7fad240ef610 2021-02-10T11:32:52.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.887 7fad22ffd700 1 -- 172.21.2.101:0/2053422933 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fad1800c5e0 con 0x7fad240ef610 2021-02-10T11:32:52.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.887 7fad2a89e700 1 -- 172.21.2.101:0/2053422933 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad240ef610 msgr2=0x7fad240efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:52.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.887 7fad2a89e700 1 --2- 172.21.2.101:0/2053422933 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad240ef610 0x7fad240efa10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fad18006650 tx=0x7fad18009a80).stop 2021-02-10T11:32:52.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.888 7fad2a89e700 1 -- 172.21.2.101:0/2053422933 shutdown_connections 2021-02-10T11:32:52.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.888 7fad2a89e700 1 --2- 172.21.2.101:0/2053422933 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad240ef610 0x7fad240efa10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:52.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.888 7fad2a89e700 1 --2- 172.21.2.101:0/2053422933 >> v2:172.21.2.108:3300/0 conn(0x7fad241b0470 0x7fad241b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:52.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.888 7fad2a89e700 1 -- 172.21.2.101:0/2053422933 >> 172.21.2.101:0/2053422933 conn(0x7fad240ce3e0 msgr2=0x7fad240e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:52.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.888 7fad2a89e700 1 -- 172.21.2.101:0/2053422933 shutdown_connections 2021-02-10T11:32:52.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.888 7fad2a89e700 1 -- 172.21.2.101:0/2053422933 wait complete. 2021-02-10T11:32:52.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.902 7fad2a89e700 1 Processor -- start 2021-02-10T11:32:52.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.902 7fad2a89e700 1 -- start start 2021-02-10T11:32:52.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.903 7fad2a89e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad240a0420 0x7fad240a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:52.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.903 7fad2a89e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad24189d00 0x7fad2418a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:52.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.903 7fad2a89e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fad2418a720 0x7fad240b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:52.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.903 7fad2a89e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad24189800 con 0x7fad24189d00 2021-02-10T11:32:52.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.903 7fad2a89e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2418ab20 con 0x7fad2418a720 2021-02-10T11:32:52.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.903 7fad2a89e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad240a1140 con 0x7fad240a0420 2021-02-10T11:32:52.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.903 7fad237fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad24189d00 0x7fad2418a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:52.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.903 7fad237fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad24189d00 0x7fad2418a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51304/0 (socket says 172.21.2.101:51304) 2021-02-10T11:32:52.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.903 7fad237fe700 1 -- 172.21.2.101:0/1294713324 learned_addr learned my addr 172.21.2.101:0/1294713324 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:52.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.903 7fad237fe700 1 -- 172.21.2.101:0/1294713324 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad240a0420 msgr2=0x7fad240a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:52.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.903 7fad237fe700 1 --2- 172.21.2.101:0/1294713324 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad240a0420 0x7fad240a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:52.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.903 7fad237fe700 1 -- 172.21.2.101:0/1294713324 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fad2418a720 msgr2=0x7fad240b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:52.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.903 7fad237fe700 1 --2- 172.21.2.101:0/1294713324 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fad2418a720 0x7fad240b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:52.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.903 7fad237fe700 1 -- 172.21.2.101:0/1294713324 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad240b8200 con 0x7fad24189d00 2021-02-10T11:32:52.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.903 7fad237fe700 1 --2- 172.21.2.101:0/1294713324 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad24189d00 0x7fad2418a100 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7fad18009630 tx=0x7fad1800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:52.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.904 7fad20ff9700 1 -- 172.21.2.101:0/1294713324 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fad180093a0 con 0x7fad24189d00 2021-02-10T11:32:52.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.904 7fad20ff9700 1 -- 172.21.2.101:0/1294713324 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1800bd70 con 0x7fad24189d00 2021-02-10T11:32:52.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.904 7fad2a89e700 1 -- 172.21.2.101:0/1294713324 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad240b8200 con 0x7fad24189d00 2021-02-10T11:32:52.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.904 7fad20ff9700 1 -- 172.21.2.101:0/1294713324 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fad18005560 con 0x7fad24189d00 2021-02-10T11:32:52.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.904 7fad2a89e700 1 -- 172.21.2.101:0/1294713324 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad240b8200 con 0x7fad24189d00 2021-02-10T11:32:52.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.904 7fad20ff9700 1 -- 172.21.2.101:0/1294713324 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fad180054f0 con 0x7fad24189d00 2021-02-10T11:32:52.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.904 7fad20ff9700 1 --2- 172.21.2.101:0/1294713324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fad04018730 0x7fad0401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:52.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.904 7fad20ff9700 1 -- 172.21.2.101:0/1294713324 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fad18032790 con 0x7fad24189d00 2021-02-10T11:32:52.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.904 7fad23fff700 1 --2- 172.21.2.101:0/1294713324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fad04018730 0x7fad0401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:52.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.905 7fad23fff700 1 --2- 172.21.2.101:0/1294713324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fad04018730 0x7fad0401abc0 crc :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:52.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.935 7fad2a89e700 1 -- 172.21.2.101:0/1294713324 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad2418ab20 con 0x7fad24189d00 2021-02-10T11:32:52.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:52.938 7fad20ff9700 1 -- 172.21.2.101:0/1294713324 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fad180052e0 con 0x7fad24189d00 2021-02-10T11:32:53.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:53.064 7fad2a89e700 1 -- 172.21.2.101:0/1294713324 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "target": ["mgr", ""], "group_name": "group_0000000000436021"}) v1 -- 0x7fad2418ab20 con 0x7fad04018730 2021-02-10T11:32:53.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:53.069 7fad20ff9700 1 -- 172.21.2.101:0/1294713324 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fad14006e70 con 0x7fad04018730 2021-02-10T11:32:53.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:53.073 7fad2a89e700 1 -- 172.21.2.101:0/1294713324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fad04018730 msgr2=0x7fad0401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:53.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:53.073 7fad2a89e700 1 --2- 172.21.2.101:0/1294713324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fad04018730 0x7fad0401abc0 crc :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:53.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:53.073 7fad2a89e700 1 -- 172.21.2.101:0/1294713324 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad24189d00 msgr2=0x7fad2418a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:53.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:53.073 7fad2a89e700 1 --2- 172.21.2.101:0/1294713324 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad24189d00 0x7fad2418a100 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7fad18009630 tx=0x7fad1800b980).stop 2021-02-10T11:32:53.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:53.074 7fad2a89e700 1 -- 172.21.2.101:0/1294713324 shutdown_connections 2021-02-10T11:32:53.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:53.074 7fad2a89e700 1 --2- 172.21.2.101:0/1294713324 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fad2418a720 0x7fad240b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:53.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:53.074 7fad2a89e700 1 --2- 172.21.2.101:0/1294713324 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad24189d00 0x7fad2418a100 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:53.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:53.074 7fad2a89e700 1 --2- 172.21.2.101:0/1294713324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fad04018730 0x7fad0401abc0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:53.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:53.074 7fad2a89e700 1 --2- 172.21.2.101:0/1294713324 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad240a0420 0x7fad240a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:53.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:53.074 7fad2a89e700 1 -- 172.21.2.101:0/1294713324 >> 172.21.2.101:0/1294713324 conn(0x7fad24094e10 msgr2=0x7fad240a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:53.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:53.074 7fad2a89e700 1 -- 172.21.2.101:0/1294713324 shutdown_connections 2021-02-10T11:32:53.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:53.074 7fad2a89e700 1 -- 172.21.2.101:0/1294713324 wait complete. 2021-02-10T11:32:53.093 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:32:53.093 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:32:53.439 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:32:53.783 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:32:53.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:53.783 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:32:53.783 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:32:54.117 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:32:54.118 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:32:54.118 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:32:54.118 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:32:54.118 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:32:54.468 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[114372]: fuse finished with error 0 and tester_r 0 2021-02-10T11:32:54.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:54.469 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:32:54.469 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:32:54.804 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:32:54.804 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:32:54.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:32:54.805 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:32:55.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:55.140 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:32:55.140 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:32:55.475 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:32:55.476 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:32:55.476 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:32:55.476 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:32:55.476 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:32:55.833 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[115013]: fuse finished with error 0 and tester_r 0 2021-02-10T11:32:56.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:56.379 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:32:56.379 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:32:56.718 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:32:56.719 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:32:56.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:32:56.720 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:32:57.058 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:57.059 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:32:57.059 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:32:57.394 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:32:57.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:32:57.395 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:32:57.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:32:57.396 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:32:57.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:32:57.730 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:32:57.730 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:32:58.065 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:32:58.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:32:58.065 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:32:58.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:32:58.066 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:32:58.154 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:32:58.158 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:32:58.400 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-02-10T11:32:58.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.803 7fcd171eb700 1 Processor -- start 2021-02-10T11:32:58.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.803 7fcd171eb700 1 -- start start 2021-02-10T11:32:58.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.803 7fcd171eb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd100d17f0 0x7fcd101b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:58.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.803 7fcd171eb700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fcd100cff00 0x7fcd100ef620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:58.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.803 7fcd171eb700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd100f0b20 con 0x7fcd100d1bf0 2021-02-10T11:32:58.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.803 7fcd171eb700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd100e5eb0 con 0x7fcd100cff00 2021-02-10T11:32:58.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.803 7fcd171eb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd101b3ff0 con 0x7fcd100d17f0 2021-02-10T11:32:58.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.804 7fcd14806700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fcd100d1bf0 0x7fcd101b3800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:45214/0 (socket says 172.21.2.101:45214) 2021-02-10T11:32:58.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.804 7fcd14806700 1 -- 172.21.2.101:0/3079810878 learned_addr learned my addr 172.21.2.101:0/3079810878 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:58.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.804 7fcd15808700 1 --2- 172.21.2.101:0/3079810878 >> v2:172.21.2.108:3300/0 conn(0x7fcd100cff00 0x7fcd100ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:58.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.804 7fcd15007700 1 --2- 172.21.2.101:0/3079810878 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd100d17f0 0x7fcd101b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:58.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.805 7fcd03fff700 1 -- 172.21.2.101:0/3079810878 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 73518674 0 0) 0x7fcd0c0010b0 con 0x7fcd100d1bf0 2021-02-10T11:32:58.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.805 7fcd03fff700 1 -- 172.21.2.101:0/3079810878 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccfc002fe0 con 0x7fcd100d1bf0 2021-02-10T11:32:58.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.805 7fcd15007700 1 -- 172.21.2.101:0/3079810878 >> v2:172.21.2.108:3300/0 conn(0x7fcd100cff00 msgr2=0x7fcd100ef620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:32:58.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.805 7fcd15007700 1 --2- 172.21.2.101:0/3079810878 >> v2:172.21.2.108:3300/0 conn(0x7fcd100cff00 0x7fcd100ef620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:58.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.805 7fcd15007700 1 -- 172.21.2.101:0/3079810878 >> v1:172.21.2.108:6789/0 conn(0x7fcd100d1bf0 legacy=0x7fcd101b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:58.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.805 7fcd15007700 1 -- 172.21.2.101:0/3079810878 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd100f20b0 con 0x7fcd100d17f0 2021-02-10T11:32:58.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.805 7fcd15007700 1 --2- 172.21.2.101:0/3079810878 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd100d17f0 0x7fcd101b31e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fcd04006650 tx=0x7fcd04009a80).ready entity=mon.2 client_cookie=8ba48dc3921e5fcb server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:58.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.805 7fcd03fff700 1 -- 172.21.2.101:0/3079810878 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd0400c0c0 con 0x7fcd100d17f0 2021-02-10T11:32:58.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.805 7fcd03fff700 1 -- 172.21.2.101:0/3079810878 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0400c340 con 0x7fcd100d17f0 2021-02-10T11:32:58.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.805 7fcd171eb700 1 -- 172.21.2.101:0/3079810878 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd100d17f0 msgr2=0x7fcd101b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:58.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.805 7fcd03fff700 1 -- 172.21.2.101:0/3079810878 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd040009b0 con 0x7fcd100d17f0 2021-02-10T11:32:58.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.805 7fcd171eb700 1 --2- 172.21.2.101:0/3079810878 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd100d17f0 0x7fcd101b31e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fcd04006650 tx=0x7fcd04009a80).stop 2021-02-10T11:32:58.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.806 7fcd171eb700 1 -- 172.21.2.101:0/3079810878 shutdown_connections 2021-02-10T11:32:58.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.806 7fcd171eb700 1 --2- 172.21.2.101:0/3079810878 >> v2:172.21.2.108:3300/0 conn(0x7fcd100cff00 0x7fcd100ef620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:58.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.806 7fcd171eb700 1 --2- 172.21.2.101:0/3079810878 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd100d17f0 0x7fcd101b31e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:58.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.806 7fcd171eb700 1 -- 172.21.2.101:0/3079810878 >> 172.21.2.101:0/3079810878 conn(0x7fcd100fde40 msgr2=0x7fcd100ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:58.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.806 7fcd171eb700 1 -- 172.21.2.101:0/3079810878 shutdown_connections 2021-02-10T11:32:58.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.806 7fcd171eb700 1 -- 172.21.2.101:0/3079810878 wait complete. 2021-02-10T11:32:58.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.820 7fcd171eb700 1 Processor -- start 2021-02-10T11:32:58.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.820 7fcd171eb700 1 -- start start 2021-02-10T11:32:58.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.820 7fcd171eb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd100ae340 0x7fcd100ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:58.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.820 7fcd171eb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd10106a40 0x7fcd10106e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:58.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.820 7fcd171eb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd10187e70 0x7fcd10188270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:58.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.820 7fcd171eb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd101076e0 con 0x7fcd100ae340 2021-02-10T11:32:58.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.820 7fcd171eb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd10095020 con 0x7fcd10106a40 2021-02-10T11:32:58.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.820 7fcd171eb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd10095200 con 0x7fcd10187e70 2021-02-10T11:32:58.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.821 7fcd15007700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd100ae340 0x7fcd100ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:58.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.821 7fcd15007700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd100ae340 0x7fcd100ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51312/0 (socket says 172.21.2.101:51312) 2021-02-10T11:32:58.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.821 7fcd15007700 1 -- 172.21.2.101:0/1034487802 learned_addr learned my addr 172.21.2.101:0/1034487802 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:58.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.821 7fcd15007700 1 -- 172.21.2.101:0/1034487802 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd10187e70 msgr2=0x7fcd10188270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:58.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.821 7fcd15007700 1 --2- 172.21.2.101:0/1034487802 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd10187e70 0x7fcd10188270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:58.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.821 7fcd15007700 1 -- 172.21.2.101:0/1034487802 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd10106a40 msgr2=0x7fcd10106e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:58.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.821 7fcd15007700 1 --2- 172.21.2.101:0/1034487802 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd10106a40 0x7fcd10106e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:58.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.821 7fcd15007700 1 -- 172.21.2.101:0/1034487802 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd101a8a40 con 0x7fcd100ae340 2021-02-10T11:32:58.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.821 7fcd15007700 1 --2- 172.21.2.101:0/1034487802 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd100ae340 0x7fcd100ae740 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7fcd04006f70 tx=0x7fcd0400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:58.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.821 7fcd01ffb700 1 -- 172.21.2.101:0/1034487802 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd040009b0 con 0x7fcd100ae340 2021-02-10T11:32:58.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.821 7fcd01ffb700 1 -- 172.21.2.101:0/1034487802 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0400bde0 con 0x7fcd100ae340 2021-02-10T11:32:58.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.821 7fcd01ffb700 1 -- 172.21.2.101:0/1034487802 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd04002030 con 0x7fcd100ae340 2021-02-10T11:32:58.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.821 7fcd171eb700 1 -- 172.21.2.101:0/1034487802 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd101a8a40 con 0x7fcd100ae340 2021-02-10T11:32:58.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.821 7fcd171eb700 1 -- 172.21.2.101:0/1034487802 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd101a8a40 con 0x7fcd100ae340 2021-02-10T11:32:58.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.822 7fcd01ffb700 1 -- 172.21.2.101:0/1034487802 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fcd04000c90 con 0x7fcd100ae340 2021-02-10T11:32:58.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.822 7fcd01ffb700 1 --2- 172.21.2.101:0/1034487802 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fccf00186c0 0x7fccf001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:58.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.822 7fcd01ffb700 1 -- 172.21.2.101:0/1034487802 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fcd0400bd50 con 0x7fcd100ae340 2021-02-10T11:32:58.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.822 7fcd14806700 1 --2- 172.21.2.101:0/1034487802 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fccf00186c0 0x7fccf001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:58.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.823 7fcd14806700 1 --2- 172.21.2.101:0/1034487802 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fccf00186c0 0x7fccf001abc0 crc :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:58.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.853 7fcd171eb700 1 -- 172.21.2.101:0/1034487802 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd10094fd0 con 0x7fcd100ae340 2021-02-10T11:32:58.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:58.856 7fcd01ffb700 1 -- 172.21.2.101:0/1034487802 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fcd04006bc0 con 0x7fcd100ae340 2021-02-10T11:32:59.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.005 7fcd171eb700 1 -- 172.21.2.101:0/1034487802 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fcd10094fd0 con 0x7fcd100ae340 2021-02-10T11:32:59.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.009 7fcd01ffb700 1 -- 172.21.2.101:0/1034487802 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcd04000c90 con 0x7fcd100ae340 2021-02-10T11:32:59.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.014 7fcd171eb700 1 -- 172.21.2.101:0/1034487802 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fccf00186c0 msgr2=0x7fccf001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:59.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.014 7fcd171eb700 1 --2- 172.21.2.101:0/1034487802 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fccf00186c0 0x7fccf001abc0 crc :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:59.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.014 7fcd171eb700 1 -- 172.21.2.101:0/1034487802 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd100ae340 msgr2=0x7fcd100ae740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:59.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.014 7fcd171eb700 1 --2- 172.21.2.101:0/1034487802 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd100ae340 0x7fcd100ae740 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7fcd04006f70 tx=0x7fcd0400b960).stop 2021-02-10T11:32:59.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.014 7fcd171eb700 1 -- 172.21.2.101:0/1034487802 shutdown_connections 2021-02-10T11:32:59.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.014 7fcd171eb700 1 --2- 172.21.2.101:0/1034487802 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fccf00186c0 0x7fccf001abc0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:59.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.014 7fcd171eb700 1 --2- 172.21.2.101:0/1034487802 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd10187e70 0x7fcd10188270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:59.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.014 7fcd171eb700 1 --2- 172.21.2.101:0/1034487802 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd10106a40 0x7fcd10106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:32:59.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.014 7fcd171eb700 1 --2- 172.21.2.101:0/1034487802 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd100ae340 0x7fcd100ae740 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:59.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.014 7fcd171eb700 1 -- 172.21.2.101:0/1034487802 >> 172.21.2.101:0/1034487802 conn(0x7fcd101ba0f0 msgr2=0x7fcd1018c240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:59.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.015 7fcd171eb700 1 -- 172.21.2.101:0/1034487802 shutdown_connections 2021-02-10T11:32:59.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.015 7fcd171eb700 1 -- 172.21.2.101:0/1034487802 wait complete. 2021-02-10T11:32:59.034 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_authorize_auth_id_not_created_by_mgr_volumes' 2021-02-10T11:32:59.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.433 7f373d691700 1 Processor -- start 2021-02-10T11:32:59.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.433 7f373d691700 1 -- start start 2021-02-10T11:32:59.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.433 7f373d691700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f373807e6a0 0x7f37380d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:59.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.433 7f373d691700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f373807ed80 con 0x7f37380e5470 2021-02-10T11:32:59.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.433 7f373d691700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f373807eaa0 con 0x7f37380e5070 2021-02-10T11:32:59.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.433 7f373d691700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37381b4080 con 0x7f373807e6a0 2021-02-10T11:32:59.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.434 7f3736abe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f37380e5470 0x7f373807f300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51814/0 (socket says 172.21.2.101:51814) 2021-02-10T11:32:59.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.434 7f3736abe700 1 -- 172.21.2.101:0/494542007 learned_addr learned my addr 172.21.2.101:0/494542007 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:32:59.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.434 7f3737ac0700 1 --2- 172.21.2.101:0/494542007 >> v2:172.21.2.108:3300/0 conn(0x7f373807e6a0 0x7f37380d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:59.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.434 7f37362bd700 1 -- 172.21.2.101:0/494542007 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 542677565 0 0) 0x7f37200009b0 con 0x7f37380e5470 2021-02-10T11:32:59.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.434 7f37362bd700 1 -- 172.21.2.101:0/494542007 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3724002fe0 con 0x7f37380e5470 2021-02-10T11:32:59.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.434 7f37362bd700 1 -- 172.21.2.101:0/494542007 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 834322978 0 0) 0x7f3728000c60 con 0x7f37380e5070 2021-02-10T11:32:59.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.434 7f37362bd700 1 -- 172.21.2.101:0/494542007 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3724004370 con 0x7f37380e5070 2021-02-10T11:32:59.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.434 7f3737ac0700 1 -- 172.21.2.101:0/494542007 >> v1:172.21.2.108:6789/0 conn(0x7f37380e5070 legacy=0x7f37381b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:59.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.434 7f3737ac0700 1 -- 172.21.2.101:0/494542007 >> v1:172.21.2.101:6789/0 conn(0x7f37380e5470 legacy=0x7f373807f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:59.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.434 7f3737ac0700 1 -- 172.21.2.101:0/494542007 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37380d1db0 con 0x7f373807e6a0 2021-02-10T11:32:59.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.434 7f3737ac0700 1 --2- 172.21.2.101:0/494542007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f373807e6a0 0x7f37380d1790 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f3730006e50 tx=0x7f3730009df0).ready entity=mon.1 client_cookie=c177ea0e497228fc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:59.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.435 7f37362bd700 1 -- 172.21.2.101:0/494542007 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f373000b510 con 0x7f373807e6a0 2021-02-10T11:32:59.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.435 7f37362bd700 1 -- 172.21.2.101:0/494542007 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3730003030 con 0x7f373807e6a0 2021-02-10T11:32:59.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.435 7f37362bd700 1 -- 172.21.2.101:0/494542007 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f373000b7d0 con 0x7f373807e6a0 2021-02-10T11:32:59.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.435 7f373d691700 1 -- 172.21.2.101:0/494542007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f373807e6a0 msgr2=0x7f37380d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:59.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.435 7f373d691700 1 --2- 172.21.2.101:0/494542007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f373807e6a0 0x7f37380d1790 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f3730006e50 tx=0x7f3730009df0).stop 2021-02-10T11:32:59.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.435 7f373d691700 1 -- 172.21.2.101:0/494542007 shutdown_connections 2021-02-10T11:32:59.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.435 7f373d691700 1 --2- 172.21.2.101:0/494542007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f373807e6a0 0x7f37380d1790 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:59.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.435 7f373d691700 1 -- 172.21.2.101:0/494542007 >> 172.21.2.101:0/494542007 conn(0x7f37380ce690 msgr2=0x7f37380e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:32:59.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.435 7f373d691700 1 -- 172.21.2.101:0/494542007 shutdown_connections 2021-02-10T11:32:59.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.435 7f373d691700 1 -- 172.21.2.101:0/494542007 wait complete. 2021-02-10T11:32:59.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.450 7f373d691700 1 Processor -- start 2021-02-10T11:32:59.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.450 7f373d691700 1 -- start start 2021-02-10T11:32:59.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.451 7f373d691700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37381ae9f0 0x7f37381a8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:59.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.451 7f373d691700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37381aedf0 0x7f37381a91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:59.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.451 7f373d691700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37381a97c0 0x7f37381a9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:59.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.451 7f373d691700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37381aa870 con 0x7f37381ae9f0 2021-02-10T11:32:59.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.451 7f373d691700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37381aaa90 con 0x7f37381aedf0 2021-02-10T11:32:59.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.451 7f373d691700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37381aacb0 con 0x7f37381a97c0 2021-02-10T11:32:59.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.451 7f3736abe700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37381aedf0 0x7f37381a91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:59.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.451 7f37372bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37381ae9f0 0x7f37381a8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:59.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.451 7f37372bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37381ae9f0 0x7f37381a8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51326/0 (socket says 172.21.2.101:51326) 2021-02-10T11:32:59.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.451 7f37372bf700 1 -- 172.21.2.101:0/2205471590 learned_addr learned my addr 172.21.2.101:0/2205471590 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:32:59.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.451 7f3737ac0700 1 --2- 172.21.2.101:0/2205471590 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37381a97c0 0x7f37381a9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:59.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.451 7f3736abe700 1 -- 172.21.2.101:0/2205471590 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37381a97c0 msgr2=0x7f37381a9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:59.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.451 7f3736abe700 1 --2- 172.21.2.101:0/2205471590 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37381a97c0 0x7f37381a9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:59.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.451 7f3736abe700 1 -- 172.21.2.101:0/2205471590 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37381ae9f0 msgr2=0x7f37381a8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:32:59.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.451 7f3736abe700 1 --2- 172.21.2.101:0/2205471590 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37381ae9f0 0x7f37381a8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:32:59.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.451 7f3736abe700 1 -- 172.21.2.101:0/2205471590 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37381aaed0 con 0x7f37381aedf0 2021-02-10T11:32:59.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.451 7f3737ac0700 1 --2- 172.21.2.101:0/2205471590 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37381a97c0 0x7f37381a9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:32:59.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.451 7f37372bf700 1 --2- 172.21.2.101:0/2205471590 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37381ae9f0 0x7f37381a8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:32:59.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.451 7f3736abe700 1 --2- 172.21.2.101:0/2205471590 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37381aedf0 0x7f37381a91a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f3720001c60 tx=0x7f372000a870).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:59.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.452 7f372ffff700 1 -- 172.21.2.101:0/2205471590 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f372000ac60 con 0x7f37381aedf0 2021-02-10T11:32:59.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.452 7f372ffff700 1 -- 172.21.2.101:0/2205471590 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3720000cc0 con 0x7f37381aedf0 2021-02-10T11:32:59.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.452 7f372ffff700 1 -- 172.21.2.101:0/2205471590 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f37200045d0 con 0x7f37381aedf0 2021-02-10T11:32:59.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.452 7f373d691700 1 -- 172.21.2.101:0/2205471590 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37381aaed0 con 0x7f37381aedf0 2021-02-10T11:32:59.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.452 7f373d691700 1 -- 172.21.2.101:0/2205471590 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37381aaed0 con 0x7f37381aedf0 2021-02-10T11:32:59.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.452 7f372ffff700 1 -- 172.21.2.101:0/2205471590 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3720004140 con 0x7f37381aedf0 2021-02-10T11:32:59.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.452 7f372ffff700 1 --2- 172.21.2.101:0/2205471590 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f371c018730 0x7f371c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:32:59.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.452 7f372ffff700 1 -- 172.21.2.101:0/2205471590 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3720032500 con 0x7f37381aedf0 2021-02-10T11:32:59.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.452 7f37372bf700 1 --2- 172.21.2.101:0/2205471590 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f371c018730 0x7f371c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:32:59.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.453 7f37372bf700 1 --2- 172.21.2.101:0/2205471590 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f371c018730 0x7f371c01ac10 crc :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:32:59.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.483 7f373d691700 1 -- 172.21.2.101:0/2205471590 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3738107170 con 0x7f37381aedf0 2021-02-10T11:32:59.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.487 7f372ffff700 1 -- 172.21.2.101:0/2205471590 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3720000cc0 con 0x7f37381aedf0 2021-02-10T11:32:59.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:32:59.622 7f373d691700 1 -- 172.21.2.101:0/2205471590 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_authorize_auth_id_not_created_by_mgr_volumes"]} v 0) v1 -- 0x7f3738107170 con 0x7f37381aedf0 2021-02-10T11:33:00.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.454 7f372ffff700 1 -- 172.21.2.101:0/2205471590 <== mon.1 v2:172.21.2.108:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_authorize_auth_id_not_created_by_mgr_volumes"]}]=0 v282) v1 ==== 166+0+0 (secure 0 0 0) 0x7f3720000ed0 con 0x7f37381aedf0 2021-02-10T11:33:00.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.494 7f373d691700 1 -- 172.21.2.101:0/2205471590 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f371c018730 msgr2=0x7f371c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:00.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.494 7f373d691700 1 --2- 172.21.2.101:0/2205471590 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f371c018730 0x7f371c01ac10 crc :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:00.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.494 7f373d691700 1 -- 172.21.2.101:0/2205471590 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37381aedf0 msgr2=0x7f37381a91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:00.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.494 7f373d691700 1 --2- 172.21.2.101:0/2205471590 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37381aedf0 0x7f37381a91a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f3720001c60 tx=0x7f372000a870).stop 2021-02-10T11:33:00.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.495 7f373d691700 1 -- 172.21.2.101:0/2205471590 shutdown_connections 2021-02-10T11:33:00.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.495 7f373d691700 1 --2- 172.21.2.101:0/2205471590 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f371c018730 0x7f371c01ac10 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:00.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.495 7f373d691700 1 --2- 172.21.2.101:0/2205471590 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37381a97c0 0x7f37381a9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:00.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.495 7f373d691700 1 --2- 172.21.2.101:0/2205471590 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37381aedf0 0x7f37381a91a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:00.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.495 7f373d691700 1 --2- 172.21.2.101:0/2205471590 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37381ae9f0 0x7f37381a8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:00.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.495 7f373d691700 1 -- 172.21.2.101:0/2205471590 >> 172.21.2.101:0/2205471590 conn(0x7f3738107d70 msgr2=0x7f37381af2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:00.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.495 7f373d691700 1 -- 172.21.2.101:0/2205471590 shutdown_connections 2021-02-10T11:33:00.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.495 7f373d691700 1 -- 172.21.2.101:0/2205471590 wait complete. 2021-02-10T11:33:00.514 INFO:tasks.cephfs_test_runner:test_authorize_auth_id_not_created_by_mgr_volumes (tasks.cephfs.test_volumes.TestVolumes) ... ok 2021-02-10T11:33:00.514 INFO:tasks.cephfs_test_runner:Starting test: test_authorize_deauthorize_legacy_subvolume (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:33:00.515 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_authorize_deauthorize_legacy_subvolume' 2021-02-10T11:33:00.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.914 7fe691ff0700 1 Processor -- start 2021-02-10T11:33:00.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.914 7fe691ff0700 1 -- start start 2021-02-10T11:33:00.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.915 7fe691ff0700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe68c0e5070 0x7fe68c1b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:00.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.915 7fe691ff0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe68c1b3e40 0x7fe68c07d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:00.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.915 7fe691ff0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe68c0e5470 0x7fe68c07dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:00.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.915 7fe691ff0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe68c07ed80 con 0x7fe68c1b3e40 2021-02-10T11:33:00.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.915 7fe691ff0700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe68c07e2d0 con 0x7fe68c0e5070 2021-02-10T11:33:00.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.915 7fe691ff0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c07e4f0 con 0x7fe68c0e5470 2021-02-10T11:33:00.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.915 7fe68affd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe68c1b3e40 0x7fe68c07d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:00.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.915 7fe68affd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe68c1b3e40 0x7fe68c07d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51334/0 (socket says 172.21.2.101:51334) 2021-02-10T11:33:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.915 7fe68affd700 1 -- 172.21.2.101:0/4078502082 learned_addr learned my addr 172.21.2.101:0/4078502082 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.915 7fe68b7fe700 1 --2- 172.21.2.101:0/4078502082 >> v2:172.21.2.108:3300/0 conn(0x7fe68c0e5070 0x7fe68c1b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.915 7fe68bfff700 1 --2- 172.21.2.101:0/4078502082 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe68c0e5470 0x7fe68c07dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.915 7fe68affd700 1 -- 172.21.2.101:0/4078502082 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe68c0e5470 msgr2=0x7fe68c07dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.915 7fe68affd700 1 --2- 172.21.2.101:0/4078502082 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe68c0e5470 0x7fe68c07dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.915 7fe68affd700 1 -- 172.21.2.101:0/4078502082 >> v2:172.21.2.108:3300/0 conn(0x7fe68c0e5070 msgr2=0x7fe68c1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.915 7fe68affd700 1 --2- 172.21.2.101:0/4078502082 >> v2:172.21.2.108:3300/0 conn(0x7fe68c0e5070 0x7fe68c1b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.915 7fe68affd700 1 -- 172.21.2.101:0/4078502082 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c07e710 con 0x7fe68c1b3e40 2021-02-10T11:33:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.915 7fe68b7fe700 1 --2- 172.21.2.101:0/4078502082 >> v2:172.21.2.108:3300/0 conn(0x7fe68c0e5070 0x7fe68c1b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:33:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.916 7fe68affd700 1 --2- 172.21.2.101:0/4078502082 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe68c1b3e40 0x7fe68c07d690 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7fe67c006700 tx=0x7fe67c009b80).ready entity=mon.0 client_cookie=16b9e7afb34c61d7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.916 7fe68a7fc700 1 -- 172.21.2.101:0/4078502082 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe67c006340 con 0x7fe68c1b3e40 2021-02-10T11:33:00.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.916 7fe68a7fc700 1 -- 172.21.2.101:0/4078502082 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67c00c0c0 con 0x7fe68c1b3e40 2021-02-10T11:33:00.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.916 7fe68a7fc700 1 -- 172.21.2.101:0/4078502082 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe67c00c5e0 con 0x7fe68c1b3e40 2021-02-10T11:33:00.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.916 7fe691ff0700 1 -- 172.21.2.101:0/4078502082 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe68c1b3e40 msgr2=0x7fe68c07d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:00.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.916 7fe691ff0700 1 --2- 172.21.2.101:0/4078502082 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe68c1b3e40 0x7fe68c07d690 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7fe67c006700 tx=0x7fe67c009b80).stop 2021-02-10T11:33:00.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.916 7fe691ff0700 1 -- 172.21.2.101:0/4078502082 shutdown_connections 2021-02-10T11:33:00.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.916 7fe691ff0700 1 --2- 172.21.2.101:0/4078502082 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe68c0e5470 0x7fe68c07dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:00.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.916 7fe691ff0700 1 --2- 172.21.2.101:0/4078502082 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe68c1b3e40 0x7fe68c07d690 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:00.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.916 7fe691ff0700 1 --2- 172.21.2.101:0/4078502082 >> v2:172.21.2.108:3300/0 conn(0x7fe68c0e5070 0x7fe68c1b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:00.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.916 7fe691ff0700 1 -- 172.21.2.101:0/4078502082 >> 172.21.2.101:0/4078502082 conn(0x7fe68c0ce690 msgr2=0x7fe68c0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:00.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.916 7fe691ff0700 1 -- 172.21.2.101:0/4078502082 shutdown_connections 2021-02-10T11:33:00.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.916 7fe691ff0700 1 -- 172.21.2.101:0/4078502082 wait complete. 2021-02-10T11:33:00.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.931 7fe691ff0700 1 Processor -- start 2021-02-10T11:33:00.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.931 7fe691ff0700 1 -- start start 2021-02-10T11:33:00.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.931 7fe691ff0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe68c1918e0 0x7fe68c189c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:00.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.932 7fe691ff0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe68c18a2a0 0x7fe68c18a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:00.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.932 7fe691ff0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe68c18acc0 0x7fe68c190070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:00.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.932 7fe691ff0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c1a2f00 con 0x7fe68c18a2a0 2021-02-10T11:33:00.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.932 7fe691ff0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c191ce0 con 0x7fe68c1918e0 2021-02-10T11:33:00.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.932 7fe691ff0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c191f00 con 0x7fe68c18acc0 2021-02-10T11:33:00.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.932 7fe68affd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe68c18a2a0 0x7fe68c18a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:00.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.932 7fe68affd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe68c18a2a0 0x7fe68c18a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51338/0 (socket says 172.21.2.101:51338) 2021-02-10T11:33:00.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.932 7fe68affd700 1 -- 172.21.2.101:0/1498688574 learned_addr learned my addr 172.21.2.101:0/1498688574 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:00.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.932 7fe68affd700 1 -- 172.21.2.101:0/1498688574 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe68c18acc0 msgr2=0x7fe68c190070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:00.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.932 7fe68affd700 1 --2- 172.21.2.101:0/1498688574 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe68c18acc0 0x7fe68c190070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:00.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.932 7fe68affd700 1 -- 172.21.2.101:0/1498688574 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe68c1918e0 msgr2=0x7fe68c189c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:00.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.932 7fe68affd700 1 --2- 172.21.2.101:0/1498688574 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe68c1918e0 0x7fe68c189c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:00.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.932 7fe68affd700 1 -- 172.21.2.101:0/1498688574 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c190690 con 0x7fe68c18a2a0 2021-02-10T11:33:00.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.932 7fe68affd700 1 --2- 172.21.2.101:0/1498688574 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe68c18a2a0 0x7fe68c18a6a0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0x7fe67c00b600 tx=0x7fe67c00b9a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:00.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.932 7fe683fff700 1 -- 172.21.2.101:0/1498688574 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe67c002030 con 0x7fe68c18a2a0 2021-02-10T11:33:00.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.933 7fe683fff700 1 -- 172.21.2.101:0/1498688574 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67c002230 con 0x7fe68c18a2a0 2021-02-10T11:33:00.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.933 7fe691ff0700 1 -- 172.21.2.101:0/1498688574 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68c190690 con 0x7fe68c18a2a0 2021-02-10T11:33:00.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.933 7fe691ff0700 1 -- 172.21.2.101:0/1498688574 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe68c190c20 con 0x7fe68c18a2a0 2021-02-10T11:33:00.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.933 7fe683fff700 1 -- 172.21.2.101:0/1498688574 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe67c002750 con 0x7fe68c18a2a0 2021-02-10T11:33:00.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.933 7fe683fff700 1 -- 172.21.2.101:0/1498688574 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe67c000c90 con 0x7fe68c18a2a0 2021-02-10T11:33:00.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.933 7fe683fff700 1 --2- 172.21.2.101:0/1498688574 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe670018700 0x7fe67001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:00.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.933 7fe683fff700 1 -- 172.21.2.101:0/1498688574 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe67c032690 con 0x7fe68c18a2a0 2021-02-10T11:33:00.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.933 7fe68b7fe700 1 --2- 172.21.2.101:0/1498688574 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe670018700 0x7fe67001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:00.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.934 7fe68b7fe700 1 --2- 172.21.2.101:0/1498688574 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe670018700 0x7fe67001abc0 crc :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:00.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.965 7fe691ff0700 1 -- 172.21.2.101:0/1498688574 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe68c189750 con 0x7fe68c18a2a0 2021-02-10T11:33:00.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:00.967 7fe683fff700 1 -- 172.21.2.101:0/1498688574 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe67c0009b0 con 0x7fe68c18a2a0 2021-02-10T11:33:01.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.100 7fe691ff0700 1 -- 172.21.2.101:0/1498688574 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_authorize_deauthorize_legacy_subvolume"]} v 0) v1 -- 0x7fe68c189750 con 0x7fe68c18a2a0 2021-02-10T11:33:01.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.457 7fe683fff700 1 -- 172.21.2.101:0/1498688574 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_authorize_deauthorize_legacy_subvolume"]}]=0 v283) v1 ==== 163+0+0 (secure 0 0 0) 0x7fe67c000c90 con 0x7fe68c18a2a0 2021-02-10T11:33:01.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.467 7fe691ff0700 1 -- 172.21.2.101:0/1498688574 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe670018700 msgr2=0x7fe67001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:01.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.467 7fe691ff0700 1 --2- 172.21.2.101:0/1498688574 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe670018700 0x7fe67001abc0 crc :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:01.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.467 7fe691ff0700 1 -- 172.21.2.101:0/1498688574 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe68c18a2a0 msgr2=0x7fe68c18a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:01.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.467 7fe691ff0700 1 --2- 172.21.2.101:0/1498688574 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe68c18a2a0 0x7fe68c18a6a0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0x7fe67c00b600 tx=0x7fe67c00b9a0).stop 2021-02-10T11:33:01.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.468 7fe691ff0700 1 -- 172.21.2.101:0/1498688574 shutdown_connections 2021-02-10T11:33:01.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.468 7fe691ff0700 1 --2- 172.21.2.101:0/1498688574 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe670018700 0x7fe67001abc0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:01.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.468 7fe691ff0700 1 --2- 172.21.2.101:0/1498688574 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe68c18acc0 0x7fe68c190070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:01.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.468 7fe691ff0700 1 --2- 172.21.2.101:0/1498688574 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe68c18a2a0 0x7fe68c18a6a0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:01.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.468 7fe691ff0700 1 --2- 172.21.2.101:0/1498688574 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe68c1918e0 0x7fe68c189c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:01.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.468 7fe691ff0700 1 -- 172.21.2.101:0/1498688574 >> 172.21.2.101:0/1498688574 conn(0x7fe68c1a31d0 msgr2=0x7fe68c08b830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:01.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.468 7fe691ff0700 1 -- 172.21.2.101:0/1498688574 shutdown_connections 2021-02-10T11:33:01.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.468 7fe691ff0700 1 -- 172.21.2.101:0/1498688574 wait complete. 2021-02-10T11:33:01.487 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:33:01.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.889 7fa847f20700 1 Processor -- start 2021-02-10T11:33:01.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.889 7fa847f20700 1 -- start start 2021-02-10T11:33:01.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.889 7fa847f20700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8401b0470 0x7fa8401b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:01.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.889 7fa847f20700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa8400ef610 0x7fa8400efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:01.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.889 7fa847f20700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8400f0b50 con 0x7fa8400f0030 2021-02-10T11:33:01.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.889 7fa847f20700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8400e52e0 con 0x7fa8400ef610 2021-02-10T11:33:01.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.889 7fa847f20700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8400e5490 con 0x7fa8401b0470 2021-02-10T11:33:01.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.890 7fa84653d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa8400f0030 0x7fa8400f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51840/0 (socket says 172.21.2.101:51840) 2021-02-10T11:33:01.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.890 7fa84653d700 1 -- 172.21.2.101:0/878272487 learned_addr learned my addr 172.21.2.101:0/878272487 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:01.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.890 7fa84553b700 1 --2- 172.21.2.101:0/878272487 >> v2:172.21.2.101:3300/0 conn(0x7fa8400ef610 0x7fa8400efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:01.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.890 7fa844d3a700 1 -- 172.21.2.101:0/878272487 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 680078398 0 0) 0x7fa838001500 con 0x7fa8400f0030 2021-02-10T11:33:01.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.890 7fa844d3a700 1 -- 172.21.2.101:0/878272487 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa82c002fe0 con 0x7fa8400f0030 2021-02-10T11:33:01.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.890 7fa84553b700 1 -- 172.21.2.101:0/878272487 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8401b0470 msgr2=0x7fa8401b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:01.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.890 7fa84553b700 1 --2- 172.21.2.101:0/878272487 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8401b0470 0x7fa8401b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:01.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.890 7fa84553b700 1 -- 172.21.2.101:0/878272487 >> v1:172.21.2.101:6789/0 conn(0x7fa8400f0030 legacy=0x7fa8400f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:01.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.890 7fa84553b700 1 -- 172.21.2.101:0/878272487 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8400f0430 con 0x7fa8400ef610 2021-02-10T11:33:01.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.890 7fa84553b700 1 --2- 172.21.2.101:0/878272487 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8400ef610 0x7fa8400efa10 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7fa834006650 tx=0x7fa834009a00).ready entity=mon.0 client_cookie=8782713e6baa0e1c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:01.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.891 7fa844d3a700 1 -- 172.21.2.101:0/878272487 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa8340009b0 con 0x7fa8400ef610 2021-02-10T11:33:01.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.891 7fa844d3a700 1 -- 172.21.2.101:0/878272487 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83400c0c0 con 0x7fa8400ef610 2021-02-10T11:33:01.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.891 7fa844d3a700 1 -- 172.21.2.101:0/878272487 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa83400c5e0 con 0x7fa8400ef610 2021-02-10T11:33:01.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.891 7fa847f20700 1 -- 172.21.2.101:0/878272487 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8400ef610 msgr2=0x7fa8400efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:01.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.891 7fa847f20700 1 --2- 172.21.2.101:0/878272487 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8400ef610 0x7fa8400efa10 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7fa834006650 tx=0x7fa834009a00).stop 2021-02-10T11:33:01.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.891 7fa847f20700 1 -- 172.21.2.101:0/878272487 shutdown_connections 2021-02-10T11:33:01.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.891 7fa847f20700 1 --2- 172.21.2.101:0/878272487 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8400ef610 0x7fa8400efa10 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:01.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.891 7fa847f20700 1 --2- 172.21.2.101:0/878272487 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8401b0470 0x7fa8401b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:01.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.891 7fa847f20700 1 -- 172.21.2.101:0/878272487 >> 172.21.2.101:0/878272487 conn(0x7fa8400ce3e0 msgr2=0x7fa8400e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:01.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.891 7fa847f20700 1 -- 172.21.2.101:0/878272487 shutdown_connections 2021-02-10T11:33:01.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.891 7fa847f20700 1 -- 172.21.2.101:0/878272487 wait complete. 2021-02-10T11:33:01.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.906 7fa847f20700 1 Processor -- start 2021-02-10T11:33:01.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.906 7fa847f20700 1 -- start start 2021-02-10T11:33:01.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.907 7fa847f20700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa84008ece0 0x7fa8400a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:01.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.907 7fa847f20700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8400a6860 0x7fa8400a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:01.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.907 7fa847f20700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8400a7280 0x7fa8400a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:01.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.907 7fa847f20700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8400a5cc0 con 0x7fa8400a6860 2021-02-10T11:33:01.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.907 7fa847f20700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84008f0e0 con 0x7fa84008ece0 2021-02-10T11:33:01.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.907 7fa847f20700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84008f2c0 con 0x7fa8400a7280 2021-02-10T11:33:01.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.907 7fa84553b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8400a6860 0x7fa8400a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:01.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.907 7fa84553b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8400a6860 0x7fa8400a6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51348/0 (socket says 172.21.2.101:51348) 2021-02-10T11:33:01.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.907 7fa84553b700 1 -- 172.21.2.101:0/2010460501 learned_addr learned my addr 172.21.2.101:0/2010460501 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:01.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.907 7fa84653d700 1 --2- 172.21.2.101:0/2010460501 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8400a7280 0x7fa8400a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:01.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.907 7fa845d3c700 1 --2- 172.21.2.101:0/2010460501 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa84008ece0 0x7fa8400a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:01.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.907 7fa84553b700 1 -- 172.21.2.101:0/2010460501 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8400a7280 msgr2=0x7fa8400a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:01.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.907 7fa84553b700 1 --2- 172.21.2.101:0/2010460501 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8400a7280 0x7fa8400a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:01.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.907 7fa84553b700 1 -- 172.21.2.101:0/2010460501 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa84008ece0 msgr2=0x7fa8400a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:01.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.907 7fa84553b700 1 --2- 172.21.2.101:0/2010460501 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa84008ece0 0x7fa8400a6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:01.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.907 7fa84553b700 1 -- 172.21.2.101:0/2010460501 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8400a4bb0 con 0x7fa8400a6860 2021-02-10T11:33:01.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.907 7fa845d3c700 1 --2- 172.21.2.101:0/2010460501 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa84008ece0 0x7fa8400a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:33:01.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.908 7fa84553b700 1 --2- 172.21.2.101:0/2010460501 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8400a6860 0x7fa8400a6c60 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7fa83400b580 tx=0x7fa83400b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:01.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.908 7fa8327fc700 1 -- 172.21.2.101:0/2010460501 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa834006860 con 0x7fa8400a6860 2021-02-10T11:33:01.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.908 7fa8327fc700 1 -- 172.21.2.101:0/2010460501 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa834002030 con 0x7fa8400a6860 2021-02-10T11:33:01.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.908 7fa8327fc700 1 -- 172.21.2.101:0/2010460501 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa834002550 con 0x7fa8400a6860 2021-02-10T11:33:01.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.908 7fa847f20700 1 -- 172.21.2.101:0/2010460501 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8400a4bb0 con 0x7fa8400a6860 2021-02-10T11:33:01.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.908 7fa847f20700 1 -- 172.21.2.101:0/2010460501 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8400a4bb0 con 0x7fa8400a6860 2021-02-10T11:33:01.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.908 7fa8327fc700 1 -- 172.21.2.101:0/2010460501 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa834000c90 con 0x7fa8400a6860 2021-02-10T11:33:01.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.909 7fa8327fc700 1 --2- 172.21.2.101:0/2010460501 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8280186c0 0x7fa82801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:01.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.909 7fa8327fc700 1 -- 172.21.2.101:0/2010460501 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa834032560 con 0x7fa8400a6860 2021-02-10T11:33:01.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.909 7fa845d3c700 1 --2- 172.21.2.101:0/2010460501 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8280186c0 0x7fa82801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:01.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.909 7fa845d3c700 1 --2- 172.21.2.101:0/2010460501 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8280186c0 0x7fa82801abc0 crc :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:01.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.939 7fa847f20700 1 -- 172.21.2.101:0/2010460501 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84019d1a0 con 0x7fa8400a6860 2021-02-10T11:33:01.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:01.942 7fa8327fc700 1 -- 172.21.2.101:0/2010460501 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa834006860 con 0x7fa8400a6860 2021-02-10T11:33:02.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:02.089 7fa847f20700 1 -- 172.21.2.101:0/2010460501 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa84019d1a0 con 0x7fa8400a6860 2021-02-10T11:33:02.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:02.093 7fa8327fc700 1 -- 172.21.2.101:0/2010460501 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v8)=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa834000c90 con 0x7fa8400a6860 2021-02-10T11:33:02.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:02.098 7fa847f20700 1 -- 172.21.2.101:0/2010460501 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8280186c0 msgr2=0x7fa82801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:02.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:02.098 7fa847f20700 1 --2- 172.21.2.101:0/2010460501 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8280186c0 0x7fa82801abc0 crc :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:02.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:02.098 7fa847f20700 1 -- 172.21.2.101:0/2010460501 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8400a6860 msgr2=0x7fa8400a6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:02.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:02.098 7fa847f20700 1 --2- 172.21.2.101:0/2010460501 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8400a6860 0x7fa8400a6c60 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7fa83400b580 tx=0x7fa83400b920).stop 2021-02-10T11:33:02.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:02.098 7fa847f20700 1 -- 172.21.2.101:0/2010460501 shutdown_connections 2021-02-10T11:33:02.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:02.098 7fa847f20700 1 --2- 172.21.2.101:0/2010460501 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8280186c0 0x7fa82801abc0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:02.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:02.098 7fa847f20700 1 --2- 172.21.2.101:0/2010460501 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8400a7280 0x7fa8400a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:02.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:02.098 7fa847f20700 1 --2- 172.21.2.101:0/2010460501 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8400a6860 0x7fa8400a6c60 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:02.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:02.098 7fa847f20700 1 --2- 172.21.2.101:0/2010460501 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa84008ece0 0x7fa8400a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:02.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:02.098 7fa847f20700 1 -- 172.21.2.101:0/2010460501 >> 172.21.2.101:0/2010460501 conn(0x7fa84008ffe0 msgr2=0x7fa8400bf050 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:02.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:02.098 7fa847f20700 1 -- 172.21.2.101:0/2010460501 shutdown_connections 2021-02-10T11:33:02.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:02.098 7fa847f20700 1 -- 172.21.2.101:0/2010460501 wait complete. 2021-02-10T11:33:02.118 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:33:02.118 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:33:02.463 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:33:02.809 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:33:02.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:02.809 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:33:02.809 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:33:03.145 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2021-02-10T11:33:03.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:33:03.146 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-02-10T11:33:03.146 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:03.146 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:33:03.146 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:33:03.481 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:33:03.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:33:03.481 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:33:03.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:03.481 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:33:03.482 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:33:03.815 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:33:03.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:33:03.816 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:33:03.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:03.816 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:33:03.816 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:33:04.149 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:33:04.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:33:04.150 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:33:04.150 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json-pretty 2021-02-10T11:33:04.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.552 7f09265ab700 1 Processor -- start 2021-02-10T11:33:04.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.552 7f09265ab700 1 -- start start 2021-02-10T11:33:04.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.552 7f09265ab700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09200e5470 0x7f092007e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:04.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.553 7f09265ab700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f092007f300 0x7f092007f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:04.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.553 7f09265ab700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092007ed80 con 0x7f09200e5070 2021-02-10T11:33:04.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.553 7f09265ab700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09201b4010 con 0x7f092007f300 2021-02-10T11:33:04.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.553 7f09265ab700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09200d1790 con 0x7f09200e5470 2021-02-10T11:33:04.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.553 7f091ffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f09200e5070 0x7f09201b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51850/0 (socket says 172.21.2.101:51850) 2021-02-10T11:33:04.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.553 7f091ffff700 1 -- 172.21.2.101:0/1854097119 learned_addr learned my addr 172.21.2.101:0/1854097119 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:04.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.553 7f091f7fe700 1 --2- 172.21.2.101:0/1854097119 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09200e5470 0x7f092007e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:04.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.553 7f0924bc8700 1 --2- 172.21.2.101:0/1854097119 >> v2:172.21.2.108:3300/0 conn(0x7f092007f300 0x7f092007f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:04.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.553 7f091effd700 1 -- 172.21.2.101:0/1854097119 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2496719706 0 0) 0x7f0910001180 con 0x7f09200e5070 2021-02-10T11:33:04.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.553 7f091effd700 1 -- 172.21.2.101:0/1854097119 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0908002fe0 con 0x7f09200e5070 2021-02-10T11:33:04.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.553 7f091f7fe700 1 -- 172.21.2.101:0/1854097119 >> v2:172.21.2.108:3300/0 conn(0x7f092007f300 msgr2=0x7f092007f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:04.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.553 7f091f7fe700 1 --2- 172.21.2.101:0/1854097119 >> v2:172.21.2.108:3300/0 conn(0x7f092007f300 0x7f092007f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:04.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.553 7f091f7fe700 1 -- 172.21.2.101:0/1854097119 >> v1:172.21.2.101:6789/0 conn(0x7f09200e5070 legacy=0x7f09201b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:04.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.553 7f091f7fe700 1 -- 172.21.2.101:0/1854097119 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09200d1980 con 0x7f09200e5470 2021-02-10T11:33:04.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.554 7f091f7fe700 1 --2- 172.21.2.101:0/1854097119 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09200e5470 0x7f092007e6a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f0914006650 tx=0x7f0914009a80).ready entity=mon.2 client_cookie=b3bc03522a0eb141 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:04.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.554 7f091effd700 1 -- 172.21.2.101:0/1854097119 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f091400c0c0 con 0x7f09200e5470 2021-02-10T11:33:04.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.554 7f091effd700 1 -- 172.21.2.101:0/1854097119 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091400c340 con 0x7f09200e5470 2021-02-10T11:33:04.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.554 7f091effd700 1 -- 172.21.2.101:0/1854097119 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f091400c860 con 0x7f09200e5470 2021-02-10T11:33:04.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.554 7f09265ab700 1 -- 172.21.2.101:0/1854097119 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09200e5470 msgr2=0x7f092007e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:04.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.554 7f09265ab700 1 --2- 172.21.2.101:0/1854097119 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09200e5470 0x7f092007e6a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f0914006650 tx=0x7f0914009a80).stop 2021-02-10T11:33:04.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.555 7f09265ab700 1 -- 172.21.2.101:0/1854097119 shutdown_connections 2021-02-10T11:33:04.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.555 7f09265ab700 1 --2- 172.21.2.101:0/1854097119 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09200e5470 0x7f092007e6a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:04.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.555 7f09265ab700 1 --2- 172.21.2.101:0/1854097119 >> v2:172.21.2.108:3300/0 conn(0x7f092007f300 0x7f092007f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:04.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.555 7f09265ab700 1 -- 172.21.2.101:0/1854097119 >> 172.21.2.101:0/1854097119 conn(0x7f09200ce690 msgr2=0x7f09200e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:04.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.555 7f09265ab700 1 -- 172.21.2.101:0/1854097119 shutdown_connections 2021-02-10T11:33:04.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.555 7f09265ab700 1 -- 172.21.2.101:0/1854097119 wait complete. 2021-02-10T11:33:04.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.569 7f09265ab700 1 Processor -- start 2021-02-10T11:33:04.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.569 7f09265ab700 1 -- start start 2021-02-10T11:33:04.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.570 7f09265ab700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0920191bb0 0x7f09200a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:04.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.570 7f09265ab700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09200a29a0 0x7f09200a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:04.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.570 7f09265ab700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09200a33c0 0x7f09200a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:04.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.570 7f09265ab700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09200adf90 con 0x7f09200a29a0 2021-02-10T11:33:04.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.570 7f09265ab700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09200a1e00 con 0x7f0920191bb0 2021-02-10T11:33:04.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.570 7f09265ab700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09200a09f0 con 0x7f09200a33c0 2021-02-10T11:33:04.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.570 7f091f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09200a29a0 0x7f09200a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:04.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.570 7f091ffff700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0920191bb0 0x7f09200a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:04.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.570 7f091f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09200a29a0 0x7f09200a2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51364/0 (socket says 172.21.2.101:51364) 2021-02-10T11:33:04.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.570 7f091f7fe700 1 -- 172.21.2.101:0/1393283161 learned_addr learned my addr 172.21.2.101:0/1393283161 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:04.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.570 7f091f7fe700 1 -- 172.21.2.101:0/1393283161 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09200a33c0 msgr2=0x7f09200a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:04.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.570 7f0924bc8700 1 --2- 172.21.2.101:0/1393283161 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09200a33c0 0x7f09200a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:04.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.570 7f091f7fe700 1 --2- 172.21.2.101:0/1393283161 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09200a33c0 0x7f09200a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:04.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.570 7f091f7fe700 1 -- 172.21.2.101:0/1393283161 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0920191bb0 msgr2=0x7f09200a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:04.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.570 7f091f7fe700 1 --2- 172.21.2.101:0/1393283161 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0920191bb0 0x7f09200a2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:04.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.570 7f091f7fe700 1 -- 172.21.2.101:0/1393283161 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09200a0bd0 con 0x7f09200a29a0 2021-02-10T11:33:04.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.570 7f0924bc8700 1 --2- 172.21.2.101:0/1393283161 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09200a33c0 0x7f09200a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:33:04.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.570 7f091f7fe700 1 --2- 172.21.2.101:0/1393283161 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09200a29a0 0x7f09200a2da0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7f0914009630 tx=0x7f091400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:04.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.571 7f091cff9700 1 -- 172.21.2.101:0/1393283161 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f09140093a0 con 0x7f09200a29a0 2021-02-10T11:33:04.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.571 7f091cff9700 1 -- 172.21.2.101:0/1393283161 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091400bd70 con 0x7f09200a29a0 2021-02-10T11:33:04.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.571 7f091cff9700 1 -- 172.21.2.101:0/1393283161 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f09140054d0 con 0x7f09200a29a0 2021-02-10T11:33:04.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.571 7f09265ab700 1 -- 172.21.2.101:0/1393283161 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09200a0bd0 con 0x7f09200a29a0 2021-02-10T11:33:04.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.571 7f09265ab700 1 -- 172.21.2.101:0/1393283161 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09200a0bd0 con 0x7f09200a29a0 2021-02-10T11:33:04.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.571 7f091cff9700 1 -- 172.21.2.101:0/1393283161 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0914000c90 con 0x7f09200a29a0 2021-02-10T11:33:04.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.571 7f091cff9700 1 --2- 172.21.2.101:0/1393283161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09000186c0 0x7f090001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:04.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.571 7f091cff9700 1 -- 172.21.2.101:0/1393283161 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0914032560 con 0x7f09200a29a0 2021-02-10T11:33:04.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.571 7f091ffff700 1 --2- 172.21.2.101:0/1393283161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09000186c0 0x7f090001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:04.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.572 7f091ffff700 1 --2- 172.21.2.101:0/1393283161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09000186c0 0x7f090001abc0 crc :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:04.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.602 7f09265ab700 1 -- 172.21.2.101:0/1393283161 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09200a1e00 con 0x7f09200a29a0 2021-02-10T11:33:04.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.605 7f091cff9700 1 -- 172.21.2.101:0/1393283161 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0914030d10 con 0x7f09200a29a0 2021-02-10T11:33:04.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.734 7f09265ab700 1 -- 172.21.2.101:0/1393283161 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f09200a1e00 con 0x7f09200a29a0 2021-02-10T11:33:04.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.735 7f091cff9700 1 -- 172.21.2.101:0/1393283161 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v47) v1 ==== 81+0+30360 (secure 0 0 0) 0x7f0914000c90 con 0x7f09200a29a0 2021-02-10T11:33:04.746 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:04.746 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:33:04.747 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 47, 2021-02-10T11:33:04.747 INFO:teuthology.orchestra.run.gibba001.stdout: "fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:33:04.747 INFO:teuthology.orchestra.run.gibba001.stdout: "created": "2021-02-10 11:12:36.788366", 2021-02-10T11:33:04.747 INFO:teuthology.orchestra.run.gibba001.stdout: "modified": "2021-02-10 11:32:25.606853", 2021-02-10T11:33:04.747 INFO:teuthology.orchestra.run.gibba001.stdout: "last_up_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:33:04.747 INFO:teuthology.orchestra.run.gibba001.stdout: "last_in_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:33:04.747 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-02-10T11:33:04.747 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_num": 5799936, 2021-02-10T11:33:04.747 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_set": [ 2021-02-10T11:33:04.747 INFO:teuthology.orchestra.run.gibba001.stdout: "pglog_hardlimit", 2021-02-10T11:33:04.747 INFO:teuthology.orchestra.run.gibba001.stdout: "purged_snapdirs", 2021-02-10T11:33:04.747 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_deletes", 2021-02-10T11:33:04.747 INFO:teuthology.orchestra.run.gibba001.stdout: "sortbitwise" 2021-02-10T11:33:04.748 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:33:04.748 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_version": 4, 2021-02-10T11:33:04.748 INFO:teuthology.orchestra.run.gibba001.stdout: "full_ratio": 0.94999998807907104, 2021-02-10T11:33:04.748 INFO:teuthology.orchestra.run.gibba001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-02-10T11:33:04.748 INFO:teuthology.orchestra.run.gibba001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-02-10T11:33:04.748 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_snapshot": "", 2021-02-10T11:33:04.748 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_max": 9, 2021-02-10T11:33:04.748 INFO:teuthology.orchestra.run.gibba001.stdout: "max_osd": 8, 2021-02-10T11:33:04.748 INFO:teuthology.orchestra.run.gibba001.stdout: "require_min_compat_client": "jewel", 2021-02-10T11:33:04.748 INFO:teuthology.orchestra.run.gibba001.stdout: "min_compat_client": "jewel", 2021-02-10T11:33:04.749 INFO:teuthology.orchestra.run.gibba001.stdout: "require_osd_release": "nautilus", 2021-02-10T11:33:04.749 INFO:teuthology.orchestra.run.gibba001.stdout: "pools": [ 2021-02-10T11:33:04.749 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.749 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 1, 2021-02-10T11:33:04.749 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "rbd", 2021-02-10T11:33:04.749 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:13:08.037472", 2021-02-10T11:33:04.749 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:33:04.749 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:33:04.749 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:33:04.749 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:33:04.749 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:33:04.749 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:33:04.749 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:33:04.750 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:33:04.750 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:33:04.750 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:33:04.750 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:33:04.750 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:33:04.750 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:33:04.750 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:33:04.750 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:33:04.750 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:33:04.750 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:33:04.750 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:33:04.750 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:33:04.750 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:33:04.750 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.751 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "16", 2021-02-10T11:33:04.751 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:33:04.751 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:33:04.751 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:33:04.751 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:33:04.751 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:33:04.751 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:33:04.751 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:33:04.751 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:33:04.751 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:33:04.751 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:33:04.751 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:33:04.751 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:33:04.751 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:33:04.752 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:33:04.752 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:33:04.752 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:33:04.752 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:33:04.752 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:33:04.752 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:33:04.752 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:33:04.752 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:33:04.752 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:33:04.752 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:33:04.752 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:33:04.752 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:33:04.752 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:33:04.752 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.753 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:33:04.753 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:33:04.753 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:33:04.753 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:33:04.753 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:33:04.753 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:33:04.753 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:33:04.753 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:33:04.753 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:33:04.753 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:33:04.753 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:33:04.753 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:33:04.753 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:33:04.753 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd": {} 2021-02-10T11:33:04.754 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.754 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.754 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.754 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 8, 2021-02-10T11:33:04.754 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_metadata", 2021-02-10T11:33:04.754 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:32:22.580828", 2021-02-10T11:33:04.754 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:33:04.754 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:33:04.754 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:33:04.754 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:33:04.754 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:33:04.754 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:33:04.754 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:33:04.754 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:33:04.755 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:33:04.755 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:33:04.755 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:33:04.755 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:33:04.755 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:33:04.755 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:33:04.755 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:33:04.755 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:33:04.755 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:33:04.755 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:33:04.755 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:33:04.755 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:33:04.755 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.755 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "46", 2021-02-10T11:33:04.756 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:33:04.756 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:33:04.756 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:33:04.756 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:33:04.756 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:33:04.756 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:33:04.756 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:33:04.756 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:33:04.756 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:33:04.756 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:33:04.756 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:33:04.756 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:33:04.756 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:33:04.756 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:33:04.757 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:33:04.757 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:33:04.757 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:33:04.757 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:33:04.757 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:33:04.757 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:33:04.757 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:33:04.757 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:33:04.757 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:33:04.757 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:33:04.757 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:33:04.757 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:33:04.757 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.757 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:33:04.758 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:33:04.758 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:33:04.758 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:33:04.758 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:33:04.758 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:33:04.758 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:33:04.758 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:33:04.758 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:33:04.758 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:33:04.758 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:33:04.758 INFO:teuthology.orchestra.run.gibba001.stdout: "options": { 2021-02-10T11:33:04.758 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_bias": 4, 2021-02-10T11:33:04.759 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_min": 16, 2021-02-10T11:33:04.759 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_priority": 5 2021-02-10T11:33:04.759 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.759 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:33:04.759 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:33:04.759 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": "cephfs" 2021-02-10T11:33:04.759 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.759 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.759 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.759 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.759 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 9, 2021-02-10T11:33:04.760 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_data", 2021-02-10T11:33:04.760 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:32:23.597744", 2021-02-10T11:33:04.760 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:33:04.760 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:33:04.760 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:33:04.760 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:33:04.760 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:33:04.760 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:33:04.760 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:33:04.760 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:33:04.761 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:33:04.761 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:33:04.761 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:33:04.761 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:33:04.761 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:33:04.761 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:33:04.761 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:33:04.761 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:33:04.761 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:33:04.761 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:33:04.761 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:33:04.761 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:33:04.761 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.762 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "47", 2021-02-10T11:33:04.762 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:33:04.762 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:33:04.762 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:33:04.762 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:33:04.762 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:33:04.762 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:33:04.762 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:33:04.762 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:33:04.762 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:33:04.763 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:33:04.763 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:33:04.763 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:33:04.763 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:33:04.763 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:33:04.763 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:33:04.763 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:33:04.763 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:33:04.763 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:33:04.763 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:33:04.763 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:33:04.763 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:33:04.763 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:33:04.763 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:33:04.764 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:33:04.764 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:33:04.764 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:33:04.764 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.764 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:33:04.764 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:33:04.764 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:33:04.764 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:33:04.764 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:33:04.765 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:33:04.765 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:33:04.765 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:33:04.765 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:33:04.765 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:33:04.765 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:33:04.765 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:33:04.765 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:33:04.765 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:33:04.765 INFO:teuthology.orchestra.run.gibba001.stdout: "data": "cephfs" 2021-02-10T11:33:04.765 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.766 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.766 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.766 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:33:04.766 INFO:teuthology.orchestra.run.gibba001.stdout: "osds": [ 2021-02-10T11:33:04.766 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.766 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:33:04.766 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5", 2021-02-10T11:33:04.766 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:33:04.766 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:33:04.766 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:33:04.767 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:33:04.767 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:33:04.767 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:33:04.767 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:33:04.767 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 45, 2021-02-10T11:33:04.767 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:33:04.767 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:33:04.767 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:33:04.767 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.767 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.767 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.767 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6802", 2021-02-10T11:33:04.767 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:33:04.768 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.768 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.768 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.768 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6805", 2021-02-10T11:33:04.768 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:33:04.768 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.768 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.768 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.768 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:33:04.768 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.768 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.768 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.768 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6807", 2021-02-10T11:33:04.769 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:33:04.769 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.769 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.769 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.769 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6809", 2021-02-10T11:33:04.769 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:33:04.769 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.769 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.769 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.769 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:33:04.769 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.769 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.769 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.770 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6817", 2021-02-10T11:33:04.770 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:33:04.770 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.770 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.770 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.770 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6820", 2021-02-10T11:33:04.770 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:33:04.770 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.770 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.770 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.770 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:33:04.770 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.770 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.771 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.771 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6812", 2021-02-10T11:33:04.771 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:33:04.771 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.771 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.771 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.771 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6814", 2021-02-10T11:33:04.771 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:33:04.771 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.771 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.771 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.771 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6805/25093", 2021-02-10T11:33:04.771 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6809/25093", 2021-02-10T11:33:04.771 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6820/25093", 2021-02-10T11:33:04.772 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6814/25093", 2021-02-10T11:33:04.772 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:33:04.772 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:33:04.772 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:33:04.772 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.772 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.772 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.772 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:33:04.772 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "ddc04ce3-b313-4707-8270-585391e38ecc", 2021-02-10T11:33:04.772 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:33:04.772 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:33:04.772 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:33:04.772 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:33:04.773 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:33:04.773 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:33:04.773 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:33:04.773 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 45, 2021-02-10T11:33:04.773 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:33:04.773 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:33:04.773 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:33:04.773 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.773 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.773 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.773 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6803", 2021-02-10T11:33:04.773 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:33:04.774 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.774 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.774 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.774 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6804", 2021-02-10T11:33:04.774 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:33:04.774 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.774 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.774 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.774 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:33:04.775 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.775 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.775 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.775 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6806", 2021-02-10T11:33:04.775 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:33:04.775 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.775 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.775 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.775 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6808", 2021-02-10T11:33:04.775 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:33:04.775 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.776 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.776 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.776 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:33:04.776 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.776 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.776 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.776 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6818", 2021-02-10T11:33:04.776 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:33:04.776 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.776 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.776 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.776 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6821", 2021-02-10T11:33:04.776 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:33:04.777 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.777 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.777 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.777 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:33:04.777 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.777 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.777 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.777 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6810", 2021-02-10T11:33:04.777 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:33:04.777 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.777 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.777 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.778 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6815", 2021-02-10T11:33:04.778 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:33:04.778 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.778 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.778 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.778 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6804/25092", 2021-02-10T11:33:04.778 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6808/25092", 2021-02-10T11:33:04.778 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6821/25092", 2021-02-10T11:33:04.778 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6815/25092", 2021-02-10T11:33:04.778 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:33:04.778 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:33:04.778 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:33:04.778 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.779 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.779 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.779 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:33:04.779 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3", 2021-02-10T11:33:04.779 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:33:04.779 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:33:04.779 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:33:04.779 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:33:04.779 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:33:04.779 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:33:04.779 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:33:04.779 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 27, 2021-02-10T11:33:04.780 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:33:04.780 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:33:04.780 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:33:04.780 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.780 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.780 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.780 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6811", 2021-02-10T11:33:04.780 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:33:04.780 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.780 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.780 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.781 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6813", 2021-02-10T11:33:04.781 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:33:04.781 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.781 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.781 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.781 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:33:04.781 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.781 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.781 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.781 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6816", 2021-02-10T11:33:04.781 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:33:04.781 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.781 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.782 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.782 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6819", 2021-02-10T11:33:04.782 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:33:04.782 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.782 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.782 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.782 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:33:04.782 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.782 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.782 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.782 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6825", 2021-02-10T11:33:04.782 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:33:04.782 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.783 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.783 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.783 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6827", 2021-02-10T11:33:04.783 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:33:04.783 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.783 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.783 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.783 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:33:04.783 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.783 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.783 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.783 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6822", 2021-02-10T11:33:04.783 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:33:04.784 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.784 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.784 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.784 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6823", 2021-02-10T11:33:04.784 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:33:04.784 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.784 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.784 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.784 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6813/25094", 2021-02-10T11:33:04.784 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6819/25094", 2021-02-10T11:33:04.784 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6827/25094", 2021-02-10T11:33:04.784 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6823/25094", 2021-02-10T11:33:04.784 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:33:04.785 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:33:04.785 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:33:04.785 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.785 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.785 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.785 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:33:04.785 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "0d9ef9b1-98aa-4d37-aa16-6305c7d46d52", 2021-02-10T11:33:04.785 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:33:04.785 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:33:04.785 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:33:04.785 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:33:04.786 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:33:04.786 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:33:04.786 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:33:04.786 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 45, 2021-02-10T11:33:04.786 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:33:04.786 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:33:04.786 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:33:04.786 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.786 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.786 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.786 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6824", 2021-02-10T11:33:04.786 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:33:04.787 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.787 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.787 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.787 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6826", 2021-02-10T11:33:04.787 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:33:04.787 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.787 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.787 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.787 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:33:04.787 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.787 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.787 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.787 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6828", 2021-02-10T11:33:04.788 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:33:04.788 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.788 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.788 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.788 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6829", 2021-02-10T11:33:04.788 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:33:04.788 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.788 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.788 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.788 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:33:04.788 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.788 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.789 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.789 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6832", 2021-02-10T11:33:04.789 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:33:04.789 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.789 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.789 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.789 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6833", 2021-02-10T11:33:04.789 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:33:04.789 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.789 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.790 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.790 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:33:04.790 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.790 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.790 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.790 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6830", 2021-02-10T11:33:04.790 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:33:04.790 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.790 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.790 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.790 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6831", 2021-02-10T11:33:04.790 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:33:04.790 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.791 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.791 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.791 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6826/25095", 2021-02-10T11:33:04.791 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6829/25095", 2021-02-10T11:33:04.791 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6833/25095", 2021-02-10T11:33:04.791 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6831/25095", 2021-02-10T11:33:04.791 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:33:04.791 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:33:04.791 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:33:04.791 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.791 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.791 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.791 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:33:04.791 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "36742c21-bfeb-4e11-bce9-8e745378b605", 2021-02-10T11:33:04.792 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:33:04.792 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:33:04.792 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:33:04.792 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:33:04.792 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:33:04.792 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:33:04.792 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:33:04.792 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 43, 2021-02-10T11:33:04.792 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:33:04.792 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:33:04.792 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:33:04.792 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.792 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.793 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.793 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6816", 2021-02-10T11:33:04.793 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:33:04.793 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.793 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.793 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.793 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6818", 2021-02-10T11:33:04.793 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:33:04.793 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.793 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.793 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.793 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:33:04.794 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.794 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.794 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.794 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6820", 2021-02-10T11:33:04.794 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:33:04.794 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.794 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.794 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.794 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6822", 2021-02-10T11:33:04.794 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:33:04.795 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.795 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.795 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.795 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:33:04.795 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.795 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.795 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.795 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6828", 2021-02-10T11:33:04.795 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:33:04.795 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.795 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.795 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.795 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6830", 2021-02-10T11:33:04.796 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:33:04.796 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.796 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.796 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.796 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:33:04.796 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.796 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.796 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.796 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6824", 2021-02-10T11:33:04.796 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:33:04.796 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.796 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.797 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.797 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6826", 2021-02-10T11:33:04.797 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:33:04.797 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.797 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.797 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.797 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6818/36380", 2021-02-10T11:33:04.797 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6822/36380", 2021-02-10T11:33:04.797 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6830/36380", 2021-02-10T11:33:04.797 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6826/36380", 2021-02-10T11:33:04.798 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:33:04.798 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:33:04.798 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:33:04.798 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.798 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.798 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.798 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:33:04.798 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "88e694e8-fe98-4d1b-88c8-8c9907ece60b", 2021-02-10T11:33:04.798 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:33:04.798 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:33:04.798 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:33:04.798 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:33:04.798 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:33:04.799 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:33:04.799 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:33:04.799 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 45, 2021-02-10T11:33:04.799 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:33:04.799 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:33:04.799 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:33:04.799 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.799 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.799 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.799 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6808", 2021-02-10T11:33:04.799 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:33:04.799 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.799 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.800 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.800 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6809", 2021-02-10T11:33:04.800 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:33:04.800 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.800 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.800 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.800 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:33:04.800 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.800 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.800 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.800 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6810", 2021-02-10T11:33:04.800 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:33:04.800 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.800 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.801 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.801 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6811", 2021-02-10T11:33:04.801 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:33:04.801 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.801 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.801 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.801 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:33:04.801 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.801 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.801 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.801 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6814", 2021-02-10T11:33:04.801 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:33:04.801 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.802 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.802 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.802 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6815", 2021-02-10T11:33:04.802 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:33:04.802 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.802 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.802 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.802 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:33:04.802 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.802 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.802 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.802 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6812", 2021-02-10T11:33:04.802 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:33:04.803 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.803 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.803 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.803 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6813", 2021-02-10T11:33:04.803 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:33:04.803 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.803 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.803 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.803 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6809/36383", 2021-02-10T11:33:04.803 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6811/36383", 2021-02-10T11:33:04.803 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6815/36383", 2021-02-10T11:33:04.803 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6813/36383", 2021-02-10T11:33:04.803 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:33:04.803 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:33:04.804 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:33:04.804 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.804 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.804 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.804 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:33:04.804 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "87a13c6c-cf85-4aaf-916b-e73913484a49", 2021-02-10T11:33:04.804 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:33:04.804 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:33:04.804 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:33:04.804 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:33:04.804 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:33:04.804 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:33:04.804 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:33:04.804 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 43, 2021-02-10T11:33:04.805 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:33:04.805 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:33:04.805 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:33:04.805 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.805 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.805 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.805 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6800", 2021-02-10T11:33:04.805 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:33:04.805 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.805 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.805 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.805 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6801", 2021-02-10T11:33:04.805 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:33:04.805 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.806 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.806 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.806 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:33:04.806 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.806 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.806 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.806 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6802", 2021-02-10T11:33:04.806 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:33:04.806 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.806 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.806 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.806 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6803", 2021-02-10T11:33:04.806 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:33:04.807 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.807 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.807 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.807 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:33:04.807 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.807 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.807 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.807 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6806", 2021-02-10T11:33:04.807 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:33:04.807 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.807 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.807 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.807 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6807", 2021-02-10T11:33:04.808 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:33:04.808 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.808 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.808 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.808 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:33:04.808 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.808 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.808 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.808 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6804", 2021-02-10T11:33:04.808 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:33:04.808 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.808 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.808 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.808 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6805", 2021-02-10T11:33:04.809 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:33:04.809 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.809 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.809 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.809 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6801/36381", 2021-02-10T11:33:04.809 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6803/36381", 2021-02-10T11:33:04.809 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6807/36381", 2021-02-10T11:33:04.809 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6805/36381", 2021-02-10T11:33:04.809 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:33:04.809 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:33:04.809 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:33:04.809 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.809 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.809 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.809 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:33:04.810 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "7b90823f-854c-4592-a95b-a7fa98f93254", 2021-02-10T11:33:04.810 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:33:04.810 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:33:04.810 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:33:04.810 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:33:04.810 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:33:04.810 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:33:04.810 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:33:04.810 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 45, 2021-02-10T11:33:04.810 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:33:04.810 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:33:04.810 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:33:04.810 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.811 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.811 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.811 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6817", 2021-02-10T11:33:04.811 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:33:04.811 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.811 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.811 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.811 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6819", 2021-02-10T11:33:04.811 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:33:04.811 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.811 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.811 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.811 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:33:04.812 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.812 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.812 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.812 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6821", 2021-02-10T11:33:04.812 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:33:04.812 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.812 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.812 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.812 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6823", 2021-02-10T11:33:04.812 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:33:04.812 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.812 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.812 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.812 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:33:04.812 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.813 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.813 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.813 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6829", 2021-02-10T11:33:04.813 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:33:04.813 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.813 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.813 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.813 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6831", 2021-02-10T11:33:04.813 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:33:04.813 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.813 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.813 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.813 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:33:04.813 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:33:04.813 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.814 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:33:04.814 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6825", 2021-02-10T11:33:04.814 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:33:04.814 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.814 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.814 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:04.814 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6827", 2021-02-10T11:33:04.814 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:33:04.814 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.814 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.814 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.814 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6819/36382", 2021-02-10T11:33:04.814 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6823/36382", 2021-02-10T11:33:04.815 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6831/36382", 2021-02-10T11:33:04.815 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6827/36382", 2021-02-10T11:33:04.815 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:33:04.815 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:33:04.815 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:33:04.815 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:04.815 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.815 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:33:04.815 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_xinfo": [ 2021-02-10T11:33:04.815 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.815 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:33:04.815 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:33:04.815 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:33:04.816 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:33:04.816 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:33:04.816 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:33:04.816 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.816 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.816 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:33:04.816 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:33:04.816 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:33:04.816 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:33:04.816 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:33:04.816 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:33:04.816 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.816 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.817 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:33:04.817 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:33:04.817 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:33:04.817 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:33:04.817 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:33:04.817 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:33:04.817 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.817 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.817 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:33:04.817 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:33:04.817 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:33:04.817 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:33:04.817 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:33:04.818 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:33:04.818 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.818 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.818 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:33:04.818 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:33:04.818 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:33:04.818 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:33:04.818 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:33:04.818 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:33:04.818 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.818 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.818 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:33:04.818 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:33:04.819 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:33:04.819 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:33:04.819 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:33:04.819 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:33:04.819 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.819 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.819 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:33:04.819 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:33:04.819 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:33:04.819 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:33:04.819 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:33:04.819 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:33:04.819 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.819 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:04.820 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:33:04.820 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:33:04.820 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:33:04.820 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:33:04.820 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:33:04.820 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:33:04.820 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.820 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:33:04.820 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap": [], 2021-02-10T11:33:04.820 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap_items": [], 2021-02-10T11:33:04.820 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_temp": [], 2021-02-10T11:33:04.820 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_temp": [], 2021-02-10T11:33:04.820 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklist": {}, 2021-02-10T11:33:04.821 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profiles": { 2021-02-10T11:33:04.821 INFO:teuthology.orchestra.run.gibba001.stdout: "default": { 2021-02-10T11:33:04.821 INFO:teuthology.orchestra.run.gibba001.stdout: "crush-failure-domain": "osd", 2021-02-10T11:33:04.821 INFO:teuthology.orchestra.run.gibba001.stdout: "k": "2", 2021-02-10T11:33:04.821 INFO:teuthology.orchestra.run.gibba001.stdout: "m": "1", 2021-02-10T11:33:04.821 INFO:teuthology.orchestra.run.gibba001.stdout: "plugin": "jerasure", 2021-02-10T11:33:04.821 INFO:teuthology.orchestra.run.gibba001.stdout: "ruleset-failure-domain": "osd", 2021-02-10T11:33:04.821 INFO:teuthology.orchestra.run.gibba001.stdout: "technique": "reed_sol_van" 2021-02-10T11:33:04.821 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:04.821 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:04.821 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps_queue": [], 2021-02-10T11:33:04.821 INFO:teuthology.orchestra.run.gibba001.stdout: "new_removed_snaps": [], 2021-02-10T11:33:04.822 INFO:teuthology.orchestra.run.gibba001.stdout: "new_purged_snaps": [], 2021-02-10T11:33:04.822 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_node_flags": {}, 2021-02-10T11:33:04.822 INFO:teuthology.orchestra.run.gibba001.stdout: "device_class_flags": {} 2021-02-10T11:33:04.822 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:33:04.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.736 7f09265ab700 1 -- 172.21.2.101:0/1393283161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09000186c0 msgr2=0x7f090001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:04.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.736 7f09265ab700 1 --2- 172.21.2.101:0/1393283161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09000186c0 0x7f090001abc0 crc :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:04.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.736 7f09265ab700 1 -- 172.21.2.101:0/1393283161 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09200a29a0 msgr2=0x7f09200a2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:04.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.736 7f09265ab700 1 --2- 172.21.2.101:0/1393283161 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09200a29a0 0x7f09200a2da0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7f0914009630 tx=0x7f091400b980).stop 2021-02-10T11:33:04.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.736 7f09265ab700 1 -- 172.21.2.101:0/1393283161 shutdown_connections 2021-02-10T11:33:04.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.736 7f09265ab700 1 --2- 172.21.2.101:0/1393283161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09000186c0 0x7f090001abc0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:04.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.736 7f09265ab700 1 --2- 172.21.2.101:0/1393283161 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09200a33c0 0x7f09200a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:04.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.736 7f09265ab700 1 --2- 172.21.2.101:0/1393283161 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09200a29a0 0x7f09200a2da0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:04.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.736 7f09265ab700 1 --2- 172.21.2.101:0/1393283161 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0920191bb0 0x7f09200a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:04.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.737 7f09265ab700 1 -- 172.21.2.101:0/1393283161 >> 172.21.2.101:0/1393283161 conn(0x7f09201a31d0 msgr2=0x7f092008b830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:04.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.737 7f09265ab700 1 -- 172.21.2.101:0/1393283161 shutdown_connections 2021-02-10T11:33:04.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:04.737 7f09265ab700 1 -- 172.21.2.101:0/1393283161 wait complete. 2021-02-10T11:33:04.825 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:33:05.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.224 7f527cbd0700 1 Processor -- start 2021-02-10T11:33:05.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.224 7f527cbd0700 1 -- start start 2021-02-10T11:33:05.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.224 7f527cbd0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f52780e5470 0x7f527807e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:05.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.224 7f527cbd0700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f527807ed80 con 0x7f527807f300 2021-02-10T11:33:05.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.224 7f527cbd0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52781b4010 con 0x7f52780e5070 2021-02-10T11:33:05.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.224 7f527cbd0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52780d1790 con 0x7f52780e5470 2021-02-10T11:33:05.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.224 7f527705f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f527807f300 0x7f527807f700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51868/0 (socket says 172.21.2.101:51868) 2021-02-10T11:33:05.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.224 7f527705f700 1 -- 172.21.2.101:0/4060458016 learned_addr learned my addr 172.21.2.101:0/4060458016 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:05.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.225 7f527585c700 1 -- 172.21.2.101:0/4060458016 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1020253248 0 0) 0x7f5270000f90 con 0x7f527807f300 2021-02-10T11:33:05.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.225 7f527585c700 1 -- 172.21.2.101:0/4060458016 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5260002fe0 con 0x7f527807f300 2021-02-10T11:33:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.225 7f527585c700 1 -- 172.21.2.101:0/4060458016 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 52403029 0 0) 0x7f5270001460 con 0x7f527807f300 2021-02-10T11:33:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.225 7f527585c700 1 -- 172.21.2.101:0/4060458016 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f52780e5470 msgr2=0x7f527807e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.225 7f527605d700 1 --2- 172.21.2.101:0/4060458016 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f52780e5470 0x7f527807e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.225 7f527585c700 1 --2- 172.21.2.101:0/4060458016 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f52780e5470 0x7f527807e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.225 7f527585c700 1 -- 172.21.2.101:0/4060458016 >> v1:172.21.2.108:6789/0 conn(0x7f52780e5070 legacy=0x7f52781b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.225 7f527585c700 1 -- 172.21.2.101:0/4060458016 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52780d1980 con 0x7f527807f300 2021-02-10T11:33:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.225 7f527585c700 1 -- 172.21.2.101:0/4060458016 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f5270001980 con 0x7f527807f300 2021-02-10T11:33:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.225 7f527585c700 1 -- 172.21.2.101:0/4060458016 >> v1:172.21.2.101:6789/0 conn(0x7f527807f300 legacy=0x7f527807f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.225 7f527585c700 1 --2- 172.21.2.101:0/4060458016 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5260003b90 0x7f5260006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.225 7f527585c700 1 --2- 172.21.2.101:0/4060458016 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f52600068c0 0x7f5260008d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.225 7f527585c700 1 --2- 172.21.2.101:0/4060458016 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5260009630 0x7f526000bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.225 7f527585c700 1 -- 172.21.2.101:0/4060458016 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526000c380 con 0x7f5260009630 2021-02-10T11:33:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.225 7f527585c700 1 -- 172.21.2.101:0/4060458016 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526000c5a0 con 0x7f5260003b90 2021-02-10T11:33:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.225 7f527585c700 1 -- 172.21.2.101:0/4060458016 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526000c7c0 con 0x7f52600068c0 2021-02-10T11:33:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.225 7f527605d700 1 --2- 172.21.2.101:0/4060458016 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f52780e5470 0x7f527807e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:33:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.226 7f527705f700 1 --2- 172.21.2.101:0/4060458016 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5260009630 0x7f526000bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.226 7f527685e700 1 --2- 172.21.2.101:0/4060458016 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5260003b90 0x7f5260006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.226 7f527605d700 1 --2- 172.21.2.101:0/4060458016 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f52600068c0 0x7f5260008d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.226 7f527705f700 1 -- 172.21.2.101:0/4060458016 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f52600068c0 msgr2=0x7f5260008d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.226 7f527705f700 1 --2- 172.21.2.101:0/4060458016 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f52600068c0 0x7f5260008d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.226 7f527705f700 1 -- 172.21.2.101:0/4060458016 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5260003b90 msgr2=0x7f5260006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.226 7f527705f700 1 --2- 172.21.2.101:0/4060458016 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5260003b90 0x7f5260006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.226 7f527705f700 1 -- 172.21.2.101:0/4060458016 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526000ca60 con 0x7f5260009630 2021-02-10T11:33:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.226 7f527705f700 1 --2- 172.21.2.101:0/4060458016 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5260009630 0x7f526000bb50 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7f5270001ae0 tx=0x7f527004e320).ready entity=mon.0 client_cookie=5410c577e1e6d984 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.226 7f527585c700 1 -- 172.21.2.101:0/4060458016 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5270001460 con 0x7f5260009630 2021-02-10T11:33:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.226 7f527585c700 1 -- 172.21.2.101:0/4060458016 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52700070d0 con 0x7f5260009630 2021-02-10T11:33:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.226 7f527585c700 1 -- 172.21.2.101:0/4060458016 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5270004720 con 0x7f5260009630 2021-02-10T11:33:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.226 7f527cbd0700 1 -- 172.21.2.101:0/4060458016 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5260009630 msgr2=0x7f526000bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:05.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.226 7f527cbd0700 1 --2- 172.21.2.101:0/4060458016 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5260009630 0x7f526000bb50 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7f5270001ae0 tx=0x7f527004e320).stop 2021-02-10T11:33:05.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.226 7f527705f700 1 -- 172.21.2.101:0/4060458016 reap_dead start 2021-02-10T11:33:05.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.227 7f527cbd0700 1 -- 172.21.2.101:0/4060458016 shutdown_connections 2021-02-10T11:33:05.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.227 7f527cbd0700 1 -- 172.21.2.101:0/4060458016 >> 172.21.2.101:0/4060458016 conn(0x7f52780ce690 msgr2=0x7f52780e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:05.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.227 7f527cbd0700 1 -- 172.21.2.101:0/4060458016 shutdown_connections 2021-02-10T11:33:05.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.227 7f527cbd0700 1 -- 172.21.2.101:0/4060458016 wait complete. 2021-02-10T11:33:05.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.242 7f527cbd0700 1 Processor -- start 2021-02-10T11:33:05.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.242 7f527cbd0700 1 -- start start 2021-02-10T11:33:05.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.243 7f527cbd0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5278194000 0x7f52780b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:05.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.243 7f527cbd0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52780b1330 0x7f5278087050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:05.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.243 7f527cbd0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5278087670 0x7f5278087a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:05.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.243 7f527cbd0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527809c0e0 con 0x7f52780b1330 2021-02-10T11:33:05.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.243 7f527cbd0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5278194440 con 0x7f5278087670 2021-02-10T11:33:05.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.243 7f527cbd0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527809c5e0 con 0x7f5278194000 2021-02-10T11:33:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.243 7f527605d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52780b1330 0x7f5278087050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.243 7f527605d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52780b1330 0x7f5278087050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51384/0 (socket says 172.21.2.101:51384) 2021-02-10T11:33:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.243 7f527605d700 1 -- 172.21.2.101:0/2945964277 learned_addr learned my addr 172.21.2.101:0/2945964277 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.243 7f527705f700 1 --2- 172.21.2.101:0/2945964277 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5278087670 0x7f5278087a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.243 7f527685e700 1 --2- 172.21.2.101:0/2945964277 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5278194000 0x7f52780b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.243 7f527605d700 1 -- 172.21.2.101:0/2945964277 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5278194000 msgr2=0x7f52780b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.243 7f527605d700 1 --2- 172.21.2.101:0/2945964277 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5278194000 0x7f52780b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.243 7f527605d700 1 -- 172.21.2.101:0/2945964277 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5278087670 msgr2=0x7f5278087a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.243 7f527605d700 1 --2- 172.21.2.101:0/2945964277 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5278087670 0x7f5278087a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.243 7f527605d700 1 -- 172.21.2.101:0/2945964277 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527809c7d0 con 0x7f52780b1330 2021-02-10T11:33:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.244 7f527605d700 1 --2- 172.21.2.101:0/2945964277 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52780b1330 0x7f5278087050 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f526c006f50 tx=0x7f526c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:05.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.244 7f52677fe700 1 -- 172.21.2.101:0/2945964277 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f526c00bd50 con 0x7f52780b1330 2021-02-10T11:33:05.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.244 7f52677fe700 1 -- 172.21.2.101:0/2945964277 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526c000c90 con 0x7f52780b1330 2021-02-10T11:33:05.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.244 7f52677fe700 1 -- 172.21.2.101:0/2945964277 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f526c002270 con 0x7f52780b1330 2021-02-10T11:33:05.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.244 7f527cbd0700 1 -- 172.21.2.101:0/2945964277 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f527809c7d0 con 0x7f52780b1330 2021-02-10T11:33:05.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.244 7f527cbd0700 1 -- 172.21.2.101:0/2945964277 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527809c7d0 con 0x7f52780b1330 2021-02-10T11:33:05.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.244 7f52677fe700 1 -- 172.21.2.101:0/2945964277 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f526c002020 con 0x7f52780b1330 2021-02-10T11:33:05.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.244 7f52677fe700 1 --2- 172.21.2.101:0/2945964277 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f525c0186c0 0x7f525c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:05.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.245 7f52677fe700 1 -- 172.21.2.101:0/2945964277 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f526c032480 con 0x7f52780b1330 2021-02-10T11:33:05.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.245 7f527685e700 1 --2- 172.21.2.101:0/2945964277 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f525c0186c0 0x7f525c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:05.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.245 7f527685e700 1 --2- 172.21.2.101:0/2945964277 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f525c0186c0 0x7f525c01abc0 crc :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:05.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.274 7f527cbd0700 1 -- 172.21.2.101:0/2945964277 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5278084b20 con 0x7f52780b1330 2021-02-10T11:33:05.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.277 7f52677fe700 1 -- 172.21.2.101:0/2945964277 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f526c030db0 con 0x7f52780b1330 2021-02-10T11:33:05.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.406 7f527cbd0700 1 -- 172.21.2.101:0/2945964277 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5278084b20 con 0x7f52780b1330 2021-02-10T11:33:05.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.406 7f52677fe700 1 -- 172.21.2.101:0/2945964277 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+2099 (secure 0 0 0) 0x7f526c00bd50 con 0x7f52780b1330 2021-02-10T11:33:05.418 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 32 2021-02-10T11:33:05.418 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:05.418 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7674,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1796077990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1796077990},{"type":"v1","addr":"172.21.2.101:6835","nonce":1796077990}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:32:24.604347","modified":"2021-02-10 11:32:26.612664","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5986},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5986":{"gid":5986,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/3346373345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3346373345},{"type":"v1","addr":"172.21.2.108:6833","nonce":3346373345}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-02-10T11:33:05.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.408 7f527cbd0700 1 -- 172.21.2.101:0/2945964277 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f525c0186c0 msgr2=0x7f525c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:05.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.408 7f527cbd0700 1 --2- 172.21.2.101:0/2945964277 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f525c0186c0 0x7f525c01abc0 crc :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:05.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.408 7f527cbd0700 1 -- 172.21.2.101:0/2945964277 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52780b1330 msgr2=0x7f5278087050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:05.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.408 7f527cbd0700 1 --2- 172.21.2.101:0/2945964277 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52780b1330 0x7f5278087050 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f526c006f50 tx=0x7f526c00b960).stop 2021-02-10T11:33:05.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.409 7f527cbd0700 1 -- 172.21.2.101:0/2945964277 shutdown_connections 2021-02-10T11:33:05.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.409 7f527cbd0700 1 --2- 172.21.2.101:0/2945964277 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5278087670 0x7f5278087a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:05.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.409 7f527cbd0700 1 --2- 172.21.2.101:0/2945964277 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52780b1330 0x7f5278087050 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:05.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.409 7f527cbd0700 1 --2- 172.21.2.101:0/2945964277 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f525c0186c0 0x7f525c01abc0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:05.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.409 7f527cbd0700 1 --2- 172.21.2.101:0/2945964277 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5278194000 0x7f52780b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:05.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.409 7f527cbd0700 1 -- 172.21.2.101:0/2945964277 >> 172.21.2.101:0/2945964277 conn(0x7f5278092c20 msgr2=0x7f5278195070 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:05.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.409 7f527cbd0700 1 -- 172.21.2.101:0/2945964277 shutdown_connections 2021-02-10T11:33:05.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.409 7f527cbd0700 1 -- 172.21.2.101:0/2945964277 wait complete. 2021-02-10T11:33:05.428 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs fail cephfs 2021-02-10T11:33:05.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.827 7f50296c6700 1 Processor -- start 2021-02-10T11:33:05.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.827 7f50296c6700 1 -- start start 2021-02-10T11:33:05.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.827 7f50296c6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f50240e5470 0x7f502407e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:05.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.827 7f50296c6700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f502407ed80 con 0x7f502407f300 2021-02-10T11:33:05.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.827 7f50296c6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50241b4010 con 0x7f50240e5070 2021-02-10T11:33:05.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.827 7f50296c6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50240d1790 con 0x7f50240e5470 2021-02-10T11:33:05.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.827 7f5023ac0700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f502407f300 0x7f502407f700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51886/0 (socket says 172.21.2.101:51886) 2021-02-10T11:33:05.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.827 7f5023ac0700 1 -- 172.21.2.101:0/2607851364 learned_addr learned my addr 172.21.2.101:0/2607851364 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:05.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.827 7f5022abe700 1 --2- 172.21.2.101:0/2607851364 >> v2:172.21.2.101:3300/0 conn(0x7f50240e5470 0x7f502407e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:05.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.828 7f5022abe700 1 -- 172.21.2.101:0/2607851364 >> v1:172.21.2.108:6789/0 conn(0x7f50240e5070 legacy=0x7f50241b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:05.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.828 7f5022abe700 1 -- 172.21.2.101:0/2607851364 >> v1:172.21.2.101:6789/0 conn(0x7f502407f300 legacy=0x7f502407f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:05.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.828 7f5022abe700 1 -- 172.21.2.101:0/2607851364 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50240d19b0 con 0x7f50240e5470 2021-02-10T11:33:05.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.828 7f5022abe700 1 --2- 172.21.2.101:0/2607851364 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50240e5470 0x7f502407e6a0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7f5014006650 tx=0x7f5014009a80).ready entity=mon.0 client_cookie=a2a60adffb9278c7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:05.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.828 7f50222bd700 1 -- 172.21.2.101:0/2607851364 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f50140009b0 con 0x7f50240e5470 2021-02-10T11:33:05.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.828 7f50222bd700 1 -- 172.21.2.101:0/2607851364 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501400c0c0 con 0x7f50240e5470 2021-02-10T11:33:05.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.828 7f50222bd700 1 -- 172.21.2.101:0/2607851364 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f501400c5e0 con 0x7f50240e5470 2021-02-10T11:33:05.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.828 7f50296c6700 1 -- 172.21.2.101:0/2607851364 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50240e5470 msgr2=0x7f502407e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:05.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.828 7f50296c6700 1 --2- 172.21.2.101:0/2607851364 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50240e5470 0x7f502407e6a0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7f5014006650 tx=0x7f5014009a80).stop 2021-02-10T11:33:05.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.829 7f50296c6700 1 -- 172.21.2.101:0/2607851364 shutdown_connections 2021-02-10T11:33:05.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.829 7f50296c6700 1 --2- 172.21.2.101:0/2607851364 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50240e5470 0x7f502407e6a0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:05.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.829 7f50296c6700 1 -- 172.21.2.101:0/2607851364 >> 172.21.2.101:0/2607851364 conn(0x7f50240ce690 msgr2=0x7f50240e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:05.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.829 7f50296c6700 1 -- 172.21.2.101:0/2607851364 shutdown_connections 2021-02-10T11:33:05.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.829 7f50296c6700 1 -- 172.21.2.101:0/2607851364 wait complete. 2021-02-10T11:33:05.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.844 7f50296c6700 1 Processor -- start 2021-02-10T11:33:05.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.844 7f50296c6700 1 -- start start 2021-02-10T11:33:05.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.845 7f50296c6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50241c24c0 0x7f50240c37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:05.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.845 7f50296c6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50240c3dc0 0x7f50240a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:05.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.845 7f50296c6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f50240a2510 0x7f50240a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:05.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.845 7f50296c6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50240b9ef0 con 0x7f50240c3dc0 2021-02-10T11:33:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.845 7f50296c6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50240a2910 con 0x7f50240a2510 2021-02-10T11:33:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.845 7f50296c6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50240a2af0 con 0x7f50241c24c0 2021-02-10T11:33:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.845 7f5022abe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50240c3dc0 0x7f50240a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.845 7f5022abe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50240c3dc0 0x7f50240a1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51394/0 (socket says 172.21.2.101:51394) 2021-02-10T11:33:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.845 7f5022abe700 1 -- 172.21.2.101:0/1857918915 learned_addr learned my addr 172.21.2.101:0/1857918915 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.845 7f5022abe700 1 -- 172.21.2.101:0/1857918915 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50241c24c0 msgr2=0x7f50240c37a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.845 7f5022abe700 1 --2- 172.21.2.101:0/1857918915 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50241c24c0 0x7f50240c37a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.845 7f5022abe700 1 -- 172.21.2.101:0/1857918915 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f50240a2510 msgr2=0x7f50240a0e90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.845 7f5022abe700 1 --2- 172.21.2.101:0/1857918915 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f50240a2510 0x7f50240a0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.845 7f5022abe700 1 -- 172.21.2.101:0/1857918915 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50240a14b0 con 0x7f50240c3dc0 2021-02-10T11:33:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.845 7f5022abe700 1 --2- 172.21.2.101:0/1857918915 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50240c3dc0 0x7f50240a1ef0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7f5014009650 tx=0x7f501400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.845 7f501bfff700 1 -- 172.21.2.101:0/1857918915 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5014009360 con 0x7f50240c3dc0 2021-02-10T11:33:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.845 7f501bfff700 1 -- 172.21.2.101:0/1857918915 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501400bd50 con 0x7f50240c3dc0 2021-02-10T11:33:05.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.845 7f50296c6700 1 -- 172.21.2.101:0/1857918915 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50240a14b0 con 0x7f50240c3dc0 2021-02-10T11:33:05.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.845 7f501bfff700 1 -- 172.21.2.101:0/1857918915 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f50140052e0 con 0x7f50240c3dc0 2021-02-10T11:33:05.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.846 7f50296c6700 1 -- 172.21.2.101:0/1857918915 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50240a14b0 con 0x7f50240c3dc0 2021-02-10T11:33:05.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.846 7f501bfff700 1 -- 172.21.2.101:0/1857918915 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5014006030 con 0x7f50240c3dc0 2021-02-10T11:33:05.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.846 7f501bfff700 1 --2- 172.21.2.101:0/1857918915 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5008018730 0x7f500801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:05.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.846 7f501bfff700 1 -- 172.21.2.101:0/1857918915 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5014032630 con 0x7f50240c3dc0 2021-02-10T11:33:05.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.846 7f50232bf700 1 --2- 172.21.2.101:0/1857918915 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5008018730 0x7f500801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:05.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.846 7f50232bf700 1 --2- 172.21.2.101:0/1857918915 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5008018730 0x7f500801ac10 crc :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:05.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.877 7f50296c6700 1 -- 172.21.2.101:0/1857918915 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50240898d0 con 0x7f50240c3dc0 2021-02-10T11:33:05.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:05.879 7f501bfff700 1 -- 172.21.2.101:0/1857918915 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5014000940 con 0x7f50240c3dc0 2021-02-10T11:33:06.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.005 7f50296c6700 1 -- 172.21.2.101:0/1857918915 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f50240ce690 con 0x7f50240c3dc0 2021-02-10T11:33:06.461 INFO:tasks.ceph.mon.a.gibba001.stderr:2021-02-10 11:33:06.450 7f974d04d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-02-10T11:33:06.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.456 7f501bfff700 1 -- 172.21.2.101:0/1857918915 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5014006030 con 0x7f50240c3dc0 2021-02-10T11:33:06.483 INFO:teuthology.orchestra.run.gibba001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-02-10T11:33:06.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.474 7f50296c6700 1 -- 172.21.2.101:0/1857918915 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5008018730 msgr2=0x7f500801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:06.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.474 7f50296c6700 1 --2- 172.21.2.101:0/1857918915 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5008018730 0x7f500801ac10 crc :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:06.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.474 7f50296c6700 1 -- 172.21.2.101:0/1857918915 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50240c3dc0 msgr2=0x7f50240a1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:06.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.474 7f50296c6700 1 --2- 172.21.2.101:0/1857918915 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50240c3dc0 0x7f50240a1ef0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7f5014009650 tx=0x7f501400b960).stop 2021-02-10T11:33:06.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.474 7f50296c6700 1 -- 172.21.2.101:0/1857918915 shutdown_connections 2021-02-10T11:33:06.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.475 7f50296c6700 1 --2- 172.21.2.101:0/1857918915 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f50240a2510 0x7f50240a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:06.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.475 7f50296c6700 1 --2- 172.21.2.101:0/1857918915 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50240c3dc0 0x7f50240a1ef0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:06.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.475 7f50296c6700 1 --2- 172.21.2.101:0/1857918915 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5008018730 0x7f500801ac10 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:06.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.475 7f50296c6700 1 --2- 172.21.2.101:0/1857918915 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50241c24c0 0x7f50240c37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:06.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.475 7f50296c6700 1 -- 172.21.2.101:0/1857918915 >> 172.21.2.101:0/1857918915 conn(0x7f50240bc2f0 msgr2=0x7f50240afea0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:06.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.475 7f50296c6700 1 -- 172.21.2.101:0/1857918915 shutdown_connections 2021-02-10T11:33:06.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.475 7f50296c6700 1 -- 172.21.2.101:0/1857918915 wait complete. 2021-02-10T11:33:06.494 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:33:06.518 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:33:06.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.892 7f4d8b2d2700 1 Processor -- start 2021-02-10T11:33:06.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.892 7f4d8b2d2700 1 -- start start 2021-02-10T11:33:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.893 7f4d8b2d2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d840e5070 0x7f4d841b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.893 7f4d8b2d2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f4d840e5470 0x7f4d8407d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.893 7f4d8b2d2700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d8407ed80 con 0x7f4d841b3e40 2021-02-10T11:33:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.893 7f4d8b2d2700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d84080310 con 0x7f4d840e5470 2021-02-10T11:33:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.893 7f4d8b2d2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d84080500 con 0x7f4d840e5070 2021-02-10T11:33:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.893 7f4d898ef700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f4d841b3e40 0x7f4d8407dcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51896/0 (socket says 172.21.2.101:51896) 2021-02-10T11:33:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.893 7f4d898ef700 1 -- 172.21.2.101:0/711982167 learned_addr learned my addr 172.21.2.101:0/711982167 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.893 7f4d888ed700 1 --2- 172.21.2.101:0/711982167 >> v2:172.21.2.101:3300/0 conn(0x7f4d840e5470 0x7f4d8407d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.893 7f4d888ed700 1 -- 172.21.2.101:0/711982167 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d840e5070 msgr2=0x7f4d841b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.893 7f4d890ee700 1 --2- 172.21.2.101:0/711982167 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d840e5070 0x7f4d841b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.893 7f4d77fff700 1 -- 172.21.2.101:0/711982167 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1838502525 0 0) 0x7f4d7c001500 con 0x7f4d841b3e40 2021-02-10T11:33:06.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.893 7f4d888ed700 1 --2- 172.21.2.101:0/711982167 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d840e5070 0x7f4d841b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:06.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.893 7f4d888ed700 1 -- 172.21.2.101:0/711982167 >> v1:172.21.2.101:6789/0 conn(0x7f4d841b3e40 legacy=0x7f4d8407dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:06.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.893 7f4d888ed700 1 -- 172.21.2.101:0/711982167 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d840806f0 con 0x7f4d840e5470 2021-02-10T11:33:06.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.893 7f4d890ee700 1 --2- 172.21.2.101:0/711982167 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d840e5070 0x7f4d841b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:33:06.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.893 7f4d888ed700 1 --2- 172.21.2.101:0/711982167 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d840e5470 0x7f4d8407d690 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0x7f4d80006650 tx=0x7f4d80009a80).ready entity=mon.0 client_cookie=659f63cef0277752 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:06.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.894 7f4d77fff700 1 -- 172.21.2.101:0/711982167 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d800009b0 con 0x7f4d840e5470 2021-02-10T11:33:06.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.894 7f4d77fff700 1 -- 172.21.2.101:0/711982167 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8000c0c0 con 0x7f4d840e5470 2021-02-10T11:33:06.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.894 7f4d77fff700 1 -- 172.21.2.101:0/711982167 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d8000c5e0 con 0x7f4d840e5470 2021-02-10T11:33:06.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.894 7f4d8b2d2700 1 -- 172.21.2.101:0/711982167 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d840e5470 msgr2=0x7f4d8407d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:06.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.894 7f4d8b2d2700 1 --2- 172.21.2.101:0/711982167 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d840e5470 0x7f4d8407d690 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0x7f4d80006650 tx=0x7f4d80009a80).stop 2021-02-10T11:33:06.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.894 7f4d8b2d2700 1 -- 172.21.2.101:0/711982167 shutdown_connections 2021-02-10T11:33:06.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.894 7f4d8b2d2700 1 --2- 172.21.2.101:0/711982167 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d840e5470 0x7f4d8407d690 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:06.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.894 7f4d8b2d2700 1 --2- 172.21.2.101:0/711982167 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d840e5070 0x7f4d841b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:06.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.894 7f4d8b2d2700 1 -- 172.21.2.101:0/711982167 >> 172.21.2.101:0/711982167 conn(0x7f4d840ce690 msgr2=0x7f4d840e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:06.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.894 7f4d8b2d2700 1 -- 172.21.2.101:0/711982167 shutdown_connections 2021-02-10T11:33:06.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.894 7f4d8b2d2700 1 -- 172.21.2.101:0/711982167 wait complete. 2021-02-10T11:33:06.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.909 7f4d8b2d2700 1 Processor -- start 2021-02-10T11:33:06.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.909 7f4d8b2d2700 1 -- start start 2021-02-10T11:33:06.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.909 7f4d8b2d2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d8408d400 0x7f4d84081230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:06.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.909 7f4d8b2d2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d84081850 0x7f4d840acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:06.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.909 7f4d8b2d2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d84081c50 0x7f4d840ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:06.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.909 7f4d8b2d2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d84082110 con 0x7f4d8408d400 2021-02-10T11:33:06.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.909 7f4d8b2d2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8408d800 con 0x7f4d84081c50 2021-02-10T11:33:06.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.909 7f4d8b2d2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d840adbc0 con 0x7f4d84081850 2021-02-10T11:33:06.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.909 7f4d890ee700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d8408d400 0x7f4d84081230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:06.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.909 7f4d890ee700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d8408d400 0x7f4d84081230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51408/0 (socket says 172.21.2.101:51408) 2021-02-10T11:33:06.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.909 7f4d890ee700 1 -- 172.21.2.101:0/390718140 learned_addr learned my addr 172.21.2.101:0/390718140 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:06.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.910 7f4d890ee700 1 -- 172.21.2.101:0/390718140 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d84081850 msgr2=0x7f4d840acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:06.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.910 7f4d890ee700 1 --2- 172.21.2.101:0/390718140 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d84081850 0x7f4d840acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:06.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.910 7f4d890ee700 1 -- 172.21.2.101:0/390718140 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d84081c50 msgr2=0x7f4d840ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:06.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.910 7f4d890ee700 1 --2- 172.21.2.101:0/390718140 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d84081c50 0x7f4d840ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:06.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.910 7f4d890ee700 1 -- 172.21.2.101:0/390718140 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d84187fc0 con 0x7f4d8408d400 2021-02-10T11:33:06.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.910 7f4d890ee700 1 --2- 172.21.2.101:0/390718140 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d8408d400 0x7f4d84081230 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 rx=0x7f4d78006f20 tx=0x7f4d7800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:06.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.910 7f4d75ffb700 1 -- 172.21.2.101:0/390718140 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d780009b0 con 0x7f4d8408d400 2021-02-10T11:33:06.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.910 7f4d75ffb700 1 -- 172.21.2.101:0/390718140 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7800bd80 con 0x7f4d8408d400 2021-02-10T11:33:06.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.910 7f4d75ffb700 1 -- 172.21.2.101:0/390718140 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d78002220 con 0x7f4d8408d400 2021-02-10T11:33:06.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.910 7f4d8b2d2700 1 -- 172.21.2.101:0/390718140 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d84187fc0 con 0x7f4d8408d400 2021-02-10T11:33:06.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.910 7f4d8b2d2700 1 -- 172.21.2.101:0/390718140 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d84188550 con 0x7f4d8408d400 2021-02-10T11:33:06.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.911 7f4d75ffb700 1 -- 172.21.2.101:0/390718140 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4d78000c90 con 0x7f4d8408d400 2021-02-10T11:33:06.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.911 7f4d75ffb700 1 --2- 172.21.2.101:0/390718140 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4d640186c0 0x7f4d6401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:06.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.911 7f4d75ffb700 1 -- 172.21.2.101:0/390718140 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f4d780325e0 con 0x7f4d8408d400 2021-02-10T11:33:06.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.911 7f4d888ed700 1 --2- 172.21.2.101:0/390718140 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4d640186c0 0x7f4d6401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:06.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.911 7f4d888ed700 1 --2- 172.21.2.101:0/390718140 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4d640186c0 0x7f4d6401abc0 crc :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:06.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.942 7f4d8b2d2700 1 -- 172.21.2.101:0/390718140 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d8407ed80 con 0x7f4d8408d400 2021-02-10T11:33:06.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:06.945 7f4d75ffb700 1 -- 172.21.2.101:0/390718140 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4d7801da10 con 0x7f4d8408d400 2021-02-10T11:33:07.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.075 7f4d8b2d2700 1 -- 172.21.2.101:0/390718140 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d8407ed80 con 0x7f4d8408d400 2021-02-10T11:33:07.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.075 7f4d75ffb700 1 -- 172.21.2.101:0/390718140 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+1754 (secure 0 0 0) 0x7f4d7801da10 con 0x7f4d8408d400 2021-02-10T11:33:07.087 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:07.087 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7674,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1796077990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1796077990},{"type":"v1","addr":"172.21.2.101:6835","nonce":1796077990}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:32:24.604347","modified":"2021-02-10 11:33:06.451605","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-02-10T11:33:07.087 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 33 2021-02-10T11:33:07.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.077 7f4d8b2d2700 1 -- 172.21.2.101:0/390718140 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4d640186c0 msgr2=0x7f4d6401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:07.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.077 7f4d8b2d2700 1 --2- 172.21.2.101:0/390718140 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4d640186c0 0x7f4d6401abc0 crc :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:07.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.077 7f4d8b2d2700 1 -- 172.21.2.101:0/390718140 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d8408d400 msgr2=0x7f4d84081230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:07.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.077 7f4d8b2d2700 1 --2- 172.21.2.101:0/390718140 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d8408d400 0x7f4d84081230 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 rx=0x7f4d78006f20 tx=0x7f4d7800b990).stop 2021-02-10T11:33:07.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.078 7f4d8b2d2700 1 -- 172.21.2.101:0/390718140 shutdown_connections 2021-02-10T11:33:07.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.078 7f4d8b2d2700 1 --2- 172.21.2.101:0/390718140 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4d640186c0 0x7f4d6401abc0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:07.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.078 7f4d8b2d2700 1 --2- 172.21.2.101:0/390718140 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d84081850 0x7f4d840acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:07.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.078 7f4d8b2d2700 1 --2- 172.21.2.101:0/390718140 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d84081c50 0x7f4d840ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:07.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.078 7f4d8b2d2700 1 --2- 172.21.2.101:0/390718140 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d8408d400 0x7f4d84081230 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:07.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.078 7f4d8b2d2700 1 -- 172.21.2.101:0/390718140 >> 172.21.2.101:0/390718140 conn(0x7f4d841ac620 msgr2=0x7f4d840896e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:07.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.078 7f4d8b2d2700 1 -- 172.21.2.101:0/390718140 shutdown_connections 2021-02-10T11:33:07.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.078 7f4d8b2d2700 1 -- 172.21.2.101:0/390718140 wait complete. 2021-02-10T11:33:07.097 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-02-10T11:33:07.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.502 7fae56694700 1 Processor -- start 2021-02-10T11:33:07.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.502 7fae56694700 1 -- start start 2021-02-10T11:33:07.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.503 7fae56694700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae500d17f0 0x7fae501b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:07.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.503 7fae56694700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fae500d1bf0 0x7fae501b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:07.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.503 7fae56694700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fae501b3e20 0x7fae500ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:07.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.503 7fae56694700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae500e5eb0 con 0x7fae500d1bf0 2021-02-10T11:33:07.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.503 7fae56694700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae500f0b20 con 0x7fae501b3e20 2021-02-10T11:33:07.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.503 7fae56694700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae500cf510 con 0x7fae500d17f0 2021-02-10T11:33:07.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.503 7fae4f7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fae500d1bf0 0x7fae501b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:07.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.503 7fae4f7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fae500d1bf0 0x7fae501b3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51414/0 (socket says 172.21.2.101:51414) 2021-02-10T11:33:07.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.503 7fae4f7fe700 1 -- 172.21.2.101:0/3014898551 learned_addr learned my addr 172.21.2.101:0/3014898551 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:07.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.503 7fae4f7fe700 1 -- 172.21.2.101:0/3014898551 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae500d17f0 msgr2=0x7fae501b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:07.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.503 7fae4ffff700 1 --2- 172.21.2.101:0/3014898551 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae500d17f0 0x7fae501b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:07.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.503 7fae4f7fe700 1 --2- 172.21.2.101:0/3014898551 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae500d17f0 0x7fae501b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:07.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.503 7fae4f7fe700 1 -- 172.21.2.101:0/3014898551 >> v2:172.21.2.108:3300/0 conn(0x7fae501b3e20 msgr2=0x7fae500ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:07.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.503 7fae54cb1700 1 --2- 172.21.2.101:0/3014898551 >> v2:172.21.2.108:3300/0 conn(0x7fae501b3e20 0x7fae500ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:07.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.503 7fae4f7fe700 1 --2- 172.21.2.101:0/3014898551 >> v2:172.21.2.108:3300/0 conn(0x7fae501b3e20 0x7fae500ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:07.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.503 7fae4f7fe700 1 -- 172.21.2.101:0/3014898551 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae500cf730 con 0x7fae500d1bf0 2021-02-10T11:33:07.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.503 7fae4ffff700 1 --2- 172.21.2.101:0/3014898551 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae500d17f0 0x7fae501b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:33:07.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.504 7fae54cb1700 1 --2- 172.21.2.101:0/3014898551 >> v2:172.21.2.108:3300/0 conn(0x7fae501b3e20 0x7fae500ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:33:07.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.504 7fae4f7fe700 1 --2- 172.21.2.101:0/3014898551 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae500d1bf0 0x7fae501b3800 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7fae44006700 tx=0x7fae44009b80).ready entity=mon.0 client_cookie=ae3c16eb1d1c243 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:07.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.504 7fae4effd700 1 -- 172.21.2.101:0/3014898551 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fae4400c0c0 con 0x7fae500d1bf0 2021-02-10T11:33:07.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.504 7fae4effd700 1 -- 172.21.2.101:0/3014898551 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4400c3c0 con 0x7fae500d1bf0 2021-02-10T11:33:07.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.504 7fae4effd700 1 -- 172.21.2.101:0/3014898551 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fae4400c8e0 con 0x7fae500d1bf0 2021-02-10T11:33:07.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.504 7fae56694700 1 -- 172.21.2.101:0/3014898551 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae500d1bf0 msgr2=0x7fae501b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:07.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.504 7fae56694700 1 --2- 172.21.2.101:0/3014898551 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae500d1bf0 0x7fae501b3800 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7fae44006700 tx=0x7fae44009b80).stop 2021-02-10T11:33:07.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.505 7fae56694700 1 -- 172.21.2.101:0/3014898551 shutdown_connections 2021-02-10T11:33:07.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.505 7fae56694700 1 --2- 172.21.2.101:0/3014898551 >> v2:172.21.2.108:3300/0 conn(0x7fae501b3e20 0x7fae500ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:07.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.505 7fae56694700 1 --2- 172.21.2.101:0/3014898551 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae500d1bf0 0x7fae501b3800 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:07.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.505 7fae56694700 1 --2- 172.21.2.101:0/3014898551 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae500d17f0 0x7fae501b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:07.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.505 7fae56694700 1 -- 172.21.2.101:0/3014898551 >> 172.21.2.101:0/3014898551 conn(0x7fae500fde40 msgr2=0x7fae500ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:07.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.505 7fae56694700 1 -- 172.21.2.101:0/3014898551 shutdown_connections 2021-02-10T11:33:07.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.505 7fae56694700 1 -- 172.21.2.101:0/3014898551 wait complete. 2021-02-10T11:33:07.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.519 7fae56694700 1 Processor -- start 2021-02-10T11:33:07.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.519 7fae56694700 1 -- start start 2021-02-10T11:33:07.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.520 7fae56694700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae50093490 0x7fae50097040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:07.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.520 7fae56694700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fae500929a0 0x7fae5018e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:07.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.520 7fae56694700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae50092da0 0x7fae5018e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:07.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.520 7fae56694700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5008bc90 con 0x7fae50093490 2021-02-10T11:33:07.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.520 7fae56694700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5018ed00 con 0x7fae500929a0 2021-02-10T11:33:07.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.520 7fae56694700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5018ef20 con 0x7fae50092da0 2021-02-10T11:33:07.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.520 7fae4ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae50093490 0x7fae50097040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:07.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.520 7fae4ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae50093490 0x7fae50097040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51422/0 (socket says 172.21.2.101:51422) 2021-02-10T11:33:07.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.520 7fae4ffff700 1 -- 172.21.2.101:0/1079074407 learned_addr learned my addr 172.21.2.101:0/1079074407 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:07.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.520 7fae4ffff700 1 -- 172.21.2.101:0/1079074407 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae50092da0 msgr2=0x7fae5018e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:07.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.520 7fae4ffff700 1 --2- 172.21.2.101:0/1079074407 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae50092da0 0x7fae5018e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:07.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.520 7fae4ffff700 1 -- 172.21.2.101:0/1079074407 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fae500929a0 msgr2=0x7fae5018e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:07.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.520 7fae4ffff700 1 --2- 172.21.2.101:0/1079074407 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fae500929a0 0x7fae5018e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:07.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.520 7fae4ffff700 1 -- 172.21.2.101:0/1079074407 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5008c210 con 0x7fae50093490 2021-02-10T11:33:07.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.520 7fae4ffff700 1 --2- 172.21.2.101:0/1079074407 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae50093490 0x7fae50097040 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 rx=0x7fae40006f20 tx=0x7fae4000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:07.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.521 7fae4cff9700 1 -- 172.21.2.101:0/1079074407 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fae4000bd80 con 0x7fae50093490 2021-02-10T11:33:07.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.521 7fae4cff9700 1 -- 172.21.2.101:0/1079074407 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae40000c90 con 0x7fae50093490 2021-02-10T11:33:07.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.521 7fae4cff9700 1 -- 172.21.2.101:0/1079074407 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fae400022b0 con 0x7fae50093490 2021-02-10T11:33:07.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.521 7fae56694700 1 -- 172.21.2.101:0/1079074407 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae5008c210 con 0x7fae50093490 2021-02-10T11:33:07.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.521 7fae56694700 1 -- 172.21.2.101:0/1079074407 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae5008c210 con 0x7fae50093490 2021-02-10T11:33:07.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.521 7fae4cff9700 1 -- 172.21.2.101:0/1079074407 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fae4000a000 con 0x7fae50093490 2021-02-10T11:33:07.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.521 7fae4cff9700 1 --2- 172.21.2.101:0/1079074407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fae300186c0 0x7fae3001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:07.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.521 7fae4cff9700 1 -- 172.21.2.101:0/1079074407 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fae40032570 con 0x7fae50093490 2021-02-10T11:33:07.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.522 7fae4f7fe700 1 --2- 172.21.2.101:0/1079074407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fae300186c0 0x7fae3001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:07.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.522 7fae4f7fe700 1 --2- 172.21.2.101:0/1079074407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fae300186c0 0x7fae3001abc0 crc :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:07.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.552 7fae56694700 1 -- 172.21.2.101:0/1079074407 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae501036f0 con 0x7fae50093490 2021-02-10T11:33:07.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.555 7fae4cff9700 1 -- 172.21.2.101:0/1079074407 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fae4002e350 con 0x7fae50093490 2021-02-10T11:33:07.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:07.682 7fae56694700 1 -- 172.21.2.101:0/1079074407 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fae500f0b20 con 0x7fae50093490 2021-02-10T11:33:08.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.460 7fae4cff9700 1 -- 172.21.2.101:0/1079074407 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7fae4002e350 con 0x7fae50093490 2021-02-10T11:33:08.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.505 7fae56694700 1 -- 172.21.2.101:0/1079074407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fae300186c0 msgr2=0x7fae3001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:08.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.505 7fae56694700 1 --2- 172.21.2.101:0/1079074407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fae300186c0 0x7fae3001abc0 crc :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:08.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.505 7fae56694700 1 -- 172.21.2.101:0/1079074407 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae50093490 msgr2=0x7fae50097040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:08.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.505 7fae56694700 1 --2- 172.21.2.101:0/1079074407 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae50093490 0x7fae50097040 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 rx=0x7fae40006f20 tx=0x7fae4000b990).stop 2021-02-10T11:33:08.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.505 7fae56694700 1 -- 172.21.2.101:0/1079074407 shutdown_connections 2021-02-10T11:33:08.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.505 7fae56694700 1 --2- 172.21.2.101:0/1079074407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fae300186c0 0x7fae3001abc0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:08.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.505 7fae56694700 1 --2- 172.21.2.101:0/1079074407 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae50092da0 0x7fae5018e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:08.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.505 7fae56694700 1 --2- 172.21.2.101:0/1079074407 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fae500929a0 0x7fae5018e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:08.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.505 7fae56694700 1 --2- 172.21.2.101:0/1079074407 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae50093490 0x7fae50097040 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:08.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.505 7fae56694700 1 -- 172.21.2.101:0/1079074407 >> 172.21.2.101:0/1079074407 conn(0x7fae50191990 msgr2=0x7fae5008db80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:08.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.505 7fae56694700 1 -- 172.21.2.101:0/1079074407 shutdown_connections 2021-02-10T11:33:08.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.505 7fae56694700 1 -- 172.21.2.101:0/1079074407 wait complete. 2021-02-10T11:33:08.525 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-02-10T11:33:08.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.926 7fb10bd9e700 1 Processor -- start 2021-02-10T11:33:08.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.926 7fb10bd9e700 1 -- start start 2021-02-10T11:33:08.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.927 7fb10bd9e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb1040d1870 0x7fb1041b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:08.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.927 7fb10bd9e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1041b0ae0 0x7fb104108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:08.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.927 7fb10bd9e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1040fe1c0 con 0x7fb1040d1c70 2021-02-10T11:33:08.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.927 7fb10bd9e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb104109a60 con 0x7fb1040d1870 2021-02-10T11:33:08.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.927 7fb10bd9e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb104109750 con 0x7fb1041b0ae0 2021-02-10T11:33:08.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.927 7fb10a3bb700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb1040d1c70 0x7fb104108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51920/0 (socket says 172.21.2.101:51920) 2021-02-10T11:33:08.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.927 7fb10a3bb700 1 -- 172.21.2.101:0/3205842541 learned_addr learned my addr 172.21.2.101:0/3205842541 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:08.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb108bb8700 1 -- 172.21.2.101:0/3205842541 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3851247809 0 0) 0x7fb0fc001500 con 0x7fb1040d1c70 2021-02-10T11:33:08.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb108bb8700 1 -- 172.21.2.101:0/3205842541 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0f0002fe0 con 0x7fb1040d1c70 2021-02-10T11:33:08.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb1093b9700 1 --2- 172.21.2.101:0/3205842541 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1041b0ae0 0x7fb104108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:08.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb109bba700 1 --2- 172.21.2.101:0/3205842541 >> v2:172.21.2.108:3300/0 conn(0x7fb1040d1870 0x7fb1041b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:08.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb108bb8700 1 -- 172.21.2.101:0/3205842541 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 648439231 0 0) 0x7fb0fc001460 con 0x7fb1040d1c70 2021-02-10T11:33:08.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb108bb8700 1 -- 172.21.2.101:0/3205842541 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1041b0ae0 msgr2=0x7fb104108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:08.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb108bb8700 1 --2- 172.21.2.101:0/3205842541 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1041b0ae0 0x7fb104108940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:08.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb108bb8700 1 -- 172.21.2.101:0/3205842541 >> v2:172.21.2.108:3300/0 conn(0x7fb1040d1870 msgr2=0x7fb1041b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:08.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb108bb8700 1 --2- 172.21.2.101:0/3205842541 >> v2:172.21.2.108:3300/0 conn(0x7fb1040d1870 0x7fb1041b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:08.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb108bb8700 1 -- 172.21.2.101:0/3205842541 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb104088020 con 0x7fb1040d1c70 2021-02-10T11:33:08.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb108bb8700 1 -- 172.21.2.101:0/3205842541 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fb0fc001980 con 0x7fb1040d1c70 2021-02-10T11:33:08.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb108bb8700 1 -- 172.21.2.101:0/3205842541 >> v1:172.21.2.101:6789/0 conn(0x7fb1040d1c70 legacy=0x7fb104108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:08.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb108bb8700 1 --2- 172.21.2.101:0/3205842541 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0f0003a60 0x7fb0f0005e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:08.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb108bb8700 1 --2- 172.21.2.101:0/3205842541 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0f004a740 0x7fb0f004cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:08.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb108bb8700 1 --2- 172.21.2.101:0/3205842541 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0f004d500 0x7fb0f004fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:08.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb108bb8700 1 -- 172.21.2.101:0/3205842541 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00502d0 con 0x7fb0f004d500 2021-02-10T11:33:08.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb108bb8700 1 -- 172.21.2.101:0/3205842541 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00504f0 con 0x7fb0f004a740 2021-02-10T11:33:08.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb108bb8700 1 -- 172.21.2.101:0/3205842541 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f0050710 con 0x7fb0f0003a60 2021-02-10T11:33:08.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb10a3bb700 1 --2- 172.21.2.101:0/3205842541 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0f004d500 0x7fb0f004fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:08.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb109bba700 1 --2- 172.21.2.101:0/3205842541 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0f0003a60 0x7fb0f0005e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:08.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb1093b9700 1 --2- 172.21.2.101:0/3205842541 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0f004a740 0x7fb0f004cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:08.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb109bba700 1 -- 172.21.2.101:0/3205842541 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0f004a740 msgr2=0x7fb0f004cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:08.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb109bba700 1 --2- 172.21.2.101:0/3205842541 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0f004a740 0x7fb0f004cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:08.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb109bba700 1 -- 172.21.2.101:0/3205842541 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0f004d500 msgr2=0x7fb0f004fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:08.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb109bba700 1 --2- 172.21.2.101:0/3205842541 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0f004d500 0x7fb0f004fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:08.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb109bba700 1 -- 172.21.2.101:0/3205842541 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f00509b0 con 0x7fb0f0003a60 2021-02-10T11:33:08.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.928 7fb10a3bb700 1 --2- 172.21.2.101:0/3205842541 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0f004d500 0x7fb0f004fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:33:08.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.929 7fb109bba700 1 --2- 172.21.2.101:0/3205842541 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0f0003a60 0x7fb0f0005e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fb10000c8f0 tx=0x7fb10000cc90).ready entity=mon.2 client_cookie=72c96301bbe26291 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:08.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.929 7fb108bb8700 1 -- 172.21.2.101:0/3205842541 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb10000cdd0 con 0x7fb0f0003a60 2021-02-10T11:33:08.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.929 7fb108bb8700 1 -- 172.21.2.101:0/3205842541 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb100009310 con 0x7fb0f0003a60 2021-02-10T11:33:08.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.929 7fb108bb8700 1 -- 172.21.2.101:0/3205842541 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1000097f0 con 0x7fb0f0003a60 2021-02-10T11:33:08.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.929 7fb10bd9e700 1 -- 172.21.2.101:0/3205842541 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0f0003a60 msgr2=0x7fb0f0005e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:08.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.929 7fb10bd9e700 1 --2- 172.21.2.101:0/3205842541 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0f0003a60 0x7fb0f0005e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fb10000c8f0 tx=0x7fb10000cc90).stop 2021-02-10T11:33:08.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.929 7fb10a3bb700 1 -- 172.21.2.101:0/3205842541 reap_dead start 2021-02-10T11:33:08.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.930 7fb10bd9e700 1 -- 172.21.2.101:0/3205842541 shutdown_connections 2021-02-10T11:33:08.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.930 7fb10bd9e700 1 -- 172.21.2.101:0/3205842541 >> 172.21.2.101:0/3205842541 conn(0x7fb1040ce420 msgr2=0x7fb1040d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:08.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.930 7fb10bd9e700 1 -- 172.21.2.101:0/3205842541 shutdown_connections 2021-02-10T11:33:08.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.930 7fb10bd9e700 1 -- 172.21.2.101:0/3205842541 wait complete. 2021-02-10T11:33:08.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.945 7fb10bd9e700 1 Processor -- start 2021-02-10T11:33:08.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.945 7fb10bd9e700 1 -- start start 2021-02-10T11:33:08.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.945 7fb10bd9e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1040b8070 0x7fb1040b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:08.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.945 7fb10bd9e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1040ba4a0 0x7fb1040ba8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:08.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.945 7fb10bd9e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1041a2fd0 0x7fb1041a33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:08.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.945 7fb10bd9e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1040baec0 con 0x7fb1041a2fd0 2021-02-10T11:33:08.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.945 7fb10bd9e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1041a39f0 con 0x7fb1040b8070 2021-02-10T11:33:08.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.945 7fb10bd9e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1041a3c10 con 0x7fb1040ba4a0 2021-02-10T11:33:08.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.945 7fb10a3bb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1041a2fd0 0x7fb1041a33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:08.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.945 7fb10a3bb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1041a2fd0 0x7fb1041a33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51436/0 (socket says 172.21.2.101:51436) 2021-02-10T11:33:08.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.945 7fb10a3bb700 1 -- 172.21.2.101:0/1261943802 learned_addr learned my addr 172.21.2.101:0/1261943802 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:08.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.945 7fb10a3bb700 1 -- 172.21.2.101:0/1261943802 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1040ba4a0 msgr2=0x7fb1040ba8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:08.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.945 7fb10a3bb700 1 --2- 172.21.2.101:0/1261943802 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1040ba4a0 0x7fb1040ba8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:08.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.945 7fb10a3bb700 1 -- 172.21.2.101:0/1261943802 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1040b8070 msgr2=0x7fb1040b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:08.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.945 7fb10a3bb700 1 --2- 172.21.2.101:0/1261943802 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1040b8070 0x7fb1040b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:08.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.945 7fb10a3bb700 1 -- 172.21.2.101:0/1261943802 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1040ad700 con 0x7fb1041a2fd0 2021-02-10T11:33:08.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.946 7fb10a3bb700 1 --2- 172.21.2.101:0/1261943802 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1041a2fd0 0x7fb1041a33d0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 rx=0x7fb0fc00a8f0 tx=0x7fb0fc00ac90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:08.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.946 7fb0f67fc700 1 -- 172.21.2.101:0/1261943802 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0fc0070d0 con 0x7fb1041a2fd0 2021-02-10T11:33:08.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.946 7fb0f67fc700 1 -- 172.21.2.101:0/1261943802 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc007350 con 0x7fb1041a2fd0 2021-02-10T11:33:08.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.946 7fb0f67fc700 1 -- 172.21.2.101:0/1261943802 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0fc0049a0 con 0x7fb1041a2fd0 2021-02-10T11:33:08.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.946 7fb10bd9e700 1 -- 172.21.2.101:0/1261943802 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1040ad700 con 0x7fb1041a2fd0 2021-02-10T11:33:08.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.946 7fb10bd9e700 1 -- 172.21.2.101:0/1261943802 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1040ad700 con 0x7fb1041a2fd0 2021-02-10T11:33:08.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.946 7fb0f67fc700 1 -- 172.21.2.101:0/1261943802 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb0fc001750 con 0x7fb1041a2fd0 2021-02-10T11:33:08.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.946 7fb0f67fc700 1 --2- 172.21.2.101:0/1261943802 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0ec0186c0 0x7fb0ec01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:08.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.946 7fb0f67fc700 1 -- 172.21.2.101:0/1261943802 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fb0fc032940 con 0x7fb1041a2fd0 2021-02-10T11:33:08.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.947 7fb109bba700 1 --2- 172.21.2.101:0/1261943802 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0ec0186c0 0x7fb0ec01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:08.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.947 7fb109bba700 1 --2- 172.21.2.101:0/1261943802 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0ec0186c0 0x7fb0ec01abc0 crc :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:08.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.977 7fb10bd9e700 1 -- 172.21.2.101:0/1261943802 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1040828e0 con 0x7fb1041a2fd0 2021-02-10T11:33:08.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:08.980 7fb0f67fc700 1 -- 172.21.2.101:0/1261943802 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb0fc001bc0 con 0x7fb1041a2fd0 2021-02-10T11:33:09.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.106 7fb10bd9e700 1 -- 172.21.2.101:0/1261943802 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb1040828e0 con 0x7fb1041a2fd0 2021-02-10T11:33:09.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.463 7fb0f67fc700 1 -- 172.21.2.101:0/1261943802 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v49) v1 ==== 185+0+0 (secure 0 0 0) 0x7fb0fc004720 con 0x7fb1041a2fd0 2021-02-10T11:33:09.482 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' removed 2021-02-10T11:33:09.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.473 7fb10bd9e700 1 -- 172.21.2.101:0/1261943802 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0ec0186c0 msgr2=0x7fb0ec01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:09.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.473 7fb10bd9e700 1 --2- 172.21.2.101:0/1261943802 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0ec0186c0 0x7fb0ec01abc0 crc :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:09.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.473 7fb10bd9e700 1 -- 172.21.2.101:0/1261943802 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1041a2fd0 msgr2=0x7fb1041a33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:09.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.473 7fb10bd9e700 1 --2- 172.21.2.101:0/1261943802 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1041a2fd0 0x7fb1041a33d0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 rx=0x7fb0fc00a8f0 tx=0x7fb0fc00ac90).stop 2021-02-10T11:33:09.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.474 7fb10bd9e700 1 -- 172.21.2.101:0/1261943802 shutdown_connections 2021-02-10T11:33:09.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.474 7fb10bd9e700 1 --2- 172.21.2.101:0/1261943802 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0ec0186c0 0x7fb0ec01abc0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:09.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.474 7fb10bd9e700 1 --2- 172.21.2.101:0/1261943802 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1040ba4a0 0x7fb1040ba8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:09.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.474 7fb10bd9e700 1 --2- 172.21.2.101:0/1261943802 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1041a2fd0 0x7fb1041a33d0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:09.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.474 7fb10bd9e700 1 --2- 172.21.2.101:0/1261943802 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1040b8070 0x7fb1040b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:09.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.474 7fb10bd9e700 1 -- 172.21.2.101:0/1261943802 >> 172.21.2.101:0/1261943802 conn(0x7fb1040b58d0 msgr2=0x7fb1040b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:09.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.474 7fb10bd9e700 1 -- 172.21.2.101:0/1261943802 shutdown_connections 2021-02-10T11:33:09.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.474 7fb10bd9e700 1 -- 172.21.2.101:0/1261943802 wait complete. 2021-02-10T11:33:09.493 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-02-10T11:33:09.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.896 7f1a4ac14700 1 Processor -- start 2021-02-10T11:33:09.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.896 7f1a4ac14700 1 -- start start 2021-02-10T11:33:09.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.896 7f1a4ac14700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1a440d1c60 0x7f1a440cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:09.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.896 7f1a4ac14700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a440cf570 0x7f1a440cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:09.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.896 7f1a4ac14700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a440fe1b0 con 0x7f1a440d1860 2021-02-10T11:33:09.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.896 7f1a4ac14700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a441b0ca0 con 0x7f1a440d1c60 2021-02-10T11:33:09.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.896 7f1a4ac14700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a44109a50 con 0x7f1a440cf570 2021-02-10T11:33:09.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.897 7f1a48a30700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f1a440d1860 0x7f1a441b04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51938/0 (socket says 172.21.2.101:51938) 2021-02-10T11:33:09.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.897 7f1a48a30700 1 -- 172.21.2.101:0/3137432190 learned_addr learned my addr 172.21.2.101:0/3137432190 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:09.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.897 7f1a43fff700 1 --2- 172.21.2.101:0/3137432190 >> v2:172.21.2.101:3300/0 conn(0x7f1a440d1c60 0x7f1a440cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:09.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.897 7f1a43fff700 1 -- 172.21.2.101:0/3137432190 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a440cf570 msgr2=0x7f1a440cf970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:33:09.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.897 7f1a43fff700 1 --2- 172.21.2.101:0/3137432190 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a440cf570 0x7f1a440cf970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:09.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.897 7f1a43fff700 1 -- 172.21.2.101:0/3137432190 >> v1:172.21.2.101:6789/0 conn(0x7f1a440d1860 legacy=0x7f1a441b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:09.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.897 7f1a43fff700 1 -- 172.21.2.101:0/3137432190 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a44088020 con 0x7f1a440d1c60 2021-02-10T11:33:09.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.897 7f1a43fff700 1 --2- 172.21.2.101:0/3137432190 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a440d1c60 0x7f1a440cef50 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7f1a34006650 tx=0x7f1a34009a00).ready entity=mon.0 client_cookie=624cced1380a687d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:09.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.897 7f1a437fe700 1 -- 172.21.2.101:0/3137432190 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1a34009df0 con 0x7f1a440d1c60 2021-02-10T11:33:09.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.897 7f1a437fe700 1 -- 172.21.2.101:0/3137432190 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a340009b0 con 0x7f1a440d1c60 2021-02-10T11:33:09.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.897 7f1a437fe700 1 -- 172.21.2.101:0/3137432190 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1a3400c2b0 con 0x7f1a440d1c60 2021-02-10T11:33:09.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.897 7f1a4ac14700 1 -- 172.21.2.101:0/3137432190 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a440d1c60 msgr2=0x7f1a440cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:09.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.897 7f1a4ac14700 1 --2- 172.21.2.101:0/3137432190 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a440d1c60 0x7f1a440cef50 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7f1a34006650 tx=0x7f1a34009a00).stop 2021-02-10T11:33:09.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.898 7f1a4ac14700 1 -- 172.21.2.101:0/3137432190 shutdown_connections 2021-02-10T11:33:09.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.898 7f1a4ac14700 1 --2- 172.21.2.101:0/3137432190 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a440cf570 0x7f1a440cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:09.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.898 7f1a4ac14700 1 --2- 172.21.2.101:0/3137432190 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a440d1c60 0x7f1a440cef50 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:09.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.898 7f1a4ac14700 1 -- 172.21.2.101:0/3137432190 >> 172.21.2.101:0/3137432190 conn(0x7f1a440ce410 msgr2=0x7f1a440d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:09.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.898 7f1a4ac14700 1 -- 172.21.2.101:0/3137432190 shutdown_connections 2021-02-10T11:33:09.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.898 7f1a4ac14700 1 -- 172.21.2.101:0/3137432190 wait complete. 2021-02-10T11:33:09.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.913 7f1a4ac14700 1 Processor -- start 2021-02-10T11:33:09.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.913 7f1a4ac14700 1 -- start start 2021-02-10T11:33:09.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.913 7f1a4ac14700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a44083ae0 0x7f1a44083ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:09.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.913 7f1a4ac14700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a44096d30 0x7f1a44097130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:09.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.913 7f1a4ac14700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a44097750 0x7f1a441aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:09.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.913 7f1a4ac14700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a441a5fb0 con 0x7f1a44096d30 2021-02-10T11:33:09.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.913 7f1a4ac14700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a441a8f50 con 0x7f1a44097750 2021-02-10T11:33:09.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.913 7f1a4ac14700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a44084500 con 0x7f1a44083ae0 2021-02-10T11:33:09.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.913 7f1a43fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a44096d30 0x7f1a44097130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:09.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.913 7f1a43fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a44096d30 0x7f1a44097130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51452/0 (socket says 172.21.2.101:51452) 2021-02-10T11:33:09.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.913 7f1a43fff700 1 -- 172.21.2.101:0/369741097 learned_addr learned my addr 172.21.2.101:0/369741097 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:09.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.913 7f1a48a30700 1 --2- 172.21.2.101:0/369741097 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a44083ae0 0x7f1a44083ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:09.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.913 7f1a49231700 1 --2- 172.21.2.101:0/369741097 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a44097750 0x7f1a441aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:09.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.914 7f1a43fff700 1 -- 172.21.2.101:0/369741097 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a44083ae0 msgr2=0x7f1a44083ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:09.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.914 7f1a43fff700 1 --2- 172.21.2.101:0/369741097 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a44083ae0 0x7f1a44083ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:09.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.914 7f1a43fff700 1 -- 172.21.2.101:0/369741097 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a44097750 msgr2=0x7f1a441aafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:09.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.914 7f1a43fff700 1 --2- 172.21.2.101:0/369741097 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a44097750 0x7f1a441aafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:09.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.914 7f1a43fff700 1 -- 172.21.2.101:0/369741097 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a441ab5f0 con 0x7f1a44096d30 2021-02-10T11:33:09.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.914 7f1a43fff700 1 --2- 172.21.2.101:0/369741097 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a44096d30 0x7f1a44097130 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7f1a340096a0 tx=0x7f1a3400b9b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:09.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.914 7f1a417fa700 1 -- 172.21.2.101:0/369741097 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1a340093a0 con 0x7f1a44096d30 2021-02-10T11:33:09.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.914 7f1a417fa700 1 -- 172.21.2.101:0/369741097 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a340067f0 con 0x7f1a44096d30 2021-02-10T11:33:09.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.914 7f1a417fa700 1 -- 172.21.2.101:0/369741097 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1a34002220 con 0x7f1a44096d30 2021-02-10T11:33:09.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.914 7f1a4ac14700 1 -- 172.21.2.101:0/369741097 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a441ab5f0 con 0x7f1a44096d30 2021-02-10T11:33:09.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.914 7f1a4ac14700 1 -- 172.21.2.101:0/369741097 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a441ab5f0 con 0x7f1a44096d30 2021-02-10T11:33:09.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.915 7f1a417fa700 1 -- 172.21.2.101:0/369741097 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1a340052e0 con 0x7f1a44096d30 2021-02-10T11:33:09.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.915 7f1a417fa700 1 --2- 172.21.2.101:0/369741097 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1a280186c0 0x7f1a2801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:09.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.915 7f1a417fa700 1 -- 172.21.2.101:0/369741097 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f1a34032380 con 0x7f1a44096d30 2021-02-10T11:33:09.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.915 7f1a48a30700 1 --2- 172.21.2.101:0/369741097 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1a280186c0 0x7f1a2801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:09.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.915 7f1a48a30700 1 --2- 172.21.2.101:0/369741097 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1a280186c0 0x7f1a2801abc0 crc :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:09.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.946 7f1a4ac14700 1 -- 172.21.2.101:0/369741097 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a441a6320 con 0x7f1a44096d30 2021-02-10T11:33:09.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:09.949 7f1a417fa700 1 -- 172.21.2.101:0/369741097 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1a3402cbd0 con 0x7f1a44096d30 2021-02-10T11:33:10.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.076 7f1a4ac14700 1 -- 172.21.2.101:0/369741097 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1a441a6320 con 0x7f1a44096d30 2021-02-10T11:33:10.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.468 7f1a417fa700 1 -- 172.21.2.101:0/369741097 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) v1 ==== 173+0+0 (secure 0 0 0) 0x7f1a3402cbd0 con 0x7f1a44096d30 2021-02-10T11:33:10.504 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' removed 2021-02-10T11:33:10.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.494 7f1a4ac14700 1 -- 172.21.2.101:0/369741097 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1a280186c0 msgr2=0x7f1a2801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:10.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.494 7f1a4ac14700 1 --2- 172.21.2.101:0/369741097 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1a280186c0 0x7f1a2801abc0 crc :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:10.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.494 7f1a4ac14700 1 -- 172.21.2.101:0/369741097 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a44096d30 msgr2=0x7f1a44097130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:10.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.494 7f1a4ac14700 1 --2- 172.21.2.101:0/369741097 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a44096d30 0x7f1a44097130 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7f1a340096a0 tx=0x7f1a3400b9b0).stop 2021-02-10T11:33:10.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.494 7f1a4ac14700 1 -- 172.21.2.101:0/369741097 shutdown_connections 2021-02-10T11:33:10.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.494 7f1a4ac14700 1 --2- 172.21.2.101:0/369741097 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a44097750 0x7f1a441aafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:10.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.494 7f1a4ac14700 1 --2- 172.21.2.101:0/369741097 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a44096d30 0x7f1a44097130 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:10.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.494 7f1a4ac14700 1 --2- 172.21.2.101:0/369741097 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1a280186c0 0x7f1a2801abc0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:10.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.494 7f1a4ac14700 1 --2- 172.21.2.101:0/369741097 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a44083ae0 0x7f1a44083ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:10.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.494 7f1a4ac14700 1 -- 172.21.2.101:0/369741097 >> 172.21.2.101:0/369741097 conn(0x7f1a4419c510 msgr2=0x7f1a44082b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:10.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.494 7f1a4ac14700 1 -- 172.21.2.101:0/369741097 shutdown_connections 2021-02-10T11:33:10.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.494 7f1a4ac14700 1 -- 172.21.2.101:0/369741097 wait complete. 2021-02-10T11:33:10.514 INFO:tasks.ceph.mds.a:Restarting daemon 2021-02-10T11:33:10.514 INFO:tasks.ceph.mds.a:Stopping old one... 2021-02-10T11:33:10.514 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-02-10T11:33:10.515 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:33:10.515 INFO:tasks.ceph.mds.b:Restarting daemon 2021-02-10T11:33:10.515 INFO:tasks.ceph.mds.b:Stopping old one... 2021-02-10T11:33:10.515 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-02-10T11:33:10.515 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:33:10.521 INFO:tasks.ceph.mds.a:Stopped 2021-02-10T11:33:10.521 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-02-10T11:33:10.522 INFO:tasks.ceph.mds.b:Stopped 2021-02-10T11:33:10.522 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-02-10T11:33:10.563 INFO:tasks.ceph.mds.a:Started 2021-02-10T11:33:10.563 DEBUG:teuthology.parallel:result is None 2021-02-10T11:33:10.564 INFO:tasks.ceph.mds.b:Started 2021-02-10T11:33:10.564 DEBUG:teuthology.parallel:result is None 2021-02-10T11:33:10.564 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd blacklist clear 2021-02-10T11:33:10.968 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:33:10.971 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:33:10.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.967 7f6c2615f700 1 Processor -- start 2021-02-10T11:33:10.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.967 7f6c2615f700 1 -- start start 2021-02-10T11:33:10.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.968 7f6c2615f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6c200e5470 0x7f6c2007e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:10.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.968 7f6c2615f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2007ed80 con 0x7f6c2007f300 2021-02-10T11:33:10.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.968 7f6c2615f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c201b4010 con 0x7f6c200e5070 2021-02-10T11:33:10.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.968 7f6c2615f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c200e5fe0 con 0x7f6c200e5470 2021-02-10T11:33:10.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.968 7f6c1ffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f6c2007f300 0x7f6c2007f700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51966/0 (socket says 172.21.2.101:51966) 2021-02-10T11:33:10.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.968 7f6c1ffff700 1 -- 172.21.2.101:0/451146025 learned_addr learned my addr 172.21.2.101:0/451146025 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:10.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.968 7f6c1effd700 1 --2- 172.21.2.101:0/451146025 >> v2:172.21.2.101:3300/0 conn(0x7f6c200e5470 0x7f6c2007e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:10.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.968 7f6c1effd700 1 -- 172.21.2.101:0/451146025 >> v1:172.21.2.108:6789/0 conn(0x7f6c200e5070 legacy=0x7f6c201b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:10.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.968 7f6c1effd700 1 -- 172.21.2.101:0/451146025 >> v1:172.21.2.101:6789/0 conn(0x7f6c2007f300 legacy=0x7f6c2007f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:10.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.968 7f6c1e7fc700 1 -- 172.21.2.101:0/451146025 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358855703 0 0) 0x7f6c18000f90 con 0x7f6c2007f300 2021-02-10T11:33:10.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.968 7f6c1effd700 1 -- 172.21.2.101:0/451146025 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c200d1790 con 0x7f6c200e5470 2021-02-10T11:33:10.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.969 7f6c1effd700 1 --2- 172.21.2.101:0/451146025 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c200e5470 0x7f6c2007e6a0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f6c10006650 tx=0x7f6c10009a00).ready entity=mon.0 client_cookie=b80acaa238d28472 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:10.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.969 7f6c1e7fc700 1 -- 172.21.2.101:0/451146025 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6c100009b0 con 0x7f6c200e5470 2021-02-10T11:33:10.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.969 7f6c1e7fc700 1 -- 172.21.2.101:0/451146025 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1000c0c0 con 0x7f6c200e5470 2021-02-10T11:33:10.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.969 7f6c1e7fc700 1 -- 172.21.2.101:0/451146025 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6c1000c5a0 con 0x7f6c200e5470 2021-02-10T11:33:10.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.969 7f6c2615f700 1 -- 172.21.2.101:0/451146025 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c200e5470 msgr2=0x7f6c2007e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:10.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.969 7f6c2615f700 1 --2- 172.21.2.101:0/451146025 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c200e5470 0x7f6c2007e6a0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f6c10006650 tx=0x7f6c10009a00).stop 2021-02-10T11:33:10.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.969 7f6c2615f700 1 -- 172.21.2.101:0/451146025 shutdown_connections 2021-02-10T11:33:10.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.969 7f6c2615f700 1 --2- 172.21.2.101:0/451146025 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c200e5470 0x7f6c2007e6a0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:10.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.969 7f6c2615f700 1 -- 172.21.2.101:0/451146025 >> 172.21.2.101:0/451146025 conn(0x7f6c200ce690 msgr2=0x7f6c200e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:10.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.969 7f6c2615f700 1 -- 172.21.2.101:0/451146025 shutdown_connections 2021-02-10T11:33:10.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.970 7f6c2615f700 1 -- 172.21.2.101:0/451146025 wait complete. 2021-02-10T11:33:10.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.984 7f6c2615f700 1 Processor -- start 2021-02-10T11:33:10.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.984 7f6c2615f700 1 -- start start 2021-02-10T11:33:10.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.984 7f6c2615f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c201a6680 0x7f6c201a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:10.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.984 7f6c2615f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c200a4640 0x7f6c200a78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:10.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.984 7f6c2615f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c200a7ef0 0x7f6c200a82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:10.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.984 7f6c2615f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c200857f0 con 0x7f6c200a7ef0 2021-02-10T11:33:10.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.984 7f6c2615f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c200a8a80 con 0x7f6c201a6680 2021-02-10T11:33:10.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.984 7f6c2615f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c20097360 con 0x7f6c200a4640 2021-02-10T11:33:10.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.985 7f6c1effd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c200a4640 0x7f6c200a78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:10.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.985 7f6c1effd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c200a4640 0x7f6c200a78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:44208/0 (socket says 172.21.2.101:44208) 2021-02-10T11:33:10.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.985 7f6c1f7fe700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c201a6680 0x7f6c201a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:11.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.985 7f6c1effd700 1 -- 172.21.2.101:0/45269598 learned_addr learned my addr 172.21.2.101:0/45269598 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:11.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.985 7f6c1ffff700 1 --2- 172.21.2.101:0/45269598 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c200a7ef0 0x7f6c200a82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:11.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.985 7f6c1f7fe700 1 -- 172.21.2.101:0/45269598 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c200a4640 msgr2=0x7f6c200a78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:11.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.985 7f6c1f7fe700 1 --2- 172.21.2.101:0/45269598 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c200a4640 0x7f6c200a78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:11.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.985 7f6c1f7fe700 1 -- 172.21.2.101:0/45269598 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c200a7ef0 msgr2=0x7f6c200a82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:11.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.985 7f6c1f7fe700 1 --2- 172.21.2.101:0/45269598 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c200a7ef0 0x7f6c200a82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:11.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.985 7f6c1f7fe700 1 -- 172.21.2.101:0/45269598 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c200a9000 con 0x7f6c201a6680 2021-02-10T11:33:11.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.985 7f6c1effd700 1 --2- 172.21.2.101:0/45269598 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c200a4640 0x7f6c200a78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:33:11.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.985 7f6c1f7fe700 1 --2- 172.21.2.101:0/45269598 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c201a6680 0x7f6c201a6a80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f6c14006e90 tx=0x7f6c1400b860).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:11.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.985 7f6c07fff700 1 -- 172.21.2.101:0/45269598 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6c14006340 con 0x7f6c201a6680 2021-02-10T11:33:11.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.985 7f6c07fff700 1 -- 172.21.2.101:0/45269598 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1400bc50 con 0x7f6c201a6680 2021-02-10T11:33:11.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.985 7f6c07fff700 1 -- 172.21.2.101:0/45269598 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6c14002220 con 0x7f6c201a6680 2021-02-10T11:33:11.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.985 7f6c2615f700 1 -- 172.21.2.101:0/45269598 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c200a9000 con 0x7f6c201a6680 2021-02-10T11:33:11.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.986 7f6c2615f700 1 -- 172.21.2.101:0/45269598 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c200a9000 con 0x7f6c201a6680 2021-02-10T11:33:11.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.986 7f6c07fff700 1 -- 172.21.2.101:0/45269598 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6c140052e0 con 0x7f6c201a6680 2021-02-10T11:33:11.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.986 7f6c07fff700 1 --2- 172.21.2.101:0/45269598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c000186d0 0x7f6c0001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:11.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.986 7f6c07fff700 1 -- 172.21.2.101:0/45269598 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4839+0+0 (secure 0 0 0) 0x7f6c14032330 con 0x7f6c201a6680 2021-02-10T11:33:11.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.986 7f6c1effd700 1 --2- 172.21.2.101:0/45269598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c000186d0 0x7f6c0001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:11.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:10.986 7f6c1effd700 1 --2- 172.21.2.101:0/45269598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c000186d0 0x7f6c0001abd0 crc :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:11.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.016 7f6c2615f700 1 -- 172.21.2.101:0/45269598 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c20097360 con 0x7f6c201a6680 2021-02-10T11:33:11.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.019 7f6c07fff700 1 -- 172.21.2.101:0/45269598 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6c14000ca0 con 0x7f6c201a6680 2021-02-10T11:33:11.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.159 7f6c2615f700 1 -- 172.21.2.101:0/45269598 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f6c20097360 con 0x7f6c201a6680 2021-02-10T11:33:11.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.472 7f6c07fff700 1 -- 172.21.2.101:0/45269598 <== mon.1 v2:172.21.2.108:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v51) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6c140052e0 con 0x7f6c201a6680 2021-02-10T11:33:11.485 INFO:teuthology.orchestra.run.gibba001.stderr: removed all blacklist entries 2021-02-10T11:33:11.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.476 7f6c2615f700 1 -- 172.21.2.101:0/45269598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c000186d0 msgr2=0x7f6c0001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:11.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.476 7f6c2615f700 1 --2- 172.21.2.101:0/45269598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c000186d0 0x7f6c0001abd0 crc :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:11.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.476 7f6c2615f700 1 -- 172.21.2.101:0/45269598 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c201a6680 msgr2=0x7f6c201a6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:11.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.476 7f6c2615f700 1 --2- 172.21.2.101:0/45269598 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c201a6680 0x7f6c201a6a80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f6c14006e90 tx=0x7f6c1400b860).stop 2021-02-10T11:33:11.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.476 7f6c2615f700 1 -- 172.21.2.101:0/45269598 shutdown_connections 2021-02-10T11:33:11.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.476 7f6c2615f700 1 --2- 172.21.2.101:0/45269598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c000186d0 0x7f6c0001abd0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:11.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.476 7f6c2615f700 1 --2- 172.21.2.101:0/45269598 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c200a4640 0x7f6c200a78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:11.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.476 7f6c2615f700 1 --2- 172.21.2.101:0/45269598 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c200a7ef0 0x7f6c200a82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:11.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.476 7f6c2615f700 1 --2- 172.21.2.101:0/45269598 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c201a6680 0x7f6c201a6a80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:11.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.476 7f6c2615f700 1 -- 172.21.2.101:0/45269598 >> 172.21.2.101:0/45269598 conn(0x7f6c200bdea0 msgr2=0x7f6c200976f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:11.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.476 7f6c2615f700 1 -- 172.21.2.101:0/45269598 shutdown_connections 2021-02-10T11:33:11.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.476 7f6c2615f700 1 -- 172.21.2.101:0/45269598 wait complete. 2021-02-10T11:33:11.495 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2021-02-10T11:33:11.496 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:33:11.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.895 7fb3df6c6700 1 Processor -- start 2021-02-10T11:33:11.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.895 7fb3df6c6700 1 -- start start 2021-02-10T11:33:11.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.895 7fb3df6c6700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb3d80e5070 0x7fb3d81b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:11.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.896 7fb3df6c6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb3d80e5470 0x7fb3d8080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:11.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.896 7fb3df6c6700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d807ed80 con 0x7fb3d81b3e40 2021-02-10T11:33:11.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.896 7fb3df6c6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d80d1790 con 0x7fb3d80e5470 2021-02-10T11:33:11.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.896 7fb3df6c6700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d80d19b0 con 0x7fb3d80e5070 2021-02-10T11:33:11.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.896 7fb3dcce1700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb3d81b3e40 0x7fb3d807d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:51976/0 (socket says 172.21.2.101:51976) 2021-02-10T11:33:11.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.896 7fb3ddce3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb3d80e5470 0x7fb3d8080310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:11.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.896 7fb3dcce1700 1 -- 172.21.2.101:0/1376840802 learned_addr learned my addr 172.21.2.101:0/1376840802 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:11.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.896 7fb3ddce3700 1 -- 172.21.2.101:0/1376840802 >> v2:172.21.2.108:3300/0 conn(0x7fb3d80e5070 msgr2=0x7fb3d81b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:11.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.896 7fb3ddce3700 1 --2- 172.21.2.101:0/1376840802 >> v2:172.21.2.108:3300/0 conn(0x7fb3d80e5070 0x7fb3d81b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:11.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.896 7fb3ddce3700 1 -- 172.21.2.101:0/1376840802 >> v1:172.21.2.101:6789/0 conn(0x7fb3d81b3e40 legacy=0x7fb3d807d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:11.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.896 7fb3ddce3700 1 -- 172.21.2.101:0/1376840802 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d80d1bd0 con 0x7fb3d80e5470 2021-02-10T11:33:11.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.896 7fb3cbfff700 1 -- 172.21.2.101:0/1376840802 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1089513577 0 0) 0x7fb3cc0010b0 con 0x7fb3d81b3e40 2021-02-10T11:33:11.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.896 7fb3ddce3700 1 --2- 172.21.2.101:0/1376840802 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3d80e5470 0x7fb3d8080310 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7fb3d0006e30 tx=0x7fb3d0009de0).ready entity=mon.0 client_cookie=ac51020a437872d5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:11.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.897 7fb3cbfff700 1 -- 172.21.2.101:0/1376840802 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb3d000b300 con 0x7fb3d80e5470 2021-02-10T11:33:11.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.897 7fb3cbfff700 1 -- 172.21.2.101:0/1376840802 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d000b680 con 0x7fb3d80e5470 2021-02-10T11:33:11.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.897 7fb3cbfff700 1 -- 172.21.2.101:0/1376840802 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb3d000bba0 con 0x7fb3d80e5470 2021-02-10T11:33:11.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.897 7fb3df6c6700 1 -- 172.21.2.101:0/1376840802 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3d80e5470 msgr2=0x7fb3d8080310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:11.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.897 7fb3df6c6700 1 --2- 172.21.2.101:0/1376840802 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3d80e5470 0x7fb3d8080310 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7fb3d0006e30 tx=0x7fb3d0009de0).stop 2021-02-10T11:33:11.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.897 7fb3df6c6700 1 -- 172.21.2.101:0/1376840802 shutdown_connections 2021-02-10T11:33:11.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.897 7fb3df6c6700 1 --2- 172.21.2.101:0/1376840802 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3d80e5470 0x7fb3d8080310 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:11.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.897 7fb3df6c6700 1 --2- 172.21.2.101:0/1376840802 >> v2:172.21.2.108:3300/0 conn(0x7fb3d80e5070 0x7fb3d81b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:11.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.897 7fb3df6c6700 1 -- 172.21.2.101:0/1376840802 >> 172.21.2.101:0/1376840802 conn(0x7fb3d80ce690 msgr2=0x7fb3d80e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:11.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.898 7fb3df6c6700 1 -- 172.21.2.101:0/1376840802 shutdown_connections 2021-02-10T11:33:11.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.898 7fb3df6c6700 1 -- 172.21.2.101:0/1376840802 wait complete. 2021-02-10T11:33:11.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.912 7fb3df6c6700 1 Processor -- start 2021-02-10T11:33:11.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.912 7fb3df6c6700 1 -- start start 2021-02-10T11:33:11.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.913 7fb3df6c6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3d818c700 0x7fb3d8096c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:11.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.913 7fb3df6c6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb3d80ae400 0x7fb3d80ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:11.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.913 7fb3df6c6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb3d8192380 0x7fb3d8192780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:11.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.913 7fb3df6c6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d8097290 con 0x7fb3d818c700 2021-02-10T11:33:11.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.913 7fb3df6c6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d8192da0 con 0x7fb3d8192380 2021-02-10T11:33:11.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.913 7fb3df6c6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d8192f80 con 0x7fb3d80ae400 2021-02-10T11:33:11.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.913 7fb3dd4e2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3d818c700 0x7fb3d8096c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:11.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.913 7fb3dd4e2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3d818c700 0x7fb3d8096c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51486/0 (socket says 172.21.2.101:51486) 2021-02-10T11:33:11.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.913 7fb3dd4e2700 1 -- 172.21.2.101:0/2733416014 learned_addr learned my addr 172.21.2.101:0/2733416014 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:11.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.913 7fb3dd4e2700 1 -- 172.21.2.101:0/2733416014 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb3d80ae400 msgr2=0x7fb3d80ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:11.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.913 7fb3dd4e2700 1 --2- 172.21.2.101:0/2733416014 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb3d80ae400 0x7fb3d80ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:11.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.913 7fb3dd4e2700 1 -- 172.21.2.101:0/2733416014 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb3d8192380 msgr2=0x7fb3d8192780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:11.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.913 7fb3dd4e2700 1 --2- 172.21.2.101:0/2733416014 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb3d8192380 0x7fb3d8192780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:11.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.913 7fb3dd4e2700 1 -- 172.21.2.101:0/2733416014 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d81931a0 con 0x7fb3d818c700 2021-02-10T11:33:11.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.913 7fb3dd4e2700 1 --2- 172.21.2.101:0/2733416014 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3d818c700 0x7fb3d8096c70 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7fb3d4006f00 tx=0x7fb3d400b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:11.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.914 7fb3c9ffb700 1 -- 172.21.2.101:0/2733416014 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb3d400bd30 con 0x7fb3d818c700 2021-02-10T11:33:11.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.914 7fb3c9ffb700 1 -- 172.21.2.101:0/2733416014 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d4006c20 con 0x7fb3d818c700 2021-02-10T11:33:11.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.914 7fb3c9ffb700 1 -- 172.21.2.101:0/2733416014 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb3d4002020 con 0x7fb3d818c700 2021-02-10T11:33:11.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.914 7fb3df6c6700 1 -- 172.21.2.101:0/2733416014 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d81931a0 con 0x7fb3d818c700 2021-02-10T11:33:11.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.914 7fb3df6c6700 1 -- 172.21.2.101:0/2733416014 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d807f510 con 0x7fb3d818c700 2021-02-10T11:33:11.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.914 7fb3c9ffb700 1 -- 172.21.2.101:0/2733416014 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb3d4005310 con 0x7fb3d818c700 2021-02-10T11:33:11.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.914 7fb3c9ffb700 1 --2- 172.21.2.101:0/2733416014 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb3b80186c0 0x7fb3b801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:11.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.914 7fb3c9ffb700 1 -- 172.21.2.101:0/2733416014 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fb3d4032230 con 0x7fb3d818c700 2021-02-10T11:33:11.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.915 7fb3dcce1700 1 --2- 172.21.2.101:0/2733416014 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb3b80186c0 0x7fb3b801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:11.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.915 7fb3dcce1700 1 --2- 172.21.2.101:0/2733416014 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb3b80186c0 0x7fb3b801abc0 crc :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:11.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.946 7fb3df6c6700 1 -- 172.21.2.101:0/2733416014 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d8107120 con 0x7fb3d818c700 2021-02-10T11:33:11.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:11.948 7fb3c9ffb700 1 -- 172.21.2.101:0/2733416014 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb3d4026030 con 0x7fb3d818c700 2021-02-10T11:33:12.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.078 7fb3df6c6700 1 -- 172.21.2.101:0/2733416014 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb3d807ed80 con 0x7fb3d818c700 2021-02-10T11:33:12.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.079 7fb3c9ffb700 1 -- 172.21.2.101:0/2733416014 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v30) v1 ==== 80+0+4513 (secure 0 0 0) 0x7fb3d4026030 con 0x7fb3d818c700 2021-02-10T11:33:12.090 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:12.091 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:33:12.091 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:33:12.091 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:12.091 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:33:12.091 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:33:12.091 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:12.091 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:33:12.091 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:33:12.091 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:33:12.091 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:12.091 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:12.091 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:12.091 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:12.092 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:33:12.092 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:33:12.092 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:12.092 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:33:12.092 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:33:12.092 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:33:12.092 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:12.092 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:12.092 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:12.092 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:12.092 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:33:12.092 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:33:12.092 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:12.092 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:33:12.093 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:33:12.093 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:12.093 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:12.093 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:12.093 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:12.093 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:33:12.093 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:33:12.093 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:12.093 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:33:12.093 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:33:12.093 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:12.093 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:12.093 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:12.094 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:12.094 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:33:12.094 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:33:12.094 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:12.094 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:33:12.094 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:33:12.094 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:12.094 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:12.094 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:12.094 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:12.094 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:33:12.094 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:33:12.094 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:12.095 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:33:12.095 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:33:12.095 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:12.095 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:12.095 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:12.095 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:12.095 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:33:12.095 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:33:12.095 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:12.095 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:33:12.095 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:33:12.095 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:12.095 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:12.096 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:12.096 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:12.096 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:33:12.096 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:33:12.096 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:12.096 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:33:12.096 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:33:12.096 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:12.096 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:12.096 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:12.096 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:12.096 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:33:12.096 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:33:12.096 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:12.097 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:33:12.097 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:33:12.097 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:12.097 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:12.097 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:12.097 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:12.097 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:33:12.097 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:33:12.097 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:12.097 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:33:12.097 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:33:12.097 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:12.097 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:12.097 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:12.097 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:12.098 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:33:12.098 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:33:12.098 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:12.098 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:33:12.098 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:33:12.098 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:33:12.098 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:12.098 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:12.098 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:12.098 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:33:12.098 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:33:12.098 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:12.098 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:33:12.098 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:33:12.099 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:33:12.099 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:12.099 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:12.099 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:12.099 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:33:12.099 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:33:12.099 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:12.099 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:33:12.099 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:33:12.099 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:33:12.099 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:12.099 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:12.099 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:12.099 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:33:12.100 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:33:12.100 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:12.100 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:33:12.100 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:33:12.100 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:33:12.100 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:12.100 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:12.100 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:12.100 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:33:12.100 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:33:12.100 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:12.100 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:33:12.100 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:33:12.100 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:33:12.101 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:12.101 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:12.101 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:12.101 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:12.101 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:33:12.101 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:33:12.101 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:12.101 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:33:12.101 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:33:12.101 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:12.101 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:12.101 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:12.101 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:12.101 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:33:12.101 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:33:12.101 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:12.102 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:33:12.102 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:33:12.102 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:12.102 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:12.102 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:12.102 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:12.102 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:33:12.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.081 7fb3df6c6700 1 -- 172.21.2.101:0/2733416014 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb3b80186c0 msgr2=0x7fb3b801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:12.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.081 7fb3df6c6700 1 --2- 172.21.2.101:0/2733416014 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb3b80186c0 0x7fb3b801abc0 crc :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:12.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.081 7fb3df6c6700 1 -- 172.21.2.101:0/2733416014 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3d818c700 msgr2=0x7fb3d8096c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:12.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.081 7fb3df6c6700 1 --2- 172.21.2.101:0/2733416014 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3d818c700 0x7fb3d8096c70 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7fb3d4006f00 tx=0x7fb3d400b940).stop 2021-02-10T11:33:12.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.081 7fb3df6c6700 1 -- 172.21.2.101:0/2733416014 shutdown_connections 2021-02-10T11:33:12.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.081 7fb3df6c6700 1 --2- 172.21.2.101:0/2733416014 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb3b80186c0 0x7fb3b801abc0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:12.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.081 7fb3df6c6700 1 --2- 172.21.2.101:0/2733416014 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb3d80ae400 0x7fb3d80ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:12.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.081 7fb3df6c6700 1 --2- 172.21.2.101:0/2733416014 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb3d8192380 0x7fb3d8192780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:12.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.081 7fb3df6c6700 1 --2- 172.21.2.101:0/2733416014 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3d818c700 0x7fb3d8096c70 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:12.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.081 7fb3df6c6700 1 -- 172.21.2.101:0/2733416014 >> 172.21.2.101:0/2733416014 conn(0x7fb3d80891f0 msgr2=0x7fb3d81a0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:12.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.081 7fb3df6c6700 1 -- 172.21.2.101:0/2733416014 shutdown_connections 2021-02-10T11:33:12.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.081 7fb3df6c6700 1 -- 172.21.2.101:0/2733416014 wait complete. 2021-02-10T11:33:12.104 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:33:12.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.504 7f1a06126700 1 Processor -- start 2021-02-10T11:33:12.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.504 7f1a06126700 1 -- start start 2021-02-10T11:33:12.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.505 7f1a06126700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1a000e5070 0x7f1a001b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:12.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.505 7f1a06126700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1a000e5470 0x7f1a00080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:12.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.505 7f1a06126700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0007ed80 con 0x7f1a001b3e40 2021-02-10T11:33:12.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.505 7f1a06126700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a000d1790 con 0x7f1a000e5070 2021-02-10T11:33:12.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.505 7f1a06126700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a000d19b0 con 0x7f1a000e5470 2021-02-10T11:33:12.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.505 7f19ff7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1a000e5070 0x7f1a001b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:12.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.505 7f19ff7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1a000e5070 0x7f1a001b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51494/0 (socket says 172.21.2.101:51494) 2021-02-10T11:33:12.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.505 7f19ff7fe700 1 -- 172.21.2.101:0/825519932 learned_addr learned my addr 172.21.2.101:0/825519932 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:12.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.505 7f19ff7fe700 1 -- 172.21.2.101:0/825519932 >> v2:172.21.2.108:3300/0 conn(0x7f1a000e5470 msgr2=0x7f1a00080310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:33:12.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.505 7f19ff7fe700 1 --2- 172.21.2.101:0/825519932 >> v2:172.21.2.108:3300/0 conn(0x7f1a000e5470 0x7f1a00080310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:12.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.505 7f19ff7fe700 1 -- 172.21.2.101:0/825519932 >> v1:172.21.2.108:6789/0 conn(0x7f1a001b3e40 legacy=0x7f1a0007d690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:33:12.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.505 7f19ff7fe700 1 -- 172.21.2.101:0/825519932 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a000d1bd0 con 0x7f1a000e5070 2021-02-10T11:33:12.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.505 7f19ff7fe700 1 --2- 172.21.2.101:0/825519932 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a000e5070 0x7f1a001b3820 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7f19f0006700 tx=0x7f19f0009b80).ready entity=mon.0 client_cookie=ac91c1a0f83a19be server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:12.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.506 7f19fe7fc700 1 -- 172.21.2.101:0/825519932 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19f000c0c0 con 0x7f1a000e5070 2021-02-10T11:33:12.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.506 7f19fe7fc700 1 -- 172.21.2.101:0/825519932 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f000c440 con 0x7f1a000e5070 2021-02-10T11:33:12.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.506 7f19fe7fc700 1 -- 172.21.2.101:0/825519932 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19f000c920 con 0x7f1a000e5070 2021-02-10T11:33:12.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.506 7f1a06126700 1 -- 172.21.2.101:0/825519932 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a000e5070 msgr2=0x7f1a001b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:12.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.506 7f1a06126700 1 --2- 172.21.2.101:0/825519932 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a000e5070 0x7f1a001b3820 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7f19f0006700 tx=0x7f19f0009b80).stop 2021-02-10T11:33:12.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.506 7f1a06126700 1 -- 172.21.2.101:0/825519932 shutdown_connections 2021-02-10T11:33:12.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.506 7f1a06126700 1 --2- 172.21.2.101:0/825519932 >> v2:172.21.2.108:3300/0 conn(0x7f1a000e5470 0x7f1a00080310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:12.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.506 7f1a06126700 1 --2- 172.21.2.101:0/825519932 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a000e5070 0x7f1a001b3820 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:12.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.506 7f1a06126700 1 -- 172.21.2.101:0/825519932 >> 172.21.2.101:0/825519932 conn(0x7f1a000ce690 msgr2=0x7f1a000e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:12.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.506 7f1a06126700 1 -- 172.21.2.101:0/825519932 shutdown_connections 2021-02-10T11:33:12.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.506 7f1a06126700 1 -- 172.21.2.101:0/825519932 wait complete. 2021-02-10T11:33:12.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.521 7f1a06126700 1 Processor -- start 2021-02-10T11:33:12.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.521 7f1a06126700 1 -- start start 2021-02-10T11:33:12.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.521 7f1a06126700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a0008c830 0x7f1a0008d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:12.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.521 7f1a06126700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a001afe20 0x7f1a000b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:12.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.521 7f1a06126700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a001b0220 0x7f1a000b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:12.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.521 7f1a06126700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0008cc30 con 0x7f1a0008c830 2021-02-10T11:33:12.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.521 7f1a06126700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a000b3b70 con 0x7f1a001b0220 2021-02-10T11:33:12.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.521 7f1a06126700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a000b3d90 con 0x7f1a001afe20 2021-02-10T11:33:12.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.521 7f19ff7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a0008c830 0x7f1a0008d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:12.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.521 7f19ff7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a0008c830 0x7f1a0008d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51498/0 (socket says 172.21.2.101:51498) 2021-02-10T11:33:12.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.521 7f19ff7fe700 1 -- 172.21.2.101:0/3524511805 learned_addr learned my addr 172.21.2.101:0/3524511805 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:12.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.522 7f19ff7fe700 1 -- 172.21.2.101:0/3524511805 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a001afe20 msgr2=0x7f1a000b2f30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:12.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.522 7f19ff7fe700 1 --2- 172.21.2.101:0/3524511805 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a001afe20 0x7f1a000b2f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:12.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.522 7f19ff7fe700 1 -- 172.21.2.101:0/3524511805 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a001b0220 msgr2=0x7f1a000b3550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:12.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.522 7f19ff7fe700 1 --2- 172.21.2.101:0/3524511805 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a001b0220 0x7f1a000b3550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:12.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.522 7f19ff7fe700 1 -- 172.21.2.101:0/3524511805 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a001b0c00 con 0x7f1a0008c830 2021-02-10T11:33:12.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.522 7f19ff7fe700 1 --2- 172.21.2.101:0/3524511805 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a0008c830 0x7f1a0008d0c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7f19f000b650 tx=0x7f19f000b9f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:12.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.522 7f19f7fff700 1 -- 172.21.2.101:0/3524511805 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19f000bde0 con 0x7f1a0008c830 2021-02-10T11:33:12.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.522 7f19f7fff700 1 -- 172.21.2.101:0/3524511805 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f0002030 con 0x7f1a0008c830 2021-02-10T11:33:12.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.522 7f19f7fff700 1 -- 172.21.2.101:0/3524511805 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19f00024d0 con 0x7f1a0008c830 2021-02-10T11:33:12.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.522 7f1a06126700 1 -- 172.21.2.101:0/3524511805 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a001b0c00 con 0x7f1a0008c830 2021-02-10T11:33:12.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.522 7f1a06126700 1 -- 172.21.2.101:0/3524511805 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a001b1150 con 0x7f1a0008c830 2021-02-10T11:33:12.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.523 7f19f7fff700 1 -- 172.21.2.101:0/3524511805 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f19f0000c90 con 0x7f1a0008c830 2021-02-10T11:33:12.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.523 7f19f7fff700 1 --2- 172.21.2.101:0/3524511805 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19e40186c0 0x7f19e401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:12.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.523 7f19f7fff700 1 -- 172.21.2.101:0/3524511805 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f19f00320f0 con 0x7f1a0008c830 2021-02-10T11:33:12.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.523 7f19feffd700 1 --2- 172.21.2.101:0/3524511805 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19e40186c0 0x7f19e401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:12.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.523 7f19feffd700 1 --2- 172.21.2.101:0/3524511805 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19e40186c0 0x7f19e401abc0 crc :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:12.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.555 7f1a06126700 1 -- 172.21.2.101:0/3524511805 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a00083db0 con 0x7f1a0008c830 2021-02-10T11:33:12.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.558 7f19f7fff700 1 -- 172.21.2.101:0/3524511805 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f19f0030d50 con 0x7f1a0008c830 2021-02-10T11:33:12.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.683 7f1a06126700 1 -- 172.21.2.101:0/3524511805 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a0007ed80 con 0x7f1a0008c830 2021-02-10T11:33:12.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.684 7f19f7fff700 1 -- 172.21.2.101:0/3524511805 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+11144 (secure 0 0 0) 0x7f19f0000c90 con 0x7f1a0008c830 2021-02-10T11:33:12.695 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:12.696 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":51,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:33:11.468343","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:33:12.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.686 7f1a06126700 1 -- 172.21.2.101:0/3524511805 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19e40186c0 msgr2=0x7f19e401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:12.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.686 7f1a06126700 1 --2- 172.21.2.101:0/3524511805 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19e40186c0 0x7f19e401abc0 crc :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:12.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.686 7f1a06126700 1 -- 172.21.2.101:0/3524511805 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a0008c830 msgr2=0x7f1a0008d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:12.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.686 7f1a06126700 1 --2- 172.21.2.101:0/3524511805 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a0008c830 0x7f1a0008d0c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7f19f000b650 tx=0x7f19f000b9f0).stop 2021-02-10T11:33:12.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.686 7f1a06126700 1 -- 172.21.2.101:0/3524511805 shutdown_connections 2021-02-10T11:33:12.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.686 7f1a06126700 1 --2- 172.21.2.101:0/3524511805 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19e40186c0 0x7f19e401abc0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:12.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.686 7f1a06126700 1 --2- 172.21.2.101:0/3524511805 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a001afe20 0x7f1a000b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:12.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.686 7f1a06126700 1 --2- 172.21.2.101:0/3524511805 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a001b0220 0x7f1a000b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:12.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.686 7f1a06126700 1 --2- 172.21.2.101:0/3524511805 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a0008c830 0x7f1a0008d0c0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:12.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.686 7f1a06126700 1 -- 172.21.2.101:0/3524511805 >> 172.21.2.101:0/3524511805 conn(0x7f1a000937a0 msgr2=0x7f1a00197e80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:12.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.686 7f1a06126700 1 -- 172.21.2.101:0/3524511805 shutdown_connections 2021-02-10T11:33:12.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:12.686 7f1a06126700 1 -- 172.21.2.101:0/3524511805 wait complete. 2021-02-10T11:33:12.706 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-10T11:33:12.706 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:33:13.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.109 7f977433a700 1 Processor -- start 2021-02-10T11:33:13.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.109 7f977433a700 1 -- start start 2021-02-10T11:33:13.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.109 7f977433a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f976c1b0470 0x7f976c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:13.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.109 7f977433a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f976c0f20e0 0x7f976c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:13.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.109 7f977433a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f976c0f0b50 con 0x7f976c0ef610 2021-02-10T11:33:13.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.109 7f977433a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f976c0f0820 con 0x7f976c1b0470 2021-02-10T11:33:13.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.109 7f977433a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f976c0f24e0 con 0x7f976c0f20e0 2021-02-10T11:33:13.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.110 7f9772156700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f976c1b0470 0x7f976c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:13.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.110 7f9771955700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f976c0ef610 0x7f976c0efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52000/0 (socket says 172.21.2.101:52000) 2021-02-10T11:33:13.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.110 7f9771955700 1 -- 172.21.2.101:0/525253280 learned_addr learned my addr 172.21.2.101:0/525253280 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:13.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.110 7f9772957700 1 --2- 172.21.2.101:0/525253280 >> v2:172.21.2.108:3300/0 conn(0x7f976c0f20e0 0x7f976c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:13.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.110 7f9771154700 1 -- 172.21.2.101:0/525253280 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722341429 0 0) 0x7f97600009b0 con 0x7f976c0ef610 2021-02-10T11:33:13.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.110 7f9771154700 1 -- 172.21.2.101:0/525253280 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9758002fe0 con 0x7f976c0ef610 2021-02-10T11:33:13.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.110 7f9772156700 1 -- 172.21.2.101:0/525253280 >> v2:172.21.2.108:3300/0 conn(0x7f976c0f20e0 msgr2=0x7f976c0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:13.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.110 7f9772156700 1 --2- 172.21.2.101:0/525253280 >> v2:172.21.2.108:3300/0 conn(0x7f976c0f20e0 0x7f976c0f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:13.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.110 7f9772156700 1 -- 172.21.2.101:0/525253280 >> v1:172.21.2.101:6789/0 conn(0x7f976c0ef610 legacy=0x7f976c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:13.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.110 7f9772156700 1 -- 172.21.2.101:0/525253280 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976c0e52e0 con 0x7f976c1b0470 2021-02-10T11:33:13.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.110 7f9772156700 1 --2- 172.21.2.101:0/525253280 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f976c1b0470 0x7f976c1b0870 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7f9768006650 tx=0x7f9768009a80).ready entity=mon.0 client_cookie=d05d613f83c50113 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:13.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.110 7f9771154700 1 -- 172.21.2.101:0/525253280 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f97680009b0 con 0x7f976c1b0470 2021-02-10T11:33:13.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.110 7f9771154700 1 -- 172.21.2.101:0/525253280 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976800c0c0 con 0x7f976c1b0470 2021-02-10T11:33:13.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.110 7f9771154700 1 -- 172.21.2.101:0/525253280 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f976800c510 con 0x7f976c1b0470 2021-02-10T11:33:13.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.110 7f977433a700 1 -- 172.21.2.101:0/525253280 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f976c1b0470 msgr2=0x7f976c1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:13.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.110 7f977433a700 1 --2- 172.21.2.101:0/525253280 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f976c1b0470 0x7f976c1b0870 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7f9768006650 tx=0x7f9768009a80).stop 2021-02-10T11:33:13.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.111 7f977433a700 1 -- 172.21.2.101:0/525253280 shutdown_connections 2021-02-10T11:33:13.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.111 7f977433a700 1 --2- 172.21.2.101:0/525253280 >> v2:172.21.2.108:3300/0 conn(0x7f976c0f20e0 0x7f976c0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:13.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.111 7f977433a700 1 --2- 172.21.2.101:0/525253280 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f976c1b0470 0x7f976c1b0870 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:13.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.111 7f977433a700 1 -- 172.21.2.101:0/525253280 >> 172.21.2.101:0/525253280 conn(0x7f976c0ce3e0 msgr2=0x7f976c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:13.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.111 7f977433a700 1 -- 172.21.2.101:0/525253280 shutdown_connections 2021-02-10T11:33:13.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.111 7f977433a700 1 -- 172.21.2.101:0/525253280 wait complete. 2021-02-10T11:33:13.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.125 7f977433a700 1 Processor -- start 2021-02-10T11:33:13.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.125 7f977433a700 1 -- start start 2021-02-10T11:33:13.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.126 7f977433a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f976c190920 0x7f976c0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:13.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.126 7f977433a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f976c0b8200 0x7f976c0b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:13.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.126 7f977433a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f976c0c1d80 0x7f976c0c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:13.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.126 7f977433a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976c190da0 con 0x7f976c0b8200 2021-02-10T11:33:13.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.126 7f977433a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976c0b8c20 con 0x7f976c190920 2021-02-10T11:33:13.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.126 7f977433a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976c0b8e10 con 0x7f976c0c1d80 2021-02-10T11:33:13.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.126 7f9772957700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f976c0c1d80 0x7f976c0c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:13.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.126 7f9771955700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f976c0b8200 0x7f976c0b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:13.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.126 7f9772957700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f976c0c1d80 0x7f976c0c2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:44248/0 (socket says 172.21.2.101:44248) 2021-02-10T11:33:13.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.126 7f9772957700 1 -- 172.21.2.101:0/3294923181 learned_addr learned my addr 172.21.2.101:0/3294923181 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:13.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.126 7f9772156700 1 --2- 172.21.2.101:0/3294923181 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f976c190920 0x7f976c0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:13.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.127 7f9772957700 1 -- 172.21.2.101:0/3294923181 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f976c190920 msgr2=0x7f976c0b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:13.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.127 7f9772957700 1 --2- 172.21.2.101:0/3294923181 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f976c190920 0x7f976c0b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:13.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.127 7f9772957700 1 -- 172.21.2.101:0/3294923181 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f976c0b8200 msgr2=0x7f976c0b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:13.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.127 7f9772957700 1 --2- 172.21.2.101:0/3294923181 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f976c0b8200 0x7f976c0b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:13.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.127 7f9772957700 1 -- 172.21.2.101:0/3294923181 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976c0c27d0 con 0x7f976c0c1d80 2021-02-10T11:33:13.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.127 7f9771955700 1 --2- 172.21.2.101:0/3294923181 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f976c0b8200 0x7f976c0b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:33:13.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.127 7f9772957700 1 --2- 172.21.2.101:0/3294923181 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f976c0c1d80 0x7f976c0c2180 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f976400b940 tx=0x7f976400bce0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:13.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.127 7f975effd700 1 -- 172.21.2.101:0/3294923181 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9764000f90 con 0x7f976c0c1d80 2021-02-10T11:33:13.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.127 7f975effd700 1 -- 172.21.2.101:0/3294923181 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97640066f0 con 0x7f976c0c1d80 2021-02-10T11:33:13.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.127 7f977433a700 1 -- 172.21.2.101:0/3294923181 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f976c0c27d0 con 0x7f976c0c1d80 2021-02-10T11:33:13.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.127 7f977433a700 1 -- 172.21.2.101:0/3294923181 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f976c0c2d60 con 0x7f976c0c1d80 2021-02-10T11:33:13.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.127 7f975effd700 1 -- 172.21.2.101:0/3294923181 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9764000f90 con 0x7f976c0c1d80 2021-02-10T11:33:13.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.128 7f975effd700 1 -- 172.21.2.101:0/3294923181 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f97640056b0 con 0x7f976c0c1d80 2021-02-10T11:33:13.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.128 7f975effd700 1 --2- 172.21.2.101:0/3294923181 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97540186c0 0x7f975401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:13.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.128 7f975effd700 1 -- 172.21.2.101:0/3294923181 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f97640325f0 con 0x7f976c0c1d80 2021-02-10T11:33:13.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.128 7f9772156700 1 --2- 172.21.2.101:0/3294923181 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97540186c0 0x7f975401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:13.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.128 7f9772156700 1 --2- 172.21.2.101:0/3294923181 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97540186c0 0x7f975401abc0 crc :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:13.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.160 7f977433a700 1 -- 172.21.2.101:0/3294923181 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f976c105ae0 con 0x7f976c0c1d80 2021-02-10T11:33:13.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.163 7f975effd700 1 -- 172.21.2.101:0/3294923181 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f97640011f0 con 0x7f976c0c1d80 2021-02-10T11:33:13.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.293 7f977433a700 1 -- 172.21.2.101:0/3294923181 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f976c0f0b50 con 0x7f976c0c1d80 2021-02-10T11:33:13.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.294 7f975effd700 1 -- 172.21.2.101:0/3294923181 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v51) v1 ==== 92+0+10 (secure 0 0 0) 0x7f97640056b0 con 0x7f976c0c1d80 2021-02-10T11:33:13.305 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:33:13.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.295 7f977433a700 1 -- 172.21.2.101:0/3294923181 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97540186c0 msgr2=0x7f975401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:13.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.295 7f977433a700 1 --2- 172.21.2.101:0/3294923181 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97540186c0 0x7f975401abc0 crc :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:13.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.295 7f977433a700 1 -- 172.21.2.101:0/3294923181 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f976c0c1d80 msgr2=0x7f976c0c2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:13.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.295 7f977433a700 1 --2- 172.21.2.101:0/3294923181 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f976c0c1d80 0x7f976c0c2180 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f976400b940 tx=0x7f976400bce0).stop 2021-02-10T11:33:13.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.296 7f977433a700 1 -- 172.21.2.101:0/3294923181 shutdown_connections 2021-02-10T11:33:13.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.296 7f977433a700 1 --2- 172.21.2.101:0/3294923181 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97540186c0 0x7f975401abc0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:13.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.296 7f977433a700 1 --2- 172.21.2.101:0/3294923181 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f976c0c1d80 0x7f976c0c2180 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:13.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.296 7f977433a700 1 --2- 172.21.2.101:0/3294923181 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f976c0b8200 0x7f976c0b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:13.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.296 7f977433a700 1 --2- 172.21.2.101:0/3294923181 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f976c190920 0x7f976c0b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:13.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.296 7f977433a700 1 -- 172.21.2.101:0/3294923181 >> 172.21.2.101:0/3294923181 conn(0x7f976c094e10 msgr2=0x7f976c0a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:13.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.296 7f977433a700 1 -- 172.21.2.101:0/3294923181 shutdown_connections 2021-02-10T11:33:13.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.296 7f977433a700 1 -- 172.21.2.101:0/3294923181 wait complete. 2021-02-10T11:33:13.315 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --format=json-pretty osd lspools 2021-02-10T11:33:13.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.714 7feeed8bc700 1 Processor -- start 2021-02-10T11:33:13.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.714 7feeed8bc700 1 -- start start 2021-02-10T11:33:13.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.714 7feeed8bc700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7feee80e5070 0x7feee81b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:13.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.715 7feeed8bc700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7feee81b3e40 0x7feee807d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:13.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.715 7feeed8bc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feee80e5470 0x7feee807dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:13.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.715 7feeed8bc700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee807ed80 con 0x7feee81b3e40 2021-02-10T11:33:13.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.715 7feeed8bc700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee807e2d0 con 0x7feee80e5070 2021-02-10T11:33:13.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.715 7feeed8bc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee807e4f0 con 0x7feee80e5470 2021-02-10T11:33:13.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.715 7feee6d73700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7feee81b3e40 0x7feee807d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:13.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.715 7feee6d73700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7feee81b3e40 0x7feee807d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51520/0 (socket says 172.21.2.101:51520) 2021-02-10T11:33:13.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.715 7feee6d73700 1 -- 172.21.2.101:0/2088063517 learned_addr learned my addr 172.21.2.101:0/2088063517 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:13.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.715 7feee6d73700 1 -- 172.21.2.101:0/2088063517 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feee80e5470 msgr2=0x7feee807dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:13.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.715 7feee6d73700 1 --2- 172.21.2.101:0/2088063517 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feee80e5470 0x7feee807dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:13.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.715 7feee6d73700 1 -- 172.21.2.101:0/2088063517 >> v2:172.21.2.108:3300/0 conn(0x7feee80e5070 msgr2=0x7feee81b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:13.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.715 7feee6d73700 1 --2- 172.21.2.101:0/2088063517 >> v2:172.21.2.108:3300/0 conn(0x7feee80e5070 0x7feee81b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:13.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.715 7feee6d73700 1 -- 172.21.2.101:0/2088063517 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee807e710 con 0x7feee81b3e40 2021-02-10T11:33:13.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.715 7feee6d73700 1 --2- 172.21.2.101:0/2088063517 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feee81b3e40 0x7feee807d690 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7feedc006700 tx=0x7feedc009b80).ready entity=mon.0 client_cookie=64b6e51fb41f0928 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:13.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.715 7feee6572700 1 -- 172.21.2.101:0/2088063517 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feedc00c0c0 con 0x7feee81b3e40 2021-02-10T11:33:13.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.715 7feee6572700 1 -- 172.21.2.101:0/2088063517 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feedc00c340 con 0x7feee81b3e40 2021-02-10T11:33:13.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.715 7feee6572700 1 -- 172.21.2.101:0/2088063517 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feedc00c820 con 0x7feee81b3e40 2021-02-10T11:33:13.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.715 7feeed8bc700 1 -- 172.21.2.101:0/2088063517 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feee81b3e40 msgr2=0x7feee807d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:13.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.715 7feeed8bc700 1 --2- 172.21.2.101:0/2088063517 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feee81b3e40 0x7feee807d690 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7feedc006700 tx=0x7feedc009b80).stop 2021-02-10T11:33:13.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.716 7feeed8bc700 1 -- 172.21.2.101:0/2088063517 shutdown_connections 2021-02-10T11:33:13.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.716 7feeed8bc700 1 --2- 172.21.2.101:0/2088063517 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feee80e5470 0x7feee807dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:13.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.716 7feeed8bc700 1 --2- 172.21.2.101:0/2088063517 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feee81b3e40 0x7feee807d690 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:13.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.716 7feeed8bc700 1 --2- 172.21.2.101:0/2088063517 >> v2:172.21.2.108:3300/0 conn(0x7feee80e5070 0x7feee81b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:13.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.716 7feeed8bc700 1 -- 172.21.2.101:0/2088063517 >> 172.21.2.101:0/2088063517 conn(0x7feee80ce690 msgr2=0x7feee80e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:13.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.716 7feeed8bc700 1 -- 172.21.2.101:0/2088063517 shutdown_connections 2021-02-10T11:33:13.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.716 7feeed8bc700 1 -- 172.21.2.101:0/2088063517 wait complete. 2021-02-10T11:33:13.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.731 7feeed8bc700 1 Processor -- start 2021-02-10T11:33:13.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.731 7feeed8bc700 1 -- start start 2021-02-10T11:33:13.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.731 7feeed8bc700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feee8192fc0 0x7feee8192780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:13.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.731 7feeed8bc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feee80b1430 0x7feee80b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:13.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.731 7feeed8bc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feee80b1e50 0x7feee808a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:13.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.731 7feeed8bc700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee80b2250 con 0x7feee80b1e50 2021-02-10T11:33:13.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.731 7feeed8bc700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee808b5d0 con 0x7feee8192fc0 2021-02-10T11:33:13.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.731 7feeed8bc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee808aa80 con 0x7feee80b1430 2021-02-10T11:33:13.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.731 7feee7d75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feee80b1e50 0x7feee808a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:13.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.732 7feee7d75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feee80b1e50 0x7feee808a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51526/0 (socket says 172.21.2.101:51526) 2021-02-10T11:33:13.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.732 7feee7d75700 1 -- 172.21.2.101:0/413174586 learned_addr learned my addr 172.21.2.101:0/413174586 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:13.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.732 7feee7d75700 1 -- 172.21.2.101:0/413174586 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feee80b1430 msgr2=0x7feee80b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:13.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.732 7feee7d75700 1 --2- 172.21.2.101:0/413174586 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feee80b1430 0x7feee80b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:13.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.732 7feee7d75700 1 -- 172.21.2.101:0/413174586 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feee8192fc0 msgr2=0x7feee8192780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:13.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.732 7feee7d75700 1 --2- 172.21.2.101:0/413174586 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feee8192fc0 0x7feee8192780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:13.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.732 7feee7d75700 1 -- 172.21.2.101:0/413174586 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee808ac60 con 0x7feee80b1e50 2021-02-10T11:33:13.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.732 7feee7d75700 1 --2- 172.21.2.101:0/413174586 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feee80b1e50 0x7feee808a460 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7feee0009980 tx=0x7feee000bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:13.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.732 7feecffff700 1 -- 172.21.2.101:0/413174586 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feee0003020 con 0x7feee80b1e50 2021-02-10T11:33:13.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.732 7feecffff700 1 -- 172.21.2.101:0/413174586 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee0005a80 con 0x7feee80b1e50 2021-02-10T11:33:13.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.732 7feecffff700 1 -- 172.21.2.101:0/413174586 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feee0006030 con 0x7feee80b1e50 2021-02-10T11:33:13.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.732 7feeed8bc700 1 -- 172.21.2.101:0/413174586 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee808ac60 con 0x7feee80b1e50 2021-02-10T11:33:13.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.732 7feeed8bc700 1 -- 172.21.2.101:0/413174586 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee808b1f0 con 0x7feee80b1e50 2021-02-10T11:33:13.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.733 7feecffff700 1 -- 172.21.2.101:0/413174586 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7feee0006030 con 0x7feee80b1e50 2021-02-10T11:33:13.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.733 7feecffff700 1 --2- 172.21.2.101:0/413174586 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feec80186c0 0x7feec801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:13.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.733 7feecffff700 1 -- 172.21.2.101:0/413174586 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4753+0+0 (secure 0 0 0) 0x7feee00325e0 con 0x7feee80b1e50 2021-02-10T11:33:13.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.733 7feee7574700 1 --2- 172.21.2.101:0/413174586 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feec80186c0 0x7feec801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:13.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.733 7feee7574700 1 --2- 172.21.2.101:0/413174586 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feec80186c0 0x7feec801abc0 crc :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:13.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.765 7feeed8bc700 1 -- 172.21.2.101:0/413174586 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee8107120 con 0x7feee80b1e50 2021-02-10T11:33:13.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.767 7feecffff700 1 -- 172.21.2.101:0/413174586 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7feee0020ee0 con 0x7feee80b1e50 2021-02-10T11:33:13.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.891 7feeed8bc700 1 -- 172.21.2.101:0/413174586 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7feee807ed80 con 0x7feee80b1e50 2021-02-10T11:33:13.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.892 7feecffff700 1 -- 172.21.2.101:0/413174586 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v51) v1 ==== 84+0+64 (secure 0 0 0) 0x7feee0020ee0 con 0x7feee80b1e50 2021-02-10T11:33:13.903 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:13.903 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:33:13.903 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:13.903 INFO:teuthology.orchestra.run.gibba001.stdout: "poolnum": 1, 2021-02-10T11:33:13.903 INFO:teuthology.orchestra.run.gibba001.stdout: "poolname": "rbd" 2021-02-10T11:33:13.903 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:13.904 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:33:13.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.894 7feeed8bc700 1 -- 172.21.2.101:0/413174586 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feec80186c0 msgr2=0x7feec801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:13.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.894 7feeed8bc700 1 --2- 172.21.2.101:0/413174586 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feec80186c0 0x7feec801abc0 crc :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:13.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.894 7feeed8bc700 1 -- 172.21.2.101:0/413174586 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feee80b1e50 msgr2=0x7feee808a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:13.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.894 7feeed8bc700 1 --2- 172.21.2.101:0/413174586 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feee80b1e50 0x7feee808a460 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7feee0009980 tx=0x7feee000bcd0).stop 2021-02-10T11:33:13.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.894 7feeed8bc700 1 -- 172.21.2.101:0/413174586 shutdown_connections 2021-02-10T11:33:13.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.894 7feeed8bc700 1 --2- 172.21.2.101:0/413174586 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feec80186c0 0x7feec801abc0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:13.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.894 7feeed8bc700 1 --2- 172.21.2.101:0/413174586 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feee80b1430 0x7feee80b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:13.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.894 7feeed8bc700 1 --2- 172.21.2.101:0/413174586 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feee80b1e50 0x7feee808a460 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:13.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.894 7feeed8bc700 1 --2- 172.21.2.101:0/413174586 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feee8192fc0 0x7feee8192780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:13.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.894 7feeed8bc700 1 -- 172.21.2.101:0/413174586 >> 172.21.2.101:0/413174586 conn(0x7feee81ac620 msgr2=0x7feee819fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:13.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.894 7feeed8bc700 1 -- 172.21.2.101:0/413174586 shutdown_connections 2021-02-10T11:33:13.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:13.894 7feeed8bc700 1 -- 172.21.2.101:0/413174586 wait complete. 2021-02-10T11:33:13.913 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-02-10T11:33:13.914 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-02-10T11:33:14.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.316 7f09c812d700 1 Processor -- start 2021-02-10T11:33:14.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.316 7f09c812d700 1 -- start start 2021-02-10T11:33:14.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.317 7f09c812d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09c01b0870 0x7f09c00f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:14.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.317 7f09c812d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f09c00efe00 0x7f09c00f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:14.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.317 7f09c812d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c01b0c70 con 0x7f09c01b0470 2021-02-10T11:33:14.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.317 7f09c812d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09c00f0b50 con 0x7f09c00efe00 2021-02-10T11:33:14.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.317 7f09c812d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c00f0820 con 0x7f09c01b0870 2021-02-10T11:33:14.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.317 7f09c674a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f09c00efe00 0x7f09c00f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:14.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.317 7f09c674a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f09c00efe00 0x7f09c00f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51534/0 (socket says 172.21.2.101:51534) 2021-02-10T11:33:14.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.317 7f09c674a700 1 -- 172.21.2.101:0/1389738400 learned_addr learned my addr 172.21.2.101:0/1389738400 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:14.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.317 7f09c674a700 1 -- 172.21.2.101:0/1389738400 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09c01b0870 msgr2=0x7f09c00f20e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:33:14.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.317 7f09c674a700 1 --2- 172.21.2.101:0/1389738400 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09c01b0870 0x7f09c00f20e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:14.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.317 7f09c674a700 1 -- 172.21.2.101:0/1389738400 >> v1:172.21.2.108:6789/0 conn(0x7f09c01b0470 legacy=0x7f09c00ef610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:33:14.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.317 7f09c674a700 1 -- 172.21.2.101:0/1389738400 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c00e52e0 con 0x7f09c00efe00 2021-02-10T11:33:14.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.317 7f09c674a700 1 --2- 172.21.2.101:0/1389738400 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09c00efe00 0x7f09c00f0200 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 rx=0x7f09b8006f00 tx=0x7f09b8009f10).ready entity=mon.0 client_cookie=fff112425827e1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:14.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.317 7f09c4f47700 1 -- 172.21.2.101:0/1389738400 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f09b800b3b0 con 0x7f09c00efe00 2021-02-10T11:33:14.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.317 7f09c4f47700 1 -- 172.21.2.101:0/1389738400 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b800b630 con 0x7f09c00efe00 2021-02-10T11:33:14.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.317 7f09c4f47700 1 -- 172.21.2.101:0/1389738400 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f09b800bb50 con 0x7f09c00efe00 2021-02-10T11:33:14.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.317 7f09c812d700 1 -- 172.21.2.101:0/1389738400 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09c00efe00 msgr2=0x7f09c00f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:14.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.317 7f09c812d700 1 --2- 172.21.2.101:0/1389738400 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09c00efe00 0x7f09c00f0200 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 rx=0x7f09b8006f00 tx=0x7f09b8009f10).stop 2021-02-10T11:33:14.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.318 7f09c812d700 1 -- 172.21.2.101:0/1389738400 shutdown_connections 2021-02-10T11:33:14.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.318 7f09c812d700 1 --2- 172.21.2.101:0/1389738400 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09c01b0870 0x7f09c00f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:14.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.318 7f09c812d700 1 --2- 172.21.2.101:0/1389738400 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09c00efe00 0x7f09c00f0200 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:14.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.318 7f09c812d700 1 -- 172.21.2.101:0/1389738400 >> 172.21.2.101:0/1389738400 conn(0x7f09c00ce3e0 msgr2=0x7f09c00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:14.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.318 7f09c812d700 1 -- 172.21.2.101:0/1389738400 shutdown_connections 2021-02-10T11:33:14.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.318 7f09c812d700 1 -- 172.21.2.101:0/1389738400 wait complete. 2021-02-10T11:33:14.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.333 7f09c812d700 1 Processor -- start 2021-02-10T11:33:14.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.333 7f09c812d700 1 -- start start 2021-02-10T11:33:14.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.333 7f09c812d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09c00a1dd0 0x7f09c0190920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:14.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.333 7f09c812d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09c008fd30 0x7f09c00effc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:14.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.333 7f09c812d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09c00f0e90 0x7f09c00f1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:14.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.333 7f09c812d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c00f06f0 con 0x7f09c00a1dd0 2021-02-10T11:33:14.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.333 7f09c812d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c0090160 con 0x7f09c00f0e90 2021-02-10T11:33:14.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.333 7f09c812d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c00f18b0 con 0x7f09c008fd30 2021-02-10T11:33:14.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.333 7f09c5f49700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09c00a1dd0 0x7f09c0190920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:14.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.333 7f09c5f49700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09c00a1dd0 0x7f09c0190920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51538/0 (socket says 172.21.2.101:51538) 2021-02-10T11:33:14.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.333 7f09c5f49700 1 -- 172.21.2.101:0/1976405341 learned_addr learned my addr 172.21.2.101:0/1976405341 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:14.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.333 7f09c5f49700 1 -- 172.21.2.101:0/1976405341 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09c008fd30 msgr2=0x7f09c00effc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:14.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.333 7f09c5f49700 1 --2- 172.21.2.101:0/1976405341 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09c008fd30 0x7f09c00effc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:14.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.334 7f09c5f49700 1 -- 172.21.2.101:0/1976405341 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09c00f0e90 msgr2=0x7f09c00f1290 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:14.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.334 7f09c5f49700 1 --2- 172.21.2.101:0/1976405341 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09c00f0e90 0x7f09c00f1290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:14.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.334 7f09c5f49700 1 -- 172.21.2.101:0/1976405341 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c00f1ad0 con 0x7f09c00a1dd0 2021-02-10T11:33:14.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.334 7f09c5f49700 1 --2- 172.21.2.101:0/1976405341 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09c00a1dd0 0x7f09c0190920 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7f09b400b5b0 tx=0x7f09b400b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:14.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.334 7f09b27fc700 1 -- 172.21.2.101:0/1976405341 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f09b400bd40 con 0x7f09c00a1dd0 2021-02-10T11:33:14.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.334 7f09b27fc700 1 -- 172.21.2.101:0/1976405341 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b4002080 con 0x7f09c00a1dd0 2021-02-10T11:33:14.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.334 7f09c812d700 1 -- 172.21.2.101:0/1976405341 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c00f1ad0 con 0x7f09c00a1dd0 2021-02-10T11:33:14.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.334 7f09b27fc700 1 -- 172.21.2.101:0/1976405341 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f09b4002490 con 0x7f09c00a1dd0 2021-02-10T11:33:14.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.334 7f09c812d700 1 -- 172.21.2.101:0/1976405341 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c00f1ad0 con 0x7f09c00a1dd0 2021-02-10T11:33:14.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.335 7f09b27fc700 1 -- 172.21.2.101:0/1976405341 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f09b4002260 con 0x7f09c00a1dd0 2021-02-10T11:33:14.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.335 7f09b27fc700 1 --2- 172.21.2.101:0/1976405341 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09a8018730 0x7f09a801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:14.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.335 7f09b27fc700 1 -- 172.21.2.101:0/1976405341 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f09b4032470 con 0x7f09c00a1dd0 2021-02-10T11:33:14.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.335 7f09c5748700 1 --2- 172.21.2.101:0/1976405341 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09a8018730 0x7f09a801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:14.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.335 7f09c5748700 1 --2- 172.21.2.101:0/1976405341 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09a8018730 0x7f09a801abc0 crc :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:14.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.366 7f09c812d700 1 -- 172.21.2.101:0/1976405341 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c0090130 con 0x7f09c00a1dd0 2021-02-10T11:33:14.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.369 7f09b27fc700 1 -- 172.21.2.101:0/1976405341 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f09b4000c60 con 0x7f09c00a1dd0 2021-02-10T11:33:14.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.497 7f09c812d700 1 -- 172.21.2.101:0/1976405341 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"} v 0) v1 -- 0x7f09c0090130 con 0x7f09c00a1dd0 2021-02-10T11:33:14.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:14.967 7f09b27fc700 1 -- 172.21.2.101:0/1976405341 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"}]=0 pool 'cephfs_metadata' created v52) v1 ==== 133+0+0 (secure 0 0 0) 0x7f09b401f4c0 con 0x7f09c00a1dd0 2021-02-10T11:33:15.024 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' created 2021-02-10T11:33:15.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.015 7f09c812d700 1 -- 172.21.2.101:0/1976405341 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09a8018730 msgr2=0x7f09a801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:15.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.015 7f09c812d700 1 --2- 172.21.2.101:0/1976405341 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09a8018730 0x7f09a801abc0 crc :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:15.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.015 7f09c812d700 1 -- 172.21.2.101:0/1976405341 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09c00a1dd0 msgr2=0x7f09c0190920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:15.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.015 7f09c812d700 1 --2- 172.21.2.101:0/1976405341 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09c00a1dd0 0x7f09c0190920 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7f09b400b5b0 tx=0x7f09b400b950).stop 2021-02-10T11:33:15.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.015 7f09c812d700 1 -- 172.21.2.101:0/1976405341 shutdown_connections 2021-02-10T11:33:15.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.015 7f09c812d700 1 --2- 172.21.2.101:0/1976405341 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09a8018730 0x7f09a801abc0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:15.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.015 7f09c812d700 1 --2- 172.21.2.101:0/1976405341 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09c008fd30 0x7f09c00effc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:15.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.015 7f09c812d700 1 --2- 172.21.2.101:0/1976405341 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09c00f0e90 0x7f09c00f1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:15.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.015 7f09c812d700 1 --2- 172.21.2.101:0/1976405341 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09c00a1dd0 0x7f09c0190920 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:15.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.015 7f09c812d700 1 -- 172.21.2.101:0/1976405341 >> 172.21.2.101:0/1976405341 conn(0x7f09c00958b0 msgr2=0x7f09c00bf040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:15.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.015 7f09c812d700 1 -- 172.21.2.101:0/1976405341 shutdown_connections 2021-02-10T11:33:15.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.015 7f09c812d700 1 -- 172.21.2.101:0/1976405341 wait complete. 2021-02-10T11:33:15.034 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_data 8 2021-02-10T11:33:15.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.436 7fd907e61700 1 Processor -- start 2021-02-10T11:33:15.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.436 7fd907e61700 1 -- start start 2021-02-10T11:33:15.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.436 7fd907e61700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9001b0470 0x7fd9001b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:15.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.436 7fd907e61700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd9000f20e0 0x7fd9000f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:15.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.436 7fd907e61700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9000f0b50 con 0x7fd9000ef610 2021-02-10T11:33:15.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.436 7fd907e61700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9000f0820 con 0x7fd9000f20e0 2021-02-10T11:33:15.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.436 7fd907e61700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9000f24e0 con 0x7fd9001b0470 2021-02-10T11:33:15.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.437 7fd90547c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd9000ef610 0x7fd9000efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52038/0 (socket says 172.21.2.101:52038) 2021-02-10T11:33:15.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.437 7fd90547c700 1 -- 172.21.2.101:0/2874181774 learned_addr learned my addr 172.21.2.101:0/2874181774 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:15.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.437 7fd90647e700 1 --2- 172.21.2.101:0/2874181774 >> v2:172.21.2.101:3300/0 conn(0x7fd9000f20e0 0x7fd9000f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:15.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.437 7fd904c7b700 1 -- 172.21.2.101:0/2874181774 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541043849 0 0) 0x7fd8fc0010b0 con 0x7fd9000ef610 2021-02-10T11:33:15.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.437 7fd904c7b700 1 -- 172.21.2.101:0/2874181774 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8ec002fe0 con 0x7fd9000ef610 2021-02-10T11:33:15.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.437 7fd90647e700 1 -- 172.21.2.101:0/2874181774 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9001b0470 msgr2=0x7fd9001b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:33:15.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.437 7fd90647e700 1 --2- 172.21.2.101:0/2874181774 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9001b0470 0x7fd9001b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:15.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.437 7fd90647e700 1 -- 172.21.2.101:0/2874181774 >> v1:172.21.2.101:6789/0 conn(0x7fd9000ef610 legacy=0x7fd9000efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:15.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.437 7fd90647e700 1 -- 172.21.2.101:0/2874181774 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9000e52e0 con 0x7fd9000f20e0 2021-02-10T11:33:15.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.437 7fd90647e700 1 --2- 172.21.2.101:0/2874181774 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9000f20e0 0x7fd9000f0200 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7fd8f8006e30 tx=0x7fd8f8009e60).ready entity=mon.0 client_cookie=f4a9bc361463d6b5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:15.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.437 7fd904c7b700 1 -- 172.21.2.101:0/2874181774 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8f8006a50 con 0x7fd9000f20e0 2021-02-10T11:33:15.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.437 7fd904c7b700 1 -- 172.21.2.101:0/2874181774 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f800b300 con 0x7fd9000f20e0 2021-02-10T11:33:15.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.437 7fd904c7b700 1 -- 172.21.2.101:0/2874181774 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8f800b820 con 0x7fd9000f20e0 2021-02-10T11:33:15.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.437 7fd907e61700 1 -- 172.21.2.101:0/2874181774 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9000f20e0 msgr2=0x7fd9000f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:15.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.437 7fd907e61700 1 --2- 172.21.2.101:0/2874181774 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9000f20e0 0x7fd9000f0200 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7fd8f8006e30 tx=0x7fd8f8009e60).stop 2021-02-10T11:33:15.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.438 7fd907e61700 1 -- 172.21.2.101:0/2874181774 shutdown_connections 2021-02-10T11:33:15.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.438 7fd907e61700 1 --2- 172.21.2.101:0/2874181774 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9000f20e0 0x7fd9000f0200 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:15.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.438 7fd907e61700 1 --2- 172.21.2.101:0/2874181774 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9001b0470 0x7fd9001b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:15.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.438 7fd907e61700 1 -- 172.21.2.101:0/2874181774 >> 172.21.2.101:0/2874181774 conn(0x7fd9000ce3e0 msgr2=0x7fd9000e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:15.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.438 7fd907e61700 1 -- 172.21.2.101:0/2874181774 shutdown_connections 2021-02-10T11:33:15.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.438 7fd907e61700 1 -- 172.21.2.101:0/2874181774 wait complete. 2021-02-10T11:33:15.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.452 7fd907e61700 1 Processor -- start 2021-02-10T11:33:15.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.453 7fd907e61700 1 -- start start 2021-02-10T11:33:15.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.453 7fd907e61700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd9000b7c20 0x7fd9000b8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:15.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.453 7fd907e61700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9000b8a30 0x7fd9000c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:15.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.453 7fd907e61700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9000c25a0 0x7fd9000c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:15.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.453 7fd907e61700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9000c1a80 con 0x7fd9000b8a30 2021-02-10T11:33:15.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.453 7fd907e61700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9000b8020 con 0x7fd9000b7c20 2021-02-10T11:33:15.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.453 7fd907e61700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90009b9c0 con 0x7fd9000c25a0 2021-02-10T11:33:15.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.453 7fd90547c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9000b8a30 0x7fd9000c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:15.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.453 7fd90547c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9000b8a30 0x7fd9000c1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51550/0 (socket says 172.21.2.101:51550) 2021-02-10T11:33:15.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.453 7fd90547c700 1 -- 172.21.2.101:0/265313468 learned_addr learned my addr 172.21.2.101:0/265313468 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:15.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.453 7fd90547c700 1 -- 172.21.2.101:0/265313468 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9000c25a0 msgr2=0x7fd9000c29a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:15.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.453 7fd90547c700 1 --2- 172.21.2.101:0/265313468 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9000c25a0 0x7fd9000c29a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:15.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.454 7fd90547c700 1 -- 172.21.2.101:0/265313468 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd9000b7c20 msgr2=0x7fd9000b8410 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:15.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.454 7fd90547c700 1 --2- 172.21.2.101:0/265313468 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd9000b7c20 0x7fd9000b8410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:15.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.454 7fd90547c700 1 -- 172.21.2.101:0/265313468 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9000b8e30 con 0x7fd9000b8a30 2021-02-10T11:33:15.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.454 7fd90547c700 1 --2- 172.21.2.101:0/265313468 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9000b8a30 0x7fd9000c1f80 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7fd8fc00a4c0 tx=0x7fd8fc00a860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:15.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.454 7fd8f27fc700 1 -- 172.21.2.101:0/265313468 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8fc00ac50 con 0x7fd9000b8a30 2021-02-10T11:33:15.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.454 7fd8f27fc700 1 -- 172.21.2.101:0/265313468 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8fc004140 con 0x7fd9000b8a30 2021-02-10T11:33:15.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.454 7fd8f27fc700 1 -- 172.21.2.101:0/265313468 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8fc004660 con 0x7fd9000b8a30 2021-02-10T11:33:15.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.454 7fd907e61700 1 -- 172.21.2.101:0/265313468 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9000b8e30 con 0x7fd9000b8a30 2021-02-10T11:33:15.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.454 7fd907e61700 1 -- 172.21.2.101:0/265313468 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9000b8e30 con 0x7fd9000b8a30 2021-02-10T11:33:15.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.455 7fd8f27fc700 1 -- 172.21.2.101:0/265313468 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd8fc004320 con 0x7fd9000b8a30 2021-02-10T11:33:15.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.455 7fd8f27fc700 1 --2- 172.21.2.101:0/265313468 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8e80186c0 0x7fd8e801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:15.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.455 7fd8f27fc700 1 -- 172.21.2.101:0/265313468 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5107+0+0 (secure 0 0 0) 0x7fd8fc032220 con 0x7fd9000b8a30 2021-02-10T11:33:15.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.455 7fd905c7d700 1 --2- 172.21.2.101:0/265313468 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8e80186c0 0x7fd8e801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:15.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.455 7fd905c7d700 1 --2- 172.21.2.101:0/265313468 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8e80186c0 0x7fd8e801abc0 crc :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:15.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.486 7fd907e61700 1 -- 172.21.2.101:0/265313468 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd900105b30 con 0x7fd9000b8a30 2021-02-10T11:33:15.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.489 7fd8f27fc700 1 -- 172.21.2.101:0/265313468 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd8fc000e00 con 0x7fd9000b8a30 2021-02-10T11:33:15.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.618 7fd907e61700 1 -- 172.21.2.101:0/265313468 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"} v 0) v1 -- 0x7fd9000f0b50 con 0x7fd9000b8a30 2021-02-10T11:33:15.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.970 7fd8f27fc700 1 -- 172.21.2.101:0/265313468 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"}]=0 pool 'cephfs_data' created v53) v1 ==== 125+0+0 (secure 0 0 0) 0x7fd8fc004320 con 0x7fd9000b8a30 2021-02-10T11:33:15.995 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' created 2021-02-10T11:33:15.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.986 7fd907e61700 1 -- 172.21.2.101:0/265313468 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8e80186c0 msgr2=0x7fd8e801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:15.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.986 7fd907e61700 1 --2- 172.21.2.101:0/265313468 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8e80186c0 0x7fd8e801abc0 crc :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:15.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.986 7fd907e61700 1 -- 172.21.2.101:0/265313468 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9000b8a30 msgr2=0x7fd9000c1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:15.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.986 7fd907e61700 1 --2- 172.21.2.101:0/265313468 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9000b8a30 0x7fd9000c1f80 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7fd8fc00a4c0 tx=0x7fd8fc00a860).stop 2021-02-10T11:33:15.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.986 7fd907e61700 1 -- 172.21.2.101:0/265313468 shutdown_connections 2021-02-10T11:33:15.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.986 7fd907e61700 1 --2- 172.21.2.101:0/265313468 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8e80186c0 0x7fd8e801abc0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:15.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.986 7fd907e61700 1 --2- 172.21.2.101:0/265313468 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9000c25a0 0x7fd9000c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:15.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.986 7fd907e61700 1 --2- 172.21.2.101:0/265313468 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9000b8a30 0x7fd9000c1f80 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:15.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.986 7fd907e61700 1 --2- 172.21.2.101:0/265313468 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd9000b7c20 0x7fd9000b8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:15.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.986 7fd907e61700 1 -- 172.21.2.101:0/265313468 >> 172.21.2.101:0/265313468 conn(0x7fd900094e10 msgr2=0x7fd9000a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:15.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.986 7fd907e61700 1 -- 172.21.2.101:0/265313468 shutdown_connections 2021-02-10T11:33:15.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:15.986 7fd907e61700 1 -- 172.21.2.101:0/265313468 wait complete. 2021-02-10T11:33:16.006 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-02-10T11:33:16.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.407 7f8f921d8700 1 Processor -- start 2021-02-10T11:33:16.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.407 7f8f921d8700 1 -- start start 2021-02-10T11:33:16.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.408 7f8f921d8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f8c0d1870 0x7f8f8c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:16.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.408 7f8f921d8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8f8c0cff70 0x7f8f8c109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:16.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.408 7f8f921d8700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f8c109a60 con 0x7f8f8c1b0ae0 2021-02-10T11:33:16.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.408 7f8f921d8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f8c109750 con 0x7f8f8c0cff70 2021-02-10T11:33:16.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.408 7f8f921d8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c088020 con 0x7f8f8c0d1870 2021-02-10T11:33:16.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.408 7f8f8affd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8f8c1b0ae0 0x7f8f8c108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52052/0 (socket says 172.21.2.101:52052) 2021-02-10T11:33:16.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.408 7f8f8affd700 1 -- 172.21.2.101:0/3911199026 learned_addr learned my addr 172.21.2.101:0/3911199026 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:16.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.408 7f8f8b7fe700 1 --2- 172.21.2.101:0/3911199026 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f8c0d1870 0x7f8f8c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:16.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.408 7f8f8bfff700 1 --2- 172.21.2.101:0/3911199026 >> v2:172.21.2.108:3300/0 conn(0x7f8f8c0cff70 0x7f8f8c109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:16.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.409 7f8f8a7fc700 1 -- 172.21.2.101:0/3911199026 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3795038341 0 0) 0x7f8f740010b0 con 0x7f8f8c1b0ae0 2021-02-10T11:33:16.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.409 7f8f8a7fc700 1 -- 172.21.2.101:0/3911199026 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f78002fe0 con 0x7f8f8c1b0ae0 2021-02-10T11:33:16.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.409 7f8f8bfff700 1 -- 172.21.2.101:0/3911199026 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f8c0d1870 msgr2=0x7f8f8c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:16.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.409 7f8f8bfff700 1 --2- 172.21.2.101:0/3911199026 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f8c0d1870 0x7f8f8c1b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:16.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.409 7f8f8bfff700 1 -- 172.21.2.101:0/3911199026 >> v1:172.21.2.101:6789/0 conn(0x7f8f8c1b0ae0 legacy=0x7f8f8c108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:16.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.409 7f8f8bfff700 1 -- 172.21.2.101:0/3911199026 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8c088210 con 0x7f8f8c0cff70 2021-02-10T11:33:16.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.409 7f8f8b7fe700 1 --2- 172.21.2.101:0/3911199026 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f8c0d1870 0x7f8f8c1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:33:16.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.409 7f8f8bfff700 1 --2- 172.21.2.101:0/3911199026 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8f8c0cff70 0x7f8f8c109130 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f8f84006e50 tx=0x7f8f84009e70).ready entity=mon.1 client_cookie=a727f082c1a105df server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:16.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.409 7f8f8a7fc700 1 -- 172.21.2.101:0/3911199026 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8f84000f90 con 0x7f8f8c0cff70 2021-02-10T11:33:16.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.409 7f8f8a7fc700 1 -- 172.21.2.101:0/3911199026 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f840066f0 con 0x7f8f8c0cff70 2021-02-10T11:33:16.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.409 7f8f8a7fc700 1 -- 172.21.2.101:0/3911199026 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8f8400b790 con 0x7f8f8c0cff70 2021-02-10T11:33:16.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.409 7f8f921d8700 1 -- 172.21.2.101:0/3911199026 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8f8c0cff70 msgr2=0x7f8f8c109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:16.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.409 7f8f921d8700 1 --2- 172.21.2.101:0/3911199026 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8f8c0cff70 0x7f8f8c109130 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f8f84006e50 tx=0x7f8f84009e70).stop 2021-02-10T11:33:16.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.410 7f8f921d8700 1 -- 172.21.2.101:0/3911199026 shutdown_connections 2021-02-10T11:33:16.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.410 7f8f921d8700 1 --2- 172.21.2.101:0/3911199026 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8f8c0cff70 0x7f8f8c109130 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:16.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.410 7f8f921d8700 1 --2- 172.21.2.101:0/3911199026 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f8c0d1870 0x7f8f8c1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:16.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.410 7f8f921d8700 1 -- 172.21.2.101:0/3911199026 >> 172.21.2.101:0/3911199026 conn(0x7f8f8c0ce420 msgr2=0x7f8f8c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:16.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.410 7f8f921d8700 1 -- 172.21.2.101:0/3911199026 shutdown_connections 2021-02-10T11:33:16.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.410 7f8f921d8700 1 -- 172.21.2.101:0/3911199026 wait complete. 2021-02-10T11:33:16.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.425 7f8f921d8700 1 Processor -- start 2021-02-10T11:33:16.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.425 7f8f921d8700 1 -- start start 2021-02-10T11:33:16.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.425 7f8f921d8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f8c0b6620 0x7f8f8c0b6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:16.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.425 7f8f921d8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8f8c1a0040 0x7f8f8c1a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:16.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.425 7f8f921d8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f8c09bd40 0x7f8f8c09c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:16.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.425 7f8f921d8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c0872b0 con 0x7f8f8c0b6620 2021-02-10T11:33:16.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.425 7f8f921d8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c09c760 con 0x7f8f8c1a0040 2021-02-10T11:33:16.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.425 7f8f921d8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c09c980 con 0x7f8f8c09bd40 2021-02-10T11:33:16.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.425 7f8f8b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f8c0b6620 0x7f8f8c0b6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:16.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.425 7f8f8b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f8c0b6620 0x7f8f8c0b6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51566/0 (socket says 172.21.2.101:51566) 2021-02-10T11:33:16.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.425 7f8f8b7fe700 1 -- 172.21.2.101:0/4027713626 learned_addr learned my addr 172.21.2.101:0/4027713626 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:16.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.426 7f8f8b7fe700 1 -- 172.21.2.101:0/4027713626 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f8c09bd40 msgr2=0x7f8f8c09c140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:16.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.426 7f8f8b7fe700 1 --2- 172.21.2.101:0/4027713626 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f8c09bd40 0x7f8f8c09c140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:16.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.426 7f8f8b7fe700 1 -- 172.21.2.101:0/4027713626 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8f8c1a0040 msgr2=0x7f8f8c1a0440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:16.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.426 7f8f8b7fe700 1 --2- 172.21.2.101:0/4027713626 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8f8c1a0040 0x7f8f8c1a0440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:16.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.426 7f8f8b7fe700 1 -- 172.21.2.101:0/4027713626 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8c09cba0 con 0x7f8f8c0b6620 2021-02-10T11:33:16.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.426 7f8f8b7fe700 1 --2- 172.21.2.101:0/4027713626 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f8c0b6620 0x7f8f8c0b6a20 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7f8f7c006f40 tx=0x7f8f7c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:16.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.426 7f8f83fff700 1 -- 172.21.2.101:0/4027713626 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8f7c00bd80 con 0x7f8f8c0b6620 2021-02-10T11:33:16.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.426 7f8f83fff700 1 -- 172.21.2.101:0/4027713626 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7c000c90 con 0x7f8f8c0b6620 2021-02-10T11:33:16.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.426 7f8f83fff700 1 -- 172.21.2.101:0/4027713626 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8f7c0022b0 con 0x7f8f8c0b6620 2021-02-10T11:33:16.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.426 7f8f921d8700 1 -- 172.21.2.101:0/4027713626 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f8c09cba0 con 0x7f8f8c0b6620 2021-02-10T11:33:16.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.426 7f8f921d8700 1 -- 172.21.2.101:0/4027713626 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8c0b78e0 con 0x7f8f8c0b6620 2021-02-10T11:33:16.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.427 7f8f83fff700 1 -- 172.21.2.101:0/4027713626 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8f7c00a000 con 0x7f8f8c0b6620 2021-02-10T11:33:16.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.427 7f8f83fff700 1 --2- 172.21.2.101:0/4027713626 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8f700186c0 0x7f8f7001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:16.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.427 7f8f83fff700 1 -- 172.21.2.101:0/4027713626 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5457+0+0 (secure 0 0 0) 0x7f8f7c032530 con 0x7f8f8c0b6620 2021-02-10T11:33:16.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.427 7f8f8affd700 1 --2- 172.21.2.101:0/4027713626 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8f700186c0 0x7f8f7001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:16.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.427 7f8f8affd700 1 --2- 172.21.2.101:0/4027713626 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8f700186c0 0x7f8f7001abc0 crc :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:16.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.458 7f8f921d8700 1 -- 172.21.2.101:0/4027713626 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f8c109a60 con 0x7f8f8c0b6620 2021-02-10T11:33:16.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.460 7f8f83fff700 1 -- 172.21.2.101:0/4027713626 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8f7c0009b0 con 0x7f8f8c0b6620 2021-02-10T11:33:16.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.588 7f8f921d8700 1 -- 172.21.2.101:0/4027713626 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"} v 0) v1 -- 0x7f8f8c109a60 con 0x7f8f8c0b6620 2021-02-10T11:33:16.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.592 7f8f83fff700 1 -- 172.21.2.101:0/4027713626 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"}]=0 new fs with metadata pool 10 and data pool 11 v36) v1 ==== 189+0+0 (secure 0 0 0) 0x7f8f7c00a000 con 0x7f8f8c0b6620 2021-02-10T11:33:16.606 INFO:teuthology.orchestra.run.gibba001.stderr:new fs with metadata pool 10 and data pool 11 2021-02-10T11:33:16.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.598 7f8f921d8700 1 -- 172.21.2.101:0/4027713626 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8f700186c0 msgr2=0x7f8f7001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:16.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.598 7f8f921d8700 1 --2- 172.21.2.101:0/4027713626 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8f700186c0 0x7f8f7001abc0 crc :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:16.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.598 7f8f921d8700 1 -- 172.21.2.101:0/4027713626 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f8c0b6620 msgr2=0x7f8f8c0b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:16.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.598 7f8f921d8700 1 --2- 172.21.2.101:0/4027713626 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f8c0b6620 0x7f8f8c0b6a20 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7f8f7c006f40 tx=0x7f8f7c00b990).stop 2021-02-10T11:33:16.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.598 7f8f921d8700 1 -- 172.21.2.101:0/4027713626 shutdown_connections 2021-02-10T11:33:16.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.598 7f8f921d8700 1 --2- 172.21.2.101:0/4027713626 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8f700186c0 0x7f8f7001abc0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:16.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.598 7f8f921d8700 1 --2- 172.21.2.101:0/4027713626 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f8c09bd40 0x7f8f8c09c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:16.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.598 7f8f921d8700 1 --2- 172.21.2.101:0/4027713626 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8f8c1a0040 0x7f8f8c1a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:16.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.598 7f8f921d8700 1 --2- 172.21.2.101:0/4027713626 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f8c0b6620 0x7f8f8c0b6a20 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:16.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.598 7f8f921d8700 1 -- 172.21.2.101:0/4027713626 >> 172.21.2.101:0/4027713626 conn(0x7f8f8c0b5e50 msgr2=0x7f8f8c0c3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:16.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.598 7f8f921d8700 1 -- 172.21.2.101:0/4027713626 shutdown_connections 2021-02-10T11:33:16.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:16.598 7f8f921d8700 1 -- 172.21.2.101:0/4027713626 wait complete. 2021-02-10T11:33:16.617 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:33:17.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.015 7f6eb8905700 1 Processor -- start 2021-02-10T11:33:17.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.015 7f6eb8905700 1 -- start start 2021-02-10T11:33:17.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.015 7f6eb8905700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6eb40e5470 0x7f6eb407e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:17.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.015 7f6eb8905700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6eb407f300 0x7f6eb407f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:17.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.015 7f6eb8905700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb407ed80 con 0x7f6eb40e5070 2021-02-10T11:33:17.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.015 7f6eb8905700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb41b4010 con 0x7f6eb40e5470 2021-02-10T11:33:17.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.015 7f6eb8905700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb40d1790 con 0x7f6eb407f300 2021-02-10T11:33:17.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.016 7f6eb1e1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6eb40e5470 0x7f6eb407e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:17.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.016 7f6eb1e1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6eb40e5470 0x7f6eb407e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51572/0 (socket says 172.21.2.101:51572) 2021-02-10T11:33:17.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.016 7f6eb1e1b700 1 -- 172.21.2.101:0/1401785218 learned_addr learned my addr 172.21.2.101:0/1401785218 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:17.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.016 7f6eb1e1b700 1 -- 172.21.2.101:0/1401785218 >> v2:172.21.2.108:3300/0 conn(0x7f6eb407f300 msgr2=0x7f6eb407f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:17.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.016 7f6eb1e1b700 1 --2- 172.21.2.101:0/1401785218 >> v2:172.21.2.108:3300/0 conn(0x7f6eb407f300 0x7f6eb407f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:17.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.016 7f6eb1e1b700 1 -- 172.21.2.101:0/1401785218 >> v1:172.21.2.108:6789/0 conn(0x7f6eb40e5070 legacy=0x7f6eb41b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:17.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.016 7f6eb1e1b700 1 -- 172.21.2.101:0/1401785218 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb40d19b0 con 0x7f6eb40e5470 2021-02-10T11:33:17.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.016 7f6eb1e1b700 1 --2- 172.21.2.101:0/1401785218 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6eb40e5470 0x7f6eb407e6a0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7f6ea4006700 tx=0x7f6ea4009b00).ready entity=mon.0 client_cookie=cc3f2462d9e28f9d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:17.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.016 7f6eb161a700 1 -- 172.21.2.101:0/1401785218 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ea400c0c0 con 0x7f6eb40e5470 2021-02-10T11:33:17.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.016 7f6eb161a700 1 -- 172.21.2.101:0/1401785218 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea400c340 con 0x7f6eb40e5470 2021-02-10T11:33:17.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.016 7f6eb161a700 1 -- 172.21.2.101:0/1401785218 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ea400c820 con 0x7f6eb40e5470 2021-02-10T11:33:17.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.016 7f6eb8905700 1 -- 172.21.2.101:0/1401785218 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6eb40e5470 msgr2=0x7f6eb407e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:17.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.016 7f6eb8905700 1 --2- 172.21.2.101:0/1401785218 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6eb40e5470 0x7f6eb407e6a0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7f6ea4006700 tx=0x7f6ea4009b00).stop 2021-02-10T11:33:17.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.017 7f6eb8905700 1 -- 172.21.2.101:0/1401785218 shutdown_connections 2021-02-10T11:33:17.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.017 7f6eb8905700 1 --2- 172.21.2.101:0/1401785218 >> v2:172.21.2.108:3300/0 conn(0x7f6eb407f300 0x7f6eb407f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:17.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.017 7f6eb8905700 1 --2- 172.21.2.101:0/1401785218 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6eb40e5470 0x7f6eb407e6a0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:17.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.017 7f6eb8905700 1 -- 172.21.2.101:0/1401785218 >> 172.21.2.101:0/1401785218 conn(0x7f6eb40ce690 msgr2=0x7f6eb40e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:17.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.017 7f6eb8905700 1 -- 172.21.2.101:0/1401785218 shutdown_connections 2021-02-10T11:33:17.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.017 7f6eb8905700 1 -- 172.21.2.101:0/1401785218 wait complete. 2021-02-10T11:33:17.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.032 7f6eb8905700 1 Processor -- start 2021-02-10T11:33:17.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.032 7f6eb8905700 1 -- start start 2021-02-10T11:33:17.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.032 7f6eb8905700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6eb41a2a90 0x7f6eb41aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:17.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.032 7f6eb8905700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6eb41aa700 0x7f6eb41a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:17.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.032 7f6eb8905700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6eb40a55e0 0x7f6eb41a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:17.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.033 7f6eb8905700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb41a34a0 con 0x7f6eb40a55e0 2021-02-10T11:33:17.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.033 7f6eb8905700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb41a2e90 con 0x7f6eb41aa700 2021-02-10T11:33:17.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.033 7f6eb8905700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb41a30b0 con 0x7f6eb41a2a90 2021-02-10T11:33:17.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.033 7f6eb2e1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6eb40a55e0 0x7f6eb41a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:17.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.033 7f6eb2e1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6eb40a55e0 0x7f6eb41a4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51582/0 (socket says 172.21.2.101:51582) 2021-02-10T11:33:17.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.033 7f6eb2e1d700 1 -- 172.21.2.101:0/3124193413 learned_addr learned my addr 172.21.2.101:0/3124193413 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:17.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.033 7f6eb261c700 1 --2- 172.21.2.101:0/3124193413 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6eb41a2a90 0x7f6eb41aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:17.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.033 7f6eb1e1b700 1 --2- 172.21.2.101:0/3124193413 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6eb41aa700 0x7f6eb41a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:17.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.033 7f6eb2e1d700 1 -- 172.21.2.101:0/3124193413 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6eb41a2a90 msgr2=0x7f6eb41aa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:17.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.033 7f6eb2e1d700 1 --2- 172.21.2.101:0/3124193413 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6eb41a2a90 0x7f6eb41aa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:17.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.033 7f6eb2e1d700 1 -- 172.21.2.101:0/3124193413 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6eb41aa700 msgr2=0x7f6eb41a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:17.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.033 7f6eb2e1d700 1 --2- 172.21.2.101:0/3124193413 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6eb41aa700 0x7f6eb41a3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:17.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.033 7f6eb2e1d700 1 -- 172.21.2.101:0/3124193413 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb41a4660 con 0x7f6eb40a55e0 2021-02-10T11:33:17.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.033 7f6eb2e1d700 1 --2- 172.21.2.101:0/3124193413 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6eb40a55e0 0x7f6eb41a4040 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0x7f6eac009980 tx=0x7f6eac00bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:17.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.034 7f6eaaffd700 1 -- 172.21.2.101:0/3124193413 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6eac003020 con 0x7f6eb40a55e0 2021-02-10T11:33:17.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.034 7f6eaaffd700 1 -- 172.21.2.101:0/3124193413 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac005a80 con 0x7f6eb40a55e0 2021-02-10T11:33:17.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.034 7f6eaaffd700 1 -- 172.21.2.101:0/3124193413 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6eac006030 con 0x7f6eb40a55e0 2021-02-10T11:33:17.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.034 7f6eb8905700 1 -- 172.21.2.101:0/3124193413 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb41a4660 con 0x7f6eb40a55e0 2021-02-10T11:33:17.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.034 7f6eb8905700 1 -- 172.21.2.101:0/3124193413 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb41a4660 con 0x7f6eb40a55e0 2021-02-10T11:33:17.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.034 7f6eaaffd700 1 -- 172.21.2.101:0/3124193413 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6eac005c60 con 0x7f6eb40a55e0 2021-02-10T11:33:17.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.034 7f6eaaffd700 1 --2- 172.21.2.101:0/3124193413 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e980186c0 0x7f6e9801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:17.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.034 7f6eaaffd700 1 -- 172.21.2.101:0/3124193413 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6eac032b30 con 0x7f6eb40a55e0 2021-02-10T11:33:17.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.035 7f6eb261c700 1 --2- 172.21.2.101:0/3124193413 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e980186c0 0x7f6e9801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:17.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.035 7f6eb261c700 1 --2- 172.21.2.101:0/3124193413 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e980186c0 0x7f6e9801abc0 crc :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:17.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.065 7f6eb8905700 1 -- 172.21.2.101:0/3124193413 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb4083da0 con 0x7f6eb40a55e0 2021-02-10T11:33:17.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.068 7f6eaaffd700 1 -- 172.21.2.101:0/3124193413 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6eac005870 con 0x7f6eb40a55e0 2021-02-10T11:33:17.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.195 7f6eb8905700 1 -- 172.21.2.101:0/3124193413 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6eb407ed80 con 0x7f6eb40a55e0 2021-02-10T11:33:17.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.195 7f6eaaffd700 1 -- 172.21.2.101:0/3124193413 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+14009 (secure 0 0 0) 0x7f6eac005a80 con 0x7f6eb40a55e0 2021-02-10T11:33:17.208 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:17.208 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":54,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:33:16.590199","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:33:14.515763","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2021-02-10 11:33:15.636274","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:33:17.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.197 7f6eb8905700 1 -- 172.21.2.101:0/3124193413 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e980186c0 msgr2=0x7f6e9801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:17.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.197 7f6eb8905700 1 --2- 172.21.2.101:0/3124193413 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e980186c0 0x7f6e9801abc0 crc :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:17.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.197 7f6eb8905700 1 -- 172.21.2.101:0/3124193413 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6eb40a55e0 msgr2=0x7f6eb41a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:17.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.197 7f6eb8905700 1 --2- 172.21.2.101:0/3124193413 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6eb40a55e0 0x7f6eb41a4040 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0x7f6eac009980 tx=0x7f6eac00bcd0).stop 2021-02-10T11:33:17.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.197 7f6eb8905700 1 -- 172.21.2.101:0/3124193413 shutdown_connections 2021-02-10T11:33:17.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.197 7f6eb8905700 1 --2- 172.21.2.101:0/3124193413 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6eb40a55e0 0x7f6eb41a4040 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:17.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.197 7f6eb8905700 1 --2- 172.21.2.101:0/3124193413 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6eb41aa700 0x7f6eb41a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:17.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.197 7f6eb8905700 1 --2- 172.21.2.101:0/3124193413 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e980186c0 0x7f6e9801abc0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:17.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.197 7f6eb8905700 1 --2- 172.21.2.101:0/3124193413 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6eb41a2a90 0x7f6eb41aa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:17.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.197 7f6eb8905700 1 -- 172.21.2.101:0/3124193413 >> 172.21.2.101:0/3124193413 conn(0x7f6eb408fe10 msgr2=0x7f6eb419c020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:17.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.197 7f6eb8905700 1 -- 172.21.2.101:0/3124193413 shutdown_connections 2021-02-10T11:33:17.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.197 7f6eb8905700 1 -- 172.21.2.101:0/3124193413 wait complete. 2021-02-10T11:33:17.217 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:33:17.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.616 7f879db6c700 1 Processor -- start 2021-02-10T11:33:17.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.616 7f879db6c700 1 -- start start 2021-02-10T11:33:17.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.616 7f879db6c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f87980d17f0 0x7f87981b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:17.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.616 7f879db6c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87980cff00 0x7f87980ef620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:17.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.616 7f879db6c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87980f0b20 con 0x7f87980d1bf0 2021-02-10T11:33:17.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.616 7f879db6c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87981b3ff0 con 0x7f87980d17f0 2021-02-10T11:33:17.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.616 7f879db6c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87980f20b0 con 0x7f87980cff00 2021-02-10T11:33:17.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.617 7f8797fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87980cff00 0x7f87980ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:17.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.617 7f8797fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87980cff00 0x7f87980ef620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:44322/0 (socket says 172.21.2.101:44322) 2021-02-10T11:33:17.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.617 7f8797fff700 1 -- 172.21.2.101:0/458415118 learned_addr learned my addr 172.21.2.101:0/458415118 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:17.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.617 7f87977fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f87980d17f0 0x7f87981b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:17.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.617 7f8796ffd700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f87980d1bf0 0x7f87981b3800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:45490/0 (socket says 172.21.2.101:45490) 2021-02-10T11:33:17.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.617 7f87977fe700 1 -- 172.21.2.101:0/458415118 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87980cff00 msgr2=0x7f87980ef620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:17.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.617 7f87977fe700 1 --2- 172.21.2.101:0/458415118 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87980cff00 0x7f87980ef620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:17.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.617 7f87977fe700 1 -- 172.21.2.101:0/458415118 >> v1:172.21.2.108:6789/0 conn(0x7f87980d1bf0 legacy=0x7f87981b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:17.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.617 7f87977fe700 1 -- 172.21.2.101:0/458415118 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87980f22d0 con 0x7f87980d17f0 2021-02-10T11:33:17.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.617 7f87977fe700 1 --2- 172.21.2.101:0/458415118 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87980d17f0 0x7f87981b31e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f878c006650 tx=0x7f878c009a80).ready entity=mon.1 client_cookie=a42a0f08f9dca92f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:17.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.618 7f87967fc700 1 -- 172.21.2.101:0/458415118 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f878c00c0c0 con 0x7f87980d17f0 2021-02-10T11:33:17.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.618 7f87967fc700 1 -- 172.21.2.101:0/458415118 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878c00c340 con 0x7f87980d17f0 2021-02-10T11:33:17.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.618 7f87967fc700 1 -- 172.21.2.101:0/458415118 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f878c00c860 con 0x7f87980d17f0 2021-02-10T11:33:17.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.618 7f879db6c700 1 -- 172.21.2.101:0/458415118 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87980d17f0 msgr2=0x7f87981b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:17.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.618 7f879db6c700 1 --2- 172.21.2.101:0/458415118 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87980d17f0 0x7f87981b31e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f878c006650 tx=0x7f878c009a80).stop 2021-02-10T11:33:17.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.618 7f879db6c700 1 -- 172.21.2.101:0/458415118 shutdown_connections 2021-02-10T11:33:17.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.618 7f879db6c700 1 --2- 172.21.2.101:0/458415118 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87980cff00 0x7f87980ef620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:17.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.618 7f879db6c700 1 --2- 172.21.2.101:0/458415118 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87980d17f0 0x7f87981b31e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:17.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.618 7f879db6c700 1 -- 172.21.2.101:0/458415118 >> 172.21.2.101:0/458415118 conn(0x7f87980fde40 msgr2=0x7f87980ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:17.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.618 7f879db6c700 1 -- 172.21.2.101:0/458415118 shutdown_connections 2021-02-10T11:33:17.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.618 7f879db6c700 1 -- 172.21.2.101:0/458415118 wait complete. 2021-02-10T11:33:17.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.633 7f879db6c700 1 Processor -- start 2021-02-10T11:33:17.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.633 7f879db6c700 1 -- start start 2021-02-10T11:33:17.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.633 7f879db6c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87981a7e80 0x7f87980ad650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:17.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.634 7f879db6c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87980adc70 0x7f87981b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:17.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.634 7f879db6c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87981b15c0 0x7f87981a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:17.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.634 7f879db6c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87981a7980 con 0x7f87981a7e80 2021-02-10T11:33:17.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.634 7f879db6c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8798198920 con 0x7f87981b15c0 2021-02-10T11:33:17.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.634 7f879db6c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87981b19c0 con 0x7f87980adc70 2021-02-10T11:33:17.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.634 7f87977fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87981a7e80 0x7f87980ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:17.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.634 7f87977fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87981a7e80 0x7f87980ad650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51592/0 (socket says 172.21.2.101:51592) 2021-02-10T11:33:17.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.634 7f87977fe700 1 -- 172.21.2.101:0/3058561179 learned_addr learned my addr 172.21.2.101:0/3058561179 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:17.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.634 7f87977fe700 1 -- 172.21.2.101:0/3058561179 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87980adc70 msgr2=0x7f87981b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:17.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.634 7f87977fe700 1 --2- 172.21.2.101:0/3058561179 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87980adc70 0x7f87981b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:17.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.634 7f87977fe700 1 -- 172.21.2.101:0/3058561179 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87981b15c0 msgr2=0x7f87981a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:17.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.634 7f87977fe700 1 --2- 172.21.2.101:0/3058561179 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87981b15c0 0x7f87981a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:17.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.634 7f87977fe700 1 -- 172.21.2.101:0/3058561179 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87981b1be0 con 0x7f87981a7e80 2021-02-10T11:33:17.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.634 7f87977fe700 1 --2- 172.21.2.101:0/3058561179 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87981a7e80 0x7f87980ad650 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0x7f878c009650 tx=0x7f878c00b8e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:17.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.634 7f877ffff700 1 -- 172.21.2.101:0/3058561179 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f878c0093a0 con 0x7f87981a7e80 2021-02-10T11:33:17.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.634 7f877ffff700 1 -- 172.21.2.101:0/3058561179 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878c006100 con 0x7f87981a7e80 2021-02-10T11:33:17.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.634 7f879db6c700 1 -- 172.21.2.101:0/3058561179 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87981b1be0 con 0x7f87981a7e80 2021-02-10T11:33:17.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.634 7f877ffff700 1 -- 172.21.2.101:0/3058561179 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f878c0052e0 con 0x7f87981a7e80 2021-02-10T11:33:17.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.634 7f879db6c700 1 -- 172.21.2.101:0/3058561179 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87981a6590 con 0x7f87981a7e80 2021-02-10T11:33:17.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.635 7f877ffff700 1 -- 172.21.2.101:0/3058561179 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f878c009300 con 0x7f87981a7e80 2021-02-10T11:33:17.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.635 7f877ffff700 1 --2- 172.21.2.101:0/3058561179 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8778018730 0x7f877801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:17.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.635 7f877ffff700 1 -- 172.21.2.101:0/3058561179 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f878c032670 con 0x7f87981a7e80 2021-02-10T11:33:17.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.635 7f8796ffd700 1 --2- 172.21.2.101:0/3058561179 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8778018730 0x7f877801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:17.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.635 7f8796ffd700 1 --2- 172.21.2.101:0/3058561179 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8778018730 0x7f877801ac10 crc :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:17.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.666 7f879db6c700 1 -- 172.21.2.101:0/3058561179 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87981036a0 con 0x7f87981a7e80 2021-02-10T11:33:17.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.669 7f877ffff700 1 -- 172.21.2.101:0/3058561179 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f878c006100 con 0x7f87981a7e80 2021-02-10T11:33:17.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.793 7f879db6c700 1 -- 172.21.2.101:0/3058561179 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87980f0b20 con 0x7f87981a7e80 2021-02-10T11:33:17.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.794 7f877ffff700 1 -- 172.21.2.101:0/3058561179 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+13996 (secure 0 0 0) 0x7f878c000940 con 0x7f87981a7e80 2021-02-10T11:33:17.805 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:17.805 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":55,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:33:17.591716","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:33:14.515763","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2021-02-10 11:33:15.636274","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:33:17.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.795 7f879db6c700 1 -- 172.21.2.101:0/3058561179 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8778018730 msgr2=0x7f877801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:17.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.795 7f879db6c700 1 --2- 172.21.2.101:0/3058561179 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8778018730 0x7f877801ac10 crc :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:17.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.796 7f879db6c700 1 -- 172.21.2.101:0/3058561179 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87981a7e80 msgr2=0x7f87980ad650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:17.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.796 7f879db6c700 1 --2- 172.21.2.101:0/3058561179 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87981a7e80 0x7f87980ad650 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0x7f878c009650 tx=0x7f878c00b8e0).stop 2021-02-10T11:33:17.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.796 7f879db6c700 1 -- 172.21.2.101:0/3058561179 shutdown_connections 2021-02-10T11:33:17.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.796 7f879db6c700 1 --2- 172.21.2.101:0/3058561179 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8778018730 0x7f877801ac10 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:17.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.796 7f879db6c700 1 --2- 172.21.2.101:0/3058561179 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87980adc70 0x7f87981b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:17.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.796 7f879db6c700 1 --2- 172.21.2.101:0/3058561179 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87981b15c0 0x7f87981a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:17.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.796 7f879db6c700 1 --2- 172.21.2.101:0/3058561179 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87981a7e80 0x7f87980ad650 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:17.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.796 7f879db6c700 1 -- 172.21.2.101:0/3058561179 >> 172.21.2.101:0/3058561179 conn(0x7f8798108000 msgr2=0x7f879808ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:17.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.796 7f879db6c700 1 -- 172.21.2.101:0/3058561179 shutdown_connections 2021-02-10T11:33:17.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:17.796 7f879db6c700 1 -- 172.21.2.101:0/3058561179 wait complete. 2021-02-10T11:33:17.815 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-02-10T11:33:18.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.217 7f61d6c21700 1 Processor -- start 2021-02-10T11:33:18.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.217 7f61d6c21700 1 -- start start 2021-02-10T11:33:18.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.218 7f61d6c21700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f61d01b0470 0x7f61d01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:18.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.218 7f61d6c21700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f61d00f20e0 0x7f61d00f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:18.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.218 7f61d6c21700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d00f0b50 con 0x7f61d00ef610 2021-02-10T11:33:18.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.218 7f61d6c21700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d00f0820 con 0x7f61d01b0470 2021-02-10T11:33:18.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.218 7f61d6c21700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d00f24e0 con 0x7f61d00f20e0 2021-02-10T11:33:18.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.218 7f61d4a3d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f61d01b0470 0x7f61d01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:18.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.218 7f61d4a3d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f61d01b0470 0x7f61d01b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51602/0 (socket says 172.21.2.101:51602) 2021-02-10T11:33:18.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.218 7f61d4a3d700 1 -- 172.21.2.101:0/3457997122 learned_addr learned my addr 172.21.2.101:0/3457997122 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:18.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.218 7f61d523e700 1 --2- 172.21.2.101:0/3457997122 >> v2:172.21.2.108:3300/0 conn(0x7f61d00f20e0 0x7f61d00f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:18.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.219 7f61d523e700 1 -- 172.21.2.101:0/3457997122 >> v2:172.21.2.101:3300/0 conn(0x7f61d01b0470 msgr2=0x7f61d01b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:18.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.219 7f61d523e700 1 --2- 172.21.2.101:0/3457997122 >> v2:172.21.2.101:3300/0 conn(0x7f61d01b0470 0x7f61d01b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:18.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.219 7f61d523e700 1 -- 172.21.2.101:0/3457997122 >> v1:172.21.2.108:6789/0 conn(0x7f61d00ef610 legacy=0x7f61d00efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:18.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.219 7f61d523e700 1 -- 172.21.2.101:0/3457997122 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d00e52e0 con 0x7f61d00f20e0 2021-02-10T11:33:18.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.219 7f61d4a3d700 1 --2- 172.21.2.101:0/3457997122 >> v2:172.21.2.101:3300/0 conn(0x7f61d01b0470 0x7f61d01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:33:18.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.219 7f61d523e700 1 --2- 172.21.2.101:0/3457997122 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f61d00f20e0 0x7f61d00f0200 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f61c8006e50 tx=0x7f61c8009e70).ready entity=mon.1 client_cookie=32034ec3264343f7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:18.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.219 7f61cf7fe700 1 -- 172.21.2.101:0/3457997122 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f61c800b300 con 0x7f61d00f20e0 2021-02-10T11:33:18.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.219 7f61cf7fe700 1 -- 172.21.2.101:0/3457997122 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c8003030 con 0x7f61d00f20e0 2021-02-10T11:33:18.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.219 7f61cf7fe700 1 -- 172.21.2.101:0/3457997122 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f61c800b7b0 con 0x7f61d00f20e0 2021-02-10T11:33:18.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.219 7f61d6c21700 1 -- 172.21.2.101:0/3457997122 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f61d00f20e0 msgr2=0x7f61d00f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:18.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.219 7f61d6c21700 1 --2- 172.21.2.101:0/3457997122 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f61d00f20e0 0x7f61d00f0200 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f61c8006e50 tx=0x7f61c8009e70).stop 2021-02-10T11:33:18.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.220 7f61d6c21700 1 -- 172.21.2.101:0/3457997122 shutdown_connections 2021-02-10T11:33:18.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.220 7f61d6c21700 1 --2- 172.21.2.101:0/3457997122 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f61d00f20e0 0x7f61d00f0200 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:18.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.220 7f61d6c21700 1 --2- 172.21.2.101:0/3457997122 >> v2:172.21.2.101:3300/0 conn(0x7f61d01b0470 0x7f61d01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:18.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.220 7f61d6c21700 1 -- 172.21.2.101:0/3457997122 >> 172.21.2.101:0/3457997122 conn(0x7f61d00ce3e0 msgr2=0x7f61d00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:18.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.220 7f61d6c21700 1 -- 172.21.2.101:0/3457997122 shutdown_connections 2021-02-10T11:33:18.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.220 7f61d6c21700 1 -- 172.21.2.101:0/3457997122 wait complete. 2021-02-10T11:33:18.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.235 7f61d6c21700 1 Processor -- start 2021-02-10T11:33:18.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.235 7f61d6c21700 1 -- start start 2021-02-10T11:33:18.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.235 7f61d6c21700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f61d00a0420 0x7f61d00a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:18.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.235 7f61d6c21700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f61d0189d00 0x7f61d018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:18.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.235 7f61d6c21700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f61d018a720 0x7f61d00b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:18.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.235 7f61d6c21700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d0189800 con 0x7f61d00a0420 2021-02-10T11:33:18.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.235 7f61d6c21700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d018ab20 con 0x7f61d018a720 2021-02-10T11:33:18.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.235 7f61d6c21700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d00a1140 con 0x7f61d0189d00 2021-02-10T11:33:18.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.236 7f61d4a3d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f61d00a0420 0x7f61d00a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:18.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.236 7f61d4a3d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f61d00a0420 0x7f61d00a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51606/0 (socket says 172.21.2.101:51606) 2021-02-10T11:33:18.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.236 7f61d4a3d700 1 -- 172.21.2.101:0/968609100 learned_addr learned my addr 172.21.2.101:0/968609100 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:18.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.236 7f61cffff700 1 --2- 172.21.2.101:0/968609100 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f61d0189d00 0x7f61d018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:18.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.236 7f61d523e700 1 --2- 172.21.2.101:0/968609100 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f61d018a720 0x7f61d00b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:18.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.236 7f61d4a3d700 1 -- 172.21.2.101:0/968609100 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f61d0189d00 msgr2=0x7f61d018a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:18.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.236 7f61d4a3d700 1 --2- 172.21.2.101:0/968609100 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f61d0189d00 0x7f61d018a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:18.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.236 7f61d4a3d700 1 -- 172.21.2.101:0/968609100 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f61d018a720 msgr2=0x7f61d00b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:18.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.236 7f61d4a3d700 1 --2- 172.21.2.101:0/968609100 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f61d018a720 0x7f61d00b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:18.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.236 7f61d4a3d700 1 -- 172.21.2.101:0/968609100 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d00b8200 con 0x7f61d00a0420 2021-02-10T11:33:18.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.236 7f61d4a3d700 1 --2- 172.21.2.101:0/968609100 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f61d00a0420 0x7f61d00a0820 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0x7f61b8006ee0 tx=0x7f61b800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:18.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.237 7f61cd7fa700 1 -- 172.21.2.101:0/968609100 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f61b800bd50 con 0x7f61d00a0420 2021-02-10T11:33:18.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.237 7f61cd7fa700 1 -- 172.21.2.101:0/968609100 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b80052e0 con 0x7f61d00a0420 2021-02-10T11:33:18.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.237 7f61d6c21700 1 -- 172.21.2.101:0/968609100 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d00b8200 con 0x7f61d00a0420 2021-02-10T11:33:18.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.237 7f61cd7fa700 1 -- 172.21.2.101:0/968609100 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f61b8005800 con 0x7f61d00a0420 2021-02-10T11:33:18.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.237 7f61d6c21700 1 -- 172.21.2.101:0/968609100 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d00b8790 con 0x7f61d00a0420 2021-02-10T11:33:18.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.237 7f61cd7fa700 1 -- 172.21.2.101:0/968609100 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f61b80054c0 con 0x7f61d00a0420 2021-02-10T11:33:18.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.237 7f61cd7fa700 1 --2- 172.21.2.101:0/968609100 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61b40186d0 0x7f61b401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:18.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.237 7f61cd7fa700 1 -- 172.21.2.101:0/968609100 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f61b8032610 con 0x7f61d00a0420 2021-02-10T11:33:18.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.237 7f61cffff700 1 --2- 172.21.2.101:0/968609100 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61b40186d0 0x7f61b401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:18.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.238 7f61cffff700 1 --2- 172.21.2.101:0/968609100 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61b40186d0 0x7f61b401abd0 crc :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.267 7f61d6c21700 1 -- 172.21.2.101:0/968609100 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d0105ae0 con 0x7f61d00a0420 2021-02-10T11:33:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.270 7f61cd7fa700 1 -- 172.21.2.101:0/968609100 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f61b80052e0 con 0x7f61d00a0420 2021-02-10T11:33:18.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.408 7f61d6c21700 1 -- 172.21.2.101:0/968609100 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"} v 0) v1 -- 0x7f61d00f0b50 con 0x7f61d00a0420 2021-02-10T11:33:18.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.597 7f61cd7fa700 1 -- 172.21.2.101:0/968609100 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"}]=0 v38) v1 ==== 118+0+0 (secure 0 0 0) 0x7f61b80054f0 con 0x7f61d00a0420 2021-02-10T11:33:18.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.626 7f61d6c21700 1 -- 172.21.2.101:0/968609100 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61b40186d0 msgr2=0x7f61b401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:18.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.626 7f61d6c21700 1 --2- 172.21.2.101:0/968609100 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61b40186d0 0x7f61b401abd0 crc :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:18.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.626 7f61d6c21700 1 -- 172.21.2.101:0/968609100 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f61d00a0420 msgr2=0x7f61d00a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:18.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.626 7f61d6c21700 1 --2- 172.21.2.101:0/968609100 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f61d00a0420 0x7f61d00a0820 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0x7f61b8006ee0 tx=0x7f61b800b960).stop 2021-02-10T11:33:18.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.627 7f61d6c21700 1 -- 172.21.2.101:0/968609100 shutdown_connections 2021-02-10T11:33:18.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.627 7f61d6c21700 1 --2- 172.21.2.101:0/968609100 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61b40186d0 0x7f61b401abd0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:18.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.627 7f61d6c21700 1 --2- 172.21.2.101:0/968609100 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f61d0189d00 0x7f61d018a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:18.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.627 7f61d6c21700 1 --2- 172.21.2.101:0/968609100 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f61d018a720 0x7f61d00b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:18.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.627 7f61d6c21700 1 --2- 172.21.2.101:0/968609100 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f61d00a0420 0x7f61d00a0820 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:18.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.627 7f61d6c21700 1 -- 172.21.2.101:0/968609100 >> 172.21.2.101:0/968609100 conn(0x7f61d0094e10 msgr2=0x7f61d00a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:18.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.627 7f61d6c21700 1 -- 172.21.2.101:0/968609100 shutdown_connections 2021-02-10T11:33:18.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:18.627 7f61d6c21700 1 -- 172.21.2.101:0/968609100 wait complete. 2021-02-10T11:33:18.646 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:33:19.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.046 7fd859ad3700 1 Processor -- start 2021-02-10T11:33:19.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.046 7fd859ad3700 1 -- start start 2021-02-10T11:33:19.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.046 7fd859ad3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd8540e5470 0x7fd85407e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:19.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.047 7fd859ad3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd85407f300 0x7fd85407f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:19.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.047 7fd859ad3700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85407ed80 con 0x7fd8540e5070 2021-02-10T11:33:19.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.047 7fd859ad3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8541b4010 con 0x7fd8540e5470 2021-02-10T11:33:19.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.047 7fd859ad3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8540d1790 con 0x7fd85407f300 2021-02-10T11:33:19.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.047 7fd8537fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd8540e5070 0x7fd8541b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52108/0 (socket says 172.21.2.101:52108) 2021-02-10T11:33:19.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.047 7fd852ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd8540e5470 0x7fd85407e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:19.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.047 7fd8537fe700 1 -- 172.21.2.101:0/1793547564 learned_addr learned my addr 172.21.2.101:0/1793547564 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:19.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.047 7fd853fff700 1 --2- 172.21.2.101:0/1793547564 >> v2:172.21.2.108:3300/0 conn(0x7fd85407f300 0x7fd85407f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:19.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.047 7fd852ffd700 1 -- 172.21.2.101:0/1793547564 >> v2:172.21.2.108:3300/0 conn(0x7fd85407f300 msgr2=0x7fd85407f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:19.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.047 7fd852ffd700 1 --2- 172.21.2.101:0/1793547564 >> v2:172.21.2.108:3300/0 conn(0x7fd85407f300 0x7fd85407f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:19.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.047 7fd852ffd700 1 -- 172.21.2.101:0/1793547564 >> v1:172.21.2.101:6789/0 conn(0x7fd8540e5070 legacy=0x7fd8541b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:19.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.047 7fd852ffd700 1 -- 172.21.2.101:0/1793547564 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8540d19b0 con 0x7fd8540e5470 2021-02-10T11:33:19.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.047 7fd853fff700 1 --2- 172.21.2.101:0/1793547564 >> v2:172.21.2.108:3300/0 conn(0x7fd85407f300 0x7fd85407f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:33:19.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.047 7fd852ffd700 1 --2- 172.21.2.101:0/1793547564 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8540e5470 0x7fd85407e6a0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7fd844006650 tx=0x7fd844009a80).ready entity=mon.0 client_cookie=2ca2549cf8397d77 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:19.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.048 7fd8527fc700 1 -- 172.21.2.101:0/1793547564 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8440009b0 con 0x7fd8540e5470 2021-02-10T11:33:19.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.048 7fd8527fc700 1 -- 172.21.2.101:0/1793547564 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84400c0c0 con 0x7fd8540e5470 2021-02-10T11:33:19.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.048 7fd8527fc700 1 -- 172.21.2.101:0/1793547564 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd84400c5a0 con 0x7fd8540e5470 2021-02-10T11:33:19.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.048 7fd859ad3700 1 -- 172.21.2.101:0/1793547564 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8540e5470 msgr2=0x7fd85407e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:19.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.048 7fd859ad3700 1 --2- 172.21.2.101:0/1793547564 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8540e5470 0x7fd85407e6a0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7fd844006650 tx=0x7fd844009a80).stop 2021-02-10T11:33:19.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.048 7fd859ad3700 1 -- 172.21.2.101:0/1793547564 shutdown_connections 2021-02-10T11:33:19.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.048 7fd859ad3700 1 --2- 172.21.2.101:0/1793547564 >> v2:172.21.2.108:3300/0 conn(0x7fd85407f300 0x7fd85407f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:19.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.048 7fd859ad3700 1 --2- 172.21.2.101:0/1793547564 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8540e5470 0x7fd85407e6a0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:19.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.048 7fd859ad3700 1 -- 172.21.2.101:0/1793547564 >> 172.21.2.101:0/1793547564 conn(0x7fd8540ce690 msgr2=0x7fd8540e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:19.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.048 7fd859ad3700 1 -- 172.21.2.101:0/1793547564 shutdown_connections 2021-02-10T11:33:19.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.048 7fd859ad3700 1 -- 172.21.2.101:0/1793547564 wait complete. 2021-02-10T11:33:19.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.063 7fd859ad3700 1 Processor -- start 2021-02-10T11:33:19.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.063 7fd859ad3700 1 -- start start 2021-02-10T11:33:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.063 7fd859ad3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd854192960 0x7fd854192d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.063 7fd859ad3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd8540ad110 0x7fd8540ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.063 7fd859ad3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8540adb30 0x7fd85408e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.063 7fd859ad3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8541a03e0 con 0x7fd8540adb30 2021-02-10T11:33:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.063 7fd859ad3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd854193380 con 0x7fd8540ad110 2021-02-10T11:33:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.063 7fd859ad3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85408ea80 con 0x7fd854192960 2021-02-10T11:33:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.064 7fd853fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8540adb30 0x7fd85408e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.064 7fd853fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8540adb30 0x7fd85408e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51624/0 (socket says 172.21.2.101:51624) 2021-02-10T11:33:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.064 7fd853fff700 1 -- 172.21.2.101:0/3081005632 learned_addr learned my addr 172.21.2.101:0/3081005632 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.064 7fd8537fe700 1 --2- 172.21.2.101:0/3081005632 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd854192960 0x7fd854192d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.064 7fd853fff700 1 -- 172.21.2.101:0/3081005632 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd854192960 msgr2=0x7fd854192d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.064 7fd853fff700 1 --2- 172.21.2.101:0/3081005632 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd854192960 0x7fd854192d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.064 7fd852ffd700 1 --2- 172.21.2.101:0/3081005632 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd8540ad110 0x7fd8540ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.064 7fd853fff700 1 -- 172.21.2.101:0/3081005632 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd8540ad110 msgr2=0x7fd8540ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.064 7fd853fff700 1 --2- 172.21.2.101:0/3081005632 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd8540ad110 0x7fd8540ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.064 7fd853fff700 1 -- 172.21.2.101:0/3081005632 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85408eca0 con 0x7fd8540adb30 2021-02-10T11:33:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.064 7fd8537fe700 1 --2- 172.21.2.101:0/3081005632 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd854192960 0x7fd854192d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:33:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.064 7fd852ffd700 1 --2- 172.21.2.101:0/3081005632 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd8540ad110 0x7fd8540ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:33:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.064 7fd853fff700 1 --2- 172.21.2.101:0/3081005632 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8540adb30 0x7fd85408e460 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7fd84c009980 tx=0x7fd84c00bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.065 7fd84bfff700 1 -- 172.21.2.101:0/3081005632 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd84c00c2f0 con 0x7fd8540adb30 2021-02-10T11:33:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.065 7fd84bfff700 1 -- 172.21.2.101:0/3081005632 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84c006790 con 0x7fd8540adb30 2021-02-10T11:33:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.065 7fd84bfff700 1 -- 172.21.2.101:0/3081005632 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd84c00c760 con 0x7fd8540adb30 2021-02-10T11:33:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.065 7fd859ad3700 1 -- 172.21.2.101:0/3081005632 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85408eca0 con 0x7fd8540adb30 2021-02-10T11:33:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.065 7fd859ad3700 1 -- 172.21.2.101:0/3081005632 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd85408f230 con 0x7fd8540adb30 2021-02-10T11:33:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.065 7fd84bfff700 1 -- 172.21.2.101:0/3081005632 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd84c0011f0 con 0x7fd8540adb30 2021-02-10T11:33:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.065 7fd84bfff700 1 --2- 172.21.2.101:0/3081005632 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8380186c0 0x7fd83801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:19.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.065 7fd84bfff700 1 -- 172.21.2.101:0/3081005632 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd84c0328c0 con 0x7fd8540adb30 2021-02-10T11:33:19.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.066 7fd8537fe700 1 --2- 172.21.2.101:0/3081005632 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8380186c0 0x7fd83801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:19.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.066 7fd8537fe700 1 --2- 172.21.2.101:0/3081005632 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8380186c0 0x7fd83801abc0 crc :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:19.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.097 7fd859ad3700 1 -- 172.21.2.101:0/3081005632 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd854107120 con 0x7fd8540adb30 2021-02-10T11:33:19.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.099 7fd84bfff700 1 -- 172.21.2.101:0/3081005632 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd84c026380 con 0x7fd8540adb30 2021-02-10T11:33:19.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.226 7fd859ad3700 1 -- 172.21.2.101:0/3081005632 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd854107120 con 0x7fd8540adb30 2021-02-10T11:33:19.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.226 7fd84bfff700 1 -- 172.21.2.101:0/3081005632 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+2098 (secure 0 0 0) 0x7fd84c026380 con 0x7fd8540adb30 2021-02-10T11:33:19.238 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 38 2021-02-10T11:33:19.238 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:19.239 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8049,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/172865610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":172865610},{"type":"v1","addr":"172.21.2.101:6835","nonce":172865610}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:33:16.590282","modified":"2021-02-10 11:33:18.594009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5579},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5579":{"gid":5579,"name":"b","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2400978339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2400978339},{"type":"v1","addr":"172.21.2.108:6833","nonce":2400978339}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-02-10T11:33:19.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.228 7fd859ad3700 1 -- 172.21.2.101:0/3081005632 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8380186c0 msgr2=0x7fd83801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:19.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.228 7fd859ad3700 1 --2- 172.21.2.101:0/3081005632 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8380186c0 0x7fd83801abc0 crc :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:19.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.228 7fd859ad3700 1 -- 172.21.2.101:0/3081005632 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8540adb30 msgr2=0x7fd85408e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:19.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.228 7fd859ad3700 1 --2- 172.21.2.101:0/3081005632 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8540adb30 0x7fd85408e460 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7fd84c009980 tx=0x7fd84c00bcd0).stop 2021-02-10T11:33:19.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.229 7fd859ad3700 1 -- 172.21.2.101:0/3081005632 shutdown_connections 2021-02-10T11:33:19.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.229 7fd859ad3700 1 --2- 172.21.2.101:0/3081005632 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8540adb30 0x7fd85408e460 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:19.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.229 7fd859ad3700 1 --2- 172.21.2.101:0/3081005632 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd8540ad110 0x7fd8540ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:19.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.229 7fd859ad3700 1 --2- 172.21.2.101:0/3081005632 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8380186c0 0x7fd83801abc0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:19.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.229 7fd859ad3700 1 --2- 172.21.2.101:0/3081005632 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd854192960 0x7fd854192d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:19.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.229 7fd859ad3700 1 -- 172.21.2.101:0/3081005632 >> 172.21.2.101:0/3081005632 conn(0x7fd8540891f0 msgr2=0x7fd8541a0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:19.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.229 7fd859ad3700 1 -- 172.21.2.101:0/3081005632 shutdown_connections 2021-02-10T11:33:19.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.229 7fd859ad3700 1 -- 172.21.2.101:0/3081005632 wait complete. 2021-02-10T11:33:19.248 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:33:19.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.648 7f2fc286d700 1 Processor -- start 2021-02-10T11:33:19.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.648 7f2fc286d700 1 -- start start 2021-02-10T11:33:19.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.649 7f2fc286d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2fbc0e5070 0x7f2fbc1b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:19.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.649 7f2fc286d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2fbc1b3e40 0x7f2fbc07d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:19.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.649 7f2fc286d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fbc07ed80 con 0x7f2fbc0e5470 2021-02-10T11:33:19.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.649 7f2fc286d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fbc080310 con 0x7f2fbc1b3e40 2021-02-10T11:33:19.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.649 7f2fc286d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fbc080530 con 0x7f2fbc0e5070 2021-02-10T11:33:19.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.649 7f2fbb7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2fbc1b3e40 0x7f2fbc07d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:19.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.649 7f2fbb7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2fbc1b3e40 0x7f2fbc07d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51628/0 (socket says 172.21.2.101:51628) 2021-02-10T11:33:19.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.649 7f2fbb7fe700 1 -- 172.21.2.101:0/592598585 learned_addr learned my addr 172.21.2.101:0/592598585 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:19.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.649 7f2fbb7fe700 1 -- 172.21.2.101:0/592598585 >> v2:172.21.2.108:3300/0 conn(0x7f2fbc0e5070 msgr2=0x7f2fbc1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:19.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.649 7f2fbb7fe700 1 --2- 172.21.2.101:0/592598585 >> v2:172.21.2.108:3300/0 conn(0x7f2fbc0e5070 0x7f2fbc1b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:19.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.649 7f2fbb7fe700 1 -- 172.21.2.101:0/592598585 >> v1:172.21.2.108:6789/0 conn(0x7f2fbc0e5470 legacy=0x7f2fbc07dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:19.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.649 7f2fbb7fe700 1 -- 172.21.2.101:0/592598585 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc080750 con 0x7f2fbc1b3e40 2021-02-10T11:33:19.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.649 7f2fbb7fe700 1 --2- 172.21.2.101:0/592598585 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fbc1b3e40 0x7f2fbc07d690 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f2fb0006700 tx=0x7f2fb0009b80).ready entity=mon.0 client_cookie=c726410ca45b8fe1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:19.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.650 7f2fbaffd700 1 -- 172.21.2.101:0/592598585 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2fb000c0c0 con 0x7f2fbc1b3e40 2021-02-10T11:33:19.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.650 7f2fbaffd700 1 -- 172.21.2.101:0/592598585 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb000c440 con 0x7f2fbc1b3e40 2021-02-10T11:33:19.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.650 7f2fbaffd700 1 -- 172.21.2.101:0/592598585 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2fb000c960 con 0x7f2fbc1b3e40 2021-02-10T11:33:19.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.650 7f2fc286d700 1 -- 172.21.2.101:0/592598585 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fbc1b3e40 msgr2=0x7f2fbc07d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:19.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.650 7f2fc286d700 1 --2- 172.21.2.101:0/592598585 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fbc1b3e40 0x7f2fbc07d690 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f2fb0006700 tx=0x7f2fb0009b80).stop 2021-02-10T11:33:19.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.650 7f2fc286d700 1 -- 172.21.2.101:0/592598585 shutdown_connections 2021-02-10T11:33:19.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.650 7f2fc286d700 1 --2- 172.21.2.101:0/592598585 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fbc1b3e40 0x7f2fbc07d690 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:19.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.650 7f2fc286d700 1 --2- 172.21.2.101:0/592598585 >> v2:172.21.2.108:3300/0 conn(0x7f2fbc0e5070 0x7f2fbc1b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:19.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.650 7f2fc286d700 1 -- 172.21.2.101:0/592598585 >> 172.21.2.101:0/592598585 conn(0x7f2fbc0ce690 msgr2=0x7f2fbc0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:19.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.650 7f2fc286d700 1 -- 172.21.2.101:0/592598585 shutdown_connections 2021-02-10T11:33:19.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.650 7f2fc286d700 1 -- 172.21.2.101:0/592598585 wait complete. 2021-02-10T11:33:19.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.665 7f2fc286d700 1 Processor -- start 2021-02-10T11:33:19.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.665 7f2fc286d700 1 -- start start 2021-02-10T11:33:19.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.665 7f2fc286d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fbc1aa290 0x7f2fbc1aa690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:19.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.665 7f2fc286d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2fbc0b7270 0x7f2fbc0b7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:19.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.665 7f2fc286d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2fbc0b7c90 0x7f2fbc084e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:19.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.665 7f2fc286d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc0b8090 con 0x7f2fbc1aa290 2021-02-10T11:33:19.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.665 7f2fc286d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc1a9df0 con 0x7f2fbc0b7c90 2021-02-10T11:33:19.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.665 7f2fc286d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc09cc30 con 0x7f2fbc0b7270 2021-02-10T11:33:19.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.666 7f2fbbfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fbc1aa290 0x7f2fbc1aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:19.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.666 7f2fbbfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fbc1aa290 0x7f2fbc1aa690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51634/0 (socket says 172.21.2.101:51634) 2021-02-10T11:33:19.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.666 7f2fbbfff700 1 -- 172.21.2.101:0/467322201 learned_addr learned my addr 172.21.2.101:0/467322201 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:19.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.666 7f2fbb7fe700 1 --2- 172.21.2.101:0/467322201 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2fbc0b7270 0x7f2fbc0b7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:19.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.666 7f2fbbfff700 1 -- 172.21.2.101:0/467322201 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2fbc0b7270 msgr2=0x7f2fbc0b7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:19.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.666 7f2fbbfff700 1 --2- 172.21.2.101:0/467322201 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2fbc0b7270 0x7f2fbc0b7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:19.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.666 7f2fc0e8a700 1 --2- 172.21.2.101:0/467322201 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2fbc0b7c90 0x7f2fbc084e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:19.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.666 7f2fbbfff700 1 -- 172.21.2.101:0/467322201 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2fbc0b7c90 msgr2=0x7f2fbc084e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:19.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.666 7f2fbbfff700 1 --2- 172.21.2.101:0/467322201 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2fbc0b7c90 0x7f2fbc084e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:19.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.666 7f2fbbfff700 1 -- 172.21.2.101:0/467322201 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc085430 con 0x7f2fbc1aa290 2021-02-10T11:33:19.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.666 7f2fbb7fe700 1 --2- 172.21.2.101:0/467322201 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2fbc0b7270 0x7f2fbc0b7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:33:19.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.666 7f2fc0e8a700 1 --2- 172.21.2.101:0/467322201 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2fbc0b7c90 0x7f2fbc084e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:33:19.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.666 7f2fbbfff700 1 --2- 172.21.2.101:0/467322201 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fbc1aa290 0x7f2fbc1aa690 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7f2fac006f00 tx=0x7f2fac00b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:19.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.667 7f2fb8ff9700 1 -- 172.21.2.101:0/467322201 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2fac0009b0 con 0x7f2fbc1aa290 2021-02-10T11:33:19.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.667 7f2fb8ff9700 1 -- 172.21.2.101:0/467322201 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fac006350 con 0x7f2fbc1aa290 2021-02-10T11:33:19.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.667 7f2fb8ff9700 1 -- 172.21.2.101:0/467322201 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2fac000c90 con 0x7f2fbc1aa290 2021-02-10T11:33:19.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.667 7f2fc286d700 1 -- 172.21.2.101:0/467322201 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fbc085430 con 0x7f2fbc1aa290 2021-02-10T11:33:19.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.667 7f2fc286d700 1 -- 172.21.2.101:0/467322201 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fbc0859c0 con 0x7f2fbc1aa290 2021-02-10T11:33:19.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.667 7f2fb8ff9700 1 -- 172.21.2.101:0/467322201 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2fac000c90 con 0x7f2fbc1aa290 2021-02-10T11:33:19.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.667 7f2fb8ff9700 1 --2- 172.21.2.101:0/467322201 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f9c0186c0 0x7f2f9c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:19.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.667 7f2fb8ff9700 1 -- 172.21.2.101:0/467322201 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2fac032590 con 0x7f2fbc1aa290 2021-02-10T11:33:19.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.668 7f2fbb7fe700 1 --2- 172.21.2.101:0/467322201 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f9c0186c0 0x7f2f9c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:19.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.668 7f2fbb7fe700 1 --2- 172.21.2.101:0/467322201 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f9c0186c0 0x7f2f9c01abc0 crc :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:19.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.699 7f2fc286d700 1 -- 172.21.2.101:0/467322201 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fbc083da0 con 0x7f2fbc1aa290 2021-02-10T11:33:19.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.701 7f2fb8ff9700 1 -- 172.21.2.101:0/467322201 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2fac000960 con 0x7f2fbc1aa290 2021-02-10T11:33:19.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.826 7f2fc286d700 1 -- 172.21.2.101:0/467322201 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fbc07ed80 con 0x7f2fbc1aa290 2021-02-10T11:33:19.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.826 7f2fb8ff9700 1 -- 172.21.2.101:0/467322201 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+13996 (secure 0 0 0) 0x7f2fac000c90 con 0x7f2fbc1aa290 2021-02-10T11:33:19.838 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:19.838 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":55,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:33:17.591716","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:33:14.515763","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2021-02-10 11:33:15.636274","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:33:19.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.828 7f2fc286d700 1 -- 172.21.2.101:0/467322201 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f9c0186c0 msgr2=0x7f2f9c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:19.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.828 7f2fc286d700 1 --2- 172.21.2.101:0/467322201 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f9c0186c0 0x7f2f9c01abc0 crc :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:19.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.828 7f2fc286d700 1 -- 172.21.2.101:0/467322201 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fbc1aa290 msgr2=0x7f2fbc1aa690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:19.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.828 7f2fc286d700 1 --2- 172.21.2.101:0/467322201 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fbc1aa290 0x7f2fbc1aa690 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7f2fac006f00 tx=0x7f2fac00b940).stop 2021-02-10T11:33:19.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.828 7f2fc286d700 1 -- 172.21.2.101:0/467322201 shutdown_connections 2021-02-10T11:33:19.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.828 7f2fc286d700 1 --2- 172.21.2.101:0/467322201 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f9c0186c0 0x7f2f9c01abc0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:19.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.828 7f2fc286d700 1 --2- 172.21.2.101:0/467322201 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2fbc0b7270 0x7f2fbc0b7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:19.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.828 7f2fc286d700 1 --2- 172.21.2.101:0/467322201 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2fbc0b7c90 0x7f2fbc084e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:19.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.828 7f2fc286d700 1 --2- 172.21.2.101:0/467322201 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fbc1aa290 0x7f2fbc1aa690 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:19.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.828 7f2fc286d700 1 -- 172.21.2.101:0/467322201 >> 172.21.2.101:0/467322201 conn(0x7f2fbc08fe10 msgr2=0x7f2fbc19c020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:19.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.828 7f2fc286d700 1 -- 172.21.2.101:0/467322201 shutdown_connections 2021-02-10T11:33:19.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:19.828 7f2fc286d700 1 -- 172.21.2.101:0/467322201 wait complete. 2021-02-10T11:33:19.848 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:33:20.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.259 7fee0f649700 1 Processor -- start 2021-02-10T11:33:20.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.259 7fee0f649700 1 -- start start 2021-02-10T11:33:20.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.260 7fee0f649700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fee081b0470 0x7fee081b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:20.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.260 7fee0f649700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fee080ef610 0x7fee080efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:20.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.260 7fee0f649700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fee080f0030 0x7fee080f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:20.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.260 7fee0f649700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee080e5ea0 con 0x7fee080f0030 2021-02-10T11:33:20.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.260 7fee0f649700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee080f0b50 con 0x7fee081b0470 2021-02-10T11:33:20.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.260 7fee0f649700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee080e52e0 con 0x7fee080ef610 2021-02-10T11:33:20.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.260 7fee0dc66700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fee080f0030 0x7fee080f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:20.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.260 7fee0dc66700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fee080f0030 0x7fee080f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51640/0 (socket says 172.21.2.101:51640) 2021-02-10T11:33:20.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.260 7fee0dc66700 1 -- 172.21.2.101:0/3154083670 learned_addr learned my addr 172.21.2.101:0/3154083670 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:20.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.261 7fee0cc64700 1 --2- 172.21.2.101:0/3154083670 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fee080ef610 0x7fee080efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:20.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.261 7fee0dc66700 1 -- 172.21.2.101:0/3154083670 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fee080ef610 msgr2=0x7fee080efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:20.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.261 7fee0d465700 1 --2- 172.21.2.101:0/3154083670 >> v2:172.21.2.108:3300/0 conn(0x7fee081b0470 0x7fee081b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:20.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.261 7fee0dc66700 1 --2- 172.21.2.101:0/3154083670 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fee080ef610 0x7fee080efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:20.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.261 7fee0dc66700 1 -- 172.21.2.101:0/3154083670 >> v2:172.21.2.108:3300/0 conn(0x7fee081b0470 msgr2=0x7fee081b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:20.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.261 7fee0dc66700 1 --2- 172.21.2.101:0/3154083670 >> v2:172.21.2.108:3300/0 conn(0x7fee081b0470 0x7fee081b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:20.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.261 7fee0dc66700 1 -- 172.21.2.101:0/3154083670 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee080f10d0 con 0x7fee080f0030 2021-02-10T11:33:20.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.261 7fee0cc64700 1 --2- 172.21.2.101:0/3154083670 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fee080ef610 0x7fee080efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:33:20.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.261 7fee0dc66700 1 --2- 172.21.2.101:0/3154083670 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fee080f0030 0x7fee080f0430 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 rx=0x7fee00006b80 tx=0x7fee00009b90).ready entity=mon.0 client_cookie=8dc08124fbbda0a4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:20.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.261 7fedfffff700 1 -- 172.21.2.101:0/3154083670 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fee000022e0 con 0x7fee080f0030 2021-02-10T11:33:20.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.261 7fedfffff700 1 -- 172.21.2.101:0/3154083670 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee00002620 con 0x7fee080f0030 2021-02-10T11:33:20.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.261 7fedfffff700 1 -- 172.21.2.101:0/3154083670 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fee00002b00 con 0x7fee080f0030 2021-02-10T11:33:20.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.261 7fee0f649700 1 -- 172.21.2.101:0/3154083670 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fee080f0030 msgr2=0x7fee080f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:20.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.261 7fee0f649700 1 --2- 172.21.2.101:0/3154083670 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fee080f0030 0x7fee080f0430 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 rx=0x7fee00006b80 tx=0x7fee00009b90).stop 2021-02-10T11:33:20.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.262 7fee0f649700 1 -- 172.21.2.101:0/3154083670 shutdown_connections 2021-02-10T11:33:20.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.262 7fee0f649700 1 --2- 172.21.2.101:0/3154083670 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fee080ef610 0x7fee080efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:20.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.262 7fee0f649700 1 --2- 172.21.2.101:0/3154083670 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fee080f0030 0x7fee080f0430 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:20.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.262 7fee0f649700 1 --2- 172.21.2.101:0/3154083670 >> v2:172.21.2.108:3300/0 conn(0x7fee081b0470 0x7fee081b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:20.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.262 7fee0f649700 1 -- 172.21.2.101:0/3154083670 >> 172.21.2.101:0/3154083670 conn(0x7fee080ce3e0 msgr2=0x7fee080e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:20.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.262 7fee0f649700 1 -- 172.21.2.101:0/3154083670 shutdown_connections 2021-02-10T11:33:20.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.262 7fee0f649700 1 -- 172.21.2.101:0/3154083670 wait complete. 2021-02-10T11:33:20.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.277 7fee0f649700 1 Processor -- start 2021-02-10T11:33:20.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.277 7fee0f649700 1 -- start start 2021-02-10T11:33:20.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.277 7fee0f649700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fee080aa9d0 0x7fee0818e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:20.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.277 7fee0f649700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fee0818ea70 0x7fee0819feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:20.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.277 7fee0f649700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fee081a04d0 0x7fee081a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:20.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.277 7fee0f649700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0818ee70 con 0x7fee0818ea70 2021-02-10T11:33:20.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.277 7fee0f649700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0818f090 con 0x7fee081a04d0 2021-02-10T11:33:20.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.277 7fee0f649700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee081a1550 con 0x7fee080aa9d0 2021-02-10T11:33:20.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.277 7fee0cc64700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fee0818ea70 0x7fee0819feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:20.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.277 7fee0cc64700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fee0818ea70 0x7fee0819feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51644/0 (socket says 172.21.2.101:51644) 2021-02-10T11:33:20.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.277 7fee0cc64700 1 -- 172.21.2.101:0/2091637646 learned_addr learned my addr 172.21.2.101:0/2091637646 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:20.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.278 7fee0d465700 1 --2- 172.21.2.101:0/2091637646 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fee080aa9d0 0x7fee0818e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:20.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.278 7fee0dc66700 1 --2- 172.21.2.101:0/2091637646 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fee081a04d0 0x7fee081a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:20.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.278 7fee0cc64700 1 -- 172.21.2.101:0/2091637646 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fee080aa9d0 msgr2=0x7fee0818e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:20.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.278 7fee0cc64700 1 --2- 172.21.2.101:0/2091637646 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fee080aa9d0 0x7fee0818e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:20.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.278 7fee0cc64700 1 -- 172.21.2.101:0/2091637646 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fee081a04d0 msgr2=0x7fee081a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:20.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.278 7fee0cc64700 1 --2- 172.21.2.101:0/2091637646 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fee081a04d0 0x7fee081a08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:20.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.278 7fee0cc64700 1 -- 172.21.2.101:0/2091637646 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee081a1770 con 0x7fee0818ea70 2021-02-10T11:33:20.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.278 7fee0cc64700 1 --2- 172.21.2.101:0/2091637646 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fee0818ea70 0x7fee0819feb0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7fedf8006f20 tx=0x7fedf800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:20.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.278 7fedfdffb700 1 -- 172.21.2.101:0/2091637646 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fedf800bd80 con 0x7fee0818ea70 2021-02-10T11:33:20.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.278 7fedfdffb700 1 -- 172.21.2.101:0/2091637646 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf8000c90 con 0x7fee0818ea70 2021-02-10T11:33:20.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.278 7fee0f649700 1 -- 172.21.2.101:0/2091637646 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee081a1770 con 0x7fee0818ea70 2021-02-10T11:33:20.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.279 7fedfdffb700 1 -- 172.21.2.101:0/2091637646 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fedf80022b0 con 0x7fee0818ea70 2021-02-10T11:33:20.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.279 7fee0f649700 1 -- 172.21.2.101:0/2091637646 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee081a1770 con 0x7fee0818ea70 2021-02-10T11:33:20.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.279 7fedfdffb700 1 -- 172.21.2.101:0/2091637646 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fedf800a000 con 0x7fee0818ea70 2021-02-10T11:33:20.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.279 7fedfdffb700 1 --2- 172.21.2.101:0/2091637646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fede80186c0 0x7fede801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:20.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.279 7fedfdffb700 1 -- 172.21.2.101:0/2091637646 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fedf8032590 con 0x7fee0818ea70 2021-02-10T11:33:20.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.279 7fee0d465700 1 --2- 172.21.2.101:0/2091637646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fede80186c0 0x7fede801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:20.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.280 7fee0d465700 1 --2- 172.21.2.101:0/2091637646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fede80186c0 0x7fede801abc0 crc :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:20.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.310 7fee0f649700 1 -- 172.21.2.101:0/2091637646 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee08105a30 con 0x7fee0818ea70 2021-02-10T11:33:20.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.312 7fedfdffb700 1 -- 172.21.2.101:0/2091637646 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fedf8006bb0 con 0x7fee0818ea70 2021-02-10T11:33:20.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.442 7fee0f649700 1 -- 172.21.2.101:0/2091637646 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fee08105a30 con 0x7fee0818ea70 2021-02-10T11:33:20.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.446 7fedfdffb700 1 -- 172.21.2.101:0/2091637646 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v31) v1 ==== 184+0+0 (secure 0 0 0) 0x7fedf801f4f0 con 0x7fee0818ea70 2021-02-10T11:33:20.460 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.0 2021-02-10T11:33:20.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.451 7fee0f649700 1 -- 172.21.2.101:0/2091637646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fede80186c0 msgr2=0x7fede801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:20.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.451 7fee0f649700 1 --2- 172.21.2.101:0/2091637646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fede80186c0 0x7fede801abc0 crc :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:20.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.451 7fee0f649700 1 -- 172.21.2.101:0/2091637646 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fee0818ea70 msgr2=0x7fee0819feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:20.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.451 7fee0f649700 1 --2- 172.21.2.101:0/2091637646 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fee0818ea70 0x7fee0819feb0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7fedf8006f20 tx=0x7fedf800b990).stop 2021-02-10T11:33:20.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.451 7fee0f649700 1 -- 172.21.2.101:0/2091637646 shutdown_connections 2021-02-10T11:33:20.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.451 7fee0f649700 1 --2- 172.21.2.101:0/2091637646 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fee081a04d0 0x7fee081a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:20.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.451 7fee0f649700 1 --2- 172.21.2.101:0/2091637646 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fee0818ea70 0x7fee0819feb0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:20.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.451 7fee0f649700 1 --2- 172.21.2.101:0/2091637646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fede80186c0 0x7fede801abc0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:20.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.451 7fee0f649700 1 --2- 172.21.2.101:0/2091637646 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fee080aa9d0 0x7fee0818e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:20.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.451 7fee0f649700 1 -- 172.21.2.101:0/2091637646 >> 172.21.2.101:0/2091637646 conn(0x7fee081a3790 msgr2=0x7fee081aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:20.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.451 7fee0f649700 1 -- 172.21.2.101:0/2091637646 shutdown_connections 2021-02-10T11:33:20.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.451 7fee0f649700 1 -- 172.21.2.101:0/2091637646 wait complete. 2021-02-10T11:33:20.470 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:33:20.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.881 7f65b0437700 1 Processor -- start 2021-02-10T11:33:20.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.881 7f65b0437700 1 -- start start 2021-02-10T11:33:20.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.882 7f65b0437700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65a81b0470 0x7f65a81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:20.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.882 7f65b0437700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f65a80ef610 0x7f65a80efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:20.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.882 7f65b0437700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a80f0b50 con 0x7f65a80f0030 2021-02-10T11:33:20.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.882 7f65b0437700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a80e52e0 con 0x7f65a80ef610 2021-02-10T11:33:20.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.882 7f65b0437700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a80e5490 con 0x7f65a81b0470 2021-02-10T11:33:20.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.882 7f65ada52700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f65a80ef610 0x7f65a80efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:20.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.882 7f65ae253700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65a81b0470 0x7f65a81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:20.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.882 7f65ae253700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65a81b0470 0x7f65a81b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:44384/0 (socket says 172.21.2.101:44384) 2021-02-10T11:33:20.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.882 7f65ada52700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f65a80ef610 0x7f65a80efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:33906/0 (socket says 172.21.2.101:33906) 2021-02-10T11:33:20.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.882 7f65ae253700 1 -- 172.21.2.101:0/3516070600 learned_addr learned my addr 172.21.2.101:0/3516070600 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:20.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.883 7f65ae253700 1 -- 172.21.2.101:0/3516070600 >> v2:172.21.2.108:3300/0 conn(0x7f65a80ef610 msgr2=0x7f65a80efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:20.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.883 7f65ae253700 1 --2- 172.21.2.101:0/3516070600 >> v2:172.21.2.108:3300/0 conn(0x7f65a80ef610 0x7f65a80efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:20.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.883 7f65ae253700 1 -- 172.21.2.101:0/3516070600 >> v1:172.21.2.108:6789/0 conn(0x7f65a80f0030 legacy=0x7f65a80f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:20.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.883 7f65ae253700 1 -- 172.21.2.101:0/3516070600 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a80f0430 con 0x7f65a81b0470 2021-02-10T11:33:20.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.883 7f65ad251700 1 -- 172.21.2.101:0/3516070600 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155543198 0 0) 0x7f65a0001400 con 0x7f65a80f0030 2021-02-10T11:33:20.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.883 7f65ada52700 1 --2- 172.21.2.101:0/3516070600 >> v2:172.21.2.108:3300/0 conn(0x7f65a80ef610 0x7f65a80efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:33:20.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.883 7f65ae253700 1 --2- 172.21.2.101:0/3516070600 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65a81b0470 0x7f65a81b0870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f65a4006700 tx=0x7f65a4009b80).ready entity=mon.2 client_cookie=9330effc96f5f703 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:20.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.884 7f65ad251700 1 -- 172.21.2.101:0/3516070600 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f65a400c0c0 con 0x7f65a81b0470 2021-02-10T11:33:20.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.884 7f65ad251700 1 -- 172.21.2.101:0/3516070600 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a400c340 con 0x7f65a81b0470 2021-02-10T11:33:20.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.884 7f65ad251700 1 -- 172.21.2.101:0/3516070600 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f65a400c860 con 0x7f65a81b0470 2021-02-10T11:33:20.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.884 7f65b0437700 1 -- 172.21.2.101:0/3516070600 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65a81b0470 msgr2=0x7f65a81b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:20.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.884 7f65b0437700 1 --2- 172.21.2.101:0/3516070600 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65a81b0470 0x7f65a81b0870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f65a4006700 tx=0x7f65a4009b80).stop 2021-02-10T11:33:20.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.885 7f65b0437700 1 -- 172.21.2.101:0/3516070600 shutdown_connections 2021-02-10T11:33:20.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.885 7f65b0437700 1 --2- 172.21.2.101:0/3516070600 >> v2:172.21.2.108:3300/0 conn(0x7f65a80ef610 0x7f65a80efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:20.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.885 7f65b0437700 1 --2- 172.21.2.101:0/3516070600 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65a81b0470 0x7f65a81b0870 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:20.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.885 7f65b0437700 1 -- 172.21.2.101:0/3516070600 >> 172.21.2.101:0/3516070600 conn(0x7f65a80ce3e0 msgr2=0x7f65a80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:20.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.885 7f65b0437700 1 -- 172.21.2.101:0/3516070600 shutdown_connections 2021-02-10T11:33:20.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.885 7f65b0437700 1 -- 172.21.2.101:0/3516070600 wait complete. 2021-02-10T11:33:20.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.900 7f65b0437700 1 Processor -- start 2021-02-10T11:33:20.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.900 7f65b0437700 1 -- start start 2021-02-10T11:33:20.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.900 7f65b0437700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65a80b65c0 0x7f65a81a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:20.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.900 7f65b0437700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65a81a8030 0x7f65a80b7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:20.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.900 7f65b0437700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65a80b69c0 0x7f65a80b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:20.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.900 7f65b0437700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a80b6ec0 con 0x7f65a80b69c0 2021-02-10T11:33:20.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.900 7f65b0437700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a80b8080 con 0x7f65a81a8030 2021-02-10T11:33:20.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.900 7f65b0437700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a80b8260 con 0x7f65a80b65c0 2021-02-10T11:33:20.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.900 7f65aea54700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65a80b69c0 0x7f65a80b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:20.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.900 7f65aea54700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65a80b69c0 0x7f65a80b7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51660/0 (socket says 172.21.2.101:51660) 2021-02-10T11:33:20.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.900 7f65aea54700 1 -- 172.21.2.101:0/1253856610 learned_addr learned my addr 172.21.2.101:0/1253856610 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:20.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.901 7f65ae253700 1 --2- 172.21.2.101:0/1253856610 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65a80b65c0 0x7f65a81a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:20.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.901 7f65aea54700 1 -- 172.21.2.101:0/1253856610 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65a80b65c0 msgr2=0x7f65a81a7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:20.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.901 7f65aea54700 1 --2- 172.21.2.101:0/1253856610 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65a80b65c0 0x7f65a81a7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:20.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.901 7f65aea54700 1 -- 172.21.2.101:0/1253856610 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65a81a8030 msgr2=0x7f65a80b7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:20.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.901 7f65aea54700 1 --2- 172.21.2.101:0/1253856610 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65a81a8030 0x7f65a80b7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:20.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.901 7f65aea54700 1 -- 172.21.2.101:0/1253856610 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a80b8480 con 0x7f65a80b69c0 2021-02-10T11:33:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.901 7f65aea54700 1 --2- 172.21.2.101:0/1253856610 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65a80b69c0 0x7f65a80b7a60 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 rx=0x7f65a0009940 tx=0x7f65a000bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.901 7f659effd700 1 -- 172.21.2.101:0/1253856610 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f65a000c2f0 con 0x7f65a80b69c0 2021-02-10T11:33:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.901 7f659effd700 1 -- 172.21.2.101:0/1253856610 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a000c570 con 0x7f65a80b69c0 2021-02-10T11:33:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.901 7f65b0437700 1 -- 172.21.2.101:0/1253856610 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a80b8480 con 0x7f65a80b69c0 2021-02-10T11:33:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.901 7f65b0437700 1 -- 172.21.2.101:0/1253856610 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a80bc420 con 0x7f65a80b69c0 2021-02-10T11:33:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.901 7f659effd700 1 -- 172.21.2.101:0/1253856610 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f65a000ca50 con 0x7f65a80b69c0 2021-02-10T11:33:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.902 7f659effd700 1 -- 172.21.2.101:0/1253856610 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f65a000c7b0 con 0x7f65a80b69c0 2021-02-10T11:33:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.902 7f659effd700 1 --2- 172.21.2.101:0/1253856610 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6590018730 0x7f659001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.902 7f659effd700 1 -- 172.21.2.101:0/1253856610 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f65a0032bd0 con 0x7f65a80b69c0 2021-02-10T11:33:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.902 7f65ae253700 1 --2- 172.21.2.101:0/1253856610 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6590018730 0x7f659001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:20.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.902 7f65ae253700 1 --2- 172.21.2.101:0/1253856610 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6590018730 0x7f659001ac10 crc :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:20.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.931 7f65b0437700 1 -- 172.21.2.101:0/1253856610 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a81059f0 con 0x7f65a80b69c0 2021-02-10T11:33:20.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:20.934 7f659effd700 1 -- 172.21.2.101:0/1253856610 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f65a000c570 con 0x7f65a80b69c0 2021-02-10T11:33:21.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.064 7f65b0437700 1 -- 172.21.2.101:0/1253856610 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f65a80f0b50 con 0x7f65a80b69c0 2021-02-10T11:33:21.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.067 7f659effd700 1 -- 172.21.2.101:0/1253856610 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v32) v1 ==== 184+0+0 (secure 0 0 0) 0x7f65a000c660 con 0x7f65a80b69c0 2021-02-10T11:33:21.082 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.1 2021-02-10T11:33:21.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.073 7f65b0437700 1 -- 172.21.2.101:0/1253856610 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6590018730 msgr2=0x7f659001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:21.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.073 7f65b0437700 1 --2- 172.21.2.101:0/1253856610 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6590018730 0x7f659001ac10 crc :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:21.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.073 7f65b0437700 1 -- 172.21.2.101:0/1253856610 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65a80b69c0 msgr2=0x7f65a80b7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:21.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.073 7f65b0437700 1 --2- 172.21.2.101:0/1253856610 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65a80b69c0 0x7f65a80b7a60 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 rx=0x7f65a0009940 tx=0x7f65a000bcf0).stop 2021-02-10T11:33:21.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.074 7f65b0437700 1 -- 172.21.2.101:0/1253856610 shutdown_connections 2021-02-10T11:33:21.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.074 7f65b0437700 1 --2- 172.21.2.101:0/1253856610 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65a80b69c0 0x7f65a80b7a60 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:21.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.074 7f65b0437700 1 --2- 172.21.2.101:0/1253856610 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65a81a8030 0x7f65a80b7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:21.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.074 7f65b0437700 1 --2- 172.21.2.101:0/1253856610 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6590018730 0x7f659001ac10 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:21.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.074 7f65b0437700 1 --2- 172.21.2.101:0/1253856610 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65a80b65c0 0x7f65a81a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:21.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.074 7f65b0437700 1 -- 172.21.2.101:0/1253856610 >> 172.21.2.101:0/1253856610 conn(0x7f65a80b5850 msgr2=0x7f65a8093a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:21.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.074 7f65b0437700 1 -- 172.21.2.101:0/1253856610 shutdown_connections 2021-02-10T11:33:21.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.074 7f65b0437700 1 -- 172.21.2.101:0/1253856610 wait complete. 2021-02-10T11:33:21.093 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:33:21.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.505 7f375085b700 1 Processor -- start 2021-02-10T11:33:21.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.505 7f375085b700 1 -- start start 2021-02-10T11:33:21.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.505 7f375085b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f374c1b0470 0x7f374c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:21.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.505 7f375085b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f374c0ef610 0x7f374c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:21.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.505 7f375085b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f374c0f0b50 con 0x7f374c0f0030 2021-02-10T11:33:21.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.505 7f375085b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f374c0e52e0 con 0x7f374c0ef610 2021-02-10T11:33:21.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.505 7f375085b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374c0e5490 con 0x7f374c1b0470 2021-02-10T11:33:21.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.506 7f374ae1d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f374c0f0030 0x7f374c0f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52158/0 (socket says 172.21.2.101:52158) 2021-02-10T11:33:21.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.506 7f374ae1d700 1 -- 172.21.2.101:0/196835670 learned_addr learned my addr 172.21.2.101:0/196835670 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:21.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.506 7f3749e1b700 1 --2- 172.21.2.101:0/196835670 >> v2:172.21.2.108:3300/0 conn(0x7f374c0ef610 0x7f374c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:21.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.506 7f374a61c700 1 --2- 172.21.2.101:0/196835670 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f374c1b0470 0x7f374c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:21.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.506 7f374961a700 1 -- 172.21.2.101:0/196835670 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2861447052 0 0) 0x7f3744001500 con 0x7f374c0f0030 2021-02-10T11:33:21.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.506 7f374961a700 1 -- 172.21.2.101:0/196835670 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3738002fe0 con 0x7f374c0f0030 2021-02-10T11:33:21.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.506 7f3749e1b700 1 -- 172.21.2.101:0/196835670 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f374c1b0470 msgr2=0x7f374c1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:21.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.506 7f3749e1b700 1 --2- 172.21.2.101:0/196835670 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f374c1b0470 0x7f374c1b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:21.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.506 7f3749e1b700 1 -- 172.21.2.101:0/196835670 >> v1:172.21.2.101:6789/0 conn(0x7f374c0f0030 legacy=0x7f374c0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:21.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.506 7f3749e1b700 1 -- 172.21.2.101:0/196835670 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374c0f0430 con 0x7f374c0ef610 2021-02-10T11:33:21.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.506 7f3749e1b700 1 --2- 172.21.2.101:0/196835670 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f374c0ef610 0x7f374c0efa10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f3734006650 tx=0x7f3734009a80).ready entity=mon.1 client_cookie=6881d921a7a69ba0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:21.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.506 7f374961a700 1 -- 172.21.2.101:0/196835670 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f373400c0c0 con 0x7f374c0ef610 2021-02-10T11:33:21.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.507 7f374961a700 1 -- 172.21.2.101:0/196835670 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373400c340 con 0x7f374c0ef610 2021-02-10T11:33:21.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.507 7f374961a700 1 -- 172.21.2.101:0/196835670 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f373400c790 con 0x7f374c0ef610 2021-02-10T11:33:21.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.507 7f375085b700 1 -- 172.21.2.101:0/196835670 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f374c0ef610 msgr2=0x7f374c0efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:21.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.507 7f375085b700 1 --2- 172.21.2.101:0/196835670 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f374c0ef610 0x7f374c0efa10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f3734006650 tx=0x7f3734009a80).stop 2021-02-10T11:33:21.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.507 7f375085b700 1 -- 172.21.2.101:0/196835670 shutdown_connections 2021-02-10T11:33:21.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.507 7f375085b700 1 --2- 172.21.2.101:0/196835670 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f374c0ef610 0x7f374c0efa10 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:21.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.507 7f375085b700 1 --2- 172.21.2.101:0/196835670 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f374c1b0470 0x7f374c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:21.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.507 7f375085b700 1 -- 172.21.2.101:0/196835670 >> 172.21.2.101:0/196835670 conn(0x7f374c0ce3e0 msgr2=0x7f374c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:21.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.507 7f375085b700 1 -- 172.21.2.101:0/196835670 shutdown_connections 2021-02-10T11:33:21.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.507 7f375085b700 1 -- 172.21.2.101:0/196835670 wait complete. 2021-02-10T11:33:21.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.522 7f375085b700 1 Processor -- start 2021-02-10T11:33:21.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.522 7f375085b700 1 -- start start 2021-02-10T11:33:21.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.522 7f375085b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f374c1a0480 0x7f374c08e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:21.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.522 7f375085b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f374c10a4b0 0x7f374c08ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:21.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.522 7f375085b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f374c18ebf0 0x7f374c18eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:21.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.522 7f375085b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374c18f720 con 0x7f374c1a0480 2021-02-10T11:33:21.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.522 7f375085b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374c18f940 con 0x7f374c10a4b0 2021-02-10T11:33:21.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.522 7f375085b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374c18fb60 con 0x7f374c18ebf0 2021-02-10T11:33:21.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.523 7f374a61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f374c1a0480 0x7f374c08e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:21.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.523 7f374a61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f374c1a0480 0x7f374c08e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51668/0 (socket says 172.21.2.101:51668) 2021-02-10T11:33:21.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.523 7f374a61c700 1 -- 172.21.2.101:0/57887684 learned_addr learned my addr 172.21.2.101:0/57887684 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:21.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.523 7f374a61c700 1 -- 172.21.2.101:0/57887684 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f374c18ebf0 msgr2=0x7f374c18eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:21.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.523 7f374a61c700 1 --2- 172.21.2.101:0/57887684 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f374c18ebf0 0x7f374c18eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:21.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.523 7f374a61c700 1 -- 172.21.2.101:0/57887684 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f374c10a4b0 msgr2=0x7f374c08ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:21.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.523 7f374a61c700 1 --2- 172.21.2.101:0/57887684 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f374c10a4b0 0x7f374c08ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:21.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.523 7f374a61c700 1 -- 172.21.2.101:0/57887684 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374c18fd80 con 0x7f374c1a0480 2021-02-10T11:33:21.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.523 7f374a61c700 1 --2- 172.21.2.101:0/57887684 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f374c1a0480 0x7f374c08e840 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 rx=0x7f373c006f60 tx=0x7f373c00b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:21.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.523 7f3742ffd700 1 -- 172.21.2.101:0/57887684 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f373c0009b0 con 0x7f374c1a0480 2021-02-10T11:33:21.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.523 7f3742ffd700 1 -- 172.21.2.101:0/57887684 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373c00bd60 con 0x7f374c1a0480 2021-02-10T11:33:21.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.523 7f3742ffd700 1 -- 172.21.2.101:0/57887684 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f373c002030 con 0x7f374c1a0480 2021-02-10T11:33:21.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.523 7f375085b700 1 -- 172.21.2.101:0/57887684 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f374c18fd80 con 0x7f374c1a0480 2021-02-10T11:33:21.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.523 7f375085b700 1 -- 172.21.2.101:0/57887684 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f374c08fdb0 con 0x7f374c1a0480 2021-02-10T11:33:21.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.524 7f3742ffd700 1 -- 172.21.2.101:0/57887684 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f373c006b00 con 0x7f374c1a0480 2021-02-10T11:33:21.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.524 7f3742ffd700 1 --2- 172.21.2.101:0/57887684 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37300186c0 0x7f373001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:21.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.524 7f3742ffd700 1 -- 172.21.2.101:0/57887684 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f373c032640 con 0x7f374c1a0480 2021-02-10T11:33:21.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.524 7f3749e1b700 1 --2- 172.21.2.101:0/57887684 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37300186c0 0x7f373001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:21.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.524 7f3749e1b700 1 --2- 172.21.2.101:0/57887684 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37300186c0 0x7f373001abc0 crc :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:21.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.555 7f375085b700 1 -- 172.21.2.101:0/57887684 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f374c1059f0 con 0x7f374c1a0480 2021-02-10T11:33:21.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.557 7f3742ffd700 1 -- 172.21.2.101:0/57887684 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f373c0009b0 con 0x7f374c1a0480 2021-02-10T11:33:21.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.686 7f375085b700 1 -- 172.21.2.101:0/57887684 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f374c0f0b50 con 0x7f374c1a0480 2021-02-10T11:33:21.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.690 7f3742ffd700 1 -- 172.21.2.101:0/57887684 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v33) v1 ==== 184+0+0 (secure 0 0 0) 0x7f373c006b00 con 0x7f374c1a0480 2021-02-10T11:33:21.704 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.2 2021-02-10T11:33:21.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.695 7f375085b700 1 -- 172.21.2.101:0/57887684 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37300186c0 msgr2=0x7f373001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:21.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.695 7f375085b700 1 --2- 172.21.2.101:0/57887684 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37300186c0 0x7f373001abc0 crc :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:21.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.695 7f375085b700 1 -- 172.21.2.101:0/57887684 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f374c1a0480 msgr2=0x7f374c08e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:21.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.695 7f375085b700 1 --2- 172.21.2.101:0/57887684 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f374c1a0480 0x7f374c08e840 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 rx=0x7f373c006f60 tx=0x7f373c00b970).stop 2021-02-10T11:33:21.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.695 7f375085b700 1 -- 172.21.2.101:0/57887684 shutdown_connections 2021-02-10T11:33:21.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.695 7f375085b700 1 --2- 172.21.2.101:0/57887684 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37300186c0 0x7f373001abc0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:21.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.695 7f375085b700 1 --2- 172.21.2.101:0/57887684 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f374c18ebf0 0x7f374c18eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:21.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.695 7f375085b700 1 --2- 172.21.2.101:0/57887684 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f374c10a4b0 0x7f374c08ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:21.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.695 7f375085b700 1 --2- 172.21.2.101:0/57887684 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f374c1a0480 0x7f374c08e840 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:21.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.695 7f375085b700 1 -- 172.21.2.101:0/57887684 >> 172.21.2.101:0/57887684 conn(0x7f374c09cd10 msgr2=0x7f374c093a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:21.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.695 7f375085b700 1 -- 172.21.2.101:0/57887684 shutdown_connections 2021-02-10T11:33:21.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:21.695 7f375085b700 1 -- 172.21.2.101:0/57887684 wait complete. 2021-02-10T11:33:21.714 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:33:22.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.124 7f1a3e382700 1 Processor -- start 2021-02-10T11:33:22.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.124 7f1a3e382700 1 -- start start 2021-02-10T11:33:22.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.124 7f1a3e382700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a381b0470 0x7f1a381b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:22.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.124 7f1a3e382700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1a380ef610 0x7f1a380efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:22.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.124 7f1a3e382700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a380f0b50 con 0x7f1a380f0030 2021-02-10T11:33:22.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.124 7f1a3e382700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a380e52e0 con 0x7f1a380ef610 2021-02-10T11:33:22.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.124 7f1a3e382700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a380e5490 con 0x7f1a381b0470 2021-02-10T11:33:22.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.125 7f1a377fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1a380ef610 0x7f1a380efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:22.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.125 7f1a377fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1a380ef610 0x7f1a380efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51676/0 (socket says 172.21.2.101:51676) 2021-02-10T11:33:22.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.125 7f1a377fe700 1 -- 172.21.2.101:0/842999937 learned_addr learned my addr 172.21.2.101:0/842999937 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:22.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.125 7f1a3c99f700 1 --1- 172.21.2.101:0/842999937 >> v1:172.21.2.101:6789/0 conn(0x7f1a380f0030 0x7f1a380f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52168/0 (socket says 172.21.2.101:52168) 2021-02-10T11:33:22.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.125 7f1a377fe700 1 -- 172.21.2.101:0/842999937 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a381b0470 msgr2=0x7f1a381b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:33:22.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.125 7f1a377fe700 1 --2- 172.21.2.101:0/842999937 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a381b0470 0x7f1a381b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:22.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.125 7f1a377fe700 1 -- 172.21.2.101:0/842999937 >> v1:172.21.2.101:6789/0 conn(0x7f1a380f0030 legacy=0x7f1a380f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:22.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.125 7f1a377fe700 1 -- 172.21.2.101:0/842999937 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a380f0430 con 0x7f1a380ef610 2021-02-10T11:33:22.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.125 7f1a377fe700 1 --2- 172.21.2.101:0/842999937 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a380ef610 0x7f1a380efa10 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7f1a2c006700 tx=0x7f1a2c009b80).ready entity=mon.0 client_cookie=f6004eaac45c0ce7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:22.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.125 7f1a36ffd700 1 -- 172.21.2.101:0/842999937 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1a2c00c0c0 con 0x7f1a380ef610 2021-02-10T11:33:22.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.125 7f1a36ffd700 1 -- 172.21.2.101:0/842999937 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2c00c340 con 0x7f1a380ef610 2021-02-10T11:33:22.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.125 7f1a36ffd700 1 -- 172.21.2.101:0/842999937 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1a2c00c820 con 0x7f1a380ef610 2021-02-10T11:33:22.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.125 7f1a3e382700 1 -- 172.21.2.101:0/842999937 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a380ef610 msgr2=0x7f1a380efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:22.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.125 7f1a3e382700 1 --2- 172.21.2.101:0/842999937 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a380ef610 0x7f1a380efa10 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7f1a2c006700 tx=0x7f1a2c009b80).stop 2021-02-10T11:33:22.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.126 7f1a3e382700 1 -- 172.21.2.101:0/842999937 shutdown_connections 2021-02-10T11:33:22.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.126 7f1a3e382700 1 --2- 172.21.2.101:0/842999937 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a380ef610 0x7f1a380efa10 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:22.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.126 7f1a3e382700 1 --2- 172.21.2.101:0/842999937 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a381b0470 0x7f1a381b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:22.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.126 7f1a3e382700 1 -- 172.21.2.101:0/842999937 >> 172.21.2.101:0/842999937 conn(0x7f1a380ce3e0 msgr2=0x7f1a380e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:22.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.126 7f1a3e382700 1 -- 172.21.2.101:0/842999937 shutdown_connections 2021-02-10T11:33:22.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.126 7f1a3e382700 1 -- 172.21.2.101:0/842999937 wait complete. 2021-02-10T11:33:22.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.140 7f1a3e382700 1 Processor -- start 2021-02-10T11:33:22.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.141 7f1a3e382700 1 -- start start 2021-02-10T11:33:22.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.141 7f1a3e382700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a381ac0b0 0x7f1a380ad920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:22.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.141 7f1a3e382700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a380adf40 0x7f1a381b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:22.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.141 7f1a3e382700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a381b7660 0x7f1a381b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:22.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.141 7f1a3e382700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a381b8080 con 0x7f1a381b7660 2021-02-10T11:33:22.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.141 7f1a3e382700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a381b8270 con 0x7f1a380adf40 2021-02-10T11:33:22.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.141 7f1a3e382700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3818d950 con 0x7f1a381ac0b0 2021-02-10T11:33:22.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.141 7f1a3c99f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a381b7660 0x7f1a381b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:22.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.141 7f1a3c99f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a381b7660 0x7f1a381b7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51682/0 (socket says 172.21.2.101:51682) 2021-02-10T11:33:22.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.141 7f1a3c99f700 1 -- 172.21.2.101:0/3212779575 learned_addr learned my addr 172.21.2.101:0/3212779575 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:22.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.142 7f1a3c99f700 1 -- 172.21.2.101:0/3212779575 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a381ac0b0 msgr2=0x7f1a380ad920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:22.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.142 7f1a3c99f700 1 --2- 172.21.2.101:0/3212779575 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a381ac0b0 0x7f1a380ad920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:22.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.142 7f1a3c99f700 1 -- 172.21.2.101:0/3212779575 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a380adf40 msgr2=0x7f1a381b7040 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:22.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.142 7f1a3c99f700 1 --2- 172.21.2.101:0/3212779575 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a380adf40 0x7f1a381b7040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:22.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.142 7f1a3c99f700 1 -- 172.21.2.101:0/3212779575 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3818db70 con 0x7f1a381b7660 2021-02-10T11:33:22.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.142 7f1a3c99f700 1 --2- 172.21.2.101:0/3212779575 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a381b7660 0x7f1a381b7a60 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 rx=0x7f1a30007f90 tx=0x7f1a3000ac50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:22.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.142 7f1a34ff9700 1 -- 172.21.2.101:0/3212779575 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1a30001270 con 0x7f1a381b7660 2021-02-10T11:33:22.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.142 7f1a34ff9700 1 -- 172.21.2.101:0/3212779575 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a30007020 con 0x7f1a381b7660 2021-02-10T11:33:22.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.142 7f1a3e382700 1 -- 172.21.2.101:0/3212779575 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a3818db70 con 0x7f1a381b7660 2021-02-10T11:33:22.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.142 7f1a34ff9700 1 -- 172.21.2.101:0/3212779575 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1a300074b0 con 0x7f1a381b7660 2021-02-10T11:33:22.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.142 7f1a3e382700 1 -- 172.21.2.101:0/3212779575 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3818db70 con 0x7f1a381b7660 2021-02-10T11:33:22.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.143 7f1a34ff9700 1 -- 172.21.2.101:0/3212779575 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1a30007200 con 0x7f1a381b7660 2021-02-10T11:33:22.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.143 7f1a34ff9700 1 --2- 172.21.2.101:0/3212779575 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1a180186d0 0x7f1a1801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:22.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.143 7f1a34ff9700 1 -- 172.21.2.101:0/3212779575 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1a30032b00 con 0x7f1a381b7660 2021-02-10T11:33:22.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.143 7f1a37fff700 1 --2- 172.21.2.101:0/3212779575 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1a180186d0 0x7f1a1801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:22.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.143 7f1a37fff700 1 --2- 172.21.2.101:0/3212779575 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1a180186d0 0x7f1a1801abd0 crc :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:22.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.174 7f1a3e382700 1 -- 172.21.2.101:0/3212779575 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a381059e0 con 0x7f1a381b7660 2021-02-10T11:33:22.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.177 7f1a34ff9700 1 -- 172.21.2.101:0/3212779575 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1a30007d00 con 0x7f1a381b7660 2021-02-10T11:33:22.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.305 7f1a3e382700 1 -- 172.21.2.101:0/3212779575 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f1a381059e0 con 0x7f1a381b7660 2021-02-10T11:33:22.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.310 7f1a34ff9700 1 -- 172.21.2.101:0/3212779575 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v34) v1 ==== 184+0+0 (secure 0 0 0) 0x7f1a30007190 con 0x7f1a381b7660 2021-02-10T11:33:22.323 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.3 2021-02-10T11:33:22.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.314 7f1a3e382700 1 -- 172.21.2.101:0/3212779575 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1a180186d0 msgr2=0x7f1a1801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:22.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.314 7f1a3e382700 1 --2- 172.21.2.101:0/3212779575 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1a180186d0 0x7f1a1801abd0 crc :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:22.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.314 7f1a3e382700 1 -- 172.21.2.101:0/3212779575 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a381b7660 msgr2=0x7f1a381b7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:22.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.314 7f1a3e382700 1 --2- 172.21.2.101:0/3212779575 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a381b7660 0x7f1a381b7a60 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 rx=0x7f1a30007f90 tx=0x7f1a3000ac50).stop 2021-02-10T11:33:22.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.315 7f1a3e382700 1 -- 172.21.2.101:0/3212779575 shutdown_connections 2021-02-10T11:33:22.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.315 7f1a3e382700 1 --2- 172.21.2.101:0/3212779575 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a381b7660 0x7f1a381b7a60 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:22.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.315 7f1a3e382700 1 --2- 172.21.2.101:0/3212779575 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a380adf40 0x7f1a381b7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:22.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.315 7f1a3e382700 1 --2- 172.21.2.101:0/3212779575 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1a180186d0 0x7f1a1801abd0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:22.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.315 7f1a3e382700 1 --2- 172.21.2.101:0/3212779575 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a381ac0b0 0x7f1a380ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:22.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.315 7f1a3e382700 1 -- 172.21.2.101:0/3212779575 >> 172.21.2.101:0/3212779575 conn(0x7f1a381a37a0 msgr2=0x7f1a3818c230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:22.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.315 7f1a3e382700 1 -- 172.21.2.101:0/3212779575 shutdown_connections 2021-02-10T11:33:22.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.315 7f1a3e382700 1 -- 172.21.2.101:0/3212779575 wait complete. 2021-02-10T11:33:22.335 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:33:22.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.733 7f0f13d92700 1 Processor -- start 2021-02-10T11:33:22.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.733 7f0f13d92700 1 -- start start 2021-02-10T11:33:22.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.734 7f0f13d92700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0f0c0e5070 0x7f0f0c1b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:22.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.734 7f0f13d92700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0f0c0e5470 0x7f0f0c080310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:22.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.734 7f0f13d92700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f0c07ed80 con 0x7f0f0c1b3e40 2021-02-10T11:33:22.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.734 7f0f13d92700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f0c0d1790 con 0x7f0f0c0e5470 2021-02-10T11:33:22.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.734 7f0f13d92700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f0c0d19b0 con 0x7f0f0c0e5070 2021-02-10T11:33:22.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.734 7f0f123af700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0f0c0e5470 0x7f0f0c080310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:22.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.734 7f0f123af700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0f0c0e5470 0x7f0f0c080310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51686/0 (socket says 172.21.2.101:51686) 2021-02-10T11:33:22.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.734 7f0f123af700 1 -- 172.21.2.101:0/36051930 learned_addr learned my addr 172.21.2.101:0/36051930 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:22.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.734 7f0f123af700 1 -- 172.21.2.101:0/36051930 >> v2:172.21.2.108:3300/0 conn(0x7f0f0c0e5070 msgr2=0x7f0f0c1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:22.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.734 7f0f123af700 1 --2- 172.21.2.101:0/36051930 >> v2:172.21.2.108:3300/0 conn(0x7f0f0c0e5070 0x7f0f0c1b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:22.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.734 7f0f123af700 1 -- 172.21.2.101:0/36051930 >> v1:172.21.2.108:6789/0 conn(0x7f0f0c1b3e40 legacy=0x7f0f0c07d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:22.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.734 7f0f123af700 1 -- 172.21.2.101:0/36051930 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c0d1bd0 con 0x7f0f0c0e5470 2021-02-10T11:33:22.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.735 7f0f123af700 1 --2- 172.21.2.101:0/36051930 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f0c0e5470 0x7f0f0c080310 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 rx=0x7f0f04006f00 tx=0x7f0f04009f10).ready entity=mon.0 client_cookie=cf54fe24eca1cadb server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:22.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.735 7f0f10bac700 1 -- 172.21.2.101:0/36051930 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0f0400b3b0 con 0x7f0f0c0e5470 2021-02-10T11:33:22.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.735 7f0f10bac700 1 -- 172.21.2.101:0/36051930 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0400b630 con 0x7f0f0c0e5470 2021-02-10T11:33:22.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.735 7f0f10bac700 1 -- 172.21.2.101:0/36051930 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0f0400bb50 con 0x7f0f0c0e5470 2021-02-10T11:33:22.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.735 7f0f13d92700 1 -- 172.21.2.101:0/36051930 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f0c0e5470 msgr2=0x7f0f0c080310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:22.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.735 7f0f13d92700 1 --2- 172.21.2.101:0/36051930 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f0c0e5470 0x7f0f0c080310 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 rx=0x7f0f04006f00 tx=0x7f0f04009f10).stop 2021-02-10T11:33:22.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.735 7f0f13d92700 1 -- 172.21.2.101:0/36051930 shutdown_connections 2021-02-10T11:33:22.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.735 7f0f13d92700 1 --2- 172.21.2.101:0/36051930 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f0c0e5470 0x7f0f0c080310 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:22.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.735 7f0f13d92700 1 --2- 172.21.2.101:0/36051930 >> v2:172.21.2.108:3300/0 conn(0x7f0f0c0e5070 0x7f0f0c1b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:22.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.735 7f0f13d92700 1 -- 172.21.2.101:0/36051930 >> 172.21.2.101:0/36051930 conn(0x7f0f0c0ce690 msgr2=0x7f0f0c0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:22.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.736 7f0f13d92700 1 -- 172.21.2.101:0/36051930 shutdown_connections 2021-02-10T11:33:22.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.736 7f0f13d92700 1 -- 172.21.2.101:0/36051930 wait complete. 2021-02-10T11:33:22.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.751 7f0f13d92700 1 Processor -- start 2021-02-10T11:33:22.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.751 7f0f13d92700 1 -- start start 2021-02-10T11:33:22.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.751 7f0f13d92700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f0c1b19d0 0x7f0f0c08cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:22.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.751 7f0f13d92700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f0c1b0d00 0x7f0f0c1b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:22.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.751 7f0f13d92700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f0c1a2b50 0x7f0f0c1a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:22.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.751 7f0f13d92700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c1b1720 con 0x7f0f0c1a2b50 2021-02-10T11:33:22.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.751 7f0f13d92700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c08c470 con 0x7f0f0c1b19d0 2021-02-10T11:33:22.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.751 7f0f13d92700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c1b1f50 con 0x7f0f0c1b0d00 2021-02-10T11:33:22.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.751 7f0f123af700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f0c1a2b50 0x7f0f0c1a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:22.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.751 7f0f123af700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f0c1a2b50 0x7f0f0c1a2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51690/0 (socket says 172.21.2.101:51690) 2021-02-10T11:33:22.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.751 7f0f123af700 1 -- 172.21.2.101:0/3446557225 learned_addr learned my addr 172.21.2.101:0/3446557225 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:22.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.751 7f0f11bae700 1 --2- 172.21.2.101:0/3446557225 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f0c1b19d0 0x7f0f0c08cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:22.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.751 7f0f113ad700 1 --2- 172.21.2.101:0/3446557225 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f0c1b0d00 0x7f0f0c1b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:22.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.752 7f0f123af700 1 -- 172.21.2.101:0/3446557225 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f0c1b0d00 msgr2=0x7f0f0c1b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:22.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.752 7f0f123af700 1 --2- 172.21.2.101:0/3446557225 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f0c1b0d00 0x7f0f0c1b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:22.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.752 7f0f123af700 1 -- 172.21.2.101:0/3446557225 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f0c1b19d0 msgr2=0x7f0f0c08cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:22.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.752 7f0f123af700 1 --2- 172.21.2.101:0/3446557225 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f0c1b19d0 0x7f0f0c08cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:22.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.752 7f0f123af700 1 -- 172.21.2.101:0/3446557225 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c1a3570 con 0x7f0f0c1a2b50 2021-02-10T11:33:22.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.752 7f0f11bae700 1 --2- 172.21.2.101:0/3446557225 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f0c1b19d0 0x7f0f0c08cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:33:22.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.752 7f0f123af700 1 --2- 172.21.2.101:0/3446557225 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f0c1a2b50 0x7f0f0c1a2f50 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7f0f0400b930 tx=0x7f0f0400bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:22.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.752 7f0efe7fc700 1 -- 172.21.2.101:0/3446557225 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0f04009450 con 0x7f0f0c1a2b50 2021-02-10T11:33:22.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.752 7f0efe7fc700 1 -- 172.21.2.101:0/3446557225 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f040068e0 con 0x7f0f0c1a2b50 2021-02-10T11:33:22.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.752 7f0efe7fc700 1 -- 172.21.2.101:0/3446557225 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0f0400c380 con 0x7f0f0c1a2b50 2021-02-10T11:33:22.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.752 7f0f13d92700 1 -- 172.21.2.101:0/3446557225 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f0c1a3570 con 0x7f0f0c1a2b50 2021-02-10T11:33:22.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.753 7f0f13d92700 1 -- 172.21.2.101:0/3446557225 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0c1a3570 con 0x7f0f0c1a2b50 2021-02-10T11:33:22.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.753 7f0efe7fc700 1 -- 172.21.2.101:0/3446557225 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0f04001270 con 0x7f0f0c1a2b50 2021-02-10T11:33:22.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.753 7f0efe7fc700 1 --2- 172.21.2.101:0/3446557225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0ef40186c0 0x7f0ef401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:22.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.753 7f0efe7fc700 1 -- 172.21.2.101:0/3446557225 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0f04032900 con 0x7f0f0c1a2b50 2021-02-10T11:33:22.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.753 7f0f11bae700 1 --2- 172.21.2.101:0/3446557225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0ef40186c0 0x7f0ef401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:22.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.754 7f0f11bae700 1 --2- 172.21.2.101:0/3446557225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0ef40186c0 0x7f0ef401abc0 crc :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:22.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.784 7f0f13d92700 1 -- 172.21.2.101:0/3446557225 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f0c083dd0 con 0x7f0f0c1a2b50 2021-02-10T11:33:22.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.786 7f0efe7fc700 1 -- 172.21.2.101:0/3446557225 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0f04009450 con 0x7f0f0c1a2b50 2021-02-10T11:33:22.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.914 7f0f13d92700 1 -- 172.21.2.101:0/3446557225 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f0c07ed80 con 0x7f0f0c1a2b50 2021-02-10T11:33:22.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.915 7f0efe7fc700 1 -- 172.21.2.101:0/3446557225 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+2098 (secure 0 0 0) 0x7f0f04001270 con 0x7f0f0c1a2b50 2021-02-10T11:33:22.926 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 38 2021-02-10T11:33:22.926 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:22.927 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8049,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/172865610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":172865610},{"type":"v1","addr":"172.21.2.101:6835","nonce":172865610}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:33:16.590282","modified":"2021-02-10 11:33:18.594009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5579},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5579":{"gid":5579,"name":"b","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2400978339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2400978339},{"type":"v1","addr":"172.21.2.108:6833","nonce":2400978339}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-02-10T11:33:22.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.917 7f0f13d92700 1 -- 172.21.2.101:0/3446557225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0ef40186c0 msgr2=0x7f0ef401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:22.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.917 7f0f13d92700 1 --2- 172.21.2.101:0/3446557225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0ef40186c0 0x7f0ef401abc0 crc :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:22.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.917 7f0f13d92700 1 -- 172.21.2.101:0/3446557225 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f0c1a2b50 msgr2=0x7f0f0c1a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:22.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.917 7f0f13d92700 1 --2- 172.21.2.101:0/3446557225 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f0c1a2b50 0x7f0f0c1a2f50 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7f0f0400b930 tx=0x7f0f0400bcd0).stop 2021-02-10T11:33:22.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.917 7f0f13d92700 1 -- 172.21.2.101:0/3446557225 shutdown_connections 2021-02-10T11:33:22.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.917 7f0f13d92700 1 --2- 172.21.2.101:0/3446557225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0ef40186c0 0x7f0ef401abc0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:22.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.917 7f0f13d92700 1 --2- 172.21.2.101:0/3446557225 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f0c1b0d00 0x7f0f0c1b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:22.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.917 7f0f13d92700 1 --2- 172.21.2.101:0/3446557225 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f0c1a2b50 0x7f0f0c1a2f50 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:22.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.917 7f0f13d92700 1 --2- 172.21.2.101:0/3446557225 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f0c1b19d0 0x7f0f0c08cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:22.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.917 7f0f13d92700 1 -- 172.21.2.101:0/3446557225 >> 172.21.2.101:0/3446557225 conn(0x7f0f0c093720 msgr2=0x7f0f0c084b80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:22.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.917 7f0f13d92700 1 -- 172.21.2.101:0/3446557225 shutdown_connections 2021-02-10T11:33:22.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:22.917 7f0f13d92700 1 -- 172.21.2.101:0/3446557225 wait complete. 2021-02-10T11:33:23.139 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:33:16.590282', 'modified': '2021-02-10 11:33:18.594009', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5579}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5579': {'gid': 5579, 'name': 'b', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/2400978339', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 2400978339}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 2400978339}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:33:23.139 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:33:23.139 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:33:23.538 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:33:23.539 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:33:23.539 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:33:23.539 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 5579, 2021-02-10T11:33:23.539 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:33:23.539 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:33:23.539 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 6.9358581360000002, 2021-02-10T11:33:23.539 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 38, 2021-02-10T11:33:23.539 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 55, 2021-02-10T11:33:23.539 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 55, 2021-02-10T11:33:23.539 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 12.568748653 2021-02-10T11:33:23.540 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:33:23.543 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 5579, "want_state": "up:active", "state": "up:active", "rank_uptime": 6.9358581360000002, "mdsmap_epoch": 38, "osdmap_epoch": 55, "osdmap_epoch_barrier": 55, "uptime": 12.568748653 } 2021-02-10T11:33:23.543 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:33:23.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.942 7fa005af5700 1 Processor -- start 2021-02-10T11:33:23.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.942 7fa005af5700 1 -- start start 2021-02-10T11:33:23.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.943 7fa005af5700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa0000e5470 0x7fa00007e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:23.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.943 7fa005af5700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa00007f300 0x7fa00007f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:23.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.943 7fa005af5700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa00007ed80 con 0x7fa0000e5070 2021-02-10T11:33:23.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.943 7fa005af5700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0001b4010 con 0x7fa0000e5470 2021-02-10T11:33:23.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.943 7fa005af5700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0000d1790 con 0x7fa00007f300 2021-02-10T11:33:23.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.943 7f9ffeffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa0000e5470 0x7fa00007e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:23.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.943 7f9ffeffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa0000e5470 0x7fa00007e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51698/0 (socket says 172.21.2.101:51698) 2021-02-10T11:33:23.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.943 7f9ffeffd700 1 -- 172.21.2.101:0/2035920969 learned_addr learned my addr 172.21.2.101:0/2035920969 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:23.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.943 7f9ffffff700 1 --2- 172.21.2.101:0/2035920969 >> v2:172.21.2.108:3300/0 conn(0x7fa00007f300 0x7fa00007f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:23.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.943 7f9ffeffd700 1 -- 172.21.2.101:0/2035920969 >> v2:172.21.2.108:3300/0 conn(0x7fa00007f300 msgr2=0x7fa00007f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:23.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.943 7f9ffeffd700 1 --2- 172.21.2.101:0/2035920969 >> v2:172.21.2.108:3300/0 conn(0x7fa00007f300 0x7fa00007f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:23.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.943 7f9ffeffd700 1 -- 172.21.2.101:0/2035920969 >> v1:172.21.2.108:6789/0 conn(0x7fa0000e5070 legacy=0x7fa0001b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:23.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.943 7f9ffeffd700 1 -- 172.21.2.101:0/2035920969 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0000d19b0 con 0x7fa0000e5470 2021-02-10T11:33:23.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.943 7f9ffffff700 1 --2- 172.21.2.101:0/2035920969 >> v2:172.21.2.108:3300/0 conn(0x7fa00007f300 0x7fa00007f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:33:23.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.943 7f9ffeffd700 1 --2- 172.21.2.101:0/2035920969 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0000e5470 0x7fa00007e6a0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7f9ff0006700 tx=0x7f9ff0009b80).ready entity=mon.0 client_cookie=f125294538e0066f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:23.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.943 7f9ffe7fc700 1 -- 172.21.2.101:0/2035920969 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9ff000c0c0 con 0x7fa0000e5470 2021-02-10T11:33:23.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.943 7f9ffe7fc700 1 -- 172.21.2.101:0/2035920969 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff000c340 con 0x7fa0000e5470 2021-02-10T11:33:23.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.943 7f9ffe7fc700 1 -- 172.21.2.101:0/2035920969 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9ff000c860 con 0x7fa0000e5470 2021-02-10T11:33:23.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.943 7fa005af5700 1 -- 172.21.2.101:0/2035920969 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0000e5470 msgr2=0x7fa00007e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:23.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.943 7fa005af5700 1 --2- 172.21.2.101:0/2035920969 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0000e5470 0x7fa00007e6a0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7f9ff0006700 tx=0x7f9ff0009b80).stop 2021-02-10T11:33:23.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.944 7fa005af5700 1 -- 172.21.2.101:0/2035920969 shutdown_connections 2021-02-10T11:33:23.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.944 7fa005af5700 1 --2- 172.21.2.101:0/2035920969 >> v2:172.21.2.108:3300/0 conn(0x7fa00007f300 0x7fa00007f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:23.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.944 7fa005af5700 1 --2- 172.21.2.101:0/2035920969 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0000e5470 0x7fa00007e6a0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:23.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.944 7fa005af5700 1 -- 172.21.2.101:0/2035920969 >> 172.21.2.101:0/2035920969 conn(0x7fa0000ce690 msgr2=0x7fa0000e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:23.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.944 7fa005af5700 1 -- 172.21.2.101:0/2035920969 shutdown_connections 2021-02-10T11:33:23.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.944 7fa005af5700 1 -- 172.21.2.101:0/2035920969 wait complete. 2021-02-10T11:33:23.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.959 7fa005af5700 1 Processor -- start 2021-02-10T11:33:23.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.959 7fa005af5700 1 -- start start 2021-02-10T11:33:23.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.959 7fa005af5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa00018d870 0x7fa000107640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:23.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.959 7fa005af5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa00018c6e0 0x7fa00018cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:23.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.959 7fa005af5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa00018dec0 0x7fa00018e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:23.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.959 7fa005af5700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00018ee70 con 0x7fa00018dec0 2021-02-10T11:33:23.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.959 7fa005af5700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00018d2f0 con 0x7fa00018d870 2021-02-10T11:33:23.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.959 7fa005af5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa000108160 con 0x7fa00018c6e0 2021-02-10T11:33:23.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.959 7f9ffffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa00018dec0 0x7fa00018e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:23.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.959 7f9ffffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa00018dec0 0x7fa00018e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51708/0 (socket says 172.21.2.101:51708) 2021-02-10T11:33:23.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.959 7f9ffffff700 1 -- 172.21.2.101:0/3077796570 learned_addr learned my addr 172.21.2.101:0/3077796570 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:23.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.959 7f9ffeffd700 1 --2- 172.21.2.101:0/3077796570 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa00018c6e0 0x7fa00018cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:23.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.960 7f9fff7fe700 1 --2- 172.21.2.101:0/3077796570 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa00018d870 0x7fa000107640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:23.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.960 7f9ffffff700 1 -- 172.21.2.101:0/3077796570 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa00018c6e0 msgr2=0x7fa00018cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:23.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.960 7f9ffffff700 1 --2- 172.21.2.101:0/3077796570 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa00018c6e0 0x7fa00018cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:23.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.960 7f9ffffff700 1 -- 172.21.2.101:0/3077796570 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa00018d870 msgr2=0x7fa000107640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:23.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.960 7f9ffffff700 1 --2- 172.21.2.101:0/3077796570 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa00018d870 0x7fa000107640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:23.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.960 7f9ffffff700 1 -- 172.21.2.101:0/3077796570 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa000107c60 con 0x7fa00018dec0 2021-02-10T11:33:23.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.960 7f9ffffff700 1 --2- 172.21.2.101:0/3077796570 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa00018dec0 0x7fa00018e2c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 rx=0x7f9ff8009980 tx=0x7f9ff800bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:23.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.960 7f9ff7fff700 1 -- 172.21.2.101:0/3077796570 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9ff800c2f0 con 0x7fa00018dec0 2021-02-10T11:33:23.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.960 7f9ff7fff700 1 -- 172.21.2.101:0/3077796570 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff8006790 con 0x7fa00018dec0 2021-02-10T11:33:23.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.960 7fa005af5700 1 -- 172.21.2.101:0/3077796570 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa000107c60 con 0x7fa00018dec0 2021-02-10T11:33:23.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.960 7f9ff7fff700 1 -- 172.21.2.101:0/3077796570 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9ff800c7f0 con 0x7fa00018dec0 2021-02-10T11:33:23.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.960 7fa005af5700 1 -- 172.21.2.101:0/3077796570 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa000107c60 con 0x7fa00018dec0 2021-02-10T11:33:23.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.961 7f9ff7fff700 1 -- 172.21.2.101:0/3077796570 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9ff80011f0 con 0x7fa00018dec0 2021-02-10T11:33:23.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.961 7f9ff7fff700 1 --2- 172.21.2.101:0/3077796570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9fe4018730 0x7f9fe401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:23.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.961 7f9ff7fff700 1 -- 172.21.2.101:0/3077796570 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9ff8032960 con 0x7fa00018dec0 2021-02-10T11:33:23.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.961 7f9fff7fe700 1 --2- 172.21.2.101:0/3077796570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9fe4018730 0x7f9fe401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:23.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.961 7f9fff7fe700 1 --2- 172.21.2.101:0/3077796570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9fe4018730 0x7f9fe401abc0 crc :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:24.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.992 7fa005af5700 1 -- 172.21.2.101:0/3077796570 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa000106cc0 con 0x7fa00018dec0 2021-02-10T11:33:24.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:23.995 7f9ff7fff700 1 -- 172.21.2.101:0/3077796570 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9ff800be10 con 0x7fa00018dec0 2021-02-10T11:33:24.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.119 7fa005af5700 1 -- 172.21.2.101:0/3077796570 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa000106cc0 con 0x7fa00018dec0 2021-02-10T11:33:24.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.119 7f9ff7fff700 1 -- 172.21.2.101:0/3077796570 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+13996 (secure 0 0 0) 0x7f9ff80011f0 con 0x7fa00018dec0 2021-02-10T11:33:24.131 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:24.131 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":55,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:33:17.591716","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:33:14.515763","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2021-02-10 11:33:15.636274","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:33:24.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.121 7fa005af5700 1 -- 172.21.2.101:0/3077796570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9fe4018730 msgr2=0x7f9fe401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:24.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.121 7fa005af5700 1 --2- 172.21.2.101:0/3077796570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9fe4018730 0x7f9fe401abc0 crc :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:24.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.121 7fa005af5700 1 -- 172.21.2.101:0/3077796570 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa00018dec0 msgr2=0x7fa00018e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:24.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.121 7fa005af5700 1 --2- 172.21.2.101:0/3077796570 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa00018dec0 0x7fa00018e2c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 rx=0x7f9ff8009980 tx=0x7f9ff800bcd0).stop 2021-02-10T11:33:24.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.121 7fa005af5700 1 -- 172.21.2.101:0/3077796570 shutdown_connections 2021-02-10T11:33:24.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.121 7fa005af5700 1 --2- 172.21.2.101:0/3077796570 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9fe4018730 0x7f9fe401abc0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:24.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.121 7fa005af5700 1 --2- 172.21.2.101:0/3077796570 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa00018c6e0 0x7fa00018cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:24.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.121 7fa005af5700 1 --2- 172.21.2.101:0/3077796570 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa00018dec0 0x7fa00018e2c0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:24.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.121 7fa005af5700 1 --2- 172.21.2.101:0/3077796570 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa00018d870 0x7fa000107640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:24.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.121 7fa005af5700 1 -- 172.21.2.101:0/3077796570 >> 172.21.2.101:0/3077796570 conn(0x7fa0001aa5e0 msgr2=0x7fa00019f190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:24.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.122 7fa005af5700 1 -- 172.21.2.101:0/3077796570 shutdown_connections 2021-02-10T11:33:24.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.122 7fa005af5700 1 -- 172.21.2.101:0/3077796570 wait complete. 2021-02-10T11:33:24.141 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:33:14.515763', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:33:15.636274', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:33:24.142 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:33:24.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.544 7f3f9d6fe700 1 Processor -- start 2021-02-10T11:33:24.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.544 7f3f9d6fe700 1 -- start start 2021-02-10T11:33:24.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.544 7f3f9d6fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3f981b0470 0x7f3f981b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:24.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.544 7f3f9d6fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3f980ef610 0x7f3f980efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:24.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.544 7f3f9d6fe700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f980f0b50 con 0x7f3f980f0030 2021-02-10T11:33:24.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.544 7f3f9d6fe700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f980e52e0 con 0x7f3f980ef610 2021-02-10T11:33:24.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.544 7f3f9d6fe700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f980e5490 con 0x7f3f981b0470 2021-02-10T11:33:24.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.544 7f3f97ac0700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f3f980f0030 0x7f3f980f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52206/0 (socket says 172.21.2.101:52206) 2021-02-10T11:33:24.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.544 7f3f97ac0700 1 -- 172.21.2.101:0/2055530181 learned_addr learned my addr 172.21.2.101:0/2055530181 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:24.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.544 7f3f96abe700 1 --2- 172.21.2.101:0/2055530181 >> v2:172.21.2.101:3300/0 conn(0x7f3f980ef610 0x7f3f980efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:24.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.545 7f3f962bd700 1 -- 172.21.2.101:0/2055530181 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 441203692 0 0) 0x7f3f90000d30 con 0x7f3f980f0030 2021-02-10T11:33:24.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.545 7f3f962bd700 1 -- 172.21.2.101:0/2055530181 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f84002fe0 con 0x7f3f980f0030 2021-02-10T11:33:24.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.545 7f3f96abe700 1 -- 172.21.2.101:0/2055530181 >> v2:172.21.2.108:3300/0 conn(0x7f3f981b0470 msgr2=0x7f3f981b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:33:24.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.545 7f3f96abe700 1 --2- 172.21.2.101:0/2055530181 >> v2:172.21.2.108:3300/0 conn(0x7f3f981b0470 0x7f3f981b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:24.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.545 7f3f96abe700 1 -- 172.21.2.101:0/2055530181 >> v1:172.21.2.101:6789/0 conn(0x7f3f980f0030 legacy=0x7f3f980f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:24.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.545 7f3f96abe700 1 -- 172.21.2.101:0/2055530181 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f980f0430 con 0x7f3f980ef610 2021-02-10T11:33:24.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.545 7f3f96abe700 1 --2- 172.21.2.101:0/2055530181 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f980ef610 0x7f3f980efa10 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7f3f80006650 tx=0x7f3f80009a00).ready entity=mon.0 client_cookie=fb93815333d4f0d4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:24.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.545 7f3f962bd700 1 -- 172.21.2.101:0/2055530181 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3f80009df0 con 0x7f3f980ef610 2021-02-10T11:33:24.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.545 7f3f962bd700 1 -- 172.21.2.101:0/2055530181 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f800009b0 con 0x7f3f980ef610 2021-02-10T11:33:24.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.545 7f3f962bd700 1 -- 172.21.2.101:0/2055530181 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3f8000c2b0 con 0x7f3f980ef610 2021-02-10T11:33:24.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.545 7f3f9d6fe700 1 -- 172.21.2.101:0/2055530181 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f980ef610 msgr2=0x7f3f980efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:24.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.545 7f3f9d6fe700 1 --2- 172.21.2.101:0/2055530181 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f980ef610 0x7f3f980efa10 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7f3f80006650 tx=0x7f3f80009a00).stop 2021-02-10T11:33:24.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.546 7f3f9d6fe700 1 -- 172.21.2.101:0/2055530181 shutdown_connections 2021-02-10T11:33:24.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.546 7f3f9d6fe700 1 --2- 172.21.2.101:0/2055530181 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f980ef610 0x7f3f980efa10 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:24.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.546 7f3f9d6fe700 1 --2- 172.21.2.101:0/2055530181 >> v2:172.21.2.108:3300/0 conn(0x7f3f981b0470 0x7f3f981b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:24.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.546 7f3f9d6fe700 1 -- 172.21.2.101:0/2055530181 >> 172.21.2.101:0/2055530181 conn(0x7f3f980ce3e0 msgr2=0x7f3f980e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:24.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.546 7f3f9d6fe700 1 -- 172.21.2.101:0/2055530181 shutdown_connections 2021-02-10T11:33:24.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.546 7f3f9d6fe700 1 -- 172.21.2.101:0/2055530181 wait complete. 2021-02-10T11:33:24.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.560 7f3f9d6fe700 1 Processor -- start 2021-02-10T11:33:24.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.560 7f3f9d6fe700 1 -- start start 2021-02-10T11:33:24.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.560 7f3f9d6fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f98190920 0x7f3f980b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:24.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.560 7f3f9d6fe700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3f980b8200 0x7f3f980b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:24.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.560 7f3f9d6fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f980c1d80 0x7f3f980c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:24.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.560 7f3f9d6fe700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f98190d20 con 0x7f3f980c1d80 2021-02-10T11:33:24.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.560 7f3f9d6fe700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f980b8c20 con 0x7f3f980b8200 2021-02-10T11:33:24.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.560 7f3f9d6fe700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f980c27a0 con 0x7f3f98190920 2021-02-10T11:33:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.561 7f3f97ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f980c1d80 0x7f3f980c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.561 7f3f97ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f980c1d80 0x7f3f980c2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51720/0 (socket says 172.21.2.101:51720) 2021-02-10T11:33:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.561 7f3f97ac0700 1 -- 172.21.2.101:0/1769126190 learned_addr learned my addr 172.21.2.101:0/1769126190 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.561 7f3f97ac0700 1 -- 172.21.2.101:0/1769126190 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f98190920 msgr2=0x7f3f980b7be0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.561 7f3f97ac0700 1 --2- 172.21.2.101:0/1769126190 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f98190920 0x7f3f980b7be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.561 7f3f97ac0700 1 -- 172.21.2.101:0/1769126190 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3f980b8200 msgr2=0x7f3f980b8600 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.561 7f3f97ac0700 1 --2- 172.21.2.101:0/1769126190 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3f980b8200 0x7f3f980b8600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.561 7f3f97ac0700 1 -- 172.21.2.101:0/1769126190 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f980c29c0 con 0x7f3f980c1d80 2021-02-10T11:33:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.561 7f3f97ac0700 1 --2- 172.21.2.101:0/1769126190 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f980c1d80 0x7f3f980c2180 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7f3f90009980 tx=0x7f3f9000bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.561 7f3f8ffff700 1 -- 172.21.2.101:0/1769126190 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3f9000be10 con 0x7f3f980c1d80 2021-02-10T11:33:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.561 7f3f8ffff700 1 -- 172.21.2.101:0/1769126190 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f90006030 con 0x7f3f980c1d80 2021-02-10T11:33:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.562 7f3f9d6fe700 1 -- 172.21.2.101:0/1769126190 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f980c29c0 con 0x7f3f980c1d80 2021-02-10T11:33:24.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.562 7f3f9d6fe700 1 -- 172.21.2.101:0/1769126190 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f980c2f50 con 0x7f3f980c1d80 2021-02-10T11:33:24.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.562 7f3f8ffff700 1 -- 172.21.2.101:0/1769126190 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3f90004720 con 0x7f3f980c1d80 2021-02-10T11:33:24.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.562 7f3f8ffff700 1 -- 172.21.2.101:0/1769126190 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3f90006210 con 0x7f3f980c1d80 2021-02-10T11:33:24.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.562 7f3f8ffff700 1 --2- 172.21.2.101:0/1769126190 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f7c0186d0 0x7f3f7c01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:24.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.562 7f3f8ffff700 1 -- 172.21.2.101:0/1769126190 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3f90032b70 con 0x7f3f980c1d80 2021-02-10T11:33:24.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.562 7f3f972bf700 1 --2- 172.21.2.101:0/1769126190 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f7c0186d0 0x7f3f7c01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:24.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.562 7f3f972bf700 1 --2- 172.21.2.101:0/1769126190 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f7c0186d0 0x7f3f7c01abd0 crc :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:24.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.594 7f3f9d6fe700 1 -- 172.21.2.101:0/1769126190 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f98105ae0 con 0x7f3f980c1d80 2021-02-10T11:33:24.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.597 7f3f8ffff700 1 -- 172.21.2.101:0/1769126190 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3f90000d30 con 0x7f3f980c1d80 2021-02-10T11:33:24.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.725 7f3f9d6fe700 1 -- 172.21.2.101:0/1769126190 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f3f980f0b50 con 0x7f3f980c1d80 2021-02-10T11:33:24.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.725 7f3f8ffff700 1 -- 172.21.2.101:0/1769126190 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v55) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3f90006030 con 0x7f3f980c1d80 2021-02-10T11:33:24.737 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:33:24.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.727 7f3f9d6fe700 1 -- 172.21.2.101:0/1769126190 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f7c0186d0 msgr2=0x7f3f7c01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:24.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.727 7f3f9d6fe700 1 --2- 172.21.2.101:0/1769126190 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f7c0186d0 0x7f3f7c01abd0 crc :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:24.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.727 7f3f9d6fe700 1 -- 172.21.2.101:0/1769126190 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f980c1d80 msgr2=0x7f3f980c2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:24.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.727 7f3f9d6fe700 1 --2- 172.21.2.101:0/1769126190 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f980c1d80 0x7f3f980c2180 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7f3f90009980 tx=0x7f3f9000bcd0).stop 2021-02-10T11:33:24.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.727 7f3f9d6fe700 1 -- 172.21.2.101:0/1769126190 shutdown_connections 2021-02-10T11:33:24.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.727 7f3f9d6fe700 1 --2- 172.21.2.101:0/1769126190 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f980c1d80 0x7f3f980c2180 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:24.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.727 7f3f9d6fe700 1 --2- 172.21.2.101:0/1769126190 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3f980b8200 0x7f3f980b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:24.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.727 7f3f9d6fe700 1 --2- 172.21.2.101:0/1769126190 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f7c0186d0 0x7f3f7c01abd0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:24.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.727 7f3f9d6fe700 1 --2- 172.21.2.101:0/1769126190 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f98190920 0x7f3f980b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:24.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.727 7f3f9d6fe700 1 -- 172.21.2.101:0/1769126190 >> 172.21.2.101:0/1769126190 conn(0x7f3f98094e10 msgr2=0x7f3f980a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:24.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.728 7f3f9d6fe700 1 -- 172.21.2.101:0/1769126190 shutdown_connections 2021-02-10T11:33:24.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:24.728 7f3f9d6fe700 1 -- 172.21.2.101:0/1769126190 wait complete. 2021-02-10T11:33:24.747 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:33:25.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.150 7fbae9534700 1 Processor -- start 2021-02-10T11:33:25.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.150 7fbae9534700 1 -- start start 2021-02-10T11:33:25.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.150 7fbae9534700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbae41b0470 0x7fbae41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:25.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.150 7fbae9534700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbae40ef610 0x7fbae40efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:25.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.151 7fbae9534700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbae40f0b50 con 0x7fbae40f0030 2021-02-10T11:33:25.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.151 7fbae9534700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae40e52e0 con 0x7fbae40ef610 2021-02-10T11:33:25.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.151 7fbae9534700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae40e5490 con 0x7fbae41b0470 2021-02-10T11:33:25.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.151 7fbae32bf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbae41b0470 0x7fbae41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:25.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.151 7fbae32bf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbae41b0470 0x7fbae41b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:44456/0 (socket says 172.21.2.101:44456) 2021-02-10T11:33:25.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.151 7fbae32bf700 1 -- 172.21.2.101:0/2966063140 learned_addr learned my addr 172.21.2.101:0/2966063140 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:25.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.151 7fbae2abe700 1 --2- 172.21.2.101:0/2966063140 >> v2:172.21.2.108:3300/0 conn(0x7fbae40ef610 0x7fbae40efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:25.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.152 7fbae32bf700 1 -- 172.21.2.101:0/2966063140 >> v2:172.21.2.108:3300/0 conn(0x7fbae40ef610 msgr2=0x7fbae40efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:25.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.152 7fbae32bf700 1 --2- 172.21.2.101:0/2966063140 >> v2:172.21.2.108:3300/0 conn(0x7fbae40ef610 0x7fbae40efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:25.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.152 7fbae32bf700 1 -- 172.21.2.101:0/2966063140 >> v1:172.21.2.108:6789/0 conn(0x7fbae40f0030 legacy=0x7fbae40f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:25.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.152 7fbae32bf700 1 -- 172.21.2.101:0/2966063140 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae40f0430 con 0x7fbae41b0470 2021-02-10T11:33:25.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.152 7fbae32bf700 1 --2- 172.21.2.101:0/2966063140 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbae41b0470 0x7fbae41b0870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fbacc006a80 tx=0x7fbacc009f00).ready entity=mon.2 client_cookie=73b11d309324006f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:25.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.152 7fbae22bd700 1 -- 172.21.2.101:0/2966063140 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbacc00b5c0 con 0x7fbae41b0470 2021-02-10T11:33:25.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.152 7fbae22bd700 1 -- 172.21.2.101:0/2966063140 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbacc00b8c0 con 0x7fbae41b0470 2021-02-10T11:33:25.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.152 7fbae22bd700 1 -- 172.21.2.101:0/2966063140 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbacc00bda0 con 0x7fbae41b0470 2021-02-10T11:33:25.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.153 7fbae9534700 1 -- 172.21.2.101:0/2966063140 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbae41b0470 msgr2=0x7fbae41b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:25.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.153 7fbae9534700 1 --2- 172.21.2.101:0/2966063140 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbae41b0470 0x7fbae41b0870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fbacc006a80 tx=0x7fbacc009f00).stop 2021-02-10T11:33:25.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.153 7fbae9534700 1 -- 172.21.2.101:0/2966063140 shutdown_connections 2021-02-10T11:33:25.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.153 7fbae9534700 1 --2- 172.21.2.101:0/2966063140 >> v2:172.21.2.108:3300/0 conn(0x7fbae40ef610 0x7fbae40efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:25.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.153 7fbae9534700 1 --2- 172.21.2.101:0/2966063140 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbae41b0470 0x7fbae41b0870 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:25.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.153 7fbae9534700 1 -- 172.21.2.101:0/2966063140 >> 172.21.2.101:0/2966063140 conn(0x7fbae40ce3e0 msgr2=0x7fbae40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:25.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.153 7fbae9534700 1 -- 172.21.2.101:0/2966063140 shutdown_connections 2021-02-10T11:33:25.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.153 7fbae9534700 1 -- 172.21.2.101:0/2966063140 wait complete. 2021-02-10T11:33:25.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.168 7fbae9534700 1 Processor -- start 2021-02-10T11:33:25.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.168 7fbae9534700 1 -- start start 2021-02-10T11:33:25.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.168 7fbae9534700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbae408ece0 0x7fbae40a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:25.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.168 7fbae9534700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbae40a6860 0x7fbae40a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:25.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.168 7fbae9534700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbae40a7280 0x7fbae40a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:25.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.168 7fbae9534700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae40a5cc0 con 0x7fbae408ece0 2021-02-10T11:33:25.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.168 7fbae9534700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae408f0e0 con 0x7fbae40a6860 2021-02-10T11:33:25.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.168 7fbae9534700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae408f2c0 con 0x7fbae40a7280 2021-02-10T11:33:25.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.168 7fbae32bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbae408ece0 0x7fbae40a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:25.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.168 7fbae32bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbae408ece0 0x7fbae40a6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51728/0 (socket says 172.21.2.101:51728) 2021-02-10T11:33:25.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.168 7fbae32bf700 1 -- 172.21.2.101:0/4283067510 learned_addr learned my addr 172.21.2.101:0/4283067510 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:25.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.169 7fbae32bf700 1 -- 172.21.2.101:0/4283067510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbae40a7280 msgr2=0x7fbae40a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:25.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.169 7fbae32bf700 1 --2- 172.21.2.101:0/4283067510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbae40a7280 0x7fbae40a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:25.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.169 7fbae32bf700 1 -- 172.21.2.101:0/4283067510 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbae40a6860 msgr2=0x7fbae40a6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:25.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.169 7fbae32bf700 1 --2- 172.21.2.101:0/4283067510 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbae40a6860 0x7fbae40a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:25.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.169 7fbae32bf700 1 -- 172.21.2.101:0/4283067510 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae40a4bb0 con 0x7fbae408ece0 2021-02-10T11:33:25.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.169 7fbae32bf700 1 --2- 172.21.2.101:0/4283067510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbae408ece0 0x7fbae40a6240 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7fbacc009450 tx=0x7fbacc00bd20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:25.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.169 7fbadbfff700 1 -- 172.21.2.101:0/4283067510 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbacc00c2f0 con 0x7fbae408ece0 2021-02-10T11:33:25.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.169 7fbadbfff700 1 -- 172.21.2.101:0/4283067510 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbacc00be60 con 0x7fbae408ece0 2021-02-10T11:33:25.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.169 7fbadbfff700 1 -- 172.21.2.101:0/4283067510 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbacc00c7f0 con 0x7fbae408ece0 2021-02-10T11:33:25.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.169 7fbae9534700 1 -- 172.21.2.101:0/4283067510 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae40a4bb0 con 0x7fbae408ece0 2021-02-10T11:33:25.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.169 7fbae9534700 1 -- 172.21.2.101:0/4283067510 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae40a5140 con 0x7fbae408ece0 2021-02-10T11:33:25.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.170 7fbadbfff700 1 -- 172.21.2.101:0/4283067510 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbacc001010 con 0x7fbae408ece0 2021-02-10T11:33:25.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.170 7fbadbfff700 1 --2- 172.21.2.101:0/4283067510 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbac80186c0 0x7fbac801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:25.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.170 7fbadbfff700 1 -- 172.21.2.101:0/4283067510 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fbacc0327a0 con 0x7fbae408ece0 2021-02-10T11:33:25.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.170 7fbae2abe700 1 --2- 172.21.2.101:0/4283067510 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbac80186c0 0x7fbac801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:25.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.171 7fbae2abe700 1 --2- 172.21.2.101:0/4283067510 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbac80186c0 0x7fbac801abc0 crc :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:25.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.201 7fbae9534700 1 -- 172.21.2.101:0/4283067510 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae4105ae0 con 0x7fbae408ece0 2021-02-10T11:33:25.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.203 7fbadbfff700 1 -- 172.21.2.101:0/4283067510 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbacc024f60 con 0x7fbae408ece0 2021-02-10T11:33:25.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.330 7fbae9534700 1 -- 172.21.2.101:0/4283067510 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7fbae40f0b50 con 0x7fbae408ece0 2021-02-10T11:33:25.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.330 7fbadbfff700 1 -- 172.21.2.101:0/4283067510 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v55) v1 ==== 104+0+10 (secure 0 0 0) 0x7fbacc024f60 con 0x7fbae408ece0 2021-02-10T11:33:25.341 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:33:25.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.332 7fbae9534700 1 -- 172.21.2.101:0/4283067510 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbac80186c0 msgr2=0x7fbac801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:25.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.332 7fbae9534700 1 --2- 172.21.2.101:0/4283067510 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbac80186c0 0x7fbac801abc0 crc :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:25.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.332 7fbae9534700 1 -- 172.21.2.101:0/4283067510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbae408ece0 msgr2=0x7fbae40a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:25.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.332 7fbae9534700 1 --2- 172.21.2.101:0/4283067510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbae408ece0 0x7fbae40a6240 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7fbacc009450 tx=0x7fbacc00bd20).stop 2021-02-10T11:33:25.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.332 7fbae9534700 1 -- 172.21.2.101:0/4283067510 shutdown_connections 2021-02-10T11:33:25.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.332 7fbae9534700 1 --2- 172.21.2.101:0/4283067510 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbac80186c0 0x7fbac801abc0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:25.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.332 7fbae9534700 1 --2- 172.21.2.101:0/4283067510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbae40a7280 0x7fbae40a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:25.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.332 7fbae9534700 1 --2- 172.21.2.101:0/4283067510 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbae40a6860 0x7fbae40a6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:25.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.332 7fbae9534700 1 --2- 172.21.2.101:0/4283067510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbae408ece0 0x7fbae40a6240 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:25.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.332 7fbae9534700 1 -- 172.21.2.101:0/4283067510 >> 172.21.2.101:0/4283067510 conn(0x7fbae408ffe0 msgr2=0x7fbae40bf050 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:25.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.332 7fbae9534700 1 -- 172.21.2.101:0/4283067510 shutdown_connections 2021-02-10T11:33:25.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.332 7fbae9534700 1 -- 172.21.2.101:0/4283067510 wait complete. 2021-02-10T11:33:25.352 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:33:25.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.753 7fe188f69700 1 Processor -- start 2021-02-10T11:33:25.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.753 7fe188f69700 1 -- start start 2021-02-10T11:33:25.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.754 7fe188f69700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe1841b0870 0x7fe1840f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:25.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.754 7fe188f69700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1840efe00 0x7fe1840f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:25.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.754 7fe188f69700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1841b0c70 con 0x7fe1841b0470 2021-02-10T11:33:25.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.754 7fe188f69700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1840f0b50 con 0x7fe1841b0870 2021-02-10T11:33:25.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.754 7fe188f69700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1840f0820 con 0x7fe1840efe00 2021-02-10T11:33:25.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.754 7fe183314700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1840efe00 0x7fe1840f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:25.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.754 7fe183314700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1840efe00 0x7fe1840f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:44472/0 (socket says 172.21.2.101:44472) 2021-02-10T11:33:25.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.754 7fe183314700 1 -- 172.21.2.101:0/132772200 learned_addr learned my addr 172.21.2.101:0/132772200 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:25.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.754 7fe182312700 1 --2- 172.21.2.101:0/132772200 >> v2:172.21.2.108:3300/0 conn(0x7fe1841b0870 0x7fe1840f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:25.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.755 7fe183314700 1 -- 172.21.2.101:0/132772200 >> v2:172.21.2.108:3300/0 conn(0x7fe1841b0870 msgr2=0x7fe1840f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:25.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.755 7fe183314700 1 --2- 172.21.2.101:0/132772200 >> v2:172.21.2.108:3300/0 conn(0x7fe1841b0870 0x7fe1840f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:25.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.755 7fe183314700 1 -- 172.21.2.101:0/132772200 >> v1:172.21.2.108:6789/0 conn(0x7fe1841b0470 legacy=0x7fe1840ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:25.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.755 7fe183314700 1 -- 172.21.2.101:0/132772200 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1840e52e0 con 0x7fe1840efe00 2021-02-10T11:33:25.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.755 7fe183314700 1 --2- 172.21.2.101:0/132772200 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1840efe00 0x7fe1840f0200 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fe17c006aa0 tx=0x7fe17c009f20).ready entity=mon.2 client_cookie=c43911adec8e3e6a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:25.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.755 7fe181b11700 1 -- 172.21.2.101:0/132772200 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe17c00b5c0 con 0x7fe1840efe00 2021-02-10T11:33:25.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.755 7fe181b11700 1 -- 172.21.2.101:0/132772200 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17c003030 con 0x7fe1840efe00 2021-02-10T11:33:25.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.755 7fe181b11700 1 -- 172.21.2.101:0/132772200 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe17c00b9f0 con 0x7fe1840efe00 2021-02-10T11:33:25.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.755 7fe188f69700 1 -- 172.21.2.101:0/132772200 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1840efe00 msgr2=0x7fe1840f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:25.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.755 7fe188f69700 1 --2- 172.21.2.101:0/132772200 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1840efe00 0x7fe1840f0200 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fe17c006aa0 tx=0x7fe17c009f20).stop 2021-02-10T11:33:25.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.756 7fe188f69700 1 -- 172.21.2.101:0/132772200 shutdown_connections 2021-02-10T11:33:25.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.756 7fe188f69700 1 --2- 172.21.2.101:0/132772200 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe1840efe00 0x7fe1840f0200 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:25.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.756 7fe188f69700 1 --2- 172.21.2.101:0/132772200 >> v2:172.21.2.108:3300/0 conn(0x7fe1841b0870 0x7fe1840f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:25.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.756 7fe188f69700 1 -- 172.21.2.101:0/132772200 >> 172.21.2.101:0/132772200 conn(0x7fe1840ce3e0 msgr2=0x7fe1840e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:25.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.756 7fe188f69700 1 -- 172.21.2.101:0/132772200 shutdown_connections 2021-02-10T11:33:25.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.756 7fe188f69700 1 -- 172.21.2.101:0/132772200 wait complete. 2021-02-10T11:33:25.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.771 7fe188f69700 1 Processor -- start 2021-02-10T11:33:25.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.771 7fe188f69700 1 -- start start 2021-02-10T11:33:25.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.771 7fe188f69700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe1840a0420 0x7fe1840a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:25.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.771 7fe188f69700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe184189d00 0x7fe18418a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:25.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.771 7fe188f69700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe18418a720 0x7fe1840b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:25.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.771 7fe188f69700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe184189800 con 0x7fe18418a720 2021-02-10T11:33:25.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.771 7fe188f69700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18418ab20 con 0x7fe1840a0420 2021-02-10T11:33:25.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.771 7fe188f69700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1840a1140 con 0x7fe184189d00 2021-02-10T11:33:25.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.771 7fe183314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe18418a720 0x7fe1840b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:25.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.771 7fe183314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe18418a720 0x7fe1840b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51742/0 (socket says 172.21.2.101:51742) 2021-02-10T11:33:25.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.771 7fe183314700 1 -- 172.21.2.101:0/341654966 learned_addr learned my addr 172.21.2.101:0/341654966 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:25.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.772 7fe183314700 1 -- 172.21.2.101:0/341654966 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe184189d00 msgr2=0x7fe18418a100 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:25.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.772 7fe183314700 1 --2- 172.21.2.101:0/341654966 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe184189d00 0x7fe18418a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:25.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.772 7fe183314700 1 -- 172.21.2.101:0/341654966 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe1840a0420 msgr2=0x7fe1840a0820 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:25.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.772 7fe183314700 1 --2- 172.21.2.101:0/341654966 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe1840a0420 0x7fe1840a0820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:25.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.772 7fe183314700 1 -- 172.21.2.101:0/341654966 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1840b8200 con 0x7fe18418a720 2021-02-10T11:33:25.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.772 7fe183314700 1 --2- 172.21.2.101:0/341654966 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe18418a720 0x7fe1840b7be0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7fe17c00b970 tx=0x7fe17c00bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:25.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.772 7fe17b7fe700 1 -- 172.21.2.101:0/341654966 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe17c009450 con 0x7fe18418a720 2021-02-10T11:33:25.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.772 7fe17b7fe700 1 -- 172.21.2.101:0/341654966 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17c00c2f0 con 0x7fe18418a720 2021-02-10T11:33:25.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.772 7fe188f69700 1 -- 172.21.2.101:0/341654966 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1840b8200 con 0x7fe18418a720 2021-02-10T11:33:25.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.772 7fe188f69700 1 -- 172.21.2.101:0/341654966 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1840b8790 con 0x7fe18418a720 2021-02-10T11:33:25.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.772 7fe17b7fe700 1 -- 172.21.2.101:0/341654966 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe17c00c790 con 0x7fe18418a720 2021-02-10T11:33:25.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.772 7fe17b7fe700 1 -- 172.21.2.101:0/341654966 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe17c009fb0 con 0x7fe18418a720 2021-02-10T11:33:25.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.773 7fe17b7fe700 1 --2- 172.21.2.101:0/341654966 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe168018730 0x7fe16801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:25.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.773 7fe17b7fe700 1 -- 172.21.2.101:0/341654966 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe17c0329b0 con 0x7fe18418a720 2021-02-10T11:33:25.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.773 7fe182b13700 1 --2- 172.21.2.101:0/341654966 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe168018730 0x7fe16801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:25.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.773 7fe182b13700 1 --2- 172.21.2.101:0/341654966 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe168018730 0x7fe16801ac10 crc :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:25.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.804 7fe188f69700 1 -- 172.21.2.101:0/341654966 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe184105ae0 con 0x7fe18418a720 2021-02-10T11:33:25.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.806 7fe17b7fe700 1 -- 172.21.2.101:0/341654966 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe17c009450 con 0x7fe18418a720 2021-02-10T11:33:25.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.934 7fe188f69700 1 -- 172.21.2.101:0/341654966 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7fe1840f0b50 con 0x7fe18418a720 2021-02-10T11:33:25.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.934 7fe17b7fe700 1 -- 172.21.2.101:0/341654966 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v55) v1 ==== 100+0+10 (secure 0 0 0) 0x7fe17c009fb0 con 0x7fe18418a720 2021-02-10T11:33:25.946 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:33:25.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.936 7fe188f69700 1 -- 172.21.2.101:0/341654966 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe168018730 msgr2=0x7fe16801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:25.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.936 7fe188f69700 1 --2- 172.21.2.101:0/341654966 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe168018730 0x7fe16801ac10 crc :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:25.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.936 7fe188f69700 1 -- 172.21.2.101:0/341654966 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe18418a720 msgr2=0x7fe1840b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:25.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.936 7fe188f69700 1 --2- 172.21.2.101:0/341654966 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe18418a720 0x7fe1840b7be0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7fe17c00b970 tx=0x7fe17c00bd10).stop 2021-02-10T11:33:25.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.936 7fe188f69700 1 -- 172.21.2.101:0/341654966 shutdown_connections 2021-02-10T11:33:25.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.936 7fe188f69700 1 --2- 172.21.2.101:0/341654966 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe168018730 0x7fe16801ac10 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:25.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.937 7fe188f69700 1 --2- 172.21.2.101:0/341654966 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe184189d00 0x7fe18418a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:25.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.937 7fe188f69700 1 --2- 172.21.2.101:0/341654966 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe18418a720 0x7fe1840b7be0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:25.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.937 7fe188f69700 1 --2- 172.21.2.101:0/341654966 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe1840a0420 0x7fe1840a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:25.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.937 7fe188f69700 1 -- 172.21.2.101:0/341654966 >> 172.21.2.101:0/341654966 conn(0x7fe184094e10 msgr2=0x7fe1840a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:25.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.937 7fe188f69700 1 -- 172.21.2.101:0/341654966 shutdown_connections 2021-02-10T11:33:25.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:25.937 7fe188f69700 1 -- 172.21.2.101:0/341654966 wait complete. 2021-02-10T11:33:25.956 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:33:25.956 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:33:26.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.355 7f2d77d65700 1 Processor -- start 2021-02-10T11:33:26.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.355 7f2d77d65700 1 -- start start 2021-02-10T11:33:26.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.356 7f2d77d65700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2d700e5470 0x7f2d7007e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:26.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.356 7f2d77d65700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d7007ed80 con 0x7f2d700e5070 2021-02-10T11:33:26.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.356 7f2d77d65700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d701b4010 con 0x7f2d7007f300 2021-02-10T11:33:26.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.356 7f2d77d65700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d700d1790 con 0x7f2d700e5470 2021-02-10T11:33:26.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.356 7f2d75b81700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f2d700e5070 0x7f2d701b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52244/0 (socket says 172.21.2.101:52244) 2021-02-10T11:33:26.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.356 7f2d75b81700 1 -- 172.21.2.101:0/858158817 learned_addr learned my addr 172.21.2.101:0/858158817 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:26.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.356 7f2d74b7f700 1 -- 172.21.2.101:0/858158817 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4234862115 0 0) 0x7f2d64000ce0 con 0x7f2d700e5070 2021-02-10T11:33:26.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.356 7f2d74b7f700 1 -- 172.21.2.101:0/858158817 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d5c002fe0 con 0x7f2d700e5070 2021-02-10T11:33:26.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.356 7f2d75380700 1 --2- 172.21.2.101:0/858158817 >> v2:172.21.2.108:3300/0 conn(0x7f2d700e5470 0x7f2d7007e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:26.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d74b7f700 1 -- 172.21.2.101:0/858158817 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3273214620 0 0) 0x7f2d640010e0 con 0x7f2d700e5070 2021-02-10T11:33:26.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d74b7f700 1 -- 172.21.2.101:0/858158817 >> v2:172.21.2.108:3300/0 conn(0x7f2d700e5470 msgr2=0x7f2d7007e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:26.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d74b7f700 1 --2- 172.21.2.101:0/858158817 >> v2:172.21.2.108:3300/0 conn(0x7f2d700e5470 0x7f2d7007e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:26.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d74b7f700 1 -- 172.21.2.101:0/858158817 >> v1:172.21.2.108:6789/0 conn(0x7f2d7007f300 legacy=0x7f2d7007f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:26.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d74b7f700 1 -- 172.21.2.101:0/858158817 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d700d19b0 con 0x7f2d700e5070 2021-02-10T11:33:26.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d74b7f700 1 -- 172.21.2.101:0/858158817 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f2d64001660 con 0x7f2d700e5070 2021-02-10T11:33:26.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d74b7f700 1 -- 172.21.2.101:0/858158817 >> v1:172.21.2.101:6789/0 conn(0x7f2d700e5070 legacy=0x7f2d701b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:26.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d74b7f700 1 --2- 172.21.2.101:0/858158817 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d5c003a60 0x7f2d5c005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:26.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d74b7f700 1 --2- 172.21.2.101:0/858158817 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d5c006790 0x7f2d5c008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:26.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d74b7f700 1 --2- 172.21.2.101:0/858158817 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d5c009590 0x7f2d5c00bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:26.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d74b7f700 1 -- 172.21.2.101:0/858158817 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c00c300 con 0x7f2d5c006790 2021-02-10T11:33:26.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d74b7f700 1 -- 172.21.2.101:0/858158817 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c00c520 con 0x7f2d5c003a60 2021-02-10T11:33:26.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d74b7f700 1 -- 172.21.2.101:0/858158817 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c00c740 con 0x7f2d5c009590 2021-02-10T11:33:26.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d75380700 1 --2- 172.21.2.101:0/858158817 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d5c006790 0x7f2d5c008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:26.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d75b81700 1 --2- 172.21.2.101:0/858158817 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d5c003a60 0x7f2d5c005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:26.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d75380700 1 -- 172.21.2.101:0/858158817 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d5c009590 msgr2=0x7f2d5c00bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:26.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d76382700 1 --2- 172.21.2.101:0/858158817 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d5c009590 0x7f2d5c00bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:26.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d75380700 1 --2- 172.21.2.101:0/858158817 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d5c009590 0x7f2d5c00bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:26.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d75380700 1 -- 172.21.2.101:0/858158817 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d5c003a60 msgr2=0x7f2d5c005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:26.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d75380700 1 --2- 172.21.2.101:0/858158817 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d5c003a60 0x7f2d5c005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:26.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d75380700 1 -- 172.21.2.101:0/858158817 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5c050980 con 0x7f2d5c006790 2021-02-10T11:33:26.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d76382700 1 --2- 172.21.2.101:0/858158817 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d5c009590 0x7f2d5c00bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:33:26.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d76382700 1 -- 172.21.2.101:0/858158817 reap_dead start 2021-02-10T11:33:26.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.357 7f2d75380700 1 --2- 172.21.2.101:0/858158817 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d5c006790 0x7f2d5c008c90 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0x7f2d6c00c8f0 tx=0x7f2d6c00cc90).ready entity=mon.0 client_cookie=85d01f29ae1d934c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:26.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.358 7f2d74b7f700 1 -- 172.21.2.101:0/858158817 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d6c0009b0 con 0x7f2d5c006790 2021-02-10T11:33:26.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.358 7f2d74b7f700 1 -- 172.21.2.101:0/858158817 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6c009310 con 0x7f2d5c006790 2021-02-10T11:33:26.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.358 7f2d74b7f700 1 -- 172.21.2.101:0/858158817 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d6c009760 con 0x7f2d5c006790 2021-02-10T11:33:26.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.358 7f2d77d65700 1 -- 172.21.2.101:0/858158817 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d5c006790 msgr2=0x7f2d5c008c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:26.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.358 7f2d77d65700 1 --2- 172.21.2.101:0/858158817 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d5c006790 0x7f2d5c008c90 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0x7f2d6c00c8f0 tx=0x7f2d6c00cc90).stop 2021-02-10T11:33:26.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.358 7f2d77d65700 1 -- 172.21.2.101:0/858158817 shutdown_connections 2021-02-10T11:33:26.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.358 7f2d77d65700 1 --2- 172.21.2.101:0/858158817 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d5c006790 0x7f2d5c008c90 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:26.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.358 7f2d77d65700 1 -- 172.21.2.101:0/858158817 >> 172.21.2.101:0/858158817 conn(0x7f2d700ce690 msgr2=0x7f2d700e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:26.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.359 7f2d77d65700 1 -- 172.21.2.101:0/858158817 shutdown_connections 2021-02-10T11:33:26.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.359 7f2d77d65700 1 -- 172.21.2.101:0/858158817 wait complete. 2021-02-10T11:33:26.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.373 7f2d77d65700 1 Processor -- start 2021-02-10T11:33:26.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.373 7f2d77d65700 1 -- start start 2021-02-10T11:33:26.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.374 7f2d77d65700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d7018d210 0x7f2d701ab320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:26.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.374 7f2d77d65700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d700984c0 0x7f2d70097290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:26.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.374 7f2d77d65700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d700978b0 0x7f2d70097cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:26.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.374 7f2d77d65700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7009a540 con 0x7f2d700978b0 2021-02-10T11:33:26.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.374 7f2d77d65700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d701aba60 con 0x7f2d7018d210 2021-02-10T11:33:26.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.374 7f2d77d65700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7009aac0 con 0x7f2d700984c0 2021-02-10T11:33:26.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.374 7f2d76382700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d700978b0 0x7f2d70097cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:26.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.374 7f2d76382700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d700978b0 0x7f2d70097cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51762/0 (socket says 172.21.2.101:51762) 2021-02-10T11:33:26.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.374 7f2d76382700 1 -- 172.21.2.101:0/2463115635 learned_addr learned my addr 172.21.2.101:0/2463115635 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:26.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.374 7f2d76382700 1 -- 172.21.2.101:0/2463115635 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d700984c0 msgr2=0x7f2d70097290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:26.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.374 7f2d76382700 1 --2- 172.21.2.101:0/2463115635 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d700984c0 0x7f2d70097290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:26.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.374 7f2d76382700 1 -- 172.21.2.101:0/2463115635 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d7018d210 msgr2=0x7f2d701ab320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:26.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.374 7f2d76382700 1 --2- 172.21.2.101:0/2463115635 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d7018d210 0x7f2d701ab320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:26.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.374 7f2d76382700 1 -- 172.21.2.101:0/2463115635 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7009ace0 con 0x7f2d700978b0 2021-02-10T11:33:26.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.374 7f2d76382700 1 --2- 172.21.2.101:0/2463115635 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d700978b0 0x7f2d70097cb0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7f2d680099a0 tx=0x7f2d6800bcc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:26.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.375 7f2d627fc700 1 -- 172.21.2.101:0/2463115635 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d68006030 con 0x7f2d700978b0 2021-02-10T11:33:26.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.375 7f2d627fc700 1 -- 172.21.2.101:0/2463115635 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d680062b0 con 0x7f2d700978b0 2021-02-10T11:33:26.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.375 7f2d627fc700 1 -- 172.21.2.101:0/2463115635 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d680056a0 con 0x7f2d700978b0 2021-02-10T11:33:26.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.375 7f2d77d65700 1 -- 172.21.2.101:0/2463115635 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7009ace0 con 0x7f2d700978b0 2021-02-10T11:33:26.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.375 7f2d77d65700 1 -- 172.21.2.101:0/2463115635 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7009ace0 con 0x7f2d700978b0 2021-02-10T11:33:26.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.375 7f2d627fc700 1 -- 172.21.2.101:0/2463115635 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2d68006490 con 0x7f2d700978b0 2021-02-10T11:33:26.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.376 7f2d627fc700 1 --2- 172.21.2.101:0/2463115635 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d580186c0 0x7f2d5801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:26.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.376 7f2d627fc700 1 -- 172.21.2.101:0/2463115635 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2d68032910 con 0x7f2d700978b0 2021-02-10T11:33:26.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.376 7f2d75b81700 1 --2- 172.21.2.101:0/2463115635 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d580186c0 0x7f2d5801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:26.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.376 7f2d75b81700 1 --2- 172.21.2.101:0/2463115635 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d580186c0 0x7f2d5801abc0 crc :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:26.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.406 7f2d77d65700 1 -- 172.21.2.101:0/2463115635 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d70084b20 con 0x7f2d700978b0 2021-02-10T11:33:26.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.408 7f2d627fc700 1 -- 172.21.2.101:0/2463115635 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2d6800be00 con 0x7f2d700978b0 2021-02-10T11:33:26.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.539 7f2d77d65700 1 -- 172.21.2.101:0/2463115635 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d70084b20 con 0x7f2d700978b0 2021-02-10T11:33:26.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.539 7f2d627fc700 1 -- 172.21.2.101:0/2463115635 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+2098 (secure 0 0 0) 0x7f2d68006490 con 0x7f2d700978b0 2021-02-10T11:33:26.551 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:26.551 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8049,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/172865610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":172865610},{"type":"v1","addr":"172.21.2.101:6835","nonce":172865610}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:33:16.590282","modified":"2021-02-10 11:33:18.594009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5579},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5579":{"gid":5579,"name":"b","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2400978339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2400978339},{"type":"v1","addr":"172.21.2.108:6833","nonce":2400978339}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-02-10T11:33:26.551 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 38 2021-02-10T11:33:26.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.541 7f2d77d65700 1 -- 172.21.2.101:0/2463115635 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d580186c0 msgr2=0x7f2d5801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:26.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.541 7f2d77d65700 1 --2- 172.21.2.101:0/2463115635 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d580186c0 0x7f2d5801abc0 crc :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:26.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.541 7f2d77d65700 1 -- 172.21.2.101:0/2463115635 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d700978b0 msgr2=0x7f2d70097cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:26.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.541 7f2d77d65700 1 --2- 172.21.2.101:0/2463115635 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d700978b0 0x7f2d70097cb0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7f2d680099a0 tx=0x7f2d6800bcc0).stop 2021-02-10T11:33:26.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.541 7f2d77d65700 1 -- 172.21.2.101:0/2463115635 shutdown_connections 2021-02-10T11:33:26.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.541 7f2d77d65700 1 --2- 172.21.2.101:0/2463115635 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d580186c0 0x7f2d5801abc0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:26.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.541 7f2d77d65700 1 --2- 172.21.2.101:0/2463115635 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d700984c0 0x7f2d70097290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:26.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.541 7f2d77d65700 1 --2- 172.21.2.101:0/2463115635 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d700978b0 0x7f2d70097cb0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:26.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.541 7f2d77d65700 1 --2- 172.21.2.101:0/2463115635 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d7018d210 0x7f2d701ab320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:26.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.541 7f2d77d65700 1 -- 172.21.2.101:0/2463115635 >> 172.21.2.101:0/2463115635 conn(0x7f2d701b2a40 msgr2=0x7f2d70198bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:26.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.541 7f2d77d65700 1 -- 172.21.2.101:0/2463115635 shutdown_connections 2021-02-10T11:33:26.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:26.541 7f2d77d65700 1 -- 172.21.2.101:0/2463115635 wait complete. 2021-02-10T11:33:26.762 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:33:16.590282', 'modified': '2021-02-10 11:33:18.594009', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5579}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5579': {'gid': 5579, 'name': 'b', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/2400978339', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 2400978339}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 2400978339}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:33:26.797 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:33:26.797 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:33:27.219 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:33:27.219 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:33:27.219 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:33:27.219 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 5579, 2021-02-10T11:33:27.219 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:33:27.219 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:33:27.219 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 10.616256962, 2021-02-10T11:33:27.219 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 38, 2021-02-10T11:33:27.219 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 55, 2021-02-10T11:33:27.220 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 55, 2021-02-10T11:33:27.220 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 16.249146845999999 2021-02-10T11:33:27.220 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:33:27.223 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 5579, "want_state": "up:active", "state": "up:active", "rank_uptime": 10.616256962, "mdsmap_epoch": 38, "osdmap_epoch": 55, "osdmap_epoch_barrier": 55, "uptime": 16.249146845999999 } 2021-02-10T11:33:27.223 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:33:27.223 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-02-10T11:33:27.223 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-02-10T11:33:27.223 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:27.224 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:33:27.561 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:33:27.906 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:33:27.906 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:33:28.249 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:33:28.249 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:33:28.250 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:33:28.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:28.251 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:33:28.551 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:33:28.555 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:33:28.585 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-02-10T11:33:28.586 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:33:28.628 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:33:28.901 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:33:28.901 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:33:28.965 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[118322]: starting ceph client 2021-02-10T11:33:28.965 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:2021-02-10 11:33:28.954 7ff4a1d0bf80 -1 init, newargv = 0x5639aaf120e0 newargc=9 2021-02-10T11:33:28.971 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[118322]: starting fuse 2021-02-10T11:33:29.244 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:33:29.244 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:33:29.244 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:33:29.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:29.245 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:33:29.618 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:33:29.619 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2021-02-10T11:33:29.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:29.619 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:33:29.619 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:33:29.619 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:33:29.619 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:33:29.619 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:33:29.619 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:33:29.619 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:33:29.619 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-02-10T11:33:29.619 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:33:29.620 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:33:29.620 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:33:29.620 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:33:29.620 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:33:29.620 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:33:29.620 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:33:29.620 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:33:29.620 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:33:29.620 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:33:29.620 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:33:29.620 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:33:29.620 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:33:29.620 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.0")) 2021-02-10T11:33:29.620 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:33:29.983 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.0.118322.asok 2021-02-10T11:33:29.986 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.118322.asok 2021-02-10T11:33:29.986 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:29.986 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.118322.asok status 2021-02-10T11:33:30.377 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:33:30.377 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:33:30.377 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:33:30.377 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:33:30.377 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "0", 2021-02-10T11:33:30.377 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:33:30.378 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-02-10T11:33:30.378 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "118322", 2021-02-10T11:33:30.378 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:33:30.378 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:30.378 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:33:30.379 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:33:30.379 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 8199, 2021-02-10T11:33:30.379 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:33:30.379 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:33:30.379 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:33:30.380 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 8199 2021-02-10T11:33:30.380 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:30.380 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:33:30.380 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:30.380 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:33:30.380 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1621462478 2021-02-10T11:33:30.380 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:30.380 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:30.380 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:33:30.380 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:30.380 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:33:30.380 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1621462478 2021-02-10T11:33:30.381 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:30.381 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.8199 172.21.2.101:0/1621462478", 2021-02-10T11:33:30.381 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/1621462478", 2021-02-10T11:33:30.381 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:33:30.381 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 38, 2021-02-10T11:33:30.381 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 55, 2021-02-10T11:33:30.381 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:33:30.381 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:33:30.381 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:33:30.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:30.383 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:33:30.383 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:33:30.718 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:33:30.718 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:33:30.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:30.718 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:33:31.063 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:33:31.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.461 7fb02e22c700 1 Processor -- start 2021-02-10T11:33:31.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.461 7fb02e22c700 1 -- start start 2021-02-10T11:33:31.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.461 7fb02e22c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb0280e5470 0x7fb02807e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:31.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.461 7fb02e22c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb02807ed80 con 0x7fb0280e5070 2021-02-10T11:33:31.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.461 7fb02e22c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0281b4010 con 0x7fb02807f300 2021-02-10T11:33:31.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.461 7fb02e22c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0280d1790 con 0x7fb0280e5470 2021-02-10T11:33:31.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.462 7fb0277fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb0280e5470 0x7fb02807e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:31.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.462 7fb0277fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb0280e5470 0x7fb02807e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51786/0 (socket says 172.21.2.101:51786) 2021-02-10T11:33:31.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.462 7fb0277fe700 1 -- 172.21.2.101:0/1936488691 learned_addr learned my addr 172.21.2.101:0/1936488691 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:31.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.462 7fb0277fe700 1 -- 172.21.2.101:0/1936488691 >> v1:172.21.2.108:6789/0 conn(0x7fb02807f300 legacy=0x7fb02807f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:31.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.462 7fb0277fe700 1 -- 172.21.2.101:0/1936488691 >> v1:172.21.2.101:6789/0 conn(0x7fb0280e5070 legacy=0x7fb0281b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:31.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.462 7fb0277fe700 1 -- 172.21.2.101:0/1936488691 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0280d19b0 con 0x7fb0280e5470 2021-02-10T11:33:31.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.462 7fb026ffd700 1 -- 172.21.2.101:0/1936488691 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 629241403 0 0) 0x7fb018000ce0 con 0x7fb0280e5070 2021-02-10T11:33:31.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.462 7fb0277fe700 1 --2- 172.21.2.101:0/1936488691 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0280e5470 0x7fb02807e6a0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0x7fb01c006700 tx=0x7fb01c009b80).ready entity=mon.0 client_cookie=45308ba4577e96f9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:31.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.463 7fb026ffd700 1 -- 172.21.2.101:0/1936488691 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb01c00c0c0 con 0x7fb0280e5470 2021-02-10T11:33:31.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.463 7fb026ffd700 1 -- 172.21.2.101:0/1936488691 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01c00c440 con 0x7fb0280e5470 2021-02-10T11:33:31.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.463 7fb026ffd700 1 -- 172.21.2.101:0/1936488691 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb01c00c960 con 0x7fb0280e5470 2021-02-10T11:33:31.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.463 7fb02e22c700 1 -- 172.21.2.101:0/1936488691 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0280e5470 msgr2=0x7fb02807e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:31.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.463 7fb02e22c700 1 --2- 172.21.2.101:0/1936488691 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0280e5470 0x7fb02807e6a0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0x7fb01c006700 tx=0x7fb01c009b80).stop 2021-02-10T11:33:31.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.463 7fb02e22c700 1 -- 172.21.2.101:0/1936488691 shutdown_connections 2021-02-10T11:33:31.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.463 7fb02e22c700 1 --2- 172.21.2.101:0/1936488691 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0280e5470 0x7fb02807e6a0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:31.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.463 7fb02e22c700 1 -- 172.21.2.101:0/1936488691 >> 172.21.2.101:0/1936488691 conn(0x7fb0280ce690 msgr2=0x7fb0280e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:31.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.463 7fb02e22c700 1 -- 172.21.2.101:0/1936488691 shutdown_connections 2021-02-10T11:33:31.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.463 7fb02e22c700 1 -- 172.21.2.101:0/1936488691 wait complete. 2021-02-10T11:33:31.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.478 7fb02e22c700 1 Processor -- start 2021-02-10T11:33:31.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.478 7fb02e22c700 1 -- start start 2021-02-10T11:33:31.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.479 7fb02e22c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb02808c830 0x7fb02808d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:31.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.479 7fb02e22c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0281afe20 0x7fb0280b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:31.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.479 7fb02e22c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0281b0220 0x7fb0280b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:31.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.479 7fb02e22c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02808cc30 con 0x7fb0281b0220 2021-02-10T11:33:31.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.479 7fb02e22c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0280b3b70 con 0x7fb0281afe20 2021-02-10T11:33:31.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.479 7fb02e22c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0280b3d90 con 0x7fb02808c830 2021-02-10T11:33:31.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.479 7fb02c849700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0281b0220 0x7fb0280b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:31.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.479 7fb02c849700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0281b0220 0x7fb0280b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51792/0 (socket says 172.21.2.101:51792) 2021-02-10T11:33:31.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.479 7fb02c849700 1 -- 172.21.2.101:0/694792987 learned_addr learned my addr 172.21.2.101:0/694792987 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:31.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.479 7fb027fff700 1 --2- 172.21.2.101:0/694792987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb02808c830 0x7fb02808d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:31.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.479 7fb0277fe700 1 --2- 172.21.2.101:0/694792987 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0281afe20 0x7fb0280b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:31.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.479 7fb02c849700 1 -- 172.21.2.101:0/694792987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb02808c830 msgr2=0x7fb02808d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:31.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.479 7fb02c849700 1 --2- 172.21.2.101:0/694792987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb02808c830 0x7fb02808d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:31.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.479 7fb02c849700 1 -- 172.21.2.101:0/694792987 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0281afe20 msgr2=0x7fb0280b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:31.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.479 7fb02c849700 1 --2- 172.21.2.101:0/694792987 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0281afe20 0x7fb0280b2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:31.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.479 7fb02c849700 1 -- 172.21.2.101:0/694792987 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0281b0c00 con 0x7fb0281b0220 2021-02-10T11:33:31.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.479 7fb02c849700 1 --2- 172.21.2.101:0/694792987 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0281b0220 0x7fb0280b3550 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7fb02000b8e0 tx=0x7fb02000bc80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:31.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.480 7fb024ff9700 1 -- 172.21.2.101:0/694792987 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb020000f90 con 0x7fb0281b0220 2021-02-10T11:33:31.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.480 7fb024ff9700 1 -- 172.21.2.101:0/694792987 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02000c2f0 con 0x7fb0281b0220 2021-02-10T11:33:31.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.480 7fb024ff9700 1 -- 172.21.2.101:0/694792987 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb02000c7d0 con 0x7fb0281b0220 2021-02-10T11:33:31.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.480 7fb02e22c700 1 -- 172.21.2.101:0/694792987 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0281b0c00 con 0x7fb0281b0220 2021-02-10T11:33:31.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.480 7fb02e22c700 1 -- 172.21.2.101:0/694792987 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0281b0c00 con 0x7fb0281b0220 2021-02-10T11:33:31.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.480 7fb024ff9700 1 -- 172.21.2.101:0/694792987 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb020001220 con 0x7fb0281b0220 2021-02-10T11:33:31.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.480 7fb024ff9700 1 --2- 172.21.2.101:0/694792987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb008018730 0x7fb00801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:31.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.480 7fb024ff9700 1 -- 172.21.2.101:0/694792987 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb020032a00 con 0x7fb0281b0220 2021-02-10T11:33:31.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.480 7fb027fff700 1 --2- 172.21.2.101:0/694792987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb008018730 0x7fb00801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:31.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.481 7fb027fff700 1 --2- 172.21.2.101:0/694792987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb008018730 0x7fb00801abc0 crc :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:31.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.511 7fb02e22c700 1 -- 172.21.2.101:0/694792987 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb028083db0 con 0x7fb0281b0220 2021-02-10T11:33:31.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.514 7fb024ff9700 1 -- 172.21.2.101:0/694792987 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb02000bdc0 con 0x7fb0281b0220 2021-02-10T11:33:31.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.639 7fb02e22c700 1 -- 172.21.2.101:0/694792987 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb02807ed80 con 0x7fb0281b0220 2021-02-10T11:33:31.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.640 7fb024ff9700 1 -- 172.21.2.101:0/694792987 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+13996 (secure 0 0 0) 0x7fb020001220 con 0x7fb0281b0220 2021-02-10T11:33:31.652 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:31.652 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":55,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:33:17.591716","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:33:14.515763","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2021-02-10 11:33:15.636274","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:33:31.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.642 7fb02e22c700 1 -- 172.21.2.101:0/694792987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb008018730 msgr2=0x7fb00801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:31.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.642 7fb02e22c700 1 --2- 172.21.2.101:0/694792987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb008018730 0x7fb00801abc0 crc :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:31.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.642 7fb02e22c700 1 -- 172.21.2.101:0/694792987 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0281b0220 msgr2=0x7fb0280b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:31.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.642 7fb02e22c700 1 --2- 172.21.2.101:0/694792987 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0281b0220 0x7fb0280b3550 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7fb02000b8e0 tx=0x7fb02000bc80).stop 2021-02-10T11:33:31.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.642 7fb02e22c700 1 -- 172.21.2.101:0/694792987 shutdown_connections 2021-02-10T11:33:31.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.642 7fb02e22c700 1 --2- 172.21.2.101:0/694792987 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0281b0220 0x7fb0280b3550 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:31.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.642 7fb02e22c700 1 --2- 172.21.2.101:0/694792987 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0281afe20 0x7fb0280b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:31.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.642 7fb02e22c700 1 --2- 172.21.2.101:0/694792987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb008018730 0x7fb00801abc0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:31.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.642 7fb02e22c700 1 --2- 172.21.2.101:0/694792987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb02808c830 0x7fb02808d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:31.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.642 7fb02e22c700 1 -- 172.21.2.101:0/694792987 >> 172.21.2.101:0/694792987 conn(0x7fb0280937a0 msgr2=0x7fb028197e80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:31.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.642 7fb02e22c700 1 -- 172.21.2.101:0/694792987 shutdown_connections 2021-02-10T11:33:31.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:31.642 7fb02e22c700 1 -- 172.21.2.101:0/694792987 wait complete. 2021-02-10T11:33:31.662 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:33:14.515763', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:33:15.636274', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:33:31.662 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:33:32.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.063 7feb27804700 1 Processor -- start 2021-02-10T11:33:32.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.064 7feb27804700 1 -- start start 2021-02-10T11:33:32.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.064 7feb27804700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feb201b0470 0x7feb201b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:32.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.064 7feb27804700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7feb200ef610 0x7feb200efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:32.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.064 7feb27804700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7feb200f0030 0x7feb200f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:32.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.064 7feb27804700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb200e5ea0 con 0x7feb200ef610 2021-02-10T11:33:32.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.064 7feb27804700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb200f0b50 con 0x7feb200f0030 2021-02-10T11:33:32.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.064 7feb27804700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb200e52e0 con 0x7feb201b0470 2021-02-10T11:33:32.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.064 7feb24e1f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7feb200ef610 0x7feb200efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:32.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.064 7feb24e1f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7feb200ef610 0x7feb200efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51798/0 (socket says 172.21.2.101:51798) 2021-02-10T11:33:32.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.064 7feb24e1f700 1 -- 172.21.2.101:0/3563341219 learned_addr learned my addr 172.21.2.101:0/3563341219 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:32.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.065 7feb24e1f700 1 -- 172.21.2.101:0/3563341219 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feb201b0470 msgr2=0x7feb201b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:32.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.065 7feb24e1f700 1 --2- 172.21.2.101:0/3563341219 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feb201b0470 0x7feb201b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:32.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.065 7feb24e1f700 1 -- 172.21.2.101:0/3563341219 >> v2:172.21.2.108:3300/0 conn(0x7feb200f0030 msgr2=0x7feb200f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:32.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.065 7feb24e1f700 1 --2- 172.21.2.101:0/3563341219 >> v2:172.21.2.108:3300/0 conn(0x7feb200f0030 0x7feb200f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:32.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.065 7feb24e1f700 1 -- 172.21.2.101:0/3563341219 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb200f10d0 con 0x7feb200ef610 2021-02-10T11:33:32.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.065 7feb24e1f700 1 --2- 172.21.2.101:0/3563341219 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb200ef610 0x7feb200efa10 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7feb0c006700 tx=0x7feb0c009b80).ready entity=mon.0 client_cookie=34f7236f4693065c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:32.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.065 7feb1ffff700 1 -- 172.21.2.101:0/3563341219 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feb0c00c0c0 con 0x7feb200ef610 2021-02-10T11:33:32.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.065 7feb1ffff700 1 -- 172.21.2.101:0/3563341219 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0c00c440 con 0x7feb200ef610 2021-02-10T11:33:32.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.065 7feb1ffff700 1 -- 172.21.2.101:0/3563341219 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feb0c00c920 con 0x7feb200ef610 2021-02-10T11:33:32.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.065 7feb27804700 1 -- 172.21.2.101:0/3563341219 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb200ef610 msgr2=0x7feb200efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:32.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.065 7feb27804700 1 --2- 172.21.2.101:0/3563341219 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb200ef610 0x7feb200efa10 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7feb0c006700 tx=0x7feb0c009b80).stop 2021-02-10T11:33:32.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.066 7feb27804700 1 -- 172.21.2.101:0/3563341219 shutdown_connections 2021-02-10T11:33:32.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.066 7feb27804700 1 --2- 172.21.2.101:0/3563341219 >> v2:172.21.2.108:3300/0 conn(0x7feb200f0030 0x7feb200f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:32.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.066 7feb27804700 1 --2- 172.21.2.101:0/3563341219 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb200ef610 0x7feb200efa10 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:32.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.066 7feb27804700 1 --2- 172.21.2.101:0/3563341219 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feb201b0470 0x7feb201b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:32.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.066 7feb27804700 1 -- 172.21.2.101:0/3563341219 >> 172.21.2.101:0/3563341219 conn(0x7feb200ce3e0 msgr2=0x7feb200e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:32.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.066 7feb27804700 1 -- 172.21.2.101:0/3563341219 shutdown_connections 2021-02-10T11:33:32.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.066 7feb27804700 1 -- 172.21.2.101:0/3563341219 wait complete. 2021-02-10T11:33:32.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.080 7feb27804700 1 Processor -- start 2021-02-10T11:33:32.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.080 7feb27804700 1 -- start start 2021-02-10T11:33:32.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.081 7feb27804700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feb200b4c60 0x7feb20106fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:32.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.081 7feb27804700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb20190630 0x7feb20099f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:32.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.081 7feb27804700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feb20190a30 0x7feb2009b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:32.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.081 7feb27804700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb20107920 con 0x7feb20190630 2021-02-10T11:33:32.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.081 7feb27804700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2009abb0 con 0x7feb20190a30 2021-02-10T11:33:32.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.081 7feb27804700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb200b5060 con 0x7feb200b4c60 2021-02-10T11:33:32.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.081 7feb24e1f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb20190630 0x7feb20099f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:32.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.081 7feb24e1f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb20190630 0x7feb20099f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51802/0 (socket says 172.21.2.101:51802) 2021-02-10T11:33:32.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.081 7feb24e1f700 1 -- 172.21.2.101:0/3447226149 learned_addr learned my addr 172.21.2.101:0/3447226149 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:32.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.081 7feb24e1f700 1 -- 172.21.2.101:0/3447226149 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feb200b4c60 msgr2=0x7feb20106fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:32.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.081 7feb24e1f700 1 --2- 172.21.2.101:0/3447226149 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feb200b4c60 0x7feb20106fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:32.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.081 7feb24e1f700 1 -- 172.21.2.101:0/3447226149 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feb20190a30 msgr2=0x7feb2009b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:32.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.081 7feb24e1f700 1 --2- 172.21.2.101:0/3447226149 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feb20190a30 0x7feb2009b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:32.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.081 7feb24e1f700 1 -- 172.21.2.101:0/3447226149 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2009a5b0 con 0x7feb20190630 2021-02-10T11:33:32.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.082 7feb24e1f700 1 --2- 172.21.2.101:0/3447226149 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb20190630 0x7feb20099f90 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0x7feb0c00b650 tx=0x7feb0c00b9f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:32.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.082 7feb1dffb700 1 -- 172.21.2.101:0/3447226149 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feb0c00bde0 con 0x7feb20190630 2021-02-10T11:33:32.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.082 7feb1dffb700 1 -- 172.21.2.101:0/3447226149 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0c000c90 con 0x7feb20190630 2021-02-10T11:33:32.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.082 7feb27804700 1 -- 172.21.2.101:0/3447226149 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2009a5b0 con 0x7feb20190630 2021-02-10T11:33:32.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.082 7feb1dffb700 1 -- 172.21.2.101:0/3447226149 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feb0c0022b0 con 0x7feb20190630 2021-02-10T11:33:32.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.082 7feb27804700 1 -- 172.21.2.101:0/3447226149 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2009b750 con 0x7feb20190630 2021-02-10T11:33:32.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.082 7feb1dffb700 1 -- 172.21.2.101:0/3447226149 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7feb0c009f90 con 0x7feb20190630 2021-02-10T11:33:32.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.082 7feb1dffb700 1 --2- 172.21.2.101:0/3447226149 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feb00018730 0x7feb0001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:32.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.082 7feb1dffb700 1 -- 172.21.2.101:0/3447226149 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7feb0c032610 con 0x7feb20190630 2021-02-10T11:33:32.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.082 7feb25620700 1 --2- 172.21.2.101:0/3447226149 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feb00018730 0x7feb0001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:32.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.083 7feb25620700 1 --2- 172.21.2.101:0/3447226149 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feb00018730 0x7feb0001ac10 crc :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:32.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.114 7feb27804700 1 -- 172.21.2.101:0/3447226149 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb2009bcb0 con 0x7feb20190630 2021-02-10T11:33:32.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.116 7feb1dffb700 1 -- 172.21.2.101:0/3447226149 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7feb0c0009b0 con 0x7feb20190630 2021-02-10T11:33:32.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.244 7feb27804700 1 -- 172.21.2.101:0/3447226149 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7feb20105af0 con 0x7feb20190630 2021-02-10T11:33:32.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.244 7feb1dffb700 1 -- 172.21.2.101:0/3447226149 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v55) v1 ==== 92+0+10 (secure 0 0 0) 0x7feb0c00bde0 con 0x7feb20190630 2021-02-10T11:33:32.256 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:33:32.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.246 7feb27804700 1 -- 172.21.2.101:0/3447226149 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feb00018730 msgr2=0x7feb0001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:32.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.246 7feb27804700 1 --2- 172.21.2.101:0/3447226149 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feb00018730 0x7feb0001ac10 crc :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:32.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.246 7feb27804700 1 -- 172.21.2.101:0/3447226149 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb20190630 msgr2=0x7feb20099f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:32.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.246 7feb27804700 1 --2- 172.21.2.101:0/3447226149 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb20190630 0x7feb20099f90 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0x7feb0c00b650 tx=0x7feb0c00b9f0).stop 2021-02-10T11:33:32.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.246 7feb27804700 1 -- 172.21.2.101:0/3447226149 shutdown_connections 2021-02-10T11:33:32.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.246 7feb27804700 1 --2- 172.21.2.101:0/3447226149 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feb20190a30 0x7feb2009b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:32.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.246 7feb27804700 1 --2- 172.21.2.101:0/3447226149 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb20190630 0x7feb20099f90 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:32.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.246 7feb27804700 1 --2- 172.21.2.101:0/3447226149 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feb00018730 0x7feb0001ac10 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:32.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.246 7feb27804700 1 --2- 172.21.2.101:0/3447226149 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feb200b4c60 0x7feb20106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:32.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.246 7feb27804700 1 -- 172.21.2.101:0/3447226149 >> 172.21.2.101:0/3447226149 conn(0x7feb200c40c0 msgr2=0x7feb200a3520 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:32.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.247 7feb27804700 1 -- 172.21.2.101:0/3447226149 shutdown_connections 2021-02-10T11:33:32.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.247 7feb27804700 1 -- 172.21.2.101:0/3447226149 wait complete. 2021-02-10T11:33:32.266 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:33:32.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.668 7f2929574700 1 Processor -- start 2021-02-10T11:33:32.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.668 7f2929574700 1 -- start start 2021-02-10T11:33:32.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.668 7f2929574700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29241b0470 0x7f29241b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:32.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.668 7f2929574700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f29240ef610 0x7f29240efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:32.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.668 7f2929574700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f29240f0030 0x7f29240f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:32.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.668 7f2929574700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29240e5ea0 con 0x7f29240f0030 2021-02-10T11:33:32.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.668 7f2929574700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29240f0b50 con 0x7f29240ef610 2021-02-10T11:33:32.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.668 7f2929574700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29240e52e0 con 0x7f29241b0470 2021-02-10T11:33:32.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.669 7f2923ac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f29240f0030 0x7f29240f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:32.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.669 7f2923ac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f29240f0030 0x7f29240f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51810/0 (socket says 172.21.2.101:51810) 2021-02-10T11:33:32.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.669 7f2922abe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f29240ef610 0x7f29240efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:32.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.669 7f2923ac0700 1 -- 172.21.2.101:0/3215163304 learned_addr learned my addr 172.21.2.101:0/3215163304 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:32.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.669 7f29232bf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29241b0470 0x7f29241b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:32.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.669 7f2923ac0700 1 -- 172.21.2.101:0/3215163304 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29241b0470 msgr2=0x7f29241b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:32.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.669 7f2923ac0700 1 --2- 172.21.2.101:0/3215163304 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29241b0470 0x7f29241b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:32.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.669 7f2923ac0700 1 -- 172.21.2.101:0/3215163304 >> v2:172.21.2.108:3300/0 conn(0x7f29240ef610 msgr2=0x7f29240efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:32.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.669 7f2923ac0700 1 --2- 172.21.2.101:0/3215163304 >> v2:172.21.2.108:3300/0 conn(0x7f29240ef610 0x7f29240efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:32.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.669 7f2923ac0700 1 -- 172.21.2.101:0/3215163304 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29240f10d0 con 0x7f29240f0030 2021-02-10T11:33:32.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.669 7f2923ac0700 1 --2- 172.21.2.101:0/3215163304 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29240f0030 0x7f29240f0430 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f291c006b80 tx=0x7f291c009b90).ready entity=mon.0 client_cookie=fff9ef9f63e1bb4f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:32.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.670 7f29222bd700 1 -- 172.21.2.101:0/3215163304 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f291c0022e0 con 0x7f29240f0030 2021-02-10T11:33:32.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.670 7f29222bd700 1 -- 172.21.2.101:0/3215163304 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291c002620 con 0x7f29240f0030 2021-02-10T11:33:32.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.670 7f29222bd700 1 -- 172.21.2.101:0/3215163304 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f291c002b00 con 0x7f29240f0030 2021-02-10T11:33:32.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.670 7f2929574700 1 -- 172.21.2.101:0/3215163304 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29240f0030 msgr2=0x7f29240f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:32.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.670 7f2929574700 1 --2- 172.21.2.101:0/3215163304 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29240f0030 0x7f29240f0430 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f291c006b80 tx=0x7f291c009b90).stop 2021-02-10T11:33:32.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.670 7f2929574700 1 -- 172.21.2.101:0/3215163304 shutdown_connections 2021-02-10T11:33:32.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.670 7f2929574700 1 --2- 172.21.2.101:0/3215163304 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29240f0030 0x7f29240f0430 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:32.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.670 7f2929574700 1 --2- 172.21.2.101:0/3215163304 >> v2:172.21.2.108:3300/0 conn(0x7f29240ef610 0x7f29240efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:32.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.670 7f2929574700 1 --2- 172.21.2.101:0/3215163304 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29241b0470 0x7f29241b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:32.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.670 7f2929574700 1 -- 172.21.2.101:0/3215163304 >> 172.21.2.101:0/3215163304 conn(0x7f29240ce3e0 msgr2=0x7f29240e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:32.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.670 7f2929574700 1 -- 172.21.2.101:0/3215163304 shutdown_connections 2021-02-10T11:33:32.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.670 7f2929574700 1 -- 172.21.2.101:0/3215163304 wait complete. 2021-02-10T11:33:32.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.684 7f2929574700 1 Processor -- start 2021-02-10T11:33:32.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.685 7f2929574700 1 -- start start 2021-02-10T11:33:32.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.685 7f2929574700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29240b7db0 0x7f29240b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:32.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.685 7f2929574700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29240b8880 0x7f29241a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:32.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.685 7f2929574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29240b8c80 0x7f29241a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:32.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.685 7f2929574700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29240b79b0 con 0x7f29240b8c80 2021-02-10T11:33:32.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.685 7f2929574700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29241a8090 con 0x7f29240b7db0 2021-02-10T11:33:32.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.685 7f2929574700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29241a82b0 con 0x7f29240b8880 2021-02-10T11:33:32.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.685 7f2923ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29240b8c80 0x7f29241a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:32.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.685 7f2923ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29240b8c80 0x7f29241a7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51818/0 (socket says 172.21.2.101:51818) 2021-02-10T11:33:32.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.685 7f2923ac0700 1 -- 172.21.2.101:0/4068381434 learned_addr learned my addr 172.21.2.101:0/4068381434 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:32.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.686 7f2923ac0700 1 -- 172.21.2.101:0/4068381434 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29240b8880 msgr2=0x7f29241a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:32.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.686 7f2923ac0700 1 --2- 172.21.2.101:0/4068381434 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29240b8880 0x7f29241a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:32.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.686 7f2923ac0700 1 -- 172.21.2.101:0/4068381434 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29240b7db0 msgr2=0x7f29240b8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:32.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.686 7f2923ac0700 1 --2- 172.21.2.101:0/4068381434 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29240b7db0 0x7f29240b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:32.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.686 7f2923ac0700 1 -- 172.21.2.101:0/4068381434 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2924190c10 con 0x7f29240b8c80 2021-02-10T11:33:32.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.686 7f2923ac0700 1 --2- 172.21.2.101:0/4068381434 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29240b8c80 0x7f29241a7a70 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7f291c006ec0 tx=0x7f291c00b9e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:32.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.686 7f291bfff700 1 -- 172.21.2.101:0/4068381434 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f291c00bdd0 con 0x7f29240b8c80 2021-02-10T11:33:32.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.686 7f291bfff700 1 -- 172.21.2.101:0/4068381434 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291c006240 con 0x7f29240b8c80 2021-02-10T11:33:32.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.686 7f291bfff700 1 -- 172.21.2.101:0/4068381434 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f291c005720 con 0x7f29240b8c80 2021-02-10T11:33:32.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.686 7f2929574700 1 -- 172.21.2.101:0/4068381434 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2924190c10 con 0x7f29240b8c80 2021-02-10T11:33:32.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.686 7f2929574700 1 -- 172.21.2.101:0/4068381434 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2924190c10 con 0x7f29240b8c80 2021-02-10T11:33:32.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.687 7f291bfff700 1 -- 172.21.2.101:0/4068381434 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f291c006a30 con 0x7f29240b8c80 2021-02-10T11:33:32.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.687 7f291bfff700 1 --2- 172.21.2.101:0/4068381434 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29080186c0 0x7f290801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:32.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.687 7f291bfff700 1 -- 172.21.2.101:0/4068381434 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f291c0325a0 con 0x7f29240b8c80 2021-02-10T11:33:32.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.687 7f29232bf700 1 --2- 172.21.2.101:0/4068381434 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29080186c0 0x7f290801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:32.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.687 7f29232bf700 1 --2- 172.21.2.101:0/4068381434 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29080186c0 0x7f290801abc0 crc :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:32.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.719 7f2929574700 1 -- 172.21.2.101:0/4068381434 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2924105b30 con 0x7f29240b8c80 2021-02-10T11:33:32.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.721 7f291bfff700 1 -- 172.21.2.101:0/4068381434 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f291c006030 con 0x7f29240b8c80 2021-02-10T11:33:32.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.847 7f2929574700 1 -- 172.21.2.101:0/4068381434 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f29240f0b50 con 0x7f29240b8c80 2021-02-10T11:33:32.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.847 7f291bfff700 1 -- 172.21.2.101:0/4068381434 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v55) v1 ==== 104+0+10 (secure 0 0 0) 0x7f291c00bdd0 con 0x7f29240b8c80 2021-02-10T11:33:32.859 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:33:32.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.849 7f2929574700 1 -- 172.21.2.101:0/4068381434 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29080186c0 msgr2=0x7f290801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:32.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.849 7f2929574700 1 --2- 172.21.2.101:0/4068381434 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29080186c0 0x7f290801abc0 crc :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:32.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.849 7f2929574700 1 -- 172.21.2.101:0/4068381434 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29240b8c80 msgr2=0x7f29241a7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:32.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.849 7f2929574700 1 --2- 172.21.2.101:0/4068381434 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29240b8c80 0x7f29241a7a70 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7f291c006ec0 tx=0x7f291c00b9e0).stop 2021-02-10T11:33:32.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.849 7f2929574700 1 -- 172.21.2.101:0/4068381434 shutdown_connections 2021-02-10T11:33:32.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.849 7f2929574700 1 --2- 172.21.2.101:0/4068381434 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29080186c0 0x7f290801abc0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:32.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.849 7f2929574700 1 --2- 172.21.2.101:0/4068381434 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29240b8880 0x7f29241a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:32.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.849 7f2929574700 1 --2- 172.21.2.101:0/4068381434 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29240b8c80 0x7f29241a7a70 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:32.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.849 7f2929574700 1 --2- 172.21.2.101:0/4068381434 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29240b7db0 0x7f29240b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:32.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.849 7f2929574700 1 -- 172.21.2.101:0/4068381434 >> 172.21.2.101:0/4068381434 conn(0x7f2924094e10 msgr2=0x7f29240a59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:32.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.850 7f2929574700 1 -- 172.21.2.101:0/4068381434 shutdown_connections 2021-02-10T11:33:32.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:32.850 7f2929574700 1 -- 172.21.2.101:0/4068381434 wait complete. 2021-02-10T11:33:32.869 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:33:33.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.271 7f2d17604700 1 Processor -- start 2021-02-10T11:33:33.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.271 7f2d17604700 1 -- start start 2021-02-10T11:33:33.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.272 7f2d17604700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d101b0870 0x7f2d100f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:33.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.272 7f2d17604700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2d100efe00 0x7f2d100f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:33.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.272 7f2d17604700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d101b0c70 con 0x7f2d101b0470 2021-02-10T11:33:33.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.272 7f2d17604700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d100f0b50 con 0x7f2d100efe00 2021-02-10T11:33:33.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.272 7f2d17604700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d100f0820 con 0x7f2d101b0870 2021-02-10T11:33:33.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.272 7f2d15c21700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2d100efe00 0x7f2d100f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:33.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.272 7f2d15c21700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2d100efe00 0x7f2d100f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51826/0 (socket says 172.21.2.101:51826) 2021-02-10T11:33:33.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.272 7f2d15c21700 1 -- 172.21.2.101:0/2389735275 learned_addr learned my addr 172.21.2.101:0/2389735275 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:33.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.272 7f2d15c21700 1 -- 172.21.2.101:0/2389735275 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d101b0870 msgr2=0x7f2d100f20e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:33:33.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.272 7f2d15c21700 1 --2- 172.21.2.101:0/2389735275 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d101b0870 0x7f2d100f20e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:33.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.272 7f2d15c21700 1 -- 172.21.2.101:0/2389735275 >> v1:172.21.2.108:6789/0 conn(0x7f2d101b0470 legacy=0x7f2d100ef610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:33:33.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.272 7f2d15c21700 1 -- 172.21.2.101:0/2389735275 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d100e52e0 con 0x7f2d100efe00 2021-02-10T11:33:33.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.272 7f2d15c21700 1 --2- 172.21.2.101:0/2389735275 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d100efe00 0x7f2d100f0200 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7f2d08006f00 tx=0x7f2d08009f10).ready entity=mon.0 client_cookie=4b75bfe5913d3cd7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:33.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.273 7f2d07fff700 1 -- 172.21.2.101:0/2389735275 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d08003030 con 0x7f2d100efe00 2021-02-10T11:33:33.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.273 7f2d07fff700 1 -- 172.21.2.101:0/2389735275 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0800b3b0 con 0x7f2d100efe00 2021-02-10T11:33:33.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.273 7f2d07fff700 1 -- 172.21.2.101:0/2389735275 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d0800b8d0 con 0x7f2d100efe00 2021-02-10T11:33:33.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.273 7f2d17604700 1 -- 172.21.2.101:0/2389735275 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d100efe00 msgr2=0x7f2d100f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:33.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.273 7f2d17604700 1 --2- 172.21.2.101:0/2389735275 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d100efe00 0x7f2d100f0200 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7f2d08006f00 tx=0x7f2d08009f10).stop 2021-02-10T11:33:33.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.273 7f2d17604700 1 -- 172.21.2.101:0/2389735275 shutdown_connections 2021-02-10T11:33:33.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.273 7f2d17604700 1 --2- 172.21.2.101:0/2389735275 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d101b0870 0x7f2d100f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:33.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.273 7f2d17604700 1 --2- 172.21.2.101:0/2389735275 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d100efe00 0x7f2d100f0200 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:33.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.273 7f2d17604700 1 -- 172.21.2.101:0/2389735275 >> 172.21.2.101:0/2389735275 conn(0x7f2d100ce3e0 msgr2=0x7f2d100e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:33.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.274 7f2d17604700 1 -- 172.21.2.101:0/2389735275 shutdown_connections 2021-02-10T11:33:33.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.274 7f2d17604700 1 -- 172.21.2.101:0/2389735275 wait complete. 2021-02-10T11:33:33.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.288 7f2d17604700 1 Processor -- start 2021-02-10T11:33:33.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.288 7f2d17604700 1 -- start start 2021-02-10T11:33:33.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.289 7f2d17604700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d100a9690 0x7f2d100a9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:33.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.289 7f2d17604700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d100aa0b0 0x7f2d100aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:33.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.289 7f2d17604700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d100f1800 0x7f2d100f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:33.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.289 7f2d17604700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d10092350 con 0x7f2d100aa0b0 2021-02-10T11:33:33.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.289 7f2d17604700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d100a6540 con 0x7f2d100f1800 2021-02-10T11:33:33.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.289 7f2d17604700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d100a6720 con 0x7f2d100a9690 2021-02-10T11:33:33.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.289 7f2d14c1f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d100aa0b0 0x7f2d100aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:33.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.289 7f2d14c1f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d100aa0b0 0x7f2d100aa4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51830/0 (socket says 172.21.2.101:51830) 2021-02-10T11:33:33.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.289 7f2d14c1f700 1 -- 172.21.2.101:0/2872315597 learned_addr learned my addr 172.21.2.101:0/2872315597 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.289 7f2d15420700 1 --2- 172.21.2.101:0/2872315597 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d100a9690 0x7f2d100a9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.289 7f2d15c21700 1 --2- 172.21.2.101:0/2872315597 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d100f1800 0x7f2d100f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.289 7f2d14c1f700 1 -- 172.21.2.101:0/2872315597 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d100a9690 msgr2=0x7f2d100a9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.289 7f2d14c1f700 1 --2- 172.21.2.101:0/2872315597 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d100a9690 0x7f2d100a9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.289 7f2d14c1f700 1 -- 172.21.2.101:0/2872315597 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d100f1800 msgr2=0x7f2d100f1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.289 7f2d14c1f700 1 --2- 172.21.2.101:0/2872315597 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d100f1800 0x7f2d100f1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.289 7f2d14c1f700 1 -- 172.21.2.101:0/2872315597 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d100a69e0 con 0x7f2d100aa0b0 2021-02-10T11:33:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.289 7f2d14c1f700 1 --2- 172.21.2.101:0/2872315597 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d100aa0b0 0x7f2d100aa4b0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7f2d0000b5c0 tx=0x7f2d0000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.290 7f2d05ffb700 1 -- 172.21.2.101:0/2872315597 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d0000bd50 con 0x7f2d100aa0b0 2021-02-10T11:33:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.290 7f2d05ffb700 1 -- 172.21.2.101:0/2872315597 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d00006030 con 0x7f2d100aa0b0 2021-02-10T11:33:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.290 7f2d05ffb700 1 -- 172.21.2.101:0/2872315597 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d000054d0 con 0x7f2d100aa0b0 2021-02-10T11:33:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.290 7f2d17604700 1 -- 172.21.2.101:0/2872315597 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d100a69e0 con 0x7f2d100aa0b0 2021-02-10T11:33:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.290 7f2d17604700 1 -- 172.21.2.101:0/2872315597 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d100a69e0 con 0x7f2d100aa0b0 2021-02-10T11:33:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.290 7f2d05ffb700 1 -- 172.21.2.101:0/2872315597 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2d00006210 con 0x7f2d100aa0b0 2021-02-10T11:33:33.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.290 7f2d05ffb700 1 --2- 172.21.2.101:0/2872315597 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2cf00186c0 0x7f2cf001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:33.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.290 7f2d05ffb700 1 -- 172.21.2.101:0/2872315597 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2d00032550 con 0x7f2d100aa0b0 2021-02-10T11:33:33.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.291 7f2d15420700 1 --2- 172.21.2.101:0/2872315597 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2cf00186c0 0x7f2cf001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:33.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.291 7f2d15420700 1 --2- 172.21.2.101:0/2872315597 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2cf00186c0 0x7f2cf001abc0 crc :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:33.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.322 7f2d17604700 1 -- 172.21.2.101:0/2872315597 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d100edd10 con 0x7f2d100aa0b0 2021-02-10T11:33:33.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.324 7f2d05ffb700 1 -- 172.21.2.101:0/2872315597 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2d00006b80 con 0x7f2d100aa0b0 2021-02-10T11:33:33.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.453 7f2d17604700 1 -- 172.21.2.101:0/2872315597 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f2d100e52e0 con 0x7f2d100aa0b0 2021-02-10T11:33:33.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.454 7f2d05ffb700 1 -- 172.21.2.101:0/2872315597 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v55) v1 ==== 100+0+10 (secure 0 0 0) 0x7f2d0000bd50 con 0x7f2d100aa0b0 2021-02-10T11:33:33.465 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:33:33.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.455 7f2d17604700 1 -- 172.21.2.101:0/2872315597 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2cf00186c0 msgr2=0x7f2cf001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:33.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.455 7f2d17604700 1 --2- 172.21.2.101:0/2872315597 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2cf00186c0 0x7f2cf001abc0 crc :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:33.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.455 7f2d17604700 1 -- 172.21.2.101:0/2872315597 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d100aa0b0 msgr2=0x7f2d100aa4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:33.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.455 7f2d17604700 1 --2- 172.21.2.101:0/2872315597 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d100aa0b0 0x7f2d100aa4b0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7f2d0000b5c0 tx=0x7f2d0000b960).stop 2021-02-10T11:33:33.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.456 7f2d17604700 1 -- 172.21.2.101:0/2872315597 shutdown_connections 2021-02-10T11:33:33.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.456 7f2d17604700 1 --2- 172.21.2.101:0/2872315597 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d100f1800 0x7f2d100f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:33.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.456 7f2d17604700 1 --2- 172.21.2.101:0/2872315597 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d100aa0b0 0x7f2d100aa4b0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:33.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.456 7f2d17604700 1 --2- 172.21.2.101:0/2872315597 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2cf00186c0 0x7f2cf001abc0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:33.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.456 7f2d17604700 1 --2- 172.21.2.101:0/2872315597 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d100a9690 0x7f2d100a9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:33.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.456 7f2d17604700 1 -- 172.21.2.101:0/2872315597 >> 172.21.2.101:0/2872315597 conn(0x7f2d100c40e0 msgr2=0x7f2d10094eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:33.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.456 7f2d17604700 1 -- 172.21.2.101:0/2872315597 shutdown_connections 2021-02-10T11:33:33.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.456 7f2d17604700 1 -- 172.21.2.101:0/2872315597 wait complete. 2021-02-10T11:33:33.475 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:33:33.475 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:33:33.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.873 7f9d6d23c700 1 Processor -- start 2021-02-10T11:33:33.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.873 7f9d6d23c700 1 -- start start 2021-02-10T11:33:33.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.874 7f9d6d23c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f9d680e5070 0x7f9d681b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:33.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.874 7f9d6d23c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d681b3e40 0x7f9d6807d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:33.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.874 7f9d6d23c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9d680e5470 0x7f9d6807dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:33.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.874 7f9d6d23c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d6807ed80 con 0x7f9d680e5470 2021-02-10T11:33:33.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.874 7f9d6d23c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d6807e2d0 con 0x7f9d680e5070 2021-02-10T11:33:33.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.874 7f9d6d23c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6807e4c0 con 0x7f9d681b3e40 2021-02-10T11:33:33.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.874 7f9d6759e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9d680e5470 0x7f9d6807dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:33.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.874 7f9d6759e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9d680e5470 0x7f9d6807dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51836/0 (socket says 172.21.2.101:51836) 2021-02-10T11:33:33.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.874 7f9d6759e700 1 -- 172.21.2.101:0/3613627648 learned_addr learned my addr 172.21.2.101:0/3613627648 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:33.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.874 7f9d6759e700 1 -- 172.21.2.101:0/3613627648 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d681b3e40 msgr2=0x7f9d6807d690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:33:33.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.874 7f9d6759e700 1 --2- 172.21.2.101:0/3613627648 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d681b3e40 0x7f9d6807d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:33.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.875 7f9d6759e700 1 -- 172.21.2.101:0/3613627648 >> v2:172.21.2.108:3300/0 conn(0x7f9d680e5070 msgr2=0x7f9d681b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:33.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.875 7f9d6759e700 1 --2- 172.21.2.101:0/3613627648 >> v2:172.21.2.108:3300/0 conn(0x7f9d680e5070 0x7f9d681b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:33.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.875 7f9d6759e700 1 -- 172.21.2.101:0/3613627648 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6807e6e0 con 0x7f9d680e5470 2021-02-10T11:33:33.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.875 7f9d6759e700 1 --2- 172.21.2.101:0/3613627648 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d680e5470 0x7f9d6807dcb0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7f9d60006f00 tx=0x7f9d60009e90).ready entity=mon.0 client_cookie=384f2c0cca6e1fd1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:33.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.875 7f9d65d9b700 1 -- 172.21.2.101:0/3613627648 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d6000b300 con 0x7f9d680e5470 2021-02-10T11:33:33.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.875 7f9d65d9b700 1 -- 172.21.2.101:0/3613627648 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6000b580 con 0x7f9d680e5470 2021-02-10T11:33:33.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.875 7f9d65d9b700 1 -- 172.21.2.101:0/3613627648 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d6000b9d0 con 0x7f9d680e5470 2021-02-10T11:33:33.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.875 7f9d6d23c700 1 -- 172.21.2.101:0/3613627648 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d680e5470 msgr2=0x7f9d6807dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:33.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.875 7f9d6d23c700 1 --2- 172.21.2.101:0/3613627648 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d680e5470 0x7f9d6807dcb0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7f9d60006f00 tx=0x7f9d60009e90).stop 2021-02-10T11:33:33.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.876 7f9d6d23c700 1 -- 172.21.2.101:0/3613627648 shutdown_connections 2021-02-10T11:33:33.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.876 7f9d6d23c700 1 --2- 172.21.2.101:0/3613627648 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d681b3e40 0x7f9d6807d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:33.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.876 7f9d6d23c700 1 --2- 172.21.2.101:0/3613627648 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d680e5470 0x7f9d6807dcb0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:33.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.876 7f9d6d23c700 1 --2- 172.21.2.101:0/3613627648 >> v2:172.21.2.108:3300/0 conn(0x7f9d680e5070 0x7f9d681b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:33.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.876 7f9d6d23c700 1 -- 172.21.2.101:0/3613627648 >> 172.21.2.101:0/3613627648 conn(0x7f9d680ce690 msgr2=0x7f9d680e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:33.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.876 7f9d6d23c700 1 -- 172.21.2.101:0/3613627648 shutdown_connections 2021-02-10T11:33:33.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.876 7f9d6d23c700 1 -- 172.21.2.101:0/3613627648 wait complete. 2021-02-10T11:33:33.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.890 7f9d6d23c700 1 Processor -- start 2021-02-10T11:33:33.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.890 7f9d6d23c700 1 -- start start 2021-02-10T11:33:33.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.891 7f9d6d23c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d68192fc0 0x7f9d68192780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:33.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.891 7f9d6d23c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d680b1430 0x7f9d680b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:33.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.891 7f9d6d23c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d680b1e50 0x7f9d6808a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:33.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.891 7f9d6d23c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d680b2250 con 0x7f9d680b1430 2021-02-10T11:33:33.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.891 7f9d6d23c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6808b5d0 con 0x7f9d680b1e50 2021-02-10T11:33:33.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.891 7f9d6d23c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6808aa80 con 0x7f9d68192fc0 2021-02-10T11:33:33.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.891 7f9d6659c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d680b1430 0x7f9d680b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:33.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.891 7f9d6659c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d680b1430 0x7f9d680b1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51844/0 (socket says 172.21.2.101:51844) 2021-02-10T11:33:33.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.891 7f9d6659c700 1 -- 172.21.2.101:0/4183859885 learned_addr learned my addr 172.21.2.101:0/4183859885 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:33.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.891 7f9d6659c700 1 -- 172.21.2.101:0/4183859885 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d68192fc0 msgr2=0x7f9d68192780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:33.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.891 7f9d6659c700 1 --2- 172.21.2.101:0/4183859885 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d68192fc0 0x7f9d68192780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:33.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.891 7f9d6659c700 1 -- 172.21.2.101:0/4183859885 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d680b1e50 msgr2=0x7f9d6808a460 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:33.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.891 7f9d6659c700 1 --2- 172.21.2.101:0/4183859885 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d680b1e50 0x7f9d6808a460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:33.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.891 7f9d6659c700 1 -- 172.21.2.101:0/4183859885 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6808ac60 con 0x7f9d680b1430 2021-02-10T11:33:33.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.891 7f9d6659c700 1 --2- 172.21.2.101:0/4183859885 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d680b1430 0x7f9d680b1830 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7f9d5c00b5c0 tx=0x7f9d5c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:33.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.892 7f9d577fe700 1 -- 172.21.2.101:0/4183859885 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d5c006b80 con 0x7f9d680b1430 2021-02-10T11:33:33.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.892 7f9d577fe700 1 -- 172.21.2.101:0/4183859885 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5c00bd50 con 0x7f9d680b1430 2021-02-10T11:33:33.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.892 7f9d6d23c700 1 -- 172.21.2.101:0/4183859885 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6808ac60 con 0x7f9d680b1430 2021-02-10T11:33:33.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.892 7f9d6d23c700 1 -- 172.21.2.101:0/4183859885 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6808b1f0 con 0x7f9d680b1430 2021-02-10T11:33:33.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.892 7f9d577fe700 1 -- 172.21.2.101:0/4183859885 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d5c0054d0 con 0x7f9d680b1430 2021-02-10T11:33:33.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.892 7f9d577fe700 1 -- 172.21.2.101:0/4183859885 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9d5c006030 con 0x7f9d680b1430 2021-02-10T11:33:33.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.893 7f9d577fe700 1 --2- 172.21.2.101:0/4183859885 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d4c018730 0x7f9d4c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:33.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.893 7f9d577fe700 1 -- 172.21.2.101:0/4183859885 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9d5c032820 con 0x7f9d680b1430 2021-02-10T11:33:33.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.893 7f9d66d9d700 1 --2- 172.21.2.101:0/4183859885 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d4c018730 0x7f9d4c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:33.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.893 7f9d66d9d700 1 --2- 172.21.2.101:0/4183859885 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d4c018730 0x7f9d4c01ac10 crc :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:33.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.924 7f9d6d23c700 1 -- 172.21.2.101:0/4183859885 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d68107120 con 0x7f9d680b1430 2021-02-10T11:33:33.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:33.927 7f9d577fe700 1 -- 172.21.2.101:0/4183859885 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9d5c0009b0 con 0x7f9d680b1430 2021-02-10T11:33:34.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:34.060 7f9d6d23c700 1 -- 172.21.2.101:0/4183859885 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d6807ed80 con 0x7f9d680b1430 2021-02-10T11:33:34.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:34.060 7f9d577fe700 1 -- 172.21.2.101:0/4183859885 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+2098 (secure 0 0 0) 0x7f9d5c006030 con 0x7f9d680b1430 2021-02-10T11:33:34.072 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:34.072 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8049,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/172865610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":172865610},{"type":"v1","addr":"172.21.2.101:6835","nonce":172865610}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:33:16.590282","modified":"2021-02-10 11:33:18.594009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5579},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5579":{"gid":5579,"name":"b","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2400978339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2400978339},{"type":"v1","addr":"172.21.2.108:6833","nonce":2400978339}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-02-10T11:33:34.072 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 38 2021-02-10T11:33:34.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:34.062 7f9d6d23c700 1 -- 172.21.2.101:0/4183859885 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d4c018730 msgr2=0x7f9d4c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:34.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:34.062 7f9d6d23c700 1 --2- 172.21.2.101:0/4183859885 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d4c018730 0x7f9d4c01ac10 crc :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:34.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:34.062 7f9d6d23c700 1 -- 172.21.2.101:0/4183859885 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d680b1430 msgr2=0x7f9d680b1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:34.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:34.062 7f9d6d23c700 1 --2- 172.21.2.101:0/4183859885 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d680b1430 0x7f9d680b1830 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7f9d5c00b5c0 tx=0x7f9d5c00b960).stop 2021-02-10T11:33:34.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:34.062 7f9d6d23c700 1 -- 172.21.2.101:0/4183859885 shutdown_connections 2021-02-10T11:33:34.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:34.062 7f9d6d23c700 1 --2- 172.21.2.101:0/4183859885 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d680b1e50 0x7f9d6808a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:34.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:34.062 7f9d6d23c700 1 --2- 172.21.2.101:0/4183859885 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d680b1430 0x7f9d680b1830 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:34.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:34.062 7f9d6d23c700 1 --2- 172.21.2.101:0/4183859885 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d4c018730 0x7f9d4c01ac10 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:34.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:34.062 7f9d6d23c700 1 --2- 172.21.2.101:0/4183859885 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d68192fc0 0x7f9d68192780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:34.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:34.062 7f9d6d23c700 1 -- 172.21.2.101:0/4183859885 >> 172.21.2.101:0/4183859885 conn(0x7f9d681ac620 msgr2=0x7f9d6819fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:34.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:34.062 7f9d6d23c700 1 -- 172.21.2.101:0/4183859885 shutdown_connections 2021-02-10T11:33:34.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:34.062 7f9d6d23c700 1 -- 172.21.2.101:0/4183859885 wait complete. 2021-02-10T11:33:34.283 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:33:16.590282', 'modified': '2021-02-10 11:33:18.594009', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5579}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5579': {'gid': 5579, 'name': 'b', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/2400978339', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 2400978339}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 2400978339}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:33:34.284 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:33:34.284 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:33:34.682 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:33:34.682 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:33:34.683 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:33:34.683 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 5579, 2021-02-10T11:33:34.683 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:33:34.683 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:33:34.683 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 18.079907473999999, 2021-02-10T11:33:34.683 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 38, 2021-02-10T11:33:34.683 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 55, 2021-02-10T11:33:34.683 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 55, 2021-02-10T11:33:34.683 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 23.712798982999999 2021-02-10T11:33:34.683 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:33:34.687 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 5579, "want_state": "up:active", "state": "up:active", "rank_uptime": 18.079907473999999, "mdsmap_epoch": 38, "osdmap_epoch": 55, "osdmap_epoch_barrier": 55, "uptime": 23.712798982999999 } 2021-02-10T11:33:34.687 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:33:34.687 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-02-10T11:33:34.687 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:33:34.687 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:34.688 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:33:35.022 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:33:35.365 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:33:35.366 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:33:35.708 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:33:35.708 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:33:35.708 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:33:35.709 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:35.709 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:33:36.043 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:33:36.043 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2021-02-10T11:33:36.043 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:33:36.046 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:33:36.389 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:33:36.390 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:33:36.456 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[118963]: starting ceph client 2021-02-10T11:33:36.456 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:2021-02-10 11:33:36.445 7f8582517f80 -1 init, newargv = 0x557af07dc100 newargc=9 2021-02-10T11:33:36.462 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[118963]: starting fuse 2021-02-10T11:33:36.732 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:33:36.732 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:33:36.733 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:33:36.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:36.733 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:33:37.068 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:33:37.068 INFO:teuthology.orchestra.run.gibba001.stdout:44 2021-02-10T11:33:37.068 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2021-02-10T11:33:37.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:37.069 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:33:37.069 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:33:37.069 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:33:37.069 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:33:37.069 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:33:37.069 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:33:37.069 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:33:37.069 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-02-10T11:33:37.069 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:33:37.069 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:33:37.070 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:33:37.070 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:33:37.070 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:33:37.070 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:33:37.070 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:33:37.070 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:33:37.070 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:33:37.070 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:33:37.070 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:33:37.070 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:33:37.070 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:33:37.070 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.1")) 2021-02-10T11:33:37.070 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:33:37.430 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.1.118963.asok 2021-02-10T11:33:37.433 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.118963.asok 2021-02-10T11:33:37.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:37.433 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.118963.asok status 2021-02-10T11:33:37.823 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:33:37.823 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:33:37.823 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:33:37.824 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:33:37.824 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "1", 2021-02-10T11:33:37.824 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:33:37.824 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:33:37.824 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "118963", 2021-02-10T11:33:37.824 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:33:37.824 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:37.824 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:33:37.824 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:33:37.824 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 8238, 2021-02-10T11:33:37.824 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:33:37.824 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:33:37.824 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:33:37.825 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 8238 2021-02-10T11:33:37.825 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:37.825 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:33:37.825 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:37.825 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:33:37.825 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 2302247019 2021-02-10T11:33:37.825 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:37.825 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:37.825 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:33:37.825 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:37.825 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:33:37.825 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 2302247019 2021-02-10T11:33:37.825 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:37.825 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.8238 172.21.2.101:0/2302247019", 2021-02-10T11:33:37.826 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/2302247019", 2021-02-10T11:33:37.826 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:33:37.826 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 38, 2021-02-10T11:33:37.826 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 55, 2021-02-10T11:33:37.826 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:33:37.826 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:33:37.826 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:33:37.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:37.827 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:33:37.827 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:33:38.163 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:33:38.163 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:33:38.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:38.164 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:33:38.510 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2021-02-10T11:33:38.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.914 7f871a6c0700 1 Processor -- start 2021-02-10T11:33:38.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.914 7f871a6c0700 1 -- start start 2021-02-10T11:33:38.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.915 7f871a6c0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87140d1870 0x7f87141b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:38.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.915 7f871a6c0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f87141b0ae0 0x7f8714108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:38.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.915 7f871a6c0700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8714109a60 con 0x7f87140d1c70 2021-02-10T11:33:38.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.915 7f871a6c0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87140e5ef0 con 0x7f87141b0ae0 2021-02-10T11:33:38.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.915 7f871a6c0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8714109750 con 0x7f87140d1870 2021-02-10T11:33:38.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.915 7f8718cdd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f87140d1c70 0x7f8714108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52356/0 (socket says 172.21.2.101:52356) 2021-02-10T11:33:38.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.915 7f8718cdd700 1 -- 172.21.2.101:0/418996515 learned_addr learned my addr 172.21.2.101:0/418996515 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:38.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.915 7f87137fe700 1 --2- 172.21.2.101:0/418996515 >> v2:172.21.2.101:3300/0 conn(0x7f87141b0ae0 0x7f8714108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:38.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.915 7f8712ffd700 1 -- 172.21.2.101:0/418996515 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1046239645 0 0) 0x7f870c001500 con 0x7f87140d1c70 2021-02-10T11:33:38.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.915 7f8712ffd700 1 -- 172.21.2.101:0/418996515 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86fc002fe0 con 0x7f87140d1c70 2021-02-10T11:33:38.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.915 7f87137fe700 1 -- 172.21.2.101:0/418996515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87140d1870 msgr2=0x7f87141b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:38.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.915 7f87137fe700 1 --2- 172.21.2.101:0/418996515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87140d1870 0x7f87141b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:38.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.915 7f87137fe700 1 -- 172.21.2.101:0/418996515 >> v1:172.21.2.101:6789/0 conn(0x7f87140d1c70 legacy=0x7f8714108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:38.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.915 7f87137fe700 1 -- 172.21.2.101:0/418996515 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8714088020 con 0x7f87141b0ae0 2021-02-10T11:33:38.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.916 7f87137fe700 1 --2- 172.21.2.101:0/418996515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87141b0ae0 0x7f8714108940 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7f8708006650 tx=0x7f8708009a80).ready entity=mon.0 client_cookie=2036a09d43fc6f6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:38.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.916 7f8712ffd700 1 -- 172.21.2.101:0/418996515 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f870800c0c0 con 0x7f87141b0ae0 2021-02-10T11:33:38.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.916 7f8712ffd700 1 -- 172.21.2.101:0/418996515 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870800c3c0 con 0x7f87141b0ae0 2021-02-10T11:33:38.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.916 7f8712ffd700 1 -- 172.21.2.101:0/418996515 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f870800c820 con 0x7f87141b0ae0 2021-02-10T11:33:38.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.916 7f871a6c0700 1 -- 172.21.2.101:0/418996515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87141b0ae0 msgr2=0x7f8714108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:38.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.916 7f871a6c0700 1 --2- 172.21.2.101:0/418996515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87141b0ae0 0x7f8714108940 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7f8708006650 tx=0x7f8708009a80).stop 2021-02-10T11:33:38.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.916 7f871a6c0700 1 -- 172.21.2.101:0/418996515 shutdown_connections 2021-02-10T11:33:38.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.916 7f871a6c0700 1 --2- 172.21.2.101:0/418996515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87141b0ae0 0x7f8714108940 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:38.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.916 7f871a6c0700 1 --2- 172.21.2.101:0/418996515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87140d1870 0x7f87141b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:38.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.916 7f871a6c0700 1 -- 172.21.2.101:0/418996515 >> 172.21.2.101:0/418996515 conn(0x7f87140ce420 msgr2=0x7f87140d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:38.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.917 7f871a6c0700 1 -- 172.21.2.101:0/418996515 shutdown_connections 2021-02-10T11:33:38.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.917 7f871a6c0700 1 -- 172.21.2.101:0/418996515 wait complete. 2021-02-10T11:33:38.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.931 7f871a6c0700 1 Processor -- start 2021-02-10T11:33:38.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.931 7f871a6c0700 1 -- start start 2021-02-10T11:33:38.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.931 7f871a6c0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87140aa9e0 0x7f871418ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:38.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.932 7f871a6c0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f871418d4d0 0x7f8714098df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:38.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.932 7f871a6c0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8714099410 0x7f87141a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:38.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.932 7f871a6c0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8714098870 con 0x7f8714099410 2021-02-10T11:33:38.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.932 7f871a6c0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87141a72e0 con 0x7f87140aa9e0 2021-02-10T11:33:38.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.932 7f871a6c0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87141a74c0 con 0x7f871418d4d0 2021-02-10T11:33:38.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.932 7f8718cdd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8714099410 0x7f87141a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:38.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.932 7f8718cdd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8714099410 0x7f87141a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51868/0 (socket says 172.21.2.101:51868) 2021-02-10T11:33:38.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.932 7f8718cdd700 1 -- 172.21.2.101:0/1916203718 learned_addr learned my addr 172.21.2.101:0/1916203718 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:38.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.932 7f87137fe700 1 --2- 172.21.2.101:0/1916203718 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f871418d4d0 0x7f8714098df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:38.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.932 7f8713fff700 1 --2- 172.21.2.101:0/1916203718 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87140aa9e0 0x7f871418ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:38.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.932 7f8718cdd700 1 -- 172.21.2.101:0/1916203718 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f871418d4d0 msgr2=0x7f8714098df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:38.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.932 7f8718cdd700 1 --2- 172.21.2.101:0/1916203718 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f871418d4d0 0x7f8714098df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:38.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.932 7f8718cdd700 1 -- 172.21.2.101:0/1916203718 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87140aa9e0 msgr2=0x7f871418ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:38.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.932 7f8718cdd700 1 --2- 172.21.2.101:0/1916203718 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87140aa9e0 0x7f871418ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:38.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.932 7f8718cdd700 1 -- 172.21.2.101:0/1916203718 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87141a76e0 con 0x7f8714099410 2021-02-10T11:33:38.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.932 7f8718cdd700 1 --2- 172.21.2.101:0/1916203718 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8714099410 0x7f87141a6cc0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7f870c001bf0 tx=0x7f870c00ac50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:38.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.933 7f8710ff9700 1 -- 172.21.2.101:0/1916203718 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f870c0070d0 con 0x7f8714099410 2021-02-10T11:33:38.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.933 7f8710ff9700 1 -- 172.21.2.101:0/1916203718 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870c007350 con 0x7f8714099410 2021-02-10T11:33:38.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.933 7f8710ff9700 1 -- 172.21.2.101:0/1916203718 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f870c004720 con 0x7f8714099410 2021-02-10T11:33:38.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.933 7f871a6c0700 1 -- 172.21.2.101:0/1916203718 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87141a76e0 con 0x7f8714099410 2021-02-10T11:33:38.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.933 7f871a6c0700 1 -- 172.21.2.101:0/1916203718 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8714097c90 con 0x7f8714099410 2021-02-10T11:33:38.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.933 7f8710ff9700 1 -- 172.21.2.101:0/1916203718 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f870c004720 con 0x7f8714099410 2021-02-10T11:33:38.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.933 7f8710ff9700 1 --2- 172.21.2.101:0/1916203718 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86f40186c0 0x7f86f401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:38.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.933 7f8710ff9700 1 -- 172.21.2.101:0/1916203718 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f870c032780 con 0x7f8714099410 2021-02-10T11:33:38.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.934 7f8713fff700 1 --2- 172.21.2.101:0/1916203718 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86f40186c0 0x7f86f401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:38.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.934 7f8713fff700 1 --2- 172.21.2.101:0/1916203718 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86f40186c0 0x7f86f401abc0 crc :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:38.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.964 7f871a6c0700 1 -- 172.21.2.101:0/1916203718 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87140828f0 con 0x7f8714099410 2021-02-10T11:33:38.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:38.967 7f8710ff9700 1 -- 172.21.2.101:0/1916203718 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f870c0075f0 con 0x7f8714099410 2021-02-10T11:33:39.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.103 7f871a6c0700 1 -- 172.21.2.101:0/1916203718 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8714109a60 con 0x7f8714099410 2021-02-10T11:33:39.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.461 7f8710ff9700 1 -- 172.21.2.101:0/1916203718 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v39) v1 ==== 136+0+0 (secure 0 0 0) 0x7f870c004720 con 0x7f8714099410 2021-02-10T11:33:39.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.471 7f871a6c0700 1 -- 172.21.2.101:0/1916203718 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86f40186c0 msgr2=0x7f86f401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:39.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.471 7f871a6c0700 1 --2- 172.21.2.101:0/1916203718 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86f40186c0 0x7f86f401abc0 crc :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:39.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.471 7f871a6c0700 1 -- 172.21.2.101:0/1916203718 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8714099410 msgr2=0x7f87141a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:39.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.471 7f871a6c0700 1 --2- 172.21.2.101:0/1916203718 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8714099410 0x7f87141a6cc0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7f870c001bf0 tx=0x7f870c00ac50).stop 2021-02-10T11:33:39.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.472 7f871a6c0700 1 -- 172.21.2.101:0/1916203718 shutdown_connections 2021-02-10T11:33:39.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.472 7f871a6c0700 1 --2- 172.21.2.101:0/1916203718 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86f40186c0 0x7f86f401abc0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:39.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.472 7f871a6c0700 1 --2- 172.21.2.101:0/1916203718 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f871418d4d0 0x7f8714098df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:39.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.472 7f871a6c0700 1 --2- 172.21.2.101:0/1916203718 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8714099410 0x7f87141a6cc0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:39.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.472 7f871a6c0700 1 --2- 172.21.2.101:0/1916203718 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87140aa9e0 0x7f871418ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:39.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.472 7f871a6c0700 1 -- 172.21.2.101:0/1916203718 >> 172.21.2.101:0/1916203718 conn(0x7f871419af60 msgr2=0x7f871418c280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:39.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.472 7f871a6c0700 1 -- 172.21.2.101:0/1916203718 shutdown_connections 2021-02-10T11:33:39.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.472 7f871a6c0700 1 -- 172.21.2.101:0/1916203718 wait complete. 2021-02-10T11:33:39.491 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs volume ls 2021-02-10T11:33:39.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.892 7f3229bda700 1 Processor -- start 2021-02-10T11:33:39.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.892 7f3229bda700 1 -- start start 2021-02-10T11:33:39.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.892 7f3229bda700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32240e5070 0x7f32241b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:39.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.892 7f3229bda700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f32240e5470 0x7f322407d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:39.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.892 7f3229bda700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322407ed80 con 0x7f32241b3e40 2021-02-10T11:33:39.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.892 7f3229bda700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3224080310 con 0x7f32240e5470 2021-02-10T11:33:39.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.892 7f3229bda700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3224080500 con 0x7f32240e5070 2021-02-10T11:33:39.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.892 7f3223fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f32241b3e40 0x7f322407dcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52368/0 (socket says 172.21.2.101:52368) 2021-02-10T11:33:39.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.892 7f3223fff700 1 -- 172.21.2.101:0/848950686 learned_addr learned my addr 172.21.2.101:0/848950686 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:39.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f32237fe700 1 --2- 172.21.2.101:0/848950686 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32240e5070 0x7f32241b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:39.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f3222ffd700 1 --2- 172.21.2.101:0/848950686 >> v2:172.21.2.108:3300/0 conn(0x7f32240e5470 0x7f322407d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:39.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f32227fc700 1 -- 172.21.2.101:0/848950686 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202240382 0 0) 0x7f321c001500 con 0x7f32241b3e40 2021-02-10T11:33:39.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f32227fc700 1 -- 172.21.2.101:0/848950686 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3210002fe0 con 0x7f32241b3e40 2021-02-10T11:33:39.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f32227fc700 1 -- 172.21.2.101:0/848950686 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 95500421 0 0) 0x7f321c001270 con 0x7f32241b3e40 2021-02-10T11:33:39.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f32227fc700 1 -- 172.21.2.101:0/848950686 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32240e5070 msgr2=0x7f32241b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:39.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f32227fc700 1 --2- 172.21.2.101:0/848950686 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32240e5070 0x7f32241b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:39.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f32227fc700 1 -- 172.21.2.101:0/848950686 >> v2:172.21.2.108:3300/0 conn(0x7f32240e5470 msgr2=0x7f322407d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:39.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f32227fc700 1 --2- 172.21.2.101:0/848950686 >> v2:172.21.2.108:3300/0 conn(0x7f32240e5470 0x7f322407d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:39.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f32227fc700 1 -- 172.21.2.101:0/848950686 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32240806f0 con 0x7f32241b3e40 2021-02-10T11:33:39.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f32227fc700 1 -- 172.21.2.101:0/848950686 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f321c001b50 con 0x7f32241b3e40 2021-02-10T11:33:39.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f32227fc700 1 -- 172.21.2.101:0/848950686 >> v1:172.21.2.101:6789/0 conn(0x7f32241b3e40 legacy=0x7f322407dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:39.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f32227fc700 1 --2- 172.21.2.101:0/848950686 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3210003a60 0x7f3210005e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:39.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f32227fc700 1 --2- 172.21.2.101:0/848950686 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3210006730 0x7f3210008c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:39.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f32227fc700 1 --2- 172.21.2.101:0/848950686 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32100094f0 0x7f321000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:39.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f32227fc700 1 -- 172.21.2.101:0/848950686 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32100502d0 con 0x7f32100094f0 2021-02-10T11:33:39.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f32227fc700 1 -- 172.21.2.101:0/848950686 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32100504f0 con 0x7f3210006730 2021-02-10T11:33:39.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f32227fc700 1 -- 172.21.2.101:0/848950686 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3210050710 con 0x7f3210003a60 2021-02-10T11:33:39.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f32237fe700 1 --2- 172.21.2.101:0/848950686 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3210003a60 0x7f3210005e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:39.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f3223fff700 1 --2- 172.21.2.101:0/848950686 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32100094f0 0x7f321000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:39.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f3222ffd700 1 --2- 172.21.2.101:0/848950686 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3210006730 0x7f3210008c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:39.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f3223fff700 1 -- 172.21.2.101:0/848950686 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3210003a60 msgr2=0x7f3210005e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:39.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.893 7f3223fff700 1 --2- 172.21.2.101:0/848950686 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3210003a60 0x7f3210005e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:39.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.894 7f3223fff700 1 -- 172.21.2.101:0/848950686 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3210006730 msgr2=0x7f3210008c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:39.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.894 7f3223fff700 1 --2- 172.21.2.101:0/848950686 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3210006730 0x7f3210008c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:39.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.894 7f3223fff700 1 -- 172.21.2.101:0/848950686 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32100509b0 con 0x7f32100094f0 2021-02-10T11:33:39.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.894 7f3223fff700 1 --2- 172.21.2.101:0/848950686 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32100094f0 0x7f321000ba10 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7f321c007cf0 tx=0x7f321c00a310).ready entity=mon.0 client_cookie=50004af64673276a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:39.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.894 7f32227fc700 1 -- 172.21.2.101:0/848950686 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f321c001480 con 0x7f32100094f0 2021-02-10T11:33:39.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.894 7f32227fc700 1 -- 172.21.2.101:0/848950686 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321c0070c0 con 0x7f32100094f0 2021-02-10T11:33:39.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.894 7f32227fc700 1 -- 172.21.2.101:0/848950686 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f321c007520 con 0x7f32100094f0 2021-02-10T11:33:39.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.894 7f3229bda700 1 -- 172.21.2.101:0/848950686 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32100094f0 msgr2=0x7f321000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:39.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.894 7f3229bda700 1 --2- 172.21.2.101:0/848950686 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32100094f0 0x7f321000ba10 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7f321c007cf0 tx=0x7f321c00a310).stop 2021-02-10T11:33:39.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.894 7f3223fff700 1 -- 172.21.2.101:0/848950686 reap_dead start 2021-02-10T11:33:39.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.894 7f3229bda700 1 -- 172.21.2.101:0/848950686 shutdown_connections 2021-02-10T11:33:39.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.894 7f3229bda700 1 -- 172.21.2.101:0/848950686 >> 172.21.2.101:0/848950686 conn(0x7f32240ce690 msgr2=0x7f32240e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:39.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.895 7f3229bda700 1 -- 172.21.2.101:0/848950686 shutdown_connections 2021-02-10T11:33:39.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.895 7f3229bda700 1 -- 172.21.2.101:0/848950686 wait complete. 2021-02-10T11:33:39.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.909 7f3229bda700 1 Processor -- start 2021-02-10T11:33:39.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.909 7f3229bda700 1 -- start start 2021-02-10T11:33:39.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.910 7f3229bda700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f322419f960 0x7f322419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:39.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.910 7f3229bda700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32240a4e30 0x7f32240a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:39.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.910 7f3229bda700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f32240a5b00 0x7f32240a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:39.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.910 7f3229bda700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32240a6520 con 0x7f322419f960 2021-02-10T11:33:39.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.910 7f3229bda700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32240a6710 con 0x7f32240a5b00 2021-02-10T11:33:39.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.910 7f3229bda700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32240a6930 con 0x7f32240a4e30 2021-02-10T11:33:39.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.910 7f32237fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f322419f960 0x7f322419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:39.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.910 7f32237fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f322419f960 0x7f322419fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51886/0 (socket says 172.21.2.101:51886) 2021-02-10T11:33:39.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.910 7f32237fe700 1 -- 172.21.2.101:0/1318235957 learned_addr learned my addr 172.21.2.101:0/1318235957 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:39.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.910 7f32237fe700 1 -- 172.21.2.101:0/1318235957 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32240a4e30 msgr2=0x7f32240a5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:39.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.910 7f32237fe700 1 --2- 172.21.2.101:0/1318235957 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32240a4e30 0x7f32240a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:39.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.910 7f32237fe700 1 -- 172.21.2.101:0/1318235957 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f32240a5b00 msgr2=0x7f32240a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:39.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.910 7f32237fe700 1 --2- 172.21.2.101:0/1318235957 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f32240a5b00 0x7f32240a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:39.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.910 7f32237fe700 1 -- 172.21.2.101:0/1318235957 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32240a6b50 con 0x7f322419f960 2021-02-10T11:33:39.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.910 7f32237fe700 1 --2- 172.21.2.101:0/1318235957 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f322419f960 0x7f322419fd60 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7f320c006f60 tx=0x7f320c00b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:39.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.911 7f321bfff700 1 -- 172.21.2.101:0/1318235957 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f320c00bd60 con 0x7f322419f960 2021-02-10T11:33:39.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.911 7f321bfff700 1 -- 172.21.2.101:0/1318235957 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320c000c90 con 0x7f322419f960 2021-02-10T11:33:39.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.911 7f321bfff700 1 -- 172.21.2.101:0/1318235957 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f320c002220 con 0x7f322419f960 2021-02-10T11:33:39.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.911 7f3229bda700 1 -- 172.21.2.101:0/1318235957 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32240a6b50 con 0x7f322419f960 2021-02-10T11:33:39.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.911 7f3229bda700 1 -- 172.21.2.101:0/1318235957 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3224091aa0 con 0x7f322419f960 2021-02-10T11:33:39.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.911 7f321bfff700 1 -- 172.21.2.101:0/1318235957 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f320c006b00 con 0x7f322419f960 2021-02-10T11:33:39.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.911 7f321bfff700 1 --2- 172.21.2.101:0/1318235957 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32080186c0 0x7f320801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:39.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.911 7f321bfff700 1 -- 172.21.2.101:0/1318235957 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f320c032580 con 0x7f322419f960 2021-02-10T11:33:39.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.912 7f3222ffd700 1 --2- 172.21.2.101:0/1318235957 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32080186c0 0x7f320801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:39.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.912 7f3222ffd700 1 --2- 172.21.2.101:0/1318235957 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32080186c0 0x7f320801abc0 crc :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:39.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.942 7f3229bda700 1 -- 172.21.2.101:0/1318235957 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3224084ad0 con 0x7f322419f960 2021-02-10T11:33:39.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:39.945 7f321bfff700 1 -- 172.21.2.101:0/1318235957 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f320c000940 con 0x7f322419f960 2021-02-10T11:33:40.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.080 7f3229bda700 1 -- 172.21.2.101:0/1318235957 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs volume ls", "target": ["mgr", ""]}) v1 -- 0x7f3224091aa0 con 0x7f32080186c0 2021-02-10T11:33:40.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.081 7f321bfff700 1 -- 172.21.2.101:0/1318235957 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7f32140052e0 con 0x7f32080186c0 2021-02-10T11:33:40.092 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:33:40.092 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:40.092 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "cephfs" 2021-02-10T11:33:40.092 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:40.093 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:33:40.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.082 7f3229bda700 1 -- 172.21.2.101:0/1318235957 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32080186c0 msgr2=0x7f320801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:40.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.082 7f3229bda700 1 --2- 172.21.2.101:0/1318235957 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32080186c0 0x7f320801abc0 crc :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:40.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.082 7f3229bda700 1 -- 172.21.2.101:0/1318235957 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f322419f960 msgr2=0x7f322419fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:40.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.082 7f3229bda700 1 --2- 172.21.2.101:0/1318235957 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f322419f960 0x7f322419fd60 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7f320c006f60 tx=0x7f320c00b970).stop 2021-02-10T11:33:40.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.083 7f3229bda700 1 -- 172.21.2.101:0/1318235957 shutdown_connections 2021-02-10T11:33:40.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.083 7f3229bda700 1 --2- 172.21.2.101:0/1318235957 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32080186c0 0x7f320801abc0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:40.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.083 7f3229bda700 1 --2- 172.21.2.101:0/1318235957 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32240a4e30 0x7f32240a5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:40.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.083 7f3229bda700 1 --2- 172.21.2.101:0/1318235957 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f32240a5b00 0x7f32240a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:40.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.083 7f3229bda700 1 --2- 172.21.2.101:0/1318235957 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f322419f960 0x7f322419fd60 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:40.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.083 7f3229bda700 1 -- 172.21.2.101:0/1318235957 >> 172.21.2.101:0/1318235957 conn(0x7f32240a3580 msgr2=0x7f32240bd230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:40.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.083 7f3229bda700 1 -- 172.21.2.101:0/1318235957 shutdown_connections 2021-02-10T11:33:40.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.083 7f3229bda700 1 -- 172.21.2.101:0/1318235957 wait complete. 2021-02-10T11:33:40.103 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:33:40.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.507 7f102fa23700 1 Processor -- start 2021-02-10T11:33:40.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.507 7f102fa23700 1 -- start start 2021-02-10T11:33:40.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.508 7f102fa23700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10281b0470 0x7f10281b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:40.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.508 7f102fa23700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f10280ef610 0x7f10280efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:40.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.508 7f102fa23700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10280f0b50 con 0x7f10280f0030 2021-02-10T11:33:40.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.508 7f102fa23700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10280e52e0 con 0x7f10280ef610 2021-02-10T11:33:40.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.508 7f102fa23700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10280e5490 con 0x7f10281b0470 2021-02-10T11:33:40.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.508 7f102d83f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10281b0470 0x7f10281b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:40.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.508 7f102d03e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f10280ef610 0x7f10280efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:40.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.508 7f102d83f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10281b0470 0x7f10281b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:44624/0 (socket says 172.21.2.101:44624) 2021-02-10T11:33:40.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.508 7f102d03e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f10280ef610 0x7f10280efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:34150/0 (socket says 172.21.2.101:34150) 2021-02-10T11:33:40.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.508 7f102d83f700 1 -- 172.21.2.101:0/30385493 learned_addr learned my addr 172.21.2.101:0/30385493 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:40.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.508 7f102e040700 1 --1- 172.21.2.101:0/30385493 >> v1:172.21.2.108:6789/0 conn(0x7f10280f0030 0x7f10280f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:45798/0 (socket says 172.21.2.101:45798) 2021-02-10T11:33:40.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.509 7f102d83f700 1 -- 172.21.2.101:0/30385493 >> v2:172.21.2.108:3300/0 conn(0x7f10280ef610 msgr2=0x7f10280efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:40.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.509 7f102c83d700 1 -- 172.21.2.101:0/30385493 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616579763 0 0) 0x7f1020000d30 con 0x7f10280f0030 2021-02-10T11:33:40.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.509 7f102d83f700 1 --2- 172.21.2.101:0/30385493 >> v2:172.21.2.108:3300/0 conn(0x7f10280ef610 0x7f10280efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:40.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.509 7f102d83f700 1 -- 172.21.2.101:0/30385493 >> v1:172.21.2.108:6789/0 conn(0x7f10280f0030 legacy=0x7f10280f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:40.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.509 7f102d83f700 1 -- 172.21.2.101:0/30385493 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10280f0430 con 0x7f10281b0470 2021-02-10T11:33:40.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.509 7f102d03e700 1 --2- 172.21.2.101:0/30385493 >> v2:172.21.2.108:3300/0 conn(0x7f10280ef610 0x7f10280efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:33:40.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.509 7f102d83f700 1 --2- 172.21.2.101:0/30385493 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10281b0470 0x7f10281b0870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f1024006700 tx=0x7f1024009b80).ready entity=mon.2 client_cookie=dd625bb95361e3b2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:40.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.509 7f102c83d700 1 -- 172.21.2.101:0/30385493 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1024006340 con 0x7f10281b0470 2021-02-10T11:33:40.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.509 7f102c83d700 1 -- 172.21.2.101:0/30385493 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102400c0c0 con 0x7f10281b0470 2021-02-10T11:33:40.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.509 7f102c83d700 1 -- 172.21.2.101:0/30385493 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f102400c520 con 0x7f10281b0470 2021-02-10T11:33:40.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.509 7f102fa23700 1 -- 172.21.2.101:0/30385493 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10281b0470 msgr2=0x7f10281b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:40.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.509 7f102fa23700 1 --2- 172.21.2.101:0/30385493 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10281b0470 0x7f10281b0870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f1024006700 tx=0x7f1024009b80).stop 2021-02-10T11:33:40.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.510 7f102fa23700 1 -- 172.21.2.101:0/30385493 shutdown_connections 2021-02-10T11:33:40.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.510 7f102fa23700 1 --2- 172.21.2.101:0/30385493 >> v2:172.21.2.108:3300/0 conn(0x7f10280ef610 0x7f10280efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:40.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.510 7f102fa23700 1 --2- 172.21.2.101:0/30385493 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10281b0470 0x7f10281b0870 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:40.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.510 7f102fa23700 1 -- 172.21.2.101:0/30385493 >> 172.21.2.101:0/30385493 conn(0x7f10280ce3e0 msgr2=0x7f10280e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:40.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.510 7f102fa23700 1 -- 172.21.2.101:0/30385493 shutdown_connections 2021-02-10T11:33:40.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.510 7f102fa23700 1 -- 172.21.2.101:0/30385493 wait complete. 2021-02-10T11:33:40.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.525 7f102fa23700 1 Processor -- start 2021-02-10T11:33:40.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.525 7f102fa23700 1 -- start start 2021-02-10T11:33:40.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.525 7f102fa23700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f102809a220 0x7f10281a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:40.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.525 7f102fa23700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1028099770 0x7f10280ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:40.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.525 7f102fa23700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10280addf0 0x7f10280a9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:40.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.525 7f102fa23700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1028099d20 con 0x7f1028099770 2021-02-10T11:33:40.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.525 7f102fa23700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10280ae1f0 con 0x7f102809a220 2021-02-10T11:33:40.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.525 7f102fa23700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10280ae530 con 0x7f10280addf0 2021-02-10T11:33:40.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.525 7f102d03e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1028099770 0x7f10280ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:40.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.525 7f102d03e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1028099770 0x7f10280ad7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51898/0 (socket says 172.21.2.101:51898) 2021-02-10T11:33:40.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.525 7f102d03e700 1 -- 172.21.2.101:0/3162630538 learned_addr learned my addr 172.21.2.101:0/3162630538 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:40.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.526 7f102d83f700 1 --2- 172.21.2.101:0/3162630538 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f102809a220 0x7f10281a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:40.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.526 7f102e040700 1 --2- 172.21.2.101:0/3162630538 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10280addf0 0x7f10280a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:40.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.526 7f102d03e700 1 -- 172.21.2.101:0/3162630538 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10280addf0 msgr2=0x7f10280a9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:40.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.526 7f102d03e700 1 --2- 172.21.2.101:0/3162630538 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10280addf0 0x7f10280a9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:40.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.526 7f102d03e700 1 -- 172.21.2.101:0/3162630538 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f102809a220 msgr2=0x7f10281a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:40.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.526 7f102d03e700 1 --2- 172.21.2.101:0/3162630538 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f102809a220 0x7f10281a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:40.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.526 7f102d03e700 1 -- 172.21.2.101:0/3162630538 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10281a49b0 con 0x7f1028099770 2021-02-10T11:33:40.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.526 7f102d83f700 1 --2- 172.21.2.101:0/3162630538 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f102809a220 0x7f10281a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:33:40.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.526 7f102e040700 1 --2- 172.21.2.101:0/3162630538 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10280addf0 0x7f10280a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:33:40.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.526 7f102d03e700 1 --2- 172.21.2.101:0/3162630538 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1028099770 0x7f10280ad7d0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7f1018006f40 tx=0x7f101800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:40.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.526 7f101e7fc700 1 -- 172.21.2.101:0/3162630538 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f10180009b0 con 0x7f1028099770 2021-02-10T11:33:40.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.526 7f101e7fc700 1 -- 172.21.2.101:0/3162630538 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101800bd80 con 0x7f1028099770 2021-02-10T11:33:40.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.526 7f102fa23700 1 -- 172.21.2.101:0/3162630538 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10281a49b0 con 0x7f1028099770 2021-02-10T11:33:40.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.526 7f101e7fc700 1 -- 172.21.2.101:0/3162630538 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1018002220 con 0x7f1028099770 2021-02-10T11:33:40.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.527 7f102fa23700 1 -- 172.21.2.101:0/3162630538 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10281a49b0 con 0x7f1028099770 2021-02-10T11:33:40.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.527 7f101e7fc700 1 -- 172.21.2.101:0/3162630538 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1018000c90 con 0x7f1028099770 2021-02-10T11:33:40.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.527 7f101e7fc700 1 --2- 172.21.2.101:0/3162630538 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10100186c0 0x7f101001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:40.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.527 7f101e7fc700 1 -- 172.21.2.101:0/3162630538 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1018032660 con 0x7f1028099770 2021-02-10T11:33:40.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.527 7f102d83f700 1 --2- 172.21.2.101:0/3162630538 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10100186c0 0x7f101001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:40.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.528 7f102d83f700 1 --2- 172.21.2.101:0/3162630538 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10100186c0 0x7f101001abc0 crc :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:40.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.557 7f102fa23700 1 -- 172.21.2.101:0/3162630538 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10280aa5d0 con 0x7f1028099770 2021-02-10T11:33:40.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.560 7f101e7fc700 1 -- 172.21.2.101:0/3162630538 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1018006b50 con 0x7f1028099770 2021-02-10T11:33:40.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.710 7f102fa23700 1 -- 172.21.2.101:0/3162630538 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f10280aa5d0 con 0x7f1028099770 2021-02-10T11:33:40.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.710 7f101e7fc700 1 -- 172.21.2.101:0/3162630538 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v8)=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1018000c90 con 0x7f1028099770 2021-02-10T11:33:40.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.712 7f102fa23700 1 -- 172.21.2.101:0/3162630538 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10100186c0 msgr2=0x7f101001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:40.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.712 7f102fa23700 1 --2- 172.21.2.101:0/3162630538 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10100186c0 0x7f101001abc0 crc :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:40.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.712 7f102fa23700 1 -- 172.21.2.101:0/3162630538 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1028099770 msgr2=0x7f10280ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:40.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.712 7f102fa23700 1 --2- 172.21.2.101:0/3162630538 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1028099770 0x7f10280ad7d0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7f1018006f40 tx=0x7f101800b990).stop 2021-02-10T11:33:40.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.713 7f102fa23700 1 -- 172.21.2.101:0/3162630538 shutdown_connections 2021-02-10T11:33:40.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.713 7f102fa23700 1 --2- 172.21.2.101:0/3162630538 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f10100186c0 0x7f101001abc0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:40.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.713 7f102fa23700 1 --2- 172.21.2.101:0/3162630538 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f10280addf0 0x7f10280a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:40.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.713 7f102fa23700 1 --2- 172.21.2.101:0/3162630538 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1028099770 0x7f10280ad7d0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:40.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.713 7f102fa23700 1 --2- 172.21.2.101:0/3162630538 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f102809a220 0x7f10281a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:40.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.713 7f102fa23700 1 -- 172.21.2.101:0/3162630538 >> 172.21.2.101:0/3162630538 conn(0x7f10280bbf60 msgr2=0x7f1028105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:40.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.713 7f102fa23700 1 -- 172.21.2.101:0/3162630538 shutdown_connections 2021-02-10T11:33:40.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:40.713 7f102fa23700 1 -- 172.21.2.101:0/3162630538 wait complete. 2021-02-10T11:33:40.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:40.732 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:33:40.732 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:33:41.068 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:33:41.069 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:33:41.069 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:33:41.069 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:33:41.069 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:33:41.424 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[118963]: fuse finished with error 0 and tester_r 0 2021-02-10T11:33:41.426 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:41.426 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:33:41.427 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:33:41.760 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:33:41.760 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:33:41.760 INFO:teuthology.orchestra.run:waiting for 900 2021-02-10T11:33:41.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:33:41.761 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:33:42.095 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo mkdir -p ./volumes/group_0000000000586931/subvolume_0000000000130711 2021-02-10T11:33:42.448 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo getfattr --only-values -n ceph.dir.layout.pool . 2021-02-10T11:33:42.797 INFO:teuthology.orchestra.run.gibba001.stdout:cephfs_data 2021-02-10T11:33:42.797 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/group_0000000000586931/subvolume_0000000000130711 2021-02-10T11:33:43.155 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume authorize cephfs subvolume_0000000000130711 alice --group_name group_0000000000586931 --tenant_id tenant_id 2021-02-10T11:33:43.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.567 7fb276682700 1 Processor -- start 2021-02-10T11:33:43.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.567 7fb276682700 1 -- start start 2021-02-10T11:33:43.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.568 7fb276682700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb2700e5010 0x7fb2701b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:43.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.568 7fb276682700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2700e5410 0x7fb2700f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:43.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.568 7fb276682700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2700f5480 con 0x7fb2701b3de0 2021-02-10T11:33:43.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.568 7fb276682700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2700d1750 con 0x7fb2700e5010 2021-02-10T11:33:43.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.568 7fb276682700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2700d1940 con 0x7fb2700e5410 2021-02-10T11:33:43.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.568 7fb26f7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb2701b3de0 0x7fb2700f3dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52400/0 (socket says 172.21.2.101:52400) 2021-02-10T11:33:43.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.568 7fb26f7fe700 1 -- 172.21.2.101:0/1693307898 learned_addr learned my addr 172.21.2.101:0/1693307898 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:43.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.568 7fb26effd700 1 -- 172.21.2.101:0/1693307898 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1053691130 0 0) 0x7fb2640010b0 con 0x7fb2701b3de0 2021-02-10T11:33:43.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.568 7fb26effd700 1 -- 172.21.2.101:0/1693307898 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb258002fe0 con 0x7fb2701b3de0 2021-02-10T11:33:43.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.568 7fb26ffff700 1 --2- 172.21.2.101:0/1693307898 >> v2:172.21.2.108:3300/0 conn(0x7fb2700e5010 0x7fb2701b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:43.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.568 7fb274c9f700 1 --2- 172.21.2.101:0/1693307898 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2700e5410 0x7fb2700f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:43.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.569 7fb26ffff700 1 -- 172.21.2.101:0/1693307898 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2700e5410 msgr2=0x7fb2700f6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:43.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.569 7fb26ffff700 1 --2- 172.21.2.101:0/1693307898 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2700e5410 0x7fb2700f6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:43.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.569 7fb26ffff700 1 -- 172.21.2.101:0/1693307898 >> v1:172.21.2.101:6789/0 conn(0x7fb2701b3de0 legacy=0x7fb2700f3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:43.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.569 7fb26ffff700 1 -- 172.21.2.101:0/1693307898 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2700d1b60 con 0x7fb2700e5010 2021-02-10T11:33:43.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.569 7fb26ffff700 1 --2- 172.21.2.101:0/1693307898 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2700e5010 0x7fb2701b37c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fb260006650 tx=0x7fb260009a80).ready entity=mon.1 client_cookie=f4d46173f34654cd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:43.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.569 7fb26effd700 1 -- 172.21.2.101:0/1693307898 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb26000c0c0 con 0x7fb2700e5010 2021-02-10T11:33:43.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.569 7fb26effd700 1 -- 172.21.2.101:0/1693307898 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26000c340 con 0x7fb2700e5010 2021-02-10T11:33:43.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.569 7fb26effd700 1 -- 172.21.2.101:0/1693307898 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb2600009b0 con 0x7fb2700e5010 2021-02-10T11:33:43.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.569 7fb276682700 1 -- 172.21.2.101:0/1693307898 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2700e5010 msgr2=0x7fb2701b37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:43.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.569 7fb276682700 1 --2- 172.21.2.101:0/1693307898 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2700e5010 0x7fb2701b37c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fb260006650 tx=0x7fb260009a80).stop 2021-02-10T11:33:43.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.570 7fb276682700 1 -- 172.21.2.101:0/1693307898 shutdown_connections 2021-02-10T11:33:43.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.570 7fb276682700 1 --2- 172.21.2.101:0/1693307898 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2700e5410 0x7fb2700f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:43.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.570 7fb276682700 1 --2- 172.21.2.101:0/1693307898 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2700e5010 0x7fb2701b37c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:43.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.570 7fb276682700 1 -- 172.21.2.101:0/1693307898 >> 172.21.2.101:0/1693307898 conn(0x7fb2700ce670 msgr2=0x7fb2700e44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:43.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.570 7fb276682700 1 -- 172.21.2.101:0/1693307898 shutdown_connections 2021-02-10T11:33:43.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.570 7fb276682700 1 -- 172.21.2.101:0/1693307898 wait complete. 2021-02-10T11:33:43.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.585 7fb276682700 1 Processor -- start 2021-02-10T11:33:43.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.585 7fb276682700 1 -- start start 2021-02-10T11:33:43.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.585 7fb276682700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb27018d2c0 0x7fb270106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:43.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.585 7fb276682700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb270104370 0x7fb270104770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:43.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.585 7fb276682700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2701026f0 0x7fb270102af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:43.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.585 7fb276682700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb270107590 con 0x7fb27018d2c0 2021-02-10T11:33:43.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.585 7fb276682700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27018d6c0 con 0x7fb2701026f0 2021-02-10T11:33:43.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.585 7fb276682700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb270104d90 con 0x7fb270104370 2021-02-10T11:33:43.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.585 7fb26ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb27018d2c0 0x7fb270106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:43.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.585 7fb26ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb27018d2c0 0x7fb270106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51912/0 (socket says 172.21.2.101:51912) 2021-02-10T11:33:43.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.585 7fb26ffff700 1 -- 172.21.2.101:0/2861465736 learned_addr learned my addr 172.21.2.101:0/2861465736 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:43.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.586 7fb26ffff700 1 -- 172.21.2.101:0/2861465736 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb270104370 msgr2=0x7fb270104770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:43.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.586 7fb26ffff700 1 --2- 172.21.2.101:0/2861465736 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb270104370 0x7fb270104770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:43.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.586 7fb26ffff700 1 -- 172.21.2.101:0/2861465736 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2701026f0 msgr2=0x7fb270102af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:43.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.586 7fb26ffff700 1 --2- 172.21.2.101:0/2861465736 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2701026f0 0x7fb270102af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:43.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.586 7fb26ffff700 1 -- 172.21.2.101:0/2861465736 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2700b5650 con 0x7fb27018d2c0 2021-02-10T11:33:43.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.586 7fb26ffff700 1 --2- 172.21.2.101:0/2861465736 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb27018d2c0 0x7fb270106f70 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7fb26000b580 tx=0x7fb26000b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:43.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.586 7fb26cff9700 1 -- 172.21.2.101:0/2861465736 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb260000c90 con 0x7fb27018d2c0 2021-02-10T11:33:43.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.586 7fb26cff9700 1 -- 172.21.2.101:0/2861465736 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb260002030 con 0x7fb27018d2c0 2021-02-10T11:33:43.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.586 7fb26cff9700 1 -- 172.21.2.101:0/2861465736 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb260002550 con 0x7fb27018d2c0 2021-02-10T11:33:43.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.586 7fb276682700 1 -- 172.21.2.101:0/2861465736 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2700b5650 con 0x7fb27018d2c0 2021-02-10T11:33:43.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.586 7fb276682700 1 -- 172.21.2.101:0/2861465736 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb270103400 con 0x7fb27018d2c0 2021-02-10T11:33:43.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.587 7fb26cff9700 1 -- 172.21.2.101:0/2861465736 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb260002210 con 0x7fb27018d2c0 2021-02-10T11:33:43.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.587 7fb26cff9700 1 --2- 172.21.2.101:0/2861465736 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2500186c0 0x7fb25001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:43.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.587 7fb26cff9700 1 -- 172.21.2.101:0/2861465736 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb260032590 con 0x7fb27018d2c0 2021-02-10T11:33:43.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.587 7fb26f7fe700 1 --2- 172.21.2.101:0/2861465736 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2500186c0 0x7fb25001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:43.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.587 7fb26f7fe700 1 --2- 172.21.2.101:0/2861465736 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2500186c0 0x7fb25001abc0 crc :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:43.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.617 7fb276682700 1 -- 172.21.2.101:0/2861465736 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2701024c0 con 0x7fb27018d2c0 2021-02-10T11:33:43.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.620 7fb26cff9700 1 -- 172.21.2.101:0/2861465736 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb2600093a0 con 0x7fb27018d2c0 2021-02-10T11:33:43.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.745 7fb276682700 1 -- 172.21.2.101:0/2861465736 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000130711", "target": ["mgr", ""], "tenant_id": "tenant_id", "auth_id": "alice", "group_name": "group_0000000000586931"}) v1 -- 0x7fb270103400 con 0x7fb2500186c0 2021-02-10T11:33:43.760 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:33:43.749 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:33:43.760 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:33:43.749 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:33:43.760 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:33:43.749 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:33:43.760 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:33:43.749 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:33:43.760 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:33:43.749 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:33:43.788 INFO:tasks.ceph.mgr.y.gibba001.stderr:Creating meta for ID alice with tenant tenant_id 2021-02-10T11:33:43.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.803 7fb26cff9700 1 -- 172.21.2.101:0/2861465736 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7fb264006e30 con 0x7fb2500186c0 2021-02-10T11:33:43.819 INFO:teuthology.orchestra.run.gibba001.stdout:AQCXxCNgV7T2LhAAoiEuMPF+Yh/GnfJFmgWXPQ== 2021-02-10T11:33:43.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.810 7fb276682700 1 -- 172.21.2.101:0/2861465736 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2500186c0 msgr2=0x7fb25001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:43.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.810 7fb276682700 1 --2- 172.21.2.101:0/2861465736 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2500186c0 0x7fb25001abc0 crc :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:43.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.810 7fb276682700 1 -- 172.21.2.101:0/2861465736 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb27018d2c0 msgr2=0x7fb270106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:43.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.810 7fb276682700 1 --2- 172.21.2.101:0/2861465736 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb27018d2c0 0x7fb270106f70 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7fb26000b580 tx=0x7fb26000b920).stop 2021-02-10T11:33:43.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.810 7fb276682700 1 -- 172.21.2.101:0/2861465736 shutdown_connections 2021-02-10T11:33:43.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.810 7fb276682700 1 --2- 172.21.2.101:0/2861465736 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2500186c0 0x7fb25001abc0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:43.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.810 7fb276682700 1 --2- 172.21.2.101:0/2861465736 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb270104370 0x7fb270104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:43.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.810 7fb276682700 1 --2- 172.21.2.101:0/2861465736 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2701026f0 0x7fb270102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:43.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.810 7fb276682700 1 --2- 172.21.2.101:0/2861465736 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb27018d2c0 0x7fb270106f70 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:43.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.810 7fb276682700 1 -- 172.21.2.101:0/2861465736 >> 172.21.2.101:0/2861465736 conn(0x7fb2700913d0 msgr2=0x7fb27019a820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:43.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.810 7fb276682700 1 -- 172.21.2.101:0/2861465736 shutdown_connections 2021-02-10T11:33:43.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:43.810 7fb276682700 1 -- 172.21.2.101:0/2861465736 wait complete. 2021-02-10T11:33:43.831 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:33:44.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.230 7f3c92f5c700 1 Processor -- start 2021-02-10T11:33:44.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.230 7f3c92f5c700 1 -- start start 2021-02-10T11:33:44.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.230 7f3c92f5c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3c8c0e5070 0x7f3c8c1b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:44.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.230 7f3c92f5c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c8c07ed80 con 0x7f3c8c0e5470 2021-02-10T11:33:44.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.230 7f3c92f5c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c8c07e670 con 0x7f3c8c1b3e40 2021-02-10T11:33:44.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.230 7f3c92f5c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c8c07e890 con 0x7f3c8c0e5070 2021-02-10T11:33:44.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.231 7f3c90d78700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3c8c0e5070 0x7f3c8c1b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:44.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.231 7f3c91579700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f3c8c0e5470 0x7f3c8c07de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52426/0 (socket says 172.21.2.101:52426) 2021-02-10T11:33:44.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.231 7f3c90d78700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3c8c0e5070 0x7f3c8c1b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51932/0 (socket says 172.21.2.101:51932) 2021-02-10T11:33:44.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.231 7f3c91579700 1 -- 172.21.2.101:0/61527353 learned_addr learned my addr 172.21.2.101:0/61527353 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:44.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.231 7f3c837fe700 1 -- 172.21.2.101:0/61527353 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953923888 0 0) 0x7f3c84001500 con 0x7f3c8c0e5470 2021-02-10T11:33:44.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.231 7f3c837fe700 1 -- 172.21.2.101:0/61527353 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c74002fe0 con 0x7f3c8c0e5470 2021-02-10T11:33:44.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.231 7f3c90d78700 1 -- 172.21.2.101:0/61527353 >> v1:172.21.2.108:6789/0 conn(0x7f3c8c1b3e40 legacy=0x7f3c8c07d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:44.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.231 7f3c90d78700 1 -- 172.21.2.101:0/61527353 >> v1:172.21.2.101:6789/0 conn(0x7f3c8c0e5470 legacy=0x7f3c8c07de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:44.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.231 7f3c90d78700 1 -- 172.21.2.101:0/61527353 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8c07eab0 con 0x7f3c8c0e5070 2021-02-10T11:33:44.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.231 7f3c90d78700 1 --2- 172.21.2.101:0/61527353 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c8c0e5070 0x7f3c8c1b3820 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7f3c7c006650 tx=0x7f3c7c009a80).ready entity=mon.0 client_cookie=fd7b6ee3ef70900d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:44.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.232 7f3c837fe700 1 -- 172.21.2.101:0/61527353 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3c7c0009b0 con 0x7f3c8c0e5070 2021-02-10T11:33:44.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.232 7f3c837fe700 1 -- 172.21.2.101:0/61527353 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7c00c0c0 con 0x7f3c8c0e5070 2021-02-10T11:33:44.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.232 7f3c837fe700 1 -- 172.21.2.101:0/61527353 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3c7c00c520 con 0x7f3c8c0e5070 2021-02-10T11:33:44.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.232 7f3c92f5c700 1 -- 172.21.2.101:0/61527353 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c8c0e5070 msgr2=0x7f3c8c1b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:44.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.232 7f3c92f5c700 1 --2- 172.21.2.101:0/61527353 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c8c0e5070 0x7f3c8c1b3820 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7f3c7c006650 tx=0x7f3c7c009a80).stop 2021-02-10T11:33:44.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.232 7f3c92f5c700 1 -- 172.21.2.101:0/61527353 shutdown_connections 2021-02-10T11:33:44.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.232 7f3c92f5c700 1 --2- 172.21.2.101:0/61527353 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c8c0e5070 0x7f3c8c1b3820 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:44.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.232 7f3c92f5c700 1 -- 172.21.2.101:0/61527353 >> 172.21.2.101:0/61527353 conn(0x7f3c8c0ce690 msgr2=0x7f3c8c0e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:44.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.232 7f3c92f5c700 1 -- 172.21.2.101:0/61527353 shutdown_connections 2021-02-10T11:33:44.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.232 7f3c92f5c700 1 -- 172.21.2.101:0/61527353 wait complete. 2021-02-10T11:33:44.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.247 7f3c92f5c700 1 Processor -- start 2021-02-10T11:33:44.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.247 7f3c92f5c700 1 -- start start 2021-02-10T11:33:44.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.247 7f3c92f5c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c8c08c620 0x7f3c8c1910b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:44.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.247 7f3c92f5c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c8c08baa0 0x7f3c8c08bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:44.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.247 7f3c92f5c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c8c18f3c0 0x7f3c8c18f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:44.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.247 7f3c92f5c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c1916d0 con 0x7f3c8c18f3c0 2021-02-10T11:33:44.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.247 7f3c92f5c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c18fde0 con 0x7f3c8c08c620 2021-02-10T11:33:44.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.247 7f3c92f5c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c190000 con 0x7f3c8c08baa0 2021-02-10T11:33:44.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.247 7f3c91579700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c8c18f3c0 0x7f3c8c18f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:44.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.247 7f3c91579700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c8c18f3c0 0x7f3c8c18f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51936/0 (socket says 172.21.2.101:51936) 2021-02-10T11:33:44.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.247 7f3c91579700 1 -- 172.21.2.101:0/4029027003 learned_addr learned my addr 172.21.2.101:0/4029027003 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:44.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.248 7f3c91579700 1 -- 172.21.2.101:0/4029027003 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c8c08baa0 msgr2=0x7f3c8c08bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:44.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.248 7f3c91579700 1 --2- 172.21.2.101:0/4029027003 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c8c08baa0 0x7f3c8c08bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:44.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.248 7f3c91579700 1 -- 172.21.2.101:0/4029027003 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c8c08c620 msgr2=0x7f3c8c1910b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:44.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.248 7f3c91579700 1 --2- 172.21.2.101:0/4029027003 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c8c08c620 0x7f3c8c1910b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:44.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.248 7f3c91579700 1 -- 172.21.2.101:0/4029027003 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8c08caf0 con 0x7f3c8c18f3c0 2021-02-10T11:33:44.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.248 7f3c91579700 1 --2- 172.21.2.101:0/4029027003 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c8c18f3c0 0x7f3c8c18f7c0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 rx=0x7f3c8400a8f0 tx=0x7f3c8400ac90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:44.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.248 7f3c817fa700 1 -- 172.21.2.101:0/4029027003 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3c84007d10 con 0x7f3c8c18f3c0 2021-02-10T11:33:44.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.248 7f3c817fa700 1 -- 172.21.2.101:0/4029027003 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c840070d0 con 0x7f3c8c18f3c0 2021-02-10T11:33:44.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.248 7f3c92f5c700 1 -- 172.21.2.101:0/4029027003 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c8c08caf0 con 0x7f3c8c18f3c0 2021-02-10T11:33:44.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.248 7f3c92f5c700 1 -- 172.21.2.101:0/4029027003 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c8c08caf0 con 0x7f3c8c18f3c0 2021-02-10T11:33:44.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.248 7f3c817fa700 1 -- 172.21.2.101:0/4029027003 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3c84004720 con 0x7f3c8c18f3c0 2021-02-10T11:33:44.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.249 7f3c817fa700 1 -- 172.21.2.101:0/4029027003 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3c840072b0 con 0x7f3c8c18f3c0 2021-02-10T11:33:44.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.249 7f3c817fa700 1 --2- 172.21.2.101:0/4029027003 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3c6c0186d0 0x7f3c6c01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:44.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.249 7f3c817fa700 1 -- 172.21.2.101:0/4029027003 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3c84032b10 con 0x7f3c8c18f3c0 2021-02-10T11:33:44.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.249 7f3c90d78700 1 --2- 172.21.2.101:0/4029027003 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3c6c0186d0 0x7f3c6c01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:44.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.249 7f3c90d78700 1 --2- 172.21.2.101:0/4029027003 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3c6c0186d0 0x7f3c6c01abd0 crc :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:44.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.280 7f3c92f5c700 1 -- 172.21.2.101:0/4029027003 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c8c07ed80 con 0x7f3c8c18f3c0 2021-02-10T11:33:44.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.283 7f3c817fa700 1 -- 172.21.2.101:0/4029027003 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3c84000f90 con 0x7f3c8c18f3c0 2021-02-10T11:33:44.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.415 7f3c92f5c700 1 -- 172.21.2.101:0/4029027003 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3c8c07ed80 con 0x7f3c8c18f3c0 2021-02-10T11:33:44.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.415 7f3c817fa700 1 -- 172.21.2.101:0/4029027003 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v35) v1 ==== 80+0+4855 (secure 0 0 0) 0x7f3c84006fb0 con 0x7f3c8c18f3c0 2021-02-10T11:33:44.427 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:44.427 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:33:44.427 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:33:44.427 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:44.427 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:33:44.427 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:33:44.427 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:44.427 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:33:44.428 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:33:44.428 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:33:44.428 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:44.428 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:44.428 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:44.428 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:44.428 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:33:44.428 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:33:44.428 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:44.428 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:33:44.428 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:33:44.428 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:33:44.429 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:44.429 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:44.429 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:44.429 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:44.429 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:33:44.429 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:33:44.429 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:44.429 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:33:44.429 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:33:44.429 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:44.429 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:44.429 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:44.429 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:44.430 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:33:44.430 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:33:44.430 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:44.430 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:33:44.430 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:33:44.430 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:44.430 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:44.430 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:44.430 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:44.430 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:33:44.430 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:33:44.430 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:44.430 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:33:44.431 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:33:44.431 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:44.431 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:44.431 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:44.431 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:44.431 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:33:44.431 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:33:44.431 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:44.431 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:33:44.431 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:33:44.431 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:44.431 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:44.431 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:44.432 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:44.432 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:33:44.432 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:33:44.432 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:44.432 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:33:44.432 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:33:44.432 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:44.432 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:44.432 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:44.432 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:44.432 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:33:44.432 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:33:44.432 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:44.433 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:33:44.433 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:33:44.433 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:44.433 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:44.433 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:44.433 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:44.433 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:33:44.433 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:33:44.433 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:44.433 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:33:44.433 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:33:44.433 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:44.433 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:44.433 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:44.434 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:44.434 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:33:44.434 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:33:44.434 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:44.434 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:33:44.434 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:33:44.434 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:44.434 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:44.434 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:44.434 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:44.434 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:33:44.434 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:33:44.435 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:44.435 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:33:44.435 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:33:44.435 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:33:44.435 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:44.435 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:44.435 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:44.435 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:33:44.435 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:33:44.435 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:44.435 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:33:44.435 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:33:44.435 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:33:44.435 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:44.436 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:44.436 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:44.436 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:33:44.436 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:33:44.436 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:44.436 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:33:44.436 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:33:44.436 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:33:44.436 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:44.436 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:44.436 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:44.436 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:33:44.436 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:33:44.437 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:44.437 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:33:44.437 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:33:44.437 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:33:44.437 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:44.437 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:44.437 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:44.437 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:33:44.437 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:33:44.437 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:44.437 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:33:44.437 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:33:44.437 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:33:44.438 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:44.438 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:44.438 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:44.438 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:44.438 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.alice", 2021-02-10T11:33:44.438 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQCXxCNgV7T2LhAAoiEuMPF+Yh/GnfJFmgWXPQ==", 2021-02-10T11:33:44.438 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:44.438 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow rw path=/volumes/group_0000000000586931/subvolume_0000000000130711", 2021-02-10T11:33:44.438 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:33:44.438 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:33:44.438 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:44.438 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:44.438 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:44.439 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:33:44.439 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:33:44.439 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:44.439 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:33:44.439 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:33:44.439 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:44.439 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:44.439 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:44.439 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:33:44.439 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:33:44.439 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:33:44.439 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:33:44.439 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:33:44.440 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:33:44.440 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:33:44.440 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:44.440 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:44.440 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:33:44.440 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:33:44.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.417 7f3c92f5c700 1 -- 172.21.2.101:0/4029027003 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3c6c0186d0 msgr2=0x7f3c6c01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:44.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.417 7f3c92f5c700 1 --2- 172.21.2.101:0/4029027003 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3c6c0186d0 0x7f3c6c01abd0 crc :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:44.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.417 7f3c92f5c700 1 -- 172.21.2.101:0/4029027003 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c8c18f3c0 msgr2=0x7f3c8c18f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:44.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.417 7f3c92f5c700 1 --2- 172.21.2.101:0/4029027003 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c8c18f3c0 0x7f3c8c18f7c0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 rx=0x7f3c8400a8f0 tx=0x7f3c8400ac90).stop 2021-02-10T11:33:44.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.417 7f3c92f5c700 1 -- 172.21.2.101:0/4029027003 shutdown_connections 2021-02-10T11:33:44.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.417 7f3c92f5c700 1 --2- 172.21.2.101:0/4029027003 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3c6c0186d0 0x7f3c6c01abd0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:44.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.417 7f3c92f5c700 1 --2- 172.21.2.101:0/4029027003 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c8c08baa0 0x7f3c8c08bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:44.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.417 7f3c92f5c700 1 --2- 172.21.2.101:0/4029027003 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c8c18f3c0 0x7f3c8c18f7c0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:44.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.417 7f3c92f5c700 1 --2- 172.21.2.101:0/4029027003 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c8c08c620 0x7f3c8c1910b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:44.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.417 7f3c92f5c700 1 -- 172.21.2.101:0/4029027003 >> 172.21.2.101:0/4029027003 conn(0x7f3c8c1909e0 msgr2=0x7f3c8c1b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:44.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.417 7f3c92f5c700 1 -- 172.21.2.101:0/4029027003 shutdown_connections 2021-02-10T11:33:44.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:44.417 7f3c92f5c700 1 -- 172.21.2.101:0/4029027003 wait complete. 2021-02-10T11:33:44.442 DEBUG:teuthology.orchestra.run.gibba001:> set -ex 2021-02-10T11:33:44.442 DEBUG:teuthology.orchestra.run.gibba001:> sudo dd of=/etc/ceph/ceph.client.alice.keyring 2021-02-10T11:33:44.785 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set client.alice debug_client 20 2021-02-10T11:33:45.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.188 7f21400bd700 1 Processor -- start 2021-02-10T11:33:45.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.188 7f21400bd700 1 -- start start 2021-02-10T11:33:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.189 7f21400bd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21381b0470 0x7f21381b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.189 7f21400bd700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f21380ef610 0x7f21380efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.189 7f21400bd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f21380f0030 0x7f21380f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.189 7f21400bd700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21380e5ea0 con 0x7f21380f0030 2021-02-10T11:33:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.189 7f21400bd700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21380f0b50 con 0x7f21380ef610 2021-02-10T11:33:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.189 7f21400bd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21380e52e0 con 0x7f21381b0470 2021-02-10T11:33:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.189 7f213e6da700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f21380f0030 0x7f21380f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.189 7f213e6da700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f21380f0030 0x7f21380f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51944/0 (socket says 172.21.2.101:51944) 2021-02-10T11:33:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.189 7f213e6da700 1 -- 172.21.2.101:0/2049678672 learned_addr learned my addr 172.21.2.101:0/2049678672 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.189 7f213e6da700 1 -- 172.21.2.101:0/2049678672 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21381b0470 msgr2=0x7f21381b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:33:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.189 7f213e6da700 1 --2- 172.21.2.101:0/2049678672 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21381b0470 0x7f21381b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.189 7f213e6da700 1 -- 172.21.2.101:0/2049678672 >> v2:172.21.2.108:3300/0 conn(0x7f21380ef610 msgr2=0x7f21380efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:33:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.189 7f213e6da700 1 --2- 172.21.2.101:0/2049678672 >> v2:172.21.2.108:3300/0 conn(0x7f21380ef610 0x7f21380efa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.189 7f213e6da700 1 -- 172.21.2.101:0/2049678672 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21380f10d0 con 0x7f21380f0030 2021-02-10T11:33:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.189 7f213e6da700 1 --2- 172.21.2.101:0/2049678672 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21380f0030 0x7f21380f0430 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7f2130006f00 tx=0x7f2130009f10).ready entity=mon.0 client_cookie=f90ebb150801a4ad server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.190 7f213ced7700 1 -- 172.21.2.101:0/2049678672 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f213000b3b0 con 0x7f21380f0030 2021-02-10T11:33:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.190 7f213ced7700 1 -- 172.21.2.101:0/2049678672 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213000b730 con 0x7f21380f0030 2021-02-10T11:33:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.190 7f213ced7700 1 -- 172.21.2.101:0/2049678672 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f213000bc10 con 0x7f21380f0030 2021-02-10T11:33:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.190 7f21400bd700 1 -- 172.21.2.101:0/2049678672 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21380f0030 msgr2=0x7f21380f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.190 7f21400bd700 1 --2- 172.21.2.101:0/2049678672 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21380f0030 0x7f21380f0430 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7f2130006f00 tx=0x7f2130009f10).stop 2021-02-10T11:33:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.190 7f21400bd700 1 -- 172.21.2.101:0/2049678672 shutdown_connections 2021-02-10T11:33:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.190 7f21400bd700 1 --2- 172.21.2.101:0/2049678672 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21380f0030 0x7f21380f0430 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.190 7f21400bd700 1 --2- 172.21.2.101:0/2049678672 >> v2:172.21.2.108:3300/0 conn(0x7f21380ef610 0x7f21380efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.190 7f21400bd700 1 --2- 172.21.2.101:0/2049678672 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21381b0470 0x7f21381b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.190 7f21400bd700 1 -- 172.21.2.101:0/2049678672 >> 172.21.2.101:0/2049678672 conn(0x7f21380ce3e0 msgr2=0x7f21380e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:45.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.190 7f21400bd700 1 -- 172.21.2.101:0/2049678672 shutdown_connections 2021-02-10T11:33:45.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.191 7f21400bd700 1 -- 172.21.2.101:0/2049678672 wait complete. 2021-02-10T11:33:45.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.205 7f21400bd700 1 Processor -- start 2021-02-10T11:33:45.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.205 7f21400bd700 1 -- start start 2021-02-10T11:33:45.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.205 7f21400bd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f213809e5d0 0x7f21380c2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:45.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.205 7f21400bd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f213809e9d0 0x7f21380c3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:45.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.205 7f21400bd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21380a9c80 0x7f21380aa080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:45.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.205 7f21400bd700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2138095040 con 0x7f213809e5d0 2021-02-10T11:33:45.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.205 7f21400bd700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21380c3820 con 0x7f213809e9d0 2021-02-10T11:33:45.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.205 7f21400bd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21380aa6a0 con 0x7f21380a9c80 2021-02-10T11:33:45.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.205 7f213ded9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f213809e5d0 0x7f21380c2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:45.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.205 7f213ded9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f213809e5d0 0x7f21380c2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51948/0 (socket says 172.21.2.101:51948) 2021-02-10T11:33:45.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.205 7f213ded9700 1 -- 172.21.2.101:0/1662954050 learned_addr learned my addr 172.21.2.101:0/1662954050 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:45.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.206 7f213ded9700 1 -- 172.21.2.101:0/1662954050 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21380a9c80 msgr2=0x7f21380aa080 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:45.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.206 7f213ded9700 1 --2- 172.21.2.101:0/1662954050 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21380a9c80 0x7f21380aa080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:45.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.206 7f213ded9700 1 -- 172.21.2.101:0/1662954050 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f213809e9d0 msgr2=0x7f21380c3200 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:45.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.206 7f213ded9700 1 --2- 172.21.2.101:0/1662954050 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f213809e9d0 0x7f21380c3200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:45.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.206 7f213ded9700 1 -- 172.21.2.101:0/1662954050 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21380aa8c0 con 0x7f213809e5d0 2021-02-10T11:33:45.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.206 7f213ded9700 1 --2- 172.21.2.101:0/1662954050 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f213809e5d0 0x7f21380c2be0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7f213400b5c0 tx=0x7f213400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:45.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.206 7f212e7fc700 1 -- 172.21.2.101:0/1662954050 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f213400bd50 con 0x7f213809e5d0 2021-02-10T11:33:45.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.206 7f212e7fc700 1 -- 172.21.2.101:0/1662954050 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2134006030 con 0x7f213809e5d0 2021-02-10T11:33:45.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.206 7f212e7fc700 1 -- 172.21.2.101:0/1662954050 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21340054d0 con 0x7f213809e5d0 2021-02-10T11:33:45.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.206 7f21400bd700 1 -- 172.21.2.101:0/1662954050 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21380aa8c0 con 0x7f213809e5d0 2021-02-10T11:33:45.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.206 7f21400bd700 1 -- 172.21.2.101:0/1662954050 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21380aa8c0 con 0x7f213809e5d0 2021-02-10T11:33:45.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.207 7f212e7fc700 1 -- 172.21.2.101:0/1662954050 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2134006210 con 0x7f213809e5d0 2021-02-10T11:33:45.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.207 7f212e7fc700 1 --2- 172.21.2.101:0/1662954050 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21200186c0 0x7f212001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:45.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.207 7f212e7fc700 1 -- 172.21.2.101:0/1662954050 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2134032590 con 0x7f213809e5d0 2021-02-10T11:33:45.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.207 7f213d6d8700 1 --2- 172.21.2.101:0/1662954050 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21200186c0 0x7f212001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:45.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.207 7f213d6d8700 1 --2- 172.21.2.101:0/1662954050 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21200186c0 0x7f212001abc0 crc :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:45.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.239 7f21400bd700 1 -- 172.21.2.101:0/1662954050 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21380edd10 con 0x7f213809e5d0 2021-02-10T11:33:45.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.241 7f212e7fc700 1 -- 172.21.2.101:0/1662954050 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2134000c40 con 0x7f213809e5d0 2021-02-10T11:33:45.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.390 7f21400bd700 1 -- 172.21.2.101:0/1662954050 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) v1 -- 0x7f21380e52e0 con 0x7f213809e5d0 2021-02-10T11:33:45.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.393 7f212e7fc700 1 -- 172.21.2.101:0/1662954050 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v9)=0 v9) v1 ==== 120+0+0 (secure 0 0 0) 0x7f2134006210 con 0x7f213809e5d0 2021-02-10T11:33:45.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.399 7f21400bd700 1 -- 172.21.2.101:0/1662954050 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21200186c0 msgr2=0x7f212001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:45.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.399 7f21400bd700 1 --2- 172.21.2.101:0/1662954050 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21200186c0 0x7f212001abc0 crc :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:45.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.399 7f21400bd700 1 -- 172.21.2.101:0/1662954050 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f213809e5d0 msgr2=0x7f21380c2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:45.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.399 7f21400bd700 1 --2- 172.21.2.101:0/1662954050 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f213809e5d0 0x7f21380c2be0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7f213400b5c0 tx=0x7f213400b960).stop 2021-02-10T11:33:45.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.399 7f21400bd700 1 -- 172.21.2.101:0/1662954050 shutdown_connections 2021-02-10T11:33:45.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.399 7f21400bd700 1 --2- 172.21.2.101:0/1662954050 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21200186c0 0x7f212001abc0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:45.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.399 7f21400bd700 1 --2- 172.21.2.101:0/1662954050 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21380a9c80 0x7f21380aa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:45.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.399 7f21400bd700 1 --2- 172.21.2.101:0/1662954050 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f213809e9d0 0x7f21380c3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:45.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.399 7f21400bd700 1 --2- 172.21.2.101:0/1662954050 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f213809e5d0 0x7f21380c2be0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:45.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.399 7f21400bd700 1 -- 172.21.2.101:0/1662954050 >> 172.21.2.101:0/1662954050 conn(0x7f213809f630 msgr2=0x7f21380a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:45.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.399 7f21400bd700 1 -- 172.21.2.101:0/1662954050 shutdown_connections 2021-02-10T11:33:45.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.400 7f21400bd700 1 -- 172.21.2.101:0/1662954050 wait complete. 2021-02-10T11:33:45.419 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set client.alice debug_objecter 20 2021-02-10T11:33:45.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.821 7fd1f713c700 1 Processor -- start 2021-02-10T11:33:45.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.821 7fd1f713c700 1 -- start start 2021-02-10T11:33:45.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.822 7fd1f713c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd1f01b0870 0x7fd1f00f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:45.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.822 7fd1f713c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f01b0c70 con 0x7fd1f01b0470 2021-02-10T11:33:45.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.822 7fd1f713c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f00f0b50 con 0x7fd1f00efe00 2021-02-10T11:33:45.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.822 7fd1f713c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1f00e5ea0 con 0x7fd1f01b0870 2021-02-10T11:33:45.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.822 7fd1f4f58700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd1f01b0470 0x7fd1f00ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52450/0 (socket says 172.21.2.101:52450) 2021-02-10T11:33:45.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.822 7fd1f4f58700 1 -- 172.21.2.101:0/2767853799 learned_addr learned my addr 172.21.2.101:0/2767853799 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:45.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.822 7fd1ef7fe700 1 -- 172.21.2.101:0/2767853799 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2286892445 0 0) 0x7fd1d8001180 con 0x7fd1f01b0470 2021-02-10T11:33:45.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.822 7fd1ef7fe700 1 -- 172.21.2.101:0/2767853799 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1dc002fe0 con 0x7fd1f01b0470 2021-02-10T11:33:45.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1effff700 1 --2- 172.21.2.101:0/2767853799 >> v2:172.21.2.108:3300/0 conn(0x7fd1f01b0870 0x7fd1f00f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:45.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1ef7fe700 1 -- 172.21.2.101:0/2767853799 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 628380622 0 0) 0x7fd1d8001180 con 0x7fd1f01b0470 2021-02-10T11:33:45.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1ef7fe700 1 -- 172.21.2.101:0/2767853799 >> v2:172.21.2.108:3300/0 conn(0x7fd1f01b0870 msgr2=0x7fd1f00f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:45.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1ef7fe700 1 --2- 172.21.2.101:0/2767853799 >> v2:172.21.2.108:3300/0 conn(0x7fd1f01b0870 0x7fd1f00f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:45.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1ef7fe700 1 -- 172.21.2.101:0/2767853799 >> v1:172.21.2.108:6789/0 conn(0x7fd1f00efe00 legacy=0x7fd1f00f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:45.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1ef7fe700 1 -- 172.21.2.101:0/2767853799 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f00e52e0 con 0x7fd1f01b0470 2021-02-10T11:33:45.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1ef7fe700 1 -- 172.21.2.101:0/2767853799 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fd1d8001680 con 0x7fd1f01b0470 2021-02-10T11:33:45.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1ef7fe700 1 -- 172.21.2.101:0/2767853799 >> v1:172.21.2.101:6789/0 conn(0x7fd1f01b0470 legacy=0x7fd1f00ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:45.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1ef7fe700 1 --2- 172.21.2.101:0/2767853799 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1dc003af0 0x7fd1dc005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:45.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1ef7fe700 1 --2- 172.21.2.101:0/2767853799 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1dc0067a0 0x7fd1dc008ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:45.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1ef7fe700 1 --2- 172.21.2.101:0/2767853799 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1dc0095a0 0x7fd1dc00baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:45.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1ef7fe700 1 -- 172.21.2.101:0/2767853799 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc00c2f0 con 0x7fd1dc0095a0 2021-02-10T11:33:45.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1ef7fe700 1 -- 172.21.2.101:0/2767853799 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc00c510 con 0x7fd1dc0067a0 2021-02-10T11:33:45.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1ef7fe700 1 -- 172.21.2.101:0/2767853799 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc00c730 con 0x7fd1dc003af0 2021-02-10T11:33:45.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1f5759700 1 --2- 172.21.2.101:0/2767853799 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1dc0095a0 0x7fd1dc00baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:45.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1f4f58700 1 --2- 172.21.2.101:0/2767853799 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1dc003af0 0x7fd1dc005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:45.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1effff700 1 --2- 172.21.2.101:0/2767853799 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1dc0067a0 0x7fd1dc008ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:45.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1f5759700 1 -- 172.21.2.101:0/2767853799 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1dc003af0 msgr2=0x7fd1dc005f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:45.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1f5759700 1 --2- 172.21.2.101:0/2767853799 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1dc003af0 0x7fd1dc005f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:45.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1f5759700 1 -- 172.21.2.101:0/2767853799 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1dc0067a0 msgr2=0x7fd1dc008ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:45.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1f5759700 1 --2- 172.21.2.101:0/2767853799 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1dc0067a0 0x7fd1dc008ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:45.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1f5759700 1 -- 172.21.2.101:0/2767853799 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc00c9d0 con 0x7fd1dc0095a0 2021-02-10T11:33:45.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1effff700 1 --2- 172.21.2.101:0/2767853799 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1dc0067a0 0x7fd1dc008ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:33:45.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1f5759700 1 -- 172.21.2.101:0/2767853799 reap_dead start 2021-02-10T11:33:45.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1f5759700 1 --2- 172.21.2.101:0/2767853799 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1dc0095a0 0x7fd1dc00baa0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7fd1e804d670 tx=0x7fd1e804de50).ready entity=mon.0 client_cookie=7794dab1bcf2105c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:45.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1ef7fe700 1 -- 172.21.2.101:0/2767853799 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd1e8006a30 con 0x7fd1dc0095a0 2021-02-10T11:33:45.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1ef7fe700 1 -- 172.21.2.101:0/2767853799 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e804f300 con 0x7fd1dc0095a0 2021-02-10T11:33:45.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1ef7fe700 1 -- 172.21.2.101:0/2767853799 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd1e8006a30 con 0x7fd1dc0095a0 2021-02-10T11:33:45.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.823 7fd1f713c700 1 -- 172.21.2.101:0/2767853799 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1dc0095a0 msgr2=0x7fd1dc00baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:45.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.824 7fd1f713c700 1 --2- 172.21.2.101:0/2767853799 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1dc0095a0 0x7fd1dc00baa0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7fd1e804d670 tx=0x7fd1e804de50).stop 2021-02-10T11:33:45.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.824 7fd1f713c700 1 -- 172.21.2.101:0/2767853799 shutdown_connections 2021-02-10T11:33:45.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.824 7fd1f713c700 1 --2- 172.21.2.101:0/2767853799 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1dc0095a0 0x7fd1dc00baa0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:45.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.824 7fd1f713c700 1 -- 172.21.2.101:0/2767853799 >> 172.21.2.101:0/2767853799 conn(0x7fd1f00ce3e0 msgr2=0x7fd1f00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:45.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.824 7fd1f713c700 1 -- 172.21.2.101:0/2767853799 shutdown_connections 2021-02-10T11:33:45.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.824 7fd1f713c700 1 -- 172.21.2.101:0/2767853799 wait complete. 2021-02-10T11:33:45.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.839 7fd1f713c700 1 Processor -- start 2021-02-10T11:33:45.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.839 7fd1f713c700 1 -- start start 2021-02-10T11:33:45.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.839 7fd1f713c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1f009e560 0x7fd1f00a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:45.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.839 7fd1f713c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1f009e960 0x7fd1f00a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:45.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.839 7fd1f713c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1f00a0d70 0x7fd1f00a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:45.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.839 7fd1f713c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f00c38f0 con 0x7fd1f009e960 2021-02-10T11:33:45.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.839 7fd1f713c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f00a1170 con 0x7fd1f00a0d70 2021-02-10T11:33:45.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.839 7fd1f713c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f00a1390 con 0x7fd1f009e560 2021-02-10T11:33:45.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.839 7fd1effff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1f009e960 0x7fd1f00a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:45.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.839 7fd1effff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1f009e960 0x7fd1f00a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51970/0 (socket says 172.21.2.101:51970) 2021-02-10T11:33:45.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.839 7fd1effff700 1 -- 172.21.2.101:0/2299807011 learned_addr learned my addr 172.21.2.101:0/2299807011 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:45.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.840 7fd1effff700 1 -- 172.21.2.101:0/2299807011 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1f009e560 msgr2=0x7fd1f00a0130 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:45.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.840 7fd1effff700 1 --2- 172.21.2.101:0/2299807011 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1f009e560 0x7fd1f00a0130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:45.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.840 7fd1effff700 1 -- 172.21.2.101:0/2299807011 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1f00a0d70 msgr2=0x7fd1f00a1af0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:45.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.840 7fd1effff700 1 --2- 172.21.2.101:0/2299807011 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1f00a0d70 0x7fd1f00a1af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:45.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.840 7fd1effff700 1 -- 172.21.2.101:0/2299807011 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f00a2110 con 0x7fd1f009e960 2021-02-10T11:33:45.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.840 7fd1effff700 1 --2- 172.21.2.101:0/2299807011 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1f009e960 0x7fd1f00a0750 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7fd1e0006f00 tx=0x7fd1e000b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:45.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.840 7fd1ed7fa700 1 -- 172.21.2.101:0/2299807011 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd1e000bd30 con 0x7fd1f009e960 2021-02-10T11:33:45.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.840 7fd1ed7fa700 1 -- 172.21.2.101:0/2299807011 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e0002030 con 0x7fd1f009e960 2021-02-10T11:33:45.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.840 7fd1ed7fa700 1 -- 172.21.2.101:0/2299807011 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd1e0002550 con 0x7fd1f009e960 2021-02-10T11:33:45.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.840 7fd1f713c700 1 -- 172.21.2.101:0/2299807011 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f00a2110 con 0x7fd1f009e960 2021-02-10T11:33:45.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.840 7fd1f713c700 1 -- 172.21.2.101:0/2299807011 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f00a26a0 con 0x7fd1f009e960 2021-02-10T11:33:45.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.841 7fd1ed7fa700 1 -- 172.21.2.101:0/2299807011 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd1e0000c90 con 0x7fd1f009e960 2021-02-10T11:33:45.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.841 7fd1ed7fa700 1 --2- 172.21.2.101:0/2299807011 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1d40186c0 0x7fd1d401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:45.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.841 7fd1ed7fa700 1 -- 172.21.2.101:0/2299807011 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd1e00325b0 con 0x7fd1f009e960 2021-02-10T11:33:45.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.841 7fd1f4f58700 1 --2- 172.21.2.101:0/2299807011 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1d40186c0 0x7fd1d401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:45.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.841 7fd1f4f58700 1 --2- 172.21.2.101:0/2299807011 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1d40186c0 0x7fd1d401abc0 crc :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:45.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.872 7fd1f713c700 1 -- 172.21.2.101:0/2299807011 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f0105b30 con 0x7fd1f009e960 2021-02-10T11:33:45.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:45.874 7fd1ed7fa700 1 -- 172.21.2.101:0/2299807011 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd1e0028f00 con 0x7fd1f009e960 2021-02-10T11:33:46.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.023 7fd1f713c700 1 -- 172.21.2.101:0/2299807011 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=debug_objecter}] v 0) v1 -- 0x7fd1f00f0b50 con 0x7fd1f009e960 2021-02-10T11:33:46.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.026 7fd1ed7fa700 1 -- 172.21.2.101:0/2299807011 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_objecter}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd1e0028f00 con 0x7fd1f009e960 2021-02-10T11:33:46.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.031 7fd1f713c700 1 -- 172.21.2.101:0/2299807011 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1d40186c0 msgr2=0x7fd1d401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:46.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.031 7fd1f713c700 1 --2- 172.21.2.101:0/2299807011 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1d40186c0 0x7fd1d401abc0 crc :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:46.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.031 7fd1f713c700 1 -- 172.21.2.101:0/2299807011 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1f009e960 msgr2=0x7fd1f00a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:46.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.031 7fd1f713c700 1 --2- 172.21.2.101:0/2299807011 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1f009e960 0x7fd1f00a0750 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7fd1e0006f00 tx=0x7fd1e000b940).stop 2021-02-10T11:33:46.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.032 7fd1f713c700 1 -- 172.21.2.101:0/2299807011 shutdown_connections 2021-02-10T11:33:46.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.032 7fd1f713c700 1 --2- 172.21.2.101:0/2299807011 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1f00a0d70 0x7fd1f00a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:46.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.032 7fd1f713c700 1 --2- 172.21.2.101:0/2299807011 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1f009e960 0x7fd1f00a0750 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:46.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.032 7fd1f713c700 1 --2- 172.21.2.101:0/2299807011 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1d40186c0 0x7fd1d401abc0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:46.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.032 7fd1f713c700 1 --2- 172.21.2.101:0/2299807011 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1f009e560 0x7fd1f00a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:46.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.032 7fd1f713c700 1 -- 172.21.2.101:0/2299807011 >> 172.21.2.101:0/2299807011 conn(0x7fd1f009fc10 msgr2=0x7fd1f0094ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:46.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.032 7fd1f713c700 1 -- 172.21.2.101:0/2299807011 shutdown_connections 2021-02-10T11:33:46.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.032 7fd1f713c700 1 -- 172.21.2.101:0/2299807011 wait complete. 2021-02-10T11:33:46.052 DEBUG:teuthology.orchestra.run.gibba001:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-02-10T11:33:46.054 DEBUG:teuthology.orchestra.run.gibba008:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-02-10T11:33:46.416 INFO:teuthology.orchestra.run.gibba001.stdout:[global] 2021-02-10T11:33:46.416 INFO:teuthology.orchestra.run.gibba001.stdout: chdir = "" 2021-02-10T11:33:46.416 INFO:teuthology.orchestra.run.gibba001.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-02-10T11:33:46.416 INFO:teuthology.orchestra.run.gibba001.stdout: auth supported = cephx 2021-02-10T11:33:46.416 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.416 INFO:teuthology.orchestra.run.gibba001.stdout: filestore xattr use omap = true 2021-02-10T11:33:46.416 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.416 INFO:teuthology.orchestra.run.gibba001.stdout: mon clock drift allowed = 1.000 2021-02-10T11:33:46.416 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.416 INFO:teuthology.orchestra.run.gibba001.stdout: osd crush chooseleaf type = 0 2021-02-10T11:33:46.416 INFO:teuthology.orchestra.run.gibba001.stdout: auth debug = true 2021-02-10T11:33:46.417 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.417 INFO:teuthology.orchestra.run.gibba001.stdout: ms die on old message = true 2021-02-10T11:33:46.417 INFO:teuthology.orchestra.run.gibba001.stdout: ms die on bug = true 2021-02-10T11:33:46.417 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.417 INFO:teuthology.orchestra.run.gibba001.stdout: mon max pg per osd = 10000 # >= luminous 2021-02-10T11:33:46.417 INFO:teuthology.orchestra.run.gibba001.stdout: mon pg warn max object skew = 0 2021-02-10T11:33:46.417 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.417 INFO:teuthology.orchestra.run.gibba001.stdout: osd pool default size = 2 2021-02-10T11:33:46.417 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.417 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd allow primary affinity = true 2021-02-10T11:33:46.417 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd allow pg remap = true 2021-02-10T11:33:46.417 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on legacy crush tunables = false 2021-02-10T11:33:46.417 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on crush straw calc version zero = false 2021-02-10T11:33:46.418 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on no sortbitwise = false 2021-02-10T11:33:46.418 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on osd down out interval zero = false 2021-02-10T11:33:46.418 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on too few osds = false 2021-02-10T11:33:46.418 INFO:teuthology.orchestra.run.gibba001.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-02-10T11:33:46.418 INFO:teuthology.orchestra.run.gibba001.stdout: mon_warn_on_pool_no_redundancy = false 2021-02-10T11:33:46.418 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.418 INFO:teuthology.orchestra.run.gibba001.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-02-10T11:33:46.418 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.418 INFO:teuthology.orchestra.run.gibba001.stdout: osd default data pool replay window = 5 2021-02-10T11:33:46.418 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.418 INFO:teuthology.orchestra.run.gibba001.stdout: mon allow pool delete = true 2021-02-10T11:33:46.418 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.418 INFO:teuthology.orchestra.run.gibba001.stdout: mon cluster log file level = debug 2021-02-10T11:33:46.418 INFO:teuthology.orchestra.run.gibba001.stdout: debug asserts on shutdown = true 2021-02-10T11:33:46.419 INFO:teuthology.orchestra.run.gibba001.stdout: mon health detail to clog = false 2021-02-10T11:33:46.419 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.419 INFO:teuthology.orchestra.run.gibba001.stdout: # we see this fail in qa on *nautilus*; bump up retries 2021-02-10T11:33:46.419 INFO:teuthology.orchestra.run.gibba001.stdout: mon_client_directed_command_retry = 4 2021-02-10T11:33:46.419 INFO:teuthology.orchestra.run.gibba001.stdout: mon host = "172.21.2.101,172.21.2.108,[v2:172.21.2.108:3301,v1:172.21.2.108:6790]" 2021-02-10T11:33:46.419 INFO:teuthology.orchestra.run.gibba001.stdout: fsid = a6663ba9-2653-4542-8b6a-9323b4e7b545 2021-02-10T11:33:46.419 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.419 INFO:teuthology.orchestra.run.gibba001.stdout:[osd] 2021-02-10T11:33:46.419 INFO:teuthology.orchestra.run.gibba001.stdout: osd journal size = 100 2021-02-10T11:33:46.419 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.419 INFO:teuthology.orchestra.run.gibba001.stdout: osd scrub load threshold = 5.0 2021-02-10T11:33:46.419 INFO:teuthology.orchestra.run.gibba001.stdout: osd scrub max interval = 600 2021-02-10T11:33:46.419 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.419 INFO:teuthology.orchestra.run.gibba001.stdout: osd recover clone overlap = true 2021-02-10T11:33:46.420 INFO:teuthology.orchestra.run.gibba001.stdout: osd recovery max chunk = 1048576 2021-02-10T11:33:46.420 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.420 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug shutdown = true 2021-02-10T11:33:46.420 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug op order = true 2021-02-10T11:33:46.420 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug verify stray on activate = true 2021-02-10T11:33:46.420 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.420 INFO:teuthology.orchestra.run.gibba001.stdout: osd open classes on start = true 2021-02-10T11:33:46.420 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug pg log writeout = true 2021-02-10T11:33:46.420 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.420 INFO:teuthology.orchestra.run.gibba001.stdout: osd deep scrub update digest min age = 30 2021-02-10T11:33:46.420 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.420 INFO:teuthology.orchestra.run.gibba001.stdout: osd map max advance = 10 2021-02-10T11:33:46.420 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.420 INFO:teuthology.orchestra.run.gibba001.stdout: journal zero on create = true 2021-02-10T11:33:46.421 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.421 INFO:teuthology.orchestra.run.gibba001.stdout: filestore ondisk finisher threads = 3 2021-02-10T11:33:46.421 INFO:teuthology.orchestra.run.gibba001.stdout: filestore apply finisher threads = 3 2021-02-10T11:33:46.421 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.421 INFO:teuthology.orchestra.run.gibba001.stdout: bdev debug aio = true 2021-02-10T11:33:46.421 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug misdirected ops = true 2021-02-10T11:33:46.421 INFO:teuthology.orchestra.run.gibba001.stdout: bdev async discard = True 2021-02-10T11:33:46.421 INFO:teuthology.orchestra.run.gibba001.stdout: bdev enable discard = True 2021-02-10T11:33:46.421 INFO:teuthology.orchestra.run.gibba001.stdout: bluestore allocator = bitmap 2021-02-10T11:33:46.421 INFO:teuthology.orchestra.run.gibba001.stdout: bluestore block size = 96636764160 2021-02-10T11:33:46.421 INFO:teuthology.orchestra.run.gibba001.stdout: bluestore fsck on mount = True 2021-02-10T11:33:46.421 INFO:teuthology.orchestra.run.gibba001.stdout: debug bluefs = 20 2021-02-10T11:33:46.421 INFO:teuthology.orchestra.run.gibba001.stdout: debug bluestore = 20 2021-02-10T11:33:46.421 INFO:teuthology.orchestra.run.gibba001.stdout: debug filestore = 20 2021-02-10T11:33:46.421 INFO:teuthology.orchestra.run.gibba001.stdout: debug journal = 20 2021-02-10T11:33:46.422 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 20 2021-02-10T11:33:46.422 INFO:teuthology.orchestra.run.gibba001.stdout: debug osd = 25 2021-02-10T11:33:46.422 INFO:teuthology.orchestra.run.gibba001.stdout: debug rocksdb = 10 2021-02-10T11:33:46.422 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd backfillfull_ratio = 0.85 2021-02-10T11:33:46.422 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd full ratio = 0.9 2021-02-10T11:33:46.422 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd nearfull ratio = 0.8 2021-02-10T11:33:46.422 INFO:teuthology.orchestra.run.gibba001.stdout: osd failsafe full ratio = 0.95 2021-02-10T11:33:46.422 INFO:teuthology.orchestra.run.gibba001.stdout: osd objectstore = bluestore 2021-02-10T11:33:46.422 INFO:teuthology.orchestra.run.gibba001.stdout: osd op complaint time = 180 2021-02-10T11:33:46.422 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.422 INFO:teuthology.orchestra.run.gibba001.stdout:[mgr] 2021-02-10T11:33:46.422 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 1 2021-02-10T11:33:46.422 INFO:teuthology.orchestra.run.gibba001.stdout: debug mgr = 20 2021-02-10T11:33:46.422 INFO:teuthology.orchestra.run.gibba001.stdout: debug mon = 20 2021-02-10T11:33:46.423 INFO:teuthology.orchestra.run.gibba001.stdout: debug auth = 20 2021-02-10T11:33:46.423 INFO:teuthology.orchestra.run.gibba001.stdout: mon reweight min pgs per osd = 4 2021-02-10T11:33:46.423 INFO:teuthology.orchestra.run.gibba001.stdout: mon reweight min bytes per osd = 10 2021-02-10T11:33:46.423 INFO:teuthology.orchestra.run.gibba001.stdout: debug client = 10 2021-02-10T11:33:46.423 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.423 INFO:teuthology.orchestra.run.gibba001.stdout:[mon] 2021-02-10T11:33:46.423 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 1 2021-02-10T11:33:46.423 INFO:teuthology.orchestra.run.gibba001.stdout: debug mon = 20 2021-02-10T11:33:46.423 INFO:teuthology.orchestra.run.gibba001.stdout: debug paxos = 20 2021-02-10T11:33:46.423 INFO:teuthology.orchestra.run.gibba001.stdout: debug auth = 20 2021-02-10T11:33:46.423 INFO:teuthology.orchestra.run.gibba001.stdout: mon data avail warn = 5 2021-02-10T11:33:46.423 INFO:teuthology.orchestra.run.gibba001.stdout: mon mgr mkfs grace = 240 2021-02-10T11:33:46.423 INFO:teuthology.orchestra.run.gibba001.stdout: mon reweight min pgs per osd = 4 2021-02-10T11:33:46.424 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd reporter subtree level = osd 2021-02-10T11:33:46.424 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd prime pg temp = true 2021-02-10T11:33:46.424 INFO:teuthology.orchestra.run.gibba001.stdout: mon reweight min bytes per osd = 10 2021-02-10T11:33:46.424 INFO:teuthology.orchestra.run.gibba001.stdout: mon op complaint time = 120 2021-02-10T11:33:46.424 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:46.424 INFO:teuthology.orchestra.run.gibba001.stdout:[client] 2021-02-10T11:33:46.424 INFO:teuthology.orchestra.run.gibba001.stdout: rgw cache enabled = true 2021-02-10T11:33:46.424 INFO:teuthology.orchestra.run.gibba001.stdout: rgw enable ops log = true 2021-02-10T11:33:46.424 INFO:teuthology.orchestra.run.gibba001.stdout: rgw enable usage log = true 2021-02-10T11:33:46.424 INFO:teuthology.orchestra.run.gibba001.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-02-10T11:33:46.424 INFO:teuthology.orchestra.run.gibba001.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-02-10T11:33:46.424 INFO:teuthology.orchestra.run.gibba001.stdout: client mount timeout = 600 2021-02-10T11:33:46.424 INFO:teuthology.orchestra.run.gibba001.stdout: debug client = 20 2021-02-10T11:33:46.424 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 1 2021-02-10T11:33:46.424 INFO:teuthology.orchestra.run.gibba001.stdout: pid file = "" 2021-02-10T11:33:46.425 INFO:teuthology.orchestra.run.gibba001.stdout:[mon.a] 2021-02-10T11:33:46.425 INFO:teuthology.orchestra.run.gibba001.stdout:[mon.b] 2021-02-10T11:33:46.425 INFO:teuthology.orchestra.run.gibba001.stdout:[mon.c] 2021-02-10T11:33:46.425 INFO:teuthology.orchestra.run.gibba001.stdout:[mds.a] 2021-02-10T11:33:46.425 INFO:teuthology.orchestra.run.gibba001.stdout:[mds.b] 2021-02-10T11:33:46.425 INFO:teuthology.orchestra.run.gibba001.stdout:[mds] 2021-02-10T11:33:46.425 INFO:teuthology.orchestra.run.gibba001.stdout: debug mds = 20 2021-02-10T11:33:46.425 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 1 2021-02-10T11:33:46.425 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal frag = True 2021-02-10T11:33:46.425 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal fragment size max = 10000 2021-02-10T11:33:46.425 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal merge size = 5 2021-02-10T11:33:46.425 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal split bits = 3 2021-02-10T11:33:46.426 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal split size = 100 2021-02-10T11:33:46.426 INFO:teuthology.orchestra.run.gibba001.stdout: mds debug frag = True 2021-02-10T11:33:46.426 INFO:teuthology.orchestra.run.gibba001.stdout: mds debug scatterstat = True 2021-02-10T11:33:46.426 INFO:teuthology.orchestra.run.gibba001.stdout: mds op complaint time = 180 2021-02-10T11:33:46.426 INFO:teuthology.orchestra.run.gibba001.stdout: mds verify scatter = True 2021-02-10T11:33:46.426 INFO:teuthology.orchestra.run.gibba001.stdout: osd op complaint time = 180 2021-02-10T11:33:46.426 INFO:teuthology.orchestra.run.gibba001.stdout:[client.alice] 2021-02-10T11:33:46.426 INFO:teuthology.orchestra.run.gibba001.stdout: keyring = /etc/ceph/ceph.client.alice.keyring 2021-02-10T11:33:46.427 INFO:teuthology.orchestra.run.gibba008.stdout:[global] 2021-02-10T11:33:46.427 INFO:teuthology.orchestra.run.gibba008.stdout: chdir = "" 2021-02-10T11:33:46.428 INFO:teuthology.orchestra.run.gibba008.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-02-10T11:33:46.428 INFO:teuthology.orchestra.run.gibba008.stdout: auth supported = cephx 2021-02-10T11:33:46.428 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.428 INFO:teuthology.orchestra.run.gibba008.stdout: filestore xattr use omap = true 2021-02-10T11:33:46.428 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.428 INFO:teuthology.orchestra.run.gibba008.stdout: mon clock drift allowed = 1.000 2021-02-10T11:33:46.428 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.428 INFO:teuthology.orchestra.run.gibba008.stdout: osd crush chooseleaf type = 0 2021-02-10T11:33:46.428 INFO:teuthology.orchestra.run.gibba008.stdout: auth debug = true 2021-02-10T11:33:46.428 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.428 INFO:teuthology.orchestra.run.gibba008.stdout: ms die on old message = true 2021-02-10T11:33:46.428 INFO:teuthology.orchestra.run.gibba008.stdout: ms die on bug = true 2021-02-10T11:33:46.428 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.429 INFO:teuthology.orchestra.run.gibba008.stdout: mon max pg per osd = 10000 # >= luminous 2021-02-10T11:33:46.429 INFO:teuthology.orchestra.run.gibba008.stdout: mon pg warn max object skew = 0 2021-02-10T11:33:46.429 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.429 INFO:teuthology.orchestra.run.gibba008.stdout: osd pool default size = 2 2021-02-10T11:33:46.429 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.429 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd allow primary affinity = true 2021-02-10T11:33:46.429 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd allow pg remap = true 2021-02-10T11:33:46.429 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on legacy crush tunables = false 2021-02-10T11:33:46.429 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on crush straw calc version zero = false 2021-02-10T11:33:46.429 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on no sortbitwise = false 2021-02-10T11:33:46.429 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on osd down out interval zero = false 2021-02-10T11:33:46.429 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on too few osds = false 2021-02-10T11:33:46.429 INFO:teuthology.orchestra.run.gibba008.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-02-10T11:33:46.429 INFO:teuthology.orchestra.run.gibba008.stdout: mon_warn_on_pool_no_redundancy = false 2021-02-10T11:33:46.430 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.430 INFO:teuthology.orchestra.run.gibba008.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-02-10T11:33:46.430 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.430 INFO:teuthology.orchestra.run.gibba008.stdout: osd default data pool replay window = 5 2021-02-10T11:33:46.430 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.430 INFO:teuthology.orchestra.run.gibba008.stdout: mon allow pool delete = true 2021-02-10T11:33:46.430 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.430 INFO:teuthology.orchestra.run.gibba008.stdout: mon cluster log file level = debug 2021-02-10T11:33:46.430 INFO:teuthology.orchestra.run.gibba008.stdout: debug asserts on shutdown = true 2021-02-10T11:33:46.430 INFO:teuthology.orchestra.run.gibba008.stdout: mon health detail to clog = false 2021-02-10T11:33:46.430 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.430 INFO:teuthology.orchestra.run.gibba008.stdout: # we see this fail in qa on *nautilus*; bump up retries 2021-02-10T11:33:46.430 INFO:teuthology.orchestra.run.gibba008.stdout: mon_client_directed_command_retry = 4 2021-02-10T11:33:46.430 INFO:teuthology.orchestra.run.gibba008.stdout: mon host = "172.21.2.101,172.21.2.108,[v2:172.21.2.108:3301,v1:172.21.2.108:6790]" 2021-02-10T11:33:46.431 INFO:teuthology.orchestra.run.gibba008.stdout: fsid = a6663ba9-2653-4542-8b6a-9323b4e7b545 2021-02-10T11:33:46.431 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.431 INFO:teuthology.orchestra.run.gibba008.stdout:[osd] 2021-02-10T11:33:46.431 INFO:teuthology.orchestra.run.gibba008.stdout: osd journal size = 100 2021-02-10T11:33:46.431 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.431 INFO:teuthology.orchestra.run.gibba008.stdout: osd scrub load threshold = 5.0 2021-02-10T11:33:46.431 INFO:teuthology.orchestra.run.gibba008.stdout: osd scrub max interval = 600 2021-02-10T11:33:46.431 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.431 INFO:teuthology.orchestra.run.gibba008.stdout: osd recover clone overlap = true 2021-02-10T11:33:46.431 INFO:teuthology.orchestra.run.gibba008.stdout: osd recovery max chunk = 1048576 2021-02-10T11:33:46.431 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.431 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug shutdown = true 2021-02-10T11:33:46.431 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug op order = true 2021-02-10T11:33:46.431 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug verify stray on activate = true 2021-02-10T11:33:46.432 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.432 INFO:teuthology.orchestra.run.gibba008.stdout: osd open classes on start = true 2021-02-10T11:33:46.432 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug pg log writeout = true 2021-02-10T11:33:46.432 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.432 INFO:teuthology.orchestra.run.gibba008.stdout: osd deep scrub update digest min age = 30 2021-02-10T11:33:46.432 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.432 INFO:teuthology.orchestra.run.gibba008.stdout: osd map max advance = 10 2021-02-10T11:33:46.432 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.432 INFO:teuthology.orchestra.run.gibba008.stdout: journal zero on create = true 2021-02-10T11:33:46.432 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.432 INFO:teuthology.orchestra.run.gibba008.stdout: filestore ondisk finisher threads = 3 2021-02-10T11:33:46.432 INFO:teuthology.orchestra.run.gibba008.stdout: filestore apply finisher threads = 3 2021-02-10T11:33:46.432 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.432 INFO:teuthology.orchestra.run.gibba008.stdout: bdev debug aio = true 2021-02-10T11:33:46.432 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug misdirected ops = true 2021-02-10T11:33:46.433 INFO:teuthology.orchestra.run.gibba008.stdout: bdev async discard = True 2021-02-10T11:33:46.433 INFO:teuthology.orchestra.run.gibba008.stdout: bdev enable discard = True 2021-02-10T11:33:46.433 INFO:teuthology.orchestra.run.gibba008.stdout: bluestore allocator = bitmap 2021-02-10T11:33:46.433 INFO:teuthology.orchestra.run.gibba008.stdout: bluestore block size = 96636764160 2021-02-10T11:33:46.433 INFO:teuthology.orchestra.run.gibba008.stdout: bluestore fsck on mount = True 2021-02-10T11:33:46.433 INFO:teuthology.orchestra.run.gibba008.stdout: debug bluefs = 20 2021-02-10T11:33:46.433 INFO:teuthology.orchestra.run.gibba008.stdout: debug bluestore = 20 2021-02-10T11:33:46.433 INFO:teuthology.orchestra.run.gibba008.stdout: debug filestore = 20 2021-02-10T11:33:46.433 INFO:teuthology.orchestra.run.gibba008.stdout: debug journal = 20 2021-02-10T11:33:46.433 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 20 2021-02-10T11:33:46.433 INFO:teuthology.orchestra.run.gibba008.stdout: debug osd = 25 2021-02-10T11:33:46.433 INFO:teuthology.orchestra.run.gibba008.stdout: debug rocksdb = 10 2021-02-10T11:33:46.433 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd backfillfull_ratio = 0.85 2021-02-10T11:33:46.433 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd full ratio = 0.9 2021-02-10T11:33:46.434 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd nearfull ratio = 0.8 2021-02-10T11:33:46.434 INFO:teuthology.orchestra.run.gibba008.stdout: osd failsafe full ratio = 0.95 2021-02-10T11:33:46.434 INFO:teuthology.orchestra.run.gibba008.stdout: osd objectstore = bluestore 2021-02-10T11:33:46.434 INFO:teuthology.orchestra.run.gibba008.stdout: osd op complaint time = 180 2021-02-10T11:33:46.434 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.434 INFO:teuthology.orchestra.run.gibba008.stdout:[mgr] 2021-02-10T11:33:46.434 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 1 2021-02-10T11:33:46.434 INFO:teuthology.orchestra.run.gibba008.stdout: debug mgr = 20 2021-02-10T11:33:46.434 INFO:teuthology.orchestra.run.gibba008.stdout: debug mon = 20 2021-02-10T11:33:46.434 INFO:teuthology.orchestra.run.gibba008.stdout: debug auth = 20 2021-02-10T11:33:46.434 INFO:teuthology.orchestra.run.gibba008.stdout: mon reweight min pgs per osd = 4 2021-02-10T11:33:46.434 INFO:teuthology.orchestra.run.gibba008.stdout: mon reweight min bytes per osd = 10 2021-02-10T11:33:46.434 INFO:teuthology.orchestra.run.gibba008.stdout: debug client = 10 2021-02-10T11:33:46.434 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.435 INFO:teuthology.orchestra.run.gibba008.stdout:[mon] 2021-02-10T11:33:46.435 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 1 2021-02-10T11:33:46.435 INFO:teuthology.orchestra.run.gibba008.stdout: debug mon = 20 2021-02-10T11:33:46.435 INFO:teuthology.orchestra.run.gibba008.stdout: debug paxos = 20 2021-02-10T11:33:46.435 INFO:teuthology.orchestra.run.gibba008.stdout: debug auth = 20 2021-02-10T11:33:46.435 INFO:teuthology.orchestra.run.gibba008.stdout: mon data avail warn = 5 2021-02-10T11:33:46.435 INFO:teuthology.orchestra.run.gibba008.stdout: mon mgr mkfs grace = 240 2021-02-10T11:33:46.435 INFO:teuthology.orchestra.run.gibba008.stdout: mon reweight min pgs per osd = 4 2021-02-10T11:33:46.435 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd reporter subtree level = osd 2021-02-10T11:33:46.435 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd prime pg temp = true 2021-02-10T11:33:46.435 INFO:teuthology.orchestra.run.gibba008.stdout: mon reweight min bytes per osd = 10 2021-02-10T11:33:46.435 INFO:teuthology.orchestra.run.gibba008.stdout: mon op complaint time = 120 2021-02-10T11:33:46.435 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:33:46.435 INFO:teuthology.orchestra.run.gibba008.stdout:[client] 2021-02-10T11:33:46.435 INFO:teuthology.orchestra.run.gibba008.stdout: rgw cache enabled = true 2021-02-10T11:33:46.436 INFO:teuthology.orchestra.run.gibba008.stdout: rgw enable ops log = true 2021-02-10T11:33:46.436 INFO:teuthology.orchestra.run.gibba008.stdout: rgw enable usage log = true 2021-02-10T11:33:46.436 INFO:teuthology.orchestra.run.gibba008.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-02-10T11:33:46.436 INFO:teuthology.orchestra.run.gibba008.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-02-10T11:33:46.436 INFO:teuthology.orchestra.run.gibba008.stdout: client mount timeout = 600 2021-02-10T11:33:46.436 INFO:teuthology.orchestra.run.gibba008.stdout: debug client = 20 2021-02-10T11:33:46.436 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 1 2021-02-10T11:33:46.436 INFO:teuthology.orchestra.run.gibba008.stdout: pid file = "" 2021-02-10T11:33:46.436 INFO:teuthology.orchestra.run.gibba008.stdout:[mon.a] 2021-02-10T11:33:46.436 INFO:teuthology.orchestra.run.gibba008.stdout:[mon.b] 2021-02-10T11:33:46.436 INFO:teuthology.orchestra.run.gibba008.stdout:[mon.c] 2021-02-10T11:33:46.436 INFO:teuthology.orchestra.run.gibba008.stdout:[mds.a] 2021-02-10T11:33:46.436 INFO:teuthology.orchestra.run.gibba008.stdout:[mds.b] 2021-02-10T11:33:46.436 INFO:teuthology.orchestra.run.gibba008.stdout:[mds] 2021-02-10T11:33:46.437 INFO:teuthology.orchestra.run.gibba008.stdout: debug mds = 20 2021-02-10T11:33:46.437 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 1 2021-02-10T11:33:46.437 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal frag = True 2021-02-10T11:33:46.437 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal fragment size max = 10000 2021-02-10T11:33:46.437 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal merge size = 5 2021-02-10T11:33:46.437 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal split bits = 3 2021-02-10T11:33:46.437 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal split size = 100 2021-02-10T11:33:46.437 INFO:teuthology.orchestra.run.gibba008.stdout: mds debug frag = True 2021-02-10T11:33:46.437 INFO:teuthology.orchestra.run.gibba008.stdout: mds debug scatterstat = True 2021-02-10T11:33:46.437 INFO:teuthology.orchestra.run.gibba008.stdout: mds op complaint time = 180 2021-02-10T11:33:46.437 INFO:teuthology.orchestra.run.gibba008.stdout: mds verify scatter = True 2021-02-10T11:33:46.437 INFO:teuthology.orchestra.run.gibba008.stdout: osd op complaint time = 180 2021-02-10T11:33:46.437 INFO:teuthology.orchestra.run.gibba008.stdout:[client.alice] 2021-02-10T11:33:46.437 INFO:teuthology.orchestra.run.gibba008.stdout: keyring = /etc/ceph/ceph.client.alice.keyring 2021-02-10T11:33:46.438 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:33:46.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.835 7fc0a49b8700 1 Processor -- start 2021-02-10T11:33:46.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.835 7fc0a49b8700 1 -- start start 2021-02-10T11:33:46.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.836 7fc0a49b8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc0a00e53e0 0x7fc0a01b0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:46.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.836 7fc0a49b8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc0a01b0b70 0x7fc0a0081910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:46.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.836 7fc0a49b8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0a0081f30 0x7fc0a0082330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:46.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.836 7fc0a49b8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0a00e5fe0 con 0x7fc0a00e53e0 2021-02-10T11:33:46.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.836 7fc0a49b8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0a0082d50 con 0x7fc0a01b0b70 2021-02-10T11:33:46.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.836 7fc0a49b8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a00e57e0 con 0x7fc0a0081f30 2021-02-10T11:33:46.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.836 7fc09e61c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc0a00e53e0 0x7fc0a01b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:46.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.836 7fc09e61c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc0a00e53e0 0x7fc0a01b0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51976/0 (socket says 172.21.2.101:51976) 2021-02-10T11:33:46.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.836 7fc09e61c700 1 -- 172.21.2.101:0/184076602 learned_addr learned my addr 172.21.2.101:0/184076602 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:46.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.836 7fc09de1b700 1 --2- 172.21.2.101:0/184076602 >> v2:172.21.2.108:3300/0 conn(0x7fc0a01b0b70 0x7fc0a0081910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:46.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.836 7fc09e61c700 1 -- 172.21.2.101:0/184076602 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0a0081f30 msgr2=0x7fc0a0082330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:46.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.836 7fc09ee1d700 1 --2- 172.21.2.101:0/184076602 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0a0081f30 0x7fc0a0082330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:46.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.837 7fc09e61c700 1 --2- 172.21.2.101:0/184076602 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0a0081f30 0x7fc0a0082330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:46.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.837 7fc09e61c700 1 -- 172.21.2.101:0/184076602 >> v2:172.21.2.108:3300/0 conn(0x7fc0a01b0b70 msgr2=0x7fc0a0081910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:46.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.837 7fc09e61c700 1 --2- 172.21.2.101:0/184076602 >> v2:172.21.2.108:3300/0 conn(0x7fc0a01b0b70 0x7fc0a0081910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:46.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.837 7fc09e61c700 1 -- 172.21.2.101:0/184076602 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a0082950 con 0x7fc0a00e53e0 2021-02-10T11:33:46.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.837 7fc09de1b700 1 --2- 172.21.2.101:0/184076602 >> v2:172.21.2.108:3300/0 conn(0x7fc0a01b0b70 0x7fc0a0081910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:33:46.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.837 7fc09ee1d700 1 --2- 172.21.2.101:0/184076602 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0a0081f30 0x7fc0a0082330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:33:46.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.837 7fc09e61c700 1 --2- 172.21.2.101:0/184076602 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0a00e53e0 0x7fc0a01b0550 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0x7fc088006700 tx=0x7fc088009b80).ready entity=mon.0 client_cookie=fc959a51913a66ee server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:46.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.837 7fc09d61a700 1 -- 172.21.2.101:0/184076602 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc08800c0c0 con 0x7fc0a00e53e0 2021-02-10T11:33:46.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.837 7fc09d61a700 1 -- 172.21.2.101:0/184076602 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08800c340 con 0x7fc0a00e53e0 2021-02-10T11:33:46.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.837 7fc09d61a700 1 -- 172.21.2.101:0/184076602 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc08800c860 con 0x7fc0a00e53e0 2021-02-10T11:33:46.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.837 7fc0a49b8700 1 -- 172.21.2.101:0/184076602 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0a00e53e0 msgr2=0x7fc0a01b0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:46.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.837 7fc0a49b8700 1 --2- 172.21.2.101:0/184076602 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0a00e53e0 0x7fc0a01b0550 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0x7fc088006700 tx=0x7fc088009b80).stop 2021-02-10T11:33:46.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.838 7fc0a49b8700 1 -- 172.21.2.101:0/184076602 shutdown_connections 2021-02-10T11:33:46.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.838 7fc0a49b8700 1 --2- 172.21.2.101:0/184076602 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0a0081f30 0x7fc0a0082330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:46.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.838 7fc0a49b8700 1 --2- 172.21.2.101:0/184076602 >> v2:172.21.2.108:3300/0 conn(0x7fc0a01b0b70 0x7fc0a0081910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:46.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.838 7fc0a49b8700 1 --2- 172.21.2.101:0/184076602 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0a00e53e0 0x7fc0a01b0550 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:46.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.838 7fc0a49b8700 1 -- 172.21.2.101:0/184076602 >> 172.21.2.101:0/184076602 conn(0x7fc0a00ce500 msgr2=0x7fc0a00e4610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:46.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.838 7fc0a49b8700 1 -- 172.21.2.101:0/184076602 shutdown_connections 2021-02-10T11:33:46.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.838 7fc0a49b8700 1 -- 172.21.2.101:0/184076602 wait complete. 2021-02-10T11:33:46.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.853 7fc0a49b8700 1 Processor -- start 2021-02-10T11:33:46.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.853 7fc0a49b8700 1 -- start start 2021-02-10T11:33:46.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.853 7fc0a49b8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0a00b1af0 0x7fc0a00b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:46.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.853 7fc0a49b8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0a00b2510 0x7fc0a01b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:46.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.853 7fc0a49b8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0a01b85c0 0x7fc0a01b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:46.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.853 7fc0a49b8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a00b2910 con 0x7fc0a00b2510 2021-02-10T11:33:46.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.853 7fc0a49b8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a01b7a50 con 0x7fc0a00b1af0 2021-02-10T11:33:46.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.853 7fc0a49b8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a01b7780 con 0x7fc0a01b85c0 2021-02-10T11:33:46.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.853 7fc09de1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0a00b2510 0x7fc0a01b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:46.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.853 7fc09de1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0a00b2510 0x7fc0a01b7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51982/0 (socket says 172.21.2.101:51982) 2021-02-10T11:33:46.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.853 7fc09de1b700 1 -- 172.21.2.101:0/3402263474 learned_addr learned my addr 172.21.2.101:0/3402263474 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:46.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.854 7fc09de1b700 1 -- 172.21.2.101:0/3402263474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0a01b85c0 msgr2=0x7fc0a01b89c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:46.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.854 7fc09de1b700 1 --2- 172.21.2.101:0/3402263474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0a01b85c0 0x7fc0a01b89c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:46.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.854 7fc09de1b700 1 -- 172.21.2.101:0/3402263474 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0a00b1af0 msgr2=0x7fc0a00b1ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:46.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.854 7fc09de1b700 1 --2- 172.21.2.101:0/3402263474 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0a00b1af0 0x7fc0a00b1ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:46.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.854 7fc09de1b700 1 -- 172.21.2.101:0/3402263474 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a01b8fe0 con 0x7fc0a00b2510 2021-02-10T11:33:46.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.854 7fc09de1b700 1 --2- 172.21.2.101:0/3402263474 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0a00b2510 0x7fc0a01b7fa0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7fc090006f20 tx=0x7fc09000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:46.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.854 7fc096ffd700 1 -- 172.21.2.101:0/3402263474 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc09000bd80 con 0x7fc0a00b2510 2021-02-10T11:33:46.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.854 7fc096ffd700 1 -- 172.21.2.101:0/3402263474 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc090000c90 con 0x7fc0a00b2510 2021-02-10T11:33:46.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.854 7fc0a49b8700 1 -- 172.21.2.101:0/3402263474 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a01b8fe0 con 0x7fc0a00b2510 2021-02-10T11:33:46.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.854 7fc0a49b8700 1 -- 172.21.2.101:0/3402263474 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a01aefb0 con 0x7fc0a00b2510 2021-02-10T11:33:46.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.854 7fc096ffd700 1 -- 172.21.2.101:0/3402263474 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc0900022b0 con 0x7fc0a00b2510 2021-02-10T11:33:46.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.855 7fc096ffd700 1 -- 172.21.2.101:0/3402263474 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc09000a000 con 0x7fc0a00b2510 2021-02-10T11:33:46.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.855 7fc096ffd700 1 --2- 172.21.2.101:0/3402263474 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc084018730 0x7fc08401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:46.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.855 7fc096ffd700 1 -- 172.21.2.101:0/3402263474 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc090032780 con 0x7fc0a00b2510 2021-02-10T11:33:46.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.855 7fc09e61c700 1 --2- 172.21.2.101:0/3402263474 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc084018730 0x7fc08401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:46.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.855 7fc09e61c700 1 --2- 172.21.2.101:0/3402263474 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc084018730 0x7fc08401abc0 crc :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:46.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.886 7fc0a49b8700 1 -- 172.21.2.101:0/3402263474 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a00ed660 con 0x7fc0a00b2510 2021-02-10T11:33:46.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:46.888 7fc096ffd700 1 -- 172.21.2.101:0/3402263474 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc0900009b0 con 0x7fc0a00b2510 2021-02-10T11:33:47.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.012 7fc0a49b8700 1 -- 172.21.2.101:0/3402263474 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0a0082d50 con 0x7fc0a00b2510 2021-02-10T11:33:47.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.013 7fc096ffd700 1 -- 172.21.2.101:0/3402263474 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+13996 (secure 0 0 0) 0x7fc09000bd80 con 0x7fc0a00b2510 2021-02-10T11:33:47.024 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:47.024 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":55,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:33:17.591716","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:33:14.515763","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2021-02-10 11:33:15.636274","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:33:47.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.014 7fc0a49b8700 1 -- 172.21.2.101:0/3402263474 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc084018730 msgr2=0x7fc08401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:47.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.014 7fc0a49b8700 1 --2- 172.21.2.101:0/3402263474 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc084018730 0x7fc08401abc0 crc :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:47.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.014 7fc0a49b8700 1 -- 172.21.2.101:0/3402263474 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0a00b2510 msgr2=0x7fc0a01b7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:47.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.014 7fc0a49b8700 1 --2- 172.21.2.101:0/3402263474 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0a00b2510 0x7fc0a01b7fa0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7fc090006f20 tx=0x7fc09000b990).stop 2021-02-10T11:33:47.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.015 7fc0a49b8700 1 -- 172.21.2.101:0/3402263474 shutdown_connections 2021-02-10T11:33:47.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.015 7fc0a49b8700 1 --2- 172.21.2.101:0/3402263474 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc084018730 0x7fc08401abc0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:47.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.015 7fc0a49b8700 1 --2- 172.21.2.101:0/3402263474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0a01b85c0 0x7fc0a01b89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:47.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.015 7fc0a49b8700 1 --2- 172.21.2.101:0/3402263474 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0a00b2510 0x7fc0a01b7fa0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:47.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.015 7fc0a49b8700 1 --2- 172.21.2.101:0/3402263474 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0a00b1af0 0x7fc0a00b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:47.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.015 7fc0a49b8700 1 -- 172.21.2.101:0/3402263474 >> 172.21.2.101:0/3402263474 conn(0x7fc0a00f2240 msgr2=0x7fc0a0094840 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:47.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.015 7fc0a49b8700 1 -- 172.21.2.101:0/3402263474 shutdown_connections 2021-02-10T11:33:47.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.015 7fc0a49b8700 1 -- 172.21.2.101:0/3402263474 wait complete. 2021-02-10T11:33:47.035 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:33:14.515763', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:33:15.636274', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:33:47.035 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:33:47.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.439 7fb202cb3700 1 Processor -- start 2021-02-10T11:33:47.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.439 7fb202cb3700 1 -- start start 2021-02-10T11:33:47.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.440 7fb202cb3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1fc0e5570 0x7fb1fc078520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:47.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.440 7fb202cb3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1fc079110 0x7fb1fc079510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:47.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.440 7fb202cb3700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1fc078b90 con 0x7fb1fc0e5170 2021-02-10T11:33:47.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.440 7fb202cb3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1fc1b40d0 con 0x7fb1fc0e5570 2021-02-10T11:33:47.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.440 7fb202cb3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc0d1900 con 0x7fb1fc079110 2021-02-10T11:33:47.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.440 7fb1fbfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1fc0e5570 0x7fb1fc078520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:47.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.440 7fb1fbfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1fc0e5570 0x7fb1fc078520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51992/0 (socket says 172.21.2.101:51992) 2021-02-10T11:33:47.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.440 7fb2012d0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1fc079110 0x7fb1fc079510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:47.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.440 7fb1fbfff700 1 -- 172.21.2.101:0/2107901809 learned_addr learned my addr 172.21.2.101:0/2107901809 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:47.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.440 7fb1fbfff700 1 -- 172.21.2.101:0/2107901809 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1fc079110 msgr2=0x7fb1fc079510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:33:47.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.440 7fb1fbfff700 1 --2- 172.21.2.101:0/2107901809 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1fc079110 0x7fb1fc079510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:47.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.440 7fb1fbfff700 1 -- 172.21.2.101:0/2107901809 >> v1:172.21.2.108:6789/0 conn(0x7fb1fc0e5170 legacy=0x7fb1fc1b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:47.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.440 7fb1fbfff700 1 -- 172.21.2.101:0/2107901809 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1fc0d1b20 con 0x7fb1fc0e5570 2021-02-10T11:33:47.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.440 7fb1fbfff700 1 --2- 172.21.2.101:0/2107901809 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1fc0e5570 0x7fb1fc078520 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7fb1ec006a80 tx=0x7fb1ec009f00).ready entity=mon.0 client_cookie=545173148d701e83 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:47.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.440 7fb1fb7fe700 1 -- 172.21.2.101:0/2107901809 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1ec00b5c0 con 0x7fb1fc0e5570 2021-02-10T11:33:47.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.441 7fb1fb7fe700 1 -- 172.21.2.101:0/2107901809 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ec00b8c0 con 0x7fb1fc0e5570 2021-02-10T11:33:47.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.441 7fb1fb7fe700 1 -- 172.21.2.101:0/2107901809 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1ec00bda0 con 0x7fb1fc0e5570 2021-02-10T11:33:47.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.441 7fb202cb3700 1 -- 172.21.2.101:0/2107901809 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1fc0e5570 msgr2=0x7fb1fc078520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:47.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.441 7fb202cb3700 1 --2- 172.21.2.101:0/2107901809 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1fc0e5570 0x7fb1fc078520 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7fb1ec006a80 tx=0x7fb1ec009f00).stop 2021-02-10T11:33:47.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.441 7fb202cb3700 1 -- 172.21.2.101:0/2107901809 shutdown_connections 2021-02-10T11:33:47.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.441 7fb202cb3700 1 --2- 172.21.2.101:0/2107901809 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1fc079110 0x7fb1fc079510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:47.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.441 7fb202cb3700 1 --2- 172.21.2.101:0/2107901809 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1fc0e5570 0x7fb1fc078520 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:47.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.441 7fb202cb3700 1 -- 172.21.2.101:0/2107901809 >> 172.21.2.101:0/2107901809 conn(0x7fb1fc0ce820 msgr2=0x7fb1fc0e4660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:47.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.441 7fb202cb3700 1 -- 172.21.2.101:0/2107901809 shutdown_connections 2021-02-10T11:33:47.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.441 7fb202cb3700 1 -- 172.21.2.101:0/2107901809 wait complete. 2021-02-10T11:33:47.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.456 7fb202cb3700 1 Processor -- start 2021-02-10T11:33:47.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.456 7fb202cb3700 1 -- start start 2021-02-10T11:33:47.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.456 7fb202cb3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1fc1a0160 0x7fb1fc0ac950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:47.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.456 7fb202cb3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1fc0acf70 0x7fb1fc091cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:47.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.456 7fb202cb3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1fc0922e0 0x7fb1fc0926e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:47.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.456 7fb202cb3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc0ad370 con 0x7fb1fc1a0160 2021-02-10T11:33:47.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.456 7fb202cb3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc0ad710 con 0x7fb1fc0acf70 2021-02-10T11:33:47.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.456 7fb202cb3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc092d00 con 0x7fb1fc0922e0 2021-02-10T11:33:47.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.456 7fb200acf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1fc1a0160 0x7fb1fc0ac950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:47.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.456 7fb200acf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1fc1a0160 0x7fb1fc0ac950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:51996/0 (socket says 172.21.2.101:51996) 2021-02-10T11:33:47.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.456 7fb200acf700 1 -- 172.21.2.101:0/3557530800 learned_addr learned my addr 172.21.2.101:0/3557530800 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:47.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.457 7fb2012d0700 1 --2- 172.21.2.101:0/3557530800 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1fc0922e0 0x7fb1fc0926e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:47.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.457 7fb1fbfff700 1 --2- 172.21.2.101:0/3557530800 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1fc0acf70 0x7fb1fc091cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:47.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.457 7fb200acf700 1 -- 172.21.2.101:0/3557530800 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1fc0922e0 msgr2=0x7fb1fc0926e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:47.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.457 7fb200acf700 1 --2- 172.21.2.101:0/3557530800 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1fc0922e0 0x7fb1fc0926e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:47.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.457 7fb200acf700 1 -- 172.21.2.101:0/3557530800 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1fc0acf70 msgr2=0x7fb1fc091cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:47.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.457 7fb200acf700 1 --2- 172.21.2.101:0/3557530800 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1fc0acf70 0x7fb1fc091cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:47.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.457 7fb200acf700 1 -- 172.21.2.101:0/3557530800 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1fc092f20 con 0x7fb1fc1a0160 2021-02-10T11:33:47.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.457 7fb1fbfff700 1 --2- 172.21.2.101:0/3557530800 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1fc0acf70 0x7fb1fc091cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:33:47.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.457 7fb2012d0700 1 --2- 172.21.2.101:0/3557530800 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1fc0922e0 0x7fb1fc0926e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:33:47.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.457 7fb200acf700 1 --2- 172.21.2.101:0/3557530800 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1fc1a0160 0x7fb1fc0ac950 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7fb1e400b5b0 tx=0x7fb1e400b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:47.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.457 7fb1f97fa700 1 -- 172.21.2.101:0/3557530800 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1e400bd40 con 0x7fb1fc1a0160 2021-02-10T11:33:47.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.457 7fb1f97fa700 1 -- 172.21.2.101:0/3557530800 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e4006020 con 0x7fb1fc1a0160 2021-02-10T11:33:47.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.457 7fb1f97fa700 1 -- 172.21.2.101:0/3557530800 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1e40054a0 con 0x7fb1fc1a0160 2021-02-10T11:33:47.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.457 7fb202cb3700 1 -- 172.21.2.101:0/3557530800 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1fc092f20 con 0x7fb1fc1a0160 2021-02-10T11:33:47.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.457 7fb202cb3700 1 -- 172.21.2.101:0/3557530800 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1fc092f20 con 0x7fb1fc1a0160 2021-02-10T11:33:47.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.458 7fb1f97fa700 1 -- 172.21.2.101:0/3557530800 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb1e4006200 con 0x7fb1fc1a0160 2021-02-10T11:33:47.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.458 7fb1f97fa700 1 --2- 172.21.2.101:0/3557530800 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1e00186c0 0x7fb1e001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:47.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.458 7fb1f97fa700 1 -- 172.21.2.101:0/3557530800 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb1e4032430 con 0x7fb1fc1a0160 2021-02-10T11:33:47.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.458 7fb1fbfff700 1 --2- 172.21.2.101:0/3557530800 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1e00186c0 0x7fb1e001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:47.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.458 7fb1fbfff700 1 --2- 172.21.2.101:0/3557530800 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1e00186c0 0x7fb1e001abc0 crc :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:47.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.489 7fb202cb3700 1 -- 172.21.2.101:0/3557530800 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1fc0ad370 con 0x7fb1fc1a0160 2021-02-10T11:33:47.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.492 7fb1f97fa700 1 -- 172.21.2.101:0/3557530800 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb1e4000c60 con 0x7fb1fc1a0160 2021-02-10T11:33:47.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.619 7fb202cb3700 1 -- 172.21.2.101:0/3557530800 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fb1fc0ad370 con 0x7fb1fc1a0160 2021-02-10T11:33:47.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.620 7fb1f97fa700 1 -- 172.21.2.101:0/3557530800 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v55) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb1e400bd40 con 0x7fb1fc1a0160 2021-02-10T11:33:47.631 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:33:47.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.622 7fb202cb3700 1 -- 172.21.2.101:0/3557530800 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1e00186c0 msgr2=0x7fb1e001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:47.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.622 7fb202cb3700 1 --2- 172.21.2.101:0/3557530800 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1e00186c0 0x7fb1e001abc0 crc :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:47.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.622 7fb202cb3700 1 -- 172.21.2.101:0/3557530800 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1fc1a0160 msgr2=0x7fb1fc0ac950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:47.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.622 7fb202cb3700 1 --2- 172.21.2.101:0/3557530800 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1fc1a0160 0x7fb1fc0ac950 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7fb1e400b5b0 tx=0x7fb1e400b950).stop 2021-02-10T11:33:47.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.622 7fb202cb3700 1 -- 172.21.2.101:0/3557530800 shutdown_connections 2021-02-10T11:33:47.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.622 7fb202cb3700 1 --2- 172.21.2.101:0/3557530800 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1e00186c0 0x7fb1e001abc0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:47.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.622 7fb202cb3700 1 --2- 172.21.2.101:0/3557530800 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1fc0922e0 0x7fb1fc0926e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:47.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.622 7fb202cb3700 1 --2- 172.21.2.101:0/3557530800 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1fc0acf70 0x7fb1fc091cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:47.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.622 7fb202cb3700 1 --2- 172.21.2.101:0/3557530800 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1fc1a0160 0x7fb1fc0ac950 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:47.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.622 7fb202cb3700 1 -- 172.21.2.101:0/3557530800 >> 172.21.2.101:0/3557530800 conn(0x7fb1fc188880 msgr2=0x7fb1fc0a4380 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:47.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.622 7fb202cb3700 1 -- 172.21.2.101:0/3557530800 shutdown_connections 2021-02-10T11:33:47.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:47.622 7fb202cb3700 1 -- 172.21.2.101:0/3557530800 wait complete. 2021-02-10T11:33:47.641 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:33:48.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.043 7fa530886700 1 Processor -- start 2021-02-10T11:33:48.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.043 7fa530886700 1 -- start start 2021-02-10T11:33:48.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.043 7fa530886700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa52c0e5170 0x7fa52c1b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:48.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.043 7fa530886700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa52c1b3f00 0x7fa52c077510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:48.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.043 7fa530886700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa52c078b90 con 0x7fa52c0e5570 2021-02-10T11:33:48.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.043 7fa530886700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52c07a120 con 0x7fa52c0e5170 2021-02-10T11:33:48.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.043 7fa530886700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52c07a2d0 con 0x7fa52c1b3f00 2021-02-10T11:33:48.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.043 7fa52a61c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa52c0e5170 0x7fa52c1b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:48.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.043 7fa52a61c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa52c0e5170 0x7fa52c1b38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52004/0 (socket says 172.21.2.101:52004) 2021-02-10T11:33:48.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.043 7fa52a61c700 1 -- 172.21.2.101:0/3308773674 learned_addr learned my addr 172.21.2.101:0/3308773674 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:48.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.044 7fa52a61c700 1 -- 172.21.2.101:0/3308773674 >> v2:172.21.2.108:3300/0 conn(0x7fa52c1b3f00 msgr2=0x7fa52c077510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:33:48.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.044 7fa52a61c700 1 --2- 172.21.2.101:0/3308773674 >> v2:172.21.2.108:3300/0 conn(0x7fa52c1b3f00 0x7fa52c077510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:48.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.044 7fa52a61c700 1 -- 172.21.2.101:0/3308773674 >> v1:172.21.2.108:6789/0 conn(0x7fa52c0e5570 legacy=0x7fa52c077b30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:33:48.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.044 7fa52a61c700 1 -- 172.21.2.101:0/3308773674 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c07a4f0 con 0x7fa52c0e5170 2021-02-10T11:33:48.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.044 7fa52a61c700 1 --2- 172.21.2.101:0/3308773674 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa52c0e5170 0x7fa52c1b38e0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7fa51c006700 tx=0x7fa51c009b00).ready entity=mon.0 client_cookie=59396a062bdb5b80 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:48.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.044 7fa52961a700 1 -- 172.21.2.101:0/3308773674 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa51c00c0c0 con 0x7fa52c0e5170 2021-02-10T11:33:48.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.044 7fa52961a700 1 -- 172.21.2.101:0/3308773674 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c00c400 con 0x7fa52c0e5170 2021-02-10T11:33:48.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.044 7fa52961a700 1 -- 172.21.2.101:0/3308773674 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa51c00c920 con 0x7fa52c0e5170 2021-02-10T11:33:48.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.044 7fa530886700 1 -- 172.21.2.101:0/3308773674 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa52c0e5170 msgr2=0x7fa52c1b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:48.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.044 7fa530886700 1 --2- 172.21.2.101:0/3308773674 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa52c0e5170 0x7fa52c1b38e0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7fa51c006700 tx=0x7fa51c009b00).stop 2021-02-10T11:33:48.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.045 7fa530886700 1 -- 172.21.2.101:0/3308773674 shutdown_connections 2021-02-10T11:33:48.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.045 7fa530886700 1 --2- 172.21.2.101:0/3308773674 >> v2:172.21.2.108:3300/0 conn(0x7fa52c1b3f00 0x7fa52c077510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:48.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.045 7fa530886700 1 --2- 172.21.2.101:0/3308773674 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa52c0e5170 0x7fa52c1b38e0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:48.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.045 7fa530886700 1 -- 172.21.2.101:0/3308773674 >> 172.21.2.101:0/3308773674 conn(0x7fa52c0ce820 msgr2=0x7fa52c0e4660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:48.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.045 7fa530886700 1 -- 172.21.2.101:0/3308773674 shutdown_connections 2021-02-10T11:33:48.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.045 7fa530886700 1 -- 172.21.2.101:0/3308773674 wait complete. 2021-02-10T11:33:48.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.060 7fa530886700 1 Processor -- start 2021-02-10T11:33:48.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.060 7fa530886700 1 -- start start 2021-02-10T11:33:48.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.060 7fa530886700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa52c18f950 0x7fa52c0a3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:48.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.060 7fa530886700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa52c0a9a40 0x7fa52c0a9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:48.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.060 7fa530886700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa52c0b72d0 0x7fa52c0b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:48.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.060 7fa530886700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c18fd50 con 0x7fa52c18f950 2021-02-10T11:33:48.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.060 7fa530886700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c0b7cf0 con 0x7fa52c0a9a40 2021-02-10T11:33:48.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.060 7fa530886700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c0a3ab0 con 0x7fa52c0b72d0 2021-02-10T11:33:48.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.060 7fa52a61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa52c18f950 0x7fa52c0a3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:48.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.060 7fa52a61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa52c18f950 0x7fa52c0a3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52008/0 (socket says 172.21.2.101:52008) 2021-02-10T11:33:48.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.060 7fa52a61c700 1 -- 172.21.2.101:0/2079561444 learned_addr learned my addr 172.21.2.101:0/2079561444 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:48.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.061 7fa52ae1d700 1 --2- 172.21.2.101:0/2079561444 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa52c0b72d0 0x7fa52c0b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:48.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.061 7fa529e1b700 1 --2- 172.21.2.101:0/2079561444 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa52c0a9a40 0x7fa52c0a9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:48.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.061 7fa52a61c700 1 -- 172.21.2.101:0/2079561444 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa52c0b72d0 msgr2=0x7fa52c0b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:48.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.061 7fa52a61c700 1 --2- 172.21.2.101:0/2079561444 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa52c0b72d0 0x7fa52c0b76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:48.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.061 7fa52a61c700 1 -- 172.21.2.101:0/2079561444 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa52c0a9a40 msgr2=0x7fa52c0a9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:48.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.061 7fa52a61c700 1 --2- 172.21.2.101:0/2079561444 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa52c0a9a40 0x7fa52c0a9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:48.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.061 7fa52a61c700 1 -- 172.21.2.101:0/2079561444 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c0b3d40 con 0x7fa52c18f950 2021-02-10T11:33:48.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.061 7fa52ae1d700 1 --2- 172.21.2.101:0/2079561444 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa52c0b72d0 0x7fa52c0b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:33:48.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.061 7fa529e1b700 1 --2- 172.21.2.101:0/2079561444 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa52c0a9a40 0x7fa52c0a9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:33:48.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.061 7fa52a61c700 1 --2- 172.21.2.101:0/2079561444 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa52c18f950 0x7fa52c0a3490 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7fa51c00b660 tx=0x7fa51c00ba00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:48.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.061 7fa51affd700 1 -- 172.21.2.101:0/2079561444 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa51c009440 con 0x7fa52c18f950 2021-02-10T11:33:48.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.061 7fa51affd700 1 -- 172.21.2.101:0/2079561444 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c000c90 con 0x7fa52c18f950 2021-02-10T11:33:48.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.061 7fa530886700 1 -- 172.21.2.101:0/2079561444 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52c0b3d40 con 0x7fa52c18f950 2021-02-10T11:33:48.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.061 7fa530886700 1 -- 172.21.2.101:0/2079561444 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52c0b42d0 con 0x7fa52c18f950 2021-02-10T11:33:48.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.061 7fa51affd700 1 -- 172.21.2.101:0/2079561444 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa51c002030 con 0x7fa52c18f950 2021-02-10T11:33:48.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.062 7fa51affd700 1 -- 172.21.2.101:0/2079561444 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa51c0093e0 con 0x7fa52c18f950 2021-02-10T11:33:48.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.062 7fa51affd700 1 --2- 172.21.2.101:0/2079561444 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa510018730 0x7fa51001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:48.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.062 7fa51affd700 1 -- 172.21.2.101:0/2079561444 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa51c0327b0 con 0x7fa52c18f950 2021-02-10T11:33:48.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.062 7fa529e1b700 1 --2- 172.21.2.101:0/2079561444 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa510018730 0x7fa51001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:48.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.063 7fa529e1b700 1 --2- 172.21.2.101:0/2079561444 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa510018730 0x7fa51001ac10 crc :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:48.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.093 7fa530886700 1 -- 172.21.2.101:0/2079561444 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa52c0fe290 con 0x7fa52c18f950 2021-02-10T11:33:48.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.095 7fa51affd700 1 -- 172.21.2.101:0/2079561444 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa51c0093e0 con 0x7fa52c18f950 2021-02-10T11:33:48.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.223 7fa530886700 1 -- 172.21.2.101:0/2079561444 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7fa52c0fe290 con 0x7fa52c18f950 2021-02-10T11:33:48.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.223 7fa51affd700 1 -- 172.21.2.101:0/2079561444 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v55) v1 ==== 104+0+10 (secure 0 0 0) 0x7fa51c00bdf0 con 0x7fa52c18f950 2021-02-10T11:33:48.235 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:33:48.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.225 7fa530886700 1 -- 172.21.2.101:0/2079561444 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa510018730 msgr2=0x7fa51001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:48.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.225 7fa530886700 1 --2- 172.21.2.101:0/2079561444 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa510018730 0x7fa51001ac10 crc :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:48.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.225 7fa530886700 1 -- 172.21.2.101:0/2079561444 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa52c18f950 msgr2=0x7fa52c0a3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:48.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.225 7fa530886700 1 --2- 172.21.2.101:0/2079561444 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa52c18f950 0x7fa52c0a3490 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7fa51c00b660 tx=0x7fa51c00ba00).stop 2021-02-10T11:33:48.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.225 7fa530886700 1 -- 172.21.2.101:0/2079561444 shutdown_connections 2021-02-10T11:33:48.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.225 7fa530886700 1 --2- 172.21.2.101:0/2079561444 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa510018730 0x7fa51001ac10 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:48.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.225 7fa530886700 1 --2- 172.21.2.101:0/2079561444 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa52c0b72d0 0x7fa52c0b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:48.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.225 7fa530886700 1 --2- 172.21.2.101:0/2079561444 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa52c0a9a40 0x7fa52c0a9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:48.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.225 7fa530886700 1 --2- 172.21.2.101:0/2079561444 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa52c18f950 0x7fa52c0a3490 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:48.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.225 7fa530886700 1 -- 172.21.2.101:0/2079561444 >> 172.21.2.101:0/2079561444 conn(0x7fa52c0a3e70 msgr2=0x7fa52c110860 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:48.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.226 7fa530886700 1 -- 172.21.2.101:0/2079561444 shutdown_connections 2021-02-10T11:33:48.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.226 7fa530886700 1 -- 172.21.2.101:0/2079561444 wait complete. 2021-02-10T11:33:48.245 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:33:48.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.647 7ff126add700 1 Processor -- start 2021-02-10T11:33:48.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.647 7ff126add700 1 -- start start 2021-02-10T11:33:48.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.648 7ff126add700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff120082920 0x7ff1200832d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:48.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.648 7ff126add700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1200838f0 0x7ff120083cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:48.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.648 7ff126add700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff120082d50 con 0x7ff1200e53d0 2021-02-10T11:33:48.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.648 7ff126add700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1200e57d0 con 0x7ff120082920 2021-02-10T11:33:48.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.648 7ff126add700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1201b0d30 con 0x7ff1200838f0 2021-02-10T11:33:48.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.648 7ff1248f9700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff1200e53d0 0x7ff1201b0540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52510/0 (socket says 172.21.2.101:52510) 2021-02-10T11:33:48.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.648 7ff1248f9700 1 -- 172.21.2.101:0/1710135651 learned_addr learned my addr 172.21.2.101:0/1710135651 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:48.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.648 7ff11ffff700 1 --2- 172.21.2.101:0/1710135651 >> v2:172.21.2.101:3300/0 conn(0x7ff120082920 0x7ff1200832d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:48.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.648 7ff11ffff700 1 -- 172.21.2.101:0/1710135651 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1200838f0 msgr2=0x7ff120083cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:33:48.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.648 7ff11ffff700 1 --2- 172.21.2.101:0/1710135651 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1200838f0 0x7ff120083cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:48.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.648 7ff11ffff700 1 -- 172.21.2.101:0/1710135651 >> v1:172.21.2.101:6789/0 conn(0x7ff1200e53d0 legacy=0x7ff1201b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:48.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.648 7ff11ffff700 1 -- 172.21.2.101:0/1710135651 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff120084310 con 0x7ff120082920 2021-02-10T11:33:48.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.648 7ff11ffff700 1 --2- 172.21.2.101:0/1710135651 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff120082920 0x7ff1200832d0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7ff110006650 tx=0x7ff110009a80).ready entity=mon.0 client_cookie=7985b6e10d3dc4ef server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:48.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.649 7ff11f7fe700 1 -- 172.21.2.101:0/1710135651 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1100009b0 con 0x7ff120082920 2021-02-10T11:33:48.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.649 7ff11f7fe700 1 -- 172.21.2.101:0/1710135651 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11000c0c0 con 0x7ff120082920 2021-02-10T11:33:48.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.649 7ff11f7fe700 1 -- 172.21.2.101:0/1710135651 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff11000c5a0 con 0x7ff120082920 2021-02-10T11:33:48.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.649 7ff126add700 1 -- 172.21.2.101:0/1710135651 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff120082920 msgr2=0x7ff1200832d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:48.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.649 7ff126add700 1 --2- 172.21.2.101:0/1710135651 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff120082920 0x7ff1200832d0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7ff110006650 tx=0x7ff110009a80).stop 2021-02-10T11:33:48.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.649 7ff126add700 1 -- 172.21.2.101:0/1710135651 shutdown_connections 2021-02-10T11:33:48.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.649 7ff126add700 1 --2- 172.21.2.101:0/1710135651 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1200838f0 0x7ff120083cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:48.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.649 7ff126add700 1 --2- 172.21.2.101:0/1710135651 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff120082920 0x7ff1200832d0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:48.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.649 7ff126add700 1 -- 172.21.2.101:0/1710135651 >> 172.21.2.101:0/1710135651 conn(0x7ff1200ce4f0 msgr2=0x7ff1200e4600 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:48.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.649 7ff126add700 1 -- 172.21.2.101:0/1710135651 shutdown_connections 2021-02-10T11:33:48.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.649 7ff126add700 1 -- 172.21.2.101:0/1710135651 wait complete. 2021-02-10T11:33:48.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.664 7ff126add700 1 Processor -- start 2021-02-10T11:33:48.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.664 7ff126add700 1 -- start start 2021-02-10T11:33:48.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.664 7ff126add700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1200b5280 0x7ff1200c0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:48.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.664 7ff126add700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff12009ac30 0x7ff12009b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:48.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.664 7ff126add700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1200ab620 0x7ff1200aba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:48.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.664 7ff126add700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12009b710 con 0x7ff12009ac30 2021-02-10T11:33:48.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.664 7ff126add700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12009bc90 con 0x7ff1200b5280 2021-02-10T11:33:48.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.664 7ff126add700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1200b4de0 con 0x7ff1200ab620 2021-02-10T11:33:48.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.665 7ff11ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff12009ac30 0x7ff12009b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:48.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.665 7ff11ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff12009ac30 0x7ff12009b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52020/0 (socket says 172.21.2.101:52020) 2021-02-10T11:33:48.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.665 7ff11ffff700 1 -- 172.21.2.101:0/2800859696 learned_addr learned my addr 172.21.2.101:0/2800859696 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:48.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.665 7ff11ffff700 1 -- 172.21.2.101:0/2800859696 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1200ab620 msgr2=0x7ff1200aba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:48.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.665 7ff11ffff700 1 --2- 172.21.2.101:0/2800859696 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1200ab620 0x7ff1200aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:48.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.665 7ff11ffff700 1 -- 172.21.2.101:0/2800859696 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1200b5280 msgr2=0x7ff1200c0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:48.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.665 7ff11ffff700 1 --2- 172.21.2.101:0/2800859696 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1200b5280 0x7ff1200c0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:48.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.665 7ff11ffff700 1 -- 172.21.2.101:0/2800859696 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1200ac040 con 0x7ff12009ac30 2021-02-10T11:33:48.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.665 7ff11ffff700 1 --2- 172.21.2.101:0/2800859696 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff12009ac30 0x7ff12009b030 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7ff11000b5e0 tx=0x7ff11000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:48.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.665 7ff11d7fa700 1 -- 172.21.2.101:0/2800859696 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff11000bd70 con 0x7ff12009ac30 2021-02-10T11:33:48.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.665 7ff11d7fa700 1 -- 172.21.2.101:0/2800859696 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff110009290 con 0x7ff12009ac30 2021-02-10T11:33:48.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.665 7ff11d7fa700 1 -- 172.21.2.101:0/2800859696 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff110002300 con 0x7ff12009ac30 2021-02-10T11:33:48.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.665 7ff126add700 1 -- 172.21.2.101:0/2800859696 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1200ac040 con 0x7ff12009ac30 2021-02-10T11:33:48.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.665 7ff126add700 1 -- 172.21.2.101:0/2800859696 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1200ac5d0 con 0x7ff12009ac30 2021-02-10T11:33:48.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.666 7ff11d7fa700 1 -- 172.21.2.101:0/2800859696 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff110000c90 con 0x7ff12009ac30 2021-02-10T11:33:48.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.666 7ff11d7fa700 1 --2- 172.21.2.101:0/2800859696 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1040186c0 0x7ff10401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:48.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.666 7ff11d7fa700 1 -- 172.21.2.101:0/2800859696 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff110032570 con 0x7ff12009ac30 2021-02-10T11:33:48.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.666 7ff1248f9700 1 --2- 172.21.2.101:0/2800859696 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1040186c0 0x7ff10401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:48.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.666 7ff1248f9700 1 --2- 172.21.2.101:0/2800859696 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1040186c0 0x7ff10401abc0 crc :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:48.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.698 7ff126add700 1 -- 172.21.2.101:0/2800859696 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1200ed680 con 0x7ff12009ac30 2021-02-10T11:33:48.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.700 7ff11d7fa700 1 -- 172.21.2.101:0/2800859696 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff110030da0 con 0x7ff12009ac30 2021-02-10T11:33:48.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.828 7ff126add700 1 -- 172.21.2.101:0/2800859696 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7ff120082d50 con 0x7ff12009ac30 2021-02-10T11:33:48.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.828 7ff11d7fa700 1 -- 172.21.2.101:0/2800859696 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v55) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff110000c90 con 0x7ff12009ac30 2021-02-10T11:33:48.840 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:33:48.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.830 7ff126add700 1 -- 172.21.2.101:0/2800859696 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1040186c0 msgr2=0x7ff10401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:48.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.830 7ff126add700 1 --2- 172.21.2.101:0/2800859696 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1040186c0 0x7ff10401abc0 crc :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:48.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.830 7ff126add700 1 -- 172.21.2.101:0/2800859696 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff12009ac30 msgr2=0x7ff12009b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:48.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.830 7ff126add700 1 --2- 172.21.2.101:0/2800859696 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff12009ac30 0x7ff12009b030 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7ff11000b5e0 tx=0x7ff11000b980).stop 2021-02-10T11:33:48.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.831 7ff126add700 1 -- 172.21.2.101:0/2800859696 shutdown_connections 2021-02-10T11:33:48.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.831 7ff126add700 1 --2- 172.21.2.101:0/2800859696 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1040186c0 0x7ff10401abc0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:48.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.831 7ff126add700 1 --2- 172.21.2.101:0/2800859696 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1200ab620 0x7ff1200aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:48.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.831 7ff126add700 1 --2- 172.21.2.101:0/2800859696 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff12009ac30 0x7ff12009b030 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:48.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.831 7ff126add700 1 --2- 172.21.2.101:0/2800859696 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1200b5280 0x7ff1200c0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:48.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.831 7ff126add700 1 -- 172.21.2.101:0/2800859696 >> 172.21.2.101:0/2800859696 conn(0x7ff1200a38c0 msgr2=0x7ff1200a2560 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:48.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.831 7ff126add700 1 -- 172.21.2.101:0/2800859696 shutdown_connections 2021-02-10T11:33:48.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:48.831 7ff126add700 1 -- 172.21.2.101:0/2800859696 wait complete. 2021-02-10T11:33:48.850 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:33:48.850 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:33:49.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.248 7f5c2b8a3700 1 Processor -- start 2021-02-10T11:33:49.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.248 7f5c2b8a3700 1 -- start start 2021-02-10T11:33:49.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.249 7f5c2b8a3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5c24082920 0x7f5c240832d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:49.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.249 7f5c2b8a3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c240838f0 0x7f5c24083cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:49.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.249 7f5c2b8a3700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c24082d50 con 0x7f5c240e53e0 2021-02-10T11:33:49.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.249 7f5c2b8a3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c240e57e0 con 0x7f5c24082920 2021-02-10T11:33:49.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.249 7f5c2b8a3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c241b0d40 con 0x7f5c240838f0 2021-02-10T11:33:49.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.249 7f5c28ebe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5c24082920 0x7f5c240832d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:49.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.249 7f5c296bf700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f5c240e53e0 0x7f5c241b0550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52522/0 (socket says 172.21.2.101:52522) 2021-02-10T11:33:49.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.249 7f5c28ebe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5c24082920 0x7f5c240832d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52028/0 (socket says 172.21.2.101:52028) 2021-02-10T11:33:49.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.249 7f5c296bf700 1 -- 172.21.2.101:0/1900867193 learned_addr learned my addr 172.21.2.101:0/1900867193 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:49.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.249 7f5c28ebe700 1 -- 172.21.2.101:0/1900867193 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c240838f0 msgr2=0x7f5c24083cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:33:49.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.249 7f5c28ebe700 1 --2- 172.21.2.101:0/1900867193 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c240838f0 0x7f5c24083cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:49.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.249 7f5c28ebe700 1 -- 172.21.2.101:0/1900867193 >> v1:172.21.2.101:6789/0 conn(0x7f5c240e53e0 legacy=0x7f5c241b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:49.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.249 7f5c28ebe700 1 -- 172.21.2.101:0/1900867193 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c24084310 con 0x7f5c24082920 2021-02-10T11:33:49.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.249 7f5c17fff700 1 -- 172.21.2.101:0/1900867193 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2453653605 0 0) 0x7f5c180010b0 con 0x7f5c240e53e0 2021-02-10T11:33:49.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.250 7f5c28ebe700 1 --2- 172.21.2.101:0/1900867193 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c24082920 0x7f5c240832d0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7f5c20006650 tx=0x7f5c20009a80).ready entity=mon.0 client_cookie=69c50566e4b8f983 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:49.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.250 7f5c17fff700 1 -- 172.21.2.101:0/1900867193 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5c200009b0 con 0x7f5c24082920 2021-02-10T11:33:49.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.250 7f5c17fff700 1 -- 172.21.2.101:0/1900867193 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2000c0c0 con 0x7f5c24082920 2021-02-10T11:33:49.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.250 7f5c17fff700 1 -- 172.21.2.101:0/1900867193 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5c2000c5e0 con 0x7f5c24082920 2021-02-10T11:33:49.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.250 7f5c2b8a3700 1 -- 172.21.2.101:0/1900867193 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c24082920 msgr2=0x7f5c240832d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:49.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.250 7f5c2b8a3700 1 --2- 172.21.2.101:0/1900867193 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c24082920 0x7f5c240832d0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7f5c20006650 tx=0x7f5c20009a80).stop 2021-02-10T11:33:49.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.250 7f5c2b8a3700 1 -- 172.21.2.101:0/1900867193 shutdown_connections 2021-02-10T11:33:49.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.251 7f5c2b8a3700 1 --2- 172.21.2.101:0/1900867193 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c240838f0 0x7f5c24083cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:49.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.251 7f5c2b8a3700 1 --2- 172.21.2.101:0/1900867193 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c24082920 0x7f5c240832d0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:49.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.251 7f5c2b8a3700 1 -- 172.21.2.101:0/1900867193 >> 172.21.2.101:0/1900867193 conn(0x7f5c240ce500 msgr2=0x7f5c240e4610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:49.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.251 7f5c2b8a3700 1 -- 172.21.2.101:0/1900867193 shutdown_connections 2021-02-10T11:33:49.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.251 7f5c2b8a3700 1 -- 172.21.2.101:0/1900867193 wait complete. 2021-02-10T11:33:49.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.265 7f5c2b8a3700 1 Processor -- start 2021-02-10T11:33:49.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.265 7f5c2b8a3700 1 -- start start 2021-02-10T11:33:49.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.266 7f5c2b8a3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c240ada90 0x7f5c240ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:49.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.266 7f5c2b8a3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c241aa540 0x7f5c241aa940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:49.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.266 7f5c2b8a3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5c241966f0 0x7f5c24196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:49.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.266 7f5c2b8a3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c241aaf60 con 0x7f5c241aa540 2021-02-10T11:33:49.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.266 7f5c2b8a3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c24197110 con 0x7f5c241966f0 2021-02-10T11:33:49.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.266 7f5c2b8a3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c24197330 con 0x7f5c240ada90 2021-02-10T11:33:49.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.266 7f5c28ebe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c241aa540 0x7f5c241aa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:49.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.266 7f5c28ebe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c241aa540 0x7f5c241aa940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52038/0 (socket says 172.21.2.101:52038) 2021-02-10T11:33:49.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.266 7f5c28ebe700 1 -- 172.21.2.101:0/607955795 learned_addr learned my addr 172.21.2.101:0/607955795 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:49.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.266 7f5c28ebe700 1 -- 172.21.2.101:0/607955795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c240ada90 msgr2=0x7f5c240ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:49.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.266 7f5c28ebe700 1 --2- 172.21.2.101:0/607955795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c240ada90 0x7f5c240ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:49.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.266 7f5c28ebe700 1 -- 172.21.2.101:0/607955795 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5c241966f0 msgr2=0x7f5c24196af0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:49.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.266 7f5c28ebe700 1 --2- 172.21.2.101:0/607955795 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5c241966f0 0x7f5c24196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:49.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.266 7f5c28ebe700 1 -- 172.21.2.101:0/607955795 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c24197550 con 0x7f5c241aa540 2021-02-10T11:33:49.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.266 7f5c28ebe700 1 --2- 172.21.2.101:0/607955795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c241aa540 0x7f5c241aa940 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x7f5c20009630 tx=0x7f5c2000b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:49.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.267 7f5c15ffb700 1 -- 172.21.2.101:0/607955795 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5c200093a0 con 0x7f5c241aa540 2021-02-10T11:33:49.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.267 7f5c15ffb700 1 -- 172.21.2.101:0/607955795 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2000bcf0 con 0x7f5c241aa540 2021-02-10T11:33:49.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.267 7f5c15ffb700 1 -- 172.21.2.101:0/607955795 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5c20005560 con 0x7f5c241aa540 2021-02-10T11:33:49.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.267 7f5c2b8a3700 1 -- 172.21.2.101:0/607955795 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c24197550 con 0x7f5c241aa540 2021-02-10T11:33:49.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.267 7f5c2b8a3700 1 -- 172.21.2.101:0/607955795 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c241981f0 con 0x7f5c241aa540 2021-02-10T11:33:49.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.267 7f5c15ffb700 1 -- 172.21.2.101:0/607955795 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5c20008240 con 0x7f5c241aa540 2021-02-10T11:33:49.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.267 7f5c15ffb700 1 --2- 172.21.2.101:0/607955795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5c040186c0 0x7f5c0401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:49.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.267 7f5c15ffb700 1 -- 172.21.2.101:0/607955795 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5c20032500 con 0x7f5c241aa540 2021-02-10T11:33:49.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.267 7f5c296bf700 1 --2- 172.21.2.101:0/607955795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5c040186c0 0x7f5c0401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:49.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.268 7f5c296bf700 1 --2- 172.21.2.101:0/607955795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5c040186c0 0x7f5c0401abc0 crc :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:49.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.299 7f5c2b8a3700 1 -- 172.21.2.101:0/607955795 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c240ed630 con 0x7f5c241aa540 2021-02-10T11:33:49.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.301 7f5c15ffb700 1 -- 172.21.2.101:0/607955795 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5c2002d010 con 0x7f5c241aa540 2021-02-10T11:33:49.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.429 7f5c2b8a3700 1 -- 172.21.2.101:0/607955795 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c240f3a10 con 0x7f5c241aa540 2021-02-10T11:33:49.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.430 7f5c15ffb700 1 -- 172.21.2.101:0/607955795 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+2098 (secure 0 0 0) 0x7f5c2002d010 con 0x7f5c241aa540 2021-02-10T11:33:49.441 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 39 2021-02-10T11:33:49.441 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:33:49.441 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8049,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/172865610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":172865610},{"type":"v1","addr":"172.21.2.101:6835","nonce":172865610}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:33:16.590282","modified":"2021-02-10 11:33:18.594009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5579},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5579":{"gid":5579,"name":"b","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2400978339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2400978339},{"type":"v1","addr":"172.21.2.108:6833","nonce":2400978339}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-02-10T11:33:49.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.431 7f5c2b8a3700 1 -- 172.21.2.101:0/607955795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5c040186c0 msgr2=0x7f5c0401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:49.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.431 7f5c2b8a3700 1 --2- 172.21.2.101:0/607955795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5c040186c0 0x7f5c0401abc0 crc :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:49.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.431 7f5c2b8a3700 1 -- 172.21.2.101:0/607955795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c241aa540 msgr2=0x7f5c241aa940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:49.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.431 7f5c2b8a3700 1 --2- 172.21.2.101:0/607955795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c241aa540 0x7f5c241aa940 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x7f5c20009630 tx=0x7f5c2000b900).stop 2021-02-10T11:33:49.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.432 7f5c2b8a3700 1 -- 172.21.2.101:0/607955795 shutdown_connections 2021-02-10T11:33:49.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.432 7f5c2b8a3700 1 --2- 172.21.2.101:0/607955795 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5c241966f0 0x7f5c24196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:49.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.432 7f5c2b8a3700 1 --2- 172.21.2.101:0/607955795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c241aa540 0x7f5c241aa940 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:49.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.432 7f5c2b8a3700 1 --2- 172.21.2.101:0/607955795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5c040186c0 0x7f5c0401abc0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:49.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.432 7f5c2b8a3700 1 --2- 172.21.2.101:0/607955795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c240ada90 0x7f5c240ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:49.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.432 7f5c2b8a3700 1 -- 172.21.2.101:0/607955795 >> 172.21.2.101:0/607955795 conn(0x7f5c24092b10 msgr2=0x7f5c2418e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:49.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.432 7f5c2b8a3700 1 -- 172.21.2.101:0/607955795 shutdown_connections 2021-02-10T11:33:49.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:49.432 7f5c2b8a3700 1 -- 172.21.2.101:0/607955795 wait complete. 2021-02-10T11:33:49.653 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:33:16.590282', 'modified': '2021-02-10 11:33:18.594009', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5579}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5579': {'gid': 5579, 'name': 'b', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/2400978339', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 2400978339}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 2400978339}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:33:49.653 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:33:49.653 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:33:50.052 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:33:50.052 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:33:50.052 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:33:50.052 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 5579, 2021-02-10T11:33:50.052 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:33:50.052 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:33:50.052 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 33.449309065000001, 2021-02-10T11:33:50.052 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 38, 2021-02-10T11:33:50.052 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 55, 2021-02-10T11:33:50.052 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 55, 2021-02-10T11:33:50.053 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 39.082200151000002 2021-02-10T11:33:50.053 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:33:50.056 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 5579, "want_state": "up:active", "state": "up:active", "rank_uptime": 33.449309065000001, "mdsmap_epoch": 38, "osdmap_epoch": 55, "osdmap_epoch_barrier": 55, "uptime": 39.082200151000002 } 2021-02-10T11:33:50.056 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:33:50.056 INFO:tasks.cephfs.fuse_mount:Client client.alice config is {} 2021-02-10T11:33:50.056 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.alice at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:33:50.056 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:50.056 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:33:50.393 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:33:50.735 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:33:50.735 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:33:51.077 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:33:51.078 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:33:51.078 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:33:51.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:51.079 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:33:51.414 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:33:51.414 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2021-02-10T11:33:51.414 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --client_mountpoint=/volumes/group_0000000000586931/subvolume_0000000000130711 --name client.alice /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:33:51.418 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:33:51.762 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:33:51.762 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:33:51.828 INFO:tasks.cephfs.fuse_mount.ceph-fuse.alice.gibba001.stderr:ceph-fuse[120270]: starting ceph client 2021-02-10T11:33:51.828 INFO:tasks.cephfs.fuse_mount.ceph-fuse.alice.gibba001.stderr:2021-02-10 11:33:51.817 7f26dc524f80 -1 init, newargv = 0x5572b6a2d450 newargc=9 2021-02-10T11:33:51.835 INFO:tasks.cephfs.fuse_mount.ceph-fuse.alice.gibba001.stderr:ceph-fuse[120270]: starting fuse 2021-02-10T11:33:52.104 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:33:52.104 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:33:52.105 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:33:52.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:52.105 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:33:52.440 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:33:52.440 INFO:teuthology.orchestra.run.gibba001.stdout:44 2021-02-10T11:33:52.440 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2021-02-10T11:33:52.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:52.440 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:33:52.441 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:33:52.441 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:33:52.441 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:33:52.441 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:33:52.441 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:33:52.441 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:33:52.441 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.alice.*.asok" 2021-02-10T11:33:52.441 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:33:52.441 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:33:52.441 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:33:52.441 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:33:52.441 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:33:52.441 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:33:52.441 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:33:52.442 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:33:52.442 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:33:52.442 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:33:52.442 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:33:52.442 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:33:52.442 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:33:52.442 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.alice")) 2021-02-10T11:33:52.442 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:33:52.804 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.alice.120270.asok 2021-02-10T11:33:52.806 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.alice.120270.asok 2021-02-10T11:33:52.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:52.806 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.alice.120270.asok status 2021-02-10T11:33:53.202 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:33:53.202 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:33:53.203 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:33:53.203 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:33:53.203 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "alice", 2021-02-10T11:33:53.203 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:33:53.203 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:33:53.203 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "120270", 2021-02-10T11:33:53.203 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/volumes/group_0000000000586931/subvolume_0000000000130711" 2021-02-10T11:33:53.203 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:53.203 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:33:53.203 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:33:53.203 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 8331, 2021-02-10T11:33:53.203 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:33:53.203 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:33:53.204 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:33:53.204 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 8331 2021-02-10T11:33:53.204 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:53.204 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:33:53.204 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:53.204 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:33:53.204 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 3664896613 2021-02-10T11:33:53.204 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:33:53.204 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:53.204 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:33:53.204 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:33:53.205 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:33:53.205 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 3664896613 2021-02-10T11:33:53.205 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:33:53.205 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.8331 172.21.2.101:0/3664896613", 2021-02-10T11:33:53.205 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/3664896613", 2021-02-10T11:33:53.205 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:33:53.205 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 38, 2021-02-10T11:33:53.205 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 55, 2021-02-10T11:33:53.205 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:33:53.205 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:33:53.206 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:33:53.207 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:53.207 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:33:53.207 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:33:53.543 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:33:53.543 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:33:53.543 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.1 && sudo dd if=/dev/urandom of=data.bin bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:33:53.918 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:33:53.919 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:33:53.919 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0249683 s, 42.0 MB/s 2021-02-10T11:33:53.920 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume authorize cephfs subvolume_0000000000130711 alice --group_name group_0000000000586931 --tenant_id tenant_id --access_level r 2021-02-10T11:33:54.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.331 7f02412c4700 1 Processor -- start 2021-02-10T11:33:54.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.331 7f02412c4700 1 -- start start 2021-02-10T11:33:54.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.331 7f02412c4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f023c0e5570 0x7f023c078520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:54.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.331 7f02412c4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f023c079110 0x7f023c079510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:54.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.331 7f02412c4700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f023c078b90 con 0x7f023c0e5170 2021-02-10T11:33:54.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.331 7f02412c4700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f023c1b40d0 con 0x7f023c079110 2021-02-10T11:33:54.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.331 7f02412c4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f023c0d1900 con 0x7f023c0e5570 2021-02-10T11:33:54.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.331 7f023b87e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f023c079110 0x7f023c079510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:54.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.331 7f023b07d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f023c0e5170 0x7f023c1b38e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52558/0 (socket says 172.21.2.101:52558) 2021-02-10T11:33:54.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.331 7f023b07d700 1 -- 172.21.2.101:0/577124377 learned_addr learned my addr 172.21.2.101:0/577124377 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:33:54.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.332 7f023b87e700 1 -- 172.21.2.101:0/577124377 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f023c0e5570 msgr2=0x7f023c078520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:33:54.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.332 7f023b87e700 1 --2- 172.21.2.101:0/577124377 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f023c0e5570 0x7f023c078520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:54.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.332 7f023b87e700 1 -- 172.21.2.101:0/577124377 >> v1:172.21.2.101:6789/0 conn(0x7f023c0e5170 legacy=0x7f023c1b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:54.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.332 7f023b87e700 1 -- 172.21.2.101:0/577124377 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023c0d1b20 con 0x7f023c079110 2021-02-10T11:33:54.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.332 7f023b87e700 1 --2- 172.21.2.101:0/577124377 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f023c079110 0x7f023c079510 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f0234006e30 tx=0x7f0234009e60).ready entity=mon.0 client_cookie=5b52608d0d397407 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:54.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.332 7f023a07b700 1 -- 172.21.2.101:0/577124377 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0234006a50 con 0x7f023c079110 2021-02-10T11:33:54.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.332 7f023a07b700 1 -- 172.21.2.101:0/577124377 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023400b300 con 0x7f023c079110 2021-02-10T11:33:54.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.332 7f023a07b700 1 -- 172.21.2.101:0/577124377 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f023400b7e0 con 0x7f023c079110 2021-02-10T11:33:54.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.332 7f02412c4700 1 -- 172.21.2.101:0/577124377 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f023c079110 msgr2=0x7f023c079510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:54.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.332 7f02412c4700 1 --2- 172.21.2.101:0/577124377 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f023c079110 0x7f023c079510 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f0234006e30 tx=0x7f0234009e60).stop 2021-02-10T11:33:54.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.333 7f02412c4700 1 -- 172.21.2.101:0/577124377 shutdown_connections 2021-02-10T11:33:54.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.333 7f02412c4700 1 --2- 172.21.2.101:0/577124377 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f023c0e5570 0x7f023c078520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:54.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.333 7f02412c4700 1 --2- 172.21.2.101:0/577124377 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f023c079110 0x7f023c079510 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:54.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.333 7f02412c4700 1 -- 172.21.2.101:0/577124377 >> 172.21.2.101:0/577124377 conn(0x7f023c0ce820 msgr2=0x7f023c0e4660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:54.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.333 7f02412c4700 1 -- 172.21.2.101:0/577124377 shutdown_connections 2021-02-10T11:33:54.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.333 7f02412c4700 1 -- 172.21.2.101:0/577124377 wait complete. 2021-02-10T11:33:54.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.347 7f02412c4700 1 Processor -- start 2021-02-10T11:33:54.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.348 7f02412c4700 1 -- start start 2021-02-10T11:33:54.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.348 7f02412c4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f023c1a23b0 0x7f023c10ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:54.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.348 7f02412c4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f023c1a1930 0x7f023c1a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:54.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.348 7f02412c4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f023c1a3400 0x7f023c093f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:54.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.348 7f02412c4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023c1a1e30 con 0x7f023c1a1930 2021-02-10T11:33:54.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.348 7f02412c4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023c1a27b0 con 0x7f023c1a23b0 2021-02-10T11:33:54.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.348 7f02412c4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f023c10d430 con 0x7f023c1a3400 2021-02-10T11:33:54.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.348 7f023a87c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f023c1a1930 0x7f023c1a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:54.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.348 7f023a87c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f023c1a1930 0x7f023c1a2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52072/0 (socket says 172.21.2.101:52072) 2021-02-10T11:33:54.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.348 7f023a87c700 1 -- 172.21.2.101:0/14653654 learned_addr learned my addr 172.21.2.101:0/14653654 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:33:54.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.348 7f023a87c700 1 -- 172.21.2.101:0/14653654 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f023c1a3400 msgr2=0x7f023c093f50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:54.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.348 7f023a87c700 1 --2- 172.21.2.101:0/14653654 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f023c1a3400 0x7f023c093f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:54.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.348 7f023a87c700 1 -- 172.21.2.101:0/14653654 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f023c1a23b0 msgr2=0x7f023c10ce10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:33:54.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.348 7f023a87c700 1 --2- 172.21.2.101:0/14653654 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f023c1a23b0 0x7f023c10ce10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:54.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.348 7f023a87c700 1 -- 172.21.2.101:0/14653654 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023c094570 con 0x7f023c1a1930 2021-02-10T11:33:54.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.349 7f023a87c700 1 --2- 172.21.2.101:0/14653654 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f023c1a1930 0x7f023c1a2de0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7f023000b560 tx=0x7f023000b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:54.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.349 7f0223fff700 1 -- 172.21.2.101:0/14653654 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f023000bcf0 con 0x7f023c1a1930 2021-02-10T11:33:54.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.349 7f0223fff700 1 -- 172.21.2.101:0/14653654 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0230006030 con 0x7f023c1a1930 2021-02-10T11:33:54.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.349 7f0223fff700 1 -- 172.21.2.101:0/14653654 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f02300054d0 con 0x7f023c1a1930 2021-02-10T11:33:54.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.349 7f02412c4700 1 -- 172.21.2.101:0/14653654 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f023c094570 con 0x7f023c1a1930 2021-02-10T11:33:54.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.349 7f02412c4700 1 -- 172.21.2.101:0/14653654 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f023c094b00 con 0x7f023c1a1930 2021-02-10T11:33:54.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.349 7f0223fff700 1 -- 172.21.2.101:0/14653654 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0230006210 con 0x7f023c1a1930 2021-02-10T11:33:54.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.349 7f0223fff700 1 --2- 172.21.2.101:0/14653654 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f021c0186c0 0x7f021c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:33:54.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.349 7f0223fff700 1 -- 172.21.2.101:0/14653654 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0230032650 con 0x7f023c1a1930 2021-02-10T11:33:54.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.350 7f023b07d700 1 --2- 172.21.2.101:0/14653654 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f021c0186c0 0x7f021c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:33:54.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.350 7f023b07d700 1 --2- 172.21.2.101:0/14653654 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f021c0186c0 0x7f021c01abc0 crc :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:33:54.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.381 7f02412c4700 1 -- 172.21.2.101:0/14653654 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f023c10bc10 con 0x7f023c1a1930 2021-02-10T11:33:54.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.384 7f0223fff700 1 -- 172.21.2.101:0/14653654 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0230006b80 con 0x7f023c1a1930 2021-02-10T11:33:54.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.509 7f02412c4700 1 -- 172.21.2.101:0/14653654 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"vol_name": "cephfs", "sub_name": "subvolume_0000000000130711", "target": ["mgr", ""], "tenant_id": "tenant_id", "group_name": "group_0000000000586931", "access_level": "r", "prefix": "fs subvolume authorize", "auth_id": "alice"}) v1 -- 0x7f023c1a1e30 con 0x7f021c0186c0 2021-02-10T11:33:54.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.552 7f0223fff700 1 -- 172.21.2.101:0/14653654 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7f022c007690 con 0x7f021c0186c0 2021-02-10T11:33:54.584 INFO:teuthology.orchestra.run.gibba001.stdout:AQCXxCNgV7T2LhAAoiEuMPF+Yh/GnfJFmgWXPQ== 2021-02-10T11:33:54.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.575 7f02412c4700 1 -- 172.21.2.101:0/14653654 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f021c0186c0 msgr2=0x7f021c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:54.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.575 7f02412c4700 1 --2- 172.21.2.101:0/14653654 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f021c0186c0 0x7f021c01abc0 crc :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:54.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.575 7f02412c4700 1 -- 172.21.2.101:0/14653654 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f023c1a1930 msgr2=0x7f023c1a2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:33:54.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.575 7f02412c4700 1 --2- 172.21.2.101:0/14653654 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f023c1a1930 0x7f023c1a2de0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7f023000b560 tx=0x7f023000b900).stop 2021-02-10T11:33:54.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.575 7f02412c4700 1 -- 172.21.2.101:0/14653654 shutdown_connections 2021-02-10T11:33:54.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.575 7f02412c4700 1 --2- 172.21.2.101:0/14653654 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f021c0186c0 0x7f021c01abc0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:54.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.575 7f02412c4700 1 --2- 172.21.2.101:0/14653654 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f023c1a3400 0x7f023c093f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:54.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.575 7f02412c4700 1 --2- 172.21.2.101:0/14653654 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f023c1a1930 0x7f023c1a2de0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:33:54.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.575 7f02412c4700 1 --2- 172.21.2.101:0/14653654 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f023c1a23b0 0x7f023c10ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:33:54.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.575 7f02412c4700 1 -- 172.21.2.101:0/14653654 >> 172.21.2.101:0/14653654 conn(0x7f023c19e600 msgr2=0x7f023c192b40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:33:54.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.575 7f02412c4700 1 -- 172.21.2.101:0/14653654 shutdown_connections 2021-02-10T11:33:54.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:33:54.575 7f02412c4700 1 -- 172.21.2.101:0/14653654 wait complete. 2021-02-10T11:33:54.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:54.595 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:33:54.595 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:33:54.930 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:33:54.930 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:33:54.930 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:33:54.930 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:33:54.930 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:33:55.284 INFO:tasks.cephfs.fuse_mount.ceph-fuse.alice.gibba001.stderr:ceph-fuse[120270]: fuse finished with error 0 and tester_r 0 2021-02-10T11:33:55.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:33:55.286 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:33:55.286 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:33:55.621 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:33:55.621 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:33:55.621 INFO:teuthology.orchestra.run:waiting for 900 2021-02-10T11:33:58.946 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:33:58.950 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:34:01.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:34:01.724 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:34:02.061 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:34:02.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.460 7f975c5d7700 1 Processor -- start 2021-02-10T11:34:02.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.460 7f975c5d7700 1 -- start start 2021-02-10T11:34:02.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.461 7f975c5d7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97540e53e0 0x7f97541b0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:02.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.461 7f975c5d7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f97541b0b70 0x7f9754081910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:02.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.461 7f975c5d7700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9754082d50 con 0x7f9754081f30 2021-02-10T11:34:02.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.461 7f975c5d7700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9754084850 con 0x7f97541b0b70 2021-02-10T11:34:02.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.461 7f975c5d7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9754082b20 con 0x7f97540e53e0 2021-02-10T11:34:02.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.461 7f9759bf2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f97541b0b70 0x7f9754081910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:02.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.461 7f9759bf2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f97541b0b70 0x7f9754081910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52080/0 (socket says 172.21.2.101:52080) 2021-02-10T11:34:02.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.461 7f9759bf2700 1 -- 172.21.2.101:0/2657730326 learned_addr learned my addr 172.21.2.101:0/2657730326 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:02.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.461 7f975a3f3700 1 --2- 172.21.2.101:0/2657730326 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97540e53e0 0x7f97541b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:02.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.462 7f9759bf2700 1 -- 172.21.2.101:0/2657730326 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97540e53e0 msgr2=0x7f97541b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:02.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.462 7f9759bf2700 1 --2- 172.21.2.101:0/2657730326 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97540e53e0 0x7f97541b0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:02.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.462 7f9759bf2700 1 -- 172.21.2.101:0/2657730326 >> v1:172.21.2.108:6789/0 conn(0x7f9754081f30 legacy=0x7f9754082330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:02.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.462 7f9759bf2700 1 -- 172.21.2.101:0/2657730326 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97540e5fe0 con 0x7f97541b0b70 2021-02-10T11:34:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.462 7f975a3f3700 1 --2- 172.21.2.101:0/2657730326 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97540e53e0 0x7f97541b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:34:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.462 7f9759bf2700 1 --2- 172.21.2.101:0/2657730326 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f97541b0b70 0x7f9754081910 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7f9748006700 tx=0x7f9748009b80).ready entity=mon.0 client_cookie=5d0ed8c97f3d7bb server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.462 7f97593f1700 1 -- 172.21.2.101:0/2657730326 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f974800c0c0 con 0x7f97541b0b70 2021-02-10T11:34:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.462 7f97593f1700 1 -- 172.21.2.101:0/2657730326 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974800c380 con 0x7f97541b0b70 2021-02-10T11:34:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.462 7f97593f1700 1 -- 172.21.2.101:0/2657730326 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f974800c8c0 con 0x7f97541b0b70 2021-02-10T11:34:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.462 7f975c5d7700 1 -- 172.21.2.101:0/2657730326 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f97541b0b70 msgr2=0x7f9754081910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.462 7f975c5d7700 1 --2- 172.21.2.101:0/2657730326 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f97541b0b70 0x7f9754081910 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7f9748006700 tx=0x7f9748009b80).stop 2021-02-10T11:34:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.463 7f975c5d7700 1 -- 172.21.2.101:0/2657730326 shutdown_connections 2021-02-10T11:34:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.463 7f975c5d7700 1 --2- 172.21.2.101:0/2657730326 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f97541b0b70 0x7f9754081910 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.463 7f975c5d7700 1 --2- 172.21.2.101:0/2657730326 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97540e53e0 0x7f97541b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.463 7f975c5d7700 1 -- 172.21.2.101:0/2657730326 >> 172.21.2.101:0/2657730326 conn(0x7f97540ce500 msgr2=0x7f97540e4610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.463 7f975c5d7700 1 -- 172.21.2.101:0/2657730326 shutdown_connections 2021-02-10T11:34:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.463 7f975c5d7700 1 -- 172.21.2.101:0/2657730326 wait complete. 2021-02-10T11:34:02.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.478 7f975c5d7700 1 Processor -- start 2021-02-10T11:34:02.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.478 7f975c5d7700 1 -- start start 2021-02-10T11:34:02.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.478 7f975c5d7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97541b7a70 0x7f97541b7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:02.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.478 7f975c5d7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97541b8490 0x7f9754097e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:02.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.478 7f975c5d7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9754098460 0x7f9754098860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:02.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.478 7f975c5d7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97541b8ff0 con 0x7f9754098460 2021-02-10T11:34:02.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.478 7f975c5d7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97541b8ae0 con 0x7f97541b7a70 2021-02-10T11:34:02.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.478 7f975c5d7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97540974e0 con 0x7f97541b8490 2021-02-10T11:34:02.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.479 7f975abf4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9754098460 0x7f9754098860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:02.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.479 7f975abf4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9754098460 0x7f9754098860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52086/0 (socket says 172.21.2.101:52086) 2021-02-10T11:34:02.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.479 7f975abf4700 1 -- 172.21.2.101:0/3760477431 learned_addr learned my addr 172.21.2.101:0/3760477431 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:02.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.479 7f975abf4700 1 -- 172.21.2.101:0/3760477431 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97541b8490 msgr2=0x7f9754097e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:02.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.479 7f975abf4700 1 --2- 172.21.2.101:0/3760477431 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97541b8490 0x7f9754097e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:02.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.479 7f975abf4700 1 -- 172.21.2.101:0/3760477431 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97541b7a70 msgr2=0x7f97541b7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:02.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.479 7f975abf4700 1 --2- 172.21.2.101:0/3760477431 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97541b7a70 0x7f97541b7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:02.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.479 7f975abf4700 1 -- 172.21.2.101:0/3760477431 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9754098e80 con 0x7f9754098460 2021-02-10T11:34:02.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.479 7f975abf4700 1 --2- 172.21.2.101:0/3760477431 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9754098460 0x7f9754098860 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7f974c006ed0 tx=0x7f974c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:02.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.479 7f9752ffd700 1 -- 172.21.2.101:0/3760477431 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f974c00bd50 con 0x7f9754098460 2021-02-10T11:34:02.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.479 7f9752ffd700 1 -- 172.21.2.101:0/3760477431 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974c000c10 con 0x7f9754098460 2021-02-10T11:34:02.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.479 7f9752ffd700 1 -- 172.21.2.101:0/3760477431 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f974c0062a0 con 0x7f9754098460 2021-02-10T11:34:02.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.479 7f975c5d7700 1 -- 172.21.2.101:0/3760477431 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9754098e80 con 0x7f9754098460 2021-02-10T11:34:02.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.480 7f975c5d7700 1 -- 172.21.2.101:0/3760477431 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9754098e80 con 0x7f9754098460 2021-02-10T11:34:02.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.480 7f9752ffd700 1 -- 172.21.2.101:0/3760477431 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f974c006020 con 0x7f9754098460 2021-02-10T11:34:02.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.480 7f9752ffd700 1 --2- 172.21.2.101:0/3760477431 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97340186c0 0x7f973401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:02.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.480 7f9752ffd700 1 -- 172.21.2.101:0/3760477431 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f974c032460 con 0x7f9754098460 2021-02-10T11:34:02.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.480 7f975a3f3700 1 --2- 172.21.2.101:0/3760477431 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97340186c0 0x7f973401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:02.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.481 7f975a3f3700 1 --2- 172.21.2.101:0/3760477431 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97340186c0 0x7f973401abc0 crc :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:02.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.511 7f975c5d7700 1 -- 172.21.2.101:0/3760477431 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97540ed660 con 0x7f9754098460 2021-02-10T11:34:02.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.513 7f9752ffd700 1 -- 172.21.2.101:0/3760477431 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f974c030d90 con 0x7f9754098460 2021-02-10T11:34:02.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.638 7f975c5d7700 1 -- 172.21.2.101:0/3760477431 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9754082d50 con 0x7f9754098460 2021-02-10T11:34:02.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.639 7f9752ffd700 1 -- 172.21.2.101:0/3760477431 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+13996 (secure 0 0 0) 0x7f974c006000 con 0x7f9754098460 2021-02-10T11:34:02.650 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:02.651 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":55,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:33:17.591716","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:33:14.515763","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2021-02-10 11:33:15.636274","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:34:02.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.641 7f975c5d7700 1 -- 172.21.2.101:0/3760477431 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97340186c0 msgr2=0x7f973401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:02.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.641 7f975c5d7700 1 --2- 172.21.2.101:0/3760477431 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97340186c0 0x7f973401abc0 crc :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:02.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.641 7f975c5d7700 1 -- 172.21.2.101:0/3760477431 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9754098460 msgr2=0x7f9754098860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:02.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.641 7f975c5d7700 1 --2- 172.21.2.101:0/3760477431 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9754098460 0x7f9754098860 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7f974c006ed0 tx=0x7f974c00b960).stop 2021-02-10T11:34:02.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.641 7f975c5d7700 1 -- 172.21.2.101:0/3760477431 shutdown_connections 2021-02-10T11:34:02.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.641 7f975c5d7700 1 --2- 172.21.2.101:0/3760477431 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97340186c0 0x7f973401abc0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:02.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.641 7f975c5d7700 1 --2- 172.21.2.101:0/3760477431 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97541b8490 0x7f9754097e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:02.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.641 7f975c5d7700 1 --2- 172.21.2.101:0/3760477431 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9754098460 0x7f9754098860 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:02.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.641 7f975c5d7700 1 --2- 172.21.2.101:0/3760477431 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97541b7a70 0x7f97541b7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:02.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.641 7f975c5d7700 1 -- 172.21.2.101:0/3760477431 >> 172.21.2.101:0/3760477431 conn(0x7f97540f2240 msgr2=0x7f9754094840 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:02.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.641 7f975c5d7700 1 -- 172.21.2.101:0/3760477431 shutdown_connections 2021-02-10T11:34:02.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:02.641 7f975c5d7700 1 -- 172.21.2.101:0/3760477431 wait complete. 2021-02-10T11:34:02.661 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:33:14.515763', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:33:15.636274', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:34:02.662 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:34:03.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.064 7f6702c79700 1 Processor -- start 2021-02-10T11:34:03.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.064 7f6702c79700 1 -- start start 2021-02-10T11:34:03.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.064 7f6702c79700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66fc0e53d0 0x7f66fc1b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:03.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.064 7f6702c79700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66fc082d50 con 0x7f66fc0842e0 2021-02-10T11:34:03.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.064 7f6702c79700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66fc0e57d0 con 0x7f66fc1b0b60 2021-02-10T11:34:03.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.064 7f6702c79700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc084850 con 0x7f66fc0e53d0 2021-02-10T11:34:03.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.064 7f6701296700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f66fc0842e0 0x7f66fc082100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52588/0 (socket says 172.21.2.101:52588) 2021-02-10T11:34:03.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.064 7f6701296700 1 -- 172.21.2.101:0/3745611796 learned_addr learned my addr 172.21.2.101:0/3745611796 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:03.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.064 7f6700a95700 1 --2- 172.21.2.101:0/3745611796 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66fc0e53d0 0x7f66fc1b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:03.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.065 7f66fb7fe700 1 -- 172.21.2.101:0/3745611796 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1072027434 0 0) 0x7f66f4001180 con 0x7f66fc0842e0 2021-02-10T11:34:03.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.065 7f66fb7fe700 1 -- 172.21.2.101:0/3745611796 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66e8002fe0 con 0x7f66fc0842e0 2021-02-10T11:34:03.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.065 7f6700a95700 1 -- 172.21.2.101:0/3745611796 >> v1:172.21.2.108:6789/0 conn(0x7f66fc1b0b60 legacy=0x7f66fc081910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:03.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.065 7f6700a95700 1 -- 172.21.2.101:0/3745611796 >> v1:172.21.2.101:6789/0 conn(0x7f66fc0842e0 legacy=0x7f66fc082100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:03.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.065 7f6700a95700 1 -- 172.21.2.101:0/3745611796 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc0e5fd0 con 0x7f66fc0e53d0 2021-02-10T11:34:03.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.065 7f6700a95700 1 --2- 172.21.2.101:0/3745611796 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66fc0e53d0 0x7f66fc1b0540 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f66e4006650 tx=0x7f66e4009a80).ready entity=mon.2 client_cookie=ca91c800a77f71eb server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:03.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.065 7f66fb7fe700 1 -- 172.21.2.101:0/3745611796 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f66e40009b0 con 0x7f66fc0e53d0 2021-02-10T11:34:03.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.066 7f66fb7fe700 1 -- 172.21.2.101:0/3745611796 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e4006740 con 0x7f66fc0e53d0 2021-02-10T11:34:03.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.066 7f66fb7fe700 1 -- 172.21.2.101:0/3745611796 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f66e400c0c0 con 0x7f66fc0e53d0 2021-02-10T11:34:03.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.066 7f6702c79700 1 -- 172.21.2.101:0/3745611796 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66fc0e53d0 msgr2=0x7f66fc1b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:03.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.066 7f6702c79700 1 --2- 172.21.2.101:0/3745611796 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66fc0e53d0 0x7f66fc1b0540 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f66e4006650 tx=0x7f66e4009a80).stop 2021-02-10T11:34:03.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.066 7f6702c79700 1 -- 172.21.2.101:0/3745611796 shutdown_connections 2021-02-10T11:34:03.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.066 7f6702c79700 1 --2- 172.21.2.101:0/3745611796 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66fc0e53d0 0x7f66fc1b0540 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:03.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.066 7f6702c79700 1 -- 172.21.2.101:0/3745611796 >> 172.21.2.101:0/3745611796 conn(0x7f66fc0ce4f0 msgr2=0x7f66fc0e4600 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:03.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.066 7f6702c79700 1 -- 172.21.2.101:0/3745611796 shutdown_connections 2021-02-10T11:34:03.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.066 7f6702c79700 1 -- 172.21.2.101:0/3745611796 wait complete. 2021-02-10T11:34:03.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.081 7f6702c79700 1 Processor -- start 2021-02-10T11:34:03.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.081 7f6702c79700 1 -- start start 2021-02-10T11:34:03.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.081 7f6702c79700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f66fc09bd80 0x7f66fc18ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:03.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.081 7f6702c79700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66fc0ba2a0 0x7f66fc0ba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:03.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.081 7f6702c79700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66fc1a7430 0x7f66fc1a7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:03.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.081 7f6702c79700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc1a7e50 con 0x7f66fc1a7430 2021-02-10T11:34:03.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.081 7f6702c79700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc1a8040 con 0x7f66fc09bd80 2021-02-10T11:34:03.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.081 7f6702c79700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc1a8260 con 0x7f66fc0ba2a0 2021-02-10T11:34:03.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.081 7f6701296700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66fc1a7430 0x7f66fc1a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:03.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.081 7f6701296700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66fc1a7430 0x7f66fc1a7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52098/0 (socket says 172.21.2.101:52098) 2021-02-10T11:34:03.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.081 7f6701296700 1 -- 172.21.2.101:0/4105292482 learned_addr learned my addr 172.21.2.101:0/4105292482 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:03.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.082 7f6701296700 1 -- 172.21.2.101:0/4105292482 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66fc0ba2a0 msgr2=0x7f66fc0ba6a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:34:03.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.082 7f6701296700 1 --2- 172.21.2.101:0/4105292482 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66fc0ba2a0 0x7f66fc0ba6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:03.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.082 7f6701296700 1 -- 172.21.2.101:0/4105292482 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f66fc09bd80 msgr2=0x7f66fc18ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:34:03.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.082 7f6701296700 1 --2- 172.21.2.101:0/4105292482 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f66fc09bd80 0x7f66fc18ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:03.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.082 7f6701296700 1 -- 172.21.2.101:0/4105292482 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc09c360 con 0x7f66fc1a7430 2021-02-10T11:34:03.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.082 7f6701296700 1 --2- 172.21.2.101:0/4105292482 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66fc1a7430 0x7f66fc1a7830 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0x7f66f4006f00 tx=0x7f66f400b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:03.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.082 7f66f97fa700 1 -- 172.21.2.101:0/4105292482 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f66f400bd30 con 0x7f66fc1a7430 2021-02-10T11:34:03.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.082 7f66f97fa700 1 -- 172.21.2.101:0/4105292482 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f4006020 con 0x7f66fc1a7430 2021-02-10T11:34:03.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.082 7f6702c79700 1 -- 172.21.2.101:0/4105292482 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66fc09c360 con 0x7f66fc1a7430 2021-02-10T11:34:03.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.082 7f66f97fa700 1 -- 172.21.2.101:0/4105292482 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f66f40043a0 con 0x7f66fc1a7430 2021-02-10T11:34:03.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.082 7f6702c79700 1 -- 172.21.2.101:0/4105292482 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66fc09c8f0 con 0x7f66fc1a7430 2021-02-10T11:34:03.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.083 7f66f97fa700 1 -- 172.21.2.101:0/4105292482 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f66f4006200 con 0x7f66fc1a7430 2021-02-10T11:34:03.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.083 7f66f97fa700 1 --2- 172.21.2.101:0/4105292482 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66e0018700 0x7f66e001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:03.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.083 7f66f97fa700 1 -- 172.21.2.101:0/4105292482 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f66f4032790 con 0x7f66fc1a7430 2021-02-10T11:34:03.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.083 7f6700a95700 1 --2- 172.21.2.101:0/4105292482 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66e0018700 0x7f66e001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:03.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.083 7f6700a95700 1 --2- 172.21.2.101:0/4105292482 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66e0018700 0x7f66e001abc0 crc :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:03.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.114 7f6702c79700 1 -- 172.21.2.101:0/4105292482 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66fc09d4a0 con 0x7f66fc1a7430 2021-02-10T11:34:03.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.117 7f66f97fa700 1 -- 172.21.2.101:0/4105292482 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f66f40009b0 con 0x7f66fc1a7430 2021-02-10T11:34:03.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.245 7f6702c79700 1 -- 172.21.2.101:0/4105292482 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f66fc09d4a0 con 0x7f66fc1a7430 2021-02-10T11:34:03.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.245 7f66f97fa700 1 -- 172.21.2.101:0/4105292482 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v55) v1 ==== 92+0+10 (secure 0 0 0) 0x7f66f400bd30 con 0x7f66fc1a7430 2021-02-10T11:34:03.257 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:34:03.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.247 7f6702c79700 1 -- 172.21.2.101:0/4105292482 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66e0018700 msgr2=0x7f66e001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:03.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.247 7f6702c79700 1 --2- 172.21.2.101:0/4105292482 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66e0018700 0x7f66e001abc0 crc :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:03.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.247 7f6702c79700 1 -- 172.21.2.101:0/4105292482 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66fc1a7430 msgr2=0x7f66fc1a7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:03.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.247 7f6702c79700 1 --2- 172.21.2.101:0/4105292482 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66fc1a7430 0x7f66fc1a7830 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0x7f66f4006f00 tx=0x7f66f400b940).stop 2021-02-10T11:34:03.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.248 7f6702c79700 1 -- 172.21.2.101:0/4105292482 shutdown_connections 2021-02-10T11:34:03.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.248 7f6702c79700 1 --2- 172.21.2.101:0/4105292482 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66e0018700 0x7f66e001abc0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:03.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.248 7f6702c79700 1 --2- 172.21.2.101:0/4105292482 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66fc0ba2a0 0x7f66fc0ba6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:03.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.248 7f6702c79700 1 --2- 172.21.2.101:0/4105292482 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66fc1a7430 0x7f66fc1a7830 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:03.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.248 7f6702c79700 1 --2- 172.21.2.101:0/4105292482 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f66fc09bd80 0x7f66fc18ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:03.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.248 7f6702c79700 1 -- 172.21.2.101:0/4105292482 >> 172.21.2.101:0/4105292482 conn(0x7f66fc0c41c0 msgr2=0x7f66fc0a0910 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:03.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.248 7f6702c79700 1 -- 172.21.2.101:0/4105292482 shutdown_connections 2021-02-10T11:34:03.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.248 7f6702c79700 1 -- 172.21.2.101:0/4105292482 wait complete. 2021-02-10T11:34:03.267 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:34:03.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.668 7fdd85f61700 1 Processor -- start 2021-02-10T11:34:03.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.668 7fdd85f61700 1 -- start start 2021-02-10T11:34:03.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.669 7fdd85f61700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fdd800e53d0 0x7fdd801b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:03.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.669 7fdd85f61700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdd801b0b60 0x7fdd80081910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:03.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.669 7fdd85f61700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd80082d50 con 0x7fdd80081f30 2021-02-10T11:34:03.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.669 7fdd85f61700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd80082b20 con 0x7fdd801b0b60 2021-02-10T11:34:03.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.669 7fdd85f61700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd800e5fd0 con 0x7fdd800e53d0 2021-02-10T11:34:03.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.669 7fdd7effd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdd801b0b60 0x7fdd80081910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:03.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.669 7fdd7effd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdd801b0b60 0x7fdd80081910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52104/0 (socket says 172.21.2.101:52104) 2021-02-10T11:34:03.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.669 7fdd7effd700 1 -- 172.21.2.101:0/3399883870 learned_addr learned my addr 172.21.2.101:0/3399883870 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:03.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.669 7fdd7effd700 1 -- 172.21.2.101:0/3399883870 >> v2:172.21.2.108:3300/0 conn(0x7fdd800e53d0 msgr2=0x7fdd801b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:03.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.669 7fdd7effd700 1 --2- 172.21.2.101:0/3399883870 >> v2:172.21.2.108:3300/0 conn(0x7fdd800e53d0 0x7fdd801b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:03.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.669 7fdd7effd700 1 -- 172.21.2.101:0/3399883870 >> v1:172.21.2.108:6789/0 conn(0x7fdd80081f30 legacy=0x7fdd80082330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:03.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.669 7fdd7effd700 1 -- 172.21.2.101:0/3399883870 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd800e57d0 con 0x7fdd801b0b60 2021-02-10T11:34:03.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.670 7fdd7effd700 1 --2- 172.21.2.101:0/3399883870 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd801b0b60 0x7fdd80081910 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7fdd68006700 tx=0x7fdd68009b80).ready entity=mon.0 client_cookie=5efe880eb5346960 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:03.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.670 7fdd7e7fc700 1 -- 172.21.2.101:0/3399883870 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd6800c0c0 con 0x7fdd801b0b60 2021-02-10T11:34:03.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.670 7fdd7e7fc700 1 -- 172.21.2.101:0/3399883870 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6800c440 con 0x7fdd801b0b60 2021-02-10T11:34:03.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.670 7fdd7e7fc700 1 -- 172.21.2.101:0/3399883870 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd6800c920 con 0x7fdd801b0b60 2021-02-10T11:34:03.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.670 7fdd85f61700 1 -- 172.21.2.101:0/3399883870 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd801b0b60 msgr2=0x7fdd80081910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:03.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.670 7fdd85f61700 1 --2- 172.21.2.101:0/3399883870 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd801b0b60 0x7fdd80081910 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7fdd68006700 tx=0x7fdd68009b80).stop 2021-02-10T11:34:03.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.670 7fdd85f61700 1 -- 172.21.2.101:0/3399883870 shutdown_connections 2021-02-10T11:34:03.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.670 7fdd85f61700 1 --2- 172.21.2.101:0/3399883870 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd801b0b60 0x7fdd80081910 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:03.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.670 7fdd85f61700 1 --2- 172.21.2.101:0/3399883870 >> v2:172.21.2.108:3300/0 conn(0x7fdd800e53d0 0x7fdd801b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:03.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.670 7fdd85f61700 1 -- 172.21.2.101:0/3399883870 >> 172.21.2.101:0/3399883870 conn(0x7fdd800ce4f0 msgr2=0x7fdd800e4600 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:03.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.670 7fdd85f61700 1 -- 172.21.2.101:0/3399883870 shutdown_connections 2021-02-10T11:34:03.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.670 7fdd85f61700 1 -- 172.21.2.101:0/3399883870 wait complete. 2021-02-10T11:34:03.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.685 7fdd85f61700 1 Processor -- start 2021-02-10T11:34:03.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.685 7fdd85f61700 1 -- start start 2021-02-10T11:34:03.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.685 7fdd85f61700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd80190d90 0x7fdd800b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:03.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.685 7fdd85f61700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd8009ba00 0x7fdd8009ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:03.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.685 7fdd85f61700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd800ab620 0x7fdd800aba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:03.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.685 7fdd85f61700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8009b250 con 0x7fdd8009ba00 2021-02-10T11:34:03.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.685 7fdd85f61700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8009b480 con 0x7fdd800ab620 2021-02-10T11:34:03.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.685 7fdd85f61700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd800b4de0 con 0x7fdd80190d90 2021-02-10T11:34:03.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.686 7fdd7effd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd8009ba00 0x7fdd8009ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:03.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.686 7fdd7effd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd8009ba00 0x7fdd8009ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52108/0 (socket says 172.21.2.101:52108) 2021-02-10T11:34:03.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.686 7fdd7effd700 1 -- 172.21.2.101:0/1086274155 learned_addr learned my addr 172.21.2.101:0/1086274155 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:03.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.686 7fdd7effd700 1 -- 172.21.2.101:0/1086274155 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd80190d90 msgr2=0x7fdd800b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:03.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.686 7fdd7effd700 1 --2- 172.21.2.101:0/1086274155 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd80190d90 0x7fdd800b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:03.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.686 7fdd7effd700 1 -- 172.21.2.101:0/1086274155 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd800ab620 msgr2=0x7fdd800aba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:03.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.686 7fdd7effd700 1 --2- 172.21.2.101:0/1086274155 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd800ab620 0x7fdd800aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:03.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.686 7fdd7effd700 1 -- 172.21.2.101:0/1086274155 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd800ac040 con 0x7fdd8009ba00 2021-02-10T11:34:03.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.686 7fdd7effd700 1 --2- 172.21.2.101:0/1086274155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd8009ba00 0x7fdd8009ac30 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7fdd6800b650 tx=0x7fdd6800b9f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:03.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.686 7fdd77fff700 1 -- 172.21.2.101:0/1086274155 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd6800bde0 con 0x7fdd8009ba00 2021-02-10T11:34:03.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.686 7fdd77fff700 1 -- 172.21.2.101:0/1086274155 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd68000c90 con 0x7fdd8009ba00 2021-02-10T11:34:03.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.686 7fdd77fff700 1 -- 172.21.2.101:0/1086274155 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd680022b0 con 0x7fdd8009ba00 2021-02-10T11:34:03.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.686 7fdd85f61700 1 -- 172.21.2.101:0/1086274155 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd800ac040 con 0x7fdd8009ba00 2021-02-10T11:34:03.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.687 7fdd85f61700 1 -- 172.21.2.101:0/1086274155 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd801a75d0 con 0x7fdd8009ba00 2021-02-10T11:34:03.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.687 7fdd77fff700 1 -- 172.21.2.101:0/1086274155 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdd68009f90 con 0x7fdd8009ba00 2021-02-10T11:34:03.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.687 7fdd77fff700 1 --2- 172.21.2.101:0/1086274155 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd640186c0 0x7fdd6401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:03.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.687 7fdd77fff700 1 -- 172.21.2.101:0/1086274155 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fdd68032560 con 0x7fdd8009ba00 2021-02-10T11:34:03.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.687 7fdd7f7fe700 1 --2- 172.21.2.101:0/1086274155 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd640186c0 0x7fdd6401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:03.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.688 7fdd7f7fe700 1 --2- 172.21.2.101:0/1086274155 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd640186c0 0x7fdd6401abc0 crc :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:03.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.719 7fdd85f61700 1 -- 172.21.2.101:0/1086274155 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd8009b250 con 0x7fdd8009ba00 2021-02-10T11:34:03.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.722 7fdd77fff700 1 -- 172.21.2.101:0/1086274155 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdd680009b0 con 0x7fdd8009ba00 2021-02-10T11:34:03.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.849 7fdd85f61700 1 -- 172.21.2.101:0/1086274155 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7fdd8009b250 con 0x7fdd8009ba00 2021-02-10T11:34:03.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.849 7fdd77fff700 1 -- 172.21.2.101:0/1086274155 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v55) v1 ==== 104+0+10 (secure 0 0 0) 0x7fdd6800bde0 con 0x7fdd8009ba00 2021-02-10T11:34:03.861 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:34:03.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.851 7fdd85f61700 1 -- 172.21.2.101:0/1086274155 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd640186c0 msgr2=0x7fdd6401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:03.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.851 7fdd85f61700 1 --2- 172.21.2.101:0/1086274155 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd640186c0 0x7fdd6401abc0 crc :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:03.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.851 7fdd85f61700 1 -- 172.21.2.101:0/1086274155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd8009ba00 msgr2=0x7fdd8009ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:03.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.851 7fdd85f61700 1 --2- 172.21.2.101:0/1086274155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd8009ba00 0x7fdd8009ac30 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7fdd6800b650 tx=0x7fdd6800b9f0).stop 2021-02-10T11:34:03.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.851 7fdd85f61700 1 -- 172.21.2.101:0/1086274155 shutdown_connections 2021-02-10T11:34:03.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.851 7fdd85f61700 1 --2- 172.21.2.101:0/1086274155 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd800ab620 0x7fdd800aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:03.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.851 7fdd85f61700 1 --2- 172.21.2.101:0/1086274155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd8009ba00 0x7fdd8009ac30 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:03.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.851 7fdd85f61700 1 --2- 172.21.2.101:0/1086274155 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd640186c0 0x7fdd6401abc0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:03.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.851 7fdd85f61700 1 --2- 172.21.2.101:0/1086274155 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd80190d90 0x7fdd800b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:03.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.851 7fdd85f61700 1 -- 172.21.2.101:0/1086274155 >> 172.21.2.101:0/1086274155 conn(0x7fdd800a38c0 msgr2=0x7fdd800a2560 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:03.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.852 7fdd85f61700 1 -- 172.21.2.101:0/1086274155 shutdown_connections 2021-02-10T11:34:03.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:03.852 7fdd85f61700 1 -- 172.21.2.101:0/1086274155 wait complete. 2021-02-10T11:34:03.871 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:34:04.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.274 7f89dc2ab700 1 Processor -- start 2021-02-10T11:34:04.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.274 7f89dc2ab700 1 -- start start 2021-02-10T11:34:04.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.274 7f89dc2ab700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89d40e5570 0x7f89d4078520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:04.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.274 7f89dc2ab700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d4078b90 con 0x7f89d40e5170 2021-02-10T11:34:04.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.274 7f89dc2ab700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d41b40d0 con 0x7f89d4079110 2021-02-10T11:34:04.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.274 7f89dc2ab700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d40d1900 con 0x7f89d40e5570 2021-02-10T11:34:04.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.275 7f89da0c7700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f89d40e5170 0x7f89d41b38e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52610/0 (socket says 172.21.2.101:52610) 2021-02-10T11:34:04.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.275 7f89da0c7700 1 -- 172.21.2.101:0/2788361184 learned_addr learned my addr 172.21.2.101:0/2788361184 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:04.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.275 7f89d90c5700 1 -- 172.21.2.101:0/2788361184 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2469329102 0 0) 0x7f89d0000ce0 con 0x7f89d40e5170 2021-02-10T11:34:04.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.275 7f89d98c6700 1 --2- 172.21.2.101:0/2788361184 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89d40e5570 0x7f89d4078520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:04.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.275 7f89d90c5700 1 -- 172.21.2.101:0/2788361184 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89c0002fe0 con 0x7f89d40e5170 2021-02-10T11:34:04.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.275 7f89d90c5700 1 -- 172.21.2.101:0/2788361184 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 898662282 0 0) 0x7f89d0001180 con 0x7f89d40e5170 2021-02-10T11:34:04.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.276 7f89d90c5700 1 -- 172.21.2.101:0/2788361184 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89d40e5570 msgr2=0x7f89d4078520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:04.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.276 7f89d90c5700 1 --2- 172.21.2.101:0/2788361184 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89d40e5570 0x7f89d4078520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:04.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.276 7f89d90c5700 1 -- 172.21.2.101:0/2788361184 >> v1:172.21.2.108:6789/0 conn(0x7f89d4079110 legacy=0x7f89d4079510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:04.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.276 7f89d90c5700 1 -- 172.21.2.101:0/2788361184 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d40d1ae0 con 0x7f89d40e5170 2021-02-10T11:34:04.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.276 7f89d90c5700 1 -- 172.21.2.101:0/2788361184 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f89d0001680 con 0x7f89d40e5170 2021-02-10T11:34:04.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.276 7f89d90c5700 1 -- 172.21.2.101:0/2788361184 >> v1:172.21.2.101:6789/0 conn(0x7f89d40e5170 legacy=0x7f89d41b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:04.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.276 7f89d90c5700 1 --2- 172.21.2.101:0/2788361184 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89c0003a60 0x7f89c0005f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:04.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.276 7f89d90c5700 1 --2- 172.21.2.101:0/2788361184 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89c00067d0 0x7f89c0008cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:04.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.276 7f89d90c5700 1 --2- 172.21.2.101:0/2788361184 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89c0009510 0x7f89c000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:04.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.276 7f89d90c5700 1 -- 172.21.2.101:0/2788361184 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c000c2e0 con 0x7f89c00067d0 2021-02-10T11:34:04.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.276 7f89d90c5700 1 -- 172.21.2.101:0/2788361184 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c000c500 con 0x7f89c0009510 2021-02-10T11:34:04.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.276 7f89d90c5700 1 -- 172.21.2.101:0/2788361184 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c0050730 con 0x7f89c0003a60 2021-02-10T11:34:04.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.276 7f89d98c6700 1 --2- 172.21.2.101:0/2788361184 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89c00067d0 0x7f89c0008cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:04.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.276 7f89da0c7700 1 --2- 172.21.2.101:0/2788361184 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89c0003a60 0x7f89c0005f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:04.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.276 7f89da8c8700 1 --2- 172.21.2.101:0/2788361184 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89c0009510 0x7f89c000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:04.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.276 7f89d98c6700 1 -- 172.21.2.101:0/2788361184 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89c0003a60 msgr2=0x7f89c0005f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:04.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.276 7f89d98c6700 1 --2- 172.21.2.101:0/2788361184 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89c0003a60 0x7f89c0005f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:04.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.276 7f89d98c6700 1 -- 172.21.2.101:0/2788361184 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89c0009510 msgr2=0x7f89c000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:04.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.276 7f89d98c6700 1 --2- 172.21.2.101:0/2788361184 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89c0009510 0x7f89c000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:04.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.276 7f89d98c6700 1 -- 172.21.2.101:0/2788361184 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c0050960 con 0x7f89c00067d0 2021-02-10T11:34:04.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.276 7f89da0c7700 1 --2- 172.21.2.101:0/2788361184 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89c0003a60 0x7f89c0005f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:34:04.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.276 7f89d98c6700 1 --2- 172.21.2.101:0/2788361184 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89c00067d0 0x7f89c0008cf0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7f89c4009310 tx=0x7f89c4009a90).ready entity=mon.0 client_cookie=79bc54728fa8c5b6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:04.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.277 7f89d90c5700 1 -- 172.21.2.101:0/2788361184 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f89c400c0c0 con 0x7f89c00067d0 2021-02-10T11:34:04.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.277 7f89d90c5700 1 -- 172.21.2.101:0/2788361184 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c4006400 con 0x7f89c00067d0 2021-02-10T11:34:04.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.277 7f89d90c5700 1 -- 172.21.2.101:0/2788361184 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f89c400c400 con 0x7f89c00067d0 2021-02-10T11:34:04.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.277 7f89dc2ab700 1 -- 172.21.2.101:0/2788361184 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89c00067d0 msgr2=0x7f89c0008cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:04.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.277 7f89dc2ab700 1 --2- 172.21.2.101:0/2788361184 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89c00067d0 0x7f89c0008cf0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7f89c4009310 tx=0x7f89c4009a90).stop 2021-02-10T11:34:04.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.277 7f89da8c8700 1 -- 172.21.2.101:0/2788361184 reap_dead start 2021-02-10T11:34:04.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.277 7f89dc2ab700 1 -- 172.21.2.101:0/2788361184 shutdown_connections 2021-02-10T11:34:04.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.277 7f89dc2ab700 1 -- 172.21.2.101:0/2788361184 >> 172.21.2.101:0/2788361184 conn(0x7f89d40ce820 msgr2=0x7f89d40e4660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:04.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.277 7f89dc2ab700 1 -- 172.21.2.101:0/2788361184 shutdown_connections 2021-02-10T11:34:04.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.277 7f89dc2ab700 1 -- 172.21.2.101:0/2788361184 wait complete. 2021-02-10T11:34:04.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.292 7f89dc2ab700 1 Processor -- start 2021-02-10T11:34:04.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.292 7f89dc2ab700 1 -- start start 2021-02-10T11:34:04.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.293 7f89dc2ab700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89d40be6d0 0x7f89d40c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:04.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.293 7f89dc2ab700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89d40bde80 0x7f89d40a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:04.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.293 7f89dc2ab700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89d40a1d60 0x7f89d40bebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:04.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.293 7f89dc2ab700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d40be300 con 0x7f89d40a1d60 2021-02-10T11:34:04.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.293 7f89dc2ab700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d40a2160 con 0x7f89d40be6d0 2021-02-10T11:34:04.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.293 7f89dc2ab700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d40a2380 con 0x7f89d40bde80 2021-02-10T11:34:04.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.293 7f89da8c8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89d40a1d60 0x7f89d40bebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:04.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.293 7f89da8c8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89d40a1d60 0x7f89d40bebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52132/0 (socket says 172.21.2.101:52132) 2021-02-10T11:34:04.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.293 7f89da8c8700 1 -- 172.21.2.101:0/3742546829 learned_addr learned my addr 172.21.2.101:0/3742546829 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:04.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.293 7f89d98c6700 1 --2- 172.21.2.101:0/3742546829 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89d40bde80 0x7f89d40a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:04.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.293 7f89da0c7700 1 --2- 172.21.2.101:0/3742546829 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89d40be6d0 0x7f89d40c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:04.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.293 7f89da8c8700 1 -- 172.21.2.101:0/3742546829 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89d40bde80 msgr2=0x7f89d40a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:04.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.293 7f89da8c8700 1 --2- 172.21.2.101:0/3742546829 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89d40bde80 0x7f89d40a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:04.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.293 7f89da8c8700 1 -- 172.21.2.101:0/3742546829 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89d40be6d0 msgr2=0x7f89d40c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:04.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.293 7f89da8c8700 1 --2- 172.21.2.101:0/3742546829 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89d40be6d0 0x7f89d40c3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:04.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.293 7f89da8c8700 1 -- 172.21.2.101:0/3742546829 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d40bf200 con 0x7f89d40a1d60 2021-02-10T11:34:04.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.293 7f89da8c8700 1 --2- 172.21.2.101:0/3742546829 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89d40a1d60 0x7f89d40bebe0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 rx=0x7f89cc00a940 tx=0x7f89cc00ace0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:04.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.294 7f89caffd700 1 -- 172.21.2.101:0/3742546829 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f89cc00c300 con 0x7f89d40a1d60 2021-02-10T11:34:04.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.294 7f89caffd700 1 -- 172.21.2.101:0/3742546829 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89cc00ae20 con 0x7f89d40a1d60 2021-02-10T11:34:04.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.294 7f89caffd700 1 -- 172.21.2.101:0/3742546829 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f89cc00c7c0 con 0x7f89d40a1d60 2021-02-10T11:34:04.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.294 7f89dc2ab700 1 -- 172.21.2.101:0/3742546829 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d40bf200 con 0x7f89d40a1d60 2021-02-10T11:34:04.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.294 7f89dc2ab700 1 -- 172.21.2.101:0/3742546829 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d40bf200 con 0x7f89d40a1d60 2021-02-10T11:34:04.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.294 7f89caffd700 1 -- 172.21.2.101:0/3742546829 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f89cc006bc0 con 0x7f89d40a1d60 2021-02-10T11:34:04.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.294 7f89caffd700 1 --2- 172.21.2.101:0/3742546829 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f89bc0186c0 0x7f89bc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:04.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.295 7f89caffd700 1 -- 172.21.2.101:0/3742546829 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f89cc032900 con 0x7f89d40a1d60 2021-02-10T11:34:04.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.295 7f89da0c7700 1 --2- 172.21.2.101:0/3742546829 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f89bc0186c0 0x7f89bc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:04.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.295 7f89da0c7700 1 --2- 172.21.2.101:0/3742546829 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f89bc0186c0 0x7f89bc01abc0 crc :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:04.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.324 7f89dc2ab700 1 -- 172.21.2.101:0/3742546829 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d410c970 con 0x7f89d40a1d60 2021-02-10T11:34:04.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.327 7f89caffd700 1 -- 172.21.2.101:0/3742546829 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f89cc022730 con 0x7f89d40a1d60 2021-02-10T11:34:04.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.455 7f89dc2ab700 1 -- 172.21.2.101:0/3742546829 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f89d40fe290 con 0x7f89d40a1d60 2021-02-10T11:34:04.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.455 7f89caffd700 1 -- 172.21.2.101:0/3742546829 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v55) v1 ==== 100+0+10 (secure 0 0 0) 0x7f89cc022730 con 0x7f89d40a1d60 2021-02-10T11:34:04.468 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:34:04.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.457 7f89dc2ab700 1 -- 172.21.2.101:0/3742546829 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f89bc0186c0 msgr2=0x7f89bc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:04.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.457 7f89dc2ab700 1 --2- 172.21.2.101:0/3742546829 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f89bc0186c0 0x7f89bc01abc0 crc :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:04.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.457 7f89dc2ab700 1 -- 172.21.2.101:0/3742546829 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89d40a1d60 msgr2=0x7f89d40bebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:04.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.457 7f89dc2ab700 1 --2- 172.21.2.101:0/3742546829 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89d40a1d60 0x7f89d40bebe0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 rx=0x7f89cc00a940 tx=0x7f89cc00ace0).stop 2021-02-10T11:34:04.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.458 7f89dc2ab700 1 -- 172.21.2.101:0/3742546829 shutdown_connections 2021-02-10T11:34:04.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.458 7f89dc2ab700 1 --2- 172.21.2.101:0/3742546829 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f89bc0186c0 0x7f89bc01abc0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:04.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.458 7f89dc2ab700 1 --2- 172.21.2.101:0/3742546829 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89d40bde80 0x7f89d40a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:04.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.458 7f89dc2ab700 1 --2- 172.21.2.101:0/3742546829 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89d40a1d60 0x7f89d40bebe0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:04.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.458 7f89dc2ab700 1 --2- 172.21.2.101:0/3742546829 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89d40be6d0 0x7f89d40c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:04.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.458 7f89dc2ab700 1 -- 172.21.2.101:0/3742546829 >> 172.21.2.101:0/3742546829 conn(0x7f89d40a12c0 msgr2=0x7f89d40c0810 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:04.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.458 7f89dc2ab700 1 -- 172.21.2.101:0/3742546829 shutdown_connections 2021-02-10T11:34:04.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.458 7f89dc2ab700 1 -- 172.21.2.101:0/3742546829 wait complete. 2021-02-10T11:34:04.478 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:34:04.478 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:34:04.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.878 7ffaea782700 1 Processor -- start 2021-02-10T11:34:04.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.878 7ffaea782700 1 -- start start 2021-02-10T11:34:04.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.878 7ffaea782700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ffae4082920 0x7ffae40832d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:04.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.878 7ffaea782700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae4082d50 con 0x7ffae40e53e0 2021-02-10T11:34:04.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.878 7ffaea782700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae40e57e0 con 0x7ffae40838f0 2021-02-10T11:34:04.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.878 7ffaea782700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae41b0d40 con 0x7ffae4082920 2021-02-10T11:34:04.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.879 7ffae3fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ffae40e53e0 0x7ffae41b0550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52630/0 (socket says 172.21.2.101:52630) 2021-02-10T11:34:04.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.879 7ffae3fff700 1 -- 172.21.2.101:0/1664857377 learned_addr learned my addr 172.21.2.101:0/1664857377 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:04.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.879 7ffae2ffd700 1 -- 172.21.2.101:0/1664857377 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2856868896 0 0) 0x7ffad40010b0 con 0x7ffae40e53e0 2021-02-10T11:34:04.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.879 7ffae2ffd700 1 -- 172.21.2.101:0/1664857377 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffacc002fe0 con 0x7ffae40e53e0 2021-02-10T11:34:04.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.879 7ffae37fe700 1 --2- 172.21.2.101:0/1664857377 >> v2:172.21.2.108:3300/0 conn(0x7ffae4082920 0x7ffae40832d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:04.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae2ffd700 1 -- 172.21.2.101:0/1664857377 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1008314675 0 0) 0x7ffad40010b0 con 0x7ffae40e53e0 2021-02-10T11:34:04.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae2ffd700 1 -- 172.21.2.101:0/1664857377 >> v2:172.21.2.108:3300/0 conn(0x7ffae4082920 msgr2=0x7ffae40832d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:04.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae2ffd700 1 --2- 172.21.2.101:0/1664857377 >> v2:172.21.2.108:3300/0 conn(0x7ffae4082920 0x7ffae40832d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:04.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae2ffd700 1 -- 172.21.2.101:0/1664857377 >> v1:172.21.2.108:6789/0 conn(0x7ffae40838f0 legacy=0x7ffae4083cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:04.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae2ffd700 1 -- 172.21.2.101:0/1664857377 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae40844e0 con 0x7ffae40e53e0 2021-02-10T11:34:04.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae2ffd700 1 -- 172.21.2.101:0/1664857377 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7ffad40015b0 con 0x7ffae40e53e0 2021-02-10T11:34:04.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae2ffd700 1 -- 172.21.2.101:0/1664857377 >> v1:172.21.2.101:6789/0 conn(0x7ffae40e53e0 legacy=0x7ffae41b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:04.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae2ffd700 1 --2- 172.21.2.101:0/1664857377 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffacc003ac0 0x7ffacc005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:04.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae2ffd700 1 --2- 172.21.2.101:0/1664857377 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffacc006790 0x7ffacc008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:04.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae2ffd700 1 --2- 172.21.2.101:0/1664857377 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffacc009590 0x7ffacc00bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:04.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae2ffd700 1 -- 172.21.2.101:0/1664857377 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc00c300 con 0x7ffacc003ac0 2021-02-10T11:34:04.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae2ffd700 1 -- 172.21.2.101:0/1664857377 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc00c520 con 0x7ffacc006790 2021-02-10T11:34:04.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae2ffd700 1 -- 172.21.2.101:0/1664857377 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc00c740 con 0x7ffacc009590 2021-02-10T11:34:04.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae3fff700 1 --2- 172.21.2.101:0/1664857377 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffacc003ac0 0x7ffacc005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:04.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae37fe700 1 --2- 172.21.2.101:0/1664857377 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffacc006790 0x7ffacc008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:04.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae8d9f700 1 --2- 172.21.2.101:0/1664857377 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffacc009590 0x7ffacc00bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:04.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae3fff700 1 -- 172.21.2.101:0/1664857377 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffacc009590 msgr2=0x7ffacc00bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:04.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae3fff700 1 --2- 172.21.2.101:0/1664857377 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffacc009590 0x7ffacc00bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:04.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae3fff700 1 -- 172.21.2.101:0/1664857377 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffacc006790 msgr2=0x7ffacc008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:04.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae3fff700 1 --2- 172.21.2.101:0/1664857377 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffacc006790 0x7ffacc008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:04.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae3fff700 1 -- 172.21.2.101:0/1664857377 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffacc0509b0 con 0x7ffacc003ac0 2021-02-10T11:34:04.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae8d9f700 1 -- 172.21.2.101:0/1664857377 reap_dead start 2021-02-10T11:34:04.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae37fe700 1 --2- 172.21.2.101:0/1664857377 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffacc006790 0x7ffacc008c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:34:04.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.880 7ffae3fff700 1 --2- 172.21.2.101:0/1664857377 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffacc003ac0 0x7ffacc005ef0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7ffad4007990 tx=0x7ffad4007d30).ready entity=mon.0 client_cookie=609543e34ce473ee server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:04.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.881 7ffae2ffd700 1 -- 172.21.2.101:0/1664857377 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffad40009b0 con 0x7ffacc003ac0 2021-02-10T11:34:04.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.881 7ffae2ffd700 1 -- 172.21.2.101:0/1664857377 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad400a310 con 0x7ffacc003ac0 2021-02-10T11:34:04.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.881 7ffae2ffd700 1 -- 172.21.2.101:0/1664857377 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffad400a7a0 con 0x7ffacc003ac0 2021-02-10T11:34:04.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.881 7ffaea782700 1 -- 172.21.2.101:0/1664857377 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffacc003ac0 msgr2=0x7ffacc005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:04.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.881 7ffaea782700 1 --2- 172.21.2.101:0/1664857377 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffacc003ac0 0x7ffacc005ef0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7ffad4007990 tx=0x7ffad4007d30).stop 2021-02-10T11:34:04.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.882 7ffaea782700 1 -- 172.21.2.101:0/1664857377 shutdown_connections 2021-02-10T11:34:04.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.882 7ffaea782700 1 --2- 172.21.2.101:0/1664857377 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffacc003ac0 0x7ffacc005ef0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:04.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.882 7ffaea782700 1 -- 172.21.2.101:0/1664857377 >> 172.21.2.101:0/1664857377 conn(0x7ffae40ce500 msgr2=0x7ffae40e4610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:04.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.882 7ffaea782700 1 -- 172.21.2.101:0/1664857377 shutdown_connections 2021-02-10T11:34:04.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.882 7ffaea782700 1 -- 172.21.2.101:0/1664857377 wait complete. 2021-02-10T11:34:04.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.896 7ffaea782700 1 Processor -- start 2021-02-10T11:34:04.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.896 7ffaea782700 1 -- start start 2021-02-10T11:34:04.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.897 7ffaea782700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffae4191bd0 0x7ffae41a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:04.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.897 7ffaea782700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffae41a3e10 0x7ffae40b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:04.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.897 7ffaea782700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffae40b78f0 0x7ffae40b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:04.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.897 7ffaea782700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae4192170 con 0x7ffae41a3e10 2021-02-10T11:34:04.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.897 7ffaea782700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae40b8310 con 0x7ffae4191bd0 2021-02-10T11:34:04.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.897 7ffaea782700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae40b8530 con 0x7ffae40b78f0 2021-02-10T11:34:04.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.897 7ffae37fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffae41a3e10 0x7ffae40b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:04.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.897 7ffae37fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffae41a3e10 0x7ffae40b72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52150/0 (socket says 172.21.2.101:52150) 2021-02-10T11:34:04.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.897 7ffae37fe700 1 -- 172.21.2.101:0/1004218433 learned_addr learned my addr 172.21.2.101:0/1004218433 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:04.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.897 7ffae37fe700 1 -- 172.21.2.101:0/1004218433 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffae40b78f0 msgr2=0x7ffae40b7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:04.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.897 7ffae37fe700 1 --2- 172.21.2.101:0/1004218433 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffae40b78f0 0x7ffae40b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:04.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.897 7ffae37fe700 1 -- 172.21.2.101:0/1004218433 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffae4191bd0 msgr2=0x7ffae41a37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:04.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.897 7ffae37fe700 1 --2- 172.21.2.101:0/1004218433 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffae4191bd0 0x7ffae41a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:04.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.897 7ffae37fe700 1 -- 172.21.2.101:0/1004218433 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae41944a0 con 0x7ffae41a3e10 2021-02-10T11:34:04.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.897 7ffae37fe700 1 --2- 172.21.2.101:0/1004218433 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffae41a3e10 0x7ffae40b72d0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 rx=0x7ffad8006b80 tx=0x7ffad800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:04.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.898 7ffae0ff9700 1 -- 172.21.2.101:0/1004218433 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffad800bd80 con 0x7ffae41a3e10 2021-02-10T11:34:04.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.898 7ffae0ff9700 1 -- 172.21.2.101:0/1004218433 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad8002030 con 0x7ffae41a3e10 2021-02-10T11:34:04.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.898 7ffae0ff9700 1 -- 172.21.2.101:0/1004218433 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffad8002550 con 0x7ffae41a3e10 2021-02-10T11:34:04.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.898 7ffaea782700 1 -- 172.21.2.101:0/1004218433 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae41944a0 con 0x7ffae41a3e10 2021-02-10T11:34:04.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.898 7ffaea782700 1 -- 172.21.2.101:0/1004218433 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae4194a30 con 0x7ffae41a3e10 2021-02-10T11:34:04.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.898 7ffae0ff9700 1 -- 172.21.2.101:0/1004218433 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ffad8002210 con 0x7ffae41a3e10 2021-02-10T11:34:04.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.898 7ffae0ff9700 1 --2- 172.21.2.101:0/1004218433 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffac40186c0 0x7ffac401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:04.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.898 7ffae0ff9700 1 -- 172.21.2.101:0/1004218433 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ffad80327d0 con 0x7ffae41a3e10 2021-02-10T11:34:04.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.899 7ffae3fff700 1 --2- 172.21.2.101:0/1004218433 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffac40186c0 0x7ffac401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:04.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.899 7ffae3fff700 1 --2- 172.21.2.101:0/1004218433 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffac40186c0 0x7ffac401abc0 crc :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:04.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.928 7ffaea782700 1 -- 172.21.2.101:0/1004218433 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae40ed5d0 con 0x7ffae41a3e10 2021-02-10T11:34:04.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:04.931 7ffae0ff9700 1 -- 172.21.2.101:0/1004218433 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ffad80009b0 con 0x7ffae41a3e10 2021-02-10T11:34:05.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:05.058 7ffaea782700 1 -- 172.21.2.101:0/1004218433 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffae4082d50 con 0x7ffae41a3e10 2021-02-10T11:34:05.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:05.059 7ffae0ff9700 1 -- 172.21.2.101:0/1004218433 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+2098 (secure 0 0 0) 0x7ffad800bd80 con 0x7ffae41a3e10 2021-02-10T11:34:05.070 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:05.071 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8049,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/172865610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":172865610},{"type":"v1","addr":"172.21.2.101:6835","nonce":172865610}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:33:16.590282","modified":"2021-02-10 11:33:18.594009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5579},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5579":{"gid":5579,"name":"b","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2400978339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2400978339},{"type":"v1","addr":"172.21.2.108:6833","nonce":2400978339}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-02-10T11:34:05.071 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 39 2021-02-10T11:34:05.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:05.061 7ffaea782700 1 -- 172.21.2.101:0/1004218433 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffac40186c0 msgr2=0x7ffac401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:05.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:05.061 7ffaea782700 1 --2- 172.21.2.101:0/1004218433 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffac40186c0 0x7ffac401abc0 crc :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:05.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:05.061 7ffaea782700 1 -- 172.21.2.101:0/1004218433 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffae41a3e10 msgr2=0x7ffae40b72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:05.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:05.061 7ffaea782700 1 --2- 172.21.2.101:0/1004218433 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffae41a3e10 0x7ffae40b72d0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 rx=0x7ffad8006b80 tx=0x7ffad800b990).stop 2021-02-10T11:34:05.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:05.061 7ffaea782700 1 -- 172.21.2.101:0/1004218433 shutdown_connections 2021-02-10T11:34:05.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:05.061 7ffaea782700 1 --2- 172.21.2.101:0/1004218433 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffac40186c0 0x7ffac401abc0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:05.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:05.061 7ffaea782700 1 --2- 172.21.2.101:0/1004218433 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffae40b78f0 0x7ffae40b7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:05.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:05.061 7ffaea782700 1 --2- 172.21.2.101:0/1004218433 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffae41a3e10 0x7ffae40b72d0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:05.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:05.061 7ffaea782700 1 --2- 172.21.2.101:0/1004218433 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffae4191bd0 0x7ffae41a37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:05.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:05.061 7ffaea782700 1 -- 172.21.2.101:0/1004218433 >> 172.21.2.101:0/1004218433 conn(0x7ffae40ad180 msgr2=0x7ffae40b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:05.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:05.061 7ffaea782700 1 -- 172.21.2.101:0/1004218433 shutdown_connections 2021-02-10T11:34:05.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:05.061 7ffaea782700 1 -- 172.21.2.101:0/1004218433 wait complete. 2021-02-10T11:34:05.282 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:33:16.590282', 'modified': '2021-02-10 11:33:18.594009', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5579}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5579': {'gid': 5579, 'name': 'b', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/2400978339', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 2400978339}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 2400978339}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:34:05.282 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:34:05.283 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:34:05.682 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:34:05.682 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:34:05.682 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:34:05.682 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 5579, 2021-02-10T11:34:05.682 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:34:05.682 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:34:05.682 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 49.079286580000002, 2021-02-10T11:34:05.682 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 38, 2021-02-10T11:34:05.682 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 55, 2021-02-10T11:34:05.682 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 55, 2021-02-10T11:34:05.682 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 54.712177996999998 2021-02-10T11:34:05.683 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:34:05.686 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 5579, "want_state": "up:active", "state": "up:active", "rank_uptime": 49.079286580000002, "mdsmap_epoch": 38, "osdmap_epoch": 55, "osdmap_epoch_barrier": 55, "uptime": 54.712177996999998 } 2021-02-10T11:34:05.686 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:34:05.686 INFO:tasks.cephfs.fuse_mount:Client client.alice config is {} 2021-02-10T11:34:05.686 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.alice at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:34:05.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:05.686 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:34:06.022 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:34:06.367 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:34:06.367 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:34:06.711 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:34:06.711 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:34:06.712 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:34:06.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:06.712 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:34:07.046 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:34:07.046 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2021-02-10T11:34:07.047 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --client_mountpoint=/volumes/group_0000000000586931/subvolume_0000000000130711 --name client.alice /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:34:07.049 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:34:07.393 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:34:07.393 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:34:07.458 INFO:tasks.cephfs.fuse_mount.ceph-fuse.alice.gibba001.stderr:ceph-fuse[121101]: starting ceph client 2021-02-10T11:34:07.458 INFO:tasks.cephfs.fuse_mount.ceph-fuse.alice.gibba001.stderr:2021-02-10 11:34:07.448 7f778bbfff80 -1 init, newargv = 0x55a8b7fef440 newargc=9 2021-02-10T11:34:07.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.alice.gibba001.stderr:ceph-fuse[121101]: starting fuse 2021-02-10T11:34:07.736 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:34:07.737 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:34:07.737 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:34:07.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:07.738 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:34:08.072 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:34:08.072 INFO:teuthology.orchestra.run.gibba001.stdout:44 2021-02-10T11:34:08.073 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2021-02-10T11:34:08.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:08.073 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:34:08.073 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:34:08.073 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:34:08.073 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:34:08.073 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:34:08.073 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:34:08.073 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:34:08.073 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.alice.*.asok" 2021-02-10T11:34:08.073 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:34:08.073 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:34:08.073 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:34:08.074 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:34:08.074 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:34:08.074 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:34:08.074 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:34:08.074 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:34:08.074 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:34:08.074 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:34:08.074 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:34:08.074 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:34:08.074 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:34:08.074 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.alice")) 2021-02-10T11:34:08.074 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:34:08.437 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.alice.121101.asok 2021-02-10T11:34:08.440 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.alice.121101.asok 2021-02-10T11:34:08.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:08.440 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.alice.121101.asok status 2021-02-10T11:34:08.830 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:34:08.830 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:34:08.830 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:34:08.830 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:34:08.831 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "alice", 2021-02-10T11:34:08.831 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:34:08.831 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:34:08.831 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "121101", 2021-02-10T11:34:08.831 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/volumes/group_0000000000586931/subvolume_0000000000130711" 2021-02-10T11:34:08.831 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:08.831 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:34:08.831 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:34:08.831 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 8376, 2021-02-10T11:34:08.831 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:34:08.831 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:34:08.831 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:34:08.831 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 8376 2021-02-10T11:34:08.831 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:08.832 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:34:08.832 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:08.832 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:34:08.832 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 4128978591 2021-02-10T11:34:08.832 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:08.832 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:08.832 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:34:08.832 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:08.832 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:34:08.832 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 4128978591 2021-02-10T11:34:08.832 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:08.832 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.8376 172.21.2.101:0/4128978591", 2021-02-10T11:34:08.832 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/4128978591", 2021-02-10T11:34:08.833 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:34:08.833 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 38, 2021-02-10T11:34:08.833 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 55, 2021-02-10T11:34:08.833 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:34:08.833 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:34:08.833 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:34:08.834 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.1 && sudo ls /home/ubuntu/cephtest/mnt.1 2021-02-10T11:34:09.180 INFO:teuthology.orchestra.run.gibba001.stdout:data.bin 2021-02-10T11:34:09.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:09.181 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:34:09.181 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:34:09.516 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:34:09.517 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:34:09.517 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.1 && sudo dd if=/dev/urandom of=rogue.bin bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:34:09.862 INFO:teuthology.orchestra.run.gibba001.stderr:dd: failed to open ‘rogue.bin’: Permission denied 2021-02-10T11:34:09.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:34:09.864 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:09.864 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:34:09.864 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:34:10.199 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:34:10.200 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:34:10.200 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:34:10.200 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:34:10.200 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:34:10.554 INFO:tasks.cephfs.fuse_mount.ceph-fuse.alice.gibba001.stderr:ceph-fuse[121101]: fuse finished with error 0 and tester_r 0 2021-02-10T11:34:10.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:10.555 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:34:10.556 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:34:10.891 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:34:10.891 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:34:10.891 INFO:teuthology.orchestra.run:waiting for 900 2021-02-10T11:34:10.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:34:10.892 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:34:11.227 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume deauthorize cephfs subvolume_0000000000130711 alice --group_name group_0000000000586931 2021-02-10T11:34:11.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.630 7f98c03f8700 1 Processor -- start 2021-02-10T11:34:11.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.630 7f98c03f8700 1 -- start start 2021-02-10T11:34:11.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.630 7f98c03f8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f98b80e51a0 0x7f98b81b3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:11.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.630 7f98c03f8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f98b80e55a0 0x7f98b807a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:11.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.630 7f98c03f8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b8078b90 con 0x7f98b81b3f30 2021-02-10T11:34:11.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.630 7f98c03f8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b80d1930 con 0x7f98b80e55a0 2021-02-10T11:34:11.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.630 7f98c03f8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b80d1b20 con 0x7f98b80e51a0 2021-02-10T11:34:11.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.630 7f98bea15700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f98b80e55a0 0x7f98b807a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:11.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.630 7f98bea15700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f98b80e55a0 0x7f98b807a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52166/0 (socket says 172.21.2.101:52166) 2021-02-10T11:34:11.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.630 7f98bea15700 1 -- 172.21.2.101:0/98364077 learned_addr learned my addr 172.21.2.101:0/98364077 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:11.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.631 7f98bea15700 1 -- 172.21.2.101:0/98364077 >> v2:172.21.2.108:3300/0 conn(0x7f98b80e51a0 msgr2=0x7f98b81b3910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:34:11.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.631 7f98bea15700 1 --2- 172.21.2.101:0/98364077 >> v2:172.21.2.108:3300/0 conn(0x7f98b80e51a0 0x7f98b81b3910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:11.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.631 7f98bea15700 1 -- 172.21.2.101:0/98364077 >> v1:172.21.2.108:6789/0 conn(0x7f98b81b3f30 legacy=0x7f98b8077510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:34:11.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.631 7f98bea15700 1 -- 172.21.2.101:0/98364077 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b80d1d40 con 0x7f98b80e55a0 2021-02-10T11:34:11.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.631 7f98bea15700 1 --2- 172.21.2.101:0/98364077 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b80e55a0 0x7f98b807a120 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7f98b0006f00 tx=0x7f98b0009f10).ready entity=mon.0 client_cookie=9ee3ca947e84faf3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:11.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.631 7f98bd212700 1 -- 172.21.2.101:0/98364077 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f98b000b3b0 con 0x7f98b80e55a0 2021-02-10T11:34:11.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.631 7f98bd212700 1 -- 172.21.2.101:0/98364077 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b000b630 con 0x7f98b80e55a0 2021-02-10T11:34:11.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.631 7f98bd212700 1 -- 172.21.2.101:0/98364077 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f98b000bb50 con 0x7f98b80e55a0 2021-02-10T11:34:11.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.631 7f98c03f8700 1 -- 172.21.2.101:0/98364077 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b80e55a0 msgr2=0x7f98b807a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:11.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.631 7f98c03f8700 1 --2- 172.21.2.101:0/98364077 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b80e55a0 0x7f98b807a120 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7f98b0006f00 tx=0x7f98b0009f10).stop 2021-02-10T11:34:11.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.632 7f98c03f8700 1 -- 172.21.2.101:0/98364077 shutdown_connections 2021-02-10T11:34:11.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.632 7f98c03f8700 1 --2- 172.21.2.101:0/98364077 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b80e55a0 0x7f98b807a120 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:11.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.632 7f98c03f8700 1 --2- 172.21.2.101:0/98364077 >> v2:172.21.2.108:3300/0 conn(0x7f98b80e51a0 0x7f98b81b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:11.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.632 7f98c03f8700 1 -- 172.21.2.101:0/98364077 >> 172.21.2.101:0/98364077 conn(0x7f98b80ce850 msgr2=0x7f98b80e4690 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:11.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.632 7f98c03f8700 1 -- 172.21.2.101:0/98364077 shutdown_connections 2021-02-10T11:34:11.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.632 7f98c03f8700 1 -- 172.21.2.101:0/98364077 wait complete. 2021-02-10T11:34:11.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.646 7f98c03f8700 1 Processor -- start 2021-02-10T11:34:11.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.646 7f98c03f8700 1 -- start start 2021-02-10T11:34:11.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.647 7f98c03f8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98b8091ec0 0x7f98b81991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:11.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.647 7f98c03f8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98b8097fa0 0x7f98b80983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:11.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.647 7f98c03f8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b80b1c60 0x7f98b80b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:11.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.647 7f98c03f8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b8098d60 con 0x7f98b80b1c60 2021-02-10T11:34:11.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.647 7f98c03f8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b80989c0 con 0x7f98b8091ec0 2021-02-10T11:34:11.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.647 7f98c03f8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b81998c0 con 0x7f98b8097fa0 2021-02-10T11:34:11.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.647 7f98bea15700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b80b1c60 0x7f98b80b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:11.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.647 7f98bea15700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b80b1c60 0x7f98b80b2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52174/0 (socket says 172.21.2.101:52174) 2021-02-10T11:34:11.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.647 7f98bea15700 1 -- 172.21.2.101:0/2826035762 learned_addr learned my addr 172.21.2.101:0/2826035762 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:11.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.647 7f98be214700 1 --2- 172.21.2.101:0/2826035762 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98b8091ec0 0x7f98b81991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:11.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.647 7f98bda13700 1 --2- 172.21.2.101:0/2826035762 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98b8097fa0 0x7f98b80983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:11.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.647 7f98bea15700 1 -- 172.21.2.101:0/2826035762 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98b8097fa0 msgr2=0x7f98b80983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:11.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.647 7f98bea15700 1 --2- 172.21.2.101:0/2826035762 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98b8097fa0 0x7f98b80983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:11.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.647 7f98bea15700 1 -- 172.21.2.101:0/2826035762 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98b8091ec0 msgr2=0x7f98b81991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:11.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.647 7f98bea15700 1 --2- 172.21.2.101:0/2826035762 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98b8091ec0 0x7f98b81991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:11.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.647 7f98bea15700 1 -- 172.21.2.101:0/2826035762 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b80b2680 con 0x7f98b80b1c60 2021-02-10T11:34:11.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.647 7f98be214700 1 --2- 172.21.2.101:0/2826035762 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98b8091ec0 0x7f98b81991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:34:11.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.647 7f98bea15700 1 --2- 172.21.2.101:0/2826035762 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b80b1c60 0x7f98b80b2060 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 rx=0x7f98b000b9b0 tx=0x7f98b000bd50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:11.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.648 7f98aaffd700 1 -- 172.21.2.101:0/2826035762 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f98b0003030 con 0x7f98b80b1c60 2021-02-10T11:34:11.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.648 7f98aaffd700 1 -- 172.21.2.101:0/2826035762 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b000c2f0 con 0x7f98b80b1c60 2021-02-10T11:34:11.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.648 7f98aaffd700 1 -- 172.21.2.101:0/2826035762 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f98b000c810 con 0x7f98b80b1c60 2021-02-10T11:34:11.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.648 7f98c03f8700 1 -- 172.21.2.101:0/2826035762 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b80b2680 con 0x7f98b80b1c60 2021-02-10T11:34:11.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.648 7f98c03f8700 1 -- 172.21.2.101:0/2826035762 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b8099260 con 0x7f98b80b1c60 2021-02-10T11:34:11.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.648 7f98aaffd700 1 -- 172.21.2.101:0/2826035762 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f98b0001270 con 0x7f98b80b1c60 2021-02-10T11:34:11.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.648 7f98aaffd700 1 --2- 172.21.2.101:0/2826035762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98a00186c0 0x7f98a001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:11.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.648 7f98aaffd700 1 -- 172.21.2.101:0/2826035762 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f98b00328c0 con 0x7f98b80b1c60 2021-02-10T11:34:11.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.649 7f98be214700 1 --2- 172.21.2.101:0/2826035762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98a00186c0 0x7f98a001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:11.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.649 7f98be214700 1 --2- 172.21.2.101:0/2826035762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98a00186c0 0x7f98a001abc0 crc :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:11.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.680 7f98c03f8700 1 -- 172.21.2.101:0/2826035762 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b8109bc0 con 0x7f98b80b1c60 2021-02-10T11:34:11.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.683 7f98aaffd700 1 -- 172.21.2.101:0/2826035762 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f98b0009450 con 0x7f98b80b1c60 2021-02-10T11:34:11.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.809 7f98c03f8700 1 -- 172.21.2.101:0/2826035762 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume deauthorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000130711", "target": ["mgr", ""], "auth_id": "alice", "group_name": "group_0000000000586931"}) v1 -- 0x7f98b8099220 con 0x7f98a00186c0 2021-02-10T11:34:11.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.846 7f98aaffd700 1 -- 172.21.2.101:0/2826035762 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f98ac0009b0 con 0x7f98a00186c0 2021-02-10T11:34:11.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.874 7f98c03f8700 1 -- 172.21.2.101:0/2826035762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98a00186c0 msgr2=0x7f98a001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:11.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.874 7f98c03f8700 1 --2- 172.21.2.101:0/2826035762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98a00186c0 0x7f98a001abc0 crc :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:11.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.874 7f98c03f8700 1 -- 172.21.2.101:0/2826035762 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b80b1c60 msgr2=0x7f98b80b2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:11.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.874 7f98c03f8700 1 --2- 172.21.2.101:0/2826035762 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b80b1c60 0x7f98b80b2060 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 rx=0x7f98b000b9b0 tx=0x7f98b000bd50).stop 2021-02-10T11:34:11.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.874 7f98c03f8700 1 -- 172.21.2.101:0/2826035762 shutdown_connections 2021-02-10T11:34:11.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.874 7f98c03f8700 1 --2- 172.21.2.101:0/2826035762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98a00186c0 0x7f98a001abc0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:11.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.874 7f98c03f8700 1 --2- 172.21.2.101:0/2826035762 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98b8097fa0 0x7f98b80983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:11.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.874 7f98c03f8700 1 --2- 172.21.2.101:0/2826035762 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b80b1c60 0x7f98b80b2060 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:11.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.874 7f98c03f8700 1 --2- 172.21.2.101:0/2826035762 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98b8091ec0 0x7f98b81991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:11.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.874 7f98c03f8700 1 -- 172.21.2.101:0/2826035762 >> 172.21.2.101:0/2826035762 conn(0x7f98b81acfe0 msgr2=0x7f98b819a100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:11.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.874 7f98c03f8700 1 -- 172.21.2.101:0/2826035762 shutdown_connections 2021-02-10T11:34:11.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:11.874 7f98c03f8700 1 -- 172.21.2.101:0/2826035762 wait complete. 2021-02-10T11:34:11.893 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:34:12.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.293 7fb974d29700 1 Processor -- start 2021-02-10T11:34:12.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.293 7fb974d29700 1 -- start start 2021-02-10T11:34:12.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.293 7fb974d29700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9700d18d0 0x7fb970083ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:12.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.293 7fb974d29700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb970082d50 con 0x7fb970082920 2021-02-10T11:34:12.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.293 7fb974d29700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9700e57e0 con 0x7fb9700e53e0 2021-02-10T11:34:12.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.293 7fb974d29700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9701b0d40 con 0x7fb9700d18d0 2021-02-10T11:34:12.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.294 7fb96e312700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb970082920 0x7fb9700832d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52672/0 (socket says 172.21.2.101:52672) 2021-02-10T11:34:12.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.294 7fb96e312700 1 -- 172.21.2.101:0/2011703681 learned_addr learned my addr 172.21.2.101:0/2011703681 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:12.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.294 7fb96f314700 1 --2- 172.21.2.101:0/2011703681 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9700d18d0 0x7fb970083ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:12.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.294 7fb96db11700 1 -- 172.21.2.101:0/2011703681 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936555241 0 0) 0x7fb9640010b0 con 0x7fb970082920 2021-02-10T11:34:12.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.294 7fb96db11700 1 -- 172.21.2.101:0/2011703681 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb958002fe0 con 0x7fb970082920 2021-02-10T11:34:12.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.294 7fb96db11700 1 -- 172.21.2.101:0/2011703681 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2108350156 0 0) 0x7fb964000e20 con 0x7fb970082920 2021-02-10T11:34:12.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.294 7fb96db11700 1 -- 172.21.2.101:0/2011703681 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9700d18d0 msgr2=0x7fb970083ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:12.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.294 7fb96db11700 1 --2- 172.21.2.101:0/2011703681 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9700d18d0 0x7fb970083ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:12.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.294 7fb96db11700 1 -- 172.21.2.101:0/2011703681 >> v1:172.21.2.108:6789/0 conn(0x7fb9700e53e0 legacy=0x7fb9701b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:12.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.294 7fb96db11700 1 -- 172.21.2.101:0/2011703681 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9700d1cd0 con 0x7fb970082920 2021-02-10T11:34:12.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.294 7fb96db11700 1 -- 172.21.2.101:0/2011703681 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fb964001700 con 0x7fb970082920 2021-02-10T11:34:12.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.294 7fb96db11700 1 -- 172.21.2.101:0/2011703681 >> v1:172.21.2.101:6789/0 conn(0x7fb970082920 legacy=0x7fb9700832d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:12.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.294 7fb96db11700 1 --2- 172.21.2.101:0/2011703681 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb958003b10 0x7fb958005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:12.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.294 7fb96db11700 1 --2- 172.21.2.101:0/2011703681 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb958006800 0x7fb958008d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:12.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.294 7fb96db11700 1 --2- 172.21.2.101:0/2011703681 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb958009620 0x7fb95800bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:12.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.294 7fb96db11700 1 -- 172.21.2.101:0/2011703681 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95800c360 con 0x7fb958003b10 2021-02-10T11:34:12.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.294 7fb96db11700 1 -- 172.21.2.101:0/2011703681 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95800c580 con 0x7fb958006800 2021-02-10T11:34:12.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.294 7fb96db11700 1 -- 172.21.2.101:0/2011703681 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95800c7a0 con 0x7fb958009620 2021-02-10T11:34:12.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.294 7fb96eb13700 1 --2- 172.21.2.101:0/2011703681 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb958003b10 0x7fb958005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:12.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.294 7fb96f314700 1 --2- 172.21.2.101:0/2011703681 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9700d18d0 0x7fb970083ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:34:12.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.295 7fb96e312700 1 --2- 172.21.2.101:0/2011703681 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb958006800 0x7fb958008d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:12.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.295 7fb96f314700 1 --2- 172.21.2.101:0/2011703681 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb958009620 0x7fb95800bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:12.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.295 7fb96eb13700 1 -- 172.21.2.101:0/2011703681 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb958009620 msgr2=0x7fb95800bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:12.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.295 7fb96eb13700 1 --2- 172.21.2.101:0/2011703681 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb958009620 0x7fb95800bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:12.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.295 7fb96eb13700 1 -- 172.21.2.101:0/2011703681 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb958006800 msgr2=0x7fb958008d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:12.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.295 7fb96eb13700 1 --2- 172.21.2.101:0/2011703681 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb958006800 0x7fb958008d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:12.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.295 7fb96eb13700 1 -- 172.21.2.101:0/2011703681 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb958050a10 con 0x7fb958003b10 2021-02-10T11:34:12.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.295 7fb96f314700 1 --2- 172.21.2.101:0/2011703681 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb958009620 0x7fb95800bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:34:12.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.295 7fb96e312700 1 --2- 172.21.2.101:0/2011703681 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb958006800 0x7fb958008d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:34:12.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.295 7fb96eb13700 1 --2- 172.21.2.101:0/2011703681 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb958003b10 0x7fb958005f80 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7fb968006e40 tx=0x7fb968009e30).ready entity=mon.0 client_cookie=307fafee5bf09b7d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:12.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.295 7fb96db11700 1 -- 172.21.2.101:0/2011703681 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb96800b580 con 0x7fb958003b10 2021-02-10T11:34:12.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.295 7fb96db11700 1 -- 172.21.2.101:0/2011703681 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb968006a60 con 0x7fb958003b10 2021-02-10T11:34:12.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.295 7fb96db11700 1 -- 172.21.2.101:0/2011703681 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb96800baf0 con 0x7fb958003b10 2021-02-10T11:34:12.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.295 7fb974d29700 1 -- 172.21.2.101:0/2011703681 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb958003b10 msgr2=0x7fb958005f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:12.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.295 7fb974d29700 1 --2- 172.21.2.101:0/2011703681 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb958003b10 0x7fb958005f80 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7fb968006e40 tx=0x7fb968009e30).stop 2021-02-10T11:34:12.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.295 7fb96f314700 1 -- 172.21.2.101:0/2011703681 reap_dead start 2021-02-10T11:34:12.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.296 7fb974d29700 1 -- 172.21.2.101:0/2011703681 shutdown_connections 2021-02-10T11:34:12.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.296 7fb974d29700 1 -- 172.21.2.101:0/2011703681 >> 172.21.2.101:0/2011703681 conn(0x7fb9700ce500 msgr2=0x7fb9700e4610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:12.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.296 7fb974d29700 1 -- 172.21.2.101:0/2011703681 shutdown_connections 2021-02-10T11:34:12.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.296 7fb974d29700 1 -- 172.21.2.101:0/2011703681 wait complete. 2021-02-10T11:34:12.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.310 7fb974d29700 1 Processor -- start 2021-02-10T11:34:12.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.310 7fb974d29700 1 -- start start 2021-02-10T11:34:12.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.311 7fb974d29700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9700abb30 0x7fb9700abf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:12.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.311 7fb974d29700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9700b7070 0x7fb9700b7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:12.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.311 7fb974d29700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9700b7a90 0x7fb97009f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:12.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.311 7fb974d29700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9700ac550 con 0x7fb9700b7a90 2021-02-10T11:34:12.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.311 7fb974d29700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9701a49a0 con 0x7fb9700b7070 2021-02-10T11:34:12.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.311 7fb974d29700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9701a4bc0 con 0x7fb9700abb30 2021-02-10T11:34:12.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.311 7fb96f314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9700b7a90 0x7fb97009f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:12.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.311 7fb96f314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9700b7a90 0x7fb97009f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52192/0 (socket says 172.21.2.101:52192) 2021-02-10T11:34:12.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.311 7fb96f314700 1 -- 172.21.2.101:0/3334456402 learned_addr learned my addr 172.21.2.101:0/3334456402 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:12.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.311 7fb96e312700 1 --2- 172.21.2.101:0/3334456402 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9700b7070 0x7fb9700b7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:12.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.311 7fb96eb13700 1 --2- 172.21.2.101:0/3334456402 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9700abb30 0x7fb9700abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:12.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.312 7fb96f314700 1 -- 172.21.2.101:0/3334456402 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9700abb30 msgr2=0x7fb9700abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:12.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.312 7fb96f314700 1 --2- 172.21.2.101:0/3334456402 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9700abb30 0x7fb9700abf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:12.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.312 7fb96f314700 1 -- 172.21.2.101:0/3334456402 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9700b7070 msgr2=0x7fb9700b7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:12.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.312 7fb96f314700 1 --2- 172.21.2.101:0/3334456402 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9700b7070 0x7fb9700b7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:12.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.312 7fb96f314700 1 -- 172.21.2.101:0/3334456402 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97009f720 con 0x7fb9700b7a90 2021-02-10T11:34:12.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.312 7fb96f314700 1 --2- 172.21.2.101:0/3334456402 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9700b7a90 0x7fb97009f100 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7fb960007630 tx=0x7fb96000a950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:12.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.312 7fb95f7fe700 1 -- 172.21.2.101:0/3334456402 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb960005540 con 0x7fb9700b7a90 2021-02-10T11:34:12.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.312 7fb95f7fe700 1 -- 172.21.2.101:0/3334456402 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb960005b80 con 0x7fb9700b7a90 2021-02-10T11:34:12.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.312 7fb95f7fe700 1 -- 172.21.2.101:0/3334456402 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb96000b070 con 0x7fb9700b7a90 2021-02-10T11:34:12.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.312 7fb974d29700 1 -- 172.21.2.101:0/3334456402 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97009f720 con 0x7fb9700b7a90 2021-02-10T11:34:12.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.312 7fb974d29700 1 -- 172.21.2.101:0/3334456402 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97009fcb0 con 0x7fb9700b7a90 2021-02-10T11:34:12.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.313 7fb95f7fe700 1 -- 172.21.2.101:0/3334456402 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb960000c00 con 0x7fb9700b7a90 2021-02-10T11:34:12.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.313 7fb95f7fe700 1 --2- 172.21.2.101:0/3334456402 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9540186c0 0x7fb95401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:12.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.313 7fb95f7fe700 1 -- 172.21.2.101:0/3334456402 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb960032570 con 0x7fb9700b7a90 2021-02-10T11:34:12.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.313 7fb96eb13700 1 --2- 172.21.2.101:0/3334456402 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9540186c0 0x7fb95401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:12.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.313 7fb96eb13700 1 --2- 172.21.2.101:0/3334456402 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9540186c0 0x7fb95401abc0 crc :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:12.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.343 7fb974d29700 1 -- 172.21.2.101:0/3334456402 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97009ff60 con 0x7fb9700b7a90 2021-02-10T11:34:12.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.346 7fb95f7fe700 1 -- 172.21.2.101:0/3334456402 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb96001cb80 con 0x7fb9700b7a90 2021-02-10T11:34:12.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.476 7fb974d29700 1 -- 172.21.2.101:0/3334456402 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb97009ff60 con 0x7fb9700b7a90 2021-02-10T11:34:12.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.477 7fb95f7fe700 1 -- 172.21.2.101:0/3334456402 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v37) v1 ==== 80+0+4513 (secure 0 0 0) 0x7fb96001cb80 con 0x7fb9700b7a90 2021-02-10T11:34:12.488 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:12.488 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:34:12.488 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:34:12.488 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:12.489 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:34:12.489 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:34:12.489 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:12.489 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:34:12.489 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:34:12.489 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:34:12.489 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:12.489 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:12.489 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:12.489 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:12.489 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:34:12.489 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:34:12.489 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:12.490 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:34:12.490 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:34:12.490 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:34:12.490 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:12.490 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:12.490 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:12.490 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:12.490 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:34:12.490 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:34:12.490 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:12.490 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:34:12.490 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:34:12.490 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:12.491 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:12.491 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:12.491 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:12.491 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:34:12.491 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:34:12.491 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:12.491 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:34:12.491 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:34:12.491 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:12.491 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:12.491 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:12.491 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:12.491 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:34:12.492 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:34:12.492 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:12.492 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:34:12.492 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:34:12.492 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:12.492 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:12.492 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:12.492 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:12.492 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:34:12.492 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:34:12.492 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:12.492 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:34:12.493 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:34:12.493 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:12.493 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:12.493 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:12.493 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:12.493 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:34:12.493 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:34:12.493 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:12.493 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:34:12.493 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:34:12.493 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:12.493 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:12.493 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:12.493 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:12.494 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:34:12.494 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:34:12.494 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:12.494 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:34:12.494 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:34:12.494 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:12.494 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:12.494 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:12.494 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:12.494 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:34:12.494 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:34:12.494 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:12.494 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:34:12.495 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:34:12.495 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:12.495 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:12.495 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:12.495 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:12.495 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:34:12.495 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:34:12.495 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:12.495 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:34:12.495 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:34:12.495 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:12.495 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:12.495 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:12.495 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:12.496 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:34:12.496 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:34:12.496 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:12.496 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:34:12.496 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:34:12.496 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:34:12.496 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:12.496 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:12.496 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:12.496 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:34:12.496 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:34:12.496 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:12.496 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:34:12.496 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:34:12.497 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:34:12.497 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:12.497 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:12.497 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:12.497 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:34:12.497 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:34:12.497 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:12.497 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:34:12.497 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:34:12.497 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:34:12.497 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:12.497 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:12.497 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:12.498 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:34:12.498 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:34:12.498 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:12.498 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:34:12.498 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:34:12.498 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:34:12.498 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:12.498 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:12.498 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:12.498 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:34:12.498 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:34:12.498 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:12.499 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:34:12.499 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:34:12.499 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:34:12.499 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:12.499 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:12.499 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:12.499 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:12.499 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:34:12.499 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:34:12.499 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:12.499 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:34:12.499 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:34:12.499 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:12.499 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:12.500 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:12.500 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:12.500 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:34:12.500 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:34:12.500 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:12.500 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:34:12.500 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:34:12.500 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:12.500 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:12.500 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:12.500 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:12.500 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:34:12.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.479 7fb974d29700 1 -- 172.21.2.101:0/3334456402 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9540186c0 msgr2=0x7fb95401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:12.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.479 7fb974d29700 1 --2- 172.21.2.101:0/3334456402 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9540186c0 0x7fb95401abc0 crc :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:12.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.479 7fb974d29700 1 -- 172.21.2.101:0/3334456402 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9700b7a90 msgr2=0x7fb97009f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:12.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.479 7fb974d29700 1 --2- 172.21.2.101:0/3334456402 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9700b7a90 0x7fb97009f100 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7fb960007630 tx=0x7fb96000a950).stop 2021-02-10T11:34:12.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.479 7fb974d29700 1 -- 172.21.2.101:0/3334456402 shutdown_connections 2021-02-10T11:34:12.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.479 7fb974d29700 1 --2- 172.21.2.101:0/3334456402 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb9700b7a90 0x7fb97009f100 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:12.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.479 7fb974d29700 1 --2- 172.21.2.101:0/3334456402 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb9700b7070 0x7fb9700b7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:12.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.479 7fb974d29700 1 --2- 172.21.2.101:0/3334456402 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb9540186c0 0x7fb95401abc0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:12.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.479 7fb974d29700 1 --2- 172.21.2.101:0/3334456402 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb9700abb30 0x7fb9700abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:12.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.479 7fb974d29700 1 -- 172.21.2.101:0/3334456402 >> 172.21.2.101:0/3334456402 conn(0x7fb9700ad180 msgr2=0x7fb9700b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:12.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.479 7fb974d29700 1 -- 172.21.2.101:0/3334456402 shutdown_connections 2021-02-10T11:34:12.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.479 7fb974d29700 1 -- 172.21.2.101:0/3334456402 wait complete. 2021-02-10T11:34:12.503 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000130711 --group_name group_0000000000586931 2021-02-10T11:34:12.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.906 7f91db9e9700 1 Processor -- start 2021-02-10T11:34:12.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.906 7f91db9e9700 1 -- start start 2021-02-10T11:34:12.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.907 7f91db9e9700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f91d40e5180 0x7f91d41b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:12.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.907 7f91db9e9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91d40e5580 0x7f91d407a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:12.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.907 7f91db9e9700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d4078b90 con 0x7f91d41b3f10 2021-02-10T11:34:12.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.907 7f91db9e9700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d40d1910 con 0x7f91d40e5180 2021-02-10T11:34:12.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.907 7f91db9e9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d40d1b30 con 0x7f91d40e5580 2021-02-10T11:34:12.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.907 7f91d9004700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f91d41b3f10 0x7f91d4077510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52690/0 (socket says 172.21.2.101:52690) 2021-02-10T11:34:12.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.907 7f91d9004700 1 -- 172.21.2.101:0/4188712079 learned_addr learned my addr 172.21.2.101:0/4188712079 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:12.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.907 7f91d8803700 1 -- 172.21.2.101:0/4188712079 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 711775292 0 0) 0x7f91c80010b0 con 0x7f91d41b3f10 2021-02-10T11:34:12.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91d9805700 1 --2- 172.21.2.101:0/4188712079 >> v2:172.21.2.108:3300/0 conn(0x7f91d40e5180 0x7f91d41b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:12.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91d8803700 1 -- 172.21.2.101:0/4188712079 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91c0002fe0 con 0x7f91d41b3f10 2021-02-10T11:34:12.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91da006700 1 --2- 172.21.2.101:0/4188712079 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91d40e5580 0x7f91d407a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:12.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91d8803700 1 -- 172.21.2.101:0/4188712079 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 919091974 0 0) 0x7f91c80010b0 con 0x7f91d41b3f10 2021-02-10T11:34:12.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91d8803700 1 -- 172.21.2.101:0/4188712079 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91d40e5580 msgr2=0x7f91d407a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:12.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91d8803700 1 --2- 172.21.2.101:0/4188712079 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91d40e5580 0x7f91d407a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:12.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91d8803700 1 -- 172.21.2.101:0/4188712079 >> v2:172.21.2.108:3300/0 conn(0x7f91d40e5180 msgr2=0x7f91d41b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:12.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91d8803700 1 --2- 172.21.2.101:0/4188712079 >> v2:172.21.2.108:3300/0 conn(0x7f91d40e5180 0x7f91d41b38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:12.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91d8803700 1 -- 172.21.2.101:0/4188712079 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d40d1d50 con 0x7f91d41b3f10 2021-02-10T11:34:12.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91d8803700 1 -- 172.21.2.101:0/4188712079 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f91c80015b0 con 0x7f91d41b3f10 2021-02-10T11:34:12.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91d8803700 1 -- 172.21.2.101:0/4188712079 >> v1:172.21.2.101:6789/0 conn(0x7f91d41b3f10 legacy=0x7f91d4077510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:12.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91d8803700 1 --2- 172.21.2.101:0/4188712079 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91c0003ae0 0x7f91c0005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:12.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91d8803700 1 --2- 172.21.2.101:0/4188712079 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f91c004a780 0x7f91c004cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:12.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91d8803700 1 --2- 172.21.2.101:0/4188712079 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91c004d5a0 0x7f91c004fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:12.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91d8803700 1 -- 172.21.2.101:0/4188712079 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c00502e0 con 0x7f91c0003ae0 2021-02-10T11:34:12.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91d8803700 1 -- 172.21.2.101:0/4188712079 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c0050500 con 0x7f91c004a780 2021-02-10T11:34:12.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91d8803700 1 -- 172.21.2.101:0/4188712079 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c0050720 con 0x7f91c004d5a0 2021-02-10T11:34:12.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91d9805700 1 --2- 172.21.2.101:0/4188712079 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91c0003ae0 0x7f91c0005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:12.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91d9805700 1 -- 172.21.2.101:0/4188712079 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91c004d5a0 msgr2=0x7f91c004fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:12.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91d9805700 1 --2- 172.21.2.101:0/4188712079 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91c004d5a0 0x7f91c004fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:12.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91d9805700 1 -- 172.21.2.101:0/4188712079 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f91c004a780 msgr2=0x7f91c004cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:12.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91d9805700 1 --2- 172.21.2.101:0/4188712079 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f91c004a780 0x7f91c004cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:12.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.908 7f91d9805700 1 -- 172.21.2.101:0/4188712079 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c00509c0 con 0x7f91c0003ae0 2021-02-10T11:34:12.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.909 7f91d9805700 1 --2- 172.21.2.101:0/4188712079 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91c0003ae0 0x7f91c0005ef0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7f91d000c8f0 tx=0x7f91d000cc90).ready entity=mon.0 client_cookie=f0a56d3fb6bcdbd5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:12.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.909 7f91d8803700 1 -- 172.21.2.101:0/4188712079 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f91d000cdd0 con 0x7f91c0003ae0 2021-02-10T11:34:12.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.909 7f91d8803700 1 -- 172.21.2.101:0/4188712079 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d0009310 con 0x7f91c0003ae0 2021-02-10T11:34:12.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.909 7f91d8803700 1 -- 172.21.2.101:0/4188712079 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f91d0009830 con 0x7f91c0003ae0 2021-02-10T11:34:12.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.909 7f91db9e9700 1 -- 172.21.2.101:0/4188712079 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91c0003ae0 msgr2=0x7f91c0005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:12.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.909 7f91db9e9700 1 --2- 172.21.2.101:0/4188712079 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91c0003ae0 0x7f91c0005ef0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7f91d000c8f0 tx=0x7f91d000cc90).stop 2021-02-10T11:34:12.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.909 7f91da006700 1 -- 172.21.2.101:0/4188712079 reap_dead start 2021-02-10T11:34:12.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.909 7f91db9e9700 1 -- 172.21.2.101:0/4188712079 shutdown_connections 2021-02-10T11:34:12.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.909 7f91db9e9700 1 -- 172.21.2.101:0/4188712079 >> 172.21.2.101:0/4188712079 conn(0x7f91d40ce830 msgr2=0x7f91d40e4670 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:12.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.910 7f91db9e9700 1 -- 172.21.2.101:0/4188712079 shutdown_connections 2021-02-10T11:34:12.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.910 7f91db9e9700 1 -- 172.21.2.101:0/4188712079 wait complete. 2021-02-10T11:34:12.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.925 7f91db9e9700 1 Processor -- start 2021-02-10T11:34:12.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.925 7f91db9e9700 1 -- start start 2021-02-10T11:34:12.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.925 7f91db9e9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91d40b6b10 0x7f91d41a2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:12.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.925 7f91db9e9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91d41a2f30 0x7f91d41a3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:12.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.925 7f91db9e9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f91d40a3ec0 0x7f91d40a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:12.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.925 7f91db9e9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d40a48e0 con 0x7f91d40b6b10 2021-02-10T11:34:12.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.925 7f91db9e9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d40a4b00 con 0x7f91d40a3ec0 2021-02-10T11:34:12.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.925 7f91db9e9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d40a4d20 con 0x7f91d41a2f30 2021-02-10T11:34:12.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.926 7f91d9805700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91d40b6b10 0x7f91d41a2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:12.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.926 7f91d9805700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91d40b6b10 0x7f91d41a2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52208/0 (socket says 172.21.2.101:52208) 2021-02-10T11:34:12.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.926 7f91d9805700 1 -- 172.21.2.101:0/3652746515 learned_addr learned my addr 172.21.2.101:0/3652746515 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:12.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.926 7f91da006700 1 --2- 172.21.2.101:0/3652746515 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f91d40a3ec0 0x7f91d40a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:12.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.926 7f91d9004700 1 --2- 172.21.2.101:0/3652746515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91d41a2f30 0x7f91d41a3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:12.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.926 7f91d9805700 1 -- 172.21.2.101:0/3652746515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91d41a2f30 msgr2=0x7f91d41a3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:12.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.926 7f91d9805700 1 --2- 172.21.2.101:0/3652746515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91d41a2f30 0x7f91d41a3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:12.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.926 7f91d9805700 1 -- 172.21.2.101:0/3652746515 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f91d40a3ec0 msgr2=0x7f91d40a42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:12.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.926 7f91d9805700 1 --2- 172.21.2.101:0/3652746515 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f91d40a3ec0 0x7f91d40a42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:12.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.926 7f91d9805700 1 -- 172.21.2.101:0/3652746515 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d40a4f40 con 0x7f91d40b6b10 2021-02-10T11:34:12.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.926 7f91d9805700 1 --2- 172.21.2.101:0/3652746515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91d40b6b10 0x7f91d41a2910 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0x7f91d0009d10 tx=0x7f91d000b300).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:12.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.926 7f91c67fc700 1 -- 172.21.2.101:0/3652746515 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f91d000c1e0 con 0x7f91d40b6b10 2021-02-10T11:34:12.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.926 7f91c67fc700 1 -- 172.21.2.101:0/3652746515 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d000c8f0 con 0x7f91d40b6b10 2021-02-10T11:34:12.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.926 7f91db9e9700 1 -- 172.21.2.101:0/3652746515 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d40a4f40 con 0x7f91d40b6b10 2021-02-10T11:34:12.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.926 7f91c67fc700 1 -- 172.21.2.101:0/3652746515 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f91d000cd80 con 0x7f91d40b6b10 2021-02-10T11:34:12.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.927 7f91db9e9700 1 -- 172.21.2.101:0/3652746515 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d40a4f40 con 0x7f91d40b6b10 2021-02-10T11:34:12.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.927 7f91c67fc700 1 -- 172.21.2.101:0/3652746515 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f91d000b6f0 con 0x7f91d40b6b10 2021-02-10T11:34:12.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.927 7f91c67fc700 1 --2- 172.21.2.101:0/3652746515 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f91bc018730 0x7f91bc01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:12.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.927 7f91c67fc700 1 -- 172.21.2.101:0/3652746515 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f91d0032640 con 0x7f91d40b6b10 2021-02-10T11:34:12.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.927 7f91d9004700 1 --2- 172.21.2.101:0/3652746515 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f91bc018730 0x7f91bc01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:12.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.928 7f91d9004700 1 --2- 172.21.2.101:0/3652746515 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f91bc018730 0x7f91bc01ac10 crc :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:12.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.957 7f91db9e9700 1 -- 172.21.2.101:0/3652746515 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d410ca50 con 0x7f91d40b6b10 2021-02-10T11:34:12.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:12.959 7f91c67fc700 1 -- 172.21.2.101:0/3652746515 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f91d0000c90 con 0x7f91d40b6b10 2021-02-10T11:34:13.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.087 7f91db9e9700 1 -- 172.21.2.101:0/3652746515 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000130711", "target": ["mgr", ""], "group_name": "group_0000000000586931"}) v1 -- 0x7f91d40fe2a0 con 0x7f91bc018730 2021-02-10T11:34:13.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.098 7f91c67fc700 1 -- 172.21.2.101:0/3652746515 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f91c8000e20 con 0x7f91bc018730 2021-02-10T11:34:13.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.104 7f91db9e9700 1 -- 172.21.2.101:0/3652746515 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f91bc018730 msgr2=0x7f91bc01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:13.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.104 7f91db9e9700 1 --2- 172.21.2.101:0/3652746515 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f91bc018730 0x7f91bc01ac10 crc :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:13.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.104 7f91db9e9700 1 -- 172.21.2.101:0/3652746515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91d40b6b10 msgr2=0x7f91d41a2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:13.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.104 7f91db9e9700 1 --2- 172.21.2.101:0/3652746515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91d40b6b10 0x7f91d41a2910 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0x7f91d0009d10 tx=0x7f91d000b300).stop 2021-02-10T11:34:13.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.104 7f91db9e9700 1 -- 172.21.2.101:0/3652746515 shutdown_connections 2021-02-10T11:34:13.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.104 7f91db9e9700 1 --2- 172.21.2.101:0/3652746515 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f91bc018730 0x7f91bc01ac10 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:13.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.104 7f91db9e9700 1 --2- 172.21.2.101:0/3652746515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91d41a2f30 0x7f91d41a3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:13.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.104 7f91db9e9700 1 --2- 172.21.2.101:0/3652746515 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f91d40a3ec0 0x7f91d40a42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:13.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.104 7f91db9e9700 1 --2- 172.21.2.101:0/3652746515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91d40b6b10 0x7f91d41a2910 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:13.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.104 7f91db9e9700 1 -- 172.21.2.101:0/3652746515 >> 172.21.2.101:0/3652746515 conn(0x7f91d40ad750 msgr2=0x7f91d418c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:13.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.104 7f91db9e9700 1 -- 172.21.2.101:0/3652746515 shutdown_connections 2021-02-10T11:34:13.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.104 7f91db9e9700 1 -- 172.21.2.101:0/3652746515 wait complete. 2021-02-10T11:34:13.124 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolumegroup rm cephfs group_0000000000586931 2021-02-10T11:34:13.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.539 7efcffe50700 1 Processor -- start 2021-02-10T11:34:13.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.539 7efcffe50700 1 -- start start 2021-02-10T11:34:13.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.540 7efcffe50700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7efcf8078520 0x7efcf80d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:13.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.540 7efcffe50700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf8078920 con 0x7efcf80e5170 2021-02-10T11:34:13.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.540 7efcffe50700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf8078b90 con 0x7efcf80e5570 2021-02-10T11:34:13.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.540 7efcffe50700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf81b40d0 con 0x7efcf8078520 2021-02-10T11:34:13.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.540 7efcfe46d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7efcf8078520 0x7efcf80d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:13.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.540 7efcfe46d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7efcf8078520 0x7efcf80d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52216/0 (socket says 172.21.2.101:52216) 2021-02-10T11:34:13.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.540 7efcfe46d700 1 -- 172.21.2.101:0/451277720 learned_addr learned my addr 172.21.2.101:0/451277720 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:13.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.540 7efcfdc6c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7efcf80e5170 0x7efcf81b38e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52708/0 (socket says 172.21.2.101:52708) 2021-02-10T11:34:13.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.541 7efcfe46d700 1 -- 172.21.2.101:0/451277720 >> v1:172.21.2.108:6789/0 conn(0x7efcf80e5570 legacy=0x7efcf8079110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:13.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.541 7efcfe46d700 1 -- 172.21.2.101:0/451277720 >> v1:172.21.2.101:6789/0 conn(0x7efcf80e5170 legacy=0x7efcf81b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:13.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.541 7efcfe46d700 1 -- 172.21.2.101:0/451277720 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf80d1f20 con 0x7efcf8078520 2021-02-10T11:34:13.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.541 7efcfe46d700 1 --2- 172.21.2.101:0/451277720 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcf8078520 0x7efcf80d1900 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0x7efcf0006f00 tx=0x7efcf0009f10).ready entity=mon.0 client_cookie=3441bf043fc13db4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:13.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.541 7efcfcc6a700 1 -- 172.21.2.101:0/451277720 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efcf000b3b0 con 0x7efcf8078520 2021-02-10T11:34:13.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.541 7efcfcc6a700 1 -- 172.21.2.101:0/451277720 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf000b630 con 0x7efcf8078520 2021-02-10T11:34:13.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.541 7efcfcc6a700 1 -- 172.21.2.101:0/451277720 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efcf000bb50 con 0x7efcf8078520 2021-02-10T11:34:13.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.541 7efcffe50700 1 -- 172.21.2.101:0/451277720 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcf8078520 msgr2=0x7efcf80d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:13.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.541 7efcffe50700 1 --2- 172.21.2.101:0/451277720 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcf8078520 0x7efcf80d1900 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0x7efcf0006f00 tx=0x7efcf0009f10).stop 2021-02-10T11:34:13.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.541 7efcffe50700 1 -- 172.21.2.101:0/451277720 shutdown_connections 2021-02-10T11:34:13.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.541 7efcffe50700 1 --2- 172.21.2.101:0/451277720 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcf8078520 0x7efcf80d1900 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:13.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.541 7efcffe50700 1 -- 172.21.2.101:0/451277720 >> 172.21.2.101:0/451277720 conn(0x7efcf80ce820 msgr2=0x7efcf80e4660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:13.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.542 7efcffe50700 1 -- 172.21.2.101:0/451277720 shutdown_connections 2021-02-10T11:34:13.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.542 7efcffe50700 1 -- 172.21.2.101:0/451277720 wait complete. 2021-02-10T11:34:13.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.557 7efcffe50700 1 Processor -- start 2021-02-10T11:34:13.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.557 7efcffe50700 1 -- start start 2021-02-10T11:34:13.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.557 7efcffe50700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcf809f2a0 0x7efcf80947c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:13.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.557 7efcffe50700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efcf81c2310 0x7efcf81c2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:13.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.557 7efcffe50700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcf810a2c0 0x7efcf810a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:13.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.557 7efcffe50700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80942c0 con 0x7efcf809f2a0 2021-02-10T11:34:13.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.557 7efcffe50700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf810ad20 con 0x7efcf81c2310 2021-02-10T11:34:13.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.557 7efcffe50700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf810af40 con 0x7efcf810a2c0 2021-02-10T11:34:13.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.557 7efcfdc6c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcf809f2a0 0x7efcf80947c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:13.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.557 7efcfdc6c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcf809f2a0 0x7efcf80947c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52222/0 (socket says 172.21.2.101:52222) 2021-02-10T11:34:13.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.557 7efcfdc6c700 1 -- 172.21.2.101:0/2619054084 learned_addr learned my addr 172.21.2.101:0/2619054084 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:13.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.557 7efcfe46d700 1 --2- 172.21.2.101:0/2619054084 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcf810a2c0 0x7efcf810a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:13.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.558 7efcfd46b700 1 --2- 172.21.2.101:0/2619054084 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efcf81c2310 0x7efcf81c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:13.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.558 7efcfdc6c700 1 -- 172.21.2.101:0/2619054084 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcf810a2c0 msgr2=0x7efcf810a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:13.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.558 7efcfdc6c700 1 --2- 172.21.2.101:0/2619054084 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcf810a2c0 0x7efcf810a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:13.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.558 7efcfdc6c700 1 -- 172.21.2.101:0/2619054084 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efcf81c2310 msgr2=0x7efcf81c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:13.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.558 7efcfdc6c700 1 --2- 172.21.2.101:0/2619054084 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efcf81c2310 0x7efcf81c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:13.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.558 7efcfdc6c700 1 -- 172.21.2.101:0/2619054084 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf80927d0 con 0x7efcf809f2a0 2021-02-10T11:34:13.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.558 7efcfe46d700 1 --2- 172.21.2.101:0/2619054084 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcf810a2c0 0x7efcf810a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:34:13.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.558 7efcfdc6c700 1 --2- 172.21.2.101:0/2619054084 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcf809f2a0 0x7efcf80947c0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0x7efcf4001c60 tx=0x7efcf400a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:13.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.558 7efcea7fc700 1 -- 172.21.2.101:0/2619054084 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efcf400acc0 con 0x7efcf809f2a0 2021-02-10T11:34:13.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.558 7efcea7fc700 1 -- 172.21.2.101:0/2619054084 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf4004140 con 0x7efcf809f2a0 2021-02-10T11:34:13.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.558 7efcea7fc700 1 -- 172.21.2.101:0/2619054084 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efcf40045a0 con 0x7efcf809f2a0 2021-02-10T11:34:13.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.558 7efcffe50700 1 -- 172.21.2.101:0/2619054084 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf80927d0 con 0x7efcf809f2a0 2021-02-10T11:34:13.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.559 7efcffe50700 1 -- 172.21.2.101:0/2619054084 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf80927d0 con 0x7efcf809f2a0 2021-02-10T11:34:13.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.559 7efcea7fc700 1 -- 172.21.2.101:0/2619054084 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7efcf4000d90 con 0x7efcf809f2a0 2021-02-10T11:34:13.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.559 7efcea7fc700 1 --2- 172.21.2.101:0/2619054084 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efce00186c0 0x7efce001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:13.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.559 7efcea7fc700 1 -- 172.21.2.101:0/2619054084 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7efcf4032530 con 0x7efcf809f2a0 2021-02-10T11:34:13.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.559 7efcfd46b700 1 --2- 172.21.2.101:0/2619054084 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efce00186c0 0x7efce001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:13.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.560 7efcfd46b700 1 --2- 172.21.2.101:0/2619054084 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efce00186c0 0x7efce001abc0 crc :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:13.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.590 7efcffe50700 1 -- 172.21.2.101:0/2619054084 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf81035c0 con 0x7efcf809f2a0 2021-02-10T11:34:13.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.592 7efcea7fc700 1 -- 172.21.2.101:0/2619054084 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7efcf40009b0 con 0x7efcf809f2a0 2021-02-10T11:34:13.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.721 7efcffe50700 1 -- 172.21.2.101:0/2619054084 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "target": ["mgr", ""], "group_name": "group_0000000000586931"}) v1 -- 0x7efcf810af40 con 0x7efce00186c0 2021-02-10T11:34:13.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.725 7efcea7fc700 1 -- 172.21.2.101:0/2619054084 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7efcec006a90 con 0x7efce00186c0 2021-02-10T11:34:13.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.730 7efcffe50700 1 -- 172.21.2.101:0/2619054084 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efce00186c0 msgr2=0x7efce001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:13.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.730 7efcffe50700 1 --2- 172.21.2.101:0/2619054084 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efce00186c0 0x7efce001abc0 crc :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:13.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.730 7efcffe50700 1 -- 172.21.2.101:0/2619054084 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcf809f2a0 msgr2=0x7efcf80947c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:13.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.730 7efcffe50700 1 --2- 172.21.2.101:0/2619054084 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcf809f2a0 0x7efcf80947c0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0x7efcf4001c60 tx=0x7efcf400a8d0).stop 2021-02-10T11:34:13.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.731 7efcffe50700 1 -- 172.21.2.101:0/2619054084 shutdown_connections 2021-02-10T11:34:13.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.731 7efcffe50700 1 --2- 172.21.2.101:0/2619054084 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efce00186c0 0x7efce001abc0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:13.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.731 7efcffe50700 1 --2- 172.21.2.101:0/2619054084 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcf810a2c0 0x7efcf810a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:13.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.731 7efcffe50700 1 --2- 172.21.2.101:0/2619054084 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efcf81c2310 0x7efcf81c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:13.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.731 7efcffe50700 1 --2- 172.21.2.101:0/2619054084 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcf809f2a0 0x7efcf80947c0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:13.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.731 7efcffe50700 1 -- 172.21.2.101:0/2619054084 >> 172.21.2.101:0/2619054084 conn(0x7efcf8091890 msgr2=0x7efcf8091ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:13.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.731 7efcffe50700 1 -- 172.21.2.101:0/2619054084 shutdown_connections 2021-02-10T11:34:13.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:13.731 7efcffe50700 1 -- 172.21.2.101:0/2619054084 wait complete. 2021-02-10T11:34:13.749 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:34:13.750 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:34:14.095 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:34:14.439 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:34:14.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:14.439 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:34:14.439 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:34:14.775 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:34:14.776 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:34:14.776 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:34:14.776 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:34:14.776 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:34:15.129 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[118322]: fuse finished with error 0 and tester_r 0 2021-02-10T11:34:15.130 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:15.130 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:34:15.130 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:34:15.465 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:34:15.465 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:34:15.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:34:15.465 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:34:15.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:15.800 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:34:15.800 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:34:16.134 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:34:16.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:34:16.134 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:34:16.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:34:16.134 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:34:16.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:16.469 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:34:16.469 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:34:16.804 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:34:16.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:34:16.805 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:34:16.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:34:16.805 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:34:17.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:17.140 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:34:17.140 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:34:17.475 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:34:17.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:34:17.475 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:34:17.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:34:17.476 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:34:17.810 DEBUG:teuthology.orchestra.run.gibba001:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-02-10T11:34:17.813 DEBUG:teuthology.orchestra.run.gibba008:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-02-10T11:34:18.172 INFO:teuthology.orchestra.run.gibba001.stdout:[global] 2021-02-10T11:34:18.172 INFO:teuthology.orchestra.run.gibba001.stdout: chdir = "" 2021-02-10T11:34:18.172 INFO:teuthology.orchestra.run.gibba001.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-02-10T11:34:18.173 INFO:teuthology.orchestra.run.gibba001.stdout: auth supported = cephx 2021-02-10T11:34:18.173 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.173 INFO:teuthology.orchestra.run.gibba001.stdout: filestore xattr use omap = true 2021-02-10T11:34:18.173 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.173 INFO:teuthology.orchestra.run.gibba001.stdout: mon clock drift allowed = 1.000 2021-02-10T11:34:18.173 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.173 INFO:teuthology.orchestra.run.gibba001.stdout: osd crush chooseleaf type = 0 2021-02-10T11:34:18.173 INFO:teuthology.orchestra.run.gibba001.stdout: auth debug = true 2021-02-10T11:34:18.173 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.173 INFO:teuthology.orchestra.run.gibba001.stdout: ms die on old message = true 2021-02-10T11:34:18.173 INFO:teuthology.orchestra.run.gibba001.stdout: ms die on bug = true 2021-02-10T11:34:18.173 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.173 INFO:teuthology.orchestra.run.gibba001.stdout: mon max pg per osd = 10000 # >= luminous 2021-02-10T11:34:18.173 INFO:teuthology.orchestra.run.gibba001.stdout: mon pg warn max object skew = 0 2021-02-10T11:34:18.174 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.174 INFO:teuthology.orchestra.run.gibba001.stdout: osd pool default size = 2 2021-02-10T11:34:18.174 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.174 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd allow primary affinity = true 2021-02-10T11:34:18.174 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd allow pg remap = true 2021-02-10T11:34:18.174 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on legacy crush tunables = false 2021-02-10T11:34:18.174 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on crush straw calc version zero = false 2021-02-10T11:34:18.174 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on no sortbitwise = false 2021-02-10T11:34:18.174 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on osd down out interval zero = false 2021-02-10T11:34:18.174 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on too few osds = false 2021-02-10T11:34:18.174 INFO:teuthology.orchestra.run.gibba001.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-02-10T11:34:18.174 INFO:teuthology.orchestra.run.gibba001.stdout: mon_warn_on_pool_no_redundancy = false 2021-02-10T11:34:18.174 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.175 INFO:teuthology.orchestra.run.gibba001.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-02-10T11:34:18.175 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.175 INFO:teuthology.orchestra.run.gibba001.stdout: osd default data pool replay window = 5 2021-02-10T11:34:18.175 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.175 INFO:teuthology.orchestra.run.gibba001.stdout: mon allow pool delete = true 2021-02-10T11:34:18.175 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.175 INFO:teuthology.orchestra.run.gibba001.stdout: mon cluster log file level = debug 2021-02-10T11:34:18.175 INFO:teuthology.orchestra.run.gibba001.stdout: debug asserts on shutdown = true 2021-02-10T11:34:18.175 INFO:teuthology.orchestra.run.gibba001.stdout: mon health detail to clog = false 2021-02-10T11:34:18.175 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.175 INFO:teuthology.orchestra.run.gibba001.stdout: # we see this fail in qa on *nautilus*; bump up retries 2021-02-10T11:34:18.175 INFO:teuthology.orchestra.run.gibba001.stdout: mon_client_directed_command_retry = 4 2021-02-10T11:34:18.175 INFO:teuthology.orchestra.run.gibba001.stdout: mon host = "172.21.2.101,172.21.2.108,[v2:172.21.2.108:3301,v1:172.21.2.108:6790]" 2021-02-10T11:34:18.176 INFO:teuthology.orchestra.run.gibba001.stdout: fsid = a6663ba9-2653-4542-8b6a-9323b4e7b545 2021-02-10T11:34:18.176 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.176 INFO:teuthology.orchestra.run.gibba001.stdout:[osd] 2021-02-10T11:34:18.176 INFO:teuthology.orchestra.run.gibba001.stdout: osd journal size = 100 2021-02-10T11:34:18.176 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.176 INFO:teuthology.orchestra.run.gibba001.stdout: osd scrub load threshold = 5.0 2021-02-10T11:34:18.176 INFO:teuthology.orchestra.run.gibba001.stdout: osd scrub max interval = 600 2021-02-10T11:34:18.176 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.176 INFO:teuthology.orchestra.run.gibba001.stdout: osd recover clone overlap = true 2021-02-10T11:34:18.176 INFO:teuthology.orchestra.run.gibba001.stdout: osd recovery max chunk = 1048576 2021-02-10T11:34:18.176 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.176 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug shutdown = true 2021-02-10T11:34:18.176 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug op order = true 2021-02-10T11:34:18.176 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug verify stray on activate = true 2021-02-10T11:34:18.177 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.177 INFO:teuthology.orchestra.run.gibba001.stdout: osd open classes on start = true 2021-02-10T11:34:18.177 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug pg log writeout = true 2021-02-10T11:34:18.177 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.177 INFO:teuthology.orchestra.run.gibba001.stdout: osd deep scrub update digest min age = 30 2021-02-10T11:34:18.177 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.177 INFO:teuthology.orchestra.run.gibba001.stdout: osd map max advance = 10 2021-02-10T11:34:18.177 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.177 INFO:teuthology.orchestra.run.gibba001.stdout: journal zero on create = true 2021-02-10T11:34:18.177 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.177 INFO:teuthology.orchestra.run.gibba001.stdout: filestore ondisk finisher threads = 3 2021-02-10T11:34:18.177 INFO:teuthology.orchestra.run.gibba001.stdout: filestore apply finisher threads = 3 2021-02-10T11:34:18.177 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.178 INFO:teuthology.orchestra.run.gibba001.stdout: bdev debug aio = true 2021-02-10T11:34:18.178 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug misdirected ops = true 2021-02-10T11:34:18.178 INFO:teuthology.orchestra.run.gibba001.stdout: bdev async discard = True 2021-02-10T11:34:18.178 INFO:teuthology.orchestra.run.gibba001.stdout: bdev enable discard = True 2021-02-10T11:34:18.178 INFO:teuthology.orchestra.run.gibba001.stdout: bluestore allocator = bitmap 2021-02-10T11:34:18.178 INFO:teuthology.orchestra.run.gibba001.stdout: bluestore block size = 96636764160 2021-02-10T11:34:18.178 INFO:teuthology.orchestra.run.gibba001.stdout: bluestore fsck on mount = True 2021-02-10T11:34:18.178 INFO:teuthology.orchestra.run.gibba001.stdout: debug bluefs = 20 2021-02-10T11:34:18.178 INFO:teuthology.orchestra.run.gibba001.stdout: debug bluestore = 20 2021-02-10T11:34:18.178 INFO:teuthology.orchestra.run.gibba001.stdout: debug filestore = 20 2021-02-10T11:34:18.178 INFO:teuthology.orchestra.run.gibba001.stdout: debug journal = 20 2021-02-10T11:34:18.178 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 20 2021-02-10T11:34:18.178 INFO:teuthology.orchestra.run.gibba001.stdout: debug osd = 25 2021-02-10T11:34:18.178 INFO:teuthology.orchestra.run.gibba001.stdout: debug rocksdb = 10 2021-02-10T11:34:18.179 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd backfillfull_ratio = 0.85 2021-02-10T11:34:18.179 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd full ratio = 0.9 2021-02-10T11:34:18.179 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd nearfull ratio = 0.8 2021-02-10T11:34:18.179 INFO:teuthology.orchestra.run.gibba001.stdout: osd failsafe full ratio = 0.95 2021-02-10T11:34:18.179 INFO:teuthology.orchestra.run.gibba001.stdout: osd objectstore = bluestore 2021-02-10T11:34:18.179 INFO:teuthology.orchestra.run.gibba001.stdout: osd op complaint time = 180 2021-02-10T11:34:18.179 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.179 INFO:teuthology.orchestra.run.gibba001.stdout:[mgr] 2021-02-10T11:34:18.179 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 1 2021-02-10T11:34:18.179 INFO:teuthology.orchestra.run.gibba001.stdout: debug mgr = 20 2021-02-10T11:34:18.179 INFO:teuthology.orchestra.run.gibba001.stdout: debug mon = 20 2021-02-10T11:34:18.179 INFO:teuthology.orchestra.run.gibba001.stdout: debug auth = 20 2021-02-10T11:34:18.179 INFO:teuthology.orchestra.run.gibba001.stdout: mon reweight min pgs per osd = 4 2021-02-10T11:34:18.179 INFO:teuthology.orchestra.run.gibba001.stdout: mon reweight min bytes per osd = 10 2021-02-10T11:34:18.180 INFO:teuthology.orchestra.run.gibba001.stdout: debug client = 10 2021-02-10T11:34:18.180 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.180 INFO:teuthology.orchestra.run.gibba001.stdout:[mon] 2021-02-10T11:34:18.180 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 1 2021-02-10T11:34:18.180 INFO:teuthology.orchestra.run.gibba001.stdout: debug mon = 20 2021-02-10T11:34:18.180 INFO:teuthology.orchestra.run.gibba001.stdout: debug paxos = 20 2021-02-10T11:34:18.180 INFO:teuthology.orchestra.run.gibba001.stdout: debug auth = 20 2021-02-10T11:34:18.180 INFO:teuthology.orchestra.run.gibba001.stdout: mon data avail warn = 5 2021-02-10T11:34:18.180 INFO:teuthology.orchestra.run.gibba001.stdout: mon mgr mkfs grace = 240 2021-02-10T11:34:18.180 INFO:teuthology.orchestra.run.gibba001.stdout: mon reweight min pgs per osd = 4 2021-02-10T11:34:18.180 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd reporter subtree level = osd 2021-02-10T11:34:18.180 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd prime pg temp = true 2021-02-10T11:34:18.180 INFO:teuthology.orchestra.run.gibba001.stdout: mon reweight min bytes per osd = 10 2021-02-10T11:34:18.180 INFO:teuthology.orchestra.run.gibba001.stdout: mon op complaint time = 120 2021-02-10T11:34:18.181 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:18.181 INFO:teuthology.orchestra.run.gibba001.stdout:[client] 2021-02-10T11:34:18.181 INFO:teuthology.orchestra.run.gibba001.stdout: rgw cache enabled = true 2021-02-10T11:34:18.181 INFO:teuthology.orchestra.run.gibba001.stdout: rgw enable ops log = true 2021-02-10T11:34:18.181 INFO:teuthology.orchestra.run.gibba001.stdout: rgw enable usage log = true 2021-02-10T11:34:18.181 INFO:teuthology.orchestra.run.gibba001.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-02-10T11:34:18.181 INFO:teuthology.orchestra.run.gibba001.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-02-10T11:34:18.181 INFO:teuthology.orchestra.run.gibba001.stdout: client mount timeout = 600 2021-02-10T11:34:18.181 INFO:teuthology.orchestra.run.gibba001.stdout: debug client = 20 2021-02-10T11:34:18.181 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 1 2021-02-10T11:34:18.181 INFO:teuthology.orchestra.run.gibba001.stdout: pid file = "" 2021-02-10T11:34:18.181 INFO:teuthology.orchestra.run.gibba001.stdout:[mon.a] 2021-02-10T11:34:18.181 INFO:teuthology.orchestra.run.gibba001.stdout:[mon.b] 2021-02-10T11:34:18.181 INFO:teuthology.orchestra.run.gibba001.stdout:[mon.c] 2021-02-10T11:34:18.182 INFO:teuthology.orchestra.run.gibba001.stdout:[mds.a] 2021-02-10T11:34:18.182 INFO:teuthology.orchestra.run.gibba001.stdout:[mds.b] 2021-02-10T11:34:18.182 INFO:teuthology.orchestra.run.gibba001.stdout:[mds] 2021-02-10T11:34:18.182 INFO:teuthology.orchestra.run.gibba001.stdout: debug mds = 20 2021-02-10T11:34:18.182 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 1 2021-02-10T11:34:18.182 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal frag = True 2021-02-10T11:34:18.182 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal fragment size max = 10000 2021-02-10T11:34:18.182 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal merge size = 5 2021-02-10T11:34:18.182 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal split bits = 3 2021-02-10T11:34:18.182 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal split size = 100 2021-02-10T11:34:18.182 INFO:teuthology.orchestra.run.gibba001.stdout: mds debug frag = True 2021-02-10T11:34:18.182 INFO:teuthology.orchestra.run.gibba001.stdout: mds debug scatterstat = True 2021-02-10T11:34:18.182 INFO:teuthology.orchestra.run.gibba001.stdout: mds op complaint time = 180 2021-02-10T11:34:18.182 INFO:teuthology.orchestra.run.gibba001.stdout: mds verify scatter = True 2021-02-10T11:34:18.183 INFO:teuthology.orchestra.run.gibba001.stdout: osd op complaint time = 180 2021-02-10T11:34:18.183 INFO:teuthology.orchestra.run.gibba001.stdout:[client.alice] 2021-02-10T11:34:18.183 INFO:teuthology.orchestra.run.gibba008.stdout:[global] 2021-02-10T11:34:18.184 INFO:teuthology.orchestra.run.gibba008.stdout: chdir = "" 2021-02-10T11:34:18.184 INFO:teuthology.orchestra.run.gibba008.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-02-10T11:34:18.184 INFO:teuthology.orchestra.run.gibba008.stdout: auth supported = cephx 2021-02-10T11:34:18.184 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.184 INFO:teuthology.orchestra.run.gibba008.stdout: filestore xattr use omap = true 2021-02-10T11:34:18.184 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.184 INFO:teuthology.orchestra.run.gibba008.stdout: mon clock drift allowed = 1.000 2021-02-10T11:34:18.184 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.184 INFO:teuthology.orchestra.run.gibba008.stdout: osd crush chooseleaf type = 0 2021-02-10T11:34:18.184 INFO:teuthology.orchestra.run.gibba008.stdout: auth debug = true 2021-02-10T11:34:18.184 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.184 INFO:teuthology.orchestra.run.gibba008.stdout: ms die on old message = true 2021-02-10T11:34:18.184 INFO:teuthology.orchestra.run.gibba008.stdout: ms die on bug = true 2021-02-10T11:34:18.185 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.185 INFO:teuthology.orchestra.run.gibba008.stdout: mon max pg per osd = 10000 # >= luminous 2021-02-10T11:34:18.185 INFO:teuthology.orchestra.run.gibba008.stdout: mon pg warn max object skew = 0 2021-02-10T11:34:18.185 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.185 INFO:teuthology.orchestra.run.gibba008.stdout: osd pool default size = 2 2021-02-10T11:34:18.185 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.185 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd allow primary affinity = true 2021-02-10T11:34:18.185 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd allow pg remap = true 2021-02-10T11:34:18.185 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on legacy crush tunables = false 2021-02-10T11:34:18.185 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on crush straw calc version zero = false 2021-02-10T11:34:18.185 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on no sortbitwise = false 2021-02-10T11:34:18.185 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on osd down out interval zero = false 2021-02-10T11:34:18.185 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on too few osds = false 2021-02-10T11:34:18.186 INFO:teuthology.orchestra.run.gibba008.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-02-10T11:34:18.186 INFO:teuthology.orchestra.run.gibba008.stdout: mon_warn_on_pool_no_redundancy = false 2021-02-10T11:34:18.186 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.186 INFO:teuthology.orchestra.run.gibba008.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-02-10T11:34:18.186 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.186 INFO:teuthology.orchestra.run.gibba008.stdout: osd default data pool replay window = 5 2021-02-10T11:34:18.186 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.186 INFO:teuthology.orchestra.run.gibba008.stdout: mon allow pool delete = true 2021-02-10T11:34:18.186 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.186 INFO:teuthology.orchestra.run.gibba008.stdout: mon cluster log file level = debug 2021-02-10T11:34:18.186 INFO:teuthology.orchestra.run.gibba008.stdout: debug asserts on shutdown = true 2021-02-10T11:34:18.186 INFO:teuthology.orchestra.run.gibba008.stdout: mon health detail to clog = false 2021-02-10T11:34:18.186 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.187 INFO:teuthology.orchestra.run.gibba008.stdout: # we see this fail in qa on *nautilus*; bump up retries 2021-02-10T11:34:18.187 INFO:teuthology.orchestra.run.gibba008.stdout: mon_client_directed_command_retry = 4 2021-02-10T11:34:18.187 INFO:teuthology.orchestra.run.gibba008.stdout: mon host = "172.21.2.101,172.21.2.108,[v2:172.21.2.108:3301,v1:172.21.2.108:6790]" 2021-02-10T11:34:18.187 INFO:teuthology.orchestra.run.gibba008.stdout: fsid = a6663ba9-2653-4542-8b6a-9323b4e7b545 2021-02-10T11:34:18.187 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.187 INFO:teuthology.orchestra.run.gibba008.stdout:[osd] 2021-02-10T11:34:18.187 INFO:teuthology.orchestra.run.gibba008.stdout: osd journal size = 100 2021-02-10T11:34:18.187 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.187 INFO:teuthology.orchestra.run.gibba008.stdout: osd scrub load threshold = 5.0 2021-02-10T11:34:18.187 INFO:teuthology.orchestra.run.gibba008.stdout: osd scrub max interval = 600 2021-02-10T11:34:18.187 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.187 INFO:teuthology.orchestra.run.gibba008.stdout: osd recover clone overlap = true 2021-02-10T11:34:18.187 INFO:teuthology.orchestra.run.gibba008.stdout: osd recovery max chunk = 1048576 2021-02-10T11:34:18.188 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.188 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug shutdown = true 2021-02-10T11:34:18.188 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug op order = true 2021-02-10T11:34:18.188 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug verify stray on activate = true 2021-02-10T11:34:18.188 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.188 INFO:teuthology.orchestra.run.gibba008.stdout: osd open classes on start = true 2021-02-10T11:34:18.188 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug pg log writeout = true 2021-02-10T11:34:18.188 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.188 INFO:teuthology.orchestra.run.gibba008.stdout: osd deep scrub update digest min age = 30 2021-02-10T11:34:18.188 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.188 INFO:teuthology.orchestra.run.gibba008.stdout: osd map max advance = 10 2021-02-10T11:34:18.188 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.188 INFO:teuthology.orchestra.run.gibba008.stdout: journal zero on create = true 2021-02-10T11:34:18.188 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.189 INFO:teuthology.orchestra.run.gibba008.stdout: filestore ondisk finisher threads = 3 2021-02-10T11:34:18.189 INFO:teuthology.orchestra.run.gibba008.stdout: filestore apply finisher threads = 3 2021-02-10T11:34:18.189 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.189 INFO:teuthology.orchestra.run.gibba008.stdout: bdev debug aio = true 2021-02-10T11:34:18.189 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug misdirected ops = true 2021-02-10T11:34:18.189 INFO:teuthology.orchestra.run.gibba008.stdout: bdev async discard = True 2021-02-10T11:34:18.189 INFO:teuthology.orchestra.run.gibba008.stdout: bdev enable discard = True 2021-02-10T11:34:18.189 INFO:teuthology.orchestra.run.gibba008.stdout: bluestore allocator = bitmap 2021-02-10T11:34:18.189 INFO:teuthology.orchestra.run.gibba008.stdout: bluestore block size = 96636764160 2021-02-10T11:34:18.189 INFO:teuthology.orchestra.run.gibba008.stdout: bluestore fsck on mount = True 2021-02-10T11:34:18.189 INFO:teuthology.orchestra.run.gibba008.stdout: debug bluefs = 20 2021-02-10T11:34:18.189 INFO:teuthology.orchestra.run.gibba008.stdout: debug bluestore = 20 2021-02-10T11:34:18.189 INFO:teuthology.orchestra.run.gibba008.stdout: debug filestore = 20 2021-02-10T11:34:18.189 INFO:teuthology.orchestra.run.gibba008.stdout: debug journal = 20 2021-02-10T11:34:18.190 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 20 2021-02-10T11:34:18.190 INFO:teuthology.orchestra.run.gibba008.stdout: debug osd = 25 2021-02-10T11:34:18.190 INFO:teuthology.orchestra.run.gibba008.stdout: debug rocksdb = 10 2021-02-10T11:34:18.190 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd backfillfull_ratio = 0.85 2021-02-10T11:34:18.190 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd full ratio = 0.9 2021-02-10T11:34:18.190 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd nearfull ratio = 0.8 2021-02-10T11:34:18.190 INFO:teuthology.orchestra.run.gibba008.stdout: osd failsafe full ratio = 0.95 2021-02-10T11:34:18.190 INFO:teuthology.orchestra.run.gibba008.stdout: osd objectstore = bluestore 2021-02-10T11:34:18.190 INFO:teuthology.orchestra.run.gibba008.stdout: osd op complaint time = 180 2021-02-10T11:34:18.190 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.190 INFO:teuthology.orchestra.run.gibba008.stdout:[mgr] 2021-02-10T11:34:18.190 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 1 2021-02-10T11:34:18.190 INFO:teuthology.orchestra.run.gibba008.stdout: debug mgr = 20 2021-02-10T11:34:18.191 INFO:teuthology.orchestra.run.gibba008.stdout: debug mon = 20 2021-02-10T11:34:18.191 INFO:teuthology.orchestra.run.gibba008.stdout: debug auth = 20 2021-02-10T11:34:18.191 INFO:teuthology.orchestra.run.gibba008.stdout: mon reweight min pgs per osd = 4 2021-02-10T11:34:18.191 INFO:teuthology.orchestra.run.gibba008.stdout: mon reweight min bytes per osd = 10 2021-02-10T11:34:18.191 INFO:teuthology.orchestra.run.gibba008.stdout: debug client = 10 2021-02-10T11:34:18.191 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.191 INFO:teuthology.orchestra.run.gibba008.stdout:[mon] 2021-02-10T11:34:18.191 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 1 2021-02-10T11:34:18.191 INFO:teuthology.orchestra.run.gibba008.stdout: debug mon = 20 2021-02-10T11:34:18.191 INFO:teuthology.orchestra.run.gibba008.stdout: debug paxos = 20 2021-02-10T11:34:18.191 INFO:teuthology.orchestra.run.gibba008.stdout: debug auth = 20 2021-02-10T11:34:18.191 INFO:teuthology.orchestra.run.gibba008.stdout: mon data avail warn = 5 2021-02-10T11:34:18.191 INFO:teuthology.orchestra.run.gibba008.stdout: mon mgr mkfs grace = 240 2021-02-10T11:34:18.191 INFO:teuthology.orchestra.run.gibba008.stdout: mon reweight min pgs per osd = 4 2021-02-10T11:34:18.192 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd reporter subtree level = osd 2021-02-10T11:34:18.192 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd prime pg temp = true 2021-02-10T11:34:18.192 INFO:teuthology.orchestra.run.gibba008.stdout: mon reweight min bytes per osd = 10 2021-02-10T11:34:18.192 INFO:teuthology.orchestra.run.gibba008.stdout: mon op complaint time = 120 2021-02-10T11:34:18.192 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:34:18.192 INFO:teuthology.orchestra.run.gibba008.stdout:[client] 2021-02-10T11:34:18.192 INFO:teuthology.orchestra.run.gibba008.stdout: rgw cache enabled = true 2021-02-10T11:34:18.192 INFO:teuthology.orchestra.run.gibba008.stdout: rgw enable ops log = true 2021-02-10T11:34:18.192 INFO:teuthology.orchestra.run.gibba008.stdout: rgw enable usage log = true 2021-02-10T11:34:18.192 INFO:teuthology.orchestra.run.gibba008.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-02-10T11:34:18.192 INFO:teuthology.orchestra.run.gibba008.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-02-10T11:34:18.192 INFO:teuthology.orchestra.run.gibba008.stdout: client mount timeout = 600 2021-02-10T11:34:18.192 INFO:teuthology.orchestra.run.gibba008.stdout: debug client = 20 2021-02-10T11:34:18.193 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 1 2021-02-10T11:34:18.193 INFO:teuthology.orchestra.run.gibba008.stdout: pid file = "" 2021-02-10T11:34:18.193 INFO:teuthology.orchestra.run.gibba008.stdout:[mon.a] 2021-02-10T11:34:18.193 INFO:teuthology.orchestra.run.gibba008.stdout:[mon.b] 2021-02-10T11:34:18.193 INFO:teuthology.orchestra.run.gibba008.stdout:[mon.c] 2021-02-10T11:34:18.193 INFO:teuthology.orchestra.run.gibba008.stdout:[mds.a] 2021-02-10T11:34:18.193 INFO:teuthology.orchestra.run.gibba008.stdout:[mds.b] 2021-02-10T11:34:18.193 INFO:teuthology.orchestra.run.gibba008.stdout:[mds] 2021-02-10T11:34:18.193 INFO:teuthology.orchestra.run.gibba008.stdout: debug mds = 20 2021-02-10T11:34:18.193 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 1 2021-02-10T11:34:18.193 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal frag = True 2021-02-10T11:34:18.193 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal fragment size max = 10000 2021-02-10T11:34:18.193 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal merge size = 5 2021-02-10T11:34:18.193 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal split bits = 3 2021-02-10T11:34:18.194 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal split size = 100 2021-02-10T11:34:18.194 INFO:teuthology.orchestra.run.gibba008.stdout: mds debug frag = True 2021-02-10T11:34:18.194 INFO:teuthology.orchestra.run.gibba008.stdout: mds debug scatterstat = True 2021-02-10T11:34:18.194 INFO:teuthology.orchestra.run.gibba008.stdout: mds op complaint time = 180 2021-02-10T11:34:18.194 INFO:teuthology.orchestra.run.gibba008.stdout: mds verify scatter = True 2021-02-10T11:34:18.194 INFO:teuthology.orchestra.run.gibba008.stdout: osd op complaint time = 180 2021-02-10T11:34:18.194 INFO:teuthology.orchestra.run.gibba008.stdout:[client.alice] 2021-02-10T11:34:18.195 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-02-10T11:34:18.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.598 7f7caefa7700 1 Processor -- start 2021-02-10T11:34:18.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.598 7f7caefa7700 1 -- start start 2021-02-10T11:34:18.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.598 7f7caefa7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ca80d1870 0x7f7ca81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:18.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.598 7f7caefa7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7ca80f0650 0x7f7ca80cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:18.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.598 7f7caefa7700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ca80f0b10 con 0x7f7ca81b0a10 2021-02-10T11:34:18.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.598 7f7caefa7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca80fe230 con 0x7f7ca80f0650 2021-02-10T11:34:18.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.598 7f7caefa7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca80e5ed0 con 0x7f7ca80d1870 2021-02-10T11:34:18.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.598 7f7cad5c4700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7ca80f0650 0x7f7ca80cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:18.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.598 7f7cacdc3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ca80d1870 0x7f7ca81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:18.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.598 7f7cad5c4700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7ca80f0650 0x7f7ca80cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:34484/0 (socket says 172.21.2.101:34484) 2021-02-10T11:34:18.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.598 7f7cacdc3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ca80d1870 0x7f7ca81b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:44958/0 (socket says 172.21.2.101:44958) 2021-02-10T11:34:18.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.599 7f7cad5c4700 1 -- 172.21.2.101:0/1120606625 learned_addr learned my addr 172.21.2.101:0/1120606625 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:18.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.599 7f7cad5c4700 1 -- 172.21.2.101:0/1120606625 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ca80d1870 msgr2=0x7f7ca81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:18.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.599 7f7cad5c4700 1 --2- 172.21.2.101:0/1120606625 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ca80d1870 0x7f7ca81b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:18.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.599 7f7cad5c4700 1 -- 172.21.2.101:0/1120606625 >> v1:172.21.2.108:6789/0 conn(0x7f7ca81b0a10 legacy=0x7f7ca80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:18.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.599 7f7cad5c4700 1 -- 172.21.2.101:0/1120606625 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca80f20a0 con 0x7f7ca80f0650 2021-02-10T11:34:18.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.599 7f7cacdc3700 1 --2- 172.21.2.101:0/1120606625 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ca80d1870 0x7f7ca81b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:34:18.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.599 7f7cad5c4700 1 --2- 172.21.2.101:0/1120606625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ca80f0650 0x7f7ca80cfb20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f7c90006720 tx=0x7f7c90009ba0).ready entity=mon.1 client_cookie=b2b5ac8a1fd14264 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:18.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.599 7f7ca77fe700 1 -- 172.21.2.101:0/1120606625 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7c900022e0 con 0x7f7ca80f0650 2021-02-10T11:34:18.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.600 7f7ca77fe700 1 -- 172.21.2.101:0/1120606625 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c90002560 con 0x7f7ca80f0650 2021-02-10T11:34:18.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.600 7f7ca77fe700 1 -- 172.21.2.101:0/1120606625 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7c90002a80 con 0x7f7ca80f0650 2021-02-10T11:34:18.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.600 7f7caefa7700 1 -- 172.21.2.101:0/1120606625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ca80f0650 msgr2=0x7f7ca80cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:18.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.600 7f7caefa7700 1 --2- 172.21.2.101:0/1120606625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ca80f0650 0x7f7ca80cfb20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f7c90006720 tx=0x7f7c90009ba0).stop 2021-02-10T11:34:18.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.600 7f7caefa7700 1 -- 172.21.2.101:0/1120606625 shutdown_connections 2021-02-10T11:34:18.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.600 7f7caefa7700 1 --2- 172.21.2.101:0/1120606625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ca80f0650 0x7f7ca80cfb20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:18.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.600 7f7caefa7700 1 --2- 172.21.2.101:0/1120606625 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ca80d1870 0x7f7ca81b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:18.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.600 7f7caefa7700 1 -- 172.21.2.101:0/1120606625 >> 172.21.2.101:0/1120606625 conn(0x7f7ca80ce450 msgr2=0x7f7ca80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:18.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.600 7f7caefa7700 1 -- 172.21.2.101:0/1120606625 shutdown_connections 2021-02-10T11:34:18.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.600 7f7caefa7700 1 -- 172.21.2.101:0/1120606625 wait complete. 2021-02-10T11:34:18.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.615 7f7caefa7700 1 Processor -- start 2021-02-10T11:34:18.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.615 7f7caefa7700 1 -- start start 2021-02-10T11:34:18.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.616 7f7caefa7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ca80b5a90 0x7f7ca81a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:18.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.616 7f7caefa7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ca81a4a40 0x7f7ca80c1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:18.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.616 7f7caefa7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ca80c2410 0x7f7ca80ec470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:18.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.616 7f7caefa7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca80ebf30 con 0x7f7ca80b5a90 2021-02-10T11:34:18.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.616 7f7caefa7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca80c2810 con 0x7f7ca80c2410 2021-02-10T11:34:18.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.616 7f7caefa7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca80b55f0 con 0x7f7ca81a4a40 2021-02-10T11:34:18.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.616 7f7cacdc3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ca80b5a90 0x7f7ca81a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:18.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.616 7f7cacdc3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ca80b5a90 0x7f7ca81a4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52232/0 (socket says 172.21.2.101:52232) 2021-02-10T11:34:18.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.616 7f7cacdc3700 1 -- 172.21.2.101:0/3061888926 learned_addr learned my addr 172.21.2.101:0/3061888926 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:18.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.616 7f7ca7fff700 1 --2- 172.21.2.101:0/3061888926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ca81a4a40 0x7f7ca80c1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:18.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.616 7f7cad5c4700 1 --2- 172.21.2.101:0/3061888926 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ca80c2410 0x7f7ca80ec470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:18.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.616 7f7cacdc3700 1 -- 172.21.2.101:0/3061888926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ca81a4a40 msgr2=0x7f7ca80c1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:18.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.616 7f7cacdc3700 1 --2- 172.21.2.101:0/3061888926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ca81a4a40 0x7f7ca80c1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:18.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.616 7f7cacdc3700 1 -- 172.21.2.101:0/3061888926 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ca80c2410 msgr2=0x7f7ca80ec470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:18.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.616 7f7cacdc3700 1 --2- 172.21.2.101:0/3061888926 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ca80c2410 0x7f7ca80ec470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:18.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.616 7f7cacdc3700 1 -- 172.21.2.101:0/3061888926 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca80eca90 con 0x7f7ca80b5a90 2021-02-10T11:34:18.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.616 7f7cacdc3700 1 --2- 172.21.2.101:0/3061888926 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ca80b5a90 0x7f7ca81a4420 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7f7ca0006f40 tx=0x7f7ca000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:18.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.617 7f7ca57fa700 1 -- 172.21.2.101:0/3061888926 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ca000bd80 con 0x7f7ca80b5a90 2021-02-10T11:34:18.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.617 7f7ca57fa700 1 -- 172.21.2.101:0/3061888926 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca0000c90 con 0x7f7ca80b5a90 2021-02-10T11:34:18.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.617 7f7ca57fa700 1 -- 172.21.2.101:0/3061888926 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ca00022b0 con 0x7f7ca80b5a90 2021-02-10T11:34:18.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.617 7f7caefa7700 1 -- 172.21.2.101:0/3061888926 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca80eca90 con 0x7f7ca80b5a90 2021-02-10T11:34:18.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.617 7f7caefa7700 1 -- 172.21.2.101:0/3061888926 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca80ed020 con 0x7f7ca80b5a90 2021-02-10T11:34:18.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.617 7f7ca57fa700 1 -- 172.21.2.101:0/3061888926 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7ca000a000 con 0x7f7ca80b5a90 2021-02-10T11:34:18.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.617 7f7ca57fa700 1 --2- 172.21.2.101:0/3061888926 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c8c0186c0 0x7f7c8c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:18.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.617 7f7ca57fa700 1 -- 172.21.2.101:0/3061888926 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7ca0032550 con 0x7f7ca80b5a90 2021-02-10T11:34:18.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.617 7f7ca7fff700 1 --2- 172.21.2.101:0/3061888926 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c8c0186c0 0x7f7c8c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:18.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.618 7f7ca7fff700 1 --2- 172.21.2.101:0/3061888926 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c8c0186c0 0x7f7c8c01abc0 crc :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:18.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.648 7f7caefa7700 1 -- 172.21.2.101:0/3061888926 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca80ebf30 con 0x7f7ca80b5a90 2021-02-10T11:34:18.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.650 7f7ca57fa700 1 -- 172.21.2.101:0/3061888926 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7ca00009b0 con 0x7f7ca80b5a90 2021-02-10T11:34:18.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.801 7f7caefa7700 1 -- 172.21.2.101:0/3061888926 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7ca80ebf30 con 0x7f7ca80b5a90 2021-02-10T11:34:18.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.806 7f7ca57fa700 1 -- 172.21.2.101:0/3061888926 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7ca000a000 con 0x7f7ca80b5a90 2021-02-10T11:34:18.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.810 7f7caefa7700 1 -- 172.21.2.101:0/3061888926 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c8c0186c0 msgr2=0x7f7c8c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:18.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.810 7f7caefa7700 1 --2- 172.21.2.101:0/3061888926 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c8c0186c0 0x7f7c8c01abc0 crc :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:18.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.810 7f7caefa7700 1 -- 172.21.2.101:0/3061888926 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ca80b5a90 msgr2=0x7f7ca81a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:18.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.810 7f7caefa7700 1 --2- 172.21.2.101:0/3061888926 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ca80b5a90 0x7f7ca81a4420 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7f7ca0006f40 tx=0x7f7ca000b990).stop 2021-02-10T11:34:18.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.810 7f7caefa7700 1 -- 172.21.2.101:0/3061888926 shutdown_connections 2021-02-10T11:34:18.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.810 7f7caefa7700 1 --2- 172.21.2.101:0/3061888926 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c8c0186c0 0x7f7c8c01abc0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:18.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.810 7f7caefa7700 1 --2- 172.21.2.101:0/3061888926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ca81a4a40 0x7f7ca80c1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:18.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.810 7f7caefa7700 1 --2- 172.21.2.101:0/3061888926 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ca80c2410 0x7f7ca80ec470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:18.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.810 7f7caefa7700 1 --2- 172.21.2.101:0/3061888926 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ca80b5a90 0x7f7ca81a4420 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:18.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.810 7f7caefa7700 1 -- 172.21.2.101:0/3061888926 >> 172.21.2.101:0/3061888926 conn(0x7f7ca809e5c0 msgr2=0x7f7ca80c2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:18.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.811 7f7caefa7700 1 -- 172.21.2.101:0/3061888926 shutdown_connections 2021-02-10T11:34:18.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:18.811 7f7caefa7700 1 -- 172.21.2.101:0/3061888926 wait complete. 2021-02-10T11:34:18.830 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm client.alice debug_client 2021-02-10T11:34:19.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.233 7fcdef7d9700 1 Processor -- start 2021-02-10T11:34:19.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.233 7fcdef7d9700 1 -- start start 2021-02-10T11:34:19.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.234 7fcdef7d9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcde80d0330 0x7fcde80ef630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:19.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.234 7fcdef7d9700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcde80d1c60 0x7fcde80efc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:19.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.234 7fcdef7d9700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde80f0b00 con 0x7fcde80d1860 2021-02-10T11:34:19.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.234 7fcdef7d9700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde80e5ec0 con 0x7fcde80d1c60 2021-02-10T11:34:19.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.234 7fcdef7d9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde81b0bd0 con 0x7fcde80d0330 2021-02-10T11:34:19.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.234 7fcded5f5700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fcde80d1860 0x7fcde81b03e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52734/0 (socket says 172.21.2.101:52734) 2021-02-10T11:34:19.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.234 7fcdeddf6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcde80d1c60 0x7fcde80efc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:19.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.234 7fcded5f5700 1 -- 172.21.2.101:0/2759904189 learned_addr learned my addr 172.21.2.101:0/2759904189 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:19.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.234 7fcdeddf6700 1 -- 172.21.2.101:0/2759904189 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcde80d0330 msgr2=0x7fcde80ef630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:34:19.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.234 7fcdeddf6700 1 --2- 172.21.2.101:0/2759904189 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcde80d0330 0x7fcde80ef630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:19.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.234 7fcdeddf6700 1 -- 172.21.2.101:0/2759904189 >> v1:172.21.2.101:6789/0 conn(0x7fcde80d1860 legacy=0x7fcde81b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:19.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.234 7fcdeddf6700 1 -- 172.21.2.101:0/2759904189 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde80f2090 con 0x7fcde80d1c60 2021-02-10T11:34:19.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.235 7fcdeddf6700 1 --2- 172.21.2.101:0/2759904189 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcde80d1c60 0x7fcde80efc50 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 rx=0x7fcde0006e30 tx=0x7fcde0009e60).ready entity=mon.0 client_cookie=3b69a6ef3cf78ff9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:19.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.235 7fcddbfff700 1 -- 172.21.2.101:0/2759904189 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcde0006a50 con 0x7fcde80d1c60 2021-02-10T11:34:19.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.235 7fcddbfff700 1 -- 172.21.2.101:0/2759904189 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde000b300 con 0x7fcde80d1c60 2021-02-10T11:34:19.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.235 7fcddbfff700 1 -- 172.21.2.101:0/2759904189 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcde000b820 con 0x7fcde80d1c60 2021-02-10T11:34:19.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.235 7fcdef7d9700 1 -- 172.21.2.101:0/2759904189 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcde80d1c60 msgr2=0x7fcde80efc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:19.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.235 7fcdef7d9700 1 --2- 172.21.2.101:0/2759904189 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcde80d1c60 0x7fcde80efc50 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 rx=0x7fcde0006e30 tx=0x7fcde0009e60).stop 2021-02-10T11:34:19.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.235 7fcdef7d9700 1 -- 172.21.2.101:0/2759904189 shutdown_connections 2021-02-10T11:34:19.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.235 7fcdef7d9700 1 --2- 172.21.2.101:0/2759904189 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcde80d0330 0x7fcde80ef630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:19.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.235 7fcdef7d9700 1 --2- 172.21.2.101:0/2759904189 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcde80d1c60 0x7fcde80efc50 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:19.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.235 7fcdef7d9700 1 -- 172.21.2.101:0/2759904189 >> 172.21.2.101:0/2759904189 conn(0x7fcde80ce440 msgr2=0x7fcde80d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:19.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.236 7fcdef7d9700 1 -- 172.21.2.101:0/2759904189 shutdown_connections 2021-02-10T11:34:19.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.236 7fcdef7d9700 1 -- 172.21.2.101:0/2759904189 wait complete. 2021-02-10T11:34:19.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.250 7fcdef7d9700 1 Processor -- start 2021-02-10T11:34:19.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.250 7fcdef7d9700 1 -- start start 2021-02-10T11:34:19.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.251 7fcdef7d9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcde81a4440 0x7fcde81a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:19.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.251 7fcdef7d9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcde8093210 0x7fcde8093610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:19.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.251 7fcdef7d9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcde818a880 0x7fcde818ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:19.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.251 7fcdef7d9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde8094430 con 0x7fcde81a4440 2021-02-10T11:34:19.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.251 7fcdef7d9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde8093f70 con 0x7fcde818a880 2021-02-10T11:34:19.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.251 7fcdef7d9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde8189ef0 con 0x7fcde8093210 2021-02-10T11:34:19.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.251 7fcded5f5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcde81a4440 0x7fcde81a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:19.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.251 7fcded5f5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcde81a4440 0x7fcde81a4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52244/0 (socket says 172.21.2.101:52244) 2021-02-10T11:34:19.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.251 7fcded5f5700 1 -- 172.21.2.101:0/1337210263 learned_addr learned my addr 172.21.2.101:0/1337210263 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:19.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.252 7fcded5f5700 1 -- 172.21.2.101:0/1337210263 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcde8093210 msgr2=0x7fcde8093610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:19.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.252 7fcded5f5700 1 --2- 172.21.2.101:0/1337210263 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcde8093210 0x7fcde8093610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:19.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.252 7fcded5f5700 1 -- 172.21.2.101:0/1337210263 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcde818a880 msgr2=0x7fcde818ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:19.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.252 7fcded5f5700 1 --2- 172.21.2.101:0/1337210263 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcde818a880 0x7fcde818ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:19.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.252 7fcded5f5700 1 -- 172.21.2.101:0/1337210263 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde818b2a0 con 0x7fcde81a4440 2021-02-10T11:34:19.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.252 7fcded5f5700 1 --2- 172.21.2.101:0/1337210263 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcde81a4440 0x7fcde81a4840 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7fcde4001c60 tx=0x7fcde400a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:19.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.252 7fcdd9ffb700 1 -- 172.21.2.101:0/1337210263 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcde400acc0 con 0x7fcde81a4440 2021-02-10T11:34:19.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.252 7fcdd9ffb700 1 -- 172.21.2.101:0/1337210263 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde4004140 con 0x7fcde81a4440 2021-02-10T11:34:19.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.252 7fcdd9ffb700 1 -- 172.21.2.101:0/1337210263 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcde4004660 con 0x7fcde81a4440 2021-02-10T11:34:19.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.252 7fcdef7d9700 1 -- 172.21.2.101:0/1337210263 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde818b2a0 con 0x7fcde81a4440 2021-02-10T11:34:19.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.252 7fcdef7d9700 1 -- 172.21.2.101:0/1337210263 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde818b830 con 0x7fcde81a4440 2021-02-10T11:34:19.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.253 7fcdd9ffb700 1 -- 172.21.2.101:0/1337210263 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fcde4004320 con 0x7fcde81a4440 2021-02-10T11:34:19.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.253 7fcdd9ffb700 1 --2- 172.21.2.101:0/1337210263 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcdc80186c0 0x7fcdc801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:19.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.253 7fcdd9ffb700 1 -- 172.21.2.101:0/1337210263 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fcde4032420 con 0x7fcde81a4440 2021-02-10T11:34:19.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.253 7fcdecdf4700 1 --2- 172.21.2.101:0/1337210263 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcdc80186c0 0x7fcdc801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:19.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.253 7fcdecdf4700 1 --2- 172.21.2.101:0/1337210263 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcdc80186c0 0x7fcdc801abc0 crc :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:19.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.283 7fcdef7d9700 1 -- 172.21.2.101:0/1337210263 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde8093cc0 con 0x7fcde81a4440 2021-02-10T11:34:19.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.286 7fcdd9ffb700 1 -- 172.21.2.101:0/1337210263 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fcde4000dd0 con 0x7fcde81a4440 2021-02-10T11:34:19.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.435 7fcdef7d9700 1 -- 172.21.2.101:0/1337210263 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "client.alice", "name": "debug_client"} v 0) v1 -- 0x7fcde8093cc0 con 0x7fcde81a4440 2021-02-10T11:34:19.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.439 7fcdd9ffb700 1 -- 172.21.2.101:0/1337210263 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "client.alice", "name": "debug_client"}]=0 v12) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcde4004200 con 0x7fcde81a4440 2021-02-10T11:34:19.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.444 7fcdef7d9700 1 -- 172.21.2.101:0/1337210263 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcdc80186c0 msgr2=0x7fcdc801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:19.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.444 7fcdef7d9700 1 --2- 172.21.2.101:0/1337210263 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcdc80186c0 0x7fcdc801abc0 crc :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:19.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.444 7fcdef7d9700 1 -- 172.21.2.101:0/1337210263 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcde81a4440 msgr2=0x7fcde81a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:19.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.444 7fcdef7d9700 1 --2- 172.21.2.101:0/1337210263 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcde81a4440 0x7fcde81a4840 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7fcde4001c60 tx=0x7fcde400a8d0).stop 2021-02-10T11:34:19.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.444 7fcdef7d9700 1 -- 172.21.2.101:0/1337210263 shutdown_connections 2021-02-10T11:34:19.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.444 7fcdef7d9700 1 --2- 172.21.2.101:0/1337210263 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcdc80186c0 0x7fcdc801abc0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:19.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.444 7fcdef7d9700 1 --2- 172.21.2.101:0/1337210263 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcde8093210 0x7fcde8093610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:19.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.444 7fcdef7d9700 1 --2- 172.21.2.101:0/1337210263 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcde818a880 0x7fcde818ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:19.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.444 7fcdef7d9700 1 --2- 172.21.2.101:0/1337210263 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcde81a4440 0x7fcde81a4840 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:19.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.444 7fcdef7d9700 1 -- 172.21.2.101:0/1337210263 >> 172.21.2.101:0/1337210263 conn(0x7fcde8094850 msgr2=0x7fcde80b81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:19.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.444 7fcdef7d9700 1 -- 172.21.2.101:0/1337210263 shutdown_connections 2021-02-10T11:34:19.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.444 7fcdef7d9700 1 -- 172.21.2.101:0/1337210263 wait complete. 2021-02-10T11:34:19.463 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm client.alice debug_objecter 2021-02-10T11:34:19.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.868 7fba91fd9700 1 Processor -- start 2021-02-10T11:34:19.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.868 7fba91fd9700 1 -- start start 2021-02-10T11:34:19.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.868 7fba91fd9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba8c0d1870 0x7fba8c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:19.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.868 7fba91fd9700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fba8c1b0a10 0x7fba8c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:19.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.868 7fba91fd9700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba8c0f0b10 con 0x7fba8c0d1c70 2021-02-10T11:34:19.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.868 7fba91fd9700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba8c0fe230 con 0x7fba8c1b0a10 2021-02-10T11:34:19.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.868 7fba91fd9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c0e5ed0 con 0x7fba8c0d1870 2021-02-10T11:34:19.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.870 7fba8bfff700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fba8c0d1c70 0x7fba8c0cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:46156/0 (socket says 172.21.2.101:46156) 2021-02-10T11:34:19.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.870 7fba8bfff700 1 -- 172.21.2.101:0/2910747283 learned_addr learned my addr 172.21.2.101:0/2910747283 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:19.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.870 7fba8affd700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fba8c1b0a10 0x7fba8c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:19.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.870 7fba8b7fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba8c0d1870 0x7fba8c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:19.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.871 7fba8affd700 1 -- 172.21.2.101:0/2910747283 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba8c0d1870 msgr2=0x7fba8c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:19.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.871 7fba8affd700 1 --2- 172.21.2.101:0/2910747283 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba8c0d1870 0x7fba8c1b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:19.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.871 7fba8affd700 1 -- 172.21.2.101:0/2910747283 >> v1:172.21.2.108:6789/0 conn(0x7fba8c0d1c70 legacy=0x7fba8c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:19.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.871 7fba8affd700 1 -- 172.21.2.101:0/2910747283 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8c0f0650 con 0x7fba8c1b0a10 2021-02-10T11:34:19.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.871 7fba8b7fe700 1 --2- 172.21.2.101:0/2910747283 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba8c0d1870 0x7fba8c1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:34:19.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.871 7fba8affd700 1 --2- 172.21.2.101:0/2910747283 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba8c1b0a10 0x7fba8c0cf330 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7fba80006650 tx=0x7fba80009a80).ready entity=mon.1 client_cookie=2f6467fc143cb6c1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:19.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.871 7fba8a7fc700 1 -- 172.21.2.101:0/2910747283 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fba8000c0c0 con 0x7fba8c1b0a10 2021-02-10T11:34:19.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.871 7fba8a7fc700 1 -- 172.21.2.101:0/2910747283 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8000c340 con 0x7fba8c1b0a10 2021-02-10T11:34:19.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.871 7fba8a7fc700 1 -- 172.21.2.101:0/2910747283 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fba8000c860 con 0x7fba8c1b0a10 2021-02-10T11:34:19.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.871 7fba91fd9700 1 -- 172.21.2.101:0/2910747283 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba8c1b0a10 msgr2=0x7fba8c0cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:19.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.871 7fba91fd9700 1 --2- 172.21.2.101:0/2910747283 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba8c1b0a10 0x7fba8c0cf330 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7fba80006650 tx=0x7fba80009a80).stop 2021-02-10T11:34:19.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.872 7fba91fd9700 1 -- 172.21.2.101:0/2910747283 shutdown_connections 2021-02-10T11:34:19.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.872 7fba91fd9700 1 --2- 172.21.2.101:0/2910747283 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba8c1b0a10 0x7fba8c0cf330 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:19.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.872 7fba91fd9700 1 --2- 172.21.2.101:0/2910747283 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba8c0d1870 0x7fba8c1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:19.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.872 7fba91fd9700 1 -- 172.21.2.101:0/2910747283 >> 172.21.2.101:0/2910747283 conn(0x7fba8c0ce450 msgr2=0x7fba8c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:19.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.872 7fba91fd9700 1 -- 172.21.2.101:0/2910747283 shutdown_connections 2021-02-10T11:34:19.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.872 7fba91fd9700 1 -- 172.21.2.101:0/2910747283 wait complete. 2021-02-10T11:34:19.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.887 7fba91fd9700 1 Processor -- start 2021-02-10T11:34:19.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.887 7fba91fd9700 1 -- start start 2021-02-10T11:34:19.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.887 7fba91fd9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba8c1a06c0 0x7fba8c0ebe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:19.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.887 7fba91fd9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba8c0ec490 0x7fba8c0c30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:19.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.887 7fba91fd9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba8c0c3710 0x7fba8c0bc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:19.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.887 7fba91fd9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c0c3b10 con 0x7fba8c1a06c0 2021-02-10T11:34:19.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.887 7fba91fd9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c0eca90 con 0x7fba8c0ec490 2021-02-10T11:34:19.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.887 7fba91fd9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c0bc690 con 0x7fba8c0c3710 2021-02-10T11:34:19.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.888 7fba8b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba8c1a06c0 0x7fba8c0ebe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:19.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.888 7fba8b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba8c1a06c0 0x7fba8c0ebe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52254/0 (socket says 172.21.2.101:52254) 2021-02-10T11:34:19.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.888 7fba8b7fe700 1 -- 172.21.2.101:0/1088236505 learned_addr learned my addr 172.21.2.101:0/1088236505 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:19.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.888 7fba8bfff700 1 --2- 172.21.2.101:0/1088236505 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba8c0c3710 0x7fba8c0bc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:19.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.888 7fba8affd700 1 --2- 172.21.2.101:0/1088236505 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba8c0ec490 0x7fba8c0c30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:19.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.888 7fba8b7fe700 1 -- 172.21.2.101:0/1088236505 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba8c0c3710 msgr2=0x7fba8c0bc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:19.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.888 7fba8b7fe700 1 --2- 172.21.2.101:0/1088236505 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba8c0c3710 0x7fba8c0bc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:19.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.888 7fba8b7fe700 1 -- 172.21.2.101:0/1088236505 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba8c0ec490 msgr2=0x7fba8c0c30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:19.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.888 7fba8b7fe700 1 --2- 172.21.2.101:0/1088236505 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba8c0ec490 0x7fba8c0c30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:19.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.888 7fba8b7fe700 1 -- 172.21.2.101:0/1088236505 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8c0bc870 con 0x7fba8c1a06c0 2021-02-10T11:34:19.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.888 7fba8b7fe700 1 --2- 172.21.2.101:0/1088236505 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba8c1a06c0 0x7fba8c0ebe70 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7fba7c006f40 tx=0x7fba7c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:19.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.888 7fba73fff700 1 -- 172.21.2.101:0/1088236505 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fba7c0009b0 con 0x7fba8c1a06c0 2021-02-10T11:34:19.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.889 7fba73fff700 1 -- 172.21.2.101:0/1088236505 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7c00bd80 con 0x7fba8c1a06c0 2021-02-10T11:34:19.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.889 7fba73fff700 1 -- 172.21.2.101:0/1088236505 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fba7c002220 con 0x7fba8c1a06c0 2021-02-10T11:34:19.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.889 7fba91fd9700 1 -- 172.21.2.101:0/1088236505 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba8c0bc870 con 0x7fba8c1a06c0 2021-02-10T11:34:19.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.889 7fba91fd9700 1 -- 172.21.2.101:0/1088236505 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba8c0bc870 con 0x7fba8c1a06c0 2021-02-10T11:34:19.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.889 7fba73fff700 1 -- 172.21.2.101:0/1088236505 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fba7c000c90 con 0x7fba8c1a06c0 2021-02-10T11:34:19.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.889 7fba73fff700 1 --2- 172.21.2.101:0/1088236505 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba6c0186c0 0x7fba6c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:19.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.889 7fba73fff700 1 -- 172.21.2.101:0/1088236505 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fba7c032420 con 0x7fba8c1a06c0 2021-02-10T11:34:19.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.889 7fba8affd700 1 --2- 172.21.2.101:0/1088236505 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba6c0186c0 0x7fba6c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:19.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.890 7fba8affd700 1 --2- 172.21.2.101:0/1088236505 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba6c0186c0 0x7fba6c01abc0 crc :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:19.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.918 7fba91fd9700 1 -- 172.21.2.101:0/1088236505 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba8c0ebc40 con 0x7fba8c1a06c0 2021-02-10T11:34:19.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:19.920 7fba73fff700 1 -- 172.21.2.101:0/1088236505 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fba7c025fd0 con 0x7fba8c1a06c0 2021-02-10T11:34:20.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.070 7fba91fd9700 1 -- 172.21.2.101:0/1088236505 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "client.alice", "name": "debug_objecter"} v 0) v1 -- 0x7fba8c0ebc40 con 0x7fba8c1a06c0 2021-02-10T11:34:20.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.074 7fba73fff700 1 -- 172.21.2.101:0/1088236505 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "client.alice", "name": "debug_objecter"}]=0 v13) v1 ==== 106+0+0 (secure 0 0 0) 0x7fba7c025fd0 con 0x7fba8c1a06c0 2021-02-10T11:34:20.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.078 7fba91fd9700 1 -- 172.21.2.101:0/1088236505 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba6c0186c0 msgr2=0x7fba6c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:20.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.078 7fba91fd9700 1 --2- 172.21.2.101:0/1088236505 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba6c0186c0 0x7fba6c01abc0 crc :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:20.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.078 7fba91fd9700 1 -- 172.21.2.101:0/1088236505 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba8c1a06c0 msgr2=0x7fba8c0ebe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:20.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.078 7fba91fd9700 1 --2- 172.21.2.101:0/1088236505 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba8c1a06c0 0x7fba8c0ebe70 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7fba7c006f40 tx=0x7fba7c00b990).stop 2021-02-10T11:34:20.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.079 7fba91fd9700 1 -- 172.21.2.101:0/1088236505 shutdown_connections 2021-02-10T11:34:20.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.079 7fba91fd9700 1 --2- 172.21.2.101:0/1088236505 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba6c0186c0 0x7fba6c01abc0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:20.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.079 7fba91fd9700 1 --2- 172.21.2.101:0/1088236505 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba8c0c3710 0x7fba8c0bc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:20.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.079 7fba91fd9700 1 --2- 172.21.2.101:0/1088236505 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba8c0ec490 0x7fba8c0c30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:20.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.079 7fba91fd9700 1 --2- 172.21.2.101:0/1088236505 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba8c1a06c0 0x7fba8c0ebe70 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:20.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.079 7fba91fd9700 1 -- 172.21.2.101:0/1088236505 >> 172.21.2.101:0/1088236505 conn(0x7fba8c0a6bb0 msgr2=0x7fba8c095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:20.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.079 7fba91fd9700 1 -- 172.21.2.101:0/1088236505 shutdown_connections 2021-02-10T11:34:20.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.079 7fba91fd9700 1 -- 172.21.2.101:0/1088236505 wait complete. 2021-02-10T11:34:20.098 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_authorize_deauthorize_legacy_subvolume' 2021-02-10T11:34:20.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.497 7f709e965700 1 Processor -- start 2021-02-10T11:34:20.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.497 7f709e965700 1 -- start start 2021-02-10T11:34:20.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.498 7f709e965700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f70980d0340 0x7f70980ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:20.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.498 7f709e965700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f70980d1c70 0x7f70980efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:20.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.498 7f709e965700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70980f0b10 con 0x7f70980d1870 2021-02-10T11:34:20.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.498 7f709e965700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70980e5ed0 con 0x7f70980d0340 2021-02-10T11:34:20.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.498 7f709e965700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70981b0be0 con 0x7f70980d1c70 2021-02-10T11:34:20.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.498 7f70977fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f70980d0340 0x7f70980ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:20.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.498 7f70977fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f70980d0340 0x7f70980ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52262/0 (socket says 172.21.2.101:52262) 2021-02-10T11:34:20.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.498 7f70977fe700 1 -- 172.21.2.101:0/18392839 learned_addr learned my addr 172.21.2.101:0/18392839 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:20.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.498 7f70977fe700 1 -- 172.21.2.101:0/18392839 >> v2:172.21.2.108:3300/0 conn(0x7f70980d1c70 msgr2=0x7f70980efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:20.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.498 7f70977fe700 1 --2- 172.21.2.101:0/18392839 >> v2:172.21.2.108:3300/0 conn(0x7f70980d1c70 0x7f70980efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:20.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.498 7f70977fe700 1 -- 172.21.2.101:0/18392839 >> v1:172.21.2.108:6789/0 conn(0x7f70980d1870 legacy=0x7f70981b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:20.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.498 7f70977fe700 1 -- 172.21.2.101:0/18392839 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70980f20a0 con 0x7f70980d0340 2021-02-10T11:34:20.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.498 7f70977fe700 1 --2- 172.21.2.101:0/18392839 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70980d0340 0x7f70980ef640 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7f708c006700 tx=0x7f708c009b80).ready entity=mon.0 client_cookie=3c06ebaae499614d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:20.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.498 7f7096ffd700 1 -- 172.21.2.101:0/18392839 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f708c00c0c0 con 0x7f70980d0340 2021-02-10T11:34:20.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.498 7f7096ffd700 1 -- 172.21.2.101:0/18392839 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708c00c440 con 0x7f70980d0340 2021-02-10T11:34:20.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.498 7f7096ffd700 1 -- 172.21.2.101:0/18392839 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f708c00c8a0 con 0x7f70980d0340 2021-02-10T11:34:20.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.498 7f709e965700 1 -- 172.21.2.101:0/18392839 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70980d0340 msgr2=0x7f70980ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:20.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.499 7f709e965700 1 --2- 172.21.2.101:0/18392839 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70980d0340 0x7f70980ef640 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7f708c006700 tx=0x7f708c009b80).stop 2021-02-10T11:34:20.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.499 7f709e965700 1 -- 172.21.2.101:0/18392839 shutdown_connections 2021-02-10T11:34:20.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.499 7f709e965700 1 --2- 172.21.2.101:0/18392839 >> v2:172.21.2.108:3300/0 conn(0x7f70980d1c70 0x7f70980efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:20.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.499 7f709e965700 1 --2- 172.21.2.101:0/18392839 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70980d0340 0x7f70980ef640 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:20.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.499 7f709e965700 1 -- 172.21.2.101:0/18392839 >> 172.21.2.101:0/18392839 conn(0x7f70980ce450 msgr2=0x7f70980d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:20.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.499 7f709e965700 1 -- 172.21.2.101:0/18392839 shutdown_connections 2021-02-10T11:34:20.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.499 7f709e965700 1 -- 172.21.2.101:0/18392839 wait complete. 2021-02-10T11:34:20.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.514 7f709e965700 1 Processor -- start 2021-02-10T11:34:20.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.514 7f709e965700 1 -- start start 2021-02-10T11:34:20.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.514 7f709e965700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70980b4e00 0x7f70980a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:20.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.514 7f709e965700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70980a3a60 0x7f70980a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:20.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.514 7f709e965700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7098094750 0x7f7098094b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:20.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.514 7f709e965700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70980a4550 con 0x7f70980b4e00 2021-02-10T11:34:20.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.514 7f709e965700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709818bf30 con 0x7f7098094750 2021-02-10T11:34:20.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.514 7f709e965700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709818ba60 con 0x7f70980a3a60 2021-02-10T11:34:20.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.514 7f7097fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70980b4e00 0x7f70980a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:20.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.514 7f7097fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70980b4e00 0x7f70980a0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52268/0 (socket says 172.21.2.101:52268) 2021-02-10T11:34:20.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.514 7f7097fff700 1 -- 172.21.2.101:0/3183908981 learned_addr learned my addr 172.21.2.101:0/3183908981 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:20.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.515 7f70977fe700 1 --2- 172.21.2.101:0/3183908981 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70980a3a60 0x7f70980a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:20.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.515 7f709cf82700 1 --2- 172.21.2.101:0/3183908981 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7098094750 0x7f7098094b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:20.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.515 7f7097fff700 1 -- 172.21.2.101:0/3183908981 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70980a3a60 msgr2=0x7f70980a3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:20.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.515 7f7097fff700 1 --2- 172.21.2.101:0/3183908981 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70980a3a60 0x7f70980a3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:20.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.515 7f7097fff700 1 -- 172.21.2.101:0/3183908981 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7098094750 msgr2=0x7f7098094b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:20.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.515 7f7097fff700 1 --2- 172.21.2.101:0/3183908981 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7098094750 0x7f7098094b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:20.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.515 7f7097fff700 1 -- 172.21.2.101:0/3183908981 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70980a15c0 con 0x7f70980b4e00 2021-02-10T11:34:20.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.515 7f7097fff700 1 --2- 172.21.2.101:0/3183908981 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70980b4e00 0x7f70980a0fa0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7f708800b5b0 tx=0x7f708800b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:20.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.515 7f7094ff9700 1 -- 172.21.2.101:0/3183908981 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f708800bd40 con 0x7f70980b4e00 2021-02-10T11:34:20.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.515 7f7094ff9700 1 -- 172.21.2.101:0/3183908981 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7088000e20 con 0x7f70980b4e00 2021-02-10T11:34:20.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.515 7f709e965700 1 -- 172.21.2.101:0/3183908981 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70980a15c0 con 0x7f70980b4e00 2021-02-10T11:34:20.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.515 7f7094ff9700 1 -- 172.21.2.101:0/3183908981 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f70880054a0 con 0x7f70980b4e00 2021-02-10T11:34:20.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.515 7f709e965700 1 -- 172.21.2.101:0/3183908981 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70980a15c0 con 0x7f70980b4e00 2021-02-10T11:34:20.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.516 7f7094ff9700 1 -- 172.21.2.101:0/3183908981 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7088006030 con 0x7f70980b4e00 2021-02-10T11:34:20.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.516 7f7094ff9700 1 --2- 172.21.2.101:0/3183908981 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70780186d0 0x7f707801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:20.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.516 7f7094ff9700 1 -- 172.21.2.101:0/3183908981 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7088032650 con 0x7f70980b4e00 2021-02-10T11:34:20.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.516 7f70977fe700 1 --2- 172.21.2.101:0/3183908981 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70780186d0 0x7f707801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:20.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.516 7f70977fe700 1 --2- 172.21.2.101:0/3183908981 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70780186d0 0x7f707801abd0 crc :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:20.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.547 7f709e965700 1 -- 172.21.2.101:0/3183908981 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70980ee0e0 con 0x7f70980b4e00 2021-02-10T11:34:20.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.550 7f7094ff9700 1 -- 172.21.2.101:0/3183908981 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7088000ce0 con 0x7f70980b4e00 2021-02-10T11:34:20.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:20.681 7f709e965700 1 -- 172.21.2.101:0/3183908981 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_authorize_deauthorize_legacy_subvolume"]} v 0) v1 -- 0x7f70980ee0e0 con 0x7f70980b4e00 2021-02-10T11:34:21.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.473 7f7094ff9700 1 -- 172.21.2.101:0/3183908981 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_authorize_deauthorize_legacy_subvolume"]}]=0 v331) v1 ==== 160+0+0 (secure 0 0 0) 0x7f7088000ef0 con 0x7f70980b4e00 2021-02-10T11:34:21.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.502 7f709e965700 1 -- 172.21.2.101:0/3183908981 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70780186d0 msgr2=0x7f707801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:21.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.502 7f709e965700 1 --2- 172.21.2.101:0/3183908981 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70780186d0 0x7f707801abd0 crc :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:21.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.502 7f709e965700 1 -- 172.21.2.101:0/3183908981 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70980b4e00 msgr2=0x7f70980a0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:21.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.502 7f709e965700 1 --2- 172.21.2.101:0/3183908981 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70980b4e00 0x7f70980a0fa0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7f708800b5b0 tx=0x7f708800b950).stop 2021-02-10T11:34:21.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.503 7f709e965700 1 -- 172.21.2.101:0/3183908981 shutdown_connections 2021-02-10T11:34:21.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.503 7f709e965700 1 --2- 172.21.2.101:0/3183908981 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70780186d0 0x7f707801abd0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:21.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.503 7f709e965700 1 --2- 172.21.2.101:0/3183908981 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70980a3a60 0x7f70980a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:21.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.503 7f709e965700 1 --2- 172.21.2.101:0/3183908981 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7098094750 0x7f7098094b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:21.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.503 7f709e965700 1 --2- 172.21.2.101:0/3183908981 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70980b4e00 0x7f70980a0fa0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:21.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.503 7f709e965700 1 -- 172.21.2.101:0/3183908981 >> 172.21.2.101:0/3183908981 conn(0x7f709818cab0 msgr2=0x7f7098092c60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:21.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.503 7f709e965700 1 -- 172.21.2.101:0/3183908981 shutdown_connections 2021-02-10T11:34:21.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.503 7f709e965700 1 -- 172.21.2.101:0/3183908981 wait complete. 2021-02-10T11:34:21.521 INFO:tasks.cephfs_test_runner:test_authorize_deauthorize_legacy_subvolume (tasks.cephfs.test_volumes.TestVolumes) ... ok 2021-02-10T11:34:21.521 INFO:tasks.cephfs_test_runner:Starting test: test_authorize_deauthorize_subvolume (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:34:21.522 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_authorize_deauthorize_subvolume' 2021-02-10T11:34:21.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.921 7f94ae770700 1 Processor -- start 2021-02-10T11:34:21.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.921 7f94ae770700 1 -- start start 2021-02-10T11:34:21.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.921 7f94ae770700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f94a80d1870 0x7f94a81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:21.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.921 7f94ae770700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94a81b0a10 0x7f94a80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:21.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.921 7f94ae770700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a80f0b10 con 0x7f94a80d1c70 2021-02-10T11:34:21.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.921 7f94ae770700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a80e5ed0 con 0x7f94a80d1870 2021-02-10T11:34:21.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.921 7f94ae770700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a80f0650 con 0x7f94a81b0a10 2021-02-10T11:34:21.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.922 7f94acd8d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f94a80d1c70 0x7f94a80cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52770/0 (socket says 172.21.2.101:52770) 2021-02-10T11:34:21.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.922 7f94acd8d700 1 -- 172.21.2.101:0/3672517231 learned_addr learned my addr 172.21.2.101:0/3672517231 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:21.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.922 7f94a7fff700 1 --2- 172.21.2.101:0/3672517231 >> v2:172.21.2.108:3300/0 conn(0x7f94a80d1870 0x7f94a81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:21.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.922 7f94a77fe700 1 --2- 172.21.2.101:0/3672517231 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94a81b0a10 0x7f94a80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:21.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.922 7f94a6ffd700 1 -- 172.21.2.101:0/3672517231 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4050354036 0 0) 0x7f94a0001500 con 0x7f94a80d1c70 2021-02-10T11:34:21.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.922 7f94a6ffd700 1 -- 172.21.2.101:0/3672517231 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9490002fe0 con 0x7f94a80d1c70 2021-02-10T11:34:21.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.922 7f94a7fff700 1 -- 172.21.2.101:0/3672517231 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94a81b0a10 msgr2=0x7f94a80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:21.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.922 7f94a7fff700 1 --2- 172.21.2.101:0/3672517231 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94a81b0a10 0x7f94a80cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:21.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.922 7f94a7fff700 1 -- 172.21.2.101:0/3672517231 >> v1:172.21.2.101:6789/0 conn(0x7f94a80d1c70 legacy=0x7f94a80cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:21.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.922 7f94a7fff700 1 -- 172.21.2.101:0/3672517231 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a80f0870 con 0x7f94a80d1870 2021-02-10T11:34:21.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.922 7f94a7fff700 1 --2- 172.21.2.101:0/3672517231 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94a80d1870 0x7f94a81b03f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f9498006650 tx=0x7f9498009a00).ready entity=mon.1 client_cookie=20b2eac381da8bda server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:21.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.923 7f94a6ffd700 1 -- 172.21.2.101:0/3672517231 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f94980009b0 con 0x7f94a80d1870 2021-02-10T11:34:21.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.923 7f94a6ffd700 1 -- 172.21.2.101:0/3672517231 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949800c0c0 con 0x7f94a80d1870 2021-02-10T11:34:21.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.923 7f94a6ffd700 1 -- 172.21.2.101:0/3672517231 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f949800c5e0 con 0x7f94a80d1870 2021-02-10T11:34:21.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.923 7f94ae770700 1 -- 172.21.2.101:0/3672517231 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94a80d1870 msgr2=0x7f94a81b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:21.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.923 7f94ae770700 1 --2- 172.21.2.101:0/3672517231 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94a80d1870 0x7f94a81b03f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f9498006650 tx=0x7f9498009a00).stop 2021-02-10T11:34:21.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.923 7f94ae770700 1 -- 172.21.2.101:0/3672517231 shutdown_connections 2021-02-10T11:34:21.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.923 7f94ae770700 1 --2- 172.21.2.101:0/3672517231 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94a81b0a10 0x7f94a80cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:21.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.923 7f94ae770700 1 --2- 172.21.2.101:0/3672517231 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94a80d1870 0x7f94a81b03f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:21.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.923 7f94ae770700 1 -- 172.21.2.101:0/3672517231 >> 172.21.2.101:0/3672517231 conn(0x7f94a80ce450 msgr2=0x7f94a80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:21.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.923 7f94ae770700 1 -- 172.21.2.101:0/3672517231 shutdown_connections 2021-02-10T11:34:21.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.923 7f94ae770700 1 -- 172.21.2.101:0/3672517231 wait complete. 2021-02-10T11:34:21.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.938 7f94ae770700 1 Processor -- start 2021-02-10T11:34:21.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.938 7f94ae770700 1 -- start start 2021-02-10T11:34:21.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.938 7f94ae770700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94a81942b0 0x7f94a80b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:21.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.938 7f94ae770700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94a8195070 0x7f94a8195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:21.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.938 7f94ae770700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94a80bb8e0 0x7f94a80bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:21.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.938 7f94ae770700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a81946b0 con 0x7f94a8195070 2021-02-10T11:34:21.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.938 7f94ae770700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a80b3b70 con 0x7f94a80bb8e0 2021-02-10T11:34:21.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.938 7f94ae770700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a80b3d90 con 0x7f94a81942b0 2021-02-10T11:34:21.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.939 7f94a77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94a8195070 0x7f94a8195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:21.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.939 7f94a77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94a8195070 0x7f94a8195470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52282/0 (socket says 172.21.2.101:52282) 2021-02-10T11:34:21.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.939 7f94a77fe700 1 -- 172.21.2.101:0/611834694 learned_addr learned my addr 172.21.2.101:0/611834694 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:21.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.939 7f94a77fe700 1 -- 172.21.2.101:0/611834694 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94a81942b0 msgr2=0x7f94a80b3550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:34:21.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.939 7f94a77fe700 1 --2- 172.21.2.101:0/611834694 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94a81942b0 0x7f94a80b3550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:21.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.939 7f94a77fe700 1 -- 172.21.2.101:0/611834694 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94a80bb8e0 msgr2=0x7f94a80bbce0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:34:21.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.939 7f94a77fe700 1 --2- 172.21.2.101:0/611834694 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94a80bb8e0 0x7f94a80bbce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:21.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.939 7f94a77fe700 1 -- 172.21.2.101:0/611834694 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a81a49e0 con 0x7f94a8195070 2021-02-10T11:34:21.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.939 7f94a77fe700 1 --2- 172.21.2.101:0/611834694 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94a8195070 0x7f94a8195470 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0x7f949c006f60 tx=0x7f949c00b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:21.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.939 7f94a4ff9700 1 -- 172.21.2.101:0/611834694 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f949c0009b0 con 0x7f94a8195070 2021-02-10T11:34:21.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.939 7f94a4ff9700 1 -- 172.21.2.101:0/611834694 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949c00bdf0 con 0x7f94a8195070 2021-02-10T11:34:21.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.939 7f94a4ff9700 1 -- 172.21.2.101:0/611834694 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f949c002030 con 0x7f94a8195070 2021-02-10T11:34:21.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.939 7f94ae770700 1 -- 172.21.2.101:0/611834694 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a81a49e0 con 0x7f94a8195070 2021-02-10T11:34:21.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.939 7f94ae770700 1 -- 172.21.2.101:0/611834694 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a81a49e0 con 0x7f94a8195070 2021-02-10T11:34:21.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.940 7f94a4ff9700 1 -- 172.21.2.101:0/611834694 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f949c006b00 con 0x7f94a8195070 2021-02-10T11:34:21.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.940 7f94a4ff9700 1 --2- 172.21.2.101:0/611834694 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94880186c0 0x7f948801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:21.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.940 7f94a4ff9700 1 -- 172.21.2.101:0/611834694 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f949c032440 con 0x7f94a8195070 2021-02-10T11:34:21.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.940 7f94a7fff700 1 --2- 172.21.2.101:0/611834694 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94880186c0 0x7f948801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:21.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.940 7f94a7fff700 1 --2- 172.21.2.101:0/611834694 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94880186c0 0x7f948801abc0 crc :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:21.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.971 7f94ae770700 1 -- 172.21.2.101:0/611834694 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a80bc5f0 con 0x7f94a8195070 2021-02-10T11:34:21.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:21.973 7f94a4ff9700 1 -- 172.21.2.101:0/611834694 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f949c006340 con 0x7f94a8195070 2021-02-10T11:34:22.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.105 7f94ae770700 1 -- 172.21.2.101:0/611834694 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_authorize_deauthorize_subvolume"]} v 0) v1 -- 0x7f94a80bc5f0 con 0x7f94a8195070 2021-02-10T11:34:22.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.478 7f94a4ff9700 1 -- 172.21.2.101:0/611834694 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_authorize_deauthorize_subvolume"]}]=0 v332) v1 ==== 156+0+0 (secure 0 0 0) 0x7f949c006b00 con 0x7f94a8195070 2021-02-10T11:34:22.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.524 7f94ae770700 1 -- 172.21.2.101:0/611834694 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94880186c0 msgr2=0x7f948801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:22.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.524 7f94ae770700 1 --2- 172.21.2.101:0/611834694 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94880186c0 0x7f948801abc0 crc :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:22.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.524 7f94ae770700 1 -- 172.21.2.101:0/611834694 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94a8195070 msgr2=0x7f94a8195470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:22.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.524 7f94ae770700 1 --2- 172.21.2.101:0/611834694 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94a8195070 0x7f94a8195470 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0x7f949c006f60 tx=0x7f949c00b970).stop 2021-02-10T11:34:22.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.524 7f94ae770700 1 -- 172.21.2.101:0/611834694 shutdown_connections 2021-02-10T11:34:22.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.524 7f94ae770700 1 --2- 172.21.2.101:0/611834694 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94a80bb8e0 0x7f94a80bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:22.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.524 7f94ae770700 1 --2- 172.21.2.101:0/611834694 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94a8195070 0x7f94a8195470 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:22.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.524 7f94ae770700 1 --2- 172.21.2.101:0/611834694 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94880186c0 0x7f948801abc0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:22.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.524 7f94ae770700 1 --2- 172.21.2.101:0/611834694 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94a81942b0 0x7f94a80b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:22.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.524 7f94ae770700 1 -- 172.21.2.101:0/611834694 >> 172.21.2.101:0/611834694 conn(0x7f94a80b3040 msgr2=0x7f94a80c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:22.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.524 7f94ae770700 1 -- 172.21.2.101:0/611834694 shutdown_connections 2021-02-10T11:34:22.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.524 7f94ae770700 1 -- 172.21.2.101:0/611834694 wait complete. 2021-02-10T11:34:22.543 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:34:22.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.949 7f28517dd700 1 Processor -- start 2021-02-10T11:34:22.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.949 7f28517dd700 1 -- start start 2021-02-10T11:34:22.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.949 7f28517dd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f284c0d1870 0x7f284c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:22.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.949 7f28517dd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f284c0cff70 0x7f284c109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:22.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.949 7f28517dd700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f284c109a60 con 0x7f284c1b0ae0 2021-02-10T11:34:22.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.949 7f28517dd700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f284c109750 con 0x7f284c0cff70 2021-02-10T11:34:22.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.949 7f28517dd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284c088070 con 0x7f284c0d1870 2021-02-10T11:34:22.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.950 7f284ad73700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f284c1b0ae0 0x7f284c108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52784/0 (socket says 172.21.2.101:52784) 2021-02-10T11:34:22.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.950 7f284ad73700 1 -- 172.21.2.101:0/1201440259 learned_addr learned my addr 172.21.2.101:0/1201440259 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:22.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.950 7f284bd75700 1 --2- 172.21.2.101:0/1201440259 >> v2:172.21.2.101:3300/0 conn(0x7f284c0cff70 0x7f284c109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:22.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.950 7f284bd75700 1 -- 172.21.2.101:0/1201440259 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f284c0d1870 msgr2=0x7f284c1b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:34:22.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.950 7f284bd75700 1 --2- 172.21.2.101:0/1201440259 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f284c0d1870 0x7f284c1b04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:22.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.950 7f284bd75700 1 -- 172.21.2.101:0/1201440259 >> v1:172.21.2.101:6789/0 conn(0x7f284c1b0ae0 legacy=0x7f284c108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:22.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.950 7f284bd75700 1 -- 172.21.2.101:0/1201440259 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284c088260 con 0x7f284c0cff70 2021-02-10T11:34:22.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.950 7f284bd75700 1 --2- 172.21.2.101:0/1201440259 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f284c0cff70 0x7f284c109130 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f2844006e30 tx=0x7f2844009e60).ready entity=mon.0 client_cookie=b7fd21feb7941aa1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:22.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.950 7f284a572700 1 -- 172.21.2.101:0/1201440259 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f284400b300 con 0x7f284c0cff70 2021-02-10T11:34:22.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.950 7f284a572700 1 -- 172.21.2.101:0/1201440259 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284400b580 con 0x7f284c0cff70 2021-02-10T11:34:22.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.950 7f284a572700 1 -- 172.21.2.101:0/1201440259 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f284400ba10 con 0x7f284c0cff70 2021-02-10T11:34:22.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.951 7f28517dd700 1 -- 172.21.2.101:0/1201440259 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f284c0cff70 msgr2=0x7f284c109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:22.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.951 7f28517dd700 1 --2- 172.21.2.101:0/1201440259 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f284c0cff70 0x7f284c109130 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f2844006e30 tx=0x7f2844009e60).stop 2021-02-10T11:34:22.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.951 7f28517dd700 1 -- 172.21.2.101:0/1201440259 shutdown_connections 2021-02-10T11:34:22.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.951 7f28517dd700 1 --2- 172.21.2.101:0/1201440259 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f284c0cff70 0x7f284c109130 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:22.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.951 7f28517dd700 1 --2- 172.21.2.101:0/1201440259 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f284c0d1870 0x7f284c1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:22.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.951 7f28517dd700 1 -- 172.21.2.101:0/1201440259 >> 172.21.2.101:0/1201440259 conn(0x7f284c0ce420 msgr2=0x7f284c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:22.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.951 7f28517dd700 1 -- 172.21.2.101:0/1201440259 shutdown_connections 2021-02-10T11:34:22.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.951 7f28517dd700 1 -- 172.21.2.101:0/1201440259 wait complete. 2021-02-10T11:34:22.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.966 7f28517dd700 1 Processor -- start 2021-02-10T11:34:22.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.966 7f28517dd700 1 -- start start 2021-02-10T11:34:22.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.966 7f28517dd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f284c0c3820 0x7f284c084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:22.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.966 7f28517dd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f284c0a0290 0x7f284c0852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:22.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.966 7f28517dd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f284c086270 0x7f284c086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:22.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.966 7f28517dd700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c0c0700 con 0x7f284c086270 2021-02-10T11:34:22.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.966 7f28517dd700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c0858f0 con 0x7f284c0a0290 2021-02-10T11:34:22.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.966 7f28517dd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284c085b10 con 0x7f284c0c3820 2021-02-10T11:34:22.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.966 7f284bd75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f284c086270 0x7f284c086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:22.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.966 7f284bd75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f284c086270 0x7f284c086670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52296/0 (socket says 172.21.2.101:52296) 2021-02-10T11:34:22.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.966 7f284bd75700 1 -- 172.21.2.101:0/3100377877 learned_addr learned my addr 172.21.2.101:0/3100377877 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:22.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.967 7f284bd75700 1 -- 172.21.2.101:0/3100377877 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f284c0c3820 msgr2=0x7f284c084cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:22.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.967 7f284bd75700 1 --2- 172.21.2.101:0/3100377877 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f284c0c3820 0x7f284c084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:22.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.967 7f284bd75700 1 -- 172.21.2.101:0/3100377877 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f284c0a0290 msgr2=0x7f284c0852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:22.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.967 7f284bd75700 1 --2- 172.21.2.101:0/3100377877 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f284c0a0290 0x7f284c0852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:22.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.967 7f284bd75700 1 -- 172.21.2.101:0/3100377877 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284c086c90 con 0x7f284c086270 2021-02-10T11:34:22.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.967 7f284bd75700 1 --2- 172.21.2.101:0/3100377877 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f284c086270 0x7f284c086670 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7f284400b910 tx=0x7f284400bcb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:22.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.967 7f2843fff700 1 -- 172.21.2.101:0/3100377877 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2844000fa0 con 0x7f284c086270 2021-02-10T11:34:22.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.967 7f2843fff700 1 -- 172.21.2.101:0/3100377877 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284400bdf0 con 0x7f284c086270 2021-02-10T11:34:22.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.967 7f28517dd700 1 -- 172.21.2.101:0/3100377877 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f284c086c90 con 0x7f284c086270 2021-02-10T11:34:22.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.967 7f28517dd700 1 -- 172.21.2.101:0/3100377877 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f284c1a2f00 con 0x7f284c086270 2021-02-10T11:34:22.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.967 7f2843fff700 1 -- 172.21.2.101:0/3100377877 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2844005a50 con 0x7f284c086270 2021-02-10T11:34:22.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.968 7f2843fff700 1 -- 172.21.2.101:0/3100377877 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2844005820 con 0x7f284c086270 2021-02-10T11:34:22.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.968 7f2843fff700 1 --2- 172.21.2.101:0/3100377877 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28300186c0 0x7f283001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:22.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.968 7f2843fff700 1 -- 172.21.2.101:0/3100377877 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2844032ba0 con 0x7f284c086270 2021-02-10T11:34:22.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.968 7f284b574700 1 --2- 172.21.2.101:0/3100377877 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28300186c0 0x7f283001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:22.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:22.968 7f284b574700 1 --2- 172.21.2.101:0/3100377877 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28300186c0 0x7f283001abc0 crc :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:23.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:23.000 7f28517dd700 1 -- 172.21.2.101:0/3100377877 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f284c082a70 con 0x7f284c086270 2021-02-10T11:34:23.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:23.002 7f2843fff700 1 -- 172.21.2.101:0/3100377877 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f28440093b0 con 0x7f284c086270 2021-02-10T11:34:23.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:23.151 7f28517dd700 1 -- 172.21.2.101:0/3100377877 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f284c082a70 con 0x7f284c086270 2021-02-10T11:34:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:23.155 7f2843fff700 1 -- 172.21.2.101:0/3100377877 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2844005820 con 0x7f284c086270 2021-02-10T11:34:23.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:23.160 7f28517dd700 1 -- 172.21.2.101:0/3100377877 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28300186c0 msgr2=0x7f283001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:23.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:23.160 7f28517dd700 1 --2- 172.21.2.101:0/3100377877 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28300186c0 0x7f283001abc0 crc :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:23.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:23.160 7f28517dd700 1 -- 172.21.2.101:0/3100377877 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f284c086270 msgr2=0x7f284c086670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:23.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:23.160 7f28517dd700 1 --2- 172.21.2.101:0/3100377877 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f284c086270 0x7f284c086670 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7f284400b910 tx=0x7f284400bcb0).stop 2021-02-10T11:34:23.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:23.160 7f28517dd700 1 -- 172.21.2.101:0/3100377877 shutdown_connections 2021-02-10T11:34:23.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:23.160 7f28517dd700 1 --2- 172.21.2.101:0/3100377877 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f284c086270 0x7f284c086670 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:23.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:23.160 7f28517dd700 1 --2- 172.21.2.101:0/3100377877 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f284c0a0290 0x7f284c0852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:23.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:23.160 7f28517dd700 1 --2- 172.21.2.101:0/3100377877 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28300186c0 0x7f283001abc0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:23.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:23.160 7f28517dd700 1 --2- 172.21.2.101:0/3100377877 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f284c0c3820 0x7f284c084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:23.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:23.160 7f28517dd700 1 -- 172.21.2.101:0/3100377877 >> 172.21.2.101:0/3100377877 conn(0x7f284c0872c0 msgr2=0x7f284c095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:23.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:23.160 7f28517dd700 1 -- 172.21.2.101:0/3100377877 shutdown_connections 2021-02-10T11:34:23.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:23.160 7f28517dd700 1 -- 172.21.2.101:0/3100377877 wait complete. 2021-02-10T11:34:23.179 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:34:23.179 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:34:23.523 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:34:23.869 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:34:23.869 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:23.869 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:34:23.870 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:34:24.204 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2021-02-10T11:34:24.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:34:24.205 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-02-10T11:34:24.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:24.205 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:34:24.205 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:34:24.539 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:34:24.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:34:24.540 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:34:24.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:24.540 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:34:24.540 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:34:24.875 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:34:24.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:34:24.875 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:34:24.875 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:24.876 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:34:24.876 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:34:25.210 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:34:25.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:34:25.210 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:34:25.210 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json-pretty 2021-02-10T11:34:25.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.612 7fc359352700 1 Processor -- start 2021-02-10T11:34:25.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.612 7fc359352700 1 -- start start 2021-02-10T11:34:25.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.613 7fc359352700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc3541b0870 0x7fc3540f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:25.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.613 7fc359352700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3541b0c70 con 0x7fc3541b0470 2021-02-10T11:34:25.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.613 7fc359352700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3540f0b50 con 0x7fc3540efe00 2021-02-10T11:34:25.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.613 7fc359352700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3540e5ea0 con 0x7fc3541b0870 2021-02-10T11:34:25.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.613 7fc35287c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc3541b0870 0x7fc3540f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:25.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.613 7fc35307d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc3541b0470 0x7fc3540ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52796/0 (socket says 172.21.2.101:52796) 2021-02-10T11:34:25.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.613 7fc35307d700 1 -- 172.21.2.101:0/2497598884 learned_addr learned my addr 172.21.2.101:0/2497598884 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:25.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.613 7fc35207b700 1 -- 172.21.2.101:0/2497598884 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2164930395 0 0) 0x7fc33c0010b0 con 0x7fc3541b0470 2021-02-10T11:34:25.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.613 7fc35207b700 1 -- 172.21.2.101:0/2497598884 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc340002fe0 con 0x7fc3541b0470 2021-02-10T11:34:25.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.613 7fc35287c700 1 -- 172.21.2.101:0/2497598884 >> v1:172.21.2.108:6789/0 conn(0x7fc3540efe00 legacy=0x7fc3540f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:25.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.613 7fc35287c700 1 -- 172.21.2.101:0/2497598884 >> v1:172.21.2.101:6789/0 conn(0x7fc3541b0470 legacy=0x7fc3540ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:25.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.613 7fc35287c700 1 -- 172.21.2.101:0/2497598884 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3540e52e0 con 0x7fc3541b0870 2021-02-10T11:34:25.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.613 7fc35287c700 1 --2- 172.21.2.101:0/2497598884 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3541b0870 0x7fc3540f20e0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0x7fc344006650 tx=0x7fc344009a80).ready entity=mon.0 client_cookie=c54643b67af95119 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:25.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.614 7fc35207b700 1 -- 172.21.2.101:0/2497598884 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc3440009b0 con 0x7fc3541b0870 2021-02-10T11:34:25.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.614 7fc35207b700 1 -- 172.21.2.101:0/2497598884 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34400c0c0 con 0x7fc3541b0870 2021-02-10T11:34:25.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.614 7fc35207b700 1 -- 172.21.2.101:0/2497598884 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc34400c5e0 con 0x7fc3541b0870 2021-02-10T11:34:25.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.614 7fc359352700 1 -- 172.21.2.101:0/2497598884 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3541b0870 msgr2=0x7fc3540f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:25.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.614 7fc359352700 1 --2- 172.21.2.101:0/2497598884 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3541b0870 0x7fc3540f20e0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0x7fc344006650 tx=0x7fc344009a80).stop 2021-02-10T11:34:25.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.614 7fc359352700 1 -- 172.21.2.101:0/2497598884 shutdown_connections 2021-02-10T11:34:25.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.614 7fc359352700 1 --2- 172.21.2.101:0/2497598884 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3541b0870 0x7fc3540f20e0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:25.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.614 7fc359352700 1 -- 172.21.2.101:0/2497598884 >> 172.21.2.101:0/2497598884 conn(0x7fc3540ce3e0 msgr2=0x7fc3540e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:25.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.614 7fc359352700 1 -- 172.21.2.101:0/2497598884 shutdown_connections 2021-02-10T11:34:25.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.614 7fc359352700 1 -- 172.21.2.101:0/2497598884 wait complete. 2021-02-10T11:34:25.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.629 7fc359352700 1 Processor -- start 2021-02-10T11:34:25.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.629 7fc359352700 1 -- start start 2021-02-10T11:34:25.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.629 7fc359352700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc354092e40 0x7fc354093240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:25.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.629 7fc359352700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc354093860 0x7fc3541a83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:25.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.629 7fc359352700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3541a89f0 0x7fc3541a8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:25.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.629 7fc359352700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc354195ad0 con 0x7fc354092e40 2021-02-10T11:34:25.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.629 7fc359352700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3541a9450 con 0x7fc354093860 2021-02-10T11:34:25.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.629 7fc359352700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3540b1b00 con 0x7fc3541a89f0 2021-02-10T11:34:25.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.630 7fc35307d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc354092e40 0x7fc354093240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:25.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.630 7fc35307d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc354092e40 0x7fc354093240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52308/0 (socket says 172.21.2.101:52308) 2021-02-10T11:34:25.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.630 7fc35307d700 1 -- 172.21.2.101:0/2034879868 learned_addr learned my addr 172.21.2.101:0/2034879868 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:25.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.630 7fc35307d700 1 -- 172.21.2.101:0/2034879868 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3541a89f0 msgr2=0x7fc3541a8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:25.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.630 7fc35307d700 1 --2- 172.21.2.101:0/2034879868 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3541a89f0 0x7fc3541a8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:25.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.630 7fc35307d700 1 -- 172.21.2.101:0/2034879868 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc354093860 msgr2=0x7fc3541a83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:25.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.630 7fc35307d700 1 --2- 172.21.2.101:0/2034879868 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc354093860 0x7fc3541a83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:25.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.630 7fc35307d700 1 -- 172.21.2.101:0/2034879868 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3541a99d0 con 0x7fc354092e40 2021-02-10T11:34:25.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.630 7fc35307d700 1 --2- 172.21.2.101:0/2034879868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc354092e40 0x7fc354093240 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7fc33c001c60 tx=0x7fc33c00a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:25.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.630 7fc34bfff700 1 -- 172.21.2.101:0/2034879868 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc33c00acc0 con 0x7fc354092e40 2021-02-10T11:34:25.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.630 7fc34bfff700 1 -- 172.21.2.101:0/2034879868 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33c004350 con 0x7fc354092e40 2021-02-10T11:34:25.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.630 7fc359352700 1 -- 172.21.2.101:0/2034879868 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3541a99d0 con 0x7fc354092e40 2021-02-10T11:34:25.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.630 7fc359352700 1 -- 172.21.2.101:0/2034879868 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3541a99d0 con 0x7fc354092e40 2021-02-10T11:34:25.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.630 7fc34bfff700 1 -- 172.21.2.101:0/2034879868 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc33c004830 con 0x7fc354092e40 2021-02-10T11:34:25.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.631 7fc34bfff700 1 -- 172.21.2.101:0/2034879868 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc33c006020 con 0x7fc354092e40 2021-02-10T11:34:25.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.631 7fc34bfff700 1 --2- 172.21.2.101:0/2034879868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3380186c0 0x7fc33801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:25.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.631 7fc34bfff700 1 -- 172.21.2.101:0/2034879868 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc33c032510 con 0x7fc354092e40 2021-02-10T11:34:25.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.631 7fc35287c700 1 --2- 172.21.2.101:0/2034879868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3380186c0 0x7fc33801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:25.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.631 7fc35287c700 1 --2- 172.21.2.101:0/2034879868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3380186c0 0x7fc33801abc0 crc :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:25.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.663 7fc359352700 1 -- 172.21.2.101:0/2034879868 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3540b1b00 con 0x7fc354092e40 2021-02-10T11:34:25.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.665 7fc34bfff700 1 -- 172.21.2.101:0/2034879868 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc33c004140 con 0x7fc354092e40 2021-02-10T11:34:25.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.793 7fc359352700 1 -- 172.21.2.101:0/2034879868 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fc3540b1b00 con 0x7fc354092e40 2021-02-10T11:34:25.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.794 7fc34bfff700 1 -- 172.21.2.101:0/2034879868 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v55) v1 ==== 81+0+30363 (secure 0 0 0) 0x7fc33c006000 con 0x7fc354092e40 2021-02-10T11:34:25.804 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:25.804 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:34:25.805 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 55, 2021-02-10T11:34:25.805 INFO:teuthology.orchestra.run.gibba001.stdout: "fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:34:25.805 INFO:teuthology.orchestra.run.gibba001.stdout: "created": "2021-02-10 11:12:36.788366", 2021-02-10T11:34:25.805 INFO:teuthology.orchestra.run.gibba001.stdout: "modified": "2021-02-10 11:33:17.591716", 2021-02-10T11:34:25.805 INFO:teuthology.orchestra.run.gibba001.stdout: "last_up_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:34:25.805 INFO:teuthology.orchestra.run.gibba001.stdout: "last_in_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:34:25.805 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-02-10T11:34:25.805 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_num": 5799936, 2021-02-10T11:34:25.805 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_set": [ 2021-02-10T11:34:25.805 INFO:teuthology.orchestra.run.gibba001.stdout: "pglog_hardlimit", 2021-02-10T11:34:25.805 INFO:teuthology.orchestra.run.gibba001.stdout: "purged_snapdirs", 2021-02-10T11:34:25.805 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_deletes", 2021-02-10T11:34:25.805 INFO:teuthology.orchestra.run.gibba001.stdout: "sortbitwise" 2021-02-10T11:34:25.806 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:34:25.806 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_version": 4, 2021-02-10T11:34:25.806 INFO:teuthology.orchestra.run.gibba001.stdout: "full_ratio": 0.94999998807907104, 2021-02-10T11:34:25.806 INFO:teuthology.orchestra.run.gibba001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-02-10T11:34:25.806 INFO:teuthology.orchestra.run.gibba001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-02-10T11:34:25.806 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_snapshot": "", 2021-02-10T11:34:25.806 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_max": 11, 2021-02-10T11:34:25.806 INFO:teuthology.orchestra.run.gibba001.stdout: "max_osd": 8, 2021-02-10T11:34:25.806 INFO:teuthology.orchestra.run.gibba001.stdout: "require_min_compat_client": "jewel", 2021-02-10T11:34:25.806 INFO:teuthology.orchestra.run.gibba001.stdout: "min_compat_client": "jewel", 2021-02-10T11:34:25.806 INFO:teuthology.orchestra.run.gibba001.stdout: "require_osd_release": "nautilus", 2021-02-10T11:34:25.806 INFO:teuthology.orchestra.run.gibba001.stdout: "pools": [ 2021-02-10T11:34:25.806 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.807 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 1, 2021-02-10T11:34:25.807 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "rbd", 2021-02-10T11:34:25.807 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:13:08.037472", 2021-02-10T11:34:25.807 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:34:25.807 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:34:25.807 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:34:25.807 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:34:25.807 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:34:25.807 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:34:25.807 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:34:25.807 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:34:25.807 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:34:25.807 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:34:25.807 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:34:25.808 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:34:25.808 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:34:25.808 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:34:25.808 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:34:25.808 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:34:25.808 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:34:25.808 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:34:25.808 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:34:25.809 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:34:25.809 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.809 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "16", 2021-02-10T11:34:25.809 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:34:25.809 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:34:25.809 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:34:25.809 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:34:25.809 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:34:25.809 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:34:25.809 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:34:25.809 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:34:25.809 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:34:25.810 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:34:25.810 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:34:25.810 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:34:25.810 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:34:25.810 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:34:25.810 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:34:25.810 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:34:25.810 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:34:25.810 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:34:25.810 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:34:25.810 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:34:25.810 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:34:25.810 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:34:25.811 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:34:25.811 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:34:25.811 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:34:25.811 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:34:25.811 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.811 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:34:25.811 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:34:25.811 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:34:25.811 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:34:25.811 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:34:25.811 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:34:25.811 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:34:25.811 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:34:25.811 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:34:25.812 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:34:25.812 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:34:25.812 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:34:25.812 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:34:25.812 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd": {} 2021-02-10T11:34:25.812 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.812 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.812 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.812 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 10, 2021-02-10T11:34:25.812 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_metadata", 2021-02-10T11:34:25.812 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:33:14.515763", 2021-02-10T11:34:25.812 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:34:25.812 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:34:25.813 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:34:25.813 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:34:25.813 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:34:25.813 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:34:25.813 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:34:25.813 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:34:25.813 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:34:25.813 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:34:25.813 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:34:25.813 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:34:25.813 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:34:25.813 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:34:25.813 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:34:25.814 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:34:25.814 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:34:25.814 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:34:25.814 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:34:25.814 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:34:25.814 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.814 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "54", 2021-02-10T11:34:25.814 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:34:25.814 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:34:25.814 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:34:25.814 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:34:25.814 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:34:25.815 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:34:25.815 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:34:25.815 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:34:25.815 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:34:25.815 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:34:25.815 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:34:25.816 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:34:25.816 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:34:25.816 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:34:25.816 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:34:25.816 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:34:25.816 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:34:25.816 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:34:25.816 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:34:25.816 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:34:25.816 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:34:25.816 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:34:25.816 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:34:25.816 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:34:25.817 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:34:25.817 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:34:25.817 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.817 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:34:25.817 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:34:25.817 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:34:25.817 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:34:25.817 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:34:25.817 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:34:25.817 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:34:25.817 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:34:25.817 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:34:25.817 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:34:25.817 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:34:25.818 INFO:teuthology.orchestra.run.gibba001.stdout: "options": { 2021-02-10T11:34:25.818 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_bias": 4, 2021-02-10T11:34:25.818 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_min": 16, 2021-02-10T11:34:25.818 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_priority": 5 2021-02-10T11:34:25.818 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.818 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:34:25.818 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:34:25.818 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": "cephfs" 2021-02-10T11:34:25.818 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.819 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.819 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.819 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.819 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 11, 2021-02-10T11:34:25.819 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_data", 2021-02-10T11:34:25.819 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:33:15.636274", 2021-02-10T11:34:25.819 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:34:25.819 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:34:25.819 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:34:25.819 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:34:25.819 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:34:25.819 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:34:25.820 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:34:25.820 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:34:25.820 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:34:25.820 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:34:25.820 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:34:25.820 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:34:25.820 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:34:25.820 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:34:25.820 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:34:25.821 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:34:25.821 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:34:25.821 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:34:25.821 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:34:25.821 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:34:25.821 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.821 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "55", 2021-02-10T11:34:25.821 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:34:25.822 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:34:25.822 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:34:25.822 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:34:25.822 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:34:25.822 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:34:25.822 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:34:25.822 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:34:25.822 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:34:25.822 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:34:25.822 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:34:25.822 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:34:25.822 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:34:25.822 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:34:25.823 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:34:25.823 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:34:25.823 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:34:25.823 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:34:25.823 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:34:25.823 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:34:25.823 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:34:25.823 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:34:25.823 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:34:25.823 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:34:25.823 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:34:25.823 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:34:25.823 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.824 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:34:25.824 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:34:25.824 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:34:25.824 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:34:25.824 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:34:25.824 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:34:25.824 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:34:25.824 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:34:25.824 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:34:25.824 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:34:25.824 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:34:25.824 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:34:25.824 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:34:25.824 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:34:25.825 INFO:teuthology.orchestra.run.gibba001.stdout: "data": "cephfs" 2021-02-10T11:34:25.825 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.825 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.825 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.825 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:34:25.825 INFO:teuthology.orchestra.run.gibba001.stdout: "osds": [ 2021-02-10T11:34:25.825 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.825 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:34:25.825 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5", 2021-02-10T11:34:25.825 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:34:25.825 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:34:25.825 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:34:25.825 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:34:25.826 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:34:25.826 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:34:25.826 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:34:25.826 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 53, 2021-02-10T11:34:25.826 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:34:25.826 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:34:25.826 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:34:25.826 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.826 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.826 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.826 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6802", 2021-02-10T11:34:25.827 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:34:25.827 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.827 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.827 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.827 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6805", 2021-02-10T11:34:25.827 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:34:25.827 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.827 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.827 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.827 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:34:25.827 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.827 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.828 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.828 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6807", 2021-02-10T11:34:25.828 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:34:25.828 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.828 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.828 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.828 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6809", 2021-02-10T11:34:25.828 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:34:25.828 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.828 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.828 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.828 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:34:25.828 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.828 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.829 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.829 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6817", 2021-02-10T11:34:25.829 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:34:25.829 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.829 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.829 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.829 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6820", 2021-02-10T11:34:25.829 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:34:25.829 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.829 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.829 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.829 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:34:25.829 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.830 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.830 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.830 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6812", 2021-02-10T11:34:25.830 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:34:25.830 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.830 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.830 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.830 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6814", 2021-02-10T11:34:25.830 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:34:25.830 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.830 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.830 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.830 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6805/25093", 2021-02-10T11:34:25.830 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6809/25093", 2021-02-10T11:34:25.831 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6820/25093", 2021-02-10T11:34:25.831 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6814/25093", 2021-02-10T11:34:25.831 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:34:25.831 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:34:25.831 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:34:25.831 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.831 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.831 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.831 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:34:25.831 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "ddc04ce3-b313-4707-8270-585391e38ecc", 2021-02-10T11:34:25.831 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:34:25.831 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:34:25.831 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:34:25.832 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:34:25.832 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:34:25.832 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:34:25.832 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:34:25.832 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 53, 2021-02-10T11:34:25.832 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:34:25.832 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:34:25.832 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:34:25.832 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.832 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.832 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.832 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6803", 2021-02-10T11:34:25.832 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:34:25.833 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.833 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.833 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.833 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6804", 2021-02-10T11:34:25.833 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:34:25.833 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.833 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.833 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.833 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:34:25.833 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.833 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.833 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.833 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6806", 2021-02-10T11:34:25.833 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:34:25.834 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.834 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.834 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.834 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6808", 2021-02-10T11:34:25.834 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:34:25.834 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.834 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.834 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.834 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:34:25.834 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.834 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.834 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.834 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6818", 2021-02-10T11:34:25.835 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:34:25.835 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.835 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.835 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.835 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6821", 2021-02-10T11:34:25.835 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:34:25.835 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.835 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.835 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.835 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:34:25.835 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.835 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.835 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.835 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6810", 2021-02-10T11:34:25.836 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:34:25.836 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.836 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.836 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.836 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6815", 2021-02-10T11:34:25.836 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:34:25.836 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.836 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.836 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.836 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6804/25092", 2021-02-10T11:34:25.836 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6808/25092", 2021-02-10T11:34:25.836 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6821/25092", 2021-02-10T11:34:25.836 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6815/25092", 2021-02-10T11:34:25.837 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:34:25.837 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:34:25.837 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:34:25.837 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.837 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.837 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.837 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:34:25.837 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3", 2021-02-10T11:34:25.837 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:34:25.837 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:34:25.837 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:34:25.837 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:34:25.837 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:34:25.837 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:34:25.838 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:34:25.838 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 53, 2021-02-10T11:34:25.838 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:34:25.838 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:34:25.838 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:34:25.838 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.838 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.838 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.838 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6811", 2021-02-10T11:34:25.838 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:34:25.838 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.838 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.838 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.838 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6813", 2021-02-10T11:34:25.839 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:34:25.839 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.839 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.839 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.839 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:34:25.839 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.839 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.839 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.839 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6816", 2021-02-10T11:34:25.839 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:34:25.839 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.839 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.839 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.840 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6819", 2021-02-10T11:34:25.840 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:34:25.840 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.840 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.840 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.840 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:34:25.840 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.840 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.840 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.840 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6825", 2021-02-10T11:34:25.840 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:34:25.840 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.840 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.840 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.841 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6827", 2021-02-10T11:34:25.841 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:34:25.841 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.841 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.841 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.841 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:34:25.841 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.841 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.841 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.841 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6822", 2021-02-10T11:34:25.841 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:34:25.841 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.841 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.841 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.842 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6823", 2021-02-10T11:34:25.842 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:34:25.842 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.842 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.842 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.842 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6813/25094", 2021-02-10T11:34:25.842 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6819/25094", 2021-02-10T11:34:25.842 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6827/25094", 2021-02-10T11:34:25.842 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6823/25094", 2021-02-10T11:34:25.842 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:34:25.842 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:34:25.842 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:34:25.842 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.843 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.843 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.843 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:34:25.843 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "0d9ef9b1-98aa-4d37-aa16-6305c7d46d52", 2021-02-10T11:34:25.843 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:34:25.843 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:34:25.843 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:34:25.843 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:34:25.843 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:34:25.843 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:34:25.843 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:34:25.843 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 45, 2021-02-10T11:34:25.843 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:34:25.843 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:34:25.844 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:34:25.844 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.844 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.844 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.844 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6824", 2021-02-10T11:34:25.844 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:34:25.844 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.844 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.844 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.844 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6826", 2021-02-10T11:34:25.844 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:34:25.844 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.844 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.845 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.845 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:34:25.845 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.845 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.845 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.845 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6828", 2021-02-10T11:34:25.845 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:34:25.845 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.845 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.845 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.845 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6829", 2021-02-10T11:34:25.845 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:34:25.845 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.845 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.846 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.846 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:34:25.846 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.846 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.846 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.846 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6832", 2021-02-10T11:34:25.846 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:34:25.846 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.846 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.846 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.846 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6833", 2021-02-10T11:34:25.846 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:34:25.846 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.846 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.847 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.847 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:34:25.847 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.847 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.847 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.847 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6830", 2021-02-10T11:34:25.847 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:34:25.847 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.847 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.847 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.847 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6831", 2021-02-10T11:34:25.847 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:34:25.847 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.848 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.848 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.848 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6826/25095", 2021-02-10T11:34:25.848 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6829/25095", 2021-02-10T11:34:25.848 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6833/25095", 2021-02-10T11:34:25.848 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6831/25095", 2021-02-10T11:34:25.848 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:34:25.848 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:34:25.848 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:34:25.848 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.848 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.848 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.848 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:34:25.848 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "36742c21-bfeb-4e11-bce9-8e745378b605", 2021-02-10T11:34:25.849 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:34:25.849 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:34:25.849 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:34:25.849 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:34:25.849 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:34:25.849 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:34:25.849 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:34:25.849 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 52, 2021-02-10T11:34:25.849 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:34:25.849 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:34:25.849 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:34:25.849 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.849 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.849 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.850 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6816", 2021-02-10T11:34:25.850 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:34:25.850 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.850 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.850 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.850 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6818", 2021-02-10T11:34:25.850 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:34:25.850 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.850 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.850 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.850 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:34:25.850 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.850 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.851 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.851 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6820", 2021-02-10T11:34:25.851 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:34:25.851 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.851 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.851 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.851 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6822", 2021-02-10T11:34:25.851 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:34:25.851 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.851 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.851 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.851 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:34:25.851 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.851 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.852 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.852 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6828", 2021-02-10T11:34:25.852 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:34:25.852 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.852 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.852 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.852 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6830", 2021-02-10T11:34:25.852 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:34:25.852 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.852 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.852 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.852 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:34:25.852 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.852 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.853 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.853 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6824", 2021-02-10T11:34:25.853 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:34:25.853 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.853 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.853 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.853 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6826", 2021-02-10T11:34:25.853 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:34:25.853 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.853 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.853 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.853 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6818/36380", 2021-02-10T11:34:25.853 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6822/36380", 2021-02-10T11:34:25.853 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6830/36380", 2021-02-10T11:34:25.854 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6826/36380", 2021-02-10T11:34:25.854 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:34:25.854 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:34:25.854 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:34:25.854 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.854 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.854 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.854 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:34:25.854 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "88e694e8-fe98-4d1b-88c8-8c9907ece60b", 2021-02-10T11:34:25.854 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:34:25.854 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:34:25.854 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:34:25.854 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:34:25.855 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:34:25.855 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:34:25.855 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:34:25.855 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 53, 2021-02-10T11:34:25.855 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:34:25.855 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:34:25.855 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:34:25.855 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.855 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.855 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.855 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6808", 2021-02-10T11:34:25.855 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:34:25.855 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.855 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.856 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.856 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6809", 2021-02-10T11:34:25.856 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:34:25.856 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.856 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.856 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.856 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:34:25.856 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.856 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.856 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.856 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6810", 2021-02-10T11:34:25.856 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:34:25.856 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.856 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.857 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.857 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6811", 2021-02-10T11:34:25.857 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:34:25.857 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.857 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.857 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.857 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:34:25.857 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.857 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.857 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.857 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6814", 2021-02-10T11:34:25.857 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:34:25.857 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.857 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.858 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.858 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6815", 2021-02-10T11:34:25.858 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:34:25.858 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.858 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.858 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.858 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:34:25.858 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.858 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.858 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.858 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6812", 2021-02-10T11:34:25.858 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:34:25.858 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.859 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.859 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.859 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6813", 2021-02-10T11:34:25.859 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:34:25.859 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.859 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.859 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.859 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6809/36383", 2021-02-10T11:34:25.859 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6811/36383", 2021-02-10T11:34:25.859 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6815/36383", 2021-02-10T11:34:25.859 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6813/36383", 2021-02-10T11:34:25.859 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:34:25.859 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:34:25.859 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:34:25.860 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.860 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.860 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.860 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:34:25.860 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "87a13c6c-cf85-4aaf-916b-e73913484a49", 2021-02-10T11:34:25.860 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:34:25.860 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:34:25.860 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:34:25.860 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:34:25.860 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:34:25.860 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:34:25.860 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:34:25.860 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 53, 2021-02-10T11:34:25.860 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:34:25.861 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:34:25.861 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:34:25.861 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.861 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.861 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.861 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6800", 2021-02-10T11:34:25.861 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:34:25.861 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.861 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.861 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.861 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6801", 2021-02-10T11:34:25.861 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:34:25.861 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.861 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.862 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.862 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:34:25.862 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.862 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.862 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.862 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6802", 2021-02-10T11:34:25.862 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:34:25.862 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.862 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.862 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.862 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6803", 2021-02-10T11:34:25.862 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:34:25.862 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.862 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.863 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.863 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:34:25.863 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.863 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.863 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.863 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6806", 2021-02-10T11:34:25.863 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:34:25.863 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.863 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.863 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.863 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6807", 2021-02-10T11:34:25.863 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:34:25.863 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.864 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.864 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.864 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:34:25.864 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.864 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.864 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.864 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6804", 2021-02-10T11:34:25.864 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:34:25.864 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.864 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.864 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.864 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6805", 2021-02-10T11:34:25.864 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:34:25.864 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.865 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.865 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.865 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6801/36381", 2021-02-10T11:34:25.865 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6803/36381", 2021-02-10T11:34:25.865 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6807/36381", 2021-02-10T11:34:25.865 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6805/36381", 2021-02-10T11:34:25.865 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:34:25.865 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:34:25.865 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:34:25.865 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.865 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.865 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.865 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:34:25.865 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "7b90823f-854c-4592-a95b-a7fa98f93254", 2021-02-10T11:34:25.866 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:34:25.866 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:34:25.866 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:34:25.866 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:34:25.866 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:34:25.866 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:34:25.866 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:34:25.866 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 53, 2021-02-10T11:34:25.866 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:34:25.866 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:34:25.866 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:34:25.866 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.866 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.867 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.867 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6817", 2021-02-10T11:34:25.867 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:34:25.867 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.867 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.867 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.867 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6819", 2021-02-10T11:34:25.867 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:34:25.867 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.867 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.867 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.867 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:34:25.867 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.867 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.868 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.868 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6821", 2021-02-10T11:34:25.868 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:34:25.868 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.868 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.868 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.868 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6823", 2021-02-10T11:34:25.868 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:34:25.868 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.868 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.868 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.868 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:34:25.868 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.868 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.869 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.869 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6829", 2021-02-10T11:34:25.869 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:34:25.869 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.869 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.869 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.869 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6831", 2021-02-10T11:34:25.869 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:34:25.869 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.869 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.869 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.869 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:34:25.869 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:34:25.869 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.870 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:34:25.870 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6825", 2021-02-10T11:34:25.870 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:34:25.870 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.870 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.870 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:25.870 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6827", 2021-02-10T11:34:25.870 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:34:25.870 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.870 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.870 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.870 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6819/36382", 2021-02-10T11:34:25.870 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6823/36382", 2021-02-10T11:34:25.870 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6831/36382", 2021-02-10T11:34:25.871 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6827/36382", 2021-02-10T11:34:25.871 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:34:25.871 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:34:25.871 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:34:25.871 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:25.871 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.871 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:34:25.871 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_xinfo": [ 2021-02-10T11:34:25.871 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.871 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:34:25.871 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:34:25.871 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:34:25.871 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:34:25.871 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:34:25.872 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:34:25.872 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.872 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.872 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:34:25.872 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:34:25.872 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:34:25.872 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:34:25.872 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:34:25.872 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:34:25.872 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.872 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.872 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:34:25.872 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:34:25.872 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:34:25.873 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:34:25.873 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:34:25.873 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:34:25.873 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.873 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.873 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:34:25.873 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:34:25.873 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:34:25.873 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:34:25.873 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:34:25.873 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:34:25.873 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.873 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.873 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:34:25.874 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:34:25.874 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:34:25.874 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:34:25.874 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:34:25.874 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:34:25.874 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.874 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.874 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:34:25.874 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:34:25.874 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:34:25.874 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:34:25.874 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:34:25.874 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:34:25.875 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.875 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.875 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:34:25.875 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:34:25.875 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:34:25.875 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:34:25.875 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:34:25.875 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:34:25.875 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.875 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:25.875 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:34:25.875 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:34:25.875 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:34:25.875 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:34:25.876 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:34:25.876 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:34:25.876 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.876 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:34:25.876 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap": [], 2021-02-10T11:34:25.876 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap_items": [], 2021-02-10T11:34:25.876 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_temp": [], 2021-02-10T11:34:25.876 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_temp": [], 2021-02-10T11:34:25.876 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklist": {}, 2021-02-10T11:34:25.876 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profiles": { 2021-02-10T11:34:25.876 INFO:teuthology.orchestra.run.gibba001.stdout: "default": { 2021-02-10T11:34:25.876 INFO:teuthology.orchestra.run.gibba001.stdout: "crush-failure-domain": "osd", 2021-02-10T11:34:25.876 INFO:teuthology.orchestra.run.gibba001.stdout: "k": "2", 2021-02-10T11:34:25.876 INFO:teuthology.orchestra.run.gibba001.stdout: "m": "1", 2021-02-10T11:34:25.877 INFO:teuthology.orchestra.run.gibba001.stdout: "plugin": "jerasure", 2021-02-10T11:34:25.877 INFO:teuthology.orchestra.run.gibba001.stdout: "ruleset-failure-domain": "osd", 2021-02-10T11:34:25.877 INFO:teuthology.orchestra.run.gibba001.stdout: "technique": "reed_sol_van" 2021-02-10T11:34:25.877 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:25.877 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:25.877 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps_queue": [], 2021-02-10T11:34:25.877 INFO:teuthology.orchestra.run.gibba001.stdout: "new_removed_snaps": [], 2021-02-10T11:34:25.877 INFO:teuthology.orchestra.run.gibba001.stdout: "new_purged_snaps": [], 2021-02-10T11:34:25.877 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_node_flags": {}, 2021-02-10T11:34:25.877 INFO:teuthology.orchestra.run.gibba001.stdout: "device_class_flags": {} 2021-02-10T11:34:25.877 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:34:25.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.795 7fc359352700 1 -- 172.21.2.101:0/2034879868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3380186c0 msgr2=0x7fc33801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:25.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.795 7fc359352700 1 --2- 172.21.2.101:0/2034879868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3380186c0 0x7fc33801abc0 crc :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:25.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.795 7fc359352700 1 -- 172.21.2.101:0/2034879868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc354092e40 msgr2=0x7fc354093240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:25.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.795 7fc359352700 1 --2- 172.21.2.101:0/2034879868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc354092e40 0x7fc354093240 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7fc33c001c60 tx=0x7fc33c00a8d0).stop 2021-02-10T11:34:25.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.795 7fc359352700 1 -- 172.21.2.101:0/2034879868 shutdown_connections 2021-02-10T11:34:25.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.795 7fc359352700 1 --2- 172.21.2.101:0/2034879868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3380186c0 0x7fc33801abc0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:25.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.795 7fc359352700 1 --2- 172.21.2.101:0/2034879868 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3541a89f0 0x7fc3541a8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:25.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.795 7fc359352700 1 --2- 172.21.2.101:0/2034879868 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc354093860 0x7fc3541a83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:25.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.795 7fc359352700 1 --2- 172.21.2.101:0/2034879868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc354092e40 0x7fc354093240 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:25.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.795 7fc359352700 1 -- 172.21.2.101:0/2034879868 >> 172.21.2.101:0/2034879868 conn(0x7fc35419f7a0 msgr2=0x7fc3540940e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:25.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.796 7fc359352700 1 -- 172.21.2.101:0/2034879868 shutdown_connections 2021-02-10T11:34:25.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:25.796 7fc359352700 1 -- 172.21.2.101:0/2034879868 wait complete. 2021-02-10T11:34:25.880 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:34:26.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.280 7ff0fa2f5700 1 Processor -- start 2021-02-10T11:34:26.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.280 7ff0fa2f5700 1 -- start start 2021-02-10T11:34:26.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.281 7ff0fa2f5700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff0f40d1870 0x7ff0f41b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:26.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.281 7ff0fa2f5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0f40f0650 0x7ff0f40cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:26.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.281 7ff0fa2f5700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f40f0b10 con 0x7ff0f41b0a10 2021-02-10T11:34:26.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.281 7ff0fa2f5700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f40e5ed0 con 0x7ff0f40d1870 2021-02-10T11:34:26.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.281 7ff0fa2f5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f40d1c70 con 0x7ff0f40f0650 2021-02-10T11:34:26.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.281 7ff0f3fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff0f40d1870 0x7ff0f41b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:26.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.281 7ff0f37fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff0f41b0a10 0x7ff0f40cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52810/0 (socket says 172.21.2.101:52810) 2021-02-10T11:34:26.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.281 7ff0f3fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff0f40d1870 0x7ff0f41b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52316/0 (socket says 172.21.2.101:52316) 2021-02-10T11:34:26.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.281 7ff0f37fe700 1 -- 172.21.2.101:0/2736704094 learned_addr learned my addr 172.21.2.101:0/2736704094 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:26.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.281 7ff0f8912700 1 --2- 172.21.2.101:0/2736704094 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0f40f0650 0x7ff0f40cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:26.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.281 7ff0f3fff700 1 -- 172.21.2.101:0/2736704094 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0f40f0650 msgr2=0x7ff0f40cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:26.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.282 7ff0f3fff700 1 --2- 172.21.2.101:0/2736704094 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0f40f0650 0x7ff0f40cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:26.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.282 7ff0f3fff700 1 -- 172.21.2.101:0/2736704094 >> v1:172.21.2.101:6789/0 conn(0x7ff0f41b0a10 legacy=0x7ff0f40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:26.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.282 7ff0f3fff700 1 -- 172.21.2.101:0/2736704094 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f40d1e90 con 0x7ff0f40d1870 2021-02-10T11:34:26.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.282 7ff0f2ffd700 1 -- 172.21.2.101:0/2736704094 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3562511928 0 0) 0x7ff0e80010b0 con 0x7ff0f41b0a10 2021-02-10T11:34:26.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.282 7ff0f8912700 1 --2- 172.21.2.101:0/2736704094 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0f40f0650 0x7ff0f40cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:34:26.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.282 7ff0f3fff700 1 --2- 172.21.2.101:0/2736704094 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0f40d1870 0x7ff0f41b03f0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7ff0e4006650 tx=0x7ff0e4009a80).ready entity=mon.0 client_cookie=72aae58a18ddb5c1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:26.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.282 7ff0f2ffd700 1 -- 172.21.2.101:0/2736704094 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0e400c0c0 con 0x7ff0f40d1870 2021-02-10T11:34:26.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.282 7ff0f2ffd700 1 -- 172.21.2.101:0/2736704094 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e400c340 con 0x7ff0f40d1870 2021-02-10T11:34:26.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.282 7ff0f2ffd700 1 -- 172.21.2.101:0/2736704094 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0e400c860 con 0x7ff0f40d1870 2021-02-10T11:34:26.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.282 7ff0fa2f5700 1 -- 172.21.2.101:0/2736704094 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0f40d1870 msgr2=0x7ff0f41b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:26.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.282 7ff0fa2f5700 1 --2- 172.21.2.101:0/2736704094 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0f40d1870 0x7ff0f41b03f0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7ff0e4006650 tx=0x7ff0e4009a80).stop 2021-02-10T11:34:26.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.283 7ff0fa2f5700 1 -- 172.21.2.101:0/2736704094 shutdown_connections 2021-02-10T11:34:26.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.283 7ff0fa2f5700 1 --2- 172.21.2.101:0/2736704094 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0f40f0650 0x7ff0f40cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:26.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.283 7ff0fa2f5700 1 --2- 172.21.2.101:0/2736704094 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0f40d1870 0x7ff0f41b03f0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:26.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.283 7ff0fa2f5700 1 -- 172.21.2.101:0/2736704094 >> 172.21.2.101:0/2736704094 conn(0x7ff0f40ce450 msgr2=0x7ff0f40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:26.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.283 7ff0fa2f5700 1 -- 172.21.2.101:0/2736704094 shutdown_connections 2021-02-10T11:34:26.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.283 7ff0fa2f5700 1 -- 172.21.2.101:0/2736704094 wait complete. 2021-02-10T11:34:26.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.298 7ff0fa2f5700 1 Processor -- start 2021-02-10T11:34:26.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.298 7ff0fa2f5700 1 -- start start 2021-02-10T11:34:26.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.298 7ff0fa2f5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0f40ada90 0x7ff0f40ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:26.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.298 7ff0fa2f5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0f4099d20 0x7ff0f409a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:26.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.298 7ff0fa2f5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0f409a740 0x7ff0f4097bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:26.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.298 7ff0fa2f5700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f40997a0 con 0x7ff0f4099d20 2021-02-10T11:34:26.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.298 7ff0fa2f5700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f418f0c0 con 0x7ff0f409a740 2021-02-10T11:34:26.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.298 7ff0fa2f5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f409ab40 con 0x7ff0f40ada90 2021-02-10T11:34:26.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.298 7ff0f37fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0f4099d20 0x7ff0f409a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:26.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.298 7ff0f37fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0f4099d20 0x7ff0f409a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52322/0 (socket says 172.21.2.101:52322) 2021-02-10T11:34:26.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.298 7ff0f37fe700 1 -- 172.21.2.101:0/2407032794 learned_addr learned my addr 172.21.2.101:0/2407032794 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:26.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.299 7ff0f37fe700 1 -- 172.21.2.101:0/2407032794 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0f40ada90 msgr2=0x7ff0f40ade90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:34:26.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.299 7ff0f37fe700 1 --2- 172.21.2.101:0/2407032794 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0f40ada90 0x7ff0f40ade90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:26.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.299 7ff0f37fe700 1 -- 172.21.2.101:0/2407032794 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0f409a740 msgr2=0x7ff0f4097bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:34:26.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.299 7ff0f37fe700 1 --2- 172.21.2.101:0/2407032794 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0f409a740 0x7ff0f4097bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:26.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.299 7ff0f37fe700 1 -- 172.21.2.101:0/2407032794 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f409ad60 con 0x7ff0f4099d20 2021-02-10T11:34:26.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.299 7ff0f37fe700 1 --2- 172.21.2.101:0/2407032794 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0f4099d20 0x7ff0f409a120 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7ff0e8001c60 tx=0x7ff0e800a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:26.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.299 7ff0f0ff9700 1 -- 172.21.2.101:0/2407032794 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0e800acc0 con 0x7ff0f4099d20 2021-02-10T11:34:26.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.299 7ff0f0ff9700 1 -- 172.21.2.101:0/2407032794 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e8004140 con 0x7ff0f4099d20 2021-02-10T11:34:26.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.299 7ff0f0ff9700 1 -- 172.21.2.101:0/2407032794 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0e8004660 con 0x7ff0f4099d20 2021-02-10T11:34:26.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.299 7ff0fa2f5700 1 -- 172.21.2.101:0/2407032794 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f409ad60 con 0x7ff0f4099d20 2021-02-10T11:34:26.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.299 7ff0fa2f5700 1 -- 172.21.2.101:0/2407032794 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f40984d0 con 0x7ff0f4099d20 2021-02-10T11:34:26.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.300 7ff0f0ff9700 1 -- 172.21.2.101:0/2407032794 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff0e8004320 con 0x7ff0f4099d20 2021-02-10T11:34:26.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.300 7ff0f0ff9700 1 --2- 172.21.2.101:0/2407032794 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0d40186c0 0x7ff0d401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:26.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.300 7ff0f0ff9700 1 -- 172.21.2.101:0/2407032794 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff0e80325b0 con 0x7ff0f4099d20 2021-02-10T11:34:26.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.300 7ff0f3fff700 1 --2- 172.21.2.101:0/2407032794 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0d40186c0 0x7ff0d401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:26.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.300 7ff0f3fff700 1 --2- 172.21.2.101:0/2407032794 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0d40186c0 0x7ff0d401abc0 crc :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:26.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.331 7ff0fa2f5700 1 -- 172.21.2.101:0/2407032794 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f418f070 con 0x7ff0f4099d20 2021-02-10T11:34:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.333 7ff0f0ff9700 1 -- 172.21.2.101:0/2407032794 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff0e8030d60 con 0x7ff0f4099d20 2021-02-10T11:34:26.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.460 7ff0fa2f5700 1 -- 172.21.2.101:0/2407032794 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0f418f070 con 0x7ff0f4099d20 2021-02-10T11:34:26.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.461 7ff0f0ff9700 1 -- 172.21.2.101:0/2407032794 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+2098 (secure 0 0 0) 0x7ff0e800acc0 con 0x7ff0f4099d20 2021-02-10T11:34:26.472 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:26.472 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8049,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/172865610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":172865610},{"type":"v1","addr":"172.21.2.101:6835","nonce":172865610}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:33:16.590282","modified":"2021-02-10 11:33:18.594009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5579},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5579":{"gid":5579,"name":"b","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2400978339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2400978339},{"type":"v1","addr":"172.21.2.108:6833","nonce":2400978339}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-02-10T11:34:26.472 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 39 2021-02-10T11:34:26.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.463 7ff0fa2f5700 1 -- 172.21.2.101:0/2407032794 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0d40186c0 msgr2=0x7ff0d401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:26.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.463 7ff0fa2f5700 1 --2- 172.21.2.101:0/2407032794 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0d40186c0 0x7ff0d401abc0 crc :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:26.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.463 7ff0fa2f5700 1 -- 172.21.2.101:0/2407032794 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0f4099d20 msgr2=0x7ff0f409a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:26.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.463 7ff0fa2f5700 1 --2- 172.21.2.101:0/2407032794 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0f4099d20 0x7ff0f409a120 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7ff0e8001c60 tx=0x7ff0e800a8d0).stop 2021-02-10T11:34:26.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.463 7ff0fa2f5700 1 -- 172.21.2.101:0/2407032794 shutdown_connections 2021-02-10T11:34:26.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.463 7ff0fa2f5700 1 --2- 172.21.2.101:0/2407032794 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0f409a740 0x7ff0f4097bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:26.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.463 7ff0fa2f5700 1 --2- 172.21.2.101:0/2407032794 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0f4099d20 0x7ff0f409a120 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:26.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.463 7ff0fa2f5700 1 --2- 172.21.2.101:0/2407032794 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0d40186c0 0x7ff0d401abc0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:26.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.463 7ff0fa2f5700 1 --2- 172.21.2.101:0/2407032794 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0f40ada90 0x7ff0f40ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:26.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.463 7ff0fa2f5700 1 -- 172.21.2.101:0/2407032794 >> 172.21.2.101:0/2407032794 conn(0x7ff0f40f0550 msgr2=0x7ff0f4191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:26.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.463 7ff0fa2f5700 1 -- 172.21.2.101:0/2407032794 shutdown_connections 2021-02-10T11:34:26.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.463 7ff0fa2f5700 1 -- 172.21.2.101:0/2407032794 wait complete. 2021-02-10T11:34:26.482 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs fail cephfs 2021-02-10T11:34:26.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.881 7f1974297700 1 Processor -- start 2021-02-10T11:34:26.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.881 7f1974297700 1 -- start start 2021-02-10T11:34:26.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.882 7f1974297700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f196c0d1870 0x7f196c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:26.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.882 7f1974297700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f196c1b0a10 0x7f196c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:26.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.882 7f1974297700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c0f0b10 con 0x7f196c0d1c70 2021-02-10T11:34:26.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.882 7f1974297700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196c0fe230 con 0x7f196c1b0a10 2021-02-10T11:34:26.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.882 7f1974297700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0e5ed0 con 0x7f196c0d1870 2021-02-10T11:34:26.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.882 7f19720b3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f196c0d1870 0x7f196c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:26.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.882 7f19718b2700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f196c1b0a10 0x7f196c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:26.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.882 7f19728b4700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f196c0d1c70 0x7f196c0cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:46232/0 (socket says 172.21.2.101:46232) 2021-02-10T11:34:26.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.882 7f19728b4700 1 -- 172.21.2.101:0/4208926705 learned_addr learned my addr 172.21.2.101:0/4208926705 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:26.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.882 7f19718b2700 1 -- 172.21.2.101:0/4208926705 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f196c0d1870 msgr2=0x7f196c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:26.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.882 7f19718b2700 1 --2- 172.21.2.101:0/4208926705 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f196c0d1870 0x7f196c1b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:26.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.882 7f19718b2700 1 -- 172.21.2.101:0/4208926705 >> v1:172.21.2.108:6789/0 conn(0x7f196c0d1c70 legacy=0x7f196c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:26.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.882 7f19718b2700 1 -- 172.21.2.101:0/4208926705 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c0f0650 con 0x7f196c1b0a10 2021-02-10T11:34:26.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.883 7f19718b2700 1 --2- 172.21.2.101:0/4208926705 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f196c1b0a10 0x7f196c0cf330 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f195c006650 tx=0x7f195c009a80).ready entity=mon.1 client_cookie=b17712573ad0d705 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:26.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.883 7f19710b1700 1 -- 172.21.2.101:0/4208926705 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f195c00c0c0 con 0x7f196c1b0a10 2021-02-10T11:34:26.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.883 7f19710b1700 1 -- 172.21.2.101:0/4208926705 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c00c340 con 0x7f196c1b0a10 2021-02-10T11:34:26.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.883 7f19710b1700 1 -- 172.21.2.101:0/4208926705 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f195c00c820 con 0x7f196c1b0a10 2021-02-10T11:34:26.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.883 7f1974297700 1 -- 172.21.2.101:0/4208926705 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f196c1b0a10 msgr2=0x7f196c0cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:26.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.883 7f1974297700 1 --2- 172.21.2.101:0/4208926705 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f196c1b0a10 0x7f196c0cf330 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f195c006650 tx=0x7f195c009a80).stop 2021-02-10T11:34:26.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.884 7f1974297700 1 -- 172.21.2.101:0/4208926705 shutdown_connections 2021-02-10T11:34:26.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.884 7f1974297700 1 --2- 172.21.2.101:0/4208926705 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f196c1b0a10 0x7f196c0cf330 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:26.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.884 7f1974297700 1 --2- 172.21.2.101:0/4208926705 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f196c0d1870 0x7f196c1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:26.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.884 7f1974297700 1 -- 172.21.2.101:0/4208926705 >> 172.21.2.101:0/4208926705 conn(0x7f196c0ce450 msgr2=0x7f196c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:26.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.884 7f1974297700 1 -- 172.21.2.101:0/4208926705 shutdown_connections 2021-02-10T11:34:26.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.884 7f1974297700 1 -- 172.21.2.101:0/4208926705 wait complete. 2021-02-10T11:34:26.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.899 7f1974297700 1 Processor -- start 2021-02-10T11:34:26.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.899 7f1974297700 1 -- start start 2021-02-10T11:34:26.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.899 7f1974297700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f196c19ffa0 0x7f196c0eda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:26.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.899 7f1974297700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f196c1a03a0 0x7f196c0ee030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:26.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.899 7f1974297700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f196c0ee650 0x7f196c0ebe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:26.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.899 7f1974297700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0ed490 con 0x7f196c19ffa0 2021-02-10T11:34:26.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.899 7f1974297700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0eea50 con 0x7f196c0ee650 2021-02-10T11:34:26.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.899 7f1974297700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0eec30 con 0x7f196c1a03a0 2021-02-10T11:34:26.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.899 7f19720b3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f196c19ffa0 0x7f196c0eda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:26.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.899 7f19720b3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f196c19ffa0 0x7f196c0eda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52334/0 (socket says 172.21.2.101:52334) 2021-02-10T11:34:26.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.899 7f19720b3700 1 -- 172.21.2.101:0/3779546748 learned_addr learned my addr 172.21.2.101:0/3779546748 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:26.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.900 7f19720b3700 1 -- 172.21.2.101:0/3779546748 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f196c1a03a0 msgr2=0x7f196c0ee030 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:34:26.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.900 7f19720b3700 1 --2- 172.21.2.101:0/3779546748 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f196c1a03a0 0x7f196c0ee030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:26.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.900 7f19720b3700 1 -- 172.21.2.101:0/3779546748 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f196c0ee650 msgr2=0x7f196c0ebe80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:34:26.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.900 7f19720b3700 1 --2- 172.21.2.101:0/3779546748 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f196c0ee650 0x7f196c0ebe80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:26.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.900 7f19720b3700 1 -- 172.21.2.101:0/3779546748 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c0ec4a0 con 0x7f196c19ffa0 2021-02-10T11:34:26.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.900 7f19720b3700 1 --2- 172.21.2.101:0/3779546748 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f196c19ffa0 0x7f196c0eda10 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7f1968006f60 tx=0x7f196800b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:26.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.900 7f1962ffd700 1 -- 172.21.2.101:0/3779546748 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f196800bd60 con 0x7f196c19ffa0 2021-02-10T11:34:26.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.900 7f1962ffd700 1 -- 172.21.2.101:0/3779546748 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1968000c90 con 0x7f196c19ffa0 2021-02-10T11:34:26.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.900 7f1962ffd700 1 -- 172.21.2.101:0/3779546748 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1968002220 con 0x7f196c19ffa0 2021-02-10T11:34:26.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.900 7f1974297700 1 -- 172.21.2.101:0/3779546748 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196c0ec4a0 con 0x7f196c19ffa0 2021-02-10T11:34:26.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.900 7f1974297700 1 -- 172.21.2.101:0/3779546748 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196c0ec4a0 con 0x7f196c19ffa0 2021-02-10T11:34:26.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.900 7f1962ffd700 1 -- 172.21.2.101:0/3779546748 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1968006b00 con 0x7f196c19ffa0 2021-02-10T11:34:26.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.900 7f1962ffd700 1 --2- 172.21.2.101:0/3779546748 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19540186c0 0x7f195401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:26.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.900 7f1962ffd700 1 -- 172.21.2.101:0/3779546748 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f19680327c0 con 0x7f196c19ffa0 2021-02-10T11:34:26.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.901 7f19718b2700 1 --2- 172.21.2.101:0/3779546748 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19540186c0 0x7f195401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:26.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.901 7f19718b2700 1 --2- 172.21.2.101:0/3779546748 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19540186c0 0x7f195401abc0 crc :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:26.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.931 7f1974297700 1 -- 172.21.2.101:0/3779546748 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196c0ebc50 con 0x7f196c19ffa0 2021-02-10T11:34:26.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:26.934 7f1962ffd700 1 -- 172.21.2.101:0/3779546748 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1968006b00 con 0x7f196c19ffa0 2021-02-10T11:34:27.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:27.061 7f1974297700 1 -- 172.21.2.101:0/3779546748 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f196c0ebc50 con 0x7f196c19ffa0 2021-02-10T11:34:27.862 INFO:tasks.ceph.mon.a.gibba001.stderr:2021-02-10 11:34:27.853 7f974d04d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-02-10T11:34:27.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:27.860 7f1962ffd700 1 -- 172.21.2.101:0/3779546748 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) v1 ==== 161+0+0 (secure 0 0 0) 0x7f196801f500 con 0x7f196c19ffa0 2021-02-10T11:34:27.891 INFO:teuthology.orchestra.run.gibba001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-02-10T11:34:27.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:27.884 7f1974297700 1 -- 172.21.2.101:0/3779546748 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19540186c0 msgr2=0x7f195401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:27.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:27.884 7f1974297700 1 --2- 172.21.2.101:0/3779546748 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19540186c0 0x7f195401abc0 crc :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:27.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:27.884 7f1974297700 1 -- 172.21.2.101:0/3779546748 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f196c19ffa0 msgr2=0x7f196c0eda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:27.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:27.884 7f1974297700 1 --2- 172.21.2.101:0/3779546748 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f196c19ffa0 0x7f196c0eda10 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7f1968006f60 tx=0x7f196800b970).stop 2021-02-10T11:34:27.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:27.884 7f1974297700 1 -- 172.21.2.101:0/3779546748 shutdown_connections 2021-02-10T11:34:27.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:27.884 7f1974297700 1 --2- 172.21.2.101:0/3779546748 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19540186c0 0x7f195401abc0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:27.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:27.884 7f1974297700 1 --2- 172.21.2.101:0/3779546748 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f196c1a03a0 0x7f196c0ee030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:27.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:27.884 7f1974297700 1 --2- 172.21.2.101:0/3779546748 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f196c0ee650 0x7f196c0ebe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:27.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:27.884 7f1974297700 1 --2- 172.21.2.101:0/3779546748 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f196c19ffa0 0x7f196c0eda10 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:27.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:27.884 7f1974297700 1 -- 172.21.2.101:0/3779546748 >> 172.21.2.101:0/3779546748 conn(0x7f196c094d90 msgr2=0x7f196c08a420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:27.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:27.884 7f1974297700 1 -- 172.21.2.101:0/3779546748 shutdown_connections 2021-02-10T11:34:27.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:27.884 7f1974297700 1 -- 172.21.2.101:0/3779546748 wait complete. 2021-02-10T11:34:27.902 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:34:27.919 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:34:28.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.302 7fc80d18d700 1 Processor -- start 2021-02-10T11:34:28.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.302 7fc80d18d700 1 -- start start 2021-02-10T11:34:28.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.303 7fc80d18d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc8080d1870 0x7fc8081b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:28.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.303 7fc80d18d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8081b0a10 0x7fc8080cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:28.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.303 7fc80d18d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8080f0b10 con 0x7fc8080d1c70 2021-02-10T11:34:28.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.303 7fc80d18d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8080e5ed0 con 0x7fc8080d1870 2021-02-10T11:34:28.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.303 7fc80d18d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8080f0650 con 0x7fc8081b0a10 2021-02-10T11:34:28.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.303 7fc80759e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc8080d1c70 0x7fc8080cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52838/0 (socket says 172.21.2.101:52838) 2021-02-10T11:34:28.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.303 7fc80759e700 1 -- 172.21.2.101:0/1113856463 learned_addr learned my addr 172.21.2.101:0/1113856463 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:28.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.303 7fc805d9b700 1 -- 172.21.2.101:0/1113856463 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 528446651 0 0) 0x7fc800001500 con 0x7fc8080d1c70 2021-02-10T11:34:28.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc805d9b700 1 -- 172.21.2.101:0/1113856463 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7f4002fe0 con 0x7fc8080d1c70 2021-02-10T11:34:28.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc80659c700 1 --2- 172.21.2.101:0/1113856463 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8081b0a10 0x7fc8080cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:28.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc806d9d700 1 --2- 172.21.2.101:0/1113856463 >> v2:172.21.2.108:3300/0 conn(0x7fc8080d1870 0x7fc8081b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:28.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc805d9b700 1 -- 172.21.2.101:0/1113856463 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2957210126 0 0) 0x7fc800001460 con 0x7fc8080d1c70 2021-02-10T11:34:28.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc805d9b700 1 -- 172.21.2.101:0/1113856463 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8081b0a10 msgr2=0x7fc8080cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:28.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc805d9b700 1 --2- 172.21.2.101:0/1113856463 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8081b0a10 0x7fc8080cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:28.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc805d9b700 1 -- 172.21.2.101:0/1113856463 >> v2:172.21.2.108:3300/0 conn(0x7fc8080d1870 msgr2=0x7fc8081b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:28.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc805d9b700 1 --2- 172.21.2.101:0/1113856463 >> v2:172.21.2.108:3300/0 conn(0x7fc8080d1870 0x7fc8081b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:28.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc805d9b700 1 -- 172.21.2.101:0/1113856463 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8080f0870 con 0x7fc8080d1c70 2021-02-10T11:34:28.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc805d9b700 1 -- 172.21.2.101:0/1113856463 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fc800001980 con 0x7fc8080d1c70 2021-02-10T11:34:28.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc805d9b700 1 -- 172.21.2.101:0/1113856463 >> v1:172.21.2.101:6789/0 conn(0x7fc8080d1c70 legacy=0x7fc8080cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:28.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc805d9b700 1 --2- 172.21.2.101:0/1113856463 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7f4003ac0 0x7fc7f4005ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:28.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc805d9b700 1 --2- 172.21.2.101:0/1113856463 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7f4006770 0x7fc7f4008c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:28.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc805d9b700 1 --2- 172.21.2.101:0/1113856463 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7f4009500 0x7fc7f400ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:28.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc805d9b700 1 -- 172.21.2.101:0/1113856463 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f400c2d0 con 0x7fc7f4009500 2021-02-10T11:34:28.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc805d9b700 1 -- 172.21.2.101:0/1113856463 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f40504c0 con 0x7fc7f4003ac0 2021-02-10T11:34:28.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc805d9b700 1 -- 172.21.2.101:0/1113856463 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f40506e0 con 0x7fc7f4006770 2021-02-10T11:34:28.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc80759e700 1 --2- 172.21.2.101:0/1113856463 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7f4009500 0x7fc7f400ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:28.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc806d9d700 1 --2- 172.21.2.101:0/1113856463 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7f4003ac0 0x7fc7f4005ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:28.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc80659c700 1 --2- 172.21.2.101:0/1113856463 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7f4006770 0x7fc7f4008c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:28.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc80759e700 1 -- 172.21.2.101:0/1113856463 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7f4006770 msgr2=0x7fc7f4008c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:28.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc80759e700 1 --2- 172.21.2.101:0/1113856463 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7f4006770 0x7fc7f4008c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:28.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc80759e700 1 -- 172.21.2.101:0/1113856463 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7f4003ac0 msgr2=0x7fc7f4005ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:28.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc80759e700 1 --2- 172.21.2.101:0/1113856463 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7f4003ac0 0x7fc7f4005ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:28.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc80759e700 1 -- 172.21.2.101:0/1113856463 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f4050980 con 0x7fc7f4009500 2021-02-10T11:34:28.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc80659c700 1 --2- 172.21.2.101:0/1113856463 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7f4006770 0x7fc7f4008c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:34:28.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.304 7fc80759e700 1 --2- 172.21.2.101:0/1113856463 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7f4009500 0x7fc7f400ba20 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7fc800001160 tx=0x7fc80000a310).ready entity=mon.0 client_cookie=d7d9d36a5c2a0a2f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:28.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.305 7fc805d9b700 1 -- 172.21.2.101:0/1113856463 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc800001460 con 0x7fc7f4009500 2021-02-10T11:34:28.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.305 7fc805d9b700 1 -- 172.21.2.101:0/1113856463 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8000070c0 con 0x7fc7f4009500 2021-02-10T11:34:28.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.305 7fc805d9b700 1 -- 172.21.2.101:0/1113856463 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc800001460 con 0x7fc7f4009500 2021-02-10T11:34:28.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.305 7fc80d18d700 1 -- 172.21.2.101:0/1113856463 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7f4009500 msgr2=0x7fc7f400ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:28.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.305 7fc80d18d700 1 --2- 172.21.2.101:0/1113856463 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7f4009500 0x7fc7f400ba20 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7fc800001160 tx=0x7fc80000a310).stop 2021-02-10T11:34:28.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.305 7fc80759e700 1 -- 172.21.2.101:0/1113856463 reap_dead start 2021-02-10T11:34:28.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.305 7fc80d18d700 1 -- 172.21.2.101:0/1113856463 shutdown_connections 2021-02-10T11:34:28.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.305 7fc80d18d700 1 -- 172.21.2.101:0/1113856463 >> 172.21.2.101:0/1113856463 conn(0x7fc8080ce450 msgr2=0x7fc8080d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:28.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.305 7fc80d18d700 1 -- 172.21.2.101:0/1113856463 shutdown_connections 2021-02-10T11:34:28.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.305 7fc80d18d700 1 -- 172.21.2.101:0/1113856463 wait complete. 2021-02-10T11:34:28.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.320 7fc80d18d700 1 Processor -- start 2021-02-10T11:34:28.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.320 7fc80d18d700 1 -- start start 2021-02-10T11:34:28.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.320 7fc80d18d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8080f0280 0x7fc8080a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:28.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.320 7fc80d18d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8080f0680 0x7fc8080a2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:28.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.320 7fc80d18d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8080ec180 0x7fc8080ec580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:28.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.320 7fc80d18d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8080efd00 con 0x7fc8080f0280 2021-02-10T11:34:28.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.320 7fc80d18d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8080ecba0 con 0x7fc8080ec180 2021-02-10T11:34:28.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.320 7fc80d18d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8080ecd80 con 0x7fc8080f0680 2021-02-10T11:34:28.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.321 7fc806d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8080f0280 0x7fc8080a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:28.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.321 7fc806d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8080f0280 0x7fc8080a2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52354/0 (socket says 172.21.2.101:52354) 2021-02-10T11:34:28.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.321 7fc806d9d700 1 -- 172.21.2.101:0/724387338 learned_addr learned my addr 172.21.2.101:0/724387338 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:28.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.321 7fc806d9d700 1 -- 172.21.2.101:0/724387338 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8080f0680 msgr2=0x7fc8080a2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:28.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.321 7fc806d9d700 1 --2- 172.21.2.101:0/724387338 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8080f0680 0x7fc8080a2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:28.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.321 7fc806d9d700 1 -- 172.21.2.101:0/724387338 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8080ec180 msgr2=0x7fc8080ec580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:28.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.321 7fc806d9d700 1 --2- 172.21.2.101:0/724387338 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8080ec180 0x7fc8080ec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:28.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.321 7fc806d9d700 1 -- 172.21.2.101:0/724387338 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8080ecfa0 con 0x7fc8080f0280 2021-02-10T11:34:28.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.321 7fc806d9d700 1 --2- 172.21.2.101:0/724387338 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8080f0280 0x7fc8080a2310 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7fc7f8006950 tx=0x7fc7f800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:28.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.321 7fc7ff7fe700 1 -- 172.21.2.101:0/724387338 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc7f800bd50 con 0x7fc8080f0280 2021-02-10T11:34:28.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.321 7fc7ff7fe700 1 -- 172.21.2.101:0/724387338 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f8000c90 con 0x7fc8080f0280 2021-02-10T11:34:28.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.321 7fc7ff7fe700 1 -- 172.21.2.101:0/724387338 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc7f80022b0 con 0x7fc8080f0280 2021-02-10T11:34:28.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.321 7fc80d18d700 1 -- 172.21.2.101:0/724387338 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8080ecfa0 con 0x7fc8080f0280 2021-02-10T11:34:28.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.322 7fc80d18d700 1 -- 172.21.2.101:0/724387338 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8080ed530 con 0x7fc8080f0280 2021-02-10T11:34:28.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.322 7fc7ff7fe700 1 -- 172.21.2.101:0/724387338 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc7f8009fe0 con 0x7fc8080f0280 2021-02-10T11:34:28.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.322 7fc7ff7fe700 1 --2- 172.21.2.101:0/724387338 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7ec0186c0 0x7fc7ec01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:28.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.322 7fc7ff7fe700 1 -- 172.21.2.101:0/724387338 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fc7f80325d0 con 0x7fc8080f0280 2021-02-10T11:34:28.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.322 7fc80659c700 1 --2- 172.21.2.101:0/724387338 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7ec0186c0 0x7fc7ec01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:28.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.323 7fc80659c700 1 --2- 172.21.2.101:0/724387338 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7ec0186c0 0x7fc7ec01abc0 crc :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:28.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.353 7fc80d18d700 1 -- 172.21.2.101:0/724387338 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8080ebc60 con 0x7fc8080f0280 2021-02-10T11:34:28.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.355 7fc7ff7fe700 1 -- 172.21.2.101:0/724387338 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc7f80249e0 con 0x7fc8080f0280 2021-02-10T11:34:28.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.483 7fc80d18d700 1 -- 172.21.2.101:0/724387338 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8080ebc60 con 0x7fc8080f0280 2021-02-10T11:34:28.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.483 7fc7ff7fe700 1 -- 172.21.2.101:0/724387338 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+1753 (secure 0 0 0) 0x7fc7f80249e0 con 0x7fc8080f0280 2021-02-10T11:34:28.493 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:28.493 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8049,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/172865610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":172865610},{"type":"v1","addr":"172.21.2.101:6835","nonce":172865610}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:33:16.590282","modified":"2021-02-10 11:34:27.853692","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-02-10T11:34:28.493 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 40 2021-02-10T11:34:28.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.485 7fc80d18d700 1 -- 172.21.2.101:0/724387338 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7ec0186c0 msgr2=0x7fc7ec01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.485 7fc80d18d700 1 --2- 172.21.2.101:0/724387338 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7ec0186c0 0x7fc7ec01abc0 crc :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.485 7fc80d18d700 1 -- 172.21.2.101:0/724387338 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8080f0280 msgr2=0x7fc8080a2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.485 7fc80d18d700 1 --2- 172.21.2.101:0/724387338 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8080f0280 0x7fc8080a2310 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7fc7f8006950 tx=0x7fc7f800b960).stop 2021-02-10T11:34:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.485 7fc80d18d700 1 -- 172.21.2.101:0/724387338 shutdown_connections 2021-02-10T11:34:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.485 7fc80d18d700 1 --2- 172.21.2.101:0/724387338 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7ec0186c0 0x7fc7ec01abc0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.485 7fc80d18d700 1 --2- 172.21.2.101:0/724387338 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8080f0680 0x7fc8080a2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.485 7fc80d18d700 1 --2- 172.21.2.101:0/724387338 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8080ec180 0x7fc8080ec580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.485 7fc80d18d700 1 --2- 172.21.2.101:0/724387338 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8080f0280 0x7fc8080a2310 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.485 7fc80d18d700 1 -- 172.21.2.101:0/724387338 >> 172.21.2.101:0/724387338 conn(0x7fc8080c2130 msgr2=0x7fc80809e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.485 7fc80d18d700 1 -- 172.21.2.101:0/724387338 shutdown_connections 2021-02-10T11:34:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.485 7fc80d18d700 1 -- 172.21.2.101:0/724387338 wait complete. 2021-02-10T11:34:28.503 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-02-10T11:34:28.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.909 7f0c88652700 1 Processor -- start 2021-02-10T11:34:28.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.909 7f0c88652700 1 -- start start 2021-02-10T11:34:28.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.909 7f0c88652700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0c800d1c70 0x7f0c800f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:28.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.909 7f0c88652700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c800f0b10 con 0x7f0c800d0340 2021-02-10T11:34:28.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.909 7f0c88652700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c801b0be0 con 0x7f0c800d1870 2021-02-10T11:34:28.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.909 7f0c88652700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c800e5ed0 con 0x7f0c800d1c70 2021-02-10T11:34:28.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.910 7f0c85c6d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f0c800d0340 0x7f0c800ef640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52854/0 (socket says 172.21.2.101:52854) 2021-02-10T11:34:28.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.910 7f0c85c6d700 1 -- 172.21.2.101:0/4274313323 learned_addr learned my addr 172.21.2.101:0/4274313323 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:28.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.910 7f0c8546c700 1 -- 172.21.2.101:0/4274313323 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011282231 0 0) 0x7f0c7c001180 con 0x7f0c800d0340 2021-02-10T11:34:28.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.910 7f0c8546c700 1 -- 172.21.2.101:0/4274313323 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c6c002fe0 con 0x7f0c800d0340 2021-02-10T11:34:28.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.910 7f0c86c6f700 1 --2- 172.21.2.101:0/4274313323 >> v2:172.21.2.108:3300/0 conn(0x7f0c800d1c70 0x7f0c800f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:28.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.910 7f0c8546c700 1 -- 172.21.2.101:0/4274313323 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2473036406 0 0) 0x7f0c7c001180 con 0x7f0c800d0340 2021-02-10T11:34:28.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.910 7f0c8546c700 1 -- 172.21.2.101:0/4274313323 >> v2:172.21.2.108:3300/0 conn(0x7f0c800d1c70 msgr2=0x7f0c800f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:28.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.910 7f0c8546c700 1 --2- 172.21.2.101:0/4274313323 >> v2:172.21.2.108:3300/0 conn(0x7f0c800d1c70 0x7f0c800f20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:28.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.910 7f0c8546c700 1 -- 172.21.2.101:0/4274313323 >> v1:172.21.2.108:6789/0 conn(0x7f0c800d1870 legacy=0x7f0c801b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:28.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.910 7f0c8546c700 1 -- 172.21.2.101:0/4274313323 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c800efe30 con 0x7f0c800d0340 2021-02-10T11:34:28.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.910 7f0c8546c700 1 -- 172.21.2.101:0/4274313323 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f0c7c001680 con 0x7f0c800d0340 2021-02-10T11:34:28.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.910 7f0c8546c700 1 -- 172.21.2.101:0/4274313323 >> v1:172.21.2.101:6789/0 conn(0x7f0c800d0340 legacy=0x7f0c800ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:28.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.910 7f0c8546c700 1 --2- 172.21.2.101:0/4274313323 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c6c003a60 0x7f0c6c005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:28.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.910 7f0c8546c700 1 --2- 172.21.2.101:0/4274313323 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c6c006780 0x7f0c6c008ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:28.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.910 7f0c8546c700 1 --2- 172.21.2.101:0/4274313323 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c6c0095a0 0x7f0c6c00baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:28.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.910 7f0c8546c700 1 -- 172.21.2.101:0/4274313323 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c00c2f0 con 0x7f0c6c006780 2021-02-10T11:34:28.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.910 7f0c8546c700 1 -- 172.21.2.101:0/4274313323 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c00c510 con 0x7f0c6c003a60 2021-02-10T11:34:28.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.910 7f0c8546c700 1 -- 172.21.2.101:0/4274313323 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c00c730 con 0x7f0c6c0095a0 2021-02-10T11:34:28.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.910 7f0c85c6d700 1 --2- 172.21.2.101:0/4274313323 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c6c006780 0x7f0c6c008ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:28.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.911 7f0c8646e700 1 --2- 172.21.2.101:0/4274313323 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c6c003a60 0x7f0c6c005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:28.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.911 7f0c86c6f700 1 --2- 172.21.2.101:0/4274313323 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c6c0095a0 0x7f0c6c00baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.911 7f0c85c6d700 1 -- 172.21.2.101:0/4274313323 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c6c0095a0 msgr2=0x7f0c6c00baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.911 7f0c85c6d700 1 --2- 172.21.2.101:0/4274313323 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c6c0095a0 0x7f0c6c00baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.911 7f0c85c6d700 1 -- 172.21.2.101:0/4274313323 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c6c003a60 msgr2=0x7f0c6c005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.911 7f0c85c6d700 1 --2- 172.21.2.101:0/4274313323 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c6c003a60 0x7f0c6c005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.911 7f0c85c6d700 1 -- 172.21.2.101:0/4274313323 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6c00c9d0 con 0x7f0c6c006780 2021-02-10T11:34:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.911 7f0c86c6f700 1 --2- 172.21.2.101:0/4274313323 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c6c0095a0 0x7f0c6c00baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:34:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.911 7f0c86c6f700 1 -- 172.21.2.101:0/4274313323 reap_dead start 2021-02-10T11:34:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.911 7f0c85c6d700 1 --2- 172.21.2.101:0/4274313323 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c6c006780 0x7f0c6c008ca0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7f0c7c000e80 tx=0x7f0c7c04bd00).ready entity=mon.0 client_cookie=b9f4bf817ec16bde server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.911 7f0c8546c700 1 -- 172.21.2.101:0/4274313323 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0c7c001180 con 0x7f0c6c006780 2021-02-10T11:34:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.911 7f0c8546c700 1 -- 172.21.2.101:0/4274313323 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7c04e310 con 0x7f0c6c006780 2021-02-10T11:34:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.911 7f0c8546c700 1 -- 172.21.2.101:0/4274313323 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0c7c04e6e0 con 0x7f0c6c006780 2021-02-10T11:34:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.911 7f0c88652700 1 -- 172.21.2.101:0/4274313323 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c6c006780 msgr2=0x7f0c6c008ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:28.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.911 7f0c88652700 1 --2- 172.21.2.101:0/4274313323 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c6c006780 0x7f0c6c008ca0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7f0c7c000e80 tx=0x7f0c7c04bd00).stop 2021-02-10T11:34:28.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.912 7f0c88652700 1 -- 172.21.2.101:0/4274313323 shutdown_connections 2021-02-10T11:34:28.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.912 7f0c88652700 1 --2- 172.21.2.101:0/4274313323 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c6c006780 0x7f0c6c008ca0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:28.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.912 7f0c88652700 1 -- 172.21.2.101:0/4274313323 >> 172.21.2.101:0/4274313323 conn(0x7f0c800ce450 msgr2=0x7f0c800d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:28.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.912 7f0c88652700 1 -- 172.21.2.101:0/4274313323 shutdown_connections 2021-02-10T11:34:28.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.912 7f0c88652700 1 -- 172.21.2.101:0/4274313323 wait complete. 2021-02-10T11:34:28.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.927 7f0c88652700 1 Processor -- start 2021-02-10T11:34:28.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.927 7f0c88652700 1 -- start start 2021-02-10T11:34:28.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.927 7f0c88652700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c80094310 0x7f0c800ebe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:28.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.927 7f0c88652700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c80094710 0x7f0c800bca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:28.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.927 7f0c88652700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c800ec490 0x7f0c800bd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:28.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.927 7f0c88652700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c80093d90 con 0x7f0c800ec490 2021-02-10T11:34:28.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.927 7f0c88652700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c800bd670 con 0x7f0c80094710 2021-02-10T11:34:28.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.927 7f0c88652700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c800939f0 con 0x7f0c80094310 2021-02-10T11:34:28.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.927 7f0c86c6f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c800ec490 0x7f0c800bd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:28.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.927 7f0c86c6f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c800ec490 0x7f0c800bd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52374/0 (socket says 172.21.2.101:52374) 2021-02-10T11:34:28.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.927 7f0c86c6f700 1 -- 172.21.2.101:0/3217473773 learned_addr learned my addr 172.21.2.101:0/3217473773 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:28.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.928 7f0c86c6f700 1 -- 172.21.2.101:0/3217473773 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c80094310 msgr2=0x7f0c800ebe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:28.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.928 7f0c86c6f700 1 --2- 172.21.2.101:0/3217473773 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c80094310 0x7f0c800ebe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:28.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.928 7f0c86c6f700 1 -- 172.21.2.101:0/3217473773 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c80094710 msgr2=0x7f0c800bca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:28.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.928 7f0c86c6f700 1 --2- 172.21.2.101:0/3217473773 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c80094710 0x7f0c800bca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:28.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.928 7f0c86c6f700 1 -- 172.21.2.101:0/3217473773 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c800bda80 con 0x7f0c800ec490 2021-02-10T11:34:28.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.928 7f0c86c6f700 1 --2- 172.21.2.101:0/3217473773 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c800ec490 0x7f0c800bd050 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7f0c7800b8c0 tx=0x7f0c7800bc60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:28.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.928 7f0c72ffd700 1 -- 172.21.2.101:0/3217473773 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0c78006030 con 0x7f0c800ec490 2021-02-10T11:34:28.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.928 7f0c72ffd700 1 -- 172.21.2.101:0/3217473773 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c78006370 con 0x7f0c800ec490 2021-02-10T11:34:28.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.928 7f0c88652700 1 -- 172.21.2.101:0/3217473773 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c800bda80 con 0x7f0c800ec490 2021-02-10T11:34:28.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.928 7f0c88652700 1 -- 172.21.2.101:0/3217473773 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c800bda80 con 0x7f0c800ec490 2021-02-10T11:34:28.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.928 7f0c72ffd700 1 -- 172.21.2.101:0/3217473773 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0c780056b0 con 0x7f0c800ec490 2021-02-10T11:34:28.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.929 7f0c72ffd700 1 -- 172.21.2.101:0/3217473773 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0c78006030 con 0x7f0c800ec490 2021-02-10T11:34:28.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.929 7f0c72ffd700 1 --2- 172.21.2.101:0/3217473773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c680186d0 0x7f0c6801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:28.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.929 7f0c72ffd700 1 -- 172.21.2.101:0/3217473773 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f0c78032b90 con 0x7f0c800ec490 2021-02-10T11:34:28.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.929 7f0c8646e700 1 --2- 172.21.2.101:0/3217473773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c680186d0 0x7f0c6801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:28.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.929 7f0c8646e700 1 --2- 172.21.2.101:0/3217473773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c680186d0 0x7f0c6801abd0 crc :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:28.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.959 7f0c88652700 1 -- 172.21.2.101:0/3217473773 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c800ebc40 con 0x7f0c800ec490 2021-02-10T11:34:28.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:28.963 7f0c72ffd700 1 -- 172.21.2.101:0/3217473773 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0c7800bda0 con 0x7f0c800ec490 2021-02-10T11:34:29.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:29.091 7f0c88652700 1 -- 172.21.2.101:0/3217473773 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0c800f0b10 con 0x7f0c800ec490 2021-02-10T11:34:29.317 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:34:29.320 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:34:29.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:29.867 7f0c72ffd700 1 -- 172.21.2.101:0/3217473773 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0c78006030 con 0x7f0c800ec490 2021-02-10T11:34:29.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:29.912 7f0c88652700 1 -- 172.21.2.101:0/3217473773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c680186d0 msgr2=0x7f0c6801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:29.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:29.912 7f0c88652700 1 --2- 172.21.2.101:0/3217473773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c680186d0 0x7f0c6801abd0 crc :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:29.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:29.912 7f0c88652700 1 -- 172.21.2.101:0/3217473773 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c800ec490 msgr2=0x7f0c800bd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:29.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:29.912 7f0c88652700 1 --2- 172.21.2.101:0/3217473773 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c800ec490 0x7f0c800bd050 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7f0c7800b8c0 tx=0x7f0c7800bc60).stop 2021-02-10T11:34:29.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:29.912 7f0c88652700 1 -- 172.21.2.101:0/3217473773 shutdown_connections 2021-02-10T11:34:29.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:29.912 7f0c88652700 1 --2- 172.21.2.101:0/3217473773 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c800ec490 0x7f0c800bd050 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:29.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:29.912 7f0c88652700 1 --2- 172.21.2.101:0/3217473773 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c80094710 0x7f0c800bca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:29.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:29.912 7f0c88652700 1 --2- 172.21.2.101:0/3217473773 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c680186d0 0x7f0c6801abd0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:29.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:29.912 7f0c88652700 1 --2- 172.21.2.101:0/3217473773 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c80094310 0x7f0c800ebe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:29.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:29.912 7f0c88652700 1 -- 172.21.2.101:0/3217473773 >> 172.21.2.101:0/3217473773 conn(0x7f0c8008fa10 msgr2=0x7f0c800bfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:29.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:29.912 7f0c88652700 1 -- 172.21.2.101:0/3217473773 shutdown_connections 2021-02-10T11:34:29.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:29.912 7f0c88652700 1 -- 172.21.2.101:0/3217473773 wait complete. 2021-02-10T11:34:29.931 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-02-10T11:34:30.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.335 7fadfc475700 1 Processor -- start 2021-02-10T11:34:30.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.335 7fadfc475700 1 -- start start 2021-02-10T11:34:30.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.336 7fadfc475700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fadf40d1880 0x7fadf41b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:30.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.336 7fadfc475700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fadf40cff80 0x7fadf4109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:30.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.336 7fadfc475700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf4109a70 con 0x7fadf41b0af0 2021-02-10T11:34:30.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.336 7fadfc475700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf4109760 con 0x7fadf40d1880 2021-02-10T11:34:30.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.336 7fadfc475700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf4088070 con 0x7fadf40cff80 2021-02-10T11:34:30.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.336 7fadfa291700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fadf40d1880 0x7fadf41b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:30.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.336 7fadf9a90700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fadf41b0af0 0x7fadf4108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52874/0 (socket says 172.21.2.101:52874) 2021-02-10T11:34:30.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.336 7fadf9a90700 1 -- 172.21.2.101:0/2875362520 learned_addr learned my addr 172.21.2.101:0/2875362520 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:30.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.336 7fadfa291700 1 -- 172.21.2.101:0/2875362520 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fadf40cff80 msgr2=0x7fadf4109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:30.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.336 7fadfaa92700 1 --2- 172.21.2.101:0/2875362520 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fadf40cff80 0x7fadf4109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:30.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.336 7fadfa291700 1 --2- 172.21.2.101:0/2875362520 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fadf40cff80 0x7fadf4109140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:30.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.336 7fadfa291700 1 -- 172.21.2.101:0/2875362520 >> v1:172.21.2.101:6789/0 conn(0x7fadf41b0af0 legacy=0x7fadf4108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:30.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.336 7fadf928f700 1 -- 172.21.2.101:0/2875362520 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3043174266 0 0) 0x7fade40010b0 con 0x7fadf41b0af0 2021-02-10T11:34:30.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.336 7fadfa291700 1 -- 172.21.2.101:0/2875362520 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf40d1c80 con 0x7fadf40d1880 2021-02-10T11:34:30.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.336 7fadfaa92700 1 --2- 172.21.2.101:0/2875362520 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fadf40cff80 0x7fadf4109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:34:30.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.336 7fadfa291700 1 --2- 172.21.2.101:0/2875362520 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fadf40d1880 0x7fadf41b04d0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0x7fadf0006650 tx=0x7fadf0009a80).ready entity=mon.0 client_cookie=db177bdcd92f561b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:30.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.337 7fadf928f700 1 -- 172.21.2.101:0/2875362520 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fadf00009b0 con 0x7fadf40d1880 2021-02-10T11:34:30.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.337 7fadf928f700 1 -- 172.21.2.101:0/2875362520 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf000c0c0 con 0x7fadf40d1880 2021-02-10T11:34:30.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.337 7fadf928f700 1 -- 172.21.2.101:0/2875362520 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fadf000c5e0 con 0x7fadf40d1880 2021-02-10T11:34:30.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.337 7fadfc475700 1 -- 172.21.2.101:0/2875362520 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fadf40d1880 msgr2=0x7fadf41b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:30.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.337 7fadfc475700 1 --2- 172.21.2.101:0/2875362520 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fadf40d1880 0x7fadf41b04d0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0x7fadf0006650 tx=0x7fadf0009a80).stop 2021-02-10T11:34:30.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.338 7fadfc475700 1 -- 172.21.2.101:0/2875362520 shutdown_connections 2021-02-10T11:34:30.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.338 7fadfc475700 1 --2- 172.21.2.101:0/2875362520 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fadf40cff80 0x7fadf4109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:30.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.338 7fadfc475700 1 --2- 172.21.2.101:0/2875362520 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fadf40d1880 0x7fadf41b04d0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:30.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.338 7fadfc475700 1 -- 172.21.2.101:0/2875362520 >> 172.21.2.101:0/2875362520 conn(0x7fadf40ce430 msgr2=0x7fadf40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:30.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.338 7fadfc475700 1 -- 172.21.2.101:0/2875362520 shutdown_connections 2021-02-10T11:34:30.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.338 7fadfc475700 1 -- 172.21.2.101:0/2875362520 wait complete. 2021-02-10T11:34:30.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.352 7fadfc475700 1 Processor -- start 2021-02-10T11:34:30.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.352 7fadfc475700 1 -- start start 2021-02-10T11:34:30.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.352 7fadfc475700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fadf40ad520 0x7fadf40ad920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:30.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.352 7fadfc475700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fadf40adf40 0x7fadf4097c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:30.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.352 7fadfc475700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fadf4098250 0x7fadf4098650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:30.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.352 7fadfc475700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf4098c70 con 0x7fadf40adf40 2021-02-10T11:34:30.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.352 7fadfc475700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf40910f0 con 0x7fadf4098250 2021-02-10T11:34:30.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.352 7fadfc475700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf4091310 con 0x7fadf40ad520 2021-02-10T11:34:30.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.353 7fadf9a90700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fadf40adf40 0x7fadf4097c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:30.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.353 7fadf9a90700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fadf40adf40 0x7fadf4097c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52388/0 (socket says 172.21.2.101:52388) 2021-02-10T11:34:30.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.353 7fadf9a90700 1 -- 172.21.2.101:0/2734516987 learned_addr learned my addr 172.21.2.101:0/2734516987 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:30.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.353 7fadfa291700 1 --2- 172.21.2.101:0/2734516987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fadf40ad520 0x7fadf40ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:30.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.353 7fadf9a90700 1 -- 172.21.2.101:0/2734516987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fadf40ad520 msgr2=0x7fadf40ad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:30.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.353 7fadf9a90700 1 --2- 172.21.2.101:0/2734516987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fadf40ad520 0x7fadf40ad920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:30.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.353 7fadf9a90700 1 -- 172.21.2.101:0/2734516987 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fadf4098250 msgr2=0x7fadf4098650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:30.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.353 7fadf9a90700 1 --2- 172.21.2.101:0/2734516987 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fadf4098250 0x7fadf4098650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:30.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.353 7fadf9a90700 1 -- 172.21.2.101:0/2734516987 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf419a3b0 con 0x7fadf40adf40 2021-02-10T11:34:30.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.353 7fadf9a90700 1 --2- 172.21.2.101:0/2734516987 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fadf40adf40 0x7fadf4097c30 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7fade4001c10 tx=0x7fade400a870).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:30.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.353 7fadeaffd700 1 -- 172.21.2.101:0/2734516987 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fade40009b0 con 0x7fadf40adf40 2021-02-10T11:34:30.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.353 7fadeaffd700 1 -- 172.21.2.101:0/2734516987 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade400ac60 con 0x7fadf40adf40 2021-02-10T11:34:30.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.353 7fadeaffd700 1 -- 172.21.2.101:0/2734516987 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fade40045d0 con 0x7fadf40adf40 2021-02-10T11:34:30.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.353 7fadfc475700 1 -- 172.21.2.101:0/2734516987 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf419a3b0 con 0x7fadf40adf40 2021-02-10T11:34:30.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.354 7fadfc475700 1 -- 172.21.2.101:0/2734516987 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf419a3b0 con 0x7fadf40adf40 2021-02-10T11:34:30.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.354 7fadeaffd700 1 -- 172.21.2.101:0/2734516987 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fade4004140 con 0x7fadf40adf40 2021-02-10T11:34:30.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.354 7fadeaffd700 1 --2- 172.21.2.101:0/2734516987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faddc0186c0 0x7faddc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:30.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.354 7fadeaffd700 1 -- 172.21.2.101:0/2734516987 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fade40325e0 con 0x7fadf40adf40 2021-02-10T11:34:30.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.354 7fadfa291700 1 --2- 172.21.2.101:0/2734516987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faddc0186c0 0x7faddc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:30.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.355 7fadfa291700 1 --2- 172.21.2.101:0/2734516987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faddc0186c0 0x7faddc01abc0 crc :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:30.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.386 7fadfc475700 1 -- 172.21.2.101:0/2734516987 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf4098c70 con 0x7fadf40adf40 2021-02-10T11:34:30.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.388 7fadeaffd700 1 -- 172.21.2.101:0/2734516987 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fade40267d0 con 0x7fadf40adf40 2021-02-10T11:34:30.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.516 7fadfc475700 1 -- 172.21.2.101:0/2734516987 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fadf4098c70 con 0x7fadf40adf40 2021-02-10T11:34:30.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.870 7fadeaffd700 1 -- 172.21.2.101:0/2734516987 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v57) v1 ==== 185+0+0 (secure 0 0 0) 0x7fade4004140 con 0x7fadf40adf40 2021-02-10T11:34:30.892 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' removed 2021-02-10T11:34:30.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.883 7fadfc475700 1 -- 172.21.2.101:0/2734516987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faddc0186c0 msgr2=0x7faddc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:30.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.883 7fadfc475700 1 --2- 172.21.2.101:0/2734516987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faddc0186c0 0x7faddc01abc0 crc :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:30.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.883 7fadfc475700 1 -- 172.21.2.101:0/2734516987 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fadf40adf40 msgr2=0x7fadf4097c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:30.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.883 7fadfc475700 1 --2- 172.21.2.101:0/2734516987 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fadf40adf40 0x7fadf4097c30 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7fade4001c10 tx=0x7fade400a870).stop 2021-02-10T11:34:30.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.884 7fadfc475700 1 -- 172.21.2.101:0/2734516987 shutdown_connections 2021-02-10T11:34:30.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.884 7fadfc475700 1 --2- 172.21.2.101:0/2734516987 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fadf4098250 0x7fadf4098650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:30.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.884 7fadfc475700 1 --2- 172.21.2.101:0/2734516987 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fadf40adf40 0x7fadf4097c30 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:30.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.884 7fadfc475700 1 --2- 172.21.2.101:0/2734516987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faddc0186c0 0x7faddc01abc0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:30.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.884 7fadfc475700 1 --2- 172.21.2.101:0/2734516987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fadf40ad520 0x7fadf40ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:30.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.884 7fadfc475700 1 -- 172.21.2.101:0/2734516987 >> 172.21.2.101:0/2734516987 conn(0x7fadf418e810 msgr2=0x7fadf418c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:30.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.884 7fadfc475700 1 -- 172.21.2.101:0/2734516987 shutdown_connections 2021-02-10T11:34:30.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:30.884 7fadfc475700 1 -- 172.21.2.101:0/2734516987 wait complete. 2021-02-10T11:34:30.902 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-02-10T11:34:31.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.307 7f0095ae9700 1 Processor -- start 2021-02-10T11:34:31.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.307 7f0095ae9700 1 -- start start 2021-02-10T11:34:31.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.307 7f0095ae9700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f00900d1c80 0x7f00900cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:31.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.307 7f0095ae9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00900cf590 0x7f00900cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:31.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.307 7f0095ae9700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00900fe1d0 con 0x7f00900d1880 2021-02-10T11:34:31.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.307 7f0095ae9700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00901b0cc0 con 0x7f00900d1c80 2021-02-10T11:34:31.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.307 7f0095ae9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0090109a70 con 0x7f00900cf590 2021-02-10T11:34:31.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.308 7f008f7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f00900d1880 0x7f00901b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52886/0 (socket says 172.21.2.101:52886) 2021-02-10T11:34:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.308 7f008f7fe700 1 -- 172.21.2.101:0/2313428222 learned_addr learned my addr 172.21.2.101:0/2313428222 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.308 7f008ffff700 1 --2- 172.21.2.101:0/2313428222 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00900cf590 0x7f00900cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.308 7f0086ffd700 1 --2- 172.21.2.101:0/2313428222 >> v2:172.21.2.108:3300/0 conn(0x7f00900d1c80 0x7f00900cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.308 7f008effd700 1 -- 172.21.2.101:0/2313428222 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2165172570 0 0) 0x7f0080001060 con 0x7f00900d1880 2021-02-10T11:34:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.308 7f008effd700 1 -- 172.21.2.101:0/2313428222 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f007c002fe0 con 0x7f00900d1880 2021-02-10T11:34:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.308 7f008ffff700 1 -- 172.21.2.101:0/2313428222 >> v2:172.21.2.108:3300/0 conn(0x7f00900d1c80 msgr2=0x7f00900cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.308 7f008ffff700 1 --2- 172.21.2.101:0/2313428222 >> v2:172.21.2.108:3300/0 conn(0x7f00900d1c80 0x7f00900cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.308 7f008ffff700 1 -- 172.21.2.101:0/2313428222 >> v1:172.21.2.101:6789/0 conn(0x7f00900d1880 legacy=0x7f00901b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.308 7f008ffff700 1 -- 172.21.2.101:0/2313428222 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0090088070 con 0x7f00900cf590 2021-02-10T11:34:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.308 7f008ffff700 1 --2- 172.21.2.101:0/2313428222 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00900cf590 0x7f00900cf990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f0088006ad0 tx=0x7f0088009ad0).ready entity=mon.2 client_cookie=8648c6458e8135e0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.308 7f008effd700 1 -- 172.21.2.101:0/2313428222 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f00880066a0 con 0x7f00900cf590 2021-02-10T11:34:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.308 7f008effd700 1 -- 172.21.2.101:0/2313428222 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0088006410 con 0x7f00900cf590 2021-02-10T11:34:31.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.308 7f008effd700 1 -- 172.21.2.101:0/2313428222 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f00880024d0 con 0x7f00900cf590 2021-02-10T11:34:31.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.308 7f0095ae9700 1 -- 172.21.2.101:0/2313428222 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00900cf590 msgr2=0x7f00900cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:31.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.308 7f0095ae9700 1 --2- 172.21.2.101:0/2313428222 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00900cf590 0x7f00900cf990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f0088006ad0 tx=0x7f0088009ad0).stop 2021-02-10T11:34:31.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.309 7f0095ae9700 1 -- 172.21.2.101:0/2313428222 shutdown_connections 2021-02-10T11:34:31.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.309 7f0095ae9700 1 --2- 172.21.2.101:0/2313428222 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00900cf590 0x7f00900cf990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:31.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.309 7f0095ae9700 1 --2- 172.21.2.101:0/2313428222 >> v2:172.21.2.108:3300/0 conn(0x7f00900d1c80 0x7f00900cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:31.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.309 7f0095ae9700 1 -- 172.21.2.101:0/2313428222 >> 172.21.2.101:0/2313428222 conn(0x7f00900ce430 msgr2=0x7f00900d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:31.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.309 7f0095ae9700 1 -- 172.21.2.101:0/2313428222 shutdown_connections 2021-02-10T11:34:31.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.309 7f0095ae9700 1 -- 172.21.2.101:0/2313428222 wait complete. 2021-02-10T11:34:31.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.324 7f0095ae9700 1 Processor -- start 2021-02-10T11:34:31.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.324 7f0095ae9700 1 -- start start 2021-02-10T11:34:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.325 7f0095ae9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00901a3570 0x7f00901a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.325 7f0095ae9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f009009c9f0 0x7f009009cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.325 7f0095ae9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f009009d410 0x7f0090098170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.325 7f0095ae9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009009c470 con 0x7f00901a3570 2021-02-10T11:34:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.325 7f0095ae9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00900b2840 con 0x7f009009d410 2021-02-10T11:34:31.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.325 7f0095ae9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00900b2a20 con 0x7f009009c9f0 2021-02-10T11:34:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.325 7f008f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00901a3570 0x7f00901a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.325 7f008f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00901a3570 0x7f00901a3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52398/0 (socket says 172.21.2.101:52398) 2021-02-10T11:34:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.325 7f008f7fe700 1 -- 172.21.2.101:0/1557597871 learned_addr learned my addr 172.21.2.101:0/1557597871 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.325 7f008ffff700 1 --2- 172.21.2.101:0/1557597871 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f009009d410 0x7f0090098170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.325 7f0086ffd700 1 --2- 172.21.2.101:0/1557597871 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f009009c9f0 0x7f009009cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.325 7f008f7fe700 1 -- 172.21.2.101:0/1557597871 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f009009c9f0 msgr2=0x7f009009cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.325 7f008f7fe700 1 --2- 172.21.2.101:0/1557597871 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f009009c9f0 0x7f009009cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.325 7f008f7fe700 1 -- 172.21.2.101:0/1557597871 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f009009d410 msgr2=0x7f0090098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.325 7f008f7fe700 1 --2- 172.21.2.101:0/1557597871 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f009009d410 0x7f0090098170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.325 7f008f7fe700 1 -- 172.21.2.101:0/1557597871 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009009d880 con 0x7f00901a3570 2021-02-10T11:34:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.325 7f008f7fe700 1 --2- 172.21.2.101:0/1557597871 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00901a3570 0x7f00901a3970 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7f0080001c10 tx=0x7f008000aca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.325 7f008cff9700 1 -- 172.21.2.101:0/1557597871 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0080007ce0 con 0x7f00901a3570 2021-02-10T11:34:31.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.325 7f008cff9700 1 -- 172.21.2.101:0/1557597871 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00800070d0 con 0x7f00901a3570 2021-02-10T11:34:31.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.325 7f0095ae9700 1 -- 172.21.2.101:0/1557597871 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f009009d880 con 0x7f00901a3570 2021-02-10T11:34:31.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.325 7f008cff9700 1 -- 172.21.2.101:0/1557597871 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0080004930 con 0x7f00901a3570 2021-02-10T11:34:31.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.326 7f0095ae9700 1 -- 172.21.2.101:0/1557597871 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009009d880 con 0x7f00901a3570 2021-02-10T11:34:31.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.326 7f008cff9700 1 -- 172.21.2.101:0/1557597871 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f00800072b0 con 0x7f00901a3570 2021-02-10T11:34:31.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.326 7f008cff9700 1 --2- 172.21.2.101:0/1557597871 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0074018730 0x7f007401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:31.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.326 7f008cff9700 1 -- 172.21.2.101:0/1557597871 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f0080032840 con 0x7f00901a3570 2021-02-10T11:34:31.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.326 7f0086ffd700 1 --2- 172.21.2.101:0/1557597871 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0074018730 0x7f007401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:31.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.326 7f0086ffd700 1 --2- 172.21.2.101:0/1557597871 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0074018730 0x7f007401ac10 crc :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:31.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.356 7f0095ae9700 1 -- 172.21.2.101:0/1557597871 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0090082a70 con 0x7f00901a3570 2021-02-10T11:34:31.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.359 7f008cff9700 1 -- 172.21.2.101:0/1557597871 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0080007ce0 con 0x7f00901a3570 2021-02-10T11:34:31.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.487 7f0095ae9700 1 -- 172.21.2.101:0/1557597871 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0090082a70 con 0x7f00901a3570 2021-02-10T11:34:31.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.877 7f008cff9700 1 -- 172.21.2.101:0/1557597871 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v58) v1 ==== 173+0+0 (secure 0 0 0) 0x7f00800070d0 con 0x7f00901a3570 2021-02-10T11:34:31.913 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' removed 2021-02-10T11:34:31.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.905 7f0095ae9700 1 -- 172.21.2.101:0/1557597871 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0074018730 msgr2=0x7f007401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:31.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.905 7f0095ae9700 1 --2- 172.21.2.101:0/1557597871 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0074018730 0x7f007401ac10 crc :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:31.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.905 7f0095ae9700 1 -- 172.21.2.101:0/1557597871 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00901a3570 msgr2=0x7f00901a3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:31.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.905 7f0095ae9700 1 --2- 172.21.2.101:0/1557597871 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00901a3570 0x7f00901a3970 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7f0080001c10 tx=0x7f008000aca0).stop 2021-02-10T11:34:31.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.905 7f0095ae9700 1 -- 172.21.2.101:0/1557597871 shutdown_connections 2021-02-10T11:34:31.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.905 7f0095ae9700 1 --2- 172.21.2.101:0/1557597871 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0074018730 0x7f007401ac10 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:31.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.905 7f0095ae9700 1 --2- 172.21.2.101:0/1557597871 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f009009c9f0 0x7f009009cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:31.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.905 7f0095ae9700 1 --2- 172.21.2.101:0/1557597871 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f009009d410 0x7f0090098170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:31.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.905 7f0095ae9700 1 --2- 172.21.2.101:0/1557597871 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00901a3570 0x7f00901a3970 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:31.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.905 7f0095ae9700 1 -- 172.21.2.101:0/1557597871 >> 172.21.2.101:0/1557597871 conn(0x7f00900c2cd0 msgr2=0x7f00900b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:31.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.905 7f0095ae9700 1 -- 172.21.2.101:0/1557597871 shutdown_connections 2021-02-10T11:34:31.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:31.905 7f0095ae9700 1 -- 172.21.2.101:0/1557597871 wait complete. 2021-02-10T11:34:31.924 INFO:tasks.ceph.mds.a:Restarting daemon 2021-02-10T11:34:31.924 INFO:tasks.ceph.mds.a:Stopping old one... 2021-02-10T11:34:31.924 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-02-10T11:34:31.924 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:34:31.924 INFO:tasks.ceph.mds.b:Restarting daemon 2021-02-10T11:34:31.925 INFO:tasks.ceph.mds.b:Stopping old one... 2021-02-10T11:34:31.925 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-02-10T11:34:31.925 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:34:31.930 INFO:tasks.ceph.mds.a:Stopped 2021-02-10T11:34:31.930 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-02-10T11:34:31.931 INFO:tasks.ceph.mds.b:Stopped 2021-02-10T11:34:31.931 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-02-10T11:34:31.972 INFO:tasks.ceph.mds.b:Started 2021-02-10T11:34:31.973 DEBUG:teuthology.parallel:result is None 2021-02-10T11:34:31.973 INFO:tasks.ceph.mds.a:Started 2021-02-10T11:34:31.974 DEBUG:teuthology.parallel:result is None 2021-02-10T11:34:31.974 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd blacklist clear 2021-02-10T11:34:32.376 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:34:32.387 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:34:32.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.380 7f94ed499700 1 Processor -- start 2021-02-10T11:34:32.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.380 7f94ed499700 1 -- start start 2021-02-10T11:34:32.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.381 7f94ed499700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f94e80d1870 0x7f94e81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:32.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.381 7f94ed499700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f94e81b0a10 0x7f94e80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:32.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.381 7f94ed499700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e80f0b10 con 0x7f94e80d1c70 2021-02-10T11:34:32.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.381 7f94ed499700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e80e5ed0 con 0x7f94e81b0a10 2021-02-10T11:34:32.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.381 7f94ed499700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e80f0650 con 0x7f94e80d1870 2021-02-10T11:34:32.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.381 7f94de87c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f94e81b0a10 0x7f94e80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:32.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.381 7f94e787e700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f94e80d1c70 0x7f94e80cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:46324/0 (socket says 172.21.2.101:46324) 2021-02-10T11:34:32.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.381 7f94de87c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f94e81b0a10 0x7f94e80cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52422/0 (socket says 172.21.2.101:52422) 2021-02-10T11:34:32.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.381 7f94de87c700 1 -- 172.21.2.101:0/3751921685 learned_addr learned my addr 172.21.2.101:0/3751921685 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:32.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.381 7f94e707d700 1 --2- 172.21.2.101:0/3751921685 >> v2:172.21.2.108:3300/0 conn(0x7f94e80d1870 0x7f94e81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:32.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.381 7f94e707d700 1 -- 172.21.2.101:0/3751921685 >> v2:172.21.2.101:3300/0 conn(0x7f94e81b0a10 msgr2=0x7f94e80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:32.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.381 7f94e707d700 1 --2- 172.21.2.101:0/3751921685 >> v2:172.21.2.101:3300/0 conn(0x7f94e81b0a10 0x7f94e80cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:32.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.381 7f94e707d700 1 -- 172.21.2.101:0/3751921685 >> v1:172.21.2.108:6789/0 conn(0x7f94e80d1c70 legacy=0x7f94e80cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:32.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.382 7f94e707d700 1 -- 172.21.2.101:0/3751921685 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e80f0840 con 0x7f94e80d1870 2021-02-10T11:34:32.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.381 7f94e687c700 1 -- 172.21.2.101:0/3751921685 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3520654798 0 0) 0x7f94e00009b0 con 0x7f94e80d1c70 2021-02-10T11:34:32.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.382 7f94e707d700 1 --2- 172.21.2.101:0/3751921685 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94e80d1870 0x7f94e81b03f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f94d80069d0 tx=0x7f94d8009dc0).ready entity=mon.1 client_cookie=d0e64742390ac5d8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:32.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.382 7f94e687c700 1 -- 172.21.2.101:0/3751921685 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f94d800b510 con 0x7f94e80d1870 2021-02-10T11:34:32.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.382 7f94e687c700 1 -- 172.21.2.101:0/3751921685 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d8006ac0 con 0x7f94e80d1870 2021-02-10T11:34:32.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.382 7f94ed499700 1 -- 172.21.2.101:0/3751921685 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94e80d1870 msgr2=0x7f94e81b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:32.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.382 7f94ed499700 1 --2- 172.21.2.101:0/3751921685 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94e80d1870 0x7f94e81b03f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f94d80069d0 tx=0x7f94d8009dc0).stop 2021-02-10T11:34:32.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.382 7f94e687c700 1 -- 172.21.2.101:0/3751921685 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f94d8000d30 con 0x7f94e80d1870 2021-02-10T11:34:32.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.382 7f94ed499700 1 -- 172.21.2.101:0/3751921685 shutdown_connections 2021-02-10T11:34:32.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.382 7f94ed499700 1 --2- 172.21.2.101:0/3751921685 >> v2:172.21.2.101:3300/0 conn(0x7f94e81b0a10 0x7f94e80cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:32.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.382 7f94ed499700 1 --2- 172.21.2.101:0/3751921685 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94e80d1870 0x7f94e81b03f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:32.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.382 7f94ed499700 1 -- 172.21.2.101:0/3751921685 >> 172.21.2.101:0/3751921685 conn(0x7f94e80ce450 msgr2=0x7f94e80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:32.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.382 7f94ed499700 1 -- 172.21.2.101:0/3751921685 shutdown_connections 2021-02-10T11:34:32.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.382 7f94ed499700 1 -- 172.21.2.101:0/3751921685 wait complete. 2021-02-10T11:34:32.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.397 7f94ed499700 1 Processor -- start 2021-02-10T11:34:32.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.397 7f94ed499700 1 -- start start 2021-02-10T11:34:32.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94ed499700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94e80b7510 0x7f94e80b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:32.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94ed499700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94e809d530 0x7f94e809d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:32.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94ed499700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94e809df50 0x7f94e809eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:32.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94ed499700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e80b8140 con 0x7f94e809df50 2021-02-10T11:34:32.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94ed499700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e809e750 con 0x7f94e80b7510 2021-02-10T11:34:32.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94ed499700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e809e350 con 0x7f94e809d530 2021-02-10T11:34:32.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94e787e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94e809df50 0x7f94e809eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:32.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94e787e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94e809df50 0x7f94e809eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52424/0 (socket says 172.21.2.101:52424) 2021-02-10T11:34:32.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94e787e700 1 -- 172.21.2.101:0/190919343 learned_addr learned my addr 172.21.2.101:0/190919343 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:32.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94e707d700 1 --2- 172.21.2.101:0/190919343 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94e80b7510 0x7f94e80b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:32.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94e787e700 1 -- 172.21.2.101:0/190919343 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94e809d530 msgr2=0x7f94e809d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:32.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94de87c700 1 --2- 172.21.2.101:0/190919343 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94e809d530 0x7f94e809d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:32.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94e787e700 1 --2- 172.21.2.101:0/190919343 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94e809d530 0x7f94e809d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:32.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94e787e700 1 -- 172.21.2.101:0/190919343 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94e80b7510 msgr2=0x7f94e80b7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:32.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94e787e700 1 --2- 172.21.2.101:0/190919343 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94e80b7510 0x7f94e80b7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:32.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94e787e700 1 -- 172.21.2.101:0/190919343 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e809f2c0 con 0x7f94e809df50 2021-02-10T11:34:32.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94de87c700 1 --2- 172.21.2.101:0/190919343 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94e809d530 0x7f94e809d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:34:32.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94e707d700 1 --2- 172.21.2.101:0/190919343 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94e80b7510 0x7f94e80b7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:34:32.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94e787e700 1 --2- 172.21.2.101:0/190919343 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94e809df50 0x7f94e809eca0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7f94e000a590 tx=0x7f94e000a930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:32.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94e4878700 1 -- 172.21.2.101:0/190919343 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f94e0007560 con 0x7f94e809df50 2021-02-10T11:34:32.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94e4878700 1 -- 172.21.2.101:0/190919343 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e000ad20 con 0x7f94e809df50 2021-02-10T11:34:32.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94ed499700 1 -- 172.21.2.101:0/190919343 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e809f2c0 con 0x7f94e809df50 2021-02-10T11:34:32.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94e4878700 1 -- 172.21.2.101:0/190919343 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f94e0004530 con 0x7f94e809df50 2021-02-10T11:34:32.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.398 7f94ed499700 1 -- 172.21.2.101:0/190919343 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e809f850 con 0x7f94e809df50 2021-02-10T11:34:32.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.399 7f94e4878700 1 -- 172.21.2.101:0/190919343 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f94e0007fe0 con 0x7f94e809df50 2021-02-10T11:34:32.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.399 7f94e4878700 1 --2- 172.21.2.101:0/190919343 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94cc018730 0x7f94cc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:32.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.399 7f94e4878700 1 -- 172.21.2.101:0/190919343 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4839+0+0 (secure 0 0 0) 0x7f94e0032350 con 0x7f94e809df50 2021-02-10T11:34:32.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.399 7f94e707d700 1 --2- 172.21.2.101:0/190919343 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94cc018730 0x7f94cc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:32.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.399 7f94e707d700 1 --2- 172.21.2.101:0/190919343 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94cc018730 0x7f94cc01abc0 crc :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:32.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.429 7f94ed499700 1 -- 172.21.2.101:0/190919343 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e80ebc50 con 0x7f94e809df50 2021-02-10T11:34:32.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.432 7f94e4878700 1 -- 172.21.2.101:0/190919343 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f94e0000e70 con 0x7f94e809df50 2021-02-10T11:34:32.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.572 7f94ed499700 1 -- 172.21.2.101:0/190919343 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f94e80f0b10 con 0x7f94e809df50 2021-02-10T11:34:32.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.920 7f94e4878700 1 -- 172.21.2.101:0/190919343 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v59) v1 ==== 97+0+0 (secure 0 0 0) 0x7f94e0007fe0 con 0x7f94e809df50 2021-02-10T11:34:32.948 INFO:teuthology.orchestra.run.gibba001.stderr: removed all blacklist entries 2021-02-10T11:34:32.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.940 7f94ed499700 1 -- 172.21.2.101:0/190919343 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94cc018730 msgr2=0x7f94cc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:32.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.940 7f94ed499700 1 --2- 172.21.2.101:0/190919343 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94cc018730 0x7f94cc01abc0 crc :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:32.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.940 7f94ed499700 1 -- 172.21.2.101:0/190919343 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94e809df50 msgr2=0x7f94e809eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:32.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.940 7f94ed499700 1 --2- 172.21.2.101:0/190919343 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94e809df50 0x7f94e809eca0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7f94e000a590 tx=0x7f94e000a930).stop 2021-02-10T11:34:32.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.940 7f94ed499700 1 -- 172.21.2.101:0/190919343 shutdown_connections 2021-02-10T11:34:32.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.940 7f94ed499700 1 --2- 172.21.2.101:0/190919343 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94cc018730 0x7f94cc01abc0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:32.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.940 7f94ed499700 1 --2- 172.21.2.101:0/190919343 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94e809d530 0x7f94e809d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:32.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.940 7f94ed499700 1 --2- 172.21.2.101:0/190919343 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94e809df50 0x7f94e809eca0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:32.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.940 7f94ed499700 1 --2- 172.21.2.101:0/190919343 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94e80b7510 0x7f94e80b7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:32.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.940 7f94ed499700 1 -- 172.21.2.101:0/190919343 >> 172.21.2.101:0/190919343 conn(0x7f94e80b83b0 msgr2=0x7f94e80a6850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:32.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.941 7f94ed499700 1 -- 172.21.2.101:0/190919343 shutdown_connections 2021-02-10T11:34:32.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:32.941 7f94ed499700 1 -- 172.21.2.101:0/190919343 wait complete. 2021-02-10T11:34:32.959 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2021-02-10T11:34:32.959 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:34:33.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.359 7fcd1766e700 1 Processor -- start 2021-02-10T11:34:33.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.359 7fcd1766e700 1 -- start start 2021-02-10T11:34:33.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.359 7fcd1766e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fcd100d1c70 0x7fcd100f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:33.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.359 7fcd1766e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd100f0b10 con 0x7fcd100d1870 2021-02-10T11:34:33.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.359 7fcd1766e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd101b0be0 con 0x7fcd100d0340 2021-02-10T11:34:33.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.360 7fcd1766e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd100e5ed0 con 0x7fcd100d1c70 2021-02-10T11:34:33.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.360 7fcd1548a700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fcd100d1870 0x7fcd101b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52926/0 (socket says 172.21.2.101:52926) 2021-02-10T11:34:33.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.360 7fcd1548a700 1 -- 172.21.2.101:0/2148217904 learned_addr learned my addr 172.21.2.101:0/2148217904 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:33.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.360 7fcd07fff700 1 -- 172.21.2.101:0/2148217904 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265512081 0 0) 0x7fcd00001180 con 0x7fcd100d1870 2021-02-10T11:34:33.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.360 7fcd15c8b700 1 --2- 172.21.2.101:0/2148217904 >> v2:172.21.2.108:3300/0 conn(0x7fcd100d1c70 0x7fcd100f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:33.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.360 7fcd07fff700 1 -- 172.21.2.101:0/2148217904 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccfc002fe0 con 0x7fcd100d1870 2021-02-10T11:34:33.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.361 7fcd15c8b700 1 -- 172.21.2.101:0/2148217904 >> v1:172.21.2.108:6789/0 conn(0x7fcd100d0340 legacy=0x7fcd100ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:33.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.361 7fcd15c8b700 1 -- 172.21.2.101:0/2148217904 >> v1:172.21.2.101:6789/0 conn(0x7fcd100d1870 legacy=0x7fcd101b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:33.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.361 7fcd07fff700 1 -- 172.21.2.101:0/2148217904 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3859607365 0 0) 0x7fcd00000ed0 con 0x7fcd100d1870 2021-02-10T11:34:33.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.361 7fcd15c8b700 1 -- 172.21.2.101:0/2148217904 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd100efe30 con 0x7fcd100d1c70 2021-02-10T11:34:33.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.361 7fcd15c8b700 1 --2- 172.21.2.101:0/2148217904 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd100d1c70 0x7fcd100f20a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7fcd08006e50 tx=0x7fcd08009e70).ready entity=mon.1 client_cookie=b485b8989c202bf1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:33.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.361 7fcd07fff700 1 -- 172.21.2.101:0/2148217904 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd08006a20 con 0x7fcd100d1c70 2021-02-10T11:34:33.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.361 7fcd07fff700 1 -- 172.21.2.101:0/2148217904 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd08006790 con 0x7fcd100d1c70 2021-02-10T11:34:33.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.361 7fcd07fff700 1 -- 172.21.2.101:0/2148217904 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd0800b700 con 0x7fcd100d1c70 2021-02-10T11:34:33.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.361 7fcd1766e700 1 -- 172.21.2.101:0/2148217904 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd100d1c70 msgr2=0x7fcd100f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:33.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.361 7fcd1766e700 1 --2- 172.21.2.101:0/2148217904 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd100d1c70 0x7fcd100f20a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7fcd08006e50 tx=0x7fcd08009e70).stop 2021-02-10T11:34:33.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.362 7fcd1766e700 1 -- 172.21.2.101:0/2148217904 shutdown_connections 2021-02-10T11:34:33.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.362 7fcd1766e700 1 --2- 172.21.2.101:0/2148217904 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd100d1c70 0x7fcd100f20a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:33.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.362 7fcd1766e700 1 -- 172.21.2.101:0/2148217904 >> 172.21.2.101:0/2148217904 conn(0x7fcd100ce450 msgr2=0x7fcd100d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:33.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.362 7fcd1766e700 1 -- 172.21.2.101:0/2148217904 shutdown_connections 2021-02-10T11:34:33.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.362 7fcd1766e700 1 -- 172.21.2.101:0/2148217904 wait complete. 2021-02-10T11:34:33.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.377 7fcd1766e700 1 Processor -- start 2021-02-10T11:34:33.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.377 7fcd1766e700 1 -- start start 2021-02-10T11:34:33.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.377 7fcd1766e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd101b7cf0 0x7fcd1019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:33.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.377 7fcd1766e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd101b80f0 0x7fcd101b84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:33.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.377 7fcd1766e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd1019da20 0x7fcd1019de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:33.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.377 7fcd1766e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd100a77c0 con 0x7fcd1019da20 2021-02-10T11:34:33.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.377 7fcd1766e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1019d4a0 con 0x7fcd101b7cf0 2021-02-10T11:34:33.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.377 7fcd1766e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd100954f0 con 0x7fcd101b80f0 2021-02-10T11:34:33.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.377 7fcd15c8b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd1019da20 0x7fcd1019de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:33.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.377 7fcd15c8b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd1019da20 0x7fcd1019de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52442/0 (socket says 172.21.2.101:52442) 2021-02-10T11:34:33.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.377 7fcd15c8b700 1 -- 172.21.2.101:0/398144359 learned_addr learned my addr 172.21.2.101:0/398144359 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:33.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.378 7fcd1548a700 1 --2- 172.21.2.101:0/398144359 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd101b7cf0 0x7fcd1019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:33.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.378 7fcd14c89700 1 --2- 172.21.2.101:0/398144359 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd101b80f0 0x7fcd101b84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:33.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.378 7fcd15c8b700 1 -- 172.21.2.101:0/398144359 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd101b80f0 msgr2=0x7fcd101b84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:33.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.378 7fcd15c8b700 1 --2- 172.21.2.101:0/398144359 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd101b80f0 0x7fcd101b84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:33.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.378 7fcd15c8b700 1 -- 172.21.2.101:0/398144359 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd101b7cf0 msgr2=0x7fcd1019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:33.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.378 7fcd15c8b700 1 --2- 172.21.2.101:0/398144359 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd101b7cf0 0x7fcd1019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:33.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.378 7fcd15c8b700 1 -- 172.21.2.101:0/398144359 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd101b8d60 con 0x7fcd1019da20 2021-02-10T11:34:33.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.378 7fcd14c89700 1 --2- 172.21.2.101:0/398144359 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd101b80f0 0x7fcd101b84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:34:33.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.378 7fcd15c8b700 1 --2- 172.21.2.101:0/398144359 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd1019da20 0x7fcd1019de20 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7fcd08006e60 tx=0x7fcd0800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:33.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.378 7fcd05ffb700 1 -- 172.21.2.101:0/398144359 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd08009130 con 0x7fcd1019da20 2021-02-10T11:34:33.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.379 7fcd05ffb700 1 -- 172.21.2.101:0/398144359 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd08000e80 con 0x7fcd1019da20 2021-02-10T11:34:33.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.379 7fcd05ffb700 1 -- 172.21.2.101:0/398144359 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd080013d0 con 0x7fcd1019da20 2021-02-10T11:34:33.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.379 7fcd1766e700 1 -- 172.21.2.101:0/398144359 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd101b8d60 con 0x7fcd1019da20 2021-02-10T11:34:33.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.379 7fcd1766e700 1 -- 172.21.2.101:0/398144359 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd101b8d60 con 0x7fcd1019da20 2021-02-10T11:34:33.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.379 7fcd05ffb700 1 -- 172.21.2.101:0/398144359 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fcd08009130 con 0x7fcd1019da20 2021-02-10T11:34:33.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.379 7fcd05ffb700 1 --2- 172.21.2.101:0/398144359 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fccf00186c0 0x7fccf001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:33.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.379 7fcd05ffb700 1 -- 172.21.2.101:0/398144359 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fcd080325d0 con 0x7fcd1019da20 2021-02-10T11:34:33.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.380 7fcd1548a700 1 --2- 172.21.2.101:0/398144359 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fccf00186c0 0x7fccf001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:33.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.380 7fcd1548a700 1 --2- 172.21.2.101:0/398144359 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fccf00186c0 0x7fccf001abc0 crc :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:33.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.409 7fcd1766e700 1 -- 172.21.2.101:0/398144359 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd100ebc00 con 0x7fcd1019da20 2021-02-10T11:34:33.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.412 7fcd05ffb700 1 -- 172.21.2.101:0/398144359 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fcd0800be30 con 0x7fcd1019da20 2021-02-10T11:34:33.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.541 7fcd1766e700 1 -- 172.21.2.101:0/398144359 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fcd100f0b10 con 0x7fcd1019da20 2021-02-10T11:34:33.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.542 7fcd05ffb700 1 -- 172.21.2.101:0/398144359 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v37) v1 ==== 80+0+4513 (secure 0 0 0) 0x7fcd08009130 con 0x7fcd1019da20 2021-02-10T11:34:33.552 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:33.552 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:34:33.552 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:34:33.552 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:33.552 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:34:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:34:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:34:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:34:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:34:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:34:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:34:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:33.553 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:34:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:34:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:34:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:34:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:34:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:34:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:34:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:33.554 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:34:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:34:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:34:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:34:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:34:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:34:33.555 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:34:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:34:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:34:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:34:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:34:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:34:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:33.556 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:34:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:34:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:34:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:34:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:34:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:34:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:33.557 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:34:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:34:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:34:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:34:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:34:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:34:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:33.558 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:34:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:34:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:34:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:34:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:34:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:34:33.559 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:34:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:34:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:34:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:34:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:34:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:34:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:34:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:34:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:33.560 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:34:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:34:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:34:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:34:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:34:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:34:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:34:33.561 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:34:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:34:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:34:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:34:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:34:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:34:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:34:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:34:33.562 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:34:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:34:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:34:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:34:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:33.563 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:34:33.564 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:34:33.564 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:34:33.564 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:34:33.564 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:34:33.564 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:34:33.564 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:33.564 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:33.564 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:34:33.564 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:34:33.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.543 7fcd1766e700 1 -- 172.21.2.101:0/398144359 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fccf00186c0 msgr2=0x7fccf001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:33.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.543 7fcd1766e700 1 --2- 172.21.2.101:0/398144359 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fccf00186c0 0x7fccf001abc0 crc :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:33.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.543 7fcd1766e700 1 -- 172.21.2.101:0/398144359 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd1019da20 msgr2=0x7fcd1019de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:33.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.543 7fcd1766e700 1 --2- 172.21.2.101:0/398144359 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd1019da20 0x7fcd1019de20 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7fcd08006e60 tx=0x7fcd0800bcf0).stop 2021-02-10T11:34:33.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.544 7fcd1766e700 1 -- 172.21.2.101:0/398144359 shutdown_connections 2021-02-10T11:34:33.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.544 7fcd1766e700 1 --2- 172.21.2.101:0/398144359 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fccf00186c0 0x7fccf001abc0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:33.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.544 7fcd1766e700 1 --2- 172.21.2.101:0/398144359 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd101b80f0 0x7fcd101b84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:33.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.544 7fcd1766e700 1 --2- 172.21.2.101:0/398144359 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd1019da20 0x7fcd1019de20 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:33.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.544 7fcd1766e700 1 --2- 172.21.2.101:0/398144359 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd101b7cf0 0x7fcd1019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:33.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.544 7fcd1766e700 1 -- 172.21.2.101:0/398144359 >> 172.21.2.101:0/398144359 conn(0x7fcd100ed810 msgr2=0x7fcd10094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:33.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.544 7fcd1766e700 1 -- 172.21.2.101:0/398144359 shutdown_connections 2021-02-10T11:34:33.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.544 7fcd1766e700 1 -- 172.21.2.101:0/398144359 wait complete. 2021-02-10T11:34:33.566 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:34:33.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.968 7fabf31bd700 1 Processor -- start 2021-02-10T11:34:36.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.968 7fabf31bd700 1 -- start start 2021-02-10T11:34:36.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.968 7fabf31bd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabec0d0340 0x7fabec0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:36.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.968 7fabf31bd700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fabec0d1c70 0x7fabec0efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:36.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.968 7fabf31bd700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabec0f0b10 con 0x7fabec0d1870 2021-02-10T11:34:36.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.968 7fabf31bd700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabec0e5ed0 con 0x7fabec0d1c70 2021-02-10T11:34:36.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.968 7fabf31bd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec1b0be0 con 0x7fabec0d0340 2021-02-10T11:34:36.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.969 7fabf0fd9700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fabec0d1870 0x7fabec1b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:46350/0 (socket says 172.21.2.101:46350) 2021-02-10T11:34:36.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.969 7fabf0fd9700 1 -- 172.21.2.101:0/1865033168 learned_addr learned my addr 172.21.2.101:0/1865033168 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:36.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.969 7fabebfff700 1 --2- 172.21.2.101:0/1865033168 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabec0d0340 0x7fabec0ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:36.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.969 7fabf17da700 1 --2- 172.21.2.101:0/1865033168 >> v2:172.21.2.108:3300/0 conn(0x7fabec0d1c70 0x7fabec0efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:36.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.970 7fabf17da700 1 -- 172.21.2.101:0/1865033168 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabec0d0340 msgr2=0x7fabec0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:36.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.970 7fabf17da700 1 --2- 172.21.2.101:0/1865033168 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabec0d0340 0x7fabec0ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:36.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.970 7fabf17da700 1 -- 172.21.2.101:0/1865033168 >> v1:172.21.2.108:6789/0 conn(0x7fabec0d1870 legacy=0x7fabec1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:36.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.970 7fabf17da700 1 -- 172.21.2.101:0/1865033168 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec0f20a0 con 0x7fabec0d1c70 2021-02-10T11:34:36.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.970 7fabebfff700 1 --2- 172.21.2.101:0/1865033168 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabec0d0340 0x7fabec0ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:34:36.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.970 7fabf17da700 1 --2- 172.21.2.101:0/1865033168 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabec0d1c70 0x7fabec0efc60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fabe4006ad0 tx=0x7fabe4009ad0).ready entity=mon.1 client_cookie=5392235c7057ea8f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:36.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.970 7fabeb7fe700 1 -- 172.21.2.101:0/1865033168 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fabe4006370 con 0x7fabec0d1c70 2021-02-10T11:34:36.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.970 7fabeb7fe700 1 -- 172.21.2.101:0/1865033168 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe40022e0 con 0x7fabec0d1c70 2021-02-10T11:34:36.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.970 7fabeb7fe700 1 -- 172.21.2.101:0/1865033168 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fabe4002800 con 0x7fabec0d1c70 2021-02-10T11:34:36.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.970 7fabf31bd700 1 -- 172.21.2.101:0/1865033168 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabec0d1c70 msgr2=0x7fabec0efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:36.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.970 7fabf31bd700 1 --2- 172.21.2.101:0/1865033168 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabec0d1c70 0x7fabec0efc60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fabe4006ad0 tx=0x7fabe4009ad0).stop 2021-02-10T11:34:36.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.971 7fabf31bd700 1 -- 172.21.2.101:0/1865033168 shutdown_connections 2021-02-10T11:34:36.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.971 7fabf31bd700 1 --2- 172.21.2.101:0/1865033168 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabec0d0340 0x7fabec0ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:36.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.971 7fabf31bd700 1 --2- 172.21.2.101:0/1865033168 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabec0d1c70 0x7fabec0efc60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:36.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.971 7fabf31bd700 1 -- 172.21.2.101:0/1865033168 >> 172.21.2.101:0/1865033168 conn(0x7fabec0ce450 msgr2=0x7fabec0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:36.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.971 7fabf31bd700 1 -- 172.21.2.101:0/1865033168 shutdown_connections 2021-02-10T11:34:36.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.971 7fabf31bd700 1 -- 172.21.2.101:0/1865033168 wait complete. 2021-02-10T11:34:36.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.985 7fabf31bd700 1 Processor -- start 2021-02-10T11:34:36.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.985 7fabf31bd700 1 -- start start 2021-02-10T11:34:36.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.986 7fabf31bd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabec18d870 0x7fabec0b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:36.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.986 7fabf31bd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabec0b1fd0 0x7fabec099ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:36.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.986 7fabf31bd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabec0b23d0 0x7fabec09a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:36.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.986 7fabf31bd700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec18dc70 con 0x7fabec18d870 2021-02-10T11:34:36.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.986 7fabf31bd700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec199e60 con 0x7fabec0b1fd0 2021-02-10T11:34:36.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.986 7fabf31bd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec09ab30 con 0x7fabec0b23d0 2021-02-10T11:34:36.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.986 7fabf0fd9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabec18d870 0x7fabec0b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:36.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.986 7fabf0fd9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabec18d870 0x7fabec0b19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52448/0 (socket says 172.21.2.101:52448) 2021-02-10T11:34:36.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.986 7fabf0fd9700 1 -- 172.21.2.101:0/595779826 learned_addr learned my addr 172.21.2.101:0/595779826 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:36.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.987 7fabf0fd9700 1 -- 172.21.2.101:0/595779826 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabec0b23d0 msgr2=0x7fabec09a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:36.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.987 7fabf0fd9700 1 --2- 172.21.2.101:0/595779826 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabec0b23d0 0x7fabec09a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:36.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.987 7fabf0fd9700 1 -- 172.21.2.101:0/595779826 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabec0b1fd0 msgr2=0x7fabec099ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:36.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.987 7fabf0fd9700 1 --2- 172.21.2.101:0/595779826 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabec0b1fd0 0x7fabec099ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:36.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.987 7fabf0fd9700 1 -- 172.21.2.101:0/595779826 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec09ad50 con 0x7fabec18d870 2021-02-10T11:34:36.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.987 7fabf0fd9700 1 --2- 172.21.2.101:0/595779826 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabec18d870 0x7fabec0b19b0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7fabd400a5c0 tx=0x7fabd400a960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:36.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.987 7fabe97fa700 1 -- 172.21.2.101:0/595779826 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fabd400ad50 con 0x7fabec18d870 2021-02-10T11:34:36.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.987 7fabe97fa700 1 -- 172.21.2.101:0/595779826 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd40070d0 con 0x7fabec18d870 2021-02-10T11:34:36.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.987 7fabe97fa700 1 -- 172.21.2.101:0/595779826 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fabd4004510 con 0x7fabec18d870 2021-02-10T11:34:36.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.987 7fabf31bd700 1 -- 172.21.2.101:0/595779826 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabec09ad50 con 0x7fabec18d870 2021-02-10T11:34:36.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.987 7fabf31bd700 1 -- 172.21.2.101:0/595779826 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabec09ad50 con 0x7fabec18d870 2021-02-10T11:34:36.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.988 7fabe97fa700 1 -- 172.21.2.101:0/595779826 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fabd4004320 con 0x7fabec18d870 2021-02-10T11:34:36.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.988 7fabe97fa700 1 --2- 172.21.2.101:0/595779826 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fabd00186c0 0x7fabd001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:36.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.988 7fabe97fa700 1 -- 172.21.2.101:0/595779826 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fabd4032270 con 0x7fabec18d870 2021-02-10T11:34:36.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.988 7fabebfff700 1 --2- 172.21.2.101:0/595779826 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fabd00186c0 0x7fabd001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:36.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:33.989 7fabebfff700 1 --2- 172.21.2.101:0/595779826 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fabd00186c0 0x7fabd001abc0 crc :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:36.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:34.018 7fabf31bd700 1 -- 172.21.2.101:0/595779826 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabec0999c0 con 0x7fabec18d870 2021-02-10T11:34:36.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:34.021 7fabe97fa700 1 -- 172.21.2.101:0/595779826 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fabd4000c10 con 0x7fabec18d870 2021-02-10T11:34:36.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:34.145 7fabf31bd700 1 -- 172.21.2.101:0/595779826 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabec0999c0 con 0x7fabec18d870 2021-02-10T11:34:36.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:34.146 7fabe97fa700 1 -- 172.21.2.101:0/595779826 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+11145 (secure 0 0 0) 0x7fabd4004320 con 0x7fabec18d870 2021-02-10T11:34:36.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:34.148 7fabf31bd700 1 -- 172.21.2.101:0/595779826 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fabd00186c0 msgr2=0x7fabd001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:36.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:34.148 7fabf31bd700 1 --2- 172.21.2.101:0/595779826 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fabd00186c0 0x7fabd001abc0 crc :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:36.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:34.148 7fabf31bd700 1 -- 172.21.2.101:0/595779826 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabec18d870 msgr2=0x7fabec0b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:36.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:34.148 7fabf31bd700 1 --2- 172.21.2.101:0/595779826 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabec18d870 0x7fabec0b19b0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7fabd400a5c0 tx=0x7fabd400a960).stop 2021-02-10T11:34:36.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:34.148 7fabf31bd700 1 -- 172.21.2.101:0/595779826 shutdown_connections 2021-02-10T11:34:36.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:34.148 7fabf31bd700 1 --2- 172.21.2.101:0/595779826 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fabd00186c0 0x7fabd001abc0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:36.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:34.148 7fabf31bd700 1 --2- 172.21.2.101:0/595779826 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabec0b23d0 0x7fabec09a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:36.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:34.148 7fabf31bd700 1 --2- 172.21.2.101:0/595779826 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabec0b1fd0 0x7fabec099ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:36.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:34.148 7fabf31bd700 1 --2- 172.21.2.101:0/595779826 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabec18d870 0x7fabec0b19b0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:36.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:34.148 7fabf31bd700 1 -- 172.21.2.101:0/595779826 >> 172.21.2.101:0/595779826 conn(0x7fabec0f0550 msgr2=0x7fabec191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:36.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:34.148 7fabf31bd700 1 -- 172.21.2.101:0/595779826 shutdown_connections 2021-02-10T11:34:36.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:34.148 7fabf31bd700 1 -- 172.21.2.101:0/595779826 wait complete. 2021-02-10T11:34:36.797 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:36.797 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":59,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:34:32.916300","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:34:36.797 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-10T11:34:36.798 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:34:37.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.203 7fdd52658700 1 Processor -- start 2021-02-10T11:34:37.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.203 7fdd52658700 1 -- start start 2021-02-10T11:34:37.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.204 7fdd52658700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd4c0d1c70 0x7fdd4c0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:37.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.204 7fdd52658700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fdd4c0cf580 0x7fdd4c0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:37.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.204 7fdd52658700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd4c0fe1c0 con 0x7fdd4c0d1870 2021-02-10T11:34:37.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.204 7fdd52658700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd4c1b0cb0 con 0x7fdd4c0cf580 2021-02-10T11:34:37.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.204 7fdd52658700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c109a60 con 0x7fdd4c0d1c70 2021-02-10T11:34:37.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.204 7fdd4bfff700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fdd4c0d1870 0x7fdd4c1b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:46362/0 (socket says 172.21.2.101:46362) 2021-02-10T11:34:37.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.204 7fdd4bfff700 1 -- 172.21.2.101:0/2172651950 learned_addr learned my addr 172.21.2.101:0/2172651950 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:37.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.204 7fdd50c75700 1 --2- 172.21.2.101:0/2172651950 >> v2:172.21.2.108:3300/0 conn(0x7fdd4c0cf580 0x7fdd4c0cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:37.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.204 7fdd4b7fe700 1 --2- 172.21.2.101:0/2172651950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd4c0d1c70 0x7fdd4c0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:37.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.204 7fdd4affd700 1 -- 172.21.2.101:0/2172651950 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3498312472 0 0) 0x7fdd3c0010b0 con 0x7fdd4c0d1870 2021-02-10T11:34:37.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.204 7fdd4affd700 1 -- 172.21.2.101:0/2172651950 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd34002fe0 con 0x7fdd4c0d1870 2021-02-10T11:34:37.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.205 7fdd4b7fe700 1 -- 172.21.2.101:0/2172651950 >> v2:172.21.2.108:3300/0 conn(0x7fdd4c0cf580 msgr2=0x7fdd4c0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:37.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.205 7fdd4b7fe700 1 --2- 172.21.2.101:0/2172651950 >> v2:172.21.2.108:3300/0 conn(0x7fdd4c0cf580 0x7fdd4c0cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:37.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.205 7fdd4b7fe700 1 -- 172.21.2.101:0/2172651950 >> v1:172.21.2.108:6789/0 conn(0x7fdd4c0d1870 legacy=0x7fdd4c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:37.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.205 7fdd4b7fe700 1 -- 172.21.2.101:0/2172651950 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c088070 con 0x7fdd4c0d1c70 2021-02-10T11:34:37.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.205 7fdd4b7fe700 1 --2- 172.21.2.101:0/2172651950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd4c0d1c70 0x7fdd4c0cef60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fdd40006650 tx=0x7fdd40009a80).ready entity=mon.2 client_cookie=babf53a6e31f7005 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:37.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.205 7fdd4affd700 1 -- 172.21.2.101:0/2172651950 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd4000c0c0 con 0x7fdd4c0d1c70 2021-02-10T11:34:37.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.205 7fdd4affd700 1 -- 172.21.2.101:0/2172651950 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4000c340 con 0x7fdd4c0d1c70 2021-02-10T11:34:37.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.205 7fdd4affd700 1 -- 172.21.2.101:0/2172651950 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd4000c860 con 0x7fdd4c0d1c70 2021-02-10T11:34:37.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.205 7fdd52658700 1 -- 172.21.2.101:0/2172651950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd4c0d1c70 msgr2=0x7fdd4c0cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:37.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.205 7fdd52658700 1 --2- 172.21.2.101:0/2172651950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd4c0d1c70 0x7fdd4c0cef60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fdd40006650 tx=0x7fdd40009a80).stop 2021-02-10T11:34:37.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.206 7fdd52658700 1 -- 172.21.2.101:0/2172651950 shutdown_connections 2021-02-10T11:34:37.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.206 7fdd52658700 1 --2- 172.21.2.101:0/2172651950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd4c0d1c70 0x7fdd4c0cef60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:37.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.206 7fdd52658700 1 --2- 172.21.2.101:0/2172651950 >> v2:172.21.2.108:3300/0 conn(0x7fdd4c0cf580 0x7fdd4c0cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:37.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.206 7fdd52658700 1 -- 172.21.2.101:0/2172651950 >> 172.21.2.101:0/2172651950 conn(0x7fdd4c0ce420 msgr2=0x7fdd4c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:37.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.206 7fdd52658700 1 -- 172.21.2.101:0/2172651950 shutdown_connections 2021-02-10T11:34:37.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.206 7fdd52658700 1 -- 172.21.2.101:0/2172651950 wait complete. 2021-02-10T11:34:37.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.221 7fdd52658700 1 Processor -- start 2021-02-10T11:34:37.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.221 7fdd52658700 1 -- start start 2021-02-10T11:34:37.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.221 7fdd52658700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd4c09bcc0 0x7fdd4c09c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:37.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.221 7fdd52658700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd4c0a0820 0x7fdd4c0a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:37.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.221 7fdd52658700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd4c0a1240 0x7fdd4c0858d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:37.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.221 7fdd52658700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c0a66b0 con 0x7fdd4c0a0820 2021-02-10T11:34:37.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.221 7fdd52658700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c1a49b0 con 0x7fdd4c0a1240 2021-02-10T11:34:37.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.221 7fdd52658700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c1a4b90 con 0x7fdd4c09bcc0 2021-02-10T11:34:37.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.221 7fdd4b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd4c0a0820 0x7fdd4c0a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:37.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.221 7fdd4b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd4c0a0820 0x7fdd4c0a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52468/0 (socket says 172.21.2.101:52468) 2021-02-10T11:34:37.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.221 7fdd4b7fe700 1 -- 172.21.2.101:0/2784526863 learned_addr learned my addr 172.21.2.101:0/2784526863 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:37.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.222 7fdd50c75700 1 --2- 172.21.2.101:0/2784526863 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd4c0a1240 0x7fdd4c0858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:37.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.222 7fdd4b7fe700 1 -- 172.21.2.101:0/2784526863 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd4c09bcc0 msgr2=0x7fdd4c09c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:37.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.222 7fdd4b7fe700 1 --2- 172.21.2.101:0/2784526863 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd4c09bcc0 0x7fdd4c09c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:37.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.222 7fdd4b7fe700 1 -- 172.21.2.101:0/2784526863 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd4c0a1240 msgr2=0x7fdd4c0858d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:37.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.222 7fdd4b7fe700 1 --2- 172.21.2.101:0/2784526863 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd4c0a1240 0x7fdd4c0858d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:37.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.222 7fdd4b7fe700 1 -- 172.21.2.101:0/2784526863 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c0a1640 con 0x7fdd4c0a0820 2021-02-10T11:34:37.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.222 7fdd50c75700 1 --2- 172.21.2.101:0/2784526863 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd4c0a1240 0x7fdd4c0858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:34:37.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.222 7fdd4b7fe700 1 --2- 172.21.2.101:0/2784526863 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd4c0a0820 0x7fdd4c0a0c20 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7fdd40009630 tx=0x7fdd4000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:37.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.222 7fdd48ff9700 1 -- 172.21.2.101:0/2784526863 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd4000bd70 con 0x7fdd4c0a0820 2021-02-10T11:34:37.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.222 7fdd48ff9700 1 -- 172.21.2.101:0/2784526863 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd40000960 con 0x7fdd4c0a0820 2021-02-10T11:34:37.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.222 7fdd52658700 1 -- 172.21.2.101:0/2784526863 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd4c0a1640 con 0x7fdd4c0a0820 2021-02-10T11:34:37.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.222 7fdd48ff9700 1 -- 172.21.2.101:0/2784526863 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd40005560 con 0x7fdd4c0a0820 2021-02-10T11:34:37.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.222 7fdd52658700 1 -- 172.21.2.101:0/2784526863 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4c0861e0 con 0x7fdd4c0a0820 2021-02-10T11:34:37.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.222 7fdd48ff9700 1 -- 172.21.2.101:0/2784526863 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdd4000a000 con 0x7fdd4c0a0820 2021-02-10T11:34:37.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.223 7fdd48ff9700 1 --2- 172.21.2.101:0/2784526863 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd2c018730 0x7fdd2c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:37.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.223 7fdd48ff9700 1 -- 172.21.2.101:0/2784526863 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fdd40032340 con 0x7fdd4c0a0820 2021-02-10T11:34:37.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.223 7fdd4bfff700 1 --2- 172.21.2.101:0/2784526863 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd2c018730 0x7fdd2c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:37.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.223 7fdd4bfff700 1 --2- 172.21.2.101:0/2784526863 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd2c018730 0x7fdd2c01ac10 crc :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:37.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.252 7fdd52658700 1 -- 172.21.2.101:0/2784526863 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd4c086a70 con 0x7fdd4c0a0820 2021-02-10T11:34:37.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.255 7fdd48ff9700 1 -- 172.21.2.101:0/2784526863 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdd40000940 con 0x7fdd4c0a0820 2021-02-10T11:34:37.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.382 7fdd52658700 1 -- 172.21.2.101:0/2784526863 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fdd4c086a70 con 0x7fdd4c0a0820 2021-02-10T11:34:37.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.382 7fdd48ff9700 1 -- 172.21.2.101:0/2784526863 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v61) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdd40000940 con 0x7fdd4c0a0820 2021-02-10T11:34:37.392 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:34:37.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.384 7fdd52658700 1 -- 172.21.2.101:0/2784526863 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd2c018730 msgr2=0x7fdd2c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:37.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.384 7fdd52658700 1 --2- 172.21.2.101:0/2784526863 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd2c018730 0x7fdd2c01ac10 crc :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:37.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.384 7fdd52658700 1 -- 172.21.2.101:0/2784526863 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd4c0a0820 msgr2=0x7fdd4c0a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:37.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.384 7fdd52658700 1 --2- 172.21.2.101:0/2784526863 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd4c0a0820 0x7fdd4c0a0c20 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7fdd40009630 tx=0x7fdd4000b980).stop 2021-02-10T11:34:37.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.384 7fdd52658700 1 -- 172.21.2.101:0/2784526863 shutdown_connections 2021-02-10T11:34:37.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.384 7fdd52658700 1 --2- 172.21.2.101:0/2784526863 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd4c0a1240 0x7fdd4c0858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:37.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.384 7fdd52658700 1 --2- 172.21.2.101:0/2784526863 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd4c0a0820 0x7fdd4c0a0c20 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:37.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.384 7fdd52658700 1 --2- 172.21.2.101:0/2784526863 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd2c018730 0x7fdd2c01ac10 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:37.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.384 7fdd52658700 1 --2- 172.21.2.101:0/2784526863 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd4c09bcc0 0x7fdd4c09c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:37.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.384 7fdd52658700 1 -- 172.21.2.101:0/2784526863 >> 172.21.2.101:0/2784526863 conn(0x7fdd4c0bef00 msgr2=0x7fdd4c09d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:37.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.384 7fdd52658700 1 -- 172.21.2.101:0/2784526863 shutdown_connections 2021-02-10T11:34:37.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.384 7fdd52658700 1 -- 172.21.2.101:0/2784526863 wait complete. 2021-02-10T11:34:37.403 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --format=json-pretty osd lspools 2021-02-10T11:34:37.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.803 7fc88d212700 1 Processor -- start 2021-02-10T11:34:37.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.803 7fc88d212700 1 -- start start 2021-02-10T11:34:37.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.804 7fc88d212700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc8880d0340 0x7fc8880ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:37.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.804 7fc88d212700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8880f0b10 con 0x7fc8880d1870 2021-02-10T11:34:37.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.804 7fc88d212700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8881b0be0 con 0x7fc8880d1c70 2021-02-10T11:34:37.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.804 7fc88d212700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8880e5ed0 con 0x7fc8880d0340 2021-02-10T11:34:37.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.804 7fc886d9d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc8880d1870 0x7fc8881b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52966/0 (socket says 172.21.2.101:52966) 2021-02-10T11:34:37.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.804 7fc886d9d700 1 -- 172.21.2.101:0/1134973198 learned_addr learned my addr 172.21.2.101:0/1134973198 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:37.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.804 7fc885d9b700 1 -- 172.21.2.101:0/1134973198 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 125428602 0 0) 0x7fc870001180 con 0x7fc8880d1870 2021-02-10T11:34:37.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.804 7fc885d9b700 1 -- 172.21.2.101:0/1134973198 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc874002fe0 con 0x7fc8880d1870 2021-02-10T11:34:37.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.805 7fc88659c700 1 --2- 172.21.2.101:0/1134973198 >> v2:172.21.2.108:3300/0 conn(0x7fc8880d0340 0x7fc8880ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:37.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.805 7fc885d9b700 1 -- 172.21.2.101:0/1134973198 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2476687282 0 0) 0x7fc870001180 con 0x7fc8880d1870 2021-02-10T11:34:37.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.805 7fc885d9b700 1 -- 172.21.2.101:0/1134973198 >> v2:172.21.2.108:3300/0 conn(0x7fc8880d0340 msgr2=0x7fc8880ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:37.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.805 7fc885d9b700 1 --2- 172.21.2.101:0/1134973198 >> v2:172.21.2.108:3300/0 conn(0x7fc8880d0340 0x7fc8880ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:37.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.805 7fc885d9b700 1 -- 172.21.2.101:0/1134973198 >> v1:172.21.2.108:6789/0 conn(0x7fc8880d1c70 legacy=0x7fc8880efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:37.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.805 7fc885d9b700 1 -- 172.21.2.101:0/1134973198 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8880f0450 con 0x7fc8880d1870 2021-02-10T11:34:37.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.805 7fc885d9b700 1 -- 172.21.2.101:0/1134973198 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fc870001680 con 0x7fc8880d1870 2021-02-10T11:34:37.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.805 7fc885d9b700 1 -- 172.21.2.101:0/1134973198 >> v1:172.21.2.101:6789/0 conn(0x7fc8880d1870 legacy=0x7fc8881b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:37.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.805 7fc885d9b700 1 --2- 172.21.2.101:0/1134973198 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc874003af0 0x7fc874005f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:37.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.805 7fc885d9b700 1 --2- 172.21.2.101:0/1134973198 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc874006780 0x7fc874008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:37.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.805 7fc885d9b700 1 --2- 172.21.2.101:0/1134973198 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc874009580 0x7fc87400baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:37.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.805 7fc885d9b700 1 -- 172.21.2.101:0/1134973198 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87400c2f0 con 0x7fc874003af0 2021-02-10T11:34:37.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.805 7fc885d9b700 1 -- 172.21.2.101:0/1134973198 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87400c510 con 0x7fc874006780 2021-02-10T11:34:37.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.805 7fc885d9b700 1 -- 172.21.2.101:0/1134973198 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87400c730 con 0x7fc874009580 2021-02-10T11:34:37.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.805 7fc886d9d700 1 --2- 172.21.2.101:0/1134973198 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc874003af0 0x7fc874005f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:37.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.805 7fc88659c700 1 --2- 172.21.2.101:0/1134973198 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc874006780 0x7fc874008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:37.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.805 7fc886d9d700 1 -- 172.21.2.101:0/1134973198 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc874009580 msgr2=0x7fc87400baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:37.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.805 7fc886d9d700 1 --2- 172.21.2.101:0/1134973198 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc874009580 0x7fc87400baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:37.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.805 7fc886d9d700 1 -- 172.21.2.101:0/1134973198 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc874006780 msgr2=0x7fc874008c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:37.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.805 7fc886d9d700 1 --2- 172.21.2.101:0/1134973198 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc874006780 0x7fc874008c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:37.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.805 7fc886d9d700 1 -- 172.21.2.101:0/1134973198 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8740509a0 con 0x7fc874003af0 2021-02-10T11:34:37.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.806 7fc88759e700 1 -- 172.21.2.101:0/1134973198 reap_dead start 2021-02-10T11:34:37.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.806 7fc886d9d700 1 --2- 172.21.2.101:0/1134973198 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc874003af0 0x7fc874005f00 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7fc870000eb0 tx=0x7fc870007d00).ready entity=mon.0 client_cookie=5a74b4735e14a745 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:37.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.806 7fc885d9b700 1 -- 172.21.2.101:0/1134973198 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc870001180 con 0x7fc874003af0 2021-02-10T11:34:37.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.806 7fc885d9b700 1 -- 172.21.2.101:0/1134973198 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87000a520 con 0x7fc874003af0 2021-02-10T11:34:37.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.806 7fc885d9b700 1 -- 172.21.2.101:0/1134973198 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc8700070c0 con 0x7fc874003af0 2021-02-10T11:34:37.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.806 7fc88d212700 1 -- 172.21.2.101:0/1134973198 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc874003af0 msgr2=0x7fc874005f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:37.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.806 7fc88d212700 1 --2- 172.21.2.101:0/1134973198 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc874003af0 0x7fc874005f00 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7fc870000eb0 tx=0x7fc870007d00).stop 2021-02-10T11:34:37.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.806 7fc88d212700 1 -- 172.21.2.101:0/1134973198 shutdown_connections 2021-02-10T11:34:37.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.806 7fc88d212700 1 --2- 172.21.2.101:0/1134973198 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc874003af0 0x7fc874005f00 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:37.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.806 7fc88d212700 1 -- 172.21.2.101:0/1134973198 >> 172.21.2.101:0/1134973198 conn(0x7fc8880ce450 msgr2=0x7fc8880d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:37.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.806 7fc88d212700 1 -- 172.21.2.101:0/1134973198 shutdown_connections 2021-02-10T11:34:37.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.806 7fc88d212700 1 -- 172.21.2.101:0/1134973198 wait complete. 2021-02-10T11:34:37.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.821 7fc88d212700 1 Processor -- start 2021-02-10T11:34:37.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.821 7fc88d212700 1 -- start start 2021-02-10T11:34:37.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.821 7fc88d212700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc888195650 0x7fc8880b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:37.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.821 7fc88d212700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8880b5b80 0x7fc8881a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:37.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.821 7fc88d212700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8880acfa0 0x7fc8881a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:37.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.821 7fc88d212700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8880b6010 con 0x7fc888195650 2021-02-10T11:34:37.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.821 7fc88d212700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8881a3a60 con 0x7fc8880acfa0 2021-02-10T11:34:37.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.821 7fc88d212700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8881a3c40 con 0x7fc8880b5b80 2021-02-10T11:34:37.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.822 7fc886d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc888195650 0x7fc8880b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:37.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.822 7fc886d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc888195650 0x7fc8880b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52484/0 (socket says 172.21.2.101:52484) 2021-02-10T11:34:37.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.822 7fc886d9d700 1 -- 172.21.2.101:0/1086507079 learned_addr learned my addr 172.21.2.101:0/1086507079 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:37.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.822 7fc886d9d700 1 -- 172.21.2.101:0/1086507079 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8880b5b80 msgr2=0x7fc8881a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:37.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.822 7fc886d9d700 1 --2- 172.21.2.101:0/1086507079 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8880b5b80 0x7fc8881a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:37.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.822 7fc886d9d700 1 -- 172.21.2.101:0/1086507079 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8880acfa0 msgr2=0x7fc8881a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:37.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.822 7fc886d9d700 1 --2- 172.21.2.101:0/1086507079 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8880acfa0 0x7fc8881a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:37.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.822 7fc886d9d700 1 -- 172.21.2.101:0/1086507079 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8880bcae0 con 0x7fc888195650 2021-02-10T11:34:37.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.822 7fc886d9d700 1 --2- 172.21.2.101:0/1086507079 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc888195650 0x7fc8880b5560 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7fc8700055e0 tx=0x7fc870005980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:37.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.822 7fc87f7fe700 1 -- 172.21.2.101:0/1086507079 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc870007970 con 0x7fc888195650 2021-02-10T11:34:37.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.822 7fc87f7fe700 1 -- 172.21.2.101:0/1086507079 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc870007bf0 con 0x7fc888195650 2021-02-10T11:34:37.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.822 7fc88d212700 1 -- 172.21.2.101:0/1086507079 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8880bcae0 con 0x7fc888195650 2021-02-10T11:34:37.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.822 7fc88d212700 1 -- 172.21.2.101:0/1086507079 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8880bd070 con 0x7fc888195650 2021-02-10T11:34:37.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.822 7fc87f7fe700 1 -- 172.21.2.101:0/1086507079 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc87000a350 con 0x7fc888195650 2021-02-10T11:34:37.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.823 7fc87f7fe700 1 -- 172.21.2.101:0/1086507079 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc8700043a0 con 0x7fc888195650 2021-02-10T11:34:37.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.823 7fc87f7fe700 1 --2- 172.21.2.101:0/1086507079 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc86c018730 0x7fc86c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:37.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.823 7fc87f7fe700 1 -- 172.21.2.101:0/1086507079 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fc870032470 con 0x7fc888195650 2021-02-10T11:34:37.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.823 7fc88659c700 1 --2- 172.21.2.101:0/1086507079 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc86c018730 0x7fc86c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:37.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.824 7fc88659c700 1 --2- 172.21.2.101:0/1086507079 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc86c018730 0x7fc86c01ac10 crc :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:37.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.854 7fc88d212700 1 -- 172.21.2.101:0/1086507079 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8880b6010 con 0x7fc888195650 2021-02-10T11:34:37.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.856 7fc87f7fe700 1 -- 172.21.2.101:0/1086507079 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc870001440 con 0x7fc888195650 2021-02-10T11:34:37.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.981 7fc88d212700 1 -- 172.21.2.101:0/1086507079 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc8880b6010 con 0x7fc888195650 2021-02-10T11:34:37.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.982 7fc87f7fe700 1 -- 172.21.2.101:0/1086507079 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v61) v1 ==== 84+0+64 (secure 0 0 0) 0x7fc870007bf0 con 0x7fc888195650 2021-02-10T11:34:37.992 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:37.992 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:34:37.992 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:34:37.992 INFO:teuthology.orchestra.run.gibba001.stdout: "poolnum": 1, 2021-02-10T11:34:37.992 INFO:teuthology.orchestra.run.gibba001.stdout: "poolname": "rbd" 2021-02-10T11:34:37.992 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:37.992 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:34:37.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.984 7fc88d212700 1 -- 172.21.2.101:0/1086507079 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc86c018730 msgr2=0x7fc86c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:37.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.984 7fc88d212700 1 --2- 172.21.2.101:0/1086507079 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc86c018730 0x7fc86c01ac10 crc :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:37.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.984 7fc88d212700 1 -- 172.21.2.101:0/1086507079 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc888195650 msgr2=0x7fc8880b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:37.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.984 7fc88d212700 1 --2- 172.21.2.101:0/1086507079 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc888195650 0x7fc8880b5560 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7fc8700055e0 tx=0x7fc870005980).stop 2021-02-10T11:34:37.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.984 7fc88d212700 1 -- 172.21.2.101:0/1086507079 shutdown_connections 2021-02-10T11:34:37.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.984 7fc88d212700 1 --2- 172.21.2.101:0/1086507079 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc86c018730 0x7fc86c01ac10 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:37.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.984 7fc88d212700 1 --2- 172.21.2.101:0/1086507079 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8880b5b80 0x7fc8881a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:37.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.984 7fc88d212700 1 --2- 172.21.2.101:0/1086507079 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8880acfa0 0x7fc8881a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:37.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.984 7fc88d212700 1 --2- 172.21.2.101:0/1086507079 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc888195650 0x7fc8880b5560 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:37.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.984 7fc88d212700 1 -- 172.21.2.101:0/1086507079 >> 172.21.2.101:0/1086507079 conn(0x7fc8880b3040 msgr2=0x7fc8880c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:37.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.984 7fc88d212700 1 -- 172.21.2.101:0/1086507079 shutdown_connections 2021-02-10T11:34:37.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:37.984 7fc88d212700 1 -- 172.21.2.101:0/1086507079 wait complete. 2021-02-10T11:34:38.002 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-02-10T11:34:38.003 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-02-10T11:34:38.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.407 7fe142065700 1 Processor -- start 2021-02-10T11:34:38.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.407 7fe142065700 1 -- start start 2021-02-10T11:34:38.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.407 7fe142065700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe13c0d1c70 0x7fe13c0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:38.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.407 7fe142065700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe13c0d0170 con 0x7fe13c0cf580 2021-02-10T11:34:38.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.407 7fe142065700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe13c0fe1c0 con 0x7fe13c0d1870 2021-02-10T11:34:38.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.407 7fe142065700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe13c1b0cb0 con 0x7fe13c0d1c70 2021-02-10T11:34:38.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.408 7fe13bfff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe13c0cf580 0x7fe13c0cf980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:52990/0 (socket says 172.21.2.101:52990) 2021-02-10T11:34:38.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.408 7fe13bfff700 1 -- 172.21.2.101:0/3419809724 learned_addr learned my addr 172.21.2.101:0/3419809724 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:38.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.408 7fe13a7fc700 1 -- 172.21.2.101:0/3419809724 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4268759151 0 0) 0x7fe134001500 con 0x7fe13c0cf580 2021-02-10T11:34:38.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.408 7fe13affd700 1 --2- 172.21.2.101:0/3419809724 >> v2:172.21.2.108:3300/0 conn(0x7fe13c0d1c70 0x7fe13c0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:38.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.408 7fe13a7fc700 1 -- 172.21.2.101:0/3419809724 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe128002fe0 con 0x7fe13c0cf580 2021-02-10T11:34:38.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.408 7fe13a7fc700 1 -- 172.21.2.101:0/3419809724 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 84970936 0 0) 0x7fe134001460 con 0x7fe13c0cf580 2021-02-10T11:34:38.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.408 7fe13a7fc700 1 -- 172.21.2.101:0/3419809724 >> v2:172.21.2.108:3300/0 conn(0x7fe13c0d1c70 msgr2=0x7fe13c0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:38.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.408 7fe13a7fc700 1 --2- 172.21.2.101:0/3419809724 >> v2:172.21.2.108:3300/0 conn(0x7fe13c0d1c70 0x7fe13c0cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:38.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.408 7fe13a7fc700 1 -- 172.21.2.101:0/3419809724 >> v1:172.21.2.108:6789/0 conn(0x7fe13c0d1870 legacy=0x7fe13c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:38.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.409 7fe13a7fc700 1 -- 172.21.2.101:0/3419809724 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13c109a60 con 0x7fe13c0cf580 2021-02-10T11:34:38.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.409 7fe13a7fc700 1 -- 172.21.2.101:0/3419809724 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fe134001980 con 0x7fe13c0cf580 2021-02-10T11:34:38.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.409 7fe13a7fc700 1 -- 172.21.2.101:0/3419809724 >> v1:172.21.2.101:6789/0 conn(0x7fe13c0cf580 legacy=0x7fe13c0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:38.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.409 7fe13a7fc700 1 --2- 172.21.2.101:0/3419809724 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe128003a40 0x7fe128005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:38.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.409 7fe13a7fc700 1 --2- 172.21.2.101:0/3419809724 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe128006770 0x7fe128008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:38.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.409 7fe13a7fc700 1 --2- 172.21.2.101:0/3419809724 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe128009590 0x7fe12800bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:38.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.409 7fe13a7fc700 1 -- 172.21.2.101:0/3419809724 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12800c300 con 0x7fe128009590 2021-02-10T11:34:38.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.409 7fe13a7fc700 1 -- 172.21.2.101:0/3419809724 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12800c520 con 0x7fe128003a40 2021-02-10T11:34:38.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.409 7fe13a7fc700 1 -- 172.21.2.101:0/3419809724 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12800c740 con 0x7fe128006770 2021-02-10T11:34:38.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.409 7fe13bfff700 1 --2- 172.21.2.101:0/3419809724 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe128009590 0x7fe12800bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:38.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.409 7fe13b7fe700 1 --2- 172.21.2.101:0/3419809724 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe128003a40 0x7fe128005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:38.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.409 7fe13affd700 1 --2- 172.21.2.101:0/3419809724 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe128006770 0x7fe128008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:38.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.409 7fe13bfff700 1 -- 172.21.2.101:0/3419809724 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe128006770 msgr2=0x7fe128008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:38.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.409 7fe13bfff700 1 --2- 172.21.2.101:0/3419809724 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe128006770 0x7fe128008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:38.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.409 7fe13bfff700 1 -- 172.21.2.101:0/3419809724 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe128003a40 msgr2=0x7fe128005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:38.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.409 7fe13bfff700 1 --2- 172.21.2.101:0/3419809724 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe128003a40 0x7fe128005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:38.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.409 7fe13bfff700 1 -- 172.21.2.101:0/3419809724 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe128050980 con 0x7fe128009590 2021-02-10T11:34:38.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.409 7fe13bfff700 1 -- 172.21.2.101:0/3419809724 reap_dead start 2021-02-10T11:34:38.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.409 7fe13bfff700 1 --2- 172.21.2.101:0/3419809724 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe128009590 0x7fe12800bab0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7fe134001160 tx=0x7fe13400a310).ready entity=mon.0 client_cookie=9b428e066feccc2e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:38.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.409 7fe13a7fc700 1 -- 172.21.2.101:0/3419809724 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe1340072b0 con 0x7fe128009590 2021-02-10T11:34:38.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.409 7fe13a7fc700 1 -- 172.21.2.101:0/3419809724 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe134004720 con 0x7fe128009590 2021-02-10T11:34:38.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.410 7fe13a7fc700 1 -- 172.21.2.101:0/3419809724 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe1340072b0 con 0x7fe128009590 2021-02-10T11:34:38.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.410 7fe142065700 1 -- 172.21.2.101:0/3419809724 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe128009590 msgr2=0x7fe12800bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:38.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.410 7fe142065700 1 --2- 172.21.2.101:0/3419809724 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe128009590 0x7fe12800bab0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7fe134001160 tx=0x7fe13400a310).stop 2021-02-10T11:34:38.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.410 7fe142065700 1 -- 172.21.2.101:0/3419809724 shutdown_connections 2021-02-10T11:34:38.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.410 7fe142065700 1 --2- 172.21.2.101:0/3419809724 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe128009590 0x7fe12800bab0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:38.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.410 7fe142065700 1 -- 172.21.2.101:0/3419809724 >> 172.21.2.101:0/3419809724 conn(0x7fe13c0ce420 msgr2=0x7fe13c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:38.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.410 7fe142065700 1 -- 172.21.2.101:0/3419809724 shutdown_connections 2021-02-10T11:34:38.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.410 7fe142065700 1 -- 172.21.2.101:0/3419809724 wait complete. 2021-02-10T11:34:38.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.425 7fe142065700 1 Processor -- start 2021-02-10T11:34:38.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.425 7fe142065700 1 -- start start 2021-02-10T11:34:38.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.425 7fe142065700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe13c0ac2c0 0x7fe13c1a7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:38.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.425 7fe142065700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe13c0bfb60 0x7fe13c1a7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:38.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.425 7fe142065700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe13c083980 0x7fe13c083d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:38.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.425 7fe142065700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c0bf740 con 0x7fe13c0bfb60 2021-02-10T11:34:38.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.425 7fe142065700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c1a80c0 con 0x7fe13c083980 2021-02-10T11:34:38.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.425 7fe142065700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c1a82e0 con 0x7fe13c0ac2c0 2021-02-10T11:34:38.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.425 7fe13affd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe13c0bfb60 0x7fe13c1a7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:38.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.425 7fe13affd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe13c0bfb60 0x7fe13c1a7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52508/0 (socket says 172.21.2.101:52508) 2021-02-10T11:34:38.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.425 7fe13affd700 1 -- 172.21.2.101:0/3801589685 learned_addr learned my addr 172.21.2.101:0/3801589685 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:38.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.426 7fe13affd700 1 -- 172.21.2.101:0/3801589685 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe13c0ac2c0 msgr2=0x7fe13c1a7480 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:34:38.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.426 7fe13affd700 1 --2- 172.21.2.101:0/3801589685 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe13c0ac2c0 0x7fe13c1a7480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:38.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.426 7fe13affd700 1 -- 172.21.2.101:0/3801589685 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe13c083980 msgr2=0x7fe13c083d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:38.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.426 7fe13affd700 1 --2- 172.21.2.101:0/3801589685 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe13c083980 0x7fe13c083d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:38.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.426 7fe13affd700 1 -- 172.21.2.101:0/3801589685 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13c0843a0 con 0x7fe13c0bfb60 2021-02-10T11:34:38.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.426 7fe13affd700 1 --2- 172.21.2.101:0/3801589685 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe13c0bfb60 0x7fe13c1a7aa0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 rx=0x7fe12c00b560 tx=0x7fe12c00b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:38.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.426 7fe133fff700 1 -- 172.21.2.101:0/3801589685 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe12c00bcf0 con 0x7fe13c0bfb60 2021-02-10T11:34:38.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.426 7fe133fff700 1 -- 172.21.2.101:0/3801589685 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12c000c90 con 0x7fe13c0bfb60 2021-02-10T11:34:38.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.426 7fe133fff700 1 -- 172.21.2.101:0/3801589685 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe12c002220 con 0x7fe13c0bfb60 2021-02-10T11:34:38.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.426 7fe142065700 1 -- 172.21.2.101:0/3801589685 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13c0843a0 con 0x7fe13c0bfb60 2021-02-10T11:34:38.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.426 7fe142065700 1 -- 172.21.2.101:0/3801589685 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13c0843a0 con 0x7fe13c0bfb60 2021-02-10T11:34:38.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.427 7fe133fff700 1 -- 172.21.2.101:0/3801589685 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe12c006b00 con 0x7fe13c0bfb60 2021-02-10T11:34:38.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.427 7fe133fff700 1 --2- 172.21.2.101:0/3801589685 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe120018730 0x7fe12001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:38.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.427 7fe133fff700 1 -- 172.21.2.101:0/3801589685 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fe12c032340 con 0x7fe13c0bfb60 2021-02-10T11:34:38.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.427 7fe13b7fe700 1 --2- 172.21.2.101:0/3801589685 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe120018730 0x7fe12001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:38.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.427 7fe13b7fe700 1 --2- 172.21.2.101:0/3801589685 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe120018730 0x7fe12001abc0 crc :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:38.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.458 7fe142065700 1 -- 172.21.2.101:0/3801589685 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe13c082a70 con 0x7fe13c0bfb60 2021-02-10T11:34:38.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.460 7fe133fff700 1 -- 172.21.2.101:0/3801589685 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe12c0063a0 con 0x7fe13c0bfb60 2021-02-10T11:34:38.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:38.590 7fe142065700 1 -- 172.21.2.101:0/3801589685 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"} v 0) v1 -- 0x7fe13c082a70 con 0x7fe13c0bfb60 2021-02-10T11:34:39.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.440 7fe133fff700 1 -- 172.21.2.101:0/3801589685 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"}]=0 pool 'cephfs_metadata' created v62) v1 ==== 133+0+0 (secure 0 0 0) 0x7fe12c006b00 con 0x7fe13c0bfb60 2021-02-10T11:34:39.469 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' created 2021-02-10T11:34:39.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.461 7fe142065700 1 -- 172.21.2.101:0/3801589685 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe120018730 msgr2=0x7fe12001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:39.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.461 7fe142065700 1 --2- 172.21.2.101:0/3801589685 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe120018730 0x7fe12001abc0 crc :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:39.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.461 7fe142065700 1 -- 172.21.2.101:0/3801589685 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe13c0bfb60 msgr2=0x7fe13c1a7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:39.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.461 7fe142065700 1 --2- 172.21.2.101:0/3801589685 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe13c0bfb60 0x7fe13c1a7aa0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 rx=0x7fe12c00b560 tx=0x7fe12c00b900).stop 2021-02-10T11:34:39.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.461 7fe142065700 1 -- 172.21.2.101:0/3801589685 shutdown_connections 2021-02-10T11:34:39.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.461 7fe142065700 1 --2- 172.21.2.101:0/3801589685 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe13c083980 0x7fe13c083d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:39.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.461 7fe142065700 1 --2- 172.21.2.101:0/3801589685 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe13c0bfb60 0x7fe13c1a7aa0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:39.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.461 7fe142065700 1 --2- 172.21.2.101:0/3801589685 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe120018730 0x7fe12001abc0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:39.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.461 7fe142065700 1 --2- 172.21.2.101:0/3801589685 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe13c0ac2c0 0x7fe13c1a7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:39.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.461 7fe142065700 1 -- 172.21.2.101:0/3801589685 >> 172.21.2.101:0/3801589685 conn(0x7fe13c09ffd0 msgr2=0x7fe13c082ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:39.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.461 7fe142065700 1 -- 172.21.2.101:0/3801589685 shutdown_connections 2021-02-10T11:34:39.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.461 7fe142065700 1 -- 172.21.2.101:0/3801589685 wait complete. 2021-02-10T11:34:39.480 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_data 8 2021-02-10T11:34:39.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.884 7f3635c04700 1 Processor -- start 2021-02-10T11:34:39.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.884 7f3635c04700 1 -- start start 2021-02-10T11:34:39.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.885 7f3635c04700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f36300d1870 0x7f36301b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:39.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.885 7f3635c04700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f36300cff70 0x7f3630109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:39.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.885 7f3635c04700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36300fe1c0 con 0x7f36301b0ae0 2021-02-10T11:34:39.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.885 7f3635c04700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3630109ad0 con 0x7f36300d1870 2021-02-10T11:34:39.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.885 7f3635c04700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3630109750 con 0x7f36300cff70 2021-02-10T11:34:39.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.885 7f362f7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f36300d1870 0x7f36301b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:39.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.885 7f362f7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f36300d1870 0x7f36301b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52512/0 (socket says 172.21.2.101:52512) 2021-02-10T11:34:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.885 7f362f7fe700 1 -- 172.21.2.101:0/2030047354 learned_addr learned my addr 172.21.2.101:0/2030047354 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.886 7f362f7fe700 1 -- 172.21.2.101:0/2030047354 >> v2:172.21.2.108:3300/0 conn(0x7f36300cff70 msgr2=0x7f3630109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.886 7f362f7fe700 1 --2- 172.21.2.101:0/2030047354 >> v2:172.21.2.108:3300/0 conn(0x7f36300cff70 0x7f3630109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.886 7f362f7fe700 1 -- 172.21.2.101:0/2030047354 >> v1:172.21.2.108:6789/0 conn(0x7f36301b0ae0 legacy=0x7f3630108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.886 7f362f7fe700 1 -- 172.21.2.101:0/2030047354 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3630088070 con 0x7f36300d1870 2021-02-10T11:34:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.886 7f362f7fe700 1 --2- 172.21.2.101:0/2030047354 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36300d1870 0x7f36301b04c0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7f3620006700 tx=0x7f3620009b80).ready entity=mon.0 client_cookie=20129bdecb248e03 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.886 7f362e7fc700 1 -- 172.21.2.101:0/2030047354 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f362000c0c0 con 0x7f36300d1870 2021-02-10T11:34:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.886 7f362e7fc700 1 -- 172.21.2.101:0/2030047354 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362000c340 con 0x7f36300d1870 2021-02-10T11:34:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.886 7f362e7fc700 1 -- 172.21.2.101:0/2030047354 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f362000c7a0 con 0x7f36300d1870 2021-02-10T11:34:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.886 7f3635c04700 1 -- 172.21.2.101:0/2030047354 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36300d1870 msgr2=0x7f36301b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.886 7f3635c04700 1 --2- 172.21.2.101:0/2030047354 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36300d1870 0x7f36301b04c0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7f3620006700 tx=0x7f3620009b80).stop 2021-02-10T11:34:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.887 7f3635c04700 1 -- 172.21.2.101:0/2030047354 shutdown_connections 2021-02-10T11:34:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.887 7f3635c04700 1 --2- 172.21.2.101:0/2030047354 >> v2:172.21.2.108:3300/0 conn(0x7f36300cff70 0x7f3630109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:39.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.887 7f3635c04700 1 --2- 172.21.2.101:0/2030047354 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36300d1870 0x7f36301b04c0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:39.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.887 7f3635c04700 1 -- 172.21.2.101:0/2030047354 >> 172.21.2.101:0/2030047354 conn(0x7f36300ce420 msgr2=0x7f36300d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:39.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.887 7f3635c04700 1 -- 172.21.2.101:0/2030047354 shutdown_connections 2021-02-10T11:34:39.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.887 7f3635c04700 1 -- 172.21.2.101:0/2030047354 wait complete. 2021-02-10T11:34:39.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.902 7f3635c04700 1 Processor -- start 2021-02-10T11:34:39.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.902 7f3635c04700 1 -- start start 2021-02-10T11:34:39.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.902 7f3635c04700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36301906c0 0x7f36301a78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:39.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.902 7f3635c04700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f36301a7f10 0x7f3630195400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:39.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.902 7f3635c04700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3630190ac0 0x7f3630195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:39.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.902 7f3635c04700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3630196040 con 0x7f36301906c0 2021-02-10T11:34:39.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.902 7f3635c04700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36300c2730 con 0x7f36301a7f10 2021-02-10T11:34:39.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.902 7f3635c04700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36300c2950 con 0x7f3630190ac0 2021-02-10T11:34:39.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.902 7f362f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36301906c0 0x7f36301a78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:39.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.902 7f362f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36301906c0 0x7f36301a78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52516/0 (socket says 172.21.2.101:52516) 2021-02-10T11:34:39.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.902 7f362f7fe700 1 -- 172.21.2.101:0/3176416146 learned_addr learned my addr 172.21.2.101:0/3176416146 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:39.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.903 7f362f7fe700 1 -- 172.21.2.101:0/3176416146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3630190ac0 msgr2=0x7f3630195a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:39.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.903 7f362f7fe700 1 --2- 172.21.2.101:0/3176416146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3630190ac0 0x7f3630195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:39.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.903 7f362f7fe700 1 -- 172.21.2.101:0/3176416146 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f36301a7f10 msgr2=0x7f3630195400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:39.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.903 7f362f7fe700 1 --2- 172.21.2.101:0/3176416146 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f36301a7f10 0x7f3630195400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:39.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.903 7f362f7fe700 1 -- 172.21.2.101:0/3176416146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363009ca10 con 0x7f36301906c0 2021-02-10T11:34:39.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.903 7f362f7fe700 1 --2- 172.21.2.101:0/3176416146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36301906c0 0x7f36301a78f0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0x7f362000b5e0 tx=0x7f362000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:39.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.903 7f3617fff700 1 -- 172.21.2.101:0/3176416146 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f362000bd70 con 0x7f36301906c0 2021-02-10T11:34:39.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.903 7f3617fff700 1 -- 172.21.2.101:0/3176416146 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3620002030 con 0x7f36301906c0 2021-02-10T11:34:39.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.903 7f3617fff700 1 -- 172.21.2.101:0/3176416146 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3620002550 con 0x7f36301906c0 2021-02-10T11:34:39.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.903 7f3635c04700 1 -- 172.21.2.101:0/3176416146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f363009ca10 con 0x7f36301906c0 2021-02-10T11:34:39.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.903 7f3635c04700 1 -- 172.21.2.101:0/3176416146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363009cfa0 con 0x7f36301906c0 2021-02-10T11:34:39.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.904 7f3617fff700 1 -- 172.21.2.101:0/3176416146 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3620000c90 con 0x7f36301906c0 2021-02-10T11:34:39.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.904 7f3617fff700 1 --2- 172.21.2.101:0/3176416146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36100186c0 0x7f361001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:39.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.904 7f3617fff700 1 -- 172.21.2.101:0/3176416146 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5107+0+0 (secure 0 0 0) 0x7f36200323b0 con 0x7f36301906c0 2021-02-10T11:34:39.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.904 7f362effd700 1 --2- 172.21.2.101:0/3176416146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36100186c0 0x7f361001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:39.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.904 7f362effd700 1 --2- 172.21.2.101:0/3176416146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36100186c0 0x7f361001abc0 crc :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:39.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.935 7f3635c04700 1 -- 172.21.2.101:0/3176416146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3630082a70 con 0x7f36301906c0 2021-02-10T11:34:39.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:39.938 7f3617fff700 1 -- 172.21.2.101:0/3176416146 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f36200009b0 con 0x7f36301906c0 2021-02-10T11:34:40.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.066 7f3635c04700 1 -- 172.21.2.101:0/3176416146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"} v 0) v1 -- 0x7f3630082a70 con 0x7f36301906c0 2021-02-10T11:34:40.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.446 7f3617fff700 1 -- 172.21.2.101:0/3176416146 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"}]=0 pool 'cephfs_data' created v63) v1 ==== 125+0+0 (secure 0 0 0) 0x7f3620000c90 con 0x7f36301906c0 2021-02-10T11:34:40.491 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' created 2021-02-10T11:34:40.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.483 7f3635c04700 1 -- 172.21.2.101:0/3176416146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36100186c0 msgr2=0x7f361001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:40.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.483 7f3635c04700 1 --2- 172.21.2.101:0/3176416146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36100186c0 0x7f361001abc0 crc :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:40.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.483 7f3635c04700 1 -- 172.21.2.101:0/3176416146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36301906c0 msgr2=0x7f36301a78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:40.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.483 7f3635c04700 1 --2- 172.21.2.101:0/3176416146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36301906c0 0x7f36301a78f0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0x7f362000b5e0 tx=0x7f362000b980).stop 2021-02-10T11:34:40.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.483 7f3635c04700 1 -- 172.21.2.101:0/3176416146 shutdown_connections 2021-02-10T11:34:40.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.483 7f3635c04700 1 --2- 172.21.2.101:0/3176416146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36100186c0 0x7f361001abc0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:40.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.483 7f3635c04700 1 --2- 172.21.2.101:0/3176416146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3630190ac0 0x7f3630195a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:40.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.483 7f3635c04700 1 --2- 172.21.2.101:0/3176416146 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f36301a7f10 0x7f3630195400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:40.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.483 7f3635c04700 1 --2- 172.21.2.101:0/3176416146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36301906c0 0x7f36301a78f0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:40.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.483 7f3635c04700 1 -- 172.21.2.101:0/3176416146 >> 172.21.2.101:0/3176416146 conn(0x7f36300b6070 msgr2=0x7f3630095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:40.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.484 7f3635c04700 1 -- 172.21.2.101:0/3176416146 shutdown_connections 2021-02-10T11:34:40.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.484 7f3635c04700 1 -- 172.21.2.101:0/3176416146 wait complete. 2021-02-10T11:34:40.502 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-02-10T11:34:40.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.906 7f7f64848700 1 Processor -- start 2021-02-10T11:34:40.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.906 7f7f64848700 1 -- start start 2021-02-10T11:34:40.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.906 7f7f64848700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7f6010b000 0x7f7f600cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:40.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.906 7f7f64848700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f600fe1d0 con 0x7f7f600d1880 2021-02-10T11:34:40.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.906 7f7f64848700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f601b0cc0 con 0x7f7f600d1c80 2021-02-10T11:34:40.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.906 7f7f64848700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f60109a70 con 0x7f7f6010b000 2021-02-10T11:34:40.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.906 7f7f5e61c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f7f600d1880 0x7f7f601b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53018/0 (socket says 172.21.2.101:53018) 2021-02-10T11:34:40.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.906 7f7f5e61c700 1 -- 172.21.2.101:0/3626433151 learned_addr learned my addr 172.21.2.101:0/3626433151 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.906 7f7f5ee1d700 1 --2- 172.21.2.101:0/3626433151 >> v2:172.21.2.101:3300/0 conn(0x7f7f6010b000 0x7f7f600cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.907 7f7f5ee1d700 1 -- 172.21.2.101:0/3626433151 >> v1:172.21.2.108:6789/0 conn(0x7f7f600d1c80 legacy=0x7f7f600cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.907 7f7f5ee1d700 1 -- 172.21.2.101:0/3626433151 >> v1:172.21.2.101:6789/0 conn(0x7f7f600d1880 legacy=0x7f7f601b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.907 7f7f5ee1d700 1 -- 172.21.2.101:0/3626433151 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6010b400 con 0x7f7f6010b000 2021-02-10T11:34:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.907 7f7f5ee1d700 1 --2- 172.21.2.101:0/3626433151 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f6010b000 0x7f7f600cf760 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f7f58006e30 tx=0x7f7f58009e60).ready entity=mon.0 client_cookie=16330b0f096a37c2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.907 7f7f5d61a700 1 -- 172.21.2.101:0/3626433151 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7f5800b300 con 0x7f7f6010b000 2021-02-10T11:34:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.907 7f7f5d61a700 1 -- 172.21.2.101:0/3626433151 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f58000d30 con 0x7f7f6010b000 2021-02-10T11:34:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.907 7f7f5d61a700 1 -- 172.21.2.101:0/3626433151 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7f5800b870 con 0x7f7f6010b000 2021-02-10T11:34:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.907 7f7f64848700 1 -- 172.21.2.101:0/3626433151 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f6010b000 msgr2=0x7f7f600cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.907 7f7f64848700 1 --2- 172.21.2.101:0/3626433151 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f6010b000 0x7f7f600cf760 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f7f58006e30 tx=0x7f7f58009e60).stop 2021-02-10T11:34:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.908 7f7f64848700 1 -- 172.21.2.101:0/3626433151 shutdown_connections 2021-02-10T11:34:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.908 7f7f64848700 1 --2- 172.21.2.101:0/3626433151 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f6010b000 0x7f7f600cf760 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:40.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.908 7f7f64848700 1 -- 172.21.2.101:0/3626433151 >> 172.21.2.101:0/3626433151 conn(0x7f7f600ce430 msgr2=0x7f7f600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:40.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.908 7f7f64848700 1 -- 172.21.2.101:0/3626433151 shutdown_connections 2021-02-10T11:34:40.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.908 7f7f64848700 1 -- 172.21.2.101:0/3626433151 wait complete. 2021-02-10T11:34:40.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.923 7f7f64848700 1 Processor -- start 2021-02-10T11:34:40.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.923 7f7f64848700 1 -- start start 2021-02-10T11:34:40.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.923 7f7f64848700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7f600be9d0 0x7f7f600bedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:40.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.923 7f7f64848700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f60106af0 0x7f7f60106ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:40.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.923 7f7f64848700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7f60094cd0 0x7f7f600950d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.923 7f7f64848700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f601085b0 con 0x7f7f60106af0 2021-02-10T11:34:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.923 7f7f64848700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f60107510 con 0x7f7f60094cd0 2021-02-10T11:34:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.923 7f7f64848700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f600956f0 con 0x7f7f600be9d0 2021-02-10T11:34:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.923 7f7f5de1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f60106af0 0x7f7f60106ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.923 7f7f5de1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f60106af0 0x7f7f60106ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52530/0 (socket says 172.21.2.101:52530) 2021-02-10T11:34:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.923 7f7f5de1b700 1 -- 172.21.2.101:0/2509218775 learned_addr learned my addr 172.21.2.101:0/2509218775 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.924 7f7f5de1b700 1 -- 172.21.2.101:0/2509218775 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7f600be9d0 msgr2=0x7f7f600bedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.924 7f7f5de1b700 1 --2- 172.21.2.101:0/2509218775 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7f600be9d0 0x7f7f600bedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.924 7f7f5de1b700 1 -- 172.21.2.101:0/2509218775 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7f60094cd0 msgr2=0x7f7f600950d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.924 7f7f5de1b700 1 --2- 172.21.2.101:0/2509218775 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7f60094cd0 0x7f7f600950d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.924 7f7f5de1b700 1 -- 172.21.2.101:0/2509218775 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f60095910 con 0x7f7f60106af0 2021-02-10T11:34:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.924 7f7f5de1b700 1 --2- 172.21.2.101:0/2509218775 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f60106af0 0x7f7f60106ef0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f7f5000b530 tx=0x7f7f5000b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.924 7f7f56ffd700 1 -- 172.21.2.101:0/2509218775 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7f5000bcc0 con 0x7f7f60106af0 2021-02-10T11:34:40.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.924 7f7f56ffd700 1 -- 172.21.2.101:0/2509218775 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f50000e20 con 0x7f7f60106af0 2021-02-10T11:34:40.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.924 7f7f56ffd700 1 -- 172.21.2.101:0/2509218775 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7f50006210 con 0x7f7f60106af0 2021-02-10T11:34:40.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.924 7f7f64848700 1 -- 172.21.2.101:0/2509218775 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f60095910 con 0x7f7f60106af0 2021-02-10T11:34:40.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.924 7f7f64848700 1 -- 172.21.2.101:0/2509218775 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f60095910 con 0x7f7f60106af0 2021-02-10T11:34:40.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.925 7f7f56ffd700 1 -- 172.21.2.101:0/2509218775 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7f50006020 con 0x7f7f60106af0 2021-02-10T11:34:40.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.925 7f7f56ffd700 1 --2- 172.21.2.101:0/2509218775 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7f440186c0 0x7f7f4401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:40.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.925 7f7f56ffd700 1 -- 172.21.2.101:0/2509218775 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5457+0+0 (secure 0 0 0) 0x7f7f50032520 con 0x7f7f60106af0 2021-02-10T11:34:40.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.925 7f7f5e61c700 1 --2- 172.21.2.101:0/2509218775 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7f440186c0 0x7f7f4401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:40.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.925 7f7f5e61c700 1 --2- 172.21.2.101:0/2509218775 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7f440186c0 0x7f7f4401abc0 crc :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:40.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.956 7f7f64848700 1 -- 172.21.2.101:0/2509218775 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f60080340 con 0x7f7f60106af0 2021-02-10T11:34:40.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:40.959 7f7f56ffd700 1 -- 172.21.2.101:0/2509218775 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7f50030d40 con 0x7f7f60106af0 2021-02-10T11:34:41.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.086 7f7f64848700 1 -- 172.21.2.101:0/2509218775 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"} v 0) v1 -- 0x7f7f60088070 con 0x7f7f60106af0 2021-02-10T11:34:41.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.091 7f7f56ffd700 1 -- 172.21.2.101:0/2509218775 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"}]=0 new fs with metadata pool 12 and data pool 13 v44) v1 ==== 189+0+0 (secure 0 0 0) 0x7f7f50006020 con 0x7f7f60106af0 2021-02-10T11:34:41.103 INFO:teuthology.orchestra.run.gibba001.stderr:new fs with metadata pool 12 and data pool 13 2021-02-10T11:34:41.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.096 7f7f64848700 1 -- 172.21.2.101:0/2509218775 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7f440186c0 msgr2=0x7f7f4401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:41.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.096 7f7f64848700 1 --2- 172.21.2.101:0/2509218775 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7f440186c0 0x7f7f4401abc0 crc :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:41.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.096 7f7f64848700 1 -- 172.21.2.101:0/2509218775 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f60106af0 msgr2=0x7f7f60106ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:41.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.096 7f7f64848700 1 --2- 172.21.2.101:0/2509218775 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f60106af0 0x7f7f60106ef0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f7f5000b530 tx=0x7f7f5000b8d0).stop 2021-02-10T11:34:41.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.096 7f7f64848700 1 -- 172.21.2.101:0/2509218775 shutdown_connections 2021-02-10T11:34:41.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.096 7f7f64848700 1 --2- 172.21.2.101:0/2509218775 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7f60094cd0 0x7f7f600950d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:41.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.096 7f7f64848700 1 --2- 172.21.2.101:0/2509218775 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7f60106af0 0x7f7f60106ef0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:41.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.096 7f7f64848700 1 --2- 172.21.2.101:0/2509218775 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7f440186c0 0x7f7f4401abc0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:41.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.096 7f7f64848700 1 --2- 172.21.2.101:0/2509218775 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7f600be9d0 0x7f7f600bedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:41.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.096 7f7f64848700 1 -- 172.21.2.101:0/2509218775 >> 172.21.2.101:0/2509218775 conn(0x7f7f6009fb30 msgr2=0x7f7f601b5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:41.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.096 7f7f64848700 1 -- 172.21.2.101:0/2509218775 shutdown_connections 2021-02-10T11:34:41.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.096 7f7f64848700 1 -- 172.21.2.101:0/2509218775 wait complete. 2021-02-10T11:34:41.114 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:34:41.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.513 7f9646257700 1 Processor -- start 2021-02-10T11:34:41.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.513 7f9646257700 1 -- start start 2021-02-10T11:34:41.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.514 7f9646257700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f96400d1c70 0x7f96400f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:41.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.514 7f9646257700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96400f0b10 con 0x7f96400d0340 2021-02-10T11:34:41.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.514 7f9646257700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96401b0be0 con 0x7f96400d1870 2021-02-10T11:34:41.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.514 7f9646257700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96400e5ed0 con 0x7f96400d1c70 2021-02-10T11:34:41.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.514 7f963f7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f96400d0340 0x7f96400ef640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53032/0 (socket says 172.21.2.101:53032) 2021-02-10T11:34:41.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.514 7f963f7fe700 1 -- 172.21.2.101:0/895614962 learned_addr learned my addr 172.21.2.101:0/895614962 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:41.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.514 7f9644874700 1 --2- 172.21.2.101:0/895614962 >> v2:172.21.2.108:3300/0 conn(0x7f96400d1c70 0x7f96400f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:41.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.514 7f963effd700 1 -- 172.21.2.101:0/895614962 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2407310715 0 0) 0x7f96340010b0 con 0x7f96400d0340 2021-02-10T11:34:41.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.514 7f963effd700 1 -- 172.21.2.101:0/895614962 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9628002fe0 con 0x7f96400d0340 2021-02-10T11:34:41.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.515 7f9644874700 1 -- 172.21.2.101:0/895614962 >> v1:172.21.2.108:6789/0 conn(0x7f96400d1870 legacy=0x7f96401b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:41.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.515 7f9644874700 1 -- 172.21.2.101:0/895614962 >> v1:172.21.2.101:6789/0 conn(0x7f96400d0340 legacy=0x7f96400ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:41.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.515 7f9644874700 1 -- 172.21.2.101:0/895614962 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96400efe30 con 0x7f96400d1c70 2021-02-10T11:34:41.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.515 7f9644874700 1 --2- 172.21.2.101:0/895614962 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96400d1c70 0x7f96400f20a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f9638006e50 tx=0x7f9638009e70).ready entity=mon.1 client_cookie=a20ddb9cef654427 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:41.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.515 7f963effd700 1 -- 172.21.2.101:0/895614962 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9638006a20 con 0x7f96400d1c70 2021-02-10T11:34:41.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.515 7f963effd700 1 -- 172.21.2.101:0/895614962 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9638006790 con 0x7f96400d1c70 2021-02-10T11:34:41.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.515 7f963effd700 1 -- 172.21.2.101:0/895614962 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f963800b700 con 0x7f96400d1c70 2021-02-10T11:34:41.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.515 7f9646257700 1 -- 172.21.2.101:0/895614962 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96400d1c70 msgr2=0x7f96400f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:41.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.515 7f9646257700 1 --2- 172.21.2.101:0/895614962 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96400d1c70 0x7f96400f20a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f9638006e50 tx=0x7f9638009e70).stop 2021-02-10T11:34:41.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.516 7f9646257700 1 -- 172.21.2.101:0/895614962 shutdown_connections 2021-02-10T11:34:41.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.516 7f9646257700 1 --2- 172.21.2.101:0/895614962 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96400d1c70 0x7f96400f20a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:41.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.516 7f9646257700 1 -- 172.21.2.101:0/895614962 >> 172.21.2.101:0/895614962 conn(0x7f96400ce450 msgr2=0x7f96400d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:41.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.516 7f9646257700 1 -- 172.21.2.101:0/895614962 shutdown_connections 2021-02-10T11:34:41.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.516 7f9646257700 1 -- 172.21.2.101:0/895614962 wait complete. 2021-02-10T11:34:41.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.530 7f9646257700 1 Processor -- start 2021-02-10T11:34:41.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.530 7f9646257700 1 -- start start 2021-02-10T11:34:41.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.531 7f9646257700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f964018f0a0 0x7f9640196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:41.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.531 7f9646257700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96401970d0 0x7f96401974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:41.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.531 7f9646257700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9640094650 0x7f9640094a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:41.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.531 7f9646257700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9640095bd0 con 0x7f9640094650 2021-02-10T11:34:41.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.531 7f9646257700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96400ad7c0 con 0x7f964018f0a0 2021-02-10T11:34:41.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.531 7f9646257700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964018ec70 con 0x7f96401970d0 2021-02-10T11:34:41.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.531 7f9644874700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9640094650 0x7f9640094a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:41.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.531 7f9644874700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9640094650 0x7f9640094a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52542/0 (socket says 172.21.2.101:52542) 2021-02-10T11:34:41.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.531 7f9644874700 1 -- 172.21.2.101:0/743518831 learned_addr learned my addr 172.21.2.101:0/743518831 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.531 7f963f7fe700 1 --2- 172.21.2.101:0/743518831 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96401970d0 0x7f96401974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.531 7f963ffff700 1 --2- 172.21.2.101:0/743518831 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f964018f0a0 0x7f9640196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.531 7f9644874700 1 -- 172.21.2.101:0/743518831 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96401970d0 msgr2=0x7f96401974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.531 7f9644874700 1 --2- 172.21.2.101:0/743518831 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96401970d0 0x7f96401974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.531 7f9644874700 1 -- 172.21.2.101:0/743518831 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f964018f0a0 msgr2=0x7f9640196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.531 7f9644874700 1 --2- 172.21.2.101:0/743518831 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f964018f0a0 0x7f9640196ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.531 7f9644874700 1 -- 172.21.2.101:0/743518831 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9640095070 con 0x7f9640094650 2021-02-10T11:34:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.531 7f963ffff700 1 --2- 172.21.2.101:0/743518831 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f964018f0a0 0x7f9640196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:34:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.532 7f9644874700 1 --2- 172.21.2.101:0/743518831 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9640094650 0x7f9640094a50 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7f9638006e60 tx=0x7f963800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.532 7f963cff9700 1 -- 172.21.2.101:0/743518831 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9638009130 con 0x7f9640094650 2021-02-10T11:34:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.532 7f963cff9700 1 -- 172.21.2.101:0/743518831 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9638009430 con 0x7f9640094650 2021-02-10T11:34:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.532 7f9646257700 1 -- 172.21.2.101:0/743518831 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9640095070 con 0x7f9640094650 2021-02-10T11:34:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.532 7f9646257700 1 -- 172.21.2.101:0/743518831 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9640095070 con 0x7f9640094650 2021-02-10T11:34:41.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.532 7f963cff9700 1 -- 172.21.2.101:0/743518831 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9638001390 con 0x7f9640094650 2021-02-10T11:34:41.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.533 7f963cff9700 1 -- 172.21.2.101:0/743518831 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9638009130 con 0x7f9640094650 2021-02-10T11:34:41.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.533 7f963cff9700 1 --2- 172.21.2.101:0/743518831 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96200186c0 0x7f962001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:41.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.533 7f963cff9700 1 -- 172.21.2.101:0/743518831 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9638032930 con 0x7f9640094650 2021-02-10T11:34:41.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.533 7f963ffff700 1 --2- 172.21.2.101:0/743518831 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96200186c0 0x7f962001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:41.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.533 7f963ffff700 1 --2- 172.21.2.101:0/743518831 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96200186c0 0x7f962001abc0 crc :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:41.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.564 7f9646257700 1 -- 172.21.2.101:0/743518831 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96400ad790 con 0x7f9640094650 2021-02-10T11:34:41.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.566 7f963cff9700 1 -- 172.21.2.101:0/743518831 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9638009430 con 0x7f9640094650 2021-02-10T11:34:41.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.690 7f9646257700 1 -- 172.21.2.101:0/743518831 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96400ad790 con 0x7f9640094650 2021-02-10T11:34:41.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.690 7f963cff9700 1 -- 172.21.2.101:0/743518831 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+14009 (secure 0 0 0) 0x7f9638009430 con 0x7f9640094650 2021-02-10T11:34:41.700 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:41.700 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":64,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:34:41.087989","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:34:38.607957","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2021-02-10 11:34:40.084211","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:34:41.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.692 7f9646257700 1 -- 172.21.2.101:0/743518831 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96200186c0 msgr2=0x7f962001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:41.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.692 7f9646257700 1 --2- 172.21.2.101:0/743518831 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96200186c0 0x7f962001abc0 crc :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:41.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.692 7f9646257700 1 -- 172.21.2.101:0/743518831 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9640094650 msgr2=0x7f9640094a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:41.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.692 7f9646257700 1 --2- 172.21.2.101:0/743518831 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9640094650 0x7f9640094a50 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7f9638006e60 tx=0x7f963800bcf0).stop 2021-02-10T11:34:41.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.692 7f9646257700 1 -- 172.21.2.101:0/743518831 shutdown_connections 2021-02-10T11:34:41.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.692 7f9646257700 1 --2- 172.21.2.101:0/743518831 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96200186c0 0x7f962001abc0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:41.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.692 7f9646257700 1 --2- 172.21.2.101:0/743518831 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96401970d0 0x7f96401974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:41.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.692 7f9646257700 1 --2- 172.21.2.101:0/743518831 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9640094650 0x7f9640094a50 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:41.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.692 7f9646257700 1 --2- 172.21.2.101:0/743518831 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f964018f0a0 0x7f9640196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:41.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.692 7f9646257700 1 -- 172.21.2.101:0/743518831 >> 172.21.2.101:0/743518831 conn(0x7f96401a38c0 msgr2=0x7f96401a2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:41.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.692 7f9646257700 1 -- 172.21.2.101:0/743518831 shutdown_connections 2021-02-10T11:34:41.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:41.692 7f9646257700 1 -- 172.21.2.101:0/743518831 wait complete. 2021-02-10T11:34:41.710 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:34:42.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.109 7efeecc6c700 1 Processor -- start 2021-02-10T11:34:42.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.109 7efeecc6c700 1 -- start start 2021-02-10T11:34:42.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.110 7efeecc6c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efee81b0870 0x7efee80f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:42.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.110 7efeecc6c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7efee80efe00 0x7efee80f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:42.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.110 7efeecc6c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee81b0c70 con 0x7efee81b0470 2021-02-10T11:34:42.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.110 7efeecc6c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee80f0b50 con 0x7efee80efe00 2021-02-10T11:34:42.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.110 7efeecc6c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee80f0820 con 0x7efee81b0870 2021-02-10T11:34:42.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.110 7efee685e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7efee81b0470 0x7efee80ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53044/0 (socket says 172.21.2.101:53044) 2021-02-10T11:34:42.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.110 7efee685e700 1 -- 172.21.2.101:0/1352383756 learned_addr learned my addr 172.21.2.101:0/1352383756 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:42.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.110 7efee705f700 1 --2- 172.21.2.101:0/1352383756 >> v2:172.21.2.101:3300/0 conn(0x7efee80efe00 0x7efee80f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:42.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.110 7efee605d700 1 --2- 172.21.2.101:0/1352383756 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efee81b0870 0x7efee80f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:42.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.110 7efee585c700 1 -- 172.21.2.101:0/1352383756 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 591246727 0 0) 0x7efed8000ce0 con 0x7efee81b0470 2021-02-10T11:34:42.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.110 7efee585c700 1 -- 172.21.2.101:0/1352383756 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efed4002fe0 con 0x7efee81b0470 2021-02-10T11:34:42.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.110 7efee705f700 1 -- 172.21.2.101:0/1352383756 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efee81b0870 msgr2=0x7efee80f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:42.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.110 7efee705f700 1 --2- 172.21.2.101:0/1352383756 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efee81b0870 0x7efee80f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:42.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.110 7efee705f700 1 -- 172.21.2.101:0/1352383756 >> v1:172.21.2.101:6789/0 conn(0x7efee81b0470 legacy=0x7efee80ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:42.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.110 7efee705f700 1 -- 172.21.2.101:0/1352383756 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee80e52e0 con 0x7efee80efe00 2021-02-10T11:34:42.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.111 7efee705f700 1 --2- 172.21.2.101:0/1352383756 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efee80efe00 0x7efee80f0200 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7efee0006e30 tx=0x7efee0009e60).ready entity=mon.0 client_cookie=676234de23729476 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:42.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.111 7efee585c700 1 -- 172.21.2.101:0/1352383756 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efee0006a50 con 0x7efee80efe00 2021-02-10T11:34:42.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.111 7efee585c700 1 -- 172.21.2.101:0/1352383756 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee000b300 con 0x7efee80efe00 2021-02-10T11:34:42.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.111 7efee585c700 1 -- 172.21.2.101:0/1352383756 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efee000b820 con 0x7efee80efe00 2021-02-10T11:34:42.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.111 7efeecc6c700 1 -- 172.21.2.101:0/1352383756 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efee80efe00 msgr2=0x7efee80f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:42.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.111 7efeecc6c700 1 --2- 172.21.2.101:0/1352383756 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efee80efe00 0x7efee80f0200 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7efee0006e30 tx=0x7efee0009e60).stop 2021-02-10T11:34:42.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.112 7efeecc6c700 1 -- 172.21.2.101:0/1352383756 shutdown_connections 2021-02-10T11:34:42.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.112 7efeecc6c700 1 --2- 172.21.2.101:0/1352383756 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efee81b0870 0x7efee80f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:42.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.112 7efeecc6c700 1 --2- 172.21.2.101:0/1352383756 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efee80efe00 0x7efee80f0200 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:42.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.112 7efeecc6c700 1 -- 172.21.2.101:0/1352383756 >> 172.21.2.101:0/1352383756 conn(0x7efee80ce3e0 msgr2=0x7efee80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:42.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.112 7efeecc6c700 1 -- 172.21.2.101:0/1352383756 shutdown_connections 2021-02-10T11:34:42.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.112 7efeecc6c700 1 -- 172.21.2.101:0/1352383756 wait complete. 2021-02-10T11:34:42.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.127 7efeecc6c700 1 Processor -- start 2021-02-10T11:34:42.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.127 7efeecc6c700 1 -- start start 2021-02-10T11:34:42.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.127 7efeecc6c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efee808d510 0x7efee81b99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:42.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.127 7efeecc6c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efee8106c20 0x7efee8107020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:42.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.127 7efeecc6c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efee808e0f0 0x7efee808e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:42.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.127 7efeecc6c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee808db70 con 0x7efee808d510 2021-02-10T11:34:42.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.127 7efeecc6c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee8107640 con 0x7efee8106c20 2021-02-10T11:34:42.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.127 7efeecc6c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee8107820 con 0x7efee808e0f0 2021-02-10T11:34:42.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.128 7efee685e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efee808d510 0x7efee81b99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:42.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.128 7efee685e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efee808d510 0x7efee81b99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52558/0 (socket says 172.21.2.101:52558) 2021-02-10T11:34:42.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.128 7efee685e700 1 -- 172.21.2.101:0/4140571552 learned_addr learned my addr 172.21.2.101:0/4140571552 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:42.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.128 7efee705f700 1 --2- 172.21.2.101:0/4140571552 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efee808e0f0 0x7efee808e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:42.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.128 7efee605d700 1 --2- 172.21.2.101:0/4140571552 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efee8106c20 0x7efee8107020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:42.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.128 7efee685e700 1 -- 172.21.2.101:0/4140571552 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efee808e0f0 msgr2=0x7efee808e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:42.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.128 7efee685e700 1 --2- 172.21.2.101:0/4140571552 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efee808e0f0 0x7efee808e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:42.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.128 7efee685e700 1 -- 172.21.2.101:0/4140571552 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efee8106c20 msgr2=0x7efee8107020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:42.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.128 7efee685e700 1 --2- 172.21.2.101:0/4140571552 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efee8106c20 0x7efee8107020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:42.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.128 7efee685e700 1 -- 172.21.2.101:0/4140571552 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee81b4f60 con 0x7efee808d510 2021-02-10T11:34:42.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.128 7efee685e700 1 --2- 172.21.2.101:0/4140571552 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efee808d510 0x7efee81b99e0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7efed800a5d0 tx=0x7efed800a970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:42.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.129 7efedf7fe700 1 -- 172.21.2.101:0/4140571552 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efed8000ce0 con 0x7efee808d510 2021-02-10T11:34:42.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.129 7efedf7fe700 1 -- 172.21.2.101:0/4140571552 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed80043a0 con 0x7efee808d510 2021-02-10T11:34:42.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.129 7efeecc6c700 1 -- 172.21.2.101:0/4140571552 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee81b4f60 con 0x7efee808d510 2021-02-10T11:34:42.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.129 7efeecc6c700 1 -- 172.21.2.101:0/4140571552 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee808ee00 con 0x7efee808d510 2021-02-10T11:34:42.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.129 7efedf7fe700 1 -- 172.21.2.101:0/4140571552 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efed80048c0 con 0x7efee808d510 2021-02-10T11:34:42.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.129 7efedf7fe700 1 -- 172.21.2.101:0/4140571552 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7efed8000ce0 con 0x7efee808d510 2021-02-10T11:34:42.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.129 7efedf7fe700 1 --2- 172.21.2.101:0/4140571552 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efecc0186d0 0x7efecc01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:42.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.129 7efedf7fe700 1 -- 172.21.2.101:0/4140571552 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7efed80327b0 con 0x7efee808d510 2021-02-10T11:34:42.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.129 7efee605d700 1 --2- 172.21.2.101:0/4140571552 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efecc0186d0 0x7efecc01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:42.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.130 7efee605d700 1 --2- 172.21.2.101:0/4140571552 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efecc0186d0 0x7efecc01abd0 crc :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:42.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.160 7efeecc6c700 1 -- 172.21.2.101:0/4140571552 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee8105a20 con 0x7efee808d510 2021-02-10T11:34:42.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.162 7efedf7fe700 1 -- 172.21.2.101:0/4140571552 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7efed80043a0 con 0x7efee808d510 2021-02-10T11:34:42.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.289 7efeecc6c700 1 -- 172.21.2.101:0/4140571552 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efee80f0b50 con 0x7efee808d510 2021-02-10T11:34:42.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.289 7efedf7fe700 1 -- 172.21.2.101:0/4140571552 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+13996 (secure 0 0 0) 0x7efed8004b80 con 0x7efee808d510 2021-02-10T11:34:42.301 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:42.301 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":65,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:34:42.090329","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:34:38.607957","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2021-02-10 11:34:40.084211","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:34:42.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.291 7efeecc6c700 1 -- 172.21.2.101:0/4140571552 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efecc0186d0 msgr2=0x7efecc01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:42.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.291 7efeecc6c700 1 --2- 172.21.2.101:0/4140571552 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efecc0186d0 0x7efecc01abd0 crc :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:42.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.291 7efeecc6c700 1 -- 172.21.2.101:0/4140571552 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efee808d510 msgr2=0x7efee81b99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:42.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.291 7efeecc6c700 1 --2- 172.21.2.101:0/4140571552 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efee808d510 0x7efee81b99e0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7efed800a5d0 tx=0x7efed800a970).stop 2021-02-10T11:34:42.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.291 7efeecc6c700 1 -- 172.21.2.101:0/4140571552 shutdown_connections 2021-02-10T11:34:42.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.291 7efeecc6c700 1 --2- 172.21.2.101:0/4140571552 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efecc0186d0 0x7efecc01abd0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:42.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.291 7efeecc6c700 1 --2- 172.21.2.101:0/4140571552 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efee808e0f0 0x7efee808e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:42.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.291 7efeecc6c700 1 --2- 172.21.2.101:0/4140571552 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efee8106c20 0x7efee8107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:42.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.291 7efeecc6c700 1 --2- 172.21.2.101:0/4140571552 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efee808d510 0x7efee81b99e0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:42.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.291 7efeecc6c700 1 -- 172.21.2.101:0/4140571552 >> 172.21.2.101:0/4140571552 conn(0x7efee81a04a0 msgr2=0x7efee80916f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:42.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.292 7efeecc6c700 1 -- 172.21.2.101:0/4140571552 shutdown_connections 2021-02-10T11:34:42.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.292 7efeecc6c700 1 -- 172.21.2.101:0/4140571552 wait complete. 2021-02-10T11:34:42.310 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-02-10T11:34:42.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.714 7ff9d0e21700 1 Processor -- start 2021-02-10T11:34:42.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.714 7ff9d0e21700 1 -- start start 2021-02-10T11:34:42.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.714 7ff9d0e21700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff9cc0d1c70 0x7ff9cc0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:42.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.714 7ff9d0e21700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9cc0d0170 con 0x7ff9cc0d1870 2021-02-10T11:34:42.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.714 7ff9d0e21700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9cc0fe1c0 con 0x7ff9cc0cf580 2021-02-10T11:34:42.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.714 7ff9d0e21700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9cc1b0cb0 con 0x7ff9cc0d1c70 2021-02-10T11:34:42.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.714 7ff9cab13700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff9cc0d1870 0x7ff9cc1b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53058/0 (socket says 172.21.2.101:53058) 2021-02-10T11:34:42.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.714 7ff9cab13700 1 -- 172.21.2.101:0/1949560018 learned_addr learned my addr 172.21.2.101:0/1949560018 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:42.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.714 7ff9ca312700 1 --2- 172.21.2.101:0/1949560018 >> v2:172.21.2.101:3300/0 conn(0x7ff9cc0d1c70 0x7ff9cc0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:42.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.715 7ff9c9b11700 1 -- 172.21.2.101:0/1949560018 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2458017200 0 0) 0x7ff9b4001180 con 0x7ff9cc0d1870 2021-02-10T11:34:42.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.715 7ff9c9b11700 1 -- 172.21.2.101:0/1949560018 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9b8002fe0 con 0x7ff9cc0d1870 2021-02-10T11:34:42.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.715 7ff9ca312700 1 -- 172.21.2.101:0/1949560018 >> v1:172.21.2.108:6789/0 conn(0x7ff9cc0cf580 legacy=0x7ff9cc0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:42.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.715 7ff9ca312700 1 -- 172.21.2.101:0/1949560018 >> v1:172.21.2.101:6789/0 conn(0x7ff9cc0d1870 legacy=0x7ff9cc1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:42.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.715 7ff9ca312700 1 -- 172.21.2.101:0/1949560018 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9cc109a60 con 0x7ff9cc0d1c70 2021-02-10T11:34:42.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.715 7ff9ca312700 1 --2- 172.21.2.101:0/1949560018 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9cc0d1c70 0x7ff9cc0cef60 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7ff9bc006650 tx=0x7ff9bc009a80).ready entity=mon.0 client_cookie=d2445336be589b12 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:42.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.715 7ff9c9b11700 1 -- 172.21.2.101:0/1949560018 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff9bc00c0c0 con 0x7ff9cc0d1c70 2021-02-10T11:34:42.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.715 7ff9c9b11700 1 -- 172.21.2.101:0/1949560018 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9bc00c340 con 0x7ff9cc0d1c70 2021-02-10T11:34:42.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.715 7ff9c9b11700 1 -- 172.21.2.101:0/1949560018 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff9bc00c860 con 0x7ff9cc0d1c70 2021-02-10T11:34:42.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.715 7ff9d0e21700 1 -- 172.21.2.101:0/1949560018 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9cc0d1c70 msgr2=0x7ff9cc0cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:42.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.715 7ff9d0e21700 1 --2- 172.21.2.101:0/1949560018 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9cc0d1c70 0x7ff9cc0cef60 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7ff9bc006650 tx=0x7ff9bc009a80).stop 2021-02-10T11:34:42.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.716 7ff9d0e21700 1 -- 172.21.2.101:0/1949560018 shutdown_connections 2021-02-10T11:34:42.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.716 7ff9d0e21700 1 --2- 172.21.2.101:0/1949560018 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9cc0d1c70 0x7ff9cc0cef60 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:42.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.716 7ff9d0e21700 1 -- 172.21.2.101:0/1949560018 >> 172.21.2.101:0/1949560018 conn(0x7ff9cc0ce420 msgr2=0x7ff9cc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:42.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.716 7ff9d0e21700 1 -- 172.21.2.101:0/1949560018 shutdown_connections 2021-02-10T11:34:42.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.716 7ff9d0e21700 1 -- 172.21.2.101:0/1949560018 wait complete. 2021-02-10T11:34:42.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.731 7ff9d0e21700 1 Processor -- start 2021-02-10T11:34:42.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.731 7ff9d0e21700 1 -- start start 2021-02-10T11:34:42.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.731 7ff9d0e21700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9cc0c2370 0x7ff9cc18f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:42.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.731 7ff9d0e21700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9cc084cb0 0x7ff9cc1a7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:42.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.731 7ff9d0e21700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff9cc1a7a90 0x7ff9cc085700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:42.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.731 7ff9d0e21700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc0be840 con 0x7ff9cc0c2370 2021-02-10T11:34:42.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.731 7ff9d0e21700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc0c29a0 con 0x7ff9cc1a7a90 2021-02-10T11:34:42.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.731 7ff9d0e21700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc1a7e90 con 0x7ff9cc084cb0 2021-02-10T11:34:42.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.731 7ff9cab13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9cc0c2370 0x7ff9cc18f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:42.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.731 7ff9cab13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9cc0c2370 0x7ff9cc18f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52572/0 (socket says 172.21.2.101:52572) 2021-02-10T11:34:42.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.731 7ff9cab13700 1 -- 172.21.2.101:0/2208116002 learned_addr learned my addr 172.21.2.101:0/2208116002 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:42.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.732 7ff9cab13700 1 -- 172.21.2.101:0/2208116002 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9cc084cb0 msgr2=0x7ff9cc1a7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:42.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.732 7ff9cab13700 1 --2- 172.21.2.101:0/2208116002 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9cc084cb0 0x7ff9cc1a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:42.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.732 7ff9cab13700 1 -- 172.21.2.101:0/2208116002 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff9cc1a7a90 msgr2=0x7ff9cc085700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:42.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.732 7ff9cab13700 1 --2- 172.21.2.101:0/2208116002 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff9cc1a7a90 0x7ff9cc085700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:42.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.732 7ff9cab13700 1 -- 172.21.2.101:0/2208116002 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9cc1a80b0 con 0x7ff9cc0c2370 2021-02-10T11:34:42.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.732 7ff9cab13700 1 --2- 172.21.2.101:0/2208116002 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9cc0c2370 0x7ff9cc18f870 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 rx=0x7ff9b400b530 tx=0x7ff9b400b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:42.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.732 7ff9c37fe700 1 -- 172.21.2.101:0/2208116002 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff9b400bcc0 con 0x7ff9cc0c2370 2021-02-10T11:34:42.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.732 7ff9c37fe700 1 -- 172.21.2.101:0/2208116002 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b4006030 con 0x7ff9cc0c2370 2021-02-10T11:34:42.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.732 7ff9c37fe700 1 -- 172.21.2.101:0/2208116002 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff9b4004620 con 0x7ff9cc0c2370 2021-02-10T11:34:42.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.732 7ff9d0e21700 1 -- 172.21.2.101:0/2208116002 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9cc1a80b0 con 0x7ff9cc0c2370 2021-02-10T11:34:42.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.732 7ff9d0e21700 1 -- 172.21.2.101:0/2208116002 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9cc1a80b0 con 0x7ff9cc0c2370 2021-02-10T11:34:42.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.733 7ff9c37fe700 1 -- 172.21.2.101:0/2208116002 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff9b40043a0 con 0x7ff9cc0c2370 2021-02-10T11:34:42.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.733 7ff9c37fe700 1 --2- 172.21.2.101:0/2208116002 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff9b00186c0 0x7ff9b001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:42.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.733 7ff9c37fe700 1 -- 172.21.2.101:0/2208116002 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff9b4032760 con 0x7ff9cc0c2370 2021-02-10T11:34:42.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.733 7ff9ca312700 1 --2- 172.21.2.101:0/2208116002 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff9b00186c0 0x7ff9b001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:42.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.733 7ff9ca312700 1 --2- 172.21.2.101:0/2208116002 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff9b00186c0 0x7ff9b001abc0 crc :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:42.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.764 7ff9d0e21700 1 -- 172.21.2.101:0/2208116002 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9cc0be840 con 0x7ff9cc0c2370 2021-02-10T11:34:42.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.767 7ff9c37fe700 1 -- 172.21.2.101:0/2208116002 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff9b4006b90 con 0x7ff9cc0c2370 2021-02-10T11:34:42.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:42.904 7ff9d0e21700 1 -- 172.21.2.101:0/2208116002 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"} v 0) v1 -- 0x7ff9cc0be840 con 0x7ff9cc0c2370 2021-02-10T11:34:43.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.455 7ff9c37fe700 1 -- 172.21.2.101:0/2208116002 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"}]=0 v46) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff9b40043a0 con 0x7ff9cc0c2370 2021-02-10T11:34:43.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.475 7ff9d0e21700 1 -- 172.21.2.101:0/2208116002 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff9b00186c0 msgr2=0x7ff9b001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:43.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.475 7ff9d0e21700 1 --2- 172.21.2.101:0/2208116002 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff9b00186c0 0x7ff9b001abc0 crc :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:43.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.475 7ff9d0e21700 1 -- 172.21.2.101:0/2208116002 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9cc0c2370 msgr2=0x7ff9cc18f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:43.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.475 7ff9d0e21700 1 --2- 172.21.2.101:0/2208116002 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9cc0c2370 0x7ff9cc18f870 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 rx=0x7ff9b400b530 tx=0x7ff9b400b8d0).stop 2021-02-10T11:34:43.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.475 7ff9d0e21700 1 -- 172.21.2.101:0/2208116002 shutdown_connections 2021-02-10T11:34:43.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.475 7ff9d0e21700 1 --2- 172.21.2.101:0/2208116002 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff9b00186c0 0x7ff9b001abc0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:43.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.475 7ff9d0e21700 1 --2- 172.21.2.101:0/2208116002 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9cc084cb0 0x7ff9cc1a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:43.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.475 7ff9d0e21700 1 --2- 172.21.2.101:0/2208116002 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff9cc1a7a90 0x7ff9cc085700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:43.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.475 7ff9d0e21700 1 --2- 172.21.2.101:0/2208116002 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9cc0c2370 0x7ff9cc18f870 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:43.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.475 7ff9d0e21700 1 -- 172.21.2.101:0/2208116002 >> 172.21.2.101:0/2208116002 conn(0x7ff9cc0b5dc0 msgr2=0x7ff9cc0c2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:43.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.475 7ff9d0e21700 1 -- 172.21.2.101:0/2208116002 shutdown_connections 2021-02-10T11:34:43.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.475 7ff9d0e21700 1 -- 172.21.2.101:0/2208116002 wait complete. 2021-02-10T11:34:43.493 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:34:43.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.895 7f5f49ca8700 1 Processor -- start 2021-02-10T11:34:43.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.895 7f5f49ca8700 1 -- start start 2021-02-10T11:34:43.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.896 7f5f49ca8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5f440d1870 0x7f5f441b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:43.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.896 7f5f49ca8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5f441b0a10 0x7f5f440cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:43.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.896 7f5f49ca8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f5f440d1c70 0x7f5f440cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:43.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.896 7f5f49ca8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f440fe200 con 0x7f5f440d1870 2021-02-10T11:34:43.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.896 7f5f49ca8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f440e5f10 con 0x7f5f440d1c70 2021-02-10T11:34:43.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.896 7f5f49ca8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f440f0b50 con 0x7f5f441b0a10 2021-02-10T11:34:43.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.896 7f5f437fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5f440d1870 0x7f5f441b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:43.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.896 7f5f437fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5f440d1870 0x7f5f441b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52578/0 (socket says 172.21.2.101:52578) 2021-02-10T11:34:43.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.896 7f5f437fe700 1 -- 172.21.2.101:0/767357005 learned_addr learned my addr 172.21.2.101:0/767357005 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:43.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.896 7f5f437fe700 1 -- 172.21.2.101:0/767357005 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5f441b0a10 msgr2=0x7f5f440cf330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:34:43.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.896 7f5f437fe700 1 --2- 172.21.2.101:0/767357005 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5f441b0a10 0x7f5f440cf330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:43.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.896 7f5f437fe700 1 -- 172.21.2.101:0/767357005 >> v2:172.21.2.108:3300/0 conn(0x7f5f440d1c70 msgr2=0x7f5f440cf950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:34:43.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.896 7f5f437fe700 1 --2- 172.21.2.101:0/767357005 >> v2:172.21.2.108:3300/0 conn(0x7f5f440d1c70 0x7f5f440cf950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:43.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.896 7f5f437fe700 1 -- 172.21.2.101:0/767357005 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f440cff70 con 0x7f5f440d1870 2021-02-10T11:34:43.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.896 7f5f437fe700 1 --2- 172.21.2.101:0/767357005 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f440d1870 0x7f5f441b03f0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7f5f2c006700 tx=0x7f5f2c009b00).ready entity=mon.0 client_cookie=981deb87482ed33f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:43.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.896 7f5f427fc700 1 -- 172.21.2.101:0/767357005 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5f2c00c0c0 con 0x7f5f440d1870 2021-02-10T11:34:43.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.896 7f5f427fc700 1 -- 172.21.2.101:0/767357005 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2c00c340 con 0x7f5f440d1870 2021-02-10T11:34:43.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.896 7f5f427fc700 1 -- 172.21.2.101:0/767357005 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5f2c00c820 con 0x7f5f440d1870 2021-02-10T11:34:43.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.896 7f5f49ca8700 1 -- 172.21.2.101:0/767357005 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f440d1870 msgr2=0x7f5f441b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:43.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.896 7f5f49ca8700 1 --2- 172.21.2.101:0/767357005 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f440d1870 0x7f5f441b03f0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7f5f2c006700 tx=0x7f5f2c009b00).stop 2021-02-10T11:34:43.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.897 7f5f49ca8700 1 -- 172.21.2.101:0/767357005 shutdown_connections 2021-02-10T11:34:43.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.897 7f5f49ca8700 1 --2- 172.21.2.101:0/767357005 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5f441b0a10 0x7f5f440cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:43.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.897 7f5f49ca8700 1 --2- 172.21.2.101:0/767357005 >> v2:172.21.2.108:3300/0 conn(0x7f5f440d1c70 0x7f5f440cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:43.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.897 7f5f49ca8700 1 --2- 172.21.2.101:0/767357005 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f440d1870 0x7f5f441b03f0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:43.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.897 7f5f49ca8700 1 -- 172.21.2.101:0/767357005 >> 172.21.2.101:0/767357005 conn(0x7f5f440ce450 msgr2=0x7f5f440d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:43.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.897 7f5f49ca8700 1 -- 172.21.2.101:0/767357005 shutdown_connections 2021-02-10T11:34:43.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.897 7f5f49ca8700 1 -- 172.21.2.101:0/767357005 wait complete. 2021-02-10T11:34:43.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.911 7f5f49ca8700 1 Processor -- start 2021-02-10T11:34:43.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.912 7f5f49ca8700 1 -- start start 2021-02-10T11:34:43.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.912 7f5f49ca8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5f441b8140 0x7f5f441b8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:43.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.912 7f5f49ca8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f441b8b60 0x7f5f441969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:43.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.912 7f5f49ca8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5f44196fd0 0x7f5f44197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:43.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.912 7f5f49ca8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f44197990 con 0x7f5f441b8b60 2021-02-10T11:34:43.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.912 7f5f49ca8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f441973d0 con 0x7f5f441b8140 2021-02-10T11:34:43.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.912 7f5f49ca8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f441975f0 con 0x7f5f44196fd0 2021-02-10T11:34:43.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.912 7f5f42ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f441b8b60 0x7f5f441969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:43.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.912 7f5f42ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f441b8b60 0x7f5f441969b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52584/0 (socket says 172.21.2.101:52584) 2021-02-10T11:34:43.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.912 7f5f42ffd700 1 -- 172.21.2.101:0/3104375523 learned_addr learned my addr 172.21.2.101:0/3104375523 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:43.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.913 7f5f42ffd700 1 -- 172.21.2.101:0/3104375523 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5f44196fd0 msgr2=0x7f5f44197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:43.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.913 7f5f42ffd700 1 --2- 172.21.2.101:0/3104375523 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5f44196fd0 0x7f5f44197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:43.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.913 7f5f42ffd700 1 -- 172.21.2.101:0/3104375523 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5f441b8140 msgr2=0x7f5f441b8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:43.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.913 7f5f42ffd700 1 --2- 172.21.2.101:0/3104375523 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5f441b8140 0x7f5f441b8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:43.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.913 7f5f42ffd700 1 -- 172.21.2.101:0/3104375523 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f44198530 con 0x7f5f441b8b60 2021-02-10T11:34:43.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.913 7f5f42ffd700 1 --2- 172.21.2.101:0/3104375523 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f441b8b60 0x7f5f441969b0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7f5f3400b5c0 tx=0x7f5f3400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:43.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.913 7f5f3bfff700 1 -- 172.21.2.101:0/3104375523 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5f3400bd50 con 0x7f5f441b8b60 2021-02-10T11:34:43.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.913 7f5f3bfff700 1 -- 172.21.2.101:0/3104375523 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f34006030 con 0x7f5f441b8b60 2021-02-10T11:34:43.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.913 7f5f49ca8700 1 -- 172.21.2.101:0/3104375523 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f44198530 con 0x7f5f441b8b60 2021-02-10T11:34:43.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.913 7f5f49ca8700 1 -- 172.21.2.101:0/3104375523 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f44198ac0 con 0x7f5f441b8b60 2021-02-10T11:34:43.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.913 7f5f3bfff700 1 -- 172.21.2.101:0/3104375523 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5f340054d0 con 0x7f5f441b8b60 2021-02-10T11:34:43.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.914 7f5f3bfff700 1 -- 172.21.2.101:0/3104375523 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5f34006210 con 0x7f5f441b8b60 2021-02-10T11:34:43.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.914 7f5f3bfff700 1 --2- 172.21.2.101:0/3104375523 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5f280186c0 0x7f5f2801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:43.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.914 7f5f3bfff700 1 -- 172.21.2.101:0/3104375523 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5f340325b0 con 0x7f5f441b8b60 2021-02-10T11:34:43.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.914 7f5f437fe700 1 --2- 172.21.2.101:0/3104375523 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5f280186c0 0x7f5f2801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:43.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.914 7f5f437fe700 1 --2- 172.21.2.101:0/3104375523 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5f280186c0 0x7f5f2801abc0 crc :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:43.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.945 7f5f49ca8700 1 -- 172.21.2.101:0/3104375523 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f440ce450 con 0x7f5f441b8b60 2021-02-10T11:34:43.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:43.948 7f5f3bfff700 1 -- 172.21.2.101:0/3104375523 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5f34000940 con 0x7f5f441b8b60 2021-02-10T11:34:44.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.076 7f5f49ca8700 1 -- 172.21.2.101:0/3104375523 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f440ce450 con 0x7f5f441b8b60 2021-02-10T11:34:44.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.076 7f5f3bfff700 1 -- 172.21.2.101:0/3104375523 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f5f3400bd50 con 0x7f5f441b8b60 2021-02-10T11:34:44.086 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 46 2021-02-10T11:34:44.086 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:44.087 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1297756180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1297756180},{"type":"v1","addr":"172.21.2.101:6835","nonce":1297756180}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:34:41.088085","modified":"2021-02-10 11:34:43.450805","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6400":{"gid":6400,"name":"b","rank":0,"incarnation":44,"state":"up:active","state_seq":4,"addr":"172.21.2.108:6833/3585539482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3585539482},{"type":"v1","addr":"172.21.2.108:6833","nonce":3585539482}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-02-10T11:34:44.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.078 7f5f49ca8700 1 -- 172.21.2.101:0/3104375523 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5f280186c0 msgr2=0x7f5f2801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:44.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.078 7f5f49ca8700 1 --2- 172.21.2.101:0/3104375523 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5f280186c0 0x7f5f2801abc0 crc :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:44.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.078 7f5f49ca8700 1 -- 172.21.2.101:0/3104375523 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f441b8b60 msgr2=0x7f5f441969b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:44.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.078 7f5f49ca8700 1 --2- 172.21.2.101:0/3104375523 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f441b8b60 0x7f5f441969b0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7f5f3400b5c0 tx=0x7f5f3400b960).stop 2021-02-10T11:34:44.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.078 7f5f49ca8700 1 -- 172.21.2.101:0/3104375523 shutdown_connections 2021-02-10T11:34:44.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.078 7f5f49ca8700 1 --2- 172.21.2.101:0/3104375523 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5f280186c0 0x7f5f2801abc0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:44.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.078 7f5f49ca8700 1 --2- 172.21.2.101:0/3104375523 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5f44196fd0 0x7f5f44197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:44.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.078 7f5f49ca8700 1 --2- 172.21.2.101:0/3104375523 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f441b8b60 0x7f5f441969b0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:44.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.078 7f5f49ca8700 1 --2- 172.21.2.101:0/3104375523 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5f441b8140 0x7f5f441b8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:44.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.078 7f5f49ca8700 1 -- 172.21.2.101:0/3104375523 >> 172.21.2.101:0/3104375523 conn(0x7f5f440efc40 msgr2=0x7f5f441b9080 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:44.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.079 7f5f49ca8700 1 -- 172.21.2.101:0/3104375523 shutdown_connections 2021-02-10T11:34:44.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.079 7f5f49ca8700 1 -- 172.21.2.101:0/3104375523 wait complete. 2021-02-10T11:34:44.097 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:34:44.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.506 7fd4658f4700 1 Processor -- start 2021-02-10T11:34:44.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.506 7fd4658f4700 1 -- start start 2021-02-10T11:34:44.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.507 7fd4658f4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4601b0470 0x7fd4601b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:44.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.507 7fd4658f4700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd4600f20e0 0x7fd4600f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:44.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.507 7fd4658f4700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4600f0b50 con 0x7fd4600ef610 2021-02-10T11:34:44.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.507 7fd4658f4700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4600f0820 con 0x7fd4600f20e0 2021-02-10T11:34:44.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.507 7fd4658f4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4600f24e0 con 0x7fd4601b0470 2021-02-10T11:34:44.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.507 7fd45ed73700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd4600ef610 0x7fd4600efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53086/0 (socket says 172.21.2.101:53086) 2021-02-10T11:34:44.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.507 7fd45ed73700 1 -- 172.21.2.101:0/3074581035 learned_addr learned my addr 172.21.2.101:0/3074581035 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:44.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.507 7fd45e572700 1 -- 172.21.2.101:0/3074581035 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1119418619 0 0) 0x7fd4540010b0 con 0x7fd4600ef610 2021-02-10T11:34:44.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.507 7fd45e572700 1 -- 172.21.2.101:0/3074581035 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd448002fe0 con 0x7fd4600ef610 2021-02-10T11:34:44.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.507 7fd45fd75700 1 --2- 172.21.2.101:0/3074581035 >> v2:172.21.2.108:3300/0 conn(0x7fd4600f20e0 0x7fd4600f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:44.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.507 7fd45f574700 1 --2- 172.21.2.101:0/3074581035 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4601b0470 0x7fd4601b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:44.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45e572700 1 -- 172.21.2.101:0/3074581035 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2276601898 0 0) 0x7fd4540010b0 con 0x7fd4600ef610 2021-02-10T11:34:44.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45e572700 1 -- 172.21.2.101:0/3074581035 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4601b0470 msgr2=0x7fd4601b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:44.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45e572700 1 --2- 172.21.2.101:0/3074581035 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4601b0470 0x7fd4601b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:44.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45e572700 1 -- 172.21.2.101:0/3074581035 >> v2:172.21.2.108:3300/0 conn(0x7fd4600f20e0 msgr2=0x7fd4600f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:44.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45e572700 1 --2- 172.21.2.101:0/3074581035 >> v2:172.21.2.108:3300/0 conn(0x7fd4600f20e0 0x7fd4600f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:44.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45e572700 1 -- 172.21.2.101:0/3074581035 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4600e52e0 con 0x7fd4600ef610 2021-02-10T11:34:44.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45e572700 1 -- 172.21.2.101:0/3074581035 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fd4540015b0 con 0x7fd4600ef610 2021-02-10T11:34:44.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45e572700 1 -- 172.21.2.101:0/3074581035 >> v1:172.21.2.101:6789/0 conn(0x7fd4600ef610 legacy=0x7fd4600efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:44.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45e572700 1 --2- 172.21.2.101:0/3074581035 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd448003a60 0x7fd448005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:44.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45e572700 1 --2- 172.21.2.101:0/3074581035 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd44804a790 0x7fd44804cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:44.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45e572700 1 --2- 172.21.2.101:0/3074581035 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd44804d590 0x7fd44804fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:44.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45e572700 1 -- 172.21.2.101:0/3074581035 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4480502d0 con 0x7fd44804a790 2021-02-10T11:34:44.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45e572700 1 -- 172.21.2.101:0/3074581035 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4480504f0 con 0x7fd448003a60 2021-02-10T11:34:44.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45e572700 1 -- 172.21.2.101:0/3074581035 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd448050710 con 0x7fd44804d590 2021-02-10T11:34:44.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45ed73700 1 --2- 172.21.2.101:0/3074581035 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd44804a790 0x7fd44804cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:44.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45f574700 1 --2- 172.21.2.101:0/3074581035 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd448003a60 0x7fd448005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:44.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45fd75700 1 --2- 172.21.2.101:0/3074581035 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd44804d590 0x7fd44804fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:44.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45ed73700 1 -- 172.21.2.101:0/3074581035 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd44804d590 msgr2=0x7fd44804fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:44.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45ed73700 1 --2- 172.21.2.101:0/3074581035 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd44804d590 0x7fd44804fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:44.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45ed73700 1 -- 172.21.2.101:0/3074581035 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd448003a60 msgr2=0x7fd448005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:44.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45ed73700 1 --2- 172.21.2.101:0/3074581035 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd448003a60 0x7fd448005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:44.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45ed73700 1 -- 172.21.2.101:0/3074581035 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4480509b0 con 0x7fd44804a790 2021-02-10T11:34:44.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45f574700 1 --2- 172.21.2.101:0/3074581035 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd448003a60 0x7fd448005f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:34:44.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45ed73700 1 --2- 172.21.2.101:0/3074581035 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd44804a790 0x7fd44804cc90 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 rx=0x7fd4540017d0 tx=0x7fd454007d30).ready entity=mon.0 client_cookie=b8e5db4811aa05a9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:44.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45e572700 1 -- 172.21.2.101:0/3074581035 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd4540013a0 con 0x7fd44804a790 2021-02-10T11:34:44.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45e572700 1 -- 172.21.2.101:0/3074581035 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45400a310 con 0x7fd44804a790 2021-02-10T11:34:44.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45e572700 1 -- 172.21.2.101:0/3074581035 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd45400a830 con 0x7fd44804a790 2021-02-10T11:34:44.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd4658f4700 1 -- 172.21.2.101:0/3074581035 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd44804a790 msgr2=0x7fd44804cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:44.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd4658f4700 1 --2- 172.21.2.101:0/3074581035 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd44804a790 0x7fd44804cc90 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 rx=0x7fd4540017d0 tx=0x7fd454007d30).stop 2021-02-10T11:34:44.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.508 7fd45fd75700 1 -- 172.21.2.101:0/3074581035 reap_dead start 2021-02-10T11:34:44.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.509 7fd4658f4700 1 -- 172.21.2.101:0/3074581035 shutdown_connections 2021-02-10T11:34:44.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.509 7fd4658f4700 1 -- 172.21.2.101:0/3074581035 >> 172.21.2.101:0/3074581035 conn(0x7fd4600ce3e0 msgr2=0x7fd4600e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:44.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.509 7fd4658f4700 1 -- 172.21.2.101:0/3074581035 shutdown_connections 2021-02-10T11:34:44.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.509 7fd4658f4700 1 -- 172.21.2.101:0/3074581035 wait complete. 2021-02-10T11:34:44.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.524 7fd4658f4700 1 Processor -- start 2021-02-10T11:34:44.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.524 7fd4658f4700 1 -- start start 2021-02-10T11:34:44.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.524 7fd4658f4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd460108ce0 0x7fd46009da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:44.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.524 7fd4658f4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4601090e0 0x7fd46009e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:44.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.524 7fd4658f4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd46009eee0 0x7fd46009f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:44.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.525 7fd4658f4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4601094e0 con 0x7fd460108ce0 2021-02-10T11:34:44.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.525 7fd4658f4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46009e960 con 0x7fd4601090e0 2021-02-10T11:34:44.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.525 7fd4658f4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46009e690 con 0x7fd46009eee0 2021-02-10T11:34:44.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.525 7fd45f574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd460108ce0 0x7fd46009da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:44.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.525 7fd45f574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd460108ce0 0x7fd46009da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52604/0 (socket says 172.21.2.101:52604) 2021-02-10T11:34:44.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.525 7fd45f574700 1 -- 172.21.2.101:0/497859511 learned_addr learned my addr 172.21.2.101:0/497859511 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:44.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.525 7fd45ed73700 1 --2- 172.21.2.101:0/497859511 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4601090e0 0x7fd46009e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:44.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.525 7fd45fd75700 1 --2- 172.21.2.101:0/497859511 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd46009eee0 0x7fd46009f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:44.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.525 7fd45f574700 1 -- 172.21.2.101:0/497859511 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd46009eee0 msgr2=0x7fd46009f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:44.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.525 7fd45f574700 1 --2- 172.21.2.101:0/497859511 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd46009eee0 0x7fd46009f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:44.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.525 7fd45f574700 1 -- 172.21.2.101:0/497859511 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4601090e0 msgr2=0x7fd46009e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:44.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.525 7fd45f574700 1 --2- 172.21.2.101:0/497859511 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4601090e0 0x7fd46009e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:44.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.525 7fd45f574700 1 -- 172.21.2.101:0/497859511 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46009f900 con 0x7fd460108ce0 2021-02-10T11:34:44.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.525 7fd45fd75700 1 --2- 172.21.2.101:0/497859511 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd46009eee0 0x7fd46009f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:34:44.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.525 7fd45ed73700 1 --2- 172.21.2.101:0/497859511 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4601090e0 0x7fd46009e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:34:44.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.525 7fd45f574700 1 --2- 172.21.2.101:0/497859511 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd460108ce0 0x7fd46009da50 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 rx=0x7fd450006f40 tx=0x7fd45000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:44.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.526 7fd447fff700 1 -- 172.21.2.101:0/497859511 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd4500067f0 con 0x7fd460108ce0 2021-02-10T11:34:44.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.526 7fd447fff700 1 -- 172.21.2.101:0/497859511 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd450006c40 con 0x7fd460108ce0 2021-02-10T11:34:44.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.526 7fd447fff700 1 -- 172.21.2.101:0/497859511 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd4500054d0 con 0x7fd460108ce0 2021-02-10T11:34:44.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.526 7fd4658f4700 1 -- 172.21.2.101:0/497859511 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd46009f900 con 0x7fd460108ce0 2021-02-10T11:34:44.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.526 7fd4658f4700 1 -- 172.21.2.101:0/497859511 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46009f900 con 0x7fd460108ce0 2021-02-10T11:34:44.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.526 7fd447fff700 1 -- 172.21.2.101:0/497859511 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd45000a000 con 0x7fd460108ce0 2021-02-10T11:34:44.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.526 7fd447fff700 1 --2- 172.21.2.101:0/497859511 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4400186c0 0x7fd44001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:44.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.526 7fd447fff700 1 -- 172.21.2.101:0/497859511 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd4500323e0 con 0x7fd460108ce0 2021-02-10T11:34:44.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.527 7fd45ed73700 1 --2- 172.21.2.101:0/497859511 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4400186c0 0x7fd44001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:44.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.527 7fd45ed73700 1 --2- 172.21.2.101:0/497859511 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4400186c0 0x7fd44001abc0 crc :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:44.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.556 7fd4658f4700 1 -- 172.21.2.101:0/497859511 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4600f0b50 con 0x7fd460108ce0 2021-02-10T11:34:44.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.559 7fd447fff700 1 -- 172.21.2.101:0/497859511 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd4500067f0 con 0x7fd460108ce0 2021-02-10T11:34:44.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.684 7fd4658f4700 1 -- 172.21.2.101:0/497859511 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4600f0b50 con 0x7fd460108ce0 2021-02-10T11:34:44.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.685 7fd447fff700 1 -- 172.21.2.101:0/497859511 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+13996 (secure 0 0 0) 0x7fd45001f4d0 con 0x7fd460108ce0 2021-02-10T11:34:44.694 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:44.695 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":65,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:34:42.090329","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:34:38.607957","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2021-02-10 11:34:40.084211","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:34:44.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.686 7fd4658f4700 1 -- 172.21.2.101:0/497859511 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4400186c0 msgr2=0x7fd44001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:44.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.686 7fd4658f4700 1 --2- 172.21.2.101:0/497859511 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4400186c0 0x7fd44001abc0 crc :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:44.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.686 7fd4658f4700 1 -- 172.21.2.101:0/497859511 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd460108ce0 msgr2=0x7fd46009da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:44.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.686 7fd4658f4700 1 --2- 172.21.2.101:0/497859511 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd460108ce0 0x7fd46009da50 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 rx=0x7fd450006f40 tx=0x7fd45000b990).stop 2021-02-10T11:34:44.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.686 7fd4658f4700 1 -- 172.21.2.101:0/497859511 shutdown_connections 2021-02-10T11:34:44.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.686 7fd4658f4700 1 --2- 172.21.2.101:0/497859511 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4400186c0 0x7fd44001abc0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:44.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.686 7fd4658f4700 1 --2- 172.21.2.101:0/497859511 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd46009eee0 0x7fd46009f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:44.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.686 7fd4658f4700 1 --2- 172.21.2.101:0/497859511 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4601090e0 0x7fd46009e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:44.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.686 7fd4658f4700 1 --2- 172.21.2.101:0/497859511 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd460108ce0 0x7fd46009da50 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:44.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.686 7fd4658f4700 1 -- 172.21.2.101:0/497859511 >> 172.21.2.101:0/497859511 conn(0x7fd4600bc6c0 msgr2=0x7fd4600b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:44.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.687 7fd4658f4700 1 -- 172.21.2.101:0/497859511 shutdown_connections 2021-02-10T11:34:44.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:44.687 7fd4658f4700 1 -- 172.21.2.101:0/497859511 wait complete. 2021-02-10T11:34:44.705 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:34:45.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.118 7fc948be7700 1 Processor -- start 2021-02-10T11:34:45.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.118 7fc948be7700 1 -- start start 2021-02-10T11:34:45.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.119 7fc948be7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9440d1c80 0x7fc9440cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:45.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.119 7fc948be7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc9440cf590 0x7fc9440cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:45.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.119 7fc948be7700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9440fe1d0 con 0x7fc9440d1880 2021-02-10T11:34:45.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.119 7fc948be7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9441b0cc0 con 0x7fc9440cf590 2021-02-10T11:34:45.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.119 7fc948be7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc944109a70 con 0x7fc9440d1c80 2021-02-10T11:34:45.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.119 7fc94305f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc9440cf590 0x7fc9440cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:45.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.119 7fc94305f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc9440cf590 0x7fc9440cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:34868/0 (socket says 172.21.2.101:34868) 2021-02-10T11:34:45.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.119 7fc94305f700 1 -- 172.21.2.101:0/1890616593 learned_addr learned my addr 172.21.2.101:0/1890616593 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:45.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.119 7fc94205d700 1 --2- 172.21.2.101:0/1890616593 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9440d1c80 0x7fc9440cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:45.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.120 7fc94205d700 1 -- 172.21.2.101:0/1890616593 >> v2:172.21.2.108:3300/0 conn(0x7fc9440cf590 msgr2=0x7fc9440cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:45.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.120 7fc94205d700 1 --2- 172.21.2.101:0/1890616593 >> v2:172.21.2.108:3300/0 conn(0x7fc9440cf590 0x7fc9440cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:45.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.120 7fc94205d700 1 -- 172.21.2.101:0/1890616593 >> v1:172.21.2.108:6789/0 conn(0x7fc9440d1880 legacy=0x7fc9441b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:45.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.120 7fc94205d700 1 -- 172.21.2.101:0/1890616593 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc944088070 con 0x7fc9440d1c80 2021-02-10T11:34:45.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.120 7fc94205d700 1 --2- 172.21.2.101:0/1890616593 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9440d1c80 0x7fc9440cef70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fc92c006650 tx=0x7fc92c009a80).ready entity=mon.2 client_cookie=d3c7b940eb8b3983 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:45.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.121 7fc94185c700 1 -- 172.21.2.101:0/1890616593 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc92c00c0c0 con 0x7fc9440d1c80 2021-02-10T11:34:45.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.121 7fc94185c700 1 -- 172.21.2.101:0/1890616593 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92c00c340 con 0x7fc9440d1c80 2021-02-10T11:34:45.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.121 7fc94185c700 1 -- 172.21.2.101:0/1890616593 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc92c00c860 con 0x7fc9440d1c80 2021-02-10T11:34:45.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.121 7fc948be7700 1 -- 172.21.2.101:0/1890616593 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9440d1c80 msgr2=0x7fc9440cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:45.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.121 7fc948be7700 1 --2- 172.21.2.101:0/1890616593 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9440d1c80 0x7fc9440cef70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fc92c006650 tx=0x7fc92c009a80).stop 2021-02-10T11:34:45.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.122 7fc948be7700 1 -- 172.21.2.101:0/1890616593 shutdown_connections 2021-02-10T11:34:45.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.122 7fc948be7700 1 --2- 172.21.2.101:0/1890616593 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9440d1c80 0x7fc9440cef70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:45.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.122 7fc948be7700 1 --2- 172.21.2.101:0/1890616593 >> v2:172.21.2.108:3300/0 conn(0x7fc9440cf590 0x7fc9440cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:45.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.122 7fc948be7700 1 -- 172.21.2.101:0/1890616593 >> 172.21.2.101:0/1890616593 conn(0x7fc9440ce430 msgr2=0x7fc9440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:45.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.122 7fc948be7700 1 -- 172.21.2.101:0/1890616593 shutdown_connections 2021-02-10T11:34:45.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.122 7fc948be7700 1 -- 172.21.2.101:0/1890616593 wait complete. 2021-02-10T11:34:45.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.136 7fc948be7700 1 Processor -- start 2021-02-10T11:34:45.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.136 7fc948be7700 1 -- start start 2021-02-10T11:34:45.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.136 7fc948be7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc94419a070 0x7fc94408c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:45.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.136 7fc948be7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc94419a470 0x7fc94408cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:45.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.136 7fc948be7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc944095520 0x7fc944095920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:45.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.136 7fc948be7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc944096900 con 0x7fc944095520 2021-02-10T11:34:45.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.136 7fc948be7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94408d4c0 con 0x7fc94419a070 2021-02-10T11:34:45.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.136 7fc948be7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94419e9e0 con 0x7fc94419a470 2021-02-10T11:34:45.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.137 7fc94305f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc944095520 0x7fc944095920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:45.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.137 7fc94305f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc944095520 0x7fc944095920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52616/0 (socket says 172.21.2.101:52616) 2021-02-10T11:34:45.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.137 7fc94305f700 1 -- 172.21.2.101:0/3864345683 learned_addr learned my addr 172.21.2.101:0/3864345683 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:45.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.137 7fc94305f700 1 -- 172.21.2.101:0/3864345683 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc94419a470 msgr2=0x7fc94408cda0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:34:45.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.137 7fc94305f700 1 --2- 172.21.2.101:0/3864345683 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc94419a470 0x7fc94408cda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:45.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.137 7fc94305f700 1 -- 172.21.2.101:0/3864345683 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc94419a070 msgr2=0x7fc94408c780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:34:45.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.137 7fc94305f700 1 --2- 172.21.2.101:0/3864345683 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc94419a070 0x7fc94408c780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:45.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.137 7fc94305f700 1 -- 172.21.2.101:0/3864345683 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc944095f40 con 0x7fc944095520 2021-02-10T11:34:45.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.137 7fc94305f700 1 --2- 172.21.2.101:0/3864345683 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc944095520 0x7fc944095920 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 rx=0x7fc93c009940 tx=0x7fc93c00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:45.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.137 7fc93b7fe700 1 -- 172.21.2.101:0/3864345683 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc93c0011f0 con 0x7fc944095520 2021-02-10T11:34:45.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.137 7fc93b7fe700 1 -- 172.21.2.101:0/3864345683 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c003030 con 0x7fc944095520 2021-02-10T11:34:45.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.137 7fc93b7fe700 1 -- 172.21.2.101:0/3864345683 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc93c0017b0 con 0x7fc944095520 2021-02-10T11:34:45.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.137 7fc948be7700 1 -- 172.21.2.101:0/3864345683 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc944095f40 con 0x7fc944095520 2021-02-10T11:34:45.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.137 7fc948be7700 1 -- 172.21.2.101:0/3864345683 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9440964d0 con 0x7fc944095520 2021-02-10T11:34:45.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.138 7fc93b7fe700 1 -- 172.21.2.101:0/3864345683 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc93c0056b0 con 0x7fc944095520 2021-02-10T11:34:45.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.138 7fc93b7fe700 1 --2- 172.21.2.101:0/3864345683 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9280186c0 0x7fc92801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:45.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.138 7fc93b7fe700 1 -- 172.21.2.101:0/3864345683 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc93c032910 con 0x7fc944095520 2021-02-10T11:34:45.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.138 7fc94285e700 1 --2- 172.21.2.101:0/3864345683 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9280186c0 0x7fc92801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:45.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.138 7fc94285e700 1 --2- 172.21.2.101:0/3864345683 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9280186c0 0x7fc92801abc0 crc :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:45.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.168 7fc948be7700 1 -- 172.21.2.101:0/3864345683 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc944082980 con 0x7fc944095520 2021-02-10T11:34:45.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.171 7fc93b7fe700 1 -- 172.21.2.101:0/3864345683 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc93c000f40 con 0x7fc944095520 2021-02-10T11:34:45.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.302 7fc948be7700 1 -- 172.21.2.101:0/3864345683 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fc944082980 con 0x7fc944095520 2021-02-10T11:34:45.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.305 7fc93b7fe700 1 -- 172.21.2.101:0/3864345683 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v38) v1 ==== 184+0+0 (secure 0 0 0) 0x7fc93c005a60 con 0x7fc944095520 2021-02-10T11:34:45.318 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.0 2021-02-10T11:34:45.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.311 7fc948be7700 1 -- 172.21.2.101:0/3864345683 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9280186c0 msgr2=0x7fc92801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:45.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.311 7fc948be7700 1 --2- 172.21.2.101:0/3864345683 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9280186c0 0x7fc92801abc0 crc :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:45.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.311 7fc948be7700 1 -- 172.21.2.101:0/3864345683 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc944095520 msgr2=0x7fc944095920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:45.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.311 7fc948be7700 1 --2- 172.21.2.101:0/3864345683 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc944095520 0x7fc944095920 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 rx=0x7fc93c009940 tx=0x7fc93c00bcf0).stop 2021-02-10T11:34:45.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.311 7fc948be7700 1 -- 172.21.2.101:0/3864345683 shutdown_connections 2021-02-10T11:34:45.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.311 7fc948be7700 1 --2- 172.21.2.101:0/3864345683 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9280186c0 0x7fc92801abc0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:45.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.311 7fc948be7700 1 --2- 172.21.2.101:0/3864345683 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc94419a470 0x7fc94408cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:45.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.311 7fc948be7700 1 --2- 172.21.2.101:0/3864345683 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc944095520 0x7fc944095920 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:45.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.311 7fc948be7700 1 --2- 172.21.2.101:0/3864345683 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc94419a070 0x7fc94408c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:45.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.311 7fc948be7700 1 -- 172.21.2.101:0/3864345683 >> 172.21.2.101:0/3864345683 conn(0x7fc9441a6190 msgr2=0x7fc9440949b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:45.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.311 7fc948be7700 1 -- 172.21.2.101:0/3864345683 shutdown_connections 2021-02-10T11:34:45.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.311 7fc948be7700 1 -- 172.21.2.101:0/3864345683 wait complete. 2021-02-10T11:34:45.329 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:34:45.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.743 7f8d06162700 1 Processor -- start 2021-02-10T11:34:45.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.743 7f8d06162700 1 -- start start 2021-02-10T11:34:45.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.743 7f8d06162700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8d000d1880 0x7f8d001b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:45.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.743 7f8d06162700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d000fe1d0 con 0x7f8d000cff80 2021-02-10T11:34:45.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.743 7f8d06162700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d00109a70 con 0x7f8d001b0af0 2021-02-10T11:34:45.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.743 7f8d06162700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d00088070 con 0x7f8d000d1880 2021-02-10T11:34:45.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.743 7f8cfffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8d000cff80 0x7f8d00109140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53118/0 (socket says 172.21.2.101:53118) 2021-02-10T11:34:45.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.743 7f8cfffff700 1 -- 172.21.2.101:0/4260647118 learned_addr learned my addr 172.21.2.101:0/4260647118 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:45.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.744 7f8cfe7fc700 1 -- 172.21.2.101:0/4260647118 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048207519 0 0) 0x7f8cf8001500 con 0x7f8d000cff80 2021-02-10T11:34:45.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.744 7f8cfe7fc700 1 -- 172.21.2.101:0/4260647118 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cec002fe0 con 0x7f8d000cff80 2021-02-10T11:34:45.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.744 7f8cff7fe700 1 --2- 172.21.2.101:0/4260647118 >> v2:172.21.2.108:3300/0 conn(0x7f8d000d1880 0x7f8d001b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:45.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.744 7f8cfe7fc700 1 -- 172.21.2.101:0/4260647118 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3011905903 0 0) 0x7f8cf8001460 con 0x7f8d000cff80 2021-02-10T11:34:45.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.744 7f8cfe7fc700 1 -- 172.21.2.101:0/4260647118 >> v2:172.21.2.108:3300/0 conn(0x7f8d000d1880 msgr2=0x7f8d001b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:45.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.744 7f8cfe7fc700 1 --2- 172.21.2.101:0/4260647118 >> v2:172.21.2.108:3300/0 conn(0x7f8d000d1880 0x7f8d001b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:45.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.744 7f8cfe7fc700 1 -- 172.21.2.101:0/4260647118 >> v1:172.21.2.108:6789/0 conn(0x7f8d001b0af0 legacy=0x7f8d00108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:45.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.744 7f8cfe7fc700 1 -- 172.21.2.101:0/4260647118 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d00088260 con 0x7f8d000cff80 2021-02-10T11:34:45.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.744 7f8cfe7fc700 1 -- 172.21.2.101:0/4260647118 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f8cf8001980 con 0x7f8d000cff80 2021-02-10T11:34:45.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.744 7f8cfe7fc700 1 -- 172.21.2.101:0/4260647118 >> v1:172.21.2.101:6789/0 conn(0x7f8d000cff80 legacy=0x7f8d00109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:45.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.744 7f8cfe7fc700 1 --2- 172.21.2.101:0/4260647118 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8cec003b90 0x7f8cec006000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:45.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.744 7f8cfe7fc700 1 --2- 172.21.2.101:0/4260647118 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8cec006820 0x7f8cec008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:45.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.744 7f8cfe7fc700 1 --2- 172.21.2.101:0/4260647118 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8cec009640 0x7f8cec00bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:45.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.744 7f8cfe7fc700 1 -- 172.21.2.101:0/4260647118 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec00c390 con 0x7f8cec009640 2021-02-10T11:34:45.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.744 7f8cfe7fc700 1 -- 172.21.2.101:0/4260647118 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec00c5b0 con 0x7f8cec003b90 2021-02-10T11:34:45.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.744 7f8cfe7fc700 1 -- 172.21.2.101:0/4260647118 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec00c7d0 con 0x7f8cec006820 2021-02-10T11:34:45.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.744 7f8cff7fe700 1 --2- 172.21.2.101:0/4260647118 >> v2:172.21.2.108:3300/0 conn(0x7f8d000d1880 0x7f8d001b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:34:45.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.744 7f8cfffff700 1 --2- 172.21.2.101:0/4260647118 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8cec009640 0x7f8cec00bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:45.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.745 7f8cff7fe700 1 --2- 172.21.2.101:0/4260647118 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8cec003b90 0x7f8cec006000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:45.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.745 7f8cfeffd700 1 --2- 172.21.2.101:0/4260647118 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8cec006820 0x7f8cec008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:45.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.745 7f8cfffff700 1 -- 172.21.2.101:0/4260647118 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8cec006820 msgr2=0x7f8cec008d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:45.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.745 7f8cfffff700 1 --2- 172.21.2.101:0/4260647118 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8cec006820 0x7f8cec008d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:45.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.745 7f8cfffff700 1 -- 172.21.2.101:0/4260647118 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8cec003b90 msgr2=0x7f8cec006000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:45.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.745 7f8cfffff700 1 --2- 172.21.2.101:0/4260647118 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8cec003b90 0x7f8cec006000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:45.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.745 7f8cfffff700 1 -- 172.21.2.101:0/4260647118 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cec00ca70 con 0x7f8cec009640 2021-02-10T11:34:45.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.745 7f8cfffff700 1 -- 172.21.2.101:0/4260647118 reap_dead start 2021-02-10T11:34:45.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.745 7f8cfffff700 1 --2- 172.21.2.101:0/4260647118 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8cec009640 0x7f8cec00bb40 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7f8cf804be20 tx=0x7f8cf804e310).ready entity=mon.0 client_cookie=f4ea6638be1484d6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:45.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.745 7f8cfe7fc700 1 -- 172.21.2.101:0/4260647118 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8cf804e700 con 0x7f8cec009640 2021-02-10T11:34:45.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.745 7f8cfe7fc700 1 -- 172.21.2.101:0/4260647118 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf804ba80 con 0x7f8cec009640 2021-02-10T11:34:45.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.745 7f8cfe7fc700 1 -- 172.21.2.101:0/4260647118 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8cf804ebb0 con 0x7f8cec009640 2021-02-10T11:34:45.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.745 7f8d06162700 1 -- 172.21.2.101:0/4260647118 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8cec009640 msgr2=0x7f8cec00bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:45.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.745 7f8d06162700 1 --2- 172.21.2.101:0/4260647118 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8cec009640 0x7f8cec00bb40 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7f8cf804be20 tx=0x7f8cf804e310).stop 2021-02-10T11:34:45.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.746 7f8d06162700 1 -- 172.21.2.101:0/4260647118 shutdown_connections 2021-02-10T11:34:45.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.746 7f8d06162700 1 --2- 172.21.2.101:0/4260647118 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8cec009640 0x7f8cec00bb40 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:45.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.746 7f8d06162700 1 -- 172.21.2.101:0/4260647118 >> 172.21.2.101:0/4260647118 conn(0x7f8d000ce430 msgr2=0x7f8d000d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:45.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.746 7f8d06162700 1 -- 172.21.2.101:0/4260647118 shutdown_connections 2021-02-10T11:34:45.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.746 7f8d06162700 1 -- 172.21.2.101:0/4260647118 wait complete. 2021-02-10T11:34:45.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.761 7f8d06162700 1 Processor -- start 2021-02-10T11:34:45.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.761 7f8d06162700 1 -- start start 2021-02-10T11:34:45.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.761 7f8d06162700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d001a0580 0x7f8d00086fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:45.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.761 7f8d06162700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d000b66b0 0x7f8d000b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:45.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.761 7f8d06162700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d00082bb0 0x7f8d00082fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:45.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.761 7f8d06162700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d000875d0 con 0x7f8d000b66b0 2021-02-10T11:34:45.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.761 7f8d06162700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d000835d0 con 0x7f8d001a0580 2021-02-10T11:34:45.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.761 7f8d06162700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d000837f0 con 0x7f8d00082bb0 2021-02-10T11:34:45.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.762 7f8cfeffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d000b66b0 0x7f8d000b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:45.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.762 7f8cfffff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d00082bb0 0x7f8d00082fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:45.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.762 7f8cfeffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d000b66b0 0x7f8d000b6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52636/0 (socket says 172.21.2.101:52636) 2021-02-10T11:34:45.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.762 7f8cfeffd700 1 -- 172.21.2.101:0/2863258140 learned_addr learned my addr 172.21.2.101:0/2863258140 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:45.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.762 7f8cff7fe700 1 --2- 172.21.2.101:0/2863258140 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d001a0580 0x7f8d00086fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:45.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.762 7f8cfffff700 1 -- 172.21.2.101:0/2863258140 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d001a0580 msgr2=0x7f8d00086fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:45.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.762 7f8cfffff700 1 --2- 172.21.2.101:0/2863258140 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d001a0580 0x7f8d00086fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:45.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.762 7f8cfffff700 1 -- 172.21.2.101:0/2863258140 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d000b66b0 msgr2=0x7f8d000b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.762 7f8cfffff700 1 --2- 172.21.2.101:0/2863258140 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d000b66b0 0x7f8d000b6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.762 7f8cfffff700 1 -- 172.21.2.101:0/2863258140 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d00083a10 con 0x7f8d00082bb0 2021-02-10T11:34:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.762 7f8cfffff700 1 --2- 172.21.2.101:0/2863258140 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d00082bb0 0x7f8d00082fb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f8cf8001270 tx=0x7f8cf804ec00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.763 7f8cf7fff700 1 -- 172.21.2.101:0/2863258140 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8cf804f070 con 0x7f8d00082bb0 2021-02-10T11:34:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.763 7f8cf7fff700 1 -- 172.21.2.101:0/2863258140 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf804f2f0 con 0x7f8d00082bb0 2021-02-10T11:34:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.763 7f8d06162700 1 -- 172.21.2.101:0/2863258140 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d00083a10 con 0x7f8d00082bb0 2021-02-10T11:34:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.763 7f8d06162700 1 -- 172.21.2.101:0/2863258140 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d00083fa0 con 0x7f8d00082bb0 2021-02-10T11:34:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.763 7f8cf7fff700 1 -- 172.21.2.101:0/2863258140 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8cf804f810 con 0x7f8d00082bb0 2021-02-10T11:34:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.763 7f8cf7fff700 1 -- 172.21.2.101:0/2863258140 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8cf8001560 con 0x7f8d00082bb0 2021-02-10T11:34:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.763 7f8cf7fff700 1 --2- 172.21.2.101:0/2863258140 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ce40186c0 0x7f8ce401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.763 7f8cf7fff700 1 -- 172.21.2.101:0/2863258140 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8cf80767f0 con 0x7f8d00082bb0 2021-02-10T11:34:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.763 7f8cff7fe700 1 --2- 172.21.2.101:0/2863258140 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ce40186c0 0x7f8ce401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.764 7f8cff7fe700 1 --2- 172.21.2.101:0/2863258140 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ce40186c0 0x7f8ce401abc0 crc :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:45.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.793 7f8d06162700 1 -- 172.21.2.101:0/2863258140 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d00082980 con 0x7f8d00082bb0 2021-02-10T11:34:45.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.796 7f8cf7fff700 1 -- 172.21.2.101:0/2863258140 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8cf8000f40 con 0x7f8d00082bb0 2021-02-10T11:34:45.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.926 7f8d06162700 1 -- 172.21.2.101:0/2863258140 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f8d00082980 con 0x7f8d00082bb0 2021-02-10T11:34:45.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.932 7f8cf7fff700 1 -- 172.21.2.101:0/2863258140 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v39) v1 ==== 184+0+0 (secure 0 0 0) 0x7f8cf80637a0 con 0x7f8d00082bb0 2021-02-10T11:34:45.943 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.1 2021-02-10T11:34:45.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.935 7f8d06162700 1 -- 172.21.2.101:0/2863258140 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ce40186c0 msgr2=0x7f8ce401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:45.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.935 7f8d06162700 1 --2- 172.21.2.101:0/2863258140 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ce40186c0 0x7f8ce401abc0 crc :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:45.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.935 7f8d06162700 1 -- 172.21.2.101:0/2863258140 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d00082bb0 msgr2=0x7f8d00082fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:45.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.935 7f8d06162700 1 --2- 172.21.2.101:0/2863258140 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d00082bb0 0x7f8d00082fb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f8cf8001270 tx=0x7f8cf804ec00).stop 2021-02-10T11:34:45.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.935 7f8d06162700 1 -- 172.21.2.101:0/2863258140 shutdown_connections 2021-02-10T11:34:45.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.935 7f8d06162700 1 --2- 172.21.2.101:0/2863258140 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ce40186c0 0x7f8ce401abc0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:45.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.935 7f8d06162700 1 --2- 172.21.2.101:0/2863258140 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d00082bb0 0x7f8d00082fb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:45.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.935 7f8d06162700 1 --2- 172.21.2.101:0/2863258140 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d000b66b0 0x7f8d000b6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:45.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.935 7f8d06162700 1 --2- 172.21.2.101:0/2863258140 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d001a0580 0x7f8d00086fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:45.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.935 7f8d06162700 1 -- 172.21.2.101:0/2863258140 >> 172.21.2.101:0/2863258140 conn(0x7f8d000afc60 msgr2=0x7f8d000b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:45.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.935 7f8d06162700 1 -- 172.21.2.101:0/2863258140 shutdown_connections 2021-02-10T11:34:45.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:45.936 7f8d06162700 1 -- 172.21.2.101:0/2863258140 wait complete. 2021-02-10T11:34:45.953 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:34:46.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.356 7f99c8635700 1 Processor -- start 2021-02-10T11:34:46.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.356 7f99c8635700 1 -- start start 2021-02-10T11:34:46.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.357 7f99c8635700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f99c00d1880 0x7f99c01b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:46.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.357 7f99c8635700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99c01b0af0 0x7f99c0108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:46.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.357 7f99c8635700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c00fe1d0 con 0x7f99c00d1c80 2021-02-10T11:34:46.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.357 7f99c8635700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c0109a70 con 0x7f99c00d1880 2021-02-10T11:34:46.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.357 7f99c8635700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c0109760 con 0x7f99c01b0af0 2021-02-10T11:34:46.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.357 7f99c6c52700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f99c00d1c80 0x7f99c0108f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53136/0 (socket says 172.21.2.101:53136) 2021-02-10T11:34:46.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.357 7f99c6c52700 1 -- 172.21.2.101:0/3249983228 learned_addr learned my addr 172.21.2.101:0/3249983228 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:46.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.357 7f99c6451700 1 --2- 172.21.2.101:0/3249983228 >> v2:172.21.2.101:3300/0 conn(0x7f99c00d1880 0x7f99c01b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:46.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.357 7f99c6451700 1 -- 172.21.2.101:0/3249983228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99c01b0af0 msgr2=0x7f99c0108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:46.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.357 7f99c6451700 1 --2- 172.21.2.101:0/3249983228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99c01b0af0 0x7f99c0108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:46.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.357 7f99c6451700 1 -- 172.21.2.101:0/3249983228 >> v1:172.21.2.101:6789/0 conn(0x7f99c00d1c80 legacy=0x7f99c0108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:46.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.358 7f99c6451700 1 -- 172.21.2.101:0/3249983228 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c0088070 con 0x7f99c00d1880 2021-02-10T11:34:46.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.358 7f99c6451700 1 --2- 172.21.2.101:0/3249983228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99c00d1880 0x7f99c01b04d0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7f99b4006650 tx=0x7f99b4009a80).ready entity=mon.0 client_cookie=9c3ee3e958d0c9cf server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:46.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.358 7f99c544f700 1 -- 172.21.2.101:0/3249983228 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f99b400c0c0 con 0x7f99c00d1880 2021-02-10T11:34:46.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.358 7f99c544f700 1 -- 172.21.2.101:0/3249983228 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b400c340 con 0x7f99c00d1880 2021-02-10T11:34:46.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.358 7f99c544f700 1 -- 172.21.2.101:0/3249983228 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f99b400c860 con 0x7f99c00d1880 2021-02-10T11:34:46.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.358 7f99c8635700 1 -- 172.21.2.101:0/3249983228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99c00d1880 msgr2=0x7f99c01b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:46.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.358 7f99c8635700 1 --2- 172.21.2.101:0/3249983228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99c00d1880 0x7f99c01b04d0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7f99b4006650 tx=0x7f99b4009a80).stop 2021-02-10T11:34:46.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.359 7f99c8635700 1 -- 172.21.2.101:0/3249983228 shutdown_connections 2021-02-10T11:34:46.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.359 7f99c8635700 1 --2- 172.21.2.101:0/3249983228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99c01b0af0 0x7f99c0108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:46.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.359 7f99c8635700 1 --2- 172.21.2.101:0/3249983228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99c00d1880 0x7f99c01b04d0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:46.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.359 7f99c8635700 1 -- 172.21.2.101:0/3249983228 >> 172.21.2.101:0/3249983228 conn(0x7f99c00ce430 msgr2=0x7f99c00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:46.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.359 7f99c8635700 1 -- 172.21.2.101:0/3249983228 shutdown_connections 2021-02-10T11:34:46.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.359 7f99c8635700 1 -- 172.21.2.101:0/3249983228 wait complete. 2021-02-10T11:34:46.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.374 7f99c8635700 1 Processor -- start 2021-02-10T11:34:46.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.374 7f99c8635700 1 -- start start 2021-02-10T11:34:46.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.374 7f99c8635700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99c01ac650 0x7f99c009ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:46.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.374 7f99c8635700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99c009d0b0 0x7f99c01b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:46.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.375 7f99c8635700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99c01b18d0 0x7f99c01b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:46.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.375 7f99c8635700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c01b2170 con 0x7f99c01ac650 2021-02-10T11:34:46.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.375 7f99c8635700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c01b1cd0 con 0x7f99c01b18d0 2021-02-10T11:34:46.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.375 7f99c8635700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c01b1eb0 con 0x7f99c009d0b0 2021-02-10T11:34:46.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.375 7f99c6451700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99c01ac650 0x7f99c009ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:46.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.375 7f99c6451700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99c01ac650 0x7f99c009ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52650/0 (socket says 172.21.2.101:52650) 2021-02-10T11:34:46.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.375 7f99c6451700 1 -- 172.21.2.101:0/1787043297 learned_addr learned my addr 172.21.2.101:0/1787043297 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:46.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.375 7f99c6c52700 1 --2- 172.21.2.101:0/1787043297 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99c01b18d0 0x7f99c01b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:46.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.375 7f99c6451700 1 -- 172.21.2.101:0/1787043297 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99c009d0b0 msgr2=0x7f99c01b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:46.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.375 7f99c5c50700 1 --2- 172.21.2.101:0/1787043297 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99c009d0b0 0x7f99c01b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:46.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.375 7f99c6451700 1 --2- 172.21.2.101:0/1787043297 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99c009d0b0 0x7f99c01b12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:46.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.375 7f99c6451700 1 -- 172.21.2.101:0/1787043297 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99c01b18d0 msgr2=0x7f99c01b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:46.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.375 7f99c6451700 1 --2- 172.21.2.101:0/1787043297 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99c01b18d0 0x7f99c01b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:46.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.375 7f99c6451700 1 -- 172.21.2.101:0/1787043297 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c01b2d10 con 0x7f99c01ac650 2021-02-10T11:34:46.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.375 7f99c6c52700 1 --2- 172.21.2.101:0/1787043297 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99c01b18d0 0x7f99c01b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:34:46.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.375 7f99c5c50700 1 --2- 172.21.2.101:0/1787043297 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99c009d0b0 0x7f99c01b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:34:46.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.375 7f99c6451700 1 --2- 172.21.2.101:0/1787043297 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99c01ac650 0x7f99c009ca90 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7f99b4009630 tx=0x7f99b400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:46.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.376 7f99b2ffd700 1 -- 172.21.2.101:0/1787043297 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f99b40093a0 con 0x7f99c01ac650 2021-02-10T11:34:46.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.376 7f99b2ffd700 1 -- 172.21.2.101:0/1787043297 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b400bd70 con 0x7f99c01ac650 2021-02-10T11:34:46.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.376 7f99b2ffd700 1 -- 172.21.2.101:0/1787043297 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f99b4005560 con 0x7f99c01ac650 2021-02-10T11:34:46.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.376 7f99c8635700 1 -- 172.21.2.101:0/1787043297 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c01b2d10 con 0x7f99c01ac650 2021-02-10T11:34:46.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.376 7f99c8635700 1 -- 172.21.2.101:0/1787043297 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c01b32a0 con 0x7f99c01ac650 2021-02-10T11:34:46.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.376 7f99b2ffd700 1 -- 172.21.2.101:0/1787043297 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f99b40052e0 con 0x7f99c01ac650 2021-02-10T11:34:46.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.377 7f99b2ffd700 1 --2- 172.21.2.101:0/1787043297 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f99a80186c0 0x7f99a801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:46.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.377 7f99b2ffd700 1 -- 172.21.2.101:0/1787043297 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f99b4032550 con 0x7f99c01ac650 2021-02-10T11:34:46.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.377 7f99c5c50700 1 --2- 172.21.2.101:0/1787043297 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f99a80186c0 0x7f99a801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:46.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.377 7f99c5c50700 1 --2- 172.21.2.101:0/1787043297 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f99a80186c0 0x7f99a801abc0 crc :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:46.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.407 7f99c8635700 1 -- 172.21.2.101:0/1787043297 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c00fe1d0 con 0x7f99c01ac650 2021-02-10T11:34:46.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.409 7f99b2ffd700 1 -- 172.21.2.101:0/1787043297 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f99b4009340 con 0x7f99c01ac650 2021-02-10T11:34:46.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.539 7f99c8635700 1 -- 172.21.2.101:0/1787043297 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f99c00fe1d0 con 0x7f99c01ac650 2021-02-10T11:34:46.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.543 7f99b2ffd700 1 -- 172.21.2.101:0/1787043297 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v40) v1 ==== 184+0+0 (secure 0 0 0) 0x7f99b401f4e0 con 0x7f99c01ac650 2021-02-10T11:34:46.556 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.2 2021-02-10T11:34:46.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.548 7f99c8635700 1 -- 172.21.2.101:0/1787043297 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f99a80186c0 msgr2=0x7f99a801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:46.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.548 7f99c8635700 1 --2- 172.21.2.101:0/1787043297 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f99a80186c0 0x7f99a801abc0 crc :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:46.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.548 7f99c8635700 1 -- 172.21.2.101:0/1787043297 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99c01ac650 msgr2=0x7f99c009ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:46.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.548 7f99c8635700 1 --2- 172.21.2.101:0/1787043297 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99c01ac650 0x7f99c009ca90 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7f99b4009630 tx=0x7f99b400b980).stop 2021-02-10T11:34:46.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.548 7f99c8635700 1 -- 172.21.2.101:0/1787043297 shutdown_connections 2021-02-10T11:34:46.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.548 7f99c8635700 1 --2- 172.21.2.101:0/1787043297 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f99a80186c0 0x7f99a801abc0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:46.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.548 7f99c8635700 1 --2- 172.21.2.101:0/1787043297 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99c009d0b0 0x7f99c01b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:46.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.548 7f99c8635700 1 --2- 172.21.2.101:0/1787043297 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99c01b18d0 0x7f99c01b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:46.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.548 7f99c8635700 1 --2- 172.21.2.101:0/1787043297 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99c01ac650 0x7f99c009ca90 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:46.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.548 7f99c8635700 1 -- 172.21.2.101:0/1787043297 >> 172.21.2.101:0/1787043297 conn(0x7f99c019c580 msgr2=0x7f99c01a9030 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:46.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.549 7f99c8635700 1 -- 172.21.2.101:0/1787043297 shutdown_connections 2021-02-10T11:34:46.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.549 7f99c8635700 1 -- 172.21.2.101:0/1787043297 wait complete. 2021-02-10T11:34:46.567 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:34:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.980 7f37ce590700 1 Processor -- start 2021-02-10T11:34:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.980 7f37ce590700 1 -- start start 2021-02-10T11:34:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.981 7f37ce590700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f37c80d1880 0x7f37c81b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.981 7f37ce590700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37c80cff80 0x7f37c8109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.981 7f37ce590700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c8109a70 con 0x7f37c81b0af0 2021-02-10T11:34:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.981 7f37ce590700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c8109760 con 0x7f37c80d1880 2021-02-10T11:34:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.981 7f37ce590700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c8088070 con 0x7f37c80cff80 2021-02-10T11:34:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.982 7f37ccbad700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37c80cff80 0x7f37c8109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.982 7f37c77fe700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f37c81b0af0 0x7f37c8108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:46560/0 (socket says 172.21.2.101:46560) 2021-02-10T11:34:47.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.982 7f37c7fff700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f37c80d1880 0x7f37c81b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:47.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.982 7f37c77fe700 1 -- 172.21.2.101:0/347704596 learned_addr learned my addr 172.21.2.101:0/347704596 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:47.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.983 7f37ccbad700 1 -- 172.21.2.101:0/347704596 >> v2:172.21.2.108:3300/0 conn(0x7f37c80d1880 msgr2=0x7f37c81b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:47.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.983 7f37ccbad700 1 --2- 172.21.2.101:0/347704596 >> v2:172.21.2.108:3300/0 conn(0x7f37c80d1880 0x7f37c81b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:47.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.983 7f37ccbad700 1 -- 172.21.2.101:0/347704596 >> v1:172.21.2.108:6789/0 conn(0x7f37c81b0af0 legacy=0x7f37c8108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:47.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.983 7f37ccbad700 1 -- 172.21.2.101:0/347704596 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c80d1c80 con 0x7f37c80cff80 2021-02-10T11:34:47.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.983 7f37c6ffd700 1 -- 172.21.2.101:0/347704596 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1159577649 0 0) 0x7f37bc0010b0 con 0x7f37c81b0af0 2021-02-10T11:34:47.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.983 7f37c7fff700 1 --2- 172.21.2.101:0/347704596 >> v2:172.21.2.108:3300/0 conn(0x7f37c80d1880 0x7f37c81b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:34:47.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.983 7f37ccbad700 1 --2- 172.21.2.101:0/347704596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37c80cff80 0x7f37c8109140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f37c0006e50 tx=0x7f37c0009e70).ready entity=mon.2 client_cookie=5d50b6cf46e26ccc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:47.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.983 7f37c6ffd700 1 -- 172.21.2.101:0/347704596 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f37c000b510 con 0x7f37c80cff80 2021-02-10T11:34:47.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.983 7f37c6ffd700 1 -- 172.21.2.101:0/347704596 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c0006790 con 0x7f37c80cff80 2021-02-10T11:34:47.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.983 7f37c6ffd700 1 -- 172.21.2.101:0/347704596 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f37c000b940 con 0x7f37c80cff80 2021-02-10T11:34:47.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.983 7f37ce590700 1 -- 172.21.2.101:0/347704596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37c80cff80 msgr2=0x7f37c8109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:47.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.983 7f37ce590700 1 --2- 172.21.2.101:0/347704596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37c80cff80 0x7f37c8109140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f37c0006e50 tx=0x7f37c0009e70).stop 2021-02-10T11:34:47.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.984 7f37ce590700 1 -- 172.21.2.101:0/347704596 shutdown_connections 2021-02-10T11:34:47.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.984 7f37ce590700 1 --2- 172.21.2.101:0/347704596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37c80cff80 0x7f37c8109140 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:47.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.984 7f37ce590700 1 --2- 172.21.2.101:0/347704596 >> v2:172.21.2.108:3300/0 conn(0x7f37c80d1880 0x7f37c81b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:47.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.984 7f37ce590700 1 -- 172.21.2.101:0/347704596 >> 172.21.2.101:0/347704596 conn(0x7f37c80ce430 msgr2=0x7f37c80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:47.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.984 7f37ce590700 1 -- 172.21.2.101:0/347704596 shutdown_connections 2021-02-10T11:34:47.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.984 7f37ce590700 1 -- 172.21.2.101:0/347704596 wait complete. 2021-02-10T11:34:47.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.999 7f37ce590700 1 Processor -- start 2021-02-10T11:34:47.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.999 7f37ce590700 1 -- start start 2021-02-10T11:34:47.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.999 7f37ce590700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37c808d090 0x7f37c808c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:47.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.999 7f37ce590700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37c808d490 0x7f37c80beec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:47.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.999 7f37ce590700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37c80bf4e0 0x7f37c809fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:47.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.999 7f37ce590700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c80bf8e0 con 0x7f37c80bf4e0 2021-02-10T11:34:47.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.999 7f37ce590700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c80a0c90 con 0x7f37c808d490 2021-02-10T11:34:47.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.999 7f37ce590700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c80a0eb0 con 0x7f37c808d090 2021-02-10T11:34:47.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.999 7f37ccbad700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37c80bf4e0 0x7f37c809fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:47.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.999 7f37ccbad700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37c80bf4e0 0x7f37c809fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52664/0 (socket says 172.21.2.101:52664) 2021-02-10T11:34:47.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:46.999 7f37ccbad700 1 -- 172.21.2.101:0/3149080069 learned_addr learned my addr 172.21.2.101:0/3149080069 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:47.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.000 7f37ccbad700 1 -- 172.21.2.101:0/3149080069 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37c808d090 msgr2=0x7f37c808c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:47.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.000 7f37ccbad700 1 --2- 172.21.2.101:0/3149080069 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37c808d090 0x7f37c808c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:47.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.000 7f37ccbad700 1 -- 172.21.2.101:0/3149080069 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37c808d490 msgr2=0x7f37c80beec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:47.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.000 7f37ccbad700 1 --2- 172.21.2.101:0/3149080069 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37c808d490 0x7f37c80beec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:47.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.000 7f37ccbad700 1 -- 172.21.2.101:0/3149080069 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c80a0260 con 0x7f37c80bf4e0 2021-02-10T11:34:47.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.000 7f37ccbad700 1 --2- 172.21.2.101:0/3149080069 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37c80bf4e0 0x7f37c809fc40 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 rx=0x7f37c0006c50 tx=0x7f37c000bd20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:47.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.000 7f37c4ff9700 1 -- 172.21.2.101:0/3149080069 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f37c00093d0 con 0x7f37c80bf4e0 2021-02-10T11:34:47.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.000 7f37c4ff9700 1 -- 172.21.2.101:0/3149080069 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c000be60 con 0x7f37c80bf4e0 2021-02-10T11:34:47.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.000 7f37c4ff9700 1 -- 172.21.2.101:0/3149080069 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f37c000c4e0 con 0x7f37c80bf4e0 2021-02-10T11:34:47.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.000 7f37ce590700 1 -- 172.21.2.101:0/3149080069 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c80a0260 con 0x7f37c80bf4e0 2021-02-10T11:34:47.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.000 7f37ce590700 1 -- 172.21.2.101:0/3149080069 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c80a07f0 con 0x7f37c80bf4e0 2021-02-10T11:34:47.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.001 7f37c4ff9700 1 -- 172.21.2.101:0/3149080069 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f37c00093d0 con 0x7f37c80bf4e0 2021-02-10T11:34:47.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.001 7f37c4ff9700 1 --2- 172.21.2.101:0/3149080069 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37a80186c0 0x7f37a801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:47.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.001 7f37c4ff9700 1 -- 172.21.2.101:0/3149080069 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f37c0032910 con 0x7f37c80bf4e0 2021-02-10T11:34:47.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.001 7f37c7fff700 1 --2- 172.21.2.101:0/3149080069 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37a80186c0 0x7f37a801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:47.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.001 7f37c7fff700 1 --2- 172.21.2.101:0/3149080069 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37a80186c0 0x7f37a801abc0 crc :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:47.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.031 7f37ce590700 1 -- 172.21.2.101:0/3149080069 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c808cb60 con 0x7f37c80bf4e0 2021-02-10T11:34:47.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.033 7f37c4ff9700 1 -- 172.21.2.101:0/3149080069 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f37c0026fb0 con 0x7f37c80bf4e0 2021-02-10T11:34:47.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.164 7f37ce590700 1 -- 172.21.2.101:0/3149080069 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f37c808cb60 con 0x7f37c80bf4e0 2021-02-10T11:34:47.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.167 7f37c4ff9700 1 -- 172.21.2.101:0/3149080069 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v41) v1 ==== 184+0+0 (secure 0 0 0) 0x7f37c00263e0 con 0x7f37c80bf4e0 2021-02-10T11:34:47.177 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.3 2021-02-10T11:34:47.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.169 7f37ce590700 1 -- 172.21.2.101:0/3149080069 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37a80186c0 msgr2=0x7f37a801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:47.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.169 7f37ce590700 1 --2- 172.21.2.101:0/3149080069 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37a80186c0 0x7f37a801abc0 crc :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:47.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.169 7f37ce590700 1 -- 172.21.2.101:0/3149080069 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37c80bf4e0 msgr2=0x7f37c809fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:47.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.169 7f37ce590700 1 --2- 172.21.2.101:0/3149080069 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37c80bf4e0 0x7f37c809fc40 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 rx=0x7f37c0006c50 tx=0x7f37c000bd20).stop 2021-02-10T11:34:47.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.169 7f37ce590700 1 -- 172.21.2.101:0/3149080069 shutdown_connections 2021-02-10T11:34:47.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.169 7f37ce590700 1 --2- 172.21.2.101:0/3149080069 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37c80bf4e0 0x7f37c809fc40 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:47.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.169 7f37ce590700 1 --2- 172.21.2.101:0/3149080069 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37c808d490 0x7f37c80beec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:47.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.169 7f37ce590700 1 --2- 172.21.2.101:0/3149080069 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37a80186c0 0x7f37a801abc0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:47.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.169 7f37ce590700 1 --2- 172.21.2.101:0/3149080069 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37c808d090 0x7f37c808c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:47.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.169 7f37ce590700 1 -- 172.21.2.101:0/3149080069 >> 172.21.2.101:0/3149080069 conn(0x7f37c8087390 msgr2=0x7f37c80a6770 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:47.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.169 7f37ce590700 1 -- 172.21.2.101:0/3149080069 shutdown_connections 2021-02-10T11:34:47.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.170 7f37ce590700 1 -- 172.21.2.101:0/3149080069 wait complete. 2021-02-10T11:34:47.187 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:34:47.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.587 7f8aa2db0700 1 Processor -- start 2021-02-10T11:34:47.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.587 7f8aa2db0700 1 -- start start 2021-02-10T11:34:47.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.588 7f8aa2db0700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8a9c0d1870 0x7f8a9c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:47.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.588 7f8aa2db0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a9c1b0a10 0x7f8a9c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:47.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.588 7f8aa2db0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a9c0f0b10 con 0x7f8a9c0d1c70 2021-02-10T11:34:47.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.588 7f8aa2db0700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a9c0e5ed0 con 0x7f8a9c0d1870 2021-02-10T11:34:47.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.588 7f8aa2db0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c0f0650 con 0x7f8a9c1b0a10 2021-02-10T11:34:47.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.588 7f8a9bfff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a9c1b0a10 0x7f8a9c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:47.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.588 7f8a9bfff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a9c1b0a10 0x7f8a9c0cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:45406/0 (socket says 172.21.2.101:45406) 2021-02-10T11:34:47.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.588 7f8a9bfff700 1 -- 172.21.2.101:0/840276316 learned_addr learned my addr 172.21.2.101:0/840276316 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:47.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.588 7f8aa13cd700 1 --1- 172.21.2.101:0/840276316 >> v1:172.21.2.108:6789/0 conn(0x7f8a9c0d1c70 0x7f8a9c0cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:46576/0 (socket says 172.21.2.101:46576) 2021-02-10T11:34:47.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.588 7f8aa0bcc700 1 --2- 172.21.2.101:0/840276316 >> v2:172.21.2.108:3300/0 conn(0x7f8a9c0d1870 0x7f8a9c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:47.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.589 7f8aa0bcc700 1 -- 172.21.2.101:0/840276316 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a9c1b0a10 msgr2=0x7f8a9c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:47.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.589 7f8aa0bcc700 1 --2- 172.21.2.101:0/840276316 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a9c1b0a10 0x7f8a9c0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:47.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.589 7f8aa0bcc700 1 -- 172.21.2.101:0/840276316 >> v1:172.21.2.108:6789/0 conn(0x7f8a9c0d1c70 legacy=0x7f8a9c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:47.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.589 7f8aa0bcc700 1 -- 172.21.2.101:0/840276316 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9c0f0870 con 0x7f8a9c0d1870 2021-02-10T11:34:47.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.589 7f8aa0bcc700 1 --2- 172.21.2.101:0/840276316 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a9c0d1870 0x7f8a9c1b03f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f8a8c006650 tx=0x7f8a8c009a80).ready entity=mon.1 client_cookie=13193aad6e5213e5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:47.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.589 7f8a9b7fe700 1 -- 172.21.2.101:0/840276316 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8a8c0009b0 con 0x7f8a9c0d1870 2021-02-10T11:34:47.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.589 7f8a9b7fe700 1 -- 172.21.2.101:0/840276316 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8c00c0c0 con 0x7f8a9c0d1870 2021-02-10T11:34:47.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.589 7f8a9b7fe700 1 -- 172.21.2.101:0/840276316 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8a8c00c510 con 0x7f8a9c0d1870 2021-02-10T11:34:47.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.589 7f8aa2db0700 1 -- 172.21.2.101:0/840276316 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a9c0d1870 msgr2=0x7f8a9c1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:47.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.589 7f8aa2db0700 1 --2- 172.21.2.101:0/840276316 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a9c0d1870 0x7f8a9c1b03f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f8a8c006650 tx=0x7f8a8c009a80).stop 2021-02-10T11:34:47.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.590 7f8aa2db0700 1 -- 172.21.2.101:0/840276316 shutdown_connections 2021-02-10T11:34:47.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.590 7f8aa2db0700 1 --2- 172.21.2.101:0/840276316 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a9c1b0a10 0x7f8a9c0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:47.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.590 7f8aa2db0700 1 --2- 172.21.2.101:0/840276316 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a9c0d1870 0x7f8a9c1b03f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:47.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.590 7f8aa2db0700 1 -- 172.21.2.101:0/840276316 >> 172.21.2.101:0/840276316 conn(0x7f8a9c0ce450 msgr2=0x7f8a9c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:47.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.590 7f8aa2db0700 1 -- 172.21.2.101:0/840276316 shutdown_connections 2021-02-10T11:34:47.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.590 7f8aa2db0700 1 -- 172.21.2.101:0/840276316 wait complete. 2021-02-10T11:34:47.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.605 7f8aa2db0700 1 Processor -- start 2021-02-10T11:34:47.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.605 7f8aa2db0700 1 -- start start 2021-02-10T11:34:47.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.605 7f8aa2db0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a9c0f0280 0x7f8a9c0a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:47.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.605 7f8aa2db0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a9c0f0680 0x7f8a9c0a2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:47.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.605 7f8aa2db0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a9c0ec180 0x7f8a9c0ec580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:47.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.605 7f8aa2db0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c0efd00 con 0x7f8a9c0f0280 2021-02-10T11:34:47.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.605 7f8aa2db0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c0ecba0 con 0x7f8a9c0f0680 2021-02-10T11:34:47.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.605 7f8aa2db0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c0ecd80 con 0x7f8a9c0ec180 2021-02-10T11:34:47.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.606 7f8aa0bcc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a9c0f0280 0x7f8a9c0a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:47.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.606 7f8aa0bcc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a9c0f0280 0x7f8a9c0a2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52676/0 (socket says 172.21.2.101:52676) 2021-02-10T11:34:47.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.606 7f8aa0bcc700 1 -- 172.21.2.101:0/170625317 learned_addr learned my addr 172.21.2.101:0/170625317 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:47.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.606 7f8aa0bcc700 1 -- 172.21.2.101:0/170625317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a9c0ec180 msgr2=0x7f8a9c0ec580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:47.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.606 7f8aa0bcc700 1 --2- 172.21.2.101:0/170625317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a9c0ec180 0x7f8a9c0ec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:47.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.606 7f8aa0bcc700 1 -- 172.21.2.101:0/170625317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a9c0f0680 msgr2=0x7f8a9c0a2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:47.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.606 7f8aa0bcc700 1 --2- 172.21.2.101:0/170625317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a9c0f0680 0x7f8a9c0a2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:47.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.606 7f8aa0bcc700 1 -- 172.21.2.101:0/170625317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9c0ecfa0 con 0x7f8a9c0f0280 2021-02-10T11:34:47.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.606 7f8aa0bcc700 1 --2- 172.21.2.101:0/170625317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a9c0f0280 0x7f8a9c0a2310 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 rx=0x7f8a8c009630 tx=0x7f8a8c00b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:47.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.606 7f8a997fa700 1 -- 172.21.2.101:0/170625317 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8a8c0093a0 con 0x7f8a9c0f0280 2021-02-10T11:34:47.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.606 7f8a997fa700 1 -- 172.21.2.101:0/170625317 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8c00bd70 con 0x7f8a9c0f0280 2021-02-10T11:34:47.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.606 7f8a997fa700 1 -- 172.21.2.101:0/170625317 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8a8c0054d0 con 0x7f8a9c0f0280 2021-02-10T11:34:47.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.606 7f8aa2db0700 1 -- 172.21.2.101:0/170625317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a9c0ecfa0 con 0x7f8a9c0f0280 2021-02-10T11:34:47.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.606 7f8aa2db0700 1 -- 172.21.2.101:0/170625317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a9c0ed530 con 0x7f8a9c0f0280 2021-02-10T11:34:47.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.607 7f8a997fa700 1 -- 172.21.2.101:0/170625317 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8a8c00a000 con 0x7f8a9c0f0280 2021-02-10T11:34:47.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.607 7f8a997fa700 1 --2- 172.21.2.101:0/170625317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a7c0186c0 0x7f8a7c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:47.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.607 7f8a997fa700 1 -- 172.21.2.101:0/170625317 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8a8c032560 con 0x7f8a9c0f0280 2021-02-10T11:34:47.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.607 7f8a9bfff700 1 --2- 172.21.2.101:0/170625317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a7c0186c0 0x7f8a7c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:47.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.607 7f8a9bfff700 1 --2- 172.21.2.101:0/170625317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a7c0186c0 0x7f8a7c01abc0 crc :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:47.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.638 7f8aa2db0700 1 -- 172.21.2.101:0/170625317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a9c0ebc60 con 0x7f8a9c0f0280 2021-02-10T11:34:47.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.641 7f8a997fa700 1 -- 172.21.2.101:0/170625317 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8a8c009340 con 0x7f8a9c0f0280 2021-02-10T11:34:47.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.769 7f8aa2db0700 1 -- 172.21.2.101:0/170625317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a9c0ebc60 con 0x7f8a9c0f0280 2021-02-10T11:34:47.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.770 7f8a997fa700 1 -- 172.21.2.101:0/170625317 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f8a8c00a000 con 0x7f8a9c0f0280 2021-02-10T11:34:47.780 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:47.780 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1297756180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1297756180},{"type":"v1","addr":"172.21.2.101:6835","nonce":1297756180}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:34:41.088085","modified":"2021-02-10 11:34:43.450805","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6400":{"gid":6400,"name":"b","rank":0,"incarnation":44,"state":"up:active","state_seq":4,"addr":"172.21.2.108:6833/3585539482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3585539482},{"type":"v1","addr":"172.21.2.108:6833","nonce":3585539482}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-02-10T11:34:47.780 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 46 2021-02-10T11:34:47.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.772 7f8aa2db0700 1 -- 172.21.2.101:0/170625317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a7c0186c0 msgr2=0x7f8a7c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:47.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.772 7f8aa2db0700 1 --2- 172.21.2.101:0/170625317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a7c0186c0 0x7f8a7c01abc0 crc :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:47.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.772 7f8aa2db0700 1 -- 172.21.2.101:0/170625317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a9c0f0280 msgr2=0x7f8a9c0a2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:47.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.772 7f8aa2db0700 1 --2- 172.21.2.101:0/170625317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a9c0f0280 0x7f8a9c0a2310 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 rx=0x7f8a8c009630 tx=0x7f8a8c00b980).stop 2021-02-10T11:34:47.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.772 7f8aa2db0700 1 -- 172.21.2.101:0/170625317 shutdown_connections 2021-02-10T11:34:47.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.772 7f8aa2db0700 1 --2- 172.21.2.101:0/170625317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a7c0186c0 0x7f8a7c01abc0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:47.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.772 7f8aa2db0700 1 --2- 172.21.2.101:0/170625317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a9c0ec180 0x7f8a9c0ec580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:47.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.772 7f8aa2db0700 1 --2- 172.21.2.101:0/170625317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a9c0f0680 0x7f8a9c0a2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:47.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.772 7f8aa2db0700 1 --2- 172.21.2.101:0/170625317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a9c0f0280 0x7f8a9c0a2310 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:47.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.772 7f8aa2db0700 1 -- 172.21.2.101:0/170625317 >> 172.21.2.101:0/170625317 conn(0x7f8a9c0c2130 msgr2=0x7f8a9c09e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:47.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.772 7f8aa2db0700 1 -- 172.21.2.101:0/170625317 shutdown_connections 2021-02-10T11:34:47.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:47.772 7f8aa2db0700 1 -- 172.21.2.101:0/170625317 wait complete. 2021-02-10T11:34:47.993 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:34:41.088085', 'modified': '2021-02-10 11:34:43.450805', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6400}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6400': {'gid': 6400, 'name': 'b', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.2.108:6833/3585539482', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 3585539482}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 3585539482}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:34:47.994 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:34:47.994 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:34:48.393 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:34:48.393 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:34:48.393 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:34:48.393 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 6400, 2021-02-10T11:34:48.394 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:34:48.394 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:34:48.394 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 7.2926503729999999, 2021-02-10T11:34:48.394 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 46, 2021-02-10T11:34:48.394 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 65, 2021-02-10T11:34:48.394 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 65, 2021-02-10T11:34:48.394 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 16.014105751999999 2021-02-10T11:34:48.394 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:34:48.397 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 6400, "want_state": "up:active", "state": "up:active", "rank_uptime": 7.2926503729999999, "mdsmap_epoch": 46, "osdmap_epoch": 65, "osdmap_epoch_barrier": 65, "uptime": 16.014105751999999 } 2021-02-10T11:34:48.397 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:34:48.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.800 7f86980fd700 1 Processor -- start 2021-02-10T11:34:48.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.800 7f86980fd700 1 -- start start 2021-02-10T11:34:48.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.800 7f86980fd700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f86900d1870 0x7f86901b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:48.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.800 7f86980fd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f86901b0a10 0x7f86900cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:48.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.800 7f86980fd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86900d1c70 0x7f86900cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:48.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.800 7f86980fd700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86900e5ed0 con 0x7f86901b0a10 2021-02-10T11:34:48.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.800 7f86980fd700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86900f0b10 con 0x7f86900d1870 2021-02-10T11:34:48.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.800 7f86980fd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86900cff70 con 0x7f86900d1c70 2021-02-10T11:34:48.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.801 7f8695718700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f86901b0a10 0x7f86900cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:48.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.801 7f8695718700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f86901b0a10 0x7f86900cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52684/0 (socket says 172.21.2.101:52684) 2021-02-10T11:34:48.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.801 7f8695718700 1 -- 172.21.2.101:0/1060751683 learned_addr learned my addr 172.21.2.101:0/1060751683 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:48.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.801 7f8695718700 1 -- 172.21.2.101:0/1060751683 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86900d1c70 msgr2=0x7f86900cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:48.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.801 7f8695718700 1 --2- 172.21.2.101:0/1060751683 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86900d1c70 0x7f86900cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:48.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.801 7f8695718700 1 -- 172.21.2.101:0/1060751683 >> v2:172.21.2.108:3300/0 conn(0x7f86900d1870 msgr2=0x7f86901b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:48.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.801 7f8695718700 1 --2- 172.21.2.101:0/1060751683 >> v2:172.21.2.108:3300/0 conn(0x7f86900d1870 0x7f86901b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:48.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.801 7f8695718700 1 -- 172.21.2.101:0/1060751683 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86900d0150 con 0x7f86901b0a10 2021-02-10T11:34:48.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.801 7f8695718700 1 --2- 172.21.2.101:0/1060751683 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86901b0a10 0x7f86900cf330 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7f868c006700 tx=0x7f868c009b80).ready entity=mon.0 client_cookie=764e33beb71773cf server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:48.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.801 7f8694f17700 1 -- 172.21.2.101:0/1060751683 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f868c00c0c0 con 0x7f86901b0a10 2021-02-10T11:34:48.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.801 7f8694f17700 1 -- 172.21.2.101:0/1060751683 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c00c440 con 0x7f86901b0a10 2021-02-10T11:34:48.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.801 7f8694f17700 1 -- 172.21.2.101:0/1060751683 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f868c00c8a0 con 0x7f86901b0a10 2021-02-10T11:34:48.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.801 7f86980fd700 1 -- 172.21.2.101:0/1060751683 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86901b0a10 msgr2=0x7f86900cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:48.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.801 7f86980fd700 1 --2- 172.21.2.101:0/1060751683 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86901b0a10 0x7f86900cf330 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7f868c006700 tx=0x7f868c009b80).stop 2021-02-10T11:34:48.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.802 7f86980fd700 1 -- 172.21.2.101:0/1060751683 shutdown_connections 2021-02-10T11:34:48.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.802 7f86980fd700 1 --2- 172.21.2.101:0/1060751683 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86900d1c70 0x7f86900cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:48.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.802 7f86980fd700 1 --2- 172.21.2.101:0/1060751683 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86901b0a10 0x7f86900cf330 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:48.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.802 7f86980fd700 1 --2- 172.21.2.101:0/1060751683 >> v2:172.21.2.108:3300/0 conn(0x7f86900d1870 0x7f86901b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:48.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.802 7f86980fd700 1 -- 172.21.2.101:0/1060751683 >> 172.21.2.101:0/1060751683 conn(0x7f86900ce450 msgr2=0x7f86900d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:48.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.802 7f86980fd700 1 -- 172.21.2.101:0/1060751683 shutdown_connections 2021-02-10T11:34:48.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.802 7f86980fd700 1 -- 172.21.2.101:0/1060751683 wait complete. 2021-02-10T11:34:48.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.816 7f86980fd700 1 Processor -- start 2021-02-10T11:34:48.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.816 7f86980fd700 1 -- start start 2021-02-10T11:34:48.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.817 7f86980fd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8690093060 0x7f869019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:48.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.817 7f86980fd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f869019aeb0 0x7f86900b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:48.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.817 7f86980fd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f869019b2b0 0x7f86900b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:48.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.817 7f86980fd700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86900efd70 con 0x7f869019aeb0 2021-02-10T11:34:48.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.817 7f86980fd700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869019b8a0 con 0x7f869019b2b0 2021-02-10T11:34:48.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.817 7f86980fd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86900b25d0 con 0x7f8690093060 2021-02-10T11:34:48.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.817 7f8695718700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f869019aeb0 0x7f86900b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:48.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.817 7f8695718700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f869019aeb0 0x7f86900b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52690/0 (socket says 172.21.2.101:52690) 2021-02-10T11:34:48.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.817 7f8695718700 1 -- 172.21.2.101:0/1483756822 learned_addr learned my addr 172.21.2.101:0/1483756822 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:48.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.817 7f8695718700 1 -- 172.21.2.101:0/1483756822 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8690093060 msgr2=0x7f869019a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:48.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.817 7f8695718700 1 --2- 172.21.2.101:0/1483756822 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8690093060 0x7f869019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:48.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.817 7f8695718700 1 -- 172.21.2.101:0/1483756822 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f869019b2b0 msgr2=0x7f86900b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:48.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.817 7f8695718700 1 --2- 172.21.2.101:0/1483756822 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f869019b2b0 0x7f86900b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:48.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.817 7f8695718700 1 -- 172.21.2.101:0/1483756822 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869018e410 con 0x7f869019aeb0 2021-02-10T11:34:48.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.818 7f8695718700 1 --2- 172.21.2.101:0/1483756822 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f869019aeb0 0x7f86900b1990 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f868c00b5e0 tx=0x7f868c00b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:48.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.818 7f86827fc700 1 -- 172.21.2.101:0/1483756822 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f868c00bd70 con 0x7f869019aeb0 2021-02-10T11:34:48.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.818 7f86827fc700 1 -- 172.21.2.101:0/1483756822 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c002030 con 0x7f869019aeb0 2021-02-10T11:34:48.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.818 7f86980fd700 1 -- 172.21.2.101:0/1483756822 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869018e410 con 0x7f869019aeb0 2021-02-10T11:34:48.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.818 7f86827fc700 1 -- 172.21.2.101:0/1483756822 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f868c002550 con 0x7f869019aeb0 2021-02-10T11:34:48.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.818 7f86980fd700 1 -- 172.21.2.101:0/1483756822 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869018e410 con 0x7f869019aeb0 2021-02-10T11:34:48.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.818 7f86827fc700 1 -- 172.21.2.101:0/1483756822 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f868c000c90 con 0x7f869019aeb0 2021-02-10T11:34:48.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.819 7f86827fc700 1 --2- 172.21.2.101:0/1483756822 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8678018700 0x7f867801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:48.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.819 7f86827fc700 1 -- 172.21.2.101:0/1483756822 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f868c0327c0 con 0x7f869019aeb0 2021-02-10T11:34:48.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.819 7f8695f19700 1 --2- 172.21.2.101:0/1483756822 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8678018700 0x7f867801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:48.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.819 7f8695f19700 1 --2- 172.21.2.101:0/1483756822 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8678018700 0x7f867801abc0 crc :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:48.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.850 7f86980fd700 1 -- 172.21.2.101:0/1483756822 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86900ebc00 con 0x7f869019aeb0 2021-02-10T11:34:48.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.853 7f86827fc700 1 -- 172.21.2.101:0/1483756822 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f868c01f500 con 0x7f869019aeb0 2021-02-10T11:34:48.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.978 7f86980fd700 1 -- 172.21.2.101:0/1483756822 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86900ebc00 con 0x7f869019aeb0 2021-02-10T11:34:48.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.979 7f86827fc700 1 -- 172.21.2.101:0/1483756822 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+13996 (secure 0 0 0) 0x7f868c000c90 con 0x7f869019aeb0 2021-02-10T11:34:48.988 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:48.989 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":65,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:34:42.090329","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:34:38.607957","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2021-02-10 11:34:40.084211","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:34:48.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.980 7f86980fd700 1 -- 172.21.2.101:0/1483756822 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8678018700 msgr2=0x7f867801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:48.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.980 7f86980fd700 1 --2- 172.21.2.101:0/1483756822 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8678018700 0x7f867801abc0 crc :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:48.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.980 7f86980fd700 1 -- 172.21.2.101:0/1483756822 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f869019aeb0 msgr2=0x7f86900b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:48.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.980 7f86980fd700 1 --2- 172.21.2.101:0/1483756822 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f869019aeb0 0x7f86900b1990 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f868c00b5e0 tx=0x7f868c00b980).stop 2021-02-10T11:34:48.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.981 7f86980fd700 1 -- 172.21.2.101:0/1483756822 shutdown_connections 2021-02-10T11:34:48.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.981 7f86980fd700 1 --2- 172.21.2.101:0/1483756822 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f869019b2b0 0x7f86900b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:48.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.981 7f86980fd700 1 --2- 172.21.2.101:0/1483756822 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f869019aeb0 0x7f86900b1990 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:48.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.981 7f86980fd700 1 --2- 172.21.2.101:0/1483756822 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8678018700 0x7f867801abc0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:48.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.981 7f86980fd700 1 --2- 172.21.2.101:0/1483756822 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8690093060 0x7f869019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:48.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.981 7f86980fd700 1 -- 172.21.2.101:0/1483756822 >> 172.21.2.101:0/1483756822 conn(0x7f869019f880 msgr2=0x7f8690199a80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:48.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.981 7f86980fd700 1 -- 172.21.2.101:0/1483756822 shutdown_connections 2021-02-10T11:34:48.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:48.981 7f86980fd700 1 -- 172.21.2.101:0/1483756822 wait complete. 2021-02-10T11:34:48.999 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:34:38.607957', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:34:40.084211', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:34:48.999 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:34:49.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.403 7fcc18da6700 1 Processor -- start 2021-02-10T11:34:49.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.403 7fcc18da6700 1 -- start start 2021-02-10T11:34:49.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.403 7fcc18da6700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fcc1410aff0 0x7fcc140cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:49.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.403 7fcc18da6700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc140fe1c0 con 0x7fcc140d1c70 2021-02-10T11:34:49.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.403 7fcc18da6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc141b0cb0 con 0x7fcc140d1870 2021-02-10T11:34:49.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.403 7fcc18da6700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc14109a60 con 0x7fcc1410aff0 2021-02-10T11:34:49.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.404 7fcc12312700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fcc140d1c70 0x7fcc140cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53188/0 (socket says 172.21.2.101:53188) 2021-02-10T11:34:49.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.404 7fcc12312700 1 -- 172.21.2.101:0/3575075459 learned_addr learned my addr 172.21.2.101:0/3575075459 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:49.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.404 7fcc11b11700 1 -- 172.21.2.101:0/3575075459 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1824986069 0 0) 0x7fcbfc0010b0 con 0x7fcc140d1c70 2021-02-10T11:34:49.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.404 7fcc11b11700 1 -- 172.21.2.101:0/3575075459 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc00002fe0 con 0x7fcc140d1c70 2021-02-10T11:34:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.404 7fcc13314700 1 --2- 172.21.2.101:0/3575075459 >> v2:172.21.2.108:3300/0 conn(0x7fcc1410aff0 0x7fcc140cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc11b11700 1 -- 172.21.2.101:0/3575075459 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 999421297 0 0) 0x7fcbfc0010b0 con 0x7fcc140d1c70 2021-02-10T11:34:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc11b11700 1 -- 172.21.2.101:0/3575075459 >> v2:172.21.2.108:3300/0 conn(0x7fcc1410aff0 msgr2=0x7fcc140cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc11b11700 1 --2- 172.21.2.101:0/3575075459 >> v2:172.21.2.108:3300/0 conn(0x7fcc1410aff0 0x7fcc140cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc11b11700 1 -- 172.21.2.101:0/3575075459 >> v1:172.21.2.108:6789/0 conn(0x7fcc140d1870 legacy=0x7fcc141b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc11b11700 1 -- 172.21.2.101:0/3575075459 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1410b3f0 con 0x7fcc140d1c70 2021-02-10T11:34:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc11b11700 1 -- 172.21.2.101:0/3575075459 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fcbfc0015b0 con 0x7fcc140d1c70 2021-02-10T11:34:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc11b11700 1 -- 172.21.2.101:0/3575075459 >> v1:172.21.2.101:6789/0 conn(0x7fcc140d1c70 legacy=0x7fcc140cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc11b11700 1 --2- 172.21.2.101:0/3575075459 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcc00003a60 0x7fcc00005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc11b11700 1 --2- 172.21.2.101:0/3575075459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc00006790 0x7fcc00008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc11b11700 1 --2- 172.21.2.101:0/3575075459 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcc00009590 0x7fcc0000bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc11b11700 1 -- 172.21.2.101:0/3575075459 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0000c300 con 0x7fcc00006790 2021-02-10T11:34:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc11b11700 1 -- 172.21.2.101:0/3575075459 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0000c520 con 0x7fcc00003a60 2021-02-10T11:34:49.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc11b11700 1 -- 172.21.2.101:0/3575075459 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0000c740 con 0x7fcc00009590 2021-02-10T11:34:49.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc12312700 1 --2- 172.21.2.101:0/3575075459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc00006790 0x7fcc00008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:49.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc13314700 1 --2- 172.21.2.101:0/3575075459 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcc00009590 0x7fcc0000bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:49.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc12b13700 1 --2- 172.21.2.101:0/3575075459 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcc00003a60 0x7fcc00005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:49.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc12312700 1 -- 172.21.2.101:0/3575075459 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcc00009590 msgr2=0x7fcc0000bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:49.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc12312700 1 --2- 172.21.2.101:0/3575075459 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcc00009590 0x7fcc0000bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:49.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc12312700 1 -- 172.21.2.101:0/3575075459 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcc00003a60 msgr2=0x7fcc00005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:49.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc12312700 1 --2- 172.21.2.101:0/3575075459 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcc00003a60 0x7fcc00005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:49.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc12312700 1 -- 172.21.2.101:0/3575075459 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0000c9e0 con 0x7fcc00006790 2021-02-10T11:34:49.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc13314700 1 -- 172.21.2.101:0/3575075459 reap_dead start 2021-02-10T11:34:49.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc12312700 1 --2- 172.21.2.101:0/3575075459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc00006790 0x7fcc00008c90 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7fcbfc001c60 tx=0x7fcbfc007d40).ready entity=mon.0 client_cookie=37fc239135167949 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:49.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc11b11700 1 -- 172.21.2.101:0/3575075459 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcbfc0009b0 con 0x7fcc00006790 2021-02-10T11:34:49.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc11b11700 1 -- 172.21.2.101:0/3575075459 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc0013f0 con 0x7fcc00006790 2021-02-10T11:34:49.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc11b11700 1 -- 172.21.2.101:0/3575075459 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcbfc04e510 con 0x7fcc00006790 2021-02-10T11:34:49.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc18da6700 1 -- 172.21.2.101:0/3575075459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc00006790 msgr2=0x7fcc00008c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:49.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.405 7fcc18da6700 1 --2- 172.21.2.101:0/3575075459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc00006790 0x7fcc00008c90 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7fcbfc001c60 tx=0x7fcbfc007d40).stop 2021-02-10T11:34:49.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.406 7fcc18da6700 1 -- 172.21.2.101:0/3575075459 shutdown_connections 2021-02-10T11:34:49.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.406 7fcc18da6700 1 --2- 172.21.2.101:0/3575075459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc00006790 0x7fcc00008c90 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:49.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.406 7fcc18da6700 1 -- 172.21.2.101:0/3575075459 >> 172.21.2.101:0/3575075459 conn(0x7fcc140ce420 msgr2=0x7fcc140d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:49.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.406 7fcc18da6700 1 -- 172.21.2.101:0/3575075459 shutdown_connections 2021-02-10T11:34:49.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.406 7fcc18da6700 1 -- 172.21.2.101:0/3575075459 wait complete. 2021-02-10T11:34:49.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.421 7fcc18da6700 1 Processor -- start 2021-02-10T11:34:49.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.421 7fcc18da6700 1 -- start start 2021-02-10T11:34:49.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.421 7fcc18da6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcc14086140 0x7fcc14086540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:49.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.421 7fcc18da6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc14086b60 0x7fcc1409c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:49.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.421 7fcc18da6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcc1409c990 0x7fcc1409cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:49.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.421 7fcc18da6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1409d980 con 0x7fcc14086b60 2021-02-10T11:34:49.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.421 7fcc18da6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1409db70 con 0x7fcc1409c990 2021-02-10T11:34:49.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.421 7fcc18da6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1409dd90 con 0x7fcc14086140 2021-02-10T11:34:49.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.422 7fcc12312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc14086b60 0x7fcc1409c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:49.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.422 7fcc12312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc14086b60 0x7fcc1409c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52708/0 (socket says 172.21.2.101:52708) 2021-02-10T11:34:49.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.422 7fcc12312700 1 -- 172.21.2.101:0/2151751639 learned_addr learned my addr 172.21.2.101:0/2151751639 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:49.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.422 7fcc13314700 1 --2- 172.21.2.101:0/2151751639 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcc1409c990 0x7fcc1409cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:49.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.422 7fcc12b13700 1 --2- 172.21.2.101:0/2151751639 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcc14086140 0x7fcc14086540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:49.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.422 7fcc12312700 1 -- 172.21.2.101:0/2151751639 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcc14086140 msgr2=0x7fcc14086540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:49.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.422 7fcc12312700 1 --2- 172.21.2.101:0/2151751639 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcc14086140 0x7fcc14086540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:49.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.422 7fcc12312700 1 -- 172.21.2.101:0/2151751639 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcc1409c990 msgr2=0x7fcc1409cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:49.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.422 7fcc12312700 1 --2- 172.21.2.101:0/2151751639 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcc1409c990 0x7fcc1409cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:49.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.422 7fcc12312700 1 -- 172.21.2.101:0/2151751639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1409dfb0 con 0x7fcc14086b60 2021-02-10T11:34:49.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.422 7fcc12312700 1 --2- 172.21.2.101:0/2151751639 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc14086b60 0x7fcc1409c370 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 rx=0x7fcbfc04f570 tx=0x7fcbfc04f910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:49.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.422 7fcc0b7fe700 1 -- 172.21.2.101:0/2151751639 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcbfc04fd00 con 0x7fcc14086b60 2021-02-10T11:34:49.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.422 7fcc0b7fe700 1 -- 172.21.2.101:0/2151751639 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc050070 con 0x7fcc14086b60 2021-02-10T11:34:49.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.422 7fcc0b7fe700 1 -- 172.21.2.101:0/2151751639 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcbfc0504d0 con 0x7fcc14086b60 2021-02-10T11:34:49.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.422 7fcc18da6700 1 -- 172.21.2.101:0/2151751639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1409dfb0 con 0x7fcc14086b60 2021-02-10T11:34:49.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.422 7fcc18da6700 1 -- 172.21.2.101:0/2151751639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1409e540 con 0x7fcc14086b60 2021-02-10T11:34:49.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.423 7fcc0b7fe700 1 -- 172.21.2.101:0/2151751639 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fcbfc04fd00 con 0x7fcc14086b60 2021-02-10T11:34:49.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.423 7fcc0b7fe700 1 --2- 172.21.2.101:0/2151751639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcbf80186c0 0x7fcbf801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:49.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.423 7fcc0b7fe700 1 -- 172.21.2.101:0/2151751639 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fcbfc076410 con 0x7fcc14086b60 2021-02-10T11:34:49.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.423 7fcc12b13700 1 --2- 172.21.2.101:0/2151751639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcbf80186c0 0x7fcbf801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:49.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.423 7fcc12b13700 1 --2- 172.21.2.101:0/2151751639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcbf80186c0 0x7fcbf801abc0 crc :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:49.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.453 7fcc18da6700 1 -- 172.21.2.101:0/2151751639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc1409edd0 con 0x7fcc14086b60 2021-02-10T11:34:49.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.456 7fcc0b7fe700 1 -- 172.21.2.101:0/2151751639 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fcbfc0733e0 con 0x7fcc14086b60 2021-02-10T11:34:49.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.586 7fcc18da6700 1 -- 172.21.2.101:0/2151751639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fcc14082a70 con 0x7fcc14086b60 2021-02-10T11:34:49.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.587 7fcc0b7fe700 1 -- 172.21.2.101:0/2151751639 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcbfc0733e0 con 0x7fcc14086b60 2021-02-10T11:34:49.597 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:34:49.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.589 7fcc18da6700 1 -- 172.21.2.101:0/2151751639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcbf80186c0 msgr2=0x7fcbf801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:49.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.589 7fcc18da6700 1 --2- 172.21.2.101:0/2151751639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcbf80186c0 0x7fcbf801abc0 crc :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:49.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.589 7fcc18da6700 1 -- 172.21.2.101:0/2151751639 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc14086b60 msgr2=0x7fcc1409c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:49.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.589 7fcc18da6700 1 --2- 172.21.2.101:0/2151751639 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc14086b60 0x7fcc1409c370 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 rx=0x7fcbfc04f570 tx=0x7fcbfc04f910).stop 2021-02-10T11:34:49.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.589 7fcc18da6700 1 -- 172.21.2.101:0/2151751639 shutdown_connections 2021-02-10T11:34:49.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.589 7fcc18da6700 1 --2- 172.21.2.101:0/2151751639 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcc1409c990 0x7fcc1409cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:49.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.589 7fcc18da6700 1 --2- 172.21.2.101:0/2151751639 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc14086b60 0x7fcc1409c370 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:49.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.589 7fcc18da6700 1 --2- 172.21.2.101:0/2151751639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcbf80186c0 0x7fcbf801abc0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:49.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.589 7fcc18da6700 1 --2- 172.21.2.101:0/2151751639 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcc14086140 0x7fcc14086540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:49.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.589 7fcc18da6700 1 -- 172.21.2.101:0/2151751639 >> 172.21.2.101:0/2151751639 conn(0x7fcc1409ffd0 msgr2=0x7fcc140a4890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:49.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.589 7fcc18da6700 1 -- 172.21.2.101:0/2151751639 shutdown_connections 2021-02-10T11:34:49.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:49.589 7fcc18da6700 1 -- 172.21.2.101:0/2151751639 wait complete. 2021-02-10T11:34:49.607 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:34:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.015 7f4442639700 1 Processor -- start 2021-02-10T11:34:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.015 7f4442639700 1 -- start start 2021-02-10T11:34:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.016 7f4442639700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f443c0d1c60 0x7f443c0cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.016 7f4442639700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f443c0cf570 0x7f443c0cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.016 7f4442639700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f443c0fe1b0 con 0x7f443c0d1860 2021-02-10T11:34:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.016 7f4442639700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f443c1b0ca0 con 0x7f443c0d1c60 2021-02-10T11:34:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.016 7f4442639700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443c109a50 con 0x7f443c0cf570 2021-02-10T11:34:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.016 7f443b7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f443c0d1c60 0x7f443c0cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.016 7f443bfff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f443c0d1860 0x7f443c1b04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53208/0 (socket says 172.21.2.101:53208) 2021-02-10T11:34:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.016 7f443bfff700 1 -- 172.21.2.101:0/3529940470 learned_addr learned my addr 172.21.2.101:0/3529940470 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.016 7f443b7fe700 1 -- 172.21.2.101:0/3529940470 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f443c0cf570 msgr2=0x7f443c0cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.016 7f443b7fe700 1 --2- 172.21.2.101:0/3529940470 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f443c0cf570 0x7f443c0cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.017 7f443b7fe700 1 -- 172.21.2.101:0/3529940470 >> v1:172.21.2.101:6789/0 conn(0x7f443c0d1860 legacy=0x7f443c1b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.017 7f443affd700 1 -- 172.21.2.101:0/3529940470 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3058469328 0 0) 0x7f442c000ce0 con 0x7f443c0d1860 2021-02-10T11:34:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.017 7f443b7fe700 1 -- 172.21.2.101:0/3529940470 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443c088070 con 0x7f443c0d1c60 2021-02-10T11:34:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.017 7f443b7fe700 1 --2- 172.21.2.101:0/3529940470 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f443c0d1c60 0x7f443c0cef50 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f4430006650 tx=0x7f4430009a80).ready entity=mon.0 client_cookie=9a03da637de074c9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.017 7f443affd700 1 -- 172.21.2.101:0/3529940470 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f443000c0c0 con 0x7f443c0d1c60 2021-02-10T11:34:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.017 7f443affd700 1 -- 172.21.2.101:0/3529940470 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443000c340 con 0x7f443c0d1c60 2021-02-10T11:34:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.017 7f443affd700 1 -- 172.21.2.101:0/3529940470 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f443000c860 con 0x7f443c0d1c60 2021-02-10T11:34:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.017 7f4442639700 1 -- 172.21.2.101:0/3529940470 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f443c0d1c60 msgr2=0x7f443c0cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.017 7f4442639700 1 --2- 172.21.2.101:0/3529940470 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f443c0d1c60 0x7f443c0cef50 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f4430006650 tx=0x7f4430009a80).stop 2021-02-10T11:34:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.018 7f4442639700 1 -- 172.21.2.101:0/3529940470 shutdown_connections 2021-02-10T11:34:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.018 7f4442639700 1 --2- 172.21.2.101:0/3529940470 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f443c0cf570 0x7f443c0cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.018 7f4442639700 1 --2- 172.21.2.101:0/3529940470 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f443c0d1c60 0x7f443c0cef50 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:50.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.018 7f4442639700 1 -- 172.21.2.101:0/3529940470 >> 172.21.2.101:0/3529940470 conn(0x7f443c0ce410 msgr2=0x7f443c0d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:50.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.018 7f4442639700 1 -- 172.21.2.101:0/3529940470 shutdown_connections 2021-02-10T11:34:50.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.018 7f4442639700 1 -- 172.21.2.101:0/3529940470 wait complete. 2021-02-10T11:34:50.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.033 7f4442639700 1 Processor -- start 2021-02-10T11:34:50.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.033 7f4442639700 1 -- start start 2021-02-10T11:34:50.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.033 7f4442639700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f443c0bfa10 0x7f443c0b82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:50.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.033 7f4442639700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f443c1a4a70 0x7f443c094c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:50.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.033 7f4442639700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f443c0c0410 0x7f443c0c0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:50.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.033 7f4442639700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c0c3a50 con 0x7f443c0c0410 2021-02-10T11:34:50.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.033 7f4442639700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c0b88c0 con 0x7f443c0bfa10 2021-02-10T11:34:50.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.033 7f4442639700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443c095260 con 0x7f443c1a4a70 2021-02-10T11:34:50.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.033 7f4440c56700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f443c0c0410 0x7f443c0c0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:50.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.034 7f4440c56700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f443c0c0410 0x7f443c0c0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52722/0 (socket says 172.21.2.101:52722) 2021-02-10T11:34:50.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.034 7f4440c56700 1 -- 172.21.2.101:0/2370052781 learned_addr learned my addr 172.21.2.101:0/2370052781 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:50.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.034 7f4440c56700 1 -- 172.21.2.101:0/2370052781 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f443c1a4a70 msgr2=0x7f443c094c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:50.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.034 7f4440c56700 1 --2- 172.21.2.101:0/2370052781 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f443c1a4a70 0x7f443c094c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:50.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.034 7f4440c56700 1 -- 172.21.2.101:0/2370052781 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f443c0bfa10 msgr2=0x7f443c0b82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:50.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.034 7f4440c56700 1 --2- 172.21.2.101:0/2370052781 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f443c0bfa10 0x7f443c0b82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:50.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.034 7f4440c56700 1 -- 172.21.2.101:0/2370052781 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443c0c3540 con 0x7f443c0c0410 2021-02-10T11:34:50.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.034 7f4440c56700 1 --2- 172.21.2.101:0/2370052781 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f443c0c0410 0x7f443c0c0810 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7f4434009980 tx=0x7f443400bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:50.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.034 7f4438ff9700 1 -- 172.21.2.101:0/2370052781 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4434005870 con 0x7f443c0c0410 2021-02-10T11:34:50.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.034 7f4438ff9700 1 -- 172.21.2.101:0/2370052781 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4434005b30 con 0x7f443c0c0410 2021-02-10T11:34:50.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.034 7f4438ff9700 1 -- 172.21.2.101:0/2370052781 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4434006030 con 0x7f443c0c0410 2021-02-10T11:34:50.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.035 7f4442639700 1 -- 172.21.2.101:0/2370052781 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443c0c3540 con 0x7f443c0c0410 2021-02-10T11:34:50.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.035 7f4442639700 1 -- 172.21.2.101:0/2370052781 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443c0c1120 con 0x7f443c0c0410 2021-02-10T11:34:50.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.035 7f4438ff9700 1 -- 172.21.2.101:0/2370052781 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4434005d10 con 0x7f443c0c0410 2021-02-10T11:34:50.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.035 7f4438ff9700 1 --2- 172.21.2.101:0/2370052781 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f441c0186c0 0x7f441c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:50.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.035 7f4438ff9700 1 -- 172.21.2.101:0/2370052781 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f44340327d0 con 0x7f443c0c0410 2021-02-10T11:34:50.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.036 7f443bfff700 1 --2- 172.21.2.101:0/2370052781 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f441c0186c0 0x7f441c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:50.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.036 7f443bfff700 1 --2- 172.21.2.101:0/2370052781 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f441c0186c0 0x7f441c01abc0 crc :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:50.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.066 7f4442639700 1 -- 172.21.2.101:0/2370052781 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f443c0bfee0 con 0x7f443c0c0410 2021-02-10T11:34:50.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.068 7f4438ff9700 1 -- 172.21.2.101:0/2370052781 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4434020f50 con 0x7f443c0c0410 2021-02-10T11:34:50.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.194 7f4442639700 1 -- 172.21.2.101:0/2370052781 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f443c0bfee0 con 0x7f443c0c0410 2021-02-10T11:34:50.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.195 7f4438ff9700 1 -- 172.21.2.101:0/2370052781 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v65) v1 ==== 104+0+10 (secure 0 0 0) 0x7f4434006f20 con 0x7f443c0c0410 2021-02-10T11:34:50.205 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:34:50.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.197 7f4442639700 1 -- 172.21.2.101:0/2370052781 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f441c0186c0 msgr2=0x7f441c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:50.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.197 7f4442639700 1 --2- 172.21.2.101:0/2370052781 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f441c0186c0 0x7f441c01abc0 crc :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:50.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.197 7f4442639700 1 -- 172.21.2.101:0/2370052781 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f443c0c0410 msgr2=0x7f443c0c0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:50.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.197 7f4442639700 1 --2- 172.21.2.101:0/2370052781 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f443c0c0410 0x7f443c0c0810 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7f4434009980 tx=0x7f443400bcd0).stop 2021-02-10T11:34:50.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.197 7f4442639700 1 -- 172.21.2.101:0/2370052781 shutdown_connections 2021-02-10T11:34:50.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.197 7f4442639700 1 --2- 172.21.2.101:0/2370052781 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f441c0186c0 0x7f441c01abc0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:50.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.197 7f4442639700 1 --2- 172.21.2.101:0/2370052781 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f443c1a4a70 0x7f443c094c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:50.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.197 7f4442639700 1 --2- 172.21.2.101:0/2370052781 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f443c0c0410 0x7f443c0c0810 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:50.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.197 7f4442639700 1 --2- 172.21.2.101:0/2370052781 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f443c0bfa10 0x7f443c0b82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:50.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.197 7f4442639700 1 -- 172.21.2.101:0/2370052781 >> 172.21.2.101:0/2370052781 conn(0x7f443c0a6a80 msgr2=0x7f443c0ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:50.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.198 7f4442639700 1 -- 172.21.2.101:0/2370052781 shutdown_connections 2021-02-10T11:34:50.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.198 7f4442639700 1 -- 172.21.2.101:0/2370052781 wait complete. 2021-02-10T11:34:50.215 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:34:50.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.619 7f482495b700 1 Processor -- start 2021-02-10T11:34:50.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.619 7f482495b700 1 -- start start 2021-02-10T11:34:50.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.619 7f482495b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48200d1870 0x7f48201b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:50.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.619 7f482495b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f48201b0ae0 0x7f4820108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:50.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.619 7f482495b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4820109a60 con 0x7f48200d1c70 2021-02-10T11:34:50.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.619 7f482495b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48200e5ef0 con 0x7f48201b0ae0 2021-02-10T11:34:50.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.619 7f482495b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4820109750 con 0x7f48200d1870 2021-02-10T11:34:50.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.619 7f481de1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f48201b0ae0 0x7f4820108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:50.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.619 7f481ee1d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f48200d1c70 0x7f4820108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53220/0 (socket says 172.21.2.101:53220) 2021-02-10T11:34:50.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.619 7f481ee1d700 1 -- 172.21.2.101:0/3148526867 learned_addr learned my addr 172.21.2.101:0/3148526867 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:50.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.620 7f481d61a700 1 -- 172.21.2.101:0/3148526867 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2841681631 0 0) 0x7f4818001500 con 0x7f48200d1c70 2021-02-10T11:34:50.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.620 7f481de1b700 1 -- 172.21.2.101:0/3148526867 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48200d1870 msgr2=0x7f48201b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:50.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.620 7f481de1b700 1 --2- 172.21.2.101:0/3148526867 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48200d1870 0x7f48201b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:50.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.620 7f481de1b700 1 -- 172.21.2.101:0/3148526867 >> v1:172.21.2.101:6789/0 conn(0x7f48200d1c70 legacy=0x7f4820108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:50.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.620 7f481de1b700 1 -- 172.21.2.101:0/3148526867 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4820088070 con 0x7f48201b0ae0 2021-02-10T11:34:50.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.620 7f481de1b700 1 --2- 172.21.2.101:0/3148526867 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48201b0ae0 0x7f4820108940 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7f4810006650 tx=0x7f4810009a80).ready entity=mon.0 client_cookie=6e09305d82698ca4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:50.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.620 7f481d61a700 1 -- 172.21.2.101:0/3148526867 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f481000c0c0 con 0x7f48201b0ae0 2021-02-10T11:34:50.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.620 7f481d61a700 1 -- 172.21.2.101:0/3148526867 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481000c340 con 0x7f48201b0ae0 2021-02-10T11:34:50.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.620 7f481d61a700 1 -- 172.21.2.101:0/3148526867 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f481000c860 con 0x7f48201b0ae0 2021-02-10T11:34:50.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.620 7f482495b700 1 -- 172.21.2.101:0/3148526867 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48201b0ae0 msgr2=0x7f4820108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:50.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.620 7f482495b700 1 --2- 172.21.2.101:0/3148526867 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48201b0ae0 0x7f4820108940 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7f4810006650 tx=0x7f4810009a80).stop 2021-02-10T11:34:50.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.621 7f482495b700 1 -- 172.21.2.101:0/3148526867 shutdown_connections 2021-02-10T11:34:50.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.621 7f482495b700 1 --2- 172.21.2.101:0/3148526867 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48201b0ae0 0x7f4820108940 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:50.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.621 7f482495b700 1 --2- 172.21.2.101:0/3148526867 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48200d1870 0x7f48201b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:50.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.621 7f482495b700 1 -- 172.21.2.101:0/3148526867 >> 172.21.2.101:0/3148526867 conn(0x7f48200ce420 msgr2=0x7f48200d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:50.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.621 7f482495b700 1 -- 172.21.2.101:0/3148526867 shutdown_connections 2021-02-10T11:34:50.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.621 7f482495b700 1 -- 172.21.2.101:0/3148526867 wait complete. 2021-02-10T11:34:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.636 7f482495b700 1 Processor -- start 2021-02-10T11:34:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.636 7f482495b700 1 -- start start 2021-02-10T11:34:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.636 7f482495b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48200a73c0 0x7f48200a6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.636 7f482495b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48200add40 0x7f48200ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.636 7f482495b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f48200ae760 0x7f48200af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.636 7f482495b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48200aec00 con 0x7f48200a73c0 2021-02-10T11:34:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.636 7f482495b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48200875d0 con 0x7f48200ae760 2021-02-10T11:34:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.636 7f482495b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48200af7a0 con 0x7f48200add40 2021-02-10T11:34:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.636 7f481e61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48200a73c0 0x7f48200a6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:50.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.636 7f481e61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48200a73c0 0x7f48200a6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52732/0 (socket says 172.21.2.101:52732) 2021-02-10T11:34:50.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.636 7f481e61c700 1 -- 172.21.2.101:0/1528986166 learned_addr learned my addr 172.21.2.101:0/1528986166 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:50.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.637 7f481e61c700 1 -- 172.21.2.101:0/1528986166 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48200add40 msgr2=0x7f48200ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:50.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.637 7f481e61c700 1 --2- 172.21.2.101:0/1528986166 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48200add40 0x7f48200ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:50.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.637 7f481e61c700 1 -- 172.21.2.101:0/1528986166 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f48200ae760 msgr2=0x7f48200af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:50.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.637 7f481e61c700 1 --2- 172.21.2.101:0/1528986166 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f48200ae760 0x7f48200af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:50.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.637 7f481e61c700 1 -- 172.21.2.101:0/1528986166 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48200af9c0 con 0x7f48200a73c0 2021-02-10T11:34:50.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.637 7f481e61c700 1 --2- 172.21.2.101:0/1528986166 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48200a73c0 0x7f48200a6b50 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7f4808006f00 tx=0x7f480800b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:50.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.637 7f4816ffd700 1 -- 172.21.2.101:0/1528986166 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f480800bd30 con 0x7f48200a73c0 2021-02-10T11:34:50.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.637 7f4816ffd700 1 -- 172.21.2.101:0/1528986166 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4808000c90 con 0x7f48200a73c0 2021-02-10T11:34:50.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.637 7f482495b700 1 -- 172.21.2.101:0/1528986166 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48200af9c0 con 0x7f48200a73c0 2021-02-10T11:34:50.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.637 7f482495b700 1 -- 172.21.2.101:0/1528986166 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48200aff50 con 0x7f48200a73c0 2021-02-10T11:34:50.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.637 7f4816ffd700 1 -- 172.21.2.101:0/1528986166 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f48080022a0 con 0x7f48200a73c0 2021-02-10T11:34:50.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.638 7f4816ffd700 1 -- 172.21.2.101:0/1528986166 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4808005310 con 0x7f48200a73c0 2021-02-10T11:34:50.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.638 7f4816ffd700 1 --2- 172.21.2.101:0/1528986166 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4804018730 0x7f480401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:50.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.638 7f4816ffd700 1 -- 172.21.2.101:0/1528986166 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4808032770 con 0x7f48200a73c0 2021-02-10T11:34:50.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.638 7f481de1b700 1 --2- 172.21.2.101:0/1528986166 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4804018730 0x7f480401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:50.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.638 7f481de1b700 1 --2- 172.21.2.101:0/1528986166 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4804018730 0x7f480401abc0 crc :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:50.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.669 7f482495b700 1 -- 172.21.2.101:0/1528986166 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48200b04f0 con 0x7f48200a73c0 2021-02-10T11:34:50.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.672 7f4816ffd700 1 -- 172.21.2.101:0/1528986166 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4808000960 con 0x7f48200a73c0 2021-02-10T11:34:50.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.798 7f482495b700 1 -- 172.21.2.101:0/1528986166 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f48200b04f0 con 0x7f48200a73c0 2021-02-10T11:34:50.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.799 7f4816ffd700 1 -- 172.21.2.101:0/1528986166 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v65) v1 ==== 100+0+10 (secure 0 0 0) 0x7f480800bd30 con 0x7f48200a73c0 2021-02-10T11:34:50.808 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:34:50.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.801 7f482495b700 1 -- 172.21.2.101:0/1528986166 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4804018730 msgr2=0x7f480401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:50.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.801 7f482495b700 1 --2- 172.21.2.101:0/1528986166 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4804018730 0x7f480401abc0 crc :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:50.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.801 7f482495b700 1 -- 172.21.2.101:0/1528986166 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48200a73c0 msgr2=0x7f48200a6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:50.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.801 7f482495b700 1 --2- 172.21.2.101:0/1528986166 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48200a73c0 0x7f48200a6b50 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7f4808006f00 tx=0x7f480800b940).stop 2021-02-10T11:34:50.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.801 7f482495b700 1 -- 172.21.2.101:0/1528986166 shutdown_connections 2021-02-10T11:34:50.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.801 7f482495b700 1 --2- 172.21.2.101:0/1528986166 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4804018730 0x7f480401abc0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:50.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.801 7f482495b700 1 --2- 172.21.2.101:0/1528986166 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48200add40 0x7f48200ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:50.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.801 7f482495b700 1 --2- 172.21.2.101:0/1528986166 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f48200ae760 0x7f48200af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:50.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.801 7f482495b700 1 --2- 172.21.2.101:0/1528986166 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48200a73c0 0x7f48200a6b50 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:50.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.801 7f482495b700 1 -- 172.21.2.101:0/1528986166 >> 172.21.2.101:0/1528986166 conn(0x7f48200be980 msgr2=0x7f4820092cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:50.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.801 7f482495b700 1 -- 172.21.2.101:0/1528986166 shutdown_connections 2021-02-10T11:34:50.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:50.801 7f482495b700 1 -- 172.21.2.101:0/1528986166 wait complete. 2021-02-10T11:34:50.818 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:34:50.818 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:34:51.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.220 7f07ffd95700 1 Processor -- start 2021-02-10T11:34:51.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.220 7f07ffd95700 1 -- start start 2021-02-10T11:34:51.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.221 7f07ffd95700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07f80d1870 0x7f07f81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:51.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.221 7f07ffd95700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07f80f0b10 con 0x7f07f81b0a10 2021-02-10T11:34:51.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.221 7f07ffd95700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07f80d0310 con 0x7f07f80f0650 2021-02-10T11:34:51.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.221 7f07ffd95700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f80e5ed0 con 0x7f07f80d1870 2021-02-10T11:34:51.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.221 7f07fd3b0700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f07f81b0a10 0x7f07f80cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53234/0 (socket says 172.21.2.101:53234) 2021-02-10T11:34:51.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.221 7f07fd3b0700 1 -- 172.21.2.101:0/3125543092 learned_addr learned my addr 172.21.2.101:0/3125543092 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:51.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.221 7f07fcbaf700 1 -- 172.21.2.101:0/3125543092 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2410644493 0 0) 0x7f07f40010b0 con 0x7f07f81b0a10 2021-02-10T11:34:51.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.221 7f07fcbaf700 1 -- 172.21.2.101:0/3125543092 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07e4002fe0 con 0x7f07f81b0a10 2021-02-10T11:34:51.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.221 7f07fdbb1700 1 --2- 172.21.2.101:0/3125543092 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07f80d1870 0x7f07f81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:51.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.222 7f07fcbaf700 1 -- 172.21.2.101:0/3125543092 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1034072478 0 0) 0x7f07f40010b0 con 0x7f07f81b0a10 2021-02-10T11:34:51.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.222 7f07fcbaf700 1 -- 172.21.2.101:0/3125543092 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07f80d1870 msgr2=0x7f07f81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:51.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.222 7f07fcbaf700 1 --2- 172.21.2.101:0/3125543092 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07f80d1870 0x7f07f81b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:51.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.222 7f07fcbaf700 1 -- 172.21.2.101:0/3125543092 >> v1:172.21.2.108:6789/0 conn(0x7f07f80f0650 legacy=0x7f07f80cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:51.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.222 7f07fcbaf700 1 -- 172.21.2.101:0/3125543092 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f80d0530 con 0x7f07f81b0a10 2021-02-10T11:34:51.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.222 7f07fcbaf700 1 -- 172.21.2.101:0/3125543092 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f07f40015b0 con 0x7f07f81b0a10 2021-02-10T11:34:51.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.222 7f07fcbaf700 1 -- 172.21.2.101:0/3125543092 >> v1:172.21.2.101:6789/0 conn(0x7f07f81b0a10 legacy=0x7f07f80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:51.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.222 7f07fcbaf700 1 --2- 172.21.2.101:0/3125543092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07e4003b10 0x7f07e4005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:51.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.222 7f07fcbaf700 1 --2- 172.21.2.101:0/3125543092 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07e4006790 0x7f07e4008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:51.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.222 7f07fcbaf700 1 --2- 172.21.2.101:0/3125543092 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07e4009520 0x7f07e400ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:51.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.222 7f07fcbaf700 1 -- 172.21.2.101:0/3125543092 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e400c2d0 con 0x7f07e4003b10 2021-02-10T11:34:51.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.222 7f07fcbaf700 1 -- 172.21.2.101:0/3125543092 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e400c4f0 con 0x7f07e4009520 2021-02-10T11:34:51.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.222 7f07fcbaf700 1 -- 172.21.2.101:0/3125543092 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e400c710 con 0x7f07e4006790 2021-02-10T11:34:51.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.222 7f07fdbb1700 1 --2- 172.21.2.101:0/3125543092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07e4003b10 0x7f07e4005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:51.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.222 7f07fe3b2700 1 --2- 172.21.2.101:0/3125543092 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07e4009520 0x7f07e400ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:51.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.222 7f07fd3b0700 1 --2- 172.21.2.101:0/3125543092 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07e4006790 0x7f07e4008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:51.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.222 7f07fdbb1700 1 -- 172.21.2.101:0/3125543092 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07e4006790 msgr2=0x7f07e4008c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:51.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.222 7f07fdbb1700 1 --2- 172.21.2.101:0/3125543092 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07e4006790 0x7f07e4008c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:51.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.222 7f07fdbb1700 1 -- 172.21.2.101:0/3125543092 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07e4009520 msgr2=0x7f07e400ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:51.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.222 7f07fdbb1700 1 --2- 172.21.2.101:0/3125543092 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07e4009520 0x7f07e400ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:51.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.222 7f07fdbb1700 1 -- 172.21.2.101:0/3125543092 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e4050980 con 0x7f07e4003b10 2021-02-10T11:34:51.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.222 7f07fd3b0700 1 --2- 172.21.2.101:0/3125543092 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07e4006790 0x7f07e4008c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:34:51.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.223 7f07fdbb1700 1 --2- 172.21.2.101:0/3125543092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07e4003b10 0x7f07e4005f80 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 rx=0x7f07ec009310 tx=0x7f07ec009a90).ready entity=mon.0 client_cookie=87e7565faa142d0d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:51.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.223 7f07fcbaf700 1 -- 172.21.2.101:0/3125543092 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f07ec00c0c0 con 0x7f07e4003b10 2021-02-10T11:34:51.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.223 7f07fcbaf700 1 -- 172.21.2.101:0/3125543092 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07ec006400 con 0x7f07e4003b10 2021-02-10T11:34:51.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.223 7f07fcbaf700 1 -- 172.21.2.101:0/3125543092 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f07ec00c400 con 0x7f07e4003b10 2021-02-10T11:34:51.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.223 7f07ffd95700 1 -- 172.21.2.101:0/3125543092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07e4003b10 msgr2=0x7f07e4005f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:51.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.223 7f07ffd95700 1 --2- 172.21.2.101:0/3125543092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07e4003b10 0x7f07e4005f80 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 rx=0x7f07ec009310 tx=0x7f07ec009a90).stop 2021-02-10T11:34:51.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.223 7f07fe3b2700 1 -- 172.21.2.101:0/3125543092 reap_dead start 2021-02-10T11:34:51.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.223 7f07ffd95700 1 -- 172.21.2.101:0/3125543092 shutdown_connections 2021-02-10T11:34:51.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.223 7f07ffd95700 1 -- 172.21.2.101:0/3125543092 >> 172.21.2.101:0/3125543092 conn(0x7f07f80ce450 msgr2=0x7f07f80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:51.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.224 7f07ffd95700 1 -- 172.21.2.101:0/3125543092 shutdown_connections 2021-02-10T11:34:51.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.224 7f07ffd95700 1 -- 172.21.2.101:0/3125543092 wait complete. 2021-02-10T11:34:51.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.238 7f07ffd95700 1 Processor -- start 2021-02-10T11:34:51.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.238 7f07ffd95700 1 -- start start 2021-02-10T11:34:51.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.239 7f07ffd95700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07f80c37f0 0x7f07f80907f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:51.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.239 7f07ffd95700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07f8090e10 0x7f07f80a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:51.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.239 7f07ffd95700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07f8091210 0x7f07f80a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:51.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.239 7f07ffd95700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f8091830 con 0x7f07f8091210 2021-02-10T11:34:51.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.239 7f07ffd95700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f809e5d0 con 0x7f07f80c37f0 2021-02-10T11:34:51.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.239 7f07ffd95700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f80c3bf0 con 0x7f07f8090e10 2021-02-10T11:34:51.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.239 7f07fe3b2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07f8091210 0x7f07f80a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:51.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.239 7f07fe3b2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07f8091210 0x7f07f80a4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52750/0 (socket says 172.21.2.101:52750) 2021-02-10T11:34:51.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.239 7f07fe3b2700 1 -- 172.21.2.101:0/3336995757 learned_addr learned my addr 172.21.2.101:0/3336995757 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:51.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.239 7f07fd3b0700 1 --2- 172.21.2.101:0/3336995757 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07f8090e10 0x7f07f80a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:51.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.239 7f07fdbb1700 1 --2- 172.21.2.101:0/3336995757 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07f80c37f0 0x7f07f80907f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:51.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.240 7f07fe3b2700 1 -- 172.21.2.101:0/3336995757 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07f8090e10 msgr2=0x7f07f80a3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:51.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.240 7f07fe3b2700 1 --2- 172.21.2.101:0/3336995757 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07f8090e10 0x7f07f80a3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:51.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.240 7f07fe3b2700 1 -- 172.21.2.101:0/3336995757 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07f80c37f0 msgr2=0x7f07f80907f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:51.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.240 7f07fe3b2700 1 --2- 172.21.2.101:0/3336995757 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07f80c37f0 0x7f07f80907f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:51.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.240 7f07fe3b2700 1 -- 172.21.2.101:0/3336995757 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f80a4830 con 0x7f07f8091210 2021-02-10T11:34:51.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.240 7f07fe3b2700 1 --2- 172.21.2.101:0/3336995757 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07f8091210 0x7f07f80a4210 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7f07f000a8f0 tx=0x7f07f000ac90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:51.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.240 7f07ea7fc700 1 -- 172.21.2.101:0/3336995757 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f07f00066e0 con 0x7f07f8091210 2021-02-10T11:34:51.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.240 7f07ea7fc700 1 -- 172.21.2.101:0/3336995757 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f000c300 con 0x7f07f8091210 2021-02-10T11:34:51.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.240 7f07ea7fc700 1 -- 172.21.2.101:0/3336995757 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f07f000c820 con 0x7f07f8091210 2021-02-10T11:34:51.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.240 7f07ffd95700 1 -- 172.21.2.101:0/3336995757 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f80a4830 con 0x7f07f8091210 2021-02-10T11:34:51.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.240 7f07ffd95700 1 -- 172.21.2.101:0/3336995757 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f80a4830 con 0x7f07f8091210 2021-02-10T11:34:51.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.241 7f07ea7fc700 1 -- 172.21.2.101:0/3336995757 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f07f0001010 con 0x7f07f8091210 2021-02-10T11:34:51.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.241 7f07ea7fc700 1 --2- 172.21.2.101:0/3336995757 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07e00186c0 0x7f07e001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:51.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.241 7f07ea7fc700 1 -- 172.21.2.101:0/3336995757 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f07f0032910 con 0x7f07f8091210 2021-02-10T11:34:51.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.241 7f07fdbb1700 1 --2- 172.21.2.101:0/3336995757 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07e00186c0 0x7f07e001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:51.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.241 7f07fdbb1700 1 --2- 172.21.2.101:0/3336995757 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07e00186c0 0x7f07e001abc0 crc :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:51.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.270 7f07ffd95700 1 -- 172.21.2.101:0/3336995757 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f80ebc80 con 0x7f07f8091210 2021-02-10T11:34:51.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.273 7f07ea7fc700 1 -- 172.21.2.101:0/3336995757 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f07f00066e0 con 0x7f07f8091210 2021-02-10T11:34:51.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.401 7f07ffd95700 1 -- 172.21.2.101:0/3336995757 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07f80f0b10 con 0x7f07f8091210 2021-02-10T11:34:51.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.402 7f07ea7fc700 1 -- 172.21.2.101:0/3336995757 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f07f0001010 con 0x7f07f8091210 2021-02-10T11:34:51.411 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 46 2021-02-10T11:34:51.411 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:51.411 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1297756180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1297756180},{"type":"v1","addr":"172.21.2.101:6835","nonce":1297756180}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:34:41.088085","modified":"2021-02-10 11:34:43.450805","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6400":{"gid":6400,"name":"b","rank":0,"incarnation":44,"state":"up:active","state_seq":4,"addr":"172.21.2.108:6833/3585539482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3585539482},{"type":"v1","addr":"172.21.2.108:6833","nonce":3585539482}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-02-10T11:34:51.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.404 7f07ffd95700 1 -- 172.21.2.101:0/3336995757 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07e00186c0 msgr2=0x7f07e001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:51.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.404 7f07ffd95700 1 --2- 172.21.2.101:0/3336995757 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07e00186c0 0x7f07e001abc0 crc :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:51.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.404 7f07ffd95700 1 -- 172.21.2.101:0/3336995757 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07f8091210 msgr2=0x7f07f80a4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:51.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.404 7f07ffd95700 1 --2- 172.21.2.101:0/3336995757 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07f8091210 0x7f07f80a4210 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7f07f000a8f0 tx=0x7f07f000ac90).stop 2021-02-10T11:34:51.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.404 7f07ffd95700 1 -- 172.21.2.101:0/3336995757 shutdown_connections 2021-02-10T11:34:51.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.404 7f07ffd95700 1 --2- 172.21.2.101:0/3336995757 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07e00186c0 0x7f07e001abc0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:51.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.404 7f07ffd95700 1 --2- 172.21.2.101:0/3336995757 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07f8090e10 0x7f07f80a3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:51.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.404 7f07ffd95700 1 --2- 172.21.2.101:0/3336995757 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07f8091210 0x7f07f80a4210 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:51.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.404 7f07ffd95700 1 --2- 172.21.2.101:0/3336995757 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07f80c37f0 0x7f07f80907f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:51.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.404 7f07ffd95700 1 -- 172.21.2.101:0/3336995757 >> 172.21.2.101:0/3336995757 conn(0x7f07f80a2b90 msgr2=0x7f07f808b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:51.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.404 7f07ffd95700 1 -- 172.21.2.101:0/3336995757 shutdown_connections 2021-02-10T11:34:51.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:51.404 7f07ffd95700 1 -- 172.21.2.101:0/3336995757 wait complete. 2021-02-10T11:34:51.623 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:34:41.088085', 'modified': '2021-02-10 11:34:43.450805', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6400}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6400': {'gid': 6400, 'name': 'b', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.2.108:6833/3585539482', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 3585539482}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 3585539482}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:34:51.623 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:34:51.623 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:34:52.021 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:34:52.021 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:34:52.021 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:34:52.021 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 6400, 2021-02-10T11:34:52.021 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:34:52.022 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:34:52.022 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 10.920625428999999, 2021-02-10T11:34:52.022 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 46, 2021-02-10T11:34:52.022 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 65, 2021-02-10T11:34:52.022 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 65, 2021-02-10T11:34:52.022 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 19.642080719999999 2021-02-10T11:34:52.022 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:34:52.025 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 6400, "want_state": "up:active", "state": "up:active", "rank_uptime": 10.920625428999999, "mdsmap_epoch": 46, "osdmap_epoch": 65, "osdmap_epoch_barrier": 65, "uptime": 19.642080719999999 } 2021-02-10T11:34:52.025 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:34:52.026 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-02-10T11:34:52.026 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-02-10T11:34:52.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:52.026 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:34:52.362 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:34:52.706 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:34:52.706 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:34:53.048 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:34:53.049 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:34:53.049 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:34:53.050 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:53.050 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:34:53.384 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-02-10T11:34:53.385 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:34:53.388 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:34:53.730 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:34:53.731 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:34:53.796 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[124259]: starting ceph client 2021-02-10T11:34:53.797 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:2021-02-10 11:34:53.787 7f064eaeff80 -1 init, newargv = 0x55bac02451f0 newargc=9 2021-02-10T11:34:53.803 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[124259]: starting fuse 2021-02-10T11:34:54.074 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:34:54.074 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:34:54.075 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:34:54.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:54.075 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:34:54.410 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:34:54.411 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2021-02-10T11:34:54.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:54.411 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:34:54.411 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:34:54.411 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:34:54.411 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:34:54.411 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:34:54.411 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:34:54.411 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:34:54.411 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-02-10T11:34:54.412 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:34:54.412 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:34:54.412 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:34:54.412 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:34:54.412 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:34:54.412 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:34:54.412 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:34:54.412 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:34:54.412 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:34:54.412 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:34:54.412 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:34:54.412 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:34:54.412 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:34:54.412 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.0")) 2021-02-10T11:34:54.412 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:34:54.776 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.0.124259.asok 2021-02-10T11:34:54.779 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.124259.asok 2021-02-10T11:34:54.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:54.779 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.124259.asok status 2021-02-10T11:34:55.168 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:34:55.169 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:34:55.169 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:34:55.169 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:34:55.169 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "0", 2021-02-10T11:34:55.169 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:34:55.169 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-02-10T11:34:55.169 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "124259", 2021-02-10T11:34:55.169 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:34:55.169 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:55.169 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:34:55.169 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:34:55.169 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 8658, 2021-02-10T11:34:55.170 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:34:55.170 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:34:55.170 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:34:55.170 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 8658 2021-02-10T11:34:55.170 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:55.170 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:34:55.170 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:55.170 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:34:55.170 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1894466675 2021-02-10T11:34:55.170 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:34:55.170 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:55.170 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:34:55.170 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:34:55.171 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:34:55.171 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1894466675 2021-02-10T11:34:55.171 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:34:55.171 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.8658 172.21.2.101:0/1894466675", 2021-02-10T11:34:55.171 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/1894466675", 2021-02-10T11:34:55.171 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:34:55.171 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 46, 2021-02-10T11:34:55.171 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 65, 2021-02-10T11:34:55.171 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:34:55.171 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:34:55.171 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:34:55.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:55.173 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:34:55.173 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:34:55.507 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:34:55.507 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:34:55.507 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:55.508 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:34:55.853 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:34:56.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.252 7f4117e30700 1 Processor -- start 2021-02-10T11:34:56.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.252 7f4117e30700 1 -- start start 2021-02-10T11:34:56.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.253 7f4117e30700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41100d0340 0x7f41100ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:56.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.253 7f4117e30700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f41100d1c70 0x7f41100efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:56.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.253 7f4117e30700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41100f0b10 con 0x7f41100d1870 2021-02-10T11:34:56.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.253 7f4117e30700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41100e5ed0 con 0x7f41100d1c70 2021-02-10T11:34:56.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.253 7f4117e30700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41101b0be0 con 0x7f41100d0340 2021-02-10T11:34:56.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.253 7f411644d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f41100d1c70 0x7f41100efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:56.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.253 7f411644d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f41100d1c70 0x7f41100efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52774/0 (socket says 172.21.2.101:52774) 2021-02-10T11:34:56.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.253 7f411644d700 1 -- 172.21.2.101:0/4025383440 learned_addr learned my addr 172.21.2.101:0/4025383440 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:56.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.253 7f411644d700 1 -- 172.21.2.101:0/4025383440 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41100d0340 msgr2=0x7f41100ef640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:34:56.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.253 7f411644d700 1 --2- 172.21.2.101:0/4025383440 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41100d0340 0x7f41100ef640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:56.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.253 7f411644d700 1 -- 172.21.2.101:0/4025383440 >> v1:172.21.2.108:6789/0 conn(0x7f41100d1870 legacy=0x7f41101b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:34:56.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.253 7f411644d700 1 -- 172.21.2.101:0/4025383440 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41100f20a0 con 0x7f41100d1c70 2021-02-10T11:34:56.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.253 7f411644d700 1 --2- 172.21.2.101:0/4025383440 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41100d1c70 0x7f41100efc60 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 rx=0x7f4108006b80 tx=0x7f4108009b90).ready entity=mon.0 client_cookie=ca232d606e66df3a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:56.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.253 7f4114c4a700 1 -- 172.21.2.101:0/4025383440 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f41080022e0 con 0x7f41100d1c70 2021-02-10T11:34:56.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.253 7f4114c4a700 1 -- 172.21.2.101:0/4025383440 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4108002620 con 0x7f41100d1c70 2021-02-10T11:34:56.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.254 7f4114c4a700 1 -- 172.21.2.101:0/4025383440 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4108002b00 con 0x7f41100d1c70 2021-02-10T11:34:56.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.254 7f4117e30700 1 -- 172.21.2.101:0/4025383440 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41100d1c70 msgr2=0x7f41100efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:56.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.254 7f4117e30700 1 --2- 172.21.2.101:0/4025383440 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41100d1c70 0x7f41100efc60 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 rx=0x7f4108006b80 tx=0x7f4108009b90).stop 2021-02-10T11:34:56.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.254 7f4117e30700 1 -- 172.21.2.101:0/4025383440 shutdown_connections 2021-02-10T11:34:56.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.254 7f4117e30700 1 --2- 172.21.2.101:0/4025383440 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41100d0340 0x7f41100ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:56.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.254 7f4117e30700 1 --2- 172.21.2.101:0/4025383440 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41100d1c70 0x7f41100efc60 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:56.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.254 7f4117e30700 1 -- 172.21.2.101:0/4025383440 >> 172.21.2.101:0/4025383440 conn(0x7f41100ce450 msgr2=0x7f41100d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:56.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.254 7f4117e30700 1 -- 172.21.2.101:0/4025383440 shutdown_connections 2021-02-10T11:34:56.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.254 7f4117e30700 1 -- 172.21.2.101:0/4025383440 wait complete. 2021-02-10T11:34:56.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.269 7f4117e30700 1 Processor -- start 2021-02-10T11:34:56.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.269 7f4117e30700 1 -- start start 2021-02-10T11:34:56.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.269 7f4117e30700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f411019ba40 0x7f411019bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:56.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.269 7f4117e30700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f411019aec0 0x7f411019b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:56.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.269 7f4117e30700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f411008a9d0 0x7f411008add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:56.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.269 7f4117e30700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411008fb50 con 0x7f411019aec0 2021-02-10T11:34:56.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.269 7f4117e30700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411019c530 con 0x7f411008a9d0 2021-02-10T11:34:56.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.269 7f4117e30700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411008b3f0 con 0x7f411019ba40 2021-02-10T11:34:56.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.269 7f411544b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f411019aec0 0x7f411019b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:56.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.269 7f411544b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f411019aec0 0x7f411019b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52778/0 (socket says 172.21.2.101:52778) 2021-02-10T11:34:56.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.269 7f411544b700 1 -- 172.21.2.101:0/1874614146 learned_addr learned my addr 172.21.2.101:0/1874614146 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:56.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.270 7f411544b700 1 -- 172.21.2.101:0/1874614146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f411019ba40 msgr2=0x7f411019bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:56.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.270 7f411544b700 1 --2- 172.21.2.101:0/1874614146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f411019ba40 0x7f411019bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:56.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.270 7f411544b700 1 -- 172.21.2.101:0/1874614146 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f411008a9d0 msgr2=0x7f411008add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:56.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.270 7f411544b700 1 --2- 172.21.2.101:0/1874614146 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f411008a9d0 0x7f411008add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:56.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.270 7f411544b700 1 -- 172.21.2.101:0/1874614146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411008b610 con 0x7f411019aec0 2021-02-10T11:34:56.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.270 7f411544b700 1 --2- 172.21.2.101:0/1874614146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f411019aec0 0x7f411019b2c0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 rx=0x7f410c009960 tx=0x7f410c00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:56.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.270 7f41027fc700 1 -- 172.21.2.101:0/1874614146 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f410c003030 con 0x7f411019aec0 2021-02-10T11:34:56.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.270 7f41027fc700 1 -- 172.21.2.101:0/1874614146 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410c001310 con 0x7f411019aec0 2021-02-10T11:34:56.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.270 7f4117e30700 1 -- 172.21.2.101:0/1874614146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411008b610 con 0x7f411019aec0 2021-02-10T11:34:56.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.270 7f41027fc700 1 -- 172.21.2.101:0/1874614146 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f410c001770 con 0x7f411019aec0 2021-02-10T11:34:56.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.270 7f4117e30700 1 -- 172.21.2.101:0/1874614146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411008b610 con 0x7f411019aec0 2021-02-10T11:34:56.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.271 7f41027fc700 1 -- 172.21.2.101:0/1874614146 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f410c0014f0 con 0x7f411019aec0 2021-02-10T11:34:56.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.271 7f41027fc700 1 --2- 172.21.2.101:0/1874614146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f40f8018730 0x7f40f801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:56.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.271 7f41027fc700 1 -- 172.21.2.101:0/1874614146 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f410c032b40 con 0x7f411019aec0 2021-02-10T11:34:56.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.271 7f4115c4c700 1 --2- 172.21.2.101:0/1874614146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f40f8018730 0x7f40f801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:56.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.272 7f4115c4c700 1 --2- 172.21.2.101:0/1874614146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f40f8018730 0x7f40f801abc0 crc :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:56.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.303 7f4117e30700 1 -- 172.21.2.101:0/1874614146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41100ebc60 con 0x7f411019aec0 2021-02-10T11:34:56.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.306 7f41027fc700 1 -- 172.21.2.101:0/1874614146 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f410c00be30 con 0x7f411019aec0 2021-02-10T11:34:56.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.431 7f4117e30700 1 -- 172.21.2.101:0/1874614146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41100f0b10 con 0x7f411019aec0 2021-02-10T11:34:56.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.432 7f41027fc700 1 -- 172.21.2.101:0/1874614146 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+13996 (secure 0 0 0) 0x7f410c001310 con 0x7f411019aec0 2021-02-10T11:34:56.441 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:56.441 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":65,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:34:42.090329","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:34:38.607957","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2021-02-10 11:34:40.084211","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:34:56.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.434 7f4117e30700 1 -- 172.21.2.101:0/1874614146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f40f8018730 msgr2=0x7f40f801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:56.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.434 7f4117e30700 1 --2- 172.21.2.101:0/1874614146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f40f8018730 0x7f40f801abc0 crc :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:56.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.434 7f4117e30700 1 -- 172.21.2.101:0/1874614146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f411019aec0 msgr2=0x7f411019b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:56.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.434 7f4117e30700 1 --2- 172.21.2.101:0/1874614146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f411019aec0 0x7f411019b2c0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 rx=0x7f410c009960 tx=0x7f410c00bcf0).stop 2021-02-10T11:34:56.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.434 7f4117e30700 1 -- 172.21.2.101:0/1874614146 shutdown_connections 2021-02-10T11:34:56.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.434 7f4117e30700 1 --2- 172.21.2.101:0/1874614146 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f411008a9d0 0x7f411008add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:56.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.434 7f4117e30700 1 --2- 172.21.2.101:0/1874614146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f411019aec0 0x7f411019b2c0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:56.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.434 7f4117e30700 1 --2- 172.21.2.101:0/1874614146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f40f8018730 0x7f40f801abc0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:56.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.434 7f4117e30700 1 --2- 172.21.2.101:0/1874614146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f411019ba40 0x7f411019bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:56.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.434 7f4117e30700 1 -- 172.21.2.101:0/1874614146 >> 172.21.2.101:0/1874614146 conn(0x7f411019f8c0 msgr2=0x7f41101a5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:56.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.434 7f4117e30700 1 -- 172.21.2.101:0/1874614146 shutdown_connections 2021-02-10T11:34:56.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.434 7f4117e30700 1 -- 172.21.2.101:0/1874614146 wait complete. 2021-02-10T11:34:56.451 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:34:38.607957', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:34:40.084211', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:34:56.452 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:34:56.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.855 7f5d77500700 1 Processor -- start 2021-02-10T11:34:56.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.855 7f5d77500700 1 -- start start 2021-02-10T11:34:56.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.855 7f5d77500700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f5d700d1c60 0x7f5d700cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:56.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.856 7f5d77500700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d700cf570 0x7f5d700cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:56.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.856 7f5d77500700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d700fe1b0 con 0x7f5d700d1860 2021-02-10T11:34:56.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.856 7f5d77500700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d701b0ca0 con 0x7f5d700d1c60 2021-02-10T11:34:56.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.856 7f5d77500700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d70109a50 con 0x7f5d700cf570 2021-02-10T11:34:56.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.856 7f5d7531c700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f5d700d1860 0x7f5d701b04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:46688/0 (socket says 172.21.2.101:46688) 2021-02-10T11:34:56.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.856 7f5d7531c700 1 -- 172.21.2.101:0/1035180866 learned_addr learned my addr 172.21.2.101:0/1035180866 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:56.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.856 7f5d74b1b700 1 --2- 172.21.2.101:0/1035180866 >> v2:172.21.2.108:3300/0 conn(0x7f5d700d1c60 0x7f5d700cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:56.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.856 7f5d75b1d700 1 --2- 172.21.2.101:0/1035180866 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d700cf570 0x7f5d700cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:56.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.857 7f5d74b1b700 1 -- 172.21.2.101:0/1035180866 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d700cf570 msgr2=0x7f5d700cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:56.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.857 7f5d74b1b700 1 --2- 172.21.2.101:0/1035180866 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d700cf570 0x7f5d700cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:56.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.857 7f5d74b1b700 1 -- 172.21.2.101:0/1035180866 >> v1:172.21.2.108:6789/0 conn(0x7f5d700d1860 legacy=0x7f5d701b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:56.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.857 7f5d74b1b700 1 -- 172.21.2.101:0/1035180866 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d70088070 con 0x7f5d700d1c60 2021-02-10T11:34:56.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.857 7f5d74b1b700 1 --2- 172.21.2.101:0/1035180866 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d700d1c60 0x7f5d700cef50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f5d6c006650 tx=0x7f5d6c009a80).ready entity=mon.1 client_cookie=221c797e6e67c21 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:56.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.858 7f5d63fff700 1 -- 172.21.2.101:0/1035180866 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5d6c00c0c0 con 0x7f5d700d1c60 2021-02-10T11:34:56.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.858 7f5d63fff700 1 -- 172.21.2.101:0/1035180866 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6c00c340 con 0x7f5d700d1c60 2021-02-10T11:34:56.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.858 7f5d63fff700 1 -- 172.21.2.101:0/1035180866 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5d6c00c860 con 0x7f5d700d1c60 2021-02-10T11:34:56.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.858 7f5d77500700 1 -- 172.21.2.101:0/1035180866 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d700d1c60 msgr2=0x7f5d700cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:56.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.858 7f5d77500700 1 --2- 172.21.2.101:0/1035180866 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d700d1c60 0x7f5d700cef50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f5d6c006650 tx=0x7f5d6c009a80).stop 2021-02-10T11:34:56.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.858 7f5d77500700 1 -- 172.21.2.101:0/1035180866 shutdown_connections 2021-02-10T11:34:56.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.858 7f5d77500700 1 --2- 172.21.2.101:0/1035180866 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d700cf570 0x7f5d700cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:56.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.858 7f5d77500700 1 --2- 172.21.2.101:0/1035180866 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d700d1c60 0x7f5d700cef50 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:56.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.858 7f5d77500700 1 -- 172.21.2.101:0/1035180866 >> 172.21.2.101:0/1035180866 conn(0x7f5d700ce410 msgr2=0x7f5d700d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:56.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.859 7f5d77500700 1 -- 172.21.2.101:0/1035180866 shutdown_connections 2021-02-10T11:34:56.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.859 7f5d77500700 1 -- 172.21.2.101:0/1035180866 wait complete. 2021-02-10T11:34:56.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.873 7f5d77500700 1 Processor -- start 2021-02-10T11:34:56.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.873 7f5d77500700 1 -- start start 2021-02-10T11:34:56.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.873 7f5d77500700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d700c2a30 0x7f5d700c2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:56.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.873 7f5d77500700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d700a7950 0x7f5d700a7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:56.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.873 7f5d77500700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d701a71f0 0x7f5d701a75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:56.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.873 7f5d77500700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d700a8440 con 0x7f5d701a71f0 2021-02-10T11:34:56.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.873 7f5d77500700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d700c3790 con 0x7f5d700c2a30 2021-02-10T11:34:56.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.873 7f5d77500700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d701a7c10 con 0x7f5d700a7950 2021-02-10T11:34:56.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.874 7f5d75b1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d701a71f0 0x7f5d701a75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:56.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.874 7f5d75b1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d701a71f0 0x7f5d701a75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52792/0 (socket says 172.21.2.101:52792) 2021-02-10T11:34:56.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.874 7f5d75b1d700 1 -- 172.21.2.101:0/3019043756 learned_addr learned my addr 172.21.2.101:0/3019043756 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:56.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.874 7f5d7531c700 1 --2- 172.21.2.101:0/3019043756 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d700c2a30 0x7f5d700c2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:56.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.874 7f5d74b1b700 1 --2- 172.21.2.101:0/3019043756 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d700a7950 0x7f5d700a7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:56.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.874 7f5d75b1d700 1 -- 172.21.2.101:0/3019043756 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d700a7950 msgr2=0x7f5d700a7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:56.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.874 7f5d75b1d700 1 --2- 172.21.2.101:0/3019043756 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d700a7950 0x7f5d700a7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:56.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.874 7f5d75b1d700 1 -- 172.21.2.101:0/3019043756 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d700c2a30 msgr2=0x7f5d700c2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:56.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.874 7f5d75b1d700 1 --2- 172.21.2.101:0/3019043756 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d700c2a30 0x7f5d700c2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:56.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.874 7f5d75b1d700 1 -- 172.21.2.101:0/3019043756 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d701a7e30 con 0x7f5d701a71f0 2021-02-10T11:34:56.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.874 7f5d7531c700 1 --2- 172.21.2.101:0/3019043756 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d700c2a30 0x7f5d700c2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:34:56.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.874 7f5d74b1b700 1 --2- 172.21.2.101:0/3019043756 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d700a7950 0x7f5d700a7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:34:56.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.874 7f5d75b1d700 1 --2- 172.21.2.101:0/3019043756 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d701a71f0 0x7f5d701a75f0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 rx=0x7f5d6800b970 tx=0x7f5d6800bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:56.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.875 7f5d61ffb700 1 -- 172.21.2.101:0/3019043756 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5d68000f90 con 0x7f5d701a71f0 2021-02-10T11:34:56.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.875 7f5d61ffb700 1 -- 172.21.2.101:0/3019043756 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d68003030 con 0x7f5d701a71f0 2021-02-10T11:34:56.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.875 7f5d61ffb700 1 -- 172.21.2.101:0/3019043756 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5d680015f0 con 0x7f5d701a71f0 2021-02-10T11:34:56.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.875 7f5d77500700 1 -- 172.21.2.101:0/3019043756 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d701a7e30 con 0x7f5d701a71f0 2021-02-10T11:34:56.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.875 7f5d77500700 1 -- 172.21.2.101:0/3019043756 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d701a7e30 con 0x7f5d701a71f0 2021-02-10T11:34:56.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.875 7f5d61ffb700 1 -- 172.21.2.101:0/3019043756 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5d68000df0 con 0x7f5d701a71f0 2021-02-10T11:34:56.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.876 7f5d61ffb700 1 --2- 172.21.2.101:0/3019043756 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5d500186c0 0x7f5d5001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:56.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.876 7f5d61ffb700 1 -- 172.21.2.101:0/3019043756 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5d680328f0 con 0x7f5d701a71f0 2021-02-10T11:34:56.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.876 7f5d7531c700 1 --2- 172.21.2.101:0/3019043756 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5d500186c0 0x7f5d5001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:56.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.876 7f5d7531c700 1 --2- 172.21.2.101:0/3019043756 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5d500186c0 0x7f5d5001abc0 crc :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:56.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.906 7f5d77500700 1 -- 172.21.2.101:0/3019043756 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d70082a20 con 0x7f5d701a71f0 2021-02-10T11:34:56.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:56.909 7f5d61ffb700 1 -- 172.21.2.101:0/3019043756 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5d680066f0 con 0x7f5d701a71f0 2021-02-10T11:34:57.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.035 7f5d77500700 1 -- 172.21.2.101:0/3019043756 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f5d70082a20 con 0x7f5d701a71f0 2021-02-10T11:34:57.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.036 7f5d61ffb700 1 -- 172.21.2.101:0/3019043756 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5d68000df0 con 0x7f5d701a71f0 2021-02-10T11:34:57.045 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:34:57.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.038 7f5d77500700 1 -- 172.21.2.101:0/3019043756 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5d500186c0 msgr2=0x7f5d5001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:57.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.038 7f5d77500700 1 --2- 172.21.2.101:0/3019043756 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5d500186c0 0x7f5d5001abc0 crc :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:57.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.038 7f5d77500700 1 -- 172.21.2.101:0/3019043756 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d701a71f0 msgr2=0x7f5d701a75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:57.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.038 7f5d77500700 1 --2- 172.21.2.101:0/3019043756 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d701a71f0 0x7f5d701a75f0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 rx=0x7f5d6800b970 tx=0x7f5d6800bd10).stop 2021-02-10T11:34:57.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.038 7f5d77500700 1 -- 172.21.2.101:0/3019043756 shutdown_connections 2021-02-10T11:34:57.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.038 7f5d77500700 1 --2- 172.21.2.101:0/3019043756 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5d500186c0 0x7f5d5001abc0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:57.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.038 7f5d77500700 1 --2- 172.21.2.101:0/3019043756 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5d700a7950 0x7f5d700a7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:57.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.038 7f5d77500700 1 --2- 172.21.2.101:0/3019043756 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5d701a71f0 0x7f5d701a75f0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:57.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.038 7f5d77500700 1 --2- 172.21.2.101:0/3019043756 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5d700c2a30 0x7f5d700c2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:57.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.038 7f5d77500700 1 -- 172.21.2.101:0/3019043756 >> 172.21.2.101:0/3019043756 conn(0x7f5d700b5f10 msgr2=0x7f5d7009f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:57.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.038 7f5d77500700 1 -- 172.21.2.101:0/3019043756 shutdown_connections 2021-02-10T11:34:57.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.038 7f5d77500700 1 -- 172.21.2.101:0/3019043756 wait complete. 2021-02-10T11:34:57.056 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:34:57.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.461 7f2a3471e700 1 Processor -- start 2021-02-10T11:34:57.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.461 7f2a3471e700 1 -- start start 2021-02-10T11:34:57.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.461 7f2a3471e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2a2c0d1870 0x7f2a2c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:57.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.461 7f2a3471e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2a2c0cff70 0x7f2a2c109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:57.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.462 7f2a3471e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a2c0fe1c0 con 0x7f2a2c1b0ae0 2021-02-10T11:34:57.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.462 7f2a3471e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a2c109ad0 con 0x7f2a2c0d1870 2021-02-10T11:34:57.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.462 7f2a3471e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a2c109750 con 0x7f2a2c0cff70 2021-02-10T11:34:57.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.462 7f2a3253a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2a2c0d1870 0x7f2a2c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:57.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.462 7f2a3253a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2a2c0d1870 0x7f2a2c1b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52798/0 (socket says 172.21.2.101:52798) 2021-02-10T11:34:57.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.462 7f2a3253a700 1 -- 172.21.2.101:0/3357114314 learned_addr learned my addr 172.21.2.101:0/3357114314 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:57.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.462 7f2a3253a700 1 -- 172.21.2.101:0/3357114314 >> v2:172.21.2.108:3300/0 conn(0x7f2a2c0cff70 msgr2=0x7f2a2c109130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:34:57.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.462 7f2a3253a700 1 --2- 172.21.2.101:0/3357114314 >> v2:172.21.2.108:3300/0 conn(0x7f2a2c0cff70 0x7f2a2c109130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:57.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.462 7f2a3253a700 1 -- 172.21.2.101:0/3357114314 >> v1:172.21.2.108:6789/0 conn(0x7f2a2c1b0ae0 legacy=0x7f2a2c108940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:34:57.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.462 7f2a3253a700 1 -- 172.21.2.101:0/3357114314 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c088070 con 0x7f2a2c0d1870 2021-02-10T11:34:57.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.462 7f2a3253a700 1 --2- 172.21.2.101:0/3357114314 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c0d1870 0x7f2a2c1b04c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7f2a28006700 tx=0x7f2a28009b80).ready entity=mon.0 client_cookie=51e6891524cab9ad server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:57.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.462 7f2a31538700 1 -- 172.21.2.101:0/3357114314 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2a2800c0c0 con 0x7f2a2c0d1870 2021-02-10T11:34:57.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.462 7f2a31538700 1 -- 172.21.2.101:0/3357114314 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2800c400 con 0x7f2a2c0d1870 2021-02-10T11:34:57.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.462 7f2a31538700 1 -- 172.21.2.101:0/3357114314 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2a2800c860 con 0x7f2a2c0d1870 2021-02-10T11:34:57.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.462 7f2a3471e700 1 -- 172.21.2.101:0/3357114314 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c0d1870 msgr2=0x7f2a2c1b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:57.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.462 7f2a3471e700 1 --2- 172.21.2.101:0/3357114314 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c0d1870 0x7f2a2c1b04c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7f2a28006700 tx=0x7f2a28009b80).stop 2021-02-10T11:34:57.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.463 7f2a3471e700 1 -- 172.21.2.101:0/3357114314 shutdown_connections 2021-02-10T11:34:57.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.463 7f2a3471e700 1 --2- 172.21.2.101:0/3357114314 >> v2:172.21.2.108:3300/0 conn(0x7f2a2c0cff70 0x7f2a2c109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:57.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.463 7f2a3471e700 1 --2- 172.21.2.101:0/3357114314 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c0d1870 0x7f2a2c1b04c0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:57.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.463 7f2a3471e700 1 -- 172.21.2.101:0/3357114314 >> 172.21.2.101:0/3357114314 conn(0x7f2a2c0ce420 msgr2=0x7f2a2c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:57.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.463 7f2a3471e700 1 -- 172.21.2.101:0/3357114314 shutdown_connections 2021-02-10T11:34:57.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.463 7f2a3471e700 1 -- 172.21.2.101:0/3357114314 wait complete. 2021-02-10T11:34:57.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.477 7f2a3471e700 1 Processor -- start 2021-02-10T11:34:57.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.477 7f2a3471e700 1 -- start start 2021-02-10T11:34:57.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.478 7f2a3471e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a2c08da20 0x7f2a2c1a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:57.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.478 7f2a3471e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c1a15d0 0x7f2a2c0902c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:57.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.478 7f2a3471e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2a2c1a19d0 0x7f2a2c0908e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:57.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.478 7f2a3471e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c091390 con 0x7f2a2c1a15d0 2021-02-10T11:34:57.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.478 7f2a3471e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c08de20 con 0x7f2a2c1a19d0 2021-02-10T11:34:57.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.478 7f2a3471e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c090f00 con 0x7f2a2c08da20 2021-02-10T11:34:57.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.478 7f2a31d39700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c1a15d0 0x7f2a2c0902c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:57.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.478 7f2a31d39700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c1a15d0 0x7f2a2c0902c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52806/0 (socket says 172.21.2.101:52806) 2021-02-10T11:34:57.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.478 7f2a31d39700 1 -- 172.21.2.101:0/618235385 learned_addr learned my addr 172.21.2.101:0/618235385 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.478 7f2a31d39700 1 -- 172.21.2.101:0/618235385 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a2c08da20 msgr2=0x7f2a2c1a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.478 7f2a31d39700 1 --2- 172.21.2.101:0/618235385 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a2c08da20 0x7f2a2c1a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.478 7f2a31d39700 1 -- 172.21.2.101:0/618235385 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2a2c1a19d0 msgr2=0x7f2a2c0908e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:34:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.478 7f2a31d39700 1 --2- 172.21.2.101:0/618235385 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2a2c1a19d0 0x7f2a2c0908e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.478 7f2a31d39700 1 -- 172.21.2.101:0/618235385 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c091120 con 0x7f2a2c1a15d0 2021-02-10T11:34:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.479 7f2a31d39700 1 --2- 172.21.2.101:0/618235385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c1a15d0 0x7f2a2c0902c0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 rx=0x7f2a2000b5b0 tx=0x7f2a2000b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.479 7f2a1effd700 1 -- 172.21.2.101:0/618235385 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2a2000bd40 con 0x7f2a2c1a15d0 2021-02-10T11:34:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.479 7f2a1effd700 1 -- 172.21.2.101:0/618235385 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a20006240 con 0x7f2a2c1a15d0 2021-02-10T11:34:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.479 7f2a1effd700 1 -- 172.21.2.101:0/618235385 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2a20005690 con 0x7f2a2c1a15d0 2021-02-10T11:34:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.479 7f2a3471e700 1 -- 172.21.2.101:0/618235385 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2c091390 con 0x7f2a2c1a15d0 2021-02-10T11:34:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.479 7f2a3471e700 1 -- 172.21.2.101:0/618235385 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2c082780 con 0x7f2a2c1a15d0 2021-02-10T11:34:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.479 7f2a1effd700 1 -- 172.21.2.101:0/618235385 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2a200054a0 con 0x7f2a2c1a15d0 2021-02-10T11:34:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.479 7f2a1effd700 1 --2- 172.21.2.101:0/618235385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a140186c0 0x7f2a1401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:57.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.479 7f2a1effd700 1 -- 172.21.2.101:0/618235385 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2a20032410 con 0x7f2a2c1a15d0 2021-02-10T11:34:57.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.480 7f2a3253a700 1 --2- 172.21.2.101:0/618235385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a140186c0 0x7f2a1401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:57.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.480 7f2a3253a700 1 --2- 172.21.2.101:0/618235385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a140186c0 0x7f2a1401abc0 crc :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:57.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.511 7f2a3471e700 1 -- 172.21.2.101:0/618235385 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a2c082a70 con 0x7f2a2c1a15d0 2021-02-10T11:34:57.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.514 7f2a1effd700 1 -- 172.21.2.101:0/618235385 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2a200009b0 con 0x7f2a2c1a15d0 2021-02-10T11:34:57.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.641 7f2a3471e700 1 -- 172.21.2.101:0/618235385 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f2a2c091390 con 0x7f2a2c1a15d0 2021-02-10T11:34:57.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.641 7f2a1effd700 1 -- 172.21.2.101:0/618235385 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v65) v1 ==== 104+0+10 (secure 0 0 0) 0x7f2a200054a0 con 0x7f2a2c1a15d0 2021-02-10T11:34:57.651 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:34:57.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.643 7f2a3471e700 1 -- 172.21.2.101:0/618235385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a140186c0 msgr2=0x7f2a1401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:57.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.643 7f2a3471e700 1 --2- 172.21.2.101:0/618235385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a140186c0 0x7f2a1401abc0 crc :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:57.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.643 7f2a3471e700 1 -- 172.21.2.101:0/618235385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c1a15d0 msgr2=0x7f2a2c0902c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:57.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.643 7f2a3471e700 1 --2- 172.21.2.101:0/618235385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c1a15d0 0x7f2a2c0902c0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 rx=0x7f2a2000b5b0 tx=0x7f2a2000b950).stop 2021-02-10T11:34:57.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.643 7f2a3471e700 1 -- 172.21.2.101:0/618235385 shutdown_connections 2021-02-10T11:34:57.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.643 7f2a3471e700 1 --2- 172.21.2.101:0/618235385 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2a2c1a19d0 0x7f2a2c0908e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:57.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.643 7f2a3471e700 1 --2- 172.21.2.101:0/618235385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c1a15d0 0x7f2a2c0902c0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:57.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.643 7f2a3471e700 1 --2- 172.21.2.101:0/618235385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a140186c0 0x7f2a1401abc0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:57.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.643 7f2a3471e700 1 --2- 172.21.2.101:0/618235385 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a2c08da20 0x7f2a2c1a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:57.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.643 7f2a3471e700 1 -- 172.21.2.101:0/618235385 >> 172.21.2.101:0/618235385 conn(0x7f2a2c09bb00 msgr2=0x7f2a2c09bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:57.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.644 7f2a3471e700 1 -- 172.21.2.101:0/618235385 shutdown_connections 2021-02-10T11:34:57.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:57.644 7f2a3471e700 1 -- 172.21.2.101:0/618235385 wait complete. 2021-02-10T11:34:57.661 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:34:58.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.065 7f0e8f949700 1 Processor -- start 2021-02-10T11:34:58.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.065 7f0e8f949700 1 -- start start 2021-02-10T11:34:58.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.066 7f0e8f949700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0e880d1c70 0x7f0e880cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:58.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.066 7f0e8f949700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0e880cf580 0x7f0e880cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:58.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.066 7f0e8f949700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e880fe1c0 con 0x7f0e880d1870 2021-02-10T11:34:58.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.066 7f0e8f949700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e881b0cb0 con 0x7f0e880cf580 2021-02-10T11:34:58.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.066 7f0e8f949700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e88109a60 con 0x7f0e880d1c70 2021-02-10T11:34:58.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.066 7f0e8df66700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0e880cf580 0x7f0e880cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:58.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.066 7f0e8df66700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0e880cf580 0x7f0e880cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52810/0 (socket says 172.21.2.101:52810) 2021-02-10T11:34:58.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.066 7f0e8df66700 1 -- 172.21.2.101:0/1278741173 learned_addr learned my addr 172.21.2.101:0/1278741173 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:58.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.066 7f0e8d765700 1 --1- 172.21.2.101:0/1278741173 >> v1:172.21.2.101:6789/0 conn(0x7f0e880d1870 0x7f0e881b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53304/0 (socket says 172.21.2.101:53304) 2021-02-10T11:34:58.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.066 7f0e8df66700 1 -- 172.21.2.101:0/1278741173 >> v2:172.21.2.108:3300/0 conn(0x7f0e880d1c70 msgr2=0x7f0e880cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:34:58.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.066 7f0e8df66700 1 --2- 172.21.2.101:0/1278741173 >> v2:172.21.2.108:3300/0 conn(0x7f0e880d1c70 0x7f0e880cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:58.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.066 7f0e8df66700 1 -- 172.21.2.101:0/1278741173 >> v1:172.21.2.101:6789/0 conn(0x7f0e880d1870 legacy=0x7f0e881b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:58.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.066 7f0e8df66700 1 -- 172.21.2.101:0/1278741173 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e88088070 con 0x7f0e880cf580 2021-02-10T11:34:58.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.066 7f0e8df66700 1 --2- 172.21.2.101:0/1278741173 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e880cf580 0x7f0e880cf980 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7f0e80006f00 tx=0x7f0e80009f10).ready entity=mon.0 client_cookie=8e3fade33465ca12 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:58.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.067 7f0e7bfff700 1 -- 172.21.2.101:0/1278741173 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e8000b3b0 con 0x7f0e880cf580 2021-02-10T11:34:58.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.067 7f0e7bfff700 1 -- 172.21.2.101:0/1278741173 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8000b630 con 0x7f0e880cf580 2021-02-10T11:34:58.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.067 7f0e7bfff700 1 -- 172.21.2.101:0/1278741173 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e8000bb50 con 0x7f0e880cf580 2021-02-10T11:34:58.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.067 7f0e8f949700 1 -- 172.21.2.101:0/1278741173 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e880cf580 msgr2=0x7f0e880cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:58.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.067 7f0e8f949700 1 --2- 172.21.2.101:0/1278741173 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e880cf580 0x7f0e880cf980 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7f0e80006f00 tx=0x7f0e80009f10).stop 2021-02-10T11:34:58.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.067 7f0e8f949700 1 -- 172.21.2.101:0/1278741173 shutdown_connections 2021-02-10T11:34:58.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.067 7f0e8f949700 1 --2- 172.21.2.101:0/1278741173 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e880cf580 0x7f0e880cf980 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:58.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.067 7f0e8f949700 1 --2- 172.21.2.101:0/1278741173 >> v2:172.21.2.108:3300/0 conn(0x7f0e880d1c70 0x7f0e880cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:58.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.067 7f0e8f949700 1 -- 172.21.2.101:0/1278741173 >> 172.21.2.101:0/1278741173 conn(0x7f0e880ce420 msgr2=0x7f0e880d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:58.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.068 7f0e8f949700 1 -- 172.21.2.101:0/1278741173 shutdown_connections 2021-02-10T11:34:58.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.068 7f0e8f949700 1 -- 172.21.2.101:0/1278741173 wait complete. 2021-02-10T11:34:58.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.082 7f0e8f949700 1 Processor -- start 2021-02-10T11:34:58.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.082 7f0e8f949700 1 -- start start 2021-02-10T11:34:58.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.083 7f0e8f949700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e880c3820 0x7f0e88084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:58.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.083 7f0e8f949700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e880a0290 0x7f0e880852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:58.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.083 7f0e8f949700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e88086270 0x7f0e88086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:58.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.083 7f0e8f949700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e880c0700 con 0x7f0e88086270 2021-02-10T11:34:58.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.083 7f0e8f949700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e880858f0 con 0x7f0e880c3820 2021-02-10T11:34:58.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.083 7f0e8f949700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e88085b10 con 0x7f0e880a0290 2021-02-10T11:34:58.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.083 7f0e8df66700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e88086270 0x7f0e88086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:58.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.083 7f0e8df66700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e88086270 0x7f0e88086670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52820/0 (socket says 172.21.2.101:52820) 2021-02-10T11:34:58.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.083 7f0e8df66700 1 -- 172.21.2.101:0/2685843014 learned_addr learned my addr 172.21.2.101:0/2685843014 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:58.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.083 7f0e8df66700 1 -- 172.21.2.101:0/2685843014 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e880a0290 msgr2=0x7f0e880852d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:34:58.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.083 7f0e8df66700 1 --2- 172.21.2.101:0/2685843014 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e880a0290 0x7f0e880852d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:58.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.083 7f0e8df66700 1 -- 172.21.2.101:0/2685843014 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e880c3820 msgr2=0x7f0e88084cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:58.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.083 7f0e8df66700 1 --2- 172.21.2.101:0/2685843014 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e880c3820 0x7f0e88084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:58.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.083 7f0e8df66700 1 -- 172.21.2.101:0/2685843014 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e88086c90 con 0x7f0e88086270 2021-02-10T11:34:58.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.084 7f0e8df66700 1 --2- 172.21.2.101:0/2685843014 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e88086270 0x7f0e88086670 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7f0e8000b8e0 tx=0x7f0e8000bc80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:58.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.084 7f0e79ffb700 1 -- 172.21.2.101:0/2685843014 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e80003030 con 0x7f0e88086270 2021-02-10T11:34:58.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.084 7f0e79ffb700 1 -- 172.21.2.101:0/2685843014 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8000bdc0 con 0x7f0e88086270 2021-02-10T11:34:58.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.084 7f0e8f949700 1 -- 172.21.2.101:0/2685843014 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e88086c90 con 0x7f0e88086270 2021-02-10T11:34:58.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.084 7f0e8f949700 1 -- 172.21.2.101:0/2685843014 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e88086c90 con 0x7f0e88086270 2021-02-10T11:34:58.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.084 7f0e79ffb700 1 -- 172.21.2.101:0/2685843014 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e8000c570 con 0x7f0e88086270 2021-02-10T11:34:58.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.084 7f0e79ffb700 1 -- 172.21.2.101:0/2685843014 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0e80005820 con 0x7f0e88086270 2021-02-10T11:34:58.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.084 7f0e79ffb700 1 --2- 172.21.2.101:0/2685843014 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e68018730 0x7f0e6801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:58.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.084 7f0e79ffb700 1 -- 172.21.2.101:0/2685843014 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0e80032b40 con 0x7f0e88086270 2021-02-10T11:34:58.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.085 7f0e8d765700 1 --2- 172.21.2.101:0/2685843014 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e68018730 0x7f0e6801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:58.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.085 7f0e8d765700 1 --2- 172.21.2.101:0/2685843014 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e68018730 0x7f0e6801abc0 crc :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:58.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.116 7f0e8f949700 1 -- 172.21.2.101:0/2685843014 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e88082a70 con 0x7f0e88086270 2021-02-10T11:34:58.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.118 7f0e79ffb700 1 -- 172.21.2.101:0/2685843014 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0e80003020 con 0x7f0e88086270 2021-02-10T11:34:58.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.245 7f0e8f949700 1 -- 172.21.2.101:0/2685843014 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f0e88082a70 con 0x7f0e88086270 2021-02-10T11:34:58.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.245 7f0e79ffb700 1 -- 172.21.2.101:0/2685843014 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v65) v1 ==== 100+0+10 (secure 0 0 0) 0x7f0e8000bdc0 con 0x7f0e88086270 2021-02-10T11:34:58.255 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:34:58.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.247 7f0e8f949700 1 -- 172.21.2.101:0/2685843014 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e68018730 msgr2=0x7f0e6801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:58.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.247 7f0e8f949700 1 --2- 172.21.2.101:0/2685843014 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e68018730 0x7f0e6801abc0 crc :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:58.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.247 7f0e8f949700 1 -- 172.21.2.101:0/2685843014 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e88086270 msgr2=0x7f0e88086670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:58.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.247 7f0e8f949700 1 --2- 172.21.2.101:0/2685843014 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e88086270 0x7f0e88086670 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7f0e8000b8e0 tx=0x7f0e8000bc80).stop 2021-02-10T11:34:58.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.248 7f0e8f949700 1 -- 172.21.2.101:0/2685843014 shutdown_connections 2021-02-10T11:34:58.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.248 7f0e8f949700 1 --2- 172.21.2.101:0/2685843014 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e68018730 0x7f0e6801abc0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:58.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.248 7f0e8f949700 1 --2- 172.21.2.101:0/2685843014 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e880a0290 0x7f0e880852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:58.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.248 7f0e8f949700 1 --2- 172.21.2.101:0/2685843014 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e88086270 0x7f0e88086670 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:58.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.248 7f0e8f949700 1 --2- 172.21.2.101:0/2685843014 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e880c3820 0x7f0e88084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:34:58.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.248 7f0e8f949700 1 -- 172.21.2.101:0/2685843014 >> 172.21.2.101:0/2685843014 conn(0x7f0e880872c0 msgr2=0x7f0e88095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:58.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.248 7f0e8f949700 1 -- 172.21.2.101:0/2685843014 shutdown_connections 2021-02-10T11:34:58.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.248 7f0e8f949700 1 -- 172.21.2.101:0/2685843014 wait complete. 2021-02-10T11:34:58.265 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:34:58.265 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:34:58.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.667 7f6932bd7700 1 Processor -- start 2021-02-10T11:34:58.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.667 7f6932bd7700 1 -- start start 2021-02-10T11:34:58.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.667 7f6932bd7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f692c0d0340 0x7f692c0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:58.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.667 7f6932bd7700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f692c0f0b10 con 0x7f692c0d1c70 2021-02-10T11:34:58.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.667 7f6932bd7700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f692c1b0be0 con 0x7f692c0d1870 2021-02-10T11:34:58.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.667 7f6932bd7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f692c0e5ed0 con 0x7f692c0d0340 2021-02-10T11:34:58.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.668 7f69311f4700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f692c0d1c70 0x7f692c0efc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53318/0 (socket says 172.21.2.101:53318) 2021-02-10T11:34:58.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.668 7f69311f4700 1 -- 172.21.2.101:0/1274388217 learned_addr learned my addr 172.21.2.101:0/1274388217 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:34:58.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.668 7f692bfff700 1 --2- 172.21.2.101:0/1274388217 >> v2:172.21.2.108:3300/0 conn(0x7f692c0d0340 0x7f692c0ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:58.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.668 7f692b7fe700 1 -- 172.21.2.101:0/1274388217 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250991720 0 0) 0x7f6924000c10 con 0x7f692c0d1c70 2021-02-10T11:34:58.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.668 7f692bfff700 1 -- 172.21.2.101:0/1274388217 >> v1:172.21.2.108:6789/0 conn(0x7f692c0d1870 legacy=0x7f692c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:58.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.668 7f692bfff700 1 -- 172.21.2.101:0/1274388217 >> v1:172.21.2.101:6789/0 conn(0x7f692c0d1c70 legacy=0x7f692c0efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:58.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.668 7f692bfff700 1 -- 172.21.2.101:0/1274388217 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692c0f0450 con 0x7f692c0d0340 2021-02-10T11:34:58.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.668 7f692bfff700 1 --2- 172.21.2.101:0/1274388217 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f692c0d0340 0x7f692c0ef640 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f691c0069d0 tx=0x7f691c009e40).ready entity=mon.1 client_cookie=97c66e3e69cb54a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:58.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.668 7f692b7fe700 1 -- 172.21.2.101:0/1274388217 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f691c00b510 con 0x7f692c0d0340 2021-02-10T11:34:58.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.668 7f692b7fe700 1 -- 172.21.2.101:0/1274388217 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691c00b890 con 0x7f692c0d0340 2021-02-10T11:34:58.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.668 7f692b7fe700 1 -- 172.21.2.101:0/1274388217 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f691c00bcf0 con 0x7f692c0d0340 2021-02-10T11:34:58.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.668 7f6932bd7700 1 -- 172.21.2.101:0/1274388217 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f692c0d0340 msgr2=0x7f692c0ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:58.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.668 7f6932bd7700 1 --2- 172.21.2.101:0/1274388217 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f692c0d0340 0x7f692c0ef640 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f691c0069d0 tx=0x7f691c009e40).stop 2021-02-10T11:34:58.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.669 7f6932bd7700 1 -- 172.21.2.101:0/1274388217 shutdown_connections 2021-02-10T11:34:58.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.669 7f6932bd7700 1 --2- 172.21.2.101:0/1274388217 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f692c0d0340 0x7f692c0ef640 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:58.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.669 7f6932bd7700 1 -- 172.21.2.101:0/1274388217 >> 172.21.2.101:0/1274388217 conn(0x7f692c0ce450 msgr2=0x7f692c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:58.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.669 7f6932bd7700 1 -- 172.21.2.101:0/1274388217 shutdown_connections 2021-02-10T11:34:58.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.669 7f6932bd7700 1 -- 172.21.2.101:0/1274388217 wait complete. 2021-02-10T11:34:58.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.684 7f6932bd7700 1 Processor -- start 2021-02-10T11:34:58.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.684 7f6932bd7700 1 -- start start 2021-02-10T11:34:58.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.684 7f6932bd7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f692c19a3d0 0x7f692c1ab160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:58.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.684 7f6932bd7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f692c1ab780 0x7f692c0b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:58.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.684 7f6932bd7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f692c1abb80 0x7f692c0b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:58.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.684 7f6932bd7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c1ae200 con 0x7f692c1abb80 2021-02-10T11:34:58.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.684 7f6932bd7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c092930 con 0x7f692c19a3d0 2021-02-10T11:34:58.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.684 7f6932bd7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692c0b4290 con 0x7f692c1ab780 2021-02-10T11:34:58.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.684 7f69311f4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f692c1abb80 0x7f692c0b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:58.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.684 7f69311f4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f692c1abb80 0x7f692c0b3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52826/0 (socket says 172.21.2.101:52826) 2021-02-10T11:34:58.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.684 7f69311f4700 1 -- 172.21.2.101:0/3713781801 learned_addr learned my addr 172.21.2.101:0/3713781801 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:34:58.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.685 7f69309f3700 1 --2- 172.21.2.101:0/3713781801 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f692c19a3d0 0x7f692c1ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:58.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.685 7f692bfff700 1 --2- 172.21.2.101:0/3713781801 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f692c1ab780 0x7f692c0b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:58.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.685 7f69311f4700 1 -- 172.21.2.101:0/3713781801 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f692c1ab780 msgr2=0x7f692c0b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:58.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.685 7f69311f4700 1 --2- 172.21.2.101:0/3713781801 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f692c1ab780 0x7f692c0b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:58.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.685 7f69311f4700 1 -- 172.21.2.101:0/3713781801 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f692c19a3d0 msgr2=0x7f692c1ab160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:58.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.685 7f69311f4700 1 --2- 172.21.2.101:0/3713781801 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f692c19a3d0 0x7f692c1ab160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:58.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.685 7f69311f4700 1 -- 172.21.2.101:0/3713781801 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692c0ee360 con 0x7f692c1abb80 2021-02-10T11:34:58.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.685 7f69309f3700 1 --2- 172.21.2.101:0/3713781801 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f692c19a3d0 0x7f692c1ab160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:34:58.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.685 7f69311f4700 1 --2- 172.21.2.101:0/3713781801 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f692c1abb80 0x7f692c0b3c70 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 rx=0x7f692400a590 tx=0x7f692400a930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:58.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.685 7f69297fa700 1 -- 172.21.2.101:0/3713781801 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f692400ad20 con 0x7f692c1abb80 2021-02-10T11:34:58.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.686 7f69297fa700 1 -- 172.21.2.101:0/3713781801 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692400ad20 con 0x7f692c1abb80 2021-02-10T11:34:58.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.686 7f6932bd7700 1 -- 172.21.2.101:0/3713781801 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692c0ee360 con 0x7f692c1abb80 2021-02-10T11:34:58.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.686 7f69297fa700 1 -- 172.21.2.101:0/3713781801 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f69240043a0 con 0x7f692c1abb80 2021-02-10T11:34:58.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.686 7f6932bd7700 1 -- 172.21.2.101:0/3713781801 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692c0ee8f0 con 0x7f692c1abb80 2021-02-10T11:34:58.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.686 7f69297fa700 1 -- 172.21.2.101:0/3713781801 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f69240043a0 con 0x7f692c1abb80 2021-02-10T11:34:58.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.686 7f69297fa700 1 --2- 172.21.2.101:0/3713781801 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f69100186d0 0x7f691001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:34:58.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.686 7f69297fa700 1 -- 172.21.2.101:0/3713781801 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6924032800 con 0x7f692c1abb80 2021-02-10T11:34:58.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.687 7f69309f3700 1 --2- 172.21.2.101:0/3713781801 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f69100186d0 0x7f691001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:34:58.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.687 7f69309f3700 1 --2- 172.21.2.101:0/3713781801 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f69100186d0 0x7f691001abd0 crc :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:34:58.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.717 7f6932bd7700 1 -- 172.21.2.101:0/3713781801 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f692c0ce450 con 0x7f692c1abb80 2021-02-10T11:34:58.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.720 7f69297fa700 1 -- 172.21.2.101:0/3713781801 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6924000ef0 con 0x7f692c1abb80 2021-02-10T11:34:58.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.850 7f6932bd7700 1 -- 172.21.2.101:0/3713781801 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f692c0ce450 con 0x7f692c1abb80 2021-02-10T11:34:58.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.850 7f69297fa700 1 -- 172.21.2.101:0/3713781801 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f69240043a0 con 0x7f692c1abb80 2021-02-10T11:34:58.860 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:34:58.860 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1297756180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1297756180},{"type":"v1","addr":"172.21.2.101:6835","nonce":1297756180}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:34:41.088085","modified":"2021-02-10 11:34:43.450805","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6400":{"gid":6400,"name":"b","rank":0,"incarnation":44,"state":"up:active","state_seq":4,"addr":"172.21.2.108:6833/3585539482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3585539482},{"type":"v1","addr":"172.21.2.108:6833","nonce":3585539482}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-02-10T11:34:58.860 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 46 2021-02-10T11:34:58.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.852 7f6932bd7700 1 -- 172.21.2.101:0/3713781801 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f69100186d0 msgr2=0x7f691001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:58.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.852 7f6932bd7700 1 --2- 172.21.2.101:0/3713781801 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f69100186d0 0x7f691001abd0 crc :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:58.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.852 7f6932bd7700 1 -- 172.21.2.101:0/3713781801 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f692c1abb80 msgr2=0x7f692c0b3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:34:58.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.852 7f6932bd7700 1 --2- 172.21.2.101:0/3713781801 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f692c1abb80 0x7f692c0b3c70 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 rx=0x7f692400a590 tx=0x7f692400a930).stop 2021-02-10T11:34:58.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.853 7f6932bd7700 1 -- 172.21.2.101:0/3713781801 shutdown_connections 2021-02-10T11:34:58.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.853 7f6932bd7700 1 --2- 172.21.2.101:0/3713781801 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f69100186d0 0x7f691001abd0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:58.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.853 7f6932bd7700 1 --2- 172.21.2.101:0/3713781801 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f692c1ab780 0x7f692c0b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:58.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.853 7f6932bd7700 1 --2- 172.21.2.101:0/3713781801 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f692c1abb80 0x7f692c0b3c70 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:58.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.853 7f6932bd7700 1 --2- 172.21.2.101:0/3713781801 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f692c19a3d0 0x7f692c1ab160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:34:58.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.853 7f6932bd7700 1 -- 172.21.2.101:0/3713781801 >> 172.21.2.101:0/3713781801 conn(0x7f692c093640 msgr2=0x7f692c18eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:34:58.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.853 7f6932bd7700 1 -- 172.21.2.101:0/3713781801 shutdown_connections 2021-02-10T11:34:58.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:34:58.853 7f6932bd7700 1 -- 172.21.2.101:0/3713781801 wait complete. 2021-02-10T11:34:59.072 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:34:41.088085', 'modified': '2021-02-10 11:34:43.450805', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6400}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6400': {'gid': 6400, 'name': 'b', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.2.108:6833/3585539482', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 3585539482}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 3585539482}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:34:59.072 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:34:59.072 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:34:59.470 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:34:59.470 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:34:59.470 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:34:59.470 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 6400, 2021-02-10T11:34:59.470 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:34:59.470 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:34:59.470 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 18.368953640000001, 2021-02-10T11:34:59.471 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 46, 2021-02-10T11:34:59.471 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 65, 2021-02-10T11:34:59.471 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 65, 2021-02-10T11:34:59.471 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 27.090408772 2021-02-10T11:34:59.471 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:34:59.474 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 6400, "want_state": "up:active", "state": "up:active", "rank_uptime": 18.368953640000001, "mdsmap_epoch": 46, "osdmap_epoch": 65, "osdmap_epoch_barrier": 65, "uptime": 27.090408772 } 2021-02-10T11:34:59.474 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:34:59.474 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-02-10T11:34:59.474 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:34:59.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:34:59.475 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:34:59.715 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:34:59.719 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:34:59.810 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:35:00.060 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:35:00.061 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:35:00.405 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:35:00.405 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:35:00.406 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:35:00.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:00.406 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:35:00.740 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:35:00.741 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2021-02-10T11:35:00.741 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:00.745 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:35:01.088 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:35:01.089 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:35:01.155 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[124926]: starting ceph client 2021-02-10T11:35:01.156 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:2021-02-10 11:35:01.146 7faf7b73ff80 -1 init, newargv = 0x561efd3c81a0 newargc=9 2021-02-10T11:35:01.161 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[124926]: starting fuse 2021-02-10T11:35:01.431 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:35:01.431 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:35:01.431 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:35:01.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:01.432 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:35:01.766 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:35:01.767 INFO:teuthology.orchestra.run.gibba001.stdout:44 2021-02-10T11:35:01.767 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2021-02-10T11:35:01.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:01.767 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:35:01.767 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:35:01.767 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:35:01.767 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:35:01.768 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:35:01.768 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:35:01.768 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:35:01.768 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-02-10T11:35:01.768 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:35:01.768 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:35:01.768 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:35:01.768 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:35:01.768 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:35:01.768 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:35:01.768 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:35:01.768 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:35:01.768 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:35:01.768 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:35:01.769 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:35:01.769 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:35:01.769 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:35:01.769 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.1")) 2021-02-10T11:35:01.769 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:35:02.130 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.1.124926.asok 2021-02-10T11:35:02.132 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.124926.asok 2021-02-10T11:35:02.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:02.133 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.124926.asok status 2021-02-10T11:35:02.522 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:35:02.522 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:35:02.522 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:35:02.522 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:35:02.522 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "1", 2021-02-10T11:35:02.522 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:35:02.522 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:35:02.522 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "124926", 2021-02-10T11:35:02.522 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:35:02.523 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:02.523 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:35:02.523 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:35:02.523 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 8691, 2021-02-10T11:35:02.523 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:35:02.523 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:35:02.523 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:35:02.523 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 8691 2021-02-10T11:35:02.523 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:02.523 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:35:02.523 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:02.523 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:35:02.523 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 2480420452 2021-02-10T11:35:02.524 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:02.524 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:02.524 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:35:02.524 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:02.524 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:35:02.524 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 2480420452 2021-02-10T11:35:02.524 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:02.524 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.8691 172.21.2.101:0/2480420452", 2021-02-10T11:35:02.524 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/2480420452", 2021-02-10T11:35:02.524 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:35:02.524 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 46, 2021-02-10T11:35:02.524 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 65, 2021-02-10T11:35:02.524 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:35:02.524 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:35:02.525 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:35:02.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:02.526 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:35:02.526 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:02.861 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:35:02.861 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:35:02.861 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:02.861 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:03.207 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2021-02-10T11:35:03.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.613 7ff2afe33700 1 Processor -- start 2021-02-10T11:35:03.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.613 7ff2afe33700 1 -- start start 2021-02-10T11:35:03.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.614 7ff2afe33700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff2a80d1880 0x7ff2a81b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:03.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.614 7ff2afe33700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2a81b0af0 0x7ff2a8108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:03.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.614 7ff2afe33700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a80fe1d0 con 0x7ff2a80d1c80 2021-02-10T11:35:03.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.614 7ff2afe33700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a8109a70 con 0x7ff2a80d1880 2021-02-10T11:35:03.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.614 7ff2afe33700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a8109760 con 0x7ff2a81b0af0 2021-02-10T11:35:03.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.614 7ff2adc4f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff2a80d1880 0x7ff2a81b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:03.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.614 7ff2adc4f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff2a80d1880 0x7ff2a81b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52848/0 (socket says 172.21.2.101:52848) 2021-02-10T11:35:03.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.614 7ff2adc4f700 1 -- 172.21.2.101:0/2733894391 learned_addr learned my addr 172.21.2.101:0/2733894391 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:03.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.614 7ff2ad44e700 1 --2- 172.21.2.101:0/2733894391 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2a81b0af0 0x7ff2a8108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:03.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.614 7ff2adc4f700 1 -- 172.21.2.101:0/2733894391 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2a81b0af0 msgr2=0x7ff2a8108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:03.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.614 7ff2adc4f700 1 --2- 172.21.2.101:0/2733894391 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2a81b0af0 0x7ff2a8108950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:03.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.614 7ff2adc4f700 1 -- 172.21.2.101:0/2733894391 >> v1:172.21.2.108:6789/0 conn(0x7ff2a80d1c80 legacy=0x7ff2a8108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:03.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.614 7ff2adc4f700 1 -- 172.21.2.101:0/2733894391 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a8088070 con 0x7ff2a80d1880 2021-02-10T11:35:03.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.614 7ff2adc4f700 1 --2- 172.21.2.101:0/2733894391 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2a80d1880 0x7ff2a81b04d0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 rx=0x7ff29c006700 tx=0x7ff29c009b00).ready entity=mon.0 client_cookie=ba08ec2dd7729b50 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:03.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.615 7ff2acc4d700 1 -- 172.21.2.101:0/2733894391 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff29c00c0c0 con 0x7ff2a80d1880 2021-02-10T11:35:03.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.615 7ff2acc4d700 1 -- 172.21.2.101:0/2733894391 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c00c400 con 0x7ff2a80d1880 2021-02-10T11:35:03.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.615 7ff2acc4d700 1 -- 172.21.2.101:0/2733894391 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff29c00c920 con 0x7ff2a80d1880 2021-02-10T11:35:03.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.615 7ff2afe33700 1 -- 172.21.2.101:0/2733894391 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2a80d1880 msgr2=0x7ff2a81b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:03.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.615 7ff2afe33700 1 --2- 172.21.2.101:0/2733894391 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2a80d1880 0x7ff2a81b04d0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 rx=0x7ff29c006700 tx=0x7ff29c009b00).stop 2021-02-10T11:35:03.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.615 7ff2afe33700 1 -- 172.21.2.101:0/2733894391 shutdown_connections 2021-02-10T11:35:03.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.615 7ff2afe33700 1 --2- 172.21.2.101:0/2733894391 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2a81b0af0 0x7ff2a8108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:03.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.615 7ff2afe33700 1 --2- 172.21.2.101:0/2733894391 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2a80d1880 0x7ff2a81b04d0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:03.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.615 7ff2afe33700 1 -- 172.21.2.101:0/2733894391 >> 172.21.2.101:0/2733894391 conn(0x7ff2a80ce430 msgr2=0x7ff2a80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:03.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.616 7ff2afe33700 1 -- 172.21.2.101:0/2733894391 shutdown_connections 2021-02-10T11:35:03.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.616 7ff2afe33700 1 -- 172.21.2.101:0/2733894391 wait complete. 2021-02-10T11:35:03.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.630 7ff2afe33700 1 Processor -- start 2021-02-10T11:35:03.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.630 7ff2afe33700 1 -- start start 2021-02-10T11:35:03.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.631 7ff2afe33700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2a80ad520 0x7ff2a80ad920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:03.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.631 7ff2afe33700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff2a80adf40 0x7ff2a8097c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:03.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.631 7ff2afe33700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2a8098250 0x7ff2a8098650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:03.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.631 7ff2afe33700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a8098c70 con 0x7ff2a80ad520 2021-02-10T11:35:03.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.631 7ff2afe33700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80910f0 con 0x7ff2a80adf40 2021-02-10T11:35:03.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.631 7ff2afe33700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a8091310 con 0x7ff2a8098250 2021-02-10T11:35:03.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.631 7ff2adc4f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2a80ad520 0x7ff2a80ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:03.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.631 7ff2adc4f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2a80ad520 0x7ff2a80ad920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52852/0 (socket says 172.21.2.101:52852) 2021-02-10T11:35:03.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.631 7ff2adc4f700 1 -- 172.21.2.101:0/2596855717 learned_addr learned my addr 172.21.2.101:0/2596855717 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:03.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.631 7ff2ad44e700 1 --2- 172.21.2.101:0/2596855717 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff2a80adf40 0x7ff2a8097c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:03.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.631 7ff2ae450700 1 --2- 172.21.2.101:0/2596855717 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2a8098250 0x7ff2a8098650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:03.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.631 7ff2adc4f700 1 -- 172.21.2.101:0/2596855717 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2a8098250 msgr2=0x7ff2a8098650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:03.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.631 7ff2adc4f700 1 --2- 172.21.2.101:0/2596855717 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2a8098250 0x7ff2a8098650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:03.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.631 7ff2adc4f700 1 -- 172.21.2.101:0/2596855717 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff2a80adf40 msgr2=0x7ff2a8097c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:03.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.631 7ff2adc4f700 1 --2- 172.21.2.101:0/2596855717 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff2a80adf40 0x7ff2a8097c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:03.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.631 7ff2adc4f700 1 -- 172.21.2.101:0/2596855717 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a819a3b0 con 0x7ff2a80ad520 2021-02-10T11:35:03.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.631 7ff2ad44e700 1 --2- 172.21.2.101:0/2596855717 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff2a80adf40 0x7ff2a8097c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:35:03.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.631 7ff2adc4f700 1 --2- 172.21.2.101:0/2596855717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2a80ad520 0x7ff2a80ad920 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 rx=0x7ff29c00b660 tx=0x7ff29c00ba00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:03.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.632 7ff29a7fc700 1 -- 172.21.2.101:0/2596855717 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff29c009440 con 0x7ff2a80ad520 2021-02-10T11:35:03.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.632 7ff29a7fc700 1 -- 172.21.2.101:0/2596855717 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c000c90 con 0x7ff2a80ad520 2021-02-10T11:35:03.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.632 7ff29a7fc700 1 -- 172.21.2.101:0/2596855717 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff29c002030 con 0x7ff2a80ad520 2021-02-10T11:35:03.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.632 7ff2afe33700 1 -- 172.21.2.101:0/2596855717 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a819a3b0 con 0x7ff2a80ad520 2021-02-10T11:35:03.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.632 7ff2afe33700 1 -- 172.21.2.101:0/2596855717 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a819a3b0 con 0x7ff2a80ad520 2021-02-10T11:35:03.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.632 7ff29a7fc700 1 -- 172.21.2.101:0/2596855717 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff29c0093e0 con 0x7ff2a80ad520 2021-02-10T11:35:03.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.632 7ff29a7fc700 1 --2- 172.21.2.101:0/2596855717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff290018730 0x7ff29001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:03.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.632 7ff29a7fc700 1 -- 172.21.2.101:0/2596855717 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff29c032650 con 0x7ff2a80ad520 2021-02-10T11:35:03.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.633 7ff2ad44e700 1 --2- 172.21.2.101:0/2596855717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff290018730 0x7ff29001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:03.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.633 7ff2ad44e700 1 --2- 172.21.2.101:0/2596855717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff290018730 0x7ff29001abc0 crc :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:03.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.664 7ff2afe33700 1 -- 172.21.2.101:0/2596855717 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a8082a70 con 0x7ff2a80ad520 2021-02-10T11:35:03.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.666 7ff29a7fc700 1 -- 172.21.2.101:0/2596855717 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff29c0090c0 con 0x7ff2a80ad520 2021-02-10T11:35:03.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.802 7ff2afe33700 1 -- 172.21.2.101:0/2596855717 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff2a8082a70 con 0x7ff2a80ad520 2021-02-10T11:35:03.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.857 7ff29a7fc700 1 -- 172.21.2.101:0/2596855717 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v47) v1 ==== 136+0+0 (secure 0 0 0) 0x7ff29c000c90 con 0x7ff2a80ad520 2021-02-10T11:35:03.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.869 7ff2afe33700 1 -- 172.21.2.101:0/2596855717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff290018730 msgr2=0x7ff29001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:03.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.869 7ff2afe33700 1 --2- 172.21.2.101:0/2596855717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff290018730 0x7ff29001abc0 crc :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:03.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.869 7ff2afe33700 1 -- 172.21.2.101:0/2596855717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2a80ad520 msgr2=0x7ff2a80ad920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:03.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.869 7ff2afe33700 1 --2- 172.21.2.101:0/2596855717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2a80ad520 0x7ff2a80ad920 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 rx=0x7ff29c00b660 tx=0x7ff29c00ba00).stop 2021-02-10T11:35:03.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.869 7ff2afe33700 1 -- 172.21.2.101:0/2596855717 shutdown_connections 2021-02-10T11:35:03.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.869 7ff2afe33700 1 --2- 172.21.2.101:0/2596855717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff290018730 0x7ff29001abc0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:03.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.869 7ff2afe33700 1 --2- 172.21.2.101:0/2596855717 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2a8098250 0x7ff2a8098650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:03.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.869 7ff2afe33700 1 --2- 172.21.2.101:0/2596855717 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff2a80adf40 0x7ff2a8097c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:03.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.869 7ff2afe33700 1 --2- 172.21.2.101:0/2596855717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2a80ad520 0x7ff2a80ad920 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:03.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.869 7ff2afe33700 1 -- 172.21.2.101:0/2596855717 >> 172.21.2.101:0/2596855717 conn(0x7ff2a818e810 msgr2=0x7ff2a818c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:03.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.869 7ff2afe33700 1 -- 172.21.2.101:0/2596855717 shutdown_connections 2021-02-10T11:35:03.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:03.870 7ff2afe33700 1 -- 172.21.2.101:0/2596855717 wait complete. 2021-02-10T11:35:03.886 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs volume ls 2021-02-10T11:35:04.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.287 7f18d6d4c700 1 Processor -- start 2021-02-10T11:35:04.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.287 7f18d6d4c700 1 -- start start 2021-02-10T11:35:04.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.288 7f18d6d4c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f18d00d1c70 0x7f18d00f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:04.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.288 7f18d6d4c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d00f0b10 con 0x7f18d00d1870 2021-02-10T11:35:04.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.288 7f18d6d4c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d01b0be0 con 0x7f18d00d0340 2021-02-10T11:35:04.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.288 7f18d6d4c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d00e5ed0 con 0x7f18d00d1c70 2021-02-10T11:35:04.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.288 7f18d4b68700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f18d00d1870 0x7f18d01b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53354/0 (socket says 172.21.2.101:53354) 2021-02-10T11:35:04.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.288 7f18d4b68700 1 -- 172.21.2.101:0/3567674542 learned_addr learned my addr 172.21.2.101:0/3567674542 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:04.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.288 7f18d5369700 1 --2- 172.21.2.101:0/3567674542 >> v2:172.21.2.108:3300/0 conn(0x7f18d00d1c70 0x7f18d00f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:04.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.288 7f18cf7fe700 1 -- 172.21.2.101:0/3567674542 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2463161418 0 0) 0x7f18b80010b0 con 0x7f18d00d1870 2021-02-10T11:35:04.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.288 7f18cf7fe700 1 -- 172.21.2.101:0/3567674542 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18bc002fe0 con 0x7f18d00d1870 2021-02-10T11:35:04.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.289 7f18d5369700 1 -- 172.21.2.101:0/3567674542 >> v1:172.21.2.108:6789/0 conn(0x7f18d00d0340 legacy=0x7f18d00ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:04.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.289 7f18d5369700 1 -- 172.21.2.101:0/3567674542 >> v1:172.21.2.101:6789/0 conn(0x7f18d00d1870 legacy=0x7f18d01b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:04.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.289 7f18d5369700 1 -- 172.21.2.101:0/3567674542 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d00efe30 con 0x7f18d00d1c70 2021-02-10T11:35:04.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.289 7f18d5369700 1 --2- 172.21.2.101:0/3567674542 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f18d00d1c70 0x7f18d00f20a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f18c8006e50 tx=0x7f18c8009e70).ready entity=mon.1 client_cookie=bb0358a0c556588 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:04.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.289 7f18cf7fe700 1 -- 172.21.2.101:0/3567674542 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f18c8006a20 con 0x7f18d00d1c70 2021-02-10T11:35:04.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.289 7f18cf7fe700 1 -- 172.21.2.101:0/3567674542 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c8006790 con 0x7f18d00d1c70 2021-02-10T11:35:04.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.289 7f18cf7fe700 1 -- 172.21.2.101:0/3567674542 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f18c800b700 con 0x7f18d00d1c70 2021-02-10T11:35:04.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.289 7f18d6d4c700 1 -- 172.21.2.101:0/3567674542 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f18d00d1c70 msgr2=0x7f18d00f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:04.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.289 7f18d6d4c700 1 --2- 172.21.2.101:0/3567674542 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f18d00d1c70 0x7f18d00f20a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f18c8006e50 tx=0x7f18c8009e70).stop 2021-02-10T11:35:04.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.290 7f18d6d4c700 1 -- 172.21.2.101:0/3567674542 shutdown_connections 2021-02-10T11:35:04.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.290 7f18d6d4c700 1 --2- 172.21.2.101:0/3567674542 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f18d00d1c70 0x7f18d00f20a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:04.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.290 7f18d6d4c700 1 -- 172.21.2.101:0/3567674542 >> 172.21.2.101:0/3567674542 conn(0x7f18d00ce450 msgr2=0x7f18d00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:04.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.290 7f18d6d4c700 1 -- 172.21.2.101:0/3567674542 shutdown_connections 2021-02-10T11:35:04.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.290 7f18d6d4c700 1 -- 172.21.2.101:0/3567674542 wait complete. 2021-02-10T11:35:04.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.305 7f18d6d4c700 1 Processor -- start 2021-02-10T11:35:04.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.305 7f18d6d4c700 1 -- start start 2021-02-10T11:35:04.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.305 7f18d6d4c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18d00b4840 0x7f18d0093890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:04.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.305 7f18d6d4c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f18d0093eb0 0x7f18d0188490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:04.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.305 7f18d6d4c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18d00942b0 0x7f18d0188ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:04.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.305 7f18d6d4c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00b4c40 con 0x7f18d00942b0 2021-02-10T11:35:04.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.305 7f18d6d4c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00b4e30 con 0x7f18d0093eb0 2021-02-10T11:35:04.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.305 7f18d6d4c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d0189100 con 0x7f18d00b4840 2021-02-10T11:35:04.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.306 7f18d5369700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18d00942b0 0x7f18d0188ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:04.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.306 7f18d5369700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18d00942b0 0x7f18d0188ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52866/0 (socket says 172.21.2.101:52866) 2021-02-10T11:35:04.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.306 7f18d5369700 1 -- 172.21.2.101:0/1037812415 learned_addr learned my addr 172.21.2.101:0/1037812415 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:04.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.306 7f18d5369700 1 -- 172.21.2.101:0/1037812415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18d00b4840 msgr2=0x7f18d0093890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:04.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.306 7f18d5369700 1 --2- 172.21.2.101:0/1037812415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18d00b4840 0x7f18d0093890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:04.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.306 7f18d5369700 1 -- 172.21.2.101:0/1037812415 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f18d0093eb0 msgr2=0x7f18d0188490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:04.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.306 7f18d5369700 1 --2- 172.21.2.101:0/1037812415 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f18d0093eb0 0x7f18d0188490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:04.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.306 7f18d5369700 1 -- 172.21.2.101:0/1037812415 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d0189320 con 0x7f18d00942b0 2021-02-10T11:35:04.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.306 7f18d5369700 1 --2- 172.21.2.101:0/1037812415 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18d00942b0 0x7f18d0188ab0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 rx=0x7f18c8006e60 tx=0x7f18c800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:04.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.306 7f18cd7fa700 1 -- 172.21.2.101:0/1037812415 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f18c8009130 con 0x7f18d00942b0 2021-02-10T11:35:04.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.306 7f18cd7fa700 1 -- 172.21.2.101:0/1037812415 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c8000df0 con 0x7f18d00942b0 2021-02-10T11:35:04.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.306 7f18d6d4c700 1 -- 172.21.2.101:0/1037812415 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d0189320 con 0x7f18d00942b0 2021-02-10T11:35:04.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.306 7f18d6d4c700 1 -- 172.21.2.101:0/1037812415 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d0092700 con 0x7f18d00942b0 2021-02-10T11:35:04.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.307 7f18cd7fa700 1 -- 172.21.2.101:0/1037812415 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f18c8001420 con 0x7f18d00942b0 2021-02-10T11:35:04.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.307 7f18cd7fa700 1 -- 172.21.2.101:0/1037812415 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f18c80011a0 con 0x7f18d00942b0 2021-02-10T11:35:04.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.307 7f18cd7fa700 1 --2- 172.21.2.101:0/1037812415 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18b4018700 0x7f18b401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:04.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.307 7f18cd7fa700 1 -- 172.21.2.101:0/1037812415 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f18c8032b40 con 0x7f18d00942b0 2021-02-10T11:35:04.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.307 7f18d4b68700 1 --2- 172.21.2.101:0/1037812415 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18b4018700 0x7f18b401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:04.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.308 7f18d4b68700 1 --2- 172.21.2.101:0/1037812415 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18b4018700 0x7f18b401abc0 crc :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:04.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.338 7f18d6d4c700 1 -- 172.21.2.101:0/1037812415 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d00ee0f0 con 0x7f18d00942b0 2021-02-10T11:35:04.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.340 7f18cd7fa700 1 -- 172.21.2.101:0/1037812415 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f18c80093d0 con 0x7f18d00942b0 2021-02-10T11:35:04.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.475 7f18d6d4c700 1 -- 172.21.2.101:0/1037812415 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs volume ls", "target": ["mgr", ""]}) v1 -- 0x7f18d01890d0 con 0x7f18b4018700 2021-02-10T11:35:04.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.476 7f18cd7fa700 1 -- 172.21.2.101:0/1037812415 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7f18b8001520 con 0x7f18b4018700 2021-02-10T11:35:04.484 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:35:04.485 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:04.485 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "cephfs" 2021-02-10T11:35:04.485 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:04.485 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:35:04.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.477 7f18d6d4c700 1 -- 172.21.2.101:0/1037812415 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18b4018700 msgr2=0x7f18b401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:04.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.477 7f18d6d4c700 1 --2- 172.21.2.101:0/1037812415 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18b4018700 0x7f18b401abc0 crc :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:04.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.477 7f18d6d4c700 1 -- 172.21.2.101:0/1037812415 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18d00942b0 msgr2=0x7f18d0188ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:04.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.477 7f18d6d4c700 1 --2- 172.21.2.101:0/1037812415 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18d00942b0 0x7f18d0188ab0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 rx=0x7f18c8006e60 tx=0x7f18c800bcf0).stop 2021-02-10T11:35:04.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.477 7f18d6d4c700 1 -- 172.21.2.101:0/1037812415 shutdown_connections 2021-02-10T11:35:04.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.477 7f18d6d4c700 1 --2- 172.21.2.101:0/1037812415 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18d00942b0 0x7f18d0188ab0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:04.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.477 7f18d6d4c700 1 --2- 172.21.2.101:0/1037812415 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f18d0093eb0 0x7f18d0188490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:04.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.477 7f18d6d4c700 1 --2- 172.21.2.101:0/1037812415 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18b4018700 0x7f18b401abc0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:04.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.477 7f18d6d4c700 1 --2- 172.21.2.101:0/1037812415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18d00b4840 0x7f18d0093890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:04.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.477 7f18d6d4c700 1 -- 172.21.2.101:0/1037812415 >> 172.21.2.101:0/1037812415 conn(0x7f18d00946f0 msgr2=0x7f18d0095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:04.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.478 7f18d6d4c700 1 -- 172.21.2.101:0/1037812415 shutdown_connections 2021-02-10T11:35:04.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.478 7f18d6d4c700 1 -- 172.21.2.101:0/1037812415 wait complete. 2021-02-10T11:35:04.495 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:35:04.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.901 7f02d300e700 1 Processor -- start 2021-02-10T11:35:04.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.901 7f02d300e700 1 -- start start 2021-02-10T11:35:04.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.902 7f02d300e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f02cc0d1870 0x7f02cc1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:04.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.902 7f02d300e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02cc0cff70 0x7f02cc109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:04.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.902 7f02d300e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02cc109a60 con 0x7f02cc1b0ae0 2021-02-10T11:35:04.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.902 7f02d300e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02cc109750 con 0x7f02cc0d1870 2021-02-10T11:35:04.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.902 7f02d300e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc088070 con 0x7f02cc0cff70 2021-02-10T11:35:04.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.902 7f02cbfff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f02cc1b0ae0 0x7f02cc108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53364/0 (socket says 172.21.2.101:53364) 2021-02-10T11:35:04.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.902 7f02cbfff700 1 -- 172.21.2.101:0/3304642304 learned_addr learned my addr 172.21.2.101:0/3304642304 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:04.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.902 7f02d0e2a700 1 --2- 172.21.2.101:0/3304642304 >> v2:172.21.2.101:3300/0 conn(0x7f02cc0d1870 0x7f02cc1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:04.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.902 7f02cb7fe700 1 -- 172.21.2.101:0/3304642304 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792085144 0 0) 0x7f02bc000ce0 con 0x7f02cc1b0ae0 2021-02-10T11:35:04.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.902 7f02cb7fe700 1 -- 172.21.2.101:0/3304642304 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02b8002fe0 con 0x7f02cc1b0ae0 2021-02-10T11:35:04.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.902 7f02d0e2a700 1 -- 172.21.2.101:0/3304642304 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02cc0cff70 msgr2=0x7f02cc109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:04.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.902 7f02d0e2a700 1 --2- 172.21.2.101:0/3304642304 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02cc0cff70 0x7f02cc109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:04.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.902 7f02d0e2a700 1 -- 172.21.2.101:0/3304642304 >> v1:172.21.2.101:6789/0 conn(0x7f02cc1b0ae0 legacy=0x7f02cc108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:04.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.902 7f02d0e2a700 1 -- 172.21.2.101:0/3304642304 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02cc0d1c70 con 0x7f02cc0d1870 2021-02-10T11:35:04.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.902 7f02d0e2a700 1 --2- 172.21.2.101:0/3304642304 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02cc0d1870 0x7f02cc1b04c0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 rx=0x7f02b4006650 tx=0x7f02b4009a80).ready entity=mon.0 client_cookie=fbc56e6cce9786e6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:04.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.903 7f02cb7fe700 1 -- 172.21.2.101:0/3304642304 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f02b400c0c0 con 0x7f02cc0d1870 2021-02-10T11:35:04.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.903 7f02cb7fe700 1 -- 172.21.2.101:0/3304642304 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b400c340 con 0x7f02cc0d1870 2021-02-10T11:35:04.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.903 7f02cb7fe700 1 -- 172.21.2.101:0/3304642304 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f02b400c820 con 0x7f02cc0d1870 2021-02-10T11:35:04.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.903 7f02d300e700 1 -- 172.21.2.101:0/3304642304 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02cc0d1870 msgr2=0x7f02cc1b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:04.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.903 7f02d300e700 1 --2- 172.21.2.101:0/3304642304 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02cc0d1870 0x7f02cc1b04c0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 rx=0x7f02b4006650 tx=0x7f02b4009a80).stop 2021-02-10T11:35:04.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.903 7f02d300e700 1 -- 172.21.2.101:0/3304642304 shutdown_connections 2021-02-10T11:35:04.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.903 7f02d300e700 1 --2- 172.21.2.101:0/3304642304 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02cc0cff70 0x7f02cc109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:04.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.903 7f02d300e700 1 --2- 172.21.2.101:0/3304642304 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02cc0d1870 0x7f02cc1b04c0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:04.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.903 7f02d300e700 1 -- 172.21.2.101:0/3304642304 >> 172.21.2.101:0/3304642304 conn(0x7f02cc0ce420 msgr2=0x7f02cc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:04.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.903 7f02d300e700 1 -- 172.21.2.101:0/3304642304 shutdown_connections 2021-02-10T11:35:04.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.903 7f02d300e700 1 -- 172.21.2.101:0/3304642304 wait complete. 2021-02-10T11:35:04.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.918 7f02d300e700 1 Processor -- start 2021-02-10T11:35:04.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.918 7f02d300e700 1 -- start start 2021-02-10T11:35:04.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.919 7f02d300e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02cc0c3820 0x7f02cc084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:04.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.919 7f02d300e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02cc0a0290 0x7f02cc0852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:04.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.919 7f02d300e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02cc086270 0x7f02cc086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:04.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.919 7f02d300e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc0c0700 con 0x7f02cc086270 2021-02-10T11:35:04.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.919 7f02d300e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc0858f0 con 0x7f02cc0c3820 2021-02-10T11:35:04.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.919 7f02d300e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc085b10 con 0x7f02cc0a0290 2021-02-10T11:35:04.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.919 7f02d162b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02cc086270 0x7f02cc086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:04.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.919 7f02d162b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02cc086270 0x7f02cc086670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52880/0 (socket says 172.21.2.101:52880) 2021-02-10T11:35:04.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.919 7f02d162b700 1 -- 172.21.2.101:0/1687907285 learned_addr learned my addr 172.21.2.101:0/1687907285 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:04.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.919 7f02cbfff700 1 --2- 172.21.2.101:0/1687907285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02cc0a0290 0x7f02cc0852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:04.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.919 7f02d0e2a700 1 --2- 172.21.2.101:0/1687907285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02cc0c3820 0x7f02cc084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:04.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.919 7f02d162b700 1 -- 172.21.2.101:0/1687907285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02cc0a0290 msgr2=0x7f02cc0852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:04.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.919 7f02d162b700 1 --2- 172.21.2.101:0/1687907285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02cc0a0290 0x7f02cc0852d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:04.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.919 7f02d162b700 1 -- 172.21.2.101:0/1687907285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02cc0c3820 msgr2=0x7f02cc084cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:04.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.919 7f02d162b700 1 --2- 172.21.2.101:0/1687907285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02cc0c3820 0x7f02cc084cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:04.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.919 7f02d162b700 1 -- 172.21.2.101:0/1687907285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02cc086c90 con 0x7f02cc086270 2021-02-10T11:35:04.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.919 7f02d162b700 1 --2- 172.21.2.101:0/1687907285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02cc086270 0x7f02cc086670 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 rx=0x7f02c400b8c0 tx=0x7f02c400bc60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:04.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.919 7f02c97fa700 1 -- 172.21.2.101:0/1687907285 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f02c4005af0 con 0x7f02cc086270 2021-02-10T11:35:04.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.919 7f02c97fa700 1 -- 172.21.2.101:0/1687907285 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c4005d30 con 0x7f02cc086270 2021-02-10T11:35:04.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.919 7f02c97fa700 1 -- 172.21.2.101:0/1687907285 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f02c40062b0 con 0x7f02cc086270 2021-02-10T11:35:04.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.919 7f02d300e700 1 -- 172.21.2.101:0/1687907285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02cc086c90 con 0x7f02cc086270 2021-02-10T11:35:04.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.919 7f02d300e700 1 -- 172.21.2.101:0/1687907285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02cc086c90 con 0x7f02cc086270 2021-02-10T11:35:04.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.920 7f02c97fa700 1 -- 172.21.2.101:0/1687907285 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f02c4006030 con 0x7f02cc086270 2021-02-10T11:35:04.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.920 7f02c97fa700 1 --2- 172.21.2.101:0/1687907285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02b00186c0 0x7f02b001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:04.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.920 7f02c97fa700 1 -- 172.21.2.101:0/1687907285 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f02c4032930 con 0x7f02cc086270 2021-02-10T11:35:04.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.920 7f02d0e2a700 1 --2- 172.21.2.101:0/1687907285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02b00186c0 0x7f02b001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:04.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.920 7f02d0e2a700 1 --2- 172.21.2.101:0/1687907285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02b00186c0 0x7f02b001abc0 crc :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:04.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.951 7f02d300e700 1 -- 172.21.2.101:0/1687907285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02cc082a70 con 0x7f02cc086270 2021-02-10T11:35:04.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:04.954 7f02c97fa700 1 -- 172.21.2.101:0/1687907285 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f02c4005ad0 con 0x7f02cc086270 2021-02-10T11:35:05.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:05.102 7f02d300e700 1 -- 172.21.2.101:0/1687907285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f02cc082a70 con 0x7f02cc086270 2021-02-10T11:35:05.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:05.102 7f02c97fa700 1 -- 172.21.2.101:0/1687907285 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f02c4006030 con 0x7f02cc086270 2021-02-10T11:35:05.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:05.104 7f02d300e700 1 -- 172.21.2.101:0/1687907285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02b00186c0 msgr2=0x7f02b001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:05.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:05.104 7f02d300e700 1 --2- 172.21.2.101:0/1687907285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02b00186c0 0x7f02b001abc0 crc :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:05.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:05.104 7f02d300e700 1 -- 172.21.2.101:0/1687907285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02cc086270 msgr2=0x7f02cc086670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:05.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:05.104 7f02d300e700 1 --2- 172.21.2.101:0/1687907285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02cc086270 0x7f02cc086670 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 rx=0x7f02c400b8c0 tx=0x7f02c400bc60).stop 2021-02-10T11:35:05.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:05.104 7f02d300e700 1 -- 172.21.2.101:0/1687907285 shutdown_connections 2021-02-10T11:35:05.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:05.104 7f02d300e700 1 --2- 172.21.2.101:0/1687907285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f02b00186c0 0x7f02b001abc0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:05.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:05.104 7f02d300e700 1 --2- 172.21.2.101:0/1687907285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02cc0a0290 0x7f02cc0852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:05.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:05.104 7f02d300e700 1 --2- 172.21.2.101:0/1687907285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02cc086270 0x7f02cc086670 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:05.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:05.104 7f02d300e700 1 --2- 172.21.2.101:0/1687907285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02cc0c3820 0x7f02cc084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:05.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:05.104 7f02d300e700 1 -- 172.21.2.101:0/1687907285 >> 172.21.2.101:0/1687907285 conn(0x7f02cc0872c0 msgr2=0x7f02cc095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:05.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:05.104 7f02d300e700 1 -- 172.21.2.101:0/1687907285 shutdown_connections 2021-02-10T11:35:05.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:05.104 7f02d300e700 1 -- 172.21.2.101:0/1687907285 wait complete. 2021-02-10T11:35:05.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:05.122 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:35:05.122 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:05.458 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:35:05.458 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:35:05.458 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:35:05.458 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:35:05.458 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:05.812 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[124926]: fuse finished with error 0 and tester_r 0 2021-02-10T11:35:05.813 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:05.813 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:35:05.814 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:06.148 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:35:06.149 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:35:06.149 INFO:teuthology.orchestra.run:waiting for 900 2021-02-10T11:35:06.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:35:06.149 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:06.487 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolumegroup create cephfs group_0000000000497714 2021-02-10T11:35:06.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.893 7f9fbb1c7700 1 Processor -- start 2021-02-10T11:35:06.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.893 7f9fbb1c7700 1 -- start start 2021-02-10T11:35:06.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.893 7f9fbb1c7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f9fb40d1c70 0x7f9fb40cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:06.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.893 7f9fbb1c7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fb40cf580 0x7f9fb40cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:06.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.893 7f9fbb1c7700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb40fe1c0 con 0x7f9fb40d1870 2021-02-10T11:35:06.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.893 7f9fbb1c7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb41b0cb0 con 0x7f9fb40d1c70 2021-02-10T11:35:06.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.893 7f9fbb1c7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb4109a60 con 0x7f9fb40cf580 2021-02-10T11:35:06.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.894 7f9fb8fe3700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f9fb40d1870 0x7f9fb41b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53378/0 (socket says 172.21.2.101:53378) 2021-02-10T11:35:06.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.894 7f9fb8fe3700 1 -- 172.21.2.101:0/1205235992 learned_addr learned my addr 172.21.2.101:0/1205235992 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:06.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.894 7f9fb97e4700 1 --2- 172.21.2.101:0/1205235992 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fb40cf580 0x7f9fb40cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:06.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.894 7f9fb3fff700 1 --2- 172.21.2.101:0/1205235992 >> v2:172.21.2.108:3300/0 conn(0x7f9fb40d1c70 0x7f9fb40cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:06.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.894 7f9fb37fe700 1 -- 172.21.2.101:0/1205235992 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3424556563 0 0) 0x7f9f9c000ce0 con 0x7f9fb40d1870 2021-02-10T11:35:06.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.894 7f9fb37fe700 1 -- 172.21.2.101:0/1205235992 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fa0002fe0 con 0x7f9fb40d1870 2021-02-10T11:35:06.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.894 7f9fb97e4700 1 -- 172.21.2.101:0/1205235992 >> v2:172.21.2.108:3300/0 conn(0x7f9fb40d1c70 msgr2=0x7f9fb40cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:06.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.894 7f9fb97e4700 1 --2- 172.21.2.101:0/1205235992 >> v2:172.21.2.108:3300/0 conn(0x7f9fb40d1c70 0x7f9fb40cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:06.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.894 7f9fb97e4700 1 -- 172.21.2.101:0/1205235992 >> v1:172.21.2.101:6789/0 conn(0x7f9fb40d1870 legacy=0x7f9fb41b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.894 7f9fb97e4700 1 -- 172.21.2.101:0/1205235992 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb4088070 con 0x7f9fb40cf580 2021-02-10T11:35:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.894 7f9fb97e4700 1 --2- 172.21.2.101:0/1205235992 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fb40cf580 0x7f9fb40cf980 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f9fac006e50 tx=0x7f9fac009e70).ready entity=mon.2 client_cookie=c705fa3296c586bc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.895 7f9fb37fe700 1 -- 172.21.2.101:0/1205235992 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9fac006a20 con 0x7f9fb40cf580 2021-02-10T11:35:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.895 7f9fb37fe700 1 -- 172.21.2.101:0/1205235992 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fac006790 con 0x7f9fb40cf580 2021-02-10T11:35:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.895 7f9fb37fe700 1 -- 172.21.2.101:0/1205235992 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9fac00b700 con 0x7f9fb40cf580 2021-02-10T11:35:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.895 7f9fbb1c7700 1 -- 172.21.2.101:0/1205235992 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fb40cf580 msgr2=0x7f9fb40cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.895 7f9fbb1c7700 1 --2- 172.21.2.101:0/1205235992 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fb40cf580 0x7f9fb40cf980 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f9fac006e50 tx=0x7f9fac009e70).stop 2021-02-10T11:35:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.895 7f9fbb1c7700 1 -- 172.21.2.101:0/1205235992 shutdown_connections 2021-02-10T11:35:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.895 7f9fbb1c7700 1 --2- 172.21.2.101:0/1205235992 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fb40cf580 0x7f9fb40cf980 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.895 7f9fbb1c7700 1 --2- 172.21.2.101:0/1205235992 >> v2:172.21.2.108:3300/0 conn(0x7f9fb40d1c70 0x7f9fb40cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.895 7f9fbb1c7700 1 -- 172.21.2.101:0/1205235992 >> 172.21.2.101:0/1205235992 conn(0x7f9fb40ce420 msgr2=0x7f9fb40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.895 7f9fbb1c7700 1 -- 172.21.2.101:0/1205235992 shutdown_connections 2021-02-10T11:35:06.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.895 7f9fbb1c7700 1 -- 172.21.2.101:0/1205235992 wait complete. 2021-02-10T11:35:06.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.910 7f9fbb1c7700 1 Processor -- start 2021-02-10T11:35:06.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.910 7f9fbb1c7700 1 -- start start 2021-02-10T11:35:06.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.910 7f9fbb1c7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9fb40bf840 0x7f9fb4093d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:06.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.910 7f9fbb1c7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fb4094350 0x7f9fb40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:06.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.910 7f9fbb1c7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fb40a3cb0 0x7f9fb40a40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:06.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.910 7f9fbb1c7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb40c0310 con 0x7f9fb40a3cb0 2021-02-10T11:35:06.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.910 7f9fbb1c7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb40c04c0 con 0x7f9fb40bf840 2021-02-10T11:35:06.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.910 7f9fbb1c7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb40c06e0 con 0x7f9fb4094350 2021-02-10T11:35:06.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.910 7f9fb97e4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fb40a3cb0 0x7f9fb40a40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:06.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.910 7f9fb97e4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fb40a3cb0 0x7f9fb40a40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52892/0 (socket says 172.21.2.101:52892) 2021-02-10T11:35:06.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.910 7f9fb97e4700 1 -- 172.21.2.101:0/3317861912 learned_addr learned my addr 172.21.2.101:0/3317861912 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:06.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.910 7f9fb97e4700 1 -- 172.21.2.101:0/3317861912 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fb4094350 msgr2=0x7f9fb40a3690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:35:06.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.910 7f9fb97e4700 1 --2- 172.21.2.101:0/3317861912 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fb4094350 0x7f9fb40a3690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:06.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.910 7f9fb97e4700 1 -- 172.21.2.101:0/3317861912 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9fb40bf840 msgr2=0x7f9fb4093d30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:35:06.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.910 7f9fb97e4700 1 --2- 172.21.2.101:0/3317861912 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9fb40bf840 0x7f9fb4093d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:06.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.910 7f9fb97e4700 1 -- 172.21.2.101:0/3317861912 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb40c0900 con 0x7f9fb40a3cb0 2021-02-10T11:35:06.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.910 7f9fb97e4700 1 --2- 172.21.2.101:0/3317861912 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fb40a3cb0 0x7f9fb40a40b0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 rx=0x7f9fac0099f0 tx=0x7f9fac00bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:06.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.911 7f9fb17fa700 1 -- 172.21.2.101:0/3317861912 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9fac0093d0 con 0x7f9fb40a3cb0 2021-02-10T11:35:06.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.911 7f9fb17fa700 1 -- 172.21.2.101:0/3317861912 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fac0090c0 con 0x7f9fb40a3cb0 2021-02-10T11:35:06.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.911 7f9fb17fa700 1 -- 172.21.2.101:0/3317861912 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9fac001390 con 0x7f9fb40a3cb0 2021-02-10T11:35:06.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.911 7f9fbb1c7700 1 -- 172.21.2.101:0/3317861912 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb40c0900 con 0x7f9fb40a3cb0 2021-02-10T11:35:06.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.911 7f9fbb1c7700 1 -- 172.21.2.101:0/3317861912 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb40c0ba0 con 0x7f9fb40a3cb0 2021-02-10T11:35:06.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.911 7f9fb17fa700 1 -- 172.21.2.101:0/3317861912 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9fac0056b0 con 0x7f9fb40a3cb0 2021-02-10T11:35:06.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.911 7f9fb17fa700 1 --2- 172.21.2.101:0/3317861912 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f980186c0 0x7f9f9801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:06.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.911 7f9fb17fa700 1 -- 172.21.2.101:0/3317861912 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9fac032740 con 0x7f9fb40a3cb0 2021-02-10T11:35:06.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.911 7f9fb8fe3700 1 --2- 172.21.2.101:0/3317861912 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f980186c0 0x7f9f9801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:06.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.912 7f9fb8fe3700 1 --2- 172.21.2.101:0/3317861912 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f980186c0 0x7f9f9801abc0 crc :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:06.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.943 7f9fbb1c7700 1 -- 172.21.2.101:0/3317861912 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb40bfde0 con 0x7f9fb40a3cb0 2021-02-10T11:35:06.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:06.946 7f9fb17fa700 1 -- 172.21.2.101:0/3317861912 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9fac028450 con 0x7f9fb40a3cb0 2021-02-10T11:35:07.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.073 7f9fbb1c7700 1 -- 172.21.2.101:0/3317861912 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "target": ["mgr", ""], "group_name": "group_0000000000497714"}) v1 -- 0x7f9fb40c0ba0 con 0x7f9f980186c0 2021-02-10T11:35:07.085 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:35:07.076 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:35:07.085 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:35:07.076 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:35:07.085 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:35:07.076 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:35:07.085 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:35:07.076 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:35:07.085 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:35:07.076 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:35:07.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.089 7f9fb17fa700 1 -- 172.21.2.101:0/3317861912 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f9f9c000ce0 con 0x7f9f980186c0 2021-02-10T11:35:07.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.106 7f9fbb1c7700 1 -- 172.21.2.101:0/3317861912 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f980186c0 msgr2=0x7f9f9801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:07.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.106 7f9fbb1c7700 1 --2- 172.21.2.101:0/3317861912 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f980186c0 0x7f9f9801abc0 crc :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:07.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.106 7f9fbb1c7700 1 -- 172.21.2.101:0/3317861912 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fb40a3cb0 msgr2=0x7f9fb40a40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:07.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.106 7f9fbb1c7700 1 --2- 172.21.2.101:0/3317861912 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fb40a3cb0 0x7f9fb40a40b0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 rx=0x7f9fac0099f0 tx=0x7f9fac00bd10).stop 2021-02-10T11:35:07.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.106 7f9fbb1c7700 1 -- 172.21.2.101:0/3317861912 shutdown_connections 2021-02-10T11:35:07.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.106 7f9fbb1c7700 1 --2- 172.21.2.101:0/3317861912 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f980186c0 0x7f9f9801abc0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:07.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.106 7f9fbb1c7700 1 --2- 172.21.2.101:0/3317861912 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fb4094350 0x7f9fb40a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:07.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.106 7f9fbb1c7700 1 --2- 172.21.2.101:0/3317861912 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fb40a3cb0 0x7f9fb40a40b0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:07.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.106 7f9fbb1c7700 1 --2- 172.21.2.101:0/3317861912 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9fb40bf840 0x7f9fb4093d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:07.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.106 7f9fbb1c7700 1 -- 172.21.2.101:0/3317861912 >> 172.21.2.101:0/3317861912 conn(0x7f9fb4094b90 msgr2=0x7f9fb40b8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:07.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.107 7f9fbb1c7700 1 -- 172.21.2.101:0/3317861912 shutdown_connections 2021-02-10T11:35:07.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.107 7f9fbb1c7700 1 -- 172.21.2.101:0/3317861912 wait complete. 2021-02-10T11:35:07.123 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000063820 --group_name group_0000000000497714 2021-02-10T11:35:07.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.529 7f83b91c1700 1 Processor -- start 2021-02-10T11:35:07.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.529 7f83b91c1700 1 -- start start 2021-02-10T11:35:07.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.530 7f83b91c1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83b40d1880 0x7f83b41b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:07.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.530 7f83b91c1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f83b40cff80 0x7f83b4109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:07.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.530 7f83b91c1700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b4109a70 con 0x7f83b41b0af0 2021-02-10T11:35:07.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.530 7f83b91c1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b4109760 con 0x7f83b40cff80 2021-02-10T11:35:07.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.530 7f83b91c1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b4088070 con 0x7f83b40d1880 2021-02-10T11:35:07.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.530 7f83b259c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f83b41b0af0 0x7f83b4108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53410/0 (socket says 172.21.2.101:53410) 2021-02-10T11:35:07.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.530 7f83b259c700 1 -- 172.21.2.101:0/523281281 learned_addr learned my addr 172.21.2.101:0/523281281 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:07.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.530 7f83b359e700 1 --2- 172.21.2.101:0/523281281 >> v2:172.21.2.108:3300/0 conn(0x7f83b40cff80 0x7f83b4109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:07.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.530 7f83b2d9d700 1 --2- 172.21.2.101:0/523281281 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83b40d1880 0x7f83b41b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:07.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.530 7f83b1d9b700 1 -- 172.21.2.101:0/523281281 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 566611647 0 0) 0x7f83a40009b0 con 0x7f83b41b0af0 2021-02-10T11:35:07.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.531 7f83b1d9b700 1 -- 172.21.2.101:0/523281281 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83a0002fe0 con 0x7f83b41b0af0 2021-02-10T11:35:07.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.531 7f83b359e700 1 -- 172.21.2.101:0/523281281 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83b40d1880 msgr2=0x7f83b41b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:07.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.531 7f83b359e700 1 --2- 172.21.2.101:0/523281281 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83b40d1880 0x7f83b41b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:07.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.531 7f83b359e700 1 -- 172.21.2.101:0/523281281 >> v1:172.21.2.101:6789/0 conn(0x7f83b41b0af0 legacy=0x7f83b4108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:07.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.531 7f83b359e700 1 -- 172.21.2.101:0/523281281 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b4088260 con 0x7f83b40cff80 2021-02-10T11:35:07.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.531 7f83b359e700 1 --2- 172.21.2.101:0/523281281 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83b40cff80 0x7f83b4109140 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f83ac006e50 tx=0x7f83ac009e70).ready entity=mon.1 client_cookie=b5e0b6a7906f4aab server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:07.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.531 7f83b1d9b700 1 -- 172.21.2.101:0/523281281 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f83ac00b510 con 0x7f83b40cff80 2021-02-10T11:35:07.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.531 7f83b1d9b700 1 -- 172.21.2.101:0/523281281 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ac006790 con 0x7f83b40cff80 2021-02-10T11:35:07.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.531 7f83b1d9b700 1 -- 172.21.2.101:0/523281281 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f83ac00b9d0 con 0x7f83b40cff80 2021-02-10T11:35:07.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.531 7f83b91c1700 1 -- 172.21.2.101:0/523281281 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83b40cff80 msgr2=0x7f83b4109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:07.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.531 7f83b91c1700 1 --2- 172.21.2.101:0/523281281 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83b40cff80 0x7f83b4109140 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f83ac006e50 tx=0x7f83ac009e70).stop 2021-02-10T11:35:07.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.532 7f83b91c1700 1 -- 172.21.2.101:0/523281281 shutdown_connections 2021-02-10T11:35:07.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.532 7f83b91c1700 1 --2- 172.21.2.101:0/523281281 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83b40cff80 0x7f83b4109140 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:07.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.532 7f83b91c1700 1 --2- 172.21.2.101:0/523281281 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83b40d1880 0x7f83b41b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:07.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.532 7f83b91c1700 1 -- 172.21.2.101:0/523281281 >> 172.21.2.101:0/523281281 conn(0x7f83b40ce430 msgr2=0x7f83b40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:07.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.532 7f83b91c1700 1 -- 172.21.2.101:0/523281281 shutdown_connections 2021-02-10T11:35:07.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.532 7f83b91c1700 1 -- 172.21.2.101:0/523281281 wait complete. 2021-02-10T11:35:07.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.547 7f83b91c1700 1 Processor -- start 2021-02-10T11:35:07.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.547 7f83b91c1700 1 -- start start 2021-02-10T11:35:07.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.547 7f83b91c1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83b41a0580 0x7f83b4082c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:07.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.547 7f83b91c1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83b40832a0 0x7f83b40836a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:07.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.547 7f83b91c1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83b40b66d0 0x7f83b40b6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:07.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.547 7f83b91c1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b40ba540 con 0x7f83b41a0580 2021-02-10T11:35:07.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.547 7f83b91c1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b40baac0 con 0x7f83b40832a0 2021-02-10T11:35:07.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.547 7f83b91c1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b40b70f0 con 0x7f83b40b66d0 2021-02-10T11:35:07.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.548 7f83b2d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83b41a0580 0x7f83b4082c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:07.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.548 7f83b2d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83b41a0580 0x7f83b4082c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52918/0 (socket says 172.21.2.101:52918) 2021-02-10T11:35:07.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.548 7f83b2d9d700 1 -- 172.21.2.101:0/582542763 learned_addr learned my addr 172.21.2.101:0/582542763 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:07.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.548 7f83b359e700 1 --2- 172.21.2.101:0/582542763 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83b40b66d0 0x7f83b40b6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:07.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.548 7f83b259c700 1 --2- 172.21.2.101:0/582542763 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83b40832a0 0x7f83b40836a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:07.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.548 7f83b2d9d700 1 -- 172.21.2.101:0/582542763 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83b40b66d0 msgr2=0x7f83b40b6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:07.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.548 7f83b2d9d700 1 --2- 172.21.2.101:0/582542763 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83b40b66d0 0x7f83b40b6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:07.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.548 7f83b2d9d700 1 -- 172.21.2.101:0/582542763 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83b40832a0 msgr2=0x7f83b40836a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:07.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.548 7f83b2d9d700 1 --2- 172.21.2.101:0/582542763 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83b40832a0 0x7f83b40836a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:07.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.548 7f83b2d9d700 1 -- 172.21.2.101:0/582542763 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b40b7310 con 0x7f83b41a0580 2021-02-10T11:35:07.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.548 7f83b2d9d700 1 --2- 172.21.2.101:0/582542763 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83b41a0580 0x7f83b4082c80 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 rx=0x7f839c006f60 tx=0x7f839c00b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:07.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.548 7f83ab7fe700 1 -- 172.21.2.101:0/582542763 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f839c00bd60 con 0x7f83b41a0580 2021-02-10T11:35:07.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.548 7f83ab7fe700 1 -- 172.21.2.101:0/582542763 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839c000c90 con 0x7f83b41a0580 2021-02-10T11:35:07.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.548 7f83b91c1700 1 -- 172.21.2.101:0/582542763 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b40b7310 con 0x7f83b41a0580 2021-02-10T11:35:07.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.548 7f83ab7fe700 1 -- 172.21.2.101:0/582542763 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f839c002220 con 0x7f83b41a0580 2021-02-10T11:35:07.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.548 7f83b91c1700 1 -- 172.21.2.101:0/582542763 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b40b7310 con 0x7f83b41a0580 2021-02-10T11:35:07.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.549 7f83ab7fe700 1 -- 172.21.2.101:0/582542763 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f839c0052e0 con 0x7f83b41a0580 2021-02-10T11:35:07.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.549 7f83ab7fe700 1 --2- 172.21.2.101:0/582542763 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8398018730 0x7f839801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:07.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.549 7f83ab7fe700 1 -- 172.21.2.101:0/582542763 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f839c032640 con 0x7f83b41a0580 2021-02-10T11:35:07.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.549 7f83b259c700 1 --2- 172.21.2.101:0/582542763 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8398018730 0x7f839801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:07.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.549 7f83b259c700 1 --2- 172.21.2.101:0/582542763 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8398018730 0x7f839801ac10 crc :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:07.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.580 7f83b91c1700 1 -- 172.21.2.101:0/582542763 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b4082a50 con 0x7f83b41a0580 2021-02-10T11:35:07.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.582 7f83ab7fe700 1 -- 172.21.2.101:0/582542763 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f839c000c90 con 0x7f83b41a0580 2021-02-10T11:35:07.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.710 7f83b91c1700 1 -- 172.21.2.101:0/582542763 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000063820", "target": ["mgr", ""], "group_name": "group_0000000000497714"}) v1 -- 0x7f83b4082a50 con 0x7f8398018730 2021-02-10T11:35:07.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.735 7f83ab7fe700 1 -- 172.21.2.101:0/582542763 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f83a4000f80 con 0x7f8398018730 2021-02-10T11:35:07.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.743 7f83b91c1700 1 -- 172.21.2.101:0/582542763 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8398018730 msgr2=0x7f839801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:07.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.743 7f83b91c1700 1 --2- 172.21.2.101:0/582542763 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8398018730 0x7f839801ac10 crc :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:07.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.743 7f83b91c1700 1 -- 172.21.2.101:0/582542763 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83b41a0580 msgr2=0x7f83b4082c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:07.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.743 7f83b91c1700 1 --2- 172.21.2.101:0/582542763 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83b41a0580 0x7f83b4082c80 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 rx=0x7f839c006f60 tx=0x7f839c00b970).stop 2021-02-10T11:35:07.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.743 7f83b91c1700 1 -- 172.21.2.101:0/582542763 shutdown_connections 2021-02-10T11:35:07.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.743 7f83b91c1700 1 --2- 172.21.2.101:0/582542763 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8398018730 0x7f839801ac10 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:07.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.743 7f83b91c1700 1 --2- 172.21.2.101:0/582542763 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83b40b66d0 0x7f83b40b6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:07.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.743 7f83b91c1700 1 --2- 172.21.2.101:0/582542763 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83b40832a0 0x7f83b40836a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:07.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.743 7f83b91c1700 1 --2- 172.21.2.101:0/582542763 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83b41a0580 0x7f83b4082c80 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:07.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.743 7f83b91c1700 1 -- 172.21.2.101:0/582542763 >> 172.21.2.101:0/582542763 conn(0x7f83b40c20f0 msgr2=0x7f83b409e410 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:07.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.743 7f83b91c1700 1 -- 172.21.2.101:0/582542763 shutdown_connections 2021-02-10T11:35:07.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:07.743 7f83b91c1700 1 -- 172.21.2.101:0/582542763 wait complete. 2021-02-10T11:35:07.760 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000063820 --group_name group_0000000000497714 2021-02-10T11:35:08.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.165 7f14649db700 1 Processor -- start 2021-02-10T11:35:08.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.165 7f14649db700 1 -- start start 2021-02-10T11:35:08.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.166 7f14649db700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f14600d1880 0x7f14601b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:08.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.166 7f14649db700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14601b0af0 0x7f1460108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:08.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.166 7f14649db700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f14600d1c80 0x7f1460108f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:08.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.166 7f14649db700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14600fe1d0 con 0x7f14600d1880 2021-02-10T11:35:08.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.166 7f14649db700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1460109a70 con 0x7f14600d1c80 2021-02-10T11:35:08.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.166 7f14649db700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1460088070 con 0x7f14601b0af0 2021-02-10T11:35:08.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.166 7f145e61c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f14600d1880 0x7f14601b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:08.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.166 7f145e61c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f14600d1880 0x7f14601b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52928/0 (socket says 172.21.2.101:52928) 2021-02-10T11:35:08.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.166 7f145e61c700 1 -- 172.21.2.101:0/2693776230 learned_addr learned my addr 172.21.2.101:0/2693776230 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:08.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.166 7f145e61c700 1 -- 172.21.2.101:0/2693776230 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14601b0af0 msgr2=0x7f1460108950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:35:08.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.166 7f145e61c700 1 --2- 172.21.2.101:0/2693776230 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14601b0af0 0x7f1460108950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:08.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.166 7f145e61c700 1 -- 172.21.2.101:0/2693776230 >> v2:172.21.2.108:3300/0 conn(0x7f14600d1c80 msgr2=0x7f1460108f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:35:08.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.166 7f145e61c700 1 --2- 172.21.2.101:0/2693776230 >> v2:172.21.2.108:3300/0 conn(0x7f14600d1c80 0x7f1460108f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:08.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.166 7f145e61c700 1 -- 172.21.2.101:0/2693776230 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1460109590 con 0x7f14600d1880 2021-02-10T11:35:08.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.166 7f145e61c700 1 --2- 172.21.2.101:0/2693776230 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14600d1880 0x7f14601b04d0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7f1448006700 tx=0x7f1448009b80).ready entity=mon.0 client_cookie=9492a50afcdaba1f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:08.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.167 7f145d61a700 1 -- 172.21.2.101:0/2693776230 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f144800c0c0 con 0x7f14600d1880 2021-02-10T11:35:08.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.167 7f145d61a700 1 -- 172.21.2.101:0/2693776230 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144800c0c0 con 0x7f14600d1880 2021-02-10T11:35:08.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.167 7f145d61a700 1 -- 172.21.2.101:0/2693776230 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f144800c340 con 0x7f14600d1880 2021-02-10T11:35:08.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.167 7f14649db700 1 -- 172.21.2.101:0/2693776230 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14600d1880 msgr2=0x7f14601b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:08.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.167 7f14649db700 1 --2- 172.21.2.101:0/2693776230 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14600d1880 0x7f14601b04d0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7f1448006700 tx=0x7f1448009b80).stop 2021-02-10T11:35:08.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.167 7f14649db700 1 -- 172.21.2.101:0/2693776230 shutdown_connections 2021-02-10T11:35:08.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.168 7f14649db700 1 --2- 172.21.2.101:0/2693776230 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14601b0af0 0x7f1460108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:08.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.168 7f14649db700 1 --2- 172.21.2.101:0/2693776230 >> v2:172.21.2.108:3300/0 conn(0x7f14600d1c80 0x7f1460108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:08.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.168 7f14649db700 1 --2- 172.21.2.101:0/2693776230 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14600d1880 0x7f14601b04d0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:08.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.168 7f14649db700 1 -- 172.21.2.101:0/2693776230 >> 172.21.2.101:0/2693776230 conn(0x7f14600ce430 msgr2=0x7f14600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:08.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.168 7f14649db700 1 -- 172.21.2.101:0/2693776230 shutdown_connections 2021-02-10T11:35:08.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.168 7f14649db700 1 -- 172.21.2.101:0/2693776230 wait complete. 2021-02-10T11:35:08.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.182 7f14649db700 1 Processor -- start 2021-02-10T11:35:08.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.182 7f14649db700 1 -- start start 2021-02-10T11:35:08.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.183 7f14649db700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14601891e0 0x7f146019c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:08.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.183 7f14649db700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f146018e660 0x7f146018ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:08.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.183 7f14649db700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f146018f080 0x7f1460196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:08.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.183 7f14649db700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1460189700 con 0x7f146018e660 2021-02-10T11:35:08.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.183 7f14649db700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1460196c60 con 0x7f14601891e0 2021-02-10T11:35:08.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.183 7f14649db700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1460196e80 con 0x7f146018f080 2021-02-10T11:35:08.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.183 7f145de1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f146018e660 0x7f146018ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:08.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.183 7f145de1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f146018e660 0x7f146018ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52932/0 (socket says 172.21.2.101:52932) 2021-02-10T11:35:08.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.183 7f145de1b700 1 -- 172.21.2.101:0/865926721 learned_addr learned my addr 172.21.2.101:0/865926721 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:08.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.183 7f145de1b700 1 -- 172.21.2.101:0/865926721 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f146018f080 msgr2=0x7f1460196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:08.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.183 7f145de1b700 1 --2- 172.21.2.101:0/865926721 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f146018f080 0x7f1460196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:08.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.183 7f145de1b700 1 -- 172.21.2.101:0/865926721 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14601891e0 msgr2=0x7f146019c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:08.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.183 7f145de1b700 1 --2- 172.21.2.101:0/865926721 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14601891e0 0x7f146019c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:08.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.183 7f145de1b700 1 -- 172.21.2.101:0/865926721 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14601970a0 con 0x7f146018e660 2021-02-10T11:35:08.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.183 7f145de1b700 1 --2- 172.21.2.101:0/865926721 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f146018e660 0x7f146018ea60 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 rx=0x7f145000b560 tx=0x7f145000b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:08.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.184 7f1456ffd700 1 -- 172.21.2.101:0/865926721 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f145000bcf0 con 0x7f146018e660 2021-02-10T11:35:08.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.184 7f1456ffd700 1 -- 172.21.2.101:0/865926721 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1450006030 con 0x7f146018e660 2021-02-10T11:35:08.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.184 7f1456ffd700 1 -- 172.21.2.101:0/865926721 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f14500054d0 con 0x7f146018e660 2021-02-10T11:35:08.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.184 7f14649db700 1 -- 172.21.2.101:0/865926721 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14601970a0 con 0x7f146018e660 2021-02-10T11:35:08.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.184 7f14649db700 1 -- 172.21.2.101:0/865926721 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1460195890 con 0x7f146018e660 2021-02-10T11:35:08.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.184 7f1456ffd700 1 -- 172.21.2.101:0/865926721 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1450006210 con 0x7f146018e660 2021-02-10T11:35:08.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.184 7f1456ffd700 1 --2- 172.21.2.101:0/865926721 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14440186c0 0x7f144401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:08.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.185 7f1456ffd700 1 -- 172.21.2.101:0/865926721 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f14500324a0 con 0x7f146018e660 2021-02-10T11:35:08.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.185 7f145e61c700 1 --2- 172.21.2.101:0/865926721 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14440186c0 0x7f144401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:08.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.185 7f145e61c700 1 --2- 172.21.2.101:0/865926721 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14440186c0 0x7f144401abc0 crc :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:08.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.216 7f14649db700 1 -- 172.21.2.101:0/865926721 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14600fe1d0 con 0x7f146018e660 2021-02-10T11:35:08.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.218 7f1456ffd700 1 -- 172.21.2.101:0/865926721 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1450025ef0 con 0x7f146018e660 2021-02-10T11:35:08.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.346 7f14649db700 1 -- 172.21.2.101:0/865926721 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000063820", "target": ["mgr", ""], "group_name": "group_0000000000497714"}) v1 -- 0x7f14600fe1d0 con 0x7f14440186c0 2021-02-10T11:35:08.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.350 7f1456ffd700 1 -- 172.21.2.101:0/865926721 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+96 (crc 0 0 0) 0x7f1448000970 con 0x7f14440186c0 2021-02-10T11:35:08.362 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/group_0000000000497714/subvolume_0000000000063820/d66a765b-b620-4859-b962-f6b2ea79bb52 2021-02-10T11:35:08.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.355 7f14649db700 1 -- 172.21.2.101:0/865926721 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14440186c0 msgr2=0x7f144401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:08.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.355 7f14649db700 1 --2- 172.21.2.101:0/865926721 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14440186c0 0x7f144401abc0 crc :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:08.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.355 7f14649db700 1 -- 172.21.2.101:0/865926721 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f146018e660 msgr2=0x7f146018ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:08.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.355 7f14649db700 1 --2- 172.21.2.101:0/865926721 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f146018e660 0x7f146018ea60 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 rx=0x7f145000b560 tx=0x7f145000b900).stop 2021-02-10T11:35:08.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.355 7f14649db700 1 -- 172.21.2.101:0/865926721 shutdown_connections 2021-02-10T11:35:08.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.355 7f14649db700 1 --2- 172.21.2.101:0/865926721 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14440186c0 0x7f144401abc0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:08.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.355 7f14649db700 1 --2- 172.21.2.101:0/865926721 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f146018f080 0x7f1460196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:08.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.355 7f14649db700 1 --2- 172.21.2.101:0/865926721 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f146018e660 0x7f146018ea60 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:08.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.355 7f14649db700 1 --2- 172.21.2.101:0/865926721 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14601891e0 0x7f146019c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:08.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.355 7f14649db700 1 -- 172.21.2.101:0/865926721 >> 172.21.2.101:0/865926721 conn(0x7f146019f940 msgr2=0x7f146019ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:08.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.355 7f14649db700 1 -- 172.21.2.101:0/865926721 shutdown_connections 2021-02-10T11:35:08.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.355 7f14649db700 1 -- 172.21.2.101:0/865926721 wait complete. 2021-02-10T11:35:08.372 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume authorize cephfs subvolume_0000000000063820 alice --group_name group_0000000000497714 --tenant_id tenant_id 2021-02-10T11:35:08.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.787 7f5030ec9700 1 Processor -- start 2021-02-10T11:35:08.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.787 7f5030ec9700 1 -- start start 2021-02-10T11:35:08.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.788 7f5030ec9700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f502c10a040 0x7f502c10a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:08.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.788 7f5030ec9700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f502c10aa60 0x7f502c10ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:08.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.788 7f5030ec9700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f502c1b0d40 con 0x7f502c0e53a0 2021-02-10T11:35:08.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.788 7f5030ec9700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f502c0e5f80 con 0x7f502c10a040 2021-02-10T11:35:08.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.788 7f5030ec9700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f502c109ac0 con 0x7f502c10aa60 2021-02-10T11:35:08.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.788 7f502a312700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f502c10a040 0x7f502c10a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:08.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.788 7f502a312700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f502c10a040 0x7f502c10a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52942/0 (socket says 172.21.2.101:52942) 2021-02-10T11:35:08.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.788 7f502a312700 1 -- 172.21.2.101:0/3573228325 learned_addr learned my addr 172.21.2.101:0/3573228325 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:08.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.788 7f502b314700 1 --2- 172.21.2.101:0/3573228325 >> v2:172.21.2.108:3300/0 conn(0x7f502c10aa60 0x7f502c10ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:08.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.789 7f502a312700 1 -- 172.21.2.101:0/3573228325 >> v2:172.21.2.108:3300/0 conn(0x7f502c10aa60 msgr2=0x7f502c10ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:08.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.789 7f502a312700 1 --2- 172.21.2.101:0/3573228325 >> v2:172.21.2.108:3300/0 conn(0x7f502c10aa60 0x7f502c10ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:08.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.789 7f502a312700 1 -- 172.21.2.101:0/3573228325 >> v1:172.21.2.108:6789/0 conn(0x7f502c0e53a0 legacy=0x7f502c1b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:08.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.789 7f502a312700 1 -- 172.21.2.101:0/3573228325 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c0e57a0 con 0x7f502c10a040 2021-02-10T11:35:08.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.789 7f502a312700 1 --2- 172.21.2.101:0/3573228325 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f502c10a040 0x7f502c10a440 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 rx=0x7f5020006700 tx=0x7f5020009b80).ready entity=mon.0 client_cookie=3922a2e43d5ec175 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:08.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.789 7f5029b11700 1 -- 172.21.2.101:0/3573228325 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f502000c0c0 con 0x7f502c10a040 2021-02-10T11:35:08.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.789 7f5029b11700 1 -- 172.21.2.101:0/3573228325 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502000c340 con 0x7f502c10a040 2021-02-10T11:35:08.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.789 7f5029b11700 1 -- 172.21.2.101:0/3573228325 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f502000c860 con 0x7f502c10a040 2021-02-10T11:35:08.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.789 7f5030ec9700 1 -- 172.21.2.101:0/3573228325 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f502c10a040 msgr2=0x7f502c10a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:08.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.789 7f5030ec9700 1 --2- 172.21.2.101:0/3573228325 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f502c10a040 0x7f502c10a440 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 rx=0x7f5020006700 tx=0x7f5020009b80).stop 2021-02-10T11:35:08.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.790 7f5030ec9700 1 -- 172.21.2.101:0/3573228325 shutdown_connections 2021-02-10T11:35:08.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.790 7f5030ec9700 1 --2- 172.21.2.101:0/3573228325 >> v2:172.21.2.108:3300/0 conn(0x7f502c10aa60 0x7f502c10ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:08.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.790 7f5030ec9700 1 --2- 172.21.2.101:0/3573228325 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f502c10a040 0x7f502c10a440 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:08.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.790 7f5030ec9700 1 -- 172.21.2.101:0/3573228325 >> 172.21.2.101:0/3573228325 conn(0x7f502c0ce4b0 msgr2=0x7f502c0e45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:08.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.790 7f5030ec9700 1 -- 172.21.2.101:0/3573228325 shutdown_connections 2021-02-10T11:35:08.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.790 7f5030ec9700 1 -- 172.21.2.101:0/3573228325 wait complete. 2021-02-10T11:35:08.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.805 7f5030ec9700 1 Processor -- start 2021-02-10T11:35:08.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.805 7f5030ec9700 1 -- start start 2021-02-10T11:35:08.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.805 7f5030ec9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f502c0aaa90 0x7f502c095ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:08.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.805 7f5030ec9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f502c0962c0 0x7f502c0966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:08.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.805 7f5030ec9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f502c096ce0 0x7f502c0908d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:08.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.805 7f5030ec9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c19eb80 con 0x7f502c096ce0 2021-02-10T11:35:08.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.805 7f5030ec9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c090ef0 con 0x7f502c0962c0 2021-02-10T11:35:08.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.805 7f5030ec9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502c091110 con 0x7f502c0aaa90 2021-02-10T11:35:08.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.805 7f502b314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f502c096ce0 0x7f502c0908d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:08.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.805 7f502b314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f502c096ce0 0x7f502c0908d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52950/0 (socket says 172.21.2.101:52950) 2021-02-10T11:35:08.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.805 7f502b314700 1 -- 172.21.2.101:0/3122364072 learned_addr learned my addr 172.21.2.101:0/3122364072 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:08.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.805 7f502a312700 1 --2- 172.21.2.101:0/3122364072 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f502c0962c0 0x7f502c0966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:08.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.805 7f502ab13700 1 --2- 172.21.2.101:0/3122364072 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f502c0aaa90 0x7f502c095ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:08.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.806 7f502b314700 1 -- 172.21.2.101:0/3122364072 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f502c0aaa90 msgr2=0x7f502c095ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:08.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.806 7f502b314700 1 --2- 172.21.2.101:0/3122364072 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f502c0aaa90 0x7f502c095ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:08.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.806 7f502b314700 1 -- 172.21.2.101:0/3122364072 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f502c0962c0 msgr2=0x7f502c0966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:08.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.806 7f502b314700 1 --2- 172.21.2.101:0/3122364072 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f502c0962c0 0x7f502c0966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:08.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.806 7f502b314700 1 -- 172.21.2.101:0/3122364072 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c091330 con 0x7f502c096ce0 2021-02-10T11:35:08.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.806 7f502b314700 1 --2- 172.21.2.101:0/3122364072 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f502c096ce0 0x7f502c0908d0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 rx=0x7f502400b970 tx=0x7f502400bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:08.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.806 7f501b7fe700 1 -- 172.21.2.101:0/3122364072 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5024000f90 con 0x7f502c096ce0 2021-02-10T11:35:08.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.806 7f501b7fe700 1 -- 172.21.2.101:0/3122364072 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5024006aa0 con 0x7f502c096ce0 2021-02-10T11:35:08.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.806 7f501b7fe700 1 -- 172.21.2.101:0/3122364072 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f50240015f0 con 0x7f502c096ce0 2021-02-10T11:35:08.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.806 7f5030ec9700 1 -- 172.21.2.101:0/3122364072 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f502c091330 con 0x7f502c096ce0 2021-02-10T11:35:08.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.806 7f5030ec9700 1 -- 172.21.2.101:0/3122364072 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502c091330 con 0x7f502c096ce0 2021-02-10T11:35:08.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.806 7f501b7fe700 1 -- 172.21.2.101:0/3122364072 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5024000e70 con 0x7f502c096ce0 2021-02-10T11:35:08.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.807 7f501b7fe700 1 --2- 172.21.2.101:0/3122364072 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5010018730 0x7f501001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:08.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.807 7f501b7fe700 1 -- 172.21.2.101:0/3122364072 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f50240329c0 con 0x7f502c096ce0 2021-02-10T11:35:08.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.807 7f502ab13700 1 --2- 172.21.2.101:0/3122364072 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5010018730 0x7f501001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:08.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.807 7f502ab13700 1 --2- 172.21.2.101:0/3122364072 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5010018730 0x7f501001ac10 crc :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:08.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.838 7f5030ec9700 1 -- 172.21.2.101:0/3122364072 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f502c19eb80 con 0x7f502c096ce0 2021-02-10T11:35:08.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.840 7f501b7fe700 1 -- 172.21.2.101:0/3122364072 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f50240066f0 con 0x7f502c096ce0 2021-02-10T11:35:08.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.964 7f5030ec9700 1 -- 172.21.2.101:0/3122364072 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000063820", "target": ["mgr", ""], "tenant_id": "tenant_id", "auth_id": "alice", "group_name": "group_0000000000497714"}) v1 -- 0x7f502c090ef0 con 0x7f5010018730 2021-02-10T11:35:08.977 INFO:tasks.ceph.mgr.y.gibba001.stderr:Creating meta for ID alice with tenant tenant_id 2021-02-10T11:35:09.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.995 7f501b7fe700 1 -- 172.21.2.101:0/3122364072 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7f501c000960 con 0x7f5010018730 2021-02-10T11:35:09.004 INFO:teuthology.orchestra.run.gibba001.stdout:AQDsxCNgecBnOhAAZSFsyDL12TdycLp/UDwgKQ== 2021-02-10T11:35:09.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.997 7f5030ec9700 1 -- 172.21.2.101:0/3122364072 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5010018730 msgr2=0x7f501001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:09.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.997 7f5030ec9700 1 --2- 172.21.2.101:0/3122364072 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5010018730 0x7f501001ac10 crc :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:09.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.997 7f5030ec9700 1 -- 172.21.2.101:0/3122364072 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f502c096ce0 msgr2=0x7f502c0908d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:09.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.997 7f5030ec9700 1 --2- 172.21.2.101:0/3122364072 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f502c096ce0 0x7f502c0908d0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 rx=0x7f502400b970 tx=0x7f502400bd10).stop 2021-02-10T11:35:09.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.997 7f5030ec9700 1 -- 172.21.2.101:0/3122364072 shutdown_connections 2021-02-10T11:35:09.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.997 7f5030ec9700 1 --2- 172.21.2.101:0/3122364072 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f502c096ce0 0x7f502c0908d0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:09.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.997 7f5030ec9700 1 --2- 172.21.2.101:0/3122364072 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f502c0962c0 0x7f502c0966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:09.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.997 7f5030ec9700 1 --2- 172.21.2.101:0/3122364072 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5010018730 0x7f501001ac10 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:09.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.997 7f5030ec9700 1 --2- 172.21.2.101:0/3122364072 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f502c0aaa90 0x7f502c095ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:09.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.997 7f5030ec9700 1 -- 172.21.2.101:0/3122364072 >> 172.21.2.101:0/3122364072 conn(0x7f502c19b000 msgr2=0x7f502c18c310 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:09.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.997 7f5030ec9700 1 -- 172.21.2.101:0/3122364072 shutdown_connections 2021-02-10T11:35:09.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:08.997 7f5030ec9700 1 -- 172.21.2.101:0/3122364072 wait complete. 2021-02-10T11:35:09.014 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:35:09.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.415 7f56ec9ee700 1 Processor -- start 2021-02-10T11:35:09.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.415 7f56ec9ee700 1 -- start start 2021-02-10T11:35:09.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.416 7f56ec9ee700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f56e80d1870 0x7f56e81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:09.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.416 7f56ec9ee700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e80f0b10 con 0x7f56e81b0a10 2021-02-10T11:35:09.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.416 7f56ec9ee700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e80d0310 con 0x7f56e80f0650 2021-02-10T11:35:09.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.416 7f56ec9ee700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e80d0530 con 0x7f56e80d1870 2021-02-10T11:35:09.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.416 7f56e5e1b700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f56e81b0a10 0x7f56e80cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53450/0 (socket says 172.21.2.101:53450) 2021-02-10T11:35:09.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.416 7f56e5e1b700 1 -- 172.21.2.101:0/1090295113 learned_addr learned my addr 172.21.2.101:0/1090295113 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:09.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.416 7f56e561a700 1 -- 172.21.2.101:0/1090295113 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 884673463 0 0) 0x7f56d80009b0 con 0x7f56e81b0a10 2021-02-10T11:35:09.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.416 7f56e561a700 1 -- 172.21.2.101:0/1090295113 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56d4002fe0 con 0x7f56e81b0a10 2021-02-10T11:35:09.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.416 7f56e661c700 1 --2- 172.21.2.101:0/1090295113 >> v2:172.21.2.108:3300/0 conn(0x7f56e80d1870 0x7f56e81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:09.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e561a700 1 -- 172.21.2.101:0/1090295113 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2522254196 0 0) 0x7f56d8001010 con 0x7f56e81b0a10 2021-02-10T11:35:09.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e561a700 1 -- 172.21.2.101:0/1090295113 >> v2:172.21.2.108:3300/0 conn(0x7f56e80d1870 msgr2=0x7f56e81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:09.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e561a700 1 --2- 172.21.2.101:0/1090295113 >> v2:172.21.2.108:3300/0 conn(0x7f56e80d1870 0x7f56e81b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:09.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e561a700 1 -- 172.21.2.101:0/1090295113 >> v1:172.21.2.108:6789/0 conn(0x7f56e80f0650 legacy=0x7f56e80cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:09.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e561a700 1 -- 172.21.2.101:0/1090295113 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e80d1c70 con 0x7f56e81b0a10 2021-02-10T11:35:09.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e561a700 1 -- 172.21.2.101:0/1090295113 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f56d8001590 con 0x7f56e81b0a10 2021-02-10T11:35:09.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e561a700 1 -- 172.21.2.101:0/1090295113 >> v1:172.21.2.101:6789/0 conn(0x7f56e81b0a10 legacy=0x7f56e80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:09.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e561a700 1 --2- 172.21.2.101:0/1090295113 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56d4003aa0 0x7f56d4005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:09.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e561a700 1 --2- 172.21.2.101:0/1090295113 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56d4006770 0x7f56d4008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:09.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e561a700 1 --2- 172.21.2.101:0/1090295113 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56d4009590 0x7f56d400bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:09.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e561a700 1 -- 172.21.2.101:0/1090295113 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d400c300 con 0x7f56d4009590 2021-02-10T11:35:09.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e561a700 1 -- 172.21.2.101:0/1090295113 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d400c520 con 0x7f56d4006770 2021-02-10T11:35:09.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e561a700 1 -- 172.21.2.101:0/1090295113 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d400c740 con 0x7f56d4003aa0 2021-02-10T11:35:09.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e6e1d700 1 --2- 172.21.2.101:0/1090295113 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56d4009590 0x7f56d400bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:09.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e661c700 1 --2- 172.21.2.101:0/1090295113 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56d4003aa0 0x7f56d4005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:09.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e5e1b700 1 --2- 172.21.2.101:0/1090295113 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56d4006770 0x7f56d4008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:09.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e6e1d700 1 -- 172.21.2.101:0/1090295113 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56d4003aa0 msgr2=0x7f56d4005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:09.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e6e1d700 1 --2- 172.21.2.101:0/1090295113 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56d4003aa0 0x7f56d4005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:09.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e6e1d700 1 -- 172.21.2.101:0/1090295113 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56d4006770 msgr2=0x7f56d4008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:09.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e6e1d700 1 --2- 172.21.2.101:0/1090295113 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56d4006770 0x7f56d4008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:09.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e6e1d700 1 -- 172.21.2.101:0/1090295113 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d40509b0 con 0x7f56d4009590 2021-02-10T11:35:09.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e6e1d700 1 -- 172.21.2.101:0/1090295113 reap_dead start 2021-02-10T11:35:09.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e6e1d700 1 --2- 172.21.2.101:0/1090295113 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56d4009590 0x7f56d400bab0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 rx=0x7f56e0006e40 tx=0x7f56e0009db0).ready entity=mon.0 client_cookie=ec3f44542a4480bf server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:09.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e561a700 1 -- 172.21.2.101:0/1090295113 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f56e000b300 con 0x7f56d4009590 2021-02-10T11:35:09.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e561a700 1 -- 172.21.2.101:0/1090295113 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e000b580 con 0x7f56d4009590 2021-02-10T11:35:09.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56e561a700 1 -- 172.21.2.101:0/1090295113 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f56e000baa0 con 0x7f56d4009590 2021-02-10T11:35:09.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56ec9ee700 1 -- 172.21.2.101:0/1090295113 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56d4009590 msgr2=0x7f56d400bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:09.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.417 7f56ec9ee700 1 --2- 172.21.2.101:0/1090295113 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56d4009590 0x7f56d400bab0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 rx=0x7f56e0006e40 tx=0x7f56e0009db0).stop 2021-02-10T11:35:09.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.418 7f56ec9ee700 1 -- 172.21.2.101:0/1090295113 shutdown_connections 2021-02-10T11:35:09.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.418 7f56ec9ee700 1 --2- 172.21.2.101:0/1090295113 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56d4009590 0x7f56d400bab0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:09.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.418 7f56ec9ee700 1 -- 172.21.2.101:0/1090295113 >> 172.21.2.101:0/1090295113 conn(0x7f56e80ce450 msgr2=0x7f56e80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:09.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.418 7f56ec9ee700 1 -- 172.21.2.101:0/1090295113 shutdown_connections 2021-02-10T11:35:09.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.418 7f56ec9ee700 1 -- 172.21.2.101:0/1090295113 wait complete. 2021-02-10T11:35:09.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.433 7f56ec9ee700 1 Processor -- start 2021-02-10T11:35:09.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.433 7f56ec9ee700 1 -- start start 2021-02-10T11:35:09.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.433 7f56ec9ee700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56e81942b0 0x7f56e80b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:09.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.433 7f56ec9ee700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56e8195070 0x7f56e8195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:09.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.433 7f56ec9ee700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56e80bb8e0 0x7f56e80bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:09.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.433 7f56ec9ee700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e81946b0 con 0x7f56e80bb8e0 2021-02-10T11:35:09.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.433 7f56ec9ee700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e80b3b70 con 0x7f56e81942b0 2021-02-10T11:35:09.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.433 7f56ec9ee700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e80b3d90 con 0x7f56e8195070 2021-02-10T11:35:09.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.434 7f56e6e1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56e80bb8e0 0x7f56e80bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:09.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.434 7f56e6e1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56e80bb8e0 0x7f56e80bbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52968/0 (socket says 172.21.2.101:52968) 2021-02-10T11:35:09.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.434 7f56e6e1d700 1 -- 172.21.2.101:0/1218827225 learned_addr learned my addr 172.21.2.101:0/1218827225 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:09.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.434 7f56e5e1b700 1 --2- 172.21.2.101:0/1218827225 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56e8195070 0x7f56e8195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:09.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.434 7f56e661c700 1 --2- 172.21.2.101:0/1218827225 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56e81942b0 0x7f56e80b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:09.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.434 7f56e6e1d700 1 -- 172.21.2.101:0/1218827225 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56e8195070 msgr2=0x7f56e8195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:09.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.434 7f56e6e1d700 1 --2- 172.21.2.101:0/1218827225 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56e8195070 0x7f56e8195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:09.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.434 7f56e6e1d700 1 -- 172.21.2.101:0/1218827225 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56e81942b0 msgr2=0x7f56e80b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:09.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.434 7f56e6e1d700 1 --2- 172.21.2.101:0/1218827225 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56e81942b0 0x7f56e80b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:09.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.434 7f56e6e1d700 1 -- 172.21.2.101:0/1218827225 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e81a49e0 con 0x7f56e80bb8e0 2021-02-10T11:35:09.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.434 7f56e5e1b700 1 --2- 172.21.2.101:0/1218827225 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56e8195070 0x7f56e8195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:35:09.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.434 7f56e661c700 1 --2- 172.21.2.101:0/1218827225 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56e81942b0 0x7f56e80b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:35:09.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.434 7f56e6e1d700 1 --2- 172.21.2.101:0/1218827225 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56e80bb8e0 0x7f56e80bbce0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7f56e000b960 tx=0x7f56e000bd00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:09.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.435 7f56deffd700 1 -- 172.21.2.101:0/1218827225 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f56e0003080 con 0x7f56e80bb8e0 2021-02-10T11:35:09.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.435 7f56deffd700 1 -- 172.21.2.101:0/1218827225 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e0000f90 con 0x7f56e80bb8e0 2021-02-10T11:35:09.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.435 7f56ec9ee700 1 -- 172.21.2.101:0/1218827225 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e81a49e0 con 0x7f56e80bb8e0 2021-02-10T11:35:09.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.435 7f56ec9ee700 1 -- 172.21.2.101:0/1218827225 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e80bc5f0 con 0x7f56e80bb8e0 2021-02-10T11:35:09.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.435 7f56deffd700 1 -- 172.21.2.101:0/1218827225 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f56e0001430 con 0x7f56e80bb8e0 2021-02-10T11:35:09.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.435 7f56deffd700 1 -- 172.21.2.101:0/1218827225 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f56e0001220 con 0x7f56e80bb8e0 2021-02-10T11:35:09.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.435 7f56deffd700 1 --2- 172.21.2.101:0/1218827225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56cc018730 0x7f56cc01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:09.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.435 7f56deffd700 1 -- 172.21.2.101:0/1218827225 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f56e0032b70 con 0x7f56e80bb8e0 2021-02-10T11:35:09.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.436 7f56e661c700 1 --2- 172.21.2.101:0/1218827225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56cc018730 0x7f56cc01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:09.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.436 7f56e661c700 1 --2- 172.21.2.101:0/1218827225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56cc018730 0x7f56cc01ac10 crc :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:09.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.465 7f56ec9ee700 1 -- 172.21.2.101:0/1218827225 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e80bc5f0 con 0x7f56e80bb8e0 2021-02-10T11:35:09.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.468 7f56deffd700 1 -- 172.21.2.101:0/1218827225 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f56e0000f90 con 0x7f56e80bb8e0 2021-02-10T11:35:09.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.602 7f56ec9ee700 1 -- 172.21.2.101:0/1218827225 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f56e80bc5f0 con 0x7f56e80bb8e0 2021-02-10T11:35:09.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.603 7f56deffd700 1 -- 172.21.2.101:0/1218827225 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v42) v1 ==== 80+0+4892 (secure 0 0 0) 0x7f56e0001220 con 0x7f56e80bb8e0 2021-02-10T11:35:09.614 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:09.614 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:35:09.614 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:35:09.614 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:09.614 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:35:09.615 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:35:09.615 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:09.615 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:35:09.615 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:09.615 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:09.615 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:09.615 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:09.615 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:09.615 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:09.615 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:35:09.615 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:35:09.615 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:09.615 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:35:09.616 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:09.616 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:09.616 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:09.616 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:09.616 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:09.616 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:09.616 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:35:09.616 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:35:09.616 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:09.616 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:09.616 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:09.616 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:09.616 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:09.617 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:09.617 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:09.617 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:35:09.617 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:35:09.617 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:09.617 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:09.617 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:09.617 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:09.617 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:09.617 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:09.617 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:09.617 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:35:09.617 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:35:09.618 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:09.618 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:09.618 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:09.618 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:09.618 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:09.618 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:09.618 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:09.618 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:35:09.618 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:35:09.618 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:09.618 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:09.618 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:09.619 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:09.619 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:09.619 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:09.619 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:09.619 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:35:09.619 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:35:09.619 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:09.619 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:09.619 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:09.619 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:09.619 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:09.619 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:09.619 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:09.619 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:35:09.620 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:35:09.620 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:09.620 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:09.620 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:09.620 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:09.620 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:09.620 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:09.620 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:09.620 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:35:09.620 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:35:09.620 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:09.620 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:09.621 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:09.621 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:09.621 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:09.621 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:09.621 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:09.621 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:35:09.621 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:35:09.621 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:09.621 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:09.622 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:09.622 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:09.622 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:09.622 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:09.622 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:09.622 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:35:09.622 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:35:09.622 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:09.622 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:35:09.622 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:35:09.622 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:35:09.622 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:09.622 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:09.623 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:09.623 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:35:09.623 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:35:09.623 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:09.623 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:35:09.623 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:35:09.623 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:35:09.623 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:09.623 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:09.623 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:09.623 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:35:09.623 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:35:09.623 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:09.623 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:35:09.624 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:35:09.624 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:35:09.624 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:09.624 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:09.624 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:09.624 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:35:09.624 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:35:09.624 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:09.624 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:35:09.624 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:35:09.624 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:35:09.624 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:09.624 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:09.625 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:09.625 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:35:09.625 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:35:09.625 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:09.625 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:35:09.625 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:09.625 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:09.625 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:09.625 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:09.625 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:09.625 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:09.625 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.alice", 2021-02-10T11:35:09.625 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQDsxCNgecBnOhAAZSFsyDL12TdycLp/UDwgKQ==", 2021-02-10T11:35:09.625 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:09.626 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow rw path=/volumes/group_0000000000497714/subvolume_0000000000063820/d66a765b-b620-4859-b962-f6b2ea79bb52", 2021-02-10T11:35:09.626 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:35:09.626 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:35:09.626 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:09.626 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:09.626 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:09.626 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:35:09.626 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:35:09.626 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:09.626 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:35:09.626 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:35:09.626 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:09.626 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:09.627 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:09.627 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:09.627 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:35:09.627 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:35:09.627 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:09.627 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:35:09.627 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:35:09.627 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:09.627 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:09.627 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:09.627 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:09.627 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:35:09.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.604 7f56ec9ee700 1 -- 172.21.2.101:0/1218827225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56cc018730 msgr2=0x7f56cc01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:09.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.604 7f56ec9ee700 1 --2- 172.21.2.101:0/1218827225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56cc018730 0x7f56cc01ac10 crc :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:09.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.604 7f56ec9ee700 1 -- 172.21.2.101:0/1218827225 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56e80bb8e0 msgr2=0x7f56e80bbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:09.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.604 7f56ec9ee700 1 --2- 172.21.2.101:0/1218827225 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56e80bb8e0 0x7f56e80bbce0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7f56e000b960 tx=0x7f56e000bd00).stop 2021-02-10T11:35:09.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.605 7f56ec9ee700 1 -- 172.21.2.101:0/1218827225 shutdown_connections 2021-02-10T11:35:09.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.605 7f56ec9ee700 1 --2- 172.21.2.101:0/1218827225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56cc018730 0x7f56cc01ac10 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:09.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.605 7f56ec9ee700 1 --2- 172.21.2.101:0/1218827225 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56e8195070 0x7f56e8195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:09.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.605 7f56ec9ee700 1 --2- 172.21.2.101:0/1218827225 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56e80bb8e0 0x7f56e80bbce0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:09.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.605 7f56ec9ee700 1 --2- 172.21.2.101:0/1218827225 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56e81942b0 0x7f56e80b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:09.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.605 7f56ec9ee700 1 -- 172.21.2.101:0/1218827225 >> 172.21.2.101:0/1218827225 conn(0x7f56e80b3040 msgr2=0x7f56e80c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:09.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.605 7f56ec9ee700 1 -- 172.21.2.101:0/1218827225 shutdown_connections 2021-02-10T11:35:09.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:09.605 7f56ec9ee700 1 -- 172.21.2.101:0/1218827225 wait complete. 2021-02-10T11:35:09.629 DEBUG:teuthology.orchestra.run.gibba001:> set -ex 2021-02-10T11:35:09.629 DEBUG:teuthology.orchestra.run.gibba001:> sudo dd of=/etc/ceph/ceph.client.alice.keyring 2021-02-10T11:35:09.973 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set client.alice debug_client 20 2021-02-10T11:35:10.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.380 7f87fed74700 1 Processor -- start 2021-02-10T11:35:10.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.380 7f87fed74700 1 -- start start 2021-02-10T11:35:10.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.381 7f87fed74700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87f80d1870 0x7f87f81b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:10.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.381 7f87fed74700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f8109a60 con 0x7f87f80cff70 2021-02-10T11:35:10.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.381 7f87fed74700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f8088070 con 0x7f87f81b0ae0 2021-02-10T11:35:10.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.381 7f87fed74700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f80e5ef0 con 0x7f87f80d1870 2021-02-10T11:35:10.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.381 7f87fd391700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f87f80cff70 0x7f87f8109130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53470/0 (socket says 172.21.2.101:53470) 2021-02-10T11:35:10.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.381 7f87fd391700 1 -- 172.21.2.101:0/2071112279 learned_addr learned my addr 172.21.2.101:0/2071112279 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:10.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.381 7f87f77fe700 1 -- 172.21.2.101:0/2071112279 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24794700 0 0) 0x7f87f0001180 con 0x7f87f80cff70 2021-02-10T11:35:10.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87f77fe700 1 -- 172.21.2.101:0/2071112279 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87e4002fe0 con 0x7f87f80cff70 2021-02-10T11:35:10.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87fcb90700 1 --2- 172.21.2.101:0/2071112279 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87f80d1870 0x7f87f81b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:10.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87f77fe700 1 -- 172.21.2.101:0/2071112279 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3071842629 0 0) 0x7f87f0001180 con 0x7f87f80cff70 2021-02-10T11:35:10.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87f77fe700 1 -- 172.21.2.101:0/2071112279 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87f80d1870 msgr2=0x7f87f81b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:10.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87f77fe700 1 --2- 172.21.2.101:0/2071112279 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87f80d1870 0x7f87f81b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:10.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87f77fe700 1 -- 172.21.2.101:0/2071112279 >> v1:172.21.2.108:6789/0 conn(0x7f87f81b0ae0 legacy=0x7f87f8108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:10.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87f77fe700 1 -- 172.21.2.101:0/2071112279 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f80d1c70 con 0x7f87f80cff70 2021-02-10T11:35:10.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87f77fe700 1 -- 172.21.2.101:0/2071112279 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f87f0001680 con 0x7f87f80cff70 2021-02-10T11:35:10.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87f77fe700 1 -- 172.21.2.101:0/2071112279 >> v1:172.21.2.101:6789/0 conn(0x7f87f80cff70 legacy=0x7f87f8109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:10.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87f77fe700 1 --2- 172.21.2.101:0/2071112279 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87e4003aa0 0x7f87e4005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:10.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87f77fe700 1 --2- 172.21.2.101:0/2071112279 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87e4006790 0x7f87e4008c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:10.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87f77fe700 1 --2- 172.21.2.101:0/2071112279 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87e4009500 0x7f87e400ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:10.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87f77fe700 1 -- 172.21.2.101:0/2071112279 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e400c2d0 con 0x7f87e4003aa0 2021-02-10T11:35:10.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87f77fe700 1 -- 172.21.2.101:0/2071112279 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e400c4f0 con 0x7f87e4009500 2021-02-10T11:35:10.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87f77fe700 1 -- 172.21.2.101:0/2071112279 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e4050720 con 0x7f87e4006790 2021-02-10T11:35:10.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87fcb90700 1 --2- 172.21.2.101:0/2071112279 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87e4003aa0 0x7f87e4005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:10.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87f7fff700 1 --2- 172.21.2.101:0/2071112279 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87e4006790 0x7f87e4008c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:10.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87fd391700 1 --2- 172.21.2.101:0/2071112279 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87e4009500 0x7f87e400ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:10.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87fcb90700 1 -- 172.21.2.101:0/2071112279 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87e4006790 msgr2=0x7f87e4008c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:10.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87fcb90700 1 --2- 172.21.2.101:0/2071112279 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87e4006790 0x7f87e4008c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:10.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87fcb90700 1 -- 172.21.2.101:0/2071112279 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87e4009500 msgr2=0x7f87e400ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:10.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87fcb90700 1 --2- 172.21.2.101:0/2071112279 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87e4009500 0x7f87e400ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:10.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87fcb90700 1 -- 172.21.2.101:0/2071112279 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e4050950 con 0x7f87e4003aa0 2021-02-10T11:35:10.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.382 7f87fcb90700 1 --2- 172.21.2.101:0/2071112279 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87e4003aa0 0x7f87e4005ef0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7f87e0009310 tx=0x7f87e0009a90).ready entity=mon.0 client_cookie=ad4554d710774a12 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:10.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.383 7f87f77fe700 1 -- 172.21.2.101:0/2071112279 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f87e000c0c0 con 0x7f87e4003aa0 2021-02-10T11:35:10.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.383 7f87f77fe700 1 -- 172.21.2.101:0/2071112279 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e0006400 con 0x7f87e4003aa0 2021-02-10T11:35:10.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.383 7f87f77fe700 1 -- 172.21.2.101:0/2071112279 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f87e000c340 con 0x7f87e4003aa0 2021-02-10T11:35:10.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.383 7f87fed74700 1 -- 172.21.2.101:0/2071112279 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87e4003aa0 msgr2=0x7f87e4005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:10.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.383 7f87fed74700 1 --2- 172.21.2.101:0/2071112279 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87e4003aa0 0x7f87e4005ef0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7f87e0009310 tx=0x7f87e0009a90).stop 2021-02-10T11:35:10.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.383 7f87fd391700 1 -- 172.21.2.101:0/2071112279 reap_dead start 2021-02-10T11:35:10.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.383 7f87fed74700 1 -- 172.21.2.101:0/2071112279 shutdown_connections 2021-02-10T11:35:10.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.383 7f87fed74700 1 -- 172.21.2.101:0/2071112279 >> 172.21.2.101:0/2071112279 conn(0x7f87f80ce420 msgr2=0x7f87f80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:10.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.383 7f87fed74700 1 -- 172.21.2.101:0/2071112279 shutdown_connections 2021-02-10T11:35:10.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.383 7f87fed74700 1 -- 172.21.2.101:0/2071112279 wait complete. 2021-02-10T11:35:10.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.398 7f87fed74700 1 Processor -- start 2021-02-10T11:35:10.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.398 7f87fed74700 1 -- start start 2021-02-10T11:35:10.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.398 7f87fed74700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87f80926d0 0x7f87f80b6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:10.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.398 7f87fed74700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87f80c3840 0x7f87f8093270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:10.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.398 7f87fed74700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87f80c3c40 0x7f87f8093890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:10.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.398 7f87fed74700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f8092ad0 con 0x7f87f80c3840 2021-02-10T11:35:10.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.398 7f87fed74700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f8092cf0 con 0x7f87f80c3c40 2021-02-10T11:35:10.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.398 7f87fed74700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f80b6f80 con 0x7f87f80926d0 2021-02-10T11:35:10.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.398 7f87f7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87f80c3840 0x7f87f8093270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:10.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.399 7f87f7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87f80c3840 0x7f87f8093270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52984/0 (socket says 172.21.2.101:52984) 2021-02-10T11:35:10.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.399 7f87f7fff700 1 -- 172.21.2.101:0/1211666131 learned_addr learned my addr 172.21.2.101:0/1211666131 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:10.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.399 7f87f7fff700 1 -- 172.21.2.101:0/1211666131 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87f80926d0 msgr2=0x7f87f80b6820 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:35:10.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.399 7f87f7fff700 1 --2- 172.21.2.101:0/1211666131 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87f80926d0 0x7f87f80b6820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:10.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.399 7f87f7fff700 1 -- 172.21.2.101:0/1211666131 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87f80c3c40 msgr2=0x7f87f8093890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:35:10.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.399 7f87f7fff700 1 --2- 172.21.2.101:0/1211666131 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87f80c3c40 0x7f87f8093890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:10.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.399 7f87f7fff700 1 -- 172.21.2.101:0/1211666131 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f8093eb0 con 0x7f87f80c3840 2021-02-10T11:35:10.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.399 7f87f7fff700 1 --2- 172.21.2.101:0/1211666131 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87f80c3840 0x7f87f8093270 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x7f87e8009980 tx=0x7f87e800bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:10.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.399 7f87f57fa700 1 -- 172.21.2.101:0/1211666131 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f87e8006030 con 0x7f87f80c3840 2021-02-10T11:35:10.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.399 7f87f57fa700 1 -- 172.21.2.101:0/1211666131 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e8006270 con 0x7f87f80c3840 2021-02-10T11:35:10.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.399 7f87f57fa700 1 -- 172.21.2.101:0/1211666131 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f87e80066d0 con 0x7f87f80c3840 2021-02-10T11:35:10.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.399 7f87fed74700 1 -- 172.21.2.101:0/1211666131 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f8093eb0 con 0x7f87f80c3840 2021-02-10T11:35:10.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.399 7f87fed74700 1 -- 172.21.2.101:0/1211666131 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f8093eb0 con 0x7f87f80c3840 2021-02-10T11:35:10.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.400 7f87f57fa700 1 -- 172.21.2.101:0/1211666131 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f87e8006450 con 0x7f87f80c3840 2021-02-10T11:35:10.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.400 7f87f57fa700 1 --2- 172.21.2.101:0/1211666131 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87dc018730 0x7f87dc01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:10.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.400 7f87f57fa700 1 -- 172.21.2.101:0/1211666131 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f87e8032ba0 con 0x7f87f80c3840 2021-02-10T11:35:10.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.400 7f87fcb90700 1 --2- 172.21.2.101:0/1211666131 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87dc018730 0x7f87dc01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:10.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.400 7f87fcb90700 1 --2- 172.21.2.101:0/1211666131 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87dc018730 0x7f87dc01ac10 crc :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:10.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.431 7f87fed74700 1 -- 172.21.2.101:0/1211666131 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f8082a70 con 0x7f87f80c3840 2021-02-10T11:35:10.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.433 7f87f57fa700 1 -- 172.21.2.101:0/1211666131 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f87e800be10 con 0x7f87f80c3840 2021-02-10T11:35:10.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.582 7f87fed74700 1 -- 172.21.2.101:0/1211666131 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) v1 -- 0x7f87f8082a70 con 0x7f87f80c3840 2021-02-10T11:35:10.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.585 7f87f57fa700 1 -- 172.21.2.101:0/1211666131 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v15)=0 v15) v1 ==== 120+0+0 (secure 0 0 0) 0x7f87e8006270 con 0x7f87f80c3840 2021-02-10T11:35:10.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.587 7f87fed74700 1 -- 172.21.2.101:0/1211666131 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87dc018730 msgr2=0x7f87dc01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:10.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.587 7f87fed74700 1 --2- 172.21.2.101:0/1211666131 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87dc018730 0x7f87dc01ac10 crc :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:10.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.587 7f87fed74700 1 -- 172.21.2.101:0/1211666131 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87f80c3840 msgr2=0x7f87f8093270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:10.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.587 7f87fed74700 1 --2- 172.21.2.101:0/1211666131 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87f80c3840 0x7f87f8093270 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x7f87e8009980 tx=0x7f87e800bcd0).stop 2021-02-10T11:35:10.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.587 7f87fed74700 1 -- 172.21.2.101:0/1211666131 shutdown_connections 2021-02-10T11:35:10.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.587 7f87fed74700 1 --2- 172.21.2.101:0/1211666131 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87f80c3c40 0x7f87f8093890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:10.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.587 7f87fed74700 1 --2- 172.21.2.101:0/1211666131 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87f80c3840 0x7f87f8093270 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:10.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.587 7f87fed74700 1 --2- 172.21.2.101:0/1211666131 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87dc018730 0x7f87dc01ac10 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:10.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.587 7f87fed74700 1 --2- 172.21.2.101:0/1211666131 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87f80926d0 0x7f87f80b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:10.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.587 7f87fed74700 1 -- 172.21.2.101:0/1211666131 >> 172.21.2.101:0/1211666131 conn(0x7f87f809ffd0 msgr2=0x7f87f8087030 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:10.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.587 7f87fed74700 1 -- 172.21.2.101:0/1211666131 shutdown_connections 2021-02-10T11:35:10.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:10.587 7f87fed74700 1 -- 172.21.2.101:0/1211666131 wait complete. 2021-02-10T11:35:10.604 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set client.alice debug_objecter 20 2021-02-10T11:35:11.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.010 7f5ba54df700 1 Processor -- start 2021-02-10T11:35:11.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.010 7f5ba54df700 1 -- start start 2021-02-10T11:35:11.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.011 7f5ba54df700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5ba010aff0 0x7f5ba00cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:11.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.011 7f5ba54df700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba00fe1c0 con 0x7f5ba00d1c70 2021-02-10T11:35:11.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.011 7f5ba54df700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba01b0cb0 con 0x7f5ba00d1870 2021-02-10T11:35:11.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.011 7f5ba54df700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba0109a60 con 0x7f5ba010aff0 2021-02-10T11:35:11.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.011 7f5b9fac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5ba010aff0 0x7f5ba00cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:11.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.011 7f5b9fac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5ba010aff0 0x7f5ba00cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:52996/0 (socket says 172.21.2.101:52996) 2021-02-10T11:35:11.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.011 7f5b9eabe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f5ba00d1c70 0x7f5ba00cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53486/0 (socket says 172.21.2.101:53486) 2021-02-10T11:35:11.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.011 7f5b9fac0700 1 -- 172.21.2.101:0/619632027 learned_addr learned my addr 172.21.2.101:0/619632027 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:11.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.011 7f5b9fac0700 1 -- 172.21.2.101:0/619632027 >> v1:172.21.2.108:6789/0 conn(0x7f5ba00d1870 legacy=0x7f5ba01b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:35:11.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.011 7f5b9fac0700 1 -- 172.21.2.101:0/619632027 >> v1:172.21.2.101:6789/0 conn(0x7f5ba00d1c70 legacy=0x7f5ba00cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:11.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.011 7f5b9fac0700 1 -- 172.21.2.101:0/619632027 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba010b3f0 con 0x7f5ba010aff0 2021-02-10T11:35:11.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.011 7f5b9fac0700 1 --2- 172.21.2.101:0/619632027 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ba010aff0 0x7f5ba00cf750 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 rx=0x7f5b98006f00 tx=0x7f5b98009f10).ready entity=mon.0 client_cookie=d82f0fbd12abb1df server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:11.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.011 7f5b9e2bd700 1 -- 172.21.2.101:0/619632027 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5b9800b3b0 con 0x7f5ba010aff0 2021-02-10T11:35:11.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.011 7f5b9e2bd700 1 -- 172.21.2.101:0/619632027 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9800b630 con 0x7f5ba010aff0 2021-02-10T11:35:11.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.011 7f5b9e2bd700 1 -- 172.21.2.101:0/619632027 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5b9800bb50 con 0x7f5ba010aff0 2021-02-10T11:35:11.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.011 7f5ba54df700 1 -- 172.21.2.101:0/619632027 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ba010aff0 msgr2=0x7f5ba00cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:11.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.011 7f5ba54df700 1 --2- 172.21.2.101:0/619632027 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ba010aff0 0x7f5ba00cf750 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 rx=0x7f5b98006f00 tx=0x7f5b98009f10).stop 2021-02-10T11:35:11.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.012 7f5ba54df700 1 -- 172.21.2.101:0/619632027 shutdown_connections 2021-02-10T11:35:11.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.012 7f5ba54df700 1 --2- 172.21.2.101:0/619632027 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ba010aff0 0x7f5ba00cf750 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:11.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.012 7f5ba54df700 1 -- 172.21.2.101:0/619632027 >> 172.21.2.101:0/619632027 conn(0x7f5ba00ce420 msgr2=0x7f5ba00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:11.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.012 7f5ba54df700 1 -- 172.21.2.101:0/619632027 shutdown_connections 2021-02-10T11:35:11.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.012 7f5ba54df700 1 -- 172.21.2.101:0/619632027 wait complete. 2021-02-10T11:35:11.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.026 7f5ba54df700 1 Processor -- start 2021-02-10T11:35:11.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.026 7f5ba54df700 1 -- start start 2021-02-10T11:35:11.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.027 7f5ba54df700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ba00a25d0 0x7f5ba00a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:11.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.027 7f5ba54df700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ba00a2ff0 0x7f5ba00a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:11.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.027 7f5ba54df700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ba00b5650 0x7f5ba00b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:11.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.027 7f5ba54df700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba01c3780 con 0x7f5ba00a25d0 2021-02-10T11:35:11.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.027 7f5ba54df700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba00a20c0 con 0x7f5ba00a2ff0 2021-02-10T11:35:11.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.027 7f5ba54df700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba00b60a0 con 0x7f5ba00b5650 2021-02-10T11:35:11.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.027 7f5b9f2bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ba00a25d0 0x7f5ba00a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:11.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.027 7f5b9f2bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ba00a25d0 0x7f5ba00a29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53000/0 (socket says 172.21.2.101:53000) 2021-02-10T11:35:11.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.027 7f5b9f2bf700 1 -- 172.21.2.101:0/4064082490 learned_addr learned my addr 172.21.2.101:0/4064082490 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:11.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.027 7f5b9f2bf700 1 -- 172.21.2.101:0/4064082490 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ba00b5650 msgr2=0x7f5ba00b5a50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:35:11.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.027 7f5b9f2bf700 1 --2- 172.21.2.101:0/4064082490 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ba00b5650 0x7f5ba00b5a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:11.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.027 7f5b9f2bf700 1 -- 172.21.2.101:0/4064082490 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ba00a2ff0 msgr2=0x7f5ba00a33f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:35:11.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.027 7f5b9f2bf700 1 --2- 172.21.2.101:0/4064082490 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ba00a2ff0 0x7f5ba00a33f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:11.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.027 7f5b9f2bf700 1 -- 172.21.2.101:0/4064082490 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba00b62c0 con 0x7f5ba00a25d0 2021-02-10T11:35:11.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.027 7f5b9f2bf700 1 --2- 172.21.2.101:0/4064082490 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ba00a25d0 0x7f5ba00a29d0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 rx=0x7f5b8800b5b0 tx=0x7f5b8800b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:11.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.027 7f5b97fff700 1 -- 172.21.2.101:0/4064082490 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5b8800bd40 con 0x7f5ba00a25d0 2021-02-10T11:35:11.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.027 7f5b97fff700 1 -- 172.21.2.101:0/4064082490 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b88006020 con 0x7f5ba00a25d0 2021-02-10T11:35:11.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.027 7f5b97fff700 1 -- 172.21.2.101:0/4064082490 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5b880054a0 con 0x7f5ba00a25d0 2021-02-10T11:35:11.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.027 7f5ba54df700 1 -- 172.21.2.101:0/4064082490 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba00b62c0 con 0x7f5ba00a25d0 2021-02-10T11:35:11.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.027 7f5ba54df700 1 -- 172.21.2.101:0/4064082490 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba00b6850 con 0x7f5ba00a25d0 2021-02-10T11:35:11.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.028 7f5b97fff700 1 -- 172.21.2.101:0/4064082490 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5b88006200 con 0x7f5ba00a25d0 2021-02-10T11:35:11.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.028 7f5b97fff700 1 --2- 172.21.2.101:0/4064082490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b840186c0 0x7f5b8401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:11.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.028 7f5b97fff700 1 -- 172.21.2.101:0/4064082490 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5b88032650 con 0x7f5ba00a25d0 2021-02-10T11:35:11.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.028 7f5b9eabe700 1 --2- 172.21.2.101:0/4064082490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b840186c0 0x7f5b8401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:11.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.029 7f5b9eabe700 1 --2- 172.21.2.101:0/4064082490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b840186c0 0x7f5b8401abc0 crc :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:11.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.029 7f5ba54df700 1 -- 172.21.2.101:0/4064082490 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba00807d0 con 0x7f5ba00a25d0 2021-02-10T11:35:11.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.031 7f5b97fff700 1 -- 172.21.2.101:0/4064082490 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5b8801f510 con 0x7f5ba00a25d0 2021-02-10T11:35:11.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.181 7f5ba54df700 1 -- 172.21.2.101:0/4064082490 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=debug_objecter}] v 0) v1 -- 0x7f5ba0088070 con 0x7f5ba00a25d0 2021-02-10T11:35:11.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.185 7f5b97fff700 1 -- 172.21.2.101:0/4064082490 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_objecter}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5b88006020 con 0x7f5ba00a25d0 2021-02-10T11:35:11.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.190 7f5ba54df700 1 -- 172.21.2.101:0/4064082490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b840186c0 msgr2=0x7f5b8401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:11.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.190 7f5ba54df700 1 --2- 172.21.2.101:0/4064082490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b840186c0 0x7f5b8401abc0 crc :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:11.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.190 7f5ba54df700 1 -- 172.21.2.101:0/4064082490 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ba00a25d0 msgr2=0x7f5ba00a29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:11.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.190 7f5ba54df700 1 --2- 172.21.2.101:0/4064082490 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ba00a25d0 0x7f5ba00a29d0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 rx=0x7f5b8800b5b0 tx=0x7f5b8800b950).stop 2021-02-10T11:35:11.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.190 7f5ba54df700 1 -- 172.21.2.101:0/4064082490 shutdown_connections 2021-02-10T11:35:11.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.190 7f5ba54df700 1 --2- 172.21.2.101:0/4064082490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b840186c0 0x7f5b8401abc0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:11.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.190 7f5ba54df700 1 --2- 172.21.2.101:0/4064082490 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ba00b5650 0x7f5ba00b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:11.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.190 7f5ba54df700 1 --2- 172.21.2.101:0/4064082490 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ba00a2ff0 0x7f5ba00a33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:11.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.190 7f5ba54df700 1 --2- 172.21.2.101:0/4064082490 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ba00a25d0 0x7f5ba00a29d0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:11.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.190 7f5ba54df700 1 -- 172.21.2.101:0/4064082490 >> 172.21.2.101:0/4064082490 conn(0x7f5ba00c4120 msgr2=0x7f5ba00ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:11.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.191 7f5ba54df700 1 -- 172.21.2.101:0/4064082490 shutdown_connections 2021-02-10T11:35:11.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.191 7f5ba54df700 1 -- 172.21.2.101:0/4064082490 wait complete. 2021-02-10T11:35:11.208 DEBUG:teuthology.orchestra.run.gibba001:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-02-10T11:35:11.211 DEBUG:teuthology.orchestra.run.gibba008:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-02-10T11:35:11.570 INFO:teuthology.orchestra.run.gibba001.stdout:[global] 2021-02-10T11:35:11.571 INFO:teuthology.orchestra.run.gibba001.stdout: chdir = "" 2021-02-10T11:35:11.571 INFO:teuthology.orchestra.run.gibba001.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-02-10T11:35:11.571 INFO:teuthology.orchestra.run.gibba001.stdout: auth supported = cephx 2021-02-10T11:35:11.571 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.571 INFO:teuthology.orchestra.run.gibba001.stdout: filestore xattr use omap = true 2021-02-10T11:35:11.571 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.571 INFO:teuthology.orchestra.run.gibba001.stdout: mon clock drift allowed = 1.000 2021-02-10T11:35:11.571 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.571 INFO:teuthology.orchestra.run.gibba001.stdout: osd crush chooseleaf type = 0 2021-02-10T11:35:11.571 INFO:teuthology.orchestra.run.gibba001.stdout: auth debug = true 2021-02-10T11:35:11.571 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.572 INFO:teuthology.orchestra.run.gibba001.stdout: ms die on old message = true 2021-02-10T11:35:11.572 INFO:teuthology.orchestra.run.gibba001.stdout: ms die on bug = true 2021-02-10T11:35:11.572 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.572 INFO:teuthology.orchestra.run.gibba001.stdout: mon max pg per osd = 10000 # >= luminous 2021-02-10T11:35:11.572 INFO:teuthology.orchestra.run.gibba001.stdout: mon pg warn max object skew = 0 2021-02-10T11:35:11.572 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.572 INFO:teuthology.orchestra.run.gibba001.stdout: osd pool default size = 2 2021-02-10T11:35:11.572 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.572 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd allow primary affinity = true 2021-02-10T11:35:11.572 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd allow pg remap = true 2021-02-10T11:35:11.572 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on legacy crush tunables = false 2021-02-10T11:35:11.572 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on crush straw calc version zero = false 2021-02-10T11:35:11.572 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on no sortbitwise = false 2021-02-10T11:35:11.573 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on osd down out interval zero = false 2021-02-10T11:35:11.573 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on too few osds = false 2021-02-10T11:35:11.573 INFO:teuthology.orchestra.run.gibba001.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-02-10T11:35:11.573 INFO:teuthology.orchestra.run.gibba001.stdout: mon_warn_on_pool_no_redundancy = false 2021-02-10T11:35:11.573 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.573 INFO:teuthology.orchestra.run.gibba001.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-02-10T11:35:11.573 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.573 INFO:teuthology.orchestra.run.gibba001.stdout: osd default data pool replay window = 5 2021-02-10T11:35:11.573 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.573 INFO:teuthology.orchestra.run.gibba001.stdout: mon allow pool delete = true 2021-02-10T11:35:11.573 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.573 INFO:teuthology.orchestra.run.gibba001.stdout: mon cluster log file level = debug 2021-02-10T11:35:11.573 INFO:teuthology.orchestra.run.gibba001.stdout: debug asserts on shutdown = true 2021-02-10T11:35:11.573 INFO:teuthology.orchestra.run.gibba001.stdout: mon health detail to clog = false 2021-02-10T11:35:11.574 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.574 INFO:teuthology.orchestra.run.gibba001.stdout: # we see this fail in qa on *nautilus*; bump up retries 2021-02-10T11:35:11.574 INFO:teuthology.orchestra.run.gibba001.stdout: mon_client_directed_command_retry = 4 2021-02-10T11:35:11.574 INFO:teuthology.orchestra.run.gibba001.stdout: mon host = "172.21.2.101,172.21.2.108,[v2:172.21.2.108:3301,v1:172.21.2.108:6790]" 2021-02-10T11:35:11.574 INFO:teuthology.orchestra.run.gibba001.stdout: fsid = a6663ba9-2653-4542-8b6a-9323b4e7b545 2021-02-10T11:35:11.574 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.574 INFO:teuthology.orchestra.run.gibba001.stdout:[osd] 2021-02-10T11:35:11.574 INFO:teuthology.orchestra.run.gibba001.stdout: osd journal size = 100 2021-02-10T11:35:11.574 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.574 INFO:teuthology.orchestra.run.gibba001.stdout: osd scrub load threshold = 5.0 2021-02-10T11:35:11.574 INFO:teuthology.orchestra.run.gibba001.stdout: osd scrub max interval = 600 2021-02-10T11:35:11.574 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.574 INFO:teuthology.orchestra.run.gibba001.stdout: osd recover clone overlap = true 2021-02-10T11:35:11.575 INFO:teuthology.orchestra.run.gibba001.stdout: osd recovery max chunk = 1048576 2021-02-10T11:35:11.575 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.575 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug shutdown = true 2021-02-10T11:35:11.575 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug op order = true 2021-02-10T11:35:11.575 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug verify stray on activate = true 2021-02-10T11:35:11.575 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.575 INFO:teuthology.orchestra.run.gibba001.stdout: osd open classes on start = true 2021-02-10T11:35:11.575 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug pg log writeout = true 2021-02-10T11:35:11.575 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.575 INFO:teuthology.orchestra.run.gibba001.stdout: osd deep scrub update digest min age = 30 2021-02-10T11:35:11.575 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.575 INFO:teuthology.orchestra.run.gibba001.stdout: osd map max advance = 10 2021-02-10T11:35:11.575 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.575 INFO:teuthology.orchestra.run.gibba001.stdout: journal zero on create = true 2021-02-10T11:35:11.575 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.576 INFO:teuthology.orchestra.run.gibba001.stdout: filestore ondisk finisher threads = 3 2021-02-10T11:35:11.576 INFO:teuthology.orchestra.run.gibba001.stdout: filestore apply finisher threads = 3 2021-02-10T11:35:11.576 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.576 INFO:teuthology.orchestra.run.gibba001.stdout: bdev debug aio = true 2021-02-10T11:35:11.576 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug misdirected ops = true 2021-02-10T11:35:11.576 INFO:teuthology.orchestra.run.gibba001.stdout: bdev async discard = True 2021-02-10T11:35:11.576 INFO:teuthology.orchestra.run.gibba001.stdout: bdev enable discard = True 2021-02-10T11:35:11.576 INFO:teuthology.orchestra.run.gibba001.stdout: bluestore allocator = bitmap 2021-02-10T11:35:11.576 INFO:teuthology.orchestra.run.gibba001.stdout: bluestore block size = 96636764160 2021-02-10T11:35:11.576 INFO:teuthology.orchestra.run.gibba001.stdout: bluestore fsck on mount = True 2021-02-10T11:35:11.576 INFO:teuthology.orchestra.run.gibba001.stdout: debug bluefs = 20 2021-02-10T11:35:11.576 INFO:teuthology.orchestra.run.gibba001.stdout: debug bluestore = 20 2021-02-10T11:35:11.576 INFO:teuthology.orchestra.run.gibba001.stdout: debug filestore = 20 2021-02-10T11:35:11.576 INFO:teuthology.orchestra.run.gibba001.stdout: debug journal = 20 2021-02-10T11:35:11.577 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 20 2021-02-10T11:35:11.577 INFO:teuthology.orchestra.run.gibba001.stdout: debug osd = 25 2021-02-10T11:35:11.577 INFO:teuthology.orchestra.run.gibba001.stdout: debug rocksdb = 10 2021-02-10T11:35:11.577 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd backfillfull_ratio = 0.85 2021-02-10T11:35:11.577 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd full ratio = 0.9 2021-02-10T11:35:11.577 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd nearfull ratio = 0.8 2021-02-10T11:35:11.577 INFO:teuthology.orchestra.run.gibba001.stdout: osd failsafe full ratio = 0.95 2021-02-10T11:35:11.577 INFO:teuthology.orchestra.run.gibba001.stdout: osd objectstore = bluestore 2021-02-10T11:35:11.577 INFO:teuthology.orchestra.run.gibba001.stdout: osd op complaint time = 180 2021-02-10T11:35:11.577 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.577 INFO:teuthology.orchestra.run.gibba001.stdout:[mgr] 2021-02-10T11:35:11.577 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 1 2021-02-10T11:35:11.577 INFO:teuthology.orchestra.run.gibba001.stdout: debug mgr = 20 2021-02-10T11:35:11.577 INFO:teuthology.orchestra.run.gibba001.stdout: debug mon = 20 2021-02-10T11:35:11.578 INFO:teuthology.orchestra.run.gibba001.stdout: debug auth = 20 2021-02-10T11:35:11.578 INFO:teuthology.orchestra.run.gibba001.stdout: mon reweight min pgs per osd = 4 2021-02-10T11:35:11.578 INFO:teuthology.orchestra.run.gibba001.stdout: mon reweight min bytes per osd = 10 2021-02-10T11:35:11.578 INFO:teuthology.orchestra.run.gibba001.stdout: debug client = 10 2021-02-10T11:35:11.578 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.578 INFO:teuthology.orchestra.run.gibba001.stdout:[mon] 2021-02-10T11:35:11.578 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 1 2021-02-10T11:35:11.578 INFO:teuthology.orchestra.run.gibba001.stdout: debug mon = 20 2021-02-10T11:35:11.578 INFO:teuthology.orchestra.run.gibba001.stdout: debug paxos = 20 2021-02-10T11:35:11.578 INFO:teuthology.orchestra.run.gibba001.stdout: debug auth = 20 2021-02-10T11:35:11.578 INFO:teuthology.orchestra.run.gibba001.stdout: mon data avail warn = 5 2021-02-10T11:35:11.578 INFO:teuthology.orchestra.run.gibba001.stdout: mon mgr mkfs grace = 240 2021-02-10T11:35:11.579 INFO:teuthology.orchestra.run.gibba001.stdout: mon reweight min pgs per osd = 4 2021-02-10T11:35:11.579 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd reporter subtree level = osd 2021-02-10T11:35:11.579 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd prime pg temp = true 2021-02-10T11:35:11.579 INFO:teuthology.orchestra.run.gibba001.stdout: mon reweight min bytes per osd = 10 2021-02-10T11:35:11.579 INFO:teuthology.orchestra.run.gibba001.stdout: mon op complaint time = 120 2021-02-10T11:35:11.579 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:11.579 INFO:teuthology.orchestra.run.gibba001.stdout:[client] 2021-02-10T11:35:11.579 INFO:teuthology.orchestra.run.gibba001.stdout: rgw cache enabled = true 2021-02-10T11:35:11.579 INFO:teuthology.orchestra.run.gibba001.stdout: rgw enable ops log = true 2021-02-10T11:35:11.579 INFO:teuthology.orchestra.run.gibba001.stdout: rgw enable usage log = true 2021-02-10T11:35:11.579 INFO:teuthology.orchestra.run.gibba001.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-02-10T11:35:11.579 INFO:teuthology.orchestra.run.gibba001.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-02-10T11:35:11.579 INFO:teuthology.orchestra.run.gibba001.stdout: client mount timeout = 600 2021-02-10T11:35:11.579 INFO:teuthology.orchestra.run.gibba001.stdout: debug client = 20 2021-02-10T11:35:11.580 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 1 2021-02-10T11:35:11.580 INFO:teuthology.orchestra.run.gibba001.stdout: pid file = "" 2021-02-10T11:35:11.580 INFO:teuthology.orchestra.run.gibba001.stdout:[mon.a] 2021-02-10T11:35:11.580 INFO:teuthology.orchestra.run.gibba001.stdout:[mon.b] 2021-02-10T11:35:11.580 INFO:teuthology.orchestra.run.gibba001.stdout:[mon.c] 2021-02-10T11:35:11.580 INFO:teuthology.orchestra.run.gibba001.stdout:[mds.a] 2021-02-10T11:35:11.580 INFO:teuthology.orchestra.run.gibba001.stdout:[mds.b] 2021-02-10T11:35:11.580 INFO:teuthology.orchestra.run.gibba001.stdout:[mds] 2021-02-10T11:35:11.580 INFO:teuthology.orchestra.run.gibba001.stdout: debug mds = 20 2021-02-10T11:35:11.581 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 1 2021-02-10T11:35:11.581 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal frag = True 2021-02-10T11:35:11.581 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal fragment size max = 10000 2021-02-10T11:35:11.581 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal merge size = 5 2021-02-10T11:35:11.581 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal split bits = 3 2021-02-10T11:35:11.581 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal split size = 100 2021-02-10T11:35:11.581 INFO:teuthology.orchestra.run.gibba001.stdout: mds debug frag = True 2021-02-10T11:35:11.581 INFO:teuthology.orchestra.run.gibba001.stdout: mds debug scatterstat = True 2021-02-10T11:35:11.581 INFO:teuthology.orchestra.run.gibba001.stdout: mds op complaint time = 180 2021-02-10T11:35:11.581 INFO:teuthology.orchestra.run.gibba001.stdout: mds verify scatter = True 2021-02-10T11:35:11.581 INFO:teuthology.orchestra.run.gibba001.stdout: osd op complaint time = 180 2021-02-10T11:35:11.581 INFO:teuthology.orchestra.run.gibba001.stdout:[client.alice] 2021-02-10T11:35:11.581 INFO:teuthology.orchestra.run.gibba001.stdout: keyring = /etc/ceph/ceph.client.alice.keyring 2021-02-10T11:35:11.582 INFO:teuthology.orchestra.run.gibba008.stdout:[global] 2021-02-10T11:35:11.583 INFO:teuthology.orchestra.run.gibba008.stdout: chdir = "" 2021-02-10T11:35:11.583 INFO:teuthology.orchestra.run.gibba008.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-02-10T11:35:11.583 INFO:teuthology.orchestra.run.gibba008.stdout: auth supported = cephx 2021-02-10T11:35:11.583 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.583 INFO:teuthology.orchestra.run.gibba008.stdout: filestore xattr use omap = true 2021-02-10T11:35:11.583 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.583 INFO:teuthology.orchestra.run.gibba008.stdout: mon clock drift allowed = 1.000 2021-02-10T11:35:11.583 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.583 INFO:teuthology.orchestra.run.gibba008.stdout: osd crush chooseleaf type = 0 2021-02-10T11:35:11.583 INFO:teuthology.orchestra.run.gibba008.stdout: auth debug = true 2021-02-10T11:35:11.583 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.583 INFO:teuthology.orchestra.run.gibba008.stdout: ms die on old message = true 2021-02-10T11:35:11.583 INFO:teuthology.orchestra.run.gibba008.stdout: ms die on bug = true 2021-02-10T11:35:11.584 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.584 INFO:teuthology.orchestra.run.gibba008.stdout: mon max pg per osd = 10000 # >= luminous 2021-02-10T11:35:11.584 INFO:teuthology.orchestra.run.gibba008.stdout: mon pg warn max object skew = 0 2021-02-10T11:35:11.584 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.584 INFO:teuthology.orchestra.run.gibba008.stdout: osd pool default size = 2 2021-02-10T11:35:11.584 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.584 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd allow primary affinity = true 2021-02-10T11:35:11.584 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd allow pg remap = true 2021-02-10T11:35:11.584 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on legacy crush tunables = false 2021-02-10T11:35:11.584 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on crush straw calc version zero = false 2021-02-10T11:35:11.584 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on no sortbitwise = false 2021-02-10T11:35:11.584 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on osd down out interval zero = false 2021-02-10T11:35:11.584 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on too few osds = false 2021-02-10T11:35:11.584 INFO:teuthology.orchestra.run.gibba008.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-02-10T11:35:11.585 INFO:teuthology.orchestra.run.gibba008.stdout: mon_warn_on_pool_no_redundancy = false 2021-02-10T11:35:11.585 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.585 INFO:teuthology.orchestra.run.gibba008.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-02-10T11:35:11.585 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.585 INFO:teuthology.orchestra.run.gibba008.stdout: osd default data pool replay window = 5 2021-02-10T11:35:11.585 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.585 INFO:teuthology.orchestra.run.gibba008.stdout: mon allow pool delete = true 2021-02-10T11:35:11.585 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.585 INFO:teuthology.orchestra.run.gibba008.stdout: mon cluster log file level = debug 2021-02-10T11:35:11.585 INFO:teuthology.orchestra.run.gibba008.stdout: debug asserts on shutdown = true 2021-02-10T11:35:11.585 INFO:teuthology.orchestra.run.gibba008.stdout: mon health detail to clog = false 2021-02-10T11:35:11.585 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.585 INFO:teuthology.orchestra.run.gibba008.stdout: # we see this fail in qa on *nautilus*; bump up retries 2021-02-10T11:35:11.585 INFO:teuthology.orchestra.run.gibba008.stdout: mon_client_directed_command_retry = 4 2021-02-10T11:35:11.586 INFO:teuthology.orchestra.run.gibba008.stdout: mon host = "172.21.2.101,172.21.2.108,[v2:172.21.2.108:3301,v1:172.21.2.108:6790]" 2021-02-10T11:35:11.586 INFO:teuthology.orchestra.run.gibba008.stdout: fsid = a6663ba9-2653-4542-8b6a-9323b4e7b545 2021-02-10T11:35:11.586 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.586 INFO:teuthology.orchestra.run.gibba008.stdout:[osd] 2021-02-10T11:35:11.586 INFO:teuthology.orchestra.run.gibba008.stdout: osd journal size = 100 2021-02-10T11:35:11.586 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.586 INFO:teuthology.orchestra.run.gibba008.stdout: osd scrub load threshold = 5.0 2021-02-10T11:35:11.586 INFO:teuthology.orchestra.run.gibba008.stdout: osd scrub max interval = 600 2021-02-10T11:35:11.586 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.586 INFO:teuthology.orchestra.run.gibba008.stdout: osd recover clone overlap = true 2021-02-10T11:35:11.586 INFO:teuthology.orchestra.run.gibba008.stdout: osd recovery max chunk = 1048576 2021-02-10T11:35:11.586 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.586 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug shutdown = true 2021-02-10T11:35:11.587 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug op order = true 2021-02-10T11:35:11.587 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug verify stray on activate = true 2021-02-10T11:35:11.587 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.587 INFO:teuthology.orchestra.run.gibba008.stdout: osd open classes on start = true 2021-02-10T11:35:11.587 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug pg log writeout = true 2021-02-10T11:35:11.587 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.587 INFO:teuthology.orchestra.run.gibba008.stdout: osd deep scrub update digest min age = 30 2021-02-10T11:35:11.587 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.587 INFO:teuthology.orchestra.run.gibba008.stdout: osd map max advance = 10 2021-02-10T11:35:11.587 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.587 INFO:teuthology.orchestra.run.gibba008.stdout: journal zero on create = true 2021-02-10T11:35:11.587 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.587 INFO:teuthology.orchestra.run.gibba008.stdout: filestore ondisk finisher threads = 3 2021-02-10T11:35:11.587 INFO:teuthology.orchestra.run.gibba008.stdout: filestore apply finisher threads = 3 2021-02-10T11:35:11.588 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.588 INFO:teuthology.orchestra.run.gibba008.stdout: bdev debug aio = true 2021-02-10T11:35:11.588 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug misdirected ops = true 2021-02-10T11:35:11.588 INFO:teuthology.orchestra.run.gibba008.stdout: bdev async discard = True 2021-02-10T11:35:11.588 INFO:teuthology.orchestra.run.gibba008.stdout: bdev enable discard = True 2021-02-10T11:35:11.588 INFO:teuthology.orchestra.run.gibba008.stdout: bluestore allocator = bitmap 2021-02-10T11:35:11.588 INFO:teuthology.orchestra.run.gibba008.stdout: bluestore block size = 96636764160 2021-02-10T11:35:11.588 INFO:teuthology.orchestra.run.gibba008.stdout: bluestore fsck on mount = True 2021-02-10T11:35:11.588 INFO:teuthology.orchestra.run.gibba008.stdout: debug bluefs = 20 2021-02-10T11:35:11.588 INFO:teuthology.orchestra.run.gibba008.stdout: debug bluestore = 20 2021-02-10T11:35:11.588 INFO:teuthology.orchestra.run.gibba008.stdout: debug filestore = 20 2021-02-10T11:35:11.588 INFO:teuthology.orchestra.run.gibba008.stdout: debug journal = 20 2021-02-10T11:35:11.588 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 20 2021-02-10T11:35:11.588 INFO:teuthology.orchestra.run.gibba008.stdout: debug osd = 25 2021-02-10T11:35:11.589 INFO:teuthology.orchestra.run.gibba008.stdout: debug rocksdb = 10 2021-02-10T11:35:11.589 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd backfillfull_ratio = 0.85 2021-02-10T11:35:11.589 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd full ratio = 0.9 2021-02-10T11:35:11.589 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd nearfull ratio = 0.8 2021-02-10T11:35:11.589 INFO:teuthology.orchestra.run.gibba008.stdout: osd failsafe full ratio = 0.95 2021-02-10T11:35:11.589 INFO:teuthology.orchestra.run.gibba008.stdout: osd objectstore = bluestore 2021-02-10T11:35:11.589 INFO:teuthology.orchestra.run.gibba008.stdout: osd op complaint time = 180 2021-02-10T11:35:11.589 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.589 INFO:teuthology.orchestra.run.gibba008.stdout:[mgr] 2021-02-10T11:35:11.589 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 1 2021-02-10T11:35:11.589 INFO:teuthology.orchestra.run.gibba008.stdout: debug mgr = 20 2021-02-10T11:35:11.589 INFO:teuthology.orchestra.run.gibba008.stdout: debug mon = 20 2021-02-10T11:35:11.589 INFO:teuthology.orchestra.run.gibba008.stdout: debug auth = 20 2021-02-10T11:35:11.589 INFO:teuthology.orchestra.run.gibba008.stdout: mon reweight min pgs per osd = 4 2021-02-10T11:35:11.590 INFO:teuthology.orchestra.run.gibba008.stdout: mon reweight min bytes per osd = 10 2021-02-10T11:35:11.590 INFO:teuthology.orchestra.run.gibba008.stdout: debug client = 10 2021-02-10T11:35:11.590 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.590 INFO:teuthology.orchestra.run.gibba008.stdout:[mon] 2021-02-10T11:35:11.590 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 1 2021-02-10T11:35:11.590 INFO:teuthology.orchestra.run.gibba008.stdout: debug mon = 20 2021-02-10T11:35:11.590 INFO:teuthology.orchestra.run.gibba008.stdout: debug paxos = 20 2021-02-10T11:35:11.590 INFO:teuthology.orchestra.run.gibba008.stdout: debug auth = 20 2021-02-10T11:35:11.590 INFO:teuthology.orchestra.run.gibba008.stdout: mon data avail warn = 5 2021-02-10T11:35:11.590 INFO:teuthology.orchestra.run.gibba008.stdout: mon mgr mkfs grace = 240 2021-02-10T11:35:11.590 INFO:teuthology.orchestra.run.gibba008.stdout: mon reweight min pgs per osd = 4 2021-02-10T11:35:11.590 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd reporter subtree level = osd 2021-02-10T11:35:11.590 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd prime pg temp = true 2021-02-10T11:35:11.590 INFO:teuthology.orchestra.run.gibba008.stdout: mon reweight min bytes per osd = 10 2021-02-10T11:35:11.591 INFO:teuthology.orchestra.run.gibba008.stdout: mon op complaint time = 120 2021-02-10T11:35:11.591 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:11.591 INFO:teuthology.orchestra.run.gibba008.stdout:[client] 2021-02-10T11:35:11.591 INFO:teuthology.orchestra.run.gibba008.stdout: rgw cache enabled = true 2021-02-10T11:35:11.591 INFO:teuthology.orchestra.run.gibba008.stdout: rgw enable ops log = true 2021-02-10T11:35:11.591 INFO:teuthology.orchestra.run.gibba008.stdout: rgw enable usage log = true 2021-02-10T11:35:11.591 INFO:teuthology.orchestra.run.gibba008.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-02-10T11:35:11.591 INFO:teuthology.orchestra.run.gibba008.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-02-10T11:35:11.591 INFO:teuthology.orchestra.run.gibba008.stdout: client mount timeout = 600 2021-02-10T11:35:11.591 INFO:teuthology.orchestra.run.gibba008.stdout: debug client = 20 2021-02-10T11:35:11.591 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 1 2021-02-10T11:35:11.591 INFO:teuthology.orchestra.run.gibba008.stdout: pid file = "" 2021-02-10T11:35:11.591 INFO:teuthology.orchestra.run.gibba008.stdout:[mon.a] 2021-02-10T11:35:11.591 INFO:teuthology.orchestra.run.gibba008.stdout:[mon.b] 2021-02-10T11:35:11.592 INFO:teuthology.orchestra.run.gibba008.stdout:[mon.c] 2021-02-10T11:35:11.592 INFO:teuthology.orchestra.run.gibba008.stdout:[mds.a] 2021-02-10T11:35:11.592 INFO:teuthology.orchestra.run.gibba008.stdout:[mds.b] 2021-02-10T11:35:11.592 INFO:teuthology.orchestra.run.gibba008.stdout:[mds] 2021-02-10T11:35:11.592 INFO:teuthology.orchestra.run.gibba008.stdout: debug mds = 20 2021-02-10T11:35:11.592 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 1 2021-02-10T11:35:11.592 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal frag = True 2021-02-10T11:35:11.592 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal fragment size max = 10000 2021-02-10T11:35:11.592 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal merge size = 5 2021-02-10T11:35:11.592 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal split bits = 3 2021-02-10T11:35:11.592 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal split size = 100 2021-02-10T11:35:11.592 INFO:teuthology.orchestra.run.gibba008.stdout: mds debug frag = True 2021-02-10T11:35:11.592 INFO:teuthology.orchestra.run.gibba008.stdout: mds debug scatterstat = True 2021-02-10T11:35:11.592 INFO:teuthology.orchestra.run.gibba008.stdout: mds op complaint time = 180 2021-02-10T11:35:11.593 INFO:teuthology.orchestra.run.gibba008.stdout: mds verify scatter = True 2021-02-10T11:35:11.593 INFO:teuthology.orchestra.run.gibba008.stdout: osd op complaint time = 180 2021-02-10T11:35:11.593 INFO:teuthology.orchestra.run.gibba008.stdout:[client.alice] 2021-02-10T11:35:11.593 INFO:teuthology.orchestra.run.gibba008.stdout: keyring = /etc/ceph/ceph.client.alice.keyring 2021-02-10T11:35:11.594 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:35:12.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.995 7f8170352700 1 Processor -- start 2021-02-10T11:35:12.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.995 7f8170352700 1 -- start start 2021-02-10T11:35:12.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.996 7f8170352700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81680d18d0 0x7f8168083ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:12.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.996 7f8170352700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8168082d50 con 0x7f8168082920 2021-02-10T11:35:12.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.996 7f8170352700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81680e57e0 con 0x7f81680e53e0 2021-02-10T11:35:12.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.996 7f8170352700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81681b0d40 con 0x7f81680d18d0 2021-02-10T11:35:12.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.996 7f816d96d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8168082920 0x7f81680832d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53500/0 (socket says 172.21.2.101:53500) 2021-02-10T11:35:12.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.996 7f816d96d700 1 -- 172.21.2.101:0/511358057 learned_addr learned my addr 172.21.2.101:0/511358057 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:12.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.996 7f816e96f700 1 --2- 172.21.2.101:0/511358057 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81680d18d0 0x7f8168083ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:12.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.996 7f816d16c700 1 -- 172.21.2.101:0/511358057 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 257701343 0 0) 0x7f81640010b0 con 0x7f8168082920 2021-02-10T11:35:12.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.996 7f816d16c700 1 -- 172.21.2.101:0/511358057 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8154002fe0 con 0x7f8168082920 2021-02-10T11:35:12.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.997 7f816d16c700 1 -- 172.21.2.101:0/511358057 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2728943815 0 0) 0x7f81640010b0 con 0x7f8168082920 2021-02-10T11:35:12.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.997 7f816d16c700 1 -- 172.21.2.101:0/511358057 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81680d18d0 msgr2=0x7f8168083ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:12.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.997 7f816d16c700 1 --2- 172.21.2.101:0/511358057 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81680d18d0 0x7f8168083ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:12.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.997 7f816d16c700 1 -- 172.21.2.101:0/511358057 >> v1:172.21.2.108:6789/0 conn(0x7f81680e53e0 legacy=0x7f81681b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:12.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.997 7f816d16c700 1 -- 172.21.2.101:0/511358057 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81680d1cd0 con 0x7f8168082920 2021-02-10T11:35:12.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.997 7f816d16c700 1 -- 172.21.2.101:0/511358057 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f81640015b0 con 0x7f8168082920 2021-02-10T11:35:12.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.997 7f816d16c700 1 -- 172.21.2.101:0/511358057 >> v1:172.21.2.101:6789/0 conn(0x7f8168082920 legacy=0x7f81680832d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:12.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.997 7f816d16c700 1 --2- 172.21.2.101:0/511358057 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8154003a60 0x7f8154005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:12.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.997 7f816d16c700 1 --2- 172.21.2.101:0/511358057 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8154006780 0x7f8154008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:12.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.997 7f816d16c700 1 --2- 172.21.2.101:0/511358057 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8154009510 0x7f815400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:12.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.997 7f816d16c700 1 -- 172.21.2.101:0/511358057 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815400c2c0 con 0x7f8154009510 2021-02-10T11:35:12.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.997 7f816d16c700 1 -- 172.21.2.101:0/511358057 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815400c4e0 con 0x7f8154003a60 2021-02-10T11:35:12.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.997 7f816d16c700 1 -- 172.21.2.101:0/511358057 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815400c700 con 0x7f8154006780 2021-02-10T11:35:12.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.997 7f816e96f700 1 --2- 172.21.2.101:0/511358057 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8154009510 0x7f815400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:12.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.997 7f816d96d700 1 --2- 172.21.2.101:0/511358057 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8154006780 0x7f8154008c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:12.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.997 7f816e16e700 1 --2- 172.21.2.101:0/511358057 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8154003a60 0x7f8154005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:12.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.997 7f816e96f700 1 -- 172.21.2.101:0/511358057 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8154006780 msgr2=0x7f8154008c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:12.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.997 7f816e96f700 1 --2- 172.21.2.101:0/511358057 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8154006780 0x7f8154008c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:12.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.997 7f816e96f700 1 -- 172.21.2.101:0/511358057 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8154003a60 msgr2=0x7f8154005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:12.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.997 7f816e96f700 1 --2- 172.21.2.101:0/511358057 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8154003a60 0x7f8154005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:12.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.997 7f816e96f700 1 -- 172.21.2.101:0/511358057 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8154050970 con 0x7f8154009510 2021-02-10T11:35:12.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.997 7f816e96f700 1 --2- 172.21.2.101:0/511358057 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8154009510 0x7f815400ba10 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7f8160009b10 tx=0x7f8160009eb0).ready entity=mon.0 client_cookie=1ab4bd19b5035490 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:12.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.998 7f816d16c700 1 -- 172.21.2.101:0/511358057 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f816000b510 con 0x7f8154009510 2021-02-10T11:35:12.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.998 7f816d16c700 1 -- 172.21.2.101:0/511358057 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8160003030 con 0x7f8154009510 2021-02-10T11:35:12.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.998 7f816d16c700 1 -- 172.21.2.101:0/511358057 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f816000b790 con 0x7f8154009510 2021-02-10T11:35:12.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.998 7f8170352700 1 -- 172.21.2.101:0/511358057 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8154009510 msgr2=0x7f815400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:12.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.998 7f8170352700 1 --2- 172.21.2.101:0/511358057 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8154009510 0x7f815400ba10 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7f8160009b10 tx=0x7f8160009eb0).stop 2021-02-10T11:35:12.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.998 7f816e96f700 1 -- 172.21.2.101:0/511358057 reap_dead start 2021-02-10T11:35:12.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.998 7f8170352700 1 -- 172.21.2.101:0/511358057 shutdown_connections 2021-02-10T11:35:12.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.998 7f8170352700 1 -- 172.21.2.101:0/511358057 >> 172.21.2.101:0/511358057 conn(0x7f81680ce500 msgr2=0x7f81680e4610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:12.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.998 7f8170352700 1 -- 172.21.2.101:0/511358057 shutdown_connections 2021-02-10T11:35:12.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:11.998 7f8170352700 1 -- 172.21.2.101:0/511358057 wait complete. 2021-02-10T11:35:12.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.013 7f8170352700 1 Processor -- start 2021-02-10T11:35:12.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.013 7f8170352700 1 -- start start 2021-02-10T11:35:12.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.014 7f8170352700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8168191bd0 0x7f81681a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:12.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.014 7f8170352700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81681a3e10 0x7f81680b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:12.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.014 7f8170352700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81680b78f0 0x7f81680b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:12.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.014 7f8170352700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8168192170 con 0x7f81681a3e10 2021-02-10T11:35:12.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.014 7f8170352700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81680b8310 con 0x7f81680b78f0 2021-02-10T11:35:12.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.014 7f8170352700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81680b8530 con 0x7f8168191bd0 2021-02-10T11:35:12.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.014 7f816d96d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81681a3e10 0x7f81680b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:12.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.014 7f816d96d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81681a3e10 0x7f81680b72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53020/0 (socket says 172.21.2.101:53020) 2021-02-10T11:35:12.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.014 7f816d96d700 1 -- 172.21.2.101:0/1799670014 learned_addr learned my addr 172.21.2.101:0/1799670014 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:12.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.014 7f816e96f700 1 --2- 172.21.2.101:0/1799670014 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81680b78f0 0x7f81680b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:12.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.014 7f816e16e700 1 --2- 172.21.2.101:0/1799670014 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8168191bd0 0x7f81681a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:12.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.014 7f816d96d700 1 -- 172.21.2.101:0/1799670014 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8168191bd0 msgr2=0x7f81681a37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:12.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.014 7f816d96d700 1 --2- 172.21.2.101:0/1799670014 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8168191bd0 0x7f81681a37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:12.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.014 7f816d96d700 1 -- 172.21.2.101:0/1799670014 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81680b78f0 msgr2=0x7f81680b7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:12.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.014 7f816d96d700 1 --2- 172.21.2.101:0/1799670014 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81680b78f0 0x7f81680b7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:12.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.014 7f816d96d700 1 -- 172.21.2.101:0/1799670014 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81681944a0 con 0x7f81681a3e10 2021-02-10T11:35:12.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.014 7f816e96f700 1 --2- 172.21.2.101:0/1799670014 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81680b78f0 0x7f81680b7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:35:12.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.014 7f816e16e700 1 --2- 172.21.2.101:0/1799670014 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8168191bd0 0x7f81681a37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:35:12.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.014 7f816d96d700 1 --2- 172.21.2.101:0/1799670014 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81681a3e10 0x7f81680b72d0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 rx=0x7f8164001ad0 tx=0x7f816400a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:12.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.015 7f815affd700 1 -- 172.21.2.101:0/1799670014 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81640009b0 con 0x7f81681a3e10 2021-02-10T11:35:12.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.015 7f815affd700 1 -- 172.21.2.101:0/1799670014 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816400acc0 con 0x7f81681a3e10 2021-02-10T11:35:12.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.015 7f815affd700 1 -- 172.21.2.101:0/1799670014 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81640045d0 con 0x7f81681a3e10 2021-02-10T11:35:12.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.015 7f8170352700 1 -- 172.21.2.101:0/1799670014 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81681944a0 con 0x7f81681a3e10 2021-02-10T11:35:12.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.015 7f8170352700 1 -- 172.21.2.101:0/1799670014 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8168194a30 con 0x7f81681a3e10 2021-02-10T11:35:12.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.015 7f815affd700 1 -- 172.21.2.101:0/1799670014 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8164004140 con 0x7f81681a3e10 2021-02-10T11:35:12.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.015 7f815affd700 1 --2- 172.21.2.101:0/1799670014 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81500186c0 0x7f815001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:12.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.016 7f815affd700 1 -- 172.21.2.101:0/1799670014 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8164032510 con 0x7f81681a3e10 2021-02-10T11:35:12.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.016 7f816e16e700 1 --2- 172.21.2.101:0/1799670014 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81500186c0 0x7f815001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:12.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.016 7f816e16e700 1 --2- 172.21.2.101:0/1799670014 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81500186c0 0x7f815001abc0 crc :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:12.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.046 7f8170352700 1 -- 172.21.2.101:0/1799670014 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81680ed5d0 con 0x7f81681a3e10 2021-02-10T11:35:12.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.048 7f815affd700 1 -- 172.21.2.101:0/1799670014 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8164022350 con 0x7f81681a3e10 2021-02-10T11:35:12.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.174 7f8170352700 1 -- 172.21.2.101:0/1799670014 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8168082d50 con 0x7f81681a3e10 2021-02-10T11:35:12.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.175 7f815affd700 1 -- 172.21.2.101:0/1799670014 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+13996 (secure 0 0 0) 0x7f8164022350 con 0x7f81681a3e10 2021-02-10T11:35:12.184 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:12.184 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":65,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:34:42.090329","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:34:38.607957","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2021-02-10 11:34:40.084211","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:35:12.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.176 7f8170352700 1 -- 172.21.2.101:0/1799670014 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81500186c0 msgr2=0x7f815001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:12.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.176 7f8170352700 1 --2- 172.21.2.101:0/1799670014 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81500186c0 0x7f815001abc0 crc :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:12.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.176 7f8170352700 1 -- 172.21.2.101:0/1799670014 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81681a3e10 msgr2=0x7f81680b72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:12.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.176 7f8170352700 1 --2- 172.21.2.101:0/1799670014 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81681a3e10 0x7f81680b72d0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 rx=0x7f8164001ad0 tx=0x7f816400a8d0).stop 2021-02-10T11:35:12.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.177 7f8170352700 1 -- 172.21.2.101:0/1799670014 shutdown_connections 2021-02-10T11:35:12.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.177 7f8170352700 1 --2- 172.21.2.101:0/1799670014 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81680b78f0 0x7f81680b7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:12.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.177 7f8170352700 1 --2- 172.21.2.101:0/1799670014 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81681a3e10 0x7f81680b72d0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:12.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.177 7f8170352700 1 --2- 172.21.2.101:0/1799670014 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81500186c0 0x7f815001abc0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:12.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.177 7f8170352700 1 --2- 172.21.2.101:0/1799670014 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8168191bd0 0x7f81681a37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:12.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.177 7f8170352700 1 -- 172.21.2.101:0/1799670014 >> 172.21.2.101:0/1799670014 conn(0x7f81680ad180 msgr2=0x7f81680b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:12.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.177 7f8170352700 1 -- 172.21.2.101:0/1799670014 shutdown_connections 2021-02-10T11:35:12.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.177 7f8170352700 1 -- 172.21.2.101:0/1799670014 wait complete. 2021-02-10T11:35:12.194 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:34:38.607957', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:34:40.084211', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:35:12.194 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:35:12.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.601 7f8c23f73700 1 Processor -- start 2021-02-10T11:35:12.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.601 7f8c23f73700 1 -- start start 2021-02-10T11:35:12.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.601 7f8c23f73700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8c1c0e5170 0x7f8c1c1b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:12.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.602 7f8c23f73700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8c1c0e5570 0x7f8c1c07a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:12.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.602 7f8c23f73700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1c078b90 con 0x7f8c1c1b3f00 2021-02-10T11:35:12.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.602 7f8c23f73700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c1c0d1900 con 0x7f8c1c0e5170 2021-02-10T11:35:12.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.602 7f8c23f73700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c1c0d1af0 con 0x7f8c1c0e5570 2021-02-10T11:35:12.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.602 7f8c21d8f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8c1c0e5170 0x7f8c1c1b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:12.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.602 7f8c2158e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8c1c1b3f00 0x7f8c1c077510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53520/0 (socket says 172.21.2.101:53520) 2021-02-10T11:35:12.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.602 7f8c2158e700 1 -- 172.21.2.101:0/593512537 learned_addr learned my addr 172.21.2.101:0/593512537 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:12.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.602 7f8c22590700 1 --2- 172.21.2.101:0/593512537 >> v2:172.21.2.108:3300/0 conn(0x7f8c1c0e5570 0x7f8c1c07a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:12.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.602 7f8c21d8f700 1 -- 172.21.2.101:0/593512537 >> v2:172.21.2.108:3300/0 conn(0x7f8c1c0e5570 msgr2=0x7f8c1c07a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:12.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.602 7f8c21d8f700 1 --2- 172.21.2.101:0/593512537 >> v2:172.21.2.108:3300/0 conn(0x7f8c1c0e5570 0x7f8c1c07a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:12.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.602 7f8c21d8f700 1 -- 172.21.2.101:0/593512537 >> v1:172.21.2.101:6789/0 conn(0x7f8c1c1b3f00 legacy=0x7f8c1c077510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:12.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.602 7f8c21d8f700 1 -- 172.21.2.101:0/593512537 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c0d1d10 con 0x7f8c1c0e5170 2021-02-10T11:35:12.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.602 7f8c21d8f700 1 --2- 172.21.2.101:0/593512537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c1c0e5170 0x7f8c1c1b38e0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 rx=0x7f8c10006650 tx=0x7f8c10009a80).ready entity=mon.0 client_cookie=bc610d8d1e0021d3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:12.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.602 7f8c20d8d700 1 -- 172.21.2.101:0/593512537 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c100009b0 con 0x7f8c1c0e5170 2021-02-10T11:35:12.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.602 7f8c20d8d700 1 -- 172.21.2.101:0/593512537 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1000c0c0 con 0x7f8c1c0e5170 2021-02-10T11:35:12.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.602 7f8c20d8d700 1 -- 172.21.2.101:0/593512537 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c1000c520 con 0x7f8c1c0e5170 2021-02-10T11:35:12.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.602 7f8c23f73700 1 -- 172.21.2.101:0/593512537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c1c0e5170 msgr2=0x7f8c1c1b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:12.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.602 7f8c23f73700 1 --2- 172.21.2.101:0/593512537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c1c0e5170 0x7f8c1c1b38e0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 rx=0x7f8c10006650 tx=0x7f8c10009a80).stop 2021-02-10T11:35:12.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.603 7f8c23f73700 1 -- 172.21.2.101:0/593512537 shutdown_connections 2021-02-10T11:35:12.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.603 7f8c23f73700 1 --2- 172.21.2.101:0/593512537 >> v2:172.21.2.108:3300/0 conn(0x7f8c1c0e5570 0x7f8c1c07a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:12.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.603 7f8c23f73700 1 --2- 172.21.2.101:0/593512537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c1c0e5170 0x7f8c1c1b38e0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:12.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.603 7f8c23f73700 1 -- 172.21.2.101:0/593512537 >> 172.21.2.101:0/593512537 conn(0x7f8c1c0ce820 msgr2=0x7f8c1c0e4660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:12.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.603 7f8c23f73700 1 -- 172.21.2.101:0/593512537 shutdown_connections 2021-02-10T11:35:12.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.603 7f8c23f73700 1 -- 172.21.2.101:0/593512537 wait complete. 2021-02-10T11:35:12.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.618 7f8c23f73700 1 Processor -- start 2021-02-10T11:35:12.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.618 7f8c23f73700 1 -- start start 2021-02-10T11:35:12.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.618 7f8c23f73700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c1c10d1a0 0x7f8c1c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:12.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.618 7f8c23f73700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c1c1987f0 0x7f8c1c10da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:12.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.618 7f8c23f73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c1c10e040 0x7f8c1c1a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:12.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.618 7f8c23f73700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c198d30 con 0x7f8c1c10e040 2021-02-10T11:35:12.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.618 7f8c23f73700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c10d5a0 con 0x7f8c1c1987f0 2021-02-10T11:35:12.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.618 7f8c23f73700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c1a4590 con 0x7f8c1c10d1a0 2021-02-10T11:35:12.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.618 7f8c22590700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c1c10e040 0x7f8c1c1a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:12.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.618 7f8c22590700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c1c10e040 0x7f8c1c1a3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53034/0 (socket says 172.21.2.101:53034) 2021-02-10T11:35:12.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.618 7f8c22590700 1 -- 172.21.2.101:0/4227643629 learned_addr learned my addr 172.21.2.101:0/4227643629 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:12.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.619 7f8c21d8f700 1 --2- 172.21.2.101:0/4227643629 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c1c10d1a0 0x7f8c1c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:12.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.619 7f8c2158e700 1 --2- 172.21.2.101:0/4227643629 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c1c1987f0 0x7f8c1c10da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:12.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.619 7f8c22590700 1 -- 172.21.2.101:0/4227643629 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c1c10d1a0 msgr2=0x7f8c1c1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:12.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.619 7f8c22590700 1 --2- 172.21.2.101:0/4227643629 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c1c10d1a0 0x7f8c1c1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:12.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.619 7f8c22590700 1 -- 172.21.2.101:0/4227643629 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c1c1987f0 msgr2=0x7f8c1c10da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:12.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.619 7f8c22590700 1 --2- 172.21.2.101:0/4227643629 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c1c1987f0 0x7f8c1c10da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:12.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.619 7f8c22590700 1 -- 172.21.2.101:0/4227643629 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c1a47b0 con 0x7f8c1c10e040 2021-02-10T11:35:12.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.619 7f8c21d8f700 1 --2- 172.21.2.101:0/4227643629 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c1c10d1a0 0x7f8c1c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:35:12.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.619 7f8c2158e700 1 --2- 172.21.2.101:0/4227643629 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c1c1987f0 0x7f8c1c10da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:35:12.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.619 7f8c22590700 1 --2- 172.21.2.101:0/4227643629 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c1c10e040 0x7f8c1c1a3f70 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 rx=0x7f8c1400b970 tx=0x7f8c1400bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:12.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.619 7f8c0e7fc700 1 -- 172.21.2.101:0/4227643629 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c14000f90 con 0x7f8c1c10e040 2021-02-10T11:35:12.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.619 7f8c0e7fc700 1 -- 172.21.2.101:0/4227643629 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c140066f0 con 0x7f8c1c10e040 2021-02-10T11:35:12.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.619 7f8c0e7fc700 1 -- 172.21.2.101:0/4227643629 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c14001680 con 0x7f8c1c10e040 2021-02-10T11:35:12.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.619 7f8c23f73700 1 -- 172.21.2.101:0/4227643629 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1c1a47b0 con 0x7f8c1c10e040 2021-02-10T11:35:12.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.619 7f8c23f73700 1 -- 172.21.2.101:0/4227643629 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1c1a47b0 con 0x7f8c1c10e040 2021-02-10T11:35:12.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.620 7f8c0e7fc700 1 -- 172.21.2.101:0/4227643629 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8c14000df0 con 0x7f8c1c10e040 2021-02-10T11:35:12.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.620 7f8c0e7fc700 1 --2- 172.21.2.101:0/4227643629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c040186c0 0x7f8c0401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:12.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.620 7f8c0e7fc700 1 -- 172.21.2.101:0/4227643629 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8c14032750 con 0x7f8c1c10e040 2021-02-10T11:35:12.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.620 7f8c21d8f700 1 --2- 172.21.2.101:0/4227643629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c040186c0 0x7f8c0401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:12.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.620 7f8c21d8f700 1 --2- 172.21.2.101:0/4227643629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c040186c0 0x7f8c0401abc0 crc :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:12.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.650 7f8c23f73700 1 -- 172.21.2.101:0/4227643629 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1c10c990 con 0x7f8c1c10e040 2021-02-10T11:35:12.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.653 7f8c0e7fc700 1 -- 172.21.2.101:0/4227643629 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8c1402cf20 con 0x7f8c1c10e040 2021-02-10T11:35:12.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.779 7f8c23f73700 1 -- 172.21.2.101:0/4227643629 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f8c1c078b90 con 0x7f8c1c10e040 2021-02-10T11:35:12.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.780 7f8c0e7fc700 1 -- 172.21.2.101:0/4227643629 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8c1402cf20 con 0x7f8c1c10e040 2021-02-10T11:35:12.789 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:35:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.782 7f8c23f73700 1 -- 172.21.2.101:0/4227643629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c040186c0 msgr2=0x7f8c0401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.782 7f8c23f73700 1 --2- 172.21.2.101:0/4227643629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c040186c0 0x7f8c0401abc0 crc :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.782 7f8c23f73700 1 -- 172.21.2.101:0/4227643629 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c1c10e040 msgr2=0x7f8c1c1a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.782 7f8c23f73700 1 --2- 172.21.2.101:0/4227643629 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c1c10e040 0x7f8c1c1a3f70 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 rx=0x7f8c1400b970 tx=0x7f8c1400bd10).stop 2021-02-10T11:35:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.782 7f8c23f73700 1 -- 172.21.2.101:0/4227643629 shutdown_connections 2021-02-10T11:35:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.782 7f8c23f73700 1 --2- 172.21.2.101:0/4227643629 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c1c10e040 0x7f8c1c1a3f70 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.782 7f8c23f73700 1 --2- 172.21.2.101:0/4227643629 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c1c1987f0 0x7f8c1c10da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.782 7f8c23f73700 1 --2- 172.21.2.101:0/4227643629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c040186c0 0x7f8c0401abc0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.782 7f8c23f73700 1 --2- 172.21.2.101:0/4227643629 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c1c10d1a0 0x7f8c1c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.782 7f8c23f73700 1 -- 172.21.2.101:0/4227643629 >> 172.21.2.101:0/4227643629 conn(0x7f8c1c0a1220 msgr2=0x7f8c1c097d40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.782 7f8c23f73700 1 -- 172.21.2.101:0/4227643629 shutdown_connections 2021-02-10T11:35:12.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:12.782 7f8c23f73700 1 -- 172.21.2.101:0/4227643629 wait complete. 2021-02-10T11:35:12.799 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:35:13.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.205 7f584a5d8700 1 Processor -- start 2021-02-10T11:35:13.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.205 7f584a5d8700 1 -- start start 2021-02-10T11:35:13.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.205 7f584a5d8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f58440e5170 0x7f58441b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:13.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.205 7f584a5d8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f58441b3f00 0x7f5844077510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:13.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.205 7f584a5d8700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5844078b90 con 0x7f58440e5570 2021-02-10T11:35:13.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.205 7f584a5d8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f584407a120 con 0x7f58440e5170 2021-02-10T11:35:13.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.205 7f584a5d8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584407a310 con 0x7f58441b3f00 2021-02-10T11:35:13.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.205 7f5848bf5700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f58440e5570 0x7f5844077b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53534/0 (socket says 172.21.2.101:53534) 2021-02-10T11:35:13.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.206 7f5848bf5700 1 -- 172.21.2.101:0/3139583526 learned_addr learned my addr 172.21.2.101:0/3139583526 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:13.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.206 7f5843fff700 1 --2- 172.21.2.101:0/3139583526 >> v2:172.21.2.108:3300/0 conn(0x7f58440e5170 0x7f58441b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:13.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.206 7f58437fe700 1 --2- 172.21.2.101:0/3139583526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f58441b3f00 0x7f5844077510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:13.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.206 7f5842ffd700 1 -- 172.21.2.101:0/3139583526 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2040715060 0 0) 0x7f583c001500 con 0x7f58440e5570 2021-02-10T11:35:13.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.206 7f5842ffd700 1 -- 172.21.2.101:0/3139583526 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f582c002fe0 con 0x7f58440e5570 2021-02-10T11:35:13.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.206 7f5842ffd700 1 -- 172.21.2.101:0/3139583526 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1937072548 0 0) 0x7f583c001500 con 0x7f58440e5570 2021-02-10T11:35:13.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.206 7f58437fe700 1 -- 172.21.2.101:0/3139583526 >> v2:172.21.2.108:3300/0 conn(0x7f58440e5170 msgr2=0x7f58441b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:13.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.206 7f58437fe700 1 --2- 172.21.2.101:0/3139583526 >> v2:172.21.2.108:3300/0 conn(0x7f58440e5170 0x7f58441b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:13.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.206 7f58437fe700 1 -- 172.21.2.101:0/3139583526 >> v1:172.21.2.101:6789/0 conn(0x7f58440e5570 legacy=0x7f5844077b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:13.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.206 7f58437fe700 1 -- 172.21.2.101:0/3139583526 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584407a530 con 0x7f58441b3f00 2021-02-10T11:35:13.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.207 7f58437fe700 1 --2- 172.21.2.101:0/3139583526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f58441b3f00 0x7f5844077510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f5838006650 tx=0x7f5838009a80).ready entity=mon.2 client_cookie=4abaf779c6b008df server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:13.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.207 7f5842ffd700 1 -- 172.21.2.101:0/3139583526 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f583800c0c0 con 0x7f58441b3f00 2021-02-10T11:35:13.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.207 7f5842ffd700 1 -- 172.21.2.101:0/3139583526 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583800c340 con 0x7f58441b3f00 2021-02-10T11:35:13.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.207 7f5842ffd700 1 -- 172.21.2.101:0/3139583526 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f583800c860 con 0x7f58441b3f00 2021-02-10T11:35:13.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.207 7f584a5d8700 1 -- 172.21.2.101:0/3139583526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f58441b3f00 msgr2=0x7f5844077510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:13.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.207 7f584a5d8700 1 --2- 172.21.2.101:0/3139583526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f58441b3f00 0x7f5844077510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f5838006650 tx=0x7f5838009a80).stop 2021-02-10T11:35:13.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.207 7f584a5d8700 1 -- 172.21.2.101:0/3139583526 shutdown_connections 2021-02-10T11:35:13.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.207 7f584a5d8700 1 --2- 172.21.2.101:0/3139583526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f58441b3f00 0x7f5844077510 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:13.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.207 7f584a5d8700 1 --2- 172.21.2.101:0/3139583526 >> v2:172.21.2.108:3300/0 conn(0x7f58440e5170 0x7f58441b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:13.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.207 7f584a5d8700 1 -- 172.21.2.101:0/3139583526 >> 172.21.2.101:0/3139583526 conn(0x7f58440ce820 msgr2=0x7f58440e4660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:13.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.208 7f584a5d8700 1 -- 172.21.2.101:0/3139583526 shutdown_connections 2021-02-10T11:35:13.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.208 7f584a5d8700 1 -- 172.21.2.101:0/3139583526 wait complete. 2021-02-10T11:35:13.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.222 7f584a5d8700 1 Processor -- start 2021-02-10T11:35:13.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.222 7f584a5d8700 1 -- start start 2021-02-10T11:35:13.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.222 7f584a5d8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f58440933d0 0x7f5844096430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:13.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.222 7f584a5d8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f58441a2b60 0x7f58441a2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:13.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.222 7f584a5d8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f58441a3580 0x7f5844094f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:13.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.222 7f584a5d8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58440b0340 con 0x7f58441a3580 2021-02-10T11:35:13.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.222 7f584a5d8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5844096a90 con 0x7f58441a2b60 2021-02-10T11:35:13.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.222 7f584a5d8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5844095580 con 0x7f58440933d0 2021-02-10T11:35:13.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.223 7f5848bf5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f58441a3580 0x7f5844094f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:13.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.223 7f5848bf5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f58441a3580 0x7f5844094f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53044/0 (socket says 172.21.2.101:53044) 2021-02-10T11:35:13.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.223 7f5848bf5700 1 -- 172.21.2.101:0/451482317 learned_addr learned my addr 172.21.2.101:0/451482317 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:13.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.223 7f58437fe700 1 --2- 172.21.2.101:0/451482317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f58441a2b60 0x7f58441a2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:13.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.223 7f5843fff700 1 --2- 172.21.2.101:0/451482317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f58440933d0 0x7f5844096430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:13.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.223 7f5848bf5700 1 -- 172.21.2.101:0/451482317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f58440933d0 msgr2=0x7f5844096430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:13.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.223 7f5848bf5700 1 --2- 172.21.2.101:0/451482317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f58440933d0 0x7f5844096430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:13.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.223 7f5848bf5700 1 -- 172.21.2.101:0/451482317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f58441a2b60 msgr2=0x7f58441a2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:13.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.223 7f5848bf5700 1 --2- 172.21.2.101:0/451482317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f58441a2b60 0x7f58441a2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:13.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.223 7f5848bf5700 1 -- 172.21.2.101:0/451482317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58440957a0 con 0x7f58441a3580 2021-02-10T11:35:13.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.223 7f5848bf5700 1 --2- 172.21.2.101:0/451482317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f58441a3580 0x7f5844094f60 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 rx=0x7f583c00a900 tx=0x7f583c00aca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:13.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.224 7f5840ff9700 1 -- 172.21.2.101:0/451482317 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f583c004930 con 0x7f58441a3580 2021-02-10T11:35:13.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.224 7f5840ff9700 1 -- 172.21.2.101:0/451482317 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583c000d30 con 0x7f58441a3580 2021-02-10T11:35:13.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.224 7f5840ff9700 1 -- 172.21.2.101:0/451482317 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f583c007020 con 0x7f58441a3580 2021-02-10T11:35:13.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.224 7f584a5d8700 1 -- 172.21.2.101:0/451482317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58440957a0 con 0x7f58441a3580 2021-02-10T11:35:13.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.224 7f584a5d8700 1 -- 172.21.2.101:0/451482317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5844095d30 con 0x7f58441a3580 2021-02-10T11:35:13.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.225 7f5840ff9700 1 -- 172.21.2.101:0/451482317 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f583c004720 con 0x7f58441a3580 2021-02-10T11:35:13.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.225 7f5840ff9700 1 --2- 172.21.2.101:0/451482317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f58240186c0 0x7f582401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:13.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.225 7f5840ff9700 1 -- 172.21.2.101:0/451482317 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f583c0328b0 con 0x7f58441a3580 2021-02-10T11:35:13.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.225 7f5843fff700 1 --2- 172.21.2.101:0/451482317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f58240186c0 0x7f582401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:13.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.225 7f5843fff700 1 --2- 172.21.2.101:0/451482317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f58240186c0 0x7f582401abc0 crc :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:13.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.255 7f584a5d8700 1 -- 172.21.2.101:0/451482317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584410c9c0 con 0x7f58441a3580 2021-02-10T11:35:13.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.258 7f5840ff9700 1 -- 172.21.2.101:0/451482317 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f583c004c70 con 0x7f58441a3580 2021-02-10T11:35:13.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.388 7f584a5d8700 1 -- 172.21.2.101:0/451482317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f58440fe290 con 0x7f58441a3580 2021-02-10T11:35:13.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.388 7f5840ff9700 1 -- 172.21.2.101:0/451482317 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v65) v1 ==== 104+0+10 (secure 0 0 0) 0x7f583c004720 con 0x7f58441a3580 2021-02-10T11:35:13.397 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:35:13.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.390 7f584a5d8700 1 -- 172.21.2.101:0/451482317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f58240186c0 msgr2=0x7f582401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:13.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.390 7f584a5d8700 1 --2- 172.21.2.101:0/451482317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f58240186c0 0x7f582401abc0 crc :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:13.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.390 7f584a5d8700 1 -- 172.21.2.101:0/451482317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f58441a3580 msgr2=0x7f5844094f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:13.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.390 7f584a5d8700 1 --2- 172.21.2.101:0/451482317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f58441a3580 0x7f5844094f60 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 rx=0x7f583c00a900 tx=0x7f583c00aca0).stop 2021-02-10T11:35:13.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.390 7f584a5d8700 1 -- 172.21.2.101:0/451482317 shutdown_connections 2021-02-10T11:35:13.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.390 7f584a5d8700 1 --2- 172.21.2.101:0/451482317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f58441a3580 0x7f5844094f60 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:13.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.390 7f584a5d8700 1 --2- 172.21.2.101:0/451482317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f58441a2b60 0x7f58441a2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:13.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.390 7f584a5d8700 1 --2- 172.21.2.101:0/451482317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f58240186c0 0x7f582401abc0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:13.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.390 7f584a5d8700 1 --2- 172.21.2.101:0/451482317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f58440933d0 0x7f5844096430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:13.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.390 7f584a5d8700 1 -- 172.21.2.101:0/451482317 >> 172.21.2.101:0/451482317 conn(0x7f58440b8d00 msgr2=0x7f58440b7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:13.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.390 7f584a5d8700 1 -- 172.21.2.101:0/451482317 shutdown_connections 2021-02-10T11:35:13.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.390 7f584a5d8700 1 -- 172.21.2.101:0/451482317 wait complete. 2021-02-10T11:35:13.408 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:35:13.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.812 7fca34873700 1 Processor -- start 2021-02-10T11:35:13.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.812 7fca34873700 1 -- start start 2021-02-10T11:35:13.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.812 7fca34873700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fca300e5170 0x7fca301b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:13.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.812 7fca34873700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca30078b90 con 0x7fca300e5570 2021-02-10T11:35:13.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.812 7fca34873700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca300784f0 con 0x7fca301b3f00 2021-02-10T11:35:13.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.812 7fca34873700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca30078710 con 0x7fca300e5170 2021-02-10T11:35:13.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.813 7fca2ee1d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fca300e5570 0x7fca30077d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53546/0 (socket says 172.21.2.101:53546) 2021-02-10T11:35:13.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.813 7fca2ee1d700 1 -- 172.21.2.101:0/42501765 learned_addr learned my addr 172.21.2.101:0/42501765 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:13.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.813 7fca2d61a700 1 -- 172.21.2.101:0/42501765 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214884576 0 0) 0x7fca28001500 con 0x7fca300e5570 2021-02-10T11:35:13.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.813 7fca2d61a700 1 -- 172.21.2.101:0/42501765 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca1c002fe0 con 0x7fca300e5570 2021-02-10T11:35:13.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.813 7fca2d61a700 1 -- 172.21.2.101:0/42501765 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3215541888 0 0) 0x7fca28001460 con 0x7fca300e5570 2021-02-10T11:35:13.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.813 7fca2d61a700 1 -- 172.21.2.101:0/42501765 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fca300e5170 msgr2=0x7fca301b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:13.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.813 7fca2d61a700 1 --2- 172.21.2.101:0/42501765 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fca300e5170 0x7fca301b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:13.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.813 7fca2d61a700 1 -- 172.21.2.101:0/42501765 >> v1:172.21.2.108:6789/0 conn(0x7fca301b3f00 legacy=0x7fca30077510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:13.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.813 7fca2d61a700 1 -- 172.21.2.101:0/42501765 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca300788c0 con 0x7fca300e5570 2021-02-10T11:35:13.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.813 7fca2d61a700 1 -- 172.21.2.101:0/42501765 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fca28001980 con 0x7fca300e5570 2021-02-10T11:35:13.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.813 7fca2d61a700 1 -- 172.21.2.101:0/42501765 >> v1:172.21.2.101:6789/0 conn(0x7fca300e5570 legacy=0x7fca30077d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:13.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.813 7fca2d61a700 1 --2- 172.21.2.101:0/42501765 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fca1c003a70 0x7fca1c005ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:13.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.813 7fca2d61a700 1 --2- 172.21.2.101:0/42501765 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fca1c006740 0x7fca1c008c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:13.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.813 7fca2d61a700 1 --2- 172.21.2.101:0/42501765 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca1c009500 0x7fca1c00ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:13.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.813 7fca2d61a700 1 -- 172.21.2.101:0/42501765 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c00c2d0 con 0x7fca1c009500 2021-02-10T11:35:13.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.813 7fca2d61a700 1 -- 172.21.2.101:0/42501765 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c00c4f0 con 0x7fca1c006740 2021-02-10T11:35:13.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.813 7fca2d61a700 1 -- 172.21.2.101:0/42501765 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c00c710 con 0x7fca1c003a70 2021-02-10T11:35:13.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.813 7fca2ee1d700 1 --2- 172.21.2.101:0/42501765 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca1c009500 0x7fca1c00ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:13.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.813 7fca2e61c700 1 --2- 172.21.2.101:0/42501765 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fca1c003a70 0x7fca1c005ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:13.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.814 7fca2de1b700 1 --2- 172.21.2.101:0/42501765 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fca1c006740 0x7fca1c008c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:13.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.814 7fca2ee1d700 1 -- 172.21.2.101:0/42501765 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fca1c003a70 msgr2=0x7fca1c005ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:13.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.814 7fca2ee1d700 1 --2- 172.21.2.101:0/42501765 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fca1c003a70 0x7fca1c005ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:13.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.814 7fca2ee1d700 1 -- 172.21.2.101:0/42501765 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fca1c006740 msgr2=0x7fca1c008c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:13.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.814 7fca2ee1d700 1 --2- 172.21.2.101:0/42501765 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fca1c006740 0x7fca1c008c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:13.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.814 7fca2ee1d700 1 -- 172.21.2.101:0/42501765 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c00c9b0 con 0x7fca1c009500 2021-02-10T11:35:13.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.814 7fca2ee1d700 1 --2- 172.21.2.101:0/42501765 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca1c009500 0x7fca1c00ba20 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 rx=0x7fca28001c20 tx=0x7fca28050310).ready entity=mon.0 client_cookie=360463965c3b87b6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:13.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.814 7fca2d61a700 1 -- 172.21.2.101:0/42501765 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fca28007020 con 0x7fca1c009500 2021-02-10T11:35:13.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.814 7fca2d61a700 1 -- 172.21.2.101:0/42501765 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca280072a0 con 0x7fca1c009500 2021-02-10T11:35:13.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.814 7fca2d61a700 1 -- 172.21.2.101:0/42501765 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fca28004720 con 0x7fca1c009500 2021-02-10T11:35:13.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.814 7fca34873700 1 -- 172.21.2.101:0/42501765 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca1c009500 msgr2=0x7fca1c00ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:13.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.814 7fca34873700 1 --2- 172.21.2.101:0/42501765 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca1c009500 0x7fca1c00ba20 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 rx=0x7fca28001c20 tx=0x7fca28050310).stop 2021-02-10T11:35:13.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.814 7fca2ee1d700 1 -- 172.21.2.101:0/42501765 reap_dead start 2021-02-10T11:35:13.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.814 7fca34873700 1 -- 172.21.2.101:0/42501765 shutdown_connections 2021-02-10T11:35:13.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.814 7fca34873700 1 -- 172.21.2.101:0/42501765 >> 172.21.2.101:0/42501765 conn(0x7fca300ce820 msgr2=0x7fca300e4660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:13.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.815 7fca34873700 1 -- 172.21.2.101:0/42501765 shutdown_connections 2021-02-10T11:35:13.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.815 7fca34873700 1 -- 172.21.2.101:0/42501765 wait complete. 2021-02-10T11:35:13.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.829 7fca34873700 1 Processor -- start 2021-02-10T11:35:13.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.829 7fca34873700 1 -- start start 2021-02-10T11:35:13.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.829 7fca34873700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca301a0160 0x7fca300ac950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:13.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.829 7fca34873700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fca300acf70 0x7fca30091cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:13.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.829 7fca34873700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fca300922e0 0x7fca300926e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:13.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.829 7fca34873700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca300ad370 con 0x7fca301a0160 2021-02-10T11:35:13.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.829 7fca34873700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca300ad710 con 0x7fca300acf70 2021-02-10T11:35:13.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.829 7fca34873700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca30092d00 con 0x7fca300922e0 2021-02-10T11:35:13.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.830 7fca2e61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca301a0160 0x7fca300ac950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:13.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.830 7fca2e61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca301a0160 0x7fca300ac950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53064/0 (socket says 172.21.2.101:53064) 2021-02-10T11:35:13.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.830 7fca2e61c700 1 -- 172.21.2.101:0/125962762 learned_addr learned my addr 172.21.2.101:0/125962762 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:13.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.830 7fca2e61c700 1 -- 172.21.2.101:0/125962762 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fca300922e0 msgr2=0x7fca300926e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:35:13.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.830 7fca2e61c700 1 --2- 172.21.2.101:0/125962762 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fca300922e0 0x7fca300926e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:13.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.830 7fca2e61c700 1 -- 172.21.2.101:0/125962762 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fca300acf70 msgr2=0x7fca30091cc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:35:13.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.830 7fca2e61c700 1 --2- 172.21.2.101:0/125962762 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fca300acf70 0x7fca30091cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:13.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.830 7fca2e61c700 1 -- 172.21.2.101:0/125962762 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca30092f20 con 0x7fca301a0160 2021-02-10T11:35:13.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.830 7fca2e61c700 1 --2- 172.21.2.101:0/125962762 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca301a0160 0x7fca300ac950 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 rx=0x7fca1800b530 tx=0x7fca1800b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:13.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.830 7fca26ffd700 1 -- 172.21.2.101:0/125962762 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fca1800bcc0 con 0x7fca301a0160 2021-02-10T11:35:13.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.830 7fca26ffd700 1 -- 172.21.2.101:0/125962762 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca18002030 con 0x7fca301a0160 2021-02-10T11:35:13.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.830 7fca34873700 1 -- 172.21.2.101:0/125962762 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca30092f20 con 0x7fca301a0160 2021-02-10T11:35:13.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.830 7fca26ffd700 1 -- 172.21.2.101:0/125962762 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fca18002480 con 0x7fca301a0160 2021-02-10T11:35:13.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.830 7fca34873700 1 -- 172.21.2.101:0/125962762 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca30090450 con 0x7fca301a0160 2021-02-10T11:35:13.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.831 7fca26ffd700 1 -- 172.21.2.101:0/125962762 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fca18000c90 con 0x7fca301a0160 2021-02-10T11:35:13.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.831 7fca26ffd700 1 --2- 172.21.2.101:0/125962762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fca140186c0 0x7fca1401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:13.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.831 7fca26ffd700 1 -- 172.21.2.101:0/125962762 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fca18032430 con 0x7fca301a0160 2021-02-10T11:35:13.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.831 7fca2de1b700 1 --2- 172.21.2.101:0/125962762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fca140186c0 0x7fca1401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:13.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.831 7fca2de1b700 1 --2- 172.21.2.101:0/125962762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fca140186c0 0x7fca1401abc0 crc :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:13.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.862 7fca34873700 1 -- 172.21.2.101:0/125962762 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3010ca30 con 0x7fca301a0160 2021-02-10T11:35:13.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.865 7fca26ffd700 1 -- 172.21.2.101:0/125962762 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fca18000940 con 0x7fca301a0160 2021-02-10T11:35:14.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.991 7fca34873700 1 -- 172.21.2.101:0/125962762 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7fca300fe290 con 0x7fca301a0160 2021-02-10T11:35:14.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.991 7fca26ffd700 1 -- 172.21.2.101:0/125962762 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v65) v1 ==== 100+0+10 (secure 0 0 0) 0x7fca18000c90 con 0x7fca301a0160 2021-02-10T11:35:14.000 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:35:14.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.993 7fca34873700 1 -- 172.21.2.101:0/125962762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fca140186c0 msgr2=0x7fca1401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:14.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.993 7fca34873700 1 --2- 172.21.2.101:0/125962762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fca140186c0 0x7fca1401abc0 crc :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:14.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.993 7fca34873700 1 -- 172.21.2.101:0/125962762 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca301a0160 msgr2=0x7fca300ac950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:14.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.993 7fca34873700 1 --2- 172.21.2.101:0/125962762 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca301a0160 0x7fca300ac950 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 rx=0x7fca1800b530 tx=0x7fca1800b8d0).stop 2021-02-10T11:35:14.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.994 7fca34873700 1 -- 172.21.2.101:0/125962762 shutdown_connections 2021-02-10T11:35:14.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.994 7fca34873700 1 --2- 172.21.2.101:0/125962762 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fca140186c0 0x7fca1401abc0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:14.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.994 7fca34873700 1 --2- 172.21.2.101:0/125962762 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fca300922e0 0x7fca300926e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:14.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.994 7fca34873700 1 --2- 172.21.2.101:0/125962762 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fca300acf70 0x7fca30091cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:14.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.994 7fca34873700 1 --2- 172.21.2.101:0/125962762 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca301a0160 0x7fca300ac950 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:14.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.994 7fca34873700 1 -- 172.21.2.101:0/125962762 >> 172.21.2.101:0/125962762 conn(0x7fca30188880 msgr2=0x7fca300a4380 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:14.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.994 7fca34873700 1 -- 172.21.2.101:0/125962762 shutdown_connections 2021-02-10T11:35:14.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:13.994 7fca34873700 1 -- 172.21.2.101:0/125962762 wait complete. 2021-02-10T11:35:14.011 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:35:14.011 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:35:14.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.412 7fa26767f700 1 Processor -- start 2021-02-10T11:35:14.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.412 7fa26767f700 1 -- start start 2021-02-10T11:35:14.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.413 7fa26767f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa2600e53e0 0x7fa2601b0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:14.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.413 7fa26767f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa2600842e0 0x7fa260082100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:14.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.413 7fa26767f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa260082d50 con 0x7fa2601b0b70 2021-02-10T11:35:14.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.413 7fa26767f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2600e5fe0 con 0x7fa2600e53e0 2021-02-10T11:35:14.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.413 7fa26767f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2600e57e0 con 0x7fa2600842e0 2021-02-10T11:35:14.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.413 7fa26549b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa2600e53e0 0x7fa2601b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:14.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.413 7fa264c9a700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa2601b0b70 0x7fa260081910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53564/0 (socket says 172.21.2.101:53564) 2021-02-10T11:35:14.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.413 7fa26549b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa2600e53e0 0x7fa2601b0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53070/0 (socket says 172.21.2.101:53070) 2021-02-10T11:35:14.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.413 7fa264c9a700 1 -- 172.21.2.101:0/3495638425 learned_addr learned my addr 172.21.2.101:0/3495638425 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:14.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.413 7fa265c9c700 1 --2- 172.21.2.101:0/3495638425 >> v2:172.21.2.108:3300/0 conn(0x7fa2600842e0 0x7fa260082100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:14.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.413 7fa26549b700 1 -- 172.21.2.101:0/3495638425 >> v2:172.21.2.108:3300/0 conn(0x7fa2600842e0 msgr2=0x7fa260082100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:14.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.413 7fa26549b700 1 --2- 172.21.2.101:0/3495638425 >> v2:172.21.2.108:3300/0 conn(0x7fa2600842e0 0x7fa260082100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:14.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.413 7fa26549b700 1 -- 172.21.2.101:0/3495638425 >> v1:172.21.2.101:6789/0 conn(0x7fa2601b0b70 legacy=0x7fa260081910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:14.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.413 7fa26549b700 1 -- 172.21.2.101:0/3495638425 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2600846e0 con 0x7fa2600e53e0 2021-02-10T11:35:14.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.413 7fa26549b700 1 --2- 172.21.2.101:0/3495638425 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2600e53e0 0x7fa2601b0550 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 rx=0x7fa254006650 tx=0x7fa254009a80).ready entity=mon.0 client_cookie=e31e7bb0c01ecd9f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:14.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.414 7fa253fff700 1 -- 172.21.2.101:0/3495638425 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa25400c0c0 con 0x7fa2600e53e0 2021-02-10T11:35:14.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.414 7fa253fff700 1 -- 172.21.2.101:0/3495638425 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25400c340 con 0x7fa2600e53e0 2021-02-10T11:35:14.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.414 7fa253fff700 1 -- 172.21.2.101:0/3495638425 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa25400c860 con 0x7fa2600e53e0 2021-02-10T11:35:14.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.414 7fa26767f700 1 -- 172.21.2.101:0/3495638425 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2600e53e0 msgr2=0x7fa2601b0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:14.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.414 7fa26767f700 1 --2- 172.21.2.101:0/3495638425 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2600e53e0 0x7fa2601b0550 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 rx=0x7fa254006650 tx=0x7fa254009a80).stop 2021-02-10T11:35:14.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.414 7fa26767f700 1 -- 172.21.2.101:0/3495638425 shutdown_connections 2021-02-10T11:35:14.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.414 7fa26767f700 1 --2- 172.21.2.101:0/3495638425 >> v2:172.21.2.108:3300/0 conn(0x7fa2600842e0 0x7fa260082100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:14.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.414 7fa26767f700 1 --2- 172.21.2.101:0/3495638425 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2600e53e0 0x7fa2601b0550 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:14.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.414 7fa26767f700 1 -- 172.21.2.101:0/3495638425 >> 172.21.2.101:0/3495638425 conn(0x7fa2600ce500 msgr2=0x7fa2600e4610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:14.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.414 7fa26767f700 1 -- 172.21.2.101:0/3495638425 shutdown_connections 2021-02-10T11:35:14.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.414 7fa26767f700 1 -- 172.21.2.101:0/3495638425 wait complete. 2021-02-10T11:35:14.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.429 7fa26767f700 1 Processor -- start 2021-02-10T11:35:14.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.429 7fa26767f700 1 -- start start 2021-02-10T11:35:14.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.429 7fa26767f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2600b1af0 0x7fa2600b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:14.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.429 7fa26767f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2600b2510 0x7fa2601b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:14.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.429 7fa26767f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2601b85c0 0x7fa2601b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:14.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.429 7fa26767f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2600b2910 con 0x7fa2600b2510 2021-02-10T11:35:14.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.429 7fa26767f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2601b7a50 con 0x7fa2600b1af0 2021-02-10T11:35:14.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.429 7fa26767f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2601b7780 con 0x7fa2601b85c0 2021-02-10T11:35:14.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.429 7fa264c9a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2600b2510 0x7fa2601b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:14.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.429 7fa264c9a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2600b2510 0x7fa2601b7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53076/0 (socket says 172.21.2.101:53076) 2021-02-10T11:35:14.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.429 7fa264c9a700 1 -- 172.21.2.101:0/2909026275 learned_addr learned my addr 172.21.2.101:0/2909026275 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:14.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.430 7fa26549b700 1 --2- 172.21.2.101:0/2909026275 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2600b1af0 0x7fa2600b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:14.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.430 7fa265c9c700 1 --2- 172.21.2.101:0/2909026275 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2601b85c0 0x7fa2601b89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:14.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.430 7fa264c9a700 1 -- 172.21.2.101:0/2909026275 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2601b85c0 msgr2=0x7fa2601b89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:14.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.430 7fa264c9a700 1 --2- 172.21.2.101:0/2909026275 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2601b85c0 0x7fa2601b89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:14.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.430 7fa264c9a700 1 -- 172.21.2.101:0/2909026275 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2600b1af0 msgr2=0x7fa2600b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:14.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.430 7fa264c9a700 1 --2- 172.21.2.101:0/2909026275 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2600b1af0 0x7fa2600b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:14.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.430 7fa264c9a700 1 -- 172.21.2.101:0/2909026275 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2601b8fe0 con 0x7fa2600b2510 2021-02-10T11:35:14.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.430 7fa264c9a700 1 --2- 172.21.2.101:0/2909026275 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2600b2510 0x7fa2601b7fa0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 rx=0x7fa25c00a4c0 tx=0x7fa25c00a860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:14.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.430 7fa251ffb700 1 -- 172.21.2.101:0/2909026275 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa25c00ac50 con 0x7fa2600b2510 2021-02-10T11:35:14.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.430 7fa251ffb700 1 -- 172.21.2.101:0/2909026275 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25c0012a0 con 0x7fa2600b2510 2021-02-10T11:35:14.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.430 7fa26767f700 1 -- 172.21.2.101:0/2909026275 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2601b8fe0 con 0x7fa2600b2510 2021-02-10T11:35:14.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.430 7fa251ffb700 1 -- 172.21.2.101:0/2909026275 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa25c004330 con 0x7fa2600b2510 2021-02-10T11:35:14.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.430 7fa26767f700 1 -- 172.21.2.101:0/2909026275 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2601b8fe0 con 0x7fa2600b2510 2021-02-10T11:35:14.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.430 7fa251ffb700 1 -- 172.21.2.101:0/2909026275 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa25c004140 con 0x7fa2600b2510 2021-02-10T11:35:14.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.431 7fa251ffb700 1 --2- 172.21.2.101:0/2909026275 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa240018730 0x7fa24001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:14.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.431 7fa251ffb700 1 -- 172.21.2.101:0/2909026275 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa25c0325e0 con 0x7fa2600b2510 2021-02-10T11:35:14.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.431 7fa26549b700 1 --2- 172.21.2.101:0/2909026275 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa240018730 0x7fa24001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:14.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.431 7fa26549b700 1 --2- 172.21.2.101:0/2909026275 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa240018730 0x7fa24001abc0 crc :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:14.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.463 7fa26767f700 1 -- 172.21.2.101:0/2909026275 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2600ed660 con 0x7fa2600b2510 2021-02-10T11:35:14.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.465 7fa251ffb700 1 -- 172.21.2.101:0/2909026275 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa25c0009b0 con 0x7fa2600b2510 2021-02-10T11:35:14.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.596 7fa26767f700 1 -- 172.21.2.101:0/2909026275 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa260082d50 con 0x7fa2600b2510 2021-02-10T11:35:14.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.597 7fa251ffb700 1 -- 172.21.2.101:0/2909026275 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+2101 (secure 0 0 0) 0x7fa25c0012a0 con 0x7fa2600b2510 2021-02-10T11:35:14.605 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 47 2021-02-10T11:35:14.605 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:14.605 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1297756180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1297756180},{"type":"v1","addr":"172.21.2.101:6835","nonce":1297756180}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:34:41.088085","modified":"2021-02-10 11:34:43.450805","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6400":{"gid":6400,"name":"b","rank":0,"incarnation":44,"state":"up:active","state_seq":4,"addr":"172.21.2.108:6833/3585539482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3585539482},{"type":"v1","addr":"172.21.2.108:6833","nonce":3585539482}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-02-10T11:35:14.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.599 7fa26767f700 1 -- 172.21.2.101:0/2909026275 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa240018730 msgr2=0x7fa24001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:14.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.599 7fa26767f700 1 --2- 172.21.2.101:0/2909026275 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa240018730 0x7fa24001abc0 crc :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:14.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.599 7fa26767f700 1 -- 172.21.2.101:0/2909026275 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2600b2510 msgr2=0x7fa2601b7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:14.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.599 7fa26767f700 1 --2- 172.21.2.101:0/2909026275 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2600b2510 0x7fa2601b7fa0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 rx=0x7fa25c00a4c0 tx=0x7fa25c00a860).stop 2021-02-10T11:35:14.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.599 7fa26767f700 1 -- 172.21.2.101:0/2909026275 shutdown_connections 2021-02-10T11:35:14.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.599 7fa26767f700 1 --2- 172.21.2.101:0/2909026275 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa240018730 0x7fa24001abc0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:14.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.599 7fa26767f700 1 --2- 172.21.2.101:0/2909026275 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2601b85c0 0x7fa2601b89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:14.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.599 7fa26767f700 1 --2- 172.21.2.101:0/2909026275 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2600b2510 0x7fa2601b7fa0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:14.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.599 7fa26767f700 1 --2- 172.21.2.101:0/2909026275 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2600b1af0 0x7fa2600b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:14.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.599 7fa26767f700 1 -- 172.21.2.101:0/2909026275 >> 172.21.2.101:0/2909026275 conn(0x7fa2600f2240 msgr2=0x7fa260094840 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:14.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.599 7fa26767f700 1 -- 172.21.2.101:0/2909026275 shutdown_connections 2021-02-10T11:35:14.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:14.599 7fa26767f700 1 -- 172.21.2.101:0/2909026275 wait complete. 2021-02-10T11:35:14.816 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:34:41.088085', 'modified': '2021-02-10 11:34:43.450805', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6400}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6400': {'gid': 6400, 'name': 'b', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.2.108:6833/3585539482', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 3585539482}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 3585539482}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:35:14.816 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:35:14.817 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:35:15.214 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:35:15.214 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:35:15.214 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:35:15.214 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 6400, 2021-02-10T11:35:15.214 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:35:15.215 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:35:15.215 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 34.111705616000002, 2021-02-10T11:35:15.215 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 46, 2021-02-10T11:35:15.215 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 65, 2021-02-10T11:35:15.215 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 65, 2021-02-10T11:35:15.215 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 42.833162244 2021-02-10T11:35:15.216 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:35:15.218 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 6400, "want_state": "up:active", "state": "up:active", "rank_uptime": 34.111705616000002, "mdsmap_epoch": 46, "osdmap_epoch": 65, "osdmap_epoch_barrier": 65, "uptime": 42.833162244 } 2021-02-10T11:35:15.219 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:35:15.219 INFO:tasks.cephfs.fuse_mount:Client client.alice config is {} 2021-02-10T11:35:15.219 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.alice at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:35:15.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:15.219 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:15.558 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:35:15.906 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:35:15.906 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:35:16.250 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:35:16.250 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:35:16.251 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:35:16.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:16.252 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:35:16.585 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:35:16.585 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2021-02-10T11:35:16.585 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --client_mountpoint=/volumes/group_0000000000497714/subvolume_0000000000063820/d66a765b-b620-4859-b962-f6b2ea79bb52 --name client.alice /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:16.588 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:35:16.932 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:35:16.932 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:35:16.999 INFO:tasks.cephfs.fuse_mount.ceph-fuse.alice.gibba001.stderr:ceph-fuse[126321]: starting ceph client 2021-02-10T11:35:16.999 INFO:tasks.cephfs.fuse_mount.ceph-fuse.alice.gibba001.stderr:2021-02-10 11:35:16.990 7fec3e053f80 -1 init, newargv = 0x557a9d0a9450 newargc=9 2021-02-10T11:35:17.010 INFO:tasks.cephfs.fuse_mount.ceph-fuse.alice.gibba001.stderr:ceph-fuse[126321]: starting fuse 2021-02-10T11:35:17.275 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:35:17.275 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:35:17.276 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:35:17.276 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:17.276 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:35:17.612 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:35:17.612 INFO:teuthology.orchestra.run.gibba001.stdout:44 2021-02-10T11:35:17.612 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2021-02-10T11:35:17.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:17.612 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:35:17.613 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:35:17.613 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:35:17.613 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:35:17.613 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:35:17.613 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:35:17.613 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:35:17.613 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.alice.*.asok" 2021-02-10T11:35:17.613 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:35:17.613 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:35:17.613 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:35:17.613 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:35:17.613 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:35:17.613 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:35:17.614 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:35:17.614 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:35:17.614 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:35:17.614 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:35:17.614 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:35:17.614 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:35:17.614 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:35:17.614 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.alice")) 2021-02-10T11:35:17.614 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:35:17.976 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.alice.126321.asok 2021-02-10T11:35:17.978 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.alice.126321.asok 2021-02-10T11:35:17.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:17.979 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.alice.126321.asok status 2021-02-10T11:35:18.369 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:35:18.369 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:35:18.369 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:35:18.369 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:35:18.370 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "alice", 2021-02-10T11:35:18.370 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:35:18.370 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:35:18.370 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "126321", 2021-02-10T11:35:18.370 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/volumes/group_0000000000497714/subvolume_0000000000063820/d66a765b-b620-4859-b962-f6b2ea79bb52" 2021-02-10T11:35:18.370 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:18.370 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:35:18.370 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:35:18.370 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 8802, 2021-02-10T11:35:18.370 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:35:18.371 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:35:18.371 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:35:18.371 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 8802 2021-02-10T11:35:18.371 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:18.371 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:35:18.371 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:18.371 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:35:18.371 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1476870269 2021-02-10T11:35:18.371 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:18.371 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:18.371 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:35:18.371 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:18.372 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:35:18.372 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1476870269 2021-02-10T11:35:18.372 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:18.372 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.8802 172.21.2.101:0/1476870269", 2021-02-10T11:35:18.372 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/1476870269", 2021-02-10T11:35:18.372 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:35:18.372 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 46, 2021-02-10T11:35:18.372 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 65, 2021-02-10T11:35:18.372 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:35:18.372 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:35:18.372 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:35:18.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:18.374 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:35:18.374 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:18.709 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:35:18.709 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:35:18.709 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.1 && sudo dd if=/dev/urandom of=data.bin bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:35:19.086 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:35:19.086 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:35:19.086 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0268177 s, 39.1 MB/s 2021-02-10T11:35:19.087 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume authorize cephfs subvolume_0000000000063820 alice --group_name group_0000000000497714 --tenant_id tenant_id --access_level r 2021-02-10T11:35:19.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.500 7f1898994700 1 Processor -- start 2021-02-10T11:35:19.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.500 7f1898994700 1 -- start start 2021-02-10T11:35:19.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.501 7f1898994700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18940e5170 0x7f18941b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:19.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.501 7f1898994700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1894078b90 con 0x7f18940e5570 2021-02-10T11:35:19.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.501 7f1898994700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18940784f0 con 0x7f18941b3f00 2021-02-10T11:35:19.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.501 7f1898994700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1894078710 con 0x7f18940e5170 2021-02-10T11:35:19.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.501 7f1892e1d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f18940e5570 0x7f1894077d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53592/0 (socket says 172.21.2.101:53592) 2021-02-10T11:35:19.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.501 7f1892e1d700 1 -- 172.21.2.101:0/715968024 learned_addr learned my addr 172.21.2.101:0/715968024 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.501 7f189161a700 1 -- 172.21.2.101:0/715968024 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2379592416 0 0) 0x7f188c001500 con 0x7f18940e5570 2021-02-10T11:35:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.501 7f189161a700 1 -- 172.21.2.101:0/715968024 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1880002fe0 con 0x7f18940e5570 2021-02-10T11:35:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f189261c700 1 --2- 172.21.2.101:0/715968024 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18940e5170 0x7f18941b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f189161a700 1 -- 172.21.2.101:0/715968024 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3188057214 0 0) 0x7f188c001460 con 0x7f18940e5570 2021-02-10T11:35:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f189161a700 1 -- 172.21.2.101:0/715968024 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18940e5170 msgr2=0x7f18941b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f189161a700 1 --2- 172.21.2.101:0/715968024 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18940e5170 0x7f18941b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f189161a700 1 -- 172.21.2.101:0/715968024 >> v1:172.21.2.108:6789/0 conn(0x7f18941b3f00 legacy=0x7f1894077510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f189161a700 1 -- 172.21.2.101:0/715968024 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18940788c0 con 0x7f18940e5570 2021-02-10T11:35:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f189161a700 1 -- 172.21.2.101:0/715968024 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f188c001980 con 0x7f18940e5570 2021-02-10T11:35:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f189161a700 1 -- 172.21.2.101:0/715968024 >> v1:172.21.2.101:6789/0 conn(0x7f18940e5570 legacy=0x7f1894077d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f189161a700 1 --2- 172.21.2.101:0/715968024 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1880003a70 0x7f1880005ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f189161a700 1 --2- 172.21.2.101:0/715968024 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1880006780 0x7f1880008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f189161a700 1 --2- 172.21.2.101:0/715968024 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1880009510 0x7f188000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f189161a700 1 -- 172.21.2.101:0/715968024 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188000c2e0 con 0x7f1880009510 2021-02-10T11:35:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f189161a700 1 -- 172.21.2.101:0/715968024 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188000c500 con 0x7f1880003a70 2021-02-10T11:35:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f189161a700 1 -- 172.21.2.101:0/715968024 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188000c720 con 0x7f1880006780 2021-02-10T11:35:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f1892e1d700 1 --2- 172.21.2.101:0/715968024 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1880009510 0x7f188000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f1891e1b700 1 --2- 172.21.2.101:0/715968024 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1880006780 0x7f1880008c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f189261c700 1 --2- 172.21.2.101:0/715968024 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1880003a70 0x7f1880005ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f1892e1d700 1 -- 172.21.2.101:0/715968024 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1880006780 msgr2=0x7f1880008c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f1892e1d700 1 --2- 172.21.2.101:0/715968024 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1880006780 0x7f1880008c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f1892e1d700 1 -- 172.21.2.101:0/715968024 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1880003a70 msgr2=0x7f1880005ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f1892e1d700 1 --2- 172.21.2.101:0/715968024 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1880003a70 0x7f1880005ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f1892e1d700 1 -- 172.21.2.101:0/715968024 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188000c9c0 con 0x7f1880009510 2021-02-10T11:35:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.502 7f1892e1d700 1 --2- 172.21.2.101:0/715968024 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1880009510 0x7f188000ba30 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 rx=0x7f188c001140 tx=0x7f188c04e320).ready entity=mon.0 client_cookie=a955ab3e4966f4ce server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.503 7f189161a700 1 -- 172.21.2.101:0/715968024 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f188c001460 con 0x7f1880009510 2021-02-10T11:35:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.503 7f189161a700 1 -- 172.21.2.101:0/715968024 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c0070c0 con 0x7f1880009510 2021-02-10T11:35:19.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.503 7f189161a700 1 -- 172.21.2.101:0/715968024 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f188c004720 con 0x7f1880009510 2021-02-10T11:35:19.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.503 7f1898994700 1 -- 172.21.2.101:0/715968024 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1880009510 msgr2=0x7f188000ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:19.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.503 7f1898994700 1 --2- 172.21.2.101:0/715968024 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1880009510 0x7f188000ba30 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 rx=0x7f188c001140 tx=0x7f188c04e320).stop 2021-02-10T11:35:19.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.503 7f1892e1d700 1 -- 172.21.2.101:0/715968024 reap_dead start 2021-02-10T11:35:19.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.503 7f1898994700 1 -- 172.21.2.101:0/715968024 shutdown_connections 2021-02-10T11:35:19.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.503 7f1898994700 1 -- 172.21.2.101:0/715968024 >> 172.21.2.101:0/715968024 conn(0x7f18940ce820 msgr2=0x7f18940e4660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:19.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.503 7f1898994700 1 -- 172.21.2.101:0/715968024 shutdown_connections 2021-02-10T11:35:19.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.503 7f1898994700 1 -- 172.21.2.101:0/715968024 wait complete. 2021-02-10T11:35:19.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.518 7f1898994700 1 Processor -- start 2021-02-10T11:35:19.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.518 7f1898994700 1 -- start start 2021-02-10T11:35:19.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.518 7f1898994700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18941ac100 0x7f18941a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:19.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.518 7f1898994700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18941b2e60 0x7f18941944f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:19.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.518 7f1898994700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1894194b10 0x7f1894194f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:19.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.518 7f1898994700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18941a3500 con 0x7f18941ac100 2021-02-10T11:35:19.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.518 7f1898994700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1894092dc0 con 0x7f1894194b10 2021-02-10T11:35:19.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.518 7f1898994700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1894093220 con 0x7f18941b2e60 2021-02-10T11:35:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.518 7f189261c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18941ac100 0x7f18941a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.518 7f189261c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18941ac100 0x7f18941a2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53110/0 (socket says 172.21.2.101:53110) 2021-02-10T11:35:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.518 7f189261c700 1 -- 172.21.2.101:0/4176105459 learned_addr learned my addr 172.21.2.101:0/4176105459 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.519 7f189261c700 1 -- 172.21.2.101:0/4176105459 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18941b2e60 msgr2=0x7f18941944f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.519 7f189261c700 1 --2- 172.21.2.101:0/4176105459 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18941b2e60 0x7f18941944f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.519 7f189261c700 1 -- 172.21.2.101:0/4176105459 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1894194b10 msgr2=0x7f1894194f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.519 7f189261c700 1 --2- 172.21.2.101:0/4176105459 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1894194b10 0x7f1894194f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.519 7f189261c700 1 -- 172.21.2.101:0/4176105459 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189410d8b0 con 0x7f18941ac100 2021-02-10T11:35:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.519 7f189261c700 1 --2- 172.21.2.101:0/4176105459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18941ac100 0x7f18941a2e10 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 rx=0x7f187c006f10 tx=0x7f187c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.519 7f188affd700 1 -- 172.21.2.101:0/4176105459 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f187c00bd80 con 0x7f18941ac100 2021-02-10T11:35:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.519 7f188affd700 1 -- 172.21.2.101:0/4176105459 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187c000d20 con 0x7f18941ac100 2021-02-10T11:35:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.519 7f1898994700 1 -- 172.21.2.101:0/4176105459 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189410d8b0 con 0x7f18941ac100 2021-02-10T11:35:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.519 7f188affd700 1 -- 172.21.2.101:0/4176105459 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f187c002220 con 0x7f18941ac100 2021-02-10T11:35:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.519 7f1898994700 1 -- 172.21.2.101:0/4176105459 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189410de40 con 0x7f18941ac100 2021-02-10T11:35:19.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.520 7f188affd700 1 -- 172.21.2.101:0/4176105459 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f187c009fe0 con 0x7f18941ac100 2021-02-10T11:35:19.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.520 7f188affd700 1 --2- 172.21.2.101:0/4176105459 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1878018730 0x7f187801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:19.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.520 7f188affd700 1 -- 172.21.2.101:0/4176105459 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f187c0327a0 con 0x7f18941ac100 2021-02-10T11:35:19.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.520 7f1891e1b700 1 --2- 172.21.2.101:0/4176105459 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1878018730 0x7f187801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:19.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.520 7f1891e1b700 1 --2- 172.21.2.101:0/4176105459 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1878018730 0x7f187801ac10 crc :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:19.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.551 7f1898994700 1 -- 172.21.2.101:0/4176105459 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189410c9a0 con 0x7f18941ac100 2021-02-10T11:35:19.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.553 7f188affd700 1 -- 172.21.2.101:0/4176105459 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f187c000940 con 0x7f18941ac100 2021-02-10T11:35:19.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.679 7f1898994700 1 -- 172.21.2.101:0/4176105459 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"vol_name": "cephfs", "sub_name": "subvolume_0000000000063820", "target": ["mgr", ""], "tenant_id": "tenant_id", "group_name": "group_0000000000497714", "access_level": "r", "prefix": "fs subvolume authorize", "auth_id": "alice"}) v1 -- 0x7f189410de40 con 0x7f1878018730 2021-02-10T11:35:19.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.718 7f188affd700 1 -- 172.21.2.101:0/4176105459 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7f1884006e70 con 0x7f1878018730 2021-02-10T11:35:19.750 INFO:teuthology.orchestra.run.gibba001.stdout:AQDsxCNgecBnOhAAZSFsyDL12TdycLp/UDwgKQ== 2021-02-10T11:35:19.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.744 7f1898994700 1 -- 172.21.2.101:0/4176105459 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1878018730 msgr2=0x7f187801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:19.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.744 7f1898994700 1 --2- 172.21.2.101:0/4176105459 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1878018730 0x7f187801ac10 crc :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:19.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.744 7f1898994700 1 -- 172.21.2.101:0/4176105459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18941ac100 msgr2=0x7f18941a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:19.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.744 7f1898994700 1 --2- 172.21.2.101:0/4176105459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18941ac100 0x7f18941a2e10 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 rx=0x7f187c006f10 tx=0x7f187c00b990).stop 2021-02-10T11:35:19.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.744 7f1898994700 1 -- 172.21.2.101:0/4176105459 shutdown_connections 2021-02-10T11:35:19.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.744 7f1898994700 1 --2- 172.21.2.101:0/4176105459 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1878018730 0x7f187801ac10 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:19.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.744 7f1898994700 1 --2- 172.21.2.101:0/4176105459 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18941b2e60 0x7f18941944f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:19.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.744 7f1898994700 1 --2- 172.21.2.101:0/4176105459 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1894194b10 0x7f1894194f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:19.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.744 7f1898994700 1 --2- 172.21.2.101:0/4176105459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18941ac100 0x7f18941a2e10 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:19.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.744 7f1898994700 1 -- 172.21.2.101:0/4176105459 >> 172.21.2.101:0/4176105459 conn(0x7f189419c030 msgr2=0x7f189410cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:19.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.744 7f1898994700 1 -- 172.21.2.101:0/4176105459 shutdown_connections 2021-02-10T11:35:19.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:19.744 7f1898994700 1 -- 172.21.2.101:0/4176105459 wait complete. 2021-02-10T11:35:19.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:19.761 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:35:19.761 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:20.096 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:35:20.096 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:35:20.097 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:35:20.097 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:35:20.097 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:20.449 INFO:tasks.cephfs.fuse_mount.ceph-fuse.alice.gibba001.stderr:ceph-fuse[126321]: fuse finished with error 0 and tester_r 0 2021-02-10T11:35:20.451 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:20.451 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:35:20.451 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:20.786 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:35:20.786 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:35:20.786 INFO:teuthology.orchestra.run:waiting for 900 2021-02-10T11:35:26.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:35:26.888 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:27.225 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:35:27.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.627 7f0e7d568700 1 Processor -- start 2021-02-10T11:35:27.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.627 7f0e7d568700 1 -- start start 2021-02-10T11:35:27.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.627 7f0e7d568700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0e780d1ad0 0x7f0e780e10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:27.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.627 7f0e7d568700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e780dfb40 con 0x7f0e781b5c20 2021-02-10T11:35:27.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.627 7f0e7d568700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e780d0f80 con 0x7f0e780d0b80 2021-02-10T11:35:27.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.627 7f0e7d568700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e780f73f0 con 0x7f0e780d1ad0 2021-02-10T11:35:27.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.627 7f0e77ac0700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f0e781b5c20 0x7f0e781b8030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53614/0 (socket says 172.21.2.101:53614) 2021-02-10T11:35:27.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.627 7f0e77ac0700 1 -- 172.21.2.101:0/3423340606 learned_addr learned my addr 172.21.2.101:0/3423340606 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:27.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.628 7f0e762bd700 1 -- 172.21.2.101:0/3423340606 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3558943037 0 0) 0x7f0e70001500 con 0x7f0e781b5c20 2021-02-10T11:35:27.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.628 7f0e762bd700 1 -- 172.21.2.101:0/3423340606 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e64002fe0 con 0x7f0e781b5c20 2021-02-10T11:35:27.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.628 7f0e76abe700 1 --2- 172.21.2.101:0/3423340606 >> v2:172.21.2.108:3300/0 conn(0x7f0e780d1ad0 0x7f0e780e10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:27.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.628 7f0e762bd700 1 -- 172.21.2.101:0/3423340606 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3699244996 0 0) 0x7f0e70001500 con 0x7f0e781b5c20 2021-02-10T11:35:27.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.628 7f0e762bd700 1 -- 172.21.2.101:0/3423340606 >> v2:172.21.2.108:3300/0 conn(0x7f0e780d1ad0 msgr2=0x7f0e780e10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:27.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.628 7f0e762bd700 1 --2- 172.21.2.101:0/3423340606 >> v2:172.21.2.108:3300/0 conn(0x7f0e780d1ad0 0x7f0e780e10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:27.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.628 7f0e762bd700 1 -- 172.21.2.101:0/3423340606 >> v1:172.21.2.108:6789/0 conn(0x7f0e780d0b80 legacy=0x7f0e780d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:27.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.628 7f0e762bd700 1 -- 172.21.2.101:0/3423340606 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e780df760 con 0x7f0e781b5c20 2021-02-10T11:35:27.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.628 7f0e762bd700 1 -- 172.21.2.101:0/3423340606 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f0e70001a00 con 0x7f0e781b5c20 2021-02-10T11:35:27.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.628 7f0e762bd700 1 -- 172.21.2.101:0/3423340606 >> v1:172.21.2.101:6789/0 conn(0x7f0e781b5c20 legacy=0x7f0e781b8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:27.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.628 7f0e762bd700 1 --2- 172.21.2.101:0/3423340606 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e64003b10 0x7f0e64005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:27.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.628 7f0e762bd700 1 --2- 172.21.2.101:0/3423340606 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e64006820 0x7f0e64008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:27.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.628 7f0e762bd700 1 --2- 172.21.2.101:0/3423340606 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e640095b0 0x7f0e6400bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:27.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.628 7f0e762bd700 1 -- 172.21.2.101:0/3423340606 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6400c320 con 0x7f0e64006820 2021-02-10T11:35:27.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.628 7f0e762bd700 1 -- 172.21.2.101:0/3423340606 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6400c540 con 0x7f0e640095b0 2021-02-10T11:35:27.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.628 7f0e762bd700 1 -- 172.21.2.101:0/3423340606 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6400c760 con 0x7f0e64003b10 2021-02-10T11:35:27.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.628 7f0e76abe700 1 --2- 172.21.2.101:0/3423340606 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e64006820 0x7f0e64008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:27.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.629 7f0e77ac0700 1 --2- 172.21.2.101:0/3423340606 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e640095b0 0x7f0e6400bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:27.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.629 7f0e772bf700 1 --2- 172.21.2.101:0/3423340606 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e64003b10 0x7f0e64005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:27.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.629 7f0e76abe700 1 -- 172.21.2.101:0/3423340606 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e64003b10 msgr2=0x7f0e64005f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:27.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.629 7f0e76abe700 1 --2- 172.21.2.101:0/3423340606 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e64003b10 0x7f0e64005f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:27.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.629 7f0e76abe700 1 -- 172.21.2.101:0/3423340606 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e640095b0 msgr2=0x7f0e6400bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:27.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.629 7f0e76abe700 1 --2- 172.21.2.101:0/3423340606 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e640095b0 0x7f0e6400bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:27.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.629 7f0e76abe700 1 -- 172.21.2.101:0/3423340606 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6400ca00 con 0x7f0e64006820 2021-02-10T11:35:27.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.629 7f0e77ac0700 1 -- 172.21.2.101:0/3423340606 reap_dead start 2021-02-10T11:35:27.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.629 7f0e76abe700 1 --2- 172.21.2.101:0/3423340606 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e64006820 0x7f0e64008d40 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 rx=0x7f0e68006a40 tx=0x7f0e6804dbe0).ready entity=mon.0 client_cookie=e21b1439b730ff2f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:27.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.629 7f0e762bd700 1 -- 172.21.2.101:0/3423340606 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e68006540 con 0x7f0e64006820 2021-02-10T11:35:27.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.629 7f0e762bd700 1 -- 172.21.2.101:0/3423340606 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e68002030 con 0x7f0e64006820 2021-02-10T11:35:27.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.629 7f0e762bd700 1 -- 172.21.2.101:0/3423340606 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e68002550 con 0x7f0e64006820 2021-02-10T11:35:27.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.629 7f0e7d568700 1 -- 172.21.2.101:0/3423340606 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e64006820 msgr2=0x7f0e64008d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:27.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.629 7f0e7d568700 1 --2- 172.21.2.101:0/3423340606 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e64006820 0x7f0e64008d40 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 rx=0x7f0e68006a40 tx=0x7f0e6804dbe0).stop 2021-02-10T11:35:27.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.630 7f0e7d568700 1 -- 172.21.2.101:0/3423340606 shutdown_connections 2021-02-10T11:35:27.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.630 7f0e7d568700 1 --2- 172.21.2.101:0/3423340606 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e64006820 0x7f0e64008d40 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:27.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.630 7f0e7d568700 1 -- 172.21.2.101:0/3423340606 >> 172.21.2.101:0/3423340606 conn(0x7f0e780ea6c0 msgr2=0x7f0e780dcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:27.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.630 7f0e7d568700 1 -- 172.21.2.101:0/3423340606 shutdown_connections 2021-02-10T11:35:27.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.630 7f0e7d568700 1 -- 172.21.2.101:0/3423340606 wait complete. 2021-02-10T11:35:27.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.644 7f0e7d568700 1 Processor -- start 2021-02-10T11:35:27.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.644 7f0e7d568700 1 -- start start 2021-02-10T11:35:27.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.645 7f0e7d568700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e780aedb0 0x7f0e780be560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:27.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.645 7f0e7d568700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e780bfbc0 0x7f0e780a8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:27.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.645 7f0e7d568700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e780a94c0 0x7f0e780c0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:27.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.645 7f0e7d568700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e780af230 con 0x7f0e780bfbc0 2021-02-10T11:35:27.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.645 7f0e7d568700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e780bffc0 con 0x7f0e780a94c0 2021-02-10T11:35:27.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.645 7f0e7d568700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e780a98c0 con 0x7f0e780aedb0 2021-02-10T11:35:27.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.645 7f0e76abe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e780bfbc0 0x7f0e780a8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:27.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.645 7f0e76abe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e780bfbc0 0x7f0e780a8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53130/0 (socket says 172.21.2.101:53130) 2021-02-10T11:35:27.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.645 7f0e76abe700 1 -- 172.21.2.101:0/3248512327 learned_addr learned my addr 172.21.2.101:0/3248512327 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:27.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.645 7f0e76abe700 1 -- 172.21.2.101:0/3248512327 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e780aedb0 msgr2=0x7f0e780be560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:27.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.645 7f0e76abe700 1 --2- 172.21.2.101:0/3248512327 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e780aedb0 0x7f0e780be560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:27.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.645 7f0e76abe700 1 -- 172.21.2.101:0/3248512327 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e780a94c0 msgr2=0x7f0e780c0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:27.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.645 7f0e76abe700 1 --2- 172.21.2.101:0/3248512327 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e780a94c0 0x7f0e780c0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:27.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.645 7f0e76abe700 1 -- 172.21.2.101:0/3248512327 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e780c0e80 con 0x7f0e780bfbc0 2021-02-10T11:35:27.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.645 7f0e76abe700 1 --2- 172.21.2.101:0/3248512327 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e780bfbc0 0x7f0e780a8ea0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 rx=0x7f0e6804e670 tx=0x7f0e680508f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:27.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.646 7f0e6ffff700 1 -- 172.21.2.101:0/3248512327 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e68000c90 con 0x7f0e780bfbc0 2021-02-10T11:35:27.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.646 7f0e6ffff700 1 -- 172.21.2.101:0/3248512327 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e68002030 con 0x7f0e780bfbc0 2021-02-10T11:35:27.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.646 7f0e6ffff700 1 -- 172.21.2.101:0/3248512327 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e68002550 con 0x7f0e780bfbc0 2021-02-10T11:35:27.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.646 7f0e7d568700 1 -- 172.21.2.101:0/3248512327 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e780c0e80 con 0x7f0e780bfbc0 2021-02-10T11:35:27.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.646 7f0e7d568700 1 -- 172.21.2.101:0/3248512327 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e780c1410 con 0x7f0e780bfbc0 2021-02-10T11:35:27.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.646 7f0e6ffff700 1 -- 172.21.2.101:0/3248512327 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0e68000c90 con 0x7f0e780bfbc0 2021-02-10T11:35:27.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.646 7f0e6ffff700 1 --2- 172.21.2.101:0/3248512327 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e5c0186c0 0x7f0e5c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:27.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.646 7f0e6ffff700 1 -- 172.21.2.101:0/3248512327 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0e68076440 con 0x7f0e780bfbc0 2021-02-10T11:35:27.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.647 7f0e772bf700 1 --2- 172.21.2.101:0/3248512327 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e5c0186c0 0x7f0e5c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:27.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.647 7f0e772bf700 1 --2- 172.21.2.101:0/3248512327 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e5c0186c0 0x7f0e5c01abc0 crc :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:27.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.677 7f0e7d568700 1 -- 172.21.2.101:0/3248512327 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e7807b2c0 con 0x7f0e780bfbc0 2021-02-10T11:35:27.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.680 7f0e6ffff700 1 -- 172.21.2.101:0/3248512327 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0e68074d70 con 0x7f0e780bfbc0 2021-02-10T11:35:27.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.804 7f0e7d568700 1 -- 172.21.2.101:0/3248512327 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e780dfb40 con 0x7f0e780bfbc0 2021-02-10T11:35:27.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.805 7f0e6ffff700 1 -- 172.21.2.101:0/3248512327 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+13996 (secure 0 0 0) 0x7f0e68000c90 con 0x7f0e780bfbc0 2021-02-10T11:35:27.813 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:27.813 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":65,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:34:42.090329","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:34:38.607957","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2021-02-10 11:34:40.084211","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:35:27.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.806 7f0e7d568700 1 -- 172.21.2.101:0/3248512327 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e5c0186c0 msgr2=0x7f0e5c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:27.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.806 7f0e7d568700 1 --2- 172.21.2.101:0/3248512327 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e5c0186c0 0x7f0e5c01abc0 crc :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:27.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.806 7f0e7d568700 1 -- 172.21.2.101:0/3248512327 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e780bfbc0 msgr2=0x7f0e780a8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:27.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.806 7f0e7d568700 1 --2- 172.21.2.101:0/3248512327 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e780bfbc0 0x7f0e780a8ea0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 rx=0x7f0e6804e670 tx=0x7f0e680508f0).stop 2021-02-10T11:35:27.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.806 7f0e7d568700 1 -- 172.21.2.101:0/3248512327 shutdown_connections 2021-02-10T11:35:27.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.806 7f0e7d568700 1 --2- 172.21.2.101:0/3248512327 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e780a94c0 0x7f0e780c0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:27.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.806 7f0e7d568700 1 --2- 172.21.2.101:0/3248512327 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e780bfbc0 0x7f0e780a8ea0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:27.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.806 7f0e7d568700 1 --2- 172.21.2.101:0/3248512327 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e5c0186c0 0x7f0e5c01abc0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:27.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.806 7f0e7d568700 1 --2- 172.21.2.101:0/3248512327 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e780aedb0 0x7f0e780be560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:27.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.806 7f0e7d568700 1 -- 172.21.2.101:0/3248512327 >> 172.21.2.101:0/3248512327 conn(0x7f0e7809e280 msgr2=0x7f0e780aaac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:27.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.807 7f0e7d568700 1 -- 172.21.2.101:0/3248512327 shutdown_connections 2021-02-10T11:35:27.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:27.807 7f0e7d568700 1 -- 172.21.2.101:0/3248512327 wait complete. 2021-02-10T11:35:27.823 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:34:38.607957', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:34:40.084211', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:35:27.823 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:35:28.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.228 7f9ee18bd700 1 Processor -- start 2021-02-10T11:35:28.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.228 7f9ee18bd700 1 -- start start 2021-02-10T11:35:28.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.229 7f9ee18bd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9edc0e5570 0x7f9edc078520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:28.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.229 7f9ee18bd700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9edc078b90 con 0x7f9edc079110 2021-02-10T11:35:28.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.229 7f9ee18bd700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9edc1b40d0 con 0x7f9edc0e5170 2021-02-10T11:35:28.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.229 7f9ee18bd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc0d1900 con 0x7f9edc0e5570 2021-02-10T11:35:28.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.229 7f9edbd75700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f9edc079110 0x7f9edc079510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53628/0 (socket says 172.21.2.101:53628) 2021-02-10T11:35:28.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.229 7f9edbd75700 1 -- 172.21.2.101:0/4120585700 learned_addr learned my addr 172.21.2.101:0/4120585700 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:28.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.229 7f9ed2d73700 1 --2- 172.21.2.101:0/4120585700 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9edc0e5570 0x7f9edc078520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:28.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.229 7f9edad73700 1 -- 172.21.2.101:0/4120585700 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4204607409 0 0) 0x7f9ed4001180 con 0x7f9edc079110 2021-02-10T11:35:28.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.230 7f9edad73700 1 -- 172.21.2.101:0/4120585700 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ec8002fe0 con 0x7f9edc079110 2021-02-10T11:35:28.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.230 7f9edad73700 1 -- 172.21.2.101:0/4120585700 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 988166917 0 0) 0x7f9ecc001400 con 0x7f9edc0e5170 2021-02-10T11:35:28.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.230 7f9edad73700 1 -- 172.21.2.101:0/4120585700 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ec8003210 con 0x7f9edc0e5170 2021-02-10T11:35:28.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.230 7f9ed2d73700 1 -- 172.21.2.101:0/4120585700 >> v1:172.21.2.108:6789/0 conn(0x7f9edc0e5170 legacy=0x7f9edc1b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:28.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.230 7f9ed2d73700 1 -- 172.21.2.101:0/4120585700 >> v1:172.21.2.101:6789/0 conn(0x7f9edc079110 legacy=0x7f9edc079510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:28.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.230 7f9ed2d73700 1 -- 172.21.2.101:0/4120585700 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9edc0d1ae0 con 0x7f9edc0e5570 2021-02-10T11:35:28.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.230 7f9ed2d73700 1 --2- 172.21.2.101:0/4120585700 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9edc0e5570 0x7f9edc078520 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f9ec4006650 tx=0x7f9ec4009a80).ready entity=mon.2 client_cookie=70146e6ada6a6724 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:28.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.230 7f9edad73700 1 -- 172.21.2.101:0/4120585700 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9ec40009b0 con 0x7f9edc0e5570 2021-02-10T11:35:28.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.230 7f9edad73700 1 -- 172.21.2.101:0/4120585700 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec400c0c0 con 0x7f9edc0e5570 2021-02-10T11:35:28.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.230 7f9edad73700 1 -- 172.21.2.101:0/4120585700 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9ec400c5e0 con 0x7f9edc0e5570 2021-02-10T11:35:28.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.230 7f9ee18bd700 1 -- 172.21.2.101:0/4120585700 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9edc0e5570 msgr2=0x7f9edc078520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:28.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.230 7f9ee18bd700 1 --2- 172.21.2.101:0/4120585700 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9edc0e5570 0x7f9edc078520 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f9ec4006650 tx=0x7f9ec4009a80).stop 2021-02-10T11:35:28.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.231 7f9ee18bd700 1 -- 172.21.2.101:0/4120585700 shutdown_connections 2021-02-10T11:35:28.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.231 7f9ee18bd700 1 --2- 172.21.2.101:0/4120585700 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9edc0e5570 0x7f9edc078520 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:28.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.231 7f9ee18bd700 1 -- 172.21.2.101:0/4120585700 >> 172.21.2.101:0/4120585700 conn(0x7f9edc0ce820 msgr2=0x7f9edc0e4660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:28.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.231 7f9ee18bd700 1 -- 172.21.2.101:0/4120585700 shutdown_connections 2021-02-10T11:35:28.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.231 7f9ee18bd700 1 -- 172.21.2.101:0/4120585700 wait complete. 2021-02-10T11:35:28.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.245 7f9ee18bd700 1 Processor -- start 2021-02-10T11:35:28.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.246 7f9ee18bd700 1 -- start start 2021-02-10T11:35:28.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.246 7f9ee18bd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9edc109170 0x7f9edc109570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:28.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.246 7f9ee18bd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9edc08b240 0x7f9edc08b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:28.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.246 7f9ee18bd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9edc08bc60 0x7f9edc0c0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:28.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.246 7f9ee18bd700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc08c060 con 0x7f9edc109170 2021-02-10T11:35:28.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.246 7f9ee18bd700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc09dc60 con 0x7f9edc08b240 2021-02-10T11:35:28.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.246 7f9ee18bd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc196000 con 0x7f9edc08bc60 2021-02-10T11:35:28.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.246 7f9edb574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9edc109170 0x7f9edc109570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:28.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.246 7f9edb574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9edc109170 0x7f9edc109570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53140/0 (socket says 172.21.2.101:53140) 2021-02-10T11:35:28.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.246 7f9edb574700 1 -- 172.21.2.101:0/2462447980 learned_addr learned my addr 172.21.2.101:0/2462447980 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:28.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.246 7f9ed2d73700 1 --2- 172.21.2.101:0/2462447980 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9edc08b240 0x7f9edc08b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:28.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.246 7f9edbd75700 1 --2- 172.21.2.101:0/2462447980 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9edc08bc60 0x7f9edc0c0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:28.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.247 7f9edb574700 1 -- 172.21.2.101:0/2462447980 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9edc08bc60 msgr2=0x7f9edc0c0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:28.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.247 7f9edb574700 1 --2- 172.21.2.101:0/2462447980 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9edc08bc60 0x7f9edc0c0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:28.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.247 7f9edb574700 1 -- 172.21.2.101:0/2462447980 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9edc08b240 msgr2=0x7f9edc08b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:28.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.247 7f9edb574700 1 --2- 172.21.2.101:0/2462447980 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9edc08b240 0x7f9edc08b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:28.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.247 7f9edb574700 1 -- 172.21.2.101:0/2462447980 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9edc0c11b0 con 0x7f9edc109170 2021-02-10T11:35:28.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.247 7f9edb574700 1 --2- 172.21.2.101:0/2462447980 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9edc109170 0x7f9edc109570 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 rx=0x7f9ecc009980 tx=0x7f9ecc00bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:28.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.247 7f9ed8d6f700 1 -- 172.21.2.101:0/2462447980 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9ecc003020 con 0x7f9edc109170 2021-02-10T11:35:28.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.247 7f9ed8d6f700 1 -- 172.21.2.101:0/2462447980 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ecc005870 con 0x7f9edc109170 2021-02-10T11:35:28.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.247 7f9ee18bd700 1 -- 172.21.2.101:0/2462447980 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9edc0c11b0 con 0x7f9edc109170 2021-02-10T11:35:28.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.247 7f9ee18bd700 1 -- 172.21.2.101:0/2462447980 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9edc0c1740 con 0x7f9edc109170 2021-02-10T11:35:28.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.247 7f9ed8d6f700 1 -- 172.21.2.101:0/2462447980 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9ecc005d50 con 0x7f9edc109170 2021-02-10T11:35:28.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.248 7f9ed8d6f700 1 -- 172.21.2.101:0/2462447980 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9ecc005a50 con 0x7f9edc109170 2021-02-10T11:35:28.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.248 7f9ed8d6f700 1 --2- 172.21.2.101:0/2462447980 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ec0018730 0x7f9ec001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:28.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.248 7f9ed8d6f700 1 -- 172.21.2.101:0/2462447980 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9ecc032b70 con 0x7f9edc109170 2021-02-10T11:35:28.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.248 7f9ed2d73700 1 --2- 172.21.2.101:0/2462447980 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ec0018730 0x7f9ec001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:28.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.248 7f9ed2d73700 1 --2- 172.21.2.101:0/2462447980 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ec0018730 0x7f9ec001ac10 crc :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:28.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.279 7f9ee18bd700 1 -- 172.21.2.101:0/2462447980 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9edc105c40 con 0x7f9edc109170 2021-02-10T11:35:28.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.281 7f9ed8d6f700 1 -- 172.21.2.101:0/2462447980 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9ecc000f40 con 0x7f9edc109170 2021-02-10T11:35:28.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.412 7f9ee18bd700 1 -- 172.21.2.101:0/2462447980 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f9edc08c060 con 0x7f9edc109170 2021-02-10T11:35:28.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.412 7f9ed8d6f700 1 -- 172.21.2.101:0/2462447980 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9ecc005870 con 0x7f9edc109170 2021-02-10T11:35:28.421 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:35:28.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.414 7f9ee18bd700 1 -- 172.21.2.101:0/2462447980 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ec0018730 msgr2=0x7f9ec001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:28.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.414 7f9ee18bd700 1 --2- 172.21.2.101:0/2462447980 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ec0018730 0x7f9ec001ac10 crc :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:28.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.414 7f9ee18bd700 1 -- 172.21.2.101:0/2462447980 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9edc109170 msgr2=0x7f9edc109570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:28.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.414 7f9ee18bd700 1 --2- 172.21.2.101:0/2462447980 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9edc109170 0x7f9edc109570 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 rx=0x7f9ecc009980 tx=0x7f9ecc00bcd0).stop 2021-02-10T11:35:28.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.414 7f9ee18bd700 1 -- 172.21.2.101:0/2462447980 shutdown_connections 2021-02-10T11:35:28.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.414 7f9ee18bd700 1 --2- 172.21.2.101:0/2462447980 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ec0018730 0x7f9ec001ac10 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:28.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.414 7f9ee18bd700 1 --2- 172.21.2.101:0/2462447980 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9edc08bc60 0x7f9edc0c0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:28.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.414 7f9ee18bd700 1 --2- 172.21.2.101:0/2462447980 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9edc08b240 0x7f9edc08b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:28.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.414 7f9ee18bd700 1 --2- 172.21.2.101:0/2462447980 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9edc109170 0x7f9edc109570 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:28.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.414 7f9ee18bd700 1 -- 172.21.2.101:0/2462447980 >> 172.21.2.101:0/2462447980 conn(0x7f9edc0a3c70 msgr2=0x7f9edc08a540 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:28.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.414 7f9ee18bd700 1 -- 172.21.2.101:0/2462447980 shutdown_connections 2021-02-10T11:35:28.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.414 7f9ee18bd700 1 -- 172.21.2.101:0/2462447980 wait complete. 2021-02-10T11:35:28.431 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:35:28.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.836 7f9bcfe91700 1 Processor -- start 2021-02-10T11:35:28.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.836 7f9bcfe91700 1 -- start start 2021-02-10T11:35:28.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.837 7f9bcfe91700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9bc80e5170 0x7f9bc81b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:28.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.837 7f9bcfe91700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9bc81b3f00 0x7f9bc8077510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:28.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.837 7f9bcfe91700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc8078b90 con 0x7f9bc80e5570 2021-02-10T11:35:28.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.837 7f9bcfe91700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bc807a120 con 0x7f9bc81b3f00 2021-02-10T11:35:28.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.837 7f9bcfe91700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc807a310 con 0x7f9bc80e5170 2021-02-10T11:35:28.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.837 7f9bcd4ac700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9bc81b3f00 0x7f9bc8077510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:28.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.837 7f9bcd4ac700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9bc81b3f00 0x7f9bc8077510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53146/0 (socket says 172.21.2.101:53146) 2021-02-10T11:35:28.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.837 7f9bce4ae700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f9bc80e5570 0x7f9bc8077b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53644/0 (socket says 172.21.2.101:53644) 2021-02-10T11:35:28.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.837 7f9bcd4ac700 1 -- 172.21.2.101:0/1391997861 learned_addr learned my addr 172.21.2.101:0/1391997861 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:28.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.837 7f9bcd4ac700 1 -- 172.21.2.101:0/1391997861 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9bc80e5170 msgr2=0x7f9bc81b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:28.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.837 7f9bcd4ac700 1 --2- 172.21.2.101:0/1391997861 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9bc80e5170 0x7f9bc81b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:28.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.837 7f9bcd4ac700 1 -- 172.21.2.101:0/1391997861 >> v1:172.21.2.101:6789/0 conn(0x7f9bc80e5570 legacy=0x7f9bc8077b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:28.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.837 7f9bcd4ac700 1 -- 172.21.2.101:0/1391997861 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc807a500 con 0x7f9bc81b3f00 2021-02-10T11:35:28.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.838 7f9bcd4ac700 1 --2- 172.21.2.101:0/1391997861 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9bc81b3f00 0x7f9bc8077510 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 rx=0x7f9bc4006700 tx=0x7f9bc4009b80).ready entity=mon.0 client_cookie=6b128ee8727ffec8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:28.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.838 7f9bcccab700 1 -- 172.21.2.101:0/1391997861 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9bc400c0c0 con 0x7f9bc81b3f00 2021-02-10T11:35:28.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.838 7f9bcccab700 1 -- 172.21.2.101:0/1391997861 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc400c340 con 0x7f9bc81b3f00 2021-02-10T11:35:28.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.838 7f9bcccab700 1 -- 172.21.2.101:0/1391997861 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9bc400c860 con 0x7f9bc81b3f00 2021-02-10T11:35:28.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.838 7f9bcfe91700 1 -- 172.21.2.101:0/1391997861 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9bc81b3f00 msgr2=0x7f9bc8077510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:28.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.838 7f9bcfe91700 1 --2- 172.21.2.101:0/1391997861 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9bc81b3f00 0x7f9bc8077510 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 rx=0x7f9bc4006700 tx=0x7f9bc4009b80).stop 2021-02-10T11:35:28.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.838 7f9bcfe91700 1 -- 172.21.2.101:0/1391997861 shutdown_connections 2021-02-10T11:35:28.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.838 7f9bcfe91700 1 --2- 172.21.2.101:0/1391997861 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9bc81b3f00 0x7f9bc8077510 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:28.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.838 7f9bcfe91700 1 --2- 172.21.2.101:0/1391997861 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9bc80e5170 0x7f9bc81b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:28.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.838 7f9bcfe91700 1 -- 172.21.2.101:0/1391997861 >> 172.21.2.101:0/1391997861 conn(0x7f9bc80ce820 msgr2=0x7f9bc80e4660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:28.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.839 7f9bcfe91700 1 -- 172.21.2.101:0/1391997861 shutdown_connections 2021-02-10T11:35:28.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.839 7f9bcfe91700 1 -- 172.21.2.101:0/1391997861 wait complete. 2021-02-10T11:35:28.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.853 7f9bcfe91700 1 Processor -- start 2021-02-10T11:35:28.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.853 7f9bcfe91700 1 -- start start 2021-02-10T11:35:28.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.854 7f9bcfe91700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9bc80b80e0 0x7f9bc80c34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:28.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.854 7f9bcfe91700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9bc8109e80 0x7f9bc810a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:28.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.854 7f9bcfe91700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9bc810a8a0 0x7f9bc810b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:28.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.854 7f9bcfe91700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc810ae60 con 0x7f9bc8109e80 2021-02-10T11:35:28.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.854 7f9bcfe91700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80b7c40 con 0x7f9bc810a8a0 2021-02-10T11:35:28.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.854 7f9bcfe91700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80b7e20 con 0x7f9bc80b80e0 2021-02-10T11:35:28.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.854 7f9bcd4ac700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9bc8109e80 0x7f9bc810a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:28.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.854 7f9bcd4ac700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9bc8109e80 0x7f9bc810a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53152/0 (socket says 172.21.2.101:53152) 2021-02-10T11:35:28.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.854 7f9bcd4ac700 1 -- 172.21.2.101:0/923154707 learned_addr learned my addr 172.21.2.101:0/923154707 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:28.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.854 7f9bcd4ac700 1 -- 172.21.2.101:0/923154707 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9bc80b80e0 msgr2=0x7f9bc80c34a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:35:28.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.854 7f9bcd4ac700 1 --2- 172.21.2.101:0/923154707 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9bc80b80e0 0x7f9bc80c34a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:28.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.854 7f9bcd4ac700 1 -- 172.21.2.101:0/923154707 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9bc810a8a0 msgr2=0x7f9bc810b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:35:28.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.854 7f9bcd4ac700 1 --2- 172.21.2.101:0/923154707 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9bc810a8a0 0x7f9bc810b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:28.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.854 7f9bcd4ac700 1 -- 172.21.2.101:0/923154707 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc810ba00 con 0x7f9bc8109e80 2021-02-10T11:35:28.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.854 7f9bcd4ac700 1 --2- 172.21.2.101:0/923154707 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9bc8109e80 0x7f9bc810a280 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 rx=0x7f9bc400b650 tx=0x7f9bc400b9f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:28.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.854 7f9bba7fc700 1 -- 172.21.2.101:0/923154707 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9bc400bde0 con 0x7f9bc8109e80 2021-02-10T11:35:28.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.855 7f9bba7fc700 1 -- 172.21.2.101:0/923154707 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc4002030 con 0x7f9bc8109e80 2021-02-10T11:35:28.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.855 7f9bba7fc700 1 -- 172.21.2.101:0/923154707 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9bc4002550 con 0x7f9bc8109e80 2021-02-10T11:35:28.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.855 7f9bcfe91700 1 -- 172.21.2.101:0/923154707 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc810ba00 con 0x7f9bc8109e80 2021-02-10T11:35:28.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.855 7f9bcfe91700 1 -- 172.21.2.101:0/923154707 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc810ba00 con 0x7f9bc8109e80 2021-02-10T11:35:28.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.855 7f9bba7fc700 1 -- 172.21.2.101:0/923154707 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9bc4000c90 con 0x7f9bc8109e80 2021-02-10T11:35:28.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.855 7f9bba7fc700 1 --2- 172.21.2.101:0/923154707 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9bb00186c0 0x7f9bb001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:28.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.855 7f9bba7fc700 1 -- 172.21.2.101:0/923154707 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9bc4032440 con 0x7f9bc8109e80 2021-02-10T11:35:28.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.855 7f9bcdcad700 1 --2- 172.21.2.101:0/923154707 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9bb00186c0 0x7f9bb001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:28.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.856 7f9bcdcad700 1 --2- 172.21.2.101:0/923154707 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9bb00186c0 0x7f9bb001abc0 crc :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:28.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.887 7f9bcfe91700 1 -- 172.21.2.101:0/923154707 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc8105c50 con 0x7f9bc8109e80 2021-02-10T11:35:28.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:28.889 7f9bba7fc700 1 -- 172.21.2.101:0/923154707 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9bc402e0b0 con 0x7f9bc8109e80 2021-02-10T11:35:29.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.018 7f9bcfe91700 1 -- 172.21.2.101:0/923154707 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f9bc80fe290 con 0x7f9bc8109e80 2021-02-10T11:35:29.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.019 7f9bba7fc700 1 -- 172.21.2.101:0/923154707 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v65) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9bc402e0b0 con 0x7f9bc8109e80 2021-02-10T11:35:29.027 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:35:29.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.021 7f9bcfe91700 1 -- 172.21.2.101:0/923154707 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9bb00186c0 msgr2=0x7f9bb001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:29.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.021 7f9bcfe91700 1 --2- 172.21.2.101:0/923154707 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9bb00186c0 0x7f9bb001abc0 crc :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:29.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.021 7f9bcfe91700 1 -- 172.21.2.101:0/923154707 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9bc8109e80 msgr2=0x7f9bc810a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:29.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.021 7f9bcfe91700 1 --2- 172.21.2.101:0/923154707 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9bc8109e80 0x7f9bc810a280 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 rx=0x7f9bc400b650 tx=0x7f9bc400b9f0).stop 2021-02-10T11:35:29.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.021 7f9bcfe91700 1 -- 172.21.2.101:0/923154707 shutdown_connections 2021-02-10T11:35:29.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.021 7f9bcfe91700 1 --2- 172.21.2.101:0/923154707 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9bc810a8a0 0x7f9bc810b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:29.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.021 7f9bcfe91700 1 --2- 172.21.2.101:0/923154707 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9bc8109e80 0x7f9bc810a280 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:29.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.021 7f9bcfe91700 1 --2- 172.21.2.101:0/923154707 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9bb00186c0 0x7f9bb001abc0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:29.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.021 7f9bcfe91700 1 --2- 172.21.2.101:0/923154707 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9bc80b80e0 0x7f9bc80c34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:29.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.021 7f9bcfe91700 1 -- 172.21.2.101:0/923154707 >> 172.21.2.101:0/923154707 conn(0x7f9bc809f300 msgr2=0x7f9bc80c3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:29.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.021 7f9bcfe91700 1 -- 172.21.2.101:0/923154707 shutdown_connections 2021-02-10T11:35:29.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.021 7f9bcfe91700 1 -- 172.21.2.101:0/923154707 wait complete. 2021-02-10T11:35:29.038 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:35:29.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.443 7efde6349700 1 Processor -- start 2021-02-10T11:35:29.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.443 7efde6349700 1 -- start start 2021-02-10T11:35:29.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.444 7efde6349700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efde00e5170 0x7efde01b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:29.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.444 7efde6349700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7efde01b3f00 0x7efde0077510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:29.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.444 7efde6349700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde0078b90 con 0x7efde00e5570 2021-02-10T11:35:29.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.444 7efde6349700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efde007a120 con 0x7efde01b3f00 2021-02-10T11:35:29.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.444 7efde6349700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde007a310 con 0x7efde00e5170 2021-02-10T11:35:29.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.444 7efde4966700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7efde00e5570 0x7efde0077b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53654/0 (socket says 172.21.2.101:53654) 2021-02-10T11:35:29.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.444 7efde4966700 1 -- 172.21.2.101:0/2170796860 learned_addr learned my addr 172.21.2.101:0/2170796860 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:29.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.444 7efddeffd700 1 -- 172.21.2.101:0/2170796860 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599854016 0 0) 0x7efdd8001500 con 0x7efde00e5570 2021-02-10T11:35:29.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddeffd700 1 -- 172.21.2.101:0/2170796860 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdc8002fe0 con 0x7efde00e5570 2021-02-10T11:35:29.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddf7fe700 1 --2- 172.21.2.101:0/2170796860 >> v2:172.21.2.108:3300/0 conn(0x7efde01b3f00 0x7efde0077510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:29.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddffff700 1 --2- 172.21.2.101:0/2170796860 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efde00e5170 0x7efde01b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:29.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddeffd700 1 -- 172.21.2.101:0/2170796860 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 508571254 0 0) 0x7efdd8001460 con 0x7efde00e5570 2021-02-10T11:35:29.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddeffd700 1 -- 172.21.2.101:0/2170796860 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efde00e5170 msgr2=0x7efde01b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:29.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddeffd700 1 --2- 172.21.2.101:0/2170796860 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efde00e5170 0x7efde01b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:29.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddeffd700 1 -- 172.21.2.101:0/2170796860 >> v2:172.21.2.108:3300/0 conn(0x7efde01b3f00 msgr2=0x7efde0077510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:29.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddeffd700 1 --2- 172.21.2.101:0/2170796860 >> v2:172.21.2.108:3300/0 conn(0x7efde01b3f00 0x7efde0077510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:29.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddeffd700 1 -- 172.21.2.101:0/2170796860 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde007a500 con 0x7efde00e5570 2021-02-10T11:35:29.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddeffd700 1 -- 172.21.2.101:0/2170796860 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7efdd8001980 con 0x7efde00e5570 2021-02-10T11:35:29.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddeffd700 1 -- 172.21.2.101:0/2170796860 >> v1:172.21.2.101:6789/0 conn(0x7efde00e5570 legacy=0x7efde0077b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:29.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddeffd700 1 --2- 172.21.2.101:0/2170796860 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efdc8003a60 0x7efdc8005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:29.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddeffd700 1 --2- 172.21.2.101:0/2170796860 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efdc804a7a0 0x7efdc804cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:29.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddeffd700 1 --2- 172.21.2.101:0/2170796860 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efdc804d510 0x7efdc804fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:29.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddeffd700 1 -- 172.21.2.101:0/2170796860 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc80502e0 con 0x7efdc8003a60 2021-02-10T11:35:29.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddeffd700 1 -- 172.21.2.101:0/2170796860 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc8050500 con 0x7efdc804d510 2021-02-10T11:35:29.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddeffd700 1 -- 172.21.2.101:0/2170796860 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc8050720 con 0x7efdc804a7a0 2021-02-10T11:35:29.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddffff700 1 --2- 172.21.2.101:0/2170796860 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efdc8003a60 0x7efdc8005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:29.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddf7fe700 1 --2- 172.21.2.101:0/2170796860 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efdc804a7a0 0x7efdc804cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:29.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efde4966700 1 --2- 172.21.2.101:0/2170796860 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efdc804d510 0x7efdc804fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:29.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddffff700 1 -- 172.21.2.101:0/2170796860 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efdc804a7a0 msgr2=0x7efdc804cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:29.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddffff700 1 --2- 172.21.2.101:0/2170796860 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efdc804a7a0 0x7efdc804cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:29.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddffff700 1 -- 172.21.2.101:0/2170796860 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efdc804d510 msgr2=0x7efdc804fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:29.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.445 7efddffff700 1 --2- 172.21.2.101:0/2170796860 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efdc804d510 0x7efdc804fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:29.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.446 7efddffff700 1 -- 172.21.2.101:0/2170796860 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc80509c0 con 0x7efdc8003a60 2021-02-10T11:35:29.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.446 7efddffff700 1 --2- 172.21.2.101:0/2170796860 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efdc8003a60 0x7efdc8005ef0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 rx=0x7efdd0006ac0 tx=0x7efdd0009ab0).ready entity=mon.0 client_cookie=c86075c64829bb42 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:29.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.446 7efddeffd700 1 -- 172.21.2.101:0/2170796860 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efdd000c0c0 con 0x7efdc8003a60 2021-02-10T11:35:29.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.446 7efddeffd700 1 -- 172.21.2.101:0/2170796860 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd000c3c0 con 0x7efdc8003a60 2021-02-10T11:35:29.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.446 7efddeffd700 1 -- 172.21.2.101:0/2170796860 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efdd000c8a0 con 0x7efdc8003a60 2021-02-10T11:35:29.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.446 7efde6349700 1 -- 172.21.2.101:0/2170796860 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efdc8003a60 msgr2=0x7efdc8005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:29.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.446 7efde6349700 1 --2- 172.21.2.101:0/2170796860 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efdc8003a60 0x7efdc8005ef0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 rx=0x7efdd0006ac0 tx=0x7efdd0009ab0).stop 2021-02-10T11:35:29.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.446 7efde4966700 1 -- 172.21.2.101:0/2170796860 reap_dead start 2021-02-10T11:35:29.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.446 7efde6349700 1 -- 172.21.2.101:0/2170796860 shutdown_connections 2021-02-10T11:35:29.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.446 7efde6349700 1 -- 172.21.2.101:0/2170796860 >> 172.21.2.101:0/2170796860 conn(0x7efde00ce820 msgr2=0x7efde00e4660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:29.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.447 7efde6349700 1 -- 172.21.2.101:0/2170796860 shutdown_connections 2021-02-10T11:35:29.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.447 7efde6349700 1 -- 172.21.2.101:0/2170796860 wait complete. 2021-02-10T11:35:29.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.461 7efde6349700 1 Processor -- start 2021-02-10T11:35:29.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.461 7efde6349700 1 -- start start 2021-02-10T11:35:29.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.462 7efde6349700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efde0188d50 0x7efde010cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:29.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.462 7efde6349700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efde018b760 0x7efde010d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:29.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.462 7efde6349700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efde010d810 0x7efde00bc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:29.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.462 7efde6349700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde00bc8f0 con 0x7efde010d810 2021-02-10T11:35:29.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.462 7efde6349700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde00bcb10 con 0x7efde0188d50 2021-02-10T11:35:29.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.462 7efde6349700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde00bcd30 con 0x7efde018b760 2021-02-10T11:35:29.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.462 7efde4966700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efde010d810 0x7efde00bc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:29.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.462 7efde4966700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efde010d810 0x7efde00bc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53172/0 (socket says 172.21.2.101:53172) 2021-02-10T11:35:29.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.462 7efde4966700 1 -- 172.21.2.101:0/663692275 learned_addr learned my addr 172.21.2.101:0/663692275 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:29.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.462 7efde4966700 1 -- 172.21.2.101:0/663692275 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efde018b760 msgr2=0x7efde010d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:29.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.462 7efde4966700 1 --2- 172.21.2.101:0/663692275 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efde018b760 0x7efde010d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:29.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.462 7efde4966700 1 -- 172.21.2.101:0/663692275 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efde0188d50 msgr2=0x7efde010cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:29.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.462 7efde4966700 1 --2- 172.21.2.101:0/663692275 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efde0188d50 0x7efde010cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:29.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.462 7efde4966700 1 -- 172.21.2.101:0/663692275 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde00bcf50 con 0x7efde010d810 2021-02-10T11:35:29.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.463 7efde4966700 1 --2- 172.21.2.101:0/663692275 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efde010d810 0x7efde00bc2d0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 rx=0x7efdd800a940 tx=0x7efdd800ace0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:29.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.463 7efddcff9700 1 -- 172.21.2.101:0/663692275 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efdd8007020 con 0x7efde010d810 2021-02-10T11:35:29.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.463 7efddcff9700 1 -- 172.21.2.101:0/663692275 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd8001780 con 0x7efde010d810 2021-02-10T11:35:29.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.463 7efde6349700 1 -- 172.21.2.101:0/663692275 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde00bcf50 con 0x7efde010d810 2021-02-10T11:35:29.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.463 7efde6349700 1 -- 172.21.2.101:0/663692275 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde00bd4e0 con 0x7efde010d810 2021-02-10T11:35:29.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.463 7efddcff9700 1 -- 172.21.2.101:0/663692275 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efdd8007520 con 0x7efde010d810 2021-02-10T11:35:29.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.463 7efddcff9700 1 -- 172.21.2.101:0/663692275 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7efdd800ae20 con 0x7efde010d810 2021-02-10T11:35:29.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.463 7efddcff9700 1 --2- 172.21.2.101:0/663692275 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efdc00186d0 0x7efdc001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:29.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.463 7efddcff9700 1 -- 172.21.2.101:0/663692275 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7efdd8032b40 con 0x7efde010d810 2021-02-10T11:35:29.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.464 7efddffff700 1 --2- 172.21.2.101:0/663692275 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efdc00186d0 0x7efdc001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:29.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.464 7efddffff700 1 --2- 172.21.2.101:0/663692275 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efdc00186d0 0x7efdc001abd0 crc :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:29.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.494 7efde6349700 1 -- 172.21.2.101:0/663692275 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde010c9a0 con 0x7efde010d810 2021-02-10T11:35:29.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.496 7efddcff9700 1 -- 172.21.2.101:0/663692275 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7efdd8001780 con 0x7efde010d810 2021-02-10T11:35:29.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.622 7efde6349700 1 -- 172.21.2.101:0/663692275 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7efde00fe290 con 0x7efde010d810 2021-02-10T11:35:29.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.623 7efddcff9700 1 -- 172.21.2.101:0/663692275 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v65) v1 ==== 100+0+10 (secure 0 0 0) 0x7efdd8001780 con 0x7efde010d810 2021-02-10T11:35:29.631 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:35:29.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.625 7efde6349700 1 -- 172.21.2.101:0/663692275 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efdc00186d0 msgr2=0x7efdc001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:29.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.625 7efde6349700 1 --2- 172.21.2.101:0/663692275 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efdc00186d0 0x7efdc001abd0 crc :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:29.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.625 7efde6349700 1 -- 172.21.2.101:0/663692275 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efde010d810 msgr2=0x7efde00bc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:29.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.625 7efde6349700 1 --2- 172.21.2.101:0/663692275 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efde010d810 0x7efde00bc2d0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 rx=0x7efdd800a940 tx=0x7efdd800ace0).stop 2021-02-10T11:35:29.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.625 7efde6349700 1 -- 172.21.2.101:0/663692275 shutdown_connections 2021-02-10T11:35:29.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.625 7efde6349700 1 --2- 172.21.2.101:0/663692275 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efdc00186d0 0x7efdc001abd0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:29.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.625 7efde6349700 1 --2- 172.21.2.101:0/663692275 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efde018b760 0x7efde010d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:29.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.625 7efde6349700 1 --2- 172.21.2.101:0/663692275 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efde010d810 0x7efde00bc2d0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:29.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.625 7efde6349700 1 --2- 172.21.2.101:0/663692275 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efde0188d50 0x7efde010cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:29.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.625 7efde6349700 1 -- 172.21.2.101:0/663692275 >> 172.21.2.101:0/663692275 conn(0x7efde00a1790 msgr2=0x7efde0187fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:29.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.625 7efde6349700 1 -- 172.21.2.101:0/663692275 shutdown_connections 2021-02-10T11:35:29.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:29.625 7efde6349700 1 -- 172.21.2.101:0/663692275 wait complete. 2021-02-10T11:35:29.642 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:35:29.642 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:35:30.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.045 7fc79a4db700 1 Processor -- start 2021-02-10T11:35:30.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.045 7fc79a4db700 1 -- start start 2021-02-10T11:35:30.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.045 7fc79a4db700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc7940e53e0 0x7fc7941b0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:30.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.045 7fc79a4db700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc794082d50 con 0x7fc7941b0b70 2021-02-10T11:35:30.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.045 7fc79a4db700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7940e57e0 con 0x7fc7940842e0 2021-02-10T11:35:30.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.045 7fc79a4db700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7940e5fe0 con 0x7fc7940e53e0 2021-02-10T11:35:30.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.046 7fc793fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc7940e53e0 0x7fc7941b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:30.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.046 7fc793fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc7940e53e0 0x7fc7941b0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53180/0 (socket says 172.21.2.101:53180) 2021-02-10T11:35:30.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.046 7fc793fff700 1 -- 172.21.2.101:0/674155708 learned_addr learned my addr 172.21.2.101:0/674155708 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:30.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.046 7fc7937fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc7941b0b70 0x7fc794081910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53672/0 (socket says 172.21.2.101:53672) 2021-02-10T11:35:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.046 7fc793fff700 1 -- 172.21.2.101:0/674155708 >> v1:172.21.2.108:6789/0 conn(0x7fc7940842e0 legacy=0x7fc794082100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.046 7fc793fff700 1 -- 172.21.2.101:0/674155708 >> v1:172.21.2.101:6789/0 conn(0x7fc7941b0b70 legacy=0x7fc794081910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.046 7fc793fff700 1 -- 172.21.2.101:0/674155708 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7940846e0 con 0x7fc7940e53e0 2021-02-10T11:35:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.046 7fc793fff700 1 --2- 172.21.2.101:0/674155708 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7940e53e0 0x7fc7941b0550 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 rx=0x7fc77c006700 tx=0x7fc77c009b80).ready entity=mon.0 client_cookie=e9af41253b028588 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.046 7fc792ffd700 1 -- 172.21.2.101:0/674155708 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc77c00c0c0 con 0x7fc7940e53e0 2021-02-10T11:35:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.046 7fc792ffd700 1 -- 172.21.2.101:0/674155708 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77c00c440 con 0x7fc7940e53e0 2021-02-10T11:35:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.046 7fc792ffd700 1 -- 172.21.2.101:0/674155708 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc77c00c960 con 0x7fc7940e53e0 2021-02-10T11:35:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.046 7fc79a4db700 1 -- 172.21.2.101:0/674155708 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7940e53e0 msgr2=0x7fc7941b0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.046 7fc79a4db700 1 --2- 172.21.2.101:0/674155708 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7940e53e0 0x7fc7941b0550 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 rx=0x7fc77c006700 tx=0x7fc77c009b80).stop 2021-02-10T11:35:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.047 7fc79a4db700 1 -- 172.21.2.101:0/674155708 shutdown_connections 2021-02-10T11:35:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.047 7fc79a4db700 1 --2- 172.21.2.101:0/674155708 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7940e53e0 0x7fc7941b0550 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.047 7fc79a4db700 1 -- 172.21.2.101:0/674155708 >> 172.21.2.101:0/674155708 conn(0x7fc7940ce500 msgr2=0x7fc7940e4610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:30.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.047 7fc79a4db700 1 -- 172.21.2.101:0/674155708 shutdown_connections 2021-02-10T11:35:30.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.047 7fc79a4db700 1 -- 172.21.2.101:0/674155708 wait complete. 2021-02-10T11:35:30.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.062 7fc79a4db700 1 Processor -- start 2021-02-10T11:35:30.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.062 7fc79a4db700 1 -- start start 2021-02-10T11:35:30.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.062 7fc79a4db700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7940f1530 0x7fc7940bfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.062 7fc79a4db700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7940c0220 0x7fc7940ebe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.062 7fc79a4db700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7940ec430 0x7fc7940ec830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.062 7fc79a4db700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7941c92f0 con 0x7fc7940ec430 2021-02-10T11:35:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.062 7fc79a4db700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7941c9510 con 0x7fc7940c0220 2021-02-10T11:35:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.062 7fc79a4db700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7941c9730 con 0x7fc7940f1530 2021-02-10T11:35:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.063 7fc798af8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7940ec430 0x7fc7940ec830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.063 7fc798af8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7940ec430 0x7fc7940ec830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53184/0 (socket says 172.21.2.101:53184) 2021-02-10T11:35:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.063 7fc798af8700 1 -- 172.21.2.101:0/3096134763 learned_addr learned my addr 172.21.2.101:0/3096134763 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.063 7fc793fff700 1 --2- 172.21.2.101:0/3096134763 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7940f1530 0x7fc7940bfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.063 7fc7937fe700 1 --2- 172.21.2.101:0/3096134763 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7940c0220 0x7fc7940ebe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.063 7fc793fff700 1 -- 172.21.2.101:0/3096134763 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7940c0220 msgr2=0x7fc7940ebe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.063 7fc793fff700 1 --2- 172.21.2.101:0/3096134763 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7940c0220 0x7fc7940ebe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.063 7fc793fff700 1 -- 172.21.2.101:0/3096134763 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7940ec430 msgr2=0x7fc7940ec830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.063 7fc793fff700 1 --2- 172.21.2.101:0/3096134763 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7940ec430 0x7fc7940ec830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.063 7fc793fff700 1 -- 172.21.2.101:0/3096134763 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7941c9950 con 0x7fc7940f1530 2021-02-10T11:35:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.063 7fc798af8700 1 --2- 172.21.2.101:0/3096134763 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7940ec430 0x7fc7940ec830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:35:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.063 7fc793fff700 1 --2- 172.21.2.101:0/3096134763 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7940f1530 0x7fc7940bfc00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fc77c006f20 tx=0x7fc77c00b8e0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.063 7fc790ff9700 1 -- 172.21.2.101:0/3096134763 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc77c0091d0 con 0x7fc7940f1530 2021-02-10T11:35:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.063 7fc790ff9700 1 -- 172.21.2.101:0/3096134763 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77c00bda0 con 0x7fc7940f1530 2021-02-10T11:35:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.063 7fc79a4db700 1 -- 172.21.2.101:0/3096134763 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7941c9950 con 0x7fc7940f1530 2021-02-10T11:35:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.063 7fc79a4db700 1 -- 172.21.2.101:0/3096134763 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7941c9ee0 con 0x7fc7940f1530 2021-02-10T11:35:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.063 7fc790ff9700 1 -- 172.21.2.101:0/3096134763 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc77c006220 con 0x7fc7940f1530 2021-02-10T11:35:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.064 7fc790ff9700 1 -- 172.21.2.101:0/3096134763 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc77c009100 con 0x7fc7940f1530 2021-02-10T11:35:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.064 7fc790ff9700 1 --2- 172.21.2.101:0/3096134763 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7780186d0 0x7fc77801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.064 7fc790ff9700 1 -- 172.21.2.101:0/3096134763 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc77c032700 con 0x7fc7940f1530 2021-02-10T11:35:30.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.064 7fc7937fe700 1 --2- 172.21.2.101:0/3096134763 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7780186d0 0x7fc77801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:30.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.064 7fc7937fe700 1 --2- 172.21.2.101:0/3096134763 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7780186d0 0x7fc77801abd0 crc :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:30.089 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:35:30.092 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:35:30.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.095 7fc79a4db700 1 -- 172.21.2.101:0/3096134763 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc76c002fd0 con 0x7fc7940f1530 2021-02-10T11:35:30.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.098 7fc790ff9700 1 -- 172.21.2.101:0/3096134763 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc77c0009b0 con 0x7fc7940f1530 2021-02-10T11:35:30.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.228 7fc79a4db700 1 -- 172.21.2.101:0/3096134763 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc76c002de0 con 0x7fc7940f1530 2021-02-10T11:35:30.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.229 7fc790ff9700 1 -- 172.21.2.101:0/3096134763 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+2101 (secure 0 0 0) 0x7fc77c00bda0 con 0x7fc7940f1530 2021-02-10T11:35:30.237 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 47 2021-02-10T11:35:30.237 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:30.237 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1297756180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1297756180},{"type":"v1","addr":"172.21.2.101:6835","nonce":1297756180}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:34:41.088085","modified":"2021-02-10 11:34:43.450805","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6400":{"gid":6400,"name":"b","rank":0,"incarnation":44,"state":"up:active","state_seq":4,"addr":"172.21.2.108:6833/3585539482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3585539482},{"type":"v1","addr":"172.21.2.108:6833","nonce":3585539482}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-02-10T11:35:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.230 7fc79a4db700 1 -- 172.21.2.101:0/3096134763 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7780186d0 msgr2=0x7fc77801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.230 7fc79a4db700 1 --2- 172.21.2.101:0/3096134763 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7780186d0 0x7fc77801abd0 crc :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.230 7fc79a4db700 1 -- 172.21.2.101:0/3096134763 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7940f1530 msgr2=0x7fc7940bfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.230 7fc79a4db700 1 --2- 172.21.2.101:0/3096134763 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7940f1530 0x7fc7940bfc00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fc77c006f20 tx=0x7fc77c00b8e0).stop 2021-02-10T11:35:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.230 7fc79a4db700 1 -- 172.21.2.101:0/3096134763 shutdown_connections 2021-02-10T11:35:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.230 7fc79a4db700 1 --2- 172.21.2.101:0/3096134763 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7940ec430 0x7fc7940ec830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.230 7fc79a4db700 1 --2- 172.21.2.101:0/3096134763 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7940c0220 0x7fc7940ebe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.230 7fc79a4db700 1 --2- 172.21.2.101:0/3096134763 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7780186d0 0x7fc77801abd0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:30.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.230 7fc79a4db700 1 --2- 172.21.2.101:0/3096134763 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7940f1530 0x7fc7940bfc00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:30.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.230 7fc79a4db700 1 -- 172.21.2.101:0/3096134763 >> 172.21.2.101:0/3096134763 conn(0x7fc79409fbf0 msgr2=0x7fc79408e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:30.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.231 7fc79a4db700 1 -- 172.21.2.101:0/3096134763 shutdown_connections 2021-02-10T11:35:30.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:30.231 7fc79a4db700 1 -- 172.21.2.101:0/3096134763 wait complete. 2021-02-10T11:35:30.450 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:34:41.088085', 'modified': '2021-02-10 11:34:43.450805', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6400}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6400': {'gid': 6400, 'name': 'b', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.2.108:6833/3585539482', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 3585539482}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 3585539482}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:35:30.450 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:35:30.450 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:35:30.848 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:35:30.849 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:35:30.849 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:35:30.849 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 6400, 2021-02-10T11:35:30.849 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:35:30.849 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:35:30.849 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 49.745667337, 2021-02-10T11:35:30.849 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 46, 2021-02-10T11:35:30.849 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 65, 2021-02-10T11:35:30.849 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 65, 2021-02-10T11:35:30.849 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 58.467139547999999 2021-02-10T11:35:30.849 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:35:30.853 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 6400, "want_state": "up:active", "state": "up:active", "rank_uptime": 49.745667337, "mdsmap_epoch": 46, "osdmap_epoch": 65, "osdmap_epoch_barrier": 65, "uptime": 58.467139547999999 } 2021-02-10T11:35:30.853 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:35:30.853 INFO:tasks.cephfs.fuse_mount:Client client.alice config is {} 2021-02-10T11:35:30.853 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.alice at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:35:30.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:30.853 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:31.188 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:35:31.532 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:35:31.532 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:35:31.876 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:35:31.876 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:35:31.877 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:35:31.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:31.877 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:35:32.212 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:35:32.212 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2021-02-10T11:35:32.212 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --client_mountpoint=/volumes/group_0000000000497714/subvolume_0000000000063820/d66a765b-b620-4859-b962-f6b2ea79bb52 --name client.alice /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:32.215 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:35:32.558 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:35:32.559 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:35:32.625 INFO:tasks.cephfs.fuse_mount.ceph-fuse.alice.gibba001.stderr:2021-02-10 11:35:32.617 7f1e759d9f80 -1 init, newargv = 0x560983ee1130 newargc=9 2021-02-10T11:35:32.626 INFO:tasks.cephfs.fuse_mount.ceph-fuse.alice.gibba001.stderr:ceph-fuse[127153]: starting ceph client 2021-02-10T11:35:32.632 INFO:tasks.cephfs.fuse_mount.ceph-fuse.alice.gibba001.stderr:ceph-fuse[127153]: starting fuse 2021-02-10T11:35:32.901 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:35:32.901 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:35:32.902 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:35:32.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:32.902 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:35:33.237 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:35:33.237 INFO:teuthology.orchestra.run.gibba001.stdout:44 2021-02-10T11:35:33.237 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2021-02-10T11:35:33.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:33.238 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:35:33.238 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:35:33.238 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:35:33.238 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:35:33.238 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:35:33.238 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:35:33.238 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:35:33.238 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.alice.*.asok" 2021-02-10T11:35:33.238 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:35:33.238 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:35:33.238 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:35:33.239 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:35:33.239 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:35:33.239 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:35:33.239 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:35:33.239 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:35:33.239 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:35:33.239 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:35:33.239 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:35:33.239 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:35:33.239 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:35:33.239 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.alice")) 2021-02-10T11:35:33.239 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:35:33.601 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.alice.127153.asok 2021-02-10T11:35:33.604 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.alice.127153.asok 2021-02-10T11:35:33.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:33.604 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.alice.127153.asok status 2021-02-10T11:35:33.994 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:35:33.994 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:35:33.995 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:35:33.995 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:35:33.995 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "alice", 2021-02-10T11:35:33.995 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:35:33.995 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:35:33.995 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "127153", 2021-02-10T11:35:33.995 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/volumes/group_0000000000497714/subvolume_0000000000063820/d66a765b-b620-4859-b962-f6b2ea79bb52" 2021-02-10T11:35:33.995 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:33.995 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:35:33.995 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:35:33.995 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 8850, 2021-02-10T11:35:33.995 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:35:33.995 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:35:33.996 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:35:33.996 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 8850 2021-02-10T11:35:33.996 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:33.996 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:35:33.996 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:33.996 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:35:33.996 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 3514370052 2021-02-10T11:35:33.996 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:33.996 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:33.996 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:35:33.996 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:33.996 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:35:33.996 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 3514370052 2021-02-10T11:35:33.996 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:33.997 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.8850 172.21.2.101:0/3514370052", 2021-02-10T11:35:33.997 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/3514370052", 2021-02-10T11:35:33.997 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:35:33.997 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 46, 2021-02-10T11:35:33.997 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 65, 2021-02-10T11:35:33.997 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:35:33.997 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:35:33.997 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:35:33.998 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.1 && sudo ls /home/ubuntu/cephtest/mnt.1 2021-02-10T11:35:34.344 INFO:teuthology.orchestra.run.gibba001.stdout:data.bin 2021-02-10T11:35:34.346 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:34.346 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:35:34.346 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:34.682 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:35:34.682 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:35:34.682 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.1 && sudo dd if=/dev/urandom of=rogue.bin bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:35:35.028 INFO:teuthology.orchestra.run.gibba001.stderr:dd: failed to open ‘rogue.bin’: Permission denied 2021-02-10T11:35:35.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:35:35.029 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:35.029 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:35:35.029 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:35.364 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:35:35.365 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:35:35.365 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:35:35.365 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:35:35.365 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:35.716 INFO:tasks.cephfs.fuse_mount.ceph-fuse.alice.gibba001.stderr:ceph-fuse[127153]: fuse finished with error 0 and tester_r 0 2021-02-10T11:35:35.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:35.784 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:35:35.784 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:36.120 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:35:36.120 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:35:36.120 INFO:teuthology.orchestra.run:waiting for 900 2021-02-10T11:35:36.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:35:36.121 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:36.456 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume deauthorize cephfs subvolume_0000000000063820 alice --group_name group_0000000000497714 2021-02-10T11:35:36.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.863 7f4532fe9700 1 Processor -- start 2021-02-10T11:35:36.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.863 7f4532fe9700 1 -- start start 2021-02-10T11:35:36.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.864 7f4532fe9700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f452c0e51a0 0x7f452c1b3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:36.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.864 7f4532fe9700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f452c1b3f30 0x7f452c077510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:36.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.864 7f4532fe9700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452c078b90 con 0x7f452c0e55a0 2021-02-10T11:35:36.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.864 7f4532fe9700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452c07a120 con 0x7f452c0e51a0 2021-02-10T11:35:36.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.864 7f4532fe9700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452c07a2d0 con 0x7f452c1b3f30 2021-02-10T11:35:36.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.864 7f4530e05700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f452c0e51a0 0x7f452c1b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:36.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.864 7f4530e05700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f452c0e51a0 0x7f452c1b3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53212/0 (socket says 172.21.2.101:53212) 2021-02-10T11:35:36.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.864 7f4530e05700 1 -- 172.21.2.101:0/3301034924 learned_addr learned my addr 172.21.2.101:0/3301034924 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:36.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.864 7f4531606700 1 --1- 172.21.2.101:0/3301034924 >> v1:172.21.2.101:6789/0 conn(0x7f452c0e55a0 0x7f452c077b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53706/0 (socket says 172.21.2.101:53706) 2021-02-10T11:35:36.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.865 7f452b7fe700 1 -- 172.21.2.101:0/3301034924 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 187168296 0 0) 0x7f4524000c10 con 0x7f452c0e55a0 2021-02-10T11:35:36.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.865 7f452b7fe700 1 -- 172.21.2.101:0/3301034924 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4518002fe0 con 0x7f452c0e55a0 2021-02-10T11:35:36.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.865 7f452bfff700 1 --2- 172.21.2.101:0/3301034924 >> v2:172.21.2.108:3300/0 conn(0x7f452c1b3f30 0x7f452c077510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:36.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.865 7f4530e05700 1 -- 172.21.2.101:0/3301034924 >> v2:172.21.2.108:3300/0 conn(0x7f452c1b3f30 msgr2=0x7f452c077510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:36.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.865 7f4530e05700 1 --2- 172.21.2.101:0/3301034924 >> v2:172.21.2.108:3300/0 conn(0x7f452c1b3f30 0x7f452c077510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:36.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.865 7f4530e05700 1 -- 172.21.2.101:0/3301034924 >> v1:172.21.2.101:6789/0 conn(0x7f452c0e55a0 legacy=0x7f452c077b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.865 7f4530e05700 1 -- 172.21.2.101:0/3301034924 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c07a4f0 con 0x7f452c0e51a0 2021-02-10T11:35:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.865 7f452bfff700 1 --2- 172.21.2.101:0/3301034924 >> v2:172.21.2.108:3300/0 conn(0x7f452c1b3f30 0x7f452c077510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:35:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.865 7f4530e05700 1 --2- 172.21.2.101:0/3301034924 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f452c0e51a0 0x7f452c1b3910 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 rx=0x7f4514006700 tx=0x7f4514009b80).ready entity=mon.0 client_cookie=799409c865ce3269 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.865 7f452b7fe700 1 -- 172.21.2.101:0/3301034924 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f451400c0c0 con 0x7f452c0e51a0 2021-02-10T11:35:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.865 7f452b7fe700 1 -- 172.21.2.101:0/3301034924 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451400c340 con 0x7f452c0e51a0 2021-02-10T11:35:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.865 7f452b7fe700 1 -- 172.21.2.101:0/3301034924 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f451400c860 con 0x7f452c0e51a0 2021-02-10T11:35:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.865 7f4532fe9700 1 -- 172.21.2.101:0/3301034924 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f452c0e51a0 msgr2=0x7f452c1b3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.865 7f4532fe9700 1 --2- 172.21.2.101:0/3301034924 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f452c0e51a0 0x7f452c1b3910 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 rx=0x7f4514006700 tx=0x7f4514009b80).stop 2021-02-10T11:35:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.866 7f4532fe9700 1 -- 172.21.2.101:0/3301034924 shutdown_connections 2021-02-10T11:35:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.866 7f4532fe9700 1 --2- 172.21.2.101:0/3301034924 >> v2:172.21.2.108:3300/0 conn(0x7f452c1b3f30 0x7f452c077510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.866 7f4532fe9700 1 --2- 172.21.2.101:0/3301034924 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f452c0e51a0 0x7f452c1b3910 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.866 7f4532fe9700 1 -- 172.21.2.101:0/3301034924 >> 172.21.2.101:0/3301034924 conn(0x7f452c0ce850 msgr2=0x7f452c0e4690 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:36.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.866 7f4532fe9700 1 -- 172.21.2.101:0/3301034924 shutdown_connections 2021-02-10T11:35:36.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.866 7f4532fe9700 1 -- 172.21.2.101:0/3301034924 wait complete. 2021-02-10T11:35:36.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.880 7f4532fe9700 1 Processor -- start 2021-02-10T11:35:36.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.880 7f4532fe9700 1 -- start start 2021-02-10T11:35:36.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.881 7f4532fe9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f452c09afe0 0x7f452c096070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:36.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.881 7f4532fe9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f452c096690 0x7f452c096a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:36.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.881 7f4532fe9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f452c0970b0 0x7f452c1ac800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:36.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.881 7f4532fe9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c095af0 con 0x7f452c096690 2021-02-10T11:35:36.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.881 7f4532fe9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c1ace20 con 0x7f452c09afe0 2021-02-10T11:35:36.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.881 7f4532fe9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452c1ad000 con 0x7f452c0970b0 2021-02-10T11:35:36.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.881 7f452bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f452c096690 0x7f452c096a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:36.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.881 7f452bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f452c096690 0x7f452c096a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53218/0 (socket says 172.21.2.101:53218) 2021-02-10T11:35:36.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.881 7f452bfff700 1 -- 172.21.2.101:0/1980016255 learned_addr learned my addr 172.21.2.101:0/1980016255 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:36.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.882 7f452bfff700 1 -- 172.21.2.101:0/1980016255 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f452c0970b0 msgr2=0x7f452c1ac800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:36.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.882 7f452bfff700 1 --2- 172.21.2.101:0/1980016255 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f452c0970b0 0x7f452c1ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:36.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.882 7f452bfff700 1 -- 172.21.2.101:0/1980016255 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f452c09afe0 msgr2=0x7f452c096070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:36.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.882 7f452bfff700 1 --2- 172.21.2.101:0/1980016255 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f452c09afe0 0x7f452c096070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:36.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.882 7f452bfff700 1 -- 172.21.2.101:0/1980016255 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c1ad220 con 0x7f452c096690 2021-02-10T11:35:36.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.882 7f452bfff700 1 --2- 172.21.2.101:0/1980016255 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f452c096690 0x7f452c096a90 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 rx=0x7f451c0099a0 tx=0x7f451c00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:36.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.882 7f45297fa700 1 -- 172.21.2.101:0/1980016255 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f451c00c2f0 con 0x7f452c096690 2021-02-10T11:35:36.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.882 7f4532fe9700 1 -- 172.21.2.101:0/1980016255 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452c1ad220 con 0x7f452c096690 2021-02-10T11:35:36.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.882 7f4532fe9700 1 -- 172.21.2.101:0/1980016255 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452c1ad220 con 0x7f452c096690 2021-02-10T11:35:36.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.882 7f45297fa700 1 -- 172.21.2.101:0/1980016255 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451c00c2f0 con 0x7f452c096690 2021-02-10T11:35:36.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.882 7f45297fa700 1 -- 172.21.2.101:0/1980016255 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f451c00c670 con 0x7f452c096690 2021-02-10T11:35:36.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.883 7f45297fa700 1 -- 172.21.2.101:0/1980016255 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f451c001010 con 0x7f452c096690 2021-02-10T11:35:36.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.883 7f45297fa700 1 --2- 172.21.2.101:0/1980016255 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4510018730 0x7f451001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:36.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.883 7f45297fa700 1 -- 172.21.2.101:0/1980016255 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f451c032b70 con 0x7f452c096690 2021-02-10T11:35:36.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.883 7f4530e05700 1 --2- 172.21.2.101:0/1980016255 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4510018730 0x7f451001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:36.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.883 7f4530e05700 1 --2- 172.21.2.101:0/1980016255 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4510018730 0x7f451001ac10 crc :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:36.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.914 7f4532fe9700 1 -- 172.21.2.101:0/1980016255 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452c10a8d0 con 0x7f452c096690 2021-02-10T11:35:36.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:36.916 7f45297fa700 1 -- 172.21.2.101:0/1980016255 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f451c00be30 con 0x7f452c096690 2021-02-10T11:35:37.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.041 7f4532fe9700 1 -- 172.21.2.101:0/1980016255 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume deauthorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000063820", "target": ["mgr", ""], "auth_id": "alice", "group_name": "group_0000000000497714"}) v1 -- 0x7f452c10a8d0 con 0x7f4510018730 2021-02-10T11:35:37.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.076 7f45297fa700 1 -- 172.21.2.101:0/1980016255 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f4514006390 con 0x7f4510018730 2021-02-10T11:35:37.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.106 7f4532fe9700 1 -- 172.21.2.101:0/1980016255 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4510018730 msgr2=0x7f451001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:37.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.106 7f4532fe9700 1 --2- 172.21.2.101:0/1980016255 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4510018730 0x7f451001ac10 crc :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:37.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.106 7f4532fe9700 1 -- 172.21.2.101:0/1980016255 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f452c096690 msgr2=0x7f452c096a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:37.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.106 7f4532fe9700 1 --2- 172.21.2.101:0/1980016255 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f452c096690 0x7f452c096a90 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 rx=0x7f451c0099a0 tx=0x7f451c00bcf0).stop 2021-02-10T11:35:37.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.106 7f4532fe9700 1 -- 172.21.2.101:0/1980016255 shutdown_connections 2021-02-10T11:35:37.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.106 7f4532fe9700 1 --2- 172.21.2.101:0/1980016255 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4510018730 0x7f451001ac10 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:37.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.106 7f4532fe9700 1 --2- 172.21.2.101:0/1980016255 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f452c0970b0 0x7f452c1ac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:37.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.106 7f4532fe9700 1 --2- 172.21.2.101:0/1980016255 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f452c096690 0x7f452c096a90 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:37.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.106 7f4532fe9700 1 --2- 172.21.2.101:0/1980016255 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f452c09afe0 0x7f452c096070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:37.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.106 7f4532fe9700 1 -- 172.21.2.101:0/1980016255 >> 172.21.2.101:0/1980016255 conn(0x7f452c1a12f0 msgr2=0x7f452c1935c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:37.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.106 7f4532fe9700 1 -- 172.21.2.101:0/1980016255 shutdown_connections 2021-02-10T11:35:37.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.106 7f4532fe9700 1 -- 172.21.2.101:0/1980016255 wait complete. 2021-02-10T11:35:37.122 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:35:37.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.525 7f2a33df3700 1 Processor -- start 2021-02-10T11:35:37.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.525 7f2a33df3700 1 -- start start 2021-02-10T11:35:37.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.526 7f2a33df3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2a2c0e53e0 0x7f2a2c1b0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:37.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.526 7f2a33df3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2a2c1b0b70 0x7f2a2c081910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:37.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.526 7f2a33df3700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a2c082d50 con 0x7f2a2c081f30 2021-02-10T11:35:37.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.526 7f2a33df3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a2c082b20 con 0x7f2a2c1b0b70 2021-02-10T11:35:37.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.526 7f2a33df3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a2c0e5fe0 con 0x7f2a2c0e53e0 2021-02-10T11:35:37.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.526 7f2a3140e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2a2c1b0b70 0x7f2a2c081910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:37.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.526 7f2a3140e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2a2c1b0b70 0x7f2a2c081910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53224/0 (socket says 172.21.2.101:53224) 2021-02-10T11:35:37.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.526 7f2a3140e700 1 -- 172.21.2.101:0/4270489347 learned_addr learned my addr 172.21.2.101:0/4270489347 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:37.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.526 7f2a3140e700 1 -- 172.21.2.101:0/4270489347 >> v2:172.21.2.108:3300/0 conn(0x7f2a2c0e53e0 msgr2=0x7f2a2c1b0550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:35:37.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.526 7f2a3140e700 1 --2- 172.21.2.101:0/4270489347 >> v2:172.21.2.108:3300/0 conn(0x7f2a2c0e53e0 0x7f2a2c1b0550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:37.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.526 7f2a3140e700 1 -- 172.21.2.101:0/4270489347 >> v1:172.21.2.108:6789/0 conn(0x7f2a2c081f30 legacy=0x7f2a2c082330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:35:37.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.526 7f2a3140e700 1 -- 172.21.2.101:0/4270489347 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c0e57e0 con 0x7f2a2c1b0b70 2021-02-10T11:35:37.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.526 7f2a3140e700 1 --2- 172.21.2.101:0/4270489347 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c1b0b70 0x7f2a2c081910 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 rx=0x7f2a20006700 tx=0x7f2a20009b80).ready entity=mon.0 client_cookie=44c7c5c01bf1b6a6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:37.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.527 7f2a30c0d700 1 -- 172.21.2.101:0/4270489347 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2a2000c0c0 con 0x7f2a2c1b0b70 2021-02-10T11:35:37.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.527 7f2a30c0d700 1 -- 172.21.2.101:0/4270489347 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2000c340 con 0x7f2a2c1b0b70 2021-02-10T11:35:37.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.527 7f2a30c0d700 1 -- 172.21.2.101:0/4270489347 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2a2000c860 con 0x7f2a2c1b0b70 2021-02-10T11:35:37.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.527 7f2a33df3700 1 -- 172.21.2.101:0/4270489347 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c1b0b70 msgr2=0x7f2a2c081910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:37.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.527 7f2a33df3700 1 --2- 172.21.2.101:0/4270489347 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c1b0b70 0x7f2a2c081910 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 rx=0x7f2a20006700 tx=0x7f2a20009b80).stop 2021-02-10T11:35:37.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.527 7f2a33df3700 1 -- 172.21.2.101:0/4270489347 shutdown_connections 2021-02-10T11:35:37.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.527 7f2a33df3700 1 --2- 172.21.2.101:0/4270489347 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c1b0b70 0x7f2a2c081910 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:37.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.527 7f2a33df3700 1 --2- 172.21.2.101:0/4270489347 >> v2:172.21.2.108:3300/0 conn(0x7f2a2c0e53e0 0x7f2a2c1b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:37.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.527 7f2a33df3700 1 -- 172.21.2.101:0/4270489347 >> 172.21.2.101:0/4270489347 conn(0x7f2a2c0ce500 msgr2=0x7f2a2c0e4610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:37.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.527 7f2a33df3700 1 -- 172.21.2.101:0/4270489347 shutdown_connections 2021-02-10T11:35:37.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.527 7f2a33df3700 1 -- 172.21.2.101:0/4270489347 wait complete. 2021-02-10T11:35:37.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.542 7f2a33df3700 1 Processor -- start 2021-02-10T11:35:37.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.542 7f2a33df3700 1 -- start start 2021-02-10T11:35:37.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.542 7f2a33df3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a2c0b7fc0 0x7f2a2c0b83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:37.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.542 7f2a33df3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2a2c094b00 0x7f2a2c094f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:37.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.542 7f2a33df3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c095520 0x7f2a2c095920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:37.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.542 7f2a33df3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c094580 con 0x7f2a2c095520 2021-02-10T11:35:37.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.542 7f2a33df3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c1a2b70 con 0x7f2a2c094b00 2021-02-10T11:35:37.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.542 7f2a33df3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c1b13f0 con 0x7f2a2c0b7fc0 2021-02-10T11:35:37.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.543 7f2a32410700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c095520 0x7f2a2c095920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:37.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.543 7f2a32410700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c095520 0x7f2a2c095920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53230/0 (socket says 172.21.2.101:53230) 2021-02-10T11:35:37.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.543 7f2a32410700 1 -- 172.21.2.101:0/3610252447 learned_addr learned my addr 172.21.2.101:0/3610252447 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:37.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.543 7f2a32410700 1 -- 172.21.2.101:0/3610252447 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a2c0b7fc0 msgr2=0x7f2a2c0b83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:37.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.543 7f2a32410700 1 --2- 172.21.2.101:0/3610252447 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a2c0b7fc0 0x7f2a2c0b83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:37.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.543 7f2a32410700 1 -- 172.21.2.101:0/3610252447 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2a2c094b00 msgr2=0x7f2a2c094f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:37.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.543 7f2a32410700 1 --2- 172.21.2.101:0/3610252447 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2a2c094b00 0x7f2a2c094f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:37.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.543 7f2a32410700 1 -- 172.21.2.101:0/3610252447 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c1b1610 con 0x7f2a2c095520 2021-02-10T11:35:37.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.543 7f2a32410700 1 --2- 172.21.2.101:0/3610252447 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c095520 0x7f2a2c095920 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 rx=0x7f2a24009940 tx=0x7f2a2400bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:37.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.543 7f2a1e7fc700 1 -- 172.21.2.101:0/3610252447 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2a24003030 con 0x7f2a2c095520 2021-02-10T11:35:37.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.543 7f2a1e7fc700 1 -- 172.21.2.101:0/3610252447 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a24001270 con 0x7f2a2c095520 2021-02-10T11:35:37.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.543 7f2a1e7fc700 1 -- 172.21.2.101:0/3610252447 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2a24001750 con 0x7f2a2c095520 2021-02-10T11:35:37.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.543 7f2a33df3700 1 -- 172.21.2.101:0/3610252447 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2c1b1610 con 0x7f2a2c095520 2021-02-10T11:35:37.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.543 7f2a33df3700 1 -- 172.21.2.101:0/3610252447 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2c1b1610 con 0x7f2a2c095520 2021-02-10T11:35:37.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.544 7f2a1e7fc700 1 -- 172.21.2.101:0/3610252447 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2a24001450 con 0x7f2a2c095520 2021-02-10T11:35:37.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.544 7f2a1e7fc700 1 --2- 172.21.2.101:0/3610252447 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a140186d0 0x7f2a1401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:37.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.544 7f2a1e7fc700 1 -- 172.21.2.101:0/3610252447 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2a24032b60 con 0x7f2a2c095520 2021-02-10T11:35:37.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.544 7f2a31c0f700 1 --2- 172.21.2.101:0/3610252447 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a140186d0 0x7f2a1401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:37.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.544 7f2a31c0f700 1 --2- 172.21.2.101:0/3610252447 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a140186d0 0x7f2a1401abd0 crc :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:37.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.576 7f2a33df3700 1 -- 172.21.2.101:0/3610252447 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a2c0ed660 con 0x7f2a2c095520 2021-02-10T11:35:37.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.578 7f2a1e7fc700 1 -- 172.21.2.101:0/3610252447 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2a2400be30 con 0x7f2a2c095520 2021-02-10T11:35:37.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.709 7f2a33df3700 1 -- 172.21.2.101:0/3610252447 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2a2c0ed660 con 0x7f2a2c095520 2021-02-10T11:35:37.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.710 7f2a1e7fc700 1 -- 172.21.2.101:0/3610252447 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v44) v1 ==== 80+0+4513 (secure 0 0 0) 0x7f2a24000db0 con 0x7f2a2c095520 2021-02-10T11:35:37.719 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:37.719 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:35:37.719 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:35:37.719 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:37.720 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:35:37.720 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:35:37.720 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:37.720 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:35:37.720 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:37.721 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:37.721 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:37.721 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:37.721 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:37.721 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:37.721 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:35:37.722 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:35:37.722 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:37.722 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:35:37.722 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:37.723 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:37.723 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:37.723 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:37.723 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:37.723 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:37.723 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:35:37.724 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:35:37.724 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:37.724 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:37.724 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:37.724 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:37.724 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:37.724 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:37.724 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:37.724 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:35:37.725 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:35:37.725 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:37.725 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:37.725 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:37.725 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:37.725 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:37.725 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:37.725 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:37.725 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:35:37.725 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:35:37.725 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:37.725 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:37.726 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:37.726 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:37.726 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:37.726 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:37.726 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:37.726 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:35:37.726 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:35:37.726 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:37.726 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:37.726 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:37.726 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:37.726 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:37.726 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:37.727 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:37.727 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:35:37.727 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:35:37.727 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:37.727 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:37.727 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:37.727 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:37.727 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:37.727 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:37.727 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:37.727 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:35:37.727 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:35:37.728 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:37.728 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:37.728 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:37.728 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:37.728 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:37.728 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:37.728 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:37.728 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:35:37.728 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:35:37.728 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:37.728 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:37.728 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:37.728 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:37.729 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:37.729 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:37.729 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:37.729 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:35:37.729 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:35:37.729 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:37.729 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:37.729 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:37.729 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:37.729 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:37.729 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:37.729 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:37.730 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:35:37.730 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:35:37.730 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:37.730 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:35:37.730 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:35:37.730 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:35:37.730 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:37.730 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:37.730 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:37.730 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:35:37.730 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:35:37.730 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:37.731 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:35:37.731 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:35:37.731 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:35:37.731 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:37.731 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:37.731 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:37.731 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:35:37.731 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:35:37.731 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:37.731 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:35:37.731 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:35:37.731 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:35:37.731 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:37.732 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:37.732 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:37.732 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:35:37.732 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:35:37.732 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:37.732 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:35:37.732 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:35:37.732 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:35:37.732 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:37.732 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:37.732 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:37.732 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:35:37.732 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:35:37.732 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:37.733 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:35:37.733 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:37.733 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:37.733 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:37.733 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:37.733 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:37.733 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:37.733 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:35:37.733 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:35:37.733 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:37.733 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:35:37.733 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:35:37.733 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:37.734 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:37.734 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:37.734 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:37.734 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:35:37.734 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:35:37.734 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:37.734 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:35:37.734 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:35:37.734 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:37.734 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:37.734 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:37.734 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:37.735 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:35:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.712 7f2a33df3700 1 -- 172.21.2.101:0/3610252447 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a140186d0 msgr2=0x7f2a1401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.712 7f2a33df3700 1 --2- 172.21.2.101:0/3610252447 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a140186d0 0x7f2a1401abd0 crc :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.712 7f2a33df3700 1 -- 172.21.2.101:0/3610252447 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c095520 msgr2=0x7f2a2c095920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.712 7f2a33df3700 1 --2- 172.21.2.101:0/3610252447 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c095520 0x7f2a2c095920 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 rx=0x7f2a24009940 tx=0x7f2a2400bcf0).stop 2021-02-10T11:35:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.712 7f2a33df3700 1 -- 172.21.2.101:0/3610252447 shutdown_connections 2021-02-10T11:35:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.712 7f2a33df3700 1 --2- 172.21.2.101:0/3610252447 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c095520 0x7f2a2c095920 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.712 7f2a33df3700 1 --2- 172.21.2.101:0/3610252447 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2a2c094b00 0x7f2a2c094f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.712 7f2a33df3700 1 --2- 172.21.2.101:0/3610252447 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a140186d0 0x7f2a1401abd0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.712 7f2a33df3700 1 --2- 172.21.2.101:0/3610252447 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a2c0b7fc0 0x7f2a2c0b83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.712 7f2a33df3700 1 -- 172.21.2.101:0/3610252447 >> 172.21.2.101:0/3610252447 conn(0x7f2a2c0a0c60 msgr2=0x7f2a2c1ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:37.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.712 7f2a33df3700 1 -- 172.21.2.101:0/3610252447 shutdown_connections 2021-02-10T11:35:37.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:37.712 7f2a33df3700 1 -- 172.21.2.101:0/3610252447 wait complete. 2021-02-10T11:35:37.737 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000063820 --group_name group_0000000000497714 2021-02-10T11:35:38.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.146 7fdcf8643700 1 Processor -- start 2021-02-10T11:35:38.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.146 7fdcf8643700 1 -- start start 2021-02-10T11:35:38.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.147 7fdcf8643700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdcf00e5580 0x7fdcf0078520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:38.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.147 7fdcf8643700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fdcf0079110 0x7fdcf0079510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:38.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.147 7fdcf8643700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf0078b90 con 0x7fdcf00e5180 2021-02-10T11:35:38.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.147 7fdcf8643700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf01b40e0 con 0x7fdcf0079110 2021-02-10T11:35:38.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.147 7fdcf8643700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf00d1910 con 0x7fdcf00e5580 2021-02-10T11:35:38.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.147 7fdcf6c60700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fdcf0079110 0x7fdcf0079510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:38.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.147 7fdcf5c5e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdcf00e5580 0x7fdcf0078520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:38.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.147 7fdcf645f700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fdcf00e5180 0x7fdcf01b38f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:47140/0 (socket says 172.21.2.101:47140) 2021-02-10T11:35:38.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.147 7fdcf645f700 1 -- 172.21.2.101:0/3278468299 learned_addr learned my addr 172.21.2.101:0/3278468299 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:38.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.148 7fdcf5c5e700 1 -- 172.21.2.101:0/3278468299 >> v2:172.21.2.108:3300/0 conn(0x7fdcf0079110 msgr2=0x7fdcf0079510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:38.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.148 7fdcf5c5e700 1 --2- 172.21.2.101:0/3278468299 >> v2:172.21.2.108:3300/0 conn(0x7fdcf0079110 0x7fdcf0079510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:38.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.148 7fdcf5c5e700 1 -- 172.21.2.101:0/3278468299 >> v1:172.21.2.108:6789/0 conn(0x7fdcf00e5180 legacy=0x7fdcf01b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:38.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.148 7fdcf5c5e700 1 -- 172.21.2.101:0/3278468299 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf00d1b30 con 0x7fdcf00e5580 2021-02-10T11:35:38.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.148 7fdcf6c60700 1 --2- 172.21.2.101:0/3278468299 >> v2:172.21.2.108:3300/0 conn(0x7fdcf0079110 0x7fdcf0079510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:35:38.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.148 7fdcf5c5e700 1 --2- 172.21.2.101:0/3278468299 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdcf00e5580 0x7fdcf0078520 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fdcec006650 tx=0x7fdcec009a80).ready entity=mon.2 client_cookie=def6ae8d2ea45299 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:38.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.148 7fdcf545d700 1 -- 172.21.2.101:0/3278468299 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdcec0009b0 con 0x7fdcf00e5580 2021-02-10T11:35:38.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.148 7fdcf545d700 1 -- 172.21.2.101:0/3278468299 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcec00c0c0 con 0x7fdcf00e5580 2021-02-10T11:35:38.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.148 7fdcf545d700 1 -- 172.21.2.101:0/3278468299 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdcec00c5e0 con 0x7fdcf00e5580 2021-02-10T11:35:38.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.148 7fdcf8643700 1 -- 172.21.2.101:0/3278468299 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdcf00e5580 msgr2=0x7fdcf0078520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:38.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.148 7fdcf8643700 1 --2- 172.21.2.101:0/3278468299 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdcf00e5580 0x7fdcf0078520 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fdcec006650 tx=0x7fdcec009a80).stop 2021-02-10T11:35:38.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.149 7fdcf8643700 1 -- 172.21.2.101:0/3278468299 shutdown_connections 2021-02-10T11:35:38.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.149 7fdcf8643700 1 --2- 172.21.2.101:0/3278468299 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdcf00e5580 0x7fdcf0078520 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:38.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.149 7fdcf8643700 1 --2- 172.21.2.101:0/3278468299 >> v2:172.21.2.108:3300/0 conn(0x7fdcf0079110 0x7fdcf0079510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:38.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.149 7fdcf8643700 1 -- 172.21.2.101:0/3278468299 >> 172.21.2.101:0/3278468299 conn(0x7fdcf00ce830 msgr2=0x7fdcf00e4670 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:38.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.149 7fdcf8643700 1 -- 172.21.2.101:0/3278468299 shutdown_connections 2021-02-10T11:35:38.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.149 7fdcf8643700 1 -- 172.21.2.101:0/3278468299 wait complete. 2021-02-10T11:35:38.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.164 7fdcf8643700 1 Processor -- start 2021-02-10T11:35:38.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.164 7fdcf8643700 1 -- start start 2021-02-10T11:35:38.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.164 7fdcf8643700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdcf009f560 0x7fdcf0110300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:38.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.164 7fdcf8643700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdcf0105ee0 0x7fdcf00adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:38.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.164 7fdcf8643700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdcf00ae550 0x7fdcf0111060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:38.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.164 7fdcf8643700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf0110b60 con 0x7fdcf0105ee0 2021-02-10T11:35:38.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.164 7fdcf8643700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf009f9a0 con 0x7fdcf00ae550 2021-02-10T11:35:38.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.164 7fdcf8643700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf01062e0 con 0x7fdcf009f560 2021-02-10T11:35:38.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.164 7fdcf5c5e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdcf0105ee0 0x7fdcf00adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:38.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.164 7fdcf5c5e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdcf0105ee0 0x7fdcf00adf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53242/0 (socket says 172.21.2.101:53242) 2021-02-10T11:35:38.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.164 7fdcf5c5e700 1 -- 172.21.2.101:0/2373574099 learned_addr learned my addr 172.21.2.101:0/2373574099 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:38.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.165 7fdcf6c60700 1 --2- 172.21.2.101:0/2373574099 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdcf00ae550 0x7fdcf0111060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:38.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.165 7fdcf645f700 1 --2- 172.21.2.101:0/2373574099 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdcf009f560 0x7fdcf0110300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:38.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.165 7fdcf5c5e700 1 -- 172.21.2.101:0/2373574099 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdcf009f560 msgr2=0x7fdcf0110300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:38.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.165 7fdcf5c5e700 1 --2- 172.21.2.101:0/2373574099 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdcf009f560 0x7fdcf0110300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:38.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.165 7fdcf5c5e700 1 -- 172.21.2.101:0/2373574099 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdcf00ae550 msgr2=0x7fdcf0111060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:38.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.165 7fdcf5c5e700 1 --2- 172.21.2.101:0/2373574099 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdcf00ae550 0x7fdcf0111060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:38.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.165 7fdcf5c5e700 1 -- 172.21.2.101:0/2373574099 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf0106500 con 0x7fdcf0105ee0 2021-02-10T11:35:38.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.165 7fdcf5c5e700 1 --2- 172.21.2.101:0/2373574099 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdcf0105ee0 0x7fdcf00adf30 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 rx=0x7fdcec009630 tx=0x7fdcec00b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:38.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.165 7fdce2ffd700 1 -- 172.21.2.101:0/2373574099 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdcec0093a0 con 0x7fdcf0105ee0 2021-02-10T11:35:38.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.165 7fdce2ffd700 1 -- 172.21.2.101:0/2373574099 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcec00bd70 con 0x7fdcf0105ee0 2021-02-10T11:35:38.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.165 7fdcf8643700 1 -- 172.21.2.101:0/2373574099 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf0106500 con 0x7fdcf0105ee0 2021-02-10T11:35:38.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.165 7fdce2ffd700 1 -- 172.21.2.101:0/2373574099 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdcec0054d0 con 0x7fdcf0105ee0 2021-02-10T11:35:38.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.165 7fdcf8643700 1 -- 172.21.2.101:0/2373574099 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf0106500 con 0x7fdcf0105ee0 2021-02-10T11:35:38.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.166 7fdce2ffd700 1 -- 172.21.2.101:0/2373574099 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdcec0052e0 con 0x7fdcf0105ee0 2021-02-10T11:35:38.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.166 7fdce2ffd700 1 --2- 172.21.2.101:0/2373574099 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdcd80186c0 0x7fdcd801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:38.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.166 7fdce2ffd700 1 -- 172.21.2.101:0/2373574099 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fdcec0327a0 con 0x7fdcf0105ee0 2021-02-10T11:35:38.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.166 7fdcf645f700 1 --2- 172.21.2.101:0/2373574099 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdcd80186c0 0x7fdcd801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:38.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.166 7fdcf645f700 1 --2- 172.21.2.101:0/2373574099 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdcd80186c0 0x7fdcd801abc0 crc :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:38.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.197 7fdcf8643700 1 -- 172.21.2.101:0/2373574099 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf0105cb0 con 0x7fdcf0105ee0 2021-02-10T11:35:38.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.199 7fdce2ffd700 1 -- 172.21.2.101:0/2373574099 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdcec01f4e0 con 0x7fdcf0105ee0 2021-02-10T11:35:38.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.328 7fdcf8643700 1 -- 172.21.2.101:0/2373574099 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000063820", "target": ["mgr", ""], "group_name": "group_0000000000497714"}) v1 -- 0x7fdcf009f960 con 0x7fdcd80186c0 2021-02-10T11:35:38.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.338 7fdce2ffd700 1 -- 172.21.2.101:0/2373574099 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fdce40077a0 con 0x7fdcd80186c0 2021-02-10T11:35:38.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.345 7fdcf8643700 1 -- 172.21.2.101:0/2373574099 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdcd80186c0 msgr2=0x7fdcd801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:38.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.345 7fdcf8643700 1 --2- 172.21.2.101:0/2373574099 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdcd80186c0 0x7fdcd801abc0 crc :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:38.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.345 7fdcf8643700 1 -- 172.21.2.101:0/2373574099 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdcf0105ee0 msgr2=0x7fdcf00adf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:38.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.345 7fdcf8643700 1 --2- 172.21.2.101:0/2373574099 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdcf0105ee0 0x7fdcf00adf30 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 rx=0x7fdcec009630 tx=0x7fdcec00b980).stop 2021-02-10T11:35:38.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.345 7fdcf8643700 1 -- 172.21.2.101:0/2373574099 shutdown_connections 2021-02-10T11:35:38.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.345 7fdcf8643700 1 --2- 172.21.2.101:0/2373574099 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdcf00ae550 0x7fdcf0111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:38.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.345 7fdcf8643700 1 --2- 172.21.2.101:0/2373574099 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdcf0105ee0 0x7fdcf00adf30 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:38.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.345 7fdcf8643700 1 --2- 172.21.2.101:0/2373574099 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdcd80186c0 0x7fdcd801abc0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:38.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.345 7fdcf8643700 1 --2- 172.21.2.101:0/2373574099 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdcf009f560 0x7fdcf0110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:38.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.345 7fdcf8643700 1 -- 172.21.2.101:0/2373574099 >> 172.21.2.101:0/2373574099 conn(0x7fdcf00bb1c0 msgr2=0x7fdcf00b3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:38.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.345 7fdcf8643700 1 -- 172.21.2.101:0/2373574099 shutdown_connections 2021-02-10T11:35:38.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.345 7fdcf8643700 1 -- 172.21.2.101:0/2373574099 wait complete. 2021-02-10T11:35:38.362 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolumegroup rm cephfs group_0000000000497714 2021-02-10T11:35:38.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.768 7f1b8553b700 1 Processor -- start 2021-02-10T11:35:38.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.768 7f1b8553b700 1 -- start start 2021-02-10T11:35:38.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.769 7f1b8553b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1b800e5170 0x7f1b801b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:38.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.769 7f1b8553b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1b801b3f00 0x7f1b80077510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:38.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.769 7f1b8553b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b80078b90 con 0x7f1b800e5570 2021-02-10T11:35:38.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.769 7f1b8553b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b8007a120 con 0x7f1b801b3f00 2021-02-10T11:35:38.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.769 7f1b8553b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b8007a2d0 con 0x7f1b800e5170 2021-02-10T11:35:38.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.769 7f1b7eabe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1b801b3f00 0x7f1b80077510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:38.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.769 7f1b7eabe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1b801b3f00 0x7f1b80077510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53252/0 (socket says 172.21.2.101:53252) 2021-02-10T11:35:38.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.769 7f1b7eabe700 1 -- 172.21.2.101:0/3830223647 learned_addr learned my addr 172.21.2.101:0/3830223647 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:38.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.769 7f1b7fac0700 1 --1- 172.21.2.101:0/3830223647 >> v1:172.21.2.101:6789/0 conn(0x7f1b800e5570 0x7f1b80077b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53744/0 (socket says 172.21.2.101:53744) 2021-02-10T11:35:38.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.769 7f1b7eabe700 1 -- 172.21.2.101:0/3830223647 >> v2:172.21.2.108:3300/0 conn(0x7f1b800e5170 msgr2=0x7f1b801b38e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:35:38.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.769 7f1b7eabe700 1 --2- 172.21.2.101:0/3830223647 >> v2:172.21.2.108:3300/0 conn(0x7f1b800e5170 0x7f1b801b38e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:38.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.769 7f1b7eabe700 1 -- 172.21.2.101:0/3830223647 >> v1:172.21.2.101:6789/0 conn(0x7f1b800e5570 legacy=0x7f1b80077b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:38.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.769 7f1b7eabe700 1 -- 172.21.2.101:0/3830223647 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8007a4f0 con 0x7f1b801b3f00 2021-02-10T11:35:38.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.770 7f1b7eabe700 1 --2- 172.21.2.101:0/3830223647 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b801b3f00 0x7f1b80077510 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 rx=0x7f1b68006700 tx=0x7f1b68009b80).ready entity=mon.0 client_cookie=4ee1409d112e5be4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:38.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.770 7f1b7e2bd700 1 -- 172.21.2.101:0/3830223647 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1b6800c0c0 con 0x7f1b801b3f00 2021-02-10T11:35:38.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.770 7f1b7e2bd700 1 -- 172.21.2.101:0/3830223647 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6800c440 con 0x7f1b801b3f00 2021-02-10T11:35:38.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.770 7f1b7e2bd700 1 -- 172.21.2.101:0/3830223647 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1b6800c8a0 con 0x7f1b801b3f00 2021-02-10T11:35:38.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.770 7f1b8553b700 1 -- 172.21.2.101:0/3830223647 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b801b3f00 msgr2=0x7f1b80077510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:38.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.770 7f1b8553b700 1 --2- 172.21.2.101:0/3830223647 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b801b3f00 0x7f1b80077510 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 rx=0x7f1b68006700 tx=0x7f1b68009b80).stop 2021-02-10T11:35:38.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.771 7f1b8553b700 1 -- 172.21.2.101:0/3830223647 shutdown_connections 2021-02-10T11:35:38.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.771 7f1b8553b700 1 --2- 172.21.2.101:0/3830223647 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b801b3f00 0x7f1b80077510 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:38.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.771 7f1b8553b700 1 --2- 172.21.2.101:0/3830223647 >> v2:172.21.2.108:3300/0 conn(0x7f1b800e5170 0x7f1b801b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:38.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.771 7f1b8553b700 1 -- 172.21.2.101:0/3830223647 >> 172.21.2.101:0/3830223647 conn(0x7f1b800ce820 msgr2=0x7f1b800e4660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:38.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.771 7f1b8553b700 1 -- 172.21.2.101:0/3830223647 shutdown_connections 2021-02-10T11:35:38.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.771 7f1b8553b700 1 -- 172.21.2.101:0/3830223647 wait complete. 2021-02-10T11:35:38.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.785 7f1b8553b700 1 Processor -- start 2021-02-10T11:35:38.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.785 7f1b8553b700 1 -- start start 2021-02-10T11:35:38.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.786 7f1b8553b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b801117f0 0x7f1b800aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:38.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.786 7f1b8553b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b800af160 0x7f1b800af560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:38.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.786 7f1b8553b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1b800bb520 0x7f1b800bb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:38.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.786 7f1b8553b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8010da90 con 0x7f1b800af160 2021-02-10T11:35:38.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.786 7f1b8553b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b800afb80 con 0x7f1b800bb520 2021-02-10T11:35:38.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.786 7f1b8553b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b80111bf0 con 0x7f1b801117f0 2021-02-10T11:35:38.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.786 7f1b7eabe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b800af160 0x7f1b800af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:38.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.786 7f1b7eabe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b800af160 0x7f1b800af560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53260/0 (socket says 172.21.2.101:53260) 2021-02-10T11:35:38.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.786 7f1b7eabe700 1 -- 172.21.2.101:0/1773021859 learned_addr learned my addr 172.21.2.101:0/1773021859 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:38.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.786 7f1b7eabe700 1 -- 172.21.2.101:0/1773021859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b801117f0 msgr2=0x7f1b800aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:38.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.786 7f1b7eabe700 1 --2- 172.21.2.101:0/1773021859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b801117f0 0x7f1b800aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:38.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.786 7f1b7eabe700 1 -- 172.21.2.101:0/1773021859 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1b800bb520 msgr2=0x7f1b800bb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:38.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.786 7f1b7eabe700 1 --2- 172.21.2.101:0/1773021859 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1b800bb520 0x7f1b800bb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:38.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.786 7f1b7eabe700 1 -- 172.21.2.101:0/1773021859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b800bc9d0 con 0x7f1b800af160 2021-02-10T11:35:38.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.786 7f1b7eabe700 1 --2- 172.21.2.101:0/1773021859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b800af160 0x7f1b800af560 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 rx=0x7f1b6800b5e0 tx=0x7f1b6800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:38.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.787 7f1b77fff700 1 -- 172.21.2.101:0/1773021859 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1b6800bd70 con 0x7f1b800af160 2021-02-10T11:35:38.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.787 7f1b77fff700 1 -- 172.21.2.101:0/1773021859 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b68002030 con 0x7f1b800af160 2021-02-10T11:35:38.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.787 7f1b77fff700 1 -- 172.21.2.101:0/1773021859 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1b68002550 con 0x7f1b800af160 2021-02-10T11:35:38.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.787 7f1b8553b700 1 -- 172.21.2.101:0/1773021859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b800bc9d0 con 0x7f1b800af160 2021-02-10T11:35:38.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.787 7f1b8553b700 1 -- 172.21.2.101:0/1773021859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b800bc230 con 0x7f1b800af160 2021-02-10T11:35:38.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.788 7f1b77fff700 1 -- 172.21.2.101:0/1773021859 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1b68000c90 con 0x7f1b800af160 2021-02-10T11:35:38.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.788 7f1b77fff700 1 --2- 172.21.2.101:0/1773021859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b640186c0 0x7f1b6401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:38.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.788 7f1b77fff700 1 -- 172.21.2.101:0/1773021859 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1b680325b0 con 0x7f1b800af160 2021-02-10T11:35:38.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.788 7f1b7f2bf700 1 --2- 172.21.2.101:0/1773021859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b640186c0 0x7f1b6401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:38.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.788 7f1b7f2bf700 1 --2- 172.21.2.101:0/1773021859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b640186c0 0x7f1b6401abc0 crc :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:38.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.819 7f1b8553b700 1 -- 172.21.2.101:0/1773021859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b80105c20 con 0x7f1b800af160 2021-02-10T11:35:38.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.821 7f1b77fff700 1 -- 172.21.2.101:0/1773021859 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1b680009b0 con 0x7f1b800af160 2021-02-10T11:35:38.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.946 7f1b8553b700 1 -- 172.21.2.101:0/1773021859 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "target": ["mgr", ""], "group_name": "group_0000000000497714"}) v1 -- 0x7f1b800bc230 con 0x7f1b640186c0 2021-02-10T11:35:38.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.950 7f1b77fff700 1 -- 172.21.2.101:0/1773021859 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f1b70006e40 con 0x7f1b640186c0 2021-02-10T11:35:38.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.955 7f1b8553b700 1 -- 172.21.2.101:0/1773021859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b640186c0 msgr2=0x7f1b6401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:38.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.955 7f1b8553b700 1 --2- 172.21.2.101:0/1773021859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b640186c0 0x7f1b6401abc0 crc :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:38.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.955 7f1b8553b700 1 -- 172.21.2.101:0/1773021859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b800af160 msgr2=0x7f1b800af560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:38.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.955 7f1b8553b700 1 --2- 172.21.2.101:0/1773021859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b800af160 0x7f1b800af560 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 rx=0x7f1b6800b5e0 tx=0x7f1b6800b980).stop 2021-02-10T11:35:38.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.955 7f1b8553b700 1 -- 172.21.2.101:0/1773021859 shutdown_connections 2021-02-10T11:35:38.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.955 7f1b8553b700 1 --2- 172.21.2.101:0/1773021859 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1b800bb520 0x7f1b800bb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:38.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.955 7f1b8553b700 1 --2- 172.21.2.101:0/1773021859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b800af160 0x7f1b800af560 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:38.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.956 7f1b8553b700 1 --2- 172.21.2.101:0/1773021859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b640186c0 0x7f1b6401abc0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:38.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.956 7f1b8553b700 1 --2- 172.21.2.101:0/1773021859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b801117f0 0x7f1b800aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:38.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.956 7f1b8553b700 1 -- 172.21.2.101:0/1773021859 >> 172.21.2.101:0/1773021859 conn(0x7f1b80105e50 msgr2=0x7f1b800b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:38.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.956 7f1b8553b700 1 -- 172.21.2.101:0/1773021859 shutdown_connections 2021-02-10T11:35:38.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:38.956 7f1b8553b700 1 -- 172.21.2.101:0/1773021859 wait complete. 2021-02-10T11:35:38.971 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:35:38.972 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:35:39.316 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:35:39.661 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:35:39.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:39.662 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:35:39.662 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:35:39.999 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:35:40.000 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:35:40.000 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:35:40.000 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:35:40.000 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:35:40.353 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[124259]: fuse finished with error 0 and tester_r 0 2021-02-10T11:35:40.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:40.354 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:35:40.354 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:35:40.689 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:35:40.689 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:35:40.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:35:40.689 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:35:41.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:41.024 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:35:41.024 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:41.360 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:35:41.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:35:41.361 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:35:41.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:35:41.361 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:41.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:41.696 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:35:41.696 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:35:42.030 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:35:42.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:35:42.031 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:35:42.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:35:42.031 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:35:42.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:42.365 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:35:42.365 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:35:42.699 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:35:42.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:35:42.700 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:35:42.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:35:42.700 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:35:43.036 DEBUG:teuthology.orchestra.run.gibba001:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-02-10T11:35:43.039 DEBUG:teuthology.orchestra.run.gibba008:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-02-10T11:35:43.398 INFO:teuthology.orchestra.run.gibba001.stdout:[global] 2021-02-10T11:35:43.398 INFO:teuthology.orchestra.run.gibba001.stdout: chdir = "" 2021-02-10T11:35:43.398 INFO:teuthology.orchestra.run.gibba001.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-02-10T11:35:43.398 INFO:teuthology.orchestra.run.gibba001.stdout: auth supported = cephx 2021-02-10T11:35:43.399 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.399 INFO:teuthology.orchestra.run.gibba001.stdout: filestore xattr use omap = true 2021-02-10T11:35:43.399 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.399 INFO:teuthology.orchestra.run.gibba001.stdout: mon clock drift allowed = 1.000 2021-02-10T11:35:43.399 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.400 INFO:teuthology.orchestra.run.gibba001.stdout: osd crush chooseleaf type = 0 2021-02-10T11:35:43.400 INFO:teuthology.orchestra.run.gibba001.stdout: auth debug = true 2021-02-10T11:35:43.400 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.400 INFO:teuthology.orchestra.run.gibba001.stdout: ms die on old message = true 2021-02-10T11:35:43.400 INFO:teuthology.orchestra.run.gibba001.stdout: ms die on bug = true 2021-02-10T11:35:43.400 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.401 INFO:teuthology.orchestra.run.gibba001.stdout: mon max pg per osd = 10000 # >= luminous 2021-02-10T11:35:43.401 INFO:teuthology.orchestra.run.gibba001.stdout: mon pg warn max object skew = 0 2021-02-10T11:35:43.401 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.401 INFO:teuthology.orchestra.run.gibba001.stdout: osd pool default size = 2 2021-02-10T11:35:43.401 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.401 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd allow primary affinity = true 2021-02-10T11:35:43.402 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd allow pg remap = true 2021-02-10T11:35:43.402 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on legacy crush tunables = false 2021-02-10T11:35:43.402 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on crush straw calc version zero = false 2021-02-10T11:35:43.402 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on no sortbitwise = false 2021-02-10T11:35:43.402 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on osd down out interval zero = false 2021-02-10T11:35:43.403 INFO:teuthology.orchestra.run.gibba001.stdout: mon warn on too few osds = false 2021-02-10T11:35:43.403 INFO:teuthology.orchestra.run.gibba001.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-02-10T11:35:43.403 INFO:teuthology.orchestra.run.gibba001.stdout: mon_warn_on_pool_no_redundancy = false 2021-02-10T11:35:43.403 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.403 INFO:teuthology.orchestra.run.gibba001.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-02-10T11:35:43.404 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.404 INFO:teuthology.orchestra.run.gibba001.stdout: osd default data pool replay window = 5 2021-02-10T11:35:43.404 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.404 INFO:teuthology.orchestra.run.gibba001.stdout: mon allow pool delete = true 2021-02-10T11:35:43.404 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.404 INFO:teuthology.orchestra.run.gibba001.stdout: mon cluster log file level = debug 2021-02-10T11:35:43.405 INFO:teuthology.orchestra.run.gibba001.stdout: debug asserts on shutdown = true 2021-02-10T11:35:43.405 INFO:teuthology.orchestra.run.gibba001.stdout: mon health detail to clog = false 2021-02-10T11:35:43.405 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.405 INFO:teuthology.orchestra.run.gibba001.stdout: # we see this fail in qa on *nautilus*; bump up retries 2021-02-10T11:35:43.405 INFO:teuthology.orchestra.run.gibba001.stdout: mon_client_directed_command_retry = 4 2021-02-10T11:35:43.406 INFO:teuthology.orchestra.run.gibba001.stdout: mon host = "172.21.2.101,172.21.2.108,[v2:172.21.2.108:3301,v1:172.21.2.108:6790]" 2021-02-10T11:35:43.406 INFO:teuthology.orchestra.run.gibba001.stdout: fsid = a6663ba9-2653-4542-8b6a-9323b4e7b545 2021-02-10T11:35:43.406 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.406 INFO:teuthology.orchestra.run.gibba001.stdout:[osd] 2021-02-10T11:35:43.406 INFO:teuthology.orchestra.run.gibba001.stdout: osd journal size = 100 2021-02-10T11:35:43.407 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.407 INFO:teuthology.orchestra.run.gibba001.stdout: osd scrub load threshold = 5.0 2021-02-10T11:35:43.407 INFO:teuthology.orchestra.run.gibba001.stdout: osd scrub max interval = 600 2021-02-10T11:35:43.407 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.407 INFO:teuthology.orchestra.run.gibba001.stdout: osd recover clone overlap = true 2021-02-10T11:35:43.407 INFO:teuthology.orchestra.run.gibba001.stdout: osd recovery max chunk = 1048576 2021-02-10T11:35:43.408 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.408 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug shutdown = true 2021-02-10T11:35:43.408 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug op order = true 2021-02-10T11:35:43.408 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug verify stray on activate = true 2021-02-10T11:35:43.408 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.408 INFO:teuthology.orchestra.run.gibba001.stdout: osd open classes on start = true 2021-02-10T11:35:43.409 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug pg log writeout = true 2021-02-10T11:35:43.409 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.409 INFO:teuthology.orchestra.run.gibba001.stdout: osd deep scrub update digest min age = 30 2021-02-10T11:35:43.409 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.409 INFO:teuthology.orchestra.run.gibba001.stdout: osd map max advance = 10 2021-02-10T11:35:43.410 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.410 INFO:teuthology.orchestra.run.gibba001.stdout: journal zero on create = true 2021-02-10T11:35:43.410 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.410 INFO:teuthology.orchestra.run.gibba001.stdout: filestore ondisk finisher threads = 3 2021-02-10T11:35:43.410 INFO:teuthology.orchestra.run.gibba001.stdout: filestore apply finisher threads = 3 2021-02-10T11:35:43.410 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.411 INFO:teuthology.orchestra.run.gibba001.stdout: bdev debug aio = true 2021-02-10T11:35:43.411 INFO:teuthology.orchestra.run.gibba001.stdout: osd debug misdirected ops = true 2021-02-10T11:35:43.411 INFO:teuthology.orchestra.run.gibba001.stdout: bdev async discard = True 2021-02-10T11:35:43.411 INFO:teuthology.orchestra.run.gibba001.stdout: bdev enable discard = True 2021-02-10T11:35:43.411 INFO:teuthology.orchestra.run.gibba001.stdout: bluestore allocator = bitmap 2021-02-10T11:35:43.411 INFO:teuthology.orchestra.run.gibba001.stdout: bluestore block size = 96636764160 2021-02-10T11:35:43.412 INFO:teuthology.orchestra.run.gibba001.stdout: bluestore fsck on mount = True 2021-02-10T11:35:43.412 INFO:teuthology.orchestra.run.gibba001.stdout: debug bluefs = 20 2021-02-10T11:35:43.412 INFO:teuthology.orchestra.run.gibba001.stdout: debug bluestore = 20 2021-02-10T11:35:43.412 INFO:teuthology.orchestra.run.gibba001.stdout: debug filestore = 20 2021-02-10T11:35:43.412 INFO:teuthology.orchestra.run.gibba001.stdout: debug journal = 20 2021-02-10T11:35:43.413 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 20 2021-02-10T11:35:43.413 INFO:teuthology.orchestra.run.gibba001.stdout: debug osd = 25 2021-02-10T11:35:43.413 INFO:teuthology.orchestra.run.gibba001.stdout: debug rocksdb = 10 2021-02-10T11:35:43.413 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd backfillfull_ratio = 0.85 2021-02-10T11:35:43.413 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd full ratio = 0.9 2021-02-10T11:35:43.414 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd nearfull ratio = 0.8 2021-02-10T11:35:43.414 INFO:teuthology.orchestra.run.gibba001.stdout: osd failsafe full ratio = 0.95 2021-02-10T11:35:43.414 INFO:teuthology.orchestra.run.gibba001.stdout: osd objectstore = bluestore 2021-02-10T11:35:43.414 INFO:teuthology.orchestra.run.gibba001.stdout: osd op complaint time = 180 2021-02-10T11:35:43.414 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.414 INFO:teuthology.orchestra.run.gibba001.stdout:[mgr] 2021-02-10T11:35:43.415 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 1 2021-02-10T11:35:43.415 INFO:teuthology.orchestra.run.gibba001.stdout: debug mgr = 20 2021-02-10T11:35:43.415 INFO:teuthology.orchestra.run.gibba001.stdout: debug mon = 20 2021-02-10T11:35:43.415 INFO:teuthology.orchestra.run.gibba001.stdout: debug auth = 20 2021-02-10T11:35:43.416 INFO:teuthology.orchestra.run.gibba001.stdout: mon reweight min pgs per osd = 4 2021-02-10T11:35:43.416 INFO:teuthology.orchestra.run.gibba001.stdout: mon reweight min bytes per osd = 10 2021-02-10T11:35:43.416 INFO:teuthology.orchestra.run.gibba001.stdout: debug client = 10 2021-02-10T11:35:43.416 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.416 INFO:teuthology.orchestra.run.gibba001.stdout:[mon] 2021-02-10T11:35:43.417 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 1 2021-02-10T11:35:43.417 INFO:teuthology.orchestra.run.gibba001.stdout: debug mon = 20 2021-02-10T11:35:43.417 INFO:teuthology.orchestra.run.gibba001.stdout: debug paxos = 20 2021-02-10T11:35:43.417 INFO:teuthology.orchestra.run.gibba001.stdout: debug auth = 20 2021-02-10T11:35:43.417 INFO:teuthology.orchestra.run.gibba001.stdout: mon data avail warn = 5 2021-02-10T11:35:43.417 INFO:teuthology.orchestra.run.gibba001.stdout: mon mgr mkfs grace = 240 2021-02-10T11:35:43.418 INFO:teuthology.orchestra.run.gibba001.stdout: mon reweight min pgs per osd = 4 2021-02-10T11:35:43.418 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd reporter subtree level = osd 2021-02-10T11:35:43.418 INFO:teuthology.orchestra.run.gibba001.stdout: mon osd prime pg temp = true 2021-02-10T11:35:43.419 INFO:teuthology.orchestra.run.gibba001.stdout: mon reweight min bytes per osd = 10 2021-02-10T11:35:43.419 INFO:teuthology.orchestra.run.gibba001.stdout: mon op complaint time = 120 2021-02-10T11:35:43.419 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:43.419 INFO:teuthology.orchestra.run.gibba001.stdout:[client] 2021-02-10T11:35:43.419 INFO:teuthology.orchestra.run.gibba001.stdout: rgw cache enabled = true 2021-02-10T11:35:43.419 INFO:teuthology.orchestra.run.gibba001.stdout: rgw enable ops log = true 2021-02-10T11:35:43.419 INFO:teuthology.orchestra.run.gibba001.stdout: rgw enable usage log = true 2021-02-10T11:35:43.419 INFO:teuthology.orchestra.run.gibba001.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-02-10T11:35:43.419 INFO:teuthology.orchestra.run.gibba001.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-02-10T11:35:43.419 INFO:teuthology.orchestra.run.gibba001.stdout: client mount timeout = 600 2021-02-10T11:35:43.419 INFO:teuthology.orchestra.run.gibba001.stdout: debug client = 20 2021-02-10T11:35:43.419 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 1 2021-02-10T11:35:43.419 INFO:teuthology.orchestra.run.gibba001.stdout: pid file = "" 2021-02-10T11:35:43.420 INFO:teuthology.orchestra.run.gibba001.stdout:[mon.a] 2021-02-10T11:35:43.420 INFO:teuthology.orchestra.run.gibba001.stdout:[mon.b] 2021-02-10T11:35:43.420 INFO:teuthology.orchestra.run.gibba001.stdout:[mon.c] 2021-02-10T11:35:43.420 INFO:teuthology.orchestra.run.gibba001.stdout:[mds.a] 2021-02-10T11:35:43.420 INFO:teuthology.orchestra.run.gibba001.stdout:[mds.b] 2021-02-10T11:35:43.420 INFO:teuthology.orchestra.run.gibba001.stdout:[mds] 2021-02-10T11:35:43.420 INFO:teuthology.orchestra.run.gibba001.stdout: debug mds = 20 2021-02-10T11:35:43.420 INFO:teuthology.orchestra.run.gibba001.stdout: debug ms = 1 2021-02-10T11:35:43.420 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal frag = True 2021-02-10T11:35:43.420 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal fragment size max = 10000 2021-02-10T11:35:43.420 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal merge size = 5 2021-02-10T11:35:43.420 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal split bits = 3 2021-02-10T11:35:43.420 INFO:teuthology.orchestra.run.gibba001.stdout: mds bal split size = 100 2021-02-10T11:35:43.420 INFO:teuthology.orchestra.run.gibba001.stdout: mds debug frag = True 2021-02-10T11:35:43.421 INFO:teuthology.orchestra.run.gibba001.stdout: mds debug scatterstat = True 2021-02-10T11:35:43.421 INFO:teuthology.orchestra.run.gibba001.stdout: mds op complaint time = 180 2021-02-10T11:35:43.421 INFO:teuthology.orchestra.run.gibba001.stdout: mds verify scatter = True 2021-02-10T11:35:43.421 INFO:teuthology.orchestra.run.gibba001.stdout: osd op complaint time = 180 2021-02-10T11:35:43.421 INFO:teuthology.orchestra.run.gibba001.stdout:[client.alice] 2021-02-10T11:35:43.422 INFO:teuthology.orchestra.run.gibba008.stdout:[global] 2021-02-10T11:35:43.422 INFO:teuthology.orchestra.run.gibba008.stdout: chdir = "" 2021-02-10T11:35:43.423 INFO:teuthology.orchestra.run.gibba008.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-02-10T11:35:43.423 INFO:teuthology.orchestra.run.gibba008.stdout: auth supported = cephx 2021-02-10T11:35:43.423 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.423 INFO:teuthology.orchestra.run.gibba008.stdout: filestore xattr use omap = true 2021-02-10T11:35:43.423 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.423 INFO:teuthology.orchestra.run.gibba008.stdout: mon clock drift allowed = 1.000 2021-02-10T11:35:43.423 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.423 INFO:teuthology.orchestra.run.gibba008.stdout: osd crush chooseleaf type = 0 2021-02-10T11:35:43.423 INFO:teuthology.orchestra.run.gibba008.stdout: auth debug = true 2021-02-10T11:35:43.423 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.423 INFO:teuthology.orchestra.run.gibba008.stdout: ms die on old message = true 2021-02-10T11:35:43.423 INFO:teuthology.orchestra.run.gibba008.stdout: ms die on bug = true 2021-02-10T11:35:43.423 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.424 INFO:teuthology.orchestra.run.gibba008.stdout: mon max pg per osd = 10000 # >= luminous 2021-02-10T11:35:43.424 INFO:teuthology.orchestra.run.gibba008.stdout: mon pg warn max object skew = 0 2021-02-10T11:35:43.424 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.424 INFO:teuthology.orchestra.run.gibba008.stdout: osd pool default size = 2 2021-02-10T11:35:43.424 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.424 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd allow primary affinity = true 2021-02-10T11:35:43.424 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd allow pg remap = true 2021-02-10T11:35:43.424 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on legacy crush tunables = false 2021-02-10T11:35:43.424 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on crush straw calc version zero = false 2021-02-10T11:35:43.424 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on no sortbitwise = false 2021-02-10T11:35:43.424 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on osd down out interval zero = false 2021-02-10T11:35:43.424 INFO:teuthology.orchestra.run.gibba008.stdout: mon warn on too few osds = false 2021-02-10T11:35:43.424 INFO:teuthology.orchestra.run.gibba008.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-02-10T11:35:43.424 INFO:teuthology.orchestra.run.gibba008.stdout: mon_warn_on_pool_no_redundancy = false 2021-02-10T11:35:43.425 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.425 INFO:teuthology.orchestra.run.gibba008.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-02-10T11:35:43.425 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.425 INFO:teuthology.orchestra.run.gibba008.stdout: osd default data pool replay window = 5 2021-02-10T11:35:43.425 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.425 INFO:teuthology.orchestra.run.gibba008.stdout: mon allow pool delete = true 2021-02-10T11:35:43.425 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.425 INFO:teuthology.orchestra.run.gibba008.stdout: mon cluster log file level = debug 2021-02-10T11:35:43.425 INFO:teuthology.orchestra.run.gibba008.stdout: debug asserts on shutdown = true 2021-02-10T11:35:43.425 INFO:teuthology.orchestra.run.gibba008.stdout: mon health detail to clog = false 2021-02-10T11:35:43.425 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.425 INFO:teuthology.orchestra.run.gibba008.stdout: # we see this fail in qa on *nautilus*; bump up retries 2021-02-10T11:35:43.425 INFO:teuthology.orchestra.run.gibba008.stdout: mon_client_directed_command_retry = 4 2021-02-10T11:35:43.426 INFO:teuthology.orchestra.run.gibba008.stdout: mon host = "172.21.2.101,172.21.2.108,[v2:172.21.2.108:3301,v1:172.21.2.108:6790]" 2021-02-10T11:35:43.426 INFO:teuthology.orchestra.run.gibba008.stdout: fsid = a6663ba9-2653-4542-8b6a-9323b4e7b545 2021-02-10T11:35:43.426 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.426 INFO:teuthology.orchestra.run.gibba008.stdout:[osd] 2021-02-10T11:35:43.426 INFO:teuthology.orchestra.run.gibba008.stdout: osd journal size = 100 2021-02-10T11:35:43.426 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.426 INFO:teuthology.orchestra.run.gibba008.stdout: osd scrub load threshold = 5.0 2021-02-10T11:35:43.426 INFO:teuthology.orchestra.run.gibba008.stdout: osd scrub max interval = 600 2021-02-10T11:35:43.426 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.426 INFO:teuthology.orchestra.run.gibba008.stdout: osd recover clone overlap = true 2021-02-10T11:35:43.426 INFO:teuthology.orchestra.run.gibba008.stdout: osd recovery max chunk = 1048576 2021-02-10T11:35:43.426 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.426 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug shutdown = true 2021-02-10T11:35:43.426 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug op order = true 2021-02-10T11:35:43.427 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug verify stray on activate = true 2021-02-10T11:35:43.427 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.427 INFO:teuthology.orchestra.run.gibba008.stdout: osd open classes on start = true 2021-02-10T11:35:43.427 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug pg log writeout = true 2021-02-10T11:35:43.427 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.427 INFO:teuthology.orchestra.run.gibba008.stdout: osd deep scrub update digest min age = 30 2021-02-10T11:35:43.427 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.427 INFO:teuthology.orchestra.run.gibba008.stdout: osd map max advance = 10 2021-02-10T11:35:43.427 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.427 INFO:teuthology.orchestra.run.gibba008.stdout: journal zero on create = true 2021-02-10T11:35:43.427 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.427 INFO:teuthology.orchestra.run.gibba008.stdout: filestore ondisk finisher threads = 3 2021-02-10T11:35:43.427 INFO:teuthology.orchestra.run.gibba008.stdout: filestore apply finisher threads = 3 2021-02-10T11:35:43.427 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.428 INFO:teuthology.orchestra.run.gibba008.stdout: bdev debug aio = true 2021-02-10T11:35:43.428 INFO:teuthology.orchestra.run.gibba008.stdout: osd debug misdirected ops = true 2021-02-10T11:35:43.428 INFO:teuthology.orchestra.run.gibba008.stdout: bdev async discard = True 2021-02-10T11:35:43.428 INFO:teuthology.orchestra.run.gibba008.stdout: bdev enable discard = True 2021-02-10T11:35:43.428 INFO:teuthology.orchestra.run.gibba008.stdout: bluestore allocator = bitmap 2021-02-10T11:35:43.428 INFO:teuthology.orchestra.run.gibba008.stdout: bluestore block size = 96636764160 2021-02-10T11:35:43.428 INFO:teuthology.orchestra.run.gibba008.stdout: bluestore fsck on mount = True 2021-02-10T11:35:43.428 INFO:teuthology.orchestra.run.gibba008.stdout: debug bluefs = 20 2021-02-10T11:35:43.428 INFO:teuthology.orchestra.run.gibba008.stdout: debug bluestore = 20 2021-02-10T11:35:43.428 INFO:teuthology.orchestra.run.gibba008.stdout: debug filestore = 20 2021-02-10T11:35:43.428 INFO:teuthology.orchestra.run.gibba008.stdout: debug journal = 20 2021-02-10T11:35:43.428 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 20 2021-02-10T11:35:43.428 INFO:teuthology.orchestra.run.gibba008.stdout: debug osd = 25 2021-02-10T11:35:43.428 INFO:teuthology.orchestra.run.gibba008.stdout: debug rocksdb = 10 2021-02-10T11:35:43.428 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd backfillfull_ratio = 0.85 2021-02-10T11:35:43.429 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd full ratio = 0.9 2021-02-10T11:35:43.429 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd nearfull ratio = 0.8 2021-02-10T11:35:43.429 INFO:teuthology.orchestra.run.gibba008.stdout: osd failsafe full ratio = 0.95 2021-02-10T11:35:43.429 INFO:teuthology.orchestra.run.gibba008.stdout: osd objectstore = bluestore 2021-02-10T11:35:43.429 INFO:teuthology.orchestra.run.gibba008.stdout: osd op complaint time = 180 2021-02-10T11:35:43.429 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.429 INFO:teuthology.orchestra.run.gibba008.stdout:[mgr] 2021-02-10T11:35:43.429 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 1 2021-02-10T11:35:43.429 INFO:teuthology.orchestra.run.gibba008.stdout: debug mgr = 20 2021-02-10T11:35:43.429 INFO:teuthology.orchestra.run.gibba008.stdout: debug mon = 20 2021-02-10T11:35:43.429 INFO:teuthology.orchestra.run.gibba008.stdout: debug auth = 20 2021-02-10T11:35:43.429 INFO:teuthology.orchestra.run.gibba008.stdout: mon reweight min pgs per osd = 4 2021-02-10T11:35:43.429 INFO:teuthology.orchestra.run.gibba008.stdout: mon reweight min bytes per osd = 10 2021-02-10T11:35:43.430 INFO:teuthology.orchestra.run.gibba008.stdout: debug client = 10 2021-02-10T11:35:43.430 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.430 INFO:teuthology.orchestra.run.gibba008.stdout:[mon] 2021-02-10T11:35:43.430 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 1 2021-02-10T11:35:43.430 INFO:teuthology.orchestra.run.gibba008.stdout: debug mon = 20 2021-02-10T11:35:43.430 INFO:teuthology.orchestra.run.gibba008.stdout: debug paxos = 20 2021-02-10T11:35:43.430 INFO:teuthology.orchestra.run.gibba008.stdout: debug auth = 20 2021-02-10T11:35:43.430 INFO:teuthology.orchestra.run.gibba008.stdout: mon data avail warn = 5 2021-02-10T11:35:43.430 INFO:teuthology.orchestra.run.gibba008.stdout: mon mgr mkfs grace = 240 2021-02-10T11:35:43.430 INFO:teuthology.orchestra.run.gibba008.stdout: mon reweight min pgs per osd = 4 2021-02-10T11:35:43.430 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd reporter subtree level = osd 2021-02-10T11:35:43.430 INFO:teuthology.orchestra.run.gibba008.stdout: mon osd prime pg temp = true 2021-02-10T11:35:43.430 INFO:teuthology.orchestra.run.gibba008.stdout: mon reweight min bytes per osd = 10 2021-02-10T11:35:43.431 INFO:teuthology.orchestra.run.gibba008.stdout: mon op complaint time = 120 2021-02-10T11:35:43.431 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-02-10T11:35:43.431 INFO:teuthology.orchestra.run.gibba008.stdout:[client] 2021-02-10T11:35:43.431 INFO:teuthology.orchestra.run.gibba008.stdout: rgw cache enabled = true 2021-02-10T11:35:43.431 INFO:teuthology.orchestra.run.gibba008.stdout: rgw enable ops log = true 2021-02-10T11:35:43.431 INFO:teuthology.orchestra.run.gibba008.stdout: rgw enable usage log = true 2021-02-10T11:35:43.431 INFO:teuthology.orchestra.run.gibba008.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-02-10T11:35:43.431 INFO:teuthology.orchestra.run.gibba008.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-02-10T11:35:43.431 INFO:teuthology.orchestra.run.gibba008.stdout: client mount timeout = 600 2021-02-10T11:35:43.431 INFO:teuthology.orchestra.run.gibba008.stdout: debug client = 20 2021-02-10T11:35:43.431 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 1 2021-02-10T11:35:43.431 INFO:teuthology.orchestra.run.gibba008.stdout: pid file = "" 2021-02-10T11:35:43.431 INFO:teuthology.orchestra.run.gibba008.stdout:[mon.a] 2021-02-10T11:35:43.432 INFO:teuthology.orchestra.run.gibba008.stdout:[mon.b] 2021-02-10T11:35:43.432 INFO:teuthology.orchestra.run.gibba008.stdout:[mon.c] 2021-02-10T11:35:43.432 INFO:teuthology.orchestra.run.gibba008.stdout:[mds.a] 2021-02-10T11:35:43.432 INFO:teuthology.orchestra.run.gibba008.stdout:[mds.b] 2021-02-10T11:35:43.432 INFO:teuthology.orchestra.run.gibba008.stdout:[mds] 2021-02-10T11:35:43.432 INFO:teuthology.orchestra.run.gibba008.stdout: debug mds = 20 2021-02-10T11:35:43.432 INFO:teuthology.orchestra.run.gibba008.stdout: debug ms = 1 2021-02-10T11:35:43.432 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal frag = True 2021-02-10T11:35:43.432 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal fragment size max = 10000 2021-02-10T11:35:43.432 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal merge size = 5 2021-02-10T11:35:43.432 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal split bits = 3 2021-02-10T11:35:43.433 INFO:teuthology.orchestra.run.gibba008.stdout: mds bal split size = 100 2021-02-10T11:35:43.433 INFO:teuthology.orchestra.run.gibba008.stdout: mds debug frag = True 2021-02-10T11:35:43.433 INFO:teuthology.orchestra.run.gibba008.stdout: mds debug scatterstat = True 2021-02-10T11:35:43.433 INFO:teuthology.orchestra.run.gibba008.stdout: mds op complaint time = 180 2021-02-10T11:35:43.433 INFO:teuthology.orchestra.run.gibba008.stdout: mds verify scatter = True 2021-02-10T11:35:43.433 INFO:teuthology.orchestra.run.gibba008.stdout: osd op complaint time = 180 2021-02-10T11:35:43.433 INFO:teuthology.orchestra.run.gibba008.stdout:[client.alice] 2021-02-10T11:35:43.433 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-02-10T11:35:43.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.838 7f374083f700 1 Processor -- start 2021-02-10T11:35:43.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.838 7f374083f700 1 -- start start 2021-02-10T11:35:43.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.839 7f374083f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f373c0d0340 0x7f373c0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:43.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.839 7f374083f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f373c0d1c70 0x7f373c0efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:43.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.839 7f374083f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f373c0f0b10 con 0x7f373c0d1870 2021-02-10T11:35:43.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.839 7f374083f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f373c0e5ed0 con 0x7f373c0d1c70 2021-02-10T11:35:43.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.839 7f374083f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f373c1b0be0 con 0x7f373c0d0340 2021-02-10T11:35:43.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.839 7f373ae1d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f373c0d1c70 0x7f373c0efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:43.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.839 7f373a61c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f373c0d1870 0x7f373c1b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53758/0 (socket says 172.21.2.101:53758) 2021-02-10T11:35:43.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.839 7f373a61c700 1 -- 172.21.2.101:0/4201258997 learned_addr learned my addr 172.21.2.101:0/4201258997 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:43.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.839 7f373ae1d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f373c0d1c70 0x7f373c0efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53268/0 (socket says 172.21.2.101:53268) 2021-02-10T11:35:43.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.839 7f373ae1d700 1 -- 172.21.2.101:0/4201258997 >> v2:172.21.2.108:3300/0 conn(0x7f373c0d0340 msgr2=0x7f373c0ef640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:35:43.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.839 7f373ae1d700 1 --2- 172.21.2.101:0/4201258997 >> v2:172.21.2.108:3300/0 conn(0x7f373c0d0340 0x7f373c0ef640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:43.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.839 7f373ae1d700 1 -- 172.21.2.101:0/4201258997 >> v1:172.21.2.101:6789/0 conn(0x7f373c0d1870 legacy=0x7f373c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:43.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.839 7f373961a700 1 -- 172.21.2.101:0/4201258997 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1206438539 0 0) 0x7f37240010b0 con 0x7f373c0d1870 2021-02-10T11:35:43.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.839 7f373ae1d700 1 -- 172.21.2.101:0/4201258997 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373c0f20a0 con 0x7f373c0d1c70 2021-02-10T11:35:43.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.839 7f373ae1d700 1 --2- 172.21.2.101:0/4201258997 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f373c0d1c70 0x7f373c0efc60 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 rx=0x7f3734006e30 tx=0x7f3734009de0).ready entity=mon.0 client_cookie=9151acb14cfd206a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:43.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.840 7f373961a700 1 -- 172.21.2.101:0/4201258997 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f373400b300 con 0x7f373c0d1c70 2021-02-10T11:35:43.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.840 7f373961a700 1 -- 172.21.2.101:0/4201258997 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373400b300 con 0x7f373c0d1c70 2021-02-10T11:35:43.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.840 7f373961a700 1 -- 172.21.2.101:0/4201258997 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f373400b580 con 0x7f373c0d1c70 2021-02-10T11:35:43.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.840 7f374083f700 1 -- 172.21.2.101:0/4201258997 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f373c0d1c70 msgr2=0x7f373c0efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:43.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.840 7f374083f700 1 --2- 172.21.2.101:0/4201258997 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f373c0d1c70 0x7f373c0efc60 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 rx=0x7f3734006e30 tx=0x7f3734009de0).stop 2021-02-10T11:35:43.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.840 7f374083f700 1 -- 172.21.2.101:0/4201258997 shutdown_connections 2021-02-10T11:35:43.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.840 7f374083f700 1 --2- 172.21.2.101:0/4201258997 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f373c0d1c70 0x7f373c0efc60 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:43.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.840 7f374083f700 1 --2- 172.21.2.101:0/4201258997 >> v2:172.21.2.108:3300/0 conn(0x7f373c0d0340 0x7f373c0ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:43.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.840 7f374083f700 1 -- 172.21.2.101:0/4201258997 >> 172.21.2.101:0/4201258997 conn(0x7f373c0ce450 msgr2=0x7f373c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:43.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.841 7f374083f700 1 -- 172.21.2.101:0/4201258997 shutdown_connections 2021-02-10T11:35:43.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.841 7f374083f700 1 -- 172.21.2.101:0/4201258997 wait complete. 2021-02-10T11:35:43.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.855 7f374083f700 1 Processor -- start 2021-02-10T11:35:43.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.855 7f374083f700 1 -- start start 2021-02-10T11:35:43.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.855 7f374083f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f373c0ee060 0x7f373c0ade80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:43.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.855 7f374083f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f373c0ee460 0x7f373c1a7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:43.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.855 7f374083f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f373c1a8180 0x7f373c094860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:43.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.855 7f374083f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c0ae4a0 con 0x7f373c1a8180 2021-02-10T11:35:43.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.855 7f374083f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c094e80 con 0x7f373c0ee460 2021-02-10T11:35:43.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.855 7f374083f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373c0950a0 con 0x7f373c0ee060 2021-02-10T11:35:43.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.856 7f373ae1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f373c1a8180 0x7f373c094860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:43.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.856 7f373ae1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f373c1a8180 0x7f373c094860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53274/0 (socket says 172.21.2.101:53274) 2021-02-10T11:35:43.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.856 7f373ae1d700 1 -- 172.21.2.101:0/65596713 learned_addr learned my addr 172.21.2.101:0/65596713 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:43.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.856 7f373ae1d700 1 -- 172.21.2.101:0/65596713 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f373c0ee060 msgr2=0x7f373c0ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:43.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.856 7f373ae1d700 1 --2- 172.21.2.101:0/65596713 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f373c0ee060 0x7f373c0ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:43.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.856 7f373ae1d700 1 -- 172.21.2.101:0/65596713 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f373c0ee460 msgr2=0x7f373c1a7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:43.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.856 7f373ae1d700 1 --2- 172.21.2.101:0/65596713 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f373c0ee460 0x7f373c1a7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:43.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.856 7f373ae1d700 1 -- 172.21.2.101:0/65596713 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373c0952c0 con 0x7f373c1a8180 2021-02-10T11:35:43.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.856 7f373ae1d700 1 --2- 172.21.2.101:0/65596713 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f373c1a8180 0x7f373c094860 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 rx=0x7f3734006b60 tx=0x7f373400bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:43.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.856 7f3732ffd700 1 -- 172.21.2.101:0/65596713 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3734009690 con 0x7f373c1a8180 2021-02-10T11:35:43.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.856 7f3732ffd700 1 -- 172.21.2.101:0/65596713 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373400c2f0 con 0x7f373c1a8180 2021-02-10T11:35:43.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.856 7f374083f700 1 -- 172.21.2.101:0/65596713 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373c0952c0 con 0x7f373c1a8180 2021-02-10T11:35:43.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.856 7f374083f700 1 -- 172.21.2.101:0/65596713 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f373c0952c0 con 0x7f373c1a8180 2021-02-10T11:35:43.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.856 7f3732ffd700 1 -- 172.21.2.101:0/65596713 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f373400c810 con 0x7f373c1a8180 2021-02-10T11:35:43.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.857 7f3732ffd700 1 -- 172.21.2.101:0/65596713 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3734009690 con 0x7f373c1a8180 2021-02-10T11:35:43.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.857 7f3732ffd700 1 --2- 172.21.2.101:0/65596713 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3720018730 0x7f372001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:43.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.857 7f3732ffd700 1 -- 172.21.2.101:0/65596713 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3734032b80 con 0x7f373c1a8180 2021-02-10T11:35:43.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.857 7f373a61c700 1 --2- 172.21.2.101:0/65596713 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3720018730 0x7f372001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:43.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.857 7f373a61c700 1 --2- 172.21.2.101:0/65596713 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3720018730 0x7f372001ac10 crc :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:43.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.889 7f374083f700 1 -- 172.21.2.101:0/65596713 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f373c094330 con 0x7f373c1a8180 2021-02-10T11:35:43.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:43.891 7f3732ffd700 1 -- 172.21.2.101:0/65596713 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3734003070 con 0x7f373c1a8180 2021-02-10T11:35:44.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.039 7f374083f700 1 -- 172.21.2.101:0/65596713 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f373c094330 con 0x7f373c1a8180 2021-02-10T11:35:44.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.043 7f3732ffd700 1 -- 172.21.2.101:0/65596713 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3734003070 con 0x7f373c1a8180 2021-02-10T11:35:44.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.048 7f374083f700 1 -- 172.21.2.101:0/65596713 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3720018730 msgr2=0x7f372001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:44.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.048 7f374083f700 1 --2- 172.21.2.101:0/65596713 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3720018730 0x7f372001ac10 crc :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:44.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.048 7f374083f700 1 -- 172.21.2.101:0/65596713 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f373c1a8180 msgr2=0x7f373c094860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:44.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.048 7f374083f700 1 --2- 172.21.2.101:0/65596713 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f373c1a8180 0x7f373c094860 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 rx=0x7f3734006b60 tx=0x7f373400bd40).stop 2021-02-10T11:35:44.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.048 7f374083f700 1 -- 172.21.2.101:0/65596713 shutdown_connections 2021-02-10T11:35:44.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.048 7f374083f700 1 --2- 172.21.2.101:0/65596713 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f373c1a8180 0x7f373c094860 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:44.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.048 7f374083f700 1 --2- 172.21.2.101:0/65596713 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f373c0ee460 0x7f373c1a7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:44.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.048 7f374083f700 1 --2- 172.21.2.101:0/65596713 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3720018730 0x7f372001ac10 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:44.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.048 7f374083f700 1 --2- 172.21.2.101:0/65596713 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f373c0ee060 0x7f373c0ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:44.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.048 7f374083f700 1 -- 172.21.2.101:0/65596713 >> 172.21.2.101:0/65596713 conn(0x7f373c0955e0 msgr2=0x7f373c0bfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:44.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.048 7f374083f700 1 -- 172.21.2.101:0/65596713 shutdown_connections 2021-02-10T11:35:44.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.048 7f374083f700 1 -- 172.21.2.101:0/65596713 wait complete. 2021-02-10T11:35:44.064 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm client.alice debug_client 2021-02-10T11:35:44.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.471 7f31497a0700 1 Processor -- start 2021-02-10T11:35:44.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.471 7f31497a0700 1 -- start start 2021-02-10T11:35:44.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.472 7f31497a0700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f31440d1870 0x7f31441b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:44.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.472 7f31497a0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f31440f0650 0x7f31440cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:44.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.472 7f31497a0700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31440f0b10 con 0x7f31441b0a10 2021-02-10T11:35:44.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.472 7f31497a0700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31440e5ed0 con 0x7f31440d1870 2021-02-10T11:35:44.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.472 7f31497a0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31440d1c70 con 0x7f31440f0650 2021-02-10T11:35:44.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.472 7f3142d73700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f31441b0a10 0x7f31440cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53772/0 (socket says 172.21.2.101:53772) 2021-02-10T11:35:44.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.472 7f3142d73700 1 -- 172.21.2.101:0/1398825710 learned_addr learned my addr 172.21.2.101:0/1398825710 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:44.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.472 7f3143d75700 1 --2- 172.21.2.101:0/1398825710 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f31440f0650 0x7f31440cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:44.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.472 7f3143574700 1 --2- 172.21.2.101:0/1398825710 >> v2:172.21.2.108:3300/0 conn(0x7f31440d1870 0x7f31441b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.472 7f3142572700 1 -- 172.21.2.101:0/1398825710 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459554694 0 0) 0x7f31340009b0 con 0x7f31441b0a10 2021-02-10T11:35:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.472 7f3142572700 1 -- 172.21.2.101:0/1398825710 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3130002fe0 con 0x7f31441b0a10 2021-02-10T11:35:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.473 7f3143d75700 1 -- 172.21.2.101:0/1398825710 >> v2:172.21.2.108:3300/0 conn(0x7f31440d1870 msgr2=0x7f31441b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.473 7f3143d75700 1 --2- 172.21.2.101:0/1398825710 >> v2:172.21.2.108:3300/0 conn(0x7f31440d1870 0x7f31441b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.473 7f3143d75700 1 -- 172.21.2.101:0/1398825710 >> v1:172.21.2.101:6789/0 conn(0x7f31441b0a10 legacy=0x7f31440cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.473 7f3143d75700 1 -- 172.21.2.101:0/1398825710 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31440d1e90 con 0x7f31440f0650 2021-02-10T11:35:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.473 7f3143574700 1 --2- 172.21.2.101:0/1398825710 >> v2:172.21.2.108:3300/0 conn(0x7f31440d1870 0x7f31441b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:35:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.473 7f3143d75700 1 --2- 172.21.2.101:0/1398825710 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f31440f0650 0x7f31440cfb20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f313c006e50 tx=0x7f313c009e70).ready entity=mon.2 client_cookie=7c797576a239a153 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.473 7f3142572700 1 -- 172.21.2.101:0/1398825710 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f313c006a20 con 0x7f31440f0650 2021-02-10T11:35:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.473 7f3142572700 1 -- 172.21.2.101:0/1398825710 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c006790 con 0x7f31440f0650 2021-02-10T11:35:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.473 7f3142572700 1 -- 172.21.2.101:0/1398825710 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f313c00b700 con 0x7f31440f0650 2021-02-10T11:35:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.473 7f31497a0700 1 -- 172.21.2.101:0/1398825710 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f31440f0650 msgr2=0x7f31440cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.473 7f31497a0700 1 --2- 172.21.2.101:0/1398825710 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f31440f0650 0x7f31440cfb20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f313c006e50 tx=0x7f313c009e70).stop 2021-02-10T11:35:44.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.474 7f31497a0700 1 -- 172.21.2.101:0/1398825710 shutdown_connections 2021-02-10T11:35:44.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.474 7f31497a0700 1 --2- 172.21.2.101:0/1398825710 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f31440f0650 0x7f31440cfb20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:44.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.474 7f31497a0700 1 --2- 172.21.2.101:0/1398825710 >> v2:172.21.2.108:3300/0 conn(0x7f31440d1870 0x7f31441b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:44.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.474 7f31497a0700 1 -- 172.21.2.101:0/1398825710 >> 172.21.2.101:0/1398825710 conn(0x7f31440ce450 msgr2=0x7f31440d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:44.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.474 7f31497a0700 1 -- 172.21.2.101:0/1398825710 shutdown_connections 2021-02-10T11:35:44.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.474 7f31497a0700 1 -- 172.21.2.101:0/1398825710 wait complete. 2021-02-10T11:35:44.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.488 7f31497a0700 1 Processor -- start 2021-02-10T11:35:44.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.488 7f31497a0700 1 -- start start 2021-02-10T11:35:44.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.489 7f31497a0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f314409ed10 0x7f3144192d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:44.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.489 7f31497a0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f314409f110 0x7f314409f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:44.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.489 7f31497a0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f31440ba110 0x7f31440ba510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:44.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.489 7f31497a0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3144193500 con 0x7f31440ba110 2021-02-10T11:35:44.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.489 7f31497a0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3144193a00 con 0x7f314409ed10 2021-02-10T11:35:44.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.489 7f31497a0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f314409fd60 con 0x7f314409f110 2021-02-10T11:35:44.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.489 7f3143d75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f31440ba110 0x7f31440ba510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:44.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.489 7f3143d75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f31440ba110 0x7f31440ba510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53284/0 (socket says 172.21.2.101:53284) 2021-02-10T11:35:44.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.489 7f3143d75700 1 -- 172.21.2.101:0/1595994584 learned_addr learned my addr 172.21.2.101:0/1595994584 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:44.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.490 7f3143d75700 1 -- 172.21.2.101:0/1595994584 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f314409f110 msgr2=0x7f314409f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:44.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.490 7f3143d75700 1 --2- 172.21.2.101:0/1595994584 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f314409f110 0x7f314409f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:44.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.490 7f3143d75700 1 -- 172.21.2.101:0/1595994584 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f314409ed10 msgr2=0x7f3144192d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:44.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.490 7f3143d75700 1 --2- 172.21.2.101:0/1595994584 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f314409ed10 0x7f3144192d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:44.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.490 7f3143d75700 1 -- 172.21.2.101:0/1595994584 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31440bab30 con 0x7f31440ba110 2021-02-10T11:35:44.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.490 7f3143d75700 1 --2- 172.21.2.101:0/1595994584 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f31440ba110 0x7f31440ba510 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 rx=0x7f313c006e60 tx=0x7f313c00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:44.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.490 7f313bfff700 1 -- 172.21.2.101:0/1595994584 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f313c009130 con 0x7f31440ba110 2021-02-10T11:35:44.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.490 7f313bfff700 1 -- 172.21.2.101:0/1595994584 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c000e80 con 0x7f31440ba110 2021-02-10T11:35:44.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.490 7f31497a0700 1 -- 172.21.2.101:0/1595994584 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31440bab30 con 0x7f31440ba110 2021-02-10T11:35:44.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.490 7f31497a0700 1 -- 172.21.2.101:0/1595994584 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31441aba40 con 0x7f31440ba110 2021-02-10T11:35:44.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.490 7f313bfff700 1 -- 172.21.2.101:0/1595994584 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f313c0013d0 con 0x7f31440ba110 2021-02-10T11:35:44.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.491 7f313bfff700 1 -- 172.21.2.101:0/1595994584 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f313c0056b0 con 0x7f31440ba110 2021-02-10T11:35:44.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.491 7f313bfff700 1 --2- 172.21.2.101:0/1595994584 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f31280186c0 0x7f312801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:44.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.491 7f313bfff700 1 -- 172.21.2.101:0/1595994584 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f313c032900 con 0x7f31440ba110 2021-02-10T11:35:44.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.491 7f3143574700 1 --2- 172.21.2.101:0/1595994584 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f31280186c0 0x7f312801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:44.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.491 7f3143574700 1 --2- 172.21.2.101:0/1595994584 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f31280186c0 0x7f312801abc0 crc :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:44.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.522 7f31497a0700 1 -- 172.21.2.101:0/1595994584 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31440bb0c0 con 0x7f31440ba110 2021-02-10T11:35:44.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.525 7f313bfff700 1 -- 172.21.2.101:0/1595994584 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f313c0093d0 con 0x7f31440ba110 2021-02-10T11:35:44.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.673 7f31497a0700 1 -- 172.21.2.101:0/1595994584 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "client.alice", "name": "debug_client"} v 0) v1 -- 0x7f31440bb0c0 con 0x7f31440ba110 2021-02-10T11:35:44.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.677 7f313bfff700 1 -- 172.21.2.101:0/1595994584 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "client.alice", "name": "debug_client"}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f313c01f8b0 con 0x7f31440ba110 2021-02-10T11:35:44.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.682 7f31497a0700 1 -- 172.21.2.101:0/1595994584 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f31280186c0 msgr2=0x7f312801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:44.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.682 7f31497a0700 1 --2- 172.21.2.101:0/1595994584 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f31280186c0 0x7f312801abc0 crc :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:44.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.682 7f31497a0700 1 -- 172.21.2.101:0/1595994584 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f31440ba110 msgr2=0x7f31440ba510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:44.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.682 7f31497a0700 1 --2- 172.21.2.101:0/1595994584 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f31440ba110 0x7f31440ba510 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 rx=0x7f313c006e60 tx=0x7f313c00bcf0).stop 2021-02-10T11:35:44.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.682 7f31497a0700 1 -- 172.21.2.101:0/1595994584 shutdown_connections 2021-02-10T11:35:44.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.682 7f31497a0700 1 --2- 172.21.2.101:0/1595994584 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f31280186c0 0x7f312801abc0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:44.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.682 7f31497a0700 1 --2- 172.21.2.101:0/1595994584 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f314409f110 0x7f314409f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:44.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.682 7f31497a0700 1 --2- 172.21.2.101:0/1595994584 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f31440ba110 0x7f31440ba510 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:44.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.682 7f31497a0700 1 --2- 172.21.2.101:0/1595994584 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f314409ed10 0x7f3144192d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:44.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.682 7f31497a0700 1 -- 172.21.2.101:0/1595994584 >> 172.21.2.101:0/1595994584 conn(0x7f31440a0290 msgr2=0x7f31440b9050 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:44.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.683 7f31497a0700 1 -- 172.21.2.101:0/1595994584 shutdown_connections 2021-02-10T11:35:44.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:44.683 7f31497a0700 1 -- 172.21.2.101:0/1595994584 wait complete. 2021-02-10T11:35:44.699 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm client.alice debug_objecter 2021-02-10T11:35:45.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.108 7f737a781700 1 Processor -- start 2021-02-10T11:35:45.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.108 7f737a781700 1 -- start start 2021-02-10T11:35:45.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.108 7f737a781700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f73740d0330 0x7f73740ef630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:45.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.108 7f737a781700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73740f0b00 con 0x7f73740d1c60 2021-02-10T11:35:45.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.108 7f737a781700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73741b0bd0 con 0x7f73740d1860 2021-02-10T11:35:45.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.108 7f737a781700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73740e5ec0 con 0x7f73740d0330 2021-02-10T11:35:45.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.108 7f7378d9e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f73740d1c60 0x7f73740efc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53784/0 (socket says 172.21.2.101:53784) 2021-02-10T11:35:45.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.108 7f7378d9e700 1 -- 172.21.2.101:0/194682940 learned_addr learned my addr 172.21.2.101:0/194682940 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:45.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.109 7f73737fe700 1 --2- 172.21.2.101:0/194682940 >> v2:172.21.2.101:3300/0 conn(0x7f73740d0330 0x7f73740ef630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:45.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.109 7f7372ffd700 1 -- 172.21.2.101:0/194682940 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226841228 0 0) 0x7f736c000d30 con 0x7f73740d1c60 2021-02-10T11:35:45.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.109 7f7372ffd700 1 -- 172.21.2.101:0/194682940 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f735c002fe0 con 0x7f73740d1c60 2021-02-10T11:35:45.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.109 7f73737fe700 1 -- 172.21.2.101:0/194682940 >> v1:172.21.2.108:6789/0 conn(0x7f73740d1860 legacy=0x7f73741b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:45.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.109 7f73737fe700 1 -- 172.21.2.101:0/194682940 >> v1:172.21.2.101:6789/0 conn(0x7f73740d1c60 legacy=0x7f73740efc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:45.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.109 7f73737fe700 1 -- 172.21.2.101:0/194682940 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73740f0440 con 0x7f73740d0330 2021-02-10T11:35:45.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.109 7f73737fe700 1 --2- 172.21.2.101:0/194682940 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73740d0330 0x7f73740ef630 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 rx=0x7f7368006650 tx=0x7f7368009a80).ready entity=mon.0 client_cookie=c43431d4ec997eab server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:45.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.109 7f7372ffd700 1 -- 172.21.2.101:0/194682940 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f736800c0c0 con 0x7f73740d0330 2021-02-10T11:35:45.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.109 7f7372ffd700 1 -- 172.21.2.101:0/194682940 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736800c340 con 0x7f73740d0330 2021-02-10T11:35:45.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.109 7f7372ffd700 1 -- 172.21.2.101:0/194682940 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f736800c860 con 0x7f73740d0330 2021-02-10T11:35:45.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.109 7f737a781700 1 -- 172.21.2.101:0/194682940 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73740d0330 msgr2=0x7f73740ef630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:45.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.109 7f737a781700 1 --2- 172.21.2.101:0/194682940 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73740d0330 0x7f73740ef630 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 rx=0x7f7368006650 tx=0x7f7368009a80).stop 2021-02-10T11:35:45.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.110 7f737a781700 1 -- 172.21.2.101:0/194682940 shutdown_connections 2021-02-10T11:35:45.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.110 7f737a781700 1 --2- 172.21.2.101:0/194682940 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73740d0330 0x7f73740ef630 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:45.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.110 7f737a781700 1 -- 172.21.2.101:0/194682940 >> 172.21.2.101:0/194682940 conn(0x7f73740ce440 msgr2=0x7f73740d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:45.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.110 7f737a781700 1 -- 172.21.2.101:0/194682940 shutdown_connections 2021-02-10T11:35:45.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.110 7f737a781700 1 -- 172.21.2.101:0/194682940 wait complete. 2021-02-10T11:35:45.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.125 7f737a781700 1 Processor -- start 2021-02-10T11:35:45.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.125 7f737a781700 1 -- start start 2021-02-10T11:35:45.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.125 7f737a781700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f737409f360 0x7f73740b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:45.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.125 7f737a781700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f737408d7a0 0x7f737408dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:45.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.125 7f737a781700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f737408c430 0x7f737408c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:45.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.125 7f737a781700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73740b5620 con 0x7f737408d7a0 2021-02-10T11:35:45.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.125 7f737a781700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737408e1c0 con 0x7f737409f360 2021-02-10T11:35:45.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.125 7f737a781700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f737409f870 con 0x7f737408c430 2021-02-10T11:35:45.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.125 7f73737fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f737408d7a0 0x7f737408dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:45.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.125 7f73737fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f737408d7a0 0x7f737408dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53298/0 (socket says 172.21.2.101:53298) 2021-02-10T11:35:45.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.125 7f73737fe700 1 -- 172.21.2.101:0/1678703300 learned_addr learned my addr 172.21.2.101:0/1678703300 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:45.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.126 7f73737fe700 1 -- 172.21.2.101:0/1678703300 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f737408c430 msgr2=0x7f737408c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:45.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.126 7f73737fe700 1 --2- 172.21.2.101:0/1678703300 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f737408c430 0x7f737408c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:45.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.126 7f73737fe700 1 -- 172.21.2.101:0/1678703300 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f737409f360 msgr2=0x7f73740b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:45.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.126 7f73737fe700 1 --2- 172.21.2.101:0/1678703300 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f737409f360 0x7f73740b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:45.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.126 7f73737fe700 1 -- 172.21.2.101:0/1678703300 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737408ce50 con 0x7f737408d7a0 2021-02-10T11:35:45.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.126 7f73737fe700 1 --2- 172.21.2.101:0/1678703300 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f737408d7a0 0x7f737408dba0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 rx=0x7f7368009630 tx=0x7f736800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:45.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.126 7f7370ff9700 1 -- 172.21.2.101:0/1678703300 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73680093a0 con 0x7f737408d7a0 2021-02-10T11:35:45.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.126 7f7370ff9700 1 -- 172.21.2.101:0/1678703300 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736800bd70 con 0x7f737408d7a0 2021-02-10T11:35:45.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.126 7f737a781700 1 -- 172.21.2.101:0/1678703300 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f737408ce50 con 0x7f737408d7a0 2021-02-10T11:35:45.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.126 7f7370ff9700 1 -- 172.21.2.101:0/1678703300 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73680054d0 con 0x7f737408d7a0 2021-02-10T11:35:45.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.126 7f737a781700 1 -- 172.21.2.101:0/1678703300 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73741a3160 con 0x7f737408d7a0 2021-02-10T11:35:45.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.127 7f7370ff9700 1 -- 172.21.2.101:0/1678703300 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f736800a000 con 0x7f737408d7a0 2021-02-10T11:35:45.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.127 7f7370ff9700 1 --2- 172.21.2.101:0/1678703300 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7354018700 0x7f735401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:45.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.127 7f7370ff9700 1 -- 172.21.2.101:0/1678703300 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f73680327d0 con 0x7f737408d7a0 2021-02-10T11:35:45.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.127 7f7373fff700 1 --2- 172.21.2.101:0/1678703300 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7354018700 0x7f735401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:45.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.127 7f7373fff700 1 --2- 172.21.2.101:0/1678703300 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7354018700 0x7f735401abc0 crc :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:45.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.157 7f737a781700 1 -- 172.21.2.101:0/1678703300 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f737408d270 con 0x7f737408d7a0 2021-02-10T11:35:45.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.160 7f7370ff9700 1 -- 172.21.2.101:0/1678703300 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7368009340 con 0x7f737408d7a0 2021-02-10T11:35:45.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.308 7f737a781700 1 -- 172.21.2.101:0/1678703300 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "client.alice", "name": "debug_objecter"} v 0) v1 -- 0x7f737408d270 con 0x7f737408d7a0 2021-02-10T11:35:45.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.312 7f7370ff9700 1 -- 172.21.2.101:0/1678703300 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "client.alice", "name": "debug_objecter"}]=0 v19) v1 ==== 106+0+0 (secure 0 0 0) 0x7f736800a000 con 0x7f737408d7a0 2021-02-10T11:35:45.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.317 7f737a781700 1 -- 172.21.2.101:0/1678703300 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7354018700 msgr2=0x7f735401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:45.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.317 7f737a781700 1 --2- 172.21.2.101:0/1678703300 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7354018700 0x7f735401abc0 crc :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:45.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.317 7f737a781700 1 -- 172.21.2.101:0/1678703300 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f737408d7a0 msgr2=0x7f737408dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:45.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.317 7f737a781700 1 --2- 172.21.2.101:0/1678703300 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f737408d7a0 0x7f737408dba0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 rx=0x7f7368009630 tx=0x7f736800b980).stop 2021-02-10T11:35:45.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.317 7f737a781700 1 -- 172.21.2.101:0/1678703300 shutdown_connections 2021-02-10T11:35:45.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.317 7f737a781700 1 --2- 172.21.2.101:0/1678703300 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7354018700 0x7f735401abc0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:45.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.317 7f737a781700 1 --2- 172.21.2.101:0/1678703300 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f737408c430 0x7f737408c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:45.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.317 7f737a781700 1 --2- 172.21.2.101:0/1678703300 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f737408d7a0 0x7f737408dba0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:45.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.317 7f737a781700 1 --2- 172.21.2.101:0/1678703300 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f737409f360 0x7f73740b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:45.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.317 7f737a781700 1 -- 172.21.2.101:0/1678703300 >> 172.21.2.101:0/1678703300 conn(0x7f73740a67e0 msgr2=0x7f7374094e60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:45.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.317 7f737a781700 1 -- 172.21.2.101:0/1678703300 shutdown_connections 2021-02-10T11:35:45.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.317 7f737a781700 1 -- 172.21.2.101:0/1678703300 wait complete. 2021-02-10T11:35:45.333 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_authorize_deauthorize_subvolume' 2021-02-10T11:35:45.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.735 7f14a7863700 1 Processor -- start 2021-02-10T11:35:45.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.736 7f14a7863700 1 -- start start 2021-02-10T11:35:45.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.736 7f14a7863700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f14a00d0340 0x7f14a00ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:45.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.736 7f14a7863700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a00f0b10 con 0x7f14a00d1c70 2021-02-10T11:35:45.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.736 7f14a7863700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a01b0be0 con 0x7f14a00d1870 2021-02-10T11:35:45.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.736 7f14a7863700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14a00e5ed0 con 0x7f14a00d0340 2021-02-10T11:35:45.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.736 7f14a5e80700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f14a00d1c70 0x7f14a00efc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53802/0 (socket says 172.21.2.101:53802) 2021-02-10T11:35:45.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.736 7f14a5e80700 1 -- 172.21.2.101:0/3401704012 learned_addr learned my addr 172.21.2.101:0/3401704012 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:45.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.736 7f14a4e7e700 1 --2- 172.21.2.101:0/3401704012 >> v2:172.21.2.108:3300/0 conn(0x7f14a00d0340 0x7f14a00ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:45.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.737 7f1493fff700 1 -- 172.21.2.101:0/3401704012 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288771393 0 0) 0x7f1498000f90 con 0x7f14a00d1c70 2021-02-10T11:35:45.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.737 7f1493fff700 1 -- 172.21.2.101:0/3401704012 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f148c002fe0 con 0x7f14a00d1c70 2021-02-10T11:35:45.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.737 7f1493fff700 1 -- 172.21.2.101:0/3401704012 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2937534549 0 0) 0x7f149c000ff0 con 0x7f14a00d1870 2021-02-10T11:35:45.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.737 7f1493fff700 1 -- 172.21.2.101:0/3401704012 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f148c004370 con 0x7f14a00d1870 2021-02-10T11:35:45.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.737 7f14a4e7e700 1 -- 172.21.2.101:0/3401704012 >> v1:172.21.2.108:6789/0 conn(0x7f14a00d1870 legacy=0x7f14a01b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:45.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.737 7f14a4e7e700 1 -- 172.21.2.101:0/3401704012 >> v1:172.21.2.101:6789/0 conn(0x7f14a00d1c70 legacy=0x7f14a00efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:45.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.737 7f14a4e7e700 1 -- 172.21.2.101:0/3401704012 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a00f0450 con 0x7f14a00d0340 2021-02-10T11:35:45.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.737 7f14a4e7e700 1 --2- 172.21.2.101:0/3401704012 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14a00d0340 0x7f14a00ef640 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f1494006650 tx=0x7f1494009a00).ready entity=mon.1 client_cookie=c7eef30511712c04 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:45.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.737 7f1493fff700 1 -- 172.21.2.101:0/3401704012 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f14940009b0 con 0x7f14a00d0340 2021-02-10T11:35:45.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.737 7f1493fff700 1 -- 172.21.2.101:0/3401704012 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149400c0c0 con 0x7f14a00d0340 2021-02-10T11:35:45.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.737 7f1493fff700 1 -- 172.21.2.101:0/3401704012 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f149400c5a0 con 0x7f14a00d0340 2021-02-10T11:35:45.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.737 7f14a7863700 1 -- 172.21.2.101:0/3401704012 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14a00d0340 msgr2=0x7f14a00ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:45.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.737 7f14a7863700 1 --2- 172.21.2.101:0/3401704012 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14a00d0340 0x7f14a00ef640 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f1494006650 tx=0x7f1494009a00).stop 2021-02-10T11:35:45.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.738 7f14a7863700 1 -- 172.21.2.101:0/3401704012 shutdown_connections 2021-02-10T11:35:45.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.738 7f14a7863700 1 --2- 172.21.2.101:0/3401704012 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14a00d0340 0x7f14a00ef640 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:45.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.738 7f14a7863700 1 -- 172.21.2.101:0/3401704012 >> 172.21.2.101:0/3401704012 conn(0x7f14a00ce450 msgr2=0x7f14a00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:45.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.738 7f14a7863700 1 -- 172.21.2.101:0/3401704012 shutdown_connections 2021-02-10T11:35:45.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.738 7f14a7863700 1 -- 172.21.2.101:0/3401704012 wait complete. 2021-02-10T11:35:45.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.753 7f14a7863700 1 Processor -- start 2021-02-10T11:35:45.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.753 7f14a7863700 1 -- start start 2021-02-10T11:35:45.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.753 7f14a7863700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14a00ada90 0x7f14a00ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:45.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.753 7f14a7863700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14a0099d20 0x7f14a009a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:45.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.753 7f14a7863700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14a009a740 0x7f14a0097bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:45.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.753 7f14a7863700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a00997a0 con 0x7f14a009a740 2021-02-10T11:35:45.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.753 7f14a7863700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a018f0c0 con 0x7f14a0099d20 2021-02-10T11:35:45.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.753 7f14a7863700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a009ab40 con 0x7f14a00ada90 2021-02-10T11:35:45.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.753 7f14a5e80700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14a009a740 0x7f14a0097bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:45.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.753 7f14a5e80700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14a009a740 0x7f14a0097bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53314/0 (socket says 172.21.2.101:53314) 2021-02-10T11:35:45.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.753 7f14a5e80700 1 -- 172.21.2.101:0/2658073393 learned_addr learned my addr 172.21.2.101:0/2658073393 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:45.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.754 7f14a5e80700 1 -- 172.21.2.101:0/2658073393 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14a00ada90 msgr2=0x7f14a00ade90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:35:45.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.754 7f14a5e80700 1 --2- 172.21.2.101:0/2658073393 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14a00ada90 0x7f14a00ade90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:45.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.754 7f14a5e80700 1 -- 172.21.2.101:0/2658073393 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14a0099d20 msgr2=0x7f14a009a120 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:35:45.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.754 7f14a5e80700 1 --2- 172.21.2.101:0/2658073393 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14a0099d20 0x7f14a009a120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:45.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.754 7f14a5e80700 1 -- 172.21.2.101:0/2658073393 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a009ad60 con 0x7f14a009a740 2021-02-10T11:35:45.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.754 7f14a5e80700 1 --2- 172.21.2.101:0/2658073393 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14a009a740 0x7f14a0097bc0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 rx=0x7f1498009960 tx=0x7f149800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:45.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.754 7f1491ffb700 1 -- 172.21.2.101:0/2658073393 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1498006030 con 0x7f14a009a740 2021-02-10T11:35:45.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.754 7f1491ffb700 1 -- 172.21.2.101:0/2658073393 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1498006370 con 0x7f14a009a740 2021-02-10T11:35:45.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.754 7f1491ffb700 1 -- 172.21.2.101:0/2658073393 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f14980049a0 con 0x7f14a009a740 2021-02-10T11:35:45.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.754 7f14a7863700 1 -- 172.21.2.101:0/2658073393 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a009ad60 con 0x7f14a009a740 2021-02-10T11:35:45.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.754 7f14a7863700 1 -- 172.21.2.101:0/2658073393 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a00984d0 con 0x7f14a009a740 2021-02-10T11:35:45.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.755 7f1491ffb700 1 -- 172.21.2.101:0/2658073393 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1498006550 con 0x7f14a009a740 2021-02-10T11:35:45.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.755 7f1491ffb700 1 --2- 172.21.2.101:0/2658073393 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14800186c0 0x7f148001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:45.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.755 7f1491ffb700 1 -- 172.21.2.101:0/2658073393 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1498032750 con 0x7f14a009a740 2021-02-10T11:35:45.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.755 7f14a567f700 1 --2- 172.21.2.101:0/2658073393 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14800186c0 0x7f148001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:45.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.755 7f14a567f700 1 --2- 172.21.2.101:0/2658073393 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14800186c0 0x7f148001abc0 crc :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:45.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.786 7f14a7863700 1 -- 172.21.2.101:0/2658073393 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a00ebc00 con 0x7f14a009a740 2021-02-10T11:35:45.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.789 7f1491ffb700 1 -- 172.21.2.101:0/2658073393 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1498000ee0 con 0x7f14a009a740 2021-02-10T11:35:45.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:45.919 7f14a7863700 1 -- 172.21.2.101:0/2658073393 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_authorize_deauthorize_subvolume"]} v 0) v1 -- 0x7f14a00ebc00 con 0x7f14a009a740 2021-02-10T11:35:46.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.101 7f1491ffb700 1 -- 172.21.2.101:0/2658073393 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_authorize_deauthorize_subvolume"]}]=0 v385) v1 ==== 153+0+0 (secure 0 0 0) 0x7f1498006550 con 0x7f14a009a740 2021-02-10T11:35:46.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.136 7f14a7863700 1 -- 172.21.2.101:0/2658073393 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14800186c0 msgr2=0x7f148001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:46.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.136 7f14a7863700 1 --2- 172.21.2.101:0/2658073393 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14800186c0 0x7f148001abc0 crc :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:46.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.136 7f14a7863700 1 -- 172.21.2.101:0/2658073393 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14a009a740 msgr2=0x7f14a0097bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:46.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.136 7f14a7863700 1 --2- 172.21.2.101:0/2658073393 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14a009a740 0x7f14a0097bc0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 rx=0x7f1498009960 tx=0x7f149800bcf0).stop 2021-02-10T11:35:46.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.137 7f14a7863700 1 -- 172.21.2.101:0/2658073393 shutdown_connections 2021-02-10T11:35:46.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.137 7f14a7863700 1 --2- 172.21.2.101:0/2658073393 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14a009a740 0x7f14a0097bc0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:46.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.137 7f14a7863700 1 --2- 172.21.2.101:0/2658073393 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14a0099d20 0x7f14a009a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:46.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.137 7f14a7863700 1 --2- 172.21.2.101:0/2658073393 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14800186c0 0x7f148001abc0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:46.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.137 7f14a7863700 1 --2- 172.21.2.101:0/2658073393 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14a00ada90 0x7f14a00ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:46.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.137 7f14a7863700 1 -- 172.21.2.101:0/2658073393 >> 172.21.2.101:0/2658073393 conn(0x7f14a00f0550 msgr2=0x7f14a0191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:46.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.137 7f14a7863700 1 -- 172.21.2.101:0/2658073393 shutdown_connections 2021-02-10T11:35:46.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.137 7f14a7863700 1 -- 172.21.2.101:0/2658073393 wait complete. 2021-02-10T11:35:46.153 INFO:tasks.cephfs_test_runner:test_authorize_deauthorize_subvolume (tasks.cephfs.test_volumes.TestVolumes) ... ok 2021-02-10T11:35:46.153 INFO:tasks.cephfs_test_runner:Starting test: test_clone_subvolume_info (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:35:46.153 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_clone_subvolume_info' 2021-02-10T11:35:46.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.564 7ff80ff69700 1 Processor -- start 2021-02-10T11:35:46.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.564 7ff80ff69700 1 -- start start 2021-02-10T11:35:46.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.564 7ff80ff69700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff8080d1870 0x7ff8081b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:46.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.564 7ff80ff69700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8080f0650 0x7ff8080cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:46.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.564 7ff80ff69700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8080f0b10 con 0x7ff8081b0a10 2021-02-10T11:35:46.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.564 7ff80ff69700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8080e5ed0 con 0x7ff8080d1870 2021-02-10T11:35:46.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.564 7ff80ff69700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8080d1c70 con 0x7ff8080f0650 2021-02-10T11:35:46.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.564 7ff80dd85700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff8080d1870 0x7ff8081b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:46.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.564 7ff80dd85700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff8080d1870 0x7ff8081b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53318/0 (socket says 172.21.2.101:53318) 2021-02-10T11:35:46.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.564 7ff80dd85700 1 -- 172.21.2.101:0/2962946066 learned_addr learned my addr 172.21.2.101:0/2962946066 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:46.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.565 7ff80dd85700 1 -- 172.21.2.101:0/2962946066 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8080f0650 msgr2=0x7ff8080cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:46.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.565 7ff80dd85700 1 --2- 172.21.2.101:0/2962946066 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8080f0650 0x7ff8080cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:46.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.565 7ff80dd85700 1 -- 172.21.2.101:0/2962946066 >> v1:172.21.2.108:6789/0 conn(0x7ff8081b0a10 legacy=0x7ff8080cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:46.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.565 7ff80dd85700 1 -- 172.21.2.101:0/2962946066 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8080d1e90 con 0x7ff8080d1870 2021-02-10T11:35:46.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.565 7ff80dd85700 1 --2- 172.21.2.101:0/2962946066 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8080d1870 0x7ff8081b03f0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 rx=0x7ff7fc006700 tx=0x7ff7fc009b80).ready entity=mon.0 client_cookie=d8b61ab5545207bf server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:46.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.565 7ff80cd83700 1 -- 172.21.2.101:0/2962946066 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff7fc00c0c0 con 0x7ff8080d1870 2021-02-10T11:35:46.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.565 7ff80cd83700 1 -- 172.21.2.101:0/2962946066 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7fc00c440 con 0x7ff8080d1870 2021-02-10T11:35:46.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.565 7ff80cd83700 1 -- 172.21.2.101:0/2962946066 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff7fc00c960 con 0x7ff8080d1870 2021-02-10T11:35:46.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.565 7ff80ff69700 1 -- 172.21.2.101:0/2962946066 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8080d1870 msgr2=0x7ff8081b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:46.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.565 7ff80ff69700 1 --2- 172.21.2.101:0/2962946066 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8080d1870 0x7ff8081b03f0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 rx=0x7ff7fc006700 tx=0x7ff7fc009b80).stop 2021-02-10T11:35:46.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.566 7ff80ff69700 1 -- 172.21.2.101:0/2962946066 shutdown_connections 2021-02-10T11:35:46.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.566 7ff80ff69700 1 --2- 172.21.2.101:0/2962946066 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8080f0650 0x7ff8080cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:46.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.566 7ff80ff69700 1 --2- 172.21.2.101:0/2962946066 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8080d1870 0x7ff8081b03f0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:46.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.566 7ff80ff69700 1 -- 172.21.2.101:0/2962946066 >> 172.21.2.101:0/2962946066 conn(0x7ff8080ce450 msgr2=0x7ff8080d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:46.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.566 7ff80ff69700 1 -- 172.21.2.101:0/2962946066 shutdown_connections 2021-02-10T11:35:46.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.566 7ff80ff69700 1 -- 172.21.2.101:0/2962946066 wait complete. 2021-02-10T11:35:46.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.581 7ff80ff69700 1 Processor -- start 2021-02-10T11:35:46.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.581 7ff80ff69700 1 -- start start 2021-02-10T11:35:46.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.581 7ff80ff69700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8080b4f30 0x7ff80818b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:46.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.581 7ff80ff69700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff808093f40 0x7ff808094340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:46.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.581 7ff80ff69700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff808094960 0x7ff808187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:46.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.581 7ff80ff69700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff808094e90 con 0x7ff8080b4f30 2021-02-10T11:35:46.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.581 7ff80ff69700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80818bff0 con 0x7ff808094960 2021-02-10T11:35:46.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.581 7ff80ff69700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80808f780 con 0x7ff808093f40 2021-02-10T11:35:46.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.581 7ff80dd85700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8080b4f30 0x7ff80818b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:46.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.581 7ff80dd85700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8080b4f30 0x7ff80818b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53324/0 (socket says 172.21.2.101:53324) 2021-02-10T11:35:46.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.581 7ff80dd85700 1 -- 172.21.2.101:0/1257938159 learned_addr learned my addr 172.21.2.101:0/1257938159 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:46.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.582 7ff80d584700 1 --2- 172.21.2.101:0/1257938159 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff808093f40 0x7ff808094340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:46.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.582 7ff80e586700 1 --2- 172.21.2.101:0/1257938159 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff808094960 0x7ff808187e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:46.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.582 7ff80dd85700 1 -- 172.21.2.101:0/1257938159 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff808093f40 msgr2=0x7ff808094340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:46.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.582 7ff80dd85700 1 --2- 172.21.2.101:0/1257938159 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff808093f40 0x7ff808094340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:46.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.582 7ff80dd85700 1 -- 172.21.2.101:0/1257938159 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff808094960 msgr2=0x7ff808187e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:46.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.582 7ff80dd85700 1 --2- 172.21.2.101:0/1257938159 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff808094960 0x7ff808187e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:46.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.582 7ff80dd85700 1 -- 172.21.2.101:0/1257938159 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff808188470 con 0x7ff8080b4f30 2021-02-10T11:35:46.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.582 7ff80dd85700 1 --2- 172.21.2.101:0/1257938159 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8080b4f30 0x7ff80818b9d0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 rx=0x7ff7fc00b580 tx=0x7ff7fc00b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:46.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.582 7ff7fa7fc700 1 -- 172.21.2.101:0/1257938159 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff7fc00bd10 con 0x7ff8080b4f30 2021-02-10T11:35:46.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.582 7ff7fa7fc700 1 -- 172.21.2.101:0/1257938159 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7fc002030 con 0x7ff8080b4f30 2021-02-10T11:35:46.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.582 7ff7fa7fc700 1 -- 172.21.2.101:0/1257938159 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff7fc002400 con 0x7ff8080b4f30 2021-02-10T11:35:46.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.582 7ff80ff69700 1 -- 172.21.2.101:0/1257938159 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff808188470 con 0x7ff8080b4f30 2021-02-10T11:35:46.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.582 7ff80ff69700 1 -- 172.21.2.101:0/1257938159 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff808188470 con 0x7ff8080b4f30 2021-02-10T11:35:46.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.583 7ff7fa7fc700 1 -- 172.21.2.101:0/1257938159 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff7fc000c90 con 0x7ff8080b4f30 2021-02-10T11:35:46.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.583 7ff7fa7fc700 1 --2- 172.21.2.101:0/1257938159 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7f00186c0 0x7ff7f001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:46.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.583 7ff7fa7fc700 1 -- 172.21.2.101:0/1257938159 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff7fc032430 con 0x7ff8080b4f30 2021-02-10T11:35:46.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.583 7ff80d584700 1 --2- 172.21.2.101:0/1257938159 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7f00186c0 0x7ff7f001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:46.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.583 7ff80d584700 1 --2- 172.21.2.101:0/1257938159 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7f00186c0 0x7ff7f001abc0 crc :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:46.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.614 7ff80ff69700 1 -- 172.21.2.101:0/1257938159 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8080ce450 con 0x7ff8080b4f30 2021-02-10T11:35:46.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.617 7ff7fa7fc700 1 -- 172.21.2.101:0/1257938159 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff7fc030d60 con 0x7ff8080b4f30 2021-02-10T11:35:46.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:46.747 7ff80ff69700 1 -- 172.21.2.101:0/1257938159 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_clone_subvolume_info"]} v 0) v1 -- 0x7ff8080ce450 con 0x7ff8080b4f30 2021-02-10T11:35:47.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.106 7ff7fa7fc700 1 -- 172.21.2.101:0/1257938159 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_clone_subvolume_info"]}]=0 v386) v1 ==== 145+0+0 (secure 0 0 0) 0x7ff7fc000c90 con 0x7ff8080b4f30 2021-02-10T11:35:47.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.115 7ff80ff69700 1 -- 172.21.2.101:0/1257938159 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7f00186c0 msgr2=0x7ff7f001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:47.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.115 7ff80ff69700 1 --2- 172.21.2.101:0/1257938159 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7f00186c0 0x7ff7f001abc0 crc :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:47.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.115 7ff80ff69700 1 -- 172.21.2.101:0/1257938159 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8080b4f30 msgr2=0x7ff80818b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:47.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.115 7ff80ff69700 1 --2- 172.21.2.101:0/1257938159 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8080b4f30 0x7ff80818b9d0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 rx=0x7ff7fc00b580 tx=0x7ff7fc00b920).stop 2021-02-10T11:35:47.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.115 7ff80ff69700 1 -- 172.21.2.101:0/1257938159 shutdown_connections 2021-02-10T11:35:47.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.115 7ff80ff69700 1 --2- 172.21.2.101:0/1257938159 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7f00186c0 0x7ff7f001abc0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:47.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.115 7ff80ff69700 1 --2- 172.21.2.101:0/1257938159 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff808093f40 0x7ff808094340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:47.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.115 7ff80ff69700 1 --2- 172.21.2.101:0/1257938159 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff808094960 0x7ff808187e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:47.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.115 7ff80ff69700 1 --2- 172.21.2.101:0/1257938159 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8080b4f30 0x7ff80818b9d0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:47.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.115 7ff80ff69700 1 -- 172.21.2.101:0/1257938159 >> 172.21.2.101:0/1257938159 conn(0x7ff80819f8f0 msgr2=0x7ff8081a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:47.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.115 7ff80ff69700 1 -- 172.21.2.101:0/1257938159 shutdown_connections 2021-02-10T11:35:47.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.115 7ff80ff69700 1 -- 172.21.2.101:0/1257938159 wait complete. 2021-02-10T11:35:47.131 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:35:47.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.539 7fd183d77700 1 Processor -- start 2021-02-10T11:35:47.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.539 7fd183d77700 1 -- start start 2021-02-10T11:35:47.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.540 7fd183d77700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd17c0d1c70 0x7fd17c0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:47.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.540 7fd183d77700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17c0d0170 con 0x7fd17c0cf580 2021-02-10T11:35:47.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.540 7fd183d77700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17c0fe1c0 con 0x7fd17c0d1870 2021-02-10T11:35:47.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.540 7fd183d77700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c1b0cb0 con 0x7fd17c0d1c70 2021-02-10T11:35:47.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.540 7fd182394700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd17c0cf580 0x7fd17c0cf980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53828/0 (socket says 172.21.2.101:53828) 2021-02-10T11:35:47.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.540 7fd182394700 1 -- 172.21.2.101:0/4131110525 learned_addr learned my addr 172.21.2.101:0/4131110525 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:47.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.540 7fd180b91700 1 -- 172.21.2.101:0/4131110525 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084506691 0 0) 0x7fd174001500 con 0x7fd17c0cf580 2021-02-10T11:35:47.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.540 7fd180b91700 1 -- 172.21.2.101:0/4131110525 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd168002fe0 con 0x7fd17c0cf580 2021-02-10T11:35:47.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd180b91700 1 -- 172.21.2.101:0/4131110525 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1166277710 0 0) 0x7fd174001460 con 0x7fd17c0cf580 2021-02-10T11:35:47.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd180b91700 1 -- 172.21.2.101:0/4131110525 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd17c0d1c70 msgr2=0x7fd17c0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:47.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd180b91700 1 --2- 172.21.2.101:0/4131110525 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd17c0d1c70 0x7fd17c0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:47.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd180b91700 1 -- 172.21.2.101:0/4131110525 >> v1:172.21.2.108:6789/0 conn(0x7fd17c0d1870 legacy=0x7fd17c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:47.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd180b91700 1 -- 172.21.2.101:0/4131110525 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c109a60 con 0x7fd17c0cf580 2021-02-10T11:35:47.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd180b91700 1 -- 172.21.2.101:0/4131110525 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fd174001980 con 0x7fd17c0cf580 2021-02-10T11:35:47.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd180b91700 1 -- 172.21.2.101:0/4131110525 >> v1:172.21.2.101:6789/0 conn(0x7fd17c0cf580 legacy=0x7fd17c0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:47.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd180b91700 1 --2- 172.21.2.101:0/4131110525 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd168003b20 0x7fd168005f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:47.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd180b91700 1 --2- 172.21.2.101:0/4131110525 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1680067b0 0x7fd168008c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:47.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd180b91700 1 --2- 172.21.2.101:0/4131110525 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd168009500 0x7fd16800ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:47.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd180b91700 1 -- 172.21.2.101:0/4131110525 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16800c2d0 con 0x7fd168009500 2021-02-10T11:35:47.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd180b91700 1 -- 172.21.2.101:0/4131110525 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16800c4f0 con 0x7fd168003b20 2021-02-10T11:35:47.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd180b91700 1 -- 172.21.2.101:0/4131110525 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16800c710 con 0x7fd1680067b0 2021-02-10T11:35:47.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd182394700 1 --2- 172.21.2.101:0/4131110525 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd168009500 0x7fd16800ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:47.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd181b93700 1 --2- 172.21.2.101:0/4131110525 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd168003b20 0x7fd168005f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:47.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd181392700 1 --2- 172.21.2.101:0/4131110525 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1680067b0 0x7fd168008c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:47.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd182394700 1 -- 172.21.2.101:0/4131110525 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1680067b0 msgr2=0x7fd168008c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:47.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd182394700 1 --2- 172.21.2.101:0/4131110525 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1680067b0 0x7fd168008c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:47.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd182394700 1 -- 172.21.2.101:0/4131110525 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd168003b20 msgr2=0x7fd168005f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:47.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd182394700 1 --2- 172.21.2.101:0/4131110525 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd168003b20 0x7fd168005f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:47.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd182394700 1 -- 172.21.2.101:0/4131110525 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16800c9b0 con 0x7fd168009500 2021-02-10T11:35:47.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd182394700 1 --2- 172.21.2.101:0/4131110525 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd168009500 0x7fd16800ba20 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 rx=0x7fd174001c20 tx=0x7fd174050310).ready entity=mon.0 client_cookie=9478473e773bf767 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:47.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd180b91700 1 -- 172.21.2.101:0/4131110525 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd174007020 con 0x7fd168009500 2021-02-10T11:35:47.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd180b91700 1 -- 172.21.2.101:0/4131110525 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd174007320 con 0x7fd168009500 2021-02-10T11:35:47.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd180b91700 1 -- 172.21.2.101:0/4131110525 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd1740049a0 con 0x7fd168009500 2021-02-10T11:35:47.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd183d77700 1 -- 172.21.2.101:0/4131110525 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd168009500 msgr2=0x7fd16800ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:47.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd183d77700 1 --2- 172.21.2.101:0/4131110525 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd168009500 0x7fd16800ba20 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 rx=0x7fd174001c20 tx=0x7fd174050310).stop 2021-02-10T11:35:47.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.541 7fd182394700 1 -- 172.21.2.101:0/4131110525 reap_dead start 2021-02-10T11:35:47.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.542 7fd183d77700 1 -- 172.21.2.101:0/4131110525 shutdown_connections 2021-02-10T11:35:47.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.542 7fd183d77700 1 -- 172.21.2.101:0/4131110525 >> 172.21.2.101:0/4131110525 conn(0x7fd17c0ce420 msgr2=0x7fd17c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:47.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.542 7fd183d77700 1 -- 172.21.2.101:0/4131110525 shutdown_connections 2021-02-10T11:35:47.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.542 7fd183d77700 1 -- 172.21.2.101:0/4131110525 wait complete. 2021-02-10T11:35:47.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.557 7fd183d77700 1 Processor -- start 2021-02-10T11:35:47.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.557 7fd183d77700 1 -- start start 2021-02-10T11:35:47.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.557 7fd183d77700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd17c085bc0 0x7fd17c0863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:47.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.557 7fd183d77700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd17c0954c0 0x7fd17c096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:47.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.557 7fd183d77700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd17c0958c0 0x7fd17c096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:47.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.557 7fd183d77700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c095d80 con 0x7fd17c0958c0 2021-02-10T11:35:47.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.557 7fd183d77700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0bfa60 con 0x7fd17c085bc0 2021-02-10T11:35:47.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.557 7fd183d77700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c085fc0 con 0x7fd17c0954c0 2021-02-10T11:35:47.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.557 7fd182394700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd17c0958c0 0x7fd17c096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:47.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.557 7fd182394700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd17c0958c0 0x7fd17c096920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53346/0 (socket says 172.21.2.101:53346) 2021-02-10T11:35:47.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.557 7fd182394700 1 -- 172.21.2.101:0/2049256298 learned_addr learned my addr 172.21.2.101:0/2049256298 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:47.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.558 7fd182394700 1 -- 172.21.2.101:0/2049256298 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd17c0954c0 msgr2=0x7fd17c096300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:47.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.558 7fd182394700 1 --2- 172.21.2.101:0/2049256298 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd17c0954c0 0x7fd17c096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:47.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.558 7fd182394700 1 -- 172.21.2.101:0/2049256298 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd17c085bc0 msgr2=0x7fd17c0863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:47.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.558 7fd182394700 1 --2- 172.21.2.101:0/2049256298 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd17c085bc0 0x7fd17c0863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:47.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.558 7fd182394700 1 -- 172.21.2.101:0/2049256298 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c086c10 con 0x7fd17c0958c0 2021-02-10T11:35:47.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.558 7fd182394700 1 --2- 172.21.2.101:0/2049256298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd17c0958c0 0x7fd17c096920 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 rx=0x7fd174051930 tx=0x7fd174051cd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:47.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.558 7fd16e7fc700 1 -- 172.21.2.101:0/2049256298 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd174000f70 con 0x7fd17c0958c0 2021-02-10T11:35:47.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.558 7fd16e7fc700 1 -- 172.21.2.101:0/2049256298 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1740070c0 con 0x7fd17c0958c0 2021-02-10T11:35:47.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.558 7fd183d77700 1 -- 172.21.2.101:0/2049256298 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd17c086c10 con 0x7fd17c0958c0 2021-02-10T11:35:47.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.558 7fd16e7fc700 1 -- 172.21.2.101:0/2049256298 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd1740075e0 con 0x7fd17c0958c0 2021-02-10T11:35:47.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.558 7fd183d77700 1 -- 172.21.2.101:0/2049256298 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17c086c10 con 0x7fd17c0958c0 2021-02-10T11:35:47.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.559 7fd16e7fc700 1 -- 172.21.2.101:0/2049256298 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd1740074b0 con 0x7fd17c0958c0 2021-02-10T11:35:47.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.559 7fd16e7fc700 1 --2- 172.21.2.101:0/2049256298 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd164018730 0x7fd16401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:47.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.559 7fd16e7fc700 1 -- 172.21.2.101:0/2049256298 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd1740769b0 con 0x7fd17c0958c0 2021-02-10T11:35:47.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.559 7fd181b93700 1 --2- 172.21.2.101:0/2049256298 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd164018730 0x7fd16401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:47.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.559 7fd181b93700 1 --2- 172.21.2.101:0/2049256298 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd164018730 0x7fd16401ac10 crc :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:47.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.589 7fd183d77700 1 -- 172.21.2.101:0/2049256298 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd17c0fe1c0 con 0x7fd17c0958c0 2021-02-10T11:35:47.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.592 7fd16e7fc700 1 -- 172.21.2.101:0/2049256298 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd174076bc0 con 0x7fd17c0958c0 2021-02-10T11:35:47.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.740 7fd183d77700 1 -- 172.21.2.101:0/2049256298 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd17c0fe1c0 con 0x7fd17c0958c0 2021-02-10T11:35:47.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.744 7fd16e7fc700 1 -- 172.21.2.101:0/2049256298 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd1740072b0 con 0x7fd17c0958c0 2021-02-10T11:35:47.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.749 7fd183d77700 1 -- 172.21.2.101:0/2049256298 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd164018730 msgr2=0x7fd16401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:47.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.749 7fd183d77700 1 --2- 172.21.2.101:0/2049256298 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd164018730 0x7fd16401ac10 crc :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:47.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.749 7fd183d77700 1 -- 172.21.2.101:0/2049256298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd17c0958c0 msgr2=0x7fd17c096920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:47.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.749 7fd183d77700 1 --2- 172.21.2.101:0/2049256298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd17c0958c0 0x7fd17c096920 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 rx=0x7fd174051930 tx=0x7fd174051cd0).stop 2021-02-10T11:35:47.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.749 7fd183d77700 1 -- 172.21.2.101:0/2049256298 shutdown_connections 2021-02-10T11:35:47.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.749 7fd183d77700 1 --2- 172.21.2.101:0/2049256298 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd164018730 0x7fd16401ac10 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:47.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.749 7fd183d77700 1 --2- 172.21.2.101:0/2049256298 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd17c0954c0 0x7fd17c096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:47.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.749 7fd183d77700 1 --2- 172.21.2.101:0/2049256298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd17c0958c0 0x7fd17c096920 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:47.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.749 7fd183d77700 1 --2- 172.21.2.101:0/2049256298 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd17c085bc0 0x7fd17c0863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:47.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.749 7fd183d77700 1 -- 172.21.2.101:0/2049256298 >> 172.21.2.101:0/2049256298 conn(0x7fd17c0bef00 msgr2=0x7fd17c09d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:47.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.749 7fd183d77700 1 -- 172.21.2.101:0/2049256298 shutdown_connections 2021-02-10T11:35:47.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:47.749 7fd183d77700 1 -- 172.21.2.101:0/2049256298 wait complete. 2021-02-10T11:35:47.765 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:35:47.765 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:35:48.110 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:35:48.455 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:35:48.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:48.455 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:35:48.456 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:35:48.792 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2021-02-10T11:35:48.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:35:48.792 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-02-10T11:35:48.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:48.793 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:35:48.793 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:35:49.127 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:35:49.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:35:49.127 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:35:49.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:49.127 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:35:49.128 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:35:49.463 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:35:49.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:35:49.463 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:35:49.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:35:49.463 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:35:49.464 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:35:49.798 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:35:49.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:35:49.799 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:35:49.799 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json-pretty 2021-02-10T11:35:50.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.203 7fd4137e4700 1 Processor -- start 2021-02-10T11:35:50.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.203 7fd4137e4700 1 -- start start 2021-02-10T11:35:50.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.203 7fd4137e4700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd40c0d1870 0x7fd40c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:50.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.203 7fd4137e4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd40c1b0a10 0x7fd40c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:50.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.203 7fd4137e4700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40c0f0b10 con 0x7fd40c0d1c70 2021-02-10T11:35:50.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.203 7fd4137e4700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd40c0e5ed0 con 0x7fd40c0d1870 2021-02-10T11:35:50.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.203 7fd4137e4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0f0650 con 0x7fd40c1b0a10 2021-02-10T11:35:50.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.204 7fd411e01700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fd40c0d1c70 0x7fd40c0cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:47254/0 (socket says 172.21.2.101:47254) 2021-02-10T11:35:50.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.204 7fd411e01700 1 -- 172.21.2.101:0/868025590 learned_addr learned my addr 172.21.2.101:0/868025590 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:50.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.204 7fd410dff700 1 --2- 172.21.2.101:0/868025590 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd40c1b0a10 0x7fd40c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:50.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.204 7fd411600700 1 --2- 172.21.2.101:0/868025590 >> v2:172.21.2.108:3300/0 conn(0x7fd40c0d1870 0x7fd40c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:50.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.204 7fd410dff700 1 -- 172.21.2.101:0/868025590 >> v2:172.21.2.108:3300/0 conn(0x7fd40c0d1870 msgr2=0x7fd40c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:50.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.204 7fd410dff700 1 --2- 172.21.2.101:0/868025590 >> v2:172.21.2.108:3300/0 conn(0x7fd40c0d1870 0x7fd40c1b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:50.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.204 7fd410dff700 1 -- 172.21.2.101:0/868025590 >> v1:172.21.2.108:6789/0 conn(0x7fd40c0d1c70 legacy=0x7fd40c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:50.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.204 7fd410dff700 1 -- 172.21.2.101:0/868025590 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c0f0870 con 0x7fd40c1b0a10 2021-02-10T11:35:50.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.204 7fd411600700 1 --2- 172.21.2.101:0/868025590 >> v2:172.21.2.108:3300/0 conn(0x7fd40c0d1870 0x7fd40c1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:35:50.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.205 7fd410dff700 1 --2- 172.21.2.101:0/868025590 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd40c1b0a10 0x7fd40c0cf330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fd408006650 tx=0x7fd408009a80).ready entity=mon.2 client_cookie=72ad5e945d85124d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:50.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.205 7fd3fffff700 1 -- 172.21.2.101:0/868025590 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd4080009b0 con 0x7fd40c1b0a10 2021-02-10T11:35:50.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.205 7fd3fffff700 1 -- 172.21.2.101:0/868025590 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40800c0c0 con 0x7fd40c1b0a10 2021-02-10T11:35:50.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.205 7fd3fffff700 1 -- 172.21.2.101:0/868025590 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd40800c5e0 con 0x7fd40c1b0a10 2021-02-10T11:35:50.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.205 7fd4137e4700 1 -- 172.21.2.101:0/868025590 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd40c1b0a10 msgr2=0x7fd40c0cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:50.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.205 7fd4137e4700 1 --2- 172.21.2.101:0/868025590 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd40c1b0a10 0x7fd40c0cf330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fd408006650 tx=0x7fd408009a80).stop 2021-02-10T11:35:50.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.206 7fd4137e4700 1 -- 172.21.2.101:0/868025590 shutdown_connections 2021-02-10T11:35:50.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.206 7fd4137e4700 1 --2- 172.21.2.101:0/868025590 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd40c1b0a10 0x7fd40c0cf330 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:50.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.206 7fd4137e4700 1 --2- 172.21.2.101:0/868025590 >> v2:172.21.2.108:3300/0 conn(0x7fd40c0d1870 0x7fd40c1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:50.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.206 7fd4137e4700 1 -- 172.21.2.101:0/868025590 >> 172.21.2.101:0/868025590 conn(0x7fd40c0ce450 msgr2=0x7fd40c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:50.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.206 7fd4137e4700 1 -- 172.21.2.101:0/868025590 shutdown_connections 2021-02-10T11:35:50.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.206 7fd4137e4700 1 -- 172.21.2.101:0/868025590 wait complete. 2021-02-10T11:35:50.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.220 7fd4137e4700 1 Processor -- start 2021-02-10T11:35:50.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.220 7fd4137e4700 1 -- start start 2021-02-10T11:35:50.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.221 7fd4137e4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd40c1942b0 0x7fd40c0b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:50.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.221 7fd4137e4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd40c195070 0x7fd40c195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:50.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.221 7fd4137e4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd40c0bb8e0 0x7fd40c0bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:50.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.221 7fd4137e4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c1946b0 con 0x7fd40c0bb8e0 2021-02-10T11:35:50.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.221 7fd4137e4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0b3b70 con 0x7fd40c195070 2021-02-10T11:35:50.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.221 7fd4137e4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0b3d90 con 0x7fd40c1942b0 2021-02-10T11:35:50.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.221 7fd411e01700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd40c0bb8e0 0x7fd40c0bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:50.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.221 7fd411e01700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd40c0bb8e0 0x7fd40c0bbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53360/0 (socket says 172.21.2.101:53360) 2021-02-10T11:35:50.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.221 7fd411e01700 1 -- 172.21.2.101:0/2986644757 learned_addr learned my addr 172.21.2.101:0/2986644757 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:50.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.221 7fd411600700 1 --2- 172.21.2.101:0/2986644757 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd40c1942b0 0x7fd40c0b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:50.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.221 7fd410dff700 1 --2- 172.21.2.101:0/2986644757 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd40c195070 0x7fd40c195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:50.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.221 7fd411e01700 1 -- 172.21.2.101:0/2986644757 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd40c1942b0 msgr2=0x7fd40c0b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:50.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.221 7fd411e01700 1 --2- 172.21.2.101:0/2986644757 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd40c1942b0 0x7fd40c0b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:50.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.221 7fd411e01700 1 -- 172.21.2.101:0/2986644757 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd40c195070 msgr2=0x7fd40c195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:50.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.221 7fd411e01700 1 --2- 172.21.2.101:0/2986644757 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd40c195070 0x7fd40c195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:50.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.221 7fd411e01700 1 -- 172.21.2.101:0/2986644757 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c1a49e0 con 0x7fd40c0bb8e0 2021-02-10T11:35:50.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.221 7fd411e01700 1 --2- 172.21.2.101:0/2986644757 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd40c0bb8e0 0x7fd40c0bbce0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 rx=0x7fd404007f90 tx=0x7fd40400ac70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:50.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.222 7fd3fdffb700 1 -- 172.21.2.101:0/2986644757 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd404001500 con 0x7fd40c0bb8e0 2021-02-10T11:35:50.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.222 7fd3fdffb700 1 -- 172.21.2.101:0/2986644757 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4040070d0 con 0x7fd40c0bb8e0 2021-02-10T11:35:50.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.222 7fd3fdffb700 1 -- 172.21.2.101:0/2986644757 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd4040075f0 con 0x7fd40c0bb8e0 2021-02-10T11:35:50.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.222 7fd4137e4700 1 -- 172.21.2.101:0/2986644757 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40c1a49e0 con 0x7fd40c0bb8e0 2021-02-10T11:35:50.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.222 7fd4137e4700 1 -- 172.21.2.101:0/2986644757 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40c0bc5f0 con 0x7fd40c0bb8e0 2021-02-10T11:35:50.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.222 7fd3fdffb700 1 -- 172.21.2.101:0/2986644757 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd4040072b0 con 0x7fd40c0bb8e0 2021-02-10T11:35:50.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.222 7fd3fdffb700 1 --2- 172.21.2.101:0/2986644757 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3ec0186c0 0x7fd3ec01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:50.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.222 7fd3fdffb700 1 -- 172.21.2.101:0/2986644757 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd4040328f0 con 0x7fd40c0bb8e0 2021-02-10T11:35:50.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.222 7fd411600700 1 --2- 172.21.2.101:0/2986644757 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3ec0186c0 0x7fd3ec01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:50.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.223 7fd411600700 1 --2- 172.21.2.101:0/2986644757 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3ec0186c0 0x7fd3ec01abc0 crc :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:50.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.253 7fd4137e4700 1 -- 172.21.2.101:0/2986644757 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40c0bc5f0 con 0x7fd40c0bb8e0 2021-02-10T11:35:50.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.255 7fd3fdffb700 1 -- 172.21.2.101:0/2986644757 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd404001310 con 0x7fd40c0bb8e0 2021-02-10T11:35:50.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.378 7fd4137e4700 1 -- 172.21.2.101:0/2986644757 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd40c0bc5f0 con 0x7fd40c0bb8e0 2021-02-10T11:35:50.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.379 7fd3fdffb700 1 -- 172.21.2.101:0/2986644757 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v65) v1 ==== 81+0+30363 (secure 0 0 0) 0x7fd4040072b0 con 0x7fd40c0bb8e0 2021-02-10T11:35:50.387 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:50.387 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:35:50.387 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 65, 2021-02-10T11:35:50.387 INFO:teuthology.orchestra.run.gibba001.stdout: "fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:35:50.387 INFO:teuthology.orchestra.run.gibba001.stdout: "created": "2021-02-10 11:12:36.788366", 2021-02-10T11:35:50.387 INFO:teuthology.orchestra.run.gibba001.stdout: "modified": "2021-02-10 11:34:42.090329", 2021-02-10T11:35:50.387 INFO:teuthology.orchestra.run.gibba001.stdout: "last_up_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:35:50.388 INFO:teuthology.orchestra.run.gibba001.stdout: "last_in_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:35:50.388 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-02-10T11:35:50.388 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_num": 5799936, 2021-02-10T11:35:50.388 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_set": [ 2021-02-10T11:35:50.388 INFO:teuthology.orchestra.run.gibba001.stdout: "pglog_hardlimit", 2021-02-10T11:35:50.388 INFO:teuthology.orchestra.run.gibba001.stdout: "purged_snapdirs", 2021-02-10T11:35:50.388 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_deletes", 2021-02-10T11:35:50.388 INFO:teuthology.orchestra.run.gibba001.stdout: "sortbitwise" 2021-02-10T11:35:50.388 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:35:50.388 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_version": 4, 2021-02-10T11:35:50.388 INFO:teuthology.orchestra.run.gibba001.stdout: "full_ratio": 0.94999998807907104, 2021-02-10T11:35:50.388 INFO:teuthology.orchestra.run.gibba001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-02-10T11:35:50.388 INFO:teuthology.orchestra.run.gibba001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-02-10T11:35:50.389 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_snapshot": "", 2021-02-10T11:35:50.389 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_max": 13, 2021-02-10T11:35:50.389 INFO:teuthology.orchestra.run.gibba001.stdout: "max_osd": 8, 2021-02-10T11:35:50.389 INFO:teuthology.orchestra.run.gibba001.stdout: "require_min_compat_client": "jewel", 2021-02-10T11:35:50.389 INFO:teuthology.orchestra.run.gibba001.stdout: "min_compat_client": "jewel", 2021-02-10T11:35:50.389 INFO:teuthology.orchestra.run.gibba001.stdout: "require_osd_release": "nautilus", 2021-02-10T11:35:50.389 INFO:teuthology.orchestra.run.gibba001.stdout: "pools": [ 2021-02-10T11:35:50.389 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.389 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 1, 2021-02-10T11:35:50.389 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "rbd", 2021-02-10T11:35:50.389 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:13:08.037472", 2021-02-10T11:35:50.389 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:35:50.389 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:35:50.389 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:35:50.390 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:35:50.390 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:35:50.390 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:35:50.390 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:35:50.390 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:35:50.390 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:35:50.390 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:35:50.390 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:35:50.390 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:35:50.390 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:35:50.390 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:35:50.390 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:35:50.390 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:35:50.391 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:35:50.391 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:35:50.391 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:35:50.391 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:35:50.391 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.391 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "16", 2021-02-10T11:35:50.391 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:35:50.391 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:35:50.391 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:35:50.391 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:35:50.391 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:35:50.391 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:35:50.391 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:35:50.391 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:35:50.392 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:35:50.392 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:35:50.392 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:35:50.392 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:35:50.392 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:35:50.392 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:35:50.392 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:35:50.392 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:35:50.392 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:35:50.392 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:35:50.392 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:35:50.392 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:35:50.393 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:35:50.393 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:35:50.393 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:35:50.393 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:35:50.393 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:35:50.393 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:35:50.393 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.393 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:35:50.393 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:35:50.394 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:35:50.394 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:35:50.394 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:35:50.394 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:35:50.394 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:35:50.394 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:35:50.394 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:35:50.394 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:35:50.394 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:35:50.395 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:35:50.395 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:35:50.395 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd": {} 2021-02-10T11:35:50.395 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.395 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.395 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.395 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 12, 2021-02-10T11:35:50.395 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_metadata", 2021-02-10T11:35:50.395 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:34:38.607957", 2021-02-10T11:35:50.395 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:35:50.395 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:35:50.396 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:35:50.396 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:35:50.396 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:35:50.396 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:35:50.396 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:35:50.396 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:35:50.396 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:35:50.396 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:35:50.396 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:35:50.396 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:35:50.396 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:35:50.396 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:35:50.396 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:35:50.397 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:35:50.397 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:35:50.397 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:35:50.397 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:35:50.397 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:35:50.397 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.397 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "64", 2021-02-10T11:35:50.397 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:35:50.397 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:35:50.397 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:35:50.397 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:35:50.397 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:35:50.397 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:35:50.398 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:35:50.398 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:35:50.398 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:35:50.398 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:35:50.398 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:35:50.398 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:35:50.398 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:35:50.398 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:35:50.398 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:35:50.398 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:35:50.398 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:35:50.398 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:35:50.398 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:35:50.399 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:35:50.399 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:35:50.399 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:35:50.399 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:35:50.399 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:35:50.399 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:35:50.399 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:35:50.399 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.399 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:35:50.399 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:35:50.399 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:35:50.399 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:35:50.399 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:35:50.399 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:35:50.400 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:35:50.400 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:35:50.400 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:35:50.400 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:35:50.400 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:35:50.400 INFO:teuthology.orchestra.run.gibba001.stdout: "options": { 2021-02-10T11:35:50.400 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_bias": 4, 2021-02-10T11:35:50.400 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_min": 16, 2021-02-10T11:35:50.400 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_priority": 5 2021-02-10T11:35:50.400 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.400 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:35:50.400 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:35:50.400 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": "cephfs" 2021-02-10T11:35:50.400 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.401 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.401 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.401 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.401 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 13, 2021-02-10T11:35:50.401 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_data", 2021-02-10T11:35:50.401 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:34:40.084211", 2021-02-10T11:35:50.401 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:35:50.401 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:35:50.401 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:35:50.401 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:35:50.401 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:35:50.401 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:35:50.401 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:35:50.401 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:35:50.402 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:35:50.402 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:35:50.402 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:35:50.402 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:35:50.402 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:35:50.402 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:35:50.402 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:35:50.402 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:35:50.402 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:35:50.402 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:35:50.402 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:35:50.402 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:35:50.402 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.402 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "65", 2021-02-10T11:35:50.403 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:35:50.403 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:35:50.403 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:35:50.403 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:35:50.403 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:35:50.403 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:35:50.403 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:35:50.403 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:35:50.403 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:35:50.403 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:35:50.403 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:35:50.403 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:35:50.403 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:35:50.404 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:35:50.404 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:35:50.404 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:35:50.404 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:35:50.404 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:35:50.404 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:35:50.404 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:35:50.404 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:35:50.404 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:35:50.404 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:35:50.404 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:35:50.404 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:35:50.404 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:35:50.404 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.405 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:35:50.405 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:35:50.405 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:35:50.405 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:35:50.405 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:35:50.405 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:35:50.405 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:35:50.405 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:35:50.405 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:35:50.405 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:35:50.405 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:35:50.405 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:35:50.405 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:35:50.405 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:35:50.406 INFO:teuthology.orchestra.run.gibba001.stdout: "data": "cephfs" 2021-02-10T11:35:50.406 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.406 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.406 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.406 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:35:50.406 INFO:teuthology.orchestra.run.gibba001.stdout: "osds": [ 2021-02-10T11:35:50.406 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.406 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:35:50.406 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5", 2021-02-10T11:35:50.406 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:35:50.406 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:35:50.406 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:35:50.406 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:35:50.407 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:35:50.407 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:35:50.407 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:35:50.407 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 63, 2021-02-10T11:35:50.407 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:35:50.407 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:35:50.407 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:35:50.407 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.407 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.407 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.407 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6802", 2021-02-10T11:35:50.407 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:35:50.407 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.408 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.408 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.408 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6805", 2021-02-10T11:35:50.408 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:35:50.408 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.408 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.408 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.408 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:35:50.408 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.408 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.408 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.408 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6807", 2021-02-10T11:35:50.408 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:35:50.408 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.409 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.409 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.409 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6809", 2021-02-10T11:35:50.409 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:35:50.409 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.409 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.409 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.409 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:35:50.409 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.409 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.409 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.409 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6817", 2021-02-10T11:35:50.409 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:35:50.410 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.410 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.410 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.410 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6820", 2021-02-10T11:35:50.410 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:35:50.410 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.410 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.410 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.410 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:35:50.410 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.410 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.410 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.410 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6812", 2021-02-10T11:35:50.410 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:35:50.411 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.411 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.411 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.411 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6814", 2021-02-10T11:35:50.411 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:35:50.411 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.411 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.411 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.411 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6805/25093", 2021-02-10T11:35:50.411 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6809/25093", 2021-02-10T11:35:50.411 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6820/25093", 2021-02-10T11:35:50.411 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6814/25093", 2021-02-10T11:35:50.411 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:35:50.411 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:35:50.412 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:35:50.412 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.412 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.412 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.412 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:35:50.412 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "ddc04ce3-b313-4707-8270-585391e38ecc", 2021-02-10T11:35:50.412 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:35:50.412 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:35:50.412 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:35:50.412 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:35:50.412 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:35:50.412 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:35:50.412 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:35:50.413 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 62, 2021-02-10T11:35:50.413 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:35:50.413 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:35:50.413 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:35:50.413 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.413 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.413 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.413 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6803", 2021-02-10T11:35:50.413 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:35:50.413 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.413 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.413 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.413 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6804", 2021-02-10T11:35:50.413 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:35:50.414 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.414 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.414 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.414 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:35:50.414 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.414 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.414 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.414 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6806", 2021-02-10T11:35:50.414 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:35:50.414 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.414 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.414 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.414 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6808", 2021-02-10T11:35:50.414 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:35:50.415 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.415 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.415 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.415 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:35:50.415 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.415 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.415 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.415 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6818", 2021-02-10T11:35:50.415 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:35:50.415 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.415 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.415 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.415 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6821", 2021-02-10T11:35:50.415 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:35:50.416 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.416 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.416 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.416 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:35:50.416 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.416 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.416 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.416 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6810", 2021-02-10T11:35:50.416 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:35:50.416 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.416 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.416 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.416 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6815", 2021-02-10T11:35:50.417 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:35:50.417 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.417 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.417 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.417 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6804/25092", 2021-02-10T11:35:50.417 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6808/25092", 2021-02-10T11:35:50.417 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6821/25092", 2021-02-10T11:35:50.417 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6815/25092", 2021-02-10T11:35:50.417 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:35:50.417 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:35:50.417 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:35:50.417 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.417 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.417 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.418 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:35:50.418 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3", 2021-02-10T11:35:50.418 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:35:50.418 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:35:50.418 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:35:50.418 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:35:50.418 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:35:50.418 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:35:50.418 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:35:50.418 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 53, 2021-02-10T11:35:50.418 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:35:50.418 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:35:50.418 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:35:50.418 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.419 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.419 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.419 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6811", 2021-02-10T11:35:50.419 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:35:50.419 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.419 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.419 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.419 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6813", 2021-02-10T11:35:50.419 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:35:50.419 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.419 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.419 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.419 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:35:50.420 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.420 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.420 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.420 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6816", 2021-02-10T11:35:50.420 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:35:50.420 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.420 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.420 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.420 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6819", 2021-02-10T11:35:50.420 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:35:50.420 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.420 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.420 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.420 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:35:50.421 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.421 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.421 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.421 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6825", 2021-02-10T11:35:50.421 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:35:50.421 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.421 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.421 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.421 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6827", 2021-02-10T11:35:50.421 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:35:50.421 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.421 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.421 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.421 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:35:50.422 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.422 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.422 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.422 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6822", 2021-02-10T11:35:50.422 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:35:50.422 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.422 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.422 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.422 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6823", 2021-02-10T11:35:50.422 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:35:50.422 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.422 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.422 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.423 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6813/25094", 2021-02-10T11:35:50.423 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6819/25094", 2021-02-10T11:35:50.423 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6827/25094", 2021-02-10T11:35:50.423 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6823/25094", 2021-02-10T11:35:50.423 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:35:50.423 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:35:50.423 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:35:50.423 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.423 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.423 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.423 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:35:50.423 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "0d9ef9b1-98aa-4d37-aa16-6305c7d46d52", 2021-02-10T11:35:50.423 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:35:50.423 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:35:50.424 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:35:50.424 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:35:50.424 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:35:50.424 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:35:50.424 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:35:50.424 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 45, 2021-02-10T11:35:50.424 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:35:50.424 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:35:50.424 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:35:50.424 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.424 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.424 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.424 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6824", 2021-02-10T11:35:50.424 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:35:50.425 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.425 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.425 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.425 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6826", 2021-02-10T11:35:50.425 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:35:50.425 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.425 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.425 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.425 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:35:50.425 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.425 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.425 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.425 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6828", 2021-02-10T11:35:50.425 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:35:50.426 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.426 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.426 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.426 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6829", 2021-02-10T11:35:50.426 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:35:50.426 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.426 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.426 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.426 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:35:50.426 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.426 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.426 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.426 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6832", 2021-02-10T11:35:50.427 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:35:50.427 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.427 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.427 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.427 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6833", 2021-02-10T11:35:50.427 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:35:50.427 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.427 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.427 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.427 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:35:50.427 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.427 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.427 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.427 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6830", 2021-02-10T11:35:50.428 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:35:50.428 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.428 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.428 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.428 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6831", 2021-02-10T11:35:50.428 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:35:50.428 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.428 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.428 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.428 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6826/25095", 2021-02-10T11:35:50.428 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6829/25095", 2021-02-10T11:35:50.428 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6833/25095", 2021-02-10T11:35:50.428 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6831/25095", 2021-02-10T11:35:50.428 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:35:50.429 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:35:50.429 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:35:50.429 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.429 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.429 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.429 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:35:50.429 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "36742c21-bfeb-4e11-bce9-8e745378b605", 2021-02-10T11:35:50.429 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:35:50.429 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:35:50.429 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:35:50.429 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:35:50.429 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:35:50.429 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:35:50.429 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:35:50.430 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 63, 2021-02-10T11:35:50.430 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:35:50.430 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:35:50.430 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:35:50.430 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.430 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.430 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.430 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6816", 2021-02-10T11:35:50.430 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:35:50.430 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.430 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.430 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.430 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6818", 2021-02-10T11:35:50.431 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:35:50.431 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.431 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.431 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.431 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:35:50.431 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.431 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.431 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.431 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6820", 2021-02-10T11:35:50.431 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:35:50.431 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.431 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.431 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.431 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6822", 2021-02-10T11:35:50.432 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:35:50.432 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.432 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.432 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.432 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:35:50.432 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.432 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.432 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.432 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6828", 2021-02-10T11:35:50.432 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:35:50.432 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.432 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.432 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.432 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6830", 2021-02-10T11:35:50.433 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:35:50.433 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.433 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.433 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.433 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:35:50.433 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.433 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.433 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.433 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6824", 2021-02-10T11:35:50.433 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:35:50.433 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.433 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.433 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.433 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6826", 2021-02-10T11:35:50.434 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:35:50.434 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.434 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.434 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.434 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6818/36380", 2021-02-10T11:35:50.434 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6822/36380", 2021-02-10T11:35:50.434 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6830/36380", 2021-02-10T11:35:50.434 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6826/36380", 2021-02-10T11:35:50.434 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:35:50.434 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:35:50.434 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:35:50.434 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.434 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.435 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.435 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:35:50.435 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "88e694e8-fe98-4d1b-88c8-8c9907ece60b", 2021-02-10T11:35:50.435 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:35:50.435 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:35:50.435 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:35:50.435 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:35:50.435 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:35:50.435 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:35:50.435 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:35:50.435 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 63, 2021-02-10T11:35:50.435 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:35:50.435 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:35:50.435 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:35:50.436 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.436 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.436 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.436 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6808", 2021-02-10T11:35:50.436 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:35:50.436 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.436 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.436 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.436 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6809", 2021-02-10T11:35:50.436 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:35:50.436 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.436 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.436 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.436 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:35:50.437 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.437 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.437 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.437 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6810", 2021-02-10T11:35:50.437 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:35:50.437 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.437 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.437 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.437 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6811", 2021-02-10T11:35:50.437 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:35:50.437 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.437 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.437 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.438 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:35:50.438 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.438 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.438 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.438 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6814", 2021-02-10T11:35:50.438 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:35:50.438 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.438 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.438 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.438 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6815", 2021-02-10T11:35:50.438 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:35:50.438 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.439 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.439 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.439 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:35:50.439 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.439 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.439 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.440 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6812", 2021-02-10T11:35:50.440 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:35:50.440 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.440 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.440 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.440 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6813", 2021-02-10T11:35:50.440 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:35:50.440 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.440 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.440 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.440 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6809/36383", 2021-02-10T11:35:50.440 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6811/36383", 2021-02-10T11:35:50.440 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6815/36383", 2021-02-10T11:35:50.440 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6813/36383", 2021-02-10T11:35:50.441 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:35:50.441 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:35:50.441 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:35:50.441 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.441 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.441 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.441 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:35:50.441 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "87a13c6c-cf85-4aaf-916b-e73913484a49", 2021-02-10T11:35:50.441 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:35:50.441 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:35:50.441 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:35:50.441 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:35:50.441 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:35:50.441 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:35:50.442 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:35:50.442 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 63, 2021-02-10T11:35:50.442 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:35:50.442 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:35:50.442 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:35:50.442 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.442 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.442 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.442 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6800", 2021-02-10T11:35:50.442 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:35:50.442 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.442 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.442 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.443 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6801", 2021-02-10T11:35:50.443 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:35:50.443 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.443 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.443 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.443 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:35:50.443 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.443 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.443 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.443 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6802", 2021-02-10T11:35:50.443 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:35:50.444 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.444 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.444 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.444 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6803", 2021-02-10T11:35:50.444 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:35:50.444 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.444 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.444 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.444 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:35:50.444 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.444 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.444 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.444 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6806", 2021-02-10T11:35:50.444 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:35:50.445 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.445 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.445 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.445 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6807", 2021-02-10T11:35:50.445 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:35:50.445 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.445 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.445 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.445 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:35:50.445 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.445 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.445 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.445 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6804", 2021-02-10T11:35:50.446 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:35:50.446 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.446 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.446 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.446 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6805", 2021-02-10T11:35:50.446 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:35:50.446 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.446 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.446 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.446 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6801/36381", 2021-02-10T11:35:50.446 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6803/36381", 2021-02-10T11:35:50.446 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6807/36381", 2021-02-10T11:35:50.446 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6805/36381", 2021-02-10T11:35:50.447 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:35:50.447 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:35:50.447 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:35:50.447 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.447 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.447 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.447 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:35:50.447 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "7b90823f-854c-4592-a95b-a7fa98f93254", 2021-02-10T11:35:50.447 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:35:50.447 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:35:50.447 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:35:50.447 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:35:50.447 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:35:50.447 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:35:50.448 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:35:50.448 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 63, 2021-02-10T11:35:50.448 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:35:50.448 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:35:50.448 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:35:50.448 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.448 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.448 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.448 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6817", 2021-02-10T11:35:50.448 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:35:50.448 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.448 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.448 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.448 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6819", 2021-02-10T11:35:50.449 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:35:50.449 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.449 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.449 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.449 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:35:50.449 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.449 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.449 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.449 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6821", 2021-02-10T11:35:50.449 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:35:50.449 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.449 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.449 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.449 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6823", 2021-02-10T11:35:50.450 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:35:50.450 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.450 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.450 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.450 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:35:50.450 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.450 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.450 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.450 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6829", 2021-02-10T11:35:50.450 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:35:50.450 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.450 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.450 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.450 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6831", 2021-02-10T11:35:50.451 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:35:50.451 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.451 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.451 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.451 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:35:50.451 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:35:50.451 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.451 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:35:50.451 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6825", 2021-02-10T11:35:50.451 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:35:50.451 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.451 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.451 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:35:50.451 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6827", 2021-02-10T11:35:50.452 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:35:50.452 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.452 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.452 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.452 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6819/36382", 2021-02-10T11:35:50.452 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6823/36382", 2021-02-10T11:35:50.452 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6831/36382", 2021-02-10T11:35:50.452 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6827/36382", 2021-02-10T11:35:50.452 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:35:50.452 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:35:50.452 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:35:50.452 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:50.452 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.452 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:35:50.453 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_xinfo": [ 2021-02-10T11:35:50.453 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.453 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:35:50.453 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:35:50.453 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:35:50.453 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:35:50.453 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:35:50.453 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:35:50.453 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.453 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.453 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:35:50.453 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:35:50.453 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:35:50.453 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:35:50.454 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:35:50.454 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:35:50.454 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.454 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.454 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:35:50.454 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:35:50.454 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:35:50.454 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:35:50.454 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:35:50.454 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:35:50.454 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.454 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.454 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:35:50.454 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:35:50.455 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:35:50.455 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:35:50.455 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:35:50.455 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:35:50.455 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.455 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.455 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:35:50.455 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:35:50.455 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:35:50.455 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:35:50.455 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:35:50.455 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:35:50.455 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.456 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.456 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:35:50.456 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:35:50.456 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:35:50.456 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:35:50.456 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:35:50.456 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:35:50.456 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.456 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.456 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:35:50.456 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:35:50.456 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:35:50.456 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:35:50.457 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:35:50.457 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:35:50.457 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.457 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:50.457 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:35:50.458 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:35:50.458 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:35:50.458 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:35:50.458 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:35:50.458 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:35:50.458 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.459 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:35:50.459 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap": [], 2021-02-10T11:35:50.459 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap_items": [], 2021-02-10T11:35:50.459 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_temp": [], 2021-02-10T11:35:50.459 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_temp": [], 2021-02-10T11:35:50.459 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklist": {}, 2021-02-10T11:35:50.459 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profiles": { 2021-02-10T11:35:50.459 INFO:teuthology.orchestra.run.gibba001.stdout: "default": { 2021-02-10T11:35:50.459 INFO:teuthology.orchestra.run.gibba001.stdout: "crush-failure-domain": "osd", 2021-02-10T11:35:50.459 INFO:teuthology.orchestra.run.gibba001.stdout: "k": "2", 2021-02-10T11:35:50.459 INFO:teuthology.orchestra.run.gibba001.stdout: "m": "1", 2021-02-10T11:35:50.459 INFO:teuthology.orchestra.run.gibba001.stdout: "plugin": "jerasure", 2021-02-10T11:35:50.459 INFO:teuthology.orchestra.run.gibba001.stdout: "ruleset-failure-domain": "osd", 2021-02-10T11:35:50.459 INFO:teuthology.orchestra.run.gibba001.stdout: "technique": "reed_sol_van" 2021-02-10T11:35:50.460 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:50.460 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:50.460 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps_queue": [], 2021-02-10T11:35:50.460 INFO:teuthology.orchestra.run.gibba001.stdout: "new_removed_snaps": [], 2021-02-10T11:35:50.460 INFO:teuthology.orchestra.run.gibba001.stdout: "new_purged_snaps": [], 2021-02-10T11:35:50.460 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_node_flags": {}, 2021-02-10T11:35:50.460 INFO:teuthology.orchestra.run.gibba001.stdout: "device_class_flags": {} 2021-02-10T11:35:50.460 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:35:50.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.380 7fd4137e4700 1 -- 172.21.2.101:0/2986644757 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3ec0186c0 msgr2=0x7fd3ec01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:50.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.380 7fd4137e4700 1 --2- 172.21.2.101:0/2986644757 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3ec0186c0 0x7fd3ec01abc0 crc :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:50.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.380 7fd4137e4700 1 -- 172.21.2.101:0/2986644757 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd40c0bb8e0 msgr2=0x7fd40c0bbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:50.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.380 7fd4137e4700 1 --2- 172.21.2.101:0/2986644757 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd40c0bb8e0 0x7fd40c0bbce0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 rx=0x7fd404007f90 tx=0x7fd40400ac70).stop 2021-02-10T11:35:50.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.381 7fd4137e4700 1 -- 172.21.2.101:0/2986644757 shutdown_connections 2021-02-10T11:35:50.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.381 7fd4137e4700 1 --2- 172.21.2.101:0/2986644757 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd40c0bb8e0 0x7fd40c0bbce0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:50.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.381 7fd4137e4700 1 --2- 172.21.2.101:0/2986644757 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd40c195070 0x7fd40c195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:50.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.381 7fd4137e4700 1 --2- 172.21.2.101:0/2986644757 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3ec0186c0 0x7fd3ec01abc0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:50.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.381 7fd4137e4700 1 --2- 172.21.2.101:0/2986644757 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd40c1942b0 0x7fd40c0b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:50.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.381 7fd4137e4700 1 -- 172.21.2.101:0/2986644757 >> 172.21.2.101:0/2986644757 conn(0x7fd40c0b3040 msgr2=0x7fd40c0c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:50.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.381 7fd4137e4700 1 -- 172.21.2.101:0/2986644757 shutdown_connections 2021-02-10T11:35:50.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.381 7fd4137e4700 1 -- 172.21.2.101:0/2986644757 wait complete. 2021-02-10T11:35:50.463 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:35:50.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.866 7eff83540700 1 Processor -- start 2021-02-10T11:35:50.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.866 7eff83540700 1 -- start start 2021-02-10T11:35:50.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.866 7eff83540700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7eff7c0d1c70 0x7eff7c0f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:50.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.866 7eff83540700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff7c0f0b10 con 0x7eff7c0d1870 2021-02-10T11:35:50.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.866 7eff83540700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff7c1b0be0 con 0x7eff7c0d0340 2021-02-10T11:35:50.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.866 7eff83540700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff7c0e5ed0 con 0x7eff7c0d1c70 2021-02-10T11:35:50.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.866 7eff81b5d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7eff7c0d1c70 0x7eff7c0f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:50.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.866 7eff81b5d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7eff7c0d1c70 0x7eff7c0f20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53364/0 (socket says 172.21.2.101:53364) 2021-02-10T11:35:50.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.866 7eff8135c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7eff7c0d1870 0x7eff7c1b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53858/0 (socket says 172.21.2.101:53858) 2021-02-10T11:35:50.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.866 7eff81b5d700 1 -- 172.21.2.101:0/4025169363 learned_addr learned my addr 172.21.2.101:0/4025169363 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:50.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.867 7eff73fff700 1 -- 172.21.2.101:0/4025169363 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2988176143 0 0) 0x7eff78001100 con 0x7eff7c0d1870 2021-02-10T11:35:50.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.867 7eff73fff700 1 -- 172.21.2.101:0/4025169363 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff68002fe0 con 0x7eff7c0d1870 2021-02-10T11:35:50.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.867 7eff81b5d700 1 -- 172.21.2.101:0/4025169363 >> v1:172.21.2.108:6789/0 conn(0x7eff7c0d0340 legacy=0x7eff7c0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:50.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.867 7eff81b5d700 1 -- 172.21.2.101:0/4025169363 >> v1:172.21.2.101:6789/0 conn(0x7eff7c0d1870 legacy=0x7eff7c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:50.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.867 7eff81b5d700 1 -- 172.21.2.101:0/4025169363 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7c0efe30 con 0x7eff7c0d1c70 2021-02-10T11:35:50.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.867 7eff81b5d700 1 --2- 172.21.2.101:0/4025169363 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff7c0d1c70 0x7eff7c0f20a0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 rx=0x7eff74006f00 tx=0x7eff74009f10).ready entity=mon.0 client_cookie=3e6b79588a2a620e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:50.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.867 7eff73fff700 1 -- 172.21.2.101:0/4025169363 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7eff7400b3b0 con 0x7eff7c0d1c70 2021-02-10T11:35:50.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.867 7eff73fff700 1 -- 172.21.2.101:0/4025169363 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7400b630 con 0x7eff7c0d1c70 2021-02-10T11:35:50.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.867 7eff73fff700 1 -- 172.21.2.101:0/4025169363 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7eff7400bb50 con 0x7eff7c0d1c70 2021-02-10T11:35:50.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.868 7eff83540700 1 -- 172.21.2.101:0/4025169363 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff7c0d1c70 msgr2=0x7eff7c0f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:50.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.868 7eff83540700 1 --2- 172.21.2.101:0/4025169363 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff7c0d1c70 0x7eff7c0f20a0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 rx=0x7eff74006f00 tx=0x7eff74009f10).stop 2021-02-10T11:35:50.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.868 7eff83540700 1 -- 172.21.2.101:0/4025169363 shutdown_connections 2021-02-10T11:35:50.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.868 7eff83540700 1 --2- 172.21.2.101:0/4025169363 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff7c0d1c70 0x7eff7c0f20a0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:50.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.868 7eff83540700 1 -- 172.21.2.101:0/4025169363 >> 172.21.2.101:0/4025169363 conn(0x7eff7c0ce450 msgr2=0x7eff7c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:50.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.868 7eff83540700 1 -- 172.21.2.101:0/4025169363 shutdown_connections 2021-02-10T11:35:50.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.868 7eff83540700 1 -- 172.21.2.101:0/4025169363 wait complete. 2021-02-10T11:35:50.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.883 7eff83540700 1 Processor -- start 2021-02-10T11:35:50.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.883 7eff83540700 1 -- start start 2021-02-10T11:35:50.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.884 7eff83540700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff7c1b8ad0 0x7eff7c187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:50.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.884 7eff83540700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7eff7c1b8ed0 0x7eff7c0b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:50.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.884 7eff83540700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7eff7c0b2110 0x7eff7c1b7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:50.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.884 7eff83540700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c188470 con 0x7eff7c1b8ad0 2021-02-10T11:35:50.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.884 7eff83540700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c18be10 con 0x7eff7c0b2110 2021-02-10T11:35:50.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.884 7eff83540700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c18c030 con 0x7eff7c1b8ed0 2021-02-10T11:35:50.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.884 7eff8135c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff7c1b8ad0 0x7eff7c187e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:50.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.884 7eff8135c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff7c1b8ad0 0x7eff7c187e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53372/0 (socket says 172.21.2.101:53372) 2021-02-10T11:35:50.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.884 7eff8135c700 1 -- 172.21.2.101:0/2330259907 learned_addr learned my addr 172.21.2.101:0/2330259907 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:50.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.884 7eff8135c700 1 -- 172.21.2.101:0/2330259907 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7eff7c1b8ed0 msgr2=0x7eff7c0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:50.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.884 7eff8135c700 1 --2- 172.21.2.101:0/2330259907 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7eff7c1b8ed0 0x7eff7c0b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:50.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.884 7eff8135c700 1 -- 172.21.2.101:0/2330259907 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7eff7c0b2110 msgr2=0x7eff7c1b7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:50.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.884 7eff8135c700 1 --2- 172.21.2.101:0/2330259907 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7eff7c0b2110 0x7eff7c1b7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:50.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.884 7eff8135c700 1 -- 172.21.2.101:0/2330259907 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7c0b2510 con 0x7eff7c1b8ad0 2021-02-10T11:35:50.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.884 7eff8135c700 1 --2- 172.21.2.101:0/2330259907 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff7c1b8ad0 0x7eff7c187e50 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 rx=0x7eff7800b560 tx=0x7eff7800b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:50.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.885 7eff71ffb700 1 -- 172.21.2.101:0/2330259907 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7eff78006030 con 0x7eff7c1b8ad0 2021-02-10T11:35:50.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.885 7eff71ffb700 1 -- 172.21.2.101:0/2330259907 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff780043a0 con 0x7eff7c1b8ad0 2021-02-10T11:35:50.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.885 7eff71ffb700 1 -- 172.21.2.101:0/2330259907 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7eff780048c0 con 0x7eff7c1b8ad0 2021-02-10T11:35:50.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.885 7eff83540700 1 -- 172.21.2.101:0/2330259907 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7c0b2510 con 0x7eff7c1b8ad0 2021-02-10T11:35:50.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.885 7eff83540700 1 -- 172.21.2.101:0/2330259907 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7c0b2510 con 0x7eff7c1b8ad0 2021-02-10T11:35:50.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.885 7eff71ffb700 1 -- 172.21.2.101:0/2330259907 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7eff78004580 con 0x7eff7c1b8ad0 2021-02-10T11:35:50.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.885 7eff71ffb700 1 --2- 172.21.2.101:0/2330259907 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7eff5c0186c0 0x7eff5c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:50.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.885 7eff71ffb700 1 -- 172.21.2.101:0/2330259907 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7eff78032790 con 0x7eff7c1b8ad0 2021-02-10T11:35:50.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.886 7eff80b5b700 1 --2- 172.21.2.101:0/2330259907 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7eff5c0186c0 0x7eff5c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:50.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.886 7eff80b5b700 1 --2- 172.21.2.101:0/2330259907 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7eff5c0186c0 0x7eff5c01abc0 crc :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:50.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.916 7eff83540700 1 -- 172.21.2.101:0/2330259907 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff7c0ebc00 con 0x7eff7c1b8ad0 2021-02-10T11:35:50.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:50.919 7eff71ffb700 1 -- 172.21.2.101:0/2330259907 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7eff78006b80 con 0x7eff7c1b8ad0 2021-02-10T11:35:51.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.045 7eff83540700 1 -- 172.21.2.101:0/2330259907 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff7c0ebc00 con 0x7eff7c1b8ad0 2021-02-10T11:35:51.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.046 7eff71ffb700 1 -- 172.21.2.101:0/2330259907 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+2101 (secure 0 0 0) 0x7eff7800bcf0 con 0x7eff7c1b8ad0 2021-02-10T11:35:51.054 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:51.054 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1297756180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1297756180},{"type":"v1","addr":"172.21.2.101:6835","nonce":1297756180}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:34:41.088085","modified":"2021-02-10 11:34:43.450805","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6400":{"gid":6400,"name":"b","rank":0,"incarnation":44,"state":"up:active","state_seq":4,"addr":"172.21.2.108:6833/3585539482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3585539482},{"type":"v1","addr":"172.21.2.108:6833","nonce":3585539482}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-02-10T11:35:51.054 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 47 2021-02-10T11:35:51.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.047 7eff83540700 1 -- 172.21.2.101:0/2330259907 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7eff5c0186c0 msgr2=0x7eff5c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:51.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.047 7eff83540700 1 --2- 172.21.2.101:0/2330259907 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7eff5c0186c0 0x7eff5c01abc0 crc :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:51.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.047 7eff83540700 1 -- 172.21.2.101:0/2330259907 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff7c1b8ad0 msgr2=0x7eff7c187e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:51.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.047 7eff83540700 1 --2- 172.21.2.101:0/2330259907 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff7c1b8ad0 0x7eff7c187e50 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 rx=0x7eff7800b560 tx=0x7eff7800b900).stop 2021-02-10T11:35:51.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.048 7eff83540700 1 -- 172.21.2.101:0/2330259907 shutdown_connections 2021-02-10T11:35:51.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.048 7eff83540700 1 --2- 172.21.2.101:0/2330259907 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7eff5c0186c0 0x7eff5c01abc0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:51.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.048 7eff83540700 1 --2- 172.21.2.101:0/2330259907 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7eff7c1b8ed0 0x7eff7c0b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:51.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.048 7eff83540700 1 --2- 172.21.2.101:0/2330259907 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7eff7c0b2110 0x7eff7c1b7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:51.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.048 7eff83540700 1 --2- 172.21.2.101:0/2330259907 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff7c1b8ad0 0x7eff7c187e50 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:51.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.048 7eff83540700 1 -- 172.21.2.101:0/2330259907 >> 172.21.2.101:0/2330259907 conn(0x7eff7c0ed810 msgr2=0x7eff7c094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:51.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.048 7eff83540700 1 -- 172.21.2.101:0/2330259907 shutdown_connections 2021-02-10T11:35:51.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.048 7eff83540700 1 -- 172.21.2.101:0/2330259907 wait complete. 2021-02-10T11:35:51.064 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs fail cephfs 2021-02-10T11:35:51.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.468 7f8844c97700 1 Processor -- start 2021-02-10T11:35:51.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.468 7f8844c97700 1 -- start start 2021-02-10T11:35:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.469 7f8844c97700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88400d1870 0x7f88401b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.469 7f8844c97700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f88400f0650 0x7f88400cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.469 7f8844c97700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88400f0b10 con 0x7f88401b0a10 2021-02-10T11:35:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.469 7f8844c97700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88400fe230 con 0x7f88400f0650 2021-02-10T11:35:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.469 7f8844c97700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88400e5ed0 con 0x7f88400d1870 2021-02-10T11:35:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.469 7f883e05d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f88401b0a10 0x7f88400cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53872/0 (socket says 172.21.2.101:53872) 2021-02-10T11:35:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.469 7f883e05d700 1 -- 172.21.2.101:0/708756856 learned_addr learned my addr 172.21.2.101:0/708756856 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.470 7f883e85e700 1 --2- 172.21.2.101:0/708756856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88400d1870 0x7f88401b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.470 7f883f05f700 1 --2- 172.21.2.101:0/708756856 >> v2:172.21.2.108:3300/0 conn(0x7f88400f0650 0x7f88400cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.470 7f883d85c700 1 -- 172.21.2.101:0/708756856 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 545106207 0 0) 0x7f8828001180 con 0x7f88401b0a10 2021-02-10T11:35:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.470 7f883d85c700 1 -- 172.21.2.101:0/708756856 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f882c002fe0 con 0x7f88401b0a10 2021-02-10T11:35:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.470 7f883d85c700 1 -- 172.21.2.101:0/708756856 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2212296779 0 0) 0x7f8828001180 con 0x7f88401b0a10 2021-02-10T11:35:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.470 7f883d85c700 1 -- 172.21.2.101:0/708756856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88400d1870 msgr2=0x7f88401b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:51.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.470 7f883d85c700 1 --2- 172.21.2.101:0/708756856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88400d1870 0x7f88401b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:51.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.470 7f883d85c700 1 -- 172.21.2.101:0/708756856 >> v2:172.21.2.108:3300/0 conn(0x7f88400f0650 msgr2=0x7f88400cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:51.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.470 7f883d85c700 1 --2- 172.21.2.101:0/708756856 >> v2:172.21.2.108:3300/0 conn(0x7f88400f0650 0x7f88400cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:51.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.470 7f883d85c700 1 -- 172.21.2.101:0/708756856 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88400f20a0 con 0x7f88401b0a10 2021-02-10T11:35:51.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.470 7f883d85c700 1 -- 172.21.2.101:0/708756856 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f8828001680 con 0x7f88401b0a10 2021-02-10T11:35:51.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.470 7f883d85c700 1 -- 172.21.2.101:0/708756856 >> v1:172.21.2.101:6789/0 conn(0x7f88401b0a10 legacy=0x7f88400cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:51.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.470 7f883d85c700 1 --2- 172.21.2.101:0/708756856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f882c003b10 0x7f882c005fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:51.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.470 7f883d85c700 1 --2- 172.21.2.101:0/708756856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f882c04a7d0 0x7f882c04cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:51.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.470 7f883d85c700 1 --2- 172.21.2.101:0/708756856 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f882c04d520 0x7f882c04fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:51.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.470 7f883d85c700 1 -- 172.21.2.101:0/708756856 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c0502f0 con 0x7f882c003b10 2021-02-10T11:35:51.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.470 7f883e85e700 1 --2- 172.21.2.101:0/708756856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f882c003b10 0x7f882c005fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:51.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.470 7f883d85c700 1 -- 172.21.2.101:0/708756856 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c050510 con 0x7f882c04d520 2021-02-10T11:35:51.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.470 7f883d85c700 1 -- 172.21.2.101:0/708756856 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882c050730 con 0x7f882c04a7d0 2021-02-10T11:35:51.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.470 7f883e05d700 1 --2- 172.21.2.101:0/708756856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f882c04a7d0 0x7f882c04cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:51.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.471 7f883f05f700 1 --2- 172.21.2.101:0/708756856 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f882c04d520 0x7f882c04fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:51.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.471 7f883e85e700 1 -- 172.21.2.101:0/708756856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f882c04a7d0 msgr2=0x7f882c04cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:51.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.471 7f883e85e700 1 --2- 172.21.2.101:0/708756856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f882c04a7d0 0x7f882c04cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:51.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.471 7f883e85e700 1 -- 172.21.2.101:0/708756856 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f882c04d520 msgr2=0x7f882c04fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:51.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.471 7f883e85e700 1 --2- 172.21.2.101:0/708756856 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f882c04d520 0x7f882c04fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:51.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.471 7f883e85e700 1 -- 172.21.2.101:0/708756856 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882c0509d0 con 0x7f882c003b10 2021-02-10T11:35:51.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.471 7f883f05f700 1 --2- 172.21.2.101:0/708756856 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f882c04d520 0x7f882c04fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:35:51.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.471 7f883e85e700 1 --2- 172.21.2.101:0/708756856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f882c003b10 0x7f882c005fc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 rx=0x7f8830006a00 tx=0x7f8830009b00).ready entity=mon.0 client_cookie=850b7df256c791f7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:51.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.471 7f883d85c700 1 -- 172.21.2.101:0/708756856 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f883000c0c0 con 0x7f882c003b10 2021-02-10T11:35:51.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.471 7f883d85c700 1 -- 172.21.2.101:0/708756856 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883000c400 con 0x7f882c003b10 2021-02-10T11:35:51.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.471 7f883d85c700 1 -- 172.21.2.101:0/708756856 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f883000c920 con 0x7f882c003b10 2021-02-10T11:35:51.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.471 7f8844c97700 1 -- 172.21.2.101:0/708756856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f882c003b10 msgr2=0x7f882c005fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:51.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.471 7f8844c97700 1 --2- 172.21.2.101:0/708756856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f882c003b10 0x7f882c005fc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 rx=0x7f8830006a00 tx=0x7f8830009b00).stop 2021-02-10T11:35:51.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.471 7f883f05f700 1 -- 172.21.2.101:0/708756856 reap_dead start 2021-02-10T11:35:51.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.472 7f8844c97700 1 -- 172.21.2.101:0/708756856 shutdown_connections 2021-02-10T11:35:51.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.472 7f8844c97700 1 -- 172.21.2.101:0/708756856 >> 172.21.2.101:0/708756856 conn(0x7f88400ce450 msgr2=0x7f88400d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:51.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.472 7f8844c97700 1 -- 172.21.2.101:0/708756856 shutdown_connections 2021-02-10T11:35:51.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.472 7f8844c97700 1 -- 172.21.2.101:0/708756856 wait complete. 2021-02-10T11:35:51.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.487 7f8844c97700 1 Processor -- start 2021-02-10T11:35:51.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.487 7f8844c97700 1 -- start start 2021-02-10T11:35:51.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.487 7f8844c97700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88400a3060 0x7f88400af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:51.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.487 7f8844c97700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88400ece50 0x7f88400ed250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:51.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.487 7f8844c97700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f884009c920 0x7f884009cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:51.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.487 7f8844c97700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88400af740 con 0x7f88400ece50 2021-02-10T11:35:51.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.487 7f8844c97700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884009c070 con 0x7f884009c920 2021-02-10T11:35:51.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.487 7f8844c97700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88400ef6f0 con 0x7f88400a3060 2021-02-10T11:35:51.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.487 7f883e05d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88400ece50 0x7f88400ed250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:51.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.487 7f883e05d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88400ece50 0x7f88400ed250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53390/0 (socket says 172.21.2.101:53390) 2021-02-10T11:35:51.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.487 7f883e05d700 1 -- 172.21.2.101:0/4251514152 learned_addr learned my addr 172.21.2.101:0/4251514152 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:51.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.487 7f883e85e700 1 --2- 172.21.2.101:0/4251514152 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88400a3060 0x7f88400af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:51.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.487 7f883f05f700 1 --2- 172.21.2.101:0/4251514152 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f884009c920 0x7f884009cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:51.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.488 7f883e85e700 1 -- 172.21.2.101:0/4251514152 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f884009c920 msgr2=0x7f884009cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:51.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.488 7f883e85e700 1 --2- 172.21.2.101:0/4251514152 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f884009c920 0x7f884009cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:51.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.488 7f883e85e700 1 -- 172.21.2.101:0/4251514152 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88400ece50 msgr2=0x7f88400ed250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:51.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.488 7f883e85e700 1 --2- 172.21.2.101:0/4251514152 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88400ece50 0x7f88400ed250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:51.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.488 7f883e85e700 1 -- 172.21.2.101:0/4251514152 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88400ed870 con 0x7f88400a3060 2021-02-10T11:35:51.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.488 7f883e05d700 1 --2- 172.21.2.101:0/4251514152 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88400ece50 0x7f88400ed250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:35:51.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.488 7f883e85e700 1 --2- 172.21.2.101:0/4251514152 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88400a3060 0x7f88400af120 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f8830009610 tx=0x7f883000b8b0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:51.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.488 7f88377fe700 1 -- 172.21.2.101:0/4251514152 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8830006740 con 0x7f88400a3060 2021-02-10T11:35:51.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.488 7f88377fe700 1 -- 172.21.2.101:0/4251514152 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8830000940 con 0x7f88400a3060 2021-02-10T11:35:51.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.488 7f8844c97700 1 -- 172.21.2.101:0/4251514152 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88400ed870 con 0x7f88400a3060 2021-02-10T11:35:51.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.488 7f8844c97700 1 -- 172.21.2.101:0/4251514152 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88400ed870 con 0x7f88400a3060 2021-02-10T11:35:51.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.488 7f88377fe700 1 -- 172.21.2.101:0/4251514152 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f88300052e0 con 0x7f88400a3060 2021-02-10T11:35:51.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.489 7f88377fe700 1 -- 172.21.2.101:0/4251514152 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f88300052e0 con 0x7f88400a3060 2021-02-10T11:35:51.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.489 7f88377fe700 1 --2- 172.21.2.101:0/4251514152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8824018730 0x7f882401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:51.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.489 7f88377fe700 1 -- 172.21.2.101:0/4251514152 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8830032740 con 0x7f88400a3060 2021-02-10T11:35:51.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.489 7f883e05d700 1 --2- 172.21.2.101:0/4251514152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8824018730 0x7f882401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:51.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.490 7f883e05d700 1 --2- 172.21.2.101:0/4251514152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8824018730 0x7f882401ac10 crc :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:51.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.519 7f8844c97700 1 -- 172.21.2.101:0/4251514152 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88400af740 con 0x7f88400a3060 2021-02-10T11:35:51.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.522 7f88377fe700 1 -- 172.21.2.101:0/4251514152 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8830006030 con 0x7f88400a3060 2021-02-10T11:35:51.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:51.651 7f8844c97700 1 -- 172.21.2.101:0/4251514152 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f88400af740 con 0x7f88400a3060 2021-02-10T11:35:52.125 INFO:tasks.ceph.mon.a.gibba001.stderr:2021-02-10 11:35:52.118 7f974d04d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-02-10T11:35:52.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.121 7f88377fe700 1 -- 172.21.2.101:0/4251514152 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) v1 ==== 161+0+0 (secure 0 0 0) 0x7f88300052e0 con 0x7f88400a3060 2021-02-10T11:35:52.176 INFO:teuthology.orchestra.run.gibba001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-02-10T11:35:52.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.170 7f8844c97700 1 -- 172.21.2.101:0/4251514152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8824018730 msgr2=0x7f882401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:52.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.170 7f8844c97700 1 --2- 172.21.2.101:0/4251514152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8824018730 0x7f882401ac10 crc :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:52.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.170 7f8844c97700 1 -- 172.21.2.101:0/4251514152 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88400a3060 msgr2=0x7f88400af120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:52.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.170 7f8844c97700 1 --2- 172.21.2.101:0/4251514152 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88400a3060 0x7f88400af120 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f8830009610 tx=0x7f883000b8b0).stop 2021-02-10T11:35:52.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.170 7f8844c97700 1 -- 172.21.2.101:0/4251514152 shutdown_connections 2021-02-10T11:35:52.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.170 7f8844c97700 1 --2- 172.21.2.101:0/4251514152 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f884009c920 0x7f884009cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:52.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.170 7f8844c97700 1 --2- 172.21.2.101:0/4251514152 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88400ece50 0x7f88400ed250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:52.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.170 7f8844c97700 1 --2- 172.21.2.101:0/4251514152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8824018730 0x7f882401ac10 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:52.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.170 7f8844c97700 1 --2- 172.21.2.101:0/4251514152 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88400a3060 0x7f88400af120 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:52.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.170 7f8844c97700 1 -- 172.21.2.101:0/4251514152 >> 172.21.2.101:0/4251514152 conn(0x7f88400ae980 msgr2=0x7f88400baef0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:52.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.170 7f8844c97700 1 -- 172.21.2.101:0/4251514152 shutdown_connections 2021-02-10T11:35:52.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.170 7f8844c97700 1 -- 172.21.2.101:0/4251514152 wait complete. 2021-02-10T11:35:52.179 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:35:52.187 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:35:52.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.591 7f5bf455b700 1 Processor -- start 2021-02-10T11:35:52.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.591 7f5bf455b700 1 -- start start 2021-02-10T11:35:52.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.592 7f5bf455b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f5bec0d1870 0x7f5bec1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:52.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.592 7f5bf455b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bec0f0650 0x7f5bec0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:52.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.592 7f5bf455b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bec0f0b10 con 0x7f5bec1b0a10 2021-02-10T11:35:52.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.592 7f5bf455b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bec0e5ed0 con 0x7f5bec0d1870 2021-02-10T11:35:52.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.592 7f5bf455b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec0d1c70 con 0x7f5bec0f0650 2021-02-10T11:35:52.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.592 7f5bf1b76700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f5bec1b0a10 0x7f5bec0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53890/0 (socket says 172.21.2.101:53890) 2021-02-10T11:35:52.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.592 7f5bf1b76700 1 -- 172.21.2.101:0/1467278712 learned_addr learned my addr 172.21.2.101:0/1467278712 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:52.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.592 7f5bf2377700 1 --2- 172.21.2.101:0/1467278712 >> v2:172.21.2.108:3300/0 conn(0x7f5bec0d1870 0x7f5bec1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:52.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.592 7f5bf2b78700 1 --2- 172.21.2.101:0/1467278712 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bec0f0650 0x7f5bec0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:52.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.592 7f5bf1375700 1 -- 172.21.2.101:0/1467278712 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2004009976 0 0) 0x7f5bdc0010b0 con 0x7f5bec1b0a10 2021-02-10T11:35:52.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.592 7f5bf1375700 1 -- 172.21.2.101:0/1467278712 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bd8002fe0 con 0x7f5bec1b0a10 2021-02-10T11:35:52.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.593 7f5bf1375700 1 -- 172.21.2.101:0/1467278712 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 876856279 0 0) 0x7f5bdc0010b0 con 0x7f5bec1b0a10 2021-02-10T11:35:52.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.593 7f5bf1375700 1 -- 172.21.2.101:0/1467278712 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bec0f0650 msgr2=0x7f5bec0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:52.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.593 7f5bf1375700 1 --2- 172.21.2.101:0/1467278712 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bec0f0650 0x7f5bec0cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:52.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.593 7f5bf1375700 1 -- 172.21.2.101:0/1467278712 >> v2:172.21.2.108:3300/0 conn(0x7f5bec0d1870 msgr2=0x7f5bec1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:52.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.593 7f5bf1375700 1 --2- 172.21.2.101:0/1467278712 >> v2:172.21.2.108:3300/0 conn(0x7f5bec0d1870 0x7f5bec1b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:52.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.593 7f5bf1375700 1 -- 172.21.2.101:0/1467278712 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec0d1e90 con 0x7f5bec1b0a10 2021-02-10T11:35:52.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.593 7f5bf1375700 1 -- 172.21.2.101:0/1467278712 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f5bdc0015b0 con 0x7f5bec1b0a10 2021-02-10T11:35:52.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.593 7f5bf1375700 1 -- 172.21.2.101:0/1467278712 >> v1:172.21.2.101:6789/0 conn(0x7f5bec1b0a10 legacy=0x7f5bec0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:52.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.593 7f5bf1375700 1 --2- 172.21.2.101:0/1467278712 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bd8003ac0 0x7f5bd8005ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:52.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.593 7f5bf1375700 1 --2- 172.21.2.101:0/1467278712 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5bd804a780 0x7f5bd804cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:52.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.593 7f5bf2377700 1 --2- 172.21.2.101:0/1467278712 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bd8003ac0 0x7f5bd8005ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:52.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.593 7f5bf1375700 1 --2- 172.21.2.101:0/1467278712 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bd804d5a0 0x7f5bd804fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:52.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.593 7f5bf1375700 1 -- 172.21.2.101:0/1467278712 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd80502e0 con 0x7f5bd8003ac0 2021-02-10T11:35:52.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.593 7f5bf1375700 1 -- 172.21.2.101:0/1467278712 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd8050500 con 0x7f5bd804a780 2021-02-10T11:35:52.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.593 7f5bf1375700 1 -- 172.21.2.101:0/1467278712 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd8050720 con 0x7f5bd804d5a0 2021-02-10T11:35:52.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.593 7f5bf1b76700 1 --2- 172.21.2.101:0/1467278712 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5bd804a780 0x7f5bd804cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:52.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.593 7f5bf2377700 1 -- 172.21.2.101:0/1467278712 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bd804d5a0 msgr2=0x7f5bd804fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:52.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.593 7f5bf2377700 1 --2- 172.21.2.101:0/1467278712 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bd804d5a0 0x7f5bd804fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:52.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.593 7f5bf2377700 1 -- 172.21.2.101:0/1467278712 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5bd804a780 msgr2=0x7f5bd804cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:52.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.593 7f5bf2377700 1 --2- 172.21.2.101:0/1467278712 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5bd804a780 0x7f5bd804cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:52.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.593 7f5bf2377700 1 -- 172.21.2.101:0/1467278712 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd80509c0 con 0x7f5bd8003ac0 2021-02-10T11:35:52.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.594 7f5bf2377700 1 --2- 172.21.2.101:0/1467278712 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bd8003ac0 0x7f5bd8005ed0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 rx=0x7f5be800c8f0 tx=0x7f5be800cc90).ready entity=mon.0 client_cookie=bddfc4afe457bad4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:52.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.594 7f5bf1375700 1 -- 172.21.2.101:0/1467278712 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5be800cdd0 con 0x7f5bd8003ac0 2021-02-10T11:35:52.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.594 7f5bf1375700 1 -- 172.21.2.101:0/1467278712 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be8009310 con 0x7f5bd8003ac0 2021-02-10T11:35:52.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.594 7f5bf1375700 1 -- 172.21.2.101:0/1467278712 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5be8009830 con 0x7f5bd8003ac0 2021-02-10T11:35:52.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.594 7f5bf455b700 1 -- 172.21.2.101:0/1467278712 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bd8003ac0 msgr2=0x7f5bd8005ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:52.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.594 7f5bf455b700 1 --2- 172.21.2.101:0/1467278712 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bd8003ac0 0x7f5bd8005ed0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 rx=0x7f5be800c8f0 tx=0x7f5be800cc90).stop 2021-02-10T11:35:52.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.594 7f5bf2b78700 1 -- 172.21.2.101:0/1467278712 reap_dead start 2021-02-10T11:35:52.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.594 7f5bf455b700 1 -- 172.21.2.101:0/1467278712 shutdown_connections 2021-02-10T11:35:52.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.594 7f5bf455b700 1 -- 172.21.2.101:0/1467278712 >> 172.21.2.101:0/1467278712 conn(0x7f5bec0ce450 msgr2=0x7f5bec0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:52.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.594 7f5bf455b700 1 -- 172.21.2.101:0/1467278712 shutdown_connections 2021-02-10T11:35:52.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.595 7f5bf455b700 1 -- 172.21.2.101:0/1467278712 wait complete. 2021-02-10T11:35:52.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.609 7f5bf455b700 1 Processor -- start 2021-02-10T11:35:52.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.609 7f5bf455b700 1 -- start start 2021-02-10T11:35:52.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.609 7f5bf455b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bec195650 0x7f5bec0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:52.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.609 7f5bf455b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5bec0b5b80 0x7f5bec1a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:52.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.609 7f5bf455b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bec0acfa0 0x7f5bec1a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:52.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.609 7f5bf455b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec0b6010 con 0x7f5bec0acfa0 2021-02-10T11:35:52.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.609 7f5bf455b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec1a3a60 con 0x7f5bec0b5b80 2021-02-10T11:35:52.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.609 7f5bf455b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec1a3c40 con 0x7f5bec195650 2021-02-10T11:35:52.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.609 7f5bf2b78700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bec0acfa0 0x7f5bec1a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:52.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.610 7f5bf2b78700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bec0acfa0 0x7f5bec1a3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53404/0 (socket says 172.21.2.101:53404) 2021-02-10T11:35:52.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.610 7f5bf2b78700 1 -- 172.21.2.101:0/2523439412 learned_addr learned my addr 172.21.2.101:0/2523439412 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:52.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.610 7f5bf2b78700 1 -- 172.21.2.101:0/2523439412 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bec195650 msgr2=0x7f5bec0b5560 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:35:52.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.610 7f5bf2b78700 1 --2- 172.21.2.101:0/2523439412 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bec195650 0x7f5bec0b5560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:52.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.610 7f5bf2b78700 1 -- 172.21.2.101:0/2523439412 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5bec0b5b80 msgr2=0x7f5bec1a2e20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:35:52.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.610 7f5bf2b78700 1 --2- 172.21.2.101:0/2523439412 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5bec0b5b80 0x7f5bec1a2e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:52.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.610 7f5bf2b78700 1 -- 172.21.2.101:0/2523439412 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec0bcae0 con 0x7f5bec0acfa0 2021-02-10T11:35:52.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.610 7f5bf2b78700 1 --2- 172.21.2.101:0/2523439412 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bec0acfa0 0x7f5bec1a3440 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 rx=0x7f5be4009960 tx=0x7f5be400bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:52.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.610 7f5be2ffd700 1 -- 172.21.2.101:0/2523439412 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5be40011f0 con 0x7f5bec0acfa0 2021-02-10T11:35:52.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.610 7f5bf455b700 1 -- 172.21.2.101:0/2523439412 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bec0bcae0 con 0x7f5bec0acfa0 2021-02-10T11:35:52.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.610 7f5be2ffd700 1 -- 172.21.2.101:0/2523439412 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be4003030 con 0x7f5bec0acfa0 2021-02-10T11:35:52.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.610 7f5be2ffd700 1 -- 172.21.2.101:0/2523439412 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5be40066f0 con 0x7f5bec0acfa0 2021-02-10T11:35:52.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.610 7f5bf455b700 1 -- 172.21.2.101:0/2523439412 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bec0bcae0 con 0x7f5bec0acfa0 2021-02-10T11:35:52.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.610 7f5be2ffd700 1 -- 172.21.2.101:0/2523439412 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5be4000e40 con 0x7f5bec0acfa0 2021-02-10T11:35:52.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.611 7f5be2ffd700 1 --2- 172.21.2.101:0/2523439412 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5bd4018730 0x7f5bd401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:52.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.611 7f5be2ffd700 1 -- 172.21.2.101:0/2523439412 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f5be4032a40 con 0x7f5bec0acfa0 2021-02-10T11:35:52.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.611 7f5bf2377700 1 --2- 172.21.2.101:0/2523439412 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5bd4018730 0x7f5bd401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:52.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.611 7f5bf2377700 1 --2- 172.21.2.101:0/2523439412 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5bd4018730 0x7f5bd401ac10 crc :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:52.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.641 7f5bf455b700 1 -- 172.21.2.101:0/2523439412 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bec0b6010 con 0x7f5bec0acfa0 2021-02-10T11:35:52.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.644 7f5be2ffd700 1 -- 172.21.2.101:0/2523439412 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5be40066f0 con 0x7f5bec0acfa0 2021-02-10T11:35:52.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.773 7f5bf455b700 1 -- 172.21.2.101:0/2523439412 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5bec0b6010 con 0x7f5bec0acfa0 2021-02-10T11:35:52.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.773 7f5be2ffd700 1 -- 172.21.2.101:0/2523439412 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+1756 (secure 0 0 0) 0x7f5be4000e40 con 0x7f5bec0acfa0 2021-02-10T11:35:52.781 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:52.781 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1297756180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1297756180},{"type":"v1","addr":"172.21.2.101:6835","nonce":1297756180}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:34:41.088085","modified":"2021-02-10 11:35:52.118535","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-02-10T11:35:52.782 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 48 2021-02-10T11:35:52.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.775 7f5bf455b700 1 -- 172.21.2.101:0/2523439412 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5bd4018730 msgr2=0x7f5bd401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:52.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.775 7f5bf455b700 1 --2- 172.21.2.101:0/2523439412 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5bd4018730 0x7f5bd401ac10 crc :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:52.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.775 7f5bf455b700 1 -- 172.21.2.101:0/2523439412 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bec0acfa0 msgr2=0x7f5bec1a3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:52.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.775 7f5bf455b700 1 --2- 172.21.2.101:0/2523439412 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bec0acfa0 0x7f5bec1a3440 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 rx=0x7f5be4009960 tx=0x7f5be400bd40).stop 2021-02-10T11:35:52.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.775 7f5bf455b700 1 -- 172.21.2.101:0/2523439412 shutdown_connections 2021-02-10T11:35:52.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.775 7f5bf455b700 1 --2- 172.21.2.101:0/2523439412 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bec0acfa0 0x7f5bec1a3440 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:52.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.775 7f5bf455b700 1 --2- 172.21.2.101:0/2523439412 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5bec0b5b80 0x7f5bec1a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:52.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.775 7f5bf455b700 1 --2- 172.21.2.101:0/2523439412 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5bd4018730 0x7f5bd401ac10 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:52.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.775 7f5bf455b700 1 --2- 172.21.2.101:0/2523439412 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bec195650 0x7f5bec0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:52.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.775 7f5bf455b700 1 -- 172.21.2.101:0/2523439412 >> 172.21.2.101:0/2523439412 conn(0x7f5bec0b3040 msgr2=0x7f5bec0c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:52.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.776 7f5bf455b700 1 -- 172.21.2.101:0/2523439412 shutdown_connections 2021-02-10T11:35:52.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:52.776 7f5bf455b700 1 -- 172.21.2.101:0/2523439412 wait complete. 2021-02-10T11:35:52.792 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-02-10T11:35:53.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.201 7f2830500700 1 Processor -- start 2021-02-10T11:35:53.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.201 7f2830500700 1 -- start start 2021-02-10T11:35:53.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.201 7f2830500700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f28280d0340 0x7f28280ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:53.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.201 7f2830500700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28280d1c70 0x7f28280efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:53.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.201 7f2830500700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28280f0b10 con 0x7f28280d1870 2021-02-10T11:35:53.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.201 7f2830500700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28280e5ed0 con 0x7f28280d0340 2021-02-10T11:35:53.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.201 7f2830500700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28281b0be0 con 0x7f28280d1c70 2021-02-10T11:35:53.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.201 7f282db1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f28280d0340 0x7f28280ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:53.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.201 7f282db1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f28280d0340 0x7f28280ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53412/0 (socket says 172.21.2.101:53412) 2021-02-10T11:35:53.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.201 7f282db1b700 1 -- 172.21.2.101:0/3888065530 learned_addr learned my addr 172.21.2.101:0/3888065530 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:53.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.202 7f282db1b700 1 -- 172.21.2.101:0/3888065530 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28280d1c70 msgr2=0x7f28280efc60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:35:53.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.202 7f282db1b700 1 --2- 172.21.2.101:0/3888065530 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28280d1c70 0x7f28280efc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:53.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.202 7f282db1b700 1 -- 172.21.2.101:0/3888065530 >> v1:172.21.2.101:6789/0 conn(0x7f28280d1870 legacy=0x7f28281b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:53.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.202 7f282db1b700 1 -- 172.21.2.101:0/3888065530 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28280f20a0 con 0x7f28280d0340 2021-02-10T11:35:53.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.202 7f282db1b700 1 --2- 172.21.2.101:0/3888065530 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28280d0340 0x7f28280ef640 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 rx=0x7f281c006700 tx=0x7f281c009b80).ready entity=mon.0 client_cookie=4cac1009b5f2d47d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:53.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.202 7f282d31a700 1 -- 172.21.2.101:0/3888065530 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f281c00c0c0 con 0x7f28280d0340 2021-02-10T11:35:53.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.202 7f282d31a700 1 -- 172.21.2.101:0/3888065530 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281c00c340 con 0x7f28280d0340 2021-02-10T11:35:53.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.202 7f282d31a700 1 -- 172.21.2.101:0/3888065530 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f281c00c860 con 0x7f28280d0340 2021-02-10T11:35:53.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.202 7f2830500700 1 -- 172.21.2.101:0/3888065530 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28280d0340 msgr2=0x7f28280ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:53.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.202 7f2830500700 1 --2- 172.21.2.101:0/3888065530 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28280d0340 0x7f28280ef640 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 rx=0x7f281c006700 tx=0x7f281c009b80).stop 2021-02-10T11:35:53.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.203 7f2830500700 1 -- 172.21.2.101:0/3888065530 shutdown_connections 2021-02-10T11:35:53.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.203 7f2830500700 1 --2- 172.21.2.101:0/3888065530 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28280d1c70 0x7f28280efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:53.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.203 7f2830500700 1 --2- 172.21.2.101:0/3888065530 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28280d0340 0x7f28280ef640 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:53.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.203 7f2830500700 1 -- 172.21.2.101:0/3888065530 >> 172.21.2.101:0/3888065530 conn(0x7f28280ce450 msgr2=0x7f28280d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:53.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.203 7f2830500700 1 -- 172.21.2.101:0/3888065530 shutdown_connections 2021-02-10T11:35:53.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.203 7f2830500700 1 -- 172.21.2.101:0/3888065530 wait complete. 2021-02-10T11:35:53.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.218 7f2830500700 1 Processor -- start 2021-02-10T11:35:53.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.218 7f2830500700 1 -- start start 2021-02-10T11:35:53.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f2830500700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f282818e0c0 0x7f28280f2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:53.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f2830500700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28281a2e20 0x7f28281a3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:53.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f2830500700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28281a3840 0x7f282818ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:53.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f2830500700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282818e7c0 con 0x7f28281a2e20 2021-02-10T11:35:53.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f2830500700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28280f2ca0 con 0x7f282818e0c0 2021-02-10T11:35:53.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f2830500700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282818e4c0 con 0x7f28281a3840 2021-02-10T11:35:53.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f282db1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28281a2e20 0x7f28281a3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:53.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f282db1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28281a2e20 0x7f28281a3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53420/0 (socket says 172.21.2.101:53420) 2021-02-10T11:35:53.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f282db1b700 1 -- 172.21.2.101:0/3100512766 learned_addr learned my addr 172.21.2.101:0/3100512766 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:53.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f282eb1d700 1 --2- 172.21.2.101:0/3100512766 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28281a3840 0x7f282818ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:53.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f282e31c700 1 --2- 172.21.2.101:0/3100512766 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f282818e0c0 0x7f28280f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:53.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f282db1b700 1 -- 172.21.2.101:0/3100512766 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28281a3840 msgr2=0x7f282818ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:53.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f282db1b700 1 --2- 172.21.2.101:0/3100512766 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28281a3840 0x7f282818ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:53.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f282db1b700 1 -- 172.21.2.101:0/3100512766 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f282818e0c0 msgr2=0x7f28280f2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:53.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f282db1b700 1 --2- 172.21.2.101:0/3100512766 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f282818e0c0 0x7f28280f2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:53.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f282db1b700 1 -- 172.21.2.101:0/3100512766 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28281a3c40 con 0x7f28281a2e20 2021-02-10T11:35:53.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f282e31c700 1 --2- 172.21.2.101:0/3100512766 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f282818e0c0 0x7f28280f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:35:53.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f282db1b700 1 --2- 172.21.2.101:0/3100512766 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28281a2e20 0x7f28281a3220 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 rx=0x7f281c00b5e0 tx=0x7f281c00b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:53.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f281affd700 1 -- 172.21.2.101:0/3100512766 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f281c00bd70 con 0x7f28281a2e20 2021-02-10T11:35:53.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f281affd700 1 -- 172.21.2.101:0/3100512766 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281c002030 con 0x7f28281a2e20 2021-02-10T11:35:53.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f2830500700 1 -- 172.21.2.101:0/3100512766 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28281a3c40 con 0x7f28281a2e20 2021-02-10T11:35:53.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f281affd700 1 -- 172.21.2.101:0/3100512766 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f281c002550 con 0x7f28281a2e20 2021-02-10T11:35:53.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.219 7f2830500700 1 -- 172.21.2.101:0/3100512766 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282818f360 con 0x7f28281a2e20 2021-02-10T11:35:53.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.220 7f281affd700 1 -- 172.21.2.101:0/3100512766 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f281c002420 con 0x7f28281a2e20 2021-02-10T11:35:53.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.220 7f281affd700 1 --2- 172.21.2.101:0/3100512766 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28100186c0 0x7f281001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:53.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.220 7f281affd700 1 -- 172.21.2.101:0/3100512766 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f281c032430 con 0x7f28281a2e20 2021-02-10T11:35:53.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.220 7f282e31c700 1 --2- 172.21.2.101:0/3100512766 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28100186c0 0x7f281001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:53.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.220 7f282e31c700 1 --2- 172.21.2.101:0/3100512766 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28100186c0 0x7f281001abc0 crc :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:53.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.251 7f2830500700 1 -- 172.21.2.101:0/3100512766 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28280ee060 con 0x7f28281a2e20 2021-02-10T11:35:53.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.253 7f281affd700 1 -- 172.21.2.101:0/3100512766 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f281c025520 con 0x7f28281a2e20 2021-02-10T11:35:53.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:53.381 7f2830500700 1 -- 172.21.2.101:0/3100512766 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f28280ce450 con 0x7f28281a2e20 2021-02-10T11:35:54.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.135 7f281affd700 1 -- 172.21.2.101:0/3100512766 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7f281c025520 con 0x7f28281a2e20 2021-02-10T11:35:54.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.151 7f2830500700 1 -- 172.21.2.101:0/3100512766 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28100186c0 msgr2=0x7f281001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:54.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.152 7f2830500700 1 --2- 172.21.2.101:0/3100512766 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28100186c0 0x7f281001abc0 crc :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:54.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.152 7f2830500700 1 -- 172.21.2.101:0/3100512766 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28281a2e20 msgr2=0x7f28281a3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:54.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.152 7f2830500700 1 --2- 172.21.2.101:0/3100512766 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28281a2e20 0x7f28281a3220 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 rx=0x7f281c00b5e0 tx=0x7f281c00b980).stop 2021-02-10T11:35:54.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.152 7f2830500700 1 -- 172.21.2.101:0/3100512766 shutdown_connections 2021-02-10T11:35:54.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.152 7f2830500700 1 --2- 172.21.2.101:0/3100512766 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28100186c0 0x7f281001abc0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:54.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.152 7f2830500700 1 --2- 172.21.2.101:0/3100512766 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28281a3840 0x7f282818ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:54.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.152 7f2830500700 1 --2- 172.21.2.101:0/3100512766 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28281a2e20 0x7f28281a3220 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:54.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.152 7f2830500700 1 --2- 172.21.2.101:0/3100512766 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f282818e0c0 0x7f28280f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:54.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.152 7f2830500700 1 -- 172.21.2.101:0/3100512766 >> 172.21.2.101:0/3100512766 conn(0x7f28280bc0b0 msgr2=0x7f28280ee290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:54.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.152 7f2830500700 1 -- 172.21.2.101:0/3100512766 shutdown_connections 2021-02-10T11:35:54.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.152 7f2830500700 1 -- 172.21.2.101:0/3100512766 wait complete. 2021-02-10T11:35:54.168 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-02-10T11:35:54.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.573 7f8c5d57f700 1 Processor -- start 2021-02-10T11:35:54.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.573 7f8c5d57f700 1 -- start start 2021-02-10T11:35:54.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.574 7f8c5d57f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8c580d1880 0x7f8c581b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:54.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.574 7f8c5d57f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c580fe1d0 con 0x7f8c580cff80 2021-02-10T11:35:54.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.574 7f8c5d57f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c58109a70 con 0x7f8c581b0af0 2021-02-10T11:35:54.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.574 7f8c5d57f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c58088070 con 0x7f8c580d1880 2021-02-10T11:35:54.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.574 7f8c572bf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8c580d1880 0x7f8c581b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:54.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.574 7f8c572bf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8c580d1880 0x7f8c581b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53426/0 (socket says 172.21.2.101:53426) 2021-02-10T11:35:54.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.574 7f8c572bf700 1 -- 172.21.2.101:0/2982423691 learned_addr learned my addr 172.21.2.101:0/2982423691 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:54.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.574 7f8c57ac0700 1 --1- 172.21.2.101:0/2982423691 >> v1:172.21.2.101:6789/0 conn(0x7f8c580cff80 0x7f8c58109140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53920/0 (socket says 172.21.2.101:53920) 2021-02-10T11:35:54.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.574 7f8c572bf700 1 -- 172.21.2.101:0/2982423691 >> v1:172.21.2.108:6789/0 conn(0x7f8c581b0af0 legacy=0x7f8c58108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:54.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.574 7f8c572bf700 1 -- 172.21.2.101:0/2982423691 >> v1:172.21.2.101:6789/0 conn(0x7f8c580cff80 legacy=0x7f8c58109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:54.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.574 7f8c572bf700 1 -- 172.21.2.101:0/2982423691 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c58088260 con 0x7f8c580d1880 2021-02-10T11:35:54.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.575 7f8c572bf700 1 --2- 172.21.2.101:0/2982423691 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c580d1880 0x7f8c581b04d0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 rx=0x7f8c48006700 tx=0x7f8c48009b80).ready entity=mon.0 client_cookie=9fa30be1143842fa server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:54.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.575 7f8c562bd700 1 -- 172.21.2.101:0/2982423691 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c4800c0c0 con 0x7f8c580d1880 2021-02-10T11:35:54.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.575 7f8c562bd700 1 -- 172.21.2.101:0/2982423691 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4800c440 con 0x7f8c580d1880 2021-02-10T11:35:54.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.575 7f8c562bd700 1 -- 172.21.2.101:0/2982423691 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c4800c960 con 0x7f8c580d1880 2021-02-10T11:35:54.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.575 7f8c5d57f700 1 -- 172.21.2.101:0/2982423691 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c580d1880 msgr2=0x7f8c581b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:54.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.575 7f8c5d57f700 1 --2- 172.21.2.101:0/2982423691 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c580d1880 0x7f8c581b04d0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 rx=0x7f8c48006700 tx=0x7f8c48009b80).stop 2021-02-10T11:35:54.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.575 7f8c5d57f700 1 -- 172.21.2.101:0/2982423691 shutdown_connections 2021-02-10T11:35:54.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.575 7f8c5d57f700 1 --2- 172.21.2.101:0/2982423691 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c580d1880 0x7f8c581b04d0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:54.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.575 7f8c5d57f700 1 -- 172.21.2.101:0/2982423691 >> 172.21.2.101:0/2982423691 conn(0x7f8c580ce430 msgr2=0x7f8c580d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:54.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.575 7f8c5d57f700 1 -- 172.21.2.101:0/2982423691 shutdown_connections 2021-02-10T11:35:54.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.575 7f8c5d57f700 1 -- 172.21.2.101:0/2982423691 wait complete. 2021-02-10T11:35:54.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.590 7f8c5d57f700 1 Processor -- start 2021-02-10T11:35:54.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.590 7f8c5d57f700 1 -- start start 2021-02-10T11:35:54.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.590 7f8c5d57f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c580af8f0 0x7f8c580aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:54.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.590 7f8c5d57f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c580bfb70 0x7f8c580bf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:54.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.590 7f8c5d57f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c580bf660 0x7f8c580ba810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:54.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.590 7f8c5d57f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c580af370 con 0x7f8c580af8f0 2021-02-10T11:35:54.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.590 7f8c5d57f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c580afcf0 con 0x7f8c580bfb70 2021-02-10T11:35:54.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.590 7f8c5d57f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c580bff70 con 0x7f8c580bf660 2021-02-10T11:35:54.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.591 7f8c572bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c580af8f0 0x7f8c580aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:54.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.591 7f8c572bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c580af8f0 0x7f8c580aecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53430/0 (socket says 172.21.2.101:53430) 2021-02-10T11:35:54.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.591 7f8c572bf700 1 -- 172.21.2.101:0/2410333965 learned_addr learned my addr 172.21.2.101:0/2410333965 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:54.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.591 7f8c572bf700 1 -- 172.21.2.101:0/2410333965 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c580bf660 msgr2=0x7f8c580ba810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:54.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.591 7f8c572bf700 1 --2- 172.21.2.101:0/2410333965 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c580bf660 0x7f8c580ba810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:54.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.591 7f8c572bf700 1 -- 172.21.2.101:0/2410333965 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c580bfb70 msgr2=0x7f8c580bf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:54.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.591 7f8c572bf700 1 --2- 172.21.2.101:0/2410333965 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c580bfb70 0x7f8c580bf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:54.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.591 7f8c572bf700 1 -- 172.21.2.101:0/2410333965 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c580c0190 con 0x7f8c580af8f0 2021-02-10T11:35:54.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.591 7f8c572bf700 1 --2- 172.21.2.101:0/2410333965 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c580af8f0 0x7f8c580aecf0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 rx=0x7f8c4800b5e0 tx=0x7f8c4800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:54.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.591 7f8c3ffff700 1 -- 172.21.2.101:0/2410333965 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c4800bd70 con 0x7f8c580af8f0 2021-02-10T11:35:54.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.591 7f8c3ffff700 1 -- 172.21.2.101:0/2410333965 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c48002030 con 0x7f8c580af8f0 2021-02-10T11:35:54.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.591 7f8c5d57f700 1 -- 172.21.2.101:0/2410333965 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c580c0190 con 0x7f8c580af8f0 2021-02-10T11:35:54.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.591 7f8c5d57f700 1 -- 172.21.2.101:0/2410333965 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c580bb120 con 0x7f8c580af8f0 2021-02-10T11:35:54.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.591 7f8c3ffff700 1 -- 172.21.2.101:0/2410333965 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c48002550 con 0x7f8c580af8f0 2021-02-10T11:35:54.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.592 7f8c3ffff700 1 -- 172.21.2.101:0/2410333965 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8c48000c90 con 0x7f8c580af8f0 2021-02-10T11:35:54.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.592 7f8c3ffff700 1 --2- 172.21.2.101:0/2410333965 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c380186c0 0x7f8c3801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:54.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.592 7f8c3ffff700 1 -- 172.21.2.101:0/2410333965 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f8c48032610 con 0x7f8c580af8f0 2021-02-10T11:35:54.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.592 7f8c56abe700 1 --2- 172.21.2.101:0/2410333965 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c380186c0 0x7f8c3801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:54.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.592 7f8c56abe700 1 --2- 172.21.2.101:0/2410333965 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c380186c0 0x7f8c3801abc0 crc :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:54.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.624 7f8c5d57f700 1 -- 172.21.2.101:0/2410333965 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c580bb400 con 0x7f8c580af8f0 2021-02-10T11:35:54.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.626 7f8c3ffff700 1 -- 172.21.2.101:0/2410333965 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8c4802df50 con 0x7f8c580af8f0 2021-02-10T11:35:54.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:54.752 7f8c5d57f700 1 -- 172.21.2.101:0/2410333965 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8c580bb400 con 0x7f8c580af8f0 2021-02-10T11:35:55.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.141 7f8c3ffff700 1 -- 172.21.2.101:0/2410333965 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v67) v1 ==== 185+0+0 (secure 0 0 0) 0x7f8c4802df50 con 0x7f8c580af8f0 2021-02-10T11:35:55.175 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' removed 2021-02-10T11:35:55.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.169 7f8c5d57f700 1 -- 172.21.2.101:0/2410333965 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c380186c0 msgr2=0x7f8c3801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:55.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.169 7f8c5d57f700 1 --2- 172.21.2.101:0/2410333965 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c380186c0 0x7f8c3801abc0 crc :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:55.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.169 7f8c5d57f700 1 -- 172.21.2.101:0/2410333965 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c580af8f0 msgr2=0x7f8c580aecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:55.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.169 7f8c5d57f700 1 --2- 172.21.2.101:0/2410333965 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c580af8f0 0x7f8c580aecf0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 rx=0x7f8c4800b5e0 tx=0x7f8c4800b980).stop 2021-02-10T11:35:55.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.169 7f8c5d57f700 1 -- 172.21.2.101:0/2410333965 shutdown_connections 2021-02-10T11:35:55.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.169 7f8c5d57f700 1 --2- 172.21.2.101:0/2410333965 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c380186c0 0x7f8c3801abc0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:55.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.169 7f8c5d57f700 1 --2- 172.21.2.101:0/2410333965 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c580bf660 0x7f8c580ba810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:55.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.169 7f8c5d57f700 1 --2- 172.21.2.101:0/2410333965 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c580bfb70 0x7f8c580bf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:55.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.169 7f8c5d57f700 1 --2- 172.21.2.101:0/2410333965 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c580af8f0 0x7f8c580aecf0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:55.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.169 7f8c5d57f700 1 -- 172.21.2.101:0/2410333965 >> 172.21.2.101:0/2410333965 conn(0x7f8c580ba0f0 msgr2=0x7f8c580b9820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:55.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.170 7f8c5d57f700 1 -- 172.21.2.101:0/2410333965 shutdown_connections 2021-02-10T11:35:55.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.170 7f8c5d57f700 1 -- 172.21.2.101:0/2410333965 wait complete. 2021-02-10T11:35:55.186 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-02-10T11:35:55.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.592 7f94f3da2700 1 Processor -- start 2021-02-10T11:35:55.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.592 7f94f3da2700 1 -- start start 2021-02-10T11:35:55.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.592 7f94f3da2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f94ec0d1c70 0x7f94ec0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:55.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.592 7f94f3da2700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f94ec0cf580 0x7f94ec0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:55.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.592 7f94f3da2700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ec0fe1c0 con 0x7f94ec0d1870 2021-02-10T11:35:55.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.592 7f94f3da2700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94ec1b0cb0 con 0x7f94ec0d1c70 2021-02-10T11:35:55.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.592 7f94f3da2700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94ec109a60 con 0x7f94ec0cf580 2021-02-10T11:35:55.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.593 7f94f13bd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f94ec0d1c70 0x7f94ec0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:55.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.593 7f94f13bd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f94ec0d1c70 0x7f94ec0cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53438/0 (socket says 172.21.2.101:53438) 2021-02-10T11:35:55.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.593 7f94f13bd700 1 -- 172.21.2.101:0/1648467441 learned_addr learned my addr 172.21.2.101:0/1648467441 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:55.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.593 7f94f13bd700 1 -- 172.21.2.101:0/1648467441 >> v2:172.21.2.108:3300/0 conn(0x7f94ec0cf580 msgr2=0x7f94ec0cf980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:35:55.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.593 7f94f13bd700 1 --2- 172.21.2.101:0/1648467441 >> v2:172.21.2.108:3300/0 conn(0x7f94ec0cf580 0x7f94ec0cf980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:55.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.593 7f94f13bd700 1 -- 172.21.2.101:0/1648467441 >> v1:172.21.2.108:6789/0 conn(0x7f94ec0d1870 legacy=0x7f94ec1b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:35:55.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.593 7f94f13bd700 1 -- 172.21.2.101:0/1648467441 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec088070 con 0x7f94ec0d1c70 2021-02-10T11:35:55.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.593 7f94f13bd700 1 --2- 172.21.2.101:0/1648467441 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94ec0d1c70 0x7f94ec0cef60 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 rx=0x7f94e0006700 tx=0x7f94e0009b80).ready entity=mon.0 client_cookie=648b9d4381d6af64 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:55.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.593 7f94f0bbc700 1 -- 172.21.2.101:0/1648467441 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f94e000c0c0 con 0x7f94ec0d1c70 2021-02-10T11:35:55.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.593 7f94f0bbc700 1 -- 172.21.2.101:0/1648467441 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e000c340 con 0x7f94ec0d1c70 2021-02-10T11:35:55.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.593 7f94f0bbc700 1 -- 172.21.2.101:0/1648467441 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f94e000c860 con 0x7f94ec0d1c70 2021-02-10T11:35:55.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.593 7f94f3da2700 1 -- 172.21.2.101:0/1648467441 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94ec0d1c70 msgr2=0x7f94ec0cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:55.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.593 7f94f3da2700 1 --2- 172.21.2.101:0/1648467441 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94ec0d1c70 0x7f94ec0cef60 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 rx=0x7f94e0006700 tx=0x7f94e0009b80).stop 2021-02-10T11:35:55.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.594 7f94f3da2700 1 -- 172.21.2.101:0/1648467441 shutdown_connections 2021-02-10T11:35:55.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.594 7f94f3da2700 1 --2- 172.21.2.101:0/1648467441 >> v2:172.21.2.108:3300/0 conn(0x7f94ec0cf580 0x7f94ec0cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:55.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.594 7f94f3da2700 1 --2- 172.21.2.101:0/1648467441 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94ec0d1c70 0x7f94ec0cef60 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:55.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.594 7f94f3da2700 1 -- 172.21.2.101:0/1648467441 >> 172.21.2.101:0/1648467441 conn(0x7f94ec0ce420 msgr2=0x7f94ec0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:55.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.594 7f94f3da2700 1 -- 172.21.2.101:0/1648467441 shutdown_connections 2021-02-10T11:35:55.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.594 7f94f3da2700 1 -- 172.21.2.101:0/1648467441 wait complete. 2021-02-10T11:35:55.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.609 7f94f3da2700 1 Processor -- start 2021-02-10T11:35:55.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.609 7f94f3da2700 1 -- start start 2021-02-10T11:35:55.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.609 7f94f3da2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94ec086440 0x7f94ec0b7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:55.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.609 7f94f3da2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94ec0b7eb0 0x7f94ec1a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:55.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.609 7f94f3da2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94ec0b82b0 0x7f94ec1a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:55.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.609 7f94f3da2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec187e70 con 0x7f94ec086440 2021-02-10T11:35:55.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.610 7f94f3da2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec1880e0 con 0x7f94ec0b7eb0 2021-02-10T11:35:55.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.610 7f94f3da2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec1a0e70 con 0x7f94ec0b82b0 2021-02-10T11:35:55.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.610 7f94f1bbe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94ec086440 0x7f94ec0b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:55.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.610 7f94f1bbe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94ec086440 0x7f94ec0b7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53444/0 (socket says 172.21.2.101:53444) 2021-02-10T11:35:55.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.610 7f94f1bbe700 1 -- 172.21.2.101:0/3426502742 learned_addr learned my addr 172.21.2.101:0/3426502742 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:55.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.610 7f94f1bbe700 1 -- 172.21.2.101:0/3426502742 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94ec0b82b0 msgr2=0x7f94ec1a0850 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:35:55.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.610 7f94f1bbe700 1 --2- 172.21.2.101:0/3426502742 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94ec0b82b0 0x7f94ec1a0850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:55.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.610 7f94f1bbe700 1 -- 172.21.2.101:0/3426502742 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94ec0b7eb0 msgr2=0x7f94ec1a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:55.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.610 7f94f1bbe700 1 --2- 172.21.2.101:0/3426502742 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94ec0b7eb0 0x7f94ec1a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:55.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.610 7f94f1bbe700 1 -- 172.21.2.101:0/3426502742 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec1a1050 con 0x7f94ec086440 2021-02-10T11:35:55.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.610 7f94f1bbe700 1 --2- 172.21.2.101:0/3426502742 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94ec086440 0x7f94ec0b7890 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 rx=0x7f94e800b5b0 tx=0x7f94e800b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:55.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.611 7f94de7fc700 1 -- 172.21.2.101:0/3426502742 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f94e800bd40 con 0x7f94ec086440 2021-02-10T11:35:55.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.611 7f94de7fc700 1 -- 172.21.2.101:0/3426502742 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e80009b0 con 0x7f94ec086440 2021-02-10T11:35:55.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.611 7f94de7fc700 1 -- 172.21.2.101:0/3426502742 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f94e8006210 con 0x7f94ec086440 2021-02-10T11:35:55.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.611 7f94f3da2700 1 -- 172.21.2.101:0/3426502742 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ec1a1050 con 0x7f94ec086440 2021-02-10T11:35:55.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.611 7f94f3da2700 1 -- 172.21.2.101:0/3426502742 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ec1a1050 con 0x7f94ec086440 2021-02-10T11:35:55.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.611 7f94de7fc700 1 -- 172.21.2.101:0/3426502742 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f94e8006020 con 0x7f94ec086440 2021-02-10T11:35:55.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.611 7f94de7fc700 1 --2- 172.21.2.101:0/3426502742 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94d40186c0 0x7f94d401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:55.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.611 7f94de7fc700 1 -- 172.21.2.101:0/3426502742 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f94e80323f0 con 0x7f94ec086440 2021-02-10T11:35:55.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.612 7f94f13bd700 1 --2- 172.21.2.101:0/3426502742 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94d40186c0 0x7f94d401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:55.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.612 7f94f13bd700 1 --2- 172.21.2.101:0/3426502742 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94d40186c0 0x7f94d401abc0 crc :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:55.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.642 7f94f3da2700 1 -- 172.21.2.101:0/3426502742 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ec0825c0 con 0x7f94ec086440 2021-02-10T11:35:55.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.645 7f94de7fc700 1 -- 172.21.2.101:0/3426502742 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f94e8006230 con 0x7f94ec086440 2021-02-10T11:35:55.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:55.773 7f94f3da2700 1 -- 172.21.2.101:0/3426502742 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f94ec0825c0 con 0x7f94ec086440 2021-02-10T11:35:56.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.146 7f94de7fc700 1 -- 172.21.2.101:0/3426502742 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v68) v1 ==== 173+0+0 (secure 0 0 0) 0x7f94e801f4e0 con 0x7f94ec086440 2021-02-10T11:35:56.197 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' removed 2021-02-10T11:35:56.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.191 7f94f3da2700 1 -- 172.21.2.101:0/3426502742 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94d40186c0 msgr2=0x7f94d401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:56.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.191 7f94f3da2700 1 --2- 172.21.2.101:0/3426502742 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94d40186c0 0x7f94d401abc0 crc :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:56.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.191 7f94f3da2700 1 -- 172.21.2.101:0/3426502742 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94ec086440 msgr2=0x7f94ec0b7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:56.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.191 7f94f3da2700 1 --2- 172.21.2.101:0/3426502742 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94ec086440 0x7f94ec0b7890 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 rx=0x7f94e800b5b0 tx=0x7f94e800b950).stop 2021-02-10T11:35:56.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.191 7f94f3da2700 1 -- 172.21.2.101:0/3426502742 shutdown_connections 2021-02-10T11:35:56.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.191 7f94f3da2700 1 --2- 172.21.2.101:0/3426502742 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94d40186c0 0x7f94d401abc0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:56.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.191 7f94f3da2700 1 --2- 172.21.2.101:0/3426502742 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94ec0b82b0 0x7f94ec1a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:56.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.191 7f94f3da2700 1 --2- 172.21.2.101:0/3426502742 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94ec0b7eb0 0x7f94ec1a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:56.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.191 7f94f3da2700 1 --2- 172.21.2.101:0/3426502742 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94ec086440 0x7f94ec0b7890 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:56.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.191 7f94f3da2700 1 -- 172.21.2.101:0/3426502742 >> 172.21.2.101:0/3426502742 conn(0x7f94ec098ab0 msgr2=0x7f94ec1974b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:56.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.192 7f94f3da2700 1 -- 172.21.2.101:0/3426502742 shutdown_connections 2021-02-10T11:35:56.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.192 7f94f3da2700 1 -- 172.21.2.101:0/3426502742 wait complete. 2021-02-10T11:35:56.208 INFO:tasks.ceph.mds.a:Restarting daemon 2021-02-10T11:35:56.208 INFO:tasks.ceph.mds.a:Stopping old one... 2021-02-10T11:35:56.208 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-02-10T11:35:56.208 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:35:56.208 INFO:tasks.ceph.mds.b:Restarting daemon 2021-02-10T11:35:56.209 INFO:tasks.ceph.mds.b:Stopping old one... 2021-02-10T11:35:56.209 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-02-10T11:35:56.209 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:35:56.214 INFO:tasks.ceph.mds.a:Stopped 2021-02-10T11:35:56.214 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-02-10T11:35:56.217 INFO:tasks.ceph.mds.b:Stopped 2021-02-10T11:35:56.217 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-02-10T11:35:56.256 INFO:tasks.ceph.mds.a:Started 2021-02-10T11:35:56.257 DEBUG:teuthology.parallel:result is None 2021-02-10T11:35:56.259 INFO:tasks.ceph.mds.b:Started 2021-02-10T11:35:56.259 DEBUG:teuthology.parallel:result is None 2021-02-10T11:35:56.260 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd blacklist clear 2021-02-10T11:35:56.664 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:35:56.669 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:35:56.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.668 7fdb48b58700 1 Processor -- start 2021-02-10T11:35:56.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.668 7fdb48b58700 1 -- start start 2021-02-10T11:35:56.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.668 7fdb48b58700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb440d1870 0x7fdb441b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:56.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.668 7fdb48b58700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fdb441b0a10 0x7fdb440cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:56.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.668 7fdb48b58700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdb440d1c70 0x7fdb440cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:56.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.668 7fdb48b58700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb440fe230 con 0x7fdb440d1c70 2021-02-10T11:35:56.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.668 7fdb48b58700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb440e5ed0 con 0x7fdb441b0a10 2021-02-10T11:35:56.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.668 7fdb48b58700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440f0b10 con 0x7fdb440d1870 2021-02-10T11:35:56.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.668 7fdb4305f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdb440d1c70 0x7fdb440cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:56.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.669 7fdb4305f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdb440d1c70 0x7fdb440cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53468/0 (socket says 172.21.2.101:53468) 2021-02-10T11:35:56.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.669 7fdb4305f700 1 -- 172.21.2.101:0/879690542 learned_addr learned my addr 172.21.2.101:0/879690542 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:56.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.669 7fdb4285e700 1 --2- 172.21.2.101:0/879690542 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb440d1870 0x7fdb441b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:56.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.669 7fdb4205d700 1 --2- 172.21.2.101:0/879690542 >> v2:172.21.2.108:3300/0 conn(0x7fdb441b0a10 0x7fdb440cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:56.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.669 7fdb4305f700 1 -- 172.21.2.101:0/879690542 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb440d1870 msgr2=0x7fdb441b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:56.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.669 7fdb4305f700 1 --2- 172.21.2.101:0/879690542 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb440d1870 0x7fdb441b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:56.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.669 7fdb4305f700 1 -- 172.21.2.101:0/879690542 >> v2:172.21.2.108:3300/0 conn(0x7fdb441b0a10 msgr2=0x7fdb440cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:56.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.669 7fdb4305f700 1 --2- 172.21.2.101:0/879690542 >> v2:172.21.2.108:3300/0 conn(0x7fdb441b0a10 0x7fdb440cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:56.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.669 7fdb4305f700 1 -- 172.21.2.101:0/879690542 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb440cff70 con 0x7fdb440d1c70 2021-02-10T11:35:56.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.669 7fdb4305f700 1 --2- 172.21.2.101:0/879690542 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb440d1c70 0x7fdb440cf950 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 rx=0x7fdb34006b80 tx=0x7fdb34009b90).ready entity=mon.0 client_cookie=1b4e1355d876f685 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:56.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.669 7fdb4185c700 1 -- 172.21.2.101:0/879690542 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdb340022e0 con 0x7fdb440d1c70 2021-02-10T11:35:56.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.669 7fdb4185c700 1 -- 172.21.2.101:0/879690542 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb34002560 con 0x7fdb440d1c70 2021-02-10T11:35:56.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.669 7fdb4185c700 1 -- 172.21.2.101:0/879690542 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdb34002a80 con 0x7fdb440d1c70 2021-02-10T11:35:56.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.669 7fdb48b58700 1 -- 172.21.2.101:0/879690542 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb440d1c70 msgr2=0x7fdb440cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:56.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.669 7fdb48b58700 1 --2- 172.21.2.101:0/879690542 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb440d1c70 0x7fdb440cf950 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 rx=0x7fdb34006b80 tx=0x7fdb34009b90).stop 2021-02-10T11:35:56.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.670 7fdb48b58700 1 -- 172.21.2.101:0/879690542 shutdown_connections 2021-02-10T11:35:56.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.670 7fdb48b58700 1 --2- 172.21.2.101:0/879690542 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb440d1c70 0x7fdb440cf950 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:56.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.670 7fdb48b58700 1 --2- 172.21.2.101:0/879690542 >> v2:172.21.2.108:3300/0 conn(0x7fdb441b0a10 0x7fdb440cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:56.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.670 7fdb48b58700 1 --2- 172.21.2.101:0/879690542 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb440d1870 0x7fdb441b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:56.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.670 7fdb48b58700 1 -- 172.21.2.101:0/879690542 >> 172.21.2.101:0/879690542 conn(0x7fdb440ce450 msgr2=0x7fdb440d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:56.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.670 7fdb48b58700 1 -- 172.21.2.101:0/879690542 shutdown_connections 2021-02-10T11:35:56.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.670 7fdb48b58700 1 -- 172.21.2.101:0/879690542 wait complete. 2021-02-10T11:35:56.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.685 7fdb48b58700 1 Processor -- start 2021-02-10T11:35:56.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.685 7fdb48b58700 1 -- start start 2021-02-10T11:35:56.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.685 7fdb48b58700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb440ebe80 0x7fdb440b8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:56.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.685 7fdb48b58700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb440afa30 0x7fdb440afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:56.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.685 7fdb48b58700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdb440b0450 0x7fdb440ec8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:56.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.685 7fdb48b58700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440b8d50 con 0x7fdb440ebe80 2021-02-10T11:35:56.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.685 7fdb48b58700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4409df70 con 0x7fdb440b0450 2021-02-10T11:35:56.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.685 7fdb48b58700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440b0850 con 0x7fdb440afa30 2021-02-10T11:35:56.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.686 7fdb4285e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb440ebe80 0x7fdb440b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:56.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.686 7fdb4285e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb440ebe80 0x7fdb440b8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53472/0 (socket says 172.21.2.101:53472) 2021-02-10T11:35:56.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.686 7fdb4285e700 1 -- 172.21.2.101:0/232303764 learned_addr learned my addr 172.21.2.101:0/232303764 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:56.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.686 7fdb4285e700 1 -- 172.21.2.101:0/232303764 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb440afa30 msgr2=0x7fdb440afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:56.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.686 7fdb4205d700 1 --2- 172.21.2.101:0/232303764 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb440afa30 0x7fdb440afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:56.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.686 7fdb4305f700 1 --2- 172.21.2.101:0/232303764 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdb440b0450 0x7fdb440ec8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:56.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.686 7fdb4285e700 1 --2- 172.21.2.101:0/232303764 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb440afa30 0x7fdb440afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:56.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.686 7fdb4285e700 1 -- 172.21.2.101:0/232303764 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdb440b0450 msgr2=0x7fdb440ec8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:56.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.686 7fdb4285e700 1 --2- 172.21.2.101:0/232303764 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdb440b0450 0x7fdb440ec8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:56.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.686 7fdb4285e700 1 -- 172.21.2.101:0/232303764 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb440ecef0 con 0x7fdb440ebe80 2021-02-10T11:35:56.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.686 7fdb4205d700 1 --2- 172.21.2.101:0/232303764 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb440afa30 0x7fdb440afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:35:56.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.686 7fdb4305f700 1 --2- 172.21.2.101:0/232303764 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdb440b0450 0x7fdb440ec8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:35:56.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.686 7fdb4285e700 1 --2- 172.21.2.101:0/232303764 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb440ebe80 0x7fdb440b8730 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 rx=0x7fdb3c0099e0 tx=0x7fdb3c00bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:56.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.686 7fdb337fe700 1 -- 172.21.2.101:0/232303764 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdb3c00c2f0 con 0x7fdb440ebe80 2021-02-10T11:35:56.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.686 7fdb337fe700 1 -- 172.21.2.101:0/232303764 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3c00c670 con 0x7fdb440ebe80 2021-02-10T11:35:56.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.686 7fdb337fe700 1 -- 172.21.2.101:0/232303764 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdb3c00cb90 con 0x7fdb440ebe80 2021-02-10T11:35:56.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.686 7fdb48b58700 1 -- 172.21.2.101:0/232303764 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb440ecef0 con 0x7fdb440ebe80 2021-02-10T11:35:56.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.686 7fdb48b58700 1 -- 172.21.2.101:0/232303764 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb440ed480 con 0x7fdb440ebe80 2021-02-10T11:35:56.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.687 7fdb337fe700 1 -- 172.21.2.101:0/232303764 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdb3c00c2f0 con 0x7fdb440ebe80 2021-02-10T11:35:56.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.687 7fdb337fe700 1 --2- 172.21.2.101:0/232303764 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdb280186c0 0x7fdb2801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:56.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.687 7fdb337fe700 1 -- 172.21.2.101:0/232303764 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4839+0+0 (secure 0 0 0) 0x7fdb3c032610 con 0x7fdb440ebe80 2021-02-10T11:35:56.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.687 7fdb4205d700 1 --2- 172.21.2.101:0/232303764 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdb280186c0 0x7fdb2801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:56.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.687 7fdb4205d700 1 --2- 172.21.2.101:0/232303764 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdb280186c0 0x7fdb2801abc0 crc :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:56.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.717 7fdb48b58700 1 -- 172.21.2.101:0/232303764 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb440ebc50 con 0x7fdb440ebe80 2021-02-10T11:35:56.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.720 7fdb337fe700 1 -- 172.21.2.101:0/232303764 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdb3c000cc0 con 0x7fdb440ebe80 2021-02-10T11:35:56.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:56.860 7fdb48b58700 1 -- 172.21.2.101:0/232303764 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fdb440f0b10 con 0x7fdb440ebe80 2021-02-10T11:35:57.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.184 7fdb337fe700 1 -- 172.21.2.101:0/232303764 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v69) v1 ==== 97+0+0 (secure 0 0 0) 0x7fdb3c00c2f0 con 0x7fdb440ebe80 2021-02-10T11:35:57.233 INFO:teuthology.orchestra.run.gibba001.stderr: removed all blacklist entries 2021-02-10T11:35:57.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.228 7fdb48b58700 1 -- 172.21.2.101:0/232303764 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdb280186c0 msgr2=0x7fdb2801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:57.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.228 7fdb48b58700 1 --2- 172.21.2.101:0/232303764 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdb280186c0 0x7fdb2801abc0 crc :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:57.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.228 7fdb48b58700 1 -- 172.21.2.101:0/232303764 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb440ebe80 msgr2=0x7fdb440b8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:57.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.228 7fdb48b58700 1 --2- 172.21.2.101:0/232303764 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb440ebe80 0x7fdb440b8730 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 rx=0x7fdb3c0099e0 tx=0x7fdb3c00bd40).stop 2021-02-10T11:35:57.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.228 7fdb48b58700 1 -- 172.21.2.101:0/232303764 shutdown_connections 2021-02-10T11:35:57.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.228 7fdb48b58700 1 --2- 172.21.2.101:0/232303764 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdb280186c0 0x7fdb2801abc0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:57.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.228 7fdb48b58700 1 --2- 172.21.2.101:0/232303764 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb440afa30 0x7fdb440afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:57.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.228 7fdb48b58700 1 --2- 172.21.2.101:0/232303764 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdb440b0450 0x7fdb440ec8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:57.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.228 7fdb48b58700 1 --2- 172.21.2.101:0/232303764 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb440ebe80 0x7fdb440b8730 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:57.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.228 7fdb48b58700 1 -- 172.21.2.101:0/232303764 >> 172.21.2.101:0/232303764 conn(0x7fdb440a6b90 msgr2=0x7fdb440af100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:57.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.228 7fdb48b58700 1 -- 172.21.2.101:0/232303764 shutdown_connections 2021-02-10T11:35:57.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.228 7fdb48b58700 1 -- 172.21.2.101:0/232303764 wait complete. 2021-02-10T11:35:57.244 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2021-02-10T11:35:57.244 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:35:57.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.646 7fbcc139b700 1 Processor -- start 2021-02-10T11:35:57.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.646 7fbcc139b700 1 -- start start 2021-02-10T11:35:57.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.647 7fbcc139b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbcbc0d1c70 0x7fbcbc0f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:57.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.647 7fbcc139b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcbc0f0b10 con 0x7fbcbc0d1870 2021-02-10T11:35:57.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.647 7fbcc139b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcbc1b0be0 con 0x7fbcbc0d0340 2021-02-10T11:35:57.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.647 7fbcc139b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcbc0e5ed0 con 0x7fbcbc0d1c70 2021-02-10T11:35:57.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.647 7fbcbb07d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fbcbc0d1870 0x7fbcbc1b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53972/0 (socket says 172.21.2.101:53972) 2021-02-10T11:35:57.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.647 7fbcbb07d700 1 -- 172.21.2.101:0/277080056 learned_addr learned my addr 172.21.2.101:0/277080056 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:57.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.647 7fbcbb87e700 1 --2- 172.21.2.101:0/277080056 >> v2:172.21.2.108:3300/0 conn(0x7fbcbc0d1c70 0x7fbcbc0f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:57.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.647 7fbcba07b700 1 -- 172.21.2.101:0/277080056 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1128984584 0 0) 0x7fbcac0009b0 con 0x7fbcbc0d1870 2021-02-10T11:35:57.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.647 7fbcba07b700 1 -- 172.21.2.101:0/277080056 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbca8002fe0 con 0x7fbcbc0d1870 2021-02-10T11:35:57.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.648 7fbcbb87e700 1 -- 172.21.2.101:0/277080056 >> v1:172.21.2.108:6789/0 conn(0x7fbcbc0d0340 legacy=0x7fbcbc0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:57.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.648 7fbcbb87e700 1 -- 172.21.2.101:0/277080056 >> v1:172.21.2.101:6789/0 conn(0x7fbcbc0d1870 legacy=0x7fbcbc1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:57.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.648 7fbcbb87e700 1 -- 172.21.2.101:0/277080056 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc0efe30 con 0x7fbcbc0d1c70 2021-02-10T11:35:57.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.648 7fbcbb87e700 1 --2- 172.21.2.101:0/277080056 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbcbc0d1c70 0x7fbcbc0f20a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7fbcb4006e50 tx=0x7fbcb4009e70).ready entity=mon.1 client_cookie=c201112aa01b0d46 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:57.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.648 7fbcba07b700 1 -- 172.21.2.101:0/277080056 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbcb4006a20 con 0x7fbcbc0d1c70 2021-02-10T11:35:57.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.648 7fbcba07b700 1 -- 172.21.2.101:0/277080056 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb4006790 con 0x7fbcbc0d1c70 2021-02-10T11:35:57.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.648 7fbcba07b700 1 -- 172.21.2.101:0/277080056 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbcb400b700 con 0x7fbcbc0d1c70 2021-02-10T11:35:57.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.648 7fbcc139b700 1 -- 172.21.2.101:0/277080056 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbcbc0d1c70 msgr2=0x7fbcbc0f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:57.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.648 7fbcc139b700 1 --2- 172.21.2.101:0/277080056 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbcbc0d1c70 0x7fbcbc0f20a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7fbcb4006e50 tx=0x7fbcb4009e70).stop 2021-02-10T11:35:57.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.649 7fbcc139b700 1 -- 172.21.2.101:0/277080056 shutdown_connections 2021-02-10T11:35:57.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.649 7fbcc139b700 1 --2- 172.21.2.101:0/277080056 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbcbc0d1c70 0x7fbcbc0f20a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:57.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.649 7fbcc139b700 1 -- 172.21.2.101:0/277080056 >> 172.21.2.101:0/277080056 conn(0x7fbcbc0ce450 msgr2=0x7fbcbc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:57.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.649 7fbcc139b700 1 -- 172.21.2.101:0/277080056 shutdown_connections 2021-02-10T11:35:57.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.649 7fbcc139b700 1 -- 172.21.2.101:0/277080056 wait complete. 2021-02-10T11:35:57.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.663 7fbcc139b700 1 Processor -- start 2021-02-10T11:35:57.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.664 7fbcc139b700 1 -- start start 2021-02-10T11:35:57.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.664 7fbcc139b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbcbc19a0e0 0x7fbcbc09cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:57.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.664 7fbcc139b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbcbc18aee0 0x7fbcbc18b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:57.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.664 7fbcc139b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbcbc18b900 0x7fbcbc187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:57.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.664 7fbcc139b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc09d1a0 con 0x7fbcbc19a0e0 2021-02-10T11:35:57.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.664 7fbcc139b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc19a4e0 con 0x7fbcbc18aee0 2021-02-10T11:35:57.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.664 7fbcc139b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc1ab160 con 0x7fbcbc18b900 2021-02-10T11:35:57.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.664 7fbcbb07d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbcbc19a0e0 0x7fbcbc09cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:57.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.664 7fbcbb07d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbcbc19a0e0 0x7fbcbc09cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53486/0 (socket says 172.21.2.101:53486) 2021-02-10T11:35:57.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.664 7fbcbb07d700 1 -- 172.21.2.101:0/926949781 learned_addr learned my addr 172.21.2.101:0/926949781 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:57.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.665 7fbcbb07d700 1 -- 172.21.2.101:0/926949781 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbcbc18b900 msgr2=0x7fbcbc187e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:57.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.665 7fbcbb07d700 1 --2- 172.21.2.101:0/926949781 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbcbc18b900 0x7fbcbc187e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:57.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.665 7fbcbb07d700 1 -- 172.21.2.101:0/926949781 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbcbc18aee0 msgr2=0x7fbcbc18b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:57.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.665 7fbcbb07d700 1 --2- 172.21.2.101:0/926949781 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbcbc18aee0 0x7fbcbc18b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:57.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.665 7fbcbb07d700 1 -- 172.21.2.101:0/926949781 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc18bd00 con 0x7fbcbc19a0e0 2021-02-10T11:35:57.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.665 7fbcbb07d700 1 --2- 172.21.2.101:0/926949781 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbcbc19a0e0 0x7fbcbc09cb00 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 rx=0x7fbcac001c60 tx=0x7fbcac00a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:57.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.665 7fbcb3fff700 1 -- 172.21.2.101:0/926949781 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbcac00acc0 con 0x7fbcbc19a0e0 2021-02-10T11:35:57.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.665 7fbcb3fff700 1 -- 172.21.2.101:0/926949781 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcac004140 con 0x7fbcbc19a0e0 2021-02-10T11:35:57.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.665 7fbcb3fff700 1 -- 172.21.2.101:0/926949781 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbcac004660 con 0x7fbcbc19a0e0 2021-02-10T11:35:57.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.665 7fbcc139b700 1 -- 172.21.2.101:0/926949781 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcbc18bd00 con 0x7fbcbc19a0e0 2021-02-10T11:35:57.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.665 7fbcc139b700 1 -- 172.21.2.101:0/926949781 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcbc188760 con 0x7fbcbc19a0e0 2021-02-10T11:35:57.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.666 7fbcb3fff700 1 -- 172.21.2.101:0/926949781 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbcac004320 con 0x7fbcbc19a0e0 2021-02-10T11:35:57.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.666 7fbcb3fff700 1 --2- 172.21.2.101:0/926949781 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbca00186c0 0x7fbca001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:57.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.666 7fbcb3fff700 1 -- 172.21.2.101:0/926949781 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fbcac032090 con 0x7fbcbc19a0e0 2021-02-10T11:35:57.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.666 7fbcba87c700 1 --2- 172.21.2.101:0/926949781 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbca00186c0 0x7fbca001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:57.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.666 7fbcba87c700 1 --2- 172.21.2.101:0/926949781 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbca00186c0 0x7fbca001abc0 crc :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:57.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.697 7fbcc139b700 1 -- 172.21.2.101:0/926949781 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcbc0ee080 con 0x7fbcbc19a0e0 2021-02-10T11:35:57.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.700 7fbcb3fff700 1 -- 172.21.2.101:0/926949781 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbcac022560 con 0x7fbcbc19a0e0 2021-02-10T11:35:57.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.830 7fbcc139b700 1 -- 172.21.2.101:0/926949781 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbcbc0ce450 con 0x7fbcbc19a0e0 2021-02-10T11:35:57.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.830 7fbcb3fff700 1 -- 172.21.2.101:0/926949781 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v44) v1 ==== 80+0+4513 (secure 0 0 0) 0x7fbcac022560 con 0x7fbcbc19a0e0 2021-02-10T11:35:57.838 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:57.838 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:35:57.838 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:35:57.838 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:57.838 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:35:57.839 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:35:57.839 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:57.839 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:35:57.839 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:57.839 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:57.839 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:57.839 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:57.839 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:57.839 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:57.839 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:35:57.839 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:35:57.839 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:57.839 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:35:57.840 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:57.840 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:57.840 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:57.840 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:57.840 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:57.840 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:57.840 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:35:57.840 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:35:57.840 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:57.840 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:57.840 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:57.840 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:57.840 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:57.841 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:57.841 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:57.841 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:35:57.841 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:35:57.841 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:57.841 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:57.841 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:57.841 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:57.841 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:57.841 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:57.841 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:57.841 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:35:57.841 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:35:57.842 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:57.842 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:57.842 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:57.842 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:57.842 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:57.842 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:57.842 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:57.842 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:35:57.842 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:35:57.842 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:57.842 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:57.842 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:57.843 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:57.843 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:57.843 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:57.843 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:57.843 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:35:57.843 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:35:57.843 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:57.843 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:57.843 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:57.843 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:57.843 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:57.844 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:57.844 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:57.844 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:35:57.844 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:35:57.844 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:57.844 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:57.844 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:57.844 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:57.844 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:57.844 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:57.845 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:57.845 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:35:57.845 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:35:57.845 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:57.845 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:57.845 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:57.845 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:57.845 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:57.845 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:57.845 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:57.845 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:35:57.845 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:35:57.846 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:57.846 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:57.846 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:57.846 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:57.846 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:57.846 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:57.846 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:57.846 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:35:57.846 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:35:57.846 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:57.846 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:35:57.847 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:35:57.847 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:35:57.847 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:57.847 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:57.847 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:57.847 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:35:57.847 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:35:57.847 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:57.847 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:35:57.847 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:35:57.847 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:35:57.848 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:57.848 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:57.848 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:57.848 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:35:57.848 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:35:57.848 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:57.848 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:35:57.848 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:35:57.848 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:35:57.848 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:57.848 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:57.848 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:57.848 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:35:57.849 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:35:57.849 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:57.849 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:35:57.849 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:35:57.849 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:35:57.850 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:57.850 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:57.850 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:57.850 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:35:57.850 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:35:57.850 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:57.850 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:35:57.850 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:35:57.850 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:35:57.851 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:57.851 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:57.851 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:57.851 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:57.851 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:35:57.851 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:35:57.851 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:57.851 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:35:57.851 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:35:57.851 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:57.851 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:57.851 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:35:57.852 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:57.852 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:35:57.852 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:35:57.852 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:35:57.852 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:35:57.852 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:35:57.852 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:35:57.852 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:57.852 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:57.852 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:35:57.852 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:35:57.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.832 7fbcc139b700 1 -- 172.21.2.101:0/926949781 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbca00186c0 msgr2=0x7fbca001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:57.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.832 7fbcc139b700 1 --2- 172.21.2.101:0/926949781 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbca00186c0 0x7fbca001abc0 crc :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:57.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.832 7fbcc139b700 1 -- 172.21.2.101:0/926949781 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbcbc19a0e0 msgr2=0x7fbcbc09cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:57.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.832 7fbcc139b700 1 --2- 172.21.2.101:0/926949781 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbcbc19a0e0 0x7fbcbc09cb00 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 rx=0x7fbcac001c60 tx=0x7fbcac00a8d0).stop 2021-02-10T11:35:57.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.832 7fbcc139b700 1 -- 172.21.2.101:0/926949781 shutdown_connections 2021-02-10T11:35:57.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.832 7fbcc139b700 1 --2- 172.21.2.101:0/926949781 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbca00186c0 0x7fbca001abc0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:57.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.832 7fbcc139b700 1 --2- 172.21.2.101:0/926949781 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbcbc18b900 0x7fbcbc187e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:57.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.832 7fbcc139b700 1 --2- 172.21.2.101:0/926949781 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbcbc18aee0 0x7fbcbc18b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:57.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.832 7fbcc139b700 1 --2- 172.21.2.101:0/926949781 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbcbc19a0e0 0x7fbcbc09cb00 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:57.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.832 7fbcc139b700 1 -- 172.21.2.101:0/926949781 >> 172.21.2.101:0/926949781 conn(0x7fbcbc093640 msgr2=0x7fbcbc18eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:57.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.833 7fbcc139b700 1 -- 172.21.2.101:0/926949781 shutdown_connections 2021-02-10T11:35:57.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:57.833 7fbcc139b700 1 -- 172.21.2.101:0/926949781 wait complete. 2021-02-10T11:35:57.855 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:35:58.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.259 7f16e38c0700 1 Processor -- start 2021-02-10T11:35:58.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.259 7f16e38c0700 1 -- start start 2021-02-10T11:35:58.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.259 7f16e38c0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16dc0d1c70 0x7f16dc0f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:58.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.259 7f16e38c0700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16dc0f0b10 con 0x7f16dc0d1870 2021-02-10T11:35:58.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.259 7f16e38c0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16dc1b0be0 con 0x7f16dc0d0340 2021-02-10T11:35:58.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.259 7f16e38c0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc0e5ed0 con 0x7f16dc0d1c70 2021-02-10T11:35:58.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.260 7f16e16dc700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f16dc0d1870 0x7f16dc1b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53986/0 (socket says 172.21.2.101:53986) 2021-02-10T11:35:58.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.260 7f16e16dc700 1 -- 172.21.2.101:0/1652393494 learned_addr learned my addr 172.21.2.101:0/1652393494 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:58.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.260 7f16e1edd700 1 --2- 172.21.2.101:0/1652393494 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16dc0d1c70 0x7f16dc0f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:58.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.260 7f16cffff700 1 -- 172.21.2.101:0/1652393494 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345853137 0 0) 0x7f16d8001180 con 0x7f16dc0d1870 2021-02-10T11:35:58.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.260 7f16cffff700 1 -- 172.21.2.101:0/1652393494 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16c8002fe0 con 0x7f16dc0d1870 2021-02-10T11:35:58.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.260 7f16e1edd700 1 -- 172.21.2.101:0/1652393494 >> v1:172.21.2.108:6789/0 conn(0x7f16dc0d0340 legacy=0x7f16dc0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:58.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.260 7f16e1edd700 1 -- 172.21.2.101:0/1652393494 >> v1:172.21.2.101:6789/0 conn(0x7f16dc0d1870 legacy=0x7f16dc1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:58.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.260 7f16e1edd700 1 -- 172.21.2.101:0/1652393494 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc0efe30 con 0x7f16dc0d1c70 2021-02-10T11:35:58.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.261 7f16e1edd700 1 --2- 172.21.2.101:0/1652393494 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16dc0d1c70 0x7f16dc0f20a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f16d4006e50 tx=0x7f16d4009e70).ready entity=mon.2 client_cookie=730372ae82b03029 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:58.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.261 7f16cffff700 1 -- 172.21.2.101:0/1652393494 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f16d400b510 con 0x7f16dc0d1c70 2021-02-10T11:35:58.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.261 7f16cffff700 1 -- 172.21.2.101:0/1652393494 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d4006790 con 0x7f16dc0d1c70 2021-02-10T11:35:58.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.261 7f16cffff700 1 -- 172.21.2.101:0/1652393494 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f16d400b940 con 0x7f16dc0d1c70 2021-02-10T11:35:58.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.261 7f16e38c0700 1 -- 172.21.2.101:0/1652393494 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16dc0d1c70 msgr2=0x7f16dc0f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:58.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.261 7f16e38c0700 1 --2- 172.21.2.101:0/1652393494 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16dc0d1c70 0x7f16dc0f20a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f16d4006e50 tx=0x7f16d4009e70).stop 2021-02-10T11:35:58.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.261 7f16e38c0700 1 -- 172.21.2.101:0/1652393494 shutdown_connections 2021-02-10T11:35:58.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.261 7f16e38c0700 1 --2- 172.21.2.101:0/1652393494 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16dc0d1c70 0x7f16dc0f20a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:58.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.261 7f16e38c0700 1 -- 172.21.2.101:0/1652393494 >> 172.21.2.101:0/1652393494 conn(0x7f16dc0ce450 msgr2=0x7f16dc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:58.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.262 7f16e38c0700 1 -- 172.21.2.101:0/1652393494 shutdown_connections 2021-02-10T11:35:58.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.262 7f16e38c0700 1 -- 172.21.2.101:0/1652393494 wait complete. 2021-02-10T11:35:58.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.276 7f16e38c0700 1 Processor -- start 2021-02-10T11:35:58.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.276 7f16e38c0700 1 -- start start 2021-02-10T11:35:58.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.277 7f16e38c0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16dc19af00 0x7f16dc19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:58.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.277 7f16e38c0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f16dc19ba80 0x7f16dc19be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:58.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.277 7f16e38c0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16dc19c4a0 0x7f16dc08a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:58.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.277 7f16e38c0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc08fb50 con 0x7f16dc19c4a0 2021-02-10T11:35:58.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.277 7f16e38c0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc08afb0 con 0x7f16dc19ba80 2021-02-10T11:35:58.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.277 7f16e38c0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc08b1a0 con 0x7f16dc19af00 2021-02-10T11:35:58.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.277 7f16e1edd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16dc19c4a0 0x7f16dc08a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:58.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.277 7f16e1edd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16dc19c4a0 0x7f16dc08a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53500/0 (socket says 172.21.2.101:53500) 2021-02-10T11:35:58.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.277 7f16e1edd700 1 -- 172.21.2.101:0/247364690 learned_addr learned my addr 172.21.2.101:0/247364690 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:58.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.277 7f16e1edd700 1 -- 172.21.2.101:0/247364690 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16dc19af00 msgr2=0x7f16dc19b300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:35:58.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.277 7f16e1edd700 1 --2- 172.21.2.101:0/247364690 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16dc19af00 0x7f16dc19b300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:58.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.277 7f16e1edd700 1 -- 172.21.2.101:0/247364690 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f16dc19ba80 msgr2=0x7f16dc19be80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:35:58.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.277 7f16e1edd700 1 --2- 172.21.2.101:0/247364690 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f16dc19ba80 0x7f16dc19be80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:58.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.277 7f16e1edd700 1 -- 172.21.2.101:0/247364690 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc08b3c0 con 0x7f16dc19c4a0 2021-02-10T11:35:58.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.277 7f16e1edd700 1 --2- 172.21.2.101:0/247364690 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16dc19c4a0 0x7f16dc08a990 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 rx=0x7f16d4006c50 tx=0x7f16d400bd20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:58.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.278 7f16cdffb700 1 -- 172.21.2.101:0/247364690 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f16d40093d0 con 0x7f16dc19c4a0 2021-02-10T11:35:58.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.278 7f16cdffb700 1 -- 172.21.2.101:0/247364690 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d400be60 con 0x7f16dc19c4a0 2021-02-10T11:35:58.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.278 7f16cdffb700 1 -- 172.21.2.101:0/247364690 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f16d400c4e0 con 0x7f16dc19c4a0 2021-02-10T11:35:58.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.278 7f16e38c0700 1 -- 172.21.2.101:0/247364690 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16dc08b3c0 con 0x7f16dc19c4a0 2021-02-10T11:35:58.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.278 7f16e38c0700 1 -- 172.21.2.101:0/247364690 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16dc08b950 con 0x7f16dc19c4a0 2021-02-10T11:35:58.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.278 7f16cdffb700 1 -- 172.21.2.101:0/247364690 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f16d40093d0 con 0x7f16dc19c4a0 2021-02-10T11:35:58.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.278 7f16cdffb700 1 --2- 172.21.2.101:0/247364690 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16bc0186c0 0x7f16bc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:58.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.279 7f16cdffb700 1 -- 172.21.2.101:0/247364690 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f16d40325b0 con 0x7f16dc19c4a0 2021-02-10T11:35:58.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.279 7f16e16dc700 1 --2- 172.21.2.101:0/247364690 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16bc0186c0 0x7f16bc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:58.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.279 7f16e16dc700 1 --2- 172.21.2.101:0/247364690 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16bc0186c0 0x7f16bc01abc0 crc :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:58.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.309 7f16e38c0700 1 -- 172.21.2.101:0/247364690 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16dc0ee030 con 0x7f16dc19c4a0 2021-02-10T11:35:58.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.312 7f16cdffb700 1 -- 172.21.2.101:0/247364690 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f16d4002ef0 con 0x7f16dc19c4a0 2021-02-10T11:35:58.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.436 7f16e38c0700 1 -- 172.21.2.101:0/247364690 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16dc0ce450 con 0x7f16dc19c4a0 2021-02-10T11:35:58.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.436 7f16cdffb700 1 -- 172.21.2.101:0/247364690 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+11145 (secure 0 0 0) 0x7f16d40093d0 con 0x7f16dc19c4a0 2021-02-10T11:35:58.444 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:58.444 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":69,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:35:57.179410","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:35:58.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.438 7f16e38c0700 1 -- 172.21.2.101:0/247364690 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16bc0186c0 msgr2=0x7f16bc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:58.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.438 7f16e38c0700 1 --2- 172.21.2.101:0/247364690 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16bc0186c0 0x7f16bc01abc0 crc :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:58.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.438 7f16e38c0700 1 -- 172.21.2.101:0/247364690 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16dc19c4a0 msgr2=0x7f16dc08a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:58.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.438 7f16e38c0700 1 --2- 172.21.2.101:0/247364690 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16dc19c4a0 0x7f16dc08a990 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 rx=0x7f16d4006c50 tx=0x7f16d400bd20).stop 2021-02-10T11:35:58.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.438 7f16e38c0700 1 -- 172.21.2.101:0/247364690 shutdown_connections 2021-02-10T11:35:58.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.438 7f16e38c0700 1 --2- 172.21.2.101:0/247364690 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16dc19c4a0 0x7f16dc08a990 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:58.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.438 7f16e38c0700 1 --2- 172.21.2.101:0/247364690 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f16dc19ba80 0x7f16dc19be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:58.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.438 7f16e38c0700 1 --2- 172.21.2.101:0/247364690 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16bc0186c0 0x7f16bc01abc0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:58.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.438 7f16e38c0700 1 --2- 172.21.2.101:0/247364690 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16dc19af00 0x7f16dc19b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:58.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.438 7f16e38c0700 1 -- 172.21.2.101:0/247364690 >> 172.21.2.101:0/247364690 conn(0x7f16dc19f8c0 msgr2=0x7f16dc1a5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:58.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.439 7f16e38c0700 1 -- 172.21.2.101:0/247364690 shutdown_connections 2021-02-10T11:35:58.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.439 7f16e38c0700 1 -- 172.21.2.101:0/247364690 wait complete. 2021-02-10T11:35:58.455 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-10T11:35:58.455 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:35:58.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.861 7faf68080700 1 Processor -- start 2021-02-10T11:35:58.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.861 7faf68080700 1 -- start start 2021-02-10T11:35:58.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.861 7faf68080700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7faf600d1870 0x7faf601b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:58.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.861 7faf68080700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf600fe1c0 con 0x7faf600cff70 2021-02-10T11:35:58.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.861 7faf68080700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf60109a60 con 0x7faf601b0ae0 2021-02-10T11:35:58.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.861 7faf68080700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf60088070 con 0x7faf600d1870 2021-02-10T11:35:58.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.862 7faf65e9c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7faf600d1870 0x7faf601b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:58.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.862 7faf65e9c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7faf600d1870 0x7faf601b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53504/0 (socket says 172.21.2.101:53504) 2021-02-10T11:35:58.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.862 7faf65e9c700 1 -- 172.21.2.101:0/3194318849 learned_addr learned my addr 172.21.2.101:0/3194318849 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:58.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.862 7faf6669d700 1 --1- 172.21.2.101:0/3194318849 >> v1:172.21.2.101:6789/0 conn(0x7faf600cff70 0x7faf60109130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:53998/0 (socket says 172.21.2.101:53998) 2021-02-10T11:35:58.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.862 7faf65e9c700 1 -- 172.21.2.101:0/3194318849 >> v1:172.21.2.108:6789/0 conn(0x7faf601b0ae0 legacy=0x7faf60108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:58.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.862 7faf65e9c700 1 -- 172.21.2.101:0/3194318849 >> v1:172.21.2.101:6789/0 conn(0x7faf600cff70 legacy=0x7faf60109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:58.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.862 7faf65e9c700 1 -- 172.21.2.101:0/3194318849 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf60088260 con 0x7faf600d1870 2021-02-10T11:35:58.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.862 7faf65e9c700 1 --2- 172.21.2.101:0/3194318849 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faf600d1870 0x7faf601b04c0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 rx=0x7faf5c006a80 tx=0x7faf5c009e80).ready entity=mon.0 client_cookie=40964db495371c86 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:58.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.863 7faf64e9a700 1 -- 172.21.2.101:0/3194318849 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faf5c00b510 con 0x7faf600d1870 2021-02-10T11:35:58.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.863 7faf64e9a700 1 -- 172.21.2.101:0/3194318849 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5c00b7d0 con 0x7faf600d1870 2021-02-10T11:35:58.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.863 7faf64e9a700 1 -- 172.21.2.101:0/3194318849 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faf5c00bc20 con 0x7faf600d1870 2021-02-10T11:35:58.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.863 7faf68080700 1 -- 172.21.2.101:0/3194318849 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faf600d1870 msgr2=0x7faf601b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:58.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.863 7faf68080700 1 --2- 172.21.2.101:0/3194318849 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faf600d1870 0x7faf601b04c0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 rx=0x7faf5c006a80 tx=0x7faf5c009e80).stop 2021-02-10T11:35:58.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.863 7faf68080700 1 -- 172.21.2.101:0/3194318849 shutdown_connections 2021-02-10T11:35:58.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.863 7faf68080700 1 --2- 172.21.2.101:0/3194318849 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faf600d1870 0x7faf601b04c0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:58.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.863 7faf68080700 1 -- 172.21.2.101:0/3194318849 >> 172.21.2.101:0/3194318849 conn(0x7faf600ce420 msgr2=0x7faf600d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:58.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.864 7faf68080700 1 -- 172.21.2.101:0/3194318849 shutdown_connections 2021-02-10T11:35:58.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.864 7faf68080700 1 -- 172.21.2.101:0/3194318849 wait complete. 2021-02-10T11:35:58.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.878 7faf68080700 1 Processor -- start 2021-02-10T11:35:58.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.878 7faf68080700 1 -- start start 2021-02-10T11:35:58.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.879 7faf68080700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faf601042d0 0x7faf6009fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:58.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.879 7faf68080700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faf601046d0 0x7faf600a02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:58.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.879 7faf68080700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faf60107900 0x7faf60107d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:58.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.879 7faf68080700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf60104ad0 con 0x7faf60107900 2021-02-10T11:35:58.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.879 7faf68080700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf60104d30 con 0x7faf601042d0 2021-02-10T11:35:58.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.879 7faf68080700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf60101830 con 0x7faf601046d0 2021-02-10T11:35:58.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.879 7faf6669d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faf60107900 0x7faf60107d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:58.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.879 7faf6669d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faf60107900 0x7faf60107d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53508/0 (socket says 172.21.2.101:53508) 2021-02-10T11:35:58.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.879 7faf6669d700 1 -- 172.21.2.101:0/3920224367 learned_addr learned my addr 172.21.2.101:0/3920224367 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:58.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.879 7faf6669d700 1 -- 172.21.2.101:0/3920224367 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faf601046d0 msgr2=0x7faf600a02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:58.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.879 7faf6669d700 1 --2- 172.21.2.101:0/3920224367 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faf601046d0 0x7faf600a02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:58.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.879 7faf6669d700 1 -- 172.21.2.101:0/3920224367 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faf601042d0 msgr2=0x7faf6009fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:58.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.879 7faf6669d700 1 --2- 172.21.2.101:0/3920224367 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faf601042d0 0x7faf6009fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:58.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.879 7faf6669d700 1 -- 172.21.2.101:0/3920224367 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf60108320 con 0x7faf60107900 2021-02-10T11:35:58.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.879 7faf6669d700 1 --2- 172.21.2.101:0/3920224367 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faf60107900 0x7faf60107d00 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 rx=0x7faf5800a590 tx=0x7faf5800a930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:58.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.880 7faf567fc700 1 -- 172.21.2.101:0/3920224367 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faf5800ad20 con 0x7faf60107900 2021-02-10T11:35:58.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.880 7faf567fc700 1 -- 172.21.2.101:0/3920224367 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf580070d0 con 0x7faf60107900 2021-02-10T11:35:58.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.880 7faf567fc700 1 -- 172.21.2.101:0/3920224367 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faf58004620 con 0x7faf60107900 2021-02-10T11:35:58.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.880 7faf68080700 1 -- 172.21.2.101:0/3920224367 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf60108320 con 0x7faf60107900 2021-02-10T11:35:58.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.880 7faf68080700 1 -- 172.21.2.101:0/3920224367 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf60108320 con 0x7faf60107900 2021-02-10T11:35:58.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.880 7faf567fc700 1 -- 172.21.2.101:0/3920224367 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7faf580043a0 con 0x7faf60107900 2021-02-10T11:35:58.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.880 7faf567fc700 1 --2- 172.21.2.101:0/3920224367 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faf480186c0 0x7faf4801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:58.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.881 7faf567fc700 1 -- 172.21.2.101:0/3920224367 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 4753+0+0 (secure 0 0 0) 0x7faf58032310 con 0x7faf60107900 2021-02-10T11:35:58.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.881 7faf65e9c700 1 --2- 172.21.2.101:0/3920224367 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faf480186c0 0x7faf4801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:58.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.881 7faf65e9c700 1 --2- 172.21.2.101:0/3920224367 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faf480186c0 0x7faf4801abc0 crc :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:58.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.911 7faf68080700 1 -- 172.21.2.101:0/3920224367 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf60080780 con 0x7faf60107900 2021-02-10T11:35:58.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:58.914 7faf567fc700 1 -- 172.21.2.101:0/3920224367 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7faf58000b40 con 0x7faf60107900 2021-02-10T11:35:59.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.040 7faf68080700 1 -- 172.21.2.101:0/3920224367 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7faf600ce420 con 0x7faf60107900 2021-02-10T11:35:59.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.041 7faf567fc700 1 -- 172.21.2.101:0/3920224367 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v69) v1 ==== 92+0+10 (secure 0 0 0) 0x7faf5800ad20 con 0x7faf60107900 2021-02-10T11:35:59.049 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:35:59.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.043 7faf68080700 1 -- 172.21.2.101:0/3920224367 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faf480186c0 msgr2=0x7faf4801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:59.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.043 7faf68080700 1 --2- 172.21.2.101:0/3920224367 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faf480186c0 0x7faf4801abc0 crc :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:59.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.043 7faf68080700 1 -- 172.21.2.101:0/3920224367 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faf60107900 msgr2=0x7faf60107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:59.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.043 7faf68080700 1 --2- 172.21.2.101:0/3920224367 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faf60107900 0x7faf60107d00 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 rx=0x7faf5800a590 tx=0x7faf5800a930).stop 2021-02-10T11:35:59.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.043 7faf68080700 1 -- 172.21.2.101:0/3920224367 shutdown_connections 2021-02-10T11:35:59.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.043 7faf68080700 1 --2- 172.21.2.101:0/3920224367 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faf480186c0 0x7faf4801abc0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:59.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.043 7faf68080700 1 --2- 172.21.2.101:0/3920224367 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faf601046d0 0x7faf600a02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:59.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.043 7faf68080700 1 --2- 172.21.2.101:0/3920224367 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faf60107900 0x7faf60107d00 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:59.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.043 7faf68080700 1 --2- 172.21.2.101:0/3920224367 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faf601042d0 0x7faf6009fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:35:59.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.043 7faf68080700 1 -- 172.21.2.101:0/3920224367 >> 172.21.2.101:0/3920224367 conn(0x7faf600c1c40 msgr2=0x7faf600a0980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:59.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.043 7faf68080700 1 -- 172.21.2.101:0/3920224367 shutdown_connections 2021-02-10T11:35:59.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.043 7faf68080700 1 -- 172.21.2.101:0/3920224367 wait complete. 2021-02-10T11:35:59.059 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --format=json-pretty osd lspools 2021-02-10T11:35:59.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.461 7f1f8a48a700 1 Processor -- start 2021-02-10T11:35:59.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.461 7f1f8a48a700 1 -- start start 2021-02-10T11:35:59.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.462 7f1f8a48a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1f840d1c70 0x7f1f840f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:59.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.462 7f1f8a48a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f840f0b10 con 0x7f1f840d0340 2021-02-10T11:35:59.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.462 7f1f8a48a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f841b0be0 con 0x7f1f840d1870 2021-02-10T11:35:59.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.462 7f1f8a48a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f840e5ed0 con 0x7f1f840d1c70 2021-02-10T11:35:59.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.462 7f1f837fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f1f840d0340 0x7f1f840ef640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54010/0 (socket says 172.21.2.101:54010) 2021-02-10T11:35:59.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.462 7f1f837fe700 1 -- 172.21.2.101:0/2216445192 learned_addr learned my addr 172.21.2.101:0/2216445192 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:35:59.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.462 7f1f82ffd700 1 -- 172.21.2.101:0/2216445192 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2030904441 0 0) 0x7f1f78000ce0 con 0x7f1f840d0340 2021-02-10T11:35:59.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.462 7f1f82ffd700 1 -- 172.21.2.101:0/2216445192 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f6c002fe0 con 0x7f1f840d0340 2021-02-10T11:35:59.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.462 7f1f82ffd700 1 -- 172.21.2.101:0/2216445192 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1500701819 0 0) 0x7f1f78001180 con 0x7f1f840d0340 2021-02-10T11:35:59.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.462 7f1f82ffd700 1 -- 172.21.2.101:0/2216445192 >> v2:172.21.2.108:3300/0 conn(0x7f1f840d1c70 msgr2=0x7f1f840f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:59.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.462 7f1f88aa7700 1 --2- 172.21.2.101:0/2216445192 >> v2:172.21.2.108:3300/0 conn(0x7f1f840d1c70 0x7f1f840f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:59.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.463 7f1f82ffd700 1 --2- 172.21.2.101:0/2216445192 >> v2:172.21.2.108:3300/0 conn(0x7f1f840d1c70 0x7f1f840f20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:59.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.463 7f1f82ffd700 1 -- 172.21.2.101:0/2216445192 >> v1:172.21.2.108:6789/0 conn(0x7f1f840d1870 legacy=0x7f1f841b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:59.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.463 7f1f82ffd700 1 -- 172.21.2.101:0/2216445192 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f840efe30 con 0x7f1f840d0340 2021-02-10T11:35:59.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.463 7f1f82ffd700 1 -- 172.21.2.101:0/2216445192 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f1f78001680 con 0x7f1f840d0340 2021-02-10T11:35:59.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.463 7f1f82ffd700 1 -- 172.21.2.101:0/2216445192 >> v1:172.21.2.101:6789/0 conn(0x7f1f840d0340 legacy=0x7f1f840ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:59.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.463 7f1f82ffd700 1 --2- 172.21.2.101:0/2216445192 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f6c003b70 0x7f1f6c005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:59.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.463 7f1f82ffd700 1 --2- 172.21.2.101:0/2216445192 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f6c006840 0x7f1f6c008d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:59.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.463 7f1f82ffd700 1 --2- 172.21.2.101:0/2216445192 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f6c009620 0x7f1f6c00bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:59.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.463 7f1f82ffd700 1 -- 172.21.2.101:0/2216445192 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c00c390 con 0x7f1f6c003b70 2021-02-10T11:35:59.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.463 7f1f82ffd700 1 -- 172.21.2.101:0/2216445192 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c00c5b0 con 0x7f1f6c009620 2021-02-10T11:35:59.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.463 7f1f82ffd700 1 -- 172.21.2.101:0/2216445192 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c00c7d0 con 0x7f1f6c006840 2021-02-10T11:35:59.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.463 7f1f88aa7700 1 --2- 172.21.2.101:0/2216445192 >> v2:172.21.2.108:3300/0 conn(0x7f1f840d1c70 0x7f1f840f20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:35:59.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.463 7f1f83fff700 1 --2- 172.21.2.101:0/2216445192 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f6c003b70 0x7f1f6c005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:59.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.463 7f1f837fe700 1 --2- 172.21.2.101:0/2216445192 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f6c006840 0x7f1f6c008d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:59.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.463 7f1f88aa7700 1 --2- 172.21.2.101:0/2216445192 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f6c009620 0x7f1f6c00bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:59.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.463 7f1f83fff700 1 -- 172.21.2.101:0/2216445192 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f6c006840 msgr2=0x7f1f6c008d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:59.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.463 7f1f83fff700 1 --2- 172.21.2.101:0/2216445192 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f6c006840 0x7f1f6c008d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:59.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.463 7f1f83fff700 1 -- 172.21.2.101:0/2216445192 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f6c009620 msgr2=0x7f1f6c00bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:35:59.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.463 7f1f83fff700 1 --2- 172.21.2.101:0/2216445192 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f6c009620 0x7f1f6c00bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:59.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.463 7f1f83fff700 1 -- 172.21.2.101:0/2216445192 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c00ca70 con 0x7f1f6c003b70 2021-02-10T11:35:59.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.463 7f1f88aa7700 1 -- 172.21.2.101:0/2216445192 reap_dead start 2021-02-10T11:35:59.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.463 7f1f83fff700 1 --2- 172.21.2.101:0/2216445192 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f6c003b70 0x7f1f6c005fa0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 rx=0x7f1f74006ab0 tx=0x7f1f7404daa0).ready entity=mon.0 client_cookie=a58eb23fe07b0ac server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:59.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.464 7f1f82ffd700 1 -- 172.21.2.101:0/2216445192 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f740066f0 con 0x7f1f6c003b70 2021-02-10T11:35:59.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.464 7f1f82ffd700 1 -- 172.21.2.101:0/2216445192 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f740500c0 con 0x7f1f6c003b70 2021-02-10T11:35:59.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.464 7f1f82ffd700 1 -- 172.21.2.101:0/2216445192 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f740505e0 con 0x7f1f6c003b70 2021-02-10T11:35:59.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.464 7f1f8a48a700 1 -- 172.21.2.101:0/2216445192 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f6c003b70 msgr2=0x7f1f6c005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:59.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.464 7f1f8a48a700 1 --2- 172.21.2.101:0/2216445192 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f6c003b70 0x7f1f6c005fa0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 rx=0x7f1f74006ab0 tx=0x7f1f7404daa0).stop 2021-02-10T11:35:59.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.464 7f1f8a48a700 1 -- 172.21.2.101:0/2216445192 shutdown_connections 2021-02-10T11:35:59.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.464 7f1f8a48a700 1 --2- 172.21.2.101:0/2216445192 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f6c003b70 0x7f1f6c005fa0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:59.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.464 7f1f8a48a700 1 -- 172.21.2.101:0/2216445192 >> 172.21.2.101:0/2216445192 conn(0x7f1f840ce450 msgr2=0x7f1f840d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:59.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.465 7f1f8a48a700 1 -- 172.21.2.101:0/2216445192 shutdown_connections 2021-02-10T11:35:59.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.465 7f1f8a48a700 1 -- 172.21.2.101:0/2216445192 wait complete. 2021-02-10T11:35:59.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.479 7f1f8a48a700 1 Processor -- start 2021-02-10T11:35:59.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.479 7f1f8a48a700 1 -- start start 2021-02-10T11:35:59.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.480 7f1f8a48a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f841942b0 0x7f1f840b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:59.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.480 7f1f8a48a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f84195070 0x7f1f84195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:59.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.480 7f1f8a48a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f840bb8e0 0x7f1f840bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:59.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.480 7f1f8a48a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f841946b0 con 0x7f1f84195070 2021-02-10T11:35:59.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.480 7f1f8a48a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f840b3b70 con 0x7f1f841942b0 2021-02-10T11:35:59.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.480 7f1f8a48a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f840b3d90 con 0x7f1f840bb8e0 2021-02-10T11:35:59.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.480 7f1f837fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f84195070 0x7f1f84195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:59.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.480 7f1f837fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f84195070 0x7f1f84195470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53526/0 (socket says 172.21.2.101:53526) 2021-02-10T11:35:59.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.480 7f1f837fe700 1 -- 172.21.2.101:0/3973527168 learned_addr learned my addr 172.21.2.101:0/3973527168 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:35:59.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.480 7f1f83fff700 1 --2- 172.21.2.101:0/3973527168 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f841942b0 0x7f1f840b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:59.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.480 7f1f88aa7700 1 --2- 172.21.2.101:0/3973527168 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f840bb8e0 0x7f1f840bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:59.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.481 7f1f837fe700 1 -- 172.21.2.101:0/3973527168 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f840bb8e0 msgr2=0x7f1f840bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:59.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.481 7f1f837fe700 1 --2- 172.21.2.101:0/3973527168 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f840bb8e0 0x7f1f840bbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:59.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.481 7f1f837fe700 1 -- 172.21.2.101:0/3973527168 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f841942b0 msgr2=0x7f1f840b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:59.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.481 7f1f837fe700 1 --2- 172.21.2.101:0/3973527168 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f841942b0 0x7f1f840b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:59.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.481 7f1f837fe700 1 -- 172.21.2.101:0/3973527168 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f841a49e0 con 0x7f1f84195070 2021-02-10T11:35:59.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.481 7f1f83fff700 1 --2- 172.21.2.101:0/3973527168 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f841942b0 0x7f1f840b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:35:59.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.481 7f1f837fe700 1 --2- 172.21.2.101:0/3973527168 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f84195070 0x7f1f84195470 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 rx=0x7f1f78009610 tx=0x7f1f780099b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:59.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.481 7f1f80ff9700 1 -- 172.21.2.101:0/3973527168 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f78007a70 con 0x7f1f84195070 2021-02-10T11:35:59.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.481 7f1f80ff9700 1 -- 172.21.2.101:0/3973527168 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f780043a0 con 0x7f1f84195070 2021-02-10T11:35:59.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.481 7f1f80ff9700 1 -- 172.21.2.101:0/3973527168 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f780047f0 con 0x7f1f84195070 2021-02-10T11:35:59.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.481 7f1f8a48a700 1 -- 172.21.2.101:0/3973527168 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f841a49e0 con 0x7f1f84195070 2021-02-10T11:35:59.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.481 7f1f8a48a700 1 -- 172.21.2.101:0/3973527168 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f841a49e0 con 0x7f1f84195070 2021-02-10T11:35:59.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.482 7f1f80ff9700 1 -- 172.21.2.101:0/3973527168 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1f78007a70 con 0x7f1f84195070 2021-02-10T11:35:59.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.482 7f1f80ff9700 1 --2- 172.21.2.101:0/3973527168 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f640186d0 0x7f1f6401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:35:59.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.482 7f1f80ff9700 1 -- 172.21.2.101:0/3973527168 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f1f780324b0 con 0x7f1f84195070 2021-02-10T11:35:59.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.482 7f1f83fff700 1 --2- 172.21.2.101:0/3973527168 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f640186d0 0x7f1f6401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:35:59.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.482 7f1f83fff700 1 --2- 172.21.2.101:0/3973527168 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f640186d0 0x7f1f6401abd0 crc :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:35:59.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.512 7f1f8a48a700 1 -- 172.21.2.101:0/3973527168 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f840bc5f0 con 0x7f1f84195070 2021-02-10T11:35:59.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.514 7f1f80ff9700 1 -- 172.21.2.101:0/3973527168 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1f780043a0 con 0x7f1f84195070 2021-02-10T11:35:59.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.637 7f1f8a48a700 1 -- 172.21.2.101:0/3973527168 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1f840bc5f0 con 0x7f1f84195070 2021-02-10T11:35:59.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.638 7f1f80ff9700 1 -- 172.21.2.101:0/3973527168 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v69) v1 ==== 84+0+64 (secure 0 0 0) 0x7f1f780043a0 con 0x7f1f84195070 2021-02-10T11:35:59.645 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:35:59.645 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:35:59.645 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:35:59.646 INFO:teuthology.orchestra.run.gibba001.stdout: "poolnum": 1, 2021-02-10T11:35:59.646 INFO:teuthology.orchestra.run.gibba001.stdout: "poolname": "rbd" 2021-02-10T11:35:59.646 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:35:59.646 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:35:59.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.639 7f1f8a48a700 1 -- 172.21.2.101:0/3973527168 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f640186d0 msgr2=0x7f1f6401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:59.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.639 7f1f8a48a700 1 --2- 172.21.2.101:0/3973527168 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f640186d0 0x7f1f6401abd0 crc :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:59.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.639 7f1f8a48a700 1 -- 172.21.2.101:0/3973527168 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f84195070 msgr2=0x7f1f84195470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:35:59.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.639 7f1f8a48a700 1 --2- 172.21.2.101:0/3973527168 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f84195070 0x7f1f84195470 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 rx=0x7f1f78009610 tx=0x7f1f780099b0).stop 2021-02-10T11:35:59.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.639 7f1f8a48a700 1 -- 172.21.2.101:0/3973527168 shutdown_connections 2021-02-10T11:35:59.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.639 7f1f8a48a700 1 --2- 172.21.2.101:0/3973527168 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f640186d0 0x7f1f6401abd0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:59.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.639 7f1f8a48a700 1 --2- 172.21.2.101:0/3973527168 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f840bb8e0 0x7f1f840bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:59.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.639 7f1f8a48a700 1 --2- 172.21.2.101:0/3973527168 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f84195070 0x7f1f84195470 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:59.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.639 7f1f8a48a700 1 --2- 172.21.2.101:0/3973527168 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f841942b0 0x7f1f840b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:35:59.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.639 7f1f8a48a700 1 -- 172.21.2.101:0/3973527168 >> 172.21.2.101:0/3973527168 conn(0x7f1f840b3040 msgr2=0x7f1f840c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:35:59.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.640 7f1f8a48a700 1 -- 172.21.2.101:0/3973527168 shutdown_connections 2021-02-10T11:35:59.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:35:59.640 7f1f8a48a700 1 -- 172.21.2.101:0/3973527168 wait complete. 2021-02-10T11:35:59.655 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-02-10T11:35:59.656 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-02-10T11:36:00.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.062 7fb5a4f49700 1 Processor -- start 2021-02-10T11:36:00.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.062 7fb5a4f49700 1 -- start start 2021-02-10T11:36:00.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.062 7fb5a4f49700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5a00d1c70 0x7fb5a00cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:00.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.063 7fb5a4f49700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb5a00cf580 0x7fb5a00cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:00.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.063 7fb5a4f49700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5a00fe1c0 con 0x7fb5a00d1870 2021-02-10T11:36:00.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.063 7fb5a4f49700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a01b0cb0 con 0x7fb5a00cf580 2021-02-10T11:36:00.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.063 7fb5a4f49700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a0109a60 con 0x7fb5a00d1c70 2021-02-10T11:36:00.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.063 7fb59f314700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb5a00cf580 0x7fb5a00cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:00.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.063 7fb59e312700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5a00d1c70 0x7fb5a00cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:00.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.063 7fb59eb13700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fb5a00d1870 0x7fb5a01b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:47438/0 (socket says 172.21.2.101:47438) 2021-02-10T11:36:00.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.064 7fb59e312700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5a00d1c70 0x7fb5a00cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:46266/0 (socket says 172.21.2.101:46266) 2021-02-10T11:36:00.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.064 7fb59f314700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb5a00cf580 0x7fb5a00cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:35790/0 (socket says 172.21.2.101:35790) 2021-02-10T11:36:00.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.064 7fb59f314700 1 -- 172.21.2.101:0/3119460585 learned_addr learned my addr 172.21.2.101:0/3119460585 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:00.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.064 7fb59f314700 1 -- 172.21.2.101:0/3119460585 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5a00d1c70 msgr2=0x7fb5a00cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:00.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.064 7fb59f314700 1 --2- 172.21.2.101:0/3119460585 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5a00d1c70 0x7fb5a00cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:00.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.064 7fb59f314700 1 -- 172.21.2.101:0/3119460585 >> v1:172.21.2.108:6789/0 conn(0x7fb5a00d1870 legacy=0x7fb5a01b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:00.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.064 7fb59db11700 1 -- 172.21.2.101:0/3119460585 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3607659289 0 0) 0x7fb590000c60 con 0x7fb5a00d1870 2021-02-10T11:36:00.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.064 7fb59f314700 1 -- 172.21.2.101:0/3119460585 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a0088070 con 0x7fb5a00cf580 2021-02-10T11:36:00.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.064 7fb59f314700 1 --2- 172.21.2.101:0/3119460585 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb5a00cf580 0x7fb5a00cf980 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7fb598006aa0 tx=0x7fb598009ea0).ready entity=mon.1 client_cookie=f8ef72fdfd1308e7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:00.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.065 7fb59db11700 1 -- 172.21.2.101:0/3119460585 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb59800b510 con 0x7fb5a00cf580 2021-02-10T11:36:00.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.065 7fb59db11700 1 -- 172.21.2.101:0/3119460585 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb598003030 con 0x7fb5a00cf580 2021-02-10T11:36:00.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.065 7fb59db11700 1 -- 172.21.2.101:0/3119460585 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb59800ba10 con 0x7fb5a00cf580 2021-02-10T11:36:00.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.065 7fb5a4f49700 1 -- 172.21.2.101:0/3119460585 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb5a00cf580 msgr2=0x7fb5a00cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:00.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.065 7fb5a4f49700 1 --2- 172.21.2.101:0/3119460585 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb5a00cf580 0x7fb5a00cf980 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7fb598006aa0 tx=0x7fb598009ea0).stop 2021-02-10T11:36:00.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.066 7fb5a4f49700 1 -- 172.21.2.101:0/3119460585 shutdown_connections 2021-02-10T11:36:00.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.066 7fb5a4f49700 1 --2- 172.21.2.101:0/3119460585 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5a00d1c70 0x7fb5a00cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:00.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.066 7fb5a4f49700 1 --2- 172.21.2.101:0/3119460585 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb5a00cf580 0x7fb5a00cf980 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:00.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.066 7fb5a4f49700 1 -- 172.21.2.101:0/3119460585 >> 172.21.2.101:0/3119460585 conn(0x7fb5a00ce420 msgr2=0x7fb5a00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:00.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.066 7fb5a4f49700 1 -- 172.21.2.101:0/3119460585 shutdown_connections 2021-02-10T11:36:00.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.066 7fb5a4f49700 1 -- 172.21.2.101:0/3119460585 wait complete. 2021-02-10T11:36:00.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.080 7fb5a4f49700 1 Processor -- start 2021-02-10T11:36:00.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.080 7fb5a4f49700 1 -- start start 2021-02-10T11:36:00.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.081 7fb5a4f49700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5a00926d0 0x7fb5a00b6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:00.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.081 7fb5a4f49700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb5a00c3840 0x7fb5a0093270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:00.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.081 7fb5a4f49700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5a00c3c40 0x7fb5a0093890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:00.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.081 7fb5a4f49700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a0092ad0 con 0x7fb5a00926d0 2021-02-10T11:36:00.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.081 7fb5a4f49700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a0092cf0 con 0x7fb5a00c3840 2021-02-10T11:36:00.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.081 7fb5a4f49700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a00b6f80 con 0x7fb5a00c3c40 2021-02-10T11:36:00.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.081 7fb59eb13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5a00926d0 0x7fb5a00b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:00.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.081 7fb59eb13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5a00926d0 0x7fb5a00b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53538/0 (socket says 172.21.2.101:53538) 2021-02-10T11:36:00.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.081 7fb59eb13700 1 -- 172.21.2.101:0/3919495511 learned_addr learned my addr 172.21.2.101:0/3919495511 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:00.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.081 7fb59eb13700 1 -- 172.21.2.101:0/3919495511 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5a00c3c40 msgr2=0x7fb5a0093890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:00.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.081 7fb59eb13700 1 --2- 172.21.2.101:0/3919495511 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5a00c3c40 0x7fb5a0093890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:00.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.081 7fb59eb13700 1 -- 172.21.2.101:0/3919495511 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb5a00c3840 msgr2=0x7fb5a0093270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:00.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.081 7fb59eb13700 1 --2- 172.21.2.101:0/3919495511 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb5a00c3840 0x7fb5a0093270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:00.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.081 7fb59eb13700 1 -- 172.21.2.101:0/3919495511 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a0093eb0 con 0x7fb5a00926d0 2021-02-10T11:36:00.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.082 7fb59eb13700 1 --2- 172.21.2.101:0/3919495511 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5a00926d0 0x7fb5a00b6820 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 rx=0x7fb59000a5d0 tx=0x7fb59000a970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:00.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.082 7fb5977fe700 1 -- 172.21.2.101:0/3919495511 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb590000c10 con 0x7fb5a00926d0 2021-02-10T11:36:00.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.082 7fb5977fe700 1 -- 172.21.2.101:0/3919495511 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5900070d0 con 0x7fb5a00926d0 2021-02-10T11:36:00.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.082 7fb5a4f49700 1 -- 172.21.2.101:0/3919495511 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a0093eb0 con 0x7fb5a00926d0 2021-02-10T11:36:00.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.082 7fb5977fe700 1 -- 172.21.2.101:0/3919495511 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb590004530 con 0x7fb5a00926d0 2021-02-10T11:36:00.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.082 7fb5a4f49700 1 -- 172.21.2.101:0/3919495511 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a0094440 con 0x7fb5a00926d0 2021-02-10T11:36:00.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.082 7fb5977fe700 1 -- 172.21.2.101:0/3919495511 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb590000c10 con 0x7fb5a00926d0 2021-02-10T11:36:00.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.082 7fb5977fe700 1 --2- 172.21.2.101:0/3919495511 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb584018730 0x7fb58401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:00.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.082 7fb5977fe700 1 -- 172.21.2.101:0/3919495511 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fb590032460 con 0x7fb5a00926d0 2021-02-10T11:36:00.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.083 7fb59e312700 1 --2- 172.21.2.101:0/3919495511 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb584018730 0x7fb58401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:00.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.083 7fb59e312700 1 --2- 172.21.2.101:0/3919495511 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb584018730 0x7fb58401abc0 crc :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:00.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.113 7fb5a4f49700 1 -- 172.21.2.101:0/3919495511 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5a0082a70 con 0x7fb5a00926d0 2021-02-10T11:36:00.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.115 7fb5977fe700 1 -- 172.21.2.101:0/3919495511 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb5900070d0 con 0x7fb5a00926d0 2021-02-10T11:36:00.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.243 7fb5a4f49700 1 -- 172.21.2.101:0/3919495511 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"} v 0) v1 -- 0x7fb5a0082a70 con 0x7fb5a00926d0 2021-02-10T11:36:00.501 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:36:00.504 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:36:00.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.667 7fb5977fe700 1 -- 172.21.2.101:0/3919495511 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"}]=0 pool 'cephfs_metadata' created v70) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb590000c10 con 0x7fb5a00926d0 2021-02-10T11:36:00.717 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' created 2021-02-10T11:36:00.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.711 7fb5a4f49700 1 -- 172.21.2.101:0/3919495511 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb584018730 msgr2=0x7fb58401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:00.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.711 7fb5a4f49700 1 --2- 172.21.2.101:0/3919495511 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb584018730 0x7fb58401abc0 crc :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:00.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.711 7fb5a4f49700 1 -- 172.21.2.101:0/3919495511 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5a00926d0 msgr2=0x7fb5a00b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:00.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.711 7fb5a4f49700 1 --2- 172.21.2.101:0/3919495511 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5a00926d0 0x7fb5a00b6820 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 rx=0x7fb59000a5d0 tx=0x7fb59000a970).stop 2021-02-10T11:36:00.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.711 7fb5a4f49700 1 -- 172.21.2.101:0/3919495511 shutdown_connections 2021-02-10T11:36:00.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.711 7fb5a4f49700 1 --2- 172.21.2.101:0/3919495511 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb584018730 0x7fb58401abc0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:00.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.711 7fb5a4f49700 1 --2- 172.21.2.101:0/3919495511 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb5a00c3c40 0x7fb5a0093890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:00.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.711 7fb5a4f49700 1 --2- 172.21.2.101:0/3919495511 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb5a00c3840 0x7fb5a0093270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:00.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.711 7fb5a4f49700 1 --2- 172.21.2.101:0/3919495511 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb5a00926d0 0x7fb5a00b6820 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:00.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.711 7fb5a4f49700 1 -- 172.21.2.101:0/3919495511 >> 172.21.2.101:0/3919495511 conn(0x7fb5a009ffd0 msgr2=0x7fb5a0087030 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:00.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.711 7fb5a4f49700 1 -- 172.21.2.101:0/3919495511 shutdown_connections 2021-02-10T11:36:00.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:00.711 7fb5a4f49700 1 -- 172.21.2.101:0/3919495511 wait complete. 2021-02-10T11:36:00.727 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_data 8 2021-02-10T11:36:01.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.132 7f22aa2f2700 1 Processor -- start 2021-02-10T11:36:01.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.132 7f22aa2f2700 1 -- start start 2021-02-10T11:36:01.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.132 7f22aa2f2700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f22a40d1870 0x7f22a41b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:01.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.132 7f22aa2f2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22a41b0ae0 0x7f22a4108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:01.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.132 7f22aa2f2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f22a40d1c70 0x7f22a4108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:01.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.132 7f22aa2f2700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a40fe1c0 con 0x7f22a40d1c70 2021-02-10T11:36:01.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.132 7f22aa2f2700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a4109a60 con 0x7f22a40d1870 2021-02-10T11:36:01.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.132 7f22aa2f2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a4088070 con 0x7f22a41b0ae0 2021-02-10T11:36:01.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.132 7f22a890f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f22a40d1c70 0x7f22a4108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:01.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.132 7f22a890f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f22a40d1c70 0x7f22a4108f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53546/0 (socket says 172.21.2.101:53546) 2021-02-10T11:36:01.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.132 7f22a890f700 1 -- 172.21.2.101:0/3154315310 learned_addr learned my addr 172.21.2.101:0/3154315310 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:01.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.133 7f22a890f700 1 -- 172.21.2.101:0/3154315310 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22a41b0ae0 msgr2=0x7f22a4108940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:36:01.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.133 7f22a890f700 1 --2- 172.21.2.101:0/3154315310 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22a41b0ae0 0x7f22a4108940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:01.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.133 7f22a890f700 1 -- 172.21.2.101:0/3154315310 >> v2:172.21.2.108:3300/0 conn(0x7f22a40d1870 msgr2=0x7f22a41b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:36:01.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.133 7f22a890f700 1 --2- 172.21.2.101:0/3154315310 >> v2:172.21.2.108:3300/0 conn(0x7f22a40d1870 0x7f22a41b04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:01.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.133 7f22a890f700 1 -- 172.21.2.101:0/3154315310 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a4109580 con 0x7f22a40d1c70 2021-02-10T11:36:01.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.133 7f22a890f700 1 --2- 172.21.2.101:0/3154315310 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22a40d1c70 0x7f22a4108f60 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 rx=0x7f229c006f00 tx=0x7f229c009f10).ready entity=mon.0 client_cookie=ad82fbb42eac9d33 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:01.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.133 7f22a2ffd700 1 -- 172.21.2.101:0/3154315310 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f229c00b3b0 con 0x7f22a40d1c70 2021-02-10T11:36:01.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.133 7f22a2ffd700 1 -- 172.21.2.101:0/3154315310 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229c00b630 con 0x7f22a40d1c70 2021-02-10T11:36:01.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.133 7f22a2ffd700 1 -- 172.21.2.101:0/3154315310 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f229c00bb50 con 0x7f22a40d1c70 2021-02-10T11:36:01.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.133 7f22aa2f2700 1 -- 172.21.2.101:0/3154315310 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22a40d1c70 msgr2=0x7f22a4108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:01.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.133 7f22aa2f2700 1 --2- 172.21.2.101:0/3154315310 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22a40d1c70 0x7f22a4108f60 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 rx=0x7f229c006f00 tx=0x7f229c009f10).stop 2021-02-10T11:36:01.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.133 7f22aa2f2700 1 -- 172.21.2.101:0/3154315310 shutdown_connections 2021-02-10T11:36:01.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.133 7f22aa2f2700 1 --2- 172.21.2.101:0/3154315310 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22a41b0ae0 0x7f22a4108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:01.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.133 7f22aa2f2700 1 --2- 172.21.2.101:0/3154315310 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22a40d1c70 0x7f22a4108f60 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:01.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.133 7f22aa2f2700 1 --2- 172.21.2.101:0/3154315310 >> v2:172.21.2.108:3300/0 conn(0x7f22a40d1870 0x7f22a41b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:01.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.133 7f22aa2f2700 1 -- 172.21.2.101:0/3154315310 >> 172.21.2.101:0/3154315310 conn(0x7f22a40ce420 msgr2=0x7f22a40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:01.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.133 7f22aa2f2700 1 -- 172.21.2.101:0/3154315310 shutdown_connections 2021-02-10T11:36:01.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.134 7f22aa2f2700 1 -- 172.21.2.101:0/3154315310 wait complete. 2021-02-10T11:36:01.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.148 7f22aa2f2700 1 Processor -- start 2021-02-10T11:36:01.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.148 7f22aa2f2700 1 -- start start 2021-02-10T11:36:01.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.148 7f22aa2f2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22a40c3820 0x7f22a4084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:01.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.148 7f22aa2f2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22a40a0290 0x7f22a40852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.148 7f22aa2f2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22a4086270 0x7f22a4086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.148 7f22aa2f2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a40c0700 con 0x7f22a40c3820 2021-02-10T11:36:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.148 7f22aa2f2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a40858f0 con 0x7f22a4086270 2021-02-10T11:36:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.148 7f22aa2f2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a4085b10 con 0x7f22a40a0290 2021-02-10T11:36:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.148 7f22a3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22a40c3820 0x7f22a4084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.148 7f22a3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22a40c3820 0x7f22a4084cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53556/0 (socket says 172.21.2.101:53556) 2021-02-10T11:36:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.148 7f22a3fff700 1 -- 172.21.2.101:0/3829384862 learned_addr learned my addr 172.21.2.101:0/3829384862 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.149 7f22a3fff700 1 -- 172.21.2.101:0/3829384862 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22a40a0290 msgr2=0x7f22a40852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.149 7f22a3fff700 1 --2- 172.21.2.101:0/3829384862 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22a40a0290 0x7f22a40852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.149 7f22a3fff700 1 -- 172.21.2.101:0/3829384862 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22a4086270 msgr2=0x7f22a4086670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.149 7f22a3fff700 1 --2- 172.21.2.101:0/3829384862 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22a4086270 0x7f22a4086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.149 7f22a3fff700 1 -- 172.21.2.101:0/3829384862 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a4086c90 con 0x7f22a40c3820 2021-02-10T11:36:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.149 7f22a3fff700 1 --2- 172.21.2.101:0/3829384862 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22a40c3820 0x7f22a4084cb0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 rx=0x7f229400b5c0 tx=0x7f229400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:01.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.149 7f22a0ff9700 1 -- 172.21.2.101:0/3829384862 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2294000c40 con 0x7f22a40c3820 2021-02-10T11:36:01.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.149 7f22a0ff9700 1 -- 172.21.2.101:0/3829384862 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229400be20 con 0x7f22a40c3820 2021-02-10T11:36:01.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.149 7f22a0ff9700 1 -- 172.21.2.101:0/3829384862 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2294006220 con 0x7f22a40c3820 2021-02-10T11:36:01.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.149 7f22aa2f2700 1 -- 172.21.2.101:0/3829384862 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a4086c90 con 0x7f22a40c3820 2021-02-10T11:36:01.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.149 7f22aa2f2700 1 -- 172.21.2.101:0/3829384862 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a41a2f00 con 0x7f22a40c3820 2021-02-10T11:36:01.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.150 7f22a0ff9700 1 -- 172.21.2.101:0/3829384862 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2294006030 con 0x7f22a40c3820 2021-02-10T11:36:01.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.150 7f22a0ff9700 1 --2- 172.21.2.101:0/3829384862 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22840186c0 0x7f228401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:01.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.150 7f22a0ff9700 1 -- 172.21.2.101:0/3829384862 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5107+0+0 (secure 0 0 0) 0x7f22940323a0 con 0x7f22a40c3820 2021-02-10T11:36:01.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.150 7f22a37fe700 1 --2- 172.21.2.101:0/3829384862 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22840186c0 0x7f228401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:01.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.150 7f22a37fe700 1 --2- 172.21.2.101:0/3829384862 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22840186c0 0x7f228401abc0 crc :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:01.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.182 7f22aa2f2700 1 -- 172.21.2.101:0/3829384862 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a4082a70 con 0x7f22a40c3820 2021-02-10T11:36:01.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.185 7f22a0ff9700 1 -- 172.21.2.101:0/3829384862 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2294006b80 con 0x7f22a40c3820 2021-02-10T11:36:01.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.315 7f22aa2f2700 1 -- 172.21.2.101:0/3829384862 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"} v 0) v1 -- 0x7f22a4082a70 con 0x7f22a40c3820 2021-02-10T11:36:01.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.675 7f22a0ff9700 1 -- 172.21.2.101:0/3829384862 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"}]=0 pool 'cephfs_data' created v72) v1 ==== 125+0+0 (secure 0 0 0) 0x7f2294006030 con 0x7f22a40c3820 2021-02-10T11:36:01.687 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' created 2021-02-10T11:36:01.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.682 7f22aa2f2700 1 -- 172.21.2.101:0/3829384862 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22840186c0 msgr2=0x7f228401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:01.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.682 7f22aa2f2700 1 --2- 172.21.2.101:0/3829384862 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22840186c0 0x7f228401abc0 crc :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:01.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.682 7f22aa2f2700 1 -- 172.21.2.101:0/3829384862 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22a40c3820 msgr2=0x7f22a4084cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:01.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.682 7f22aa2f2700 1 --2- 172.21.2.101:0/3829384862 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22a40c3820 0x7f22a4084cb0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 rx=0x7f229400b5c0 tx=0x7f229400b960).stop 2021-02-10T11:36:01.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.682 7f22aa2f2700 1 -- 172.21.2.101:0/3829384862 shutdown_connections 2021-02-10T11:36:01.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.682 7f22aa2f2700 1 --2- 172.21.2.101:0/3829384862 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22840186c0 0x7f228401abc0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:01.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.682 7f22aa2f2700 1 --2- 172.21.2.101:0/3829384862 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22a40a0290 0x7f22a40852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:01.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.682 7f22aa2f2700 1 --2- 172.21.2.101:0/3829384862 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22a4086270 0x7f22a4086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:01.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.682 7f22aa2f2700 1 --2- 172.21.2.101:0/3829384862 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22a40c3820 0x7f22a4084cb0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:01.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.682 7f22aa2f2700 1 -- 172.21.2.101:0/3829384862 >> 172.21.2.101:0/3829384862 conn(0x7f22a40872c0 msgr2=0x7f22a4095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:01.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.682 7f22aa2f2700 1 -- 172.21.2.101:0/3829384862 shutdown_connections 2021-02-10T11:36:01.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:01.682 7f22aa2f2700 1 -- 172.21.2.101:0/3829384862 wait complete. 2021-02-10T11:36:01.698 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-02-10T11:36:02.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.104 7f36e8924700 1 Processor -- start 2021-02-10T11:36:02.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.104 7f36e8924700 1 -- start start 2021-02-10T11:36:02.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.105 7f36e8924700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f36e40d1880 0x7f36e41b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:02.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.105 7f36e8924700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e4109a70 con 0x7f36e40cff80 2021-02-10T11:36:02.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.105 7f36e8924700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e4088070 con 0x7f36e41b0af0 2021-02-10T11:36:02.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.105 7f36e8924700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e40e5f00 con 0x7f36e40d1880 2021-02-10T11:36:02.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.105 7f36e2e1d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f36e40cff80 0x7f36e4109140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54056/0 (socket says 172.21.2.101:54056) 2021-02-10T11:36:02.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.105 7f36e2e1d700 1 -- 172.21.2.101:0/3849207896 learned_addr learned my addr 172.21.2.101:0/3849207896 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:02.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.105 7f36e161a700 1 -- 172.21.2.101:0/3849207896 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3913911579 0 0) 0x7f36dc001500 con 0x7f36e40cff80 2021-02-10T11:36:02.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.105 7f36e161a700 1 -- 172.21.2.101:0/3849207896 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36d0002fe0 con 0x7f36e40cff80 2021-02-10T11:36:02.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.106 7f36e161a700 1 -- 172.21.2.101:0/3849207896 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3516394787 0 0) 0x7f36dc001500 con 0x7f36e40cff80 2021-02-10T11:36:02.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.106 7f36e161a700 1 -- 172.21.2.101:0/3849207896 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f36e40d1880 msgr2=0x7f36e41b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:02.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.106 7f36e161a700 1 --2- 172.21.2.101:0/3849207896 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f36e40d1880 0x7f36e41b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:02.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.106 7f36e161a700 1 -- 172.21.2.101:0/3849207896 >> v1:172.21.2.108:6789/0 conn(0x7f36e41b0af0 legacy=0x7f36e4108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:02.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.106 7f36e161a700 1 -- 172.21.2.101:0/3849207896 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e40d1c80 con 0x7f36e40cff80 2021-02-10T11:36:02.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.106 7f36e161a700 1 -- 172.21.2.101:0/3849207896 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f36dc001a00 con 0x7f36e40cff80 2021-02-10T11:36:02.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.106 7f36e161a700 1 -- 172.21.2.101:0/3849207896 >> v1:172.21.2.101:6789/0 conn(0x7f36e40cff80 legacy=0x7f36e4109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:02.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.106 7f36e161a700 1 --2- 172.21.2.101:0/3849207896 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f36d0003b90 0x7f36d0006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:02.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.106 7f36e161a700 1 --2- 172.21.2.101:0/3849207896 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36d00067b0 0x7f36d0008cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:02.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.106 7f36e161a700 1 --2- 172.21.2.101:0/3849207896 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f36d00095d0 0x7f36d000ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:02.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.106 7f36e161a700 1 -- 172.21.2.101:0/3849207896 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d000c2d0 con 0x7f36d00067b0 2021-02-10T11:36:02.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.106 7f36e161a700 1 -- 172.21.2.101:0/3849207896 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d000c4f0 con 0x7f36d0003b90 2021-02-10T11:36:02.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.106 7f36e161a700 1 -- 172.21.2.101:0/3849207896 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d000c710 con 0x7f36d00095d0 2021-02-10T11:36:02.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.106 7f36e1e1b700 1 --2- 172.21.2.101:0/3849207896 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36d00067b0 0x7f36d0008cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:02.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.106 7f36e261c700 1 --2- 172.21.2.101:0/3849207896 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f36d0003b90 0x7f36d0006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:02.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.106 7f36e2e1d700 1 --2- 172.21.2.101:0/3849207896 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f36d00095d0 0x7f36d000ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:02.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.106 7f36e1e1b700 1 -- 172.21.2.101:0/3849207896 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f36d00095d0 msgr2=0x7f36d000ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:02.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.106 7f36e1e1b700 1 --2- 172.21.2.101:0/3849207896 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f36d00095d0 0x7f36d000ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:02.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.106 7f36e1e1b700 1 -- 172.21.2.101:0/3849207896 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f36d0003b90 msgr2=0x7f36d0006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:02.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.106 7f36e1e1b700 1 --2- 172.21.2.101:0/3849207896 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f36d0003b90 0x7f36d0006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:02.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.106 7f36e1e1b700 1 -- 172.21.2.101:0/3849207896 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d000c9b0 con 0x7f36d00067b0 2021-02-10T11:36:02.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.106 7f36e1e1b700 1 --2- 172.21.2.101:0/3849207896 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36d00067b0 0x7f36d0008cd0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 rx=0x7f36d4006ab0 tx=0x7f36d404fbc0).ready entity=mon.0 client_cookie=7a66c51b7893d23f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:02.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.107 7f36e161a700 1 -- 172.21.2.101:0/3849207896 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f36d4006240 con 0x7f36d00067b0 2021-02-10T11:36:02.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.107 7f36e161a700 1 -- 172.21.2.101:0/3849207896 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d4006240 con 0x7f36d00067b0 2021-02-10T11:36:02.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.107 7f36e161a700 1 -- 172.21.2.101:0/3849207896 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f36d40066d0 con 0x7f36d00067b0 2021-02-10T11:36:02.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.107 7f36e8924700 1 -- 172.21.2.101:0/3849207896 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36d00067b0 msgr2=0x7f36d0008cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:02.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.107 7f36e8924700 1 --2- 172.21.2.101:0/3849207896 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36d00067b0 0x7f36d0008cd0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 rx=0x7f36d4006ab0 tx=0x7f36d404fbc0).stop 2021-02-10T11:36:02.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.107 7f36e2e1d700 1 -- 172.21.2.101:0/3849207896 reap_dead start 2021-02-10T11:36:02.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.108 7f36e8924700 1 -- 172.21.2.101:0/3849207896 shutdown_connections 2021-02-10T11:36:02.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.108 7f36e8924700 1 -- 172.21.2.101:0/3849207896 >> 172.21.2.101:0/3849207896 conn(0x7f36e40ce430 msgr2=0x7f36e40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:02.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.108 7f36e8924700 1 -- 172.21.2.101:0/3849207896 shutdown_connections 2021-02-10T11:36:02.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.108 7f36e8924700 1 -- 172.21.2.101:0/3849207896 wait complete. 2021-02-10T11:36:02.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.122 7f36e8924700 1 Processor -- start 2021-02-10T11:36:02.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.122 7f36e8924700 1 -- start start 2021-02-10T11:36:02.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.123 7f36e8924700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f36e40913d0 0x7f36e41a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:02.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.123 7f36e8924700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36e41a3c30 0x7f36e40b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:02.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.123 7f36e8924700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f36e40b1790 0x7f36e40b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:02.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.123 7f36e8924700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e4091850 con 0x7f36e41a3c30 2021-02-10T11:36:02.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.123 7f36e8924700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e40b21b0 con 0x7f36e40913d0 2021-02-10T11:36:02.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.123 7f36e8924700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e40b23d0 con 0x7f36e40b1790 2021-02-10T11:36:02.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.123 7f36e1e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36e41a3c30 0x7f36e40b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:02.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.123 7f36e1e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36e41a3c30 0x7f36e40b1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53574/0 (socket says 172.21.2.101:53574) 2021-02-10T11:36:02.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.123 7f36e1e1b700 1 -- 172.21.2.101:0/2450466555 learned_addr learned my addr 172.21.2.101:0/2450466555 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:02.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.123 7f36e1e1b700 1 -- 172.21.2.101:0/2450466555 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f36e40b1790 msgr2=0x7f36e40b1b90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:02.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.123 7f36e1e1b700 1 --2- 172.21.2.101:0/2450466555 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f36e40b1790 0x7f36e40b1b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:02.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.123 7f36e1e1b700 1 -- 172.21.2.101:0/2450466555 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f36e40913d0 msgr2=0x7f36e41a3610 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:02.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.123 7f36e1e1b700 1 --2- 172.21.2.101:0/2450466555 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f36e40913d0 0x7f36e41a3610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:02.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.123 7f36e1e1b700 1 -- 172.21.2.101:0/2450466555 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e40af2a0 con 0x7f36e41a3c30 2021-02-10T11:36:02.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.123 7f36e1e1b700 1 --2- 172.21.2.101:0/2450466555 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36e41a3c30 0x7f36e40b1170 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 rx=0x7f36d4006ef0 tx=0x7f36d4050940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:02.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.124 7f36daffd700 1 -- 172.21.2.101:0/2450466555 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f36d40009b0 con 0x7f36e41a3c30 2021-02-10T11:36:02.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.124 7f36daffd700 1 -- 172.21.2.101:0/2450466555 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d4050d30 con 0x7f36e41a3c30 2021-02-10T11:36:02.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.124 7f36daffd700 1 -- 172.21.2.101:0/2450466555 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f36d40092c0 con 0x7f36e41a3c30 2021-02-10T11:36:02.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.124 7f36e8924700 1 -- 172.21.2.101:0/2450466555 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e40af2a0 con 0x7f36e41a3c30 2021-02-10T11:36:02.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.124 7f36e8924700 1 -- 172.21.2.101:0/2450466555 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e40af830 con 0x7f36e41a3c30 2021-02-10T11:36:02.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.124 7f36daffd700 1 -- 172.21.2.101:0/2450466555 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f36d4001030 con 0x7f36e41a3c30 2021-02-10T11:36:02.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.124 7f36daffd700 1 --2- 172.21.2.101:0/2450466555 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36c80186c0 0x7f36c801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:02.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.124 7f36daffd700 1 -- 172.21.2.101:0/2450466555 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5457+0+0 (secure 0 0 0) 0x7f36d4076530 con 0x7f36e41a3c30 2021-02-10T11:36:02.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.125 7f36e261c700 1 --2- 172.21.2.101:0/2450466555 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36c80186c0 0x7f36c801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:02.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.125 7f36e261c700 1 --2- 172.21.2.101:0/2450466555 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36c80186c0 0x7f36c801abc0 crc :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:02.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.155 7f36e8924700 1 -- 172.21.2.101:0/2450466555 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e4082aa0 con 0x7f36e41a3c30 2021-02-10T11:36:02.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.158 7f36daffd700 1 -- 172.21.2.101:0/2450466555 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f36d4069fd0 con 0x7f36e41a3c30 2021-02-10T11:36:02.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.285 7f36e8924700 1 -- 172.21.2.101:0/2450466555 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"} v 0) v1 -- 0x7f36e4082aa0 con 0x7f36e41a3c30 2021-02-10T11:36:02.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.289 7f36daffd700 1 -- 172.21.2.101:0/2450466555 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"}]=0 new fs with metadata pool 14 and data pool 15 v52) v1 ==== 189+0+0 (secure 0 0 0) 0x7f36d4001030 con 0x7f36e41a3c30 2021-02-10T11:36:02.300 INFO:teuthology.orchestra.run.gibba001.stderr:new fs with metadata pool 14 and data pool 15 2021-02-10T11:36:02.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.295 7f36e8924700 1 -- 172.21.2.101:0/2450466555 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36c80186c0 msgr2=0x7f36c801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:02.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.295 7f36e8924700 1 --2- 172.21.2.101:0/2450466555 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36c80186c0 0x7f36c801abc0 crc :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:02.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.295 7f36e8924700 1 -- 172.21.2.101:0/2450466555 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36e41a3c30 msgr2=0x7f36e40b1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:02.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.295 7f36e8924700 1 --2- 172.21.2.101:0/2450466555 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36e41a3c30 0x7f36e40b1170 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 rx=0x7f36d4006ef0 tx=0x7f36d4050940).stop 2021-02-10T11:36:02.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.295 7f36e8924700 1 -- 172.21.2.101:0/2450466555 shutdown_connections 2021-02-10T11:36:02.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.295 7f36e8924700 1 --2- 172.21.2.101:0/2450466555 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f36c80186c0 0x7f36c801abc0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:02.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.295 7f36e8924700 1 --2- 172.21.2.101:0/2450466555 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f36e40b1790 0x7f36e40b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:02.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.295 7f36e8924700 1 --2- 172.21.2.101:0/2450466555 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36e41a3c30 0x7f36e40b1170 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:02.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.295 7f36e8924700 1 --2- 172.21.2.101:0/2450466555 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f36e40913d0 0x7f36e41a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:02.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.295 7f36e8924700 1 -- 172.21.2.101:0/2450466555 >> 172.21.2.101:0/2450466555 conn(0x7f36e4087360 msgr2=0x7f36e40b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:02.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.295 7f36e8924700 1 -- 172.21.2.101:0/2450466555 shutdown_connections 2021-02-10T11:36:02.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.295 7f36e8924700 1 -- 172.21.2.101:0/2450466555 wait complete. 2021-02-10T11:36:02.311 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:36:02.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.715 7fc66cc9a700 1 Processor -- start 2021-02-10T11:36:02.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.715 7fc66cc9a700 1 -- start start 2021-02-10T11:36:02.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.716 7fc66cc9a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc6680d1870 0x7fc6681b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:02.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.716 7fc66cc9a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6680f0b10 con 0x7fc6680f0650 2021-02-10T11:36:02.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.716 7fc66cc9a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6680d0310 con 0x7fc6681b0a10 2021-02-10T11:36:02.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.716 7fc66cc9a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6680d0530 con 0x7fc6680d1870 2021-02-10T11:36:02.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.716 7fc66705f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc6680f0650 0x7fc6680cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54084/0 (socket says 172.21.2.101:54084) 2021-02-10T11:36:02.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.716 7fc66705f700 1 -- 172.21.2.101:0/1217145317 learned_addr learned my addr 172.21.2.101:0/1217145317 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:02.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.716 7fc66585c700 1 -- 172.21.2.101:0/1217145317 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1222079633 0 0) 0x7fc660001500 con 0x7fc6680f0650 2021-02-10T11:36:02.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.716 7fc66585c700 1 -- 172.21.2.101:0/1217145317 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc654002fe0 con 0x7fc6680f0650 2021-02-10T11:36:02.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66585c700 1 -- 172.21.2.101:0/1217145317 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 551389179 0 0) 0x7fc660001500 con 0x7fc6680f0650 2021-02-10T11:36:02.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66585c700 1 -- 172.21.2.101:0/1217145317 >> v2:172.21.2.108:3300/0 conn(0x7fc6680d1870 msgr2=0x7fc6681b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:02.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66585c700 1 --2- 172.21.2.101:0/1217145317 >> v2:172.21.2.108:3300/0 conn(0x7fc6680d1870 0x7fc6681b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:02.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66585c700 1 -- 172.21.2.101:0/1217145317 >> v1:172.21.2.108:6789/0 conn(0x7fc6681b0a10 legacy=0x7fc6680cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:02.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66585c700 1 -- 172.21.2.101:0/1217145317 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6680d1c70 con 0x7fc6680f0650 2021-02-10T11:36:02.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66585c700 1 -- 172.21.2.101:0/1217145317 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fc660001a00 con 0x7fc6680f0650 2021-02-10T11:36:02.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66585c700 1 -- 172.21.2.101:0/1217145317 >> v1:172.21.2.101:6789/0 conn(0x7fc6680f0650 legacy=0x7fc6680cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:02.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66585c700 1 --2- 172.21.2.101:0/1217145317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc654003af0 0x7fc654005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:02.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66585c700 1 --2- 172.21.2.101:0/1217145317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6540067b0 0x7fc654008cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:02.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66585c700 1 --2- 172.21.2.101:0/1217145317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6540095b0 0x7fc65400bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:02.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66585c700 1 -- 172.21.2.101:0/1217145317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65400c300 con 0x7fc6540095b0 2021-02-10T11:36:02.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66585c700 1 -- 172.21.2.101:0/1217145317 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65400c520 con 0x7fc6540067b0 2021-02-10T11:36:02.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66585c700 1 -- 172.21.2.101:0/1217145317 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65400c740 con 0x7fc654003af0 2021-02-10T11:36:02.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66705f700 1 --2- 172.21.2.101:0/1217145317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6540095b0 0x7fc65400bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:02.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66685e700 1 --2- 172.21.2.101:0/1217145317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc654003af0 0x7fc654005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:02.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66605d700 1 --2- 172.21.2.101:0/1217145317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6540067b0 0x7fc654008cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:02.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66705f700 1 -- 172.21.2.101:0/1217145317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc654003af0 msgr2=0x7fc654005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:02.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66705f700 1 --2- 172.21.2.101:0/1217145317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc654003af0 0x7fc654005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:02.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66705f700 1 -- 172.21.2.101:0/1217145317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6540067b0 msgr2=0x7fc654008cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:02.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66705f700 1 --2- 172.21.2.101:0/1217145317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6540067b0 0x7fc654008cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:02.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66705f700 1 -- 172.21.2.101:0/1217145317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65400c9e0 con 0x7fc6540095b0 2021-02-10T11:36:02.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66685e700 1 --2- 172.21.2.101:0/1217145317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc654003af0 0x7fc654005fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:36:02.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66705f700 1 -- 172.21.2.101:0/1217145317 reap_dead start 2021-02-10T11:36:02.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.717 7fc66705f700 1 --2- 172.21.2.101:0/1217145317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6540095b0 0x7fc65400bab0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 rx=0x7fc660007a70 tx=0x7fc660050310).ready entity=mon.0 client_cookie=f985b4780c0364c5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:02.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.718 7fc66585c700 1 -- 172.21.2.101:0/1217145317 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc660004720 con 0x7fc6540095b0 2021-02-10T11:36:02.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.718 7fc66585c700 1 -- 172.21.2.101:0/1217145317 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc660004aa0 con 0x7fc6540095b0 2021-02-10T11:36:02.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.718 7fc66585c700 1 -- 172.21.2.101:0/1217145317 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc660050700 con 0x7fc6540095b0 2021-02-10T11:36:02.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.718 7fc66cc9a700 1 -- 172.21.2.101:0/1217145317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6540095b0 msgr2=0x7fc65400bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:02.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.718 7fc66cc9a700 1 --2- 172.21.2.101:0/1217145317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6540095b0 0x7fc65400bab0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 rx=0x7fc660007a70 tx=0x7fc660050310).stop 2021-02-10T11:36:02.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.718 7fc66cc9a700 1 -- 172.21.2.101:0/1217145317 shutdown_connections 2021-02-10T11:36:02.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.718 7fc66cc9a700 1 --2- 172.21.2.101:0/1217145317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6540095b0 0x7fc65400bab0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:02.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.718 7fc66cc9a700 1 -- 172.21.2.101:0/1217145317 >> 172.21.2.101:0/1217145317 conn(0x7fc6680ce450 msgr2=0x7fc6680d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:02.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.718 7fc66cc9a700 1 -- 172.21.2.101:0/1217145317 shutdown_connections 2021-02-10T11:36:02.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.718 7fc66cc9a700 1 -- 172.21.2.101:0/1217145317 wait complete. 2021-02-10T11:36:02.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.733 7fc66cc9a700 1 Processor -- start 2021-02-10T11:36:02.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.733 7fc66cc9a700 1 -- start start 2021-02-10T11:36:02.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.733 7fc66cc9a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6680ba6d0 0x7fc6680bb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:02.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.733 7fc66cc9a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6680b0590 0x7fc6680b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:02.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.733 7fc66cc9a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6680b0fb0 0x7fc6681a7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:02.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.733 7fc66cc9a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6680baf30 con 0x7fc6680b0fb0 2021-02-10T11:36:02.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.733 7fc66cc9a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6680b0010 con 0x7fc6680b0590 2021-02-10T11:36:02.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.733 7fc66cc9a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6680baad0 con 0x7fc6680ba6d0 2021-02-10T11:36:02.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.734 7fc66705f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6680b0fb0 0x7fc6681a7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:02.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.734 7fc66705f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6680b0fb0 0x7fc6681a7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53602/0 (socket says 172.21.2.101:53602) 2021-02-10T11:36:02.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.734 7fc66705f700 1 -- 172.21.2.101:0/1308919106 learned_addr learned my addr 172.21.2.101:0/1308919106 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:02.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.734 7fc66705f700 1 -- 172.21.2.101:0/1308919106 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6680ba6d0 msgr2=0x7fc6680bb200 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:02.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.734 7fc66705f700 1 --2- 172.21.2.101:0/1308919106 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6680ba6d0 0x7fc6680bb200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:02.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.734 7fc66705f700 1 -- 172.21.2.101:0/1308919106 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6680b0590 msgr2=0x7fc6680b0990 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:02.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.734 7fc66705f700 1 --2- 172.21.2.101:0/1308919106 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6680b0590 0x7fc6680b0990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:02.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.734 7fc66705f700 1 -- 172.21.2.101:0/1308919106 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6681a7ab0 con 0x7fc6680b0fb0 2021-02-10T11:36:02.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.734 7fc66705f700 1 --2- 172.21.2.101:0/1308919106 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6680b0fb0 0x7fc6681a7490 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 rx=0x7fc660007fc0 tx=0x7fc660051ca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:02.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.734 7fc65f7fe700 1 -- 172.21.2.101:0/1308919106 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc660007030 con 0x7fc6680b0fb0 2021-02-10T11:36:02.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.734 7fc65f7fe700 1 -- 172.21.2.101:0/1308919106 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc660007330 con 0x7fc6680b0fb0 2021-02-10T11:36:02.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.734 7fc66cc9a700 1 -- 172.21.2.101:0/1308919106 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6681a7ab0 con 0x7fc6680b0fb0 2021-02-10T11:36:02.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.734 7fc66cc9a700 1 -- 172.21.2.101:0/1308919106 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6681a8040 con 0x7fc6680b0fb0 2021-02-10T11:36:02.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.734 7fc65f7fe700 1 -- 172.21.2.101:0/1308919106 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc66000a0c0 con 0x7fc6680b0fb0 2021-02-10T11:36:02.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.735 7fc65f7fe700 1 -- 172.21.2.101:0/1308919106 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc660001670 con 0x7fc6680b0fb0 2021-02-10T11:36:02.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.735 7fc65f7fe700 1 --2- 172.21.2.101:0/1308919106 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc64c018730 0x7fc64c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:02.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.735 7fc65f7fe700 1 -- 172.21.2.101:0/1308919106 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc6600769f0 con 0x7fc6680b0fb0 2021-02-10T11:36:02.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.735 7fc66685e700 1 --2- 172.21.2.101:0/1308919106 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc64c018730 0x7fc64c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:02.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.735 7fc66685e700 1 --2- 172.21.2.101:0/1308919106 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc64c018730 0x7fc64c01ac10 crc :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:02.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.766 7fc66cc9a700 1 -- 172.21.2.101:0/1308919106 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6680f0b10 con 0x7fc6680b0fb0 2021-02-10T11:36:02.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.769 7fc65f7fe700 1 -- 172.21.2.101:0/1308919106 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc660000f40 con 0x7fc6680b0fb0 2021-02-10T11:36:02.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.894 7fc66cc9a700 1 -- 172.21.2.101:0/1308919106 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6680f0b10 con 0x7fc6680b0fb0 2021-02-10T11:36:02.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.895 7fc65f7fe700 1 -- 172.21.2.101:0/1308919106 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+14009 (secure 0 0 0) 0x7fc660001670 con 0x7fc6680b0fb0 2021-02-10T11:36:02.902 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:36:02.903 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":73,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:36:02.287104","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:36:00.261115","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2021-02-10 11:36:01.333230","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:36:02.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.897 7fc66cc9a700 1 -- 172.21.2.101:0/1308919106 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc64c018730 msgr2=0x7fc64c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:02.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.897 7fc66cc9a700 1 --2- 172.21.2.101:0/1308919106 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc64c018730 0x7fc64c01ac10 crc :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:02.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.897 7fc66cc9a700 1 -- 172.21.2.101:0/1308919106 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6680b0fb0 msgr2=0x7fc6681a7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:02.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.897 7fc66cc9a700 1 --2- 172.21.2.101:0/1308919106 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6680b0fb0 0x7fc6681a7490 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 rx=0x7fc660007fc0 tx=0x7fc660051ca0).stop 2021-02-10T11:36:02.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.897 7fc66cc9a700 1 -- 172.21.2.101:0/1308919106 shutdown_connections 2021-02-10T11:36:02.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.897 7fc66cc9a700 1 --2- 172.21.2.101:0/1308919106 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6680b0fb0 0x7fc6681a7490 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:02.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.897 7fc66cc9a700 1 --2- 172.21.2.101:0/1308919106 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6680b0590 0x7fc6680b0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:02.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.897 7fc66cc9a700 1 --2- 172.21.2.101:0/1308919106 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc64c018730 0x7fc64c01ac10 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:02.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.897 7fc66cc9a700 1 --2- 172.21.2.101:0/1308919106 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6680ba6d0 0x7fc6680bb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:02.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.897 7fc66cc9a700 1 -- 172.21.2.101:0/1308919106 >> 172.21.2.101:0/1308919106 conn(0x7fc66809eb60 msgr2=0x7fc6680a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:02.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.897 7fc66cc9a700 1 -- 172.21.2.101:0/1308919106 shutdown_connections 2021-02-10T11:36:02.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:02.897 7fc66cc9a700 1 -- 172.21.2.101:0/1308919106 wait complete. 2021-02-10T11:36:02.913 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:36:03.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.319 7fecbf68d700 1 Processor -- start 2021-02-10T11:36:03.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.320 7fecbf68d700 1 -- start start 2021-02-10T11:36:03.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.320 7fecbf68d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fecb80d1870 0x7fecb81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:03.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.320 7fecbf68d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fecb80f0650 0x7fecb80cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:03.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.320 7fecbf68d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb80f0b10 con 0x7fecb81b0a10 2021-02-10T11:36:03.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.320 7fecbf68d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb80e5ed0 con 0x7fecb80d1870 2021-02-10T11:36:03.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.320 7fecbf68d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb80d1c70 con 0x7fecb80f0650 2021-02-10T11:36:03.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.320 7fecbd4a9700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fecb80d1870 0x7fecb81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:03.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.320 7fecbcca8700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fecb81b0a10 0x7fecb80cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54100/0 (socket says 172.21.2.101:54100) 2021-02-10T11:36:03.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.320 7fecbd4a9700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fecb80d1870 0x7fecb81b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53606/0 (socket says 172.21.2.101:53606) 2021-02-10T11:36:03.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.320 7fecbcca8700 1 -- 172.21.2.101:0/3501686297 learned_addr learned my addr 172.21.2.101:0/3501686297 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:03.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.320 7fecbdcaa700 1 --2- 172.21.2.101:0/3501686297 >> v2:172.21.2.108:3300/0 conn(0x7fecb80f0650 0x7fecb80cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:03.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.320 7fecbd4a9700 1 -- 172.21.2.101:0/3501686297 >> v2:172.21.2.108:3300/0 conn(0x7fecb80f0650 msgr2=0x7fecb80cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:03.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.320 7fecbd4a9700 1 --2- 172.21.2.101:0/3501686297 >> v2:172.21.2.108:3300/0 conn(0x7fecb80f0650 0x7fecb80cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:03.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.320 7fecbd4a9700 1 -- 172.21.2.101:0/3501686297 >> v1:172.21.2.101:6789/0 conn(0x7fecb81b0a10 legacy=0x7fecb80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:03.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.321 7fecbd4a9700 1 -- 172.21.2.101:0/3501686297 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb80d1e90 con 0x7fecb80d1870 2021-02-10T11:36:03.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.321 7fecbd4a9700 1 --2- 172.21.2.101:0/3501686297 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fecb80d1870 0x7fecb81b03f0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 rx=0x7fecb4006650 tx=0x7fecb4009a00).ready entity=mon.0 client_cookie=a5ed9a460ca321a0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:03.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.321 7fecabfff700 1 -- 172.21.2.101:0/3501686297 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fecb40009b0 con 0x7fecb80d1870 2021-02-10T11:36:03.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.321 7fecabfff700 1 -- 172.21.2.101:0/3501686297 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb400c0c0 con 0x7fecb80d1870 2021-02-10T11:36:03.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.321 7fecabfff700 1 -- 172.21.2.101:0/3501686297 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fecb400c5a0 con 0x7fecb80d1870 2021-02-10T11:36:03.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.321 7fecbf68d700 1 -- 172.21.2.101:0/3501686297 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fecb80d1870 msgr2=0x7fecb81b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:03.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.321 7fecbf68d700 1 --2- 172.21.2.101:0/3501686297 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fecb80d1870 0x7fecb81b03f0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 rx=0x7fecb4006650 tx=0x7fecb4009a00).stop 2021-02-10T11:36:03.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.321 7fecbf68d700 1 -- 172.21.2.101:0/3501686297 shutdown_connections 2021-02-10T11:36:03.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.321 7fecbf68d700 1 --2- 172.21.2.101:0/3501686297 >> v2:172.21.2.108:3300/0 conn(0x7fecb80f0650 0x7fecb80cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:03.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.321 7fecbf68d700 1 --2- 172.21.2.101:0/3501686297 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fecb80d1870 0x7fecb81b03f0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:03.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.321 7fecbf68d700 1 -- 172.21.2.101:0/3501686297 >> 172.21.2.101:0/3501686297 conn(0x7fecb80ce450 msgr2=0x7fecb80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:03.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.322 7fecbf68d700 1 -- 172.21.2.101:0/3501686297 shutdown_connections 2021-02-10T11:36:03.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.322 7fecbf68d700 1 -- 172.21.2.101:0/3501686297 wait complete. 2021-02-10T11:36:03.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.336 7fecbf68d700 1 Processor -- start 2021-02-10T11:36:03.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.336 7fecbf68d700 1 -- start start 2021-02-10T11:36:03.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.337 7fecbf68d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fecb80ada90 0x7fecb80ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:03.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.337 7fecbf68d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fecb8099d20 0x7fecb809a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:03.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.337 7fecbf68d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fecb809a740 0x7fecb8097bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:03.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.337 7fecbf68d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb80997a0 con 0x7fecb8099d20 2021-02-10T11:36:03.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.337 7fecbf68d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb818f0c0 con 0x7fecb80ada90 2021-02-10T11:36:03.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.337 7fecbf68d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb809ab40 con 0x7fecb809a740 2021-02-10T11:36:03.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.337 7fecbcca8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fecb8099d20 0x7fecb809a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:03.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.337 7fecbcca8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fecb8099d20 0x7fecb809a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53612/0 (socket says 172.21.2.101:53612) 2021-02-10T11:36:03.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.337 7fecbcca8700 1 -- 172.21.2.101:0/246025592 learned_addr learned my addr 172.21.2.101:0/246025592 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:03.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.337 7fecbdcaa700 1 --2- 172.21.2.101:0/246025592 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fecb809a740 0x7fecb8097bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:03.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.337 7fecbd4a9700 1 --2- 172.21.2.101:0/246025592 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fecb80ada90 0x7fecb80ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:03.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.337 7fecbcca8700 1 -- 172.21.2.101:0/246025592 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fecb809a740 msgr2=0x7fecb8097bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:03.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.337 7fecbcca8700 1 --2- 172.21.2.101:0/246025592 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fecb809a740 0x7fecb8097bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:03.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.337 7fecbcca8700 1 -- 172.21.2.101:0/246025592 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fecb80ada90 msgr2=0x7fecb80ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:03.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.337 7fecbcca8700 1 --2- 172.21.2.101:0/246025592 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fecb80ada90 0x7fecb80ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:03.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.337 7fecbcca8700 1 -- 172.21.2.101:0/246025592 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb809ad60 con 0x7fecb8099d20 2021-02-10T11:36:03.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.337 7fecbcca8700 1 --2- 172.21.2.101:0/246025592 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fecb8099d20 0x7fecb809a120 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 rx=0x7fecac00a4c0 tx=0x7fecac00a860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:03.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.338 7feca9ffb700 1 -- 172.21.2.101:0/246025592 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fecac0010b0 con 0x7fecb8099d20 2021-02-10T11:36:03.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.338 7feca9ffb700 1 -- 172.21.2.101:0/246025592 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecac00ac50 con 0x7fecb8099d20 2021-02-10T11:36:03.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.338 7feca9ffb700 1 -- 172.21.2.101:0/246025592 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fecac0045d0 con 0x7fecb8099d20 2021-02-10T11:36:03.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.338 7fecbf68d700 1 -- 172.21.2.101:0/246025592 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb809ad60 con 0x7fecb8099d20 2021-02-10T11:36:03.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.338 7fecbf68d700 1 -- 172.21.2.101:0/246025592 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb80984d0 con 0x7fecb8099d20 2021-02-10T11:36:03.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.338 7feca9ffb700 1 -- 172.21.2.101:0/246025592 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fecac001050 con 0x7fecb8099d20 2021-02-10T11:36:03.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.338 7feca9ffb700 1 --2- 172.21.2.101:0/246025592 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fec980186c0 0x7fec9801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:03.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.338 7feca9ffb700 1 -- 172.21.2.101:0/246025592 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fecac032550 con 0x7fecb8099d20 2021-02-10T11:36:03.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.338 7fecbd4a9700 1 --2- 172.21.2.101:0/246025592 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fec980186c0 0x7fec9801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:03.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.339 7fecbd4a9700 1 --2- 172.21.2.101:0/246025592 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fec980186c0 0x7fec9801abc0 crc :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:03.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.370 7fecbf68d700 1 -- 172.21.2.101:0/246025592 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb818f070 con 0x7fecb8099d20 2021-02-10T11:36:03.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.372 7feca9ffb700 1 -- 172.21.2.101:0/246025592 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fecac024ec0 con 0x7fecb8099d20 2021-02-10T11:36:03.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.497 7fecbf68d700 1 -- 172.21.2.101:0/246025592 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fecb818f070 con 0x7fecb8099d20 2021-02-10T11:36:03.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.498 7feca9ffb700 1 -- 172.21.2.101:0/246025592 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+13996 (secure 0 0 0) 0x7fecac024ec0 con 0x7fecb8099d20 2021-02-10T11:36:03.505 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:36:03.505 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":74,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:36:03.289219","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:36:00.261115","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2021-02-10 11:36:01.333230","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:36:03.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.499 7fecbf68d700 1 -- 172.21.2.101:0/246025592 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fec980186c0 msgr2=0x7fec9801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:03.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.499 7fecbf68d700 1 --2- 172.21.2.101:0/246025592 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fec980186c0 0x7fec9801abc0 crc :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:03.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.499 7fecbf68d700 1 -- 172.21.2.101:0/246025592 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fecb8099d20 msgr2=0x7fecb809a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:03.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.499 7fecbf68d700 1 --2- 172.21.2.101:0/246025592 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fecb8099d20 0x7fecb809a120 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 rx=0x7fecac00a4c0 tx=0x7fecac00a860).stop 2021-02-10T11:36:03.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.499 7fecbf68d700 1 -- 172.21.2.101:0/246025592 shutdown_connections 2021-02-10T11:36:03.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.499 7fecbf68d700 1 --2- 172.21.2.101:0/246025592 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fec980186c0 0x7fec9801abc0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:03.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.499 7fecbf68d700 1 --2- 172.21.2.101:0/246025592 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fecb809a740 0x7fecb8097bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:03.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.499 7fecbf68d700 1 --2- 172.21.2.101:0/246025592 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fecb8099d20 0x7fecb809a120 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:03.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.499 7fecbf68d700 1 --2- 172.21.2.101:0/246025592 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fecb80ada90 0x7fecb80ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:03.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.499 7fecbf68d700 1 -- 172.21.2.101:0/246025592 >> 172.21.2.101:0/246025592 conn(0x7fecb80f0550 msgr2=0x7fecb8191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:03.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.500 7fecbf68d700 1 -- 172.21.2.101:0/246025592 shutdown_connections 2021-02-10T11:36:03.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.500 7fecbf68d700 1 -- 172.21.2.101:0/246025592 wait complete. 2021-02-10T11:36:03.516 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-02-10T11:36:03.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.923 7f21d59b9700 1 Processor -- start 2021-02-10T11:36:03.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.923 7f21d59b9700 1 -- start start 2021-02-10T11:36:03.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.924 7f21d59b9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21d00d1870 0x7f21d01b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:03.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.924 7f21d59b9700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f21d01b0ae0 0x7f21d0108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:03.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.924 7f21d59b9700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d0109a60 con 0x7f21d00d1c70 2021-02-10T11:36:03.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.924 7f21d59b9700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d00e5ef0 con 0x7f21d01b0ae0 2021-02-10T11:36:03.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.924 7f21d59b9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d0109750 con 0x7f21d00d1870 2021-02-10T11:36:03.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.924 7f21ced73700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f21d01b0ae0 0x7f21d0108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:03.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.924 7f21ced73700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f21d01b0ae0 0x7f21d0108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53618/0 (socket says 172.21.2.101:53618) 2021-02-10T11:36:03.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.924 7f21ced73700 1 -- 172.21.2.101:0/1326261117 learned_addr learned my addr 172.21.2.101:0/1326261117 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:03.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.924 7f21cf574700 1 --2- 172.21.2.101:0/1326261117 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21d00d1870 0x7f21d01b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:03.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.924 7f21ced73700 1 -- 172.21.2.101:0/1326261117 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21d00d1870 msgr2=0x7f21d01b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:03.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.924 7f21ced73700 1 --2- 172.21.2.101:0/1326261117 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21d00d1870 0x7f21d01b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:03.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.924 7f21ced73700 1 -- 172.21.2.101:0/1326261117 >> v1:172.21.2.108:6789/0 conn(0x7f21d00d1c70 legacy=0x7f21d0108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:03.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.924 7f21ced73700 1 -- 172.21.2.101:0/1326261117 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d0088070 con 0x7f21d01b0ae0 2021-02-10T11:36:03.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.924 7f21cf574700 1 --2- 172.21.2.101:0/1326261117 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21d00d1870 0x7f21d01b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:36:03.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.924 7f21ced73700 1 --2- 172.21.2.101:0/1326261117 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21d01b0ae0 0x7f21d0108940 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 rx=0x7f21b8006700 tx=0x7f21b8009b80).ready entity=mon.0 client_cookie=e379947d7824e03f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:03.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.925 7f21ce572700 1 -- 172.21.2.101:0/1326261117 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21b800c0c0 con 0x7f21d01b0ae0 2021-02-10T11:36:03.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.925 7f21ce572700 1 -- 172.21.2.101:0/1326261117 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b800c440 con 0x7f21d01b0ae0 2021-02-10T11:36:03.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.925 7f21ce572700 1 -- 172.21.2.101:0/1326261117 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21b800c8a0 con 0x7f21d01b0ae0 2021-02-10T11:36:03.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.925 7f21d59b9700 1 -- 172.21.2.101:0/1326261117 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21d01b0ae0 msgr2=0x7f21d0108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:03.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.925 7f21d59b9700 1 --2- 172.21.2.101:0/1326261117 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21d01b0ae0 0x7f21d0108940 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 rx=0x7f21b8006700 tx=0x7f21b8009b80).stop 2021-02-10T11:36:03.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.925 7f21d59b9700 1 -- 172.21.2.101:0/1326261117 shutdown_connections 2021-02-10T11:36:03.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.925 7f21d59b9700 1 --2- 172.21.2.101:0/1326261117 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21d01b0ae0 0x7f21d0108940 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:03.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.925 7f21d59b9700 1 --2- 172.21.2.101:0/1326261117 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21d00d1870 0x7f21d01b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:03.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.925 7f21d59b9700 1 -- 172.21.2.101:0/1326261117 >> 172.21.2.101:0/1326261117 conn(0x7f21d00ce420 msgr2=0x7f21d00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:03.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.925 7f21d59b9700 1 -- 172.21.2.101:0/1326261117 shutdown_connections 2021-02-10T11:36:03.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.925 7f21d59b9700 1 -- 172.21.2.101:0/1326261117 wait complete. 2021-02-10T11:36:03.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.940 7f21d59b9700 1 Processor -- start 2021-02-10T11:36:03.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.940 7f21d59b9700 1 -- start start 2021-02-10T11:36:03.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.940 7f21d59b9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21d00b1ba0 0x7f21d01a78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:03.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.940 7f21d59b9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21d01a7ef0 0x7f21d00a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:03.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.940 7f21d59b9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21d00a7060 0x7f21d00b2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:03.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.940 7f21d59b9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d0190c20 con 0x7f21d01a7ef0 2021-02-10T11:36:03.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.940 7f21d59b9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d0190710 con 0x7f21d00a7060 2021-02-10T11:36:03.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.940 7f21d59b9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d00b1fa0 con 0x7f21d00b1ba0 2021-02-10T11:36:03.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.941 7f21ced73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21d01a7ef0 0x7f21d00a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:03.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.941 7f21cf574700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21d00b1ba0 0x7f21d01a78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:03.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.941 7f21ced73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21d01a7ef0 0x7f21d00a6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53624/0 (socket says 172.21.2.101:53624) 2021-02-10T11:36:03.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.941 7f21ced73700 1 -- 172.21.2.101:0/1421413152 learned_addr learned my addr 172.21.2.101:0/1421413152 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:03.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.941 7f21cfd75700 1 --2- 172.21.2.101:0/1421413152 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21d00a7060 0x7f21d00b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:03.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.941 7f21ced73700 1 -- 172.21.2.101:0/1421413152 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21d00b1ba0 msgr2=0x7f21d01a78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:03.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.941 7f21ced73700 1 --2- 172.21.2.101:0/1421413152 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21d00b1ba0 0x7f21d01a78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:03.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.941 7f21ced73700 1 -- 172.21.2.101:0/1421413152 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21d00a7060 msgr2=0x7f21d00b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:03.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.941 7f21ced73700 1 --2- 172.21.2.101:0/1421413152 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21d00a7060 0x7f21d00b2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:03.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.941 7f21ced73700 1 -- 172.21.2.101:0/1421413152 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d00a7460 con 0x7f21d01a7ef0 2021-02-10T11:36:03.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.941 7f21ced73700 1 --2- 172.21.2.101:0/1421413152 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21d01a7ef0 0x7f21d00a6a40 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 rx=0x7f21b800b580 tx=0x7f21b800b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:03.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.941 7f21c7fff700 1 -- 172.21.2.101:0/1421413152 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21b800bd10 con 0x7f21d01a7ef0 2021-02-10T11:36:03.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.941 7f21c7fff700 1 -- 172.21.2.101:0/1421413152 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b8000c90 con 0x7f21d01a7ef0 2021-02-10T11:36:03.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.941 7f21d59b9700 1 -- 172.21.2.101:0/1421413152 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d00a7460 con 0x7f21d01a7ef0 2021-02-10T11:36:03.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.941 7f21c7fff700 1 -- 172.21.2.101:0/1421413152 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21b80022b0 con 0x7f21d01a7ef0 2021-02-10T11:36:03.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.941 7f21d59b9700 1 -- 172.21.2.101:0/1421413152 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d00a7460 con 0x7f21d01a7ef0 2021-02-10T11:36:03.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.942 7f21c7fff700 1 -- 172.21.2.101:0/1421413152 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f21b8009f90 con 0x7f21d01a7ef0 2021-02-10T11:36:03.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.942 7f21c7fff700 1 --2- 172.21.2.101:0/1421413152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21b40186d0 0x7f21b401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:03.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.942 7f21c7fff700 1 -- 172.21.2.101:0/1421413152 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f21b80325e0 con 0x7f21d01a7ef0 2021-02-10T11:36:03.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.942 7f21cf574700 1 --2- 172.21.2.101:0/1421413152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21b40186d0 0x7f21b401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:03.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.943 7f21cf574700 1 --2- 172.21.2.101:0/1421413152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21b40186d0 0x7f21b401abd0 crc :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:03.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.974 7f21d59b9700 1 -- 172.21.2.101:0/1421413152 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d0082980 con 0x7f21d01a7ef0 2021-02-10T11:36:03.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:03.977 7f21c7fff700 1 -- 172.21.2.101:0/1421413152 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f21b80091e0 con 0x7f21d01a7ef0 2021-02-10T11:36:04.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.112 7f21d59b9700 1 -- 172.21.2.101:0/1421413152 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"} v 0) v1 -- 0x7f21d0109a60 con 0x7f21d01a7ef0 2021-02-10T11:36:04.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.297 7f21c7fff700 1 -- 172.21.2.101:0/1421413152 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"}]=0 v55) v1 ==== 118+0+0 (secure 0 0 0) 0x7f21b8009f90 con 0x7f21d01a7ef0 2021-02-10T11:36:04.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.328 7f21d59b9700 1 -- 172.21.2.101:0/1421413152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21b40186d0 msgr2=0x7f21b401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:04.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.328 7f21d59b9700 1 --2- 172.21.2.101:0/1421413152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21b40186d0 0x7f21b401abd0 crc :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:04.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.328 7f21d59b9700 1 -- 172.21.2.101:0/1421413152 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21d01a7ef0 msgr2=0x7f21d00a6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:04.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.328 7f21d59b9700 1 --2- 172.21.2.101:0/1421413152 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21d01a7ef0 0x7f21d00a6a40 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 rx=0x7f21b800b580 tx=0x7f21b800b920).stop 2021-02-10T11:36:04.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.328 7f21d59b9700 1 -- 172.21.2.101:0/1421413152 shutdown_connections 2021-02-10T11:36:04.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.328 7f21d59b9700 1 --2- 172.21.2.101:0/1421413152 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21d00a7060 0x7f21d00b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:04.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.328 7f21d59b9700 1 --2- 172.21.2.101:0/1421413152 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21d01a7ef0 0x7f21d00a6a40 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:04.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.328 7f21d59b9700 1 --2- 172.21.2.101:0/1421413152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21b40186d0 0x7f21b401abd0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:04.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.328 7f21d59b9700 1 --2- 172.21.2.101:0/1421413152 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21d00b1ba0 0x7f21d01a78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:04.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.328 7f21d59b9700 1 -- 172.21.2.101:0/1421413152 >> 172.21.2.101:0/1421413152 conn(0x7f21d00b5dc0 msgr2=0x7f21d0095960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:04.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.328 7f21d59b9700 1 -- 172.21.2.101:0/1421413152 shutdown_connections 2021-02-10T11:36:04.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.328 7f21d59b9700 1 -- 172.21.2.101:0/1421413152 wait complete. 2021-02-10T11:36:04.344 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:36:04.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.747 7f43a1411700 1 Processor -- start 2021-02-10T11:36:04.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.747 7f43a1411700 1 -- start start 2021-02-10T11:36:04.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.747 7f43a1411700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f439c1b0470 0x7f439c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:04.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.747 7f43a1411700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f439c0ef610 0x7f439c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:04.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.747 7f43a1411700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f439c0f0b50 con 0x7f439c0f0030 2021-02-10T11:36:04.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.747 7f43a1411700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f439c0e52e0 con 0x7f439c0ef610 2021-02-10T11:36:04.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.747 7f43a1411700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f439c0e5490 con 0x7f439c1b0470 2021-02-10T11:36:04.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.747 7f439a87c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f439c0ef610 0x7f439c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:04.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.747 7f439a87c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f439c0ef610 0x7f439c0efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53632/0 (socket says 172.21.2.101:53632) 2021-02-10T11:36:04.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.747 7f439a87c700 1 -- 172.21.2.101:0/2138912351 learned_addr learned my addr 172.21.2.101:0/2138912351 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:04.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.748 7f439a87c700 1 -- 172.21.2.101:0/2138912351 >> v2:172.21.2.108:3300/0 conn(0x7f439c1b0470 msgr2=0x7f439c1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:04.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.748 7f439a87c700 1 --2- 172.21.2.101:0/2138912351 >> v2:172.21.2.108:3300/0 conn(0x7f439c1b0470 0x7f439c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:04.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.748 7f439a87c700 1 -- 172.21.2.101:0/2138912351 >> v1:172.21.2.108:6789/0 conn(0x7f439c0f0030 legacy=0x7f439c0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:04.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.748 7f439a87c700 1 -- 172.21.2.101:0/2138912351 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439c0f0430 con 0x7f439c0ef610 2021-02-10T11:36:04.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.748 7f439a87c700 1 --2- 172.21.2.101:0/2138912351 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f439c0ef610 0x7f439c0efa10 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 rx=0x7f438c006700 tx=0x7f438c009b80).ready entity=mon.0 client_cookie=82e05a81a32cbfb9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:04.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.748 7f439a07b700 1 -- 172.21.2.101:0/2138912351 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f438c00c0c0 con 0x7f439c0ef610 2021-02-10T11:36:04.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.748 7f439a07b700 1 -- 172.21.2.101:0/2138912351 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f438c00c3c0 con 0x7f439c0ef610 2021-02-10T11:36:04.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.748 7f439a07b700 1 -- 172.21.2.101:0/2138912351 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f438c00c8e0 con 0x7f439c0ef610 2021-02-10T11:36:04.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.748 7f43a1411700 1 -- 172.21.2.101:0/2138912351 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f439c0ef610 msgr2=0x7f439c0efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:04.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.748 7f43a1411700 1 --2- 172.21.2.101:0/2138912351 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f439c0ef610 0x7f439c0efa10 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 rx=0x7f438c006700 tx=0x7f438c009b80).stop 2021-02-10T11:36:04.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.749 7f43a1411700 1 -- 172.21.2.101:0/2138912351 shutdown_connections 2021-02-10T11:36:04.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.749 7f43a1411700 1 --2- 172.21.2.101:0/2138912351 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f439c0ef610 0x7f439c0efa10 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:04.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.749 7f43a1411700 1 --2- 172.21.2.101:0/2138912351 >> v2:172.21.2.108:3300/0 conn(0x7f439c1b0470 0x7f439c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:04.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.749 7f43a1411700 1 -- 172.21.2.101:0/2138912351 >> 172.21.2.101:0/2138912351 conn(0x7f439c0ce3e0 msgr2=0x7f439c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:04.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.749 7f43a1411700 1 -- 172.21.2.101:0/2138912351 shutdown_connections 2021-02-10T11:36:04.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.749 7f43a1411700 1 -- 172.21.2.101:0/2138912351 wait complete. 2021-02-10T11:36:04.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.764 7f43a1411700 1 Processor -- start 2021-02-10T11:36:04.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.764 7f43a1411700 1 -- start start 2021-02-10T11:36:04.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.764 7f43a1411700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f439c1a2600 0x7f439c09ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:04.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.764 7f43a1411700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f439c09f580 0x7f439c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:04.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.764 7f43a1411700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f439c19d340 0x7f439c19d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:04.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.764 7f43a1411700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439c1a2d60 con 0x7f439c09f580 2021-02-10T11:36:04.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.764 7f43a1411700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439c1a2a00 con 0x7f439c19d340 2021-02-10T11:36:04.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.764 7f43a1411700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f439c09f980 con 0x7f439c1a2600 2021-02-10T11:36:04.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.764 7f439a87c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f439c09f580 0x7f439c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:04.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.764 7f439a87c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f439c09f580 0x7f439c19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53638/0 (socket says 172.21.2.101:53638) 2021-02-10T11:36:04.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.764 7f439a87c700 1 -- 172.21.2.101:0/3940747916 learned_addr learned my addr 172.21.2.101:0/3940747916 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:04.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.765 7f439a87c700 1 -- 172.21.2.101:0/3940747916 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f439c1a2600 msgr2=0x7f439c09ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:04.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.765 7f439a87c700 1 --2- 172.21.2.101:0/3940747916 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f439c1a2600 0x7f439c09ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:04.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.765 7f439a87c700 1 -- 172.21.2.101:0/3940747916 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f439c19d340 msgr2=0x7f439c19d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:04.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.765 7f439a87c700 1 --2- 172.21.2.101:0/3940747916 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f439c19d340 0x7f439c19d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:04.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.765 7f439a87c700 1 -- 172.21.2.101:0/3940747916 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439c19c470 con 0x7f439c09f580 2021-02-10T11:36:04.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.765 7f439a87c700 1 --2- 172.21.2.101:0/3940747916 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f439c09f580 0x7f439c19cd20 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 rx=0x7f438c00b650 tx=0x7f438c00b9f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:04.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.765 7f4393fff700 1 -- 172.21.2.101:0/3940747916 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f438c0090b0 con 0x7f439c09f580 2021-02-10T11:36:04.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.765 7f4393fff700 1 -- 172.21.2.101:0/3940747916 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f438c00bde0 con 0x7f439c09f580 2021-02-10T11:36:04.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.765 7f4393fff700 1 -- 172.21.2.101:0/3940747916 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f438c002220 con 0x7f439c09f580 2021-02-10T11:36:04.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.765 7f43a1411700 1 -- 172.21.2.101:0/3940747916 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f439c19c470 con 0x7f439c09f580 2021-02-10T11:36:04.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.765 7f43a1411700 1 -- 172.21.2.101:0/3940747916 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f439c19e050 con 0x7f439c09f580 2021-02-10T11:36:04.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.766 7f4393fff700 1 -- 172.21.2.101:0/3940747916 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f438c000c90 con 0x7f439c09f580 2021-02-10T11:36:04.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.766 7f4393fff700 1 --2- 172.21.2.101:0/3940747916 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f43800186c0 0x7f438001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:04.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.766 7f4393fff700 1 -- 172.21.2.101:0/3940747916 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f438c032560 con 0x7f439c09f580 2021-02-10T11:36:04.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.766 7f439b07d700 1 --2- 172.21.2.101:0/3940747916 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f43800186c0 0x7f438001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:04.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.766 7f439b07d700 1 --2- 172.21.2.101:0/3940747916 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f43800186c0 0x7f438001abc0 crc :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:04.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.797 7f43a1411700 1 -- 172.21.2.101:0/3940747916 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f439c1a2d40 con 0x7f439c09f580 2021-02-10T11:36:04.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.800 7f4393fff700 1 -- 172.21.2.101:0/3940747916 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f438c0064d0 con 0x7f439c09f580 2021-02-10T11:36:04.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.927 7f43a1411700 1 -- 172.21.2.101:0/3940747916 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f439c1a2d40 con 0x7f439c09f580 2021-02-10T11:36:04.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.927 7f4393fff700 1 -- 172.21.2.101:0/3940747916 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f438c000c90 con 0x7f439c09f580 2021-02-10T11:36:04.935 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 55 2021-02-10T11:36:04.935 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:36:04.935 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":55,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/3016216705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3016216705},{"type":"v1","addr":"172.21.2.108:6833","nonce":3016216705}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":51}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:02.287195","modified":"2021-02-10 11:36:04.295896","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5900},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5900":{"gid":5900,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.2.101:6835/2977571942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2977571942},{"type":"v1","addr":"172.21.2.101:6835","nonce":2977571942}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-02-10T11:36:04.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.929 7f43a1411700 1 -- 172.21.2.101:0/3940747916 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f43800186c0 msgr2=0x7f438001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:04.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.929 7f43a1411700 1 --2- 172.21.2.101:0/3940747916 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f43800186c0 0x7f438001abc0 crc :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:04.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.929 7f43a1411700 1 -- 172.21.2.101:0/3940747916 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f439c09f580 msgr2=0x7f439c19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:04.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.929 7f43a1411700 1 --2- 172.21.2.101:0/3940747916 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f439c09f580 0x7f439c19cd20 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 rx=0x7f438c00b650 tx=0x7f438c00b9f0).stop 2021-02-10T11:36:04.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.929 7f43a1411700 1 -- 172.21.2.101:0/3940747916 shutdown_connections 2021-02-10T11:36:04.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.929 7f43a1411700 1 --2- 172.21.2.101:0/3940747916 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f439c19d340 0x7f439c19d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:04.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.929 7f43a1411700 1 --2- 172.21.2.101:0/3940747916 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f439c09f580 0x7f439c19cd20 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:04.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.929 7f43a1411700 1 --2- 172.21.2.101:0/3940747916 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f43800186c0 0x7f438001abc0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:04.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.929 7f43a1411700 1 --2- 172.21.2.101:0/3940747916 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f439c1a2600 0x7f439c09ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:04.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.929 7f43a1411700 1 -- 172.21.2.101:0/3940747916 >> 172.21.2.101:0/3940747916 conn(0x7f439c09e730 msgr2=0x7f439c19eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:04.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.929 7f43a1411700 1 -- 172.21.2.101:0/3940747916 shutdown_connections 2021-02-10T11:36:04.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:04.929 7f43a1411700 1 -- 172.21.2.101:0/3940747916 wait complete. 2021-02-10T11:36:04.945 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:36:05.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.350 7f53059fb700 1 Processor -- start 2021-02-10T11:36:05.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.350 7f53059fb700 1 -- start start 2021-02-10T11:36:05.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.350 7f53059fb700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f53000d1870 0x7f53001b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:05.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.350 7f53059fb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53001b0a10 0x7f53000cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:05.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.350 7f53059fb700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53000f0b10 con 0x7f53000d1c70 2021-02-10T11:36:05.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.350 7f53059fb700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53000e5ed0 con 0x7f53000d1870 2021-02-10T11:36:05.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.350 7f53059fb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53000f0650 con 0x7f53001b0a10 2021-02-10T11:36:05.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.351 7f52feffd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53001b0a10 0x7f53000cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:05.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.351 7f52feffd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53001b0a10 0x7f53000cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:46376/0 (socket says 172.21.2.101:46376) 2021-02-10T11:36:05.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.351 7f52feffd700 1 -- 172.21.2.101:0/2537190624 learned_addr learned my addr 172.21.2.101:0/2537190624 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:05.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.351 7f52ff7fe700 1 --2- 172.21.2.101:0/2537190624 >> v2:172.21.2.108:3300/0 conn(0x7f53000d1870 0x7f53001b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:05.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.351 7f52feffd700 1 -- 172.21.2.101:0/2537190624 >> v2:172.21.2.108:3300/0 conn(0x7f53000d1870 msgr2=0x7f53001b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:05.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.351 7f52feffd700 1 --2- 172.21.2.101:0/2537190624 >> v2:172.21.2.108:3300/0 conn(0x7f53000d1870 0x7f53001b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:05.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.351 7f52feffd700 1 -- 172.21.2.101:0/2537190624 >> v1:172.21.2.108:6789/0 conn(0x7f53000d1c70 legacy=0x7f53000cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:05.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.351 7f52feffd700 1 -- 172.21.2.101:0/2537190624 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53000f0870 con 0x7f53001b0a10 2021-02-10T11:36:05.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.351 7f52ff7fe700 1 --2- 172.21.2.101:0/2537190624 >> v2:172.21.2.108:3300/0 conn(0x7f53000d1870 0x7f53001b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:36:05.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.351 7f52feffd700 1 --2- 172.21.2.101:0/2537190624 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53001b0a10 0x7f53000cf330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f52f0006700 tx=0x7f52f0009b80).ready entity=mon.2 client_cookie=fd4236a0971c0375 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:05.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.351 7f52fe7fc700 1 -- 172.21.2.101:0/2537190624 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f52f000c0c0 con 0x7f53001b0a10 2021-02-10T11:36:05.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.351 7f52fe7fc700 1 -- 172.21.2.101:0/2537190624 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f000c440 con 0x7f53001b0a10 2021-02-10T11:36:05.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.351 7f52fe7fc700 1 -- 172.21.2.101:0/2537190624 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f52f000c8a0 con 0x7f53001b0a10 2021-02-10T11:36:05.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.351 7f53059fb700 1 -- 172.21.2.101:0/2537190624 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53001b0a10 msgr2=0x7f53000cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:05.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.352 7f53059fb700 1 --2- 172.21.2.101:0/2537190624 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53001b0a10 0x7f53000cf330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f52f0006700 tx=0x7f52f0009b80).stop 2021-02-10T11:36:05.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.352 7f53059fb700 1 -- 172.21.2.101:0/2537190624 shutdown_connections 2021-02-10T11:36:05.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.352 7f53059fb700 1 --2- 172.21.2.101:0/2537190624 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53001b0a10 0x7f53000cf330 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:05.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.352 7f53059fb700 1 --2- 172.21.2.101:0/2537190624 >> v2:172.21.2.108:3300/0 conn(0x7f53000d1870 0x7f53001b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:05.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.352 7f53059fb700 1 -- 172.21.2.101:0/2537190624 >> 172.21.2.101:0/2537190624 conn(0x7f53000ce450 msgr2=0x7f53000d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:05.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.352 7f53059fb700 1 -- 172.21.2.101:0/2537190624 shutdown_connections 2021-02-10T11:36:05.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.352 7f53059fb700 1 -- 172.21.2.101:0/2537190624 wait complete. 2021-02-10T11:36:05.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.366 7f53059fb700 1 Processor -- start 2021-02-10T11:36:05.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.367 7f53059fb700 1 -- start start 2021-02-10T11:36:05.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.367 7f53059fb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53000970b0 0x7f53000ebe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:05.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.367 7f53059fb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5300098810 0x7f5300098c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:05.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.367 7f53059fb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5300099230 0x7f53000b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:05.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.367 7f53059fb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53000ec6d0 con 0x7f5300099230 2021-02-10T11:36:05.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.367 7f53059fb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53000974b0 con 0x7f53000970b0 2021-02-10T11:36:05.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.367 7f53059fb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53000b1ee0 con 0x7f5300098810 2021-02-10T11:36:05.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.367 7f52fffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5300099230 0x7f53000b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:05.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.367 7f52fffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5300099230 0x7f53000b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53650/0 (socket says 172.21.2.101:53650) 2021-02-10T11:36:05.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.367 7f52fffff700 1 -- 172.21.2.101:0/1253395098 learned_addr learned my addr 172.21.2.101:0/1253395098 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:05.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.367 7f52fffff700 1 -- 172.21.2.101:0/1253395098 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5300098810 msgr2=0x7f5300098c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:05.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.368 7f52fffff700 1 --2- 172.21.2.101:0/1253395098 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5300098810 0x7f5300098c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:05.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.368 7f52fffff700 1 -- 172.21.2.101:0/1253395098 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53000970b0 msgr2=0x7f53000ebe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:05.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.368 7f52fffff700 1 --2- 172.21.2.101:0/1253395098 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53000970b0 0x7f53000ebe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:05.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.368 7f52fffff700 1 -- 172.21.2.101:0/1253395098 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53000b2100 con 0x7f5300099230 2021-02-10T11:36:05.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.368 7f52fffff700 1 --2- 172.21.2.101:0/1253395098 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5300099230 0x7f53000b18c0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 rx=0x7f52f8009940 tx=0x7f52f800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:05.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.368 7f52f7fff700 1 -- 172.21.2.101:0/1253395098 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f52f8000ce0 con 0x7f5300099230 2021-02-10T11:36:05.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.368 7f52f7fff700 1 -- 172.21.2.101:0/1253395098 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f800c2f0 con 0x7f5300099230 2021-02-10T11:36:05.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.368 7f53059fb700 1 -- 172.21.2.101:0/1253395098 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53000b2100 con 0x7f5300099230 2021-02-10T11:36:05.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.368 7f52f7fff700 1 -- 172.21.2.101:0/1253395098 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f52f800c750 con 0x7f5300099230 2021-02-10T11:36:05.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.368 7f53059fb700 1 -- 172.21.2.101:0/1253395098 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53000b2100 con 0x7f5300099230 2021-02-10T11:36:05.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.368 7f52f7fff700 1 -- 172.21.2.101:0/1253395098 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f52f8001220 con 0x7f5300099230 2021-02-10T11:36:05.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.369 7f52f7fff700 1 --2- 172.21.2.101:0/1253395098 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52e40186c0 0x7f52e401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:05.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.369 7f52f7fff700 1 -- 172.21.2.101:0/1253395098 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f52f8032910 con 0x7f5300099230 2021-02-10T11:36:05.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.369 7f52ff7fe700 1 --2- 172.21.2.101:0/1253395098 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52e40186c0 0x7f52e401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:05.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.369 7f52ff7fe700 1 --2- 172.21.2.101:0/1253395098 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52e40186c0 0x7f52e401abc0 crc :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:05.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.400 7f53059fb700 1 -- 172.21.2.101:0/1253395098 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53000ebc00 con 0x7f5300099230 2021-02-10T11:36:05.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.403 7f52f7fff700 1 -- 172.21.2.101:0/1253395098 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f52f8000f90 con 0x7f5300099230 2021-02-10T11:36:05.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.526 7f53059fb700 1 -- 172.21.2.101:0/1253395098 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53000f0b10 con 0x7f5300099230 2021-02-10T11:36:05.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.527 7f52f7fff700 1 -- 172.21.2.101:0/1253395098 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+13996 (secure 0 0 0) 0x7f52f8001220 con 0x7f5300099230 2021-02-10T11:36:05.534 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:36:05.534 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":74,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:36:03.289219","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:36:00.261115","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2021-02-10 11:36:01.333230","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:36:05.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.528 7f53059fb700 1 -- 172.21.2.101:0/1253395098 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52e40186c0 msgr2=0x7f52e401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:05.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.528 7f53059fb700 1 --2- 172.21.2.101:0/1253395098 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52e40186c0 0x7f52e401abc0 crc :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:05.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.528 7f53059fb700 1 -- 172.21.2.101:0/1253395098 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5300099230 msgr2=0x7f53000b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:05.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.528 7f53059fb700 1 --2- 172.21.2.101:0/1253395098 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5300099230 0x7f53000b18c0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 rx=0x7f52f8009940 tx=0x7f52f800bcf0).stop 2021-02-10T11:36:05.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.528 7f53059fb700 1 -- 172.21.2.101:0/1253395098 shutdown_connections 2021-02-10T11:36:05.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.528 7f53059fb700 1 --2- 172.21.2.101:0/1253395098 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52e40186c0 0x7f52e401abc0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:05.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.528 7f53059fb700 1 --2- 172.21.2.101:0/1253395098 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5300098810 0x7f5300098c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:05.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.528 7f53059fb700 1 --2- 172.21.2.101:0/1253395098 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5300099230 0x7f53000b18c0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:05.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.528 7f53059fb700 1 --2- 172.21.2.101:0/1253395098 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53000970b0 0x7f53000ebe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:05.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.528 7f53059fb700 1 -- 172.21.2.101:0/1253395098 >> 172.21.2.101:0/1253395098 conn(0x7f530019a730 msgr2=0x7f530019e480 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:05.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.529 7f53059fb700 1 -- 172.21.2.101:0/1253395098 shutdown_connections 2021-02-10T11:36:05.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.529 7f53059fb700 1 -- 172.21.2.101:0/1253395098 wait complete. 2021-02-10T11:36:05.545 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:36:05.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.960 7efeba177700 1 Processor -- start 2021-02-10T11:36:05.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.960 7efeba177700 1 -- start start 2021-02-10T11:36:05.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.961 7efeba177700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efeb40d1880 0x7efeb41b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:05.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.961 7efeba177700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb4109a70 con 0x7efeb40cff80 2021-02-10T11:36:05.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.961 7efeba177700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb4088070 con 0x7efeb41b0af0 2021-02-10T11:36:05.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.961 7efeba177700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb40e5f00 con 0x7efeb40d1880 2021-02-10T11:36:05.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.961 7efeb3fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7efeb40cff80 0x7efeb4109140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54148/0 (socket says 172.21.2.101:54148) 2021-02-10T11:36:05.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.961 7efeb3fff700 1 -- 172.21.2.101:0/2030056303 learned_addr learned my addr 172.21.2.101:0/2030056303 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:05.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.961 7efeb37fe700 1 --2- 172.21.2.101:0/2030056303 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efeb40d1880 0x7efeb41b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:05.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.961 7efeb27fc700 1 -- 172.21.2.101:0/2030056303 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 902451848 0 0) 0x7efeac001500 con 0x7efeb40cff80 2021-02-10T11:36:05.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.961 7efeb27fc700 1 -- 172.21.2.101:0/2030056303 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efea0002fe0 con 0x7efeb40cff80 2021-02-10T11:36:05.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.962 7efeb37fe700 1 -- 172.21.2.101:0/2030056303 >> v1:172.21.2.108:6789/0 conn(0x7efeb41b0af0 legacy=0x7efeb4108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:05.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.962 7efeb27fc700 1 -- 172.21.2.101:0/2030056303 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 866134755 0 0) 0x7efea4000ff0 con 0x7efeb41b0af0 2021-02-10T11:36:05.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.962 7efeb37fe700 1 -- 172.21.2.101:0/2030056303 >> v1:172.21.2.101:6789/0 conn(0x7efeb40cff80 legacy=0x7efeb4109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:05.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.962 7efeb37fe700 1 -- 172.21.2.101:0/2030056303 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb40d1c80 con 0x7efeb40d1880 2021-02-10T11:36:05.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.962 7efeb37fe700 1 --2- 172.21.2.101:0/2030056303 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efeb40d1880 0x7efeb41b04d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7efe9c006650 tx=0x7efe9c009a00).ready entity=mon.2 client_cookie=2582036edf573336 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:05.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.962 7efeb27fc700 1 -- 172.21.2.101:0/2030056303 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efe9c0009b0 con 0x7efeb40d1880 2021-02-10T11:36:05.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.962 7efeb27fc700 1 -- 172.21.2.101:0/2030056303 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9c00c0c0 con 0x7efeb40d1880 2021-02-10T11:36:05.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.962 7efeb27fc700 1 -- 172.21.2.101:0/2030056303 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efe9c00c5a0 con 0x7efeb40d1880 2021-02-10T11:36:05.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.962 7efeba177700 1 -- 172.21.2.101:0/2030056303 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efeb40d1880 msgr2=0x7efeb41b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:05.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.962 7efeba177700 1 --2- 172.21.2.101:0/2030056303 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efeb40d1880 0x7efeb41b04d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7efe9c006650 tx=0x7efe9c009a00).stop 2021-02-10T11:36:05.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.963 7efeba177700 1 -- 172.21.2.101:0/2030056303 shutdown_connections 2021-02-10T11:36:05.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.963 7efeba177700 1 --2- 172.21.2.101:0/2030056303 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efeb40d1880 0x7efeb41b04d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:05.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.963 7efeba177700 1 -- 172.21.2.101:0/2030056303 >> 172.21.2.101:0/2030056303 conn(0x7efeb40ce430 msgr2=0x7efeb40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:05.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.963 7efeba177700 1 -- 172.21.2.101:0/2030056303 shutdown_connections 2021-02-10T11:36:05.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.963 7efeba177700 1 -- 172.21.2.101:0/2030056303 wait complete. 2021-02-10T11:36:05.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.977 7efeba177700 1 Processor -- start 2021-02-10T11:36:05.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.977 7efeba177700 1 -- start start 2021-02-10T11:36:05.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.978 7efeba177700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efeb4086dd0 0x7efeb4082b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:05.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.978 7efeba177700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efeb40ae8f0 0x7efeb40aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:05.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.978 7efeba177700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efeb41adfb0 0x7efeb41ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:05.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.978 7efeba177700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb40871d0 con 0x7efeb41adfb0 2021-02-10T11:36:05.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.978 7efeba177700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb41ae9d0 con 0x7efeb4086dd0 2021-02-10T11:36:05.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.978 7efeba177700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb41aebf0 con 0x7efeb40ae8f0 2021-02-10T11:36:05.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.978 7efeb3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efeb41adfb0 0x7efeb41ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:05.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.978 7efeb3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efeb41adfb0 0x7efeb41ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53660/0 (socket says 172.21.2.101:53660) 2021-02-10T11:36:05.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.978 7efeb3fff700 1 -- 172.21.2.101:0/3512331985 learned_addr learned my addr 172.21.2.101:0/3512331985 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:05.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.978 7efeb3fff700 1 -- 172.21.2.101:0/3512331985 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efeb40ae8f0 msgr2=0x7efeb40aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:05.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.978 7efeb3fff700 1 --2- 172.21.2.101:0/3512331985 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efeb40ae8f0 0x7efeb40aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:05.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.978 7efeb3fff700 1 -- 172.21.2.101:0/3512331985 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efeb4086dd0 msgr2=0x7efeb4082b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:05.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.978 7efeb3fff700 1 --2- 172.21.2.101:0/3512331985 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efeb4086dd0 0x7efeb4082b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:05.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.978 7efeb3fff700 1 -- 172.21.2.101:0/3512331985 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb4083870 con 0x7efeb41adfb0 2021-02-10T11:36:05.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.978 7efeb3fff700 1 --2- 172.21.2.101:0/3512331985 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efeb41adfb0 0x7efeb41ae3b0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 rx=0x7efeac009980 tx=0x7efeac00bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:05.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.979 7efeabfff700 1 -- 172.21.2.101:0/3512331985 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efeac00be10 con 0x7efeb41adfb0 2021-02-10T11:36:05.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.979 7efeabfff700 1 -- 172.21.2.101:0/3512331985 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeac006030 con 0x7efeb41adfb0 2021-02-10T11:36:05.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.979 7efeba177700 1 -- 172.21.2.101:0/3512331985 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb4083870 con 0x7efeb41adfb0 2021-02-10T11:36:05.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.979 7efeabfff700 1 -- 172.21.2.101:0/3512331985 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efeac00be10 con 0x7efeb41adfb0 2021-02-10T11:36:05.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.979 7efeba177700 1 -- 172.21.2.101:0/3512331985 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb4083870 con 0x7efeb41adfb0 2021-02-10T11:36:05.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.979 7efeabfff700 1 -- 172.21.2.101:0/3512331985 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7efeac000d30 con 0x7efeb41adfb0 2021-02-10T11:36:05.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.979 7efeabfff700 1 --2- 172.21.2.101:0/3512331985 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efe980186c0 0x7efe9801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:05.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.979 7efeabfff700 1 -- 172.21.2.101:0/3512331985 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7efeac032af0 con 0x7efeb41adfb0 2021-02-10T11:36:05.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.979 7efeb37fe700 1 --2- 172.21.2.101:0/3512331985 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efe980186c0 0x7efe9801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:05.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:05.980 7efeb37fe700 1 --2- 172.21.2.101:0/3512331985 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efe980186c0 0x7efe9801abc0 crc :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:06.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.011 7efeba177700 1 -- 172.21.2.101:0/3512331985 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb4082900 con 0x7efeb41adfb0 2021-02-10T11:36:06.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.013 7efeabfff700 1 -- 172.21.2.101:0/3512331985 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7efeac006030 con 0x7efeb41adfb0 2021-02-10T11:36:06.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.143 7efeba177700 1 -- 172.21.2.101:0/3512331985 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7efeb4082900 con 0x7efeb41adfb0 2021-02-10T11:36:06.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.146 7efeabfff700 1 -- 172.21.2.101:0/3512331985 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v45) v1 ==== 184+0+0 (secure 0 0 0) 0x7efeac000ef0 con 0x7efeb41adfb0 2021-02-10T11:36:06.157 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.0 2021-02-10T11:36:06.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.151 7efeba177700 1 -- 172.21.2.101:0/3512331985 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efe980186c0 msgr2=0x7efe9801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:06.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.151 7efeba177700 1 --2- 172.21.2.101:0/3512331985 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efe980186c0 0x7efe9801abc0 crc :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:06.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.151 7efeba177700 1 -- 172.21.2.101:0/3512331985 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efeb41adfb0 msgr2=0x7efeb41ae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:06.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.151 7efeba177700 1 --2- 172.21.2.101:0/3512331985 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efeb41adfb0 0x7efeb41ae3b0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 rx=0x7efeac009980 tx=0x7efeac00bcd0).stop 2021-02-10T11:36:06.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.152 7efeba177700 1 -- 172.21.2.101:0/3512331985 shutdown_connections 2021-02-10T11:36:06.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.152 7efeba177700 1 --2- 172.21.2.101:0/3512331985 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efe980186c0 0x7efe9801abc0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:06.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.152 7efeba177700 1 --2- 172.21.2.101:0/3512331985 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efeb40ae8f0 0x7efeb40aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:06.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.152 7efeba177700 1 --2- 172.21.2.101:0/3512331985 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efeb41adfb0 0x7efeb41ae3b0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:06.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.152 7efeba177700 1 --2- 172.21.2.101:0/3512331985 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efeb4086dd0 0x7efeb4082b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:06.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.152 7efeba177700 1 -- 172.21.2.101:0/3512331985 >> 172.21.2.101:0/3512331985 conn(0x7efeb41b9600 msgr2=0x7efeb41b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:06.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.152 7efeba177700 1 -- 172.21.2.101:0/3512331985 shutdown_connections 2021-02-10T11:36:06.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.152 7efeba177700 1 -- 172.21.2.101:0/3512331985 wait complete. 2021-02-10T11:36:06.167 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:36:06.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.574 7f1db22d3700 1 Processor -- start 2021-02-10T11:36:06.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.574 7f1db22d3700 1 -- start start 2021-02-10T11:36:06.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.575 7f1db22d3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1dac0d1c80 0x7f1dac0cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:06.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.575 7f1db22d3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1dac0cf590 0x7f1dac0cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:06.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.575 7f1db22d3700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dac0fe1d0 con 0x7f1dac0d1880 2021-02-10T11:36:06.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.575 7f1db22d3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dac1b0cc0 con 0x7f1dac0cf590 2021-02-10T11:36:06.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.575 7f1db22d3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dac109a70 con 0x7f1dac0d1c80 2021-02-10T11:36:06.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.575 7f1db08f0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1dac0cf590 0x7f1dac0cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:06.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.575 7f1db08f0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1dac0cf590 0x7f1dac0cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53668/0 (socket says 172.21.2.101:53668) 2021-02-10T11:36:06.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.575 7f1db08f0700 1 -- 172.21.2.101:0/1517375094 learned_addr learned my addr 172.21.2.101:0/1517375094 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:06.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.575 7f1dabfff700 1 --1- 172.21.2.101:0/1517375094 >> v1:172.21.2.101:6789/0 conn(0x7f1dac0d1880 0x7f1dac1b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54158/0 (socket says 172.21.2.101:54158) 2021-02-10T11:36:06.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.576 7f1db08f0700 1 -- 172.21.2.101:0/1517375094 >> v2:172.21.2.108:3300/0 conn(0x7f1dac0d1c80 msgr2=0x7f1dac0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:06.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.576 7f1db08f0700 1 --2- 172.21.2.101:0/1517375094 >> v2:172.21.2.108:3300/0 conn(0x7f1dac0d1c80 0x7f1dac0cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:06.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.576 7f1db08f0700 1 -- 172.21.2.101:0/1517375094 >> v1:172.21.2.101:6789/0 conn(0x7f1dac0d1880 legacy=0x7f1dac1b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:06.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.576 7f1db08f0700 1 -- 172.21.2.101:0/1517375094 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dac088070 con 0x7f1dac0cf590 2021-02-10T11:36:06.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.576 7f1db08f0700 1 --2- 172.21.2.101:0/1517375094 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dac0cf590 0x7f1dac0cf990 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 rx=0x7f1da4006f00 tx=0x7f1da4009f10).ready entity=mon.0 client_cookie=c614817cc0337f28 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:06.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.576 7f1daaffd700 1 -- 172.21.2.101:0/1517375094 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1da400b3b0 con 0x7f1dac0cf590 2021-02-10T11:36:06.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.576 7f1daaffd700 1 -- 172.21.2.101:0/1517375094 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da400b730 con 0x7f1dac0cf590 2021-02-10T11:36:06.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.576 7f1daaffd700 1 -- 172.21.2.101:0/1517375094 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1da400bb90 con 0x7f1dac0cf590 2021-02-10T11:36:06.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.576 7f1db22d3700 1 -- 172.21.2.101:0/1517375094 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dac0cf590 msgr2=0x7f1dac0cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:06.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.576 7f1db22d3700 1 --2- 172.21.2.101:0/1517375094 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dac0cf590 0x7f1dac0cf990 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 rx=0x7f1da4006f00 tx=0x7f1da4009f10).stop 2021-02-10T11:36:06.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.577 7f1db22d3700 1 -- 172.21.2.101:0/1517375094 shutdown_connections 2021-02-10T11:36:06.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.577 7f1db22d3700 1 --2- 172.21.2.101:0/1517375094 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dac0cf590 0x7f1dac0cf990 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:06.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.577 7f1db22d3700 1 --2- 172.21.2.101:0/1517375094 >> v2:172.21.2.108:3300/0 conn(0x7f1dac0d1c80 0x7f1dac0cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:06.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.577 7f1db22d3700 1 -- 172.21.2.101:0/1517375094 >> 172.21.2.101:0/1517375094 conn(0x7f1dac0ce430 msgr2=0x7f1dac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:06.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.577 7f1db22d3700 1 -- 172.21.2.101:0/1517375094 shutdown_connections 2021-02-10T11:36:06.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.577 7f1db22d3700 1 -- 172.21.2.101:0/1517375094 wait complete. 2021-02-10T11:36:06.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.592 7f1db22d3700 1 Processor -- start 2021-02-10T11:36:06.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.592 7f1db22d3700 1 -- start start 2021-02-10T11:36:06.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.592 7f1db22d3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dac1b9280 0x7f1dac18f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:06.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.592 7f1db22d3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1dac1b9680 0x7f1dac18fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:06.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.592 7f1db22d3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1dac1acd60 0x7f1dac1ad160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:06.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.592 7f1db22d3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac190390 con 0x7f1dac1b9280 2021-02-10T11:36:06.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.592 7f1db22d3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac1907c0 con 0x7f1dac1b9680 2021-02-10T11:36:06.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.592 7f1db22d3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac1ac3d0 con 0x7f1dac1acd60 2021-02-10T11:36:06.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.592 7f1dabfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dac1b9280 0x7f1dac18f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:06.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.593 7f1dabfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dac1b9280 0x7f1dac18f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53670/0 (socket says 172.21.2.101:53670) 2021-02-10T11:36:06.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.593 7f1dabfff700 1 -- 172.21.2.101:0/2082457082 learned_addr learned my addr 172.21.2.101:0/2082457082 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:06.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.593 7f1dabfff700 1 -- 172.21.2.101:0/2082457082 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1dac1acd60 msgr2=0x7f1dac1ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:06.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.593 7f1dabfff700 1 --2- 172.21.2.101:0/2082457082 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1dac1acd60 0x7f1dac1ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:06.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.593 7f1dabfff700 1 -- 172.21.2.101:0/2082457082 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1dac1b9680 msgr2=0x7f1dac18fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:06.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.593 7f1dabfff700 1 --2- 172.21.2.101:0/2082457082 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1dac1b9680 0x7f1dac18fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:06.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.593 7f1dabfff700 1 -- 172.21.2.101:0/2082457082 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dac1ad780 con 0x7f1dac1b9280 2021-02-10T11:36:06.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.593 7f1dabfff700 1 --2- 172.21.2.101:0/2082457082 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dac1b9280 0x7f1dac18f750 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 rx=0x7f1d9c006f00 tx=0x7f1d9c00b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:06.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.593 7f1da8ff9700 1 -- 172.21.2.101:0/2082457082 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1d9c006020 con 0x7f1dac1b9280 2021-02-10T11:36:06.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.593 7f1da8ff9700 1 -- 172.21.2.101:0/2082457082 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9c0063a0 con 0x7f1dac1b9280 2021-02-10T11:36:06.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.593 7f1da8ff9700 1 -- 172.21.2.101:0/2082457082 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1d9c004620 con 0x7f1dac1b9280 2021-02-10T11:36:06.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.593 7f1db22d3700 1 -- 172.21.2.101:0/2082457082 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dac1ad780 con 0x7f1dac1b9280 2021-02-10T11:36:06.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.593 7f1db22d3700 1 -- 172.21.2.101:0/2082457082 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dac1add10 con 0x7f1dac1b9280 2021-02-10T11:36:06.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.594 7f1da8ff9700 1 -- 172.21.2.101:0/2082457082 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1d9c00a000 con 0x7f1dac1b9280 2021-02-10T11:36:06.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.594 7f1da8ff9700 1 --2- 172.21.2.101:0/2082457082 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d8c018730 0x7f1d8c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:06.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.594 7f1da8ff9700 1 -- 172.21.2.101:0/2082457082 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1d9c032600 con 0x7f1dac1b9280 2021-02-10T11:36:06.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.594 7f1dab7fe700 1 --2- 172.21.2.101:0/2082457082 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d8c018730 0x7f1d8c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:06.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.594 7f1dab7fe700 1 --2- 172.21.2.101:0/2082457082 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d8c018730 0x7f1d8c01abc0 crc :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:06.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.625 7f1db22d3700 1 -- 172.21.2.101:0/2082457082 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dac190390 con 0x7f1dac1b9280 2021-02-10T11:36:06.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.627 7f1da8ff9700 1 -- 172.21.2.101:0/2082457082 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1d9c006b90 con 0x7f1dac1b9280 2021-02-10T11:36:06.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.758 7f1db22d3700 1 -- 172.21.2.101:0/2082457082 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f1dac190390 con 0x7f1dac1b9280 2021-02-10T11:36:06.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.762 7f1da8ff9700 1 -- 172.21.2.101:0/2082457082 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v46) v1 ==== 184+0+0 (secure 0 0 0) 0x7f1d9c00a000 con 0x7f1dac1b9280 2021-02-10T11:36:06.772 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.1 2021-02-10T11:36:06.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.767 7f1db22d3700 1 -- 172.21.2.101:0/2082457082 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d8c018730 msgr2=0x7f1d8c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:06.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.767 7f1db22d3700 1 --2- 172.21.2.101:0/2082457082 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d8c018730 0x7f1d8c01abc0 crc :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:06.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.767 7f1db22d3700 1 -- 172.21.2.101:0/2082457082 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dac1b9280 msgr2=0x7f1dac18f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:06.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.767 7f1db22d3700 1 --2- 172.21.2.101:0/2082457082 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dac1b9280 0x7f1dac18f750 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 rx=0x7f1d9c006f00 tx=0x7f1d9c00b940).stop 2021-02-10T11:36:06.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.767 7f1db22d3700 1 -- 172.21.2.101:0/2082457082 shutdown_connections 2021-02-10T11:36:06.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.767 7f1db22d3700 1 --2- 172.21.2.101:0/2082457082 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d8c018730 0x7f1d8c01abc0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:06.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.767 7f1db22d3700 1 --2- 172.21.2.101:0/2082457082 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1dac1acd60 0x7f1dac1ad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:06.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.767 7f1db22d3700 1 --2- 172.21.2.101:0/2082457082 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1dac1b9680 0x7f1dac18fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:06.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.767 7f1db22d3700 1 --2- 172.21.2.101:0/2082457082 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dac1b9280 0x7f1dac18f750 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:06.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.767 7f1db22d3700 1 -- 172.21.2.101:0/2082457082 >> 172.21.2.101:0/2082457082 conn(0x7f1dac0845b0 msgr2=0x7f1dac18c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:06.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.767 7f1db22d3700 1 -- 172.21.2.101:0/2082457082 shutdown_connections 2021-02-10T11:36:06.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:06.767 7f1db22d3700 1 -- 172.21.2.101:0/2082457082 wait complete. 2021-02-10T11:36:06.783 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:36:07.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.197 7ff07cb37700 1 Processor -- start 2021-02-10T11:36:07.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.197 7ff07cb37700 1 -- start start 2021-02-10T11:36:07.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.197 7ff07cb37700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff0780d1c80 0x7ff0780cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:07.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.197 7ff07cb37700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0780d0180 con 0x7ff0780cf590 2021-02-10T11:36:07.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.197 7ff07cb37700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0780fe1d0 con 0x7ff0780d1880 2021-02-10T11:36:07.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.197 7ff07cb37700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0781b0cc0 con 0x7ff0780d1c80 2021-02-10T11:36:07.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.198 7ff07705f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff0780cf590 0x7ff0780cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54176/0 (socket says 172.21.2.101:54176) 2021-02-10T11:36:07.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.198 7ff07705f700 1 -- 172.21.2.101:0/1192341273 learned_addr learned my addr 172.21.2.101:0/1192341273 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:07.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.198 7ff07585c700 1 -- 172.21.2.101:0/1192341273 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2599016202 0 0) 0x7ff070001500 con 0x7ff0780cf590 2021-02-10T11:36:07.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.198 7ff07585c700 1 -- 172.21.2.101:0/1192341273 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff064002fe0 con 0x7ff0780cf590 2021-02-10T11:36:07.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.198 7ff07605d700 1 --2- 172.21.2.101:0/1192341273 >> v2:172.21.2.108:3300/0 conn(0x7ff0780d1c80 0x7ff0780cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:07.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.198 7ff07585c700 1 -- 172.21.2.101:0/1192341273 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3986193684 0 0) 0x7ff070001460 con 0x7ff0780cf590 2021-02-10T11:36:07.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.198 7ff07585c700 1 -- 172.21.2.101:0/1192341273 >> v2:172.21.2.108:3300/0 conn(0x7ff0780d1c80 msgr2=0x7ff0780cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:07.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.198 7ff07585c700 1 --2- 172.21.2.101:0/1192341273 >> v2:172.21.2.108:3300/0 conn(0x7ff0780d1c80 0x7ff0780cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:07.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.198 7ff07585c700 1 -- 172.21.2.101:0/1192341273 >> v1:172.21.2.108:6789/0 conn(0x7ff0780d1880 legacy=0x7ff0781b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:07.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.198 7ff07585c700 1 -- 172.21.2.101:0/1192341273 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff078109a70 con 0x7ff0780cf590 2021-02-10T11:36:07.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.198 7ff07585c700 1 -- 172.21.2.101:0/1192341273 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7ff070001980 con 0x7ff0780cf590 2021-02-10T11:36:07.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.198 7ff07585c700 1 -- 172.21.2.101:0/1192341273 >> v1:172.21.2.101:6789/0 conn(0x7ff0780cf590 legacy=0x7ff0780cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:07.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.198 7ff07585c700 1 --2- 172.21.2.101:0/1192341273 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff064003b10 0x7ff064005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:07.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.198 7ff07585c700 1 --2- 172.21.2.101:0/1192341273 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff064006800 0x7ff064008d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:07.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.198 7ff07585c700 1 --2- 172.21.2.101:0/1192341273 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff064009620 0x7ff06400bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:07.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.198 7ff07585c700 1 -- 172.21.2.101:0/1192341273 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06400c390 con 0x7ff064009620 2021-02-10T11:36:07.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.198 7ff07585c700 1 -- 172.21.2.101:0/1192341273 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06400c5b0 con 0x7ff064003b10 2021-02-10T11:36:07.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.198 7ff07585c700 1 -- 172.21.2.101:0/1192341273 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06400c7d0 con 0x7ff064006800 2021-02-10T11:36:07.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.198 7ff07605d700 1 --2- 172.21.2.101:0/1192341273 >> v2:172.21.2.108:3300/0 conn(0x7ff0780d1c80 0x7ff0780cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:36:07.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.198 7ff07705f700 1 --2- 172.21.2.101:0/1192341273 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff064009620 0x7ff06400bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:07.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.199 7ff07685e700 1 --2- 172.21.2.101:0/1192341273 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff064003b10 0x7ff064005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:07.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.199 7ff07605d700 1 --2- 172.21.2.101:0/1192341273 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff064006800 0x7ff064008d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:07.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.199 7ff07705f700 1 -- 172.21.2.101:0/1192341273 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff064006800 msgr2=0x7ff064008d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:07.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.199 7ff07705f700 1 --2- 172.21.2.101:0/1192341273 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff064006800 0x7ff064008d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:07.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.199 7ff07705f700 1 -- 172.21.2.101:0/1192341273 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff064003b10 msgr2=0x7ff064005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:07.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.199 7ff07705f700 1 --2- 172.21.2.101:0/1192341273 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff064003b10 0x7ff064005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:07.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.199 7ff07705f700 1 -- 172.21.2.101:0/1192341273 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06400ca70 con 0x7ff064009620 2021-02-10T11:36:07.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.199 7ff07705f700 1 -- 172.21.2.101:0/1192341273 reap_dead start 2021-02-10T11:36:07.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.199 7ff07705f700 1 --2- 172.21.2.101:0/1192341273 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff064009620 0x7ff06400bb40 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 rx=0x7ff070001ae0 tx=0x7ff07004e320).ready entity=mon.0 client_cookie=9f77124406c6c46b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:07.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.199 7ff07585c700 1 -- 172.21.2.101:0/1192341273 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff070001460 con 0x7ff064009620 2021-02-10T11:36:07.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.199 7ff07585c700 1 -- 172.21.2.101:0/1192341273 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0700070c0 con 0x7ff064009620 2021-02-10T11:36:07.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.199 7ff07585c700 1 -- 172.21.2.101:0/1192341273 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff070004720 con 0x7ff064009620 2021-02-10T11:36:07.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.199 7ff07cb37700 1 -- 172.21.2.101:0/1192341273 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff064009620 msgr2=0x7ff06400bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:07.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.199 7ff07cb37700 1 --2- 172.21.2.101:0/1192341273 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff064009620 0x7ff06400bb40 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 rx=0x7ff070001ae0 tx=0x7ff07004e320).stop 2021-02-10T11:36:07.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.200 7ff07cb37700 1 -- 172.21.2.101:0/1192341273 shutdown_connections 2021-02-10T11:36:07.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.200 7ff07cb37700 1 --2- 172.21.2.101:0/1192341273 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff064009620 0x7ff06400bb40 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:07.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.200 7ff07cb37700 1 -- 172.21.2.101:0/1192341273 >> 172.21.2.101:0/1192341273 conn(0x7ff0780ce430 msgr2=0x7ff0780d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:07.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.200 7ff07cb37700 1 -- 172.21.2.101:0/1192341273 shutdown_connections 2021-02-10T11:36:07.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.200 7ff07cb37700 1 -- 172.21.2.101:0/1192341273 wait complete. 2021-02-10T11:36:07.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.215 7ff07cb37700 1 Processor -- start 2021-02-10T11:36:07.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.215 7ff07cb37700 1 -- start start 2021-02-10T11:36:07.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.215 7ff07cb37700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0780abd30 0x7ff078082bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:07.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.215 7ff07cb37700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0780831d0 0x7ff07808a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:07.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.215 7ff07cb37700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0780ac130 0x7ff07808aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:07.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.215 7ff07cb37700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff078083700 con 0x7ff0780abd30 2021-02-10T11:36:07.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.215 7ff07cb37700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07808b0c0 con 0x7ff0780ac130 2021-02-10T11:36:07.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.215 7ff07cb37700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07808b2a0 con 0x7ff0780831d0 2021-02-10T11:36:07.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.215 7ff07685e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0780abd30 0x7ff078082bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:07.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.215 7ff07685e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0780abd30 0x7ff078082bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53690/0 (socket says 172.21.2.101:53690) 2021-02-10T11:36:07.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.215 7ff07685e700 1 -- 172.21.2.101:0/4141920192 learned_addr learned my addr 172.21.2.101:0/4141920192 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:07.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.215 7ff07685e700 1 -- 172.21.2.101:0/4141920192 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0780831d0 msgr2=0x7ff07808a480 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:07.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.215 7ff07685e700 1 --2- 172.21.2.101:0/4141920192 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0780831d0 0x7ff07808a480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:07.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.215 7ff07685e700 1 -- 172.21.2.101:0/4141920192 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0780ac130 msgr2=0x7ff07808aaa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:07.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.215 7ff07685e700 1 --2- 172.21.2.101:0/4141920192 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0780ac130 0x7ff07808aaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:07.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.215 7ff07685e700 1 -- 172.21.2.101:0/4141920192 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07808b5f0 con 0x7ff0780abd30 2021-02-10T11:36:07.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.216 7ff07685e700 1 --2- 172.21.2.101:0/4141920192 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0780abd30 0x7ff078082bb0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 rx=0x7ff068006990 tx=0x7ff06800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:07.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.216 7ff06f7fe700 1 -- 172.21.2.101:0/4141920192 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff068006a80 con 0x7ff0780abd30 2021-02-10T11:36:07.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.216 7ff06f7fe700 1 -- 172.21.2.101:0/4141920192 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff068000c60 con 0x7ff0780abd30 2021-02-10T11:36:07.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.216 7ff06f7fe700 1 -- 172.21.2.101:0/4141920192 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0680022c0 con 0x7ff0780abd30 2021-02-10T11:36:07.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.216 7ff07cb37700 1 -- 172.21.2.101:0/4141920192 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff07808b5f0 con 0x7ff0780abd30 2021-02-10T11:36:07.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.216 7ff07cb37700 1 -- 172.21.2.101:0/4141920192 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff07808b5f0 con 0x7ff0780abd30 2021-02-10T11:36:07.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.217 7ff06f7fe700 1 -- 172.21.2.101:0/4141920192 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff068009fe0 con 0x7ff0780abd30 2021-02-10T11:36:07.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.217 7ff06f7fe700 1 --2- 172.21.2.101:0/4141920192 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff05c0186c0 0x7ff05c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:07.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.217 7ff06f7fe700 1 -- 172.21.2.101:0/4141920192 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff068032590 con 0x7ff0780abd30 2021-02-10T11:36:07.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.217 7ff07605d700 1 --2- 172.21.2.101:0/4141920192 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff05c0186c0 0x7ff05c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:07.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.217 7ff07605d700 1 --2- 172.21.2.101:0/4141920192 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff05c0186c0 0x7ff05c01abc0 crc :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:07.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.247 7ff07cb37700 1 -- 172.21.2.101:0/4141920192 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff078082980 con 0x7ff0780abd30 2021-02-10T11:36:07.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.250 7ff06f7fe700 1 -- 172.21.2.101:0/4141920192 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff068006a80 con 0x7ff0780abd30 2021-02-10T11:36:07.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.382 7ff07cb37700 1 -- 172.21.2.101:0/4141920192 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ff078082980 con 0x7ff0780abd30 2021-02-10T11:36:07.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.385 7ff06f7fe700 1 -- 172.21.2.101:0/4141920192 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v47) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff068009fe0 con 0x7ff0780abd30 2021-02-10T11:36:07.392 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.2 2021-02-10T11:36:07.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.386 7ff07cb37700 1 -- 172.21.2.101:0/4141920192 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff05c0186c0 msgr2=0x7ff05c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:07.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.386 7ff07cb37700 1 --2- 172.21.2.101:0/4141920192 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff05c0186c0 0x7ff05c01abc0 crc :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:07.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.386 7ff07cb37700 1 -- 172.21.2.101:0/4141920192 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0780abd30 msgr2=0x7ff078082bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:07.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.386 7ff07cb37700 1 --2- 172.21.2.101:0/4141920192 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0780abd30 0x7ff078082bb0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 rx=0x7ff068006990 tx=0x7ff06800b990).stop 2021-02-10T11:36:07.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.387 7ff07cb37700 1 -- 172.21.2.101:0/4141920192 shutdown_connections 2021-02-10T11:36:07.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.387 7ff07cb37700 1 --2- 172.21.2.101:0/4141920192 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff05c0186c0 0x7ff05c01abc0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:07.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.387 7ff07cb37700 1 --2- 172.21.2.101:0/4141920192 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0780831d0 0x7ff07808a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:07.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.387 7ff07cb37700 1 --2- 172.21.2.101:0/4141920192 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0780ac130 0x7ff07808aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:07.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.387 7ff07cb37700 1 --2- 172.21.2.101:0/4141920192 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0780abd30 0x7ff078082bb0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:07.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.387 7ff07cb37700 1 -- 172.21.2.101:0/4141920192 >> 172.21.2.101:0/4141920192 conn(0x7ff0780afc60 msgr2=0x7ff0780b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:07.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.387 7ff07cb37700 1 -- 172.21.2.101:0/4141920192 shutdown_connections 2021-02-10T11:36:07.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.387 7ff07cb37700 1 -- 172.21.2.101:0/4141920192 wait complete. 2021-02-10T11:36:07.402 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:36:07.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.817 7fb7ed841700 1 Processor -- start 2021-02-10T11:36:07.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.817 7fb7ed841700 1 -- start start 2021-02-10T11:36:07.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.818 7fb7ed841700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb7e80d1880 0x7fb7e81b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:07.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.818 7fb7ed841700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7e80cff80 0x7fb7e8109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:07.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.818 7fb7ed841700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e8109a70 con 0x7fb7e81b0af0 2021-02-10T11:36:07.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.818 7fb7ed841700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e8109760 con 0x7fb7e80d1880 2021-02-10T11:36:07.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.818 7fb7ed841700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e8088070 con 0x7fb7e80cff80 2021-02-10T11:36:07.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.818 7fb7e7d75700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7e80cff80 0x7fb7e8109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:07.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.818 7fb7e7d75700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7e80cff80 0x7fb7e8109140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:46430/0 (socket says 172.21.2.101:46430) 2021-02-10T11:36:07.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.818 7fb7e7d75700 1 -- 172.21.2.101:0/2476609359 learned_addr learned my addr 172.21.2.101:0/2476609359 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:07.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.818 7fb7e7574700 1 --2- 172.21.2.101:0/2476609359 >> v2:172.21.2.108:3300/0 conn(0x7fb7e80d1880 0x7fb7e81b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:07.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.819 7fb7e7d75700 1 -- 172.21.2.101:0/2476609359 >> v2:172.21.2.108:3300/0 conn(0x7fb7e80d1880 msgr2=0x7fb7e81b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:07.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.819 7fb7e7d75700 1 --2- 172.21.2.101:0/2476609359 >> v2:172.21.2.108:3300/0 conn(0x7fb7e80d1880 0x7fb7e81b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:07.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.819 7fb7e7d75700 1 -- 172.21.2.101:0/2476609359 >> v1:172.21.2.108:6789/0 conn(0x7fb7e81b0af0 legacy=0x7fb7e8108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:07.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.819 7fb7e7d75700 1 -- 172.21.2.101:0/2476609359 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e80d1c80 con 0x7fb7e80cff80 2021-02-10T11:36:07.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.819 7fb7e6572700 1 -- 172.21.2.101:0/2476609359 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2693990666 0 0) 0x7fb7d0000ff0 con 0x7fb7e81b0af0 2021-02-10T11:36:07.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.819 7fb7e7d75700 1 --2- 172.21.2.101:0/2476609359 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7e80cff80 0x7fb7e8109140 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fb7e0006aa0 tx=0x7fb7e0009f20).ready entity=mon.2 client_cookie=b2c72b5818c35402 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:07.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.819 7fb7e6572700 1 -- 172.21.2.101:0/2476609359 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb7e000b5c0 con 0x7fb7e80cff80 2021-02-10T11:36:07.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.819 7fb7e6572700 1 -- 172.21.2.101:0/2476609359 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e0003030 con 0x7fb7e80cff80 2021-02-10T11:36:07.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.819 7fb7e6572700 1 -- 172.21.2.101:0/2476609359 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb7e000ba30 con 0x7fb7e80cff80 2021-02-10T11:36:07.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.819 7fb7ed841700 1 -- 172.21.2.101:0/2476609359 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7e80cff80 msgr2=0x7fb7e8109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:07.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.819 7fb7ed841700 1 --2- 172.21.2.101:0/2476609359 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7e80cff80 0x7fb7e8109140 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fb7e0006aa0 tx=0x7fb7e0009f20).stop 2021-02-10T11:36:07.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.820 7fb7ed841700 1 -- 172.21.2.101:0/2476609359 shutdown_connections 2021-02-10T11:36:07.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.820 7fb7ed841700 1 --2- 172.21.2.101:0/2476609359 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7e80cff80 0x7fb7e8109140 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:07.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.820 7fb7ed841700 1 --2- 172.21.2.101:0/2476609359 >> v2:172.21.2.108:3300/0 conn(0x7fb7e80d1880 0x7fb7e81b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:07.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.820 7fb7ed841700 1 -- 172.21.2.101:0/2476609359 >> 172.21.2.101:0/2476609359 conn(0x7fb7e80ce430 msgr2=0x7fb7e80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:07.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.820 7fb7ed841700 1 -- 172.21.2.101:0/2476609359 shutdown_connections 2021-02-10T11:36:07.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.820 7fb7ed841700 1 -- 172.21.2.101:0/2476609359 wait complete. 2021-02-10T11:36:07.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.835 7fb7ed841700 1 Processor -- start 2021-02-10T11:36:07.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.835 7fb7ed841700 1 -- start start 2021-02-10T11:36:07.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.835 7fb7ed841700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb7e8085610 0x7fb7e80ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:07.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.835 7fb7ed841700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7e8085a10 0x7fb7e8086460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:07.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.835 7fb7ed841700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7e8084bd0 0x7fb7e8086a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:07.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.835 7fb7ed841700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e8085090 con 0x7fb7e8084bd0 2021-02-10T11:36:07.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.835 7fb7ed841700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e80aede0 con 0x7fb7e8085610 2021-02-10T11:36:07.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.835 7fb7ed841700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e80aefc0 con 0x7fb7e8085a10 2021-02-10T11:36:07.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.836 7fb7e7d75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7e8084bd0 0x7fb7e8086a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:07.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.836 7fb7e7d75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7e8084bd0 0x7fb7e8086a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53706/0 (socket says 172.21.2.101:53706) 2021-02-10T11:36:07.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.836 7fb7e7d75700 1 -- 172.21.2.101:0/316150557 learned_addr learned my addr 172.21.2.101:0/316150557 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:07.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.836 7fb7e6d73700 1 --2- 172.21.2.101:0/316150557 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7e8085a10 0x7fb7e8086460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:07.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.836 7fb7e7d75700 1 -- 172.21.2.101:0/316150557 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7e8085a10 msgr2=0x7fb7e8086460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:07.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.836 7fb7e7d75700 1 --2- 172.21.2.101:0/316150557 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7e8085a10 0x7fb7e8086460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:07.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.836 7fb7e7d75700 1 -- 172.21.2.101:0/316150557 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb7e8085610 msgr2=0x7fb7e80ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:07.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.836 7fb7e7d75700 1 --2- 172.21.2.101:0/316150557 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb7e8085610 0x7fb7e80ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:07.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.836 7fb7e7d75700 1 -- 172.21.2.101:0/316150557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e80871c0 con 0x7fb7e8084bd0 2021-02-10T11:36:07.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.836 7fb7e6d73700 1 --2- 172.21.2.101:0/316150557 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7e8085a10 0x7fb7e8086460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:36:07.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.836 7fb7e7d75700 1 --2- 172.21.2.101:0/316150557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7e8084bd0 0x7fb7e8086a80 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 rx=0x7fb7e000b970 tx=0x7fb7e000bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:07.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.836 7fb7dffff700 1 -- 172.21.2.101:0/316150557 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb7e0009450 con 0x7fb7e8084bd0 2021-02-10T11:36:07.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.836 7fb7dffff700 1 -- 172.21.2.101:0/316150557 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e0006030 con 0x7fb7e8084bd0 2021-02-10T11:36:07.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.836 7fb7dffff700 1 -- 172.21.2.101:0/316150557 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb7e0006550 con 0x7fb7e8084bd0 2021-02-10T11:36:07.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.836 7fb7ed841700 1 -- 172.21.2.101:0/316150557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e80871c0 con 0x7fb7e8084bd0 2021-02-10T11:36:07.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.836 7fb7ed841700 1 -- 172.21.2.101:0/316150557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e81a93e0 con 0x7fb7e8084bd0 2021-02-10T11:36:07.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.837 7fb7dffff700 1 -- 172.21.2.101:0/316150557 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb7e0006210 con 0x7fb7e8084bd0 2021-02-10T11:36:07.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.837 7fb7dffff700 1 --2- 172.21.2.101:0/316150557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb7cc0186c0 0x7fb7cc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:07.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.837 7fb7dffff700 1 -- 172.21.2.101:0/316150557 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb7e00328d0 con 0x7fb7e8084bd0 2021-02-10T11:36:07.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.837 7fb7e7574700 1 --2- 172.21.2.101:0/316150557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb7cc0186c0 0x7fb7cc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:07.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.837 7fb7e7574700 1 --2- 172.21.2.101:0/316150557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb7cc0186c0 0x7fb7cc01abc0 crc :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:07.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.868 7fb7ed841700 1 -- 172.21.2.101:0/316150557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e8082990 con 0x7fb7e8084bd0 2021-02-10T11:36:07.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:07.871 7fb7dffff700 1 -- 172.21.2.101:0/316150557 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb7e0009450 con 0x7fb7e8084bd0 2021-02-10T11:36:08.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.000 7fb7ed841700 1 -- 172.21.2.101:0/316150557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fb7e8082990 con 0x7fb7e8084bd0 2021-02-10T11:36:08.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.004 7fb7dffff700 1 -- 172.21.2.101:0/316150557 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v48) v1 ==== 184+0+0 (secure 0 0 0) 0x7fb7e0006210 con 0x7fb7e8084bd0 2021-02-10T11:36:08.015 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.3 2021-02-10T11:36:08.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.009 7fb7ed841700 1 -- 172.21.2.101:0/316150557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb7cc0186c0 msgr2=0x7fb7cc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:08.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.009 7fb7ed841700 1 --2- 172.21.2.101:0/316150557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb7cc0186c0 0x7fb7cc01abc0 crc :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:08.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.009 7fb7ed841700 1 -- 172.21.2.101:0/316150557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7e8084bd0 msgr2=0x7fb7e8086a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:08.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.009 7fb7ed841700 1 --2- 172.21.2.101:0/316150557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7e8084bd0 0x7fb7e8086a80 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 rx=0x7fb7e000b970 tx=0x7fb7e000bd10).stop 2021-02-10T11:36:08.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.009 7fb7ed841700 1 -- 172.21.2.101:0/316150557 shutdown_connections 2021-02-10T11:36:08.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.009 7fb7ed841700 1 --2- 172.21.2.101:0/316150557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb7cc0186c0 0x7fb7cc01abc0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:08.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.009 7fb7ed841700 1 --2- 172.21.2.101:0/316150557 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7e8085a10 0x7fb7e8086460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:08.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.009 7fb7ed841700 1 --2- 172.21.2.101:0/316150557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7e8084bd0 0x7fb7e8086a80 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:08.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.009 7fb7ed841700 1 --2- 172.21.2.101:0/316150557 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb7e8085610 0x7fb7e80ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:08.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.009 7fb7ed841700 1 -- 172.21.2.101:0/316150557 >> 172.21.2.101:0/316150557 conn(0x7fb7e81ac500 msgr2=0x7fb7e81b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:08.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.010 7fb7ed841700 1 -- 172.21.2.101:0/316150557 shutdown_connections 2021-02-10T11:36:08.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.010 7fb7ed841700 1 -- 172.21.2.101:0/316150557 wait complete. 2021-02-10T11:36:08.025 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:36:08.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.427 7f1592507700 1 Processor -- start 2021-02-10T11:36:08.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.427 7f1592507700 1 -- start start 2021-02-10T11:36:08.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.428 7f1592507700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f158c0d1870 0x7f158c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:08.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.428 7f1592507700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f158c1b0a10 0x7f158c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:08.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.428 7f1592507700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158c0f0b10 con 0x7f158c0d1c70 2021-02-10T11:36:08.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.428 7f1592507700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158c0e5ed0 con 0x7f158c0d1870 2021-02-10T11:36:08.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.428 7f1592507700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158c0f0650 con 0x7f158c1b0a10 2021-02-10T11:36:08.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.428 7f158bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f158c0d1870 0x7f158c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:08.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.428 7f158bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f158c0d1870 0x7f158c1b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53712/0 (socket says 172.21.2.101:53712) 2021-02-10T11:36:08.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.428 7f158bfff700 1 -- 172.21.2.101:0/3012807658 learned_addr learned my addr 172.21.2.101:0/3012807658 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:08.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.428 7f158bfff700 1 -- 172.21.2.101:0/3012807658 >> v2:172.21.2.108:3300/0 conn(0x7f158c1b0a10 msgr2=0x7f158c0cf330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:36:08.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.428 7f158bfff700 1 --2- 172.21.2.101:0/3012807658 >> v2:172.21.2.108:3300/0 conn(0x7f158c1b0a10 0x7f158c0cf330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:08.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.428 7f158bfff700 1 -- 172.21.2.101:0/3012807658 >> v1:172.21.2.108:6789/0 conn(0x7f158c0d1c70 legacy=0x7f158c0cf950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:36:08.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.428 7f158bfff700 1 -- 172.21.2.101:0/3012807658 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c0f0840 con 0x7f158c0d1870 2021-02-10T11:36:08.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.428 7f158bfff700 1 --2- 172.21.2.101:0/3012807658 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f158c0d1870 0x7f158c1b03f0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 rx=0x7f157c006700 tx=0x7f157c009b80).ready entity=mon.0 client_cookie=cf69a234828d25eb server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:08.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.428 7f158affd700 1 -- 172.21.2.101:0/3012807658 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f157c00c0c0 con 0x7f158c0d1870 2021-02-10T11:36:08.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.429 7f158affd700 1 -- 172.21.2.101:0/3012807658 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c00c340 con 0x7f158c0d1870 2021-02-10T11:36:08.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.429 7f158affd700 1 -- 172.21.2.101:0/3012807658 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f157c00c820 con 0x7f158c0d1870 2021-02-10T11:36:08.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.429 7f1592507700 1 -- 172.21.2.101:0/3012807658 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f158c0d1870 msgr2=0x7f158c1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:08.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.429 7f1592507700 1 --2- 172.21.2.101:0/3012807658 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f158c0d1870 0x7f158c1b03f0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 rx=0x7f157c006700 tx=0x7f157c009b80).stop 2021-02-10T11:36:08.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.429 7f1592507700 1 -- 172.21.2.101:0/3012807658 shutdown_connections 2021-02-10T11:36:08.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.429 7f1592507700 1 --2- 172.21.2.101:0/3012807658 >> v2:172.21.2.108:3300/0 conn(0x7f158c1b0a10 0x7f158c0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:08.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.429 7f1592507700 1 --2- 172.21.2.101:0/3012807658 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f158c0d1870 0x7f158c1b03f0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:08.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.429 7f1592507700 1 -- 172.21.2.101:0/3012807658 >> 172.21.2.101:0/3012807658 conn(0x7f158c0ce450 msgr2=0x7f158c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:08.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.429 7f1592507700 1 -- 172.21.2.101:0/3012807658 shutdown_connections 2021-02-10T11:36:08.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.429 7f1592507700 1 -- 172.21.2.101:0/3012807658 wait complete. 2021-02-10T11:36:08.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.443 7f1592507700 1 Processor -- start 2021-02-10T11:36:08.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.444 7f1592507700 1 -- start start 2021-02-10T11:36:08.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.444 7f1592507700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f158c0b4f30 0x7f158c18b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:08.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.444 7f1592507700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f158c093f40 0x7f158c094340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:08.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.444 7f1592507700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f158c094960 0x7f158c187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:08.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.444 7f1592507700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c094e90 con 0x7f158c0b4f30 2021-02-10T11:36:08.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.444 7f1592507700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c18bff0 con 0x7f158c094960 2021-02-10T11:36:08.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.444 7f1592507700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158c08f780 con 0x7f158c093f40 2021-02-10T11:36:08.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.444 7f158bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f158c0b4f30 0x7f158c18b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:08.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.444 7f158bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f158c0b4f30 0x7f158c18b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53718/0 (socket says 172.21.2.101:53718) 2021-02-10T11:36:08.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.444 7f158bfff700 1 -- 172.21.2.101:0/1290898695 learned_addr learned my addr 172.21.2.101:0/1290898695 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:08.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.444 7f158bfff700 1 -- 172.21.2.101:0/1290898695 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f158c093f40 msgr2=0x7f158c094340 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:08.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.444 7f158bfff700 1 --2- 172.21.2.101:0/1290898695 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f158c093f40 0x7f158c094340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:08.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.444 7f158bfff700 1 -- 172.21.2.101:0/1290898695 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f158c094960 msgr2=0x7f158c187e50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:08.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.444 7f158bfff700 1 --2- 172.21.2.101:0/1290898695 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f158c094960 0x7f158c187e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:08.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.444 7f158bfff700 1 -- 172.21.2.101:0/1290898695 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c188470 con 0x7f158c0b4f30 2021-02-10T11:36:08.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.445 7f158bfff700 1 --2- 172.21.2.101:0/1290898695 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f158c0b4f30 0x7f158c18b9d0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 rx=0x7f157c00b5e0 tx=0x7f157c00b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:08.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.445 7f1588ff9700 1 -- 172.21.2.101:0/1290898695 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f157c00bd70 con 0x7f158c0b4f30 2021-02-10T11:36:08.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.445 7f1588ff9700 1 -- 172.21.2.101:0/1290898695 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c002030 con 0x7f158c0b4f30 2021-02-10T11:36:08.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.445 7f1592507700 1 -- 172.21.2.101:0/1290898695 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f158c188470 con 0x7f158c0b4f30 2021-02-10T11:36:08.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.445 7f1592507700 1 -- 172.21.2.101:0/1290898695 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158c188a00 con 0x7f158c0b4f30 2021-02-10T11:36:08.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.445 7f1588ff9700 1 -- 172.21.2.101:0/1290898695 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f157c002550 con 0x7f158c0b4f30 2021-02-10T11:36:08.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.446 7f1588ff9700 1 -- 172.21.2.101:0/1290898695 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f157c002210 con 0x7f158c0b4f30 2021-02-10T11:36:08.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.446 7f1588ff9700 1 --2- 172.21.2.101:0/1290898695 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f156c0186c0 0x7f156c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:08.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.446 7f1588ff9700 1 -- 172.21.2.101:0/1290898695 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f157c032560 con 0x7f158c0b4f30 2021-02-10T11:36:08.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.446 7f158b7fe700 1 --2- 172.21.2.101:0/1290898695 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f156c0186c0 0x7f156c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:08.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.446 7f158b7fe700 1 --2- 172.21.2.101:0/1290898695 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f156c0186c0 0x7f156c01abc0 crc :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:08.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.477 7f1592507700 1 -- 172.21.2.101:0/1290898695 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158c0ce450 con 0x7f158c0b4f30 2021-02-10T11:36:08.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.480 7f1588ff9700 1 -- 172.21.2.101:0/1290898695 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f157c0009b0 con 0x7f158c0b4f30 2021-02-10T11:36:08.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.607 7f1592507700 1 -- 172.21.2.101:0/1290898695 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f158c0ce450 con 0x7f158c0b4f30 2021-02-10T11:36:08.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.608 7f1588ff9700 1 -- 172.21.2.101:0/1290898695 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f157c0021f0 con 0x7f158c0b4f30 2021-02-10T11:36:08.616 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 55 2021-02-10T11:36:08.616 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:36:08.616 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":55,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/3016216705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3016216705},{"type":"v1","addr":"172.21.2.108:6833","nonce":3016216705}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":51}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:02.287195","modified":"2021-02-10 11:36:04.295896","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5900},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5900":{"gid":5900,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.2.101:6835/2977571942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2977571942},{"type":"v1","addr":"172.21.2.101:6835","nonce":2977571942}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-02-10T11:36:08.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.610 7f1592507700 1 -- 172.21.2.101:0/1290898695 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f156c0186c0 msgr2=0x7f156c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:08.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.610 7f1592507700 1 --2- 172.21.2.101:0/1290898695 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f156c0186c0 0x7f156c01abc0 crc :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:08.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.610 7f1592507700 1 -- 172.21.2.101:0/1290898695 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f158c0b4f30 msgr2=0x7f158c18b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:08.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.610 7f1592507700 1 --2- 172.21.2.101:0/1290898695 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f158c0b4f30 0x7f158c18b9d0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 rx=0x7f157c00b5e0 tx=0x7f157c00b980).stop 2021-02-10T11:36:08.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.610 7f1592507700 1 -- 172.21.2.101:0/1290898695 shutdown_connections 2021-02-10T11:36:08.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.610 7f1592507700 1 --2- 172.21.2.101:0/1290898695 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f156c0186c0 0x7f156c01abc0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:08.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.610 7f1592507700 1 --2- 172.21.2.101:0/1290898695 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f158c093f40 0x7f158c094340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:08.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.610 7f1592507700 1 --2- 172.21.2.101:0/1290898695 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f158c094960 0x7f158c187e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:08.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.610 7f1592507700 1 --2- 172.21.2.101:0/1290898695 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f158c0b4f30 0x7f158c18b9d0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:08.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.610 7f1592507700 1 -- 172.21.2.101:0/1290898695 >> 172.21.2.101:0/1290898695 conn(0x7f158c19f8f0 msgr2=0x7f158c1a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:08.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.610 7f1592507700 1 -- 172.21.2.101:0/1290898695 shutdown_connections 2021-02-10T11:36:08.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:08.610 7f1592507700 1 -- 172.21.2.101:0/1290898695 wait complete. 2021-02-10T11:36:08.827 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:36:02.287195', 'modified': '2021-02-10 11:36:04.295896', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5900}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5900': {'gid': 5900, 'name': 'a', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.101:6835/2977571942', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.101:6834', 'nonce': 2977571942}, {'type': 'v1', 'addr': '172.21.2.101:6835', 'nonce': 2977571942}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:36:08.827 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:36:08.828 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok status 2021-02-10T11:36:09.228 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:36:09.229 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:36:09.229 INFO:teuthology.orchestra.run.gibba001.stdout: "whoami": 0, 2021-02-10T11:36:09.229 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 5900, 2021-02-10T11:36:09.229 INFO:teuthology.orchestra.run.gibba001.stdout: "want_state": "up:active", 2021-02-10T11:36:09.229 INFO:teuthology.orchestra.run.gibba001.stdout: "state": "up:active", 2021-02-10T11:36:09.229 INFO:teuthology.orchestra.run.gibba001.stdout: "rank_uptime": 6.9329001149999998, 2021-02-10T11:36:09.229 INFO:teuthology.orchestra.run.gibba001.stdout: "mdsmap_epoch": 55, 2021-02-10T11:36:09.229 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch": 74, 2021-02-10T11:36:09.229 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch_barrier": 74, 2021-02-10T11:36:09.229 INFO:teuthology.orchestra.run.gibba001.stdout: "uptime": 12.558579943 2021-02-10T11:36:09.229 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:36:09.233 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 5900, "want_state": "up:active", "state": "up:active", "rank_uptime": 6.9329001149999998, "mdsmap_epoch": 55, "osdmap_epoch": 74, "osdmap_epoch_barrier": 74, "uptime": 12.558579943 } 2021-02-10T11:36:09.234 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:36:09.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.638 7fba91ad4700 1 Processor -- start 2021-02-10T11:36:09.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.638 7fba91ad4700 1 -- start start 2021-02-10T11:36:09.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.638 7fba91ad4700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fba8c0d0340 0x7fba8c0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:09.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.638 7fba91ad4700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba8c0f0b10 con 0x7fba8c0d1870 2021-02-10T11:36:09.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.638 7fba91ad4700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba8c1b0be0 con 0x7fba8c0d1c70 2021-02-10T11:36:09.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.638 7fba91ad4700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba8c0e5ed0 con 0x7fba8c0d0340 2021-02-10T11:36:09.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.639 7fba8b7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fba8c0d1870 0x7fba8c1b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54218/0 (socket says 172.21.2.101:54218) 2021-02-10T11:36:09.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.639 7fba8b7fe700 1 -- 172.21.2.101:0/9991465 learned_addr learned my addr 172.21.2.101:0/9991465 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:09.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.639 7fba8affd700 1 --2- 172.21.2.101:0/9991465 >> v2:172.21.2.108:3300/0 conn(0x7fba8c0d0340 0x7fba8c0ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:09.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.639 7fba8a7fc700 1 -- 172.21.2.101:0/9991465 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4267362362 0 0) 0x7fba740010b0 con 0x7fba8c0d1870 2021-02-10T11:36:09.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.639 7fba8a7fc700 1 -- 172.21.2.101:0/9991465 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba78002fe0 con 0x7fba8c0d1870 2021-02-10T11:36:09.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.639 7fba8affd700 1 -- 172.21.2.101:0/9991465 >> v1:172.21.2.108:6789/0 conn(0x7fba8c0d1c70 legacy=0x7fba8c0efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:09.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.639 7fba8affd700 1 -- 172.21.2.101:0/9991465 >> v1:172.21.2.101:6789/0 conn(0x7fba8c0d1870 legacy=0x7fba8c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:09.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.639 7fba8a7fc700 1 -- 172.21.2.101:0/9991465 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1186904760 0 0) 0x7fba84001400 con 0x7fba8c0d1c70 2021-02-10T11:36:09.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.639 7fba8affd700 1 -- 172.21.2.101:0/9991465 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8c0f0450 con 0x7fba8c0d0340 2021-02-10T11:36:09.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.639 7fba8affd700 1 --2- 172.21.2.101:0/9991465 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba8c0d0340 0x7fba8c0ef640 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fba7c006650 tx=0x7fba7c009a80).ready entity=mon.1 client_cookie=a4edb08f4e10ecb3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:09.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.640 7fba8a7fc700 1 -- 172.21.2.101:0/9991465 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fba7c00c0c0 con 0x7fba8c0d0340 2021-02-10T11:36:09.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.640 7fba8a7fc700 1 -- 172.21.2.101:0/9991465 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7c00c340 con 0x7fba8c0d0340 2021-02-10T11:36:09.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.640 7fba8a7fc700 1 -- 172.21.2.101:0/9991465 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fba7c00c820 con 0x7fba8c0d0340 2021-02-10T11:36:09.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.640 7fba91ad4700 1 -- 172.21.2.101:0/9991465 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba8c0d0340 msgr2=0x7fba8c0ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:09.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.640 7fba91ad4700 1 --2- 172.21.2.101:0/9991465 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba8c0d0340 0x7fba8c0ef640 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fba7c006650 tx=0x7fba7c009a80).stop 2021-02-10T11:36:09.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.640 7fba91ad4700 1 -- 172.21.2.101:0/9991465 shutdown_connections 2021-02-10T11:36:09.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.640 7fba91ad4700 1 --2- 172.21.2.101:0/9991465 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba8c0d0340 0x7fba8c0ef640 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:09.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.640 7fba91ad4700 1 -- 172.21.2.101:0/9991465 >> 172.21.2.101:0/9991465 conn(0x7fba8c0ce450 msgr2=0x7fba8c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:09.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.640 7fba91ad4700 1 -- 172.21.2.101:0/9991465 shutdown_connections 2021-02-10T11:36:09.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.640 7fba91ad4700 1 -- 172.21.2.101:0/9991465 wait complete. 2021-02-10T11:36:09.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.655 7fba91ad4700 1 Processor -- start 2021-02-10T11:36:09.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.655 7fba91ad4700 1 -- start start 2021-02-10T11:36:09.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.655 7fba91ad4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba8c1b7cf0 0x7fba8c19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:09.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.655 7fba91ad4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba8c1b80f0 0x7fba8c1b84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:09.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.655 7fba91ad4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba8c19da20 0x7fba8c19de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:09.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.655 7fba91ad4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c0a77c0 con 0x7fba8c1b7cf0 2021-02-10T11:36:09.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.655 7fba91ad4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c19d4a0 con 0x7fba8c1b80f0 2021-02-10T11:36:09.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.655 7fba91ad4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c0954f0 con 0x7fba8c19da20 2021-02-10T11:36:09.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.655 7fba8b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba8c1b7cf0 0x7fba8c19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:09.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.656 7fba8b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba8c1b7cf0 0x7fba8c19cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53732/0 (socket says 172.21.2.101:53732) 2021-02-10T11:36:09.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.656 7fba8b7fe700 1 -- 172.21.2.101:0/2904469042 learned_addr learned my addr 172.21.2.101:0/2904469042 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:09.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.656 7fba8b7fe700 1 -- 172.21.2.101:0/2904469042 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba8c19da20 msgr2=0x7fba8c19de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:09.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.656 7fba8b7fe700 1 --2- 172.21.2.101:0/2904469042 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba8c19da20 0x7fba8c19de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:09.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.656 7fba8b7fe700 1 -- 172.21.2.101:0/2904469042 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba8c1b80f0 msgr2=0x7fba8c1b84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:09.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.656 7fba8b7fe700 1 --2- 172.21.2.101:0/2904469042 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba8c1b80f0 0x7fba8c1b84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:09.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.656 7fba8b7fe700 1 -- 172.21.2.101:0/2904469042 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8c1b8d60 con 0x7fba8c1b7cf0 2021-02-10T11:36:09.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.656 7fba8b7fe700 1 --2- 172.21.2.101:0/2904469042 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba8c1b7cf0 0x7fba8c19cdb0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 rx=0x7fba74001c60 tx=0x7fba7400a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:09.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.656 7fba83fff700 1 -- 172.21.2.101:0/2904469042 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fba74000e00 con 0x7fba8c1b7cf0 2021-02-10T11:36:09.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.656 7fba83fff700 1 -- 172.21.2.101:0/2904469042 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7400acc0 con 0x7fba8c1b7cf0 2021-02-10T11:36:09.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.656 7fba83fff700 1 -- 172.21.2.101:0/2904469042 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fba740045d0 con 0x7fba8c1b7cf0 2021-02-10T11:36:09.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.656 7fba91ad4700 1 -- 172.21.2.101:0/2904469042 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba8c1b8d60 con 0x7fba8c1b7cf0 2021-02-10T11:36:09.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.656 7fba91ad4700 1 -- 172.21.2.101:0/2904469042 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba8c19e730 con 0x7fba8c1b7cf0 2021-02-10T11:36:09.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.657 7fba83fff700 1 -- 172.21.2.101:0/2904469042 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fba74004140 con 0x7fba8c1b7cf0 2021-02-10T11:36:09.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.657 7fba83fff700 1 --2- 172.21.2.101:0/2904469042 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba700186c0 0x7fba7001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:09.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.657 7fba83fff700 1 -- 172.21.2.101:0/2904469042 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fba74032580 con 0x7fba8c1b7cf0 2021-02-10T11:36:09.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.657 7fba8affd700 1 --2- 172.21.2.101:0/2904469042 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba700186c0 0x7fba7001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:09.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.658 7fba8affd700 1 --2- 172.21.2.101:0/2904469042 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba700186c0 0x7fba7001abc0 crc :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:09.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.688 7fba91ad4700 1 -- 172.21.2.101:0/2904469042 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba8c0ebc00 con 0x7fba8c1b7cf0 2021-02-10T11:36:09.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.691 7fba83fff700 1 -- 172.21.2.101:0/2904469042 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fba74000960 con 0x7fba8c1b7cf0 2021-02-10T11:36:09.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.815 7fba91ad4700 1 -- 172.21.2.101:0/2904469042 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba8c0f0b10 con 0x7fba8c1b7cf0 2021-02-10T11:36:09.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.816 7fba83fff700 1 -- 172.21.2.101:0/2904469042 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+13996 (secure 0 0 0) 0x7fba7401f2c0 con 0x7fba8c1b7cf0 2021-02-10T11:36:09.822 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:36:09.823 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":74,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:36:03.289219","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:36:00.261115","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2021-02-10 11:36:01.333230","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:36:09.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.817 7fba91ad4700 1 -- 172.21.2.101:0/2904469042 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba700186c0 msgr2=0x7fba7001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:09.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.817 7fba91ad4700 1 --2- 172.21.2.101:0/2904469042 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba700186c0 0x7fba7001abc0 crc :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:09.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.817 7fba91ad4700 1 -- 172.21.2.101:0/2904469042 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba8c1b7cf0 msgr2=0x7fba8c19cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:09.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.817 7fba91ad4700 1 --2- 172.21.2.101:0/2904469042 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba8c1b7cf0 0x7fba8c19cdb0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 rx=0x7fba74001c60 tx=0x7fba7400a8d0).stop 2021-02-10T11:36:09.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.818 7fba91ad4700 1 -- 172.21.2.101:0/2904469042 shutdown_connections 2021-02-10T11:36:09.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.818 7fba91ad4700 1 --2- 172.21.2.101:0/2904469042 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba700186c0 0x7fba7001abc0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:09.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.818 7fba91ad4700 1 --2- 172.21.2.101:0/2904469042 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba8c19da20 0x7fba8c19de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:09.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.818 7fba91ad4700 1 --2- 172.21.2.101:0/2904469042 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba8c1b80f0 0x7fba8c1b84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:09.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.818 7fba91ad4700 1 --2- 172.21.2.101:0/2904469042 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba8c1b7cf0 0x7fba8c19cdb0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:09.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.818 7fba91ad4700 1 -- 172.21.2.101:0/2904469042 >> 172.21.2.101:0/2904469042 conn(0x7fba8c0ed810 msgr2=0x7fba8c094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:09.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.818 7fba91ad4700 1 -- 172.21.2.101:0/2904469042 shutdown_connections 2021-02-10T11:36:09.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:09.818 7fba91ad4700 1 -- 172.21.2.101:0/2904469042 wait complete. 2021-02-10T11:36:09.833 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:36:00.261115', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:36:01.333230', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:36:09.833 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:36:10.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.241 7fef68b06700 1 Processor -- start 2021-02-10T11:36:10.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.242 7fef68b06700 1 -- start start 2021-02-10T11:36:10.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.242 7fef68b06700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef640d1870 0x7fef641b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:10.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.242 7fef68b06700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fef641b0ae0 0x7fef64108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:10.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.242 7fef68b06700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef64109a60 con 0x7fef640d1c70 2021-02-10T11:36:10.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.242 7fef68b06700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef640e5ef0 con 0x7fef641b0ae0 2021-02-10T11:36:10.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.242 7fef68b06700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef64109750 con 0x7fef640d1870 2021-02-10T11:36:10.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.242 7fef6305f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fef640d1c70 0x7fef64108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54236/0 (socket says 172.21.2.101:54236) 2021-02-10T11:36:10.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.242 7fef6305f700 1 -- 172.21.2.101:0/659419183 learned_addr learned my addr 172.21.2.101:0/659419183 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:10.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.243 7fef6185c700 1 -- 172.21.2.101:0/659419183 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2269557023 0 0) 0x7fef5c001500 con 0x7fef640d1c70 2021-02-10T11:36:10.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.243 7fef6185c700 1 -- 172.21.2.101:0/659419183 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef50002fe0 con 0x7fef640d1c70 2021-02-10T11:36:10.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.243 7fef6285e700 1 --2- 172.21.2.101:0/659419183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef640d1870 0x7fef641b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:10.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.243 7fef6205d700 1 --2- 172.21.2.101:0/659419183 >> v2:172.21.2.108:3300/0 conn(0x7fef641b0ae0 0x7fef64108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:10.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.243 7fef6185c700 1 -- 172.21.2.101:0/659419183 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1055981328 0 0) 0x7fef5c001500 con 0x7fef640d1c70 2021-02-10T11:36:10.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.243 7fef6185c700 1 -- 172.21.2.101:0/659419183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef640d1870 msgr2=0x7fef641b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:10.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.243 7fef6185c700 1 --2- 172.21.2.101:0/659419183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef640d1870 0x7fef641b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:10.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.243 7fef6185c700 1 -- 172.21.2.101:0/659419183 >> v2:172.21.2.108:3300/0 conn(0x7fef641b0ae0 msgr2=0x7fef64108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:10.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.243 7fef6185c700 1 --2- 172.21.2.101:0/659419183 >> v2:172.21.2.108:3300/0 conn(0x7fef641b0ae0 0x7fef64108940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:10.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.243 7fef6185c700 1 -- 172.21.2.101:0/659419183 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef64088070 con 0x7fef640d1c70 2021-02-10T11:36:10.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.243 7fef6185c700 1 -- 172.21.2.101:0/659419183 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fef5c001a00 con 0x7fef640d1c70 2021-02-10T11:36:10.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.243 7fef6185c700 1 -- 172.21.2.101:0/659419183 >> v1:172.21.2.101:6789/0 conn(0x7fef640d1c70 legacy=0x7fef64108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:10.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.243 7fef6185c700 1 --2- 172.21.2.101:0/659419183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef50003af0 0x7fef50005f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:10.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.243 7fef6185c700 1 --2- 172.21.2.101:0/659419183 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef50006780 0x7fef50008c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:10.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.243 7fef6185c700 1 --2- 172.21.2.101:0/659419183 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef50009500 0x7fef5000ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:10.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.243 7fef6185c700 1 -- 172.21.2.101:0/659419183 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5000c2d0 con 0x7fef50009500 2021-02-10T11:36:10.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.243 7fef6185c700 1 -- 172.21.2.101:0/659419183 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef500504c0 con 0x7fef50006780 2021-02-10T11:36:10.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.243 7fef6185c700 1 -- 172.21.2.101:0/659419183 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef500506e0 con 0x7fef50003af0 2021-02-10T11:36:10.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.243 7fef6305f700 1 --2- 172.21.2.101:0/659419183 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef50009500 0x7fef5000ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:10.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.243 7fef6285e700 1 --2- 172.21.2.101:0/659419183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef50003af0 0x7fef50005f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:10.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.243 7fef6205d700 1 --2- 172.21.2.101:0/659419183 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef50006780 0x7fef50008c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:10.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.244 7fef6305f700 1 -- 172.21.2.101:0/659419183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef50003af0 msgr2=0x7fef50005f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:10.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.244 7fef6305f700 1 --2- 172.21.2.101:0/659419183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef50003af0 0x7fef50005f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:10.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.244 7fef6305f700 1 -- 172.21.2.101:0/659419183 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef50006780 msgr2=0x7fef50008c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:10.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.244 7fef6305f700 1 --2- 172.21.2.101:0/659419183 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef50006780 0x7fef50008c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:10.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.244 7fef6305f700 1 -- 172.21.2.101:0/659419183 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef50050980 con 0x7fef50009500 2021-02-10T11:36:10.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.244 7fef6305f700 1 --2- 172.21.2.101:0/659419183 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef50009500 0x7fef5000ba20 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 rx=0x7fef5c001860 tx=0x7fef5c00a310).ready entity=mon.0 client_cookie=f9753701970b7cf3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:10.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.244 7fef6185c700 1 -- 172.21.2.101:0/659419183 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fef5c0070c0 con 0x7fef50009500 2021-02-10T11:36:10.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.244 7fef6185c700 1 -- 172.21.2.101:0/659419183 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c0073c0 con 0x7fef50009500 2021-02-10T11:36:10.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.244 7fef6185c700 1 -- 172.21.2.101:0/659419183 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fef5c0049a0 con 0x7fef50009500 2021-02-10T11:36:10.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.244 7fef68b06700 1 -- 172.21.2.101:0/659419183 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef50009500 msgr2=0x7fef5000ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:10.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.244 7fef68b06700 1 --2- 172.21.2.101:0/659419183 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef50009500 0x7fef5000ba20 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 rx=0x7fef5c001860 tx=0x7fef5c00a310).stop 2021-02-10T11:36:10.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.244 7fef6305f700 1 -- 172.21.2.101:0/659419183 reap_dead start 2021-02-10T11:36:10.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.245 7fef68b06700 1 -- 172.21.2.101:0/659419183 shutdown_connections 2021-02-10T11:36:10.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.245 7fef68b06700 1 -- 172.21.2.101:0/659419183 >> 172.21.2.101:0/659419183 conn(0x7fef640ce420 msgr2=0x7fef640d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:10.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.245 7fef68b06700 1 -- 172.21.2.101:0/659419183 shutdown_connections 2021-02-10T11:36:10.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.245 7fef68b06700 1 -- 172.21.2.101:0/659419183 wait complete. 2021-02-10T11:36:10.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.260 7fef68b06700 1 Processor -- start 2021-02-10T11:36:10.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.260 7fef68b06700 1 -- start start 2021-02-10T11:36:10.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.260 7fef68b06700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef64085bc0 0x7fef640863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:10.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.260 7fef68b06700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef640954c0 0x7fef64096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:10.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.260 7fef68b06700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef640958c0 0x7fef64096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:10.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.260 7fef68b06700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef64095d80 con 0x7fef640958c0 2021-02-10T11:36:10.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.260 7fef68b06700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef640bfa60 con 0x7fef64085bc0 2021-02-10T11:36:10.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.260 7fef68b06700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef64085fc0 con 0x7fef640954c0 2021-02-10T11:36:10.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.260 7fef6305f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef640958c0 0x7fef64096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:10.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.260 7fef6305f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef640958c0 0x7fef64096920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53752/0 (socket says 172.21.2.101:53752) 2021-02-10T11:36:10.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.260 7fef6305f700 1 -- 172.21.2.101:0/3171178919 learned_addr learned my addr 172.21.2.101:0/3171178919 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:10.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.261 7fef6305f700 1 -- 172.21.2.101:0/3171178919 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef640954c0 msgr2=0x7fef64096300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:10.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.261 7fef6305f700 1 --2- 172.21.2.101:0/3171178919 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef640954c0 0x7fef64096300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:10.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.261 7fef6305f700 1 -- 172.21.2.101:0/3171178919 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef64085bc0 msgr2=0x7fef640863e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:10.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.261 7fef6305f700 1 --2- 172.21.2.101:0/3171178919 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef64085bc0 0x7fef640863e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:10.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.261 7fef6305f700 1 -- 172.21.2.101:0/3171178919 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef64086c10 con 0x7fef640958c0 2021-02-10T11:36:10.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.261 7fef6305f700 1 --2- 172.21.2.101:0/3171178919 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef640958c0 0x7fef64096920 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 rx=0x7fef5c00a940 tx=0x7fef5c00ace0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:10.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.261 7fef5b7fe700 1 -- 172.21.2.101:0/3171178919 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fef5c004930 con 0x7fef640958c0 2021-02-10T11:36:10.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.261 7fef5b7fe700 1 -- 172.21.2.101:0/3171178919 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c004b70 con 0x7fef640958c0 2021-02-10T11:36:10.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.261 7fef5b7fe700 1 -- 172.21.2.101:0/3171178919 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fef5c007020 con 0x7fef640958c0 2021-02-10T11:36:10.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.261 7fef68b06700 1 -- 172.21.2.101:0/3171178919 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef64086c10 con 0x7fef640958c0 2021-02-10T11:36:10.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.261 7fef68b06700 1 -- 172.21.2.101:0/3171178919 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef64097230 con 0x7fef640958c0 2021-02-10T11:36:10.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.262 7fef5b7fe700 1 -- 172.21.2.101:0/3171178919 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fef5c004720 con 0x7fef640958c0 2021-02-10T11:36:10.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.262 7fef5b7fe700 1 --2- 172.21.2.101:0/3171178919 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fef480186c0 0x7fef4801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:10.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.262 7fef5b7fe700 1 -- 172.21.2.101:0/3171178919 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fef5c032920 con 0x7fef640958c0 2021-02-10T11:36:10.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.262 7fef6285e700 1 --2- 172.21.2.101:0/3171178919 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fef480186c0 0x7fef4801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:10.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.262 7fef6285e700 1 --2- 172.21.2.101:0/3171178919 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fef480186c0 0x7fef4801abc0 crc :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:10.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.292 7fef68b06700 1 -- 172.21.2.101:0/3171178919 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef640fe1c0 con 0x7fef640958c0 2021-02-10T11:36:10.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.294 7fef5b7fe700 1 -- 172.21.2.101:0/3171178919 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fef5c026090 con 0x7fef640958c0 2021-02-10T11:36:10.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.422 7fef68b06700 1 -- 172.21.2.101:0/3171178919 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fef640fe1c0 con 0x7fef640958c0 2021-02-10T11:36:10.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.423 7fef5b7fe700 1 -- 172.21.2.101:0/3171178919 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v74) v1 ==== 92+0+10 (secure 0 0 0) 0x7fef5c026090 con 0x7fef640958c0 2021-02-10T11:36:10.429 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:36:10.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.425 7fef68b06700 1 -- 172.21.2.101:0/3171178919 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fef480186c0 msgr2=0x7fef4801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:10.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.425 7fef68b06700 1 --2- 172.21.2.101:0/3171178919 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fef480186c0 0x7fef4801abc0 crc :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:10.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.425 7fef68b06700 1 -- 172.21.2.101:0/3171178919 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef640958c0 msgr2=0x7fef64096920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:10.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.425 7fef68b06700 1 --2- 172.21.2.101:0/3171178919 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef640958c0 0x7fef64096920 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 rx=0x7fef5c00a940 tx=0x7fef5c00ace0).stop 2021-02-10T11:36:10.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.425 7fef68b06700 1 -- 172.21.2.101:0/3171178919 shutdown_connections 2021-02-10T11:36:10.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.425 7fef68b06700 1 --2- 172.21.2.101:0/3171178919 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fef480186c0 0x7fef4801abc0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:10.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.425 7fef68b06700 1 --2- 172.21.2.101:0/3171178919 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef640954c0 0x7fef64096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:10.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.425 7fef68b06700 1 --2- 172.21.2.101:0/3171178919 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef640958c0 0x7fef64096920 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:10.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.425 7fef68b06700 1 --2- 172.21.2.101:0/3171178919 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef64085bc0 0x7fef640863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:10.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.425 7fef68b06700 1 -- 172.21.2.101:0/3171178919 >> 172.21.2.101:0/3171178919 conn(0x7fef640bef00 msgr2=0x7fef6409d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:10.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.425 7fef68b06700 1 -- 172.21.2.101:0/3171178919 shutdown_connections 2021-02-10T11:36:10.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.425 7fef68b06700 1 -- 172.21.2.101:0/3171178919 wait complete. 2021-02-10T11:36:10.439 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:36:10.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.845 7f0c28032700 1 Processor -- start 2021-02-10T11:36:10.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.845 7f0c28032700 1 -- start start 2021-02-10T11:36:10.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.846 7f0c28032700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0c200d1870 0x7f0c201b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:10.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.846 7f0c28032700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c201b0ae0 0x7f0c20108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:10.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.846 7f0c28032700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c200fe1c0 con 0x7f0c200d1c70 2021-02-10T11:36:10.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.846 7f0c28032700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c20109a60 con 0x7f0c200d1870 2021-02-10T11:36:10.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.846 7f0c28032700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c20109750 con 0x7f0c201b0ae0 2021-02-10T11:36:10.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.846 7f0c25e4e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0c200d1870 0x7f0c201b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:10.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.846 7f0c25e4e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0c200d1870 0x7f0c201b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53758/0 (socket says 172.21.2.101:53758) 2021-02-10T11:36:10.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.846 7f0c25e4e700 1 -- 172.21.2.101:0/2803823648 learned_addr learned my addr 172.21.2.101:0/2803823648 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:10.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.846 7f0c25e4e700 1 -- 172.21.2.101:0/2803823648 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c201b0ae0 msgr2=0x7f0c20108940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:36:10.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.846 7f0c25e4e700 1 --2- 172.21.2.101:0/2803823648 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c201b0ae0 0x7f0c20108940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:10.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.846 7f0c25e4e700 1 -- 172.21.2.101:0/2803823648 >> v1:172.21.2.108:6789/0 conn(0x7f0c200d1c70 legacy=0x7f0c20108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:10.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.846 7f0c25e4e700 1 -- 172.21.2.101:0/2803823648 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c20088070 con 0x7f0c200d1870 2021-02-10T11:36:10.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.846 7f0c25e4e700 1 --2- 172.21.2.101:0/2803823648 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c200d1870 0x7f0c201b04c0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 rx=0x7f0c1c006700 tx=0x7f0c1c009b00).ready entity=mon.0 client_cookie=85e6a49b3ae93ab2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:10.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.847 7f0c24e4c700 1 -- 172.21.2.101:0/2803823648 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0c1c00c0c0 con 0x7f0c200d1870 2021-02-10T11:36:10.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.847 7f0c24e4c700 1 -- 172.21.2.101:0/2803823648 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1c00c400 con 0x7f0c200d1870 2021-02-10T11:36:10.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.847 7f0c24e4c700 1 -- 172.21.2.101:0/2803823648 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0c1c00c920 con 0x7f0c200d1870 2021-02-10T11:36:10.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.847 7f0c28032700 1 -- 172.21.2.101:0/2803823648 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c200d1870 msgr2=0x7f0c201b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:10.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.847 7f0c28032700 1 --2- 172.21.2.101:0/2803823648 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c200d1870 0x7f0c201b04c0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 rx=0x7f0c1c006700 tx=0x7f0c1c009b00).stop 2021-02-10T11:36:10.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.848 7f0c28032700 1 -- 172.21.2.101:0/2803823648 shutdown_connections 2021-02-10T11:36:10.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.848 7f0c28032700 1 --2- 172.21.2.101:0/2803823648 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c201b0ae0 0x7f0c20108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:10.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.848 7f0c28032700 1 --2- 172.21.2.101:0/2803823648 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c200d1870 0x7f0c201b04c0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:10.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.848 7f0c28032700 1 -- 172.21.2.101:0/2803823648 >> 172.21.2.101:0/2803823648 conn(0x7f0c200ce420 msgr2=0x7f0c200d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:10.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.848 7f0c28032700 1 -- 172.21.2.101:0/2803823648 shutdown_connections 2021-02-10T11:36:10.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.848 7f0c28032700 1 -- 172.21.2.101:0/2803823648 wait complete. 2021-02-10T11:36:10.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.862 7f0c28032700 1 Processor -- start 2021-02-10T11:36:10.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.863 7f0c28032700 1 -- start start 2021-02-10T11:36:10.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.863 7f0c28032700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c200c3820 0x7f0c20084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:10.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.863 7f0c28032700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c200a0290 0x7f0c200852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:10.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.863 7f0c28032700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c20086270 0x7f0c20086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:10.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.863 7f0c28032700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200c0700 con 0x7f0c20086270 2021-02-10T11:36:10.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.863 7f0c28032700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200858f0 con 0x7f0c200c3820 2021-02-10T11:36:10.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.863 7f0c28032700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c20085b10 con 0x7f0c200a0290 2021-02-10T11:36:10.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.863 7f0c2664f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c20086270 0x7f0c20086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:10.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.863 7f0c2664f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c20086270 0x7f0c20086670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53768/0 (socket says 172.21.2.101:53768) 2021-02-10T11:36:10.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.863 7f0c2664f700 1 -- 172.21.2.101:0/872951506 learned_addr learned my addr 172.21.2.101:0/872951506 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:10.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.863 7f0c2664f700 1 -- 172.21.2.101:0/872951506 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c200a0290 msgr2=0x7f0c200852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:10.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.863 7f0c2664f700 1 --2- 172.21.2.101:0/872951506 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c200a0290 0x7f0c200852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:10.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.863 7f0c2664f700 1 -- 172.21.2.101:0/872951506 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c200c3820 msgr2=0x7f0c20084cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:10.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.863 7f0c2664f700 1 --2- 172.21.2.101:0/872951506 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c200c3820 0x7f0c20084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:10.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.863 7f0c2664f700 1 -- 172.21.2.101:0/872951506 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c20086c90 con 0x7f0c20086270 2021-02-10T11:36:10.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.864 7f0c2664f700 1 --2- 172.21.2.101:0/872951506 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c20086270 0x7f0c20086670 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 rx=0x7f0c18009940 tx=0x7f0c1800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:10.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.864 7f0c127fc700 1 -- 172.21.2.101:0/872951506 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0c18000d30 con 0x7f0c20086270 2021-02-10T11:36:10.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.864 7f0c127fc700 1 -- 172.21.2.101:0/872951506 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c18000f90 con 0x7f0c20086270 2021-02-10T11:36:10.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.864 7f0c127fc700 1 -- 172.21.2.101:0/872951506 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0c1800c570 con 0x7f0c20086270 2021-02-10T11:36:10.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.864 7f0c28032700 1 -- 172.21.2.101:0/872951506 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c20086c90 con 0x7f0c20086270 2021-02-10T11:36:10.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.864 7f0c28032700 1 -- 172.21.2.101:0/872951506 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c20086c90 con 0x7f0c20086270 2021-02-10T11:36:10.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.865 7f0c127fc700 1 -- 172.21.2.101:0/872951506 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0c18001220 con 0x7f0c20086270 2021-02-10T11:36:10.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.865 7f0c127fc700 1 --2- 172.21.2.101:0/872951506 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c080186c0 0x7f0c0801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:10.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.865 7f0c127fc700 1 -- 172.21.2.101:0/872951506 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0c18032920 con 0x7f0c20086270 2021-02-10T11:36:10.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.865 7f0c25e4e700 1 --2- 172.21.2.101:0/872951506 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c080186c0 0x7f0c0801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:10.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.865 7f0c25e4e700 1 --2- 172.21.2.101:0/872951506 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c080186c0 0x7f0c0801abc0 crc :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:10.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.896 7f0c28032700 1 -- 172.21.2.101:0/872951506 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c20082a70 con 0x7f0c20086270 2021-02-10T11:36:10.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:10.899 7f0c127fc700 1 -- 172.21.2.101:0/872951506 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0c1800be30 con 0x7f0c20086270 2021-02-10T11:36:11.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.025 7f0c28032700 1 -- 172.21.2.101:0/872951506 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f0c20082a70 con 0x7f0c20086270 2021-02-10T11:36:11.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.026 7f0c127fc700 1 -- 172.21.2.101:0/872951506 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v74) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0c18001220 con 0x7f0c20086270 2021-02-10T11:36:11.033 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:36:11.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.028 7f0c28032700 1 -- 172.21.2.101:0/872951506 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c080186c0 msgr2=0x7f0c0801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:11.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.028 7f0c28032700 1 --2- 172.21.2.101:0/872951506 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c080186c0 0x7f0c0801abc0 crc :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:11.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.028 7f0c28032700 1 -- 172.21.2.101:0/872951506 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c20086270 msgr2=0x7f0c20086670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:11.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.028 7f0c28032700 1 --2- 172.21.2.101:0/872951506 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c20086270 0x7f0c20086670 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 rx=0x7f0c18009940 tx=0x7f0c1800bcf0).stop 2021-02-10T11:36:11.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.028 7f0c28032700 1 -- 172.21.2.101:0/872951506 shutdown_connections 2021-02-10T11:36:11.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.028 7f0c28032700 1 --2- 172.21.2.101:0/872951506 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c080186c0 0x7f0c0801abc0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:11.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.028 7f0c28032700 1 --2- 172.21.2.101:0/872951506 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c200a0290 0x7f0c200852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:11.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.028 7f0c28032700 1 --2- 172.21.2.101:0/872951506 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c20086270 0x7f0c20086670 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:11.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.028 7f0c28032700 1 --2- 172.21.2.101:0/872951506 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c200c3820 0x7f0c20084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:11.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.028 7f0c28032700 1 -- 172.21.2.101:0/872951506 >> 172.21.2.101:0/872951506 conn(0x7f0c200872c0 msgr2=0x7f0c20095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:11.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.028 7f0c28032700 1 -- 172.21.2.101:0/872951506 shutdown_connections 2021-02-10T11:36:11.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.028 7f0c28032700 1 -- 172.21.2.101:0/872951506 wait complete. 2021-02-10T11:36:11.044 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:36:11.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.449 7f8e73c62700 1 Processor -- start 2021-02-10T11:36:11.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.449 7f8e73c62700 1 -- start start 2021-02-10T11:36:11.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.449 7f8e73c62700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8e6c0d1870 0x7f8e6c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:11.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.449 7f8e73c62700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8e6c1b0ae0 0x7f8e6c108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:11.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.449 7f8e73c62700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e6c0fe1c0 con 0x7f8e6c0d1c70 2021-02-10T11:36:11.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.449 7f8e73c62700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e6c109a60 con 0x7f8e6c0d1870 2021-02-10T11:36:11.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.449 7f8e73c62700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e6c109750 con 0x7f8e6c1b0ae0 2021-02-10T11:36:11.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.449 7f8e71a7e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8e6c0d1870 0x7f8e6c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.449 7f8e71a7e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8e6c0d1870 0x7f8e6c1b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53772/0 (socket says 172.21.2.101:53772) 2021-02-10T11:36:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.449 7f8e71a7e700 1 -- 172.21.2.101:0/2115210863 learned_addr learned my addr 172.21.2.101:0/2115210863 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.450 7f8e71a7e700 1 -- 172.21.2.101:0/2115210863 >> v2:172.21.2.108:3300/0 conn(0x7f8e6c1b0ae0 msgr2=0x7f8e6c108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.450 7f8e71a7e700 1 --2- 172.21.2.101:0/2115210863 >> v2:172.21.2.108:3300/0 conn(0x7f8e6c1b0ae0 0x7f8e6c108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.450 7f8e71a7e700 1 -- 172.21.2.101:0/2115210863 >> v1:172.21.2.108:6789/0 conn(0x7f8e6c0d1c70 legacy=0x7f8e6c108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.450 7f8e71a7e700 1 -- 172.21.2.101:0/2115210863 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6c088070 con 0x7f8e6c0d1870 2021-02-10T11:36:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.450 7f8e71a7e700 1 --2- 172.21.2.101:0/2115210863 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e6c0d1870 0x7f8e6c1b04c0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 rx=0x7f8e68006700 tx=0x7f8e68009b80).ready entity=mon.0 client_cookie=53f31800a98e8e5a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.450 7f8e70a7c700 1 -- 172.21.2.101:0/2115210863 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8e6800c0c0 con 0x7f8e6c0d1870 2021-02-10T11:36:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.450 7f8e70a7c700 1 -- 172.21.2.101:0/2115210863 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6800c340 con 0x7f8e6c0d1870 2021-02-10T11:36:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.450 7f8e70a7c700 1 -- 172.21.2.101:0/2115210863 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8e6800c820 con 0x7f8e6c0d1870 2021-02-10T11:36:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.450 7f8e73c62700 1 -- 172.21.2.101:0/2115210863 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e6c0d1870 msgr2=0x7f8e6c1b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.450 7f8e73c62700 1 --2- 172.21.2.101:0/2115210863 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e6c0d1870 0x7f8e6c1b04c0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 rx=0x7f8e68006700 tx=0x7f8e68009b80).stop 2021-02-10T11:36:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.450 7f8e73c62700 1 -- 172.21.2.101:0/2115210863 shutdown_connections 2021-02-10T11:36:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.450 7f8e73c62700 1 --2- 172.21.2.101:0/2115210863 >> v2:172.21.2.108:3300/0 conn(0x7f8e6c1b0ae0 0x7f8e6c108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:11.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.450 7f8e73c62700 1 --2- 172.21.2.101:0/2115210863 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e6c0d1870 0x7f8e6c1b04c0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:11.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.450 7f8e73c62700 1 -- 172.21.2.101:0/2115210863 >> 172.21.2.101:0/2115210863 conn(0x7f8e6c0ce420 msgr2=0x7f8e6c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:11.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.451 7f8e73c62700 1 -- 172.21.2.101:0/2115210863 shutdown_connections 2021-02-10T11:36:11.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.451 7f8e73c62700 1 -- 172.21.2.101:0/2115210863 wait complete. 2021-02-10T11:36:11.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.465 7f8e73c62700 1 Processor -- start 2021-02-10T11:36:11.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.465 7f8e73c62700 1 -- start start 2021-02-10T11:36:11.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.466 7f8e73c62700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e6c0c2370 0x7f8e6c18f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:11.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.466 7f8e73c62700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8e6c084cb0 0x7f8e6c1a7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:11.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.466 7f8e73c62700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8e6c1a7a90 0x7f8e6c085700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:11.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.466 7f8e73c62700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c0be840 con 0x7f8e6c0c2370 2021-02-10T11:36:11.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.466 7f8e73c62700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c0c29a0 con 0x7f8e6c084cb0 2021-02-10T11:36:11.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.466 7f8e73c62700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c1a7e90 con 0x7f8e6c1a7a90 2021-02-10T11:36:11.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.466 7f8e71a7e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e6c0c2370 0x7f8e6c18f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:11.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.466 7f8e71a7e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e6c0c2370 0x7f8e6c18f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53778/0 (socket says 172.21.2.101:53778) 2021-02-10T11:36:11.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.466 7f8e71a7e700 1 -- 172.21.2.101:0/2343969952 learned_addr learned my addr 172.21.2.101:0/2343969952 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:11.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.466 7f8e71a7e700 1 -- 172.21.2.101:0/2343969952 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8e6c1a7a90 msgr2=0x7f8e6c085700 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:11.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.466 7f8e71a7e700 1 --2- 172.21.2.101:0/2343969952 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8e6c1a7a90 0x7f8e6c085700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:11.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.466 7f8e71a7e700 1 -- 172.21.2.101:0/2343969952 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8e6c084cb0 msgr2=0x7f8e6c1a7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:11.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.466 7f8e71a7e700 1 --2- 172.21.2.101:0/2343969952 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8e6c084cb0 0x7f8e6c1a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:11.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.466 7f8e71a7e700 1 -- 172.21.2.101:0/2343969952 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6c1a80b0 con 0x7f8e6c0c2370 2021-02-10T11:36:11.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.466 7f8e71a7e700 1 --2- 172.21.2.101:0/2343969952 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e6c0c2370 0x7f8e6c18f870 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 rx=0x7f8e6800b650 tx=0x7f8e6800b9f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:11.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.467 7f8e627fc700 1 -- 172.21.2.101:0/2343969952 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8e680090b0 con 0x7f8e6c0c2370 2021-02-10T11:36:11.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.467 7f8e627fc700 1 -- 172.21.2.101:0/2343969952 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6800bde0 con 0x7f8e6c0c2370 2021-02-10T11:36:11.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.467 7f8e627fc700 1 -- 172.21.2.101:0/2343969952 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8e68002220 con 0x7f8e6c0c2370 2021-02-10T11:36:11.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.467 7f8e73c62700 1 -- 172.21.2.101:0/2343969952 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e6c1a80b0 con 0x7f8e6c0c2370 2021-02-10T11:36:11.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.467 7f8e73c62700 1 -- 172.21.2.101:0/2343969952 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e6c1a80b0 con 0x7f8e6c0c2370 2021-02-10T11:36:11.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.467 7f8e627fc700 1 -- 172.21.2.101:0/2343969952 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8e680064d0 con 0x7f8e6c0c2370 2021-02-10T11:36:11.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.467 7f8e627fc700 1 --2- 172.21.2.101:0/2343969952 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8e540186c0 0x7f8e5401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:11.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.467 7f8e627fc700 1 -- 172.21.2.101:0/2343969952 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8e68032570 con 0x7f8e6c0c2370 2021-02-10T11:36:11.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.467 7f8e7127d700 1 --2- 172.21.2.101:0/2343969952 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8e540186c0 0x7f8e5401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:11.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.468 7f8e7127d700 1 --2- 172.21.2.101:0/2343969952 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8e540186c0 0x7f8e5401abc0 crc :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:11.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.499 7f8e73c62700 1 -- 172.21.2.101:0/2343969952 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e6c082a70 con 0x7f8e6c0c2370 2021-02-10T11:36:11.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.502 7f8e627fc700 1 -- 172.21.2.101:0/2343969952 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8e680219e0 con 0x7f8e6c0c2370 2021-02-10T11:36:11.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.628 7f8e73c62700 1 -- 172.21.2.101:0/2343969952 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f8e6c082a70 con 0x7f8e6c0c2370 2021-02-10T11:36:11.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.628 7f8e627fc700 1 -- 172.21.2.101:0/2343969952 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v74) v1 ==== 100+0+10 (secure 0 0 0) 0x7f8e680219e0 con 0x7f8e6c0c2370 2021-02-10T11:36:11.636 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:36:11.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.630 7f8e73c62700 1 -- 172.21.2.101:0/2343969952 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8e540186c0 msgr2=0x7f8e5401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:11.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.630 7f8e73c62700 1 --2- 172.21.2.101:0/2343969952 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8e540186c0 0x7f8e5401abc0 crc :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:11.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.630 7f8e73c62700 1 -- 172.21.2.101:0/2343969952 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e6c0c2370 msgr2=0x7f8e6c18f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:11.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.630 7f8e73c62700 1 --2- 172.21.2.101:0/2343969952 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e6c0c2370 0x7f8e6c18f870 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 rx=0x7f8e6800b650 tx=0x7f8e6800b9f0).stop 2021-02-10T11:36:11.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.630 7f8e73c62700 1 -- 172.21.2.101:0/2343969952 shutdown_connections 2021-02-10T11:36:11.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.630 7f8e73c62700 1 --2- 172.21.2.101:0/2343969952 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8e540186c0 0x7f8e5401abc0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:11.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.630 7f8e73c62700 1 --2- 172.21.2.101:0/2343969952 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8e6c1a7a90 0x7f8e6c085700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:11.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.630 7f8e73c62700 1 --2- 172.21.2.101:0/2343969952 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8e6c084cb0 0x7f8e6c1a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:11.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.630 7f8e73c62700 1 --2- 172.21.2.101:0/2343969952 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8e6c0c2370 0x7f8e6c18f870 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:11.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.630 7f8e73c62700 1 -- 172.21.2.101:0/2343969952 >> 172.21.2.101:0/2343969952 conn(0x7f8e6c0b5dc0 msgr2=0x7f8e6c0c2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:11.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.631 7f8e73c62700 1 -- 172.21.2.101:0/2343969952 shutdown_connections 2021-02-10T11:36:11.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:11.631 7f8e73c62700 1 -- 172.21.2.101:0/2343969952 wait complete. 2021-02-10T11:36:11.646 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:36:11.646 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:36:12.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.050 7f9f59103700 1 Processor -- start 2021-02-10T11:36:12.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.050 7f9f59103700 1 -- start start 2021-02-10T11:36:12.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.051 7f9f59103700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f9f540d1870 0x7f9f541b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:12.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.051 7f9f59103700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f540f0650 0x7f9f540cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:12.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.051 7f9f59103700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f540f0b10 con 0x7f9f541b0a10 2021-02-10T11:36:12.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.051 7f9f59103700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f540e5ed0 con 0x7f9f540d1870 2021-02-10T11:36:12.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.051 7f9f59103700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f540d1c70 con 0x7f9f540f0650 2021-02-10T11:36:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.052 7f9f5359e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f540f0650 0x7f9f540cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.052 7f9f5359e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f540f0650 0x7f9f540cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:46522/0 (socket says 172.21.2.101:46522) 2021-02-10T11:36:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.052 7f9f5359e700 1 -- 172.21.2.101:0/933958805 learned_addr learned my addr 172.21.2.101:0/933958805 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.052 7f9f5259c700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f9f541b0a10 0x7f9f540cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:47692/0 (socket says 172.21.2.101:47692) 2021-02-10T11:36:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.052 7f9f52d9d700 1 --2- 172.21.2.101:0/933958805 >> v2:172.21.2.108:3300/0 conn(0x7f9f540d1870 0x7f9f541b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.052 7f9f51d9b700 1 -- 172.21.2.101:0/933958805 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3508567357 0 0) 0x7f9f3c0010b0 con 0x7f9f541b0a10 2021-02-10T11:36:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.052 7f9f5359e700 1 -- 172.21.2.101:0/933958805 >> v2:172.21.2.108:3300/0 conn(0x7f9f540d1870 msgr2=0x7f9f541b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.052 7f9f5359e700 1 --2- 172.21.2.101:0/933958805 >> v2:172.21.2.108:3300/0 conn(0x7f9f540d1870 0x7f9f541b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.052 7f9f5359e700 1 -- 172.21.2.101:0/933958805 >> v1:172.21.2.108:6789/0 conn(0x7f9f541b0a10 legacy=0x7f9f540cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.052 7f9f5359e700 1 -- 172.21.2.101:0/933958805 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f540d1e90 con 0x7f9f540f0650 2021-02-10T11:36:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.052 7f9f5359e700 1 --2- 172.21.2.101:0/933958805 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f540f0650 0x7f9f540cfb20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f9f4c006aa0 tx=0x7f9f4c009f20).ready entity=mon.2 client_cookie=4bdd487aaa0ef753 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.053 7f9f51d9b700 1 -- 172.21.2.101:0/933958805 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9f4c00b5c0 con 0x7f9f540f0650 2021-02-10T11:36:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.053 7f9f51d9b700 1 -- 172.21.2.101:0/933958805 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4c003030 con 0x7f9f540f0650 2021-02-10T11:36:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.053 7f9f51d9b700 1 -- 172.21.2.101:0/933958805 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9f4c00b9f0 con 0x7f9f540f0650 2021-02-10T11:36:12.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.053 7f9f59103700 1 -- 172.21.2.101:0/933958805 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f540f0650 msgr2=0x7f9f540cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:12.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.053 7f9f59103700 1 --2- 172.21.2.101:0/933958805 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f540f0650 0x7f9f540cfb20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f9f4c006aa0 tx=0x7f9f4c009f20).stop 2021-02-10T11:36:12.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.054 7f9f59103700 1 -- 172.21.2.101:0/933958805 shutdown_connections 2021-02-10T11:36:12.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.054 7f9f59103700 1 --2- 172.21.2.101:0/933958805 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f540f0650 0x7f9f540cfb20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:12.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.054 7f9f59103700 1 --2- 172.21.2.101:0/933958805 >> v2:172.21.2.108:3300/0 conn(0x7f9f540d1870 0x7f9f541b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:12.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.054 7f9f59103700 1 -- 172.21.2.101:0/933958805 >> 172.21.2.101:0/933958805 conn(0x7f9f540ce450 msgr2=0x7f9f540d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:12.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.054 7f9f59103700 1 -- 172.21.2.101:0/933958805 shutdown_connections 2021-02-10T11:36:12.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.054 7f9f59103700 1 -- 172.21.2.101:0/933958805 wait complete. 2021-02-10T11:36:12.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.068 7f9f59103700 1 Processor -- start 2021-02-10T11:36:12.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.068 7f9f59103700 1 -- start start 2021-02-10T11:36:12.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.069 7f9f59103700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f541942b0 0x7f9f540b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:12.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.069 7f9f59103700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f54195070 0x7f9f54195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:12.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.069 7f9f59103700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f540bb8e0 0x7f9f540bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:12.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.069 7f9f59103700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541946b0 con 0x7f9f54195070 2021-02-10T11:36:12.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.069 7f9f59103700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f540b3b70 con 0x7f9f541942b0 2021-02-10T11:36:12.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.069 7f9f59103700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f540b3d90 con 0x7f9f540bb8e0 2021-02-10T11:36:12.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.069 7f9f5259c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f54195070 0x7f9f54195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:12.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.069 7f9f5259c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f54195070 0x7f9f54195470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53796/0 (socket says 172.21.2.101:53796) 2021-02-10T11:36:12.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.069 7f9f5259c700 1 -- 172.21.2.101:0/4275396418 learned_addr learned my addr 172.21.2.101:0/4275396418 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:12.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.069 7f9f5359e700 1 --2- 172.21.2.101:0/4275396418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f540bb8e0 0x7f9f540bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:12.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.069 7f9f52d9d700 1 --2- 172.21.2.101:0/4275396418 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f541942b0 0x7f9f540b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:12.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.069 7f9f5259c700 1 -- 172.21.2.101:0/4275396418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f540bb8e0 msgr2=0x7f9f540bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:12.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.069 7f9f5259c700 1 --2- 172.21.2.101:0/4275396418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f540bb8e0 0x7f9f540bbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:12.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.069 7f9f5259c700 1 -- 172.21.2.101:0/4275396418 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f541942b0 msgr2=0x7f9f540b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:12.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.069 7f9f5259c700 1 --2- 172.21.2.101:0/4275396418 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f541942b0 0x7f9f540b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:12.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.069 7f9f5259c700 1 -- 172.21.2.101:0/4275396418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f541a49e0 con 0x7f9f54195070 2021-02-10T11:36:12.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.069 7f9f5359e700 1 --2- 172.21.2.101:0/4275396418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f540bb8e0 0x7f9f540bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:36:12.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.069 7f9f52d9d700 1 --2- 172.21.2.101:0/4275396418 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f541942b0 0x7f9f540b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:36:12.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.070 7f9f5259c700 1 --2- 172.21.2.101:0/4275396418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f54195070 0x7f9f54195470 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 rx=0x7f9f3c001c10 tx=0x7f9f3c00a8f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.070 7f9f4b7fe700 1 -- 172.21.2.101:0/4275396418 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9f3c0009b0 con 0x7f9f54195070 2021-02-10T11:36:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.070 7f9f4b7fe700 1 -- 172.21.2.101:0/4275396418 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3c0012c0 con 0x7f9f54195070 2021-02-10T11:36:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.070 7f9f4b7fe700 1 -- 172.21.2.101:0/4275396418 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9f3c004350 con 0x7f9f54195070 2021-02-10T11:36:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.070 7f9f59103700 1 -- 172.21.2.101:0/4275396418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f541a49e0 con 0x7f9f54195070 2021-02-10T11:36:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.070 7f9f59103700 1 -- 172.21.2.101:0/4275396418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f541a49e0 con 0x7f9f54195070 2021-02-10T11:36:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.070 7f9f4b7fe700 1 -- 172.21.2.101:0/4275396418 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9f3c004140 con 0x7f9f54195070 2021-02-10T11:36:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.071 7f9f4b7fe700 1 --2- 172.21.2.101:0/4275396418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f380186c0 0x7f9f3801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.071 7f9f4b7fe700 1 -- 172.21.2.101:0/4275396418 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9f3c032480 con 0x7f9f54195070 2021-02-10T11:36:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.071 7f9f52d9d700 1 --2- 172.21.2.101:0/4275396418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f380186c0 0x7f9f3801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.071 7f9f52d9d700 1 --2- 172.21.2.101:0/4275396418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f380186c0 0x7f9f3801abc0 crc :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:12.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.101 7f9f59103700 1 -- 172.21.2.101:0/4275396418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f540bc5f0 con 0x7f9f54195070 2021-02-10T11:36:12.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.103 7f9f4b7fe700 1 -- 172.21.2.101:0/4275396418 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9f3c030db0 con 0x7f9f54195070 2021-02-10T11:36:12.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.231 7f9f59103700 1 -- 172.21.2.101:0/4275396418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f540bc5f0 con 0x7f9f54195070 2021-02-10T11:36:12.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.231 7f9f4b7fe700 1 -- 172.21.2.101:0/4275396418 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f9f3c004140 con 0x7f9f54195070 2021-02-10T11:36:12.239 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 55 2021-02-10T11:36:12.239 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:36:12.239 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":55,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/3016216705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3016216705},{"type":"v1","addr":"172.21.2.108:6833","nonce":3016216705}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":51}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:02.287195","modified":"2021-02-10 11:36:04.295896","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5900},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5900":{"gid":5900,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.2.101:6835/2977571942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2977571942},{"type":"v1","addr":"172.21.2.101:6835","nonce":2977571942}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-02-10T11:36:12.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.233 7f9f59103700 1 -- 172.21.2.101:0/4275396418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f380186c0 msgr2=0x7f9f3801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:12.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.233 7f9f59103700 1 --2- 172.21.2.101:0/4275396418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f380186c0 0x7f9f3801abc0 crc :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:12.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.233 7f9f59103700 1 -- 172.21.2.101:0/4275396418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f54195070 msgr2=0x7f9f54195470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:12.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.233 7f9f59103700 1 --2- 172.21.2.101:0/4275396418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f54195070 0x7f9f54195470 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 rx=0x7f9f3c001c10 tx=0x7f9f3c00a8f0).stop 2021-02-10T11:36:12.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.233 7f9f59103700 1 -- 172.21.2.101:0/4275396418 shutdown_connections 2021-02-10T11:36:12.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.233 7f9f59103700 1 --2- 172.21.2.101:0/4275396418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f380186c0 0x7f9f3801abc0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:12.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.233 7f9f59103700 1 --2- 172.21.2.101:0/4275396418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f540bb8e0 0x7f9f540bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:12.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.233 7f9f59103700 1 --2- 172.21.2.101:0/4275396418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f54195070 0x7f9f54195470 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:12.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.233 7f9f59103700 1 --2- 172.21.2.101:0/4275396418 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f541942b0 0x7f9f540b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:12.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.233 7f9f59103700 1 -- 172.21.2.101:0/4275396418 >> 172.21.2.101:0/4275396418 conn(0x7f9f540b3040 msgr2=0x7f9f540c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:12.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.234 7f9f59103700 1 -- 172.21.2.101:0/4275396418 shutdown_connections 2021-02-10T11:36:12.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:12.234 7f9f59103700 1 -- 172.21.2.101:0/4275396418 wait complete. 2021-02-10T11:36:12.451 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:36:02.287195', 'modified': '2021-02-10 11:36:04.295896', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5900}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5900': {'gid': 5900, 'name': 'a', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.101:6835/2977571942', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.101:6834', 'nonce': 2977571942}, {'type': 'v1', 'addr': '172.21.2.101:6835', 'nonce': 2977571942}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:36:12.451 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:36:12.451 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok status 2021-02-10T11:36:12.850 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:36:12.850 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:36:12.850 INFO:teuthology.orchestra.run.gibba001.stdout: "whoami": 0, 2021-02-10T11:36:12.850 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 5900, 2021-02-10T11:36:12.850 INFO:teuthology.orchestra.run.gibba001.stdout: "want_state": "up:active", 2021-02-10T11:36:12.850 INFO:teuthology.orchestra.run.gibba001.stdout: "state": "up:active", 2021-02-10T11:36:12.850 INFO:teuthology.orchestra.run.gibba001.stdout: "rank_uptime": 10.554516303, 2021-02-10T11:36:12.851 INFO:teuthology.orchestra.run.gibba001.stdout: "mdsmap_epoch": 55, 2021-02-10T11:36:12.851 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch": 74, 2021-02-10T11:36:12.851 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch_barrier": 74, 2021-02-10T11:36:12.851 INFO:teuthology.orchestra.run.gibba001.stdout: "uptime": 16.180196749 2021-02-10T11:36:12.851 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:36:12.854 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 5900, "want_state": "up:active", "state": "up:active", "rank_uptime": 10.554516303, "mdsmap_epoch": 55, "osdmap_epoch": 74, "osdmap_epoch_barrier": 74, "uptime": 16.180196749 } 2021-02-10T11:36:12.855 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:36:12.855 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-02-10T11:36:12.855 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-02-10T11:36:12.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:12.855 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:36:13.192 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:36:13.536 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:36:13.536 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:36:13.878 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:36:13.879 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:36:13.879 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:36:13.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:13.880 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:36:14.214 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-02-10T11:36:14.214 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:36:14.217 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:36:14.560 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:36:14.560 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:36:14.626 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[130383]: starting ceph client 2021-02-10T11:36:14.626 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:2021-02-10 11:36:14.619 7f2fadd46f80 -1 init, newargv = 0x55d4760c51a0 newargc=9 2021-02-10T11:36:14.632 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[130383]: starting fuse 2021-02-10T11:36:14.904 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:36:14.904 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:36:14.905 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:36:14.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:14.905 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:36:15.239 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:36:15.239 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2021-02-10T11:36:15.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:15.240 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:36:15.240 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:36:15.240 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:36:15.240 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:36:15.240 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:36:15.240 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:36:15.240 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:36:15.240 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-02-10T11:36:15.240 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:36:15.240 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:36:15.240 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:36:15.240 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:36:15.240 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:36:15.241 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:36:15.241 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:36:15.241 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:36:15.241 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:36:15.241 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:36:15.241 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:36:15.241 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:36:15.241 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:36:15.241 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.0")) 2021-02-10T11:36:15.241 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:36:15.601 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.0.130383.asok 2021-02-10T11:36:15.604 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.130383.asok 2021-02-10T11:36:15.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:15.604 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.130383.asok status 2021-02-10T11:36:15.994 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:36:15.994 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:36:15.994 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:36:15.994 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:36:15.994 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "0", 2021-02-10T11:36:15.994 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:36:15.995 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-02-10T11:36:15.995 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "130383", 2021-02-10T11:36:15.995 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:36:15.995 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:15.995 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:36:15.995 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:36:15.995 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 9132, 2021-02-10T11:36:15.995 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:36:15.995 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:36:15.995 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:36:15.995 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 9132 2021-02-10T11:36:15.995 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:15.995 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:36:15.995 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:15.996 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:36:15.996 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1853189196 2021-02-10T11:36:15.996 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:15.996 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:15.996 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:36:15.996 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:15.996 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:36:15.996 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1853189196 2021-02-10T11:36:15.996 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:15.996 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.9132 172.21.2.101:0/1853189196", 2021-02-10T11:36:15.996 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/1853189196", 2021-02-10T11:36:15.996 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:36:15.996 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 55, 2021-02-10T11:36:15.996 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 74, 2021-02-10T11:36:15.997 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:36:15.997 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:36:15.997 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:36:15.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:15.999 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:36:15.999 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:36:16.333 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:36:16.333 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:36:16.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:16.334 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:36:16.681 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:36:17.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.083 7fadf3b61700 1 Processor -- start 2021-02-10T11:36:17.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.083 7fadf3b61700 1 -- start start 2021-02-10T11:36:17.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.084 7fadf3b61700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fadec0d1870 0x7fadec1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:17.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.084 7fadf3b61700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fadec1b0a10 0x7fadec0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:17.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.084 7fadf3b61700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadec0f0b10 con 0x7fadec0d1c70 2021-02-10T11:36:17.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.084 7fadf3b61700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadec0fe230 con 0x7fadec1b0a10 2021-02-10T11:36:17.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.084 7fadf3b61700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadec0e5ed0 con 0x7fadec0d1870 2021-02-10T11:36:17.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.084 7fadf217e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fadec0d1c70 0x7fadec0cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54312/0 (socket says 172.21.2.101:54312) 2021-02-10T11:36:17.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.084 7fadf217e700 1 -- 172.21.2.101:0/1240135947 learned_addr learned my addr 172.21.2.101:0/1240135947 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:17.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.084 7fadf117c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fadec1b0a10 0x7fadec0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:17.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.084 7fadf097b700 1 -- 172.21.2.101:0/1240135947 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 702176605 0 0) 0x7fade4000f90 con 0x7fadec0d1c70 2021-02-10T11:36:17.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.084 7fadf097b700 1 -- 172.21.2.101:0/1240135947 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadd8002fe0 con 0x7fadec0d1c70 2021-02-10T11:36:17.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.084 7fadf117c700 1 -- 172.21.2.101:0/1240135947 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fadec0d1870 msgr2=0x7fadec1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:17.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.084 7fadf117c700 1 --2- 172.21.2.101:0/1240135947 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fadec0d1870 0x7fadec1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:17.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.084 7fadf117c700 1 -- 172.21.2.101:0/1240135947 >> v1:172.21.2.101:6789/0 conn(0x7fadec0d1c70 legacy=0x7fadec0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:17.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.084 7fadf117c700 1 -- 172.21.2.101:0/1240135947 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec0f0650 con 0x7fadec1b0a10 2021-02-10T11:36:17.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.085 7fadf117c700 1 --2- 172.21.2.101:0/1240135947 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fadec1b0a10 0x7fadec0cf330 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 rx=0x7faddc006650 tx=0x7faddc009a80).ready entity=mon.0 client_cookie=6e4b356ede16f877 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:17.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.085 7fadf097b700 1 -- 172.21.2.101:0/1240135947 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faddc0009b0 con 0x7fadec1b0a10 2021-02-10T11:36:17.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.085 7fadf097b700 1 -- 172.21.2.101:0/1240135947 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc00c0c0 con 0x7fadec1b0a10 2021-02-10T11:36:17.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.085 7fadf097b700 1 -- 172.21.2.101:0/1240135947 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faddc00c520 con 0x7fadec1b0a10 2021-02-10T11:36:17.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.085 7fadf3b61700 1 -- 172.21.2.101:0/1240135947 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fadec1b0a10 msgr2=0x7fadec0cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:17.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.085 7fadf3b61700 1 --2- 172.21.2.101:0/1240135947 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fadec1b0a10 0x7fadec0cf330 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 rx=0x7faddc006650 tx=0x7faddc009a80).stop 2021-02-10T11:36:17.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.086 7fadf3b61700 1 -- 172.21.2.101:0/1240135947 shutdown_connections 2021-02-10T11:36:17.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.086 7fadf3b61700 1 --2- 172.21.2.101:0/1240135947 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fadec1b0a10 0x7fadec0cf330 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:17.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.086 7fadf3b61700 1 --2- 172.21.2.101:0/1240135947 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fadec0d1870 0x7fadec1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:17.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.086 7fadf3b61700 1 -- 172.21.2.101:0/1240135947 >> 172.21.2.101:0/1240135947 conn(0x7fadec0ce450 msgr2=0x7fadec0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:17.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.086 7fadf3b61700 1 -- 172.21.2.101:0/1240135947 shutdown_connections 2021-02-10T11:36:17.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.086 7fadf3b61700 1 -- 172.21.2.101:0/1240135947 wait complete. 2021-02-10T11:36:17.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.100 7fadf3b61700 1 Processor -- start 2021-02-10T11:36:17.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.100 7fadf3b61700 1 -- start start 2021-02-10T11:36:17.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.101 7fadf3b61700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fadec195650 0x7fadec0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:17.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.101 7fadf3b61700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fadec0b5b80 0x7fadec1a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:17.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.101 7fadf3b61700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fadec0acfa0 0x7fadec1a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:17.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.101 7fadf3b61700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec0b6010 con 0x7fadec0b5b80 2021-02-10T11:36:17.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.101 7fadf3b61700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec1a3a60 con 0x7fadec195650 2021-02-10T11:36:17.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.101 7fadf3b61700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadec1a3c40 con 0x7fadec0acfa0 2021-02-10T11:36:17.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.101 7fadf117c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fadec0b5b80 0x7fadec1a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:17.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.101 7fadf117c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fadec0b5b80 0x7fadec1a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53820/0 (socket says 172.21.2.101:53820) 2021-02-10T11:36:17.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.101 7fadf117c700 1 -- 172.21.2.101:0/2497335641 learned_addr learned my addr 172.21.2.101:0/2497335641 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:17.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.101 7fadf117c700 1 -- 172.21.2.101:0/2497335641 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fadec0acfa0 msgr2=0x7fadec1a3440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:17.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.101 7fadf117c700 1 --2- 172.21.2.101:0/2497335641 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fadec0acfa0 0x7fadec1a3440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:17.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.101 7fadf117c700 1 -- 172.21.2.101:0/2497335641 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fadec195650 msgr2=0x7fadec0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:17.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.101 7fadf117c700 1 --2- 172.21.2.101:0/2497335641 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fadec195650 0x7fadec0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:17.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.101 7fadf117c700 1 -- 172.21.2.101:0/2497335641 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec0bcae0 con 0x7fadec0b5b80 2021-02-10T11:36:17.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.102 7fadf117c700 1 --2- 172.21.2.101:0/2497335641 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fadec0b5b80 0x7fadec1a2e20 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 rx=0x7faddc009650 tx=0x7faddc00b8e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:17.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.102 7fade27fc700 1 -- 172.21.2.101:0/2497335641 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faddc0093a0 con 0x7fadec0b5b80 2021-02-10T11:36:17.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.102 7fade27fc700 1 -- 172.21.2.101:0/2497335641 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc006100 con 0x7fadec0b5b80 2021-02-10T11:36:17.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.102 7fade27fc700 1 -- 172.21.2.101:0/2497335641 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faddc0052e0 con 0x7fadec0b5b80 2021-02-10T11:36:17.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.102 7fadf3b61700 1 -- 172.21.2.101:0/2497335641 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadec0bcae0 con 0x7fadec0b5b80 2021-02-10T11:36:17.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.102 7fadf3b61700 1 -- 172.21.2.101:0/2497335641 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadec0bcae0 con 0x7fadec0b5b80 2021-02-10T11:36:17.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.103 7fade27fc700 1 -- 172.21.2.101:0/2497335641 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7faddc009300 con 0x7fadec0b5b80 2021-02-10T11:36:17.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.103 7fade27fc700 1 --2- 172.21.2.101:0/2497335641 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fadd40186c0 0x7fadd401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:17.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.103 7fade27fc700 1 -- 172.21.2.101:0/2497335641 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7faddc0323e0 con 0x7fadec0b5b80 2021-02-10T11:36:17.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.103 7fadf197d700 1 --2- 172.21.2.101:0/2497335641 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fadd40186c0 0x7fadd401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:17.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.103 7fadf197d700 1 --2- 172.21.2.101:0/2497335641 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fadd40186c0 0x7fadd401abc0 crc :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:17.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.134 7fadf3b61700 1 -- 172.21.2.101:0/2497335641 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadec0b6010 con 0x7fadec0b5b80 2021-02-10T11:36:17.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.136 7fade27fc700 1 -- 172.21.2.101:0/2497335641 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7faddc026030 con 0x7fadec0b5b80 2021-02-10T11:36:17.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.262 7fadf3b61700 1 -- 172.21.2.101:0/2497335641 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadec0b6010 con 0x7fadec0b5b80 2021-02-10T11:36:17.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.262 7fade27fc700 1 -- 172.21.2.101:0/2497335641 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+13996 (secure 0 0 0) 0x7faddc026030 con 0x7fadec0b5b80 2021-02-10T11:36:17.270 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:36:17.270 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":74,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:36:03.289219","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:36:00.261115","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2021-02-10 11:36:01.333230","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:36:17.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.264 7fadf3b61700 1 -- 172.21.2.101:0/2497335641 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fadd40186c0 msgr2=0x7fadd401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:17.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.264 7fadf3b61700 1 --2- 172.21.2.101:0/2497335641 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fadd40186c0 0x7fadd401abc0 crc :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:17.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.264 7fadf3b61700 1 -- 172.21.2.101:0/2497335641 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fadec0b5b80 msgr2=0x7fadec1a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:17.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.264 7fadf3b61700 1 --2- 172.21.2.101:0/2497335641 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fadec0b5b80 0x7fadec1a2e20 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 rx=0x7faddc009650 tx=0x7faddc00b8e0).stop 2021-02-10T11:36:17.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.264 7fadf3b61700 1 -- 172.21.2.101:0/2497335641 shutdown_connections 2021-02-10T11:36:17.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.264 7fadf3b61700 1 --2- 172.21.2.101:0/2497335641 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fadd40186c0 0x7fadd401abc0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:17.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.264 7fadf3b61700 1 --2- 172.21.2.101:0/2497335641 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fadec0acfa0 0x7fadec1a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:17.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.264 7fadf3b61700 1 --2- 172.21.2.101:0/2497335641 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fadec0b5b80 0x7fadec1a2e20 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:17.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.264 7fadf3b61700 1 --2- 172.21.2.101:0/2497335641 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fadec195650 0x7fadec0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:17.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.264 7fadf3b61700 1 -- 172.21.2.101:0/2497335641 >> 172.21.2.101:0/2497335641 conn(0x7fadec0b3040 msgr2=0x7fadec0c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:17.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.264 7fadf3b61700 1 -- 172.21.2.101:0/2497335641 shutdown_connections 2021-02-10T11:36:17.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.264 7fadf3b61700 1 -- 172.21.2.101:0/2497335641 wait complete. 2021-02-10T11:36:17.280 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:36:00.261115', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:36:01.333230', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:36:17.280 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:36:17.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.687 7fb4c1785700 1 Processor -- start 2021-02-10T11:36:17.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.687 7fb4c1785700 1 -- start start 2021-02-10T11:36:17.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.687 7fb4c1785700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4bc0d1c70 0x7fb4bc0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:17.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.687 7fb4c1785700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4bc0d0170 con 0x7fb4bc0d1870 2021-02-10T11:36:17.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.687 7fb4c1785700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4bc0fe1c0 con 0x7fb4bc0cf580 2021-02-10T11:36:17.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.687 7fb4c1785700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc1b0cb0 con 0x7fb4bc0d1c70 2021-02-10T11:36:17.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.688 7fb4bb574700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb4bc0d1870 0x7fb4bc1b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54322/0 (socket says 172.21.2.101:54322) 2021-02-10T11:36:17.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.688 7fb4bb574700 1 -- 172.21.2.101:0/3747293790 learned_addr learned my addr 172.21.2.101:0/3747293790 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:17.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.688 7fb4ba572700 1 -- 172.21.2.101:0/3747293790 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654681576 0 0) 0x7fb4ac001180 con 0x7fb4bc0d1870 2021-02-10T11:36:17.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.688 7fb4ba572700 1 -- 172.21.2.101:0/3747293790 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4a4002fe0 con 0x7fb4bc0d1870 2021-02-10T11:36:17.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.688 7fb4ba572700 1 -- 172.21.2.101:0/3747293790 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 861228618 0 0) 0x7fb4ac0010e0 con 0x7fb4bc0d1870 2021-02-10T11:36:17.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.688 7fb4ba572700 1 -- 172.21.2.101:0/3747293790 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4bc0d1c70 msgr2=0x7fb4bc0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:17.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.688 7fb4ba572700 1 --2- 172.21.2.101:0/3747293790 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4bc0d1c70 0x7fb4bc0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:17.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.688 7fb4ba572700 1 -- 172.21.2.101:0/3747293790 >> v1:172.21.2.108:6789/0 conn(0x7fb4bc0cf580 legacy=0x7fb4bc0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:17.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.688 7fb4ba572700 1 -- 172.21.2.101:0/3747293790 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4bc109a60 con 0x7fb4bc0d1870 2021-02-10T11:36:17.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.688 7fb4ba572700 1 -- 172.21.2.101:0/3747293790 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fb4ac001600 con 0x7fb4bc0d1870 2021-02-10T11:36:17.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.688 7fb4ba572700 1 -- 172.21.2.101:0/3747293790 >> v1:172.21.2.101:6789/0 conn(0x7fb4bc0d1870 legacy=0x7fb4bc1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:17.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.688 7fb4ba572700 1 --2- 172.21.2.101:0/3747293790 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4a4003a60 0x7fb4a4005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:17.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.688 7fb4ba572700 1 --2- 172.21.2.101:0/3747293790 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4a4006790 0x7fb4a4008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:17.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.688 7fb4ba572700 1 --2- 172.21.2.101:0/3747293790 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4a4009590 0x7fb4a400ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:17.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.688 7fb4ba572700 1 -- 172.21.2.101:0/3747293790 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a400c2d0 con 0x7fb4a4006790 2021-02-10T11:36:17.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.688 7fb4ba572700 1 -- 172.21.2.101:0/3747293790 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a400c4f0 con 0x7fb4a4003a60 2021-02-10T11:36:17.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.688 7fb4ba572700 1 -- 172.21.2.101:0/3747293790 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a400c710 con 0x7fb4a4009590 2021-02-10T11:36:17.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.689 7fb4bad73700 1 --2- 172.21.2.101:0/3747293790 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4a4006790 0x7fb4a4008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:17.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.689 7fb4bbd75700 1 --2- 172.21.2.101:0/3747293790 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4a4009590 0x7fb4a400ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:17.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.689 7fb4bb574700 1 --2- 172.21.2.101:0/3747293790 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4a4003a60 0x7fb4a4005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:17.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.689 7fb4bad73700 1 -- 172.21.2.101:0/3747293790 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4a4009590 msgr2=0x7fb4a400ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:17.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.689 7fb4bad73700 1 --2- 172.21.2.101:0/3747293790 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4a4009590 0x7fb4a400ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:17.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.689 7fb4bad73700 1 -- 172.21.2.101:0/3747293790 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4a4003a60 msgr2=0x7fb4a4005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:17.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.689 7fb4bad73700 1 --2- 172.21.2.101:0/3747293790 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4a4003a60 0x7fb4a4005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:17.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.689 7fb4bad73700 1 -- 172.21.2.101:0/3747293790 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a400c9b0 con 0x7fb4a4006790 2021-02-10T11:36:17.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.689 7fb4bad73700 1 --2- 172.21.2.101:0/3747293790 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4a4006790 0x7fb4a4008c90 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 rx=0x7fb4b0006ab0 tx=0x7fb4b004fbc0).ready entity=mon.0 client_cookie=eb1e935d9c4334d6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:17.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.689 7fb4ba572700 1 -- 172.21.2.101:0/3747293790 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb4b0002080 con 0x7fb4a4006790 2021-02-10T11:36:17.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.689 7fb4ba572700 1 -- 172.21.2.101:0/3747293790 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b00063f0 con 0x7fb4a4006790 2021-02-10T11:36:17.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.689 7fb4ba572700 1 -- 172.21.2.101:0/3747293790 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb4b0002380 con 0x7fb4a4006790 2021-02-10T11:36:17.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.689 7fb4c1785700 1 -- 172.21.2.101:0/3747293790 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4a4006790 msgr2=0x7fb4a4008c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:17.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.689 7fb4c1785700 1 --2- 172.21.2.101:0/3747293790 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4a4006790 0x7fb4a4008c90 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 rx=0x7fb4b0006ab0 tx=0x7fb4b004fbc0).stop 2021-02-10T11:36:17.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.689 7fb4bbd75700 1 -- 172.21.2.101:0/3747293790 reap_dead start 2021-02-10T11:36:17.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.690 7fb4c1785700 1 -- 172.21.2.101:0/3747293790 shutdown_connections 2021-02-10T11:36:17.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.690 7fb4c1785700 1 -- 172.21.2.101:0/3747293790 >> 172.21.2.101:0/3747293790 conn(0x7fb4bc0ce420 msgr2=0x7fb4bc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:17.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.690 7fb4c1785700 1 -- 172.21.2.101:0/3747293790 shutdown_connections 2021-02-10T11:36:17.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.690 7fb4c1785700 1 -- 172.21.2.101:0/3747293790 wait complete. 2021-02-10T11:36:17.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.705 7fb4c1785700 1 Processor -- start 2021-02-10T11:36:17.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.705 7fb4c1785700 1 -- start start 2021-02-10T11:36:17.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.705 7fb4c1785700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4bc085bc0 0x7fb4bc0863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:17.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.705 7fb4c1785700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4bc0954c0 0x7fb4bc096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:17.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.705 7fb4c1785700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4bc0958c0 0x7fb4bc096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:17.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.705 7fb4c1785700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc095d80 con 0x7fb4bc0954c0 2021-02-10T11:36:17.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.705 7fb4c1785700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc0bfa60 con 0x7fb4bc085bc0 2021-02-10T11:36:17.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.705 7fb4c1785700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc085fc0 con 0x7fb4bc0958c0 2021-02-10T11:36:17.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.705 7fb4bad73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4bc0954c0 0x7fb4bc096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:17.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.705 7fb4bad73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4bc0954c0 0x7fb4bc096300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53840/0 (socket says 172.21.2.101:53840) 2021-02-10T11:36:17.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.705 7fb4bad73700 1 -- 172.21.2.101:0/2760764638 learned_addr learned my addr 172.21.2.101:0/2760764638 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:17.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.706 7fb4bad73700 1 -- 172.21.2.101:0/2760764638 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4bc0958c0 msgr2=0x7fb4bc096920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:17.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.706 7fb4bad73700 1 --2- 172.21.2.101:0/2760764638 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4bc0958c0 0x7fb4bc096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:17.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.706 7fb4bad73700 1 -- 172.21.2.101:0/2760764638 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4bc085bc0 msgr2=0x7fb4bc0863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:17.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.706 7fb4bad73700 1 --2- 172.21.2.101:0/2760764638 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4bc085bc0 0x7fb4bc0863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:17.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.706 7fb4bad73700 1 -- 172.21.2.101:0/2760764638 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4bc086c10 con 0x7fb4bc0954c0 2021-02-10T11:36:17.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.706 7fb4bad73700 1 --2- 172.21.2.101:0/2760764638 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4bc0954c0 0x7fb4bc096300 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 rx=0x7fb4b0002390 tx=0x7fb4b004f970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:17.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.706 7fb4a3fff700 1 -- 172.21.2.101:0/2760764638 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb4b004fd60 con 0x7fb4bc0954c0 2021-02-10T11:36:17.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.706 7fb4a3fff700 1 -- 172.21.2.101:0/2760764638 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b0006cb0 con 0x7fb4bc0954c0 2021-02-10T11:36:17.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.706 7fb4c1785700 1 -- 172.21.2.101:0/2760764638 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4bc086c10 con 0x7fb4bc0954c0 2021-02-10T11:36:17.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.706 7fb4a3fff700 1 -- 172.21.2.101:0/2760764638 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb4b00072a0 con 0x7fb4bc0954c0 2021-02-10T11:36:17.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.706 7fb4c1785700 1 -- 172.21.2.101:0/2760764638 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4bc097230 con 0x7fb4bc0954c0 2021-02-10T11:36:17.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.707 7fb4a3fff700 1 -- 172.21.2.101:0/2760764638 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb4b0007230 con 0x7fb4bc0954c0 2021-02-10T11:36:17.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.707 7fb4a3fff700 1 --2- 172.21.2.101:0/2760764638 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb49c0186c0 0x7fb49c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:17.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.707 7fb4a3fff700 1 -- 172.21.2.101:0/2760764638 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb4b0076400 con 0x7fb4bc0954c0 2021-02-10T11:36:17.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.707 7fb4bb574700 1 --2- 172.21.2.101:0/2760764638 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb49c0186c0 0x7fb49c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:17.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.707 7fb4bb574700 1 --2- 172.21.2.101:0/2760764638 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb49c0186c0 0x7fb49c01abc0 crc :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:17.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.738 7fb4c1785700 1 -- 172.21.2.101:0/2760764638 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4bc0fe1c0 con 0x7fb4bc0954c0 2021-02-10T11:36:17.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.740 7fb4a3fff700 1 -- 172.21.2.101:0/2760764638 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb4b0069fe0 con 0x7fb4bc0954c0 2021-02-10T11:36:17.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.867 7fb4c1785700 1 -- 172.21.2.101:0/2760764638 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fb4bc0fe1c0 con 0x7fb4bc0954c0 2021-02-10T11:36:17.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.867 7fb4a3fff700 1 -- 172.21.2.101:0/2760764638 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v74) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb4b0069fe0 con 0x7fb4bc0954c0 2021-02-10T11:36:17.875 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:36:17.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.869 7fb4c1785700 1 -- 172.21.2.101:0/2760764638 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb49c0186c0 msgr2=0x7fb49c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:17.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.869 7fb4c1785700 1 --2- 172.21.2.101:0/2760764638 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb49c0186c0 0x7fb49c01abc0 crc :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:17.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.869 7fb4c1785700 1 -- 172.21.2.101:0/2760764638 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4bc0954c0 msgr2=0x7fb4bc096300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:17.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.869 7fb4c1785700 1 --2- 172.21.2.101:0/2760764638 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4bc0954c0 0x7fb4bc096300 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 rx=0x7fb4b0002390 tx=0x7fb4b004f970).stop 2021-02-10T11:36:17.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.869 7fb4c1785700 1 -- 172.21.2.101:0/2760764638 shutdown_connections 2021-02-10T11:36:17.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.869 7fb4c1785700 1 --2- 172.21.2.101:0/2760764638 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb49c0186c0 0x7fb49c01abc0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:17.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.869 7fb4c1785700 1 --2- 172.21.2.101:0/2760764638 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4bc0958c0 0x7fb4bc096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:17.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.869 7fb4c1785700 1 --2- 172.21.2.101:0/2760764638 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4bc0954c0 0x7fb4bc096300 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:17.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.869 7fb4c1785700 1 --2- 172.21.2.101:0/2760764638 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4bc085bc0 0x7fb4bc0863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:17.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.869 7fb4c1785700 1 -- 172.21.2.101:0/2760764638 >> 172.21.2.101:0/2760764638 conn(0x7fb4bc0bef00 msgr2=0x7fb4bc09d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:17.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.870 7fb4c1785700 1 -- 172.21.2.101:0/2760764638 shutdown_connections 2021-02-10T11:36:17.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:17.870 7fb4c1785700 1 -- 172.21.2.101:0/2760764638 wait complete. 2021-02-10T11:36:17.884 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:36:18.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.292 7f96bab7d700 1 Processor -- start 2021-02-10T11:36:18.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.292 7f96bab7d700 1 -- start start 2021-02-10T11:36:18.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.293 7f96bab7d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f96b40d1870 0x7f96b41b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:18.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.293 7f96bab7d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f96b40cff70 0x7f96b4109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:18.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.293 7f96bab7d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b40fe1c0 con 0x7f96b41b0ae0 2021-02-10T11:36:18.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.293 7f96bab7d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b4109ad0 con 0x7f96b40d1870 2021-02-10T11:36:18.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.293 7f96bab7d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b4109750 con 0x7f96b40cff70 2021-02-10T11:36:18.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.293 7f96b8999700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f96b40d1870 0x7f96b41b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:18.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.293 7f96b8999700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f96b40d1870 0x7f96b41b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53848/0 (socket says 172.21.2.101:53848) 2021-02-10T11:36:18.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.293 7f96b8999700 1 -- 172.21.2.101:0/3604191248 learned_addr learned my addr 172.21.2.101:0/3604191248 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:18.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.294 7f96b8999700 1 -- 172.21.2.101:0/3604191248 >> v2:172.21.2.108:3300/0 conn(0x7f96b40cff70 msgr2=0x7f96b4109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:18.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.294 7f96b8999700 1 --2- 172.21.2.101:0/3604191248 >> v2:172.21.2.108:3300/0 conn(0x7f96b40cff70 0x7f96b4109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:18.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.294 7f96b8999700 1 -- 172.21.2.101:0/3604191248 >> v1:172.21.2.108:6789/0 conn(0x7f96b41b0ae0 legacy=0x7f96b4108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:18.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.294 7f96b8999700 1 -- 172.21.2.101:0/3604191248 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b4088070 con 0x7f96b40d1870 2021-02-10T11:36:18.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.294 7f96b8999700 1 --2- 172.21.2.101:0/3604191248 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96b40d1870 0x7f96b41b04c0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 rx=0x7f969c006700 tx=0x7f969c009b80).ready entity=mon.0 client_cookie=14cd88d83070af13 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:18.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.294 7f96b37fe700 1 -- 172.21.2.101:0/3604191248 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f969c00c0c0 con 0x7f96b40d1870 2021-02-10T11:36:18.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.294 7f96b37fe700 1 -- 172.21.2.101:0/3604191248 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969c00c400 con 0x7f96b40d1870 2021-02-10T11:36:18.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.294 7f96b37fe700 1 -- 172.21.2.101:0/3604191248 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f969c00c920 con 0x7f96b40d1870 2021-02-10T11:36:18.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.294 7f96bab7d700 1 -- 172.21.2.101:0/3604191248 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96b40d1870 msgr2=0x7f96b41b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:18.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.294 7f96bab7d700 1 --2- 172.21.2.101:0/3604191248 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96b40d1870 0x7f96b41b04c0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 rx=0x7f969c006700 tx=0x7f969c009b80).stop 2021-02-10T11:36:18.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.295 7f96bab7d700 1 -- 172.21.2.101:0/3604191248 shutdown_connections 2021-02-10T11:36:18.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.295 7f96bab7d700 1 --2- 172.21.2.101:0/3604191248 >> v2:172.21.2.108:3300/0 conn(0x7f96b40cff70 0x7f96b4109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:18.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.295 7f96bab7d700 1 --2- 172.21.2.101:0/3604191248 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96b40d1870 0x7f96b41b04c0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:18.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.295 7f96bab7d700 1 -- 172.21.2.101:0/3604191248 >> 172.21.2.101:0/3604191248 conn(0x7f96b40ce420 msgr2=0x7f96b40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:18.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.295 7f96bab7d700 1 -- 172.21.2.101:0/3604191248 shutdown_connections 2021-02-10T11:36:18.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.295 7f96bab7d700 1 -- 172.21.2.101:0/3604191248 wait complete. 2021-02-10T11:36:18.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.310 7f96bab7d700 1 Processor -- start 2021-02-10T11:36:18.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.310 7f96bab7d700 1 -- start start 2021-02-10T11:36:18.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.310 7f96bab7d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96b40c3820 0x7f96b40a7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:18.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.310 7f96bab7d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96b41a44c0 0x7f96b41a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:18.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.310 7f96bab7d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96b40863d0 0x7f96b40867d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:18.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.310 7f96bab7d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b4085ed0 con 0x7f96b41a44c0 2021-02-10T11:36:18.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.310 7f96bab7d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b40c0740 con 0x7f96b40863d0 2021-02-10T11:36:18.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.310 7f96bab7d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b4086df0 con 0x7f96b40c3820 2021-02-10T11:36:18.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.310 7f96b3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96b41a44c0 0x7f96b41a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:18.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.310 7f96b3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96b41a44c0 0x7f96b41a48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53852/0 (socket says 172.21.2.101:53852) 2021-02-10T11:36:18.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.310 7f96b3fff700 1 -- 172.21.2.101:0/1998755246 learned_addr learned my addr 172.21.2.101:0/1998755246 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:18.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.311 7f96b3fff700 1 -- 172.21.2.101:0/1998755246 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96b40c3820 msgr2=0x7f96b40a7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:18.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.311 7f96b3fff700 1 --2- 172.21.2.101:0/1998755246 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96b40c3820 0x7f96b40a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:18.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.311 7f96b3fff700 1 -- 172.21.2.101:0/1998755246 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96b40863d0 msgr2=0x7f96b40867d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:18.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.311 7f96b3fff700 1 --2- 172.21.2.101:0/1998755246 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96b40863d0 0x7f96b40867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:18.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.311 7f96b3fff700 1 -- 172.21.2.101:0/1998755246 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b4087010 con 0x7f96b41a44c0 2021-02-10T11:36:18.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.311 7f96b3fff700 1 --2- 172.21.2.101:0/1998755246 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96b41a44c0 0x7f96b41a48c0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 rx=0x7f96a4009940 tx=0x7f96a400bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:18.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.311 7f96b17fa700 1 -- 172.21.2.101:0/1998755246 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96a4000fe0 con 0x7f96b41a44c0 2021-02-10T11:36:18.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.311 7f96b17fa700 1 -- 172.21.2.101:0/1998755246 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a4000d30 con 0x7f96b41a44c0 2021-02-10T11:36:18.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.311 7f96b17fa700 1 -- 172.21.2.101:0/1998755246 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96a400c570 con 0x7f96b41a44c0 2021-02-10T11:36:18.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.311 7f96bab7d700 1 -- 172.21.2.101:0/1998755246 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b4087010 con 0x7f96b41a44c0 2021-02-10T11:36:18.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.311 7f96bab7d700 1 -- 172.21.2.101:0/1998755246 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b4083000 con 0x7f96b41a44c0 2021-02-10T11:36:18.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.312 7f96b17fa700 1 -- 172.21.2.101:0/1998755246 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f96a4001230 con 0x7f96b41a44c0 2021-02-10T11:36:18.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.312 7f96b17fa700 1 --2- 172.21.2.101:0/1998755246 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96980186c0 0x7f969801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:18.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.312 7f96b17fa700 1 -- 172.21.2.101:0/1998755246 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f96a4032920 con 0x7f96b41a44c0 2021-02-10T11:36:18.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.312 7f96b8999700 1 --2- 172.21.2.101:0/1998755246 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96980186c0 0x7f969801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:18.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.312 7f96b8999700 1 --2- 172.21.2.101:0/1998755246 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96980186c0 0x7f969801abc0 crc :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:18.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.342 7f96bab7d700 1 -- 172.21.2.101:0/1998755246 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b4085e50 con 0x7f96b41a44c0 2021-02-10T11:36:18.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.345 7f96b17fa700 1 -- 172.21.2.101:0/1998755246 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f96a4027c10 con 0x7f96b41a44c0 2021-02-10T11:36:18.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.473 7f96bab7d700 1 -- 172.21.2.101:0/1998755246 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f96b4085e50 con 0x7f96b41a44c0 2021-02-10T11:36:18.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.474 7f96b17fa700 1 -- 172.21.2.101:0/1998755246 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v74) v1 ==== 104+0+10 (secure 0 0 0) 0x7f96a4027c10 con 0x7f96b41a44c0 2021-02-10T11:36:18.481 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:36:18.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.476 7f96bab7d700 1 -- 172.21.2.101:0/1998755246 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96980186c0 msgr2=0x7f969801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:18.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.476 7f96bab7d700 1 --2- 172.21.2.101:0/1998755246 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96980186c0 0x7f969801abc0 crc :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:18.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.476 7f96bab7d700 1 -- 172.21.2.101:0/1998755246 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96b41a44c0 msgr2=0x7f96b41a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:18.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.476 7f96bab7d700 1 --2- 172.21.2.101:0/1998755246 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96b41a44c0 0x7f96b41a48c0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 rx=0x7f96a4009940 tx=0x7f96a400bcf0).stop 2021-02-10T11:36:18.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.476 7f96bab7d700 1 -- 172.21.2.101:0/1998755246 shutdown_connections 2021-02-10T11:36:18.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.476 7f96bab7d700 1 --2- 172.21.2.101:0/1998755246 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96b40863d0 0x7f96b40867d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:18.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.476 7f96bab7d700 1 --2- 172.21.2.101:0/1998755246 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96b41a44c0 0x7f96b41a48c0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:18.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.476 7f96bab7d700 1 --2- 172.21.2.101:0/1998755246 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96980186c0 0x7f969801abc0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:18.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.476 7f96bab7d700 1 --2- 172.21.2.101:0/1998755246 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96b40c3820 0x7f96b40a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:18.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.476 7f96bab7d700 1 -- 172.21.2.101:0/1998755246 >> 172.21.2.101:0/1998755246 conn(0x7f96b40872c0 msgr2=0x7f96b4095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:18.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.476 7f96bab7d700 1 -- 172.21.2.101:0/1998755246 shutdown_connections 2021-02-10T11:36:18.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.476 7f96bab7d700 1 -- 172.21.2.101:0/1998755246 wait complete. 2021-02-10T11:36:18.491 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:36:18.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.899 7ff35d7a5700 1 Processor -- start 2021-02-10T11:36:18.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.899 7ff35d7a5700 1 -- start start 2021-02-10T11:36:18.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.900 7ff35d7a5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3580d1c70 0x7ff3580cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:18.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.900 7ff35d7a5700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3580d0170 con 0x7ff3580cf580 2021-02-10T11:36:18.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.900 7ff35d7a5700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3580fe1c0 con 0x7ff3580d1870 2021-02-10T11:36:18.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.900 7ff35d7a5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3581b0cb0 con 0x7ff3580d1c70 2021-02-10T11:36:18.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.900 7ff357d75700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff3580cf580 0x7ff3580cf980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54356/0 (socket says 172.21.2.101:54356) 2021-02-10T11:36:18.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.900 7ff357d75700 1 -- 172.21.2.101:0/247871180 learned_addr learned my addr 172.21.2.101:0/247871180 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:18.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.900 7ff34ed73700 1 --2- 172.21.2.101:0/247871180 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3580d1c70 0x7ff3580cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:18.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.900 7ff356d73700 1 -- 172.21.2.101:0/247871180 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3287676376 0 0) 0x7ff350000d30 con 0x7ff3580cf580 2021-02-10T11:36:18.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.900 7ff356d73700 1 -- 172.21.2.101:0/247871180 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff344002fe0 con 0x7ff3580cf580 2021-02-10T11:36:18.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.900 7ff356d73700 1 -- 172.21.2.101:0/247871180 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660411187 0 0) 0x7ff348000c60 con 0x7ff3580d1870 2021-02-10T11:36:18.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.900 7ff356d73700 1 -- 172.21.2.101:0/247871180 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff344003210 con 0x7ff3580d1870 2021-02-10T11:36:18.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.900 7ff34ed73700 1 -- 172.21.2.101:0/247871180 >> v1:172.21.2.108:6789/0 conn(0x7ff3580d1870 legacy=0x7ff3581b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:18.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.900 7ff34ed73700 1 -- 172.21.2.101:0/247871180 >> v1:172.21.2.101:6789/0 conn(0x7ff3580cf580 legacy=0x7ff3580cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:18.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.900 7ff34ed73700 1 -- 172.21.2.101:0/247871180 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff358109a60 con 0x7ff3580d1c70 2021-02-10T11:36:18.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.901 7ff34ed73700 1 --2- 172.21.2.101:0/247871180 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3580d1c70 0x7ff3580cef60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7ff340006650 tx=0x7ff340009a00).ready entity=mon.2 client_cookie=501801cbefdd41b9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:18.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.901 7ff356d73700 1 -- 172.21.2.101:0/247871180 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff340009df0 con 0x7ff3580d1c70 2021-02-10T11:36:18.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.901 7ff356d73700 1 -- 172.21.2.101:0/247871180 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3400009b0 con 0x7ff3580d1c70 2021-02-10T11:36:18.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.901 7ff356d73700 1 -- 172.21.2.101:0/247871180 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff34000c2b0 con 0x7ff3580d1c70 2021-02-10T11:36:18.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.901 7ff35d7a5700 1 -- 172.21.2.101:0/247871180 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3580d1c70 msgr2=0x7ff3580cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:18.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.901 7ff35d7a5700 1 --2- 172.21.2.101:0/247871180 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3580d1c70 0x7ff3580cef60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7ff340006650 tx=0x7ff340009a00).stop 2021-02-10T11:36:18.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.901 7ff35d7a5700 1 -- 172.21.2.101:0/247871180 shutdown_connections 2021-02-10T11:36:18.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.901 7ff35d7a5700 1 --2- 172.21.2.101:0/247871180 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3580d1c70 0x7ff3580cef60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:18.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.901 7ff35d7a5700 1 -- 172.21.2.101:0/247871180 >> 172.21.2.101:0/247871180 conn(0x7ff3580ce420 msgr2=0x7ff3580d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:18.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.901 7ff35d7a5700 1 -- 172.21.2.101:0/247871180 shutdown_connections 2021-02-10T11:36:18.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.901 7ff35d7a5700 1 -- 172.21.2.101:0/247871180 wait complete. 2021-02-10T11:36:18.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.916 7ff35d7a5700 1 Processor -- start 2021-02-10T11:36:18.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.916 7ff35d7a5700 1 -- start start 2021-02-10T11:36:18.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.916 7ff35d7a5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff35818c6c0 0x7ff358092e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:18.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.916 7ff35d7a5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff358097850 0x7ff3581a7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:18.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.916 7ff35d7a5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3581a7860 0x7ff3581a7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:18.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.916 7ff35d7a5700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff358097f20 con 0x7ff3581a7860 2021-02-10T11:36:18.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.916 7ff35d7a5700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff358097c50 con 0x7ff35818c6c0 2021-02-10T11:36:18.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.916 7ff35d7a5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3580869b0 con 0x7ff358097850 2021-02-10T11:36:18.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.917 7ff357d75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3581a7860 0x7ff3581a7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:18.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.917 7ff357d75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3581a7860 0x7ff3581a7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53870/0 (socket says 172.21.2.101:53870) 2021-02-10T11:36:18.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.917 7ff357d75700 1 -- 172.21.2.101:0/2680679285 learned_addr learned my addr 172.21.2.101:0/2680679285 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:18.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.917 7ff357d75700 1 -- 172.21.2.101:0/2680679285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff358097850 msgr2=0x7ff3581a7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:18.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.917 7ff357d75700 1 --2- 172.21.2.101:0/2680679285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff358097850 0x7ff3581a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:18.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.917 7ff357d75700 1 -- 172.21.2.101:0/2680679285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff35818c6c0 msgr2=0x7ff358092e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:18.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.917 7ff357d75700 1 --2- 172.21.2.101:0/2680679285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff35818c6c0 0x7ff358092e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:18.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.917 7ff357d75700 1 -- 172.21.2.101:0/2680679285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3580984a0 con 0x7ff3581a7860 2021-02-10T11:36:18.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.917 7ff357d75700 1 --2- 172.21.2.101:0/2680679285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3581a7860 0x7ff3581a7c60 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 rx=0x7ff350009980 tx=0x7ff35000bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:18.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.917 7ff354d6f700 1 -- 172.21.2.101:0/2680679285 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff35000be10 con 0x7ff3581a7860 2021-02-10T11:36:18.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.917 7ff354d6f700 1 -- 172.21.2.101:0/2680679285 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3500060c0 con 0x7ff3581a7860 2021-02-10T11:36:18.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.917 7ff354d6f700 1 -- 172.21.2.101:0/2680679285 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff350006490 con 0x7ff3581a7860 2021-02-10T11:36:18.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.917 7ff35d7a5700 1 -- 172.21.2.101:0/2680679285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3580984a0 con 0x7ff3581a7860 2021-02-10T11:36:18.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.918 7ff35d7a5700 1 -- 172.21.2.101:0/2680679285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3580984a0 con 0x7ff3581a7860 2021-02-10T11:36:18.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.918 7ff354d6f700 1 -- 172.21.2.101:0/2680679285 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff3500062a0 con 0x7ff3581a7860 2021-02-10T11:36:18.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.918 7ff354d6f700 1 --2- 172.21.2.101:0/2680679285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff33c0186c0 0x7ff33c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:18.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.918 7ff354d6f700 1 -- 172.21.2.101:0/2680679285 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff350032a00 con 0x7ff3581a7860 2021-02-10T11:36:18.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.918 7ff357574700 1 --2- 172.21.2.101:0/2680679285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff33c0186c0 0x7ff33c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:18.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.919 7ff357574700 1 --2- 172.21.2.101:0/2680679285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff33c0186c0 0x7ff33c01abc0 crc :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:18.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.949 7ff35d7a5700 1 -- 172.21.2.101:0/2680679285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff358082a70 con 0x7ff3581a7860 2021-02-10T11:36:18.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:18.952 7ff354d6f700 1 -- 172.21.2.101:0/2680679285 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff350000f90 con 0x7ff3581a7860 2021-02-10T11:36:19.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.079 7ff35d7a5700 1 -- 172.21.2.101:0/2680679285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7ff358082a70 con 0x7ff3581a7860 2021-02-10T11:36:19.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.079 7ff354d6f700 1 -- 172.21.2.101:0/2680679285 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v74) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff3500060c0 con 0x7ff3581a7860 2021-02-10T11:36:19.087 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:36:19.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.081 7ff35d7a5700 1 -- 172.21.2.101:0/2680679285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff33c0186c0 msgr2=0x7ff33c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:19.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.081 7ff35d7a5700 1 --2- 172.21.2.101:0/2680679285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff33c0186c0 0x7ff33c01abc0 crc :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:19.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.081 7ff35d7a5700 1 -- 172.21.2.101:0/2680679285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3581a7860 msgr2=0x7ff3581a7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:19.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.081 7ff35d7a5700 1 --2- 172.21.2.101:0/2680679285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3581a7860 0x7ff3581a7c60 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 rx=0x7ff350009980 tx=0x7ff35000bcd0).stop 2021-02-10T11:36:19.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.081 7ff35d7a5700 1 -- 172.21.2.101:0/2680679285 shutdown_connections 2021-02-10T11:36:19.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.081 7ff35d7a5700 1 --2- 172.21.2.101:0/2680679285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff33c0186c0 0x7ff33c01abc0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:19.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.081 7ff35d7a5700 1 --2- 172.21.2.101:0/2680679285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff358097850 0x7ff3581a7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:19.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.081 7ff35d7a5700 1 --2- 172.21.2.101:0/2680679285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3581a7860 0x7ff3581a7c60 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:19.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.081 7ff35d7a5700 1 --2- 172.21.2.101:0/2680679285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff35818c6c0 0x7ff358092e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:19.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.081 7ff35d7a5700 1 -- 172.21.2.101:0/2680679285 >> 172.21.2.101:0/2680679285 conn(0x7ff3580a74f0 msgr2=0x7ff3580b5140 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:19.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.082 7ff35d7a5700 1 -- 172.21.2.101:0/2680679285 shutdown_connections 2021-02-10T11:36:19.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.082 7ff35d7a5700 1 -- 172.21.2.101:0/2680679285 wait complete. 2021-02-10T11:36:19.097 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:36:19.097 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:36:19.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.503 7f838a876700 1 Processor -- start 2021-02-10T11:36:19.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.503 7f838a876700 1 -- start start 2021-02-10T11:36:19.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.503 7f838a876700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f83840d1870 0x7f83841b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:19.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.503 7f838a876700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f83841b0a10 0x7f83840cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:19.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.503 7f838a876700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83840d1c70 0x7f83840cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.503 7f838a876700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83840e5ed0 con 0x7f83841b0a10 2021-02-10T11:36:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.503 7f838a876700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83840f0b10 con 0x7f83840d1870 2021-02-10T11:36:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.503 7f838a876700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83840cff70 con 0x7f83840d1c70 2021-02-10T11:36:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.503 7f83837fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f83841b0a10 0x7f83840cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.503 7f83837fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f83841b0a10 0x7f83840cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53876/0 (socket says 172.21.2.101:53876) 2021-02-10T11:36:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.503 7f83837fe700 1 -- 172.21.2.101:0/2005978052 learned_addr learned my addr 172.21.2.101:0/2005978052 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.503 7f8388e93700 1 --2- 172.21.2.101:0/2005978052 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83840d1c70 0x7f83840cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.503 7f8383fff700 1 --2- 172.21.2.101:0/2005978052 >> v2:172.21.2.108:3300/0 conn(0x7f83840d1870 0x7f83841b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.504 7f83837fe700 1 -- 172.21.2.101:0/2005978052 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83840d1c70 msgr2=0x7f83840cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.504 7f83837fe700 1 --2- 172.21.2.101:0/2005978052 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83840d1c70 0x7f83840cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.504 7f83837fe700 1 -- 172.21.2.101:0/2005978052 >> v2:172.21.2.108:3300/0 conn(0x7f83840d1870 msgr2=0x7f83841b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:19.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.504 7f83837fe700 1 --2- 172.21.2.101:0/2005978052 >> v2:172.21.2.108:3300/0 conn(0x7f83840d1870 0x7f83841b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.504 7f83837fe700 1 -- 172.21.2.101:0/2005978052 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83840d0150 con 0x7f83841b0a10 2021-02-10T11:36:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.504 7f83837fe700 1 --2- 172.21.2.101:0/2005978052 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83841b0a10 0x7f83840cf330 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 rx=0x7f8378006700 tx=0x7f8378009b80).ready entity=mon.0 client_cookie=39921db8e97300d4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.504 7f8382ffd700 1 -- 172.21.2.101:0/2005978052 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f837800c0c0 con 0x7f83841b0a10 2021-02-10T11:36:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.504 7f8382ffd700 1 -- 172.21.2.101:0/2005978052 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837800c400 con 0x7f83841b0a10 2021-02-10T11:36:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.504 7f8382ffd700 1 -- 172.21.2.101:0/2005978052 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f837800c920 con 0x7f83841b0a10 2021-02-10T11:36:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.504 7f838a876700 1 -- 172.21.2.101:0/2005978052 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83841b0a10 msgr2=0x7f83840cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.504 7f838a876700 1 --2- 172.21.2.101:0/2005978052 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83841b0a10 0x7f83840cf330 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 rx=0x7f8378006700 tx=0x7f8378009b80).stop 2021-02-10T11:36:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.505 7f838a876700 1 -- 172.21.2.101:0/2005978052 shutdown_connections 2021-02-10T11:36:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.505 7f838a876700 1 --2- 172.21.2.101:0/2005978052 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83840d1c70 0x7f83840cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.505 7f838a876700 1 --2- 172.21.2.101:0/2005978052 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83841b0a10 0x7f83840cf330 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.505 7f838a876700 1 --2- 172.21.2.101:0/2005978052 >> v2:172.21.2.108:3300/0 conn(0x7f83840d1870 0x7f83841b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.505 7f838a876700 1 -- 172.21.2.101:0/2005978052 >> 172.21.2.101:0/2005978052 conn(0x7f83840ce450 msgr2=0x7f83840d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.505 7f838a876700 1 -- 172.21.2.101:0/2005978052 shutdown_connections 2021-02-10T11:36:19.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.505 7f838a876700 1 -- 172.21.2.101:0/2005978052 wait complete. 2021-02-10T11:36:19.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.520 7f838a876700 1 Processor -- start 2021-02-10T11:36:19.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.520 7f838a876700 1 -- start start 2021-02-10T11:36:19.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.520 7f838a876700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83841942b0 0x7f83840b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:19.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.520 7f838a876700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8384195070 0x7f8384195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:19.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.520 7f838a876700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83840bb8e0 0x7f83840bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:19.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.520 7f838a876700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83841946b0 con 0x7f83841942b0 2021-02-10T11:36:19.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.520 7f838a876700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83840b3b70 con 0x7f8384195070 2021-02-10T11:36:19.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.520 7f838a876700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83840b3d90 con 0x7f83840bb8e0 2021-02-10T11:36:19.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.520 7f8383fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83841942b0 0x7f83840b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:19.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.521 7f8383fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83841942b0 0x7f83840b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53882/0 (socket says 172.21.2.101:53882) 2021-02-10T11:36:19.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.521 7f8383fff700 1 -- 172.21.2.101:0/1163978657 learned_addr learned my addr 172.21.2.101:0/1163978657 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:19.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.521 7f8388e93700 1 --2- 172.21.2.101:0/1163978657 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83840bb8e0 0x7f83840bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:19.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.521 7f8383fff700 1 -- 172.21.2.101:0/1163978657 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83840bb8e0 msgr2=0x7f83840bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:19.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.521 7f8383fff700 1 --2- 172.21.2.101:0/1163978657 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83840bb8e0 0x7f83840bbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:19.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.521 7f8383fff700 1 -- 172.21.2.101:0/1163978657 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8384195070 msgr2=0x7f8384195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:19.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.521 7f8383fff700 1 --2- 172.21.2.101:0/1163978657 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8384195070 0x7f8384195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:19.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.521 7f8383fff700 1 -- 172.21.2.101:0/1163978657 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83841a49e0 con 0x7f83841942b0 2021-02-10T11:36:19.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.521 7f8388e93700 1 --2- 172.21.2.101:0/1163978657 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83840bb8e0 0x7f83840bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:36:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.521 7f8383fff700 1 --2- 172.21.2.101:0/1163978657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83841942b0 0x7f83840b3550 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 rx=0x7f837c0099e0 tx=0x7f837c00bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.521 7f8380ff9700 1 -- 172.21.2.101:0/1163978657 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f837c00c2f0 con 0x7f83841942b0 2021-02-10T11:36:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.521 7f8380ff9700 1 -- 172.21.2.101:0/1163978657 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837c00c630 con 0x7f83841942b0 2021-02-10T11:36:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.521 7f8380ff9700 1 -- 172.21.2.101:0/1163978657 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f837c00cb50 con 0x7f83841942b0 2021-02-10T11:36:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.521 7f838a876700 1 -- 172.21.2.101:0/1163978657 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83841a49e0 con 0x7f83841942b0 2021-02-10T11:36:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.522 7f838a876700 1 -- 172.21.2.101:0/1163978657 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83840bc5f0 con 0x7f83841942b0 2021-02-10T11:36:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.522 7f8380ff9700 1 -- 172.21.2.101:0/1163978657 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f837c00a020 con 0x7f83841942b0 2021-02-10T11:36:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.522 7f8380ff9700 1 --2- 172.21.2.101:0/1163978657 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83640186c0 0x7f836401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.522 7f8380ff9700 1 -- 172.21.2.101:0/1163978657 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f837c032b50 con 0x7f83841942b0 2021-02-10T11:36:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.522 7f83837fe700 1 --2- 172.21.2.101:0/1163978657 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83640186c0 0x7f836401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:19.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.522 7f83837fe700 1 --2- 172.21.2.101:0/1163978657 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83640186c0 0x7f836401abc0 crc :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:19.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.553 7f838a876700 1 -- 172.21.2.101:0/1163978657 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83840bc5f0 con 0x7f83841942b0 2021-02-10T11:36:19.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.556 7f8380ff9700 1 -- 172.21.2.101:0/1163978657 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f837c00c2f0 con 0x7f83841942b0 2021-02-10T11:36:19.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.685 7f838a876700 1 -- 172.21.2.101:0/1163978657 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83840bc5f0 con 0x7f83841942b0 2021-02-10T11:36:19.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.685 7f8380ff9700 1 -- 172.21.2.101:0/1163978657 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f837c009fe0 con 0x7f83841942b0 2021-02-10T11:36:19.693 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:36:19.693 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":55,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/3016216705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3016216705},{"type":"v1","addr":"172.21.2.108:6833","nonce":3016216705}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":51}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:02.287195","modified":"2021-02-10 11:36:04.295896","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5900},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5900":{"gid":5900,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.2.101:6835/2977571942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2977571942},{"type":"v1","addr":"172.21.2.101:6835","nonce":2977571942}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-02-10T11:36:19.693 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 55 2021-02-10T11:36:19.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.687 7f838a876700 1 -- 172.21.2.101:0/1163978657 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83640186c0 msgr2=0x7f836401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:19.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.687 7f838a876700 1 --2- 172.21.2.101:0/1163978657 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83640186c0 0x7f836401abc0 crc :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:19.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.687 7f838a876700 1 -- 172.21.2.101:0/1163978657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83841942b0 msgr2=0x7f83840b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:19.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.687 7f838a876700 1 --2- 172.21.2.101:0/1163978657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83841942b0 0x7f83840b3550 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 rx=0x7f837c0099e0 tx=0x7f837c00bd40).stop 2021-02-10T11:36:19.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.687 7f838a876700 1 -- 172.21.2.101:0/1163978657 shutdown_connections 2021-02-10T11:36:19.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.687 7f838a876700 1 --2- 172.21.2.101:0/1163978657 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83640186c0 0x7f836401abc0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:19.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.687 7f838a876700 1 --2- 172.21.2.101:0/1163978657 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f83840bb8e0 0x7f83840bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:19.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.687 7f838a876700 1 --2- 172.21.2.101:0/1163978657 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8384195070 0x7f8384195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:19.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.687 7f838a876700 1 --2- 172.21.2.101:0/1163978657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83841942b0 0x7f83840b3550 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:19.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.687 7f838a876700 1 -- 172.21.2.101:0/1163978657 >> 172.21.2.101:0/1163978657 conn(0x7f83840b3040 msgr2=0x7f83840c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:19.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.688 7f838a876700 1 -- 172.21.2.101:0/1163978657 shutdown_connections 2021-02-10T11:36:19.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:19.688 7f838a876700 1 -- 172.21.2.101:0/1163978657 wait complete. 2021-02-10T11:36:19.904 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:36:02.287195', 'modified': '2021-02-10 11:36:04.295896', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5900}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5900': {'gid': 5900, 'name': 'a', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.101:6835/2977571942', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.101:6834', 'nonce': 2977571942}, {'type': 'v1', 'addr': '172.21.2.101:6835', 'nonce': 2977571942}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:36:19.905 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:36:19.905 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok status 2021-02-10T11:36:20.305 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:36:20.305 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:36:20.305 INFO:teuthology.orchestra.run.gibba001.stdout: "whoami": 0, 2021-02-10T11:36:20.306 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 5900, 2021-02-10T11:36:20.306 INFO:teuthology.orchestra.run.gibba001.stdout: "want_state": "up:active", 2021-02-10T11:36:20.306 INFO:teuthology.orchestra.run.gibba001.stdout: "state": "up:active", 2021-02-10T11:36:20.306 INFO:teuthology.orchestra.run.gibba001.stdout: "rank_uptime": 18.010159000000002, 2021-02-10T11:36:20.306 INFO:teuthology.orchestra.run.gibba001.stdout: "mdsmap_epoch": 55, 2021-02-10T11:36:20.306 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch": 74, 2021-02-10T11:36:20.306 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch_barrier": 74, 2021-02-10T11:36:20.306 INFO:teuthology.orchestra.run.gibba001.stdout: "uptime": 23.635836823000002 2021-02-10T11:36:20.306 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:36:20.309 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 5900, "want_state": "up:active", "state": "up:active", "rank_uptime": 18.010159000000002, "mdsmap_epoch": 55, "osdmap_epoch": 74, "osdmap_epoch_barrier": 74, "uptime": 23.635836823000002 } 2021-02-10T11:36:20.310 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:36:20.310 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-02-10T11:36:20.310 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:36:20.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:20.310 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:36:20.645 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:36:20.993 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:36:20.993 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:36:21.335 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:36:21.336 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:36:21.337 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:36:21.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:21.337 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:36:21.673 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:36:21.673 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2021-02-10T11:36:21.673 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:36:21.676 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:36:22.020 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:36:22.020 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:36:22.086 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:2021-02-10 11:36:22.079 7fdbf8424f80 -1 init, newargv = 0x555d00dfd1a0 newargc=9 2021-02-10T11:36:22.086 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[131056]: starting ceph client 2021-02-10T11:36:22.092 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[131056]: starting fuse 2021-02-10T11:36:22.363 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:36:22.364 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:36:22.364 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:36:22.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:22.365 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:36:22.700 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:36:22.700 INFO:teuthology.orchestra.run.gibba001.stdout:44 2021-02-10T11:36:22.700 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2021-02-10T11:36:22.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:22.701 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:36:22.701 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:36:22.701 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:36:22.701 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:36:22.701 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:36:22.701 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:36:22.701 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:36:22.701 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-02-10T11:36:22.701 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:36:22.702 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:36:22.702 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:36:22.702 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:36:22.702 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:36:22.702 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:36:22.702 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:36:22.702 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:36:22.702 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:36:22.702 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:36:22.702 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:36:22.702 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:36:22.702 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:36:22.703 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.1")) 2021-02-10T11:36:22.703 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:36:23.064 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.1.131056.asok 2021-02-10T11:36:23.067 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.131056.asok 2021-02-10T11:36:23.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:23.067 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.131056.asok status 2021-02-10T11:36:23.456 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:36:23.456 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:36:23.456 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:36:23.456 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:36:23.456 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "1", 2021-02-10T11:36:23.457 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:36:23.457 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:36:23.457 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "131056", 2021-02-10T11:36:23.457 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:36:23.457 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:23.457 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:36:23.457 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:36:23.457 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 9171, 2021-02-10T11:36:23.457 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:36:23.457 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:36:23.457 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:36:23.457 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 9171 2021-02-10T11:36:23.457 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:23.458 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:36:23.458 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:23.458 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:36:23.458 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 3725000031 2021-02-10T11:36:23.458 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:23.458 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:23.458 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:36:23.458 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:23.458 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:36:23.458 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 3725000031 2021-02-10T11:36:23.458 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:23.458 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.9171 172.21.2.101:0/3725000031", 2021-02-10T11:36:23.458 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/3725000031", 2021-02-10T11:36:23.458 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:36:23.459 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 55, 2021-02-10T11:36:23.459 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 74, 2021-02-10T11:36:23.459 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:36:23.459 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:36:23.459 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:36:23.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:23.460 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:36:23.461 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:36:23.796 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:36:23.796 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:36:23.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:23.796 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:36:24.143 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2021-02-10T11:36:24.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.551 7fb04f073700 1 Processor -- start 2021-02-10T11:36:24.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.551 7fb04f073700 1 -- start start 2021-02-10T11:36:24.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.551 7fb04f073700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb0480d1880 0x7fb0481b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.551 7fb04f073700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0480cff80 0x7fb048109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.551 7fb04f073700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb048109a70 con 0x7fb0481b0af0 2021-02-10T11:36:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.551 7fb04f073700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb048109760 con 0x7fb0480d1880 2021-02-10T11:36:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.551 7fb04f073700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb048088070 con 0x7fb0480cff80 2021-02-10T11:36:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.552 7fb04ce8f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb0480d1880 0x7fb0481b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.552 7fb04ce8f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb0480d1880 0x7fb0481b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53904/0 (socket says 172.21.2.101:53904) 2021-02-10T11:36:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.552 7fb04ce8f700 1 -- 172.21.2.101:0/3886729205 learned_addr learned my addr 172.21.2.101:0/3886729205 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.552 7fb047fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb0481b0af0 0x7fb048108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54396/0 (socket says 172.21.2.101:54396) 2021-02-10T11:36:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.552 7fb04d690700 1 --2- 172.21.2.101:0/3886729205 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0480cff80 0x7fb048109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.552 7fb04ce8f700 1 -- 172.21.2.101:0/3886729205 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0480cff80 msgr2=0x7fb048109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.552 7fb0477fe700 1 -- 172.21.2.101:0/3886729205 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2987299063 0 0) 0x7fb038000ce0 con 0x7fb0481b0af0 2021-02-10T11:36:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.552 7fb04ce8f700 1 --2- 172.21.2.101:0/3886729205 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0480cff80 0x7fb048109140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.552 7fb04ce8f700 1 -- 172.21.2.101:0/3886729205 >> v1:172.21.2.101:6789/0 conn(0x7fb0481b0af0 legacy=0x7fb048108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:24.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.552 7fb04ce8f700 1 -- 172.21.2.101:0/3886729205 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0480d1c80 con 0x7fb0480d1880 2021-02-10T11:36:24.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.552 7fb04d690700 1 --2- 172.21.2.101:0/3886729205 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0480cff80 0x7fb048109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:36:24.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.552 7fb04ce8f700 1 --2- 172.21.2.101:0/3886729205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0480d1880 0x7fb0481b04d0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 rx=0x7fb030006700 tx=0x7fb030009b80).ready entity=mon.0 client_cookie=2096310fc685c1b4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:24.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.552 7fb0477fe700 1 -- 172.21.2.101:0/3886729205 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb03000c0c0 con 0x7fb0480d1880 2021-02-10T11:36:24.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.553 7fb0477fe700 1 -- 172.21.2.101:0/3886729205 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03000c340 con 0x7fb0480d1880 2021-02-10T11:36:24.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.553 7fb0477fe700 1 -- 172.21.2.101:0/3886729205 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb03000c860 con 0x7fb0480d1880 2021-02-10T11:36:24.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.553 7fb04f073700 1 -- 172.21.2.101:0/3886729205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0480d1880 msgr2=0x7fb0481b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:24.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.553 7fb04f073700 1 --2- 172.21.2.101:0/3886729205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0480d1880 0x7fb0481b04d0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 rx=0x7fb030006700 tx=0x7fb030009b80).stop 2021-02-10T11:36:24.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.553 7fb04f073700 1 -- 172.21.2.101:0/3886729205 shutdown_connections 2021-02-10T11:36:24.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.553 7fb04f073700 1 --2- 172.21.2.101:0/3886729205 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0480cff80 0x7fb048109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:24.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.553 7fb04f073700 1 --2- 172.21.2.101:0/3886729205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0480d1880 0x7fb0481b04d0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:24.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.553 7fb04f073700 1 -- 172.21.2.101:0/3886729205 >> 172.21.2.101:0/3886729205 conn(0x7fb0480ce430 msgr2=0x7fb0480d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:24.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.553 7fb04f073700 1 -- 172.21.2.101:0/3886729205 shutdown_connections 2021-02-10T11:36:24.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.553 7fb04f073700 1 -- 172.21.2.101:0/3886729205 wait complete. 2021-02-10T11:36:24.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.568 7fb04f073700 1 Processor -- start 2021-02-10T11:36:24.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.568 7fb04f073700 1 -- start start 2021-02-10T11:36:24.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.569 7fb04f073700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb04808e830 0x7fb0480adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:24.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.569 7fb04f073700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0480ae120 0x7fb0481ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:24.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.569 7fb04f073700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0481abb10 0x7fb0481abf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:24.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.569 7fb04f073700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04808ed50 con 0x7fb0481abb10 2021-02-10T11:36:24.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.569 7fb04f073700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb048085930 con 0x7fb0480ae120 2021-02-10T11:36:24.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.569 7fb04f073700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04809c6d0 con 0x7fb04808e830 2021-02-10T11:36:24.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.569 7fb04d690700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0481abb10 0x7fb0481abf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:24.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.569 7fb04d690700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0481abb10 0x7fb0481abf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53910/0 (socket says 172.21.2.101:53910) 2021-02-10T11:36:24.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.569 7fb04d690700 1 -- 172.21.2.101:0/2590829633 learned_addr learned my addr 172.21.2.101:0/2590829633 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:24.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.569 7fb04ce8f700 1 --2- 172.21.2.101:0/2590829633 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb04808e830 0x7fb0480adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:24.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.569 7fb04d690700 1 -- 172.21.2.101:0/2590829633 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb04808e830 msgr2=0x7fb0480adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:24.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.569 7fb047fff700 1 --2- 172.21.2.101:0/2590829633 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0480ae120 0x7fb0481ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:24.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.569 7fb04d690700 1 --2- 172.21.2.101:0/2590829633 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb04808e830 0x7fb0480adb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:24.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.569 7fb04d690700 1 -- 172.21.2.101:0/2590829633 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0480ae120 msgr2=0x7fb0481ab4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:24.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.570 7fb04d690700 1 --2- 172.21.2.101:0/2590829633 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0480ae120 0x7fb0481ab4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:24.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.570 7fb04d690700 1 -- 172.21.2.101:0/2590829633 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04808f2d0 con 0x7fb0481abb10 2021-02-10T11:36:24.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.570 7fb04ce8f700 1 --2- 172.21.2.101:0/2590829633 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb04808e830 0x7fb0480adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:36:24.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.570 7fb047fff700 1 --2- 172.21.2.101:0/2590829633 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0480ae120 0x7fb0481ab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:36:24.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.570 7fb04d690700 1 --2- 172.21.2.101:0/2590829633 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0481abb10 0x7fb0481abf10 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 rx=0x7fb040009980 tx=0x7fb04000bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:24.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.570 7fb0457fa700 1 -- 172.21.2.101:0/2590829633 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb04000c2f0 con 0x7fb0481abb10 2021-02-10T11:36:24.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.570 7fb0457fa700 1 -- 172.21.2.101:0/2590829633 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb040006790 con 0x7fb0481abb10 2021-02-10T11:36:24.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.570 7fb0457fa700 1 -- 172.21.2.101:0/2590829633 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb04000c870 con 0x7fb0481abb10 2021-02-10T11:36:24.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.570 7fb04f073700 1 -- 172.21.2.101:0/2590829633 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04808f2d0 con 0x7fb0481abb10 2021-02-10T11:36:24.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.570 7fb04f073700 1 -- 172.21.2.101:0/2590829633 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04808f2d0 con 0x7fb0481abb10 2021-02-10T11:36:24.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.570 7fb0457fa700 1 -- 172.21.2.101:0/2590829633 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb0400011f0 con 0x7fb0481abb10 2021-02-10T11:36:24.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.571 7fb0457fa700 1 --2- 172.21.2.101:0/2590829633 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb02c0186c0 0x7fb02c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:24.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.571 7fb0457fa700 1 -- 172.21.2.101:0/2590829633 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb0400327a0 con 0x7fb0481abb10 2021-02-10T11:36:24.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.571 7fb04ce8f700 1 --2- 172.21.2.101:0/2590829633 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb02c0186c0 0x7fb02c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:24.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.571 7fb04ce8f700 1 --2- 172.21.2.101:0/2590829633 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb02c0186c0 0x7fb02c01abc0 crc :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:24.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.601 7fb04f073700 1 -- 172.21.2.101:0/2590829633 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb048082a70 con 0x7fb0481abb10 2021-02-10T11:36:24.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.604 7fb0457fa700 1 -- 172.21.2.101:0/2590829633 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb040030150 con 0x7fb0481abb10 2021-02-10T11:36:24.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:24.739 7fb04f073700 1 -- 172.21.2.101:0/2590829633 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb048082a70 con 0x7fb0481abb10 2021-02-10T11:36:25.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.498 7fb0457fa700 1 -- 172.21.2.101:0/2590829633 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v56) v1 ==== 136+0+0 (secure 0 0 0) 0x7fb040030150 con 0x7fb0481abb10 2021-02-10T11:36:25.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.511 7fb04f073700 1 -- 172.21.2.101:0/2590829633 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb02c0186c0 msgr2=0x7fb02c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:25.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.511 7fb04f073700 1 --2- 172.21.2.101:0/2590829633 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb02c0186c0 0x7fb02c01abc0 crc :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:25.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.511 7fb04f073700 1 -- 172.21.2.101:0/2590829633 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0481abb10 msgr2=0x7fb0481abf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:25.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.511 7fb04f073700 1 --2- 172.21.2.101:0/2590829633 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0481abb10 0x7fb0481abf10 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 rx=0x7fb040009980 tx=0x7fb04000bcd0).stop 2021-02-10T11:36:25.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.511 7fb04f073700 1 -- 172.21.2.101:0/2590829633 shutdown_connections 2021-02-10T11:36:25.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.511 7fb04f073700 1 --2- 172.21.2.101:0/2590829633 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0481abb10 0x7fb0481abf10 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:25.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.511 7fb04f073700 1 --2- 172.21.2.101:0/2590829633 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0480ae120 0x7fb0481ab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:25.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.511 7fb04f073700 1 --2- 172.21.2.101:0/2590829633 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb02c0186c0 0x7fb02c01abc0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:25.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.511 7fb04f073700 1 --2- 172.21.2.101:0/2590829633 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb04808e830 0x7fb0480adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:25.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.511 7fb04f073700 1 -- 172.21.2.101:0/2590829633 >> 172.21.2.101:0/2590829633 conn(0x7fb04819c630 msgr2=0x7fb048082ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:25.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.511 7fb04f073700 1 -- 172.21.2.101:0/2590829633 shutdown_connections 2021-02-10T11:36:25.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.511 7fb04f073700 1 -- 172.21.2.101:0/2590829633 wait complete. 2021-02-10T11:36:25.526 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs volume ls 2021-02-10T11:36:25.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.929 7f068f871700 1 Processor -- start 2021-02-10T11:36:25.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.929 7f068f871700 1 -- start start 2021-02-10T11:36:25.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.929 7f068f871700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f06880d0340 0x7f06880ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:25.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.929 7f068f871700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06880d1c70 0x7f06880efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:25.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.929 7f068f871700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06880f0b10 con 0x7f06880d1870 2021-02-10T11:36:25.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.929 7f068f871700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06880e5ed0 con 0x7f06880d0340 2021-02-10T11:36:25.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.929 7f068f871700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06881b0be0 con 0x7f06880d1c70 2021-02-10T11:36:25.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.930 7f068d68d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f06880d1870 0x7f06881b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54410/0 (socket says 172.21.2.101:54410) 2021-02-10T11:36:25.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.930 7f068d68d700 1 -- 172.21.2.101:0/4087151443 learned_addr learned my addr 172.21.2.101:0/4087151443 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:25.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.930 7f067bfff700 1 -- 172.21.2.101:0/4087151443 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960888266 0 0) 0x7f067c0010b0 con 0x7f06880d1870 2021-02-10T11:36:25.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.930 7f067bfff700 1 -- 172.21.2.101:0/4087151443 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0674002fe0 con 0x7f06880d1870 2021-02-10T11:36:25.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.930 7f068de8e700 1 --2- 172.21.2.101:0/4087151443 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06880d1c70 0x7f06880efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:25.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.930 7f068ce8c700 1 --2- 172.21.2.101:0/4087151443 >> v2:172.21.2.108:3300/0 conn(0x7f06880d0340 0x7f06880ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:25.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.930 7f067bfff700 1 -- 172.21.2.101:0/4087151443 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2227649507 0 0) 0x7f067c0010b0 con 0x7f06880d1870 2021-02-10T11:36:25.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.930 7f067bfff700 1 -- 172.21.2.101:0/4087151443 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06880d1c70 msgr2=0x7f06880efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:25.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.930 7f067bfff700 1 --2- 172.21.2.101:0/4087151443 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06880d1c70 0x7f06880efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:25.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.930 7f067bfff700 1 -- 172.21.2.101:0/4087151443 >> v2:172.21.2.108:3300/0 conn(0x7f06880d0340 msgr2=0x7f06880ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:25.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.930 7f067bfff700 1 --2- 172.21.2.101:0/4087151443 >> v2:172.21.2.108:3300/0 conn(0x7f06880d0340 0x7f06880ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:25.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.930 7f067bfff700 1 -- 172.21.2.101:0/4087151443 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06880f20a0 con 0x7f06880d1870 2021-02-10T11:36:25.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.930 7f067bfff700 1 -- 172.21.2.101:0/4087151443 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f067c0015b0 con 0x7f06880d1870 2021-02-10T11:36:25.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.930 7f067bfff700 1 -- 172.21.2.101:0/4087151443 >> v1:172.21.2.101:6789/0 conn(0x7f06880d1870 legacy=0x7f06881b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:25.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.930 7f067bfff700 1 --2- 172.21.2.101:0/4087151443 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0674003ac0 0x7f0674005ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:25.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.930 7f067bfff700 1 --2- 172.21.2.101:0/4087151443 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f067404a780 0x7f067404cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:25.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.930 7f067bfff700 1 --2- 172.21.2.101:0/4087151443 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f067404d520 0x7f067404fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:25.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.930 7f067bfff700 1 -- 172.21.2.101:0/4087151443 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06740502f0 con 0x7f067404d520 2021-02-10T11:36:25.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.930 7f067bfff700 1 -- 172.21.2.101:0/4087151443 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0674050510 con 0x7f067404a780 2021-02-10T11:36:25.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.930 7f067bfff700 1 -- 172.21.2.101:0/4087151443 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0674050730 con 0x7f0674003ac0 2021-02-10T11:36:25.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.931 7f068de8e700 1 --2- 172.21.2.101:0/4087151443 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f067404d520 0x7f067404fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:25.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.931 7f068de8e700 1 -- 172.21.2.101:0/4087151443 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0674003ac0 msgr2=0x7f0674005ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:25.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.931 7f068de8e700 1 --2- 172.21.2.101:0/4087151443 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0674003ac0 0x7f0674005ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:25.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.931 7f068de8e700 1 -- 172.21.2.101:0/4087151443 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f067404a780 msgr2=0x7f067404cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:25.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.931 7f068de8e700 1 --2- 172.21.2.101:0/4087151443 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f067404a780 0x7f067404cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:25.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.931 7f068de8e700 1 -- 172.21.2.101:0/4087151443 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06740509d0 con 0x7f067404d520 2021-02-10T11:36:25.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.931 7f068de8e700 1 --2- 172.21.2.101:0/4087151443 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f067404d520 0x7f067404fa40 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 rx=0x7f0680009b10 tx=0x7f0680009eb0).ready entity=mon.0 client_cookie=a09502d22b86f89c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:25.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.931 7f067bfff700 1 -- 172.21.2.101:0/4087151443 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f068000b510 con 0x7f067404d520 2021-02-10T11:36:25.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.931 7f067bfff700 1 -- 172.21.2.101:0/4087151443 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0680003030 con 0x7f067404d520 2021-02-10T11:36:25.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.931 7f067bfff700 1 -- 172.21.2.101:0/4087151443 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f068000b790 con 0x7f067404d520 2021-02-10T11:36:25.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.931 7f068f871700 1 -- 172.21.2.101:0/4087151443 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f067404d520 msgr2=0x7f067404fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:25.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.931 7f068f871700 1 --2- 172.21.2.101:0/4087151443 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f067404d520 0x7f067404fa40 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 rx=0x7f0680009b10 tx=0x7f0680009eb0).stop 2021-02-10T11:36:25.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.931 7f068de8e700 1 -- 172.21.2.101:0/4087151443 reap_dead start 2021-02-10T11:36:25.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.931 7f068f871700 1 -- 172.21.2.101:0/4087151443 shutdown_connections 2021-02-10T11:36:25.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.931 7f068f871700 1 -- 172.21.2.101:0/4087151443 >> 172.21.2.101:0/4087151443 conn(0x7f06880ce450 msgr2=0x7f06880d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:25.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.932 7f068f871700 1 -- 172.21.2.101:0/4087151443 shutdown_connections 2021-02-10T11:36:25.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.932 7f068f871700 1 -- 172.21.2.101:0/4087151443 wait complete. 2021-02-10T11:36:25.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.946 7f068f871700 1 Processor -- start 2021-02-10T11:36:25.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.946 7f068f871700 1 -- start start 2021-02-10T11:36:25.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.947 7f068f871700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f068819ffa0 0x7f06880eda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:25.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.947 7f068f871700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06881a03a0 0x7f06880ee030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:25.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.947 7f068f871700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06880ee650 0x7f06880ebe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:25.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.947 7f068f871700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06880ed490 con 0x7f06881a03a0 2021-02-10T11:36:25.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.947 7f068f871700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06880eea50 con 0x7f068819ffa0 2021-02-10T11:36:25.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.947 7f068f871700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06880eec30 con 0x7f06880ee650 2021-02-10T11:36:25.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.947 7f068ce8c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06881a03a0 0x7f06880ee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:25.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.947 7f068ce8c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06881a03a0 0x7f06880ee030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53926/0 (socket says 172.21.2.101:53926) 2021-02-10T11:36:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.947 7f068ce8c700 1 -- 172.21.2.101:0/1939322488 learned_addr learned my addr 172.21.2.101:0/1939322488 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.947 7f068de8e700 1 --2- 172.21.2.101:0/1939322488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06880ee650 0x7f06880ebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.947 7f068d68d700 1 --2- 172.21.2.101:0/1939322488 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f068819ffa0 0x7f06880eda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.948 7f068d68d700 1 -- 172.21.2.101:0/1939322488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06880ee650 msgr2=0x7f06880ebe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.948 7f068d68d700 1 --2- 172.21.2.101:0/1939322488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06880ee650 0x7f06880ebe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.948 7f068d68d700 1 -- 172.21.2.101:0/1939322488 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06881a03a0 msgr2=0x7f06880ee030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.948 7f068d68d700 1 --2- 172.21.2.101:0/1939322488 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06881a03a0 0x7f06880ee030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.948 7f068d68d700 1 -- 172.21.2.101:0/1939322488 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06880ec4a0 con 0x7f068819ffa0 2021-02-10T11:36:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.948 7f068ce8c700 1 --2- 172.21.2.101:0/1939322488 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06881a03a0 0x7f06880ee030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:36:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.948 7f068d68d700 1 --2- 172.21.2.101:0/1939322488 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f068819ffa0 0x7f06880eda10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f067c007f90 tx=0x7f067c00a800).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.948 7f0679ffb700 1 -- 172.21.2.101:0/1939322488 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f067c000e20 con 0x7f068819ffa0 2021-02-10T11:36:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.948 7f0679ffb700 1 -- 172.21.2.101:0/1939322488 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067c000940 con 0x7f068819ffa0 2021-02-10T11:36:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.948 7f0679ffb700 1 -- 172.21.2.101:0/1939322488 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f067c004350 con 0x7f068819ffa0 2021-02-10T11:36:25.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.948 7f068f871700 1 -- 172.21.2.101:0/1939322488 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06880ec4a0 con 0x7f068819ffa0 2021-02-10T11:36:25.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.948 7f068f871700 1 -- 172.21.2.101:0/1939322488 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06880eca30 con 0x7f068819ffa0 2021-02-10T11:36:25.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.949 7f0679ffb700 1 -- 172.21.2.101:0/1939322488 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f067c009030 con 0x7f068819ffa0 2021-02-10T11:36:25.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.949 7f0679ffb700 1 --2- 172.21.2.101:0/1939322488 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06680186c0 0x7f066801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:25.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.949 7f0679ffb700 1 -- 172.21.2.101:0/1939322488 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f067c0324c0 con 0x7f068819ffa0 2021-02-10T11:36:25.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.949 7f068ce8c700 1 --2- 172.21.2.101:0/1939322488 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06680186c0 0x7f066801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:25.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.949 7f068ce8c700 1 --2- 172.21.2.101:0/1939322488 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06680186c0 0x7f066801abc0 crc :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:25.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.979 7f068f871700 1 -- 172.21.2.101:0/1939322488 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06880ebc50 con 0x7f068819ffa0 2021-02-10T11:36:25.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:25.982 7f0679ffb700 1 -- 172.21.2.101:0/1939322488 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f067c0257e0 con 0x7f068819ffa0 2021-02-10T11:36:26.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.116 7f068f871700 1 -- 172.21.2.101:0/1939322488 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs volume ls", "target": ["mgr", ""]}) v1 -- 0x7f06880eca30 con 0x7f06680186c0 2021-02-10T11:36:26.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.117 7f0679ffb700 1 -- 172.21.2.101:0/1939322488 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7f0684006d80 con 0x7f06680186c0 2021-02-10T11:36:26.124 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:36:26.124 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:26.124 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "cephfs" 2021-02-10T11:36:26.124 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:26.124 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:36:26.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.118 7f068f871700 1 -- 172.21.2.101:0/1939322488 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06680186c0 msgr2=0x7f066801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:26.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.118 7f068f871700 1 --2- 172.21.2.101:0/1939322488 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06680186c0 0x7f066801abc0 crc :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:26.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.118 7f068f871700 1 -- 172.21.2.101:0/1939322488 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f068819ffa0 msgr2=0x7f06880eda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:26.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.118 7f068f871700 1 --2- 172.21.2.101:0/1939322488 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f068819ffa0 0x7f06880eda10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f067c007f90 tx=0x7f067c00a800).stop 2021-02-10T11:36:26.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.119 7f068f871700 1 -- 172.21.2.101:0/1939322488 shutdown_connections 2021-02-10T11:36:26.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.119 7f068f871700 1 --2- 172.21.2.101:0/1939322488 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06680186c0 0x7f066801abc0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:26.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.119 7f068f871700 1 --2- 172.21.2.101:0/1939322488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06880ee650 0x7f06880ebe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:26.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.119 7f068f871700 1 --2- 172.21.2.101:0/1939322488 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06881a03a0 0x7f06880ee030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:26.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.119 7f068f871700 1 --2- 172.21.2.101:0/1939322488 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f068819ffa0 0x7f06880eda10 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:26.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.119 7f068f871700 1 -- 172.21.2.101:0/1939322488 >> 172.21.2.101:0/1939322488 conn(0x7f0688094d90 msgr2=0x7f068808a420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:26.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.119 7f068f871700 1 -- 172.21.2.101:0/1939322488 shutdown_connections 2021-02-10T11:36:26.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.119 7f068f871700 1 -- 172.21.2.101:0/1939322488 wait complete. 2021-02-10T11:36:26.134 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:36:26.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.543 7f143e10a700 1 Processor -- start 2021-02-10T11:36:26.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.543 7f143e10a700 1 -- start start 2021-02-10T11:36:26.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.543 7f143e10a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f14380d1c70 0x7f14380cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:26.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.543 7f143e10a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14380cf580 0x7f14380cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:26.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.543 7f143e10a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14380fe1c0 con 0x7f14380d1870 2021-02-10T11:36:26.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.543 7f143e10a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14381b0cb0 con 0x7f14380d1c70 2021-02-10T11:36:26.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.543 7f143e10a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1438109a60 con 0x7f14380cf580 2021-02-10T11:36:26.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.543 7f1436ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f14380d1c70 0x7f14380cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:26.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.543 7f1436ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f14380d1c70 0x7f14380cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53934/0 (socket says 172.21.2.101:53934) 2021-02-10T11:36:26.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.543 7f1436ffd700 1 -- 172.21.2.101:0/3061869355 learned_addr learned my addr 172.21.2.101:0/3061869355 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:26.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.544 7f1437fff700 1 --2- 172.21.2.101:0/3061869355 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14380cf580 0x7f14380cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:26.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.544 7f1436ffd700 1 -- 172.21.2.101:0/3061869355 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14380cf580 msgr2=0x7f14380cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:26.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.544 7f1436ffd700 1 --2- 172.21.2.101:0/3061869355 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14380cf580 0x7f14380cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:26.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.544 7f1436ffd700 1 -- 172.21.2.101:0/3061869355 >> v1:172.21.2.108:6789/0 conn(0x7f14380d1870 legacy=0x7f14381b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:26.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.544 7f1436ffd700 1 -- 172.21.2.101:0/3061869355 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1438088070 con 0x7f14380d1c70 2021-02-10T11:36:26.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.544 7f1437fff700 1 --2- 172.21.2.101:0/3061869355 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14380cf580 0x7f14380cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:36:26.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.544 7f1436ffd700 1 --2- 172.21.2.101:0/3061869355 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14380d1c70 0x7f14380cef60 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 rx=0x7f142c006700 tx=0x7f142c009b80).ready entity=mon.0 client_cookie=4f23b1eaa45dc50f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:26.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.544 7f14367fc700 1 -- 172.21.2.101:0/3061869355 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f142c00c0c0 con 0x7f14380d1c70 2021-02-10T11:36:26.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.544 7f14367fc700 1 -- 172.21.2.101:0/3061869355 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142c00c400 con 0x7f14380d1c70 2021-02-10T11:36:26.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.544 7f14367fc700 1 -- 172.21.2.101:0/3061869355 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f142c00c8e0 con 0x7f14380d1c70 2021-02-10T11:36:26.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.544 7f143e10a700 1 -- 172.21.2.101:0/3061869355 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14380d1c70 msgr2=0x7f14380cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:26.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.544 7f143e10a700 1 --2- 172.21.2.101:0/3061869355 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14380d1c70 0x7f14380cef60 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 rx=0x7f142c006700 tx=0x7f142c009b80).stop 2021-02-10T11:36:26.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.545 7f143e10a700 1 -- 172.21.2.101:0/3061869355 shutdown_connections 2021-02-10T11:36:26.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.545 7f143e10a700 1 --2- 172.21.2.101:0/3061869355 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14380cf580 0x7f14380cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:26.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.545 7f143e10a700 1 --2- 172.21.2.101:0/3061869355 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14380d1c70 0x7f14380cef60 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:26.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.545 7f143e10a700 1 -- 172.21.2.101:0/3061869355 >> 172.21.2.101:0/3061869355 conn(0x7f14380ce420 msgr2=0x7f14380d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:26.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.545 7f143e10a700 1 -- 172.21.2.101:0/3061869355 shutdown_connections 2021-02-10T11:36:26.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.545 7f143e10a700 1 -- 172.21.2.101:0/3061869355 wait complete. 2021-02-10T11:36:26.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.559 7f143e10a700 1 Processor -- start 2021-02-10T11:36:26.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.560 7f143e10a700 1 -- start start 2021-02-10T11:36:26.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.560 7f143e10a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1438191b10 0x7f14380ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:26.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.560 7f143e10a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1438191f10 0x7f143819ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:26.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.560 7f143e10a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14381a05f0 0x7f1438082ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:26.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.560 7f143e10a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1438192440 con 0x7f1438191f10 2021-02-10T11:36:26.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.560 7f143e10a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14381916e0 con 0x7f14381a05f0 2021-02-10T11:36:26.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.560 7f143e10a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1438191900 con 0x7f1438191b10 2021-02-10T11:36:26.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.560 7f1436ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1438191f10 0x7f143819ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:26.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.560 7f1436ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1438191f10 0x7f143819ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53942/0 (socket says 172.21.2.101:53942) 2021-02-10T11:36:26.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.560 7f1436ffd700 1 -- 172.21.2.101:0/2131727413 learned_addr learned my addr 172.21.2.101:0/2131727413 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:26.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.561 7f14377fe700 1 --2- 172.21.2.101:0/2131727413 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1438191b10 0x7f14380ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:26.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.561 7f1437fff700 1 --2- 172.21.2.101:0/2131727413 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14381a05f0 0x7f1438082ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:26.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.561 7f1436ffd700 1 -- 172.21.2.101:0/2131727413 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1438191b10 msgr2=0x7f14380ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:26.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.561 7f1436ffd700 1 --2- 172.21.2.101:0/2131727413 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1438191b10 0x7f14380ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:26.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.561 7f1436ffd700 1 -- 172.21.2.101:0/2131727413 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14381a05f0 msgr2=0x7f1438082ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:26.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.561 7f1436ffd700 1 --2- 172.21.2.101:0/2131727413 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14381a05f0 0x7f1438082ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:26.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.561 7f1436ffd700 1 -- 172.21.2.101:0/2131727413 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14380834f0 con 0x7f1438191f10 2021-02-10T11:36:26.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.561 7f1436ffd700 1 --2- 172.21.2.101:0/2131727413 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1438191f10 0x7f143819ffd0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 rx=0x7f142c006420 tx=0x7f142c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:26.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.561 7f141ffff700 1 -- 172.21.2.101:0/2131727413 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f142c009440 con 0x7f1438191f10 2021-02-10T11:36:26.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.561 7f141ffff700 1 -- 172.21.2.101:0/2131727413 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142c00bd50 con 0x7f1438191f10 2021-02-10T11:36:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.561 7f141ffff700 1 -- 172.21.2.101:0/2131727413 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f142c005560 con 0x7f1438191f10 2021-02-10T11:36:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.561 7f143e10a700 1 -- 172.21.2.101:0/2131727413 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14380834f0 con 0x7f1438191f10 2021-02-10T11:36:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.562 7f143e10a700 1 -- 172.21.2.101:0/2131727413 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1438083a80 con 0x7f1438191f10 2021-02-10T11:36:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.562 7f141ffff700 1 -- 172.21.2.101:0/2131727413 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f142c0052e0 con 0x7f1438191f10 2021-02-10T11:36:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.562 7f141ffff700 1 --2- 172.21.2.101:0/2131727413 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14180186c0 0x7f141801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.562 7f141ffff700 1 -- 172.21.2.101:0/2131727413 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f142c032400 con 0x7f1438191f10 2021-02-10T11:36:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.562 7f14377fe700 1 --2- 172.21.2.101:0/2131727413 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14180186c0 0x7f141801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.563 7f14377fe700 1 --2- 172.21.2.101:0/2131727413 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14180186c0 0x7f141801abc0 crc :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:26.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.593 7f143e10a700 1 -- 172.21.2.101:0/2131727413 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1438192440 con 0x7f1438191f10 2021-02-10T11:36:26.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.596 7f141ffff700 1 -- 172.21.2.101:0/2131727413 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f142c021bb0 con 0x7f1438191f10 2021-02-10T11:36:26.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.745 7f143e10a700 1 -- 172.21.2.101:0/2131727413 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1438192440 con 0x7f1438191f10 2021-02-10T11:36:26.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.746 7f141ffff700 1 -- 172.21.2.101:0/2131727413 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f142c021bb0 con 0x7f1438191f10 2021-02-10T11:36:26.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.747 7f143e10a700 1 -- 172.21.2.101:0/2131727413 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14180186c0 msgr2=0x7f141801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:26.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.747 7f143e10a700 1 --2- 172.21.2.101:0/2131727413 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14180186c0 0x7f141801abc0 crc :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:26.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.747 7f143e10a700 1 -- 172.21.2.101:0/2131727413 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1438191f10 msgr2=0x7f143819ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:26.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.747 7f143e10a700 1 --2- 172.21.2.101:0/2131727413 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1438191f10 0x7f143819ffd0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 rx=0x7f142c006420 tx=0x7f142c00b960).stop 2021-02-10T11:36:26.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.747 7f143e10a700 1 -- 172.21.2.101:0/2131727413 shutdown_connections 2021-02-10T11:36:26.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.747 7f143e10a700 1 --2- 172.21.2.101:0/2131727413 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14381a05f0 0x7f1438082ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:26.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.747 7f143e10a700 1 --2- 172.21.2.101:0/2131727413 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1438191f10 0x7f143819ffd0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:26.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.747 7f143e10a700 1 --2- 172.21.2.101:0/2131727413 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14180186c0 0x7f141801abc0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:26.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.747 7f143e10a700 1 --2- 172.21.2.101:0/2131727413 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1438191b10 0x7f14380ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:26.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.748 7f143e10a700 1 -- 172.21.2.101:0/2131727413 >> 172.21.2.101:0/2131727413 conn(0x7f14380a02d0 msgr2=0x7f143809f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:26.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.748 7f143e10a700 1 -- 172.21.2.101:0/2131727413 shutdown_connections 2021-02-10T11:36:26.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:26.748 7f143e10a700 1 -- 172.21.2.101:0/2131727413 wait complete. 2021-02-10T11:36:26.762 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000678861 2021-02-10T11:36:27.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.169 7fcc5de6c700 1 Processor -- start 2021-02-10T11:36:27.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.169 7fcc5de6c700 1 -- start start 2021-02-10T11:36:27.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.170 7fcc5de6c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcc580d1870 0x7fcc581b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:27.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.170 7fcc5de6c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc580fe1c0 con 0x7fcc581b0ae0 2021-02-10T11:36:27.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.170 7fcc5de6c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc58109a60 con 0x7fcc580cff70 2021-02-10T11:36:27.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.170 7fcc5de6c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc58088070 con 0x7fcc580d1870 2021-02-10T11:36:27.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.170 7fcc577fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcc580d1870 0x7fcc581b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:27.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.170 7fcc56ffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fcc581b0ae0 0x7fcc58108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54440/0 (socket says 172.21.2.101:54440) 2021-02-10T11:36:27.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.170 7fcc56ffd700 1 -- 172.21.2.101:0/584557282 learned_addr learned my addr 172.21.2.101:0/584557282 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:27.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.170 7fcc567fc700 1 -- 172.21.2.101:0/584557282 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3943363989 0 0) 0x7fcc480009b0 con 0x7fcc581b0ae0 2021-02-10T11:36:27.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.170 7fcc567fc700 1 -- 172.21.2.101:0/584557282 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc44002fe0 con 0x7fcc581b0ae0 2021-02-10T11:36:27.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.170 7fcc577fe700 1 -- 172.21.2.101:0/584557282 >> v1:172.21.2.108:6789/0 conn(0x7fcc580cff70 legacy=0x7fcc58109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:27.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.171 7fcc577fe700 1 -- 172.21.2.101:0/584557282 >> v1:172.21.2.101:6789/0 conn(0x7fcc581b0ae0 legacy=0x7fcc58108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:27.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.171 7fcc577fe700 1 -- 172.21.2.101:0/584557282 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc58088260 con 0x7fcc580d1870 2021-02-10T11:36:27.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.171 7fcc577fe700 1 --2- 172.21.2.101:0/584557282 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc580d1870 0x7fcc581b04c0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 rx=0x7fcc400069d0 tx=0x7fcc40009e40).ready entity=mon.0 client_cookie=845fc8574736c179 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:27.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.171 7fcc567fc700 1 -- 172.21.2.101:0/584557282 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcc4000b510 con 0x7fcc580d1870 2021-02-10T11:36:27.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.171 7fcc567fc700 1 -- 172.21.2.101:0/584557282 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4000b890 con 0x7fcc580d1870 2021-02-10T11:36:27.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.171 7fcc5de6c700 1 -- 172.21.2.101:0/584557282 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc580d1870 msgr2=0x7fcc581b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:27.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.171 7fcc567fc700 1 -- 172.21.2.101:0/584557282 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcc4000bdb0 con 0x7fcc580d1870 2021-02-10T11:36:27.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.171 7fcc5de6c700 1 --2- 172.21.2.101:0/584557282 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc580d1870 0x7fcc581b04c0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 rx=0x7fcc400069d0 tx=0x7fcc40009e40).stop 2021-02-10T11:36:27.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.172 7fcc5de6c700 1 -- 172.21.2.101:0/584557282 shutdown_connections 2021-02-10T11:36:27.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.172 7fcc5de6c700 1 --2- 172.21.2.101:0/584557282 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc580d1870 0x7fcc581b04c0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:27.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.172 7fcc5de6c700 1 -- 172.21.2.101:0/584557282 >> 172.21.2.101:0/584557282 conn(0x7fcc580ce420 msgr2=0x7fcc580d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:27.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.172 7fcc5de6c700 1 -- 172.21.2.101:0/584557282 shutdown_connections 2021-02-10T11:36:27.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.172 7fcc5de6c700 1 -- 172.21.2.101:0/584557282 wait complete. 2021-02-10T11:36:27.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.186 7fcc5de6c700 1 Processor -- start 2021-02-10T11:36:27.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.187 7fcc5de6c700 1 -- start start 2021-02-10T11:36:27.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.187 7fcc5de6c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc580c3820 0x7fcc580a7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:27.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.187 7fcc5de6c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcc581a44c0 0x7fcc581a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:27.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.187 7fcc5de6c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcc580863d0 0x7fcc580867d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:27.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.187 7fcc5de6c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc58085ed0 con 0x7fcc580c3820 2021-02-10T11:36:27.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.187 7fcc5de6c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc580c0740 con 0x7fcc580863d0 2021-02-10T11:36:27.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.187 7fcc5de6c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc58086df0 con 0x7fcc581a44c0 2021-02-10T11:36:27.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.187 7fcc577fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc580c3820 0x7fcc580a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:27.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.187 7fcc577fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc580c3820 0x7fcc580a7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53952/0 (socket says 172.21.2.101:53952) 2021-02-10T11:36:27.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.187 7fcc577fe700 1 -- 172.21.2.101:0/4107350205 learned_addr learned my addr 172.21.2.101:0/4107350205 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:27.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.188 7fcc577fe700 1 -- 172.21.2.101:0/4107350205 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcc581a44c0 msgr2=0x7fcc581a48c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:27.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.188 7fcc577fe700 1 --2- 172.21.2.101:0/4107350205 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcc581a44c0 0x7fcc581a48c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:27.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.188 7fcc577fe700 1 -- 172.21.2.101:0/4107350205 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcc580863d0 msgr2=0x7fcc580867d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:27.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.188 7fcc577fe700 1 --2- 172.21.2.101:0/4107350205 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcc580863d0 0x7fcc580867d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:27.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.188 7fcc577fe700 1 -- 172.21.2.101:0/4107350205 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc58087010 con 0x7fcc580c3820 2021-02-10T11:36:27.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.188 7fcc577fe700 1 --2- 172.21.2.101:0/4107350205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc580c3820 0x7fcc580a7950 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 rx=0x7fcc4000c9c0 tx=0x7fcc4000cd60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:27.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.188 7fcc4ffff700 1 -- 172.21.2.101:0/4107350205 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcc40006280 con 0x7fcc580c3820 2021-02-10T11:36:27.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.188 7fcc4ffff700 1 -- 172.21.2.101:0/4107350205 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc40006580 con 0x7fcc580c3820 2021-02-10T11:36:27.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.188 7fcc4ffff700 1 -- 172.21.2.101:0/4107350205 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcc40005850 con 0x7fcc580c3820 2021-02-10T11:36:27.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.188 7fcc5de6c700 1 -- 172.21.2.101:0/4107350205 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc58087010 con 0x7fcc580c3820 2021-02-10T11:36:27.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.188 7fcc5de6c700 1 -- 172.21.2.101:0/4107350205 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc58087010 con 0x7fcc580c3820 2021-02-10T11:36:27.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.189 7fcc4ffff700 1 -- 172.21.2.101:0/4107350205 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fcc40005660 con 0x7fcc580c3820 2021-02-10T11:36:27.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.189 7fcc4ffff700 1 --2- 172.21.2.101:0/4107350205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcc3c0186c0 0x7fcc3c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:27.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.189 7fcc4ffff700 1 -- 172.21.2.101:0/4107350205 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fcc40032b40 con 0x7fcc580c3820 2021-02-10T11:36:27.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.189 7fcc56ffd700 1 --2- 172.21.2.101:0/4107350205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcc3c0186c0 0x7fcc3c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:27.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.189 7fcc56ffd700 1 --2- 172.21.2.101:0/4107350205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcc3c0186c0 0x7fcc3c01abc0 crc :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:27.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.220 7fcc5de6c700 1 -- 172.21.2.101:0/4107350205 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc580fe1c0 con 0x7fcc580c3820 2021-02-10T11:36:27.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.222 7fcc4ffff700 1 -- 172.21.2.101:0/4107350205 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fcc40006580 con 0x7fcc580c3820 2021-02-10T11:36:27.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.349 7fcc5de6c700 1 -- 172.21.2.101:0/4107350205 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000678861", "target": ["mgr", ""]}) v1 -- 0x7fcc580fe1c0 con 0x7fcc3c0186c0 2021-02-10T11:36:27.359 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:36:27.353 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:36:27.359 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:36:27.353 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:36:27.359 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:36:27.353 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:36:27.360 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:36:27.353 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:36:27.360 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:36:27.353 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:36:27.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.384 7fcc4ffff700 1 -- 172.21.2.101:0/4107350205 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fcc480014c0 con 0x7fcc3c0186c0 2021-02-10T11:36:27.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.414 7fcc5de6c700 1 -- 172.21.2.101:0/4107350205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcc3c0186c0 msgr2=0x7fcc3c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:27.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.414 7fcc5de6c700 1 --2- 172.21.2.101:0/4107350205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcc3c0186c0 0x7fcc3c01abc0 crc :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:27.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.414 7fcc5de6c700 1 -- 172.21.2.101:0/4107350205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc580c3820 msgr2=0x7fcc580a7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:27.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.414 7fcc5de6c700 1 --2- 172.21.2.101:0/4107350205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc580c3820 0x7fcc580a7950 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 rx=0x7fcc4000c9c0 tx=0x7fcc4000cd60).stop 2021-02-10T11:36:27.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.414 7fcc5de6c700 1 -- 172.21.2.101:0/4107350205 shutdown_connections 2021-02-10T11:36:27.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.414 7fcc5de6c700 1 --2- 172.21.2.101:0/4107350205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcc3c0186c0 0x7fcc3c01abc0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:27.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.414 7fcc5de6c700 1 --2- 172.21.2.101:0/4107350205 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcc581a44c0 0x7fcc581a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:27.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.414 7fcc5de6c700 1 --2- 172.21.2.101:0/4107350205 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcc580863d0 0x7fcc580867d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:27.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.414 7fcc5de6c700 1 --2- 172.21.2.101:0/4107350205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcc580c3820 0x7fcc580a7950 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:27.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.414 7fcc5de6c700 1 -- 172.21.2.101:0/4107350205 >> 172.21.2.101:0/4107350205 conn(0x7fcc580872c0 msgr2=0x7fcc58095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:27.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.414 7fcc5de6c700 1 -- 172.21.2.101:0/4107350205 shutdown_connections 2021-02-10T11:36:27.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.414 7fcc5de6c700 1 -- 172.21.2.101:0/4107350205 wait complete. 2021-02-10T11:36:27.430 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000678861 2021-02-10T11:36:27.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.838 7f9f4a04d700 1 Processor -- start 2021-02-10T11:36:27.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.838 7f9f4a04d700 1 -- start start 2021-02-10T11:36:27.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.838 7f9f4a04d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f4410aff0 0x7f9f440cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:27.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.838 7f9f4a04d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f440fe1c0 con 0x7f9f440d1c70 2021-02-10T11:36:27.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.838 7f9f4a04d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f441b0cb0 con 0x7f9f440d1870 2021-02-10T11:36:27.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.838 7f9f4a04d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f44109a60 con 0x7f9f4410aff0 2021-02-10T11:36:27.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.838 7f9f42ffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f9f440d1c70 0x7f9f440cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54466/0 (socket says 172.21.2.101:54466) 2021-02-10T11:36:27.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.839 7f9f42ffd700 1 -- 172.21.2.101:0/201391595 learned_addr learned my addr 172.21.2.101:0/201391595 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:27.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.839 7f9f43fff700 1 --2- 172.21.2.101:0/201391595 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f4410aff0 0x7f9f440cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:27.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.839 7f9f427fc700 1 -- 172.21.2.101:0/201391595 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892171868 0 0) 0x7f9f2c0009b0 con 0x7f9f440d1c70 2021-02-10T11:36:27.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.839 7f9f427fc700 1 -- 172.21.2.101:0/201391595 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f30002fe0 con 0x7f9f440d1c70 2021-02-10T11:36:27.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.839 7f9f43fff700 1 -- 172.21.2.101:0/201391595 >> v1:172.21.2.108:6789/0 conn(0x7f9f440d1870 legacy=0x7f9f441b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:27.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.839 7f9f43fff700 1 -- 172.21.2.101:0/201391595 >> v1:172.21.2.101:6789/0 conn(0x7f9f440d1c70 legacy=0x7f9f440cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:27.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.839 7f9f43fff700 1 -- 172.21.2.101:0/201391595 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4410b3f0 con 0x7f9f4410aff0 2021-02-10T11:36:27.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.839 7f9f43fff700 1 --2- 172.21.2.101:0/201391595 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f4410aff0 0x7f9f440cf750 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f9f3c006e50 tx=0x7f9f3c009e70).ready entity=mon.2 client_cookie=bc90de04237f496a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:27.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.839 7f9f427fc700 1 -- 172.21.2.101:0/201391595 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9f3c00b510 con 0x7f9f4410aff0 2021-02-10T11:36:27.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.839 7f9f427fc700 1 -- 172.21.2.101:0/201391595 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3c006790 con 0x7f9f4410aff0 2021-02-10T11:36:27.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.839 7f9f427fc700 1 -- 172.21.2.101:0/201391595 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9f3c00ba90 con 0x7f9f4410aff0 2021-02-10T11:36:27.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.839 7f9f4a04d700 1 -- 172.21.2.101:0/201391595 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f4410aff0 msgr2=0x7f9f440cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:27.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.840 7f9f4a04d700 1 --2- 172.21.2.101:0/201391595 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f4410aff0 0x7f9f440cf750 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f9f3c006e50 tx=0x7f9f3c009e70).stop 2021-02-10T11:36:27.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.840 7f9f4a04d700 1 -- 172.21.2.101:0/201391595 shutdown_connections 2021-02-10T11:36:27.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.840 7f9f4a04d700 1 --2- 172.21.2.101:0/201391595 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f4410aff0 0x7f9f440cf750 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:27.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.840 7f9f4a04d700 1 -- 172.21.2.101:0/201391595 >> 172.21.2.101:0/201391595 conn(0x7f9f440ce420 msgr2=0x7f9f440d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:27.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.840 7f9f4a04d700 1 -- 172.21.2.101:0/201391595 shutdown_connections 2021-02-10T11:36:27.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.840 7f9f4a04d700 1 -- 172.21.2.101:0/201391595 wait complete. 2021-02-10T11:36:27.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.854 7f9f4a04d700 1 Processor -- start 2021-02-10T11:36:27.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.855 7f9f4a04d700 1 -- start start 2021-02-10T11:36:27.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.855 7f9f4a04d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f440c3820 0x7f9f44084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:27.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.855 7f9f4a04d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f440a0290 0x7f9f440852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:27.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.855 7f9f4a04d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f44086270 0x7f9f44086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:27.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.855 7f9f4a04d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f440c0700 con 0x7f9f440c3820 2021-02-10T11:36:27.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.855 7f9f4a04d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f440858f0 con 0x7f9f44086270 2021-02-10T11:36:27.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.855 7f9f4a04d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f44085b10 con 0x7f9f440a0290 2021-02-10T11:36:27.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.855 7f9f437fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f440c3820 0x7f9f44084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:27.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.855 7f9f437fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f440c3820 0x7f9f44084cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:53980/0 (socket says 172.21.2.101:53980) 2021-02-10T11:36:27.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.855 7f9f437fe700 1 -- 172.21.2.101:0/3111429225 learned_addr learned my addr 172.21.2.101:0/3111429225 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:27.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.855 7f9f437fe700 1 -- 172.21.2.101:0/3111429225 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f440a0290 msgr2=0x7f9f440852d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:27.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.855 7f9f437fe700 1 --2- 172.21.2.101:0/3111429225 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f440a0290 0x7f9f440852d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:27.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.855 7f9f437fe700 1 -- 172.21.2.101:0/3111429225 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f44086270 msgr2=0x7f9f44086670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:27.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.855 7f9f437fe700 1 --2- 172.21.2.101:0/3111429225 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f44086270 0x7f9f44086670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:27.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.855 7f9f437fe700 1 -- 172.21.2.101:0/3111429225 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f44086c90 con 0x7f9f440c3820 2021-02-10T11:36:27.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.856 7f9f437fe700 1 --2- 172.21.2.101:0/3111429225 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f440c3820 0x7f9f44084cb0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 rx=0x7f9f34006ed0 tx=0x7f9f3400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:27.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.856 7f9f3bfff700 1 -- 172.21.2.101:0/3111429225 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9f34006020 con 0x7f9f440c3820 2021-02-10T11:36:27.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.856 7f9f3bfff700 1 -- 172.21.2.101:0/3111429225 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f34006360 con 0x7f9f440c3820 2021-02-10T11:36:27.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.856 7f9f4a04d700 1 -- 172.21.2.101:0/3111429225 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f44086c90 con 0x7f9f440c3820 2021-02-10T11:36:27.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.856 7f9f4a04d700 1 -- 172.21.2.101:0/3111429225 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f441a2f00 con 0x7f9f440c3820 2021-02-10T11:36:27.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.856 7f9f3bfff700 1 -- 172.21.2.101:0/3111429225 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9f34005730 con 0x7f9f440c3820 2021-02-10T11:36:27.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.856 7f9f3bfff700 1 -- 172.21.2.101:0/3111429225 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9f340054b0 con 0x7f9f440c3820 2021-02-10T11:36:27.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.856 7f9f3bfff700 1 --2- 172.21.2.101:0/3111429225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f280186c0 0x7f9f2801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:27.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.856 7f9f3bfff700 1 -- 172.21.2.101:0/3111429225 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9f34032840 con 0x7f9f440c3820 2021-02-10T11:36:27.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.856 7f9f42ffd700 1 --2- 172.21.2.101:0/3111429225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f280186c0 0x7f9f2801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:27.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.857 7f9f42ffd700 1 --2- 172.21.2.101:0/3111429225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f280186c0 0x7f9f2801abc0 crc :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:27.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.888 7f9f4a04d700 1 -- 172.21.2.101:0/3111429225 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f44082a70 con 0x7f9f440c3820 2021-02-10T11:36:27.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:27.891 7f9f3bfff700 1 -- 172.21.2.101:0/3111429225 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9f34005890 con 0x7f9f440c3820 2021-02-10T11:36:28.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:28.018 7f9f4a04d700 1 -- 172.21.2.101:0/3111429225 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000678861", "target": ["mgr", ""]}) v1 -- 0x7f9f441a2f00 con 0x7f9f280186c0 2021-02-10T11:36:28.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:28.021 7f9f3bfff700 1 -- 172.21.2.101:0/3111429225 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f9f2c000dd0 con 0x7f9f280186c0 2021-02-10T11:36:28.028 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000678861/46fd4fcc-a1b5-4ba3-a4c9-3517eccae3f5 2021-02-10T11:36:28.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:28.022 7f9f4a04d700 1 -- 172.21.2.101:0/3111429225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f280186c0 msgr2=0x7f9f2801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:28.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:28.022 7f9f4a04d700 1 --2- 172.21.2.101:0/3111429225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f280186c0 0x7f9f2801abc0 crc :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:28.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:28.022 7f9f4a04d700 1 -- 172.21.2.101:0/3111429225 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f440c3820 msgr2=0x7f9f44084cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:28.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:28.022 7f9f4a04d700 1 --2- 172.21.2.101:0/3111429225 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f440c3820 0x7f9f44084cb0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 rx=0x7f9f34006ed0 tx=0x7f9f3400b960).stop 2021-02-10T11:36:28.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:28.023 7f9f4a04d700 1 -- 172.21.2.101:0/3111429225 shutdown_connections 2021-02-10T11:36:28.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:28.023 7f9f4a04d700 1 --2- 172.21.2.101:0/3111429225 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f280186c0 0x7f9f2801abc0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:28.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:28.023 7f9f4a04d700 1 --2- 172.21.2.101:0/3111429225 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f440a0290 0x7f9f440852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:28.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:28.023 7f9f4a04d700 1 --2- 172.21.2.101:0/3111429225 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f44086270 0x7f9f44086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:28.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:28.023 7f9f4a04d700 1 --2- 172.21.2.101:0/3111429225 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f440c3820 0x7f9f44084cb0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:28.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:28.023 7f9f4a04d700 1 -- 172.21.2.101:0/3111429225 >> 172.21.2.101:0/3111429225 conn(0x7f9f440872c0 msgr2=0x7f9f44095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:28.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:28.023 7f9f4a04d700 1 -- 172.21.2.101:0/3111429225 shutdown_connections 2021-02-10T11:36:28.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:28.023 7f9f4a04d700 1 -- 172.21.2.101:0/3111429225 wait complete. 2021-02-10T11:36:28.038 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000678861 with 1 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000678861/46fd4fcc-a1b5-4ba3-a4c9-3517eccae3f5 2021-02-10T11:36:28.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:28.038 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:36:28.038 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:36:28.373 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:36:28.373 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:36:28.373 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000678861/46fd4fcc-a1b5-4ba3-a4c9-3517eccae3f5/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:36:28.757 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:36:28.758 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:36:28.758 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0277811 s, 37.7 MB/s 2021-02-10T11:36:28.759 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume snapshot create cephfs subvolume_0000000000678861 snapshot_0000000000465306 2021-02-10T11:36:29.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.167 7f3ce6cb6700 1 Processor -- start 2021-02-10T11:36:29.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.167 7f3ce6cb6700 1 -- start start 2021-02-10T11:36:29.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.167 7f3ce6cb6700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3ce00d1880 0x7f3ce01b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:29.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.167 7f3ce6cb6700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce00fe1d0 con 0x7f3ce01b0af0 2021-02-10T11:36:29.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.167 7f3ce6cb6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce0109a70 con 0x7f3ce00cff80 2021-02-10T11:36:29.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.167 7f3ce6cb6700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce0088070 con 0x7f3ce00d1880 2021-02-10T11:36:29.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.168 7f3cdffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f3ce01b0af0 0x7f3ce0108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54482/0 (socket says 172.21.2.101:54482) 2021-02-10T11:36:29.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.168 7f3cdffff700 1 -- 172.21.2.101:0/192614091 learned_addr learned my addr 172.21.2.101:0/192614091 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:29.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.168 7f3cdf7fe700 1 -- 172.21.2.101:0/192614091 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3163340153 0 0) 0x7f3cd0001060 con 0x7f3ce01b0af0 2021-02-10T11:36:29.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.168 7f3cdf7fe700 1 -- 172.21.2.101:0/192614091 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ccc002fe0 con 0x7f3ce01b0af0 2021-02-10T11:36:29.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.168 7f3cdf7fe700 1 -- 172.21.2.101:0/192614091 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2068911748 0 0) 0x7f3cd0001460 con 0x7f3ce01b0af0 2021-02-10T11:36:29.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.168 7f3cdf7fe700 1 -- 172.21.2.101:0/192614091 >> v2:172.21.2.108:3300/0 conn(0x7f3ce00d1880 msgr2=0x7f3ce01b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:29.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.168 7f3cdf7fe700 1 --2- 172.21.2.101:0/192614091 >> v2:172.21.2.108:3300/0 conn(0x7f3ce00d1880 0x7f3ce01b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:29.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.168 7f3cdf7fe700 1 -- 172.21.2.101:0/192614091 >> v1:172.21.2.108:6789/0 conn(0x7f3ce00cff80 legacy=0x7f3ce0109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:29.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.168 7f3cdf7fe700 1 -- 172.21.2.101:0/192614091 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce0088260 con 0x7f3ce01b0af0 2021-02-10T11:36:29.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.168 7f3cdf7fe700 1 -- 172.21.2.101:0/192614091 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f3cd00019e0 con 0x7f3ce01b0af0 2021-02-10T11:36:29.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.168 7f3cdf7fe700 1 -- 172.21.2.101:0/192614091 >> v1:172.21.2.101:6789/0 conn(0x7f3ce01b0af0 legacy=0x7f3ce0108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:29.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.168 7f3cdf7fe700 1 --2- 172.21.2.101:0/192614091 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ccc003b10 0x7f3ccc005f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:29.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.168 7f3cdf7fe700 1 --2- 172.21.2.101:0/192614091 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ccc0067a0 0x7f3ccc008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:29.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.168 7f3cdf7fe700 1 --2- 172.21.2.101:0/192614091 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ccc009580 0x7f3ccc00baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:29.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.168 7f3cdf7fe700 1 -- 172.21.2.101:0/192614091 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc00c2f0 con 0x7f3ccc003b10 2021-02-10T11:36:29.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.168 7f3cdf7fe700 1 -- 172.21.2.101:0/192614091 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc00c510 con 0x7f3ccc0067a0 2021-02-10T11:36:29.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.168 7f3cdf7fe700 1 -- 172.21.2.101:0/192614091 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc00c730 con 0x7f3ccc009580 2021-02-10T11:36:29.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.168 7f3ce4ad2700 1 --2- 172.21.2.101:0/192614091 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ccc003b10 0x7f3ccc005f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:29.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.169 7f3ce52d3700 1 --2- 172.21.2.101:0/192614091 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ccc009580 0x7f3ccc00baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:29.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.169 7f3cdffff700 1 --2- 172.21.2.101:0/192614091 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ccc0067a0 0x7f3ccc008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:29.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.169 7f3ce4ad2700 1 -- 172.21.2.101:0/192614091 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ccc009580 msgr2=0x7f3ccc00baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:29.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.169 7f3ce4ad2700 1 --2- 172.21.2.101:0/192614091 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ccc009580 0x7f3ccc00baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:29.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.169 7f3ce4ad2700 1 -- 172.21.2.101:0/192614091 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ccc0067a0 msgr2=0x7f3ccc008c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:29.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.169 7f3ce4ad2700 1 --2- 172.21.2.101:0/192614091 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ccc0067a0 0x7f3ccc008c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:29.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.169 7f3ce4ad2700 1 -- 172.21.2.101:0/192614091 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ccc00c9d0 con 0x7f3ccc003b10 2021-02-10T11:36:29.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.169 7f3cdffff700 1 --2- 172.21.2.101:0/192614091 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ccc0067a0 0x7f3ccc008c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:36:29.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.169 7f3ce52d3700 1 -- 172.21.2.101:0/192614091 reap_dead start 2021-02-10T11:36:29.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.169 7f3ce4ad2700 1 --2- 172.21.2.101:0/192614091 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ccc003b10 0x7f3ccc005f20 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 rx=0x7f3cc8006bf0 tx=0x7f3cc804fcd0).ready entity=mon.0 client_cookie=8e3a71c6051f92ab server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:29.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.169 7f3cdf7fe700 1 -- 172.21.2.101:0/192614091 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3cc80009b0 con 0x7f3ccc003b10 2021-02-10T11:36:29.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.169 7f3cdf7fe700 1 -- 172.21.2.101:0/192614091 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc8002030 con 0x7f3ccc003b10 2021-02-10T11:36:29.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.169 7f3cdf7fe700 1 -- 172.21.2.101:0/192614091 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3cc8002510 con 0x7f3ccc003b10 2021-02-10T11:36:29.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.169 7f3ce6cb6700 1 -- 172.21.2.101:0/192614091 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ccc003b10 msgr2=0x7f3ccc005f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:29.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.169 7f3ce6cb6700 1 --2- 172.21.2.101:0/192614091 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ccc003b10 0x7f3ccc005f20 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 rx=0x7f3cc8006bf0 tx=0x7f3cc804fcd0).stop 2021-02-10T11:36:29.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.170 7f3ce6cb6700 1 -- 172.21.2.101:0/192614091 shutdown_connections 2021-02-10T11:36:29.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.170 7f3ce6cb6700 1 --2- 172.21.2.101:0/192614091 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ccc003b10 0x7f3ccc005f20 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:29.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.170 7f3ce6cb6700 1 -- 172.21.2.101:0/192614091 >> 172.21.2.101:0/192614091 conn(0x7f3ce00ce430 msgr2=0x7f3ce00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:29.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.170 7f3ce6cb6700 1 -- 172.21.2.101:0/192614091 shutdown_connections 2021-02-10T11:36:29.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.170 7f3ce6cb6700 1 -- 172.21.2.101:0/192614091 wait complete. 2021-02-10T11:36:29.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.185 7f3ce6cb6700 1 Processor -- start 2021-02-10T11:36:29.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.185 7f3ce6cb6700 1 -- start start 2021-02-10T11:36:29.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.185 7f3ce6cb6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ce01a3570 0x7f3ce01a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:29.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.185 7f3ce6cb6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ce009c9f0 0x7f3ce009cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:29.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.185 7f3ce6cb6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ce009d410 0x7f3ce0098170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:29.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.185 7f3ce6cb6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce009c470 con 0x7f3ce009d410 2021-02-10T11:36:29.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.185 7f3ce6cb6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce00b2840 con 0x7f3ce009c9f0 2021-02-10T11:36:29.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.185 7f3ce6cb6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce00b2a20 con 0x7f3ce01a3570 2021-02-10T11:36:29.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.186 7f3ce52d3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ce009d410 0x7f3ce0098170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:29.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.186 7f3ce52d3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ce009d410 0x7f3ce0098170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54002/0 (socket says 172.21.2.101:54002) 2021-02-10T11:36:29.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.186 7f3ce52d3700 1 -- 172.21.2.101:0/3674095285 learned_addr learned my addr 172.21.2.101:0/3674095285 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:29.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.186 7f3ce4ad2700 1 --2- 172.21.2.101:0/3674095285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ce01a3570 0x7f3ce01a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:29.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.186 7f3cdffff700 1 --2- 172.21.2.101:0/3674095285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ce009c9f0 0x7f3ce009cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:29.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.186 7f3ce52d3700 1 -- 172.21.2.101:0/3674095285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ce01a3570 msgr2=0x7f3ce01a3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:29.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.186 7f3ce52d3700 1 --2- 172.21.2.101:0/3674095285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ce01a3570 0x7f3ce01a3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:29.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.186 7f3ce52d3700 1 -- 172.21.2.101:0/3674095285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ce009c9f0 msgr2=0x7f3ce009cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:29.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.186 7f3ce52d3700 1 --2- 172.21.2.101:0/3674095285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ce009c9f0 0x7f3ce009cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:29.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.186 7f3ce52d3700 1 -- 172.21.2.101:0/3674095285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce009d880 con 0x7f3ce009d410 2021-02-10T11:36:29.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.186 7f3cdffff700 1 --2- 172.21.2.101:0/3674095285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ce009c9f0 0x7f3ce009cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:36:29.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.186 7f3ce52d3700 1 --2- 172.21.2.101:0/3674095285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ce009d410 0x7f3ce0098170 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 rx=0x7f3cd800b590 tx=0x7f3cd800b930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:29.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.186 7f3cdd7fa700 1 -- 172.21.2.101:0/3674095285 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3cd8006240 con 0x7f3ce009d410 2021-02-10T11:36:29.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.186 7f3cdd7fa700 1 -- 172.21.2.101:0/3674095285 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd8006bb0 con 0x7f3ce009d410 2021-02-10T11:36:29.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.186 7f3cdd7fa700 1 -- 172.21.2.101:0/3674095285 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3cd8005330 con 0x7f3ce009d410 2021-02-10T11:36:29.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.187 7f3ce6cb6700 1 -- 172.21.2.101:0/3674095285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce009d880 con 0x7f3ce009d410 2021-02-10T11:36:29.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.187 7f3ce6cb6700 1 -- 172.21.2.101:0/3674095285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce0098a80 con 0x7f3ce009d410 2021-02-10T11:36:29.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.187 7f3cdd7fa700 1 -- 172.21.2.101:0/3674095285 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3cd8005330 con 0x7f3ce009d410 2021-02-10T11:36:29.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.187 7f3cdd7fa700 1 --2- 172.21.2.101:0/3674095285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cc40186c0 0x7f3cc401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:29.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.187 7f3cdd7fa700 1 -- 172.21.2.101:0/3674095285 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3cd8032800 con 0x7f3ce009d410 2021-02-10T11:36:29.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.187 7f3ce4ad2700 1 --2- 172.21.2.101:0/3674095285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cc40186c0 0x7f3cc401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:29.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.188 7f3ce4ad2700 1 --2- 172.21.2.101:0/3674095285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cc40186c0 0x7f3cc401abc0 crc :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:29.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.217 7f3ce6cb6700 1 -- 172.21.2.101:0/3674095285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce0082a70 con 0x7f3ce009d410 2021-02-10T11:36:29.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.220 7f3cdd7fa700 1 -- 172.21.2.101:0/3674095285 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3cd8006030 con 0x7f3ce009d410 2021-02-10T11:36:29.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.349 7f3ce6cb6700 1 -- 172.21.2.101:0/3674095285 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000678861", "target": ["mgr", ""], "snap_name": "snapshot_0000000000465306"}) v1 -- 0x7f3ce0098a80 con 0x7f3cc40186c0 2021-02-10T11:36:29.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.359 7f3cdd7fa700 1 -- 172.21.2.101:0/3674095285 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f3cc80009b0 con 0x7f3cc40186c0 2021-02-10T11:36:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.365 7f3ce6cb6700 1 -- 172.21.2.101:0/3674095285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cc40186c0 msgr2=0x7f3cc401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.365 7f3ce6cb6700 1 --2- 172.21.2.101:0/3674095285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cc40186c0 0x7f3cc401abc0 crc :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.365 7f3ce6cb6700 1 -- 172.21.2.101:0/3674095285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ce009d410 msgr2=0x7f3ce0098170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.365 7f3ce6cb6700 1 --2- 172.21.2.101:0/3674095285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ce009d410 0x7f3ce0098170 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 rx=0x7f3cd800b590 tx=0x7f3cd800b930).stop 2021-02-10T11:36:29.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.366 7f3ce6cb6700 1 -- 172.21.2.101:0/3674095285 shutdown_connections 2021-02-10T11:36:29.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.366 7f3ce6cb6700 1 --2- 172.21.2.101:0/3674095285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ce009d410 0x7f3ce0098170 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:29.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.366 7f3ce6cb6700 1 --2- 172.21.2.101:0/3674095285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ce009c9f0 0x7f3ce009cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:29.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.366 7f3ce6cb6700 1 --2- 172.21.2.101:0/3674095285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cc40186c0 0x7f3cc401abc0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:29.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.366 7f3ce6cb6700 1 --2- 172.21.2.101:0/3674095285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ce01a3570 0x7f3ce01a3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:29.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.366 7f3ce6cb6700 1 -- 172.21.2.101:0/3674095285 >> 172.21.2.101:0/3674095285 conn(0x7f3ce00c2cd0 msgr2=0x7f3ce00b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:29.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.366 7f3ce6cb6700 1 -- 172.21.2.101:0/3674095285 shutdown_connections 2021-02-10T11:36:29.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.366 7f3ce6cb6700 1 -- 172.21.2.101:0/3674095285 wait complete. 2021-02-10T11:36:29.380 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume snapshot clone cephfs subvolume_0000000000678861 snapshot_0000000000465306 clone_0000000000005193 2021-02-10T11:36:29.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.790 7f9521f31700 1 Processor -- start 2021-02-10T11:36:29.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.790 7f9521f31700 1 -- start start 2021-02-10T11:36:29.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.790 7f9521f31700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f951c0dce80 0x7f951c0d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:29.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.790 7f9521f31700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f951c10a550 con 0x7f951c10b2c0 2021-02-10T11:36:29.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.790 7f9521f31700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f951c088240 con 0x7f951c0eb220 2021-02-10T11:36:29.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.790 7f9521f31700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f951c0d1fe0 con 0x7f951c0dce80 2021-02-10T11:36:29.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.790 7f951b7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f951c0dce80 0x7f951c0d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:29.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.790 7f951b7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f951c0dce80 0x7f951c0d14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54008/0 (socket says 172.21.2.101:54008) 2021-02-10T11:36:29.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.790 7f951b7fe700 1 -- 172.21.2.101:0/405025601 learned_addr learned my addr 172.21.2.101:0/405025601 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:29.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.791 7f951b7fe700 1 -- 172.21.2.101:0/405025601 >> v1:172.21.2.108:6789/0 conn(0x7f951c0eb220 legacy=0x7f951c10aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:29.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.791 7f951b7fe700 1 -- 172.21.2.101:0/405025601 >> v1:172.21.2.101:6789/0 conn(0x7f951c10b2c0 legacy=0x7f951c1b8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:29.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.791 7f951b7fe700 1 -- 172.21.2.101:0/405025601 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c0dd280 con 0x7f951c0dce80 2021-02-10T11:36:29.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.791 7f951b7fe700 1 --2- 172.21.2.101:0/405025601 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f951c0dce80 0x7f951c0d14f0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 rx=0x7f950c006700 tx=0x7f950c009b00).ready entity=mon.0 client_cookie=c414b1c5c4f6cd35 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:29.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.791 7f951a7fc700 1 -- 172.21.2.101:0/405025601 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f950c00c0c0 con 0x7f951c0dce80 2021-02-10T11:36:29.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.791 7f951a7fc700 1 -- 172.21.2.101:0/405025601 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950c00c340 con 0x7f951c0dce80 2021-02-10T11:36:29.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.792 7f951a7fc700 1 -- 172.21.2.101:0/405025601 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f950c00c820 con 0x7f951c0dce80 2021-02-10T11:36:29.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.792 7f9521f31700 1 -- 172.21.2.101:0/405025601 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f951c0dce80 msgr2=0x7f951c0d14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:29.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.792 7f9521f31700 1 --2- 172.21.2.101:0/405025601 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f951c0dce80 0x7f951c0d14f0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 rx=0x7f950c006700 tx=0x7f950c009b00).stop 2021-02-10T11:36:29.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.792 7f9521f31700 1 -- 172.21.2.101:0/405025601 shutdown_connections 2021-02-10T11:36:29.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.792 7f9521f31700 1 --2- 172.21.2.101:0/405025601 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f951c0dce80 0x7f951c0d14f0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:29.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.792 7f9521f31700 1 -- 172.21.2.101:0/405025601 >> 172.21.2.101:0/405025601 conn(0x7f951c0ea630 msgr2=0x7f951c0d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:29.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.792 7f9521f31700 1 -- 172.21.2.101:0/405025601 shutdown_connections 2021-02-10T11:36:29.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.792 7f9521f31700 1 -- 172.21.2.101:0/405025601 wait complete. 2021-02-10T11:36:29.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.807 7f9521f31700 1 Processor -- start 2021-02-10T11:36:29.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.807 7f9521f31700 1 -- start start 2021-02-10T11:36:29.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.807 7f9521f31700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f951c094ae0 0x7f951c1c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:29.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.807 7f9521f31700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f951c1c6a50 0x7f951c0f87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:29.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.807 7f9521f31700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f951c0f8dd0 0x7f951c107230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:29.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.807 7f9521f31700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c0f92a0 con 0x7f951c0f8dd0 2021-02-10T11:36:29.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.807 7f9521f31700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c0b75f0 con 0x7f951c1c6a50 2021-02-10T11:36:29.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.807 7f9521f31700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951c094ee0 con 0x7f951c094ae0 2021-02-10T11:36:29.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.807 7f951bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f951c0f8dd0 0x7f951c107230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:29.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.807 7f951bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f951c0f8dd0 0x7f951c107230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54016/0 (socket says 172.21.2.101:54016) 2021-02-10T11:36:29.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.808 7f951bfff700 1 -- 172.21.2.101:0/2291863703 learned_addr learned my addr 172.21.2.101:0/2291863703 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:29.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.808 7f951bfff700 1 -- 172.21.2.101:0/2291863703 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f951c094ae0 msgr2=0x7f951c1c6430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:29.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.808 7f951bfff700 1 --2- 172.21.2.101:0/2291863703 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f951c094ae0 0x7f951c1c6430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:29.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.808 7f951bfff700 1 -- 172.21.2.101:0/2291863703 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f951c1c6a50 msgr2=0x7f951c0f87b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:29.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.808 7f951bfff700 1 --2- 172.21.2.101:0/2291863703 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f951c1c6a50 0x7f951c0f87b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:29.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.808 7f951bfff700 1 -- 172.21.2.101:0/2291863703 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c1b7750 con 0x7f951c0f8dd0 2021-02-10T11:36:29.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.808 7f951bfff700 1 --2- 172.21.2.101:0/2291863703 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f951c0f8dd0 0x7f951c107230 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 rx=0x7f9514009980 tx=0x7f951400bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:29.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.808 7f9503fff700 1 -- 172.21.2.101:0/2291863703 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9514005af0 con 0x7f951c0f8dd0 2021-02-10T11:36:29.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.808 7f9503fff700 1 -- 172.21.2.101:0/2291863703 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9514005d70 con 0x7f951c0f8dd0 2021-02-10T11:36:29.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.808 7f9521f31700 1 -- 172.21.2.101:0/2291863703 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951c1b7750 con 0x7f951c0f8dd0 2021-02-10T11:36:29.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.808 7f9521f31700 1 -- 172.21.2.101:0/2291863703 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951c1b7750 con 0x7f951c0f8dd0 2021-02-10T11:36:29.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.808 7f9503fff700 1 -- 172.21.2.101:0/2291863703 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9514006220 con 0x7f951c0f8dd0 2021-02-10T11:36:29.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.809 7f9503fff700 1 -- 172.21.2.101:0/2291863703 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9514005870 con 0x7f951c0f8dd0 2021-02-10T11:36:29.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.809 7f9503fff700 1 --2- 172.21.2.101:0/2291863703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94fc0186c0 0x7f94fc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:29.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.809 7f9503fff700 1 -- 172.21.2.101:0/2291863703 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f95140327a0 con 0x7f951c0f8dd0 2021-02-10T11:36:29.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.809 7f951b7fe700 1 --2- 172.21.2.101:0/2291863703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94fc0186c0 0x7f94fc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:29.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.809 7f951b7fe700 1 --2- 172.21.2.101:0/2291863703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94fc0186c0 0x7f94fc01abc0 crc :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:29.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.840 7f9521f31700 1 -- 172.21.2.101:0/2291863703 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f951c0f7a70 con 0x7f951c0f8dd0 2021-02-10T11:36:29.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.843 7f9503fff700 1 -- 172.21.2.101:0/2291863703 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f951400be10 con 0x7f951c0f8dd0 2021-02-10T11:36:29.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:29.972 7f9521f31700 1 -- 172.21.2.101:0/2291863703 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvolume_0000000000678861", "target": ["mgr", ""], "snap_name": "snapshot_0000000000465306", "target_sub_name": "clone_0000000000005193"}) v1 -- 0x7f951c0f7a70 con 0x7f94fc0186c0 2021-02-10T11:36:30.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.014 7f9503fff700 1 -- 172.21.2.101:0/2291863703 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f950c009390 con 0x7f94fc0186c0 2021-02-10T11:36:30.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.037 7f9521f31700 1 -- 172.21.2.101:0/2291863703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94fc0186c0 msgr2=0x7f94fc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:30.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.037 7f9521f31700 1 --2- 172.21.2.101:0/2291863703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94fc0186c0 0x7f94fc01abc0 crc :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:30.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.037 7f9521f31700 1 -- 172.21.2.101:0/2291863703 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f951c0f8dd0 msgr2=0x7f951c107230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:30.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.037 7f9521f31700 1 --2- 172.21.2.101:0/2291863703 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f951c0f8dd0 0x7f951c107230 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 rx=0x7f9514009980 tx=0x7f951400bcd0).stop 2021-02-10T11:36:30.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.037 7f9521f31700 1 -- 172.21.2.101:0/2291863703 shutdown_connections 2021-02-10T11:36:30.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.037 7f9521f31700 1 --2- 172.21.2.101:0/2291863703 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f951c0f8dd0 0x7f951c107230 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:30.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.037 7f9521f31700 1 --2- 172.21.2.101:0/2291863703 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f951c1c6a50 0x7f951c0f87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:30.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.037 7f9521f31700 1 --2- 172.21.2.101:0/2291863703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94fc0186c0 0x7f94fc01abc0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:30.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.037 7f9521f31700 1 --2- 172.21.2.101:0/2291863703 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f951c094ae0 0x7f951c1c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:30.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.037 7f9521f31700 1 -- 172.21.2.101:0/2291863703 >> 172.21.2.101:0/2291863703 conn(0x7f951c095270 msgr2=0x7f951c0fb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:30.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.037 7f9521f31700 1 -- 172.21.2.101:0/2291863703 shutdown_connections 2021-02-10T11:36:30.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.037 7f9521f31700 1 -- 172.21.2.101:0/2291863703 wait complete. 2021-02-10T11:36:30.053 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs clone status cephfs clone_0000000000005193 2021-02-10T11:36:30.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.461 7f0c9d773700 1 Processor -- start 2021-02-10T11:36:30.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.461 7f0c9d773700 1 -- start start 2021-02-10T11:36:30.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.461 7f0c9d773700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0c980d1c70 0x7f0c980cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:30.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.462 7f0c9d773700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c980cf580 0x7f0c980cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:30.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.462 7f0c9d773700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c980fe1c0 con 0x7f0c980d1870 2021-02-10T11:36:30.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.462 7f0c9d773700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c981b0cb0 con 0x7f0c980d1c70 2021-02-10T11:36:30.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.462 7f0c9d773700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c98109a60 con 0x7f0c980cf580 2021-02-10T11:36:30.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.462 7f0c96d73700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0c980d1c70 0x7f0c980cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:30.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.462 7f0c97574700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f0c980d1870 0x7f0c981b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54522/0 (socket says 172.21.2.101:54522) 2021-02-10T11:36:30.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.462 7f0c97574700 1 -- 172.21.2.101:0/2748510494 learned_addr learned my addr 172.21.2.101:0/2748510494 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:30.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.462 7f0c97d75700 1 --2- 172.21.2.101:0/2748510494 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c980cf580 0x7f0c980cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:30.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.462 7f0c96d73700 1 -- 172.21.2.101:0/2748510494 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c980cf580 msgr2=0x7f0c980cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:30.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.462 7f0c96d73700 1 --2- 172.21.2.101:0/2748510494 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c980cf580 0x7f0c980cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:30.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.462 7f0c96d73700 1 -- 172.21.2.101:0/2748510494 >> v1:172.21.2.101:6789/0 conn(0x7f0c980d1870 legacy=0x7f0c981b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:30.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.462 7f0c96d73700 1 -- 172.21.2.101:0/2748510494 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c98088070 con 0x7f0c980d1c70 2021-02-10T11:36:30.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.462 7f0c96d73700 1 --2- 172.21.2.101:0/2748510494 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c980d1c70 0x7f0c980cef60 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 rx=0x7f0c900069d0 tx=0x7f0c90009e40).ready entity=mon.0 client_cookie=58be3633fcb9a8a8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:30.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.462 7f0c96572700 1 -- 172.21.2.101:0/2748510494 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0c9000b510 con 0x7f0c980d1c70 2021-02-10T11:36:30.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.462 7f0c96572700 1 -- 172.21.2.101:0/2748510494 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9000b890 con 0x7f0c980d1c70 2021-02-10T11:36:30.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.462 7f0c96572700 1 -- 172.21.2.101:0/2748510494 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0c9000bcf0 con 0x7f0c980d1c70 2021-02-10T11:36:30.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.462 7f0c9d773700 1 -- 172.21.2.101:0/2748510494 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c980d1c70 msgr2=0x7f0c980cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:30.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.462 7f0c9d773700 1 --2- 172.21.2.101:0/2748510494 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c980d1c70 0x7f0c980cef60 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 rx=0x7f0c900069d0 tx=0x7f0c90009e40).stop 2021-02-10T11:36:30.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.463 7f0c9d773700 1 -- 172.21.2.101:0/2748510494 shutdown_connections 2021-02-10T11:36:30.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.463 7f0c9d773700 1 --2- 172.21.2.101:0/2748510494 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c980cf580 0x7f0c980cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:30.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.463 7f0c9d773700 1 --2- 172.21.2.101:0/2748510494 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c980d1c70 0x7f0c980cef60 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:30.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.463 7f0c9d773700 1 -- 172.21.2.101:0/2748510494 >> 172.21.2.101:0/2748510494 conn(0x7f0c980ce420 msgr2=0x7f0c980d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:30.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.463 7f0c9d773700 1 -- 172.21.2.101:0/2748510494 shutdown_connections 2021-02-10T11:36:30.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.463 7f0c9d773700 1 -- 172.21.2.101:0/2748510494 wait complete. 2021-02-10T11:36:30.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.477 7f0c9d773700 1 Processor -- start 2021-02-10T11:36:30.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.477 7f0c9d773700 1 -- start start 2021-02-10T11:36:30.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.478 7f0c9d773700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c9809bcc0 0x7f0c9809c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:30.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.478 7f0c9d773700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c980a0820 0x7f0c980a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:30.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.478 7f0c9d773700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c980a1240 0x7f0c980858d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:30.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.478 7f0c9d773700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c980a66b0 con 0x7f0c9809bcc0 2021-02-10T11:36:30.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.478 7f0c9d773700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c981a49b0 con 0x7f0c980a1240 2021-02-10T11:36:30.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.478 7f0c9d773700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c981a4b90 con 0x7f0c980a0820 2021-02-10T11:36:30.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.478 7f0c97574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c9809bcc0 0x7f0c9809c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:30.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.478 7f0c97574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c9809bcc0 0x7f0c9809c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54036/0 (socket says 172.21.2.101:54036) 2021-02-10T11:36:30.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.478 7f0c97574700 1 -- 172.21.2.101:0/709651791 learned_addr learned my addr 172.21.2.101:0/709651791 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:30.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.478 7f0c97d75700 1 --2- 172.21.2.101:0/709651791 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c980a1240 0x7f0c980858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:30.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.478 7f0c96d73700 1 --2- 172.21.2.101:0/709651791 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c980a0820 0x7f0c980a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:30.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.478 7f0c97574700 1 -- 172.21.2.101:0/709651791 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c980a0820 msgr2=0x7f0c980a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:30.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.478 7f0c97574700 1 --2- 172.21.2.101:0/709651791 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c980a0820 0x7f0c980a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:30.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.478 7f0c97574700 1 -- 172.21.2.101:0/709651791 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c980a1240 msgr2=0x7f0c980858d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:30.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.478 7f0c97574700 1 --2- 172.21.2.101:0/709651791 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c980a1240 0x7f0c980858d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:30.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.478 7f0c97574700 1 -- 172.21.2.101:0/709651791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c980a1640 con 0x7f0c9809bcc0 2021-02-10T11:36:30.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.478 7f0c96d73700 1 --2- 172.21.2.101:0/709651791 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c980a0820 0x7f0c980a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:36:30.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.479 7f0c97574700 1 --2- 172.21.2.101:0/709651791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c9809bcc0 0x7f0c9809c0c0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 rx=0x7f0c8c001410 tx=0x7f0c8c00a950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:30.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.479 7f0c7ffff700 1 -- 172.21.2.101:0/709651791 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0c8c00ad40 con 0x7f0c9809bcc0 2021-02-10T11:36:30.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.479 7f0c7ffff700 1 -- 172.21.2.101:0/709651791 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8c000c90 con 0x7f0c9809bcc0 2021-02-10T11:36:30.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.479 7f0c9d773700 1 -- 172.21.2.101:0/709651791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c980a1640 con 0x7f0c9809bcc0 2021-02-10T11:36:30.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.479 7f0c7ffff700 1 -- 172.21.2.101:0/709651791 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0c8c004590 con 0x7f0c9809bcc0 2021-02-10T11:36:30.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.479 7f0c9d773700 1 -- 172.21.2.101:0/709651791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c980861e0 con 0x7f0c9809bcc0 2021-02-10T11:36:30.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.479 7f0c7ffff700 1 -- 172.21.2.101:0/709651791 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0c8c0043a0 con 0x7f0c9809bcc0 2021-02-10T11:36:30.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.479 7f0c7ffff700 1 --2- 172.21.2.101:0/709651791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c780186c0 0x7f0c7801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:30.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.479 7f0c7ffff700 1 -- 172.21.2.101:0/709651791 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0c8c0325e0 con 0x7f0c9809bcc0 2021-02-10T11:36:30.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.480 7f0c96d73700 1 --2- 172.21.2.101:0/709651791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c780186c0 0x7f0c7801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:30.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.480 7f0c96d73700 1 --2- 172.21.2.101:0/709651791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c780186c0 0x7f0c7801abc0 crc :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:30.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.511 7f0c9d773700 1 -- 172.21.2.101:0/709651791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c98086a70 con 0x7f0c9809bcc0 2021-02-10T11:36:30.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.514 7f0c7ffff700 1 -- 172.21.2.101:0/709651791 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0c8c000ed0 con 0x7f0c9809bcc0 2021-02-10T11:36:30.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.646 7f0c9d773700 1 -- 172.21.2.101:0/709651791 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "clone_0000000000005193", "target": ["mgr", ""]}) v1 -- 0x7f0c980861e0 con 0x7f0c780186c0 2021-02-10T11:36:30.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.649 7f0c7ffff700 1 -- 172.21.2.101:0/709651791 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+46 (crc 0 0 0) 0x7f0c90000d10 con 0x7f0c780186c0 2021-02-10T11:36:30.656 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:36:30.656 INFO:teuthology.orchestra.run.gibba001.stdout: "status": { 2021-02-10T11:36:30.656 INFO:teuthology.orchestra.run.gibba001.stdout: "state": "complete" 2021-02-10T11:36:30.656 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:30.656 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:36:30.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.651 7f0c9d773700 1 -- 172.21.2.101:0/709651791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c780186c0 msgr2=0x7f0c7801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:30.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.651 7f0c9d773700 1 --2- 172.21.2.101:0/709651791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c780186c0 0x7f0c7801abc0 crc :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:30.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.651 7f0c9d773700 1 -- 172.21.2.101:0/709651791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c9809bcc0 msgr2=0x7f0c9809c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:30.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.651 7f0c9d773700 1 --2- 172.21.2.101:0/709651791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c9809bcc0 0x7f0c9809c0c0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 rx=0x7f0c8c001410 tx=0x7f0c8c00a950).stop 2021-02-10T11:36:30.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.651 7f0c9d773700 1 -- 172.21.2.101:0/709651791 shutdown_connections 2021-02-10T11:36:30.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.651 7f0c9d773700 1 --2- 172.21.2.101:0/709651791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0c780186c0 0x7f0c7801abc0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:30.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.651 7f0c9d773700 1 --2- 172.21.2.101:0/709651791 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0c980a0820 0x7f0c980a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:30.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.651 7f0c9d773700 1 --2- 172.21.2.101:0/709651791 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0c980a1240 0x7f0c980858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:30.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.651 7f0c9d773700 1 --2- 172.21.2.101:0/709651791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0c9809bcc0 0x7f0c9809c0c0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:30.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.651 7f0c9d773700 1 -- 172.21.2.101:0/709651791 >> 172.21.2.101:0/709651791 conn(0x7f0c980bef00 msgr2=0x7f0c9809d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:30.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.651 7f0c9d773700 1 -- 172.21.2.101:0/709651791 shutdown_connections 2021-02-10T11:36:30.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:30.651 7f0c9d773700 1 -- 172.21.2.101:0/709651791 wait complete. 2021-02-10T11:36:30.666 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume snapshot rm cephfs subvolume_0000000000678861 snapshot_0000000000465306 2021-02-10T11:36:30.878 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:36:30.881 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:36:31.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.073 7f8d6d7ba700 1 Processor -- start 2021-02-10T11:36:31.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.073 7f8d6d7ba700 1 -- start start 2021-02-10T11:36:31.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.073 7f8d6d7ba700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8d680d1c80 0x7f8d680cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:31.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.073 7f8d6d7ba700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d680cf590 0x7f8d680cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:31.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.073 7f8d6d7ba700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d680fe1d0 con 0x7f8d680d1880 2021-02-10T11:36:31.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.073 7f8d6d7ba700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d681b0cc0 con 0x7f8d680d1c80 2021-02-10T11:36:31.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.073 7f8d6d7ba700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d68109a70 con 0x7f8d680cf590 2021-02-10T11:36:31.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.074 7f8d67d75700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d680cf590 0x7f8d680cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:31.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.074 7f8d66d73700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8d680d1c80 0x7f8d680cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:31.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.074 7f8d67d75700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d680cf590 0x7f8d680cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:46778/0 (socket says 172.21.2.101:46778) 2021-02-10T11:36:31.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.074 7f8d67574700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f8d680d1880 0x7f8d681b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:47948/0 (socket says 172.21.2.101:47948) 2021-02-10T11:36:31.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.074 7f8d67d75700 1 -- 172.21.2.101:0/3383959725 learned_addr learned my addr 172.21.2.101:0/3383959725 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:31.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.074 7f8d66d73700 1 -- 172.21.2.101:0/3383959725 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d680cf590 msgr2=0x7f8d680cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:31.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.074 7f8d66d73700 1 --2- 172.21.2.101:0/3383959725 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d680cf590 0x7f8d680cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:31.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.074 7f8d66d73700 1 -- 172.21.2.101:0/3383959725 >> v1:172.21.2.108:6789/0 conn(0x7f8d680d1880 legacy=0x7f8d681b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:31.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.074 7f8d66572700 1 -- 172.21.2.101:0/3383959725 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288452871 0 0) 0x7f8d580010b0 con 0x7f8d680d1880 2021-02-10T11:36:31.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.074 7f8d66d73700 1 -- 172.21.2.101:0/3383959725 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d68088070 con 0x7f8d680d1c80 2021-02-10T11:36:31.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.074 7f8d67d75700 1 --2- 172.21.2.101:0/3383959725 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d680cf590 0x7f8d680cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:36:31.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.075 7f8d66d73700 1 --2- 172.21.2.101:0/3383959725 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d680d1c80 0x7f8d680cef70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f8d5c006650 tx=0x7f8d5c009a80).ready entity=mon.1 client_cookie=3f986cfbe1d53960 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:31.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.075 7f8d66572700 1 -- 172.21.2.101:0/3383959725 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d5c0009b0 con 0x7f8d680d1c80 2021-02-10T11:36:31.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.075 7f8d66572700 1 -- 172.21.2.101:0/3383959725 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5c00c0c0 con 0x7f8d680d1c80 2021-02-10T11:36:31.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.075 7f8d66572700 1 -- 172.21.2.101:0/3383959725 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d5c00c5e0 con 0x7f8d680d1c80 2021-02-10T11:36:31.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.075 7f8d6d7ba700 1 -- 172.21.2.101:0/3383959725 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d680d1c80 msgr2=0x7f8d680cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:31.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.075 7f8d6d7ba700 1 --2- 172.21.2.101:0/3383959725 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d680d1c80 0x7f8d680cef70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f8d5c006650 tx=0x7f8d5c009a80).stop 2021-02-10T11:36:31.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.075 7f8d6d7ba700 1 -- 172.21.2.101:0/3383959725 shutdown_connections 2021-02-10T11:36:31.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.075 7f8d6d7ba700 1 --2- 172.21.2.101:0/3383959725 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d680cf590 0x7f8d680cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:31.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.075 7f8d6d7ba700 1 --2- 172.21.2.101:0/3383959725 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d680d1c80 0x7f8d680cef70 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:31.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.075 7f8d6d7ba700 1 -- 172.21.2.101:0/3383959725 >> 172.21.2.101:0/3383959725 conn(0x7f8d680ce430 msgr2=0x7f8d680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:31.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.075 7f8d6d7ba700 1 -- 172.21.2.101:0/3383959725 shutdown_connections 2021-02-10T11:36:31.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.075 7f8d6d7ba700 1 -- 172.21.2.101:0/3383959725 wait complete. 2021-02-10T11:36:31.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.090 7f8d6d7ba700 1 Processor -- start 2021-02-10T11:36:31.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.090 7f8d6d7ba700 1 -- start start 2021-02-10T11:36:31.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.090 7f8d6d7ba700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d680b4bb0 0x7f8d6809ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:31.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.090 7f8d6d7ba700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d6809b400 0x7f8d6809b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:31.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.090 7f8d6d7ba700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d68086480 0x7f8d68086880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:31.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.091 7f8d6d7ba700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d680b4100 con 0x7f8d680b4bb0 2021-02-10T11:36:31.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.091 7f8d6d7ba700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6809be60 con 0x7f8d68086480 2021-02-10T11:36:31.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.091 7f8d6d7ba700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d680b4600 con 0x7f8d6809b400 2021-02-10T11:36:31.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.091 7f8d67574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d680b4bb0 0x7f8d6809ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:31.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.091 7f8d67574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d680b4bb0 0x7f8d6809ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54048/0 (socket says 172.21.2.101:54048) 2021-02-10T11:36:31.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.091 7f8d67574700 1 -- 172.21.2.101:0/79081235 learned_addr learned my addr 172.21.2.101:0/79081235 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:31.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.091 7f8d66d73700 1 --2- 172.21.2.101:0/79081235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d6809b400 0x7f8d6809b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:31.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.091 7f8d67d75700 1 --2- 172.21.2.101:0/79081235 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d68086480 0x7f8d68086880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:31.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.091 7f8d67574700 1 -- 172.21.2.101:0/79081235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d6809b400 msgr2=0x7f8d6809b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:31.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.091 7f8d67574700 1 --2- 172.21.2.101:0/79081235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d6809b400 0x7f8d6809b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:31.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.091 7f8d67574700 1 -- 172.21.2.101:0/79081235 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d68086480 msgr2=0x7f8d68086880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:31.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.091 7f8d67574700 1 --2- 172.21.2.101:0/79081235 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d68086480 0x7f8d68086880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:31.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.091 7f8d67574700 1 -- 172.21.2.101:0/79081235 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d680b4fb0 con 0x7f8d680b4bb0 2021-02-10T11:36:31.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.091 7f8d67d75700 1 --2- 172.21.2.101:0/79081235 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d68086480 0x7f8d68086880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:36:31.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.091 7f8d67574700 1 --2- 172.21.2.101:0/79081235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d680b4bb0 0x7f8d6809ade0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 rx=0x7f8d5800a4c0 tx=0x7f8d5800a860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:31.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.092 7f8d4ffff700 1 -- 172.21.2.101:0/79081235 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d580009b0 con 0x7f8d680b4bb0 2021-02-10T11:36:31.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.092 7f8d4ffff700 1 -- 172.21.2.101:0/79081235 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d58004140 con 0x7f8d680b4bb0 2021-02-10T11:36:31.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.092 7f8d4ffff700 1 -- 172.21.2.101:0/79081235 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d580045a0 con 0x7f8d680b4bb0 2021-02-10T11:36:31.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.092 7f8d6d7ba700 1 -- 172.21.2.101:0/79081235 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d680b4fb0 con 0x7f8d680b4bb0 2021-02-10T11:36:31.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.092 7f8d6d7ba700 1 -- 172.21.2.101:0/79081235 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d680b4fb0 con 0x7f8d680b4bb0 2021-02-10T11:36:31.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.092 7f8d4ffff700 1 -- 172.21.2.101:0/79081235 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8d5800ac50 con 0x7f8d680b4bb0 2021-02-10T11:36:31.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.092 7f8d4ffff700 1 --2- 172.21.2.101:0/79081235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d480186c0 0x7f8d4801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:31.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.092 7f8d4ffff700 1 -- 172.21.2.101:0/79081235 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8d58034600 con 0x7f8d680b4bb0 2021-02-10T11:36:31.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.093 7f8d66d73700 1 --2- 172.21.2.101:0/79081235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d480186c0 0x7f8d4801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:31.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.093 7f8d66d73700 1 --2- 172.21.2.101:0/79081235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d480186c0 0x7f8d4801abc0 crc :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:31.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.123 7f8d6d7ba700 1 -- 172.21.2.101:0/79081235 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d680fe1d0 con 0x7f8d680b4bb0 2021-02-10T11:36:31.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.126 7f8d4ffff700 1 -- 172.21.2.101:0/79081235 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8d58000cc0 con 0x7f8d680b4bb0 2021-02-10T11:36:31.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.257 7f8d6d7ba700 1 -- 172.21.2.101:0/79081235 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000678861", "target": ["mgr", ""], "snap_name": "snapshot_0000000000465306"}) v1 -- 0x7f8d680b45c0 con 0x7f8d480186c0 2021-02-10T11:36:31.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.268 7f8d4ffff700 1 -- 172.21.2.101:0/79081235 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f8d5c000c90 con 0x7f8d480186c0 2021-02-10T11:36:31.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.273 7f8d6d7ba700 1 -- 172.21.2.101:0/79081235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d480186c0 msgr2=0x7f8d4801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:31.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.273 7f8d6d7ba700 1 --2- 172.21.2.101:0/79081235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d480186c0 0x7f8d4801abc0 crc :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:31.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.273 7f8d6d7ba700 1 -- 172.21.2.101:0/79081235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d680b4bb0 msgr2=0x7f8d6809ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:31.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.273 7f8d6d7ba700 1 --2- 172.21.2.101:0/79081235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d680b4bb0 0x7f8d6809ade0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 rx=0x7f8d5800a4c0 tx=0x7f8d5800a860).stop 2021-02-10T11:36:31.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.274 7f8d6d7ba700 1 -- 172.21.2.101:0/79081235 shutdown_connections 2021-02-10T11:36:31.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.274 7f8d6d7ba700 1 --2- 172.21.2.101:0/79081235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d480186c0 0x7f8d4801abc0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:31.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.274 7f8d6d7ba700 1 --2- 172.21.2.101:0/79081235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d6809b400 0x7f8d6809b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:31.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.274 7f8d6d7ba700 1 --2- 172.21.2.101:0/79081235 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d68086480 0x7f8d68086880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:31.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.274 7f8d6d7ba700 1 --2- 172.21.2.101:0/79081235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d680b4bb0 0x7f8d6809ade0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:31.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.274 7f8d6d7ba700 1 -- 172.21.2.101:0/79081235 >> 172.21.2.101:0/79081235 conn(0x7f8d6809ebf0 msgr2=0x7f8d680b53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:31.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.274 7f8d6d7ba700 1 -- 172.21.2.101:0/79081235 shutdown_connections 2021-02-10T11:36:31.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.274 7f8d6d7ba700 1 -- 172.21.2.101:0/79081235 wait complete. 2021-02-10T11:36:31.289 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume info cephfs clone_0000000000005193 2021-02-10T11:36:31.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.735 7f64368ab700 1 Processor -- start 2021-02-10T11:36:31.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.735 7f64368ab700 1 -- start start 2021-02-10T11:36:31.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.736 7f64368ab700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f64300d1860 0x7f64301b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:31.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.736 7f64368ab700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f64301b0ad0 0x7f6430108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:31.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.736 7f64368ab700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6430109a50 con 0x7f64300d1c60 2021-02-10T11:36:31.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.736 7f64368ab700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64300e5ee0 con 0x7f64301b0ad0 2021-02-10T11:36:31.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.736 7f64368ab700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6430109740 con 0x7f64300d1860 2021-02-10T11:36:31.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.736 7f642f7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f64301b0ad0 0x7f6430108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:31.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.736 7f6434ec8700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f64300d1c60 0x7f6430108f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54552/0 (socket says 172.21.2.101:54552) 2021-02-10T11:36:31.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.736 7f6434ec8700 1 -- 172.21.2.101:0/153117448 learned_addr learned my addr 172.21.2.101:0/153117448 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:31.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.736 7f642ffff700 1 --2- 172.21.2.101:0/153117448 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f64300d1860 0x7f64301b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:31.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.737 7f642f7fe700 1 -- 172.21.2.101:0/153117448 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f64300d1860 msgr2=0x7f64301b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:31.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.737 7f642f7fe700 1 --2- 172.21.2.101:0/153117448 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f64300d1860 0x7f64301b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:31.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.737 7f642f7fe700 1 -- 172.21.2.101:0/153117448 >> v1:172.21.2.101:6789/0 conn(0x7f64300d1c60 legacy=0x7f6430108f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:31.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.737 7f642f7fe700 1 -- 172.21.2.101:0/153117448 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6430088070 con 0x7f64301b0ad0 2021-02-10T11:36:31.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.737 7f642ffff700 1 --2- 172.21.2.101:0/153117448 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f64300d1860 0x7f64301b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:36:31.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.737 7f642f7fe700 1 --2- 172.21.2.101:0/153117448 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f64301b0ad0 0x7f6430108930 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 rx=0x7f6424006650 tx=0x7f6424009a80).ready entity=mon.0 client_cookie=1bd89390601df77c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:31.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.737 7f642effd700 1 -- 172.21.2.101:0/153117448 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f642400c0c0 con 0x7f64301b0ad0 2021-02-10T11:36:31.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.737 7f642effd700 1 -- 172.21.2.101:0/153117448 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642400c340 con 0x7f64301b0ad0 2021-02-10T11:36:31.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.737 7f642effd700 1 -- 172.21.2.101:0/153117448 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f642400c860 con 0x7f64301b0ad0 2021-02-10T11:36:31.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.737 7f64368ab700 1 -- 172.21.2.101:0/153117448 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f64301b0ad0 msgr2=0x7f6430108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:31.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.737 7f64368ab700 1 --2- 172.21.2.101:0/153117448 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f64301b0ad0 0x7f6430108930 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 rx=0x7f6424006650 tx=0x7f6424009a80).stop 2021-02-10T11:36:31.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.738 7f64368ab700 1 -- 172.21.2.101:0/153117448 shutdown_connections 2021-02-10T11:36:31.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.738 7f64368ab700 1 --2- 172.21.2.101:0/153117448 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f64301b0ad0 0x7f6430108930 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:31.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.738 7f64368ab700 1 --2- 172.21.2.101:0/153117448 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f64300d1860 0x7f64301b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:31.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.738 7f64368ab700 1 -- 172.21.2.101:0/153117448 >> 172.21.2.101:0/153117448 conn(0x7f64300ce410 msgr2=0x7f64300d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:31.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.738 7f64368ab700 1 -- 172.21.2.101:0/153117448 shutdown_connections 2021-02-10T11:36:31.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.738 7f64368ab700 1 -- 172.21.2.101:0/153117448 wait complete. 2021-02-10T11:36:31.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.753 7f64368ab700 1 Processor -- start 2021-02-10T11:36:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.753 7f64368ab700 1 -- start start 2021-02-10T11:36:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.753 7f64368ab700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f643009f9f0 0x7f643009f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.753 7f64368ab700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6430187e60 0x7f64300c1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.753 7f64368ab700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f64300c2030 0x7f64300c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.753 7f64368ab700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6430188260 con 0x7f643009f9f0 2021-02-10T11:36:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.753 7f64368ab700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6430188480 con 0x7f64300c2030 2021-02-10T11:36:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.753 7f64368ab700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64300c3020 con 0x7f6430187e60 2021-02-10T11:36:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.753 7f642ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f643009f9f0 0x7f643009f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.753 7f642ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f643009f9f0 0x7f643009f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54064/0 (socket says 172.21.2.101:54064) 2021-02-10T11:36:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.753 7f642ffff700 1 -- 172.21.2.101:0/2401983241 learned_addr learned my addr 172.21.2.101:0/2401983241 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.754 7f642ffff700 1 -- 172.21.2.101:0/2401983241 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6430187e60 msgr2=0x7f64300c1a10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.754 7f642ffff700 1 --2- 172.21.2.101:0/2401983241 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6430187e60 0x7f64300c1a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.754 7f642ffff700 1 -- 172.21.2.101:0/2401983241 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f64300c2030 msgr2=0x7f64300c2a00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.754 7f642ffff700 1 --2- 172.21.2.101:0/2401983241 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f64300c2030 0x7f64300c2a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.754 7f642ffff700 1 -- 172.21.2.101:0/2401983241 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64300c3240 con 0x7f643009f9f0 2021-02-10T11:36:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.754 7f642ffff700 1 --2- 172.21.2.101:0/2401983241 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f643009f9f0 0x7f643009f190 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 rx=0x7f6420006f20 tx=0x7f642000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.754 7f642cff9700 1 -- 172.21.2.101:0/2401983241 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f642000bd80 con 0x7f643009f9f0 2021-02-10T11:36:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.754 7f642cff9700 1 -- 172.21.2.101:0/2401983241 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6420000c90 con 0x7f643009f9f0 2021-02-10T11:36:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.754 7f642cff9700 1 -- 172.21.2.101:0/2401983241 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f64200022b0 con 0x7f643009f9f0 2021-02-10T11:36:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.754 7f64368ab700 1 -- 172.21.2.101:0/2401983241 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64300c3240 con 0x7f643009f9f0 2021-02-10T11:36:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.754 7f64368ab700 1 -- 172.21.2.101:0/2401983241 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64301a31c0 con 0x7f643009f9f0 2021-02-10T11:36:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.755 7f642cff9700 1 -- 172.21.2.101:0/2401983241 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f642000a000 con 0x7f643009f9f0 2021-02-10T11:36:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.755 7f642cff9700 1 --2- 172.21.2.101:0/2401983241 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f64100186c0 0x7f641001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.755 7f642cff9700 1 -- 172.21.2.101:0/2401983241 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6420032520 con 0x7f643009f9f0 2021-02-10T11:36:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.755 7f642f7fe700 1 --2- 172.21.2.101:0/2401983241 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f64100186c0 0x7f641001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.755 7f642f7fe700 1 --2- 172.21.2.101:0/2401983241 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f64100186c0 0x7f641001abc0 crc :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:31.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.786 7f64368ab700 1 -- 172.21.2.101:0/2401983241 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64300c24d0 con 0x7f643009f9f0 2021-02-10T11:36:31.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.789 7f642cff9700 1 -- 172.21.2.101:0/2401983241 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f64200009b0 con 0x7f643009f9f0 2021-02-10T11:36:31.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.916 7f64368ab700 1 -- 172.21.2.101:0/2401983241 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "clone_0000000000005193", "target": ["mgr", ""]}) v1 -- 0x7f6430188260 con 0x7f64100186c0 2021-02-10T11:36:31.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.922 7f642cff9700 1 -- 172.21.2.101:0/2401983241 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+724 (crc 0 0 0) 0x7f6424000c90 con 0x7f64100186c0 2021-02-10T11:36:31.930 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:36:31.930 INFO:teuthology.orchestra.run.gibba001.stdout: "atime": "2021-02-10 11:36:27", 2021-02-10T11:36:31.930 INFO:teuthology.orchestra.run.gibba001.stdout: "bytes_pcent": "undefined", 2021-02-10T11:36:31.930 INFO:teuthology.orchestra.run.gibba001.stdout: "bytes_quota": "infinite", 2021-02-10T11:36:31.930 INFO:teuthology.orchestra.run.gibba001.stdout: "bytes_used": 0, 2021-02-10T11:36:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: "created_at": "2021-02-10 11:36:29", 2021-02-10T11:36:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: "ctime": "2021-02-10 11:36:30", 2021-02-10T11:36:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: "data_pool": "cephfs_data", 2021-02-10T11:36:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: "features": [ 2021-02-10T11:36:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: "snapshot-clone", 2021-02-10T11:36:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: "snapshot-autoprotect", 2021-02-10T11:36:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: "snapshot-retention" 2021-02-10T11:36:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:36:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: "gid": 0, 2021-02-10T11:36:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: "mode": 16877, 2021-02-10T11:36:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: "mon_addrs": [ 2021-02-10T11:36:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: "172.21.2.101:6789", 2021-02-10T11:36:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: "172.21.2.108:6789", 2021-02-10T11:36:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: "172.21.2.108:6790" 2021-02-10T11:36:31.932 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:36:31.932 INFO:teuthology.orchestra.run.gibba001.stdout: "mtime": "2021-02-10 11:36:28", 2021-02-10T11:36:31.932 INFO:teuthology.orchestra.run.gibba001.stdout: "path": "/volumes/_nogroup/clone_0000000000005193/25e5112a-38c8-43c8-95fc-073aaaaf925f", 2021-02-10T11:36:31.932 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_namespace": "", 2021-02-10T11:36:31.932 INFO:teuthology.orchestra.run.gibba001.stdout: "state": "complete", 2021-02-10T11:36:31.932 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "clone", 2021-02-10T11:36:31.932 INFO:teuthology.orchestra.run.gibba001.stdout: "uid": 0 2021-02-10T11:36:31.932 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:36:31.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.925 7f64368ab700 1 -- 172.21.2.101:0/2401983241 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f64100186c0 msgr2=0x7f641001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:31.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.925 7f64368ab700 1 --2- 172.21.2.101:0/2401983241 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f64100186c0 0x7f641001abc0 crc :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:31.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.925 7f64368ab700 1 -- 172.21.2.101:0/2401983241 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f643009f9f0 msgr2=0x7f643009f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:31.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.925 7f64368ab700 1 --2- 172.21.2.101:0/2401983241 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f643009f9f0 0x7f643009f190 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 rx=0x7f6420006f20 tx=0x7f642000b990).stop 2021-02-10T11:36:31.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.925 7f64368ab700 1 -- 172.21.2.101:0/2401983241 shutdown_connections 2021-02-10T11:36:31.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.925 7f64368ab700 1 --2- 172.21.2.101:0/2401983241 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f64100186c0 0x7f641001abc0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:31.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.925 7f64368ab700 1 --2- 172.21.2.101:0/2401983241 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6430187e60 0x7f64300c1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:31.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.925 7f64368ab700 1 --2- 172.21.2.101:0/2401983241 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f64300c2030 0x7f64300c2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:31.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.925 7f64368ab700 1 --2- 172.21.2.101:0/2401983241 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f643009f9f0 0x7f643009f190 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:31.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.925 7f64368ab700 1 -- 172.21.2.101:0/2401983241 >> 172.21.2.101:0/2401983241 conn(0x7f64300c3740 msgr2=0x7f64300ae030 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:31.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.926 7f64368ab700 1 -- 172.21.2.101:0/2401983241 shutdown_connections 2021-02-10T11:36:31.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:31.926 7f64368ab700 1 -- 172.21.2.101:0/2401983241 wait complete. 2021-02-10T11:36:31.941 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000678861 2021-02-10T11:36:32.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.348 7fdbeb659700 1 Processor -- start 2021-02-10T11:36:32.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.348 7fdbeb659700 1 -- start start 2021-02-10T11:36:32.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.349 7fdbeb659700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdbe40d1870 0x7fdbe41b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:32.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.349 7fdbeb659700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe40fe1c0 con 0x7fdbe41b0ae0 2021-02-10T11:36:32.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.349 7fdbeb659700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe4109a60 con 0x7fdbe40cff70 2021-02-10T11:36:32.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.349 7fdbeb659700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe4088070 con 0x7fdbe40d1870 2021-02-10T11:36:32.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.349 7fdbe9475700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdbe40d1870 0x7fdbe41b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:32.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.349 7fdbe8c74700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fdbe41b0ae0 0x7fdbe4108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54562/0 (socket says 172.21.2.101:54562) 2021-02-10T11:36:32.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.349 7fdbe8c74700 1 -- 172.21.2.101:0/2527340671 learned_addr learned my addr 172.21.2.101:0/2527340671 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:32.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.349 7fdbd7fff700 1 -- 172.21.2.101:0/2527340671 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229367461 0 0) 0x7fdbe00010b0 con 0x7fdbe41b0ae0 2021-02-10T11:36:32.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.349 7fdbd7fff700 1 -- 172.21.2.101:0/2527340671 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbd0002fe0 con 0x7fdbe41b0ae0 2021-02-10T11:36:32.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.349 7fdbe9475700 1 -- 172.21.2.101:0/2527340671 >> v1:172.21.2.108:6789/0 conn(0x7fdbe40cff70 legacy=0x7fdbe4109130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:36:32.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.349 7fdbe9475700 1 -- 172.21.2.101:0/2527340671 >> v1:172.21.2.101:6789/0 conn(0x7fdbe41b0ae0 legacy=0x7fdbe4108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:32.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.349 7fdbe9475700 1 -- 172.21.2.101:0/2527340671 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe4088260 con 0x7fdbe40d1870 2021-02-10T11:36:32.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.349 7fdbe9475700 1 --2- 172.21.2.101:0/2527340671 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdbe40d1870 0x7fdbe41b04c0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 rx=0x7fdbd8006650 tx=0x7fdbd8009a80).ready entity=mon.0 client_cookie=187660c025bad3e4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:32.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.350 7fdbd7fff700 1 -- 172.21.2.101:0/2527340671 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdbd80009b0 con 0x7fdbe40d1870 2021-02-10T11:36:32.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.350 7fdbd7fff700 1 -- 172.21.2.101:0/2527340671 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd800c0c0 con 0x7fdbe40d1870 2021-02-10T11:36:32.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.350 7fdbd7fff700 1 -- 172.21.2.101:0/2527340671 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdbd800c5e0 con 0x7fdbe40d1870 2021-02-10T11:36:32.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.350 7fdbeb659700 1 -- 172.21.2.101:0/2527340671 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdbe40d1870 msgr2=0x7fdbe41b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:32.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.350 7fdbeb659700 1 --2- 172.21.2.101:0/2527340671 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdbe40d1870 0x7fdbe41b04c0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 rx=0x7fdbd8006650 tx=0x7fdbd8009a80).stop 2021-02-10T11:36:32.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.350 7fdbeb659700 1 -- 172.21.2.101:0/2527340671 shutdown_connections 2021-02-10T11:36:32.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.350 7fdbeb659700 1 --2- 172.21.2.101:0/2527340671 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdbe40d1870 0x7fdbe41b04c0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:32.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.350 7fdbeb659700 1 -- 172.21.2.101:0/2527340671 >> 172.21.2.101:0/2527340671 conn(0x7fdbe40ce420 msgr2=0x7fdbe40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:32.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.351 7fdbeb659700 1 -- 172.21.2.101:0/2527340671 shutdown_connections 2021-02-10T11:36:32.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.351 7fdbeb659700 1 -- 172.21.2.101:0/2527340671 wait complete. 2021-02-10T11:36:32.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.365 7fdbeb659700 1 Processor -- start 2021-02-10T11:36:32.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.365 7fdbeb659700 1 -- start start 2021-02-10T11:36:32.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.365 7fdbeb659700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdbe40c3820 0x7fdbe4084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:32.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.365 7fdbeb659700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdbe40a0290 0x7fdbe40852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:32.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.365 7fdbeb659700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdbe4086270 0x7fdbe4086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:32.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.365 7fdbeb659700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe40c0700 con 0x7fdbe40c3820 2021-02-10T11:36:32.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.365 7fdbeb659700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe40858f0 con 0x7fdbe4086270 2021-02-10T11:36:32.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.365 7fdbeb659700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe4085b10 con 0x7fdbe40a0290 2021-02-10T11:36:32.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.366 7fdbe9475700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdbe40c3820 0x7fdbe4084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:32.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.366 7fdbe9475700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdbe40c3820 0x7fdbe4084cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54072/0 (socket says 172.21.2.101:54072) 2021-02-10T11:36:32.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.366 7fdbe9475700 1 -- 172.21.2.101:0/337195977 learned_addr learned my addr 172.21.2.101:0/337195977 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:32.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.366 7fdbe9475700 1 -- 172.21.2.101:0/337195977 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdbe40a0290 msgr2=0x7fdbe40852d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:32.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.366 7fdbe9475700 1 --2- 172.21.2.101:0/337195977 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdbe40a0290 0x7fdbe40852d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:32.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.366 7fdbe9475700 1 -- 172.21.2.101:0/337195977 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdbe4086270 msgr2=0x7fdbe4086670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:32.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.366 7fdbe9475700 1 --2- 172.21.2.101:0/337195977 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdbe4086270 0x7fdbe4086670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:32.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.366 7fdbe9475700 1 -- 172.21.2.101:0/337195977 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe4086c90 con 0x7fdbe40c3820 2021-02-10T11:36:32.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.366 7fdbe9475700 1 --2- 172.21.2.101:0/337195977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdbe40c3820 0x7fdbe4084cb0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 rx=0x7fdbd8009630 tx=0x7fdbd800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:32.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.366 7fdbd5ffb700 1 -- 172.21.2.101:0/337195977 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdbd80093a0 con 0x7fdbe40c3820 2021-02-10T11:36:32.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.366 7fdbd5ffb700 1 -- 172.21.2.101:0/337195977 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd800bd70 con 0x7fdbe40c3820 2021-02-10T11:36:32.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.366 7fdbd5ffb700 1 -- 172.21.2.101:0/337195977 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdbd8005560 con 0x7fdbe40c3820 2021-02-10T11:36:32.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.366 7fdbeb659700 1 -- 172.21.2.101:0/337195977 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe4086c90 con 0x7fdbe40c3820 2021-02-10T11:36:32.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.366 7fdbeb659700 1 -- 172.21.2.101:0/337195977 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe41a2f00 con 0x7fdbe40c3820 2021-02-10T11:36:32.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.367 7fdbd5ffb700 1 -- 172.21.2.101:0/337195977 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdbd800a000 con 0x7fdbe40c3820 2021-02-10T11:36:32.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.367 7fdbd5ffb700 1 --2- 172.21.2.101:0/337195977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdbc40186c0 0x7fdbc401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:32.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.367 7fdbd5ffb700 1 -- 172.21.2.101:0/337195977 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fdbd8032580 con 0x7fdbe40c3820 2021-02-10T11:36:32.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.367 7fdbe8c74700 1 --2- 172.21.2.101:0/337195977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdbc40186c0 0x7fdbc401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:32.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.368 7fdbe8c74700 1 --2- 172.21.2.101:0/337195977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdbc40186c0 0x7fdbc401abc0 crc :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:32.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.399 7fdbeb659700 1 -- 172.21.2.101:0/337195977 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe4082a70 con 0x7fdbe40c3820 2021-02-10T11:36:32.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.401 7fdbd5ffb700 1 -- 172.21.2.101:0/337195977 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdbd802acb0 con 0x7fdbe40c3820 2021-02-10T11:36:32.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.528 7fdbeb659700 1 -- 172.21.2.101:0/337195977 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000678861", "target": ["mgr", ""]}) v1 -- 0x7fdbe41a2f00 con 0x7fdbc40186c0 2021-02-10T11:36:32.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.539 7fdbd5ffb700 1 -- 172.21.2.101:0/337195977 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fdbe00014c0 con 0x7fdbc40186c0 2021-02-10T11:36:32.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.545 7fdbeb659700 1 -- 172.21.2.101:0/337195977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdbc40186c0 msgr2=0x7fdbc401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:32.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.545 7fdbeb659700 1 --2- 172.21.2.101:0/337195977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdbc40186c0 0x7fdbc401abc0 crc :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:32.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.545 7fdbeb659700 1 -- 172.21.2.101:0/337195977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdbe40c3820 msgr2=0x7fdbe4084cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:32.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.545 7fdbeb659700 1 --2- 172.21.2.101:0/337195977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdbe40c3820 0x7fdbe4084cb0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 rx=0x7fdbd8009630 tx=0x7fdbd800b980).stop 2021-02-10T11:36:32.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.545 7fdbeb659700 1 -- 172.21.2.101:0/337195977 shutdown_connections 2021-02-10T11:36:32.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.545 7fdbeb659700 1 --2- 172.21.2.101:0/337195977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdbc40186c0 0x7fdbc401abc0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:32.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.545 7fdbeb659700 1 --2- 172.21.2.101:0/337195977 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdbe40a0290 0x7fdbe40852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:32.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.545 7fdbeb659700 1 --2- 172.21.2.101:0/337195977 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdbe4086270 0x7fdbe4086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:32.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.545 7fdbeb659700 1 --2- 172.21.2.101:0/337195977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdbe40c3820 0x7fdbe4084cb0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:32.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.545 7fdbeb659700 1 -- 172.21.2.101:0/337195977 >> 172.21.2.101:0/337195977 conn(0x7fdbe40872c0 msgr2=0x7fdbe4095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:32.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.546 7fdbeb659700 1 -- 172.21.2.101:0/337195977 shutdown_connections 2021-02-10T11:36:32.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.546 7fdbeb659700 1 -- 172.21.2.101:0/337195977 wait complete. 2021-02-10T11:36:32.563 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs clone_0000000000005193 2021-02-10T11:36:32.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.973 7f2f889cb700 1 Processor -- start 2021-02-10T11:36:32.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.973 7f2f889cb700 1 -- start start 2021-02-10T11:36:32.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.974 7f2f889cb700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2f840d1860 0x7f2f841b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.974 7f2f889cb700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f840fe1b0 con 0x7f2f841b0ad0 2021-02-10T11:36:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.974 7f2f889cb700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f84109a50 con 0x7f2f840cff60 2021-02-10T11:36:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.974 7f2f889cb700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f84088070 con 0x7f2f840d1860 2021-02-10T11:36:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.974 7f2f81e1b700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f2f841b0ad0 0x7f2f84108930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54574/0 (socket says 172.21.2.101:54574) 2021-02-10T11:36:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.974 7f2f81e1b700 1 -- 172.21.2.101:0/204101231 learned_addr learned my addr 172.21.2.101:0/204101231 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.974 7f2f8261c700 1 --2- 172.21.2.101:0/204101231 >> v2:172.21.2.108:3300/0 conn(0x7f2f840d1860 0x7f2f841b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.974 7f2f8161a700 1 -- 172.21.2.101:0/204101231 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3594020529 0 0) 0x7f2f780010b0 con 0x7f2f841b0ad0 2021-02-10T11:36:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.974 7f2f8161a700 1 -- 172.21.2.101:0/204101231 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f6c002fe0 con 0x7f2f841b0ad0 2021-02-10T11:36:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.974 7f2f8261c700 1 -- 172.21.2.101:0/204101231 >> v1:172.21.2.108:6789/0 conn(0x7f2f840cff60 legacy=0x7f2f84109120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.974 7f2f8261c700 1 -- 172.21.2.101:0/204101231 >> v1:172.21.2.101:6789/0 conn(0x7f2f841b0ad0 legacy=0x7f2f84108930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.974 7f2f8261c700 1 -- 172.21.2.101:0/204101231 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f84088260 con 0x7f2f840d1860 2021-02-10T11:36:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.975 7f2f8261c700 1 --2- 172.21.2.101:0/204101231 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2f840d1860 0x7f2f841b04b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f2f74006650 tx=0x7f2f74009a80).ready entity=mon.1 client_cookie=b73d143c39059823 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.975 7f2f8161a700 1 -- 172.21.2.101:0/204101231 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2f7400c0c0 con 0x7f2f840d1860 2021-02-10T11:36:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.975 7f2f8161a700 1 -- 172.21.2.101:0/204101231 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7400c340 con 0x7f2f840d1860 2021-02-10T11:36:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.975 7f2f8161a700 1 -- 172.21.2.101:0/204101231 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2f740009b0 con 0x7f2f840d1860 2021-02-10T11:36:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.975 7f2f889cb700 1 -- 172.21.2.101:0/204101231 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2f840d1860 msgr2=0x7f2f841b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.975 7f2f889cb700 1 --2- 172.21.2.101:0/204101231 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2f840d1860 0x7f2f841b04b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f2f74006650 tx=0x7f2f74009a80).stop 2021-02-10T11:36:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.976 7f2f889cb700 1 -- 172.21.2.101:0/204101231 shutdown_connections 2021-02-10T11:36:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.976 7f2f889cb700 1 --2- 172.21.2.101:0/204101231 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2f840d1860 0x7f2f841b04b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.976 7f2f889cb700 1 -- 172.21.2.101:0/204101231 >> 172.21.2.101:0/204101231 conn(0x7f2f840ce410 msgr2=0x7f2f840d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.976 7f2f889cb700 1 -- 172.21.2.101:0/204101231 shutdown_connections 2021-02-10T11:36:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.976 7f2f889cb700 1 -- 172.21.2.101:0/204101231 wait complete. 2021-02-10T11:36:33.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.990 7f2f889cb700 1 Processor -- start 2021-02-10T11:36:33.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.990 7f2f889cb700 1 -- start start 2021-02-10T11:36:33.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.991 7f2f889cb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2f841a05e0 0x7f2f8418b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:33.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.991 7f2f889cb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2f840b8780 0x7f2f840bfa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:33.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.991 7f2f889cb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2f840c0030 0x7f2f84189e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.991 7f2f889cb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f840b82a0 con 0x7f2f840c0030 2021-02-10T11:36:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.991 7f2f889cb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f840c0430 con 0x7f2f840b8780 2021-02-10T11:36:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.991 7f2f889cb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f840c0650 con 0x7f2f841a05e0 2021-02-10T11:36:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.991 7f2f82e1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2f840c0030 0x7f2f84189e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.991 7f2f8261c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2f841a05e0 0x7f2f8418b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.991 7f2f82e1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2f840c0030 0x7f2f84189e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54088/0 (socket says 172.21.2.101:54088) 2021-02-10T11:36:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.991 7f2f82e1d700 1 -- 172.21.2.101:0/3055170349 learned_addr learned my addr 172.21.2.101:0/3055170349 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.991 7f2f81e1b700 1 --2- 172.21.2.101:0/3055170349 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2f840b8780 0x7f2f840bfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.992 7f2f82e1d700 1 -- 172.21.2.101:0/3055170349 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2f841a05e0 msgr2=0x7f2f8418b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.992 7f2f82e1d700 1 --2- 172.21.2.101:0/3055170349 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2f841a05e0 0x7f2f8418b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.992 7f2f82e1d700 1 -- 172.21.2.101:0/3055170349 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2f840b8780 msgr2=0x7f2f840bfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.992 7f2f82e1d700 1 --2- 172.21.2.101:0/3055170349 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2f840b8780 0x7f2f840bfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.992 7f2f82e1d700 1 -- 172.21.2.101:0/3055170349 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8418a490 con 0x7f2f840c0030 2021-02-10T11:36:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.992 7f2f82e1d700 1 --2- 172.21.2.101:0/3055170349 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2f840c0030 0x7f2f84189e70 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 rx=0x7f2f7c009940 tx=0x7f2f7c00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.992 7f2f72ffd700 1 -- 172.21.2.101:0/3055170349 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2f7c000ce0 con 0x7f2f840c0030 2021-02-10T11:36:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.992 7f2f72ffd700 1 -- 172.21.2.101:0/3055170349 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7c00c2f0 con 0x7f2f840c0030 2021-02-10T11:36:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.992 7f2f889cb700 1 -- 172.21.2.101:0/3055170349 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8418a490 con 0x7f2f840c0030 2021-02-10T11:36:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.992 7f2f889cb700 1 -- 172.21.2.101:0/3055170349 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8418aa20 con 0x7f2f840c0030 2021-02-10T11:36:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.992 7f2f72ffd700 1 -- 172.21.2.101:0/3055170349 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2f7c00c810 con 0x7f2f840c0030 2021-02-10T11:36:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.993 7f2f72ffd700 1 -- 172.21.2.101:0/3055170349 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2f7c001220 con 0x7f2f840c0030 2021-02-10T11:36:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.993 7f2f72ffd700 1 --2- 172.21.2.101:0/3055170349 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f680186c0 0x7f2f6801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.993 7f2f72ffd700 1 -- 172.21.2.101:0/3055170349 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2f7c032b20 con 0x7f2f840c0030 2021-02-10T11:36:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.993 7f2f8261c700 1 --2- 172.21.2.101:0/3055170349 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f680186c0 0x7f2f6801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:32.993 7f2f8261c700 1 --2- 172.21.2.101:0/3055170349 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f680186c0 0x7f2f6801abc0 crc :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:33.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:33.024 7f2f889cb700 1 -- 172.21.2.101:0/3055170349 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f84082a20 con 0x7f2f840c0030 2021-02-10T11:36:33.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:33.026 7f2f72ffd700 1 -- 172.21.2.101:0/3055170349 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2f7c000cc0 con 0x7f2f840c0030 2021-02-10T11:36:33.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:33.155 7f2f889cb700 1 -- 172.21.2.101:0/3055170349 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "clone_0000000000005193", "target": ["mgr", ""]}) v1 -- 0x7f2f8418aa20 con 0x7f2f680186c0 2021-02-10T11:36:33.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:33.163 7f2f72ffd700 1 -- 172.21.2.101:0/3055170349 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f2f740009b0 con 0x7f2f680186c0 2021-02-10T11:36:33.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:33.171 7f2f889cb700 1 -- 172.21.2.101:0/3055170349 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f680186c0 msgr2=0x7f2f6801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:33.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:33.171 7f2f889cb700 1 --2- 172.21.2.101:0/3055170349 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f680186c0 0x7f2f6801abc0 crc :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:33.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:33.171 7f2f889cb700 1 -- 172.21.2.101:0/3055170349 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2f840c0030 msgr2=0x7f2f84189e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:33.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:33.171 7f2f889cb700 1 --2- 172.21.2.101:0/3055170349 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2f840c0030 0x7f2f84189e70 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 rx=0x7f2f7c009940 tx=0x7f2f7c00bcf0).stop 2021-02-10T11:36:33.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:33.171 7f2f889cb700 1 -- 172.21.2.101:0/3055170349 shutdown_connections 2021-02-10T11:36:33.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:33.171 7f2f889cb700 1 --2- 172.21.2.101:0/3055170349 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2f840c0030 0x7f2f84189e70 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:33.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:33.171 7f2f889cb700 1 --2- 172.21.2.101:0/3055170349 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2f840b8780 0x7f2f840bfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:33.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:33.171 7f2f889cb700 1 --2- 172.21.2.101:0/3055170349 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f680186c0 0x7f2f6801abc0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:33.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:33.171 7f2f889cb700 1 --2- 172.21.2.101:0/3055170349 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2f841a05e0 0x7f2f8418b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:33.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:33.171 7f2f889cb700 1 -- 172.21.2.101:0/3055170349 >> 172.21.2.101:0/3055170349 conn(0x7f2f840a6a80 msgr2=0x7f2f840ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:33.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:33.172 7f2f889cb700 1 -- 172.21.2.101:0/3055170349 shutdown_connections 2021-02-10T11:36:33.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:33.172 7f2f889cb700 1 -- 172.21.2.101:0/3055170349 wait complete. 2021-02-10T11:36:33.186 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo getfattr --only-values -n ceph.dir.entries /home/ubuntu/cephtest/mnt.0/./volumes/_deleting 2021-02-10T11:36:33.537 INFO:teuthology.orchestra.run.gibba001.stderr:getfattr: Removing leading '/' from absolute path names 2021-02-10T11:36:33.538 INFO:teuthology.orchestra.run.gibba001.stdout:0 2021-02-10T11:36:33.538 DEBUG:tasks.cephfs.mount:Directory ./volumes/_deleting seen empty from 0 after 0s 2021-02-10T11:36:33.538 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:36:33.538 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:36:33.883 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:36:34.229 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:36:34.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:34.229 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:36:34.229 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:36:34.564 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:36:34.564 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:36:34.564 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:36:34.564 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:36:34.564 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:36:34.913 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[130383]: fuse finished with error 0 and tester_r 0 2021-02-10T11:36:34.915 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:34.915 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:36:34.915 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:36:35.250 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:36:35.250 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:36:35.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:36:35.250 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:36:35.586 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:35.586 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:36:35.586 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:36:35.921 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:36:35.921 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:36:35.922 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:36:35.922 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:36:35.922 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:36:36.276 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[131056]: fuse finished with error 0 and tester_r 0 2021-02-10T11:36:36.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:36.278 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:36:36.278 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:36:36.612 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:36:36.613 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:36:36.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:36:36.613 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:36:36.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:36.947 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:36:36.947 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:36:37.283 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:36:37.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:36:37.284 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:36:37.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:36:37.284 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:36:37.588 INFO:tasks.ceph.osd.6.gibba008.stderr:2021-02-10 11:36:37.600 7f5a06d54700 -1 osd.6 pg_epoch: 75 pg[15.5( v 74'8 (0'0,74'8] local-lis/les=72/73 n=2 ec=72/72 lis/c 72/72 les/c/f 73/73/0 72/72/72) [6,3] r=0 lpr=72 crt=74'8 lcod 74'7 mlcod 74'7 active+clean+snaptrim trimq=[2~2]] removing snap head 2021-02-10T11:36:37.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:37.973 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:36:37.973 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:36:38.308 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:36:38.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:36:38.309 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:36:38.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:36:38.309 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:36:38.643 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-02-10T11:36:39.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.052 7eff3dca6700 1 Processor -- start 2021-02-10T11:36:39.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.053 7eff3dca6700 1 -- start start 2021-02-10T11:36:39.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.053 7eff3dca6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7eff380d0340 0x7eff380ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:39.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.053 7eff3dca6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7eff380d1c70 0x7eff380efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:39.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.053 7eff3dca6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff380f0b10 con 0x7eff380d1870 2021-02-10T11:36:39.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.053 7eff3dca6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff380e5ed0 con 0x7eff380d1c70 2021-02-10T11:36:39.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.053 7eff3dca6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff381b0be0 con 0x7eff380d0340 2021-02-10T11:36:39.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.053 7eff37fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7eff380d1c70 0x7eff380efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:39.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.053 7eff37fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7eff380d1c70 0x7eff380efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54092/0 (socket says 172.21.2.101:54092) 2021-02-10T11:36:39.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.053 7eff37fff700 1 -- 172.21.2.101:0/2468831449 learned_addr learned my addr 172.21.2.101:0/2468831449 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:39.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.053 7eff37fff700 1 -- 172.21.2.101:0/2468831449 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7eff380d0340 msgr2=0x7eff380ef640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:36:39.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.053 7eff37fff700 1 --2- 172.21.2.101:0/2468831449 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7eff380d0340 0x7eff380ef640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:39.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.053 7eff37fff700 1 -- 172.21.2.101:0/2468831449 >> v1:172.21.2.108:6789/0 conn(0x7eff380d1870 legacy=0x7eff381b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:36:39.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.053 7eff37fff700 1 -- 172.21.2.101:0/2468831449 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff380f20a0 con 0x7eff380d1c70 2021-02-10T11:36:39.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.054 7eff37fff700 1 --2- 172.21.2.101:0/2468831449 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff380d1c70 0x7eff380efc60 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 rx=0x7eff30006f00 tx=0x7eff30009f10).ready entity=mon.0 client_cookie=92f0ba4a28ec19a8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:39.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.054 7eff367fc700 1 -- 172.21.2.101:0/2468831449 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7eff3000b3b0 con 0x7eff380d1c70 2021-02-10T11:36:39.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.054 7eff367fc700 1 -- 172.21.2.101:0/2468831449 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3000b730 con 0x7eff380d1c70 2021-02-10T11:36:39.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.054 7eff367fc700 1 -- 172.21.2.101:0/2468831449 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7eff3000bc50 con 0x7eff380d1c70 2021-02-10T11:36:39.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.054 7eff3dca6700 1 -- 172.21.2.101:0/2468831449 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff380d1c70 msgr2=0x7eff380efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:39.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.054 7eff3dca6700 1 --2- 172.21.2.101:0/2468831449 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff380d1c70 0x7eff380efc60 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 rx=0x7eff30006f00 tx=0x7eff30009f10).stop 2021-02-10T11:36:39.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.055 7eff3dca6700 1 -- 172.21.2.101:0/2468831449 shutdown_connections 2021-02-10T11:36:39.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.055 7eff3dca6700 1 --2- 172.21.2.101:0/2468831449 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7eff380d0340 0x7eff380ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:39.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.055 7eff3dca6700 1 --2- 172.21.2.101:0/2468831449 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff380d1c70 0x7eff380efc60 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:39.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.055 7eff3dca6700 1 -- 172.21.2.101:0/2468831449 >> 172.21.2.101:0/2468831449 conn(0x7eff380ce450 msgr2=0x7eff380d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:39.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.055 7eff3dca6700 1 -- 172.21.2.101:0/2468831449 shutdown_connections 2021-02-10T11:36:39.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.055 7eff3dca6700 1 -- 172.21.2.101:0/2468831449 wait complete. 2021-02-10T11:36:39.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.069 7eff3dca6700 1 Processor -- start 2021-02-10T11:36:39.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.069 7eff3dca6700 1 -- start start 2021-02-10T11:36:39.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.070 7eff3dca6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff381881a0 0x7eff380ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:39.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.070 7eff3dca6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7eff380ae660 0x7eff38188de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:39.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.070 7eff3dca6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7eff38189400 0x7eff38189800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:39.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.070 7eff3dca6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff380be9f0 con 0x7eff381881a0 2021-02-10T11:36:39.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.070 7eff3dca6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff38188860 con 0x7eff380ae660 2021-02-10T11:36:39.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.070 7eff3dca6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff381885a0 con 0x7eff38189400 2021-02-10T11:36:39.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.070 7eff377fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff381881a0 0x7eff380ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:39.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.070 7eff377fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff381881a0 0x7eff380ae040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54098/0 (socket says 172.21.2.101:54098) 2021-02-10T11:36:39.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.070 7eff377fe700 1 -- 172.21.2.101:0/2990028185 learned_addr learned my addr 172.21.2.101:0/2990028185 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:39.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.070 7eff37fff700 1 --2- 172.21.2.101:0/2990028185 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7eff38189400 0x7eff38189800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:39.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.070 7eff377fe700 1 -- 172.21.2.101:0/2990028185 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7eff38189400 msgr2=0x7eff38189800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:39.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.070 7eff377fe700 1 --2- 172.21.2.101:0/2990028185 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7eff38189400 0x7eff38189800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:39.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.070 7eff377fe700 1 -- 172.21.2.101:0/2990028185 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7eff380ae660 msgr2=0x7eff38188de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:39.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.070 7eff377fe700 1 --2- 172.21.2.101:0/2990028185 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7eff380ae660 0x7eff38188de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:39.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.071 7eff377fe700 1 -- 172.21.2.101:0/2990028185 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff380aea60 con 0x7eff381881a0 2021-02-10T11:36:39.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.071 7eff377fe700 1 --2- 172.21.2.101:0/2990028185 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff381881a0 0x7eff380ae040 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 rx=0x7eff2000b5b0 tx=0x7eff2000b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:39.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.071 7eff2ffff700 1 -- 172.21.2.101:0/2990028185 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7eff2000bd40 con 0x7eff381881a0 2021-02-10T11:36:39.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.071 7eff2ffff700 1 -- 172.21.2.101:0/2990028185 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff20000c60 con 0x7eff381881a0 2021-02-10T11:36:39.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.071 7eff2ffff700 1 -- 172.21.2.101:0/2990028185 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7eff20006210 con 0x7eff381881a0 2021-02-10T11:36:39.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.071 7eff3dca6700 1 -- 172.21.2.101:0/2990028185 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff380aea60 con 0x7eff381881a0 2021-02-10T11:36:39.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.071 7eff3dca6700 1 -- 172.21.2.101:0/2990028185 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff380aea60 con 0x7eff381881a0 2021-02-10T11:36:39.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.072 7eff2ffff700 1 -- 172.21.2.101:0/2990028185 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7eff20006020 con 0x7eff381881a0 2021-02-10T11:36:39.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.072 7eff2ffff700 1 --2- 172.21.2.101:0/2990028185 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7eff1c0186c0 0x7eff1c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:39.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.072 7eff2ffff700 1 -- 172.21.2.101:0/2990028185 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5645+0+0 (secure 0 0 0) 0x7eff20032810 con 0x7eff381881a0 2021-02-10T11:36:39.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.072 7eff36ffd700 1 --2- 172.21.2.101:0/2990028185 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7eff1c0186c0 0x7eff1c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:39.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.072 7eff36ffd700 1 --2- 172.21.2.101:0/2990028185 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7eff1c0186c0 0x7eff1c01abc0 crc :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:39.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.103 7eff3dca6700 1 -- 172.21.2.101:0/2990028185 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff380ebc50 con 0x7eff381881a0 2021-02-10T11:36:39.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.106 7eff2ffff700 1 -- 172.21.2.101:0/2990028185 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7eff2001f510 con 0x7eff381881a0 2021-02-10T11:36:39.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.256 7eff3dca6700 1 -- 172.21.2.101:0/2990028185 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7eff380f0b10 con 0x7eff381881a0 2021-02-10T11:36:39.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.260 7eff2ffff700 1 -- 172.21.2.101:0/2990028185 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7eff20006020 con 0x7eff381881a0 2021-02-10T11:36:39.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.265 7eff3dca6700 1 -- 172.21.2.101:0/2990028185 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7eff1c0186c0 msgr2=0x7eff1c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:39.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.265 7eff3dca6700 1 --2- 172.21.2.101:0/2990028185 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7eff1c0186c0 0x7eff1c01abc0 crc :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:39.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.265 7eff3dca6700 1 -- 172.21.2.101:0/2990028185 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff381881a0 msgr2=0x7eff380ae040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:39.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.265 7eff3dca6700 1 --2- 172.21.2.101:0/2990028185 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff381881a0 0x7eff380ae040 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 rx=0x7eff2000b5b0 tx=0x7eff2000b950).stop 2021-02-10T11:36:39.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.265 7eff3dca6700 1 -- 172.21.2.101:0/2990028185 shutdown_connections 2021-02-10T11:36:39.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.265 7eff3dca6700 1 --2- 172.21.2.101:0/2990028185 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7eff1c0186c0 0x7eff1c01abc0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:39.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.265 7eff3dca6700 1 --2- 172.21.2.101:0/2990028185 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7eff38189400 0x7eff38189800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:39.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.265 7eff3dca6700 1 --2- 172.21.2.101:0/2990028185 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7eff380ae660 0x7eff38188de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:39.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.265 7eff3dca6700 1 --2- 172.21.2.101:0/2990028185 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff381881a0 0x7eff380ae040 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:39.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.265 7eff3dca6700 1 -- 172.21.2.101:0/2990028185 >> 172.21.2.101:0/2990028185 conn(0x7eff380b8ae0 msgr2=0x7eff38095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:39.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.265 7eff3dca6700 1 -- 172.21.2.101:0/2990028185 shutdown_connections 2021-02-10T11:36:39.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.266 7eff3dca6700 1 -- 172.21.2.101:0/2990028185 wait complete. 2021-02-10T11:36:39.279 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_clone_subvolume_info' 2021-02-10T11:36:39.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.683 7f4ea9cf7700 1 Processor -- start 2021-02-10T11:36:39.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.683 7f4ea9cf7700 1 -- start start 2021-02-10T11:36:39.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.683 7f4ea9cf7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4ea40d0340 0x7f4ea40ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:39.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.683 7f4ea9cf7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea40d1c70 0x7f4ea40efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:39.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.683 7f4ea9cf7700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea40f0b10 con 0x7f4ea40d1870 2021-02-10T11:36:39.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.683 7f4ea9cf7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea40e5ed0 con 0x7f4ea40d0340 2021-02-10T11:36:39.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.683 7f4ea9cf7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea41b0be0 con 0x7f4ea40d1c70 2021-02-10T11:36:39.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.684 7f4ea3fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea40d1c70 0x7f4ea40efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:39.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.684 7f4ea2ffd700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4ea40d0340 0x7f4ea40ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:39.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.684 7f4ea37fe700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f4ea40d1870 0x7f4ea41b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:48008/0 (socket says 172.21.2.101:48008) 2021-02-10T11:36:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.684 7f4ea37fe700 1 -- 172.21.2.101:0/815944789 learned_addr learned my addr 172.21.2.101:0/815944789 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.685 7f4ea3fff700 1 -- 172.21.2.101:0/815944789 >> v2:172.21.2.108:3300/0 conn(0x7f4ea40d0340 msgr2=0x7f4ea40ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.685 7f4ea3fff700 1 --2- 172.21.2.101:0/815944789 >> v2:172.21.2.108:3300/0 conn(0x7f4ea40d0340 0x7f4ea40ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.685 7f4ea3fff700 1 -- 172.21.2.101:0/815944789 >> v1:172.21.2.108:6789/0 conn(0x7f4ea40d1870 legacy=0x7f4ea41b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.685 7f4ea3fff700 1 -- 172.21.2.101:0/815944789 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea40f20a0 con 0x7f4ea40d1c70 2021-02-10T11:36:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.685 7f4ea27fc700 1 -- 172.21.2.101:0/815944789 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2195511252 0 0) 0x7f4e8c001190 con 0x7f4ea40d1870 2021-02-10T11:36:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.685 7f4ea2ffd700 1 --2- 172.21.2.101:0/815944789 >> v2:172.21.2.108:3300/0 conn(0x7f4ea40d0340 0x7f4ea40ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:36:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.686 7f4ea3fff700 1 --2- 172.21.2.101:0/815944789 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea40d1c70 0x7f4ea40efc60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f4e9c006e50 tx=0x7f4e9c009e70).ready entity=mon.2 client_cookie=a5fdc77c5872d54c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.686 7f4ea27fc700 1 -- 172.21.2.101:0/815944789 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e9c006a20 con 0x7f4ea40d1c70 2021-02-10T11:36:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.686 7f4ea27fc700 1 -- 172.21.2.101:0/815944789 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c006790 con 0x7f4ea40d1c70 2021-02-10T11:36:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.686 7f4ea27fc700 1 -- 172.21.2.101:0/815944789 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e9c00b700 con 0x7f4ea40d1c70 2021-02-10T11:36:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.686 7f4ea9cf7700 1 -- 172.21.2.101:0/815944789 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea40d1c70 msgr2=0x7f4ea40efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.686 7f4ea9cf7700 1 --2- 172.21.2.101:0/815944789 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea40d1c70 0x7f4ea40efc60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f4e9c006e50 tx=0x7f4e9c009e70).stop 2021-02-10T11:36:39.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.687 7f4ea9cf7700 1 -- 172.21.2.101:0/815944789 shutdown_connections 2021-02-10T11:36:39.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.687 7f4ea9cf7700 1 --2- 172.21.2.101:0/815944789 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea40d1c70 0x7f4ea40efc60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:39.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.687 7f4ea9cf7700 1 --2- 172.21.2.101:0/815944789 >> v2:172.21.2.108:3300/0 conn(0x7f4ea40d0340 0x7f4ea40ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:39.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.687 7f4ea9cf7700 1 -- 172.21.2.101:0/815944789 >> 172.21.2.101:0/815944789 conn(0x7f4ea40ce450 msgr2=0x7f4ea40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:39.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.687 7f4ea9cf7700 1 -- 172.21.2.101:0/815944789 shutdown_connections 2021-02-10T11:36:39.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.687 7f4ea9cf7700 1 -- 172.21.2.101:0/815944789 wait complete. 2021-02-10T11:36:39.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.702 7f4ea9cf7700 1 Processor -- start 2021-02-10T11:36:39.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.702 7f4ea9cf7700 1 -- start start 2021-02-10T11:36:39.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.702 7f4ea9cf7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ea4195650 0x7f4ea40b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:39.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.702 7f4ea9cf7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea40b5b80 0x7f4ea41a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:39.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.702 7f4ea9cf7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ea40acfa0 0x7f4ea41a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:39.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.702 7f4ea9cf7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea40b6010 con 0x7f4ea40acfa0 2021-02-10T11:36:39.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.702 7f4ea9cf7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea41a3a60 con 0x7f4ea4195650 2021-02-10T11:36:39.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.702 7f4ea9cf7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea41a3c40 con 0x7f4ea40b5b80 2021-02-10T11:36:39.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.702 7f4ea3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ea40acfa0 0x7f4ea41a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:39.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.702 7f4ea3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ea40acfa0 0x7f4ea41a3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54108/0 (socket says 172.21.2.101:54108) 2021-02-10T11:36:39.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.702 7f4ea3fff700 1 -- 172.21.2.101:0/1448462183 learned_addr learned my addr 172.21.2.101:0/1448462183 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:39.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.702 7f4ea3fff700 1 -- 172.21.2.101:0/1448462183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea40b5b80 msgr2=0x7f4ea41a2e20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:39.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.702 7f4ea3fff700 1 --2- 172.21.2.101:0/1448462183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea40b5b80 0x7f4ea41a2e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:39.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.702 7f4ea3fff700 1 -- 172.21.2.101:0/1448462183 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ea4195650 msgr2=0x7f4ea40b5560 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:39.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.702 7f4ea3fff700 1 --2- 172.21.2.101:0/1448462183 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ea4195650 0x7f4ea40b5560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:39.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.702 7f4ea3fff700 1 -- 172.21.2.101:0/1448462183 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea40bcae0 con 0x7f4ea40acfa0 2021-02-10T11:36:39.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.703 7f4ea3fff700 1 --2- 172.21.2.101:0/1448462183 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ea40acfa0 0x7f4ea41a3440 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 rx=0x7f4e9c00b960 tx=0x7f4e9c00bd00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:39.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.703 7f4e9bfff700 1 -- 172.21.2.101:0/1448462183 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e9c0093d0 con 0x7f4ea40acfa0 2021-02-10T11:36:39.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.703 7f4e9bfff700 1 -- 172.21.2.101:0/1448462183 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c009710 con 0x7f4ea40acfa0 2021-02-10T11:36:39.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.703 7f4e9bfff700 1 -- 172.21.2.101:0/1448462183 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e9c0022a0 con 0x7f4ea40acfa0 2021-02-10T11:36:39.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.703 7f4ea9cf7700 1 -- 172.21.2.101:0/1448462183 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea40bcae0 con 0x7f4ea40acfa0 2021-02-10T11:36:39.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.703 7f4ea9cf7700 1 -- 172.21.2.101:0/1448462183 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea40bd070 con 0x7f4ea40acfa0 2021-02-10T11:36:39.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.703 7f4e9bfff700 1 -- 172.21.2.101:0/1448462183 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4e9c0056b0 con 0x7f4ea40acfa0 2021-02-10T11:36:39.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.704 7f4e9bfff700 1 --2- 172.21.2.101:0/1448462183 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4e880186c0 0x7f4e8801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:39.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.704 7f4e9bfff700 1 -- 172.21.2.101:0/1448462183 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5645+0+0 (secure 0 0 0) 0x7f4e9c032910 con 0x7f4ea40acfa0 2021-02-10T11:36:39.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.704 7f4ea37fe700 1 --2- 172.21.2.101:0/1448462183 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4e880186c0 0x7f4e8801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:39.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.704 7f4ea37fe700 1 --2- 172.21.2.101:0/1448462183 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4e880186c0 0x7f4e8801abc0 crc :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:39.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.733 7f4ea9cf7700 1 -- 172.21.2.101:0/1448462183 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea40f0ab0 con 0x7f4ea40acfa0 2021-02-10T11:36:39.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.736 7f4e9bfff700 1 -- 172.21.2.101:0/1448462183 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4e9c0263a0 con 0x7f4ea40acfa0 2021-02-10T11:36:39.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:39.869 7f4ea9cf7700 1 -- 172.21.2.101:0/1448462183 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_clone_subvolume_info"]} v 0) v1 -- 0x7f4ea40f0ab0 con 0x7f4ea40acfa0 2021-02-10T11:36:40.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.516 7f4e9bfff700 1 -- 172.21.2.101:0/1448462183 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_clone_subvolume_info"]}]=0 v420) v1 ==== 142+0+0 (secure 0 0 0) 0x7f4e9c0263a0 con 0x7f4ea40acfa0 2021-02-10T11:36:40.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.539 7f4ea9cf7700 1 -- 172.21.2.101:0/1448462183 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4e880186c0 msgr2=0x7f4e8801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:40.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.539 7f4ea9cf7700 1 --2- 172.21.2.101:0/1448462183 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4e880186c0 0x7f4e8801abc0 crc :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:40.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.539 7f4ea9cf7700 1 -- 172.21.2.101:0/1448462183 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ea40acfa0 msgr2=0x7f4ea41a3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:40.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.539 7f4ea9cf7700 1 --2- 172.21.2.101:0/1448462183 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ea40acfa0 0x7f4ea41a3440 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 rx=0x7f4e9c00b960 tx=0x7f4e9c00bd00).stop 2021-02-10T11:36:40.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.539 7f4ea9cf7700 1 -- 172.21.2.101:0/1448462183 shutdown_connections 2021-02-10T11:36:40.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.539 7f4ea9cf7700 1 --2- 172.21.2.101:0/1448462183 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4e880186c0 0x7f4e8801abc0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:40.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.539 7f4ea9cf7700 1 --2- 172.21.2.101:0/1448462183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea40b5b80 0x7f4ea41a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:40.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.539 7f4ea9cf7700 1 --2- 172.21.2.101:0/1448462183 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ea40acfa0 0x7f4ea41a3440 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:40.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.539 7f4ea9cf7700 1 --2- 172.21.2.101:0/1448462183 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ea4195650 0x7f4ea40b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:40.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.539 7f4ea9cf7700 1 -- 172.21.2.101:0/1448462183 >> 172.21.2.101:0/1448462183 conn(0x7f4ea40b3040 msgr2=0x7f4ea40c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:40.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.539 7f4ea9cf7700 1 -- 172.21.2.101:0/1448462183 shutdown_connections 2021-02-10T11:36:40.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.539 7f4ea9cf7700 1 -- 172.21.2.101:0/1448462183 wait complete. 2021-02-10T11:36:40.553 INFO:tasks.cephfs_test_runner:test_clone_subvolume_info (tasks.cephfs.test_volumes.TestVolumes) ... ok 2021-02-10T11:36:40.553 INFO:tasks.cephfs_test_runner:Starting test: test_connection_expiration (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:36:40.554 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_connection_expiration' 2021-02-10T11:36:40.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.958 7f699c08c700 1 Processor -- start 2021-02-10T11:36:40.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.958 7f699c08c700 1 -- start start 2021-02-10T11:36:40.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.958 7f699c08c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f69940d0340 0x7f69940ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:40.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.958 7f699c08c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69940f0b10 con 0x7f69940d1c70 2021-02-10T11:36:40.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.958 7f699c08c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69941b0be0 con 0x7f69940d1870 2021-02-10T11:36:40.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.958 7f699c08c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69940e5ed0 con 0x7f69940d0340 2021-02-10T11:36:40.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.959 7f69996a7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f69940d0340 0x7f69940ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:40.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.959 7f69996a7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f69940d0340 0x7f69940ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54116/0 (socket says 172.21.2.101:54116) 2021-02-10T11:36:40.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.959 7f69996a7700 1 -- 172.21.2.101:0/773035566 learned_addr learned my addr 172.21.2.101:0/773035566 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:40.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.959 7f69996a7700 1 -- 172.21.2.101:0/773035566 >> v1:172.21.2.108:6789/0 conn(0x7f69940d1870 legacy=0x7f69941b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:36:40.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.959 7f69996a7700 1 -- 172.21.2.101:0/773035566 >> v1:172.21.2.101:6789/0 conn(0x7f69940d1c70 legacy=0x7f69940efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:40.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.959 7f69996a7700 1 -- 172.21.2.101:0/773035566 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69940f0450 con 0x7f69940d0340 2021-02-10T11:36:40.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.959 7f69996a7700 1 --2- 172.21.2.101:0/773035566 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f69940d0340 0x7f69940ef640 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 rx=0x7f6988006700 tx=0x7f6988009b00).ready entity=mon.0 client_cookie=a81b9a1cb7d9db5d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:40.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.959 7f6998ea6700 1 -- 172.21.2.101:0/773035566 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f698800c0c0 con 0x7f69940d0340 2021-02-10T11:36:40.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.959 7f6998ea6700 1 -- 172.21.2.101:0/773035566 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698800c400 con 0x7f69940d0340 2021-02-10T11:36:40.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.959 7f6998ea6700 1 -- 172.21.2.101:0/773035566 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f698800c920 con 0x7f69940d0340 2021-02-10T11:36:40.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.959 7f699c08c700 1 -- 172.21.2.101:0/773035566 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f69940d0340 msgr2=0x7f69940ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:40.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.959 7f699c08c700 1 --2- 172.21.2.101:0/773035566 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f69940d0340 0x7f69940ef640 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 rx=0x7f6988006700 tx=0x7f6988009b00).stop 2021-02-10T11:36:40.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.960 7f699c08c700 1 -- 172.21.2.101:0/773035566 shutdown_connections 2021-02-10T11:36:40.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.960 7f699c08c700 1 --2- 172.21.2.101:0/773035566 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f69940d0340 0x7f69940ef640 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:40.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.960 7f699c08c700 1 -- 172.21.2.101:0/773035566 >> 172.21.2.101:0/773035566 conn(0x7f69940ce450 msgr2=0x7f69940d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:40.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.960 7f699c08c700 1 -- 172.21.2.101:0/773035566 shutdown_connections 2021-02-10T11:36:40.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.960 7f699c08c700 1 -- 172.21.2.101:0/773035566 wait complete. 2021-02-10T11:36:40.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.975 7f699c08c700 1 Processor -- start 2021-02-10T11:36:40.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.975 7f699c08c700 1 -- start start 2021-02-10T11:36:40.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.975 7f699c08c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f69940c0060 0x7f69940c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:40.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.975 7f699c08c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f69941c69b0 0x7f69940c3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:40.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.975 7f699c08c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f69940c3830 0x7f69940b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:40.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.975 7f699c08c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69940b33f0 con 0x7f69941c69b0 2021-02-10T11:36:40.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.975 7f699c08c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69940c3c30 con 0x7f69940c0060 2021-02-10T11:36:40.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.975 7f699c08c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69940bfbc0 con 0x7f69940c3830 2021-02-10T11:36:40.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.976 7f69996a7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f69941c69b0 0x7f69940c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:40.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.976 7f69996a7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f69941c69b0 0x7f69940c3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54122/0 (socket says 172.21.2.101:54122) 2021-02-10T11:36:40.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.976 7f69996a7700 1 -- 172.21.2.101:0/772227942 learned_addr learned my addr 172.21.2.101:0/772227942 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:40.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.976 7f69996a7700 1 -- 172.21.2.101:0/772227942 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f69940c3830 msgr2=0x7f69940b3970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:40.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.976 7f69996a7700 1 --2- 172.21.2.101:0/772227942 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f69940c3830 0x7f69940b3970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:40.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.976 7f69996a7700 1 -- 172.21.2.101:0/772227942 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f69940c0060 msgr2=0x7f69940c2bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:40.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.976 7f69996a7700 1 --2- 172.21.2.101:0/772227942 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f69940c0060 0x7f69940c2bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:40.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.976 7f69996a7700 1 -- 172.21.2.101:0/772227942 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69940b3f90 con 0x7f69941c69b0 2021-02-10T11:36:40.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.976 7f69996a7700 1 --2- 172.21.2.101:0/772227942 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f69941c69b0 0x7f69940c3210 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 rx=0x7f6988006f20 tx=0x7f698800b9e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:40.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.976 7f69867fc700 1 -- 172.21.2.101:0/772227942 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f698800bdd0 con 0x7f69941c69b0 2021-02-10T11:36:40.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.976 7f69867fc700 1 -- 172.21.2.101:0/772227942 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6988006440 con 0x7f69941c69b0 2021-02-10T11:36:40.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.976 7f69867fc700 1 -- 172.21.2.101:0/772227942 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f69880052e0 con 0x7f69941c69b0 2021-02-10T11:36:40.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.976 7f699c08c700 1 -- 172.21.2.101:0/772227942 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69940b3f90 con 0x7f69941c69b0 2021-02-10T11:36:40.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.976 7f699c08c700 1 -- 172.21.2.101:0/772227942 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69940b3f90 con 0x7f69941c69b0 2021-02-10T11:36:40.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.977 7f69867fc700 1 -- 172.21.2.101:0/772227942 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6988006030 con 0x7f69941c69b0 2021-02-10T11:36:40.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.977 7f69867fc700 1 --2- 172.21.2.101:0/772227942 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f697c0186c0 0x7f697c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:40.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.977 7f69867fc700 1 -- 172.21.2.101:0/772227942 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5645+0+0 (secure 0 0 0) 0x7f6988032810 con 0x7f69941c69b0 2021-02-10T11:36:40.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.977 7f6999ea8700 1 --2- 172.21.2.101:0/772227942 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f697c0186c0 0x7f697c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:40.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:40.977 7f6999ea8700 1 --2- 172.21.2.101:0/772227942 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f697c0186c0 0x7f697c01abc0 crc :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:41.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.009 7f699c08c700 1 -- 172.21.2.101:0/772227942 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69940f06f0 con 0x7f69941c69b0 2021-02-10T11:36:41.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.011 7f69867fc700 1 -- 172.21.2.101:0/772227942 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6988006030 con 0x7f69941c69b0 2021-02-10T11:36:41.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.142 7f699c08c700 1 -- 172.21.2.101:0/772227942 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_connection_expiration"]} v 0) v1 -- 0x7f69940f06f0 con 0x7f69941c69b0 2021-02-10T11:36:41.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.517 7f69867fc700 1 -- 172.21.2.101:0/772227942 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_connection_expiration"]}]=0 v421) v1 ==== 146+0+0 (secure 0 0 0) 0x7f6988006440 con 0x7f69941c69b0 2021-02-10T11:36:41.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.561 7f699c08c700 1 -- 172.21.2.101:0/772227942 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f697c0186c0 msgr2=0x7f697c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:41.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.561 7f699c08c700 1 --2- 172.21.2.101:0/772227942 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f697c0186c0 0x7f697c01abc0 crc :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:41.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.561 7f699c08c700 1 -- 172.21.2.101:0/772227942 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f69941c69b0 msgr2=0x7f69940c3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:41.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.561 7f699c08c700 1 --2- 172.21.2.101:0/772227942 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f69941c69b0 0x7f69940c3210 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 rx=0x7f6988006f20 tx=0x7f698800b9e0).stop 2021-02-10T11:36:41.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.561 7f699c08c700 1 -- 172.21.2.101:0/772227942 shutdown_connections 2021-02-10T11:36:41.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.561 7f699c08c700 1 --2- 172.21.2.101:0/772227942 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f697c0186c0 0x7f697c01abc0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:41.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.561 7f699c08c700 1 --2- 172.21.2.101:0/772227942 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f69940c3830 0x7f69940b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:41.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.561 7f699c08c700 1 --2- 172.21.2.101:0/772227942 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f69941c69b0 0x7f69940c3210 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:41.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.561 7f699c08c700 1 --2- 172.21.2.101:0/772227942 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f69940c0060 0x7f69940c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:41.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.561 7f699c08c700 1 -- 172.21.2.101:0/772227942 >> 172.21.2.101:0/772227942 conn(0x7f699410a820 msgr2=0x7f6994108150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:41.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.561 7f699c08c700 1 -- 172.21.2.101:0/772227942 shutdown_connections 2021-02-10T11:36:41.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.561 7f699c08c700 1 -- 172.21.2.101:0/772227942 wait complete. 2021-02-10T11:36:41.576 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:36:42.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.985 7ff033aa2700 1 Processor -- start 2021-02-10T11:36:42.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.985 7ff033aa2700 1 -- start start 2021-02-10T11:36:42.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.985 7ff033aa2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff02c0d1860 0x7ff02c1b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:42.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.985 7ff033aa2700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff02c0fe1b0 con 0x7ff02c0cff60 2021-02-10T11:36:42.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.985 7ff033aa2700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff02c109a50 con 0x7ff02c1b0ad0 2021-02-10T11:36:42.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.985 7ff033aa2700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff02c088070 con 0x7ff02c0d1860 2021-02-10T11:36:42.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.986 7ff0318be700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff02c0d1860 0x7ff02c1b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:42.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.986 7ff0310bd700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7ff02c1b0ad0 0x7ff02c108930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:48030/0 (socket says 172.21.2.101:48030) 2021-02-10T11:36:42.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.986 7ff0318be700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff02c0d1860 0x7ff02c1b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54126/0 (socket says 172.21.2.101:54126) 2021-02-10T11:36:42.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.986 7ff0320bf700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff02c0cff60 0x7ff02c109120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54620/0 (socket says 172.21.2.101:54620) 2021-02-10T11:36:42.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.986 7ff0310bd700 1 -- 172.21.2.101:0/69170375 learned_addr learned my addr 172.21.2.101:0/69170375 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:42.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.986 7ff0308bc700 1 -- 172.21.2.101:0/69170375 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812494922 0 0) 0x7ff01c0009b0 con 0x7ff02c1b0ad0 2021-02-10T11:36:42.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.986 7ff0308bc700 1 -- 172.21.2.101:0/69170375 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff018002fe0 con 0x7ff02c1b0ad0 2021-02-10T11:36:42.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.986 7ff0318be700 1 -- 172.21.2.101:0/69170375 >> v1:172.21.2.108:6789/0 conn(0x7ff02c1b0ad0 legacy=0x7ff02c108930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:42.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.986 7ff0318be700 1 -- 172.21.2.101:0/69170375 >> v1:172.21.2.101:6789/0 conn(0x7ff02c0cff60 legacy=0x7ff02c109120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:42.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.986 7ff0318be700 1 -- 172.21.2.101:0/69170375 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02c088260 con 0x7ff02c0d1860 2021-02-10T11:36:42.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.987 7ff0318be700 1 --2- 172.21.2.101:0/69170375 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff02c0d1860 0x7ff02c1b04b0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 rx=0x7ff028006650 tx=0x7ff028009a80).ready entity=mon.0 client_cookie=53efc128f3272d8c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:42.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.987 7ff0308bc700 1 -- 172.21.2.101:0/69170375 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff02800c0c0 con 0x7ff02c0d1860 2021-02-10T11:36:42.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.987 7ff0308bc700 1 -- 172.21.2.101:0/69170375 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02800c340 con 0x7ff02c0d1860 2021-02-10T11:36:42.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.987 7ff0308bc700 1 -- 172.21.2.101:0/69170375 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff02800c860 con 0x7ff02c0d1860 2021-02-10T11:36:42.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.987 7ff033aa2700 1 -- 172.21.2.101:0/69170375 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff02c0d1860 msgr2=0x7ff02c1b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:42.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.987 7ff033aa2700 1 --2- 172.21.2.101:0/69170375 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff02c0d1860 0x7ff02c1b04b0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 rx=0x7ff028006650 tx=0x7ff028009a80).stop 2021-02-10T11:36:42.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.987 7ff033aa2700 1 -- 172.21.2.101:0/69170375 shutdown_connections 2021-02-10T11:36:42.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.987 7ff033aa2700 1 --2- 172.21.2.101:0/69170375 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff02c0d1860 0x7ff02c1b04b0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:42.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.987 7ff033aa2700 1 -- 172.21.2.101:0/69170375 >> 172.21.2.101:0/69170375 conn(0x7ff02c0ce410 msgr2=0x7ff02c0d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:42.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.988 7ff033aa2700 1 -- 172.21.2.101:0/69170375 shutdown_connections 2021-02-10T11:36:42.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:41.988 7ff033aa2700 1 -- 172.21.2.101:0/69170375 wait complete. 2021-02-10T11:36:42.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.002 7ff033aa2700 1 Processor -- start 2021-02-10T11:36:42.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.002 7ff033aa2700 1 -- start start 2021-02-10T11:36:42.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.003 7ff033aa2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff02c1a05e0 0x7ff02c18b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:42.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.003 7ff033aa2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff02c0b8780 0x7ff02c0bfa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:42.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.003 7ff033aa2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff02c0c0030 0x7ff02c189e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:42.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.003 7ff033aa2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c0b82a0 con 0x7ff02c1a05e0 2021-02-10T11:36:42.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.003 7ff033aa2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c0c0430 con 0x7ff02c0c0030 2021-02-10T11:36:42.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.003 7ff033aa2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c0c0650 con 0x7ff02c0b8780 2021-02-10T11:36:42.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.003 7ff0318be700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff02c1a05e0 0x7ff02c18b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:42.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.003 7ff0318be700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff02c1a05e0 0x7ff02c18b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54130/0 (socket says 172.21.2.101:54130) 2021-02-10T11:36:42.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.003 7ff0318be700 1 -- 172.21.2.101:0/3875367542 learned_addr learned my addr 172.21.2.101:0/3875367542 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:42.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.003 7ff0318be700 1 -- 172.21.2.101:0/3875367542 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff02c0b8780 msgr2=0x7ff02c0bfa10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:42.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.003 7ff0318be700 1 --2- 172.21.2.101:0/3875367542 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff02c0b8780 0x7ff02c0bfa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:42.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.003 7ff0318be700 1 -- 172.21.2.101:0/3875367542 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff02c0c0030 msgr2=0x7ff02c189e70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:42.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.003 7ff0318be700 1 --2- 172.21.2.101:0/3875367542 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff02c0c0030 0x7ff02c189e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:42.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.003 7ff0318be700 1 -- 172.21.2.101:0/3875367542 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02c18a490 con 0x7ff02c1a05e0 2021-02-10T11:36:42.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.003 7ff0318be700 1 --2- 172.21.2.101:0/3875367542 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff02c1a05e0 0x7ff02c18b970 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 rx=0x7ff028009630 tx=0x7ff02800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:42.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.003 7ff0227fc700 1 -- 172.21.2.101:0/3875367542 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0280093a0 con 0x7ff02c1a05e0 2021-02-10T11:36:42.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.004 7ff0227fc700 1 -- 172.21.2.101:0/3875367542 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02800bd70 con 0x7ff02c1a05e0 2021-02-10T11:36:42.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.004 7ff0227fc700 1 -- 172.21.2.101:0/3875367542 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0280054d0 con 0x7ff02c1a05e0 2021-02-10T11:36:42.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.004 7ff033aa2700 1 -- 172.21.2.101:0/3875367542 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff02c18a490 con 0x7ff02c1a05e0 2021-02-10T11:36:42.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.004 7ff033aa2700 1 -- 172.21.2.101:0/3875367542 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff02c18a490 con 0x7ff02c1a05e0 2021-02-10T11:36:42.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.004 7ff0227fc700 1 -- 172.21.2.101:0/3875367542 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff02800a000 con 0x7ff02c1a05e0 2021-02-10T11:36:42.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.004 7ff0227fc700 1 --2- 172.21.2.101:0/3875367542 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff014018730 0x7ff01401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:42.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.004 7ff0227fc700 1 -- 172.21.2.101:0/3875367542 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5617+0+0 (secure 0 0 0) 0x7ff028032800 con 0x7ff02c1a05e0 2021-02-10T11:36:42.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.004 7ff0310bd700 1 --2- 172.21.2.101:0/3875367542 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff014018730 0x7ff01401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:42.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.005 7ff0310bd700 1 --2- 172.21.2.101:0/3875367542 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff014018730 0x7ff01401ac10 crc :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:42.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.036 7ff033aa2700 1 -- 172.21.2.101:0/3875367542 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff02c082a20 con 0x7ff02c1a05e0 2021-02-10T11:36:42.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.038 7ff0227fc700 1 -- 172.21.2.101:0/3875367542 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff02801f510 con 0x7ff02c1a05e0 2021-02-10T11:36:42.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.188 7ff033aa2700 1 -- 172.21.2.101:0/3875367542 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff02c082a20 con 0x7ff02c1a05e0 2021-02-10T11:36:42.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.191 7ff0227fc700 1 -- 172.21.2.101:0/3875367542 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff02800a000 con 0x7ff02c1a05e0 2021-02-10T11:36:42.202 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.196 7ff033aa2700 1 -- 172.21.2.101:0/3875367542 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff014018730 msgr2=0x7ff01401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:42.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.196 7ff033aa2700 1 --2- 172.21.2.101:0/3875367542 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff014018730 0x7ff01401ac10 crc :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:42.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.196 7ff033aa2700 1 -- 172.21.2.101:0/3875367542 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff02c1a05e0 msgr2=0x7ff02c18b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:42.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.196 7ff033aa2700 1 --2- 172.21.2.101:0/3875367542 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff02c1a05e0 0x7ff02c18b970 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 rx=0x7ff028009630 tx=0x7ff02800b980).stop 2021-02-10T11:36:42.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.196 7ff033aa2700 1 -- 172.21.2.101:0/3875367542 shutdown_connections 2021-02-10T11:36:42.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.196 7ff033aa2700 1 --2- 172.21.2.101:0/3875367542 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff014018730 0x7ff01401ac10 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:42.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.196 7ff033aa2700 1 --2- 172.21.2.101:0/3875367542 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff02c0b8780 0x7ff02c0bfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:42.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.196 7ff033aa2700 1 --2- 172.21.2.101:0/3875367542 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff02c0c0030 0x7ff02c189e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:42.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.196 7ff033aa2700 1 --2- 172.21.2.101:0/3875367542 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff02c1a05e0 0x7ff02c18b970 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:42.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.196 7ff033aa2700 1 -- 172.21.2.101:0/3875367542 >> 172.21.2.101:0/3875367542 conn(0x7ff02c0a6a80 msgr2=0x7ff02c0ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:42.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.197 7ff033aa2700 1 -- 172.21.2.101:0/3875367542 shutdown_connections 2021-02-10T11:36:42.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:42.197 7ff033aa2700 1 -- 172.21.2.101:0/3875367542 wait complete. 2021-02-10T11:36:42.211 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:36:42.212 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:36:42.557 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:36:42.903 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:36:42.903 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:42.904 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:36:42.904 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:36:43.240 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2021-02-10T11:36:43.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:36:43.241 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-02-10T11:36:43.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:43.241 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:36:43.241 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:36:43.575 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:36:43.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:36:43.576 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:36:43.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:43.576 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:36:43.576 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:36:43.910 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:36:43.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:36:43.911 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:36:43.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:36:43.911 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:36:43.911 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:36:44.244 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:36:44.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:36:44.244 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:36:44.245 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json-pretty 2021-02-10T11:36:44.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.649 7f7a954ad700 1 Processor -- start 2021-02-10T11:36:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.650 7f7a954ad700 1 -- start start 2021-02-10T11:36:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.650 7f7a954ad700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7a900d1870 0x7f7a901b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.650 7f7a954ad700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a901b0a10 0x7f7a900cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.650 7f7a954ad700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7a900d1c70 0x7f7a900cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.650 7f7a954ad700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a900fe200 con 0x7f7a900d1c70 2021-02-10T11:36:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.650 7f7a954ad700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a900e5f10 con 0x7f7a900d1870 2021-02-10T11:36:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.650 7f7a954ad700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a900f0b50 con 0x7f7a901b0a10 2021-02-10T11:36:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.650 7f7a8fac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7a900d1c70 0x7f7a900cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.650 7f7a8fac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7a900d1c70 0x7f7a900cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54138/0 (socket says 172.21.2.101:54138) 2021-02-10T11:36:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.650 7f7a8fac0700 1 -- 172.21.2.101:0/2109491031 learned_addr learned my addr 172.21.2.101:0/2109491031 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.650 7f7a8fac0700 1 -- 172.21.2.101:0/2109491031 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a901b0a10 msgr2=0x7f7a900cf330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:36:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.650 7f7a8fac0700 1 --2- 172.21.2.101:0/2109491031 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a901b0a10 0x7f7a900cf330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.650 7f7a8fac0700 1 -- 172.21.2.101:0/2109491031 >> v2:172.21.2.108:3300/0 conn(0x7f7a900d1870 msgr2=0x7f7a901b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:36:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.650 7f7a8fac0700 1 --2- 172.21.2.101:0/2109491031 >> v2:172.21.2.108:3300/0 conn(0x7f7a900d1870 0x7f7a901b03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.650 7f7a8fac0700 1 -- 172.21.2.101:0/2109491031 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a900cff70 con 0x7f7a900d1c70 2021-02-10T11:36:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.651 7f7a8fac0700 1 --2- 172.21.2.101:0/2109491031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a900d1c70 0x7f7a900cf950 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 rx=0x7f7a88006f00 tx=0x7f7a88009f10).ready entity=mon.0 client_cookie=4096f396ea49b5da server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.651 7f7a8e2bd700 1 -- 172.21.2.101:0/2109491031 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7a8800b3b0 con 0x7f7a900d1c70 2021-02-10T11:36:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.651 7f7a8e2bd700 1 -- 172.21.2.101:0/2109491031 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8800b630 con 0x7f7a900d1c70 2021-02-10T11:36:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.651 7f7a8e2bd700 1 -- 172.21.2.101:0/2109491031 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7a8800bb50 con 0x7f7a900d1c70 2021-02-10T11:36:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.651 7f7a954ad700 1 -- 172.21.2.101:0/2109491031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a900d1c70 msgr2=0x7f7a900cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.651 7f7a954ad700 1 --2- 172.21.2.101:0/2109491031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a900d1c70 0x7f7a900cf950 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 rx=0x7f7a88006f00 tx=0x7f7a88009f10).stop 2021-02-10T11:36:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.651 7f7a954ad700 1 -- 172.21.2.101:0/2109491031 shutdown_connections 2021-02-10T11:36:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.651 7f7a954ad700 1 --2- 172.21.2.101:0/2109491031 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a901b0a10 0x7f7a900cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.651 7f7a954ad700 1 --2- 172.21.2.101:0/2109491031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a900d1c70 0x7f7a900cf950 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.651 7f7a954ad700 1 --2- 172.21.2.101:0/2109491031 >> v2:172.21.2.108:3300/0 conn(0x7f7a900d1870 0x7f7a901b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.651 7f7a954ad700 1 -- 172.21.2.101:0/2109491031 >> 172.21.2.101:0/2109491031 conn(0x7f7a900ce450 msgr2=0x7f7a900d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:44.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.651 7f7a954ad700 1 -- 172.21.2.101:0/2109491031 shutdown_connections 2021-02-10T11:36:44.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.651 7f7a954ad700 1 -- 172.21.2.101:0/2109491031 wait complete. 2021-02-10T11:36:44.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.666 7f7a954ad700 1 Processor -- start 2021-02-10T11:36:44.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.666 7f7a954ad700 1 -- start start 2021-02-10T11:36:44.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.666 7f7a954ad700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a9018e930 0x7f7a900aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:44.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.666 7f7a954ad700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a900aaaf0 0x7f7a90093700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:44.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.666 7f7a954ad700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a9018ed30 0x7f7a90093d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:44.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.666 7f7a954ad700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a900944e0 con 0x7f7a900aaaf0 2021-02-10T11:36:44.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.666 7f7a954ad700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9008a590 con 0x7f7a9018ed30 2021-02-10T11:36:44.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.666 7f7a954ad700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9008a770 con 0x7f7a9018e930 2021-02-10T11:36:44.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.666 7f7a8eabe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a900aaaf0 0x7f7a90093700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:44.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.666 7f7a8eabe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a900aaaf0 0x7f7a90093700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54144/0 (socket says 172.21.2.101:54144) 2021-02-10T11:36:44.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.666 7f7a8eabe700 1 -- 172.21.2.101:0/1744203397 learned_addr learned my addr 172.21.2.101:0/1744203397 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:44.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.667 7f7a8eabe700 1 -- 172.21.2.101:0/1744203397 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a9018e930 msgr2=0x7f7a900aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:44.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.667 7f7a8eabe700 1 --2- 172.21.2.101:0/1744203397 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a9018e930 0x7f7a900aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:44.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.667 7f7a8eabe700 1 -- 172.21.2.101:0/1744203397 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a9018ed30 msgr2=0x7f7a90093d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:44.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.667 7f7a8eabe700 1 --2- 172.21.2.101:0/1744203397 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a9018ed30 0x7f7a90093d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:44.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.667 7f7a8eabe700 1 -- 172.21.2.101:0/1744203397 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9008a990 con 0x7f7a900aaaf0 2021-02-10T11:36:44.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.667 7f7a8eabe700 1 --2- 172.21.2.101:0/1744203397 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a900aaaf0 0x7f7a90093700 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 rx=0x7f7a8000b5c0 tx=0x7f7a8000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:44.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.667 7f7a87fff700 1 -- 172.21.2.101:0/1744203397 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7a8000bd50 con 0x7f7a900aaaf0 2021-02-10T11:36:44.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.667 7f7a87fff700 1 -- 172.21.2.101:0/1744203397 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a80006030 con 0x7f7a900aaaf0 2021-02-10T11:36:44.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.667 7f7a87fff700 1 -- 172.21.2.101:0/1744203397 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7a800054d0 con 0x7f7a900aaaf0 2021-02-10T11:36:44.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.667 7f7a954ad700 1 -- 172.21.2.101:0/1744203397 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a9008a990 con 0x7f7a900aaaf0 2021-02-10T11:36:44.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.667 7f7a954ad700 1 -- 172.21.2.101:0/1744203397 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a9008af20 con 0x7f7a900aaaf0 2021-02-10T11:36:44.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.668 7f7a87fff700 1 -- 172.21.2.101:0/1744203397 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7a80006210 con 0x7f7a900aaaf0 2021-02-10T11:36:44.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.668 7f7a87fff700 1 --2- 172.21.2.101:0/1744203397 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a740186c0 0x7f7a7401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:44.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.668 7f7a87fff700 1 -- 172.21.2.101:0/1744203397 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5617+0+0 (secure 0 0 0) 0x7f7a800325d0 con 0x7f7a900aaaf0 2021-02-10T11:36:44.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.668 7f7a8f2bf700 1 --2- 172.21.2.101:0/1744203397 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a740186c0 0x7f7a7401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:44.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.669 7f7a8f2bf700 1 --2- 172.21.2.101:0/1744203397 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a740186c0 0x7f7a7401abc0 crc :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:44.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.700 7f7a954ad700 1 -- 172.21.2.101:0/1744203397 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a900ee060 con 0x7f7a900aaaf0 2021-02-10T11:36:44.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.703 7f7a87fff700 1 -- 172.21.2.101:0/1744203397 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7a80000940 con 0x7f7a900aaaf0 2021-02-10T11:36:44.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.828 7f7a954ad700 1 -- 172.21.2.101:0/1744203397 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f7a900ce450 con 0x7f7a900aaaf0 2021-02-10T11:36:44.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.829 7f7a87fff700 1 -- 172.21.2.101:0/1744203397 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v76) v1 ==== 81+0+30574 (secure 0 0 0) 0x7f7a80030d50 con 0x7f7a900aaaf0 2021-02-10T11:36:44.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.831 7f7a954ad700 1 -- 172.21.2.101:0/1744203397 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a740186c0 msgr2=0x7f7a7401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:44.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.831 7f7a954ad700 1 --2- 172.21.2.101:0/1744203397 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a740186c0 0x7f7a7401abc0 crc :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:44.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.831 7f7a954ad700 1 -- 172.21.2.101:0/1744203397 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a900aaaf0 msgr2=0x7f7a90093700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:44.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.831 7f7a954ad700 1 --2- 172.21.2.101:0/1744203397 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a900aaaf0 0x7f7a90093700 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 rx=0x7f7a8000b5c0 tx=0x7f7a8000b960).stop 2021-02-10T11:36:44.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.831 7f7a954ad700 1 -- 172.21.2.101:0/1744203397 shutdown_connections 2021-02-10T11:36:44.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.831 7f7a954ad700 1 --2- 172.21.2.101:0/1744203397 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7a9018ed30 0x7f7a90093d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:44.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.831 7f7a954ad700 1 --2- 172.21.2.101:0/1744203397 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7a900aaaf0 0x7f7a90093700 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:44.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.831 7f7a954ad700 1 --2- 172.21.2.101:0/1744203397 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7a740186c0 0x7f7a7401abc0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:44.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.831 7f7a954ad700 1 --2- 172.21.2.101:0/1744203397 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7a9018e930 0x7f7a900aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:44.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.831 7f7a954ad700 1 -- 172.21.2.101:0/1744203397 >> 172.21.2.101:0/1744203397 conn(0x7f7a9019f8f0 msgr2=0x7f7a9018c260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:44.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.831 7f7a954ad700 1 -- 172.21.2.101:0/1744203397 shutdown_connections 2021-02-10T11:36:44.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:44.831 7f7a954ad700 1 -- 172.21.2.101:0/1744203397 wait complete. 2021-02-10T11:36:44.840 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:36:44.840 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:36:44.840 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 76, 2021-02-10T11:36:44.840 INFO:teuthology.orchestra.run.gibba001.stdout: "fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:36:44.840 INFO:teuthology.orchestra.run.gibba001.stdout: "created": "2021-02-10 11:12:36.788366", 2021-02-10T11:36:44.840 INFO:teuthology.orchestra.run.gibba001.stdout: "modified": "2021-02-10 11:36:41.922401", 2021-02-10T11:36:44.840 INFO:teuthology.orchestra.run.gibba001.stdout: "last_up_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:36:44.841 INFO:teuthology.orchestra.run.gibba001.stdout: "last_in_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:36:44.841 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-02-10T11:36:44.841 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_num": 5799936, 2021-02-10T11:36:44.841 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_set": [ 2021-02-10T11:36:44.841 INFO:teuthology.orchestra.run.gibba001.stdout: "pglog_hardlimit", 2021-02-10T11:36:44.841 INFO:teuthology.orchestra.run.gibba001.stdout: "purged_snapdirs", 2021-02-10T11:36:44.841 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_deletes", 2021-02-10T11:36:44.841 INFO:teuthology.orchestra.run.gibba001.stdout: "sortbitwise" 2021-02-10T11:36:44.841 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:36:44.841 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_version": 4, 2021-02-10T11:36:44.841 INFO:teuthology.orchestra.run.gibba001.stdout: "full_ratio": 0.94999998807907104, 2021-02-10T11:36:44.841 INFO:teuthology.orchestra.run.gibba001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-02-10T11:36:44.841 INFO:teuthology.orchestra.run.gibba001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-02-10T11:36:44.842 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_snapshot": "", 2021-02-10T11:36:44.842 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_max": 15, 2021-02-10T11:36:44.842 INFO:teuthology.orchestra.run.gibba001.stdout: "max_osd": 8, 2021-02-10T11:36:44.842 INFO:teuthology.orchestra.run.gibba001.stdout: "require_min_compat_client": "jewel", 2021-02-10T11:36:44.842 INFO:teuthology.orchestra.run.gibba001.stdout: "min_compat_client": "jewel", 2021-02-10T11:36:44.842 INFO:teuthology.orchestra.run.gibba001.stdout: "require_osd_release": "nautilus", 2021-02-10T11:36:44.842 INFO:teuthology.orchestra.run.gibba001.stdout: "pools": [ 2021-02-10T11:36:44.842 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.842 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 1, 2021-02-10T11:36:44.842 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "rbd", 2021-02-10T11:36:44.842 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:13:08.037472", 2021-02-10T11:36:44.842 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:36:44.843 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:36:44.843 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:36:44.843 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:36:44.843 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:36:44.843 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:36:44.843 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:36:44.843 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:36:44.843 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:36:44.843 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:36:44.843 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:36:44.843 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:36:44.843 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:36:44.843 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:36:44.844 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:36:44.844 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:36:44.844 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:36:44.844 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:36:44.844 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:36:44.844 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:36:44.844 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.844 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "16", 2021-02-10T11:36:44.844 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:36:44.844 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:36:44.844 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:36:44.844 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:36:44.844 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:36:44.845 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:36:44.845 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:36:44.845 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:36:44.845 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:36:44.845 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:36:44.845 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:36:44.845 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:36:44.845 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:36:44.845 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:36:44.845 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:36:44.845 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:36:44.845 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:36:44.845 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:36:44.846 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:36:44.846 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:36:44.846 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:36:44.846 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:36:44.846 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:36:44.846 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:36:44.846 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:36:44.846 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:36:44.846 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.846 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:36:44.846 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:36:44.846 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:36:44.847 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:36:44.847 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:36:44.847 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:36:44.847 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:36:44.847 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:36:44.847 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:36:44.847 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:36:44.847 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:36:44.847 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:36:44.847 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:36:44.847 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd": {} 2021-02-10T11:36:44.847 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.847 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.848 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.848 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 14, 2021-02-10T11:36:44.848 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_metadata", 2021-02-10T11:36:44.848 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:36:00.261115", 2021-02-10T11:36:44.848 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:36:44.848 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:36:44.848 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:36:44.848 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:36:44.848 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:36:44.848 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:36:44.848 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:36:44.848 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:36:44.848 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:36:44.849 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:36:44.849 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:36:44.849 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:36:44.849 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:36:44.849 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:36:44.849 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:36:44.849 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:36:44.849 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:36:44.849 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:36:44.849 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:36:44.849 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:36:44.849 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.849 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "73", 2021-02-10T11:36:44.850 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:36:44.850 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:36:44.850 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:36:44.850 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:36:44.850 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:36:44.850 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:36:44.850 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:36:44.850 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:36:44.850 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:36:44.850 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:36:44.850 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:36:44.851 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:36:44.851 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:36:44.851 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:36:44.851 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:36:44.851 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:36:44.851 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:36:44.851 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:36:44.851 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:36:44.851 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:36:44.851 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:36:44.851 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:36:44.851 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:36:44.851 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:36:44.852 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:36:44.852 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:36:44.852 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.852 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:36:44.852 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:36:44.852 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:36:44.852 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:36:44.852 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:36:44.852 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:36:44.852 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:36:44.852 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:36:44.852 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:36:44.852 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:36:44.853 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:36:44.853 INFO:teuthology.orchestra.run.gibba001.stdout: "options": { 2021-02-10T11:36:44.853 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_bias": 4, 2021-02-10T11:36:44.853 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_min": 16, 2021-02-10T11:36:44.853 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_priority": 5 2021-02-10T11:36:44.853 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.853 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:36:44.853 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:36:44.853 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": "cephfs" 2021-02-10T11:36:44.853 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.853 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.853 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.854 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.854 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 15, 2021-02-10T11:36:44.854 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_data", 2021-02-10T11:36:44.854 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:36:01.333230", 2021-02-10T11:36:44.854 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 8193, 2021-02-10T11:36:44.854 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2021-02-10T11:36:44.854 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:36:44.854 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:36:44.854 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:36:44.854 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:36:44.854 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:36:44.854 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:36:44.854 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:36:44.855 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:36:44.855 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:36:44.855 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:36:44.855 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:36:44.855 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:36:44.855 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:36:44.855 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:36:44.855 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:36:44.855 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:36:44.855 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:36:44.855 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:36:44.855 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.855 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "75", 2021-02-10T11:36:44.856 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:36:44.856 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:36:44.856 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:36:44.856 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:36:44.856 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:36:44.856 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 3, 2021-02-10T11:36:44.856 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 75, 2021-02-10T11:36:44.856 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:36:44.856 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[2~2]", 2021-02-10T11:36:44.856 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:36:44.856 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:36:44.856 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:36:44.857 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:36:44.857 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:36:44.857 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:36:44.857 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:36:44.857 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:36:44.857 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:36:44.857 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:36:44.857 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:36:44.857 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:36:44.857 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:36:44.857 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:36:44.857 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:36:44.857 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:36:44.858 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:36:44.858 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.858 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:36:44.858 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:36:44.858 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:36:44.858 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:36:44.858 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:36:44.858 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:36:44.858 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:36:44.858 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:36:44.858 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:36:44.858 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:36:44.859 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:36:44.859 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:36:44.859 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:36:44.859 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:36:44.859 INFO:teuthology.orchestra.run.gibba001.stdout: "data": "cephfs" 2021-02-10T11:36:44.859 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.859 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.859 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.859 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:36:44.859 INFO:teuthology.orchestra.run.gibba001.stdout: "osds": [ 2021-02-10T11:36:44.859 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.859 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:36:44.859 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5", 2021-02-10T11:36:44.860 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:36:44.860 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:36:44.860 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:36:44.860 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:36:44.860 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:36:44.860 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:36:44.860 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:36:44.860 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 72, 2021-02-10T11:36:44.860 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:36:44.860 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:36:44.860 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:36:44.860 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.860 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.860 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.861 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6802", 2021-02-10T11:36:44.861 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:36:44.861 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.861 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.861 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.861 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6805", 2021-02-10T11:36:44.861 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:36:44.861 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.861 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.861 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.861 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:36:44.861 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.861 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.862 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.862 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6807", 2021-02-10T11:36:44.862 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:36:44.862 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.862 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.862 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.862 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6809", 2021-02-10T11:36:44.862 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:36:44.862 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.862 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.862 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.862 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:36:44.862 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.863 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.863 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.863 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6817", 2021-02-10T11:36:44.863 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:36:44.863 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.863 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.863 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.863 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6820", 2021-02-10T11:36:44.863 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:36:44.863 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.863 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.863 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.863 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:36:44.863 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.864 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.864 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.864 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6812", 2021-02-10T11:36:44.864 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:36:44.864 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.864 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.864 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.864 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6814", 2021-02-10T11:36:44.864 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:36:44.864 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.864 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.864 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.864 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6805/25093", 2021-02-10T11:36:44.865 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6809/25093", 2021-02-10T11:36:44.865 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6820/25093", 2021-02-10T11:36:44.865 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6814/25093", 2021-02-10T11:36:44.865 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:36:44.865 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:36:44.865 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:36:44.865 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.865 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.865 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.865 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:36:44.865 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "ddc04ce3-b313-4707-8270-585391e38ecc", 2021-02-10T11:36:44.865 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:36:44.865 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:36:44.866 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:36:44.866 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:36:44.866 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:36:44.866 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:36:44.866 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:36:44.866 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 72, 2021-02-10T11:36:44.866 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:36:44.866 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:36:44.866 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:36:44.866 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.866 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.866 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.866 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6803", 2021-02-10T11:36:44.867 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:36:44.867 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.867 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.867 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.867 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6804", 2021-02-10T11:36:44.867 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:36:44.867 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.867 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.867 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.867 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:36:44.867 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.867 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.867 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.867 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6806", 2021-02-10T11:36:44.868 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:36:44.868 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.868 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.868 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.868 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6808", 2021-02-10T11:36:44.868 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:36:44.868 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.868 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.868 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.868 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:36:44.868 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.868 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.868 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.868 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6818", 2021-02-10T11:36:44.869 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:36:44.869 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.869 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.869 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.869 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6821", 2021-02-10T11:36:44.869 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:36:44.869 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.869 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.869 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.869 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:36:44.869 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.869 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.869 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.870 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6810", 2021-02-10T11:36:44.870 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:36:44.870 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.870 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.870 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.870 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6815", 2021-02-10T11:36:44.870 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:36:44.870 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.870 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.870 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.870 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6804/25092", 2021-02-10T11:36:44.870 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6808/25092", 2021-02-10T11:36:44.870 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6821/25092", 2021-02-10T11:36:44.870 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6815/25092", 2021-02-10T11:36:44.871 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:36:44.871 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:36:44.871 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:36:44.871 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.871 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.871 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.871 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:36:44.871 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3", 2021-02-10T11:36:44.871 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:36:44.871 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:36:44.871 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:36:44.871 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:36:44.871 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:36:44.871 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:36:44.872 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:36:44.872 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 72, 2021-02-10T11:36:44.872 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:36:44.872 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:36:44.872 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:36:44.872 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.872 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.872 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.872 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6811", 2021-02-10T11:36:44.872 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:36:44.872 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.872 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.872 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.873 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6813", 2021-02-10T11:36:44.873 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:36:44.873 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.873 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.873 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.873 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:36:44.873 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.873 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.873 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.873 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6816", 2021-02-10T11:36:44.873 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:36:44.873 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.873 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.873 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.874 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6819", 2021-02-10T11:36:44.874 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:36:44.874 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.874 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.874 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.874 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:36:44.874 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.874 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.874 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.874 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6825", 2021-02-10T11:36:44.874 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:36:44.874 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.874 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.875 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.875 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6827", 2021-02-10T11:36:44.875 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:36:44.875 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.875 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.875 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.875 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:36:44.875 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.875 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.875 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.875 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6822", 2021-02-10T11:36:44.875 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:36:44.875 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.876 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.876 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.876 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6823", 2021-02-10T11:36:44.876 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:36:44.876 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.876 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.876 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.876 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6813/25094", 2021-02-10T11:36:44.876 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6819/25094", 2021-02-10T11:36:44.876 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6827/25094", 2021-02-10T11:36:44.876 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6823/25094", 2021-02-10T11:36:44.876 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:36:44.876 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:36:44.876 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:36:44.877 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.877 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.877 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.877 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:36:44.877 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "0d9ef9b1-98aa-4d37-aa16-6305c7d46d52", 2021-02-10T11:36:44.877 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:36:44.877 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:36:44.877 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:36:44.877 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:36:44.877 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:36:44.877 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:36:44.877 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:36:44.877 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 70, 2021-02-10T11:36:44.878 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:36:44.878 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:36:44.878 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:36:44.878 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.878 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.878 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.878 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6824", 2021-02-10T11:36:44.878 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:36:44.878 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.878 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.878 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.878 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6826", 2021-02-10T11:36:44.878 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:36:44.879 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.879 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.879 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.879 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:36:44.879 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.879 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.879 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.879 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6828", 2021-02-10T11:36:44.879 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:36:44.879 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.879 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.879 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.879 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6829", 2021-02-10T11:36:44.880 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:36:44.880 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.880 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.880 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.880 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:36:44.880 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.880 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.880 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.880 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6832", 2021-02-10T11:36:44.880 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:36:44.880 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.880 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.880 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.880 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6833", 2021-02-10T11:36:44.881 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:36:44.881 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.881 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.881 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.881 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:36:44.881 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.881 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.881 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.881 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6830", 2021-02-10T11:36:44.881 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:36:44.881 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.881 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.881 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.881 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6831", 2021-02-10T11:36:44.882 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:36:44.882 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.882 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.882 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.882 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6826/25095", 2021-02-10T11:36:44.882 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6829/25095", 2021-02-10T11:36:44.882 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6833/25095", 2021-02-10T11:36:44.882 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6831/25095", 2021-02-10T11:36:44.882 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:36:44.882 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:36:44.882 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:36:44.882 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.883 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.883 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.883 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:36:44.883 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "36742c21-bfeb-4e11-bce9-8e745378b605", 2021-02-10T11:36:44.883 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:36:44.883 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:36:44.883 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:36:44.883 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:36:44.883 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:36:44.883 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:36:44.883 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:36:44.883 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 72, 2021-02-10T11:36:44.883 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:36:44.884 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:36:44.884 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:36:44.884 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.884 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.884 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.884 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6816", 2021-02-10T11:36:44.884 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:36:44.884 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.884 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.884 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.884 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6818", 2021-02-10T11:36:44.884 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:36:44.884 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.884 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.885 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.885 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:36:44.885 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.885 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.885 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.885 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6820", 2021-02-10T11:36:44.885 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:36:44.885 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.885 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.885 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.885 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6822", 2021-02-10T11:36:44.885 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:36:44.886 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.886 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.886 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.886 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:36:44.886 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.886 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.886 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.886 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6828", 2021-02-10T11:36:44.886 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:36:44.886 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.886 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.887 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.887 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6830", 2021-02-10T11:36:44.887 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:36:44.887 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.887 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.887 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.887 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:36:44.887 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.887 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.887 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.887 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6824", 2021-02-10T11:36:44.887 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:36:44.887 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.888 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.888 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.888 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6826", 2021-02-10T11:36:44.888 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:36:44.888 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.888 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.888 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.888 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6818/36380", 2021-02-10T11:36:44.888 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6822/36380", 2021-02-10T11:36:44.888 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6830/36380", 2021-02-10T11:36:44.888 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6826/36380", 2021-02-10T11:36:44.888 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:36:44.888 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:36:44.889 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:36:44.889 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.889 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.889 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.889 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:36:44.889 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "88e694e8-fe98-4d1b-88c8-8c9907ece60b", 2021-02-10T11:36:44.889 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:36:44.889 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:36:44.889 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:36:44.889 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:36:44.889 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:36:44.889 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:36:44.890 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:36:44.890 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 70, 2021-02-10T11:36:44.890 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:36:44.890 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:36:44.890 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:36:44.890 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.890 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.890 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.890 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6808", 2021-02-10T11:36:44.890 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:36:44.890 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.890 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.891 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.891 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6809", 2021-02-10T11:36:44.891 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:36:44.891 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.891 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.891 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.891 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:36:44.891 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.891 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.891 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.891 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6810", 2021-02-10T11:36:44.891 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:36:44.891 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.892 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.892 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.892 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6811", 2021-02-10T11:36:44.892 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:36:44.892 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.892 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.892 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.892 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:36:44.892 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.892 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.892 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.892 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6814", 2021-02-10T11:36:44.892 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:36:44.893 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.893 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.893 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.893 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6815", 2021-02-10T11:36:44.893 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:36:44.893 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.893 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.893 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.893 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:36:44.893 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.893 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.893 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.893 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6812", 2021-02-10T11:36:44.893 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:36:44.894 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.894 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.894 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.894 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6813", 2021-02-10T11:36:44.894 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:36:44.894 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.894 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.894 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.894 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6809/36383", 2021-02-10T11:36:44.894 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6811/36383", 2021-02-10T11:36:44.894 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6815/36383", 2021-02-10T11:36:44.894 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6813/36383", 2021-02-10T11:36:44.894 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:36:44.895 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:36:44.895 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:36:44.895 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.895 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.895 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.895 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:36:44.895 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "87a13c6c-cf85-4aaf-916b-e73913484a49", 2021-02-10T11:36:44.895 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:36:44.895 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:36:44.895 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:36:44.895 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:36:44.895 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:36:44.895 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:36:44.896 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:36:44.896 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 72, 2021-02-10T11:36:44.896 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:36:44.896 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:36:44.896 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:36:44.896 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.896 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.896 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.896 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6800", 2021-02-10T11:36:44.896 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:36:44.896 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.896 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.897 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.897 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6801", 2021-02-10T11:36:44.897 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:36:44.897 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.897 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.897 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.897 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:36:44.897 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.897 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.897 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.897 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6802", 2021-02-10T11:36:44.898 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:36:44.898 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.898 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.898 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.898 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6803", 2021-02-10T11:36:44.898 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:36:44.898 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.898 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.898 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.898 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:36:44.899 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.899 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.899 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.899 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6806", 2021-02-10T11:36:44.899 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:36:44.899 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.899 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.899 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.899 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6807", 2021-02-10T11:36:44.899 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:36:44.899 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.900 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.900 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.900 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:36:44.900 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.900 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.900 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.900 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6804", 2021-02-10T11:36:44.900 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:36:44.900 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.900 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.900 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.901 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6805", 2021-02-10T11:36:44.901 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:36:44.901 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.901 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.901 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.901 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6801/36381", 2021-02-10T11:36:44.901 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6803/36381", 2021-02-10T11:36:44.901 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6807/36381", 2021-02-10T11:36:44.901 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6805/36381", 2021-02-10T11:36:44.901 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:36:44.901 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:36:44.901 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:36:44.901 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.902 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.902 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.902 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:36:44.902 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "7b90823f-854c-4592-a95b-a7fa98f93254", 2021-02-10T11:36:44.902 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:36:44.902 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:36:44.902 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:36:44.902 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:36:44.902 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:36:44.902 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:36:44.902 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:36:44.902 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 63, 2021-02-10T11:36:44.903 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:36:44.903 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:36:44.903 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:36:44.903 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.903 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.903 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.903 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6817", 2021-02-10T11:36:44.903 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:36:44.903 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.903 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.903 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.903 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6819", 2021-02-10T11:36:44.903 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:36:44.903 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.904 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.904 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.904 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:36:44.904 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.904 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.904 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.904 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6821", 2021-02-10T11:36:44.904 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:36:44.904 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.904 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.904 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.904 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6823", 2021-02-10T11:36:44.904 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:36:44.905 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.905 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.905 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.905 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:36:44.905 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.905 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.905 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.905 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6829", 2021-02-10T11:36:44.905 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:36:44.905 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.905 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.905 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.905 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6831", 2021-02-10T11:36:44.905 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:36:44.906 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.906 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.906 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.906 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:36:44.906 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:36:44.906 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.906 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:36:44.906 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6825", 2021-02-10T11:36:44.906 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:36:44.906 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.906 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.906 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:36:44.907 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6827", 2021-02-10T11:36:44.907 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:36:44.907 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.907 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.907 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.907 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6819/36382", 2021-02-10T11:36:44.907 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6823/36382", 2021-02-10T11:36:44.907 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6831/36382", 2021-02-10T11:36:44.907 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6827/36382", 2021-02-10T11:36:44.907 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:36:44.908 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:36:44.908 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:36:44.908 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.908 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.908 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:36:44.908 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_xinfo": [ 2021-02-10T11:36:44.908 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.908 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:36:44.908 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:36:44.908 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:36:44.908 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:36:44.908 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:36:44.908 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:36:44.909 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.909 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.909 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:36:44.909 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:36:44.909 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:36:44.909 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:36:44.909 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:36:44.909 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:36:44.909 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.909 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.909 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:36:44.909 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:36:44.909 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:36:44.909 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:36:44.910 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:36:44.910 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:36:44.910 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.910 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.910 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:36:44.910 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:36:44.910 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:36:44.910 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:36:44.910 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:36:44.910 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:36:44.910 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.910 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.911 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:36:44.911 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:36:44.911 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:36:44.911 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:36:44.911 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:36:44.911 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:36:44.911 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.911 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.911 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:36:44.911 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:36:44.911 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:36:44.911 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:36:44.911 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:36:44.911 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:36:44.912 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.912 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.912 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:36:44.912 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:36:44.912 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:36:44.912 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:36:44.912 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:36:44.912 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:36:44.912 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.912 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.912 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:36:44.912 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:36:44.912 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:36:44.912 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:36:44.913 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:36:44.913 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:36:44.913 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.913 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:36:44.913 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap": [], 2021-02-10T11:36:44.913 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap_items": [], 2021-02-10T11:36:44.913 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_temp": [], 2021-02-10T11:36:44.913 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_temp": [], 2021-02-10T11:36:44.913 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklist": {}, 2021-02-10T11:36:44.913 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profiles": { 2021-02-10T11:36:44.913 INFO:teuthology.orchestra.run.gibba001.stdout: "default": { 2021-02-10T11:36:44.913 INFO:teuthology.orchestra.run.gibba001.stdout: "crush-failure-domain": "osd", 2021-02-10T11:36:44.913 INFO:teuthology.orchestra.run.gibba001.stdout: "k": "2", 2021-02-10T11:36:44.914 INFO:teuthology.orchestra.run.gibba001.stdout: "m": "1", 2021-02-10T11:36:44.914 INFO:teuthology.orchestra.run.gibba001.stdout: "plugin": "jerasure", 2021-02-10T11:36:44.914 INFO:teuthology.orchestra.run.gibba001.stdout: "ruleset-failure-domain": "osd", 2021-02-10T11:36:44.914 INFO:teuthology.orchestra.run.gibba001.stdout: "technique": "reed_sol_van" 2021-02-10T11:36:44.914 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.914 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:44.914 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps_queue": [], 2021-02-10T11:36:44.914 INFO:teuthology.orchestra.run.gibba001.stdout: "new_removed_snaps": [], 2021-02-10T11:36:44.914 INFO:teuthology.orchestra.run.gibba001.stdout: "new_purged_snaps": [ 2021-02-10T11:36:44.914 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.914 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 15, 2021-02-10T11:36:44.914 INFO:teuthology.orchestra.run.gibba001.stdout: "snaps": [ 2021-02-10T11:36:44.915 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:44.915 INFO:teuthology.orchestra.run.gibba001.stdout: "begin": 2, 2021-02-10T11:36:44.915 INFO:teuthology.orchestra.run.gibba001.stdout: "length": 2 2021-02-10T11:36:44.915 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.915 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:44.915 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:44.915 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:36:44.915 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_node_flags": {}, 2021-02-10T11:36:44.915 INFO:teuthology.orchestra.run.gibba001.stdout: "device_class_flags": {} 2021-02-10T11:36:44.915 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:36:44.917 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:36:45.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.320 7fb11ad44700 1 Processor -- start 2021-02-10T11:36:45.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.320 7fb11ad44700 1 -- start start 2021-02-10T11:36:45.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.321 7fb11ad44700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1140d1870 0x7fb1141b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:45.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.321 7fb11ad44700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1140f0b10 con 0x7fb1140f0650 2021-02-10T11:36:45.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.321 7fb11ad44700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1140d0310 con 0x7fb1141b0a10 2021-02-10T11:36:45.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.321 7fb11ad44700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1140d0530 con 0x7fb1140d1870 2021-02-10T11:36:45.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.321 7fb118b60700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1140d1870 0x7fb1141b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:45.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.321 7fb119361700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb1140f0650 0x7fb1140cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54648/0 (socket says 172.21.2.101:54648) 2021-02-10T11:36:45.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.321 7fb119361700 1 -- 172.21.2.101:0/872313298 learned_addr learned my addr 172.21.2.101:0/872313298 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:45.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.321 7fb1137fe700 1 -- 172.21.2.101:0/872313298 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 868888037 0 0) 0x7fb10c000c10 con 0x7fb1140f0650 2021-02-10T11:36:45.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.322 7fb1137fe700 1 -- 172.21.2.101:0/872313298 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb100002fe0 con 0x7fb1140f0650 2021-02-10T11:36:45.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.322 7fb1137fe700 1 -- 172.21.2.101:0/872313298 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2963345596 0 0) 0x7fb104000fe0 con 0x7fb1141b0a10 2021-02-10T11:36:45.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.322 7fb1137fe700 1 -- 172.21.2.101:0/872313298 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb100004370 con 0x7fb1141b0a10 2021-02-10T11:36:45.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.322 7fb118b60700 1 -- 172.21.2.101:0/872313298 >> v1:172.21.2.108:6789/0 conn(0x7fb1141b0a10 legacy=0x7fb1140cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:45.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.322 7fb118b60700 1 -- 172.21.2.101:0/872313298 >> v1:172.21.2.101:6789/0 conn(0x7fb1140f0650 legacy=0x7fb1140cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:45.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.322 7fb118b60700 1 -- 172.21.2.101:0/872313298 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1140d1c70 con 0x7fb1140d1870 2021-02-10T11:36:45.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.322 7fb118b60700 1 --2- 172.21.2.101:0/872313298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1140d1870 0x7fb1141b03f0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 rx=0x7fb0fc006650 tx=0x7fb0fc009a80).ready entity=mon.0 client_cookie=96a77912bf601e92 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:45.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.322 7fb1137fe700 1 -- 172.21.2.101:0/872313298 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0fc00c0c0 con 0x7fb1140d1870 2021-02-10T11:36:45.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.322 7fb1137fe700 1 -- 172.21.2.101:0/872313298 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc00c340 con 0x7fb1140d1870 2021-02-10T11:36:45.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.322 7fb1137fe700 1 -- 172.21.2.101:0/872313298 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0fc00c860 con 0x7fb1140d1870 2021-02-10T11:36:45.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.322 7fb11ad44700 1 -- 172.21.2.101:0/872313298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1140d1870 msgr2=0x7fb1141b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:45.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.322 7fb11ad44700 1 --2- 172.21.2.101:0/872313298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1140d1870 0x7fb1141b03f0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 rx=0x7fb0fc006650 tx=0x7fb0fc009a80).stop 2021-02-10T11:36:45.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.323 7fb11ad44700 1 -- 172.21.2.101:0/872313298 shutdown_connections 2021-02-10T11:36:45.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.323 7fb11ad44700 1 --2- 172.21.2.101:0/872313298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1140d1870 0x7fb1141b03f0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:45.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.323 7fb11ad44700 1 -- 172.21.2.101:0/872313298 >> 172.21.2.101:0/872313298 conn(0x7fb1140ce450 msgr2=0x7fb1140d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:45.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.323 7fb11ad44700 1 -- 172.21.2.101:0/872313298 shutdown_connections 2021-02-10T11:36:45.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.323 7fb11ad44700 1 -- 172.21.2.101:0/872313298 wait complete. 2021-02-10T11:36:45.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.337 7fb11ad44700 1 Processor -- start 2021-02-10T11:36:45.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.337 7fb11ad44700 1 -- start start 2021-02-10T11:36:45.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.338 7fb11ad44700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1140ada90 0x7fb1140ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:45.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.338 7fb11ad44700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb114099d20 0x7fb11409a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:45.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.338 7fb11ad44700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb11409a740 0x7fb114097bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:45.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.338 7fb11ad44700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1140997a0 con 0x7fb1140ada90 2021-02-10T11:36:45.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.338 7fb11ad44700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11418f0c0 con 0x7fb11409a740 2021-02-10T11:36:45.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.338 7fb11ad44700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11409ab40 con 0x7fb114099d20 2021-02-10T11:36:45.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.338 7fb118b60700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1140ada90 0x7fb1140ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:45.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.338 7fb118b60700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1140ada90 0x7fb1140ade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54158/0 (socket says 172.21.2.101:54158) 2021-02-10T11:36:45.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.338 7fb118b60700 1 -- 172.21.2.101:0/3729652744 learned_addr learned my addr 172.21.2.101:0/3729652744 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:45.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.338 7fb113fff700 1 --2- 172.21.2.101:0/3729652744 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb114099d20 0x7fb11409a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:45.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.338 7fb119361700 1 --2- 172.21.2.101:0/3729652744 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb11409a740 0x7fb114097bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:45.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.338 7fb118b60700 1 -- 172.21.2.101:0/3729652744 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb114099d20 msgr2=0x7fb11409a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:45.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.338 7fb118b60700 1 --2- 172.21.2.101:0/3729652744 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb114099d20 0x7fb11409a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:45.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.338 7fb118b60700 1 -- 172.21.2.101:0/3729652744 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb11409a740 msgr2=0x7fb114097bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:45.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.339 7fb118b60700 1 --2- 172.21.2.101:0/3729652744 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb11409a740 0x7fb114097bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:45.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.339 7fb118b60700 1 -- 172.21.2.101:0/3729652744 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11409ad60 con 0x7fb1140ada90 2021-02-10T11:36:45.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.339 7fb113fff700 1 --2- 172.21.2.101:0/3729652744 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb114099d20 0x7fb11409a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:36:45.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.339 7fb118b60700 1 --2- 172.21.2.101:0/3729652744 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1140ada90 0x7fb1140ade90 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 rx=0x7fb0fc009630 tx=0x7fb0fc00b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:45.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.339 7fb1117fa700 1 -- 172.21.2.101:0/3729652744 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0fc0093a0 con 0x7fb1140ada90 2021-02-10T11:36:45.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.339 7fb1117fa700 1 -- 172.21.2.101:0/3729652744 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc00bd70 con 0x7fb1140ada90 2021-02-10T11:36:45.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.339 7fb1117fa700 1 -- 172.21.2.101:0/3729652744 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0fc0054d0 con 0x7fb1140ada90 2021-02-10T11:36:45.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.339 7fb11ad44700 1 -- 172.21.2.101:0/3729652744 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11409ad60 con 0x7fb1140ada90 2021-02-10T11:36:45.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.339 7fb11ad44700 1 -- 172.21.2.101:0/3729652744 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1140984d0 con 0x7fb1140ada90 2021-02-10T11:36:45.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.340 7fb1117fa700 1 -- 172.21.2.101:0/3729652744 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb0fc007020 con 0x7fb1140ada90 2021-02-10T11:36:45.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.340 7fb1117fa700 1 --2- 172.21.2.101:0/3729652744 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0f80186c0 0x7fb0f801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:45.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.340 7fb1117fa700 1 -- 172.21.2.101:0/3729652744 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5617+0+0 (secure 0 0 0) 0x7fb0fc032590 con 0x7fb1140ada90 2021-02-10T11:36:45.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.340 7fb113fff700 1 --2- 172.21.2.101:0/3729652744 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0f80186c0 0x7fb0f801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:45.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.340 7fb113fff700 1 --2- 172.21.2.101:0/3729652744 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0f80186c0 0x7fb0f801abc0 crc :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:45.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.371 7fb11ad44700 1 -- 172.21.2.101:0/3729652744 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1140ebc00 con 0x7fb1140ada90 2021-02-10T11:36:45.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.374 7fb1117fa700 1 -- 172.21.2.101:0/3729652744 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb0fc01b400 con 0x7fb1140ada90 2021-02-10T11:36:45.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.502 7fb11ad44700 1 -- 172.21.2.101:0/3729652744 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1140ebc00 con 0x7fb1140ada90 2021-02-10T11:36:45.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.502 7fb1117fa700 1 -- 172.21.2.101:0/3729652744 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+2101 (secure 0 0 0) 0x7fb0fc01b400 con 0x7fb1140ada90 2021-02-10T11:36:45.509 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 56 2021-02-10T11:36:45.509 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:36:45.509 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/3016216705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3016216705},{"type":"v1","addr":"172.21.2.108:6833","nonce":3016216705}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":51}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:02.287195","modified":"2021-02-10 11:36:04.295896","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5900},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5900":{"gid":5900,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.2.101:6835/2977571942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2977571942},{"type":"v1","addr":"172.21.2.101:6835","nonce":2977571942}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-02-10T11:36:45.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.504 7fb11ad44700 1 -- 172.21.2.101:0/3729652744 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0f80186c0 msgr2=0x7fb0f801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:45.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.504 7fb11ad44700 1 --2- 172.21.2.101:0/3729652744 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0f80186c0 0x7fb0f801abc0 crc :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:45.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.504 7fb11ad44700 1 -- 172.21.2.101:0/3729652744 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1140ada90 msgr2=0x7fb1140ade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:45.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.504 7fb11ad44700 1 --2- 172.21.2.101:0/3729652744 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1140ada90 0x7fb1140ade90 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 rx=0x7fb0fc009630 tx=0x7fb0fc00b980).stop 2021-02-10T11:36:45.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.505 7fb11ad44700 1 -- 172.21.2.101:0/3729652744 shutdown_connections 2021-02-10T11:36:45.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.505 7fb11ad44700 1 --2- 172.21.2.101:0/3729652744 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0f80186c0 0x7fb0f801abc0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:45.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.505 7fb11ad44700 1 --2- 172.21.2.101:0/3729652744 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb114099d20 0x7fb11409a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:45.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.505 7fb11ad44700 1 --2- 172.21.2.101:0/3729652744 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb11409a740 0x7fb114097bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:45.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.505 7fb11ad44700 1 --2- 172.21.2.101:0/3729652744 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1140ada90 0x7fb1140ade90 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:45.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.505 7fb11ad44700 1 -- 172.21.2.101:0/3729652744 >> 172.21.2.101:0/3729652744 conn(0x7fb1140f0550 msgr2=0x7fb114191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:45.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.505 7fb11ad44700 1 -- 172.21.2.101:0/3729652744 shutdown_connections 2021-02-10T11:36:45.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.505 7fb11ad44700 1 -- 172.21.2.101:0/3729652744 wait complete. 2021-02-10T11:36:45.520 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs fail cephfs 2021-02-10T11:36:45.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.923 7ff68cdc6700 1 Processor -- start 2021-02-10T11:36:45.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.923 7ff68cdc6700 1 -- start start 2021-02-10T11:36:45.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.924 7ff68cdc6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff6880d1870 0x7ff6881b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:45.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.924 7ff68cdc6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6880f0650 0x7ff6880cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:45.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.924 7ff68cdc6700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6880f0b10 con 0x7ff6881b0a10 2021-02-10T11:36:45.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.924 7ff68cdc6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6880e5ed0 con 0x7ff6880d1870 2021-02-10T11:36:45.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.924 7ff68cdc6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6880d1c70 con 0x7ff6880f0650 2021-02-10T11:36:45.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.924 7ff686b13700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff6880d1870 0x7ff6881b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:45.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.924 7ff686312700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff6881b0a10 0x7ff6880cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54662/0 (socket says 172.21.2.101:54662) 2021-02-10T11:36:45.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.924 7ff686312700 1 -- 172.21.2.101:0/4262191150 learned_addr learned my addr 172.21.2.101:0/4262191150 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:45.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.924 7ff686b13700 1 -- 172.21.2.101:0/4262191150 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6880f0650 msgr2=0x7ff6880cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:45.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.924 7ff686b13700 1 --2- 172.21.2.101:0/4262191150 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6880f0650 0x7ff6880cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:45.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.924 7ff686b13700 1 -- 172.21.2.101:0/4262191150 >> v1:172.21.2.101:6789/0 conn(0x7ff6881b0a10 legacy=0x7ff6880cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:45.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.924 7ff686b13700 1 -- 172.21.2.101:0/4262191150 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6880d1e90 con 0x7ff6880d1870 2021-02-10T11:36:45.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.924 7ff686b13700 1 --2- 172.21.2.101:0/4262191150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6880d1870 0x7ff6881b03f0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 rx=0x7ff678006650 tx=0x7ff678009a80).ready entity=mon.0 client_cookie=b3b80e19226da3c3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:45.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.925 7ff685b11700 1 -- 172.21.2.101:0/4262191150 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff67800c0c0 con 0x7ff6880d1870 2021-02-10T11:36:45.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.925 7ff685b11700 1 -- 172.21.2.101:0/4262191150 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67800c340 con 0x7ff6880d1870 2021-02-10T11:36:45.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.925 7ff685b11700 1 -- 172.21.2.101:0/4262191150 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff67800c860 con 0x7ff6880d1870 2021-02-10T11:36:45.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.925 7ff68cdc6700 1 -- 172.21.2.101:0/4262191150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6880d1870 msgr2=0x7ff6881b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:45.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.925 7ff68cdc6700 1 --2- 172.21.2.101:0/4262191150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6880d1870 0x7ff6881b03f0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 rx=0x7ff678006650 tx=0x7ff678009a80).stop 2021-02-10T11:36:45.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.925 7ff68cdc6700 1 -- 172.21.2.101:0/4262191150 shutdown_connections 2021-02-10T11:36:45.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.925 7ff68cdc6700 1 --2- 172.21.2.101:0/4262191150 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6880f0650 0x7ff6880cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:45.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.925 7ff68cdc6700 1 --2- 172.21.2.101:0/4262191150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6880d1870 0x7ff6881b03f0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:45.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.925 7ff68cdc6700 1 -- 172.21.2.101:0/4262191150 >> 172.21.2.101:0/4262191150 conn(0x7ff6880ce450 msgr2=0x7ff6880d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:45.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.925 7ff68cdc6700 1 -- 172.21.2.101:0/4262191150 shutdown_connections 2021-02-10T11:36:45.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.925 7ff68cdc6700 1 -- 172.21.2.101:0/4262191150 wait complete. 2021-02-10T11:36:45.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.940 7ff68cdc6700 1 Processor -- start 2021-02-10T11:36:45.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.940 7ff68cdc6700 1 -- start start 2021-02-10T11:36:45.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.940 7ff68cdc6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6880bdda0 0x7ff68818b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:45.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.940 7ff68cdc6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff68818b870 0x7ff6880c0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:45.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.940 7ff68cdc6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff68818bc70 0x7ff6880c0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:45.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.940 7ff68cdc6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68819d390 con 0x7ff68818b870 2021-02-10T11:36:45.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.940 7ff68cdc6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6880bd740 con 0x7ff68818bc70 2021-02-10T11:36:45.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.940 7ff68cdc6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6880be1a0 con 0x7ff6880bdda0 2021-02-10T11:36:45.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.941 7ff686312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff68818b870 0x7ff6880c0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:45.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.941 7ff686312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff68818b870 0x7ff6880c0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54170/0 (socket says 172.21.2.101:54170) 2021-02-10T11:36:45.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.941 7ff686312700 1 -- 172.21.2.101:0/2378824802 learned_addr learned my addr 172.21.2.101:0/2378824802 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:45.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.941 7ff686b13700 1 --2- 172.21.2.101:0/2378824802 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6880bdda0 0x7ff68818b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:45.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.941 7ff687314700 1 --2- 172.21.2.101:0/2378824802 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff68818bc70 0x7ff6880c0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:45.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.941 7ff686312700 1 -- 172.21.2.101:0/2378824802 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6880bdda0 msgr2=0x7ff68818b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:45.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.941 7ff686312700 1 --2- 172.21.2.101:0/2378824802 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6880bdda0 0x7ff68818b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:45.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.941 7ff686312700 1 -- 172.21.2.101:0/2378824802 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff68818bc70 msgr2=0x7ff6880c0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:45.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.941 7ff686312700 1 --2- 172.21.2.101:0/2378824802 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff68818bc70 0x7ff6880c0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:45.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.941 7ff686312700 1 -- 172.21.2.101:0/2378824802 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6880c0c70 con 0x7ff68818b870 2021-02-10T11:36:45.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.941 7ff687314700 1 --2- 172.21.2.101:0/2378824802 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff68818bc70 0x7ff6880c0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:36:45.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.941 7ff686312700 1 --2- 172.21.2.101:0/2378824802 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff68818b870 0x7ff6880c0030 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 rx=0x7ff670001c60 tx=0x7ff67000a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:45.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.941 7ff67f7fe700 1 -- 172.21.2.101:0/2378824802 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff67000acc0 con 0x7ff68818b870 2021-02-10T11:36:45.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.941 7ff67f7fe700 1 -- 172.21.2.101:0/2378824802 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff670004140 con 0x7ff68818b870 2021-02-10T11:36:45.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.941 7ff68cdc6700 1 -- 172.21.2.101:0/2378824802 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6880c0c70 con 0x7ff68818b870 2021-02-10T11:36:45.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.941 7ff68cdc6700 1 -- 172.21.2.101:0/2378824802 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6880c1200 con 0x7ff68818b870 2021-02-10T11:36:45.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.941 7ff67f7fe700 1 -- 172.21.2.101:0/2378824802 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff670004620 con 0x7ff68818b870 2021-02-10T11:36:45.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.942 7ff67f7fe700 1 -- 172.21.2.101:0/2378824802 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff670004320 con 0x7ff68818b870 2021-02-10T11:36:45.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.942 7ff67f7fe700 1 --2- 172.21.2.101:0/2378824802 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff66c018730 0x7ff66c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:45.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.942 7ff67f7fe700 1 -- 172.21.2.101:0/2378824802 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5617+0+0 (secure 0 0 0) 0x7ff6700327e0 con 0x7ff68818b870 2021-02-10T11:36:45.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.942 7ff686b13700 1 --2- 172.21.2.101:0/2378824802 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff66c018730 0x7ff66c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:45.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.943 7ff686b13700 1 --2- 172.21.2.101:0/2378824802 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff66c018730 0x7ff66c01abc0 crc :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:45.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.974 7ff68cdc6700 1 -- 172.21.2.101:0/2378824802 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6880ee0b0 con 0x7ff68818b870 2021-02-10T11:36:45.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:45.976 7ff67f7fe700 1 -- 172.21.2.101:0/2378824802 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff670000dd0 con 0x7ff68818b870 2021-02-10T11:36:46.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.105 7ff68cdc6700 1 -- 172.21.2.101:0/2378824802 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff6880ce450 con 0x7ff68818b870 2021-02-10T11:36:46.507 INFO:tasks.ceph.mon.a.gibba001.stderr:2021-02-10 11:36:46.501 7f974d04d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-02-10T11:36:46.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.507 7ff67f7fe700 1 -- 172.21.2.101:0/2378824802 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v57) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff670004140 con 0x7ff68818b870 2021-02-10T11:36:46.527 INFO:teuthology.orchestra.run.gibba001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-02-10T11:36:46.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.523 7ff68cdc6700 1 -- 172.21.2.101:0/2378824802 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff66c018730 msgr2=0x7ff66c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:46.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.523 7ff68cdc6700 1 --2- 172.21.2.101:0/2378824802 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff66c018730 0x7ff66c01abc0 crc :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:46.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.523 7ff68cdc6700 1 -- 172.21.2.101:0/2378824802 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff68818b870 msgr2=0x7ff6880c0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:46.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.523 7ff68cdc6700 1 --2- 172.21.2.101:0/2378824802 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff68818b870 0x7ff6880c0030 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 rx=0x7ff670001c60 tx=0x7ff67000a8d0).stop 2021-02-10T11:36:46.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.523 7ff68cdc6700 1 -- 172.21.2.101:0/2378824802 shutdown_connections 2021-02-10T11:36:46.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.523 7ff68cdc6700 1 --2- 172.21.2.101:0/2378824802 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff68818bc70 0x7ff6880c0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:46.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.523 7ff68cdc6700 1 --2- 172.21.2.101:0/2378824802 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff68818b870 0x7ff6880c0030 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:46.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.523 7ff68cdc6700 1 --2- 172.21.2.101:0/2378824802 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff66c018730 0x7ff66c01abc0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:46.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.523 7ff68cdc6700 1 --2- 172.21.2.101:0/2378824802 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6880bdda0 0x7ff68818b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:46.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.523 7ff68cdc6700 1 -- 172.21.2.101:0/2378824802 >> 172.21.2.101:0/2378824802 conn(0x7ff6880a6eb0 msgr2=0x7ff688098ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:46.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.523 7ff68cdc6700 1 -- 172.21.2.101:0/2378824802 shutdown_connections 2021-02-10T11:36:46.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.523 7ff68cdc6700 1 -- 172.21.2.101:0/2378824802 wait complete. 2021-02-10T11:36:46.538 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:36:46.566 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:36:46.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.943 7fbd14972700 1 Processor -- start 2021-02-10T11:36:46.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.943 7fbd14972700 1 -- start start 2021-02-10T11:36:46.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.943 7fbd14972700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbd100d0340 0x7fbd100ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:46.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.943 7fbd14972700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd100d1c70 0x7fbd100efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:46.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.943 7fbd14972700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd100f0b10 con 0x7fbd100d1870 2021-02-10T11:36:46.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.943 7fbd14972700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd100e5ed0 con 0x7fbd100d0340 2021-02-10T11:36:46.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.943 7fbd14972700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd101b0be0 con 0x7fbd100d1c70 2021-02-10T11:36:46.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.943 7fbd0de1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbd100d0340 0x7fbd100ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:46.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.943 7fbd0de1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbd100d0340 0x7fbd100ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54190/0 (socket says 172.21.2.101:54190) 2021-02-10T11:36:46.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.943 7fbd0de1b700 1 -- 172.21.2.101:0/3349368458 learned_addr learned my addr 172.21.2.101:0/3349368458 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:46.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.944 7fbd0ee1d700 1 --2- 172.21.2.101:0/3349368458 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd100d1c70 0x7fbd100efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:46.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.944 7fbd0de1b700 1 -- 172.21.2.101:0/3349368458 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd100d1c70 msgr2=0x7fbd100efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:46.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.944 7fbd0de1b700 1 --2- 172.21.2.101:0/3349368458 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd100d1c70 0x7fbd100efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:46.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.944 7fbd0de1b700 1 -- 172.21.2.101:0/3349368458 >> v1:172.21.2.108:6789/0 conn(0x7fbd100d1870 legacy=0x7fbd101b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:46.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.944 7fbd0de1b700 1 -- 172.21.2.101:0/3349368458 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd100f20a0 con 0x7fbd100d0340 2021-02-10T11:36:46.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.944 7fbd0de1b700 1 --2- 172.21.2.101:0/3349368458 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd100d0340 0x7fbd100ef640 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 rx=0x7fbd00006700 tx=0x7fbd00009b80).ready entity=mon.0 client_cookie=3a736155f029e41d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:46.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.944 7fbd0d61a700 1 -- 172.21.2.101:0/3349368458 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbd0000c0c0 con 0x7fbd100d0340 2021-02-10T11:36:46.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.944 7fbd0d61a700 1 -- 172.21.2.101:0/3349368458 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0000c440 con 0x7fbd100d0340 2021-02-10T11:36:46.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.944 7fbd0d61a700 1 -- 172.21.2.101:0/3349368458 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbd0000c8a0 con 0x7fbd100d0340 2021-02-10T11:36:46.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.944 7fbd14972700 1 -- 172.21.2.101:0/3349368458 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd100d0340 msgr2=0x7fbd100ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:46.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.944 7fbd14972700 1 --2- 172.21.2.101:0/3349368458 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd100d0340 0x7fbd100ef640 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 rx=0x7fbd00006700 tx=0x7fbd00009b80).stop 2021-02-10T11:36:46.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.945 7fbd14972700 1 -- 172.21.2.101:0/3349368458 shutdown_connections 2021-02-10T11:36:46.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.945 7fbd14972700 1 --2- 172.21.2.101:0/3349368458 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd100d1c70 0x7fbd100efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:46.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.945 7fbd14972700 1 --2- 172.21.2.101:0/3349368458 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd100d0340 0x7fbd100ef640 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:46.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.945 7fbd14972700 1 -- 172.21.2.101:0/3349368458 >> 172.21.2.101:0/3349368458 conn(0x7fbd100ce450 msgr2=0x7fbd100d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:46.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.945 7fbd14972700 1 -- 172.21.2.101:0/3349368458 shutdown_connections 2021-02-10T11:36:46.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.945 7fbd14972700 1 -- 172.21.2.101:0/3349368458 wait complete. 2021-02-10T11:36:46.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.960 7fbd14972700 1 Processor -- start 2021-02-10T11:36:46.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.960 7fbd14972700 1 -- start start 2021-02-10T11:36:46.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.960 7fbd14972700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd100ada90 0x7fbd100ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:46.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.960 7fbd14972700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbd10099d20 0x7fbd1009a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:46.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.960 7fbd14972700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd1009a740 0x7fbd10097bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:46.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.960 7fbd14972700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd100997a0 con 0x7fbd100ada90 2021-02-10T11:36:46.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.960 7fbd14972700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1018f0c0 con 0x7fbd10099d20 2021-02-10T11:36:46.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.960 7fbd14972700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1009ab40 con 0x7fbd1009a740 2021-02-10T11:36:46.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.960 7fbd0e61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd100ada90 0x7fbd100ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:46.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.961 7fbd0e61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd100ada90 0x7fbd100ade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54196/0 (socket says 172.21.2.101:54196) 2021-02-10T11:36:46.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.961 7fbd0e61c700 1 -- 172.21.2.101:0/1695884484 learned_addr learned my addr 172.21.2.101:0/1695884484 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:46.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.961 7fbd0e61c700 1 -- 172.21.2.101:0/1695884484 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd1009a740 msgr2=0x7fbd10097bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:46.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.961 7fbd0e61c700 1 --2- 172.21.2.101:0/1695884484 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd1009a740 0x7fbd10097bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:46.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.961 7fbd0e61c700 1 -- 172.21.2.101:0/1695884484 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbd10099d20 msgr2=0x7fbd1009a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:46.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.961 7fbd0e61c700 1 --2- 172.21.2.101:0/1695884484 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbd10099d20 0x7fbd1009a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:46.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.961 7fbd0e61c700 1 -- 172.21.2.101:0/1695884484 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd1009ad60 con 0x7fbd100ada90 2021-02-10T11:36:46.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.961 7fbd0e61c700 1 --2- 172.21.2.101:0/1695884484 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd100ada90 0x7fbd100ade90 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 rx=0x7fbcf800b530 tx=0x7fbcf800b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:46.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.961 7fbd06ffd700 1 -- 172.21.2.101:0/1695884484 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbcf800bcc0 con 0x7fbd100ada90 2021-02-10T11:36:46.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.961 7fbd06ffd700 1 -- 172.21.2.101:0/1695884484 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf8006020 con 0x7fbd100ada90 2021-02-10T11:36:46.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.961 7fbd06ffd700 1 -- 172.21.2.101:0/1695884484 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbcf80054a0 con 0x7fbd100ada90 2021-02-10T11:36:46.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.961 7fbd14972700 1 -- 172.21.2.101:0/1695884484 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd1009ad60 con 0x7fbd100ada90 2021-02-10T11:36:46.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.961 7fbd14972700 1 -- 172.21.2.101:0/1695884484 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd100984d0 con 0x7fbd100ada90 2021-02-10T11:36:46.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.962 7fbd06ffd700 1 -- 172.21.2.101:0/1695884484 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbcf8006200 con 0x7fbd100ada90 2021-02-10T11:36:46.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.962 7fbd06ffd700 1 --2- 172.21.2.101:0/1695884484 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbcf40186c0 0x7fbcf401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:46.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.962 7fbd06ffd700 1 -- 172.21.2.101:0/1695884484 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5675+0+0 (secure 0 0 0) 0x7fbcf8032490 con 0x7fbd100ada90 2021-02-10T11:36:46.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.962 7fbd0de1b700 1 --2- 172.21.2.101:0/1695884484 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbcf40186c0 0x7fbcf401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:46.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.963 7fbd0de1b700 1 --2- 172.21.2.101:0/1695884484 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbcf40186c0 0x7fbcf401abc0 crc :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:47.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.993 7fbd14972700 1 -- 172.21.2.101:0/1695884484 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd100ebc00 con 0x7fbd100ada90 2021-02-10T11:36:47.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:46.996 7fbd06ffd700 1 -- 172.21.2.101:0/1695884484 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbcf8000940 con 0x7fbd100ada90 2021-02-10T11:36:47.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.124 7fbd14972700 1 -- 172.21.2.101:0/1695884484 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd100ebc00 con 0x7fbd100ada90 2021-02-10T11:36:47.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.125 7fbd06ffd700 1 -- 172.21.2.101:0/1695884484 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+1756 (secure 0 0 0) 0x7fbcf80061e0 con 0x7fbd100ada90 2021-02-10T11:36:47.131 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:36:47.132 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/3016216705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3016216705},{"type":"v1","addr":"172.21.2.108:6833","nonce":3016216705}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":51}],"filesystems":[{"mdsmap":{"epoch":57,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:02.287195","modified":"2021-02-10 11:36:46.502195","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-02-10T11:36:47.132 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 57 2021-02-10T11:36:47.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.127 7fbd14972700 1 -- 172.21.2.101:0/1695884484 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbcf40186c0 msgr2=0x7fbcf401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:47.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.127 7fbd14972700 1 --2- 172.21.2.101:0/1695884484 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbcf40186c0 0x7fbcf401abc0 crc :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:47.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.127 7fbd14972700 1 -- 172.21.2.101:0/1695884484 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd100ada90 msgr2=0x7fbd100ade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:47.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.127 7fbd14972700 1 --2- 172.21.2.101:0/1695884484 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd100ada90 0x7fbd100ade90 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 rx=0x7fbcf800b530 tx=0x7fbcf800b8d0).stop 2021-02-10T11:36:47.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.127 7fbd14972700 1 -- 172.21.2.101:0/1695884484 shutdown_connections 2021-02-10T11:36:47.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.127 7fbd14972700 1 --2- 172.21.2.101:0/1695884484 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbcf40186c0 0x7fbcf401abc0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:47.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.127 7fbd14972700 1 --2- 172.21.2.101:0/1695884484 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd1009a740 0x7fbd10097bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:47.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.127 7fbd14972700 1 --2- 172.21.2.101:0/1695884484 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbd10099d20 0x7fbd1009a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:47.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.127 7fbd14972700 1 --2- 172.21.2.101:0/1695884484 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd100ada90 0x7fbd100ade90 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:47.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.127 7fbd14972700 1 -- 172.21.2.101:0/1695884484 >> 172.21.2.101:0/1695884484 conn(0x7fbd100f0550 msgr2=0x7fbd10191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:47.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.127 7fbd14972700 1 -- 172.21.2.101:0/1695884484 shutdown_connections 2021-02-10T11:36:47.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.127 7fbd14972700 1 -- 172.21.2.101:0/1695884484 wait complete. 2021-02-10T11:36:47.141 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-02-10T11:36:47.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.590 7f8c1820b700 1 Processor -- start 2021-02-10T11:36:47.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.590 7f8c1820b700 1 -- start start 2021-02-10T11:36:47.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.591 7f8c1820b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8c100d1870 0x7f8c101b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:47.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.591 7f8c1820b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c100f0b10 con 0x7f8c100f0650 2021-02-10T11:36:47.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.591 7f8c1820b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c100d0310 con 0x7f8c101b0a10 2021-02-10T11:36:47.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.591 7f8c1820b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c100d0530 con 0x7f8c100d1870 2021-02-10T11:36:47.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.591 7f8c16828700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8c100f0650 0x7f8c100cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54700/0 (socket says 172.21.2.101:54700) 2021-02-10T11:36:47.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.591 7f8c16828700 1 -- 172.21.2.101:0/1166441824 learned_addr learned my addr 172.21.2.101:0/1166441824 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:47.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.591 7f8c16027700 1 --2- 172.21.2.101:0/1166441824 >> v2:172.21.2.108:3300/0 conn(0x7f8c100d1870 0x7f8c101b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:47.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.592 7f8c15025700 1 -- 172.21.2.101:0/1166441824 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1732798648 0 0) 0x7f8c08000f90 con 0x7f8c100f0650 2021-02-10T11:36:47.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.592 7f8c15025700 1 -- 172.21.2.101:0/1166441824 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bfc002fe0 con 0x7f8c100f0650 2021-02-10T11:36:47.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.592 7f8c15025700 1 -- 172.21.2.101:0/1166441824 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 157792680 0 0) 0x7f8c0c000c60 con 0x7f8c101b0a10 2021-02-10T11:36:47.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.592 7f8c15025700 1 -- 172.21.2.101:0/1166441824 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bfc004370 con 0x7f8c101b0a10 2021-02-10T11:36:47.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.592 7f8c16027700 1 -- 172.21.2.101:0/1166441824 >> v1:172.21.2.108:6789/0 conn(0x7f8c101b0a10 legacy=0x7f8c100cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:47.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.592 7f8c16027700 1 -- 172.21.2.101:0/1166441824 >> v1:172.21.2.101:6789/0 conn(0x7f8c100f0650 legacy=0x7f8c100cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:47.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.592 7f8c16027700 1 -- 172.21.2.101:0/1166441824 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c100d1c70 con 0x7f8c100d1870 2021-02-10T11:36:47.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.592 7f8c16027700 1 --2- 172.21.2.101:0/1166441824 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c100d1870 0x7f8c101b03f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f8c00006650 tx=0x7f8c00009a80).ready entity=mon.1 client_cookie=b60a6885f28190a7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:47.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.592 7f8c15025700 1 -- 172.21.2.101:0/1166441824 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c000009b0 con 0x7f8c100d1870 2021-02-10T11:36:47.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.592 7f8c15025700 1 -- 172.21.2.101:0/1166441824 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0000c0c0 con 0x7f8c100d1870 2021-02-10T11:36:47.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.592 7f8c15025700 1 -- 172.21.2.101:0/1166441824 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c0000c520 con 0x7f8c100d1870 2021-02-10T11:36:47.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.592 7f8c1820b700 1 -- 172.21.2.101:0/1166441824 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c100d1870 msgr2=0x7f8c101b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:47.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.592 7f8c1820b700 1 --2- 172.21.2.101:0/1166441824 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c100d1870 0x7f8c101b03f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f8c00006650 tx=0x7f8c00009a80).stop 2021-02-10T11:36:47.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.593 7f8c1820b700 1 -- 172.21.2.101:0/1166441824 shutdown_connections 2021-02-10T11:36:47.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.593 7f8c1820b700 1 --2- 172.21.2.101:0/1166441824 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c100d1870 0x7f8c101b03f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:47.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.593 7f8c1820b700 1 -- 172.21.2.101:0/1166441824 >> 172.21.2.101:0/1166441824 conn(0x7f8c100ce450 msgr2=0x7f8c100d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:47.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.593 7f8c1820b700 1 -- 172.21.2.101:0/1166441824 shutdown_connections 2021-02-10T11:36:47.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.593 7f8c1820b700 1 -- 172.21.2.101:0/1166441824 wait complete. 2021-02-10T11:36:47.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.608 7f8c1820b700 1 Processor -- start 2021-02-10T11:36:47.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.608 7f8c1820b700 1 -- start start 2021-02-10T11:36:47.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.608 7f8c1820b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c101a4a80 0x7f8c100906b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:47.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.608 7f8c1820b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c100c30f0 0x7f8c100c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:47.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.608 7f8c1820b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c100ef5a0 0x7f8c100ef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:47.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.608 7f8c1820b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c100c3b10 con 0x7f8c101a4a80 2021-02-10T11:36:47.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.608 7f8c1820b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c100ef020 con 0x7f8c100ef5a0 2021-02-10T11:36:47.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.608 7f8c1820b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c100effc0 con 0x7f8c100c30f0 2021-02-10T11:36:47.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.608 7f8c16027700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c101a4a80 0x7f8c100906b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:47.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.608 7f8c16027700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c101a4a80 0x7f8c100906b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54210/0 (socket says 172.21.2.101:54210) 2021-02-10T11:36:47.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.608 7f8c16027700 1 -- 172.21.2.101:0/1528257410 learned_addr learned my addr 172.21.2.101:0/1528257410 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:47.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.609 7f8c16027700 1 -- 172.21.2.101:0/1528257410 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c100c30f0 msgr2=0x7f8c100c34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:47.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.609 7f8c16027700 1 --2- 172.21.2.101:0/1528257410 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c100c30f0 0x7f8c100c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:47.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.609 7f8c16027700 1 -- 172.21.2.101:0/1528257410 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c100ef5a0 msgr2=0x7f8c100ef9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:47.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.609 7f8c16027700 1 --2- 172.21.2.101:0/1528257410 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c100ef5a0 0x7f8c100ef9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:47.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.609 7f8c16027700 1 -- 172.21.2.101:0/1528257410 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c100ebe70 con 0x7f8c101a4a80 2021-02-10T11:36:47.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.609 7f8c16027700 1 --2- 172.21.2.101:0/1528257410 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c101a4a80 0x7f8c100906b0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 rx=0x7f8c00009630 tx=0x7f8c0000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:47.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.609 7f8c06ffd700 1 -- 172.21.2.101:0/1528257410 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c000093a0 con 0x7f8c101a4a80 2021-02-10T11:36:47.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.609 7f8c06ffd700 1 -- 172.21.2.101:0/1528257410 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0000bd70 con 0x7f8c101a4a80 2021-02-10T11:36:47.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.609 7f8c06ffd700 1 -- 172.21.2.101:0/1528257410 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c00005560 con 0x7f8c101a4a80 2021-02-10T11:36:47.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.609 7f8c1820b700 1 -- 172.21.2.101:0/1528257410 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c100ebe70 con 0x7f8c101a4a80 2021-02-10T11:36:47.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.609 7f8c1820b700 1 -- 172.21.2.101:0/1528257410 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c100ec400 con 0x7f8c101a4a80 2021-02-10T11:36:47.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.610 7f8c06ffd700 1 -- 172.21.2.101:0/1528257410 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8c0000a000 con 0x7f8c101a4a80 2021-02-10T11:36:47.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.610 7f8c06ffd700 1 --2- 172.21.2.101:0/1528257410 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8bf80186c0 0x7f8bf801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:47.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.610 7f8c06ffd700 1 -- 172.21.2.101:0/1528257410 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5675+0+0 (secure 0 0 0) 0x7f8c000325c0 con 0x7f8c101a4a80 2021-02-10T11:36:47.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.610 7f8c15826700 1 --2- 172.21.2.101:0/1528257410 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8bf80186c0 0x7f8bf801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:47.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.610 7f8c15826700 1 --2- 172.21.2.101:0/1528257410 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8bf80186c0 0x7f8bf801abc0 crc :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:47.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.641 7f8c1820b700 1 -- 172.21.2.101:0/1528257410 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c100f0b10 con 0x7f8c101a4a80 2021-02-10T11:36:47.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.643 7f8c06ffd700 1 -- 172.21.2.101:0/1528257410 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8c00025ff0 con 0x7f8c101a4a80 2021-02-10T11:36:47.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:47.772 7f8c1820b700 1 -- 172.21.2.101:0/1528257410 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8c100f0b10 con 0x7f8c101a4a80 2021-02-10T11:36:48.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.512 7f8c06ffd700 1 -- 172.21.2.101:0/1528257410 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v58) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8c00025ff0 con 0x7f8c101a4a80 2021-02-10T11:36:48.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.542 7f8c1820b700 1 -- 172.21.2.101:0/1528257410 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8bf80186c0 msgr2=0x7f8bf801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:48.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.543 7f8c1820b700 1 --2- 172.21.2.101:0/1528257410 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8bf80186c0 0x7f8bf801abc0 crc :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:48.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.543 7f8c1820b700 1 -- 172.21.2.101:0/1528257410 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c101a4a80 msgr2=0x7f8c100906b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:48.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.543 7f8c1820b700 1 --2- 172.21.2.101:0/1528257410 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c101a4a80 0x7f8c100906b0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 rx=0x7f8c00009630 tx=0x7f8c0000b980).stop 2021-02-10T11:36:48.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.543 7f8c1820b700 1 -- 172.21.2.101:0/1528257410 shutdown_connections 2021-02-10T11:36:48.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.543 7f8c1820b700 1 --2- 172.21.2.101:0/1528257410 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8bf80186c0 0x7f8bf801abc0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:48.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.543 7f8c1820b700 1 --2- 172.21.2.101:0/1528257410 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c100c30f0 0x7f8c100c34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:48.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.543 7f8c1820b700 1 --2- 172.21.2.101:0/1528257410 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c100ef5a0 0x7f8c100ef9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:48.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.543 7f8c1820b700 1 --2- 172.21.2.101:0/1528257410 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c101a4a80 0x7f8c100906b0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:48.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.543 7f8c1820b700 1 -- 172.21.2.101:0/1528257410 >> 172.21.2.101:0/1528257410 conn(0x7f8c100a6bb0 msgr2=0x7f8c10095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:48.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.543 7f8c1820b700 1 -- 172.21.2.101:0/1528257410 shutdown_connections 2021-02-10T11:36:48.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.543 7f8c1820b700 1 -- 172.21.2.101:0/1528257410 wait complete. 2021-02-10T11:36:48.558 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-02-10T11:36:48.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.966 7fd442bee700 1 Processor -- start 2021-02-10T11:36:48.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.966 7fd442bee700 1 -- start start 2021-02-10T11:36:48.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.967 7fd442bee700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd43c0d1880 0x7fd43c1b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:48.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.967 7fd442bee700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd43c0cff80 0x7fd43c109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:48.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.967 7fd442bee700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd43c109a70 con 0x7fd43c1b0af0 2021-02-10T11:36:48.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.967 7fd442bee700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43c109760 con 0x7fd43c0cff80 2021-02-10T11:36:48.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.967 7fd442bee700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c088070 con 0x7fd43c0d1880 2021-02-10T11:36:48.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.967 7fd44120b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd43c0cff80 0x7fd43c109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:48.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.967 7fd44120b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd43c0cff80 0x7fd43c109140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54216/0 (socket says 172.21.2.101:54216) 2021-02-10T11:36:48.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.967 7fd44120b700 1 -- 172.21.2.101:0/250358915 learned_addr learned my addr 172.21.2.101:0/250358915 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:48.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.967 7fd43b7fe700 1 -- 172.21.2.101:0/250358915 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249506735 0 0) 0x7fd42c001060 con 0x7fd43c1b0af0 2021-02-10T11:36:48.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.967 7fd43b7fe700 1 -- 172.21.2.101:0/250358915 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd428002fe0 con 0x7fd43c1b0af0 2021-02-10T11:36:48.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.967 7fd44120b700 1 -- 172.21.2.101:0/250358915 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd43c0d1880 msgr2=0x7fd43c1b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:48.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.967 7fd44120b700 1 --2- 172.21.2.101:0/250358915 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd43c0d1880 0x7fd43c1b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:48.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.967 7fd44120b700 1 -- 172.21.2.101:0/250358915 >> v1:172.21.2.101:6789/0 conn(0x7fd43c1b0af0 legacy=0x7fd43c108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:48.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.967 7fd44120b700 1 -- 172.21.2.101:0/250358915 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43c088260 con 0x7fd43c0cff80 2021-02-10T11:36:48.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.967 7fd44120b700 1 --2- 172.21.2.101:0/250358915 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd43c0cff80 0x7fd43c109140 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 rx=0x7fd434006b80 tx=0x7fd434009b10).ready entity=mon.0 client_cookie=81e7af829bcad404 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:48.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.968 7fd43b7fe700 1 -- 172.21.2.101:0/250358915 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd4340022e0 con 0x7fd43c0cff80 2021-02-10T11:36:48.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.968 7fd43b7fe700 1 -- 172.21.2.101:0/250358915 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd434002620 con 0x7fd43c0cff80 2021-02-10T11:36:48.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.968 7fd43b7fe700 1 -- 172.21.2.101:0/250358915 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd434002b40 con 0x7fd43c0cff80 2021-02-10T11:36:48.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.968 7fd442bee700 1 -- 172.21.2.101:0/250358915 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd43c0cff80 msgr2=0x7fd43c109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:48.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.968 7fd442bee700 1 --2- 172.21.2.101:0/250358915 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd43c0cff80 0x7fd43c109140 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 rx=0x7fd434006b80 tx=0x7fd434009b10).stop 2021-02-10T11:36:48.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.968 7fd442bee700 1 -- 172.21.2.101:0/250358915 shutdown_connections 2021-02-10T11:36:48.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.968 7fd442bee700 1 --2- 172.21.2.101:0/250358915 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd43c0cff80 0x7fd43c109140 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:48.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.968 7fd442bee700 1 --2- 172.21.2.101:0/250358915 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd43c0d1880 0x7fd43c1b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:48.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.968 7fd442bee700 1 -- 172.21.2.101:0/250358915 >> 172.21.2.101:0/250358915 conn(0x7fd43c0ce430 msgr2=0x7fd43c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:48.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.968 7fd442bee700 1 -- 172.21.2.101:0/250358915 shutdown_connections 2021-02-10T11:36:48.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.968 7fd442bee700 1 -- 172.21.2.101:0/250358915 wait complete. 2021-02-10T11:36:48.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.983 7fd442bee700 1 Processor -- start 2021-02-10T11:36:48.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.983 7fd442bee700 1 -- start start 2021-02-10T11:36:48.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.983 7fd442bee700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd43c09a690 0x7fd43c09aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:48.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.983 7fd442bee700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd43c1b4160 0x7fd43c1b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:48.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.983 7fd442bee700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd43c1b7dd0 0x7fd43c1b81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:48.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.983 7fd442bee700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c1b4b80 con 0x7fd43c1b7dd0 2021-02-10T11:36:48.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.983 7fd442bee700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c1b87f0 con 0x7fd43c09a690 2021-02-10T11:36:48.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.983 7fd442bee700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c1b8a10 con 0x7fd43c1b4160 2021-02-10T11:36:48.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.983 7fd44120b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd43c1b7dd0 0x7fd43c1b81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:48.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.983 7fd44120b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd43c1b7dd0 0x7fd43c1b81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54224/0 (socket says 172.21.2.101:54224) 2021-02-10T11:36:48.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.983 7fd44120b700 1 -- 172.21.2.101:0/1527189360 learned_addr learned my addr 172.21.2.101:0/1527189360 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:48.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.984 7fd44120b700 1 -- 172.21.2.101:0/1527189360 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd43c1b4160 msgr2=0x7fd43c1b4560 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:48.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.984 7fd44120b700 1 --2- 172.21.2.101:0/1527189360 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd43c1b4160 0x7fd43c1b4560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:48.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.984 7fd44120b700 1 -- 172.21.2.101:0/1527189360 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd43c09a690 msgr2=0x7fd43c09aa90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:48.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.984 7fd44120b700 1 --2- 172.21.2.101:0/1527189360 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd43c09a690 0x7fd43c09aa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:48.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.984 7fd44120b700 1 -- 172.21.2.101:0/1527189360 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43c1b8c30 con 0x7fd43c1b7dd0 2021-02-10T11:36:48.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.984 7fd44120b700 1 --2- 172.21.2.101:0/1527189360 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd43c1b7dd0 0x7fd43c1b81d0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 rx=0x7fd434009640 tx=0x7fd43400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:48.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.984 7fd4397fa700 1 -- 172.21.2.101:0/1527189360 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd434009100 con 0x7fd43c1b7dd0 2021-02-10T11:36:48.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.984 7fd4397fa700 1 -- 172.21.2.101:0/1527189360 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43400bde0 con 0x7fd43c1b7dd0 2021-02-10T11:36:48.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.984 7fd4397fa700 1 -- 172.21.2.101:0/1527189360 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd434006470 con 0x7fd43c1b7dd0 2021-02-10T11:36:48.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.984 7fd442bee700 1 -- 172.21.2.101:0/1527189360 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd43c1b8c30 con 0x7fd43c1b7dd0 2021-02-10T11:36:48.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.984 7fd442bee700 1 -- 172.21.2.101:0/1527189360 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd43c1b91c0 con 0x7fd43c1b7dd0 2021-02-10T11:36:48.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.985 7fd4397fa700 1 -- 172.21.2.101:0/1527189360 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd434006030 con 0x7fd43c1b7dd0 2021-02-10T11:36:48.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.985 7fd4397fa700 1 --2- 172.21.2.101:0/1527189360 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4200186c0 0x7fd42001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:48.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.985 7fd4397fa700 1 -- 172.21.2.101:0/1527189360 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5675+0+0 (secure 0 0 0) 0x7fd4340325b0 con 0x7fd43c1b7dd0 2021-02-10T11:36:48.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.985 7fd440a0a700 1 --2- 172.21.2.101:0/1527189360 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4200186c0 0x7fd42001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:48.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:48.985 7fd440a0a700 1 --2- 172.21.2.101:0/1527189360 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4200186c0 0x7fd42001abc0 crc :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:49.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.016 7fd442bee700 1 -- 172.21.2.101:0/1527189360 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd43c09a160 con 0x7fd43c1b7dd0 2021-02-10T11:36:49.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.019 7fd4397fa700 1 -- 172.21.2.101:0/1527189360 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd434009100 con 0x7fd43c1b7dd0 2021-02-10T11:36:49.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.145 7fd442bee700 1 -- 172.21.2.101:0/1527189360 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd43c09a160 con 0x7fd43c1b7dd0 2021-02-10T11:36:49.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.516 7fd4397fa700 1 -- 172.21.2.101:0/1527189360 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v78) v1 ==== 185+0+0 (secure 0 0 0) 0x7fd434006030 con 0x7fd43c1b7dd0 2021-02-10T11:36:49.567 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' removed 2021-02-10T11:36:49.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.563 7fd442bee700 1 -- 172.21.2.101:0/1527189360 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4200186c0 msgr2=0x7fd42001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:49.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.563 7fd442bee700 1 --2- 172.21.2.101:0/1527189360 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4200186c0 0x7fd42001abc0 crc :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:49.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.563 7fd442bee700 1 -- 172.21.2.101:0/1527189360 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd43c1b7dd0 msgr2=0x7fd43c1b81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:49.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.563 7fd442bee700 1 --2- 172.21.2.101:0/1527189360 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd43c1b7dd0 0x7fd43c1b81d0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 rx=0x7fd434009640 tx=0x7fd43400b960).stop 2021-02-10T11:36:49.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.563 7fd442bee700 1 -- 172.21.2.101:0/1527189360 shutdown_connections 2021-02-10T11:36:49.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.563 7fd442bee700 1 --2- 172.21.2.101:0/1527189360 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4200186c0 0x7fd42001abc0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:49.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.563 7fd442bee700 1 --2- 172.21.2.101:0/1527189360 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd43c1b4160 0x7fd43c1b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:49.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.563 7fd442bee700 1 --2- 172.21.2.101:0/1527189360 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd43c1b7dd0 0x7fd43c1b81d0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:49.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.563 7fd442bee700 1 --2- 172.21.2.101:0/1527189360 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd43c09a690 0x7fd43c09aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:49.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.563 7fd442bee700 1 -- 172.21.2.101:0/1527189360 >> 172.21.2.101:0/1527189360 conn(0x7fd43c18e870 msgr2=0x7fd43c18ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:49.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.563 7fd442bee700 1 -- 172.21.2.101:0/1527189360 shutdown_connections 2021-02-10T11:36:49.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.563 7fd442bee700 1 -- 172.21.2.101:0/1527189360 wait complete. 2021-02-10T11:36:49.578 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-02-10T11:36:50.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.986 7f5348843700 1 Processor -- start 2021-02-10T11:36:50.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.986 7f5348843700 1 -- start start 2021-02-10T11:36:50.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.987 7f5348843700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f53440d1c80 0x7f53440cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:50.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.987 7f5348843700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53440cf590 0x7f53440cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:50.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.987 7f5348843700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53440fe1d0 con 0x7f53440d1880 2021-02-10T11:36:50.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.987 7f5348843700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53441b0cc0 con 0x7f53440d1c80 2021-02-10T11:36:50.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.987 7f5348843700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5344109a70 con 0x7f53440cf590 2021-02-10T11:36:50.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.987 7f5341e1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f53440d1c80 0x7f53440cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:50.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.987 7f5341e1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f53440d1c80 0x7f53440cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54228/0 (socket says 172.21.2.101:54228) 2021-02-10T11:36:50.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.987 7f5341e1b700 1 -- 172.21.2.101:0/3841985792 learned_addr learned my addr 172.21.2.101:0/3841985792 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:50.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.987 7f5342e1d700 1 --2- 172.21.2.101:0/3841985792 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53440cf590 0x7f53440cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:50.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.987 7f534161a700 1 -- 172.21.2.101:0/3841985792 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 368172852 0 0) 0x7f5334001080 con 0x7f53440d1880 2021-02-10T11:36:50.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.987 7f5341e1b700 1 -- 172.21.2.101:0/3841985792 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53440cf590 msgr2=0x7f53440cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:50.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.987 7f5341e1b700 1 --2- 172.21.2.101:0/3841985792 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53440cf590 0x7f53440cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:50.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.987 7f5341e1b700 1 -- 172.21.2.101:0/3841985792 >> v1:172.21.2.108:6789/0 conn(0x7f53440d1880 legacy=0x7f53441b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:50.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.987 7f5341e1b700 1 -- 172.21.2.101:0/3841985792 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5344088070 con 0x7f53440d1c80 2021-02-10T11:36:50.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.987 7f5342e1d700 1 --2- 172.21.2.101:0/3841985792 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53440cf590 0x7f53440cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:36:50.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.988 7f5341e1b700 1 --2- 172.21.2.101:0/3841985792 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53440d1c80 0x7f53440cef70 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 rx=0x7f5338006700 tx=0x7f5338009b80).ready entity=mon.0 client_cookie=11d8f84dafc7c2dc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:50.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.988 7f534161a700 1 -- 172.21.2.101:0/3841985792 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5338006340 con 0x7f53440d1c80 2021-02-10T11:36:50.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.988 7f534161a700 1 -- 172.21.2.101:0/3841985792 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533800c0c0 con 0x7f53440d1c80 2021-02-10T11:36:50.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.988 7f534161a700 1 -- 172.21.2.101:0/3841985792 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f533800c510 con 0x7f53440d1c80 2021-02-10T11:36:50.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.988 7f5348843700 1 -- 172.21.2.101:0/3841985792 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53440d1c80 msgr2=0x7f53440cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:50.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.988 7f5348843700 1 --2- 172.21.2.101:0/3841985792 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53440d1c80 0x7f53440cef70 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 rx=0x7f5338006700 tx=0x7f5338009b80).stop 2021-02-10T11:36:50.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.989 7f5348843700 1 -- 172.21.2.101:0/3841985792 shutdown_connections 2021-02-10T11:36:50.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.989 7f5348843700 1 --2- 172.21.2.101:0/3841985792 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53440cf590 0x7f53440cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:50.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.989 7f5348843700 1 --2- 172.21.2.101:0/3841985792 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53440d1c80 0x7f53440cef70 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:50.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.989 7f5348843700 1 -- 172.21.2.101:0/3841985792 >> 172.21.2.101:0/3841985792 conn(0x7f53440ce430 msgr2=0x7f53440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:50.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.989 7f5348843700 1 -- 172.21.2.101:0/3841985792 shutdown_connections 2021-02-10T11:36:50.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:49.989 7f5348843700 1 -- 172.21.2.101:0/3841985792 wait complete. 2021-02-10T11:36:50.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.003 7f5348843700 1 Processor -- start 2021-02-10T11:36:50.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.003 7f5348843700 1 -- start start 2021-02-10T11:36:50.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.004 7f5348843700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53441a3570 0x7f53441a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:50.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.004 7f5348843700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f534409c9f0 0x7f534409cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:50.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.004 7f5348843700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f534409d410 0x7f5344098170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:50.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.004 7f5348843700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534409c470 con 0x7f534409c9f0 2021-02-10T11:36:50.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.004 7f5348843700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53440b2840 con 0x7f534409d410 2021-02-10T11:36:50.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.004 7f5348843700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53440b2a20 con 0x7f53441a3570 2021-02-10T11:36:50.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.004 7f5341e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f534409c9f0 0x7f534409cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:50.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.004 7f5341e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f534409c9f0 0x7f534409cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54234/0 (socket says 172.21.2.101:54234) 2021-02-10T11:36:50.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.004 7f5341e1b700 1 -- 172.21.2.101:0/2551333709 learned_addr learned my addr 172.21.2.101:0/2551333709 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:50.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.004 7f534261c700 1 --2- 172.21.2.101:0/2551333709 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53441a3570 0x7f53441a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:50.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.004 7f5342e1d700 1 --2- 172.21.2.101:0/2551333709 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f534409d410 0x7f5344098170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:50.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.004 7f534261c700 1 -- 172.21.2.101:0/2551333709 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f534409d410 msgr2=0x7f5344098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:50.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.004 7f534261c700 1 --2- 172.21.2.101:0/2551333709 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f534409d410 0x7f5344098170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:50.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.004 7f534261c700 1 -- 172.21.2.101:0/2551333709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f534409c9f0 msgr2=0x7f534409cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:50.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.004 7f534261c700 1 --2- 172.21.2.101:0/2551333709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f534409c9f0 0x7f534409cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:50.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.004 7f534261c700 1 -- 172.21.2.101:0/2551333709 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534409d880 con 0x7f53441a3570 2021-02-10T11:36:50.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.004 7f5341e1b700 1 --2- 172.21.2.101:0/2551333709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f534409c9f0 0x7f534409cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:36:50.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.004 7f534261c700 1 --2- 172.21.2.101:0/2551333709 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53441a3570 0x7f53441a3970 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f5334006e90 tx=0x7f533400b860).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:50.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.004 7f5332ffd700 1 -- 172.21.2.101:0/2551333709 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5334000bc0 con 0x7f53441a3570 2021-02-10T11:36:50.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.004 7f5332ffd700 1 -- 172.21.2.101:0/2551333709 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5334000e20 con 0x7f53441a3570 2021-02-10T11:36:50.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.004 7f5348843700 1 -- 172.21.2.101:0/2551333709 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f534409d880 con 0x7f53441a3570 2021-02-10T11:36:50.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.004 7f5332ffd700 1 -- 172.21.2.101:0/2551333709 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f53340022e0 con 0x7f53441a3570 2021-02-10T11:36:50.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.005 7f5348843700 1 -- 172.21.2.101:0/2551333709 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534409d880 con 0x7f53441a3570 2021-02-10T11:36:50.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.005 7f5332ffd700 1 -- 172.21.2.101:0/2551333709 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5334000ae0 con 0x7f53441a3570 2021-02-10T11:36:50.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.005 7f5332ffd700 1 --2- 172.21.2.101:0/2551333709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5328018730 0x7f532801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:50.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.005 7f5332ffd700 1 -- 172.21.2.101:0/2551333709 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5237+0+0 (secure 0 0 0) 0x7f5334032550 con 0x7f53441a3570 2021-02-10T11:36:50.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.005 7f5341e1b700 1 --2- 172.21.2.101:0/2551333709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5328018730 0x7f532801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:50.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.006 7f5341e1b700 1 --2- 172.21.2.101:0/2551333709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5328018730 0x7f532801ac10 crc :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.036 7f5348843700 1 -- 172.21.2.101:0/2551333709 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5344082a70 con 0x7f53441a3570 2021-02-10T11:36:50.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.040 7f5332ffd700 1 -- 172.21.2.101:0/2551333709 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5334000e20 con 0x7f53441a3570 2021-02-10T11:36:50.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.174 7f5348843700 1 -- 172.21.2.101:0/2551333709 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5344082a70 con 0x7f53441a3570 2021-02-10T11:36:50.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.522 7f5332ffd700 1 -- 172.21.2.101:0/2551333709 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v79) v1 ==== 173+0+0 (secure 0 0 0) 0x7f533401f420 con 0x7f53441a3570 2021-02-10T11:36:50.545 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' removed 2021-02-10T11:36:50.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.540 7f5348843700 1 -- 172.21.2.101:0/2551333709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5328018730 msgr2=0x7f532801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:50.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.540 7f5348843700 1 --2- 172.21.2.101:0/2551333709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5328018730 0x7f532801ac10 crc :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:50.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.540 7f5348843700 1 -- 172.21.2.101:0/2551333709 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53441a3570 msgr2=0x7f53441a3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:50.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.540 7f5348843700 1 --2- 172.21.2.101:0/2551333709 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53441a3570 0x7f53441a3970 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f5334006e90 tx=0x7f533400b860).stop 2021-02-10T11:36:50.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.541 7f5348843700 1 -- 172.21.2.101:0/2551333709 shutdown_connections 2021-02-10T11:36:50.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.541 7f5348843700 1 --2- 172.21.2.101:0/2551333709 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f534409d410 0x7f5344098170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:50.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.541 7f5348843700 1 --2- 172.21.2.101:0/2551333709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f534409c9f0 0x7f534409cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:50.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.541 7f5348843700 1 --2- 172.21.2.101:0/2551333709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5328018730 0x7f532801ac10 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:50.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.541 7f5348843700 1 --2- 172.21.2.101:0/2551333709 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53441a3570 0x7f53441a3970 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:50.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.541 7f5348843700 1 -- 172.21.2.101:0/2551333709 >> 172.21.2.101:0/2551333709 conn(0x7f53440c2cd0 msgr2=0x7f53440b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:50.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.541 7f5348843700 1 -- 172.21.2.101:0/2551333709 shutdown_connections 2021-02-10T11:36:50.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:50.541 7f5348843700 1 -- 172.21.2.101:0/2551333709 wait complete. 2021-02-10T11:36:50.556 INFO:tasks.ceph.mds.a:Restarting daemon 2021-02-10T11:36:50.556 INFO:tasks.ceph.mds.a:Stopping old one... 2021-02-10T11:36:50.556 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-02-10T11:36:50.557 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:36:50.557 INFO:tasks.ceph.mds.b:Restarting daemon 2021-02-10T11:36:50.558 INFO:tasks.ceph.mds.b:Stopping old one... 2021-02-10T11:36:50.558 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-02-10T11:36:50.558 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:36:50.563 INFO:tasks.ceph.mds.a:Stopped 2021-02-10T11:36:50.563 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-02-10T11:36:50.564 INFO:tasks.ceph.mds.b:Stopped 2021-02-10T11:36:50.565 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-02-10T11:36:50.605 INFO:tasks.ceph.mds.a:Started 2021-02-10T11:36:50.606 DEBUG:teuthology.parallel:result is None 2021-02-10T11:36:50.609 INFO:tasks.ceph.mds.b:Started 2021-02-10T11:36:50.609 DEBUG:teuthology.parallel:result is None 2021-02-10T11:36:50.609 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd blacklist clear 2021-02-10T11:36:51.013 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:36:51.015 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:36:51.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.011 7fdc71653700 1 Processor -- start 2021-02-10T11:36:51.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.011 7fdc71653700 1 -- start start 2021-02-10T11:36:51.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.012 7fdc71653700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdc6c0d1870 0x7fdc6c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:51.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.012 7fdc71653700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc6c0f0650 0x7fdc6c0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:51.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.012 7fdc71653700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc6c0f0b10 con 0x7fdc6c1b0a10 2021-02-10T11:36:51.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.012 7fdc71653700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc6c0e5ed0 con 0x7fdc6c0d1870 2021-02-10T11:36:51.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.012 7fdc71653700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0d1c70 con 0x7fdc6c0f0650 2021-02-10T11:36:51.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.012 7fdc6b2bf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdc6c0d1870 0x7fdc6c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:51.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.012 7fdc6b2bf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdc6c0d1870 0x7fdc6c1b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54248/0 (socket says 172.21.2.101:54248) 2021-02-10T11:36:51.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.012 7fdc6b2bf700 1 -- 172.21.2.101:0/1151242456 learned_addr learned my addr 172.21.2.101:0/1151242456 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:51.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.012 7fdc6bac0700 1 --2- 172.21.2.101:0/1151242456 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc6c0f0650 0x7fdc6c0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:51.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.012 7fdc6b2bf700 1 -- 172.21.2.101:0/1151242456 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc6c0f0650 msgr2=0x7fdc6c0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:51.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.012 7fdc6b2bf700 1 --2- 172.21.2.101:0/1151242456 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc6c0f0650 0x7fdc6c0cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:51.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.012 7fdc6b2bf700 1 -- 172.21.2.101:0/1151242456 >> v1:172.21.2.108:6789/0 conn(0x7fdc6c1b0a10 legacy=0x7fdc6c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:51.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.012 7fdc6b2bf700 1 -- 172.21.2.101:0/1151242456 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c0d1e90 con 0x7fdc6c0d1870 2021-02-10T11:36:51.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.012 7fdc6b2bf700 1 --2- 172.21.2.101:0/1151242456 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6c0d1870 0x7fdc6c1b03f0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 rx=0x7fdc64006a80 tx=0x7fdc64009e80).ready entity=mon.0 client_cookie=b596bd33a066d2d7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:51.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.012 7fdc6a2bd700 1 -- 172.21.2.101:0/1151242456 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc6400b510 con 0x7fdc6c0d1870 2021-02-10T11:36:51.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.012 7fdc6a2bd700 1 -- 172.21.2.101:0/1151242456 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6400b810 con 0x7fdc6c0d1870 2021-02-10T11:36:51.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.012 7fdc6a2bd700 1 -- 172.21.2.101:0/1151242456 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc6400bcf0 con 0x7fdc6c0d1870 2021-02-10T11:36:51.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.012 7fdc71653700 1 -- 172.21.2.101:0/1151242456 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6c0d1870 msgr2=0x7fdc6c1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:51.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.012 7fdc71653700 1 --2- 172.21.2.101:0/1151242456 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6c0d1870 0x7fdc6c1b03f0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 rx=0x7fdc64006a80 tx=0x7fdc64009e80).stop 2021-02-10T11:36:51.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.013 7fdc71653700 1 -- 172.21.2.101:0/1151242456 shutdown_connections 2021-02-10T11:36:51.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.013 7fdc71653700 1 --2- 172.21.2.101:0/1151242456 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc6c0f0650 0x7fdc6c0cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:51.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.013 7fdc71653700 1 --2- 172.21.2.101:0/1151242456 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6c0d1870 0x7fdc6c1b03f0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:51.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.013 7fdc71653700 1 -- 172.21.2.101:0/1151242456 >> 172.21.2.101:0/1151242456 conn(0x7fdc6c0ce450 msgr2=0x7fdc6c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:51.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.013 7fdc71653700 1 -- 172.21.2.101:0/1151242456 shutdown_connections 2021-02-10T11:36:51.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.013 7fdc71653700 1 -- 172.21.2.101:0/1151242456 wait complete. 2021-02-10T11:36:51.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.028 7fdc71653700 1 Processor -- start 2021-02-10T11:36:51.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.028 7fdc71653700 1 -- start start 2021-02-10T11:36:51.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.028 7fdc71653700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc6c0a5ef0 0x7fdc6c188f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:51.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.028 7fdc71653700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc6c0a62f0 0x7fdc6c189570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:51.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.028 7fdc71653700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6c189b90 0x7fdc6c08cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:51.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.028 7fdc71653700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c08c360 con 0x7fdc6c189b90 2021-02-10T11:36:51.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.028 7fdc71653700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c189f90 con 0x7fdc6c0a5ef0 2021-02-10T11:36:51.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.028 7fdc71653700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c08d1b0 con 0x7fdc6c0a62f0 2021-02-10T11:36:51.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.029 7fdc6bac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6c189b90 0x7fdc6c08cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:51.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.029 7fdc6bac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6c189b90 0x7fdc6c08cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54258/0 (socket says 172.21.2.101:54258) 2021-02-10T11:36:51.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.029 7fdc6bac0700 1 -- 172.21.2.101:0/766328326 learned_addr learned my addr 172.21.2.101:0/766328326 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:51.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.029 7fdc6bac0700 1 -- 172.21.2.101:0/766328326 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc6c0a62f0 msgr2=0x7fdc6c189570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:51.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.029 7fdc6aabe700 1 --2- 172.21.2.101:0/766328326 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc6c0a62f0 0x7fdc6c189570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:51.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.029 7fdc6b2bf700 1 --2- 172.21.2.101:0/766328326 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc6c0a5ef0 0x7fdc6c188f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:51.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.029 7fdc6bac0700 1 --2- 172.21.2.101:0/766328326 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc6c0a62f0 0x7fdc6c189570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:51.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.029 7fdc6bac0700 1 -- 172.21.2.101:0/766328326 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc6c0a5ef0 msgr2=0x7fdc6c188f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:51.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.029 7fdc6bac0700 1 --2- 172.21.2.101:0/766328326 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc6c0a5ef0 0x7fdc6c188f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:51.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.029 7fdc6bac0700 1 -- 172.21.2.101:0/766328326 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c08d3d0 con 0x7fdc6c189b90 2021-02-10T11:36:51.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.029 7fdc6aabe700 1 --2- 172.21.2.101:0/766328326 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc6c0a62f0 0x7fdc6c189570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:36:51.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.029 7fdc6b2bf700 1 --2- 172.21.2.101:0/766328326 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc6c0a5ef0 0x7fdc6c188f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:36:51.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.029 7fdc6bac0700 1 --2- 172.21.2.101:0/766328326 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6c189b90 0x7fdc6c08cb90 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 rx=0x7fdc5c006f20 tx=0x7fdc5c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:51.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.029 7fdc53fff700 1 -- 172.21.2.101:0/766328326 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc5c000c10 con 0x7fdc6c189b90 2021-02-10T11:36:51.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.030 7fdc53fff700 1 -- 172.21.2.101:0/766328326 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5c00bd80 con 0x7fdc6c189b90 2021-02-10T11:36:51.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.030 7fdc53fff700 1 -- 172.21.2.101:0/766328326 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc5c0022b0 con 0x7fdc6c189b90 2021-02-10T11:36:51.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.030 7fdc71653700 1 -- 172.21.2.101:0/766328326 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc6c08d3d0 con 0x7fdc6c189b90 2021-02-10T11:36:51.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.030 7fdc71653700 1 -- 172.21.2.101:0/766328326 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6c08d3d0 con 0x7fdc6c189b90 2021-02-10T11:36:51.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.030 7fdc53fff700 1 -- 172.21.2.101:0/766328326 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdc5c002020 con 0x7fdc6c189b90 2021-02-10T11:36:51.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.030 7fdc53fff700 1 --2- 172.21.2.101:0/766328326 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc4c0186c0 0x7fdc4c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:51.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.030 7fdc53fff700 1 -- 172.21.2.101:0/766328326 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4839+0+0 (secure 0 0 0) 0x7fdc5c0322d0 con 0x7fdc6c189b90 2021-02-10T11:36:51.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.030 7fdc6b2bf700 1 --2- 172.21.2.101:0/766328326 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc4c0186c0 0x7fdc4c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:51.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.030 7fdc6b2bf700 1 --2- 172.21.2.101:0/766328326 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc4c0186c0 0x7fdc4c01abc0 crc :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:51.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.061 7fdc71653700 1 -- 172.21.2.101:0/766328326 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc6c0ee0b0 con 0x7fdc6c189b90 2021-02-10T11:36:51.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.064 7fdc53fff700 1 -- 172.21.2.101:0/766328326 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdc5c006370 con 0x7fdc6c189b90 2021-02-10T11:36:51.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.203 7fdc71653700 1 -- 172.21.2.101:0/766328326 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fdc6c0ce450 con 0x7fdc6c189b90 2021-02-10T11:36:51.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.525 7fdc53fff700 1 -- 172.21.2.101:0/766328326 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v80) v1 ==== 97+0+0 (secure 0 0 0) 0x7fdc5c002000 con 0x7fdc6c189b90 2021-02-10T11:36:51.576 INFO:teuthology.orchestra.run.gibba001.stderr: removed all blacklist entries 2021-02-10T11:36:51.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.572 7fdc71653700 1 -- 172.21.2.101:0/766328326 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc4c0186c0 msgr2=0x7fdc4c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:51.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.572 7fdc71653700 1 --2- 172.21.2.101:0/766328326 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc4c0186c0 0x7fdc4c01abc0 crc :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:51.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.572 7fdc71653700 1 -- 172.21.2.101:0/766328326 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6c189b90 msgr2=0x7fdc6c08cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:51.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.572 7fdc71653700 1 --2- 172.21.2.101:0/766328326 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6c189b90 0x7fdc6c08cb90 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 rx=0x7fdc5c006f20 tx=0x7fdc5c00b990).stop 2021-02-10T11:36:51.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.572 7fdc71653700 1 -- 172.21.2.101:0/766328326 shutdown_connections 2021-02-10T11:36:51.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.572 7fdc71653700 1 --2- 172.21.2.101:0/766328326 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc4c0186c0 0x7fdc4c01abc0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:51.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.572 7fdc71653700 1 --2- 172.21.2.101:0/766328326 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc6c0a62f0 0x7fdc6c189570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:51.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.572 7fdc71653700 1 --2- 172.21.2.101:0/766328326 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6c189b90 0x7fdc6c08cb90 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:51.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.572 7fdc71653700 1 --2- 172.21.2.101:0/766328326 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc6c0a5ef0 0x7fdc6c188f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:51.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.572 7fdc71653700 1 -- 172.21.2.101:0/766328326 >> 172.21.2.101:0/766328326 conn(0x7fdc6c0a6eb0 msgr2=0x7fdc6c098ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:51.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.573 7fdc71653700 1 -- 172.21.2.101:0/766328326 shutdown_connections 2021-02-10T11:36:51.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.573 7fdc71653700 1 -- 172.21.2.101:0/766328326 wait complete. 2021-02-10T11:36:51.587 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2021-02-10T11:36:51.587 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:36:52.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.991 7fe6c7dc8700 1 Processor -- start 2021-02-10T11:36:52.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.991 7fe6c7dc8700 1 -- start start 2021-02-10T11:36:52.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.992 7fe6c7dc8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe6c00d1870 0x7fe6c01b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:52.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.992 7fe6c7dc8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6c01b0a10 0x7fe6c00cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:52.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.992 7fe6c7dc8700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6c00f0b10 con 0x7fe6c00d1c70 2021-02-10T11:36:52.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.992 7fe6c7dc8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c00e5ed0 con 0x7fe6c00d1870 2021-02-10T11:36:52.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.992 7fe6c7dc8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c00f0650 con 0x7fe6c01b0a10 2021-02-10T11:36:52.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.992 7fe6c63e5700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe6c00d1c70 0x7fe6c00cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54756/0 (socket says 172.21.2.101:54756) 2021-02-10T11:36:52.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.992 7fe6c63e5700 1 -- 172.21.2.101:0/4002731263 learned_addr learned my addr 172.21.2.101:0/4002731263 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:52.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.992 7fe6c4be2700 1 -- 172.21.2.101:0/4002731263 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 770082206 0 0) 0x7fe6b8001500 con 0x7fe6c00d1c70 2021-02-10T11:36:52.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.992 7fe6c4be2700 1 -- 172.21.2.101:0/4002731263 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6ac002fe0 con 0x7fe6c00d1c70 2021-02-10T11:36:52.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.992 7fe6c53e3700 1 --2- 172.21.2.101:0/4002731263 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6c01b0a10 0x7fe6c00cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:52.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.992 7fe6c4be2700 1 -- 172.21.2.101:0/4002731263 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1824340110 0 0) 0x7fe6b8001460 con 0x7fe6c00d1c70 2021-02-10T11:36:52.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.992 7fe6c4be2700 1 -- 172.21.2.101:0/4002731263 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6c01b0a10 msgr2=0x7fe6c00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:52.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.992 7fe6c4be2700 1 --2- 172.21.2.101:0/4002731263 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6c01b0a10 0x7fe6c00cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:52.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.992 7fe6c4be2700 1 -- 172.21.2.101:0/4002731263 >> v2:172.21.2.108:3300/0 conn(0x7fe6c00d1870 msgr2=0x7fe6c01b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:52.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.992 7fe6c5be4700 1 --2- 172.21.2.101:0/4002731263 >> v2:172.21.2.108:3300/0 conn(0x7fe6c00d1870 0x7fe6c01b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:52.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.992 7fe6c4be2700 1 --2- 172.21.2.101:0/4002731263 >> v2:172.21.2.108:3300/0 conn(0x7fe6c00d1870 0x7fe6c01b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:52.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.992 7fe6c4be2700 1 -- 172.21.2.101:0/4002731263 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c00f0870 con 0x7fe6c00d1c70 2021-02-10T11:36:52.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.992 7fe6c4be2700 1 -- 172.21.2.101:0/4002731263 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fe6b8001980 con 0x7fe6c00d1c70 2021-02-10T11:36:52.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.992 7fe6c53e3700 1 --2- 172.21.2.101:0/4002731263 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6c01b0a10 0x7fe6c00cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:36:52.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.992 7fe6c4be2700 1 -- 172.21.2.101:0/4002731263 >> v1:172.21.2.101:6789/0 conn(0x7fe6c00d1c70 legacy=0x7fe6c00cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:52.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.992 7fe6c4be2700 1 --2- 172.21.2.101:0/4002731263 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6ac003af0 0x7fe6ac005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:52.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.993 7fe6c4be2700 1 --2- 172.21.2.101:0/4002731263 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6ac006800 0x7fe6ac008d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:52.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.993 7fe6c4be2700 1 --2- 172.21.2.101:0/4002731263 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6ac0095a0 0x7fe6ac00bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:52.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.993 7fe6c4be2700 1 -- 172.21.2.101:0/4002731263 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac00c370 con 0x7fe6ac006800 2021-02-10T11:36:52.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.993 7fe6c4be2700 1 -- 172.21.2.101:0/4002731263 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac00c590 con 0x7fe6ac0095a0 2021-02-10T11:36:52.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.993 7fe6c4be2700 1 -- 172.21.2.101:0/4002731263 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac0507c0 con 0x7fe6ac003af0 2021-02-10T11:36:52.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.993 7fe6c53e3700 1 --2- 172.21.2.101:0/4002731263 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6ac006800 0x7fe6ac008d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:52.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.993 7fe6c5be4700 1 --2- 172.21.2.101:0/4002731263 >> v2:172.21.2.108:3300/0 conn(0x7fe6c00d1870 0x7fe6c01b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:36:52.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.993 7fe6c53e3700 1 -- 172.21.2.101:0/4002731263 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6ac003af0 msgr2=0x7fe6ac005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:52.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.993 7fe6c53e3700 1 --2- 172.21.2.101:0/4002731263 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6ac003af0 0x7fe6ac005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:52.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.993 7fe6c53e3700 1 -- 172.21.2.101:0/4002731263 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6ac0095a0 msgr2=0x7fe6ac00bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:52.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.993 7fe6c53e3700 1 --2- 172.21.2.101:0/4002731263 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6ac0095a0 0x7fe6ac00bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:52.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.993 7fe6c53e3700 1 -- 172.21.2.101:0/4002731263 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ac050a20 con 0x7fe6ac006800 2021-02-10T11:36:52.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.993 7fe6c53e3700 1 --2- 172.21.2.101:0/4002731263 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6ac006800 0x7fe6ac008d00 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 rx=0x7fe6b4006670 tx=0x7fe6b4009ac0).ready entity=mon.0 client_cookie=fc6b57ee650a4b7f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:52.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.993 7fe6c4be2700 1 -- 172.21.2.101:0/4002731263 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe6b400c0c0 con 0x7fe6ac006800 2021-02-10T11:36:52.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.993 7fe6c4be2700 1 -- 172.21.2.101:0/4002731263 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b40063e0 con 0x7fe6ac006800 2021-02-10T11:36:52.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.993 7fe6c4be2700 1 -- 172.21.2.101:0/4002731263 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe6b400c5f0 con 0x7fe6ac006800 2021-02-10T11:36:52.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.993 7fe6c7dc8700 1 -- 172.21.2.101:0/4002731263 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6ac006800 msgr2=0x7fe6ac008d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:52.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.993 7fe6c7dc8700 1 --2- 172.21.2.101:0/4002731263 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6ac006800 0x7fe6ac008d00 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 rx=0x7fe6b4006670 tx=0x7fe6b4009ac0).stop 2021-02-10T11:36:52.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.993 7fe6c63e5700 1 -- 172.21.2.101:0/4002731263 reap_dead start 2021-02-10T11:36:52.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.994 7fe6c7dc8700 1 -- 172.21.2.101:0/4002731263 shutdown_connections 2021-02-10T11:36:52.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.994 7fe6c7dc8700 1 -- 172.21.2.101:0/4002731263 >> 172.21.2.101:0/4002731263 conn(0x7fe6c00ce450 msgr2=0x7fe6c00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:52.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.994 7fe6c7dc8700 1 -- 172.21.2.101:0/4002731263 shutdown_connections 2021-02-10T11:36:52.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:51.994 7fe6c7dc8700 1 -- 172.21.2.101:0/4002731263 wait complete. 2021-02-10T11:36:52.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.008 7fe6c7dc8700 1 Processor -- start 2021-02-10T11:36:52.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.008 7fe6c7dc8700 1 -- start start 2021-02-10T11:36:52.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.009 7fe6c7dc8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6c01942b0 0x7fe6c00b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:52.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.009 7fe6c7dc8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6c0195070 0x7fe6c0195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:52.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.009 7fe6c7dc8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6c00bb8e0 0x7fe6c00bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:52.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.009 7fe6c7dc8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c01946b0 con 0x7fe6c01942b0 2021-02-10T11:36:52.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.009 7fe6c7dc8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c00b3b70 con 0x7fe6c00bb8e0 2021-02-10T11:36:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.009 7fe6c7dc8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c00b3d90 con 0x7fe6c0195070 2021-02-10T11:36:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.009 7fe6c5be4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6c01942b0 0x7fe6c00b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.009 7fe6c5be4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6c01942b0 0x7fe6c00b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54272/0 (socket says 172.21.2.101:54272) 2021-02-10T11:36:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.009 7fe6c5be4700 1 -- 172.21.2.101:0/3375808689 learned_addr learned my addr 172.21.2.101:0/3375808689 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.009 7fe6c5be4700 1 -- 172.21.2.101:0/3375808689 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6c0195070 msgr2=0x7fe6c0195470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.009 7fe6c5be4700 1 --2- 172.21.2.101:0/3375808689 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6c0195070 0x7fe6c0195470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.009 7fe6c5be4700 1 -- 172.21.2.101:0/3375808689 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6c00bb8e0 msgr2=0x7fe6c00bbce0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.009 7fe6c5be4700 1 --2- 172.21.2.101:0/3375808689 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6c00bb8e0 0x7fe6c00bbce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.009 7fe6c5be4700 1 -- 172.21.2.101:0/3375808689 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c01a49e0 con 0x7fe6c01942b0 2021-02-10T11:36:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.009 7fe6c5be4700 1 --2- 172.21.2.101:0/3375808689 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6c01942b0 0x7fe6c00b3550 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 rx=0x7fe6bc00b5e0 tx=0x7fe6bc00b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.010 7fe6b27fc700 1 -- 172.21.2.101:0/3375808689 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe6bc0065d0 con 0x7fe6c01942b0 2021-02-10T11:36:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.010 7fe6b27fc700 1 -- 172.21.2.101:0/3375808689 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6bc0052e0 con 0x7fe6c01942b0 2021-02-10T11:36:52.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.010 7fe6b27fc700 1 -- 172.21.2.101:0/3375808689 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe6bc005800 con 0x7fe6c01942b0 2021-02-10T11:36:52.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.010 7fe6c7dc8700 1 -- 172.21.2.101:0/3375808689 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c01a49e0 con 0x7fe6c01942b0 2021-02-10T11:36:52.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.010 7fe6c7dc8700 1 -- 172.21.2.101:0/3375808689 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c01a49e0 con 0x7fe6c01942b0 2021-02-10T11:36:52.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.010 7fe6b27fc700 1 -- 172.21.2.101:0/3375808689 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe6bc0054c0 con 0x7fe6c01942b0 2021-02-10T11:36:52.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.010 7fe6b27fc700 1 --2- 172.21.2.101:0/3375808689 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6a80186c0 0x7fe6a801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:52.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.011 7fe6b27fc700 1 -- 172.21.2.101:0/3375808689 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fe6bc032240 con 0x7fe6c01942b0 2021-02-10T11:36:52.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.011 7fe6c53e3700 1 --2- 172.21.2.101:0/3375808689 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6a80186c0 0x7fe6a801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:52.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.011 7fe6c53e3700 1 --2- 172.21.2.101:0/3375808689 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6a80186c0 0x7fe6a801abc0 crc :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:52.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.042 7fe6c7dc8700 1 -- 172.21.2.101:0/3375808689 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c01a49e0 con 0x7fe6c01942b0 2021-02-10T11:36:52.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.044 7fe6b27fc700 1 -- 172.21.2.101:0/3375808689 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe6bc030d20 con 0x7fe6c01942b0 2021-02-10T11:36:52.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.179 7fe6c7dc8700 1 -- 172.21.2.101:0/3375808689 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe6c01a49e0 con 0x7fe6c01942b0 2021-02-10T11:36:52.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.180 7fe6b27fc700 1 -- 172.21.2.101:0/3375808689 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v48) v1 ==== 80+0+4513 (secure 0 0 0) 0x7fe6bc0054c0 con 0x7fe6c01942b0 2021-02-10T11:36:52.186 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:36:52.186 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:36:52.186 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:36:52.186 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:52.187 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:36:52.187 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:36:52.187 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:36:52.187 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:36:52.187 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:36:52.187 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:36:52.187 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:36:52.187 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:52.187 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:52.187 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:52.187 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:36:52.187 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:36:52.187 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:36:52.187 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:36:52.188 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:36:52.188 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:36:52.188 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:36:52.188 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:52.188 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:52.188 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:52.188 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:36:52.188 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:36:52.188 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:36:52.188 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:36:52.188 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:36:52.188 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:36:52.188 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:52.188 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:52.189 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:52.189 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:36:52.189 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:36:52.189 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:36:52.189 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:36:52.189 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:36:52.189 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:36:52.189 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:52.189 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:52.189 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:52.189 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:36:52.189 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:36:52.189 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:36:52.189 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:36:52.190 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:36:52.190 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:36:52.190 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:52.190 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:52.190 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:52.190 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:36:52.190 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:36:52.190 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:36:52.190 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:36:52.190 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:36:52.190 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:36:52.190 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:52.190 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:52.191 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:52.191 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:36:52.191 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:36:52.191 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:36:52.191 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:36:52.191 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:36:52.191 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:36:52.191 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:52.191 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:52.191 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:52.191 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:36:52.191 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:36:52.191 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:36:52.191 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:36:52.192 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:36:52.192 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:36:52.192 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:52.192 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:52.192 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:52.192 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:36:52.192 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:36:52.192 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:36:52.192 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:36:52.192 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:36:52.192 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:36:52.192 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:52.192 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:52.193 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:52.193 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:36:52.193 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:36:52.193 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:36:52.193 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:36:52.193 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:36:52.193 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:36:52.193 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:52.193 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:52.193 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:52.193 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:36:52.193 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:36:52.194 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:36:52.194 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:36:52.194 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:36:52.194 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:36:52.194 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:52.194 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:52.194 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:52.194 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:36:52.194 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:36:52.194 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:36:52.195 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:36:52.195 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:36:52.195 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:36:52.195 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:52.195 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:52.195 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:52.195 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:36:52.195 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:36:52.195 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:36:52.195 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:36:52.195 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:36:52.195 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:36:52.195 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:52.195 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:52.196 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:52.196 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:36:52.196 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:36:52.196 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:36:52.196 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:36:52.196 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:36:52.196 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:36:52.196 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:52.196 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:52.196 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:52.196 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:36:52.196 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:36:52.196 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:36:52.196 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:36:52.197 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:36:52.197 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:36:52.197 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:36:52.197 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:52.197 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:52.197 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:52.197 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:36:52.197 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:36:52.197 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:36:52.197 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:36:52.197 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:36:52.197 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:36:52.197 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:52.197 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:36:52.198 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:52.198 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:36:52.198 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:36:52.198 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:36:52.198 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:36:52.198 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:36:52.198 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:36:52.198 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:52.198 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:52.198 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:36:52.198 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:36:52.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.182 7fe6c7dc8700 1 -- 172.21.2.101:0/3375808689 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6a80186c0 msgr2=0x7fe6a801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:52.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.182 7fe6c7dc8700 1 --2- 172.21.2.101:0/3375808689 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6a80186c0 0x7fe6a801abc0 crc :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:52.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.182 7fe6c7dc8700 1 -- 172.21.2.101:0/3375808689 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6c01942b0 msgr2=0x7fe6c00b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:52.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.182 7fe6c7dc8700 1 --2- 172.21.2.101:0/3375808689 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6c01942b0 0x7fe6c00b3550 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 rx=0x7fe6bc00b5e0 tx=0x7fe6bc00b980).stop 2021-02-10T11:36:52.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.182 7fe6c7dc8700 1 -- 172.21.2.101:0/3375808689 shutdown_connections 2021-02-10T11:36:52.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.182 7fe6c7dc8700 1 --2- 172.21.2.101:0/3375808689 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6a80186c0 0x7fe6a801abc0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:52.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.182 7fe6c7dc8700 1 --2- 172.21.2.101:0/3375808689 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6c0195070 0x7fe6c0195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:52.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.182 7fe6c7dc8700 1 --2- 172.21.2.101:0/3375808689 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6c00bb8e0 0x7fe6c00bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:52.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.182 7fe6c7dc8700 1 --2- 172.21.2.101:0/3375808689 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6c01942b0 0x7fe6c00b3550 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:52.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.182 7fe6c7dc8700 1 -- 172.21.2.101:0/3375808689 >> 172.21.2.101:0/3375808689 conn(0x7fe6c00b3040 msgr2=0x7fe6c00c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:52.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.182 7fe6c7dc8700 1 -- 172.21.2.101:0/3375808689 shutdown_connections 2021-02-10T11:36:52.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.182 7fe6c7dc8700 1 -- 172.21.2.101:0/3375808689 wait complete. 2021-02-10T11:36:52.201 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:36:52.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.607 7fc8e7c0f700 1 Processor -- start 2021-02-10T11:36:52.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.607 7fc8e7c0f700 1 -- start start 2021-02-10T11:36:52.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.608 7fc8e7c0f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc8e00d1870 0x7fc8e01b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:52.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.608 7fc8e7c0f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e00f0b10 con 0x7fc8e01b0a10 2021-02-10T11:36:52.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.608 7fc8e7c0f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e00d0310 con 0x7fc8e00f0650 2021-02-10T11:36:52.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.608 7fc8e7c0f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e00d0530 con 0x7fc8e00d1870 2021-02-10T11:36:52.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.608 7fc8e522a700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc8e01b0a10 0x7fc8e00cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54774/0 (socket says 172.21.2.101:54774) 2021-02-10T11:36:52.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.608 7fc8e522a700 1 -- 172.21.2.101:0/658892172 learned_addr learned my addr 172.21.2.101:0/658892172 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:52.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.608 7fc8e5a2b700 1 --2- 172.21.2.101:0/658892172 >> v2:172.21.2.108:3300/0 conn(0x7fc8e00d1870 0x7fc8e01b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:52.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.608 7fc8e4a29700 1 -- 172.21.2.101:0/658892172 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2444665221 0 0) 0x7fc8dc000ce0 con 0x7fc8e01b0a10 2021-02-10T11:36:52.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.608 7fc8e4a29700 1 -- 172.21.2.101:0/658892172 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8cc002fe0 con 0x7fc8e01b0a10 2021-02-10T11:36:52.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e4a29700 1 -- 172.21.2.101:0/658892172 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1771614864 0 0) 0x7fc8dc0010e0 con 0x7fc8e01b0a10 2021-02-10T11:36:52.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e4a29700 1 -- 172.21.2.101:0/658892172 >> v2:172.21.2.108:3300/0 conn(0x7fc8e00d1870 msgr2=0x7fc8e01b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:52.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e4a29700 1 --2- 172.21.2.101:0/658892172 >> v2:172.21.2.108:3300/0 conn(0x7fc8e00d1870 0x7fc8e01b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:52.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e4a29700 1 -- 172.21.2.101:0/658892172 >> v1:172.21.2.108:6789/0 conn(0x7fc8e00f0650 legacy=0x7fc8e00cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:52.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e4a29700 1 -- 172.21.2.101:0/658892172 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e00d1c70 con 0x7fc8e01b0a10 2021-02-10T11:36:52.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e4a29700 1 -- 172.21.2.101:0/658892172 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fc8dc001600 con 0x7fc8e01b0a10 2021-02-10T11:36:52.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e4a29700 1 -- 172.21.2.101:0/658892172 >> v1:172.21.2.101:6789/0 conn(0x7fc8e01b0a10 legacy=0x7fc8e00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:52.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e4a29700 1 --2- 172.21.2.101:0/658892172 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8cc003af0 0x7fc8cc005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:52.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e4a29700 1 --2- 172.21.2.101:0/658892172 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8cc006820 0x7fc8cc008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:52.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e4a29700 1 --2- 172.21.2.101:0/658892172 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8cc009640 0x7fc8cc00bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:52.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e4a29700 1 -- 172.21.2.101:0/658892172 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc00c330 con 0x7fc8cc009640 2021-02-10T11:36:52.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e4a29700 1 -- 172.21.2.101:0/658892172 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc00c550 con 0x7fc8cc006820 2021-02-10T11:36:52.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e4a29700 1 -- 172.21.2.101:0/658892172 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc00c770 con 0x7fc8cc003af0 2021-02-10T11:36:52.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e622c700 1 --2- 172.21.2.101:0/658892172 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8cc009640 0x7fc8cc00bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:52.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e522a700 1 --2- 172.21.2.101:0/658892172 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8cc006820 0x7fc8cc008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:52.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e5a2b700 1 --2- 172.21.2.101:0/658892172 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8cc003af0 0x7fc8cc005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:52.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e622c700 1 -- 172.21.2.101:0/658892172 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8cc003af0 msgr2=0x7fc8cc005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:52.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e622c700 1 --2- 172.21.2.101:0/658892172 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8cc003af0 0x7fc8cc005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:52.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e622c700 1 -- 172.21.2.101:0/658892172 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8cc006820 msgr2=0x7fc8cc008d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:52.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e622c700 1 --2- 172.21.2.101:0/658892172 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8cc006820 0x7fc8cc008d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:52.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e622c700 1 -- 172.21.2.101:0/658892172 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8cc0509e0 con 0x7fc8cc009640 2021-02-10T11:36:52.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e622c700 1 -- 172.21.2.101:0/658892172 reap_dead start 2021-02-10T11:36:52.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e622c700 1 --2- 172.21.2.101:0/658892172 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8cc009640 0x7fc8cc00bb60 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 rx=0x7fc8d8006e40 tx=0x7fc8d8009e30).ready entity=mon.0 client_cookie=d1a3f19afe098f55 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:52.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e4a29700 1 -- 172.21.2.101:0/658892172 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc8d800b300 con 0x7fc8cc009640 2021-02-10T11:36:52.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e4a29700 1 -- 172.21.2.101:0/658892172 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d800b300 con 0x7fc8cc009640 2021-02-10T11:36:52.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e4a29700 1 -- 172.21.2.101:0/658892172 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc8d8006a60 con 0x7fc8cc009640 2021-02-10T11:36:52.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e7c0f700 1 -- 172.21.2.101:0/658892172 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8cc009640 msgr2=0x7fc8cc00bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:52.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.609 7fc8e7c0f700 1 --2- 172.21.2.101:0/658892172 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8cc009640 0x7fc8cc00bb60 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 rx=0x7fc8d8006e40 tx=0x7fc8d8009e30).stop 2021-02-10T11:36:52.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.610 7fc8e7c0f700 1 -- 172.21.2.101:0/658892172 shutdown_connections 2021-02-10T11:36:52.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.610 7fc8e7c0f700 1 --2- 172.21.2.101:0/658892172 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8cc009640 0x7fc8cc00bb60 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:52.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.610 7fc8e7c0f700 1 -- 172.21.2.101:0/658892172 >> 172.21.2.101:0/658892172 conn(0x7fc8e00ce450 msgr2=0x7fc8e00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:52.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.610 7fc8e7c0f700 1 -- 172.21.2.101:0/658892172 shutdown_connections 2021-02-10T11:36:52.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.610 7fc8e7c0f700 1 -- 172.21.2.101:0/658892172 wait complete. 2021-02-10T11:36:52.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.625 7fc8e7c0f700 1 Processor -- start 2021-02-10T11:36:52.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.625 7fc8e7c0f700 1 -- start start 2021-02-10T11:36:52.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.625 7fc8e7c0f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8e018b820 0x7fc8e00a7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:52.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.625 7fc8e7c0f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8e00a7a60 0x7fc8e00bf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:52.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.625 7fc8e7c0f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8e00bfee0 0x7fc8e00ed150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:52.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.625 7fc8e7c0f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e00bf340 con 0x7fc8e00bfee0 2021-02-10T11:36:52.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.625 7fc8e7c0f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e00ed770 con 0x7fc8e018b820 2021-02-10T11:36:52.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.625 7fc8e7c0f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e00ed990 con 0x7fc8e00a7a60 2021-02-10T11:36:52.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.625 7fc8e622c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8e00bfee0 0x7fc8e00ed150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:52.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.625 7fc8e622c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8e00bfee0 0x7fc8e00ed150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54294/0 (socket says 172.21.2.101:54294) 2021-02-10T11:36:52.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.625 7fc8e622c700 1 -- 172.21.2.101:0/3501777312 learned_addr learned my addr 172.21.2.101:0/3501777312 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:52.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.625 7fc8e522a700 1 --2- 172.21.2.101:0/3501777312 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8e00a7a60 0x7fc8e00bf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:52.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.625 7fc8e5a2b700 1 --2- 172.21.2.101:0/3501777312 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8e018b820 0x7fc8e00a7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:52.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.626 7fc8e622c700 1 -- 172.21.2.101:0/3501777312 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8e00a7a60 msgr2=0x7fc8e00bf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:52.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.626 7fc8e622c700 1 --2- 172.21.2.101:0/3501777312 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8e00a7a60 0x7fc8e00bf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:52.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.626 7fc8e622c700 1 -- 172.21.2.101:0/3501777312 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8e018b820 msgr2=0x7fc8e00a7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:52.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.626 7fc8e622c700 1 --2- 172.21.2.101:0/3501777312 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8e018b820 0x7fc8e00a7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:52.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.626 7fc8e622c700 1 -- 172.21.2.101:0/3501777312 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e01a74f0 con 0x7fc8e00bfee0 2021-02-10T11:36:52.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.626 7fc8e622c700 1 --2- 172.21.2.101:0/3501777312 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8e00bfee0 0x7fc8e00ed150 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 rx=0x7fc8d80093a0 tx=0x7fc8d800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:52.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.626 7fc8d67fc700 1 -- 172.21.2.101:0/3501777312 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc8d8000d30 con 0x7fc8e00bfee0 2021-02-10T11:36:52.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.626 7fc8d67fc700 1 -- 172.21.2.101:0/3501777312 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d800c2f0 con 0x7fc8e00bfee0 2021-02-10T11:36:52.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.626 7fc8d67fc700 1 -- 172.21.2.101:0/3501777312 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc8d800c740 con 0x7fc8e00bfee0 2021-02-10T11:36:52.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.626 7fc8e7c0f700 1 -- 172.21.2.101:0/3501777312 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e01a74f0 con 0x7fc8e00bfee0 2021-02-10T11:36:52.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.626 7fc8e7c0f700 1 -- 172.21.2.101:0/3501777312 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e01a74f0 con 0x7fc8e00bfee0 2021-02-10T11:36:52.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.627 7fc8d67fc700 1 -- 172.21.2.101:0/3501777312 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc8d8001220 con 0x7fc8e00bfee0 2021-02-10T11:36:52.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.627 7fc8d67fc700 1 --2- 172.21.2.101:0/3501777312 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc8c80186c0 0x7fc8c801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:52.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.627 7fc8d67fc700 1 -- 172.21.2.101:0/3501777312 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fc8d8032400 con 0x7fc8e00bfee0 2021-02-10T11:36:52.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.627 7fc8e5a2b700 1 --2- 172.21.2.101:0/3501777312 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc8c80186c0 0x7fc8c801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:52.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.627 7fc8e5a2b700 1 --2- 172.21.2.101:0/3501777312 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc8c80186c0 0x7fc8c801abc0 crc :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:52.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.657 7fc8e7c0f700 1 -- 172.21.2.101:0/3501777312 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e00ebc60 con 0x7fc8e00bfee0 2021-02-10T11:36:52.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.660 7fc8d67fc700 1 -- 172.21.2.101:0/3501777312 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc8d8009020 con 0x7fc8e00bfee0 2021-02-10T11:36:52.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.788 7fc8e7c0f700 1 -- 172.21.2.101:0/3501777312 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8e00ebc60 con 0x7fc8e00bfee0 2021-02-10T11:36:52.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.789 7fc8d67fc700 1 -- 172.21.2.101:0/3501777312 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+11145 (secure 0 0 0) 0x7fc8d8001220 con 0x7fc8e00bfee0 2021-02-10T11:36:52.796 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:36:52.796 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":80,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:36:51.521823","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:36:52.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.791 7fc8e7c0f700 1 -- 172.21.2.101:0/3501777312 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc8c80186c0 msgr2=0x7fc8c801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:52.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.791 7fc8e7c0f700 1 --2- 172.21.2.101:0/3501777312 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc8c80186c0 0x7fc8c801abc0 crc :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:52.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.791 7fc8e7c0f700 1 -- 172.21.2.101:0/3501777312 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8e00bfee0 msgr2=0x7fc8e00ed150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:52.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.791 7fc8e7c0f700 1 --2- 172.21.2.101:0/3501777312 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8e00bfee0 0x7fc8e00ed150 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 rx=0x7fc8d80093a0 tx=0x7fc8d800bcf0).stop 2021-02-10T11:36:52.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.791 7fc8e7c0f700 1 -- 172.21.2.101:0/3501777312 shutdown_connections 2021-02-10T11:36:52.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.791 7fc8e7c0f700 1 --2- 172.21.2.101:0/3501777312 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc8c80186c0 0x7fc8c801abc0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:52.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.791 7fc8e7c0f700 1 --2- 172.21.2.101:0/3501777312 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8e00a7a60 0x7fc8e00bf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:52.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.791 7fc8e7c0f700 1 --2- 172.21.2.101:0/3501777312 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8e00bfee0 0x7fc8e00ed150 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:52.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.791 7fc8e7c0f700 1 --2- 172.21.2.101:0/3501777312 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8e018b820 0x7fc8e00a7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:52.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.791 7fc8e7c0f700 1 -- 172.21.2.101:0/3501777312 >> 172.21.2.101:0/3501777312 conn(0x7fc8e009eba0 msgr2=0x7fc8e00ebe90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:52.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.791 7fc8e7c0f700 1 -- 172.21.2.101:0/3501777312 shutdown_connections 2021-02-10T11:36:52.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:52.791 7fc8e7c0f700 1 -- 172.21.2.101:0/3501777312 wait complete. 2021-02-10T11:36:52.806 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-10T11:36:52.806 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:36:53.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.214 7f35f5c86700 1 Processor -- start 2021-02-10T11:36:53.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.215 7f35f5c86700 1 -- start start 2021-02-10T11:36:53.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.215 7f35f5c86700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f35f00d1c70 0x7f35f00cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:53.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.215 7f35f5c86700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35f00cf580 0x7f35f00cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:53.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.215 7f35f5c86700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f00fe1c0 con 0x7f35f00d1870 2021-02-10T11:36:53.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.215 7f35f5c86700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f01b0cb0 con 0x7f35f00d1c70 2021-02-10T11:36:53.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.215 7f35f5c86700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f0109a60 con 0x7f35f00cf580 2021-02-10T11:36:53.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.215 7f35eeffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f35f00d1c70 0x7f35f00cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:53.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.215 7f35ef7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f35f00d1870 0x7f35f01b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54792/0 (socket says 172.21.2.101:54792) 2021-02-10T11:36:53.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.215 7f35ef7fe700 1 -- 172.21.2.101:0/181360091 learned_addr learned my addr 172.21.2.101:0/181360091 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:53.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.215 7f35eeffd700 1 -- 172.21.2.101:0/181360091 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35f00cf580 msgr2=0x7f35f00cf980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:36:53.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.215 7f35eeffd700 1 --2- 172.21.2.101:0/181360091 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35f00cf580 0x7f35f00cf980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:53.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.215 7f35eeffd700 1 -- 172.21.2.101:0/181360091 >> v1:172.21.2.101:6789/0 conn(0x7f35f00d1870 legacy=0x7f35f01b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:53.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.215 7f35eeffd700 1 -- 172.21.2.101:0/181360091 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f0088070 con 0x7f35f00d1c70 2021-02-10T11:36:53.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.216 7f35eeffd700 1 --2- 172.21.2.101:0/181360091 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35f00d1c70 0x7f35f00cef60 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 rx=0x7f35e0006650 tx=0x7f35e0009a80).ready entity=mon.0 client_cookie=e23823ef1ea16a44 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:53.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.216 7f35ee7fc700 1 -- 172.21.2.101:0/181360091 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f35e000c0c0 con 0x7f35f00d1c70 2021-02-10T11:36:53.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.216 7f35ee7fc700 1 -- 172.21.2.101:0/181360091 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e000c340 con 0x7f35f00d1c70 2021-02-10T11:36:53.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.216 7f35ee7fc700 1 -- 172.21.2.101:0/181360091 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f35e000c7a0 con 0x7f35f00d1c70 2021-02-10T11:36:53.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.216 7f35f5c86700 1 -- 172.21.2.101:0/181360091 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35f00d1c70 msgr2=0x7f35f00cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:53.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.216 7f35f5c86700 1 --2- 172.21.2.101:0/181360091 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35f00d1c70 0x7f35f00cef60 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 rx=0x7f35e0006650 tx=0x7f35e0009a80).stop 2021-02-10T11:36:53.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.216 7f35f5c86700 1 -- 172.21.2.101:0/181360091 shutdown_connections 2021-02-10T11:36:53.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.216 7f35f5c86700 1 --2- 172.21.2.101:0/181360091 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35f00cf580 0x7f35f00cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:53.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.216 7f35f5c86700 1 --2- 172.21.2.101:0/181360091 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35f00d1c70 0x7f35f00cef60 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:53.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.216 7f35f5c86700 1 -- 172.21.2.101:0/181360091 >> 172.21.2.101:0/181360091 conn(0x7f35f00ce420 msgr2=0x7f35f00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:53.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.217 7f35f5c86700 1 -- 172.21.2.101:0/181360091 shutdown_connections 2021-02-10T11:36:53.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.217 7f35f5c86700 1 -- 172.21.2.101:0/181360091 wait complete. 2021-02-10T11:36:53.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.231 7f35f5c86700 1 Processor -- start 2021-02-10T11:36:53.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.231 7f35f5c86700 1 -- start start 2021-02-10T11:36:53.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.231 7f35f5c86700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f35f00c3820 0x7f35f00a7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:53.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.231 7f35f5c86700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35f01a44c0 0x7f35f01a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:53.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.231 7f35f5c86700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35f00863d0 0x7f35f00867d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:53.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.231 7f35f5c86700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f0085ed0 con 0x7f35f01a44c0 2021-02-10T11:36:53.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.231 7f35f5c86700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f00c0740 con 0x7f35f00c3820 2021-02-10T11:36:53.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.231 7f35f5c86700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f0086df0 con 0x7f35f00863d0 2021-02-10T11:36:53.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.231 7f35eeffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35f01a44c0 0x7f35f01a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:53.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.231 7f35eeffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35f01a44c0 0x7f35f01a48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54306/0 (socket says 172.21.2.101:54306) 2021-02-10T11:36:53.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.232 7f35eeffd700 1 -- 172.21.2.101:0/4284344626 learned_addr learned my addr 172.21.2.101:0/4284344626 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:53.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.232 7f35eeffd700 1 -- 172.21.2.101:0/4284344626 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35f00863d0 msgr2=0x7f35f00867d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:53.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.232 7f35eeffd700 1 --2- 172.21.2.101:0/4284344626 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35f00863d0 0x7f35f00867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:53.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.232 7f35eeffd700 1 -- 172.21.2.101:0/4284344626 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f35f00c3820 msgr2=0x7f35f00a7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:53.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.232 7f35eeffd700 1 --2- 172.21.2.101:0/4284344626 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f35f00c3820 0x7f35f00a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:53.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.232 7f35eeffd700 1 -- 172.21.2.101:0/4284344626 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f0087010 con 0x7f35f01a44c0 2021-02-10T11:36:53.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.232 7f35eeffd700 1 --2- 172.21.2.101:0/4284344626 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35f01a44c0 0x7f35f01a48c0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 rx=0x7f35e0009650 tx=0x7f35e000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:53.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.232 7f35e7fff700 1 -- 172.21.2.101:0/4284344626 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f35e000bd50 con 0x7f35f01a44c0 2021-02-10T11:36:53.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.232 7f35e7fff700 1 -- 172.21.2.101:0/4284344626 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e0006030 con 0x7f35f01a44c0 2021-02-10T11:36:53.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.232 7f35e7fff700 1 -- 172.21.2.101:0/4284344626 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f35e00054d0 con 0x7f35f01a44c0 2021-02-10T11:36:53.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.232 7f35f5c86700 1 -- 172.21.2.101:0/4284344626 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f0087010 con 0x7f35f01a44c0 2021-02-10T11:36:53.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.232 7f35f5c86700 1 -- 172.21.2.101:0/4284344626 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f0083000 con 0x7f35f01a44c0 2021-02-10T11:36:53.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.233 7f35e7fff700 1 -- 172.21.2.101:0/4284344626 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f35e00052e0 con 0x7f35f01a44c0 2021-02-10T11:36:53.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.233 7f35e7fff700 1 --2- 172.21.2.101:0/4284344626 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f35d40186c0 0x7f35d401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:53.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.233 7f35e7fff700 1 -- 172.21.2.101:0/4284344626 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f35e0032220 con 0x7f35f01a44c0 2021-02-10T11:36:53.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.233 7f35ef7fe700 1 --2- 172.21.2.101:0/4284344626 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f35d40186c0 0x7f35d401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:53.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.234 7f35ef7fe700 1 --2- 172.21.2.101:0/4284344626 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f35d40186c0 0x7f35d401abc0 crc :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:53.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.265 7f35f5c86700 1 -- 172.21.2.101:0/4284344626 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f0085e50 con 0x7f35f01a44c0 2021-02-10T11:36:53.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.267 7f35e7fff700 1 -- 172.21.2.101:0/4284344626 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f35e0009300 con 0x7f35f01a44c0 2021-02-10T11:36:53.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.394 7f35f5c86700 1 -- 172.21.2.101:0/4284344626 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f35f0085e50 con 0x7f35f01a44c0 2021-02-10T11:36:53.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.394 7f35e7fff700 1 -- 172.21.2.101:0/4284344626 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v80) v1 ==== 92+0+10 (secure 0 0 0) 0x7f35e000bd50 con 0x7f35f01a44c0 2021-02-10T11:36:53.401 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:36:53.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.396 7f35f5c86700 1 -- 172.21.2.101:0/4284344626 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f35d40186c0 msgr2=0x7f35d401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:53.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.396 7f35f5c86700 1 --2- 172.21.2.101:0/4284344626 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f35d40186c0 0x7f35d401abc0 crc :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:53.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.396 7f35f5c86700 1 -- 172.21.2.101:0/4284344626 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35f01a44c0 msgr2=0x7f35f01a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:53.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.396 7f35f5c86700 1 --2- 172.21.2.101:0/4284344626 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35f01a44c0 0x7f35f01a48c0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 rx=0x7f35e0009650 tx=0x7f35e000b960).stop 2021-02-10T11:36:53.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.397 7f35f5c86700 1 -- 172.21.2.101:0/4284344626 shutdown_connections 2021-02-10T11:36:53.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.397 7f35f5c86700 1 --2- 172.21.2.101:0/4284344626 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f35d40186c0 0x7f35d401abc0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:53.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.397 7f35f5c86700 1 --2- 172.21.2.101:0/4284344626 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35f00863d0 0x7f35f00867d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:53.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.397 7f35f5c86700 1 --2- 172.21.2.101:0/4284344626 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35f01a44c0 0x7f35f01a48c0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:53.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.397 7f35f5c86700 1 --2- 172.21.2.101:0/4284344626 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f35f00c3820 0x7f35f00a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:53.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.397 7f35f5c86700 1 -- 172.21.2.101:0/4284344626 >> 172.21.2.101:0/4284344626 conn(0x7f35f00872c0 msgr2=0x7f35f0095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:53.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.397 7f35f5c86700 1 -- 172.21.2.101:0/4284344626 shutdown_connections 2021-02-10T11:36:53.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.397 7f35f5c86700 1 -- 172.21.2.101:0/4284344626 wait complete. 2021-02-10T11:36:53.412 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --format=json-pretty osd lspools 2021-02-10T11:36:53.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.816 7f7c6d14a700 1 Processor -- start 2021-02-10T11:36:53.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.816 7f7c6d14a700 1 -- start start 2021-02-10T11:36:53.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.817 7f7c6d14a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c681b0870 0x7f7c680f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:53.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.817 7f7c6d14a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7c680efe00 0x7f7c680f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:53.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.817 7f7c6d14a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c681b0c70 con 0x7f7c681b0470 2021-02-10T11:36:53.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.817 7f7c6d14a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c680f0b50 con 0x7f7c680efe00 2021-02-10T11:36:53.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.817 7f7c6d14a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c680f0820 con 0x7f7c681b0870 2021-02-10T11:36:53.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.817 7f7c66d9d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f7c681b0470 0x7f7c680ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54804/0 (socket says 172.21.2.101:54804) 2021-02-10T11:36:53.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.817 7f7c66d9d700 1 -- 172.21.2.101:0/278558867 learned_addr learned my addr 172.21.2.101:0/278558867 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:53.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.817 7f7c65d9b700 1 -- 172.21.2.101:0/278558867 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554530204 0 0) 0x7f7c50001180 con 0x7f7c681b0470 2021-02-10T11:36:53.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.817 7f7c65d9b700 1 -- 172.21.2.101:0/278558867 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c54002fe0 con 0x7f7c681b0470 2021-02-10T11:36:53.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.817 7f7c6759e700 1 --2- 172.21.2.101:0/278558867 >> v2:172.21.2.108:3300/0 conn(0x7f7c680efe00 0x7f7c680f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:53.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.817 7f7c6659c700 1 --2- 172.21.2.101:0/278558867 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c681b0870 0x7f7c680f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:53.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c65d9b700 1 -- 172.21.2.101:0/278558867 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2424924281 0 0) 0x7f7c50001180 con 0x7f7c681b0470 2021-02-10T11:36:53.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c65d9b700 1 -- 172.21.2.101:0/278558867 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c681b0870 msgr2=0x7f7c680f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:53.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c65d9b700 1 --2- 172.21.2.101:0/278558867 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c681b0870 0x7f7c680f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:53.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c65d9b700 1 -- 172.21.2.101:0/278558867 >> v2:172.21.2.108:3300/0 conn(0x7f7c680efe00 msgr2=0x7f7c680f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:53.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c65d9b700 1 --2- 172.21.2.101:0/278558867 >> v2:172.21.2.108:3300/0 conn(0x7f7c680efe00 0x7f7c680f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:53.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c65d9b700 1 -- 172.21.2.101:0/278558867 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c680e52e0 con 0x7f7c681b0470 2021-02-10T11:36:53.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c65d9b700 1 -- 172.21.2.101:0/278558867 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f7c50001680 con 0x7f7c681b0470 2021-02-10T11:36:53.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c65d9b700 1 -- 172.21.2.101:0/278558867 >> v1:172.21.2.101:6789/0 conn(0x7f7c681b0470 legacy=0x7f7c680ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:53.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c65d9b700 1 --2- 172.21.2.101:0/278558867 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c54003a40 0x7f7c54005eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:53.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c65d9b700 1 --2- 172.21.2.101:0/278558867 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c5404a760 0x7f7c5404cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:53.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c65d9b700 1 --2- 172.21.2.101:0/278558867 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c5404d500 0x7f7c5404fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:53.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c65d9b700 1 -- 172.21.2.101:0/278558867 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c540502d0 con 0x7f7c5404d500 2021-02-10T11:36:53.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c65d9b700 1 -- 172.21.2.101:0/278558867 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c540504f0 con 0x7f7c5404a760 2021-02-10T11:36:53.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c65d9b700 1 -- 172.21.2.101:0/278558867 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c54050710 con 0x7f7c54003a40 2021-02-10T11:36:53.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c6759e700 1 --2- 172.21.2.101:0/278558867 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c5404d500 0x7f7c5404fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:53.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c66d9d700 1 --2- 172.21.2.101:0/278558867 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c54003a40 0x7f7c54005eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:53.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c6659c700 1 --2- 172.21.2.101:0/278558867 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c5404a760 0x7f7c5404cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:53.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c6759e700 1 -- 172.21.2.101:0/278558867 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c54003a40 msgr2=0x7f7c54005eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:53.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c6759e700 1 --2- 172.21.2.101:0/278558867 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c54003a40 0x7f7c54005eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:53.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c6759e700 1 -- 172.21.2.101:0/278558867 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c5404a760 msgr2=0x7f7c5404cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:53.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c6759e700 1 --2- 172.21.2.101:0/278558867 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c5404a760 0x7f7c5404cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:53.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c6759e700 1 -- 172.21.2.101:0/278558867 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c540509b0 con 0x7f7c5404d500 2021-02-10T11:36:53.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.818 7f7c6759e700 1 --2- 172.21.2.101:0/278558867 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c5404d500 0x7f7c5404fa20 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 rx=0x7f7c60006a40 tx=0x7f7c6000c0c0).ready entity=mon.0 client_cookie=9b8c8576deece47f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:53.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.819 7f7c65d9b700 1 -- 172.21.2.101:0/278558867 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7c6000c4b0 con 0x7f7c5404d500 2021-02-10T11:36:53.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.819 7f7c65d9b700 1 -- 172.21.2.101:0/278558867 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6000c770 con 0x7f7c5404d500 2021-02-10T11:36:53.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.819 7f7c65d9b700 1 -- 172.21.2.101:0/278558867 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7c6000cbd0 con 0x7f7c5404d500 2021-02-10T11:36:53.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.819 7f7c6d14a700 1 -- 172.21.2.101:0/278558867 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c5404d500 msgr2=0x7f7c5404fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:53.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.819 7f7c6d14a700 1 --2- 172.21.2.101:0/278558867 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c5404d500 0x7f7c5404fa20 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 rx=0x7f7c60006a40 tx=0x7f7c6000c0c0).stop 2021-02-10T11:36:53.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.819 7f7c6759e700 1 -- 172.21.2.101:0/278558867 reap_dead start 2021-02-10T11:36:53.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.819 7f7c6d14a700 1 -- 172.21.2.101:0/278558867 shutdown_connections 2021-02-10T11:36:53.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.819 7f7c6d14a700 1 -- 172.21.2.101:0/278558867 >> 172.21.2.101:0/278558867 conn(0x7f7c680ce3e0 msgr2=0x7f7c680e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:53.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.819 7f7c6d14a700 1 -- 172.21.2.101:0/278558867 shutdown_connections 2021-02-10T11:36:53.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.819 7f7c6d14a700 1 -- 172.21.2.101:0/278558867 wait complete. 2021-02-10T11:36:53.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.834 7f7c6d14a700 1 Processor -- start 2021-02-10T11:36:53.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.834 7f7c6d14a700 1 -- start start 2021-02-10T11:36:53.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.834 7f7c6d14a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c681a0500 0x7f7c6809e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:53.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.834 7f7c6d14a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c6809eab0 0x7f7c680ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:53.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.834 7f7c6d14a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c680ae6f0 0x7f7c68092690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:53.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.834 7f7c6d14a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c680adbd0 con 0x7f7c6809eab0 2021-02-10T11:36:53.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.834 7f7c6d14a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c68092cb0 con 0x7f7c680ae6f0 2021-02-10T11:36:53.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.834 7f7c6d14a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c68092ed0 con 0x7f7c681a0500 2021-02-10T11:36:53.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.834 7f7c6659c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c6809eab0 0x7f7c680ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:53.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.834 7f7c6659c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c6809eab0 0x7f7c680ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54326/0 (socket says 172.21.2.101:54326) 2021-02-10T11:36:53.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.835 7f7c6659c700 1 -- 172.21.2.101:0/2930064669 learned_addr learned my addr 172.21.2.101:0/2930064669 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:53.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.835 7f7c6659c700 1 -- 172.21.2.101:0/2930064669 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c681a0500 msgr2=0x7f7c6809e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:53.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.835 7f7c6659c700 1 --2- 172.21.2.101:0/2930064669 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c681a0500 0x7f7c6809e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:53.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.835 7f7c6659c700 1 -- 172.21.2.101:0/2930064669 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c680ae6f0 msgr2=0x7f7c68092690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:53.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.835 7f7c6659c700 1 --2- 172.21.2.101:0/2930064669 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c680ae6f0 0x7f7c68092690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:53.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.835 7f7c6659c700 1 -- 172.21.2.101:0/2930064669 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c680930f0 con 0x7f7c6809eab0 2021-02-10T11:36:53.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.835 7f7c6659c700 1 --2- 172.21.2.101:0/2930064669 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c6809eab0 0x7f7c680ae0d0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 rx=0x7f7c58006f60 tx=0x7f7c5800b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:53.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.835 7f7c5f7fe700 1 -- 172.21.2.101:0/2930064669 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7c5800bd60 con 0x7f7c6809eab0 2021-02-10T11:36:53.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.835 7f7c5f7fe700 1 -- 172.21.2.101:0/2930064669 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c58000c90 con 0x7f7c6809eab0 2021-02-10T11:36:53.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.835 7f7c6d14a700 1 -- 172.21.2.101:0/2930064669 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c680930f0 con 0x7f7c6809eab0 2021-02-10T11:36:53.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.835 7f7c6d14a700 1 -- 172.21.2.101:0/2930064669 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c680930f0 con 0x7f7c6809eab0 2021-02-10T11:36:53.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.835 7f7c5f7fe700 1 -- 172.21.2.101:0/2930064669 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7c58002300 con 0x7f7c6809eab0 2021-02-10T11:36:53.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.836 7f7c5f7fe700 1 -- 172.21.2.101:0/2930064669 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7c58002020 con 0x7f7c6809eab0 2021-02-10T11:36:53.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.836 7f7c5f7fe700 1 --2- 172.21.2.101:0/2930064669 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c4c018730 0x7f7c4c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:53.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.836 7f7c5f7fe700 1 -- 172.21.2.101:0/2930064669 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f7c58032470 con 0x7f7c6809eab0 2021-02-10T11:36:53.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.836 7f7c66d9d700 1 --2- 172.21.2.101:0/2930064669 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c4c018730 0x7f7c4c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:53.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.836 7f7c66d9d700 1 --2- 172.21.2.101:0/2930064669 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c4c018730 0x7f7c4c01ac10 crc :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:53.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.866 7f7c6d14a700 1 -- 172.21.2.101:0/2930064669 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c68105a80 con 0x7f7c6809eab0 2021-02-10T11:36:53.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.869 7f7c5f7fe700 1 -- 172.21.2.101:0/2930064669 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7c58000c90 con 0x7f7c6809eab0 2021-02-10T11:36:53.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.992 7f7c6d14a700 1 -- 172.21.2.101:0/2930064669 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7c680f0b50 con 0x7f7c6809eab0 2021-02-10T11:36:53.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.993 7f7c5f7fe700 1 -- 172.21.2.101:0/2930064669 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+64 (secure 0 0 0) 0x7f7c5800bd60 con 0x7f7c6809eab0 2021-02-10T11:36:54.000 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:36:54.000 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:36:54.000 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:36:54.000 INFO:teuthology.orchestra.run.gibba001.stdout: "poolnum": 1, 2021-02-10T11:36:54.000 INFO:teuthology.orchestra.run.gibba001.stdout: "poolname": "rbd" 2021-02-10T11:36:54.000 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:36:54.001 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:36:54.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.995 7f7c6d14a700 1 -- 172.21.2.101:0/2930064669 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c4c018730 msgr2=0x7f7c4c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:54.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.995 7f7c6d14a700 1 --2- 172.21.2.101:0/2930064669 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c4c018730 0x7f7c4c01ac10 crc :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:54.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.995 7f7c6d14a700 1 -- 172.21.2.101:0/2930064669 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c6809eab0 msgr2=0x7f7c680ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:54.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.995 7f7c6d14a700 1 --2- 172.21.2.101:0/2930064669 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c6809eab0 0x7f7c680ae0d0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 rx=0x7f7c58006f60 tx=0x7f7c5800b970).stop 2021-02-10T11:36:54.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.995 7f7c6d14a700 1 -- 172.21.2.101:0/2930064669 shutdown_connections 2021-02-10T11:36:54.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.995 7f7c6d14a700 1 --2- 172.21.2.101:0/2930064669 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c680ae6f0 0x7f7c68092690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:54.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.995 7f7c6d14a700 1 --2- 172.21.2.101:0/2930064669 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c6809eab0 0x7f7c680ae0d0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:54.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.995 7f7c6d14a700 1 --2- 172.21.2.101:0/2930064669 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c4c018730 0x7f7c4c01ac10 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:54.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.995 7f7c6d14a700 1 --2- 172.21.2.101:0/2930064669 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c681a0500 0x7f7c6809e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:54.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.995 7f7c6d14a700 1 -- 172.21.2.101:0/2930064669 >> 172.21.2.101:0/2930064669 conn(0x7f7c680a1dc0 msgr2=0x7f7c6808e860 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:54.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.995 7f7c6d14a700 1 -- 172.21.2.101:0/2930064669 shutdown_connections 2021-02-10T11:36:54.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:53.995 7f7c6d14a700 1 -- 172.21.2.101:0/2930064669 wait complete. 2021-02-10T11:36:54.009 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-02-10T11:36:54.009 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-02-10T11:36:54.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.417 7f6171c15700 1 Processor -- start 2021-02-10T11:36:54.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.417 7f6171c15700 1 -- start start 2021-02-10T11:36:54.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.418 7f6171c15700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f616c0d1c70 0x7f616c0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:54.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.418 7f6171c15700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f616c0cf580 0x7f616c0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:54.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.418 7f6171c15700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f616c0fe1c0 con 0x7f616c0d1870 2021-02-10T11:36:54.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.418 7f6171c15700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f616c1b0cb0 con 0x7f616c0d1c70 2021-02-10T11:36:54.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.418 7f6171c15700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f616c109a60 con 0x7f616c0cf580 2021-02-10T11:36:54.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.418 7f616affd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f616c0d1c70 0x7f616c0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:54.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.418 7f616affd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f616c0d1c70 0x7f616c0cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54330/0 (socket says 172.21.2.101:54330) 2021-02-10T11:36:54.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.418 7f616affd700 1 -- 172.21.2.101:0/2835852716 learned_addr learned my addr 172.21.2.101:0/2835852716 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:54.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.418 7f616b7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f616c0d1870 0x7f616c1b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54824/0 (socket says 172.21.2.101:54824) 2021-02-10T11:36:54.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.418 7f616affd700 1 -- 172.21.2.101:0/2835852716 >> v2:172.21.2.108:3300/0 conn(0x7f616c0cf580 msgr2=0x7f616c0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:54.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.418 7f616bfff700 1 --2- 172.21.2.101:0/2835852716 >> v2:172.21.2.108:3300/0 conn(0x7f616c0cf580 0x7f616c0cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:54.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.418 7f616a7fc700 1 -- 172.21.2.101:0/2835852716 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2302582259 0 0) 0x7f615c0010b0 con 0x7f616c0d1870 2021-02-10T11:36:54.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.418 7f616affd700 1 --2- 172.21.2.101:0/2835852716 >> v2:172.21.2.108:3300/0 conn(0x7f616c0cf580 0x7f616c0cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:54.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.418 7f616affd700 1 -- 172.21.2.101:0/2835852716 >> v1:172.21.2.101:6789/0 conn(0x7f616c0d1870 legacy=0x7f616c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:54.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.418 7f616affd700 1 -- 172.21.2.101:0/2835852716 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616c088070 con 0x7f616c0d1c70 2021-02-10T11:36:54.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.418 7f616bfff700 1 --2- 172.21.2.101:0/2835852716 >> v2:172.21.2.108:3300/0 conn(0x7f616c0cf580 0x7f616c0cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:36:54.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.418 7f616affd700 1 --2- 172.21.2.101:0/2835852716 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f616c0d1c70 0x7f616c0cef60 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 rx=0x7f6154006700 tx=0x7f6154009b80).ready entity=mon.0 client_cookie=762345e232f6dc66 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:54.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.419 7f616a7fc700 1 -- 172.21.2.101:0/2835852716 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6154006340 con 0x7f616c0d1c70 2021-02-10T11:36:54.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.419 7f616a7fc700 1 -- 172.21.2.101:0/2835852716 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615400c0c0 con 0x7f616c0d1c70 2021-02-10T11:36:54.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.419 7f616a7fc700 1 -- 172.21.2.101:0/2835852716 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f615400c5a0 con 0x7f616c0d1c70 2021-02-10T11:36:54.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.419 7f6171c15700 1 -- 172.21.2.101:0/2835852716 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f616c0d1c70 msgr2=0x7f616c0cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:54.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.419 7f6171c15700 1 --2- 172.21.2.101:0/2835852716 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f616c0d1c70 0x7f616c0cef60 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 rx=0x7f6154006700 tx=0x7f6154009b80).stop 2021-02-10T11:36:54.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.419 7f6171c15700 1 -- 172.21.2.101:0/2835852716 shutdown_connections 2021-02-10T11:36:54.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.419 7f6171c15700 1 --2- 172.21.2.101:0/2835852716 >> v2:172.21.2.108:3300/0 conn(0x7f616c0cf580 0x7f616c0cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:54.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.419 7f6171c15700 1 --2- 172.21.2.101:0/2835852716 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f616c0d1c70 0x7f616c0cef60 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:54.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.419 7f6171c15700 1 -- 172.21.2.101:0/2835852716 >> 172.21.2.101:0/2835852716 conn(0x7f616c0ce420 msgr2=0x7f616c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:54.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.419 7f6171c15700 1 -- 172.21.2.101:0/2835852716 shutdown_connections 2021-02-10T11:36:54.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.419 7f6171c15700 1 -- 172.21.2.101:0/2835852716 wait complete. 2021-02-10T11:36:54.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.434 7f6171c15700 1 Processor -- start 2021-02-10T11:36:54.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.434 7f6171c15700 1 -- start start 2021-02-10T11:36:54.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.434 7f6171c15700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f616c191b10 0x7f616c0ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:54.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.434 7f6171c15700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f616c191f10 0x7f616c19ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:54.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.434 7f6171c15700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f616c1a05f0 0x7f616c082ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:54.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.434 7f6171c15700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c192440 con 0x7f616c191f10 2021-02-10T11:36:54.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.434 7f6171c15700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c1916e0 con 0x7f616c1a05f0 2021-02-10T11:36:54.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.434 7f6171c15700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616c191900 con 0x7f616c191b10 2021-02-10T11:36:54.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.434 7f616affd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f616c191f10 0x7f616c19ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:54.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.434 7f616affd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f616c191f10 0x7f616c19ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54338/0 (socket says 172.21.2.101:54338) 2021-02-10T11:36:54.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.434 7f616b7fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f616c191b10 0x7f616c0ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:54.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.434 7f616affd700 1 -- 172.21.2.101:0/1368394031 learned_addr learned my addr 172.21.2.101:0/1368394031 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:54.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.435 7f616bfff700 1 --2- 172.21.2.101:0/1368394031 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f616c1a05f0 0x7f616c082ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:54.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.435 7f616affd700 1 -- 172.21.2.101:0/1368394031 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f616c191b10 msgr2=0x7f616c0ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:54.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.435 7f616affd700 1 --2- 172.21.2.101:0/1368394031 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f616c191b10 0x7f616c0ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:54.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.435 7f616affd700 1 -- 172.21.2.101:0/1368394031 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f616c1a05f0 msgr2=0x7f616c082ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:54.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.435 7f616affd700 1 --2- 172.21.2.101:0/1368394031 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f616c1a05f0 0x7f616c082ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:54.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.435 7f616affd700 1 -- 172.21.2.101:0/1368394031 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616c0834f0 con 0x7f616c191f10 2021-02-10T11:36:54.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.435 7f616affd700 1 --2- 172.21.2.101:0/1368394031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f616c191f10 0x7f616c19ffd0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 rx=0x7f6154009270 tx=0x7f615400b8e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:54.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.435 7f6163fff700 1 -- 172.21.2.101:0/1368394031 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f615400bcd0 con 0x7f616c191f10 2021-02-10T11:36:54.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.435 7f6163fff700 1 -- 172.21.2.101:0/1368394031 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61540052e0 con 0x7f616c191f10 2021-02-10T11:36:54.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.435 7f6171c15700 1 -- 172.21.2.101:0/1368394031 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616c0834f0 con 0x7f616c191f10 2021-02-10T11:36:54.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.435 7f6163fff700 1 -- 172.21.2.101:0/1368394031 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6154005800 con 0x7f616c191f10 2021-02-10T11:36:54.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.435 7f6171c15700 1 -- 172.21.2.101:0/1368394031 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616c0834f0 con 0x7f616c191f10 2021-02-10T11:36:54.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.436 7f6163fff700 1 -- 172.21.2.101:0/1368394031 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f61540056d0 con 0x7f616c191f10 2021-02-10T11:36:54.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.436 7f6163fff700 1 --2- 172.21.2.101:0/1368394031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61500186c0 0x7f615001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:54.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.436 7f6163fff700 1 -- 172.21.2.101:0/1368394031 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f6154032450 con 0x7f616c191f10 2021-02-10T11:36:54.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.436 7f616b7fe700 1 --2- 172.21.2.101:0/1368394031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61500186c0 0x7f615001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:54.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.436 7f616b7fe700 1 --2- 172.21.2.101:0/1368394031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61500186c0 0x7f615001abc0 crc :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:54.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.468 7f6171c15700 1 -- 172.21.2.101:0/1368394031 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f616c191900 con 0x7f616c191f10 2021-02-10T11:36:54.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.470 7f6163fff700 1 -- 172.21.2.101:0/1368394031 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f61540052e0 con 0x7f616c191f10 2021-02-10T11:36:54.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:54.600 7f6171c15700 1 -- 172.21.2.101:0/1368394031 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"} v 0) v1 -- 0x7f616c191900 con 0x7f616c191f10 2021-02-10T11:36:55.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.016 7f6163fff700 1 -- 172.21.2.101:0/1368394031 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"}]=0 pool 'cephfs_metadata' created v81) v1 ==== 133+0+0 (secure 0 0 0) 0x7f61540052e0 con 0x7f616c191f10 2021-02-10T11:36:55.023 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' created 2021-02-10T11:36:55.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.019 7f6171c15700 1 -- 172.21.2.101:0/1368394031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61500186c0 msgr2=0x7f615001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:55.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.019 7f6171c15700 1 --2- 172.21.2.101:0/1368394031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61500186c0 0x7f615001abc0 crc :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:55.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.019 7f6171c15700 1 -- 172.21.2.101:0/1368394031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f616c191f10 msgr2=0x7f616c19ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:55.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.019 7f6171c15700 1 --2- 172.21.2.101:0/1368394031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f616c191f10 0x7f616c19ffd0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 rx=0x7f6154009270 tx=0x7f615400b8e0).stop 2021-02-10T11:36:55.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.019 7f6171c15700 1 -- 172.21.2.101:0/1368394031 shutdown_connections 2021-02-10T11:36:55.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.019 7f6171c15700 1 --2- 172.21.2.101:0/1368394031 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f616c1a05f0 0x7f616c082ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:55.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.019 7f6171c15700 1 --2- 172.21.2.101:0/1368394031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f616c191f10 0x7f616c19ffd0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:55.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.019 7f6171c15700 1 --2- 172.21.2.101:0/1368394031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61500186c0 0x7f615001abc0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:55.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.019 7f6171c15700 1 --2- 172.21.2.101:0/1368394031 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f616c191b10 0x7f616c0ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:55.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.019 7f6171c15700 1 -- 172.21.2.101:0/1368394031 >> 172.21.2.101:0/1368394031 conn(0x7f616c0a02d0 msgr2=0x7f616c09f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:55.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.019 7f6171c15700 1 -- 172.21.2.101:0/1368394031 shutdown_connections 2021-02-10T11:36:55.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.019 7f6171c15700 1 -- 172.21.2.101:0/1368394031 wait complete. 2021-02-10T11:36:55.033 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_data 8 2021-02-10T11:36:55.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.442 7fa5da68b700 1 Processor -- start 2021-02-10T11:36:55.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.442 7fa5da68b700 1 -- start start 2021-02-10T11:36:55.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.443 7fa5da68b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa5d40d1860 0x7fa5d41b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:55.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.443 7fa5da68b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa5d41b0ad0 0x7fa5d4108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:55.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.443 7fa5da68b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d40fe1b0 con 0x7fa5d40d1c60 2021-02-10T11:36:55.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.443 7fa5da68b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d4109a50 con 0x7fa5d41b0ad0 2021-02-10T11:36:55.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.443 7fa5da68b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d4109740 con 0x7fa5d40d1860 2021-02-10T11:36:55.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.443 7fa5d37fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa5d41b0ad0 0x7fa5d4108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:55.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.443 7fa5d37fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa5d41b0ad0 0x7fa5d4108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54348/0 (socket says 172.21.2.101:54348) 2021-02-10T11:36:55.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.443 7fa5d37fe700 1 -- 172.21.2.101:0/3520591321 learned_addr learned my addr 172.21.2.101:0/3520591321 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:55.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.443 7fa5d8ca8700 1 --1- 172.21.2.101:0/3520591321 >> v1:172.21.2.101:6789/0 conn(0x7fa5d40d1c60 0x7fa5d4108f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54840/0 (socket says 172.21.2.101:54840) 2021-02-10T11:36:55.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.444 7fa5d37fe700 1 -- 172.21.2.101:0/3520591321 >> v2:172.21.2.108:3300/0 conn(0x7fa5d40d1860 msgr2=0x7fa5d41b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:55.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.444 7fa5d37fe700 1 --2- 172.21.2.101:0/3520591321 >> v2:172.21.2.108:3300/0 conn(0x7fa5d40d1860 0x7fa5d41b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:55.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.444 7fa5d37fe700 1 -- 172.21.2.101:0/3520591321 >> v1:172.21.2.101:6789/0 conn(0x7fa5d40d1c60 legacy=0x7fa5d4108f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:55.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.444 7fa5d37fe700 1 -- 172.21.2.101:0/3520591321 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d4088070 con 0x7fa5d41b0ad0 2021-02-10T11:36:55.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.444 7fa5d2ffd700 1 -- 172.21.2.101:0/3520591321 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939606819 0 0) 0x7fa5cc001480 con 0x7fa5d40d1c60 2021-02-10T11:36:55.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.444 7fa5d37fe700 1 --2- 172.21.2.101:0/3520591321 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5d41b0ad0 0x7fa5d4108930 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 rx=0x7fa5c8006700 tx=0x7fa5c8009b80).ready entity=mon.0 client_cookie=68960e1504968498 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:55.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.444 7fa5d2ffd700 1 -- 172.21.2.101:0/3520591321 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa5c800c0c0 con 0x7fa5d41b0ad0 2021-02-10T11:36:55.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.444 7fa5d2ffd700 1 -- 172.21.2.101:0/3520591321 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c800c340 con 0x7fa5d41b0ad0 2021-02-10T11:36:55.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.444 7fa5d2ffd700 1 -- 172.21.2.101:0/3520591321 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa5c800c860 con 0x7fa5d41b0ad0 2021-02-10T11:36:55.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.444 7fa5da68b700 1 -- 172.21.2.101:0/3520591321 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5d41b0ad0 msgr2=0x7fa5d4108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:55.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.444 7fa5da68b700 1 --2- 172.21.2.101:0/3520591321 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5d41b0ad0 0x7fa5d4108930 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 rx=0x7fa5c8006700 tx=0x7fa5c8009b80).stop 2021-02-10T11:36:55.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.445 7fa5da68b700 1 -- 172.21.2.101:0/3520591321 shutdown_connections 2021-02-10T11:36:55.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.445 7fa5da68b700 1 --2- 172.21.2.101:0/3520591321 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5d41b0ad0 0x7fa5d4108930 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:55.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.445 7fa5da68b700 1 --2- 172.21.2.101:0/3520591321 >> v2:172.21.2.108:3300/0 conn(0x7fa5d40d1860 0x7fa5d41b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:55.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.445 7fa5da68b700 1 -- 172.21.2.101:0/3520591321 >> 172.21.2.101:0/3520591321 conn(0x7fa5d40ce410 msgr2=0x7fa5d40d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:55.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.445 7fa5da68b700 1 -- 172.21.2.101:0/3520591321 shutdown_connections 2021-02-10T11:36:55.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.445 7fa5da68b700 1 -- 172.21.2.101:0/3520591321 wait complete. 2021-02-10T11:36:55.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.459 7fa5da68b700 1 Processor -- start 2021-02-10T11:36:55.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.460 7fa5da68b700 1 -- start start 2021-02-10T11:36:55.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.460 7fa5da68b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa5d41a05e0 0x7fa5d418b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:55.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.460 7fa5da68b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5d40b8780 0x7fa5d40bfa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:55.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.460 7fa5da68b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5d40c0030 0x7fa5d4189e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:55.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.460 7fa5da68b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d40b82a0 con 0x7fa5d40b8780 2021-02-10T11:36:55.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.460 7fa5da68b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d40c0430 con 0x7fa5d41a05e0 2021-02-10T11:36:55.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.460 7fa5da68b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d40c0650 con 0x7fa5d40c0030 2021-02-10T11:36:55.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.460 7fa5d37fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5d40b8780 0x7fa5d40bfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:55.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.460 7fa5d37fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5d40b8780 0x7fa5d40bfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54350/0 (socket says 172.21.2.101:54350) 2021-02-10T11:36:55.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.460 7fa5d37fe700 1 -- 172.21.2.101:0/235330083 learned_addr learned my addr 172.21.2.101:0/235330083 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:55.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.461 7fa5d37fe700 1 -- 172.21.2.101:0/235330083 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5d40c0030 msgr2=0x7fa5d4189e70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:55.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.461 7fa5d37fe700 1 --2- 172.21.2.101:0/235330083 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5d40c0030 0x7fa5d4189e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:55.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.461 7fa5d37fe700 1 -- 172.21.2.101:0/235330083 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa5d41a05e0 msgr2=0x7fa5d418b970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:55.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.461 7fa5d37fe700 1 --2- 172.21.2.101:0/235330083 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa5d41a05e0 0x7fa5d418b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:55.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.461 7fa5d37fe700 1 -- 172.21.2.101:0/235330083 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d418a490 con 0x7fa5d40b8780 2021-02-10T11:36:55.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.461 7fa5d37fe700 1 --2- 172.21.2.101:0/235330083 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5d40b8780 0x7fa5d40bfa10 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 rx=0x7fa5c800b5e0 tx=0x7fa5c800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:55.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.461 7fa5d0ff9700 1 -- 172.21.2.101:0/235330083 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa5c800bd70 con 0x7fa5d40b8780 2021-02-10T11:36:55.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.461 7fa5d0ff9700 1 -- 172.21.2.101:0/235330083 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c8002030 con 0x7fa5d40b8780 2021-02-10T11:36:55.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.461 7fa5d0ff9700 1 -- 172.21.2.101:0/235330083 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa5c8002550 con 0x7fa5d40b8780 2021-02-10T11:36:55.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.461 7fa5da68b700 1 -- 172.21.2.101:0/235330083 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d418a490 con 0x7fa5d40b8780 2021-02-10T11:36:55.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.461 7fa5da68b700 1 -- 172.21.2.101:0/235330083 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d418a490 con 0x7fa5d40b8780 2021-02-10T11:36:55.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.462 7fa5d0ff9700 1 -- 172.21.2.101:0/235330083 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa5c8000c90 con 0x7fa5d40b8780 2021-02-10T11:36:55.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.462 7fa5d0ff9700 1 --2- 172.21.2.101:0/235330083 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5b40186c0 0x7fa5b401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:55.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.462 7fa5d0ff9700 1 -- 172.21.2.101:0/235330083 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5107+0+0 (secure 0 0 0) 0x7fa5c8032430 con 0x7fa5d40b8780 2021-02-10T11:36:55.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.462 7fa5d3fff700 1 --2- 172.21.2.101:0/235330083 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5b40186c0 0x7fa5b401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:55.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.462 7fa5d3fff700 1 --2- 172.21.2.101:0/235330083 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5b40186c0 0x7fa5b401abc0 crc :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:55.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.493 7fa5da68b700 1 -- 172.21.2.101:0/235330083 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d4082a20 con 0x7fa5d40b8780 2021-02-10T11:36:55.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.495 7fa5d0ff9700 1 -- 172.21.2.101:0/235330083 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa5c8030db0 con 0x7fa5d40b8780 2021-02-10T11:36:55.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:55.626 7fa5da68b700 1 -- 172.21.2.101:0/235330083 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"} v 0) v1 -- 0x7fa5d4082a20 con 0x7fa5d40b8780 2021-02-10T11:36:56.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.022 7fa5d0ff9700 1 -- 172.21.2.101:0/235330083 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"}]=0 pool 'cephfs_data' created v82) v1 ==== 125+0+0 (secure 0 0 0) 0x7fa5c8000c90 con 0x7fa5d40b8780 2021-02-10T11:36:56.047 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' created 2021-02-10T11:36:56.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.043 7fa5da68b700 1 -- 172.21.2.101:0/235330083 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5b40186c0 msgr2=0x7fa5b401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:56.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.043 7fa5da68b700 1 --2- 172.21.2.101:0/235330083 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5b40186c0 0x7fa5b401abc0 crc :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:56.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.043 7fa5da68b700 1 -- 172.21.2.101:0/235330083 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5d40b8780 msgr2=0x7fa5d40bfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:56.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.043 7fa5da68b700 1 --2- 172.21.2.101:0/235330083 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5d40b8780 0x7fa5d40bfa10 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 rx=0x7fa5c800b5e0 tx=0x7fa5c800b980).stop 2021-02-10T11:36:56.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.043 7fa5da68b700 1 -- 172.21.2.101:0/235330083 shutdown_connections 2021-02-10T11:36:56.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.043 7fa5da68b700 1 --2- 172.21.2.101:0/235330083 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5b40186c0 0x7fa5b401abc0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:56.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.043 7fa5da68b700 1 --2- 172.21.2.101:0/235330083 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5d40c0030 0x7fa5d4189e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:56.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.043 7fa5da68b700 1 --2- 172.21.2.101:0/235330083 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5d40b8780 0x7fa5d40bfa10 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:56.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.043 7fa5da68b700 1 --2- 172.21.2.101:0/235330083 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa5d41a05e0 0x7fa5d418b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:56.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.043 7fa5da68b700 1 -- 172.21.2.101:0/235330083 >> 172.21.2.101:0/235330083 conn(0x7fa5d40a6a80 msgr2=0x7fa5d40ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:56.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.043 7fa5da68b700 1 -- 172.21.2.101:0/235330083 shutdown_connections 2021-02-10T11:36:56.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.043 7fa5da68b700 1 -- 172.21.2.101:0/235330083 wait complete. 2021-02-10T11:36:56.057 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-02-10T11:36:56.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.466 7efd7e086700 1 Processor -- start 2021-02-10T11:36:56.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.466 7efd7e086700 1 -- start start 2021-02-10T11:36:56.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.466 7efd7e086700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7efd780d1880 0x7efd781b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:56.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.466 7efd7e086700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7efd781b0af0 0x7efd78108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:56.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.466 7efd7e086700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd780fe1d0 con 0x7efd780d1c80 2021-02-10T11:36:56.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.466 7efd7e086700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd78109a70 con 0x7efd780d1880 2021-02-10T11:36:56.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.466 7efd7e086700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd78109760 con 0x7efd781b0af0 2021-02-10T11:36:56.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.467 7efd777fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7efd780d1880 0x7efd781b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:56.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.467 7efd77fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7efd780d1c80 0x7efd78108f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54854/0 (socket says 172.21.2.101:54854) 2021-02-10T11:36:56.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.467 7efd77fff700 1 -- 172.21.2.101:0/1882110664 learned_addr learned my addr 172.21.2.101:0/1882110664 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:56.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.467 7efd76ffd700 1 --2- 172.21.2.101:0/1882110664 >> v2:172.21.2.108:3300/0 conn(0x7efd781b0af0 0x7efd78108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:56.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.467 7efd767fc700 1 -- 172.21.2.101:0/1882110664 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307499678 0 0) 0x7efd70001500 con 0x7efd780d1c80 2021-02-10T11:36:56.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.467 7efd76ffd700 1 -- 172.21.2.101:0/1882110664 >> v2:172.21.2.101:3300/0 conn(0x7efd780d1880 msgr2=0x7efd781b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:56.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.467 7efd76ffd700 1 --2- 172.21.2.101:0/1882110664 >> v2:172.21.2.101:3300/0 conn(0x7efd780d1880 0x7efd781b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:56.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.467 7efd76ffd700 1 -- 172.21.2.101:0/1882110664 >> v1:172.21.2.101:6789/0 conn(0x7efd780d1c80 legacy=0x7efd78108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:56.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.467 7efd76ffd700 1 -- 172.21.2.101:0/1882110664 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd78088070 con 0x7efd781b0af0 2021-02-10T11:36:56.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.467 7efd777fe700 1 --2- 172.21.2.101:0/1882110664 >> v2:172.21.2.101:3300/0 conn(0x7efd780d1880 0x7efd781b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:36:56.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.467 7efd76ffd700 1 --2- 172.21.2.101:0/1882110664 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd781b0af0 0x7efd78108950 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7efd68006650 tx=0x7efd68009a80).ready entity=mon.1 client_cookie=11bec8c1c1e4742 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:56.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.468 7efd767fc700 1 -- 172.21.2.101:0/1882110664 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd680009b0 con 0x7efd781b0af0 2021-02-10T11:36:56.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.468 7efd767fc700 1 -- 172.21.2.101:0/1882110664 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6800c0c0 con 0x7efd781b0af0 2021-02-10T11:36:56.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.468 7efd767fc700 1 -- 172.21.2.101:0/1882110664 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd6800c5e0 con 0x7efd781b0af0 2021-02-10T11:36:56.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.468 7efd7e086700 1 -- 172.21.2.101:0/1882110664 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd781b0af0 msgr2=0x7efd78108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:56.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.468 7efd7e086700 1 --2- 172.21.2.101:0/1882110664 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd781b0af0 0x7efd78108950 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7efd68006650 tx=0x7efd68009a80).stop 2021-02-10T11:36:56.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.469 7efd7e086700 1 -- 172.21.2.101:0/1882110664 shutdown_connections 2021-02-10T11:36:56.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.469 7efd7e086700 1 --2- 172.21.2.101:0/1882110664 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd781b0af0 0x7efd78108950 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:56.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.469 7efd7e086700 1 --2- 172.21.2.101:0/1882110664 >> v2:172.21.2.101:3300/0 conn(0x7efd780d1880 0x7efd781b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:56.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.469 7efd7e086700 1 -- 172.21.2.101:0/1882110664 >> 172.21.2.101:0/1882110664 conn(0x7efd780ce430 msgr2=0x7efd780d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:56.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.469 7efd7e086700 1 -- 172.21.2.101:0/1882110664 shutdown_connections 2021-02-10T11:36:56.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.469 7efd7e086700 1 -- 172.21.2.101:0/1882110664 wait complete. 2021-02-10T11:36:56.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.483 7efd7e086700 1 Processor -- start 2021-02-10T11:36:56.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.483 7efd7e086700 1 -- start start 2021-02-10T11:36:56.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.484 7efd7e086700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd781a3570 0x7efd781a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:56.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.484 7efd7e086700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd7809c9f0 0x7efd7809cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:56.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.484 7efd7e086700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd7809d410 0x7efd78098170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:56.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.484 7efd7e086700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7809c470 con 0x7efd7809c9f0 2021-02-10T11:36:56.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.484 7efd7e086700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd780b2840 con 0x7efd7809d410 2021-02-10T11:36:56.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.484 7efd7e086700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd780b2a20 con 0x7efd781a3570 2021-02-10T11:36:56.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.484 7efd76ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd7809c9f0 0x7efd7809cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:56.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.484 7efd76ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd7809c9f0 0x7efd7809cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54366/0 (socket says 172.21.2.101:54366) 2021-02-10T11:36:56.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.484 7efd76ffd700 1 -- 172.21.2.101:0/214648331 learned_addr learned my addr 172.21.2.101:0/214648331 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:56.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.484 7efd777fe700 1 --2- 172.21.2.101:0/214648331 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd781a3570 0x7efd781a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:56.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.484 7efd77fff700 1 --2- 172.21.2.101:0/214648331 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd7809d410 0x7efd78098170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:56.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.484 7efd76ffd700 1 -- 172.21.2.101:0/214648331 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd781a3570 msgr2=0x7efd781a3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:56.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.484 7efd76ffd700 1 --2- 172.21.2.101:0/214648331 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd781a3570 0x7efd781a3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:56.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.485 7efd76ffd700 1 -- 172.21.2.101:0/214648331 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd7809d410 msgr2=0x7efd78098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:56.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.485 7efd76ffd700 1 --2- 172.21.2.101:0/214648331 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd7809d410 0x7efd78098170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:56.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.485 7efd76ffd700 1 -- 172.21.2.101:0/214648331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7809d880 con 0x7efd7809c9f0 2021-02-10T11:36:56.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.485 7efd77fff700 1 --2- 172.21.2.101:0/214648331 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd7809d410 0x7efd78098170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:36:56.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.485 7efd76ffd700 1 --2- 172.21.2.101:0/214648331 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd7809c9f0 0x7efd7809cdf0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 rx=0x7efd68009630 tx=0x7efd6800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:56.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.485 7efd6ffff700 1 -- 172.21.2.101:0/214648331 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd680093a0 con 0x7efd7809c9f0 2021-02-10T11:36:56.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.485 7efd6ffff700 1 -- 172.21.2.101:0/214648331 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6800bd70 con 0x7efd7809c9f0 2021-02-10T11:36:56.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.485 7efd6ffff700 1 -- 172.21.2.101:0/214648331 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd68005560 con 0x7efd7809c9f0 2021-02-10T11:36:56.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.485 7efd7e086700 1 -- 172.21.2.101:0/214648331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7809d880 con 0x7efd7809c9f0 2021-02-10T11:36:56.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.485 7efd7e086700 1 -- 172.21.2.101:0/214648331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd7809d880 con 0x7efd7809c9f0 2021-02-10T11:36:56.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.486 7efd6ffff700 1 -- 172.21.2.101:0/214648331 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7efd680054f0 con 0x7efd7809c9f0 2021-02-10T11:36:56.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.486 7efd6ffff700 1 --2- 172.21.2.101:0/214648331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd5c0186c0 0x7efd5c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:56.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.486 7efd6ffff700 1 -- 172.21.2.101:0/214648331 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5457+0+0 (secure 0 0 0) 0x7efd68032390 con 0x7efd7809c9f0 2021-02-10T11:36:56.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.486 7efd777fe700 1 --2- 172.21.2.101:0/214648331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd5c0186c0 0x7efd5c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:56.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.486 7efd777fe700 1 --2- 172.21.2.101:0/214648331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd5c0186c0 0x7efd5c01abc0 crc :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:56.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.517 7efd7e086700 1 -- 172.21.2.101:0/214648331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd78082a70 con 0x7efd7809c9f0 2021-02-10T11:36:56.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.519 7efd6ffff700 1 -- 172.21.2.101:0/214648331 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7efd680052e0 con 0x7efd7809c9f0 2021-02-10T11:36:56.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.647 7efd7e086700 1 -- 172.21.2.101:0/214648331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"} v 0) v1 -- 0x7efd78082a70 con 0x7efd7809c9f0 2021-02-10T11:36:56.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.651 7efd6ffff700 1 -- 172.21.2.101:0/214648331 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"}]=0 new fs with metadata pool 16 and data pool 17 v60) v1 ==== 189+0+0 (secure 0 0 0) 0x7efd680054f0 con 0x7efd7809c9f0 2021-02-10T11:36:56.660 INFO:teuthology.orchestra.run.gibba001.stderr:new fs with metadata pool 16 and data pool 17 2021-02-10T11:36:56.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.656 7efd7e086700 1 -- 172.21.2.101:0/214648331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd5c0186c0 msgr2=0x7efd5c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:56.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.656 7efd7e086700 1 --2- 172.21.2.101:0/214648331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd5c0186c0 0x7efd5c01abc0 crc :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:56.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.656 7efd7e086700 1 -- 172.21.2.101:0/214648331 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd7809c9f0 msgr2=0x7efd7809cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:56.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.656 7efd7e086700 1 --2- 172.21.2.101:0/214648331 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd7809c9f0 0x7efd7809cdf0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 rx=0x7efd68009630 tx=0x7efd6800b980).stop 2021-02-10T11:36:56.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.657 7efd7e086700 1 -- 172.21.2.101:0/214648331 shutdown_connections 2021-02-10T11:36:56.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.657 7efd7e086700 1 --2- 172.21.2.101:0/214648331 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd7809d410 0x7efd78098170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:56.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.657 7efd7e086700 1 --2- 172.21.2.101:0/214648331 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd7809c9f0 0x7efd7809cdf0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:56.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.657 7efd7e086700 1 --2- 172.21.2.101:0/214648331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd5c0186c0 0x7efd5c01abc0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:56.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.657 7efd7e086700 1 --2- 172.21.2.101:0/214648331 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd781a3570 0x7efd781a3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:56.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.657 7efd7e086700 1 -- 172.21.2.101:0/214648331 >> 172.21.2.101:0/214648331 conn(0x7efd780c2cd0 msgr2=0x7efd780b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:56.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.657 7efd7e086700 1 -- 172.21.2.101:0/214648331 shutdown_connections 2021-02-10T11:36:56.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:56.657 7efd7e086700 1 -- 172.21.2.101:0/214648331 wait complete. 2021-02-10T11:36:56.671 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:36:57.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.075 7f1d3fee2700 1 Processor -- start 2021-02-10T11:36:57.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.075 7f1d3fee2700 1 -- start start 2021-02-10T11:36:57.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.076 7f1d3fee2700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1d380d1c70 0x7f1d380f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:57.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.076 7f1d3fee2700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d380f0b10 con 0x7f1d380d0340 2021-02-10T11:36:57.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.076 7f1d3fee2700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d381b0be0 con 0x7f1d380d1870 2021-02-10T11:36:57.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.076 7f1d3fee2700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d380e5ed0 con 0x7f1d380d1c70 2021-02-10T11:36:57.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.076 7f1d3d4fd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f1d380d0340 0x7f1d380ef640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54866/0 (socket says 172.21.2.101:54866) 2021-02-10T11:36:57.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.076 7f1d3d4fd700 1 -- 172.21.2.101:0/3442970489 learned_addr learned my addr 172.21.2.101:0/3442970489 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:57.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.076 7f1d3ccfc700 1 -- 172.21.2.101:0/3442970489 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229021751 0 0) 0x7f1d2c001180 con 0x7f1d380d0340 2021-02-10T11:36:57.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.076 7f1d3ccfc700 1 -- 172.21.2.101:0/3442970489 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d24002fe0 con 0x7f1d380d0340 2021-02-10T11:36:57.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.077 7f1d3e4ff700 1 --2- 172.21.2.101:0/3442970489 >> v2:172.21.2.108:3300/0 conn(0x7f1d380d1c70 0x7f1d380f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:57.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.077 7f1d3ccfc700 1 -- 172.21.2.101:0/3442970489 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4193356952 0 0) 0x7f1d2c001180 con 0x7f1d380d0340 2021-02-10T11:36:57.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.077 7f1d3ccfc700 1 -- 172.21.2.101:0/3442970489 >> v2:172.21.2.108:3300/0 conn(0x7f1d380d1c70 msgr2=0x7f1d380f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:57.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.077 7f1d3ccfc700 1 --2- 172.21.2.101:0/3442970489 >> v2:172.21.2.108:3300/0 conn(0x7f1d380d1c70 0x7f1d380f20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:57.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.077 7f1d3ccfc700 1 -- 172.21.2.101:0/3442970489 >> v1:172.21.2.108:6789/0 conn(0x7f1d380d1870 legacy=0x7f1d381b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:57.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.077 7f1d3ccfc700 1 -- 172.21.2.101:0/3442970489 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d380efe30 con 0x7f1d380d0340 2021-02-10T11:36:57.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.077 7f1d3ccfc700 1 -- 172.21.2.101:0/3442970489 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f1d2c001680 con 0x7f1d380d0340 2021-02-10T11:36:57.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.077 7f1d3ccfc700 1 -- 172.21.2.101:0/3442970489 >> v1:172.21.2.101:6789/0 conn(0x7f1d380d0340 legacy=0x7f1d380ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:57.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.077 7f1d3ccfc700 1 --2- 172.21.2.101:0/3442970489 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1d24003af0 0x7f1d24005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:57.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.077 7f1d3ccfc700 1 --2- 172.21.2.101:0/3442970489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d24006820 0x7f1d24008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:57.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.077 7f1d3ccfc700 1 --2- 172.21.2.101:0/3442970489 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d24009640 0x7f1d2400bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:57.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.077 7f1d3ccfc700 1 -- 172.21.2.101:0/3442970489 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2400c3b0 con 0x7f1d24006820 2021-02-10T11:36:57.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.077 7f1d3ccfc700 1 -- 172.21.2.101:0/3442970489 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2400c5d0 con 0x7f1d24003af0 2021-02-10T11:36:57.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.077 7f1d3ccfc700 1 -- 172.21.2.101:0/3442970489 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2400c7f0 con 0x7f1d24009640 2021-02-10T11:36:57.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.077 7f1d3e4ff700 1 --2- 172.21.2.101:0/3442970489 >> v2:172.21.2.108:3300/0 conn(0x7f1d380d1c70 0x7f1d380f20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:36:57.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.077 7f1d3dcfe700 1 --2- 172.21.2.101:0/3442970489 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1d24003af0 0x7f1d24005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:57.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.077 7f1d3d4fd700 1 --2- 172.21.2.101:0/3442970489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d24006820 0x7f1d24008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:57.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.077 7f1d3e4ff700 1 --2- 172.21.2.101:0/3442970489 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d24009640 0x7f1d2400bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:57.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.077 7f1d3d4fd700 1 -- 172.21.2.101:0/3442970489 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d24009640 msgr2=0x7f1d2400bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:57.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.077 7f1d3d4fd700 1 --2- 172.21.2.101:0/3442970489 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d24009640 0x7f1d2400bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:57.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.077 7f1d3d4fd700 1 -- 172.21.2.101:0/3442970489 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1d24003af0 msgr2=0x7f1d24005f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:57.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.078 7f1d3d4fd700 1 --2- 172.21.2.101:0/3442970489 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1d24003af0 0x7f1d24005f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:57.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.078 7f1d3d4fd700 1 -- 172.21.2.101:0/3442970489 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2400ca90 con 0x7f1d24006820 2021-02-10T11:36:57.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.078 7f1d3e4ff700 1 -- 172.21.2.101:0/3442970489 reap_dead start 2021-02-10T11:36:57.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.078 7f1d3dcfe700 1 --2- 172.21.2.101:0/3442970489 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1d24003af0 0x7f1d24005f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:36:57.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.078 7f1d3d4fd700 1 --2- 172.21.2.101:0/3442970489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d24006820 0x7f1d24008d40 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 rx=0x7f1d2c001250 tx=0x7f1d2c007cf0).ready entity=mon.0 client_cookie=ff4a2d33c0968508 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:57.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.078 7f1d3ccfc700 1 -- 172.21.2.101:0/3442970489 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1d2c0070c0 con 0x7f1d24006820 2021-02-10T11:36:57.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.078 7f1d3ccfc700 1 -- 172.21.2.101:0/3442970489 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2c0043a0 con 0x7f1d24006820 2021-02-10T11:36:57.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.078 7f1d3ccfc700 1 -- 172.21.2.101:0/3442970489 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1d2c0047f0 con 0x7f1d24006820 2021-02-10T11:36:57.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.078 7f1d3fee2700 1 -- 172.21.2.101:0/3442970489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d24006820 msgr2=0x7f1d24008d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:57.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.078 7f1d3fee2700 1 --2- 172.21.2.101:0/3442970489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d24006820 0x7f1d24008d40 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 rx=0x7f1d2c001250 tx=0x7f1d2c007cf0).stop 2021-02-10T11:36:57.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.079 7f1d3fee2700 1 -- 172.21.2.101:0/3442970489 shutdown_connections 2021-02-10T11:36:57.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.079 7f1d3fee2700 1 --2- 172.21.2.101:0/3442970489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d24006820 0x7f1d24008d40 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:57.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.079 7f1d3fee2700 1 -- 172.21.2.101:0/3442970489 >> 172.21.2.101:0/3442970489 conn(0x7f1d380ce450 msgr2=0x7f1d380d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:57.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.079 7f1d3fee2700 1 -- 172.21.2.101:0/3442970489 shutdown_connections 2021-02-10T11:36:57.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.079 7f1d3fee2700 1 -- 172.21.2.101:0/3442970489 wait complete. 2021-02-10T11:36:57.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.093 7f1d3fee2700 1 Processor -- start 2021-02-10T11:36:57.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.093 7f1d3fee2700 1 -- start start 2021-02-10T11:36:57.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.094 7f1d3fee2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d38195650 0x7f1d380b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:57.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.094 7f1d3fee2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1d380b5b80 0x7f1d381a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:57.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.094 7f1d3fee2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d380acfa0 0x7f1d381a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:57.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.094 7f1d3fee2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d380b6010 con 0x7f1d38195650 2021-02-10T11:36:57.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.094 7f1d3fee2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d381a3a60 con 0x7f1d380b5b80 2021-02-10T11:36:57.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.094 7f1d3fee2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d381a3c40 con 0x7f1d380acfa0 2021-02-10T11:36:57.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.094 7f1d3dcfe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d38195650 0x7f1d380b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:57.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.094 7f1d3dcfe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d38195650 0x7f1d380b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54386/0 (socket says 172.21.2.101:54386) 2021-02-10T11:36:57.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.094 7f1d3dcfe700 1 -- 172.21.2.101:0/3444549874 learned_addr learned my addr 172.21.2.101:0/3444549874 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:57.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.094 7f1d3dcfe700 1 -- 172.21.2.101:0/3444549874 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d380acfa0 msgr2=0x7f1d381a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:57.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.094 7f1d3dcfe700 1 --2- 172.21.2.101:0/3444549874 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d380acfa0 0x7f1d381a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:57.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.094 7f1d3dcfe700 1 -- 172.21.2.101:0/3444549874 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1d380b5b80 msgr2=0x7f1d381a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:57.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.094 7f1d3dcfe700 1 --2- 172.21.2.101:0/3444549874 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1d380b5b80 0x7f1d381a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:57.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.094 7f1d3dcfe700 1 -- 172.21.2.101:0/3444549874 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d380bcae0 con 0x7f1d38195650 2021-02-10T11:36:57.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.094 7f1d3dcfe700 1 --2- 172.21.2.101:0/3444549874 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d38195650 0x7f1d380b5560 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 rx=0x7f1d3404aac0 tx=0x7f1d3404f990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:57.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.095 7f1d2a7fc700 1 -- 172.21.2.101:0/3444549874 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1d3404abf0 con 0x7f1d38195650 2021-02-10T11:36:57.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.095 7f1d2a7fc700 1 -- 172.21.2.101:0/3444549874 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3404a030 con 0x7f1d38195650 2021-02-10T11:36:57.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.095 7f1d2a7fc700 1 -- 172.21.2.101:0/3444549874 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1d34049550 con 0x7f1d38195650 2021-02-10T11:36:57.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.095 7f1d3fee2700 1 -- 172.21.2.101:0/3444549874 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d380bcae0 con 0x7f1d38195650 2021-02-10T11:36:57.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.095 7f1d3fee2700 1 -- 172.21.2.101:0/3444549874 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d380bd070 con 0x7f1d38195650 2021-02-10T11:36:57.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.095 7f1d2a7fc700 1 -- 172.21.2.101:0/3444549874 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1d3404a210 con 0x7f1d38195650 2021-02-10T11:36:57.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.095 7f1d2a7fc700 1 --2- 172.21.2.101:0/3444549874 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d200186c0 0x7f1d2001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:57.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.095 7f1d2a7fc700 1 -- 172.21.2.101:0/3444549874 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1d34076570 con 0x7f1d38195650 2021-02-10T11:36:57.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.096 7f1d3d4fd700 1 --2- 172.21.2.101:0/3444549874 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d200186c0 0x7f1d2001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:57.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.096 7f1d3d4fd700 1 --2- 172.21.2.101:0/3444549874 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d200186c0 0x7f1d2001abc0 crc :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:57.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.126 7f1d3fee2700 1 -- 172.21.2.101:0/3444549874 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d380b6010 con 0x7f1d38195650 2021-02-10T11:36:57.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.128 7f1d2a7fc700 1 -- 172.21.2.101:0/3444549874 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1d3404a720 con 0x7f1d38195650 2021-02-10T11:36:57.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.253 7f1d3fee2700 1 -- 172.21.2.101:0/3444549874 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d380b6010 con 0x7f1d38195650 2021-02-10T11:36:57.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.254 7f1d2a7fc700 1 -- 172.21.2.101:0/3444549874 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+14009 (secure 0 0 0) 0x7f1d3404a210 con 0x7f1d38195650 2021-02-10T11:36:57.260 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:36:57.261 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":83,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:36:56.648670","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:36:54.618463","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2021-02-10 11:36:55.644563","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:36:57.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.256 7f1d3fee2700 1 -- 172.21.2.101:0/3444549874 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d200186c0 msgr2=0x7f1d2001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:57.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.256 7f1d3fee2700 1 --2- 172.21.2.101:0/3444549874 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d200186c0 0x7f1d2001abc0 crc :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:57.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.256 7f1d3fee2700 1 -- 172.21.2.101:0/3444549874 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d38195650 msgr2=0x7f1d380b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:57.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.256 7f1d3fee2700 1 --2- 172.21.2.101:0/3444549874 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d38195650 0x7f1d380b5560 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 rx=0x7f1d3404aac0 tx=0x7f1d3404f990).stop 2021-02-10T11:36:57.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.256 7f1d3fee2700 1 -- 172.21.2.101:0/3444549874 shutdown_connections 2021-02-10T11:36:57.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.256 7f1d3fee2700 1 --2- 172.21.2.101:0/3444549874 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d200186c0 0x7f1d2001abc0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:57.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.256 7f1d3fee2700 1 --2- 172.21.2.101:0/3444549874 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d380acfa0 0x7f1d381a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:57.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.256 7f1d3fee2700 1 --2- 172.21.2.101:0/3444549874 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1d380b5b80 0x7f1d381a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:57.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.256 7f1d3fee2700 1 --2- 172.21.2.101:0/3444549874 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d38195650 0x7f1d380b5560 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:57.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.256 7f1d3fee2700 1 -- 172.21.2.101:0/3444549874 >> 172.21.2.101:0/3444549874 conn(0x7f1d380b3040 msgr2=0x7f1d380c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:57.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.256 7f1d3fee2700 1 -- 172.21.2.101:0/3444549874 shutdown_connections 2021-02-10T11:36:57.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.256 7f1d3fee2700 1 -- 172.21.2.101:0/3444549874 wait complete. 2021-02-10T11:36:57.271 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:36:57.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.675 7f77c0fd4700 1 Processor -- start 2021-02-10T11:36:57.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.676 7f77c0fd4700 1 -- start start 2021-02-10T11:36:57.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.676 7f77c0fd4700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f77bc0d0340 0x7f77bc0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:57.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.676 7f77c0fd4700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77bc0f0b10 con 0x7f77bc0d1870 2021-02-10T11:36:57.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.676 7f77c0fd4700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77bc1b0be0 con 0x7f77bc0d1c70 2021-02-10T11:36:57.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.676 7f77c0fd4700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77bc0e5ed0 con 0x7f77bc0d0340 2021-02-10T11:36:57.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.676 7f77bad9d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f77bc0d1870 0x7f77bc1b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54884/0 (socket says 172.21.2.101:54884) 2021-02-10T11:36:57.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.676 7f77bad9d700 1 -- 172.21.2.101:0/2299561520 learned_addr learned my addr 172.21.2.101:0/2299561520 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:36:57.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.676 7f77ba59c700 1 --2- 172.21.2.101:0/2299561520 >> v2:172.21.2.108:3300/0 conn(0x7f77bc0d0340 0x7f77bc0ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:57.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.677 7f77b9d9b700 1 -- 172.21.2.101:0/2299561520 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053909413 0 0) 0x7f77b0001180 con 0x7f77bc0d1870 2021-02-10T11:36:57.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.677 7f77b9d9b700 1 -- 172.21.2.101:0/2299561520 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77a4002fe0 con 0x7f77bc0d1870 2021-02-10T11:36:57.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.677 7f77ba59c700 1 -- 172.21.2.101:0/2299561520 >> v1:172.21.2.108:6789/0 conn(0x7f77bc0d1c70 legacy=0x7f77bc0efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:57.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.677 7f77ba59c700 1 -- 172.21.2.101:0/2299561520 >> v1:172.21.2.101:6789/0 conn(0x7f77bc0d1870 legacy=0x7f77bc1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:57.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.677 7f77ba59c700 1 -- 172.21.2.101:0/2299561520 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77bc0f0450 con 0x7f77bc0d0340 2021-02-10T11:36:57.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.677 7f77ba59c700 1 --2- 172.21.2.101:0/2299561520 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77bc0d0340 0x7f77bc0ef640 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f77b40069d0 tx=0x7f77b4009e40).ready entity=mon.1 client_cookie=5ecdad3f271cfab3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:57.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.677 7f77b9d9b700 1 -- 172.21.2.101:0/2299561520 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f77b400b510 con 0x7f77bc0d0340 2021-02-10T11:36:57.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.677 7f77b9d9b700 1 -- 172.21.2.101:0/2299561520 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b400b790 con 0x7f77bc0d0340 2021-02-10T11:36:57.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.677 7f77b9d9b700 1 -- 172.21.2.101:0/2299561520 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f77b400bc70 con 0x7f77bc0d0340 2021-02-10T11:36:57.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.677 7f77c0fd4700 1 -- 172.21.2.101:0/2299561520 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77bc0d0340 msgr2=0x7f77bc0ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:57.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.677 7f77c0fd4700 1 --2- 172.21.2.101:0/2299561520 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77bc0d0340 0x7f77bc0ef640 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f77b40069d0 tx=0x7f77b4009e40).stop 2021-02-10T11:36:57.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.678 7f77c0fd4700 1 -- 172.21.2.101:0/2299561520 shutdown_connections 2021-02-10T11:36:57.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.678 7f77c0fd4700 1 --2- 172.21.2.101:0/2299561520 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77bc0d0340 0x7f77bc0ef640 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:57.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.678 7f77c0fd4700 1 -- 172.21.2.101:0/2299561520 >> 172.21.2.101:0/2299561520 conn(0x7f77bc0ce450 msgr2=0x7f77bc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:57.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.678 7f77c0fd4700 1 -- 172.21.2.101:0/2299561520 shutdown_connections 2021-02-10T11:36:57.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.678 7f77c0fd4700 1 -- 172.21.2.101:0/2299561520 wait complete. 2021-02-10T11:36:57.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.692 7f77c0fd4700 1 Processor -- start 2021-02-10T11:36:57.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.692 7f77c0fd4700 1 -- start start 2021-02-10T11:36:57.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.693 7f77c0fd4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77bc18f0a0 0x7f77bc196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:57.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.693 7f77c0fd4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77bc1970d0 0x7f77bc1974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:57.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.693 7f77c0fd4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77bc094650 0x7f77bc094a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:57.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.693 7f77c0fd4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc095bd0 con 0x7f77bc18f0a0 2021-02-10T11:36:57.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.693 7f77c0fd4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc0ad7c0 con 0x7f77bc1970d0 2021-02-10T11:36:57.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.693 7f77c0fd4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc18ec70 con 0x7f77bc094650 2021-02-10T11:36:57.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.693 7f77bad9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77bc18f0a0 0x7f77bc196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:57.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.693 7f77bad9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77bc18f0a0 0x7f77bc196ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54394/0 (socket says 172.21.2.101:54394) 2021-02-10T11:36:57.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.693 7f77bad9d700 1 -- 172.21.2.101:0/2023050598 learned_addr learned my addr 172.21.2.101:0/2023050598 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:57.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.693 7f77bb59e700 1 --2- 172.21.2.101:0/2023050598 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77bc094650 0x7f77bc094a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:57.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.693 7f77ba59c700 1 --2- 172.21.2.101:0/2023050598 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77bc1970d0 0x7f77bc1974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:57.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.693 7f77bb59e700 1 -- 172.21.2.101:0/2023050598 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77bc1970d0 msgr2=0x7f77bc1974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:57.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.693 7f77bb59e700 1 --2- 172.21.2.101:0/2023050598 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77bc1970d0 0x7f77bc1974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:57.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.693 7f77bb59e700 1 -- 172.21.2.101:0/2023050598 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77bc18f0a0 msgr2=0x7f77bc196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:57.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.693 7f77bb59e700 1 --2- 172.21.2.101:0/2023050598 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77bc18f0a0 0x7f77bc196ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:57.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.694 7f77bb59e700 1 -- 172.21.2.101:0/2023050598 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77bc095070 con 0x7f77bc094650 2021-02-10T11:36:57.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.694 7f77ba59c700 1 --2- 172.21.2.101:0/2023050598 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77bc1970d0 0x7f77bc1974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:36:57.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.694 7f77bad9d700 1 --2- 172.21.2.101:0/2023050598 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77bc18f0a0 0x7f77bc196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:36:57.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.694 7f77bb59e700 1 --2- 172.21.2.101:0/2023050598 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77bc094650 0x7f77bc094a50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f77ac006e80 tx=0x7f77ac00b8b0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:57.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.694 7f77ab7fe700 1 -- 172.21.2.101:0/2023050598 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f77ac006b50 con 0x7f77bc094650 2021-02-10T11:36:57.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.694 7f77ab7fe700 1 -- 172.21.2.101:0/2023050598 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ac006360 con 0x7f77bc094650 2021-02-10T11:36:57.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.694 7f77ab7fe700 1 -- 172.21.2.101:0/2023050598 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f77ac002080 con 0x7f77bc094650 2021-02-10T11:36:57.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.694 7f77c0fd4700 1 -- 172.21.2.101:0/2023050598 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77bc095070 con 0x7f77bc094650 2021-02-10T11:36:57.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.694 7f77c0fd4700 1 -- 172.21.2.101:0/2023050598 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77bc095070 con 0x7f77bc094650 2021-02-10T11:36:57.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.694 7f77ab7fe700 1 -- 172.21.2.101:0/2023050598 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f77ac002020 con 0x7f77bc094650 2021-02-10T11:36:57.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.694 7f77ab7fe700 1 --2- 172.21.2.101:0/2023050598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77a00186d0 0x7f77a001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:57.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.694 7f77ab7fe700 1 -- 172.21.2.101:0/2023050598 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f77ac032710 con 0x7f77bc094650 2021-02-10T11:36:57.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.695 7f77bad9d700 1 --2- 172.21.2.101:0/2023050598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77a00186d0 0x7f77a001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:57.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.695 7f77bad9d700 1 --2- 172.21.2.101:0/2023050598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77a00186d0 0x7f77a001abd0 crc :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:57.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.726 7f77c0fd4700 1 -- 172.21.2.101:0/2023050598 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77bc0ad790 con 0x7f77bc094650 2021-02-10T11:36:57.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.730 7f77ab7fe700 1 -- 172.21.2.101:0/2023050598 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f77ac000c90 con 0x7f77bc094650 2021-02-10T11:36:57.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.858 7f77c0fd4700 1 -- 172.21.2.101:0/2023050598 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77bc0ad790 con 0x7f77bc094650 2021-02-10T11:36:57.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.859 7f77ab7fe700 1 -- 172.21.2.101:0/2023050598 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+14009 (secure 0 0 0) 0x7f77ac006360 con 0x7f77bc094650 2021-02-10T11:36:57.867 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:36:57.867 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":84,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:36:57.650115","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:36:54.618463","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2021-02-10 11:36:55.644563","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:36:57.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.862 7f77c0fd4700 1 -- 172.21.2.101:0/2023050598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77a00186d0 msgr2=0x7f77a001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:57.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.862 7f77c0fd4700 1 --2- 172.21.2.101:0/2023050598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77a00186d0 0x7f77a001abd0 crc :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:57.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.863 7f77c0fd4700 1 -- 172.21.2.101:0/2023050598 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77bc094650 msgr2=0x7f77bc094a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:57.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.863 7f77c0fd4700 1 --2- 172.21.2.101:0/2023050598 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77bc094650 0x7f77bc094a50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f77ac006e80 tx=0x7f77ac00b8b0).stop 2021-02-10T11:36:57.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.863 7f77c0fd4700 1 -- 172.21.2.101:0/2023050598 shutdown_connections 2021-02-10T11:36:57.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.863 7f77c0fd4700 1 --2- 172.21.2.101:0/2023050598 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77a00186d0 0x7f77a001abd0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:57.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.863 7f77c0fd4700 1 --2- 172.21.2.101:0/2023050598 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77bc094650 0x7f77bc094a50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:57.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.863 7f77c0fd4700 1 --2- 172.21.2.101:0/2023050598 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77bc1970d0 0x7f77bc1974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:57.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.863 7f77c0fd4700 1 --2- 172.21.2.101:0/2023050598 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77bc18f0a0 0x7f77bc196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:57.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.863 7f77c0fd4700 1 -- 172.21.2.101:0/2023050598 >> 172.21.2.101:0/2023050598 conn(0x7f77bc1a38c0 msgr2=0x7f77bc1a2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:57.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.863 7f77c0fd4700 1 -- 172.21.2.101:0/2023050598 shutdown_connections 2021-02-10T11:36:57.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:57.863 7f77c0fd4700 1 -- 172.21.2.101:0/2023050598 wait complete. 2021-02-10T11:36:57.877 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-02-10T11:36:58.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.284 7f3ed4046700 1 Processor -- start 2021-02-10T11:36:58.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.284 7f3ed4046700 1 -- start start 2021-02-10T11:36:58.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.285 7f3ed4046700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ecc0d1870 0x7f3ecc1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:58.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.285 7f3ed4046700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3ecc0cff70 0x7f3ecc109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:58.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.285 7f3ed4046700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ecc109a60 con 0x7f3ecc1b0ae0 2021-02-10T11:36:58.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.285 7f3ed4046700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ecc109750 con 0x7f3ecc0cff70 2021-02-10T11:36:58.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.285 7f3ed4046700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc088070 con 0x7f3ecc0d1870 2021-02-10T11:36:58.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.285 7f3ed1e62700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ecc0d1870 0x7f3ecc1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:58.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.285 7f3ed1e62700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ecc0d1870 0x7f3ecc1b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:47134/0 (socket says 172.21.2.101:47134) 2021-02-10T11:36:58.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.285 7f3ed1e62700 1 -- 172.21.2.101:0/1221670249 learned_addr learned my addr 172.21.2.101:0/1221670249 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:58.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.285 7f3ed2663700 1 --2- 172.21.2.101:0/1221670249 >> v2:172.21.2.108:3300/0 conn(0x7f3ecc0cff70 0x7f3ecc109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:58.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.286 7f3ed1e62700 1 -- 172.21.2.101:0/1221670249 >> v2:172.21.2.108:3300/0 conn(0x7f3ecc0cff70 msgr2=0x7f3ecc109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:58.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.286 7f3ed1e62700 1 --2- 172.21.2.101:0/1221670249 >> v2:172.21.2.108:3300/0 conn(0x7f3ecc0cff70 0x7f3ecc109130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:58.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.286 7f3ed1e62700 1 -- 172.21.2.101:0/1221670249 >> v1:172.21.2.108:6789/0 conn(0x7f3ecc1b0ae0 legacy=0x7f3ecc108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:58.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.286 7f3ed1e62700 1 -- 172.21.2.101:0/1221670249 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc088260 con 0x7f3ecc0d1870 2021-02-10T11:36:58.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.286 7f3ed1e62700 1 --2- 172.21.2.101:0/1221670249 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ecc0d1870 0x7f3ecc1b04c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f3ec0006700 tx=0x7f3ec0009b80).ready entity=mon.2 client_cookie=6fb6f9f46cce1903 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:58.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.286 7f3ed0e60700 1 -- 172.21.2.101:0/1221670249 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3ec000c0c0 con 0x7f3ecc0d1870 2021-02-10T11:36:58.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.286 7f3ed0e60700 1 -- 172.21.2.101:0/1221670249 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec000c440 con 0x7f3ecc0d1870 2021-02-10T11:36:58.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.286 7f3ed0e60700 1 -- 172.21.2.101:0/1221670249 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3ec000c960 con 0x7f3ecc0d1870 2021-02-10T11:36:58.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.286 7f3ed4046700 1 -- 172.21.2.101:0/1221670249 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ecc0d1870 msgr2=0x7f3ecc1b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:58.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.286 7f3ed4046700 1 --2- 172.21.2.101:0/1221670249 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ecc0d1870 0x7f3ecc1b04c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f3ec0006700 tx=0x7f3ec0009b80).stop 2021-02-10T11:36:58.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.287 7f3ed4046700 1 -- 172.21.2.101:0/1221670249 shutdown_connections 2021-02-10T11:36:58.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.287 7f3ed4046700 1 --2- 172.21.2.101:0/1221670249 >> v2:172.21.2.108:3300/0 conn(0x7f3ecc0cff70 0x7f3ecc109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:58.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.287 7f3ed4046700 1 --2- 172.21.2.101:0/1221670249 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ecc0d1870 0x7f3ecc1b04c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:58.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.287 7f3ed4046700 1 -- 172.21.2.101:0/1221670249 >> 172.21.2.101:0/1221670249 conn(0x7f3ecc0ce420 msgr2=0x7f3ecc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:58.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.287 7f3ed4046700 1 -- 172.21.2.101:0/1221670249 shutdown_connections 2021-02-10T11:36:58.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.287 7f3ed4046700 1 -- 172.21.2.101:0/1221670249 wait complete. 2021-02-10T11:36:58.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.302 7f3ed4046700 1 Processor -- start 2021-02-10T11:36:58.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.302 7f3ed4046700 1 -- start start 2021-02-10T11:36:58.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.302 7f3ed4046700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ecc0b1ba0 0x7f3ecc1a78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:58.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.302 7f3ed4046700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ecc1a7ef0 0x7f3ecc0a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:58.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.302 7f3ed4046700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ecc0a7060 0x7f3ecc0b2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:58.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.302 7f3ed4046700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc190c20 con 0x7f3ecc0b1ba0 2021-02-10T11:36:58.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.302 7f3ed4046700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc190710 con 0x7f3ecc1a7ef0 2021-02-10T11:36:58.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.302 7f3ed4046700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc0b1fa0 con 0x7f3ecc0a7060 2021-02-10T11:36:58.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.302 7f3ed1e62700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ecc0b1ba0 0x7f3ecc1a78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:58.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.302 7f3ed1e62700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ecc0b1ba0 0x7f3ecc1a78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54408/0 (socket says 172.21.2.101:54408) 2021-02-10T11:36:58.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.302 7f3ed1e62700 1 -- 172.21.2.101:0/3865627173 learned_addr learned my addr 172.21.2.101:0/3865627173 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:58.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.303 7f3ed1e62700 1 -- 172.21.2.101:0/3865627173 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ecc0a7060 msgr2=0x7f3ecc0b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:58.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.303 7f3ed1e62700 1 --2- 172.21.2.101:0/3865627173 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ecc0a7060 0x7f3ecc0b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:58.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.303 7f3ed1e62700 1 -- 172.21.2.101:0/3865627173 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ecc1a7ef0 msgr2=0x7f3ecc0a6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:58.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.303 7f3ed1e62700 1 --2- 172.21.2.101:0/3865627173 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ecc1a7ef0 0x7f3ecc0a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:58.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.303 7f3ed1e62700 1 -- 172.21.2.101:0/3865627173 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc0a7460 con 0x7f3ecc0b1ba0 2021-02-10T11:36:58.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.303 7f3ed1e62700 1 --2- 172.21.2.101:0/3865627173 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ecc0b1ba0 0x7f3ecc1a78d0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 rx=0x7f3ec000b580 tx=0x7f3ec000b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:58.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.303 7f3ebe7fc700 1 -- 172.21.2.101:0/3865627173 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3ec000bd10 con 0x7f3ecc0b1ba0 2021-02-10T11:36:58.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.303 7f3ebe7fc700 1 -- 172.21.2.101:0/3865627173 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec0002030 con 0x7f3ecc0b1ba0 2021-02-10T11:36:58.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.303 7f3ebe7fc700 1 -- 172.21.2.101:0/3865627173 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3ec0002480 con 0x7f3ecc0b1ba0 2021-02-10T11:36:58.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.303 7f3ed4046700 1 -- 172.21.2.101:0/3865627173 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ecc0a7460 con 0x7f3ecc0b1ba0 2021-02-10T11:36:58.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.303 7f3ed4046700 1 -- 172.21.2.101:0/3865627173 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ecc0a7460 con 0x7f3ecc0b1ba0 2021-02-10T11:36:58.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.304 7f3ebe7fc700 1 -- 172.21.2.101:0/3865627173 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3ec0000c90 con 0x7f3ecc0b1ba0 2021-02-10T11:36:58.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.304 7f3ebe7fc700 1 --2- 172.21.2.101:0/3865627173 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3eb40186c0 0x7f3eb401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:58.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.304 7f3ebe7fc700 1 -- 172.21.2.101:0/3865627173 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3ec0032580 con 0x7f3ecc0b1ba0 2021-02-10T11:36:58.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.304 7f3ed1661700 1 --2- 172.21.2.101:0/3865627173 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3eb40186c0 0x7f3eb401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:58.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.304 7f3ed1661700 1 --2- 172.21.2.101:0/3865627173 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3eb40186c0 0x7f3eb401abc0 crc :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:58.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.334 7f3ed4046700 1 -- 172.21.2.101:0/3865627173 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ecc082980 con 0x7f3ecc0b1ba0 2021-02-10T11:36:58.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.337 7f3ebe7fc700 1 -- 172.21.2.101:0/3865627173 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3ec0030d30 con 0x7f3ecc0b1ba0 2021-02-10T11:36:58.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.472 7f3ed4046700 1 -- 172.21.2.101:0/3865627173 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"} v 0) v1 -- 0x7f3ecc109a60 con 0x7f3ecc0b1ba0 2021-02-10T11:36:58.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.657 7f3ebe7fc700 1 -- 172.21.2.101:0/3865627173 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"}]=0 v62) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3ec0000c90 con 0x7f3ecc0b1ba0 2021-02-10T11:36:58.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.689 7f3ed4046700 1 -- 172.21.2.101:0/3865627173 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3eb40186c0 msgr2=0x7f3eb401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:58.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.689 7f3ed4046700 1 --2- 172.21.2.101:0/3865627173 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3eb40186c0 0x7f3eb401abc0 crc :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:58.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.689 7f3ed4046700 1 -- 172.21.2.101:0/3865627173 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ecc0b1ba0 msgr2=0x7f3ecc1a78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:58.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.689 7f3ed4046700 1 --2- 172.21.2.101:0/3865627173 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ecc0b1ba0 0x7f3ecc1a78d0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 rx=0x7f3ec000b580 tx=0x7f3ec000b920).stop 2021-02-10T11:36:58.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.690 7f3ed4046700 1 -- 172.21.2.101:0/3865627173 shutdown_connections 2021-02-10T11:36:58.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.690 7f3ed4046700 1 --2- 172.21.2.101:0/3865627173 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3eb40186c0 0x7f3eb401abc0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:58.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.690 7f3ed4046700 1 --2- 172.21.2.101:0/3865627173 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ecc0a7060 0x7f3ecc0b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:58.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.690 7f3ed4046700 1 --2- 172.21.2.101:0/3865627173 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ecc1a7ef0 0x7f3ecc0a6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:58.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.690 7f3ed4046700 1 --2- 172.21.2.101:0/3865627173 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ecc0b1ba0 0x7f3ecc1a78d0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:58.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.690 7f3ed4046700 1 -- 172.21.2.101:0/3865627173 >> 172.21.2.101:0/3865627173 conn(0x7f3ecc0b5dc0 msgr2=0x7f3ecc095960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:58.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.690 7f3ed4046700 1 -- 172.21.2.101:0/3865627173 shutdown_connections 2021-02-10T11:36:58.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:58.690 7f3ed4046700 1 -- 172.21.2.101:0/3865627173 wait complete. 2021-02-10T11:36:58.704 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:36:59.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.107 7f61e0472700 1 Processor -- start 2021-02-10T11:36:59.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.107 7f61e0472700 1 -- start start 2021-02-10T11:36:59.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.108 7f61e0472700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f61d80d0340 0x7f61d80ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:59.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.108 7f61e0472700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f61d80d1c70 0x7f61d80efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:59.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.108 7f61e0472700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d80f0b10 con 0x7f61d80d1870 2021-02-10T11:36:59.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.108 7f61e0472700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d80e5ed0 con 0x7f61d80d1c70 2021-02-10T11:36:59.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.108 7f61e0472700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d81b0be0 con 0x7f61d80d0340 2021-02-10T11:36:59.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.108 7f61dea8f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f61d80d1c70 0x7f61d80efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:59.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.108 7f61dea8f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f61d80d1c70 0x7f61d80efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54418/0 (socket says 172.21.2.101:54418) 2021-02-10T11:36:59.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.108 7f61dea8f700 1 -- 172.21.2.101:0/1524044883 learned_addr learned my addr 172.21.2.101:0/1524044883 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:59.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.108 7f61dda8d700 1 --2- 172.21.2.101:0/1524044883 >> v2:172.21.2.108:3300/0 conn(0x7f61d80d0340 0x7f61d80ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:59.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.108 7f61dd28c700 1 -- 172.21.2.101:0/1524044883 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198249231 0 0) 0x7f61cc000ff0 con 0x7f61d80d1870 2021-02-10T11:36:59.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.108 7f61dd28c700 1 -- 172.21.2.101:0/1524044883 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61c4002fe0 con 0x7f61d80d1870 2021-02-10T11:36:59.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.108 7f61dea8f700 1 -- 172.21.2.101:0/1524044883 >> v2:172.21.2.108:3300/0 conn(0x7f61d80d0340 msgr2=0x7f61d80ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:59.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.108 7f61dea8f700 1 --2- 172.21.2.101:0/1524044883 >> v2:172.21.2.108:3300/0 conn(0x7f61d80d0340 0x7f61d80ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:59.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.108 7f61dea8f700 1 -- 172.21.2.101:0/1524044883 >> v1:172.21.2.101:6789/0 conn(0x7f61d80d1870 legacy=0x7f61d81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.108 7f61dea8f700 1 -- 172.21.2.101:0/1524044883 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d80f20a0 con 0x7f61d80d1c70 2021-02-10T11:36:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.109 7f61dea8f700 1 --2- 172.21.2.101:0/1524044883 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f61d80d1c70 0x7f61d80efc60 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 rx=0x7f61d0006f00 tx=0x7f61d0009f10).ready entity=mon.0 client_cookie=98fdd9fe1ff3bb6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.109 7f61dd28c700 1 -- 172.21.2.101:0/1524044883 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f61d000b3b0 con 0x7f61d80d1c70 2021-02-10T11:36:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.109 7f61dd28c700 1 -- 172.21.2.101:0/1524044883 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d000b730 con 0x7f61d80d1c70 2021-02-10T11:36:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.109 7f61dd28c700 1 -- 172.21.2.101:0/1524044883 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f61d000bb90 con 0x7f61d80d1c70 2021-02-10T11:36:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.109 7f61e0472700 1 -- 172.21.2.101:0/1524044883 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f61d80d1c70 msgr2=0x7f61d80efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.109 7f61e0472700 1 --2- 172.21.2.101:0/1524044883 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f61d80d1c70 0x7f61d80efc60 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 rx=0x7f61d0006f00 tx=0x7f61d0009f10).stop 2021-02-10T11:36:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.109 7f61e0472700 1 -- 172.21.2.101:0/1524044883 shutdown_connections 2021-02-10T11:36:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.109 7f61e0472700 1 --2- 172.21.2.101:0/1524044883 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f61d80d1c70 0x7f61d80efc60 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.109 7f61e0472700 1 --2- 172.21.2.101:0/1524044883 >> v2:172.21.2.108:3300/0 conn(0x7f61d80d0340 0x7f61d80ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.109 7f61e0472700 1 -- 172.21.2.101:0/1524044883 >> 172.21.2.101:0/1524044883 conn(0x7f61d80ce450 msgr2=0x7f61d80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.109 7f61e0472700 1 -- 172.21.2.101:0/1524044883 shutdown_connections 2021-02-10T11:36:59.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.109 7f61e0472700 1 -- 172.21.2.101:0/1524044883 wait complete. 2021-02-10T11:36:59.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.124 7f61e0472700 1 Processor -- start 2021-02-10T11:36:59.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.124 7f61e0472700 1 -- start start 2021-02-10T11:36:59.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.124 7f61e0472700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f61d80ada90 0x7f61d80ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:59.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.124 7f61e0472700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f61d8099d20 0x7f61d809a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:59.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.124 7f61e0472700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f61d809a740 0x7f61d8097bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:59.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.124 7f61e0472700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d80997a0 con 0x7f61d80ada90 2021-02-10T11:36:59.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.124 7f61e0472700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d818f0c0 con 0x7f61d809a740 2021-02-10T11:36:59.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.124 7f61e0472700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d809ab40 con 0x7f61d8099d20 2021-02-10T11:36:59.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.124 7f61de28e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f61d80ada90 0x7f61d80ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:59.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.124 7f61de28e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f61d80ada90 0x7f61d80ade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54420/0 (socket says 172.21.2.101:54420) 2021-02-10T11:36:59.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.124 7f61de28e700 1 -- 172.21.2.101:0/2321172369 learned_addr learned my addr 172.21.2.101:0/2321172369 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.125 7f61de28e700 1 -- 172.21.2.101:0/2321172369 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f61d8099d20 msgr2=0x7f61d809a120 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.125 7f61de28e700 1 --2- 172.21.2.101:0/2321172369 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f61d8099d20 0x7f61d809a120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.125 7f61de28e700 1 -- 172.21.2.101:0/2321172369 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f61d809a740 msgr2=0x7f61d8097bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:36:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.125 7f61de28e700 1 --2- 172.21.2.101:0/2321172369 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f61d809a740 0x7f61d8097bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.125 7f61de28e700 1 -- 172.21.2.101:0/2321172369 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d809ad60 con 0x7f61d80ada90 2021-02-10T11:36:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.125 7f61de28e700 1 --2- 172.21.2.101:0/2321172369 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f61d80ada90 0x7f61d80ade90 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 rx=0x7f61cc006f20 tx=0x7f61cc00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.125 7f61caffd700 1 -- 172.21.2.101:0/2321172369 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f61cc0009b0 con 0x7f61d80ada90 2021-02-10T11:36:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.125 7f61caffd700 1 -- 172.21.2.101:0/2321172369 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61cc00bd50 con 0x7f61d80ada90 2021-02-10T11:36:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.125 7f61caffd700 1 -- 172.21.2.101:0/2321172369 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f61cc005530 con 0x7f61d80ada90 2021-02-10T11:36:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.125 7f61e0472700 1 -- 172.21.2.101:0/2321172369 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d809ad60 con 0x7f61d80ada90 2021-02-10T11:36:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.125 7f61e0472700 1 -- 172.21.2.101:0/2321172369 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d80984d0 con 0x7f61d80ada90 2021-02-10T11:36:59.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.126 7f61caffd700 1 -- 172.21.2.101:0/2321172369 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f61cc00a000 con 0x7f61d80ada90 2021-02-10T11:36:59.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.126 7f61caffd700 1 --2- 172.21.2.101:0/2321172369 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61c00186c0 0x7f61c001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:59.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.126 7f61caffd700 1 -- 172.21.2.101:0/2321172369 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f61cc032510 con 0x7f61d80ada90 2021-02-10T11:36:59.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.126 7f61dda8d700 1 --2- 172.21.2.101:0/2321172369 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61c00186c0 0x7f61c001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:59.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.126 7f61dda8d700 1 --2- 172.21.2.101:0/2321172369 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61c00186c0 0x7f61c001abc0 crc :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:59.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.158 7f61e0472700 1 -- 172.21.2.101:0/2321172369 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d80ebc00 con 0x7f61d80ada90 2021-02-10T11:36:59.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.160 7f61caffd700 1 -- 172.21.2.101:0/2321172369 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f61cc020280 con 0x7f61d80ada90 2021-02-10T11:36:59.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.289 7f61e0472700 1 -- 172.21.2.101:0/2321172369 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61d80ebc00 con 0x7f61d80ada90 2021-02-10T11:36:59.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.290 7f61caffd700 1 -- 172.21.2.101:0/2321172369 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f61cc020280 con 0x7f61d80ada90 2021-02-10T11:36:59.296 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:36:59.296 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9360,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2529021396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2529021396},{"type":"v1","addr":"172.21.2.101:6835","nonce":2529021396}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:56.648753","modified":"2021-02-10 11:36:58.653009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6005":{"gid":6005,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2363068042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2363068042},{"type":"v1","addr":"172.21.2.108:6833","nonce":2363068042}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-02-10T11:36:59.297 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 62 2021-02-10T11:36:59.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.292 7f61e0472700 1 -- 172.21.2.101:0/2321172369 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61c00186c0 msgr2=0x7f61c001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:59.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.292 7f61e0472700 1 --2- 172.21.2.101:0/2321172369 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61c00186c0 0x7f61c001abc0 crc :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:59.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.292 7f61e0472700 1 -- 172.21.2.101:0/2321172369 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f61d80ada90 msgr2=0x7f61d80ade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:59.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.292 7f61e0472700 1 --2- 172.21.2.101:0/2321172369 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f61d80ada90 0x7f61d80ade90 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 rx=0x7f61cc006f20 tx=0x7f61cc00b960).stop 2021-02-10T11:36:59.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.292 7f61e0472700 1 -- 172.21.2.101:0/2321172369 shutdown_connections 2021-02-10T11:36:59.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.292 7f61e0472700 1 --2- 172.21.2.101:0/2321172369 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f61c00186c0 0x7f61c001abc0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:59.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.292 7f61e0472700 1 --2- 172.21.2.101:0/2321172369 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f61d8099d20 0x7f61d809a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:59.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.292 7f61e0472700 1 --2- 172.21.2.101:0/2321172369 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f61d809a740 0x7f61d8097bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:36:59.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.292 7f61e0472700 1 --2- 172.21.2.101:0/2321172369 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f61d80ada90 0x7f61d80ade90 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:59.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.292 7f61e0472700 1 -- 172.21.2.101:0/2321172369 >> 172.21.2.101:0/2321172369 conn(0x7f61d80f0550 msgr2=0x7f61d8191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:59.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.292 7f61e0472700 1 -- 172.21.2.101:0/2321172369 shutdown_connections 2021-02-10T11:36:59.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.292 7f61e0472700 1 -- 172.21.2.101:0/2321172369 wait complete. 2021-02-10T11:36:59.306 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:36:59.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.711 7f5df17d9700 1 Processor -- start 2021-02-10T11:36:59.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.711 7f5df17d9700 1 -- start start 2021-02-10T11:36:59.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.712 7f5df17d9700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f5dec0d1870 0x7f5dec1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.712 7f5df17d9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5dec1b0a10 0x7f5dec0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.712 7f5df17d9700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dec0f0b10 con 0x7f5dec0d1c70 2021-02-10T11:36:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.712 7f5df17d9700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dec0e5ed0 con 0x7f5dec0d1870 2021-02-10T11:36:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.712 7f5df17d9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec0f0650 con 0x7f5dec1b0a10 2021-02-10T11:36:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.712 7f5deb574700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f5dec0d1870 0x7f5dec1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.712 7f5deb574700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f5dec0d1870 0x7f5dec1b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:36682/0 (socket says 172.21.2.101:36682) 2021-02-10T11:36:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.712 7f5deb574700 1 -- 172.21.2.101:0/4257174051 learned_addr learned my addr 172.21.2.101:0/4257174051 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.712 7f5dead73700 1 --2- 172.21.2.101:0/4257174051 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5dec1b0a10 0x7f5dec0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.712 7f5deb574700 1 -- 172.21.2.101:0/4257174051 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5dec1b0a10 msgr2=0x7f5dec0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:36:59.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.712 7f5deb574700 1 --2- 172.21.2.101:0/4257174051 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5dec1b0a10 0x7f5dec0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.712 7f5deb574700 1 -- 172.21.2.101:0/4257174051 >> v1:172.21.2.108:6789/0 conn(0x7f5dec0d1c70 legacy=0x7f5dec0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.712 7f5deb574700 1 -- 172.21.2.101:0/4257174051 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dec0f0870 con 0x7f5dec0d1870 2021-02-10T11:36:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.712 7f5deb574700 1 --2- 172.21.2.101:0/4257174051 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5dec0d1870 0x7f5dec1b03f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f5dd4006700 tx=0x7f5dd4009b80).ready entity=mon.1 client_cookie=2a70853558b190cc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.713 7f5dea572700 1 -- 172.21.2.101:0/4257174051 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5dd400c0c0 con 0x7f5dec0d1870 2021-02-10T11:36:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.713 7f5dea572700 1 -- 172.21.2.101:0/4257174051 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd400c440 con 0x7f5dec0d1870 2021-02-10T11:36:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.713 7f5dea572700 1 -- 172.21.2.101:0/4257174051 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5dd400c960 con 0x7f5dec0d1870 2021-02-10T11:36:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.713 7f5df17d9700 1 -- 172.21.2.101:0/4257174051 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5dec0d1870 msgr2=0x7f5dec1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.713 7f5df17d9700 1 --2- 172.21.2.101:0/4257174051 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5dec0d1870 0x7f5dec1b03f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f5dd4006700 tx=0x7f5dd4009b80).stop 2021-02-10T11:36:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.713 7f5df17d9700 1 -- 172.21.2.101:0/4257174051 shutdown_connections 2021-02-10T11:36:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.713 7f5df17d9700 1 --2- 172.21.2.101:0/4257174051 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5dec1b0a10 0x7f5dec0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.713 7f5df17d9700 1 --2- 172.21.2.101:0/4257174051 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5dec0d1870 0x7f5dec1b03f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.713 7f5df17d9700 1 -- 172.21.2.101:0/4257174051 >> 172.21.2.101:0/4257174051 conn(0x7f5dec0ce450 msgr2=0x7f5dec0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:59.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.713 7f5df17d9700 1 -- 172.21.2.101:0/4257174051 shutdown_connections 2021-02-10T11:36:59.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.713 7f5df17d9700 1 -- 172.21.2.101:0/4257174051 wait complete. 2021-02-10T11:36:59.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.728 7f5df17d9700 1 Processor -- start 2021-02-10T11:36:59.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.728 7f5df17d9700 1 -- start start 2021-02-10T11:36:59.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.728 7f5df17d9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5dec18d870 0x7f5dec0b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:59.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.728 7f5df17d9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5dec0b1fd0 0x7f5dec099ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:59.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.728 7f5df17d9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5dec0b23d0 0x7f5dec09a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:59.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.728 7f5df17d9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec18dc70 con 0x7f5dec18d870 2021-02-10T11:36:59.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.728 7f5df17d9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec199e60 con 0x7f5dec0b23d0 2021-02-10T11:36:59.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.728 7f5df17d9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec09ab30 con 0x7f5dec0b1fd0 2021-02-10T11:36:59.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.729 7f5deb574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5dec18d870 0x7f5dec0b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:59.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.729 7f5deb574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5dec18d870 0x7f5dec0b19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54432/0 (socket says 172.21.2.101:54432) 2021-02-10T11:36:59.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.729 7f5deb574700 1 -- 172.21.2.101:0/813737286 learned_addr learned my addr 172.21.2.101:0/813737286 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:36:59.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.729 7f5dead73700 1 --2- 172.21.2.101:0/813737286 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5dec0b1fd0 0x7f5dec099ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:59.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.729 7f5debd75700 1 --2- 172.21.2.101:0/813737286 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5dec0b23d0 0x7f5dec09a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:59.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.729 7f5deb574700 1 -- 172.21.2.101:0/813737286 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5dec0b1fd0 msgr2=0x7f5dec099ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:59.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.729 7f5deb574700 1 --2- 172.21.2.101:0/813737286 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5dec0b1fd0 0x7f5dec099ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:59.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.729 7f5deb574700 1 -- 172.21.2.101:0/813737286 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5dec0b23d0 msgr2=0x7f5dec09a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:59.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.729 7f5deb574700 1 --2- 172.21.2.101:0/813737286 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5dec0b23d0 0x7f5dec09a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:59.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.729 7f5deb574700 1 -- 172.21.2.101:0/813737286 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dec09ad50 con 0x7f5dec18d870 2021-02-10T11:36:59.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.729 7f5deb574700 1 --2- 172.21.2.101:0/813737286 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5dec18d870 0x7f5dec0b19b0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 rx=0x7f5dd400b5e0 tx=0x7f5dd400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:59.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.729 7f5de3fff700 1 -- 172.21.2.101:0/813737286 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5dd400bd70 con 0x7f5dec18d870 2021-02-10T11:36:59.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.729 7f5de3fff700 1 -- 172.21.2.101:0/813737286 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd4002030 con 0x7f5dec18d870 2021-02-10T11:36:59.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.729 7f5de3fff700 1 -- 172.21.2.101:0/813737286 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5dd4002550 con 0x7f5dec18d870 2021-02-10T11:36:59.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.729 7f5df17d9700 1 -- 172.21.2.101:0/813737286 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dec09ad50 con 0x7f5dec18d870 2021-02-10T11:36:59.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.729 7f5df17d9700 1 -- 172.21.2.101:0/813737286 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dec18e3f0 con 0x7f5dec18d870 2021-02-10T11:36:59.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.730 7f5de3fff700 1 -- 172.21.2.101:0/813737286 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5dd4002210 con 0x7f5dec18d870 2021-02-10T11:36:59.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.730 7f5de3fff700 1 --2- 172.21.2.101:0/813737286 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5dd00186c0 0x7f5dd001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:36:59.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.730 7f5de3fff700 1 -- 172.21.2.101:0/813737286 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5dd4032540 con 0x7f5dec18d870 2021-02-10T11:36:59.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.730 7f5dead73700 1 --2- 172.21.2.101:0/813737286 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5dd00186c0 0x7f5dd001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:36:59.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.730 7f5dead73700 1 --2- 172.21.2.101:0/813737286 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5dd00186c0 0x7f5dd001abc0 crc :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:36:59.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.762 7f5df17d9700 1 -- 172.21.2.101:0/813737286 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dec0999c0 con 0x7f5dec18d870 2021-02-10T11:36:59.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.765 7f5de3fff700 1 -- 172.21.2.101:0/813737286 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5dd40009b0 con 0x7f5dec18d870 2021-02-10T11:36:59.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.889 7f5df17d9700 1 -- 172.21.2.101:0/813737286 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5dec0999c0 con 0x7f5dec18d870 2021-02-10T11:36:59.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.889 7f5de3fff700 1 -- 172.21.2.101:0/813737286 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+13996 (secure 0 0 0) 0x7f5dd4002210 con 0x7f5dec18d870 2021-02-10T11:36:59.896 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:36:59.896 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":85,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:36:58.653077","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:36:54.618463","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2021-02-10 11:36:55.644563","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:36:59.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.891 7f5df17d9700 1 -- 172.21.2.101:0/813737286 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5dd00186c0 msgr2=0x7f5dd001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:59.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.891 7f5df17d9700 1 --2- 172.21.2.101:0/813737286 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5dd00186c0 0x7f5dd001abc0 crc :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:59.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.891 7f5df17d9700 1 -- 172.21.2.101:0/813737286 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5dec18d870 msgr2=0x7f5dec0b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:36:59.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.891 7f5df17d9700 1 --2- 172.21.2.101:0/813737286 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5dec18d870 0x7f5dec0b19b0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 rx=0x7f5dd400b5e0 tx=0x7f5dd400b980).stop 2021-02-10T11:36:59.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.891 7f5df17d9700 1 -- 172.21.2.101:0/813737286 shutdown_connections 2021-02-10T11:36:59.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.891 7f5df17d9700 1 --2- 172.21.2.101:0/813737286 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5dd00186c0 0x7f5dd001abc0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:59.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.891 7f5df17d9700 1 --2- 172.21.2.101:0/813737286 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5dec0b1fd0 0x7f5dec099ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:59.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.891 7f5df17d9700 1 --2- 172.21.2.101:0/813737286 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5dec0b23d0 0x7f5dec09a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:59.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.891 7f5df17d9700 1 --2- 172.21.2.101:0/813737286 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5dec18d870 0x7f5dec0b19b0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:36:59.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.891 7f5df17d9700 1 -- 172.21.2.101:0/813737286 >> 172.21.2.101:0/813737286 conn(0x7f5dec0f0550 msgr2=0x7f5dec191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:36:59.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.892 7f5df17d9700 1 -- 172.21.2.101:0/813737286 shutdown_connections 2021-02-10T11:36:59.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:36:59.892 7f5df17d9700 1 -- 172.21.2.101:0/813737286 wait complete. 2021-02-10T11:36:59.906 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:37:00.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.322 7f2fa593b700 1 Processor -- start 2021-02-10T11:37:00.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.322 7f2fa593b700 1 -- start start 2021-02-10T11:37:00.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.323 7f2fa593b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2fa00d1c80 0x7f2fa00cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:00.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.323 7f2fa593b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa00d0180 con 0x7f2fa00cf590 2021-02-10T11:37:00.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.323 7f2fa593b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa00fe1d0 con 0x7f2fa00d1880 2021-02-10T11:37:00.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.323 7f2fa593b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa01b0cc0 con 0x7f2fa00d1c80 2021-02-10T11:37:00.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.323 7f2f9ed73700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2fa00d1c80 0x7f2fa00cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:00.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.323 7f2f9ed73700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2fa00d1c80 0x7f2fa00cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54440/0 (socket says 172.21.2.101:54440) 2021-02-10T11:37:00.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.323 7f2f9ed73700 1 -- 172.21.2.101:0/1443472671 learned_addr learned my addr 172.21.2.101:0/1443472671 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:00.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.324 7f2f9ed73700 1 -- 172.21.2.101:0/1443472671 >> v1:172.21.2.108:6789/0 conn(0x7f2fa00d1880 legacy=0x7f2fa01b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:00.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.324 7f2f9ed73700 1 -- 172.21.2.101:0/1443472671 >> v1:172.21.2.101:6789/0 conn(0x7f2fa00cf590 legacy=0x7f2fa00cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:00.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.324 7f2f9ed73700 1 -- 172.21.2.101:0/1443472671 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa0109a70 con 0x7f2fa00d1c80 2021-02-10T11:37:00.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.324 7f2f9ed73700 1 --2- 172.21.2.101:0/1443472671 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fa00d1c80 0x7f2fa00cef70 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 rx=0x7f2f88006700 tx=0x7f2f88009b00).ready entity=mon.0 client_cookie=754f066659200db4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:00.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.324 7f2f9e572700 1 -- 172.21.2.101:0/1443472671 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2f8800c0c0 con 0x7f2fa00d1c80 2021-02-10T11:37:00.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.324 7f2f9e572700 1 -- 172.21.2.101:0/1443472671 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8800c400 con 0x7f2fa00d1c80 2021-02-10T11:37:00.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.324 7f2f9e572700 1 -- 172.21.2.101:0/1443472671 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2f8800c8e0 con 0x7f2fa00d1c80 2021-02-10T11:37:00.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.324 7f2fa593b700 1 -- 172.21.2.101:0/1443472671 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fa00d1c80 msgr2=0x7f2fa00cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:00.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.324 7f2fa593b700 1 --2- 172.21.2.101:0/1443472671 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fa00d1c80 0x7f2fa00cef70 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 rx=0x7f2f88006700 tx=0x7f2f88009b00).stop 2021-02-10T11:37:00.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.325 7f2fa593b700 1 -- 172.21.2.101:0/1443472671 shutdown_connections 2021-02-10T11:37:00.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.325 7f2fa593b700 1 --2- 172.21.2.101:0/1443472671 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fa00d1c80 0x7f2fa00cef70 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:00.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.325 7f2fa593b700 1 -- 172.21.2.101:0/1443472671 >> 172.21.2.101:0/1443472671 conn(0x7f2fa00ce430 msgr2=0x7f2fa00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:00.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.325 7f2fa593b700 1 -- 172.21.2.101:0/1443472671 shutdown_connections 2021-02-10T11:37:00.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.325 7f2fa593b700 1 -- 172.21.2.101:0/1443472671 wait complete. 2021-02-10T11:37:00.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.339 7f2fa593b700 1 Processor -- start 2021-02-10T11:37:00.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.339 7f2fa593b700 1 -- start start 2021-02-10T11:37:00.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.340 7f2fa593b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2fa0108070 0x7f2fa00ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:00.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.340 7f2fa593b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2fa0108470 0x7f2fa00c1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:00.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.340 7f2fa593b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fa00c1e20 0x7f2fa00c2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:00.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.340 7f2fa593b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa00c1300 con 0x7f2fa00c1e20 2021-02-10T11:37:00.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.340 7f2fa593b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa00b84b0 con 0x7f2fa0108070 2021-02-10T11:37:00.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.340 7f2fa593b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa00c0f70 con 0x7f2fa0108470 2021-02-10T11:37:00.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.340 7f2f9fd75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fa00c1e20 0x7f2fa00c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:00.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.340 7f2f9fd75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fa00c1e20 0x7f2fa00c2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54450/0 (socket says 172.21.2.101:54450) 2021-02-10T11:37:00.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.340 7f2f9fd75700 1 -- 172.21.2.101:0/25837691 learned_addr learned my addr 172.21.2.101:0/25837691 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:00.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.340 7f2f9fd75700 1 -- 172.21.2.101:0/25837691 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2fa0108470 msgr2=0x7f2fa00c1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:00.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.340 7f2f9fd75700 1 --2- 172.21.2.101:0/25837691 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2fa0108470 0x7f2fa00c1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:00.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.340 7f2f9fd75700 1 -- 172.21.2.101:0/25837691 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2fa0108070 msgr2=0x7f2fa00ffcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:00.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.340 7f2f9fd75700 1 --2- 172.21.2.101:0/25837691 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2fa0108070 0x7f2fa00ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:00.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.340 7f2f9fd75700 1 -- 172.21.2.101:0/25837691 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa01002f0 con 0x7f2fa00c1e20 2021-02-10T11:37:00.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.340 7f2f9fd75700 1 --2- 172.21.2.101:0/25837691 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fa00c1e20 0x7f2fa00c2220 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 rx=0x7f2f98009980 tx=0x7f2f9800bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:00.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.341 7f2f97fff700 1 -- 172.21.2.101:0/25837691 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2f98005870 con 0x7f2fa00c1e20 2021-02-10T11:37:00.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.341 7f2f97fff700 1 -- 172.21.2.101:0/25837691 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f98005a70 con 0x7f2fa00c1e20 2021-02-10T11:37:00.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.341 7f2f97fff700 1 -- 172.21.2.101:0/25837691 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2f98006030 con 0x7f2fa00c1e20 2021-02-10T11:37:00.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.341 7f2fa593b700 1 -- 172.21.2.101:0/25837691 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa01002f0 con 0x7f2fa00c1e20 2021-02-10T11:37:00.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.341 7f2fa593b700 1 -- 172.21.2.101:0/25837691 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa00c2b30 con 0x7f2fa00c1e20 2021-02-10T11:37:00.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.342 7f2f97fff700 1 -- 172.21.2.101:0/25837691 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2f98006030 con 0x7f2fa00c1e20 2021-02-10T11:37:00.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.342 7f2f97fff700 1 --2- 172.21.2.101:0/25837691 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f840186c0 0x7f2f8401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:00.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.342 7f2f97fff700 1 -- 172.21.2.101:0/25837691 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2f98032950 con 0x7f2fa00c1e20 2021-02-10T11:37:00.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.342 7f2f9f574700 1 --2- 172.21.2.101:0/25837691 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f840186c0 0x7f2f8401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:00.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.342 7f2f9f574700 1 --2- 172.21.2.101:0/25837691 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f840186c0 0x7f2f8401abc0 crc :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:00.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.373 7f2fa593b700 1 -- 172.21.2.101:0/25837691 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa0088070 con 0x7f2fa00c1e20 2021-02-10T11:37:00.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.376 7f2f97fff700 1 -- 172.21.2.101:0/25837691 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2f98005d10 con 0x7f2fa00c1e20 2021-02-10T11:37:00.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.504 7f2fa593b700 1 -- 172.21.2.101:0/25837691 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f2fa00b8470 con 0x7f2fa00c1e20 2021-02-10T11:37:00.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.507 7f2f97fff700 1 -- 172.21.2.101:0/25837691 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v49) v1 ==== 184+0+0 (secure 0 0 0) 0x7f2f98006030 con 0x7f2fa00c1e20 2021-02-10T11:37:00.517 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.0 2021-02-10T11:37:00.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.512 7f2fa593b700 1 -- 172.21.2.101:0/25837691 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f840186c0 msgr2=0x7f2f8401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:00.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.512 7f2fa593b700 1 --2- 172.21.2.101:0/25837691 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f840186c0 0x7f2f8401abc0 crc :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:00.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.512 7f2fa593b700 1 -- 172.21.2.101:0/25837691 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fa00c1e20 msgr2=0x7f2fa00c2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:00.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.512 7f2fa593b700 1 --2- 172.21.2.101:0/25837691 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fa00c1e20 0x7f2fa00c2220 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 rx=0x7f2f98009980 tx=0x7f2f9800bcd0).stop 2021-02-10T11:37:00.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.512 7f2fa593b700 1 -- 172.21.2.101:0/25837691 shutdown_connections 2021-02-10T11:37:00.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.512 7f2fa593b700 1 --2- 172.21.2.101:0/25837691 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f840186c0 0x7f2f8401abc0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:00.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.512 7f2fa593b700 1 --2- 172.21.2.101:0/25837691 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2fa0108470 0x7f2fa00c1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:00.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.512 7f2fa593b700 1 --2- 172.21.2.101:0/25837691 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2fa00c1e20 0x7f2fa00c2220 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:00.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.513 7f2fa593b700 1 --2- 172.21.2.101:0/25837691 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2fa0108070 0x7f2fa00ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:00.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.513 7f2fa593b700 1 -- 172.21.2.101:0/25837691 >> 172.21.2.101:0/25837691 conn(0x7f2fa0100b70 msgr2=0x7f2fa00c3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:00.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.513 7f2fa593b700 1 -- 172.21.2.101:0/25837691 shutdown_connections 2021-02-10T11:37:00.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.513 7f2fa593b700 1 -- 172.21.2.101:0/25837691 wait complete. 2021-02-10T11:37:00.527 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:37:00.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.945 7fcd5e476700 1 Processor -- start 2021-02-10T11:37:00.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.945 7fcd5e476700 1 -- start start 2021-02-10T11:37:00.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.945 7fcd5e476700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fcd5810afe0 0x7fcd580cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:00.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.945 7fcd5e476700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd580fe1b0 con 0x7fcd580d1c60 2021-02-10T11:37:00.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.945 7fcd5e476700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd581b0ca0 con 0x7fcd580d1860 2021-02-10T11:37:00.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.945 7fcd5e476700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd58109a50 con 0x7fcd5810afe0 2021-02-10T11:37:00.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.945 7fcd577fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fcd580d1c60 0x7fcd580cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54948/0 (socket says 172.21.2.101:54948) 2021-02-10T11:37:00.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.945 7fcd577fe700 1 -- 172.21.2.101:0/1510852781 learned_addr learned my addr 172.21.2.101:0/1510852781 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:37:00.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.946 7fcd56ffd700 1 -- 172.21.2.101:0/1510852781 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4143924148 0 0) 0x7fcd4c000ce0 con 0x7fcd580d1c60 2021-02-10T11:37:00.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.946 7fcd56ffd700 1 -- 172.21.2.101:0/1510852781 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd40002fe0 con 0x7fcd580d1c60 2021-02-10T11:37:00.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.946 7fcd5ca93700 1 --2- 172.21.2.101:0/1510852781 >> v2:172.21.2.108:3300/0 conn(0x7fcd5810afe0 0x7fcd580cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:00.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.946 7fcd56ffd700 1 -- 172.21.2.101:0/1510852781 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3241690980 0 0) 0x7fcd4c0010e0 con 0x7fcd580d1c60 2021-02-10T11:37:00.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.946 7fcd56ffd700 1 -- 172.21.2.101:0/1510852781 >> v2:172.21.2.108:3300/0 conn(0x7fcd5810afe0 msgr2=0x7fcd580cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:00.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.946 7fcd56ffd700 1 --2- 172.21.2.101:0/1510852781 >> v2:172.21.2.108:3300/0 conn(0x7fcd5810afe0 0x7fcd580cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:00.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.946 7fcd56ffd700 1 -- 172.21.2.101:0/1510852781 >> v1:172.21.2.108:6789/0 conn(0x7fcd580d1860 legacy=0x7fcd581b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:00.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.946 7fcd56ffd700 1 -- 172.21.2.101:0/1510852781 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5810b3e0 con 0x7fcd580d1c60 2021-02-10T11:37:00.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.946 7fcd56ffd700 1 -- 172.21.2.101:0/1510852781 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fcd4c001660 con 0x7fcd580d1c60 2021-02-10T11:37:00.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.946 7fcd56ffd700 1 -- 172.21.2.101:0/1510852781 >> v1:172.21.2.101:6789/0 conn(0x7fcd580d1c60 legacy=0x7fcd580cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:00.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.946 7fcd56ffd700 1 --2- 172.21.2.101:0/1510852781 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd40003a60 0x7fcd40005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:00.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.946 7fcd56ffd700 1 --2- 172.21.2.101:0/1510852781 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd40006770 0x7fcd40008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:00.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.946 7fcd56ffd700 1 --2- 172.21.2.101:0/1510852781 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd40009590 0x7fcd4000bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:00.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.946 7fcd56ffd700 1 -- 172.21.2.101:0/1510852781 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4000c300 con 0x7fcd40009590 2021-02-10T11:37:00.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.946 7fcd56ffd700 1 -- 172.21.2.101:0/1510852781 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4000c520 con 0x7fcd40003a60 2021-02-10T11:37:00.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.946 7fcd56ffd700 1 -- 172.21.2.101:0/1510852781 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4000c740 con 0x7fcd40006770 2021-02-10T11:37:00.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.946 7fcd5ca93700 1 --2- 172.21.2.101:0/1510852781 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd40009590 0x7fcd4000bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:00.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.946 7fcd57fff700 1 --2- 172.21.2.101:0/1510852781 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd40003a60 0x7fcd40005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:00.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.947 7fcd577fe700 1 --2- 172.21.2.101:0/1510852781 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd40006770 0x7fcd40008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:00.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.947 7fcd5ca93700 1 -- 172.21.2.101:0/1510852781 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd40006770 msgr2=0x7fcd40008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:00.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.947 7fcd5ca93700 1 --2- 172.21.2.101:0/1510852781 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd40006770 0x7fcd40008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:00.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.947 7fcd5ca93700 1 -- 172.21.2.101:0/1510852781 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd40003a60 msgr2=0x7fcd40005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:00.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.947 7fcd5ca93700 1 --2- 172.21.2.101:0/1510852781 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd40003a60 0x7fcd40005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:00.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.947 7fcd5ca93700 1 -- 172.21.2.101:0/1510852781 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4000c9e0 con 0x7fcd40009590 2021-02-10T11:37:00.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.947 7fcd577fe700 1 --2- 172.21.2.101:0/1510852781 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd40006770 0x7fcd40008c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:37:00.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.947 7fcd5ca93700 1 -- 172.21.2.101:0/1510852781 reap_dead start 2021-02-10T11:37:00.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.947 7fcd5ca93700 1 --2- 172.21.2.101:0/1510852781 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd40009590 0x7fcd4000bab0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 rx=0x7fcd500068b0 tx=0x7fcd5004f050).ready entity=mon.0 client_cookie=a25fe965842b1b96 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:00.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.947 7fcd56ffd700 1 -- 172.21.2.101:0/1510852781 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd5004f440 con 0x7fcd40009590 2021-02-10T11:37:00.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.947 7fcd56ffd700 1 -- 172.21.2.101:0/1510852781 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd50006d10 con 0x7fcd40009590 2021-02-10T11:37:00.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.947 7fcd56ffd700 1 -- 172.21.2.101:0/1510852781 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd5004fa00 con 0x7fcd40009590 2021-02-10T11:37:00.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.947 7fcd5e476700 1 -- 172.21.2.101:0/1510852781 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd40009590 msgr2=0x7fcd4000bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:00.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.947 7fcd5e476700 1 --2- 172.21.2.101:0/1510852781 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd40009590 0x7fcd4000bab0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 rx=0x7fcd500068b0 tx=0x7fcd5004f050).stop 2021-02-10T11:37:00.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.948 7fcd5e476700 1 -- 172.21.2.101:0/1510852781 shutdown_connections 2021-02-10T11:37:00.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.948 7fcd5e476700 1 --2- 172.21.2.101:0/1510852781 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd40009590 0x7fcd4000bab0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:00.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.948 7fcd5e476700 1 -- 172.21.2.101:0/1510852781 >> 172.21.2.101:0/1510852781 conn(0x7fcd580ce410 msgr2=0x7fcd580d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:00.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.948 7fcd5e476700 1 -- 172.21.2.101:0/1510852781 shutdown_connections 2021-02-10T11:37:00.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.948 7fcd5e476700 1 -- 172.21.2.101:0/1510852781 wait complete. 2021-02-10T11:37:00.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.962 7fcd5e476700 1 Processor -- start 2021-02-10T11:37:00.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.963 7fcd5e476700 1 -- start start 2021-02-10T11:37:00.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.963 7fcd5e476700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd580b5fb0 0x7fcd580a81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:00.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.963 7fcd5e476700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd580a8810 0x7fcd580a2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:00.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.963 7fcd5e476700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd580a8c10 0x7fcd580a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:00.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.963 7fcd5e476700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd580b63b0 con 0x7fcd580a8c10 2021-02-10T11:37:00.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.963 7fcd5e476700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd580a9210 con 0x7fcd580b5fb0 2021-02-10T11:37:00.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.963 7fcd5e476700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd580b2650 con 0x7fcd580a8810 2021-02-10T11:37:00.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.963 7fcd5ca93700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd580a8c10 0x7fcd580a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:00.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.963 7fcd5ca93700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd580a8c10 0x7fcd580a2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54466/0 (socket says 172.21.2.101:54466) 2021-02-10T11:37:00.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.963 7fcd5ca93700 1 -- 172.21.2.101:0/2335452305 learned_addr learned my addr 172.21.2.101:0/2335452305 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:00.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.964 7fcd5ca93700 1 -- 172.21.2.101:0/2335452305 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd580a8810 msgr2=0x7fcd580a2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:00.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.964 7fcd5ca93700 1 --2- 172.21.2.101:0/2335452305 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd580a8810 0x7fcd580a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:00.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.964 7fcd5ca93700 1 -- 172.21.2.101:0/2335452305 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd580b5fb0 msgr2=0x7fcd580a81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:00.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.964 7fcd5ca93700 1 --2- 172.21.2.101:0/2335452305 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd580b5fb0 0x7fcd580a81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:00.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.964 7fcd5ca93700 1 -- 172.21.2.101:0/2335452305 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd580a3180 con 0x7fcd580a8c10 2021-02-10T11:37:00.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.964 7fcd5ca93700 1 --2- 172.21.2.101:0/2335452305 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd580a8c10 0x7fcd580a2b60 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 rx=0x7fcd50006970 tx=0x7fcd5004fd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:00.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.964 7fcd54ff9700 1 -- 172.21.2.101:0/2335452305 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd5004d8b0 con 0x7fcd580a8c10 2021-02-10T11:37:00.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.964 7fcd54ff9700 1 -- 172.21.2.101:0/2335452305 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd500502f0 con 0x7fcd580a8c10 2021-02-10T11:37:00.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.964 7fcd54ff9700 1 -- 172.21.2.101:0/2335452305 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcd50050810 con 0x7fcd580a8c10 2021-02-10T11:37:00.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.964 7fcd5e476700 1 -- 172.21.2.101:0/2335452305 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd580a3180 con 0x7fcd580a8c10 2021-02-10T11:37:00.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.964 7fcd5e476700 1 -- 172.21.2.101:0/2335452305 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd580a3710 con 0x7fcd580a8c10 2021-02-10T11:37:00.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.965 7fcd54ff9700 1 -- 172.21.2.101:0/2335452305 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fcd5004d8b0 con 0x7fcd580a8c10 2021-02-10T11:37:00.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.965 7fcd54ff9700 1 --2- 172.21.2.101:0/2335452305 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcd380186c0 0x7fcd3801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:00.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.965 7fcd54ff9700 1 -- 172.21.2.101:0/2335452305 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fcd50076ba0 con 0x7fcd580a8c10 2021-02-10T11:37:00.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.965 7fcd57fff700 1 --2- 172.21.2.101:0/2335452305 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcd380186c0 0x7fcd3801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:00.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.965 7fcd57fff700 1 --2- 172.21.2.101:0/2335452305 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcd380186c0 0x7fcd3801abc0 crc :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:01.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.995 7fcd5e476700 1 -- 172.21.2.101:0/2335452305 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd580fe1b0 con 0x7fcd580a8c10 2021-02-10T11:37:01.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:00.998 7fcd54ff9700 1 -- 172.21.2.101:0/2335452305 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fcd500638e0 con 0x7fcd580a8c10 2021-02-10T11:37:01.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.130 7fcd5e476700 1 -- 172.21.2.101:0/2335452305 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fcd580b63b0 con 0x7fcd580a8c10 2021-02-10T11:37:01.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.134 7fcd54ff9700 1 -- 172.21.2.101:0/2335452305 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v50) v1 ==== 184+0+0 (secure 0 0 0) 0x7fcd5004d8b0 con 0x7fcd580a8c10 2021-02-10T11:37:01.143 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.1 2021-02-10T11:37:01.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.139 7fcd5e476700 1 -- 172.21.2.101:0/2335452305 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcd380186c0 msgr2=0x7fcd3801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:01.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.139 7fcd5e476700 1 --2- 172.21.2.101:0/2335452305 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcd380186c0 0x7fcd3801abc0 crc :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:01.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.139 7fcd5e476700 1 -- 172.21.2.101:0/2335452305 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd580a8c10 msgr2=0x7fcd580a2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:01.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.139 7fcd5e476700 1 --2- 172.21.2.101:0/2335452305 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd580a8c10 0x7fcd580a2b60 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 rx=0x7fcd50006970 tx=0x7fcd5004fd10).stop 2021-02-10T11:37:01.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.139 7fcd5e476700 1 -- 172.21.2.101:0/2335452305 shutdown_connections 2021-02-10T11:37:01.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.139 7fcd5e476700 1 --2- 172.21.2.101:0/2335452305 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcd380186c0 0x7fcd3801abc0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:01.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.139 7fcd5e476700 1 --2- 172.21.2.101:0/2335452305 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcd580a8810 0x7fcd580a2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:01.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.139 7fcd5e476700 1 --2- 172.21.2.101:0/2335452305 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcd580a8c10 0x7fcd580a2b60 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:01.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.139 7fcd5e476700 1 --2- 172.21.2.101:0/2335452305 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcd580b5fb0 0x7fcd580a81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:01.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.139 7fcd5e476700 1 -- 172.21.2.101:0/2335452305 >> 172.21.2.101:0/2335452305 conn(0x7fcd5809e660 msgr2=0x7fcd580a9550 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:01.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.139 7fcd5e476700 1 -- 172.21.2.101:0/2335452305 shutdown_connections 2021-02-10T11:37:01.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.139 7fcd5e476700 1 -- 172.21.2.101:0/2335452305 wait complete. 2021-02-10T11:37:01.153 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:37:01.237 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:37:01.240 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:37:01.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.570 7f17ee73f700 1 Processor -- start 2021-02-10T11:37:01.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.570 7f17ee73f700 1 -- start start 2021-02-10T11:37:01.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.570 7f17ee73f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f17e80d1880 0x7f17e81b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:01.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.570 7f17ee73f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17e81b0af0 0x7f17e8108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:01.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.570 7f17ee73f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e80fe1d0 con 0x7f17e80d1c80 2021-02-10T11:37:01.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.570 7f17ee73f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e8109a70 con 0x7f17e80d1880 2021-02-10T11:37:01.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.570 7f17ee73f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e8109760 con 0x7f17e81b0af0 2021-02-10T11:37:01.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.571 7f17e77fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17e81b0af0 0x7f17e8108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:01.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.571 7f17e77fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17e81b0af0 0x7f17e8108950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:47206/0 (socket says 172.21.2.101:47206) 2021-02-10T11:37:01.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.571 7f17e77fe700 1 -- 172.21.2.101:0/2140157284 learned_addr learned my addr 172.21.2.101:0/2140157284 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:01.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.571 7f17e7fff700 1 --2- 172.21.2.101:0/2140157284 >> v2:172.21.2.108:3300/0 conn(0x7f17e80d1880 0x7f17e81b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:01.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.571 7f17e77fe700 1 -- 172.21.2.101:0/2140157284 >> v2:172.21.2.108:3300/0 conn(0x7f17e80d1880 msgr2=0x7f17e81b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:01.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.571 7f17e77fe700 1 --2- 172.21.2.101:0/2140157284 >> v2:172.21.2.108:3300/0 conn(0x7f17e80d1880 0x7f17e81b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:01.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.571 7f17e77fe700 1 -- 172.21.2.101:0/2140157284 >> v1:172.21.2.108:6789/0 conn(0x7f17e80d1c80 legacy=0x7f17e8108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:01.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.571 7f17e77fe700 1 -- 172.21.2.101:0/2140157284 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e8088070 con 0x7f17e81b0af0 2021-02-10T11:37:01.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.571 7f17e77fe700 1 --2- 172.21.2.101:0/2140157284 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17e81b0af0 0x7f17e8108950 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f17dc006700 tx=0x7f17dc009b80).ready entity=mon.2 client_cookie=8f2efa6e6dc061b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:01.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.572 7f17e6ffd700 1 -- 172.21.2.101:0/2140157284 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f17dc000c90 con 0x7f17e81b0af0 2021-02-10T11:37:01.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.572 7f17e6ffd700 1 -- 172.21.2.101:0/2140157284 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17dc00c0c0 con 0x7f17e81b0af0 2021-02-10T11:37:01.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.572 7f17e6ffd700 1 -- 172.21.2.101:0/2140157284 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f17dc00c5e0 con 0x7f17e81b0af0 2021-02-10T11:37:01.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.572 7f17ee73f700 1 -- 172.21.2.101:0/2140157284 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17e81b0af0 msgr2=0x7f17e8108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:01.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.572 7f17ee73f700 1 --2- 172.21.2.101:0/2140157284 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17e81b0af0 0x7f17e8108950 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f17dc006700 tx=0x7f17dc009b80).stop 2021-02-10T11:37:01.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.572 7f17ee73f700 1 -- 172.21.2.101:0/2140157284 shutdown_connections 2021-02-10T11:37:01.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.572 7f17ee73f700 1 --2- 172.21.2.101:0/2140157284 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17e81b0af0 0x7f17e8108950 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:01.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.572 7f17ee73f700 1 --2- 172.21.2.101:0/2140157284 >> v2:172.21.2.108:3300/0 conn(0x7f17e80d1880 0x7f17e81b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:01.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.572 7f17ee73f700 1 -- 172.21.2.101:0/2140157284 >> 172.21.2.101:0/2140157284 conn(0x7f17e80ce430 msgr2=0x7f17e80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:01.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.572 7f17ee73f700 1 -- 172.21.2.101:0/2140157284 shutdown_connections 2021-02-10T11:37:01.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.572 7f17ee73f700 1 -- 172.21.2.101:0/2140157284 wait complete. 2021-02-10T11:37:01.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.587 7f17ee73f700 1 Processor -- start 2021-02-10T11:37:01.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.587 7f17ee73f700 1 -- start start 2021-02-10T11:37:01.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.588 7f17ee73f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f17e8095600 0x7f17e8095a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:01.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.588 7f17ee73f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f17e8096020 0x7f17e8096420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:01.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.588 7f17ee73f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17e8092a10 0x7f17e8092e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:01.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.588 7f17ee73f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e8094100 con 0x7f17e8096020 2021-02-10T11:37:01.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.588 7f17ee73f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e819e9e0 con 0x7f17e8095600 2021-02-10T11:37:01.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.588 7f17ee73f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e8093430 con 0x7f17e8092a10 2021-02-10T11:37:01.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.588 7f17e77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f17e8096020 0x7f17e8096420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:01.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.588 7f17e77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f17e8096020 0x7f17e8096420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54476/0 (socket says 172.21.2.101:54476) 2021-02-10T11:37:01.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.588 7f17e77fe700 1 -- 172.21.2.101:0/1746352365 learned_addr learned my addr 172.21.2.101:0/1746352365 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:01.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.588 7f17e7fff700 1 --2- 172.21.2.101:0/1746352365 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f17e8095600 0x7f17e8095a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:01.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.588 7f17ecd5c700 1 --2- 172.21.2.101:0/1746352365 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17e8092a10 0x7f17e8092e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:01.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.588 7f17e77fe700 1 -- 172.21.2.101:0/1746352365 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17e8092a10 msgr2=0x7f17e8092e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:01.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.588 7f17e77fe700 1 --2- 172.21.2.101:0/1746352365 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17e8092a10 0x7f17e8092e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:01.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.588 7f17e77fe700 1 -- 172.21.2.101:0/1746352365 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f17e8095600 msgr2=0x7f17e8095a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:01.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.588 7f17e77fe700 1 --2- 172.21.2.101:0/1746352365 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f17e8095600 0x7f17e8095a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:01.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.588 7f17e77fe700 1 -- 172.21.2.101:0/1746352365 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e8093650 con 0x7f17e8096020 2021-02-10T11:37:01.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.588 7f17e77fe700 1 --2- 172.21.2.101:0/1746352365 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f17e8096020 0x7f17e8096420 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 rx=0x7f17dc009fb0 tx=0x7f17dc00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:01.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.589 7f17e4ff9700 1 -- 172.21.2.101:0/1746352365 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f17dc006340 con 0x7f17e8096020 2021-02-10T11:37:01.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.589 7f17e4ff9700 1 -- 172.21.2.101:0/1746352365 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17dc002080 con 0x7f17e8096020 2021-02-10T11:37:01.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.589 7f17ee73f700 1 -- 172.21.2.101:0/1746352365 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e8093650 con 0x7f17e8096020 2021-02-10T11:37:01.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.589 7f17ee73f700 1 -- 172.21.2.101:0/1746352365 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e8093be0 con 0x7f17e8096020 2021-02-10T11:37:01.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.589 7f17e4ff9700 1 -- 172.21.2.101:0/1746352365 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f17dc0024e0 con 0x7f17e8096020 2021-02-10T11:37:01.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.589 7f17e4ff9700 1 -- 172.21.2.101:0/1746352365 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f17dc0029b0 con 0x7f17e8096020 2021-02-10T11:37:01.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.589 7f17e4ff9700 1 --2- 172.21.2.101:0/1746352365 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f17c8018730 0x7f17c801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:01.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.589 7f17e4ff9700 1 -- 172.21.2.101:0/1746352365 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f17dc032880 con 0x7f17e8096020 2021-02-10T11:37:01.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.590 7f17e7fff700 1 --2- 172.21.2.101:0/1746352365 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f17c8018730 0x7f17c801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:01.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.590 7f17e7fff700 1 --2- 172.21.2.101:0/1746352365 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f17c8018730 0x7f17c801abc0 crc :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:01.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.620 7f17ee73f700 1 -- 172.21.2.101:0/1746352365 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e8082980 con 0x7f17e8096020 2021-02-10T11:37:01.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.622 7f17e4ff9700 1 -- 172.21.2.101:0/1746352365 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f17dc0027a0 con 0x7f17e8096020 2021-02-10T11:37:01.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.753 7f17ee73f700 1 -- 172.21.2.101:0/1746352365 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f17e8094080 con 0x7f17e8096020 2021-02-10T11:37:01.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.756 7f17e4ff9700 1 -- 172.21.2.101:0/1746352365 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v51) v1 ==== 184+0+0 (secure 0 0 0) 0x7f17dc006940 con 0x7f17e8096020 2021-02-10T11:37:01.766 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.2 2021-02-10T11:37:01.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.762 7f17ee73f700 1 -- 172.21.2.101:0/1746352365 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f17c8018730 msgr2=0x7f17c801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:01.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.762 7f17ee73f700 1 --2- 172.21.2.101:0/1746352365 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f17c8018730 0x7f17c801abc0 crc :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:01.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.762 7f17ee73f700 1 -- 172.21.2.101:0/1746352365 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f17e8096020 msgr2=0x7f17e8096420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:01.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.762 7f17ee73f700 1 --2- 172.21.2.101:0/1746352365 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f17e8096020 0x7f17e8096420 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 rx=0x7f17dc009fb0 tx=0x7f17dc00b990).stop 2021-02-10T11:37:01.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.762 7f17ee73f700 1 -- 172.21.2.101:0/1746352365 shutdown_connections 2021-02-10T11:37:01.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.762 7f17ee73f700 1 --2- 172.21.2.101:0/1746352365 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f17c8018730 0x7f17c801abc0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:01.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.762 7f17ee73f700 1 --2- 172.21.2.101:0/1746352365 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17e8092a10 0x7f17e8092e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:01.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.762 7f17ee73f700 1 --2- 172.21.2.101:0/1746352365 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f17e8096020 0x7f17e8096420 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:01.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.762 7f17ee73f700 1 --2- 172.21.2.101:0/1746352365 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f17e8095600 0x7f17e8095a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:01.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.762 7f17ee73f700 1 -- 172.21.2.101:0/1746352365 >> 172.21.2.101:0/1746352365 conn(0x7f17e81a6190 msgr2=0x7f17e80949b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:01.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.763 7f17ee73f700 1 -- 172.21.2.101:0/1746352365 shutdown_connections 2021-02-10T11:37:01.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:01.763 7f17ee73f700 1 -- 172.21.2.101:0/1746352365 wait complete. 2021-02-10T11:37:01.777 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:37:02.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.193 7f889ca2b700 1 Processor -- start 2021-02-10T11:37:02.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.193 7f889ca2b700 1 -- start start 2021-02-10T11:37:02.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.194 7f889ca2b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88980d1c80 0x7f88980cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:02.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.194 7f889ca2b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88980d0180 con 0x7f88980cf590 2021-02-10T11:37:02.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.194 7f889ca2b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88980fe1d0 con 0x7f88980d1880 2021-02-10T11:37:02.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.194 7f889ca2b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88981b0cc0 con 0x7f88980d1c80 2021-02-10T11:37:02.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.194 7f8896e1d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f88980cf590 0x7f88980cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:54978/0 (socket says 172.21.2.101:54978) 2021-02-10T11:37:02.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.194 7f8896e1d700 1 -- 172.21.2.101:0/815529037 learned_addr learned my addr 172.21.2.101:0/815529037 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:37:02.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.194 7f888de1b700 1 --2- 172.21.2.101:0/815529037 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88980d1c80 0x7f88980cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:02.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.194 7f8895e1b700 1 -- 172.21.2.101:0/815529037 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4014376693 0 0) 0x7f8890000c10 con 0x7f88980cf590 2021-02-10T11:37:02.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.194 7f8895e1b700 1 -- 172.21.2.101:0/815529037 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8884002fe0 con 0x7f88980cf590 2021-02-10T11:37:02.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.195 7f888de1b700 1 -- 172.21.2.101:0/815529037 >> v1:172.21.2.108:6789/0 conn(0x7f88980d1880 legacy=0x7f88981b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:02.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.195 7f888de1b700 1 -- 172.21.2.101:0/815529037 >> v1:172.21.2.101:6789/0 conn(0x7f88980cf590 legacy=0x7f88980cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:02.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.195 7f888de1b700 1 -- 172.21.2.101:0/815529037 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8898109a70 con 0x7f88980d1c80 2021-02-10T11:37:02.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.195 7f888de1b700 1 --2- 172.21.2.101:0/815529037 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88980d1c80 0x7f88980cef70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f8880006650 tx=0x7f8880009a80).ready entity=mon.2 client_cookie=908b0f3b71003982 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:02.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.195 7f8895e1b700 1 -- 172.21.2.101:0/815529037 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f888000c0c0 con 0x7f88980d1c80 2021-02-10T11:37:02.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.195 7f8895e1b700 1 -- 172.21.2.101:0/815529037 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888000c340 con 0x7f88980d1c80 2021-02-10T11:37:02.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.195 7f8895e1b700 1 -- 172.21.2.101:0/815529037 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f888000c860 con 0x7f88980d1c80 2021-02-10T11:37:02.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.195 7f889ca2b700 1 -- 172.21.2.101:0/815529037 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88980d1c80 msgr2=0x7f88980cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:02.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.195 7f889ca2b700 1 --2- 172.21.2.101:0/815529037 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88980d1c80 0x7f88980cef70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f8880006650 tx=0x7f8880009a80).stop 2021-02-10T11:37:02.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.196 7f889ca2b700 1 -- 172.21.2.101:0/815529037 shutdown_connections 2021-02-10T11:37:02.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.196 7f889ca2b700 1 --2- 172.21.2.101:0/815529037 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88980d1c80 0x7f88980cef70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:02.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.196 7f889ca2b700 1 -- 172.21.2.101:0/815529037 >> 172.21.2.101:0/815529037 conn(0x7f88980ce430 msgr2=0x7f88980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:02.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.196 7f889ca2b700 1 -- 172.21.2.101:0/815529037 shutdown_connections 2021-02-10T11:37:02.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.196 7f889ca2b700 1 -- 172.21.2.101:0/815529037 wait complete. 2021-02-10T11:37:02.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.211 7f889ca2b700 1 Processor -- start 2021-02-10T11:37:02.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.211 7f889ca2b700 1 -- start start 2021-02-10T11:37:02.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.211 7f889ca2b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8898086520 0x7f88981a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:02.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.211 7f889ca2b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8898086920 0x7f8898187e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:02.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.211 7f889ca2b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88981a22d0 0x7f88981884a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:02.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.211 7f889ca2b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8898099600 con 0x7f8898086920 2021-02-10T11:37:02.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.211 7f889ca2b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88980999f0 con 0x7f8898086520 2021-02-10T11:37:02.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.211 7f889ca2b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8898188ac0 con 0x7f88981a22d0 2021-02-10T11:37:02.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.211 7f888de1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8898086920 0x7f8898187e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:02.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.211 7f888de1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8898086920 0x7f8898187e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54488/0 (socket says 172.21.2.101:54488) 2021-02-10T11:37:02.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.211 7f888de1b700 1 -- 172.21.2.101:0/4027161737 learned_addr learned my addr 172.21.2.101:0/4027161737 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:02.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.211 7f8896e1d700 1 --2- 172.21.2.101:0/4027161737 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88981a22d0 0x7f88981884a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:02.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.211 7f889661c700 1 --2- 172.21.2.101:0/4027161737 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8898086520 0x7f88981a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:02.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.212 7f888de1b700 1 -- 172.21.2.101:0/4027161737 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88981a22d0 msgr2=0x7f88981884a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:02.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.212 7f888de1b700 1 --2- 172.21.2.101:0/4027161737 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88981a22d0 0x7f88981884a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:02.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.212 7f888de1b700 1 -- 172.21.2.101:0/4027161737 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8898086520 msgr2=0x7f88981a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:02.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.212 7f888de1b700 1 --2- 172.21.2.101:0/4027161737 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8898086520 0x7f88981a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:02.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.212 7f888de1b700 1 -- 172.21.2.101:0/4027161737 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8898188ce0 con 0x7f8898086920 2021-02-10T11:37:02.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.212 7f888de1b700 1 --2- 172.21.2.101:0/4027161737 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8898086920 0x7f8898187e80 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 rx=0x7f8880009630 tx=0x7f888000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:02.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.212 7f888f7fe700 1 -- 172.21.2.101:0/4027161737 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f88800093a0 con 0x7f8898086920 2021-02-10T11:37:02.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.212 7f888f7fe700 1 -- 172.21.2.101:0/4027161737 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888000bd70 con 0x7f8898086920 2021-02-10T11:37:02.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.212 7f889ca2b700 1 -- 172.21.2.101:0/4027161737 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8898188ce0 con 0x7f8898086920 2021-02-10T11:37:02.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.212 7f888f7fe700 1 -- 172.21.2.101:0/4027161737 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8880005560 con 0x7f8898086920 2021-02-10T11:37:02.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.212 7f889ca2b700 1 -- 172.21.2.101:0/4027161737 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8898189270 con 0x7f8898086920 2021-02-10T11:37:02.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.213 7f888f7fe700 1 -- 172.21.2.101:0/4027161737 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8880008240 con 0x7f8898086920 2021-02-10T11:37:02.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.213 7f888f7fe700 1 --2- 172.21.2.101:0/4027161737 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f887c0186c0 0x7f887c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:02.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.213 7f888f7fe700 1 -- 172.21.2.101:0/4027161737 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8880032400 con 0x7f8898086920 2021-02-10T11:37:02.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.213 7f889661c700 1 --2- 172.21.2.101:0/4027161737 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f887c0186c0 0x7f887c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:02.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.213 7f889661c700 1 --2- 172.21.2.101:0/4027161737 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f887c0186c0 0x7f887c01abc0 crc :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:02.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.243 7f889ca2b700 1 -- 172.21.2.101:0/4027161737 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88980fe1d0 con 0x7f8898086920 2021-02-10T11:37:02.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.246 7f888f7fe700 1 -- 172.21.2.101:0/4027161737 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8880008010 con 0x7f8898086920 2021-02-10T11:37:02.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.377 7f889ca2b700 1 -- 172.21.2.101:0/4027161737 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f88980fe1d0 con 0x7f8898086920 2021-02-10T11:37:02.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.381 7f888f7fe700 1 -- 172.21.2.101:0/4027161737 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v52) v1 ==== 184+0+0 (secure 0 0 0) 0x7f888002df40 con 0x7f8898086920 2021-02-10T11:37:02.391 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.3 2021-02-10T11:37:02.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.386 7f889ca2b700 1 -- 172.21.2.101:0/4027161737 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f887c0186c0 msgr2=0x7f887c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.386 7f889ca2b700 1 --2- 172.21.2.101:0/4027161737 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f887c0186c0 0x7f887c01abc0 crc :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.386 7f889ca2b700 1 -- 172.21.2.101:0/4027161737 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8898086920 msgr2=0x7f8898187e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.386 7f889ca2b700 1 --2- 172.21.2.101:0/4027161737 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8898086920 0x7f8898187e80 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 rx=0x7f8880009630 tx=0x7f888000b980).stop 2021-02-10T11:37:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.387 7f889ca2b700 1 -- 172.21.2.101:0/4027161737 shutdown_connections 2021-02-10T11:37:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.387 7f889ca2b700 1 --2- 172.21.2.101:0/4027161737 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f887c0186c0 0x7f887c01abc0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.387 7f889ca2b700 1 --2- 172.21.2.101:0/4027161737 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88981a22d0 0x7f88981884a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.387 7f889ca2b700 1 --2- 172.21.2.101:0/4027161737 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8898086920 0x7f8898187e80 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.387 7f889ca2b700 1 --2- 172.21.2.101:0/4027161737 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8898086520 0x7f88981a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.387 7f889ca2b700 1 -- 172.21.2.101:0/4027161737 >> 172.21.2.101:0/4027161737 conn(0x7f889819f880 msgr2=0x7f889808f890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.387 7f889ca2b700 1 -- 172.21.2.101:0/4027161737 shutdown_connections 2021-02-10T11:37:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.387 7f889ca2b700 1 -- 172.21.2.101:0/4027161737 wait complete. 2021-02-10T11:37:02.401 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:37:02.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.806 7f7555b30700 1 Processor -- start 2021-02-10T11:37:02.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.806 7f7555b30700 1 -- start start 2021-02-10T11:37:02.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.806 7f7555b30700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f75500d0340 0x7f75500ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:02.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.806 7f7555b30700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75500d1c70 0x7f75500efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:02.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.806 7f7555b30700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75500f0b10 con 0x7f75500d1870 2021-02-10T11:37:02.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.806 7f7555b30700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75500e5ed0 con 0x7f75500d0340 2021-02-10T11:37:02.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.806 7f7555b30700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75501b0be0 con 0x7f75500d1c70 2021-02-10T11:37:02.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.807 7f754effd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f75500d0340 0x7f75500ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:02.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.807 7f754effd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f75500d0340 0x7f75500ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54496/0 (socket says 172.21.2.101:54496) 2021-02-10T11:37:02.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.807 7f754effd700 1 -- 172.21.2.101:0/3100146600 learned_addr learned my addr 172.21.2.101:0/3100146600 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:02.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.807 7f754effd700 1 -- 172.21.2.101:0/3100146600 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75500d1c70 msgr2=0x7f75500efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:02.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.807 7f754effd700 1 --2- 172.21.2.101:0/3100146600 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75500d1c70 0x7f75500efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:02.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.807 7f754effd700 1 -- 172.21.2.101:0/3100146600 >> v1:172.21.2.108:6789/0 conn(0x7f75500d1870 legacy=0x7f75501b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:02.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.807 7f754effd700 1 -- 172.21.2.101:0/3100146600 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75500f20a0 con 0x7f75500d0340 2021-02-10T11:37:02.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.807 7f754effd700 1 --2- 172.21.2.101:0/3100146600 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75500d0340 0x7f75500ef640 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 rx=0x7f7540006700 tx=0x7f7540009b80).ready entity=mon.0 client_cookie=4179572704ad7008 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:02.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.807 7f754e7fc700 1 -- 172.21.2.101:0/3100146600 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7540000c90 con 0x7f75500d0340 2021-02-10T11:37:02.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.807 7f754e7fc700 1 -- 172.21.2.101:0/3100146600 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754000c0c0 con 0x7f75500d0340 2021-02-10T11:37:02.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.807 7f754e7fc700 1 -- 172.21.2.101:0/3100146600 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f754000c5e0 con 0x7f75500d0340 2021-02-10T11:37:02.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.807 7f7555b30700 1 -- 172.21.2.101:0/3100146600 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75500d0340 msgr2=0x7f75500ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:02.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.807 7f7555b30700 1 --2- 172.21.2.101:0/3100146600 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75500d0340 0x7f75500ef640 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 rx=0x7f7540006700 tx=0x7f7540009b80).stop 2021-02-10T11:37:02.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.808 7f7555b30700 1 -- 172.21.2.101:0/3100146600 shutdown_connections 2021-02-10T11:37:02.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.808 7f7555b30700 1 --2- 172.21.2.101:0/3100146600 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75500d1c70 0x7f75500efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:02.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.808 7f7555b30700 1 --2- 172.21.2.101:0/3100146600 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75500d0340 0x7f75500ef640 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:02.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.808 7f7555b30700 1 -- 172.21.2.101:0/3100146600 >> 172.21.2.101:0/3100146600 conn(0x7f75500ce450 msgr2=0x7f75500d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:02.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.808 7f7555b30700 1 -- 172.21.2.101:0/3100146600 shutdown_connections 2021-02-10T11:37:02.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.808 7f7555b30700 1 -- 172.21.2.101:0/3100146600 wait complete. 2021-02-10T11:37:02.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.823 7f7555b30700 1 Processor -- start 2021-02-10T11:37:02.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.823 7f7555b30700 1 -- start start 2021-02-10T11:37:02.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.823 7f7555b30700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75500b0ad0 0x7f7550091f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:02.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.823 7f7555b30700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75501a8850 0x7f75501a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:02.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.823 7f7555b30700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75501a6fa0 0x7f75501a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:02.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.823 7f7555b30700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75501abfe0 con 0x7f75501a6fa0 2021-02-10T11:37:02.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.823 7f7555b30700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75501a9c40 con 0x7f75500b0ad0 2021-02-10T11:37:02.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.823 7f7555b30700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75501a9730 con 0x7f75501a8850 2021-02-10T11:37:02.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.824 7f754ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75501a6fa0 0x7f75501a73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:02.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.824 7f754ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75501a6fa0 0x7f75501a73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54504/0 (socket says 172.21.2.101:54504) 2021-02-10T11:37:02.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.824 7f754ffff700 1 -- 172.21.2.101:0/2849832857 learned_addr learned my addr 172.21.2.101:0/2849832857 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:02.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.824 7f754ffff700 1 -- 172.21.2.101:0/2849832857 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75501a8850 msgr2=0x7f75501a8c50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:37:02.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.824 7f754ffff700 1 --2- 172.21.2.101:0/2849832857 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75501a8850 0x7f75501a8c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:02.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.824 7f754ffff700 1 -- 172.21.2.101:0/2849832857 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75500b0ad0 msgr2=0x7f7550091f30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:37:02.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.824 7f754ffff700 1 --2- 172.21.2.101:0/2849832857 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75500b0ad0 0x7f7550091f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:02.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.824 7f754ffff700 1 -- 172.21.2.101:0/2849832857 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75501a9270 con 0x7f75501a6fa0 2021-02-10T11:37:02.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.824 7f754ffff700 1 --2- 172.21.2.101:0/2849832857 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75501a6fa0 0x7f75501a73a0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 rx=0x7f7548009980 tx=0x7f754800bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:02.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.824 7f7547fff700 1 -- 172.21.2.101:0/2849832857 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7548005af0 con 0x7f75501a6fa0 2021-02-10T11:37:02.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.824 7f7547fff700 1 -- 172.21.2.101:0/2849832857 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7548005d70 con 0x7f75501a6fa0 2021-02-10T11:37:02.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.824 7f7547fff700 1 -- 172.21.2.101:0/2849832857 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f75480062b0 con 0x7f75501a6fa0 2021-02-10T11:37:02.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.824 7f7555b30700 1 -- 172.21.2.101:0/2849832857 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75501a9270 con 0x7f75501a6fa0 2021-02-10T11:37:02.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.824 7f7555b30700 1 -- 172.21.2.101:0/2849832857 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75501a7cb0 con 0x7f75501a6fa0 2021-02-10T11:37:02.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.825 7f7547fff700 1 -- 172.21.2.101:0/2849832857 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7548006030 con 0x7f75501a6fa0 2021-02-10T11:37:02.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.825 7f7547fff700 1 --2- 172.21.2.101:0/2849832857 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75340186c0 0x7f753401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:02.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.825 7f7547fff700 1 -- 172.21.2.101:0/2849832857 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f75480328e0 con 0x7f75501a6fa0 2021-02-10T11:37:02.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.825 7f754f7fe700 1 --2- 172.21.2.101:0/2849832857 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75340186c0 0x7f753401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:02.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.825 7f754f7fe700 1 --2- 172.21.2.101:0/2849832857 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75340186c0 0x7f753401abc0 crc :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:02.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.856 7f7555b30700 1 -- 172.21.2.101:0/2849832857 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75501a9710 con 0x7f75501a6fa0 2021-02-10T11:37:02.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.859 7f7547fff700 1 -- 172.21.2.101:0/2849832857 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f754802e710 con 0x7f75501a6fa0 2021-02-10T11:37:02.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.987 7f7555b30700 1 -- 172.21.2.101:0/2849832857 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75501a9710 con 0x7f75501a6fa0 2021-02-10T11:37:02.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.988 7f7547fff700 1 -- 172.21.2.101:0/2849832857 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f754802e710 con 0x7f75501a6fa0 2021-02-10T11:37:02.994 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:37:02.995 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9360,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2529021396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2529021396},{"type":"v1","addr":"172.21.2.101:6835","nonce":2529021396}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:56.648753","modified":"2021-02-10 11:36:58.653009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6005":{"gid":6005,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2363068042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2363068042},{"type":"v1","addr":"172.21.2.108:6833","nonce":2363068042}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-02-10T11:37:02.995 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 62 2021-02-10T11:37:02.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.990 7f7555b30700 1 -- 172.21.2.101:0/2849832857 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75340186c0 msgr2=0x7f753401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:02.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.990 7f7555b30700 1 --2- 172.21.2.101:0/2849832857 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75340186c0 0x7f753401abc0 crc :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:02.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.990 7f7555b30700 1 -- 172.21.2.101:0/2849832857 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75501a6fa0 msgr2=0x7f75501a73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:02.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.990 7f7555b30700 1 --2- 172.21.2.101:0/2849832857 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75501a6fa0 0x7f75501a73a0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 rx=0x7f7548009980 tx=0x7f754800bcd0).stop 2021-02-10T11:37:02.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.990 7f7555b30700 1 -- 172.21.2.101:0/2849832857 shutdown_connections 2021-02-10T11:37:02.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.990 7f7555b30700 1 --2- 172.21.2.101:0/2849832857 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f75340186c0 0x7f753401abc0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:02.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.990 7f7555b30700 1 --2- 172.21.2.101:0/2849832857 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75501a8850 0x7f75501a8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:02.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.990 7f7555b30700 1 --2- 172.21.2.101:0/2849832857 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75501a6fa0 0x7f75501a73a0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:02.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.990 7f7555b30700 1 --2- 172.21.2.101:0/2849832857 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75500b0ad0 0x7f7550091f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:02.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.990 7f7555b30700 1 -- 172.21.2.101:0/2849832857 >> 172.21.2.101:0/2849832857 conn(0x7f755019f8f0 msgr2=0x7f75501a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:02.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.990 7f7555b30700 1 -- 172.21.2.101:0/2849832857 shutdown_connections 2021-02-10T11:37:02.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:02.990 7f7555b30700 1 -- 172.21.2.101:0/2849832857 wait complete. 2021-02-10T11:37:03.206 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:36:56.648753', 'modified': '2021-02-10 11:36:58.653009', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6005}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6005': {'gid': 6005, 'name': 'b', 'rank': 0, 'incarnation': 60, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/2363068042', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 2363068042}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 2363068042}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:37:03.206 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:37:03.206 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:37:03.605 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:37:03.605 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:37:03.606 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:37:03.606 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 6005, 2021-02-10T11:37:03.606 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:37:03.606 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:37:03.606 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 6.9488036989999999, 2021-02-10T11:37:03.606 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 62, 2021-02-10T11:37:03.607 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 85, 2021-02-10T11:37:03.607 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 84, 2021-02-10T11:37:03.607 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 12.589646870999999 2021-02-10T11:37:03.607 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:37:03.609 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 6005, "want_state": "up:active", "state": "up:active", "rank_uptime": 6.9488036989999999, "mdsmap_epoch": 62, "osdmap_epoch": 85, "osdmap_epoch_barrier": 84, "uptime": 12.589646870999999 } 2021-02-10T11:37:03.610 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:37:04.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.018 7fb07e443700 1 Processor -- start 2021-02-10T11:37:04.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.018 7fb07e443700 1 -- start start 2021-02-10T11:37:04.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.018 7fb07e443700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb0780d1870 0x7fb0781b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:04.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.018 7fb07e443700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0781b0a10 0x7fb0780cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:04.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.018 7fb07e443700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0780f0b10 con 0x7fb0780d1c70 2021-02-10T11:37:04.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.018 7fb07e443700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0780e5ed0 con 0x7fb0780d1870 2021-02-10T11:37:04.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.018 7fb07e443700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0780f0650 con 0x7fb0781b0a10 2021-02-10T11:37:04.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.018 7fb077fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb0780d1870 0x7fb0781b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:04.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.019 7fb077fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb0780d1870 0x7fb0781b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54508/0 (socket says 172.21.2.101:54508) 2021-02-10T11:37:04.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.018 7fb07ca60700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb0780d1c70 0x7fb0780cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55006/0 (socket says 172.21.2.101:55006) 2021-02-10T11:37:04.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.019 7fb077fff700 1 -- 172.21.2.101:0/652908228 learned_addr learned my addr 172.21.2.101:0/652908228 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:04.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.019 7fb077fff700 1 -- 172.21.2.101:0/652908228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0781b0a10 msgr2=0x7fb0780cf330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:37:04.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.019 7fb077fff700 1 --2- 172.21.2.101:0/652908228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0781b0a10 0x7fb0780cf330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:04.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.019 7fb077fff700 1 -- 172.21.2.101:0/652908228 >> v1:172.21.2.101:6789/0 conn(0x7fb0780d1c70 legacy=0x7fb0780cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:04.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.019 7fb077fff700 1 -- 172.21.2.101:0/652908228 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0780f0870 con 0x7fb0780d1870 2021-02-10T11:37:04.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.019 7fb077fff700 1 --2- 172.21.2.101:0/652908228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0780d1870 0x7fb0781b03f0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 rx=0x7fb068006700 tx=0x7fb068009b80).ready entity=mon.0 client_cookie=ca3493eedae96ffd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:04.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.019 7fb076ffd700 1 -- 172.21.2.101:0/652908228 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb06800c0c0 con 0x7fb0780d1870 2021-02-10T11:37:04.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.019 7fb076ffd700 1 -- 172.21.2.101:0/652908228 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06800c340 con 0x7fb0780d1870 2021-02-10T11:37:04.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.019 7fb076ffd700 1 -- 172.21.2.101:0/652908228 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb06800c860 con 0x7fb0780d1870 2021-02-10T11:37:04.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.019 7fb07e443700 1 -- 172.21.2.101:0/652908228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0780d1870 msgr2=0x7fb0781b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:04.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.019 7fb07e443700 1 --2- 172.21.2.101:0/652908228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0780d1870 0x7fb0781b03f0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 rx=0x7fb068006700 tx=0x7fb068009b80).stop 2021-02-10T11:37:04.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.020 7fb07e443700 1 -- 172.21.2.101:0/652908228 shutdown_connections 2021-02-10T11:37:04.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.020 7fb07e443700 1 --2- 172.21.2.101:0/652908228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0781b0a10 0x7fb0780cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:04.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.020 7fb07e443700 1 --2- 172.21.2.101:0/652908228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0780d1870 0x7fb0781b03f0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:04.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.020 7fb07e443700 1 -- 172.21.2.101:0/652908228 >> 172.21.2.101:0/652908228 conn(0x7fb0780ce450 msgr2=0x7fb0780d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:04.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.020 7fb07e443700 1 -- 172.21.2.101:0/652908228 shutdown_connections 2021-02-10T11:37:04.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.020 7fb07e443700 1 -- 172.21.2.101:0/652908228 wait complete. 2021-02-10T11:37:04.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.035 7fb07e443700 1 Processor -- start 2021-02-10T11:37:04.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.035 7fb07e443700 1 -- start start 2021-02-10T11:37:04.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.035 7fb07e443700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb07819d440 0x7fb078091060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:04.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.035 7fb07e443700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0781b2da0 0x7fb0780b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:04.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.035 7fb07e443700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0780b2080 0x7fb07819e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:04.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.035 7fb07e443700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07819d840 con 0x7fb0781b2da0 2021-02-10T11:37:04.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.035 7fb07e443700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0781b31a0 con 0x7fb0780b2080 2021-02-10T11:37:04.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.035 7fb07e443700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0781b35d0 con 0x7fb07819d440 2021-02-10T11:37:04.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.035 7fb0777fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0781b2da0 0x7fb0780b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:04.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.035 7fb0777fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0781b2da0 0x7fb0780b1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54516/0 (socket says 172.21.2.101:54516) 2021-02-10T11:37:04.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.035 7fb0777fe700 1 -- 172.21.2.101:0/3127795739 learned_addr learned my addr 172.21.2.101:0/3127795739 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:04.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.036 7fb0777fe700 1 -- 172.21.2.101:0/3127795739 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb07819d440 msgr2=0x7fb078091060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:04.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.036 7fb0777fe700 1 --2- 172.21.2.101:0/3127795739 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb07819d440 0x7fb078091060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:04.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.036 7fb0777fe700 1 -- 172.21.2.101:0/3127795739 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0780b2080 msgr2=0x7fb07819e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:04.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.036 7fb0777fe700 1 --2- 172.21.2.101:0/3127795739 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0780b2080 0x7fb07819e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:04.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.036 7fb0777fe700 1 -- 172.21.2.101:0/3127795739 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0780b2480 con 0x7fb0781b2da0 2021-02-10T11:37:04.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.036 7fb0777fe700 1 --2- 172.21.2.101:0/3127795739 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0781b2da0 0x7fb0780b1a60 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 rx=0x7fb06c00b5c0 tx=0x7fb06c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:04.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.036 7fb074ff9700 1 -- 172.21.2.101:0/3127795739 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb06c006b80 con 0x7fb0781b2da0 2021-02-10T11:37:04.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.036 7fb074ff9700 1 -- 172.21.2.101:0/3127795739 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06c00bd50 con 0x7fb0781b2da0 2021-02-10T11:37:04.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.036 7fb074ff9700 1 -- 172.21.2.101:0/3127795739 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb06c0054d0 con 0x7fb0781b2da0 2021-02-10T11:37:04.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.036 7fb07e443700 1 -- 172.21.2.101:0/3127795739 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0780b2480 con 0x7fb0781b2da0 2021-02-10T11:37:04.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.036 7fb07e443700 1 -- 172.21.2.101:0/3127795739 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07819e680 con 0x7fb0781b2da0 2021-02-10T11:37:04.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.037 7fb074ff9700 1 -- 172.21.2.101:0/3127795739 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb06c006030 con 0x7fb0781b2da0 2021-02-10T11:37:04.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.037 7fb074ff9700 1 --2- 172.21.2.101:0/3127795739 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0580186c0 0x7fb05801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:04.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.037 7fb074ff9700 1 -- 172.21.2.101:0/3127795739 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb06c032460 con 0x7fb0781b2da0 2021-02-10T11:37:04.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.037 7fb077fff700 1 --2- 172.21.2.101:0/3127795739 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0580186c0 0x7fb05801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:04.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.037 7fb077fff700 1 --2- 172.21.2.101:0/3127795739 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0580186c0 0x7fb05801abc0 crc :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:04.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.068 7fb07e443700 1 -- 172.21.2.101:0/3127795739 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0780ebc80 con 0x7fb0781b2da0 2021-02-10T11:37:04.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.071 7fb074ff9700 1 -- 172.21.2.101:0/3127795739 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb06c0009b0 con 0x7fb0781b2da0 2021-02-10T11:37:04.202 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.196 7fb07e443700 1 -- 172.21.2.101:0/3127795739 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0780ebc80 con 0x7fb0781b2da0 2021-02-10T11:37:04.202 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.197 7fb074ff9700 1 -- 172.21.2.101:0/3127795739 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+13996 (secure 0 0 0) 0x7fb06c006030 con 0x7fb0781b2da0 2021-02-10T11:37:04.203 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:37:04.203 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":85,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:36:58.653077","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:36:54.618463","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2021-02-10 11:36:55.644563","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:37:04.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.198 7fb07e443700 1 -- 172.21.2.101:0/3127795739 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0580186c0 msgr2=0x7fb05801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:04.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.198 7fb07e443700 1 --2- 172.21.2.101:0/3127795739 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0580186c0 0x7fb05801abc0 crc :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:04.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.198 7fb07e443700 1 -- 172.21.2.101:0/3127795739 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0781b2da0 msgr2=0x7fb0780b1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:04.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.198 7fb07e443700 1 --2- 172.21.2.101:0/3127795739 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0781b2da0 0x7fb0780b1a60 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 rx=0x7fb06c00b5c0 tx=0x7fb06c00b960).stop 2021-02-10T11:37:04.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.198 7fb07e443700 1 -- 172.21.2.101:0/3127795739 shutdown_connections 2021-02-10T11:37:04.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.198 7fb07e443700 1 --2- 172.21.2.101:0/3127795739 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0780b2080 0x7fb07819e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:04.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.198 7fb07e443700 1 --2- 172.21.2.101:0/3127795739 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0781b2da0 0x7fb0780b1a60 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:04.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.198 7fb07e443700 1 --2- 172.21.2.101:0/3127795739 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0580186c0 0x7fb05801abc0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:04.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.198 7fb07e443700 1 --2- 172.21.2.101:0/3127795739 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb07819d440 0x7fb078091060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:04.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.198 7fb07e443700 1 -- 172.21.2.101:0/3127795739 >> 172.21.2.101:0/3127795739 conn(0x7fb0781a05b0 msgr2=0x7fb0780ebeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:04.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.199 7fb07e443700 1 -- 172.21.2.101:0/3127795739 shutdown_connections 2021-02-10T11:37:04.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.199 7fb07e443700 1 -- 172.21.2.101:0/3127795739 wait complete. 2021-02-10T11:37:04.213 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:36:54.618463', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:36:55.644563', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:37:04.213 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:37:04.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.623 7fd7b9948700 1 Processor -- start 2021-02-10T11:37:04.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.623 7fd7b9948700 1 -- start start 2021-02-10T11:37:04.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.623 7fd7b9948700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd7b40d1c70 0x7fd7b40cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:04.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.623 7fd7b9948700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7b40cf580 0x7fd7b40cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:04.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.623 7fd7b9948700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b40fe1c0 con 0x7fd7b40d1870 2021-02-10T11:37:04.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.623 7fd7b9948700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b41b0cb0 con 0x7fd7b40d1c70 2021-02-10T11:37:04.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.623 7fd7b9948700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b4109a60 con 0x7fd7b40cf580 2021-02-10T11:37:04.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.623 7fd7b2d73700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd7b40d1c70 0x7fd7b40cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:04.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.623 7fd7b2d73700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd7b40d1c70 0x7fd7b40cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54522/0 (socket says 172.21.2.101:54522) 2021-02-10T11:37:04.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.623 7fd7b2d73700 1 -- 172.21.2.101:0/3520117344 learned_addr learned my addr 172.21.2.101:0/3520117344 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:04.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.624 7fd7b2d73700 1 -- 172.21.2.101:0/3520117344 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7b40cf580 msgr2=0x7fd7b40cf980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:37:04.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.624 7fd7b2d73700 1 --2- 172.21.2.101:0/3520117344 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7b40cf580 0x7fd7b40cf980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:04.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.624 7fd7b2d73700 1 -- 172.21.2.101:0/3520117344 >> v1:172.21.2.108:6789/0 conn(0x7fd7b40d1870 legacy=0x7fd7b41b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:37:04.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.624 7fd7b2d73700 1 -- 172.21.2.101:0/3520117344 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b4088070 con 0x7fd7b40d1c70 2021-02-10T11:37:04.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.624 7fd7b2d73700 1 --2- 172.21.2.101:0/3520117344 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7b40d1c70 0x7fd7b40cef60 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 rx=0x7fd7a8006700 tx=0x7fd7a8009b80).ready entity=mon.0 client_cookie=c1b9de231c697652 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:04.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.624 7fd7b2572700 1 -- 172.21.2.101:0/3520117344 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd7a800c0c0 con 0x7fd7b40d1c70 2021-02-10T11:37:04.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.624 7fd7b2572700 1 -- 172.21.2.101:0/3520117344 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a800c340 con 0x7fd7b40d1c70 2021-02-10T11:37:04.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.624 7fd7b2572700 1 -- 172.21.2.101:0/3520117344 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd7a800c860 con 0x7fd7b40d1c70 2021-02-10T11:37:04.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.624 7fd7b9948700 1 -- 172.21.2.101:0/3520117344 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7b40d1c70 msgr2=0x7fd7b40cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:04.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.624 7fd7b9948700 1 --2- 172.21.2.101:0/3520117344 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7b40d1c70 0x7fd7b40cef60 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 rx=0x7fd7a8006700 tx=0x7fd7a8009b80).stop 2021-02-10T11:37:04.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.625 7fd7b9948700 1 -- 172.21.2.101:0/3520117344 shutdown_connections 2021-02-10T11:37:04.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.625 7fd7b9948700 1 --2- 172.21.2.101:0/3520117344 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7b40cf580 0x7fd7b40cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:04.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.625 7fd7b9948700 1 --2- 172.21.2.101:0/3520117344 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7b40d1c70 0x7fd7b40cef60 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:04.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.625 7fd7b9948700 1 -- 172.21.2.101:0/3520117344 >> 172.21.2.101:0/3520117344 conn(0x7fd7b40ce420 msgr2=0x7fd7b40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:04.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.625 7fd7b9948700 1 -- 172.21.2.101:0/3520117344 shutdown_connections 2021-02-10T11:37:04.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.625 7fd7b9948700 1 -- 172.21.2.101:0/3520117344 wait complete. 2021-02-10T11:37:04.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.639 7fd7b9948700 1 Processor -- start 2021-02-10T11:37:04.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.639 7fd7b9948700 1 -- start start 2021-02-10T11:37:04.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.640 7fd7b9948700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7b40c3820 0x7fd7b40a7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:04.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.640 7fd7b9948700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7b41a44c0 0x7fd7b41a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:04.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.640 7fd7b9948700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7b40863d0 0x7fd7b40867d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:04.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.640 7fd7b9948700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b4085ed0 con 0x7fd7b40863d0 2021-02-10T11:37:04.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.640 7fd7b9948700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b40c0740 con 0x7fd7b40c3820 2021-02-10T11:37:04.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.640 7fd7b9948700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b4086df0 con 0x7fd7b41a44c0 2021-02-10T11:37:04.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.640 7fd7b3d75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7b40863d0 0x7fd7b40867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:04.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.640 7fd7b3d75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7b40863d0 0x7fd7b40867d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54532/0 (socket says 172.21.2.101:54532) 2021-02-10T11:37:04.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.640 7fd7b3d75700 1 -- 172.21.2.101:0/3887333561 learned_addr learned my addr 172.21.2.101:0/3887333561 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:04.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.640 7fd7b3d75700 1 -- 172.21.2.101:0/3887333561 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7b41a44c0 msgr2=0x7fd7b41a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:04.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.640 7fd7b3d75700 1 --2- 172.21.2.101:0/3887333561 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7b41a44c0 0x7fd7b41a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:04.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.640 7fd7b3d75700 1 -- 172.21.2.101:0/3887333561 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7b40c3820 msgr2=0x7fd7b40a7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:04.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.640 7fd7b3d75700 1 --2- 172.21.2.101:0/3887333561 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7b40c3820 0x7fd7b40a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:04.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.640 7fd7b3d75700 1 -- 172.21.2.101:0/3887333561 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b4087010 con 0x7fd7b40863d0 2021-02-10T11:37:04.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.641 7fd7b3d75700 1 --2- 172.21.2.101:0/3887333561 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7b40863d0 0x7fd7b40867d0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 rx=0x7fd7ac009fc0 tx=0x7fd7ac00bc40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:04.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.641 7fd79bfff700 1 -- 172.21.2.101:0/3887333561 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd7ac003020 con 0x7fd7b40863d0 2021-02-10T11:37:04.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.641 7fd79bfff700 1 -- 172.21.2.101:0/3887333561 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ac005850 con 0x7fd7b40863d0 2021-02-10T11:37:04.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.641 7fd79bfff700 1 -- 172.21.2.101:0/3887333561 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd7ac005d70 con 0x7fd7b40863d0 2021-02-10T11:37:04.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.641 7fd7b9948700 1 -- 172.21.2.101:0/3887333561 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b4087010 con 0x7fd7b40863d0 2021-02-10T11:37:04.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.641 7fd7b9948700 1 -- 172.21.2.101:0/3887333561 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b4087010 con 0x7fd7b40863d0 2021-02-10T11:37:04.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.641 7fd79bfff700 1 -- 172.21.2.101:0/3887333561 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd7ac003020 con 0x7fd7b40863d0 2021-02-10T11:37:04.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.641 7fd79bfff700 1 --2- 172.21.2.101:0/3887333561 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7940186c0 0x7fd79401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:04.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.641 7fd79bfff700 1 -- 172.21.2.101:0/3887333561 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd7ac0328e0 con 0x7fd7b40863d0 2021-02-10T11:37:04.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.642 7fd7b3574700 1 --2- 172.21.2.101:0/3887333561 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7940186c0 0x7fd79401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:04.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.642 7fd7b3574700 1 --2- 172.21.2.101:0/3887333561 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7940186c0 0x7fd79401abc0 crc :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:04.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.673 7fd7b9948700 1 -- 172.21.2.101:0/3887333561 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b40fe1c0 con 0x7fd7b40863d0 2021-02-10T11:37:04.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.676 7fd79bfff700 1 -- 172.21.2.101:0/3887333561 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd7ac021330 con 0x7fd7b40863d0 2021-02-10T11:37:04.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.807 7fd7b9948700 1 -- 172.21.2.101:0/3887333561 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fd7b40fe1c0 con 0x7fd7b40863d0 2021-02-10T11:37:04.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.808 7fd79bfff700 1 -- 172.21.2.101:0/3887333561 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v85) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd7ac021330 con 0x7fd7b40863d0 2021-02-10T11:37:04.814 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:37:04.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.810 7fd7b9948700 1 -- 172.21.2.101:0/3887333561 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7940186c0 msgr2=0x7fd79401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:04.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.810 7fd7b9948700 1 --2- 172.21.2.101:0/3887333561 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7940186c0 0x7fd79401abc0 crc :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:04.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.810 7fd7b9948700 1 -- 172.21.2.101:0/3887333561 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7b40863d0 msgr2=0x7fd7b40867d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:04.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.810 7fd7b9948700 1 --2- 172.21.2.101:0/3887333561 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7b40863d0 0x7fd7b40867d0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 rx=0x7fd7ac009fc0 tx=0x7fd7ac00bc40).stop 2021-02-10T11:37:04.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.810 7fd7b9948700 1 -- 172.21.2.101:0/3887333561 shutdown_connections 2021-02-10T11:37:04.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.810 7fd7b9948700 1 --2- 172.21.2.101:0/3887333561 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7940186c0 0x7fd79401abc0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:04.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.810 7fd7b9948700 1 --2- 172.21.2.101:0/3887333561 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7b41a44c0 0x7fd7b41a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:04.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.810 7fd7b9948700 1 --2- 172.21.2.101:0/3887333561 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7b40863d0 0x7fd7b40867d0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:04.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.810 7fd7b9948700 1 --2- 172.21.2.101:0/3887333561 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7b40c3820 0x7fd7b40a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:04.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.810 7fd7b9948700 1 -- 172.21.2.101:0/3887333561 >> 172.21.2.101:0/3887333561 conn(0x7fd7b40872c0 msgr2=0x7fd7b4095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:04.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.810 7fd7b9948700 1 -- 172.21.2.101:0/3887333561 shutdown_connections 2021-02-10T11:37:04.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:04.810 7fd7b9948700 1 -- 172.21.2.101:0/3887333561 wait complete. 2021-02-10T11:37:04.824 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:37:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.233 7fed2a082700 1 Processor -- start 2021-02-10T11:37:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.233 7fed2a082700 1 -- start start 2021-02-10T11:37:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.233 7fed2a082700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fed240d1c70 0x7fed240cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.233 7fed2a082700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fed240cf580 0x7fed240cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.233 7fed2a082700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed240fe1c0 con 0x7fed240d1870 2021-02-10T11:37:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.233 7fed2a082700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed241b0cb0 con 0x7fed240d1c70 2021-02-10T11:37:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.233 7fed2a082700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed24109a60 con 0x7fed240cf580 2021-02-10T11:37:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.233 7fed22ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fed240d1c70 0x7fed240cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.233 7fed22ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fed240d1c70 0x7fed240cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54536/0 (socket says 172.21.2.101:54536) 2021-02-10T11:37:05.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.233 7fed22ffd700 1 -- 172.21.2.101:0/2004149108 learned_addr learned my addr 172.21.2.101:0/2004149108 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.234 7fed22ffd700 1 -- 172.21.2.101:0/2004149108 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fed240cf580 msgr2=0x7fed240cf980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:37:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.234 7fed22ffd700 1 --2- 172.21.2.101:0/2004149108 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fed240cf580 0x7fed240cf980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.234 7fed22ffd700 1 -- 172.21.2.101:0/2004149108 >> v1:172.21.2.101:6789/0 conn(0x7fed240d1870 legacy=0x7fed241b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.234 7fed22ffd700 1 -- 172.21.2.101:0/2004149108 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed24088070 con 0x7fed240d1c70 2021-02-10T11:37:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.234 7fed22ffd700 1 --2- 172.21.2.101:0/2004149108 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fed240d1c70 0x7fed240cef60 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 rx=0x7fed14006700 tx=0x7fed14009b80).ready entity=mon.0 client_cookie=2db5f85bd4b190c8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.234 7fed227fc700 1 -- 172.21.2.101:0/2004149108 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fed1400c0c0 con 0x7fed240d1c70 2021-02-10T11:37:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.234 7fed227fc700 1 -- 172.21.2.101:0/2004149108 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1400c340 con 0x7fed240d1c70 2021-02-10T11:37:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.234 7fed227fc700 1 -- 172.21.2.101:0/2004149108 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fed1400c860 con 0x7fed240d1c70 2021-02-10T11:37:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.234 7fed2a082700 1 -- 172.21.2.101:0/2004149108 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fed240d1c70 msgr2=0x7fed240cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.234 7fed2a082700 1 --2- 172.21.2.101:0/2004149108 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fed240d1c70 0x7fed240cef60 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 rx=0x7fed14006700 tx=0x7fed14009b80).stop 2021-02-10T11:37:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.235 7fed2a082700 1 -- 172.21.2.101:0/2004149108 shutdown_connections 2021-02-10T11:37:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.235 7fed2a082700 1 --2- 172.21.2.101:0/2004149108 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fed240cf580 0x7fed240cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.235 7fed2a082700 1 --2- 172.21.2.101:0/2004149108 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fed240d1c70 0x7fed240cef60 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.235 7fed2a082700 1 -- 172.21.2.101:0/2004149108 >> 172.21.2.101:0/2004149108 conn(0x7fed240ce420 msgr2=0x7fed240d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:05.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.235 7fed2a082700 1 -- 172.21.2.101:0/2004149108 shutdown_connections 2021-02-10T11:37:05.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.235 7fed2a082700 1 -- 172.21.2.101:0/2004149108 wait complete. 2021-02-10T11:37:05.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.249 7fed2a082700 1 Processor -- start 2021-02-10T11:37:05.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.249 7fed2a082700 1 -- start start 2021-02-10T11:37:05.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.250 7fed2a082700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fed240c3820 0x7fed24084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:05.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.250 7fed2a082700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fed240a0290 0x7fed240852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:05.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.250 7fed2a082700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fed24086270 0x7fed24086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:05.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.250 7fed2a082700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed240c0700 con 0x7fed240a0290 2021-02-10T11:37:05.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.250 7fed2a082700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed240858f0 con 0x7fed240c3820 2021-02-10T11:37:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.250 7fed2a082700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed24085b10 con 0x7fed24086270 2021-02-10T11:37:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.250 7fed22ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fed240a0290 0x7fed240852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.250 7fed22ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fed240a0290 0x7fed240852d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54540/0 (socket says 172.21.2.101:54540) 2021-02-10T11:37:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.250 7fed22ffd700 1 -- 172.21.2.101:0/2799354036 learned_addr learned my addr 172.21.2.101:0/2799354036 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.250 7fed22ffd700 1 -- 172.21.2.101:0/2799354036 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fed24086270 msgr2=0x7fed24086670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:37:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.250 7fed22ffd700 1 --2- 172.21.2.101:0/2799354036 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fed24086270 0x7fed24086670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.250 7fed22ffd700 1 -- 172.21.2.101:0/2799354036 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fed240c3820 msgr2=0x7fed24084cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:37:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.250 7fed22ffd700 1 --2- 172.21.2.101:0/2799354036 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fed240c3820 0x7fed24084cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.250 7fed22ffd700 1 -- 172.21.2.101:0/2799354036 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed24086c90 con 0x7fed240a0290 2021-02-10T11:37:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.250 7fed22ffd700 1 --2- 172.21.2.101:0/2799354036 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fed240a0290 0x7fed240852d0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 rx=0x7fed1400b5e0 tx=0x7fed1400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.251 7fed1bfff700 1 -- 172.21.2.101:0/2799354036 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fed1400bd70 con 0x7fed240a0290 2021-02-10T11:37:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.251 7fed1bfff700 1 -- 172.21.2.101:0/2799354036 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed14002030 con 0x7fed240a0290 2021-02-10T11:37:05.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.251 7fed1bfff700 1 -- 172.21.2.101:0/2799354036 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fed14002550 con 0x7fed240a0290 2021-02-10T11:37:05.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.251 7fed2a082700 1 -- 172.21.2.101:0/2799354036 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed24086c90 con 0x7fed240a0290 2021-02-10T11:37:05.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.251 7fed2a082700 1 -- 172.21.2.101:0/2799354036 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed241a2f00 con 0x7fed240a0290 2021-02-10T11:37:05.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.251 7fed1bfff700 1 -- 172.21.2.101:0/2799354036 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fed14000c90 con 0x7fed240a0290 2021-02-10T11:37:05.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.251 7fed1bfff700 1 --2- 172.21.2.101:0/2799354036 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fed080186c0 0x7fed0801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:05.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.251 7fed1bfff700 1 -- 172.21.2.101:0/2799354036 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fed140327e0 con 0x7fed240a0290 2021-02-10T11:37:05.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.251 7fed237fe700 1 --2- 172.21.2.101:0/2799354036 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fed080186c0 0x7fed0801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:05.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.252 7fed237fe700 1 --2- 172.21.2.101:0/2799354036 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fed080186c0 0x7fed0801abc0 crc :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:05.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.283 7fed2a082700 1 -- 172.21.2.101:0/2799354036 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed24082a70 con 0x7fed240a0290 2021-02-10T11:37:05.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.286 7fed1bfff700 1 -- 172.21.2.101:0/2799354036 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fed140009b0 con 0x7fed240a0290 2021-02-10T11:37:05.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.413 7fed2a082700 1 -- 172.21.2.101:0/2799354036 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7fed24082a70 con 0x7fed240a0290 2021-02-10T11:37:05.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.413 7fed1bfff700 1 -- 172.21.2.101:0/2799354036 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v85) v1 ==== 104+0+10 (secure 0 0 0) 0x7fed1400bd70 con 0x7fed240a0290 2021-02-10T11:37:05.420 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:37:05.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.415 7fed2a082700 1 -- 172.21.2.101:0/2799354036 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fed080186c0 msgr2=0x7fed0801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:05.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.415 7fed2a082700 1 --2- 172.21.2.101:0/2799354036 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fed080186c0 0x7fed0801abc0 crc :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:05.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.415 7fed2a082700 1 -- 172.21.2.101:0/2799354036 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fed240a0290 msgr2=0x7fed240852d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:05.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.415 7fed2a082700 1 --2- 172.21.2.101:0/2799354036 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fed240a0290 0x7fed240852d0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 rx=0x7fed1400b5e0 tx=0x7fed1400b980).stop 2021-02-10T11:37:05.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.416 7fed2a082700 1 -- 172.21.2.101:0/2799354036 shutdown_connections 2021-02-10T11:37:05.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.416 7fed2a082700 1 --2- 172.21.2.101:0/2799354036 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fed080186c0 0x7fed0801abc0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:05.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.416 7fed2a082700 1 --2- 172.21.2.101:0/2799354036 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fed24086270 0x7fed24086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:05.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.416 7fed2a082700 1 --2- 172.21.2.101:0/2799354036 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fed240a0290 0x7fed240852d0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:05.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.416 7fed2a082700 1 --2- 172.21.2.101:0/2799354036 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fed240c3820 0x7fed24084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:05.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.416 7fed2a082700 1 -- 172.21.2.101:0/2799354036 >> 172.21.2.101:0/2799354036 conn(0x7fed240872c0 msgr2=0x7fed24095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:05.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.416 7fed2a082700 1 -- 172.21.2.101:0/2799354036 shutdown_connections 2021-02-10T11:37:05.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.416 7fed2a082700 1 -- 172.21.2.101:0/2799354036 wait complete. 2021-02-10T11:37:05.430 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:37:05.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.839 7f30746ab700 1 Processor -- start 2021-02-10T11:37:05.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.839 7f30746ab700 1 -- start start 2021-02-10T11:37:05.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.839 7f30746ab700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f306c0d1870 0x7f306c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:05.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.839 7f30746ab700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f306c0cff70 0x7f306c109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:05.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.839 7f30746ab700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306c109a60 con 0x7f306c1b0ae0 2021-02-10T11:37:05.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.839 7f30746ab700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f306c109750 con 0x7f306c0cff70 2021-02-10T11:37:05.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.839 7f30746ab700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306c088070 con 0x7f306c0d1870 2021-02-10T11:37:05.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.839 7f3072cc8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f306c0cff70 0x7f306c109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:05.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.840 7f3072cc8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f306c0cff70 0x7f306c109130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54552/0 (socket says 172.21.2.101:54552) 2021-02-10T11:37:05.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.840 7f3072cc8700 1 -- 172.21.2.101:0/81549397 learned_addr learned my addr 172.21.2.101:0/81549397 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:05.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.840 7f3072cc8700 1 -- 172.21.2.101:0/81549397 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f306c0d1870 msgr2=0x7f306c1b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:37:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.840 7f3072cc8700 1 --2- 172.21.2.101:0/81549397 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f306c0d1870 0x7f306c1b04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.840 7f3072cc8700 1 -- 172.21.2.101:0/81549397 >> v1:172.21.2.108:6789/0 conn(0x7f306c1b0ae0 legacy=0x7f306c108940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:37:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.840 7f3072cc8700 1 -- 172.21.2.101:0/81549397 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c088260 con 0x7f306c0cff70 2021-02-10T11:37:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.840 7f3072cc8700 1 --2- 172.21.2.101:0/81549397 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f306c0cff70 0x7f306c109130 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 rx=0x7f3064006f00 tx=0x7f3064009e90).ready entity=mon.0 client_cookie=9d602dbd35379e40 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.840 7f30714c5700 1 -- 172.21.2.101:0/81549397 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f306400b300 con 0x7f306c0cff70 2021-02-10T11:37:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.840 7f30714c5700 1 -- 172.21.2.101:0/81549397 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306400b580 con 0x7f306c0cff70 2021-02-10T11:37:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.840 7f30714c5700 1 -- 172.21.2.101:0/81549397 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f306400ba60 con 0x7f306c0cff70 2021-02-10T11:37:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.840 7f30746ab700 1 -- 172.21.2.101:0/81549397 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f306c0cff70 msgr2=0x7f306c109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.840 7f30746ab700 1 --2- 172.21.2.101:0/81549397 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f306c0cff70 0x7f306c109130 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 rx=0x7f3064006f00 tx=0x7f3064009e90).stop 2021-02-10T11:37:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.841 7f30746ab700 1 -- 172.21.2.101:0/81549397 shutdown_connections 2021-02-10T11:37:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.841 7f30746ab700 1 --2- 172.21.2.101:0/81549397 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f306c0cff70 0x7f306c109130 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.841 7f30746ab700 1 --2- 172.21.2.101:0/81549397 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f306c0d1870 0x7f306c1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:05.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.841 7f30746ab700 1 -- 172.21.2.101:0/81549397 >> 172.21.2.101:0/81549397 conn(0x7f306c0ce420 msgr2=0x7f306c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:05.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.841 7f30746ab700 1 -- 172.21.2.101:0/81549397 shutdown_connections 2021-02-10T11:37:05.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.841 7f30746ab700 1 -- 172.21.2.101:0/81549397 wait complete. 2021-02-10T11:37:05.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.855 7f30746ab700 1 Processor -- start 2021-02-10T11:37:05.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.855 7f30746ab700 1 -- start start 2021-02-10T11:37:05.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.856 7f30746ab700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f306c083ed0 0x7f306c0842d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:05.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.856 7f30746ab700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f306c0848f0 0x7f306c084cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:05.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.856 7f30746ab700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f306c085f20 0x7f306c086320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:05.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.856 7f30746ab700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c083980 con 0x7f306c085f20 2021-02-10T11:37:05.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.856 7f30746ab700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c086940 con 0x7f306c083ed0 2021-02-10T11:37:05.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.856 7f30746ab700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306c086b60 con 0x7f306c0848f0 2021-02-10T11:37:05.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.856 7f3072cc8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f306c085f20 0x7f306c086320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:05.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.856 7f3072cc8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f306c085f20 0x7f306c086320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54558/0 (socket says 172.21.2.101:54558) 2021-02-10T11:37:05.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.856 7f3072cc8700 1 -- 172.21.2.101:0/3472414636 learned_addr learned my addr 172.21.2.101:0/3472414636 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:05.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.856 7f3072cc8700 1 -- 172.21.2.101:0/3472414636 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f306c0848f0 msgr2=0x7f306c084cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:37:05.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.856 7f3072cc8700 1 --2- 172.21.2.101:0/3472414636 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f306c0848f0 0x7f306c084cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:05.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.856 7f3072cc8700 1 -- 172.21.2.101:0/3472414636 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f306c083ed0 msgr2=0x7f306c0842d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:37:05.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.856 7f3072cc8700 1 --2- 172.21.2.101:0/3472414636 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f306c083ed0 0x7f306c0842d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:05.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.856 7f3072cc8700 1 -- 172.21.2.101:0/3472414636 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c086d80 con 0x7f306c085f20 2021-02-10T11:37:05.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.856 7f3072cc8700 1 --2- 172.21.2.101:0/3472414636 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f306c085f20 0x7f306c086320 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 rx=0x7f3064009980 tx=0x7f306400bcc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:05.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.857 7f305effd700 1 -- 172.21.2.101:0/3472414636 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f306400be00 con 0x7f306c085f20 2021-02-10T11:37:05.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.857 7f305effd700 1 -- 172.21.2.101:0/3472414636 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3064005820 con 0x7f306c085f20 2021-02-10T11:37:05.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.857 7f305effd700 1 -- 172.21.2.101:0/3472414636 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3064005d40 con 0x7f306c085f20 2021-02-10T11:37:05.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.857 7f30746ab700 1 -- 172.21.2.101:0/3472414636 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f306c086d80 con 0x7f306c085f20 2021-02-10T11:37:05.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.857 7f30746ab700 1 -- 172.21.2.101:0/3472414636 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306c087310 con 0x7f306c085f20 2021-02-10T11:37:05.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.858 7f305effd700 1 -- 172.21.2.101:0/3472414636 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3064005820 con 0x7f306c085f20 2021-02-10T11:37:05.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.858 7f305effd700 1 --2- 172.21.2.101:0/3472414636 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f305401cad0 0x7f305401eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:05.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.858 7f305effd700 1 -- 172.21.2.101:0/3472414636 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3064032730 con 0x7f306c085f20 2021-02-10T11:37:05.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.858 7f30724c7700 1 --2- 172.21.2.101:0/3472414636 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f305401cad0 0x7f305401eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:05.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.858 7f30724c7700 1 --2- 172.21.2.101:0/3472414636 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f305401cad0 0x7f305401eff0 crc :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:05.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.889 7f30746ab700 1 -- 172.21.2.101:0/3472414636 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306c082a70 con 0x7f306c085f20 2021-02-10T11:37:05.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:05.892 7f305effd700 1 -- 172.21.2.101:0/3472414636 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3064000f90 con 0x7f306c085f20 2021-02-10T11:37:06.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.020 7f30746ab700 1 -- 172.21.2.101:0/3472414636 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f306c083980 con 0x7f306c085f20 2021-02-10T11:37:06.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.020 7f305effd700 1 -- 172.21.2.101:0/3472414636 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v85) v1 ==== 100+0+10 (secure 0 0 0) 0x7f3064006ee0 con 0x7f306c085f20 2021-02-10T11:37:06.027 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:37:06.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.023 7f30746ab700 1 -- 172.21.2.101:0/3472414636 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f305401cad0 msgr2=0x7f305401eff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:06.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.023 7f30746ab700 1 --2- 172.21.2.101:0/3472414636 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f305401cad0 0x7f305401eff0 crc :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:06.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.023 7f30746ab700 1 -- 172.21.2.101:0/3472414636 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f306c085f20 msgr2=0x7f306c086320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:06.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.023 7f30746ab700 1 --2- 172.21.2.101:0/3472414636 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f306c085f20 0x7f306c086320 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 rx=0x7f3064009980 tx=0x7f306400bcc0).stop 2021-02-10T11:37:06.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.023 7f30746ab700 1 -- 172.21.2.101:0/3472414636 shutdown_connections 2021-02-10T11:37:06.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.023 7f30746ab700 1 --2- 172.21.2.101:0/3472414636 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f305401cad0 0x7f305401eff0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:06.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.023 7f30746ab700 1 --2- 172.21.2.101:0/3472414636 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f306c0848f0 0x7f306c084cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:06.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.023 7f30746ab700 1 --2- 172.21.2.101:0/3472414636 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f306c085f20 0x7f306c086320 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:06.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.023 7f30746ab700 1 --2- 172.21.2.101:0/3472414636 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f306c083ed0 0x7f306c0842d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:06.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.023 7f30746ab700 1 -- 172.21.2.101:0/3472414636 >> 172.21.2.101:0/3472414636 conn(0x7f306c0bc0c0 msgr2=0x7f306c082ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:06.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.023 7f30746ab700 1 -- 172.21.2.101:0/3472414636 shutdown_connections 2021-02-10T11:37:06.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.023 7f30746ab700 1 -- 172.21.2.101:0/3472414636 wait complete. 2021-02-10T11:37:06.037 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:37:06.037 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:37:06.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.442 7ffac7a6e700 1 Processor -- start 2021-02-10T11:37:06.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.442 7ffac7a6e700 1 -- start start 2021-02-10T11:37:06.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.442 7ffac7a6e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ffac00d1870 0x7ffac01b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:06.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.442 7ffac7a6e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ffac01b0a10 0x7ffac00cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:06.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.442 7ffac7a6e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffac00f0b10 con 0x7ffac00d1c70 2021-02-10T11:37:06.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.442 7ffac7a6e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac00e5ed0 con 0x7ffac01b0a10 2021-02-10T11:37:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.442 7ffac7a6e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac00f0650 con 0x7ffac00d1870 2021-02-10T11:37:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.442 7ffac5089700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ffac01b0a10 0x7ffac00cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.442 7ffac5089700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ffac01b0a10 0x7ffac00cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54562/0 (socket says 172.21.2.101:54562) 2021-02-10T11:37:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.442 7ffac5089700 1 -- 172.21.2.101:0/3938349652 learned_addr learned my addr 172.21.2.101:0/3938349652 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.443 7ffac5089700 1 -- 172.21.2.101:0/3938349652 >> v2:172.21.2.108:3300/0 conn(0x7ffac00d1870 msgr2=0x7ffac01b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:37:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.443 7ffac5089700 1 --2- 172.21.2.101:0/3938349652 >> v2:172.21.2.108:3300/0 conn(0x7ffac00d1870 0x7ffac01b03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.443 7ffac5089700 1 -- 172.21.2.101:0/3938349652 >> v1:172.21.2.101:6789/0 conn(0x7ffac00d1c70 legacy=0x7ffac00cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.443 7ffac5089700 1 -- 172.21.2.101:0/3938349652 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac00f0840 con 0x7ffac01b0a10 2021-02-10T11:37:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.443 7ffac4888700 1 -- 172.21.2.101:0/3938349652 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4232621038 0 0) 0x7ffab8000f90 con 0x7ffac00d1c70 2021-02-10T11:37:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.443 7ffac5089700 1 --2- 172.21.2.101:0/3938349652 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffac01b0a10 0x7ffac00cf330 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 rx=0x7ffabc006700 tx=0x7ffabc009b00).ready entity=mon.0 client_cookie=c24c81d4d0909cb7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.443 7ffac4888700 1 -- 172.21.2.101:0/3938349652 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffabc00c0c0 con 0x7ffac01b0a10 2021-02-10T11:37:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.443 7ffac4888700 1 -- 172.21.2.101:0/3938349652 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffabc00c340 con 0x7ffac01b0a10 2021-02-10T11:37:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.443 7ffac4888700 1 -- 172.21.2.101:0/3938349652 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffabc00c820 con 0x7ffac01b0a10 2021-02-10T11:37:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.443 7ffac7a6e700 1 -- 172.21.2.101:0/3938349652 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffac01b0a10 msgr2=0x7ffac00cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:06.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.443 7ffac7a6e700 1 --2- 172.21.2.101:0/3938349652 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffac01b0a10 0x7ffac00cf330 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 rx=0x7ffabc006700 tx=0x7ffabc009b00).stop 2021-02-10T11:37:06.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.444 7ffac7a6e700 1 -- 172.21.2.101:0/3938349652 shutdown_connections 2021-02-10T11:37:06.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.444 7ffac7a6e700 1 --2- 172.21.2.101:0/3938349652 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffac01b0a10 0x7ffac00cf330 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:06.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.444 7ffac7a6e700 1 --2- 172.21.2.101:0/3938349652 >> v2:172.21.2.108:3300/0 conn(0x7ffac00d1870 0x7ffac01b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:06.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.444 7ffac7a6e700 1 -- 172.21.2.101:0/3938349652 >> 172.21.2.101:0/3938349652 conn(0x7ffac00ce450 msgr2=0x7ffac00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:06.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.444 7ffac7a6e700 1 -- 172.21.2.101:0/3938349652 shutdown_connections 2021-02-10T11:37:06.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.444 7ffac7a6e700 1 -- 172.21.2.101:0/3938349652 wait complete. 2021-02-10T11:37:06.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.459 7ffac7a6e700 1 Processor -- start 2021-02-10T11:37:06.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.459 7ffac7a6e700 1 -- start start 2021-02-10T11:37:06.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.459 7ffac7a6e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffac01b8ad0 0x7ffac0187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:06.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.459 7ffac7a6e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffac01b8ed0 0x7ffac00b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:06.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.459 7ffac7a6e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffac00b2110 0x7ffac01b7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:06.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.459 7ffac7a6e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac0188470 con 0x7ffac01b8ed0 2021-02-10T11:37:06.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.459 7ffac7a6e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac018be10 con 0x7ffac01b8ad0 2021-02-10T11:37:06.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.459 7ffac7a6e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac018c030 con 0x7ffac00b2110 2021-02-10T11:37:06.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.459 7ffac5089700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffac01b8ed0 0x7ffac00b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:06.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.459 7ffac5089700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffac01b8ed0 0x7ffac00b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54566/0 (socket says 172.21.2.101:54566) 2021-02-10T11:37:06.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.459 7ffac5089700 1 -- 172.21.2.101:0/3540971025 learned_addr learned my addr 172.21.2.101:0/3540971025 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:06.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.460 7ffac5089700 1 -- 172.21.2.101:0/3540971025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffac00b2110 msgr2=0x7ffac01b7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:06.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.460 7ffac5089700 1 --2- 172.21.2.101:0/3540971025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffac00b2110 0x7ffac01b7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:06.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.460 7ffac5089700 1 -- 172.21.2.101:0/3540971025 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffac01b8ad0 msgr2=0x7ffac0187e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:06.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.460 7ffac5089700 1 --2- 172.21.2.101:0/3540971025 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffac01b8ad0 0x7ffac0187e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:06.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.460 7ffac5089700 1 -- 172.21.2.101:0/3540971025 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac00b2510 con 0x7ffac01b8ed0 2021-02-10T11:37:06.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.460 7ffac5089700 1 --2- 172.21.2.101:0/3540971025 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffac01b8ed0 0x7ffac00b1af0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 rx=0x7ffabc006f20 tx=0x7ffabc00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:06.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.460 7ffab27fc700 1 -- 172.21.2.101:0/3540971025 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffabc000c90 con 0x7ffac01b8ed0 2021-02-10T11:37:06.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.460 7ffab27fc700 1 -- 172.21.2.101:0/3540971025 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffabc009100 con 0x7ffac01b8ed0 2021-02-10T11:37:06.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.460 7ffab27fc700 1 -- 172.21.2.101:0/3540971025 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffabc002220 con 0x7ffac01b8ed0 2021-02-10T11:37:06.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.460 7ffac7a6e700 1 -- 172.21.2.101:0/3540971025 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac00b2510 con 0x7ffac01b8ed0 2021-02-10T11:37:06.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.460 7ffac7a6e700 1 -- 172.21.2.101:0/3540971025 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac00b2510 con 0x7ffac01b8ed0 2021-02-10T11:37:06.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.461 7ffab27fc700 1 -- 172.21.2.101:0/3540971025 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ffabc00a000 con 0x7ffac01b8ed0 2021-02-10T11:37:06.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.461 7ffab27fc700 1 --2- 172.21.2.101:0/3540971025 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffaa80186c0 0x7ffaa801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:06.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.461 7ffab27fc700 1 -- 172.21.2.101:0/3540971025 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ffabc0325b0 con 0x7ffac01b8ed0 2021-02-10T11:37:06.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.461 7ffac588a700 1 --2- 172.21.2.101:0/3540971025 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffaa80186c0 0x7ffaa801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:06.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.461 7ffac588a700 1 --2- 172.21.2.101:0/3540971025 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffaa80186c0 0x7ffaa801abc0 crc :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:06.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.492 7ffac7a6e700 1 -- 172.21.2.101:0/3540971025 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac00ebc00 con 0x7ffac01b8ed0 2021-02-10T11:37:06.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.495 7ffab27fc700 1 -- 172.21.2.101:0/3540971025 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ffabc030d60 con 0x7ffac01b8ed0 2021-02-10T11:37:06.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.622 7ffac7a6e700 1 -- 172.21.2.101:0/3540971025 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffac00f0b10 con 0x7ffac01b8ed0 2021-02-10T11:37:06.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.622 7ffab27fc700 1 -- 172.21.2.101:0/3540971025 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+2101 (secure 0 0 0) 0x7ffabc00a000 con 0x7ffac01b8ed0 2021-02-10T11:37:06.629 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 62 2021-02-10T11:37:06.629 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:37:06.629 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9360,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2529021396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2529021396},{"type":"v1","addr":"172.21.2.101:6835","nonce":2529021396}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:56.648753","modified":"2021-02-10 11:36:58.653009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6005":{"gid":6005,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2363068042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2363068042},{"type":"v1","addr":"172.21.2.108:6833","nonce":2363068042}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-02-10T11:37:06.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.624 7ffac7a6e700 1 -- 172.21.2.101:0/3540971025 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffaa80186c0 msgr2=0x7ffaa801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:06.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.624 7ffac7a6e700 1 --2- 172.21.2.101:0/3540971025 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffaa80186c0 0x7ffaa801abc0 crc :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:06.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.624 7ffac7a6e700 1 -- 172.21.2.101:0/3540971025 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffac01b8ed0 msgr2=0x7ffac00b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:06.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.624 7ffac7a6e700 1 --2- 172.21.2.101:0/3540971025 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffac01b8ed0 0x7ffac00b1af0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 rx=0x7ffabc006f20 tx=0x7ffabc00b990).stop 2021-02-10T11:37:06.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.625 7ffac7a6e700 1 -- 172.21.2.101:0/3540971025 shutdown_connections 2021-02-10T11:37:06.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.625 7ffac7a6e700 1 --2- 172.21.2.101:0/3540971025 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffaa80186c0 0x7ffaa801abc0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:06.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.625 7ffac7a6e700 1 --2- 172.21.2.101:0/3540971025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffac00b2110 0x7ffac01b7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:06.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.625 7ffac7a6e700 1 --2- 172.21.2.101:0/3540971025 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffac01b8ed0 0x7ffac00b1af0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:06.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.625 7ffac7a6e700 1 --2- 172.21.2.101:0/3540971025 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffac01b8ad0 0x7ffac0187e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:06.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.625 7ffac7a6e700 1 -- 172.21.2.101:0/3540971025 >> 172.21.2.101:0/3540971025 conn(0x7ffac00ed810 msgr2=0x7ffac0094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:06.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.625 7ffac7a6e700 1 -- 172.21.2.101:0/3540971025 shutdown_connections 2021-02-10T11:37:06.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:06.625 7ffac7a6e700 1 -- 172.21.2.101:0/3540971025 wait complete. 2021-02-10T11:37:06.840 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:36:56.648753', 'modified': '2021-02-10 11:36:58.653009', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6005}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6005': {'gid': 6005, 'name': 'b', 'rank': 0, 'incarnation': 60, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/2363068042', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 2363068042}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 2363068042}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:37:06.840 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:37:06.840 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:37:07.241 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:37:07.241 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:37:07.241 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:37:07.241 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 6005, 2021-02-10T11:37:07.241 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:37:07.241 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:37:07.241 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 10.584530088999999, 2021-02-10T11:37:07.241 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 62, 2021-02-10T11:37:07.241 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 85, 2021-02-10T11:37:07.242 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 84, 2021-02-10T11:37:07.242 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 16.225375648 2021-02-10T11:37:07.242 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:37:07.245 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 6005, "want_state": "up:active", "state": "up:active", "rank_uptime": 10.584530088999999, "mdsmap_epoch": 62, "osdmap_epoch": 85, "osdmap_epoch_barrier": 84, "uptime": 16.225375648 } 2021-02-10T11:37:07.245 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:37:07.245 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-02-10T11:37:07.245 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-02-10T11:37:07.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:37:07.246 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:37:07.580 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:37:07.924 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:37:07.924 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:37:08.267 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:37:08.267 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:37:08.268 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:37:08.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:37:08.268 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:37:08.602 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-02-10T11:37:09.396 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:37:09.400 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:37:09.745 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:37:09.745 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:37:09.811 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[134576]: starting ceph client 2021-02-10T11:37:09.811 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:2021-02-10 11:37:09.806 7ffae85bff80 -1 init, newargv = 0x55e5dcf63190 newargc=9 2021-02-10T11:37:09.817 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[134576]: starting fuse 2021-02-10T11:37:10.087 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:37:10.088 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:37:10.088 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:37:10.089 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:37:10.089 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:37:10.423 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:37:10.423 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2021-02-10T11:37:10.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:37:10.424 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:37:10.424 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:37:10.424 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:37:10.424 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:37:10.424 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:37:10.424 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:37:10.424 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:37:10.424 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-02-10T11:37:10.424 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:37:10.424 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:37:10.424 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:37:10.424 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:37:10.424 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:37:10.424 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:37:10.425 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:37:10.425 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:37:10.425 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:37:10.425 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:37:10.425 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:37:10.425 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:37:10.425 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:37:10.425 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.0")) 2021-02-10T11:37:10.425 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:37:10.785 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.0.134576.asok 2021-02-10T11:37:10.788 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.134576.asok 2021-02-10T11:37:10.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:37:10.788 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.134576.asok status 2021-02-10T11:37:11.178 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:37:11.178 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:37:11.178 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:37:11.178 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:37:11.178 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "0", 2021-02-10T11:37:11.178 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:37:11.178 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-02-10T11:37:11.178 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "134576", 2021-02-10T11:37:11.178 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:37:11.178 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:37:11.179 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:37:11.179 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:37:11.179 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 9522, 2021-02-10T11:37:11.179 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:37:11.179 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:37:11.179 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:37:11.179 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 9522 2021-02-10T11:37:11.179 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:37:11.179 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:37:11.179 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:37:11.179 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:37:11.179 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1691593197 2021-02-10T11:37:11.180 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:37:11.180 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:37:11.180 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:37:11.180 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:37:11.180 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:37:11.180 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1691593197 2021-02-10T11:37:11.180 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:37:11.180 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.9522 172.21.2.101:0/1691593197", 2021-02-10T11:37:11.180 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/1691593197", 2021-02-10T11:37:11.180 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:37:11.180 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 62, 2021-02-10T11:37:11.180 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 85, 2021-02-10T11:37:11.180 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:37:11.180 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:37:11.181 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:37:11.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:37:11.182 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:37:11.182 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:37:11.516 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:37:11.516 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:37:11.517 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:37:11.517 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:37:11.863 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:37:12.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.268 7f63fd7ac700 1 Processor -- start 2021-02-10T11:37:12.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.268 7f63fd7ac700 1 -- start start 2021-02-10T11:37:12.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.268 7f63fd7ac700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f63f80d0340 0x7f63f80ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:12.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.268 7f63fd7ac700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63f80d1c70 0x7f63f80efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:12.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.268 7f63fd7ac700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f80f0b10 con 0x7f63f80d1870 2021-02-10T11:37:12.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.268 7f63fd7ac700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63f80e5ed0 con 0x7f63f80d0340 2021-02-10T11:37:12.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.268 7f63fd7ac700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f81b0be0 con 0x7f63f80d1c70 2021-02-10T11:37:12.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.270 7f63f7d75700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63f80d1c70 0x7f63f80efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:12.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.270 7f63f6d73700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f63f80d0340 0x7f63f80ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:12.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.270 7f63f7d75700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63f80d1c70 0x7f63f80efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:47324/0 (socket says 172.21.2.101:47324) 2021-02-10T11:37:12.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.270 7f63f6d73700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f63f80d0340 0x7f63f80ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:36842/0 (socket says 172.21.2.101:36842) 2021-02-10T11:37:12.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.270 7f63f7d75700 1 -- 172.21.2.101:0/1921879885 learned_addr learned my addr 172.21.2.101:0/1921879885 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:12.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.270 7f63f7574700 1 --1- 172.21.2.101:0/1921879885 >> v1:172.21.2.108:6789/0 conn(0x7f63f80d1870 0x7f63f81b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:48492/0 (socket says 172.21.2.101:48492) 2021-02-10T11:37:12.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.270 7f63f7d75700 1 -- 172.21.2.101:0/1921879885 >> v2:172.21.2.108:3300/0 conn(0x7f63f80d0340 msgr2=0x7f63f80ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:12.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.270 7f63f7d75700 1 --2- 172.21.2.101:0/1921879885 >> v2:172.21.2.108:3300/0 conn(0x7f63f80d0340 0x7f63f80ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:12.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.270 7f63f7d75700 1 -- 172.21.2.101:0/1921879885 >> v1:172.21.2.108:6789/0 conn(0x7f63f80d1870 legacy=0x7f63f81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:12.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.270 7f63f7d75700 1 -- 172.21.2.101:0/1921879885 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f80f20a0 con 0x7f63f80d1c70 2021-02-10T11:37:12.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.270 7f63f6d73700 1 --2- 172.21.2.101:0/1921879885 >> v2:172.21.2.108:3300/0 conn(0x7f63f80d0340 0x7f63f80ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:37:12.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.270 7f63f7d75700 1 --2- 172.21.2.101:0/1921879885 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63f80d1c70 0x7f63f80efc60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f63f0006aa0 tx=0x7f63f0009f20).ready entity=mon.2 client_cookie=942d0919e0af530d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:12.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.271 7f63f6572700 1 -- 172.21.2.101:0/1921879885 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f63f000b5c0 con 0x7f63f80d1c70 2021-02-10T11:37:12.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.271 7f63f6572700 1 -- 172.21.2.101:0/1921879885 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f0003030 con 0x7f63f80d1c70 2021-02-10T11:37:12.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.271 7f63f6572700 1 -- 172.21.2.101:0/1921879885 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f63f000b9f0 con 0x7f63f80d1c70 2021-02-10T11:37:12.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.271 7f63fd7ac700 1 -- 172.21.2.101:0/1921879885 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63f80d1c70 msgr2=0x7f63f80efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:12.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.271 7f63fd7ac700 1 --2- 172.21.2.101:0/1921879885 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63f80d1c70 0x7f63f80efc60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f63f0006aa0 tx=0x7f63f0009f20).stop 2021-02-10T11:37:12.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.272 7f63fd7ac700 1 -- 172.21.2.101:0/1921879885 shutdown_connections 2021-02-10T11:37:12.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.272 7f63fd7ac700 1 --2- 172.21.2.101:0/1921879885 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63f80d1c70 0x7f63f80efc60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:12.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.272 7f63fd7ac700 1 --2- 172.21.2.101:0/1921879885 >> v2:172.21.2.108:3300/0 conn(0x7f63f80d0340 0x7f63f80ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:12.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.272 7f63fd7ac700 1 -- 172.21.2.101:0/1921879885 >> 172.21.2.101:0/1921879885 conn(0x7f63f80ce450 msgr2=0x7f63f80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:12.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.272 7f63fd7ac700 1 -- 172.21.2.101:0/1921879885 shutdown_connections 2021-02-10T11:37:12.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.272 7f63fd7ac700 1 -- 172.21.2.101:0/1921879885 wait complete. 2021-02-10T11:37:12.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.287 7f63fd7ac700 1 Processor -- start 2021-02-10T11:37:12.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.287 7f63fd7ac700 1 -- start start 2021-02-10T11:37:12.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.287 7f63fd7ac700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63f8195650 0x7f63f80b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:12.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.287 7f63fd7ac700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63f80b5b80 0x7f63f81a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:12.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.287 7f63fd7ac700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63f80acfa0 0x7f63f81a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:12.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.287 7f63fd7ac700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f80b6010 con 0x7f63f80b5b80 2021-02-10T11:37:12.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.287 7f63fd7ac700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f81a3a60 con 0x7f63f80acfa0 2021-02-10T11:37:12.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.287 7f63fd7ac700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f81a3c40 con 0x7f63f8195650 2021-02-10T11:37:12.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.287 7f63f6d73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63f80b5b80 0x7f63f81a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:12.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.287 7f63f6d73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63f80b5b80 0x7f63f81a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54594/0 (socket says 172.21.2.101:54594) 2021-02-10T11:37:12.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.287 7f63f6d73700 1 -- 172.21.2.101:0/937035495 learned_addr learned my addr 172.21.2.101:0/937035495 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:12.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.288 7f63f7574700 1 --2- 172.21.2.101:0/937035495 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63f8195650 0x7f63f80b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:12.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.288 7f63f7d75700 1 --2- 172.21.2.101:0/937035495 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63f80acfa0 0x7f63f81a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:12.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.288 7f63f6d73700 1 -- 172.21.2.101:0/937035495 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63f8195650 msgr2=0x7f63f80b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:12.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.288 7f63f6d73700 1 --2- 172.21.2.101:0/937035495 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63f8195650 0x7f63f80b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:12.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.288 7f63f6d73700 1 -- 172.21.2.101:0/937035495 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63f80acfa0 msgr2=0x7f63f81a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:12.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.288 7f63f6d73700 1 --2- 172.21.2.101:0/937035495 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63f80acfa0 0x7f63f81a3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:12.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.288 7f63f6d73700 1 -- 172.21.2.101:0/937035495 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f80bcae0 con 0x7f63f80b5b80 2021-02-10T11:37:12.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.288 7f63f7574700 1 --2- 172.21.2.101:0/937035495 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63f8195650 0x7f63f80b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:37:12.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.288 7f63f6d73700 1 --2- 172.21.2.101:0/937035495 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63f80b5b80 0x7f63f81a2e20 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 rx=0x7f63ec006f40 tx=0x7f63ec00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:12.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.288 7f63dffff700 1 -- 172.21.2.101:0/937035495 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f63ec00bd80 con 0x7f63f80b5b80 2021-02-10T11:37:12.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.288 7f63dffff700 1 -- 172.21.2.101:0/937035495 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ec002030 con 0x7f63f80b5b80 2021-02-10T11:37:12.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.288 7f63fd7ac700 1 -- 172.21.2.101:0/937035495 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f80bcae0 con 0x7f63f80b5b80 2021-02-10T11:37:12.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.288 7f63fd7ac700 1 -- 172.21.2.101:0/937035495 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f80bd070 con 0x7f63f80b5b80 2021-02-10T11:37:12.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.288 7f63dffff700 1 -- 172.21.2.101:0/937035495 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f63ec002480 con 0x7f63f80b5b80 2021-02-10T11:37:12.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.289 7f63dffff700 1 -- 172.21.2.101:0/937035495 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f63ec006b50 con 0x7f63f80b5b80 2021-02-10T11:37:12.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.289 7f63dffff700 1 --2- 172.21.2.101:0/937035495 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f63d80186c0 0x7f63d801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:12.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.289 7f63dffff700 1 -- 172.21.2.101:0/937035495 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f63ec0327b0 con 0x7f63f80b5b80 2021-02-10T11:37:12.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.289 7f63f7574700 1 --2- 172.21.2.101:0/937035495 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f63d80186c0 0x7f63d801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:12.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.289 7f63f7574700 1 --2- 172.21.2.101:0/937035495 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f63d80186c0 0x7f63d801abc0 crc :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:12.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.319 7f63fd7ac700 1 -- 172.21.2.101:0/937035495 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f80f0ab0 con 0x7f63f80b5b80 2021-02-10T11:37:12.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.321 7f63dffff700 1 -- 172.21.2.101:0/937035495 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f63ec000940 con 0x7f63f80b5b80 2021-02-10T11:37:12.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.447 7f63fd7ac700 1 -- 172.21.2.101:0/937035495 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63f80f0ab0 con 0x7f63f80b5b80 2021-02-10T11:37:12.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.448 7f63dffff700 1 -- 172.21.2.101:0/937035495 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+13996 (secure 0 0 0) 0x7f63ec006b50 con 0x7f63f80b5b80 2021-02-10T11:37:12.453 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:37:12.453 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":85,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:36:58.653077","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:36:54.618463","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2021-02-10 11:36:55.644563","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:37:12.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.449 7f63fd7ac700 1 -- 172.21.2.101:0/937035495 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f63d80186c0 msgr2=0x7f63d801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:12.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.449 7f63fd7ac700 1 --2- 172.21.2.101:0/937035495 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f63d80186c0 0x7f63d801abc0 crc :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:12.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.449 7f63fd7ac700 1 -- 172.21.2.101:0/937035495 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63f80b5b80 msgr2=0x7f63f81a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:12.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.449 7f63fd7ac700 1 --2- 172.21.2.101:0/937035495 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63f80b5b80 0x7f63f81a2e20 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 rx=0x7f63ec006f40 tx=0x7f63ec00b990).stop 2021-02-10T11:37:12.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.449 7f63fd7ac700 1 -- 172.21.2.101:0/937035495 shutdown_connections 2021-02-10T11:37:12.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.449 7f63fd7ac700 1 --2- 172.21.2.101:0/937035495 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63f80acfa0 0x7f63f81a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:12.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.449 7f63fd7ac700 1 --2- 172.21.2.101:0/937035495 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63f80b5b80 0x7f63f81a2e20 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:12.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.449 7f63fd7ac700 1 --2- 172.21.2.101:0/937035495 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f63d80186c0 0x7f63d801abc0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:12.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.449 7f63fd7ac700 1 --2- 172.21.2.101:0/937035495 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63f8195650 0x7f63f80b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:12.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.449 7f63fd7ac700 1 -- 172.21.2.101:0/937035495 >> 172.21.2.101:0/937035495 conn(0x7f63f80b3040 msgr2=0x7f63f80c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:12.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.450 7f63fd7ac700 1 -- 172.21.2.101:0/937035495 shutdown_connections 2021-02-10T11:37:12.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.450 7f63fd7ac700 1 -- 172.21.2.101:0/937035495 wait complete. 2021-02-10T11:37:12.463 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:36:54.618463', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:36:55.644563', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:37:12.463 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:37:12.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.874 7fb38f5ca700 1 Processor -- start 2021-02-10T11:37:12.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.874 7fb38f5ca700 1 -- start start 2021-02-10T11:37:12.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.874 7fb38f5ca700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb38810aff0 0x7fb3880cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:12.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.874 7fb38f5ca700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3880fe1c0 con 0x7fb3880d1870 2021-02-10T11:37:12.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.874 7fb38f5ca700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3881b0cb0 con 0x7fb3880d1c70 2021-02-10T11:37:12.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.874 7fb38f5ca700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb388109a60 con 0x7fb38810aff0 2021-02-10T11:37:12.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.875 7fb38d3e6700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb3880d1870 0x7fb3881b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55096/0 (socket says 172.21.2.101:55096) 2021-02-10T11:37:12.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.875 7fb38d3e6700 1 -- 172.21.2.101:0/1041961048 learned_addr learned my addr 172.21.2.101:0/1041961048 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:37:12.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.875 7fb37ffff700 1 -- 172.21.2.101:0/1041961048 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487665029 0 0) 0x7fb384001180 con 0x7fb3880d1870 2021-02-10T11:37:12.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.875 7fb37ffff700 1 -- 172.21.2.101:0/1041961048 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb374002fe0 con 0x7fb3880d1870 2021-02-10T11:37:12.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.875 7fb38dbe7700 1 --2- 172.21.2.101:0/1041961048 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb38810aff0 0x7fb3880cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:12.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.875 7fb37ffff700 1 -- 172.21.2.101:0/1041961048 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1227756325 0 0) 0x7fb384001180 con 0x7fb3880d1870 2021-02-10T11:37:12.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.875 7fb37ffff700 1 -- 172.21.2.101:0/1041961048 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb38810aff0 msgr2=0x7fb3880cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:12.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.875 7fb37ffff700 1 --2- 172.21.2.101:0/1041961048 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb38810aff0 0x7fb3880cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:12.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.875 7fb37ffff700 1 -- 172.21.2.101:0/1041961048 >> v1:172.21.2.108:6789/0 conn(0x7fb3880d1c70 legacy=0x7fb3880cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:12.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.875 7fb37ffff700 1 -- 172.21.2.101:0/1041961048 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38810b3f0 con 0x7fb3880d1870 2021-02-10T11:37:12.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.875 7fb37ffff700 1 -- 172.21.2.101:0/1041961048 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fb384001680 con 0x7fb3880d1870 2021-02-10T11:37:12.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.875 7fb37ffff700 1 -- 172.21.2.101:0/1041961048 >> v1:172.21.2.101:6789/0 conn(0x7fb3880d1870 legacy=0x7fb3881b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:12.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.875 7fb37ffff700 1 --2- 172.21.2.101:0/1041961048 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb374003a60 0x7fb374005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:12.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.875 7fb37ffff700 1 --2- 172.21.2.101:0/1041961048 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb374006790 0x7fb374008c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:12.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.875 7fb37ffff700 1 --2- 172.21.2.101:0/1041961048 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb374009500 0x7fb37400ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:12.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.875 7fb37ffff700 1 -- 172.21.2.101:0/1041961048 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37400c2d0 con 0x7fb374009500 2021-02-10T11:37:12.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.875 7fb37ffff700 1 -- 172.21.2.101:0/1041961048 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37400c4f0 con 0x7fb374003a60 2021-02-10T11:37:12.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.876 7fb37ffff700 1 -- 172.21.2.101:0/1041961048 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37400c710 con 0x7fb374006790 2021-02-10T11:37:12.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.876 7fb38dbe7700 1 --2- 172.21.2.101:0/1041961048 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb374009500 0x7fb37400ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:12.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.876 7fb38cbe5700 1 --2- 172.21.2.101:0/1041961048 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb374006790 0x7fb374008c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:12.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.876 7fb38d3e6700 1 --2- 172.21.2.101:0/1041961048 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb374003a60 0x7fb374005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:12.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.876 7fb38dbe7700 1 -- 172.21.2.101:0/1041961048 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb374006790 msgr2=0x7fb374008c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:12.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.876 7fb38dbe7700 1 --2- 172.21.2.101:0/1041961048 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb374006790 0x7fb374008c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:12.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.876 7fb38dbe7700 1 -- 172.21.2.101:0/1041961048 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb374003a60 msgr2=0x7fb374005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:12.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.876 7fb38dbe7700 1 --2- 172.21.2.101:0/1041961048 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb374003a60 0x7fb374005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:12.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.876 7fb38dbe7700 1 -- 172.21.2.101:0/1041961048 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37400c9b0 con 0x7fb374009500 2021-02-10T11:37:12.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.876 7fb38dbe7700 1 --2- 172.21.2.101:0/1041961048 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb374009500 0x7fb37400ba20 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 rx=0x7fb38004ae60 tx=0x7fb38004de00).ready entity=mon.0 client_cookie=630c48e8eaddba8c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:12.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.876 7fb37ffff700 1 -- 172.21.2.101:0/1041961048 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb38004aa30 con 0x7fb374009500 2021-02-10T11:37:12.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.876 7fb37ffff700 1 -- 172.21.2.101:0/1041961048 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38004f300 con 0x7fb374009500 2021-02-10T11:37:12.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.876 7fb37ffff700 1 -- 172.21.2.101:0/1041961048 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb38004f760 con 0x7fb374009500 2021-02-10T11:37:12.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.876 7fb38f5ca700 1 -- 172.21.2.101:0/1041961048 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb374009500 msgr2=0x7fb37400ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:12.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.876 7fb38f5ca700 1 --2- 172.21.2.101:0/1041961048 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb374009500 0x7fb37400ba20 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 rx=0x7fb38004ae60 tx=0x7fb38004de00).stop 2021-02-10T11:37:12.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.876 7fb38dbe7700 1 -- 172.21.2.101:0/1041961048 reap_dead start 2021-02-10T11:37:12.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.877 7fb38f5ca700 1 -- 172.21.2.101:0/1041961048 shutdown_connections 2021-02-10T11:37:12.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.877 7fb38f5ca700 1 -- 172.21.2.101:0/1041961048 >> 172.21.2.101:0/1041961048 conn(0x7fb3880ce420 msgr2=0x7fb3880d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:12.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.877 7fb38f5ca700 1 -- 172.21.2.101:0/1041961048 shutdown_connections 2021-02-10T11:37:12.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.877 7fb38f5ca700 1 -- 172.21.2.101:0/1041961048 wait complete. 2021-02-10T11:37:12.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.892 7fb38f5ca700 1 Processor -- start 2021-02-10T11:37:12.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.892 7fb38f5ca700 1 -- start start 2021-02-10T11:37:12.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.892 7fb38f5ca700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb388083c70 0x7fb388084070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:12.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.892 7fb38f5ca700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb3880c3050 0x7fb3880c3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:12.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.892 7fb38f5ca700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3880c3a70 0x7fb3880ac460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:12.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.892 7fb38f5ca700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880abe50 con 0x7fb3880c3a70 2021-02-10T11:37:12.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.892 7fb38f5ca700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880c3e70 con 0x7fb388083c70 2021-02-10T11:37:12.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.892 7fb38f5ca700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880aca80 con 0x7fb3880c3050 2021-02-10T11:37:12.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.893 7fb38dbe7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3880c3a70 0x7fb3880ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:12.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.893 7fb38dbe7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3880c3a70 0x7fb3880ac460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54614/0 (socket says 172.21.2.101:54614) 2021-02-10T11:37:12.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.893 7fb38dbe7700 1 -- 172.21.2.101:0/1836275804 learned_addr learned my addr 172.21.2.101:0/1836275804 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:12.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.893 7fb38d3e6700 1 --2- 172.21.2.101:0/1836275804 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb388083c70 0x7fb388084070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:12.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.893 7fb38cbe5700 1 --2- 172.21.2.101:0/1836275804 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb3880c3050 0x7fb3880c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:12.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.893 7fb38dbe7700 1 -- 172.21.2.101:0/1836275804 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb3880c3050 msgr2=0x7fb3880c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:12.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.893 7fb38dbe7700 1 --2- 172.21.2.101:0/1836275804 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb3880c3050 0x7fb3880c3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:12.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.893 7fb38dbe7700 1 -- 172.21.2.101:0/1836275804 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb388083c70 msgr2=0x7fb388084070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:12.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.893 7fb38dbe7700 1 --2- 172.21.2.101:0/1836275804 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb388083c70 0x7fb388084070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:12.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.893 7fb38dbe7700 1 -- 172.21.2.101:0/1836275804 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3880acca0 con 0x7fb3880c3a70 2021-02-10T11:37:12.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.893 7fb38dbe7700 1 --2- 172.21.2.101:0/1836275804 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3880c3a70 0x7fb3880ac460 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 rx=0x7fb38004d880 tx=0x7fb38004fd20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:12.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.893 7fb37dffb700 1 -- 172.21.2.101:0/1836275804 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb38004aa20 con 0x7fb3880c3a70 2021-02-10T11:37:12.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.893 7fb37dffb700 1 -- 172.21.2.101:0/1836275804 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38004fe60 con 0x7fb3880c3a70 2021-02-10T11:37:12.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.893 7fb37dffb700 1 -- 172.21.2.101:0/1836275804 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb38004b500 con 0x7fb3880c3a70 2021-02-10T11:37:12.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.893 7fb38f5ca700 1 -- 172.21.2.101:0/1836275804 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3880acca0 con 0x7fb3880c3a70 2021-02-10T11:37:12.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.893 7fb38f5ca700 1 -- 172.21.2.101:0/1836275804 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3880acca0 con 0x7fb3880c3a70 2021-02-10T11:37:12.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.894 7fb37dffb700 1 -- 172.21.2.101:0/1836275804 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb38004aa20 con 0x7fb3880c3a70 2021-02-10T11:37:12.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.894 7fb37dffb700 1 --2- 172.21.2.101:0/1836275804 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb368018730 0x7fb36801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:12.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.894 7fb37dffb700 1 -- 172.21.2.101:0/1836275804 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb380076b20 con 0x7fb3880c3a70 2021-02-10T11:37:12.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.894 7fb38d3e6700 1 --2- 172.21.2.101:0/1836275804 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb368018730 0x7fb36801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:12.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.894 7fb38d3e6700 1 --2- 172.21.2.101:0/1836275804 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb368018730 0x7fb36801ac10 crc :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:12.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.925 7fb38f5ca700 1 -- 172.21.2.101:0/1836275804 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb388082a70 con 0x7fb3880c3a70 2021-02-10T11:37:12.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:12.927 7fb37dffb700 1 -- 172.21.2.101:0/1836275804 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb380076d30 con 0x7fb3880c3a70 2021-02-10T11:37:13.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.056 7fb38f5ca700 1 -- 172.21.2.101:0/1836275804 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fb3880abdd0 con 0x7fb3880c3a70 2021-02-10T11:37:13.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.056 7fb37dffb700 1 -- 172.21.2.101:0/1836275804 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v85) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb380063840 con 0x7fb3880c3a70 2021-02-10T11:37:13.061 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:37:13.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.058 7fb38f5ca700 1 -- 172.21.2.101:0/1836275804 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb368018730 msgr2=0x7fb36801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:13.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.058 7fb38f5ca700 1 --2- 172.21.2.101:0/1836275804 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb368018730 0x7fb36801ac10 crc :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:13.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.058 7fb38f5ca700 1 -- 172.21.2.101:0/1836275804 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3880c3a70 msgr2=0x7fb3880ac460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:13.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.058 7fb38f5ca700 1 --2- 172.21.2.101:0/1836275804 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3880c3a70 0x7fb3880ac460 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 rx=0x7fb38004d880 tx=0x7fb38004fd20).stop 2021-02-10T11:37:13.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.058 7fb38f5ca700 1 -- 172.21.2.101:0/1836275804 shutdown_connections 2021-02-10T11:37:13.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.058 7fb38f5ca700 1 --2- 172.21.2.101:0/1836275804 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb368018730 0x7fb36801ac10 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:13.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.058 7fb38f5ca700 1 --2- 172.21.2.101:0/1836275804 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb3880c3050 0x7fb3880c3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:13.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.058 7fb38f5ca700 1 --2- 172.21.2.101:0/1836275804 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3880c3a70 0x7fb3880ac460 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:13.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.058 7fb38f5ca700 1 --2- 172.21.2.101:0/1836275804 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb388083c70 0x7fb388084070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:13.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.058 7fb38f5ca700 1 -- 172.21.2.101:0/1836275804 >> 172.21.2.101:0/1836275804 conn(0x7fb38809ffd0 msgr2=0x7fb388082ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:13.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.059 7fb38f5ca700 1 -- 172.21.2.101:0/1836275804 shutdown_connections 2021-02-10T11:37:13.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.059 7fb38f5ca700 1 -- 172.21.2.101:0/1836275804 wait complete. 2021-02-10T11:37:13.072 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:37:13.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.482 7f6d79a04700 1 Processor -- start 2021-02-10T11:37:13.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.482 7f6d79a04700 1 -- start start 2021-02-10T11:37:13.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.482 7f6d79a04700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6d740d1870 0x7f6d741b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:13.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.482 7f6d79a04700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6d740cff70 0x7f6d74109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:13.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.482 7f6d79a04700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d74109a60 con 0x7f6d741b0ae0 2021-02-10T11:37:13.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.482 7f6d79a04700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d74109750 con 0x7f6d740d1870 2021-02-10T11:37:13.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.482 7f6d79a04700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d74088070 con 0x7f6d740cff70 2021-02-10T11:37:13.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.483 7f6d73fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6d740cff70 0x7f6d74109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:13.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.484 7f6d73fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6d740cff70 0x7f6d74109130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:47354/0 (socket says 172.21.2.101:47354) 2021-02-10T11:37:13.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.484 7f6d73fff700 1 -- 172.21.2.101:0/2816048426 learned_addr learned my addr 172.21.2.101:0/2816048426 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:13.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.484 7f6d737fe700 1 --2- 172.21.2.101:0/2816048426 >> v2:172.21.2.108:3300/0 conn(0x7f6d740d1870 0x7f6d741b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:13.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.484 7f6d73fff700 1 -- 172.21.2.101:0/2816048426 >> v2:172.21.2.108:3300/0 conn(0x7f6d740d1870 msgr2=0x7f6d741b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:13.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.484 7f6d73fff700 1 --2- 172.21.2.101:0/2816048426 >> v2:172.21.2.108:3300/0 conn(0x7f6d740d1870 0x7f6d741b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:13.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.484 7f6d73fff700 1 -- 172.21.2.101:0/2816048426 >> v1:172.21.2.108:6789/0 conn(0x7f6d741b0ae0 legacy=0x7f6d74108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:13.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.484 7f6d73fff700 1 -- 172.21.2.101:0/2816048426 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d740d1c70 con 0x7f6d740cff70 2021-02-10T11:37:13.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.484 7f6d737fe700 1 --2- 172.21.2.101:0/2816048426 >> v2:172.21.2.108:3300/0 conn(0x7f6d740d1870 0x7f6d741b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:37:13.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.484 7f6d73fff700 1 --2- 172.21.2.101:0/2816048426 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6d740cff70 0x7f6d74109130 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f6d6c006aa0 tx=0x7f6d6c009f20).ready entity=mon.2 client_cookie=b4de1d5ef13ee12d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:13.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.485 7f6d727fc700 1 -- 172.21.2.101:0/2816048426 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6d6c00b5c0 con 0x7f6d740cff70 2021-02-10T11:37:13.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.485 7f6d727fc700 1 -- 172.21.2.101:0/2816048426 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6c003030 con 0x7f6d740cff70 2021-02-10T11:37:13.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.485 7f6d727fc700 1 -- 172.21.2.101:0/2816048426 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6d6c00bab0 con 0x7f6d740cff70 2021-02-10T11:37:13.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.485 7f6d79a04700 1 -- 172.21.2.101:0/2816048426 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6d740cff70 msgr2=0x7f6d74109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:13.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.485 7f6d79a04700 1 --2- 172.21.2.101:0/2816048426 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6d740cff70 0x7f6d74109130 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f6d6c006aa0 tx=0x7f6d6c009f20).stop 2021-02-10T11:37:13.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.485 7f6d79a04700 1 -- 172.21.2.101:0/2816048426 shutdown_connections 2021-02-10T11:37:13.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.485 7f6d79a04700 1 --2- 172.21.2.101:0/2816048426 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6d740cff70 0x7f6d74109130 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:13.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.485 7f6d79a04700 1 --2- 172.21.2.101:0/2816048426 >> v2:172.21.2.108:3300/0 conn(0x7f6d740d1870 0x7f6d741b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:13.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.485 7f6d79a04700 1 -- 172.21.2.101:0/2816048426 >> 172.21.2.101:0/2816048426 conn(0x7f6d740ce420 msgr2=0x7f6d740d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:13.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.485 7f6d79a04700 1 -- 172.21.2.101:0/2816048426 shutdown_connections 2021-02-10T11:37:13.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.485 7f6d79a04700 1 -- 172.21.2.101:0/2816048426 wait complete. 2021-02-10T11:37:13.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.499 7f6d79a04700 1 Processor -- start 2021-02-10T11:37:13.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.500 7f6d79a04700 1 -- start start 2021-02-10T11:37:13.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.500 7f6d79a04700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6d74191b10 0x7f6d740ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:13.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.500 7f6d79a04700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6d74191f10 0x7f6d7419ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:13.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.500 7f6d79a04700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6d741a05f0 0x7f6d74082ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:13.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.500 7f6d79a04700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d74192440 con 0x7f6d74191b10 2021-02-10T11:37:13.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.500 7f6d79a04700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d741916e0 con 0x7f6d74191f10 2021-02-10T11:37:13.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.500 7f6d79a04700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d74191900 con 0x7f6d741a05f0 2021-02-10T11:37:13.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.500 7f6d737fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6d74191b10 0x7f6d740ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:13.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.500 7f6d737fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6d74191b10 0x7f6d740ae640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54626/0 (socket says 172.21.2.101:54626) 2021-02-10T11:37:13.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.500 7f6d737fe700 1 -- 172.21.2.101:0/3174985506 learned_addr learned my addr 172.21.2.101:0/3174985506 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:13.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.500 7f6d73fff700 1 --2- 172.21.2.101:0/3174985506 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6d741a05f0 0x7f6d74082ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:13.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.500 7f6d72ffd700 1 --2- 172.21.2.101:0/3174985506 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6d74191f10 0x7f6d7419ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:13.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.501 7f6d73fff700 1 -- 172.21.2.101:0/3174985506 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6d74191f10 msgr2=0x7f6d7419ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:13.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.501 7f6d73fff700 1 --2- 172.21.2.101:0/3174985506 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6d74191f10 0x7f6d7419ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:13.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.501 7f6d73fff700 1 -- 172.21.2.101:0/3174985506 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6d74191b10 msgr2=0x7f6d740ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:13.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.501 7f6d73fff700 1 --2- 172.21.2.101:0/3174985506 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6d74191b10 0x7f6d740ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:13.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.501 7f6d73fff700 1 -- 172.21.2.101:0/3174985506 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d740834f0 con 0x7f6d741a05f0 2021-02-10T11:37:13.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.501 7f6d73fff700 1 --2- 172.21.2.101:0/3174985506 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6d741a05f0 0x7f6d74082ed0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f6d6c009410 tx=0x7f6d6c00bc10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:13.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.501 7f6d5bfff700 1 -- 172.21.2.101:0/3174985506 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6d6c009450 con 0x7f6d741a05f0 2021-02-10T11:37:13.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.501 7f6d5bfff700 1 -- 172.21.2.101:0/3174985506 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6c002030 con 0x7f6d741a05f0 2021-02-10T11:37:13.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.501 7f6d5bfff700 1 -- 172.21.2.101:0/3174985506 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6d6c002480 con 0x7f6d741a05f0 2021-02-10T11:37:13.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.501 7f6d79a04700 1 -- 172.21.2.101:0/3174985506 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d740834f0 con 0x7f6d741a05f0 2021-02-10T11:37:13.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.501 7f6d79a04700 1 -- 172.21.2.101:0/3174985506 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d74083a80 con 0x7f6d741a05f0 2021-02-10T11:37:13.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.502 7f6d5bfff700 1 -- 172.21.2.101:0/3174985506 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6d6c002210 con 0x7f6d741a05f0 2021-02-10T11:37:13.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.502 7f6d5bfff700 1 --2- 172.21.2.101:0/3174985506 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6d540186c0 0x7f6d5401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:13.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.502 7f6d5bfff700 1 -- 172.21.2.101:0/3174985506 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6d6c032690 con 0x7f6d741a05f0 2021-02-10T11:37:13.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.502 7f6d737fe700 1 --2- 172.21.2.101:0/3174985506 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6d540186c0 0x7f6d5401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:13.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.502 7f6d737fe700 1 --2- 172.21.2.101:0/3174985506 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6d540186c0 0x7f6d5401abc0 crc :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:13.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.532 7f6d79a04700 1 -- 172.21.2.101:0/3174985506 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d740fe1c0 con 0x7f6d741a05f0 2021-02-10T11:37:13.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.536 7f6d5bfff700 1 -- 172.21.2.101:0/3174985506 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6d6c023400 con 0x7f6d741a05f0 2021-02-10T11:37:13.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.668 7f6d79a04700 1 -- 172.21.2.101:0/3174985506 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f6d740fe1c0 con 0x7f6d741a05f0 2021-02-10T11:37:13.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.669 7f6d5bfff700 1 -- 172.21.2.101:0/3174985506 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v85) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6d6c023400 con 0x7f6d741a05f0 2021-02-10T11:37:13.673 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:37:13.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.670 7f6d79a04700 1 -- 172.21.2.101:0/3174985506 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6d540186c0 msgr2=0x7f6d5401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:13.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.670 7f6d79a04700 1 --2- 172.21.2.101:0/3174985506 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6d540186c0 0x7f6d5401abc0 crc :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:13.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.670 7f6d79a04700 1 -- 172.21.2.101:0/3174985506 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6d741a05f0 msgr2=0x7f6d74082ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:13.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.670 7f6d79a04700 1 --2- 172.21.2.101:0/3174985506 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6d741a05f0 0x7f6d74082ed0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f6d6c009410 tx=0x7f6d6c00bc10).stop 2021-02-10T11:37:13.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.670 7f6d79a04700 1 -- 172.21.2.101:0/3174985506 shutdown_connections 2021-02-10T11:37:13.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.670 7f6d79a04700 1 --2- 172.21.2.101:0/3174985506 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6d540186c0 0x7f6d5401abc0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:13.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.670 7f6d79a04700 1 --2- 172.21.2.101:0/3174985506 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6d741a05f0 0x7f6d74082ed0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:13.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.670 7f6d79a04700 1 --2- 172.21.2.101:0/3174985506 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6d74191f10 0x7f6d7419ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:13.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.670 7f6d79a04700 1 --2- 172.21.2.101:0/3174985506 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6d74191b10 0x7f6d740ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:13.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.670 7f6d79a04700 1 -- 172.21.2.101:0/3174985506 >> 172.21.2.101:0/3174985506 conn(0x7f6d740a02d0 msgr2=0x7f6d7409f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:13.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.670 7f6d79a04700 1 -- 172.21.2.101:0/3174985506 shutdown_connections 2021-02-10T11:37:13.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:13.670 7f6d79a04700 1 -- 172.21.2.101:0/3174985506 wait complete. 2021-02-10T11:37:13.683 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:37:14.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.093 7fa25fb2a700 1 Processor -- start 2021-02-10T11:37:14.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.093 7fa25fb2a700 1 -- start start 2021-02-10T11:37:14.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.094 7fa25fb2a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa2580d1870 0x7fa2581b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:14.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.094 7fa25fb2a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2580fe1c0 con 0x7fa2581b0ae0 2021-02-10T11:37:14.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.094 7fa25fb2a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa258109a60 con 0x7fa2580cff70 2021-02-10T11:37:14.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.094 7fa25fb2a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa258088070 con 0x7fa2580d1870 2021-02-10T11:37:14.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.094 7fa25d946700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa2580d1870 0x7fa2581b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:14.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.094 7fa25d145700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa2581b0ae0 0x7fa258108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55126/0 (socket says 172.21.2.101:55126) 2021-02-10T11:37:14.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.094 7fa25d145700 1 -- 172.21.2.101:0/1764761535 learned_addr learned my addr 172.21.2.101:0/1764761535 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:37:14.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.094 7fa25d946700 1 -- 172.21.2.101:0/1764761535 >> v1:172.21.2.108:6789/0 conn(0x7fa2580cff70 legacy=0x7fa258109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:14.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.094 7fa25d946700 1 -- 172.21.2.101:0/1764761535 >> v1:172.21.2.101:6789/0 conn(0x7fa2581b0ae0 legacy=0x7fa258108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:14.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.094 7fa25d946700 1 -- 172.21.2.101:0/1764761535 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa258088260 con 0x7fa2580d1870 2021-02-10T11:37:14.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.095 7fa25d946700 1 --2- 172.21.2.101:0/1764761535 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2580d1870 0x7fa2581b04c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7fa254006650 tx=0x7fa254009a80).ready entity=mon.1 client_cookie=760edbac3ab686b5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:14.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.095 7fa25c944700 1 -- 172.21.2.101:0/1764761535 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa25400c0c0 con 0x7fa2580d1870 2021-02-10T11:37:14.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.095 7fa25c944700 1 -- 172.21.2.101:0/1764761535 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25400c340 con 0x7fa2580d1870 2021-02-10T11:37:14.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.095 7fa25c944700 1 -- 172.21.2.101:0/1764761535 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa25400c860 con 0x7fa2580d1870 2021-02-10T11:37:14.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.095 7fa25fb2a700 1 -- 172.21.2.101:0/1764761535 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2580d1870 msgr2=0x7fa2581b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:14.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.095 7fa25fb2a700 1 --2- 172.21.2.101:0/1764761535 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2580d1870 0x7fa2581b04c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7fa254006650 tx=0x7fa254009a80).stop 2021-02-10T11:37:14.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.095 7fa25fb2a700 1 -- 172.21.2.101:0/1764761535 shutdown_connections 2021-02-10T11:37:14.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.095 7fa25fb2a700 1 --2- 172.21.2.101:0/1764761535 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2580d1870 0x7fa2581b04c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:14.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.095 7fa25fb2a700 1 -- 172.21.2.101:0/1764761535 >> 172.21.2.101:0/1764761535 conn(0x7fa2580ce420 msgr2=0x7fa2580d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:14.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.096 7fa25fb2a700 1 -- 172.21.2.101:0/1764761535 shutdown_connections 2021-02-10T11:37:14.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.096 7fa25fb2a700 1 -- 172.21.2.101:0/1764761535 wait complete. 2021-02-10T11:37:14.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.110 7fa25fb2a700 1 Processor -- start 2021-02-10T11:37:14.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.110 7fa25fb2a700 1 -- start start 2021-02-10T11:37:14.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.111 7fa25fb2a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa25808cc50 0x7fa2580a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:14.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.111 7fa25fb2a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa25808d550 0x7fa25808d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:14.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.111 7fa25fb2a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa25808df70 0x7fa25808fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:14.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.111 7fa25fb2a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25808e370 con 0x7fa25808df70 2021-02-10T11:37:14.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.111 7fa25fb2a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2580a6c10 con 0x7fa25808d550 2021-02-10T11:37:14.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.111 7fa25fb2a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25808d050 con 0x7fa25808cc50 2021-02-10T11:37:14.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.111 7fa25d946700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa25808cc50 0x7fa2580a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:14.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.111 7fa25d946700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa25808cc50 0x7fa2580a65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:47368/0 (socket says 172.21.2.101:47368) 2021-02-10T11:37:14.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.111 7fa25d946700 1 -- 172.21.2.101:0/1540700283 learned_addr learned my addr 172.21.2.101:0/1540700283 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:14.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.111 7fa25e147700 1 --2- 172.21.2.101:0/1540700283 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa25808df70 0x7fa25808fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:14.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.111 7fa25d145700 1 --2- 172.21.2.101:0/1540700283 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa25808d550 0x7fa25808d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:14.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.111 7fa25d946700 1 -- 172.21.2.101:0/1540700283 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa25808d550 msgr2=0x7fa25808d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:14.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.111 7fa25d946700 1 --2- 172.21.2.101:0/1540700283 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa25808d550 0x7fa25808d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:14.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.111 7fa25d946700 1 -- 172.21.2.101:0/1540700283 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa25808df70 msgr2=0x7fa25808fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:14.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.111 7fa25d946700 1 --2- 172.21.2.101:0/1540700283 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa25808df70 0x7fa25808fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:14.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.111 7fa25d946700 1 -- 172.21.2.101:0/1540700283 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa258090090 con 0x7fa25808cc50 2021-02-10T11:37:14.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.111 7fa25e147700 1 --2- 172.21.2.101:0/1540700283 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa25808df70 0x7fa25808fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:37:14.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.111 7fa25d946700 1 --2- 172.21.2.101:0/1540700283 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa25808cc50 0x7fa2580a65f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fa254009630 tx=0x7fa25400b970).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:14.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.112 7fa24e7fc700 1 -- 172.21.2.101:0/1540700283 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa25400bd60 con 0x7fa25808cc50 2021-02-10T11:37:14.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.112 7fa24e7fc700 1 -- 172.21.2.101:0/1540700283 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa254006740 con 0x7fa25808cc50 2021-02-10T11:37:14.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.112 7fa25fb2a700 1 -- 172.21.2.101:0/1540700283 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa258090090 con 0x7fa25808cc50 2021-02-10T11:37:14.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.112 7fa24e7fc700 1 -- 172.21.2.101:0/1540700283 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa254002270 con 0x7fa25808cc50 2021-02-10T11:37:14.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.112 7fa25fb2a700 1 -- 172.21.2.101:0/1540700283 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa258090090 con 0x7fa25808cc50 2021-02-10T11:37:14.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.112 7fa24e7fc700 1 -- 172.21.2.101:0/1540700283 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa254000c90 con 0x7fa25808cc50 2021-02-10T11:37:14.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.112 7fa24e7fc700 1 --2- 172.21.2.101:0/1540700283 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2400186d0 0x7fa24001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:14.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.112 7fa24e7fc700 1 -- 172.21.2.101:0/1540700283 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa2540326a0 con 0x7fa25808cc50 2021-02-10T11:37:14.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.112 7fa25d145700 1 --2- 172.21.2.101:0/1540700283 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2400186d0 0x7fa24001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:14.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.113 7fa25d145700 1 --2- 172.21.2.101:0/1540700283 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2400186d0 0x7fa24001abd0 crc :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:14.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.144 7fa25fb2a700 1 -- 172.21.2.101:0/1540700283 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa258080750 con 0x7fa25808cc50 2021-02-10T11:37:14.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.146 7fa24e7fc700 1 -- 172.21.2.101:0/1540700283 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa25401f560 con 0x7fa25808cc50 2021-02-10T11:37:14.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.274 7fa25fb2a700 1 -- 172.21.2.101:0/1540700283 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7fa2580ce420 con 0x7fa25808cc50 2021-02-10T11:37:14.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.276 7fa24e7fc700 1 -- 172.21.2.101:0/1540700283 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v85) v1 ==== 100+0+10 (secure 0 0 0) 0x7fa25400bd60 con 0x7fa25808cc50 2021-02-10T11:37:14.281 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:37:14.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.276 7fa25fb2a700 1 -- 172.21.2.101:0/1540700283 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2400186d0 msgr2=0x7fa24001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:14.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.276 7fa25fb2a700 1 --2- 172.21.2.101:0/1540700283 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2400186d0 0x7fa24001abd0 crc :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:14.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.276 7fa25fb2a700 1 -- 172.21.2.101:0/1540700283 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa25808cc50 msgr2=0x7fa2580a65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:14.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.276 7fa25fb2a700 1 --2- 172.21.2.101:0/1540700283 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa25808cc50 0x7fa2580a65f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fa254009630 tx=0x7fa25400b970).stop 2021-02-10T11:37:14.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.276 7fa25fb2a700 1 -- 172.21.2.101:0/1540700283 shutdown_connections 2021-02-10T11:37:14.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.276 7fa25fb2a700 1 --2- 172.21.2.101:0/1540700283 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa25808df70 0x7fa25808fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:14.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.276 7fa25fb2a700 1 --2- 172.21.2.101:0/1540700283 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa25808d550 0x7fa25808d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:14.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.276 7fa25fb2a700 1 --2- 172.21.2.101:0/1540700283 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2400186d0 0x7fa24001abd0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:14.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.276 7fa25fb2a700 1 --2- 172.21.2.101:0/1540700283 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa25808cc50 0x7fa2580a65f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:14.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.276 7fa25fb2a700 1 -- 172.21.2.101:0/1540700283 >> 172.21.2.101:0/1540700283 conn(0x7fa2580bcff0 msgr2=0x7fa258107630 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:14.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.277 7fa25fb2a700 1 -- 172.21.2.101:0/1540700283 shutdown_connections 2021-02-10T11:37:14.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.277 7fa25fb2a700 1 -- 172.21.2.101:0/1540700283 wait complete. 2021-02-10T11:37:14.291 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:37:14.291 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:37:14.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.694 7fa66b58a700 1 Processor -- start 2021-02-10T11:37:14.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.694 7fa66b58a700 1 -- start start 2021-02-10T11:37:14.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.695 7fa66b58a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa6640d1870 0x7fa6641b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:14.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.695 7fa66b58a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6640f0b10 con 0x7fa6641b0a10 2021-02-10T11:37:14.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.695 7fa66b58a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6640d0310 con 0x7fa6640f0650 2021-02-10T11:37:14.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.695 7fa66b58a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6640d0530 con 0x7fa6640d1870 2021-02-10T11:37:14.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.695 7fa668ba5700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa6641b0a10 0x7fa6640cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55138/0 (socket says 172.21.2.101:55138) 2021-02-10T11:37:14.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.695 7fa668ba5700 1 -- 172.21.2.101:0/1363059954 learned_addr learned my addr 172.21.2.101:0/1363059954 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:37:14.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.695 7fa6693a6700 1 --2- 172.21.2.101:0/1363059954 >> v2:172.21.2.101:3300/0 conn(0x7fa6640d1870 0x7fa6641b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:14.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.695 7fa6693a6700 1 -- 172.21.2.101:0/1363059954 >> v1:172.21.2.108:6789/0 conn(0x7fa6640f0650 legacy=0x7fa6640cfb20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:37:14.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.695 7fa6693a6700 1 -- 172.21.2.101:0/1363059954 >> v1:172.21.2.101:6789/0 conn(0x7fa6641b0a10 legacy=0x7fa6640cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:14.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.695 7fa657fff700 1 -- 172.21.2.101:0/1363059954 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3403509338 0 0) 0x7fa6600010b0 con 0x7fa6641b0a10 2021-02-10T11:37:14.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.695 7fa6693a6700 1 -- 172.21.2.101:0/1363059954 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6640d1c70 con 0x7fa6640d1870 2021-02-10T11:37:14.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.696 7fa6693a6700 1 --2- 172.21.2.101:0/1363059954 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa6640d1870 0x7fa6641b03f0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 rx=0x7fa658006650 tx=0x7fa658009a00).ready entity=mon.0 client_cookie=579522c6a1e942f6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:14.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.696 7fa657fff700 1 -- 172.21.2.101:0/1363059954 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa6580009b0 con 0x7fa6640d1870 2021-02-10T11:37:14.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.696 7fa657fff700 1 -- 172.21.2.101:0/1363059954 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65800c0c0 con 0x7fa6640d1870 2021-02-10T11:37:14.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.696 7fa657fff700 1 -- 172.21.2.101:0/1363059954 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa65800c5a0 con 0x7fa6640d1870 2021-02-10T11:37:14.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.696 7fa66b58a700 1 -- 172.21.2.101:0/1363059954 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa6640d1870 msgr2=0x7fa6641b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:14.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.696 7fa66b58a700 1 --2- 172.21.2.101:0/1363059954 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa6640d1870 0x7fa6641b03f0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 rx=0x7fa658006650 tx=0x7fa658009a00).stop 2021-02-10T11:37:14.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.696 7fa66b58a700 1 -- 172.21.2.101:0/1363059954 shutdown_connections 2021-02-10T11:37:14.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.696 7fa66b58a700 1 --2- 172.21.2.101:0/1363059954 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa6640d1870 0x7fa6641b03f0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:14.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.696 7fa66b58a700 1 -- 172.21.2.101:0/1363059954 >> 172.21.2.101:0/1363059954 conn(0x7fa6640ce450 msgr2=0x7fa6640d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:14.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.697 7fa66b58a700 1 -- 172.21.2.101:0/1363059954 shutdown_connections 2021-02-10T11:37:14.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.697 7fa66b58a700 1 -- 172.21.2.101:0/1363059954 wait complete. 2021-02-10T11:37:14.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.711 7fa66b58a700 1 Processor -- start 2021-02-10T11:37:14.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.711 7fa66b58a700 1 -- start start 2021-02-10T11:37:14.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.712 7fa66b58a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa66419af00 0x7fa66419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:14.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.712 7fa66b58a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa66419ba80 0x7fa66419be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:14.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.712 7fa66b58a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa66419c4a0 0x7fa66408a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:14.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.712 7fa66b58a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66408fb50 con 0x7fa66419ba80 2021-02-10T11:37:14.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.712 7fa66b58a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66408afb0 con 0x7fa66419af00 2021-02-10T11:37:14.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.712 7fa66b58a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66408b1a0 con 0x7fa66419c4a0 2021-02-10T11:37:14.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.712 7fa668ba5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa66419ba80 0x7fa66419be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:14.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.712 7fa668ba5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa66419ba80 0x7fa66419be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54650/0 (socket says 172.21.2.101:54650) 2021-02-10T11:37:14.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.712 7fa668ba5700 1 -- 172.21.2.101:0/814405981 learned_addr learned my addr 172.21.2.101:0/814405981 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:14.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.712 7fa668ba5700 1 -- 172.21.2.101:0/814405981 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa66419c4a0 msgr2=0x7fa66408a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:14.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.712 7fa668ba5700 1 --2- 172.21.2.101:0/814405981 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa66419c4a0 0x7fa66408a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:14.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.712 7fa668ba5700 1 -- 172.21.2.101:0/814405981 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa66419af00 msgr2=0x7fa66419b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:14.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.712 7fa668ba5700 1 --2- 172.21.2.101:0/814405981 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa66419af00 0x7fa66419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:14.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.712 7fa668ba5700 1 -- 172.21.2.101:0/814405981 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66408b3c0 con 0x7fa66419ba80 2021-02-10T11:37:14.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.712 7fa668ba5700 1 --2- 172.21.2.101:0/814405981 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa66419ba80 0x7fa66419be80 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 rx=0x7fa660001c10 tx=0x7fa66000a8f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:14.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.713 7fa655ffb700 1 -- 172.21.2.101:0/814405981 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa6600009b0 con 0x7fa66419ba80 2021-02-10T11:37:14.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.713 7fa655ffb700 1 -- 172.21.2.101:0/814405981 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6600012c0 con 0x7fa66419ba80 2021-02-10T11:37:14.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.713 7fa655ffb700 1 -- 172.21.2.101:0/814405981 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa660004350 con 0x7fa66419ba80 2021-02-10T11:37:14.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.713 7fa66b58a700 1 -- 172.21.2.101:0/814405981 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa66408b3c0 con 0x7fa66419ba80 2021-02-10T11:37:14.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.713 7fa66b58a700 1 -- 172.21.2.101:0/814405981 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66408b3c0 con 0x7fa66419ba80 2021-02-10T11:37:14.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.713 7fa655ffb700 1 -- 172.21.2.101:0/814405981 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa660004140 con 0x7fa66419ba80 2021-02-10T11:37:14.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.713 7fa655ffb700 1 --2- 172.21.2.101:0/814405981 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa6440186c0 0x7fa64401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:14.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.713 7fa655ffb700 1 -- 172.21.2.101:0/814405981 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa660032510 con 0x7fa66419ba80 2021-02-10T11:37:14.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.713 7fa6693a6700 1 --2- 172.21.2.101:0/814405981 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa6440186c0 0x7fa64401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:14.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.714 7fa6693a6700 1 --2- 172.21.2.101:0/814405981 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa6440186c0 0x7fa64401abc0 crc :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:14.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.745 7fa66b58a700 1 -- 172.21.2.101:0/814405981 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6640ee030 con 0x7fa66419ba80 2021-02-10T11:37:14.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.747 7fa655ffb700 1 -- 172.21.2.101:0/814405981 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa660004140 con 0x7fa66419ba80 2021-02-10T11:37:14.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.875 7fa66b58a700 1 -- 172.21.2.101:0/814405981 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6640ee030 con 0x7fa66419ba80 2021-02-10T11:37:14.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.875 7fa655ffb700 1 -- 172.21.2.101:0/814405981 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+2101 (secure 0 0 0) 0x7fa66000ace0 con 0x7fa66419ba80 2021-02-10T11:37:14.881 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:37:14.881 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9360,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2529021396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2529021396},{"type":"v1","addr":"172.21.2.101:6835","nonce":2529021396}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:56.648753","modified":"2021-02-10 11:36:58.653009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6005":{"gid":6005,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2363068042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2363068042},{"type":"v1","addr":"172.21.2.108:6833","nonce":2363068042}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-02-10T11:37:14.882 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 62 2021-02-10T11:37:14.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.877 7fa66b58a700 1 -- 172.21.2.101:0/814405981 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa6440186c0 msgr2=0x7fa64401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:14.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.877 7fa66b58a700 1 --2- 172.21.2.101:0/814405981 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa6440186c0 0x7fa64401abc0 crc :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:14.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.877 7fa66b58a700 1 -- 172.21.2.101:0/814405981 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa66419ba80 msgr2=0x7fa66419be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:14.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.877 7fa66b58a700 1 --2- 172.21.2.101:0/814405981 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa66419ba80 0x7fa66419be80 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 rx=0x7fa660001c10 tx=0x7fa66000a8f0).stop 2021-02-10T11:37:14.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.877 7fa66b58a700 1 -- 172.21.2.101:0/814405981 shutdown_connections 2021-02-10T11:37:14.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.877 7fa66b58a700 1 --2- 172.21.2.101:0/814405981 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa6440186c0 0x7fa64401abc0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:14.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.877 7fa66b58a700 1 --2- 172.21.2.101:0/814405981 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa66419c4a0 0x7fa66408a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:14.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.877 7fa66b58a700 1 --2- 172.21.2.101:0/814405981 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa66419ba80 0x7fa66419be80 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:14.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.877 7fa66b58a700 1 --2- 172.21.2.101:0/814405981 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa66419af00 0x7fa66419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:14.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.877 7fa66b58a700 1 -- 172.21.2.101:0/814405981 >> 172.21.2.101:0/814405981 conn(0x7fa66419f8c0 msgr2=0x7fa6641a5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:14.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.878 7fa66b58a700 1 -- 172.21.2.101:0/814405981 shutdown_connections 2021-02-10T11:37:14.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:14.878 7fa66b58a700 1 -- 172.21.2.101:0/814405981 wait complete. 2021-02-10T11:37:15.093 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:36:56.648753', 'modified': '2021-02-10 11:36:58.653009', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6005}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6005': {'gid': 6005, 'name': 'b', 'rank': 0, 'incarnation': 60, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/2363068042', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 2363068042}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 2363068042}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:37:15.093 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:37:15.093 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:37:15.489 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:37:15.490 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:37:15.490 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:37:15.490 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 6005, 2021-02-10T11:37:15.490 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:37:15.490 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:37:15.490 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 18.832943886999999, 2021-02-10T11:37:15.490 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 62, 2021-02-10T11:37:15.490 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 85, 2021-02-10T11:37:15.490 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 84, 2021-02-10T11:37:15.490 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 24.473789012000001 2021-02-10T11:37:15.491 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:37:15.493 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 6005, "want_state": "up:active", "state": "up:active", "rank_uptime": 18.832943886999999, "mdsmap_epoch": 62, "osdmap_epoch": 85, "osdmap_epoch_barrier": 84, "uptime": 24.473789012000001 } 2021-02-10T11:37:15.494 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:37:15.494 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-02-10T11:37:15.494 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:37:15.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:37:15.494 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:37:15.831 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:37:16.175 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:37:16.175 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:37:16.518 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:37:16.519 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:37:16.520 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:37:16.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:37:16.520 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:37:16.855 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:37:16.855 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2021-02-10T11:37:16.855 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:37:16.858 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:37:17.202 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:37:17.202 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:37:17.267 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[135217]: starting ceph client 2021-02-10T11:37:17.267 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:2021-02-10 11:37:17.261 7f3b6a729f80 -1 init, newargv = 0x55b170f601a0 newargc=9 2021-02-10T11:37:17.272 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[135217]: starting fuse 2021-02-10T11:37:17.546 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:37:17.546 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:37:17.547 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:37:17.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:37:17.547 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:37:17.880 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:37:17.881 INFO:teuthology.orchestra.run.gibba001.stdout:44 2021-02-10T11:37:17.881 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2021-02-10T11:37:17.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:37:17.881 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:37:17.881 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:37:17.881 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:37:17.881 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:37:17.882 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:37:17.882 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:37:17.882 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:37:17.882 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-02-10T11:37:17.882 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:37:17.882 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:37:17.882 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:37:17.882 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:37:17.882 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:37:17.882 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:37:17.882 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:37:17.882 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:37:17.882 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:37:17.882 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:37:17.883 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:37:17.883 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:37:17.883 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:37:17.883 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.1")) 2021-02-10T11:37:17.883 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:37:18.243 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.1.135217.asok 2021-02-10T11:37:18.246 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.135217.asok 2021-02-10T11:37:18.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:37:18.246 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.135217.asok status 2021-02-10T11:37:18.635 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:37:18.635 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:37:18.635 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:37:18.635 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:37:18.635 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "1", 2021-02-10T11:37:18.635 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:37:18.635 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:37:18.635 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "135217", 2021-02-10T11:37:18.636 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:37:18.636 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:37:18.636 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:37:18.636 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:37:18.636 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 9558, 2021-02-10T11:37:18.636 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:37:18.636 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:37:18.636 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:37:18.636 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 9558 2021-02-10T11:37:18.636 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:37:18.636 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:37:18.636 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:37:18.636 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:37:18.636 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 4181133942 2021-02-10T11:37:18.636 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:37:18.637 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:37:18.637 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:37:18.637 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:37:18.637 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:37:18.637 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 4181133942 2021-02-10T11:37:18.637 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:37:18.637 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.9558 172.21.2.101:0/4181133942", 2021-02-10T11:37:18.637 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/4181133942", 2021-02-10T11:37:18.637 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:37:18.637 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 62, 2021-02-10T11:37:18.637 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 85, 2021-02-10T11:37:18.637 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:37:18.637 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:37:18.637 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:37:18.639 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:37:18.639 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:37:18.639 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:37:18.974 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:37:18.975 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:37:18.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:37:18.975 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:37:19.320 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2021-02-10T11:37:19.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.729 7fb07cce0700 1 Processor -- start 2021-02-10T11:37:19.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.729 7fb07cce0700 1 -- start start 2021-02-10T11:37:19.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.730 7fb07cce0700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb0780d1880 0x7fb0781b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:19.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.730 7fb07cce0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0781b0af0 0x7fb078108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:19.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.730 7fb07cce0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0780fe1d0 con 0x7fb0780d1c80 2021-02-10T11:37:19.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.730 7fb07cce0700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb078109a70 con 0x7fb0780d1880 2021-02-10T11:37:19.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.730 7fb07cce0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb078109760 con 0x7fb0781b0af0 2021-02-10T11:37:19.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.730 7fb0762e7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0781b0af0 0x7fb078108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:19.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.730 7fb076ae8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb0780d1880 0x7fb0781b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:19.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.730 7fb0762e7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0781b0af0 0x7fb078108950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:47402/0 (socket says 172.21.2.101:47402) 2021-02-10T11:37:19.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.730 7fb076ae8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb0780d1880 0x7fb0781b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:36922/0 (socket says 172.21.2.101:36922) 2021-02-10T11:37:19.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.730 7fb0762e7700 1 -- 172.21.2.101:0/2201994485 learned_addr learned my addr 172.21.2.101:0/2201994485 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:19.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.731 7fb076ae8700 1 -- 172.21.2.101:0/2201994485 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0781b0af0 msgr2=0x7fb078108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:19.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.731 7fb076ae8700 1 --2- 172.21.2.101:0/2201994485 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0781b0af0 0x7fb078108950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:19.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.731 7fb076ae8700 1 -- 172.21.2.101:0/2201994485 >> v1:172.21.2.108:6789/0 conn(0x7fb0780d1c80 legacy=0x7fb078108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:19.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.731 7fb076ae8700 1 -- 172.21.2.101:0/2201994485 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb078088070 con 0x7fb0780d1880 2021-02-10T11:37:19.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.731 7fb0762e7700 1 --2- 172.21.2.101:0/2201994485 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0781b0af0 0x7fb078108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:37:19.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.731 7fb076ae8700 1 --2- 172.21.2.101:0/2201994485 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0780d1880 0x7fb0781b04d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7fb068006650 tx=0x7fb068009a80).ready entity=mon.1 client_cookie=f6d958c61344ca93 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:19.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.731 7fb075ae6700 1 -- 172.21.2.101:0/2201994485 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0680009b0 con 0x7fb0780d1880 2021-02-10T11:37:19.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.731 7fb075ae6700 1 -- 172.21.2.101:0/2201994485 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06800c0c0 con 0x7fb0780d1880 2021-02-10T11:37:19.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.731 7fb075ae6700 1 -- 172.21.2.101:0/2201994485 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb06800c5e0 con 0x7fb0780d1880 2021-02-10T11:37:19.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.732 7fb07cce0700 1 -- 172.21.2.101:0/2201994485 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0780d1880 msgr2=0x7fb0781b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:19.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.732 7fb07cce0700 1 --2- 172.21.2.101:0/2201994485 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0780d1880 0x7fb0781b04d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7fb068006650 tx=0x7fb068009a80).stop 2021-02-10T11:37:19.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.732 7fb07cce0700 1 -- 172.21.2.101:0/2201994485 shutdown_connections 2021-02-10T11:37:19.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.732 7fb07cce0700 1 --2- 172.21.2.101:0/2201994485 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0781b0af0 0x7fb078108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:19.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.732 7fb07cce0700 1 --2- 172.21.2.101:0/2201994485 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0780d1880 0x7fb0781b04d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:19.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.732 7fb07cce0700 1 -- 172.21.2.101:0/2201994485 >> 172.21.2.101:0/2201994485 conn(0x7fb0780ce430 msgr2=0x7fb0780d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:19.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.733 7fb07cce0700 1 -- 172.21.2.101:0/2201994485 shutdown_connections 2021-02-10T11:37:19.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.733 7fb07cce0700 1 -- 172.21.2.101:0/2201994485 wait complete. 2021-02-10T11:37:19.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.747 7fb07cce0700 1 Processor -- start 2021-02-10T11:37:19.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.747 7fb07cce0700 1 -- start start 2021-02-10T11:37:19.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.747 7fb07cce0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb078095710 0x7fb07819dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:19.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.747 7fb07cce0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0780aeeb0 0x7fb07819e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:19.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.747 7fb07cce0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb07819e900 0x7fb0781b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:19.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.747 7fb07cce0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07819ed00 con 0x7fb0780aeeb0 2021-02-10T11:37:19.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.747 7fb07cce0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb078094050 con 0x7fb07819e900 2021-02-10T11:37:19.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.747 7fb07cce0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0781b7fb0 con 0x7fb078095710 2021-02-10T11:37:19.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.748 7fb0762e7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0780aeeb0 0x7fb07819e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:19.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.748 7fb0762e7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0780aeeb0 0x7fb07819e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54676/0 (socket says 172.21.2.101:54676) 2021-02-10T11:37:19.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.748 7fb0762e7700 1 -- 172.21.2.101:0/2192310246 learned_addr learned my addr 172.21.2.101:0/2192310246 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:19.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.748 7fb076ae8700 1 --2- 172.21.2.101:0/2192310246 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb078095710 0x7fb07819dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:19.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.748 7fb0772e9700 1 --2- 172.21.2.101:0/2192310246 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb07819e900 0x7fb0781b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:19.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.748 7fb0762e7700 1 -- 172.21.2.101:0/2192310246 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb078095710 msgr2=0x7fb07819dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:19.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.748 7fb0762e7700 1 --2- 172.21.2.101:0/2192310246 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb078095710 0x7fb07819dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:19.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.748 7fb0762e7700 1 -- 172.21.2.101:0/2192310246 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb07819e900 msgr2=0x7fb0781b7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:19.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.748 7fb0762e7700 1 --2- 172.21.2.101:0/2192310246 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb07819e900 0x7fb0781b7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:19.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.748 7fb0762e7700 1 -- 172.21.2.101:0/2192310246 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0781b81d0 con 0x7fb0780aeeb0 2021-02-10T11:37:19.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.748 7fb0762e7700 1 --2- 172.21.2.101:0/2192310246 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0780aeeb0 0x7fb07819e2e0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 rx=0x7fb060006ee0 tx=0x7fb06000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:19.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.748 7fb06f7fe700 1 -- 172.21.2.101:0/2192310246 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb06000bd50 con 0x7fb0780aeeb0 2021-02-10T11:37:19.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.748 7fb06f7fe700 1 -- 172.21.2.101:0/2192310246 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0600052e0 con 0x7fb0780aeeb0 2021-02-10T11:37:19.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.748 7fb06f7fe700 1 -- 172.21.2.101:0/2192310246 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb060005800 con 0x7fb0780aeeb0 2021-02-10T11:37:19.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.748 7fb07cce0700 1 -- 172.21.2.101:0/2192310246 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0781b81d0 con 0x7fb0780aeeb0 2021-02-10T11:37:19.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.749 7fb07cce0700 1 -- 172.21.2.101:0/2192310246 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0781b8760 con 0x7fb0780aeeb0 2021-02-10T11:37:19.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.749 7fb06f7fe700 1 -- 172.21.2.101:0/2192310246 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb0600054c0 con 0x7fb0780aeeb0 2021-02-10T11:37:19.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.749 7fb06f7fe700 1 --2- 172.21.2.101:0/2192310246 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb05c0186d0 0x7fb05c01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:19.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.749 7fb06f7fe700 1 -- 172.21.2.101:0/2192310246 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb060032670 con 0x7fb0780aeeb0 2021-02-10T11:37:19.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.749 7fb076ae8700 1 --2- 172.21.2.101:0/2192310246 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb05c0186d0 0x7fb05c01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:19.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.749 7fb076ae8700 1 --2- 172.21.2.101:0/2192310246 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb05c0186d0 0x7fb05c01abd0 crc :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:19.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.779 7fb07cce0700 1 -- 172.21.2.101:0/2192310246 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb078082aa0 con 0x7fb0780aeeb0 2021-02-10T11:37:19.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.782 7fb06f7fe700 1 -- 172.21.2.101:0/2192310246 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb0600009b0 con 0x7fb0780aeeb0 2021-02-10T11:37:19.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:19.919 7fb07cce0700 1 -- 172.21.2.101:0/2192310246 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb078082aa0 con 0x7fb0780aeeb0 2021-02-10T11:37:20.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:20.921 7fb06f7fe700 1 -- 172.21.2.101:0/2192310246 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v63) v1 ==== 136+0+0 (secure 0 0 0) 0x7fb0600052e0 con 0x7fb0780aeeb0 2021-02-10T11:37:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:20.941 7fb07cce0700 1 -- 172.21.2.101:0/2192310246 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb05c0186d0 msgr2=0x7fb05c01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:20.941 7fb07cce0700 1 --2- 172.21.2.101:0/2192310246 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb05c0186d0 0x7fb05c01abd0 crc :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:20.941 7fb07cce0700 1 -- 172.21.2.101:0/2192310246 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0780aeeb0 msgr2=0x7fb07819e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:20.941 7fb07cce0700 1 --2- 172.21.2.101:0/2192310246 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0780aeeb0 0x7fb07819e2e0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 rx=0x7fb060006ee0 tx=0x7fb06000b960).stop 2021-02-10T11:37:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:20.941 7fb07cce0700 1 -- 172.21.2.101:0/2192310246 shutdown_connections 2021-02-10T11:37:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:20.941 7fb07cce0700 1 --2- 172.21.2.101:0/2192310246 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb07819e900 0x7fb0781b7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:20.941 7fb07cce0700 1 --2- 172.21.2.101:0/2192310246 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0780aeeb0 0x7fb07819e2e0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:20.941 7fb07cce0700 1 --2- 172.21.2.101:0/2192310246 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb05c0186d0 0x7fb05c01abd0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:20.941 7fb07cce0700 1 --2- 172.21.2.101:0/2192310246 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb078095710 0x7fb07819dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:20.941 7fb07cce0700 1 -- 172.21.2.101:0/2192310246 >> 172.21.2.101:0/2192310246 conn(0x7fb0780846b0 msgr2=0x7fb078094950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:20.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:20.941 7fb07cce0700 1 -- 172.21.2.101:0/2192310246 shutdown_connections 2021-02-10T11:37:20.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:20.941 7fb07cce0700 1 -- 172.21.2.101:0/2192310246 wait complete. 2021-02-10T11:37:20.955 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs volume ls 2021-02-10T11:37:21.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.358 7ff9a7de6700 1 Processor -- start 2021-02-10T11:37:21.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.358 7ff9a7de6700 1 -- start start 2021-02-10T11:37:21.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.359 7ff9a7de6700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff9a00d1870 0x7ff9a01b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:21.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.359 7ff9a7de6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff9a00f0650 0x7ff9a00cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:21.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.359 7ff9a7de6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a00f0b10 con 0x7ff9a01b0a10 2021-02-10T11:37:21.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.359 7ff9a7de6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a00e5ed0 con 0x7ff9a00f0650 2021-02-10T11:37:21.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.359 7ff9a7de6700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a00d1c70 con 0x7ff9a00d1870 2021-02-10T11:37:21.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.359 7ff9a6403700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff9a00f0650 0x7ff9a00cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:21.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.359 7ff9a6403700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff9a00f0650 0x7ff9a00cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54682/0 (socket says 172.21.2.101:54682) 2021-02-10T11:37:21.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.359 7ff9a6403700 1 -- 172.21.2.101:0/1790449401 learned_addr learned my addr 172.21.2.101:0/1790449401 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:21.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.359 7ff9a6403700 1 -- 172.21.2.101:0/1790449401 >> v2:172.21.2.108:3300/0 conn(0x7ff9a00d1870 msgr2=0x7ff9a01b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:37:21.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.359 7ff9a6403700 1 --2- 172.21.2.101:0/1790449401 >> v2:172.21.2.108:3300/0 conn(0x7ff9a00d1870 0x7ff9a01b03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:21.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.359 7ff9a6403700 1 -- 172.21.2.101:0/1790449401 >> v1:172.21.2.108:6789/0 conn(0x7ff9a01b0a10 legacy=0x7ff9a00cf330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:37:21.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.359 7ff9a6403700 1 -- 172.21.2.101:0/1790449401 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a00d1e90 con 0x7ff9a00f0650 2021-02-10T11:37:21.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.360 7ff9a6403700 1 --2- 172.21.2.101:0/1790449401 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9a00f0650 0x7ff9a00cfb20 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 rx=0x7ff998009b50 tx=0x7ff998009ef0).ready entity=mon.0 client_cookie=9049bce4f0fc28be server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:21.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.360 7ff9a4c00700 1 -- 172.21.2.101:0/1790449401 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff998000f90 con 0x7ff9a00f0650 2021-02-10T11:37:21.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.360 7ff9a4c00700 1 -- 172.21.2.101:0/1790449401 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99800b3b0 con 0x7ff9a00f0650 2021-02-10T11:37:21.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.360 7ff9a4c00700 1 -- 172.21.2.101:0/1790449401 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff99800b8d0 con 0x7ff9a00f0650 2021-02-10T11:37:21.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.360 7ff9a7de6700 1 -- 172.21.2.101:0/1790449401 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9a00f0650 msgr2=0x7ff9a00cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:21.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.360 7ff9a7de6700 1 --2- 172.21.2.101:0/1790449401 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9a00f0650 0x7ff9a00cfb20 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 rx=0x7ff998009b50 tx=0x7ff998009ef0).stop 2021-02-10T11:37:21.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.360 7ff9a7de6700 1 -- 172.21.2.101:0/1790449401 shutdown_connections 2021-02-10T11:37:21.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.360 7ff9a7de6700 1 --2- 172.21.2.101:0/1790449401 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9a00f0650 0x7ff9a00cfb20 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:21.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.360 7ff9a7de6700 1 --2- 172.21.2.101:0/1790449401 >> v2:172.21.2.108:3300/0 conn(0x7ff9a00d1870 0x7ff9a01b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:21.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.360 7ff9a7de6700 1 -- 172.21.2.101:0/1790449401 >> 172.21.2.101:0/1790449401 conn(0x7ff9a00ce450 msgr2=0x7ff9a00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:21.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.361 7ff9a7de6700 1 -- 172.21.2.101:0/1790449401 shutdown_connections 2021-02-10T11:37:21.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.361 7ff9a7de6700 1 -- 172.21.2.101:0/1790449401 wait complete. 2021-02-10T11:37:21.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.375 7ff9a7de6700 1 Processor -- start 2021-02-10T11:37:21.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.375 7ff9a7de6700 1 -- start start 2021-02-10T11:37:21.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.375 7ff9a7de6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff9a00bb5d0 0x7ff9a00bfac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:21.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.375 7ff9a7de6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9a00a9b50 0x7ff9a00a9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:21.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.375 7ff9a7de6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9a01a7550 0x7ff9a01a7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:21.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.375 7ff9a7de6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a00bb9d0 con 0x7ff9a00a9b50 2021-02-10T11:37:21.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.375 7ff9a7de6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a00aa570 con 0x7ff9a00bb5d0 2021-02-10T11:37:21.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.375 7ff9a7de6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a00aa790 con 0x7ff9a01a7550 2021-02-10T11:37:21.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.375 7ff9a5401700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9a00a9b50 0x7ff9a00a9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:21.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.375 7ff9a5401700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9a00a9b50 0x7ff9a00a9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54690/0 (socket says 172.21.2.101:54690) 2021-02-10T11:37:21.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.375 7ff9a5401700 1 -- 172.21.2.101:0/2158862320 learned_addr learned my addr 172.21.2.101:0/2158862320 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:21.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.376 7ff9a5401700 1 -- 172.21.2.101:0/2158862320 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9a01a7550 msgr2=0x7ff9a01a7950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:37:21.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.376 7ff9a5401700 1 --2- 172.21.2.101:0/2158862320 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9a01a7550 0x7ff9a01a7950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:21.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.376 7ff9a5401700 1 -- 172.21.2.101:0/2158862320 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff9a00bb5d0 msgr2=0x7ff9a00bfac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:37:21.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.376 7ff9a5401700 1 --2- 172.21.2.101:0/2158862320 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff9a00bb5d0 0x7ff9a00bfac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:21.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.376 7ff9a5401700 1 -- 172.21.2.101:0/2158862320 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a01a7f70 con 0x7ff9a00a9b50 2021-02-10T11:37:21.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.376 7ff9a5401700 1 --2- 172.21.2.101:0/2158862320 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9a00a9b50 0x7ff9a00a9f50 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 rx=0x7ff99c00b5b0 tx=0x7ff99c00b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:21.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.376 7ff9927fc700 1 -- 172.21.2.101:0/2158862320 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff99c00bd40 con 0x7ff9a00a9b50 2021-02-10T11:37:21.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.376 7ff9927fc700 1 -- 172.21.2.101:0/2158862320 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99c002080 con 0x7ff9a00a9b50 2021-02-10T11:37:21.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.376 7ff9a7de6700 1 -- 172.21.2.101:0/2158862320 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a01a7f70 con 0x7ff9a00a9b50 2021-02-10T11:37:21.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.376 7ff9927fc700 1 -- 172.21.2.101:0/2158862320 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff99c0025a0 con 0x7ff9a00a9b50 2021-02-10T11:37:21.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.376 7ff9a7de6700 1 -- 172.21.2.101:0/2158862320 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a01a7f70 con 0x7ff9a00a9b50 2021-02-10T11:37:21.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.377 7ff9927fc700 1 -- 172.21.2.101:0/2158862320 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff99c006a70 con 0x7ff9a00a9b50 2021-02-10T11:37:21.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.377 7ff9927fc700 1 --2- 172.21.2.101:0/2158862320 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff988018730 0x7ff98801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:21.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.377 7ff9927fc700 1 -- 172.21.2.101:0/2158862320 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff99c0327e0 con 0x7ff9a00a9b50 2021-02-10T11:37:21.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.377 7ff9a5c02700 1 --2- 172.21.2.101:0/2158862320 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff988018730 0x7ff98801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:21.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.377 7ff9a5c02700 1 --2- 172.21.2.101:0/2158862320 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff988018730 0x7ff98801ac10 crc :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:21.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.409 7ff9a7de6700 1 -- 172.21.2.101:0/2158862320 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a00ee0b0 con 0x7ff9a00a9b50 2021-02-10T11:37:21.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.411 7ff9927fc700 1 -- 172.21.2.101:0/2158862320 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff99c00bd40 con 0x7ff9a00a9b50 2021-02-10T11:37:21.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.545 7ff9a7de6700 1 -- 172.21.2.101:0/2158862320 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs volume ls", "target": ["mgr", ""]}) v1 -- 0x7ff9a00ce450 con 0x7ff988018730 2021-02-10T11:37:21.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.546 7ff9927fc700 1 -- 172.21.2.101:0/2158862320 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7ff994006970 con 0x7ff988018730 2021-02-10T11:37:21.551 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:37:21.551 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:37:21.551 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "cephfs" 2021-02-10T11:37:21.552 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:37:21.552 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:37:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.547 7ff9a7de6700 1 -- 172.21.2.101:0/2158862320 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff988018730 msgr2=0x7ff98801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.547 7ff9a7de6700 1 --2- 172.21.2.101:0/2158862320 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff988018730 0x7ff98801ac10 crc :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.547 7ff9a7de6700 1 -- 172.21.2.101:0/2158862320 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9a00a9b50 msgr2=0x7ff9a00a9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.547 7ff9a7de6700 1 --2- 172.21.2.101:0/2158862320 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9a00a9b50 0x7ff9a00a9f50 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 rx=0x7ff99c00b5b0 tx=0x7ff99c00b950).stop 2021-02-10T11:37:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.547 7ff9a7de6700 1 -- 172.21.2.101:0/2158862320 shutdown_connections 2021-02-10T11:37:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.547 7ff9a7de6700 1 --2- 172.21.2.101:0/2158862320 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff988018730 0x7ff98801ac10 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.547 7ff9a7de6700 1 --2- 172.21.2.101:0/2158862320 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9a01a7550 0x7ff9a01a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.547 7ff9a7de6700 1 --2- 172.21.2.101:0/2158862320 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9a00a9b50 0x7ff9a00a9f50 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.547 7ff9a7de6700 1 --2- 172.21.2.101:0/2158862320 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff9a00bb5d0 0x7ff9a00bfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.547 7ff9a7de6700 1 -- 172.21.2.101:0/2158862320 >> 172.21.2.101:0/2158862320 conn(0x7ff9a00a6eb0 msgr2=0x7ff9a00bf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.547 7ff9a7de6700 1 -- 172.21.2.101:0/2158862320 shutdown_connections 2021-02-10T11:37:21.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.547 7ff9a7de6700 1 -- 172.21.2.101:0/2158862320 wait complete. 2021-02-10T11:37:21.561 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:37:21.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.970 7f322b793700 1 Processor -- start 2021-02-10T11:37:21.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.970 7f322b793700 1 -- start start 2021-02-10T11:37:21.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.970 7f322b793700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f32240d1870 0x7f32241b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:21.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.970 7f322b793700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f32241b0ae0 0x7f3224108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:21.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.970 7f322b793700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32240d1c70 0x7f3224108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:21.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.970 7f322b793700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32240fe1c0 con 0x7f32240d1870 2021-02-10T11:37:21.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.970 7f322b793700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3224109a60 con 0x7f32241b0ae0 2021-02-10T11:37:21.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.970 7f322b793700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3224088070 con 0x7f32240d1c70 2021-02-10T11:37:21.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.971 7f32295af700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f32240d1870 0x7f32241b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:21.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.971 7f32295af700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f32240d1870 0x7f32241b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54694/0 (socket says 172.21.2.101:54694) 2021-02-10T11:37:21.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.971 7f32295af700 1 -- 172.21.2.101:0/887500757 learned_addr learned my addr 172.21.2.101:0/887500757 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:21.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.971 7f32295af700 1 -- 172.21.2.101:0/887500757 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32240d1c70 msgr2=0x7f3224108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:21.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.971 7f32295af700 1 --2- 172.21.2.101:0/887500757 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32240d1c70 0x7f3224108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:21.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.971 7f32295af700 1 -- 172.21.2.101:0/887500757 >> v2:172.21.2.108:3300/0 conn(0x7f32241b0ae0 msgr2=0x7f3224108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:21.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.971 7f32295af700 1 --2- 172.21.2.101:0/887500757 >> v2:172.21.2.108:3300/0 conn(0x7f32241b0ae0 0x7f3224108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:21.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.971 7f32295af700 1 -- 172.21.2.101:0/887500757 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3224109580 con 0x7f32240d1870 2021-02-10T11:37:21.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.971 7f32295af700 1 --2- 172.21.2.101:0/887500757 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32240d1870 0x7f32241b04c0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 rx=0x7f3220006700 tx=0x7f3220009b80).ready entity=mon.0 client_cookie=aa504ec8538be6f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:21.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.972 7f321bfff700 1 -- 172.21.2.101:0/887500757 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f322000c0c0 con 0x7f32240d1870 2021-02-10T11:37:21.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.972 7f321bfff700 1 -- 172.21.2.101:0/887500757 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322000c440 con 0x7f32240d1870 2021-02-10T11:37:21.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.972 7f321bfff700 1 -- 172.21.2.101:0/887500757 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f322000c8a0 con 0x7f32240d1870 2021-02-10T11:37:21.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.972 7f322b793700 1 -- 172.21.2.101:0/887500757 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32240d1870 msgr2=0x7f32241b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:21.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.972 7f322b793700 1 --2- 172.21.2.101:0/887500757 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32240d1870 0x7f32241b04c0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 rx=0x7f3220006700 tx=0x7f3220009b80).stop 2021-02-10T11:37:21.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.972 7f322b793700 1 -- 172.21.2.101:0/887500757 shutdown_connections 2021-02-10T11:37:21.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.972 7f322b793700 1 --2- 172.21.2.101:0/887500757 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32240d1c70 0x7f3224108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:21.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.972 7f322b793700 1 --2- 172.21.2.101:0/887500757 >> v2:172.21.2.108:3300/0 conn(0x7f32241b0ae0 0x7f3224108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:21.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.972 7f322b793700 1 --2- 172.21.2.101:0/887500757 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32240d1870 0x7f32241b04c0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:21.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.972 7f322b793700 1 -- 172.21.2.101:0/887500757 >> 172.21.2.101:0/887500757 conn(0x7f32240ce420 msgr2=0x7f32240d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:21.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.973 7f322b793700 1 -- 172.21.2.101:0/887500757 shutdown_connections 2021-02-10T11:37:21.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.973 7f322b793700 1 -- 172.21.2.101:0/887500757 wait complete. 2021-02-10T11:37:21.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.987 7f322b793700 1 Processor -- start 2021-02-10T11:37:21.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.987 7f322b793700 1 -- start start 2021-02-10T11:37:21.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.988 7f322b793700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f322409bd90 0x7f322419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:21.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.988 7f322b793700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f322419f740 0x7f3224085350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:21.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.988 7f322b793700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3224085970 0x7f3224085d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:21.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.988 7f322b793700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3224086410 con 0x7f3224085970 2021-02-10T11:37:21.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.988 7f322b793700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3224086600 con 0x7f322409bd90 2021-02-10T11:37:21.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.988 7f322b793700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3224086820 con 0x7f322419f740 2021-02-10T11:37:21.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.988 7f3229db0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3224085970 0x7f3224085d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:21.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.988 7f3229db0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3224085970 0x7f3224085d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54700/0 (socket says 172.21.2.101:54700) 2021-02-10T11:37:21.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.988 7f3229db0700 1 -- 172.21.2.101:0/979437870 learned_addr learned my addr 172.21.2.101:0/979437870 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:21.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.989 7f32295af700 1 --2- 172.21.2.101:0/979437870 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f322409bd90 0x7f322419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:21.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.989 7f3228dae700 1 --2- 172.21.2.101:0/979437870 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f322419f740 0x7f3224085350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:21.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.989 7f3229db0700 1 -- 172.21.2.101:0/979437870 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f322419f740 msgr2=0x7f3224085350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:21.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.989 7f3229db0700 1 --2- 172.21.2.101:0/979437870 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f322419f740 0x7f3224085350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:21.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.989 7f3229db0700 1 -- 172.21.2.101:0/979437870 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f322409bd90 msgr2=0x7f322419f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:21.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.989 7f3229db0700 1 --2- 172.21.2.101:0/979437870 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f322409bd90 0x7f322419f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:21.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.989 7f3229db0700 1 -- 172.21.2.101:0/979437870 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32240832b0 con 0x7f3224085970 2021-02-10T11:37:21.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.989 7f32295af700 1 --2- 172.21.2.101:0/979437870 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f322409bd90 0x7f322419f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:37:21.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.989 7f3228dae700 1 --2- 172.21.2.101:0/979437870 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f322419f740 0x7f3224085350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:37:21.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.989 7f3229db0700 1 --2- 172.21.2.101:0/979437870 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3224085970 0x7f3224085d70 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 rx=0x7f321c009980 tx=0x7f321c00bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:21.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.989 7f3219ffb700 1 -- 172.21.2.101:0/979437870 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f321c005af0 con 0x7f3224085970 2021-02-10T11:37:21.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.989 7f3219ffb700 1 -- 172.21.2.101:0/979437870 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321c005df0 con 0x7f3224085970 2021-02-10T11:37:21.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.989 7f3219ffb700 1 -- 172.21.2.101:0/979437870 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f321c0062b0 con 0x7f3224085970 2021-02-10T11:37:21.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.989 7f322b793700 1 -- 172.21.2.101:0/979437870 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32240832b0 con 0x7f3224085970 2021-02-10T11:37:21.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.989 7f322b793700 1 -- 172.21.2.101:0/979437870 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3224083810 con 0x7f3224085970 2021-02-10T11:37:21.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.990 7f3219ffb700 1 -- 172.21.2.101:0/979437870 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f321c005af0 con 0x7f3224085970 2021-02-10T11:37:21.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.990 7f3219ffb700 1 --2- 172.21.2.101:0/979437870 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3204018730 0x7f320401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:21.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.990 7f3219ffb700 1 -- 172.21.2.101:0/979437870 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f321c032b70 con 0x7f3224085970 2021-02-10T11:37:21.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.990 7f32295af700 1 --2- 172.21.2.101:0/979437870 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3204018730 0x7f320401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:21.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:21.990 7f32295af700 1 --2- 172.21.2.101:0/979437870 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3204018730 0x7f320401abc0 crc :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:22.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:22.020 7f322b793700 1 -- 172.21.2.101:0/979437870 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3224086820 con 0x7f3224085970 2021-02-10T11:37:22.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:22.023 7f3219ffb700 1 -- 172.21.2.101:0/979437870 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f321c006570 con 0x7f3224085970 2021-02-10T11:37:22.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:22.173 7f322b793700 1 -- 172.21.2.101:0/979437870 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3224086820 con 0x7f3224085970 2021-02-10T11:37:22.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:22.173 7f3219ffb700 1 -- 172.21.2.101:0/979437870 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f321c006f20 con 0x7f3224085970 2021-02-10T11:37:22.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:22.175 7f322b793700 1 -- 172.21.2.101:0/979437870 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3204018730 msgr2=0x7f320401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:22.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:22.175 7f322b793700 1 --2- 172.21.2.101:0/979437870 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3204018730 0x7f320401abc0 crc :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:22.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:22.175 7f322b793700 1 -- 172.21.2.101:0/979437870 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3224085970 msgr2=0x7f3224085d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:22.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:22.175 7f322b793700 1 --2- 172.21.2.101:0/979437870 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3224085970 0x7f3224085d70 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 rx=0x7f321c009980 tx=0x7f321c00bcd0).stop 2021-02-10T11:37:22.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:22.175 7f322b793700 1 -- 172.21.2.101:0/979437870 shutdown_connections 2021-02-10T11:37:22.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:22.175 7f322b793700 1 --2- 172.21.2.101:0/979437870 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3204018730 0x7f320401abc0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:22.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:22.175 7f322b793700 1 --2- 172.21.2.101:0/979437870 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f322419f740 0x7f3224085350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:22.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:22.175 7f322b793700 1 --2- 172.21.2.101:0/979437870 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3224085970 0x7f3224085d70 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:22.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:22.175 7f322b793700 1 --2- 172.21.2.101:0/979437870 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f322409bd90 0x7f322419f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:22.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:22.175 7f322b793700 1 -- 172.21.2.101:0/979437870 >> 172.21.2.101:0/979437870 conn(0x7f32240946f0 msgr2=0x7f3224095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:22.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:22.175 7f322b793700 1 -- 172.21.2.101:0/979437870 shutdown_connections 2021-02-10T11:37:22.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:22.175 7f322b793700 1 -- 172.21.2.101:0/979437870 wait complete. 2021-02-10T11:37:22.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:37:22.189 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:37:22.189 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:37:22.524 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:37:22.525 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:37:22.525 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:37:22.525 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:37:22.525 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:37:22.872 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[134576]: fuse finished with error 0 and tester_r 0 2021-02-10T11:37:22.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:37:22.874 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:37:22.874 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:37:23.208 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:37:23.208 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:37:23.209 INFO:teuthology.orchestra.run:waiting for 900 2021-02-10T11:37:23.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:37:23.209 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:37:23.545 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:37:23.545 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:37:23.545 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:37:23.881 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:37:23.881 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:37:23.881 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:37:23.882 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:37:23.882 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:37:24.231 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[135217]: fuse finished with error 0 and tester_r 0 2021-02-10T11:37:24.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:37:24.233 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:37:24.233 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:37:24.568 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:37:24.568 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:37:24.568 INFO:teuthology.orchestra.run:waiting for 900 2021-02-10T11:37:24.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:37:24.568 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:37:24.903 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:37:25.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.307 7f835d734700 1 Processor -- start 2021-02-10T11:37:25.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.307 7f835d734700 1 -- start start 2021-02-10T11:37:25.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.308 7f835d734700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f83580d1870 0x7f83581b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:25.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.308 7f835d734700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f83580f0650 0x7f83580cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:25.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.308 7f835d734700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83580f0b10 con 0x7f83581b0a10 2021-02-10T11:37:25.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.308 7f835d734700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83580e5ed0 con 0x7f83580d1870 2021-02-10T11:37:25.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.308 7f835d734700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83580d1c70 con 0x7f83580f0650 2021-02-10T11:37:25.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.308 7f8357549700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f83580d1870 0x7f83581b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:25.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.308 7f8357549700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f83580d1870 0x7f83581b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54706/0 (socket says 172.21.2.101:54706) 2021-02-10T11:37:25.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.308 7f8357549700 1 -- 172.21.2.101:0/743182665 learned_addr learned my addr 172.21.2.101:0/743182665 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:25.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.308 7f8357d4a700 1 --2- 172.21.2.101:0/743182665 >> v2:172.21.2.108:3300/0 conn(0x7f83580f0650 0x7f83580cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:25.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.308 7f8357549700 1 -- 172.21.2.101:0/743182665 >> v2:172.21.2.108:3300/0 conn(0x7f83580f0650 msgr2=0x7f83580cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:25.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.308 7f8357549700 1 --2- 172.21.2.101:0/743182665 >> v2:172.21.2.108:3300/0 conn(0x7f83580f0650 0x7f83580cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:25.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.308 7f8357549700 1 -- 172.21.2.101:0/743182665 >> v1:172.21.2.108:6789/0 conn(0x7f83581b0a10 legacy=0x7f83580cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:25.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.308 7f8357549700 1 -- 172.21.2.101:0/743182665 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83580d1e90 con 0x7f83580d1870 2021-02-10T11:37:25.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.308 7f8357549700 1 --2- 172.21.2.101:0/743182665 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83580d1870 0x7f83581b03f0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 rx=0x7f8348006700 tx=0x7f8348009b80).ready entity=mon.0 client_cookie=6ea96a68ae6df576 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:25.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.308 7f8356547700 1 -- 172.21.2.101:0/743182665 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8348000c90 con 0x7f83580d1870 2021-02-10T11:37:25.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.308 7f8356547700 1 -- 172.21.2.101:0/743182665 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834800c0c0 con 0x7f83580d1870 2021-02-10T11:37:25.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.308 7f8356547700 1 -- 172.21.2.101:0/743182665 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f834800c5a0 con 0x7f83580d1870 2021-02-10T11:37:25.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.308 7f835d734700 1 -- 172.21.2.101:0/743182665 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83580d1870 msgr2=0x7f83581b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:25.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.308 7f835d734700 1 --2- 172.21.2.101:0/743182665 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83580d1870 0x7f83581b03f0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 rx=0x7f8348006700 tx=0x7f8348009b80).stop 2021-02-10T11:37:25.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.309 7f835d734700 1 -- 172.21.2.101:0/743182665 shutdown_connections 2021-02-10T11:37:25.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.309 7f835d734700 1 --2- 172.21.2.101:0/743182665 >> v2:172.21.2.108:3300/0 conn(0x7f83580f0650 0x7f83580cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:25.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.309 7f835d734700 1 --2- 172.21.2.101:0/743182665 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83580d1870 0x7f83581b03f0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:25.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.309 7f835d734700 1 -- 172.21.2.101:0/743182665 >> 172.21.2.101:0/743182665 conn(0x7f83580ce450 msgr2=0x7f83580d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:25.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.309 7f835d734700 1 -- 172.21.2.101:0/743182665 shutdown_connections 2021-02-10T11:37:25.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.309 7f835d734700 1 -- 172.21.2.101:0/743182665 wait complete. 2021-02-10T11:37:25.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.324 7f835d734700 1 Processor -- start 2021-02-10T11:37:25.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.324 7f835d734700 1 -- start start 2021-02-10T11:37:25.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.324 7f835d734700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83581942b0 0x7f83580b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:25.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.324 7f835d734700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8358195070 0x7f8358195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:25.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.324 7f835d734700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83580bb8e0 0x7f83580bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:25.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.324 7f835d734700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83581946b0 con 0x7f83580bb8e0 2021-02-10T11:37:25.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.324 7f835d734700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83580b3b70 con 0x7f83581942b0 2021-02-10T11:37:25.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.324 7f835d734700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83580b3d90 con 0x7f8358195070 2021-02-10T11:37:25.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.325 7f8357d4a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83580bb8e0 0x7f83580bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:25.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.325 7f8357d4a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83580bb8e0 0x7f83580bbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54714/0 (socket says 172.21.2.101:54714) 2021-02-10T11:37:25.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.325 7f8357d4a700 1 -- 172.21.2.101:0/3276797625 learned_addr learned my addr 172.21.2.101:0/3276797625 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:25.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.325 7f8357549700 1 --2- 172.21.2.101:0/3276797625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83581942b0 0x7f83580b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:25.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.325 7f8356d48700 1 --2- 172.21.2.101:0/3276797625 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8358195070 0x7f8358195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:25.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.325 7f8357549700 1 -- 172.21.2.101:0/3276797625 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8358195070 msgr2=0x7f8358195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:25.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.325 7f8357549700 1 --2- 172.21.2.101:0/3276797625 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8358195070 0x7f8358195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:25.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.325 7f8357549700 1 -- 172.21.2.101:0/3276797625 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83580bb8e0 msgr2=0x7f83580bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:25.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.325 7f8357549700 1 --2- 172.21.2.101:0/3276797625 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83580bb8e0 0x7f83580bbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:25.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.325 7f8357549700 1 -- 172.21.2.101:0/3276797625 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83581a49e0 con 0x7f83581942b0 2021-02-10T11:37:25.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.325 7f8357d4a700 1 --2- 172.21.2.101:0/3276797625 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83580bb8e0 0x7f83580bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:37:25.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.325 7f8357549700 1 --2- 172.21.2.101:0/3276797625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83581942b0 0x7f83580b3550 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f83480093b0 tx=0x7f834800b8b0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:25.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.325 7f834ffff700 1 -- 172.21.2.101:0/3276797625 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8348000c90 con 0x7f83581942b0 2021-02-10T11:37:25.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.325 7f834ffff700 1 -- 172.21.2.101:0/3276797625 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8348006340 con 0x7f83581942b0 2021-02-10T11:37:25.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.325 7f834ffff700 1 -- 172.21.2.101:0/3276797625 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8348002300 con 0x7f83581942b0 2021-02-10T11:37:25.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.325 7f835d734700 1 -- 172.21.2.101:0/3276797625 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83581a49e0 con 0x7f83581942b0 2021-02-10T11:37:25.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.325 7f835d734700 1 -- 172.21.2.101:0/3276797625 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83581a49e0 con 0x7f83581942b0 2021-02-10T11:37:25.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.326 7f834ffff700 1 -- 172.21.2.101:0/3276797625 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8348002020 con 0x7f83581942b0 2021-02-10T11:37:25.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.326 7f834ffff700 1 --2- 172.21.2.101:0/3276797625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f833c018730 0x7f833c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:25.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.326 7f834ffff700 1 -- 172.21.2.101:0/3276797625 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f83480326d0 con 0x7f83581942b0 2021-02-10T11:37:25.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.326 7f8356d48700 1 --2- 172.21.2.101:0/3276797625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f833c018730 0x7f833c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:25.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.327 7f8356d48700 1 --2- 172.21.2.101:0/3276797625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f833c018730 0x7f833c01ac10 crc :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:25.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.358 7f835d734700 1 -- 172.21.2.101:0/3276797625 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83580bc5f0 con 0x7f83581942b0 2021-02-10T11:37:25.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.361 7f834ffff700 1 -- 172.21.2.101:0/3276797625 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8348006810 con 0x7f83581942b0 2021-02-10T11:37:25.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.487 7f835d734700 1 -- 172.21.2.101:0/3276797625 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83580bc5f0 con 0x7f83581942b0 2021-02-10T11:37:25.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.488 7f834ffff700 1 -- 172.21.2.101:0/3276797625 <== mon.1 v2:172.21.2.108:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f8348006340 con 0x7f83581942b0 2021-02-10T11:37:25.495 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:37:25.496 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9360,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2529021396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2529021396},{"type":"v1","addr":"172.21.2.101:6835","nonce":2529021396}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:56.648753","modified":"2021-02-10 11:36:58.653009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6005":{"gid":6005,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2363068042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2363068042},{"type":"v1","addr":"172.21.2.108:6833","nonce":2363068042}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-02-10T11:37:25.496 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 63 2021-02-10T11:37:25.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.491 7f835d734700 1 -- 172.21.2.101:0/3276797625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f833c018730 msgr2=0x7f833c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:25.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.491 7f835d734700 1 --2- 172.21.2.101:0/3276797625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f833c018730 0x7f833c01ac10 crc :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:25.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.491 7f835d734700 1 -- 172.21.2.101:0/3276797625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83581942b0 msgr2=0x7f83580b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:25.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.491 7f835d734700 1 --2- 172.21.2.101:0/3276797625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83581942b0 0x7f83580b3550 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f83480093b0 tx=0x7f834800b8b0).stop 2021-02-10T11:37:25.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.492 7f835d734700 1 -- 172.21.2.101:0/3276797625 shutdown_connections 2021-02-10T11:37:25.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.492 7f835d734700 1 --2- 172.21.2.101:0/3276797625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f833c018730 0x7f833c01ac10 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:25.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.492 7f835d734700 1 --2- 172.21.2.101:0/3276797625 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8358195070 0x7f8358195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:25.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.492 7f835d734700 1 --2- 172.21.2.101:0/3276797625 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f83580bb8e0 0x7f83580bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:25.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.492 7f835d734700 1 --2- 172.21.2.101:0/3276797625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f83581942b0 0x7f83580b3550 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:25.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.492 7f835d734700 1 -- 172.21.2.101:0/3276797625 >> 172.21.2.101:0/3276797625 conn(0x7f83580b3040 msgr2=0x7f83580c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:25.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.492 7f835d734700 1 -- 172.21.2.101:0/3276797625 shutdown_connections 2021-02-10T11:37:25.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:25.492 7f835d734700 1 -- 172.21.2.101:0/3276797625 wait complete. 2021-02-10T11:37:25.506 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok session ls 2021-02-10T11:37:25.902 INFO:teuthology.orchestra.run.gibba008.stdout:[] 2021-02-10T11:37:25.906 INFO:tasks.cephfs.filesystem:_json_asok output: [] 2021-02-10T11:37:25.907 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000679757 2021-02-10T11:37:26.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.316 7f55a6723700 1 Processor -- start 2021-02-10T11:37:26.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.316 7f55a6723700 1 -- start start 2021-02-10T11:37:26.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.317 7f55a6723700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f55a00d1870 0x7f55a01b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:26.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.317 7f55a6723700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f55a01b0ae0 0x7f55a0108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:26.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.317 7f55a6723700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a0109a60 con 0x7f55a00d1c70 2021-02-10T11:37:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.317 7f55a6723700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55a00e5ef0 con 0x7f55a01b0ae0 2021-02-10T11:37:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.317 7f55a6723700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a0109750 con 0x7f55a00d1870 2021-02-10T11:37:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.317 7f55a4d40700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f55a00d1c70 0x7f55a0108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55214/0 (socket says 172.21.2.101:55214) 2021-02-10T11:37:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.317 7f55a4d40700 1 -- 172.21.2.101:0/1958689720 learned_addr learned my addr 172.21.2.101:0/1958689720 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:37:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.317 7f559f7fe700 1 --2- 172.21.2.101:0/1958689720 >> v2:172.21.2.101:3300/0 conn(0x7f55a01b0ae0 0x7f55a0108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.317 7f559effd700 1 -- 172.21.2.101:0/1958689720 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983543069 0 0) 0x7f5598001500 con 0x7f55a00d1c70 2021-02-10T11:37:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.317 7f559effd700 1 -- 172.21.2.101:0/1958689720 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5588002fe0 con 0x7f55a00d1c70 2021-02-10T11:37:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.317 7f559f7fe700 1 -- 172.21.2.101:0/1958689720 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f55a00d1870 msgr2=0x7f55a01b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.317 7f559f7fe700 1 --2- 172.21.2.101:0/1958689720 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f55a00d1870 0x7f55a01b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.317 7f559f7fe700 1 -- 172.21.2.101:0/1958689720 >> v1:172.21.2.101:6789/0 conn(0x7f55a00d1c70 legacy=0x7f55a0108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.317 7f559effd700 1 -- 172.21.2.101:0/1958689720 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2785482662 0 0) 0x7f5598001460 con 0x7f55a00d1c70 2021-02-10T11:37:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.318 7f559f7fe700 1 -- 172.21.2.101:0/1958689720 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a0088070 con 0x7f55a01b0ae0 2021-02-10T11:37:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.318 7f559f7fe700 1 --2- 172.21.2.101:0/1958689720 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f55a01b0ae0 0x7f55a0108940 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 rx=0x7f5594006650 tx=0x7f5594009a00).ready entity=mon.0 client_cookie=22c71a366ff7dc94 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.318 7f559effd700 1 -- 172.21.2.101:0/1958689720 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f55940009b0 con 0x7f55a01b0ae0 2021-02-10T11:37:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.318 7f559effd700 1 -- 172.21.2.101:0/1958689720 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f559400c0c0 con 0x7f55a01b0ae0 2021-02-10T11:37:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.318 7f559effd700 1 -- 172.21.2.101:0/1958689720 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f559400c5a0 con 0x7f55a01b0ae0 2021-02-10T11:37:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.318 7f55a6723700 1 -- 172.21.2.101:0/1958689720 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f55a01b0ae0 msgr2=0x7f55a0108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.318 7f55a6723700 1 --2- 172.21.2.101:0/1958689720 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f55a01b0ae0 0x7f55a0108940 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 rx=0x7f5594006650 tx=0x7f5594009a00).stop 2021-02-10T11:37:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.319 7f55a6723700 1 -- 172.21.2.101:0/1958689720 shutdown_connections 2021-02-10T11:37:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.319 7f55a6723700 1 --2- 172.21.2.101:0/1958689720 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f55a01b0ae0 0x7f55a0108940 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.319 7f55a6723700 1 --2- 172.21.2.101:0/1958689720 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f55a00d1870 0x7f55a01b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.319 7f55a6723700 1 -- 172.21.2.101:0/1958689720 >> 172.21.2.101:0/1958689720 conn(0x7f55a00ce420 msgr2=0x7f55a00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.319 7f55a6723700 1 -- 172.21.2.101:0/1958689720 shutdown_connections 2021-02-10T11:37:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.319 7f55a6723700 1 -- 172.21.2.101:0/1958689720 wait complete. 2021-02-10T11:37:26.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.333 7f55a6723700 1 Processor -- start 2021-02-10T11:37:26.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.333 7f55a6723700 1 -- start start 2021-02-10T11:37:26.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.334 7f55a6723700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f55a00855d0 0x7f55a0084d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:26.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.334 7f55a6723700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f55a009e210 0x7f55a00ba330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:26.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.334 7f55a6723700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f55a009e610 0x7f55a00ba950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:26.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.334 7f55a6723700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a008a530 con 0x7f55a009e610 2021-02-10T11:37:26.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.334 7f55a6723700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a00b9b90 con 0x7f55a009e210 2021-02-10T11:37:26.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.334 7f55a6723700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a00b9de0 con 0x7f55a00855d0 2021-02-10T11:37:26.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.334 7f55a4d40700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f55a009e610 0x7f55a00ba950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:26.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.334 7f55a4d40700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f55a009e610 0x7f55a00ba950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54726/0 (socket says 172.21.2.101:54726) 2021-02-10T11:37:26.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.334 7f55a4d40700 1 -- 172.21.2.101:0/3589437671 learned_addr learned my addr 172.21.2.101:0/3589437671 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.334 7f55a4d40700 1 -- 172.21.2.101:0/3589437671 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f55a00855d0 msgr2=0x7f55a0084d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.334 7f55a4d40700 1 --2- 172.21.2.101:0/3589437671 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f55a00855d0 0x7f55a0084d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.334 7f55a4d40700 1 -- 172.21.2.101:0/3589437671 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f55a009e210 msgr2=0x7f55a00ba330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.334 7f55a4d40700 1 --2- 172.21.2.101:0/3589437671 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f55a009e210 0x7f55a00ba330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.334 7f55a4d40700 1 -- 172.21.2.101:0/3589437671 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a009ebf0 con 0x7f55a009e610 2021-02-10T11:37:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.334 7f55a4d40700 1 --2- 172.21.2.101:0/3589437671 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f55a009e610 0x7f55a00ba950 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 rx=0x7f5598009960 tx=0x7f559800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.335 7f559cff9700 1 -- 172.21.2.101:0/3589437671 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f55980060c0 con 0x7f55a009e610 2021-02-10T11:37:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.335 7f559cff9700 1 -- 172.21.2.101:0/3589437671 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5598006380 con 0x7f55a009e610 2021-02-10T11:37:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.335 7f559cff9700 1 -- 172.21.2.101:0/3589437671 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5598004720 con 0x7f55a009e610 2021-02-10T11:37:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.335 7f55a6723700 1 -- 172.21.2.101:0/3589437671 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a009ebf0 con 0x7f55a009e610 2021-02-10T11:37:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.335 7f55a6723700 1 -- 172.21.2.101:0/3589437671 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a00bb260 con 0x7f55a009e610 2021-02-10T11:37:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.335 7f559cff9700 1 -- 172.21.2.101:0/3589437671 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5598006560 con 0x7f55a009e610 2021-02-10T11:37:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.336 7f559cff9700 1 --2- 172.21.2.101:0/3589437671 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f55800186c0 0x7f558001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:26.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.336 7f559cff9700 1 -- 172.21.2.101:0/3589437671 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5598032900 con 0x7f55a009e610 2021-02-10T11:37:26.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.336 7f559ffff700 1 --2- 172.21.2.101:0/3589437671 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f55800186c0 0x7f558001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:26.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.336 7f559ffff700 1 --2- 172.21.2.101:0/3589437671 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f55800186c0 0x7f558001abc0 crc :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:26.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.367 7f55a6723700 1 -- 172.21.2.101:0/3589437671 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a00b9db0 con 0x7f55a009e610 2021-02-10T11:37:26.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.370 7f559cff9700 1 -- 172.21.2.101:0/3589437671 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5598000cc0 con 0x7f55a009e610 2021-02-10T11:37:26.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.497 7f55a6723700 1 -- 172.21.2.101:0/3589437671 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000679757", "target": ["mgr", ""]}) v1 -- 0x7f55a00b9b90 con 0x7f55800186c0 2021-02-10T11:37:26.507 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:37:26.501 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:37:26.507 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:37:26.501 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:37:26.507 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:37:26.501 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:37:26.507 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:37:26.501 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:37:26.507 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:37:26.501 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:37:26.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.531 7f559cff9700 1 -- 172.21.2.101:0/3589437671 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f55900009b0 con 0x7f55800186c0 2021-02-10T11:37:26.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.562 7f55a6723700 1 -- 172.21.2.101:0/3589437671 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f55800186c0 msgr2=0x7f558001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:26.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.562 7f55a6723700 1 --2- 172.21.2.101:0/3589437671 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f55800186c0 0x7f558001abc0 crc :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:26.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.562 7f55a6723700 1 -- 172.21.2.101:0/3589437671 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f55a009e610 msgr2=0x7f55a00ba950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:26.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.562 7f55a6723700 1 --2- 172.21.2.101:0/3589437671 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f55a009e610 0x7f55a00ba950 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 rx=0x7f5598009960 tx=0x7f559800bcf0).stop 2021-02-10T11:37:26.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.562 7f55a6723700 1 -- 172.21.2.101:0/3589437671 shutdown_connections 2021-02-10T11:37:26.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.562 7f55a6723700 1 --2- 172.21.2.101:0/3589437671 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f55a009e610 0x7f55a00ba950 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:26.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.562 7f55a6723700 1 --2- 172.21.2.101:0/3589437671 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f55a009e210 0x7f55a00ba330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:26.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.562 7f55a6723700 1 --2- 172.21.2.101:0/3589437671 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f55800186c0 0x7f558001abc0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:26.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.562 7f55a6723700 1 --2- 172.21.2.101:0/3589437671 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f55a00855d0 0x7f55a0084d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:26.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.562 7f55a6723700 1 -- 172.21.2.101:0/3589437671 >> 172.21.2.101:0/3589437671 conn(0x7f55a00bc070 msgr2=0x7f55a00a0510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:26.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.563 7f55a6723700 1 -- 172.21.2.101:0/3589437671 shutdown_connections 2021-02-10T11:37:26.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.563 7f55a6723700 1 -- 172.21.2.101:0/3589437671 wait complete. 2021-02-10T11:37:26.576 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:37:27.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.980 7fd8fdfca700 1 Processor -- start 2021-02-10T11:37:27.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.981 7fd8fdfca700 1 -- start start 2021-02-10T11:37:27.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.981 7fd8fdfca700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd8f80d1870 0x7fd8f81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:27.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.981 7fd8fdfca700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8f81b0a10 0x7fd8f80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:27.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.981 7fd8fdfca700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f80f0b10 con 0x7fd8f80d1c70 2021-02-10T11:37:27.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.981 7fd8fdfca700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f80e5ed0 con 0x7fd8f80d1870 2021-02-10T11:37:27.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.981 7fd8fdfca700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f80f0650 con 0x7fd8f81b0a10 2021-02-10T11:37:27.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.982 7fd8f6ffd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8f81b0a10 0x7fd8f80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:27.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.982 7fd8f77fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd8f80d1870 0x7fd8f81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:27.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.982 7fd8f7fff700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fd8f80d1c70 0x7fd8f80cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:48652/0 (socket says 172.21.2.101:48652) 2021-02-10T11:37:27.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.982 7fd8f7fff700 1 -- 172.21.2.101:0/3535873746 learned_addr learned my addr 172.21.2.101:0/3535873746 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:37:27.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.982 7fd8f6ffd700 1 -- 172.21.2.101:0/3535873746 >> v2:172.21.2.108:3300/0 conn(0x7fd8f80d1870 msgr2=0x7fd8f81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:27.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.982 7fd8f6ffd700 1 --2- 172.21.2.101:0/3535873746 >> v2:172.21.2.108:3300/0 conn(0x7fd8f80d1870 0x7fd8f81b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:27.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.982 7fd8f6ffd700 1 -- 172.21.2.101:0/3535873746 >> v1:172.21.2.108:6789/0 conn(0x7fd8f80d1c70 legacy=0x7fd8f80cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:27.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.982 7fd8f6ffd700 1 -- 172.21.2.101:0/3535873746 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f80f0870 con 0x7fd8f81b0a10 2021-02-10T11:37:27.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.982 7fd8f77fe700 1 --2- 172.21.2.101:0/3535873746 >> v2:172.21.2.108:3300/0 conn(0x7fd8f80d1870 0x7fd8f81b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:37:27.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.983 7fd8f6ffd700 1 --2- 172.21.2.101:0/3535873746 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8f81b0a10 0x7fd8f80cf330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fd8e8006650 tx=0x7fd8e8009a80).ready entity=mon.2 client_cookie=cbfcd1345625b868 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:27.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.983 7fd8f67fc700 1 -- 172.21.2.101:0/3535873746 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8e80009b0 con 0x7fd8f81b0a10 2021-02-10T11:37:27.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.983 7fd8f67fc700 1 -- 172.21.2.101:0/3535873746 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e800c0c0 con 0x7fd8f81b0a10 2021-02-10T11:37:27.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.983 7fd8f67fc700 1 -- 172.21.2.101:0/3535873746 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8e800c5e0 con 0x7fd8f81b0a10 2021-02-10T11:37:27.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.983 7fd8fdfca700 1 -- 172.21.2.101:0/3535873746 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8f81b0a10 msgr2=0x7fd8f80cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:27.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.983 7fd8fdfca700 1 --2- 172.21.2.101:0/3535873746 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8f81b0a10 0x7fd8f80cf330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fd8e8006650 tx=0x7fd8e8009a80).stop 2021-02-10T11:37:27.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.984 7fd8fdfca700 1 -- 172.21.2.101:0/3535873746 shutdown_connections 2021-02-10T11:37:27.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.984 7fd8fdfca700 1 --2- 172.21.2.101:0/3535873746 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8f81b0a10 0x7fd8f80cf330 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:27.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.984 7fd8fdfca700 1 --2- 172.21.2.101:0/3535873746 >> v2:172.21.2.108:3300/0 conn(0x7fd8f80d1870 0x7fd8f81b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:27.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.984 7fd8fdfca700 1 -- 172.21.2.101:0/3535873746 >> 172.21.2.101:0/3535873746 conn(0x7fd8f80ce450 msgr2=0x7fd8f80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:27.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.984 7fd8fdfca700 1 -- 172.21.2.101:0/3535873746 shutdown_connections 2021-02-10T11:37:27.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.984 7fd8fdfca700 1 -- 172.21.2.101:0/3535873746 wait complete. 2021-02-10T11:37:27.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.998 7fd8fdfca700 1 Processor -- start 2021-02-10T11:37:27.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.998 7fd8fdfca700 1 -- start start 2021-02-10T11:37:27.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.999 7fd8fdfca700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd8f80aab30 0x7fd8f808ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:27.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.999 7fd8fdfca700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8f808f2a0 0x7fd8f808f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:27.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.999 7fd8fdfca700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8f808fcc0 0x7fd8f819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:27.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.999 7fd8fdfca700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f8197820 con 0x7fd8f808fcc0 2021-02-10T11:37:27.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.999 7fd8fdfca700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f81ac2a0 con 0x7fd8f80aab30 2021-02-10T11:37:27.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.999 7fd8fdfca700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f81abde0 con 0x7fd8f808f2a0 2021-02-10T11:37:27.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.999 7fd8f7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8f808fcc0 0x7fd8f819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:27.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.999 7fd8f7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8f808fcc0 0x7fd8f819a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54752/0 (socket says 172.21.2.101:54752) 2021-02-10T11:37:27.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.999 7fd8f7fff700 1 -- 172.21.2.101:0/2083837245 learned_addr learned my addr 172.21.2.101:0/2083837245 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:27.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.999 7fd8f7fff700 1 -- 172.21.2.101:0/2083837245 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8f808f2a0 msgr2=0x7fd8f808f6a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:37:27.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.999 7fd8f7fff700 1 --2- 172.21.2.101:0/2083837245 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8f808f2a0 0x7fd8f808f6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:27.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.999 7fd8f7fff700 1 -- 172.21.2.101:0/2083837245 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd8f80aab30 msgr2=0x7fd8f808ec80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:37:27.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.999 7fd8f7fff700 1 --2- 172.21.2.101:0/2083837245 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd8f80aab30 0x7fd8f808ec80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:27.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.999 7fd8f7fff700 1 -- 172.21.2.101:0/2083837245 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f819a8f0 con 0x7fd8f808fcc0 2021-02-10T11:37:27.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:26.999 7fd8f7fff700 1 --2- 172.21.2.101:0/2083837245 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8f808fcc0 0x7fd8f819a2d0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 rx=0x7fd8f0007fc0 tx=0x7fd8f000ac80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:27.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.000 7fd8effff700 1 -- 172.21.2.101:0/2083837245 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8f0007020 con 0x7fd8f808fcc0 2021-02-10T11:37:27.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.000 7fd8effff700 1 -- 172.21.2.101:0/2083837245 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f00072e0 con 0x7fd8f808fcc0 2021-02-10T11:37:27.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.000 7fd8fdfca700 1 -- 172.21.2.101:0/2083837245 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f819a8f0 con 0x7fd8f808fcc0 2021-02-10T11:37:27.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.000 7fd8fdfca700 1 -- 172.21.2.101:0/2083837245 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f819ae80 con 0x7fd8f808fcc0 2021-02-10T11:37:27.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.000 7fd8effff700 1 -- 172.21.2.101:0/2083837245 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8f00046a0 con 0x7fd8f808fcc0 2021-02-10T11:37:27.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.001 7fd8effff700 1 -- 172.21.2.101:0/2083837245 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd8f000adc0 con 0x7fd8f808fcc0 2021-02-10T11:37:27.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.001 7fd8effff700 1 --2- 172.21.2.101:0/2083837245 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8dc0186c0 0x7fd8dc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:27.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.001 7fd8effff700 1 -- 172.21.2.101:0/2083837245 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd8f00328d0 con 0x7fd8f808fcc0 2021-02-10T11:37:27.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.001 7fd8f77fe700 1 --2- 172.21.2.101:0/2083837245 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8dc0186c0 0x7fd8dc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:27.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.001 7fd8f77fe700 1 --2- 172.21.2.101:0/2083837245 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8dc0186c0 0x7fd8dc01abc0 crc :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:27.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.031 7fd8fdfca700 1 -- 172.21.2.101:0/2083837245 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f80ebc00 con 0x7fd8f808fcc0 2021-02-10T11:37:27.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.033 7fd8effff700 1 -- 172.21.2.101:0/2083837245 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd8f00011f0 con 0x7fd8f808fcc0 2021-02-10T11:37:27.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.162 7fd8fdfca700 1 -- 172.21.2.101:0/2083837245 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8f80f0b10 con 0x7fd8f808fcc0 2021-02-10T11:37:27.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.162 7fd8effff700 1 -- 172.21.2.101:0/2083837245 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+2101 (secure 0 0 0) 0x7fd8f000adc0 con 0x7fd8f808fcc0 2021-02-10T11:37:27.168 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:37:27.168 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9360,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2529021396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2529021396},{"type":"v1","addr":"172.21.2.101:6835","nonce":2529021396}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:56.648753","modified":"2021-02-10 11:36:58.653009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6005":{"gid":6005,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2363068042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2363068042},{"type":"v1","addr":"172.21.2.108:6833","nonce":2363068042}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-02-10T11:37:27.169 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 63 2021-02-10T11:37:27.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.164 7fd8fdfca700 1 -- 172.21.2.101:0/2083837245 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8dc0186c0 msgr2=0x7fd8dc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:27.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.164 7fd8fdfca700 1 --2- 172.21.2.101:0/2083837245 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8dc0186c0 0x7fd8dc01abc0 crc :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:27.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.164 7fd8fdfca700 1 -- 172.21.2.101:0/2083837245 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8f808fcc0 msgr2=0x7fd8f819a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:27.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.164 7fd8fdfca700 1 --2- 172.21.2.101:0/2083837245 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8f808fcc0 0x7fd8f819a2d0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 rx=0x7fd8f0007fc0 tx=0x7fd8f000ac80).stop 2021-02-10T11:37:27.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.164 7fd8fdfca700 1 -- 172.21.2.101:0/2083837245 shutdown_connections 2021-02-10T11:37:27.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.164 7fd8fdfca700 1 --2- 172.21.2.101:0/2083837245 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8dc0186c0 0x7fd8dc01abc0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:27.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.164 7fd8fdfca700 1 --2- 172.21.2.101:0/2083837245 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd8f808f2a0 0x7fd8f808f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:27.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.164 7fd8fdfca700 1 --2- 172.21.2.101:0/2083837245 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd8f808fcc0 0x7fd8f819a2d0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:27.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.164 7fd8fdfca700 1 --2- 172.21.2.101:0/2083837245 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd8f80aab30 0x7fd8f808ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:27.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.164 7fd8fdfca700 1 -- 172.21.2.101:0/2083837245 >> 172.21.2.101:0/2083837245 conn(0x7fd8f819c560 msgr2=0x7fd8f80ebe30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:27.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.165 7fd8fdfca700 1 -- 172.21.2.101:0/2083837245 shutdown_connections 2021-02-10T11:37:27.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.165 7fd8fdfca700 1 -- 172.21.2.101:0/2083837245 wait complete. 2021-02-10T11:37:27.178 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok session ls 2021-02-10T11:37:27.576 INFO:teuthology.orchestra.run.gibba008.stdout:[ 2021-02-10T11:37:27.576 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:37:27.576 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 9594, 2021-02-10T11:37:27.576 INFO:teuthology.orchestra.run.gibba008.stdout: "entity": { 2021-02-10T11:37:27.576 INFO:teuthology.orchestra.run.gibba008.stdout: "name": { 2021-02-10T11:37:27.577 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "client", 2021-02-10T11:37:27.577 INFO:teuthology.orchestra.run.gibba008.stdout: "num": 9594 2021-02-10T11:37:27.577 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:27.577 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": { 2021-02-10T11:37:27.577 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "any", 2021-02-10T11:37:27.577 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:37:27.577 INFO:teuthology.orchestra.run.gibba008.stdout: "nonce": 1857925112 2021-02-10T11:37:27.577 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:27.577 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:27.577 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "open", 2021-02-10T11:37:27.577 INFO:teuthology.orchestra.run.gibba008.stdout: "num_leases": 1, 2021-02-10T11:37:27.577 INFO:teuthology.orchestra.run.gibba008.stdout: "num_caps": 7, 2021-02-10T11:37:27.577 INFO:teuthology.orchestra.run.gibba008.stdout: "request_load_avg": 13, 2021-02-10T11:37:27.577 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 1.066940199, 2021-02-10T11:37:27.578 INFO:teuthology.orchestra.run.gibba008.stdout: "requests_in_flight": 1, 2021-02-10T11:37:27.578 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": 0, 2021-02-10T11:37:27.578 INFO:teuthology.orchestra.run.gibba008.stdout: "reconnecting": false, 2021-02-10T11:37:27.578 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps": { 2021-02-10T11:37:27.578 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:27.578 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:37:27.578 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:27.578 INFO:teuthology.orchestra.run.gibba008.stdout: "release_caps": { 2021-02-10T11:37:27.578 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:27.578 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:37:27.578 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:27.578 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle": { 2021-02-10T11:37:27.578 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:27.579 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 2.5 2021-02-10T11:37:27.579 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:27.579 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle2o": { 2021-02-10T11:37:27.579 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:27.579 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 0.5 2021-02-10T11:37:27.579 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:27.579 INFO:teuthology.orchestra.run.gibba008.stdout: "session_cache_liveness": { 2021-02-10T11:37:27.579 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 19.95123699433621, 2021-02-10T11:37:27.579 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 300 2021-02-10T11:37:27.579 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:27.579 INFO:teuthology.orchestra.run.gibba008.stdout: "cap_acquisition": { 2021-02-10T11:37:27.579 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:27.579 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 10 2021-02-10T11:37:27.580 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:27.580 INFO:teuthology.orchestra.run.gibba008.stdout: "inst": "client.9594 172.21.2.101:0/1857925112", 2021-02-10T11:37:27.580 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": [], 2021-02-10T11:37:27.580 INFO:teuthology.orchestra.run.gibba008.stdout: "prealloc_inos": [ 2021-02-10T11:37:27.580 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:37:27.580 INFO:teuthology.orchestra.run.gibba008.stdout: "start": 1099511627782, 2021-02-10T11:37:27.580 INFO:teuthology.orchestra.run.gibba008.stdout: "length": 995 2021-02-10T11:37:27.580 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:27.580 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:37:27.580 INFO:teuthology.orchestra.run.gibba008.stdout: "used_inos": [ 2021-02-10T11:37:27.580 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:37:27.580 INFO:teuthology.orchestra.run.gibba008.stdout: "start": 1099511627781, 2021-02-10T11:37:27.580 INFO:teuthology.orchestra.run.gibba008.stdout: "length": 1 2021-02-10T11:37:27.580 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:27.581 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:37:27.581 INFO:teuthology.orchestra.run.gibba008.stdout: "client_metadata": { 2021-02-10T11:37:27.581 INFO:teuthology.orchestra.run.gibba008.stdout: "features": "0x0000000000001fff", 2021-02-10T11:37:27.581 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:37:27.581 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:37:27.581 INFO:teuthology.orchestra.run.gibba008.stdout: "entity_id": "y", 2021-02-10T11:37:27.581 INFO:teuthology.orchestra.run.gibba008.stdout: "hostname": "gibba001", 2021-02-10T11:37:27.581 INFO:teuthology.orchestra.run.gibba008.stdout: "pid": "24165", 2021-02-10T11:37:27.581 INFO:teuthology.orchestra.run.gibba008.stdout: "root": "/" 2021-02-10T11:37:27.581 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:27.581 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:27.581 INFO:teuthology.orchestra.run.gibba008.stdout:] 2021-02-10T11:37:27.582 INFO:tasks.cephfs.filesystem:_json_asok output: [ { "id": 9594, "entity": { "name": { "type": "client", "num": 9594 }, "addr": { "type": "any", "addr": "172.21.2.101:0", "nonce": 1857925112 } }, "state": "open", "num_leases": 1, "num_caps": 7, "request_load_avg": 13, "uptime": 1.066940199, "requests_in_flight": 1, "completed_requests": 0, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 2.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 19.95123699433621, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 10 }, "inst": "client.9594 172.21.2.101:0/1857925112", "completed_requests": [], "prealloc_inos": [ { "start": 1099511627782, "length": 995 } ], "used_inos": [ { "start": 1099511627781, "length": 1 } ], "client_metadata": { "features": "0x0000000000001fff", "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", "entity_id": "y", "hostname": "gibba001", "pid": "24165", "root": "/" } } ] 2021-02-10T11:37:27.583 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:37:28.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.988 7f6303374700 1 Processor -- start 2021-02-10T11:37:28.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.988 7f6303374700 1 -- start start 2021-02-10T11:37:28.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.988 7f6303374700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62fc0f05f0 0x7f62fc0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:28.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.988 7f6303374700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62fc1b0c70 con 0x7f62fc1b0870 2021-02-10T11:37:28.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.988 7f6303374700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62fc0f0b50 con 0x7f62fc1b0470 2021-02-10T11:37:28.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.988 7f6303374700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc0e5ea0 con 0x7f62fc0f05f0 2021-02-10T11:37:28.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.989 7f630098f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f62fc1b0870 0x7f62fc0efe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55254/0 (socket says 172.21.2.101:55254) 2021-02-10T11:37:28.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.989 7f630098f700 1 -- 172.21.2.101:0/145146235 learned_addr learned my addr 172.21.2.101:0/145146235 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:37:28.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.989 7f6301991700 1 --2- 172.21.2.101:0/145146235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62fc0f05f0 0x7f62fc0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:28.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.989 7f62effff700 1 -- 172.21.2.101:0/145146235 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996609871 0 0) 0x7f62f0001500 con 0x7f62fc1b0870 2021-02-10T11:37:28.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.989 7f6301991700 1 -- 172.21.2.101:0/145146235 >> v1:172.21.2.108:6789/0 conn(0x7f62fc1b0470 legacy=0x7f62fc0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:28.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.989 7f6301991700 1 -- 172.21.2.101:0/145146235 >> v1:172.21.2.101:6789/0 conn(0x7f62fc1b0870 legacy=0x7f62fc0efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:28.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.989 7f6301991700 1 -- 172.21.2.101:0/145146235 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62fc0e52e0 con 0x7f62fc0f05f0 2021-02-10T11:37:28.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.989 7f6301991700 1 --2- 172.21.2.101:0/145146235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62fc0f05f0 0x7f62fc0f20e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f62f4006ad0 tx=0x7f62f4009ad0).ready entity=mon.2 client_cookie=33483c0cc99dc15 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:28.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.990 7f62effff700 1 -- 172.21.2.101:0/145146235 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f62f40022e0 con 0x7f62fc0f05f0 2021-02-10T11:37:28.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.990 7f62effff700 1 -- 172.21.2.101:0/145146235 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f4006410 con 0x7f62fc0f05f0 2021-02-10T11:37:28.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.990 7f62effff700 1 -- 172.21.2.101:0/145146235 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f62f4002620 con 0x7f62fc0f05f0 2021-02-10T11:37:28.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.990 7f6303374700 1 -- 172.21.2.101:0/145146235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62fc0f05f0 msgr2=0x7f62fc0f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:28.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.990 7f6303374700 1 --2- 172.21.2.101:0/145146235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62fc0f05f0 0x7f62fc0f20e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f62f4006ad0 tx=0x7f62f4009ad0).stop 2021-02-10T11:37:28.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.990 7f6303374700 1 -- 172.21.2.101:0/145146235 shutdown_connections 2021-02-10T11:37:28.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.990 7f6303374700 1 --2- 172.21.2.101:0/145146235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62fc0f05f0 0x7f62fc0f20e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:28.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.990 7f6303374700 1 -- 172.21.2.101:0/145146235 >> 172.21.2.101:0/145146235 conn(0x7f62fc0ce3e0 msgr2=0x7f62fc0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:28.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.991 7f6303374700 1 -- 172.21.2.101:0/145146235 shutdown_connections 2021-02-10T11:37:28.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:27.991 7f6303374700 1 -- 172.21.2.101:0/145146235 wait complete. 2021-02-10T11:37:28.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.005 7f6303374700 1 Processor -- start 2021-02-10T11:37:28.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.005 7f6303374700 1 -- start start 2021-02-10T11:37:28.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.006 7f6303374700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62fc1944c0 0x7f62fc1905d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:28.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.006 7f6303374700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62fc19c7b0 0x7f62fc19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:28.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.006 7f6303374700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62fc19d1d0 0x7f62fc0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:28.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.006 7f6303374700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc193f40 con 0x7f62fc19d1d0 2021-02-10T11:37:28.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.006 7f6303374700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc0b7e90 con 0x7f62fc19c7b0 2021-02-10T11:37:28.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.006 7f6303374700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc0b80b0 con 0x7f62fc1944c0 2021-02-10T11:37:28.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.006 7f6301991700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62fc19d1d0 0x7f62fc0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:28.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.006 7f6301991700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62fc19d1d0 0x7f62fc0b7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54770/0 (socket says 172.21.2.101:54770) 2021-02-10T11:37:28.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.006 7f6301190700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62fc1944c0 0x7f62fc1905d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:28.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.006 7f6301991700 1 -- 172.21.2.101:0/2328079507 learned_addr learned my addr 172.21.2.101:0/2328079507 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:28.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.006 7f630098f700 1 --2- 172.21.2.101:0/2328079507 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62fc19c7b0 0x7f62fc19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:28.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.006 7f6301991700 1 -- 172.21.2.101:0/2328079507 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62fc1944c0 msgr2=0x7f62fc1905d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:28.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.006 7f6301991700 1 --2- 172.21.2.101:0/2328079507 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62fc1944c0 0x7f62fc1905d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:28.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.006 7f6301991700 1 -- 172.21.2.101:0/2328079507 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62fc19c7b0 msgr2=0x7f62fc19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:28.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.006 7f6301991700 1 --2- 172.21.2.101:0/2328079507 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62fc19c7b0 0x7f62fc19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:28.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.006 7f6301991700 1 -- 172.21.2.101:0/2328079507 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62fc0b82d0 con 0x7f62fc19d1d0 2021-02-10T11:37:28.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.006 7f6301190700 1 --2- 172.21.2.101:0/2328079507 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62fc1944c0 0x7f62fc1905d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:37:28.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.007 7f6301991700 1 --2- 172.21.2.101:0/2328079507 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62fc19d1d0 0x7f62fc0b7870 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 rx=0x7f62f4009670 tx=0x7f62f400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:28.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.007 7f62edffb700 1 -- 172.21.2.101:0/2328079507 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f62f400bd80 con 0x7f62fc19d1d0 2021-02-10T11:37:28.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.007 7f62edffb700 1 -- 172.21.2.101:0/2328079507 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f4002080 con 0x7f62fc19d1d0 2021-02-10T11:37:28.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.007 7f62edffb700 1 -- 172.21.2.101:0/2328079507 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f62f40024d0 con 0x7f62fc19d1d0 2021-02-10T11:37:28.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.007 7f6303374700 1 -- 172.21.2.101:0/2328079507 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62fc0b82d0 con 0x7f62fc19d1d0 2021-02-10T11:37:28.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.007 7f6303374700 1 -- 172.21.2.101:0/2328079507 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62fc192e70 con 0x7f62fc19d1d0 2021-02-10T11:37:28.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.007 7f62edffb700 1 -- 172.21.2.101:0/2328079507 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f62f4002260 con 0x7f62fc19d1d0 2021-02-10T11:37:28.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.007 7f62edffb700 1 --2- 172.21.2.101:0/2328079507 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f62dc0186c0 0x7f62dc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:28.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.008 7f62edffb700 1 -- 172.21.2.101:0/2328079507 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f62f4032580 con 0x7f62fc19d1d0 2021-02-10T11:37:28.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.008 7f6301190700 1 --2- 172.21.2.101:0/2328079507 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f62dc0186c0 0x7f62dc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:28.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.008 7f6301190700 1 --2- 172.21.2.101:0/2328079507 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f62dc0186c0 0x7f62dc01abc0 crc :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:28.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.038 7f6303374700 1 -- 172.21.2.101:0/2328079507 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62fc0eddb0 con 0x7f62fc19d1d0 2021-02-10T11:37:28.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.041 7f62edffb700 1 -- 172.21.2.101:0/2328079507 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f62f40093e0 con 0x7f62fc19d1d0 2021-02-10T11:37:28.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.168 7f6303374700 1 -- 172.21.2.101:0/2328079507 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62fc0e52e0 con 0x7f62fc19d1d0 2021-02-10T11:37:28.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.168 7f62edffb700 1 -- 172.21.2.101:0/2328079507 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f62f400bd80 con 0x7f62fc19d1d0 2021-02-10T11:37:28.174 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:37:28.174 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9360,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2529021396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2529021396},{"type":"v1","addr":"172.21.2.101:6835","nonce":2529021396}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:56.648753","modified":"2021-02-10 11:36:58.653009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6005":{"gid":6005,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2363068042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2363068042},{"type":"v1","addr":"172.21.2.108:6833","nonce":2363068042}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-02-10T11:37:28.174 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 63 2021-02-10T11:37:28.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.170 7f6303374700 1 -- 172.21.2.101:0/2328079507 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f62dc0186c0 msgr2=0x7f62dc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:28.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.170 7f6303374700 1 --2- 172.21.2.101:0/2328079507 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f62dc0186c0 0x7f62dc01abc0 crc :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:28.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.170 7f6303374700 1 -- 172.21.2.101:0/2328079507 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62fc19d1d0 msgr2=0x7f62fc0b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:28.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.170 7f6303374700 1 --2- 172.21.2.101:0/2328079507 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62fc19d1d0 0x7f62fc0b7870 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 rx=0x7f62f4009670 tx=0x7f62f400b990).stop 2021-02-10T11:37:28.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.170 7f6303374700 1 -- 172.21.2.101:0/2328079507 shutdown_connections 2021-02-10T11:37:28.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.170 7f6303374700 1 --2- 172.21.2.101:0/2328079507 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62fc19d1d0 0x7f62fc0b7870 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:28.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.170 7f6303374700 1 --2- 172.21.2.101:0/2328079507 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62fc19c7b0 0x7f62fc19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:28.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.170 7f6303374700 1 --2- 172.21.2.101:0/2328079507 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f62dc0186c0 0x7f62dc01abc0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:28.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.170 7f6303374700 1 --2- 172.21.2.101:0/2328079507 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62fc1944c0 0x7f62fc1905d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:28.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.170 7f6303374700 1 -- 172.21.2.101:0/2328079507 >> 172.21.2.101:0/2328079507 conn(0x7f62fc093c20 msgr2=0x7f62fc092f70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:28.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.170 7f6303374700 1 -- 172.21.2.101:0/2328079507 shutdown_connections 2021-02-10T11:37:28.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:28.171 7f6303374700 1 -- 172.21.2.101:0/2328079507 wait complete. 2021-02-10T11:37:28.184 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok session ls 2021-02-10T11:37:28.582 INFO:teuthology.orchestra.run.gibba008.stdout:[ 2021-02-10T11:37:28.582 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:37:28.582 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 9594, 2021-02-10T11:37:28.582 INFO:teuthology.orchestra.run.gibba008.stdout: "entity": { 2021-02-10T11:37:28.583 INFO:teuthology.orchestra.run.gibba008.stdout: "name": { 2021-02-10T11:37:28.583 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "client", 2021-02-10T11:37:28.583 INFO:teuthology.orchestra.run.gibba008.stdout: "num": 9594 2021-02-10T11:37:28.583 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:28.583 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": { 2021-02-10T11:37:28.583 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "any", 2021-02-10T11:37:28.583 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:37:28.583 INFO:teuthology.orchestra.run.gibba008.stdout: "nonce": 1857925112 2021-02-10T11:37:28.583 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:28.583 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:28.583 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "open", 2021-02-10T11:37:28.583 INFO:teuthology.orchestra.run.gibba008.stdout: "num_leases": 1, 2021-02-10T11:37:28.584 INFO:teuthology.orchestra.run.gibba008.stdout: "num_caps": 7, 2021-02-10T11:37:28.584 INFO:teuthology.orchestra.run.gibba008.stdout: "request_load_avg": 13, 2021-02-10T11:37:28.584 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 2.0738839160000002, 2021-02-10T11:37:28.584 INFO:teuthology.orchestra.run.gibba008.stdout: "requests_in_flight": 1, 2021-02-10T11:37:28.584 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": 0, 2021-02-10T11:37:28.584 INFO:teuthology.orchestra.run.gibba008.stdout: "reconnecting": false, 2021-02-10T11:37:28.584 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps": { 2021-02-10T11:37:28.584 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:28.584 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:37:28.584 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:28.584 INFO:teuthology.orchestra.run.gibba008.stdout: "release_caps": { 2021-02-10T11:37:28.585 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:28.585 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:37:28.585 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:28.585 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle": { 2021-02-10T11:37:28.585 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:28.585 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 2.5 2021-02-10T11:37:28.585 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:28.585 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle2o": { 2021-02-10T11:37:28.585 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:28.585 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 0.5 2021-02-10T11:37:28.585 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:28.585 INFO:teuthology.orchestra.run.gibba008.stdout: "session_cache_liveness": { 2021-02-10T11:37:28.586 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 19.904873717015004, 2021-02-10T11:37:28.586 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 300 2021-02-10T11:37:28.586 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:28.586 INFO:teuthology.orchestra.run.gibba008.stdout: "cap_acquisition": { 2021-02-10T11:37:28.586 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:28.586 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 10 2021-02-10T11:37:28.586 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:28.586 INFO:teuthology.orchestra.run.gibba008.stdout: "inst": "client.9594 172.21.2.101:0/1857925112", 2021-02-10T11:37:28.586 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": [], 2021-02-10T11:37:28.587 INFO:teuthology.orchestra.run.gibba008.stdout: "prealloc_inos": [ 2021-02-10T11:37:28.587 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:37:28.587 INFO:teuthology.orchestra.run.gibba008.stdout: "start": 1099511627782, 2021-02-10T11:37:28.587 INFO:teuthology.orchestra.run.gibba008.stdout: "length": 995 2021-02-10T11:37:28.587 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:28.587 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:37:28.587 INFO:teuthology.orchestra.run.gibba008.stdout: "used_inos": [ 2021-02-10T11:37:28.587 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:37:28.587 INFO:teuthology.orchestra.run.gibba008.stdout: "start": 1099511627781, 2021-02-10T11:37:28.587 INFO:teuthology.orchestra.run.gibba008.stdout: "length": 1 2021-02-10T11:37:28.588 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:28.588 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:37:28.588 INFO:teuthology.orchestra.run.gibba008.stdout: "client_metadata": { 2021-02-10T11:37:28.588 INFO:teuthology.orchestra.run.gibba008.stdout: "features": "0x0000000000001fff", 2021-02-10T11:37:28.588 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:37:28.588 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:37:28.588 INFO:teuthology.orchestra.run.gibba008.stdout: "entity_id": "y", 2021-02-10T11:37:28.588 INFO:teuthology.orchestra.run.gibba008.stdout: "hostname": "gibba001", 2021-02-10T11:37:28.588 INFO:teuthology.orchestra.run.gibba008.stdout: "pid": "24165", 2021-02-10T11:37:28.588 INFO:teuthology.orchestra.run.gibba008.stdout: "root": "/" 2021-02-10T11:37:28.588 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:28.589 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:28.589 INFO:teuthology.orchestra.run.gibba008.stdout:] 2021-02-10T11:37:28.590 INFO:tasks.cephfs.filesystem:_json_asok output: [ { "id": 9594, "entity": { "name": { "type": "client", "num": 9594 }, "addr": { "type": "any", "addr": "172.21.2.101:0", "nonce": 1857925112 } }, "state": "open", "num_leases": 1, "num_caps": 7, "request_load_avg": 13, "uptime": 2.0738839160000002, "requests_in_flight": 1, "completed_requests": 0, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 2.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 19.904873717015004, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 10 }, "inst": "client.9594 172.21.2.101:0/1857925112", "completed_requests": [], "prealloc_inos": [ { "start": 1099511627782, "length": 995 } ], "used_inos": [ { "start": 1099511627781, "length": 1 } ], "client_metadata": { "features": "0x0000000000001fff", "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", "entity_id": "y", "hostname": "gibba001", "pid": "24165", "root": "/" } } ] 2021-02-10T11:37:28.590 DEBUG:tasks.ceph_test_case:wait_until_true: waiting... 2021-02-10T11:37:31.623 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:37:31.627 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:37:33.591 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:37:34.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:33.998 7fd58e1e1700 1 Processor -- start 2021-02-10T11:37:34.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:33.998 7fd58e1e1700 1 -- start start 2021-02-10T11:37:34.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:33.999 7fd58e1e1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5880d1870 0x7fd5881b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:34.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:33.999 7fd58e1e1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd5880f0650 0x7fd5880cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:34.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:33.999 7fd58e1e1700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5880f0b10 con 0x7fd5881b0a10 2021-02-10T11:37:34.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:33.999 7fd58e1e1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5880fe230 con 0x7fd5880f0650 2021-02-10T11:37:34.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:33.999 7fd58e1e1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5880e5ed0 con 0x7fd5880d1870 2021-02-10T11:37:34.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.000 7fd5877fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5880d1870 0x7fd5881b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:34.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.000 7fd5877fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5880d1870 0x7fd5881b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:47506/0 (socket says 172.21.2.101:47506) 2021-02-10T11:37:34.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.000 7fd5877fe700 1 -- 172.21.2.101:0/1486949105 learned_addr learned my addr 172.21.2.101:0/1486949105 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:34.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.000 7fd587fff700 1 --2- 172.21.2.101:0/1486949105 >> v2:172.21.2.108:3300/0 conn(0x7fd5880f0650 0x7fd5880cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:34.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.000 7fd587fff700 1 -- 172.21.2.101:0/1486949105 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5880d1870 msgr2=0x7fd5881b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:34.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.000 7fd587fff700 1 --2- 172.21.2.101:0/1486949105 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5880d1870 0x7fd5881b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:34.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.000 7fd587fff700 1 -- 172.21.2.101:0/1486949105 >> v1:172.21.2.108:6789/0 conn(0x7fd5881b0a10 legacy=0x7fd5880cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:34.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.000 7fd5867fc700 1 -- 172.21.2.101:0/1486949105 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3337129124 0 0) 0x7fd578000ff0 con 0x7fd5881b0a10 2021-02-10T11:37:34.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.000 7fd587fff700 1 -- 172.21.2.101:0/1486949105 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5880f20a0 con 0x7fd5880f0650 2021-02-10T11:37:34.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.000 7fd5877fe700 1 --2- 172.21.2.101:0/1486949105 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5880d1870 0x7fd5881b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:37:34.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.001 7fd587fff700 1 --2- 172.21.2.101:0/1486949105 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd5880f0650 0x7fd5880cfb20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fd580006e50 tx=0x7fd580009e70).ready entity=mon.1 client_cookie=78646c01b04e0251 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:34.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.001 7fd5867fc700 1 -- 172.21.2.101:0/1486949105 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd58000b510 con 0x7fd5880f0650 2021-02-10T11:37:34.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.001 7fd5867fc700 1 -- 172.21.2.101:0/1486949105 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd580006790 con 0x7fd5880f0650 2021-02-10T11:37:34.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.001 7fd5867fc700 1 -- 172.21.2.101:0/1486949105 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd58000b940 con 0x7fd5880f0650 2021-02-10T11:37:34.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.001 7fd58e1e1700 1 -- 172.21.2.101:0/1486949105 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd5880f0650 msgr2=0x7fd5880cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:34.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.001 7fd58e1e1700 1 --2- 172.21.2.101:0/1486949105 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd5880f0650 0x7fd5880cfb20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fd580006e50 tx=0x7fd580009e70).stop 2021-02-10T11:37:34.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.002 7fd58e1e1700 1 -- 172.21.2.101:0/1486949105 shutdown_connections 2021-02-10T11:37:34.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.002 7fd58e1e1700 1 --2- 172.21.2.101:0/1486949105 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd5880f0650 0x7fd5880cfb20 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:34.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.002 7fd58e1e1700 1 --2- 172.21.2.101:0/1486949105 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5880d1870 0x7fd5881b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:34.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.002 7fd58e1e1700 1 -- 172.21.2.101:0/1486949105 >> 172.21.2.101:0/1486949105 conn(0x7fd5880ce450 msgr2=0x7fd5880d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:34.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.002 7fd58e1e1700 1 -- 172.21.2.101:0/1486949105 shutdown_connections 2021-02-10T11:37:34.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.002 7fd58e1e1700 1 -- 172.21.2.101:0/1486949105 wait complete. 2021-02-10T11:37:34.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.016 7fd58e1e1700 1 Processor -- start 2021-02-10T11:37:34.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.016 7fd58e1e1700 1 -- start start 2021-02-10T11:37:34.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.017 7fd58e1e1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd588195650 0x7fd5880b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:34.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.017 7fd58e1e1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd5880b5b80 0x7fd5881a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:34.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.017 7fd58e1e1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd5880acfa0 0x7fd5881a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:34.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.017 7fd58e1e1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5880b6010 con 0x7fd5880b5b80 2021-02-10T11:37:34.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.017 7fd58e1e1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5881a3a60 con 0x7fd5880acfa0 2021-02-10T11:37:34.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.017 7fd58e1e1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5881a3c40 con 0x7fd588195650 2021-02-10T11:37:34.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.017 7fd586ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd5880b5b80 0x7fd5881a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:34.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.017 7fd586ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd5880b5b80 0x7fd5881a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54778/0 (socket says 172.21.2.101:54778) 2021-02-10T11:37:34.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.017 7fd586ffd700 1 -- 172.21.2.101:0/80550742 learned_addr learned my addr 172.21.2.101:0/80550742 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:34.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.017 7fd586ffd700 1 -- 172.21.2.101:0/80550742 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd588195650 msgr2=0x7fd5880b5560 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:37:34.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.017 7fd586ffd700 1 --2- 172.21.2.101:0/80550742 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd588195650 0x7fd5880b5560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:34.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.017 7fd586ffd700 1 -- 172.21.2.101:0/80550742 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd5880acfa0 msgr2=0x7fd5881a3440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:37:34.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.017 7fd586ffd700 1 --2- 172.21.2.101:0/80550742 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd5880acfa0 0x7fd5881a3440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:34.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.017 7fd586ffd700 1 -- 172.21.2.101:0/80550742 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5880bcae0 con 0x7fd5880b5b80 2021-02-10T11:37:34.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.017 7fd586ffd700 1 --2- 172.21.2.101:0/80550742 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd5880b5b80 0x7fd5881a2e20 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 rx=0x7fd57800b580 tx=0x7fd57800b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:34.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.017 7fd57ffff700 1 -- 172.21.2.101:0/80550742 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd5780080d0 con 0x7fd5880b5b80 2021-02-10T11:37:34.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.017 7fd57ffff700 1 -- 172.21.2.101:0/80550742 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd578008350 con 0x7fd5880b5b80 2021-02-10T11:37:34.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.017 7fd58e1e1700 1 -- 172.21.2.101:0/80550742 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5880bcae0 con 0x7fd5880b5b80 2021-02-10T11:37:34.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.017 7fd57ffff700 1 -- 172.21.2.101:0/80550742 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd578008870 con 0x7fd5880b5b80 2021-02-10T11:37:34.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.017 7fd58e1e1700 1 -- 172.21.2.101:0/80550742 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5880bd070 con 0x7fd5880b5b80 2021-02-10T11:37:34.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.018 7fd57ffff700 1 -- 172.21.2.101:0/80550742 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd578008800 con 0x7fd5880b5b80 2021-02-10T11:37:34.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.018 7fd57ffff700 1 --2- 172.21.2.101:0/80550742 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd56c018730 0x7fd56c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:34.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.018 7fd57ffff700 1 -- 172.21.2.101:0/80550742 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd578032680 con 0x7fd5880b5b80 2021-02-10T11:37:34.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.018 7fd5877fe700 1 --2- 172.21.2.101:0/80550742 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd56c018730 0x7fd56c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:34.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.018 7fd5877fe700 1 --2- 172.21.2.101:0/80550742 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd56c018730 0x7fd56c01ac10 crc :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:34.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.049 7fd58e1e1700 1 -- 172.21.2.101:0/80550742 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5880f0ab0 con 0x7fd5880b5b80 2021-02-10T11:37:34.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.052 7fd57ffff700 1 -- 172.21.2.101:0/80550742 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd578008330 con 0x7fd5880b5b80 2021-02-10T11:37:34.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.179 7fd58e1e1700 1 -- 172.21.2.101:0/80550742 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5880f0ab0 con 0x7fd5880b5b80 2021-02-10T11:37:34.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.179 7fd57ffff700 1 -- 172.21.2.101:0/80550742 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+2101 (secure 0 0 0) 0x7fd578008330 con 0x7fd5880b5b80 2021-02-10T11:37:34.185 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:37:34.185 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9360,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2529021396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2529021396},{"type":"v1","addr":"172.21.2.101:6835","nonce":2529021396}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:56.648753","modified":"2021-02-10 11:36:58.653009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6005":{"gid":6005,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2363068042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2363068042},{"type":"v1","addr":"172.21.2.108:6833","nonce":2363068042}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-02-10T11:37:34.185 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 63 2021-02-10T11:37:34.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.181 7fd58e1e1700 1 -- 172.21.2.101:0/80550742 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd56c018730 msgr2=0x7fd56c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:34.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.181 7fd58e1e1700 1 --2- 172.21.2.101:0/80550742 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd56c018730 0x7fd56c01ac10 crc :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:34.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.181 7fd58e1e1700 1 -- 172.21.2.101:0/80550742 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd5880b5b80 msgr2=0x7fd5881a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:34.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.181 7fd58e1e1700 1 --2- 172.21.2.101:0/80550742 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd5880b5b80 0x7fd5881a2e20 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 rx=0x7fd57800b580 tx=0x7fd57800b920).stop 2021-02-10T11:37:34.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.181 7fd58e1e1700 1 -- 172.21.2.101:0/80550742 shutdown_connections 2021-02-10T11:37:34.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.181 7fd58e1e1700 1 --2- 172.21.2.101:0/80550742 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd5880acfa0 0x7fd5881a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:34.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.181 7fd58e1e1700 1 --2- 172.21.2.101:0/80550742 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd5880b5b80 0x7fd5881a2e20 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:34.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.181 7fd58e1e1700 1 --2- 172.21.2.101:0/80550742 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd56c018730 0x7fd56c01ac10 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:34.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.181 7fd58e1e1700 1 --2- 172.21.2.101:0/80550742 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd588195650 0x7fd5880b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:34.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.181 7fd58e1e1700 1 -- 172.21.2.101:0/80550742 >> 172.21.2.101:0/80550742 conn(0x7fd5880b3040 msgr2=0x7fd5880c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:34.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.181 7fd58e1e1700 1 -- 172.21.2.101:0/80550742 shutdown_connections 2021-02-10T11:37:34.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:34.181 7fd58e1e1700 1 -- 172.21.2.101:0/80550742 wait complete. 2021-02-10T11:37:34.195 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok session ls 2021-02-10T11:37:34.592 INFO:teuthology.orchestra.run.gibba008.stdout:[ 2021-02-10T11:37:34.592 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:37:34.593 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 9594, 2021-02-10T11:37:34.593 INFO:teuthology.orchestra.run.gibba008.stdout: "entity": { 2021-02-10T11:37:34.593 INFO:teuthology.orchestra.run.gibba008.stdout: "name": { 2021-02-10T11:37:34.593 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "client", 2021-02-10T11:37:34.593 INFO:teuthology.orchestra.run.gibba008.stdout: "num": 9594 2021-02-10T11:37:34.593 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:34.593 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": { 2021-02-10T11:37:34.593 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "any", 2021-02-10T11:37:34.593 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:37:34.593 INFO:teuthology.orchestra.run.gibba008.stdout: "nonce": 1857925112 2021-02-10T11:37:34.593 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:34.593 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:34.593 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "open", 2021-02-10T11:37:34.594 INFO:teuthology.orchestra.run.gibba008.stdout: "num_leases": 1, 2021-02-10T11:37:34.594 INFO:teuthology.orchestra.run.gibba008.stdout: "num_caps": 7, 2021-02-10T11:37:34.594 INFO:teuthology.orchestra.run.gibba008.stdout: "request_load_avg": 12, 2021-02-10T11:37:34.594 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 8.0835510839999998, 2021-02-10T11:37:34.594 INFO:teuthology.orchestra.run.gibba008.stdout: "requests_in_flight": 0, 2021-02-10T11:37:34.594 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": 1, 2021-02-10T11:37:34.594 INFO:teuthology.orchestra.run.gibba008.stdout: "reconnecting": false, 2021-02-10T11:37:34.594 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps": { 2021-02-10T11:37:34.594 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:34.594 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:37:34.594 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:34.594 INFO:teuthology.orchestra.run.gibba008.stdout: "release_caps": { 2021-02-10T11:37:34.594 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:34.595 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:37:34.595 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:34.595 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle": { 2021-02-10T11:37:34.595 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:34.595 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 2.5 2021-02-10T11:37:34.595 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:34.595 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle2o": { 2021-02-10T11:37:34.595 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:34.595 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 0.5 2021-02-10T11:37:34.595 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:34.595 INFO:teuthology.orchestra.run.gibba008.stdout: "session_cache_liveness": { 2021-02-10T11:37:34.595 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 20.625600344961715, 2021-02-10T11:37:34.595 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 300 2021-02-10T11:37:34.595 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:34.596 INFO:teuthology.orchestra.run.gibba008.stdout: "cap_acquisition": { 2021-02-10T11:37:34.596 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:34.596 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 10 2021-02-10T11:37:34.596 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:34.596 INFO:teuthology.orchestra.run.gibba008.stdout: "inst": "client.9594 172.21.2.101:0/1857925112", 2021-02-10T11:37:34.596 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": [ 2021-02-10T11:37:34.596 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:37:34.596 INFO:teuthology.orchestra.run.gibba008.stdout: "tid": 14, 2021-02-10T11:37:34.596 INFO:teuthology.orchestra.run.gibba008.stdout: "created_ino": "0x10000000005" 2021-02-10T11:37:34.596 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:34.596 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:37:34.596 INFO:teuthology.orchestra.run.gibba008.stdout: "prealloc_inos": [ 2021-02-10T11:37:34.596 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:37:34.597 INFO:teuthology.orchestra.run.gibba008.stdout: "start": 1099511627782, 2021-02-10T11:37:34.597 INFO:teuthology.orchestra.run.gibba008.stdout: "length": 995 2021-02-10T11:37:34.597 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:34.597 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:37:34.597 INFO:teuthology.orchestra.run.gibba008.stdout: "used_inos": [], 2021-02-10T11:37:34.597 INFO:teuthology.orchestra.run.gibba008.stdout: "client_metadata": { 2021-02-10T11:37:34.597 INFO:teuthology.orchestra.run.gibba008.stdout: "features": "0x0000000000001fff", 2021-02-10T11:37:34.597 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:37:34.597 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:37:34.597 INFO:teuthology.orchestra.run.gibba008.stdout: "entity_id": "y", 2021-02-10T11:37:34.597 INFO:teuthology.orchestra.run.gibba008.stdout: "hostname": "gibba001", 2021-02-10T11:37:34.597 INFO:teuthology.orchestra.run.gibba008.stdout: "pid": "24165", 2021-02-10T11:37:34.597 INFO:teuthology.orchestra.run.gibba008.stdout: "root": "/" 2021-02-10T11:37:34.597 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:34.598 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:34.598 INFO:teuthology.orchestra.run.gibba008.stdout:] 2021-02-10T11:37:34.598 INFO:tasks.cephfs.filesystem:_json_asok output: [ { "id": 9594, "entity": { "name": { "type": "client", "num": 9594 }, "addr": { "type": "any", "addr": "172.21.2.101:0", "nonce": 1857925112 } }, "state": "open", "num_leases": 1, "num_caps": 7, "request_load_avg": 12, "uptime": 8.0835510839999998, "requests_in_flight": 0, "completed_requests": 1, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 2.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 20.625600344961715, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 10 }, "inst": "client.9594 172.21.2.101:0/1857925112", "completed_requests": [ { "tid": 14, "created_ino": "0x10000000005" } ], "prealloc_inos": [ { "start": 1099511627782, "length": 995 } ], "used_inos": [], "client_metadata": { "features": "0x0000000000001fff", "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", "entity_id": "y", "hostname": "gibba001", "pid": "24165", "root": "/" } } ] 2021-02-10T11:37:34.599 DEBUG:tasks.ceph_test_case:wait_until_true: waiting... 2021-02-10T11:37:39.600 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:37:40.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.007 7f8ab4431700 1 Processor -- start 2021-02-10T11:37:40.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.007 7f8ab4431700 1 -- start start 2021-02-10T11:37:40.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.008 7f8ab4431700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8aac0d1870 0x7f8aac1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:40.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.008 7f8ab4431700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8aac1b0a10 0x7f8aac0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:40.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.008 7f8ab4431700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aac0f0b10 con 0x7f8aac0d1c70 2021-02-10T11:37:40.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.008 7f8ab4431700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aac0fe230 con 0x7f8aac1b0a10 2021-02-10T11:37:40.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.008 7f8ab4431700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac0e5ed0 con 0x7f8aac0d1870 2021-02-10T11:37:40.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.008 7f8ab1a4c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8aac1b0a10 0x7f8aac0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:40.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.008 7f8ab1a4c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8aac1b0a10 0x7f8aac0cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54786/0 (socket says 172.21.2.101:54786) 2021-02-10T11:37:40.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.008 7f8ab1a4c700 1 -- 172.21.2.101:0/2489306377 learned_addr learned my addr 172.21.2.101:0/2489306377 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:40.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.008 7f8ab1a4c700 1 -- 172.21.2.101:0/2489306377 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8aac0d1870 msgr2=0x7f8aac1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:40.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.008 7f8ab1a4c700 1 --2- 172.21.2.101:0/2489306377 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8aac0d1870 0x7f8aac1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:40.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.008 7f8ab1a4c700 1 -- 172.21.2.101:0/2489306377 >> v1:172.21.2.101:6789/0 conn(0x7f8aac0d1c70 legacy=0x7f8aac0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:40.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.008 7f8ab1a4c700 1 -- 172.21.2.101:0/2489306377 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac0f0650 con 0x7f8aac1b0a10 2021-02-10T11:37:40.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.009 7f8ab1a4c700 1 --2- 172.21.2.101:0/2489306377 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8aac1b0a10 0x7f8aac0cf330 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 rx=0x7f8aa0006700 tx=0x7f8aa0009b80).ready entity=mon.0 client_cookie=3e85d37b0de817a2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:40.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.009 7f8ab124b700 1 -- 172.21.2.101:0/2489306377 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8aa000c0c0 con 0x7f8aac1b0a10 2021-02-10T11:37:40.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.009 7f8ab124b700 1 -- 172.21.2.101:0/2489306377 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa000c340 con 0x7f8aac1b0a10 2021-02-10T11:37:40.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.009 7f8ab124b700 1 -- 172.21.2.101:0/2489306377 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8aa000c860 con 0x7f8aac1b0a10 2021-02-10T11:37:40.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.009 7f8ab4431700 1 -- 172.21.2.101:0/2489306377 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8aac1b0a10 msgr2=0x7f8aac0cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:40.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.009 7f8ab4431700 1 --2- 172.21.2.101:0/2489306377 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8aac1b0a10 0x7f8aac0cf330 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 rx=0x7f8aa0006700 tx=0x7f8aa0009b80).stop 2021-02-10T11:37:40.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.010 7f8ab4431700 1 -- 172.21.2.101:0/2489306377 shutdown_connections 2021-02-10T11:37:40.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.010 7f8ab4431700 1 --2- 172.21.2.101:0/2489306377 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8aac1b0a10 0x7f8aac0cf330 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:40.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.010 7f8ab4431700 1 --2- 172.21.2.101:0/2489306377 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8aac0d1870 0x7f8aac1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:40.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.010 7f8ab4431700 1 -- 172.21.2.101:0/2489306377 >> 172.21.2.101:0/2489306377 conn(0x7f8aac0ce450 msgr2=0x7f8aac0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:40.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.010 7f8ab4431700 1 -- 172.21.2.101:0/2489306377 shutdown_connections 2021-02-10T11:37:40.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.010 7f8ab4431700 1 -- 172.21.2.101:0/2489306377 wait complete. 2021-02-10T11:37:40.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.024 7f8ab4431700 1 Processor -- start 2021-02-10T11:37:40.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.024 7f8ab4431700 1 -- start start 2021-02-10T11:37:40.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.025 7f8ab4431700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8aac0b0ad0 0x7f8aac091f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:40.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.025 7f8ab4431700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8aac1a8850 0x7f8aac1a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:40.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.025 7f8ab4431700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8aac1a6fa0 0x7f8aac1a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:40.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.025 7f8ab4431700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac1abfe0 con 0x7f8aac1a6fa0 2021-02-10T11:37:40.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.025 7f8ab4431700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac1a9c40 con 0x7f8aac1a8850 2021-02-10T11:37:40.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.025 7f8ab4431700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac1a9730 con 0x7f8aac0b0ad0 2021-02-10T11:37:40.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.025 7f8ab2a4e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8aac1a6fa0 0x7f8aac1a73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:40.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.025 7f8ab2a4e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8aac1a6fa0 0x7f8aac1a73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54790/0 (socket says 172.21.2.101:54790) 2021-02-10T11:37:40.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.025 7f8ab2a4e700 1 -- 172.21.2.101:0/1755385682 learned_addr learned my addr 172.21.2.101:0/1755385682 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.025 7f8ab224d700 1 --2- 172.21.2.101:0/1755385682 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8aac0b0ad0 0x7f8aac091f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.025 7f8ab1a4c700 1 --2- 172.21.2.101:0/1755385682 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8aac1a8850 0x7f8aac1a8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.025 7f8ab2a4e700 1 -- 172.21.2.101:0/1755385682 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8aac0b0ad0 msgr2=0x7f8aac091f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.025 7f8ab2a4e700 1 --2- 172.21.2.101:0/1755385682 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8aac0b0ad0 0x7f8aac091f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.025 7f8ab2a4e700 1 -- 172.21.2.101:0/1755385682 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8aac1a8850 msgr2=0x7f8aac1a8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.025 7f8ab2a4e700 1 --2- 172.21.2.101:0/1755385682 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8aac1a8850 0x7f8aac1a8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.025 7f8ab2a4e700 1 -- 172.21.2.101:0/1755385682 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac1a9270 con 0x7f8aac1a6fa0 2021-02-10T11:37:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.025 7f8ab2a4e700 1 --2- 172.21.2.101:0/1755385682 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8aac1a6fa0 0x7f8aac1a73a0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 rx=0x7f8aa400b8c0 tx=0x7f8aa400bc60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.026 7f8a9effd700 1 -- 172.21.2.101:0/1755385682 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8aa4005a60 con 0x7f8aac1a6fa0 2021-02-10T11:37:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.026 7f8a9effd700 1 -- 172.21.2.101:0/1755385682 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa4005d20 con 0x7f8aac1a6fa0 2021-02-10T11:37:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.026 7f8ab4431700 1 -- 172.21.2.101:0/1755385682 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aac1a9270 con 0x7f8aac1a6fa0 2021-02-10T11:37:40.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.026 7f8a9effd700 1 -- 172.21.2.101:0/1755385682 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8aa40062b0 con 0x7f8aac1a6fa0 2021-02-10T11:37:40.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.026 7f8ab4431700 1 -- 172.21.2.101:0/1755385682 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aac1a9270 con 0x7f8aac1a6fa0 2021-02-10T11:37:40.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.026 7f8a9effd700 1 -- 172.21.2.101:0/1755385682 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8aa4005870 con 0x7f8aac1a6fa0 2021-02-10T11:37:40.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.026 7f8a9effd700 1 --2- 172.21.2.101:0/1755385682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a94018700 0x7f8a9401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:40.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.026 7f8a9effd700 1 -- 172.21.2.101:0/1755385682 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8aa40329e0 con 0x7f8aac1a6fa0 2021-02-10T11:37:40.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.026 7f8ab224d700 1 --2- 172.21.2.101:0/1755385682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a94018700 0x7f8a9401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:40.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.027 7f8ab224d700 1 --2- 172.21.2.101:0/1755385682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a94018700 0x7f8a9401abc0 crc :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:40.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.058 7f8ab4431700 1 -- 172.21.2.101:0/1755385682 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aac1abfe0 con 0x7f8aac1a6fa0 2021-02-10T11:37:40.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.061 7f8a9effd700 1 -- 172.21.2.101:0/1755385682 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8aa400bda0 con 0x7f8aac1a6fa0 2021-02-10T11:37:40.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.188 7f8ab4431700 1 -- 172.21.2.101:0/1755385682 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8aac1abfe0 con 0x7f8aac1a6fa0 2021-02-10T11:37:40.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.188 7f8a9effd700 1 -- 172.21.2.101:0/1755385682 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f8aa400bda0 con 0x7f8aac1a6fa0 2021-02-10T11:37:40.194 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:37:40.194 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9360,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2529021396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2529021396},{"type":"v1","addr":"172.21.2.101:6835","nonce":2529021396}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:56.648753","modified":"2021-02-10 11:36:58.653009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6005":{"gid":6005,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2363068042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2363068042},{"type":"v1","addr":"172.21.2.108:6833","nonce":2363068042}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-02-10T11:37:40.194 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 63 2021-02-10T11:37:40.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.190 7f8ab4431700 1 -- 172.21.2.101:0/1755385682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a94018700 msgr2=0x7f8a9401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:40.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.190 7f8ab4431700 1 --2- 172.21.2.101:0/1755385682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a94018700 0x7f8a9401abc0 crc :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:40.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.190 7f8ab4431700 1 -- 172.21.2.101:0/1755385682 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8aac1a6fa0 msgr2=0x7f8aac1a73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:40.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.190 7f8ab4431700 1 --2- 172.21.2.101:0/1755385682 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8aac1a6fa0 0x7f8aac1a73a0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 rx=0x7f8aa400b8c0 tx=0x7f8aa400bc60).stop 2021-02-10T11:37:40.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.191 7f8ab4431700 1 -- 172.21.2.101:0/1755385682 shutdown_connections 2021-02-10T11:37:40.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.191 7f8ab4431700 1 --2- 172.21.2.101:0/1755385682 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8aac1a6fa0 0x7f8aac1a73a0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:40.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.191 7f8ab4431700 1 --2- 172.21.2.101:0/1755385682 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8aac1a8850 0x7f8aac1a8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:40.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.191 7f8ab4431700 1 --2- 172.21.2.101:0/1755385682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a94018700 0x7f8a9401abc0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:40.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.191 7f8ab4431700 1 --2- 172.21.2.101:0/1755385682 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8aac0b0ad0 0x7f8aac091f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:40.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.191 7f8ab4431700 1 -- 172.21.2.101:0/1755385682 >> 172.21.2.101:0/1755385682 conn(0x7f8aac19f8f0 msgr2=0x7f8aac1a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:40.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.191 7f8ab4431700 1 -- 172.21.2.101:0/1755385682 shutdown_connections 2021-02-10T11:37:40.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:40.191 7f8ab4431700 1 -- 172.21.2.101:0/1755385682 wait complete. 2021-02-10T11:37:40.204 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok session ls 2021-02-10T11:37:40.601 INFO:teuthology.orchestra.run.gibba008.stdout:[ 2021-02-10T11:37:40.601 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:37:40.601 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 9594, 2021-02-10T11:37:40.602 INFO:teuthology.orchestra.run.gibba008.stdout: "entity": { 2021-02-10T11:37:40.602 INFO:teuthology.orchestra.run.gibba008.stdout: "name": { 2021-02-10T11:37:40.602 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "client", 2021-02-10T11:37:40.602 INFO:teuthology.orchestra.run.gibba008.stdout: "num": 9594 2021-02-10T11:37:40.602 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:40.602 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": { 2021-02-10T11:37:40.602 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "any", 2021-02-10T11:37:40.602 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:37:40.602 INFO:teuthology.orchestra.run.gibba008.stdout: "nonce": 1857925112 2021-02-10T11:37:40.602 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:40.602 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:40.602 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "open", 2021-02-10T11:37:40.603 INFO:teuthology.orchestra.run.gibba008.stdout: "num_leases": 1, 2021-02-10T11:37:40.603 INFO:teuthology.orchestra.run.gibba008.stdout: "num_caps": 7, 2021-02-10T11:37:40.603 INFO:teuthology.orchestra.run.gibba008.stdout: "request_load_avg": 11, 2021-02-10T11:37:40.603 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 14.092223512, 2021-02-10T11:37:40.603 INFO:teuthology.orchestra.run.gibba008.stdout: "requests_in_flight": 0, 2021-02-10T11:37:40.603 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": 1, 2021-02-10T11:37:40.603 INFO:teuthology.orchestra.run.gibba008.stdout: "reconnecting": false, 2021-02-10T11:37:40.603 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps": { 2021-02-10T11:37:40.603 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:40.603 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:37:40.603 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:40.603 INFO:teuthology.orchestra.run.gibba008.stdout: "release_caps": { 2021-02-10T11:37:40.603 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:40.604 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:37:40.604 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:40.604 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle": { 2021-02-10T11:37:40.604 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:40.604 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 2.5 2021-02-10T11:37:40.604 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:40.604 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle2o": { 2021-02-10T11:37:40.604 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:40.604 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 0.5 2021-02-10T11:37:40.604 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:40.604 INFO:teuthology.orchestra.run.gibba008.stdout: "session_cache_liveness": { 2021-02-10T11:37:40.604 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 20.341234017161149, 2021-02-10T11:37:40.604 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 300 2021-02-10T11:37:40.605 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:40.605 INFO:teuthology.orchestra.run.gibba008.stdout: "cap_acquisition": { 2021-02-10T11:37:40.605 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:40.605 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 10 2021-02-10T11:37:40.605 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:40.605 INFO:teuthology.orchestra.run.gibba008.stdout: "inst": "client.9594 172.21.2.101:0/1857925112", 2021-02-10T11:37:40.605 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": [ 2021-02-10T11:37:40.605 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:37:40.605 INFO:teuthology.orchestra.run.gibba008.stdout: "tid": 14, 2021-02-10T11:37:40.605 INFO:teuthology.orchestra.run.gibba008.stdout: "created_ino": "0x10000000005" 2021-02-10T11:37:40.605 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:40.605 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:37:40.605 INFO:teuthology.orchestra.run.gibba008.stdout: "prealloc_inos": [ 2021-02-10T11:37:40.606 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:37:40.606 INFO:teuthology.orchestra.run.gibba008.stdout: "start": 1099511627782, 2021-02-10T11:37:40.606 INFO:teuthology.orchestra.run.gibba008.stdout: "length": 995 2021-02-10T11:37:40.606 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:40.606 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:37:40.606 INFO:teuthology.orchestra.run.gibba008.stdout: "used_inos": [], 2021-02-10T11:37:40.606 INFO:teuthology.orchestra.run.gibba008.stdout: "client_metadata": { 2021-02-10T11:37:40.606 INFO:teuthology.orchestra.run.gibba008.stdout: "features": "0x0000000000001fff", 2021-02-10T11:37:40.606 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:37:40.606 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:37:40.606 INFO:teuthology.orchestra.run.gibba008.stdout: "entity_id": "y", 2021-02-10T11:37:40.606 INFO:teuthology.orchestra.run.gibba008.stdout: "hostname": "gibba001", 2021-02-10T11:37:40.606 INFO:teuthology.orchestra.run.gibba008.stdout: "pid": "24165", 2021-02-10T11:37:40.607 INFO:teuthology.orchestra.run.gibba008.stdout: "root": "/" 2021-02-10T11:37:40.607 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:40.607 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:40.607 INFO:teuthology.orchestra.run.gibba008.stdout:] 2021-02-10T11:37:40.608 INFO:tasks.cephfs.filesystem:_json_asok output: [ { "id": 9594, "entity": { "name": { "type": "client", "num": 9594 }, "addr": { "type": "any", "addr": "172.21.2.101:0", "nonce": 1857925112 } }, "state": "open", "num_leases": 1, "num_caps": 7, "request_load_avg": 11, "uptime": 14.092223512, "requests_in_flight": 0, "completed_requests": 1, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 2.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 20.341234017161149, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 10 }, "inst": "client.9594 172.21.2.101:0/1857925112", "completed_requests": [ { "tid": 14, "created_ino": "0x10000000005" } ], "prealloc_inos": [ { "start": 1099511627782, "length": 995 } ], "used_inos": [], "client_metadata": { "features": "0x0000000000001fff", "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", "entity_id": "y", "hostname": "gibba001", "pid": "24165", "root": "/" } } ] 2021-02-10T11:37:40.608 DEBUG:tasks.ceph_test_case:wait_until_true: waiting... 2021-02-10T11:37:45.609 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:37:46.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.016 7f8fcdbeb700 1 Processor -- start 2021-02-10T11:37:46.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.016 7f8fcdbeb700 1 -- start start 2021-02-10T11:37:46.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.016 7f8fcdbeb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8fc80d0340 0x7f8fc80ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:46.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.016 7f8fcdbeb700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8fc80d1c70 0x7f8fc80efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:46.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.016 7f8fcdbeb700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc80f0b10 con 0x7f8fc80d1870 2021-02-10T11:37:46.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.016 7f8fcdbeb700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc80e5ed0 con 0x7f8fc80d1c70 2021-02-10T11:37:46.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.016 7f8fcdbeb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc81b0be0 con 0x7f8fc80d0340 2021-02-10T11:37:46.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.016 7f8fc6ffd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8fc80d0340 0x7f8fc80ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:46.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.017 7f8fc6ffd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8fc80d0340 0x7f8fc80ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:47530/0 (socket says 172.21.2.101:47530) 2021-02-10T11:37:46.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.017 7f8fc6ffd700 1 -- 172.21.2.101:0/1648079085 learned_addr learned my addr 172.21.2.101:0/1648079085 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:46.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.017 7f8fc77fe700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f8fc80d1870 0x7f8fc81b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:48702/0 (socket says 172.21.2.101:48702) 2021-02-10T11:37:46.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.017 7f8fc7fff700 1 --2- 172.21.2.101:0/1648079085 >> v2:172.21.2.108:3300/0 conn(0x7f8fc80d1c70 0x7f8fc80efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:46.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.017 7f8fc6ffd700 1 -- 172.21.2.101:0/1648079085 >> v2:172.21.2.108:3300/0 conn(0x7f8fc80d1c70 msgr2=0x7f8fc80efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:46.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.017 7f8fc6ffd700 1 --2- 172.21.2.101:0/1648079085 >> v2:172.21.2.108:3300/0 conn(0x7f8fc80d1c70 0x7f8fc80efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:46.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.017 7f8fc6ffd700 1 -- 172.21.2.101:0/1648079085 >> v1:172.21.2.108:6789/0 conn(0x7f8fc80d1870 legacy=0x7f8fc81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:46.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.017 7f8fc6ffd700 1 -- 172.21.2.101:0/1648079085 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc80f20a0 con 0x7f8fc80d0340 2021-02-10T11:37:46.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.017 7f8fc6ffd700 1 --2- 172.21.2.101:0/1648079085 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8fc80d0340 0x7f8fc80ef640 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f8fb0006700 tx=0x7f8fb0009b80).ready entity=mon.2 client_cookie=177e8b6c0beceaf8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:46.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.017 7f8fc67fc700 1 -- 172.21.2.101:0/1648079085 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8fb000c0c0 con 0x7f8fc80d0340 2021-02-10T11:37:46.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.017 7f8fc67fc700 1 -- 172.21.2.101:0/1648079085 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb000c440 con 0x7f8fc80d0340 2021-02-10T11:37:46.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.017 7f8fc67fc700 1 -- 172.21.2.101:0/1648079085 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8fb000c8a0 con 0x7f8fc80d0340 2021-02-10T11:37:46.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.018 7f8fcdbeb700 1 -- 172.21.2.101:0/1648079085 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8fc80d0340 msgr2=0x7f8fc80ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:46.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.018 7f8fcdbeb700 1 --2- 172.21.2.101:0/1648079085 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8fc80d0340 0x7f8fc80ef640 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f8fb0006700 tx=0x7f8fb0009b80).stop 2021-02-10T11:37:46.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.019 7f8fcdbeb700 1 -- 172.21.2.101:0/1648079085 shutdown_connections 2021-02-10T11:37:46.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.019 7f8fcdbeb700 1 --2- 172.21.2.101:0/1648079085 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8fc80d0340 0x7f8fc80ef640 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:46.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.019 7f8fcdbeb700 1 --2- 172.21.2.101:0/1648079085 >> v2:172.21.2.108:3300/0 conn(0x7f8fc80d1c70 0x7f8fc80efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:46.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.019 7f8fcdbeb700 1 -- 172.21.2.101:0/1648079085 >> 172.21.2.101:0/1648079085 conn(0x7f8fc80ce450 msgr2=0x7f8fc80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:46.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.019 7f8fcdbeb700 1 -- 172.21.2.101:0/1648079085 shutdown_connections 2021-02-10T11:37:46.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.019 7f8fcdbeb700 1 -- 172.21.2.101:0/1648079085 wait complete. 2021-02-10T11:37:46.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.033 7f8fcdbeb700 1 Processor -- start 2021-02-10T11:37:46.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.033 7f8fcdbeb700 1 -- start start 2021-02-10T11:37:46.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.034 7f8fcdbeb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8fc80b1af0 0x7f8fc80c0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:46.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.034 7f8fcdbeb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8fc80c15f0 0x7f8fc81a7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:46.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.034 7f8fcdbeb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8fc81a7ab0 0x7f8fc80c2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:46.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.034 7f8fcdbeb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc80c1e50 con 0x7f8fc80c15f0 2021-02-10T11:37:46.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.034 7f8fcdbeb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc80a49e0 con 0x7f8fc80b1af0 2021-02-10T11:37:46.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.034 7f8fcdbeb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc80c19f0 con 0x7f8fc81a7ab0 2021-02-10T11:37:46.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.034 7f8fc6ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8fc80c15f0 0x7f8fc81a7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:46.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.034 7f8fc77fe700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8fc80b1af0 0x7f8fc80c0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:46.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.034 7f8fc7fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8fc81a7ab0 0x7f8fc80c2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:46.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.034 7f8fc6ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8fc80c15f0 0x7f8fc81a7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54808/0 (socket says 172.21.2.101:54808) 2021-02-10T11:37:46.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.034 7f8fc6ffd700 1 -- 172.21.2.101:0/626605680 learned_addr learned my addr 172.21.2.101:0/626605680 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:46.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.035 7f8fc6ffd700 1 -- 172.21.2.101:0/626605680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8fc81a7ab0 msgr2=0x7f8fc80c2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:46.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.035 7f8fc6ffd700 1 --2- 172.21.2.101:0/626605680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8fc81a7ab0 0x7f8fc80c2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:46.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.035 7f8fc6ffd700 1 -- 172.21.2.101:0/626605680 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8fc80b1af0 msgr2=0x7f8fc80c0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:46.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.035 7f8fc6ffd700 1 --2- 172.21.2.101:0/626605680 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8fc80b1af0 0x7f8fc80c0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:46.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.035 7f8fc6ffd700 1 -- 172.21.2.101:0/626605680 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc81a7eb0 con 0x7f8fc80c15f0 2021-02-10T11:37:46.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.035 7f8fc7fff700 1 --2- 172.21.2.101:0/626605680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8fc81a7ab0 0x7f8fc80c2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:37:46.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.035 7f8fc6ffd700 1 --2- 172.21.2.101:0/626605680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8fc80c15f0 0x7f8fc81a7490 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 rx=0x7f8fb000b5e0 tx=0x7f8fb000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:46.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.035 7f8fbffff700 1 -- 172.21.2.101:0/626605680 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8fb000bd70 con 0x7f8fc80c15f0 2021-02-10T11:37:46.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.035 7f8fbffff700 1 -- 172.21.2.101:0/626605680 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb0002030 con 0x7f8fc80c15f0 2021-02-10T11:37:46.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.035 7f8fcdbeb700 1 -- 172.21.2.101:0/626605680 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc81a7eb0 con 0x7f8fc80c15f0 2021-02-10T11:37:46.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.035 7f8fcdbeb700 1 -- 172.21.2.101:0/626605680 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc80c2970 con 0x7f8fc80c15f0 2021-02-10T11:37:46.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.035 7f8fbffff700 1 -- 172.21.2.101:0/626605680 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8fb0002550 con 0x7f8fc80c15f0 2021-02-10T11:37:46.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.036 7f8fbffff700 1 -- 172.21.2.101:0/626605680 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8fb0000c90 con 0x7f8fc80c15f0 2021-02-10T11:37:46.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.036 7f8fbffff700 1 --2- 172.21.2.101:0/626605680 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8fac018730 0x7f8fac01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:46.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.036 7f8fbffff700 1 -- 172.21.2.101:0/626605680 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8fb00327e0 con 0x7f8fc80c15f0 2021-02-10T11:37:46.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.036 7f8fc77fe700 1 --2- 172.21.2.101:0/626605680 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8fac018730 0x7f8fac01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:46.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.036 7f8fc77fe700 1 --2- 172.21.2.101:0/626605680 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8fac018730 0x7f8fac01abc0 crc :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:46.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.066 7f8fcdbeb700 1 -- 172.21.2.101:0/626605680 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc80ebc00 con 0x7f8fc80c15f0 2021-02-10T11:37:46.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.069 7f8fbffff700 1 -- 172.21.2.101:0/626605680 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8fb0000940 con 0x7f8fc80c15f0 2021-02-10T11:37:46.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.199 7f8fcdbeb700 1 -- 172.21.2.101:0/626605680 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fc80f0b10 con 0x7f8fc80c15f0 2021-02-10T11:37:46.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.200 7f8fbffff700 1 -- 172.21.2.101:0/626605680 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f8fb0000c90 con 0x7f8fc80c15f0 2021-02-10T11:37:46.205 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 63 2021-02-10T11:37:46.205 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:37:46.206 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9360,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2529021396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2529021396},{"type":"v1","addr":"172.21.2.101:6835","nonce":2529021396}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:56.648753","modified":"2021-02-10 11:36:58.653009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6005":{"gid":6005,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2363068042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2363068042},{"type":"v1","addr":"172.21.2.108:6833","nonce":2363068042}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-02-10T11:37:46.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.201 7f8fcdbeb700 1 -- 172.21.2.101:0/626605680 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8fac018730 msgr2=0x7f8fac01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:46.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.201 7f8fcdbeb700 1 --2- 172.21.2.101:0/626605680 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8fac018730 0x7f8fac01abc0 crc :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:46.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.201 7f8fcdbeb700 1 -- 172.21.2.101:0/626605680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8fc80c15f0 msgr2=0x7f8fc81a7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:46.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.201 7f8fcdbeb700 1 --2- 172.21.2.101:0/626605680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8fc80c15f0 0x7f8fc81a7490 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 rx=0x7f8fb000b5e0 tx=0x7f8fb000b980).stop 2021-02-10T11:37:46.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.202 7f8fcdbeb700 1 -- 172.21.2.101:0/626605680 shutdown_connections 2021-02-10T11:37:46.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.202 7f8fcdbeb700 1 --2- 172.21.2.101:0/626605680 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8fac018730 0x7f8fac01abc0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:46.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.202 7f8fcdbeb700 1 --2- 172.21.2.101:0/626605680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8fc81a7ab0 0x7f8fc80c2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:46.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.202 7f8fcdbeb700 1 --2- 172.21.2.101:0/626605680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8fc80c15f0 0x7f8fc81a7490 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:46.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.202 7f8fcdbeb700 1 --2- 172.21.2.101:0/626605680 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8fc80b1af0 0x7f8fc80c0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:46.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.202 7f8fcdbeb700 1 -- 172.21.2.101:0/626605680 >> 172.21.2.101:0/626605680 conn(0x7f8fc809eb60 msgr2=0x7f8fc80ebe30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:46.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.202 7f8fcdbeb700 1 -- 172.21.2.101:0/626605680 shutdown_connections 2021-02-10T11:37:46.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:46.202 7f8fcdbeb700 1 -- 172.21.2.101:0/626605680 wait complete. 2021-02-10T11:37:46.215 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok session ls 2021-02-10T11:37:46.613 INFO:teuthology.orchestra.run.gibba008.stdout:[ 2021-02-10T11:37:46.613 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:37:46.613 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 9594, 2021-02-10T11:37:46.613 INFO:teuthology.orchestra.run.gibba008.stdout: "entity": { 2021-02-10T11:37:46.613 INFO:teuthology.orchestra.run.gibba008.stdout: "name": { 2021-02-10T11:37:46.613 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "client", 2021-02-10T11:37:46.614 INFO:teuthology.orchestra.run.gibba008.stdout: "num": 9594 2021-02-10T11:37:46.614 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:46.614 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": { 2021-02-10T11:37:46.614 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "any", 2021-02-10T11:37:46.614 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:37:46.614 INFO:teuthology.orchestra.run.gibba008.stdout: "nonce": 1857925112 2021-02-10T11:37:46.614 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:46.614 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:46.614 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "open", 2021-02-10T11:37:46.615 INFO:teuthology.orchestra.run.gibba008.stdout: "num_leases": 1, 2021-02-10T11:37:46.615 INFO:teuthology.orchestra.run.gibba008.stdout: "num_caps": 7, 2021-02-10T11:37:46.615 INFO:teuthology.orchestra.run.gibba008.stdout: "request_load_avg": 11, 2021-02-10T11:37:46.615 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 20.103900852999999, 2021-02-10T11:37:46.615 INFO:teuthology.orchestra.run.gibba008.stdout: "requests_in_flight": 0, 2021-02-10T11:37:46.615 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": 1, 2021-02-10T11:37:46.615 INFO:teuthology.orchestra.run.gibba008.stdout: "reconnecting": false, 2021-02-10T11:37:46.615 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps": { 2021-02-10T11:37:46.615 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:46.615 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:37:46.616 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:46.616 INFO:teuthology.orchestra.run.gibba008.stdout: "release_caps": { 2021-02-10T11:37:46.616 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:46.616 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:37:46.616 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:46.616 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle": { 2021-02-10T11:37:46.616 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:46.616 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 2.5 2021-02-10T11:37:46.616 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:46.616 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle2o": { 2021-02-10T11:37:46.616 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:46.616 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 0.5 2021-02-10T11:37:46.616 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:46.617 INFO:teuthology.orchestra.run.gibba008.stdout: "session_cache_liveness": { 2021-02-10T11:37:46.617 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 20.060648986109808, 2021-02-10T11:37:46.617 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 300 2021-02-10T11:37:46.617 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:46.617 INFO:teuthology.orchestra.run.gibba008.stdout: "cap_acquisition": { 2021-02-10T11:37:46.617 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:46.617 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 10 2021-02-10T11:37:46.617 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:46.617 INFO:teuthology.orchestra.run.gibba008.stdout: "inst": "client.9594 172.21.2.101:0/1857925112", 2021-02-10T11:37:46.617 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": [ 2021-02-10T11:37:46.617 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:37:46.617 INFO:teuthology.orchestra.run.gibba008.stdout: "tid": 14, 2021-02-10T11:37:46.617 INFO:teuthology.orchestra.run.gibba008.stdout: "created_ino": "0x10000000005" 2021-02-10T11:37:46.618 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:46.618 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:37:46.618 INFO:teuthology.orchestra.run.gibba008.stdout: "prealloc_inos": [ 2021-02-10T11:37:46.618 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:37:46.618 INFO:teuthology.orchestra.run.gibba008.stdout: "start": 1099511627782, 2021-02-10T11:37:46.618 INFO:teuthology.orchestra.run.gibba008.stdout: "length": 995 2021-02-10T11:37:46.618 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:46.618 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:37:46.618 INFO:teuthology.orchestra.run.gibba008.stdout: "used_inos": [], 2021-02-10T11:37:46.618 INFO:teuthology.orchestra.run.gibba008.stdout: "client_metadata": { 2021-02-10T11:37:46.618 INFO:teuthology.orchestra.run.gibba008.stdout: "features": "0x0000000000001fff", 2021-02-10T11:37:46.618 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:37:46.618 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:37:46.618 INFO:teuthology.orchestra.run.gibba008.stdout: "entity_id": "y", 2021-02-10T11:37:46.619 INFO:teuthology.orchestra.run.gibba008.stdout: "hostname": "gibba001", 2021-02-10T11:37:46.619 INFO:teuthology.orchestra.run.gibba008.stdout: "pid": "24165", 2021-02-10T11:37:46.619 INFO:teuthology.orchestra.run.gibba008.stdout: "root": "/" 2021-02-10T11:37:46.619 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:46.619 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:46.619 INFO:teuthology.orchestra.run.gibba008.stdout:] 2021-02-10T11:37:46.620 INFO:tasks.cephfs.filesystem:_json_asok output: [ { "id": 9594, "entity": { "name": { "type": "client", "num": 9594 }, "addr": { "type": "any", "addr": "172.21.2.101:0", "nonce": 1857925112 } }, "state": "open", "num_leases": 1, "num_caps": 7, "request_load_avg": 11, "uptime": 20.103900852999999, "requests_in_flight": 0, "completed_requests": 1, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 2.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 20.060648986109808, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 10 }, "inst": "client.9594 172.21.2.101:0/1857925112", "completed_requests": [ { "tid": 14, "created_ino": "0x10000000005" } ], "prealloc_inos": [ { "start": 1099511627782, "length": 995 } ], "used_inos": [], "client_metadata": { "features": "0x0000000000001fff", "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", "entity_id": "y", "hostname": "gibba001", "pid": "24165", "root": "/" } } ] 2021-02-10T11:37:46.620 DEBUG:tasks.ceph_test_case:wait_until_true: waiting... 2021-02-10T11:37:51.621 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:37:52.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.025 7f750ada6700 1 Processor -- start 2021-02-10T11:37:52.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.025 7f750ada6700 1 -- start start 2021-02-10T11:37:52.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.026 7f750ada6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75041b0470 0x7f75041b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:52.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.026 7f750ada6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f75040ef610 0x7f75040efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:52.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.026 7f750ada6700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f75040f0030 0x7f75040f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:52.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.026 7f750ada6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75040e5ea0 con 0x7f75040ef610 2021-02-10T11:37:52.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.026 7f750ada6700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75040f0b50 con 0x7f75040f0030 2021-02-10T11:37:52.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.026 7f750ada6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75040e52e0 con 0x7f75041b0470 2021-02-10T11:37:52.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.026 7f7503fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f75040ef610 0x7f75040efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:52.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.026 7f7503fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f75040ef610 0x7f75040efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54816/0 (socket says 172.21.2.101:54816) 2021-02-10T11:37:52.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.026 7f7503fff700 1 -- 172.21.2.101:0/1343196340 learned_addr learned my addr 172.21.2.101:0/1343196340 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:52.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.026 7f7503fff700 1 -- 172.21.2.101:0/1343196340 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75041b0470 msgr2=0x7f75041b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:52.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.026 7f7503fff700 1 --2- 172.21.2.101:0/1343196340 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75041b0470 0x7f75041b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:52.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.026 7f7503fff700 1 -- 172.21.2.101:0/1343196340 >> v2:172.21.2.108:3300/0 conn(0x7f75040f0030 msgr2=0x7f75040f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:52.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.026 7f7503fff700 1 --2- 172.21.2.101:0/1343196340 >> v2:172.21.2.108:3300/0 conn(0x7f75040f0030 0x7f75040f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:52.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.026 7f7503fff700 1 -- 172.21.2.101:0/1343196340 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75040f10d0 con 0x7f75040ef610 2021-02-10T11:37:52.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.027 7f7503fff700 1 --2- 172.21.2.101:0/1343196340 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75040ef610 0x7f75040efa10 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 rx=0x7f74f4006700 tx=0x7f74f4009b80).ready entity=mon.0 client_cookie=b1c92d8a5677cd17 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:52.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.027 7f75037fe700 1 -- 172.21.2.101:0/1343196340 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f74f400c0c0 con 0x7f75040ef610 2021-02-10T11:37:52.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.027 7f75037fe700 1 -- 172.21.2.101:0/1343196340 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f400c440 con 0x7f75040ef610 2021-02-10T11:37:52.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.027 7f75037fe700 1 -- 172.21.2.101:0/1343196340 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f74f400c920 con 0x7f75040ef610 2021-02-10T11:37:52.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.027 7f750ada6700 1 -- 172.21.2.101:0/1343196340 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75040ef610 msgr2=0x7f75040efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:52.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.027 7f750ada6700 1 --2- 172.21.2.101:0/1343196340 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75040ef610 0x7f75040efa10 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 rx=0x7f74f4006700 tx=0x7f74f4009b80).stop 2021-02-10T11:37:52.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.027 7f750ada6700 1 -- 172.21.2.101:0/1343196340 shutdown_connections 2021-02-10T11:37:52.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.027 7f750ada6700 1 --2- 172.21.2.101:0/1343196340 >> v2:172.21.2.108:3300/0 conn(0x7f75040f0030 0x7f75040f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:52.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.027 7f750ada6700 1 --2- 172.21.2.101:0/1343196340 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75040ef610 0x7f75040efa10 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:52.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.027 7f750ada6700 1 --2- 172.21.2.101:0/1343196340 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75041b0470 0x7f75041b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:52.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.027 7f750ada6700 1 -- 172.21.2.101:0/1343196340 >> 172.21.2.101:0/1343196340 conn(0x7f75040ce3e0 msgr2=0x7f75040e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:52.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.027 7f750ada6700 1 -- 172.21.2.101:0/1343196340 shutdown_connections 2021-02-10T11:37:52.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.027 7f750ada6700 1 -- 172.21.2.101:0/1343196340 wait complete. 2021-02-10T11:37:52.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.042 7f750ada6700 1 Processor -- start 2021-02-10T11:37:52.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.042 7f750ada6700 1 -- start start 2021-02-10T11:37:52.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.042 7f750ada6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75041b6e50 0x7f7504198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:52.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.043 7f750ada6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75041990b0 0x7f75040ad260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:52.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.043 7f750ada6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75040ad880 0x7f75040adc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:52.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.043 7f750ada6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7504199580 con 0x7f75041b6e50 2021-02-10T11:37:52.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.043 7f750ada6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75041b96f0 con 0x7f75041990b0 2021-02-10T11:37:52.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.043 7f750ada6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75041b7250 con 0x7f75040ad880 2021-02-10T11:37:52.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.043 7f7508bc2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75041b6e50 0x7f7504198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:52.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.043 7f7508bc2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75041b6e50 0x7f7504198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54818/0 (socket says 172.21.2.101:54818) 2021-02-10T11:37:52.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.043 7f7508bc2700 1 -- 172.21.2.101:0/3127684314 learned_addr learned my addr 172.21.2.101:0/3127684314 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:52.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.043 7f7503fff700 1 --2- 172.21.2.101:0/3127684314 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75041990b0 0x7f75040ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:52.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.043 7f7508bc2700 1 -- 172.21.2.101:0/3127684314 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75040ad880 msgr2=0x7f75040adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:52.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.043 7f75093c3700 1 --2- 172.21.2.101:0/3127684314 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75040ad880 0x7f75040adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:52.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.043 7f7508bc2700 1 --2- 172.21.2.101:0/3127684314 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75040ad880 0x7f75040adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:52.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.043 7f7508bc2700 1 -- 172.21.2.101:0/3127684314 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75041990b0 msgr2=0x7f75040ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:52.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.043 7f7508bc2700 1 --2- 172.21.2.101:0/3127684314 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75041990b0 0x7f75040ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:52.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.043 7f7508bc2700 1 -- 172.21.2.101:0/3127684314 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7504199d70 con 0x7f75041b6e50 2021-02-10T11:37:52.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.043 7f75093c3700 1 --2- 172.21.2.101:0/3127684314 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75040ad880 0x7f75040adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:37:52.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.043 7f7503fff700 1 --2- 172.21.2.101:0/3127684314 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75041990b0 0x7f75040ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:37:52.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.043 7f7508bc2700 1 --2- 172.21.2.101:0/3127684314 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75041b6e50 0x7f7504198a90 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 rx=0x7f74ec006f00 tx=0x7f74ec00b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:52.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.044 7f75017fa700 1 -- 172.21.2.101:0/3127684314 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f74ec00bd30 con 0x7f75041b6e50 2021-02-10T11:37:52.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.044 7f75017fa700 1 -- 172.21.2.101:0/3127684314 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ec000c90 con 0x7f75041b6e50 2021-02-10T11:37:52.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.044 7f75017fa700 1 -- 172.21.2.101:0/3127684314 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f74ec0022a0 con 0x7f75041b6e50 2021-02-10T11:37:52.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.044 7f750ada6700 1 -- 172.21.2.101:0/3127684314 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7504199d70 con 0x7f75041b6e50 2021-02-10T11:37:52.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.044 7f750ada6700 1 -- 172.21.2.101:0/3127684314 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f750419a300 con 0x7f75041b6e50 2021-02-10T11:37:52.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.044 7f75017fa700 1 -- 172.21.2.101:0/3127684314 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f74ec005310 con 0x7f75041b6e50 2021-02-10T11:37:52.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.044 7f75017fa700 1 --2- 172.21.2.101:0/3127684314 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f74e80186c0 0x7f74e801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:52.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.044 7f75017fa700 1 -- 172.21.2.101:0/3127684314 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f74ec032560 con 0x7f75041b6e50 2021-02-10T11:37:52.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.044 7f7503fff700 1 --2- 172.21.2.101:0/3127684314 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f74e80186c0 0x7f74e801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:52.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.045 7f7503fff700 1 --2- 172.21.2.101:0/3127684314 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f74e80186c0 0x7f74e801abc0 crc :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:52.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.076 7f750ada6700 1 -- 172.21.2.101:0/3127684314 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7504105ac0 con 0x7f75041b6e50 2021-02-10T11:37:52.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.078 7f75017fa700 1 -- 172.21.2.101:0/3127684314 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f74ec025f90 con 0x7f75041b6e50 2021-02-10T11:37:52.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.205 7f750ada6700 1 -- 172.21.2.101:0/3127684314 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75040f0b50 con 0x7f75041b6e50 2021-02-10T11:37:52.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.205 7f75017fa700 1 -- 172.21.2.101:0/3127684314 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f74ec025f90 con 0x7f75041b6e50 2021-02-10T11:37:52.211 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 63 2021-02-10T11:37:52.211 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:37:52.211 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9360,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2529021396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2529021396},{"type":"v1","addr":"172.21.2.101:6835","nonce":2529021396}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:56.648753","modified":"2021-02-10 11:36:58.653009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6005":{"gid":6005,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2363068042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2363068042},{"type":"v1","addr":"172.21.2.108:6833","nonce":2363068042}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-02-10T11:37:52.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.207 7f750ada6700 1 -- 172.21.2.101:0/3127684314 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f74e80186c0 msgr2=0x7f74e801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:52.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.207 7f750ada6700 1 --2- 172.21.2.101:0/3127684314 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f74e80186c0 0x7f74e801abc0 crc :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:52.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.207 7f750ada6700 1 -- 172.21.2.101:0/3127684314 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75041b6e50 msgr2=0x7f7504198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:52.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.207 7f750ada6700 1 --2- 172.21.2.101:0/3127684314 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75041b6e50 0x7f7504198a90 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 rx=0x7f74ec006f00 tx=0x7f74ec00b940).stop 2021-02-10T11:37:52.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.208 7f750ada6700 1 -- 172.21.2.101:0/3127684314 shutdown_connections 2021-02-10T11:37:52.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.208 7f750ada6700 1 --2- 172.21.2.101:0/3127684314 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f74e80186c0 0x7f74e801abc0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:52.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.208 7f750ada6700 1 --2- 172.21.2.101:0/3127684314 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75040ad880 0x7f75040adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:52.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.208 7f750ada6700 1 --2- 172.21.2.101:0/3127684314 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75041990b0 0x7f75040ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:52.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.208 7f750ada6700 1 --2- 172.21.2.101:0/3127684314 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f75041b6e50 0x7f7504198a90 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:52.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.208 7f750ada6700 1 -- 172.21.2.101:0/3127684314 >> 172.21.2.101:0/3127684314 conn(0x7f750419f7c0 msgr2=0x7f750418c230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:52.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.208 7f750ada6700 1 -- 172.21.2.101:0/3127684314 shutdown_connections 2021-02-10T11:37:52.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:52.208 7f750ada6700 1 -- 172.21.2.101:0/3127684314 wait complete. 2021-02-10T11:37:52.221 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok session ls 2021-02-10T11:37:52.615 INFO:teuthology.orchestra.run.gibba008.stdout:[ 2021-02-10T11:37:52.615 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:37:52.615 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 9594, 2021-02-10T11:37:52.615 INFO:teuthology.orchestra.run.gibba008.stdout: "entity": { 2021-02-10T11:37:52.615 INFO:teuthology.orchestra.run.gibba008.stdout: "name": { 2021-02-10T11:37:52.615 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "client", 2021-02-10T11:37:52.615 INFO:teuthology.orchestra.run.gibba008.stdout: "num": 9594 2021-02-10T11:37:52.615 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:52.616 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": { 2021-02-10T11:37:52.616 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "any", 2021-02-10T11:37:52.616 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:37:52.616 INFO:teuthology.orchestra.run.gibba008.stdout: "nonce": 1857925112 2021-02-10T11:37:52.616 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:52.616 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:52.616 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "open", 2021-02-10T11:37:52.616 INFO:teuthology.orchestra.run.gibba008.stdout: "num_leases": 1, 2021-02-10T11:37:52.616 INFO:teuthology.orchestra.run.gibba008.stdout: "num_caps": 7, 2021-02-10T11:37:52.616 INFO:teuthology.orchestra.run.gibba008.stdout: "request_load_avg": 10, 2021-02-10T11:37:52.616 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 26.105583687999999, 2021-02-10T11:37:52.616 INFO:teuthology.orchestra.run.gibba008.stdout: "requests_in_flight": 0, 2021-02-10T11:37:52.616 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": 1, 2021-02-10T11:37:52.616 INFO:teuthology.orchestra.run.gibba008.stdout: "reconnecting": false, 2021-02-10T11:37:52.617 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps": { 2021-02-10T11:37:52.617 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:52.617 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:37:52.617 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:52.617 INFO:teuthology.orchestra.run.gibba008.stdout: "release_caps": { 2021-02-10T11:37:52.617 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:52.617 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:37:52.617 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:52.617 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle": { 2021-02-10T11:37:52.617 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:52.617 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 2.5 2021-02-10T11:37:52.617 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:52.617 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle2o": { 2021-02-10T11:37:52.617 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:52.618 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 0.5 2021-02-10T11:37:52.618 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:52.618 INFO:teuthology.orchestra.run.gibba008.stdout: "session_cache_liveness": { 2021-02-10T11:37:52.618 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 19.784391178149157, 2021-02-10T11:37:52.618 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 300 2021-02-10T11:37:52.618 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:52.618 INFO:teuthology.orchestra.run.gibba008.stdout: "cap_acquisition": { 2021-02-10T11:37:52.618 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:52.618 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 10 2021-02-10T11:37:52.618 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:52.618 INFO:teuthology.orchestra.run.gibba008.stdout: "inst": "client.9594 172.21.2.101:0/1857925112", 2021-02-10T11:37:52.618 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": [ 2021-02-10T11:37:52.618 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:37:52.619 INFO:teuthology.orchestra.run.gibba008.stdout: "tid": 14, 2021-02-10T11:37:52.619 INFO:teuthology.orchestra.run.gibba008.stdout: "created_ino": "0x10000000005" 2021-02-10T11:37:52.619 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:52.619 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:37:52.619 INFO:teuthology.orchestra.run.gibba008.stdout: "prealloc_inos": [ 2021-02-10T11:37:52.619 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:37:52.619 INFO:teuthology.orchestra.run.gibba008.stdout: "start": 1099511627782, 2021-02-10T11:37:52.619 INFO:teuthology.orchestra.run.gibba008.stdout: "length": 995 2021-02-10T11:37:52.619 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:52.619 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:37:52.619 INFO:teuthology.orchestra.run.gibba008.stdout: "used_inos": [], 2021-02-10T11:37:52.619 INFO:teuthology.orchestra.run.gibba008.stdout: "client_metadata": { 2021-02-10T11:37:52.619 INFO:teuthology.orchestra.run.gibba008.stdout: "features": "0x0000000000001fff", 2021-02-10T11:37:52.620 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:37:52.620 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:37:52.620 INFO:teuthology.orchestra.run.gibba008.stdout: "entity_id": "y", 2021-02-10T11:37:52.620 INFO:teuthology.orchestra.run.gibba008.stdout: "hostname": "gibba001", 2021-02-10T11:37:52.620 INFO:teuthology.orchestra.run.gibba008.stdout: "pid": "24165", 2021-02-10T11:37:52.620 INFO:teuthology.orchestra.run.gibba008.stdout: "root": "/" 2021-02-10T11:37:52.620 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:52.620 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:52.620 INFO:teuthology.orchestra.run.gibba008.stdout:] 2021-02-10T11:37:52.621 INFO:tasks.cephfs.filesystem:_json_asok output: [ { "id": 9594, "entity": { "name": { "type": "client", "num": 9594 }, "addr": { "type": "any", "addr": "172.21.2.101:0", "nonce": 1857925112 } }, "state": "open", "num_leases": 1, "num_caps": 7, "request_load_avg": 10, "uptime": 26.105583687999999, "requests_in_flight": 0, "completed_requests": 1, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 2.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 19.784391178149157, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 10 }, "inst": "client.9594 172.21.2.101:0/1857925112", "completed_requests": [ { "tid": 14, "created_ino": "0x10000000005" } ], "prealloc_inos": [ { "start": 1099511627782, "length": 995 } ], "used_inos": [], "client_metadata": { "features": "0x0000000000001fff", "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", "entity_id": "y", "hostname": "gibba001", "pid": "24165", "root": "/" } } ] 2021-02-10T11:37:52.621 DEBUG:tasks.ceph_test_case:wait_until_true: waiting... 2021-02-10T11:37:57.623 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:37:58.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.030 7f2dcfbd1700 1 Processor -- start 2021-02-10T11:37:58.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.031 7f2dcfbd1700 1 -- start start 2021-02-10T11:37:58.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.031 7f2dcfbd1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2dc80d1870 0x7f2dc81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:58.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.031 7f2dcfbd1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2dc80f0650 0x7f2dc80cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:58.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.031 7f2dcfbd1700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc80f0b10 con 0x7f2dc81b0a10 2021-02-10T11:37:58.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.031 7f2dcfbd1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dc80fe230 con 0x7f2dc80f0650 2021-02-10T11:37:58.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.031 7f2dcfbd1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc80e5ed0 con 0x7f2dc80d1870 2021-02-10T11:37:58.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.031 7f2dcd1ec700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f2dc81b0a10 0x7f2dc80cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55320/0 (socket says 172.21.2.101:55320) 2021-02-10T11:37:58.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.031 7f2dcd1ec700 1 -- 172.21.2.101:0/498752752 learned_addr learned my addr 172.21.2.101:0/498752752 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:37:58.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.032 7f2dcc9eb700 1 -- 172.21.2.101:0/498752752 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2917174190 0 0) 0x7f2dbc0009b0 con 0x7f2dc81b0a10 2021-02-10T11:37:58.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.032 7f2dcc9eb700 1 -- 172.21.2.101:0/498752752 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2db4002fe0 con 0x7f2dc81b0a10 2021-02-10T11:37:58.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.032 7f2dcd9ed700 1 --2- 172.21.2.101:0/498752752 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2dc80d1870 0x7f2dc81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:58.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.032 7f2dce1ee700 1 --2- 172.21.2.101:0/498752752 >> v2:172.21.2.108:3300/0 conn(0x7f2dc80f0650 0x7f2dc80cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:58.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.032 7f2dcc9eb700 1 -- 172.21.2.101:0/498752752 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3326674409 0 0) 0x7f2dbc001010 con 0x7f2dc81b0a10 2021-02-10T11:37:58.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.032 7f2dcc9eb700 1 -- 172.21.2.101:0/498752752 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2dc80d1870 msgr2=0x7f2dc81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:58.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.032 7f2dcc9eb700 1 --2- 172.21.2.101:0/498752752 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2dc80d1870 0x7f2dc81b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:58.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.032 7f2dcc9eb700 1 -- 172.21.2.101:0/498752752 >> v2:172.21.2.108:3300/0 conn(0x7f2dc80f0650 msgr2=0x7f2dc80cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:58.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.032 7f2dcc9eb700 1 --2- 172.21.2.101:0/498752752 >> v2:172.21.2.108:3300/0 conn(0x7f2dc80f0650 0x7f2dc80cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:58.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.032 7f2dcc9eb700 1 -- 172.21.2.101:0/498752752 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc80f20a0 con 0x7f2dc81b0a10 2021-02-10T11:37:58.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.032 7f2dcc9eb700 1 -- 172.21.2.101:0/498752752 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f2dbc001590 con 0x7f2dc81b0a10 2021-02-10T11:37:58.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.032 7f2dcc9eb700 1 -- 172.21.2.101:0/498752752 >> v1:172.21.2.101:6789/0 conn(0x7f2dc81b0a10 legacy=0x7f2dc80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:58.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.032 7f2dcc9eb700 1 --2- 172.21.2.101:0/498752752 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2db4003af0 0x7f2db4005f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:58.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.032 7f2dcc9eb700 1 --2- 172.21.2.101:0/498752752 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2db404a790 0x7f2db404cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:58.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.032 7f2dcc9eb700 1 --2- 172.21.2.101:0/498752752 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2db404d510 0x7f2db404fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:58.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.032 7f2dcc9eb700 1 -- 172.21.2.101:0/498752752 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db40502e0 con 0x7f2db404d510 2021-02-10T11:37:58.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.032 7f2dcc9eb700 1 -- 172.21.2.101:0/498752752 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db4050500 con 0x7f2db404a790 2021-02-10T11:37:58.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.032 7f2dcc9eb700 1 -- 172.21.2.101:0/498752752 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db4050720 con 0x7f2db4003af0 2021-02-10T11:37:58.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.032 7f2dce1ee700 1 --2- 172.21.2.101:0/498752752 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2db404d510 0x7f2db404fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:58.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.032 7f2dcd9ed700 1 --2- 172.21.2.101:0/498752752 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2db4003af0 0x7f2db4005f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:58.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.033 7f2dcd1ec700 1 --2- 172.21.2.101:0/498752752 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2db404a790 0x7f2db404cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:58.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.033 7f2dce1ee700 1 -- 172.21.2.101:0/498752752 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2db4003af0 msgr2=0x7f2db4005f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:58.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.033 7f2dce1ee700 1 --2- 172.21.2.101:0/498752752 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2db4003af0 0x7f2db4005f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:58.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.033 7f2dce1ee700 1 -- 172.21.2.101:0/498752752 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2db404a790 msgr2=0x7f2db404cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:37:58.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.033 7f2dce1ee700 1 --2- 172.21.2.101:0/498752752 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2db404a790 0x7f2db404cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:58.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.033 7f2dce1ee700 1 -- 172.21.2.101:0/498752752 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db40509c0 con 0x7f2db404d510 2021-02-10T11:37:58.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.033 7f2dce1ee700 1 --2- 172.21.2.101:0/498752752 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2db404d510 0x7f2db404fa30 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 rx=0x7f2dc0006a40 tx=0x7f2dc0009f40).ready entity=mon.0 client_cookie=de3248bef5c6389e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:58.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.033 7f2dcc9eb700 1 -- 172.21.2.101:0/498752752 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2dc000c420 con 0x7f2db404d510 2021-02-10T11:37:58.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.033 7f2dcc9eb700 1 -- 172.21.2.101:0/498752752 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc000c6a0 con 0x7f2db404d510 2021-02-10T11:37:58.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.033 7f2dcc9eb700 1 -- 172.21.2.101:0/498752752 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2dc000cb00 con 0x7f2db404d510 2021-02-10T11:37:58.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.033 7f2dcfbd1700 1 -- 172.21.2.101:0/498752752 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2db404d510 msgr2=0x7f2db404fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:58.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.033 7f2dcfbd1700 1 --2- 172.21.2.101:0/498752752 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2db404d510 0x7f2db404fa30 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 rx=0x7f2dc0006a40 tx=0x7f2dc0009f40).stop 2021-02-10T11:37:58.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.033 7f2dce1ee700 1 -- 172.21.2.101:0/498752752 reap_dead start 2021-02-10T11:37:58.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.034 7f2dcfbd1700 1 -- 172.21.2.101:0/498752752 shutdown_connections 2021-02-10T11:37:58.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.034 7f2dcfbd1700 1 -- 172.21.2.101:0/498752752 >> 172.21.2.101:0/498752752 conn(0x7f2dc80ce450 msgr2=0x7f2dc80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:58.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.034 7f2dcfbd1700 1 -- 172.21.2.101:0/498752752 shutdown_connections 2021-02-10T11:37:58.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.034 7f2dcfbd1700 1 -- 172.21.2.101:0/498752752 wait complete. 2021-02-10T11:37:58.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.049 7f2dcfbd1700 1 Processor -- start 2021-02-10T11:37:58.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.049 7f2dcfbd1700 1 -- start start 2021-02-10T11:37:58.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.049 7f2dcfbd1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2dc80edfb0 0x7f2dc80a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:58.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.049 7f2dcfbd1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2dc80bf2c0 0x7f2dc80a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:58.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.049 7f2dcfbd1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2dc80a4cf0 0x7f2dc80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:58.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.049 7f2dcfbd1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc80eec00 con 0x7f2dc80a4cf0 2021-02-10T11:37:58.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.049 7f2dcfbd1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc80a3ed0 con 0x7f2dc80bf2c0 2021-02-10T11:37:58.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.049 7f2dcfbd1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc80a4150 con 0x7f2dc80edfb0 2021-02-10T11:37:58.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.049 7f2dce1ee700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2dc80a4cf0 0x7f2dc80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:58.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.050 7f2dce1ee700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2dc80a4cf0 0x7f2dc80a50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54836/0 (socket says 172.21.2.101:54836) 2021-02-10T11:37:58.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.050 7f2dce1ee700 1 -- 172.21.2.101:0/2823009595 learned_addr learned my addr 172.21.2.101:0/2823009595 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:37:58.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.050 7f2dce1ee700 1 -- 172.21.2.101:0/2823009595 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2dc80edfb0 msgr2=0x7f2dc80a38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:58.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.050 7f2dce1ee700 1 --2- 172.21.2.101:0/2823009595 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2dc80edfb0 0x7f2dc80a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:58.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.050 7f2dce1ee700 1 -- 172.21.2.101:0/2823009595 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2dc80bf2c0 msgr2=0x7f2dc80a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:58.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.050 7f2dce1ee700 1 --2- 172.21.2.101:0/2823009595 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2dc80bf2c0 0x7f2dc80a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:58.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.050 7f2dce1ee700 1 -- 172.21.2.101:0/2823009595 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc80ee3b0 con 0x7f2dc80a4cf0 2021-02-10T11:37:58.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.050 7f2dce1ee700 1 --2- 172.21.2.101:0/2823009595 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2dc80a4cf0 0x7f2dc80a50f0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 rx=0x7f2dc000b910 tx=0x7f2dc000bcb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:58.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.050 7f2dba7fc700 1 -- 172.21.2.101:0/2823009595 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2dc0002030 con 0x7f2dc80a4cf0 2021-02-10T11:37:58.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.050 7f2dba7fc700 1 -- 172.21.2.101:0/2823009595 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc0000df0 con 0x7f2dc80a4cf0 2021-02-10T11:37:58.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.050 7f2dba7fc700 1 -- 172.21.2.101:0/2823009595 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2dc0002560 con 0x7f2dc80a4cf0 2021-02-10T11:37:58.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.050 7f2dcfbd1700 1 -- 172.21.2.101:0/2823009595 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc80ee3b0 con 0x7f2dc80a4cf0 2021-02-10T11:37:58.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.051 7f2dcfbd1700 1 -- 172.21.2.101:0/2823009595 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc80ee3b0 con 0x7f2dc80a4cf0 2021-02-10T11:37:58.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.051 7f2dba7fc700 1 -- 172.21.2.101:0/2823009595 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2dc00056b0 con 0x7f2dc80a4cf0 2021-02-10T11:37:58.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.051 7f2dba7fc700 1 --2- 172.21.2.101:0/2823009595 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2db00186c0 0x7f2db001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:37:58.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.051 7f2dba7fc700 1 -- 172.21.2.101:0/2823009595 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2dc00328e0 con 0x7f2dc80a4cf0 2021-02-10T11:37:58.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.051 7f2dcd9ed700 1 --2- 172.21.2.101:0/2823009595 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2db00186c0 0x7f2db001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:37:58.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.052 7f2dcd9ed700 1 --2- 172.21.2.101:0/2823009595 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2db00186c0 0x7f2db001abc0 crc :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:37:58.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.081 7f2dcfbd1700 1 -- 172.21.2.101:0/2823009595 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc80ebcd0 con 0x7f2dc80a4cf0 2021-02-10T11:37:58.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.083 7f2dba7fc700 1 -- 172.21.2.101:0/2823009595 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2dc000bdf0 con 0x7f2dc80a4cf0 2021-02-10T11:37:58.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.213 7f2dcfbd1700 1 -- 172.21.2.101:0/2823009595 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2dc80ebcd0 con 0x7f2dc80a4cf0 2021-02-10T11:37:58.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.213 7f2dba7fc700 1 -- 172.21.2.101:0/2823009595 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f2dc001f6b0 con 0x7f2dc80a4cf0 2021-02-10T11:37:58.219 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 63 2021-02-10T11:37:58.219 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:37:58.219 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9360,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2529021396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2529021396},{"type":"v1","addr":"172.21.2.101:6835","nonce":2529021396}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:56.648753","modified":"2021-02-10 11:36:58.653009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6005":{"gid":6005,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2363068042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2363068042},{"type":"v1","addr":"172.21.2.108:6833","nonce":2363068042}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-02-10T11:37:58.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.215 7f2dcfbd1700 1 -- 172.21.2.101:0/2823009595 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2db00186c0 msgr2=0x7f2db001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:58.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.215 7f2dcfbd1700 1 --2- 172.21.2.101:0/2823009595 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2db00186c0 0x7f2db001abc0 crc :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:58.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.215 7f2dcfbd1700 1 -- 172.21.2.101:0/2823009595 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2dc80a4cf0 msgr2=0x7f2dc80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:37:58.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.215 7f2dcfbd1700 1 --2- 172.21.2.101:0/2823009595 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2dc80a4cf0 0x7f2dc80a50f0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 rx=0x7f2dc000b910 tx=0x7f2dc000bcb0).stop 2021-02-10T11:37:58.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.215 7f2dcfbd1700 1 -- 172.21.2.101:0/2823009595 shutdown_connections 2021-02-10T11:37:58.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.215 7f2dcfbd1700 1 --2- 172.21.2.101:0/2823009595 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2dc80a4cf0 0x7f2dc80a50f0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:58.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.215 7f2dcfbd1700 1 --2- 172.21.2.101:0/2823009595 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2dc80bf2c0 0x7f2dc80a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:58.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.215 7f2dcfbd1700 1 --2- 172.21.2.101:0/2823009595 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2db00186c0 0x7f2db001abc0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:37:58.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.215 7f2dcfbd1700 1 --2- 172.21.2.101:0/2823009595 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2dc80edfb0 0x7f2dc80a38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:37:58.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.215 7f2dcfbd1700 1 -- 172.21.2.101:0/2823009595 >> 172.21.2.101:0/2823009595 conn(0x7f2dc80a2ba0 msgr2=0x7f2dc80bfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:37:58.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.215 7f2dcfbd1700 1 -- 172.21.2.101:0/2823009595 shutdown_connections 2021-02-10T11:37:58.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:37:58.216 7f2dcfbd1700 1 -- 172.21.2.101:0/2823009595 wait complete. 2021-02-10T11:37:58.229 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok session ls 2021-02-10T11:37:58.625 INFO:teuthology.orchestra.run.gibba008.stdout:[ 2021-02-10T11:37:58.626 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:37:58.626 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 9594, 2021-02-10T11:37:58.626 INFO:teuthology.orchestra.run.gibba008.stdout: "entity": { 2021-02-10T11:37:58.626 INFO:teuthology.orchestra.run.gibba008.stdout: "name": { 2021-02-10T11:37:58.626 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "client", 2021-02-10T11:37:58.626 INFO:teuthology.orchestra.run.gibba008.stdout: "num": 9594 2021-02-10T11:37:58.626 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:58.626 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": { 2021-02-10T11:37:58.626 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "any", 2021-02-10T11:37:58.626 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:37:58.627 INFO:teuthology.orchestra.run.gibba008.stdout: "nonce": 1857925112 2021-02-10T11:37:58.627 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:58.627 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:58.627 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "open", 2021-02-10T11:37:58.627 INFO:teuthology.orchestra.run.gibba008.stdout: "num_leases": 0, 2021-02-10T11:37:58.627 INFO:teuthology.orchestra.run.gibba008.stdout: "num_caps": 7, 2021-02-10T11:37:58.627 INFO:teuthology.orchestra.run.gibba008.stdout: "request_load_avg": 9, 2021-02-10T11:37:58.627 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 32.115270940999999, 2021-02-10T11:37:58.627 INFO:teuthology.orchestra.run.gibba008.stdout: "requests_in_flight": 0, 2021-02-10T11:37:58.627 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": 1, 2021-02-10T11:37:58.627 INFO:teuthology.orchestra.run.gibba008.stdout: "reconnecting": false, 2021-02-10T11:37:58.627 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps": { 2021-02-10T11:37:58.627 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:58.628 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:37:58.628 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:58.628 INFO:teuthology.orchestra.run.gibba008.stdout: "release_caps": { 2021-02-10T11:37:58.628 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:58.628 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:37:58.628 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:58.628 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle": { 2021-02-10T11:37:58.628 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:58.628 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 2.5 2021-02-10T11:37:58.628 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:58.628 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle2o": { 2021-02-10T11:37:58.628 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:58.628 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 0.5 2021-02-10T11:37:58.629 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:58.629 INFO:teuthology.orchestra.run.gibba008.stdout: "session_cache_liveness": { 2021-02-10T11:37:58.629 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 19.511576899388736, 2021-02-10T11:37:58.629 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 300 2021-02-10T11:37:58.629 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:58.629 INFO:teuthology.orchestra.run.gibba008.stdout: "cap_acquisition": { 2021-02-10T11:37:58.629 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:37:58.629 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 10 2021-02-10T11:37:58.629 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:37:58.629 INFO:teuthology.orchestra.run.gibba008.stdout: "inst": "client.9594 172.21.2.101:0/1857925112", 2021-02-10T11:37:58.629 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": [ 2021-02-10T11:37:58.629 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:37:58.629 INFO:teuthology.orchestra.run.gibba008.stdout: "tid": 14, 2021-02-10T11:37:58.630 INFO:teuthology.orchestra.run.gibba008.stdout: "created_ino": "0x10000000005" 2021-02-10T11:37:58.630 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:58.630 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:37:58.630 INFO:teuthology.orchestra.run.gibba008.stdout: "prealloc_inos": [ 2021-02-10T11:37:58.630 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:37:58.630 INFO:teuthology.orchestra.run.gibba008.stdout: "start": 1099511627782, 2021-02-10T11:37:58.630 INFO:teuthology.orchestra.run.gibba008.stdout: "length": 995 2021-02-10T11:37:58.630 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:58.630 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:37:58.630 INFO:teuthology.orchestra.run.gibba008.stdout: "used_inos": [], 2021-02-10T11:37:58.630 INFO:teuthology.orchestra.run.gibba008.stdout: "client_metadata": { 2021-02-10T11:37:58.630 INFO:teuthology.orchestra.run.gibba008.stdout: "features": "0x0000000000001fff", 2021-02-10T11:37:58.630 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:37:58.631 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:37:58.631 INFO:teuthology.orchestra.run.gibba008.stdout: "entity_id": "y", 2021-02-10T11:37:58.631 INFO:teuthology.orchestra.run.gibba008.stdout: "hostname": "gibba001", 2021-02-10T11:37:58.631 INFO:teuthology.orchestra.run.gibba008.stdout: "pid": "24165", 2021-02-10T11:37:58.631 INFO:teuthology.orchestra.run.gibba008.stdout: "root": "/" 2021-02-10T11:37:58.631 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:58.631 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:37:58.631 INFO:teuthology.orchestra.run.gibba008.stdout:] 2021-02-10T11:37:58.632 INFO:tasks.cephfs.filesystem:_json_asok output: [ { "id": 9594, "entity": { "name": { "type": "client", "num": 9594 }, "addr": { "type": "any", "addr": "172.21.2.101:0", "nonce": 1857925112 } }, "state": "open", "num_leases": 0, "num_caps": 7, "request_load_avg": 9, "uptime": 32.115270940999999, "requests_in_flight": 0, "completed_requests": 1, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 2.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 19.511576899388736, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 10 }, "inst": "client.9594 172.21.2.101:0/1857925112", "completed_requests": [ { "tid": 14, "created_ino": "0x10000000005" } ], "prealloc_inos": [ { "start": 1099511627782, "length": 995 } ], "used_inos": [], "client_metadata": { "features": "0x0000000000001fff", "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", "entity_id": "y", "hostname": "gibba001", "pid": "24165", "root": "/" } } ] 2021-02-10T11:37:58.632 DEBUG:tasks.ceph_test_case:wait_until_true: waiting... 2021-02-10T11:38:02.000 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:38:02.005 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:38:03.633 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:38:04.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.038 7efd3b036700 1 Processor -- start 2021-02-10T11:38:04.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.038 7efd3b036700 1 -- start start 2021-02-10T11:38:04.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.039 7efd3b036700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd341b0470 0x7efd341b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:04.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.039 7efd3b036700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd340f0b50 con 0x7efd340ef610 2021-02-10T11:38:04.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.039 7efd3b036700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd340f24e0 con 0x7efd340f20e0 2021-02-10T11:38:04.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.039 7efd3b036700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd340e5ea0 con 0x7efd341b0470 2021-02-10T11:38:04.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.039 7efd33fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7efd340ef610 0x7efd340efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55336/0 (socket says 172.21.2.101:55336) 2021-02-10T11:38:04.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.039 7efd33fff700 1 -- 172.21.2.101:0/2243644011 learned_addr learned my addr 172.21.2.101:0/2243644011 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:38:04.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.039 7efd337fe700 1 -- 172.21.2.101:0/2243644011 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3634968581 0 0) 0x7efd24000ce0 con 0x7efd340ef610 2021-02-10T11:38:04.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.039 7efd337fe700 1 -- 172.21.2.101:0/2243644011 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd20002fe0 con 0x7efd340ef610 2021-02-10T11:38:04.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd38e52700 1 --2- 172.21.2.101:0/2243644011 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd341b0470 0x7efd341b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:04.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd337fe700 1 -- 172.21.2.101:0/2243644011 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2739606448 0 0) 0x7efd240010e0 con 0x7efd340ef610 2021-02-10T11:38:04.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd337fe700 1 -- 172.21.2.101:0/2243644011 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd341b0470 msgr2=0x7efd341b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:04.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd337fe700 1 --2- 172.21.2.101:0/2243644011 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd341b0470 0x7efd341b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:04.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd337fe700 1 -- 172.21.2.101:0/2243644011 >> v1:172.21.2.108:6789/0 conn(0x7efd340f20e0 legacy=0x7efd340f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:04.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd337fe700 1 -- 172.21.2.101:0/2243644011 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd340e52e0 con 0x7efd340ef610 2021-02-10T11:38:04.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd337fe700 1 -- 172.21.2.101:0/2243644011 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7efd24001660 con 0x7efd340ef610 2021-02-10T11:38:04.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd337fe700 1 -- 172.21.2.101:0/2243644011 >> v1:172.21.2.101:6789/0 conn(0x7efd340ef610 legacy=0x7efd340efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:04.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd337fe700 1 --2- 172.21.2.101:0/2243644011 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd20003a60 0x7efd20005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:04.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd337fe700 1 --2- 172.21.2.101:0/2243644011 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd20006780 0x7efd20008ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:04.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd337fe700 1 --2- 172.21.2.101:0/2243644011 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd200095a0 0x7efd2000ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:04.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd337fe700 1 -- 172.21.2.101:0/2243644011 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2000c2c0 con 0x7efd20003a60 2021-02-10T11:38:04.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd337fe700 1 -- 172.21.2.101:0/2243644011 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2000c4e0 con 0x7efd20006780 2021-02-10T11:38:04.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd337fe700 1 -- 172.21.2.101:0/2243644011 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd2000c700 con 0x7efd200095a0 2021-02-10T11:38:04.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd38e52700 1 --2- 172.21.2.101:0/2243644011 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd20003a60 0x7efd20005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:04.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd33fff700 1 --2- 172.21.2.101:0/2243644011 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd20006780 0x7efd20008ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:04.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd39653700 1 --2- 172.21.2.101:0/2243644011 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd200095a0 0x7efd2000ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:04.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd38e52700 1 -- 172.21.2.101:0/2243644011 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd200095a0 msgr2=0x7efd2000ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:04.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd38e52700 1 --2- 172.21.2.101:0/2243644011 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd200095a0 0x7efd2000ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:04.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd38e52700 1 -- 172.21.2.101:0/2243644011 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd20006780 msgr2=0x7efd20008ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:04.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd38e52700 1 --2- 172.21.2.101:0/2243644011 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd20006780 0x7efd20008ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:04.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd38e52700 1 -- 172.21.2.101:0/2243644011 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd20050940 con 0x7efd20003a60 2021-02-10T11:38:04.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.040 7efd39653700 1 --2- 172.21.2.101:0/2243644011 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd200095a0 0x7efd2000ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:38:04.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.041 7efd38e52700 1 --2- 172.21.2.101:0/2243644011 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd20003a60 0x7efd20005f60 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 rx=0x7efd1c009310 tx=0x7efd1c009a90).ready entity=mon.0 client_cookie=6906e7276449bbe5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:04.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.041 7efd337fe700 1 -- 172.21.2.101:0/2243644011 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd1c00c0c0 con 0x7efd20003a60 2021-02-10T11:38:04.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.041 7efd337fe700 1 -- 172.21.2.101:0/2243644011 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1c006400 con 0x7efd20003a60 2021-02-10T11:38:04.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.041 7efd337fe700 1 -- 172.21.2.101:0/2243644011 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd1c00c400 con 0x7efd20003a60 2021-02-10T11:38:04.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.041 7efd3b036700 1 -- 172.21.2.101:0/2243644011 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd20003a60 msgr2=0x7efd20005f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:04.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.041 7efd3b036700 1 --2- 172.21.2.101:0/2243644011 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd20003a60 0x7efd20005f60 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 rx=0x7efd1c009310 tx=0x7efd1c009a90).stop 2021-02-10T11:38:04.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.041 7efd39653700 1 -- 172.21.2.101:0/2243644011 reap_dead start 2021-02-10T11:38:04.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.041 7efd3b036700 1 -- 172.21.2.101:0/2243644011 shutdown_connections 2021-02-10T11:38:04.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.041 7efd3b036700 1 -- 172.21.2.101:0/2243644011 >> 172.21.2.101:0/2243644011 conn(0x7efd340ce3e0 msgr2=0x7efd340e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:04.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.041 7efd3b036700 1 -- 172.21.2.101:0/2243644011 shutdown_connections 2021-02-10T11:38:04.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.041 7efd3b036700 1 -- 172.21.2.101:0/2243644011 wait complete. 2021-02-10T11:38:04.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.056 7efd3b036700 1 Processor -- start 2021-02-10T11:38:04.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.056 7efd3b036700 1 -- start start 2021-02-10T11:38:04.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.057 7efd3b036700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd3418c720 0x7efd3418cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:04.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.057 7efd3b036700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd3418f2b0 0x7efd3418f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:04.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.057 7efd3b036700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd34090970 0x7efd34090d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:04.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.057 7efd3b036700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3418fcd0 con 0x7efd3418c720 2021-02-10T11:38:04.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.057 7efd3b036700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd34090070 con 0x7efd3418f2b0 2021-02-10T11:38:04.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.057 7efd3b036700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd34091390 con 0x7efd34090970 2021-02-10T11:38:04.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.057 7efd38e52700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd3418c720 0x7efd3418cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:04.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.057 7efd38e52700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd3418c720 0x7efd3418cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54856/0 (socket says 172.21.2.101:54856) 2021-02-10T11:38:04.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.057 7efd38e52700 1 -- 172.21.2.101:0/1049114145 learned_addr learned my addr 172.21.2.101:0/1049114145 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:04.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.057 7efd39653700 1 --2- 172.21.2.101:0/1049114145 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd34090970 0x7efd34090d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:04.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.057 7efd33fff700 1 --2- 172.21.2.101:0/1049114145 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd3418f2b0 0x7efd3418f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:04.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.057 7efd38e52700 1 -- 172.21.2.101:0/1049114145 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd34090970 msgr2=0x7efd34090d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:04.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.057 7efd38e52700 1 --2- 172.21.2.101:0/1049114145 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd34090970 0x7efd34090d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:04.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.057 7efd38e52700 1 -- 172.21.2.101:0/1049114145 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd3418f2b0 msgr2=0x7efd3418f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:04.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.057 7efd38e52700 1 --2- 172.21.2.101:0/1049114145 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd3418f2b0 0x7efd3418f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:04.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.057 7efd38e52700 1 -- 172.21.2.101:0/1049114145 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd340915b0 con 0x7efd3418c720 2021-02-10T11:38:04.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.057 7efd39653700 1 --2- 172.21.2.101:0/1049114145 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd34090970 0x7efd34090d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:38:04.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.057 7efd38e52700 1 --2- 172.21.2.101:0/1049114145 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd3418c720 0x7efd3418cb20 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 rx=0x7efd1c009670 tx=0x7efd1c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:04.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.058 7efd317fa700 1 -- 172.21.2.101:0/1049114145 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd1c000940 con 0x7efd3418c720 2021-02-10T11:38:04.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.058 7efd317fa700 1 -- 172.21.2.101:0/1049114145 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1c0093e0 con 0x7efd3418c720 2021-02-10T11:38:04.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.058 7efd317fa700 1 -- 172.21.2.101:0/1049114145 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd1c002080 con 0x7efd3418c720 2021-02-10T11:38:04.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.058 7efd3b036700 1 -- 172.21.2.101:0/1049114145 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd340915b0 con 0x7efd3418c720 2021-02-10T11:38:04.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.058 7efd3b036700 1 -- 172.21.2.101:0/1049114145 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd34091b40 con 0x7efd3418c720 2021-02-10T11:38:04.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.058 7efd317fa700 1 -- 172.21.2.101:0/1049114145 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7efd1c002020 con 0x7efd3418c720 2021-02-10T11:38:04.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.058 7efd317fa700 1 --2- 172.21.2.101:0/1049114145 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd180186c0 0x7efd1801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:04.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.058 7efd317fa700 1 -- 172.21.2.101:0/1049114145 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7efd1c0325b0 con 0x7efd3418c720 2021-02-10T11:38:04.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.058 7efd33fff700 1 --2- 172.21.2.101:0/1049114145 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd180186c0 0x7efd1801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:04.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.059 7efd33fff700 1 --2- 172.21.2.101:0/1049114145 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd180186c0 0x7efd1801abc0 crc :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:04.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.089 7efd3b036700 1 -- 172.21.2.101:0/1049114145 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd34105ad0 con 0x7efd3418c720 2021-02-10T11:38:04.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.091 7efd317fa700 1 -- 172.21.2.101:0/1049114145 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7efd1c030d60 con 0x7efd3418c720 2021-02-10T11:38:04.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.219 7efd3b036700 1 -- 172.21.2.101:0/1049114145 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd340f0b50 con 0x7efd3418c720 2021-02-10T11:38:04.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.219 7efd317fa700 1 -- 172.21.2.101:0/1049114145 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+2101 (secure 0 0 0) 0x7efd1c00bd50 con 0x7efd3418c720 2021-02-10T11:38:04.225 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 63 2021-02-10T11:38:04.225 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:38:04.225 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9360,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2529021396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2529021396},{"type":"v1","addr":"172.21.2.101:6835","nonce":2529021396}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:56.648753","modified":"2021-02-10 11:36:58.653009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6005":{"gid":6005,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2363068042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2363068042},{"type":"v1","addr":"172.21.2.108:6833","nonce":2363068042}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-02-10T11:38:04.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.221 7efd3b036700 1 -- 172.21.2.101:0/1049114145 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd180186c0 msgr2=0x7efd1801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:04.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.221 7efd3b036700 1 --2- 172.21.2.101:0/1049114145 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd180186c0 0x7efd1801abc0 crc :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:04.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.221 7efd3b036700 1 -- 172.21.2.101:0/1049114145 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd3418c720 msgr2=0x7efd3418cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:04.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.221 7efd3b036700 1 --2- 172.21.2.101:0/1049114145 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd3418c720 0x7efd3418cb20 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 rx=0x7efd1c009670 tx=0x7efd1c00b960).stop 2021-02-10T11:38:04.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.222 7efd3b036700 1 -- 172.21.2.101:0/1049114145 shutdown_connections 2021-02-10T11:38:04.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.222 7efd3b036700 1 --2- 172.21.2.101:0/1049114145 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd180186c0 0x7efd1801abc0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:04.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.222 7efd3b036700 1 --2- 172.21.2.101:0/1049114145 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd34090970 0x7efd34090d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:04.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.222 7efd3b036700 1 --2- 172.21.2.101:0/1049114145 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd3418f2b0 0x7efd3418f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:04.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.222 7efd3b036700 1 --2- 172.21.2.101:0/1049114145 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd3418c720 0x7efd3418cb20 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:04.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.222 7efd3b036700 1 -- 172.21.2.101:0/1049114145 >> 172.21.2.101:0/1049114145 conn(0x7efd3410a400 msgr2=0x7efd340b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:04.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.222 7efd3b036700 1 -- 172.21.2.101:0/1049114145 shutdown_connections 2021-02-10T11:38:04.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:04.222 7efd3b036700 1 -- 172.21.2.101:0/1049114145 wait complete. 2021-02-10T11:38:04.235 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok session ls 2021-02-10T11:38:04.631 INFO:teuthology.orchestra.run.gibba008.stdout:[ 2021-02-10T11:38:04.631 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:38:04.631 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 9594, 2021-02-10T11:38:04.631 INFO:teuthology.orchestra.run.gibba008.stdout: "entity": { 2021-02-10T11:38:04.632 INFO:teuthology.orchestra.run.gibba008.stdout: "name": { 2021-02-10T11:38:04.632 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "client", 2021-02-10T11:38:04.632 INFO:teuthology.orchestra.run.gibba008.stdout: "num": 9594 2021-02-10T11:38:04.632 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:04.632 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": { 2021-02-10T11:38:04.632 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "any", 2021-02-10T11:38:04.632 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:38:04.632 INFO:teuthology.orchestra.run.gibba008.stdout: "nonce": 1857925112 2021-02-10T11:38:04.632 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:04.632 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:04.632 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "open", 2021-02-10T11:38:04.632 INFO:teuthology.orchestra.run.gibba008.stdout: "num_leases": 0, 2021-02-10T11:38:04.632 INFO:teuthology.orchestra.run.gibba008.stdout: "num_caps": 7, 2021-02-10T11:38:04.632 INFO:teuthology.orchestra.run.gibba008.stdout: "request_load_avg": 9, 2021-02-10T11:38:04.633 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 38.120963134, 2021-02-10T11:38:04.633 INFO:teuthology.orchestra.run.gibba008.stdout: "requests_in_flight": 0, 2021-02-10T11:38:04.633 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": 1, 2021-02-10T11:38:04.633 INFO:teuthology.orchestra.run.gibba008.stdout: "reconnecting": false, 2021-02-10T11:38:04.633 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps": { 2021-02-10T11:38:04.633 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:04.633 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:38:04.633 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:04.633 INFO:teuthology.orchestra.run.gibba008.stdout: "release_caps": { 2021-02-10T11:38:04.633 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:04.633 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:38:04.633 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:04.633 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle": { 2021-02-10T11:38:04.634 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:04.634 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 2.5 2021-02-10T11:38:04.634 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:04.634 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle2o": { 2021-02-10T11:38:04.634 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:04.634 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 0.5 2021-02-10T11:38:04.634 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:04.634 INFO:teuthology.orchestra.run.gibba008.stdout: "session_cache_liveness": { 2021-02-10T11:38:04.634 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 19.242702177379513, 2021-02-10T11:38:04.634 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 300 2021-02-10T11:38:04.634 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:04.634 INFO:teuthology.orchestra.run.gibba008.stdout: "cap_acquisition": { 2021-02-10T11:38:04.634 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:04.635 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 10 2021-02-10T11:38:04.635 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:04.635 INFO:teuthology.orchestra.run.gibba008.stdout: "inst": "client.9594 172.21.2.101:0/1857925112", 2021-02-10T11:38:04.635 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": [ 2021-02-10T11:38:04.635 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:38:04.635 INFO:teuthology.orchestra.run.gibba008.stdout: "tid": 14, 2021-02-10T11:38:04.635 INFO:teuthology.orchestra.run.gibba008.stdout: "created_ino": "0x10000000005" 2021-02-10T11:38:04.635 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:04.635 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:38:04.635 INFO:teuthology.orchestra.run.gibba008.stdout: "prealloc_inos": [ 2021-02-10T11:38:04.635 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:38:04.635 INFO:teuthology.orchestra.run.gibba008.stdout: "start": 1099511627782, 2021-02-10T11:38:04.635 INFO:teuthology.orchestra.run.gibba008.stdout: "length": 995 2021-02-10T11:38:04.635 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:04.636 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:38:04.636 INFO:teuthology.orchestra.run.gibba008.stdout: "used_inos": [], 2021-02-10T11:38:04.636 INFO:teuthology.orchestra.run.gibba008.stdout: "client_metadata": { 2021-02-10T11:38:04.636 INFO:teuthology.orchestra.run.gibba008.stdout: "features": "0x0000000000001fff", 2021-02-10T11:38:04.636 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:38:04.636 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:38:04.636 INFO:teuthology.orchestra.run.gibba008.stdout: "entity_id": "y", 2021-02-10T11:38:04.636 INFO:teuthology.orchestra.run.gibba008.stdout: "hostname": "gibba001", 2021-02-10T11:38:04.636 INFO:teuthology.orchestra.run.gibba008.stdout: "pid": "24165", 2021-02-10T11:38:04.636 INFO:teuthology.orchestra.run.gibba008.stdout: "root": "/" 2021-02-10T11:38:04.636 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:04.636 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:04.636 INFO:teuthology.orchestra.run.gibba008.stdout:] 2021-02-10T11:38:04.637 INFO:tasks.cephfs.filesystem:_json_asok output: [ { "id": 9594, "entity": { "name": { "type": "client", "num": 9594 }, "addr": { "type": "any", "addr": "172.21.2.101:0", "nonce": 1857925112 } }, "state": "open", "num_leases": 0, "num_caps": 7, "request_load_avg": 9, "uptime": 38.120963134, "requests_in_flight": 0, "completed_requests": 1, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 2.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 19.242702177379513, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 10 }, "inst": "client.9594 172.21.2.101:0/1857925112", "completed_requests": [ { "tid": 14, "created_ino": "0x10000000005" } ], "prealloc_inos": [ { "start": 1099511627782, "length": 995 } ], "used_inos": [], "client_metadata": { "features": "0x0000000000001fff", "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", "entity_id": "y", "hostname": "gibba001", "pid": "24165", "root": "/" } } ] 2021-02-10T11:38:04.637 DEBUG:tasks.ceph_test_case:wait_until_true: waiting... 2021-02-10T11:38:09.638 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:38:10.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.045 7f68f79b3700 1 Processor -- start 2021-02-10T11:38:10.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.045 7f68f79b3700 1 -- start start 2021-02-10T11:38:10.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.046 7f68f79b3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68f00d1870 0x7f68f01b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:10.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.046 7f68f79b3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f68f01b0a10 0x7f68f00cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:10.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.046 7f68f79b3700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f00f0b10 con 0x7f68f00d1c70 2021-02-10T11:38:10.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.046 7f68f79b3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f00fe230 con 0x7f68f01b0a10 2021-02-10T11:38:10.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.046 7f68f79b3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f00e5ed0 con 0x7f68f00d1870 2021-02-10T11:38:10.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.046 7f68f4fce700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f68f01b0a10 0x7f68f00cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:10.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.046 7f68f4fce700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f68f01b0a10 0x7f68f00cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54862/0 (socket says 172.21.2.101:54862) 2021-02-10T11:38:10.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.046 7f68f4fce700 1 -- 172.21.2.101:0/2158463510 learned_addr learned my addr 172.21.2.101:0/2158463510 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:10.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.046 7f68f4fce700 1 -- 172.21.2.101:0/2158463510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68f00d1870 msgr2=0x7f68f01b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:38:10.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.046 7f68f4fce700 1 --2- 172.21.2.101:0/2158463510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68f00d1870 0x7f68f01b03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:10.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.046 7f68f4fce700 1 -- 172.21.2.101:0/2158463510 >> v1:172.21.2.101:6789/0 conn(0x7f68f00d1c70 legacy=0x7f68f00cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:10.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.046 7f68f4fce700 1 -- 172.21.2.101:0/2158463510 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f00f0650 con 0x7f68f01b0a10 2021-02-10T11:38:10.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.046 7f68f4fce700 1 --2- 172.21.2.101:0/2158463510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68f01b0a10 0x7f68f00cf330 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 rx=0x7f68e4006700 tx=0x7f68e4009b00).ready entity=mon.0 client_cookie=556700e6653d9454 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:10.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.047 7f68e3fff700 1 -- 172.21.2.101:0/2158463510 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f68e400c0c0 con 0x7f68f01b0a10 2021-02-10T11:38:10.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.047 7f68e3fff700 1 -- 172.21.2.101:0/2158463510 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e400c400 con 0x7f68f01b0a10 2021-02-10T11:38:10.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.047 7f68e3fff700 1 -- 172.21.2.101:0/2158463510 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f68e400c920 con 0x7f68f01b0a10 2021-02-10T11:38:10.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.047 7f68f79b3700 1 -- 172.21.2.101:0/2158463510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68f01b0a10 msgr2=0x7f68f00cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:10.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.047 7f68f79b3700 1 --2- 172.21.2.101:0/2158463510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68f01b0a10 0x7f68f00cf330 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 rx=0x7f68e4006700 tx=0x7f68e4009b00).stop 2021-02-10T11:38:10.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.047 7f68f79b3700 1 -- 172.21.2.101:0/2158463510 shutdown_connections 2021-02-10T11:38:10.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.047 7f68f79b3700 1 --2- 172.21.2.101:0/2158463510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68f01b0a10 0x7f68f00cf330 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:10.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.047 7f68f79b3700 1 --2- 172.21.2.101:0/2158463510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68f00d1870 0x7f68f01b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:10.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.047 7f68f79b3700 1 -- 172.21.2.101:0/2158463510 >> 172.21.2.101:0/2158463510 conn(0x7f68f00ce450 msgr2=0x7f68f00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:10.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.048 7f68f79b3700 1 -- 172.21.2.101:0/2158463510 shutdown_connections 2021-02-10T11:38:10.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.048 7f68f79b3700 1 -- 172.21.2.101:0/2158463510 wait complete. 2021-02-10T11:38:10.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.062 7f68f79b3700 1 Processor -- start 2021-02-10T11:38:10.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.062 7f68f79b3700 1 -- start start 2021-02-10T11:38:10.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.063 7f68f79b3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68f00b0ad0 0x7f68f0091f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:10.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.063 7f68f79b3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68f01a8850 0x7f68f01a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:10.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.063 7f68f79b3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68f01a6fa0 0x7f68f01a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:10.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.063 7f68f79b3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f01abfe0 con 0x7f68f00b0ad0 2021-02-10T11:38:10.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.063 7f68f79b3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f01a9c40 con 0x7f68f01a8850 2021-02-10T11:38:10.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.063 7f68f79b3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f01a9730 con 0x7f68f01a6fa0 2021-02-10T11:38:10.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.063 7f68f57cf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68f00b0ad0 0x7f68f0091f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:10.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.063 7f68f57cf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68f00b0ad0 0x7f68f0091f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54866/0 (socket says 172.21.2.101:54866) 2021-02-10T11:38:10.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.063 7f68f57cf700 1 -- 172.21.2.101:0/174097935 learned_addr learned my addr 172.21.2.101:0/174097935 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:10.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.063 7f68f5fd0700 1 --2- 172.21.2.101:0/174097935 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68f01a6fa0 0x7f68f01a73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:10.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.063 7f68f4fce700 1 --2- 172.21.2.101:0/174097935 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68f01a8850 0x7f68f01a8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:10.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.063 7f68f57cf700 1 -- 172.21.2.101:0/174097935 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68f01a6fa0 msgr2=0x7f68f01a73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:10.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.063 7f68f57cf700 1 --2- 172.21.2.101:0/174097935 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68f01a6fa0 0x7f68f01a73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:10.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.063 7f68f57cf700 1 -- 172.21.2.101:0/174097935 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68f01a8850 msgr2=0x7f68f01a8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:10.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.063 7f68f57cf700 1 --2- 172.21.2.101:0/174097935 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68f01a8850 0x7f68f01a8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:10.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.063 7f68f57cf700 1 -- 172.21.2.101:0/174097935 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f01a9270 con 0x7f68f00b0ad0 2021-02-10T11:38:10.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.063 7f68f5fd0700 1 --2- 172.21.2.101:0/174097935 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68f01a6fa0 0x7f68f01a73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:38:10.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.064 7f68f57cf700 1 --2- 172.21.2.101:0/174097935 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68f00b0ad0 0x7f68f0091f30 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 rx=0x7f68ec00b5c0 tx=0x7f68ec00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:10.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.064 7f68e1ffb700 1 -- 172.21.2.101:0/174097935 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f68ec00bd50 con 0x7f68f00b0ad0 2021-02-10T11:38:10.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.064 7f68e1ffb700 1 -- 172.21.2.101:0/174097935 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec006030 con 0x7f68f00b0ad0 2021-02-10T11:38:10.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.064 7f68e1ffb700 1 -- 172.21.2.101:0/174097935 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f68ec0054d0 con 0x7f68f00b0ad0 2021-02-10T11:38:10.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.064 7f68f79b3700 1 -- 172.21.2.101:0/174097935 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f01a9270 con 0x7f68f00b0ad0 2021-02-10T11:38:10.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.064 7f68f79b3700 1 -- 172.21.2.101:0/174097935 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f01a7cb0 con 0x7f68f00b0ad0 2021-02-10T11:38:10.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.064 7f68e1ffb700 1 -- 172.21.2.101:0/174097935 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f68ec006210 con 0x7f68f00b0ad0 2021-02-10T11:38:10.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.064 7f68e1ffb700 1 --2- 172.21.2.101:0/174097935 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68d00186c0 0x7f68d001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:10.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.064 7f68e1ffb700 1 -- 172.21.2.101:0/174097935 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f68ec032650 con 0x7f68f00b0ad0 2021-02-10T11:38:10.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.064 7f68f4fce700 1 --2- 172.21.2.101:0/174097935 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68d00186c0 0x7f68d001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:10.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.065 7f68f4fce700 1 --2- 172.21.2.101:0/174097935 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68d00186c0 0x7f68d001abc0 crc :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:10.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.096 7f68f79b3700 1 -- 172.21.2.101:0/174097935 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f00ce450 con 0x7f68f00b0ad0 2021-02-10T11:38:10.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.098 7f68e1ffb700 1 -- 172.21.2.101:0/174097935 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f68ec000940 con 0x7f68f00b0ad0 2021-02-10T11:38:10.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.227 7f68f79b3700 1 -- 172.21.2.101:0/174097935 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68f00ce450 con 0x7f68f00b0ad0 2021-02-10T11:38:10.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.228 7f68e1ffb700 1 -- 172.21.2.101:0/174097935 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f68ec00bd50 con 0x7f68f00b0ad0 2021-02-10T11:38:10.233 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 63 2021-02-10T11:38:10.233 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:38:10.233 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9360,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2529021396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2529021396},{"type":"v1","addr":"172.21.2.101:6835","nonce":2529021396}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:56.648753","modified":"2021-02-10 11:36:58.653009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6005":{"gid":6005,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2363068042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2363068042},{"type":"v1","addr":"172.21.2.108:6833","nonce":2363068042}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-02-10T11:38:10.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.229 7f68f79b3700 1 -- 172.21.2.101:0/174097935 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68d00186c0 msgr2=0x7f68d001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:10.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.229 7f68f79b3700 1 --2- 172.21.2.101:0/174097935 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68d00186c0 0x7f68d001abc0 crc :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:10.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.229 7f68f79b3700 1 -- 172.21.2.101:0/174097935 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68f00b0ad0 msgr2=0x7f68f0091f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:10.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.229 7f68f79b3700 1 --2- 172.21.2.101:0/174097935 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68f00b0ad0 0x7f68f0091f30 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 rx=0x7f68ec00b5c0 tx=0x7f68ec00b960).stop 2021-02-10T11:38:10.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.230 7f68f79b3700 1 -- 172.21.2.101:0/174097935 shutdown_connections 2021-02-10T11:38:10.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.230 7f68f79b3700 1 --2- 172.21.2.101:0/174097935 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68d00186c0 0x7f68d001abc0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:10.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.230 7f68f79b3700 1 --2- 172.21.2.101:0/174097935 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68f01a6fa0 0x7f68f01a73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:10.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.230 7f68f79b3700 1 --2- 172.21.2.101:0/174097935 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68f01a8850 0x7f68f01a8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:10.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.230 7f68f79b3700 1 --2- 172.21.2.101:0/174097935 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68f00b0ad0 0x7f68f0091f30 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:10.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.230 7f68f79b3700 1 -- 172.21.2.101:0/174097935 >> 172.21.2.101:0/174097935 conn(0x7f68f019f8f0 msgr2=0x7f68f01a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:10.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.230 7f68f79b3700 1 -- 172.21.2.101:0/174097935 shutdown_connections 2021-02-10T11:38:10.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:10.230 7f68f79b3700 1 -- 172.21.2.101:0/174097935 wait complete. 2021-02-10T11:38:10.243 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok session ls 2021-02-10T11:38:10.641 INFO:teuthology.orchestra.run.gibba008.stdout:[ 2021-02-10T11:38:10.641 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:38:10.642 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 9594, 2021-02-10T11:38:10.642 INFO:teuthology.orchestra.run.gibba008.stdout: "entity": { 2021-02-10T11:38:10.642 INFO:teuthology.orchestra.run.gibba008.stdout: "name": { 2021-02-10T11:38:10.642 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "client", 2021-02-10T11:38:10.642 INFO:teuthology.orchestra.run.gibba008.stdout: "num": 9594 2021-02-10T11:38:10.642 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:10.642 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": { 2021-02-10T11:38:10.642 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "any", 2021-02-10T11:38:10.642 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:38:10.642 INFO:teuthology.orchestra.run.gibba008.stdout: "nonce": 1857925112 2021-02-10T11:38:10.642 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:10.643 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:10.643 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "open", 2021-02-10T11:38:10.643 INFO:teuthology.orchestra.run.gibba008.stdout: "num_leases": 0, 2021-02-10T11:38:10.643 INFO:teuthology.orchestra.run.gibba008.stdout: "num_caps": 7, 2021-02-10T11:38:10.643 INFO:teuthology.orchestra.run.gibba008.stdout: "request_load_avg": 8, 2021-02-10T11:38:10.643 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 44.131659689999999, 2021-02-10T11:38:10.643 INFO:teuthology.orchestra.run.gibba008.stdout: "requests_in_flight": 0, 2021-02-10T11:38:10.643 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": 1, 2021-02-10T11:38:10.643 INFO:teuthology.orchestra.run.gibba008.stdout: "reconnecting": false, 2021-02-10T11:38:10.643 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps": { 2021-02-10T11:38:10.643 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:10.644 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:38:10.644 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:10.644 INFO:teuthology.orchestra.run.gibba008.stdout: "release_caps": { 2021-02-10T11:38:10.644 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:10.644 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:38:10.644 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:10.644 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle": { 2021-02-10T11:38:10.644 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:10.644 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 2.5 2021-02-10T11:38:10.644 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:10.644 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle2o": { 2021-02-10T11:38:10.644 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:10.645 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 0.5 2021-02-10T11:38:10.645 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:10.645 INFO:teuthology.orchestra.run.gibba008.stdout: "session_cache_liveness": { 2021-02-10T11:38:10.645 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 18.977313193499828, 2021-02-10T11:38:10.645 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 300 2021-02-10T11:38:10.645 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:10.645 INFO:teuthology.orchestra.run.gibba008.stdout: "cap_acquisition": { 2021-02-10T11:38:10.645 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:10.645 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 10 2021-02-10T11:38:10.645 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:10.645 INFO:teuthology.orchestra.run.gibba008.stdout: "inst": "client.9594 172.21.2.101:0/1857925112", 2021-02-10T11:38:10.646 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": [ 2021-02-10T11:38:10.646 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:38:10.646 INFO:teuthology.orchestra.run.gibba008.stdout: "tid": 14, 2021-02-10T11:38:10.646 INFO:teuthology.orchestra.run.gibba008.stdout: "created_ino": "0x10000000005" 2021-02-10T11:38:10.646 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:10.646 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:38:10.646 INFO:teuthology.orchestra.run.gibba008.stdout: "prealloc_inos": [ 2021-02-10T11:38:10.646 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:38:10.646 INFO:teuthology.orchestra.run.gibba008.stdout: "start": 1099511627782, 2021-02-10T11:38:10.646 INFO:teuthology.orchestra.run.gibba008.stdout: "length": 995 2021-02-10T11:38:10.646 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:10.647 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:38:10.647 INFO:teuthology.orchestra.run.gibba008.stdout: "used_inos": [], 2021-02-10T11:38:10.647 INFO:teuthology.orchestra.run.gibba008.stdout: "client_metadata": { 2021-02-10T11:38:10.647 INFO:teuthology.orchestra.run.gibba008.stdout: "features": "0x0000000000001fff", 2021-02-10T11:38:10.647 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:38:10.647 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:38:10.647 INFO:teuthology.orchestra.run.gibba008.stdout: "entity_id": "y", 2021-02-10T11:38:10.647 INFO:teuthology.orchestra.run.gibba008.stdout: "hostname": "gibba001", 2021-02-10T11:38:10.647 INFO:teuthology.orchestra.run.gibba008.stdout: "pid": "24165", 2021-02-10T11:38:10.647 INFO:teuthology.orchestra.run.gibba008.stdout: "root": "/" 2021-02-10T11:38:10.647 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:10.647 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:10.647 INFO:teuthology.orchestra.run.gibba008.stdout:] 2021-02-10T11:38:10.648 INFO:tasks.cephfs.filesystem:_json_asok output: [ { "id": 9594, "entity": { "name": { "type": "client", "num": 9594 }, "addr": { "type": "any", "addr": "172.21.2.101:0", "nonce": 1857925112 } }, "state": "open", "num_leases": 0, "num_caps": 7, "request_load_avg": 8, "uptime": 44.131659689999999, "requests_in_flight": 0, "completed_requests": 1, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 2.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 18.977313193499828, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 10 }, "inst": "client.9594 172.21.2.101:0/1857925112", "completed_requests": [ { "tid": 14, "created_ino": "0x10000000005" } ], "prealloc_inos": [ { "start": 1099511627782, "length": 995 } ], "used_inos": [], "client_metadata": { "features": "0x0000000000001fff", "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", "entity_id": "y", "hostname": "gibba001", "pid": "24165", "root": "/" } } ] 2021-02-10T11:38:10.649 DEBUG:tasks.ceph_test_case:wait_until_true: waiting... 2021-02-10T11:38:15.649 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:38:16.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.056 7fd33e80a700 1 Processor -- start 2021-02-10T11:38:16.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.056 7fd33e80a700 1 -- start start 2021-02-10T11:38:16.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.056 7fd33e80a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd3380d1870 0x7fd3381b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:16.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.056 7fd33e80a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3380f0b10 con 0x7fd3380f0650 2021-02-10T11:38:16.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.056 7fd33e80a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3380d0310 con 0x7fd3381b0a10 2021-02-10T11:38:16.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.056 7fd33e80a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3380d0530 con 0x7fd3380d1870 2021-02-10T11:38:16.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.056 7fd33ce27700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd3380f0650 0x7fd3380cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55368/0 (socket says 172.21.2.101:55368) 2021-02-10T11:38:16.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.056 7fd33ce27700 1 -- 172.21.2.101:0/783646346 learned_addr learned my addr 172.21.2.101:0/783646346 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:38:16.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.057 7fd336ffd700 1 -- 172.21.2.101:0/783646346 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2008263832 0 0) 0x7fd330001500 con 0x7fd3380f0650 2021-02-10T11:38:16.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.057 7fd336ffd700 1 -- 172.21.2.101:0/783646346 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd320002fe0 con 0x7fd3380f0650 2021-02-10T11:38:16.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.057 7fd337fff700 1 --2- 172.21.2.101:0/783646346 >> v2:172.21.2.108:3300/0 conn(0x7fd3380d1870 0x7fd3381b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:16.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.057 7fd336ffd700 1 -- 172.21.2.101:0/783646346 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3952499652 0 0) 0x7fd330001460 con 0x7fd3380f0650 2021-02-10T11:38:16.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.057 7fd336ffd700 1 -- 172.21.2.101:0/783646346 >> v2:172.21.2.108:3300/0 conn(0x7fd3380d1870 msgr2=0x7fd3381b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:16.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.057 7fd336ffd700 1 --2- 172.21.2.101:0/783646346 >> v2:172.21.2.108:3300/0 conn(0x7fd3380d1870 0x7fd3381b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:16.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.057 7fd336ffd700 1 -- 172.21.2.101:0/783646346 >> v1:172.21.2.108:6789/0 conn(0x7fd3381b0a10 legacy=0x7fd3380cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:16.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.057 7fd336ffd700 1 -- 172.21.2.101:0/783646346 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3380d1c70 con 0x7fd3380f0650 2021-02-10T11:38:16.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.057 7fd336ffd700 1 -- 172.21.2.101:0/783646346 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fd330001980 con 0x7fd3380f0650 2021-02-10T11:38:16.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.057 7fd336ffd700 1 -- 172.21.2.101:0/783646346 >> v1:172.21.2.101:6789/0 conn(0x7fd3380f0650 legacy=0x7fd3380cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:16.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.057 7fd336ffd700 1 --2- 172.21.2.101:0/783646346 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd320003aa0 0x7fd320005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:16.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.057 7fd336ffd700 1 --2- 172.21.2.101:0/783646346 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd320006780 0x7fd320008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:16.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.057 7fd336ffd700 1 --2- 172.21.2.101:0/783646346 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd320009580 0x7fd32000baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:16.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.057 7fd336ffd700 1 -- 172.21.2.101:0/783646346 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32000c2f0 con 0x7fd320006780 2021-02-10T11:38:16.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.057 7fd336ffd700 1 -- 172.21.2.101:0/783646346 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32000c510 con 0x7fd320009580 2021-02-10T11:38:16.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.057 7fd336ffd700 1 -- 172.21.2.101:0/783646346 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32000c730 con 0x7fd320003aa0 2021-02-10T11:38:16.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.057 7fd3377fe700 1 --2- 172.21.2.101:0/783646346 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd320006780 0x7fd320008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:16.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.058 7fd33ce27700 1 --2- 172.21.2.101:0/783646346 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd320009580 0x7fd32000baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:16.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.058 7fd3377fe700 1 -- 172.21.2.101:0/783646346 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd320003aa0 msgr2=0x7fd320005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:16.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.058 7fd337fff700 1 --2- 172.21.2.101:0/783646346 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd320003aa0 0x7fd320005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:16.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.058 7fd3377fe700 1 --2- 172.21.2.101:0/783646346 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd320003aa0 0x7fd320005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:16.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.058 7fd3377fe700 1 -- 172.21.2.101:0/783646346 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd320009580 msgr2=0x7fd32000baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:16.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.058 7fd3377fe700 1 --2- 172.21.2.101:0/783646346 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd320009580 0x7fd32000baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:16.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.058 7fd3377fe700 1 -- 172.21.2.101:0/783646346 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32000c9d0 con 0x7fd320006780 2021-02-10T11:38:16.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.058 7fd33ce27700 1 --2- 172.21.2.101:0/783646346 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd320009580 0x7fd32000baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:38:16.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.058 7fd33ce27700 1 -- 172.21.2.101:0/783646346 reap_dead start 2021-02-10T11:38:16.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.058 7fd337fff700 1 --2- 172.21.2.101:0/783646346 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd320003aa0 0x7fd320005f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:38:16.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.058 7fd3377fe700 1 --2- 172.21.2.101:0/783646346 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd320006780 0x7fd320008c80 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 rx=0x7fd32c04d750 tx=0x7fd32c04daf0).ready entity=mon.0 client_cookie=e096fa4b2dbc7330 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:16.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.058 7fd336ffd700 1 -- 172.21.2.101:0/783646346 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd32c0066c0 con 0x7fd320006780 2021-02-10T11:38:16.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.058 7fd336ffd700 1 -- 172.21.2.101:0/783646346 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32c000990 con 0x7fd320006780 2021-02-10T11:38:16.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.058 7fd336ffd700 1 -- 172.21.2.101:0/783646346 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd32c0062a0 con 0x7fd320006780 2021-02-10T11:38:16.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.058 7fd33e80a700 1 -- 172.21.2.101:0/783646346 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd320006780 msgr2=0x7fd320008c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:16.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.058 7fd33e80a700 1 --2- 172.21.2.101:0/783646346 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd320006780 0x7fd320008c80 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 rx=0x7fd32c04d750 tx=0x7fd32c04daf0).stop 2021-02-10T11:38:16.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.059 7fd33e80a700 1 -- 172.21.2.101:0/783646346 shutdown_connections 2021-02-10T11:38:16.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.059 7fd33e80a700 1 --2- 172.21.2.101:0/783646346 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd320006780 0x7fd320008c80 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:16.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.059 7fd33e80a700 1 -- 172.21.2.101:0/783646346 >> 172.21.2.101:0/783646346 conn(0x7fd3380ce450 msgr2=0x7fd3380d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:16.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.059 7fd33e80a700 1 -- 172.21.2.101:0/783646346 shutdown_connections 2021-02-10T11:38:16.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.059 7fd33e80a700 1 -- 172.21.2.101:0/783646346 wait complete. 2021-02-10T11:38:16.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.073 7fd33e80a700 1 Processor -- start 2021-02-10T11:38:16.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.073 7fd33e80a700 1 -- start start 2021-02-10T11:38:16.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.074 7fd33e80a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3381942b0 0x7fd3380b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:16.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.074 7fd33e80a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd338195070 0x7fd338195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:16.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.074 7fd33e80a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3380bb8e0 0x7fd3380bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:16.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.074 7fd33e80a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3381946b0 con 0x7fd3380bb8e0 2021-02-10T11:38:16.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.074 7fd33e80a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3380b3b70 con 0x7fd338195070 2021-02-10T11:38:16.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.074 7fd33e80a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3380b3d90 con 0x7fd3381942b0 2021-02-10T11:38:16.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.074 7fd33ce27700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3380bb8e0 0x7fd3380bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:16.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.074 7fd33ce27700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3380bb8e0 0x7fd3380bbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54888/0 (socket says 172.21.2.101:54888) 2021-02-10T11:38:16.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.074 7fd33ce27700 1 -- 172.21.2.101:0/1438722749 learned_addr learned my addr 172.21.2.101:0/1438722749 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:16.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.074 7fd337fff700 1 --2- 172.21.2.101:0/1438722749 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3381942b0 0x7fd3380b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:16.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.074 7fd3377fe700 1 --2- 172.21.2.101:0/1438722749 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd338195070 0x7fd338195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:16.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.074 7fd33ce27700 1 -- 172.21.2.101:0/1438722749 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3381942b0 msgr2=0x7fd3380b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.074 7fd33ce27700 1 --2- 172.21.2.101:0/1438722749 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3381942b0 0x7fd3380b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.074 7fd33ce27700 1 -- 172.21.2.101:0/1438722749 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd338195070 msgr2=0x7fd338195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.074 7fd33ce27700 1 --2- 172.21.2.101:0/1438722749 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd338195070 0x7fd338195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.074 7fd33ce27700 1 -- 172.21.2.101:0/1438722749 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3381a49e0 con 0x7fd3380bb8e0 2021-02-10T11:38:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.074 7fd3377fe700 1 --2- 172.21.2.101:0/1438722749 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd338195070 0x7fd338195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:38:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.075 7fd33ce27700 1 --2- 172.21.2.101:0/1438722749 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3380bb8e0 0x7fd3380bbce0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 rx=0x7fd33000a8e0 tx=0x7fd33000ac80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.075 7fd334ff9700 1 -- 172.21.2.101:0/1438722749 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd330007020 con 0x7fd3380bb8e0 2021-02-10T11:38:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.075 7fd334ff9700 1 -- 172.21.2.101:0/1438722749 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3300073a0 con 0x7fd3380bb8e0 2021-02-10T11:38:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.075 7fd334ff9700 1 -- 172.21.2.101:0/1438722749 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd3300049a0 con 0x7fd3380bb8e0 2021-02-10T11:38:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.075 7fd33e80a700 1 -- 172.21.2.101:0/1438722749 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3381a49e0 con 0x7fd3380bb8e0 2021-02-10T11:38:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.075 7fd33e80a700 1 -- 172.21.2.101:0/1438722749 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3380bc5f0 con 0x7fd3380bb8e0 2021-02-10T11:38:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.076 7fd334ff9700 1 -- 172.21.2.101:0/1438722749 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd33000adc0 con 0x7fd3380bb8e0 2021-02-10T11:38:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.076 7fd334ff9700 1 --2- 172.21.2.101:0/1438722749 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3180186c0 0x7fd31801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.076 7fd334ff9700 1 -- 172.21.2.101:0/1438722749 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd3300328b0 con 0x7fd3380bb8e0 2021-02-10T11:38:16.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.076 7fd337fff700 1 --2- 172.21.2.101:0/1438722749 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3180186c0 0x7fd31801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:16.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.076 7fd337fff700 1 --2- 172.21.2.101:0/1438722749 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3180186c0 0x7fd31801abc0 crc :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:16.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.106 7fd33e80a700 1 -- 172.21.2.101:0/1438722749 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3380bc5f0 con 0x7fd3380bb8e0 2021-02-10T11:38:16.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.109 7fd334ff9700 1 -- 172.21.2.101:0/1438722749 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd330001270 con 0x7fd3380bb8e0 2021-02-10T11:38:16.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.235 7fd33e80a700 1 -- 172.21.2.101:0/1438722749 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3380bc5f0 con 0x7fd3380bb8e0 2021-02-10T11:38:16.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.235 7fd334ff9700 1 -- 172.21.2.101:0/1438722749 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+2101 (secure 0 0 0) 0x7fd33000adc0 con 0x7fd3380bb8e0 2021-02-10T11:38:16.241 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:38:16.241 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9360,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2529021396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2529021396},{"type":"v1","addr":"172.21.2.101:6835","nonce":2529021396}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:56.648753","modified":"2021-02-10 11:36:58.653009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6005":{"gid":6005,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2363068042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2363068042},{"type":"v1","addr":"172.21.2.108:6833","nonce":2363068042}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-02-10T11:38:16.241 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 63 2021-02-10T11:38:16.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.237 7fd33e80a700 1 -- 172.21.2.101:0/1438722749 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3180186c0 msgr2=0x7fd31801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:16.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.237 7fd33e80a700 1 --2- 172.21.2.101:0/1438722749 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3180186c0 0x7fd31801abc0 crc :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:16.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.237 7fd33e80a700 1 -- 172.21.2.101:0/1438722749 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3380bb8e0 msgr2=0x7fd3380bbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:16.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.237 7fd33e80a700 1 --2- 172.21.2.101:0/1438722749 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3380bb8e0 0x7fd3380bbce0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 rx=0x7fd33000a8e0 tx=0x7fd33000ac80).stop 2021-02-10T11:38:16.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.238 7fd33e80a700 1 -- 172.21.2.101:0/1438722749 shutdown_connections 2021-02-10T11:38:16.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.238 7fd33e80a700 1 --2- 172.21.2.101:0/1438722749 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3380bb8e0 0x7fd3380bbce0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:16.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.238 7fd33e80a700 1 --2- 172.21.2.101:0/1438722749 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd338195070 0x7fd338195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:16.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.238 7fd33e80a700 1 --2- 172.21.2.101:0/1438722749 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3180186c0 0x7fd31801abc0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:16.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.238 7fd33e80a700 1 --2- 172.21.2.101:0/1438722749 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3381942b0 0x7fd3380b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:16.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.238 7fd33e80a700 1 -- 172.21.2.101:0/1438722749 >> 172.21.2.101:0/1438722749 conn(0x7fd3380b3040 msgr2=0x7fd3380c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:16.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.238 7fd33e80a700 1 -- 172.21.2.101:0/1438722749 shutdown_connections 2021-02-10T11:38:16.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:16.238 7fd33e80a700 1 -- 172.21.2.101:0/1438722749 wait complete. 2021-02-10T11:38:16.250 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok session ls 2021-02-10T11:38:16.647 INFO:teuthology.orchestra.run.gibba008.stdout:[ 2021-02-10T11:38:16.647 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:38:16.647 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 9594, 2021-02-10T11:38:16.647 INFO:teuthology.orchestra.run.gibba008.stdout: "entity": { 2021-02-10T11:38:16.647 INFO:teuthology.orchestra.run.gibba008.stdout: "name": { 2021-02-10T11:38:16.648 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "client", 2021-02-10T11:38:16.648 INFO:teuthology.orchestra.run.gibba008.stdout: "num": 9594 2021-02-10T11:38:16.648 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:16.648 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": { 2021-02-10T11:38:16.648 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "any", 2021-02-10T11:38:16.648 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:38:16.648 INFO:teuthology.orchestra.run.gibba008.stdout: "nonce": 1857925112 2021-02-10T11:38:16.648 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:16.648 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:16.648 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "open", 2021-02-10T11:38:16.648 INFO:teuthology.orchestra.run.gibba008.stdout: "num_leases": 0, 2021-02-10T11:38:16.648 INFO:teuthology.orchestra.run.gibba008.stdout: "num_caps": 7, 2021-02-10T11:38:16.649 INFO:teuthology.orchestra.run.gibba008.stdout: "request_load_avg": 7, 2021-02-10T11:38:16.649 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 50.136361061000002, 2021-02-10T11:38:16.649 INFO:teuthology.orchestra.run.gibba008.stdout: "requests_in_flight": 0, 2021-02-10T11:38:16.649 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": 1, 2021-02-10T11:38:16.649 INFO:teuthology.orchestra.run.gibba008.stdout: "reconnecting": false, 2021-02-10T11:38:16.649 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps": { 2021-02-10T11:38:16.649 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:16.649 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:38:16.649 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:16.649 INFO:teuthology.orchestra.run.gibba008.stdout: "release_caps": { 2021-02-10T11:38:16.649 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:16.649 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:38:16.649 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:16.650 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle": { 2021-02-10T11:38:16.650 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:16.650 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 2.5 2021-02-10T11:38:16.650 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:16.650 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle2o": { 2021-02-10T11:38:16.650 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:16.650 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 0.5 2021-02-10T11:38:16.650 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:16.650 INFO:teuthology.orchestra.run.gibba008.stdout: "session_cache_liveness": { 2021-02-10T11:38:16.650 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 18.715843613383175, 2021-02-10T11:38:16.650 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 300 2021-02-10T11:38:16.651 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:16.651 INFO:teuthology.orchestra.run.gibba008.stdout: "cap_acquisition": { 2021-02-10T11:38:16.651 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:16.651 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 10 2021-02-10T11:38:16.651 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:16.651 INFO:teuthology.orchestra.run.gibba008.stdout: "inst": "client.9594 172.21.2.101:0/1857925112", 2021-02-10T11:38:16.651 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": [ 2021-02-10T11:38:16.651 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:38:16.651 INFO:teuthology.orchestra.run.gibba008.stdout: "tid": 14, 2021-02-10T11:38:16.651 INFO:teuthology.orchestra.run.gibba008.stdout: "created_ino": "0x10000000005" 2021-02-10T11:38:16.651 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:16.651 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:38:16.651 INFO:teuthology.orchestra.run.gibba008.stdout: "prealloc_inos": [ 2021-02-10T11:38:16.651 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:38:16.652 INFO:teuthology.orchestra.run.gibba008.stdout: "start": 1099511627782, 2021-02-10T11:38:16.652 INFO:teuthology.orchestra.run.gibba008.stdout: "length": 995 2021-02-10T11:38:16.652 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:16.652 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:38:16.652 INFO:teuthology.orchestra.run.gibba008.stdout: "used_inos": [], 2021-02-10T11:38:16.652 INFO:teuthology.orchestra.run.gibba008.stdout: "client_metadata": { 2021-02-10T11:38:16.652 INFO:teuthology.orchestra.run.gibba008.stdout: "features": "0x0000000000001fff", 2021-02-10T11:38:16.652 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:38:16.652 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:38:16.652 INFO:teuthology.orchestra.run.gibba008.stdout: "entity_id": "y", 2021-02-10T11:38:16.652 INFO:teuthology.orchestra.run.gibba008.stdout: "hostname": "gibba001", 2021-02-10T11:38:16.652 INFO:teuthology.orchestra.run.gibba008.stdout: "pid": "24165", 2021-02-10T11:38:16.652 INFO:teuthology.orchestra.run.gibba008.stdout: "root": "/" 2021-02-10T11:38:16.652 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:16.652 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:16.653 INFO:teuthology.orchestra.run.gibba008.stdout:] 2021-02-10T11:38:16.653 INFO:tasks.cephfs.filesystem:_json_asok output: [ { "id": 9594, "entity": { "name": { "type": "client", "num": 9594 }, "addr": { "type": "any", "addr": "172.21.2.101:0", "nonce": 1857925112 } }, "state": "open", "num_leases": 0, "num_caps": 7, "request_load_avg": 7, "uptime": 50.136361061000002, "requests_in_flight": 0, "completed_requests": 1, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 2.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 18.715843613383175, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 10 }, "inst": "client.9594 172.21.2.101:0/1857925112", "completed_requests": [ { "tid": 14, "created_ino": "0x10000000005" } ], "prealloc_inos": [ { "start": 1099511627782, "length": 995 } ], "used_inos": [], "client_metadata": { "features": "0x0000000000001fff", "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", "entity_id": "y", "hostname": "gibba001", "pid": "24165", "root": "/" } } ] 2021-02-10T11:38:16.654 DEBUG:tasks.ceph_test_case:wait_until_true: waiting... 2021-02-10T11:38:21.654 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:38:22.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.060 7f68bb24f700 1 Processor -- start 2021-02-10T11:38:22.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.060 7f68bb24f700 1 -- start start 2021-02-10T11:38:22.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.061 7f68bb24f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f68b40d1870 0x7f68b41b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:22.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.061 7f68bb24f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68b41b0a10 0x7f68b40cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:22.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.061 7f68bb24f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b40f0b10 con 0x7f68b40d1c70 2021-02-10T11:38:22.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.061 7f68bb24f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b40e5ed0 con 0x7f68b40d1870 2021-02-10T11:38:22.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.061 7f68bb24f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40f0650 con 0x7f68b41b0a10 2021-02-10T11:38:22.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.062 7f68b886a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68b41b0a10 0x7f68b40cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:22.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.062 7f68b906b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f68b40d1870 0x7f68b41b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:22.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.062 7f68b886a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68b41b0a10 0x7f68b40cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:47624/0 (socket says 172.21.2.101:47624) 2021-02-10T11:38:22.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.062 7f68b886a700 1 -- 172.21.2.101:0/2507025511 learned_addr learned my addr 172.21.2.101:0/2507025511 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:22.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.062 7f68b906b700 1 -- 172.21.2.101:0/2507025511 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68b41b0a10 msgr2=0x7f68b40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:22.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.062 7f68b906b700 1 --2- 172.21.2.101:0/2507025511 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68b41b0a10 0x7f68b40cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:22.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.062 7f68b906b700 1 -- 172.21.2.101:0/2507025511 >> v1:172.21.2.108:6789/0 conn(0x7f68b40d1c70 legacy=0x7f68b40cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:22.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.062 7f68b906b700 1 -- 172.21.2.101:0/2507025511 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b40f0870 con 0x7f68b40d1870 2021-02-10T11:38:22.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.062 7f68b886a700 1 --2- 172.21.2.101:0/2507025511 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68b41b0a10 0x7f68b40cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:38:22.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.063 7f68b906b700 1 --2- 172.21.2.101:0/2507025511 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68b40d1870 0x7f68b41b03f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f68a8006650 tx=0x7f68a8009a80).ready entity=mon.1 client_cookie=f2750120f56a0160 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:22.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.063 7f68a7fff700 1 -- 172.21.2.101:0/2507025511 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f68a800c0c0 con 0x7f68b40d1870 2021-02-10T11:38:22.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.063 7f68a7fff700 1 -- 172.21.2.101:0/2507025511 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a800c340 con 0x7f68b40d1870 2021-02-10T11:38:22.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.063 7f68a7fff700 1 -- 172.21.2.101:0/2507025511 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f68a800c860 con 0x7f68b40d1870 2021-02-10T11:38:22.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.063 7f68bb24f700 1 -- 172.21.2.101:0/2507025511 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68b40d1870 msgr2=0x7f68b41b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:22.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.063 7f68bb24f700 1 --2- 172.21.2.101:0/2507025511 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68b40d1870 0x7f68b41b03f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f68a8006650 tx=0x7f68a8009a80).stop 2021-02-10T11:38:22.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.064 7f68bb24f700 1 -- 172.21.2.101:0/2507025511 shutdown_connections 2021-02-10T11:38:22.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.064 7f68bb24f700 1 --2- 172.21.2.101:0/2507025511 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68b41b0a10 0x7f68b40cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:22.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.064 7f68bb24f700 1 --2- 172.21.2.101:0/2507025511 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68b40d1870 0x7f68b41b03f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:22.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.064 7f68bb24f700 1 -- 172.21.2.101:0/2507025511 >> 172.21.2.101:0/2507025511 conn(0x7f68b40ce450 msgr2=0x7f68b40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:22.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.064 7f68bb24f700 1 -- 172.21.2.101:0/2507025511 shutdown_connections 2021-02-10T11:38:22.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.064 7f68bb24f700 1 -- 172.21.2.101:0/2507025511 wait complete. 2021-02-10T11:38:22.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.079 7f68bb24f700 1 Processor -- start 2021-02-10T11:38:22.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.079 7f68bb24f700 1 -- start start 2021-02-10T11:38:22.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.079 7f68bb24f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68b40ada90 0x7f68b40ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:22.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.079 7f68bb24f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68b4099d20 0x7f68b409a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:22.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.079 7f68bb24f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68b409a740 0x7f68b4097bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:22.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.079 7f68bb24f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40997a0 con 0x7f68b409a740 2021-02-10T11:38:22.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.079 7f68bb24f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b418f0c0 con 0x7f68b40ada90 2021-02-10T11:38:22.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.079 7f68bb24f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b409ab40 con 0x7f68b4099d20 2021-02-10T11:38:22.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.079 7f68b986c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68b409a740 0x7f68b4097bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:22.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.079 7f68b986c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68b409a740 0x7f68b4097bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54898/0 (socket says 172.21.2.101:54898) 2021-02-10T11:38:22.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.079 7f68b986c700 1 -- 172.21.2.101:0/2769458234 learned_addr learned my addr 172.21.2.101:0/2769458234 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:22.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.080 7f68b886a700 1 --2- 172.21.2.101:0/2769458234 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68b4099d20 0x7f68b409a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:22.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.080 7f68b906b700 1 --2- 172.21.2.101:0/2769458234 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68b40ada90 0x7f68b40ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:22.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.080 7f68b986c700 1 -- 172.21.2.101:0/2769458234 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68b4099d20 msgr2=0x7f68b409a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:22.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.080 7f68b986c700 1 --2- 172.21.2.101:0/2769458234 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68b4099d20 0x7f68b409a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:22.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.080 7f68b986c700 1 -- 172.21.2.101:0/2769458234 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68b40ada90 msgr2=0x7f68b40ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:22.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.080 7f68b986c700 1 --2- 172.21.2.101:0/2769458234 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68b40ada90 0x7f68b40ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:22.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.080 7f68b986c700 1 -- 172.21.2.101:0/2769458234 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b409ad60 con 0x7f68b409a740 2021-02-10T11:38:22.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.080 7f68b886a700 1 --2- 172.21.2.101:0/2769458234 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68b4099d20 0x7f68b409a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:38:22.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.080 7f68b906b700 1 --2- 172.21.2.101:0/2769458234 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68b40ada90 0x7f68b40ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:38:22.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.080 7f68b986c700 1 --2- 172.21.2.101:0/2769458234 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68b409a740 0x7f68b4097bc0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 rx=0x7f68ac009940 tx=0x7f68ac00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:22.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.080 7f68a5ffb700 1 -- 172.21.2.101:0/2769458234 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f68ac00c2f0 con 0x7f68b409a740 2021-02-10T11:38:22.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.080 7f68a5ffb700 1 -- 172.21.2.101:0/2769458234 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ac00c570 con 0x7f68b409a740 2021-02-10T11:38:22.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.080 7f68a5ffb700 1 -- 172.21.2.101:0/2769458234 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f68ac00c9d0 con 0x7f68b409a740 2021-02-10T11:38:22.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.081 7f68bb24f700 1 -- 172.21.2.101:0/2769458234 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b409ad60 con 0x7f68b409a740 2021-02-10T11:38:22.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.081 7f68bb24f700 1 -- 172.21.2.101:0/2769458234 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b40984d0 con 0x7f68b409a740 2021-02-10T11:38:22.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.081 7f68a5ffb700 1 -- 172.21.2.101:0/2769458234 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f68ac00c750 con 0x7f68b409a740 2021-02-10T11:38:22.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.081 7f68a5ffb700 1 --2- 172.21.2.101:0/2769458234 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68940186c0 0x7f689401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:22.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.081 7f68a5ffb700 1 -- 172.21.2.101:0/2769458234 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f68ac032920 con 0x7f68b409a740 2021-02-10T11:38:22.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.081 7f68b906b700 1 --2- 172.21.2.101:0/2769458234 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68940186c0 0x7f689401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:22.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.082 7f68b906b700 1 --2- 172.21.2.101:0/2769458234 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68940186c0 0x7f689401abc0 crc :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:22.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.111 7f68bb24f700 1 -- 172.21.2.101:0/2769458234 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b418f070 con 0x7f68b409a740 2021-02-10T11:38:22.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.113 7f68a5ffb700 1 -- 172.21.2.101:0/2769458234 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f68ac025ef0 con 0x7f68b409a740 2021-02-10T11:38:22.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.243 7f68bb24f700 1 -- 172.21.2.101:0/2769458234 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68b418f070 con 0x7f68b409a740 2021-02-10T11:38:22.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.244 7f68a5ffb700 1 -- 172.21.2.101:0/2769458234 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f68ac025ef0 con 0x7f68b409a740 2021-02-10T11:38:22.249 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 63 2021-02-10T11:38:22.249 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:38:22.249 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9360,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2529021396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2529021396},{"type":"v1","addr":"172.21.2.101:6835","nonce":2529021396}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:56.648753","modified":"2021-02-10 11:36:58.653009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6005":{"gid":6005,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2363068042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2363068042},{"type":"v1","addr":"172.21.2.108:6833","nonce":2363068042}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-02-10T11:38:22.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.246 7f68bb24f700 1 -- 172.21.2.101:0/2769458234 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68940186c0 msgr2=0x7f689401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:22.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.246 7f68bb24f700 1 --2- 172.21.2.101:0/2769458234 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68940186c0 0x7f689401abc0 crc :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:22.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.246 7f68bb24f700 1 -- 172.21.2.101:0/2769458234 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68b409a740 msgr2=0x7f68b4097bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:22.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.246 7f68bb24f700 1 --2- 172.21.2.101:0/2769458234 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68b409a740 0x7f68b4097bc0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 rx=0x7f68ac009940 tx=0x7f68ac00bcf0).stop 2021-02-10T11:38:22.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.246 7f68bb24f700 1 -- 172.21.2.101:0/2769458234 shutdown_connections 2021-02-10T11:38:22.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.246 7f68bb24f700 1 --2- 172.21.2.101:0/2769458234 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f68940186c0 0x7f689401abc0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:22.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.246 7f68bb24f700 1 --2- 172.21.2.101:0/2769458234 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f68b4099d20 0x7f68b409a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:22.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.246 7f68bb24f700 1 --2- 172.21.2.101:0/2769458234 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f68b409a740 0x7f68b4097bc0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:22.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.246 7f68bb24f700 1 --2- 172.21.2.101:0/2769458234 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f68b40ada90 0x7f68b40ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:22.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.246 7f68bb24f700 1 -- 172.21.2.101:0/2769458234 >> 172.21.2.101:0/2769458234 conn(0x7f68b40f0550 msgr2=0x7f68b4191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:22.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.246 7f68bb24f700 1 -- 172.21.2.101:0/2769458234 shutdown_connections 2021-02-10T11:38:22.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:22.246 7f68bb24f700 1 -- 172.21.2.101:0/2769458234 wait complete. 2021-02-10T11:38:22.259 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok session ls 2021-02-10T11:38:22.655 INFO:teuthology.orchestra.run.gibba008.stdout:[ 2021-02-10T11:38:22.656 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:38:22.656 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 9594, 2021-02-10T11:38:22.656 INFO:teuthology.orchestra.run.gibba008.stdout: "entity": { 2021-02-10T11:38:22.656 INFO:teuthology.orchestra.run.gibba008.stdout: "name": { 2021-02-10T11:38:22.656 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "client", 2021-02-10T11:38:22.656 INFO:teuthology.orchestra.run.gibba008.stdout: "num": 9594 2021-02-10T11:38:22.656 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:22.656 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": { 2021-02-10T11:38:22.656 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "any", 2021-02-10T11:38:22.656 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:38:22.656 INFO:teuthology.orchestra.run.gibba008.stdout: "nonce": 1857925112 2021-02-10T11:38:22.657 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:22.657 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:22.657 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "open", 2021-02-10T11:38:22.657 INFO:teuthology.orchestra.run.gibba008.stdout: "num_leases": 0, 2021-02-10T11:38:22.657 INFO:teuthology.orchestra.run.gibba008.stdout: "num_caps": 7, 2021-02-10T11:38:22.657 INFO:teuthology.orchestra.run.gibba008.stdout: "request_load_avg": 7, 2021-02-10T11:38:22.657 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 56.145066641, 2021-02-10T11:38:22.657 INFO:teuthology.orchestra.run.gibba008.stdout: "requests_in_flight": 0, 2021-02-10T11:38:22.657 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": 1, 2021-02-10T11:38:22.657 INFO:teuthology.orchestra.run.gibba008.stdout: "reconnecting": false, 2021-02-10T11:38:22.657 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps": { 2021-02-10T11:38:22.657 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:22.657 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:38:22.658 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:22.658 INFO:teuthology.orchestra.run.gibba008.stdout: "release_caps": { 2021-02-10T11:38:22.658 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:22.658 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:38:22.658 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:22.658 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle": { 2021-02-10T11:38:22.658 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:22.658 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 2.5 2021-02-10T11:38:22.658 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:22.658 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle2o": { 2021-02-10T11:38:22.658 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:22.658 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 0.5 2021-02-10T11:38:22.658 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:22.659 INFO:teuthology.orchestra.run.gibba008.stdout: "session_cache_liveness": { 2021-02-10T11:38:22.659 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 18.457805796694476, 2021-02-10T11:38:22.659 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 300 2021-02-10T11:38:22.659 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:22.659 INFO:teuthology.orchestra.run.gibba008.stdout: "cap_acquisition": { 2021-02-10T11:38:22.659 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:22.659 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 10 2021-02-10T11:38:22.659 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:22.659 INFO:teuthology.orchestra.run.gibba008.stdout: "inst": "client.9594 172.21.2.101:0/1857925112", 2021-02-10T11:38:22.659 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": [ 2021-02-10T11:38:22.659 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:38:22.659 INFO:teuthology.orchestra.run.gibba008.stdout: "tid": 14, 2021-02-10T11:38:22.659 INFO:teuthology.orchestra.run.gibba008.stdout: "created_ino": "0x10000000005" 2021-02-10T11:38:22.659 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:22.660 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:38:22.660 INFO:teuthology.orchestra.run.gibba008.stdout: "prealloc_inos": [ 2021-02-10T11:38:22.660 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:38:22.660 INFO:teuthology.orchestra.run.gibba008.stdout: "start": 1099511627782, 2021-02-10T11:38:22.660 INFO:teuthology.orchestra.run.gibba008.stdout: "length": 995 2021-02-10T11:38:22.660 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:22.660 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:38:22.660 INFO:teuthology.orchestra.run.gibba008.stdout: "used_inos": [], 2021-02-10T11:38:22.660 INFO:teuthology.orchestra.run.gibba008.stdout: "client_metadata": { 2021-02-10T11:38:22.660 INFO:teuthology.orchestra.run.gibba008.stdout: "features": "0x0000000000001fff", 2021-02-10T11:38:22.660 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:38:22.660 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:38:22.661 INFO:teuthology.orchestra.run.gibba008.stdout: "entity_id": "y", 2021-02-10T11:38:22.661 INFO:teuthology.orchestra.run.gibba008.stdout: "hostname": "gibba001", 2021-02-10T11:38:22.661 INFO:teuthology.orchestra.run.gibba008.stdout: "pid": "24165", 2021-02-10T11:38:22.661 INFO:teuthology.orchestra.run.gibba008.stdout: "root": "/" 2021-02-10T11:38:22.661 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:22.661 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:22.661 INFO:teuthology.orchestra.run.gibba008.stdout:] 2021-02-10T11:38:22.662 INFO:tasks.cephfs.filesystem:_json_asok output: [ { "id": 9594, "entity": { "name": { "type": "client", "num": 9594 }, "addr": { "type": "any", "addr": "172.21.2.101:0", "nonce": 1857925112 } }, "state": "open", "num_leases": 0, "num_caps": 7, "request_load_avg": 7, "uptime": 56.145066641, "requests_in_flight": 0, "completed_requests": 1, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 2.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 18.457805796694476, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 10 }, "inst": "client.9594 172.21.2.101:0/1857925112", "completed_requests": [ { "tid": 14, "created_ino": "0x10000000005" } ], "prealloc_inos": [ { "start": 1099511627782, "length": 995 } ], "used_inos": [], "client_metadata": { "features": "0x0000000000001fff", "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", "entity_id": "y", "hostname": "gibba001", "pid": "24165", "root": "/" } } ] 2021-02-10T11:38:22.662 DEBUG:tasks.ceph_test_case:wait_until_true: waiting... 2021-02-10T11:38:27.663 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:38:28.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.069 7ff9bbb35700 1 Processor -- start 2021-02-10T11:38:28.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.070 7ff9bbb35700 1 -- start start 2021-02-10T11:38:28.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.070 7ff9bbb35700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9b40d1870 0x7ff9b41b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:28.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.070 7ff9bbb35700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff9b41b0a10 0x7ff9b40cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:28.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.070 7ff9bbb35700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b40f0b10 con 0x7ff9b40d1c70 2021-02-10T11:38:28.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.070 7ff9bbb35700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b40fe230 con 0x7ff9b41b0a10 2021-02-10T11:38:28.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.070 7ff9bbb35700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40e5ed0 con 0x7ff9b40d1870 2021-02-10T11:38:28.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.070 7ff9b9951700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9b40d1870 0x7ff9b41b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:28.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.070 7ff9b9951700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9b40d1870 0x7ff9b41b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:47636/0 (socket says 172.21.2.101:47636) 2021-02-10T11:38:28.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.071 7ff9b9951700 1 -- 172.21.2.101:0/2158897417 learned_addr learned my addr 172.21.2.101:0/2158897417 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:28.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.071 7ff9b9150700 1 --2- 172.21.2.101:0/2158897417 >> v2:172.21.2.108:3300/0 conn(0x7ff9b41b0a10 0x7ff9b40cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:28.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.071 7ff9b894f700 1 -- 172.21.2.101:0/2158897417 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473353258 0 0) 0x7ff9ac001080 con 0x7ff9b40d1c70 2021-02-10T11:38:28.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.071 7ff9b9951700 1 -- 172.21.2.101:0/2158897417 >> v2:172.21.2.108:3300/0 conn(0x7ff9b41b0a10 msgr2=0x7ff9b40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:28.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.071 7ff9b9951700 1 --2- 172.21.2.101:0/2158897417 >> v2:172.21.2.108:3300/0 conn(0x7ff9b41b0a10 0x7ff9b40cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:28.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.071 7ff9b9951700 1 -- 172.21.2.101:0/2158897417 >> v1:172.21.2.108:6789/0 conn(0x7ff9b40d1c70 legacy=0x7ff9b40cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:28.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.071 7ff9b9951700 1 -- 172.21.2.101:0/2158897417 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b40f0650 con 0x7ff9b40d1870 2021-02-10T11:38:28.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.071 7ff9b9150700 1 --2- 172.21.2.101:0/2158897417 >> v2:172.21.2.108:3300/0 conn(0x7ff9b41b0a10 0x7ff9b40cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:38:28.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.071 7ff9b9951700 1 --2- 172.21.2.101:0/2158897417 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9b40d1870 0x7ff9b41b03f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7ff9a4006a80 tx=0x7ff9a4009f00).ready entity=mon.2 client_cookie=c7094f454c449598 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:28.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.072 7ff9b894f700 1 -- 172.21.2.101:0/2158897417 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff9a40066c0 con 0x7ff9b40d1870 2021-02-10T11:38:28.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.072 7ff9b894f700 1 -- 172.21.2.101:0/2158897417 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a400b5c0 con 0x7ff9b40d1870 2021-02-10T11:38:28.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.072 7ff9b894f700 1 -- 172.21.2.101:0/2158897417 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff9a400ba10 con 0x7ff9b40d1870 2021-02-10T11:38:28.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.072 7ff9bbb35700 1 -- 172.21.2.101:0/2158897417 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9b40d1870 msgr2=0x7ff9b41b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:28.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.072 7ff9bbb35700 1 --2- 172.21.2.101:0/2158897417 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9b40d1870 0x7ff9b41b03f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7ff9a4006a80 tx=0x7ff9a4009f00).stop 2021-02-10T11:38:28.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.072 7ff9bbb35700 1 -- 172.21.2.101:0/2158897417 shutdown_connections 2021-02-10T11:38:28.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.072 7ff9bbb35700 1 --2- 172.21.2.101:0/2158897417 >> v2:172.21.2.108:3300/0 conn(0x7ff9b41b0a10 0x7ff9b40cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:28.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.072 7ff9bbb35700 1 --2- 172.21.2.101:0/2158897417 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9b40d1870 0x7ff9b41b03f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:28.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.072 7ff9bbb35700 1 -- 172.21.2.101:0/2158897417 >> 172.21.2.101:0/2158897417 conn(0x7ff9b40ce450 msgr2=0x7ff9b40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:28.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.073 7ff9bbb35700 1 -- 172.21.2.101:0/2158897417 shutdown_connections 2021-02-10T11:38:28.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.073 7ff9bbb35700 1 -- 172.21.2.101:0/2158897417 wait complete. 2021-02-10T11:38:28.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.087 7ff9bbb35700 1 Processor -- start 2021-02-10T11:38:28.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.087 7ff9bbb35700 1 -- start start 2021-02-10T11:38:28.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.088 7ff9bbb35700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9b41942b0 0x7ff9b40b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:28.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.088 7ff9bbb35700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff9b4195070 0x7ff9b4195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:28.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.088 7ff9bbb35700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9b40bb8e0 0x7ff9b40bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:28.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.088 7ff9bbb35700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b41946b0 con 0x7ff9b41942b0 2021-02-10T11:38:28.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.088 7ff9bbb35700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40b3b70 con 0x7ff9b4195070 2021-02-10T11:38:28.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.088 7ff9bbb35700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40b3d90 con 0x7ff9b40bb8e0 2021-02-10T11:38:28.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.088 7ff9b9951700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9b41942b0 0x7ff9b40b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:28.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.088 7ff9b9951700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9b41942b0 0x7ff9b40b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54910/0 (socket says 172.21.2.101:54910) 2021-02-10T11:38:28.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.088 7ff9b9951700 1 -- 172.21.2.101:0/1674683972 learned_addr learned my addr 172.21.2.101:0/1674683972 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:28.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.088 7ff9ba152700 1 --2- 172.21.2.101:0/1674683972 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9b40bb8e0 0x7ff9b40bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:28.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.088 7ff9b9150700 1 --2- 172.21.2.101:0/1674683972 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff9b4195070 0x7ff9b4195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:28.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.088 7ff9b9951700 1 -- 172.21.2.101:0/1674683972 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9b40bb8e0 msgr2=0x7ff9b40bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:28.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.088 7ff9b9951700 1 --2- 172.21.2.101:0/1674683972 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9b40bb8e0 0x7ff9b40bbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:28.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.088 7ff9b9951700 1 -- 172.21.2.101:0/1674683972 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff9b4195070 msgr2=0x7ff9b4195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:28.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.088 7ff9b9951700 1 --2- 172.21.2.101:0/1674683972 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff9b4195070 0x7ff9b4195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:28.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.088 7ff9b9951700 1 -- 172.21.2.101:0/1674683972 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b41a49e0 con 0x7ff9b41942b0 2021-02-10T11:38:28.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.088 7ff9b9150700 1 --2- 172.21.2.101:0/1674683972 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff9b4195070 0x7ff9b4195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:38:28.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.089 7ff9b9951700 1 --2- 172.21.2.101:0/1674683972 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9b41942b0 0x7ff9b40b3550 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 rx=0x7ff9a400b910 tx=0x7ff9a400bcb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:28.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.089 7ff9b27fc700 1 -- 172.21.2.101:0/1674683972 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff9a400c2f0 con 0x7ff9b41942b0 2021-02-10T11:38:28.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.089 7ff9b27fc700 1 -- 172.21.2.101:0/1674683972 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a400c5b0 con 0x7ff9b41942b0 2021-02-10T11:38:28.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.089 7ff9b27fc700 1 -- 172.21.2.101:0/1674683972 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff9a400cad0 con 0x7ff9b41942b0 2021-02-10T11:38:28.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.089 7ff9bbb35700 1 -- 172.21.2.101:0/1674683972 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b41a49e0 con 0x7ff9b41942b0 2021-02-10T11:38:28.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.089 7ff9bbb35700 1 -- 172.21.2.101:0/1674683972 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b41a49e0 con 0x7ff9b41942b0 2021-02-10T11:38:28.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.090 7ff9b27fc700 1 -- 172.21.2.101:0/1674683972 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff9a4001010 con 0x7ff9b41942b0 2021-02-10T11:38:28.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.090 7ff9b27fc700 1 --2- 172.21.2.101:0/1674683972 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff9940186c0 0x7ff99401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:28.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.090 7ff9b27fc700 1 -- 172.21.2.101:0/1674683972 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff9a4032930 con 0x7ff9b41942b0 2021-02-10T11:38:28.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.090 7ff9b9150700 1 --2- 172.21.2.101:0/1674683972 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff9940186c0 0x7ff99401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:28.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.090 7ff9b9150700 1 --2- 172.21.2.101:0/1674683972 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff9940186c0 0x7ff99401abc0 crc :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:28.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.120 7ff9bbb35700 1 -- 172.21.2.101:0/1674683972 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b41a49e0 con 0x7ff9b41942b0 2021-02-10T11:38:28.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.122 7ff9b27fc700 1 -- 172.21.2.101:0/1674683972 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff9a400bdf0 con 0x7ff9b41942b0 2021-02-10T11:38:28.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.251 7ff9bbb35700 1 -- 172.21.2.101:0/1674683972 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9b41a49e0 con 0x7ff9b41942b0 2021-02-10T11:38:28.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.252 7ff9b27fc700 1 -- 172.21.2.101:0/1674683972 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+2101 (secure 0 0 0) 0x7ff9a4001010 con 0x7ff9b41942b0 2021-02-10T11:38:28.256 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 63 2021-02-10T11:38:28.256 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:38:28.256 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9360,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2529021396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2529021396},{"type":"v1","addr":"172.21.2.101:6835","nonce":2529021396}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:56.648753","modified":"2021-02-10 11:36:58.653009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6005":{"gid":6005,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2363068042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2363068042},{"type":"v1","addr":"172.21.2.108:6833","nonce":2363068042}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-02-10T11:38:28.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.254 7ff9bbb35700 1 -- 172.21.2.101:0/1674683972 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff9940186c0 msgr2=0x7ff99401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:28.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.254 7ff9bbb35700 1 --2- 172.21.2.101:0/1674683972 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff9940186c0 0x7ff99401abc0 crc :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:28.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.254 7ff9bbb35700 1 -- 172.21.2.101:0/1674683972 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9b41942b0 msgr2=0x7ff9b40b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:28.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.254 7ff9bbb35700 1 --2- 172.21.2.101:0/1674683972 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9b41942b0 0x7ff9b40b3550 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 rx=0x7ff9a400b910 tx=0x7ff9a400bcb0).stop 2021-02-10T11:38:28.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.254 7ff9bbb35700 1 -- 172.21.2.101:0/1674683972 shutdown_connections 2021-02-10T11:38:28.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.254 7ff9bbb35700 1 --2- 172.21.2.101:0/1674683972 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff9940186c0 0x7ff99401abc0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:28.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.254 7ff9bbb35700 1 --2- 172.21.2.101:0/1674683972 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9b40bb8e0 0x7ff9b40bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:28.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.254 7ff9bbb35700 1 --2- 172.21.2.101:0/1674683972 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff9b4195070 0x7ff9b4195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:28.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.254 7ff9bbb35700 1 --2- 172.21.2.101:0/1674683972 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9b41942b0 0x7ff9b40b3550 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:28.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.254 7ff9bbb35700 1 -- 172.21.2.101:0/1674683972 >> 172.21.2.101:0/1674683972 conn(0x7ff9b40b3040 msgr2=0x7ff9b40c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:28.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.254 7ff9bbb35700 1 -- 172.21.2.101:0/1674683972 shutdown_connections 2021-02-10T11:38:28.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:28.254 7ff9bbb35700 1 -- 172.21.2.101:0/1674683972 wait complete. 2021-02-10T11:38:28.266 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok session ls 2021-02-10T11:38:28.664 INFO:teuthology.orchestra.run.gibba008.stdout:[ 2021-02-10T11:38:28.664 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:38:28.664 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 9594, 2021-02-10T11:38:28.664 INFO:teuthology.orchestra.run.gibba008.stdout: "entity": { 2021-02-10T11:38:28.664 INFO:teuthology.orchestra.run.gibba008.stdout: "name": { 2021-02-10T11:38:28.664 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "client", 2021-02-10T11:38:28.664 INFO:teuthology.orchestra.run.gibba008.stdout: "num": 9594 2021-02-10T11:38:28.664 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:28.664 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": { 2021-02-10T11:38:28.664 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "any", 2021-02-10T11:38:28.664 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:38:28.665 INFO:teuthology.orchestra.run.gibba008.stdout: "nonce": 1857925112 2021-02-10T11:38:28.665 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:28.665 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:28.665 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "open", 2021-02-10T11:38:28.665 INFO:teuthology.orchestra.run.gibba008.stdout: "num_leases": 0, 2021-02-10T11:38:28.665 INFO:teuthology.orchestra.run.gibba008.stdout: "num_caps": 7, 2021-02-10T11:38:28.665 INFO:teuthology.orchestra.run.gibba008.stdout: "request_load_avg": 6, 2021-02-10T11:38:28.665 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 62.153776522999998, 2021-02-10T11:38:28.665 INFO:teuthology.orchestra.run.gibba008.stdout: "requests_in_flight": 0, 2021-02-10T11:38:28.665 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": 1, 2021-02-10T11:38:28.665 INFO:teuthology.orchestra.run.gibba008.stdout: "reconnecting": false, 2021-02-10T11:38:28.665 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps": { 2021-02-10T11:38:28.665 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:28.666 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:38:28.666 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:28.666 INFO:teuthology.orchestra.run.gibba008.stdout: "release_caps": { 2021-02-10T11:38:28.666 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:28.666 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:38:28.666 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:28.666 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle": { 2021-02-10T11:38:28.666 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:28.666 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 2.5 2021-02-10T11:38:28.666 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:28.666 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle2o": { 2021-02-10T11:38:28.666 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:28.666 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 0.5 2021-02-10T11:38:28.666 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:28.667 INFO:teuthology.orchestra.run.gibba008.stdout: "session_cache_liveness": { 2021-02-10T11:38:28.667 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 18.203325400651313, 2021-02-10T11:38:28.667 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 300 2021-02-10T11:38:28.667 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:28.667 INFO:teuthology.orchestra.run.gibba008.stdout: "cap_acquisition": { 2021-02-10T11:38:28.667 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:28.667 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 10 2021-02-10T11:38:28.667 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:28.667 INFO:teuthology.orchestra.run.gibba008.stdout: "inst": "client.9594 172.21.2.101:0/1857925112", 2021-02-10T11:38:28.667 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": [ 2021-02-10T11:38:28.667 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:38:28.667 INFO:teuthology.orchestra.run.gibba008.stdout: "tid": 14, 2021-02-10T11:38:28.667 INFO:teuthology.orchestra.run.gibba008.stdout: "created_ino": "0x10000000005" 2021-02-10T11:38:28.668 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:28.668 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:38:28.668 INFO:teuthology.orchestra.run.gibba008.stdout: "prealloc_inos": [ 2021-02-10T11:38:28.668 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:38:28.668 INFO:teuthology.orchestra.run.gibba008.stdout: "start": 1099511627782, 2021-02-10T11:38:28.668 INFO:teuthology.orchestra.run.gibba008.stdout: "length": 995 2021-02-10T11:38:28.668 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:28.668 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:38:28.668 INFO:teuthology.orchestra.run.gibba008.stdout: "used_inos": [], 2021-02-10T11:38:28.668 INFO:teuthology.orchestra.run.gibba008.stdout: "client_metadata": { 2021-02-10T11:38:28.668 INFO:teuthology.orchestra.run.gibba008.stdout: "features": "0x0000000000001fff", 2021-02-10T11:38:28.668 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:38:28.668 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:38:28.669 INFO:teuthology.orchestra.run.gibba008.stdout: "entity_id": "y", 2021-02-10T11:38:28.669 INFO:teuthology.orchestra.run.gibba008.stdout: "hostname": "gibba001", 2021-02-10T11:38:28.669 INFO:teuthology.orchestra.run.gibba008.stdout: "pid": "24165", 2021-02-10T11:38:28.669 INFO:teuthology.orchestra.run.gibba008.stdout: "root": "/" 2021-02-10T11:38:28.669 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:28.669 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:28.669 INFO:teuthology.orchestra.run.gibba008.stdout:] 2021-02-10T11:38:28.670 INFO:tasks.cephfs.filesystem:_json_asok output: [ { "id": 9594, "entity": { "name": { "type": "client", "num": 9594 }, "addr": { "type": "any", "addr": "172.21.2.101:0", "nonce": 1857925112 } }, "state": "open", "num_leases": 0, "num_caps": 7, "request_load_avg": 6, "uptime": 62.153776522999998, "requests_in_flight": 0, "completed_requests": 1, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 2.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 18.203325400651313, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 10 }, "inst": "client.9594 172.21.2.101:0/1857925112", "completed_requests": [ { "tid": 14, "created_ino": "0x10000000005" } ], "prealloc_inos": [ { "start": 1099511627782, "length": 995 } ], "used_inos": [], "client_metadata": { "features": "0x0000000000001fff", "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", "entity_id": "y", "hostname": "gibba001", "pid": "24165", "root": "/" } } ] 2021-02-10T11:38:28.670 DEBUG:tasks.ceph_test_case:wait_until_true: waiting... 2021-02-10T11:38:32.387 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:38:32.391 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:38:33.671 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:38:34.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.078 7f07f57c1700 1 Processor -- start 2021-02-10T11:38:34.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.078 7f07f57c1700 1 -- start start 2021-02-10T11:38:34.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.078 7f07f57c1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07f00d1870 0x7f07f01b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:34.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.078 7f07f57c1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f07f00f0650 0x7f07f00cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:34.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.078 7f07f57c1700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07f00f0b10 con 0x7f07f01b0a10 2021-02-10T11:38:34.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.078 7f07f57c1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07f00fe230 con 0x7f07f00f0650 2021-02-10T11:38:34.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.078 7f07f57c1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f00e5ed0 con 0x7f07f00d1870 2021-02-10T11:38:34.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.079 7f07efd75700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f07f00f0650 0x7f07f00cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:34.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.079 7f07ef574700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07f00d1870 0x7f07f01b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:34.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.079 7f07eed73700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f07f01b0a10 0x7f07f00cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:48818/0 (socket says 172.21.2.101:48818) 2021-02-10T11:38:34.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.079 7f07eed73700 1 -- 172.21.2.101:0/1072282845 learned_addr learned my addr 172.21.2.101:0/1072282845 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:38:34.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.080 7f07efd75700 1 -- 172.21.2.101:0/1072282845 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07f00d1870 msgr2=0x7f07f01b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:34.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.080 7f07efd75700 1 --2- 172.21.2.101:0/1072282845 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07f00d1870 0x7f07f01b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:34.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.080 7f07efd75700 1 -- 172.21.2.101:0/1072282845 >> v1:172.21.2.108:6789/0 conn(0x7f07f01b0a10 legacy=0x7f07f00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:34.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.080 7f07efd75700 1 -- 172.21.2.101:0/1072282845 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f00f20a0 con 0x7f07f00f0650 2021-02-10T11:38:34.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.080 7f07efd75700 1 --2- 172.21.2.101:0/1072282845 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07f00f0650 0x7f07f00cfb20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f07e8006e50 tx=0x7f07e8009e70).ready entity=mon.1 client_cookie=b9f570f859c49d45 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:34.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.081 7f07ee572700 1 -- 172.21.2.101:0/1072282845 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f07e8006a20 con 0x7f07f00f0650 2021-02-10T11:38:34.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.081 7f07ee572700 1 -- 172.21.2.101:0/1072282845 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e8006790 con 0x7f07f00f0650 2021-02-10T11:38:34.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.081 7f07ee572700 1 -- 172.21.2.101:0/1072282845 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f07e800b700 con 0x7f07f00f0650 2021-02-10T11:38:34.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.081 7f07f57c1700 1 -- 172.21.2.101:0/1072282845 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07f00f0650 msgr2=0x7f07f00cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:34.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.081 7f07f57c1700 1 --2- 172.21.2.101:0/1072282845 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07f00f0650 0x7f07f00cfb20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f07e8006e50 tx=0x7f07e8009e70).stop 2021-02-10T11:38:34.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.081 7f07f57c1700 1 -- 172.21.2.101:0/1072282845 shutdown_connections 2021-02-10T11:38:34.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.081 7f07f57c1700 1 --2- 172.21.2.101:0/1072282845 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07f00f0650 0x7f07f00cfb20 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:34.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.081 7f07f57c1700 1 --2- 172.21.2.101:0/1072282845 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07f00d1870 0x7f07f01b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:34.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.081 7f07f57c1700 1 -- 172.21.2.101:0/1072282845 >> 172.21.2.101:0/1072282845 conn(0x7f07f00ce450 msgr2=0x7f07f00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:34.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.081 7f07f57c1700 1 -- 172.21.2.101:0/1072282845 shutdown_connections 2021-02-10T11:38:34.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.081 7f07f57c1700 1 -- 172.21.2.101:0/1072282845 wait complete. 2021-02-10T11:38:34.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.096 7f07f57c1700 1 Processor -- start 2021-02-10T11:38:34.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.096 7f07f57c1700 1 -- start start 2021-02-10T11:38:34.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.096 7f07f57c1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07f00ada90 0x7f07f00ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:34.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.096 7f07f57c1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07f0099d20 0x7f07f009a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:34.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.096 7f07f57c1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07f009a740 0x7f07f0097bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:34.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.096 7f07f57c1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f00997a0 con 0x7f07f0099d20 2021-02-10T11:38:34.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.096 7f07f57c1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f018f0c0 con 0x7f07f009a740 2021-02-10T11:38:34.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.096 7f07f57c1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f009ab40 con 0x7f07f00ada90 2021-02-10T11:38:34.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.097 7f07eed73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07f0099d20 0x7f07f009a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:34.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.097 7f07eed73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07f0099d20 0x7f07f009a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54920/0 (socket says 172.21.2.101:54920) 2021-02-10T11:38:34.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.097 7f07eed73700 1 -- 172.21.2.101:0/4203407 learned_addr learned my addr 172.21.2.101:0/4203407 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:34.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.097 7f07ef574700 1 --2- 172.21.2.101:0/4203407 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07f00ada90 0x7f07f00ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:34.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.097 7f07efd75700 1 --2- 172.21.2.101:0/4203407 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07f009a740 0x7f07f0097bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:34.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.097 7f07eed73700 1 -- 172.21.2.101:0/4203407 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07f00ada90 msgr2=0x7f07f00ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:34.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.097 7f07eed73700 1 --2- 172.21.2.101:0/4203407 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07f00ada90 0x7f07f00ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:34.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.097 7f07eed73700 1 -- 172.21.2.101:0/4203407 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07f009a740 msgr2=0x7f07f0097bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:34.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.097 7f07eed73700 1 --2- 172.21.2.101:0/4203407 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07f009a740 0x7f07f0097bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:34.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.097 7f07eed73700 1 -- 172.21.2.101:0/4203407 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f009ad60 con 0x7f07f0099d20 2021-02-10T11:38:34.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.097 7f07efd75700 1 --2- 172.21.2.101:0/4203407 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07f009a740 0x7f07f0097bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:38:34.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.097 7f07eed73700 1 --2- 172.21.2.101:0/4203407 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07f0099d20 0x7f07f009a120 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 rx=0x7f07e4001c60 tx=0x7f07e400a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:34.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.097 7f07d7fff700 1 -- 172.21.2.101:0/4203407 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f07e400acc0 con 0x7f07f0099d20 2021-02-10T11:38:34.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.097 7f07d7fff700 1 -- 172.21.2.101:0/4203407 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e4004140 con 0x7f07f0099d20 2021-02-10T11:38:34.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.097 7f07f57c1700 1 -- 172.21.2.101:0/4203407 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f009ad60 con 0x7f07f0099d20 2021-02-10T11:38:34.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.097 7f07d7fff700 1 -- 172.21.2.101:0/4203407 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f07e4004620 con 0x7f07f0099d20 2021-02-10T11:38:34.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.097 7f07f57c1700 1 -- 172.21.2.101:0/4203407 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f00984d0 con 0x7f07f0099d20 2021-02-10T11:38:34.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.098 7f07d7fff700 1 -- 172.21.2.101:0/4203407 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f07e4004320 con 0x7f07f0099d20 2021-02-10T11:38:34.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.098 7f07d7fff700 1 --2- 172.21.2.101:0/4203407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07d00186c0 0x7f07d001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:34.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.098 7f07d7fff700 1 -- 172.21.2.101:0/4203407 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f07e40327a0 con 0x7f07f0099d20 2021-02-10T11:38:34.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.098 7f07ef574700 1 --2- 172.21.2.101:0/4203407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07d00186c0 0x7f07d001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:34.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.098 7f07ef574700 1 --2- 172.21.2.101:0/4203407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07d00186c0 0x7f07d001abc0 crc :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:34.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.128 7f07f57c1700 1 -- 172.21.2.101:0/4203407 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f018f070 con 0x7f07f0099d20 2021-02-10T11:38:34.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.131 7f07d7fff700 1 -- 172.21.2.101:0/4203407 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f07e4000dd0 con 0x7f07f0099d20 2021-02-10T11:38:34.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.258 7f07f57c1700 1 -- 172.21.2.101:0/4203407 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07f018f070 con 0x7f07f0099d20 2021-02-10T11:38:34.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.258 7f07d7fff700 1 -- 172.21.2.101:0/4203407 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f07e4004270 con 0x7f07f0099d20 2021-02-10T11:38:34.264 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:38:34.264 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9360,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2529021396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2529021396},{"type":"v1","addr":"172.21.2.101:6835","nonce":2529021396}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:56.648753","modified":"2021-02-10 11:36:58.653009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6005":{"gid":6005,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2363068042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2363068042},{"type":"v1","addr":"172.21.2.108:6833","nonce":2363068042}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-02-10T11:38:34.264 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 63 2021-02-10T11:38:34.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.260 7f07f57c1700 1 -- 172.21.2.101:0/4203407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07d00186c0 msgr2=0x7f07d001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:34.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.260 7f07f57c1700 1 --2- 172.21.2.101:0/4203407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07d00186c0 0x7f07d001abc0 crc :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:34.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.260 7f07f57c1700 1 -- 172.21.2.101:0/4203407 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07f0099d20 msgr2=0x7f07f009a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:34.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.260 7f07f57c1700 1 --2- 172.21.2.101:0/4203407 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07f0099d20 0x7f07f009a120 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 rx=0x7f07e4001c60 tx=0x7f07e400a8d0).stop 2021-02-10T11:38:34.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.260 7f07f57c1700 1 -- 172.21.2.101:0/4203407 shutdown_connections 2021-02-10T11:38:34.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.260 7f07f57c1700 1 --2- 172.21.2.101:0/4203407 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07f009a740 0x7f07f0097bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:34.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.260 7f07f57c1700 1 --2- 172.21.2.101:0/4203407 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07f0099d20 0x7f07f009a120 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:34.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.260 7f07f57c1700 1 --2- 172.21.2.101:0/4203407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07d00186c0 0x7f07d001abc0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:34.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.260 7f07f57c1700 1 --2- 172.21.2.101:0/4203407 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07f00ada90 0x7f07f00ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:34.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.260 7f07f57c1700 1 -- 172.21.2.101:0/4203407 >> 172.21.2.101:0/4203407 conn(0x7f07f00f0550 msgr2=0x7f07f0191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:34.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.261 7f07f57c1700 1 -- 172.21.2.101:0/4203407 shutdown_connections 2021-02-10T11:38:34.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:34.261 7f07f57c1700 1 -- 172.21.2.101:0/4203407 wait complete. 2021-02-10T11:38:34.273 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok session ls 2021-02-10T11:38:34.670 INFO:teuthology.orchestra.run.gibba008.stdout:[ 2021-02-10T11:38:34.670 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:38:34.670 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 9594, 2021-02-10T11:38:34.670 INFO:teuthology.orchestra.run.gibba008.stdout: "entity": { 2021-02-10T11:38:34.670 INFO:teuthology.orchestra.run.gibba008.stdout: "name": { 2021-02-10T11:38:34.670 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "client", 2021-02-10T11:38:34.670 INFO:teuthology.orchestra.run.gibba008.stdout: "num": 9594 2021-02-10T11:38:34.671 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:34.671 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": { 2021-02-10T11:38:34.671 INFO:teuthology.orchestra.run.gibba008.stdout: "type": "any", 2021-02-10T11:38:34.671 INFO:teuthology.orchestra.run.gibba008.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:38:34.671 INFO:teuthology.orchestra.run.gibba008.stdout: "nonce": 1857925112 2021-02-10T11:38:34.671 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:34.671 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:34.671 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "open", 2021-02-10T11:38:34.671 INFO:teuthology.orchestra.run.gibba008.stdout: "num_leases": 0, 2021-02-10T11:38:34.671 INFO:teuthology.orchestra.run.gibba008.stdout: "num_caps": 7, 2021-02-10T11:38:34.671 INFO:teuthology.orchestra.run.gibba008.stdout: "request_load_avg": 6, 2021-02-10T11:38:34.671 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 68.159490758000004, 2021-02-10T11:38:34.671 INFO:teuthology.orchestra.run.gibba008.stdout: "requests_in_flight": 0, 2021-02-10T11:38:34.672 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": 1, 2021-02-10T11:38:34.672 INFO:teuthology.orchestra.run.gibba008.stdout: "reconnecting": false, 2021-02-10T11:38:34.672 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps": { 2021-02-10T11:38:34.672 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:34.672 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:38:34.672 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:34.672 INFO:teuthology.orchestra.run.gibba008.stdout: "release_caps": { 2021-02-10T11:38:34.672 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:34.672 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 60 2021-02-10T11:38:34.672 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:34.672 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle": { 2021-02-10T11:38:34.672 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:34.672 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 2.5 2021-02-10T11:38:34.672 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:34.673 INFO:teuthology.orchestra.run.gibba008.stdout: "recall_caps_throttle2o": { 2021-02-10T11:38:34.673 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:34.673 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 0.5 2021-02-10T11:38:34.673 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:34.673 INFO:teuthology.orchestra.run.gibba008.stdout: "session_cache_liveness": { 2021-02-10T11:38:34.673 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 17.952477818160038, 2021-02-10T11:38:34.673 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 300 2021-02-10T11:38:34.673 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:34.673 INFO:teuthology.orchestra.run.gibba008.stdout: "cap_acquisition": { 2021-02-10T11:38:34.673 INFO:teuthology.orchestra.run.gibba008.stdout: "value": 0, 2021-02-10T11:38:34.673 INFO:teuthology.orchestra.run.gibba008.stdout: "halflife": 10 2021-02-10T11:38:34.673 INFO:teuthology.orchestra.run.gibba008.stdout: }, 2021-02-10T11:38:34.673 INFO:teuthology.orchestra.run.gibba008.stdout: "inst": "client.9594 172.21.2.101:0/1857925112", 2021-02-10T11:38:34.674 INFO:teuthology.orchestra.run.gibba008.stdout: "completed_requests": [ 2021-02-10T11:38:34.674 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:38:34.674 INFO:teuthology.orchestra.run.gibba008.stdout: "tid": 14, 2021-02-10T11:38:34.674 INFO:teuthology.orchestra.run.gibba008.stdout: "created_ino": "0x10000000005" 2021-02-10T11:38:34.674 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:34.674 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:38:34.674 INFO:teuthology.orchestra.run.gibba008.stdout: "prealloc_inos": [ 2021-02-10T11:38:34.674 INFO:teuthology.orchestra.run.gibba008.stdout: { 2021-02-10T11:38:34.674 INFO:teuthology.orchestra.run.gibba008.stdout: "start": 1099511627782, 2021-02-10T11:38:34.674 INFO:teuthology.orchestra.run.gibba008.stdout: "length": 995 2021-02-10T11:38:34.674 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:34.674 INFO:teuthology.orchestra.run.gibba008.stdout: ], 2021-02-10T11:38:34.674 INFO:teuthology.orchestra.run.gibba008.stdout: "used_inos": [], 2021-02-10T11:38:34.675 INFO:teuthology.orchestra.run.gibba008.stdout: "client_metadata": { 2021-02-10T11:38:34.675 INFO:teuthology.orchestra.run.gibba008.stdout: "features": "0x0000000000001fff", 2021-02-10T11:38:34.675 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:38:34.675 INFO:teuthology.orchestra.run.gibba008.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:38:34.675 INFO:teuthology.orchestra.run.gibba008.stdout: "entity_id": "y", 2021-02-10T11:38:34.675 INFO:teuthology.orchestra.run.gibba008.stdout: "hostname": "gibba001", 2021-02-10T11:38:34.675 INFO:teuthology.orchestra.run.gibba008.stdout: "pid": "24165", 2021-02-10T11:38:34.675 INFO:teuthology.orchestra.run.gibba008.stdout: "root": "/" 2021-02-10T11:38:34.675 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:34.675 INFO:teuthology.orchestra.run.gibba008.stdout: } 2021-02-10T11:38:34.675 INFO:teuthology.orchestra.run.gibba008.stdout:] 2021-02-10T11:38:34.676 INFO:tasks.cephfs.filesystem:_json_asok output: [ { "id": 9594, "entity": { "name": { "type": "client", "num": 9594 }, "addr": { "type": "any", "addr": "172.21.2.101:0", "nonce": 1857925112 } }, "state": "open", "num_leases": 0, "num_caps": 7, "request_load_avg": 6, "uptime": 68.159490758000004, "requests_in_flight": 0, "completed_requests": 1, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 2.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 17.952477818160038, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 10 }, "inst": "client.9594 172.21.2.101:0/1857925112", "completed_requests": [ { "tid": 14, "created_ino": "0x10000000005" } ], "prealloc_inos": [ { "start": 1099511627782, "length": 995 } ], "used_inos": [], "client_metadata": { "features": "0x0000000000001fff", "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", "entity_id": "y", "hostname": "gibba001", "pid": "24165", "root": "/" } } ] 2021-02-10T11:38:34.676 DEBUG:tasks.ceph_test_case:wait_until_true: waiting... 2021-02-10T11:38:39.677 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:38:40.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.084 7fdedb262700 1 Processor -- start 2021-02-10T11:38:40.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.084 7fdedb262700 1 -- start start 2021-02-10T11:38:40.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.084 7fdedb262700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fded40d1870 0x7fded41b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:40.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.084 7fdedb262700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded40f0b10 con 0x7fded41b0a10 2021-02-10T11:38:40.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.084 7fdedb262700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded40d0310 con 0x7fded40f0650 2021-02-10T11:38:40.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.084 7fdedb262700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded40d0530 con 0x7fded40d1870 2021-02-10T11:38:40.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.085 7fded887d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fded41b0a10 0x7fded40cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55420/0 (socket says 172.21.2.101:55420) 2021-02-10T11:38:40.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.085 7fded887d700 1 -- 172.21.2.101:0/1847496317 learned_addr learned my addr 172.21.2.101:0/1847496317 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:38:40.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.085 7fded907e700 1 --2- 172.21.2.101:0/1847496317 >> v2:172.21.2.101:3300/0 conn(0x7fded40d1870 0x7fded41b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:40.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.085 7fded907e700 1 -- 172.21.2.101:0/1847496317 >> v1:172.21.2.108:6789/0 conn(0x7fded40f0650 legacy=0x7fded40cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:40.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.085 7fded907e700 1 -- 172.21.2.101:0/1847496317 >> v1:172.21.2.101:6789/0 conn(0x7fded41b0a10 legacy=0x7fded40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:40.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.085 7fded907e700 1 -- 172.21.2.101:0/1847496317 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded40d1c70 con 0x7fded40d1870 2021-02-10T11:38:40.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.085 7fded907e700 1 --2- 172.21.2.101:0/1847496317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fded40d1870 0x7fded41b03f0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 rx=0x7fdec8006650 tx=0x7fdec8009a80).ready entity=mon.0 client_cookie=7db4904cbc0ae672 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:40.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.086 7fdec7fff700 1 -- 172.21.2.101:0/1847496317 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdec80009b0 con 0x7fded40d1870 2021-02-10T11:38:40.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.086 7fdec7fff700 1 -- 172.21.2.101:0/1847496317 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec800c0c0 con 0x7fded40d1870 2021-02-10T11:38:40.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.086 7fdec7fff700 1 -- 172.21.2.101:0/1847496317 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdec800c510 con 0x7fded40d1870 2021-02-10T11:38:40.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.086 7fdedb262700 1 -- 172.21.2.101:0/1847496317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fded40d1870 msgr2=0x7fded41b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:40.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.086 7fdedb262700 1 --2- 172.21.2.101:0/1847496317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fded40d1870 0x7fded41b03f0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 rx=0x7fdec8006650 tx=0x7fdec8009a80).stop 2021-02-10T11:38:40.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.087 7fdedb262700 1 -- 172.21.2.101:0/1847496317 shutdown_connections 2021-02-10T11:38:40.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.087 7fdedb262700 1 --2- 172.21.2.101:0/1847496317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fded40d1870 0x7fded41b03f0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:40.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.087 7fdedb262700 1 -- 172.21.2.101:0/1847496317 >> 172.21.2.101:0/1847496317 conn(0x7fded40ce450 msgr2=0x7fded40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:40.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.087 7fdedb262700 1 -- 172.21.2.101:0/1847496317 shutdown_connections 2021-02-10T11:38:40.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.087 7fdedb262700 1 -- 172.21.2.101:0/1847496317 wait complete. 2021-02-10T11:38:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.102 7fdedb262700 1 Processor -- start 2021-02-10T11:38:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.102 7fdedb262700 1 -- start start 2021-02-10T11:38:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.102 7fdedb262700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fded409bca0 0x7fded409c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.102 7fdedb262700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fded409c6c0 0x7fded409cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.102 7fdedb262700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fded408c1a0 0x7fded408c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.102 7fdedb262700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded409b720 con 0x7fded409c6c0 2021-02-10T11:38:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.102 7fdedb262700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded40abcd0 con 0x7fded408c1a0 2021-02-10T11:38:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.102 7fdedb262700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded408cbc0 con 0x7fded409bca0 2021-02-10T11:38:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.102 7fded887d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fded409c6c0 0x7fded409cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.102 7fded887d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fded409c6c0 0x7fded409cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54932/0 (socket says 172.21.2.101:54932) 2021-02-10T11:38:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.102 7fded887d700 1 -- 172.21.2.101:0/1179534817 learned_addr learned my addr 172.21.2.101:0/1179534817 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.102 7fded987f700 1 --2- 172.21.2.101:0/1179534817 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fded408c1a0 0x7fded408c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.102 7fded907e700 1 --2- 172.21.2.101:0/1179534817 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fded409bca0 0x7fded409c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.103 7fded887d700 1 -- 172.21.2.101:0/1179534817 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fded409bca0 msgr2=0x7fded409c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.103 7fded887d700 1 --2- 172.21.2.101:0/1179534817 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fded409bca0 0x7fded409c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.103 7fded887d700 1 -- 172.21.2.101:0/1179534817 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fded408c1a0 msgr2=0x7fded408c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.103 7fded887d700 1 --2- 172.21.2.101:0/1179534817 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fded408c1a0 0x7fded408c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.103 7fded887d700 1 -- 172.21.2.101:0/1179534817 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded408cde0 con 0x7fded409c6c0 2021-02-10T11:38:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.103 7fded887d700 1 --2- 172.21.2.101:0/1179534817 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fded409c6c0 0x7fded409cac0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 rx=0x7fded000a4c0 tx=0x7fded000a860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.103 7fdec5ffb700 1 -- 172.21.2.101:0/1179534817 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fded000ac50 con 0x7fded409c6c0 2021-02-10T11:38:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.103 7fdec5ffb700 1 -- 172.21.2.101:0/1179534817 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded0004140 con 0x7fded409c6c0 2021-02-10T11:38:40.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.103 7fdedb262700 1 -- 172.21.2.101:0/1179534817 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded408cde0 con 0x7fded409c6c0 2021-02-10T11:38:40.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.103 7fdedb262700 1 -- 172.21.2.101:0/1179534817 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded408cde0 con 0x7fded409c6c0 2021-02-10T11:38:40.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.103 7fdec5ffb700 1 -- 172.21.2.101:0/1179534817 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fded0004660 con 0x7fded409c6c0 2021-02-10T11:38:40.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.103 7fdec5ffb700 1 -- 172.21.2.101:0/1179534817 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fded0000ce0 con 0x7fded409c6c0 2021-02-10T11:38:40.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.103 7fdec5ffb700 1 --2- 172.21.2.101:0/1179534817 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdeb4018730 0x7fdeb401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:40.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.103 7fdec5ffb700 1 -- 172.21.2.101:0/1179534817 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fded00327c0 con 0x7fded409c6c0 2021-02-10T11:38:40.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.104 7fded907e700 1 --2- 172.21.2.101:0/1179534817 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdeb4018730 0x7fdeb401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:40.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.104 7fded907e700 1 --2- 172.21.2.101:0/1179534817 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdeb4018730 0x7fdeb401ac10 crc :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:40.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.134 7fdedb262700 1 -- 172.21.2.101:0/1179534817 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded40d1ee0 con 0x7fded409c6c0 2021-02-10T11:38:40.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.137 7fdec5ffb700 1 -- 172.21.2.101:0/1179534817 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fded00009b0 con 0x7fded409c6c0 2021-02-10T11:38:40.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.264 7fdedb262700 1 -- 172.21.2.101:0/1179534817 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fded408d660 con 0x7fded409c6c0 2021-02-10T11:38:40.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.265 7fdec5ffb700 1 -- 172.21.2.101:0/1179534817 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+2101 (secure 0 0 0) 0x7fded0000ce0 con 0x7fded409c6c0 2021-02-10T11:38:40.270 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:38:40.270 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9360,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2529021396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2529021396},{"type":"v1","addr":"172.21.2.101:6835","nonce":2529021396}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:56.648753","modified":"2021-02-10 11:36:58.653009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6005":{"gid":6005,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2363068042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2363068042},{"type":"v1","addr":"172.21.2.108:6833","nonce":2363068042}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-02-10T11:38:40.270 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 63 2021-02-10T11:38:40.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.267 7fdedb262700 1 -- 172.21.2.101:0/1179534817 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdeb4018730 msgr2=0x7fdeb401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:40.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.267 7fdedb262700 1 --2- 172.21.2.101:0/1179534817 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdeb4018730 0x7fdeb401ac10 crc :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:40.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.267 7fdedb262700 1 -- 172.21.2.101:0/1179534817 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fded409c6c0 msgr2=0x7fded409cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:40.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.267 7fdedb262700 1 --2- 172.21.2.101:0/1179534817 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fded409c6c0 0x7fded409cac0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 rx=0x7fded000a4c0 tx=0x7fded000a860).stop 2021-02-10T11:38:40.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.267 7fdedb262700 1 -- 172.21.2.101:0/1179534817 shutdown_connections 2021-02-10T11:38:40.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.267 7fdedb262700 1 --2- 172.21.2.101:0/1179534817 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fded408c1a0 0x7fded408c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:40.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.267 7fdedb262700 1 --2- 172.21.2.101:0/1179534817 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fded409c6c0 0x7fded409cac0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:40.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.267 7fdedb262700 1 --2- 172.21.2.101:0/1179534817 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdeb4018730 0x7fdeb401ac10 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:40.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.267 7fdedb262700 1 --2- 172.21.2.101:0/1179534817 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fded409bca0 0x7fded409c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:40.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.267 7fdedb262700 1 -- 172.21.2.101:0/1179534817 >> 172.21.2.101:0/1179534817 conn(0x7fded40a0450 msgr2=0x7fded40f0460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:40.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.267 7fdedb262700 1 -- 172.21.2.101:0/1179534817 shutdown_connections 2021-02-10T11:38:40.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:40.267 7fdedb262700 1 -- 172.21.2.101:0/1179534817 wait complete. 2021-02-10T11:38:40.280 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok session ls 2021-02-10T11:38:40.676 INFO:teuthology.orchestra.run.gibba008.stdout:[] 2021-02-10T11:38:40.681 INFO:tasks.cephfs.filesystem:_json_asok output: [] 2021-02-10T11:38:40.681 DEBUG:tasks.ceph_test_case:wait_until_true: success in 60s 2021-02-10T11:38:40.681 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:38:40.682 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:38:41.027 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:38:41.470 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:38:41.470 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:38:41.470 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:38:41.470 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:38:41.803 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2021-02-10T11:38:41.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:38:41.804 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-02-10T11:38:41.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:38:41.804 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:38:42.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:38:42.139 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:38:42.139 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:38:42.473 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:38:42.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:38:42.474 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:38:42.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:38:42.474 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:38:42.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:38:42.810 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:38:42.810 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:38:43.145 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:38:43.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:38:43.146 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:38:43.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:38:43.146 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:38:43.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:38:43.481 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:38:43.481 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:38:43.815 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:38:43.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:38:43.816 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:38:43.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:38:43.816 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:38:44.150 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-02-10T11:38:44.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.560 7f20749b3700 1 Processor -- start 2021-02-10T11:38:44.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.560 7f20749b3700 1 -- start start 2021-02-10T11:38:44.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.561 7f20749b3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f20700d1870 0x7f20701b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:44.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.561 7f20749b3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20700f0650 0x7f20700cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:44.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.561 7f20749b3700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20700f0b10 con 0x7f20701b0a10 2021-02-10T11:38:44.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.561 7f20749b3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20700e5ed0 con 0x7f20700d1870 2021-02-10T11:38:44.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.561 7f20749b3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20700d1c70 con 0x7f20700f0650 2021-02-10T11:38:44.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.562 7f206de1b700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f20701b0a10 0x7f20700cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:48842/0 (socket says 172.21.2.101:48842) 2021-02-10T11:38:44.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.562 7f206ee1d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20700f0650 0x7f20700cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:44.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.562 7f206de1b700 1 -- 172.21.2.101:0/3782629637 learned_addr learned my addr 172.21.2.101:0/3782629637 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:38:44.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.562 7f206e61c700 1 --2- 172.21.2.101:0/3782629637 >> v2:172.21.2.108:3300/0 conn(0x7f20700d1870 0x7f20701b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:44.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.562 7f206e61c700 1 -- 172.21.2.101:0/3782629637 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20700f0650 msgr2=0x7f20700cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:44.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.562 7f206e61c700 1 --2- 172.21.2.101:0/3782629637 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20700f0650 0x7f20700cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:44.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.562 7f206e61c700 1 -- 172.21.2.101:0/3782629637 >> v1:172.21.2.108:6789/0 conn(0x7f20701b0a10 legacy=0x7f20700cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:44.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.562 7f206e61c700 1 -- 172.21.2.101:0/3782629637 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20700d1e90 con 0x7f20700d1870 2021-02-10T11:38:44.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.562 7f206ee1d700 1 --2- 172.21.2.101:0/3782629637 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20700f0650 0x7f20700cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:38:44.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.563 7f206e61c700 1 --2- 172.21.2.101:0/3782629637 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f20700d1870 0x7f20701b03f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f2058006650 tx=0x7f2058009a80).ready entity=mon.1 client_cookie=534824ccfdb0e648 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:44.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.563 7f206d61a700 1 -- 172.21.2.101:0/3782629637 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f20580009b0 con 0x7f20700d1870 2021-02-10T11:38:44.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.563 7f206d61a700 1 -- 172.21.2.101:0/3782629637 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205800c0c0 con 0x7f20700d1870 2021-02-10T11:38:44.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.563 7f206d61a700 1 -- 172.21.2.101:0/3782629637 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f205800c550 con 0x7f20700d1870 2021-02-10T11:38:44.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.563 7f20749b3700 1 -- 172.21.2.101:0/3782629637 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f20700d1870 msgr2=0x7f20701b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:44.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.563 7f20749b3700 1 --2- 172.21.2.101:0/3782629637 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f20700d1870 0x7f20701b03f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f2058006650 tx=0x7f2058009a80).stop 2021-02-10T11:38:44.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.564 7f20749b3700 1 -- 172.21.2.101:0/3782629637 shutdown_connections 2021-02-10T11:38:44.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.564 7f20749b3700 1 --2- 172.21.2.101:0/3782629637 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20700f0650 0x7f20700cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:44.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.564 7f20749b3700 1 --2- 172.21.2.101:0/3782629637 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f20700d1870 0x7f20701b03f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:44.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.564 7f20749b3700 1 -- 172.21.2.101:0/3782629637 >> 172.21.2.101:0/3782629637 conn(0x7f20700ce450 msgr2=0x7f20700d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:44.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.564 7f20749b3700 1 -- 172.21.2.101:0/3782629637 shutdown_connections 2021-02-10T11:38:44.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.564 7f20749b3700 1 -- 172.21.2.101:0/3782629637 wait complete. 2021-02-10T11:38:44.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.579 7f20749b3700 1 Processor -- start 2021-02-10T11:38:44.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.579 7f20749b3700 1 -- start start 2021-02-10T11:38:44.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.579 7f20749b3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20701a4a80 0x7f20700906b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:44.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.579 7f20749b3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f20700c30f0 0x7f20700c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:44.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.579 7f20749b3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20700ef5a0 0x7f20700ef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:44.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.579 7f20749b3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20700c3b10 con 0x7f20700ef5a0 2021-02-10T11:38:44.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.579 7f20749b3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20700ef020 con 0x7f20700c30f0 2021-02-10T11:38:44.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.579 7f20749b3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20700effc0 con 0x7f20701a4a80 2021-02-10T11:38:44.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.579 7f206ee1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20700ef5a0 0x7f20700ef9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:44.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.579 7f206ee1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20700ef5a0 0x7f20700ef9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54942/0 (socket says 172.21.2.101:54942) 2021-02-10T11:38:44.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.579 7f206ee1d700 1 -- 172.21.2.101:0/3360863463 learned_addr learned my addr 172.21.2.101:0/3360863463 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:44.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.580 7f206ee1d700 1 -- 172.21.2.101:0/3360863463 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20701a4a80 msgr2=0x7f20700906b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:44.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.580 7f206ee1d700 1 --2- 172.21.2.101:0/3360863463 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20701a4a80 0x7f20700906b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:44.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.580 7f206ee1d700 1 -- 172.21.2.101:0/3360863463 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f20700c30f0 msgr2=0x7f20700c34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:44.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.580 7f206ee1d700 1 --2- 172.21.2.101:0/3360863463 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f20700c30f0 0x7f20700c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:44.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.580 7f206ee1d700 1 -- 172.21.2.101:0/3360863463 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20700ebe70 con 0x7f20700ef5a0 2021-02-10T11:38:44.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.580 7f206ee1d700 1 --2- 172.21.2.101:0/3360863463 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20700ef5a0 0x7f20700ef9a0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 rx=0x7f206800b970 tx=0x7f206800bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:44.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.580 7f2066ffd700 1 -- 172.21.2.101:0/3360863463 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2068000f90 con 0x7f20700ef5a0 2021-02-10T11:38:44.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.580 7f2066ffd700 1 -- 172.21.2.101:0/3360863463 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2068006790 con 0x7f20700ef5a0 2021-02-10T11:38:44.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.580 7f2066ffd700 1 -- 172.21.2.101:0/3360863463 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f20680015f0 con 0x7f20700ef5a0 2021-02-10T11:38:44.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.580 7f20749b3700 1 -- 172.21.2.101:0/3360863463 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20700ebe70 con 0x7f20700ef5a0 2021-02-10T11:38:44.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.580 7f20749b3700 1 -- 172.21.2.101:0/3360863463 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20700ec400 con 0x7f20700ef5a0 2021-02-10T11:38:44.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.581 7f2066ffd700 1 -- 172.21.2.101:0/3360863463 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f20680011f0 con 0x7f20700ef5a0 2021-02-10T11:38:44.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.581 7f2066ffd700 1 --2- 172.21.2.101:0/3360863463 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f20540186c0 0x7f205401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:44.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.581 7f2066ffd700 1 -- 172.21.2.101:0/3360863463 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2068032910 con 0x7f20700ef5a0 2021-02-10T11:38:44.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.581 7f206e61c700 1 --2- 172.21.2.101:0/3360863463 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f20540186c0 0x7f205401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:44.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.581 7f206e61c700 1 --2- 172.21.2.101:0/3360863463 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f20540186c0 0x7f205401abc0 crc :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:44.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.611 7f20749b3700 1 -- 172.21.2.101:0/3360863463 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20700f0b10 con 0x7f20700ef5a0 2021-02-10T11:38:44.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.613 7f2066ffd700 1 -- 172.21.2.101:0/3360863463 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2068000f70 con 0x7f20700ef5a0 2021-02-10T11:38:44.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.763 7f20749b3700 1 -- 172.21.2.101:0/3360863463 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f20700f0b10 con 0x7f20700ef5a0 2021-02-10T11:38:44.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.767 7f2066ffd700 1 -- 172.21.2.101:0/3360863463 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f20680011f0 con 0x7f20700ef5a0 2021-02-10T11:38:44.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.772 7f20749b3700 1 -- 172.21.2.101:0/3360863463 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f20540186c0 msgr2=0x7f205401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:44.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.772 7f20749b3700 1 --2- 172.21.2.101:0/3360863463 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f20540186c0 0x7f205401abc0 crc :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:44.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.772 7f20749b3700 1 -- 172.21.2.101:0/3360863463 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20700ef5a0 msgr2=0x7f20700ef9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:44.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.772 7f20749b3700 1 --2- 172.21.2.101:0/3360863463 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20700ef5a0 0x7f20700ef9a0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 rx=0x7f206800b970 tx=0x7f206800bd10).stop 2021-02-10T11:38:44.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.772 7f20749b3700 1 -- 172.21.2.101:0/3360863463 shutdown_connections 2021-02-10T11:38:44.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.772 7f20749b3700 1 --2- 172.21.2.101:0/3360863463 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20700ef5a0 0x7f20700ef9a0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:44.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.772 7f20749b3700 1 --2- 172.21.2.101:0/3360863463 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f20700c30f0 0x7f20700c34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:44.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.772 7f20749b3700 1 --2- 172.21.2.101:0/3360863463 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f20540186c0 0x7f205401abc0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:44.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.772 7f20749b3700 1 --2- 172.21.2.101:0/3360863463 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20701a4a80 0x7f20700906b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:44.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.772 7f20749b3700 1 -- 172.21.2.101:0/3360863463 >> 172.21.2.101:0/3360863463 conn(0x7f20700a6bb0 msgr2=0x7f2070095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:44.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.772 7f20749b3700 1 -- 172.21.2.101:0/3360863463 shutdown_connections 2021-02-10T11:38:44.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:44.772 7f20749b3700 1 -- 172.21.2.101:0/3360863463 wait complete. 2021-02-10T11:38:44.784 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_connection_expiration' 2021-02-10T11:38:45.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.190 7f8d8e941700 1 Processor -- start 2021-02-10T11:38:45.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.190 7f8d8e941700 1 -- start start 2021-02-10T11:38:45.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.191 7f8d8e941700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8d880d0340 0x7f8d880ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:45.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.191 7f8d8e941700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8d880d1c70 0x7f8d880efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:45.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.191 7f8d8e941700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d880f0b10 con 0x7f8d880d1870 2021-02-10T11:38:45.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.191 7f8d8e941700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d880e5ed0 con 0x7f8d880d0340 2021-02-10T11:38:45.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.191 7f8d8e941700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d881b0be0 con 0x7f8d880d1c70 2021-02-10T11:38:45.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.191 7f8d877fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8d880d0340 0x7f8d880ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:45.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.191 7f8d8cf5e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8d880d1c70 0x7f8d880efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:45.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.191 7f8d87fff700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f8d880d1870 0x7f8d881b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:48852/0 (socket says 172.21.2.101:48852) 2021-02-10T11:38:45.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.191 7f8d87fff700 1 -- 172.21.2.101:0/3122858121 learned_addr learned my addr 172.21.2.101:0/3122858121 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:38:45.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.192 7f8d8cf5e700 1 -- 172.21.2.101:0/3122858121 >> v2:172.21.2.101:3300/0 conn(0x7f8d880d0340 msgr2=0x7f8d880ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:45.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.192 7f8d8cf5e700 1 --2- 172.21.2.101:0/3122858121 >> v2:172.21.2.101:3300/0 conn(0x7f8d880d0340 0x7f8d880ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:45.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.192 7f8d8cf5e700 1 -- 172.21.2.101:0/3122858121 >> v1:172.21.2.108:6789/0 conn(0x7f8d880d1870 legacy=0x7f8d881b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:45.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.192 7f8d8cf5e700 1 -- 172.21.2.101:0/3122858121 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d880f20a0 con 0x7f8d880d1c70 2021-02-10T11:38:45.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.192 7f8d8cf5e700 1 --2- 172.21.2.101:0/3122858121 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d880d1c70 0x7f8d880efc60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f8d80006e50 tx=0x7f8d80009e70).ready entity=mon.1 client_cookie=b1ef9c098c1ebaf4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:45.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.192 7f8d86ffd700 1 -- 172.21.2.101:0/3122858121 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d80006a20 con 0x7f8d880d1c70 2021-02-10T11:38:45.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.192 7f8d86ffd700 1 -- 172.21.2.101:0/3122858121 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d80006790 con 0x7f8d880d1c70 2021-02-10T11:38:45.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.192 7f8d86ffd700 1 -- 172.21.2.101:0/3122858121 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d8000b700 con 0x7f8d880d1c70 2021-02-10T11:38:45.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.192 7f8d8e941700 1 -- 172.21.2.101:0/3122858121 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d880d1c70 msgr2=0x7f8d880efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:45.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.192 7f8d8e941700 1 --2- 172.21.2.101:0/3122858121 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d880d1c70 0x7f8d880efc60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f8d80006e50 tx=0x7f8d80009e70).stop 2021-02-10T11:38:45.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.193 7f8d8e941700 1 -- 172.21.2.101:0/3122858121 shutdown_connections 2021-02-10T11:38:45.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.193 7f8d8e941700 1 --2- 172.21.2.101:0/3122858121 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d880d1c70 0x7f8d880efc60 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:45.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.193 7f8d8e941700 1 --2- 172.21.2.101:0/3122858121 >> v2:172.21.2.101:3300/0 conn(0x7f8d880d0340 0x7f8d880ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:45.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.193 7f8d8e941700 1 -- 172.21.2.101:0/3122858121 >> 172.21.2.101:0/3122858121 conn(0x7f8d880ce450 msgr2=0x7f8d880d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:45.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.193 7f8d8e941700 1 -- 172.21.2.101:0/3122858121 shutdown_connections 2021-02-10T11:38:45.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.193 7f8d8e941700 1 -- 172.21.2.101:0/3122858121 wait complete. 2021-02-10T11:38:45.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.208 7f8d8e941700 1 Processor -- start 2021-02-10T11:38:45.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.208 7f8d8e941700 1 -- start start 2021-02-10T11:38:45.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.208 7f8d8e941700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d8818d870 0x7f8d880b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:45.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.208 7f8d8e941700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d880b1fd0 0x7f8d88099ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.208 7f8d8e941700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d880b23d0 0x7f8d8809a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.208 7f8d8e941700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8818dc70 con 0x7f8d880b1fd0 2021-02-10T11:38:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.208 7f8d8e941700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d88199e60 con 0x7f8d8818d870 2021-02-10T11:38:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.208 7f8d8e941700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8809ab30 con 0x7f8d880b23d0 2021-02-10T11:38:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.208 7f8d877fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d880b1fd0 0x7f8d88099ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.208 7f8d877fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d880b1fd0 0x7f8d88099ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54954/0 (socket says 172.21.2.101:54954) 2021-02-10T11:38:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.208 7f8d877fe700 1 -- 172.21.2.101:0/2526236233 learned_addr learned my addr 172.21.2.101:0/2526236233 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.209 7f8d877fe700 1 -- 172.21.2.101:0/2526236233 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d880b23d0 msgr2=0x7f8d8809a510 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:38:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.209 7f8d877fe700 1 --2- 172.21.2.101:0/2526236233 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d880b23d0 0x7f8d8809a510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.209 7f8d877fe700 1 -- 172.21.2.101:0/2526236233 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d8818d870 msgr2=0x7f8d880b19b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:38:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.209 7f8d877fe700 1 --2- 172.21.2.101:0/2526236233 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d8818d870 0x7f8d880b19b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.209 7f8d877fe700 1 -- 172.21.2.101:0/2526236233 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8809ad50 con 0x7f8d880b1fd0 2021-02-10T11:38:45.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.209 7f8d877fe700 1 --2- 172.21.2.101:0/2526236233 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d880b1fd0 0x7f8d88099ef0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 rx=0x7f8d7c006f60 tx=0x7f8d7c00b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.209 7f8d84ff9700 1 -- 172.21.2.101:0/2526236233 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d7c0009b0 con 0x7f8d880b1fd0 2021-02-10T11:38:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.209 7f8d84ff9700 1 -- 172.21.2.101:0/2526236233 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7c00bd60 con 0x7f8d880b1fd0 2021-02-10T11:38:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.209 7f8d84ff9700 1 -- 172.21.2.101:0/2526236233 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d7c002030 con 0x7f8d880b1fd0 2021-02-10T11:38:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.209 7f8d8e941700 1 -- 172.21.2.101:0/2526236233 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8809ad50 con 0x7f8d880b1fd0 2021-02-10T11:38:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.209 7f8d8e941700 1 -- 172.21.2.101:0/2526236233 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8818e3f0 con 0x7f8d880b1fd0 2021-02-10T11:38:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.210 7f8d84ff9700 1 -- 172.21.2.101:0/2526236233 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8d7c000c90 con 0x7f8d880b1fd0 2021-02-10T11:38:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.210 7f8d84ff9700 1 --2- 172.21.2.101:0/2526236233 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d680186c0 0x7f8d6801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.210 7f8d84ff9700 1 -- 172.21.2.101:0/2526236233 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8d7c032570 con 0x7f8d880b1fd0 2021-02-10T11:38:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.210 7f8d87fff700 1 --2- 172.21.2.101:0/2526236233 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d680186c0 0x7f8d6801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:45.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.210 7f8d87fff700 1 --2- 172.21.2.101:0/2526236233 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d680186c0 0x7f8d6801abc0 crc :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:45.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.241 7f8d8e941700 1 -- 172.21.2.101:0/2526236233 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d880999c0 con 0x7f8d880b1fd0 2021-02-10T11:38:45.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.244 7f8d84ff9700 1 -- 172.21.2.101:0/2526236233 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8d7c025d10 con 0x7f8d880b1fd0 2021-02-10T11:38:45.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.377 7f8d8e941700 1 -- 172.21.2.101:0/2526236233 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_connection_expiration"]} v 0) v1 -- 0x7f8d880999c0 con 0x7f8d880b1fd0 2021-02-10T11:38:45.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.532 7f8d84ff9700 1 -- 172.21.2.101:0/2526236233 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_connection_expiration"]}]=0 v476) v1 ==== 143+0+0 (secure 0 0 0) 0x7f8d7c025d10 con 0x7f8d880b1fd0 2021-02-10T11:38:45.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.543 7f8d8e941700 1 -- 172.21.2.101:0/2526236233 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d680186c0 msgr2=0x7f8d6801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:45.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.543 7f8d8e941700 1 --2- 172.21.2.101:0/2526236233 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d680186c0 0x7f8d6801abc0 crc :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:45.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.543 7f8d8e941700 1 -- 172.21.2.101:0/2526236233 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d880b1fd0 msgr2=0x7f8d88099ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:45.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.543 7f8d8e941700 1 --2- 172.21.2.101:0/2526236233 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d880b1fd0 0x7f8d88099ef0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 rx=0x7f8d7c006f60 tx=0x7f8d7c00b970).stop 2021-02-10T11:38:45.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.543 7f8d8e941700 1 -- 172.21.2.101:0/2526236233 shutdown_connections 2021-02-10T11:38:45.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.543 7f8d8e941700 1 --2- 172.21.2.101:0/2526236233 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d680186c0 0x7f8d6801abc0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:45.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.543 7f8d8e941700 1 --2- 172.21.2.101:0/2526236233 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d880b23d0 0x7f8d8809a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:45.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.543 7f8d8e941700 1 --2- 172.21.2.101:0/2526236233 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d880b1fd0 0x7f8d88099ef0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:45.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.543 7f8d8e941700 1 --2- 172.21.2.101:0/2526236233 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d8818d870 0x7f8d880b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:45.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.543 7f8d8e941700 1 -- 172.21.2.101:0/2526236233 >> 172.21.2.101:0/2526236233 conn(0x7f8d880f0550 msgr2=0x7f8d88191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:45.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.543 7f8d8e941700 1 -- 172.21.2.101:0/2526236233 shutdown_connections 2021-02-10T11:38:45.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.543 7f8d8e941700 1 -- 172.21.2.101:0/2526236233 wait complete. 2021-02-10T11:38:45.555 INFO:tasks.cephfs_test_runner:test_connection_expiration (tasks.cephfs.test_volumes.TestVolumes) ... ok 2021-02-10T11:38:45.555 INFO:tasks.cephfs_test_runner:Starting test: test_deauthorize_auth_id_after_out_of_band_update (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:38:45.556 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_deauthorize_auth_id_after_out_of_band_update' 2021-02-10T11:38:45.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.962 7ff8323d2700 1 Processor -- start 2021-02-10T11:38:45.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.962 7ff8323d2700 1 -- start start 2021-02-10T11:38:45.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.962 7ff8323d2700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff82c0d1870 0x7ff82c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:45.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.962 7ff8323d2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff82c1b0a10 0x7ff82c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:45.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.962 7ff8323d2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff82c0d1c70 0x7ff82c0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:45.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.962 7ff8323d2700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82c0fe200 con 0x7ff82c0d1c70 2021-02-10T11:38:45.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.962 7ff8323d2700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82c0e5f10 con 0x7ff82c0d1870 2021-02-10T11:38:45.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.962 7ff8323d2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c0f0b50 con 0x7ff82c1b0a10 2021-02-10T11:38:45.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.963 7ff8309ef700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff82c0d1c70 0x7ff82c0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:45.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.963 7ff8309ef700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff82c0d1c70 0x7ff82c0cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54962/0 (socket says 172.21.2.101:54962) 2021-02-10T11:38:45.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.963 7ff8309ef700 1 -- 172.21.2.101:0/4138767298 learned_addr learned my addr 172.21.2.101:0/4138767298 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:45.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.963 7ff8309ef700 1 -- 172.21.2.101:0/4138767298 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff82c1b0a10 msgr2=0x7ff82c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:45.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.963 7ff8309ef700 1 --2- 172.21.2.101:0/4138767298 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff82c1b0a10 0x7ff82c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:45.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.963 7ff8309ef700 1 -- 172.21.2.101:0/4138767298 >> v2:172.21.2.108:3300/0 conn(0x7ff82c0d1870 msgr2=0x7ff82c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:45.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.963 7ff8309ef700 1 --2- 172.21.2.101:0/4138767298 >> v2:172.21.2.108:3300/0 conn(0x7ff82c0d1870 0x7ff82c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:45.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.963 7ff8309ef700 1 -- 172.21.2.101:0/4138767298 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c0cff70 con 0x7ff82c0d1c70 2021-02-10T11:38:45.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.963 7ff8309ef700 1 --2- 172.21.2.101:0/4138767298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff82c0d1c70 0x7ff82c0cf950 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 rx=0x7ff824006f00 tx=0x7ff824009f10).ready entity=mon.0 client_cookie=d31aefa6ce6f244e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:45.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.963 7ff82affd700 1 -- 172.21.2.101:0/4138767298 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff82400b3b0 con 0x7ff82c0d1c70 2021-02-10T11:38:45.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.963 7ff82affd700 1 -- 172.21.2.101:0/4138767298 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82400b630 con 0x7ff82c0d1c70 2021-02-10T11:38:45.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.963 7ff82affd700 1 -- 172.21.2.101:0/4138767298 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff82400bb50 con 0x7ff82c0d1c70 2021-02-10T11:38:45.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.963 7ff8323d2700 1 -- 172.21.2.101:0/4138767298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff82c0d1c70 msgr2=0x7ff82c0cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:45.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.963 7ff8323d2700 1 --2- 172.21.2.101:0/4138767298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff82c0d1c70 0x7ff82c0cf950 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 rx=0x7ff824006f00 tx=0x7ff824009f10).stop 2021-02-10T11:38:45.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.964 7ff8323d2700 1 -- 172.21.2.101:0/4138767298 shutdown_connections 2021-02-10T11:38:45.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.964 7ff8323d2700 1 --2- 172.21.2.101:0/4138767298 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff82c1b0a10 0x7ff82c0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:45.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.964 7ff8323d2700 1 --2- 172.21.2.101:0/4138767298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff82c0d1c70 0x7ff82c0cf950 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:45.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.964 7ff8323d2700 1 --2- 172.21.2.101:0/4138767298 >> v2:172.21.2.108:3300/0 conn(0x7ff82c0d1870 0x7ff82c1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:45.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.964 7ff8323d2700 1 -- 172.21.2.101:0/4138767298 >> 172.21.2.101:0/4138767298 conn(0x7ff82c0ce450 msgr2=0x7ff82c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:45.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.964 7ff8323d2700 1 -- 172.21.2.101:0/4138767298 shutdown_connections 2021-02-10T11:38:45.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.964 7ff8323d2700 1 -- 172.21.2.101:0/4138767298 wait complete. 2021-02-10T11:38:45.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.979 7ff8323d2700 1 Processor -- start 2021-02-10T11:38:45.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.979 7ff8323d2700 1 -- start start 2021-02-10T11:38:45.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.979 7ff8323d2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff82c1971e0 0x7ff82c18eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:45.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.979 7ff8323d2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff82c1b8180 0x7ff82c1b8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:45.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.979 7ff8323d2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff82c1b8ba0 0x7ff82c0b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:45.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.979 7ff8323d2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c1b9750 con 0x7ff82c1b8ba0 2021-02-10T11:38:45.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.979 7ff8323d2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c18e930 con 0x7ff82c1b8180 2021-02-10T11:38:45.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.979 7ff8323d2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c0b1dd0 con 0x7ff82c1971e0 2021-02-10T11:38:45.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.979 7ff8309ef700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff82c1b8ba0 0x7ff82c0b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:45.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.979 7ff8309ef700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff82c1b8ba0 0x7ff82c0b17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54966/0 (socket says 172.21.2.101:54966) 2021-02-10T11:38:45.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.979 7ff8309ef700 1 -- 172.21.2.101:0/545635375 learned_addr learned my addr 172.21.2.101:0/545635375 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:45.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.980 7ff8309ef700 1 -- 172.21.2.101:0/545635375 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff82c1971e0 msgr2=0x7ff82c18eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:45.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.980 7ff8309ef700 1 --2- 172.21.2.101:0/545635375 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff82c1971e0 0x7ff82c18eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:45.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.980 7ff8309ef700 1 -- 172.21.2.101:0/545635375 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff82c1b8180 msgr2=0x7ff82c1b8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:45.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.980 7ff8309ef700 1 --2- 172.21.2.101:0/545635375 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff82c1b8180 0x7ff82c1b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:45.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.980 7ff8309ef700 1 -- 172.21.2.101:0/545635375 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c0b1ff0 con 0x7ff82c1b8ba0 2021-02-10T11:38:45.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.980 7ff8309ef700 1 --2- 172.21.2.101:0/545635375 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff82c1b8ba0 0x7ff82c0b17b0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 rx=0x7ff824009390 tx=0x7ff82400bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:45.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.980 7ff828ff9700 1 -- 172.21.2.101:0/545635375 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff824009450 con 0x7ff82c1b8ba0 2021-02-10T11:38:45.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.980 7ff828ff9700 1 -- 172.21.2.101:0/545635375 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8240068e0 con 0x7ff82c1b8ba0 2021-02-10T11:38:45.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.980 7ff8323d2700 1 -- 172.21.2.101:0/545635375 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff82c0b1ff0 con 0x7ff82c1b8ba0 2021-02-10T11:38:45.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.980 7ff8323d2700 1 -- 172.21.2.101:0/545635375 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82c0b2580 con 0x7ff82c1b8ba0 2021-02-10T11:38:45.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.980 7ff828ff9700 1 -- 172.21.2.101:0/545635375 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff82400c570 con 0x7ff82c1b8ba0 2021-02-10T11:38:45.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.981 7ff828ff9700 1 -- 172.21.2.101:0/545635375 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff824001270 con 0x7ff82c1b8ba0 2021-02-10T11:38:45.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.981 7ff828ff9700 1 --2- 172.21.2.101:0/545635375 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff80c0186c0 0x7ff80c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:45.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.981 7ff828ff9700 1 -- 172.21.2.101:0/545635375 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff8240328d0 con 0x7ff82c1b8ba0 2021-02-10T11:38:45.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.981 7ff82bfff700 1 --2- 172.21.2.101:0/545635375 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff80c0186c0 0x7ff80c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:45.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:45.982 7ff82bfff700 1 --2- 172.21.2.101:0/545635375 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff80c0186c0 0x7ff80c01abc0 crc :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:46.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.012 7ff8323d2700 1 -- 172.21.2.101:0/545635375 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff82c0ee060 con 0x7ff82c1b8ba0 2021-02-10T11:38:46.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.015 7ff828ff9700 1 -- 172.21.2.101:0/545635375 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff82402cf60 con 0x7ff82c1b8ba0 2021-02-10T11:38:46.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.146 7ff8323d2700 1 -- 172.21.2.101:0/545635375 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_deauthorize_auth_id_after_out_of_band_update"]} v 0) v1 -- 0x7ff82c0ce450 con 0x7ff82c1b8ba0 2021-02-10T11:38:46.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.537 7ff828ff9700 1 -- 172.21.2.101:0/545635375 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_deauthorize_auth_id_after_out_of_band_update"]}]=0 v477) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff824001270 con 0x7ff82c1b8ba0 2021-02-10T11:38:46.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.564 7ff8323d2700 1 -- 172.21.2.101:0/545635375 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff80c0186c0 msgr2=0x7ff80c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:46.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.564 7ff8323d2700 1 --2- 172.21.2.101:0/545635375 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff80c0186c0 0x7ff80c01abc0 crc :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:46.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.564 7ff8323d2700 1 -- 172.21.2.101:0/545635375 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff82c1b8ba0 msgr2=0x7ff82c0b17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:46.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.564 7ff8323d2700 1 --2- 172.21.2.101:0/545635375 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff82c1b8ba0 0x7ff82c0b17b0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 rx=0x7ff824009390 tx=0x7ff82400bd40).stop 2021-02-10T11:38:46.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.564 7ff8323d2700 1 -- 172.21.2.101:0/545635375 shutdown_connections 2021-02-10T11:38:46.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.564 7ff8323d2700 1 --2- 172.21.2.101:0/545635375 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff82c1b8ba0 0x7ff82c0b17b0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:46.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.564 7ff8323d2700 1 --2- 172.21.2.101:0/545635375 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff82c1b8180 0x7ff82c1b8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:46.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.564 7ff8323d2700 1 --2- 172.21.2.101:0/545635375 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff80c0186c0 0x7ff80c01abc0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:46.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.564 7ff8323d2700 1 --2- 172.21.2.101:0/545635375 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff82c1971e0 0x7ff82c18eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:46.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.564 7ff8323d2700 1 -- 172.21.2.101:0/545635375 >> 172.21.2.101:0/545635375 conn(0x7ff82c19f8f0 msgr2=0x7ff82c18c260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:46.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.565 7ff8323d2700 1 -- 172.21.2.101:0/545635375 shutdown_connections 2021-02-10T11:38:46.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.565 7ff8323d2700 1 -- 172.21.2.101:0/545635375 wait complete. 2021-02-10T11:38:46.576 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:38:47.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.986 7f64914ca700 1 Processor -- start 2021-02-10T11:38:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.986 7f64914ca700 1 -- start start 2021-02-10T11:38:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.986 7f64914ca700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f648c0d1c70 0x7f648c0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.986 7f64914ca700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f648c0cf580 0x7f648c0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.986 7f64914ca700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f648c0fe1c0 con 0x7f648c0d1870 2021-02-10T11:38:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.986 7f64914ca700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f648c1b0cb0 con 0x7f648c0cf580 2021-02-10T11:38:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.986 7f64914ca700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f648c109a60 con 0x7f648c0d1c70 2021-02-10T11:38:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.986 7f648bac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f648c0cf580 0x7f648c0cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.986 7f648b2bf700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f648c0d1870 0x7f648c1b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55466/0 (socket says 172.21.2.101:55466) 2021-02-10T11:38:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.987 7f648b2bf700 1 -- 172.21.2.101:0/4138409955 learned_addr learned my addr 172.21.2.101:0/4138409955 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:38:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.987 7f648aabe700 1 --2- 172.21.2.101:0/4138409955 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f648c0d1c70 0x7f648c0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.987 7f648bac0700 1 -- 172.21.2.101:0/4138409955 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f648c0d1c70 msgr2=0x7f648c0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.987 7f648bac0700 1 --2- 172.21.2.101:0/4138409955 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f648c0d1c70 0x7f648c0cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.987 7f648bac0700 1 -- 172.21.2.101:0/4138409955 >> v1:172.21.2.101:6789/0 conn(0x7f648c0d1870 legacy=0x7f648c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.987 7f648a2bd700 1 -- 172.21.2.101:0/4138409955 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2685559323 0 0) 0x7f647c001180 con 0x7f648c0d1870 2021-02-10T11:38:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.987 7f648bac0700 1 -- 172.21.2.101:0/4138409955 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f648c088070 con 0x7f648c0cf580 2021-02-10T11:38:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.987 7f648bac0700 1 --2- 172.21.2.101:0/4138409955 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f648c0cf580 0x7f648c0cf980 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 rx=0x7f6484006e30 tx=0x7f6484009de0).ready entity=mon.0 client_cookie=f13cf41aec8c2a64 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.988 7f648a2bd700 1 -- 172.21.2.101:0/4138409955 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f648400b300 con 0x7f648c0cf580 2021-02-10T11:38:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.988 7f648a2bd700 1 -- 172.21.2.101:0/4138409955 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648400b580 con 0x7f648c0cf580 2021-02-10T11:38:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.988 7f648a2bd700 1 -- 172.21.2.101:0/4138409955 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f648400baa0 con 0x7f648c0cf580 2021-02-10T11:38:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.988 7f64914ca700 1 -- 172.21.2.101:0/4138409955 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f648c0cf580 msgr2=0x7f648c0cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.988 7f64914ca700 1 --2- 172.21.2.101:0/4138409955 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f648c0cf580 0x7f648c0cf980 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 rx=0x7f6484006e30 tx=0x7f6484009de0).stop 2021-02-10T11:38:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.988 7f64914ca700 1 -- 172.21.2.101:0/4138409955 shutdown_connections 2021-02-10T11:38:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.988 7f64914ca700 1 --2- 172.21.2.101:0/4138409955 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f648c0d1c70 0x7f648c0cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:47.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.988 7f64914ca700 1 --2- 172.21.2.101:0/4138409955 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f648c0cf580 0x7f648c0cf980 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:47.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.988 7f64914ca700 1 -- 172.21.2.101:0/4138409955 >> 172.21.2.101:0/4138409955 conn(0x7f648c0ce420 msgr2=0x7f648c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:47.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.988 7f64914ca700 1 -- 172.21.2.101:0/4138409955 shutdown_connections 2021-02-10T11:38:47.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:46.988 7f64914ca700 1 -- 172.21.2.101:0/4138409955 wait complete. 2021-02-10T11:38:47.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.003 7f64914ca700 1 Processor -- start 2021-02-10T11:38:47.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.003 7f64914ca700 1 -- start start 2021-02-10T11:38:47.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.003 7f64914ca700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f648c0b04d0 0x7f648c1a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:47.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.003 7f64914ca700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f648c1a8090 0x7f648c0c23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:47.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.003 7f64914ca700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f648c08d680 0x7f648c0c29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:47.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.003 7f64914ca700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648c0b0a00 con 0x7f648c1a8090 2021-02-10T11:38:47.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.003 7f64914ca700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648c0c2fe0 con 0x7f648c0b04d0 2021-02-10T11:38:47.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.003 7f64914ca700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f648c0c3200 con 0x7f648c08d680 2021-02-10T11:38:47.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.004 7f648aabe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f648c1a8090 0x7f648c0c23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:47.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.004 7f648aabe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f648c1a8090 0x7f648c0c23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54978/0 (socket says 172.21.2.101:54978) 2021-02-10T11:38:47.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.004 7f648aabe700 1 -- 172.21.2.101:0/4022882400 learned_addr learned my addr 172.21.2.101:0/4022882400 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:47.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.004 7f648aabe700 1 -- 172.21.2.101:0/4022882400 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f648c08d680 msgr2=0x7f648c0c29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:47.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.004 7f648aabe700 1 --2- 172.21.2.101:0/4022882400 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f648c08d680 0x7f648c0c29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:47.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.004 7f648aabe700 1 -- 172.21.2.101:0/4022882400 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f648c0b04d0 msgr2=0x7f648c1a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:47.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.004 7f648aabe700 1 --2- 172.21.2.101:0/4022882400 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f648c0b04d0 0x7f648c1a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:47.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.004 7f648aabe700 1 -- 172.21.2.101:0/4022882400 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f648c08e080 con 0x7f648c1a8090 2021-02-10T11:38:47.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.004 7f648aabe700 1 --2- 172.21.2.101:0/4022882400 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f648c1a8090 0x7f648c0c23a0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 rx=0x7f6480006f00 tx=0x7f648000b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:47.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.005 7f6473fff700 1 -- 172.21.2.101:0/4022882400 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f648000bd30 con 0x7f648c1a8090 2021-02-10T11:38:47.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.005 7f6473fff700 1 -- 172.21.2.101:0/4022882400 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6480002030 con 0x7f648c1a8090 2021-02-10T11:38:47.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.005 7f64914ca700 1 -- 172.21.2.101:0/4022882400 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f648c08e080 con 0x7f648c1a8090 2021-02-10T11:38:47.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.005 7f64914ca700 1 -- 172.21.2.101:0/4022882400 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648c08e610 con 0x7f648c1a8090 2021-02-10T11:38:47.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.005 7f6473fff700 1 -- 172.21.2.101:0/4022882400 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6480002510 con 0x7f648c1a8090 2021-02-10T11:38:47.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.005 7f6473fff700 1 -- 172.21.2.101:0/4022882400 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6480000c90 con 0x7f648c1a8090 2021-02-10T11:38:47.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.005 7f6473fff700 1 --2- 172.21.2.101:0/4022882400 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f646c0186c0 0x7f646c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:47.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.005 7f6473fff700 1 -- 172.21.2.101:0/4022882400 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6480032550 con 0x7f648c1a8090 2021-02-10T11:38:47.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.005 7f648b2bf700 1 --2- 172.21.2.101:0/4022882400 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f646c0186c0 0x7f646c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:47.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.006 7f648b2bf700 1 --2- 172.21.2.101:0/4022882400 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f646c0186c0 0x7f646c01abc0 crc :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:47.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.036 7f64914ca700 1 -- 172.21.2.101:0/4022882400 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f648c082a70 con 0x7f648c1a8090 2021-02-10T11:38:47.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.039 7f6473fff700 1 -- 172.21.2.101:0/4022882400 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6480000940 con 0x7f648c1a8090 2021-02-10T11:38:47.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.189 7f64914ca700 1 -- 172.21.2.101:0/4022882400 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f648c082a70 con 0x7f648c1a8090 2021-02-10T11:38:47.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.193 7f6473fff700 1 -- 172.21.2.101:0/4022882400 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6480000c90 con 0x7f648c1a8090 2021-02-10T11:38:47.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.198 7f64914ca700 1 -- 172.21.2.101:0/4022882400 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f646c0186c0 msgr2=0x7f646c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:47.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.198 7f64914ca700 1 --2- 172.21.2.101:0/4022882400 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f646c0186c0 0x7f646c01abc0 crc :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:47.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.198 7f64914ca700 1 -- 172.21.2.101:0/4022882400 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f648c1a8090 msgr2=0x7f648c0c23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:47.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.198 7f64914ca700 1 --2- 172.21.2.101:0/4022882400 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f648c1a8090 0x7f648c0c23a0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 rx=0x7f6480006f00 tx=0x7f648000b940).stop 2021-02-10T11:38:47.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.198 7f64914ca700 1 -- 172.21.2.101:0/4022882400 shutdown_connections 2021-02-10T11:38:47.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.198 7f64914ca700 1 --2- 172.21.2.101:0/4022882400 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f646c0186c0 0x7f646c01abc0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:47.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.198 7f64914ca700 1 --2- 172.21.2.101:0/4022882400 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f648c08d680 0x7f648c0c29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:47.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.198 7f64914ca700 1 --2- 172.21.2.101:0/4022882400 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f648c1a8090 0x7f648c0c23a0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:47.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.198 7f64914ca700 1 --2- 172.21.2.101:0/4022882400 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f648c0b04d0 0x7f648c1a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:47.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.198 7f64914ca700 1 -- 172.21.2.101:0/4022882400 >> 172.21.2.101:0/4022882400 conn(0x7f648c0be530 msgr2=0x7f648c09f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:47.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.199 7f64914ca700 1 -- 172.21.2.101:0/4022882400 shutdown_connections 2021-02-10T11:38:47.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:47.199 7f64914ca700 1 -- 172.21.2.101:0/4022882400 wait complete. 2021-02-10T11:38:47.211 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:38:47.211 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:38:47.559 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:38:47.904 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:38:47.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:38:47.905 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:38:47.905 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:38:48.239 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2021-02-10T11:38:48.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:38:48.240 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-02-10T11:38:48.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:38:48.241 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:38:48.241 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:38:48.576 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:38:48.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:38:48.576 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:38:48.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:38:48.576 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:38:48.576 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:38:48.911 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:38:48.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:38:48.912 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:38:48.912 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:38:48.912 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:38:48.912 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:38:49.247 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:38:49.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:38:49.248 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:38:49.248 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json-pretty 2021-02-10T11:38:49.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.655 7f46bbb2d700 1 Processor -- start 2021-02-10T11:38:49.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.655 7f46bbb2d700 1 -- start start 2021-02-10T11:38:49.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.656 7f46bbb2d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f46b40d1870 0x7f46b41b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:49.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.656 7f46bbb2d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f46b41b0a10 0x7f46b40cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:49.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.656 7f46bbb2d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f46b40d1c70 0x7f46b40cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:49.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.656 7f46bbb2d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b40fe230 con 0x7f46b40d1c70 2021-02-10T11:38:49.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.656 7f46bbb2d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b40e5ed0 con 0x7f46b41b0a10 2021-02-10T11:38:49.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.656 7f46bbb2d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b40f0b10 con 0x7f46b40d1870 2021-02-10T11:38:49.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.656 7f46ba14a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f46b40d1c70 0x7f46b40cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:49.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.656 7f46ba14a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f46b40d1c70 0x7f46b40cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54986/0 (socket says 172.21.2.101:54986) 2021-02-10T11:38:49.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.656 7f46ba14a700 1 -- 172.21.2.101:0/1020599396 learned_addr learned my addr 172.21.2.101:0/1020599396 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:49.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.656 7f46ba14a700 1 -- 172.21.2.101:0/1020599396 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f46b40d1870 msgr2=0x7f46b41b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:38:49.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.656 7f46ba14a700 1 --2- 172.21.2.101:0/1020599396 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f46b40d1870 0x7f46b41b03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:49.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.656 7f46ba14a700 1 -- 172.21.2.101:0/1020599396 >> v2:172.21.2.108:3300/0 conn(0x7f46b41b0a10 msgr2=0x7f46b40cf330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:38:49.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.656 7f46ba14a700 1 --2- 172.21.2.101:0/1020599396 >> v2:172.21.2.108:3300/0 conn(0x7f46b41b0a10 0x7f46b40cf330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:49.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.656 7f46ba14a700 1 -- 172.21.2.101:0/1020599396 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b40cff70 con 0x7f46b40d1c70 2021-02-10T11:38:49.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.657 7f46ba14a700 1 --2- 172.21.2.101:0/1020599396 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f46b40d1c70 0x7f46b40cf950 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 rx=0x7f46ac006f00 tx=0x7f46ac009f10).ready entity=mon.0 client_cookie=737bfa5452f158a1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:49.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.657 7f46b8947700 1 -- 172.21.2.101:0/1020599396 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f46ac00b3b0 con 0x7f46b40d1c70 2021-02-10T11:38:49.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.657 7f46b8947700 1 -- 172.21.2.101:0/1020599396 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ac00b630 con 0x7f46b40d1c70 2021-02-10T11:38:49.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.657 7f46b8947700 1 -- 172.21.2.101:0/1020599396 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f46ac00bb50 con 0x7f46b40d1c70 2021-02-10T11:38:49.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.657 7f46bbb2d700 1 -- 172.21.2.101:0/1020599396 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f46b40d1c70 msgr2=0x7f46b40cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:49.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.657 7f46bbb2d700 1 --2- 172.21.2.101:0/1020599396 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f46b40d1c70 0x7f46b40cf950 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 rx=0x7f46ac006f00 tx=0x7f46ac009f10).stop 2021-02-10T11:38:49.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.657 7f46bbb2d700 1 -- 172.21.2.101:0/1020599396 shutdown_connections 2021-02-10T11:38:49.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.658 7f46bbb2d700 1 --2- 172.21.2.101:0/1020599396 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f46b40d1c70 0x7f46b40cf950 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:49.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.658 7f46bbb2d700 1 --2- 172.21.2.101:0/1020599396 >> v2:172.21.2.108:3300/0 conn(0x7f46b41b0a10 0x7f46b40cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:49.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.658 7f46bbb2d700 1 --2- 172.21.2.101:0/1020599396 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f46b40d1870 0x7f46b41b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:49.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.658 7f46bbb2d700 1 -- 172.21.2.101:0/1020599396 >> 172.21.2.101:0/1020599396 conn(0x7f46b40ce450 msgr2=0x7f46b40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:49.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.658 7f46bbb2d700 1 -- 172.21.2.101:0/1020599396 shutdown_connections 2021-02-10T11:38:49.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.658 7f46bbb2d700 1 -- 172.21.2.101:0/1020599396 wait complete. 2021-02-10T11:38:49.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.672 7f46bbb2d700 1 Processor -- start 2021-02-10T11:38:49.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.672 7f46bbb2d700 1 -- start start 2021-02-10T11:38:49.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.672 7f46bbb2d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f46b41b8ad0 0x7f46b4187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:49.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.672 7f46bbb2d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f46b41b8ed0 0x7f46b40b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:49.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.672 7f46bbb2d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f46b40b2110 0x7f46b41b7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:49.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.672 7f46bbb2d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b4188470 con 0x7f46b41b8ad0 2021-02-10T11:38:49.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.672 7f46bbb2d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b418be10 con 0x7f46b40b2110 2021-02-10T11:38:49.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.672 7f46bbb2d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b418c030 con 0x7f46b41b8ed0 2021-02-10T11:38:49.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.673 7f46b9949700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f46b41b8ad0 0x7f46b4187e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:49.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.673 7f46b9949700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f46b41b8ad0 0x7f46b4187e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:54990/0 (socket says 172.21.2.101:54990) 2021-02-10T11:38:49.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.673 7f46b9949700 1 -- 172.21.2.101:0/241713191 learned_addr learned my addr 172.21.2.101:0/241713191 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:49.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.673 7f46b9148700 1 --2- 172.21.2.101:0/241713191 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f46b41b8ed0 0x7f46b40b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:49.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.673 7f46b9949700 1 -- 172.21.2.101:0/241713191 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f46b41b8ed0 msgr2=0x7f46b40b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:49.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.673 7f46b9949700 1 --2- 172.21.2.101:0/241713191 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f46b41b8ed0 0x7f46b40b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:49.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.673 7f46b9949700 1 -- 172.21.2.101:0/241713191 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f46b40b2110 msgr2=0x7f46b41b7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:49.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.673 7f46b9949700 1 --2- 172.21.2.101:0/241713191 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f46b40b2110 0x7f46b41b7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:49.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.673 7f46b9949700 1 -- 172.21.2.101:0/241713191 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b40b2510 con 0x7f46b41b8ad0 2021-02-10T11:38:49.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.673 7f46b9148700 1 --2- 172.21.2.101:0/241713191 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f46b41b8ed0 0x7f46b40b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:38:49.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.673 7f46b9949700 1 --2- 172.21.2.101:0/241713191 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f46b41b8ad0 0x7f46b4187e50 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 rx=0x7f46b000b5c0 tx=0x7f46b000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:49.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.673 7f46a67fc700 1 -- 172.21.2.101:0/241713191 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f46b000bd50 con 0x7f46b41b8ad0 2021-02-10T11:38:49.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.673 7f46a67fc700 1 -- 172.21.2.101:0/241713191 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b0006030 con 0x7f46b41b8ad0 2021-02-10T11:38:49.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.673 7f46a67fc700 1 -- 172.21.2.101:0/241713191 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f46b00054d0 con 0x7f46b41b8ad0 2021-02-10T11:38:49.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.673 7f46bbb2d700 1 -- 172.21.2.101:0/241713191 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b40b2510 con 0x7f46b41b8ad0 2021-02-10T11:38:49.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.674 7f46bbb2d700 1 -- 172.21.2.101:0/241713191 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b40b2510 con 0x7f46b41b8ad0 2021-02-10T11:38:49.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.674 7f46a67fc700 1 -- 172.21.2.101:0/241713191 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f46b0006210 con 0x7f46b41b8ad0 2021-02-10T11:38:49.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.674 7f46a67fc700 1 --2- 172.21.2.101:0/241713191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f469c0186d0 0x7f469c01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:49.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.674 7f46a67fc700 1 -- 172.21.2.101:0/241713191 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f46b00327d0 con 0x7f46b41b8ad0 2021-02-10T11:38:49.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.675 7f46b9148700 1 --2- 172.21.2.101:0/241713191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f469c0186d0 0x7f469c01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:49.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.675 7f46b9148700 1 --2- 172.21.2.101:0/241713191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f469c0186d0 0x7f469c01abd0 crc :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:49.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.706 7f46bbb2d700 1 -- 172.21.2.101:0/241713191 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b40ebc00 con 0x7f46b41b8ad0 2021-02-10T11:38:49.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.708 7f46a67fc700 1 -- 172.21.2.101:0/241713191 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f46b0006b80 con 0x7f46b41b8ad0 2021-02-10T11:38:49.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.832 7f46bbb2d700 1 -- 172.21.2.101:0/241713191 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f46b40ebc00 con 0x7f46b41b8ad0 2021-02-10T11:38:49.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.833 7f46a67fc700 1 -- 172.21.2.101:0/241713191 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v85) v1 ==== 81+0+30363 (secure 0 0 0) 0x7f46b0006030 con 0x7f46b41b8ad0 2021-02-10T11:38:49.838 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:38:49.838 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:38:49.838 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 85, 2021-02-10T11:38:49.838 INFO:teuthology.orchestra.run.gibba001.stdout: "fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:38:49.838 INFO:teuthology.orchestra.run.gibba001.stdout: "created": "2021-02-10 11:12:36.788366", 2021-02-10T11:38:49.838 INFO:teuthology.orchestra.run.gibba001.stdout: "modified": "2021-02-10 11:36:58.653077", 2021-02-10T11:38:49.838 INFO:teuthology.orchestra.run.gibba001.stdout: "last_up_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:38:49.838 INFO:teuthology.orchestra.run.gibba001.stdout: "last_in_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:38:49.838 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-02-10T11:38:49.838 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_num": 5799936, 2021-02-10T11:38:49.838 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_set": [ 2021-02-10T11:38:49.839 INFO:teuthology.orchestra.run.gibba001.stdout: "pglog_hardlimit", 2021-02-10T11:38:49.839 INFO:teuthology.orchestra.run.gibba001.stdout: "purged_snapdirs", 2021-02-10T11:38:49.839 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_deletes", 2021-02-10T11:38:49.839 INFO:teuthology.orchestra.run.gibba001.stdout: "sortbitwise" 2021-02-10T11:38:49.839 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:38:49.839 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_version": 4, 2021-02-10T11:38:49.839 INFO:teuthology.orchestra.run.gibba001.stdout: "full_ratio": 0.94999998807907104, 2021-02-10T11:38:49.839 INFO:teuthology.orchestra.run.gibba001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-02-10T11:38:49.839 INFO:teuthology.orchestra.run.gibba001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-02-10T11:38:49.839 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_snapshot": "", 2021-02-10T11:38:49.839 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_max": 17, 2021-02-10T11:38:49.839 INFO:teuthology.orchestra.run.gibba001.stdout: "max_osd": 8, 2021-02-10T11:38:49.840 INFO:teuthology.orchestra.run.gibba001.stdout: "require_min_compat_client": "jewel", 2021-02-10T11:38:49.840 INFO:teuthology.orchestra.run.gibba001.stdout: "min_compat_client": "jewel", 2021-02-10T11:38:49.840 INFO:teuthology.orchestra.run.gibba001.stdout: "require_osd_release": "nautilus", 2021-02-10T11:38:49.840 INFO:teuthology.orchestra.run.gibba001.stdout: "pools": [ 2021-02-10T11:38:49.840 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.840 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 1, 2021-02-10T11:38:49.840 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "rbd", 2021-02-10T11:38:49.840 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:13:08.037472", 2021-02-10T11:38:49.840 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:38:49.840 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:38:49.840 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:38:49.840 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:38:49.840 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:38:49.841 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:38:49.841 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:38:49.841 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:38:49.841 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:38:49.841 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:38:49.841 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:38:49.841 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:38:49.841 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:38:49.841 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:38:49.841 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:38:49.841 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:38:49.841 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:38:49.841 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:38:49.842 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:38:49.842 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:38:49.842 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.842 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "16", 2021-02-10T11:38:49.842 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:38:49.842 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:38:49.842 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:38:49.842 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:38:49.842 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:38:49.842 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:38:49.842 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:38:49.842 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:38:49.842 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:38:49.843 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:38:49.843 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:38:49.843 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:38:49.843 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:38:49.843 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:38:49.843 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:38:49.843 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:38:49.843 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:38:49.843 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:38:49.843 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:38:49.843 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:38:49.843 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:38:49.843 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:38:49.843 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:38:49.844 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:38:49.844 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:38:49.844 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:38:49.844 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.844 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:38:49.844 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:38:49.844 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:38:49.844 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:38:49.844 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:38:49.844 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:38:49.844 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:38:49.844 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:38:49.844 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:38:49.845 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:38:49.845 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:38:49.845 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:38:49.845 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:38:49.845 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd": {} 2021-02-10T11:38:49.845 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.845 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.845 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.846 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 16, 2021-02-10T11:38:49.846 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_metadata", 2021-02-10T11:38:49.846 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:36:54.618463", 2021-02-10T11:38:49.846 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:38:49.846 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:38:49.846 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:38:49.846 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:38:49.846 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:38:49.846 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:38:49.846 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:38:49.847 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:38:49.847 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:38:49.847 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:38:49.847 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:38:49.847 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:38:49.847 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:38:49.847 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:38:49.847 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:38:49.847 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:38:49.847 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:38:49.848 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:38:49.848 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:38:49.848 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:38:49.848 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.848 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "83", 2021-02-10T11:38:49.848 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:38:49.848 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:38:49.848 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:38:49.848 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:38:49.848 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:38:49.848 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:38:49.848 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:38:49.848 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:38:49.849 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:38:49.849 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:38:49.849 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:38:49.849 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:38:49.849 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:38:49.849 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:38:49.849 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:38:49.849 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:38:49.849 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:38:49.849 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:38:49.849 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:38:49.849 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:38:49.849 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:38:49.850 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:38:49.850 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:38:49.850 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:38:49.850 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:38:49.850 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:38:49.850 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.850 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:38:49.850 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:38:49.850 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:38:49.850 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:38:49.850 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:38:49.850 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:38:49.851 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:38:49.851 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:38:49.851 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:38:49.851 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:38:49.851 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:38:49.851 INFO:teuthology.orchestra.run.gibba001.stdout: "options": { 2021-02-10T11:38:49.851 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_bias": 4, 2021-02-10T11:38:49.851 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_min": 16, 2021-02-10T11:38:49.851 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_priority": 5 2021-02-10T11:38:49.851 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.851 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:38:49.851 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:38:49.851 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": "cephfs" 2021-02-10T11:38:49.852 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.852 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.852 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.852 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.852 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 17, 2021-02-10T11:38:49.852 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_data", 2021-02-10T11:38:49.852 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:36:55.644563", 2021-02-10T11:38:49.852 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:38:49.852 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:38:49.852 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:38:49.852 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:38:49.852 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:38:49.852 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:38:49.853 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:38:49.853 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:38:49.853 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:38:49.853 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:38:49.853 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:38:49.853 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:38:49.853 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:38:49.853 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:38:49.853 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:38:49.853 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:38:49.853 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:38:49.853 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:38:49.853 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:38:49.853 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:38:49.854 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.854 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "85", 2021-02-10T11:38:49.854 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:38:49.854 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:38:49.854 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:38:49.854 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:38:49.854 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:38:49.854 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:38:49.854 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:38:49.854 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:38:49.854 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:38:49.854 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:38:49.854 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:38:49.855 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:38:49.855 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:38:49.855 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:38:49.855 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:38:49.855 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:38:49.855 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:38:49.855 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:38:49.855 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:38:49.855 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:38:49.855 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:38:49.855 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:38:49.855 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:38:49.855 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:38:49.856 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:38:49.856 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:38:49.856 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.856 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:38:49.856 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:38:49.856 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:38:49.856 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:38:49.856 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:38:49.856 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:38:49.856 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:38:49.856 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:38:49.856 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:38:49.856 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:38:49.856 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:38:49.857 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:38:49.857 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:38:49.857 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:38:49.857 INFO:teuthology.orchestra.run.gibba001.stdout: "data": "cephfs" 2021-02-10T11:38:49.857 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.857 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.857 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.857 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:38:49.857 INFO:teuthology.orchestra.run.gibba001.stdout: "osds": [ 2021-02-10T11:38:49.857 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.857 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:38:49.857 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5", 2021-02-10T11:38:49.857 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:38:49.858 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:38:49.858 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:38:49.858 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:38:49.858 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:38:49.858 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:38:49.858 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:38:49.858 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 82, 2021-02-10T11:38:49.858 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:38:49.858 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:38:49.858 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:38:49.858 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.858 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.858 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.859 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6802", 2021-02-10T11:38:49.859 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:38:49.859 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.859 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.859 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.859 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6805", 2021-02-10T11:38:49.859 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:38:49.859 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.859 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.859 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.859 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:38:49.859 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.859 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.860 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.860 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6807", 2021-02-10T11:38:49.860 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:38:49.860 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.860 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.860 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.860 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6809", 2021-02-10T11:38:49.860 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:38:49.860 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.860 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.860 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.860 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:38:49.860 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.861 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.861 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.861 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6817", 2021-02-10T11:38:49.861 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:38:49.861 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.861 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.861 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.861 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6820", 2021-02-10T11:38:49.861 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:38:49.861 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.861 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.861 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.861 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:38:49.861 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.862 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.862 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.862 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6812", 2021-02-10T11:38:49.862 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:38:49.862 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.862 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.862 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.862 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6814", 2021-02-10T11:38:49.862 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:38:49.862 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.862 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.862 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.862 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6805/25093", 2021-02-10T11:38:49.863 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6809/25093", 2021-02-10T11:38:49.863 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6820/25093", 2021-02-10T11:38:49.863 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6814/25093", 2021-02-10T11:38:49.863 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:38:49.863 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:38:49.863 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:38:49.863 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.863 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.863 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.863 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:38:49.863 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "ddc04ce3-b313-4707-8270-585391e38ecc", 2021-02-10T11:38:49.863 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:38:49.864 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:38:49.864 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:38:49.864 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:38:49.864 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:38:49.864 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:38:49.864 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:38:49.864 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 72, 2021-02-10T11:38:49.864 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:38:49.864 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:38:49.864 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:38:49.865 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.865 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.865 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.865 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6803", 2021-02-10T11:38:49.865 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:38:49.865 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.865 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.865 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.865 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6804", 2021-02-10T11:38:49.865 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:38:49.865 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.865 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.865 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.866 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:38:49.866 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.866 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.866 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.866 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6806", 2021-02-10T11:38:49.866 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:38:49.866 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.866 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.866 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.866 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6808", 2021-02-10T11:38:49.866 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:38:49.867 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.867 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.867 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.867 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:38:49.867 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.867 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.867 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.867 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6818", 2021-02-10T11:38:49.867 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:38:49.867 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.867 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.867 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.868 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6821", 2021-02-10T11:38:49.868 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:38:49.868 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.868 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.868 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.868 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:38:49.868 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.868 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.868 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.868 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6810", 2021-02-10T11:38:49.868 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:38:49.868 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.869 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.869 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.869 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6815", 2021-02-10T11:38:49.869 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:38:49.869 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.869 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.869 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.869 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6804/25092", 2021-02-10T11:38:49.869 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6808/25092", 2021-02-10T11:38:49.869 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6821/25092", 2021-02-10T11:38:49.869 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6815/25092", 2021-02-10T11:38:49.869 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:38:49.869 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:38:49.870 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:38:49.870 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.870 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.870 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.870 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:38:49.870 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3", 2021-02-10T11:38:49.870 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:38:49.870 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:38:49.870 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:38:49.870 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:38:49.870 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:38:49.870 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:38:49.870 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:38:49.871 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 81, 2021-02-10T11:38:49.871 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:38:49.871 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:38:49.871 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:38:49.871 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.871 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.871 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.871 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6811", 2021-02-10T11:38:49.871 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:38:49.871 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.871 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.871 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.871 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6813", 2021-02-10T11:38:49.872 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:38:49.872 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.872 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.872 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.872 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:38:49.872 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.872 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.872 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.872 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6816", 2021-02-10T11:38:49.872 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:38:49.872 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.872 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.872 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.872 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6819", 2021-02-10T11:38:49.873 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:38:49.873 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.873 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.873 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.873 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:38:49.873 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.873 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.873 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.873 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6825", 2021-02-10T11:38:49.873 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:38:49.873 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.873 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.873 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.874 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6827", 2021-02-10T11:38:49.874 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:38:49.874 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.874 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.874 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.874 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:38:49.874 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.874 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.874 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.874 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6822", 2021-02-10T11:38:49.874 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:38:49.874 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.874 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.874 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.875 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6823", 2021-02-10T11:38:49.875 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:38:49.875 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.875 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.875 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.875 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6813/25094", 2021-02-10T11:38:49.875 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6819/25094", 2021-02-10T11:38:49.875 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6827/25094", 2021-02-10T11:38:49.875 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6823/25094", 2021-02-10T11:38:49.875 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:38:49.875 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:38:49.875 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:38:49.875 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.875 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.876 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.876 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:38:49.876 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "0d9ef9b1-98aa-4d37-aa16-6305c7d46d52", 2021-02-10T11:38:49.876 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:38:49.876 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:38:49.876 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:38:49.876 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:38:49.876 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:38:49.876 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:38:49.876 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:38:49.876 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 81, 2021-02-10T11:38:49.876 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:38:49.876 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:38:49.876 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:38:49.877 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.877 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.877 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.877 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6824", 2021-02-10T11:38:49.877 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:38:49.877 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.877 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.877 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.877 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6826", 2021-02-10T11:38:49.877 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:38:49.877 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.877 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.877 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.878 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:38:49.878 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.878 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.878 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.878 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6828", 2021-02-10T11:38:49.878 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:38:49.878 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.878 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.878 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.878 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6829", 2021-02-10T11:38:49.878 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:38:49.878 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.879 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.879 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.879 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:38:49.879 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.879 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.879 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.879 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6832", 2021-02-10T11:38:49.879 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:38:49.879 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.879 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.879 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.879 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6833", 2021-02-10T11:38:49.879 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:38:49.879 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.880 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.880 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.880 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:38:49.880 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.880 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.880 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.880 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6830", 2021-02-10T11:38:49.880 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:38:49.880 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.880 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.880 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.880 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6831", 2021-02-10T11:38:49.880 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:38:49.880 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.881 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.881 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.881 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6826/25095", 2021-02-10T11:38:49.881 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6829/25095", 2021-02-10T11:38:49.881 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6833/25095", 2021-02-10T11:38:49.881 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6831/25095", 2021-02-10T11:38:49.881 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:38:49.881 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:38:49.881 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:38:49.881 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.881 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.881 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.881 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:38:49.882 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "36742c21-bfeb-4e11-bce9-8e745378b605", 2021-02-10T11:38:49.882 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:38:49.882 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:38:49.882 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:38:49.882 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:38:49.882 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:38:49.882 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:38:49.882 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:38:49.882 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 82, 2021-02-10T11:38:49.882 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:38:49.882 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:38:49.882 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:38:49.882 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.883 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.883 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.883 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6816", 2021-02-10T11:38:49.883 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:38:49.883 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.883 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.883 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.883 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6818", 2021-02-10T11:38:49.883 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:38:49.883 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.883 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.883 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.883 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:38:49.883 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.884 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.884 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.884 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6820", 2021-02-10T11:38:49.884 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:38:49.884 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.884 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.884 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.884 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6822", 2021-02-10T11:38:49.884 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:38:49.884 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.884 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.884 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.884 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:38:49.885 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.885 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.885 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.885 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6828", 2021-02-10T11:38:49.885 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:38:49.885 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.885 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.885 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.885 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6830", 2021-02-10T11:38:49.885 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:38:49.885 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.885 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.885 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.886 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:38:49.886 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.886 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.886 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.886 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6824", 2021-02-10T11:38:49.886 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:38:49.886 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.886 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.886 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.886 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6826", 2021-02-10T11:38:49.886 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:38:49.886 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.886 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.887 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.887 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6818/36380", 2021-02-10T11:38:49.887 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6822/36380", 2021-02-10T11:38:49.887 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6830/36380", 2021-02-10T11:38:49.887 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6826/36380", 2021-02-10T11:38:49.887 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:38:49.887 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:38:49.887 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:38:49.887 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.887 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.887 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.887 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:38:49.887 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "88e694e8-fe98-4d1b-88c8-8c9907ece60b", 2021-02-10T11:38:49.888 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:38:49.888 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:38:49.888 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:38:49.888 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:38:49.888 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:38:49.888 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:38:49.888 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:38:49.888 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 81, 2021-02-10T11:38:49.888 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:38:49.888 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:38:49.888 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:38:49.888 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.888 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.889 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.889 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6808", 2021-02-10T11:38:49.889 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:38:49.889 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.889 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.889 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.889 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6809", 2021-02-10T11:38:49.889 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:38:49.889 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.889 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.889 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.889 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:38:49.889 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.889 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.890 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.890 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6810", 2021-02-10T11:38:49.890 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:38:49.890 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.890 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.890 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.890 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6811", 2021-02-10T11:38:49.890 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:38:49.890 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.890 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.890 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.891 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:38:49.891 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.891 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.891 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.891 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6814", 2021-02-10T11:38:49.891 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:38:49.891 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.891 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.891 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.891 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6815", 2021-02-10T11:38:49.891 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:38:49.891 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.891 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.892 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.892 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:38:49.892 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.892 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.892 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.892 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6812", 2021-02-10T11:38:49.892 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:38:49.892 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.892 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.892 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.892 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6813", 2021-02-10T11:38:49.892 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:38:49.893 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.893 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.893 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.893 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6809/36383", 2021-02-10T11:38:49.893 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6811/36383", 2021-02-10T11:38:49.893 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6815/36383", 2021-02-10T11:38:49.893 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6813/36383", 2021-02-10T11:38:49.893 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:38:49.893 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:38:49.893 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:38:49.893 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.893 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.893 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.894 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:38:49.894 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "87a13c6c-cf85-4aaf-916b-e73913484a49", 2021-02-10T11:38:49.894 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:38:49.894 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:38:49.894 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:38:49.894 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:38:49.894 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:38:49.894 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:38:49.894 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:38:49.894 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 82, 2021-02-10T11:38:49.894 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:38:49.894 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:38:49.894 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:38:49.895 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.895 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.895 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.895 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6800", 2021-02-10T11:38:49.895 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:38:49.895 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.895 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.895 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.895 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6801", 2021-02-10T11:38:49.895 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:38:49.895 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.895 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.895 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.895 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:38:49.896 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.896 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.896 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.896 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6802", 2021-02-10T11:38:49.896 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:38:49.896 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.896 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.896 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.896 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6803", 2021-02-10T11:38:49.896 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:38:49.896 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.896 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.896 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.897 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:38:49.897 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.897 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.897 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.897 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6806", 2021-02-10T11:38:49.897 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:38:49.897 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.897 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.897 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.897 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6807", 2021-02-10T11:38:49.897 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:38:49.897 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.897 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.898 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.898 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:38:49.898 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.898 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.898 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.898 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6804", 2021-02-10T11:38:49.898 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:38:49.898 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.898 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.898 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.898 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6805", 2021-02-10T11:38:49.898 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:38:49.899 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.899 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.899 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.899 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6801/36381", 2021-02-10T11:38:49.899 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6803/36381", 2021-02-10T11:38:49.899 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6807/36381", 2021-02-10T11:38:49.899 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6805/36381", 2021-02-10T11:38:49.899 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:38:49.899 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:38:49.899 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:38:49.899 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.899 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.899 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.900 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:38:49.900 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "7b90823f-854c-4592-a95b-a7fa98f93254", 2021-02-10T11:38:49.900 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:38:49.900 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:38:49.900 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:38:49.900 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:38:49.900 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:38:49.900 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:38:49.900 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:38:49.900 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 83, 2021-02-10T11:38:49.900 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:38:49.900 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:38:49.900 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:38:49.901 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.901 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.901 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.901 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6817", 2021-02-10T11:38:49.901 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:38:49.901 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.901 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.901 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.901 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6819", 2021-02-10T11:38:49.901 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:38:49.901 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.901 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.901 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.901 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:38:49.902 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.902 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.902 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.902 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6821", 2021-02-10T11:38:49.902 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:38:49.902 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.902 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.902 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.902 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6823", 2021-02-10T11:38:49.902 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:38:49.902 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.902 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.902 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.903 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:38:49.903 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.903 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.903 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.903 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6829", 2021-02-10T11:38:49.903 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:38:49.903 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.903 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.903 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.903 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6831", 2021-02-10T11:38:49.903 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:38:49.903 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.903 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.904 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.904 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:38:49.904 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:38:49.904 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.904 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:38:49.904 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6825", 2021-02-10T11:38:49.904 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:38:49.904 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.904 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.904 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:38:49.904 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6827", 2021-02-10T11:38:49.904 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:38:49.904 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.904 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.905 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.905 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6819/36382", 2021-02-10T11:38:49.905 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6823/36382", 2021-02-10T11:38:49.905 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6831/36382", 2021-02-10T11:38:49.905 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6827/36382", 2021-02-10T11:38:49.905 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:38:49.905 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:38:49.905 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:38:49.905 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:49.905 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.905 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:38:49.905 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_xinfo": [ 2021-02-10T11:38:49.906 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.906 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:38:49.906 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:38:49.906 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:38:49.906 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:38:49.906 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:38:49.906 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:38:49.906 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.906 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.906 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:38:49.906 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:38:49.906 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:38:49.906 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:38:49.907 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:38:49.907 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:38:49.907 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.907 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.907 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:38:49.907 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:38:49.907 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:38:49.907 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:38:49.907 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:38:49.907 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:38:49.907 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.907 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.907 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:38:49.907 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:38:49.908 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:38:49.908 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:38:49.908 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:38:49.908 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:38:49.908 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.908 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.908 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:38:49.908 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:38:49.908 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:38:49.908 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:38:49.908 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:38:49.908 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:38:49.908 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.908 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.909 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:38:49.909 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:38:49.909 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:38:49.909 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:38:49.909 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:38:49.909 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:38:49.909 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.909 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.909 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:38:49.909 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:38:49.909 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:38:49.909 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:38:49.909 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:38:49.910 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:38:49.910 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.910 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:49.910 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:38:49.910 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:38:49.910 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:38:49.910 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:38:49.910 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:38:49.910 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:38:49.910 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.910 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:38:49.910 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap": [], 2021-02-10T11:38:49.910 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap_items": [], 2021-02-10T11:38:49.911 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_temp": [], 2021-02-10T11:38:49.911 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_temp": [], 2021-02-10T11:38:49.911 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklist": {}, 2021-02-10T11:38:49.911 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profiles": { 2021-02-10T11:38:49.911 INFO:teuthology.orchestra.run.gibba001.stdout: "default": { 2021-02-10T11:38:49.911 INFO:teuthology.orchestra.run.gibba001.stdout: "crush-failure-domain": "osd", 2021-02-10T11:38:49.911 INFO:teuthology.orchestra.run.gibba001.stdout: "k": "2", 2021-02-10T11:38:49.911 INFO:teuthology.orchestra.run.gibba001.stdout: "m": "1", 2021-02-10T11:38:49.911 INFO:teuthology.orchestra.run.gibba001.stdout: "plugin": "jerasure", 2021-02-10T11:38:49.911 INFO:teuthology.orchestra.run.gibba001.stdout: "ruleset-failure-domain": "osd", 2021-02-10T11:38:49.911 INFO:teuthology.orchestra.run.gibba001.stdout: "technique": "reed_sol_van" 2021-02-10T11:38:49.911 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:49.911 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:49.912 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps_queue": [], 2021-02-10T11:38:49.912 INFO:teuthology.orchestra.run.gibba001.stdout: "new_removed_snaps": [], 2021-02-10T11:38:49.912 INFO:teuthology.orchestra.run.gibba001.stdout: "new_purged_snaps": [], 2021-02-10T11:38:49.912 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_node_flags": {}, 2021-02-10T11:38:49.912 INFO:teuthology.orchestra.run.gibba001.stdout: "device_class_flags": {} 2021-02-10T11:38:49.912 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:38:49.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.834 7f46bbb2d700 1 -- 172.21.2.101:0/241713191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f469c0186d0 msgr2=0x7f469c01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:49.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.834 7f46bbb2d700 1 --2- 172.21.2.101:0/241713191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f469c0186d0 0x7f469c01abd0 crc :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:49.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.834 7f46bbb2d700 1 -- 172.21.2.101:0/241713191 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f46b41b8ad0 msgr2=0x7f46b4187e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:49.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.834 7f46bbb2d700 1 --2- 172.21.2.101:0/241713191 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f46b41b8ad0 0x7f46b4187e50 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 rx=0x7f46b000b5c0 tx=0x7f46b000b960).stop 2021-02-10T11:38:49.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.835 7f46bbb2d700 1 -- 172.21.2.101:0/241713191 shutdown_connections 2021-02-10T11:38:49.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.835 7f46bbb2d700 1 --2- 172.21.2.101:0/241713191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f469c0186d0 0x7f469c01abd0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:49.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.835 7f46bbb2d700 1 --2- 172.21.2.101:0/241713191 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f46b41b8ed0 0x7f46b40b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:49.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.835 7f46bbb2d700 1 --2- 172.21.2.101:0/241713191 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f46b40b2110 0x7f46b41b7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:49.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.835 7f46bbb2d700 1 --2- 172.21.2.101:0/241713191 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f46b41b8ad0 0x7f46b4187e50 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:49.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.835 7f46bbb2d700 1 -- 172.21.2.101:0/241713191 >> 172.21.2.101:0/241713191 conn(0x7f46b40ed810 msgr2=0x7f46b4094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:49.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.835 7f46bbb2d700 1 -- 172.21.2.101:0/241713191 shutdown_connections 2021-02-10T11:38:49.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:49.835 7f46bbb2d700 1 -- 172.21.2.101:0/241713191 wait complete. 2021-02-10T11:38:49.915 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:38:50.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.322 7ff2b8cce700 1 Processor -- start 2021-02-10T11:38:50.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.322 7ff2b8cce700 1 -- start start 2021-02-10T11:38:50.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.323 7ff2b8cce700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff2b40d1870 0x7ff2b41b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:50.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.323 7ff2b8cce700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2b40f0650 0x7ff2b40cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:50.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.323 7ff2b8cce700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b40f0b10 con 0x7ff2b41b0a10 2021-02-10T11:38:50.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.323 7ff2b8cce700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b40e5ed0 con 0x7ff2b40d1870 2021-02-10T11:38:50.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.323 7ff2b8cce700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b40d1c70 con 0x7ff2b40f0650 2021-02-10T11:38:50.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.324 7ff2b32e9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2b40f0650 0x7ff2b40cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:50.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.324 7ff2b2ae8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff2b40d1870 0x7ff2b41b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:50.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.324 7ff2b32e9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2b40f0650 0x7ff2b40cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:47734/0 (socket says 172.21.2.101:47734) 2021-02-10T11:38:50.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.324 7ff2b2ae8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff2b40d1870 0x7ff2b41b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:37252/0 (socket says 172.21.2.101:37252) 2021-02-10T11:38:50.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.324 7ff2b32e9700 1 -- 172.21.2.101:0/3477509697 learned_addr learned my addr 172.21.2.101:0/3477509697 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:50.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.325 7ff2b2ae8700 1 -- 172.21.2.101:0/3477509697 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2b40f0650 msgr2=0x7ff2b40cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:50.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.325 7ff2b2ae8700 1 --2- 172.21.2.101:0/3477509697 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2b40f0650 0x7ff2b40cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:50.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.325 7ff2b2ae8700 1 -- 172.21.2.101:0/3477509697 >> v1:172.21.2.108:6789/0 conn(0x7ff2b41b0a10 legacy=0x7ff2b40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:50.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.325 7ff2b2ae8700 1 -- 172.21.2.101:0/3477509697 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b40d1e90 con 0x7ff2b40d1870 2021-02-10T11:38:50.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.325 7ff2b1ae6700 1 -- 172.21.2.101:0/3477509697 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090935921 0 0) 0x7ff29c000ff0 con 0x7ff2b41b0a10 2021-02-10T11:38:50.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.325 7ff2b32e9700 1 --2- 172.21.2.101:0/3477509697 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2b40f0650 0x7ff2b40cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:38:50.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.325 7ff2b2ae8700 1 --2- 172.21.2.101:0/3477509697 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff2b40d1870 0x7ff2b41b03f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7ff2a4006650 tx=0x7ff2a4009a80).ready entity=mon.1 client_cookie=b7e26c6504edce01 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:50.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.325 7ff2b1ae6700 1 -- 172.21.2.101:0/3477509697 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff2a40009b0 con 0x7ff2b40d1870 2021-02-10T11:38:50.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.325 7ff2b1ae6700 1 -- 172.21.2.101:0/3477509697 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a400c0c0 con 0x7ff2b40d1870 2021-02-10T11:38:50.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.325 7ff2b1ae6700 1 -- 172.21.2.101:0/3477509697 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff2a400c5e0 con 0x7ff2b40d1870 2021-02-10T11:38:50.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.326 7ff2b8cce700 1 -- 172.21.2.101:0/3477509697 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff2b40d1870 msgr2=0x7ff2b41b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:50.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.326 7ff2b8cce700 1 --2- 172.21.2.101:0/3477509697 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff2b40d1870 0x7ff2b41b03f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7ff2a4006650 tx=0x7ff2a4009a80).stop 2021-02-10T11:38:50.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.326 7ff2b8cce700 1 -- 172.21.2.101:0/3477509697 shutdown_connections 2021-02-10T11:38:50.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.326 7ff2b8cce700 1 --2- 172.21.2.101:0/3477509697 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2b40f0650 0x7ff2b40cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:50.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.326 7ff2b8cce700 1 --2- 172.21.2.101:0/3477509697 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff2b40d1870 0x7ff2b41b03f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:50.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.326 7ff2b8cce700 1 -- 172.21.2.101:0/3477509697 >> 172.21.2.101:0/3477509697 conn(0x7ff2b40ce450 msgr2=0x7ff2b40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:50.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.326 7ff2b8cce700 1 -- 172.21.2.101:0/3477509697 shutdown_connections 2021-02-10T11:38:50.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.326 7ff2b8cce700 1 -- 172.21.2.101:0/3477509697 wait complete. 2021-02-10T11:38:50.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.341 7ff2b8cce700 1 Processor -- start 2021-02-10T11:38:50.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.341 7ff2b8cce700 1 -- start start 2021-02-10T11:38:50.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.341 7ff2b8cce700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff2b41942b0 0x7ff2b40b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:50.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.341 7ff2b8cce700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2b4195070 0x7ff2b4195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:50.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.341 7ff2b8cce700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2b40bb8e0 0x7ff2b40bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:50.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.341 7ff2b8cce700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b41946b0 con 0x7ff2b40bb8e0 2021-02-10T11:38:50.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.341 7ff2b8cce700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b40b3b70 con 0x7ff2b41942b0 2021-02-10T11:38:50.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.341 7ff2b8cce700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b40b3d90 con 0x7ff2b4195070 2021-02-10T11:38:50.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.342 7ff2b32e9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2b40bb8e0 0x7ff2b40bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:50.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.342 7ff2b32e9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2b40bb8e0 0x7ff2b40bbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55008/0 (socket says 172.21.2.101:55008) 2021-02-10T11:38:50.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.342 7ff2b32e9700 1 -- 172.21.2.101:0/4232203893 learned_addr learned my addr 172.21.2.101:0/4232203893 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:50.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.342 7ff2b32e9700 1 -- 172.21.2.101:0/4232203893 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2b4195070 msgr2=0x7ff2b4195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:50.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.342 7ff2b32e9700 1 --2- 172.21.2.101:0/4232203893 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2b4195070 0x7ff2b4195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:50.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.342 7ff2b32e9700 1 -- 172.21.2.101:0/4232203893 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff2b41942b0 msgr2=0x7ff2b40b3550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:38:50.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.342 7ff2b32e9700 1 --2- 172.21.2.101:0/4232203893 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff2b41942b0 0x7ff2b40b3550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:50.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.342 7ff2b32e9700 1 -- 172.21.2.101:0/4232203893 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b41a49e0 con 0x7ff2b40bb8e0 2021-02-10T11:38:50.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.342 7ff2b32e9700 1 --2- 172.21.2.101:0/4232203893 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2b40bb8e0 0x7ff2b40bbce0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 rx=0x7ff2ac009940 tx=0x7ff2ac00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:50.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.342 7ff2ab7fe700 1 -- 172.21.2.101:0/4232203893 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff2ac00c2f0 con 0x7ff2b40bb8e0 2021-02-10T11:38:50.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.342 7ff2ab7fe700 1 -- 172.21.2.101:0/4232203893 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ac006790 con 0x7ff2b40bb8e0 2021-02-10T11:38:50.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.342 7ff2ab7fe700 1 -- 172.21.2.101:0/4232203893 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff2ac00c760 con 0x7ff2b40bb8e0 2021-02-10T11:38:50.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.342 7ff2b8cce700 1 -- 172.21.2.101:0/4232203893 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b41a49e0 con 0x7ff2b40bb8e0 2021-02-10T11:38:50.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.342 7ff2b8cce700 1 -- 172.21.2.101:0/4232203893 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b40bc5f0 con 0x7ff2b40bb8e0 2021-02-10T11:38:50.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.343 7ff2ab7fe700 1 -- 172.21.2.101:0/4232203893 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff2ac0011f0 con 0x7ff2b40bb8e0 2021-02-10T11:38:50.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.343 7ff2ab7fe700 1 --2- 172.21.2.101:0/4232203893 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2980186c0 0x7ff29801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:50.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.343 7ff2ab7fe700 1 -- 172.21.2.101:0/4232203893 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff2ac032920 con 0x7ff2b40bb8e0 2021-02-10T11:38:50.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.343 7ff2b2ae8700 1 --2- 172.21.2.101:0/4232203893 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2980186c0 0x7ff29801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:50.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.344 7ff2b2ae8700 1 --2- 172.21.2.101:0/4232203893 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2980186c0 0x7ff29801abc0 crc :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:50.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.373 7ff2b8cce700 1 -- 172.21.2.101:0/4232203893 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b40bc5f0 con 0x7ff2b40bb8e0 2021-02-10T11:38:50.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.375 7ff2ab7fe700 1 -- 172.21.2.101:0/4232203893 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff2ac000f40 con 0x7ff2b40bb8e0 2021-02-10T11:38:50.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.503 7ff2b8cce700 1 -- 172.21.2.101:0/4232203893 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2b40bc5f0 con 0x7ff2b40bb8e0 2021-02-10T11:38:50.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.504 7ff2ab7fe700 1 -- 172.21.2.101:0/4232203893 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+2101 (secure 0 0 0) 0x7ff2ac0011f0 con 0x7ff2b40bb8e0 2021-02-10T11:38:50.508 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:38:50.509 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9360,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2529021396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2529021396},{"type":"v1","addr":"172.21.2.101:6835","nonce":2529021396}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:56.648753","modified":"2021-02-10 11:36:58.653009","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6005":{"gid":6005,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2363068042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2363068042},{"type":"v1","addr":"172.21.2.108:6833","nonce":2363068042}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-02-10T11:38:50.509 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 63 2021-02-10T11:38:50.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.506 7ff2b8cce700 1 -- 172.21.2.101:0/4232203893 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2980186c0 msgr2=0x7ff29801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:50.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.506 7ff2b8cce700 1 --2- 172.21.2.101:0/4232203893 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2980186c0 0x7ff29801abc0 crc :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:50.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.506 7ff2b8cce700 1 -- 172.21.2.101:0/4232203893 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2b40bb8e0 msgr2=0x7ff2b40bbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:50.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.506 7ff2b8cce700 1 --2- 172.21.2.101:0/4232203893 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2b40bb8e0 0x7ff2b40bbce0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 rx=0x7ff2ac009940 tx=0x7ff2ac00bcf0).stop 2021-02-10T11:38:50.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.506 7ff2b8cce700 1 -- 172.21.2.101:0/4232203893 shutdown_connections 2021-02-10T11:38:50.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.506 7ff2b8cce700 1 --2- 172.21.2.101:0/4232203893 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2980186c0 0x7ff29801abc0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:50.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.506 7ff2b8cce700 1 --2- 172.21.2.101:0/4232203893 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2b4195070 0x7ff2b4195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:50.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.506 7ff2b8cce700 1 --2- 172.21.2.101:0/4232203893 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2b40bb8e0 0x7ff2b40bbce0 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:50.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.506 7ff2b8cce700 1 --2- 172.21.2.101:0/4232203893 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff2b41942b0 0x7ff2b40b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:50.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.506 7ff2b8cce700 1 -- 172.21.2.101:0/4232203893 >> 172.21.2.101:0/4232203893 conn(0x7ff2b40b3040 msgr2=0x7ff2b40c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:50.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.506 7ff2b8cce700 1 -- 172.21.2.101:0/4232203893 shutdown_connections 2021-02-10T11:38:50.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.506 7ff2b8cce700 1 -- 172.21.2.101:0/4232203893 wait complete. 2021-02-10T11:38:50.519 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs fail cephfs 2021-02-10T11:38:50.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.926 7fef0699d700 1 Processor -- start 2021-02-10T11:38:50.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.926 7fef0699d700 1 -- start start 2021-02-10T11:38:50.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.926 7fef0699d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef000d1870 0x7fef001b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:50.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.926 7fef0699d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fef001b0a10 0x7fef000cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:50.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.926 7fef0699d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef000f0b10 con 0x7fef000d1c70 2021-02-10T11:38:50.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.926 7fef0699d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef000fe230 con 0x7fef001b0a10 2021-02-10T11:38:50.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.926 7fef0699d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef000e5ed0 con 0x7fef000d1870 2021-02-10T11:38:50.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.927 7feeff7fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fef001b0a10 0x7fef000cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:50.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.927 7feefffff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef000d1870 0x7fef001b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:50.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.927 7feefffff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef000d1870 0x7fef001b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:47744/0 (socket says 172.21.2.101:47744) 2021-02-10T11:38:50.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.927 7feeff7fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fef001b0a10 0x7fef000cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:37268/0 (socket says 172.21.2.101:37268) 2021-02-10T11:38:50.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.927 7feeff7fe700 1 -- 172.21.2.101:0/1627365560 learned_addr learned my addr 172.21.2.101:0/1627365560 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:50.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.927 7feeff7fe700 1 -- 172.21.2.101:0/1627365560 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef000d1870 msgr2=0x7fef001b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:50.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.927 7feeff7fe700 1 --2- 172.21.2.101:0/1627365560 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef000d1870 0x7fef001b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:50.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.927 7feeff7fe700 1 -- 172.21.2.101:0/1627365560 >> v1:172.21.2.108:6789/0 conn(0x7fef000d1c70 legacy=0x7fef000cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:50.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.927 7feeff7fe700 1 -- 172.21.2.101:0/1627365560 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef000f0650 con 0x7fef001b0a10 2021-02-10T11:38:50.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.927 7feeff7fe700 1 --2- 172.21.2.101:0/1627365560 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef001b0a10 0x7fef000cf330 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7feef4006700 tx=0x7feef4009b80).ready entity=mon.1 client_cookie=4130ce2fd19f31b2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:50.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.928 7feefeffd700 1 -- 172.21.2.101:0/1627365560 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feef400c0c0 con 0x7fef001b0a10 2021-02-10T11:38:50.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.928 7feefeffd700 1 -- 172.21.2.101:0/1627365560 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef400c440 con 0x7fef001b0a10 2021-02-10T11:38:50.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.928 7feefeffd700 1 -- 172.21.2.101:0/1627365560 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feef400c960 con 0x7fef001b0a10 2021-02-10T11:38:50.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.928 7fef0699d700 1 -- 172.21.2.101:0/1627365560 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef001b0a10 msgr2=0x7fef000cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:50.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.928 7fef0699d700 1 --2- 172.21.2.101:0/1627365560 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef001b0a10 0x7fef000cf330 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7feef4006700 tx=0x7feef4009b80).stop 2021-02-10T11:38:50.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.928 7fef0699d700 1 -- 172.21.2.101:0/1627365560 shutdown_connections 2021-02-10T11:38:50.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.928 7fef0699d700 1 --2- 172.21.2.101:0/1627365560 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef001b0a10 0x7fef000cf330 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:50.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.928 7fef0699d700 1 --2- 172.21.2.101:0/1627365560 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef000d1870 0x7fef001b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:50.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.928 7fef0699d700 1 -- 172.21.2.101:0/1627365560 >> 172.21.2.101:0/1627365560 conn(0x7fef000ce450 msgr2=0x7fef000d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:50.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.928 7fef0699d700 1 -- 172.21.2.101:0/1627365560 shutdown_connections 2021-02-10T11:38:50.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.928 7fef0699d700 1 -- 172.21.2.101:0/1627365560 wait complete. 2021-02-10T11:38:50.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.943 7fef0699d700 1 Processor -- start 2021-02-10T11:38:50.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.943 7fef0699d700 1 -- start start 2021-02-10T11:38:50.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.943 7fef0699d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef001a06a0 0x7fef0009f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:50.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.943 7fef0699d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef0009f9a0 0x7fef0008b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:50.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.943 7fef0699d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef0008bb80 0x7fef0008bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:50.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.943 7fef0699d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0008c780 con 0x7fef0008bb80 2021-02-10T11:38:50.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.943 7fef0699d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0009fda0 con 0x7fef0009f9a0 2021-02-10T11:38:50.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.943 7fef0699d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0008cd00 con 0x7fef001a06a0 2021-02-10T11:38:50.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.943 7fef04fba700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef0008bb80 0x7fef0008bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:50.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.943 7fef04fba700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef0008bb80 0x7fef0008bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55018/0 (socket says 172.21.2.101:55018) 2021-02-10T11:38:50.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.944 7fef04fba700 1 -- 172.21.2.101:0/4254685093 learned_addr learned my addr 172.21.2.101:0/4254685093 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:50.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.944 7feefffff700 1 --2- 172.21.2.101:0/4254685093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef001a06a0 0x7fef0009f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:50.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.944 7fef04fba700 1 -- 172.21.2.101:0/4254685093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef001a06a0 msgr2=0x7fef0009f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:50.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.944 7fef04fba700 1 --2- 172.21.2.101:0/4254685093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef001a06a0 0x7fef0009f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:50.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.944 7fef04fba700 1 -- 172.21.2.101:0/4254685093 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef0009f9a0 msgr2=0x7fef0008b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:50.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.944 7fef04fba700 1 --2- 172.21.2.101:0/4254685093 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef0009f9a0 0x7fef0008b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:50.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.944 7fef04fba700 1 -- 172.21.2.101:0/4254685093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0008cf20 con 0x7fef0008bb80 2021-02-10T11:38:50.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.944 7fef04fba700 1 --2- 172.21.2.101:0/4254685093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef0008bb80 0x7fef0008bf80 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 rx=0x7feef8009940 tx=0x7feef800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:50.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.944 7feefcff9700 1 -- 172.21.2.101:0/4254685093 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feef8000ce0 con 0x7fef0008bb80 2021-02-10T11:38:50.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.944 7feefcff9700 1 -- 172.21.2.101:0/4254685093 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef800c2f0 con 0x7fef0008bb80 2021-02-10T11:38:50.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.944 7feefcff9700 1 -- 172.21.2.101:0/4254685093 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feef800c810 con 0x7fef0008bb80 2021-02-10T11:38:50.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.944 7fef0699d700 1 -- 172.21.2.101:0/4254685093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef0008cf20 con 0x7fef0008bb80 2021-02-10T11:38:50.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.944 7fef0699d700 1 -- 172.21.2.101:0/4254685093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef0008d4b0 con 0x7fef0008bb80 2021-02-10T11:38:50.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.945 7feefcff9700 1 -- 172.21.2.101:0/4254685093 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7feef8001220 con 0x7fef0008bb80 2021-02-10T11:38:50.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.945 7feefcff9700 1 --2- 172.21.2.101:0/4254685093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feee00186c0 0x7feee001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:50.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.945 7feefcff9700 1 -- 172.21.2.101:0/4254685093 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5573+0+0 (secure 0 0 0) 0x7feef8032900 con 0x7fef0008bb80 2021-02-10T11:38:50.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.945 7feefffff700 1 --2- 172.21.2.101:0/4254685093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feee00186c0 0x7feee001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:50.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.946 7feefffff700 1 --2- 172.21.2.101:0/4254685093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feee00186c0 0x7feee001abc0 crc :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:50.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.976 7fef0699d700 1 -- 172.21.2.101:0/4254685093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef000ebc50 con 0x7fef0008bb80 2021-02-10T11:38:50.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:50.978 7feefcff9700 1 -- 172.21.2.101:0/4254685093 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7feef8000cc0 con 0x7fef0008bb80 2021-02-10T11:38:51.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:51.106 7fef0699d700 1 -- 172.21.2.101:0/4254685093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fef000f0b10 con 0x7fef0008bb80 2021-02-10T11:38:51.896 INFO:tasks.ceph.mon.a.gibba001.stderr:2021-02-10 11:38:51.892 7f974d04d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-02-10T11:38:51.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:51.898 7feefcff9700 1 -- 172.21.2.101:0/4254685093 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v64) v1 ==== 161+0+0 (secure 0 0 0) 0x7feef801f860 con 0x7fef0008bb80 2021-02-10T11:38:51.930 INFO:teuthology.orchestra.run.gibba001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-02-10T11:38:51.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:51.927 7fef0699d700 1 -- 172.21.2.101:0/4254685093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feee00186c0 msgr2=0x7feee001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:51.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:51.927 7fef0699d700 1 --2- 172.21.2.101:0/4254685093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feee00186c0 0x7feee001abc0 crc :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:51.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:51.927 7fef0699d700 1 -- 172.21.2.101:0/4254685093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef0008bb80 msgr2=0x7fef0008bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:51.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:51.927 7fef0699d700 1 --2- 172.21.2.101:0/4254685093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef0008bb80 0x7fef0008bf80 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 rx=0x7feef8009940 tx=0x7feef800bcf0).stop 2021-02-10T11:38:51.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:51.927 7fef0699d700 1 -- 172.21.2.101:0/4254685093 shutdown_connections 2021-02-10T11:38:51.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:51.927 7fef0699d700 1 --2- 172.21.2.101:0/4254685093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef0008bb80 0x7fef0008bf80 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:51.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:51.927 7fef0699d700 1 --2- 172.21.2.101:0/4254685093 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef0009f9a0 0x7fef0008b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:51.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:51.927 7fef0699d700 1 --2- 172.21.2.101:0/4254685093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feee00186c0 0x7feee001abc0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:51.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:51.927 7fef0699d700 1 --2- 172.21.2.101:0/4254685093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef001a06a0 0x7fef0009f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:51.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:51.927 7fef0699d700 1 -- 172.21.2.101:0/4254685093 >> 172.21.2.101:0/4254685093 conn(0x7fef000b83b0 msgr2=0x7fef000a6850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:51.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:51.928 7fef0699d700 1 -- 172.21.2.101:0/4254685093 shutdown_connections 2021-02-10T11:38:51.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:51.928 7fef0699d700 1 -- 172.21.2.101:0/4254685093 wait complete. 2021-02-10T11:38:51.940 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:38:51.950 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:38:52.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.347 7f063d085700 1 Processor -- start 2021-02-10T11:38:52.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.347 7f063d085700 1 -- start start 2021-02-10T11:38:52.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.348 7f063d085700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f06380d0340 0x7f06380ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:52.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.348 7f063d085700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06380f0b10 con 0x7f06380d1c70 2021-02-10T11:38:52.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.348 7f063d085700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06381b0be0 con 0x7f06380d1870 2021-02-10T11:38:52.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.348 7f063d085700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06380e5ed0 con 0x7f06380d0340 2021-02-10T11:38:52.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.348 7f063659c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f06380d0340 0x7f06380ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:52.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.348 7f063659c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f06380d0340 0x7f06380ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55028/0 (socket says 172.21.2.101:55028) 2021-02-10T11:38:52.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.348 7f063659c700 1 -- 172.21.2.101:0/2566802684 learned_addr learned my addr 172.21.2.101:0/2566802684 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:52.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.349 7f063659c700 1 -- 172.21.2.101:0/2566802684 >> v1:172.21.2.108:6789/0 conn(0x7f06380d1870 legacy=0x7f06381b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:52.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.349 7f063659c700 1 -- 172.21.2.101:0/2566802684 >> v1:172.21.2.101:6789/0 conn(0x7f06380d1c70 legacy=0x7f06380efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:52.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.349 7f0635d9b700 1 -- 172.21.2.101:0/2566802684 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1690845977 0 0) 0x7f0630000f90 con 0x7f06380d1c70 2021-02-10T11:38:52.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.349 7f063659c700 1 -- 172.21.2.101:0/2566802684 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06380f0450 con 0x7f06380d0340 2021-02-10T11:38:52.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.349 7f063659c700 1 --2- 172.21.2.101:0/2566802684 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06380d0340 0x7f06380ef640 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 rx=0x7f0628006700 tx=0x7f0628009b80).ready entity=mon.0 client_cookie=68aa130c48ea5912 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:52.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.349 7f0635d9b700 1 -- 172.21.2.101:0/2566802684 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f062800c0c0 con 0x7f06380d0340 2021-02-10T11:38:52.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.349 7f0635d9b700 1 -- 172.21.2.101:0/2566802684 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062800c440 con 0x7f06380d0340 2021-02-10T11:38:52.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.349 7f0635d9b700 1 -- 172.21.2.101:0/2566802684 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f062800c8a0 con 0x7f06380d0340 2021-02-10T11:38:52.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.349 7f063d085700 1 -- 172.21.2.101:0/2566802684 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06380d0340 msgr2=0x7f06380ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:52.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.349 7f063d085700 1 --2- 172.21.2.101:0/2566802684 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06380d0340 0x7f06380ef640 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 rx=0x7f0628006700 tx=0x7f0628009b80).stop 2021-02-10T11:38:52.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.350 7f063d085700 1 -- 172.21.2.101:0/2566802684 shutdown_connections 2021-02-10T11:38:52.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.350 7f063d085700 1 --2- 172.21.2.101:0/2566802684 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06380d0340 0x7f06380ef640 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:52.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.350 7f063d085700 1 -- 172.21.2.101:0/2566802684 >> 172.21.2.101:0/2566802684 conn(0x7f06380ce450 msgr2=0x7f06380d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:52.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.350 7f063d085700 1 -- 172.21.2.101:0/2566802684 shutdown_connections 2021-02-10T11:38:52.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.350 7f063d085700 1 -- 172.21.2.101:0/2566802684 wait complete. 2021-02-10T11:38:52.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.365 7f063d085700 1 Processor -- start 2021-02-10T11:38:52.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.365 7f063d085700 1 -- start start 2021-02-10T11:38:52.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.365 7f063d085700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06380b4e00 0x7f06380a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:52.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.365 7f063d085700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f06380a3a60 0x7f06380a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:52.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.365 7f063d085700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0638094750 0x7f0638094b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:52.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.365 7f063d085700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06380a4550 con 0x7f06380b4e00 2021-02-10T11:38:52.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.365 7f063d085700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063818bf30 con 0x7f06380a3a60 2021-02-10T11:38:52.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.365 7f063d085700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063818ba60 con 0x7f0638094750 2021-02-10T11:38:52.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.365 7f0636d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06380b4e00 0x7f06380a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:52.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.365 7f0636d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06380b4e00 0x7f06380a0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55032/0 (socket says 172.21.2.101:55032) 2021-02-10T11:38:52.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.365 7f0636d9d700 1 -- 172.21.2.101:0/1071256850 learned_addr learned my addr 172.21.2.101:0/1071256850 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:52.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.366 7f0636d9d700 1 -- 172.21.2.101:0/1071256850 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0638094750 msgr2=0x7f0638094b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:52.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.366 7f0636d9d700 1 --2- 172.21.2.101:0/1071256850 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0638094750 0x7f0638094b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:52.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.366 7f0636d9d700 1 -- 172.21.2.101:0/1071256850 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f06380a3a60 msgr2=0x7f06380a3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:52.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.366 7f0636d9d700 1 --2- 172.21.2.101:0/1071256850 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f06380a3a60 0x7f06380a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:52.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.366 7f0636d9d700 1 -- 172.21.2.101:0/1071256850 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06380a15c0 con 0x7f06380b4e00 2021-02-10T11:38:52.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.366 7f0636d9d700 1 --2- 172.21.2.101:0/1071256850 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06380b4e00 0x7f06380a0fa0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 rx=0x7f062000b530 tx=0x7f062000b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:52.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.366 7f062f7fe700 1 -- 172.21.2.101:0/1071256850 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f062000bcc0 con 0x7f06380b4e00 2021-02-10T11:38:52.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.366 7f062f7fe700 1 -- 172.21.2.101:0/1071256850 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0620006020 con 0x7f06380b4e00 2021-02-10T11:38:52.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.366 7f063d085700 1 -- 172.21.2.101:0/1071256850 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06380a15c0 con 0x7f06380b4e00 2021-02-10T11:38:52.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.366 7f063d085700 1 -- 172.21.2.101:0/1071256850 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0638095460 con 0x7f06380b4e00 2021-02-10T11:38:52.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.366 7f062f7fe700 1 -- 172.21.2.101:0/1071256850 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f06200054a0 con 0x7f06380b4e00 2021-02-10T11:38:52.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.367 7f062f7fe700 1 -- 172.21.2.101:0/1071256850 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0620006200 con 0x7f06380b4e00 2021-02-10T11:38:52.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.367 7f062f7fe700 1 --2- 172.21.2.101:0/1071256850 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f061c018730 0x7f061c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:52.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.367 7f062f7fe700 1 -- 172.21.2.101:0/1071256850 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f06200326a0 con 0x7f06380b4e00 2021-02-10T11:38:52.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.367 7f063659c700 1 --2- 172.21.2.101:0/1071256850 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f061c018730 0x7f061c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:52.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.367 7f063659c700 1 --2- 172.21.2.101:0/1071256850 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f061c018730 0x7f061c01ac10 crc :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:52.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.398 7f063d085700 1 -- 172.21.2.101:0/1071256850 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06380ee0e0 con 0x7f06380b4e00 2021-02-10T11:38:52.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.401 7f062f7fe700 1 -- 172.21.2.101:0/1071256850 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0620000940 con 0x7f06380b4e00 2021-02-10T11:38:52.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.530 7f063d085700 1 -- 172.21.2.101:0/1071256850 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06380ee0e0 con 0x7f06380b4e00 2021-02-10T11:38:52.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.530 7f062f7fe700 1 -- 172.21.2.101:0/1071256850 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+1756 (secure 0 0 0) 0x7f062000bcc0 con 0x7f06380b4e00 2021-02-10T11:38:52.535 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 64 2021-02-10T11:38:52.535 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:38:52.535 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9360,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2529021396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2529021396},{"type":"v1","addr":"172.21.2.101:6835","nonce":2529021396}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:36:56.648753","modified":"2021-02-10 11:38:51.893428","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-02-10T11:38:52.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.532 7f063d085700 1 -- 172.21.2.101:0/1071256850 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f061c018730 msgr2=0x7f061c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:52.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.532 7f063d085700 1 --2- 172.21.2.101:0/1071256850 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f061c018730 0x7f061c01ac10 crc :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:52.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.532 7f063d085700 1 -- 172.21.2.101:0/1071256850 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06380b4e00 msgr2=0x7f06380a0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:52.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.532 7f063d085700 1 --2- 172.21.2.101:0/1071256850 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06380b4e00 0x7f06380a0fa0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 rx=0x7f062000b530 tx=0x7f062000b8d0).stop 2021-02-10T11:38:52.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.532 7f063d085700 1 -- 172.21.2.101:0/1071256850 shutdown_connections 2021-02-10T11:38:52.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.532 7f063d085700 1 --2- 172.21.2.101:0/1071256850 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f061c018730 0x7f061c01ac10 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:52.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.532 7f063d085700 1 --2- 172.21.2.101:0/1071256850 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0638094750 0x7f0638094b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:52.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.532 7f063d085700 1 --2- 172.21.2.101:0/1071256850 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f06380a3a60 0x7f06380a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:52.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.532 7f063d085700 1 --2- 172.21.2.101:0/1071256850 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06380b4e00 0x7f06380a0fa0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:52.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.532 7f063d085700 1 -- 172.21.2.101:0/1071256850 >> 172.21.2.101:0/1071256850 conn(0x7f063818cab0 msgr2=0x7f0638092c60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:52.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.532 7f063d085700 1 -- 172.21.2.101:0/1071256850 shutdown_connections 2021-02-10T11:38:52.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.532 7f063d085700 1 -- 172.21.2.101:0/1071256850 wait complete. 2021-02-10T11:38:52.545 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-02-10T11:38:52.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.957 7f56c6179700 1 Processor -- start 2021-02-10T11:38:52.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.957 7f56c6179700 1 -- start start 2021-02-10T11:38:52.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.957 7f56c6179700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f56c00d1870 0x7f56c01b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:52.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.957 7f56c6179700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56c00f0650 0x7f56c00cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:52.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.957 7f56c6179700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c00f0b10 con 0x7f56c01b0a10 2021-02-10T11:38:52.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.957 7f56c6179700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c00e5ed0 con 0x7f56c00d1870 2021-02-10T11:38:52.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.957 7f56c6179700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c00d1c70 con 0x7f56c00f0650 2021-02-10T11:38:52.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.957 7f56bf7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f56c00d1870 0x7f56c01b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:52.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.957 7f56beffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f56c01b0a10 0x7f56c00cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55534/0 (socket says 172.21.2.101:55534) 2021-02-10T11:38:52.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.957 7f56beffd700 1 -- 172.21.2.101:0/1277554502 learned_addr learned my addr 172.21.2.101:0/1277554502 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:38:52.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.957 7f56bffff700 1 --2- 172.21.2.101:0/1277554502 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56c00f0650 0x7f56c00cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:52.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.958 7f56bf7fe700 1 -- 172.21.2.101:0/1277554502 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56c00f0650 msgr2=0x7f56c00cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:52.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.958 7f56bf7fe700 1 --2- 172.21.2.101:0/1277554502 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56c00f0650 0x7f56c00cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:52.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.958 7f56bf7fe700 1 -- 172.21.2.101:0/1277554502 >> v1:172.21.2.101:6789/0 conn(0x7f56c01b0a10 legacy=0x7f56c00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:52.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.958 7f56bf7fe700 1 -- 172.21.2.101:0/1277554502 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c00d1e90 con 0x7f56c00d1870 2021-02-10T11:38:52.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.958 7f56bf7fe700 1 --2- 172.21.2.101:0/1277554502 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56c00d1870 0x7f56c01b03f0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 rx=0x7f56a8006650 tx=0x7f56a8009a80).ready entity=mon.0 client_cookie=162fdb7ee727c81f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:52.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.958 7f56be7fc700 1 -- 172.21.2.101:0/1277554502 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f56a800c0c0 con 0x7f56c00d1870 2021-02-10T11:38:52.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.958 7f56be7fc700 1 -- 172.21.2.101:0/1277554502 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a800c340 con 0x7f56c00d1870 2021-02-10T11:38:52.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.958 7f56be7fc700 1 -- 172.21.2.101:0/1277554502 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f56a800c860 con 0x7f56c00d1870 2021-02-10T11:38:52.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.958 7f56c6179700 1 -- 172.21.2.101:0/1277554502 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56c00d1870 msgr2=0x7f56c01b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:52.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.958 7f56c6179700 1 --2- 172.21.2.101:0/1277554502 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56c00d1870 0x7f56c01b03f0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 rx=0x7f56a8006650 tx=0x7f56a8009a80).stop 2021-02-10T11:38:52.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.959 7f56c6179700 1 -- 172.21.2.101:0/1277554502 shutdown_connections 2021-02-10T11:38:52.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.959 7f56c6179700 1 --2- 172.21.2.101:0/1277554502 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56c00f0650 0x7f56c00cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:52.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.959 7f56c6179700 1 --2- 172.21.2.101:0/1277554502 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56c00d1870 0x7f56c01b03f0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:52.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.959 7f56c6179700 1 -- 172.21.2.101:0/1277554502 >> 172.21.2.101:0/1277554502 conn(0x7f56c00ce450 msgr2=0x7f56c00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:52.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.959 7f56c6179700 1 -- 172.21.2.101:0/1277554502 shutdown_connections 2021-02-10T11:38:52.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.959 7f56c6179700 1 -- 172.21.2.101:0/1277554502 wait complete. 2021-02-10T11:38:52.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.974 7f56c6179700 1 Processor -- start 2021-02-10T11:38:52.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.974 7f56c6179700 1 -- start start 2021-02-10T11:38:52.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.974 7f56c6179700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56c0094310 0x7f56c00ebe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:52.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.974 7f56c6179700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56c0094710 0x7f56c00bca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:52.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.974 7f56c6179700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56c00ec490 0x7f56c00bd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:52.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.974 7f56c6179700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c0093d90 con 0x7f56c00ec490 2021-02-10T11:38:52.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.974 7f56c6179700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c00bd670 con 0x7f56c0094310 2021-02-10T11:38:52.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.974 7f56c6179700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c00939f0 con 0x7f56c0094710 2021-02-10T11:38:52.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.974 7f56bffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56c00ec490 0x7f56c00bd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:52.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.974 7f56bffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56c00ec490 0x7f56c00bd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55046/0 (socket says 172.21.2.101:55046) 2021-02-10T11:38:52.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.974 7f56bffff700 1 -- 172.21.2.101:0/743775050 learned_addr learned my addr 172.21.2.101:0/743775050 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:52.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.975 7f56bffff700 1 -- 172.21.2.101:0/743775050 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56c0094710 msgr2=0x7f56c00bca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:52.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.975 7f56bffff700 1 --2- 172.21.2.101:0/743775050 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56c0094710 0x7f56c00bca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:52.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.975 7f56bffff700 1 -- 172.21.2.101:0/743775050 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56c0094310 msgr2=0x7f56c00ebe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:52.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.975 7f56bffff700 1 --2- 172.21.2.101:0/743775050 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56c0094310 0x7f56c00ebe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:52.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.975 7f56bffff700 1 -- 172.21.2.101:0/743775050 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c00bda80 con 0x7f56c00ec490 2021-02-10T11:38:52.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.975 7f56bffff700 1 --2- 172.21.2.101:0/743775050 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56c00ec490 0x7f56c00bd050 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 rx=0x7f56b800b970 tx=0x7f56b800bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:52.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.975 7f56b7fff700 1 -- 172.21.2.101:0/743775050 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f56b8000f90 con 0x7f56c00ec490 2021-02-10T11:38:52.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.975 7f56b7fff700 1 -- 172.21.2.101:0/743775050 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b8006aa0 con 0x7f56c00ec490 2021-02-10T11:38:52.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.975 7f56b7fff700 1 -- 172.21.2.101:0/743775050 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f56b8001680 con 0x7f56c00ec490 2021-02-10T11:38:52.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.975 7f56c6179700 1 -- 172.21.2.101:0/743775050 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c00bda80 con 0x7f56c00ec490 2021-02-10T11:38:52.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.975 7f56c6179700 1 -- 172.21.2.101:0/743775050 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c00bda80 con 0x7f56c00ec490 2021-02-10T11:38:52.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.976 7f56b7fff700 1 -- 172.21.2.101:0/743775050 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f56b8000df0 con 0x7f56c00ec490 2021-02-10T11:38:52.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.976 7f56b7fff700 1 --2- 172.21.2.101:0/743775050 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56a40186c0 0x7f56a401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:52.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.976 7f56b7fff700 1 -- 172.21.2.101:0/743775050 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f56b8032940 con 0x7f56c00ec490 2021-02-10T11:38:52.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.976 7f56bf7fe700 1 --2- 172.21.2.101:0/743775050 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56a40186c0 0x7f56a401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:52.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:52.976 7f56bf7fe700 1 --2- 172.21.2.101:0/743775050 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56a40186c0 0x7f56a401abc0 crc :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:53.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:53.007 7f56c6179700 1 -- 172.21.2.101:0/743775050 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c00ebc40 con 0x7f56c00ec490 2021-02-10T11:38:53.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:53.009 7f56b7fff700 1 -- 172.21.2.101:0/743775050 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f56b8001650 con 0x7f56c00ec490 2021-02-10T11:38:53.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:53.136 7f56c6179700 1 -- 172.21.2.101:0/743775050 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f56c00f0b10 con 0x7f56c00ec490 2021-02-10T11:38:53.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:53.907 7f56b7fff700 1 -- 172.21.2.101:0/743775050 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v65) v1 ==== 104+0+0 (secure 0 0 0) 0x7f56b8000df0 con 0x7f56c00ec490 2021-02-10T11:38:53.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:53.957 7f56c6179700 1 -- 172.21.2.101:0/743775050 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56a40186c0 msgr2=0x7f56a401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:53.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:53.957 7f56c6179700 1 --2- 172.21.2.101:0/743775050 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56a40186c0 0x7f56a401abc0 crc :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:53.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:53.957 7f56c6179700 1 -- 172.21.2.101:0/743775050 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56c00ec490 msgr2=0x7f56c00bd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:53.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:53.957 7f56c6179700 1 --2- 172.21.2.101:0/743775050 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56c00ec490 0x7f56c00bd050 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 rx=0x7f56b800b970 tx=0x7f56b800bd10).stop 2021-02-10T11:38:53.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:53.958 7f56c6179700 1 -- 172.21.2.101:0/743775050 shutdown_connections 2021-02-10T11:38:53.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:53.958 7f56c6179700 1 --2- 172.21.2.101:0/743775050 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56a40186c0 0x7f56a401abc0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:53.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:53.958 7f56c6179700 1 --2- 172.21.2.101:0/743775050 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56c0094710 0x7f56c00bca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:53.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:53.958 7f56c6179700 1 --2- 172.21.2.101:0/743775050 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56c00ec490 0x7f56c00bd050 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:53.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:53.958 7f56c6179700 1 --2- 172.21.2.101:0/743775050 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56c0094310 0x7f56c00ebe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:53.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:53.958 7f56c6179700 1 -- 172.21.2.101:0/743775050 >> 172.21.2.101:0/743775050 conn(0x7f56c008fa10 msgr2=0x7f56c00bfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:53.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:53.958 7f56c6179700 1 -- 172.21.2.101:0/743775050 shutdown_connections 2021-02-10T11:38:53.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:53.958 7f56c6179700 1 -- 172.21.2.101:0/743775050 wait complete. 2021-02-10T11:38:53.970 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-02-10T11:38:54.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.380 7efd759ab700 1 Processor -- start 2021-02-10T11:38:54.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.380 7efd759ab700 1 -- start start 2021-02-10T11:38:54.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.381 7efd759ab700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7efd700d1c80 0x7efd700cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:54.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.381 7efd759ab700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd700cf590 0x7efd700cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:54.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.381 7efd759ab700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd700fe1d0 con 0x7efd700d1880 2021-02-10T11:38:54.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.381 7efd759ab700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd701b0cc0 con 0x7efd700d1c80 2021-02-10T11:38:54.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.381 7efd759ab700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd70109a70 con 0x7efd700cf590 2021-02-10T11:38:54.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.382 7efd6ed73700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7efd700d1c80 0x7efd700cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:54.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.382 7efd6fd75700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd700cf590 0x7efd700cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:54.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.382 7efd6f574700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7efd700d1880 0x7efd701b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:48956/0 (socket says 172.21.2.101:48956) 2021-02-10T11:38:54.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.382 7efd6f574700 1 -- 172.21.2.101:0/1662191150 learned_addr learned my addr 172.21.2.101:0/1662191150 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:38:54.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.382 7efd6ed73700 1 -- 172.21.2.101:0/1662191150 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd700cf590 msgr2=0x7efd700cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:54.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.382 7efd6ed73700 1 --2- 172.21.2.101:0/1662191150 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd700cf590 0x7efd700cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:54.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.382 7efd6ed73700 1 -- 172.21.2.101:0/1662191150 >> v1:172.21.2.108:6789/0 conn(0x7efd700d1880 legacy=0x7efd701b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:54.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.382 7efd6ed73700 1 -- 172.21.2.101:0/1662191150 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd70088070 con 0x7efd700d1c80 2021-02-10T11:38:54.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.382 7efd6fd75700 1 --2- 172.21.2.101:0/1662191150 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd700cf590 0x7efd700cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:38:54.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.382 7efd6ed73700 1 --2- 172.21.2.101:0/1662191150 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd700d1c80 0x7efd700cef70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7efd64006650 tx=0x7efd64009a80).ready entity=mon.1 client_cookie=64f28afdcb3d9591 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:54.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.383 7efd6e572700 1 -- 172.21.2.101:0/1662191150 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd640009b0 con 0x7efd700d1c80 2021-02-10T11:38:54.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.383 7efd6e572700 1 -- 172.21.2.101:0/1662191150 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6400c0c0 con 0x7efd700d1c80 2021-02-10T11:38:54.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.383 7efd6e572700 1 -- 172.21.2.101:0/1662191150 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd6400c5e0 con 0x7efd700d1c80 2021-02-10T11:38:54.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.383 7efd759ab700 1 -- 172.21.2.101:0/1662191150 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd700d1c80 msgr2=0x7efd700cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:54.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.383 7efd759ab700 1 --2- 172.21.2.101:0/1662191150 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd700d1c80 0x7efd700cef70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7efd64006650 tx=0x7efd64009a80).stop 2021-02-10T11:38:54.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.383 7efd759ab700 1 -- 172.21.2.101:0/1662191150 shutdown_connections 2021-02-10T11:38:54.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.383 7efd759ab700 1 --2- 172.21.2.101:0/1662191150 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd700cf590 0x7efd700cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:54.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.383 7efd759ab700 1 --2- 172.21.2.101:0/1662191150 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd700d1c80 0x7efd700cef70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:54.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.383 7efd759ab700 1 -- 172.21.2.101:0/1662191150 >> 172.21.2.101:0/1662191150 conn(0x7efd700ce430 msgr2=0x7efd700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:54.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.384 7efd759ab700 1 -- 172.21.2.101:0/1662191150 shutdown_connections 2021-02-10T11:38:54.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.384 7efd759ab700 1 -- 172.21.2.101:0/1662191150 wait complete. 2021-02-10T11:38:54.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.398 7efd759ab700 1 Processor -- start 2021-02-10T11:38:54.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.398 7efd759ab700 1 -- start start 2021-02-10T11:38:54.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.398 7efd759ab700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd701a5af0 0x7efd700b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:54.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.398 7efd759ab700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd701a6030 0x7efd700b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:54.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.398 7efd759ab700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd700b23e0 0x7efd70099e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:54.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.398 7efd759ab700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7018e0e0 con 0x7efd701a5af0 2021-02-10T11:38:54.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.398 7efd759ab700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7009a470 con 0x7efd701a6030 2021-02-10T11:38:54.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.398 7efd759ab700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd7009a690 con 0x7efd700b23e0 2021-02-10T11:38:54.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.398 7efd6f574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd701a5af0 0x7efd700b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:54.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.399 7efd6f574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd701a5af0 0x7efd700b17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55058/0 (socket says 172.21.2.101:55058) 2021-02-10T11:38:54.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.399 7efd6f574700 1 -- 172.21.2.101:0/1861886139 learned_addr learned my addr 172.21.2.101:0/1861886139 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:54.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.399 7efd6f574700 1 -- 172.21.2.101:0/1861886139 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd700b23e0 msgr2=0x7efd70099e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:54.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.399 7efd6f574700 1 --2- 172.21.2.101:0/1861886139 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd700b23e0 0x7efd70099e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:54.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.399 7efd6f574700 1 -- 172.21.2.101:0/1861886139 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd701a6030 msgr2=0x7efd700b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:54.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.399 7efd6f574700 1 --2- 172.21.2.101:0/1861886139 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd701a6030 0x7efd700b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:54.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.399 7efd6f574700 1 -- 172.21.2.101:0/1861886139 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7009a8b0 con 0x7efd701a5af0 2021-02-10T11:38:54.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.399 7efd6f574700 1 --2- 172.21.2.101:0/1861886139 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd701a5af0 0x7efd700b17a0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 rx=0x7efd600013a0 tx=0x7efd6000a8a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:54.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.399 7efd57fff700 1 -- 172.21.2.101:0/1861886139 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd600070d0 con 0x7efd701a5af0 2021-02-10T11:38:54.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.399 7efd57fff700 1 -- 172.21.2.101:0/1861886139 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6000ac90 con 0x7efd701a5af0 2021-02-10T11:38:54.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.399 7efd57fff700 1 -- 172.21.2.101:0/1861886139 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd600045a0 con 0x7efd701a5af0 2021-02-10T11:38:54.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.399 7efd759ab700 1 -- 172.21.2.101:0/1861886139 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7009a8b0 con 0x7efd701a5af0 2021-02-10T11:38:54.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.399 7efd759ab700 1 -- 172.21.2.101:0/1861886139 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd7018cc10 con 0x7efd701a5af0 2021-02-10T11:38:54.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.400 7efd57fff700 1 -- 172.21.2.101:0/1861886139 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7efd60004320 con 0x7efd701a5af0 2021-02-10T11:38:54.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.400 7efd57fff700 1 --2- 172.21.2.101:0/1861886139 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd500186c0 0x7efd5001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:54.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.400 7efd57fff700 1 -- 172.21.2.101:0/1861886139 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5659+0+0 (secure 0 0 0) 0x7efd600325a0 con 0x7efd701a5af0 2021-02-10T11:38:54.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.400 7efd6ed73700 1 --2- 172.21.2.101:0/1861886139 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd500186c0 0x7efd5001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:54.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.401 7efd6ed73700 1 --2- 172.21.2.101:0/1861886139 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd500186c0 0x7efd5001abc0 crc :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:54.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.431 7efd759ab700 1 -- 172.21.2.101:0/1861886139 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd70082a70 con 0x7efd701a5af0 2021-02-10T11:38:54.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.433 7efd57fff700 1 -- 172.21.2.101:0/1861886139 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7efd60000eb0 con 0x7efd701a5af0 2021-02-10T11:38:54.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.559 7efd759ab700 1 -- 172.21.2.101:0/1861886139 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efd7009ae40 con 0x7efd701a5af0 2021-02-10T11:38:54.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.913 7efd57fff700 1 -- 172.21.2.101:0/1861886139 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v87) v1 ==== 185+0+0 (secure 0 0 0) 0x7efd6001f4d0 con 0x7efd701a5af0 2021-02-10T11:38:54.931 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' removed 2021-02-10T11:38:54.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.927 7efd759ab700 1 -- 172.21.2.101:0/1861886139 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd500186c0 msgr2=0x7efd5001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:54.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.927 7efd759ab700 1 --2- 172.21.2.101:0/1861886139 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd500186c0 0x7efd5001abc0 crc :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:54.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.927 7efd759ab700 1 -- 172.21.2.101:0/1861886139 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd701a5af0 msgr2=0x7efd700b17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:54.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.927 7efd759ab700 1 --2- 172.21.2.101:0/1861886139 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd701a5af0 0x7efd700b17a0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 rx=0x7efd600013a0 tx=0x7efd6000a8a0).stop 2021-02-10T11:38:54.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.927 7efd759ab700 1 -- 172.21.2.101:0/1861886139 shutdown_connections 2021-02-10T11:38:54.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.927 7efd759ab700 1 --2- 172.21.2.101:0/1861886139 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd500186c0 0x7efd5001abc0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:54.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.927 7efd759ab700 1 --2- 172.21.2.101:0/1861886139 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd700b23e0 0x7efd70099e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:54.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.927 7efd759ab700 1 --2- 172.21.2.101:0/1861886139 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd701a6030 0x7efd700b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:54.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.927 7efd759ab700 1 --2- 172.21.2.101:0/1861886139 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd701a5af0 0x7efd700b17a0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:54.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.927 7efd759ab700 1 -- 172.21.2.101:0/1861886139 >> 172.21.2.101:0/1861886139 conn(0x7efd700873b0 msgr2=0x7efd701b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:54.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.928 7efd759ab700 1 -- 172.21.2.101:0/1861886139 shutdown_connections 2021-02-10T11:38:54.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:54.928 7efd759ab700 1 -- 172.21.2.101:0/1861886139 wait complete. 2021-02-10T11:38:54.940 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-02-10T11:38:55.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.352 7f9d53634700 1 Processor -- start 2021-02-10T11:38:55.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.352 7f9d53634700 1 -- start start 2021-02-10T11:38:55.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.353 7f9d53634700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f9d4c0d1870 0x7f9d4c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:55.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.353 7f9d53634700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9d4c0cff70 0x7f9d4c109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:55.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.353 7f9d53634700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d4c0fe1c0 con 0x7f9d4c1b0ae0 2021-02-10T11:38:55.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.353 7f9d53634700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d4c109ad0 con 0x7f9d4c0cff70 2021-02-10T11:38:55.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.353 7f9d53634700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d4c109750 con 0x7f9d4c0d1870 2021-02-10T11:38:55.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.353 7f9d51c51700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9d4c0cff70 0x7f9d4c109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:55.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.353 7f9d51c51700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9d4c0cff70 0x7f9d4c109130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55070/0 (socket says 172.21.2.101:55070) 2021-02-10T11:38:55.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.353 7f9d51c51700 1 -- 172.21.2.101:0/3864350739 learned_addr learned my addr 172.21.2.101:0/3864350739 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:55.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.353 7f9d50c4f700 1 --1- 172.21.2.101:0/3864350739 >> v1:172.21.2.101:6789/0 conn(0x7f9d4c1b0ae0 0x7f9d4c108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55560/0 (socket says 172.21.2.101:55560) 2021-02-10T11:38:55.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.353 7f9d51c51700 1 -- 172.21.2.101:0/3864350739 >> v2:172.21.2.108:3300/0 conn(0x7f9d4c0d1870 msgr2=0x7f9d4c1b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:38:55.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.353 7f9d51c51700 1 --2- 172.21.2.101:0/3864350739 >> v2:172.21.2.108:3300/0 conn(0x7f9d4c0d1870 0x7f9d4c1b04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:55.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.353 7f9d51c51700 1 -- 172.21.2.101:0/3864350739 >> v1:172.21.2.101:6789/0 conn(0x7f9d4c1b0ae0 legacy=0x7f9d4c108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:55.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.353 7f9d51c51700 1 -- 172.21.2.101:0/3864350739 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c088070 con 0x7f9d4c0cff70 2021-02-10T11:38:55.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.353 7f9d51c51700 1 --2- 172.21.2.101:0/3864350739 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d4c0cff70 0x7f9d4c109130 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 rx=0x7f9d44006f00 tx=0x7f9d44009f10).ready entity=mon.0 client_cookie=5221231a328bce5b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:55.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.354 7f9d3ffff700 1 -- 172.21.2.101:0/3864350739 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d44003030 con 0x7f9d4c0cff70 2021-02-10T11:38:55.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.354 7f9d3ffff700 1 -- 172.21.2.101:0/3864350739 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4400b3b0 con 0x7f9d4c0cff70 2021-02-10T11:38:55.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.354 7f9d3ffff700 1 -- 172.21.2.101:0/3864350739 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d4400b8d0 con 0x7f9d4c0cff70 2021-02-10T11:38:55.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.354 7f9d53634700 1 -- 172.21.2.101:0/3864350739 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d4c0cff70 msgr2=0x7f9d4c109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:55.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.354 7f9d53634700 1 --2- 172.21.2.101:0/3864350739 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d4c0cff70 0x7f9d4c109130 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 rx=0x7f9d44006f00 tx=0x7f9d44009f10).stop 2021-02-10T11:38:55.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.354 7f9d53634700 1 -- 172.21.2.101:0/3864350739 shutdown_connections 2021-02-10T11:38:55.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.354 7f9d53634700 1 --2- 172.21.2.101:0/3864350739 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d4c0cff70 0x7f9d4c109130 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:55.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.354 7f9d53634700 1 --2- 172.21.2.101:0/3864350739 >> v2:172.21.2.108:3300/0 conn(0x7f9d4c0d1870 0x7f9d4c1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:55.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.354 7f9d53634700 1 -- 172.21.2.101:0/3864350739 >> 172.21.2.101:0/3864350739 conn(0x7f9d4c0ce420 msgr2=0x7f9d4c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:55.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.355 7f9d53634700 1 -- 172.21.2.101:0/3864350739 shutdown_connections 2021-02-10T11:38:55.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.355 7f9d53634700 1 -- 172.21.2.101:0/3864350739 wait complete. 2021-02-10T11:38:55.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.369 7f9d53634700 1 Processor -- start 2021-02-10T11:38:55.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.369 7f9d53634700 1 -- start start 2021-02-10T11:38:55.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d53634700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d4c19fb30 0x7f9d4c19ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:55.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d53634700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d4c09a0d0 0x7f9d4c09a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:55.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d53634700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d4c09aaf0 0x7f9d4c1a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:55.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d53634700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c1a0920 con 0x7f9d4c09a0d0 2021-02-10T11:38:55.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d53634700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c09aef0 con 0x7f9d4c19fb30 2021-02-10T11:38:55.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d53634700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c1a0550 con 0x7f9d4c09aaf0 2021-02-10T11:38:55.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d50c4f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d4c09a0d0 0x7f9d4c09a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:55.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d50c4f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d4c09a0d0 0x7f9d4c09a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55074/0 (socket says 172.21.2.101:55074) 2021-02-10T11:38:55.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d50c4f700 1 -- 172.21.2.101:0/1991732025 learned_addr learned my addr 172.21.2.101:0/1991732025 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:55.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d51450700 1 --2- 172.21.2.101:0/1991732025 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d4c19fb30 0x7f9d4c19ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:55.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d50c4f700 1 -- 172.21.2.101:0/1991732025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d4c09aaf0 msgr2=0x7f9d4c1a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:55.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d51c51700 1 --2- 172.21.2.101:0/1991732025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d4c09aaf0 0x7f9d4c1a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:55.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d50c4f700 1 --2- 172.21.2.101:0/1991732025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d4c09aaf0 0x7f9d4c1a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:55.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d50c4f700 1 -- 172.21.2.101:0/1991732025 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d4c19fb30 msgr2=0x7f9d4c19ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:55.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d50c4f700 1 --2- 172.21.2.101:0/1991732025 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d4c19fb30 0x7f9d4c19ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:55.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d50c4f700 1 -- 172.21.2.101:0/1991732025 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c1a14c0 con 0x7f9d4c09a0d0 2021-02-10T11:38:55.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d51c51700 1 --2- 172.21.2.101:0/1991732025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d4c09aaf0 0x7f9d4c1a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:38:55.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d51450700 1 --2- 172.21.2.101:0/1991732025 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d4c19fb30 0x7f9d4c19ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:38:55.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d50c4f700 1 --2- 172.21.2.101:0/1991732025 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d4c09a0d0 0x7f9d4c09a4d0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 rx=0x7f9d48001c60 tx=0x7f9d4800a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:55.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d3dffb700 1 -- 172.21.2.101:0/1991732025 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d4800acc0 con 0x7f9d4c09a0d0 2021-02-10T11:38:55.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d3dffb700 1 -- 172.21.2.101:0/1991732025 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d48004140 con 0x7f9d4c09a0d0 2021-02-10T11:38:55.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d53634700 1 -- 172.21.2.101:0/1991732025 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d4c1a14c0 con 0x7f9d4c09a0d0 2021-02-10T11:38:55.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d3dffb700 1 -- 172.21.2.101:0/1991732025 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d480045a0 con 0x7f9d4c09a0d0 2021-02-10T11:38:55.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.370 7f9d53634700 1 -- 172.21.2.101:0/1991732025 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d4c1a14c0 con 0x7f9d4c09a0d0 2021-02-10T11:38:55.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.371 7f9d3dffb700 1 -- 172.21.2.101:0/1991732025 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9d48000d90 con 0x7f9d4c09a0d0 2021-02-10T11:38:55.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.371 7f9d3dffb700 1 --2- 172.21.2.101:0/1991732025 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d2c018730 0x7f9d2c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:55.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.371 7f9d3dffb700 1 -- 172.21.2.101:0/1991732025 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f9d480324d0 con 0x7f9d4c09a0d0 2021-02-10T11:38:55.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.371 7f9d51450700 1 --2- 172.21.2.101:0/1991732025 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d2c018730 0x7f9d2c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:55.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.371 7f9d51450700 1 --2- 172.21.2.101:0/1991732025 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d2c018730 0x7f9d2c01ac10 crc :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:55.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.403 7f9d53634700 1 -- 172.21.2.101:0/1991732025 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d4c0825d0 con 0x7f9d4c09a0d0 2021-02-10T11:38:55.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.405 7f9d3dffb700 1 -- 172.21.2.101:0/1991732025 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9d480009b0 con 0x7f9d4c09a0d0 2021-02-10T11:38:55.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.536 7f9d53634700 1 -- 172.21.2.101:0/1991732025 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9d4c0825d0 con 0x7f9d4c09a0d0 2021-02-10T11:38:55.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.919 7f9d3dffb700 1 -- 172.21.2.101:0/1991732025 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v88) v1 ==== 173+0+0 (secure 0 0 0) 0x7f9d4801f4f0 con 0x7f9d4c09a0d0 2021-02-10T11:38:55.956 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' removed 2021-02-10T11:38:55.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.955 7f9d53634700 1 -- 172.21.2.101:0/1991732025 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d2c018730 msgr2=0x7f9d2c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:55.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.955 7f9d53634700 1 --2- 172.21.2.101:0/1991732025 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d2c018730 0x7f9d2c01ac10 crc :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:55.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.955 7f9d53634700 1 -- 172.21.2.101:0/1991732025 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d4c09a0d0 msgr2=0x7f9d4c09a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:55.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.955 7f9d53634700 1 --2- 172.21.2.101:0/1991732025 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d4c09a0d0 0x7f9d4c09a4d0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 rx=0x7f9d48001c60 tx=0x7f9d4800a8d0).stop 2021-02-10T11:38:55.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.955 7f9d53634700 1 -- 172.21.2.101:0/1991732025 shutdown_connections 2021-02-10T11:38:55.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.955 7f9d53634700 1 --2- 172.21.2.101:0/1991732025 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d2c018730 0x7f9d2c01ac10 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:55.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.955 7f9d53634700 1 --2- 172.21.2.101:0/1991732025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d4c09aaf0 0x7f9d4c1a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:55.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.955 7f9d53634700 1 --2- 172.21.2.101:0/1991732025 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d4c09a0d0 0x7f9d4c09a4d0 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:55.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.955 7f9d53634700 1 --2- 172.21.2.101:0/1991732025 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d4c19fb30 0x7f9d4c19ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:55.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.955 7f9d53634700 1 -- 172.21.2.101:0/1991732025 >> 172.21.2.101:0/1991732025 conn(0x7f9d4c1b3560 msgr2=0x7f9d4c1b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:55.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.955 7f9d53634700 1 -- 172.21.2.101:0/1991732025 shutdown_connections 2021-02-10T11:38:55.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:55.955 7f9d53634700 1 -- 172.21.2.101:0/1991732025 wait complete. 2021-02-10T11:38:55.967 INFO:tasks.ceph.mds.a:Restarting daemon 2021-02-10T11:38:55.967 INFO:tasks.ceph.mds.a:Stopping old one... 2021-02-10T11:38:55.967 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-02-10T11:38:55.967 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:38:55.968 INFO:tasks.ceph.mds.b:Restarting daemon 2021-02-10T11:38:55.968 INFO:tasks.ceph.mds.b:Stopping old one... 2021-02-10T11:38:55.968 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-02-10T11:38:55.968 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:38:55.974 INFO:tasks.ceph.mds.a:Stopped 2021-02-10T11:38:55.974 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-02-10T11:38:55.975 INFO:tasks.ceph.mds.b:Stopped 2021-02-10T11:38:55.975 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-02-10T11:38:56.016 INFO:tasks.ceph.mds.a:Started 2021-02-10T11:38:56.016 DEBUG:teuthology.parallel:result is None 2021-02-10T11:38:56.017 INFO:tasks.ceph.mds.b:Started 2021-02-10T11:38:56.017 DEBUG:teuthology.parallel:result is None 2021-02-10T11:38:56.017 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd blacklist clear 2021-02-10T11:38:56.421 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:38:56.424 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:38:56.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.423 7f797436d700 1 Processor -- start 2021-02-10T11:38:56.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.424 7f797436d700 1 -- start start 2021-02-10T11:38:56.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.424 7f797436d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f796c0d0340 0x7f796c0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:56.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.424 7f797436d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f796c0d1c70 0x7f796c0efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:56.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.424 7f797436d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f796c0f0b10 con 0x7f796c0d1870 2021-02-10T11:38:56.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.424 7f797436d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f796c0e5ed0 con 0x7f796c0d1c70 2021-02-10T11:38:56.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.424 7f797436d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796c0d2fe0 con 0x7f796c0d0340 2021-02-10T11:38:56.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.424 7f797298a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f796c0d1c70 0x7f796c0efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:56.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.424 7f7971988700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f796c0d0340 0x7f796c0ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:56.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.424 7f7972189700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f796c0d1870 0x7f796c1b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:48996/0 (socket says 172.21.2.101:48996) 2021-02-10T11:38:56.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.424 7f7972189700 1 -- 172.21.2.101:0/197327925 learned_addr learned my addr 172.21.2.101:0/197327925 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:38:56.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.425 7f797298a700 1 -- 172.21.2.101:0/197327925 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f796c0d0340 msgr2=0x7f796c0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:56.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.425 7f797298a700 1 --2- 172.21.2.101:0/197327925 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f796c0d0340 0x7f796c0ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:56.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.425 7f797298a700 1 -- 172.21.2.101:0/197327925 >> v1:172.21.2.108:6789/0 conn(0x7f796c0d1870 legacy=0x7f796c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:56.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.425 7f797298a700 1 -- 172.21.2.101:0/197327925 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c0d3360 con 0x7f796c0d1c70 2021-02-10T11:38:56.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.425 7f797298a700 1 --2- 172.21.2.101:0/197327925 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f796c0d1c70 0x7f796c0efc60 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 rx=0x7f7964006e30 tx=0x7f7964009e60).ready entity=mon.0 client_cookie=f3dc112e1a6b4d92 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:56.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.425 7f7971187700 1 -- 172.21.2.101:0/197327925 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7964006a50 con 0x7f796c0d1c70 2021-02-10T11:38:56.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.425 7f7971187700 1 -- 172.21.2.101:0/197327925 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796400b300 con 0x7f796c0d1c70 2021-02-10T11:38:56.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.425 7f7971187700 1 -- 172.21.2.101:0/197327925 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f796400b760 con 0x7f796c0d1c70 2021-02-10T11:38:56.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.425 7f797436d700 1 -- 172.21.2.101:0/197327925 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f796c0d1c70 msgr2=0x7f796c0efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:56.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.425 7f797436d700 1 --2- 172.21.2.101:0/197327925 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f796c0d1c70 0x7f796c0efc60 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 rx=0x7f7964006e30 tx=0x7f7964009e60).stop 2021-02-10T11:38:56.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.426 7f797436d700 1 -- 172.21.2.101:0/197327925 shutdown_connections 2021-02-10T11:38:56.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.426 7f797436d700 1 --2- 172.21.2.101:0/197327925 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f796c0d0340 0x7f796c0ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:56.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.426 7f797436d700 1 --2- 172.21.2.101:0/197327925 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f796c0d1c70 0x7f796c0efc60 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:56.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.426 7f797436d700 1 -- 172.21.2.101:0/197327925 >> 172.21.2.101:0/197327925 conn(0x7f796c0ce450 msgr2=0x7f796c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:56.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.426 7f797436d700 1 -- 172.21.2.101:0/197327925 shutdown_connections 2021-02-10T11:38:56.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.426 7f797436d700 1 -- 172.21.2.101:0/197327925 wait complete. 2021-02-10T11:38:56.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.441 7f797436d700 1 Processor -- start 2021-02-10T11:38:56.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.441 7f797436d700 1 -- start start 2021-02-10T11:38:56.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.441 7f797436d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f796c19ffa0 0x7f796c0eda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:56.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.441 7f797436d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f796c1a03a0 0x7f796c0ee030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:56.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.441 7f797436d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f796c0ee650 0x7f796c0ebe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:56.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.441 7f797436d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c0ed490 con 0x7f796c1a03a0 2021-02-10T11:38:56.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.441 7f797436d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c0eea50 con 0x7f796c0ee650 2021-02-10T11:38:56.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.441 7f797436d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796c0eec30 con 0x7f796c19ffa0 2021-02-10T11:38:56.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.441 7f7971988700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f796c1a03a0 0x7f796c0ee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:56.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.441 7f7971988700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f796c1a03a0 0x7f796c0ee030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55098/0 (socket says 172.21.2.101:55098) 2021-02-10T11:38:56.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.441 7f7971988700 1 -- 172.21.2.101:0/125552845 learned_addr learned my addr 172.21.2.101:0/125552845 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:56.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.441 7f7972189700 1 --2- 172.21.2.101:0/125552845 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f796c19ffa0 0x7f796c0eda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:56.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.441 7f797298a700 1 --2- 172.21.2.101:0/125552845 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f796c0ee650 0x7f796c0ebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:56.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.442 7f7971988700 1 -- 172.21.2.101:0/125552845 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f796c19ffa0 msgr2=0x7f796c0eda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:56.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.442 7f7971988700 1 --2- 172.21.2.101:0/125552845 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f796c19ffa0 0x7f796c0eda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:56.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.442 7f7971988700 1 -- 172.21.2.101:0/125552845 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f796c0ee650 msgr2=0x7f796c0ebe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:56.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.442 7f7971988700 1 --2- 172.21.2.101:0/125552845 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f796c0ee650 0x7f796c0ebe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:56.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.442 7f7971988700 1 -- 172.21.2.101:0/125552845 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c0ec4a0 con 0x7f796c1a03a0 2021-02-10T11:38:56.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.442 7f797298a700 1 --2- 172.21.2.101:0/125552845 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f796c0ee650 0x7f796c0ebe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:38:56.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.442 7f7971988700 1 --2- 172.21.2.101:0/125552845 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f796c1a03a0 0x7f796c0ee030 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 rx=0x7f795c006f60 tx=0x7f795c00b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:56.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.442 7f7962ffd700 1 -- 172.21.2.101:0/125552845 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f795c00bd60 con 0x7f796c1a03a0 2021-02-10T11:38:56.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.442 7f7962ffd700 1 -- 172.21.2.101:0/125552845 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795c000c90 con 0x7f796c1a03a0 2021-02-10T11:38:56.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.442 7f7962ffd700 1 -- 172.21.2.101:0/125552845 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f795c0022b0 con 0x7f796c1a03a0 2021-02-10T11:38:56.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.442 7f797436d700 1 -- 172.21.2.101:0/125552845 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796c0ec4a0 con 0x7f796c1a03a0 2021-02-10T11:38:56.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.442 7f797436d700 1 -- 172.21.2.101:0/125552845 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f796c0eca30 con 0x7f796c1a03a0 2021-02-10T11:38:56.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.443 7f7962ffd700 1 -- 172.21.2.101:0/125552845 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f795c006b00 con 0x7f796c1a03a0 2021-02-10T11:38:56.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.443 7f7962ffd700 1 --2- 172.21.2.101:0/125552845 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79540186c0 0x7f795401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:56.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.443 7f7962ffd700 1 -- 172.21.2.101:0/125552845 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4839+0+0 (secure 0 0 0) 0x7f795c0324e0 con 0x7f796c1a03a0 2021-02-10T11:38:56.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.443 7f7972189700 1 --2- 172.21.2.101:0/125552845 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79540186c0 0x7f795401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:56.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.443 7f7972189700 1 --2- 172.21.2.101:0/125552845 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79540186c0 0x7f795401abc0 crc :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:56.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.472 7f797436d700 1 -- 172.21.2.101:0/125552845 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f796c0ebc50 con 0x7f796c1a03a0 2021-02-10T11:38:56.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.474 7f7962ffd700 1 -- 172.21.2.101:0/125552845 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f795c000c90 con 0x7f796c1a03a0 2021-02-10T11:38:56.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.618 7f797436d700 1 -- 172.21.2.101:0/125552845 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f796c0ebc50 con 0x7f796c1a03a0 2021-02-10T11:38:56.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.959 7f7962ffd700 1 -- 172.21.2.101:0/125552845 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v89) v1 ==== 97+0+0 (secure 0 0 0) 0x7f795c006b00 con 0x7f796c1a03a0 2021-02-10T11:38:56.988 INFO:teuthology.orchestra.run.gibba001.stderr: removed all blacklist entries 2021-02-10T11:38:56.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.986 7f797436d700 1 -- 172.21.2.101:0/125552845 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79540186c0 msgr2=0x7f795401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:56.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.986 7f797436d700 1 --2- 172.21.2.101:0/125552845 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79540186c0 0x7f795401abc0 crc :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:56.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.986 7f797436d700 1 -- 172.21.2.101:0/125552845 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f796c1a03a0 msgr2=0x7f796c0ee030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:56.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.986 7f797436d700 1 --2- 172.21.2.101:0/125552845 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f796c1a03a0 0x7f796c0ee030 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 rx=0x7f795c006f60 tx=0x7f795c00b970).stop 2021-02-10T11:38:56.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.986 7f797436d700 1 -- 172.21.2.101:0/125552845 shutdown_connections 2021-02-10T11:38:56.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.986 7f797436d700 1 --2- 172.21.2.101:0/125552845 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f796c0ee650 0x7f796c0ebe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:56.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.986 7f797436d700 1 --2- 172.21.2.101:0/125552845 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f796c1a03a0 0x7f796c0ee030 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:56.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.986 7f797436d700 1 --2- 172.21.2.101:0/125552845 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f79540186c0 0x7f795401abc0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:56.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.986 7f797436d700 1 --2- 172.21.2.101:0/125552845 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f796c19ffa0 0x7f796c0eda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:56.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.986 7f797436d700 1 -- 172.21.2.101:0/125552845 >> 172.21.2.101:0/125552845 conn(0x7f796c094d90 msgr2=0x7f796c08a420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:56.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.986 7f797436d700 1 -- 172.21.2.101:0/125552845 shutdown_connections 2021-02-10T11:38:56.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:56.986 7f797436d700 1 -- 172.21.2.101:0/125552845 wait complete. 2021-02-10T11:38:56.998 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2021-02-10T11:38:56.998 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:38:57.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.405 7ff247a4c700 1 Processor -- start 2021-02-10T11:38:57.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.405 7ff247a4c700 1 -- start start 2021-02-10T11:38:57.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.406 7ff247a4c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2400d1870 0x7ff2401b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:57.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.406 7ff247a4c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff2401b0a10 0x7ff2400cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:57.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.406 7ff247a4c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2400f0b10 con 0x7ff2400d1c70 2021-02-10T11:38:57.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.406 7ff247a4c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2400fe230 con 0x7ff2401b0a10 2021-02-10T11:38:57.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.406 7ff247a4c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2400e5ed0 con 0x7ff2400d1870 2021-02-10T11:38:57.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.406 7ff245868700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2400d1870 0x7ff2401b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:57.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.407 7ff245868700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2400d1870 0x7ff2401b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:47838/0 (socket says 172.21.2.101:47838) 2021-02-10T11:38:57.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.407 7ff245868700 1 -- 172.21.2.101:0/4284019186 learned_addr learned my addr 172.21.2.101:0/4284019186 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:57.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.407 7ff245067700 1 --2- 172.21.2.101:0/4284019186 >> v2:172.21.2.108:3300/0 conn(0x7ff2401b0a10 0x7ff2400cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:57.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.407 7ff245868700 1 -- 172.21.2.101:0/4284019186 >> v2:172.21.2.108:3300/0 conn(0x7ff2401b0a10 msgr2=0x7ff2400cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:57.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.407 7ff245868700 1 --2- 172.21.2.101:0/4284019186 >> v2:172.21.2.108:3300/0 conn(0x7ff2401b0a10 0x7ff2400cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:57.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.407 7ff245868700 1 -- 172.21.2.101:0/4284019186 >> v1:172.21.2.108:6789/0 conn(0x7ff2400d1c70 legacy=0x7ff2400cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:57.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.407 7ff245868700 1 -- 172.21.2.101:0/4284019186 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2400f0650 con 0x7ff2400d1870 2021-02-10T11:38:57.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.407 7ff245868700 1 --2- 172.21.2.101:0/4284019186 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2400d1870 0x7ff2401b03f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7ff23c006700 tx=0x7ff23c009b80).ready entity=mon.2 client_cookie=304406adc315ad1b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:57.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.408 7ff244866700 1 -- 172.21.2.101:0/4284019186 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff23c00c0c0 con 0x7ff2400d1870 2021-02-10T11:38:57.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.408 7ff244866700 1 -- 172.21.2.101:0/4284019186 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c00c340 con 0x7ff2400d1870 2021-02-10T11:38:57.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.408 7ff244866700 1 -- 172.21.2.101:0/4284019186 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff23c00c860 con 0x7ff2400d1870 2021-02-10T11:38:57.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.408 7ff247a4c700 1 -- 172.21.2.101:0/4284019186 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2400d1870 msgr2=0x7ff2401b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:57.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.408 7ff247a4c700 1 --2- 172.21.2.101:0/4284019186 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2400d1870 0x7ff2401b03f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7ff23c006700 tx=0x7ff23c009b80).stop 2021-02-10T11:38:57.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.408 7ff247a4c700 1 -- 172.21.2.101:0/4284019186 shutdown_connections 2021-02-10T11:38:57.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.408 7ff247a4c700 1 --2- 172.21.2.101:0/4284019186 >> v2:172.21.2.108:3300/0 conn(0x7ff2401b0a10 0x7ff2400cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:57.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.408 7ff247a4c700 1 --2- 172.21.2.101:0/4284019186 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2400d1870 0x7ff2401b03f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:57.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.408 7ff247a4c700 1 -- 172.21.2.101:0/4284019186 >> 172.21.2.101:0/4284019186 conn(0x7ff2400ce450 msgr2=0x7ff2400d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:57.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.409 7ff247a4c700 1 -- 172.21.2.101:0/4284019186 shutdown_connections 2021-02-10T11:38:57.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.409 7ff247a4c700 1 -- 172.21.2.101:0/4284019186 wait complete. 2021-02-10T11:38:57.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.423 7ff247a4c700 1 Processor -- start 2021-02-10T11:38:57.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.423 7ff247a4c700 1 -- start start 2021-02-10T11:38:57.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.423 7ff247a4c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff24008a740 0x7ff24008ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:57.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.423 7ff247a4c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff24008b160 0x7ff240194990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:57.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.423 7ff247a4c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2400913c0 0x7ff240194fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:57.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.423 7ff247a4c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2401ac380 con 0x7ff24008a740 2021-02-10T11:38:57.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.423 7ff247a4c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff240091970 con 0x7ff24008b160 2021-02-10T11:38:57.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.423 7ff247a4c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2401abe70 con 0x7ff2400913c0 2021-02-10T11:38:57.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.424 7ff245868700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff24008a740 0x7ff24008ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:57.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.424 7ff245868700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff24008a740 0x7ff24008ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55110/0 (socket says 172.21.2.101:55110) 2021-02-10T11:38:57.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.424 7ff245868700 1 -- 172.21.2.101:0/2705765228 learned_addr learned my addr 172.21.2.101:0/2705765228 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:57.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.424 7ff246069700 1 --2- 172.21.2.101:0/2705765228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2400913c0 0x7ff240194fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:57.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.424 7ff245067700 1 --2- 172.21.2.101:0/2705765228 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff24008b160 0x7ff240194990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:57.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.424 7ff245868700 1 -- 172.21.2.101:0/2705765228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2400913c0 msgr2=0x7ff240194fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:57.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.424 7ff245868700 1 --2- 172.21.2.101:0/2705765228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2400913c0 0x7ff240194fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:57.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.424 7ff245868700 1 -- 172.21.2.101:0/2705765228 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff24008b160 msgr2=0x7ff240194990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:57.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.424 7ff245868700 1 --2- 172.21.2.101:0/2705765228 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff24008b160 0x7ff240194990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:57.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.424 7ff245868700 1 -- 172.21.2.101:0/2705765228 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2401955d0 con 0x7ff24008a740 2021-02-10T11:38:57.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.424 7ff246069700 1 --2- 172.21.2.101:0/2705765228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2400913c0 0x7ff240194fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:38:57.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.424 7ff245067700 1 --2- 172.21.2.101:0/2705765228 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff24008b160 0x7ff240194990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:38:57.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.424 7ff245868700 1 --2- 172.21.2.101:0/2705765228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff24008a740 0x7ff24008ab40 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 rx=0x7ff23c009760 tx=0x7ff23c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:57.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.425 7ff2327fc700 1 -- 172.21.2.101:0/2705765228 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff23c006240 con 0x7ff24008a740 2021-02-10T11:38:57.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.425 7ff2327fc700 1 -- 172.21.2.101:0/2705765228 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c0052e0 con 0x7ff24008a740 2021-02-10T11:38:57.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.425 7ff2327fc700 1 -- 172.21.2.101:0/2705765228 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff23c005730 con 0x7ff24008a740 2021-02-10T11:38:57.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.425 7ff247a4c700 1 -- 172.21.2.101:0/2705765228 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2401955d0 con 0x7ff24008a740 2021-02-10T11:38:57.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.425 7ff247a4c700 1 -- 172.21.2.101:0/2705765228 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2401955d0 con 0x7ff24008a740 2021-02-10T11:38:57.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.425 7ff2327fc700 1 -- 172.21.2.101:0/2705765228 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff23c006030 con 0x7ff24008a740 2021-02-10T11:38:57.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.425 7ff2327fc700 1 --2- 172.21.2.101:0/2705765228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2280186d0 0x7ff22801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:57.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.425 7ff2327fc700 1 -- 172.21.2.101:0/2705765228 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4753+0+0 (secure 0 0 0) 0x7ff23c032470 con 0x7ff24008a740 2021-02-10T11:38:57.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.425 7ff245067700 1 --2- 172.21.2.101:0/2705765228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2280186d0 0x7ff22801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:57.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.426 7ff245067700 1 --2- 172.21.2.101:0/2705765228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2280186d0 0x7ff22801abd0 crc :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:57.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.456 7ff247a4c700 1 -- 172.21.2.101:0/2705765228 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2400ebc00 con 0x7ff24008a740 2021-02-10T11:38:57.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.458 7ff2327fc700 1 -- 172.21.2.101:0/2705765228 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff23c009030 con 0x7ff24008a740 2021-02-10T11:38:57.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.591 7ff247a4c700 1 -- 172.21.2.101:0/2705765228 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff2400f0b10 con 0x7ff24008a740 2021-02-10T11:38:57.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.591 7ff2327fc700 1 -- 172.21.2.101:0/2705765228 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v52) v1 ==== 80+0+4513 (secure 0 0 0) 0x7ff23c00bd80 con 0x7ff24008a740 2021-02-10T11:38:57.595 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:38:57.596 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:38:57.596 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:38:57.596 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:57.596 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:38:57.596 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:38:57.596 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:38:57.596 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:38:57.596 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:38:57.596 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:38:57.596 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:38:57.596 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:57.596 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:57.597 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:57.597 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:38:57.597 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:38:57.597 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:38:57.597 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:38:57.597 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:38:57.597 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:38:57.597 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:38:57.597 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:57.597 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:57.597 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:57.597 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:38:57.597 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:38:57.598 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:38:57.598 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:38:57.598 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:38:57.598 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:38:57.598 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:57.598 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:57.598 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:57.598 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:38:57.598 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:38:57.598 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:38:57.598 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:38:57.598 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:38:57.599 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:38:57.599 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:57.599 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:57.599 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:57.599 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:38:57.599 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:38:57.599 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:38:57.599 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:38:57.599 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:38:57.599 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:38:57.599 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:57.599 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:57.600 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:57.600 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:38:57.600 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:38:57.600 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:38:57.600 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:38:57.600 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:38:57.600 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:38:57.600 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:57.600 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:57.600 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:57.600 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:38:57.600 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:38:57.600 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:38:57.601 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:38:57.601 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:38:57.601 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:38:57.601 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:57.601 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:57.601 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:57.601 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:38:57.601 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:38:57.601 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:38:57.601 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:38:57.601 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:38:57.601 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:38:57.601 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:57.602 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:57.602 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:57.602 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:38:57.602 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:38:57.602 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:38:57.602 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:38:57.602 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:38:57.602 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:38:57.602 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:57.602 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:57.602 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:57.602 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:38:57.603 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:38:57.603 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:38:57.603 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:38:57.603 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:38:57.603 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:38:57.603 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:57.603 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:57.603 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:57.603 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:38:57.603 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:38:57.603 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:38:57.603 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:38:57.603 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:38:57.604 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:38:57.604 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:57.604 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:57.604 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:57.604 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:38:57.604 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:38:57.604 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:38:57.604 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:38:57.604 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:38:57.604 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:38:57.604 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:57.604 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:57.605 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:57.605 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:38:57.605 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:38:57.605 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:38:57.605 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:38:57.605 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:38:57.605 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:38:57.605 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:57.605 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:57.605 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:57.605 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:38:57.605 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:38:57.605 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:38:57.606 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:38:57.606 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:38:57.606 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:38:57.606 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:57.606 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:57.606 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:57.606 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:38:57.606 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:38:57.606 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:38:57.606 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:38:57.606 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:38:57.607 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:38:57.607 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:38:57.607 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:57.607 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:57.607 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:57.607 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:38:57.607 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:38:57.607 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:38:57.607 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:38:57.607 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:38:57.607 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:38:57.607 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:57.607 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:38:57.608 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:57.608 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:38:57.608 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:38:57.608 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:38:57.608 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:38:57.608 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:38:57.608 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:38:57.608 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:57.608 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:57.608 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:38:57.608 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:38:57.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.593 7ff247a4c700 1 -- 172.21.2.101:0/2705765228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2280186d0 msgr2=0x7ff22801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:57.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.593 7ff247a4c700 1 --2- 172.21.2.101:0/2705765228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2280186d0 0x7ff22801abd0 crc :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:57.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.593 7ff247a4c700 1 -- 172.21.2.101:0/2705765228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff24008a740 msgr2=0x7ff24008ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:57.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.593 7ff247a4c700 1 --2- 172.21.2.101:0/2705765228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff24008a740 0x7ff24008ab40 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 rx=0x7ff23c009760 tx=0x7ff23c00b990).stop 2021-02-10T11:38:57.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.593 7ff247a4c700 1 -- 172.21.2.101:0/2705765228 shutdown_connections 2021-02-10T11:38:57.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.593 7ff247a4c700 1 --2- 172.21.2.101:0/2705765228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2280186d0 0x7ff22801abd0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:57.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.593 7ff247a4c700 1 --2- 172.21.2.101:0/2705765228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2400913c0 0x7ff240194fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:57.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.593 7ff247a4c700 1 --2- 172.21.2.101:0/2705765228 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff24008b160 0x7ff240194990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:57.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.593 7ff247a4c700 1 --2- 172.21.2.101:0/2705765228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff24008a740 0x7ff24008ab40 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:57.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.593 7ff247a4c700 1 -- 172.21.2.101:0/2705765228 >> 172.21.2.101:0/2705765228 conn(0x7ff24019c560 msgr2=0x7ff2400ebe30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:57.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.594 7ff247a4c700 1 -- 172.21.2.101:0/2705765228 shutdown_connections 2021-02-10T11:38:57.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:57.594 7ff247a4c700 1 -- 172.21.2.101:0/2705765228 wait complete. 2021-02-10T11:38:57.611 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:38:58.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.017 7f9e26a76700 1 Processor -- start 2021-02-10T11:38:58.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.017 7f9e26a76700 1 -- start start 2021-02-10T11:38:58.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.017 7f9e26a76700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9e201b0470 0x7f9e201b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:58.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.017 7f9e26a76700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f9e200f20e0 0x7f9e200f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:58.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.017 7f9e26a76700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e200f0b50 con 0x7f9e200ef610 2021-02-10T11:38:58.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.017 7f9e26a76700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e200f0820 con 0x7f9e201b0470 2021-02-10T11:38:58.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.017 7f9e26a76700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e200f24e0 con 0x7f9e200f20e0 2021-02-10T11:38:58.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.018 7f9e1ffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f9e200ef610 0x7f9e200efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55610/0 (socket says 172.21.2.101:55610) 2021-02-10T11:38:58.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.018 7f9e1ffff700 1 -- 172.21.2.101:0/2110536107 learned_addr learned my addr 172.21.2.101:0/2110536107 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:38:58.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.018 7f9e24892700 1 --2- 172.21.2.101:0/2110536107 >> v2:172.21.2.101:3300/0 conn(0x7f9e201b0470 0x7f9e201b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:58.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.018 7f9e24892700 1 -- 172.21.2.101:0/2110536107 >> v2:172.21.2.108:3300/0 conn(0x7f9e200f20e0 msgr2=0x7f9e200f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:58.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.018 7f9e24892700 1 --2- 172.21.2.101:0/2110536107 >> v2:172.21.2.108:3300/0 conn(0x7f9e200f20e0 0x7f9e200f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:58.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.018 7f9e24892700 1 -- 172.21.2.101:0/2110536107 >> v1:172.21.2.101:6789/0 conn(0x7f9e200ef610 legacy=0x7f9e200efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:58.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.018 7f9e1f7fe700 1 -- 172.21.2.101:0/2110536107 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556121621 0 0) 0x7f9e10000ce0 con 0x7f9e200ef610 2021-02-10T11:38:58.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.018 7f9e24892700 1 -- 172.21.2.101:0/2110536107 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e200e52e0 con 0x7f9e201b0470 2021-02-10T11:38:58.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.018 7f9e24892700 1 --2- 172.21.2.101:0/2110536107 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9e201b0470 0x7f9e201b0870 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 rx=0x7f9e08006650 tx=0x7f9e08009a00).ready entity=mon.0 client_cookie=f3b47ba34d7433e1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:58.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.019 7f9e1f7fe700 1 -- 172.21.2.101:0/2110536107 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9e080009b0 con 0x7f9e201b0470 2021-02-10T11:38:58.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.019 7f9e1f7fe700 1 -- 172.21.2.101:0/2110536107 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0800c0c0 con 0x7f9e201b0470 2021-02-10T11:38:58.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.019 7f9e1f7fe700 1 -- 172.21.2.101:0/2110536107 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9e0800c5a0 con 0x7f9e201b0470 2021-02-10T11:38:58.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.019 7f9e26a76700 1 -- 172.21.2.101:0/2110536107 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9e201b0470 msgr2=0x7f9e201b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:58.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.019 7f9e26a76700 1 --2- 172.21.2.101:0/2110536107 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9e201b0470 0x7f9e201b0870 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 rx=0x7f9e08006650 tx=0x7f9e08009a00).stop 2021-02-10T11:38:58.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.019 7f9e26a76700 1 -- 172.21.2.101:0/2110536107 shutdown_connections 2021-02-10T11:38:58.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.019 7f9e26a76700 1 --2- 172.21.2.101:0/2110536107 >> v2:172.21.2.108:3300/0 conn(0x7f9e200f20e0 0x7f9e200f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:58.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.019 7f9e26a76700 1 --2- 172.21.2.101:0/2110536107 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9e201b0470 0x7f9e201b0870 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:58.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.019 7f9e26a76700 1 -- 172.21.2.101:0/2110536107 >> 172.21.2.101:0/2110536107 conn(0x7f9e200ce3e0 msgr2=0x7f9e200e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:58.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.020 7f9e26a76700 1 -- 172.21.2.101:0/2110536107 shutdown_connections 2021-02-10T11:38:58.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.020 7f9e26a76700 1 -- 172.21.2.101:0/2110536107 wait complete. 2021-02-10T11:38:58.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.034 7f9e26a76700 1 Processor -- start 2021-02-10T11:38:58.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.034 7f9e26a76700 1 -- start start 2021-02-10T11:38:58.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.035 7f9e26a76700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9e200aa8d0 0x7f9e2019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:58.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.035 7f9e26a76700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9e2019e9d0 0x7f9e2019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:58.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.035 7f9e26a76700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9e2019ad40 0x7f9e2019b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:58.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.035 7f9e26a76700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2019b1f0 con 0x7f9e2019ad40 2021-02-10T11:38:58.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.035 7f9e26a76700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e20109b30 con 0x7f9e2019e9d0 2021-02-10T11:38:58.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.035 7f9e26a76700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e20099620 con 0x7f9e200aa8d0 2021-02-10T11:38:58.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.035 7f9e25093700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9e2019ad40 0x7f9e2019b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:58.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.035 7f9e25093700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9e2019ad40 0x7f9e2019b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55126/0 (socket says 172.21.2.101:55126) 2021-02-10T11:38:58.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.035 7f9e25093700 1 -- 172.21.2.101:0/3340435756 learned_addr learned my addr 172.21.2.101:0/3340435756 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:58.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.035 7f9e24892700 1 --2- 172.21.2.101:0/3340435756 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9e200aa8d0 0x7f9e2019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:58.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.035 7f9e25093700 1 -- 172.21.2.101:0/3340435756 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9e200aa8d0 msgr2=0x7f9e2019a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:58.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.035 7f9e25093700 1 --2- 172.21.2.101:0/3340435756 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9e200aa8d0 0x7f9e2019a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:58.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.035 7f9e25093700 1 -- 172.21.2.101:0/3340435756 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9e2019e9d0 msgr2=0x7f9e2019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:58.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.035 7f9e1ffff700 1 --2- 172.21.2.101:0/3340435756 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9e2019e9d0 0x7f9e2019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:58.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.035 7f9e25093700 1 --2- 172.21.2.101:0/3340435756 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9e2019e9d0 0x7f9e2019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:58.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.035 7f9e25093700 1 -- 172.21.2.101:0/3340435756 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e20109640 con 0x7f9e2019ad40 2021-02-10T11:38:58.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.035 7f9e1ffff700 1 --2- 172.21.2.101:0/3340435756 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9e2019e9d0 0x7f9e2019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:38:58.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.035 7f9e25093700 1 --2- 172.21.2.101:0/3340435756 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9e2019ad40 0x7f9e2019b770 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 rx=0x7f9e1800b8c0 tx=0x7f9e1800bc60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:58.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.036 7f9e1d7fa700 1 -- 172.21.2.101:0/3340435756 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9e18003020 con 0x7f9e2019ad40 2021-02-10T11:38:58.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.036 7f9e1d7fa700 1 -- 172.21.2.101:0/3340435756 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e18005a80 con 0x7f9e2019ad40 2021-02-10T11:38:58.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.036 7f9e1d7fa700 1 -- 172.21.2.101:0/3340435756 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9e18006030 con 0x7f9e2019ad40 2021-02-10T11:38:58.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.036 7f9e26a76700 1 -- 172.21.2.101:0/3340435756 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e20109640 con 0x7f9e2019ad40 2021-02-10T11:38:58.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.036 7f9e26a76700 1 -- 172.21.2.101:0/3340435756 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e2019c080 con 0x7f9e2019ad40 2021-02-10T11:38:58.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.036 7f9e1d7fa700 1 -- 172.21.2.101:0/3340435756 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9e18006500 con 0x7f9e2019ad40 2021-02-10T11:38:58.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.037 7f9e1d7fa700 1 --2- 172.21.2.101:0/3340435756 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9e040186c0 0x7f9e0401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:58.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.037 7f9e1d7fa700 1 -- 172.21.2.101:0/3340435756 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f9e180325f0 con 0x7f9e2019ad40 2021-02-10T11:38:58.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.037 7f9e24892700 1 --2- 172.21.2.101:0/3340435756 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9e040186c0 0x7f9e0401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:58.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.037 7f9e24892700 1 --2- 172.21.2.101:0/3340435756 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9e040186c0 0x7f9e0401abc0 crc :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:58.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.067 7f9e26a76700 1 -- 172.21.2.101:0/3340435756 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e200f0b50 con 0x7f9e2019ad40 2021-02-10T11:38:58.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.069 7f9e1d7fa700 1 -- 172.21.2.101:0/3340435756 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9e180061d0 con 0x7f9e2019ad40 2021-02-10T11:38:58.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.194 7f9e26a76700 1 -- 172.21.2.101:0/3340435756 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e200f0b50 con 0x7f9e2019ad40 2021-02-10T11:38:58.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.194 7f9e1d7fa700 1 -- 172.21.2.101:0/3340435756 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+11145 (secure 0 0 0) 0x7f9e18005e00 con 0x7f9e2019ad40 2021-02-10T11:38:58.198 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:38:58.199 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":89,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:38:56.954377","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:38:58.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.196 7f9e26a76700 1 -- 172.21.2.101:0/3340435756 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9e040186c0 msgr2=0x7f9e0401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:58.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.196 7f9e26a76700 1 --2- 172.21.2.101:0/3340435756 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9e040186c0 0x7f9e0401abc0 crc :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:58.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.196 7f9e26a76700 1 -- 172.21.2.101:0/3340435756 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9e2019ad40 msgr2=0x7f9e2019b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:58.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.196 7f9e26a76700 1 --2- 172.21.2.101:0/3340435756 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9e2019ad40 0x7f9e2019b770 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 rx=0x7f9e1800b8c0 tx=0x7f9e1800bc60).stop 2021-02-10T11:38:58.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.196 7f9e26a76700 1 -- 172.21.2.101:0/3340435756 shutdown_connections 2021-02-10T11:38:58.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.196 7f9e26a76700 1 --2- 172.21.2.101:0/3340435756 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9e2019ad40 0x7f9e2019b770 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:58.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.196 7f9e26a76700 1 --2- 172.21.2.101:0/3340435756 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9e2019e9d0 0x7f9e2019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:58.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.196 7f9e26a76700 1 --2- 172.21.2.101:0/3340435756 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9e040186c0 0x7f9e0401abc0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:58.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.196 7f9e26a76700 1 --2- 172.21.2.101:0/3340435756 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9e200aa8d0 0x7f9e2019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:58.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.196 7f9e26a76700 1 -- 172.21.2.101:0/3340435756 >> 172.21.2.101:0/3340435756 conn(0x7f9e2019f7b0 msgr2=0x7f9e2018f380 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:58.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.196 7f9e26a76700 1 -- 172.21.2.101:0/3340435756 shutdown_connections 2021-02-10T11:38:58.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.196 7f9e26a76700 1 -- 172.21.2.101:0/3340435756 wait complete. 2021-02-10T11:38:58.209 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-10T11:38:58.209 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:38:58.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.621 7f660312a700 1 Processor -- start 2021-02-10T11:38:58.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.621 7f660312a700 1 -- start start 2021-02-10T11:38:58.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.621 7f660312a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f65fc0d1c70 0x7f65fc0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:58.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.621 7f660312a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65fc0d0170 con 0x7f65fc0cf580 2021-02-10T11:38:58.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.621 7f660312a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65fc0fe1c0 con 0x7f65fc0d1870 2021-02-10T11:38:58.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.621 7f660312a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65fc1b0cb0 con 0x7f65fc0d1c70 2021-02-10T11:38:58.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.621 7f6601747700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f65fc0cf580 0x7f65fc0cf980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55628/0 (socket says 172.21.2.101:55628) 2021-02-10T11:38:58.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.621 7f6601747700 1 -- 172.21.2.101:0/2226111684 learned_addr learned my addr 172.21.2.101:0/2226111684 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:38:58.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.621 7f65fbfff700 1 --2- 172.21.2.101:0/2226111684 >> v2:172.21.2.101:3300/0 conn(0x7f65fc0d1c70 0x7f65fc0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:58.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.621 7f65fb7fe700 1 -- 172.21.2.101:0/2226111684 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3230971281 0 0) 0x7f65f4000c10 con 0x7f65fc0cf580 2021-02-10T11:38:58.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.621 7f65fb7fe700 1 -- 172.21.2.101:0/2226111684 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65e8002fe0 con 0x7f65fc0cf580 2021-02-10T11:38:58.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.621 7f65fbfff700 1 -- 172.21.2.101:0/2226111684 >> v1:172.21.2.108:6789/0 conn(0x7f65fc0d1870 legacy=0x7f65fc1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:58.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.621 7f65fbfff700 1 -- 172.21.2.101:0/2226111684 >> v1:172.21.2.101:6789/0 conn(0x7f65fc0cf580 legacy=0x7f65fc0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:58.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.622 7f65fbfff700 1 -- 172.21.2.101:0/2226111684 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65fc109a60 con 0x7f65fc0d1c70 2021-02-10T11:38:58.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.622 7f65fbfff700 1 --2- 172.21.2.101:0/2226111684 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65fc0d1c70 0x7f65fc0cef60 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 rx=0x7f65ec0069d0 tx=0x7f65ec009e40).ready entity=mon.0 client_cookie=802e05f09a5a180b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:58.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.622 7f65fb7fe700 1 -- 172.21.2.101:0/2226111684 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f65ec00b510 con 0x7f65fc0d1c70 2021-02-10T11:38:58.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.622 7f65fb7fe700 1 -- 172.21.2.101:0/2226111684 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ec00b890 con 0x7f65fc0d1c70 2021-02-10T11:38:58.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.622 7f65fb7fe700 1 -- 172.21.2.101:0/2226111684 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f65ec00bcf0 con 0x7f65fc0d1c70 2021-02-10T11:38:58.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.622 7f660312a700 1 -- 172.21.2.101:0/2226111684 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65fc0d1c70 msgr2=0x7f65fc0cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:58.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.622 7f660312a700 1 --2- 172.21.2.101:0/2226111684 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65fc0d1c70 0x7f65fc0cef60 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 rx=0x7f65ec0069d0 tx=0x7f65ec009e40).stop 2021-02-10T11:38:58.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.622 7f660312a700 1 -- 172.21.2.101:0/2226111684 shutdown_connections 2021-02-10T11:38:58.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.622 7f660312a700 1 --2- 172.21.2.101:0/2226111684 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65fc0d1c70 0x7f65fc0cef60 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:58.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.622 7f660312a700 1 -- 172.21.2.101:0/2226111684 >> 172.21.2.101:0/2226111684 conn(0x7f65fc0ce420 msgr2=0x7f65fc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:58.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.623 7f660312a700 1 -- 172.21.2.101:0/2226111684 shutdown_connections 2021-02-10T11:38:58.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.623 7f660312a700 1 -- 172.21.2.101:0/2226111684 wait complete. 2021-02-10T11:38:58.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.637 7f660312a700 1 Processor -- start 2021-02-10T11:38:58.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.637 7f660312a700 1 -- start start 2021-02-10T11:38:58.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.638 7f660312a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65fc19e410 0x7f65fc08a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:58.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.638 7f660312a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65fc083980 0x7f65fc083d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:58.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.638 7f660312a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65fc0baa30 0x7f65fc0bae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:58.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.638 7f660312a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc0ba4b0 con 0x7f65fc19e410 2021-02-10T11:38:58.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.638 7f660312a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc0843a0 con 0x7f65fc0baa30 2021-02-10T11:38:58.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.638 7f660312a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc084710 con 0x7f65fc083980 2021-02-10T11:38:58.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.638 7f6600f46700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65fc19e410 0x7f65fc08a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:58.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.638 7f6600f46700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65fc19e410 0x7f65fc08a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55138/0 (socket says 172.21.2.101:55138) 2021-02-10T11:38:58.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.638 7f6600f46700 1 -- 172.21.2.101:0/1926009711 learned_addr learned my addr 172.21.2.101:0/1926009711 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:58.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.638 7f65fbfff700 1 --2- 172.21.2.101:0/1926009711 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65fc083980 0x7f65fc083d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:58.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.638 7f6601747700 1 --2- 172.21.2.101:0/1926009711 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65fc0baa30 0x7f65fc0bae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:58.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.638 7f6600f46700 1 -- 172.21.2.101:0/1926009711 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65fc083980 msgr2=0x7f65fc083d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:58.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.638 7f6600f46700 1 --2- 172.21.2.101:0/1926009711 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65fc083980 0x7f65fc083d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:58.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.638 7f6600f46700 1 -- 172.21.2.101:0/1926009711 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65fc0baa30 msgr2=0x7f65fc0bae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:58.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.639 7f6600f46700 1 --2- 172.21.2.101:0/1926009711 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65fc0baa30 0x7f65fc0bae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:58.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.639 7f6600f46700 1 -- 172.21.2.101:0/1926009711 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65fc0bb450 con 0x7f65fc19e410 2021-02-10T11:38:58.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.639 7f65fbfff700 1 --2- 172.21.2.101:0/1926009711 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65fc083980 0x7f65fc083d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:38:58.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.639 7f6601747700 1 --2- 172.21.2.101:0/1926009711 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65fc0baa30 0x7f65fc0bae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:38:58.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.639 7f6600f46700 1 --2- 172.21.2.101:0/1926009711 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65fc19e410 0x7f65fc08a420 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 rx=0x7f65e400b5b0 tx=0x7f65e400b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:58.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.639 7f65f97fa700 1 -- 172.21.2.101:0/1926009711 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f65e400bd40 con 0x7f65fc19e410 2021-02-10T11:38:58.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.639 7f65f97fa700 1 -- 172.21.2.101:0/1926009711 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e4000e20 con 0x7f65fc19e410 2021-02-10T11:38:58.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.639 7f65f97fa700 1 -- 172.21.2.101:0/1926009711 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f65e40054a0 con 0x7f65fc19e410 2021-02-10T11:38:58.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.639 7f660312a700 1 -- 172.21.2.101:0/1926009711 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65fc0bb450 con 0x7f65fc19e410 2021-02-10T11:38:58.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.639 7f660312a700 1 -- 172.21.2.101:0/1926009711 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65fc085f20 con 0x7f65fc19e410 2021-02-10T11:38:58.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.640 7f65f97fa700 1 -- 172.21.2.101:0/1926009711 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f65e4006030 con 0x7f65fc19e410 2021-02-10T11:38:58.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.640 7f65f97fa700 1 --2- 172.21.2.101:0/1926009711 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65e00186c0 0x7f65e001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:58.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.640 7f65f97fa700 1 -- 172.21.2.101:0/1926009711 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f65e40320a0 con 0x7f65fc19e410 2021-02-10T11:38:58.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.640 7f65fbfff700 1 --2- 172.21.2.101:0/1926009711 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65e00186c0 0x7f65e001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:58.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.640 7f65fbfff700 1 --2- 172.21.2.101:0/1926009711 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65e00186c0 0x7f65e001abc0 crc :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:58.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.670 7f660312a700 1 -- 172.21.2.101:0/1926009711 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65fc082a70 con 0x7f65fc19e410 2021-02-10T11:38:58.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.673 7f65f97fa700 1 -- 172.21.2.101:0/1926009711 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f65e4028e60 con 0x7f65fc19e410 2021-02-10T11:38:58.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.801 7f660312a700 1 -- 172.21.2.101:0/1926009711 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f65fc082a70 con 0x7f65fc19e410 2021-02-10T11:38:58.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.801 7f65f97fa700 1 -- 172.21.2.101:0/1926009711 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v89) v1 ==== 92+0+10 (secure 0 0 0) 0x7f65e4028e60 con 0x7f65fc19e410 2021-02-10T11:38:58.806 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:38:58.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.803 7f660312a700 1 -- 172.21.2.101:0/1926009711 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65e00186c0 msgr2=0x7f65e001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:58.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.803 7f660312a700 1 --2- 172.21.2.101:0/1926009711 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65e00186c0 0x7f65e001abc0 crc :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:58.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.803 7f660312a700 1 -- 172.21.2.101:0/1926009711 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65fc19e410 msgr2=0x7f65fc08a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:58.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.803 7f660312a700 1 --2- 172.21.2.101:0/1926009711 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65fc19e410 0x7f65fc08a420 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 rx=0x7f65e400b5b0 tx=0x7f65e400b950).stop 2021-02-10T11:38:58.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.804 7f660312a700 1 -- 172.21.2.101:0/1926009711 shutdown_connections 2021-02-10T11:38:58.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.804 7f660312a700 1 --2- 172.21.2.101:0/1926009711 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65e00186c0 0x7f65e001abc0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:58.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.804 7f660312a700 1 --2- 172.21.2.101:0/1926009711 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65fc083980 0x7f65fc083d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:58.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.804 7f660312a700 1 --2- 172.21.2.101:0/1926009711 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65fc0baa30 0x7f65fc0bae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:58.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.804 7f660312a700 1 --2- 172.21.2.101:0/1926009711 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65fc19e410 0x7f65fc08a420 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:58.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.804 7f660312a700 1 -- 172.21.2.101:0/1926009711 >> 172.21.2.101:0/1926009711 conn(0x7f65fc0bc0c0 msgr2=0x7f65fc082ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:58.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.804 7f660312a700 1 -- 172.21.2.101:0/1926009711 shutdown_connections 2021-02-10T11:38:58.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:58.804 7f660312a700 1 -- 172.21.2.101:0/1926009711 wait complete. 2021-02-10T11:38:58.816 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --format=json-pretty osd lspools 2021-02-10T11:38:59.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.224 7fdd2690a700 1 Processor -- start 2021-02-10T11:38:59.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.224 7fdd2690a700 1 -- start start 2021-02-10T11:38:59.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.224 7fdd2690a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd200d1870 0x7fdd201b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:59.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.224 7fdd2690a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fdd201b0a10 0x7fdd200cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:59.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.224 7fdd2690a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdd200d1c70 0x7fdd200cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:59.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.224 7fdd2690a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd200fe230 con 0x7fdd200d1c70 2021-02-10T11:38:59.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.224 7fdd2690a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd200e5ed0 con 0x7fdd201b0a10 2021-02-10T11:38:59.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.224 7fdd2690a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd200f0b10 con 0x7fdd200d1870 2021-02-10T11:38:59.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.225 7fdd24f27700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdd200d1c70 0x7fdd200cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:59.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.225 7fdd24f27700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdd200d1c70 0x7fdd200cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55144/0 (socket says 172.21.2.101:55144) 2021-02-10T11:38:59.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.225 7fdd24f27700 1 -- 172.21.2.101:0/4243932250 learned_addr learned my addr 172.21.2.101:0/4243932250 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:59.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.225 7fdd24f27700 1 -- 172.21.2.101:0/4243932250 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd200d1870 msgr2=0x7fdd201b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:38:59.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.225 7fdd24f27700 1 --2- 172.21.2.101:0/4243932250 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd200d1870 0x7fdd201b03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:59.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.225 7fdd24f27700 1 -- 172.21.2.101:0/4243932250 >> v2:172.21.2.108:3300/0 conn(0x7fdd201b0a10 msgr2=0x7fdd200cf330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:38:59.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.225 7fdd24f27700 1 --2- 172.21.2.101:0/4243932250 >> v2:172.21.2.108:3300/0 conn(0x7fdd201b0a10 0x7fdd200cf330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:59.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.225 7fdd24f27700 1 -- 172.21.2.101:0/4243932250 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd200cff70 con 0x7fdd200d1c70 2021-02-10T11:38:59.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.225 7fdd24f27700 1 --2- 172.21.2.101:0/4243932250 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd200d1c70 0x7fdd200cf950 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 rx=0x7fdd18006f00 tx=0x7fdd18009f10).ready entity=mon.0 client_cookie=e570f64ba0b391cf server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:59.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.225 7fdd1effd700 1 -- 172.21.2.101:0/4243932250 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd1800b3b0 con 0x7fdd200d1c70 2021-02-10T11:38:59.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.225 7fdd1effd700 1 -- 172.21.2.101:0/4243932250 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1800b630 con 0x7fdd200d1c70 2021-02-10T11:38:59.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.225 7fdd1effd700 1 -- 172.21.2.101:0/4243932250 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd1800bb50 con 0x7fdd200d1c70 2021-02-10T11:38:59.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.225 7fdd2690a700 1 -- 172.21.2.101:0/4243932250 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd200d1c70 msgr2=0x7fdd200cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:59.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.225 7fdd2690a700 1 --2- 172.21.2.101:0/4243932250 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd200d1c70 0x7fdd200cf950 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 rx=0x7fdd18006f00 tx=0x7fdd18009f10).stop 2021-02-10T11:38:59.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.226 7fdd2690a700 1 -- 172.21.2.101:0/4243932250 shutdown_connections 2021-02-10T11:38:59.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.226 7fdd2690a700 1 --2- 172.21.2.101:0/4243932250 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd200d1c70 0x7fdd200cf950 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:59.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.226 7fdd2690a700 1 --2- 172.21.2.101:0/4243932250 >> v2:172.21.2.108:3300/0 conn(0x7fdd201b0a10 0x7fdd200cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:59.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.226 7fdd2690a700 1 --2- 172.21.2.101:0/4243932250 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd200d1870 0x7fdd201b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:59.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.226 7fdd2690a700 1 -- 172.21.2.101:0/4243932250 >> 172.21.2.101:0/4243932250 conn(0x7fdd200ce450 msgr2=0x7fdd200d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:59.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.226 7fdd2690a700 1 -- 172.21.2.101:0/4243932250 shutdown_connections 2021-02-10T11:38:59.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.226 7fdd2690a700 1 -- 172.21.2.101:0/4243932250 wait complete. 2021-02-10T11:38:59.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.241 7fdd2690a700 1 Processor -- start 2021-02-10T11:38:59.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.241 7fdd2690a700 1 -- start start 2021-02-10T11:38:59.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.241 7fdd2690a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd201b7cf0 0x7fdd2019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:59.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.241 7fdd2690a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd201b80f0 0x7fdd201b84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:59.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.241 7fdd2690a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd2019da20 0x7fdd2019de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:59.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.241 7fdd2690a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd200a77c0 con 0x7fdd201b7cf0 2021-02-10T11:38:59.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.241 7fdd2690a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2019d4a0 con 0x7fdd2019da20 2021-02-10T11:38:59.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.241 7fdd2690a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd200954f0 con 0x7fdd201b80f0 2021-02-10T11:38:59.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.242 7fdd1ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd201b7cf0 0x7fdd2019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:59.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.242 7fdd1ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd201b7cf0 0x7fdd2019cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55152/0 (socket says 172.21.2.101:55152) 2021-02-10T11:38:59.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.242 7fdd1ffff700 1 -- 172.21.2.101:0/97618482 learned_addr learned my addr 172.21.2.101:0/97618482 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:59.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.242 7fdd1f7fe700 1 --2- 172.21.2.101:0/97618482 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd201b80f0 0x7fdd201b84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:59.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.242 7fdd24f27700 1 --2- 172.21.2.101:0/97618482 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd2019da20 0x7fdd2019de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:59.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.242 7fdd1ffff700 1 -- 172.21.2.101:0/97618482 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd201b80f0 msgr2=0x7fdd201b84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:59.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.242 7fdd1ffff700 1 --2- 172.21.2.101:0/97618482 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd201b80f0 0x7fdd201b84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:59.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.242 7fdd1ffff700 1 -- 172.21.2.101:0/97618482 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd2019da20 msgr2=0x7fdd2019de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:59.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.242 7fdd1ffff700 1 --2- 172.21.2.101:0/97618482 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd2019da20 0x7fdd2019de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:59.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.242 7fdd1ffff700 1 -- 172.21.2.101:0/97618482 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd201b8d60 con 0x7fdd201b7cf0 2021-02-10T11:38:59.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.242 7fdd24f27700 1 --2- 172.21.2.101:0/97618482 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd2019da20 0x7fdd2019de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:38:59.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.242 7fdd1ffff700 1 --2- 172.21.2.101:0/97618482 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd201b7cf0 0x7fdd2019cdb0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 rx=0x7fdd1000b5c0 tx=0x7fdd1000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:59.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.242 7fdd1cff9700 1 -- 172.21.2.101:0/97618482 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd1000bd50 con 0x7fdd201b7cf0 2021-02-10T11:38:59.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.242 7fdd1cff9700 1 -- 172.21.2.101:0/97618482 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd10006030 con 0x7fdd201b7cf0 2021-02-10T11:38:59.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.242 7fdd2690a700 1 -- 172.21.2.101:0/97618482 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd201b8d60 con 0x7fdd201b7cf0 2021-02-10T11:38:59.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.242 7fdd2690a700 1 -- 172.21.2.101:0/97618482 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd201b8d60 con 0x7fdd201b7cf0 2021-02-10T11:38:59.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.242 7fdd1cff9700 1 -- 172.21.2.101:0/97618482 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd10006360 con 0x7fdd201b7cf0 2021-02-10T11:38:59.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.243 7fdd1cff9700 1 -- 172.21.2.101:0/97618482 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdd10006210 con 0x7fdd201b7cf0 2021-02-10T11:38:59.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.243 7fdd1cff9700 1 --2- 172.21.2.101:0/97618482 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd000186c0 0x7fdd0001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:59.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.243 7fdd1cff9700 1 -- 172.21.2.101:0/97618482 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fdd100324a0 con 0x7fdd201b7cf0 2021-02-10T11:38:59.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.243 7fdd1f7fe700 1 --2- 172.21.2.101:0/97618482 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd000186c0 0x7fdd0001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:59.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.243 7fdd1f7fe700 1 --2- 172.21.2.101:0/97618482 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd000186c0 0x7fdd0001abc0 crc :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:59.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.274 7fdd2690a700 1 -- 172.21.2.101:0/97618482 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd200ebc00 con 0x7fdd201b7cf0 2021-02-10T11:38:59.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.277 7fdd1cff9700 1 -- 172.21.2.101:0/97618482 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdd10006b80 con 0x7fdd201b7cf0 2021-02-10T11:38:59.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.401 7fdd2690a700 1 -- 172.21.2.101:0/97618482 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdd200f0b10 con 0x7fdd201b7cf0 2021-02-10T11:38:59.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.402 7fdd1cff9700 1 -- 172.21.2.101:0/97618482 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v89) v1 ==== 84+0+64 (secure 0 0 0) 0x7fdd1000bd50 con 0x7fdd201b7cf0 2021-02-10T11:38:59.406 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:38:59.406 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:38:59.406 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:38:59.406 INFO:teuthology.orchestra.run.gibba001.stdout: "poolnum": 1, 2021-02-10T11:38:59.406 INFO:teuthology.orchestra.run.gibba001.stdout: "poolname": "rbd" 2021-02-10T11:38:59.407 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:38:59.407 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:38:59.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.404 7fdd2690a700 1 -- 172.21.2.101:0/97618482 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd000186c0 msgr2=0x7fdd0001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:59.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.404 7fdd2690a700 1 --2- 172.21.2.101:0/97618482 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd000186c0 0x7fdd0001abc0 crc :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:59.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.404 7fdd2690a700 1 -- 172.21.2.101:0/97618482 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd201b7cf0 msgr2=0x7fdd2019cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:59.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.404 7fdd2690a700 1 --2- 172.21.2.101:0/97618482 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd201b7cf0 0x7fdd2019cdb0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 rx=0x7fdd1000b5c0 tx=0x7fdd1000b960).stop 2021-02-10T11:38:59.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.404 7fdd2690a700 1 -- 172.21.2.101:0/97618482 shutdown_connections 2021-02-10T11:38:59.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.404 7fdd2690a700 1 --2- 172.21.2.101:0/97618482 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd000186c0 0x7fdd0001abc0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:59.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.404 7fdd2690a700 1 --2- 172.21.2.101:0/97618482 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd201b80f0 0x7fdd201b84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:59.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.404 7fdd2690a700 1 --2- 172.21.2.101:0/97618482 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd2019da20 0x7fdd2019de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:59.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.404 7fdd2690a700 1 --2- 172.21.2.101:0/97618482 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd201b7cf0 0x7fdd2019cdb0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:59.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.404 7fdd2690a700 1 -- 172.21.2.101:0/97618482 >> 172.21.2.101:0/97618482 conn(0x7fdd200ed810 msgr2=0x7fdd20094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:59.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.404 7fdd2690a700 1 -- 172.21.2.101:0/97618482 shutdown_connections 2021-02-10T11:38:59.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.404 7fdd2690a700 1 -- 172.21.2.101:0/97618482 wait complete. 2021-02-10T11:38:59.416 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-02-10T11:38:59.417 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-02-10T11:38:59.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.826 7f2990b5c700 1 Processor -- start 2021-02-10T11:38:59.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.826 7f2990b5c700 1 -- start start 2021-02-10T11:38:59.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.827 7f2990b5c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f298c0d1870 0x7f298c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:59.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.827 7f2990b5c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f298c0cff70 0x7f298c109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:59.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.827 7f2990b5c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f298c0fe1c0 con 0x7f298c1b0ae0 2021-02-10T11:38:59.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.827 7f2990b5c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f298c109ad0 con 0x7f298c0d1870 2021-02-10T11:38:59.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.827 7f2990b5c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f298c109750 con 0x7f298c0cff70 2021-02-10T11:38:59.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.827 7f298a05d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f298c1b0ae0 0x7f298c108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55656/0 (socket says 172.21.2.101:55656) 2021-02-10T11:38:59.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.827 7f298a05d700 1 -- 172.21.2.101:0/403377120 learned_addr learned my addr 172.21.2.101:0/403377120 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:38:59.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.827 7f298a85e700 1 --2- 172.21.2.101:0/403377120 >> v2:172.21.2.101:3300/0 conn(0x7f298c0d1870 0x7f298c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:59.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.827 7f298b05f700 1 --2- 172.21.2.101:0/403377120 >> v2:172.21.2.108:3300/0 conn(0x7f298c0cff70 0x7f298c109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:59.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.827 7f298a85e700 1 -- 172.21.2.101:0/403377120 >> v2:172.21.2.108:3300/0 conn(0x7f298c0cff70 msgr2=0x7f298c109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:38:59.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.827 7f298a85e700 1 --2- 172.21.2.101:0/403377120 >> v2:172.21.2.108:3300/0 conn(0x7f298c0cff70 0x7f298c109130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:59.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.827 7f298a85e700 1 -- 172.21.2.101:0/403377120 >> v1:172.21.2.101:6789/0 conn(0x7f298c1b0ae0 legacy=0x7f298c108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:59.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.827 7f298a85e700 1 -- 172.21.2.101:0/403377120 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298c088070 con 0x7f298c0d1870 2021-02-10T11:38:59.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.827 7f298b05f700 1 --2- 172.21.2.101:0/403377120 >> v2:172.21.2.108:3300/0 conn(0x7f298c0cff70 0x7f298c109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:38:59.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.828 7f298a85e700 1 --2- 172.21.2.101:0/403377120 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f298c0d1870 0x7f298c1b04c0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 rx=0x7f297c006650 tx=0x7f297c009a80).ready entity=mon.0 client_cookie=d696536c74280f77 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:59.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.828 7f298985c700 1 -- 172.21.2.101:0/403377120 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f297c00c0c0 con 0x7f298c0d1870 2021-02-10T11:38:59.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.828 7f298985c700 1 -- 172.21.2.101:0/403377120 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297c00c3c0 con 0x7f298c0d1870 2021-02-10T11:38:59.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.828 7f298985c700 1 -- 172.21.2.101:0/403377120 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f297c00c810 con 0x7f298c0d1870 2021-02-10T11:38:59.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.828 7f2990b5c700 1 -- 172.21.2.101:0/403377120 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f298c0d1870 msgr2=0x7f298c1b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:59.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.828 7f2990b5c700 1 --2- 172.21.2.101:0/403377120 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f298c0d1870 0x7f298c1b04c0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 rx=0x7f297c006650 tx=0x7f297c009a80).stop 2021-02-10T11:38:59.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.829 7f2990b5c700 1 -- 172.21.2.101:0/403377120 shutdown_connections 2021-02-10T11:38:59.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.829 7f2990b5c700 1 --2- 172.21.2.101:0/403377120 >> v2:172.21.2.108:3300/0 conn(0x7f298c0cff70 0x7f298c109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:59.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.829 7f2990b5c700 1 --2- 172.21.2.101:0/403377120 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f298c0d1870 0x7f298c1b04c0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:38:59.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.829 7f2990b5c700 1 -- 172.21.2.101:0/403377120 >> 172.21.2.101:0/403377120 conn(0x7f298c0ce420 msgr2=0x7f298c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:38:59.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.829 7f2990b5c700 1 -- 172.21.2.101:0/403377120 shutdown_connections 2021-02-10T11:38:59.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.829 7f2990b5c700 1 -- 172.21.2.101:0/403377120 wait complete. 2021-02-10T11:38:59.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.843 7f2990b5c700 1 Processor -- start 2021-02-10T11:38:59.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.843 7f2990b5c700 1 -- start start 2021-02-10T11:38:59.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.844 7f2990b5c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f298c0c3820 0x7f298c0a7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:59.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.844 7f2990b5c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f298c1a44c0 0x7f298c1a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:59.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.844 7f2990b5c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f298c0863d0 0x7f298c0867d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:59.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.844 7f2990b5c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c085ed0 con 0x7f298c0863d0 2021-02-10T11:38:59.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.844 7f2990b5c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c0c0740 con 0x7f298c1a44c0 2021-02-10T11:38:59.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.844 7f2990b5c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f298c086df0 con 0x7f298c0c3820 2021-02-10T11:38:59.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.844 7f298b05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f298c0863d0 0x7f298c0867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:59.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.844 7f298b05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f298c0863d0 0x7f298c0867d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55168/0 (socket says 172.21.2.101:55168) 2021-02-10T11:38:59.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.844 7f298b05f700 1 -- 172.21.2.101:0/856708012 learned_addr learned my addr 172.21.2.101:0/856708012 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:38:59.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.844 7f298b05f700 1 -- 172.21.2.101:0/856708012 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f298c0c3820 msgr2=0x7f298c0a7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:59.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.844 7f298b05f700 1 --2- 172.21.2.101:0/856708012 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f298c0c3820 0x7f298c0a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:59.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.844 7f298b05f700 1 -- 172.21.2.101:0/856708012 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f298c1a44c0 msgr2=0x7f298c1a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:38:59.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.844 7f298b05f700 1 --2- 172.21.2.101:0/856708012 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f298c1a44c0 0x7f298c1a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:38:59.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.844 7f298b05f700 1 -- 172.21.2.101:0/856708012 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298c087010 con 0x7f298c0863d0 2021-02-10T11:38:59.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.845 7f298b05f700 1 --2- 172.21.2.101:0/856708012 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f298c0863d0 0x7f298c0867d0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 rx=0x7f2984009980 tx=0x7f298400bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:59.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.845 7f29837fe700 1 -- 172.21.2.101:0/856708012 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f298400c2f0 con 0x7f298c0863d0 2021-02-10T11:38:59.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.845 7f29837fe700 1 -- 172.21.2.101:0/856708012 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2984006790 con 0x7f298c0863d0 2021-02-10T11:38:59.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.845 7f29837fe700 1 -- 172.21.2.101:0/856708012 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f298400c7f0 con 0x7f298c0863d0 2021-02-10T11:38:59.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.845 7f2990b5c700 1 -- 172.21.2.101:0/856708012 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298c087010 con 0x7f298c0863d0 2021-02-10T11:38:59.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.845 7f2990b5c700 1 -- 172.21.2.101:0/856708012 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f298c083000 con 0x7f298c0863d0 2021-02-10T11:38:59.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.845 7f29837fe700 1 -- 172.21.2.101:0/856708012 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f29840011f0 con 0x7f298c0863d0 2021-02-10T11:38:59.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.845 7f29837fe700 1 --2- 172.21.2.101:0/856708012 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2970018730 0x7f297001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:38:59.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.845 7f29837fe700 1 -- 172.21.2.101:0/856708012 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f29840326c0 con 0x7f298c0863d0 2021-02-10T11:38:59.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.846 7f298a85e700 1 --2- 172.21.2.101:0/856708012 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2970018730 0x7f297001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:38:59.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.846 7f298a85e700 1 --2- 172.21.2.101:0/856708012 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2970018730 0x7f297001ac10 crc :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:38:59.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.876 7f2990b5c700 1 -- 172.21.2.101:0/856708012 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f298c085e50 con 0x7f298c0863d0 2021-02-10T11:38:59.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:38:59.879 7f29837fe700 1 -- 172.21.2.101:0/856708012 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2984000f40 con 0x7f298c0863d0 2021-02-10T11:39:00.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.009 7f2990b5c700 1 -- 172.21.2.101:0/856708012 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"} v 0) v1 -- 0x7f298c0fe1c0 con 0x7f298c0863d0 2021-02-10T11:39:00.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.424 7f29837fe700 1 -- 172.21.2.101:0/856708012 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"}]=0 pool 'cephfs_metadata' created v90) v1 ==== 133+0+0 (secure 0 0 0) 0x7f29840011f0 con 0x7f298c0863d0 2021-02-10T11:39:00.429 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' created 2021-02-10T11:39:00.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.427 7f2990b5c700 1 -- 172.21.2.101:0/856708012 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2970018730 msgr2=0x7f297001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:00.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.428 7f2990b5c700 1 --2- 172.21.2.101:0/856708012 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2970018730 0x7f297001ac10 crc :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:00.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.428 7f2990b5c700 1 -- 172.21.2.101:0/856708012 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f298c0863d0 msgr2=0x7f298c0867d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:00.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.428 7f2990b5c700 1 --2- 172.21.2.101:0/856708012 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f298c0863d0 0x7f298c0867d0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 rx=0x7f2984009980 tx=0x7f298400bcd0).stop 2021-02-10T11:39:00.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.428 7f2990b5c700 1 -- 172.21.2.101:0/856708012 shutdown_connections 2021-02-10T11:39:00.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.428 7f2990b5c700 1 --2- 172.21.2.101:0/856708012 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f298c0863d0 0x7f298c0867d0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:00.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.428 7f2990b5c700 1 --2- 172.21.2.101:0/856708012 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f298c1a44c0 0x7f298c1a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:00.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.428 7f2990b5c700 1 --2- 172.21.2.101:0/856708012 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2970018730 0x7f297001ac10 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:00.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.428 7f2990b5c700 1 --2- 172.21.2.101:0/856708012 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f298c0c3820 0x7f298c0a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:00.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.428 7f2990b5c700 1 -- 172.21.2.101:0/856708012 >> 172.21.2.101:0/856708012 conn(0x7f298c0872c0 msgr2=0x7f298c095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:00.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.428 7f2990b5c700 1 -- 172.21.2.101:0/856708012 shutdown_connections 2021-02-10T11:39:00.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.428 7f2990b5c700 1 -- 172.21.2.101:0/856708012 wait complete. 2021-02-10T11:39:00.440 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_data 8 2021-02-10T11:39:00.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.850 7f04385e9700 1 Processor -- start 2021-02-10T11:39:00.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.850 7f04385e9700 1 -- start start 2021-02-10T11:39:00.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.850 7f04385e9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f04300d1870 0x7f04301b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:00.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.850 7f04385e9700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f04300cff70 0x7f0430109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:00.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.850 7f04385e9700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0430109a60 con 0x7f04301b0ae0 2021-02-10T11:39:00.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.850 7f04385e9700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0430109750 con 0x7f04300cff70 2021-02-10T11:39:00.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.850 7f04385e9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0430088070 con 0x7f04300d1870 2021-02-10T11:39:00.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.850 7f0436c06700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f04300cff70 0x7f0430109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:00.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.850 7f0435c04700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f04301b0ae0 0x7f0430108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55666/0 (socket says 172.21.2.101:55666) 2021-02-10T11:39:00.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.850 7f0435c04700 1 -- 172.21.2.101:0/1260457644 learned_addr learned my addr 172.21.2.101:0/1260457644 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:00.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.851 7f0436405700 1 --2- 172.21.2.101:0/1260457644 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f04300d1870 0x7f04301b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:00.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.851 7f0436c06700 1 -- 172.21.2.101:0/1260457644 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f04300d1870 msgr2=0x7f04301b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:00.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.851 7f0436c06700 1 --2- 172.21.2.101:0/1260457644 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f04300d1870 0x7f04301b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:00.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.851 7f0436c06700 1 -- 172.21.2.101:0/1260457644 >> v1:172.21.2.101:6789/0 conn(0x7f04301b0ae0 legacy=0x7f0430108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:00.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.851 7f0436c06700 1 -- 172.21.2.101:0/1260457644 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0430088260 con 0x7f04300cff70 2021-02-10T11:39:00.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.851 7f0436c06700 1 --2- 172.21.2.101:0/1260457644 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04300cff70 0x7f0430109130 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 rx=0x7f0428006e30 tx=0x7f0428009e60).ready entity=mon.0 client_cookie=cf995c6c7ca3a24c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:00.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.851 7f0435403700 1 -- 172.21.2.101:0/1260457644 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0428006a50 con 0x7f04300cff70 2021-02-10T11:39:00.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.851 7f0435403700 1 -- 172.21.2.101:0/1260457644 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042800b300 con 0x7f04300cff70 2021-02-10T11:39:00.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.851 7f0435403700 1 -- 172.21.2.101:0/1260457644 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f042800b760 con 0x7f04300cff70 2021-02-10T11:39:00.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.851 7f04385e9700 1 -- 172.21.2.101:0/1260457644 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04300cff70 msgr2=0x7f0430109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:00.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.851 7f04385e9700 1 --2- 172.21.2.101:0/1260457644 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04300cff70 0x7f0430109130 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 rx=0x7f0428006e30 tx=0x7f0428009e60).stop 2021-02-10T11:39:00.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.852 7f04385e9700 1 -- 172.21.2.101:0/1260457644 shutdown_connections 2021-02-10T11:39:00.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.852 7f04385e9700 1 --2- 172.21.2.101:0/1260457644 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04300cff70 0x7f0430109130 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:00.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.852 7f04385e9700 1 --2- 172.21.2.101:0/1260457644 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f04300d1870 0x7f04301b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:00.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.852 7f04385e9700 1 -- 172.21.2.101:0/1260457644 >> 172.21.2.101:0/1260457644 conn(0x7f04300ce420 msgr2=0x7f04300d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:00.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.852 7f04385e9700 1 -- 172.21.2.101:0/1260457644 shutdown_connections 2021-02-10T11:39:00.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.852 7f04385e9700 1 -- 172.21.2.101:0/1260457644 wait complete. 2021-02-10T11:39:00.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.867 7f04385e9700 1 Processor -- start 2021-02-10T11:39:00.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.867 7f04385e9700 1 -- start start 2021-02-10T11:39:00.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.867 7f04385e9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f04300a2580 0x7f04300a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:00.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.867 7f04385e9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0430094c40 0x7f04301a75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:00.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.867 7f04385e9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0430095040 0x7f04301a7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:00.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.867 7f04385e9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0430188180 con 0x7f0430094c40 2021-02-10T11:39:00.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.867 7f04385e9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0430187e70 con 0x7f04300a2580 2021-02-10T11:39:00.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.867 7f04385e9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04301a8220 con 0x7f0430095040 2021-02-10T11:39:00.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.868 7f0435c04700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0430094c40 0x7f04301a75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:00.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.868 7f0435c04700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0430094c40 0x7f04301a75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55178/0 (socket says 172.21.2.101:55178) 2021-02-10T11:39:00.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.868 7f0435c04700 1 -- 172.21.2.101:0/880876643 learned_addr learned my addr 172.21.2.101:0/880876643 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:00.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.868 7f0435c04700 1 -- 172.21.2.101:0/880876643 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0430095040 msgr2=0x7f04301a7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:00.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.868 7f0435c04700 1 --2- 172.21.2.101:0/880876643 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0430095040 0x7f04301a7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:00.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.868 7f0435c04700 1 -- 172.21.2.101:0/880876643 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f04300a2580 msgr2=0x7f04300a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:00.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.868 7f0435c04700 1 --2- 172.21.2.101:0/880876643 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f04300a2580 0x7f04300a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:00.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.868 7f0435c04700 1 -- 172.21.2.101:0/880876643 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0430188700 con 0x7f0430094c40 2021-02-10T11:39:00.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.868 7f0435c04700 1 --2- 172.21.2.101:0/880876643 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0430094c40 0x7f04301a75e0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 rx=0x7f042400a4c0 tx=0x7f042400a860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:00.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.868 7f0422ffd700 1 -- 172.21.2.101:0/880876643 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f042400ac50 con 0x7f0430094c40 2021-02-10T11:39:00.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.869 7f0422ffd700 1 -- 172.21.2.101:0/880876643 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0424004140 con 0x7f0430094c40 2021-02-10T11:39:00.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.869 7f0422ffd700 1 -- 172.21.2.101:0/880876643 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0424004660 con 0x7f0430094c40 2021-02-10T11:39:00.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.869 7f04385e9700 1 -- 172.21.2.101:0/880876643 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0430188700 con 0x7f0430094c40 2021-02-10T11:39:00.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.869 7f04385e9700 1 -- 172.21.2.101:0/880876643 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0430188700 con 0x7f0430094c40 2021-02-10T11:39:00.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.869 7f0422ffd700 1 -- 172.21.2.101:0/880876643 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0424000ce0 con 0x7f0430094c40 2021-02-10T11:39:00.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.869 7f0422ffd700 1 --2- 172.21.2.101:0/880876643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04180186c0 0x7f041801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:00.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.869 7f0422ffd700 1 -- 172.21.2.101:0/880876643 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5107+0+0 (secure 0 0 0) 0x7f0424032390 con 0x7f0430094c40 2021-02-10T11:39:00.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.870 7f0436405700 1 --2- 172.21.2.101:0/880876643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04180186c0 0x7f041801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:00.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.870 7f0436405700 1 --2- 172.21.2.101:0/880876643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04180186c0 0x7f041801abc0 crc :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:00.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.900 7f04385e9700 1 -- 172.21.2.101:0/880876643 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0430082980 con 0x7f0430094c40 2021-02-10T11:39:00.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:00.903 7f0422ffd700 1 -- 172.21.2.101:0/880876643 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f04240009b0 con 0x7f0430094c40 2021-02-10T11:39:01.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.035 7f04385e9700 1 -- 172.21.2.101:0/880876643 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"} v 0) v1 -- 0x7f0430082980 con 0x7f0430094c40 2021-02-10T11:39:01.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.433 7f0422ffd700 1 -- 172.21.2.101:0/880876643 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"}]=0 pool 'cephfs_data' created v92) v1 ==== 125+0+0 (secure 0 0 0) 0x7f0424000ce0 con 0x7f0430094c40 2021-02-10T11:39:01.455 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' created 2021-02-10T11:39:01.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.453 7f04385e9700 1 -- 172.21.2.101:0/880876643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04180186c0 msgr2=0x7f041801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:01.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.453 7f04385e9700 1 --2- 172.21.2.101:0/880876643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04180186c0 0x7f041801abc0 crc :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:01.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.453 7f04385e9700 1 -- 172.21.2.101:0/880876643 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0430094c40 msgr2=0x7f04301a75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:01.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.453 7f04385e9700 1 --2- 172.21.2.101:0/880876643 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0430094c40 0x7f04301a75e0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 rx=0x7f042400a4c0 tx=0x7f042400a860).stop 2021-02-10T11:39:01.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.453 7f04385e9700 1 -- 172.21.2.101:0/880876643 shutdown_connections 2021-02-10T11:39:01.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.453 7f04385e9700 1 --2- 172.21.2.101:0/880876643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04180186c0 0x7f041801abc0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:01.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.453 7f04385e9700 1 --2- 172.21.2.101:0/880876643 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0430095040 0x7f04301a7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:01.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.453 7f04385e9700 1 --2- 172.21.2.101:0/880876643 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0430094c40 0x7f04301a75e0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:01.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.453 7f04385e9700 1 --2- 172.21.2.101:0/880876643 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f04300a2580 0x7f04300a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:01.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.453 7f04385e9700 1 -- 172.21.2.101:0/880876643 >> 172.21.2.101:0/880876643 conn(0x7f04300a2b30 msgr2=0x7f043009ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:01.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.453 7f04385e9700 1 -- 172.21.2.101:0/880876643 shutdown_connections 2021-02-10T11:39:01.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.453 7f04385e9700 1 -- 172.21.2.101:0/880876643 wait complete. 2021-02-10T11:39:01.465 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-02-10T11:39:01.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.879 7fddf8e54700 1 Processor -- start 2021-02-10T11:39:01.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.879 7fddf8e54700 1 -- start start 2021-02-10T11:39:01.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.879 7fddf8e54700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fddf40d1880 0x7fddf41b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:01.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.879 7fddf8e54700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf4109a70 con 0x7fddf41b0af0 2021-02-10T11:39:01.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.879 7fddf8e54700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf4088070 con 0x7fddf40cff80 2021-02-10T11:39:01.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.879 7fddf8e54700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf40e5f00 con 0x7fddf40d1880 2021-02-10T11:39:01.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.880 7fddf2312700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fddf41b0af0 0x7fddf4108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55680/0 (socket says 172.21.2.101:55680) 2021-02-10T11:39:01.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.880 7fddf2312700 1 -- 172.21.2.101:0/2176942981 learned_addr learned my addr 172.21.2.101:0/2176942981 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:01.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.880 7fddf2b13700 1 --2- 172.21.2.101:0/2176942981 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fddf40d1880 0x7fddf41b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:01.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.880 7fddf1b11700 1 -- 172.21.2.101:0/2176942981 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200924739 0 0) 0x7fdde8001180 con 0x7fddf41b0af0 2021-02-10T11:39:01.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.880 7fddf1b11700 1 -- 172.21.2.101:0/2176942981 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdddc002fe0 con 0x7fddf41b0af0 2021-02-10T11:39:01.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.881 7fddf1b11700 1 -- 172.21.2.101:0/2176942981 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2358793189 0 0) 0x7fdde8001180 con 0x7fddf41b0af0 2021-02-10T11:39:01.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.881 7fddf1b11700 1 -- 172.21.2.101:0/2176942981 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fddf40d1880 msgr2=0x7fddf41b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:01.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.881 7fddf1b11700 1 --2- 172.21.2.101:0/2176942981 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fddf40d1880 0x7fddf41b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:01.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.881 7fddf1b11700 1 -- 172.21.2.101:0/2176942981 >> v1:172.21.2.108:6789/0 conn(0x7fddf40cff80 legacy=0x7fddf4109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:01.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.881 7fddf1b11700 1 -- 172.21.2.101:0/2176942981 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf40d1c80 con 0x7fddf41b0af0 2021-02-10T11:39:01.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.881 7fddf1b11700 1 -- 172.21.2.101:0/2176942981 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fdde8001680 con 0x7fddf41b0af0 2021-02-10T11:39:01.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.881 7fddf1b11700 1 -- 172.21.2.101:0/2176942981 >> v1:172.21.2.101:6789/0 conn(0x7fddf41b0af0 legacy=0x7fddf4108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:01.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.881 7fddf1b11700 1 --2- 172.21.2.101:0/2176942981 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdddc003ae0 0x7fdddc006000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:01.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.881 7fddf1b11700 1 --2- 172.21.2.101:0/2176942981 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdddc006820 0x7fdddc008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:01.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.881 7fddf1b11700 1 --2- 172.21.2.101:0/2176942981 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdddc009640 0x7fdddc00bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:01.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.881 7fddf1b11700 1 -- 172.21.2.101:0/2176942981 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc00c360 con 0x7fdddc006820 2021-02-10T11:39:01.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.881 7fddf1b11700 1 -- 172.21.2.101:0/2176942981 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc00c580 con 0x7fdddc003ae0 2021-02-10T11:39:01.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.881 7fddf1b11700 1 -- 172.21.2.101:0/2176942981 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc0507b0 con 0x7fdddc009640 2021-02-10T11:39:01.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.881 7fddf2b13700 1 --2- 172.21.2.101:0/2176942981 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fddf40d1880 0x7fddf41b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:39:01.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.881 7fddf2312700 1 --2- 172.21.2.101:0/2176942981 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdddc006820 0x7fdddc008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:01.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.881 7fddf3314700 1 --2- 172.21.2.101:0/2176942981 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdddc009640 0x7fdddc00bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:01.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.881 7fddf2312700 1 -- 172.21.2.101:0/2176942981 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdddc009640 msgr2=0x7fdddc00bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:01.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.881 7fddf2312700 1 --2- 172.21.2.101:0/2176942981 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdddc009640 0x7fdddc00bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:01.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.882 7fddf2312700 1 -- 172.21.2.101:0/2176942981 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdddc003ae0 msgr2=0x7fdddc006000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:01.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.882 7fddf2312700 1 --2- 172.21.2.101:0/2176942981 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdddc003ae0 0x7fdddc006000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:01.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.882 7fddf2312700 1 -- 172.21.2.101:0/2176942981 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdddc050a10 con 0x7fdddc006820 2021-02-10T11:39:01.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.882 7fddf2312700 1 --2- 172.21.2.101:0/2176942981 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdddc006820 0x7fdddc008d40 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 rx=0x7fdde80014e0 tx=0x7fdde8007d20).ready entity=mon.0 client_cookie=7f0a5e3507366686 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:01.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.882 7fddf1b11700 1 -- 172.21.2.101:0/2176942981 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdde80070c0 con 0x7fdddc006820 2021-02-10T11:39:01.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.882 7fddf1b11700 1 -- 172.21.2.101:0/2176942981 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde80043a0 con 0x7fdddc006820 2021-02-10T11:39:01.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.882 7fddf1b11700 1 -- 172.21.2.101:0/2176942981 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdde8004830 con 0x7fdddc006820 2021-02-10T11:39:01.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.882 7fddf8e54700 1 -- 172.21.2.101:0/2176942981 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdddc006820 msgr2=0x7fdddc008d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:01.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.882 7fddf8e54700 1 --2- 172.21.2.101:0/2176942981 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdddc006820 0x7fdddc008d40 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 rx=0x7fdde80014e0 tx=0x7fdde8007d20).stop 2021-02-10T11:39:01.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.882 7fddf3314700 1 -- 172.21.2.101:0/2176942981 reap_dead start 2021-02-10T11:39:01.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.883 7fddf8e54700 1 -- 172.21.2.101:0/2176942981 shutdown_connections 2021-02-10T11:39:01.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.883 7fddf8e54700 1 -- 172.21.2.101:0/2176942981 >> 172.21.2.101:0/2176942981 conn(0x7fddf40ce430 msgr2=0x7fddf40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:01.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.883 7fddf8e54700 1 -- 172.21.2.101:0/2176942981 shutdown_connections 2021-02-10T11:39:01.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.883 7fddf8e54700 1 -- 172.21.2.101:0/2176942981 wait complete. 2021-02-10T11:39:01.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.897 7fddf8e54700 1 Processor -- start 2021-02-10T11:39:01.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.897 7fddf8e54700 1 -- start start 2021-02-10T11:39:01.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.898 7fddf8e54700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fddf40a0570 0x7fddf40a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:01.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.898 7fddf8e54700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fddf4084cb0 0x7fddf419ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:01.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.898 7fddf8e54700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fddf419f060 0x7fddf419f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:01.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.898 7fddf8e54700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf40c29f0 con 0x7fddf40a0570 2021-02-10T11:39:01.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.898 7fddf8e54700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf40a0f90 con 0x7fddf4084cb0 2021-02-10T11:39:01.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.898 7fddf8e54700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf4085810 con 0x7fddf419f060 2021-02-10T11:39:01.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.898 7fddf2b13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fddf40a0570 0x7fddf40a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:01.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.898 7fddf2b13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fddf40a0570 0x7fddf40a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55196/0 (socket says 172.21.2.101:55196) 2021-02-10T11:39:01.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.898 7fddf2b13700 1 -- 172.21.2.101:0/2201362223 learned_addr learned my addr 172.21.2.101:0/2201362223 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:01.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.898 7fddf2b13700 1 -- 172.21.2.101:0/2201362223 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fddf419f060 msgr2=0x7fddf419f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:01.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.898 7fddf2b13700 1 --2- 172.21.2.101:0/2201362223 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fddf419f060 0x7fddf419f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:01.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.898 7fddf2b13700 1 -- 172.21.2.101:0/2201362223 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fddf4084cb0 msgr2=0x7fddf419ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:01.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.898 7fddf2b13700 1 --2- 172.21.2.101:0/2201362223 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fddf4084cb0 0x7fddf419ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:01.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.898 7fddf2b13700 1 -- 172.21.2.101:0/2201362223 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf4085a30 con 0x7fddf40a0570 2021-02-10T11:39:01.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.898 7fddf2b13700 1 --2- 172.21.2.101:0/2201362223 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fddf40a0570 0x7fddf40a0970 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 rx=0x7fdde400b560 tx=0x7fdde400b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:01.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.899 7fdde37fe700 1 -- 172.21.2.101:0/2201362223 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdde4000b50 con 0x7fddf40a0570 2021-02-10T11:39:01.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.899 7fdde37fe700 1 -- 172.21.2.101:0/2201362223 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde4002080 con 0x7fddf40a0570 2021-02-10T11:39:01.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.899 7fddf8e54700 1 -- 172.21.2.101:0/2201362223 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf4085a30 con 0x7fddf40a0570 2021-02-10T11:39:01.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.899 7fdde37fe700 1 -- 172.21.2.101:0/2201362223 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdde40024e0 con 0x7fddf40a0570 2021-02-10T11:39:01.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.899 7fddf8e54700 1 -- 172.21.2.101:0/2201362223 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf4085a30 con 0x7fddf40a0570 2021-02-10T11:39:01.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.899 7fdde37fe700 1 -- 172.21.2.101:0/2201362223 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdde40026f0 con 0x7fddf40a0570 2021-02-10T11:39:01.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.899 7fdde37fe700 1 --2- 172.21.2.101:0/2201362223 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fddd80186c0 0x7fddd801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:01.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.899 7fdde37fe700 1 -- 172.21.2.101:0/2201362223 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5457+0+0 (secure 0 0 0) 0x7fdde4032500 con 0x7fddf40a0570 2021-02-10T11:39:01.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.900 7fddf2312700 1 --2- 172.21.2.101:0/2201362223 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fddd80186c0 0x7fddd801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:01.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.900 7fddf2312700 1 --2- 172.21.2.101:0/2201362223 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fddd80186c0 0x7fddd801abc0 crc :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:01.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.929 7fddf8e54700 1 -- 172.21.2.101:0/2201362223 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf40852e0 con 0x7fddf40a0570 2021-02-10T11:39:01.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:01.932 7fdde37fe700 1 -- 172.21.2.101:0/2201362223 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdde40166a0 con 0x7fddf40a0570 2021-02-10T11:39:02.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.061 7fddf8e54700 1 -- 172.21.2.101:0/2201362223 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"} v 0) v1 -- 0x7fddf40852e0 con 0x7fddf40a0570 2021-02-10T11:39:02.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.065 7fdde37fe700 1 -- 172.21.2.101:0/2201362223 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"}]=0 new fs with metadata pool 18 and data pool 19 v68) v1 ==== 189+0+0 (secure 0 0 0) 0x7fdde40026f0 con 0x7fddf40a0570 2021-02-10T11:39:02.072 INFO:teuthology.orchestra.run.gibba001.stderr:new fs with metadata pool 18 and data pool 19 2021-02-10T11:39:02.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.072 7fddf8e54700 1 -- 172.21.2.101:0/2201362223 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fddd80186c0 msgr2=0x7fddd801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:02.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.072 7fddf8e54700 1 --2- 172.21.2.101:0/2201362223 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fddd80186c0 0x7fddd801abc0 crc :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:02.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.072 7fddf8e54700 1 -- 172.21.2.101:0/2201362223 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fddf40a0570 msgr2=0x7fddf40a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:02.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.072 7fddf8e54700 1 --2- 172.21.2.101:0/2201362223 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fddf40a0570 0x7fddf40a0970 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 rx=0x7fdde400b560 tx=0x7fdde400b900).stop 2021-02-10T11:39:02.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.072 7fddf8e54700 1 -- 172.21.2.101:0/2201362223 shutdown_connections 2021-02-10T11:39:02.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.072 7fddf8e54700 1 --2- 172.21.2.101:0/2201362223 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fddd80186c0 0x7fddd801abc0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:02.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.072 7fddf8e54700 1 --2- 172.21.2.101:0/2201362223 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fddf419f060 0x7fddf419f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:02.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.072 7fddf8e54700 1 --2- 172.21.2.101:0/2201362223 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fddf4084cb0 0x7fddf419ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:02.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.072 7fddf8e54700 1 --2- 172.21.2.101:0/2201362223 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fddf40a0570 0x7fddf40a0970 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:02.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.072 7fddf8e54700 1 -- 172.21.2.101:0/2201362223 >> 172.21.2.101:0/2201362223 conn(0x7fddf409daf0 msgr2=0x7fddf40b50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:02.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.072 7fddf8e54700 1 -- 172.21.2.101:0/2201362223 shutdown_connections 2021-02-10T11:39:02.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.072 7fddf8e54700 1 -- 172.21.2.101:0/2201362223 wait complete. 2021-02-10T11:39:02.083 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:39:02.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.489 7f6a1c2db700 1 Processor -- start 2021-02-10T11:39:02.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.489 7f6a1c2db700 1 -- start start 2021-02-10T11:39:02.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.490 7f6a1c2db700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6a140d0340 0x7f6a140ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:02.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.490 7f6a1c2db700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a140d1c70 0x7f6a140efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:02.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.490 7f6a1c2db700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a140f0b10 con 0x7f6a140d1870 2021-02-10T11:39:02.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.490 7f6a1c2db700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a140e5ed0 con 0x7f6a140d0340 2021-02-10T11:39:02.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.490 7f6a1c2db700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a141b0be0 con 0x7f6a140d1c70 2021-02-10T11:39:02.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.490 7f6a1a0f7700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f6a140d1870 0x7f6a141b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55706/0 (socket says 172.21.2.101:55706) 2021-02-10T11:39:02.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.490 7f6a1a0f7700 1 -- 172.21.2.101:0/1153897871 learned_addr learned my addr 172.21.2.101:0/1153897871 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:02.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.491 7f6a190f5700 1 -- 172.21.2.101:0/1153897871 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3897211811 0 0) 0x7f6a10000ce0 con 0x7f6a140d1870 2021-02-10T11:39:02.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.491 7f6a190f5700 1 -- 172.21.2.101:0/1153897871 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a00002fe0 con 0x7f6a140d1870 2021-02-10T11:39:02.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.491 7f6a1a8f8700 1 --2- 172.21.2.101:0/1153897871 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a140d1c70 0x7f6a140efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:02.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.491 7f6a198f6700 1 --2- 172.21.2.101:0/1153897871 >> v2:172.21.2.108:3300/0 conn(0x7f6a140d0340 0x7f6a140ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:02.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.491 7f6a190f5700 1 -- 172.21.2.101:0/1153897871 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1953345007 0 0) 0x7f6a10001180 con 0x7f6a140d1870 2021-02-10T11:39:02.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.491 7f6a190f5700 1 -- 172.21.2.101:0/1153897871 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a140d1c70 msgr2=0x7f6a140efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:02.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.491 7f6a190f5700 1 --2- 172.21.2.101:0/1153897871 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a140d1c70 0x7f6a140efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:02.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.491 7f6a190f5700 1 -- 172.21.2.101:0/1153897871 >> v2:172.21.2.108:3300/0 conn(0x7f6a140d0340 msgr2=0x7f6a140ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:02.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.491 7f6a190f5700 1 --2- 172.21.2.101:0/1153897871 >> v2:172.21.2.108:3300/0 conn(0x7f6a140d0340 0x7f6a140ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:02.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.491 7f6a190f5700 1 -- 172.21.2.101:0/1153897871 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a140f20a0 con 0x7f6a140d1870 2021-02-10T11:39:02.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.491 7f6a190f5700 1 -- 172.21.2.101:0/1153897871 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f6a10001680 con 0x7f6a140d1870 2021-02-10T11:39:02.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.491 7f6a1a8f8700 1 --2- 172.21.2.101:0/1153897871 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a140d1c70 0x7f6a140efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:39:02.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.491 7f6a190f5700 1 -- 172.21.2.101:0/1153897871 >> v1:172.21.2.101:6789/0 conn(0x7f6a140d1870 legacy=0x7f6a141b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:02.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.491 7f6a190f5700 1 --2- 172.21.2.101:0/1153897871 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a00003ac0 0x7f6a00005ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:02.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.491 7f6a190f5700 1 --2- 172.21.2.101:0/1153897871 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a0004a780 0x7f6a0004cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:02.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.491 7f6a190f5700 1 --2- 172.21.2.101:0/1153897871 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a0004d520 0x7f6a0004fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:02.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.491 7f6a190f5700 1 -- 172.21.2.101:0/1153897871 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a000502f0 con 0x7f6a0004a780 2021-02-10T11:39:02.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.491 7f6a190f5700 1 -- 172.21.2.101:0/1153897871 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a00050510 con 0x7f6a0004d520 2021-02-10T11:39:02.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.491 7f6a190f5700 1 -- 172.21.2.101:0/1153897871 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a00050730 con 0x7f6a00003ac0 2021-02-10T11:39:02.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.491 7f6a198f6700 1 --2- 172.21.2.101:0/1153897871 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a0004a780 0x7f6a0004cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:02.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.491 7f6a1a0f7700 1 --2- 172.21.2.101:0/1153897871 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a00003ac0 0x7f6a00005ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:02.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.491 7f6a1a8f8700 1 --2- 172.21.2.101:0/1153897871 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a0004d520 0x7f6a0004fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:02.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.492 7f6a198f6700 1 -- 172.21.2.101:0/1153897871 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a00003ac0 msgr2=0x7f6a00005ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:02.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.492 7f6a198f6700 1 --2- 172.21.2.101:0/1153897871 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a00003ac0 0x7f6a00005ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:02.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.492 7f6a198f6700 1 -- 172.21.2.101:0/1153897871 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a0004d520 msgr2=0x7f6a0004fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:02.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.492 7f6a198f6700 1 --2- 172.21.2.101:0/1153897871 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a0004d520 0x7f6a0004fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:02.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.492 7f6a198f6700 1 -- 172.21.2.101:0/1153897871 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a000509d0 con 0x7f6a0004a780 2021-02-10T11:39:02.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.492 7f6a198f6700 1 --2- 172.21.2.101:0/1153897871 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a0004a780 0x7f6a0004cc80 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 rx=0x7f6a0400c8f0 tx=0x7f6a0400cc90).ready entity=mon.0 client_cookie=7b674fea09c39e77 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:02.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.492 7f6a190f5700 1 -- 172.21.2.101:0/1153897871 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a0400cdd0 con 0x7f6a0004a780 2021-02-10T11:39:02.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.492 7f6a190f5700 1 -- 172.21.2.101:0/1153897871 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a04009310 con 0x7f6a0004a780 2021-02-10T11:39:02.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.492 7f6a190f5700 1 -- 172.21.2.101:0/1153897871 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a040097b0 con 0x7f6a0004a780 2021-02-10T11:39:02.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.492 7f6a1c2db700 1 -- 172.21.2.101:0/1153897871 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a0004a780 msgr2=0x7f6a0004cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:02.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.492 7f6a1c2db700 1 --2- 172.21.2.101:0/1153897871 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a0004a780 0x7f6a0004cc80 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 rx=0x7f6a0400c8f0 tx=0x7f6a0400cc90).stop 2021-02-10T11:39:02.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.492 7f6a1a8f8700 1 -- 172.21.2.101:0/1153897871 reap_dead start 2021-02-10T11:39:02.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.493 7f6a1c2db700 1 -- 172.21.2.101:0/1153897871 shutdown_connections 2021-02-10T11:39:02.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.493 7f6a1c2db700 1 -- 172.21.2.101:0/1153897871 >> 172.21.2.101:0/1153897871 conn(0x7f6a140ce450 msgr2=0x7f6a140d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:02.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.493 7f6a1c2db700 1 -- 172.21.2.101:0/1153897871 shutdown_connections 2021-02-10T11:39:02.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.493 7f6a1c2db700 1 -- 172.21.2.101:0/1153897871 wait complete. 2021-02-10T11:39:02.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.508 7f6a1c2db700 1 Processor -- start 2021-02-10T11:39:02.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.508 7f6a1c2db700 1 -- start start 2021-02-10T11:39:02.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.508 7f6a1c2db700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a141a4a60 0x7f6a140c1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:02.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.508 7f6a1c2db700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a140c2290 0x7f6a140c2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:02.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.508 7f6a1c2db700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a140c2cb0 0x7f6a140bfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:02.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.508 7f6a1c2db700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140f05b0 con 0x7f6a141a4a60 2021-02-10T11:39:02.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.508 7f6a1c2db700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140f07d0 con 0x7f6a140c2cb0 2021-02-10T11:39:02.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.508 7f6a1c2db700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140c0340 con 0x7f6a140c2290 2021-02-10T11:39:02.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.508 7f6a1a0f7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a141a4a60 0x7f6a140c1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:02.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.508 7f6a1a0f7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a141a4a60 0x7f6a140c1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55224/0 (socket says 172.21.2.101:55224) 2021-02-10T11:39:02.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.508 7f6a1a0f7700 1 -- 172.21.2.101:0/2235669352 learned_addr learned my addr 172.21.2.101:0/2235669352 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:02.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.509 7f6a1a0f7700 1 -- 172.21.2.101:0/2235669352 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a140c2290 msgr2=0x7f6a140c2690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:02.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.509 7f6a1a0f7700 1 --2- 172.21.2.101:0/2235669352 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a140c2290 0x7f6a140c2690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:02.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.509 7f6a1a0f7700 1 -- 172.21.2.101:0/2235669352 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a140c2cb0 msgr2=0x7f6a140bfd20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:02.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.509 7f6a1a0f7700 1 --2- 172.21.2.101:0/2235669352 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a140c2cb0 0x7f6a140bfd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:02.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.509 7f6a1a0f7700 1 -- 172.21.2.101:0/2235669352 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a140c0560 con 0x7f6a141a4a60 2021-02-10T11:39:02.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.509 7f6a1a0f7700 1 --2- 172.21.2.101:0/2235669352 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a141a4a60 0x7f6a140c1c70 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 rx=0x7f6a10007fc0 tx=0x7f6a1000a930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:02.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.509 7f6a0affd700 1 -- 172.21.2.101:0/2235669352 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a10001150 con 0x7f6a141a4a60 2021-02-10T11:39:02.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.509 7f6a0affd700 1 -- 172.21.2.101:0/2235669352 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a100043a0 con 0x7f6a141a4a60 2021-02-10T11:39:02.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.509 7f6a0affd700 1 -- 172.21.2.101:0/2235669352 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a100048c0 con 0x7f6a141a4a60 2021-02-10T11:39:02.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.509 7f6a1c2db700 1 -- 172.21.2.101:0/2235669352 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a140c0560 con 0x7f6a141a4a60 2021-02-10T11:39:02.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.509 7f6a1c2db700 1 -- 172.21.2.101:0/2235669352 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a140c0560 con 0x7f6a141a4a60 2021-02-10T11:39:02.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.510 7f6a0affd700 1 -- 172.21.2.101:0/2235669352 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6a10004580 con 0x7f6a141a4a60 2021-02-10T11:39:02.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.510 7f6a0affd700 1 --2- 172.21.2.101:0/2235669352 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f69fc0186c0 0x7f69fc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:02.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.510 7f6a0affd700 1 -- 172.21.2.101:0/2235669352 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6a10032530 con 0x7f6a141a4a60 2021-02-10T11:39:02.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.510 7f6a198f6700 1 --2- 172.21.2.101:0/2235669352 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f69fc0186c0 0x7f69fc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:02.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.510 7f6a198f6700 1 --2- 172.21.2.101:0/2235669352 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f69fc0186c0 0x7f69fc01abc0 crc :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:02.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.540 7f6a1c2db700 1 -- 172.21.2.101:0/2235669352 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a140f0af0 con 0x7f6a141a4a60 2021-02-10T11:39:02.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.542 7f6a0affd700 1 -- 172.21.2.101:0/2235669352 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6a10001150 con 0x7f6a141a4a60 2021-02-10T11:39:02.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.667 7f6a1c2db700 1 -- 172.21.2.101:0/2235669352 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a140f0af0 con 0x7f6a141a4a60 2021-02-10T11:39:02.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.667 7f6a0affd700 1 -- 172.21.2.101:0/2235669352 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+14009 (secure 0 0 0) 0x7f6a10004580 con 0x7f6a141a4a60 2021-02-10T11:39:02.671 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:39:02.672 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":93,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:39:02.062986","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:39:00.027476","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2021-02-10 11:39:01.054115","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:39:02.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.669 7f6a1c2db700 1 -- 172.21.2.101:0/2235669352 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f69fc0186c0 msgr2=0x7f69fc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:02.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.669 7f6a1c2db700 1 --2- 172.21.2.101:0/2235669352 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f69fc0186c0 0x7f69fc01abc0 crc :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:02.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.669 7f6a1c2db700 1 -- 172.21.2.101:0/2235669352 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a141a4a60 msgr2=0x7f6a140c1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:02.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.669 7f6a1c2db700 1 --2- 172.21.2.101:0/2235669352 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a141a4a60 0x7f6a140c1c70 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 rx=0x7f6a10007fc0 tx=0x7f6a1000a930).stop 2021-02-10T11:39:02.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.669 7f6a1c2db700 1 -- 172.21.2.101:0/2235669352 shutdown_connections 2021-02-10T11:39:02.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.669 7f6a1c2db700 1 --2- 172.21.2.101:0/2235669352 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f69fc0186c0 0x7f69fc01abc0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:02.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.669 7f6a1c2db700 1 --2- 172.21.2.101:0/2235669352 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a140c2290 0x7f6a140c2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:02.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.669 7f6a1c2db700 1 --2- 172.21.2.101:0/2235669352 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a140c2cb0 0x7f6a140bfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:02.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.669 7f6a1c2db700 1 --2- 172.21.2.101:0/2235669352 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a141a4a60 0x7f6a140c1c70 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:02.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.669 7f6a1c2db700 1 -- 172.21.2.101:0/2235669352 >> 172.21.2.101:0/2235669352 conn(0x7f6a140ebe30 msgr2=0x7f6a140bf200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:02.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.669 7f6a1c2db700 1 -- 172.21.2.101:0/2235669352 shutdown_connections 2021-02-10T11:39:02.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:02.669 7f6a1c2db700 1 -- 172.21.2.101:0/2235669352 wait complete. 2021-02-10T11:39:02.681 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:39:02.789 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:39:02.792 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:39:03.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.088 7f7688e6e700 1 Processor -- start 2021-02-10T11:39:03.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.088 7f7688e6e700 1 -- start start 2021-02-10T11:39:03.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.088 7f7688e6e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f76840d1870 0x7f76841b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:03.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.088 7f7688e6e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f76841b0a10 0x7f76840cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:03.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.088 7f7688e6e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f76840d1c70 0x7f76840cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:03.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.088 7f7688e6e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76840e5ed0 con 0x7f76841b0a10 2021-02-10T11:39:03.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.088 7f7688e6e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76840f0b10 con 0x7f76840d1870 2021-02-10T11:39:03.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.088 7f7688e6e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76840cff70 con 0x7f76840d1c70 2021-02-10T11:39:03.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.088 7f7682b13700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f76840d1870 0x7f76841b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:03.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.088 7f7682312700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f76841b0a10 0x7f76840cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:03.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.089 7f7682312700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f76841b0a10 0x7f76840cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55234/0 (socket says 172.21.2.101:55234) 2021-02-10T11:39:03.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.089 7f7682b13700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f76840d1870 0x7f76841b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:37486/0 (socket says 172.21.2.101:37486) 2021-02-10T11:39:03.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.089 7f7682312700 1 -- 172.21.2.101:0/1252547575 learned_addr learned my addr 172.21.2.101:0/1252547575 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:03.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.089 7f7683314700 1 --2- 172.21.2.101:0/1252547575 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f76840d1c70 0x7f76840cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:03.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.089 7f7682b13700 1 -- 172.21.2.101:0/1252547575 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f76840d1c70 msgr2=0x7f76840cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:03.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.089 7f7682b13700 1 --2- 172.21.2.101:0/1252547575 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f76840d1c70 0x7f76840cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:03.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.089 7f7682b13700 1 -- 172.21.2.101:0/1252547575 >> v2:172.21.2.101:3300/0 conn(0x7f76841b0a10 msgr2=0x7f76840cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:03.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.089 7f7682b13700 1 --2- 172.21.2.101:0/1252547575 >> v2:172.21.2.101:3300/0 conn(0x7f76841b0a10 0x7f76840cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:03.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.089 7f7682b13700 1 -- 172.21.2.101:0/1252547575 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76840d0150 con 0x7f76840d1870 2021-02-10T11:39:03.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.089 7f7683314700 1 --2- 172.21.2.101:0/1252547575 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f76840d1c70 0x7f76840cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:39:03.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.089 7f7682b13700 1 --2- 172.21.2.101:0/1252547575 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f76840d1870 0x7f76841b03f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f767c0069d0 tx=0x7f767c009e40).ready entity=mon.1 client_cookie=e230df1de450512d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:03.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.089 7f7681b11700 1 -- 172.21.2.101:0/1252547575 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f767c00b510 con 0x7f76840d1870 2021-02-10T11:39:03.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.089 7f7681b11700 1 -- 172.21.2.101:0/1252547575 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767c00b890 con 0x7f76840d1870 2021-02-10T11:39:03.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.089 7f7681b11700 1 -- 172.21.2.101:0/1252547575 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f767c00bd70 con 0x7f76840d1870 2021-02-10T11:39:03.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.089 7f7688e6e700 1 -- 172.21.2.101:0/1252547575 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f76840d1870 msgr2=0x7f76841b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:03.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.089 7f7688e6e700 1 --2- 172.21.2.101:0/1252547575 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f76840d1870 0x7f76841b03f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f767c0069d0 tx=0x7f767c009e40).stop 2021-02-10T11:39:03.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.090 7f7688e6e700 1 -- 172.21.2.101:0/1252547575 shutdown_connections 2021-02-10T11:39:03.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.090 7f7688e6e700 1 --2- 172.21.2.101:0/1252547575 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f76840d1c70 0x7f76840cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:03.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.090 7f7688e6e700 1 --2- 172.21.2.101:0/1252547575 >> v2:172.21.2.101:3300/0 conn(0x7f76841b0a10 0x7f76840cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:03.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.090 7f7688e6e700 1 --2- 172.21.2.101:0/1252547575 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f76840d1870 0x7f76841b03f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:03.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.090 7f7688e6e700 1 -- 172.21.2.101:0/1252547575 >> 172.21.2.101:0/1252547575 conn(0x7f76840ce450 msgr2=0x7f76840d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:03.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.090 7f7688e6e700 1 -- 172.21.2.101:0/1252547575 shutdown_connections 2021-02-10T11:39:03.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.090 7f7688e6e700 1 -- 172.21.2.101:0/1252547575 wait complete. 2021-02-10T11:39:03.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.105 7f7688e6e700 1 Processor -- start 2021-02-10T11:39:03.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.105 7f7688e6e700 1 -- start start 2021-02-10T11:39:03.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.105 7f7688e6e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7684195650 0x7f76840b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:03.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.105 7f7688e6e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f76840b5b80 0x7f76841a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:03.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.105 7f7688e6e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f76840acfa0 0x7f76841a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:03.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.105 7f7688e6e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76840b6010 con 0x7f76840b5b80 2021-02-10T11:39:03.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.105 7f7688e6e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76841a3a60 con 0x7f7684195650 2021-02-10T11:39:03.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.105 7f7688e6e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76841a3c40 con 0x7f76840acfa0 2021-02-10T11:39:03.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.106 7f7683314700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f76840acfa0 0x7f76841a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:03.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.106 7f7682b13700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7684195650 0x7f76840b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:03.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.106 7f7682312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f76840b5b80 0x7f76841a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:03.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.106 7f7683314700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f76840acfa0 0x7f76841a3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:47970/0 (socket says 172.21.2.101:47970) 2021-02-10T11:39:03.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.106 7f7683314700 1 -- 172.21.2.101:0/576713765 learned_addr learned my addr 172.21.2.101:0/576713765 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:03.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.106 7f7682312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f76840b5b80 0x7f76841a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55242/0 (socket says 172.21.2.101:55242) 2021-02-10T11:39:03.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.106 7f7682312700 1 -- 172.21.2.101:0/576713765 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f76840acfa0 msgr2=0x7f76841a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:03.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.106 7f7682312700 1 --2- 172.21.2.101:0/576713765 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f76840acfa0 0x7f76841a3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:03.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.106 7f7682312700 1 -- 172.21.2.101:0/576713765 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7684195650 msgr2=0x7f76840b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:03.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.106 7f7682312700 1 --2- 172.21.2.101:0/576713765 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7684195650 0x7f76840b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:03.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.106 7f7682312700 1 -- 172.21.2.101:0/576713765 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76840bcae0 con 0x7f76840b5b80 2021-02-10T11:39:03.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.106 7f7682312700 1 --2- 172.21.2.101:0/576713765 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f76840b5b80 0x7f76841a2e20 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 rx=0x7f7678006ed0 tx=0x7f767800b8c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:03.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.106 7f76737fe700 1 -- 172.21.2.101:0/576713765 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f767800bcb0 con 0x7f76840b5b80 2021-02-10T11:39:03.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.106 7f76737fe700 1 -- 172.21.2.101:0/576713765 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7678006400 con 0x7f76840b5b80 2021-02-10T11:39:03.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.106 7f76737fe700 1 -- 172.21.2.101:0/576713765 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7678002220 con 0x7f76840b5b80 2021-02-10T11:39:03.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.106 7f7688e6e700 1 -- 172.21.2.101:0/576713765 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76840bcae0 con 0x7f76840b5b80 2021-02-10T11:39:03.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.106 7f7688e6e700 1 -- 172.21.2.101:0/576713765 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76840bd070 con 0x7f76840b5b80 2021-02-10T11:39:03.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.107 7f76737fe700 1 -- 172.21.2.101:0/576713765 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7678000c90 con 0x7f76840b5b80 2021-02-10T11:39:03.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.107 7f76737fe700 1 --2- 172.21.2.101:0/576713765 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76680186c0 0x7f766801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:03.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.107 7f76737fe700 1 -- 172.21.2.101:0/576713765 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7678032320 con 0x7f76840b5b80 2021-02-10T11:39:03.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.107 7f7682b13700 1 --2- 172.21.2.101:0/576713765 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76680186c0 0x7f766801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:03.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.107 7f7682b13700 1 --2- 172.21.2.101:0/576713765 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76680186c0 0x7f766801abc0 crc :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:03.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.138 7f7688e6e700 1 -- 172.21.2.101:0/576713765 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76840f0ab0 con 0x7f76840b5b80 2021-02-10T11:39:03.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.141 7f76737fe700 1 -- 172.21.2.101:0/576713765 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f76780009b0 con 0x7f76840b5b80 2021-02-10T11:39:03.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.267 7f7688e6e700 1 -- 172.21.2.101:0/576713765 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76840f0ab0 con 0x7f76840b5b80 2021-02-10T11:39:03.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.267 7f76737fe700 1 -- 172.21.2.101:0/576713765 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+13996 (secure 0 0 0) 0x7f7678000c90 con 0x7f76840b5b80 2021-02-10T11:39:03.271 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:39:03.272 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":94,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:39:03.065120","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:39:00.027476","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2021-02-10 11:39:01.054115","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:39:03.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.269 7f7688e6e700 1 -- 172.21.2.101:0/576713765 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76680186c0 msgr2=0x7f766801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:03.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.269 7f7688e6e700 1 --2- 172.21.2.101:0/576713765 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76680186c0 0x7f766801abc0 crc :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:03.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.269 7f7688e6e700 1 -- 172.21.2.101:0/576713765 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f76840b5b80 msgr2=0x7f76841a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:03.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.269 7f7688e6e700 1 --2- 172.21.2.101:0/576713765 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f76840b5b80 0x7f76841a2e20 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 rx=0x7f7678006ed0 tx=0x7f767800b8c0).stop 2021-02-10T11:39:03.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.269 7f7688e6e700 1 -- 172.21.2.101:0/576713765 shutdown_connections 2021-02-10T11:39:03.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.269 7f7688e6e700 1 --2- 172.21.2.101:0/576713765 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76680186c0 0x7f766801abc0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:03.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.269 7f7688e6e700 1 --2- 172.21.2.101:0/576713765 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f76840acfa0 0x7f76841a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:03.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.269 7f7688e6e700 1 --2- 172.21.2.101:0/576713765 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f76840b5b80 0x7f76841a2e20 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:03.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.269 7f7688e6e700 1 --2- 172.21.2.101:0/576713765 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7684195650 0x7f76840b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:03.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.269 7f7688e6e700 1 -- 172.21.2.101:0/576713765 >> 172.21.2.101:0/576713765 conn(0x7f76840b3040 msgr2=0x7f76840c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:03.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.270 7f7688e6e700 1 -- 172.21.2.101:0/576713765 shutdown_connections 2021-02-10T11:39:03.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.270 7f7688e6e700 1 -- 172.21.2.101:0/576713765 wait complete. 2021-02-10T11:39:03.282 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-02-10T11:39:03.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.693 7fb4a3e98700 1 Processor -- start 2021-02-10T11:39:03.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.693 7fb4a3e98700 1 -- start start 2021-02-10T11:39:03.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.693 7fb4a3e98700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb49c0d1c70 0x7fb49c0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:03.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.693 7fb4a3e98700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb49c0d0170 con 0x7fb49c0cf580 2021-02-10T11:39:03.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.693 7fb4a3e98700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb49c0fe1c0 con 0x7fb49c0d1870 2021-02-10T11:39:03.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.693 7fb4a3e98700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb49c1b0cb0 con 0x7fb49c0d1c70 2021-02-10T11:39:03.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a24b5700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb49c0cf580 0x7fb49c0cf980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55740/0 (socket says 172.21.2.101:55740) 2021-02-10T11:39:03.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a24b5700 1 -- 172.21.2.101:0/1712343680 learned_addr learned my addr 172.21.2.101:0/1712343680 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:03.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a14b3700 1 --2- 172.21.2.101:0/1712343680 >> v2:172.21.2.108:3300/0 conn(0x7fb49c0d1c70 0x7fb49c0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:03.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a0cb2700 1 -- 172.21.2.101:0/1712343680 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229533855 0 0) 0x7fb494000f90 con 0x7fb49c0cf580 2021-02-10T11:39:03.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a0cb2700 1 -- 172.21.2.101:0/1712343680 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb488002fe0 con 0x7fb49c0cf580 2021-02-10T11:39:03.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a0cb2700 1 -- 172.21.2.101:0/1712343680 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1551735478 0 0) 0x7fb498000ff0 con 0x7fb49c0d1870 2021-02-10T11:39:03.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a0cb2700 1 -- 172.21.2.101:0/1712343680 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb488003210 con 0x7fb49c0d1870 2021-02-10T11:39:03.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a0cb2700 1 -- 172.21.2.101:0/1712343680 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 722889331 0 0) 0x7fb494001460 con 0x7fb49c0cf580 2021-02-10T11:39:03.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a0cb2700 1 -- 172.21.2.101:0/1712343680 >> v2:172.21.2.108:3300/0 conn(0x7fb49c0d1c70 msgr2=0x7fb49c0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:03.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a0cb2700 1 --2- 172.21.2.101:0/1712343680 >> v2:172.21.2.108:3300/0 conn(0x7fb49c0d1c70 0x7fb49c0cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:03.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a0cb2700 1 -- 172.21.2.101:0/1712343680 >> v1:172.21.2.108:6789/0 conn(0x7fb49c0d1870 legacy=0x7fb49c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:03.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a0cb2700 1 -- 172.21.2.101:0/1712343680 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49c109a60 con 0x7fb49c0cf580 2021-02-10T11:39:03.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a0cb2700 1 -- 172.21.2.101:0/1712343680 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fb494001980 con 0x7fb49c0cf580 2021-02-10T11:39:03.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a0cb2700 1 -- 172.21.2.101:0/1712343680 >> v1:172.21.2.101:6789/0 conn(0x7fb49c0cf580 legacy=0x7fb49c0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:03.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a0cb2700 1 --2- 172.21.2.101:0/1712343680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb488003b90 0x7fb488005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:03.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a0cb2700 1 --2- 172.21.2.101:0/1712343680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4880067a0 0x7fb488008ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:03.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a0cb2700 1 --2- 172.21.2.101:0/1712343680 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4880095a0 0x7fb48800bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:03.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a1cb4700 1 --2- 172.21.2.101:0/1712343680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb488003b90 0x7fb488005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:03.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a0cb2700 1 -- 172.21.2.101:0/1712343680 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb488050320 con 0x7fb488003b90 2021-02-10T11:39:03.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a0cb2700 1 -- 172.21.2.101:0/1712343680 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb488050500 con 0x7fb4880095a0 2021-02-10T11:39:03.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a0cb2700 1 -- 172.21.2.101:0/1712343680 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb488050720 con 0x7fb4880067a0 2021-02-10T11:39:03.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a14b3700 1 --2- 172.21.2.101:0/1712343680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4880067a0 0x7fb488008ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:03.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.694 7fb4a24b5700 1 --2- 172.21.2.101:0/1712343680 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4880095a0 0x7fb48800bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:03.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.695 7fb4a1cb4700 1 -- 172.21.2.101:0/1712343680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4880067a0 msgr2=0x7fb488008ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:03.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.695 7fb4a1cb4700 1 --2- 172.21.2.101:0/1712343680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4880067a0 0x7fb488008ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:03.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.695 7fb4a1cb4700 1 -- 172.21.2.101:0/1712343680 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4880095a0 msgr2=0x7fb48800bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:03.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.695 7fb4a1cb4700 1 --2- 172.21.2.101:0/1712343680 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4880095a0 0x7fb48800bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:03.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.695 7fb4a1cb4700 1 -- 172.21.2.101:0/1712343680 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4880509c0 con 0x7fb488003b90 2021-02-10T11:39:03.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.695 7fb4a24b5700 1 --2- 172.21.2.101:0/1712343680 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4880095a0 0x7fb48800bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:39:03.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.695 7fb4a24b5700 1 -- 172.21.2.101:0/1712343680 reap_dead start 2021-02-10T11:39:03.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.695 7fb4a1cb4700 1 --2- 172.21.2.101:0/1712343680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb488003b90 0x7fb488005fa0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 rx=0x7fb498006ac0 tx=0x7fb498009a70).ready entity=mon.0 client_cookie=f59e04675bf20a7f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:03.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.695 7fb4a0cb2700 1 -- 172.21.2.101:0/1712343680 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb4980066b0 con 0x7fb488003b90 2021-02-10T11:39:03.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.695 7fb4a0cb2700 1 -- 172.21.2.101:0/1712343680 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4980082e0 con 0x7fb488003b90 2021-02-10T11:39:03.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.695 7fb4a0cb2700 1 -- 172.21.2.101:0/1712343680 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb498008770 con 0x7fb488003b90 2021-02-10T11:39:03.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.695 7fb4a3e98700 1 -- 172.21.2.101:0/1712343680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb488003b90 msgr2=0x7fb488005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:03.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.695 7fb4a3e98700 1 --2- 172.21.2.101:0/1712343680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb488003b90 0x7fb488005fa0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 rx=0x7fb498006ac0 tx=0x7fb498009a70).stop 2021-02-10T11:39:03.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.695 7fb4a3e98700 1 -- 172.21.2.101:0/1712343680 shutdown_connections 2021-02-10T11:39:03.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.695 7fb4a3e98700 1 --2- 172.21.2.101:0/1712343680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb488003b90 0x7fb488005fa0 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:03.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.695 7fb4a3e98700 1 -- 172.21.2.101:0/1712343680 >> 172.21.2.101:0/1712343680 conn(0x7fb49c0ce420 msgr2=0x7fb49c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:03.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.696 7fb4a3e98700 1 -- 172.21.2.101:0/1712343680 shutdown_connections 2021-02-10T11:39:03.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.696 7fb4a3e98700 1 -- 172.21.2.101:0/1712343680 wait complete. 2021-02-10T11:39:03.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.710 7fb4a3e98700 1 Processor -- start 2021-02-10T11:39:03.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.710 7fb4a3e98700 1 -- start start 2021-02-10T11:39:03.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.711 7fb4a3e98700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb49c085bc0 0x7fb49c0863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:03.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.711 7fb4a3e98700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb49c0954c0 0x7fb49c096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:03.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.711 7fb4a3e98700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb49c0958c0 0x7fb49c096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:03.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.711 7fb4a3e98700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c095d80 con 0x7fb49c0958c0 2021-02-10T11:39:03.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.711 7fb4a3e98700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c0bfa60 con 0x7fb49c0954c0 2021-02-10T11:39:03.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.711 7fb4a3e98700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c085fc0 con 0x7fb49c085bc0 2021-02-10T11:39:03.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.711 7fb4a24b5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb49c0958c0 0x7fb49c096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:03.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.711 7fb4a24b5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb49c0958c0 0x7fb49c096920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55258/0 (socket says 172.21.2.101:55258) 2021-02-10T11:39:03.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.711 7fb4a24b5700 1 -- 172.21.2.101:0/3363206680 learned_addr learned my addr 172.21.2.101:0/3363206680 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:03.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.711 7fb4a1cb4700 1 --2- 172.21.2.101:0/3363206680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb49c085bc0 0x7fb49c0863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:03.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.711 7fb4a14b3700 1 --2- 172.21.2.101:0/3363206680 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb49c0954c0 0x7fb49c096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:03.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.712 7fb4a24b5700 1 -- 172.21.2.101:0/3363206680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb49c085bc0 msgr2=0x7fb49c0863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:03.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.712 7fb4a24b5700 1 --2- 172.21.2.101:0/3363206680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb49c085bc0 0x7fb49c0863e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:03.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.712 7fb4a24b5700 1 -- 172.21.2.101:0/3363206680 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb49c0954c0 msgr2=0x7fb49c096300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:03.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.712 7fb4a24b5700 1 --2- 172.21.2.101:0/3363206680 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb49c0954c0 0x7fb49c096300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:03.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.712 7fb4a24b5700 1 -- 172.21.2.101:0/3363206680 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49c086c10 con 0x7fb49c0958c0 2021-02-10T11:39:03.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.712 7fb4a14b3700 1 --2- 172.21.2.101:0/3363206680 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb49c0954c0 0x7fb49c096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:39:03.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.712 7fb4a24b5700 1 --2- 172.21.2.101:0/3363206680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb49c0958c0 0x7fb49c096920 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 rx=0x7fb494002240 tx=0x7fb49400ac80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:03.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.712 7fb4927fc700 1 -- 172.21.2.101:0/3363206680 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb494001270 con 0x7fb49c0958c0 2021-02-10T11:39:03.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.712 7fb4927fc700 1 -- 172.21.2.101:0/3363206680 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4940070d0 con 0x7fb49c0958c0 2021-02-10T11:39:03.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.712 7fb4927fc700 1 -- 172.21.2.101:0/3363206680 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb494004930 con 0x7fb49c0958c0 2021-02-10T11:39:03.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.712 7fb4a3e98700 1 -- 172.21.2.101:0/3363206680 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb49c086c10 con 0x7fb49c0958c0 2021-02-10T11:39:03.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.712 7fb4a3e98700 1 -- 172.21.2.101:0/3363206680 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb49c086c10 con 0x7fb49c0958c0 2021-02-10T11:39:03.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.713 7fb4927fc700 1 -- 172.21.2.101:0/3363206680 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb494001270 con 0x7fb49c0958c0 2021-02-10T11:39:03.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.713 7fb4927fc700 1 --2- 172.21.2.101:0/3363206680 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4840186c0 0x7fb48401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:03.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.713 7fb4927fc700 1 -- 172.21.2.101:0/3363206680 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb4940328b0 con 0x7fb49c0958c0 2021-02-10T11:39:03.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.713 7fb4a1cb4700 1 --2- 172.21.2.101:0/3363206680 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4840186c0 0x7fb48401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:03.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.713 7fb4a1cb4700 1 --2- 172.21.2.101:0/3363206680 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4840186c0 0x7fb48401abc0 crc :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:03.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.743 7fb4a3e98700 1 -- 172.21.2.101:0/3363206680 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb49c0fe1c0 con 0x7fb49c0958c0 2021-02-10T11:39:03.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.746 7fb4927fc700 1 -- 172.21.2.101:0/3363206680 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb49402df20 con 0x7fb49c0958c0 2021-02-10T11:39:03.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:03.885 7fb4a3e98700 1 -- 172.21.2.101:0/3363206680 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"} v 0) v1 -- 0x7fb49c0fe1c0 con 0x7fb49c0958c0 2021-02-10T11:39:04.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.077 7fb4927fc700 1 -- 172.21.2.101:0/3363206680 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"}]=0 v71) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb49402df20 con 0x7fb49c0958c0 2021-02-10T11:39:04.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.102 7fb4a3e98700 1 -- 172.21.2.101:0/3363206680 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4840186c0 msgr2=0x7fb48401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:04.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.102 7fb4a3e98700 1 --2- 172.21.2.101:0/3363206680 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4840186c0 0x7fb48401abc0 crc :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:04.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.102 7fb4a3e98700 1 -- 172.21.2.101:0/3363206680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb49c0958c0 msgr2=0x7fb49c096920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:04.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.102 7fb4a3e98700 1 --2- 172.21.2.101:0/3363206680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb49c0958c0 0x7fb49c096920 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 rx=0x7fb494002240 tx=0x7fb49400ac80).stop 2021-02-10T11:39:04.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.102 7fb4a3e98700 1 -- 172.21.2.101:0/3363206680 shutdown_connections 2021-02-10T11:39:04.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.102 7fb4a3e98700 1 --2- 172.21.2.101:0/3363206680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb49c0958c0 0x7fb49c096920 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:04.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.102 7fb4a3e98700 1 --2- 172.21.2.101:0/3363206680 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb49c0954c0 0x7fb49c096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:04.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.102 7fb4a3e98700 1 --2- 172.21.2.101:0/3363206680 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4840186c0 0x7fb48401abc0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:04.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.102 7fb4a3e98700 1 --2- 172.21.2.101:0/3363206680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb49c085bc0 0x7fb49c0863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:04.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.102 7fb4a3e98700 1 -- 172.21.2.101:0/3363206680 >> 172.21.2.101:0/3363206680 conn(0x7fb49c0bef00 msgr2=0x7fb49c09d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:04.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.102 7fb4a3e98700 1 -- 172.21.2.101:0/3363206680 shutdown_connections 2021-02-10T11:39:04.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.102 7fb4a3e98700 1 -- 172.21.2.101:0/3363206680 wait complete. 2021-02-10T11:39:04.114 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:39:04.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.520 7f220f4eb700 1 Processor -- start 2021-02-10T11:39:04.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.520 7f220f4eb700 1 -- start start 2021-02-10T11:39:04.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.520 7f220f4eb700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f22080d1870 0x7f22081b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:04.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.520 7f220f4eb700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f22080f0650 0x7f22080cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:04.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.520 7f220f4eb700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22080f0b10 con 0x7f22081b0a10 2021-02-10T11:39:04.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.520 7f220f4eb700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22080e5ed0 con 0x7f22080f0650 2021-02-10T11:39:04.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.520 7f220f4eb700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22080d1c70 con 0x7f22080d1870 2021-02-10T11:39:04.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.521 7f220cb06700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f22081b0a10 0x7f22080cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55758/0 (socket says 172.21.2.101:55758) 2021-02-10T11:39:04.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.521 7f220cb06700 1 -- 172.21.2.101:0/1721775915 learned_addr learned my addr 172.21.2.101:0/1721775915 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:04.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.521 7f220db08700 1 --2- 172.21.2.101:0/1721775915 >> v2:172.21.2.101:3300/0 conn(0x7f22080f0650 0x7f22080cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:04.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.521 7f21fbfff700 1 -- 172.21.2.101:0/1721775915 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1109267147 0 0) 0x7f22040010b0 con 0x7f22081b0a10 2021-02-10T11:39:04.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.521 7f21fbfff700 1 -- 172.21.2.101:0/1721775915 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21f4002fe0 con 0x7f22081b0a10 2021-02-10T11:39:04.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.521 7f220db08700 1 -- 172.21.2.101:0/1721775915 >> v2:172.21.2.108:3300/0 conn(0x7f22080d1870 msgr2=0x7f22081b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:04.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.521 7f220db08700 1 --2- 172.21.2.101:0/1721775915 >> v2:172.21.2.108:3300/0 conn(0x7f22080d1870 0x7f22081b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:04.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.521 7f220db08700 1 -- 172.21.2.101:0/1721775915 >> v1:172.21.2.101:6789/0 conn(0x7f22081b0a10 legacy=0x7f22080cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:04.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.521 7f220db08700 1 -- 172.21.2.101:0/1721775915 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22080d1e90 con 0x7f22080f0650 2021-02-10T11:39:04.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.521 7f220db08700 1 --2- 172.21.2.101:0/1721775915 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22080f0650 0x7f22080cfb20 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 rx=0x7f2200006e30 tx=0x7f2200009e60).ready entity=mon.0 client_cookie=43cb138203b075b7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:04.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.521 7f21fbfff700 1 -- 172.21.2.101:0/1721775915 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2200006a50 con 0x7f22080f0650 2021-02-10T11:39:04.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.521 7f21fbfff700 1 -- 172.21.2.101:0/1721775915 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220000b300 con 0x7f22080f0650 2021-02-10T11:39:04.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.521 7f21fbfff700 1 -- 172.21.2.101:0/1721775915 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f220000b820 con 0x7f22080f0650 2021-02-10T11:39:04.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.522 7f220f4eb700 1 -- 172.21.2.101:0/1721775915 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22080f0650 msgr2=0x7f22080cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:04.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.522 7f220f4eb700 1 --2- 172.21.2.101:0/1721775915 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22080f0650 0x7f22080cfb20 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 rx=0x7f2200006e30 tx=0x7f2200009e60).stop 2021-02-10T11:39:04.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.522 7f220f4eb700 1 -- 172.21.2.101:0/1721775915 shutdown_connections 2021-02-10T11:39:04.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.522 7f220f4eb700 1 --2- 172.21.2.101:0/1721775915 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22080f0650 0x7f22080cfb20 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:04.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.522 7f220f4eb700 1 --2- 172.21.2.101:0/1721775915 >> v2:172.21.2.108:3300/0 conn(0x7f22080d1870 0x7f22081b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:04.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.522 7f220f4eb700 1 -- 172.21.2.101:0/1721775915 >> 172.21.2.101:0/1721775915 conn(0x7f22080ce450 msgr2=0x7f22080d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:04.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.522 7f220f4eb700 1 -- 172.21.2.101:0/1721775915 shutdown_connections 2021-02-10T11:39:04.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.522 7f220f4eb700 1 -- 172.21.2.101:0/1721775915 wait complete. 2021-02-10T11:39:04.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.537 7f220f4eb700 1 Processor -- start 2021-02-10T11:39:04.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.537 7f220f4eb700 1 -- start start 2021-02-10T11:39:04.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.538 7f220f4eb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22080ada90 0x7f22080ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:04.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.538 7f220f4eb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2208099d20 0x7f220809a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:04.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.538 7f220f4eb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f220809a740 0x7f2208097bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:04.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.538 7f220f4eb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22080997a0 con 0x7f22080ada90 2021-02-10T11:39:04.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.538 7f220f4eb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220818f0c0 con 0x7f2208099d20 2021-02-10T11:39:04.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.538 7f220f4eb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220809ab40 con 0x7f220809a740 2021-02-10T11:39:04.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.538 7f220d307700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22080ada90 0x7f22080ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:04.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.538 7f220d307700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22080ada90 0x7f22080ade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55268/0 (socket says 172.21.2.101:55268) 2021-02-10T11:39:04.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.538 7f220d307700 1 -- 172.21.2.101:0/3055329436 learned_addr learned my addr 172.21.2.101:0/3055329436 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:04.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.538 7f220d307700 1 -- 172.21.2.101:0/3055329436 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f220809a740 msgr2=0x7f2208097bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:04.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.538 7f220d307700 1 --2- 172.21.2.101:0/3055329436 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f220809a740 0x7f2208097bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:04.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.538 7f220d307700 1 -- 172.21.2.101:0/3055329436 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2208099d20 msgr2=0x7f220809a120 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:04.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.538 7f220d307700 1 --2- 172.21.2.101:0/3055329436 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2208099d20 0x7f220809a120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:04.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.538 7f220d307700 1 -- 172.21.2.101:0/3055329436 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220809ad60 con 0x7f22080ada90 2021-02-10T11:39:04.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.538 7f220d307700 1 --2- 172.21.2.101:0/3055329436 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22080ada90 0x7f22080ade90 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 rx=0x7f21fc006f00 tx=0x7f21fc00b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:04.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.539 7f21f9ffb700 1 -- 172.21.2.101:0/3055329436 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21fc00bd30 con 0x7f22080ada90 2021-02-10T11:39:04.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.539 7f21f9ffb700 1 -- 172.21.2.101:0/3055329436 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21fc000c90 con 0x7f22080ada90 2021-02-10T11:39:04.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.539 7f21f9ffb700 1 -- 172.21.2.101:0/3055329436 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21fc0022a0 con 0x7f22080ada90 2021-02-10T11:39:04.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.539 7f220f4eb700 1 -- 172.21.2.101:0/3055329436 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f220809ad60 con 0x7f22080ada90 2021-02-10T11:39:04.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.539 7f220f4eb700 1 -- 172.21.2.101:0/3055329436 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f220809ad60 con 0x7f22080ada90 2021-02-10T11:39:04.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.539 7f21f9ffb700 1 -- 172.21.2.101:0/3055329436 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f21fc005310 con 0x7f22080ada90 2021-02-10T11:39:04.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.539 7f21f9ffb700 1 --2- 172.21.2.101:0/3055329436 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21e80186c0 0x7f21e801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:04.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.539 7f21f9ffb700 1 -- 172.21.2.101:0/3055329436 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f21fc032780 con 0x7f22080ada90 2021-02-10T11:39:04.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.539 7f220cb06700 1 --2- 172.21.2.101:0/3055329436 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21e80186c0 0x7f21e801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:04.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.540 7f220cb06700 1 --2- 172.21.2.101:0/3055329436 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21e80186c0 0x7f21e801abc0 crc :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:04.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.570 7f220f4eb700 1 -- 172.21.2.101:0/3055329436 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22080ebc00 con 0x7f22080ada90 2021-02-10T11:39:04.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.573 7f21f9ffb700 1 -- 172.21.2.101:0/3055329436 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f21fc006b90 con 0x7f22080ada90 2021-02-10T11:39:04.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.702 7f220f4eb700 1 -- 172.21.2.101:0/3055329436 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22080ebc00 con 0x7f22080ada90 2021-02-10T11:39:04.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.702 7f21f9ffb700 1 -- 172.21.2.101:0/3055329436 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f21fc00bd30 con 0x7f22080ada90 2021-02-10T11:39:04.707 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 71 2021-02-10T11:39:04.707 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:39:04.707 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/4210469578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":4210469578},{"type":"v1","addr":"172.21.2.108:6833","nonce":4210469578}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:39:02.063078","modified":"2021-02-10 11:39:04.073713","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6164},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6164":{"gid":6164,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.2.101:6835/1113812290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1113812290},{"type":"v1","addr":"172.21.2.101:6835","nonce":1113812290}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-02-10T11:39:04.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.704 7f220f4eb700 1 -- 172.21.2.101:0/3055329436 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21e80186c0 msgr2=0x7f21e801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:04.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.704 7f220f4eb700 1 --2- 172.21.2.101:0/3055329436 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21e80186c0 0x7f21e801abc0 crc :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:04.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.704 7f220f4eb700 1 -- 172.21.2.101:0/3055329436 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22080ada90 msgr2=0x7f22080ade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:04.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.704 7f220f4eb700 1 --2- 172.21.2.101:0/3055329436 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22080ada90 0x7f22080ade90 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 rx=0x7f21fc006f00 tx=0x7f21fc00b940).stop 2021-02-10T11:39:04.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.705 7f220f4eb700 1 -- 172.21.2.101:0/3055329436 shutdown_connections 2021-02-10T11:39:04.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.705 7f220f4eb700 1 --2- 172.21.2.101:0/3055329436 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21e80186c0 0x7f21e801abc0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:04.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.705 7f220f4eb700 1 --2- 172.21.2.101:0/3055329436 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f220809a740 0x7f2208097bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:04.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.705 7f220f4eb700 1 --2- 172.21.2.101:0/3055329436 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2208099d20 0x7f220809a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:04.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.705 7f220f4eb700 1 --2- 172.21.2.101:0/3055329436 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22080ada90 0x7f22080ade90 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:04.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.705 7f220f4eb700 1 -- 172.21.2.101:0/3055329436 >> 172.21.2.101:0/3055329436 conn(0x7f22080f0550 msgr2=0x7f2208191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:04.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.705 7f220f4eb700 1 -- 172.21.2.101:0/3055329436 shutdown_connections 2021-02-10T11:39:04.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:04.705 7f220f4eb700 1 -- 172.21.2.101:0/3055329436 wait complete. 2021-02-10T11:39:04.717 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:39:05.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.125 7fb8db739700 1 Processor -- start 2021-02-10T11:39:05.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.125 7fb8db739700 1 -- start start 2021-02-10T11:39:05.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.126 7fb8db739700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb8d41b0470 0x7fb8d41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:05.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.126 7fb8db739700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb8d40ef610 0x7fb8d40efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:05.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.126 7fb8db739700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d40f0b50 con 0x7fb8d40f0030 2021-02-10T11:39:05.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.126 7fb8db739700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d40e52e0 con 0x7fb8d40ef610 2021-02-10T11:39:05.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.126 7fb8db739700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d40e5490 con 0x7fb8d41b0470 2021-02-10T11:39:05.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.126 7fb8d8d54700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb8d40ef610 0x7fb8d40efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:05.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.126 7fb8d8d54700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb8d40ef610 0x7fb8d40efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55276/0 (socket says 172.21.2.101:55276) 2021-02-10T11:39:05.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.126 7fb8d8d54700 1 -- 172.21.2.101:0/4057049180 learned_addr learned my addr 172.21.2.101:0/4057049180 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:05.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.126 7fb8d9555700 1 --2- 172.21.2.101:0/4057049180 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb8d41b0470 0x7fb8d41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:05.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.126 7fb8d8d54700 1 -- 172.21.2.101:0/4057049180 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb8d41b0470 msgr2=0x7fb8d41b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:05.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.126 7fb8d8d54700 1 --2- 172.21.2.101:0/4057049180 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb8d41b0470 0x7fb8d41b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:05.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.126 7fb8d8d54700 1 -- 172.21.2.101:0/4057049180 >> v1:172.21.2.108:6789/0 conn(0x7fb8d40f0030 legacy=0x7fb8d40f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:05.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.126 7fb8d8d54700 1 -- 172.21.2.101:0/4057049180 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d40f0430 con 0x7fb8d40ef610 2021-02-10T11:39:05.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.126 7fb8d9555700 1 --2- 172.21.2.101:0/4057049180 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb8d41b0470 0x7fb8d41b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:39:05.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.127 7fb8d8d54700 1 --2- 172.21.2.101:0/4057049180 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8d40ef610 0x7fb8d40efa10 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 rx=0x7fb8d0006700 tx=0x7fb8d0009b80).ready entity=mon.0 client_cookie=9c041d81344c3783 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:05.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.127 7fb8c7fff700 1 -- 172.21.2.101:0/4057049180 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb8d000c0c0 con 0x7fb8d40ef610 2021-02-10T11:39:05.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.127 7fb8c7fff700 1 -- 172.21.2.101:0/4057049180 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d000c340 con 0x7fb8d40ef610 2021-02-10T11:39:05.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.127 7fb8c7fff700 1 -- 172.21.2.101:0/4057049180 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb8d000c860 con 0x7fb8d40ef610 2021-02-10T11:39:05.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.127 7fb8db739700 1 -- 172.21.2.101:0/4057049180 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8d40ef610 msgr2=0x7fb8d40efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:05.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.127 7fb8db739700 1 --2- 172.21.2.101:0/4057049180 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8d40ef610 0x7fb8d40efa10 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 rx=0x7fb8d0006700 tx=0x7fb8d0009b80).stop 2021-02-10T11:39:05.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.127 7fb8db739700 1 -- 172.21.2.101:0/4057049180 shutdown_connections 2021-02-10T11:39:05.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.127 7fb8db739700 1 --2- 172.21.2.101:0/4057049180 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8d40ef610 0x7fb8d40efa10 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:05.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.127 7fb8db739700 1 --2- 172.21.2.101:0/4057049180 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb8d41b0470 0x7fb8d41b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:05.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.127 7fb8db739700 1 -- 172.21.2.101:0/4057049180 >> 172.21.2.101:0/4057049180 conn(0x7fb8d40ce3e0 msgr2=0x7fb8d40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:05.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.128 7fb8db739700 1 -- 172.21.2.101:0/4057049180 shutdown_connections 2021-02-10T11:39:05.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.128 7fb8db739700 1 -- 172.21.2.101:0/4057049180 wait complete. 2021-02-10T11:39:05.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.142 7fb8db739700 1 Processor -- start 2021-02-10T11:39:05.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.142 7fb8db739700 1 -- start start 2021-02-10T11:39:05.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.143 7fb8db739700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb8d41af8a0 0x7fb8d4198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:05.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.143 7fb8db739700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8d41af050 0x7fb8d40aa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:05.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.143 7fb8db739700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb8d4197d00 0x7fb8d4198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:05.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.143 7fb8db739700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d41988e0 con 0x7fb8d41af050 2021-02-10T11:39:05.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.143 7fb8db739700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d41af450 con 0x7fb8d41af8a0 2021-02-10T11:39:05.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.143 7fb8db739700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d40aaa10 con 0x7fb8d4197d00 2021-02-10T11:39:05.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.143 7fb8d8d54700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8d41af050 0x7fb8d40aa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:05.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.143 7fb8d8d54700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8d41af050 0x7fb8d40aa3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55282/0 (socket says 172.21.2.101:55282) 2021-02-10T11:39:05.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.143 7fb8d8d54700 1 -- 172.21.2.101:0/2907967618 learned_addr learned my addr 172.21.2.101:0/2907967618 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:05.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.143 7fb8d9d56700 1 --2- 172.21.2.101:0/2907967618 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb8d4197d00 0x7fb8d4198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:05.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.143 7fb8d9555700 1 --2- 172.21.2.101:0/2907967618 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb8d41af8a0 0x7fb8d4198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:05.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.143 7fb8d8d54700 1 -- 172.21.2.101:0/2907967618 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb8d4197d00 msgr2=0x7fb8d4198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:05.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.144 7fb8d8d54700 1 --2- 172.21.2.101:0/2907967618 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb8d4197d00 0x7fb8d4198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:05.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.144 7fb8d8d54700 1 -- 172.21.2.101:0/2907967618 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb8d41af8a0 msgr2=0x7fb8d4198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:05.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.144 7fb8d8d54700 1 --2- 172.21.2.101:0/2907967618 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb8d41af8a0 0x7fb8d4198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:05.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.144 7fb8d8d54700 1 -- 172.21.2.101:0/2907967618 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d419fec0 con 0x7fb8d41af050 2021-02-10T11:39:05.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.144 7fb8d9d56700 1 --2- 172.21.2.101:0/2907967618 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb8d4197d00 0x7fb8d4198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:39:05.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.144 7fb8d9555700 1 --2- 172.21.2.101:0/2907967618 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb8d41af8a0 0x7fb8d4198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:39:05.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.144 7fb8d8d54700 1 --2- 172.21.2.101:0/2907967618 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8d41af050 0x7fb8d40aa3f0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 rx=0x7fb8d000b5e0 tx=0x7fb8d000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:05.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.144 7fb8c5ffb700 1 -- 172.21.2.101:0/2907967618 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb8d000bd70 con 0x7fb8d41af050 2021-02-10T11:39:05.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.144 7fb8c5ffb700 1 -- 172.21.2.101:0/2907967618 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d0002030 con 0x7fb8d41af050 2021-02-10T11:39:05.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.144 7fb8c5ffb700 1 -- 172.21.2.101:0/2907967618 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb8d0002550 con 0x7fb8d41af050 2021-02-10T11:39:05.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.144 7fb8db739700 1 -- 172.21.2.101:0/2907967618 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d419fec0 con 0x7fb8d41af050 2021-02-10T11:39:05.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.144 7fb8db739700 1 -- 172.21.2.101:0/2907967618 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d41a0450 con 0x7fb8d41af050 2021-02-10T11:39:05.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.145 7fb8c5ffb700 1 -- 172.21.2.101:0/2907967618 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb8d0000c90 con 0x7fb8d41af050 2021-02-10T11:39:05.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.145 7fb8c5ffb700 1 --2- 172.21.2.101:0/2907967618 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8b40186c0 0x7fb8b401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:05.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.145 7fb8c5ffb700 1 -- 172.21.2.101:0/2907967618 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb8d0032580 con 0x7fb8d41af050 2021-02-10T11:39:05.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.145 7fb8d9555700 1 --2- 172.21.2.101:0/2907967618 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8b40186c0 0x7fb8b401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:05.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.145 7fb8d9555700 1 --2- 172.21.2.101:0/2907967618 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8b40186c0 0x7fb8b401abc0 crc :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:05.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.176 7fb8db739700 1 -- 172.21.2.101:0/2907967618 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d4105a20 con 0x7fb8d41af050 2021-02-10T11:39:05.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.178 7fb8c5ffb700 1 -- 172.21.2.101:0/2907967618 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb8d00009b0 con 0x7fb8d41af050 2021-02-10T11:39:05.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.302 7fb8db739700 1 -- 172.21.2.101:0/2907967618 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8d40f0b50 con 0x7fb8d41af050 2021-02-10T11:39:05.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.303 7fb8c5ffb700 1 -- 172.21.2.101:0/2907967618 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+13996 (secure 0 0 0) 0x7fb8d0000c90 con 0x7fb8d41af050 2021-02-10T11:39:05.307 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:39:05.307 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":94,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:39:03.065120","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:39:00.027476","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2021-02-10 11:39:01.054115","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:39:05.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.304 7fb8db739700 1 -- 172.21.2.101:0/2907967618 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8b40186c0 msgr2=0x7fb8b401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:05.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.304 7fb8db739700 1 --2- 172.21.2.101:0/2907967618 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8b40186c0 0x7fb8b401abc0 crc :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:05.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.304 7fb8db739700 1 -- 172.21.2.101:0/2907967618 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8d41af050 msgr2=0x7fb8d40aa3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:05.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.304 7fb8db739700 1 --2- 172.21.2.101:0/2907967618 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8d41af050 0x7fb8d40aa3f0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 rx=0x7fb8d000b5e0 tx=0x7fb8d000b980).stop 2021-02-10T11:39:05.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.304 7fb8db739700 1 -- 172.21.2.101:0/2907967618 shutdown_connections 2021-02-10T11:39:05.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.304 7fb8db739700 1 --2- 172.21.2.101:0/2907967618 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8b40186c0 0x7fb8b401abc0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:05.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.304 7fb8db739700 1 --2- 172.21.2.101:0/2907967618 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb8d4197d00 0x7fb8d4198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:05.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.304 7fb8db739700 1 --2- 172.21.2.101:0/2907967618 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8d41af050 0x7fb8d40aa3f0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:05.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.304 7fb8db739700 1 --2- 172.21.2.101:0/2907967618 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb8d41af8a0 0x7fb8d4198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:05.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.305 7fb8db739700 1 -- 172.21.2.101:0/2907967618 >> 172.21.2.101:0/2907967618 conn(0x7fb8d41a37a0 msgr2=0x7fb8d418c230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:05.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.305 7fb8db739700 1 -- 172.21.2.101:0/2907967618 shutdown_connections 2021-02-10T11:39:05.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.305 7fb8db739700 1 -- 172.21.2.101:0/2907967618 wait complete. 2021-02-10T11:39:05.317 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:39:05.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.736 7f0e282f3700 1 Processor -- start 2021-02-10T11:39:05.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.736 7f0e282f3700 1 -- start start 2021-02-10T11:39:05.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.737 7f0e282f3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0e200d1880 0x7f0e201b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:05.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.737 7f0e282f3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e201b0af0 0x7f0e20108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:05.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.737 7f0e282f3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0e200d1c80 0x7f0e20108f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:05.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.737 7f0e282f3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e200fe1d0 con 0x7f0e200d1880 2021-02-10T11:39:05.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.737 7f0e282f3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e20109a70 con 0x7f0e200d1c80 2021-02-10T11:39:05.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.737 7f0e282f3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e20088070 con 0x7f0e201b0af0 2021-02-10T11:39:05.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.737 7f0e2610f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0e200d1880 0x7f0e201b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:05.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.737 7f0e2610f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0e200d1880 0x7f0e201b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55288/0 (socket says 172.21.2.101:55288) 2021-02-10T11:39:05.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.737 7f0e2610f700 1 -- 172.21.2.101:0/2702647725 learned_addr learned my addr 172.21.2.101:0/2702647725 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:05.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.737 7f0e2610f700 1 -- 172.21.2.101:0/2702647725 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e201b0af0 msgr2=0x7f0e20108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:05.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.737 7f0e2610f700 1 --2- 172.21.2.101:0/2702647725 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e201b0af0 0x7f0e20108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:05.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.737 7f0e2610f700 1 -- 172.21.2.101:0/2702647725 >> v2:172.21.2.108:3300/0 conn(0x7f0e200d1c80 msgr2=0x7f0e20108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:05.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.737 7f0e2610f700 1 --2- 172.21.2.101:0/2702647725 >> v2:172.21.2.108:3300/0 conn(0x7f0e200d1c80 0x7f0e20108f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:05.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.737 7f0e2610f700 1 -- 172.21.2.101:0/2702647725 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e20109590 con 0x7f0e200d1880 2021-02-10T11:39:05.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.737 7f0e2610f700 1 --2- 172.21.2.101:0/2702647725 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e200d1880 0x7f0e201b04d0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 rx=0x7f0e1c006700 tx=0x7f0e1c009b00).ready entity=mon.0 client_cookie=908463a3054395e6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:05.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.738 7f0e2510d700 1 -- 172.21.2.101:0/2702647725 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e1c00c0c0 con 0x7f0e200d1880 2021-02-10T11:39:05.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.738 7f0e2510d700 1 -- 172.21.2.101:0/2702647725 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1c00c340 con 0x7f0e200d1880 2021-02-10T11:39:05.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.738 7f0e2510d700 1 -- 172.21.2.101:0/2702647725 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e1c00c860 con 0x7f0e200d1880 2021-02-10T11:39:05.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.738 7f0e282f3700 1 -- 172.21.2.101:0/2702647725 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e200d1880 msgr2=0x7f0e201b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:05.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.738 7f0e282f3700 1 --2- 172.21.2.101:0/2702647725 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e200d1880 0x7f0e201b04d0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 rx=0x7f0e1c006700 tx=0x7f0e1c009b00).stop 2021-02-10T11:39:05.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.738 7f0e282f3700 1 -- 172.21.2.101:0/2702647725 shutdown_connections 2021-02-10T11:39:05.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.738 7f0e282f3700 1 --2- 172.21.2.101:0/2702647725 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e201b0af0 0x7f0e20108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:05.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.738 7f0e282f3700 1 --2- 172.21.2.101:0/2702647725 >> v2:172.21.2.108:3300/0 conn(0x7f0e200d1c80 0x7f0e20108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:05.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.738 7f0e282f3700 1 --2- 172.21.2.101:0/2702647725 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e200d1880 0x7f0e201b04d0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:05.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.738 7f0e282f3700 1 -- 172.21.2.101:0/2702647725 >> 172.21.2.101:0/2702647725 conn(0x7f0e200ce430 msgr2=0x7f0e200d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:05.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.739 7f0e282f3700 1 -- 172.21.2.101:0/2702647725 shutdown_connections 2021-02-10T11:39:05.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.739 7f0e282f3700 1 -- 172.21.2.101:0/2702647725 wait complete. 2021-02-10T11:39:05.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.753 7f0e282f3700 1 Processor -- start 2021-02-10T11:39:05.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.753 7f0e282f3700 1 -- start start 2021-02-10T11:39:05.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.754 7f0e282f3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e201b9280 0x7f0e2018f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:05.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.754 7f0e282f3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e201b9680 0x7f0e2018fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:05.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.754 7f0e282f3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e201acd60 0x7f0e201ad160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:05.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.754 7f0e282f3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e20190390 con 0x7f0e201b9280 2021-02-10T11:39:05.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.754 7f0e282f3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e201907c0 con 0x7f0e201b9680 2021-02-10T11:39:05.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.754 7f0e282f3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e201ac3d0 con 0x7f0e201acd60 2021-02-10T11:39:05.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.754 7f0e2610f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e201b9280 0x7f0e2018f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:05.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.754 7f0e2610f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e201b9280 0x7f0e2018f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55294/0 (socket says 172.21.2.101:55294) 2021-02-10T11:39:05.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.754 7f0e2610f700 1 -- 172.21.2.101:0/3257921499 learned_addr learned my addr 172.21.2.101:0/3257921499 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:05.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.754 7f0e2610f700 1 -- 172.21.2.101:0/3257921499 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e201acd60 msgr2=0x7f0e201ad160 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:05.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.754 7f0e2610f700 1 --2- 172.21.2.101:0/3257921499 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e201acd60 0x7f0e201ad160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:05.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.754 7f0e2610f700 1 -- 172.21.2.101:0/3257921499 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e201b9680 msgr2=0x7f0e2018fd70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:05.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.754 7f0e2610f700 1 --2- 172.21.2.101:0/3257921499 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e201b9680 0x7f0e2018fd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:05.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.754 7f0e2610f700 1 -- 172.21.2.101:0/3257921499 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e201ad780 con 0x7f0e201b9280 2021-02-10T11:39:05.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.754 7f0e2610f700 1 --2- 172.21.2.101:0/3257921499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e201b9280 0x7f0e2018f750 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 rx=0x7f0e1c00b600 tx=0x7f0e1c00b9a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:05.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.754 7f0e12ffd700 1 -- 172.21.2.101:0/3257921499 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e1c006340 con 0x7f0e201b9280 2021-02-10T11:39:05.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.754 7f0e12ffd700 1 -- 172.21.2.101:0/3257921499 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1c002080 con 0x7f0e201b9280 2021-02-10T11:39:05.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.754 7f0e12ffd700 1 -- 172.21.2.101:0/3257921499 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e1c002560 con 0x7f0e201b9280 2021-02-10T11:39:05.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.754 7f0e282f3700 1 -- 172.21.2.101:0/3257921499 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e201ad780 con 0x7f0e201b9280 2021-02-10T11:39:05.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.755 7f0e282f3700 1 -- 172.21.2.101:0/3257921499 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e201ad780 con 0x7f0e201b9280 2021-02-10T11:39:05.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.755 7f0e12ffd700 1 -- 172.21.2.101:0/3257921499 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0e1c000c90 con 0x7f0e201b9280 2021-02-10T11:39:05.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.755 7f0e12ffd700 1 --2- 172.21.2.101:0/3257921499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e080186c0 0x7f0e0801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:05.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.755 7f0e12ffd700 1 -- 172.21.2.101:0/3257921499 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0e1c032600 con 0x7f0e201b9280 2021-02-10T11:39:05.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.755 7f0e2590e700 1 --2- 172.21.2.101:0/3257921499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e080186c0 0x7f0e0801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:05.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.756 7f0e2590e700 1 --2- 172.21.2.101:0/3257921499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e080186c0 0x7f0e0801abc0 crc :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:05.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.787 7f0e282f3700 1 -- 172.21.2.101:0/3257921499 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e20190390 con 0x7f0e201b9280 2021-02-10T11:39:05.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.790 7f0e12ffd700 1 -- 172.21.2.101:0/3257921499 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0e1c02ef80 con 0x7f0e201b9280 2021-02-10T11:39:05.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.920 7f0e282f3700 1 -- 172.21.2.101:0/3257921499 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f0e20190390 con 0x7f0e201b9280 2021-02-10T11:39:05.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.924 7f0e12ffd700 1 -- 172.21.2.101:0/3257921499 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v53) v1 ==== 184+0+0 (secure 0 0 0) 0x7f0e1c000c90 con 0x7f0e201b9280 2021-02-10T11:39:05.931 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.0 2021-02-10T11:39:05.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.929 7f0e282f3700 1 -- 172.21.2.101:0/3257921499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e080186c0 msgr2=0x7f0e0801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:05.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.929 7f0e282f3700 1 --2- 172.21.2.101:0/3257921499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e080186c0 0x7f0e0801abc0 crc :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:05.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.929 7f0e282f3700 1 -- 172.21.2.101:0/3257921499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e201b9280 msgr2=0x7f0e2018f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:05.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.929 7f0e282f3700 1 --2- 172.21.2.101:0/3257921499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e201b9280 0x7f0e2018f750 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 rx=0x7f0e1c00b600 tx=0x7f0e1c00b9a0).stop 2021-02-10T11:39:05.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.930 7f0e282f3700 1 -- 172.21.2.101:0/3257921499 shutdown_connections 2021-02-10T11:39:05.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.930 7f0e282f3700 1 --2- 172.21.2.101:0/3257921499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e080186c0 0x7f0e0801abc0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:05.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.930 7f0e282f3700 1 --2- 172.21.2.101:0/3257921499 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e201acd60 0x7f0e201ad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:05.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.930 7f0e282f3700 1 --2- 172.21.2.101:0/3257921499 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e201b9680 0x7f0e2018fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:05.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.930 7f0e282f3700 1 --2- 172.21.2.101:0/3257921499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e201b9280 0x7f0e2018f750 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:05.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.930 7f0e282f3700 1 -- 172.21.2.101:0/3257921499 >> 172.21.2.101:0/3257921499 conn(0x7f0e200845b0 msgr2=0x7f0e2018c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:05.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.930 7f0e282f3700 1 -- 172.21.2.101:0/3257921499 shutdown_connections 2021-02-10T11:39:05.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:05.930 7f0e282f3700 1 -- 172.21.2.101:0/3257921499 wait complete. 2021-02-10T11:39:05.942 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:39:06.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.360 7f5416fe6700 1 Processor -- start 2021-02-10T11:39:06.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.360 7f5416fe6700 1 -- start start 2021-02-10T11:39:06.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.360 7f5416fe6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f54100d1c80 0x7f54100cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:06.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.360 7f5416fe6700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f54100cf590 0x7f54100cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:06.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.360 7f5416fe6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54100fe1d0 con 0x7f54100d1880 2021-02-10T11:39:06.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.360 7f5416fe6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54101b0cc0 con 0x7f54100d1c80 2021-02-10T11:39:06.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.360 7f5416fe6700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5410109a70 con 0x7f54100cf590 2021-02-10T11:39:06.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.360 7f540ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f54100d1c80 0x7f54100cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:06.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.360 7f540ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f54100d1c80 0x7f54100cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55300/0 (socket says 172.21.2.101:55300) 2021-02-10T11:39:06.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.360 7f540ffff700 1 -- 172.21.2.101:0/430037291 learned_addr learned my addr 172.21.2.101:0/430037291 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:06.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.361 7f540ffff700 1 -- 172.21.2.101:0/430037291 >> v2:172.21.2.108:3300/0 conn(0x7f54100cf590 msgr2=0x7f54100cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:06.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.361 7f5415603700 1 --2- 172.21.2.101:0/430037291 >> v2:172.21.2.108:3300/0 conn(0x7f54100cf590 0x7f54100cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:06.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.361 7f540ffff700 1 --2- 172.21.2.101:0/430037291 >> v2:172.21.2.108:3300/0 conn(0x7f54100cf590 0x7f54100cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:06.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.361 7f540ffff700 1 -- 172.21.2.101:0/430037291 >> v1:172.21.2.108:6789/0 conn(0x7f54100d1880 legacy=0x7f54101b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:06.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.361 7f540ffff700 1 -- 172.21.2.101:0/430037291 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5410088070 con 0x7f54100d1c80 2021-02-10T11:39:06.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.361 7f540ffff700 1 --2- 172.21.2.101:0/430037291 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54100d1c80 0x7f54100cef70 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 rx=0x7f5400006700 tx=0x7f5400009b00).ready entity=mon.0 client_cookie=163009e97ef5b268 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:06.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.361 7f540f7fe700 1 -- 172.21.2.101:0/430037291 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f540000c0c0 con 0x7f54100d1c80 2021-02-10T11:39:06.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.361 7f540f7fe700 1 -- 172.21.2.101:0/430037291 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540000c400 con 0x7f54100d1c80 2021-02-10T11:39:06.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.361 7f540f7fe700 1 -- 172.21.2.101:0/430037291 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f540000c920 con 0x7f54100d1c80 2021-02-10T11:39:06.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.361 7f5416fe6700 1 -- 172.21.2.101:0/430037291 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54100d1c80 msgr2=0x7f54100cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:06.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.361 7f5416fe6700 1 --2- 172.21.2.101:0/430037291 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54100d1c80 0x7f54100cef70 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 rx=0x7f5400006700 tx=0x7f5400009b00).stop 2021-02-10T11:39:06.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.362 7f5416fe6700 1 -- 172.21.2.101:0/430037291 shutdown_connections 2021-02-10T11:39:06.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.362 7f5416fe6700 1 --2- 172.21.2.101:0/430037291 >> v2:172.21.2.108:3300/0 conn(0x7f54100cf590 0x7f54100cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:06.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.362 7f5416fe6700 1 --2- 172.21.2.101:0/430037291 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54100d1c80 0x7f54100cef70 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:06.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.362 7f5416fe6700 1 -- 172.21.2.101:0/430037291 >> 172.21.2.101:0/430037291 conn(0x7f54100ce430 msgr2=0x7f54100d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:06.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.362 7f5416fe6700 1 -- 172.21.2.101:0/430037291 shutdown_connections 2021-02-10T11:39:06.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.362 7f5416fe6700 1 -- 172.21.2.101:0/430037291 wait complete. 2021-02-10T11:39:06.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.377 7f5416fe6700 1 Processor -- start 2021-02-10T11:39:06.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.377 7f5416fe6700 1 -- start start 2021-02-10T11:39:06.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.377 7f5416fe6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54101b9470 0x7f54101ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:06.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.377 7f5416fe6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54101aecf0 0x7f54100957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:06.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.377 7f5416fe6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54101af0f0 0x7f5410095e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:06.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.377 7f5416fe6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5410096d70 con 0x7f54101b9470 2021-02-10T11:39:06.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.377 7f5416fe6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54100a7780 con 0x7f54101af0f0 2021-02-10T11:39:06.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.377 7f5416fe6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54101b9870 con 0x7f54101aecf0 2021-02-10T11:39:06.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.378 7f5414e02700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54101b9470 0x7f54101ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:06.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.378 7f5414e02700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54101b9470 0x7f54101ae6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55304/0 (socket says 172.21.2.101:55304) 2021-02-10T11:39:06.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.378 7f5414e02700 1 -- 172.21.2.101:0/2670309490 learned_addr learned my addr 172.21.2.101:0/2670309490 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:06.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.378 7f5414e02700 1 -- 172.21.2.101:0/2670309490 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54101aecf0 msgr2=0x7f54100957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:06.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.378 7f5414e02700 1 --2- 172.21.2.101:0/2670309490 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54101aecf0 0x7f54100957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:06.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.378 7f5414e02700 1 -- 172.21.2.101:0/2670309490 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54101af0f0 msgr2=0x7f5410095e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:06.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.378 7f5414e02700 1 --2- 172.21.2.101:0/2670309490 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54101af0f0 0x7f5410095e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:06.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.378 7f5414e02700 1 -- 172.21.2.101:0/2670309490 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541019e6b0 con 0x7f54101b9470 2021-02-10T11:39:06.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.378 7f5414e02700 1 --2- 172.21.2.101:0/2670309490 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54101b9470 0x7f54101ae6d0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 rx=0x7f53f800b5b0 tx=0x7f53f800b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:06.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.378 7f540d7fa700 1 -- 172.21.2.101:0/2670309490 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f53f800bd40 con 0x7f54101b9470 2021-02-10T11:39:06.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.378 7f540d7fa700 1 -- 172.21.2.101:0/2670309490 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f8006020 con 0x7f54101b9470 2021-02-10T11:39:06.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.378 7f540d7fa700 1 -- 172.21.2.101:0/2670309490 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f53f80054a0 con 0x7f54101b9470 2021-02-10T11:39:06.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.378 7f5416fe6700 1 -- 172.21.2.101:0/2670309490 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541019e6b0 con 0x7f54101b9470 2021-02-10T11:39:06.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.378 7f5416fe6700 1 -- 172.21.2.101:0/2670309490 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5410096720 con 0x7f54101b9470 2021-02-10T11:39:06.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.379 7f540d7fa700 1 -- 172.21.2.101:0/2670309490 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f53f8006200 con 0x7f54101b9470 2021-02-10T11:39:06.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.379 7f540d7fa700 1 --2- 172.21.2.101:0/2670309490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f53f40186d0 0x7f53f401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:06.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.379 7f540d7fa700 1 -- 172.21.2.101:0/2670309490 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f53f80327f0 con 0x7f54101b9470 2021-02-10T11:39:06.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.379 7f540ffff700 1 --2- 172.21.2.101:0/2670309490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f53f40186d0 0x7f53f401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:06.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.379 7f540ffff700 1 --2- 172.21.2.101:0/2670309490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f53f40186d0 0x7f53f401abd0 crc :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:06.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.409 7f5416fe6700 1 -- 172.21.2.101:0/2670309490 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54100829a0 con 0x7f54101b9470 2021-02-10T11:39:06.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.412 7f540d7fa700 1 -- 172.21.2.101:0/2670309490 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f53f8000c60 con 0x7f54101b9470 2021-02-10T11:39:06.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.541 7f5416fe6700 1 -- 172.21.2.101:0/2670309490 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f54100a7780 con 0x7f54101b9470 2021-02-10T11:39:06.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.544 7f540d7fa700 1 -- 172.21.2.101:0/2670309490 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v54) v1 ==== 184+0+0 (secure 0 0 0) 0x7f53f80061e0 con 0x7f54101b9470 2021-02-10T11:39:06.552 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.1 2021-02-10T11:39:06.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.549 7f5416fe6700 1 -- 172.21.2.101:0/2670309490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f53f40186d0 msgr2=0x7f53f401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:06.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.549 7f5416fe6700 1 --2- 172.21.2.101:0/2670309490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f53f40186d0 0x7f53f401abd0 crc :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:06.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.549 7f5416fe6700 1 -- 172.21.2.101:0/2670309490 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54101b9470 msgr2=0x7f54101ae6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:06.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.549 7f5416fe6700 1 --2- 172.21.2.101:0/2670309490 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54101b9470 0x7f54101ae6d0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 rx=0x7f53f800b5b0 tx=0x7f53f800b950).stop 2021-02-10T11:39:06.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.550 7f5416fe6700 1 -- 172.21.2.101:0/2670309490 shutdown_connections 2021-02-10T11:39:06.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.550 7f5416fe6700 1 --2- 172.21.2.101:0/2670309490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f53f40186d0 0x7f53f401abd0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:06.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.550 7f5416fe6700 1 --2- 172.21.2.101:0/2670309490 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54101aecf0 0x7f54100957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:06.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.550 7f5416fe6700 1 --2- 172.21.2.101:0/2670309490 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54101af0f0 0x7f5410095e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:06.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.550 7f5416fe6700 1 --2- 172.21.2.101:0/2670309490 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54101b9470 0x7f54101ae6d0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:06.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.550 7f5416fe6700 1 -- 172.21.2.101:0/2670309490 >> 172.21.2.101:0/2670309490 conn(0x7f54100845b0 msgr2=0x7f541018c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:06.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.550 7f5416fe6700 1 -- 172.21.2.101:0/2670309490 shutdown_connections 2021-02-10T11:39:06.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.550 7f5416fe6700 1 -- 172.21.2.101:0/2670309490 wait complete. 2021-02-10T11:39:06.561 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:39:06.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.972 7f3dbd290700 1 Processor -- start 2021-02-10T11:39:06.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.972 7f3dbd290700 1 -- start start 2021-02-10T11:39:06.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.973 7f3dbd290700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3db80d1c80 0x7f3db80cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:06.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.973 7f3dbd290700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3db80cf590 0x7f3db80cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:06.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.973 7f3dbd290700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db80fe1d0 con 0x7f3db80d1880 2021-02-10T11:39:06.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.973 7f3dbd290700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db81b0cc0 con 0x7f3db80cf590 2021-02-10T11:39:06.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.973 7f3dbd290700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db8109a70 con 0x7f3db80d1c80 2021-02-10T11:39:06.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.973 7f3db707d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f3db80d1880 0x7f3db81b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55806/0 (socket says 172.21.2.101:55806) 2021-02-10T11:39:06.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.973 7f3db707d700 1 -- 172.21.2.101:0/2348855189 learned_addr learned my addr 172.21.2.101:0/2348855189 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:06.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.973 7f3db787e700 1 --2- 172.21.2.101:0/2348855189 >> v2:172.21.2.101:3300/0 conn(0x7f3db80cf590 0x7f3db80cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:06.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.973 7f3db787e700 1 -- 172.21.2.101:0/2348855189 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3db80d1c80 msgr2=0x7f3db80cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:06.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.973 7f3db787e700 1 --2- 172.21.2.101:0/2348855189 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3db80d1c80 0x7f3db80cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:06.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.973 7f3db787e700 1 -- 172.21.2.101:0/2348855189 >> v1:172.21.2.101:6789/0 conn(0x7f3db80d1880 legacy=0x7f3db81b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:06.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.973 7f3db787e700 1 -- 172.21.2.101:0/2348855189 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db8088070 con 0x7f3db80cf590 2021-02-10T11:39:06.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.974 7f3db787e700 1 --2- 172.21.2.101:0/2348855189 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3db80cf590 0x7f3db80cf990 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 rx=0x7f3db0006e30 tx=0x7f3db0009e60).ready entity=mon.0 client_cookie=999e82a884b06753 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:06.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.974 7f3db607b700 1 -- 172.21.2.101:0/2348855189 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3db0006a50 con 0x7f3db80cf590 2021-02-10T11:39:06.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.974 7f3db607b700 1 -- 172.21.2.101:0/2348855189 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db000b300 con 0x7f3db80cf590 2021-02-10T11:39:06.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.974 7f3db607b700 1 -- 172.21.2.101:0/2348855189 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3db000b820 con 0x7f3db80cf590 2021-02-10T11:39:06.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.974 7f3dbd290700 1 -- 172.21.2.101:0/2348855189 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3db80cf590 msgr2=0x7f3db80cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:06.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.974 7f3dbd290700 1 --2- 172.21.2.101:0/2348855189 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3db80cf590 0x7f3db80cf990 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 rx=0x7f3db0006e30 tx=0x7f3db0009e60).stop 2021-02-10T11:39:06.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.974 7f3dbd290700 1 -- 172.21.2.101:0/2348855189 shutdown_connections 2021-02-10T11:39:06.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.974 7f3dbd290700 1 --2- 172.21.2.101:0/2348855189 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3db80d1c80 0x7f3db80cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:06.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.974 7f3dbd290700 1 --2- 172.21.2.101:0/2348855189 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3db80cf590 0x7f3db80cf990 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:06.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.974 7f3dbd290700 1 -- 172.21.2.101:0/2348855189 >> 172.21.2.101:0/2348855189 conn(0x7f3db80ce430 msgr2=0x7f3db80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:06.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.975 7f3dbd290700 1 -- 172.21.2.101:0/2348855189 shutdown_connections 2021-02-10T11:39:06.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.975 7f3dbd290700 1 -- 172.21.2.101:0/2348855189 wait complete. 2021-02-10T11:39:06.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.989 7f3dbd290700 1 Processor -- start 2021-02-10T11:39:06.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.989 7f3dbd290700 1 -- start start 2021-02-10T11:39:06.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.989 7f3dbd290700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3db81aaf10 0x7f3db81ab310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:06.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.989 7f3dbd290700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3db81a3750 0x7f3db81a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:06.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.989 7f3dbd290700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3db81a4170 0x7f3db81a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:06.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.989 7f3dbd290700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db81abe10 con 0x7f3db81aaf10 2021-02-10T11:39:06.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.989 7f3dbd290700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db80b4f10 con 0x7f3db81a3750 2021-02-10T11:39:06.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.989 7f3dbd290700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db81ab930 con 0x7f3db81a4170 2021-02-10T11:39:06.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.990 7f3db707d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3db81aaf10 0x7f3db81ab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:06.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.990 7f3db707d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3db81aaf10 0x7f3db81ab310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55316/0 (socket says 172.21.2.101:55316) 2021-02-10T11:39:06.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.990 7f3db707d700 1 -- 172.21.2.101:0/2878033398 learned_addr learned my addr 172.21.2.101:0/2878033398 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:06.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.990 7f3db707d700 1 -- 172.21.2.101:0/2878033398 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3db81a4170 msgr2=0x7f3db81a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:06.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.990 7f3db707d700 1 --2- 172.21.2.101:0/2878033398 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3db81a4170 0x7f3db81a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:06.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.990 7f3db707d700 1 -- 172.21.2.101:0/2878033398 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3db81a3750 msgr2=0x7f3db81a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:06.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.990 7f3db707d700 1 --2- 172.21.2.101:0/2878033398 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3db81a3750 0x7f3db81a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:06.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.990 7f3db707d700 1 -- 172.21.2.101:0/2878033398 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db81a5260 con 0x7f3db81aaf10 2021-02-10T11:39:06.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.990 7f3db707d700 1 --2- 172.21.2.101:0/2878033398 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3db81aaf10 0x7f3db81ab310 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 rx=0x7f3da0001410 tx=0x7f3da000a950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:06.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.991 7f3daffff700 1 -- 172.21.2.101:0/2878033398 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3da000ad40 con 0x7f3db81aaf10 2021-02-10T11:39:06.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.991 7f3daffff700 1 -- 172.21.2.101:0/2878033398 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da00043a0 con 0x7f3db81aaf10 2021-02-10T11:39:06.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.991 7f3dbd290700 1 -- 172.21.2.101:0/2878033398 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db81a5260 con 0x7f3db81aaf10 2021-02-10T11:39:06.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.991 7f3dbd290700 1 -- 172.21.2.101:0/2878033398 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db81a57f0 con 0x7f3db81aaf10 2021-02-10T11:39:06.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.991 7f3daffff700 1 -- 172.21.2.101:0/2878033398 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3da00047b0 con 0x7f3db81aaf10 2021-02-10T11:39:06.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.991 7f3daffff700 1 -- 172.21.2.101:0/2878033398 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3da0004580 con 0x7f3db81aaf10 2021-02-10T11:39:06.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.992 7f3daffff700 1 --2- 172.21.2.101:0/2878033398 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3d9c0186c0 0x7f3d9c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:06.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.992 7f3daffff700 1 -- 172.21.2.101:0/2878033398 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3da00327b0 con 0x7f3db81aaf10 2021-02-10T11:39:06.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.992 7f3db687c700 1 --2- 172.21.2.101:0/2878033398 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3d9c0186c0 0x7f3d9c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:06.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:06.992 7f3db687c700 1 --2- 172.21.2.101:0/2878033398 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3d9c0186c0 0x7f3d9c01abc0 crc :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:07.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.023 7f3dbd290700 1 -- 172.21.2.101:0/2878033398 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db80829a0 con 0x7f3db81aaf10 2021-02-10T11:39:07.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.026 7f3daffff700 1 -- 172.21.2.101:0/2878033398 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3da0000ef0 con 0x7f3db81aaf10 2021-02-10T11:39:07.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.157 7f3dbd290700 1 -- 172.21.2.101:0/2878033398 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f3db80829a0 con 0x7f3db81aaf10 2021-02-10T11:39:07.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.160 7f3daffff700 1 -- 172.21.2.101:0/2878033398 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v55) v1 ==== 184+0+0 (secure 0 0 0) 0x7f3da00045c0 con 0x7f3db81aaf10 2021-02-10T11:39:07.168 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.2 2021-02-10T11:39:07.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.165 7f3dbd290700 1 -- 172.21.2.101:0/2878033398 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3d9c0186c0 msgr2=0x7f3d9c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:07.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.166 7f3dbd290700 1 --2- 172.21.2.101:0/2878033398 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3d9c0186c0 0x7f3d9c01abc0 crc :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:07.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.166 7f3dbd290700 1 -- 172.21.2.101:0/2878033398 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3db81aaf10 msgr2=0x7f3db81ab310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:07.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.166 7f3dbd290700 1 --2- 172.21.2.101:0/2878033398 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3db81aaf10 0x7f3db81ab310 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 rx=0x7f3da0001410 tx=0x7f3da000a950).stop 2021-02-10T11:39:07.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.166 7f3dbd290700 1 -- 172.21.2.101:0/2878033398 shutdown_connections 2021-02-10T11:39:07.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.166 7f3dbd290700 1 --2- 172.21.2.101:0/2878033398 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3d9c0186c0 0x7f3d9c01abc0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:07.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.166 7f3dbd290700 1 --2- 172.21.2.101:0/2878033398 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3db81a4170 0x7f3db81a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:07.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.166 7f3dbd290700 1 --2- 172.21.2.101:0/2878033398 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3db81a3750 0x7f3db81a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:07.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.166 7f3dbd290700 1 --2- 172.21.2.101:0/2878033398 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3db81aaf10 0x7f3db81ab310 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:07.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.166 7f3dbd290700 1 -- 172.21.2.101:0/2878033398 >> 172.21.2.101:0/2878033398 conn(0x7f3db819f880 msgr2=0x7f3db81a71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:07.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.166 7f3dbd290700 1 -- 172.21.2.101:0/2878033398 shutdown_connections 2021-02-10T11:39:07.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.166 7f3dbd290700 1 -- 172.21.2.101:0/2878033398 wait complete. 2021-02-10T11:39:07.178 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:39:07.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.597 7f4366588700 1 Processor -- start 2021-02-10T11:39:07.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.597 7f4366588700 1 -- start start 2021-02-10T11:39:07.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.597 7f4366588700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f43600d1c80 0x7f43600cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:07.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.597 7f4366588700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43600cf590 0x7f43600cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:07.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.597 7f4366588700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43600fe1d0 con 0x7f43600d1880 2021-02-10T11:39:07.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.597 7f4366588700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43601b0cc0 con 0x7f43600d1c80 2021-02-10T11:39:07.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.597 7f4366588700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4360109a70 con 0x7f43600cf590 2021-02-10T11:39:07.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.598 7f4364ba5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43600cf590 0x7f43600cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:07.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.598 7f435ffff700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f43600d1880 0x7f43601b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:49228/0 (socket says 172.21.2.101:49228) 2021-02-10T11:39:07.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.598 7f435ffff700 1 -- 172.21.2.101:0/1528586911 learned_addr learned my addr 172.21.2.101:0/1528586911 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:07.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.598 7f435f7fe700 1 --2- 172.21.2.101:0/1528586911 >> v2:172.21.2.108:3300/0 conn(0x7f43600d1c80 0x7f43600cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:07.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.598 7f4364ba5700 1 -- 172.21.2.101:0/1528586911 >> v2:172.21.2.108:3300/0 conn(0x7f43600d1c80 msgr2=0x7f43600cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:07.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.598 7f4364ba5700 1 --2- 172.21.2.101:0/1528586911 >> v2:172.21.2.108:3300/0 conn(0x7f43600d1c80 0x7f43600cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:07.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.598 7f4364ba5700 1 -- 172.21.2.101:0/1528586911 >> v1:172.21.2.108:6789/0 conn(0x7f43600d1880 legacy=0x7f43601b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:07.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.598 7f4364ba5700 1 -- 172.21.2.101:0/1528586911 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4360088070 con 0x7f43600cf590 2021-02-10T11:39:07.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.598 7f435f7fe700 1 --2- 172.21.2.101:0/1528586911 >> v2:172.21.2.108:3300/0 conn(0x7f43600d1c80 0x7f43600cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:39:07.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.598 7f4364ba5700 1 --2- 172.21.2.101:0/1528586911 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43600cf590 0x7f43600cf990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f4350006ad0 tx=0x7f4350009ad0).ready entity=mon.2 client_cookie=fea7b6ae6ef9f14a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:07.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.598 7f435effd700 1 -- 172.21.2.101:0/1528586911 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f435000c0c0 con 0x7f43600cf590 2021-02-10T11:39:07.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.599 7f435effd700 1 -- 172.21.2.101:0/1528586911 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435000c440 con 0x7f43600cf590 2021-02-10T11:39:07.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.599 7f435effd700 1 -- 172.21.2.101:0/1528586911 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f435000c960 con 0x7f43600cf590 2021-02-10T11:39:07.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.599 7f4366588700 1 -- 172.21.2.101:0/1528586911 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43600cf590 msgr2=0x7f43600cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:07.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.599 7f4366588700 1 --2- 172.21.2.101:0/1528586911 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43600cf590 0x7f43600cf990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f4350006ad0 tx=0x7f4350009ad0).stop 2021-02-10T11:39:07.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.599 7f4366588700 1 -- 172.21.2.101:0/1528586911 shutdown_connections 2021-02-10T11:39:07.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.599 7f4366588700 1 --2- 172.21.2.101:0/1528586911 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43600cf590 0x7f43600cf990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:07.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.599 7f4366588700 1 --2- 172.21.2.101:0/1528586911 >> v2:172.21.2.108:3300/0 conn(0x7f43600d1c80 0x7f43600cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:07.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.599 7f4366588700 1 -- 172.21.2.101:0/1528586911 >> 172.21.2.101:0/1528586911 conn(0x7f43600ce430 msgr2=0x7f43600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:07.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.599 7f4366588700 1 -- 172.21.2.101:0/1528586911 shutdown_connections 2021-02-10T11:39:07.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.599 7f4366588700 1 -- 172.21.2.101:0/1528586911 wait complete. 2021-02-10T11:39:07.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.614 7f4366588700 1 Processor -- start 2021-02-10T11:39:07.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.614 7f4366588700 1 -- start start 2021-02-10T11:39:07.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.614 7f4366588700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f43601a0580 0x7f4360086fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:07.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.614 7f4366588700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43600b66b0 0x7f43600b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:07.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.614 7f4366588700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4360082bb0 0x7f4360082fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:07.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.614 7f4366588700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43600875d0 con 0x7f43600b66b0 2021-02-10T11:39:07.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.614 7f4366588700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43600835d0 con 0x7f43601a0580 2021-02-10T11:39:07.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.614 7f4366588700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43600837f0 con 0x7f4360082bb0 2021-02-10T11:39:07.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.614 7f435f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43600b66b0 0x7f43600b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:07.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.614 7f4364ba5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4360082bb0 0x7f4360082fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:07.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.614 7f4364ba5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4360082bb0 0x7f4360082fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:48064/0 (socket says 172.21.2.101:48064) 2021-02-10T11:39:07.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.614 7f4364ba5700 1 -- 172.21.2.101:0/3411959989 learned_addr learned my addr 172.21.2.101:0/3411959989 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:07.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.614 7f435f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43600b66b0 0x7f43600b6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55330/0 (socket says 172.21.2.101:55330) 2021-02-10T11:39:07.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.614 7f435ffff700 1 --2- 172.21.2.101:0/3411959989 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f43601a0580 0x7f4360086fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:07.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.615 7f4364ba5700 1 -- 172.21.2.101:0/3411959989 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f43601a0580 msgr2=0x7f4360086fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:07.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.615 7f4364ba5700 1 --2- 172.21.2.101:0/3411959989 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f43601a0580 0x7f4360086fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:07.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.615 7f4364ba5700 1 -- 172.21.2.101:0/3411959989 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43600b66b0 msgr2=0x7f43600b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:07.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.615 7f4364ba5700 1 --2- 172.21.2.101:0/3411959989 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43600b66b0 0x7f43600b6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:07.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.615 7f4364ba5700 1 -- 172.21.2.101:0/3411959989 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4360083a10 con 0x7f4360082bb0 2021-02-10T11:39:07.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.615 7f4364ba5700 1 --2- 172.21.2.101:0/3411959989 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4360082bb0 0x7f4360082fb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f435000b610 tx=0x7f435000b9b0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:07.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.615 7f435cff9700 1 -- 172.21.2.101:0/3411959989 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f43500068a0 con 0x7f4360082bb0 2021-02-10T11:39:07.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.615 7f435cff9700 1 -- 172.21.2.101:0/3411959989 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435000bda0 con 0x7f4360082bb0 2021-02-10T11:39:07.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.615 7f4366588700 1 -- 172.21.2.101:0/3411959989 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4360083a10 con 0x7f4360082bb0 2021-02-10T11:39:07.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.615 7f435cff9700 1 -- 172.21.2.101:0/3411959989 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4350002080 con 0x7f4360082bb0 2021-02-10T11:39:07.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.615 7f4366588700 1 -- 172.21.2.101:0/3411959989 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4360083a10 con 0x7f4360082bb0 2021-02-10T11:39:07.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.616 7f435cff9700 1 -- 172.21.2.101:0/3411959989 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4350000c80 con 0x7f4360082bb0 2021-02-10T11:39:07.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.616 7f435cff9700 1 --2- 172.21.2.101:0/3411959989 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4340018730 0x7f434001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:07.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.616 7f435cff9700 1 -- 172.21.2.101:0/3411959989 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f43500326a0 con 0x7f4360082bb0 2021-02-10T11:39:07.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.616 7f435ffff700 1 --2- 172.21.2.101:0/3411959989 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4340018730 0x7f434001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:07.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.616 7f435ffff700 1 --2- 172.21.2.101:0/3411959989 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4340018730 0x7f434001abc0 crc :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:07.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.647 7f4366588700 1 -- 172.21.2.101:0/3411959989 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43600875d0 con 0x7f4360082bb0 2021-02-10T11:39:07.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.651 7f435cff9700 1 -- 172.21.2.101:0/3411959989 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4350000a70 con 0x7f4360082bb0 2021-02-10T11:39:07.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.790 7f4366588700 1 -- 172.21.2.101:0/3411959989 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f43600875d0 con 0x7f4360082bb0 2021-02-10T11:39:07.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.795 7f435cff9700 1 -- 172.21.2.101:0/3411959989 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v56) v1 ==== 184+0+0 (secure 0 0 0) 0x7f4350000c80 con 0x7f4360082bb0 2021-02-10T11:39:07.801 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.3 2021-02-10T11:39:07.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.799 7f4366588700 1 -- 172.21.2.101:0/3411959989 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4340018730 msgr2=0x7f434001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:07.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.799 7f4366588700 1 --2- 172.21.2.101:0/3411959989 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4340018730 0x7f434001abc0 crc :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:07.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.799 7f4366588700 1 -- 172.21.2.101:0/3411959989 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4360082bb0 msgr2=0x7f4360082fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:07.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.799 7f4366588700 1 --2- 172.21.2.101:0/3411959989 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4360082bb0 0x7f4360082fb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f435000b610 tx=0x7f435000b9b0).stop 2021-02-10T11:39:07.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.799 7f4366588700 1 -- 172.21.2.101:0/3411959989 shutdown_connections 2021-02-10T11:39:07.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.799 7f4366588700 1 --2- 172.21.2.101:0/3411959989 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4340018730 0x7f434001abc0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:07.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.799 7f4366588700 1 --2- 172.21.2.101:0/3411959989 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4360082bb0 0x7f4360082fb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:07.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.799 7f4366588700 1 --2- 172.21.2.101:0/3411959989 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43600b66b0 0x7f43600b6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:07.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.799 7f4366588700 1 --2- 172.21.2.101:0/3411959989 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f43601a0580 0x7f4360086fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:07.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.799 7f4366588700 1 -- 172.21.2.101:0/3411959989 >> 172.21.2.101:0/3411959989 conn(0x7f43600afc60 msgr2=0x7f43600b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:07.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.799 7f4366588700 1 -- 172.21.2.101:0/3411959989 shutdown_connections 2021-02-10T11:39:07.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:07.799 7f4366588700 1 -- 172.21.2.101:0/3411959989 wait complete. 2021-02-10T11:39:07.811 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:39:08.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.218 7f88a792d700 1 Processor -- start 2021-02-10T11:39:08.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.218 7f88a792d700 1 -- start start 2021-02-10T11:39:08.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.219 7f88a792d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88a00d1870 0x7f88a01b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:08.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.219 7f88a792d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a00f0b10 con 0x7f88a00f0650 2021-02-10T11:39:08.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.219 7f88a792d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a00d0310 con 0x7f88a01b0a10 2021-02-10T11:39:08.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.219 7f88a792d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a00e5ed0 con 0x7f88a00d1870 2021-02-10T11:39:08.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.219 7f88a5f4a700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f88a00f0650 0x7f88a00cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55832/0 (socket says 172.21.2.101:55832) 2021-02-10T11:39:08.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.219 7f88a5f4a700 1 -- 172.21.2.101:0/3690171009 learned_addr learned my addr 172.21.2.101:0/3690171009 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:08.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.219 7f88a5749700 1 --2- 172.21.2.101:0/3690171009 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88a00d1870 0x7f88a01b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:08.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.219 7f8893fff700 1 -- 172.21.2.101:0/3690171009 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3411566089 0 0) 0x7f8898001500 con 0x7f88a00f0650 2021-02-10T11:39:08.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.219 7f8893fff700 1 -- 172.21.2.101:0/3690171009 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f888c002fe0 con 0x7f88a00f0650 2021-02-10T11:39:08.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.219 7f88a5749700 1 -- 172.21.2.101:0/3690171009 >> v1:172.21.2.108:6789/0 conn(0x7f88a01b0a10 legacy=0x7f88a00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:08.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.219 7f88a5749700 1 -- 172.21.2.101:0/3690171009 >> v1:172.21.2.101:6789/0 conn(0x7f88a00f0650 legacy=0x7f88a00cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:08.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.219 7f88a5749700 1 -- 172.21.2.101:0/3690171009 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a00d0530 con 0x7f88a00d1870 2021-02-10T11:39:08.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.220 7f88a5749700 1 --2- 172.21.2.101:0/3690171009 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88a00d1870 0x7f88a01b03f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f889c006650 tx=0x7f889c009a80).ready entity=mon.2 client_cookie=d1c7170e045c3bb7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:08.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.220 7f8893fff700 1 -- 172.21.2.101:0/3690171009 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f889c00c0c0 con 0x7f88a00d1870 2021-02-10T11:39:08.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.220 7f8893fff700 1 -- 172.21.2.101:0/3690171009 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889c00c340 con 0x7f88a00d1870 2021-02-10T11:39:08.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.220 7f8893fff700 1 -- 172.21.2.101:0/3690171009 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f889c00c860 con 0x7f88a00d1870 2021-02-10T11:39:08.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.220 7f88a792d700 1 -- 172.21.2.101:0/3690171009 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88a00d1870 msgr2=0x7f88a01b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:08.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.220 7f88a792d700 1 --2- 172.21.2.101:0/3690171009 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88a00d1870 0x7f88a01b03f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f889c006650 tx=0x7f889c009a80).stop 2021-02-10T11:39:08.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.220 7f88a792d700 1 -- 172.21.2.101:0/3690171009 shutdown_connections 2021-02-10T11:39:08.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.220 7f88a792d700 1 --2- 172.21.2.101:0/3690171009 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88a00d1870 0x7f88a01b03f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:08.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.220 7f88a792d700 1 -- 172.21.2.101:0/3690171009 >> 172.21.2.101:0/3690171009 conn(0x7f88a00ce450 msgr2=0x7f88a00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:08.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.221 7f88a792d700 1 -- 172.21.2.101:0/3690171009 shutdown_connections 2021-02-10T11:39:08.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.221 7f88a792d700 1 -- 172.21.2.101:0/3690171009 wait complete. 2021-02-10T11:39:08.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.235 7f88a792d700 1 Processor -- start 2021-02-10T11:39:08.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.235 7f88a792d700 1 -- start start 2021-02-10T11:39:08.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.235 7f88a792d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88a00b4f30 0x7f88a018b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:08.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.235 7f88a792d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f88a0093f40 0x7f88a0094340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:08.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.235 7f88a792d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88a0094960 0x7f88a0187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:08.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.235 7f88a792d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a0094e90 con 0x7f88a00b4f30 2021-02-10T11:39:08.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.235 7f88a792d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a018bff0 con 0x7f88a0093f40 2021-02-10T11:39:08.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.235 7f88a792d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a008f780 con 0x7f88a0094960 2021-02-10T11:39:08.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.236 7f88a5749700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88a00b4f30 0x7f88a018b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:08.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.236 7f88a5749700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88a00b4f30 0x7f88a018b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55342/0 (socket says 172.21.2.101:55342) 2021-02-10T11:39:08.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.236 7f88a5749700 1 -- 172.21.2.101:0/1452952931 learned_addr learned my addr 172.21.2.101:0/1452952931 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:08.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.236 7f88a5749700 1 -- 172.21.2.101:0/1452952931 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88a0094960 msgr2=0x7f88a0187e50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:08.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.236 7f88a5749700 1 --2- 172.21.2.101:0/1452952931 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88a0094960 0x7f88a0187e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:08.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.236 7f88a5749700 1 -- 172.21.2.101:0/1452952931 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f88a0093f40 msgr2=0x7f88a0094340 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:08.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.236 7f88a5749700 1 --2- 172.21.2.101:0/1452952931 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f88a0093f40 0x7f88a0094340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:08.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.236 7f88a5749700 1 -- 172.21.2.101:0/1452952931 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a0188470 con 0x7f88a00b4f30 2021-02-10T11:39:08.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.236 7f88a5749700 1 --2- 172.21.2.101:0/1452952931 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88a00b4f30 0x7f88a018b9d0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 rx=0x7f889c009650 tx=0x7f889c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:08.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.236 7f8891ffb700 1 -- 172.21.2.101:0/1452952931 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f889c00bd50 con 0x7f88a00b4f30 2021-02-10T11:39:08.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.236 7f8891ffb700 1 -- 172.21.2.101:0/1452952931 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889c006030 con 0x7f88a00b4f30 2021-02-10T11:39:08.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.236 7f88a792d700 1 -- 172.21.2.101:0/1452952931 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a0188470 con 0x7f88a00b4f30 2021-02-10T11:39:08.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.236 7f88a792d700 1 -- 172.21.2.101:0/1452952931 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a0188470 con 0x7f88a00b4f30 2021-02-10T11:39:08.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.236 7f8891ffb700 1 -- 172.21.2.101:0/1452952931 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f889c005560 con 0x7f88a00b4f30 2021-02-10T11:39:08.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.237 7f8891ffb700 1 -- 172.21.2.101:0/1452952931 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f889c0052e0 con 0x7f88a00b4f30 2021-02-10T11:39:08.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.237 7f8891ffb700 1 --2- 172.21.2.101:0/1452952931 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8880018730 0x7f888001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:08.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.237 7f8891ffb700 1 -- 172.21.2.101:0/1452952931 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f889c0327c0 con 0x7f88a00b4f30 2021-02-10T11:39:08.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.237 7f88a4f48700 1 --2- 172.21.2.101:0/1452952931 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8880018730 0x7f888001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:08.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.237 7f88a4f48700 1 --2- 172.21.2.101:0/1452952931 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8880018730 0x7f888001ac10 crc :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:08.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.268 7f88a792d700 1 -- 172.21.2.101:0/1452952931 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88a00ce450 con 0x7f88a00b4f30 2021-02-10T11:39:08.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.271 7f8891ffb700 1 -- 172.21.2.101:0/1452952931 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f889c0009a0 con 0x7f88a00b4f30 2021-02-10T11:39:08.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.399 7f88a792d700 1 -- 172.21.2.101:0/1452952931 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88a00ce450 con 0x7f88a00b4f30 2021-02-10T11:39:08.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.400 7f8891ffb700 1 -- 172.21.2.101:0/1452952931 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f889c00bd50 con 0x7f88a00b4f30 2021-02-10T11:39:08.404 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:39:08.404 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/4210469578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":4210469578},{"type":"v1","addr":"172.21.2.108:6833","nonce":4210469578}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:39:02.063078","modified":"2021-02-10 11:39:04.073713","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6164},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6164":{"gid":6164,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.2.101:6835/1113812290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1113812290},{"type":"v1","addr":"172.21.2.101:6835","nonce":1113812290}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-02-10T11:39:08.404 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 71 2021-02-10T11:39:08.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.402 7f88a792d700 1 -- 172.21.2.101:0/1452952931 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8880018730 msgr2=0x7f888001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:08.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.402 7f88a792d700 1 --2- 172.21.2.101:0/1452952931 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8880018730 0x7f888001ac10 crc :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:08.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.402 7f88a792d700 1 -- 172.21.2.101:0/1452952931 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88a00b4f30 msgr2=0x7f88a018b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:08.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.402 7f88a792d700 1 --2- 172.21.2.101:0/1452952931 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88a00b4f30 0x7f88a018b9d0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 rx=0x7f889c009650 tx=0x7f889c00b960).stop 2021-02-10T11:39:08.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.402 7f88a792d700 1 -- 172.21.2.101:0/1452952931 shutdown_connections 2021-02-10T11:39:08.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.402 7f88a792d700 1 --2- 172.21.2.101:0/1452952931 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8880018730 0x7f888001ac10 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:08.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.402 7f88a792d700 1 --2- 172.21.2.101:0/1452952931 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f88a0094960 0x7f88a0187e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:08.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.402 7f88a792d700 1 --2- 172.21.2.101:0/1452952931 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f88a0093f40 0x7f88a0094340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:08.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.402 7f88a792d700 1 --2- 172.21.2.101:0/1452952931 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f88a00b4f30 0x7f88a018b9d0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:08.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.402 7f88a792d700 1 -- 172.21.2.101:0/1452952931 >> 172.21.2.101:0/1452952931 conn(0x7f88a019f8f0 msgr2=0x7f88a01a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:08.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.402 7f88a792d700 1 -- 172.21.2.101:0/1452952931 shutdown_connections 2021-02-10T11:39:08.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:08.402 7f88a792d700 1 -- 172.21.2.101:0/1452952931 wait complete. 2021-02-10T11:39:08.616 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:39:02.063078', 'modified': '2021-02-10 11:39:04.073713', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6164}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6164': {'gid': 6164, 'name': 'a', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.101:6835/1113812290', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.101:6834', 'nonce': 1113812290}, {'type': 'v1', 'addr': '172.21.2.101:6835', 'nonce': 1113812290}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:39:08.616 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:39:08.616 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok status 2021-02-10T11:39:09.017 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:39:09.017 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:39:09.017 INFO:teuthology.orchestra.run.gibba001.stdout: "whoami": 0, 2021-02-10T11:39:09.018 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 6164, 2021-02-10T11:39:09.018 INFO:teuthology.orchestra.run.gibba001.stdout: "want_state": "up:active", 2021-02-10T11:39:09.018 INFO:teuthology.orchestra.run.gibba001.stdout: "state": "up:active", 2021-02-10T11:39:09.018 INFO:teuthology.orchestra.run.gibba001.stdout: "rank_uptime": 6.9487945040000003, 2021-02-10T11:39:09.018 INFO:teuthology.orchestra.run.gibba001.stdout: "mdsmap_epoch": 71, 2021-02-10T11:39:09.018 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch": 94, 2021-02-10T11:39:09.018 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch_barrier": 94, 2021-02-10T11:39:09.018 INFO:teuthology.orchestra.run.gibba001.stdout: "uptime": 12.59173036 2021-02-10T11:39:09.018 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:39:09.022 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 6164, "want_state": "up:active", "state": "up:active", "rank_uptime": 6.9487945040000003, "mdsmap_epoch": 71, "osdmap_epoch": 94, "osdmap_epoch_barrier": 94, "uptime": 12.59173036 } 2021-02-10T11:39:09.022 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:39:09.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.429 7fb4fe5f7700 1 Processor -- start 2021-02-10T11:39:09.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.429 7fb4fe5f7700 1 -- start start 2021-02-10T11:39:09.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.430 7fb4fe5f7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb4f80d1870 0x7fb4f81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:09.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.430 7fb4fe5f7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb4f80f0650 0x7fb4f80cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:09.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.430 7fb4fe5f7700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4f80f0b10 con 0x7fb4f81b0a10 2021-02-10T11:39:09.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.430 7fb4fe5f7700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f80e5ed0 con 0x7fb4f80d1870 2021-02-10T11:39:09.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.430 7fb4fe5f7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f80d1c70 con 0x7fb4f80f0650 2021-02-10T11:39:09.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.430 7fb4f7fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb4f80d1870 0x7fb4f81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:09.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.430 7fb4f77fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb4f81b0a10 0x7fb4f80cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55844/0 (socket says 172.21.2.101:55844) 2021-02-10T11:39:09.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.430 7fb4f77fe700 1 -- 172.21.2.101:0/3345820944 learned_addr learned my addr 172.21.2.101:0/3345820944 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:09.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.430 7fb4fcc14700 1 --2- 172.21.2.101:0/3345820944 >> v2:172.21.2.108:3300/0 conn(0x7fb4f80f0650 0x7fb4f80cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:09.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.431 7fb4f7fff700 1 -- 172.21.2.101:0/3345820944 >> v2:172.21.2.108:3300/0 conn(0x7fb4f80f0650 msgr2=0x7fb4f80cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:09.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.431 7fb4f7fff700 1 --2- 172.21.2.101:0/3345820944 >> v2:172.21.2.108:3300/0 conn(0x7fb4f80f0650 0x7fb4f80cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:09.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.431 7fb4f7fff700 1 -- 172.21.2.101:0/3345820944 >> v1:172.21.2.101:6789/0 conn(0x7fb4f81b0a10 legacy=0x7fb4f80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:09.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.431 7fb4f7fff700 1 -- 172.21.2.101:0/3345820944 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f80d1e90 con 0x7fb4f80d1870 2021-02-10T11:39:09.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.431 7fb4f7fff700 1 --2- 172.21.2.101:0/3345820944 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4f80d1870 0x7fb4f81b03f0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 rx=0x7fb4e8006650 tx=0x7fb4e8009a80).ready entity=mon.0 client_cookie=40aa1435b98280e4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:09.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.431 7fb4f6ffd700 1 -- 172.21.2.101:0/3345820944 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb4e800c0c0 con 0x7fb4f80d1870 2021-02-10T11:39:09.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.431 7fb4f6ffd700 1 -- 172.21.2.101:0/3345820944 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e800c340 con 0x7fb4f80d1870 2021-02-10T11:39:09.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.431 7fb4f6ffd700 1 -- 172.21.2.101:0/3345820944 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb4e800c860 con 0x7fb4f80d1870 2021-02-10T11:39:09.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.431 7fb4fe5f7700 1 -- 172.21.2.101:0/3345820944 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4f80d1870 msgr2=0x7fb4f81b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:09.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.431 7fb4fe5f7700 1 --2- 172.21.2.101:0/3345820944 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4f80d1870 0x7fb4f81b03f0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 rx=0x7fb4e8006650 tx=0x7fb4e8009a80).stop 2021-02-10T11:39:09.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.432 7fb4fe5f7700 1 -- 172.21.2.101:0/3345820944 shutdown_connections 2021-02-10T11:39:09.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.432 7fb4fe5f7700 1 --2- 172.21.2.101:0/3345820944 >> v2:172.21.2.108:3300/0 conn(0x7fb4f80f0650 0x7fb4f80cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:09.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.432 7fb4fe5f7700 1 --2- 172.21.2.101:0/3345820944 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4f80d1870 0x7fb4f81b03f0 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:09.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.432 7fb4fe5f7700 1 -- 172.21.2.101:0/3345820944 >> 172.21.2.101:0/3345820944 conn(0x7fb4f80ce450 msgr2=0x7fb4f80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:09.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.432 7fb4fe5f7700 1 -- 172.21.2.101:0/3345820944 shutdown_connections 2021-02-10T11:39:09.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.432 7fb4fe5f7700 1 -- 172.21.2.101:0/3345820944 wait complete. 2021-02-10T11:39:09.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.446 7fb4fe5f7700 1 Processor -- start 2021-02-10T11:39:09.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.447 7fb4fe5f7700 1 -- start start 2021-02-10T11:39:09.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.447 7fb4fe5f7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4f8195650 0x7fb4f80b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:09.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.447 7fb4fe5f7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4f80b5b80 0x7fb4f81a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:09.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.447 7fb4fe5f7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4f80acfa0 0x7fb4f81a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:09.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.447 7fb4fe5f7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f80b6010 con 0x7fb4f8195650 2021-02-10T11:39:09.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.447 7fb4fe5f7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f81a3a60 con 0x7fb4f80b5b80 2021-02-10T11:39:09.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.447 7fb4fe5f7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f81a3c40 con 0x7fb4f80acfa0 2021-02-10T11:39:09.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.447 7fb4f7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4f8195650 0x7fb4f80b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:09.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.447 7fb4f77fe700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4f80b5b80 0x7fb4f81a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:09.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.447 7fb4f7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4f8195650 0x7fb4f80b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55354/0 (socket says 172.21.2.101:55354) 2021-02-10T11:39:09.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.447 7fb4fcc14700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4f80acfa0 0x7fb4f81a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:09.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.447 7fb4f7fff700 1 -- 172.21.2.101:0/2284773123 learned_addr learned my addr 172.21.2.101:0/2284773123 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:09.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.448 7fb4f7fff700 1 -- 172.21.2.101:0/2284773123 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4f80acfa0 msgr2=0x7fb4f81a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:09.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.448 7fb4f7fff700 1 --2- 172.21.2.101:0/2284773123 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4f80acfa0 0x7fb4f81a3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:09.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.448 7fb4f7fff700 1 -- 172.21.2.101:0/2284773123 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4f80b5b80 msgr2=0x7fb4f81a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:09.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.448 7fb4f7fff700 1 --2- 172.21.2.101:0/2284773123 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4f80b5b80 0x7fb4f81a2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:09.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.448 7fb4f7fff700 1 -- 172.21.2.101:0/2284773123 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f80bcae0 con 0x7fb4f8195650 2021-02-10T11:39:09.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.448 7fb4f7fff700 1 --2- 172.21.2.101:0/2284773123 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4f8195650 0x7fb4f80b5560 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 rx=0x7fb4e8009630 tx=0x7fb4e800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:09.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.448 7fb4f4ff9700 1 -- 172.21.2.101:0/2284773123 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb4e80093a0 con 0x7fb4f8195650 2021-02-10T11:39:09.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.448 7fb4f4ff9700 1 -- 172.21.2.101:0/2284773123 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e800bd70 con 0x7fb4f8195650 2021-02-10T11:39:09.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.448 7fb4f4ff9700 1 -- 172.21.2.101:0/2284773123 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb4e80054f0 con 0x7fb4f8195650 2021-02-10T11:39:09.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.448 7fb4fe5f7700 1 -- 172.21.2.101:0/2284773123 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f80bcae0 con 0x7fb4f8195650 2021-02-10T11:39:09.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.448 7fb4fe5f7700 1 -- 172.21.2.101:0/2284773123 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f80bd070 con 0x7fb4f8195650 2021-02-10T11:39:09.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.449 7fb4f4ff9700 1 -- 172.21.2.101:0/2284773123 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb4e80054f0 con 0x7fb4f8195650 2021-02-10T11:39:09.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.449 7fb4f4ff9700 1 --2- 172.21.2.101:0/2284773123 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4d80186c0 0x7fb4d801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:09.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.449 7fb4f4ff9700 1 -- 172.21.2.101:0/2284773123 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb4e8032540 con 0x7fb4f8195650 2021-02-10T11:39:09.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.449 7fb4f77fe700 1 --2- 172.21.2.101:0/2284773123 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4d80186c0 0x7fb4d801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:09.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.449 7fb4f77fe700 1 --2- 172.21.2.101:0/2284773123 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4d80186c0 0x7fb4d801abc0 crc :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:09.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.480 7fb4fe5f7700 1 -- 172.21.2.101:0/2284773123 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f80b6010 con 0x7fb4f8195650 2021-02-10T11:39:09.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.482 7fb4f4ff9700 1 -- 172.21.2.101:0/2284773123 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb4e8009340 con 0x7fb4f8195650 2021-02-10T11:39:09.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.608 7fb4fe5f7700 1 -- 172.21.2.101:0/2284773123 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4f80b6010 con 0x7fb4f8195650 2021-02-10T11:39:09.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.609 7fb4f4ff9700 1 -- 172.21.2.101:0/2284773123 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+13996 (secure 0 0 0) 0x7fb4e80054b0 con 0x7fb4f8195650 2021-02-10T11:39:09.612 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:39:09.612 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":94,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:39:03.065120","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:39:00.027476","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2021-02-10 11:39:01.054115","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:39:09.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.611 7fb4fe5f7700 1 -- 172.21.2.101:0/2284773123 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4d80186c0 msgr2=0x7fb4d801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:09.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.611 7fb4fe5f7700 1 --2- 172.21.2.101:0/2284773123 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4d80186c0 0x7fb4d801abc0 crc :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:09.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.611 7fb4fe5f7700 1 -- 172.21.2.101:0/2284773123 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4f8195650 msgr2=0x7fb4f80b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:09.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.611 7fb4fe5f7700 1 --2- 172.21.2.101:0/2284773123 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4f8195650 0x7fb4f80b5560 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 rx=0x7fb4e8009630 tx=0x7fb4e800b980).stop 2021-02-10T11:39:09.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.611 7fb4fe5f7700 1 -- 172.21.2.101:0/2284773123 shutdown_connections 2021-02-10T11:39:09.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.611 7fb4fe5f7700 1 --2- 172.21.2.101:0/2284773123 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4d80186c0 0x7fb4d801abc0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:09.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.611 7fb4fe5f7700 1 --2- 172.21.2.101:0/2284773123 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4f80acfa0 0x7fb4f81a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:09.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.611 7fb4fe5f7700 1 --2- 172.21.2.101:0/2284773123 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4f80b5b80 0x7fb4f81a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:09.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.611 7fb4fe5f7700 1 --2- 172.21.2.101:0/2284773123 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4f8195650 0x7fb4f80b5560 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:09.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.611 7fb4fe5f7700 1 -- 172.21.2.101:0/2284773123 >> 172.21.2.101:0/2284773123 conn(0x7fb4f80b3040 msgr2=0x7fb4f80c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:09.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.611 7fb4fe5f7700 1 -- 172.21.2.101:0/2284773123 shutdown_connections 2021-02-10T11:39:09.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:09.611 7fb4fe5f7700 1 -- 172.21.2.101:0/2284773123 wait complete. 2021-02-10T11:39:09.622 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:39:00.027476', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:39:01.054115', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:39:09.622 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:39:10.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.033 7f0a310e0700 1 Processor -- start 2021-02-10T11:39:10.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.033 7f0a310e0700 1 -- start start 2021-02-10T11:39:10.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.034 7f0a310e0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0a2c0d1870 0x7f0a2c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:10.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.034 7f0a310e0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a2c1b0ae0 0x7f0a2c108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:10.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.034 7f0a310e0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a2c0fe1c0 con 0x7f0a2c0d1c70 2021-02-10T11:39:10.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.034 7f0a310e0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a2c109a60 con 0x7f0a2c0d1870 2021-02-10T11:39:10.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.034 7f0a310e0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2c109750 con 0x7f0a2c1b0ae0 2021-02-10T11:39:10.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.034 7f0a2ad9d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0a2c0d1870 0x7f0a2c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:10.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.034 7f0a2ad9d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0a2c0d1870 0x7f0a2c1b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55360/0 (socket says 172.21.2.101:55360) 2021-02-10T11:39:10.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.034 7f0a2ad9d700 1 -- 172.21.2.101:0/433256717 learned_addr learned my addr 172.21.2.101:0/433256717 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:10.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.034 7f0a2ad9d700 1 -- 172.21.2.101:0/433256717 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a2c1b0ae0 msgr2=0x7f0a2c108940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:39:10.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.034 7f0a2ad9d700 1 --2- 172.21.2.101:0/433256717 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a2c1b0ae0 0x7f0a2c108940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:10.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.034 7f0a2ad9d700 1 -- 172.21.2.101:0/433256717 >> v1:172.21.2.108:6789/0 conn(0x7f0a2c0d1c70 legacy=0x7f0a2c108f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:39:10.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.034 7f0a2ad9d700 1 -- 172.21.2.101:0/433256717 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a2c088070 con 0x7f0a2c0d1870 2021-02-10T11:39:10.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.035 7f0a2ad9d700 1 --2- 172.21.2.101:0/433256717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a2c0d1870 0x7f0a2c1b04c0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 rx=0x7f0a14006700 tx=0x7f0a14009b80).ready entity=mon.0 client_cookie=6ad0e05e4332aafb server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:10.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.035 7f0a29d9b700 1 -- 172.21.2.101:0/433256717 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0a1400c0c0 con 0x7f0a2c0d1870 2021-02-10T11:39:10.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.035 7f0a29d9b700 1 -- 172.21.2.101:0/433256717 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1400c340 con 0x7f0a2c0d1870 2021-02-10T11:39:10.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.035 7f0a29d9b700 1 -- 172.21.2.101:0/433256717 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0a1400c860 con 0x7f0a2c0d1870 2021-02-10T11:39:10.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.035 7f0a310e0700 1 -- 172.21.2.101:0/433256717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a2c0d1870 msgr2=0x7f0a2c1b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:10.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.035 7f0a310e0700 1 --2- 172.21.2.101:0/433256717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a2c0d1870 0x7f0a2c1b04c0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 rx=0x7f0a14006700 tx=0x7f0a14009b80).stop 2021-02-10T11:39:10.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.036 7f0a310e0700 1 -- 172.21.2.101:0/433256717 shutdown_connections 2021-02-10T11:39:10.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.036 7f0a310e0700 1 --2- 172.21.2.101:0/433256717 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a2c1b0ae0 0x7f0a2c108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:10.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.036 7f0a310e0700 1 --2- 172.21.2.101:0/433256717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a2c0d1870 0x7f0a2c1b04c0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:10.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.036 7f0a310e0700 1 -- 172.21.2.101:0/433256717 >> 172.21.2.101:0/433256717 conn(0x7f0a2c0ce420 msgr2=0x7f0a2c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:10.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.036 7f0a310e0700 1 -- 172.21.2.101:0/433256717 shutdown_connections 2021-02-10T11:39:10.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.036 7f0a310e0700 1 -- 172.21.2.101:0/433256717 wait complete. 2021-02-10T11:39:10.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.050 7f0a310e0700 1 Processor -- start 2021-02-10T11:39:10.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.050 7f0a310e0700 1 -- start start 2021-02-10T11:39:10.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.051 7f0a310e0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a2c0c3820 0x7f0a2c084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:10.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.051 7f0a310e0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a2c0a0290 0x7f0a2c0852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:10.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.051 7f0a310e0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0a2c086270 0x7f0a2c086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:10.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.051 7f0a310e0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2c0c0700 con 0x7f0a2c0a0290 2021-02-10T11:39:10.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.051 7f0a310e0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2c0858f0 con 0x7f0a2c086270 2021-02-10T11:39:10.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.051 7f0a310e0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2c085b10 con 0x7f0a2c0c3820 2021-02-10T11:39:10.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.051 7f0a2a59c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a2c0a0290 0x7f0a2c0852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:10.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.051 7f0a2a59c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a2c0a0290 0x7f0a2c0852d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55370/0 (socket says 172.21.2.101:55370) 2021-02-10T11:39:10.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.051 7f0a2a59c700 1 -- 172.21.2.101:0/602115610 learned_addr learned my addr 172.21.2.101:0/602115610 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:10.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.051 7f0a2a59c700 1 -- 172.21.2.101:0/602115610 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a2c0c3820 msgr2=0x7f0a2c084cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:10.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.051 7f0a2a59c700 1 --2- 172.21.2.101:0/602115610 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a2c0c3820 0x7f0a2c084cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:10.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.051 7f0a2a59c700 1 -- 172.21.2.101:0/602115610 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0a2c086270 msgr2=0x7f0a2c086670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:10.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.051 7f0a2a59c700 1 --2- 172.21.2.101:0/602115610 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0a2c086270 0x7f0a2c086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:10.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.051 7f0a2a59c700 1 -- 172.21.2.101:0/602115610 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a2c086c90 con 0x7f0a2c0a0290 2021-02-10T11:39:10.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.051 7f0a2a59c700 1 --2- 172.21.2.101:0/602115610 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a2c0a0290 0x7f0a2c0852d0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 rx=0x7f0a1c00b5c0 tx=0x7f0a1c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:10.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.052 7f0a237fe700 1 -- 172.21.2.101:0/602115610 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0a1c00bd50 con 0x7f0a2c0a0290 2021-02-10T11:39:10.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.052 7f0a237fe700 1 -- 172.21.2.101:0/602115610 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1c006030 con 0x7f0a2c0a0290 2021-02-10T11:39:10.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.052 7f0a237fe700 1 -- 172.21.2.101:0/602115610 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0a1c0054d0 con 0x7f0a2c0a0290 2021-02-10T11:39:10.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.052 7f0a310e0700 1 -- 172.21.2.101:0/602115610 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a2c086c90 con 0x7f0a2c0a0290 2021-02-10T11:39:10.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.052 7f0a310e0700 1 -- 172.21.2.101:0/602115610 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a2c086c90 con 0x7f0a2c0a0290 2021-02-10T11:39:10.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.052 7f0a237fe700 1 -- 172.21.2.101:0/602115610 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0a1c006210 con 0x7f0a2c0a0290 2021-02-10T11:39:10.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.052 7f0a237fe700 1 --2- 172.21.2.101:0/602115610 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a100186c0 0x7f0a1001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:10.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.052 7f0a237fe700 1 -- 172.21.2.101:0/602115610 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0a1c0325a0 con 0x7f0a2c0a0290 2021-02-10T11:39:10.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.053 7f0a2ad9d700 1 --2- 172.21.2.101:0/602115610 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a100186c0 0x7f0a1001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:10.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.053 7f0a2ad9d700 1 --2- 172.21.2.101:0/602115610 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a100186c0 0x7f0a1001abc0 crc :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:10.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.084 7f0a310e0700 1 -- 172.21.2.101:0/602115610 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a2c082a70 con 0x7f0a2c0a0290 2021-02-10T11:39:10.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.086 7f0a237fe700 1 -- 172.21.2.101:0/602115610 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0a1c006b80 con 0x7f0a2c0a0290 2021-02-10T11:39:10.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.215 7f0a310e0700 1 -- 172.21.2.101:0/602115610 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f0a2c082a70 con 0x7f0a2c0a0290 2021-02-10T11:39:10.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.216 7f0a237fe700 1 -- 172.21.2.101:0/602115610 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v94) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0a1c00bd50 con 0x7f0a2c0a0290 2021-02-10T11:39:10.219 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:39:10.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.218 7f0a310e0700 1 -- 172.21.2.101:0/602115610 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a100186c0 msgr2=0x7f0a1001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:10.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.218 7f0a310e0700 1 --2- 172.21.2.101:0/602115610 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a100186c0 0x7f0a1001abc0 crc :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:10.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.218 7f0a310e0700 1 -- 172.21.2.101:0/602115610 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a2c0a0290 msgr2=0x7f0a2c0852d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:10.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.218 7f0a310e0700 1 --2- 172.21.2.101:0/602115610 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a2c0a0290 0x7f0a2c0852d0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 rx=0x7f0a1c00b5c0 tx=0x7f0a1c00b960).stop 2021-02-10T11:39:10.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.218 7f0a310e0700 1 -- 172.21.2.101:0/602115610 shutdown_connections 2021-02-10T11:39:10.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.218 7f0a310e0700 1 --2- 172.21.2.101:0/602115610 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0a2c086270 0x7f0a2c086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:10.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.218 7f0a310e0700 1 --2- 172.21.2.101:0/602115610 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a2c0a0290 0x7f0a2c0852d0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:10.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.218 7f0a310e0700 1 --2- 172.21.2.101:0/602115610 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a100186c0 0x7f0a1001abc0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:10.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.218 7f0a310e0700 1 --2- 172.21.2.101:0/602115610 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a2c0c3820 0x7f0a2c084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:10.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.218 7f0a310e0700 1 -- 172.21.2.101:0/602115610 >> 172.21.2.101:0/602115610 conn(0x7f0a2c0872c0 msgr2=0x7f0a2c095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:10.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.218 7f0a310e0700 1 -- 172.21.2.101:0/602115610 shutdown_connections 2021-02-10T11:39:10.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.218 7f0a310e0700 1 -- 172.21.2.101:0/602115610 wait complete. 2021-02-10T11:39:10.229 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:39:10.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.641 7f16a6ca1700 1 Processor -- start 2021-02-10T11:39:10.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.641 7f16a6ca1700 1 -- start start 2021-02-10T11:39:10.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.641 7f16a6ca1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16a00d1c70 0x7f16a00cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:10.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.641 7f16a6ca1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f16a00cf580 0x7f16a00cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:10.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.641 7f16a6ca1700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a00fe1c0 con 0x7f16a00d1870 2021-02-10T11:39:10.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.641 7f16a6ca1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a01b0cb0 con 0x7f16a00cf580 2021-02-10T11:39:10.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.641 7f16a6ca1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a0109a60 con 0x7f16a00d1c70 2021-02-10T11:39:10.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.642 7f16a4abd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f16a00d1870 0x7f16a01b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55868/0 (socket says 172.21.2.101:55868) 2021-02-10T11:39:10.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.642 7f16a4abd700 1 -- 172.21.2.101:0/1449924137 learned_addr learned my addr 172.21.2.101:0/1449924137 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:10.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.642 7f169f7fe700 1 -- 172.21.2.101:0/1449924137 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3485091444 0 0) 0x7f16880009b0 con 0x7f16a00d1870 2021-02-10T11:39:10.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.642 7f169f7fe700 1 -- 172.21.2.101:0/1449924137 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f168c002fe0 con 0x7f16a00d1870 2021-02-10T11:39:10.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.642 7f169ffff700 1 --2- 172.21.2.101:0/1449924137 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16a00d1c70 0x7f16a00cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:10.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.642 7f16a52be700 1 --2- 172.21.2.101:0/1449924137 >> v2:172.21.2.108:3300/0 conn(0x7f16a00cf580 0x7f16a00cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:10.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.642 7f169f7fe700 1 -- 172.21.2.101:0/1449924137 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3612840651 0 0) 0x7f1688001010 con 0x7f16a00d1870 2021-02-10T11:39:10.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.642 7f169f7fe700 1 -- 172.21.2.101:0/1449924137 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16a00d1c70 msgr2=0x7f16a00cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:10.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.642 7f169f7fe700 1 --2- 172.21.2.101:0/1449924137 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16a00d1c70 0x7f16a00cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:10.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.642 7f169f7fe700 1 -- 172.21.2.101:0/1449924137 >> v2:172.21.2.108:3300/0 conn(0x7f16a00cf580 msgr2=0x7f16a00cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:10.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.642 7f169f7fe700 1 --2- 172.21.2.101:0/1449924137 >> v2:172.21.2.108:3300/0 conn(0x7f16a00cf580 0x7f16a00cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:10.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.642 7f169f7fe700 1 -- 172.21.2.101:0/1449924137 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a0088070 con 0x7f16a00d1870 2021-02-10T11:39:10.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.642 7f169f7fe700 1 -- 172.21.2.101:0/1449924137 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f1688001530 con 0x7f16a00d1870 2021-02-10T11:39:10.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.642 7f169f7fe700 1 -- 172.21.2.101:0/1449924137 >> v1:172.21.2.101:6789/0 conn(0x7f16a00d1870 legacy=0x7f16a01b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:10.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.642 7f169f7fe700 1 --2- 172.21.2.101:0/1449924137 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f168c003b90 0x7f168c0060b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:10.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.642 7f169f7fe700 1 --2- 172.21.2.101:0/1449924137 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f168c006950 0x7f168c008e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:10.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.642 7f169f7fe700 1 --2- 172.21.2.101:0/1449924137 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f168c0096e0 0x7f168c00bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:10.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.643 7f169f7fe700 1 -- 172.21.2.101:0/1449924137 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168c0504c0 con 0x7f168c0096e0 2021-02-10T11:39:10.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.643 7f169f7fe700 1 -- 172.21.2.101:0/1449924137 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168c050670 con 0x7f168c003b90 2021-02-10T11:39:10.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.643 7f169f7fe700 1 -- 172.21.2.101:0/1449924137 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168c050850 con 0x7f168c006950 2021-02-10T11:39:10.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.643 7f169ffff700 1 --2- 172.21.2.101:0/1449924137 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16a00d1c70 0x7f16a00cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:39:10.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.643 7f16a52be700 1 --2- 172.21.2.101:0/1449924137 >> v2:172.21.2.108:3300/0 conn(0x7f16a00cf580 0x7f16a00cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:39:10.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.643 7f16a52be700 1 --2- 172.21.2.101:0/1449924137 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f168c0096e0 0x7f168c00bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:10.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.643 7f169ffff700 1 --2- 172.21.2.101:0/1449924137 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f168c006950 0x7f168c008e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:10.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.643 7f16a4abd700 1 --2- 172.21.2.101:0/1449924137 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f168c003b90 0x7f168c0060b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:10.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.643 7f16a52be700 1 -- 172.21.2.101:0/1449924137 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f168c006950 msgr2=0x7f168c008e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:10.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.643 7f16a52be700 1 --2- 172.21.2.101:0/1449924137 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f168c006950 0x7f168c008e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:10.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.643 7f16a52be700 1 -- 172.21.2.101:0/1449924137 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f168c003b90 msgr2=0x7f168c0060b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:10.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.643 7f16a52be700 1 --2- 172.21.2.101:0/1449924137 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f168c003b90 0x7f168c0060b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:10.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.643 7f16a52be700 1 -- 172.21.2.101:0/1449924137 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168c050af0 con 0x7f168c0096e0 2021-02-10T11:39:10.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.643 7f16a4abd700 1 --2- 172.21.2.101:0/1449924137 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f168c003b90 0x7f168c0060b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:39:10.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.643 7f16a52be700 1 --2- 172.21.2.101:0/1449924137 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f168c0096e0 0x7f168c00bc00 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 rx=0x7f1698006530 tx=0x7f1698009d30).ready entity=mon.0 client_cookie=3f9d25ec4dd5c06a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:10.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.644 7f169f7fe700 1 -- 172.21.2.101:0/1449924137 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1698000b90 con 0x7f168c0096e0 2021-02-10T11:39:10.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.644 7f169f7fe700 1 -- 172.21.2.101:0/1449924137 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169800b300 con 0x7f168c0096e0 2021-02-10T11:39:10.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.644 7f169f7fe700 1 -- 172.21.2.101:0/1449924137 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f169800b820 con 0x7f168c0096e0 2021-02-10T11:39:10.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.644 7f16a6ca1700 1 -- 172.21.2.101:0/1449924137 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f168c0096e0 msgr2=0x7f168c00bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:10.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.644 7f16a6ca1700 1 --2- 172.21.2.101:0/1449924137 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f168c0096e0 0x7f168c00bc00 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 rx=0x7f1698006530 tx=0x7f1698009d30).stop 2021-02-10T11:39:10.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.644 7f16a52be700 1 -- 172.21.2.101:0/1449924137 reap_dead start 2021-02-10T11:39:10.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.644 7f16a6ca1700 1 -- 172.21.2.101:0/1449924137 shutdown_connections 2021-02-10T11:39:10.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.644 7f16a6ca1700 1 -- 172.21.2.101:0/1449924137 >> 172.21.2.101:0/1449924137 conn(0x7f16a00ce420 msgr2=0x7f16a00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:10.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.644 7f16a6ca1700 1 -- 172.21.2.101:0/1449924137 shutdown_connections 2021-02-10T11:39:10.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.644 7f16a6ca1700 1 -- 172.21.2.101:0/1449924137 wait complete. 2021-02-10T11:39:10.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.659 7f16a6ca1700 1 Processor -- start 2021-02-10T11:39:10.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.659 7f16a6ca1700 1 -- start start 2021-02-10T11:39:10.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.660 7f16a6ca1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16a0085bc0 0x7f16a00863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:10.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.660 7f16a6ca1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16a00954c0 0x7f16a0096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:10.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.660 7f16a6ca1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f16a00958c0 0x7f16a0096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:10.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.660 7f16a6ca1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a0095d80 con 0x7f16a0085bc0 2021-02-10T11:39:10.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.660 7f16a6ca1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a00bfa60 con 0x7f16a00958c0 2021-02-10T11:39:10.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.660 7f16a6ca1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a0085fc0 con 0x7f16a00954c0 2021-02-10T11:39:10.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.660 7f16a4abd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16a0085bc0 0x7f16a00863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:10.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.660 7f16a4abd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16a0085bc0 0x7f16a00863e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55386/0 (socket says 172.21.2.101:55386) 2021-02-10T11:39:10.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.660 7f16a4abd700 1 -- 172.21.2.101:0/3715383258 learned_addr learned my addr 172.21.2.101:0/3715383258 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:10.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.660 7f16a4abd700 1 -- 172.21.2.101:0/3715383258 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16a00954c0 msgr2=0x7f16a0096300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:10.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.660 7f16a4abd700 1 --2- 172.21.2.101:0/3715383258 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16a00954c0 0x7f16a0096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:10.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.660 7f16a4abd700 1 -- 172.21.2.101:0/3715383258 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f16a00958c0 msgr2=0x7f16a0096920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:10.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.660 7f16a4abd700 1 --2- 172.21.2.101:0/3715383258 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f16a00958c0 0x7f16a0096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:10.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.660 7f16a4abd700 1 -- 172.21.2.101:0/3715383258 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a0086c10 con 0x7f16a0085bc0 2021-02-10T11:39:10.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.660 7f16a4abd700 1 --2- 172.21.2.101:0/3715383258 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16a0085bc0 0x7f16a00863e0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 rx=0x7f1688001c10 tx=0x7f168800a8f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:10.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.661 7f169d7fa700 1 -- 172.21.2.101:0/3715383258 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f16880012c0 con 0x7f16a0085bc0 2021-02-10T11:39:10.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.661 7f169d7fa700 1 -- 172.21.2.101:0/3715383258 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168800ace0 con 0x7f16a0085bc0 2021-02-10T11:39:10.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.661 7f169d7fa700 1 -- 172.21.2.101:0/3715383258 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f16880045d0 con 0x7f16a0085bc0 2021-02-10T11:39:10.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.661 7f16a6ca1700 1 -- 172.21.2.101:0/3715383258 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a0086c10 con 0x7f16a0085bc0 2021-02-10T11:39:10.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.661 7f16a6ca1700 1 -- 172.21.2.101:0/3715383258 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a0086c10 con 0x7f16a0085bc0 2021-02-10T11:39:10.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.661 7f169d7fa700 1 -- 172.21.2.101:0/3715383258 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1688004140 con 0x7f16a0085bc0 2021-02-10T11:39:10.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.662 7f169d7fa700 1 --2- 172.21.2.101:0/3715383258 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16840186c0 0x7f168401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:10.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.662 7f169d7fa700 1 -- 172.21.2.101:0/3715383258 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1688032550 con 0x7f16a0085bc0 2021-02-10T11:39:10.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.662 7f169ffff700 1 --2- 172.21.2.101:0/3715383258 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16840186c0 0x7f168401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:10.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.662 7f169ffff700 1 --2- 172.21.2.101:0/3715383258 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16840186c0 0x7f168401abc0 crc :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:10.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.691 7f16a6ca1700 1 -- 172.21.2.101:0/3715383258 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a00fe1c0 con 0x7f16a0085bc0 2021-02-10T11:39:10.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.694 7f169d7fa700 1 -- 172.21.2.101:0/3715383258 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1688000c90 con 0x7f16a0085bc0 2021-02-10T11:39:10.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.821 7f16a6ca1700 1 -- 172.21.2.101:0/3715383258 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f16a00fe1c0 con 0x7f16a0085bc0 2021-02-10T11:39:10.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.822 7f169d7fa700 1 -- 172.21.2.101:0/3715383258 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v94) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1688004140 con 0x7f16a0085bc0 2021-02-10T11:39:10.825 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:39:10.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.824 7f16a6ca1700 1 -- 172.21.2.101:0/3715383258 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16840186c0 msgr2=0x7f168401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:10.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.824 7f16a6ca1700 1 --2- 172.21.2.101:0/3715383258 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16840186c0 0x7f168401abc0 crc :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:10.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.824 7f16a6ca1700 1 -- 172.21.2.101:0/3715383258 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16a0085bc0 msgr2=0x7f16a00863e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:10.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.824 7f16a6ca1700 1 --2- 172.21.2.101:0/3715383258 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16a0085bc0 0x7f16a00863e0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 rx=0x7f1688001c10 tx=0x7f168800a8f0).stop 2021-02-10T11:39:10.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.824 7f16a6ca1700 1 -- 172.21.2.101:0/3715383258 shutdown_connections 2021-02-10T11:39:10.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.824 7f16a6ca1700 1 --2- 172.21.2.101:0/3715383258 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16840186c0 0x7f168401abc0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:10.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.824 7f16a6ca1700 1 --2- 172.21.2.101:0/3715383258 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16a00954c0 0x7f16a0096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:10.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.824 7f16a6ca1700 1 --2- 172.21.2.101:0/3715383258 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f16a00958c0 0x7f16a0096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:10.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.824 7f16a6ca1700 1 --2- 172.21.2.101:0/3715383258 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16a0085bc0 0x7f16a00863e0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:10.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.824 7f16a6ca1700 1 -- 172.21.2.101:0/3715383258 >> 172.21.2.101:0/3715383258 conn(0x7f16a00bef00 msgr2=0x7f16a009d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:10.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.824 7f16a6ca1700 1 -- 172.21.2.101:0/3715383258 shutdown_connections 2021-02-10T11:39:10.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:10.824 7f16a6ca1700 1 -- 172.21.2.101:0/3715383258 wait complete. 2021-02-10T11:39:10.835 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:39:11.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.245 7f969d98d700 1 Processor -- start 2021-02-10T11:39:11.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.245 7f969d98d700 1 -- start start 2021-02-10T11:39:11.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.245 7f969d98d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96980d1870 0x7f96981b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:11.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.245 7f969d98d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9698109a60 con 0x7f96980cff70 2021-02-10T11:39:11.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.245 7f969d98d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9698088070 con 0x7f96981b0ae0 2021-02-10T11:39:11.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.245 7f969d98d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96980e5ef0 con 0x7f96980d1870 2021-02-10T11:39:11.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.245 7f9697d75700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f96980cff70 0x7f9698109130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55886/0 (socket says 172.21.2.101:55886) 2021-02-10T11:39:11.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.245 7f9697d75700 1 -- 172.21.2.101:0/395646260 learned_addr learned my addr 172.21.2.101:0/395646260 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:11.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.246 7f9696572700 1 -- 172.21.2.101:0/395646260 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094062559 0 0) 0x7f9690001500 con 0x7f96980cff70 2021-02-10T11:39:11.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.246 7f9696572700 1 -- 172.21.2.101:0/395646260 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9684002fe0 con 0x7f96980cff70 2021-02-10T11:39:11.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.246 7f9697574700 1 --2- 172.21.2.101:0/395646260 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96980d1870 0x7f96981b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:11.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.246 7f9696572700 1 -- 172.21.2.101:0/395646260 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1862621540 0 0) 0x7f9690001500 con 0x7f96980cff70 2021-02-10T11:39:11.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.246 7f9696572700 1 -- 172.21.2.101:0/395646260 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96980d1870 msgr2=0x7f96981b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:11.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.246 7f9696572700 1 --2- 172.21.2.101:0/395646260 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96980d1870 0x7f96981b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:11.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.246 7f9696572700 1 -- 172.21.2.101:0/395646260 >> v1:172.21.2.108:6789/0 conn(0x7f96981b0ae0 legacy=0x7f9698108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:11.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.246 7f9696572700 1 -- 172.21.2.101:0/395646260 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96980d1c70 con 0x7f96980cff70 2021-02-10T11:39:11.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.246 7f9696572700 1 -- 172.21.2.101:0/395646260 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f9690001a00 con 0x7f96980cff70 2021-02-10T11:39:11.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.246 7f9696572700 1 -- 172.21.2.101:0/395646260 >> v1:172.21.2.101:6789/0 conn(0x7f96980cff70 legacy=0x7f9698109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:11.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.246 7f9696572700 1 --2- 172.21.2.101:0/395646260 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9684003a70 0x7f9684005f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:11.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.246 7f9696572700 1 --2- 172.21.2.101:0/395646260 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9684006780 0x7f9684008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:11.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.246 7f9696572700 1 --2- 172.21.2.101:0/395646260 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9684009510 0x7f968400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:11.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.246 7f9696572700 1 -- 172.21.2.101:0/395646260 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968400c2e0 con 0x7f9684003a70 2021-02-10T11:39:11.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.246 7f9696572700 1 -- 172.21.2.101:0/395646260 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968400c500 con 0x7f9684009510 2021-02-10T11:39:11.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.246 7f9696572700 1 -- 172.21.2.101:0/395646260 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9684050730 con 0x7f9684006780 2021-02-10T11:39:11.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.246 7f9697574700 1 --2- 172.21.2.101:0/395646260 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9684003a70 0x7f9684005f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:11.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.246 7f9697d75700 1 --2- 172.21.2.101:0/395646260 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9684009510 0x7f968400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:11.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.246 7f9696d73700 1 --2- 172.21.2.101:0/395646260 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9684006780 0x7f9684008c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:11.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.247 7f9697574700 1 -- 172.21.2.101:0/395646260 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9684006780 msgr2=0x7f9684008c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:11.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.247 7f9697574700 1 --2- 172.21.2.101:0/395646260 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9684006780 0x7f9684008c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:11.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.247 7f9697574700 1 -- 172.21.2.101:0/395646260 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9684009510 msgr2=0x7f968400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:11.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.247 7f9697574700 1 --2- 172.21.2.101:0/395646260 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9684009510 0x7f968400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:11.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.247 7f9697574700 1 -- 172.21.2.101:0/395646260 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9684050990 con 0x7f9684003a70 2021-02-10T11:39:11.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.247 7f9697d75700 1 --2- 172.21.2.101:0/395646260 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9684009510 0x7f968400ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:39:11.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.247 7f9697574700 1 --2- 172.21.2.101:0/395646260 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9684003a70 0x7f9684005f00 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 rx=0x7f9688009310 tx=0x7f9688009a90).ready entity=mon.0 client_cookie=c885f7a084741412 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:11.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.247 7f9696572700 1 -- 172.21.2.101:0/395646260 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f968800c0c0 con 0x7f9684003a70 2021-02-10T11:39:11.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.247 7f9696572700 1 -- 172.21.2.101:0/395646260 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9688006400 con 0x7f9684003a70 2021-02-10T11:39:11.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.247 7f9696572700 1 -- 172.21.2.101:0/395646260 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f968800c340 con 0x7f9684003a70 2021-02-10T11:39:11.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.247 7f969d98d700 1 -- 172.21.2.101:0/395646260 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9684003a70 msgr2=0x7f9684005f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:11.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.247 7f969d98d700 1 --2- 172.21.2.101:0/395646260 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9684003a70 0x7f9684005f00 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 rx=0x7f9688009310 tx=0x7f9688009a90).stop 2021-02-10T11:39:11.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.247 7f9697d75700 1 -- 172.21.2.101:0/395646260 reap_dead start 2021-02-10T11:39:11.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.247 7f969d98d700 1 -- 172.21.2.101:0/395646260 shutdown_connections 2021-02-10T11:39:11.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.247 7f969d98d700 1 -- 172.21.2.101:0/395646260 >> 172.21.2.101:0/395646260 conn(0x7f96980ce420 msgr2=0x7f96980d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:11.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.248 7f969d98d700 1 -- 172.21.2.101:0/395646260 shutdown_connections 2021-02-10T11:39:11.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.248 7f969d98d700 1 -- 172.21.2.101:0/395646260 wait complete. 2021-02-10T11:39:11.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.262 7f969d98d700 1 Processor -- start 2021-02-10T11:39:11.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.262 7f969d98d700 1 -- start start 2021-02-10T11:39:11.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.263 7f969d98d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9698085bc0 0x7f96980863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:11.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.263 7f969d98d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96980954c0 0x7f9698096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:11.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.263 7f969d98d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96980958c0 0x7f9698096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:11.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.263 7f969d98d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9698095d80 con 0x7f96980954c0 2021-02-10T11:39:11.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.263 7f969d98d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96980bfa60 con 0x7f96980958c0 2021-02-10T11:39:11.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.263 7f969d98d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9698085fc0 con 0x7f9698085bc0 2021-02-10T11:39:11.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.263 7f9696d73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96980954c0 0x7f9698096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:11.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.263 7f9696d73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96980954c0 0x7f9698096300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55408/0 (socket says 172.21.2.101:55408) 2021-02-10T11:39:11.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.263 7f9696d73700 1 -- 172.21.2.101:0/2066119761 learned_addr learned my addr 172.21.2.101:0/2066119761 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:11.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.263 7f9697d75700 1 --2- 172.21.2.101:0/2066119761 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96980958c0 0x7f9698096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:11.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.263 7f9697574700 1 --2- 172.21.2.101:0/2066119761 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9698085bc0 0x7f96980863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:11.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.265 7f9696d73700 1 -- 172.21.2.101:0/2066119761 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9698085bc0 msgr2=0x7f96980863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:11.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.265 7f9696d73700 1 --2- 172.21.2.101:0/2066119761 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9698085bc0 0x7f96980863e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:11.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.265 7f9696d73700 1 -- 172.21.2.101:0/2066119761 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96980958c0 msgr2=0x7f9698096920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:11.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.265 7f9696d73700 1 --2- 172.21.2.101:0/2066119761 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96980958c0 0x7f9698096920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:11.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.265 7f9696d73700 1 -- 172.21.2.101:0/2066119761 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9698086c10 con 0x7f96980954c0 2021-02-10T11:39:11.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.265 7f9697574700 1 --2- 172.21.2.101:0/2066119761 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9698085bc0 0x7f96980863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:39:11.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.265 7f9696d73700 1 --2- 172.21.2.101:0/2066119761 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96980954c0 0x7f9698096300 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 rx=0x7f9680006f10 tx=0x7f968000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:11.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.265 7f968ffff700 1 -- 172.21.2.101:0/2066119761 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f968000bd80 con 0x7f96980954c0 2021-02-10T11:39:11.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.265 7f968ffff700 1 -- 172.21.2.101:0/2066119761 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9680000ce0 con 0x7f96980954c0 2021-02-10T11:39:11.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.265 7f968ffff700 1 -- 172.21.2.101:0/2066119761 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9680002220 con 0x7f96980954c0 2021-02-10T11:39:11.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.265 7f969d98d700 1 -- 172.21.2.101:0/2066119761 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9698086c10 con 0x7f96980954c0 2021-02-10T11:39:11.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.265 7f969d98d700 1 -- 172.21.2.101:0/2066119761 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9698097230 con 0x7f96980954c0 2021-02-10T11:39:11.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.266 7f968ffff700 1 -- 172.21.2.101:0/2066119761 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9680009fe0 con 0x7f96980954c0 2021-02-10T11:39:11.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.266 7f968ffff700 1 --2- 172.21.2.101:0/2066119761 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f967c0186c0 0x7f967c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:11.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.266 7f968ffff700 1 -- 172.21.2.101:0/2066119761 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9680032510 con 0x7f96980954c0 2021-02-10T11:39:11.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.266 7f9697574700 1 --2- 172.21.2.101:0/2066119761 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f967c0186c0 0x7f967c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:11.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.266 7f9697574700 1 --2- 172.21.2.101:0/2066119761 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f967c0186c0 0x7f967c01abc0 crc :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:11.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.295 7f969d98d700 1 -- 172.21.2.101:0/2066119761 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96980fe1c0 con 0x7f96980954c0 2021-02-10T11:39:11.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.299 7f968ffff700 1 -- 172.21.2.101:0/2066119761 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9680000940 con 0x7f96980954c0 2021-02-10T11:39:11.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.424 7f969d98d700 1 -- 172.21.2.101:0/2066119761 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f96980fe1c0 con 0x7f96980954c0 2021-02-10T11:39:11.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.425 7f968ffff700 1 -- 172.21.2.101:0/2066119761 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v94) v1 ==== 100+0+10 (secure 0 0 0) 0x7f968000bd80 con 0x7f96980954c0 2021-02-10T11:39:11.429 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:39:11.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.427 7f969d98d700 1 -- 172.21.2.101:0/2066119761 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f967c0186c0 msgr2=0x7f967c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:11.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.427 7f969d98d700 1 --2- 172.21.2.101:0/2066119761 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f967c0186c0 0x7f967c01abc0 crc :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:11.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.427 7f969d98d700 1 -- 172.21.2.101:0/2066119761 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96980954c0 msgr2=0x7f9698096300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:11.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.427 7f969d98d700 1 --2- 172.21.2.101:0/2066119761 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96980954c0 0x7f9698096300 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 rx=0x7f9680006f10 tx=0x7f968000b990).stop 2021-02-10T11:39:11.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.427 7f969d98d700 1 -- 172.21.2.101:0/2066119761 shutdown_connections 2021-02-10T11:39:11.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.427 7f969d98d700 1 --2- 172.21.2.101:0/2066119761 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96980958c0 0x7f9698096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:11.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.427 7f969d98d700 1 --2- 172.21.2.101:0/2066119761 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96980954c0 0x7f9698096300 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:11.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.427 7f969d98d700 1 --2- 172.21.2.101:0/2066119761 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f967c0186c0 0x7f967c01abc0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:11.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.427 7f969d98d700 1 --2- 172.21.2.101:0/2066119761 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9698085bc0 0x7f96980863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:11.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.427 7f969d98d700 1 -- 172.21.2.101:0/2066119761 >> 172.21.2.101:0/2066119761 conn(0x7f96980bef00 msgr2=0x7f969809d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:11.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.427 7f969d98d700 1 -- 172.21.2.101:0/2066119761 shutdown_connections 2021-02-10T11:39:11.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.427 7f969d98d700 1 -- 172.21.2.101:0/2066119761 wait complete. 2021-02-10T11:39:11.439 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:39:11.439 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:39:11.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.846 7f9c7fdfc700 1 Processor -- start 2021-02-10T11:39:13.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.846 7f9c7fdfc700 1 -- start start 2021-02-10T11:39:13.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.847 7f9c7fdfc700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f9c780d0340 0x7f9c780ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:13.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.847 7f9c7fdfc700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c780f0b10 con 0x7f9c780d1c70 2021-02-10T11:39:13.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.847 7f9c7fdfc700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c781b0be0 con 0x7f9c780d1870 2021-02-10T11:39:13.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.847 7f9c7fdfc700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c780e5ed0 con 0x7f9c780d0340 2021-02-10T11:39:13.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.847 7f9c7e419700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f9c780d1c70 0x7f9c780efc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55906/0 (socket says 172.21.2.101:55906) 2021-02-10T11:39:13.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.847 7f9c7e419700 1 -- 172.21.2.101:0/407237928 learned_addr learned my addr 172.21.2.101:0/407237928 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:13.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.847 7f9c7cc16700 1 -- 172.21.2.101:0/407237928 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3487178910 0 0) 0x7f9c70001500 con 0x7f9c780d1c70 2021-02-10T11:39:13.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.847 7f9c7cc16700 1 -- 172.21.2.101:0/407237928 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c64002fe0 con 0x7f9c780d1c70 2021-02-10T11:39:13.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7d417700 1 --2- 172.21.2.101:0/407237928 >> v2:172.21.2.108:3300/0 conn(0x7f9c780d0340 0x7f9c780ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:13.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7cc16700 1 -- 172.21.2.101:0/407237928 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 201540768 0 0) 0x7f9c70001500 con 0x7f9c780d1c70 2021-02-10T11:39:13.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7cc16700 1 -- 172.21.2.101:0/407237928 >> v2:172.21.2.108:3300/0 conn(0x7f9c780d0340 msgr2=0x7f9c780ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:13.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7cc16700 1 --2- 172.21.2.101:0/407237928 >> v2:172.21.2.108:3300/0 conn(0x7f9c780d0340 0x7f9c780ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:13.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7cc16700 1 -- 172.21.2.101:0/407237928 >> v1:172.21.2.108:6789/0 conn(0x7f9c780d1870 legacy=0x7f9c781b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:13.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7cc16700 1 -- 172.21.2.101:0/407237928 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c780f0450 con 0x7f9c780d1c70 2021-02-10T11:39:13.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7cc16700 1 -- 172.21.2.101:0/407237928 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f9c70001a00 con 0x7f9c780d1c70 2021-02-10T11:39:13.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7cc16700 1 -- 172.21.2.101:0/407237928 >> v1:172.21.2.101:6789/0 conn(0x7f9c780d1c70 legacy=0x7f9c780efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:13.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7cc16700 1 --2- 172.21.2.101:0/407237928 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9c64003af0 0x7f9c64005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:13.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7cc16700 1 --2- 172.21.2.101:0/407237928 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9c64006840 0x7f9c64008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:13.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7cc16700 1 --2- 172.21.2.101:0/407237928 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c64009640 0x7f9c6400bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:13.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7cc16700 1 -- 172.21.2.101:0/407237928 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6400c3b0 con 0x7f9c64009640 2021-02-10T11:39:13.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7cc16700 1 -- 172.21.2.101:0/407237928 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6400c5d0 con 0x7f9c64003af0 2021-02-10T11:39:13.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7cc16700 1 -- 172.21.2.101:0/407237928 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6400c7f0 con 0x7f9c64006840 2021-02-10T11:39:13.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7d417700 1 --2- 172.21.2.101:0/407237928 >> v2:172.21.2.108:3300/0 conn(0x7f9c780d0340 0x7f9c780ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:39:13.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7e419700 1 --2- 172.21.2.101:0/407237928 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c64009640 0x7f9c6400bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:13.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7d417700 1 --2- 172.21.2.101:0/407237928 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9c64006840 0x7f9c64008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:13.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7dc18700 1 --2- 172.21.2.101:0/407237928 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9c64003af0 0x7f9c64005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:13.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7e419700 1 -- 172.21.2.101:0/407237928 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9c64006840 msgr2=0x7f9c64008d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:13.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7e419700 1 --2- 172.21.2.101:0/407237928 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9c64006840 0x7f9c64008d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:13.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7e419700 1 -- 172.21.2.101:0/407237928 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9c64003af0 msgr2=0x7f9c64005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:13.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7e419700 1 --2- 172.21.2.101:0/407237928 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9c64003af0 0x7f9c64005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:13.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7e419700 1 -- 172.21.2.101:0/407237928 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6400ca90 con 0x7f9c64009640 2021-02-10T11:39:13.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7e419700 1 -- 172.21.2.101:0/407237928 reap_dead start 2021-02-10T11:39:13.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7e419700 1 --2- 172.21.2.101:0/407237928 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c64009640 0x7f9c6400bb60 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 rx=0x7f9c70007fc0 tx=0x7f9c7004e320).ready entity=mon.0 client_cookie=130fc88b962fb007 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:13.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7cc16700 1 -- 172.21.2.101:0/407237928 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9c70007a10 con 0x7f9c64009640 2021-02-10T11:39:13.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7cc16700 1 -- 172.21.2.101:0/407237928 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c700070c0 con 0x7f9c64009640 2021-02-10T11:39:13.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7cc16700 1 -- 172.21.2.101:0/407237928 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9c70007510 con 0x7f9c64009640 2021-02-10T11:39:13.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7fdfc700 1 -- 172.21.2.101:0/407237928 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c64009640 msgr2=0x7f9c6400bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:13.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.848 7f9c7fdfc700 1 --2- 172.21.2.101:0/407237928 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c64009640 0x7f9c6400bb60 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 rx=0x7f9c70007fc0 tx=0x7f9c7004e320).stop 2021-02-10T11:39:13.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.849 7f9c7fdfc700 1 -- 172.21.2.101:0/407237928 shutdown_connections 2021-02-10T11:39:13.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.849 7f9c7fdfc700 1 --2- 172.21.2.101:0/407237928 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c64009640 0x7f9c6400bb60 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:13.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.849 7f9c7fdfc700 1 -- 172.21.2.101:0/407237928 >> 172.21.2.101:0/407237928 conn(0x7f9c780ce450 msgr2=0x7f9c780d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:13.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.849 7f9c7fdfc700 1 -- 172.21.2.101:0/407237928 shutdown_connections 2021-02-10T11:39:13.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.849 7f9c7fdfc700 1 -- 172.21.2.101:0/407237928 wait complete. 2021-02-10T11:39:13.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.864 7f9c7fdfc700 1 Processor -- start 2021-02-10T11:39:13.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.864 7f9c7fdfc700 1 -- start start 2021-02-10T11:39:13.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.864 7f9c7fdfc700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9c781942b0 0x7f9c780b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:13.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.864 7f9c7fdfc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9c78195070 0x7f9c78195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:13.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.864 7f9c7fdfc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c780bb8e0 0x7f9c780bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:13.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.864 7f9c7fdfc700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c781946b0 con 0x7f9c780bb8e0 2021-02-10T11:39:13.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.864 7f9c7fdfc700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c780b3b70 con 0x7f9c781942b0 2021-02-10T11:39:13.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.864 7f9c7fdfc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c780b3d90 con 0x7f9c78195070 2021-02-10T11:39:13.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.864 7f9c7e419700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c780bb8e0 0x7f9c780bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:13.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.864 7f9c7e419700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c780bb8e0 0x7f9c780bbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55422/0 (socket says 172.21.2.101:55422) 2021-02-10T11:39:13.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.864 7f9c7e419700 1 -- 172.21.2.101:0/4238292113 learned_addr learned my addr 172.21.2.101:0/4238292113 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:13.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.865 7f9c7e419700 1 -- 172.21.2.101:0/4238292113 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9c78195070 msgr2=0x7f9c78195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:13.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.865 7f9c7e419700 1 --2- 172.21.2.101:0/4238292113 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9c78195070 0x7f9c78195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:13.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.865 7f9c7e419700 1 -- 172.21.2.101:0/4238292113 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9c781942b0 msgr2=0x7f9c780b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:13.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.865 7f9c7e419700 1 --2- 172.21.2.101:0/4238292113 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9c781942b0 0x7f9c780b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:13.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.865 7f9c7e419700 1 -- 172.21.2.101:0/4238292113 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c781a49e0 con 0x7f9c780bb8e0 2021-02-10T11:39:13.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.865 7f9c7e419700 1 --2- 172.21.2.101:0/4238292113 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c780bb8e0 0x7f9c780bbce0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 rx=0x7f9c70007ec0 tx=0x7f9c7004fc80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:13.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.865 7f9c6e7fc700 1 -- 172.21.2.101:0/4238292113 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9c70000f90 con 0x7f9c780bb8e0 2021-02-10T11:39:13.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.865 7f9c6e7fc700 1 -- 172.21.2.101:0/4238292113 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c70007030 con 0x7f9c780bb8e0 2021-02-10T11:39:13.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.865 7f9c6e7fc700 1 -- 172.21.2.101:0/4238292113 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9c70007550 con 0x7f9c780bb8e0 2021-02-10T11:39:13.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.865 7f9c7fdfc700 1 -- 172.21.2.101:0/4238292113 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c781a49e0 con 0x7f9c780bb8e0 2021-02-10T11:39:13.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.865 7f9c7fdfc700 1 -- 172.21.2.101:0/4238292113 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c781a49e0 con 0x7f9c780bb8e0 2021-02-10T11:39:13.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.866 7f9c6e7fc700 1 -- 172.21.2.101:0/4238292113 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9c70007420 con 0x7f9c780bb8e0 2021-02-10T11:39:13.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.866 7f9c6e7fc700 1 --2- 172.21.2.101:0/4238292113 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9c600186c0 0x7f9c6001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:13.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.866 7f9c6e7fc700 1 -- 172.21.2.101:0/4238292113 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9c70076780 con 0x7f9c780bb8e0 2021-02-10T11:39:13.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.866 7f9c7dc18700 1 --2- 172.21.2.101:0/4238292113 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9c600186c0 0x7f9c6001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:13.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.866 7f9c7dc18700 1 --2- 172.21.2.101:0/4238292113 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9c600186c0 0x7f9c6001abc0 crc :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:13.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.897 7f9c7fdfc700 1 -- 172.21.2.101:0/4238292113 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c781a49e0 con 0x7f9c780bb8e0 2021-02-10T11:39:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:11.900 7f9c6e7fc700 1 -- 172.21.2.101:0/4238292113 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9c70061db0 con 0x7f9c780bb8e0 2021-02-10T11:39:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:12.027 7f9c7fdfc700 1 -- 172.21.2.101:0/4238292113 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c781a49e0 con 0x7f9c780bb8e0 2021-02-10T11:39:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:12.028 7f9c6e7fc700 1 -- 172.21.2.101:0/4238292113 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f9c70061db0 con 0x7f9c780bb8e0 2021-02-10T11:39:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 71 2021-02-10T11:39:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:12.030 7f9c7fdfc700 1 -- 172.21.2.101:0/4238292113 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9c600186c0 msgr2=0x7f9c6001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:12.030 7f9c7fdfc700 1 --2- 172.21.2.101:0/4238292113 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9c600186c0 0x7f9c6001abc0 crc :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:12.030 7f9c7fdfc700 1 -- 172.21.2.101:0/4238292113 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c780bb8e0 msgr2=0x7f9c780bbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:12.030 7f9c7fdfc700 1 --2- 172.21.2.101:0/4238292113 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c780bb8e0 0x7f9c780bbce0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 rx=0x7f9c70007ec0 tx=0x7f9c7004fc80).stop 2021-02-10T11:39:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:12.030 7f9c7fdfc700 1 -- 172.21.2.101:0/4238292113 shutdown_connections 2021-02-10T11:39:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:12.030 7f9c7fdfc700 1 --2- 172.21.2.101:0/4238292113 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9c600186c0 0x7f9c6001abc0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:12.030 7f9c7fdfc700 1 --2- 172.21.2.101:0/4238292113 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9c78195070 0x7f9c78195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:12.030 7f9c7fdfc700 1 --2- 172.21.2.101:0/4238292113 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c780bb8e0 0x7f9c780bbce0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:12.030 7f9c7fdfc700 1 --2- 172.21.2.101:0/4238292113 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9c781942b0 0x7f9c780b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:13.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:12.030 7f9c7fdfc700 1 -- 172.21.2.101:0/4238292113 >> 172.21.2.101:0/4238292113 conn(0x7f9c780b3040 msgr2=0x7f9c780c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:13.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:12.030 7f9c7fdfc700 1 -- 172.21.2.101:0/4238292113 shutdown_connections 2021-02-10T11:39:13.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:12.030 7f9c7fdfc700 1 -- 172.21.2.101:0/4238292113 wait complete. 2021-02-10T11:39:13.187 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:39:13.187 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/4210469578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":4210469578},{"type":"v1","addr":"172.21.2.108:6833","nonce":4210469578}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:39:02.063078","modified":"2021-02-10 11:39:04.073713","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6164},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6164":{"gid":6164,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.2.101:6835/1113812290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1113812290},{"type":"v1","addr":"172.21.2.101:6835","nonce":1113812290}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-02-10T11:39:13.389 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:39:02.063078', 'modified': '2021-02-10 11:39:04.073713', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6164}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6164': {'gid': 6164, 'name': 'a', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.101:6835/1113812290', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.101:6834', 'nonce': 1113812290}, {'type': 'v1', 'addr': '172.21.2.101:6835', 'nonce': 1113812290}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:39:13.389 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:39:13.389 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok status 2021-02-10T11:39:13.790 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:39:13.790 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:39:13.790 INFO:teuthology.orchestra.run.gibba001.stdout: "whoami": 0, 2021-02-10T11:39:13.790 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 6164, 2021-02-10T11:39:13.790 INFO:teuthology.orchestra.run.gibba001.stdout: "want_state": "up:active", 2021-02-10T11:39:13.790 INFO:teuthology.orchestra.run.gibba001.stdout: "state": "up:active", 2021-02-10T11:39:13.790 INFO:teuthology.orchestra.run.gibba001.stdout: "rank_uptime": 11.721499041, 2021-02-10T11:39:13.790 INFO:teuthology.orchestra.run.gibba001.stdout: "mdsmap_epoch": 71, 2021-02-10T11:39:13.791 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch": 94, 2021-02-10T11:39:13.791 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch_barrier": 94, 2021-02-10T11:39:13.791 INFO:teuthology.orchestra.run.gibba001.stdout: "uptime": 17.364437689999999 2021-02-10T11:39:13.791 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:39:13.794 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 6164, "want_state": "up:active", "state": "up:active", "rank_uptime": 11.721499041, "mdsmap_epoch": 71, "osdmap_epoch": 94, "osdmap_epoch_barrier": 94, "uptime": 17.364437689999999 } 2021-02-10T11:39:13.794 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:39:13.794 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-02-10T11:39:13.795 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-02-10T11:39:13.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:13.795 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:39:14.129 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:39:14.473 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:39:14.473 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:39:14.816 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:39:14.817 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:39:14.817 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:39:14.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:14.818 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:39:15.152 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-02-10T11:39:15.153 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:39:15.155 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:39:15.499 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:39:15.499 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:39:15.565 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[139273]: starting ceph client 2021-02-10T11:39:15.565 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:2021-02-10 11:39:15.561 7fc78938bf80 -1 init, newargv = 0x55d5ff22e9a0 newargc=9 2021-02-10T11:39:15.569 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[139273]: starting fuse 2021-02-10T11:39:15.841 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:39:15.841 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:39:15.842 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:39:15.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:15.843 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:39:16.178 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:39:16.178 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2021-02-10T11:39:16.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:16.178 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:39:16.178 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:39:16.179 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:39:16.179 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:39:16.179 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:39:16.179 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:39:16.179 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:39:16.179 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-02-10T11:39:16.179 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:39:16.179 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:39:16.179 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:39:16.179 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:39:16.179 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:39:16.179 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:39:16.179 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:39:16.179 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:39:16.180 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:39:16.180 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:39:16.180 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:39:16.180 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:39:16.180 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:39:16.180 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.0")) 2021-02-10T11:39:16.180 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:39:16.543 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.0.139273.asok 2021-02-10T11:39:16.546 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.139273.asok 2021-02-10T11:39:16.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:16.546 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.139273.asok status 2021-02-10T11:39:16.935 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:39:16.935 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:39:16.935 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:39:16.935 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:39:16.935 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "0", 2021-02-10T11:39:16.936 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:39:16.936 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-02-10T11:39:16.936 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "139273", 2021-02-10T11:39:16.936 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:39:16.936 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:16.936 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:39:16.936 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:39:16.936 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 9915, 2021-02-10T11:39:16.936 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:39:16.936 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:39:16.936 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:39:16.936 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 9915 2021-02-10T11:39:16.936 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:16.937 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:39:16.937 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:16.937 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:39:16.937 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 2493739895 2021-02-10T11:39:16.937 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:16.937 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:16.937 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:39:16.937 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:16.937 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:39:16.937 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 2493739895 2021-02-10T11:39:16.937 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:16.937 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.9915 172.21.2.101:0/2493739895", 2021-02-10T11:39:16.937 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/2493739895", 2021-02-10T11:39:16.938 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:39:16.938 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 71, 2021-02-10T11:39:16.938 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 94, 2021-02-10T11:39:16.938 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:39:16.938 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:39:16.938 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:39:16.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:16.939 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:39:16.939 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:39:17.276 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:39:17.277 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:39:17.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:17.277 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:39:17.624 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:39:18.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.031 7f41cefe5700 1 Processor -- start 2021-02-10T11:39:18.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.031 7f41cefe5700 1 -- start start 2021-02-10T11:39:18.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.032 7f41cefe5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41c80d0340 0x7f41c80ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:18.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.032 7f41cefe5700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41c80f0b10 con 0x7f41c80d1c70 2021-02-10T11:39:18.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.032 7f41cefe5700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41c81b0be0 con 0x7f41c80d1870 2021-02-10T11:39:18.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.032 7f41cefe5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41c80e5ed0 con 0x7f41c80d0340 2021-02-10T11:39:18.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.032 7f41cd602700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f41c80d1c70 0x7f41c80efc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:55942/0 (socket says 172.21.2.101:55942) 2021-02-10T11:39:18.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.032 7f41cd602700 1 -- 172.21.2.101:0/761607809 learned_addr learned my addr 172.21.2.101:0/761607809 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:18.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.032 7f41c77fe700 1 -- 172.21.2.101:0/761607809 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1159207315 0 0) 0x7f41c0001480 con 0x7f41c80d1c70 2021-02-10T11:39:18.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.032 7f41c77fe700 1 -- 172.21.2.101:0/761607809 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41b4002fe0 con 0x7f41c80d1c70 2021-02-10T11:39:18.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.032 7f41c7fff700 1 --2- 172.21.2.101:0/761607809 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41c80d0340 0x7f41c80ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:18.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41c77fe700 1 -- 172.21.2.101:0/761607809 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3530546472 0 0) 0x7f41c0001270 con 0x7f41c80d1c70 2021-02-10T11:39:18.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41c77fe700 1 -- 172.21.2.101:0/761607809 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41c80d0340 msgr2=0x7f41c80ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:18.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41c77fe700 1 --2- 172.21.2.101:0/761607809 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41c80d0340 0x7f41c80ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:18.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41c77fe700 1 -- 172.21.2.101:0/761607809 >> v1:172.21.2.108:6789/0 conn(0x7f41c80d1870 legacy=0x7f41c81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:18.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41c77fe700 1 -- 172.21.2.101:0/761607809 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c80f0450 con 0x7f41c80d1c70 2021-02-10T11:39:18.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41c77fe700 1 -- 172.21.2.101:0/761607809 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f41c0001ad0 con 0x7f41c80d1c70 2021-02-10T11:39:18.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41c77fe700 1 -- 172.21.2.101:0/761607809 >> v1:172.21.2.101:6789/0 conn(0x7f41c80d1c70 legacy=0x7f41c80efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:18.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41c77fe700 1 --2- 172.21.2.101:0/761607809 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f41b4003af0 0x7f41b4005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:18.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41c77fe700 1 --2- 172.21.2.101:0/761607809 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41b4006840 0x7f41b4008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:18.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41c77fe700 1 --2- 172.21.2.101:0/761607809 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41b4009640 0x7f41b400bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:18.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41c77fe700 1 -- 172.21.2.101:0/761607809 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b400c380 con 0x7f41b4006840 2021-02-10T11:39:18.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41c77fe700 1 -- 172.21.2.101:0/761607809 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b400c5a0 con 0x7f41b4003af0 2021-02-10T11:39:18.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41c77fe700 1 -- 172.21.2.101:0/761607809 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b400c7c0 con 0x7f41b4009640 2021-02-10T11:39:18.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41c7fff700 1 --2- 172.21.2.101:0/761607809 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41c80d0340 0x7f41c80ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:39:18.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41c7fff700 1 --2- 172.21.2.101:0/761607809 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41b4006840 0x7f41b4008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:18.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41cd602700 1 --2- 172.21.2.101:0/761607809 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41b4009640 0x7f41b400bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:18.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41cce01700 1 --2- 172.21.2.101:0/761607809 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f41b4003af0 0x7f41b4005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:18.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41c7fff700 1 -- 172.21.2.101:0/761607809 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41b4009640 msgr2=0x7f41b400bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:18.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41c7fff700 1 --2- 172.21.2.101:0/761607809 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41b4009640 0x7f41b400bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:18.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41c7fff700 1 -- 172.21.2.101:0/761607809 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f41b4003af0 msgr2=0x7f41b4005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:18.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41c7fff700 1 --2- 172.21.2.101:0/761607809 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f41b4003af0 0x7f41b4005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:18.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41c7fff700 1 -- 172.21.2.101:0/761607809 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b400ca60 con 0x7f41b4006840 2021-02-10T11:39:18.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41cce01700 1 --2- 172.21.2.101:0/761607809 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f41b4003af0 0x7f41b4005fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:39:18.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.033 7f41c7fff700 1 --2- 172.21.2.101:0/761607809 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41b4006840 0x7f41b4008d40 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 rx=0x7f41b8002e00 tx=0x7f41b8050ac0).ready entity=mon.0 client_cookie=1c2188244a5c668f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:18.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.034 7f41c77fe700 1 -- 172.21.2.101:0/761607809 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f41b80090c0 con 0x7f41b4006840 2021-02-10T11:39:18.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.034 7f41c77fe700 1 -- 172.21.2.101:0/761607809 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b80093c0 con 0x7f41b4006840 2021-02-10T11:39:18.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.034 7f41c77fe700 1 -- 172.21.2.101:0/761607809 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f41b8009820 con 0x7f41b4006840 2021-02-10T11:39:18.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.034 7f41cefe5700 1 -- 172.21.2.101:0/761607809 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41b4006840 msgr2=0x7f41b4008d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:18.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.034 7f41cefe5700 1 --2- 172.21.2.101:0/761607809 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41b4006840 0x7f41b4008d40 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 rx=0x7f41b8002e00 tx=0x7f41b8050ac0).stop 2021-02-10T11:39:18.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.034 7f41cd602700 1 -- 172.21.2.101:0/761607809 reap_dead start 2021-02-10T11:39:18.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.034 7f41cefe5700 1 -- 172.21.2.101:0/761607809 shutdown_connections 2021-02-10T11:39:18.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.034 7f41cefe5700 1 -- 172.21.2.101:0/761607809 >> 172.21.2.101:0/761607809 conn(0x7f41c80ce450 msgr2=0x7f41c80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:18.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.034 7f41cefe5700 1 -- 172.21.2.101:0/761607809 shutdown_connections 2021-02-10T11:39:18.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.034 7f41cefe5700 1 -- 172.21.2.101:0/761607809 wait complete. 2021-02-10T11:39:18.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.049 7f41cefe5700 1 Processor -- start 2021-02-10T11:39:18.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.049 7f41cefe5700 1 -- start start 2021-02-10T11:39:18.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.049 7f41cefe5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41c81942b0 0x7f41c80b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:18.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.049 7f41cefe5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41c8195070 0x7f41c8195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:18.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.049 7f41cefe5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f41c80bb8e0 0x7f41c80bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:18.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.049 7f41cefe5700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c81946b0 con 0x7f41c81942b0 2021-02-10T11:39:18.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.049 7f41cefe5700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c80b3b70 con 0x7f41c80bb8e0 2021-02-10T11:39:18.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.049 7f41cefe5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41c80b3d90 con 0x7f41c8195070 2021-02-10T11:39:18.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.050 7f41cce01700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41c81942b0 0x7f41c80b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:18.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.050 7f41cce01700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41c81942b0 0x7f41c80b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55456/0 (socket says 172.21.2.101:55456) 2021-02-10T11:39:18.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.050 7f41cce01700 1 -- 172.21.2.101:0/799006497 learned_addr learned my addr 172.21.2.101:0/799006497 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:18.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.050 7f41c7fff700 1 --2- 172.21.2.101:0/799006497 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41c8195070 0x7f41c8195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:18.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.050 7f41cd602700 1 --2- 172.21.2.101:0/799006497 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f41c80bb8e0 0x7f41c80bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:18.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.050 7f41cce01700 1 -- 172.21.2.101:0/799006497 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41c8195070 msgr2=0x7f41c8195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:18.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.050 7f41cce01700 1 --2- 172.21.2.101:0/799006497 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41c8195070 0x7f41c8195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:18.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.050 7f41cce01700 1 -- 172.21.2.101:0/799006497 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f41c80bb8e0 msgr2=0x7f41c80bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:18.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.050 7f41cce01700 1 --2- 172.21.2.101:0/799006497 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f41c80bb8e0 0x7f41c80bbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:18.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.050 7f41cce01700 1 -- 172.21.2.101:0/799006497 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c81a49e0 con 0x7f41c81942b0 2021-02-10T11:39:18.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.050 7f41cce01700 1 --2- 172.21.2.101:0/799006497 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41c81942b0 0x7f41c80b3550 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 rx=0x7f41b0006f10 tx=0x7f41b000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:18.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.050 7f41c57fa700 1 -- 172.21.2.101:0/799006497 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f41b000bd80 con 0x7f41c81942b0 2021-02-10T11:39:18.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.050 7f41c57fa700 1 -- 172.21.2.101:0/799006497 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b0000ca0 con 0x7f41c81942b0 2021-02-10T11:39:18.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.050 7f41c57fa700 1 -- 172.21.2.101:0/799006497 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f41b0002220 con 0x7f41c81942b0 2021-02-10T11:39:18.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.050 7f41cefe5700 1 -- 172.21.2.101:0/799006497 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c81a49e0 con 0x7f41c81942b0 2021-02-10T11:39:18.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.051 7f41cefe5700 1 -- 172.21.2.101:0/799006497 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c80bc5f0 con 0x7f41c81942b0 2021-02-10T11:39:18.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.051 7f41c57fa700 1 -- 172.21.2.101:0/799006497 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f41b00052e0 con 0x7f41c81942b0 2021-02-10T11:39:18.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.051 7f41c57fa700 1 --2- 172.21.2.101:0/799006497 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f41ac018700 0x7f41ac01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:18.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.051 7f41c57fa700 1 -- 172.21.2.101:0/799006497 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f41b0032610 con 0x7f41c81942b0 2021-02-10T11:39:18.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.051 7f41c7fff700 1 --2- 172.21.2.101:0/799006497 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f41ac018700 0x7f41ac01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:18.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.052 7f41c7fff700 1 --2- 172.21.2.101:0/799006497 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f41ac018700 0x7f41ac01abc0 crc :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:18.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.081 7f41cefe5700 1 -- 172.21.2.101:0/799006497 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41c80bc5f0 con 0x7f41c81942b0 2021-02-10T11:39:18.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.084 7f41c57fa700 1 -- 172.21.2.101:0/799006497 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f41b0006be0 con 0x7f41c81942b0 2021-02-10T11:39:18.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.208 7f41cefe5700 1 -- 172.21.2.101:0/799006497 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41c80bc5f0 con 0x7f41c81942b0 2021-02-10T11:39:18.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.209 7f41c57fa700 1 -- 172.21.2.101:0/799006497 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+13996 (secure 0 0 0) 0x7f41b00052e0 con 0x7f41c81942b0 2021-02-10T11:39:18.213 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:39:18.213 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":94,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:39:03.065120","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:39:00.027476","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2021-02-10 11:39:01.054115","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:39:18.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.210 7f41cefe5700 1 -- 172.21.2.101:0/799006497 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f41ac018700 msgr2=0x7f41ac01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:18.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.210 7f41cefe5700 1 --2- 172.21.2.101:0/799006497 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f41ac018700 0x7f41ac01abc0 crc :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:18.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.210 7f41cefe5700 1 -- 172.21.2.101:0/799006497 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41c81942b0 msgr2=0x7f41c80b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:18.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.210 7f41cefe5700 1 --2- 172.21.2.101:0/799006497 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41c81942b0 0x7f41c80b3550 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 rx=0x7f41b0006f10 tx=0x7f41b000b990).stop 2021-02-10T11:39:18.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.211 7f41cefe5700 1 -- 172.21.2.101:0/799006497 shutdown_connections 2021-02-10T11:39:18.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.211 7f41cefe5700 1 --2- 172.21.2.101:0/799006497 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f41ac018700 0x7f41ac01abc0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:18.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.211 7f41cefe5700 1 --2- 172.21.2.101:0/799006497 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f41c8195070 0x7f41c8195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:18.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.211 7f41cefe5700 1 --2- 172.21.2.101:0/799006497 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f41c80bb8e0 0x7f41c80bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:18.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.211 7f41cefe5700 1 --2- 172.21.2.101:0/799006497 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f41c81942b0 0x7f41c80b3550 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:18.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.211 7f41cefe5700 1 -- 172.21.2.101:0/799006497 >> 172.21.2.101:0/799006497 conn(0x7f41c80b3040 msgr2=0x7f41c80c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:18.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.211 7f41cefe5700 1 -- 172.21.2.101:0/799006497 shutdown_connections 2021-02-10T11:39:18.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.211 7f41cefe5700 1 -- 172.21.2.101:0/799006497 wait complete. 2021-02-10T11:39:18.224 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:39:00.027476', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:39:01.054115', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:39:18.224 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:39:18.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.636 7f2d394f5700 1 Processor -- start 2021-02-10T11:39:18.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.636 7f2d394f5700 1 -- start start 2021-02-10T11:39:18.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.636 7f2d394f5700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2d340d1870 0x7f2d341b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:18.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.636 7f2d394f5700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2d340cff70 0x7f2d34109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:18.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.636 7f2d394f5700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d340fe1c0 con 0x7f2d341b0ae0 2021-02-10T11:39:18.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.636 7f2d394f5700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d34109ad0 con 0x7f2d340cff70 2021-02-10T11:39:18.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.636 7f2d394f5700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d34109750 con 0x7f2d340d1870 2021-02-10T11:39:18.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.637 7f2d33ac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2d340cff70 0x7f2d34109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:18.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.637 7f2d33ac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2d340cff70 0x7f2d34109130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55464/0 (socket says 172.21.2.101:55464) 2021-02-10T11:39:18.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.637 7f2d33ac0700 1 -- 172.21.2.101:0/1295202040 learned_addr learned my addr 172.21.2.101:0/1295202040 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:18.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.637 7f2d33ac0700 1 -- 172.21.2.101:0/1295202040 >> v2:172.21.2.108:3300/0 conn(0x7f2d340d1870 msgr2=0x7f2d341b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:39:18.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.637 7f2d33ac0700 1 --2- 172.21.2.101:0/1295202040 >> v2:172.21.2.108:3300/0 conn(0x7f2d340d1870 0x7f2d341b04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:18.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.637 7f2d33ac0700 1 -- 172.21.2.101:0/1295202040 >> v1:172.21.2.101:6789/0 conn(0x7f2d341b0ae0 legacy=0x7f2d34108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:18.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.637 7f2d33ac0700 1 -- 172.21.2.101:0/1295202040 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d34088070 con 0x7f2d340cff70 2021-02-10T11:39:18.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.637 7f2d33ac0700 1 --2- 172.21.2.101:0/1295202040 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d340cff70 0x7f2d34109130 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 rx=0x7f2d2c006f00 tx=0x7f2d2c009f10).ready entity=mon.0 client_cookie=167f9abe4d128f4c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:18.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.637 7f2d322bd700 1 -- 172.21.2.101:0/1295202040 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d2c003030 con 0x7f2d340cff70 2021-02-10T11:39:18.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.637 7f2d322bd700 1 -- 172.21.2.101:0/1295202040 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2c00b3b0 con 0x7f2d340cff70 2021-02-10T11:39:18.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.637 7f2d322bd700 1 -- 172.21.2.101:0/1295202040 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d2c00b8d0 con 0x7f2d340cff70 2021-02-10T11:39:18.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.637 7f2d394f5700 1 -- 172.21.2.101:0/1295202040 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d340cff70 msgr2=0x7f2d34109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:18.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.637 7f2d394f5700 1 --2- 172.21.2.101:0/1295202040 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d340cff70 0x7f2d34109130 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 rx=0x7f2d2c006f00 tx=0x7f2d2c009f10).stop 2021-02-10T11:39:18.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.638 7f2d394f5700 1 -- 172.21.2.101:0/1295202040 shutdown_connections 2021-02-10T11:39:18.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.638 7f2d394f5700 1 --2- 172.21.2.101:0/1295202040 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d340cff70 0x7f2d34109130 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:18.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.638 7f2d394f5700 1 --2- 172.21.2.101:0/1295202040 >> v2:172.21.2.108:3300/0 conn(0x7f2d340d1870 0x7f2d341b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:18.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.638 7f2d394f5700 1 -- 172.21.2.101:0/1295202040 >> 172.21.2.101:0/1295202040 conn(0x7f2d340ce420 msgr2=0x7f2d340d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:18.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.638 7f2d394f5700 1 -- 172.21.2.101:0/1295202040 shutdown_connections 2021-02-10T11:39:18.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.638 7f2d394f5700 1 -- 172.21.2.101:0/1295202040 wait complete. 2021-02-10T11:39:18.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.653 7f2d394f5700 1 Processor -- start 2021-02-10T11:39:18.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.653 7f2d394f5700 1 -- start start 2021-02-10T11:39:18.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.653 7f2d394f5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d340c3820 0x7f2d34084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.653 7f2d394f5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d340a0290 0x7f2d340852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.653 7f2d394f5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d34086270 0x7f2d34086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.653 7f2d394f5700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d340c0700 con 0x7f2d340c3820 2021-02-10T11:39:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.653 7f2d394f5700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d340858f0 con 0x7f2d340a0290 2021-02-10T11:39:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.653 7f2d394f5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d34085b10 con 0x7f2d34086270 2021-02-10T11:39:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.653 7f2d332bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d340c3820 0x7f2d34084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.653 7f2d332bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d340c3820 0x7f2d34084cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55472/0 (socket says 172.21.2.101:55472) 2021-02-10T11:39:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.653 7f2d332bf700 1 -- 172.21.2.101:0/4265458707 learned_addr learned my addr 172.21.2.101:0/4265458707 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.654 7f2d332bf700 1 -- 172.21.2.101:0/4265458707 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d34086270 msgr2=0x7f2d34086670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.654 7f2d332bf700 1 --2- 172.21.2.101:0/4265458707 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d34086270 0x7f2d34086670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.654 7f2d332bf700 1 -- 172.21.2.101:0/4265458707 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d340a0290 msgr2=0x7f2d340852d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.654 7f2d332bf700 1 --2- 172.21.2.101:0/4265458707 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d340a0290 0x7f2d340852d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.654 7f2d332bf700 1 -- 172.21.2.101:0/4265458707 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d34086c90 con 0x7f2d340c3820 2021-02-10T11:39:18.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.654 7f2d332bf700 1 --2- 172.21.2.101:0/4265458707 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d340c3820 0x7f2d34084cb0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 rx=0x7f2d1c00b5c0 tx=0x7f2d1c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:18.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.654 7f2d2bfff700 1 -- 172.21.2.101:0/4265458707 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d1c00bd50 con 0x7f2d340c3820 2021-02-10T11:39:18.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.654 7f2d2bfff700 1 -- 172.21.2.101:0/4265458707 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1c006030 con 0x7f2d340c3820 2021-02-10T11:39:18.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.654 7f2d2bfff700 1 -- 172.21.2.101:0/4265458707 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d1c0054d0 con 0x7f2d340c3820 2021-02-10T11:39:18.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.654 7f2d394f5700 1 -- 172.21.2.101:0/4265458707 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d34086c90 con 0x7f2d340c3820 2021-02-10T11:39:18.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.654 7f2d394f5700 1 -- 172.21.2.101:0/4265458707 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d341a2f00 con 0x7f2d340c3820 2021-02-10T11:39:18.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.655 7f2d2bfff700 1 -- 172.21.2.101:0/4265458707 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2d1c006210 con 0x7f2d340c3820 2021-02-10T11:39:18.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.655 7f2d2bfff700 1 --2- 172.21.2.101:0/4265458707 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d180186c0 0x7f2d1801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:18.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.655 7f2d2bfff700 1 -- 172.21.2.101:0/4265458707 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2d1c0325b0 con 0x7f2d340c3820 2021-02-10T11:39:18.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.655 7f2d32abe700 1 --2- 172.21.2.101:0/4265458707 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d180186c0 0x7f2d1801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:18.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.655 7f2d32abe700 1 --2- 172.21.2.101:0/4265458707 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d180186c0 0x7f2d1801abc0 crc :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:18.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.687 7f2d394f5700 1 -- 172.21.2.101:0/4265458707 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d34082a70 con 0x7f2d340c3820 2021-02-10T11:39:18.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.689 7f2d2bfff700 1 -- 172.21.2.101:0/4265458707 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2d1c02cf30 con 0x7f2d340c3820 2021-02-10T11:39:18.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.816 7f2d394f5700 1 -- 172.21.2.101:0/4265458707 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f2d34082a70 con 0x7f2d340c3820 2021-02-10T11:39:18.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.816 7f2d2bfff700 1 -- 172.21.2.101:0/4265458707 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v94) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2d1c02cf30 con 0x7f2d340c3820 2021-02-10T11:39:18.821 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:39:18.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.818 7f2d394f5700 1 -- 172.21.2.101:0/4265458707 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d180186c0 msgr2=0x7f2d1801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:18.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.818 7f2d394f5700 1 --2- 172.21.2.101:0/4265458707 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d180186c0 0x7f2d1801abc0 crc :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:18.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.818 7f2d394f5700 1 -- 172.21.2.101:0/4265458707 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d340c3820 msgr2=0x7f2d34084cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:18.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.818 7f2d394f5700 1 --2- 172.21.2.101:0/4265458707 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d340c3820 0x7f2d34084cb0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 rx=0x7f2d1c00b5c0 tx=0x7f2d1c00b960).stop 2021-02-10T11:39:18.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.819 7f2d394f5700 1 -- 172.21.2.101:0/4265458707 shutdown_connections 2021-02-10T11:39:18.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.819 7f2d394f5700 1 --2- 172.21.2.101:0/4265458707 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d180186c0 0x7f2d1801abc0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:18.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.819 7f2d394f5700 1 --2- 172.21.2.101:0/4265458707 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d34086270 0x7f2d34086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:18.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.819 7f2d394f5700 1 --2- 172.21.2.101:0/4265458707 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d340a0290 0x7f2d340852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:18.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.819 7f2d394f5700 1 --2- 172.21.2.101:0/4265458707 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d340c3820 0x7f2d34084cb0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:18.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.819 7f2d394f5700 1 -- 172.21.2.101:0/4265458707 >> 172.21.2.101:0/4265458707 conn(0x7f2d340872c0 msgr2=0x7f2d34095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:18.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.819 7f2d394f5700 1 -- 172.21.2.101:0/4265458707 shutdown_connections 2021-02-10T11:39:18.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:18.819 7f2d394f5700 1 -- 172.21.2.101:0/4265458707 wait complete. 2021-02-10T11:39:18.831 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:39:19.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.241 7f4fe20dd700 1 Processor -- start 2021-02-10T11:39:19.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.241 7f4fe20dd700 1 -- start start 2021-02-10T11:39:19.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.242 7f4fe20dd700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4fdc0d1c70 0x7f4fdc0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:19.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.242 7f4fe20dd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4fdc0cf580 0x7f4fdc0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:19.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.242 7f4fe20dd700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fdc0fe1c0 con 0x7f4fdc0d1870 2021-02-10T11:39:19.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.242 7f4fe20dd700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fdc1b0cb0 con 0x7f4fdc0d1c70 2021-02-10T11:39:19.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.242 7f4fe20dd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc109a60 con 0x7f4fdc0cf580 2021-02-10T11:39:19.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.243 7f4fdbfff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4fdc0cf580 0x7f4fdc0cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:19.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.243 7f4fdaffd700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4fdc0d1c70 0x7f4fdc0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:19.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.243 7f4fdaffd700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4fdc0d1c70 0x7f4fdc0cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:37732/0 (socket says 172.21.2.101:37732) 2021-02-10T11:39:19.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.243 7f4fdbfff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4fdc0cf580 0x7f4fdc0cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:48214/0 (socket says 172.21.2.101:48214) 2021-02-10T11:39:19.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.243 7f4fdaffd700 1 -- 172.21.2.101:0/1061528839 learned_addr learned my addr 172.21.2.101:0/1061528839 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:19.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.243 7f4fdbfff700 1 -- 172.21.2.101:0/1061528839 >> v2:172.21.2.108:3300/0 conn(0x7f4fdc0d1c70 msgr2=0x7f4fdc0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:19.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.243 7f4fdbfff700 1 --2- 172.21.2.101:0/1061528839 >> v2:172.21.2.108:3300/0 conn(0x7f4fdc0d1c70 0x7f4fdc0cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:19.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.243 7f4fdbfff700 1 -- 172.21.2.101:0/1061528839 >> v1:172.21.2.108:6789/0 conn(0x7f4fdc0d1870 legacy=0x7f4fdc1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:19.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.243 7f4fda7fc700 1 -- 172.21.2.101:0/1061528839 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969954736 0 0) 0x7f4fcc000ff0 con 0x7f4fdc0d1870 2021-02-10T11:39:19.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.243 7f4fdbfff700 1 -- 172.21.2.101:0/1061528839 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc088070 con 0x7f4fdc0cf580 2021-02-10T11:39:19.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.243 7f4fdaffd700 1 --2- 172.21.2.101:0/1061528839 >> v2:172.21.2.108:3300/0 conn(0x7f4fdc0d1c70 0x7f4fdc0cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:39:19.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.243 7f4fdbfff700 1 --2- 172.21.2.101:0/1061528839 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4fdc0cf580 0x7f4fdc0cf980 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f4fd4006e50 tx=0x7f4fd4009e70).ready entity=mon.2 client_cookie=148efce59cbee194 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:19.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.244 7f4fda7fc700 1 -- 172.21.2.101:0/1061528839 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4fd400b300 con 0x7f4fdc0cf580 2021-02-10T11:39:19.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.244 7f4fda7fc700 1 -- 172.21.2.101:0/1061528839 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd4003030 con 0x7f4fdc0cf580 2021-02-10T11:39:19.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.244 7f4fda7fc700 1 -- 172.21.2.101:0/1061528839 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4fd400b770 con 0x7f4fdc0cf580 2021-02-10T11:39:19.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.244 7f4fe20dd700 1 -- 172.21.2.101:0/1061528839 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4fdc0cf580 msgr2=0x7f4fdc0cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:19.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.244 7f4fe20dd700 1 --2- 172.21.2.101:0/1061528839 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4fdc0cf580 0x7f4fdc0cf980 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f4fd4006e50 tx=0x7f4fd4009e70).stop 2021-02-10T11:39:19.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.244 7f4fe20dd700 1 -- 172.21.2.101:0/1061528839 shutdown_connections 2021-02-10T11:39:19.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.244 7f4fe20dd700 1 --2- 172.21.2.101:0/1061528839 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4fdc0cf580 0x7f4fdc0cf980 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:19.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.244 7f4fe20dd700 1 --2- 172.21.2.101:0/1061528839 >> v2:172.21.2.108:3300/0 conn(0x7f4fdc0d1c70 0x7f4fdc0cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:19.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.244 7f4fe20dd700 1 -- 172.21.2.101:0/1061528839 >> 172.21.2.101:0/1061528839 conn(0x7f4fdc0ce420 msgr2=0x7f4fdc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:19.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.244 7f4fe20dd700 1 -- 172.21.2.101:0/1061528839 shutdown_connections 2021-02-10T11:39:19.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.244 7f4fe20dd700 1 -- 172.21.2.101:0/1061528839 wait complete. 2021-02-10T11:39:19.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.259 7f4fe20dd700 1 Processor -- start 2021-02-10T11:39:19.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.259 7f4fe20dd700 1 -- start start 2021-02-10T11:39:19.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.259 7f4fe20dd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4fdc1a79a0 0x7f4fdc1a7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:19.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.259 7f4fe20dd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4fdc0bdb40 0x7f4fdc0bdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:19.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.259 7f4fe20dd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4fdc082ca0 0x7f4fdc0830a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:19.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.259 7f4fe20dd700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc19d990 con 0x7f4fdc0bdb40 2021-02-10T11:39:19.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.259 7f4fe20dd700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc19dbb0 con 0x7f4fdc1a79a0 2021-02-10T11:39:19.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.259 7f4fe20dd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc0836c0 con 0x7f4fdc082ca0 2021-02-10T11:39:19.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.259 7f4fdaffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4fdc0bdb40 0x7f4fdc0bdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:19.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.259 7f4fdaffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4fdc0bdb40 0x7f4fdc0bdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55486/0 (socket says 172.21.2.101:55486) 2021-02-10T11:39:19.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.259 7f4fdaffd700 1 -- 172.21.2.101:0/1264965294 learned_addr learned my addr 172.21.2.101:0/1264965294 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:19.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.260 7f4fdaffd700 1 -- 172.21.2.101:0/1264965294 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4fdc082ca0 msgr2=0x7f4fdc0830a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:19.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.260 7f4fdaffd700 1 --2- 172.21.2.101:0/1264965294 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4fdc082ca0 0x7f4fdc0830a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:19.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.260 7f4fdaffd700 1 -- 172.21.2.101:0/1264965294 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4fdc1a79a0 msgr2=0x7f4fdc1a7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:19.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.260 7f4fdaffd700 1 --2- 172.21.2.101:0/1264965294 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4fdc1a79a0 0x7f4fdc1a7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:19.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.260 7f4fdaffd700 1 -- 172.21.2.101:0/1264965294 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc0838e0 con 0x7f4fdc0bdb40 2021-02-10T11:39:19.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.260 7f4fdaffd700 1 --2- 172.21.2.101:0/1264965294 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4fdc0bdb40 0x7f4fdc0bdf40 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 rx=0x7f4fc4006ee0 tx=0x7f4fc400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:19.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.260 7f4fd3fff700 1 -- 172.21.2.101:0/1264965294 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4fc400bd50 con 0x7f4fdc0bdb40 2021-02-10T11:39:19.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.260 7f4fd3fff700 1 -- 172.21.2.101:0/1264965294 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc40052e0 con 0x7f4fdc0bdb40 2021-02-10T11:39:19.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.260 7f4fe20dd700 1 -- 172.21.2.101:0/1264965294 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fdc0838e0 con 0x7f4fdc0bdb40 2021-02-10T11:39:19.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.260 7f4fe20dd700 1 -- 172.21.2.101:0/1264965294 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fdc084420 con 0x7f4fdc0bdb40 2021-02-10T11:39:19.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.260 7f4fd3fff700 1 -- 172.21.2.101:0/1264965294 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4fc4005800 con 0x7f4fdc0bdb40 2021-02-10T11:39:19.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.261 7f4fd3fff700 1 -- 172.21.2.101:0/1264965294 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4fc40057d0 con 0x7f4fdc0bdb40 2021-02-10T11:39:19.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.261 7f4fd3fff700 1 --2- 172.21.2.101:0/1264965294 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4fc00186c0 0x7f4fc001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:19.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.261 7f4fd3fff700 1 -- 172.21.2.101:0/1264965294 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4fc4032530 con 0x7f4fdc0bdb40 2021-02-10T11:39:19.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.261 7f4fdb7fe700 1 --2- 172.21.2.101:0/1264965294 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4fc00186c0 0x7f4fc001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:19.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.261 7f4fdb7fe700 1 --2- 172.21.2.101:0/1264965294 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4fc00186c0 0x7f4fc001abc0 crc :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:19.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.291 7f4fe20dd700 1 -- 172.21.2.101:0/1264965294 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fdc082a70 con 0x7f4fdc0bdb40 2021-02-10T11:39:19.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.294 7f4fd3fff700 1 -- 172.21.2.101:0/1264965294 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4fc4000960 con 0x7f4fdc0bdb40 2021-02-10T11:39:19.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.420 7f4fe20dd700 1 -- 172.21.2.101:0/1264965294 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f4fdc0838e0 con 0x7f4fdc0bdb40 2021-02-10T11:39:19.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.421 7f4fd3fff700 1 -- 172.21.2.101:0/1264965294 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v94) v1 ==== 104+0+10 (secure 0 0 0) 0x7f4fc400bd50 con 0x7f4fdc0bdb40 2021-02-10T11:39:19.425 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:39:19.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.422 7f4fe20dd700 1 -- 172.21.2.101:0/1264965294 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4fc00186c0 msgr2=0x7f4fc001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:19.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.422 7f4fe20dd700 1 --2- 172.21.2.101:0/1264965294 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4fc00186c0 0x7f4fc001abc0 crc :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:19.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.422 7f4fe20dd700 1 -- 172.21.2.101:0/1264965294 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4fdc0bdb40 msgr2=0x7f4fdc0bdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:19.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.422 7f4fe20dd700 1 --2- 172.21.2.101:0/1264965294 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4fdc0bdb40 0x7f4fdc0bdf40 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 rx=0x7f4fc4006ee0 tx=0x7f4fc400b960).stop 2021-02-10T11:39:19.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.423 7f4fe20dd700 1 -- 172.21.2.101:0/1264965294 shutdown_connections 2021-02-10T11:39:19.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.423 7f4fe20dd700 1 --2- 172.21.2.101:0/1264965294 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4fc00186c0 0x7f4fc001abc0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:19.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.423 7f4fe20dd700 1 --2- 172.21.2.101:0/1264965294 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4fdc082ca0 0x7f4fdc0830a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:19.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.423 7f4fe20dd700 1 --2- 172.21.2.101:0/1264965294 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4fdc0bdb40 0x7f4fdc0bdf40 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:19.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.423 7f4fe20dd700 1 --2- 172.21.2.101:0/1264965294 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4fdc1a79a0 0x7f4fdc1a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:19.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.423 7f4fe20dd700 1 -- 172.21.2.101:0/1264965294 >> 172.21.2.101:0/1264965294 conn(0x7f4fdc0bef00 msgr2=0x7f4fdc09f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:19.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.423 7f4fe20dd700 1 -- 172.21.2.101:0/1264965294 shutdown_connections 2021-02-10T11:39:19.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.423 7f4fe20dd700 1 -- 172.21.2.101:0/1264965294 wait complete. 2021-02-10T11:39:19.435 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:39:19.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.847 7f2b8110f700 1 Processor -- start 2021-02-10T11:39:19.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.847 7f2b8110f700 1 -- start start 2021-02-10T11:39:19.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.847 7f2b8110f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2b7c0d1c60 0x7f2b7c0cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:19.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.847 7f2b8110f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2b7c0cf570 0x7f2b7c0cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:19.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.847 7f2b8110f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b7c0fe1b0 con 0x7f2b7c0d1860 2021-02-10T11:39:19.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.847 7f2b8110f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b7c1b0ca0 con 0x7f2b7c0d1c60 2021-02-10T11:39:19.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.847 7f2b8110f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b7c109a50 con 0x7f2b7c0cf570 2021-02-10T11:39:19.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.848 7f2b7a59c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2b7c0d1c60 0x7f2b7c0cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:19.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.848 7f2b7a59c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2b7c0d1c60 0x7f2b7c0cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55494/0 (socket says 172.21.2.101:55494) 2021-02-10T11:39:19.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.848 7f2b7a59c700 1 -- 172.21.2.101:0/2541697999 learned_addr learned my addr 172.21.2.101:0/2541697999 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:19.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.848 7f2b7a59c700 1 -- 172.21.2.101:0/2541697999 >> v2:172.21.2.108:3300/0 conn(0x7f2b7c0cf570 msgr2=0x7f2b7c0cf970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:39:19.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.848 7f2b7a59c700 1 --2- 172.21.2.101:0/2541697999 >> v2:172.21.2.108:3300/0 conn(0x7f2b7c0cf570 0x7f2b7c0cf970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:19.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.848 7f2b7a59c700 1 -- 172.21.2.101:0/2541697999 >> v1:172.21.2.108:6789/0 conn(0x7f2b7c0d1860 legacy=0x7f2b7c1b04b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:39:19.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.848 7f2b7a59c700 1 -- 172.21.2.101:0/2541697999 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7c088070 con 0x7f2b7c0d1c60 2021-02-10T11:39:19.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.848 7f2b7a59c700 1 --2- 172.21.2.101:0/2541697999 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b7c0d1c60 0x7f2b7c0cef50 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 rx=0x7f2b64006700 tx=0x7f2b64009b80).ready entity=mon.0 client_cookie=37c79428c0828b5e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:19.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.848 7f2b79d9b700 1 -- 172.21.2.101:0/2541697999 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b6400c0c0 con 0x7f2b7c0d1c60 2021-02-10T11:39:19.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.848 7f2b79d9b700 1 -- 172.21.2.101:0/2541697999 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6400c440 con 0x7f2b7c0d1c60 2021-02-10T11:39:19.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.848 7f2b79d9b700 1 -- 172.21.2.101:0/2541697999 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b6400c960 con 0x7f2b7c0d1c60 2021-02-10T11:39:19.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.848 7f2b8110f700 1 -- 172.21.2.101:0/2541697999 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b7c0d1c60 msgr2=0x7f2b7c0cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:19.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.848 7f2b8110f700 1 --2- 172.21.2.101:0/2541697999 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b7c0d1c60 0x7f2b7c0cef50 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 rx=0x7f2b64006700 tx=0x7f2b64009b80).stop 2021-02-10T11:39:19.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.849 7f2b8110f700 1 -- 172.21.2.101:0/2541697999 shutdown_connections 2021-02-10T11:39:19.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.849 7f2b8110f700 1 --2- 172.21.2.101:0/2541697999 >> v2:172.21.2.108:3300/0 conn(0x7f2b7c0cf570 0x7f2b7c0cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:19.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.849 7f2b8110f700 1 --2- 172.21.2.101:0/2541697999 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b7c0d1c60 0x7f2b7c0cef50 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:19.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.849 7f2b8110f700 1 -- 172.21.2.101:0/2541697999 >> 172.21.2.101:0/2541697999 conn(0x7f2b7c0ce410 msgr2=0x7f2b7c0d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:19.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.849 7f2b8110f700 1 -- 172.21.2.101:0/2541697999 shutdown_connections 2021-02-10T11:39:19.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.849 7f2b8110f700 1 -- 172.21.2.101:0/2541697999 wait complete. 2021-02-10T11:39:19.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.863 7f2b8110f700 1 Processor -- start 2021-02-10T11:39:19.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.863 7f2b8110f700 1 -- start start 2021-02-10T11:39:19.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.864 7f2b8110f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b7c0bfa10 0x7f2b7c0b82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:19.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.864 7f2b8110f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b7c1a4a70 0x7f2b7c094c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:19.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.864 7f2b8110f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b7c0c0410 0x7f2b7c0c0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:19.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.864 7f2b8110f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c0c3a50 con 0x7f2b7c1a4a70 2021-02-10T11:39:19.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.864 7f2b8110f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c0b88c0 con 0x7f2b7c0bfa10 2021-02-10T11:39:19.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.864 7f2b8110f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c095260 con 0x7f2b7c0c0410 2021-02-10T11:39:19.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.864 7f2b7a59c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b7c1a4a70 0x7f2b7c094c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:19.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.864 7f2b7a59c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b7c1a4a70 0x7f2b7c094c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55498/0 (socket says 172.21.2.101:55498) 2021-02-10T11:39:19.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.864 7f2b7a59c700 1 -- 172.21.2.101:0/408129859 learned_addr learned my addr 172.21.2.101:0/408129859 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:19.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.864 7f2b7a59c700 1 -- 172.21.2.101:0/408129859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b7c0c0410 msgr2=0x7f2b7c0c0810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:19.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.864 7f2b7a59c700 1 --2- 172.21.2.101:0/408129859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b7c0c0410 0x7f2b7c0c0810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:19.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.864 7f2b7a59c700 1 -- 172.21.2.101:0/408129859 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b7c0bfa10 msgr2=0x7f2b7c0b82a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:19.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.864 7f2b7a59c700 1 --2- 172.21.2.101:0/408129859 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b7c0bfa10 0x7f2b7c0b82a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:19.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.864 7f2b7a59c700 1 -- 172.21.2.101:0/408129859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7c0c3540 con 0x7f2b7c1a4a70 2021-02-10T11:39:19.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.865 7f2b7a59c700 1 --2- 172.21.2.101:0/408129859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b7c1a4a70 0x7f2b7c094c40 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 rx=0x7f2b6400b5e0 tx=0x7f2b6400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:19.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.865 7f2b737fe700 1 -- 172.21.2.101:0/408129859 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b6400bd70 con 0x7f2b7c1a4a70 2021-02-10T11:39:19.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.865 7f2b737fe700 1 -- 172.21.2.101:0/408129859 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b64002030 con 0x7f2b7c1a4a70 2021-02-10T11:39:19.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.865 7f2b737fe700 1 -- 172.21.2.101:0/408129859 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b640024d0 con 0x7f2b7c1a4a70 2021-02-10T11:39:19.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.865 7f2b8110f700 1 -- 172.21.2.101:0/408129859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7c0c3540 con 0x7f2b7c1a4a70 2021-02-10T11:39:19.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.865 7f2b8110f700 1 -- 172.21.2.101:0/408129859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7c0c1120 con 0x7f2b7c1a4a70 2021-02-10T11:39:19.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.866 7f2b737fe700 1 -- 172.21.2.101:0/408129859 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2b64000c90 con 0x7f2b7c1a4a70 2021-02-10T11:39:19.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.866 7f2b737fe700 1 --2- 172.21.2.101:0/408129859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b600186c0 0x7f2b6001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:19.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.866 7f2b737fe700 1 -- 172.21.2.101:0/408129859 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2b640325b0 con 0x7f2b7c1a4a70 2021-02-10T11:39:19.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.866 7f2b7ad9d700 1 --2- 172.21.2.101:0/408129859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b600186c0 0x7f2b6001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:19.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.866 7f2b7ad9d700 1 --2- 172.21.2.101:0/408129859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b600186c0 0x7f2b6001abc0 crc :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:19.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.897 7f2b8110f700 1 -- 172.21.2.101:0/408129859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b7c0bfee0 con 0x7f2b7c1a4a70 2021-02-10T11:39:19.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:19.900 7f2b737fe700 1 -- 172.21.2.101:0/408129859 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2b6402df10 con 0x7f2b7c1a4a70 2021-02-10T11:39:20.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.027 7f2b8110f700 1 -- 172.21.2.101:0/408129859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f2b7c0bfee0 con 0x7f2b7c1a4a70 2021-02-10T11:39:20.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.027 7f2b737fe700 1 -- 172.21.2.101:0/408129859 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v94) v1 ==== 100+0+10 (secure 0 0 0) 0x7f2b6402df10 con 0x7f2b7c1a4a70 2021-02-10T11:39:20.031 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:39:20.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.029 7f2b8110f700 1 -- 172.21.2.101:0/408129859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b600186c0 msgr2=0x7f2b6001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:20.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.029 7f2b8110f700 1 --2- 172.21.2.101:0/408129859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b600186c0 0x7f2b6001abc0 crc :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:20.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.029 7f2b8110f700 1 -- 172.21.2.101:0/408129859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b7c1a4a70 msgr2=0x7f2b7c094c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:20.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.029 7f2b8110f700 1 --2- 172.21.2.101:0/408129859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b7c1a4a70 0x7f2b7c094c40 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 rx=0x7f2b6400b5e0 tx=0x7f2b6400b980).stop 2021-02-10T11:39:20.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.029 7f2b8110f700 1 -- 172.21.2.101:0/408129859 shutdown_connections 2021-02-10T11:39:20.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.029 7f2b8110f700 1 --2- 172.21.2.101:0/408129859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b600186c0 0x7f2b6001abc0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:20.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.029 7f2b8110f700 1 --2- 172.21.2.101:0/408129859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b7c0c0410 0x7f2b7c0c0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:20.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.030 7f2b8110f700 1 --2- 172.21.2.101:0/408129859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b7c1a4a70 0x7f2b7c094c40 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:20.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.030 7f2b8110f700 1 --2- 172.21.2.101:0/408129859 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b7c0bfa10 0x7f2b7c0b82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:20.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.030 7f2b8110f700 1 -- 172.21.2.101:0/408129859 >> 172.21.2.101:0/408129859 conn(0x7f2b7c0a6a80 msgr2=0x7f2b7c0ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:20.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.030 7f2b8110f700 1 -- 172.21.2.101:0/408129859 shutdown_connections 2021-02-10T11:39:20.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.030 7f2b8110f700 1 -- 172.21.2.101:0/408129859 wait complete. 2021-02-10T11:39:20.041 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:39:20.042 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:39:20.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.449 7f815a107700 1 Processor -- start 2021-02-10T11:39:20.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.449 7f815a107700 1 -- start start 2021-02-10T11:39:20.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.449 7f815a107700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f81540d1870 0x7f81541b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:20.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.449 7f815a107700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81540f0650 0x7f81540cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:20.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.449 7f815a107700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81540f0b10 con 0x7f81541b0a10 2021-02-10T11:39:20.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.449 7f815a107700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81540e5ed0 con 0x7f81540d1870 2021-02-10T11:39:20.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.449 7f815a107700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81540d1c70 con 0x7f81540f0650 2021-02-10T11:39:20.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.450 7f81537fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f81540d1870 0x7f81541b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:20.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.450 7f81537fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f81540d1870 0x7f81541b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55506/0 (socket says 172.21.2.101:55506) 2021-02-10T11:39:20.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.450 7f81537fe700 1 -- 172.21.2.101:0/2523632334 learned_addr learned my addr 172.21.2.101:0/2523632334 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:20.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.450 7f81537fe700 1 -- 172.21.2.101:0/2523632334 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81540f0650 msgr2=0x7f81540cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:20.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.450 7f81537fe700 1 --2- 172.21.2.101:0/2523632334 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81540f0650 0x7f81540cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:20.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.450 7f81537fe700 1 -- 172.21.2.101:0/2523632334 >> v1:172.21.2.108:6789/0 conn(0x7f81541b0a10 legacy=0x7f81540cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:20.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.450 7f81537fe700 1 -- 172.21.2.101:0/2523632334 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81540d1e90 con 0x7f81540d1870 2021-02-10T11:39:20.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.450 7f81537fe700 1 --2- 172.21.2.101:0/2523632334 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81540d1870 0x7f81541b03f0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 rx=0x7f8144006700 tx=0x7f8144009b80).ready entity=mon.0 client_cookie=9c369bf399995c50 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:20.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.450 7f81527fc700 1 -- 172.21.2.101:0/2523632334 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f814400c0c0 con 0x7f81540d1870 2021-02-10T11:39:20.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.450 7f81527fc700 1 -- 172.21.2.101:0/2523632334 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814400c400 con 0x7f81540d1870 2021-02-10T11:39:20.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.450 7f81527fc700 1 -- 172.21.2.101:0/2523632334 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f814400c860 con 0x7f81540d1870 2021-02-10T11:39:20.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.450 7f815a107700 1 -- 172.21.2.101:0/2523632334 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81540d1870 msgr2=0x7f81541b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:20.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.450 7f815a107700 1 --2- 172.21.2.101:0/2523632334 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81540d1870 0x7f81541b03f0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 rx=0x7f8144006700 tx=0x7f8144009b80).stop 2021-02-10T11:39:20.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.451 7f815a107700 1 -- 172.21.2.101:0/2523632334 shutdown_connections 2021-02-10T11:39:20.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.451 7f815a107700 1 --2- 172.21.2.101:0/2523632334 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81540f0650 0x7f81540cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:20.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.451 7f815a107700 1 --2- 172.21.2.101:0/2523632334 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81540d1870 0x7f81541b03f0 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:20.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.451 7f815a107700 1 -- 172.21.2.101:0/2523632334 >> 172.21.2.101:0/2523632334 conn(0x7f81540ce450 msgr2=0x7f81540d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:20.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.451 7f815a107700 1 -- 172.21.2.101:0/2523632334 shutdown_connections 2021-02-10T11:39:20.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.451 7f815a107700 1 -- 172.21.2.101:0/2523632334 wait complete. 2021-02-10T11:39:20.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.465 7f815a107700 1 Processor -- start 2021-02-10T11:39:20.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.465 7f815a107700 1 -- start start 2021-02-10T11:39:20.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.466 7f815a107700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f815418a540 0x7f8154197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:20.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.466 7f815a107700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f815418aa50 0x7f81540b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:20.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.466 7f815a107700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81540b7df0 0x7f81540b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:20.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.466 7f815a107700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81541a79d0 con 0x7f815418a540 2021-02-10T11:39:20.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.466 7f815a107700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81541a9e20 con 0x7f815418aa50 2021-02-10T11:39:20.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.466 7f815a107700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81541aa780 con 0x7f81540b7df0 2021-02-10T11:39:20.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.466 7f81537fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f815418a540 0x7f8154197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:20.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.466 7f81537fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f815418a540 0x7f8154197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55512/0 (socket says 172.21.2.101:55512) 2021-02-10T11:39:20.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.466 7f81537fe700 1 -- 172.21.2.101:0/2080407252 learned_addr learned my addr 172.21.2.101:0/2080407252 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:20.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.466 7f81537fe700 1 -- 172.21.2.101:0/2080407252 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81540b7df0 msgr2=0x7f81540b81f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:20.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.466 7f81537fe700 1 --2- 172.21.2.101:0/2080407252 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81540b7df0 0x7f81540b81f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:20.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.466 7f81537fe700 1 -- 172.21.2.101:0/2080407252 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f815418aa50 msgr2=0x7f81540b77d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:20.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.466 7f81537fe700 1 --2- 172.21.2.101:0/2080407252 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f815418aa50 0x7f81540b77d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:20.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.466 7f81537fe700 1 -- 172.21.2.101:0/2080407252 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81541aa9a0 con 0x7f815418a540 2021-02-10T11:39:20.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.467 7f81537fe700 1 --2- 172.21.2.101:0/2080407252 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f815418a540 0x7f8154197480 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 rx=0x7f814400b580 tx=0x7f814400b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:20.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.467 7f813bfff700 1 -- 172.21.2.101:0/2080407252 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f814400bd10 con 0x7f815418a540 2021-02-10T11:39:20.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.467 7f813bfff700 1 -- 172.21.2.101:0/2080407252 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81440052e0 con 0x7f815418a540 2021-02-10T11:39:20.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.467 7f813bfff700 1 -- 172.21.2.101:0/2080407252 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8144005800 con 0x7f815418a540 2021-02-10T11:39:20.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.467 7f815a107700 1 -- 172.21.2.101:0/2080407252 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81541aa9a0 con 0x7f815418a540 2021-02-10T11:39:20.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.467 7f815a107700 1 -- 172.21.2.101:0/2080407252 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81541aaf30 con 0x7f815418a540 2021-02-10T11:39:20.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.468 7f813bfff700 1 -- 172.21.2.101:0/2080407252 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8144006030 con 0x7f815418a540 2021-02-10T11:39:20.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.468 7f813bfff700 1 --2- 172.21.2.101:0/2080407252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81340186c0 0x7f813401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:20.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.468 7f813bfff700 1 -- 172.21.2.101:0/2080407252 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8144032590 con 0x7f815418a540 2021-02-10T11:39:20.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.468 7f8152ffd700 1 --2- 172.21.2.101:0/2080407252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81340186c0 0x7f813401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:20.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.468 7f8152ffd700 1 --2- 172.21.2.101:0/2080407252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81340186c0 0x7f813401abc0 crc :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:20.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.499 7f815a107700 1 -- 172.21.2.101:0/2080407252 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81541aa250 con 0x7f815418a540 2021-02-10T11:39:20.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.502 7f813bfff700 1 -- 172.21.2.101:0/2080407252 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f814402cc80 con 0x7f815418a540 2021-02-10T11:39:20.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.630 7f815a107700 1 -- 172.21.2.101:0/2080407252 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81541aa250 con 0x7f815418a540 2021-02-10T11:39:20.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.630 7f813bfff700 1 -- 172.21.2.101:0/2080407252 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f814402cc80 con 0x7f815418a540 2021-02-10T11:39:20.634 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:39:20.635 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/4210469578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":4210469578},{"type":"v1","addr":"172.21.2.108:6833","nonce":4210469578}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:39:02.063078","modified":"2021-02-10 11:39:04.073713","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6164},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6164":{"gid":6164,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.2.101:6835/1113812290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1113812290},{"type":"v1","addr":"172.21.2.101:6835","nonce":1113812290}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-02-10T11:39:20.635 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 71 2021-02-10T11:39:20.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.632 7f815a107700 1 -- 172.21.2.101:0/2080407252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81340186c0 msgr2=0x7f813401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:20.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.632 7f815a107700 1 --2- 172.21.2.101:0/2080407252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81340186c0 0x7f813401abc0 crc :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:20.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.632 7f815a107700 1 -- 172.21.2.101:0/2080407252 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f815418a540 msgr2=0x7f8154197480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:20.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.632 7f815a107700 1 --2- 172.21.2.101:0/2080407252 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f815418a540 0x7f8154197480 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 rx=0x7f814400b580 tx=0x7f814400b920).stop 2021-02-10T11:39:20.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.632 7f815a107700 1 -- 172.21.2.101:0/2080407252 shutdown_connections 2021-02-10T11:39:20.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.632 7f815a107700 1 --2- 172.21.2.101:0/2080407252 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81340186c0 0x7f813401abc0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:20.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.632 7f815a107700 1 --2- 172.21.2.101:0/2080407252 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81540b7df0 0x7f81540b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:20.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.632 7f815a107700 1 --2- 172.21.2.101:0/2080407252 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f815418aa50 0x7f81540b77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:20.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.632 7f815a107700 1 --2- 172.21.2.101:0/2080407252 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f815418a540 0x7f8154197480 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:20.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.632 7f815a107700 1 -- 172.21.2.101:0/2080407252 >> 172.21.2.101:0/2080407252 conn(0x7f81540efc70 msgr2=0x7f81540ef410 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:20.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.633 7f815a107700 1 -- 172.21.2.101:0/2080407252 shutdown_connections 2021-02-10T11:39:20.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:20.633 7f815a107700 1 -- 172.21.2.101:0/2080407252 wait complete. 2021-02-10T11:39:20.846 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:39:02.063078', 'modified': '2021-02-10 11:39:04.073713', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6164}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6164': {'gid': 6164, 'name': 'a', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.101:6835/1113812290', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.101:6834', 'nonce': 1113812290}, {'type': 'v1', 'addr': '172.21.2.101:6835', 'nonce': 1113812290}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:39:20.846 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:39:20.846 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok status 2021-02-10T11:39:21.245 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:39:21.246 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:39:21.246 INFO:teuthology.orchestra.run.gibba001.stdout: "whoami": 0, 2021-02-10T11:39:21.246 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 6164, 2021-02-10T11:39:21.246 INFO:teuthology.orchestra.run.gibba001.stdout: "want_state": "up:active", 2021-02-10T11:39:21.246 INFO:teuthology.orchestra.run.gibba001.stdout: "state": "up:active", 2021-02-10T11:39:21.246 INFO:teuthology.orchestra.run.gibba001.stdout: "rank_uptime": 19.177411761999998, 2021-02-10T11:39:21.246 INFO:teuthology.orchestra.run.gibba001.stdout: "mdsmap_epoch": 71, 2021-02-10T11:39:21.246 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch": 94, 2021-02-10T11:39:21.246 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch_barrier": 94, 2021-02-10T11:39:21.246 INFO:teuthology.orchestra.run.gibba001.stdout: "uptime": 24.820349305000001 2021-02-10T11:39:21.246 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:39:21.250 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 6164, "want_state": "up:active", "state": "up:active", "rank_uptime": 19.177411761999998, "mdsmap_epoch": 71, "osdmap_epoch": 94, "osdmap_epoch_barrier": 94, "uptime": 24.820349305000001 } 2021-02-10T11:39:21.250 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:39:21.250 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-02-10T11:39:21.250 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:39:21.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:21.250 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:39:21.586 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:39:21.930 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:39:21.930 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:39:22.274 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:39:22.274 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:39:22.275 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:39:22.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:22.275 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:39:22.610 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:39:22.611 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2021-02-10T11:39:22.611 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:39:22.614 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:39:22.957 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:39:22.957 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:39:23.023 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:2021-02-10 11:39:23.019 7f1183c4ff80 -1 init, newargv = 0x561450885220 newargc=9 2021-02-10T11:39:23.023 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[139946]: starting ceph client 2021-02-10T11:39:23.027 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[139946]: starting fuse 2021-02-10T11:39:23.299 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:39:23.299 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:39:23.300 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:39:23.300 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:23.300 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:39:23.635 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:39:23.636 INFO:teuthology.orchestra.run.gibba001.stdout:44 2021-02-10T11:39:23.636 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2021-02-10T11:39:23.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:23.636 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:39:23.636 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:39:23.637 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:39:23.637 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:39:23.637 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:39:23.637 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:39:23.637 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:39:23.637 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-02-10T11:39:23.637 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:39:23.637 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:39:23.637 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:39:23.637 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:39:23.637 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:39:23.637 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:39:23.637 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:39:23.637 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:39:23.637 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:39:23.638 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:39:23.638 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:39:23.638 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:39:23.638 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:39:23.638 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.1")) 2021-02-10T11:39:23.638 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:39:23.999 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.1.139946.asok 2021-02-10T11:39:24.002 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.139946.asok 2021-02-10T11:39:24.002 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:24.002 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.139946.asok status 2021-02-10T11:39:24.393 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:39:24.393 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:39:24.393 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:39:24.393 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:39:24.393 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "1", 2021-02-10T11:39:24.394 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:39:24.394 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:39:24.394 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "139946", 2021-02-10T11:39:24.394 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:39:24.394 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:24.394 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:39:24.394 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:39:24.394 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 9948, 2021-02-10T11:39:24.394 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:39:24.394 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:39:24.394 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:39:24.394 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 9948 2021-02-10T11:39:24.394 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:24.395 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:39:24.395 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:24.395 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:39:24.395 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 3423734992 2021-02-10T11:39:24.395 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:24.395 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:24.395 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:39:24.395 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:24.395 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:39:24.395 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 3423734992 2021-02-10T11:39:24.395 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:24.395 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.9948 172.21.2.101:0/3423734992", 2021-02-10T11:39:24.395 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/3423734992", 2021-02-10T11:39:24.396 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:39:24.396 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 71, 2021-02-10T11:39:24.396 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 94, 2021-02-10T11:39:24.396 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:39:24.396 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:39:24.396 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:39:24.397 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:24.397 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:39:24.397 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:39:24.733 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:39:24.733 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:39:24.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:24.733 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:39:25.080 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2021-02-10T11:39:25.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.490 7f7fee66e700 1 Processor -- start 2021-02-10T11:39:25.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.490 7f7fee66e700 1 -- start start 2021-02-10T11:39:25.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.491 7f7fee66e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7fe80d1880 0x7f7fe81b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:25.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.491 7f7fee66e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fe81b0af0 0x7f7fe8108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:25.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.491 7f7fee66e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe80fe1d0 con 0x7f7fe80d1c80 2021-02-10T11:39:25.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.491 7f7fee66e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe8109a70 con 0x7f7fe80d1880 2021-02-10T11:39:25.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.491 7f7fee66e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe8109760 con 0x7f7fe81b0af0 2021-02-10T11:39:25.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.491 7f7fe77fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fe81b0af0 0x7f7fe8108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:25.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.492 7f7fe77fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fe81b0af0 0x7f7fe8108950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:48262/0 (socket says 172.21.2.101:48262) 2021-02-10T11:39:25.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.492 7f7fe77fe700 1 -- 172.21.2.101:0/1580555415 learned_addr learned my addr 172.21.2.101:0/1580555415 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:25.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.492 7f7fe7fff700 1 --2- 172.21.2.101:0/1580555415 >> v2:172.21.2.108:3300/0 conn(0x7f7fe80d1880 0x7f7fe81b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:25.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.492 7f7fe77fe700 1 -- 172.21.2.101:0/1580555415 >> v2:172.21.2.108:3300/0 conn(0x7f7fe80d1880 msgr2=0x7f7fe81b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:25.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.492 7f7fe77fe700 1 --2- 172.21.2.101:0/1580555415 >> v2:172.21.2.108:3300/0 conn(0x7f7fe80d1880 0x7f7fe81b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:25.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.492 7f7fe77fe700 1 -- 172.21.2.101:0/1580555415 >> v1:172.21.2.108:6789/0 conn(0x7f7fe80d1c80 legacy=0x7f7fe8108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:25.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.492 7f7fe77fe700 1 -- 172.21.2.101:0/1580555415 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe8088070 con 0x7f7fe81b0af0 2021-02-10T11:39:25.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.492 7f7fe7fff700 1 --2- 172.21.2.101:0/1580555415 >> v2:172.21.2.108:3300/0 conn(0x7f7fe80d1880 0x7f7fe81b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:39:25.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.492 7f7fe77fe700 1 --2- 172.21.2.101:0/1580555415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fe81b0af0 0x7f7fe8108950 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f7fdc006700 tx=0x7f7fdc009b80).ready entity=mon.2 client_cookie=31545d79ee39292a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:25.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.493 7f7fe6ffd700 1 -- 172.21.2.101:0/1580555415 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7fdc00c0c0 con 0x7f7fe81b0af0 2021-02-10T11:39:25.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.493 7f7fe6ffd700 1 -- 172.21.2.101:0/1580555415 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fdc00c440 con 0x7f7fe81b0af0 2021-02-10T11:39:25.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.493 7f7fe6ffd700 1 -- 172.21.2.101:0/1580555415 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7fdc00c960 con 0x7f7fe81b0af0 2021-02-10T11:39:25.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.493 7f7fee66e700 1 -- 172.21.2.101:0/1580555415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fe81b0af0 msgr2=0x7f7fe8108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:25.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.493 7f7fee66e700 1 --2- 172.21.2.101:0/1580555415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fe81b0af0 0x7f7fe8108950 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f7fdc006700 tx=0x7f7fdc009b80).stop 2021-02-10T11:39:25.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.493 7f7fee66e700 1 -- 172.21.2.101:0/1580555415 shutdown_connections 2021-02-10T11:39:25.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.493 7f7fee66e700 1 --2- 172.21.2.101:0/1580555415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fe81b0af0 0x7f7fe8108950 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:25.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.493 7f7fee66e700 1 --2- 172.21.2.101:0/1580555415 >> v2:172.21.2.108:3300/0 conn(0x7f7fe80d1880 0x7f7fe81b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:25.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.493 7f7fee66e700 1 -- 172.21.2.101:0/1580555415 >> 172.21.2.101:0/1580555415 conn(0x7f7fe80ce430 msgr2=0x7f7fe80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:25.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.493 7f7fee66e700 1 -- 172.21.2.101:0/1580555415 shutdown_connections 2021-02-10T11:39:25.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.493 7f7fee66e700 1 -- 172.21.2.101:0/1580555415 wait complete. 2021-02-10T11:39:25.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.508 7f7fee66e700 1 Processor -- start 2021-02-10T11:39:25.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.508 7f7fee66e700 1 -- start start 2021-02-10T11:39:25.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.508 7f7fee66e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7fe81a3570 0x7f7fe81a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:25.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.508 7f7fee66e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fe809c9f0 0x7f7fe809cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:25.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.508 7f7fee66e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fe809d410 0x7f7fe8098170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:25.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.508 7f7fee66e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe809c470 con 0x7f7fe809c9f0 2021-02-10T11:39:25.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.508 7f7fee66e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe80b2840 con 0x7f7fe81a3570 2021-02-10T11:39:25.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.508 7f7fee66e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe80b2a20 con 0x7f7fe809d410 2021-02-10T11:39:25.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.508 7f7fe77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fe809c9f0 0x7f7fe809cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:25.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.508 7f7fe77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fe809c9f0 0x7f7fe809cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55534/0 (socket says 172.21.2.101:55534) 2021-02-10T11:39:25.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.508 7f7fe77fe700 1 -- 172.21.2.101:0/90465407 learned_addr learned my addr 172.21.2.101:0/90465407 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:25.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.509 7f7fe77fe700 1 -- 172.21.2.101:0/90465407 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fe809d410 msgr2=0x7f7fe8098170 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:25.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.509 7f7fe77fe700 1 --2- 172.21.2.101:0/90465407 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fe809d410 0x7f7fe8098170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:25.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.509 7f7fe77fe700 1 -- 172.21.2.101:0/90465407 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7fe81a3570 msgr2=0x7f7fe81a3970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:25.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.509 7f7fe77fe700 1 --2- 172.21.2.101:0/90465407 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7fe81a3570 0x7f7fe81a3970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:25.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.509 7f7fe77fe700 1 -- 172.21.2.101:0/90465407 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe809d880 con 0x7f7fe809c9f0 2021-02-10T11:39:25.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.509 7f7fe77fe700 1 --2- 172.21.2.101:0/90465407 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fe809c9f0 0x7f7fe809cdf0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 rx=0x7f7fdc00b580 tx=0x7f7fdc00b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:25.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.509 7f7fe4ff9700 1 -- 172.21.2.101:0/90465407 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7fdc00bd10 con 0x7f7fe809c9f0 2021-02-10T11:39:25.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.509 7f7fe4ff9700 1 -- 172.21.2.101:0/90465407 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fdc002030 con 0x7f7fe809c9f0 2021-02-10T11:39:25.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.509 7f7fe4ff9700 1 -- 172.21.2.101:0/90465407 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7fdc002510 con 0x7f7fe809c9f0 2021-02-10T11:39:25.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.509 7f7fee66e700 1 -- 172.21.2.101:0/90465407 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe809d880 con 0x7f7fe809c9f0 2021-02-10T11:39:25.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.509 7f7fee66e700 1 -- 172.21.2.101:0/90465407 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe809d880 con 0x7f7fe809c9f0 2021-02-10T11:39:25.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.510 7f7fe4ff9700 1 -- 172.21.2.101:0/90465407 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7fdc002210 con 0x7f7fe809c9f0 2021-02-10T11:39:25.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.510 7f7fe4ff9700 1 --2- 172.21.2.101:0/90465407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7fc80186d0 0x7f7fc801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:25.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.510 7f7fe4ff9700 1 -- 172.21.2.101:0/90465407 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7fdc032620 con 0x7f7fe809c9f0 2021-02-10T11:39:25.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.510 7f7fe7fff700 1 --2- 172.21.2.101:0/90465407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7fc80186d0 0x7f7fc801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:25.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.510 7f7fe7fff700 1 --2- 172.21.2.101:0/90465407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7fc80186d0 0x7f7fc801abd0 crc :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:25.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.541 7f7fee66e700 1 -- 172.21.2.101:0/90465407 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe8082a70 con 0x7f7fe809c9f0 2021-02-10T11:39:25.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.543 7f7fe4ff9700 1 -- 172.21.2.101:0/90465407 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7fdc002240 con 0x7f7fe809c9f0 2021-02-10T11:39:25.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.679 7f7fee66e700 1 -- 172.21.2.101:0/90465407 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7fe8082a70 con 0x7f7fe809c9f0 2021-02-10T11:39:25.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.733 7f7fe4ff9700 1 -- 172.21.2.101:0/90465407 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v72) v1 ==== 136+0+0 (secure 0 0 0) 0x7f7fdc002030 con 0x7f7fe809c9f0 2021-02-10T11:39:25.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.745 7f7fee66e700 1 -- 172.21.2.101:0/90465407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7fc80186d0 msgr2=0x7f7fc801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:25.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.745 7f7fee66e700 1 --2- 172.21.2.101:0/90465407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7fc80186d0 0x7f7fc801abd0 crc :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:25.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.745 7f7fee66e700 1 -- 172.21.2.101:0/90465407 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fe809c9f0 msgr2=0x7f7fe809cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:25.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.745 7f7fee66e700 1 --2- 172.21.2.101:0/90465407 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fe809c9f0 0x7f7fe809cdf0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 rx=0x7f7fdc00b580 tx=0x7f7fdc00b920).stop 2021-02-10T11:39:25.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.745 7f7fee66e700 1 -- 172.21.2.101:0/90465407 shutdown_connections 2021-02-10T11:39:25.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.745 7f7fee66e700 1 --2- 172.21.2.101:0/90465407 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7fc80186d0 0x7f7fc801abd0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:25.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.745 7f7fee66e700 1 --2- 172.21.2.101:0/90465407 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7fe809d410 0x7f7fe8098170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:25.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.745 7f7fee66e700 1 --2- 172.21.2.101:0/90465407 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7fe809c9f0 0x7f7fe809cdf0 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:25.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.745 7f7fee66e700 1 --2- 172.21.2.101:0/90465407 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7fe81a3570 0x7f7fe81a3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:25.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.745 7f7fee66e700 1 -- 172.21.2.101:0/90465407 >> 172.21.2.101:0/90465407 conn(0x7f7fe80c2cd0 msgr2=0x7f7fe80b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:25.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.745 7f7fee66e700 1 -- 172.21.2.101:0/90465407 shutdown_connections 2021-02-10T11:39:25.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:25.745 7f7fee66e700 1 -- 172.21.2.101:0/90465407 wait complete. 2021-02-10T11:39:25.757 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs volume ls 2021-02-10T11:39:26.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.164 7f00783db700 1 Processor -- start 2021-02-10T11:39:26.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.164 7f00783db700 1 -- start start 2021-02-10T11:39:26.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.165 7f00783db700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00700d1870 0x7f00701b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:26.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.165 7f00783db700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f00700f0650 0x7f00700cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:26.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.165 7f00783db700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00700f0b10 con 0x7f00701b0a10 2021-02-10T11:39:26.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.165 7f00783db700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00700fe230 con 0x7f00700f0650 2021-02-10T11:39:26.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.165 7f00783db700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00700e5ed0 con 0x7f00700d1870 2021-02-10T11:39:26.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.165 7f00759f6700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f00701b0a10 0x7f00700cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56036/0 (socket says 172.21.2.101:56036) 2021-02-10T11:39:26.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.165 7f00759f6700 1 -- 172.21.2.101:0/4087630429 learned_addr learned my addr 172.21.2.101:0/4087630429 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:26.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.166 7f00751f5700 1 -- 172.21.2.101:0/4087630429 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1641202561 0 0) 0x7f006c001180 con 0x7f00701b0a10 2021-02-10T11:39:26.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.166 7f00751f5700 1 -- 172.21.2.101:0/4087630429 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f005c002fe0 con 0x7f00701b0a10 2021-02-10T11:39:26.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.166 7f00769f8700 1 --2- 172.21.2.101:0/4087630429 >> v2:172.21.2.108:3300/0 conn(0x7f00700f0650 0x7f00700cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:26.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.166 7f00761f7700 1 --2- 172.21.2.101:0/4087630429 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00700d1870 0x7f00701b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:26.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.166 7f00751f5700 1 -- 172.21.2.101:0/4087630429 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2256025254 0 0) 0x7f006c001180 con 0x7f00701b0a10 2021-02-10T11:39:26.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.166 7f00751f5700 1 -- 172.21.2.101:0/4087630429 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00700d1870 msgr2=0x7f00701b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:26.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.166 7f00751f5700 1 --2- 172.21.2.101:0/4087630429 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00700d1870 0x7f00701b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:26.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.166 7f00751f5700 1 -- 172.21.2.101:0/4087630429 >> v2:172.21.2.108:3300/0 conn(0x7f00700f0650 msgr2=0x7f00700cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:26.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.166 7f00751f5700 1 --2- 172.21.2.101:0/4087630429 >> v2:172.21.2.108:3300/0 conn(0x7f00700f0650 0x7f00700cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:26.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.166 7f00751f5700 1 -- 172.21.2.101:0/4087630429 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00700f20a0 con 0x7f00701b0a10 2021-02-10T11:39:26.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.166 7f00751f5700 1 -- 172.21.2.101:0/4087630429 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f006c001680 con 0x7f00701b0a10 2021-02-10T11:39:26.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.166 7f00751f5700 1 -- 172.21.2.101:0/4087630429 >> v1:172.21.2.101:6789/0 conn(0x7f00701b0a10 legacy=0x7f00700cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:26.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.166 7f00751f5700 1 --2- 172.21.2.101:0/4087630429 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f005c003b10 0x7f005c005f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:26.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.166 7f00751f5700 1 --2- 172.21.2.101:0/4087630429 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f005c04a780 0x7f005c04cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:26.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.166 7f00751f5700 1 --2- 172.21.2.101:0/4087630429 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f005c04d500 0x7f005c04fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:26.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.166 7f00751f5700 1 -- 172.21.2.101:0/4087630429 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005c0502d0 con 0x7f005c04d500 2021-02-10T11:39:26.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.166 7f00751f5700 1 -- 172.21.2.101:0/4087630429 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005c0504f0 con 0x7f005c04a780 2021-02-10T11:39:26.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.166 7f00751f5700 1 -- 172.21.2.101:0/4087630429 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005c050710 con 0x7f005c003b10 2021-02-10T11:39:26.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.166 7f00769f8700 1 --2- 172.21.2.101:0/4087630429 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f005c04d500 0x7f005c04fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:26.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.166 7f00761f7700 1 --2- 172.21.2.101:0/4087630429 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f005c003b10 0x7f005c005f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:26.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.166 7f00759f6700 1 --2- 172.21.2.101:0/4087630429 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f005c04a780 0x7f005c04cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:26.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.167 7f00769f8700 1 -- 172.21.2.101:0/4087630429 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f005c003b10 msgr2=0x7f005c005f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:26.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.167 7f00769f8700 1 --2- 172.21.2.101:0/4087630429 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f005c003b10 0x7f005c005f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:26.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.167 7f00769f8700 1 -- 172.21.2.101:0/4087630429 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f005c04a780 msgr2=0x7f005c04cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:26.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.167 7f00769f8700 1 --2- 172.21.2.101:0/4087630429 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f005c04a780 0x7f005c04cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:26.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.167 7f00769f8700 1 -- 172.21.2.101:0/4087630429 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005c0509b0 con 0x7f005c04d500 2021-02-10T11:39:26.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.167 7f00761f7700 1 --2- 172.21.2.101:0/4087630429 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f005c003b10 0x7f005c005f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:39:26.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.167 7f00769f8700 1 --2- 172.21.2.101:0/4087630429 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f005c04d500 0x7f005c04fa20 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 rx=0x7f0068006a40 tx=0x7f006800c0c0).ready entity=mon.0 client_cookie=78b0584c202a6fa8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:26.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.167 7f00751f5700 1 -- 172.21.2.101:0/4087630429 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f006800c4b0 con 0x7f005c04d500 2021-02-10T11:39:26.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.167 7f00751f5700 1 -- 172.21.2.101:0/4087630429 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006800c770 con 0x7f005c04d500 2021-02-10T11:39:26.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.167 7f00751f5700 1 -- 172.21.2.101:0/4087630429 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f006800cc90 con 0x7f005c04d500 2021-02-10T11:39:26.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.167 7f00783db700 1 -- 172.21.2.101:0/4087630429 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f005c04d500 msgr2=0x7f005c04fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:26.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.167 7f00783db700 1 --2- 172.21.2.101:0/4087630429 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f005c04d500 0x7f005c04fa20 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 rx=0x7f0068006a40 tx=0x7f006800c0c0).stop 2021-02-10T11:39:26.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.167 7f00769f8700 1 -- 172.21.2.101:0/4087630429 reap_dead start 2021-02-10T11:39:26.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.168 7f00783db700 1 -- 172.21.2.101:0/4087630429 shutdown_connections 2021-02-10T11:39:26.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.168 7f00783db700 1 -- 172.21.2.101:0/4087630429 >> 172.21.2.101:0/4087630429 conn(0x7f00700ce450 msgr2=0x7f00700d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:26.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.168 7f00783db700 1 -- 172.21.2.101:0/4087630429 shutdown_connections 2021-02-10T11:39:26.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.168 7f00783db700 1 -- 172.21.2.101:0/4087630429 wait complete. 2021-02-10T11:39:26.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.183 7f00783db700 1 Processor -- start 2021-02-10T11:39:26.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.183 7f00783db700 1 -- start start 2021-02-10T11:39:26.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.183 7f00783db700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f007019d270 0x7f00700a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:26.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.183 7f00783db700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00700aab90 0x7f00700aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:26.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.183 7f00783db700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f00700ee5b0 0x7f00700ee9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:26.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.183 7f00783db700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00700aba30 con 0x7f007019d270 2021-02-10T11:39:26.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.183 7f00783db700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00700edd20 con 0x7f00700ee5b0 2021-02-10T11:39:26.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.183 7f00783db700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00700ab5b0 con 0x7f00700aab90 2021-02-10T11:39:26.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.184 7f00761f7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f007019d270 0x7f00700a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:26.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.184 7f00761f7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f007019d270 0x7f00700a22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55556/0 (socket says 172.21.2.101:55556) 2021-02-10T11:39:26.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.184 7f00761f7700 1 -- 172.21.2.101:0/588247996 learned_addr learned my addr 172.21.2.101:0/588247996 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:26.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.184 7f00761f7700 1 -- 172.21.2.101:0/588247996 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00700aab90 msgr2=0x7f00700aaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:26.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.184 7f00761f7700 1 --2- 172.21.2.101:0/588247996 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00700aab90 0x7f00700aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:26.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.184 7f00761f7700 1 -- 172.21.2.101:0/588247996 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f00700ee5b0 msgr2=0x7f00700ee9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:26.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.184 7f00761f7700 1 --2- 172.21.2.101:0/588247996 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f00700ee5b0 0x7f00700ee9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:26.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.184 7f00761f7700 1 -- 172.21.2.101:0/588247996 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00700a28d0 con 0x7f007019d270 2021-02-10T11:39:26.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.184 7f00761f7700 1 --2- 172.21.2.101:0/588247996 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f007019d270 0x7f00700a22b0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 rx=0x7f006400b4f0 tx=0x7f006400b890).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:26.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.184 7f0062ffd700 1 -- 172.21.2.101:0/588247996 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0064000c90 con 0x7f007019d270 2021-02-10T11:39:26.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.184 7f0062ffd700 1 -- 172.21.2.101:0/588247996 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006400bc80 con 0x7f007019d270 2021-02-10T11:39:26.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.184 7f0062ffd700 1 -- 172.21.2.101:0/588247996 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f00640022b0 con 0x7f007019d270 2021-02-10T11:39:26.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.184 7f00783db700 1 -- 172.21.2.101:0/588247996 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00700a28d0 con 0x7f007019d270 2021-02-10T11:39:26.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.184 7f00783db700 1 -- 172.21.2.101:0/588247996 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00700a28d0 con 0x7f007019d270 2021-02-10T11:39:26.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.185 7f0062ffd700 1 -- 172.21.2.101:0/588247996 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0064000c90 con 0x7f007019d270 2021-02-10T11:39:26.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.185 7f0062ffd700 1 --2- 172.21.2.101:0/588247996 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f00580186c0 0x7f005801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:26.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.185 7f0062ffd700 1 -- 172.21.2.101:0/588247996 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0064036960 con 0x7f007019d270 2021-02-10T11:39:26.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.185 7f00759f6700 1 --2- 172.21.2.101:0/588247996 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f00580186c0 0x7f005801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:26.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.186 7f00759f6700 1 --2- 172.21.2.101:0/588247996 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f00580186c0 0x7f005801abc0 crc :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:26.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.215 7f00783db700 1 -- 172.21.2.101:0/588247996 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00700f0b10 con 0x7f007019d270 2021-02-10T11:39:26.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.218 7f0062ffd700 1 -- 172.21.2.101:0/588247996 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0064006b00 con 0x7f007019d270 2021-02-10T11:39:26.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.353 7f00783db700 1 -- 172.21.2.101:0/588247996 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs volume ls", "target": ["mgr", ""]}) v1 -- 0x7f00700f0b10 con 0x7f00580186c0 2021-02-10T11:39:26.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.353 7f0062ffd700 1 -- 172.21.2.101:0/588247996 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7f006c0073f0 con 0x7f00580186c0 2021-02-10T11:39:26.357 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:39:26.357 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:26.357 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "cephfs" 2021-02-10T11:39:26.357 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:26.357 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:39:26.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.355 7f00783db700 1 -- 172.21.2.101:0/588247996 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f00580186c0 msgr2=0x7f005801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:26.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.355 7f00783db700 1 --2- 172.21.2.101:0/588247996 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f00580186c0 0x7f005801abc0 crc :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:26.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.355 7f00783db700 1 -- 172.21.2.101:0/588247996 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f007019d270 msgr2=0x7f00700a22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:26.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.355 7f00783db700 1 --2- 172.21.2.101:0/588247996 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f007019d270 0x7f00700a22b0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 rx=0x7f006400b4f0 tx=0x7f006400b890).stop 2021-02-10T11:39:26.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.355 7f00783db700 1 -- 172.21.2.101:0/588247996 shutdown_connections 2021-02-10T11:39:26.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.355 7f00783db700 1 --2- 172.21.2.101:0/588247996 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f00580186c0 0x7f005801abc0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:26.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.355 7f00783db700 1 --2- 172.21.2.101:0/588247996 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00700aab90 0x7f00700aaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:26.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.355 7f00783db700 1 --2- 172.21.2.101:0/588247996 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f00700ee5b0 0x7f00700ee9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:26.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.355 7f00783db700 1 --2- 172.21.2.101:0/588247996 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f007019d270 0x7f00700a22b0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:26.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.355 7f00783db700 1 -- 172.21.2.101:0/588247996 >> 172.21.2.101:0/588247996 conn(0x7f00700c3e70 msgr2=0x7f00700a7060 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:26.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.355 7f00783db700 1 -- 172.21.2.101:0/588247996 shutdown_connections 2021-02-10T11:39:26.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.355 7f00783db700 1 -- 172.21.2.101:0/588247996 wait complete. 2021-02-10T11:39:26.367 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:39:26.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.779 7fc4316fe700 1 Processor -- start 2021-02-10T11:39:26.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.779 7fc4316fe700 1 -- start start 2021-02-10T11:39:26.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.779 7fc4316fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc42c0d1870 0x7fc42c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:26.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.779 7fc4316fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc42c1b0ae0 0x7fc42c108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:26.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.779 7fc4316fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc42c0d1c70 0x7fc42c108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:26.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.780 7fc4316fe700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc42c0fe1c0 con 0x7fc42c1b0ae0 2021-02-10T11:39:26.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.780 7fc4316fe700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc42c109a60 con 0x7fc42c0d1870 2021-02-10T11:39:26.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.780 7fc4316fe700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c088070 con 0x7fc42c0d1c70 2021-02-10T11:39:26.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.780 7fc42aabe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc42c1b0ae0 0x7fc42c108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:26.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.780 7fc42aabe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc42c1b0ae0 0x7fc42c108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55560/0 (socket says 172.21.2.101:55560) 2021-02-10T11:39:26.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.780 7fc42aabe700 1 -- 172.21.2.101:0/3220104116 learned_addr learned my addr 172.21.2.101:0/3220104116 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:26.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.780 7fc42b2bf700 1 --2- 172.21.2.101:0/3220104116 >> v2:172.21.2.108:3300/0 conn(0x7fc42c0d1870 0x7fc42c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:26.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.780 7fc42bac0700 1 --2- 172.21.2.101:0/3220104116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc42c0d1c70 0x7fc42c108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:26.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.780 7fc42aabe700 1 -- 172.21.2.101:0/3220104116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc42c0d1c70 msgr2=0x7fc42c108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:26.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.780 7fc42aabe700 1 --2- 172.21.2.101:0/3220104116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc42c0d1c70 0x7fc42c108f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:26.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.780 7fc42aabe700 1 -- 172.21.2.101:0/3220104116 >> v2:172.21.2.108:3300/0 conn(0x7fc42c0d1870 msgr2=0x7fc42c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:26.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.780 7fc42aabe700 1 --2- 172.21.2.101:0/3220104116 >> v2:172.21.2.108:3300/0 conn(0x7fc42c0d1870 0x7fc42c1b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:26.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.780 7fc42aabe700 1 -- 172.21.2.101:0/3220104116 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc42c109580 con 0x7fc42c1b0ae0 2021-02-10T11:39:26.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.780 7fc42bac0700 1 --2- 172.21.2.101:0/3220104116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc42c0d1c70 0x7fc42c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:39:26.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.780 7fc42b2bf700 1 --2- 172.21.2.101:0/3220104116 >> v2:172.21.2.108:3300/0 conn(0x7fc42c0d1870 0x7fc42c1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:39:26.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.780 7fc42aabe700 1 --2- 172.21.2.101:0/3220104116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc42c1b0ae0 0x7fc42c108940 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 rx=0x7fc420006700 tx=0x7fc420009b80).ready entity=mon.0 client_cookie=869202031d3e7372 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:26.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.781 7fc42a2bd700 1 -- 172.21.2.101:0/3220104116 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc420006340 con 0x7fc42c1b0ae0 2021-02-10T11:39:26.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.781 7fc42a2bd700 1 -- 172.21.2.101:0/3220104116 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42000c0c0 con 0x7fc42c1b0ae0 2021-02-10T11:39:26.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.781 7fc42a2bd700 1 -- 172.21.2.101:0/3220104116 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc42000c5e0 con 0x7fc42c1b0ae0 2021-02-10T11:39:26.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.781 7fc4316fe700 1 -- 172.21.2.101:0/3220104116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc42c1b0ae0 msgr2=0x7fc42c108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:26.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.781 7fc4316fe700 1 --2- 172.21.2.101:0/3220104116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc42c1b0ae0 0x7fc42c108940 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 rx=0x7fc420006700 tx=0x7fc420009b80).stop 2021-02-10T11:39:26.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.782 7fc4316fe700 1 -- 172.21.2.101:0/3220104116 shutdown_connections 2021-02-10T11:39:26.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.782 7fc4316fe700 1 --2- 172.21.2.101:0/3220104116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc42c0d1c70 0x7fc42c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:26.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.782 7fc4316fe700 1 --2- 172.21.2.101:0/3220104116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc42c1b0ae0 0x7fc42c108940 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:26.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.782 7fc4316fe700 1 --2- 172.21.2.101:0/3220104116 >> v2:172.21.2.108:3300/0 conn(0x7fc42c0d1870 0x7fc42c1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:26.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.782 7fc4316fe700 1 -- 172.21.2.101:0/3220104116 >> 172.21.2.101:0/3220104116 conn(0x7fc42c0ce420 msgr2=0x7fc42c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:26.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.782 7fc4316fe700 1 -- 172.21.2.101:0/3220104116 shutdown_connections 2021-02-10T11:39:26.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.782 7fc4316fe700 1 -- 172.21.2.101:0/3220104116 wait complete. 2021-02-10T11:39:26.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.796 7fc4316fe700 1 Processor -- start 2021-02-10T11:39:26.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.797 7fc4316fe700 1 -- start start 2021-02-10T11:39:26.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.797 7fc4316fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc42c085bc0 0x7fc42c0863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:26.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.797 7fc4316fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc42c0954c0 0x7fc42c096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:26.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.797 7fc4316fe700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc42c0958c0 0x7fc42c096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:26.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.797 7fc4316fe700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c095d80 con 0x7fc42c085bc0 2021-02-10T11:39:26.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.797 7fc4316fe700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c0bfa60 con 0x7fc42c0958c0 2021-02-10T11:39:26.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.797 7fc4316fe700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c085fc0 con 0x7fc42c0954c0 2021-02-10T11:39:26.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.797 7fc42b2bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc42c085bc0 0x7fc42c0863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:26.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.797 7fc42b2bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc42c085bc0 0x7fc42c0863e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55564/0 (socket says 172.21.2.101:55564) 2021-02-10T11:39:26.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.797 7fc42b2bf700 1 -- 172.21.2.101:0/1155562495 learned_addr learned my addr 172.21.2.101:0/1155562495 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:26.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.798 7fc42b2bf700 1 -- 172.21.2.101:0/1155562495 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc42c0954c0 msgr2=0x7fc42c096300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:26.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.798 7fc42b2bf700 1 --2- 172.21.2.101:0/1155562495 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc42c0954c0 0x7fc42c096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:26.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.798 7fc42b2bf700 1 -- 172.21.2.101:0/1155562495 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc42c0958c0 msgr2=0x7fc42c096920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:26.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.798 7fc42b2bf700 1 --2- 172.21.2.101:0/1155562495 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc42c0958c0 0x7fc42c096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:26.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.798 7fc42b2bf700 1 -- 172.21.2.101:0/1155562495 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc42c086c10 con 0x7fc42c085bc0 2021-02-10T11:39:26.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.798 7fc42b2bf700 1 --2- 172.21.2.101:0/1155562495 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc42c085bc0 0x7fc42c0863e0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 rx=0x7fc41c006f20 tx=0x7fc41c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:26.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.798 7fc413fff700 1 -- 172.21.2.101:0/1155562495 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc41c00bd80 con 0x7fc42c085bc0 2021-02-10T11:39:26.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.798 7fc413fff700 1 -- 172.21.2.101:0/1155562495 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41c000c90 con 0x7fc42c085bc0 2021-02-10T11:39:26.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.798 7fc413fff700 1 -- 172.21.2.101:0/1155562495 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc41c002220 con 0x7fc42c085bc0 2021-02-10T11:39:26.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.798 7fc4316fe700 1 -- 172.21.2.101:0/1155562495 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc42c086c10 con 0x7fc42c085bc0 2021-02-10T11:39:26.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.798 7fc4316fe700 1 -- 172.21.2.101:0/1155562495 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc42c097230 con 0x7fc42c085bc0 2021-02-10T11:39:26.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.799 7fc413fff700 1 -- 172.21.2.101:0/1155562495 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc41c00a000 con 0x7fc42c085bc0 2021-02-10T11:39:26.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.799 7fc413fff700 1 --2- 172.21.2.101:0/1155562495 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc40c0186c0 0x7fc40c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:26.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.799 7fc413fff700 1 -- 172.21.2.101:0/1155562495 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc41c032560 con 0x7fc42c085bc0 2021-02-10T11:39:26.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.799 7fc42aabe700 1 --2- 172.21.2.101:0/1155562495 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc40c0186c0 0x7fc40c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:26.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.799 7fc42aabe700 1 --2- 172.21.2.101:0/1155562495 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc40c0186c0 0x7fc40c01abc0 crc :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:26.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.829 7fc4316fe700 1 -- 172.21.2.101:0/1155562495 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc42c0fe1c0 con 0x7fc42c085bc0 2021-02-10T11:39:26.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.832 7fc413fff700 1 -- 172.21.2.101:0/1155562495 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc41c0009b0 con 0x7fc42c085bc0 2021-02-10T11:39:26.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.982 7fc4316fe700 1 -- 172.21.2.101:0/1155562495 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc42c0fe1c0 con 0x7fc42c085bc0 2021-02-10T11:39:26.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.983 7fc413fff700 1 -- 172.21.2.101:0/1155562495 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc41c00a000 con 0x7fc42c085bc0 2021-02-10T11:39:26.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.985 7fc4316fe700 1 -- 172.21.2.101:0/1155562495 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc40c0186c0 msgr2=0x7fc40c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:26.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.985 7fc4316fe700 1 --2- 172.21.2.101:0/1155562495 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc40c0186c0 0x7fc40c01abc0 crc :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:26.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.985 7fc4316fe700 1 -- 172.21.2.101:0/1155562495 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc42c085bc0 msgr2=0x7fc42c0863e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:26.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.985 7fc4316fe700 1 --2- 172.21.2.101:0/1155562495 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc42c085bc0 0x7fc42c0863e0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 rx=0x7fc41c006f20 tx=0x7fc41c00b990).stop 2021-02-10T11:39:26.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.985 7fc4316fe700 1 -- 172.21.2.101:0/1155562495 shutdown_connections 2021-02-10T11:39:26.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.985 7fc4316fe700 1 --2- 172.21.2.101:0/1155562495 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc40c0186c0 0x7fc40c01abc0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:26.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.985 7fc4316fe700 1 --2- 172.21.2.101:0/1155562495 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc42c0954c0 0x7fc42c096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:26.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.985 7fc4316fe700 1 --2- 172.21.2.101:0/1155562495 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc42c0958c0 0x7fc42c096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:26.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.985 7fc4316fe700 1 --2- 172.21.2.101:0/1155562495 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc42c085bc0 0x7fc42c0863e0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:26.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.985 7fc4316fe700 1 -- 172.21.2.101:0/1155562495 >> 172.21.2.101:0/1155562495 conn(0x7fc42c0bef00 msgr2=0x7fc42c09d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:26.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.985 7fc4316fe700 1 -- 172.21.2.101:0/1155562495 shutdown_connections 2021-02-10T11:39:26.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:26.985 7fc4316fe700 1 -- 172.21.2.101:0/1155562495 wait complete. 2021-02-10T11:39:26.997 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolumegroup create cephfs group_0000000000112255 2021-02-10T11:39:27.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.408 7f5a8262e700 1 Processor -- start 2021-02-10T11:39:27.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.408 7f5a8262e700 1 -- start start 2021-02-10T11:39:27.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.409 7f5a8262e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a7c0d1870 0x7f5a7c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:27.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.409 7f5a8262e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7c109a60 con 0x7f5a7c1b0ae0 2021-02-10T11:39:27.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.409 7f5a8262e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7c088070 con 0x7f5a7c0cff70 2021-02-10T11:39:27.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.409 7f5a8262e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c0e5ef0 con 0x7f5a7c0d1870 2021-02-10T11:39:27.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.409 7f5a7b7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f5a7c1b0ae0 0x7f5a7c108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56066/0 (socket says 172.21.2.101:56066) 2021-02-10T11:39:27.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.409 7f5a7b7fe700 1 -- 172.21.2.101:0/2755285200 learned_addr learned my addr 172.21.2.101:0/2755285200 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:27.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.409 7f5a7bfff700 1 --2- 172.21.2.101:0/2755285200 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a7c0d1870 0x7f5a7c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:27.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.409 7f5a7affd700 1 -- 172.21.2.101:0/2755285200 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4112624939 0 0) 0x7f5a700010b0 con 0x7f5a7c1b0ae0 2021-02-10T11:39:27.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.409 7f5a7affd700 1 -- 172.21.2.101:0/2755285200 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a64002fe0 con 0x7f5a7c1b0ae0 2021-02-10T11:39:27.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.410 7f5a7bfff700 1 -- 172.21.2.101:0/2755285200 >> v1:172.21.2.108:6789/0 conn(0x7f5a7c0cff70 legacy=0x7f5a7c109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:27.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.410 7f5a7bfff700 1 -- 172.21.2.101:0/2755285200 >> v1:172.21.2.101:6789/0 conn(0x7f5a7c1b0ae0 legacy=0x7f5a7c108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:27.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.410 7f5a7bfff700 1 -- 172.21.2.101:0/2755285200 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7c0d1c70 con 0x7f5a7c0d1870 2021-02-10T11:39:27.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.410 7f5a7bfff700 1 --2- 172.21.2.101:0/2755285200 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a7c0d1870 0x7f5a7c1b04c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f5a6c006650 tx=0x7f5a6c009a80).ready entity=mon.2 client_cookie=830660b223555b65 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:27.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.410 7f5a7affd700 1 -- 172.21.2.101:0/2755285200 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5a6c00c0c0 con 0x7f5a7c0d1870 2021-02-10T11:39:27.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.410 7f5a7affd700 1 -- 172.21.2.101:0/2755285200 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6c00c340 con 0x7f5a7c0d1870 2021-02-10T11:39:27.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.410 7f5a7affd700 1 -- 172.21.2.101:0/2755285200 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5a6c00c860 con 0x7f5a7c0d1870 2021-02-10T11:39:27.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.410 7f5a8262e700 1 -- 172.21.2.101:0/2755285200 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a7c0d1870 msgr2=0x7f5a7c1b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:27.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.410 7f5a8262e700 1 --2- 172.21.2.101:0/2755285200 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a7c0d1870 0x7f5a7c1b04c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f5a6c006650 tx=0x7f5a6c009a80).stop 2021-02-10T11:39:27.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.411 7f5a8262e700 1 -- 172.21.2.101:0/2755285200 shutdown_connections 2021-02-10T11:39:27.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.411 7f5a8262e700 1 --2- 172.21.2.101:0/2755285200 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a7c0d1870 0x7f5a7c1b04c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:27.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.411 7f5a8262e700 1 -- 172.21.2.101:0/2755285200 >> 172.21.2.101:0/2755285200 conn(0x7f5a7c0ce420 msgr2=0x7f5a7c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:27.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.411 7f5a8262e700 1 -- 172.21.2.101:0/2755285200 shutdown_connections 2021-02-10T11:39:27.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.411 7f5a8262e700 1 -- 172.21.2.101:0/2755285200 wait complete. 2021-02-10T11:39:27.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.426 7f5a8262e700 1 Processor -- start 2021-02-10T11:39:27.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.426 7f5a8262e700 1 -- start start 2021-02-10T11:39:27.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.426 7f5a8262e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5a7c18f9d0 0x7f5a7c084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:27.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.426 7f5a8262e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a7c0c23e0 0x7f5a7c1a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:27.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.426 7f5a8262e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a7c0c27e0 0x7f5a7c0859f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:27.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.426 7f5a8262e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c0854f0 con 0x7f5a7c0c23e0 2021-02-10T11:39:27.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.426 7f5a8262e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c18fdd0 con 0x7f5a7c18f9d0 2021-02-10T11:39:27.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.426 7f5a8262e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c1a4ad0 con 0x7f5a7c0c27e0 2021-02-10T11:39:27.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.426 7f5a7b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a7c0c23e0 0x7f5a7c1a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:27.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.426 7f5a7b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a7c0c23e0 0x7f5a7c1a44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55576/0 (socket says 172.21.2.101:55576) 2021-02-10T11:39:27.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.426 7f5a80c4b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a7c0c27e0 0x7f5a7c0859f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:27.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.426 7f5a7b7fe700 1 -- 172.21.2.101:0/4238162107 learned_addr learned my addr 172.21.2.101:0/4238162107 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:27.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.427 7f5a7bfff700 1 --2- 172.21.2.101:0/4238162107 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5a7c18f9d0 0x7f5a7c084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:27.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.427 7f5a7b7fe700 1 -- 172.21.2.101:0/4238162107 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a7c0c27e0 msgr2=0x7f5a7c0859f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:27.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.427 7f5a7b7fe700 1 --2- 172.21.2.101:0/4238162107 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a7c0c27e0 0x7f5a7c0859f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:27.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.427 7f5a7b7fe700 1 -- 172.21.2.101:0/4238162107 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5a7c18f9d0 msgr2=0x7f5a7c084cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:27.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.427 7f5a7b7fe700 1 --2- 172.21.2.101:0/4238162107 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5a7c18f9d0 0x7f5a7c084cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:27.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.427 7f5a7b7fe700 1 -- 172.21.2.101:0/4238162107 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7c1a4cf0 con 0x7f5a7c0c23e0 2021-02-10T11:39:27.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.427 7f5a7b7fe700 1 --2- 172.21.2.101:0/4238162107 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a7c0c23e0 0x7f5a7c1a44b0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 rx=0x7f5a70001c60 tx=0x7f5a7000a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:27.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.427 7f5a78ff9700 1 -- 172.21.2.101:0/4238162107 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5a7000acc0 con 0x7f5a7c0c23e0 2021-02-10T11:39:27.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.427 7f5a78ff9700 1 -- 172.21.2.101:0/4238162107 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a70004140 con 0x7f5a7c0c23e0 2021-02-10T11:39:27.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.427 7f5a78ff9700 1 -- 172.21.2.101:0/4238162107 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5a70004510 con 0x7f5a7c0c23e0 2021-02-10T11:39:27.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.427 7f5a8262e700 1 -- 172.21.2.101:0/4238162107 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a7c1a4cf0 con 0x7f5a7c0c23e0 2021-02-10T11:39:27.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.427 7f5a8262e700 1 -- 172.21.2.101:0/4238162107 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a7c1a4cf0 con 0x7f5a7c0c23e0 2021-02-10T11:39:27.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.428 7f5a78ff9700 1 -- 172.21.2.101:0/4238162107 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5a70004320 con 0x7f5a7c0c23e0 2021-02-10T11:39:27.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.428 7f5a78ff9700 1 --2- 172.21.2.101:0/4238162107 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5a5c018730 0x7f5a5c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:27.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.428 7f5a78ff9700 1 -- 172.21.2.101:0/4238162107 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5a700327e0 con 0x7f5a7c0c23e0 2021-02-10T11:39:27.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.428 7f5a7bfff700 1 --2- 172.21.2.101:0/4238162107 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5a5c018730 0x7f5a5c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:27.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.428 7f5a7bfff700 1 --2- 172.21.2.101:0/4238162107 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5a5c018730 0x7f5a5c01abc0 crc :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:27.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.459 7f5a8262e700 1 -- 172.21.2.101:0/4238162107 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a7c082a70 con 0x7f5a7c0c23e0 2021-02-10T11:39:27.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.461 7f5a78ff9700 1 -- 172.21.2.101:0/4238162107 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5a70000e20 con 0x7f5a7c0c23e0 2021-02-10T11:39:27.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.588 7f5a8262e700 1 -- 172.21.2.101:0/4238162107 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "target": ["mgr", ""], "group_name": "group_0000000000112255"}) v1 -- 0x7f5a7c18fdd0 con 0x7f5a5c018730 2021-02-10T11:39:27.595 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:39:27.591 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:39:27.595 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:39:27.591 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:39:27.595 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:39:27.591 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:39:27.595 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:39:27.591 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:39:27.595 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:39:27.591 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:39:27.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.603 7f5a78ff9700 1 -- 172.21.2.101:0/4238162107 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f5a6c0009b0 con 0x7f5a5c018730 2021-02-10T11:39:27.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.621 7f5a8262e700 1 -- 172.21.2.101:0/4238162107 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5a5c018730 msgr2=0x7f5a5c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:27.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.621 7f5a8262e700 1 --2- 172.21.2.101:0/4238162107 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5a5c018730 0x7f5a5c01abc0 crc :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:27.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.621 7f5a8262e700 1 -- 172.21.2.101:0/4238162107 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a7c0c23e0 msgr2=0x7f5a7c1a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:27.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.621 7f5a8262e700 1 --2- 172.21.2.101:0/4238162107 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a7c0c23e0 0x7f5a7c1a44b0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 rx=0x7f5a70001c60 tx=0x7f5a7000a8d0).stop 2021-02-10T11:39:27.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.621 7f5a8262e700 1 -- 172.21.2.101:0/4238162107 shutdown_connections 2021-02-10T11:39:27.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.621 7f5a8262e700 1 --2- 172.21.2.101:0/4238162107 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5a5c018730 0x7f5a5c01abc0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:27.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.621 7f5a8262e700 1 --2- 172.21.2.101:0/4238162107 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a7c0c27e0 0x7f5a7c0859f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:27.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.621 7f5a8262e700 1 --2- 172.21.2.101:0/4238162107 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a7c0c23e0 0x7f5a7c1a44b0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:27.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.621 7f5a8262e700 1 --2- 172.21.2.101:0/4238162107 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5a7c18f9d0 0x7f5a7c084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:27.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.621 7f5a8262e700 1 -- 172.21.2.101:0/4238162107 >> 172.21.2.101:0/4238162107 conn(0x7f5a7c0b5dc0 msgr2=0x7f5a7c0c2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:27.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.621 7f5a8262e700 1 -- 172.21.2.101:0/4238162107 shutdown_connections 2021-02-10T11:39:27.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:27.621 7f5a8262e700 1 -- 172.21.2.101:0/4238162107 wait complete. 2021-02-10T11:39:27.633 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000977298 --group_name group_0000000000112255 2021-02-10T11:39:28.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.044 7f3aa1106700 1 Processor -- start 2021-02-10T11:39:28.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.044 7f3aa1106700 1 -- start start 2021-02-10T11:39:28.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.045 7f3aa1106700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a9c0d1870 0x7f3a9c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:28.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.045 7f3aa1106700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3a9c0cff70 0x7f3a9c109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:28.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.045 7f3aa1106700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a9c109a60 con 0x7f3a9c1b0ae0 2021-02-10T11:39:28.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.045 7f3aa1106700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a9c109750 con 0x7f3a9c0cff70 2021-02-10T11:39:28.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.045 7f3aa1106700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c088070 con 0x7f3a9c0d1870 2021-02-10T11:39:28.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.045 7f3a9b59e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3a9c0cff70 0x7f3a9c109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.045 7f3a9b59e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3a9c0cff70 0x7f3a9c109130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55602/0 (socket says 172.21.2.101:55602) 2021-02-10T11:39:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.045 7f3a9b59e700 1 -- 172.21.2.101:0/918665402 learned_addr learned my addr 172.21.2.101:0/918665402 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.045 7f3a9b59e700 1 -- 172.21.2.101:0/918665402 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a9c0d1870 msgr2=0x7f3a9c1b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:39:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.045 7f3a9b59e700 1 --2- 172.21.2.101:0/918665402 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a9c0d1870 0x7f3a9c1b04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.045 7f3a9b59e700 1 -- 172.21.2.101:0/918665402 >> v1:172.21.2.108:6789/0 conn(0x7f3a9c1b0ae0 legacy=0x7f3a9c108940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:39:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.045 7f3a9b59e700 1 -- 172.21.2.101:0/918665402 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c088260 con 0x7f3a9c0cff70 2021-02-10T11:39:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.045 7f3a9b59e700 1 --2- 172.21.2.101:0/918665402 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a9c0cff70 0x7f3a9c109130 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 rx=0x7f3a94006f00 tx=0x7f3a94009f10).ready entity=mon.0 client_cookie=b69ed91c1306de9b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.045 7f3a99d9b700 1 -- 172.21.2.101:0/918665402 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3a9400b3b0 con 0x7f3a9c0cff70 2021-02-10T11:39:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.045 7f3a99d9b700 1 -- 172.21.2.101:0/918665402 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9400b730 con 0x7f3a9c0cff70 2021-02-10T11:39:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.045 7f3a99d9b700 1 -- 172.21.2.101:0/918665402 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3a9400bc50 con 0x7f3a9c0cff70 2021-02-10T11:39:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.045 7f3aa1106700 1 -- 172.21.2.101:0/918665402 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a9c0cff70 msgr2=0x7f3a9c109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.045 7f3aa1106700 1 --2- 172.21.2.101:0/918665402 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a9c0cff70 0x7f3a9c109130 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 rx=0x7f3a94006f00 tx=0x7f3a94009f10).stop 2021-02-10T11:39:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.046 7f3aa1106700 1 -- 172.21.2.101:0/918665402 shutdown_connections 2021-02-10T11:39:28.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.046 7f3aa1106700 1 --2- 172.21.2.101:0/918665402 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a9c0cff70 0x7f3a9c109130 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:28.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.046 7f3aa1106700 1 --2- 172.21.2.101:0/918665402 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a9c0d1870 0x7f3a9c1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:28.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.046 7f3aa1106700 1 -- 172.21.2.101:0/918665402 >> 172.21.2.101:0/918665402 conn(0x7f3a9c0ce420 msgr2=0x7f3a9c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:28.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.046 7f3aa1106700 1 -- 172.21.2.101:0/918665402 shutdown_connections 2021-02-10T11:39:28.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.046 7f3aa1106700 1 -- 172.21.2.101:0/918665402 wait complete. 2021-02-10T11:39:28.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.060 7f3aa1106700 1 Processor -- start 2021-02-10T11:39:28.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.061 7f3aa1106700 1 -- start start 2021-02-10T11:39:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.061 7f3aa1106700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3a9c1b3330 0x7f3a9c1b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.061 7f3aa1106700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a9c1a29d0 0x7f3a9c093130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.061 7f3aa1106700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a9c1a2dd0 0x7f3a9c187e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.061 7f3aa1106700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c0937d0 con 0x7f3a9c1a29d0 2021-02-10T11:39:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.061 7f3aa1106700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c1b3730 con 0x7f3a9c1b3330 2021-02-10T11:39:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.061 7f3aa1106700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c188490 con 0x7f3a9c1a2dd0 2021-02-10T11:39:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.061 7f3a9a59c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a9c1a29d0 0x7f3a9c093130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.061 7f3a9a59c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a9c1a29d0 0x7f3a9c093130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55606/0 (socket says 172.21.2.101:55606) 2021-02-10T11:39:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.061 7f3a9a59c700 1 -- 172.21.2.101:0/3718106302 learned_addr learned my addr 172.21.2.101:0/3718106302 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.061 7f3a9a59c700 1 -- 172.21.2.101:0/3718106302 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a9c1a2dd0 msgr2=0x7f3a9c187e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.061 7f3a9a59c700 1 --2- 172.21.2.101:0/3718106302 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a9c1a2dd0 0x7f3a9c187e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.061 7f3a9a59c700 1 -- 172.21.2.101:0/3718106302 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3a9c1b3330 msgr2=0x7f3a9c1b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.061 7f3a9a59c700 1 --2- 172.21.2.101:0/3718106302 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3a9c1b3330 0x7f3a9c1b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.061 7f3a9a59c700 1 -- 172.21.2.101:0/3718106302 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c1886b0 con 0x7f3a9c1a29d0 2021-02-10T11:39:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.062 7f3a9a59c700 1 --2- 172.21.2.101:0/3718106302 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a9c1a29d0 0x7f3a9c093130 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 rx=0x7f3a8c00b5b0 tx=0x7f3a8c00b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.062 7f3a937fe700 1 -- 172.21.2.101:0/3718106302 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3a8c00bd40 con 0x7f3a9c1a29d0 2021-02-10T11:39:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.062 7f3a937fe700 1 -- 172.21.2.101:0/3718106302 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8c006020 con 0x7f3a9c1a29d0 2021-02-10T11:39:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.062 7f3aa1106700 1 -- 172.21.2.101:0/3718106302 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9c1886b0 con 0x7f3a9c1a29d0 2021-02-10T11:39:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.062 7f3aa1106700 1 -- 172.21.2.101:0/3718106302 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9c0b7a00 con 0x7f3a9c1a29d0 2021-02-10T11:39:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.062 7f3a937fe700 1 -- 172.21.2.101:0/3718106302 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3a8c0054a0 con 0x7f3a9c1a29d0 2021-02-10T11:39:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.063 7f3a937fe700 1 -- 172.21.2.101:0/3718106302 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3a8c006200 con 0x7f3a9c1a29d0 2021-02-10T11:39:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.063 7f3a937fe700 1 --2- 172.21.2.101:0/3718106302 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a80018730 0x7f3a8001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.063 7f3a937fe700 1 -- 172.21.2.101:0/3718106302 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3a8c0327e0 con 0x7f3a9c1a29d0 2021-02-10T11:39:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.063 7f3a9ad9d700 1 --2- 172.21.2.101:0/3718106302 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a80018730 0x7f3a8001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.063 7f3a9ad9d700 1 --2- 172.21.2.101:0/3718106302 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a80018730 0x7f3a8001ac10 crc :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:28.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.095 7f3aa1106700 1 -- 172.21.2.101:0/3718106302 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9c0825c0 con 0x7f3a9c1a29d0 2021-02-10T11:39:28.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.097 7f3a937fe700 1 -- 172.21.2.101:0/3718106302 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3a8c000960 con 0x7f3a9c1a29d0 2021-02-10T11:39:28.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.223 7f3aa1106700 1 -- 172.21.2.101:0/3718106302 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000977298", "target": ["mgr", ""], "group_name": "group_0000000000112255"}) v1 -- 0x7f3a9c0b7a00 con 0x7f3a80018730 2021-02-10T11:39:28.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.247 7f3a937fe700 1 -- 172.21.2.101:0/3718106302 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f3a84006e40 con 0x7f3a80018730 2021-02-10T11:39:28.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.257 7f3aa1106700 1 -- 172.21.2.101:0/3718106302 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a80018730 msgr2=0x7f3a8001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:28.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.257 7f3aa1106700 1 --2- 172.21.2.101:0/3718106302 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a80018730 0x7f3a8001ac10 crc :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:28.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.257 7f3aa1106700 1 -- 172.21.2.101:0/3718106302 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a9c1a29d0 msgr2=0x7f3a9c093130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:28.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.257 7f3aa1106700 1 --2- 172.21.2.101:0/3718106302 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a9c1a29d0 0x7f3a9c093130 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 rx=0x7f3a8c00b5b0 tx=0x7f3a8c00b950).stop 2021-02-10T11:39:28.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.257 7f3aa1106700 1 -- 172.21.2.101:0/3718106302 shutdown_connections 2021-02-10T11:39:28.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.257 7f3aa1106700 1 --2- 172.21.2.101:0/3718106302 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3a80018730 0x7f3a8001ac10 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:28.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.257 7f3aa1106700 1 --2- 172.21.2.101:0/3718106302 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3a9c1a2dd0 0x7f3a9c187e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:28.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.257 7f3aa1106700 1 --2- 172.21.2.101:0/3718106302 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3a9c1a29d0 0x7f3a9c093130 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:28.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.257 7f3aa1106700 1 --2- 172.21.2.101:0/3718106302 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3a9c1b3330 0x7f3a9c1b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:28.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.257 7f3aa1106700 1 -- 172.21.2.101:0/3718106302 >> 172.21.2.101:0/3718106302 conn(0x7f3a9c0841d0 msgr2=0x7f3a9c0827f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:28.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.257 7f3aa1106700 1 -- 172.21.2.101:0/3718106302 shutdown_connections 2021-02-10T11:39:28.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.257 7f3aa1106700 1 -- 172.21.2.101:0/3718106302 wait complete. 2021-02-10T11:39:28.269 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume authorize cephfs subvolume_0000000000977298 guest1 --group_name group_0000000000112255 --tenant_id tenant1 2021-02-10T11:39:28.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.687 7f8f728e0700 1 Processor -- start 2021-02-10T11:39:28.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.687 7f8f728e0700 1 -- start start 2021-02-10T11:39:28.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.687 7f8f728e0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f6c0e53a0 0x7f8f6c1b0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:28.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.687 7f8f728e0700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6c109ac0 con 0x7f8f6c1b0b70 2021-02-10T11:39:28.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.687 7f8f728e0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6c0e57a0 con 0x7f8f6c10b050 2021-02-10T11:39:28.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.687 7f8f728e0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0fe2b0 con 0x7f8f6c0e53a0 2021-02-10T11:39:28.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.687 7f8f6b7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8f6c1b0b70 0x7f8f6c1089d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56108/0 (socket says 172.21.2.101:56108) 2021-02-10T11:39:28.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.687 7f8f6b7fe700 1 -- 172.21.2.101:0/898168258 learned_addr learned my addr 172.21.2.101:0/898168258 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:28.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.688 7f8f6affd700 1 -- 172.21.2.101:0/898168258 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 814452394 0 0) 0x7f8f600010b0 con 0x7f8f6c1b0b70 2021-02-10T11:39:28.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.688 7f8f6bfff700 1 --2- 172.21.2.101:0/898168258 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f6c0e53a0 0x7f8f6c1b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:28.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.688 7f8f6affd700 1 -- 172.21.2.101:0/898168258 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f54002fe0 con 0x7f8f6c1b0b70 2021-02-10T11:39:28.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.688 7f8f6affd700 1 -- 172.21.2.101:0/898168258 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2001080378 0 0) 0x7f8f60001010 con 0x7f8f6c1b0b70 2021-02-10T11:39:28.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.688 7f8f6affd700 1 -- 172.21.2.101:0/898168258 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f6c0e53a0 msgr2=0x7f8f6c1b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:28.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.688 7f8f6affd700 1 --2- 172.21.2.101:0/898168258 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f6c0e53a0 0x7f8f6c1b0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:28.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.688 7f8f6affd700 1 -- 172.21.2.101:0/898168258 >> v1:172.21.2.108:6789/0 conn(0x7f8f6c10b050 legacy=0x7f8f6c1091c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:28.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.688 7f8f6affd700 1 -- 172.21.2.101:0/898168258 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6c0e5f80 con 0x7f8f6c1b0b70 2021-02-10T11:39:28.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.688 7f8f6affd700 1 -- 172.21.2.101:0/898168258 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f8f60001530 con 0x7f8f6c1b0b70 2021-02-10T11:39:28.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.688 7f8f6affd700 1 -- 172.21.2.101:0/898168258 >> v1:172.21.2.101:6789/0 conn(0x7f8f6c1b0b70 legacy=0x7f8f6c1089d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:28.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.688 7f8f6affd700 1 --2- 172.21.2.101:0/898168258 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f54003b10 0x7f8f54005f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:28.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.688 7f8f6affd700 1 --2- 172.21.2.101:0/898168258 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f54006770 0x7f8f54008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:28.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.688 7f8f6affd700 1 --2- 172.21.2.101:0/898168258 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8f540094f0 0x7f8f5400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:28.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.688 7f8f6affd700 1 -- 172.21.2.101:0/898168258 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5400c2c0 con 0x7f8f54006770 2021-02-10T11:39:28.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.688 7f8f6affd700 1 -- 172.21.2.101:0/898168258 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5400c4e0 con 0x7f8f540094f0 2021-02-10T11:39:28.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.688 7f8f6affd700 1 -- 172.21.2.101:0/898168258 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5400c700 con 0x7f8f54003b10 2021-02-10T11:39:28.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.688 7f8f6b7fe700 1 --2- 172.21.2.101:0/898168258 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f54006770 0x7f8f54008c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:28.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.689 7f8f6bfff700 1 --2- 172.21.2.101:0/898168258 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f54003b10 0x7f8f54005f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:28.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.689 7f8f70efd700 1 --2- 172.21.2.101:0/898168258 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8f540094f0 0x7f8f5400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:28.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.689 7f8f6b7fe700 1 -- 172.21.2.101:0/898168258 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f54003b10 msgr2=0x7f8f54005f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:28.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.689 7f8f6b7fe700 1 --2- 172.21.2.101:0/898168258 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f54003b10 0x7f8f54005f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:28.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.689 7f8f6b7fe700 1 -- 172.21.2.101:0/898168258 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8f540094f0 msgr2=0x7f8f5400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:28.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.689 7f8f6b7fe700 1 --2- 172.21.2.101:0/898168258 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8f540094f0 0x7f8f5400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:28.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.689 7f8f6b7fe700 1 -- 172.21.2.101:0/898168258 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f54050970 con 0x7f8f54006770 2021-02-10T11:39:28.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.689 7f8f6b7fe700 1 --2- 172.21.2.101:0/898168258 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f54006770 0x7f8f54008c70 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 rx=0x7f8f60001380 tx=0x7f8f60007d30).ready entity=mon.0 client_cookie=59c41dcfcdf370cd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:28.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.689 7f8f6affd700 1 -- 172.21.2.101:0/898168258 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8f60001010 con 0x7f8f54006770 2021-02-10T11:39:28.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.689 7f8f6affd700 1 -- 172.21.2.101:0/898168258 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6000a310 con 0x7f8f54006770 2021-02-10T11:39:28.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.689 7f8f6affd700 1 -- 172.21.2.101:0/898168258 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8f6000a770 con 0x7f8f54006770 2021-02-10T11:39:28.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.689 7f8f728e0700 1 -- 172.21.2.101:0/898168258 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f54006770 msgr2=0x7f8f54008c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:28.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.689 7f8f728e0700 1 --2- 172.21.2.101:0/898168258 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f54006770 0x7f8f54008c70 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 rx=0x7f8f60001380 tx=0x7f8f60007d30).stop 2021-02-10T11:39:28.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.689 7f8f70efd700 1 -- 172.21.2.101:0/898168258 reap_dead start 2021-02-10T11:39:28.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.690 7f8f728e0700 1 -- 172.21.2.101:0/898168258 shutdown_connections 2021-02-10T11:39:28.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.690 7f8f728e0700 1 -- 172.21.2.101:0/898168258 >> 172.21.2.101:0/898168258 conn(0x7f8f6c0ce4b0 msgr2=0x7f8f6c0e45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:28.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.690 7f8f728e0700 1 -- 172.21.2.101:0/898168258 shutdown_connections 2021-02-10T11:39:28.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.690 7f8f728e0700 1 -- 172.21.2.101:0/898168258 wait complete. 2021-02-10T11:39:28.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.705 7f8f728e0700 1 Processor -- start 2021-02-10T11:39:28.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.705 7f8f728e0700 1 -- start start 2021-02-10T11:39:28.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.705 7f8f728e0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f6c0a5ac0 0x7f8f6c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:28.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.705 7f8f728e0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8f6c0a5fc0 0x7f8f6c1a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:28.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.705 7f8f728e0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f6c1a4b80 0x7f8f6c0c16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:28.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.705 7f8f728e0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0c1120 con 0x7f8f6c0a5ac0 2021-02-10T11:39:28.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.705 7f8f728e0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0a63c0 con 0x7f8f6c0a5fc0 2021-02-10T11:39:28.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.705 7f8f728e0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0a65e0 con 0x7f8f6c1a4b80 2021-02-10T11:39:28.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.705 7f8f6bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f6c0a5ac0 0x7f8f6c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:28.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.705 7f8f6bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f6c0a5ac0 0x7f8f6c0a5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55624/0 (socket says 172.21.2.101:55624) 2021-02-10T11:39:28.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.705 7f8f6bfff700 1 -- 172.21.2.101:0/2610988814 learned_addr learned my addr 172.21.2.101:0/2610988814 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:28.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.706 7f8f6bfff700 1 -- 172.21.2.101:0/2610988814 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f6c1a4b80 msgr2=0x7f8f6c0c16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:28.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.706 7f8f6bfff700 1 --2- 172.21.2.101:0/2610988814 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f6c1a4b80 0x7f8f6c0c16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:28.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.706 7f8f6bfff700 1 -- 172.21.2.101:0/2610988814 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8f6c0a5fc0 msgr2=0x7f8f6c1a4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:28.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.706 7f8f6bfff700 1 --2- 172.21.2.101:0/2610988814 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8f6c0a5fc0 0x7f8f6c1a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:28.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.706 7f8f6bfff700 1 -- 172.21.2.101:0/2610988814 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6c0c1cc0 con 0x7f8f6c0a5ac0 2021-02-10T11:39:28.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.706 7f8f6bfff700 1 --2- 172.21.2.101:0/2610988814 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f6c0a5ac0 0x7f8f6c0a5260 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 rx=0x7f8f5c006f60 tx=0x7f8f5c00b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:28.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.706 7f8f68ff9700 1 -- 172.21.2.101:0/2610988814 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8f5c00bd60 con 0x7f8f6c0a5ac0 2021-02-10T11:39:28.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.706 7f8f68ff9700 1 -- 172.21.2.101:0/2610988814 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5c000c90 con 0x7f8f6c0a5ac0 2021-02-10T11:39:28.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.706 7f8f728e0700 1 -- 172.21.2.101:0/2610988814 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6c0c1cc0 con 0x7f8f6c0a5ac0 2021-02-10T11:39:28.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.706 7f8f68ff9700 1 -- 172.21.2.101:0/2610988814 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8f5c002270 con 0x7f8f6c0a5ac0 2021-02-10T11:39:28.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.706 7f8f728e0700 1 -- 172.21.2.101:0/2610988814 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6c0c2250 con 0x7f8f6c0a5ac0 2021-02-10T11:39:28.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.707 7f8f68ff9700 1 -- 172.21.2.101:0/2610988814 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8f5c002020 con 0x7f8f6c0a5ac0 2021-02-10T11:39:28.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.707 7f8f68ff9700 1 --2- 172.21.2.101:0/2610988814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8f4c018730 0x7f8f4c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:28.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.707 7f8f68ff9700 1 -- 172.21.2.101:0/2610988814 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8f5c032620 con 0x7f8f6c0a5ac0 2021-02-10T11:39:28.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.707 7f8f6b7fe700 1 --2- 172.21.2.101:0/2610988814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8f4c018730 0x7f8f4c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:28.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.707 7f8f6b7fe700 1 --2- 172.21.2.101:0/2610988814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8f4c018730 0x7f8f4c01abc0 crc :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:28.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.737 7f8f728e0700 1 -- 172.21.2.101:0/2610988814 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f6c1037b0 con 0x7f8f6c0a5ac0 2021-02-10T11:39:28.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.740 7f8f68ff9700 1 -- 172.21.2.101:0/2610988814 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8f5c000c90 con 0x7f8f6c0a5ac0 2021-02-10T11:39:28.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.863 7f8f728e0700 1 -- 172.21.2.101:0/2610988814 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000977298", "target": ["mgr", ""], "tenant_id": "tenant1", "auth_id": "guest1", "group_name": "group_0000000000112255"}) v1 -- 0x7f8f6c0c1120 con 0x7f8f4c018730 2021-02-10T11:39:28.871 INFO:tasks.ceph.mgr.y.gibba001.stderr:Creating meta for ID guest1 with tenant tenant1 2021-02-10T11:39:28.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.892 7f8f68ff9700 1 -- 172.21.2.101:0/2610988814 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7f8f600009b0 con 0x7f8f4c018730 2021-02-10T11:39:28.897 INFO:teuthology.orchestra.run.gibba001.stdout:AQDwxSNgoURPNBAA3WTD9XIbUFJUSQXQj9SWWw== 2021-02-10T11:39:28.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.896 7f8f728e0700 1 -- 172.21.2.101:0/2610988814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8f4c018730 msgr2=0x7f8f4c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:28.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.896 7f8f728e0700 1 --2- 172.21.2.101:0/2610988814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8f4c018730 0x7f8f4c01abc0 crc :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:28.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.896 7f8f728e0700 1 -- 172.21.2.101:0/2610988814 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f6c0a5ac0 msgr2=0x7f8f6c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:28.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.896 7f8f728e0700 1 --2- 172.21.2.101:0/2610988814 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f6c0a5ac0 0x7f8f6c0a5260 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 rx=0x7f8f5c006f60 tx=0x7f8f5c00b970).stop 2021-02-10T11:39:28.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.896 7f8f728e0700 1 -- 172.21.2.101:0/2610988814 shutdown_connections 2021-02-10T11:39:28.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.896 7f8f728e0700 1 --2- 172.21.2.101:0/2610988814 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8f4c018730 0x7f8f4c01abc0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:28.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.896 7f8f728e0700 1 --2- 172.21.2.101:0/2610988814 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f6c1a4b80 0x7f8f6c0c16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:28.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.896 7f8f728e0700 1 --2- 172.21.2.101:0/2610988814 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8f6c0a5fc0 0x7f8f6c1a4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:28.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.896 7f8f728e0700 1 --2- 172.21.2.101:0/2610988814 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f6c0a5ac0 0x7f8f6c0a5260 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:28.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.896 7f8f728e0700 1 -- 172.21.2.101:0/2610988814 >> 172.21.2.101:0/2610988814 conn(0x7f8f6c105f10 msgr2=0x7f8f6c0b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:28.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.896 7f8f728e0700 1 -- 172.21.2.101:0/2610988814 shutdown_connections 2021-02-10T11:39:28.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:28.896 7f8f728e0700 1 -- 172.21.2.101:0/2610988814 wait complete. 2021-02-10T11:39:28.908 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000977298 --group_name group_0000000000112255 2021-02-10T11:39:29.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.320 7fe8e3b42700 1 Processor -- start 2021-02-10T11:39:29.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.320 7fe8e3b42700 1 -- start start 2021-02-10T11:39:29.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.320 7fe8e3b42700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe8dc0d1c80 0x7fe8dc0cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:29.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.320 7fe8e3b42700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8dc0d0180 con 0x7fe8dc0d1880 2021-02-10T11:39:29.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.320 7fe8e3b42700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8dc0fe1d0 con 0x7fe8dc0cf590 2021-02-10T11:39:29.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.320 7fe8e3b42700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8dc1b0cc0 con 0x7fe8dc0d1c80 2021-02-10T11:39:29.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.321 7fe8e115d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe8dc0d1c80 0x7fe8dc0cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:29.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.321 7fe8e115d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe8dc0d1c80 0x7fe8dc0cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55636/0 (socket says 172.21.2.101:55636) 2021-02-10T11:39:29.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.321 7fe8e115d700 1 -- 172.21.2.101:0/3651417824 learned_addr learned my addr 172.21.2.101:0/3651417824 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:29.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.321 7fe8e115d700 1 -- 172.21.2.101:0/3651417824 >> v1:172.21.2.108:6789/0 conn(0x7fe8dc0cf590 legacy=0x7fe8dc0cf990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:39:29.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.321 7fe8e115d700 1 -- 172.21.2.101:0/3651417824 >> v1:172.21.2.101:6789/0 conn(0x7fe8dc0d1880 legacy=0x7fe8dc1b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:29.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.321 7fe8e115d700 1 -- 172.21.2.101:0/3651417824 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc109a70 con 0x7fe8dc0d1c80 2021-02-10T11:39:29.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.321 7fe8e115d700 1 --2- 172.21.2.101:0/3651417824 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8dc0d1c80 0x7fe8dc0cef70 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 rx=0x7fe8d8006700 tx=0x7fe8d8009b80).ready entity=mon.0 client_cookie=ad328c57ee705849 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:29.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.321 7fe8e095c700 1 -- 172.21.2.101:0/3651417824 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe8d800c0c0 con 0x7fe8dc0d1c80 2021-02-10T11:39:29.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.321 7fe8e095c700 1 -- 172.21.2.101:0/3651417824 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d800c440 con 0x7fe8dc0d1c80 2021-02-10T11:39:29.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.321 7fe8e095c700 1 -- 172.21.2.101:0/3651417824 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe8d800c920 con 0x7fe8dc0d1c80 2021-02-10T11:39:29.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.321 7fe8e3b42700 1 -- 172.21.2.101:0/3651417824 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8dc0d1c80 msgr2=0x7fe8dc0cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:29.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.321 7fe8e3b42700 1 --2- 172.21.2.101:0/3651417824 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8dc0d1c80 0x7fe8dc0cef70 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 rx=0x7fe8d8006700 tx=0x7fe8d8009b80).stop 2021-02-10T11:39:29.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.322 7fe8e3b42700 1 -- 172.21.2.101:0/3651417824 shutdown_connections 2021-02-10T11:39:29.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.322 7fe8e3b42700 1 --2- 172.21.2.101:0/3651417824 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8dc0d1c80 0x7fe8dc0cef70 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:29.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.322 7fe8e3b42700 1 -- 172.21.2.101:0/3651417824 >> 172.21.2.101:0/3651417824 conn(0x7fe8dc0ce430 msgr2=0x7fe8dc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:29.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.322 7fe8e3b42700 1 -- 172.21.2.101:0/3651417824 shutdown_connections 2021-02-10T11:39:29.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.322 7fe8e3b42700 1 -- 172.21.2.101:0/3651417824 wait complete. 2021-02-10T11:39:29.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.336 7fe8e3b42700 1 Processor -- start 2021-02-10T11:39:29.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.336 7fe8e3b42700 1 -- start start 2021-02-10T11:39:29.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.336 7fe8e3b42700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe8dc0b7ea0 0x7fe8dc0c0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:29.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.336 7fe8e3b42700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8dc0c0620 0x7fe8dc09eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:29.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.336 7fe8e3b42700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8dc09f3c0 0x7fe8dc0c0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:29.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.336 7fe8e3b42700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0b7b20 con 0x7fe8dc09f3c0 2021-02-10T11:39:29.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.336 7fe8e3b42700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc1c6920 con 0x7fe8dc0b7ea0 2021-02-10T11:39:29.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.336 7fe8e3b42700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc1c6b40 con 0x7fe8dc0c0620 2021-02-10T11:39:29.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.337 7fe8e215f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8dc09f3c0 0x7fe8dc0c0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:29.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.337 7fe8e215f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8dc09f3c0 0x7fe8dc0c0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55644/0 (socket says 172.21.2.101:55644) 2021-02-10T11:39:29.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.337 7fe8e215f700 1 -- 172.21.2.101:0/2229361404 learned_addr learned my addr 172.21.2.101:0/2229361404 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:29.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.337 7fe8e115d700 1 --2- 172.21.2.101:0/2229361404 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8dc0c0620 0x7fe8dc09eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:29.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.337 7fe8e195e700 1 --2- 172.21.2.101:0/2229361404 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe8dc0b7ea0 0x7fe8dc0c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:29.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.337 7fe8e215f700 1 -- 172.21.2.101:0/2229361404 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8dc0c0620 msgr2=0x7fe8dc09eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:29.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.337 7fe8e215f700 1 --2- 172.21.2.101:0/2229361404 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8dc0c0620 0x7fe8dc09eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:29.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.337 7fe8e215f700 1 -- 172.21.2.101:0/2229361404 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe8dc0b7ea0 msgr2=0x7fe8dc0c0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:29.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.337 7fe8e215f700 1 --2- 172.21.2.101:0/2229361404 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe8dc0b7ea0 0x7fe8dc0c0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:29.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.337 7fe8e215f700 1 -- 172.21.2.101:0/2229361404 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc0b82a0 con 0x7fe8dc09f3c0 2021-02-10T11:39:29.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.337 7fe8e195e700 1 --2- 172.21.2.101:0/2229361404 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe8dc0b7ea0 0x7fe8dc0c0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:39:29.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.337 7fe8e215f700 1 --2- 172.21.2.101:0/2229361404 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8dc09f3c0 0x7fe8dc0c0fd0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 rx=0x7fe8d4009940 tx=0x7fe8d400bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:29.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.338 7fe8ce7fc700 1 -- 172.21.2.101:0/2229361404 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe8d4003030 con 0x7fe8dc09f3c0 2021-02-10T11:39:29.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.338 7fe8ce7fc700 1 -- 172.21.2.101:0/2229361404 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d4001270 con 0x7fe8dc09f3c0 2021-02-10T11:39:29.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.338 7fe8ce7fc700 1 -- 172.21.2.101:0/2229361404 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe8d4001790 con 0x7fe8dc09f3c0 2021-02-10T11:39:29.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.338 7fe8e3b42700 1 -- 172.21.2.101:0/2229361404 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8dc0b82a0 con 0x7fe8dc09f3c0 2021-02-10T11:39:29.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.338 7fe8e3b42700 1 -- 172.21.2.101:0/2229361404 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8dc0b82a0 con 0x7fe8dc09f3c0 2021-02-10T11:39:29.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.338 7fe8ce7fc700 1 -- 172.21.2.101:0/2229361404 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe8d4001270 con 0x7fe8dc09f3c0 2021-02-10T11:39:29.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.339 7fe8ce7fc700 1 --2- 172.21.2.101:0/2229361404 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8c40186c0 0x7fe8c401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:29.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.339 7fe8ce7fc700 1 -- 172.21.2.101:0/2229361404 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe8d4032930 con 0x7fe8dc09f3c0 2021-02-10T11:39:29.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.339 7fe8e195e700 1 --2- 172.21.2.101:0/2229361404 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8c40186c0 0x7fe8c401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:29.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.339 7fe8e195e700 1 --2- 172.21.2.101:0/2229361404 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8c40186c0 0x7fe8c401abc0 crc :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:29.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.370 7fe8e3b42700 1 -- 172.21.2.101:0/2229361404 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8dc0b8540 con 0x7fe8dc09f3c0 2021-02-10T11:39:29.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.373 7fe8ce7fc700 1 -- 172.21.2.101:0/2229361404 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe8d400be30 con 0x7fe8dc09f3c0 2021-02-10T11:39:29.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.502 7fe8e3b42700 1 -- 172.21.2.101:0/2229361404 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000977298", "target": ["mgr", ""], "group_name": "group_0000000000112255"}) v1 -- 0x7fe8dc0b8540 con 0x7fe8c40186c0 2021-02-10T11:39:29.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.505 7fe8ce7fc700 1 -- 172.21.2.101:0/2229361404 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+96 (crc 0 0 0) 0x7fe8d0006b10 con 0x7fe8c40186c0 2021-02-10T11:39:29.509 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/group_0000000000112255/subvolume_0000000000977298/051641b3-02be-4edf-8148-3cd1debeb31b 2021-02-10T11:39:29.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.507 7fe8e3b42700 1 -- 172.21.2.101:0/2229361404 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8c40186c0 msgr2=0x7fe8c401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:29.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.507 7fe8e3b42700 1 --2- 172.21.2.101:0/2229361404 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8c40186c0 0x7fe8c401abc0 crc :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:29.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.507 7fe8e3b42700 1 -- 172.21.2.101:0/2229361404 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8dc09f3c0 msgr2=0x7fe8dc0c0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:29.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.507 7fe8e3b42700 1 --2- 172.21.2.101:0/2229361404 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8dc09f3c0 0x7fe8dc0c0fd0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 rx=0x7fe8d4009940 tx=0x7fe8d400bcf0).stop 2021-02-10T11:39:29.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.507 7fe8e3b42700 1 -- 172.21.2.101:0/2229361404 shutdown_connections 2021-02-10T11:39:29.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.507 7fe8e3b42700 1 --2- 172.21.2.101:0/2229361404 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8c40186c0 0x7fe8c401abc0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:29.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.507 7fe8e3b42700 1 --2- 172.21.2.101:0/2229361404 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8dc0c0620 0x7fe8dc09eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:29.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.507 7fe8e3b42700 1 --2- 172.21.2.101:0/2229361404 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8dc09f3c0 0x7fe8dc0c0fd0 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:29.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.507 7fe8e3b42700 1 --2- 172.21.2.101:0/2229361404 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe8dc0b7ea0 0x7fe8dc0c0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:29.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.507 7fe8e3b42700 1 -- 172.21.2.101:0/2229361404 >> 172.21.2.101:0/2229361404 conn(0x7fe8dc0a02c0 msgr2=0x7fe8dc09b200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:29.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.507 7fe8e3b42700 1 -- 172.21.2.101:0/2229361404 shutdown_connections 2021-02-10T11:39:29.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.507 7fe8e3b42700 1 -- 172.21.2.101:0/2229361404 wait complete. 2021-02-10T11:39:29.519 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.guest1 mds 'allow rw path=/volumes/group_0000000000112255, allow rw path=/volumes/group_0000000000112255/subvolume_0000000000977298/051641b3-02be-4edf-8148-3cd1debeb31b' osd 'allow rw pool=cephfs_data' mon 'allow r' mgr 'allow *' 2021-02-10T11:39:29.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.938 7ff7b2e38700 1 Processor -- start 2021-02-10T11:39:29.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.938 7ff7b2e38700 1 -- start start 2021-02-10T11:39:29.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.938 7ff7b2e38700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff7ac0e5750 0x7ff7ac10fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:29.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.938 7ff7b2e38700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff7ac10d940 0x7ff7ac10dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:29.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.938 7ff7b2e38700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7ac10e750 con 0x7ff7ac0e5350 2021-02-10T11:39:29.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.938 7ff7b2e38700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7ac0e61d0 con 0x7ff7ac10d940 2021-02-10T11:39:29.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.938 7ff7b2e38700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7ac10e360 con 0x7ff7ac0e5750 2021-02-10T11:39:29.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.939 7ff7b1455700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff7ac10d940 0x7ff7ac10dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:29.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.939 7ff7b0c54700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff7ac0e5350 0x7ff7ac10d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56144/0 (socket says 172.21.2.101:56144) 2021-02-10T11:39:29.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.939 7ff7b0c54700 1 -- 172.21.2.101:0/3691717247 learned_addr learned my addr 172.21.2.101:0/3691717247 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:29.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.939 7ff7b1455700 1 -- 172.21.2.101:0/3691717247 >> v2:172.21.2.108:3300/0 conn(0x7ff7ac0e5750 msgr2=0x7ff7ac10fce0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:39:29.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.939 7ff7b1455700 1 --2- 172.21.2.101:0/3691717247 >> v2:172.21.2.108:3300/0 conn(0x7ff7ac0e5750 0x7ff7ac10fce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:29.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.939 7ff7b1455700 1 -- 172.21.2.101:0/3691717247 >> v1:172.21.2.101:6789/0 conn(0x7ff7ac0e5350 legacy=0x7ff7ac10d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:29.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.939 7ff7b1455700 1 -- 172.21.2.101:0/3691717247 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac1b0be0 con 0x7ff7ac10d940 2021-02-10T11:39:29.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.939 7ff7b1455700 1 --2- 172.21.2.101:0/3691717247 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff7ac10d940 0x7ff7ac10dd40 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 rx=0x7ff7a4006e30 tx=0x7ff7a4009e60).ready entity=mon.0 client_cookie=2fbc5fbb31711151 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:29.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.939 7ff7ab7fe700 1 -- 172.21.2.101:0/3691717247 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff7a4006a50 con 0x7ff7ac10d940 2021-02-10T11:39:29.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.939 7ff7ab7fe700 1 -- 172.21.2.101:0/3691717247 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a400b300 con 0x7ff7ac10d940 2021-02-10T11:39:29.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.939 7ff7ab7fe700 1 -- 172.21.2.101:0/3691717247 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff7a400b7e0 con 0x7ff7ac10d940 2021-02-10T11:39:29.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.939 7ff7b2e38700 1 -- 172.21.2.101:0/3691717247 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff7ac10d940 msgr2=0x7ff7ac10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:29.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.939 7ff7b2e38700 1 --2- 172.21.2.101:0/3691717247 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff7ac10d940 0x7ff7ac10dd40 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 rx=0x7ff7a4006e30 tx=0x7ff7a4009e60).stop 2021-02-10T11:39:29.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.940 7ff7b2e38700 1 -- 172.21.2.101:0/3691717247 shutdown_connections 2021-02-10T11:39:29.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.940 7ff7b2e38700 1 --2- 172.21.2.101:0/3691717247 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff7ac10d940 0x7ff7ac10dd40 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:29.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.940 7ff7b2e38700 1 --2- 172.21.2.101:0/3691717247 >> v2:172.21.2.108:3300/0 conn(0x7ff7ac0e5750 0x7ff7ac10fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:29.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.940 7ff7b2e38700 1 -- 172.21.2.101:0/3691717247 >> 172.21.2.101:0/3691717247 conn(0x7ff7ac0cea00 msgr2=0x7ff7ac0e4840 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:29.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.940 7ff7b2e38700 1 -- 172.21.2.101:0/3691717247 shutdown_connections 2021-02-10T11:39:29.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.940 7ff7b2e38700 1 -- 172.21.2.101:0/3691717247 wait complete. 2021-02-10T11:39:29.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.955 7ff7b2e38700 1 Processor -- start 2021-02-10T11:39:29.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.955 7ff7b2e38700 1 -- start start 2021-02-10T11:39:29.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.955 7ff7b2e38700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff7ac1a4480 0x7ff7ac0935f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:29.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.955 7ff7b2e38700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff7ac0b7d40 0x7ff7ac0b8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:29.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.955 7ff7b2e38700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff7ac0b8760 0x7ff7ac1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:29.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.955 7ff7b2e38700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac1a0db0 con 0x7ff7ac0b7d40 2021-02-10T11:39:29.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.955 7ff7b2e38700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac1a0fa0 con 0x7ff7ac0b8760 2021-02-10T11:39:29.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.955 7ff7b2e38700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac1a11c0 con 0x7ff7ac1a4480 2021-02-10T11:39:29.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.956 7ff7abfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff7ac0b7d40 0x7ff7ac0b8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:29.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.956 7ff7abfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff7ac0b7d40 0x7ff7ac0b8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55658/0 (socket says 172.21.2.101:55658) 2021-02-10T11:39:29.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.956 7ff7abfff700 1 -- 172.21.2.101:0/2709433092 learned_addr learned my addr 172.21.2.101:0/2709433092 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:29.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.956 7ff7abfff700 1 -- 172.21.2.101:0/2709433092 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff7ac1a4480 msgr2=0x7ff7ac0935f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:29.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.956 7ff7abfff700 1 --2- 172.21.2.101:0/2709433092 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff7ac1a4480 0x7ff7ac0935f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:29.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.956 7ff7abfff700 1 -- 172.21.2.101:0/2709433092 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff7ac0b8760 msgr2=0x7ff7ac1a0790 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:29.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.956 7ff7abfff700 1 --2- 172.21.2.101:0/2709433092 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff7ac0b8760 0x7ff7ac1a0790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:29.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.956 7ff7abfff700 1 -- 172.21.2.101:0/2709433092 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac1a13e0 con 0x7ff7ac0b7d40 2021-02-10T11:39:29.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.956 7ff7abfff700 1 --2- 172.21.2.101:0/2709433092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff7ac0b7d40 0x7ff7ac0b8140 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 rx=0x7ff79c00b5c0 tx=0x7ff79c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:29.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.956 7ff7a97fa700 1 -- 172.21.2.101:0/2709433092 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff79c00bd50 con 0x7ff7ac0b7d40 2021-02-10T11:39:29.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.956 7ff7a97fa700 1 -- 172.21.2.101:0/2709433092 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79c006030 con 0x7ff7ac0b7d40 2021-02-10T11:39:29.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.956 7ff7a97fa700 1 -- 172.21.2.101:0/2709433092 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff79c0054d0 con 0x7ff7ac0b7d40 2021-02-10T11:39:29.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.956 7ff7b2e38700 1 -- 172.21.2.101:0/2709433092 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ac1a13e0 con 0x7ff7ac0b7d40 2021-02-10T11:39:29.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.956 7ff7b2e38700 1 -- 172.21.2.101:0/2709433092 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ac1a51a0 con 0x7ff7ac0b7d40 2021-02-10T11:39:29.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.957 7ff7a97fa700 1 -- 172.21.2.101:0/2709433092 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff79c006210 con 0x7ff7ac0b7d40 2021-02-10T11:39:29.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.957 7ff7a97fa700 1 --2- 172.21.2.101:0/2709433092 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7900186c0 0x7ff79001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:29.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.957 7ff7a97fa700 1 -- 172.21.2.101:0/2709433092 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff79c032460 con 0x7ff7ac0b7d40 2021-02-10T11:39:29.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.957 7ff7b0c54700 1 --2- 172.21.2.101:0/2709433092 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7900186c0 0x7ff79001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:29.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.957 7ff7b0c54700 1 --2- 172.21.2.101:0/2709433092 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7900186c0 0x7ff79001abc0 crc :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:29.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.988 7ff7b2e38700 1 -- 172.21.2.101:0/2709433092 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7ac17f530 con 0x7ff7ac0b7d40 2021-02-10T11:39:29.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:29.991 7ff7a97fa700 1 -- 172.21.2.101:0/2709433092 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff79c030d90 con 0x7ff7ac0b7d40 2021-02-10T11:39:30.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.120 7ff7b2e38700 1 -- 172.21.2.101:0/2709433092 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.guest1", "caps": ["mds", "allow rw path=/volumes/group_0000000000112255, allow rw path=/volumes/group_0000000000112255/subvolume_0000000000977298/051641b3-02be-4edf-8148-3cd1debeb31b", "osd", "allow rw pool=cephfs_data", "mon", "allow r", "mgr", "allow *"]} v 0) v1 -- 0x7ff7ac1b0be0 con 0x7ff7ac0b7d40 2021-02-10T11:39:30.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.124 7ff7a97fa700 1 -- 172.21.2.101:0/2709433092 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.guest1", "caps": ["mds", "allow rw path=/volumes/group_0000000000112255, allow rw path=/volumes/group_0000000000112255/subvolume_0000000000977298/051641b3-02be-4edf-8148-3cd1debeb31b", "osd", "allow rw pool=cephfs_data", "mon", "allow r", "mgr", "allow *"]}]=0 updated caps for client.guest1 v58) v1 ==== 363+0+0 (secure 0 0 0) 0x7ff79c01f540 con 0x7ff7ac0b7d40 2021-02-10T11:39:30.131 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.guest1 2021-02-10T11:39:30.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.129 7ff7b2e38700 1 -- 172.21.2.101:0/2709433092 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7900186c0 msgr2=0x7ff79001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:30.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.129 7ff7b2e38700 1 --2- 172.21.2.101:0/2709433092 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7900186c0 0x7ff79001abc0 crc :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:30.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.129 7ff7b2e38700 1 -- 172.21.2.101:0/2709433092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff7ac0b7d40 msgr2=0x7ff7ac0b8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:30.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.129 7ff7b2e38700 1 --2- 172.21.2.101:0/2709433092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff7ac0b7d40 0x7ff7ac0b8140 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 rx=0x7ff79c00b5c0 tx=0x7ff79c00b960).stop 2021-02-10T11:39:30.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.130 7ff7b2e38700 1 -- 172.21.2.101:0/2709433092 shutdown_connections 2021-02-10T11:39:30.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.130 7ff7b2e38700 1 --2- 172.21.2.101:0/2709433092 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff7ac0b8760 0x7ff7ac1a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:30.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.130 7ff7b2e38700 1 --2- 172.21.2.101:0/2709433092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff7ac0b7d40 0x7ff7ac0b8140 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:30.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.130 7ff7b2e38700 1 --2- 172.21.2.101:0/2709433092 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7900186c0 0x7ff79001abc0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:30.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.130 7ff7b2e38700 1 --2- 172.21.2.101:0/2709433092 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff7ac1a4480 0x7ff7ac0935f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:30.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.130 7ff7b2e38700 1 -- 172.21.2.101:0/2709433092 >> 172.21.2.101:0/2709433092 conn(0x7ff7ac1b00e0 msgr2=0x7ff7ac18a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:30.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.130 7ff7b2e38700 1 -- 172.21.2.101:0/2709433092 shutdown_connections 2021-02-10T11:39:30.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.130 7ff7b2e38700 1 -- 172.21.2.101:0/2709433092 wait complete. 2021-02-10T11:39:30.142 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume deauthorize cephfs subvolume_0000000000977298 guest1 --group_name group_0000000000112255 2021-02-10T11:39:30.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.556 7fb48a209700 1 Processor -- start 2021-02-10T11:39:30.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.556 7fb48a209700 1 -- start start 2021-02-10T11:39:30.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.556 7fb48a209700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb4840dce80 0x7fb4840d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:30.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.556 7fb48a209700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb4840eb220 0x7fb4840eb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:30.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.556 7fb48a209700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48410a550 con 0x7fb48410aad0 2021-02-10T11:39:30.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.556 7fb48a209700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4840881d0 con 0x7fb4840dce80 2021-02-10T11:39:30.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.556 7fb48a209700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4840d1f30 con 0x7fb4840eb220 2021-02-10T11:39:30.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.556 7fb483fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb4840dce80 0x7fb4840d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:30.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.556 7fb488826700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb48410aad0 0x7fb48410aed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56160/0 (socket says 172.21.2.101:56160) 2021-02-10T11:39:30.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.557 7fb483fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb4840dce80 0x7fb4840d14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55662/0 (socket says 172.21.2.101:55662) 2021-02-10T11:39:30.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.557 7fb488826700 1 -- 172.21.2.101:0/3349594273 learned_addr learned my addr 172.21.2.101:0/3349594273 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:30.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.557 7fb4837fe700 1 --2- 172.21.2.101:0/3349594273 >> v2:172.21.2.108:3300/0 conn(0x7fb4840eb220 0x7fb4840eb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:30.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.557 7fb482ffd700 1 -- 172.21.2.101:0/3349594273 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3868914866 0 0) 0x7fb47c001500 con 0x7fb48410aad0 2021-02-10T11:39:30.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.557 7fb482ffd700 1 -- 172.21.2.101:0/3349594273 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb46c002fe0 con 0x7fb48410aad0 2021-02-10T11:39:30.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.557 7fb483fff700 1 -- 172.21.2.101:0/3349594273 >> v2:172.21.2.108:3300/0 conn(0x7fb4840eb220 msgr2=0x7fb4840eb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:30.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.557 7fb483fff700 1 --2- 172.21.2.101:0/3349594273 >> v2:172.21.2.108:3300/0 conn(0x7fb4840eb220 0x7fb4840eb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:30.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.557 7fb483fff700 1 -- 172.21.2.101:0/3349594273 >> v1:172.21.2.101:6789/0 conn(0x7fb48410aad0 legacy=0x7fb48410aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:30.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.557 7fb483fff700 1 -- 172.21.2.101:0/3349594273 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4840dd280 con 0x7fb4840dce80 2021-02-10T11:39:30.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.557 7fb483fff700 1 --2- 172.21.2.101:0/3349594273 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4840dce80 0x7fb4840d14f0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 rx=0x7fb474006650 tx=0x7fb474009a00).ready entity=mon.0 client_cookie=75c856f9906c91ae server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:30.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.557 7fb482ffd700 1 -- 172.21.2.101:0/3349594273 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb474009df0 con 0x7fb4840dce80 2021-02-10T11:39:30.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.557 7fb482ffd700 1 -- 172.21.2.101:0/3349594273 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4740009b0 con 0x7fb4840dce80 2021-02-10T11:39:30.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.557 7fb48a209700 1 -- 172.21.2.101:0/3349594273 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4840dce80 msgr2=0x7fb4840d14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:30.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.557 7fb48a209700 1 --2- 172.21.2.101:0/3349594273 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4840dce80 0x7fb4840d14f0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 rx=0x7fb474006650 tx=0x7fb474009a00).stop 2021-02-10T11:39:30.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.557 7fb482ffd700 1 -- 172.21.2.101:0/3349594273 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb47400c2b0 con 0x7fb4840dce80 2021-02-10T11:39:30.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.558 7fb48a209700 1 -- 172.21.2.101:0/3349594273 shutdown_connections 2021-02-10T11:39:30.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.558 7fb48a209700 1 --2- 172.21.2.101:0/3349594273 >> v2:172.21.2.108:3300/0 conn(0x7fb4840eb220 0x7fb4840eb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:30.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.558 7fb48a209700 1 --2- 172.21.2.101:0/3349594273 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4840dce80 0x7fb4840d14f0 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:30.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.558 7fb48a209700 1 -- 172.21.2.101:0/3349594273 >> 172.21.2.101:0/3349594273 conn(0x7fb4840ea630 msgr2=0x7fb4840d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:30.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.558 7fb48a209700 1 -- 172.21.2.101:0/3349594273 shutdown_connections 2021-02-10T11:39:30.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.558 7fb48a209700 1 -- 172.21.2.101:0/3349594273 wait complete. 2021-02-10T11:39:30.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.573 7fb48a209700 1 Processor -- start 2021-02-10T11:39:30.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.573 7fb48a209700 1 -- start start 2021-02-10T11:39:30.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.573 7fb48a209700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4840abd40 0x7fb484198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:30.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.573 7fb48a209700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4840fc550 0x7fb484198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:30.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.573 7fb48a209700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb48419ad30 0x7fb48419b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:30.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.573 7fb48a209700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48419b750 con 0x7fb4840fc550 2021-02-10T11:39:30.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.573 7fb48a209700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48419b970 con 0x7fb4840abd40 2021-02-10T11:39:30.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.573 7fb48a209700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48419bb90 con 0x7fb48419ad30 2021-02-10T11:39:30.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.574 7fb4837fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4840fc550 0x7fb484198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:30.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.574 7fb4837fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4840fc550 0x7fb484198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55670/0 (socket says 172.21.2.101:55670) 2021-02-10T11:39:30.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.574 7fb4837fe700 1 -- 172.21.2.101:0/4187938834 learned_addr learned my addr 172.21.2.101:0/4187938834 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:30.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.574 7fb488826700 1 --2- 172.21.2.101:0/4187938834 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb48419ad30 0x7fb48419b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:30.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.574 7fb483fff700 1 --2- 172.21.2.101:0/4187938834 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4840abd40 0x7fb484198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:30.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.574 7fb4837fe700 1 -- 172.21.2.101:0/4187938834 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb48419ad30 msgr2=0x7fb48419b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:30.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.574 7fb4837fe700 1 --2- 172.21.2.101:0/4187938834 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb48419ad30 0x7fb48419b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:30.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.574 7fb4837fe700 1 -- 172.21.2.101:0/4187938834 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4840abd40 msgr2=0x7fb484198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:30.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.574 7fb4837fe700 1 --2- 172.21.2.101:0/4187938834 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4840abd40 0x7fb484198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:30.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.574 7fb4837fe700 1 -- 172.21.2.101:0/4187938834 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb484199170 con 0x7fb4840fc550 2021-02-10T11:39:30.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.574 7fb4837fe700 1 --2- 172.21.2.101:0/4187938834 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4840fc550 0x7fb484198980 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 rx=0x7fb478006f60 tx=0x7fb47800b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:30.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.574 7fb480ff9700 1 -- 172.21.2.101:0/4187938834 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb47800bd60 con 0x7fb4840fc550 2021-02-10T11:39:30.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.575 7fb480ff9700 1 -- 172.21.2.101:0/4187938834 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb478000c90 con 0x7fb4840fc550 2021-02-10T11:39:30.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.575 7fb480ff9700 1 -- 172.21.2.101:0/4187938834 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb478002220 con 0x7fb4840fc550 2021-02-10T11:39:30.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.575 7fb48a209700 1 -- 172.21.2.101:0/4187938834 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb484199170 con 0x7fb4840fc550 2021-02-10T11:39:30.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.575 7fb48a209700 1 -- 172.21.2.101:0/4187938834 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb484199700 con 0x7fb4840fc550 2021-02-10T11:39:30.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.575 7fb480ff9700 1 -- 172.21.2.101:0/4187938834 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb478002020 con 0x7fb4840fc550 2021-02-10T11:39:30.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.575 7fb480ff9700 1 --2- 172.21.2.101:0/4187938834 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4640186c0 0x7fb46401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:30.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.575 7fb480ff9700 1 -- 172.21.2.101:0/4187938834 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb478032590 con 0x7fb4840fc550 2021-02-10T11:39:30.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.575 7fb483fff700 1 --2- 172.21.2.101:0/4187938834 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4640186c0 0x7fb46401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:30.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.576 7fb483fff700 1 --2- 172.21.2.101:0/4187938834 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4640186c0 0x7fb46401abc0 crc :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:30.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.606 7fb48a209700 1 -- 172.21.2.101:0/4187938834 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4840f93d0 con 0x7fb4840fc550 2021-02-10T11:39:30.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.608 7fb480ff9700 1 -- 172.21.2.101:0/4187938834 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb4780009b0 con 0x7fb4840fc550 2021-02-10T11:39:30.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.734 7fb48a209700 1 -- 172.21.2.101:0/4187938834 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume deauthorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000977298", "target": ["mgr", ""], "auth_id": "guest1", "group_name": "group_0000000000112255"}) v1 -- 0x7fb48419b750 con 0x7fb4640186c0 2021-02-10T11:39:30.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.764 7fb480ff9700 1 -- 172.21.2.101:0/4187938834 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fb474009320 con 0x7fb4640186c0 2021-02-10T11:39:30.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.767 7fb48a209700 1 -- 172.21.2.101:0/4187938834 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4640186c0 msgr2=0x7fb46401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:30.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.767 7fb48a209700 1 --2- 172.21.2.101:0/4187938834 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4640186c0 0x7fb46401abc0 crc :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:30.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.767 7fb48a209700 1 -- 172.21.2.101:0/4187938834 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4840fc550 msgr2=0x7fb484198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:30.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.767 7fb48a209700 1 --2- 172.21.2.101:0/4187938834 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4840fc550 0x7fb484198980 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 rx=0x7fb478006f60 tx=0x7fb47800b970).stop 2021-02-10T11:39:30.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.767 7fb48a209700 1 -- 172.21.2.101:0/4187938834 shutdown_connections 2021-02-10T11:39:30.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.767 7fb48a209700 1 --2- 172.21.2.101:0/4187938834 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4640186c0 0x7fb46401abc0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:30.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.767 7fb48a209700 1 --2- 172.21.2.101:0/4187938834 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb48419ad30 0x7fb48419b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:30.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.767 7fb48a209700 1 --2- 172.21.2.101:0/4187938834 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4840fc550 0x7fb484198980 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:30.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.767 7fb48a209700 1 --2- 172.21.2.101:0/4187938834 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4840abd40 0x7fb484198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:30.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.767 7fb48a209700 1 -- 172.21.2.101:0/4187938834 >> 172.21.2.101:0/4187938834 conn(0x7fb48419a820 msgr2=0x7fb484092be0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:30.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.767 7fb48a209700 1 -- 172.21.2.101:0/4187938834 shutdown_connections 2021-02-10T11:39:30.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:30.767 7fb48a209700 1 -- 172.21.2.101:0/4187938834 wait complete. 2021-02-10T11:39:30.779 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth get client.guest1 --format=json-pretty 2021-02-10T11:39:31.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.189 7f98b61e7700 1 Processor -- start 2021-02-10T11:39:31.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.189 7f98b61e7700 1 -- start start 2021-02-10T11:39:31.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.189 7f98b61e7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f98b00d1860 0x7f98b01b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:31.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.190 7f98b61e7700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b00f0b00 con 0x7f98b01b0a00 2021-02-10T11:39:31.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.190 7f98b61e7700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b00d0300 con 0x7f98b00f0640 2021-02-10T11:39:31.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.190 7f98b61e7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b00d0520 con 0x7f98b00d1860 2021-02-10T11:39:31.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.190 7f98af7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f98b01b0a00 0x7f98b00cf320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56170/0 (socket says 172.21.2.101:56170) 2021-02-10T11:39:31.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.190 7f98af7fe700 1 -- 172.21.2.101:0/1305213645 learned_addr learned my addr 172.21.2.101:0/1305213645 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:31.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.190 7f98affff700 1 --2- 172.21.2.101:0/1305213645 >> v2:172.21.2.108:3300/0 conn(0x7f98b00d1860 0x7f98b01b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:31.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.190 7f98aeffd700 1 -- 172.21.2.101:0/1305213645 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 28196777 0 0) 0x7f98a40010b0 con 0x7f98b01b0a00 2021-02-10T11:39:31.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.190 7f98aeffd700 1 -- 172.21.2.101:0/1305213645 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9898002fe0 con 0x7f98b01b0a00 2021-02-10T11:39:31.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.190 7f98affff700 1 -- 172.21.2.101:0/1305213645 >> v1:172.21.2.108:6789/0 conn(0x7f98b00f0640 legacy=0x7f98b00cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:31.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.190 7f98affff700 1 -- 172.21.2.101:0/1305213645 >> v1:172.21.2.101:6789/0 conn(0x7f98b01b0a00 legacy=0x7f98b00cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:31.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.190 7f98affff700 1 -- 172.21.2.101:0/1305213645 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b00d1c60 con 0x7f98b00d1860 2021-02-10T11:39:31.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.191 7f98affff700 1 --2- 172.21.2.101:0/1305213645 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98b00d1860 0x7f98b01b03e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f98a0006650 tx=0x7f98a0009a80).ready entity=mon.1 client_cookie=70dd8a23fdbc166c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:31.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.191 7f98aeffd700 1 -- 172.21.2.101:0/1305213645 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f98a000c0c0 con 0x7f98b00d1860 2021-02-10T11:39:31.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.191 7f98aeffd700 1 -- 172.21.2.101:0/1305213645 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a000c340 con 0x7f98b00d1860 2021-02-10T11:39:31.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.191 7f98aeffd700 1 -- 172.21.2.101:0/1305213645 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f98a000c860 con 0x7f98b00d1860 2021-02-10T11:39:31.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.191 7f98b61e7700 1 -- 172.21.2.101:0/1305213645 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98b00d1860 msgr2=0x7f98b01b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:31.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.191 7f98b61e7700 1 --2- 172.21.2.101:0/1305213645 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98b00d1860 0x7f98b01b03e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f98a0006650 tx=0x7f98a0009a80).stop 2021-02-10T11:39:31.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.191 7f98b61e7700 1 -- 172.21.2.101:0/1305213645 shutdown_connections 2021-02-10T11:39:31.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.191 7f98b61e7700 1 --2- 172.21.2.101:0/1305213645 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98b00d1860 0x7f98b01b03e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:31.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.191 7f98b61e7700 1 -- 172.21.2.101:0/1305213645 >> 172.21.2.101:0/1305213645 conn(0x7f98b00ce440 msgr2=0x7f98b00d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:31.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.192 7f98b61e7700 1 -- 172.21.2.101:0/1305213645 shutdown_connections 2021-02-10T11:39:31.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.192 7f98b61e7700 1 -- 172.21.2.101:0/1305213645 wait complete. 2021-02-10T11:39:31.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.206 7f98b61e7700 1 Processor -- start 2021-02-10T11:39:31.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.206 7f98b61e7700 1 -- start start 2021-02-10T11:39:31.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.207 7f98b61e7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98b00a7930 0x7f98b00f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:31.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.207 7f98b61e7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b00f23c0 0x7f98b00b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:31.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.207 7f98b61e7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98b00f27c0 0x7f98b00b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:31.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.207 7f98b61e7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b00f1820 con 0x7f98b00f23c0 2021-02-10T11:39:31.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.207 7f98b61e7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b008a8c0 con 0x7f98b00a7930 2021-02-10T11:39:31.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.207 7f98b61e7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b00f14f0 con 0x7f98b00f27c0 2021-02-10T11:39:31.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.207 7f98af7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b00f23c0 0x7f98b00b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:31.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.207 7f98af7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b00f23c0 0x7f98b00b4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55680/0 (socket says 172.21.2.101:55680) 2021-02-10T11:39:31.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.207 7f98af7fe700 1 -- 172.21.2.101:0/17122750 learned_addr learned my addr 172.21.2.101:0/17122750 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:31.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.207 7f98affff700 1 --2- 172.21.2.101:0/17122750 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98b00a7930 0x7f98b00f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:31.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.207 7f98b4804700 1 --2- 172.21.2.101:0/17122750 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98b00f27c0 0x7f98b00b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:31.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.207 7f98af7fe700 1 -- 172.21.2.101:0/17122750 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98b00f27c0 msgr2=0x7f98b00b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:31.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.207 7f98af7fe700 1 --2- 172.21.2.101:0/17122750 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98b00f27c0 0x7f98b00b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:31.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.207 7f98af7fe700 1 -- 172.21.2.101:0/17122750 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98b00a7930 msgr2=0x7f98b00f1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:31.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.207 7f98af7fe700 1 --2- 172.21.2.101:0/17122750 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98b00a7930 0x7f98b00f1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:31.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.207 7f98af7fe700 1 -- 172.21.2.101:0/17122750 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b008a420 con 0x7f98b00f23c0 2021-02-10T11:39:31.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.207 7f98affff700 1 --2- 172.21.2.101:0/17122750 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98b00a7930 0x7f98b00f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:39:31.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.208 7f98af7fe700 1 --2- 172.21.2.101:0/17122750 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b00f23c0 0x7f98b00b4840 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 rx=0x7f98a4001c60 tx=0x7f98a400a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:31.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.208 7f98acff9700 1 -- 172.21.2.101:0/17122750 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f98a400acc0 con 0x7f98b00f23c0 2021-02-10T11:39:31.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.208 7f98acff9700 1 -- 172.21.2.101:0/17122750 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a4004350 con 0x7f98b00f23c0 2021-02-10T11:39:31.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.208 7f98b61e7700 1 -- 172.21.2.101:0/17122750 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b008a420 con 0x7f98b00f23c0 2021-02-10T11:39:31.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.208 7f98acff9700 1 -- 172.21.2.101:0/17122750 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f98a40047b0 con 0x7f98b00f23c0 2021-02-10T11:39:31.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.208 7f98b61e7700 1 -- 172.21.2.101:0/17122750 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b00b5770 con 0x7f98b00f23c0 2021-02-10T11:39:31.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.209 7f98acff9700 1 -- 172.21.2.101:0/17122750 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f98a4007f30 con 0x7f98b00f23c0 2021-02-10T11:39:31.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.209 7f98acff9700 1 --2- 172.21.2.101:0/17122750 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98900186c0 0x7f989001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:31.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.209 7f98acff9700 1 -- 172.21.2.101:0/17122750 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f98a40323a0 con 0x7f98b00f23c0 2021-02-10T11:39:31.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.209 7f98affff700 1 --2- 172.21.2.101:0/17122750 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98900186c0 0x7f989001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:31.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.209 7f98affff700 1 --2- 172.21.2.101:0/17122750 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98900186c0 0x7f989001abc0 crc :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:31.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.239 7f98b61e7700 1 -- 172.21.2.101:0/17122750 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b00ee060 con 0x7f98b00f23c0 2021-02-10T11:39:31.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.242 7f98acff9700 1 -- 172.21.2.101:0/17122750 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f98a4022c70 con 0x7f98b00f23c0 2021-02-10T11:39:31.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.372 7f98b61e7700 1 -- 172.21.2.101:0/17122750 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.guest1", "format": "json-pretty"} v 0) v1 -- 0x7f98b00ce440 con 0x7f98b00f23c0 2021-02-10T11:39:31.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.373 7f98acff9700 1 -- 172.21.2.101:0/17122750 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.guest1", "format": "json-pretty"}]=0 exported keyring for client.guest1 v59) v1 ==== 142+0+235 (secure 0 0 0) 0x7f98a4022c70 con 0x7f98b00f23c0 2021-02-10T11:39:31.377 INFO:teuthology.orchestra.run.gibba001.stderr:exported keyring for client.guest1 2021-02-10T11:39:31.377 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:39:31.377 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:39:31.377 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:31.377 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.guest1", 2021-02-10T11:39:31.377 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQDwxSNgoURPNBAA3WTD9XIbUFJUSQXQj9SWWw==", 2021-02-10T11:39:31.377 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:39:31.377 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow rw path=/volumes/group_0000000000112255", 2021-02-10T11:39:31.378 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *" 2021-02-10T11:39:31.378 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:31.378 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:31.378 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:39:31.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.374 7f98b61e7700 1 -- 172.21.2.101:0/17122750 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98900186c0 msgr2=0x7f989001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:31.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.375 7f98b61e7700 1 --2- 172.21.2.101:0/17122750 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98900186c0 0x7f989001abc0 crc :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:31.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.375 7f98b61e7700 1 -- 172.21.2.101:0/17122750 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b00f23c0 msgr2=0x7f98b00b4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:31.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.375 7f98b61e7700 1 --2- 172.21.2.101:0/17122750 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b00f23c0 0x7f98b00b4840 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 rx=0x7f98a4001c60 tx=0x7f98a400a8d0).stop 2021-02-10T11:39:31.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.375 7f98b61e7700 1 -- 172.21.2.101:0/17122750 shutdown_connections 2021-02-10T11:39:31.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.375 7f98b61e7700 1 --2- 172.21.2.101:0/17122750 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98900186c0 0x7f989001abc0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:31.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.375 7f98b61e7700 1 --2- 172.21.2.101:0/17122750 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98b00f27c0 0x7f98b00b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:31.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.375 7f98b61e7700 1 --2- 172.21.2.101:0/17122750 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b00f23c0 0x7f98b00b4840 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:31.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.375 7f98b61e7700 1 --2- 172.21.2.101:0/17122750 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98b00a7930 0x7f98b00f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:31.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.375 7f98b61e7700 1 -- 172.21.2.101:0/17122750 >> 172.21.2.101:0/17122750 conn(0x7f98b00bea70 msgr2=0x7f98b0095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:31.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.375 7f98b61e7700 1 -- 172.21.2.101:0/17122750 shutdown_connections 2021-02-10T11:39:31.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.375 7f98b61e7700 1 -- 172.21.2.101:0/17122750 wait complete. 2021-02-10T11:39:31.387 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth rm client.guest1 2021-02-10T11:39:31.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.794 7f3c5f6fa700 1 Processor -- start 2021-02-10T11:39:31.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.794 7f3c5f6fa700 1 -- start start 2021-02-10T11:39:31.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.795 7f3c5f6fa700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3c580d1870 0x7f3c581b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:31.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.795 7f3c5f6fa700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3c581b0a10 0x7f3c580cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:31.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.795 7f3c5f6fa700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c580d1c70 0x7f3c580cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:31.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.795 7f3c5f6fa700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c580e5ed0 con 0x7f3c581b0a10 2021-02-10T11:39:31.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.795 7f3c5f6fa700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c580f0b10 con 0x7f3c580d1870 2021-02-10T11:39:31.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.795 7f3c5f6fa700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c580cff70 con 0x7f3c580d1c70 2021-02-10T11:39:31.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.795 7f3c5cd15700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3c581b0a10 0x7f3c580cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:31.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.795 7f3c5dd17700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c580d1c70 0x7f3c580cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:31.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.795 7f3c5cd15700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3c581b0a10 0x7f3c580cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55688/0 (socket says 172.21.2.101:55688) 2021-02-10T11:39:31.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.795 7f3c5cd15700 1 -- 172.21.2.101:0/3208754364 learned_addr learned my addr 172.21.2.101:0/3208754364 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:31.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.795 7f3c5d516700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3c580d1870 0x7f3c581b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:31.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.795 7f3c5d516700 1 -- 172.21.2.101:0/3208754364 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c580d1c70 msgr2=0x7f3c580cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:31.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.795 7f3c5d516700 1 --2- 172.21.2.101:0/3208754364 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c580d1c70 0x7f3c580cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:31.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.796 7f3c5d516700 1 -- 172.21.2.101:0/3208754364 >> v2:172.21.2.101:3300/0 conn(0x7f3c581b0a10 msgr2=0x7f3c580cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:31.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.796 7f3c5d516700 1 --2- 172.21.2.101:0/3208754364 >> v2:172.21.2.101:3300/0 conn(0x7f3c581b0a10 0x7f3c580cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:31.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.796 7f3c5d516700 1 -- 172.21.2.101:0/3208754364 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c580d0150 con 0x7f3c580d1870 2021-02-10T11:39:31.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.796 7f3c5cd15700 1 --2- 172.21.2.101:0/3208754364 >> v2:172.21.2.101:3300/0 conn(0x7f3c581b0a10 0x7f3c580cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:39:31.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.796 7f3c5d516700 1 --2- 172.21.2.101:0/3208754364 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c580d1870 0x7f3c581b03f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f3c54006650 tx=0x7f3c54009a80).ready entity=mon.1 client_cookie=a231a93f4bf8dcc5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:31.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.796 7f3c4bfff700 1 -- 172.21.2.101:0/3208754364 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3c540009b0 con 0x7f3c580d1870 2021-02-10T11:39:31.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.796 7f3c4bfff700 1 -- 172.21.2.101:0/3208754364 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5400c0c0 con 0x7f3c580d1870 2021-02-10T11:39:31.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.796 7f3c4bfff700 1 -- 172.21.2.101:0/3208754364 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3c5400c510 con 0x7f3c580d1870 2021-02-10T11:39:31.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.796 7f3c5f6fa700 1 -- 172.21.2.101:0/3208754364 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c580d1870 msgr2=0x7f3c581b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:31.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.796 7f3c5f6fa700 1 --2- 172.21.2.101:0/3208754364 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c580d1870 0x7f3c581b03f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f3c54006650 tx=0x7f3c54009a80).stop 2021-02-10T11:39:31.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.797 7f3c5f6fa700 1 -- 172.21.2.101:0/3208754364 shutdown_connections 2021-02-10T11:39:31.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.797 7f3c5f6fa700 1 --2- 172.21.2.101:0/3208754364 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c580d1c70 0x7f3c580cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:31.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.797 7f3c5f6fa700 1 --2- 172.21.2.101:0/3208754364 >> v2:172.21.2.101:3300/0 conn(0x7f3c581b0a10 0x7f3c580cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:31.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.797 7f3c5f6fa700 1 --2- 172.21.2.101:0/3208754364 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c580d1870 0x7f3c581b03f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:31.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.797 7f3c5f6fa700 1 -- 172.21.2.101:0/3208754364 >> 172.21.2.101:0/3208754364 conn(0x7f3c580ce450 msgr2=0x7f3c580d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:31.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.797 7f3c5f6fa700 1 -- 172.21.2.101:0/3208754364 shutdown_connections 2021-02-10T11:39:31.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.797 7f3c5f6fa700 1 -- 172.21.2.101:0/3208754364 wait complete. 2021-02-10T11:39:31.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.812 7f3c5f6fa700 1 Processor -- start 2021-02-10T11:39:31.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.812 7f3c5f6fa700 1 -- start start 2021-02-10T11:39:31.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.812 7f3c5f6fa700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c5819ffa0 0x7f3c580eda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:31.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.812 7f3c5f6fa700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c581a03a0 0x7f3c580ee030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:31.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.812 7f3c5f6fa700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c580ee650 0x7f3c580ebe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:31.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.812 7f3c5f6fa700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c580ed490 con 0x7f3c580ee650 2021-02-10T11:39:31.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.812 7f3c5f6fa700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c580eea50 con 0x7f3c5819ffa0 2021-02-10T11:39:31.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.812 7f3c5f6fa700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c580eec30 con 0x7f3c581a03a0 2021-02-10T11:39:31.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.813 7f3c5dd17700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c580ee650 0x7f3c580ebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:31.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.813 7f3c5dd17700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c580ee650 0x7f3c580ebe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55696/0 (socket says 172.21.2.101:55696) 2021-02-10T11:39:31.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.813 7f3c5dd17700 1 -- 172.21.2.101:0/1644331821 learned_addr learned my addr 172.21.2.101:0/1644331821 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:31.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.813 7f3c5d516700 1 --2- 172.21.2.101:0/1644331821 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c5819ffa0 0x7f3c580eda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:31.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.813 7f3c5cd15700 1 --2- 172.21.2.101:0/1644331821 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c581a03a0 0x7f3c580ee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:31.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.813 7f3c5dd17700 1 -- 172.21.2.101:0/1644331821 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c581a03a0 msgr2=0x7f3c580ee030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:31.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.813 7f3c5dd17700 1 --2- 172.21.2.101:0/1644331821 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c581a03a0 0x7f3c580ee030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:31.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.813 7f3c5dd17700 1 -- 172.21.2.101:0/1644331821 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c5819ffa0 msgr2=0x7f3c580eda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:31.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.813 7f3c5dd17700 1 --2- 172.21.2.101:0/1644331821 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c5819ffa0 0x7f3c580eda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:31.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.813 7f3c5dd17700 1 -- 172.21.2.101:0/1644331821 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c580ec4a0 con 0x7f3c580ee650 2021-02-10T11:39:31.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.813 7f3c5cd15700 1 --2- 172.21.2.101:0/1644331821 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c581a03a0 0x7f3c580ee030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:39:31.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.813 7f3c5dd17700 1 --2- 172.21.2.101:0/1644331821 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c580ee650 0x7f3c580ebe80 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 rx=0x7f3c5000b970 tx=0x7f3c5000bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:31.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.813 7f3c49ffb700 1 -- 172.21.2.101:0/1644331821 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3c50000f90 con 0x7f3c580ee650 2021-02-10T11:39:31.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.813 7f3c49ffb700 1 -- 172.21.2.101:0/1644331821 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c50003030 con 0x7f3c580ee650 2021-02-10T11:39:31.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.813 7f3c49ffb700 1 -- 172.21.2.101:0/1644331821 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3c50001680 con 0x7f3c580ee650 2021-02-10T11:39:31.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.813 7f3c5f6fa700 1 -- 172.21.2.101:0/1644331821 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c580ec4a0 con 0x7f3c580ee650 2021-02-10T11:39:31.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.813 7f3c5f6fa700 1 -- 172.21.2.101:0/1644331821 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c580ec4a0 con 0x7f3c580ee650 2021-02-10T11:39:31.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.814 7f3c49ffb700 1 -- 172.21.2.101:0/1644331821 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3c50000df0 con 0x7f3c580ee650 2021-02-10T11:39:31.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.814 7f3c49ffb700 1 --2- 172.21.2.101:0/1644331821 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3c380186c0 0x7f3c3801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:31.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.814 7f3c49ffb700 1 -- 172.21.2.101:0/1644331821 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3c500328e0 con 0x7f3c580ee650 2021-02-10T11:39:31.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.814 7f3c5d516700 1 --2- 172.21.2.101:0/1644331821 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3c380186c0 0x7f3c3801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:31.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.814 7f3c5d516700 1 --2- 172.21.2.101:0/1644331821 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3c380186c0 0x7f3c3801abc0 crc :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:31.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.845 7f3c5f6fa700 1 -- 172.21.2.101:0/1644331821 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c580ebc50 con 0x7f3c580ee650 2021-02-10T11:39:31.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.848 7f3c49ffb700 1 -- 172.21.2.101:0/1644331821 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3c50002f30 con 0x7f3c580ee650 2021-02-10T11:39:31.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.979 7f3c5f6fa700 1 -- 172.21.2.101:0/1644331821 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth rm", "entity": "client.guest1"} v 0) v1 -- 0x7f3c580ebc50 con 0x7f3c580ee650 2021-02-10T11:39:31.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.982 7f3c49ffb700 1 -- 172.21.2.101:0/1644331821 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth rm", "entity": "client.guest1"}]=0 updated v60) v1 ==== 89+0+0 (secure 0 0 0) 0x7f3c50002f30 con 0x7f3c580ee650 2021-02-10T11:39:31.989 INFO:teuthology.orchestra.run.gibba001.stderr:updated 2021-02-10T11:39:31.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.987 7f3c5f6fa700 1 -- 172.21.2.101:0/1644331821 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3c380186c0 msgr2=0x7f3c3801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:31.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.987 7f3c5f6fa700 1 --2- 172.21.2.101:0/1644331821 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3c380186c0 0x7f3c3801abc0 crc :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:31.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.987 7f3c5f6fa700 1 -- 172.21.2.101:0/1644331821 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c580ee650 msgr2=0x7f3c580ebe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:31.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.987 7f3c5f6fa700 1 --2- 172.21.2.101:0/1644331821 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c580ee650 0x7f3c580ebe80 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 rx=0x7f3c5000b970 tx=0x7f3c5000bd10).stop 2021-02-10T11:39:31.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.988 7f3c5f6fa700 1 -- 172.21.2.101:0/1644331821 shutdown_connections 2021-02-10T11:39:31.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.988 7f3c5f6fa700 1 --2- 172.21.2.101:0/1644331821 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3c380186c0 0x7f3c3801abc0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:31.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.988 7f3c5f6fa700 1 --2- 172.21.2.101:0/1644331821 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c581a03a0 0x7f3c580ee030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:31.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.988 7f3c5f6fa700 1 --2- 172.21.2.101:0/1644331821 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c580ee650 0x7f3c580ebe80 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:31.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.988 7f3c5f6fa700 1 --2- 172.21.2.101:0/1644331821 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c5819ffa0 0x7f3c580eda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:31.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.988 7f3c5f6fa700 1 -- 172.21.2.101:0/1644331821 >> 172.21.2.101:0/1644331821 conn(0x7f3c58094d90 msgr2=0x7f3c5808a420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:31.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.988 7f3c5f6fa700 1 -- 172.21.2.101:0/1644331821 shutdown_connections 2021-02-10T11:39:31.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:31.988 7f3c5f6fa700 1 -- 172.21.2.101:0/1644331821 wait complete. 2021-02-10T11:39:31.999 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000977298 --group_name group_0000000000112255 2021-02-10T11:39:32.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.411 7f9b36cac700 1 Processor -- start 2021-02-10T11:39:32.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.411 7f9b36cac700 1 -- start start 2021-02-10T11:39:32.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.411 7f9b36cac700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f9b300d1870 0x7f9b301b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:32.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.411 7f9b36cac700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9b301b0ae0 0x7f9b30108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:32.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.411 7f9b36cac700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b300fe1c0 con 0x7f9b300d1c70 2021-02-10T11:39:32.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.411 7f9b36cac700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b30109a60 con 0x7f9b301b0ae0 2021-02-10T11:39:32.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.411 7f9b36cac700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b30109750 con 0x7f9b300d1870 2021-02-10T11:39:32.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.411 7f9b2ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9b301b0ae0 0x7f9b30108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:32.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.411 7f9b352c9700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f9b300d1c70 0x7f9b30108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56200/0 (socket says 172.21.2.101:56200) 2021-02-10T11:39:32.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.411 7f9b352c9700 1 -- 172.21.2.101:0/3182743657 learned_addr learned my addr 172.21.2.101:0/3182743657 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:32.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.412 7f9b2f7fe700 1 -- 172.21.2.101:0/3182743657 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3475184108 0 0) 0x7f9b28001500 con 0x7f9b300d1c70 2021-02-10T11:39:32.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.412 7f9b2f7fe700 1 -- 172.21.2.101:0/3182743657 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b1c002fe0 con 0x7f9b300d1c70 2021-02-10T11:39:32.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.412 7f9b2ffff700 1 -- 172.21.2.101:0/3182743657 >> v2:172.21.2.108:3300/0 conn(0x7f9b300d1870 msgr2=0x7f9b301b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:32.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.412 7f9b2ffff700 1 --2- 172.21.2.101:0/3182743657 >> v2:172.21.2.108:3300/0 conn(0x7f9b300d1870 0x7f9b301b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:32.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.412 7f9b2ffff700 1 -- 172.21.2.101:0/3182743657 >> v1:172.21.2.101:6789/0 conn(0x7f9b300d1c70 legacy=0x7f9b30108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:32.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.412 7f9b2f7fe700 1 -- 172.21.2.101:0/3182743657 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2256912740 0 0) 0x7f9b28001460 con 0x7f9b300d1c70 2021-02-10T11:39:32.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.412 7f9b2ffff700 1 -- 172.21.2.101:0/3182743657 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b30088070 con 0x7f9b301b0ae0 2021-02-10T11:39:32.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.412 7f9b2ffff700 1 --2- 172.21.2.101:0/3182743657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b301b0ae0 0x7f9b30108940 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 rx=0x7f9b20006650 tx=0x7f9b20009a00).ready entity=mon.0 client_cookie=5e4aba06feccc257 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:32.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.412 7f9b2f7fe700 1 -- 172.21.2.101:0/3182743657 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9b20009df0 con 0x7f9b301b0ae0 2021-02-10T11:39:32.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.412 7f9b2f7fe700 1 -- 172.21.2.101:0/3182743657 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b200009b0 con 0x7f9b301b0ae0 2021-02-10T11:39:32.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.412 7f9b2f7fe700 1 -- 172.21.2.101:0/3182743657 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9b2000c2b0 con 0x7f9b301b0ae0 2021-02-10T11:39:32.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.412 7f9b36cac700 1 -- 172.21.2.101:0/3182743657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b301b0ae0 msgr2=0x7f9b30108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:32.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.412 7f9b36cac700 1 --2- 172.21.2.101:0/3182743657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b301b0ae0 0x7f9b30108940 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 rx=0x7f9b20006650 tx=0x7f9b20009a00).stop 2021-02-10T11:39:32.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.413 7f9b36cac700 1 -- 172.21.2.101:0/3182743657 shutdown_connections 2021-02-10T11:39:32.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.413 7f9b36cac700 1 --2- 172.21.2.101:0/3182743657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b301b0ae0 0x7f9b30108940 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:32.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.413 7f9b36cac700 1 --2- 172.21.2.101:0/3182743657 >> v2:172.21.2.108:3300/0 conn(0x7f9b300d1870 0x7f9b301b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:32.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.413 7f9b36cac700 1 -- 172.21.2.101:0/3182743657 >> 172.21.2.101:0/3182743657 conn(0x7f9b300ce420 msgr2=0x7f9b300d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:32.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.413 7f9b36cac700 1 -- 172.21.2.101:0/3182743657 shutdown_connections 2021-02-10T11:39:32.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.413 7f9b36cac700 1 -- 172.21.2.101:0/3182743657 wait complete. 2021-02-10T11:39:32.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.428 7f9b36cac700 1 Processor -- start 2021-02-10T11:39:32.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.428 7f9b36cac700 1 -- start start 2021-02-10T11:39:32.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.428 7f9b36cac700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b3019e5c0 0x7f9b30099fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:32.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.428 7f9b36cac700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b3009a5d0 0x7f9b3009a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:32.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.428 7f9b36cac700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b300982a0 0x7f9b300986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:32.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.428 7f9b36cac700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3019feb0 con 0x7f9b3009a5d0 2021-02-10T11:39:32.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.428 7f9b36cac700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b30099a30 con 0x7f9b300982a0 2021-02-10T11:39:32.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.428 7f9b36cac700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3019fb30 con 0x7f9b3019e5c0 2021-02-10T11:39:32.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.428 7f9b2ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b3009a5d0 0x7f9b3009a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:32.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.428 7f9b2ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b3009a5d0 0x7f9b3009a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55712/0 (socket says 172.21.2.101:55712) 2021-02-10T11:39:32.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.428 7f9b2ffff700 1 -- 172.21.2.101:0/2458583936 learned_addr learned my addr 172.21.2.101:0/2458583936 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:32.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.429 7f9b2ffff700 1 -- 172.21.2.101:0/2458583936 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b3019e5c0 msgr2=0x7f9b30099fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:32.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.429 7f9b2ffff700 1 --2- 172.21.2.101:0/2458583936 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b3019e5c0 0x7f9b30099fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:32.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.429 7f9b2ffff700 1 -- 172.21.2.101:0/2458583936 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b300982a0 msgr2=0x7f9b300986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:32.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.429 7f9b2ffff700 1 --2- 172.21.2.101:0/2458583936 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b300982a0 0x7f9b300986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:32.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.429 7f9b2ffff700 1 -- 172.21.2.101:0/2458583936 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3019e9c0 con 0x7f9b3009a5d0 2021-02-10T11:39:32.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.429 7f9b2ffff700 1 --2- 172.21.2.101:0/2458583936 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b3009a5d0 0x7f9b3009a9d0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 rx=0x7f9b200096a0 tx=0x7f9b2000b9b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:32.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.429 7f9b2d7fa700 1 -- 172.21.2.101:0/2458583936 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9b20000940 con 0x7f9b3009a5d0 2021-02-10T11:39:32.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.429 7f9b2d7fa700 1 -- 172.21.2.101:0/2458583936 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b200093a0 con 0x7f9b3009a5d0 2021-02-10T11:39:32.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.429 7f9b2d7fa700 1 -- 172.21.2.101:0/2458583936 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9b200022b0 con 0x7f9b3009a5d0 2021-02-10T11:39:32.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.429 7f9b36cac700 1 -- 172.21.2.101:0/2458583936 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b3019e9c0 con 0x7f9b3009a5d0 2021-02-10T11:39:32.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.429 7f9b36cac700 1 -- 172.21.2.101:0/2458583936 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b30098fb0 con 0x7f9b3009a5d0 2021-02-10T11:39:32.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.430 7f9b2d7fa700 1 -- 172.21.2.101:0/2458583936 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9b20002240 con 0x7f9b3009a5d0 2021-02-10T11:39:32.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.430 7f9b2d7fa700 1 --2- 172.21.2.101:0/2458583936 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b140186c0 0x7f9b1401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:32.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.430 7f9b2d7fa700 1 -- 172.21.2.101:0/2458583936 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9b20032590 con 0x7f9b3009a5d0 2021-02-10T11:39:32.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.430 7f9b34ac8700 1 --2- 172.21.2.101:0/2458583936 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b140186c0 0x7f9b1401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:32.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.430 7f9b34ac8700 1 --2- 172.21.2.101:0/2458583936 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b140186c0 0x7f9b1401abc0 crc :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:32.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.461 7f9b36cac700 1 -- 172.21.2.101:0/2458583936 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b300825d0 con 0x7f9b3009a5d0 2021-02-10T11:39:32.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.463 7f9b2d7fa700 1 -- 172.21.2.101:0/2458583936 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9b20002030 con 0x7f9b3009a5d0 2021-02-10T11:39:32.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.590 7f9b36cac700 1 -- 172.21.2.101:0/2458583936 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000977298", "target": ["mgr", ""], "group_name": "group_0000000000112255"}) v1 -- 0x7f9b30098fb0 con 0x7f9b140186c0 2021-02-10T11:39:32.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.598 7f9b2d7fa700 1 -- 172.21.2.101:0/2458583936 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f9b18006960 con 0x7f9b140186c0 2021-02-10T11:39:32.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.607 7f9b36cac700 1 -- 172.21.2.101:0/2458583936 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b140186c0 msgr2=0x7f9b1401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:32.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.607 7f9b36cac700 1 --2- 172.21.2.101:0/2458583936 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b140186c0 0x7f9b1401abc0 crc :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:32.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.607 7f9b36cac700 1 -- 172.21.2.101:0/2458583936 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b3009a5d0 msgr2=0x7f9b3009a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:32.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.607 7f9b36cac700 1 --2- 172.21.2.101:0/2458583936 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b3009a5d0 0x7f9b3009a9d0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 rx=0x7f9b200096a0 tx=0x7f9b2000b9b0).stop 2021-02-10T11:39:32.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.607 7f9b36cac700 1 -- 172.21.2.101:0/2458583936 shutdown_connections 2021-02-10T11:39:32.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.607 7f9b36cac700 1 --2- 172.21.2.101:0/2458583936 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b300982a0 0x7f9b300986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:32.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.607 7f9b36cac700 1 --2- 172.21.2.101:0/2458583936 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b3009a5d0 0x7f9b3009a9d0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:32.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.607 7f9b36cac700 1 --2- 172.21.2.101:0/2458583936 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b140186c0 0x7f9b1401abc0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:32.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.607 7f9b36cac700 1 --2- 172.21.2.101:0/2458583936 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b3019e5c0 0x7f9b30099fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:32.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.607 7f9b36cac700 1 -- 172.21.2.101:0/2458583936 >> 172.21.2.101:0/2458583936 conn(0x7f9b301b3560 msgr2=0x7f9b301b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:32.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.607 7f9b36cac700 1 -- 172.21.2.101:0/2458583936 shutdown_connections 2021-02-10T11:39:32.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:32.607 7f9b36cac700 1 -- 172.21.2.101:0/2458583936 wait complete. 2021-02-10T11:39:32.619 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolumegroup rm cephfs group_0000000000112255 2021-02-10T11:39:33.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.032 7ff6eca73700 1 Processor -- start 2021-02-10T11:39:33.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.032 7ff6eca73700 1 -- start start 2021-02-10T11:39:33.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.032 7ff6eca73700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6e80d1870 0x7ff6e81b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:33.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.032 7ff6eca73700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff6e80cff70 0x7ff6e8109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:33.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.032 7ff6eca73700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e8109a60 con 0x7ff6e81b0ae0 2021-02-10T11:39:33.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.032 7ff6eca73700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e8109750 con 0x7ff6e80cff70 2021-02-10T11:39:33.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.032 7ff6eca73700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e8088070 con 0x7ff6e80d1870 2021-02-10T11:39:33.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.032 7ff6e605d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff6e81b0ae0 0x7ff6e8108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56212/0 (socket says 172.21.2.101:56212) 2021-02-10T11:39:33.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.032 7ff6e605d700 1 -- 172.21.2.101:0/3093043920 learned_addr learned my addr 172.21.2.101:0/3093043920 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:33.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.032 7ff6e685e700 1 --2- 172.21.2.101:0/3093043920 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6e80d1870 0x7ff6e81b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:33.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.032 7ff6e705f700 1 --2- 172.21.2.101:0/3093043920 >> v2:172.21.2.108:3300/0 conn(0x7ff6e80cff70 0x7ff6e8109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:33.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.033 7ff6e585c700 1 -- 172.21.2.101:0/3093043920 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378667932 0 0) 0x7ff6d00010b0 con 0x7ff6e81b0ae0 2021-02-10T11:39:33.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.033 7ff6e585c700 1 -- 172.21.2.101:0/3093043920 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6d4002fe0 con 0x7ff6e81b0ae0 2021-02-10T11:39:33.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.033 7ff6e685e700 1 -- 172.21.2.101:0/3093043920 >> v2:172.21.2.108:3300/0 conn(0x7ff6e80cff70 msgr2=0x7ff6e8109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:33.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.033 7ff6e685e700 1 --2- 172.21.2.101:0/3093043920 >> v2:172.21.2.108:3300/0 conn(0x7ff6e80cff70 0x7ff6e8109130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:33.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.033 7ff6e685e700 1 -- 172.21.2.101:0/3093043920 >> v1:172.21.2.101:6789/0 conn(0x7ff6e81b0ae0 legacy=0x7ff6e8108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:33.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.033 7ff6e685e700 1 -- 172.21.2.101:0/3093043920 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e8088260 con 0x7ff6e80d1870 2021-02-10T11:39:33.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.033 7ff6e705f700 1 --2- 172.21.2.101:0/3093043920 >> v2:172.21.2.108:3300/0 conn(0x7ff6e80cff70 0x7ff6e8109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:39:33.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.033 7ff6e685e700 1 --2- 172.21.2.101:0/3093043920 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6e80d1870 0x7ff6e81b04c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7ff6d8006650 tx=0x7ff6d8009a80).ready entity=mon.2 client_cookie=f01bbcde19adfa99 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:33.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.033 7ff6e585c700 1 -- 172.21.2.101:0/3093043920 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff6d8006390 con 0x7ff6e80d1870 2021-02-10T11:39:33.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.033 7ff6e585c700 1 -- 172.21.2.101:0/3093043920 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d80009b0 con 0x7ff6e80d1870 2021-02-10T11:39:33.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.033 7ff6eca73700 1 -- 172.21.2.101:0/3093043920 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6e80d1870 msgr2=0x7ff6e81b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:33.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.033 7ff6eca73700 1 --2- 172.21.2.101:0/3093043920 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6e80d1870 0x7ff6e81b04c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7ff6d8006650 tx=0x7ff6d8009a80).stop 2021-02-10T11:39:33.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.033 7ff6e585c700 1 -- 172.21.2.101:0/3093043920 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff6d800c2b0 con 0x7ff6e80d1870 2021-02-10T11:39:33.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.034 7ff6eca73700 1 -- 172.21.2.101:0/3093043920 shutdown_connections 2021-02-10T11:39:33.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.034 7ff6eca73700 1 --2- 172.21.2.101:0/3093043920 >> v2:172.21.2.108:3300/0 conn(0x7ff6e80cff70 0x7ff6e8109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:33.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.034 7ff6eca73700 1 --2- 172.21.2.101:0/3093043920 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6e80d1870 0x7ff6e81b04c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:33.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.034 7ff6eca73700 1 -- 172.21.2.101:0/3093043920 >> 172.21.2.101:0/3093043920 conn(0x7ff6e80ce420 msgr2=0x7ff6e80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:33.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.034 7ff6eca73700 1 -- 172.21.2.101:0/3093043920 shutdown_connections 2021-02-10T11:39:33.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.034 7ff6eca73700 1 -- 172.21.2.101:0/3093043920 wait complete. 2021-02-10T11:39:33.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.048 7ff6eca73700 1 Processor -- start 2021-02-10T11:39:33.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.048 7ff6eca73700 1 -- start start 2021-02-10T11:39:33.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.049 7ff6eca73700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6e80a58c0 0x7ff6e80a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:33.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.049 7ff6eca73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6e80b1ba0 0x7ff6e80b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:33.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.049 7ff6eca73700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6e80b25c0 0x7ff6e80b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:33.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.049 7ff6eca73700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80b2a40 con 0x7ff6e80b1ba0 2021-02-10T11:39:33.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.049 7ff6eca73700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80a5340 con 0x7ff6e80b25c0 2021-02-10T11:39:33.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.049 7ff6eca73700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80a5560 con 0x7ff6e80a58c0 2021-02-10T11:39:33.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.049 7ff6e605d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6e80b1ba0 0x7ff6e80b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:33.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.049 7ff6e605d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6e80b1ba0 0x7ff6e80b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55726/0 (socket says 172.21.2.101:55726) 2021-02-10T11:39:33.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.049 7ff6e605d700 1 -- 172.21.2.101:0/1360342741 learned_addr learned my addr 172.21.2.101:0/1360342741 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:33.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.049 7ff6e705f700 1 --2- 172.21.2.101:0/1360342741 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6e80b25c0 0x7ff6e80b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:33.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.049 7ff6e685e700 1 --2- 172.21.2.101:0/1360342741 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6e80a58c0 0x7ff6e80a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:33.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.049 7ff6e705f700 1 -- 172.21.2.101:0/1360342741 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6e80a58c0 msgr2=0x7ff6e80a4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:33.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.049 7ff6e705f700 1 --2- 172.21.2.101:0/1360342741 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6e80a58c0 0x7ff6e80a4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:33.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.049 7ff6e705f700 1 -- 172.21.2.101:0/1360342741 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6e80b1ba0 msgr2=0x7ff6e80b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:33.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.049 7ff6e705f700 1 --2- 172.21.2.101:0/1360342741 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6e80b1ba0 0x7ff6e80b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:33.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.049 7ff6e705f700 1 -- 172.21.2.101:0/1360342741 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e80b3890 con 0x7ff6e80b25c0 2021-02-10T11:39:33.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.049 7ff6e605d700 1 --2- 172.21.2.101:0/1360342741 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6e80b1ba0 0x7ff6e80b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:39:33.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.050 7ff6e705f700 1 --2- 172.21.2.101:0/1360342741 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6e80b25c0 0x7ff6e80b3270 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7ff6e000b8a0 tx=0x7ff6e000bc40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:33.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.050 7ff6df7fe700 1 -- 172.21.2.101:0/1360342741 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff6e00066f0 con 0x7ff6e80b25c0 2021-02-10T11:39:33.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.050 7ff6df7fe700 1 -- 172.21.2.101:0/1360342741 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e00011f0 con 0x7ff6e80b25c0 2021-02-10T11:39:33.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.050 7ff6eca73700 1 -- 172.21.2.101:0/1360342741 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e80b3890 con 0x7ff6e80b25c0 2021-02-10T11:39:33.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.050 7ff6eca73700 1 -- 172.21.2.101:0/1360342741 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e80b3de0 con 0x7ff6e80b25c0 2021-02-10T11:39:33.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.050 7ff6df7fe700 1 -- 172.21.2.101:0/1360342741 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff6e0001710 con 0x7ff6e80b25c0 2021-02-10T11:39:33.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.051 7ff6df7fe700 1 -- 172.21.2.101:0/1360342741 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff6e00013d0 con 0x7ff6e80b25c0 2021-02-10T11:39:33.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.051 7ff6df7fe700 1 --2- 172.21.2.101:0/1360342741 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6cc0186c0 0x7ff6cc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:33.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.051 7ff6df7fe700 1 -- 172.21.2.101:0/1360342741 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff6e0032800 con 0x7ff6e80b25c0 2021-02-10T11:39:33.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.051 7ff6e685e700 1 --2- 172.21.2.101:0/1360342741 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6cc0186c0 0x7ff6cc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:33.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.051 7ff6e685e700 1 --2- 172.21.2.101:0/1360342741 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6cc0186c0 0x7ff6cc01abc0 crc :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:33.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.082 7ff6eca73700 1 -- 172.21.2.101:0/1360342741 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e8082980 con 0x7ff6e80b25c0 2021-02-10T11:39:33.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.084 7ff6df7fe700 1 -- 172.21.2.101:0/1360342741 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff6e0000f20 con 0x7ff6e80b25c0 2021-02-10T11:39:33.197 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:39:33.199 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:39:33.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.214 7ff6eca73700 1 -- 172.21.2.101:0/1360342741 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "target": ["mgr", ""], "group_name": "group_0000000000112255"}) v1 -- 0x7ff6e80b3de0 con 0x7ff6cc0186c0 2021-02-10T11:39:33.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.217 7ff6df7fe700 1 -- 172.21.2.101:0/1360342741 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7ff6d8006520 con 0x7ff6cc0186c0 2021-02-10T11:39:33.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.218 7ff6eca73700 1 -- 172.21.2.101:0/1360342741 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6cc0186c0 msgr2=0x7ff6cc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:33.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.218 7ff6eca73700 1 --2- 172.21.2.101:0/1360342741 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6cc0186c0 0x7ff6cc01abc0 crc :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:33.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.218 7ff6eca73700 1 -- 172.21.2.101:0/1360342741 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6e80b25c0 msgr2=0x7ff6e80b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:33.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.218 7ff6eca73700 1 --2- 172.21.2.101:0/1360342741 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6e80b25c0 0x7ff6e80b3270 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7ff6e000b8a0 tx=0x7ff6e000bc40).stop 2021-02-10T11:39:33.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.219 7ff6eca73700 1 -- 172.21.2.101:0/1360342741 shutdown_connections 2021-02-10T11:39:33.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.219 7ff6eca73700 1 --2- 172.21.2.101:0/1360342741 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6e80b25c0 0x7ff6e80b3270 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:33.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.219 7ff6eca73700 1 --2- 172.21.2.101:0/1360342741 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6e80b1ba0 0x7ff6e80b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:33.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.219 7ff6eca73700 1 --2- 172.21.2.101:0/1360342741 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6cc0186c0 0x7ff6cc01abc0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:33.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.219 7ff6eca73700 1 --2- 172.21.2.101:0/1360342741 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6e80a58c0 0x7ff6e80a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:33.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.219 7ff6eca73700 1 -- 172.21.2.101:0/1360342741 >> 172.21.2.101:0/1360342741 conn(0x7ff6e80b5dc0 msgr2=0x7ff6e8095960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:33.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.219 7ff6eca73700 1 -- 172.21.2.101:0/1360342741 shutdown_connections 2021-02-10T11:39:33.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:33.219 7ff6eca73700 1 -- 172.21.2.101:0/1360342741 wait complete. 2021-02-10T11:39:33.230 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:39:33.230 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:39:33.544 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:39:33.889 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:39:33.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:33.890 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:39:33.890 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:39:34.226 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:39:34.226 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:39:34.227 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:39:34.227 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:39:34.227 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:39:34.576 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[139273]: fuse finished with error 0 and tester_r 0 2021-02-10T11:39:34.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:34.578 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:39:34.578 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:39:34.912 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:39:34.912 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:39:34.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:39:34.912 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:39:35.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:35.248 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:39:35.248 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:39:35.584 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:39:35.584 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:39:35.585 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:39:35.585 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:39:35.585 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:39:35.933 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[139946]: fuse finished with error 0 and tester_r 0 2021-02-10T11:39:35.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:35.935 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:39:35.935 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:39:36.270 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:39:36.271 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:39:36.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:39:36.271 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:39:36.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:36.605 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:39:36.605 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:39:36.940 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:39:36.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:39:36.940 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:39:36.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:39:36.941 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:39:37.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:37.275 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:39:37.275 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:39:37.608 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:39:37.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:39:37.609 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:39:37.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:39:37.609 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:39:37.944 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-02-10T11:39:38.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.356 7ff737ef7700 1 Processor -- start 2021-02-10T11:39:38.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.356 7ff737ef7700 1 -- start start 2021-02-10T11:39:38.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.356 7ff737ef7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff7300d1860 0x7ff7301b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:38.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.356 7ff737ef7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff7300f0640 0x7ff7300cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:38.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.356 7ff737ef7700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7300f0b00 con 0x7ff7301b0a00 2021-02-10T11:39:38.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.356 7ff737ef7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7300e5ec0 con 0x7ff7300d1860 2021-02-10T11:39:38.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.356 7ff737ef7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7300d1c60 con 0x7ff7300f0640 2021-02-10T11:39:38.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.358 7ff736514700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff7300f0640 0x7ff7300cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:38.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.358 7ff735d13700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff7300d1860 0x7ff7301b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:38.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.358 7ff735512700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7ff7301b0a00 0x7ff7300cf320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:49634/0 (socket says 172.21.2.101:49634) 2021-02-10T11:39:38.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.358 7ff735512700 1 -- 172.21.2.101:0/3282621921 learned_addr learned my addr 172.21.2.101:0/3282621921 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:38.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.358 7ff736514700 1 -- 172.21.2.101:0/3282621921 >> v2:172.21.2.108:3300/0 conn(0x7ff7300d1860 msgr2=0x7ff7301b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:38.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.358 7ff736514700 1 --2- 172.21.2.101:0/3282621921 >> v2:172.21.2.108:3300/0 conn(0x7ff7300d1860 0x7ff7301b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:38.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.358 7ff736514700 1 -- 172.21.2.101:0/3282621921 >> v1:172.21.2.108:6789/0 conn(0x7ff7301b0a00 legacy=0x7ff7300cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:38.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.358 7ff736514700 1 -- 172.21.2.101:0/3282621921 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7300d1e80 con 0x7ff7300f0640 2021-02-10T11:39:38.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.358 7ff735d13700 1 --2- 172.21.2.101:0/3282621921 >> v2:172.21.2.108:3300/0 conn(0x7ff7300d1860 0x7ff7301b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:39:38.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.358 7ff736514700 1 --2- 172.21.2.101:0/3282621921 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff7300f0640 0x7ff7300cfb10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7ff728006e50 tx=0x7ff728009e70).ready entity=mon.2 client_cookie=4a987c21979a594c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:38.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.359 7ff734d11700 1 -- 172.21.2.101:0/3282621921 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff72800b300 con 0x7ff7300f0640 2021-02-10T11:39:38.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.359 7ff734d11700 1 -- 172.21.2.101:0/3282621921 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff728003030 con 0x7ff7300f0640 2021-02-10T11:39:38.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.359 7ff734d11700 1 -- 172.21.2.101:0/3282621921 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff72800b730 con 0x7ff7300f0640 2021-02-10T11:39:38.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.359 7ff737ef7700 1 -- 172.21.2.101:0/3282621921 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff7300f0640 msgr2=0x7ff7300cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:38.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.359 7ff737ef7700 1 --2- 172.21.2.101:0/3282621921 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff7300f0640 0x7ff7300cfb10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7ff728006e50 tx=0x7ff728009e70).stop 2021-02-10T11:39:38.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.360 7ff737ef7700 1 -- 172.21.2.101:0/3282621921 shutdown_connections 2021-02-10T11:39:38.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.360 7ff737ef7700 1 --2- 172.21.2.101:0/3282621921 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff7300f0640 0x7ff7300cfb10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:38.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.360 7ff737ef7700 1 --2- 172.21.2.101:0/3282621921 >> v2:172.21.2.108:3300/0 conn(0x7ff7300d1860 0x7ff7301b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:38.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.360 7ff737ef7700 1 -- 172.21.2.101:0/3282621921 >> 172.21.2.101:0/3282621921 conn(0x7ff7300ce440 msgr2=0x7ff7300d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:38.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.360 7ff737ef7700 1 -- 172.21.2.101:0/3282621921 shutdown_connections 2021-02-10T11:39:38.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.360 7ff737ef7700 1 -- 172.21.2.101:0/3282621921 wait complete. 2021-02-10T11:39:38.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.374 7ff737ef7700 1 Processor -- start 2021-02-10T11:39:38.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.374 7ff737ef7700 1 -- start start 2021-02-10T11:39:38.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.374 7ff737ef7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff7300b52d0 0x7ff7301a4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:38.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.374 7ff737ef7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff730094550 0x7ff730094950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:38.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.374 7ff737ef7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff730094f70 0x7ff730092c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:38.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.374 7ff737ef7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff730096af0 con 0x7ff730094f70 2021-02-10T11:39:38.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.374 7ff737ef7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff730096680 con 0x7ff730094550 2021-02-10T11:39:38.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.374 7ff737ef7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7301a4a80 con 0x7ff7300b52d0 2021-02-10T11:39:38.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.375 7ff736514700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff730094f70 0x7ff730092c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:38.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.375 7ff736514700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff730094f70 0x7ff730092c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55738/0 (socket says 172.21.2.101:55738) 2021-02-10T11:39:38.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.375 7ff736514700 1 -- 172.21.2.101:0/1549401476 learned_addr learned my addr 172.21.2.101:0/1549401476 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:38.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.375 7ff736514700 1 -- 172.21.2.101:0/1549401476 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff7300b52d0 msgr2=0x7ff7301a4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:38.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.375 7ff736514700 1 --2- 172.21.2.101:0/1549401476 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff7300b52d0 0x7ff7301a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:38.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.375 7ff736514700 1 -- 172.21.2.101:0/1549401476 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff730094550 msgr2=0x7ff730094950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:38.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.375 7ff736514700 1 --2- 172.21.2.101:0/1549401476 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff730094550 0x7ff730094950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:38.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.375 7ff736514700 1 -- 172.21.2.101:0/1549401476 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7300b56d0 con 0x7ff730094f70 2021-02-10T11:39:38.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.375 7ff736514700 1 --2- 172.21.2.101:0/1549401476 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff730094f70 0x7ff730092c30 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 rx=0x7ff72800b940 tx=0x7ff72800bce0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:38.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.376 7ff7227fc700 1 -- 172.21.2.101:0/1549401476 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff728008030 con 0x7ff730094f70 2021-02-10T11:39:38.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.376 7ff7227fc700 1 -- 172.21.2.101:0/1549401476 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff728008330 con 0x7ff730094f70 2021-02-10T11:39:38.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.376 7ff7227fc700 1 -- 172.21.2.101:0/1549401476 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff728008850 con 0x7ff730094f70 2021-02-10T11:39:38.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.376 7ff737ef7700 1 -- 172.21.2.101:0/1549401476 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7300b56d0 con 0x7ff730094f70 2021-02-10T11:39:38.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.376 7ff737ef7700 1 -- 172.21.2.101:0/1549401476 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7300b56d0 con 0x7ff730094f70 2021-02-10T11:39:38.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.376 7ff7227fc700 1 -- 172.21.2.101:0/1549401476 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff728006cf0 con 0x7ff730094f70 2021-02-10T11:39:38.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.376 7ff7227fc700 1 --2- 172.21.2.101:0/1549401476 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7180186c0 0x7ff71801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:38.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.376 7ff7227fc700 1 -- 172.21.2.101:0/1549401476 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff728032930 con 0x7ff730094f70 2021-02-10T11:39:38.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.377 7ff735d13700 1 --2- 172.21.2.101:0/1549401476 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7180186c0 0x7ff71801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:38.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.377 7ff735d13700 1 --2- 172.21.2.101:0/1549401476 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7180186c0 0x7ff71801abc0 crc :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:38.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.406 7ff737ef7700 1 -- 172.21.2.101:0/1549401476 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7300ebba0 con 0x7ff730094f70 2021-02-10T11:39:38.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.409 7ff7227fc700 1 -- 172.21.2.101:0/1549401476 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff72800be20 con 0x7ff730094f70 2021-02-10T11:39:38.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.558 7ff737ef7700 1 -- 172.21.2.101:0/1549401476 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff7300f0b00 con 0x7ff730094f70 2021-02-10T11:39:38.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.562 7ff7227fc700 1 -- 172.21.2.101:0/1549401476 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff728006cf0 con 0x7ff730094f70 2021-02-10T11:39:38.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.567 7ff737ef7700 1 -- 172.21.2.101:0/1549401476 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7180186c0 msgr2=0x7ff71801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:38.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.567 7ff737ef7700 1 --2- 172.21.2.101:0/1549401476 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7180186c0 0x7ff71801abc0 crc :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:38.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.567 7ff737ef7700 1 -- 172.21.2.101:0/1549401476 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff730094f70 msgr2=0x7ff730092c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:38.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.567 7ff737ef7700 1 --2- 172.21.2.101:0/1549401476 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff730094f70 0x7ff730092c30 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 rx=0x7ff72800b940 tx=0x7ff72800bce0).stop 2021-02-10T11:39:38.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.567 7ff737ef7700 1 -- 172.21.2.101:0/1549401476 shutdown_connections 2021-02-10T11:39:38.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.567 7ff737ef7700 1 --2- 172.21.2.101:0/1549401476 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff730094f70 0x7ff730092c30 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:38.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.567 7ff737ef7700 1 --2- 172.21.2.101:0/1549401476 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff730094550 0x7ff730094950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:38.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.567 7ff737ef7700 1 --2- 172.21.2.101:0/1549401476 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7180186c0 0x7ff71801abc0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:38.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.567 7ff737ef7700 1 --2- 172.21.2.101:0/1549401476 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff7300b52d0 0x7ff7301a4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:38.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.567 7ff737ef7700 1 -- 172.21.2.101:0/1549401476 >> 172.21.2.101:0/1549401476 conn(0x7ff7300b8df0 msgr2=0x7ff7300add50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:38.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.567 7ff737ef7700 1 -- 172.21.2.101:0/1549401476 shutdown_connections 2021-02-10T11:39:38.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.567 7ff737ef7700 1 -- 172.21.2.101:0/1549401476 wait complete. 2021-02-10T11:39:38.579 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_deauthorize_auth_id_after_out_of_band_update' 2021-02-10T11:39:39.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.986 7fcb16d9e700 1 Processor -- start 2021-02-10T11:39:39.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.986 7fcb16d9e700 1 -- start start 2021-02-10T11:39:39.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.987 7fcb16d9e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fcb100d0340 0x7fcb100ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:39.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.987 7fcb16d9e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb100d1c70 0x7fcb100efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:39.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.987 7fcb16d9e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb100f0b10 con 0x7fcb100d1870 2021-02-10T11:39:39.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.987 7fcb16d9e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb100e5ed0 con 0x7fcb100d0340 2021-02-10T11:39:39.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.987 7fcb16d9e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb101b0be0 con 0x7fcb100d1c70 2021-02-10T11:39:39.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.987 7fcb14bba700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fcb100d1870 0x7fcb101b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56236/0 (socket says 172.21.2.101:56236) 2021-02-10T11:39:39.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.987 7fcb14bba700 1 -- 172.21.2.101:0/2426111908 learned_addr learned my addr 172.21.2.101:0/2426111908 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:39.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.987 7fcb0ffff700 1 --2- 172.21.2.101:0/2426111908 >> v2:172.21.2.108:3300/0 conn(0x7fcb100d0340 0x7fcb100ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:39.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.987 7fcb153bb700 1 --2- 172.21.2.101:0/2426111908 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb100d1c70 0x7fcb100efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:39.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.987 7fcb0ffff700 1 -- 172.21.2.101:0/2426111908 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb100d1c70 msgr2=0x7fcb100efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:39.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.987 7fcb0ffff700 1 --2- 172.21.2.101:0/2426111908 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb100d1c70 0x7fcb100efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:39.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.987 7fcb0ffff700 1 -- 172.21.2.101:0/2426111908 >> v1:172.21.2.101:6789/0 conn(0x7fcb100d1870 legacy=0x7fcb101b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:39.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.987 7fcb0f7fe700 1 -- 172.21.2.101:0/2426111908 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2594519819 0 0) 0x7fcb08001430 con 0x7fcb100d1870 2021-02-10T11:39:39.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.987 7fcb0ffff700 1 -- 172.21.2.101:0/2426111908 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb100f20a0 con 0x7fcb100d0340 2021-02-10T11:39:39.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.988 7fcb153bb700 1 --2- 172.21.2.101:0/2426111908 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb100d1c70 0x7fcb100efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:39:39.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.988 7fcb0ffff700 1 --2- 172.21.2.101:0/2426111908 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcb100d0340 0x7fcb100ef640 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7fcb04006650 tx=0x7fcb04009a80).ready entity=mon.1 client_cookie=18efe2f6767c19fc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:39.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.988 7fcb0f7fe700 1 -- 172.21.2.101:0/2426111908 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcb040009b0 con 0x7fcb100d0340 2021-02-10T11:39:39.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.988 7fcb0f7fe700 1 -- 172.21.2.101:0/2426111908 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0400c0c0 con 0x7fcb100d0340 2021-02-10T11:39:39.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.988 7fcb0f7fe700 1 -- 172.21.2.101:0/2426111908 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcb0400c5e0 con 0x7fcb100d0340 2021-02-10T11:39:39.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.988 7fcb16d9e700 1 -- 172.21.2.101:0/2426111908 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcb100d0340 msgr2=0x7fcb100ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:39.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.988 7fcb16d9e700 1 --2- 172.21.2.101:0/2426111908 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcb100d0340 0x7fcb100ef640 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7fcb04006650 tx=0x7fcb04009a80).stop 2021-02-10T11:39:39.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.989 7fcb16d9e700 1 -- 172.21.2.101:0/2426111908 shutdown_connections 2021-02-10T11:39:39.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.989 7fcb16d9e700 1 --2- 172.21.2.101:0/2426111908 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb100d1c70 0x7fcb100efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:39.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.989 7fcb16d9e700 1 --2- 172.21.2.101:0/2426111908 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcb100d0340 0x7fcb100ef640 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:39.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.989 7fcb16d9e700 1 -- 172.21.2.101:0/2426111908 >> 172.21.2.101:0/2426111908 conn(0x7fcb100ce450 msgr2=0x7fcb100d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:39.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.989 7fcb16d9e700 1 -- 172.21.2.101:0/2426111908 shutdown_connections 2021-02-10T11:39:39.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:38.989 7fcb16d9e700 1 -- 172.21.2.101:0/2426111908 wait complete. 2021-02-10T11:39:39.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.003 7fcb16d9e700 1 Processor -- start 2021-02-10T11:39:39.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.003 7fcb16d9e700 1 -- start start 2021-02-10T11:39:39.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.004 7fcb16d9e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb100beeb0 0x7fcb100bf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:39.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.004 7fcb16d9e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb1018ef70 0x7fcb1018f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:39.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.004 7fcb16d9e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcb1018f990 0x7fcb100c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:39.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.004 7fcb16d9e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb100bfc50 con 0x7fcb1018ef70 2021-02-10T11:39:39.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.004 7fcb16d9e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1018fd90 con 0x7fcb1018f990 2021-02-10T11:39:39.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.004 7fcb16d9e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb100bf8d0 con 0x7fcb100beeb0 2021-02-10T11:39:39.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.004 7fcb14bba700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb100beeb0 0x7fcb100bf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:39.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.004 7fcb0ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb1018ef70 0x7fcb1018f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:39.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.004 7fcb14bba700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb100beeb0 0x7fcb100bf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:48480/0 (socket says 172.21.2.101:48480) 2021-02-10T11:39:39.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.004 7fcb0ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb1018ef70 0x7fcb1018f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55750/0 (socket says 172.21.2.101:55750) 2021-02-10T11:39:39.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.004 7fcb14bba700 1 -- 172.21.2.101:0/3562097367 learned_addr learned my addr 172.21.2.101:0/3562097367 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:39.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.004 7fcb153bb700 1 --2- 172.21.2.101:0/3562097367 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcb1018f990 0x7fcb100c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:39.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.004 7fcb14bba700 1 -- 172.21.2.101:0/3562097367 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcb1018f990 msgr2=0x7fcb100c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:39.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.004 7fcb14bba700 1 --2- 172.21.2.101:0/3562097367 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcb1018f990 0x7fcb100c01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:39.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.004 7fcb14bba700 1 -- 172.21.2.101:0/3562097367 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb1018ef70 msgr2=0x7fcb1018f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:39.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.004 7fcb14bba700 1 --2- 172.21.2.101:0/3562097367 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb1018ef70 0x7fcb1018f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:39.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.004 7fcb14bba700 1 -- 172.21.2.101:0/3562097367 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb100c07f0 con 0x7fcb100beeb0 2021-02-10T11:39:39.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.004 7fcb153bb700 1 --2- 172.21.2.101:0/3562097367 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcb1018f990 0x7fcb100c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:39:39.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.005 7fcb14bba700 1 --2- 172.21.2.101:0/3562097367 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb100beeb0 0x7fcb100bf2b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fcb08001910 tx=0x7fcb0800ac50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:39.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.005 7fcb0d7fa700 1 -- 172.21.2.101:0/3562097367 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcb08007020 con 0x7fcb100beeb0 2021-02-10T11:39:39.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.005 7fcb0d7fa700 1 -- 172.21.2.101:0/3562097367 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb080046d0 con 0x7fcb100beeb0 2021-02-10T11:39:39.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.005 7fcb0d7fa700 1 -- 172.21.2.101:0/3562097367 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcb08004bb0 con 0x7fcb100beeb0 2021-02-10T11:39:39.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.005 7fcb16d9e700 1 -- 172.21.2.101:0/3562097367 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb100c07f0 con 0x7fcb100beeb0 2021-02-10T11:39:39.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.005 7fcb16d9e700 1 -- 172.21.2.101:0/3562097367 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb100c0d80 con 0x7fcb100beeb0 2021-02-10T11:39:39.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.006 7fcb0d7fa700 1 -- 172.21.2.101:0/3562097367 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fcb0800ad90 con 0x7fcb100beeb0 2021-02-10T11:39:39.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.006 7fcb0d7fa700 1 --2- 172.21.2.101:0/3562097367 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcaf00186c0 0x7fcaf001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:39.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.006 7fcb0d7fa700 1 -- 172.21.2.101:0/3562097367 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fcb080328d0 con 0x7fcb100beeb0 2021-02-10T11:39:39.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.006 7fcb0ffff700 1 --2- 172.21.2.101:0/3562097367 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcaf00186c0 0x7fcaf001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:39.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.006 7fcb0ffff700 1 --2- 172.21.2.101:0/3562097367 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcaf00186c0 0x7fcaf001abc0 crc :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:39.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.036 7fcb16d9e700 1 -- 172.21.2.101:0/3562097367 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb100ebc60 con 0x7fcb100beeb0 2021-02-10T11:39:39.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.039 7fcb0d7fa700 1 -- 172.21.2.101:0/3562097367 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fcb08000d10 con 0x7fcb100beeb0 2021-02-10T11:39:39.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.171 7fcb16d9e700 1 -- 172.21.2.101:0/3562097367 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_deauthorize_auth_id_after_out_of_band_update"]} v 0) v1 -- 0x7fcb100f0b10 con 0x7fcb100beeb0 2021-02-10T11:39:39.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.544 7fcb0d7fa700 1 -- 172.21.2.101:0/3562097367 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_deauthorize_auth_id_after_out_of_band_update"]}]=0 v512) v1 ==== 166+0+0 (secure 0 0 0) 0x7fcb0800ad90 con 0x7fcb100beeb0 2021-02-10T11:39:39.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.590 7fcb16d9e700 1 -- 172.21.2.101:0/3562097367 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcaf00186c0 msgr2=0x7fcaf001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:39.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.590 7fcb16d9e700 1 --2- 172.21.2.101:0/3562097367 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcaf00186c0 0x7fcaf001abc0 crc :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:39.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.590 7fcb16d9e700 1 -- 172.21.2.101:0/3562097367 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb100beeb0 msgr2=0x7fcb100bf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:39.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.590 7fcb16d9e700 1 --2- 172.21.2.101:0/3562097367 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb100beeb0 0x7fcb100bf2b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fcb08001910 tx=0x7fcb0800ac50).stop 2021-02-10T11:39:39.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.590 7fcb16d9e700 1 -- 172.21.2.101:0/3562097367 shutdown_connections 2021-02-10T11:39:39.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.590 7fcb16d9e700 1 --2- 172.21.2.101:0/3562097367 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcb1018f990 0x7fcb100c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:39.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.590 7fcb16d9e700 1 --2- 172.21.2.101:0/3562097367 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb1018ef70 0x7fcb1018f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:39.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.590 7fcb16d9e700 1 --2- 172.21.2.101:0/3562097367 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcaf00186c0 0x7fcaf001abc0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:39.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.590 7fcb16d9e700 1 --2- 172.21.2.101:0/3562097367 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb100beeb0 0x7fcb100bf2b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:39.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.590 7fcb16d9e700 1 -- 172.21.2.101:0/3562097367 >> 172.21.2.101:0/3562097367 conn(0x7fcb100c2130 msgr2=0x7fcb1009e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:39.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.591 7fcb16d9e700 1 -- 172.21.2.101:0/3562097367 shutdown_connections 2021-02-10T11:39:39.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:39.591 7fcb16d9e700 1 -- 172.21.2.101:0/3562097367 wait complete. 2021-02-10T11:39:39.602 INFO:tasks.cephfs_test_runner:test_deauthorize_auth_id_after_out_of_band_update (tasks.cephfs.test_volumes.TestVolumes) ... ok 2021-02-10T11:39:39.602 INFO:tasks.cephfs_test_runner:Starting test: test_default_uid_gid_subvolume (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:39:39.602 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_default_uid_gid_subvolume' 2021-02-10T11:39:40.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.010 7f3048284700 1 Processor -- start 2021-02-10T11:39:40.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.010 7f3048284700 1 -- start start 2021-02-10T11:39:40.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.011 7f3048284700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f30400d0340 0x7f30400ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:40.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.011 7f3048284700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30400d1c70 0x7f30400efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:40.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.011 7f3048284700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30400f0b10 con 0x7f30400d1870 2021-02-10T11:39:40.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.011 7f3048284700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30400e5ed0 con 0x7f30400d0340 2021-02-10T11:39:40.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.011 7f3048284700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30401b0be0 con 0x7f30400d1c70 2021-02-10T11:39:40.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.012 7f30468a1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30400d1c70 0x7f30400efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:40.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.012 7f304589f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f30400d0340 0x7f30400ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.012 7f30468a1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30400d1c70 0x7f30400efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:48492/0 (socket says 172.21.2.101:48492) 2021-02-10T11:39:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.012 7f304589f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f30400d0340 0x7f30400ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:38010/0 (socket says 172.21.2.101:38010) 2021-02-10T11:39:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.012 7f30460a0700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f30400d1870 0x7f30401b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:49660/0 (socket says 172.21.2.101:49660) 2021-02-10T11:39:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.012 7f30468a1700 1 -- 172.21.2.101:0/784159688 learned_addr learned my addr 172.21.2.101:0/784159688 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.013 7f30468a1700 1 -- 172.21.2.101:0/784159688 >> v2:172.21.2.108:3300/0 conn(0x7f30400d0340 msgr2=0x7f30400ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.013 7f30468a1700 1 --2- 172.21.2.101:0/784159688 >> v2:172.21.2.108:3300/0 conn(0x7f30400d0340 0x7f30400ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.013 7f30468a1700 1 -- 172.21.2.101:0/784159688 >> v1:172.21.2.108:6789/0 conn(0x7f30400d1870 legacy=0x7f30401b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.013 7f30468a1700 1 -- 172.21.2.101:0/784159688 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30400f20a0 con 0x7f30400d1c70 2021-02-10T11:39:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.013 7f30468a1700 1 --2- 172.21.2.101:0/784159688 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30400d1c70 0x7f30400efc60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f3038006aa0 tx=0x7f3038009f20).ready entity=mon.2 client_cookie=3080023f06279842 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.014 7f304509e700 1 -- 172.21.2.101:0/784159688 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f303800b5c0 con 0x7f30400d1c70 2021-02-10T11:39:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.014 7f304509e700 1 -- 172.21.2.101:0/784159688 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3038003030 con 0x7f30400d1c70 2021-02-10T11:39:40.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.014 7f304509e700 1 -- 172.21.2.101:0/784159688 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f303800b9f0 con 0x7f30400d1c70 2021-02-10T11:39:40.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.014 7f3048284700 1 -- 172.21.2.101:0/784159688 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30400d1c70 msgr2=0x7f30400efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:40.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.014 7f3048284700 1 --2- 172.21.2.101:0/784159688 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30400d1c70 0x7f30400efc60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f3038006aa0 tx=0x7f3038009f20).stop 2021-02-10T11:39:40.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.015 7f3048284700 1 -- 172.21.2.101:0/784159688 shutdown_connections 2021-02-10T11:39:40.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.015 7f3048284700 1 --2- 172.21.2.101:0/784159688 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30400d1c70 0x7f30400efc60 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:40.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.015 7f3048284700 1 --2- 172.21.2.101:0/784159688 >> v2:172.21.2.108:3300/0 conn(0x7f30400d0340 0x7f30400ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:40.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.015 7f3048284700 1 -- 172.21.2.101:0/784159688 >> 172.21.2.101:0/784159688 conn(0x7f30400ce450 msgr2=0x7f30400d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:40.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.015 7f3048284700 1 -- 172.21.2.101:0/784159688 shutdown_connections 2021-02-10T11:39:40.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.015 7f3048284700 1 -- 172.21.2.101:0/784159688 wait complete. 2021-02-10T11:39:40.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.029 7f3048284700 1 Processor -- start 2021-02-10T11:39:40.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.030 7f3048284700 1 -- start start 2021-02-10T11:39:40.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.030 7f3048284700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30400ada90 0x7f30400ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:40.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.030 7f3048284700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3040099d20 0x7f304009a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:40.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.030 7f3048284700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f304009a740 0x7f3040097bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:40.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.030 7f3048284700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30400997a0 con 0x7f3040099d20 2021-02-10T11:39:40.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.030 7f3048284700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304018f0c0 con 0x7f304009a740 2021-02-10T11:39:40.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.030 7f3048284700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304009ab40 con 0x7f30400ada90 2021-02-10T11:39:40.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.030 7f30460a0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30400ada90 0x7f30400ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:40.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.030 7f30460a0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30400ada90 0x7f30400ade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:48494/0 (socket says 172.21.2.101:48494) 2021-02-10T11:39:40.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.030 7f30460a0700 1 -- 172.21.2.101:0/2416181578 learned_addr learned my addr 172.21.2.101:0/2416181578 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:40.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.030 7f30468a1700 1 --2- 172.21.2.101:0/2416181578 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f304009a740 0x7f3040097bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:40.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.030 7f304589f700 1 --2- 172.21.2.101:0/2416181578 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3040099d20 0x7f304009a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:40.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.031 7f30460a0700 1 -- 172.21.2.101:0/2416181578 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f304009a740 msgr2=0x7f3040097bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:40.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.031 7f30460a0700 1 --2- 172.21.2.101:0/2416181578 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f304009a740 0x7f3040097bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:40.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.031 7f30460a0700 1 -- 172.21.2.101:0/2416181578 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3040099d20 msgr2=0x7f304009a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:40.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.031 7f30460a0700 1 --2- 172.21.2.101:0/2416181578 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3040099d20 0x7f304009a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:40.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.031 7f30460a0700 1 -- 172.21.2.101:0/2416181578 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304009ad60 con 0x7f30400ada90 2021-02-10T11:39:40.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.031 7f30468a1700 1 --2- 172.21.2.101:0/2416181578 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f304009a740 0x7f3040097bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:39:40.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.031 7f30460a0700 1 --2- 172.21.2.101:0/2416181578 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30400ada90 0x7f30400ade90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f3034007650 tx=0x7f303400a900).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:40.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.031 7f3032ffd700 1 -- 172.21.2.101:0/2416181578 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f303400acf0 con 0x7f30400ada90 2021-02-10T11:39:40.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.031 7f3032ffd700 1 -- 172.21.2.101:0/2416181578 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3034004140 con 0x7f30400ada90 2021-02-10T11:39:40.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.031 7f3032ffd700 1 -- 172.21.2.101:0/2416181578 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3034004660 con 0x7f30400ada90 2021-02-10T11:39:40.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.031 7f3048284700 1 -- 172.21.2.101:0/2416181578 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304009ad60 con 0x7f30400ada90 2021-02-10T11:39:40.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.031 7f3048284700 1 -- 172.21.2.101:0/2416181578 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304009ad60 con 0x7f30400ada90 2021-02-10T11:39:40.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.032 7f3032ffd700 1 -- 172.21.2.101:0/2416181578 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3034000d60 con 0x7f30400ada90 2021-02-10T11:39:40.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.032 7f3032ffd700 1 --2- 172.21.2.101:0/2416181578 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30280186c0 0x7f302801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:40.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.032 7f3032ffd700 1 -- 172.21.2.101:0/2416181578 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3034032400 con 0x7f30400ada90 2021-02-10T11:39:40.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.032 7f304589f700 1 --2- 172.21.2.101:0/2416181578 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30280186c0 0x7f302801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:40.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.032 7f304589f700 1 --2- 172.21.2.101:0/2416181578 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30280186c0 0x7f302801abc0 crc :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:40.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.061 7f3048284700 1 -- 172.21.2.101:0/2416181578 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30400ebc00 con 0x7f30400ada90 2021-02-10T11:39:40.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.064 7f3032ffd700 1 -- 172.21.2.101:0/2416181578 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3034023380 con 0x7f30400ada90 2021-02-10T11:39:40.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.198 7f3048284700 1 -- 172.21.2.101:0/2416181578 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_default_uid_gid_subvolume"]} v 0) v1 -- 0x7f30400ebc00 con 0x7f30400ada90 2021-02-10T11:39:40.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.549 7f3032ffd700 1 -- 172.21.2.101:0/2416181578 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_default_uid_gid_subvolume"]}]=0 v513) v1 ==== 150+0+0 (secure 0 0 0) 0x7f3034023380 con 0x7f30400ada90 2021-02-10T11:39:40.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.567 7f3048284700 1 -- 172.21.2.101:0/2416181578 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30280186c0 msgr2=0x7f302801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:40.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.567 7f3048284700 1 --2- 172.21.2.101:0/2416181578 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30280186c0 0x7f302801abc0 crc :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:40.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.567 7f3048284700 1 -- 172.21.2.101:0/2416181578 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30400ada90 msgr2=0x7f30400ade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:40.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.567 7f3048284700 1 --2- 172.21.2.101:0/2416181578 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30400ada90 0x7f30400ade90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f3034007650 tx=0x7f303400a900).stop 2021-02-10T11:39:40.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.567 7f3048284700 1 -- 172.21.2.101:0/2416181578 shutdown_connections 2021-02-10T11:39:40.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.567 7f3048284700 1 --2- 172.21.2.101:0/2416181578 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f304009a740 0x7f3040097bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:40.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.567 7f3048284700 1 --2- 172.21.2.101:0/2416181578 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3040099d20 0x7f304009a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:40.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.567 7f3048284700 1 --2- 172.21.2.101:0/2416181578 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30280186c0 0x7f302801abc0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:40.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.567 7f3048284700 1 --2- 172.21.2.101:0/2416181578 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30400ada90 0x7f30400ade90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:40.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.567 7f3048284700 1 -- 172.21.2.101:0/2416181578 >> 172.21.2.101:0/2416181578 conn(0x7f30400f0550 msgr2=0x7f3040191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:40.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.567 7f3048284700 1 -- 172.21.2.101:0/2416181578 shutdown_connections 2021-02-10T11:39:40.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.567 7f3048284700 1 -- 172.21.2.101:0/2416181578 wait complete. 2021-02-10T11:39:40.579 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:39:41.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.991 7f2c33d7f700 1 Processor -- start 2021-02-10T11:39:41.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.991 7f2c33d7f700 1 -- start start 2021-02-10T11:39:41.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.991 7f2c33d7f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2c2c0d1c70 0x7f2c2c0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:41.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.991 7f2c33d7f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c2c0d0170 con 0x7f2c2c0d1870 2021-02-10T11:39:41.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.991 7f2c33d7f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c2c0fe1c0 con 0x7f2c2c0cf580 2021-02-10T11:39:41.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.991 7f2c33d7f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c2c1b0cb0 con 0x7f2c2c0d1c70 2021-02-10T11:39:41.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.992 7f2c31b9b700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f2c2c0d1870 0x7f2c2c1b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56264/0 (socket says 172.21.2.101:56264) 2021-02-10T11:39:41.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.992 7f2c31b9b700 1 -- 172.21.2.101:0/1772652310 learned_addr learned my addr 172.21.2.101:0/1772652310 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:41.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.992 7f2c30b99700 1 -- 172.21.2.101:0/1772652310 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1637228565 0 0) 0x7f2c28000ce0 con 0x7f2c2c0d1870 2021-02-10T11:39:41.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.992 7f2c30b99700 1 -- 172.21.2.101:0/1772652310 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c18002fe0 con 0x7f2c2c0d1870 2021-02-10T11:39:41.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.992 7f2c3139a700 1 --2- 172.21.2.101:0/1772652310 >> v2:172.21.2.108:3300/0 conn(0x7f2c2c0d1c70 0x7f2c2c0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:41.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.993 7f2c3139a700 1 -- 172.21.2.101:0/1772652310 >> v1:172.21.2.108:6789/0 conn(0x7f2c2c0cf580 legacy=0x7f2c2c0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:41.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.993 7f2c3139a700 1 -- 172.21.2.101:0/1772652310 >> v1:172.21.2.101:6789/0 conn(0x7f2c2c0d1870 legacy=0x7f2c2c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:41.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.993 7f2c3139a700 1 -- 172.21.2.101:0/1772652310 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2c109a60 con 0x7f2c2c0d1c70 2021-02-10T11:39:41.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.993 7f2c3139a700 1 --2- 172.21.2.101:0/1772652310 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2c2c0d1c70 0x7f2c2c0cef60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f2c1c006650 tx=0x7f2c1c009a80).ready entity=mon.1 client_cookie=de4c0601a9103717 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:41.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.993 7f2c30b99700 1 -- 172.21.2.101:0/1772652310 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2c1c0009b0 con 0x7f2c2c0d1c70 2021-02-10T11:39:41.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.993 7f2c30b99700 1 -- 172.21.2.101:0/1772652310 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1c00c0c0 con 0x7f2c2c0d1c70 2021-02-10T11:39:41.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.993 7f2c30b99700 1 -- 172.21.2.101:0/1772652310 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2c1c00c560 con 0x7f2c2c0d1c70 2021-02-10T11:39:41.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.993 7f2c33d7f700 1 -- 172.21.2.101:0/1772652310 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2c2c0d1c70 msgr2=0x7f2c2c0cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:41.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.993 7f2c33d7f700 1 --2- 172.21.2.101:0/1772652310 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2c2c0d1c70 0x7f2c2c0cef60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f2c1c006650 tx=0x7f2c1c009a80).stop 2021-02-10T11:39:41.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.994 7f2c33d7f700 1 -- 172.21.2.101:0/1772652310 shutdown_connections 2021-02-10T11:39:41.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.994 7f2c33d7f700 1 --2- 172.21.2.101:0/1772652310 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2c2c0d1c70 0x7f2c2c0cef60 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:41.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.994 7f2c33d7f700 1 -- 172.21.2.101:0/1772652310 >> 172.21.2.101:0/1772652310 conn(0x7f2c2c0ce420 msgr2=0x7f2c2c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:41.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.994 7f2c33d7f700 1 -- 172.21.2.101:0/1772652310 shutdown_connections 2021-02-10T11:39:41.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:40.994 7f2c33d7f700 1 -- 172.21.2.101:0/1772652310 wait complete. 2021-02-10T11:39:41.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.009 7f2c33d7f700 1 Processor -- start 2021-02-10T11:39:41.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.009 7f2c33d7f700 1 -- start start 2021-02-10T11:39:41.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.009 7f2c33d7f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2c2c0c3820 0x7f2c2c0a7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:41.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.009 7f2c33d7f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c2c1a44c0 0x7f2c2c1a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:41.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.009 7f2c33d7f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c2c0863d0 0x7f2c2c0867d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:41.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.009 7f2c33d7f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c085ed0 con 0x7f2c2c0863d0 2021-02-10T11:39:41.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.009 7f2c33d7f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c0c0740 con 0x7f2c2c0c3820 2021-02-10T11:39:41.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.009 7f2c33d7f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c086df0 con 0x7f2c2c1a44c0 2021-02-10T11:39:41.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.009 7f2c3239c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c2c0863d0 0x7f2c2c0867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:41.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.009 7f2c3239c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c2c0863d0 0x7f2c2c0867d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55778/0 (socket says 172.21.2.101:55778) 2021-02-10T11:39:41.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.009 7f2c3239c700 1 -- 172.21.2.101:0/2178516361 learned_addr learned my addr 172.21.2.101:0/2178516361 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:41.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.010 7f2c3239c700 1 -- 172.21.2.101:0/2178516361 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c2c1a44c0 msgr2=0x7f2c2c1a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:41.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.010 7f2c3239c700 1 --2- 172.21.2.101:0/2178516361 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c2c1a44c0 0x7f2c2c1a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:41.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.010 7f2c3239c700 1 -- 172.21.2.101:0/2178516361 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2c2c0c3820 msgr2=0x7f2c2c0a7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:41.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.010 7f2c3239c700 1 --2- 172.21.2.101:0/2178516361 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2c2c0c3820 0x7f2c2c0a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:41.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.010 7f2c3239c700 1 -- 172.21.2.101:0/2178516361 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2c087010 con 0x7f2c2c0863d0 2021-02-10T11:39:41.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.010 7f2c3239c700 1 --2- 172.21.2.101:0/2178516361 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c2c0863d0 0x7f2c2c0867d0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 rx=0x7f2c24009940 tx=0x7f2c2400bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:41.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.010 7f2c227fc700 1 -- 172.21.2.101:0/2178516361 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2c24000ce0 con 0x7f2c2c0863d0 2021-02-10T11:39:41.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.010 7f2c227fc700 1 -- 172.21.2.101:0/2178516361 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2400c2f0 con 0x7f2c2c0863d0 2021-02-10T11:39:41.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.010 7f2c227fc700 1 -- 172.21.2.101:0/2178516361 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2c2400c750 con 0x7f2c2c0863d0 2021-02-10T11:39:41.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.010 7f2c33d7f700 1 -- 172.21.2.101:0/2178516361 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2c087010 con 0x7f2c2c0863d0 2021-02-10T11:39:41.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.011 7f2c33d7f700 1 -- 172.21.2.101:0/2178516361 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2c087010 con 0x7f2c2c0863d0 2021-02-10T11:39:41.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.011 7f2c227fc700 1 -- 172.21.2.101:0/2178516361 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2c24001220 con 0x7f2c2c0863d0 2021-02-10T11:39:41.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.011 7f2c227fc700 1 --2- 172.21.2.101:0/2178516361 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2c140186d0 0x7f2c1401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:41.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.011 7f2c227fc700 1 -- 172.21.2.101:0/2178516361 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2c24032b20 con 0x7f2c2c0863d0 2021-02-10T11:39:41.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.011 7f2c31b9b700 1 --2- 172.21.2.101:0/2178516361 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2c140186d0 0x7f2c1401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:41.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.011 7f2c31b9b700 1 --2- 172.21.2.101:0/2178516361 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2c140186d0 0x7f2c1401abd0 crc :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:41.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.041 7f2c33d7f700 1 -- 172.21.2.101:0/2178516361 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2c0fe1c0 con 0x7f2c2c0863d0 2021-02-10T11:39:41.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.044 7f2c227fc700 1 -- 172.21.2.101:0/2178516361 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2c2400be30 con 0x7f2c2c0863d0 2021-02-10T11:39:41.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.195 7f2c33d7f700 1 -- 172.21.2.101:0/2178516361 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2c2c0fe1c0 con 0x7f2c2c0863d0 2021-02-10T11:39:41.202 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.199 7f2c227fc700 1 -- 172.21.2.101:0/2178516361 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2c24001220 con 0x7f2c2c0863d0 2021-02-10T11:39:41.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.204 7f2c33d7f700 1 -- 172.21.2.101:0/2178516361 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2c140186d0 msgr2=0x7f2c1401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:41.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.204 7f2c33d7f700 1 --2- 172.21.2.101:0/2178516361 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2c140186d0 0x7f2c1401abd0 crc :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:41.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.204 7f2c33d7f700 1 -- 172.21.2.101:0/2178516361 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c2c0863d0 msgr2=0x7f2c2c0867d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:41.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.204 7f2c33d7f700 1 --2- 172.21.2.101:0/2178516361 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c2c0863d0 0x7f2c2c0867d0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 rx=0x7f2c24009940 tx=0x7f2c2400bcf0).stop 2021-02-10T11:39:41.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.204 7f2c33d7f700 1 -- 172.21.2.101:0/2178516361 shutdown_connections 2021-02-10T11:39:41.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.204 7f2c33d7f700 1 --2- 172.21.2.101:0/2178516361 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2c140186d0 0x7f2c1401abd0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:41.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.204 7f2c33d7f700 1 --2- 172.21.2.101:0/2178516361 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c2c1a44c0 0x7f2c2c1a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:41.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.204 7f2c33d7f700 1 --2- 172.21.2.101:0/2178516361 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c2c0863d0 0x7f2c2c0867d0 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:41.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.204 7f2c33d7f700 1 --2- 172.21.2.101:0/2178516361 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2c2c0c3820 0x7f2c2c0a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:41.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.204 7f2c33d7f700 1 -- 172.21.2.101:0/2178516361 >> 172.21.2.101:0/2178516361 conn(0x7f2c2c0872c0 msgr2=0x7f2c2c095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:41.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.204 7f2c33d7f700 1 -- 172.21.2.101:0/2178516361 shutdown_connections 2021-02-10T11:39:41.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:41.204 7f2c33d7f700 1 -- 172.21.2.101:0/2178516361 wait complete. 2021-02-10T11:39:41.215 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:39:41.215 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:39:41.560 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:39:41.904 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:39:41.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:41.904 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:39:41.904 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:39:42.239 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2021-02-10T11:39:42.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:39:42.240 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-02-10T11:39:42.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:42.240 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:39:42.240 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:39:42.574 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:39:42.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:39:42.575 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:39:42.575 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:42.575 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:39:42.575 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:39:42.909 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:39:42.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:39:42.910 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:39:42.910 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:39:42.910 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:39:42.910 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:39:43.244 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:39:43.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:39:43.244 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:39:43.244 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json-pretty 2021-02-10T11:39:43.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.652 7f9c5f40e700 1 Processor -- start 2021-02-10T11:39:43.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.653 7f9c5f40e700 1 -- start start 2021-02-10T11:39:43.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.653 7f9c5f40e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f9c581b0470 0x7f9c581b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:43.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.653 7f9c5f40e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9c580ef610 0x7f9c580efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:43.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.653 7f9c5f40e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c580f0b50 con 0x7f9c580f0030 2021-02-10T11:39:43.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.653 7f9c5f40e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c580e52e0 con 0x7f9c580ef610 2021-02-10T11:39:43.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.653 7f9c5f40e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c580e5490 con 0x7f9c581b0470 2021-02-10T11:39:43.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.653 7f9c5ca29700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9c580ef610 0x7f9c580efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:43.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.653 7f9c5ca29700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9c580ef610 0x7f9c580efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55782/0 (socket says 172.21.2.101:55782) 2021-02-10T11:39:43.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.653 7f9c5ca29700 1 -- 172.21.2.101:0/916392825 learned_addr learned my addr 172.21.2.101:0/916392825 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:43.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.653 7f9c5ca29700 1 -- 172.21.2.101:0/916392825 >> v2:172.21.2.108:3300/0 conn(0x7f9c581b0470 msgr2=0x7f9c581b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:39:43.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.653 7f9c5ca29700 1 --2- 172.21.2.101:0/916392825 >> v2:172.21.2.108:3300/0 conn(0x7f9c581b0470 0x7f9c581b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:43.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.653 7f9c5ca29700 1 -- 172.21.2.101:0/916392825 >> v1:172.21.2.108:6789/0 conn(0x7f9c580f0030 legacy=0x7f9c580f10d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:39:43.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.653 7f9c5ca29700 1 -- 172.21.2.101:0/916392825 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c580f0430 con 0x7f9c580ef610 2021-02-10T11:39:43.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.654 7f9c5ca29700 1 --2- 172.21.2.101:0/916392825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c580ef610 0x7f9c580efa10 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 rx=0x7f9c54006700 tx=0x7f9c54009b00).ready entity=mon.0 client_cookie=32f00d97905d04a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:43.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.654 7f9c4bfff700 1 -- 172.21.2.101:0/916392825 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9c5400c0c0 con 0x7f9c580ef610 2021-02-10T11:39:43.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.654 7f9c4bfff700 1 -- 172.21.2.101:0/916392825 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5400c400 con 0x7f9c580ef610 2021-02-10T11:39:43.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.654 7f9c4bfff700 1 -- 172.21.2.101:0/916392825 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9c5400c8e0 con 0x7f9c580ef610 2021-02-10T11:39:43.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.654 7f9c5f40e700 1 -- 172.21.2.101:0/916392825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c580ef610 msgr2=0x7f9c580efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:43.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.654 7f9c5f40e700 1 --2- 172.21.2.101:0/916392825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c580ef610 0x7f9c580efa10 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 rx=0x7f9c54006700 tx=0x7f9c54009b00).stop 2021-02-10T11:39:43.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.654 7f9c5f40e700 1 -- 172.21.2.101:0/916392825 shutdown_connections 2021-02-10T11:39:43.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.654 7f9c5f40e700 1 --2- 172.21.2.101:0/916392825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c580ef610 0x7f9c580efa10 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:43.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.654 7f9c5f40e700 1 --2- 172.21.2.101:0/916392825 >> v2:172.21.2.108:3300/0 conn(0x7f9c581b0470 0x7f9c581b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:43.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.654 7f9c5f40e700 1 -- 172.21.2.101:0/916392825 >> 172.21.2.101:0/916392825 conn(0x7f9c580ce3e0 msgr2=0x7f9c580e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:43.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.654 7f9c5f40e700 1 -- 172.21.2.101:0/916392825 shutdown_connections 2021-02-10T11:39:43.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.654 7f9c5f40e700 1 -- 172.21.2.101:0/916392825 wait complete. 2021-02-10T11:39:43.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.669 7f9c5f40e700 1 Processor -- start 2021-02-10T11:39:43.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.669 7f9c5f40e700 1 -- start start 2021-02-10T11:39:43.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.669 7f9c5f40e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9c581acc20 0x7f9c581ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:43.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.669 7f9c5f40e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c581ad640 0x7f9c581ada40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:43.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.669 7f9c5f40e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9c58188110 0x7f9c58188510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:43.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.669 7f9c5f40e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c581ac720 con 0x7f9c581ad640 2021-02-10T11:39:43.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.669 7f9c5f40e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c580b1b10 con 0x7f9c581acc20 2021-02-10T11:39:43.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.669 7f9c5f40e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c58188b30 con 0x7f9c58188110 2021-02-10T11:39:43.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.669 7f9c5ca29700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c581ad640 0x7f9c581ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:43.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.669 7f9c5ca29700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c581ad640 0x7f9c581ada40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55788/0 (socket says 172.21.2.101:55788) 2021-02-10T11:39:43.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.669 7f9c5ca29700 1 -- 172.21.2.101:0/1126410446 learned_addr learned my addr 172.21.2.101:0/1126410446 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:43.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.670 7f9c5ca29700 1 -- 172.21.2.101:0/1126410446 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9c58188110 msgr2=0x7f9c58188510 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:43.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.670 7f9c5ca29700 1 --2- 172.21.2.101:0/1126410446 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9c58188110 0x7f9c58188510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:43.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.670 7f9c5ca29700 1 -- 172.21.2.101:0/1126410446 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9c581acc20 msgr2=0x7f9c581ad020 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:43.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.670 7f9c5ca29700 1 --2- 172.21.2.101:0/1126410446 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9c581acc20 0x7f9c581ad020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:43.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.670 7f9c5ca29700 1 -- 172.21.2.101:0/1126410446 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c58188d50 con 0x7f9c581ad640 2021-02-10T11:39:43.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.670 7f9c5ca29700 1 --2- 172.21.2.101:0/1126410446 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c581ad640 0x7f9c581ada40 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 rx=0x7f9c54006f20 tx=0x7f9c5400b9e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:43.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.670 7f9c49ffb700 1 -- 172.21.2.101:0/1126410446 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9c5400bdd0 con 0x7f9c581ad640 2021-02-10T11:39:43.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.670 7f9c49ffb700 1 -- 172.21.2.101:0/1126410446 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c54006440 con 0x7f9c581ad640 2021-02-10T11:39:43.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.670 7f9c5f40e700 1 -- 172.21.2.101:0/1126410446 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c58188d50 con 0x7f9c581ad640 2021-02-10T11:39:43.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.670 7f9c5f40e700 1 -- 172.21.2.101:0/1126410446 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c581892e0 con 0x7f9c581ad640 2021-02-10T11:39:43.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.670 7f9c49ffb700 1 -- 172.21.2.101:0/1126410446 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9c540052e0 con 0x7f9c581ad640 2021-02-10T11:39:43.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.671 7f9c49ffb700 1 -- 172.21.2.101:0/1126410446 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9c54006030 con 0x7f9c581ad640 2021-02-10T11:39:43.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.671 7f9c49ffb700 1 --2- 172.21.2.101:0/1126410446 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9c38018730 0x7f9c3801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:43.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.671 7f9c49ffb700 1 -- 172.21.2.101:0/1126410446 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9c54032620 con 0x7f9c581ad640 2021-02-10T11:39:43.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.671 7f9c5d22a700 1 --2- 172.21.2.101:0/1126410446 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9c38018730 0x7f9c3801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:43.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.671 7f9c5d22a700 1 --2- 172.21.2.101:0/1126410446 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9c38018730 0x7f9c3801ac10 crc :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:43.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.703 7f9c5f40e700 1 -- 172.21.2.101:0/1126410446 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c580eddf0 con 0x7f9c581ad640 2021-02-10T11:39:43.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.705 7f9c49ffb700 1 -- 172.21.2.101:0/1126410446 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9c54009440 con 0x7f9c581ad640 2021-02-10T11:39:43.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.830 7f9c5f40e700 1 -- 172.21.2.101:0/1126410446 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f9c580e52e0 con 0x7f9c581ad640 2021-02-10T11:39:43.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.831 7f9c49ffb700 1 -- 172.21.2.101:0/1126410446 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v94) v1 ==== 81+0+30363 (secure 0 0 0) 0x7f9c54006440 con 0x7f9c581ad640 2021-02-10T11:39:43.834 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:39:43.834 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:39:43.835 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 94, 2021-02-10T11:39:43.835 INFO:teuthology.orchestra.run.gibba001.stdout: "fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:39:43.835 INFO:teuthology.orchestra.run.gibba001.stdout: "created": "2021-02-10 11:12:36.788366", 2021-02-10T11:39:43.835 INFO:teuthology.orchestra.run.gibba001.stdout: "modified": "2021-02-10 11:39:03.065120", 2021-02-10T11:39:43.835 INFO:teuthology.orchestra.run.gibba001.stdout: "last_up_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:39:43.835 INFO:teuthology.orchestra.run.gibba001.stdout: "last_in_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:39:43.835 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-02-10T11:39:43.835 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_num": 5799936, 2021-02-10T11:39:43.835 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_set": [ 2021-02-10T11:39:43.835 INFO:teuthology.orchestra.run.gibba001.stdout: "pglog_hardlimit", 2021-02-10T11:39:43.835 INFO:teuthology.orchestra.run.gibba001.stdout: "purged_snapdirs", 2021-02-10T11:39:43.835 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_deletes", 2021-02-10T11:39:43.835 INFO:teuthology.orchestra.run.gibba001.stdout: "sortbitwise" 2021-02-10T11:39:43.836 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:39:43.836 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_version": 4, 2021-02-10T11:39:43.836 INFO:teuthology.orchestra.run.gibba001.stdout: "full_ratio": 0.94999998807907104, 2021-02-10T11:39:43.836 INFO:teuthology.orchestra.run.gibba001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-02-10T11:39:43.836 INFO:teuthology.orchestra.run.gibba001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-02-10T11:39:43.836 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_snapshot": "", 2021-02-10T11:39:43.836 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_max": 19, 2021-02-10T11:39:43.836 INFO:teuthology.orchestra.run.gibba001.stdout: "max_osd": 8, 2021-02-10T11:39:43.836 INFO:teuthology.orchestra.run.gibba001.stdout: "require_min_compat_client": "jewel", 2021-02-10T11:39:43.836 INFO:teuthology.orchestra.run.gibba001.stdout: "min_compat_client": "jewel", 2021-02-10T11:39:43.836 INFO:teuthology.orchestra.run.gibba001.stdout: "require_osd_release": "nautilus", 2021-02-10T11:39:43.836 INFO:teuthology.orchestra.run.gibba001.stdout: "pools": [ 2021-02-10T11:39:43.837 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.837 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 1, 2021-02-10T11:39:43.837 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "rbd", 2021-02-10T11:39:43.837 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:13:08.037472", 2021-02-10T11:39:43.837 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:39:43.837 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:39:43.837 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:39:43.837 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:39:43.837 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:39:43.837 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:39:43.837 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:39:43.837 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:39:43.838 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:39:43.838 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:39:43.838 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:39:43.838 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:39:43.838 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:39:43.838 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:39:43.838 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:39:43.838 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:39:43.839 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:39:43.839 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:39:43.839 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:39:43.839 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:39:43.839 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.839 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "16", 2021-02-10T11:39:43.839 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:39:43.839 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:39:43.839 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:39:43.839 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:39:43.839 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:39:43.839 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:39:43.839 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:39:43.840 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:39:43.840 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:39:43.840 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:39:43.840 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:39:43.840 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:39:43.840 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:39:43.840 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:39:43.840 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:39:43.840 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:39:43.840 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:39:43.840 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:39:43.840 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:39:43.840 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:39:43.840 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:39:43.841 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:39:43.841 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:39:43.841 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:39:43.841 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:39:43.841 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:39:43.841 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.841 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:39:43.841 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:39:43.841 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:39:43.841 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:39:43.841 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:39:43.841 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:39:43.841 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:39:43.842 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:39:43.842 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:39:43.842 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:39:43.842 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:39:43.842 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:39:43.842 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:39:43.842 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd": {} 2021-02-10T11:39:43.842 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.842 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.842 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.842 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 18, 2021-02-10T11:39:43.842 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_metadata", 2021-02-10T11:39:43.842 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:39:00.027476", 2021-02-10T11:39:43.843 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:39:43.843 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:39:43.843 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:39:43.843 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:39:43.843 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:39:43.843 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:39:43.843 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:39:43.843 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:39:43.843 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:39:43.843 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:39:43.843 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:39:43.843 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:39:43.843 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:39:43.843 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:39:43.844 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:39:43.844 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:39:43.844 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:39:43.844 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:39:43.844 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:39:43.844 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:39:43.844 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.844 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "93", 2021-02-10T11:39:43.844 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:39:43.844 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:39:43.844 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:39:43.844 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:39:43.844 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:39:43.844 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:39:43.845 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:39:43.845 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:39:43.845 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:39:43.845 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:39:43.845 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:39:43.845 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:39:43.845 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:39:43.845 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:39:43.845 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:39:43.845 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:39:43.845 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:39:43.845 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:39:43.846 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:39:43.846 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:39:43.846 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:39:43.846 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:39:43.846 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:39:43.846 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:39:43.846 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:39:43.846 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:39:43.846 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.846 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:39:43.846 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:39:43.846 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:39:43.847 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:39:43.847 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:39:43.847 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:39:43.847 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:39:43.847 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:39:43.847 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:39:43.847 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:39:43.847 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:39:43.847 INFO:teuthology.orchestra.run.gibba001.stdout: "options": { 2021-02-10T11:39:43.847 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_bias": 4, 2021-02-10T11:39:43.847 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_min": 16, 2021-02-10T11:39:43.847 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_priority": 5 2021-02-10T11:39:43.848 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.848 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:39:43.848 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:39:43.848 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": "cephfs" 2021-02-10T11:39:43.848 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.848 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.848 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.848 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.848 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 19, 2021-02-10T11:39:43.848 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_data", 2021-02-10T11:39:43.848 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:39:01.054115", 2021-02-10T11:39:43.848 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:39:43.848 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:39:43.848 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:39:43.849 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:39:43.849 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:39:43.849 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:39:43.849 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:39:43.849 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:39:43.849 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:39:43.849 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:39:43.849 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:39:43.849 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:39:43.849 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:39:43.849 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:39:43.849 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:39:43.849 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:39:43.849 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:39:43.850 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:39:43.850 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:39:43.850 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:39:43.850 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.850 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "94", 2021-02-10T11:39:43.850 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:39:43.850 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:39:43.850 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:39:43.850 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:39:43.850 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:39:43.850 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:39:43.851 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:39:43.851 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:39:43.851 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:39:43.851 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:39:43.851 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:39:43.851 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:39:43.851 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:39:43.851 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:39:43.851 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:39:43.851 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:39:43.852 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:39:43.852 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:39:43.852 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:39:43.852 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:39:43.852 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:39:43.852 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:39:43.852 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:39:43.852 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:39:43.852 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:39:43.853 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:39:43.853 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.853 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:39:43.853 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:39:43.853 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:39:43.853 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:39:43.853 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:39:43.853 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:39:43.853 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:39:43.853 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:39:43.853 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:39:43.853 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:39:43.853 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:39:43.853 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:39:43.854 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:39:43.854 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:39:43.854 INFO:teuthology.orchestra.run.gibba001.stdout: "data": "cephfs" 2021-02-10T11:39:43.854 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.854 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.854 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.854 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:39:43.854 INFO:teuthology.orchestra.run.gibba001.stdout: "osds": [ 2021-02-10T11:39:43.857 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.857 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:39:43.857 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5", 2021-02-10T11:39:43.857 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:39:43.857 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:39:43.857 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:39:43.857 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:39:43.857 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:39:43.858 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:39:43.858 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:39:43.858 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 90, 2021-02-10T11:39:43.858 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:39:43.858 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:39:43.858 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:39:43.858 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.858 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.858 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.858 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6802", 2021-02-10T11:39:43.858 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:39:43.858 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.859 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.859 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.859 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6805", 2021-02-10T11:39:43.859 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:39:43.859 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.859 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.859 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.859 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:39:43.859 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.859 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.859 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.859 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6807", 2021-02-10T11:39:43.859 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:39:43.860 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.860 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.860 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.860 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6809", 2021-02-10T11:39:43.860 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:39:43.860 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.860 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.860 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.860 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:39:43.860 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.860 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.860 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.861 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6817", 2021-02-10T11:39:43.861 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:39:43.861 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.861 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.861 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.861 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6820", 2021-02-10T11:39:43.861 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:39:43.861 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.861 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.862 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.862 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:39:43.862 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.862 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.862 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.862 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6812", 2021-02-10T11:39:43.862 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:39:43.862 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.862 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.862 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.862 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6814", 2021-02-10T11:39:43.862 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:39:43.862 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.863 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.863 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.863 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6805/25093", 2021-02-10T11:39:43.863 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6809/25093", 2021-02-10T11:39:43.863 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6820/25093", 2021-02-10T11:39:43.863 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6814/25093", 2021-02-10T11:39:43.863 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:39:43.863 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:39:43.863 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:39:43.863 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.863 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.863 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.864 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:39:43.864 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "ddc04ce3-b313-4707-8270-585391e38ecc", 2021-02-10T11:39:43.864 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:39:43.864 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:39:43.864 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:39:43.864 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:39:43.864 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:39:43.864 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:39:43.864 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:39:43.864 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 92, 2021-02-10T11:39:43.864 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:39:43.864 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:39:43.864 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:39:43.864 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.865 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.865 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.865 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6803", 2021-02-10T11:39:43.865 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:39:43.865 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.865 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.865 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.865 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6804", 2021-02-10T11:39:43.865 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:39:43.865 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.865 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.865 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.865 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:39:43.866 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.866 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.866 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.866 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6806", 2021-02-10T11:39:43.866 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:39:43.866 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.866 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.866 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.866 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6808", 2021-02-10T11:39:43.866 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:39:43.866 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.866 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.866 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.867 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:39:43.867 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.867 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.867 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.867 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6818", 2021-02-10T11:39:43.867 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:39:43.867 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.867 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.867 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.867 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6821", 2021-02-10T11:39:43.867 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:39:43.867 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.867 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.868 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.868 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:39:43.868 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.868 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.868 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.868 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6810", 2021-02-10T11:39:43.868 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:39:43.868 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.868 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.868 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.868 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6815", 2021-02-10T11:39:43.868 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:39:43.868 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.868 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.869 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.869 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6804/25092", 2021-02-10T11:39:43.869 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6808/25092", 2021-02-10T11:39:43.869 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6821/25092", 2021-02-10T11:39:43.869 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6815/25092", 2021-02-10T11:39:43.869 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:39:43.869 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:39:43.869 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:39:43.869 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.869 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.869 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.869 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:39:43.869 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3", 2021-02-10T11:39:43.869 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:39:43.870 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:39:43.870 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:39:43.870 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:39:43.870 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:39:43.870 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:39:43.870 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:39:43.870 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 92, 2021-02-10T11:39:43.870 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:39:43.870 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:39:43.870 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:39:43.870 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.870 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.870 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.871 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6811", 2021-02-10T11:39:43.871 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:39:43.871 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.871 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.871 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.871 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6813", 2021-02-10T11:39:43.871 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:39:43.871 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.871 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.871 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.871 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:39:43.871 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.871 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.871 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.872 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6816", 2021-02-10T11:39:43.872 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:39:43.872 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.872 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.872 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.872 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6819", 2021-02-10T11:39:43.872 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:39:43.872 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.872 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.872 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.872 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:39:43.872 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.872 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.872 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.873 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6825", 2021-02-10T11:39:43.873 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:39:43.873 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.873 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.873 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.873 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6827", 2021-02-10T11:39:43.873 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:39:43.873 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.873 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.873 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.873 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:39:43.873 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.873 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.874 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.874 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6822", 2021-02-10T11:39:43.874 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:39:43.874 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.874 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.874 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.874 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6823", 2021-02-10T11:39:43.874 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:39:43.874 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.874 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.874 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.875 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6813/25094", 2021-02-10T11:39:43.875 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6819/25094", 2021-02-10T11:39:43.875 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6827/25094", 2021-02-10T11:39:43.875 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6823/25094", 2021-02-10T11:39:43.875 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:39:43.875 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:39:43.875 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:39:43.875 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.875 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.875 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.875 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:39:43.875 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "0d9ef9b1-98aa-4d37-aa16-6305c7d46d52", 2021-02-10T11:39:43.875 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:39:43.876 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:39:43.876 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:39:43.876 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:39:43.876 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:39:43.876 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:39:43.876 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:39:43.876 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 92, 2021-02-10T11:39:43.876 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:39:43.876 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:39:43.876 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:39:43.876 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.876 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.876 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.876 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6824", 2021-02-10T11:39:43.877 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:39:43.877 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.877 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.877 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.877 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6826", 2021-02-10T11:39:43.877 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:39:43.877 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.877 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.877 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.877 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:39:43.877 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.877 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.877 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.878 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6828", 2021-02-10T11:39:43.878 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:39:43.878 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.878 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.878 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.878 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6829", 2021-02-10T11:39:43.878 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:39:43.878 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.878 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.878 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.878 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:39:43.878 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.878 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.878 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.879 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6832", 2021-02-10T11:39:43.879 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:39:43.879 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.879 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.879 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.879 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6833", 2021-02-10T11:39:43.879 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:39:43.879 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.879 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.879 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.879 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:39:43.879 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.879 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.880 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.880 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6830", 2021-02-10T11:39:43.880 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:39:43.880 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.880 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.880 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.880 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6831", 2021-02-10T11:39:43.880 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:39:43.880 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.880 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.880 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.880 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6826/25095", 2021-02-10T11:39:43.880 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6829/25095", 2021-02-10T11:39:43.880 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6833/25095", 2021-02-10T11:39:43.881 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6831/25095", 2021-02-10T11:39:43.881 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:39:43.881 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:39:43.881 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:39:43.881 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.881 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.881 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.881 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:39:43.881 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "36742c21-bfeb-4e11-bce9-8e745378b605", 2021-02-10T11:39:43.881 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:39:43.881 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:39:43.881 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:39:43.881 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:39:43.881 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:39:43.882 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:39:43.882 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:39:43.882 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 92, 2021-02-10T11:39:43.882 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:39:43.882 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:39:43.882 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:39:43.882 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.882 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.882 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.882 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6816", 2021-02-10T11:39:43.882 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:39:43.882 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.882 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.883 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.883 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6818", 2021-02-10T11:39:43.883 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:39:43.883 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.883 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.883 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.883 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:39:43.883 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.883 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.883 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.883 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6820", 2021-02-10T11:39:43.883 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:39:43.884 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.884 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.884 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.884 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6822", 2021-02-10T11:39:43.884 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:39:43.884 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.884 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.884 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.884 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:39:43.884 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.885 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.885 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.885 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6828", 2021-02-10T11:39:43.885 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:39:43.885 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.885 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.885 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.885 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6830", 2021-02-10T11:39:43.885 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:39:43.885 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.886 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.886 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.886 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:39:43.886 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.886 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.886 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.886 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6824", 2021-02-10T11:39:43.886 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:39:43.886 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.886 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.886 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.886 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6826", 2021-02-10T11:39:43.887 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:39:43.887 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.887 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.887 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.887 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6818/36380", 2021-02-10T11:39:43.887 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6822/36380", 2021-02-10T11:39:43.887 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6830/36380", 2021-02-10T11:39:43.887 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6826/36380", 2021-02-10T11:39:43.887 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:39:43.887 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:39:43.887 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:39:43.887 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.887 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.888 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.888 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:39:43.888 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "88e694e8-fe98-4d1b-88c8-8c9907ece60b", 2021-02-10T11:39:43.888 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:39:43.888 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:39:43.888 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:39:43.888 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:39:43.888 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:39:43.888 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:39:43.888 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:39:43.888 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 92, 2021-02-10T11:39:43.888 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:39:43.888 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:39:43.888 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:39:43.889 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.889 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.889 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.889 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6808", 2021-02-10T11:39:43.889 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:39:43.889 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.889 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.889 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.889 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6809", 2021-02-10T11:39:43.889 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:39:43.889 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.889 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.889 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.890 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:39:43.890 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.890 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.890 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.890 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6810", 2021-02-10T11:39:43.890 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:39:43.890 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.890 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.890 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.890 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6811", 2021-02-10T11:39:43.890 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:39:43.890 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.890 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.890 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.891 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:39:43.891 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.891 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.891 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.891 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6814", 2021-02-10T11:39:43.891 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:39:43.891 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.891 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.891 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.891 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6815", 2021-02-10T11:39:43.891 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:39:43.891 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.891 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.892 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.892 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:39:43.892 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.892 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.892 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.892 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6812", 2021-02-10T11:39:43.892 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:39:43.892 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.892 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.892 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.892 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6813", 2021-02-10T11:39:43.892 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:39:43.892 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.892 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.893 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.893 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6809/36383", 2021-02-10T11:39:43.893 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6811/36383", 2021-02-10T11:39:43.893 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6815/36383", 2021-02-10T11:39:43.893 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6813/36383", 2021-02-10T11:39:43.893 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:39:43.893 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:39:43.893 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:39:43.893 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.893 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.893 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.893 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:39:43.893 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "87a13c6c-cf85-4aaf-916b-e73913484a49", 2021-02-10T11:39:43.893 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:39:43.894 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:39:43.894 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:39:43.894 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:39:43.894 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:39:43.894 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:39:43.894 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:39:43.894 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 82, 2021-02-10T11:39:43.894 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:39:43.894 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:39:43.894 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:39:43.894 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.894 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.894 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.895 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6800", 2021-02-10T11:39:43.895 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:39:43.895 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.895 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.895 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.895 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6801", 2021-02-10T11:39:43.895 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:39:43.895 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.895 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.895 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.895 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:39:43.895 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.895 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.896 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.896 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6802", 2021-02-10T11:39:43.896 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:39:43.896 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.896 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.896 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.896 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6803", 2021-02-10T11:39:43.896 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:39:43.896 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.896 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.896 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.896 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:39:43.896 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.896 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.897 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.897 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6806", 2021-02-10T11:39:43.897 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:39:43.897 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.897 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.897 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.897 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6807", 2021-02-10T11:39:43.897 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:39:43.897 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.897 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.898 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.898 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:39:43.898 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.898 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.898 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.898 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6804", 2021-02-10T11:39:43.898 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:39:43.898 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.898 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.898 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.898 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6805", 2021-02-10T11:39:43.898 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:39:43.899 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.899 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.899 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.899 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6801/36381", 2021-02-10T11:39:43.899 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6803/36381", 2021-02-10T11:39:43.899 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6807/36381", 2021-02-10T11:39:43.899 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6805/36381", 2021-02-10T11:39:43.899 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:39:43.899 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:39:43.899 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:39:43.899 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.899 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.899 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.899 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:39:43.900 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "7b90823f-854c-4592-a95b-a7fa98f93254", 2021-02-10T11:39:43.900 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:39:43.900 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:39:43.900 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:39:43.900 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:39:43.900 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:39:43.900 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:39:43.900 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:39:43.900 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 92, 2021-02-10T11:39:43.900 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:39:43.900 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:39:43.900 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:39:43.900 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.900 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.901 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.901 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6817", 2021-02-10T11:39:43.901 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:39:43.901 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.901 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.901 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.901 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6819", 2021-02-10T11:39:43.901 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:39:43.901 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.901 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.901 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.901 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:39:43.901 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.902 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.902 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.902 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6821", 2021-02-10T11:39:43.902 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:39:43.902 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.902 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.902 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.902 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6823", 2021-02-10T11:39:43.902 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:39:43.902 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.902 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.902 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.902 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:39:43.903 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.903 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.903 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.903 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6829", 2021-02-10T11:39:43.903 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:39:43.903 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.903 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.903 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.903 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6831", 2021-02-10T11:39:43.903 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:39:43.904 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.904 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.904 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.904 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:39:43.904 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:39:43.904 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.904 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:39:43.904 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6825", 2021-02-10T11:39:43.904 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:39:43.904 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.904 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.904 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:39:43.904 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6827", 2021-02-10T11:39:43.904 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:39:43.905 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.905 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.905 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.905 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6819/36382", 2021-02-10T11:39:43.905 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6823/36382", 2021-02-10T11:39:43.905 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6831/36382", 2021-02-10T11:39:43.905 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6827/36382", 2021-02-10T11:39:43.905 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:39:43.905 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:39:43.905 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:39:43.905 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:43.905 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.905 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:39:43.905 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_xinfo": [ 2021-02-10T11:39:43.906 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.906 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:39:43.906 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:39:43.906 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:39:43.906 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:39:43.906 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:39:43.906 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:39:43.906 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.906 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.906 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:39:43.906 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:39:43.906 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:39:43.906 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:39:43.907 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:39:43.907 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:39:43.907 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.907 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.907 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:39:43.907 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:39:43.907 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:39:43.907 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:39:43.907 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:39:43.907 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:39:43.907 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.907 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.908 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:39:43.908 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:39:43.908 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:39:43.908 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:39:43.908 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:39:43.908 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:39:43.908 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.908 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.908 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:39:43.908 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:39:43.908 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:39:43.908 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:39:43.908 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:39:43.909 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:39:43.909 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.909 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.909 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:39:43.909 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:39:43.909 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:39:43.909 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:39:43.909 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:39:43.909 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:39:43.909 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.909 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.909 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:39:43.910 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:39:43.910 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:39:43.910 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:39:43.910 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:39:43.910 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:39:43.910 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.910 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:43.910 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:39:43.911 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:39:43.911 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:39:43.911 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:39:43.911 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:39:43.911 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:39:43.911 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.911 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:39:43.912 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap": [], 2021-02-10T11:39:43.912 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap_items": [], 2021-02-10T11:39:43.912 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_temp": [], 2021-02-10T11:39:43.912 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_temp": [], 2021-02-10T11:39:43.912 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklist": {}, 2021-02-10T11:39:43.912 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profiles": { 2021-02-10T11:39:43.912 INFO:teuthology.orchestra.run.gibba001.stdout: "default": { 2021-02-10T11:39:43.913 INFO:teuthology.orchestra.run.gibba001.stdout: "crush-failure-domain": "osd", 2021-02-10T11:39:43.913 INFO:teuthology.orchestra.run.gibba001.stdout: "k": "2", 2021-02-10T11:39:43.913 INFO:teuthology.orchestra.run.gibba001.stdout: "m": "1", 2021-02-10T11:39:43.913 INFO:teuthology.orchestra.run.gibba001.stdout: "plugin": "jerasure", 2021-02-10T11:39:43.913 INFO:teuthology.orchestra.run.gibba001.stdout: "ruleset-failure-domain": "osd", 2021-02-10T11:39:43.913 INFO:teuthology.orchestra.run.gibba001.stdout: "technique": "reed_sol_van" 2021-02-10T11:39:43.914 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:43.914 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:43.914 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps_queue": [], 2021-02-10T11:39:43.914 INFO:teuthology.orchestra.run.gibba001.stdout: "new_removed_snaps": [], 2021-02-10T11:39:43.914 INFO:teuthology.orchestra.run.gibba001.stdout: "new_purged_snaps": [], 2021-02-10T11:39:43.914 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_node_flags": {}, 2021-02-10T11:39:43.915 INFO:teuthology.orchestra.run.gibba001.stdout: "device_class_flags": {} 2021-02-10T11:39:43.915 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:39:43.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.832 7f9c5f40e700 1 -- 172.21.2.101:0/1126410446 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9c38018730 msgr2=0x7f9c3801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:43.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.832 7f9c5f40e700 1 --2- 172.21.2.101:0/1126410446 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9c38018730 0x7f9c3801ac10 crc :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:43.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.832 7f9c5f40e700 1 -- 172.21.2.101:0/1126410446 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c581ad640 msgr2=0x7f9c581ada40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:43.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.832 7f9c5f40e700 1 --2- 172.21.2.101:0/1126410446 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c581ad640 0x7f9c581ada40 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 rx=0x7f9c54006f20 tx=0x7f9c5400b9e0).stop 2021-02-10T11:39:43.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.832 7f9c5f40e700 1 -- 172.21.2.101:0/1126410446 shutdown_connections 2021-02-10T11:39:43.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.832 7f9c5f40e700 1 --2- 172.21.2.101:0/1126410446 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9c38018730 0x7f9c3801ac10 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:43.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.832 7f9c5f40e700 1 --2- 172.21.2.101:0/1126410446 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9c58188110 0x7f9c58188510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:43.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.832 7f9c5f40e700 1 --2- 172.21.2.101:0/1126410446 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c581ad640 0x7f9c581ada40 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:43.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.832 7f9c5f40e700 1 --2- 172.21.2.101:0/1126410446 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9c581acc20 0x7f9c581ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:43.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.832 7f9c5f40e700 1 -- 172.21.2.101:0/1126410446 >> 172.21.2.101:0/1126410446 conn(0x7f9c5819f7a0 msgr2=0x7f9c581922a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:43.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.832 7f9c5f40e700 1 -- 172.21.2.101:0/1126410446 shutdown_connections 2021-02-10T11:39:43.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:43.832 7f9c5f40e700 1 -- 172.21.2.101:0/1126410446 wait complete. 2021-02-10T11:39:43.921 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:39:44.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.329 7f1787da9700 1 Processor -- start 2021-02-10T11:39:44.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.329 7f1787da9700 1 -- start start 2021-02-10T11:39:44.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.330 7f1787da9700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f17800d1c70 0x7f17800f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:44.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.330 7f1787da9700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17800f0b10 con 0x7f17800d1870 2021-02-10T11:39:44.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.330 7f1787da9700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17801b0be0 con 0x7f17800d0340 2021-02-10T11:39:44.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.330 7f1787da9700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17800e5ed0 con 0x7f17800d1c70 2021-02-10T11:39:44.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.330 7f17863c6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f17800d1c70 0x7f17800f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:44.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.330 7f17863c6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f17800d1c70 0x7f17800f20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55798/0 (socket says 172.21.2.101:55798) 2021-02-10T11:39:44.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.330 7f17863c6700 1 -- 172.21.2.101:0/3195250150 learned_addr learned my addr 172.21.2.101:0/3195250150 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:44.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.330 7f17863c6700 1 -- 172.21.2.101:0/3195250150 >> v1:172.21.2.108:6789/0 conn(0x7f17800d0340 legacy=0x7f17800ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:44.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.330 7f17863c6700 1 -- 172.21.2.101:0/3195250150 >> v1:172.21.2.101:6789/0 conn(0x7f17800d1870 legacy=0x7f17801b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:44.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.330 7f17863c6700 1 -- 172.21.2.101:0/3195250150 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17800efe30 con 0x7f17800d1c70 2021-02-10T11:39:44.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.330 7f17863c6700 1 --2- 172.21.2.101:0/3195250150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f17800d1c70 0x7f17800f20a0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 rx=0x7f1778006f00 tx=0x7f1778009f10).ready entity=mon.0 client_cookie=d78bfefef1f7a04c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:44.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.331 7f1784bc3700 1 -- 172.21.2.101:0/3195250150 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1778003030 con 0x7f17800d1c70 2021-02-10T11:39:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.331 7f1784bc3700 1 -- 172.21.2.101:0/3195250150 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177800b3b0 con 0x7f17800d1c70 2021-02-10T11:39:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.331 7f1784bc3700 1 -- 172.21.2.101:0/3195250150 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f177800b8d0 con 0x7f17800d1c70 2021-02-10T11:39:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.331 7f1787da9700 1 -- 172.21.2.101:0/3195250150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f17800d1c70 msgr2=0x7f17800f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.331 7f1787da9700 1 --2- 172.21.2.101:0/3195250150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f17800d1c70 0x7f17800f20a0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 rx=0x7f1778006f00 tx=0x7f1778009f10).stop 2021-02-10T11:39:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.331 7f1787da9700 1 -- 172.21.2.101:0/3195250150 shutdown_connections 2021-02-10T11:39:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.331 7f1787da9700 1 --2- 172.21.2.101:0/3195250150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f17800d1c70 0x7f17800f20a0 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.331 7f1787da9700 1 -- 172.21.2.101:0/3195250150 >> 172.21.2.101:0/3195250150 conn(0x7f17800ce450 msgr2=0x7f17800d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.331 7f1787da9700 1 -- 172.21.2.101:0/3195250150 shutdown_connections 2021-02-10T11:39:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.331 7f1787da9700 1 -- 172.21.2.101:0/3195250150 wait complete. 2021-02-10T11:39:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.346 7f1787da9700 1 Processor -- start 2021-02-10T11:39:44.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.346 7f1787da9700 1 -- start start 2021-02-10T11:39:44.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.346 7f1787da9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1780092010 0x7f17800c33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:44.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.346 7f1787da9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17800c39e0 0x7f17800af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:44.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.346 7f1787da9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f17800af660 0x7f17800afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:44.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.346 7f1787da9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17800b0500 con 0x7f1780092010 2021-02-10T11:39:44.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.346 7f1787da9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17800b0080 con 0x7f17800af660 2021-02-10T11:39:44.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.346 7f1787da9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17800b02a0 con 0x7f17800c39e0 2021-02-10T11:39:44.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.347 7f1785bc5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1780092010 0x7f17800c33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:44.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.347 7f1785bc5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1780092010 0x7f17800c33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55802/0 (socket says 172.21.2.101:55802) 2021-02-10T11:39:44.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.347 7f1785bc5700 1 -- 172.21.2.101:0/85116968 learned_addr learned my addr 172.21.2.101:0/85116968 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:44.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.347 7f1785bc5700 1 -- 172.21.2.101:0/85116968 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17800c39e0 msgr2=0x7f17800af040 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:44.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.347 7f1785bc5700 1 --2- 172.21.2.101:0/85116968 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17800c39e0 0x7f17800af040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:44.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.347 7f1785bc5700 1 -- 172.21.2.101:0/85116968 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f17800af660 msgr2=0x7f17800afa60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:44.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.347 7f1785bc5700 1 --2- 172.21.2.101:0/85116968 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f17800af660 0x7f17800afa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:44.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.347 7f1785bc5700 1 -- 172.21.2.101:0/85116968 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17800b0a70 con 0x7f1780092010 2021-02-10T11:39:44.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.347 7f1785bc5700 1 --2- 172.21.2.101:0/85116968 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1780092010 0x7f17800c33c0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 rx=0x7f177c006ed0 tx=0x7f177c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:44.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.347 7f17727fc700 1 -- 172.21.2.101:0/85116968 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f177c006020 con 0x7f1780092010 2021-02-10T11:39:44.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.347 7f17727fc700 1 -- 172.21.2.101:0/85116968 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177c0062a0 con 0x7f1780092010 2021-02-10T11:39:44.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.347 7f17727fc700 1 -- 172.21.2.101:0/85116968 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f177c005730 con 0x7f1780092010 2021-02-10T11:39:44.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.347 7f1787da9700 1 -- 172.21.2.101:0/85116968 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17800b0a70 con 0x7f1780092010 2021-02-10T11:39:44.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.347 7f1787da9700 1 -- 172.21.2.101:0/85116968 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17800b0a70 con 0x7f1780092010 2021-02-10T11:39:44.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.348 7f17727fc700 1 -- 172.21.2.101:0/85116968 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f177c0054b0 con 0x7f1780092010 2021-02-10T11:39:44.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.348 7f17727fc700 1 --2- 172.21.2.101:0/85116968 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f17680186c0 0x7f176801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:44.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.348 7f17727fc700 1 -- 172.21.2.101:0/85116968 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f177c032570 con 0x7f1780092010 2021-02-10T11:39:44.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.348 7f17853c4700 1 --2- 172.21.2.101:0/85116968 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f17680186c0 0x7f176801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:44.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.348 7f17853c4700 1 --2- 172.21.2.101:0/85116968 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f17680186c0 0x7f176801abc0 crc :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:44.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.380 7f1787da9700 1 -- 172.21.2.101:0/85116968 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17800e7df0 con 0x7f1780092010 2021-02-10T11:39:44.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.383 7f17727fc700 1 -- 172.21.2.101:0/85116968 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f177c0009b0 con 0x7f1780092010 2021-02-10T11:39:44.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.511 7f1787da9700 1 -- 172.21.2.101:0/85116968 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17800ce450 con 0x7f1780092010 2021-02-10T11:39:44.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.512 7f17727fc700 1 -- 172.21.2.101:0/85116968 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f177c00bd50 con 0x7f1780092010 2021-02-10T11:39:44.516 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 72 2021-02-10T11:39:44.516 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:39:44.516 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":72,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/4210469578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":4210469578},{"type":"v1","addr":"172.21.2.108:6833","nonce":4210469578}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:39:02.063078","modified":"2021-02-10 11:39:04.073713","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6164},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6164":{"gid":6164,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.2.101:6835/1113812290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1113812290},{"type":"v1","addr":"172.21.2.101:6835","nonce":1113812290}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-02-10T11:39:44.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.514 7f1787da9700 1 -- 172.21.2.101:0/85116968 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f17680186c0 msgr2=0x7f176801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:44.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.514 7f1787da9700 1 --2- 172.21.2.101:0/85116968 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f17680186c0 0x7f176801abc0 crc :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:44.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.514 7f1787da9700 1 -- 172.21.2.101:0/85116968 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1780092010 msgr2=0x7f17800c33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:44.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.514 7f1787da9700 1 --2- 172.21.2.101:0/85116968 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1780092010 0x7f17800c33c0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 rx=0x7f177c006ed0 tx=0x7f177c00b960).stop 2021-02-10T11:39:44.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.514 7f1787da9700 1 -- 172.21.2.101:0/85116968 shutdown_connections 2021-02-10T11:39:44.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.514 7f1787da9700 1 --2- 172.21.2.101:0/85116968 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f17680186c0 0x7f176801abc0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:44.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.514 7f1787da9700 1 --2- 172.21.2.101:0/85116968 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17800c39e0 0x7f17800af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:44.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.514 7f1787da9700 1 --2- 172.21.2.101:0/85116968 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f17800af660 0x7f17800afa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:44.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.514 7f1787da9700 1 --2- 172.21.2.101:0/85116968 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1780092010 0x7f17800c33c0 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:44.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.514 7f1787da9700 1 -- 172.21.2.101:0/85116968 >> 172.21.2.101:0/85116968 conn(0x7f17800a0030 msgr2=0x7f17800f22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:44.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.514 7f1787da9700 1 -- 172.21.2.101:0/85116968 shutdown_connections 2021-02-10T11:39:44.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.514 7f1787da9700 1 -- 172.21.2.101:0/85116968 wait complete. 2021-02-10T11:39:44.526 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs fail cephfs 2021-02-10T11:39:44.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.934 7fc748eea700 1 Processor -- start 2021-02-10T11:39:44.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.934 7fc748eea700 1 -- start start 2021-02-10T11:39:44.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.935 7fc748eea700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7440d1870 0x7fc7441b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:44.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.935 7fc748eea700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc7440f0650 0x7fc7440cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:44.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.935 7fc748eea700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7440f0b10 con 0x7fc7441b0a10 2021-02-10T11:39:44.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.935 7fc748eea700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7440fe230 con 0x7fc7440f0650 2021-02-10T11:39:44.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.935 7fc748eea700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7440e5ed0 con 0x7fc7440d1870 2021-02-10T11:39:44.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.935 7fc743314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc7440f0650 0x7fc7440cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:44.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.935 7fc743314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc7440f0650 0x7fc7440cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55812/0 (socket says 172.21.2.101:55812) 2021-02-10T11:39:44.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.935 7fc743314700 1 -- 172.21.2.101:0/2804020880 learned_addr learned my addr 172.21.2.101:0/2804020880 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:44.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.935 7fc742b13700 1 --2- 172.21.2.101:0/2804020880 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7440d1870 0x7fc7441b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:44.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.935 7fc743314700 1 -- 172.21.2.101:0/2804020880 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7440d1870 msgr2=0x7fc7441b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:44.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.935 7fc743314700 1 --2- 172.21.2.101:0/2804020880 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7440d1870 0x7fc7441b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:44.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.935 7fc743314700 1 -- 172.21.2.101:0/2804020880 >> v1:172.21.2.108:6789/0 conn(0x7fc7441b0a10 legacy=0x7fc7440cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:44.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.935 7fc743314700 1 -- 172.21.2.101:0/2804020880 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7440f20a0 con 0x7fc7440f0650 2021-02-10T11:39:44.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.935 7fc743314700 1 --2- 172.21.2.101:0/2804020880 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7440f0650 0x7fc7440cfb20 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 rx=0x7fc73c006f00 tx=0x7fc73c009f10).ready entity=mon.0 client_cookie=b821ab287453fd45 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:44.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.935 7fc741b11700 1 -- 172.21.2.101:0/2804020880 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc73c003030 con 0x7fc7440f0650 2021-02-10T11:39:44.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.935 7fc741b11700 1 -- 172.21.2.101:0/2804020880 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73c00b3b0 con 0x7fc7440f0650 2021-02-10T11:39:44.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.935 7fc741b11700 1 -- 172.21.2.101:0/2804020880 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc73c00b8d0 con 0x7fc7440f0650 2021-02-10T11:39:44.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.935 7fc748eea700 1 -- 172.21.2.101:0/2804020880 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7440f0650 msgr2=0x7fc7440cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:44.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.935 7fc748eea700 1 --2- 172.21.2.101:0/2804020880 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7440f0650 0x7fc7440cfb20 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 rx=0x7fc73c006f00 tx=0x7fc73c009f10).stop 2021-02-10T11:39:44.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.936 7fc748eea700 1 -- 172.21.2.101:0/2804020880 shutdown_connections 2021-02-10T11:39:44.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.936 7fc748eea700 1 --2- 172.21.2.101:0/2804020880 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7440f0650 0x7fc7440cfb20 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:44.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.936 7fc748eea700 1 --2- 172.21.2.101:0/2804020880 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7440d1870 0x7fc7441b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:44.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.936 7fc748eea700 1 -- 172.21.2.101:0/2804020880 >> 172.21.2.101:0/2804020880 conn(0x7fc7440ce450 msgr2=0x7fc7440d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:44.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.936 7fc748eea700 1 -- 172.21.2.101:0/2804020880 shutdown_connections 2021-02-10T11:39:44.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.936 7fc748eea700 1 -- 172.21.2.101:0/2804020880 wait complete. 2021-02-10T11:39:44.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.951 7fc748eea700 1 Processor -- start 2021-02-10T11:39:44.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.951 7fc748eea700 1 -- start start 2021-02-10T11:39:44.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.951 7fc748eea700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7440be330 0x7fc7441a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:44.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.951 7fc748eea700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7441a4a40 0x7fc744187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:44.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.951 7fc748eea700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc744188470 0x7fc744188870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:44.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.951 7fc748eea700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7440be730 con 0x7fc7440be330 2021-02-10T11:39:44.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.951 7fc748eea700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7440b64c0 con 0x7fc744188470 2021-02-10T11:39:44.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.951 7fc748eea700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7440b66e0 con 0x7fc7441a4a40 2021-02-10T11:39:44.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.951 7fc742b13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7440be330 0x7fc7441a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:44.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.951 7fc742b13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7440be330 0x7fc7441a4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55814/0 (socket says 172.21.2.101:55814) 2021-02-10T11:39:44.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.951 7fc742b13700 1 -- 172.21.2.101:0/348626922 learned_addr learned my addr 172.21.2.101:0/348626922 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:44.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.952 7fc742b13700 1 -- 172.21.2.101:0/348626922 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7441a4a40 msgr2=0x7fc744187e50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:44.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.952 7fc742b13700 1 --2- 172.21.2.101:0/348626922 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7441a4a40 0x7fc744187e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:44.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.952 7fc742b13700 1 -- 172.21.2.101:0/348626922 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc744188470 msgr2=0x7fc744188870 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:44.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.952 7fc742b13700 1 --2- 172.21.2.101:0/348626922 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc744188470 0x7fc744188870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:44.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.952 7fc742b13700 1 -- 172.21.2.101:0/348626922 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7440b6900 con 0x7fc7440be330 2021-02-10T11:39:44.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.952 7fc742b13700 1 --2- 172.21.2.101:0/348626922 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7440be330 0x7fc7441a4420 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 rx=0x7fc72c0067e0 tx=0x7fc72c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:44.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.952 7fc73b7fe700 1 -- 172.21.2.101:0/348626922 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc72c006900 con 0x7fc7440be330 2021-02-10T11:39:44.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.952 7fc73b7fe700 1 -- 172.21.2.101:0/348626922 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72c000c90 con 0x7fc7440be330 2021-02-10T11:39:44.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.952 7fc748eea700 1 -- 172.21.2.101:0/348626922 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7440b6900 con 0x7fc7440be330 2021-02-10T11:39:44.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.952 7fc748eea700 1 -- 172.21.2.101:0/348626922 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7440b6e90 con 0x7fc7440be330 2021-02-10T11:39:44.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.952 7fc73b7fe700 1 -- 172.21.2.101:0/348626922 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc72c002300 con 0x7fc7440be330 2021-02-10T11:39:44.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.953 7fc73b7fe700 1 -- 172.21.2.101:0/348626922 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc72c0027d0 con 0x7fc7440be330 2021-02-10T11:39:44.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.953 7fc73b7fe700 1 --2- 172.21.2.101:0/348626922 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc728018730 0x7fc72801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:44.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.953 7fc73b7fe700 1 -- 172.21.2.101:0/348626922 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc72c0327f0 con 0x7fc7440be330 2021-02-10T11:39:44.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.953 7fc742312700 1 --2- 172.21.2.101:0/348626922 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc728018730 0x7fc72801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:44.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.953 7fc742312700 1 --2- 172.21.2.101:0/348626922 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc728018730 0x7fc72801ac10 crc :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:44.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.985 7fc748eea700 1 -- 172.21.2.101:0/348626922 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7440ebc90 con 0x7fc7440be330 2021-02-10T11:39:44.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:44.987 7fc73b7fe700 1 -- 172.21.2.101:0/348626922 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc72c0025a0 con 0x7fc7440be330 2021-02-10T11:39:45.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:45.115 7fc748eea700 1 -- 172.21.2.101:0/348626922 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc7440f0b10 con 0x7fc7440be330 2021-02-10T11:39:45.892 INFO:tasks.ceph.mon.a.gibba001.stderr:2021-02-10 11:39:45.889 7f974d04d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-02-10T11:39:45.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:45.895 7fc73b7fe700 1 -- 172.21.2.101:0/348626922 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v73) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc72c0027d0 con 0x7fc7440be330 2021-02-10T11:39:45.937 INFO:teuthology.orchestra.run.gibba001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-02-10T11:39:45.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:45.936 7fc748eea700 1 -- 172.21.2.101:0/348626922 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc728018730 msgr2=0x7fc72801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:45.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:45.936 7fc748eea700 1 --2- 172.21.2.101:0/348626922 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc728018730 0x7fc72801ac10 crc :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:45.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:45.936 7fc748eea700 1 -- 172.21.2.101:0/348626922 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7440be330 msgr2=0x7fc7441a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:45.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:45.936 7fc748eea700 1 --2- 172.21.2.101:0/348626922 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7440be330 0x7fc7441a4420 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 rx=0x7fc72c0067e0 tx=0x7fc72c00b990).stop 2021-02-10T11:39:45.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:45.936 7fc748eea700 1 -- 172.21.2.101:0/348626922 shutdown_connections 2021-02-10T11:39:45.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:45.936 7fc748eea700 1 --2- 172.21.2.101:0/348626922 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc728018730 0x7fc72801ac10 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:45.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:45.936 7fc748eea700 1 --2- 172.21.2.101:0/348626922 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7441a4a40 0x7fc744187e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:45.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:45.936 7fc748eea700 1 --2- 172.21.2.101:0/348626922 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc744188470 0x7fc744188870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:45.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:45.936 7fc748eea700 1 --2- 172.21.2.101:0/348626922 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7440be330 0x7fc7441a4420 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:45.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:45.936 7fc748eea700 1 -- 172.21.2.101:0/348626922 >> 172.21.2.101:0/348626922 conn(0x7fc7440a6370 msgr2=0x7fc744094c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:45.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:45.936 7fc748eea700 1 -- 172.21.2.101:0/348626922 shutdown_connections 2021-02-10T11:39:45.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:45.936 7fc748eea700 1 -- 172.21.2.101:0/348626922 wait complete. 2021-02-10T11:39:45.948 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:39:45.951 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:39:46.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.355 7f378dcd2700 1 Processor -- start 2021-02-10T11:39:46.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.355 7f378dcd2700 1 -- start start 2021-02-10T11:39:46.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.355 7f378dcd2700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f37880d0340 0x7f37880ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:46.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.355 7f378dcd2700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37880f0b10 con 0x7f37880d1870 2021-02-10T11:39:46.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.355 7f378dcd2700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37881b0be0 con 0x7f37880d1c70 2021-02-10T11:39:46.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.355 7f378dcd2700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37880e5ed0 con 0x7f37880d0340 2021-02-10T11:39:46.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.355 7f37877fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f37880d1870 0x7f37881b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56328/0 (socket says 172.21.2.101:56328) 2021-02-10T11:39:46.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.355 7f37877fe700 1 -- 172.21.2.101:0/401733631 learned_addr learned my addr 172.21.2.101:0/401733631 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:46.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.356 7f3786ffd700 1 --2- 172.21.2.101:0/401733631 >> v2:172.21.2.108:3300/0 conn(0x7f37880d0340 0x7f37880ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:46.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.356 7f37867fc700 1 -- 172.21.2.101:0/401733631 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439666728 0 0) 0x7f37780010b0 con 0x7f37880d1870 2021-02-10T11:39:46.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.356 7f37867fc700 1 -- 172.21.2.101:0/401733631 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3774002fe0 con 0x7f37880d1870 2021-02-10T11:39:46.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.356 7f37867fc700 1 -- 172.21.2.101:0/401733631 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 747543765 0 0) 0x7f3780001400 con 0x7f37880d1c70 2021-02-10T11:39:46.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.356 7f37867fc700 1 -- 172.21.2.101:0/401733631 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3774003210 con 0x7f37880d1c70 2021-02-10T11:39:46.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.356 7f3786ffd700 1 -- 172.21.2.101:0/401733631 >> v1:172.21.2.108:6789/0 conn(0x7f37880d1c70 legacy=0x7f37880efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:46.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.356 7f3786ffd700 1 -- 172.21.2.101:0/401733631 >> v1:172.21.2.101:6789/0 conn(0x7f37880d1870 legacy=0x7f37881b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:46.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.356 7f3786ffd700 1 -- 172.21.2.101:0/401733631 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37880f0450 con 0x7f37880d0340 2021-02-10T11:39:46.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.356 7f3786ffd700 1 --2- 172.21.2.101:0/401733631 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37880d0340 0x7f37880ef640 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f3770006650 tx=0x7f3770009a80).ready entity=mon.1 client_cookie=f5d3e2937c34c76e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:46.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.357 7f37867fc700 1 -- 172.21.2.101:0/401733631 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f377000c0c0 con 0x7f37880d0340 2021-02-10T11:39:46.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.357 7f37867fc700 1 -- 172.21.2.101:0/401733631 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377000c340 con 0x7f37880d0340 2021-02-10T11:39:46.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.357 7f37867fc700 1 -- 172.21.2.101:0/401733631 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f377000c860 con 0x7f37880d0340 2021-02-10T11:39:46.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.357 7f378dcd2700 1 -- 172.21.2.101:0/401733631 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37880d0340 msgr2=0x7f37880ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:46.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.357 7f378dcd2700 1 --2- 172.21.2.101:0/401733631 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37880d0340 0x7f37880ef640 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f3770006650 tx=0x7f3770009a80).stop 2021-02-10T11:39:46.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.357 7f378dcd2700 1 -- 172.21.2.101:0/401733631 shutdown_connections 2021-02-10T11:39:46.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.357 7f378dcd2700 1 --2- 172.21.2.101:0/401733631 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37880d0340 0x7f37880ef640 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:46.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.357 7f378dcd2700 1 -- 172.21.2.101:0/401733631 >> 172.21.2.101:0/401733631 conn(0x7f37880ce450 msgr2=0x7f37880d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:46.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.358 7f378dcd2700 1 -- 172.21.2.101:0/401733631 shutdown_connections 2021-02-10T11:39:46.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.358 7f378dcd2700 1 -- 172.21.2.101:0/401733631 wait complete. 2021-02-10T11:39:46.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.372 7f378dcd2700 1 Processor -- start 2021-02-10T11:39:46.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.372 7f378dcd2700 1 -- start start 2021-02-10T11:39:46.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.373 7f378dcd2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f378818d870 0x7f37880b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:46.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.373 7f378dcd2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37880b1fd0 0x7f3788099ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:46.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.373 7f378dcd2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37880b23d0 0x7f378809a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:46.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.373 7f378dcd2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378818dc70 con 0x7f37880b23d0 2021-02-10T11:39:46.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.373 7f378dcd2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3788199e60 con 0x7f37880b1fd0 2021-02-10T11:39:46.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.373 7f378dcd2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378809ab30 con 0x7f378818d870 2021-02-10T11:39:46.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.373 7f3787fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37880b23d0 0x7f378809a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:46.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.373 7f3787fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37880b23d0 0x7f378809a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55844/0 (socket says 172.21.2.101:55844) 2021-02-10T11:39:46.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.373 7f3787fff700 1 -- 172.21.2.101:0/2956391446 learned_addr learned my addr 172.21.2.101:0/2956391446 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:46.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.373 7f3786ffd700 1 --2- 172.21.2.101:0/2956391446 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37880b1fd0 0x7f3788099ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:46.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.373 7f37877fe700 1 --2- 172.21.2.101:0/2956391446 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f378818d870 0x7f37880b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:46.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.373 7f3787fff700 1 -- 172.21.2.101:0/2956391446 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f378818d870 msgr2=0x7f37880b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:46.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.373 7f3787fff700 1 --2- 172.21.2.101:0/2956391446 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f378818d870 0x7f37880b19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:46.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.373 7f3787fff700 1 -- 172.21.2.101:0/2956391446 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37880b1fd0 msgr2=0x7f3788099ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:46.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.373 7f3787fff700 1 --2- 172.21.2.101:0/2956391446 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37880b1fd0 0x7f3788099ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:46.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.373 7f3787fff700 1 -- 172.21.2.101:0/2956391446 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378809ad50 con 0x7f37880b23d0 2021-02-10T11:39:46.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.373 7f37877fe700 1 --2- 172.21.2.101:0/2956391446 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f378818d870 0x7f37880b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:39:46.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.374 7f3787fff700 1 --2- 172.21.2.101:0/2956391446 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37880b23d0 0x7f378809a510 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 rx=0x7f3780009940 tx=0x7f378000bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:46.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.374 7f377ffff700 1 -- 172.21.2.101:0/2956391446 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3780000f40 con 0x7f37880b23d0 2021-02-10T11:39:46.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.374 7f377ffff700 1 -- 172.21.2.101:0/2956391446 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378000c2f0 con 0x7f37880b23d0 2021-02-10T11:39:46.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.374 7f377ffff700 1 -- 172.21.2.101:0/2956391446 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f378000c810 con 0x7f37880b23d0 2021-02-10T11:39:46.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.374 7f378dcd2700 1 -- 172.21.2.101:0/2956391446 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f378809ad50 con 0x7f37880b23d0 2021-02-10T11:39:46.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.374 7f378dcd2700 1 -- 172.21.2.101:0/2956391446 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f378818e3f0 con 0x7f37880b23d0 2021-02-10T11:39:46.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.375 7f377ffff700 1 -- 172.21.2.101:0/2956391446 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3780001250 con 0x7f37880b23d0 2021-02-10T11:39:46.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.375 7f377ffff700 1 --2- 172.21.2.101:0/2956391446 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f376c0186c0 0x7f376c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:46.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.375 7f377ffff700 1 -- 172.21.2.101:0/2956391446 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f3780032940 con 0x7f37880b23d0 2021-02-10T11:39:46.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.375 7f37877fe700 1 --2- 172.21.2.101:0/2956391446 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f376c0186c0 0x7f376c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:46.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.375 7f37877fe700 1 --2- 172.21.2.101:0/2956391446 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f376c0186c0 0x7f376c01abc0 crc :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:46.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.405 7f378dcd2700 1 -- 172.21.2.101:0/2956391446 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37880999c0 con 0x7f37880b23d0 2021-02-10T11:39:46.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.407 7f377ffff700 1 -- 172.21.2.101:0/2956391446 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3780003000 con 0x7f37880b23d0 2021-02-10T11:39:46.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.537 7f378dcd2700 1 -- 172.21.2.101:0/2956391446 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37880999c0 con 0x7f37880b23d0 2021-02-10T11:39:46.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.537 7f377ffff700 1 -- 172.21.2.101:0/2956391446 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+1756 (secure 0 0 0) 0x7f3780001230 con 0x7f37880b23d0 2021-02-10T11:39:46.541 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:39:46.541 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":73,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/4210469578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":4210469578},{"type":"v1","addr":"172.21.2.108:6833","nonce":4210469578}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":67}],"filesystems":[{"mdsmap":{"epoch":73,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:39:02.063078","modified":"2021-02-10 11:39:45.890477","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-02-10T11:39:46.541 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 73 2021-02-10T11:39:46.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.539 7f378dcd2700 1 -- 172.21.2.101:0/2956391446 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f376c0186c0 msgr2=0x7f376c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:46.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.539 7f378dcd2700 1 --2- 172.21.2.101:0/2956391446 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f376c0186c0 0x7f376c01abc0 crc :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:46.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.539 7f378dcd2700 1 -- 172.21.2.101:0/2956391446 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37880b23d0 msgr2=0x7f378809a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:46.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.539 7f378dcd2700 1 --2- 172.21.2.101:0/2956391446 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37880b23d0 0x7f378809a510 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 rx=0x7f3780009940 tx=0x7f378000bcf0).stop 2021-02-10T11:39:46.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.539 7f378dcd2700 1 -- 172.21.2.101:0/2956391446 shutdown_connections 2021-02-10T11:39:46.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.539 7f378dcd2700 1 --2- 172.21.2.101:0/2956391446 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37880b23d0 0x7f378809a510 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:46.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.539 7f378dcd2700 1 --2- 172.21.2.101:0/2956391446 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37880b1fd0 0x7f3788099ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:46.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.539 7f378dcd2700 1 --2- 172.21.2.101:0/2956391446 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f376c0186c0 0x7f376c01abc0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:46.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.539 7f378dcd2700 1 --2- 172.21.2.101:0/2956391446 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f378818d870 0x7f37880b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:46.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.539 7f378dcd2700 1 -- 172.21.2.101:0/2956391446 >> 172.21.2.101:0/2956391446 conn(0x7f37880f0550 msgr2=0x7f3788191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:46.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.539 7f378dcd2700 1 -- 172.21.2.101:0/2956391446 shutdown_connections 2021-02-10T11:39:46.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:46.540 7f378dcd2700 1 -- 172.21.2.101:0/2956391446 wait complete. 2021-02-10T11:39:46.551 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-02-10T11:39:47.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.003 7f4561cd2700 1 Processor -- start 2021-02-10T11:39:47.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.003 7f4561cd2700 1 -- start start 2021-02-10T11:39:47.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.003 7f4561cd2700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f455c0d1870 0x7f455c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:47.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.003 7f4561cd2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f455c0f0650 0x7f455c0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:47.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.003 7f4561cd2700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f455c0f0b10 con 0x7f455c1b0a10 2021-02-10T11:39:47.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.003 7f4561cd2700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f455c0e5ed0 con 0x7f455c0d1870 2021-02-10T11:39:47.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.003 7f4561cd2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455c0d1c70 con 0x7f455c0f0650 2021-02-10T11:39:47.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.004 7f4552ffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f455c1b0a10 0x7f455c0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56346/0 (socket says 172.21.2.101:56346) 2021-02-10T11:39:47.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.004 7f4552ffd700 1 -- 172.21.2.101:0/2710218226 learned_addr learned my addr 172.21.2.101:0/2710218226 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:47.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.004 7f455bfff700 1 --2- 172.21.2.101:0/2710218226 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f455c0f0650 0x7f455c0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:47.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.004 7f455b7fe700 1 --2- 172.21.2.101:0/2710218226 >> v2:172.21.2.108:3300/0 conn(0x7f455c0d1870 0x7f455c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:47.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.004 7f455bfff700 1 -- 172.21.2.101:0/2710218226 >> v2:172.21.2.108:3300/0 conn(0x7f455c0d1870 msgr2=0x7f455c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:47.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.004 7f455bfff700 1 --2- 172.21.2.101:0/2710218226 >> v2:172.21.2.108:3300/0 conn(0x7f455c0d1870 0x7f455c1b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:47.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.004 7f455bfff700 1 -- 172.21.2.101:0/2710218226 >> v1:172.21.2.101:6789/0 conn(0x7f455c1b0a10 legacy=0x7f455c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:47.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.004 7f455bfff700 1 -- 172.21.2.101:0/2710218226 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c0d1e90 con 0x7f455c0f0650 2021-02-10T11:39:47.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.004 7f455bfff700 1 --2- 172.21.2.101:0/2710218226 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f455c0f0650 0x7f455c0cfb20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f4554006e50 tx=0x7f4554009e70).ready entity=mon.2 client_cookie=587ca49adc44472b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:47.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.004 7f455affd700 1 -- 172.21.2.101:0/2710218226 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4554006a20 con 0x7f455c0f0650 2021-02-10T11:39:47.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.004 7f455affd700 1 -- 172.21.2.101:0/2710218226 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4554006790 con 0x7f455c0f0650 2021-02-10T11:39:47.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.004 7f455affd700 1 -- 172.21.2.101:0/2710218226 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f455400b700 con 0x7f455c0f0650 2021-02-10T11:39:47.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.005 7f4561cd2700 1 -- 172.21.2.101:0/2710218226 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f455c0f0650 msgr2=0x7f455c0cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:47.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.005 7f4561cd2700 1 --2- 172.21.2.101:0/2710218226 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f455c0f0650 0x7f455c0cfb20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f4554006e50 tx=0x7f4554009e70).stop 2021-02-10T11:39:47.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.005 7f4561cd2700 1 -- 172.21.2.101:0/2710218226 shutdown_connections 2021-02-10T11:39:47.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.005 7f4561cd2700 1 --2- 172.21.2.101:0/2710218226 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f455c0f0650 0x7f455c0cfb20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:47.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.005 7f4561cd2700 1 --2- 172.21.2.101:0/2710218226 >> v2:172.21.2.108:3300/0 conn(0x7f455c0d1870 0x7f455c1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:47.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.005 7f4561cd2700 1 -- 172.21.2.101:0/2710218226 >> 172.21.2.101:0/2710218226 conn(0x7f455c0ce450 msgr2=0x7f455c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:47.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.005 7f4561cd2700 1 -- 172.21.2.101:0/2710218226 shutdown_connections 2021-02-10T11:39:47.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.005 7f4561cd2700 1 -- 172.21.2.101:0/2710218226 wait complete. 2021-02-10T11:39:47.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.020 7f4561cd2700 1 Processor -- start 2021-02-10T11:39:47.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.020 7f4561cd2700 1 -- start start 2021-02-10T11:39:47.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.021 7f4561cd2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f455c0ebe70 0x7f455c0ec270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:47.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.021 7f4561cd2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f455c093830 0x7f455c093c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:47.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.021 7f4561cd2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f455c094250 0x7f455c094650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:47.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.021 7f4561cd2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c0ec940 con 0x7f455c093830 2021-02-10T11:39:47.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.021 7f4561cd2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c0ece40 con 0x7f455c0ebe70 2021-02-10T11:39:47.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.021 7f4561cd2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455c0ed060 con 0x7f455c094250 2021-02-10T11:39:47.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.021 7f4552ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f455c093830 0x7f455c093c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:47.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.021 7f4552ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f455c093830 0x7f455c093c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55854/0 (socket says 172.21.2.101:55854) 2021-02-10T11:39:47.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.021 7f4552ffd700 1 -- 172.21.2.101:0/407357226 learned_addr learned my addr 172.21.2.101:0/407357226 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:47.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.021 7f455bfff700 1 --2- 172.21.2.101:0/407357226 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f455c094250 0x7f455c094650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:47.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.021 7f455b7fe700 1 --2- 172.21.2.101:0/407357226 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f455c0ebe70 0x7f455c0ec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:47.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.021 7f4552ffd700 1 -- 172.21.2.101:0/407357226 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f455c094250 msgr2=0x7f455c094650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:47.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.021 7f4552ffd700 1 --2- 172.21.2.101:0/407357226 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f455c094250 0x7f455c094650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:47.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.021 7f4552ffd700 1 -- 172.21.2.101:0/407357226 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f455c0ebe70 msgr2=0x7f455c0ec270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:47.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.021 7f4552ffd700 1 --2- 172.21.2.101:0/407357226 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f455c0ebe70 0x7f455c0ec270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:47.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.021 7f4552ffd700 1 -- 172.21.2.101:0/407357226 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c0ed280 con 0x7f455c093830 2021-02-10T11:39:47.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.021 7f4552ffd700 1 --2- 172.21.2.101:0/407357226 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f455c093830 0x7f455c093c30 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 rx=0x7f454400a4c0 tx=0x7f454400a860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:47.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.022 7f4558ff9700 1 -- 172.21.2.101:0/407357226 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f454400ac50 con 0x7f455c093830 2021-02-10T11:39:47.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.022 7f4558ff9700 1 -- 172.21.2.101:0/407357226 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4544004140 con 0x7f455c093830 2021-02-10T11:39:47.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.022 7f4558ff9700 1 -- 172.21.2.101:0/407357226 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f45440045a0 con 0x7f455c093830 2021-02-10T11:39:47.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.022 7f4561cd2700 1 -- 172.21.2.101:0/407357226 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f455c0ed280 con 0x7f455c093830 2021-02-10T11:39:47.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.022 7f4561cd2700 1 -- 172.21.2.101:0/407357226 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455c0ed810 con 0x7f455c093830 2021-02-10T11:39:47.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.022 7f4558ff9700 1 -- 172.21.2.101:0/407357226 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4544004320 con 0x7f455c093830 2021-02-10T11:39:47.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.022 7f4558ff9700 1 --2- 172.21.2.101:0/407357226 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45400186c0 0x7f454001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:47.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.022 7f4558ff9700 1 -- 172.21.2.101:0/407357226 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f45440325b0 con 0x7f455c093830 2021-02-10T11:39:47.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.022 7f455b7fe700 1 --2- 172.21.2.101:0/407357226 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45400186c0 0x7f454001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:47.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.023 7f455b7fe700 1 --2- 172.21.2.101:0/407357226 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45400186c0 0x7f454001abc0 crc :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:47.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.053 7f4561cd2700 1 -- 172.21.2.101:0/407357226 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f455c0ebc40 con 0x7f455c093830 2021-02-10T11:39:47.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.056 7f4558ff9700 1 -- 172.21.2.101:0/407357226 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4544004ee0 con 0x7f455c093830 2021-02-10T11:39:47.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.184 7f4561cd2700 1 -- 172.21.2.101:0/407357226 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f455c0f0b10 con 0x7f455c093830 2021-02-10T11:39:47.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.906 7f4558ff9700 1 -- 172.21.2.101:0/407357226 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v74) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4544004320 con 0x7f455c093830 2021-02-10T11:39:47.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.956 7f4561cd2700 1 -- 172.21.2.101:0/407357226 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45400186c0 msgr2=0x7f454001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:47.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.956 7f4561cd2700 1 --2- 172.21.2.101:0/407357226 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45400186c0 0x7f454001abc0 crc :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:47.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.956 7f4561cd2700 1 -- 172.21.2.101:0/407357226 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f455c093830 msgr2=0x7f455c093c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:47.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.956 7f4561cd2700 1 --2- 172.21.2.101:0/407357226 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f455c093830 0x7f455c093c30 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 rx=0x7f454400a4c0 tx=0x7f454400a860).stop 2021-02-10T11:39:47.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.957 7f4561cd2700 1 -- 172.21.2.101:0/407357226 shutdown_connections 2021-02-10T11:39:47.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.957 7f4561cd2700 1 --2- 172.21.2.101:0/407357226 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45400186c0 0x7f454001abc0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:47.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.957 7f4561cd2700 1 --2- 172.21.2.101:0/407357226 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f455c094250 0x7f455c094650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:47.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.957 7f4561cd2700 1 --2- 172.21.2.101:0/407357226 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f455c093830 0x7f455c093c30 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:47.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.957 7f4561cd2700 1 --2- 172.21.2.101:0/407357226 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f455c0ebe70 0x7f455c0ec270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:47.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.957 7f4561cd2700 1 -- 172.21.2.101:0/407357226 >> 172.21.2.101:0/407357226 conn(0x7f455c08fa10 msgr2=0x7f455c0bfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:47.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.957 7f4561cd2700 1 -- 172.21.2.101:0/407357226 shutdown_connections 2021-02-10T11:39:47.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:47.957 7f4561cd2700 1 -- 172.21.2.101:0/407357226 wait complete. 2021-02-10T11:39:47.969 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-02-10T11:39:48.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.382 7fc0dedee700 1 Processor -- start 2021-02-10T11:39:48.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.382 7fc0dedee700 1 -- start start 2021-02-10T11:39:48.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.383 7fc0dedee700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0d80d1880 0x7fc0d81b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:48.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.383 7fc0dedee700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc0d80cff80 0x7fc0d8109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:48.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.383 7fc0dedee700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0d8109a70 con 0x7fc0d81b0af0 2021-02-10T11:39:48.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.383 7fc0dedee700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0d8109760 con 0x7fc0d80cff80 2021-02-10T11:39:48.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.383 7fc0dedee700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d8088070 con 0x7fc0d80d1880 2021-02-10T11:39:48.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.383 7fc0d7fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc0d81b0af0 0x7fc0d8108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56356/0 (socket says 172.21.2.101:56356) 2021-02-10T11:39:48.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.383 7fc0d7fff700 1 -- 172.21.2.101:0/75024334 learned_addr learned my addr 172.21.2.101:0/75024334 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:48.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.384 7fc0d77fe700 1 -- 172.21.2.101:0/75024334 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1014197131 0 0) 0x7fc0c8001180 con 0x7fc0d81b0af0 2021-02-10T11:39:48.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.384 7fc0d77fe700 1 -- 172.21.2.101:0/75024334 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0c4002fe0 con 0x7fc0d81b0af0 2021-02-10T11:39:48.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.384 7fc0dd40b700 1 --2- 172.21.2.101:0/75024334 >> v2:172.21.2.108:3300/0 conn(0x7fc0d80cff80 0x7fc0d8109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:48.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.384 7fc0dcc0a700 1 --2- 172.21.2.101:0/75024334 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0d80d1880 0x7fc0d81b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:48.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.384 7fc0d77fe700 1 -- 172.21.2.101:0/75024334 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 605264221 0 0) 0x7fc0c8001180 con 0x7fc0d81b0af0 2021-02-10T11:39:48.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.384 7fc0d77fe700 1 -- 172.21.2.101:0/75024334 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0d80d1880 msgr2=0x7fc0d81b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:48.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.384 7fc0d77fe700 1 --2- 172.21.2.101:0/75024334 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0d80d1880 0x7fc0d81b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:48.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.384 7fc0d77fe700 1 -- 172.21.2.101:0/75024334 >> v2:172.21.2.108:3300/0 conn(0x7fc0d80cff80 msgr2=0x7fc0d8109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:48.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.384 7fc0d77fe700 1 --2- 172.21.2.101:0/75024334 >> v2:172.21.2.108:3300/0 conn(0x7fc0d80cff80 0x7fc0d8109140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:48.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.384 7fc0d77fe700 1 -- 172.21.2.101:0/75024334 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d8088260 con 0x7fc0d81b0af0 2021-02-10T11:39:48.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.384 7fc0d77fe700 1 -- 172.21.2.101:0/75024334 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fc0c8001680 con 0x7fc0d81b0af0 2021-02-10T11:39:48.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.384 7fc0d77fe700 1 -- 172.21.2.101:0/75024334 >> v1:172.21.2.101:6789/0 conn(0x7fc0d81b0af0 legacy=0x7fc0d8108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:48.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.384 7fc0d77fe700 1 --2- 172.21.2.101:0/75024334 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0c4003a60 0x7fc0c4005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:48.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.384 7fc0d77fe700 1 --2- 172.21.2.101:0/75024334 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0c404a790 0x7fc0c404cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:48.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.384 7fc0d77fe700 1 --2- 172.21.2.101:0/75024334 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0c404d500 0x7fc0c404fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:48.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.384 7fc0d77fe700 1 -- 172.21.2.101:0/75024334 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c40502d0 con 0x7fc0c404d500 2021-02-10T11:39:48.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.384 7fc0d77fe700 1 -- 172.21.2.101:0/75024334 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c40504f0 con 0x7fc0c4003a60 2021-02-10T11:39:48.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.384 7fc0d77fe700 1 -- 172.21.2.101:0/75024334 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c4050710 con 0x7fc0c404a790 2021-02-10T11:39:48.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.384 7fc0dd40b700 1 --2- 172.21.2.101:0/75024334 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0c404d500 0x7fc0c404fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:48.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.384 7fc0dcc0a700 1 --2- 172.21.2.101:0/75024334 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0c4003a60 0x7fc0c4005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:48.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.385 7fc0d7fff700 1 --2- 172.21.2.101:0/75024334 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0c404a790 0x7fc0c404cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:48.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.385 7fc0dd40b700 1 -- 172.21.2.101:0/75024334 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0c404a790 msgr2=0x7fc0c404cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:48.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.385 7fc0dd40b700 1 --2- 172.21.2.101:0/75024334 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0c404a790 0x7fc0c404cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:48.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.385 7fc0dd40b700 1 -- 172.21.2.101:0/75024334 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0c4003a60 msgr2=0x7fc0c4005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:48.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.385 7fc0dd40b700 1 --2- 172.21.2.101:0/75024334 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0c4003a60 0x7fc0c4005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:48.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.385 7fc0dd40b700 1 -- 172.21.2.101:0/75024334 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c40509b0 con 0x7fc0c404d500 2021-02-10T11:39:48.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.385 7fc0dd40b700 1 --2- 172.21.2.101:0/75024334 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0c404d500 0x7fc0c404fa20 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 rx=0x7fc0d0006a40 tx=0x7fc0d000c0c0).ready entity=mon.0 client_cookie=a070fca3b1f36386 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:48.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.385 7fc0d77fe700 1 -- 172.21.2.101:0/75024334 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc0d000c4b0 con 0x7fc0c404d500 2021-02-10T11:39:48.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.385 7fc0d77fe700 1 -- 172.21.2.101:0/75024334 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d000c770 con 0x7fc0c404d500 2021-02-10T11:39:48.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.385 7fc0d77fe700 1 -- 172.21.2.101:0/75024334 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc0d000cc50 con 0x7fc0c404d500 2021-02-10T11:39:48.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.385 7fc0dedee700 1 -- 172.21.2.101:0/75024334 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0c404d500 msgr2=0x7fc0c404fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:48.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.385 7fc0dedee700 1 --2- 172.21.2.101:0/75024334 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0c404d500 0x7fc0c404fa20 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 rx=0x7fc0d0006a40 tx=0x7fc0d000c0c0).stop 2021-02-10T11:39:48.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.385 7fc0dd40b700 1 -- 172.21.2.101:0/75024334 reap_dead start 2021-02-10T11:39:48.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.386 7fc0dedee700 1 -- 172.21.2.101:0/75024334 shutdown_connections 2021-02-10T11:39:48.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.386 7fc0dedee700 1 -- 172.21.2.101:0/75024334 >> 172.21.2.101:0/75024334 conn(0x7fc0d80ce430 msgr2=0x7fc0d80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:48.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.386 7fc0dedee700 1 -- 172.21.2.101:0/75024334 shutdown_connections 2021-02-10T11:39:48.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.386 7fc0dedee700 1 -- 172.21.2.101:0/75024334 wait complete. 2021-02-10T11:39:48.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.401 7fc0dedee700 1 Processor -- start 2021-02-10T11:39:48.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.401 7fc0dedee700 1 -- start start 2021-02-10T11:39:48.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.401 7fc0dedee700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0d80a11f0 0x7fc0d818a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:48.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.401 7fc0dedee700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0d80a0670 0x7fc0d80a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:48.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.401 7fc0dedee700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0d80a16c0 0x7fc0d80a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:48.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.401 7fc0dedee700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d8189fb0 con 0x7fc0d80a0670 2021-02-10T11:39:48.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.401 7fc0dedee700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d818ab50 con 0x7fc0d80a11f0 2021-02-10T11:39:48.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.401 7fc0dedee700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d80ab4c0 con 0x7fc0d80a16c0 2021-02-10T11:39:48.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.401 7fc0d7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0d80a0670 0x7fc0d80a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:48.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.401 7fc0d7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0d80a0670 0x7fc0d80a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55876/0 (socket says 172.21.2.101:55876) 2021-02-10T11:39:48.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.401 7fc0d7fff700 1 -- 172.21.2.101:0/2666463023 learned_addr learned my addr 172.21.2.101:0/2666463023 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:48.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.402 7fc0dcc0a700 1 --2- 172.21.2.101:0/2666463023 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0d80a11f0 0x7fc0d818a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:48.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.402 7fc0dd40b700 1 --2- 172.21.2.101:0/2666463023 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0d80a16c0 0x7fc0d80a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:48.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.402 7fc0d7fff700 1 -- 172.21.2.101:0/2666463023 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0d80a16c0 msgr2=0x7fc0d80a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:48.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.402 7fc0d7fff700 1 --2- 172.21.2.101:0/2666463023 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0d80a16c0 0x7fc0d80a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:48.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.402 7fc0d7fff700 1 -- 172.21.2.101:0/2666463023 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0d80a11f0 msgr2=0x7fc0d818a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:48.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.402 7fc0d7fff700 1 --2- 172.21.2.101:0/2666463023 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0d80a11f0 0x7fc0d818a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:48.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.402 7fc0d7fff700 1 -- 172.21.2.101:0/2666463023 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d80ab6e0 con 0x7fc0d80a0670 2021-02-10T11:39:48.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.402 7fc0d7fff700 1 --2- 172.21.2.101:0/2666463023 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0d80a0670 0x7fc0d80a0a70 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 rx=0x7fc0c8001e30 tx=0x7fc0c800a8f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:48.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.402 7fc0d57fa700 1 -- 172.21.2.101:0/2666463023 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc0c800ace0 con 0x7fc0d80a0670 2021-02-10T11:39:48.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.402 7fc0d57fa700 1 -- 172.21.2.101:0/2666463023 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c8000c90 con 0x7fc0d80a0670 2021-02-10T11:39:48.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.402 7fc0d57fa700 1 -- 172.21.2.101:0/2666463023 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc0c8004590 con 0x7fc0d80a0670 2021-02-10T11:39:48.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.402 7fc0dedee700 1 -- 172.21.2.101:0/2666463023 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d80ab6e0 con 0x7fc0d80a0670 2021-02-10T11:39:48.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.402 7fc0dedee700 1 -- 172.21.2.101:0/2666463023 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d80abc70 con 0x7fc0d80a0670 2021-02-10T11:39:48.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.403 7fc0d57fa700 1 -- 172.21.2.101:0/2666463023 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc0c80045b0 con 0x7fc0d80a0670 2021-02-10T11:39:48.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.403 7fc0d57fa700 1 --2- 172.21.2.101:0/2666463023 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0bc0186c0 0x7fc0bc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:48.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.403 7fc0d57fa700 1 -- 172.21.2.101:0/2666463023 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fc0c80327b0 con 0x7fc0d80a0670 2021-02-10T11:39:48.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.403 7fc0dcc0a700 1 --2- 172.21.2.101:0/2666463023 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0bc0186c0 0x7fc0bc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:48.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.403 7fc0dcc0a700 1 --2- 172.21.2.101:0/2666463023 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0bc0186c0 0x7fc0bc01abc0 crc :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:48.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.432 7fc0dedee700 1 -- 172.21.2.101:0/2666463023 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d8189fb0 con 0x7fc0d80a0670 2021-02-10T11:39:48.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.435 7fc0d57fa700 1 -- 172.21.2.101:0/2666463023 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc0c80043a0 con 0x7fc0d80a0670 2021-02-10T11:39:48.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.563 7fc0dedee700 1 -- 172.21.2.101:0/2666463023 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc0d80abee0 con 0x7fc0d80a0670 2021-02-10T11:39:48.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.911 7fc0d57fa700 1 -- 172.21.2.101:0/2666463023 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) v1 ==== 185+0+0 (secure 0 0 0) 0x7fc0c80045b0 con 0x7fc0d80a0670 2021-02-10T11:39:48.931 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' removed 2021-02-10T11:39:48.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.930 7fc0dedee700 1 -- 172.21.2.101:0/2666463023 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0bc0186c0 msgr2=0x7fc0bc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:48.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.930 7fc0dedee700 1 --2- 172.21.2.101:0/2666463023 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0bc0186c0 0x7fc0bc01abc0 crc :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:48.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.930 7fc0dedee700 1 -- 172.21.2.101:0/2666463023 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0d80a0670 msgr2=0x7fc0d80a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:48.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.930 7fc0dedee700 1 --2- 172.21.2.101:0/2666463023 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0d80a0670 0x7fc0d80a0a70 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 rx=0x7fc0c8001e30 tx=0x7fc0c800a8f0).stop 2021-02-10T11:39:48.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.930 7fc0dedee700 1 -- 172.21.2.101:0/2666463023 shutdown_connections 2021-02-10T11:39:48.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.930 7fc0dedee700 1 --2- 172.21.2.101:0/2666463023 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0bc0186c0 0x7fc0bc01abc0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:48.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.930 7fc0dedee700 1 --2- 172.21.2.101:0/2666463023 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0d80a16c0 0x7fc0d80a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:48.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.930 7fc0dedee700 1 --2- 172.21.2.101:0/2666463023 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0d80a0670 0x7fc0d80a0a70 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:48.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.930 7fc0dedee700 1 --2- 172.21.2.101:0/2666463023 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0d80a11f0 0x7fc0d818a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:48.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.930 7fc0dedee700 1 -- 172.21.2.101:0/2666463023 >> 172.21.2.101:0/2666463023 conn(0x7fc0d809ebd0 msgr2=0x7fc0d8082c80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:48.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.930 7fc0dedee700 1 -- 172.21.2.101:0/2666463023 shutdown_connections 2021-02-10T11:39:48.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:48.931 7fc0dedee700 1 -- 172.21.2.101:0/2666463023 wait complete. 2021-02-10T11:39:48.942 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-02-10T11:39:49.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.354 7f6430f2c700 1 Processor -- start 2021-02-10T11:39:49.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.355 7f6430f2c700 1 -- start start 2021-02-10T11:39:49.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.355 7f6430f2c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f642c0d1880 0x7f642c1b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:49.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.355 7f6430f2c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f642c0fe1d0 con 0x7f642c1b0af0 2021-02-10T11:39:49.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.355 7f6430f2c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f642c109a70 con 0x7f642c0cff80 2021-02-10T11:39:49.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.355 7f6430f2c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f642c088070 con 0x7f642c0d1880 2021-02-10T11:39:49.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.355 7f642a312700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f642c1b0af0 0x7f642c108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56376/0 (socket says 172.21.2.101:56376) 2021-02-10T11:39:49.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.355 7f642a312700 1 -- 172.21.2.101:0/4019348746 learned_addr learned my addr 172.21.2.101:0/4019348746 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:49.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.355 7f642ab13700 1 --2- 172.21.2.101:0/4019348746 >> v2:172.21.2.108:3300/0 conn(0x7f642c0d1880 0x7f642c1b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:49.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.356 7f642ab13700 1 -- 172.21.2.101:0/4019348746 >> v1:172.21.2.108:6789/0 conn(0x7f642c0cff80 legacy=0x7f642c109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:49.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.356 7f642ab13700 1 -- 172.21.2.101:0/4019348746 >> v1:172.21.2.101:6789/0 conn(0x7f642c1b0af0 legacy=0x7f642c108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:49.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.356 7f642ab13700 1 -- 172.21.2.101:0/4019348746 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642c088260 con 0x7f642c0d1880 2021-02-10T11:39:49.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.356 7f6429b11700 1 -- 172.21.2.101:0/4019348746 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 347249199 0 0) 0x7f641c0009b0 con 0x7f642c1b0af0 2021-02-10T11:39:49.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.356 7f642ab13700 1 --2- 172.21.2.101:0/4019348746 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f642c0d1880 0x7f642c1b04d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f6414006650 tx=0x7f6414009a80).ready entity=mon.1 client_cookie=8f1808aba964a265 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:49.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.356 7f6429b11700 1 -- 172.21.2.101:0/4019348746 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f64140009b0 con 0x7f642c0d1880 2021-02-10T11:39:49.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.356 7f6429b11700 1 -- 172.21.2.101:0/4019348746 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641400c0c0 con 0x7f642c0d1880 2021-02-10T11:39:49.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.356 7f6429b11700 1 -- 172.21.2.101:0/4019348746 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f641400c5e0 con 0x7f642c0d1880 2021-02-10T11:39:49.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.356 7f6430f2c700 1 -- 172.21.2.101:0/4019348746 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f642c0d1880 msgr2=0x7f642c1b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:49.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.356 7f6430f2c700 1 --2- 172.21.2.101:0/4019348746 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f642c0d1880 0x7f642c1b04d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f6414006650 tx=0x7f6414009a80).stop 2021-02-10T11:39:49.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.357 7f6430f2c700 1 -- 172.21.2.101:0/4019348746 shutdown_connections 2021-02-10T11:39:49.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.357 7f6430f2c700 1 --2- 172.21.2.101:0/4019348746 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f642c0d1880 0x7f642c1b04d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:49.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.357 7f6430f2c700 1 -- 172.21.2.101:0/4019348746 >> 172.21.2.101:0/4019348746 conn(0x7f642c0ce430 msgr2=0x7f642c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:49.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.357 7f6430f2c700 1 -- 172.21.2.101:0/4019348746 shutdown_connections 2021-02-10T11:39:49.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.357 7f6430f2c700 1 -- 172.21.2.101:0/4019348746 wait complete. 2021-02-10T11:39:49.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.372 7f6430f2c700 1 Processor -- start 2021-02-10T11:39:49.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.372 7f6430f2c700 1 -- start start 2021-02-10T11:39:49.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.372 7f6430f2c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f642c08fb40 0x7f642c195720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:49.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.372 7f6430f2c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f642c09b4f0 0x7f642c09b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:49.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.372 7f6430f2c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f642c09bf10 0x7f642c1940a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:49.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.372 7f6430f2c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c1951a0 con 0x7f642c09bf10 2021-02-10T11:39:49.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.372 7f6430f2c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c094990 con 0x7f642c08fb40 2021-02-10T11:39:49.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.372 7f6430f2c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642c1946c0 con 0x7f642c09b4f0 2021-02-10T11:39:49.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.372 7f642b314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f642c09bf10 0x7f642c1940a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:49.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.372 7f642b314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f642c09bf10 0x7f642c1940a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55888/0 (socket says 172.21.2.101:55888) 2021-02-10T11:39:49.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.373 7f642b314700 1 -- 172.21.2.101:0/457653426 learned_addr learned my addr 172.21.2.101:0/457653426 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:49.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.373 7f642ab13700 1 --2- 172.21.2.101:0/457653426 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f642c08fb40 0x7f642c195720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:49.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.373 7f642a312700 1 --2- 172.21.2.101:0/457653426 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f642c09b4f0 0x7f642c09b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:49.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.373 7f642b314700 1 -- 172.21.2.101:0/457653426 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f642c09b4f0 msgr2=0x7f642c09b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:49.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.373 7f642b314700 1 --2- 172.21.2.101:0/457653426 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f642c09b4f0 0x7f642c09b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:49.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.373 7f642b314700 1 -- 172.21.2.101:0/457653426 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f642c08fb40 msgr2=0x7f642c195720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:49.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.373 7f642b314700 1 --2- 172.21.2.101:0/457653426 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f642c08fb40 0x7f642c195720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:49.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.373 7f642b314700 1 -- 172.21.2.101:0/457653426 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642c1948e0 con 0x7f642c09bf10 2021-02-10T11:39:49.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.373 7f642b314700 1 --2- 172.21.2.101:0/457653426 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f642c09bf10 0x7f642c1940a0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 rx=0x7f6424009940 tx=0x7f642400bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:49.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.373 7f64237fe700 1 -- 172.21.2.101:0/457653426 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6424000ce0 con 0x7f642c09bf10 2021-02-10T11:39:49.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.373 7f64237fe700 1 -- 172.21.2.101:0/457653426 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642400c2f0 con 0x7f642c09bf10 2021-02-10T11:39:49.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.373 7f64237fe700 1 -- 172.21.2.101:0/457653426 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f642400c810 con 0x7f642c09bf10 2021-02-10T11:39:49.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.373 7f6430f2c700 1 -- 172.21.2.101:0/457653426 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f642c1948e0 con 0x7f642c09bf10 2021-02-10T11:39:49.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.373 7f6430f2c700 1 -- 172.21.2.101:0/457653426 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f642c194e70 con 0x7f642c09bf10 2021-02-10T11:39:49.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.374 7f64237fe700 1 -- 172.21.2.101:0/457653426 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6424001220 con 0x7f642c09bf10 2021-02-10T11:39:49.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.374 7f64237fe700 1 --2- 172.21.2.101:0/457653426 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f64100186c0 0x7f641001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:49.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.374 7f64237fe700 1 -- 172.21.2.101:0/457653426 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f64240329c0 con 0x7f642c09bf10 2021-02-10T11:39:49.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.374 7f642ab13700 1 --2- 172.21.2.101:0/457653426 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f64100186c0 0x7f641001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:49.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.374 7f642ab13700 1 --2- 172.21.2.101:0/457653426 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f64100186c0 0x7f641001abc0 crc :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:49.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.405 7f6430f2c700 1 -- 172.21.2.101:0/457653426 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f642c082ab0 con 0x7f642c09bf10 2021-02-10T11:39:49.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.407 7f64237fe700 1 -- 172.21.2.101:0/457653426 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6424000ce0 con 0x7f642c09bf10 2021-02-10T11:39:49.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.533 7f6430f2c700 1 -- 172.21.2.101:0/457653426 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f642c082ab0 con 0x7f642c09bf10 2021-02-10T11:39:49.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.913 7f64237fe700 1 -- 172.21.2.101:0/457653426 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) v1 ==== 173+0+0 (secure 0 0 0) 0x7f642401f860 con 0x7f642c09bf10 2021-02-10T11:39:49.953 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' removed 2021-02-10T11:39:49.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.951 7f6430f2c700 1 -- 172.21.2.101:0/457653426 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f64100186c0 msgr2=0x7f641001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:49.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.951 7f6430f2c700 1 --2- 172.21.2.101:0/457653426 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f64100186c0 0x7f641001abc0 crc :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:49.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.951 7f6430f2c700 1 -- 172.21.2.101:0/457653426 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f642c09bf10 msgr2=0x7f642c1940a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:49.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.951 7f6430f2c700 1 --2- 172.21.2.101:0/457653426 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f642c09bf10 0x7f642c1940a0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 rx=0x7f6424009940 tx=0x7f642400bcf0).stop 2021-02-10T11:39:49.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.951 7f6430f2c700 1 -- 172.21.2.101:0/457653426 shutdown_connections 2021-02-10T11:39:49.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.951 7f6430f2c700 1 --2- 172.21.2.101:0/457653426 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f64100186c0 0x7f641001abc0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:49.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.951 7f6430f2c700 1 --2- 172.21.2.101:0/457653426 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f642c09b4f0 0x7f642c09b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:49.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.951 7f6430f2c700 1 --2- 172.21.2.101:0/457653426 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f642c09bf10 0x7f642c1940a0 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:49.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.951 7f6430f2c700 1 --2- 172.21.2.101:0/457653426 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f642c08fb40 0x7f642c195720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:49.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.951 7f6430f2c700 1 -- 172.21.2.101:0/457653426 >> 172.21.2.101:0/457653426 conn(0x7f642c09afc0 msgr2=0x7f642c0953e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:49.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.951 7f6430f2c700 1 -- 172.21.2.101:0/457653426 shutdown_connections 2021-02-10T11:39:49.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:49.951 7f6430f2c700 1 -- 172.21.2.101:0/457653426 wait complete. 2021-02-10T11:39:49.968 INFO:tasks.ceph.mds.a:Restarting daemon 2021-02-10T11:39:49.968 INFO:tasks.ceph.mds.a:Stopping old one... 2021-02-10T11:39:49.968 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-02-10T11:39:49.968 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:39:49.968 INFO:tasks.ceph.mds.b:Restarting daemon 2021-02-10T11:39:49.969 INFO:tasks.ceph.mds.b:Stopping old one... 2021-02-10T11:39:49.969 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-02-10T11:39:49.969 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:39:49.974 INFO:tasks.ceph.mds.a:Stopped 2021-02-10T11:39:49.974 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-02-10T11:39:49.976 INFO:tasks.ceph.mds.b:Stopped 2021-02-10T11:39:49.976 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-02-10T11:39:50.016 INFO:tasks.ceph.mds.a:Started 2021-02-10T11:39:50.017 DEBUG:teuthology.parallel:result is None 2021-02-10T11:39:50.017 INFO:tasks.ceph.mds.b:Started 2021-02-10T11:39:50.017 DEBUG:teuthology.parallel:result is None 2021-02-10T11:39:50.018 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd blacklist clear 2021-02-10T11:39:50.421 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:39:50.426 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:39:50.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.425 7f6b7cf38700 1 Processor -- start 2021-02-10T11:39:50.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.425 7f6b7cf38700 1 -- start start 2021-02-10T11:39:50.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.425 7f6b7cf38700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6b780d0340 0x7f6b780ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:50.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.425 7f6b7cf38700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6b780d1c70 0x7f6b780efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:50.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.425 7f6b7cf38700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b780f0b10 con 0x7f6b780d1870 2021-02-10T11:39:50.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.425 7f6b7cf38700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b780e5ed0 con 0x7f6b780d0340 2021-02-10T11:39:50.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.425 7f6b7cf38700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b781b0be0 con 0x7f6b780d1c70 2021-02-10T11:39:50.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.425 7f6b76b13700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f6b780d1870 0x7f6b781b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56400/0 (socket says 172.21.2.101:56400) 2021-02-10T11:39:50.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.425 7f6b76b13700 1 -- 172.21.2.101:0/216316150 learned_addr learned my addr 172.21.2.101:0/216316150 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:50.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.425 7f6b6e312700 1 --2- 172.21.2.101:0/216316150 >> v2:172.21.2.101:3300/0 conn(0x7f6b780d0340 0x7f6b780ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:50.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.425 7f6b77314700 1 --2- 172.21.2.101:0/216316150 >> v2:172.21.2.108:3300/0 conn(0x7f6b780d1c70 0x7f6b780efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:50.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.425 7f6b76312700 1 -- 172.21.2.101:0/216316150 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623441944 0 0) 0x7f6b68001060 con 0x7f6b780d1870 2021-02-10T11:39:50.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.425 7f6b76312700 1 -- 172.21.2.101:0/216316150 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b64002fe0 con 0x7f6b780d1870 2021-02-10T11:39:50.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.426 7f6b6e312700 1 -- 172.21.2.101:0/216316150 >> v2:172.21.2.108:3300/0 conn(0x7f6b780d1c70 msgr2=0x7f6b780efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:50.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.426 7f6b6e312700 1 --2- 172.21.2.101:0/216316150 >> v2:172.21.2.108:3300/0 conn(0x7f6b780d1c70 0x7f6b780efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:50.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.426 7f6b6e312700 1 -- 172.21.2.101:0/216316150 >> v1:172.21.2.101:6789/0 conn(0x7f6b780d1870 legacy=0x7f6b781b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:50.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.426 7f6b6e312700 1 -- 172.21.2.101:0/216316150 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b780f20a0 con 0x7f6b780d0340 2021-02-10T11:39:50.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.426 7f6b77314700 1 --2- 172.21.2.101:0/216316150 >> v2:172.21.2.108:3300/0 conn(0x7f6b780d1c70 0x7f6b780efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:39:50.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.426 7f6b6e312700 1 --2- 172.21.2.101:0/216316150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b780d0340 0x7f6b780ef640 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 rx=0x7f6b60006650 tx=0x7f6b60009a80).ready entity=mon.0 client_cookie=667b2ffba92b4290 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:50.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.426 7f6b76312700 1 -- 172.21.2.101:0/216316150 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6b600009b0 con 0x7f6b780d0340 2021-02-10T11:39:50.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.426 7f6b76312700 1 -- 172.21.2.101:0/216316150 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6000c0c0 con 0x7f6b780d0340 2021-02-10T11:39:50.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.426 7f6b76312700 1 -- 172.21.2.101:0/216316150 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6b6000c510 con 0x7f6b780d0340 2021-02-10T11:39:50.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.426 7f6b7cf38700 1 -- 172.21.2.101:0/216316150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b780d0340 msgr2=0x7f6b780ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:50.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.426 7f6b7cf38700 1 --2- 172.21.2.101:0/216316150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b780d0340 0x7f6b780ef640 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 rx=0x7f6b60006650 tx=0x7f6b60009a80).stop 2021-02-10T11:39:50.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.426 7f6b7cf38700 1 -- 172.21.2.101:0/216316150 shutdown_connections 2021-02-10T11:39:50.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.426 7f6b7cf38700 1 --2- 172.21.2.101:0/216316150 >> v2:172.21.2.108:3300/0 conn(0x7f6b780d1c70 0x7f6b780efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:50.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.426 7f6b7cf38700 1 --2- 172.21.2.101:0/216316150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b780d0340 0x7f6b780ef640 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:50.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.426 7f6b7cf38700 1 -- 172.21.2.101:0/216316150 >> 172.21.2.101:0/216316150 conn(0x7f6b780ce450 msgr2=0x7f6b780d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:50.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.426 7f6b7cf38700 1 -- 172.21.2.101:0/216316150 shutdown_connections 2021-02-10T11:39:50.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.426 7f6b7cf38700 1 -- 172.21.2.101:0/216316150 wait complete. 2021-02-10T11:39:50.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.441 7f6b7cf38700 1 Processor -- start 2021-02-10T11:39:50.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.441 7f6b7cf38700 1 -- start start 2021-02-10T11:39:50.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.441 7f6b7cf38700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b781a06a0 0x7f6b7809f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:50.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.441 7f6b7cf38700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b7809f9a0 0x7f6b7808b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:50.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.441 7f6b7cf38700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b7808bb80 0x7f6b7808bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:50.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.441 7f6b7cf38700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7808c780 con 0x7f6b7809f9a0 2021-02-10T11:39:50.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.441 7f6b7cf38700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7809fda0 con 0x7f6b781a06a0 2021-02-10T11:39:50.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.441 7f6b7cf38700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7808cd00 con 0x7f6b7808bb80 2021-02-10T11:39:50.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.442 7f6b6e312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b7809f9a0 0x7f6b7808b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:50.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.442 7f6b6e312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b7809f9a0 0x7f6b7808b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55912/0 (socket says 172.21.2.101:55912) 2021-02-10T11:39:50.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.442 7f6b6e312700 1 -- 172.21.2.101:0/3566295859 learned_addr learned my addr 172.21.2.101:0/3566295859 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:50.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.442 7f6b77314700 1 --2- 172.21.2.101:0/3566295859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b7808bb80 0x7f6b7808bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:50.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.442 7f6b76b13700 1 --2- 172.21.2.101:0/3566295859 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b781a06a0 0x7f6b7809f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:50.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.442 7f6b6e312700 1 -- 172.21.2.101:0/3566295859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b7808bb80 msgr2=0x7f6b7808bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:50.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.442 7f6b6e312700 1 --2- 172.21.2.101:0/3566295859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b7808bb80 0x7f6b7808bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:50.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.442 7f6b6e312700 1 -- 172.21.2.101:0/3566295859 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b781a06a0 msgr2=0x7f6b7809f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:50.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.442 7f6b6e312700 1 --2- 172.21.2.101:0/3566295859 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b781a06a0 0x7f6b7809f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:50.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.442 7f6b6e312700 1 -- 172.21.2.101:0/3566295859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7808cf20 con 0x7f6b7809f9a0 2021-02-10T11:39:50.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.442 7f6b6e312700 1 --2- 172.21.2.101:0/3566295859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b7809f9a0 0x7f6b7808b560 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 rx=0x7f6b60009650 tx=0x7f6b6000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:50.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.442 7f6b6ffff700 1 -- 172.21.2.101:0/3566295859 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6b6000bd50 con 0x7f6b7809f9a0 2021-02-10T11:39:50.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.442 7f6b6ffff700 1 -- 172.21.2.101:0/3566295859 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b60006030 con 0x7f6b7809f9a0 2021-02-10T11:39:50.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.442 7f6b6ffff700 1 -- 172.21.2.101:0/3566295859 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6b60005560 con 0x7f6b7809f9a0 2021-02-10T11:39:50.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.442 7f6b7cf38700 1 -- 172.21.2.101:0/3566295859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b7808cf20 con 0x7f6b7809f9a0 2021-02-10T11:39:50.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.442 7f6b7cf38700 1 -- 172.21.2.101:0/3566295859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b7808cf20 con 0x7f6b7809f9a0 2021-02-10T11:39:50.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.443 7f6b6ffff700 1 -- 172.21.2.101:0/3566295859 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6b600052e0 con 0x7f6b7809f9a0 2021-02-10T11:39:50.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.443 7f6b6ffff700 1 --2- 172.21.2.101:0/3566295859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b5c018730 0x7f6b5c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:50.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.443 7f6b6ffff700 1 -- 172.21.2.101:0/3566295859 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 4839+0+0 (secure 0 0 0) 0x7f6b60032360 con 0x7f6b7809f9a0 2021-02-10T11:39:50.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.443 7f6b76b13700 1 --2- 172.21.2.101:0/3566295859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b5c018730 0x7f6b5c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:50.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.443 7f6b76b13700 1 --2- 172.21.2.101:0/3566295859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b5c018730 0x7f6b5c01abc0 crc :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:50.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.474 7f6b7cf38700 1 -- 172.21.2.101:0/3566295859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b780ebc50 con 0x7f6b7809f9a0 2021-02-10T11:39:50.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.476 7f6b6ffff700 1 -- 172.21.2.101:0/3566295859 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6b600009c0 con 0x7f6b7809f9a0 2021-02-10T11:39:50.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.615 7f6b7cf38700 1 -- 172.21.2.101:0/3566295859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f6b780f0b10 con 0x7f6b7809f9a0 2021-02-10T11:39:50.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.957 7f6b6ffff700 1 -- 172.21.2.101:0/3566295859 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v98) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6b60006030 con 0x7f6b7809f9a0 2021-02-10T11:39:50.984 INFO:teuthology.orchestra.run.gibba001.stderr: removed all blacklist entries 2021-02-10T11:39:50.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.982 7f6b7cf38700 1 -- 172.21.2.101:0/3566295859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b5c018730 msgr2=0x7f6b5c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:50.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.982 7f6b7cf38700 1 --2- 172.21.2.101:0/3566295859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b5c018730 0x7f6b5c01abc0 crc :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:50.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.982 7f6b7cf38700 1 -- 172.21.2.101:0/3566295859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b7809f9a0 msgr2=0x7f6b7808b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:50.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.982 7f6b7cf38700 1 --2- 172.21.2.101:0/3566295859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b7809f9a0 0x7f6b7808b560 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 rx=0x7f6b60009650 tx=0x7f6b6000b960).stop 2021-02-10T11:39:50.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.982 7f6b7cf38700 1 -- 172.21.2.101:0/3566295859 shutdown_connections 2021-02-10T11:39:50.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.982 7f6b7cf38700 1 --2- 172.21.2.101:0/3566295859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b5c018730 0x7f6b5c01abc0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:50.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.982 7f6b7cf38700 1 --2- 172.21.2.101:0/3566295859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b7808bb80 0x7f6b7808bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:50.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.982 7f6b7cf38700 1 --2- 172.21.2.101:0/3566295859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b7809f9a0 0x7f6b7808b560 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:50.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.982 7f6b7cf38700 1 --2- 172.21.2.101:0/3566295859 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b781a06a0 0x7f6b7809f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:50.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.982 7f6b7cf38700 1 -- 172.21.2.101:0/3566295859 >> 172.21.2.101:0/3566295859 conn(0x7f6b780b83b0 msgr2=0x7f6b780a6850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:50.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.982 7f6b7cf38700 1 -- 172.21.2.101:0/3566295859 shutdown_connections 2021-02-10T11:39:50.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:50.982 7f6b7cf38700 1 -- 172.21.2.101:0/3566295859 wait complete. 2021-02-10T11:39:50.994 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2021-02-10T11:39:50.994 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:39:51.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.403 7f0b36371700 1 Processor -- start 2021-02-10T11:39:51.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.403 7f0b36371700 1 -- start start 2021-02-10T11:39:51.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.403 7f0b36371700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b300d1870 0x7f0b301b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:51.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.403 7f0b36371700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0b300f0650 0x7f0b300cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:51.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.403 7f0b36371700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b300f0b10 con 0x7f0b301b0a10 2021-02-10T11:39:51.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.403 7f0b36371700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b300fe230 con 0x7f0b300f0650 2021-02-10T11:39:51.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.403 7f0b36371700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b300e5ed0 con 0x7f0b300d1870 2021-02-10T11:39:51.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.404 7f0b2f7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f0b301b0a10 0x7f0b300cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56418/0 (socket says 172.21.2.101:56418) 2021-02-10T11:39:51.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.404 7f0b2f7fe700 1 -- 172.21.2.101:0/2388038856 learned_addr learned my addr 172.21.2.101:0/2388038856 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:51.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.404 7f0b2effd700 1 -- 172.21.2.101:0/2388038856 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1365760145 0 0) 0x7f0b240009b0 con 0x7f0b301b0a10 2021-02-10T11:39:51.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.404 7f0b2effd700 1 -- 172.21.2.101:0/2388038856 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b18002fe0 con 0x7f0b301b0a10 2021-02-10T11:39:51.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.404 7f0b3498e700 1 --2- 172.21.2.101:0/2388038856 >> v2:172.21.2.108:3300/0 conn(0x7f0b300f0650 0x7f0b300cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:51.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.404 7f0b2ffff700 1 --2- 172.21.2.101:0/2388038856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b300d1870 0x7f0b301b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:51.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.404 7f0b2effd700 1 -- 172.21.2.101:0/2388038856 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1357370943 0 0) 0x7f0b24001010 con 0x7f0b301b0a10 2021-02-10T11:39:51.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.404 7f0b2effd700 1 -- 172.21.2.101:0/2388038856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b300d1870 msgr2=0x7f0b301b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:51.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.404 7f0b2effd700 1 --2- 172.21.2.101:0/2388038856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b300d1870 0x7f0b301b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:51.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.404 7f0b2effd700 1 -- 172.21.2.101:0/2388038856 >> v2:172.21.2.108:3300/0 conn(0x7f0b300f0650 msgr2=0x7f0b300cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:51.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.404 7f0b2effd700 1 --2- 172.21.2.101:0/2388038856 >> v2:172.21.2.108:3300/0 conn(0x7f0b300f0650 0x7f0b300cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:51.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.404 7f0b2effd700 1 -- 172.21.2.101:0/2388038856 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b300f20a0 con 0x7f0b301b0a10 2021-02-10T11:39:51.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.404 7f0b2effd700 1 -- 172.21.2.101:0/2388038856 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f0b24001530 con 0x7f0b301b0a10 2021-02-10T11:39:51.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.404 7f0b2effd700 1 -- 172.21.2.101:0/2388038856 >> v1:172.21.2.101:6789/0 conn(0x7f0b301b0a10 legacy=0x7f0b300cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:51.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.404 7f0b2effd700 1 --2- 172.21.2.101:0/2388038856 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b18003af0 0x7f0b18006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:51.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.404 7f0b2effd700 1 --2- 172.21.2.101:0/2388038856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b180068b0 0x7f0b18008da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:51.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.404 7f0b2effd700 1 --2- 172.21.2.101:0/2388038856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b18009640 0x7f0b1800bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:51.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.404 7f0b2effd700 1 -- 172.21.2.101:0/2388038856 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1800c390 con 0x7f0b18009640 2021-02-10T11:39:51.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.404 7f0b2effd700 1 -- 172.21.2.101:0/2388038856 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b18050580 con 0x7f0b18003af0 2021-02-10T11:39:51.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.404 7f0b2effd700 1 -- 172.21.2.101:0/2388038856 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b180507a0 con 0x7f0b180068b0 2021-02-10T11:39:51.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.404 7f0b3498e700 1 --2- 172.21.2.101:0/2388038856 >> v2:172.21.2.108:3300/0 conn(0x7f0b300f0650 0x7f0b300cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:39:51.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.405 7f0b2ffff700 1 --2- 172.21.2.101:0/2388038856 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b18003af0 0x7f0b18006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:51.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.405 7f0b3498e700 1 --2- 172.21.2.101:0/2388038856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b18009640 0x7f0b1800bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:51.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.405 7f0b2f7fe700 1 --2- 172.21.2.101:0/2388038856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b180068b0 0x7f0b18008da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:51.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.405 7f0b3498e700 1 -- 172.21.2.101:0/2388038856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b180068b0 msgr2=0x7f0b18008da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:51.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.405 7f0b3498e700 1 --2- 172.21.2.101:0/2388038856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b180068b0 0x7f0b18008da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:51.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.405 7f0b3498e700 1 -- 172.21.2.101:0/2388038856 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b18003af0 msgr2=0x7f0b18006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:51.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.405 7f0b3498e700 1 --2- 172.21.2.101:0/2388038856 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b18003af0 0x7f0b18006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:51.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.405 7f0b3498e700 1 -- 172.21.2.101:0/2388038856 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b18050a40 con 0x7f0b18009640 2021-02-10T11:39:51.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.405 7f0b3498e700 1 --2- 172.21.2.101:0/2388038856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b18009640 0x7f0b1800bb60 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 rx=0x7f0b280068c0 tx=0x7f0b2800b300).ready entity=mon.0 client_cookie=5cbec2d06cd66780 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:51.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.405 7f0b2effd700 1 -- 172.21.2.101:0/2388038856 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0b28000df0 con 0x7f0b18009640 2021-02-10T11:39:51.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.405 7f0b2effd700 1 -- 172.21.2.101:0/2388038856 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b28003030 con 0x7f0b18009640 2021-02-10T11:39:51.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.405 7f0b2effd700 1 -- 172.21.2.101:0/2388038856 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0b2800b8e0 con 0x7f0b18009640 2021-02-10T11:39:51.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.405 7f0b36371700 1 -- 172.21.2.101:0/2388038856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b18009640 msgr2=0x7f0b1800bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:51.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.405 7f0b36371700 1 --2- 172.21.2.101:0/2388038856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b18009640 0x7f0b1800bb60 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 rx=0x7f0b280068c0 tx=0x7f0b2800b300).stop 2021-02-10T11:39:51.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.405 7f0b3498e700 1 -- 172.21.2.101:0/2388038856 reap_dead start 2021-02-10T11:39:51.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.406 7f0b36371700 1 -- 172.21.2.101:0/2388038856 shutdown_connections 2021-02-10T11:39:51.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.406 7f0b36371700 1 -- 172.21.2.101:0/2388038856 >> 172.21.2.101:0/2388038856 conn(0x7f0b300ce450 msgr2=0x7f0b300d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:51.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.406 7f0b36371700 1 -- 172.21.2.101:0/2388038856 shutdown_connections 2021-02-10T11:39:51.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.406 7f0b36371700 1 -- 172.21.2.101:0/2388038856 wait complete. 2021-02-10T11:39:51.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.421 7f0b36371700 1 Processor -- start 2021-02-10T11:39:51.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.421 7f0b36371700 1 -- start start 2021-02-10T11:39:51.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.421 7f0b36371700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b3009bb70 0x7f0b300b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:51.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.421 7f0b36371700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b300b55e0 0x7f0b300c1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:51.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.421 7f0b36371700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b300b59e0 0x7f0b300c1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:51.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.421 7f0b36371700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b300b6010 con 0x7f0b300b59e0 2021-02-10T11:39:51.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.421 7f0b36371700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b300b6230 con 0x7f0b300b55e0 2021-02-10T11:39:51.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.421 7f0b36371700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3009bf70 con 0x7f0b3009bb70 2021-02-10T11:39:51.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.422 7f0b3498e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b300b59e0 0x7f0b300c1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:51.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.422 7f0b3498e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b300b59e0 0x7f0b300c1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55936/0 (socket says 172.21.2.101:55936) 2021-02-10T11:39:51.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.422 7f0b3498e700 1 -- 172.21.2.101:0/1330927029 learned_addr learned my addr 172.21.2.101:0/1330927029 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:51.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.422 7f0b3498e700 1 -- 172.21.2.101:0/1330927029 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b3009bb70 msgr2=0x7f0b300b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:51.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.422 7f0b3498e700 1 --2- 172.21.2.101:0/1330927029 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b3009bb70 0x7f0b300b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:51.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.422 7f0b3498e700 1 -- 172.21.2.101:0/1330927029 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b300b55e0 msgr2=0x7f0b300c1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:51.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.422 7f0b3498e700 1 --2- 172.21.2.101:0/1330927029 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b300b55e0 0x7f0b300c1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:51.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.422 7f0b3498e700 1 -- 172.21.2.101:0/1330927029 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b300c1fb0 con 0x7f0b300b59e0 2021-02-10T11:39:51.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.422 7f0b3498e700 1 --2- 172.21.2.101:0/1330927029 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b300b59e0 0x7f0b300c1990 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 rx=0x7f0b28008a20 tx=0x7f0b28008dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:51.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.422 7f0b2cff9700 1 -- 172.21.2.101:0/1330927029 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0b28009440 con 0x7f0b300b59e0 2021-02-10T11:39:51.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.422 7f0b2cff9700 1 -- 172.21.2.101:0/1330927029 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2800c400 con 0x7f0b300b59e0 2021-02-10T11:39:51.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.422 7f0b2cff9700 1 -- 172.21.2.101:0/1330927029 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0b2800c890 con 0x7f0b300b59e0 2021-02-10T11:39:51.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.422 7f0b36371700 1 -- 172.21.2.101:0/1330927029 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b300c1fb0 con 0x7f0b300b59e0 2021-02-10T11:39:51.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.422 7f0b36371700 1 -- 172.21.2.101:0/1330927029 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b300c2540 con 0x7f0b300b59e0 2021-02-10T11:39:51.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.423 7f0b2cff9700 1 -- 172.21.2.101:0/1330927029 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0b28009440 con 0x7f0b300b59e0 2021-02-10T11:39:51.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.423 7f0b2cff9700 1 --2- 172.21.2.101:0/1330927029 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b100186c0 0x7f0b1001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:51.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.423 7f0b2cff9700 1 -- 172.21.2.101:0/1330927029 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f0b280325e0 con 0x7f0b300b59e0 2021-02-10T11:39:51.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.423 7f0b2ffff700 1 --2- 172.21.2.101:0/1330927029 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b100186c0 0x7f0b1001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:51.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.423 7f0b2ffff700 1 --2- 172.21.2.101:0/1330927029 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b100186c0 0x7f0b1001abc0 crc :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:51.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.453 7f0b36371700 1 -- 172.21.2.101:0/1330927029 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b300c0e40 con 0x7f0b300b59e0 2021-02-10T11:39:51.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.455 7f0b2cff9700 1 -- 172.21.2.101:0/1330927029 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0b2802df80 con 0x7f0b300b59e0 2021-02-10T11:39:51.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.587 7f0b36371700 1 -- 172.21.2.101:0/1330927029 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0b300c0e40 con 0x7f0b300b59e0 2021-02-10T11:39:51.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.587 7f0b2cff9700 1 -- 172.21.2.101:0/1330927029 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v60) v1 ==== 80+0+4513 (secure 0 0 0) 0x7f0b2802df80 con 0x7f0b300b59e0 2021-02-10T11:39:51.591 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:39:51.591 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:39:51.591 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:39:51.591 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:51.592 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:39:51.592 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:39:51.592 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:39:51.592 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:39:51.592 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:39:51.592 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:39:51.592 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:39:51.592 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:51.592 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:51.592 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:51.592 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:39:51.593 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:39:51.593 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:39:51.593 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:39:51.593 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:39:51.593 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:39:51.593 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:39:51.593 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:51.593 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:51.593 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:51.593 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:39:51.593 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:39:51.593 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:39:51.594 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:39:51.594 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:39:51.594 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:39:51.594 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:51.594 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:51.594 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:51.594 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:39:51.594 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:39:51.594 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:39:51.594 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:39:51.594 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:39:51.595 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:39:51.595 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:51.595 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:51.595 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:51.595 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:39:51.595 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:39:51.595 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:39:51.595 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:39:51.595 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:39:51.595 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:39:51.595 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:51.595 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:51.596 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:51.596 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:39:51.596 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:39:51.596 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:39:51.596 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:39:51.596 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:39:51.596 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:39:51.596 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:51.596 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:51.596 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:51.596 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:39:51.597 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:39:51.597 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:39:51.597 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:39:51.597 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:39:51.597 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:39:51.597 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:51.597 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:51.597 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:51.597 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:39:51.597 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:39:51.597 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:39:51.597 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:39:51.598 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:39:51.598 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:39:51.598 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:51.598 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:51.598 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:51.598 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:39:51.598 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:39:51.598 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:39:51.598 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:39:51.598 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:39:51.598 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:39:51.599 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:51.599 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:51.599 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:51.599 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:39:51.599 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:39:51.599 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:39:51.599 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:39:51.599 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:39:51.599 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:39:51.599 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:51.599 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:51.599 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:51.600 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:39:51.600 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:39:51.600 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:39:51.600 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:39:51.600 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:39:51.600 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:39:51.600 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:51.600 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:51.600 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:51.600 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:39:51.600 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:39:51.601 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:39:51.601 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:39:51.601 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:39:51.601 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:39:51.601 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:51.601 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:51.601 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:51.601 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:39:51.601 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:39:51.601 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:39:51.601 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:39:51.601 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:39:51.602 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:39:51.602 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:51.602 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:51.602 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:51.602 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:39:51.602 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:39:51.602 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:39:51.602 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:39:51.602 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:39:51.602 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:39:51.603 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:51.603 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:51.603 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:51.603 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:39:51.603 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:39:51.603 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:39:51.603 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:39:51.603 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:39:51.603 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:39:51.603 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:39:51.603 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:51.603 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:51.604 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:51.604 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:39:51.604 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:39:51.604 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:39:51.604 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:39:51.604 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:39:51.604 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:39:51.604 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:51.604 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:39:51.604 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:51.604 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:39:51.605 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:39:51.605 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:39:51.605 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:39:51.605 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:39:51.605 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:39:51.605 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:51.605 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:51.605 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:39:51.605 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:39:51.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.589 7f0b36371700 1 -- 172.21.2.101:0/1330927029 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b100186c0 msgr2=0x7f0b1001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:51.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.589 7f0b36371700 1 --2- 172.21.2.101:0/1330927029 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b100186c0 0x7f0b1001abc0 crc :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:51.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.589 7f0b36371700 1 -- 172.21.2.101:0/1330927029 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b300b59e0 msgr2=0x7f0b300c1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:51.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.589 7f0b36371700 1 --2- 172.21.2.101:0/1330927029 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b300b59e0 0x7f0b300c1990 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 rx=0x7f0b28008a20 tx=0x7f0b28008dc0).stop 2021-02-10T11:39:51.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.589 7f0b36371700 1 -- 172.21.2.101:0/1330927029 shutdown_connections 2021-02-10T11:39:51.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.589 7f0b36371700 1 --2- 172.21.2.101:0/1330927029 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b300b59e0 0x7f0b300c1990 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:51.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.589 7f0b36371700 1 --2- 172.21.2.101:0/1330927029 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b300b55e0 0x7f0b300c1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:51.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.589 7f0b36371700 1 --2- 172.21.2.101:0/1330927029 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b100186c0 0x7f0b1001abc0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:51.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.589 7f0b36371700 1 --2- 172.21.2.101:0/1330927029 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b3009bb70 0x7f0b300b4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:51.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.589 7f0b36371700 1 -- 172.21.2.101:0/1330927029 >> 172.21.2.101:0/1330927029 conn(0x7f0b300ac990 msgr2=0x7f0b30091b30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:51.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.590 7f0b36371700 1 -- 172.21.2.101:0/1330927029 shutdown_connections 2021-02-10T11:39:51.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:51.590 7f0b36371700 1 -- 172.21.2.101:0/1330927029 wait complete. 2021-02-10T11:39:51.608 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:39:52.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.016 7f517c36a700 1 Processor -- start 2021-02-10T11:39:52.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.016 7f517c36a700 1 -- start start 2021-02-10T11:39:52.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.016 7f517c36a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f51740d1870 0x7f51741b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:52.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.016 7f517c36a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51741b0a10 0x7f51740cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:52.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.016 7f517c36a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51740f0b10 con 0x7f51740d1c70 2021-02-10T11:39:52.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.016 7f517c36a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51740e5ed0 con 0x7f51740d1870 2021-02-10T11:39:52.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.016 7f517c36a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51740f0650 con 0x7f51741b0a10 2021-02-10T11:39:52.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.017 7f517a186700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f51740d1870 0x7f51741b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:52.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.017 7f5179985700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51741b0a10 0x7f51740cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:52.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.017 7f517a186700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f51740d1870 0x7f51741b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:38194/0 (socket says 172.21.2.101:38194) 2021-02-10T11:39:52.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.017 7f5179985700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51741b0a10 0x7f51740cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:48672/0 (socket says 172.21.2.101:48672) 2021-02-10T11:39:52.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.017 7f5179985700 1 -- 172.21.2.101:0/2149419553 learned_addr learned my addr 172.21.2.101:0/2149419553 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:52.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.018 7f5179985700 1 -- 172.21.2.101:0/2149419553 >> v2:172.21.2.108:3300/0 conn(0x7f51740d1870 msgr2=0x7f51741b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:52.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.018 7f5179985700 1 --2- 172.21.2.101:0/2149419553 >> v2:172.21.2.108:3300/0 conn(0x7f51740d1870 0x7f51741b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:52.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.018 7f5179985700 1 -- 172.21.2.101:0/2149419553 >> v1:172.21.2.108:6789/0 conn(0x7f51740d1c70 legacy=0x7f51740cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:52.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.018 7f5179985700 1 -- 172.21.2.101:0/2149419553 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51740f0870 con 0x7f51741b0a10 2021-02-10T11:39:52.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.018 7f517a186700 1 --2- 172.21.2.101:0/2149419553 >> v2:172.21.2.108:3300/0 conn(0x7f51740d1870 0x7f51741b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:39:52.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.018 7f5179985700 1 --2- 172.21.2.101:0/2149419553 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51741b0a10 0x7f51740cf330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f5170006700 tx=0x7f5170009b80).ready entity=mon.2 client_cookie=2a0437f6cddbff26 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:52.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.018 7f5179184700 1 -- 172.21.2.101:0/2149419553 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f517000c0c0 con 0x7f51741b0a10 2021-02-10T11:39:52.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.018 7f5179184700 1 -- 172.21.2.101:0/2149419553 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517000c340 con 0x7f51741b0a10 2021-02-10T11:39:52.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.018 7f5179184700 1 -- 172.21.2.101:0/2149419553 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f517000c860 con 0x7f51741b0a10 2021-02-10T11:39:52.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.018 7f517c36a700 1 -- 172.21.2.101:0/2149419553 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51741b0a10 msgr2=0x7f51740cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:52.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.018 7f517c36a700 1 --2- 172.21.2.101:0/2149419553 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51741b0a10 0x7f51740cf330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f5170006700 tx=0x7f5170009b80).stop 2021-02-10T11:39:52.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.019 7f517c36a700 1 -- 172.21.2.101:0/2149419553 shutdown_connections 2021-02-10T11:39:52.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.019 7f517c36a700 1 --2- 172.21.2.101:0/2149419553 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51741b0a10 0x7f51740cf330 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:52.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.019 7f517c36a700 1 --2- 172.21.2.101:0/2149419553 >> v2:172.21.2.108:3300/0 conn(0x7f51740d1870 0x7f51741b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:52.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.019 7f517c36a700 1 -- 172.21.2.101:0/2149419553 >> 172.21.2.101:0/2149419553 conn(0x7f51740ce450 msgr2=0x7f51740d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:52.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.019 7f517c36a700 1 -- 172.21.2.101:0/2149419553 shutdown_connections 2021-02-10T11:39:52.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.019 7f517c36a700 1 -- 172.21.2.101:0/2149419553 wait complete. 2021-02-10T11:39:52.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.034 7f517c36a700 1 Processor -- start 2021-02-10T11:39:52.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.034 7f517c36a700 1 -- start start 2021-02-10T11:39:52.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.034 7f517c36a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51740ada90 0x7f51740ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:52.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.034 7f517c36a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5174099d20 0x7f517409a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:52.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.034 7f517c36a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f517409a740 0x7f5174097bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:52.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.034 7f517c36a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51740997a0 con 0x7f5174099d20 2021-02-10T11:39:52.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.034 7f517c36a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517418f0c0 con 0x7f517409a740 2021-02-10T11:39:52.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.034 7f517c36a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f517409ab40 con 0x7f51740ada90 2021-02-10T11:39:52.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.034 7f5179985700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5174099d20 0x7f517409a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:52.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.034 7f5179985700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5174099d20 0x7f517409a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55946/0 (socket says 172.21.2.101:55946) 2021-02-10T11:39:52.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.034 7f5179985700 1 -- 172.21.2.101:0/1261930216 learned_addr learned my addr 172.21.2.101:0/1261930216 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:52.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.035 7f517a186700 1 --2- 172.21.2.101:0/1261930216 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51740ada90 0x7f51740ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:52.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.035 7f517a987700 1 --2- 172.21.2.101:0/1261930216 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f517409a740 0x7f5174097bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:52.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.035 7f5179985700 1 -- 172.21.2.101:0/1261930216 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51740ada90 msgr2=0x7f51740ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:52.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.035 7f5179985700 1 --2- 172.21.2.101:0/1261930216 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51740ada90 0x7f51740ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:52.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.035 7f5179985700 1 -- 172.21.2.101:0/1261930216 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f517409a740 msgr2=0x7f5174097bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:52.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.035 7f5179985700 1 --2- 172.21.2.101:0/1261930216 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f517409a740 0x7f5174097bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:52.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.035 7f5179985700 1 -- 172.21.2.101:0/1261930216 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517409ad60 con 0x7f5174099d20 2021-02-10T11:39:52.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.035 7f517a987700 1 --2- 172.21.2.101:0/1261930216 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f517409a740 0x7f5174097bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:39:52.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.035 7f5179985700 1 --2- 172.21.2.101:0/1261930216 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5174099d20 0x7f517409a120 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 rx=0x7f517000b5e0 tx=0x7f517000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:52.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.035 7f5166ffd700 1 -- 172.21.2.101:0/1261930216 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f517000bd70 con 0x7f5174099d20 2021-02-10T11:39:52.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.035 7f5166ffd700 1 -- 172.21.2.101:0/1261930216 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5170002030 con 0x7f5174099d20 2021-02-10T11:39:52.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.035 7f5166ffd700 1 -- 172.21.2.101:0/1261930216 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f51700024d0 con 0x7f5174099d20 2021-02-10T11:39:52.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.035 7f517c36a700 1 -- 172.21.2.101:0/1261930216 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517409ad60 con 0x7f5174099d20 2021-02-10T11:39:52.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.035 7f517c36a700 1 -- 172.21.2.101:0/1261930216 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51740984d0 con 0x7f5174099d20 2021-02-10T11:39:52.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.036 7f5166ffd700 1 -- 172.21.2.101:0/1261930216 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5170000c90 con 0x7f5174099d20 2021-02-10T11:39:52.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.036 7f5166ffd700 1 --2- 172.21.2.101:0/1261930216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f515c0186c0 0x7f515c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:52.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.036 7f5166ffd700 1 -- 172.21.2.101:0/1261930216 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f5170032140 con 0x7f5174099d20 2021-02-10T11:39:52.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.036 7f517a186700 1 --2- 172.21.2.101:0/1261930216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f515c0186c0 0x7f515c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:52.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.036 7f517a186700 1 --2- 172.21.2.101:0/1261930216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f515c0186c0 0x7f515c01abc0 crc :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:52.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.066 7f517c36a700 1 -- 172.21.2.101:0/1261930216 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51740ebc00 con 0x7f5174099d20 2021-02-10T11:39:52.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.068 7f5166ffd700 1 -- 172.21.2.101:0/1261930216 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f517002cf70 con 0x7f5174099d20 2021-02-10T11:39:52.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.194 7f517c36a700 1 -- 172.21.2.101:0/1261930216 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51740f0b10 con 0x7f5174099d20 2021-02-10T11:39:52.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.195 7f5166ffd700 1 -- 172.21.2.101:0/1261930216 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+11145 (secure 0 0 0) 0x7f517002cf70 con 0x7f5174099d20 2021-02-10T11:39:52.199 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:39:52.199 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":98,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:39:50.954464","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:39:52.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.197 7f517c36a700 1 -- 172.21.2.101:0/1261930216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f515c0186c0 msgr2=0x7f515c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:52.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.197 7f517c36a700 1 --2- 172.21.2.101:0/1261930216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f515c0186c0 0x7f515c01abc0 crc :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:52.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.197 7f517c36a700 1 -- 172.21.2.101:0/1261930216 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5174099d20 msgr2=0x7f517409a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:52.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.197 7f517c36a700 1 --2- 172.21.2.101:0/1261930216 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5174099d20 0x7f517409a120 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 rx=0x7f517000b5e0 tx=0x7f517000b980).stop 2021-02-10T11:39:52.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.197 7f517c36a700 1 -- 172.21.2.101:0/1261930216 shutdown_connections 2021-02-10T11:39:52.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.197 7f517c36a700 1 --2- 172.21.2.101:0/1261930216 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f517409a740 0x7f5174097bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:52.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.197 7f517c36a700 1 --2- 172.21.2.101:0/1261930216 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5174099d20 0x7f517409a120 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:52.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.197 7f517c36a700 1 --2- 172.21.2.101:0/1261930216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f515c0186c0 0x7f515c01abc0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:52.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.197 7f517c36a700 1 --2- 172.21.2.101:0/1261930216 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51740ada90 0x7f51740ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:52.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.197 7f517c36a700 1 -- 172.21.2.101:0/1261930216 >> 172.21.2.101:0/1261930216 conn(0x7f51740f0550 msgr2=0x7f5174191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:52.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.197 7f517c36a700 1 -- 172.21.2.101:0/1261930216 shutdown_connections 2021-02-10T11:39:52.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.197 7f517c36a700 1 -- 172.21.2.101:0/1261930216 wait complete. 2021-02-10T11:39:52.209 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-10T11:39:52.209 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:39:52.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.620 7f129b4e1700 1 Processor -- start 2021-02-10T11:39:52.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.620 7f129b4e1700 1 -- start start 2021-02-10T11:39:52.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.621 7f129b4e1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f12940d1c60 0x7f12940cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:52.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.621 7f129b4e1700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12940d0160 con 0x7f12940cf570 2021-02-10T11:39:52.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.621 7f129b4e1700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12940fe1b0 con 0x7f12940d1860 2021-02-10T11:39:52.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.621 7f129b4e1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12941b0ca0 con 0x7f12940d1c60 2021-02-10T11:39:52.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.621 7f1299afe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f12940cf570 0x7f12940cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56450/0 (socket says 172.21.2.101:56450) 2021-02-10T11:39:52.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.621 7f1299afe700 1 -- 172.21.2.101:0/2059558925 learned_addr learned my addr 172.21.2.101:0/2059558925 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:52.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.621 7f1287fff700 1 -- 172.21.2.101:0/2059558925 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3016527255 0 0) 0x7f128c001500 con 0x7f12940cf570 2021-02-10T11:39:52.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f1287fff700 1 -- 172.21.2.101:0/2059558925 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1280002fe0 con 0x7f12940cf570 2021-02-10T11:39:52.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f1298afc700 1 --2- 172.21.2.101:0/2059558925 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f12940d1c60 0x7f12940cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:52.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f1287fff700 1 -- 172.21.2.101:0/2059558925 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2945209380 0 0) 0x7f128c001460 con 0x7f12940cf570 2021-02-10T11:39:52.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f1287fff700 1 -- 172.21.2.101:0/2059558925 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f12940d1c60 msgr2=0x7f12940cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:52.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f1287fff700 1 --2- 172.21.2.101:0/2059558925 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f12940d1c60 0x7f12940cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:52.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f1287fff700 1 -- 172.21.2.101:0/2059558925 >> v1:172.21.2.108:6789/0 conn(0x7f12940d1860 legacy=0x7f12941b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:52.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f1287fff700 1 -- 172.21.2.101:0/2059558925 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1294109a50 con 0x7f12940cf570 2021-02-10T11:39:52.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f1287fff700 1 -- 172.21.2.101:0/2059558925 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f128c001980 con 0x7f12940cf570 2021-02-10T11:39:52.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f1287fff700 1 -- 172.21.2.101:0/2059558925 >> v1:172.21.2.101:6789/0 conn(0x7f12940cf570 legacy=0x7f12940cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:52.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f1287fff700 1 --2- 172.21.2.101:0/2059558925 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1280003a60 0x7f1280005e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:52.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f1287fff700 1 --2- 172.21.2.101:0/2059558925 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1280006730 0x7f1280008c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:52.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f1287fff700 1 --2- 172.21.2.101:0/2059558925 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f12800094f0 0x7f128000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:52.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f1287fff700 1 -- 172.21.2.101:0/2059558925 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128000c2c0 con 0x7f1280006730 2021-02-10T11:39:52.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f1287fff700 1 -- 172.21.2.101:0/2059558925 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128000c4e0 con 0x7f12800094f0 2021-02-10T11:39:52.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f1287fff700 1 -- 172.21.2.101:0/2059558925 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128000c700 con 0x7f1280003a60 2021-02-10T11:39:52.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f1298afc700 1 --2- 172.21.2.101:0/2059558925 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1280006730 0x7f1280008c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:52.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f12992fd700 1 --2- 172.21.2.101:0/2059558925 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1280003a60 0x7f1280005e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:52.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f1299afe700 1 --2- 172.21.2.101:0/2059558925 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f12800094f0 0x7f128000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:52.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f1298afc700 1 -- 172.21.2.101:0/2059558925 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1280003a60 msgr2=0x7f1280005e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:52.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f1298afc700 1 --2- 172.21.2.101:0/2059558925 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1280003a60 0x7f1280005e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:52.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f1298afc700 1 -- 172.21.2.101:0/2059558925 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f12800094f0 msgr2=0x7f128000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:52.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f1298afc700 1 --2- 172.21.2.101:0/2059558925 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f12800094f0 0x7f128000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:52.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.622 7f1298afc700 1 -- 172.21.2.101:0/2059558925 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128000c9a0 con 0x7f1280006730 2021-02-10T11:39:52.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.623 7f1298afc700 1 --2- 172.21.2.101:0/2059558925 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1280006730 0x7f1280008c50 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 rx=0x7f129004d6a0 tx=0x7f129004da40).ready entity=mon.0 client_cookie=2c61059779e8d51d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:52.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.623 7f1287fff700 1 -- 172.21.2.101:0/2059558925 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f12900066d0 con 0x7f1280006730 2021-02-10T11:39:52.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.623 7f1287fff700 1 -- 172.21.2.101:0/2059558925 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1290002030 con 0x7f1280006730 2021-02-10T11:39:52.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.623 7f1287fff700 1 -- 172.21.2.101:0/2059558925 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1290002510 con 0x7f1280006730 2021-02-10T11:39:52.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.623 7f129b4e1700 1 -- 172.21.2.101:0/2059558925 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1280006730 msgr2=0x7f1280008c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:52.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.623 7f129b4e1700 1 --2- 172.21.2.101:0/2059558925 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1280006730 0x7f1280008c50 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 rx=0x7f129004d6a0 tx=0x7f129004da40).stop 2021-02-10T11:39:52.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.623 7f1299afe700 1 -- 172.21.2.101:0/2059558925 reap_dead start 2021-02-10T11:39:52.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.624 7f129b4e1700 1 -- 172.21.2.101:0/2059558925 shutdown_connections 2021-02-10T11:39:52.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.624 7f129b4e1700 1 -- 172.21.2.101:0/2059558925 >> 172.21.2.101:0/2059558925 conn(0x7f12940ce410 msgr2=0x7f12940d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:52.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.624 7f129b4e1700 1 -- 172.21.2.101:0/2059558925 shutdown_connections 2021-02-10T11:39:52.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.624 7f129b4e1700 1 -- 172.21.2.101:0/2059558925 wait complete. 2021-02-10T11:39:52.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.639 7f129b4e1700 1 Processor -- start 2021-02-10T11:39:52.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.639 7f129b4e1700 1 -- start start 2021-02-10T11:39:52.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.639 7f129b4e1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12940adf50 0x7f12940ae350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:52.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.639 7f129b4e1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f12940ae970 0x7f12940aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:52.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.639 7f129b4e1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f12940982c0 0x7f12940986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:52.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.639 7f129b4e1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129409f450 con 0x7f12940adf50 2021-02-10T11:39:52.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.639 7f129b4e1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1294098ce0 con 0x7f12940ae970 2021-02-10T11:39:52.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.639 7f129b4e1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1294098ed0 con 0x7f12940982c0 2021-02-10T11:39:52.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.639 7f12992fd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12940adf50 0x7f12940ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:52.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.639 7f12992fd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12940adf50 0x7f12940ae350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55964/0 (socket says 172.21.2.101:55964) 2021-02-10T11:39:52.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.639 7f12992fd700 1 -- 172.21.2.101:0/3801478658 learned_addr learned my addr 172.21.2.101:0/3801478658 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:52.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.640 7f12992fd700 1 -- 172.21.2.101:0/3801478658 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f12940982c0 msgr2=0x7f12940986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:52.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.640 7f12992fd700 1 --2- 172.21.2.101:0/3801478658 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f12940982c0 0x7f12940986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:52.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.640 7f12992fd700 1 -- 172.21.2.101:0/3801478658 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f12940ae970 msgr2=0x7f12940aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:52.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.640 7f12992fd700 1 --2- 172.21.2.101:0/3801478658 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f12940ae970 0x7f12940aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:52.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.640 7f12992fd700 1 -- 172.21.2.101:0/3801478658 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12940990f0 con 0x7f12940adf50 2021-02-10T11:39:52.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.640 7f12992fd700 1 --2- 172.21.2.101:0/3801478658 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12940adf50 0x7f12940ae350 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 rx=0x7f128800b5e0 tx=0x7f128800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:52.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.640 7f1285ffb700 1 -- 172.21.2.101:0/3801478658 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f128800bd70 con 0x7f12940adf50 2021-02-10T11:39:52.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.640 7f1285ffb700 1 -- 172.21.2.101:0/3801478658 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12880063c0 con 0x7f12940adf50 2021-02-10T11:39:52.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.640 7f1285ffb700 1 -- 172.21.2.101:0/3801478658 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f12880054d0 con 0x7f12940adf50 2021-02-10T11:39:52.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.640 7f129b4e1700 1 -- 172.21.2.101:0/3801478658 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12940990f0 con 0x7f12940adf50 2021-02-10T11:39:52.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.640 7f129b4e1700 1 -- 172.21.2.101:0/3801478658 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12940990f0 con 0x7f12940adf50 2021-02-10T11:39:52.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.641 7f1285ffb700 1 -- 172.21.2.101:0/3801478658 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f12880052e0 con 0x7f12940adf50 2021-02-10T11:39:52.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.641 7f1285ffb700 1 --2- 172.21.2.101:0/3801478658 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f12740186c0 0x7f127401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:52.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.641 7f1285ffb700 1 -- 172.21.2.101:0/3801478658 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f1288032230 con 0x7f12940adf50 2021-02-10T11:39:52.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.641 7f1298afc700 1 --2- 172.21.2.101:0/3801478658 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f12740186c0 0x7f127401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:52.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.641 7f1298afc700 1 --2- 172.21.2.101:0/3801478658 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f12740186c0 0x7f127401abc0 crc :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:52.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.671 7f129b4e1700 1 -- 172.21.2.101:0/3801478658 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1294082a20 con 0x7f12940adf50 2021-02-10T11:39:52.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.673 7f1285ffb700 1 -- 172.21.2.101:0/3801478658 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1288006030 con 0x7f12940adf50 2021-02-10T11:39:52.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.800 7f129b4e1700 1 -- 172.21.2.101:0/3801478658 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f1294082a20 con 0x7f12940adf50 2021-02-10T11:39:52.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.801 7f1285ffb700 1 -- 172.21.2.101:0/3801478658 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v98) v1 ==== 92+0+10 (secure 0 0 0) 0x7f128800bd70 con 0x7f12940adf50 2021-02-10T11:39:52.804 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:39:52.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.802 7f129b4e1700 1 -- 172.21.2.101:0/3801478658 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f12740186c0 msgr2=0x7f127401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:52.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.802 7f129b4e1700 1 --2- 172.21.2.101:0/3801478658 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f12740186c0 0x7f127401abc0 crc :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:52.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.803 7f129b4e1700 1 -- 172.21.2.101:0/3801478658 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12940adf50 msgr2=0x7f12940ae350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:52.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.803 7f129b4e1700 1 --2- 172.21.2.101:0/3801478658 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12940adf50 0x7f12940ae350 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 rx=0x7f128800b5e0 tx=0x7f128800b980).stop 2021-02-10T11:39:52.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.803 7f129b4e1700 1 -- 172.21.2.101:0/3801478658 shutdown_connections 2021-02-10T11:39:52.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.803 7f129b4e1700 1 --2- 172.21.2.101:0/3801478658 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f12740186c0 0x7f127401abc0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:52.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.803 7f129b4e1700 1 --2- 172.21.2.101:0/3801478658 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f12940982c0 0x7f12940986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:52.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.803 7f129b4e1700 1 --2- 172.21.2.101:0/3801478658 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f12940ae970 0x7f12940aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:52.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.803 7f129b4e1700 1 --2- 172.21.2.101:0/3801478658 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12940adf50 0x7f12940ae350 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:52.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.803 7f129b4e1700 1 -- 172.21.2.101:0/3801478658 >> 172.21.2.101:0/3801478658 conn(0x7f12940bee90 msgr2=0x7f12940bb880 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:52.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.803 7f129b4e1700 1 -- 172.21.2.101:0/3801478658 shutdown_connections 2021-02-10T11:39:52.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:52.803 7f129b4e1700 1 -- 172.21.2.101:0/3801478658 wait complete. 2021-02-10T11:39:52.815 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --format=json-pretty osd lspools 2021-02-10T11:39:53.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.223 7efcd00d4700 1 Processor -- start 2021-02-10T11:39:53.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.223 7efcd00d4700 1 -- start start 2021-02-10T11:39:53.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.223 7efcd00d4700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7efcc80d1870 0x7efcc81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:53.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.223 7efcd00d4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcc81b0a10 0x7efcc80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:53.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.223 7efcd00d4700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcc80f0b10 con 0x7efcc80d1c70 2021-02-10T11:39:53.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.223 7efcd00d4700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcc80e5ed0 con 0x7efcc80d1870 2021-02-10T11:39:53.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.223 7efcd00d4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc80f0650 con 0x7efcc81b0a10 2021-02-10T11:39:53.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.223 7efcce6f1700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7efcc80d1c70 0x7efcc80cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56466/0 (socket says 172.21.2.101:56466) 2021-02-10T11:39:53.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.223 7efcce6f1700 1 -- 172.21.2.101:0/1250026014 learned_addr learned my addr 172.21.2.101:0/1250026014 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:53.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.224 7efccceee700 1 -- 172.21.2.101:0/1250026014 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3161833095 0 0) 0x7efcc0001500 con 0x7efcc80d1c70 2021-02-10T11:39:53.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.224 7efccceee700 1 -- 172.21.2.101:0/1250026014 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcb4002fe0 con 0x7efcc80d1c70 2021-02-10T11:39:53.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.224 7efccdef0700 1 --2- 172.21.2.101:0/1250026014 >> v2:172.21.2.108:3300/0 conn(0x7efcc80d1870 0x7efcc81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:53.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.224 7efccd6ef700 1 --2- 172.21.2.101:0/1250026014 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcc81b0a10 0x7efcc80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:53.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.224 7efccceee700 1 -- 172.21.2.101:0/1250026014 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4081266848 0 0) 0x7efcc0001500 con 0x7efcc80d1c70 2021-02-10T11:39:53.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.224 7efccceee700 1 -- 172.21.2.101:0/1250026014 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcc81b0a10 msgr2=0x7efcc80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:53.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.224 7efccceee700 1 --2- 172.21.2.101:0/1250026014 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcc81b0a10 0x7efcc80cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:53.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.224 7efccceee700 1 -- 172.21.2.101:0/1250026014 >> v2:172.21.2.108:3300/0 conn(0x7efcc80d1870 msgr2=0x7efcc81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:53.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.224 7efccceee700 1 --2- 172.21.2.101:0/1250026014 >> v2:172.21.2.108:3300/0 conn(0x7efcc80d1870 0x7efcc81b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:53.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.224 7efccceee700 1 -- 172.21.2.101:0/1250026014 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc80f0870 con 0x7efcc80d1c70 2021-02-10T11:39:53.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.224 7efccceee700 1 -- 172.21.2.101:0/1250026014 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7efcc0001a00 con 0x7efcc80d1c70 2021-02-10T11:39:53.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.224 7efccceee700 1 -- 172.21.2.101:0/1250026014 >> v1:172.21.2.101:6789/0 conn(0x7efcc80d1c70 legacy=0x7efcc80cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:53.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.224 7efccceee700 1 --2- 172.21.2.101:0/1250026014 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efcb4003b90 0x7efcb4005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:53.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.224 7efccceee700 1 --2- 172.21.2.101:0/1250026014 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcb40067c0 0x7efcb4008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:53.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.224 7efccceee700 1 --2- 172.21.2.101:0/1250026014 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcb4009510 0x7efcb400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:53.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.224 7efccceee700 1 -- 172.21.2.101:0/1250026014 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb400c2c0 con 0x7efcb4009510 2021-02-10T11:39:53.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.224 7efccceee700 1 -- 172.21.2.101:0/1250026014 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb40504b0 con 0x7efcb4003b90 2021-02-10T11:39:53.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.224 7efccceee700 1 -- 172.21.2.101:0/1250026014 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb40506d0 con 0x7efcb40067c0 2021-02-10T11:39:53.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.224 7efcce6f1700 1 --2- 172.21.2.101:0/1250026014 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcb4009510 0x7efcb400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:53.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.225 7efccdef0700 1 --2- 172.21.2.101:0/1250026014 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efcb4003b90 0x7efcb4005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:53.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.225 7efccd6ef700 1 --2- 172.21.2.101:0/1250026014 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcb40067c0 0x7efcb4008c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:53.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.225 7efcce6f1700 1 -- 172.21.2.101:0/1250026014 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcb40067c0 msgr2=0x7efcb4008c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:53.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.225 7efcce6f1700 1 --2- 172.21.2.101:0/1250026014 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcb40067c0 0x7efcb4008c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:53.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.225 7efcce6f1700 1 -- 172.21.2.101:0/1250026014 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efcb4003b90 msgr2=0x7efcb4005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:53.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.225 7efcce6f1700 1 --2- 172.21.2.101:0/1250026014 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efcb4003b90 0x7efcb4005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:53.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.225 7efcce6f1700 1 -- 172.21.2.101:0/1250026014 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb4050970 con 0x7efcb4009510 2021-02-10T11:39:53.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.225 7efccd6ef700 1 --2- 172.21.2.101:0/1250026014 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcb40067c0 0x7efcb4008c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:39:53.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.225 7efcce6f1700 1 --2- 172.21.2.101:0/1250026014 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcb4009510 0x7efcb400ba10 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 rx=0x7efcc0007a70 tx=0x7efcc000a310).ready entity=mon.0 client_cookie=5e4c92a9713088d2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:53.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.225 7efccceee700 1 -- 172.21.2.101:0/1250026014 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efcc000a910 con 0x7efcb4009510 2021-02-10T11:39:53.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.225 7efccceee700 1 -- 172.21.2.101:0/1250026014 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc000ab90 con 0x7efcb4009510 2021-02-10T11:39:53.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.225 7efccceee700 1 -- 172.21.2.101:0/1250026014 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efcc0007340 con 0x7efcb4009510 2021-02-10T11:39:53.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.225 7efcd00d4700 1 -- 172.21.2.101:0/1250026014 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcb4009510 msgr2=0x7efcb400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:53.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.225 7efcd00d4700 1 --2- 172.21.2.101:0/1250026014 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcb4009510 0x7efcb400ba10 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 rx=0x7efcc0007a70 tx=0x7efcc000a310).stop 2021-02-10T11:39:53.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.225 7efcce6f1700 1 -- 172.21.2.101:0/1250026014 reap_dead start 2021-02-10T11:39:53.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.226 7efcd00d4700 1 -- 172.21.2.101:0/1250026014 shutdown_connections 2021-02-10T11:39:53.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.226 7efcd00d4700 1 -- 172.21.2.101:0/1250026014 >> 172.21.2.101:0/1250026014 conn(0x7efcc80ce450 msgr2=0x7efcc80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:53.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.226 7efcd00d4700 1 -- 172.21.2.101:0/1250026014 shutdown_connections 2021-02-10T11:39:53.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.226 7efcd00d4700 1 -- 172.21.2.101:0/1250026014 wait complete. 2021-02-10T11:39:53.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.241 7efcd00d4700 1 Processor -- start 2021-02-10T11:39:53.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.241 7efcd00d4700 1 -- start start 2021-02-10T11:39:53.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.241 7efcd00d4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcc8195650 0x7efcc80b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:53.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.241 7efcd00d4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efcc80b5b80 0x7efcc81a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:53.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.241 7efcd00d4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcc80acfa0 0x7efcc81a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:53.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.241 7efcd00d4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc80b6010 con 0x7efcc8195650 2021-02-10T11:39:53.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.241 7efcd00d4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc81a3a60 con 0x7efcc80b5b80 2021-02-10T11:39:53.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.241 7efcd00d4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc81a3c40 con 0x7efcc80acfa0 2021-02-10T11:39:53.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.241 7efccdef0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcc8195650 0x7efcc80b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:53.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.241 7efccdef0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcc8195650 0x7efcc80b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:55984/0 (socket says 172.21.2.101:55984) 2021-02-10T11:39:53.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.241 7efccdef0700 1 -- 172.21.2.101:0/1704730827 learned_addr learned my addr 172.21.2.101:0/1704730827 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:53.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.241 7efccdef0700 1 -- 172.21.2.101:0/1704730827 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcc80acfa0 msgr2=0x7efcc81a3440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:53.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.241 7efccdef0700 1 --2- 172.21.2.101:0/1704730827 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcc80acfa0 0x7efcc81a3440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:53.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.241 7efccdef0700 1 -- 172.21.2.101:0/1704730827 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efcc80b5b80 msgr2=0x7efcc81a2e20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:53.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.241 7efccdef0700 1 --2- 172.21.2.101:0/1704730827 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efcc80b5b80 0x7efcc81a2e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:53.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.241 7efccdef0700 1 -- 172.21.2.101:0/1704730827 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc80bcae0 con 0x7efcc8195650 2021-02-10T11:39:53.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.242 7efccdef0700 1 --2- 172.21.2.101:0/1704730827 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcc8195650 0x7efcc80b5560 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 rx=0x7efcbc006f00 tx=0x7efcbc00b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:53.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.242 7efcba7fc700 1 -- 172.21.2.101:0/1704730827 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efcbc00bd30 con 0x7efcc8195650 2021-02-10T11:39:53.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.242 7efcba7fc700 1 -- 172.21.2.101:0/1704730827 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcbc002030 con 0x7efcc8195650 2021-02-10T11:39:53.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.242 7efcba7fc700 1 -- 172.21.2.101:0/1704730827 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efcbc002480 con 0x7efcc8195650 2021-02-10T11:39:53.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.242 7efcd00d4700 1 -- 172.21.2.101:0/1704730827 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc80bcae0 con 0x7efcc8195650 2021-02-10T11:39:53.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.242 7efcd00d4700 1 -- 172.21.2.101:0/1704730827 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc80bcae0 con 0x7efcc8195650 2021-02-10T11:39:53.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.242 7efcba7fc700 1 -- 172.21.2.101:0/1704730827 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7efcbc000c90 con 0x7efcc8195650 2021-02-10T11:39:53.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.242 7efcba7fc700 1 --2- 172.21.2.101:0/1704730827 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efcb00186c0 0x7efcb001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:53.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.242 7efcba7fc700 1 -- 172.21.2.101:0/1704730827 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4753+0+0 (secure 0 0 0) 0x7efcbc032260 con 0x7efcc8195650 2021-02-10T11:39:53.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.243 7efccd6ef700 1 --2- 172.21.2.101:0/1704730827 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efcb00186c0 0x7efcb001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:53.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.243 7efccd6ef700 1 --2- 172.21.2.101:0/1704730827 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efcb00186c0 0x7efcb001abc0 crc :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:53.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.273 7efcd00d4700 1 -- 172.21.2.101:0/1704730827 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc80b6010 con 0x7efcc8195650 2021-02-10T11:39:53.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.275 7efcba7fc700 1 -- 172.21.2.101:0/1704730827 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7efcbc030d40 con 0x7efcc8195650 2021-02-10T11:39:53.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.399 7efcd00d4700 1 -- 172.21.2.101:0/1704730827 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7efcc80b6010 con 0x7efcc8195650 2021-02-10T11:39:53.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.399 7efcba7fc700 1 -- 172.21.2.101:0/1704730827 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v98) v1 ==== 84+0+64 (secure 0 0 0) 0x7efcbc000c90 con 0x7efcc8195650 2021-02-10T11:39:53.403 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:39:53.403 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:39:53.403 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:39:53.403 INFO:teuthology.orchestra.run.gibba001.stdout: "poolnum": 1, 2021-02-10T11:39:53.403 INFO:teuthology.orchestra.run.gibba001.stdout: "poolname": "rbd" 2021-02-10T11:39:53.404 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:39:53.404 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:39:53.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.401 7efcd00d4700 1 -- 172.21.2.101:0/1704730827 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efcb00186c0 msgr2=0x7efcb001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:53.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.401 7efcd00d4700 1 --2- 172.21.2.101:0/1704730827 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efcb00186c0 0x7efcb001abc0 crc :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:53.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.401 7efcd00d4700 1 -- 172.21.2.101:0/1704730827 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcc8195650 msgr2=0x7efcc80b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:53.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.401 7efcd00d4700 1 --2- 172.21.2.101:0/1704730827 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcc8195650 0x7efcc80b5560 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 rx=0x7efcbc006f00 tx=0x7efcbc00b940).stop 2021-02-10T11:39:53.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.402 7efcd00d4700 1 -- 172.21.2.101:0/1704730827 shutdown_connections 2021-02-10T11:39:53.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.402 7efcd00d4700 1 --2- 172.21.2.101:0/1704730827 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efcb00186c0 0x7efcb001abc0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:53.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.402 7efcd00d4700 1 --2- 172.21.2.101:0/1704730827 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efcc80acfa0 0x7efcc81a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:53.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.402 7efcd00d4700 1 --2- 172.21.2.101:0/1704730827 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efcc80b5b80 0x7efcc81a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:53.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.402 7efcd00d4700 1 --2- 172.21.2.101:0/1704730827 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efcc8195650 0x7efcc80b5560 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:53.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.402 7efcd00d4700 1 -- 172.21.2.101:0/1704730827 >> 172.21.2.101:0/1704730827 conn(0x7efcc80b3040 msgr2=0x7efcc80c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:53.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.402 7efcd00d4700 1 -- 172.21.2.101:0/1704730827 shutdown_connections 2021-02-10T11:39:53.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.402 7efcd00d4700 1 -- 172.21.2.101:0/1704730827 wait complete. 2021-02-10T11:39:53.413 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-02-10T11:39:53.414 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-02-10T11:39:53.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.825 7f44af182700 1 Processor -- start 2021-02-10T11:39:53.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.825 7f44af182700 1 -- start start 2021-02-10T11:39:53.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.826 7f44af182700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f44a810aff0 0x7f44a80cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:53.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.826 7f44af182700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a80fe1c0 con 0x7f44a80d1c70 2021-02-10T11:39:53.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.826 7f44af182700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a81b0cb0 con 0x7f44a80d1870 2021-02-10T11:39:53.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.826 7f44af182700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a8109a60 con 0x7f44a810aff0 2021-02-10T11:39:53.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.826 7f44a7fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f44a80d1c70 0x7f44a80cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56486/0 (socket says 172.21.2.101:56486) 2021-02-10T11:39:53.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.826 7f44a7fff700 1 -- 172.21.2.101:0/4270392114 learned_addr learned my addr 172.21.2.101:0/4270392114 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:53.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.826 7f44a77fe700 1 -- 172.21.2.101:0/4270392114 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1377345428 0 0) 0x7f44980009b0 con 0x7f44a80d1c70 2021-02-10T11:39:53.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.826 7f44a77fe700 1 -- 172.21.2.101:0/4270392114 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4494002fe0 con 0x7f44a80d1c70 2021-02-10T11:39:53.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.826 7f44ad79f700 1 --2- 172.21.2.101:0/4270392114 >> v2:172.21.2.108:3300/0 conn(0x7f44a810aff0 0x7f44a80cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:53.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.826 7f44a77fe700 1 -- 172.21.2.101:0/4270392114 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1252496799 0 0) 0x7f4498001010 con 0x7f44a80d1c70 2021-02-10T11:39:53.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.826 7f44a77fe700 1 -- 172.21.2.101:0/4270392114 >> v2:172.21.2.108:3300/0 conn(0x7f44a810aff0 msgr2=0x7f44a80cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:53.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.826 7f44a77fe700 1 --2- 172.21.2.101:0/4270392114 >> v2:172.21.2.108:3300/0 conn(0x7f44a810aff0 0x7f44a80cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:53.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.826 7f44a77fe700 1 -- 172.21.2.101:0/4270392114 >> v1:172.21.2.108:6789/0 conn(0x7f44a80d1870 legacy=0x7f44a81b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:53.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.826 7f44a77fe700 1 -- 172.21.2.101:0/4270392114 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a810b3f0 con 0x7f44a80d1c70 2021-02-10T11:39:53.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.826 7f44a77fe700 1 -- 172.21.2.101:0/4270392114 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f4498001590 con 0x7f44a80d1c70 2021-02-10T11:39:53.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.826 7f44a77fe700 1 -- 172.21.2.101:0/4270392114 >> v1:172.21.2.101:6789/0 conn(0x7f44a80d1c70 legacy=0x7f44a80cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:53.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.827 7f44a77fe700 1 --2- 172.21.2.101:0/4270392114 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4494003aa0 0x7f4494005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:53.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.827 7f44a77fe700 1 --2- 172.21.2.101:0/4270392114 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4494006780 0x7f4494008ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:53.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.827 7f44a77fe700 1 --2- 172.21.2.101:0/4270392114 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4494009580 0x7f449400baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:53.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.827 7f44a77fe700 1 -- 172.21.2.101:0/4270392114 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449400c2f0 con 0x7f4494006780 2021-02-10T11:39:53.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.827 7f44a77fe700 1 -- 172.21.2.101:0/4270392114 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449400c510 con 0x7f4494003aa0 2021-02-10T11:39:53.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.827 7f44a77fe700 1 -- 172.21.2.101:0/4270392114 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4494050740 con 0x7f4494009580 2021-02-10T11:39:53.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.827 7f44a7fff700 1 --2- 172.21.2.101:0/4270392114 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4494006780 0x7f4494008ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:53.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.827 7f44acf9e700 1 --2- 172.21.2.101:0/4270392114 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4494003aa0 0x7f4494005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:53.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.827 7f44ad79f700 1 --2- 172.21.2.101:0/4270392114 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4494009580 0x7f449400baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:53.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.827 7f44a7fff700 1 -- 172.21.2.101:0/4270392114 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4494009580 msgr2=0x7f449400baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:53.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.827 7f44a7fff700 1 --2- 172.21.2.101:0/4270392114 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4494009580 0x7f449400baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:53.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.827 7f44a7fff700 1 -- 172.21.2.101:0/4270392114 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4494003aa0 msgr2=0x7f4494005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:53.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.827 7f44a7fff700 1 --2- 172.21.2.101:0/4270392114 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4494003aa0 0x7f4494005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:53.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.827 7f44a7fff700 1 -- 172.21.2.101:0/4270392114 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4494050970 con 0x7f4494006780 2021-02-10T11:39:53.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.827 7f44ad79f700 1 -- 172.21.2.101:0/4270392114 reap_dead start 2021-02-10T11:39:53.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.827 7f44acf9e700 1 --2- 172.21.2.101:0/4270392114 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4494003aa0 0x7f4494005f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:39:53.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.827 7f44a7fff700 1 --2- 172.21.2.101:0/4270392114 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4494006780 0x7f4494008ca0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 rx=0x7f4498000b60 tx=0x7f4498007c90).ready entity=mon.0 client_cookie=9158689dc119784e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:53.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.827 7f44a77fe700 1 -- 172.21.2.101:0/4270392114 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4498001010 con 0x7f4494006780 2021-02-10T11:39:53.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.827 7f44a77fe700 1 -- 172.21.2.101:0/4270392114 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44980013d0 con 0x7f4494006780 2021-02-10T11:39:53.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.827 7f44a77fe700 1 -- 172.21.2.101:0/4270392114 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f449800a710 con 0x7f4494006780 2021-02-10T11:39:53.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.827 7f44af182700 1 -- 172.21.2.101:0/4270392114 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4494006780 msgr2=0x7f4494008ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:53.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.827 7f44af182700 1 --2- 172.21.2.101:0/4270392114 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4494006780 0x7f4494008ca0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 rx=0x7f4498000b60 tx=0x7f4498007c90).stop 2021-02-10T11:39:53.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.828 7f44af182700 1 -- 172.21.2.101:0/4270392114 shutdown_connections 2021-02-10T11:39:53.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.828 7f44af182700 1 --2- 172.21.2.101:0/4270392114 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4494006780 0x7f4494008ca0 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:53.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.828 7f44af182700 1 -- 172.21.2.101:0/4270392114 >> 172.21.2.101:0/4270392114 conn(0x7f44a80ce420 msgr2=0x7f44a80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:53.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.828 7f44af182700 1 -- 172.21.2.101:0/4270392114 shutdown_connections 2021-02-10T11:39:53.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.828 7f44af182700 1 -- 172.21.2.101:0/4270392114 wait complete. 2021-02-10T11:39:53.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.843 7f44af182700 1 Processor -- start 2021-02-10T11:39:53.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.843 7f44af182700 1 -- start start 2021-02-10T11:39:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.843 7f44af182700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44a8085bc0 0x7f44a80863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.843 7f44af182700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44a80954c0 0x7f44a8096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.843 7f44af182700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44a80958c0 0x7f44a8096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.843 7f44af182700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a8095d80 con 0x7f44a80954c0 2021-02-10T11:39:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.843 7f44af182700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a80bfa60 con 0x7f44a8085bc0 2021-02-10T11:39:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.843 7f44af182700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a8085fc0 con 0x7f44a80958c0 2021-02-10T11:39:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.843 7f44a7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44a80954c0 0x7f44a8096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.843 7f44a7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44a80954c0 0x7f44a8096300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56004/0 (socket says 172.21.2.101:56004) 2021-02-10T11:39:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.843 7f44a7fff700 1 -- 172.21.2.101:0/3530874355 learned_addr learned my addr 172.21.2.101:0/3530874355 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.844 7f44a7fff700 1 -- 172.21.2.101:0/3530874355 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44a80958c0 msgr2=0x7f44a8096920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.844 7f44a7fff700 1 --2- 172.21.2.101:0/3530874355 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44a80958c0 0x7f44a8096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:53.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.844 7f44a7fff700 1 -- 172.21.2.101:0/3530874355 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44a8085bc0 msgr2=0x7f44a80863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.844 7f44a7fff700 1 --2- 172.21.2.101:0/3530874355 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44a8085bc0 0x7f44a80863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.844 7f44a7fff700 1 -- 172.21.2.101:0/3530874355 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a8086c10 con 0x7f44a80954c0 2021-02-10T11:39:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.844 7f44a7fff700 1 --2- 172.21.2.101:0/3530874355 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44a80954c0 0x7f44a8096300 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 rx=0x7f449800a580 tx=0x7f449800a920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.844 7f44a57fa700 1 -- 172.21.2.101:0/3530874355 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f449800ad10 con 0x7f44a80954c0 2021-02-10T11:39:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.844 7f44a57fa700 1 -- 172.21.2.101:0/3530874355 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44980018a0 con 0x7f44a80954c0 2021-02-10T11:39:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.844 7f44a57fa700 1 -- 172.21.2.101:0/3530874355 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4498004330 con 0x7f44a80954c0 2021-02-10T11:39:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.844 7f44af182700 1 -- 172.21.2.101:0/3530874355 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a8086c10 con 0x7f44a80954c0 2021-02-10T11:39:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.844 7f44af182700 1 -- 172.21.2.101:0/3530874355 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a8086c10 con 0x7f44a80954c0 2021-02-10T11:39:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.845 7f44a57fa700 1 -- 172.21.2.101:0/3530874355 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4498004140 con 0x7f44a80954c0 2021-02-10T11:39:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.845 7f44a57fa700 1 --2- 172.21.2.101:0/3530874355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f448c0186c0 0x7f448c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.845 7f44a57fa700 1 -- 172.21.2.101:0/3530874355 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f44980322c0 con 0x7f44a80954c0 2021-02-10T11:39:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.845 7f44acf9e700 1 --2- 172.21.2.101:0/3530874355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f448c0186c0 0x7f448c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:53.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.845 7f44acf9e700 1 --2- 172.21.2.101:0/3530874355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f448c0186c0 0x7f448c01abc0 crc :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:53.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.875 7f44af182700 1 -- 172.21.2.101:0/3530874355 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a80fe1c0 con 0x7f44a80954c0 2021-02-10T11:39:53.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:53.878 7f44a57fa700 1 -- 172.21.2.101:0/3530874355 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4498000940 con 0x7f44a80954c0 2021-02-10T11:39:54.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.004 7f44af182700 1 -- 172.21.2.101:0/3530874355 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"} v 0) v1 -- 0x7f44a80fe1c0 con 0x7f44a80954c0 2021-02-10T11:39:54.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.427 7f44a57fa700 1 -- 172.21.2.101:0/3530874355 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"}]=0 pool 'cephfs_metadata' created v99) v1 ==== 133+0+0 (secure 0 0 0) 0x7f4498004140 con 0x7f44a80954c0 2021-02-10T11:39:54.473 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' created 2021-02-10T11:39:54.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.471 7f44af182700 1 -- 172.21.2.101:0/3530874355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f448c0186c0 msgr2=0x7f448c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:54.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.471 7f44af182700 1 --2- 172.21.2.101:0/3530874355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f448c0186c0 0x7f448c01abc0 crc :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:54.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.472 7f44af182700 1 -- 172.21.2.101:0/3530874355 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44a80954c0 msgr2=0x7f44a8096300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:54.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.472 7f44af182700 1 --2- 172.21.2.101:0/3530874355 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44a80954c0 0x7f44a8096300 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 rx=0x7f449800a580 tx=0x7f449800a920).stop 2021-02-10T11:39:54.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.472 7f44af182700 1 -- 172.21.2.101:0/3530874355 shutdown_connections 2021-02-10T11:39:54.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.472 7f44af182700 1 --2- 172.21.2.101:0/3530874355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f448c0186c0 0x7f448c01abc0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:54.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.472 7f44af182700 1 --2- 172.21.2.101:0/3530874355 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44a80958c0 0x7f44a8096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:54.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.472 7f44af182700 1 --2- 172.21.2.101:0/3530874355 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f44a80954c0 0x7f44a8096300 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:54.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.472 7f44af182700 1 --2- 172.21.2.101:0/3530874355 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44a8085bc0 0x7f44a80863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:54.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.472 7f44af182700 1 -- 172.21.2.101:0/3530874355 >> 172.21.2.101:0/3530874355 conn(0x7f44a80bef00 msgr2=0x7f44a809d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:54.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.472 7f44af182700 1 -- 172.21.2.101:0/3530874355 shutdown_connections 2021-02-10T11:39:54.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.472 7f44af182700 1 -- 172.21.2.101:0/3530874355 wait complete. 2021-02-10T11:39:54.483 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_data 8 2021-02-10T11:39:54.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.895 7fbb2d8f3700 1 Processor -- start 2021-02-10T11:39:54.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.895 7fbb2d8f3700 1 -- start start 2021-02-10T11:39:54.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.896 7fbb2d8f3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb280d1870 0x7fbb281b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:54.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.896 7fbb2d8f3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbb281b0ae0 0x7fbb28108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:54.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.896 7fbb2d8f3700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb28109a60 con 0x7fbb280d1c70 2021-02-10T11:39:54.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.896 7fbb2d8f3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb280e5ef0 con 0x7fbb281b0ae0 2021-02-10T11:39:54.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.896 7fbb2d8f3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb28109750 con 0x7fbb280d1870 2021-02-10T11:39:54.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.896 7fbb27d75700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fbb280d1c70 0x7fbb28108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56502/0 (socket says 172.21.2.101:56502) 2021-02-10T11:39:54.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.896 7fbb27d75700 1 -- 172.21.2.101:0/1991112963 learned_addr learned my addr 172.21.2.101:0/1991112963 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:54.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.896 7fbb26572700 1 -- 172.21.2.101:0/1991112963 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3213526595 0 0) 0x7fbb20001500 con 0x7fbb280d1c70 2021-02-10T11:39:54.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.896 7fbb26572700 1 -- 172.21.2.101:0/1991112963 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb10002fe0 con 0x7fbb280d1c70 2021-02-10T11:39:54.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.897 7fbb27574700 1 --2- 172.21.2.101:0/1991112963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb280d1870 0x7fbb281b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:54.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.897 7fbb26d73700 1 --2- 172.21.2.101:0/1991112963 >> v2:172.21.2.108:3300/0 conn(0x7fbb281b0ae0 0x7fbb28108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:54.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.897 7fbb26572700 1 -- 172.21.2.101:0/1991112963 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2097195718 0 0) 0x7fbb20001500 con 0x7fbb280d1c70 2021-02-10T11:39:54.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.897 7fbb26572700 1 -- 172.21.2.101:0/1991112963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb280d1870 msgr2=0x7fbb281b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:54.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.897 7fbb26572700 1 --2- 172.21.2.101:0/1991112963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb280d1870 0x7fbb281b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:54.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.897 7fbb26572700 1 -- 172.21.2.101:0/1991112963 >> v2:172.21.2.108:3300/0 conn(0x7fbb281b0ae0 msgr2=0x7fbb28108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:54.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.897 7fbb26572700 1 --2- 172.21.2.101:0/1991112963 >> v2:172.21.2.108:3300/0 conn(0x7fbb281b0ae0 0x7fbb28108940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:54.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.897 7fbb26572700 1 -- 172.21.2.101:0/1991112963 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb28088070 con 0x7fbb280d1c70 2021-02-10T11:39:54.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.897 7fbb26572700 1 -- 172.21.2.101:0/1991112963 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fbb20001a00 con 0x7fbb280d1c70 2021-02-10T11:39:54.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.897 7fbb26572700 1 -- 172.21.2.101:0/1991112963 >> v1:172.21.2.101:6789/0 conn(0x7fbb280d1c70 legacy=0x7fbb28108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:54.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.897 7fbb26572700 1 --2- 172.21.2.101:0/1991112963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb10003a40 0x7fbb10005e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:54.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.897 7fbb26572700 1 --2- 172.21.2.101:0/1991112963 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb1004a740 0x7fbb1004cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:54.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.897 7fbb26572700 1 --2- 172.21.2.101:0/1991112963 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb1004d500 0x7fbb1004fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:54.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.897 7fbb26572700 1 -- 172.21.2.101:0/1991112963 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb100502d0 con 0x7fbb1004a740 2021-02-10T11:39:54.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.897 7fbb26572700 1 -- 172.21.2.101:0/1991112963 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb100504f0 con 0x7fbb1004d500 2021-02-10T11:39:54.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.897 7fbb26572700 1 -- 172.21.2.101:0/1991112963 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb10050710 con 0x7fbb10003a40 2021-02-10T11:39:54.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.897 7fbb26d73700 1 --2- 172.21.2.101:0/1991112963 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb1004a740 0x7fbb1004cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:54.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.897 7fbb27574700 1 --2- 172.21.2.101:0/1991112963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb10003a40 0x7fbb10005e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:54.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.897 7fbb27d75700 1 --2- 172.21.2.101:0/1991112963 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb1004d500 0x7fbb1004fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:54.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.898 7fbb26d73700 1 -- 172.21.2.101:0/1991112963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb10003a40 msgr2=0x7fbb10005e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:54.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.898 7fbb26d73700 1 --2- 172.21.2.101:0/1991112963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb10003a40 0x7fbb10005e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:54.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.898 7fbb26d73700 1 -- 172.21.2.101:0/1991112963 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb1004d500 msgr2=0x7fbb1004fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:54.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.898 7fbb26d73700 1 --2- 172.21.2.101:0/1991112963 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb1004d500 0x7fbb1004fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:54.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.898 7fbb26d73700 1 -- 172.21.2.101:0/1991112963 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb100509b0 con 0x7fbb1004a740 2021-02-10T11:39:54.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.898 7fbb27d75700 1 --2- 172.21.2.101:0/1991112963 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb1004d500 0x7fbb1004fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:39:54.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.898 7fbb26d73700 1 --2- 172.21.2.101:0/1991112963 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb1004a740 0x7fbb1004cc60 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 rx=0x7fbb1c00c8f0 tx=0x7fbb1c00cc90).ready entity=mon.0 client_cookie=fa33562197ee1e9c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:54.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.898 7fbb26572700 1 -- 172.21.2.101:0/1991112963 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbb1c00cdd0 con 0x7fbb1004a740 2021-02-10T11:39:54.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.898 7fbb26572700 1 -- 172.21.2.101:0/1991112963 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1c009310 con 0x7fbb1004a740 2021-02-10T11:39:54.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.898 7fbb26572700 1 -- 172.21.2.101:0/1991112963 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbb1c009830 con 0x7fbb1004a740 2021-02-10T11:39:54.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.898 7fbb2d8f3700 1 -- 172.21.2.101:0/1991112963 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb1004a740 msgr2=0x7fbb1004cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:54.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.898 7fbb2d8f3700 1 --2- 172.21.2.101:0/1991112963 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb1004a740 0x7fbb1004cc60 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 rx=0x7fbb1c00c8f0 tx=0x7fbb1c00cc90).stop 2021-02-10T11:39:54.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.898 7fbb27d75700 1 -- 172.21.2.101:0/1991112963 reap_dead start 2021-02-10T11:39:54.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.898 7fbb2d8f3700 1 -- 172.21.2.101:0/1991112963 shutdown_connections 2021-02-10T11:39:54.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.898 7fbb2d8f3700 1 -- 172.21.2.101:0/1991112963 >> 172.21.2.101:0/1991112963 conn(0x7fbb280ce420 msgr2=0x7fbb280d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:54.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.899 7fbb2d8f3700 1 -- 172.21.2.101:0/1991112963 shutdown_connections 2021-02-10T11:39:54.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.899 7fbb2d8f3700 1 -- 172.21.2.101:0/1991112963 wait complete. 2021-02-10T11:39:54.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.913 7fbb2d8f3700 1 Processor -- start 2021-02-10T11:39:54.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.913 7fbb2d8f3700 1 -- start start 2021-02-10T11:39:54.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.914 7fbb2d8f3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb2809bcc0 0x7fbb2809c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:54.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.914 7fbb2d8f3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb280a0820 0x7fbb280a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:54.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.914 7fbb2d8f3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb280a1240 0x7fbb280858d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:54.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.914 7fbb2d8f3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb280a66b0 con 0x7fbb280a0820 2021-02-10T11:39:54.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.914 7fbb2d8f3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb281a49b0 con 0x7fbb280a1240 2021-02-10T11:39:54.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.914 7fbb2d8f3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb281a4b90 con 0x7fbb2809bcc0 2021-02-10T11:39:54.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.914 7fbb26d73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb280a0820 0x7fbb280a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:54.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.914 7fbb26d73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb280a0820 0x7fbb280a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56020/0 (socket says 172.21.2.101:56020) 2021-02-10T11:39:54.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.914 7fbb26d73700 1 -- 172.21.2.101:0/335642380 learned_addr learned my addr 172.21.2.101:0/335642380 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:54.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.914 7fbb26d73700 1 -- 172.21.2.101:0/335642380 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb2809bcc0 msgr2=0x7fbb2809c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:54.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.914 7fbb26d73700 1 --2- 172.21.2.101:0/335642380 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb2809bcc0 0x7fbb2809c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:54.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.914 7fbb26d73700 1 -- 172.21.2.101:0/335642380 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb280a1240 msgr2=0x7fbb280858d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:54.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.914 7fbb26d73700 1 --2- 172.21.2.101:0/335642380 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb280a1240 0x7fbb280858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:54.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.914 7fbb26d73700 1 -- 172.21.2.101:0/335642380 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb280a1640 con 0x7fbb280a0820 2021-02-10T11:39:54.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.915 7fbb26d73700 1 --2- 172.21.2.101:0/335642380 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb280a0820 0x7fbb280a0c20 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 rx=0x7fbb1c009e30 tx=0x7fbb1c00b300).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:54.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.915 7fbb0ffff700 1 -- 172.21.2.101:0/335642380 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbb1c00c870 con 0x7fbb280a0820 2021-02-10T11:39:54.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.915 7fbb0ffff700 1 -- 172.21.2.101:0/335642380 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1c00cbb0 con 0x7fbb280a0820 2021-02-10T11:39:54.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.915 7fbb0ffff700 1 -- 172.21.2.101:0/335642380 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbb1c00b970 con 0x7fbb280a0820 2021-02-10T11:39:54.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.915 7fbb2d8f3700 1 -- 172.21.2.101:0/335642380 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb280a1640 con 0x7fbb280a0820 2021-02-10T11:39:54.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.915 7fbb2d8f3700 1 -- 172.21.2.101:0/335642380 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb280a1640 con 0x7fbb280a0820 2021-02-10T11:39:54.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.916 7fbb0ffff700 1 -- 172.21.2.101:0/335642380 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbb1c000c90 con 0x7fbb280a0820 2021-02-10T11:39:54.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.916 7fbb0ffff700 1 --2- 172.21.2.101:0/335642380 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbb080186c0 0x7fbb0801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:54.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.916 7fbb0ffff700 1 -- 172.21.2.101:0/335642380 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5107+0+0 (secure 0 0 0) 0x7fbb1c032400 con 0x7fbb280a0820 2021-02-10T11:39:54.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.916 7fbb27574700 1 --2- 172.21.2.101:0/335642380 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbb080186c0 0x7fbb0801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:54.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.916 7fbb27574700 1 --2- 172.21.2.101:0/335642380 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbb080186c0 0x7fbb0801abc0 crc :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:54.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.946 7fbb2d8f3700 1 -- 172.21.2.101:0/335642380 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb280fe1c0 con 0x7fbb280a0820 2021-02-10T11:39:54.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:54.948 7fbb0ffff700 1 -- 172.21.2.101:0/335642380 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbb1c001f00 con 0x7fbb280a0820 2021-02-10T11:39:55.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.077 7fbb2d8f3700 1 -- 172.21.2.101:0/335642380 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"} v 0) v1 -- 0x7fbb280fe1c0 con 0x7fbb280a0820 2021-02-10T11:39:55.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.436 7fbb0ffff700 1 -- 172.21.2.101:0/335642380 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"}]=0 pool 'cephfs_data' created v101) v1 ==== 125+0+0 (secure 0 0 0) 0x7fbb1c000c90 con 0x7fbb280a0820 2021-02-10T11:39:55.444 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' created 2021-02-10T11:39:55.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.443 7fbb2d8f3700 1 -- 172.21.2.101:0/335642380 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbb080186c0 msgr2=0x7fbb0801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:55.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.443 7fbb2d8f3700 1 --2- 172.21.2.101:0/335642380 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbb080186c0 0x7fbb0801abc0 crc :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:55.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.443 7fbb2d8f3700 1 -- 172.21.2.101:0/335642380 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb280a0820 msgr2=0x7fbb280a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:55.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.443 7fbb2d8f3700 1 --2- 172.21.2.101:0/335642380 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb280a0820 0x7fbb280a0c20 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 rx=0x7fbb1c009e30 tx=0x7fbb1c00b300).stop 2021-02-10T11:39:55.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.443 7fbb2d8f3700 1 -- 172.21.2.101:0/335642380 shutdown_connections 2021-02-10T11:39:55.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.443 7fbb2d8f3700 1 --2- 172.21.2.101:0/335642380 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb280a1240 0x7fbb280858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:55.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.443 7fbb2d8f3700 1 --2- 172.21.2.101:0/335642380 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb280a0820 0x7fbb280a0c20 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:55.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.443 7fbb2d8f3700 1 --2- 172.21.2.101:0/335642380 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbb080186c0 0x7fbb0801abc0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:55.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.443 7fbb2d8f3700 1 --2- 172.21.2.101:0/335642380 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb2809bcc0 0x7fbb2809c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:55.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.443 7fbb2d8f3700 1 -- 172.21.2.101:0/335642380 >> 172.21.2.101:0/335642380 conn(0x7fbb280bef00 msgr2=0x7fbb2809d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:55.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.443 7fbb2d8f3700 1 -- 172.21.2.101:0/335642380 shutdown_connections 2021-02-10T11:39:55.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.443 7fbb2d8f3700 1 -- 172.21.2.101:0/335642380 wait complete. 2021-02-10T11:39:55.454 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-02-10T11:39:55.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.866 7f0d6b61e700 1 Processor -- start 2021-02-10T11:39:55.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.866 7f0d6b61e700 1 -- start start 2021-02-10T11:39:55.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.866 7f0d6b61e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0d640d1c80 0x7f0d640cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:55.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.866 7f0d6b61e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d640d0180 con 0x7f0d640d1880 2021-02-10T11:39:55.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.866 7f0d6b61e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d640fe1d0 con 0x7f0d640cf590 2021-02-10T11:39:55.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.866 7f0d6b61e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d641b0cc0 con 0x7f0d640d1c80 2021-02-10T11:39:55.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.866 7f0d68c39700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0d640d1c80 0x7f0d640cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:55.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.866 7f0d6943a700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f0d640d1880 0x7f0d641b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56522/0 (socket says 172.21.2.101:56522) 2021-02-10T11:39:55.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.866 7f0d6943a700 1 -- 172.21.2.101:0/2761953635 learned_addr learned my addr 172.21.2.101:0/2761953635 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:55.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.867 7f0d57fff700 1 -- 172.21.2.101:0/2761953635 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 697419293 0 0) 0x7f0d580009b0 con 0x7f0d640d1880 2021-02-10T11:39:55.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.867 7f0d57fff700 1 -- 172.21.2.101:0/2761953635 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d50002fe0 con 0x7f0d640d1880 2021-02-10T11:39:55.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.867 7f0d68c39700 1 -- 172.21.2.101:0/2761953635 >> v1:172.21.2.108:6789/0 conn(0x7f0d640cf590 legacy=0x7f0d640cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:55.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.867 7f0d68c39700 1 -- 172.21.2.101:0/2761953635 >> v1:172.21.2.101:6789/0 conn(0x7f0d640d1880 legacy=0x7f0d641b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:55.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.867 7f0d68c39700 1 -- 172.21.2.101:0/2761953635 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d64109a70 con 0x7f0d640d1c80 2021-02-10T11:39:55.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.867 7f0d68c39700 1 --2- 172.21.2.101:0/2761953635 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d640d1c80 0x7f0d640cef70 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 rx=0x7f0d60006650 tx=0x7f0d60009a80).ready entity=mon.0 client_cookie=861e825761868743 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:55.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.867 7f0d57fff700 1 -- 172.21.2.101:0/2761953635 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0d6000c0c0 con 0x7f0d640d1c80 2021-02-10T11:39:55.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.867 7f0d57fff700 1 -- 172.21.2.101:0/2761953635 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6000c340 con 0x7f0d640d1c80 2021-02-10T11:39:55.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.867 7f0d57fff700 1 -- 172.21.2.101:0/2761953635 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0d6000c860 con 0x7f0d640d1c80 2021-02-10T11:39:55.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.867 7f0d6b61e700 1 -- 172.21.2.101:0/2761953635 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d640d1c80 msgr2=0x7f0d640cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:55.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.867 7f0d6b61e700 1 --2- 172.21.2.101:0/2761953635 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d640d1c80 0x7f0d640cef70 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 rx=0x7f0d60006650 tx=0x7f0d60009a80).stop 2021-02-10T11:39:55.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.868 7f0d6b61e700 1 -- 172.21.2.101:0/2761953635 shutdown_connections 2021-02-10T11:39:55.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.868 7f0d6b61e700 1 --2- 172.21.2.101:0/2761953635 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d640d1c80 0x7f0d640cef70 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:55.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.868 7f0d6b61e700 1 -- 172.21.2.101:0/2761953635 >> 172.21.2.101:0/2761953635 conn(0x7f0d640ce430 msgr2=0x7f0d640d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:55.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.868 7f0d6b61e700 1 -- 172.21.2.101:0/2761953635 shutdown_connections 2021-02-10T11:39:55.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.868 7f0d6b61e700 1 -- 172.21.2.101:0/2761953635 wait complete. 2021-02-10T11:39:55.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.883 7f0d6b61e700 1 Processor -- start 2021-02-10T11:39:55.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.883 7f0d6b61e700 1 -- start start 2021-02-10T11:39:55.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.883 7f0d6b61e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d641a9100 0x7f0d641ab830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:55.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.883 7f0d6b61e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0d640b81b0 0x7f0d640b85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:55.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.883 7f0d6b61e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0d64191da0 0x7f0d641921a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:55.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.883 7f0d6b61e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d64192900 con 0x7f0d641a9100 2021-02-10T11:39:55.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.883 7f0d6b61e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d641a9540 con 0x7f0d64191da0 2021-02-10T11:39:55.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.883 7f0d6b61e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d64094900 con 0x7f0d640b81b0 2021-02-10T11:39:55.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.883 7f0d6943a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d641a9100 0x7f0d641ab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:55.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.883 7f0d6943a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d641a9100 0x7f0d641ab830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56034/0 (socket says 172.21.2.101:56034) 2021-02-10T11:39:55.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.883 7f0d6943a700 1 -- 172.21.2.101:0/1515833512 learned_addr learned my addr 172.21.2.101:0/1515833512 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:55.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.884 7f0d6943a700 1 -- 172.21.2.101:0/1515833512 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0d640b81b0 msgr2=0x7f0d640b85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:55.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.884 7f0d6943a700 1 --2- 172.21.2.101:0/1515833512 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0d640b81b0 0x7f0d640b85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:55.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.884 7f0d6943a700 1 -- 172.21.2.101:0/1515833512 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0d64191da0 msgr2=0x7f0d641921a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:55.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.884 7f0d6943a700 1 --2- 172.21.2.101:0/1515833512 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0d64191da0 0x7f0d641921a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:55.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.884 7f0d6943a700 1 -- 172.21.2.101:0/1515833512 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d64192e00 con 0x7f0d641a9100 2021-02-10T11:39:55.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.884 7f0d6943a700 1 --2- 172.21.2.101:0/1515833512 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d641a9100 0x7f0d641ab830 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 rx=0x7f0d58001c60 tx=0x7f0d5800a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:55.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.884 7f0d55ffb700 1 -- 172.21.2.101:0/1515833512 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0d5800acc0 con 0x7f0d641a9100 2021-02-10T11:39:55.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.884 7f0d55ffb700 1 -- 172.21.2.101:0/1515833512 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d58004140 con 0x7f0d641a9100 2021-02-10T11:39:55.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.884 7f0d55ffb700 1 -- 172.21.2.101:0/1515833512 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0d58004660 con 0x7f0d641a9100 2021-02-10T11:39:55.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.884 7f0d6b61e700 1 -- 172.21.2.101:0/1515833512 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d64192e00 con 0x7f0d641a9100 2021-02-10T11:39:55.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.884 7f0d6b61e700 1 -- 172.21.2.101:0/1515833512 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d64193390 con 0x7f0d641a9100 2021-02-10T11:39:55.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.885 7f0d55ffb700 1 -- 172.21.2.101:0/1515833512 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0d58004140 con 0x7f0d641a9100 2021-02-10T11:39:55.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.885 7f0d55ffb700 1 --2- 172.21.2.101:0/1515833512 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0d440186c0 0x7f0d4401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:55.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.885 7f0d55ffb700 1 -- 172.21.2.101:0/1515833512 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5457+0+0 (secure 0 0 0) 0x7f0d580324d0 con 0x7f0d641a9100 2021-02-10T11:39:55.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.885 7f0d68c39700 1 --2- 172.21.2.101:0/1515833512 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0d440186c0 0x7f0d4401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:55.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.886 7f0d68c39700 1 --2- 172.21.2.101:0/1515833512 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0d440186c0 0x7f0d4401abc0 crc :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:55.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.916 7f0d6b61e700 1 -- 172.21.2.101:0/1515833512 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d64082ab0 con 0x7f0d641a9100 2021-02-10T11:39:55.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:55.919 7f0d55ffb700 1 -- 172.21.2.101:0/1515833512 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0d58000940 con 0x7f0d641a9100 2021-02-10T11:39:56.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.053 7f0d6b61e700 1 -- 172.21.2.101:0/1515833512 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"} v 0) v1 -- 0x7f0d64082ab0 con 0x7f0d641a9100 2021-02-10T11:39:56.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.057 7f0d55ffb700 1 -- 172.21.2.101:0/1515833512 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"}]=0 new fs with metadata pool 20 and data pool 21 v77) v1 ==== 189+0+0 (secure 0 0 0) 0x7f0d5801f280 con 0x7f0d641a9100 2021-02-10T11:39:56.063 INFO:teuthology.orchestra.run.gibba001.stderr:new fs with metadata pool 20 and data pool 21 2021-02-10T11:39:56.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.063 7f0d6b61e700 1 -- 172.21.2.101:0/1515833512 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0d440186c0 msgr2=0x7f0d4401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:56.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.063 7f0d6b61e700 1 --2- 172.21.2.101:0/1515833512 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0d440186c0 0x7f0d4401abc0 crc :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:56.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.063 7f0d6b61e700 1 -- 172.21.2.101:0/1515833512 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d641a9100 msgr2=0x7f0d641ab830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:56.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.063 7f0d6b61e700 1 --2- 172.21.2.101:0/1515833512 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d641a9100 0x7f0d641ab830 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 rx=0x7f0d58001c60 tx=0x7f0d5800a8d0).stop 2021-02-10T11:39:56.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.064 7f0d6b61e700 1 -- 172.21.2.101:0/1515833512 shutdown_connections 2021-02-10T11:39:56.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.064 7f0d6b61e700 1 --2- 172.21.2.101:0/1515833512 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0d440186c0 0x7f0d4401abc0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:56.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.064 7f0d6b61e700 1 --2- 172.21.2.101:0/1515833512 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0d640b81b0 0x7f0d640b85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:56.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.064 7f0d6b61e700 1 --2- 172.21.2.101:0/1515833512 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0d64191da0 0x7f0d641921a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:56.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.064 7f0d6b61e700 1 --2- 172.21.2.101:0/1515833512 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d641a9100 0x7f0d641ab830 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:56.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.064 7f0d6b61e700 1 -- 172.21.2.101:0/1515833512 >> 172.21.2.101:0/1515833512 conn(0x7f0d6409afc0 msgr2=0x7f0d640953e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:56.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.064 7f0d6b61e700 1 -- 172.21.2.101:0/1515833512 shutdown_connections 2021-02-10T11:39:56.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.064 7f0d6b61e700 1 -- 172.21.2.101:0/1515833512 wait complete. 2021-02-10T11:39:56.074 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:39:56.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.482 7f99dfb48700 1 Processor -- start 2021-02-10T11:39:56.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.482 7f99dfb48700 1 -- start start 2021-02-10T11:39:56.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.482 7f99dfb48700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f99d80d1870 0x7f99d81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:56.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.482 7f99dfb48700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f99d80f0650 0x7f99d80cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:56.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.482 7f99dfb48700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99d80f0b10 con 0x7f99d81b0a10 2021-02-10T11:39:56.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.482 7f99dfb48700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d80e5ed0 con 0x7f99d80f0650 2021-02-10T11:39:56.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.482 7f99dfb48700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d80d1c70 con 0x7f99d80d1870 2021-02-10T11:39:56.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.483 7f99dd163700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f99d81b0a10 0x7f99d80cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56536/0 (socket says 172.21.2.101:56536) 2021-02-10T11:39:56.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.483 7f99dd163700 1 -- 172.21.2.101:0/3996238311 learned_addr learned my addr 172.21.2.101:0/3996238311 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:56.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.483 7f99de165700 1 --2- 172.21.2.101:0/3996238311 >> v2:172.21.2.101:3300/0 conn(0x7f99d80f0650 0x7f99d80cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:56.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.483 7f99de165700 1 -- 172.21.2.101:0/3996238311 >> v2:172.21.2.108:3300/0 conn(0x7f99d80d1870 msgr2=0x7f99d81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:56.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.483 7f99de165700 1 --2- 172.21.2.101:0/3996238311 >> v2:172.21.2.108:3300/0 conn(0x7f99d80d1870 0x7f99d81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:56.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.483 7f99de165700 1 -- 172.21.2.101:0/3996238311 >> v1:172.21.2.101:6789/0 conn(0x7f99d81b0a10 legacy=0x7f99d80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:56.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.483 7f99dc962700 1 -- 172.21.2.101:0/3996238311 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2217458779 0 0) 0x7f99c80009b0 con 0x7f99d81b0a10 2021-02-10T11:39:56.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.483 7f99de165700 1 -- 172.21.2.101:0/3996238311 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d80d1e90 con 0x7f99d80f0650 2021-02-10T11:39:56.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.483 7f99de165700 1 --2- 172.21.2.101:0/3996238311 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99d80f0650 0x7f99d80cfb20 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 rx=0x7f99d0006e30 tx=0x7f99d0009de0).ready entity=mon.0 client_cookie=8a5c70774c47c0dd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:56.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.484 7f99dc962700 1 -- 172.21.2.101:0/3996238311 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f99d000b300 con 0x7f99d80f0650 2021-02-10T11:39:56.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.484 7f99dc962700 1 -- 172.21.2.101:0/3996238311 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d000b680 con 0x7f99d80f0650 2021-02-10T11:39:56.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.484 7f99dc962700 1 -- 172.21.2.101:0/3996238311 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f99d000bba0 con 0x7f99d80f0650 2021-02-10T11:39:56.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.484 7f99dfb48700 1 -- 172.21.2.101:0/3996238311 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99d80f0650 msgr2=0x7f99d80cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:56.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.484 7f99dfb48700 1 --2- 172.21.2.101:0/3996238311 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99d80f0650 0x7f99d80cfb20 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 rx=0x7f99d0006e30 tx=0x7f99d0009de0).stop 2021-02-10T11:39:56.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.485 7f99dfb48700 1 -- 172.21.2.101:0/3996238311 shutdown_connections 2021-02-10T11:39:56.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.485 7f99dfb48700 1 --2- 172.21.2.101:0/3996238311 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99d80f0650 0x7f99d80cfb20 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:56.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.485 7f99dfb48700 1 --2- 172.21.2.101:0/3996238311 >> v2:172.21.2.108:3300/0 conn(0x7f99d80d1870 0x7f99d81b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:56.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.485 7f99dfb48700 1 -- 172.21.2.101:0/3996238311 >> 172.21.2.101:0/3996238311 conn(0x7f99d80ce450 msgr2=0x7f99d80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:56.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.485 7f99dfb48700 1 -- 172.21.2.101:0/3996238311 shutdown_connections 2021-02-10T11:39:56.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.485 7f99dfb48700 1 -- 172.21.2.101:0/3996238311 wait complete. 2021-02-10T11:39:56.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.499 7f99dfb48700 1 Processor -- start 2021-02-10T11:39:56.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.499 7f99dfb48700 1 -- start start 2021-02-10T11:39:56.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.500 7f99dfb48700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99d81b8ad0 0x7f99d8187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:56.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.500 7f99dfb48700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99d81b8ed0 0x7f99d80b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:56.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.500 7f99dfb48700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99d80b2110 0x7f99d81b7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:56.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.500 7f99dfb48700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d8188470 con 0x7f99d81b8ad0 2021-02-10T11:39:56.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.500 7f99dfb48700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d818be10 con 0x7f99d80b2110 2021-02-10T11:39:56.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.500 7f99dfb48700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d818c030 con 0x7f99d81b8ed0 2021-02-10T11:39:56.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.500 7f99dd964700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99d81b8ad0 0x7f99d8187e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:56.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.500 7f99dd964700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99d81b8ad0 0x7f99d8187e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56050/0 (socket says 172.21.2.101:56050) 2021-02-10T11:39:56.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.500 7f99dd964700 1 -- 172.21.2.101:0/1989876959 learned_addr learned my addr 172.21.2.101:0/1989876959 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:56.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.500 7f99dd964700 1 -- 172.21.2.101:0/1989876959 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99d81b8ed0 msgr2=0x7f99d80b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:56.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.500 7f99dd964700 1 --2- 172.21.2.101:0/1989876959 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99d81b8ed0 0x7f99d80b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:56.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.500 7f99dd964700 1 -- 172.21.2.101:0/1989876959 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99d80b2110 msgr2=0x7f99d81b7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:56.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.500 7f99dd964700 1 --2- 172.21.2.101:0/1989876959 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99d80b2110 0x7f99d81b7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:56.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.500 7f99dd964700 1 -- 172.21.2.101:0/1989876959 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d80b2510 con 0x7f99d81b8ad0 2021-02-10T11:39:56.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.500 7f99dd964700 1 --2- 172.21.2.101:0/1989876959 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99d81b8ad0 0x7f99d8187e50 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 rx=0x7f99d4006f00 tx=0x7f99d400b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:56.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.501 7f99ce7fc700 1 -- 172.21.2.101:0/1989876959 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f99d40009b0 con 0x7f99d81b8ad0 2021-02-10T11:39:56.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.501 7f99ce7fc700 1 -- 172.21.2.101:0/1989876959 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d40063d0 con 0x7f99d81b8ad0 2021-02-10T11:39:56.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.501 7f99ce7fc700 1 -- 172.21.2.101:0/1989876959 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f99d4000c90 con 0x7f99d81b8ad0 2021-02-10T11:39:56.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.501 7f99dfb48700 1 -- 172.21.2.101:0/1989876959 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99d80b2510 con 0x7f99d81b8ad0 2021-02-10T11:39:56.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.501 7f99dfb48700 1 -- 172.21.2.101:0/1989876959 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d80b2510 con 0x7f99d81b8ad0 2021-02-10T11:39:56.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.501 7f99ce7fc700 1 -- 172.21.2.101:0/1989876959 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f99d4007000 con 0x7f99d81b8ad0 2021-02-10T11:39:56.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.501 7f99ce7fc700 1 --2- 172.21.2.101:0/1989876959 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f99c00186c0 0x7f99c001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:56.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.501 7f99ce7fc700 1 -- 172.21.2.101:0/1989876959 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f99d4032510 con 0x7f99d81b8ad0 2021-02-10T11:39:56.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.501 7f99dd163700 1 --2- 172.21.2.101:0/1989876959 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f99c00186c0 0x7f99c001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:56.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.502 7f99dd163700 1 --2- 172.21.2.101:0/1989876959 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f99c00186c0 0x7f99c001abc0 crc :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:56.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.532 7f99dfb48700 1 -- 172.21.2.101:0/1989876959 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99d80ebc00 con 0x7f99d81b8ad0 2021-02-10T11:39:56.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.535 7f99ce7fc700 1 -- 172.21.2.101:0/1989876959 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f99d4006b90 con 0x7f99d81b8ad0 2021-02-10T11:39:56.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.659 7f99dfb48700 1 -- 172.21.2.101:0/1989876959 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99d80ebc00 con 0x7f99d81b8ad0 2021-02-10T11:39:56.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.660 7f99ce7fc700 1 -- 172.21.2.101:0/1989876959 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+14018 (secure 0 0 0) 0x7f99d4007000 con 0x7f99d81b8ad0 2021-02-10T11:39:56.662 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:39:56.662 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":102,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:39:56.054050","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:39:54.022810","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2021-02-10 11:39:55.095134","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:39:56.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.661 7f99dfb48700 1 -- 172.21.2.101:0/1989876959 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f99c00186c0 msgr2=0x7f99c001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:56.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.661 7f99dfb48700 1 --2- 172.21.2.101:0/1989876959 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f99c00186c0 0x7f99c001abc0 crc :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:56.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.661 7f99dfb48700 1 -- 172.21.2.101:0/1989876959 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99d81b8ad0 msgr2=0x7f99d8187e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:56.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.661 7f99dfb48700 1 --2- 172.21.2.101:0/1989876959 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99d81b8ad0 0x7f99d8187e50 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 rx=0x7f99d4006f00 tx=0x7f99d400b940).stop 2021-02-10T11:39:56.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.661 7f99dfb48700 1 -- 172.21.2.101:0/1989876959 shutdown_connections 2021-02-10T11:39:56.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.661 7f99dfb48700 1 --2- 172.21.2.101:0/1989876959 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f99c00186c0 0x7f99c001abc0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:56.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.662 7f99dfb48700 1 --2- 172.21.2.101:0/1989876959 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99d81b8ed0 0x7f99d80b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:56.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.662 7f99dfb48700 1 --2- 172.21.2.101:0/1989876959 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99d80b2110 0x7f99d81b7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:56.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.662 7f99dfb48700 1 --2- 172.21.2.101:0/1989876959 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99d81b8ad0 0x7f99d8187e50 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:56.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.662 7f99dfb48700 1 -- 172.21.2.101:0/1989876959 >> 172.21.2.101:0/1989876959 conn(0x7f99d80ed810 msgr2=0x7f99d8094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:56.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.662 7f99dfb48700 1 -- 172.21.2.101:0/1989876959 shutdown_connections 2021-02-10T11:39:56.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:56.662 7f99dfb48700 1 -- 172.21.2.101:0/1989876959 wait complete. 2021-02-10T11:39:56.673 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:39:57.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.080 7f910cf4b700 1 Processor -- start 2021-02-10T11:39:57.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.081 7f910cf4b700 1 -- start start 2021-02-10T11:39:57.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.081 7f910cf4b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91081b0470 0x7f91081b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:57.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.081 7f910cf4b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91080f0b50 con 0x7f91080f20e0 2021-02-10T11:39:57.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.081 7f910cf4b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91080f24e0 con 0x7f91080ef610 2021-02-10T11:39:57.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.081 7f910cf4b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91080e5ea0 con 0x7f91081b0470 2021-02-10T11:39:57.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.081 7f9107314700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f91080f20e0 0x7f91080f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56548/0 (socket says 172.21.2.101:56548) 2021-02-10T11:39:57.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.081 7f9107314700 1 -- 172.21.2.101:0/1626886232 learned_addr learned my addr 172.21.2.101:0/1626886232 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:57.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.081 7f9106b13700 1 --2- 172.21.2.101:0/1626886232 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91081b0470 0x7f91081b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:57.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.082 7f9105b11700 1 -- 172.21.2.101:0/1626886232 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 616941691 0 0) 0x7f9100001500 con 0x7f91080f20e0 2021-02-10T11:39:57.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.082 7f9105b11700 1 -- 172.21.2.101:0/1626886232 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90f4002fe0 con 0x7f91080f20e0 2021-02-10T11:39:57.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.082 7f9105b11700 1 -- 172.21.2.101:0/1626886232 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 423339351 0 0) 0x7f90f8000ff0 con 0x7f91080ef610 2021-02-10T11:39:57.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.082 7f9106b13700 1 -- 172.21.2.101:0/1626886232 >> v1:172.21.2.108:6789/0 conn(0x7f91080ef610 legacy=0x7f91080efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:57.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.082 7f9106b13700 1 -- 172.21.2.101:0/1626886232 >> v1:172.21.2.101:6789/0 conn(0x7f91080f20e0 legacy=0x7f91080f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:57.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.082 7f9106b13700 1 -- 172.21.2.101:0/1626886232 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91080e52e0 con 0x7f91081b0470 2021-02-10T11:39:57.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.082 7f9106b13700 1 --2- 172.21.2.101:0/1626886232 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91081b0470 0x7f91081b0870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f90f0006650 tx=0x7f90f0009a80).ready entity=mon.2 client_cookie=7b8c20d46b5001e6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:57.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.082 7f9105b11700 1 -- 172.21.2.101:0/1626886232 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f90f000c0c0 con 0x7f91081b0470 2021-02-10T11:39:57.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.082 7f9105b11700 1 -- 172.21.2.101:0/1626886232 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f000c340 con 0x7f91081b0470 2021-02-10T11:39:57.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.082 7f9105b11700 1 -- 172.21.2.101:0/1626886232 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f90f000c860 con 0x7f91081b0470 2021-02-10T11:39:57.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.082 7f910cf4b700 1 -- 172.21.2.101:0/1626886232 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91081b0470 msgr2=0x7f91081b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:57.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.082 7f910cf4b700 1 --2- 172.21.2.101:0/1626886232 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91081b0470 0x7f91081b0870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f90f0006650 tx=0x7f90f0009a80).stop 2021-02-10T11:39:57.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.083 7f910cf4b700 1 -- 172.21.2.101:0/1626886232 shutdown_connections 2021-02-10T11:39:57.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.083 7f910cf4b700 1 --2- 172.21.2.101:0/1626886232 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91081b0470 0x7f91081b0870 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:57.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.083 7f910cf4b700 1 -- 172.21.2.101:0/1626886232 >> 172.21.2.101:0/1626886232 conn(0x7f91080ce3e0 msgr2=0x7f91080e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:57.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.083 7f910cf4b700 1 -- 172.21.2.101:0/1626886232 shutdown_connections 2021-02-10T11:39:57.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.083 7f910cf4b700 1 -- 172.21.2.101:0/1626886232 wait complete. 2021-02-10T11:39:57.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.098 7f910cf4b700 1 Processor -- start 2021-02-10T11:39:57.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.098 7f910cf4b700 1 -- start start 2021-02-10T11:39:57.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.098 7f910cf4b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f91080aa8d0 0x7f910819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:57.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.098 7f910cf4b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f910819e9d0 0x7f910819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:57.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.098 7f910cf4b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f910819ad40 0x7f910819b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:57.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.098 7f910cf4b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910819b1f0 con 0x7f910819e9d0 2021-02-10T11:39:57.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.098 7f910cf4b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9108109b30 con 0x7f91080aa8d0 2021-02-10T11:39:57.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.098 7f910cf4b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9108099620 con 0x7f910819ad40 2021-02-10T11:39:57.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.098 7f9106312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f910819e9d0 0x7f910819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:57.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.098 7f9106312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f910819e9d0 0x7f910819a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56058/0 (socket says 172.21.2.101:56058) 2021-02-10T11:39:57.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.098 7f9106312700 1 -- 172.21.2.101:0/4182851666 learned_addr learned my addr 172.21.2.101:0/4182851666 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:57.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.099 7f9106312700 1 -- 172.21.2.101:0/4182851666 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f910819ad40 msgr2=0x7f910819b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:57.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.099 7f9106312700 1 --2- 172.21.2.101:0/4182851666 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f910819ad40 0x7f910819b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:57.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.099 7f9106312700 1 -- 172.21.2.101:0/4182851666 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f91080aa8d0 msgr2=0x7f910819a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:57.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.099 7f9106312700 1 --2- 172.21.2.101:0/4182851666 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f91080aa8d0 0x7f910819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:57.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.099 7f9106312700 1 -- 172.21.2.101:0/4182851666 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9108109640 con 0x7f910819e9d0 2021-02-10T11:39:57.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.099 7f9106312700 1 --2- 172.21.2.101:0/4182851666 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f910819e9d0 0x7f910819a720 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 rx=0x7f90f800a4d0 tx=0x7f90f800a870).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:57.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.099 7f90ff7fe700 1 -- 172.21.2.101:0/4182851666 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f90f800c0d0 con 0x7f910819e9d0 2021-02-10T11:39:57.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.099 7f90ff7fe700 1 -- 172.21.2.101:0/4182851666 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f800c350 con 0x7f910819e9d0 2021-02-10T11:39:57.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.099 7f90ff7fe700 1 -- 172.21.2.101:0/4182851666 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f90f800c870 con 0x7f910819e9d0 2021-02-10T11:39:57.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.099 7f910cf4b700 1 -- 172.21.2.101:0/4182851666 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9108109640 con 0x7f910819e9d0 2021-02-10T11:39:57.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.099 7f910cf4b700 1 -- 172.21.2.101:0/4182851666 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910819c080 con 0x7f910819e9d0 2021-02-10T11:39:57.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.100 7f90ff7fe700 1 -- 172.21.2.101:0/4182851666 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f90f8008030 con 0x7f910819e9d0 2021-02-10T11:39:57.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.100 7f90ff7fe700 1 --2- 172.21.2.101:0/4182851666 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90ec0186c0 0x7f90ec01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:57.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.100 7f90ff7fe700 1 -- 172.21.2.101:0/4182851666 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f90f8032570 con 0x7f910819e9d0 2021-02-10T11:39:57.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.100 7f9106b13700 1 --2- 172.21.2.101:0/4182851666 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90ec0186c0 0x7f90ec01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:57.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.100 7f9106b13700 1 --2- 172.21.2.101:0/4182851666 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90ec0186c0 0x7f90ec01abc0 crc :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:57.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.131 7f910cf4b700 1 -- 172.21.2.101:0/4182851666 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91080f0b50 con 0x7f910819e9d0 2021-02-10T11:39:57.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.133 7f90ff7fe700 1 -- 172.21.2.101:0/4182851666 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f90f8008ee0 con 0x7f910819e9d0 2021-02-10T11:39:57.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.259 7f910cf4b700 1 -- 172.21.2.101:0/4182851666 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91080f0b50 con 0x7f910819e9d0 2021-02-10T11:39:57.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.260 7f90ff7fe700 1 -- 172.21.2.101:0/4182851666 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+14005 (secure 0 0 0) 0x7f90f800ac60 con 0x7f910819e9d0 2021-02-10T11:39:57.263 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:39:57.263 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":103,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:39:57.056338","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:39:54.022810","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2021-02-10 11:39:55.095134","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:39:57.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.262 7f910cf4b700 1 -- 172.21.2.101:0/4182851666 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90ec0186c0 msgr2=0x7f90ec01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:57.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.262 7f910cf4b700 1 --2- 172.21.2.101:0/4182851666 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90ec0186c0 0x7f90ec01abc0 crc :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:57.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.262 7f910cf4b700 1 -- 172.21.2.101:0/4182851666 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f910819e9d0 msgr2=0x7f910819a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:57.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.262 7f910cf4b700 1 --2- 172.21.2.101:0/4182851666 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f910819e9d0 0x7f910819a720 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 rx=0x7f90f800a4d0 tx=0x7f90f800a870).stop 2021-02-10T11:39:57.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.262 7f910cf4b700 1 -- 172.21.2.101:0/4182851666 shutdown_connections 2021-02-10T11:39:57.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.262 7f910cf4b700 1 --2- 172.21.2.101:0/4182851666 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90ec0186c0 0x7f90ec01abc0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:57.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.262 7f910cf4b700 1 --2- 172.21.2.101:0/4182851666 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f910819ad40 0x7f910819b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:57.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.262 7f910cf4b700 1 --2- 172.21.2.101:0/4182851666 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f910819e9d0 0x7f910819a720 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:57.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.262 7f910cf4b700 1 --2- 172.21.2.101:0/4182851666 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f91080aa8d0 0x7f910819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:57.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.262 7f910cf4b700 1 -- 172.21.2.101:0/4182851666 >> 172.21.2.101:0/4182851666 conn(0x7f910819f7b0 msgr2=0x7f910818f380 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:57.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.262 7f910cf4b700 1 -- 172.21.2.101:0/4182851666 shutdown_connections 2021-02-10T11:39:57.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.262 7f910cf4b700 1 -- 172.21.2.101:0/4182851666 wait complete. 2021-02-10T11:39:57.274 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-02-10T11:39:57.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.688 7f9701e84700 1 Processor -- start 2021-02-10T11:39:57.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.688 7f9701e84700 1 -- start start 2021-02-10T11:39:57.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.688 7f9701e84700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f96fc0d1870 0x7f96fc1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:57.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.688 7f9701e84700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96fc0fe1c0 con 0x7f96fc0cff70 2021-02-10T11:39:57.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.688 7f9701e84700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96fc109a60 con 0x7f96fc1b0ae0 2021-02-10T11:39:57.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.688 7f9701e84700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96fc088070 con 0x7f96fc0d1870 2021-02-10T11:39:57.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.690 7f96fbfff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f96fc0cff70 0x7f96fc109130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56558/0 (socket says 172.21.2.101:56558) 2021-02-10T11:39:57.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.690 7f96fbfff700 1 -- 172.21.2.101:0/536819224 learned_addr learned my addr 172.21.2.101:0/536819224 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:57.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.690 7f96fb7fe700 1 --2- 172.21.2.101:0/536819224 >> v2:172.21.2.101:3300/0 conn(0x7f96fc0d1870 0x7f96fc1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:57.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.690 7f96fa7fc700 1 -- 172.21.2.101:0/536819224 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711116869 0 0) 0x7f96f4001500 con 0x7f96fc0cff70 2021-02-10T11:39:57.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.690 7f96fb7fe700 1 -- 172.21.2.101:0/536819224 >> v1:172.21.2.108:6789/0 conn(0x7f96fc1b0ae0 legacy=0x7f96fc108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:57.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.690 7f96fb7fe700 1 -- 172.21.2.101:0/536819224 >> v1:172.21.2.101:6789/0 conn(0x7f96fc0cff70 legacy=0x7f96fc109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:57.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.690 7f96fb7fe700 1 -- 172.21.2.101:0/536819224 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96fc088260 con 0x7f96fc0d1870 2021-02-10T11:39:57.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.690 7f96fb7fe700 1 --2- 172.21.2.101:0/536819224 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96fc0d1870 0x7f96fc1b04c0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 rx=0x7f96ec006650 tx=0x7f96ec009a80).ready entity=mon.0 client_cookie=2ff3a8940bb32b44 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:57.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.690 7f96fa7fc700 1 -- 172.21.2.101:0/536819224 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96ec0009b0 con 0x7f96fc0d1870 2021-02-10T11:39:57.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.690 7f96fa7fc700 1 -- 172.21.2.101:0/536819224 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ec00c0c0 con 0x7f96fc0d1870 2021-02-10T11:39:57.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.690 7f96fa7fc700 1 -- 172.21.2.101:0/536819224 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96ec00c520 con 0x7f96fc0d1870 2021-02-10T11:39:57.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.690 7f9701e84700 1 -- 172.21.2.101:0/536819224 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96fc0d1870 msgr2=0x7f96fc1b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:57.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.690 7f9701e84700 1 --2- 172.21.2.101:0/536819224 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96fc0d1870 0x7f96fc1b04c0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 rx=0x7f96ec006650 tx=0x7f96ec009a80).stop 2021-02-10T11:39:57.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.690 7f9701e84700 1 -- 172.21.2.101:0/536819224 shutdown_connections 2021-02-10T11:39:57.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.690 7f9701e84700 1 --2- 172.21.2.101:0/536819224 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96fc0d1870 0x7f96fc1b04c0 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:57.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.690 7f9701e84700 1 -- 172.21.2.101:0/536819224 >> 172.21.2.101:0/536819224 conn(0x7f96fc0ce420 msgr2=0x7f96fc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:57.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.690 7f9701e84700 1 -- 172.21.2.101:0/536819224 shutdown_connections 2021-02-10T11:39:57.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.690 7f9701e84700 1 -- 172.21.2.101:0/536819224 wait complete. 2021-02-10T11:39:57.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.705 7f9701e84700 1 Processor -- start 2021-02-10T11:39:57.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.705 7f9701e84700 1 -- start start 2021-02-10T11:39:57.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.706 7f9701e84700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96fc0c3820 0x7f96fc084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:57.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.706 7f9701e84700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96fc0a0290 0x7f96fc0852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:57.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.706 7f9701e84700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96fc086270 0x7f96fc086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:57.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.706 7f9701e84700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc0c0700 con 0x7f96fc0c3820 2021-02-10T11:39:57.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.706 7f9701e84700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc0858f0 con 0x7f96fc086270 2021-02-10T11:39:57.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.706 7f9701e84700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc085b10 con 0x7f96fc0a0290 2021-02-10T11:39:57.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.706 7f96fb7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96fc0c3820 0x7f96fc084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:57.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.706 7f96fb7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96fc0c3820 0x7f96fc084cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56068/0 (socket says 172.21.2.101:56068) 2021-02-10T11:39:57.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.706 7f96fb7fe700 1 -- 172.21.2.101:0/284612896 learned_addr learned my addr 172.21.2.101:0/284612896 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:57.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.706 7f96fb7fe700 1 -- 172.21.2.101:0/284612896 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96fc0a0290 msgr2=0x7f96fc0852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:57.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.706 7f96fb7fe700 1 --2- 172.21.2.101:0/284612896 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96fc0a0290 0x7f96fc0852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:57.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.706 7f96fb7fe700 1 -- 172.21.2.101:0/284612896 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96fc086270 msgr2=0x7f96fc086670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:57.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.706 7f96fb7fe700 1 --2- 172.21.2.101:0/284612896 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96fc086270 0x7f96fc086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:57.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.706 7f96fb7fe700 1 -- 172.21.2.101:0/284612896 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96fc086c90 con 0x7f96fc0c3820 2021-02-10T11:39:57.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.706 7f96fb7fe700 1 --2- 172.21.2.101:0/284612896 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96fc0c3820 0x7f96fc084cb0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 rx=0x7f96ec009630 tx=0x7f96ec00b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:57.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.706 7f96e3fff700 1 -- 172.21.2.101:0/284612896 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96ec0093a0 con 0x7f96fc0c3820 2021-02-10T11:39:57.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.707 7f9701e84700 1 -- 172.21.2.101:0/284612896 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96fc086c90 con 0x7f96fc0c3820 2021-02-10T11:39:57.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.707 7f9701e84700 1 -- 172.21.2.101:0/284612896 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96fc1a2f00 con 0x7f96fc0c3820 2021-02-10T11:39:57.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.707 7f96e3fff700 1 -- 172.21.2.101:0/284612896 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ec000940 con 0x7f96fc0c3820 2021-02-10T11:39:57.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.707 7f96e3fff700 1 -- 172.21.2.101:0/284612896 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96ec00bd70 con 0x7f96fc0c3820 2021-02-10T11:39:57.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.707 7f96e3fff700 1 -- 172.21.2.101:0/284612896 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f96ec0054f0 con 0x7f96fc0c3820 2021-02-10T11:39:57.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.707 7f96e3fff700 1 --2- 172.21.2.101:0/284612896 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96dc0186c0 0x7f96dc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:57.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.707 7f96e3fff700 1 -- 172.21.2.101:0/284612896 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f96ec032440 con 0x7f96fc0c3820 2021-02-10T11:39:57.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.707 7f96faffd700 1 --2- 172.21.2.101:0/284612896 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96dc0186c0 0x7f96dc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:57.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.708 7f96faffd700 1 --2- 172.21.2.101:0/284612896 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96dc0186c0 0x7f96dc01abc0 crc :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:57.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.739 7f9701e84700 1 -- 172.21.2.101:0/284612896 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96fc082a70 con 0x7f96fc0c3820 2021-02-10T11:39:57.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.741 7f96e3fff700 1 -- 172.21.2.101:0/284612896 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f96ec0052e0 con 0x7f96fc0c3820 2021-02-10T11:39:57.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:57.878 7f9701e84700 1 -- 172.21.2.101:0/284612896 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"} v 0) v1 -- 0x7f96fc082a70 con 0x7f96fc0c3820 2021-02-10T11:39:58.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.062 7f96e3fff700 1 -- 172.21.2.101:0/284612896 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"}]=0 v79) v1 ==== 118+0+0 (secure 0 0 0) 0x7f96ec0054f0 con 0x7f96fc0c3820 2021-02-10T11:39:58.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.095 7f9701e84700 1 -- 172.21.2.101:0/284612896 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96dc0186c0 msgr2=0x7f96dc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:58.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.095 7f9701e84700 1 --2- 172.21.2.101:0/284612896 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96dc0186c0 0x7f96dc01abc0 crc :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:58.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.095 7f9701e84700 1 -- 172.21.2.101:0/284612896 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96fc0c3820 msgr2=0x7f96fc084cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:58.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.095 7f9701e84700 1 --2- 172.21.2.101:0/284612896 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96fc0c3820 0x7f96fc084cb0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 rx=0x7f96ec009630 tx=0x7f96ec00b980).stop 2021-02-10T11:39:58.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.095 7f9701e84700 1 -- 172.21.2.101:0/284612896 shutdown_connections 2021-02-10T11:39:58.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.095 7f9701e84700 1 --2- 172.21.2.101:0/284612896 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96dc0186c0 0x7f96dc01abc0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:58.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.095 7f9701e84700 1 --2- 172.21.2.101:0/284612896 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96fc0a0290 0x7f96fc0852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:58.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.095 7f9701e84700 1 --2- 172.21.2.101:0/284612896 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96fc086270 0x7f96fc086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:58.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.095 7f9701e84700 1 --2- 172.21.2.101:0/284612896 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96fc0c3820 0x7f96fc084cb0 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:58.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.095 7f9701e84700 1 -- 172.21.2.101:0/284612896 >> 172.21.2.101:0/284612896 conn(0x7f96fc0872c0 msgr2=0x7f96fc095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:58.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.095 7f9701e84700 1 -- 172.21.2.101:0/284612896 shutdown_connections 2021-02-10T11:39:58.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.095 7f9701e84700 1 -- 172.21.2.101:0/284612896 wait complete. 2021-02-10T11:39:58.106 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:39:58.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.512 7f40a3e2d700 1 Processor -- start 2021-02-10T11:39:58.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.512 7f40a3e2d700 1 -- start start 2021-02-10T11:39:58.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.513 7f40a3e2d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f409c0d1870 0x7f409c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:58.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.513 7f40a3e2d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f409c0f0650 0x7f409c0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:58.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.513 7f40a3e2d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409c0f0b10 con 0x7f409c1b0a10 2021-02-10T11:39:58.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.513 7f40a3e2d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409c0e5ed0 con 0x7f409c0d1870 2021-02-10T11:39:58.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.513 7f40a3e2d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409c0d1c70 con 0x7f409c0f0650 2021-02-10T11:39:58.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.513 7f40a1c49700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f409c0d1870 0x7f409c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:58.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.513 7f40a1c49700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f409c0d1870 0x7f409c1b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56076/0 (socket says 172.21.2.101:56076) 2021-02-10T11:39:58.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.513 7f40a1c49700 1 -- 172.21.2.101:0/1767034640 learned_addr learned my addr 172.21.2.101:0/1767034640 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:58.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.513 7f40a244a700 1 --2- 172.21.2.101:0/1767034640 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f409c0f0650 0x7f409c0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:58.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.514 7f40a1c49700 1 -- 172.21.2.101:0/1767034640 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f409c0f0650 msgr2=0x7f409c0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:58.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.514 7f40a1c49700 1 --2- 172.21.2.101:0/1767034640 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f409c0f0650 0x7f409c0cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:58.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.514 7f40a1c49700 1 -- 172.21.2.101:0/1767034640 >> v1:172.21.2.108:6789/0 conn(0x7f409c1b0a10 legacy=0x7f409c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:58.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.514 7f40a1c49700 1 -- 172.21.2.101:0/1767034640 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c0d1e90 con 0x7f409c0d1870 2021-02-10T11:39:58.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.514 7f40a1c49700 1 --2- 172.21.2.101:0/1767034640 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f409c0d1870 0x7f409c1b03f0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 rx=0x7f4098006700 tx=0x7f4098009b00).ready entity=mon.0 client_cookie=e8ed2a9fb161b1aa server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:58.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.514 7f40a0c47700 1 -- 172.21.2.101:0/1767034640 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f409800c0c0 con 0x7f409c0d1870 2021-02-10T11:39:58.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.514 7f40a0c47700 1 -- 172.21.2.101:0/1767034640 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409800c400 con 0x7f409c0d1870 2021-02-10T11:39:58.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.514 7f40a0c47700 1 -- 172.21.2.101:0/1767034640 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f409800c8e0 con 0x7f409c0d1870 2021-02-10T11:39:58.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.514 7f40a3e2d700 1 -- 172.21.2.101:0/1767034640 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f409c0d1870 msgr2=0x7f409c1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:58.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.514 7f40a3e2d700 1 --2- 172.21.2.101:0/1767034640 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f409c0d1870 0x7f409c1b03f0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 rx=0x7f4098006700 tx=0x7f4098009b00).stop 2021-02-10T11:39:58.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.516 7f40a3e2d700 1 -- 172.21.2.101:0/1767034640 shutdown_connections 2021-02-10T11:39:58.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.516 7f40a3e2d700 1 --2- 172.21.2.101:0/1767034640 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f409c0f0650 0x7f409c0cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:58.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.516 7f40a3e2d700 1 --2- 172.21.2.101:0/1767034640 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f409c0d1870 0x7f409c1b03f0 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:58.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.516 7f40a3e2d700 1 -- 172.21.2.101:0/1767034640 >> 172.21.2.101:0/1767034640 conn(0x7f409c0ce450 msgr2=0x7f409c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:58.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.516 7f40a3e2d700 1 -- 172.21.2.101:0/1767034640 shutdown_connections 2021-02-10T11:39:58.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.516 7f40a3e2d700 1 -- 172.21.2.101:0/1767034640 wait complete. 2021-02-10T11:39:58.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.529 7f40a3e2d700 1 Processor -- start 2021-02-10T11:39:58.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.529 7f40a3e2d700 1 -- start start 2021-02-10T11:39:58.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.530 7f40a3e2d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f409c195650 0x7f409c0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:58.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.530 7f40a3e2d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f409c0b5b80 0x7f409c1a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:58.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.530 7f40a3e2d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f409c0acfa0 0x7f409c1a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:58.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.530 7f40a3e2d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c0b6010 con 0x7f409c195650 2021-02-10T11:39:58.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.530 7f40a3e2d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c1a3a60 con 0x7f409c0acfa0 2021-02-10T11:39:58.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.530 7f40a3e2d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409c1a3c40 con 0x7f409c0b5b80 2021-02-10T11:39:58.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.530 7f40a1c49700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f409c195650 0x7f409c0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:58.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.530 7f40a1c49700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f409c195650 0x7f409c0b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56082/0 (socket says 172.21.2.101:56082) 2021-02-10T11:39:58.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.530 7f40a1c49700 1 -- 172.21.2.101:0/2047309161 learned_addr learned my addr 172.21.2.101:0/2047309161 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:58.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.530 7f40a244a700 1 --2- 172.21.2.101:0/2047309161 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f409c0acfa0 0x7f409c1a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:58.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.530 7f40a1448700 1 --2- 172.21.2.101:0/2047309161 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f409c0b5b80 0x7f409c1a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:58.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.530 7f40a1c49700 1 -- 172.21.2.101:0/2047309161 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f409c0b5b80 msgr2=0x7f409c1a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:58.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.530 7f40a1c49700 1 --2- 172.21.2.101:0/2047309161 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f409c0b5b80 0x7f409c1a2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:58.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.530 7f40a1c49700 1 -- 172.21.2.101:0/2047309161 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f409c0acfa0 msgr2=0x7f409c1a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:58.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.530 7f40a1c49700 1 --2- 172.21.2.101:0/2047309161 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f409c0acfa0 0x7f409c1a3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:58.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.530 7f40a1c49700 1 -- 172.21.2.101:0/2047309161 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c0bcae0 con 0x7f409c195650 2021-02-10T11:39:58.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.531 7f40a1448700 1 --2- 172.21.2.101:0/2047309161 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f409c0b5b80 0x7f409c1a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:39:58.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.531 7f40a1c49700 1 --2- 172.21.2.101:0/2047309161 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f409c195650 0x7f409c0b5560 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 rx=0x7f409800b660 tx=0x7f409800ba00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:58.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.531 7f408e7fc700 1 -- 172.21.2.101:0/2047309161 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4098009440 con 0x7f409c195650 2021-02-10T11:39:58.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.531 7f408e7fc700 1 -- 172.21.2.101:0/2047309161 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4098000c90 con 0x7f409c195650 2021-02-10T11:39:58.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.531 7f408e7fc700 1 -- 172.21.2.101:0/2047309161 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4098002030 con 0x7f409c195650 2021-02-10T11:39:58.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.531 7f40a3e2d700 1 -- 172.21.2.101:0/2047309161 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409c0bcae0 con 0x7f409c195650 2021-02-10T11:39:58.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.531 7f40a3e2d700 1 -- 172.21.2.101:0/2047309161 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409c0bcae0 con 0x7f409c195650 2021-02-10T11:39:58.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.532 7f408e7fc700 1 -- 172.21.2.101:0/2047309161 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f40980093e0 con 0x7f409c195650 2021-02-10T11:39:58.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.532 7f408e7fc700 1 --2- 172.21.2.101:0/2047309161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f40840186c0 0x7f408401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:58.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.532 7f408e7fc700 1 -- 172.21.2.101:0/2047309161 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4098032420 con 0x7f409c195650 2021-02-10T11:39:58.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.532 7f40a1448700 1 --2- 172.21.2.101:0/2047309161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f40840186c0 0x7f408401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:58.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.532 7f40a1448700 1 --2- 172.21.2.101:0/2047309161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f40840186c0 0x7f408401abc0 crc :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:58.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.563 7f40a3e2d700 1 -- 172.21.2.101:0/2047309161 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409c0b6010 con 0x7f409c195650 2021-02-10T11:39:58.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.566 7f408e7fc700 1 -- 172.21.2.101:0/2047309161 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f409802b160 con 0x7f409c195650 2021-02-10T11:39:58.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.693 7f40a3e2d700 1 -- 172.21.2.101:0/2047309161 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f409c0b6010 con 0x7f409c195650 2021-02-10T11:39:58.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.693 7f408e7fc700 1 -- 172.21.2.101:0/2047309161 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+2107 (secure 0 0 0) 0x7f409802b160 con 0x7f409c195650 2021-02-10T11:39:58.697 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:39:58.697 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":79,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10125,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/3121560299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3121560299},{"type":"v1","addr":"172.21.2.101:6835","nonce":3121560299}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":76}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:39:56.054150","modified":"2021-02-10 11:39:58.060269","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10122},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10122":{"gid":10122,"name":"b","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/1690286235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":1690286235},{"type":"v1","addr":"172.21.2.108:6833","nonce":1690286235}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2021-02-10T11:39:58.697 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 79 2021-02-10T11:39:58.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.696 7f40a3e2d700 1 -- 172.21.2.101:0/2047309161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f40840186c0 msgr2=0x7f408401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:58.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.696 7f40a3e2d700 1 --2- 172.21.2.101:0/2047309161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f40840186c0 0x7f408401abc0 crc :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:58.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.696 7f40a3e2d700 1 -- 172.21.2.101:0/2047309161 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f409c195650 msgr2=0x7f409c0b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:58.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.696 7f40a3e2d700 1 --2- 172.21.2.101:0/2047309161 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f409c195650 0x7f409c0b5560 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 rx=0x7f409800b660 tx=0x7f409800ba00).stop 2021-02-10T11:39:58.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.696 7f40a3e2d700 1 -- 172.21.2.101:0/2047309161 shutdown_connections 2021-02-10T11:39:58.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.696 7f40a3e2d700 1 --2- 172.21.2.101:0/2047309161 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f40840186c0 0x7f408401abc0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:58.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.696 7f40a3e2d700 1 --2- 172.21.2.101:0/2047309161 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f409c0b5b80 0x7f409c1a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:58.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.696 7f40a3e2d700 1 --2- 172.21.2.101:0/2047309161 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f409c0acfa0 0x7f409c1a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:58.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.696 7f40a3e2d700 1 --2- 172.21.2.101:0/2047309161 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f409c195650 0x7f409c0b5560 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:58.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.696 7f40a3e2d700 1 -- 172.21.2.101:0/2047309161 >> 172.21.2.101:0/2047309161 conn(0x7f409c0b3040 msgr2=0x7f409c0c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:58.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.696 7f40a3e2d700 1 -- 172.21.2.101:0/2047309161 shutdown_connections 2021-02-10T11:39:58.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:58.696 7f40a3e2d700 1 -- 172.21.2.101:0/2047309161 wait complete. 2021-02-10T11:39:58.707 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:39:59.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.117 7f9107d32700 1 Processor -- start 2021-02-10T11:39:59.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.117 7f9107d32700 1 -- start start 2021-02-10T11:39:59.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.117 7f9107d32700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f91000d1870 0x7f91001b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:59.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.117 7f9107d32700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91000f0650 0x7f91000cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:59.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.117 7f9107d32700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91000f0b10 con 0x7f91001b0a10 2021-02-10T11:39:59.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.117 7f9107d32700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91000e5ed0 con 0x7f91000d1870 2021-02-10T11:39:59.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.117 7f9107d32700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91000d1c70 con 0x7f91000f0650 2021-02-10T11:39:59.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.117 7f9105b4e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f91000d1870 0x7f91001b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:59.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.117 7f9105b4e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f91000d1870 0x7f91001b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56090/0 (socket says 172.21.2.101:56090) 2021-02-10T11:39:59.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.117 7f9105b4e700 1 -- 172.21.2.101:0/3409298122 learned_addr learned my addr 172.21.2.101:0/3409298122 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:59.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.118 7f9105b4e700 1 -- 172.21.2.101:0/3409298122 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91000f0650 msgr2=0x7f91000cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:59.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.118 7f9105b4e700 1 --2- 172.21.2.101:0/3409298122 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91000f0650 0x7f91000cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:59.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.118 7f9105b4e700 1 -- 172.21.2.101:0/3409298122 >> v1:172.21.2.108:6789/0 conn(0x7f91001b0a10 legacy=0x7f91000cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:59.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.118 7f9105b4e700 1 -- 172.21.2.101:0/3409298122 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91000d1e90 con 0x7f91000d1870 2021-02-10T11:39:59.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.118 7f9105b4e700 1 --2- 172.21.2.101:0/3409298122 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91000d1870 0x7f91001b03f0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 rx=0x7f90f4006700 tx=0x7f90f4009b80).ready entity=mon.0 client_cookie=d694a41a96f46bc1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:59.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.118 7f9104b4c700 1 -- 172.21.2.101:0/3409298122 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f90f400c0c0 con 0x7f91000d1870 2021-02-10T11:39:59.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.118 7f9104b4c700 1 -- 172.21.2.101:0/3409298122 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f400c340 con 0x7f91000d1870 2021-02-10T11:39:59.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.118 7f9104b4c700 1 -- 172.21.2.101:0/3409298122 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f90f400c860 con 0x7f91000d1870 2021-02-10T11:39:59.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.118 7f9107d32700 1 -- 172.21.2.101:0/3409298122 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91000d1870 msgr2=0x7f91001b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:59.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.118 7f9107d32700 1 --2- 172.21.2.101:0/3409298122 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91000d1870 0x7f91001b03f0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 rx=0x7f90f4006700 tx=0x7f90f4009b80).stop 2021-02-10T11:39:59.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.119 7f9107d32700 1 -- 172.21.2.101:0/3409298122 shutdown_connections 2021-02-10T11:39:59.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.119 7f9107d32700 1 --2- 172.21.2.101:0/3409298122 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91000f0650 0x7f91000cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:59.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.119 7f9107d32700 1 --2- 172.21.2.101:0/3409298122 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91000d1870 0x7f91001b03f0 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:59.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.119 7f9107d32700 1 -- 172.21.2.101:0/3409298122 >> 172.21.2.101:0/3409298122 conn(0x7f91000ce450 msgr2=0x7f91000d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:59.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.119 7f9107d32700 1 -- 172.21.2.101:0/3409298122 shutdown_connections 2021-02-10T11:39:59.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.119 7f9107d32700 1 -- 172.21.2.101:0/3409298122 wait complete. 2021-02-10T11:39:59.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.134 7f9107d32700 1 Processor -- start 2021-02-10T11:39:59.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.134 7f9107d32700 1 -- start start 2021-02-10T11:39:59.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.134 7f9107d32700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f910018aa60 0x7f910019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:59.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.134 7f9107d32700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f91000b7a50 0x7f91000b7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:59.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.134 7f9107d32700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91000b8470 0x7f91001aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:59.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.134 7f9107d32700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91000b8870 con 0x7f910018aa60 2021-02-10T11:39:59.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.134 7f9107d32700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91001aa030 con 0x7f91000b7a50 2021-02-10T11:39:59.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.134 7f9107d32700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910019f210 con 0x7f91000b8470 2021-02-10T11:39:59.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.135 7f9105b4e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f910018aa60 0x7f910019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:59.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.135 7f9105b4e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f910018aa60 0x7f910019ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56096/0 (socket says 172.21.2.101:56096) 2021-02-10T11:39:59.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.135 7f9105b4e700 1 -- 172.21.2.101:0/4006061672 learned_addr learned my addr 172.21.2.101:0/4006061672 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:59.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.135 7f9105b4e700 1 -- 172.21.2.101:0/4006061672 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91000b8470 msgr2=0x7f91001aa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:59.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.135 7f9105b4e700 1 --2- 172.21.2.101:0/4006061672 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91000b8470 0x7f91001aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:59.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.135 7f9105b4e700 1 -- 172.21.2.101:0/4006061672 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f91000b7a50 msgr2=0x7f91000b7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:59.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.135 7f9105b4e700 1 --2- 172.21.2.101:0/4006061672 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f91000b7a50 0x7f91000b7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:59.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.135 7f9105b4e700 1 -- 172.21.2.101:0/4006061672 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91001a7a00 con 0x7f910018aa60 2021-02-10T11:39:59.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.135 7f9105b4e700 1 --2- 172.21.2.101:0/4006061672 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f910018aa60 0x7f910019ebf0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 rx=0x7f90f400b580 tx=0x7f90f400b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:59.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.135 7f90f27fc700 1 -- 172.21.2.101:0/4006061672 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f90f400bd10 con 0x7f910018aa60 2021-02-10T11:39:59.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.135 7f90f27fc700 1 -- 172.21.2.101:0/4006061672 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f4006500 con 0x7f910018aa60 2021-02-10T11:39:59.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.135 7f90f27fc700 1 -- 172.21.2.101:0/4006061672 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f90f40022b0 con 0x7f910018aa60 2021-02-10T11:39:59.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.135 7f9107d32700 1 -- 172.21.2.101:0/4006061672 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91001a7a00 con 0x7f910018aa60 2021-02-10T11:39:59.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.136 7f9107d32700 1 -- 172.21.2.101:0/4006061672 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91001a7a00 con 0x7f910018aa60 2021-02-10T11:39:59.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.136 7f90f27fc700 1 -- 172.21.2.101:0/4006061672 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f90f4000c90 con 0x7f910018aa60 2021-02-10T11:39:59.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.136 7f90f27fc700 1 --2- 172.21.2.101:0/4006061672 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90e8018730 0x7f90e801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:59.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.136 7f90f27fc700 1 -- 172.21.2.101:0/4006061672 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f90f4032680 con 0x7f910018aa60 2021-02-10T11:39:59.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.136 7f910534d700 1 --2- 172.21.2.101:0/4006061672 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90e8018730 0x7f90e801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:59.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.136 7f910534d700 1 --2- 172.21.2.101:0/4006061672 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90e8018730 0x7f90e801ac10 crc :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:59.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.167 7f9107d32700 1 -- 172.21.2.101:0/4006061672 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91000ce450 con 0x7f910018aa60 2021-02-10T11:39:59.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.170 7f90f27fc700 1 -- 172.21.2.101:0/4006061672 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f90f401f540 con 0x7f910018aa60 2021-02-10T11:39:59.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.294 7f9107d32700 1 -- 172.21.2.101:0/4006061672 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91000ce450 con 0x7f910018aa60 2021-02-10T11:39:59.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.294 7f90f27fc700 1 -- 172.21.2.101:0/4006061672 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+14005 (secure 0 0 0) 0x7f90f4000c90 con 0x7f910018aa60 2021-02-10T11:39:59.298 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:39:59.298 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":103,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:39:57.056338","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:39:54.022810","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2021-02-10 11:39:55.095134","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:39:59.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.296 7f9107d32700 1 -- 172.21.2.101:0/4006061672 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90e8018730 msgr2=0x7f90e801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:59.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.296 7f9107d32700 1 --2- 172.21.2.101:0/4006061672 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90e8018730 0x7f90e801ac10 crc :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:59.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.296 7f9107d32700 1 -- 172.21.2.101:0/4006061672 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f910018aa60 msgr2=0x7f910019ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:59.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.296 7f9107d32700 1 --2- 172.21.2.101:0/4006061672 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f910018aa60 0x7f910019ebf0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 rx=0x7f90f400b580 tx=0x7f90f400b920).stop 2021-02-10T11:39:59.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.296 7f9107d32700 1 -- 172.21.2.101:0/4006061672 shutdown_connections 2021-02-10T11:39:59.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.296 7f9107d32700 1 --2- 172.21.2.101:0/4006061672 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90e8018730 0x7f90e801ac10 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:59.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.296 7f9107d32700 1 --2- 172.21.2.101:0/4006061672 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91000b8470 0x7f91001aa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:59.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.296 7f9107d32700 1 --2- 172.21.2.101:0/4006061672 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f91000b7a50 0x7f91000b7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:59.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.296 7f9107d32700 1 --2- 172.21.2.101:0/4006061672 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f910018aa60 0x7f910019ebf0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:59.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.296 7f9107d32700 1 -- 172.21.2.101:0/4006061672 >> 172.21.2.101:0/4006061672 conn(0x7f91000efc70 msgr2=0x7f91000ef410 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:59.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.296 7f9107d32700 1 -- 172.21.2.101:0/4006061672 shutdown_connections 2021-02-10T11:39:59.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.297 7f9107d32700 1 -- 172.21.2.101:0/4006061672 wait complete. 2021-02-10T11:39:59.308 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:39:59.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.729 7f1670597700 1 Processor -- start 2021-02-10T11:39:59.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.729 7f1670597700 1 -- start start 2021-02-10T11:39:59.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.730 7f1670597700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16680d1880 0x7f16681b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:59.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.730 7f1670597700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1668109a70 con 0x7f16681b0af0 2021-02-10T11:39:59.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.730 7f1670597700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1668088070 con 0x7f16680cff80 2021-02-10T11:39:59.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.730 7f1670597700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16680e5f00 con 0x7f16680d1880 2021-02-10T11:39:59.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.730 7f166dbb2700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f16681b0af0 0x7f1668108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56598/0 (socket says 172.21.2.101:56598) 2021-02-10T11:39:59.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.730 7f166dbb2700 1 -- 172.21.2.101:0/33288515 learned_addr learned my addr 172.21.2.101:0/33288515 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:39:59.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.731 7f166d3b1700 1 -- 172.21.2.101:0/33288515 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029136213 0 0) 0x7f16640010b0 con 0x7f16681b0af0 2021-02-10T11:39:59.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.731 7f166d3b1700 1 -- 172.21.2.101:0/33288515 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1654002fe0 con 0x7f16681b0af0 2021-02-10T11:39:59.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.731 7f166e3b3700 1 --2- 172.21.2.101:0/33288515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16680d1880 0x7f16681b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:59.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.731 7f166d3b1700 1 -- 172.21.2.101:0/33288515 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1045448234 0 0) 0x7f16640010b0 con 0x7f16681b0af0 2021-02-10T11:39:59.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.731 7f166d3b1700 1 -- 172.21.2.101:0/33288515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16680d1880 msgr2=0x7f16681b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:59.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.731 7f166d3b1700 1 --2- 172.21.2.101:0/33288515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16680d1880 0x7f16681b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:59.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.731 7f166d3b1700 1 -- 172.21.2.101:0/33288515 >> v1:172.21.2.108:6789/0 conn(0x7f16680cff80 legacy=0x7f1668109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:59.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.731 7f166d3b1700 1 -- 172.21.2.101:0/33288515 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16680d1c80 con 0x7f16681b0af0 2021-02-10T11:39:59.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.731 7f166d3b1700 1 -- 172.21.2.101:0/33288515 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f16640015b0 con 0x7f16681b0af0 2021-02-10T11:39:59.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.731 7f166d3b1700 1 -- 172.21.2.101:0/33288515 >> v1:172.21.2.101:6789/0 conn(0x7f16681b0af0 legacy=0x7f1668108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:59.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.731 7f166d3b1700 1 --2- 172.21.2.101:0/33288515 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1654003aa0 0x7f1654005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:59.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.731 7f166d3b1700 1 --2- 172.21.2.101:0/33288515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1654006790 0x7f1654008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:59.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.731 7f166d3b1700 1 --2- 172.21.2.101:0/33288515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1654009590 0x7f165400ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:59.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.731 7f166d3b1700 1 -- 172.21.2.101:0/33288515 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165400c2d0 con 0x7f1654006790 2021-02-10T11:39:59.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.731 7f166d3b1700 1 -- 172.21.2.101:0/33288515 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165400c4f0 con 0x7f1654003aa0 2021-02-10T11:39:59.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.731 7f166d3b1700 1 -- 172.21.2.101:0/33288515 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f165400c710 con 0x7f1654009590 2021-02-10T11:39:59.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.731 7f166dbb2700 1 --2- 172.21.2.101:0/33288515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1654006790 0x7f1654008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:59.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.731 7f166ebb4700 1 --2- 172.21.2.101:0/33288515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1654009590 0x7f165400ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.731 7f166e3b3700 1 --2- 172.21.2.101:0/33288515 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1654003aa0 0x7f1654005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.732 7f166dbb2700 1 -- 172.21.2.101:0/33288515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1654009590 msgr2=0x7f165400ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.732 7f166dbb2700 1 --2- 172.21.2.101:0/33288515 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1654009590 0x7f165400ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.732 7f166dbb2700 1 -- 172.21.2.101:0/33288515 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1654003aa0 msgr2=0x7f1654005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:39:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.732 7f166dbb2700 1 --2- 172.21.2.101:0/33288515 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1654003aa0 0x7f1654005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.732 7f166dbb2700 1 -- 172.21.2.101:0/33288515 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1654050950 con 0x7f1654006790 2021-02-10T11:39:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.732 7f166e3b3700 1 --2- 172.21.2.101:0/33288515 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1654003aa0 0x7f1654005ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:39:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.732 7f166dbb2700 1 --2- 172.21.2.101:0/33288515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1654006790 0x7f1654008c90 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 rx=0x7f1664001370 tx=0x7f1664007d30).ready entity=mon.0 client_cookie=7e37e39e15a3b108 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.732 7f166d3b1700 1 -- 172.21.2.101:0/33288515 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f16640010b0 con 0x7f1654006790 2021-02-10T11:39:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.732 7f166d3b1700 1 -- 172.21.2.101:0/33288515 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1664000990 con 0x7f1654006790 2021-02-10T11:39:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.732 7f166d3b1700 1 -- 172.21.2.101:0/33288515 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f166400a710 con 0x7f1654006790 2021-02-10T11:39:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.732 7f1670597700 1 -- 172.21.2.101:0/33288515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1654006790 msgr2=0x7f1654008c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:59.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.732 7f1670597700 1 --2- 172.21.2.101:0/33288515 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1654006790 0x7f1654008c90 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 rx=0x7f1664001370 tx=0x7f1664007d30).stop 2021-02-10T11:39:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.732 7f166ebb4700 1 -- 172.21.2.101:0/33288515 reap_dead start 2021-02-10T11:39:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.733 7f1670597700 1 -- 172.21.2.101:0/33288515 shutdown_connections 2021-02-10T11:39:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.733 7f1670597700 1 -- 172.21.2.101:0/33288515 >> 172.21.2.101:0/33288515 conn(0x7f16680ce430 msgr2=0x7f16680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.733 7f1670597700 1 -- 172.21.2.101:0/33288515 shutdown_connections 2021-02-10T11:39:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.733 7f1670597700 1 -- 172.21.2.101:0/33288515 wait complete. 2021-02-10T11:39:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.747 7f1670597700 1 Processor -- start 2021-02-10T11:39:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.747 7f1670597700 1 -- start start 2021-02-10T11:39:59.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.748 7f1670597700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f166808d090 0x7f166808c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:59.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.748 7f1670597700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f166808d490 0x7f16680beec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:59.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.748 7f1670597700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16680bf4e0 0x7f166809fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:59.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.748 7f1670597700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16680bf8e0 con 0x7f166808d090 2021-02-10T11:39:59.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.748 7f1670597700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16680a0c90 con 0x7f166808d490 2021-02-10T11:39:59.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.748 7f1670597700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16680a0eb0 con 0x7f16680bf4e0 2021-02-10T11:39:59.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.748 7f166e3b3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f166808d090 0x7f166808c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:59.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.748 7f166e3b3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f166808d090 0x7f166808c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56114/0 (socket says 172.21.2.101:56114) 2021-02-10T11:39:59.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.748 7f166e3b3700 1 -- 172.21.2.101:0/599362709 learned_addr learned my addr 172.21.2.101:0/599362709 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:39:59.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.748 7f166e3b3700 1 -- 172.21.2.101:0/599362709 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16680bf4e0 msgr2=0x7f166809fc40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:59.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.748 7f166e3b3700 1 --2- 172.21.2.101:0/599362709 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16680bf4e0 0x7f166809fc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:59.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.748 7f166e3b3700 1 -- 172.21.2.101:0/599362709 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f166808d490 msgr2=0x7f16680beec0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:39:59.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.748 7f166e3b3700 1 --2- 172.21.2.101:0/599362709 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f166808d490 0x7f16680beec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:59.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.748 7f166e3b3700 1 -- 172.21.2.101:0/599362709 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16680a0260 con 0x7f166808d090 2021-02-10T11:39:59.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.748 7f166e3b3700 1 --2- 172.21.2.101:0/599362709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f166808d090 0x7f166808c490 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 rx=0x7f165c006f40 tx=0x7f165c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:59.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.749 7f165affd700 1 -- 172.21.2.101:0/599362709 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f165c00bd80 con 0x7f166808d090 2021-02-10T11:39:59.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.749 7f165affd700 1 -- 172.21.2.101:0/599362709 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165c000c90 con 0x7f166808d090 2021-02-10T11:39:59.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.749 7f165affd700 1 -- 172.21.2.101:0/599362709 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f165c002030 con 0x7f166808d090 2021-02-10T11:39:59.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.749 7f1670597700 1 -- 172.21.2.101:0/599362709 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16680a0260 con 0x7f166808d090 2021-02-10T11:39:59.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.749 7f1670597700 1 -- 172.21.2.101:0/599362709 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16680a07f0 con 0x7f166808d090 2021-02-10T11:39:59.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.749 7f165affd700 1 -- 172.21.2.101:0/599362709 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f165c006b00 con 0x7f166808d090 2021-02-10T11:39:59.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.749 7f165affd700 1 --2- 172.21.2.101:0/599362709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1650018730 0x7f165001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:39:59.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.749 7f165affd700 1 -- 172.21.2.101:0/599362709 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f165c032640 con 0x7f166808d090 2021-02-10T11:39:59.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.749 7f166dbb2700 1 --2- 172.21.2.101:0/599362709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1650018730 0x7f165001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:39:59.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.750 7f166dbb2700 1 --2- 172.21.2.101:0/599362709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1650018730 0x7f165001abc0 crc :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:39:59.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.780 7f1670597700 1 -- 172.21.2.101:0/599362709 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f166808cb60 con 0x7f166808d090 2021-02-10T11:39:59.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.782 7f165affd700 1 -- 172.21.2.101:0/599362709 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f165c000940 con 0x7f166808d090 2021-02-10T11:39:59.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.912 7f1670597700 1 -- 172.21.2.101:0/599362709 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f166808cb60 con 0x7f166808d090 2021-02-10T11:39:59.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.916 7f165affd700 1 -- 172.21.2.101:0/599362709 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v61) v1 ==== 184+0+0 (secure 0 0 0) 0x7f165c000c90 con 0x7f166808d090 2021-02-10T11:39:59.922 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.0 2021-02-10T11:39:59.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.921 7f1670597700 1 -- 172.21.2.101:0/599362709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1650018730 msgr2=0x7f165001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:59.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.921 7f1670597700 1 --2- 172.21.2.101:0/599362709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1650018730 0x7f165001abc0 crc :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:59.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.921 7f1670597700 1 -- 172.21.2.101:0/599362709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f166808d090 msgr2=0x7f166808c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:39:59.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.921 7f1670597700 1 --2- 172.21.2.101:0/599362709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f166808d090 0x7f166808c490 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 rx=0x7f165c006f40 tx=0x7f165c00b990).stop 2021-02-10T11:39:59.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.921 7f1670597700 1 -- 172.21.2.101:0/599362709 shutdown_connections 2021-02-10T11:39:59.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.921 7f1670597700 1 --2- 172.21.2.101:0/599362709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1650018730 0x7f165001abc0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:59.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.921 7f1670597700 1 --2- 172.21.2.101:0/599362709 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16680bf4e0 0x7f166809fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:59.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.921 7f1670597700 1 --2- 172.21.2.101:0/599362709 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f166808d490 0x7f16680beec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:39:59.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.921 7f1670597700 1 --2- 172.21.2.101:0/599362709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f166808d090 0x7f166808c490 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:39:59.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.921 7f1670597700 1 -- 172.21.2.101:0/599362709 >> 172.21.2.101:0/599362709 conn(0x7f1668087390 msgr2=0x7f16680a6770 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:39:59.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.922 7f1670597700 1 -- 172.21.2.101:0/599362709 shutdown_connections 2021-02-10T11:39:59.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:39:59.922 7f1670597700 1 -- 172.21.2.101:0/599362709 wait complete. 2021-02-10T11:39:59.933 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:40:00.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.352 7f2f8a7e7700 1 Processor -- start 2021-02-10T11:40:00.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.353 7f2f8a7e7700 1 -- start start 2021-02-10T11:40:00.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.353 7f2f8a7e7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2f840d1880 0x7f2f841b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:00.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.353 7f2f8a7e7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2f841b0af0 0x7f2f84108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:00.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.353 7f2f8a7e7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2f840d1c80 0x7f2f84108f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:00.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.353 7f2f8a7e7700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f840fe1d0 con 0x7f2f841b0af0 2021-02-10T11:40:00.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.353 7f2f8a7e7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f84109a70 con 0x7f2f840d1880 2021-02-10T11:40:00.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.353 7f2f8a7e7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f84088070 con 0x7f2f840d1c80 2021-02-10T11:40:00.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.353 7f2f837fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2f841b0af0 0x7f2f84108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:00.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.353 7f2f837fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2f841b0af0 0x7f2f84108950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56122/0 (socket says 172.21.2.101:56122) 2021-02-10T11:40:00.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.353 7f2f837fe700 1 -- 172.21.2.101:0/4139130522 learned_addr learned my addr 172.21.2.101:0/4139130522 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:00.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.354 7f2f837fe700 1 -- 172.21.2.101:0/4139130522 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2f840d1c80 msgr2=0x7f2f84108f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:40:00.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.354 7f2f837fe700 1 --2- 172.21.2.101:0/4139130522 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2f840d1c80 0x7f2f84108f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:00.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.354 7f2f837fe700 1 -- 172.21.2.101:0/4139130522 >> v2:172.21.2.108:3300/0 conn(0x7f2f840d1880 msgr2=0x7f2f841b04d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:40:00.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.354 7f2f837fe700 1 --2- 172.21.2.101:0/4139130522 >> v2:172.21.2.108:3300/0 conn(0x7f2f840d1880 0x7f2f841b04d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:00.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.354 7f2f837fe700 1 -- 172.21.2.101:0/4139130522 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f84109590 con 0x7f2f841b0af0 2021-02-10T11:40:00.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.354 7f2f837fe700 1 --2- 172.21.2.101:0/4139130522 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2f841b0af0 0x7f2f84108950 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 rx=0x7f2f78006700 tx=0x7f2f78009b80).ready entity=mon.0 client_cookie=e4b556cae7d872c4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:00.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.354 7f2f82ffd700 1 -- 172.21.2.101:0/4139130522 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2f7800c0c0 con 0x7f2f841b0af0 2021-02-10T11:40:00.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.354 7f2f82ffd700 1 -- 172.21.2.101:0/4139130522 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7800c340 con 0x7f2f841b0af0 2021-02-10T11:40:00.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.354 7f2f82ffd700 1 -- 172.21.2.101:0/4139130522 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2f7800c860 con 0x7f2f841b0af0 2021-02-10T11:40:00.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.354 7f2f8a7e7700 1 -- 172.21.2.101:0/4139130522 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2f841b0af0 msgr2=0x7f2f84108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:00.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.354 7f2f8a7e7700 1 --2- 172.21.2.101:0/4139130522 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2f841b0af0 0x7f2f84108950 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 rx=0x7f2f78006700 tx=0x7f2f78009b80).stop 2021-02-10T11:40:00.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.355 7f2f8a7e7700 1 -- 172.21.2.101:0/4139130522 shutdown_connections 2021-02-10T11:40:00.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.355 7f2f8a7e7700 1 --2- 172.21.2.101:0/4139130522 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2f840d1c80 0x7f2f84108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:00.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.355 7f2f8a7e7700 1 --2- 172.21.2.101:0/4139130522 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2f841b0af0 0x7f2f84108950 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:00.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.355 7f2f8a7e7700 1 --2- 172.21.2.101:0/4139130522 >> v2:172.21.2.108:3300/0 conn(0x7f2f840d1880 0x7f2f841b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:00.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.355 7f2f8a7e7700 1 -- 172.21.2.101:0/4139130522 >> 172.21.2.101:0/4139130522 conn(0x7f2f840ce430 msgr2=0x7f2f840d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:00.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.355 7f2f8a7e7700 1 -- 172.21.2.101:0/4139130522 shutdown_connections 2021-02-10T11:40:00.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.355 7f2f8a7e7700 1 -- 172.21.2.101:0/4139130522 wait complete. 2021-02-10T11:40:00.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.369 7f2f8a7e7700 1 Processor -- start 2021-02-10T11:40:00.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.369 7f2f8a7e7700 1 -- start start 2021-02-10T11:40:00.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.370 7f2f8a7e7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2f841b9470 0x7f2f841ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:00.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.370 7f2f8a7e7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2f841aecf0 0x7f2f840957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:00.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.370 7f2f8a7e7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2f841af0f0 0x7f2f84095e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:00.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.370 7f2f8a7e7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f84096d70 con 0x7f2f841aecf0 2021-02-10T11:40:00.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.370 7f2f8a7e7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f840a7780 con 0x7f2f841af0f0 2021-02-10T11:40:00.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.370 7f2f8a7e7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f841b9870 con 0x7f2f841b9470 2021-02-10T11:40:00.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.370 7f2f837fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2f841aecf0 0x7f2f840957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:00.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.370 7f2f837fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2f841aecf0 0x7f2f840957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56130/0 (socket says 172.21.2.101:56130) 2021-02-10T11:40:00.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.370 7f2f837fe700 1 -- 172.21.2.101:0/3518144058 learned_addr learned my addr 172.21.2.101:0/3518144058 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:00.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.370 7f2f837fe700 1 -- 172.21.2.101:0/3518144058 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2f841b9470 msgr2=0x7f2f841ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:00.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.370 7f2f837fe700 1 --2- 172.21.2.101:0/3518144058 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2f841b9470 0x7f2f841ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:00.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.370 7f2f837fe700 1 -- 172.21.2.101:0/3518144058 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2f841af0f0 msgr2=0x7f2f84095e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:00.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.370 7f2f837fe700 1 --2- 172.21.2.101:0/3518144058 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2f841af0f0 0x7f2f84095e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:00.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.370 7f2f837fe700 1 -- 172.21.2.101:0/3518144058 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8419e6b0 con 0x7f2f841aecf0 2021-02-10T11:40:00.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.370 7f2f837fe700 1 --2- 172.21.2.101:0/3518144058 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2f841aecf0 0x7f2f840957f0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 rx=0x7f2f7800b5e0 tx=0x7f2f7800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:00.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.371 7f2f80ff9700 1 -- 172.21.2.101:0/3518144058 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2f7800bd70 con 0x7f2f841aecf0 2021-02-10T11:40:00.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.371 7f2f80ff9700 1 -- 172.21.2.101:0/3518144058 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f78002030 con 0x7f2f841aecf0 2021-02-10T11:40:00.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.371 7f2f80ff9700 1 -- 172.21.2.101:0/3518144058 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2f78002550 con 0x7f2f841aecf0 2021-02-10T11:40:00.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.371 7f2f8a7e7700 1 -- 172.21.2.101:0/3518144058 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8419e6b0 con 0x7f2f841aecf0 2021-02-10T11:40:00.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.371 7f2f8a7e7700 1 -- 172.21.2.101:0/3518144058 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8419e6b0 con 0x7f2f841aecf0 2021-02-10T11:40:00.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.371 7f2f80ff9700 1 -- 172.21.2.101:0/3518144058 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2f78002210 con 0x7f2f841aecf0 2021-02-10T11:40:00.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.371 7f2f80ff9700 1 --2- 172.21.2.101:0/3518144058 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f64018730 0x7f2f6401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:00.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.371 7f2f80ff9700 1 -- 172.21.2.101:0/3518144058 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2f780325e0 con 0x7f2f841aecf0 2021-02-10T11:40:00.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.371 7f2f83fff700 1 --2- 172.21.2.101:0/3518144058 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f64018730 0x7f2f6401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:00.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.372 7f2f83fff700 1 --2- 172.21.2.101:0/3518144058 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f64018730 0x7f2f6401ac10 crc :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:00.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.403 7f2f8a7e7700 1 -- 172.21.2.101:0/3518144058 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f840829a0 con 0x7f2f841aecf0 2021-02-10T11:40:00.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.405 7f2f80ff9700 1 -- 172.21.2.101:0/3518144058 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2f78002240 con 0x7f2f841aecf0 2021-02-10T11:40:00.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.535 7f2f8a7e7700 1 -- 172.21.2.101:0/3518144058 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f2f840a7780 con 0x7f2f841aecf0 2021-02-10T11:40:00.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.539 7f2f80ff9700 1 -- 172.21.2.101:0/3518144058 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v62) v1 ==== 184+0+0 (secure 0 0 0) 0x7f2f7801f4a0 con 0x7f2f841aecf0 2021-02-10T11:40:00.545 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.1 2021-02-10T11:40:00.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.544 7f2f8a7e7700 1 -- 172.21.2.101:0/3518144058 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f64018730 msgr2=0x7f2f6401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:00.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.544 7f2f8a7e7700 1 --2- 172.21.2.101:0/3518144058 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f64018730 0x7f2f6401ac10 crc :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:00.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.544 7f2f8a7e7700 1 -- 172.21.2.101:0/3518144058 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2f841aecf0 msgr2=0x7f2f840957f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:00.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.544 7f2f8a7e7700 1 --2- 172.21.2.101:0/3518144058 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2f841aecf0 0x7f2f840957f0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 rx=0x7f2f7800b5e0 tx=0x7f2f7800b980).stop 2021-02-10T11:40:00.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.544 7f2f8a7e7700 1 -- 172.21.2.101:0/3518144058 shutdown_connections 2021-02-10T11:40:00.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.544 7f2f8a7e7700 1 --2- 172.21.2.101:0/3518144058 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2f841af0f0 0x7f2f84095e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:00.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.544 7f2f8a7e7700 1 --2- 172.21.2.101:0/3518144058 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2f841aecf0 0x7f2f840957f0 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:00.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.544 7f2f8a7e7700 1 --2- 172.21.2.101:0/3518144058 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2f64018730 0x7f2f6401ac10 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:00.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.544 7f2f8a7e7700 1 --2- 172.21.2.101:0/3518144058 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2f841b9470 0x7f2f841ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:00.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.544 7f2f8a7e7700 1 -- 172.21.2.101:0/3518144058 >> 172.21.2.101:0/3518144058 conn(0x7f2f840845b0 msgr2=0x7f2f8418c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:00.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.544 7f2f8a7e7700 1 -- 172.21.2.101:0/3518144058 shutdown_connections 2021-02-10T11:40:00.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.544 7f2f8a7e7700 1 -- 172.21.2.101:0/3518144058 wait complete. 2021-02-10T11:40:00.555 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:40:00.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.965 7f1b86117700 1 Processor -- start 2021-02-10T11:40:00.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.965 7f1b86117700 1 -- start start 2021-02-10T11:40:00.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.966 7f1b86117700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b800d1880 0x7f1b801b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:00.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.966 7f1b86117700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1b801b0af0 0x7f1b80108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:00.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.966 7f1b86117700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1b800d1c80 0x7f1b80108f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:00.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.966 7f1b86117700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b800e5f00 con 0x7f1b801b0af0 2021-02-10T11:40:00.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.966 7f1b86117700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b80109a70 con 0x7f1b800d1c80 2021-02-10T11:40:00.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.966 7f1b86117700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b80088070 con 0x7f1b800d1880 2021-02-10T11:40:00.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.966 7f1b7effd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1b801b0af0 0x7f1b80108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:00.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.966 7f1b7effd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1b801b0af0 0x7f1b80108950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56134/0 (socket says 172.21.2.101:56134) 2021-02-10T11:40:00.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.966 7f1b7effd700 1 -- 172.21.2.101:0/140181452 learned_addr learned my addr 172.21.2.101:0/140181452 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:00.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.966 7f1b7f7fe700 1 --2- 172.21.2.101:0/140181452 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b800d1880 0x7f1b801b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:00.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.966 7f1b7ffff700 1 --2- 172.21.2.101:0/140181452 >> v2:172.21.2.108:3300/0 conn(0x7f1b800d1c80 0x7f1b80108f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:00.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.967 7f1b7effd700 1 -- 172.21.2.101:0/140181452 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b800d1880 msgr2=0x7f1b801b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:00.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.967 7f1b7effd700 1 --2- 172.21.2.101:0/140181452 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b800d1880 0x7f1b801b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:00.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.967 7f1b7effd700 1 -- 172.21.2.101:0/140181452 >> v2:172.21.2.108:3300/0 conn(0x7f1b800d1c80 msgr2=0x7f1b80108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:00.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.967 7f1b7effd700 1 --2- 172.21.2.101:0/140181452 >> v2:172.21.2.108:3300/0 conn(0x7f1b800d1c80 0x7f1b80108f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:00.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.967 7f1b7effd700 1 -- 172.21.2.101:0/140181452 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b80109590 con 0x7f1b801b0af0 2021-02-10T11:40:00.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.967 7f1b7effd700 1 --2- 172.21.2.101:0/140181452 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b801b0af0 0x7f1b80108950 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 rx=0x7f1b70006700 tx=0x7f1b70009b80).ready entity=mon.0 client_cookie=261c308570fb017b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:00.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.967 7f1b7e7fc700 1 -- 172.21.2.101:0/140181452 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1b7000c0c0 con 0x7f1b801b0af0 2021-02-10T11:40:00.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.967 7f1b7e7fc700 1 -- 172.21.2.101:0/140181452 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7000c340 con 0x7f1b801b0af0 2021-02-10T11:40:00.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.967 7f1b7e7fc700 1 -- 172.21.2.101:0/140181452 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1b7000c860 con 0x7f1b801b0af0 2021-02-10T11:40:00.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.967 7f1b86117700 1 -- 172.21.2.101:0/140181452 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b801b0af0 msgr2=0x7f1b80108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:00.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.967 7f1b86117700 1 --2- 172.21.2.101:0/140181452 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b801b0af0 0x7f1b80108950 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 rx=0x7f1b70006700 tx=0x7f1b70009b80).stop 2021-02-10T11:40:00.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.968 7f1b86117700 1 -- 172.21.2.101:0/140181452 shutdown_connections 2021-02-10T11:40:00.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.968 7f1b86117700 1 --2- 172.21.2.101:0/140181452 >> v2:172.21.2.108:3300/0 conn(0x7f1b800d1c80 0x7f1b80108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:00.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.968 7f1b86117700 1 --2- 172.21.2.101:0/140181452 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b801b0af0 0x7f1b80108950 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:00.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.968 7f1b86117700 1 --2- 172.21.2.101:0/140181452 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b800d1880 0x7f1b801b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:00.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.968 7f1b86117700 1 -- 172.21.2.101:0/140181452 >> 172.21.2.101:0/140181452 conn(0x7f1b800ce430 msgr2=0x7f1b800d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:00.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.968 7f1b86117700 1 -- 172.21.2.101:0/140181452 shutdown_connections 2021-02-10T11:40:00.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.968 7f1b86117700 1 -- 172.21.2.101:0/140181452 wait complete. 2021-02-10T11:40:00.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.983 7f1b86117700 1 Processor -- start 2021-02-10T11:40:00.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.983 7f1b86117700 1 -- start start 2021-02-10T11:40:00.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.983 7f1b86117700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b80085300 0x7f1b800bf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:00.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.983 7f1b86117700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b800bf7d0 0x7f1b80085fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:00.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.983 7f1b86117700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1b800bfbd0 0x7f1b800865d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:00.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.983 7f1b86117700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b80084d80 con 0x7f1b80085300 2021-02-10T11:40:00.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.983 7f1b86117700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b80085700 con 0x7f1b800bfbd0 2021-02-10T11:40:00.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.983 7f1b86117700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b80086bf0 con 0x7f1b800bf7d0 2021-02-10T11:40:00.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.984 7f1b7f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b80085300 0x7f1b800bf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:00.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.984 7f1b7f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b80085300 0x7f1b800bf1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56140/0 (socket says 172.21.2.101:56140) 2021-02-10T11:40:00.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.984 7f1b7f7fe700 1 -- 172.21.2.101:0/963408984 learned_addr learned my addr 172.21.2.101:0/963408984 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:00.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.984 7f1b7f7fe700 1 -- 172.21.2.101:0/963408984 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b800bf7d0 msgr2=0x7f1b80085fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:00.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.984 7f1b7f7fe700 1 --2- 172.21.2.101:0/963408984 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b800bf7d0 0x7f1b80085fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:00.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.984 7f1b7f7fe700 1 -- 172.21.2.101:0/963408984 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1b800bfbd0 msgr2=0x7f1b800865d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:00.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.984 7f1b7f7fe700 1 --2- 172.21.2.101:0/963408984 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1b800bfbd0 0x7f1b800865d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:00.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.984 7f1b7f7fe700 1 -- 172.21.2.101:0/963408984 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b80086e10 con 0x7f1b80085300 2021-02-10T11:40:00.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.984 7f1b7f7fe700 1 --2- 172.21.2.101:0/963408984 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b80085300 0x7f1b800bf1b0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 rx=0x7f1b6800b560 tx=0x7f1b6800b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:00.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.984 7f1b77fff700 1 -- 172.21.2.101:0/963408984 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1b6800bcf0 con 0x7f1b80085300 2021-02-10T11:40:00.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.984 7f1b77fff700 1 -- 172.21.2.101:0/963408984 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b68000c90 con 0x7f1b80085300 2021-02-10T11:40:00.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.984 7f1b86117700 1 -- 172.21.2.101:0/963408984 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b80086e10 con 0x7f1b80085300 2021-02-10T11:40:00.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.984 7f1b86117700 1 -- 172.21.2.101:0/963408984 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b800873a0 con 0x7f1b80085300 2021-02-10T11:40:00.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.984 7f1b77fff700 1 -- 172.21.2.101:0/963408984 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1b680022b0 con 0x7f1b80085300 2021-02-10T11:40:00.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.985 7f1b77fff700 1 -- 172.21.2.101:0/963408984 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1b68006b00 con 0x7f1b80085300 2021-02-10T11:40:00.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.985 7f1b77fff700 1 --2- 172.21.2.101:0/963408984 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b640186c0 0x7f1b6401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:00.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.985 7f1b77fff700 1 -- 172.21.2.101:0/963408984 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1b68032590 con 0x7f1b80085300 2021-02-10T11:40:00.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.985 7f1b7effd700 1 --2- 172.21.2.101:0/963408984 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b640186c0 0x7f1b6401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:00.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:00.985 7f1b7effd700 1 --2- 172.21.2.101:0/963408984 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b640186c0 0x7f1b6401abc0 crc :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:01.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.016 7f1b86117700 1 -- 172.21.2.101:0/963408984 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b800829a0 con 0x7f1b80085300 2021-02-10T11:40:01.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.019 7f1b77fff700 1 -- 172.21.2.101:0/963408984 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1b68000940 con 0x7f1b80085300 2021-02-10T11:40:01.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.151 7f1b86117700 1 -- 172.21.2.101:0/963408984 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f1b80109a70 con 0x7f1b80085300 2021-02-10T11:40:01.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.153 7f1b77fff700 1 -- 172.21.2.101:0/963408984 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v63) v1 ==== 184+0+0 (secure 0 0 0) 0x7f1b68006b00 con 0x7f1b80085300 2021-02-10T11:40:01.157 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.2 2021-02-10T11:40:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.155 7f1b86117700 1 -- 172.21.2.101:0/963408984 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b640186c0 msgr2=0x7f1b6401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.155 7f1b86117700 1 --2- 172.21.2.101:0/963408984 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b640186c0 0x7f1b6401abc0 crc :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.155 7f1b86117700 1 -- 172.21.2.101:0/963408984 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b80085300 msgr2=0x7f1b800bf1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.155 7f1b86117700 1 --2- 172.21.2.101:0/963408984 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b80085300 0x7f1b800bf1b0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 rx=0x7f1b6800b560 tx=0x7f1b6800b900).stop 2021-02-10T11:40:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.155 7f1b86117700 1 -- 172.21.2.101:0/963408984 shutdown_connections 2021-02-10T11:40:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.155 7f1b86117700 1 --2- 172.21.2.101:0/963408984 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b640186c0 0x7f1b6401abc0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.155 7f1b86117700 1 --2- 172.21.2.101:0/963408984 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b800bf7d0 0x7f1b80085fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.155 7f1b86117700 1 --2- 172.21.2.101:0/963408984 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1b800bfbd0 0x7f1b800865d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.155 7f1b86117700 1 --2- 172.21.2.101:0/963408984 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b80085300 0x7f1b800bf1b0 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.155 7f1b86117700 1 -- 172.21.2.101:0/963408984 >> 172.21.2.101:0/963408984 conn(0x7f1b800bd7a0 msgr2=0x7f1b800b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:01.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.156 7f1b86117700 1 -- 172.21.2.101:0/963408984 shutdown_connections 2021-02-10T11:40:01.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.156 7f1b86117700 1 -- 172.21.2.101:0/963408984 wait complete. 2021-02-10T11:40:01.167 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:40:01.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.587 7ff16ed0b700 1 Processor -- start 2021-02-10T11:40:01.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.587 7ff16ed0b700 1 -- start start 2021-02-10T11:40:01.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.588 7ff16ed0b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1680d1880 0x7ff1681b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:01.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.588 7ff16ed0b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff1681b0af0 0x7ff168108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:01.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.588 7ff16ed0b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff1680d1c80 0x7ff168108f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:01.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.588 7ff16ed0b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1680e5f00 con 0x7ff1680d1c80 2021-02-10T11:40:01.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.588 7ff16ed0b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff168109a70 con 0x7ff1681b0af0 2021-02-10T11:40:01.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.588 7ff16ed0b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff168088070 con 0x7ff1680d1880 2021-02-10T11:40:01.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.588 7ff16d328700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff1680d1c80 0x7ff168108f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:01.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.588 7ff16d328700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff1680d1c80 0x7ff168108f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56148/0 (socket says 172.21.2.101:56148) 2021-02-10T11:40:01.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.588 7ff16d328700 1 -- 172.21.2.101:0/1722420045 learned_addr learned my addr 172.21.2.101:0/1722420045 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:01.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.588 7ff16d328700 1 -- 172.21.2.101:0/1722420045 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1680d1880 msgr2=0x7ff1681b04d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:40:01.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.588 7ff16d328700 1 --2- 172.21.2.101:0/1722420045 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1680d1880 0x7ff1681b04d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:01.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.588 7ff16d328700 1 -- 172.21.2.101:0/1722420045 >> v2:172.21.2.108:3300/0 conn(0x7ff1681b0af0 msgr2=0x7ff168108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:01.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.588 7ff16d328700 1 --2- 172.21.2.101:0/1722420045 >> v2:172.21.2.108:3300/0 conn(0x7ff1681b0af0 0x7ff168108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:01.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.588 7ff16d328700 1 -- 172.21.2.101:0/1722420045 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff168109590 con 0x7ff1680d1c80 2021-02-10T11:40:01.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.588 7ff16d328700 1 --2- 172.21.2.101:0/1722420045 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1680d1c80 0x7ff168108f70 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 rx=0x7ff160006f00 tx=0x7ff160009f10).ready entity=mon.0 client_cookie=a7ac83633f1c5ccc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:01.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.588 7ff1677fe700 1 -- 172.21.2.101:0/1722420045 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff16000b3b0 con 0x7ff1680d1c80 2021-02-10T11:40:01.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.588 7ff1677fe700 1 -- 172.21.2.101:0/1722420045 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16000b630 con 0x7ff1680d1c80 2021-02-10T11:40:01.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.588 7ff1677fe700 1 -- 172.21.2.101:0/1722420045 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff16000bb10 con 0x7ff1680d1c80 2021-02-10T11:40:01.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.588 7ff16ed0b700 1 -- 172.21.2.101:0/1722420045 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1680d1c80 msgr2=0x7ff168108f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:01.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.588 7ff16ed0b700 1 --2- 172.21.2.101:0/1722420045 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1680d1c80 0x7ff168108f70 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 rx=0x7ff160006f00 tx=0x7ff160009f10).stop 2021-02-10T11:40:01.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.589 7ff16ed0b700 1 -- 172.21.2.101:0/1722420045 shutdown_connections 2021-02-10T11:40:01.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.589 7ff16ed0b700 1 --2- 172.21.2.101:0/1722420045 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1680d1c80 0x7ff168108f70 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:01.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.589 7ff16ed0b700 1 --2- 172.21.2.101:0/1722420045 >> v2:172.21.2.108:3300/0 conn(0x7ff1681b0af0 0x7ff168108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:01.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.589 7ff16ed0b700 1 --2- 172.21.2.101:0/1722420045 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1680d1880 0x7ff1681b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:01.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.589 7ff16ed0b700 1 -- 172.21.2.101:0/1722420045 >> 172.21.2.101:0/1722420045 conn(0x7ff1680ce430 msgr2=0x7ff1680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:01.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.589 7ff16ed0b700 1 -- 172.21.2.101:0/1722420045 shutdown_connections 2021-02-10T11:40:01.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.589 7ff16ed0b700 1 -- 172.21.2.101:0/1722420045 wait complete. 2021-02-10T11:40:01.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.604 7ff16ed0b700 1 Processor -- start 2021-02-10T11:40:01.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.604 7ff16ed0b700 1 -- start start 2021-02-10T11:40:01.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.604 7ff16ed0b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff168082b30 0x7ff168096e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:01.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.604 7ff16ed0b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1681a1cf0 0x7ff1681a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:01.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.604 7ff16ed0b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1680a90e0 0x7ff1680a94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:01.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.604 7ff16ed0b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff168097830 con 0x7ff1680a90e0 2021-02-10T11:40:01.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.604 7ff16ed0b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1681a2b50 con 0x7ff168082b30 2021-02-10T11:40:01.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.604 7ff16ed0b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1681a2740 con 0x7ff1681a1cf0 2021-02-10T11:40:01.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.605 7ff16d328700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1680a90e0 0x7ff1680a94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:01.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.605 7ff16d328700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1680a90e0 0x7ff1680a94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56156/0 (socket says 172.21.2.101:56156) 2021-02-10T11:40:01.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.605 7ff16d328700 1 -- 172.21.2.101:0/1601486124 learned_addr learned my addr 172.21.2.101:0/1601486124 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:01.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.605 7ff16cb27700 1 --2- 172.21.2.101:0/1601486124 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff168082b30 0x7ff168096e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:01.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.605 7ff167fff700 1 --2- 172.21.2.101:0/1601486124 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1681a1cf0 0x7ff1681a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:01.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.605 7ff16d328700 1 -- 172.21.2.101:0/1601486124 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1681a1cf0 msgr2=0x7ff1681a20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:01.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.605 7ff16d328700 1 --2- 172.21.2.101:0/1601486124 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1681a1cf0 0x7ff1681a20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:01.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.605 7ff16d328700 1 -- 172.21.2.101:0/1601486124 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff168082b30 msgr2=0x7ff168096e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:01.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.605 7ff16d328700 1 --2- 172.21.2.101:0/1601486124 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff168082b30 0x7ff168096e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:01.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.605 7ff16d328700 1 -- 172.21.2.101:0/1601486124 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1680a9b00 con 0x7ff1680a90e0 2021-02-10T11:40:01.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.605 7ff16d328700 1 --2- 172.21.2.101:0/1601486124 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1680a90e0 0x7ff1680a94e0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 rx=0x7ff16000b8e0 tx=0x7ff16000bc80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:01.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.605 7ff1657fa700 1 -- 172.21.2.101:0/1601486124 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff160009450 con 0x7ff1680a90e0 2021-02-10T11:40:01.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.605 7ff1657fa700 1 -- 172.21.2.101:0/1601486124 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1600058b0 con 0x7ff1680a90e0 2021-02-10T11:40:01.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.605 7ff1657fa700 1 -- 172.21.2.101:0/1601486124 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff160005d60 con 0x7ff1680a90e0 2021-02-10T11:40:01.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.605 7ff16ed0b700 1 -- 172.21.2.101:0/1601486124 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1680a9b00 con 0x7ff1680a90e0 2021-02-10T11:40:01.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.605 7ff16ed0b700 1 -- 172.21.2.101:0/1601486124 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1681a3060 con 0x7ff1680a90e0 2021-02-10T11:40:01.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.606 7ff1657fa700 1 -- 172.21.2.101:0/1601486124 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff160009450 con 0x7ff1680a90e0 2021-02-10T11:40:01.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.606 7ff1657fa700 1 --2- 172.21.2.101:0/1601486124 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff14c0186c0 0x7ff14c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:01.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.606 7ff1657fa700 1 -- 172.21.2.101:0/1601486124 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff160032920 con 0x7ff1680a90e0 2021-02-10T11:40:01.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.606 7ff16cb27700 1 --2- 172.21.2.101:0/1601486124 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff14c0186c0 0x7ff14c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:01.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.607 7ff16cb27700 1 --2- 172.21.2.101:0/1601486124 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff14c0186c0 0x7ff14c01abc0 crc :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:01.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.637 7ff16ed0b700 1 -- 172.21.2.101:0/1601486124 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1681a2710 con 0x7ff1680a90e0 2021-02-10T11:40:01.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.640 7ff1657fa700 1 -- 172.21.2.101:0/1601486124 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff16001f8b0 con 0x7ff1680a90e0 2021-02-10T11:40:01.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.770 7ff16ed0b700 1 -- 172.21.2.101:0/1601486124 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ff1681a2710 con 0x7ff1680a90e0 2021-02-10T11:40:01.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.774 7ff1657fa700 1 -- 172.21.2.101:0/1601486124 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v64) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff16001fac0 con 0x7ff1680a90e0 2021-02-10T11:40:01.780 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.3 2021-02-10T11:40:01.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.779 7ff16ed0b700 1 -- 172.21.2.101:0/1601486124 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff14c0186c0 msgr2=0x7ff14c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:01.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.779 7ff16ed0b700 1 --2- 172.21.2.101:0/1601486124 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff14c0186c0 0x7ff14c01abc0 crc :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:01.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.779 7ff16ed0b700 1 -- 172.21.2.101:0/1601486124 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1680a90e0 msgr2=0x7ff1680a94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:01.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.779 7ff16ed0b700 1 --2- 172.21.2.101:0/1601486124 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1680a90e0 0x7ff1680a94e0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 rx=0x7ff16000b8e0 tx=0x7ff16000bc80).stop 2021-02-10T11:40:01.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.779 7ff16ed0b700 1 -- 172.21.2.101:0/1601486124 shutdown_connections 2021-02-10T11:40:01.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.779 7ff16ed0b700 1 --2- 172.21.2.101:0/1601486124 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff14c0186c0 0x7ff14c01abc0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:01.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.779 7ff16ed0b700 1 --2- 172.21.2.101:0/1601486124 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1681a1cf0 0x7ff1681a20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:01.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.779 7ff16ed0b700 1 --2- 172.21.2.101:0/1601486124 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1680a90e0 0x7ff1680a94e0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:01.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.779 7ff16ed0b700 1 --2- 172.21.2.101:0/1601486124 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff168082b30 0x7ff168096e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:01.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.779 7ff16ed0b700 1 -- 172.21.2.101:0/1601486124 >> 172.21.2.101:0/1601486124 conn(0x7ff168191380 msgr2=0x7ff168187e80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:01.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.779 7ff16ed0b700 1 -- 172.21.2.101:0/1601486124 shutdown_connections 2021-02-10T11:40:01.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:01.779 7ff16ed0b700 1 -- 172.21.2.101:0/1601486124 wait complete. 2021-02-10T11:40:01.791 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:40:02.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.202 7fd55d5e0700 1 Processor -- start 2021-02-10T11:40:02.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.202 7fd55d5e0700 1 -- start start 2021-02-10T11:40:02.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.202 7fd55d5e0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5581b0870 0x7fd5580f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:02.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.202 7fd55d5e0700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd5580efe00 0x7fd5580f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:02.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.202 7fd55d5e0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5581b0c70 con 0x7fd5581b0470 2021-02-10T11:40:02.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.202 7fd55d5e0700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5580f0b50 con 0x7fd5580efe00 2021-02-10T11:40:02.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.202 7fd55d5e0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5580f0820 con 0x7fd5581b0870 2021-02-10T11:40:02.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.203 7fd5572bf700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fd5581b0470 0x7fd5580ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:50068/0 (socket says 172.21.2.101:50068) 2021-02-10T11:40:02.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.203 7fd5572bf700 1 -- 172.21.2.101:0/2025184329 learned_addr learned my addr 172.21.2.101:0/2025184329 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:02.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.203 7fd556abe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5581b0870 0x7fd5580f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:02.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.203 7fd557ac0700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd5580efe00 0x7fd5580f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:02.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.204 7fd557ac0700 1 -- 172.21.2.101:0/2025184329 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5581b0870 msgr2=0x7fd5580f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:02.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.204 7fd557ac0700 1 --2- 172.21.2.101:0/2025184329 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5581b0870 0x7fd5580f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:02.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.204 7fd557ac0700 1 -- 172.21.2.101:0/2025184329 >> v1:172.21.2.108:6789/0 conn(0x7fd5581b0470 legacy=0x7fd5580ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:02.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.204 7fd557ac0700 1 -- 172.21.2.101:0/2025184329 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5580e52e0 con 0x7fd5580efe00 2021-02-10T11:40:02.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.204 7fd556abe700 1 --2- 172.21.2.101:0/2025184329 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5581b0870 0x7fd5580f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:40:02.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.204 7fd557ac0700 1 --2- 172.21.2.101:0/2025184329 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd5580efe00 0x7fd5580f0200 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fd54c006ad0 tx=0x7fd54c009ad0).ready entity=mon.1 client_cookie=c3c082bd188fdaea server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:02.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.204 7fd5562bd700 1 -- 172.21.2.101:0/2025184329 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd54c000c10 con 0x7fd5580efe00 2021-02-10T11:40:02.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.204 7fd5562bd700 1 -- 172.21.2.101:0/2025184329 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54c006410 con 0x7fd5580efe00 2021-02-10T11:40:02.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.204 7fd5562bd700 1 -- 172.21.2.101:0/2025184329 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd54c0024d0 con 0x7fd5580efe00 2021-02-10T11:40:02.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.204 7fd55d5e0700 1 -- 172.21.2.101:0/2025184329 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd5580efe00 msgr2=0x7fd5580f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:02.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.204 7fd55d5e0700 1 --2- 172.21.2.101:0/2025184329 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd5580efe00 0x7fd5580f0200 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fd54c006ad0 tx=0x7fd54c009ad0).stop 2021-02-10T11:40:02.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.205 7fd55d5e0700 1 -- 172.21.2.101:0/2025184329 shutdown_connections 2021-02-10T11:40:02.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.205 7fd55d5e0700 1 --2- 172.21.2.101:0/2025184329 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5581b0870 0x7fd5580f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:02.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.205 7fd55d5e0700 1 --2- 172.21.2.101:0/2025184329 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd5580efe00 0x7fd5580f0200 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:02.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.205 7fd55d5e0700 1 -- 172.21.2.101:0/2025184329 >> 172.21.2.101:0/2025184329 conn(0x7fd5580ce3e0 msgr2=0x7fd5580e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:02.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.205 7fd55d5e0700 1 -- 172.21.2.101:0/2025184329 shutdown_connections 2021-02-10T11:40:02.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.205 7fd55d5e0700 1 -- 172.21.2.101:0/2025184329 wait complete. 2021-02-10T11:40:02.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.219 7fd55d5e0700 1 Processor -- start 2021-02-10T11:40:02.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.219 7fd55d5e0700 1 -- start start 2021-02-10T11:40:02.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.220 7fd55d5e0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd5581ac1a0 0x7fd55818e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:02.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.220 7fd55d5e0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5581ab670 0x7fd55818ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:02.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.220 7fd55d5e0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd55819fee0 0x7fd5581a02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:02.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.220 7fd55d5e0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5581abc20 con 0x7fd55819fee0 2021-02-10T11:40:02.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.220 7fd55d5e0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5581ac5a0 con 0x7fd5581ac1a0 2021-02-10T11:40:02.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.220 7fd55d5e0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5581991c0 con 0x7fd5581ab670 2021-02-10T11:40:02.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.220 7fd5572bf700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd5581ac1a0 0x7fd55818e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:02.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.220 7fd557ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd55819fee0 0x7fd5581a02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:02.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.220 7fd557ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd55819fee0 0x7fd5581a02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56168/0 (socket says 172.21.2.101:56168) 2021-02-10T11:40:02.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.220 7fd557ac0700 1 -- 172.21.2.101:0/4095608941 learned_addr learned my addr 172.21.2.101:0/4095608941 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:02.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.220 7fd5572bf700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd5581ac1a0 0x7fd55818e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:38422/0 (socket says 172.21.2.101:38422) 2021-02-10T11:40:02.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.220 7fd556abe700 1 --2- 172.21.2.101:0/4095608941 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5581ab670 0x7fd55818ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:02.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.221 7fd5572bf700 1 -- 172.21.2.101:0/4095608941 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5581ab670 msgr2=0x7fd55818ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:02.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.221 7fd5572bf700 1 --2- 172.21.2.101:0/4095608941 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5581ab670 0x7fd55818ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:02.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.221 7fd5572bf700 1 -- 172.21.2.101:0/4095608941 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd55819fee0 msgr2=0x7fd5581a02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:02.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.221 7fd5572bf700 1 --2- 172.21.2.101:0/4095608941 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd55819fee0 0x7fd5581a02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:02.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.221 7fd5572bf700 1 -- 172.21.2.101:0/4095608941 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd558198d10 con 0x7fd5581ac1a0 2021-02-10T11:40:02.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.221 7fd557ac0700 1 --2- 172.21.2.101:0/4095608941 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd55819fee0 0x7fd5581a02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:40:02.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.221 7fd5572bf700 1 --2- 172.21.2.101:0/4095608941 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd5581ac1a0 0x7fd55818e6f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7fd550007f70 tx=0x7fd55000abf0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:02.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.221 7fd53ffff700 1 -- 172.21.2.101:0/4095608941 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd550000f90 con 0x7fd5581ac1a0 2021-02-10T11:40:02.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.221 7fd53ffff700 1 -- 172.21.2.101:0/4095608941 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd550007280 con 0x7fd5581ac1a0 2021-02-10T11:40:02.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.221 7fd53ffff700 1 -- 172.21.2.101:0/4095608941 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd5500046a0 con 0x7fd5581ac1a0 2021-02-10T11:40:02.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.221 7fd55d5e0700 1 -- 172.21.2.101:0/4095608941 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd558198d10 con 0x7fd5581ac1a0 2021-02-10T11:40:02.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.221 7fd55d5e0700 1 -- 172.21.2.101:0/4095608941 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd558198d10 con 0x7fd5581ac1a0 2021-02-10T11:40:02.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.222 7fd53ffff700 1 -- 172.21.2.101:0/4095608941 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd550000f90 con 0x7fd5581ac1a0 2021-02-10T11:40:02.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.222 7fd53ffff700 1 --2- 172.21.2.101:0/4095608941 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd5380186c0 0x7fd53801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:02.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.222 7fd53ffff700 1 -- 172.21.2.101:0/4095608941 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd5500326a0 con 0x7fd5581ac1a0 2021-02-10T11:40:02.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.222 7fd556abe700 1 --2- 172.21.2.101:0/4095608941 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd5380186c0 0x7fd53801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:02.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.222 7fd556abe700 1 --2- 172.21.2.101:0/4095608941 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd5380186c0 0x7fd53801abc0 crc :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:02.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.252 7fd55d5e0700 1 -- 172.21.2.101:0/4095608941 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd558105a40 con 0x7fd5581ac1a0 2021-02-10T11:40:02.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.256 7fd53ffff700 1 -- 172.21.2.101:0/4095608941 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd550007360 con 0x7fd5581ac1a0 2021-02-10T11:40:02.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.387 7fd55d5e0700 1 -- 172.21.2.101:0/4095608941 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5580f0b50 con 0x7fd5581ac1a0 2021-02-10T11:40:02.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.389 7fd53ffff700 1 -- 172.21.2.101:0/4095608941 <== mon.1 v2:172.21.2.108:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+2107 (secure 0 0 0) 0x7fd550000f90 con 0x7fd5581ac1a0 2021-02-10T11:40:02.392 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:40:02.392 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":79,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10125,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/3121560299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3121560299},{"type":"v1","addr":"172.21.2.101:6835","nonce":3121560299}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":76}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:39:56.054150","modified":"2021-02-10 11:39:58.060269","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10122},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10122":{"gid":10122,"name":"b","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/1690286235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":1690286235},{"type":"v1","addr":"172.21.2.108:6833","nonce":1690286235}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2021-02-10T11:40:02.392 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 79 2021-02-10T11:40:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.390 7fd55d5e0700 1 -- 172.21.2.101:0/4095608941 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd5380186c0 msgr2=0x7fd53801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.390 7fd55d5e0700 1 --2- 172.21.2.101:0/4095608941 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd5380186c0 0x7fd53801abc0 crc :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.390 7fd55d5e0700 1 -- 172.21.2.101:0/4095608941 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd5581ac1a0 msgr2=0x7fd55818e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.390 7fd55d5e0700 1 --2- 172.21.2.101:0/4095608941 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd5581ac1a0 0x7fd55818e6f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7fd550007f70 tx=0x7fd55000abf0).stop 2021-02-10T11:40:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.390 7fd55d5e0700 1 -- 172.21.2.101:0/4095608941 shutdown_connections 2021-02-10T11:40:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.390 7fd55d5e0700 1 --2- 172.21.2.101:0/4095608941 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd5380186c0 0x7fd53801abc0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.390 7fd55d5e0700 1 --2- 172.21.2.101:0/4095608941 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5581ab670 0x7fd55818ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.390 7fd55d5e0700 1 --2- 172.21.2.101:0/4095608941 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd55819fee0 0x7fd5581a02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.390 7fd55d5e0700 1 --2- 172.21.2.101:0/4095608941 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd5581ac1a0 0x7fd55818e6f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.390 7fd55d5e0700 1 -- 172.21.2.101:0/4095608941 >> 172.21.2.101:0/4095608941 conn(0x7fd5581a37c0 msgr2=0x7fd5581b9230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:02.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.390 7fd55d5e0700 1 -- 172.21.2.101:0/4095608941 shutdown_connections 2021-02-10T11:40:02.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:02.390 7fd55d5e0700 1 -- 172.21.2.101:0/4095608941 wait complete. 2021-02-10T11:40:02.603 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 79, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:39:56.054150', 'modified': '2021-02-10 11:39:58.060269', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10122}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10122': {'gid': 10122, 'name': 'b', 'rank': 0, 'incarnation': 77, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/1690286235', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 1690286235}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 1690286235}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:40:02.603 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:40:02.603 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:40:03.002 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:40:03.002 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:40:03.002 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:40:03.002 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 10122, 2021-02-10T11:40:03.003 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:40:03.003 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:40:03.003 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 6.9436534449999998, 2021-02-10T11:40:03.003 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 79, 2021-02-10T11:40:03.003 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 103, 2021-02-10T11:40:03.003 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 103, 2021-02-10T11:40:03.003 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 12.579452482000001 2021-02-10T11:40:03.003 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:40:03.006 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 10122, "want_state": "up:active", "state": "up:active", "rank_uptime": 6.9436534449999998, "mdsmap_epoch": 79, "osdmap_epoch": 103, "osdmap_epoch_barrier": 103, "uptime": 12.579452482000001 } 2021-02-10T11:40:03.006 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:40:03.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.415 7f5eb3bfc700 1 Processor -- start 2021-02-10T11:40:03.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.415 7f5eb3bfc700 1 -- start start 2021-02-10T11:40:03.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.415 7f5eb3bfc700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5eac1b0470 0x7f5eac1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:03.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.415 7f5eb3bfc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eac0ef610 0x7f5eac0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:03.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.415 7f5eb3bfc700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eac0f0b50 con 0x7f5eac0f0030 2021-02-10T11:40:03.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.415 7f5eb3bfc700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eac0e52e0 con 0x7f5eac1b0470 2021-02-10T11:40:03.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.415 7f5eb3bfc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac0e54d0 con 0x7f5eac0ef610 2021-02-10T11:40:03.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.415 7f5eb1a18700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5eac1b0470 0x7f5eac1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:03.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.415 7f5eb1a18700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5eac1b0470 0x7f5eac1b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56174/0 (socket says 172.21.2.101:56174) 2021-02-10T11:40:03.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.415 7f5eb1a18700 1 -- 172.21.2.101:0/4283977868 learned_addr learned my addr 172.21.2.101:0/4283977868 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:03.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.416 7f5eb1a18700 1 -- 172.21.2.101:0/4283977868 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eac0ef610 msgr2=0x7f5eac0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:03.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.416 7f5eb1a18700 1 --2- 172.21.2.101:0/4283977868 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eac0ef610 0x7f5eac0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:03.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.416 7f5eb1a18700 1 -- 172.21.2.101:0/4283977868 >> v1:172.21.2.108:6789/0 conn(0x7f5eac0f0030 legacy=0x7f5eac0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:03.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.416 7f5eb1a18700 1 -- 172.21.2.101:0/4283977868 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eac0f0430 con 0x7f5eac1b0470 2021-02-10T11:40:03.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.416 7f5eb1a18700 1 --2- 172.21.2.101:0/4283977868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eac1b0470 0x7f5eac1b0870 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 rx=0x7f5ea8006700 tx=0x7f5ea8009b80).ready entity=mon.0 client_cookie=2d07eaa1115ae707 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:03.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.416 7f5eb0a16700 1 -- 172.21.2.101:0/4283977868 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5ea800c0c0 con 0x7f5eac1b0470 2021-02-10T11:40:03.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.416 7f5eb0a16700 1 -- 172.21.2.101:0/4283977868 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea800c340 con 0x7f5eac1b0470 2021-02-10T11:40:03.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.416 7f5eb0a16700 1 -- 172.21.2.101:0/4283977868 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5ea800c860 con 0x7f5eac1b0470 2021-02-10T11:40:03.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.416 7f5eb3bfc700 1 -- 172.21.2.101:0/4283977868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eac1b0470 msgr2=0x7f5eac1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:03.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.416 7f5eb3bfc700 1 --2- 172.21.2.101:0/4283977868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eac1b0470 0x7f5eac1b0870 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 rx=0x7f5ea8006700 tx=0x7f5ea8009b80).stop 2021-02-10T11:40:03.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.417 7f5eb3bfc700 1 -- 172.21.2.101:0/4283977868 shutdown_connections 2021-02-10T11:40:03.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.417 7f5eb3bfc700 1 --2- 172.21.2.101:0/4283977868 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eac0ef610 0x7f5eac0efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:03.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.417 7f5eb3bfc700 1 --2- 172.21.2.101:0/4283977868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eac1b0470 0x7f5eac1b0870 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:03.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.417 7f5eb3bfc700 1 -- 172.21.2.101:0/4283977868 >> 172.21.2.101:0/4283977868 conn(0x7f5eac0ce3e0 msgr2=0x7f5eac0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:03.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.417 7f5eb3bfc700 1 -- 172.21.2.101:0/4283977868 shutdown_connections 2021-02-10T11:40:03.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.417 7f5eb3bfc700 1 -- 172.21.2.101:0/4283977868 wait complete. 2021-02-10T11:40:03.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.432 7f5eb3bfc700 1 Processor -- start 2021-02-10T11:40:03.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.432 7f5eb3bfc700 1 -- start start 2021-02-10T11:40:03.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.432 7f5eb3bfc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eac0b0970 0x7f5eac092ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:03.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.432 7f5eb3bfc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eac0934f0 0x7f5eac094dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:03.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.432 7f5eb3bfc700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5eac0938f0 0x7f5eac0953e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:03.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.432 7f5eb3bfc700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac195ad0 con 0x7f5eac0b0970 2021-02-10T11:40:03.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.432 7f5eb3bfc700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac095f40 con 0x7f5eac0938f0 2021-02-10T11:40:03.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.432 7f5eb3bfc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac095a30 con 0x7f5eac0934f0 2021-02-10T11:40:03.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.432 7f5eb1a18700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eac0b0970 0x7f5eac092ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:03.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.432 7f5eb1a18700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eac0b0970 0x7f5eac092ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56180/0 (socket says 172.21.2.101:56180) 2021-02-10T11:40:03.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.432 7f5eb1a18700 1 -- 172.21.2.101:0/4032385825 learned_addr learned my addr 172.21.2.101:0/4032385825 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:03.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.433 7f5eb1a18700 1 -- 172.21.2.101:0/4032385825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eac0934f0 msgr2=0x7f5eac094dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:03.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.433 7f5eb1a18700 1 --2- 172.21.2.101:0/4032385825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eac0934f0 0x7f5eac094dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:03.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.433 7f5eb1a18700 1 -- 172.21.2.101:0/4032385825 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5eac0938f0 msgr2=0x7f5eac0953e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:03.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.433 7f5eb1a18700 1 --2- 172.21.2.101:0/4032385825 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5eac0938f0 0x7f5eac0953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:03.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.433 7f5eb1a18700 1 -- 172.21.2.101:0/4032385825 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eac095c50 con 0x7f5eac0b0970 2021-02-10T11:40:03.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.433 7f5eb1a18700 1 --2- 172.21.2.101:0/4032385825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eac0b0970 0x7f5eac092ed0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 rx=0x7f5ea800b580 tx=0x7f5ea800b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:03.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.433 7f5e9e7fc700 1 -- 172.21.2.101:0/4032385825 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5ea800bd10 con 0x7f5eac0b0970 2021-02-10T11:40:03.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.433 7f5e9e7fc700 1 -- 172.21.2.101:0/4032385825 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea8002030 con 0x7f5eac0b0970 2021-02-10T11:40:03.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.433 7f5e9e7fc700 1 -- 172.21.2.101:0/4032385825 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5ea8002490 con 0x7f5eac0b0970 2021-02-10T11:40:03.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.433 7f5eb3bfc700 1 -- 172.21.2.101:0/4032385825 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eac095c50 con 0x7f5eac0b0970 2021-02-10T11:40:03.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.433 7f5eb3bfc700 1 -- 172.21.2.101:0/4032385825 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eac095c50 con 0x7f5eac0b0970 2021-02-10T11:40:03.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.434 7f5e9e7fc700 1 -- 172.21.2.101:0/4032385825 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5ea8000c90 con 0x7f5eac0b0970 2021-02-10T11:40:03.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.434 7f5e9e7fc700 1 --2- 172.21.2.101:0/4032385825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5e940186c0 0x7f5e9401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:03.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.434 7f5e9e7fc700 1 -- 172.21.2.101:0/4032385825 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5ea8032590 con 0x7f5eac0b0970 2021-02-10T11:40:03.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.434 7f5eb1217700 1 --2- 172.21.2.101:0/4032385825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5e940186c0 0x7f5e9401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:03.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.434 7f5eb1217700 1 --2- 172.21.2.101:0/4032385825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5e940186c0 0x7f5e9401abc0 crc :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:03.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.465 7f5eb3bfc700 1 -- 172.21.2.101:0/4032385825 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eac0e52e0 con 0x7f5eac0b0970 2021-02-10T11:40:03.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.467 7f5e9e7fc700 1 -- 172.21.2.101:0/4032385825 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5ea8030d40 con 0x7f5eac0b0970 2021-02-10T11:40:03.577 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:40:03.580 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:40:03.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.593 7f5eb3bfc700 1 -- 172.21.2.101:0/4032385825 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5eac0e52e0 con 0x7f5eac0b0970 2021-02-10T11:40:03.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.594 7f5e9e7fc700 1 -- 172.21.2.101:0/4032385825 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+14005 (secure 0 0 0) 0x7f5ea8000c90 con 0x7f5eac0b0970 2021-02-10T11:40:03.598 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:40:03.598 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":103,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:39:57.056338","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:39:54.022810","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2021-02-10 11:39:55.095134","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:40:03.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.596 7f5eb3bfc700 1 -- 172.21.2.101:0/4032385825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5e940186c0 msgr2=0x7f5e9401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:03.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.596 7f5eb3bfc700 1 --2- 172.21.2.101:0/4032385825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5e940186c0 0x7f5e9401abc0 crc :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:03.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.596 7f5eb3bfc700 1 -- 172.21.2.101:0/4032385825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eac0b0970 msgr2=0x7f5eac092ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:03.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.596 7f5eb3bfc700 1 --2- 172.21.2.101:0/4032385825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eac0b0970 0x7f5eac092ed0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 rx=0x7f5ea800b580 tx=0x7f5ea800b920).stop 2021-02-10T11:40:03.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.596 7f5eb3bfc700 1 -- 172.21.2.101:0/4032385825 shutdown_connections 2021-02-10T11:40:03.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.596 7f5eb3bfc700 1 --2- 172.21.2.101:0/4032385825 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5e940186c0 0x7f5e9401abc0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:03.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.596 7f5eb3bfc700 1 --2- 172.21.2.101:0/4032385825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eac0934f0 0x7f5eac094dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:03.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.596 7f5eb3bfc700 1 --2- 172.21.2.101:0/4032385825 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5eac0938f0 0x7f5eac0953e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:03.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.596 7f5eb3bfc700 1 --2- 172.21.2.101:0/4032385825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eac0b0970 0x7f5eac092ed0 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:03.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.596 7f5eb3bfc700 1 -- 172.21.2.101:0/4032385825 >> 172.21.2.101:0/4032385825 conn(0x7f5eac19f7a0 msgr2=0x7f5eac0940e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:03.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.596 7f5eb3bfc700 1 -- 172.21.2.101:0/4032385825 shutdown_connections 2021-02-10T11:40:03.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.596 7f5eb3bfc700 1 -- 172.21.2.101:0/4032385825 wait complete. 2021-02-10T11:40:03.608 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:39:54.022810', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:39:55.095134', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:40:03.609 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:40:04.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.991 7fbb1d4e1700 1 Processor -- start 2021-02-10T11:40:04.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.991 7fbb1d4e1700 1 -- start start 2021-02-10T11:40:04.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.991 7fbb1d4e1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbb180d1870 0x7fbb181b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:04.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.991 7fbb1d4e1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb181b0ae0 0x7fbb18108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:04.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.991 7fbb1d4e1700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb180fe1c0 con 0x7fbb180d1c70 2021-02-10T11:40:04.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.991 7fbb1d4e1700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb18109a60 con 0x7fbb180d1870 2021-02-10T11:40:04.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.991 7fbb1d4e1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb18109750 con 0x7fbb181b0ae0 2021-02-10T11:40:04.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.992 7fbb17ac0700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fbb180d1c70 0x7fbb18108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56680/0 (socket says 172.21.2.101:56680) 2021-02-10T11:40:04.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.992 7fbb17ac0700 1 -- 172.21.2.101:0/1163383395 learned_addr learned my addr 172.21.2.101:0/1163383395 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:04.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.992 7fbb172bf700 1 --2- 172.21.2.101:0/1163383395 >> v2:172.21.2.101:3300/0 conn(0x7fbb180d1870 0x7fbb181b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:04.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.992 7fbb16abe700 1 --2- 172.21.2.101:0/1163383395 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb181b0ae0 0x7fbb18108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:04.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.992 7fbb172bf700 1 -- 172.21.2.101:0/1163383395 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb181b0ae0 msgr2=0x7fbb18108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:04.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.992 7fbb172bf700 1 --2- 172.21.2.101:0/1163383395 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb181b0ae0 0x7fbb18108940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:04.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.992 7fbb172bf700 1 -- 172.21.2.101:0/1163383395 >> v1:172.21.2.101:6789/0 conn(0x7fbb180d1c70 legacy=0x7fbb18108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:04.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.992 7fbb172bf700 1 -- 172.21.2.101:0/1163383395 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb18088070 con 0x7fbb180d1870 2021-02-10T11:40:04.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.992 7fbb162bd700 1 -- 172.21.2.101:0/1163383395 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2298643524 0 0) 0x7fbb10000f90 con 0x7fbb180d1c70 2021-02-10T11:40:04.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.992 7fbb172bf700 1 --2- 172.21.2.101:0/1163383395 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb180d1870 0x7fbb181b04c0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 rx=0x7fbb08006650 tx=0x7fbb08009a00).ready entity=mon.0 client_cookie=994e544f65cb7647 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:04.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.993 7fbb162bd700 1 -- 172.21.2.101:0/1163383395 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbb08009df0 con 0x7fbb180d1870 2021-02-10T11:40:04.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.993 7fbb162bd700 1 -- 172.21.2.101:0/1163383395 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb080009b0 con 0x7fbb180d1870 2021-02-10T11:40:04.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.993 7fbb162bd700 1 -- 172.21.2.101:0/1163383395 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbb0800c2b0 con 0x7fbb180d1870 2021-02-10T11:40:04.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.993 7fbb1d4e1700 1 -- 172.21.2.101:0/1163383395 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb180d1870 msgr2=0x7fbb181b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:04.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.993 7fbb1d4e1700 1 --2- 172.21.2.101:0/1163383395 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb180d1870 0x7fbb181b04c0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 rx=0x7fbb08006650 tx=0x7fbb08009a00).stop 2021-02-10T11:40:04.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.993 7fbb1d4e1700 1 -- 172.21.2.101:0/1163383395 shutdown_connections 2021-02-10T11:40:04.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.993 7fbb1d4e1700 1 --2- 172.21.2.101:0/1163383395 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb181b0ae0 0x7fbb18108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:04.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.993 7fbb1d4e1700 1 --2- 172.21.2.101:0/1163383395 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb180d1870 0x7fbb181b04c0 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:04.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.993 7fbb1d4e1700 1 -- 172.21.2.101:0/1163383395 >> 172.21.2.101:0/1163383395 conn(0x7fbb180ce420 msgr2=0x7fbb180d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:04.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.993 7fbb1d4e1700 1 -- 172.21.2.101:0/1163383395 shutdown_connections 2021-02-10T11:40:04.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:03.993 7fbb1d4e1700 1 -- 172.21.2.101:0/1163383395 wait complete. 2021-02-10T11:40:04.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.008 7fbb1d4e1700 1 Processor -- start 2021-02-10T11:40:04.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.008 7fbb1d4e1700 1 -- start start 2021-02-10T11:40:04.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.008 7fbb1d4e1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb1809efa0 0x7fbb181a79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:04.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.008 7fbb1d4e1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb181a7fc0 0x7fbb180c1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:04.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.008 7fbb1d4e1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb1809f3a0 0x7fbb180c1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:04.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.008 7fbb1d4e1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb18086890 con 0x7fbb1809f3a0 2021-02-10T11:40:04.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.008 7fbb1d4e1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180c1da0 con 0x7fbb181a7fc0 2021-02-10T11:40:04.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.008 7fbb1d4e1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180c1fc0 con 0x7fbb1809efa0 2021-02-10T11:40:04.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.009 7fbb17ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb1809f3a0 0x7fbb180c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:04.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.009 7fbb17ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb1809f3a0 0x7fbb180c1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56192/0 (socket says 172.21.2.101:56192) 2021-02-10T11:40:04.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.009 7fbb17ac0700 1 -- 172.21.2.101:0/1082580795 learned_addr learned my addr 172.21.2.101:0/1082580795 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:04.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.009 7fbb16abe700 1 --2- 172.21.2.101:0/1082580795 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb181a7fc0 0x7fbb180c1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:04.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.009 7fbb172bf700 1 --2- 172.21.2.101:0/1082580795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb1809efa0 0x7fbb181a79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:04.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.009 7fbb17ac0700 1 -- 172.21.2.101:0/1082580795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb1809efa0 msgr2=0x7fbb181a79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:04.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.009 7fbb17ac0700 1 --2- 172.21.2.101:0/1082580795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb1809efa0 0x7fbb181a79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:04.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.009 7fbb17ac0700 1 -- 172.21.2.101:0/1082580795 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb181a7fc0 msgr2=0x7fbb180c1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:04.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.009 7fbb17ac0700 1 --2- 172.21.2.101:0/1082580795 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb181a7fc0 0x7fbb180c1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:04.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.009 7fbb17ac0700 1 -- 172.21.2.101:0/1082580795 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb180c2890 con 0x7fbb1809f3a0 2021-02-10T11:40:04.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.009 7fbb16abe700 1 --2- 172.21.2.101:0/1082580795 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb181a7fc0 0x7fbb180c1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:40:04.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.009 7fbb17ac0700 1 --2- 172.21.2.101:0/1082580795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb1809f3a0 0x7fbb180c1780 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 rx=0x7fbb1000a880 tx=0x7fbb1000ac20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:04.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.010 7fbafffff700 1 -- 172.21.2.101:0/1082580795 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbb10000f90 con 0x7fbb1809f3a0 2021-02-10T11:40:04.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.010 7fbafffff700 1 -- 172.21.2.101:0/1082580795 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb100070b0 con 0x7fbb1809f3a0 2021-02-10T11:40:04.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.010 7fbb1d4e1700 1 -- 172.21.2.101:0/1082580795 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb180c2890 con 0x7fbb1809f3a0 2021-02-10T11:40:04.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.010 7fbb1d4e1700 1 -- 172.21.2.101:0/1082580795 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb180c2e20 con 0x7fbb1809f3a0 2021-02-10T11:40:04.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.010 7fbafffff700 1 -- 172.21.2.101:0/1082580795 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbb10007540 con 0x7fbb1809f3a0 2021-02-10T11:40:04.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.010 7fbafffff700 1 -- 172.21.2.101:0/1082580795 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbb10007290 con 0x7fbb1809f3a0 2021-02-10T11:40:04.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.010 7fbafffff700 1 --2- 172.21.2.101:0/1082580795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbaf80186c0 0x7fbaf801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:04.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.010 7fbafffff700 1 -- 172.21.2.101:0/1082580795 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fbb100328a0 con 0x7fbb1809f3a0 2021-02-10T11:40:04.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.010 7fbb172bf700 1 --2- 172.21.2.101:0/1082580795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbaf80186c0 0x7fbaf801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:04.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.011 7fbb172bf700 1 --2- 172.21.2.101:0/1082580795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbaf80186c0 0x7fbaf801abc0 crc :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:04.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.042 7fbb1d4e1700 1 -- 172.21.2.101:0/1082580795 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb180fe1c0 con 0x7fbb1809f3a0 2021-02-10T11:40:04.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.044 7fbafffff700 1 -- 172.21.2.101:0/1082580795 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbb10000f90 con 0x7fbb1809f3a0 2021-02-10T11:40:04.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.171 7fbb1d4e1700 1 -- 172.21.2.101:0/1082580795 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fbb18086840 con 0x7fbb1809f3a0 2021-02-10T11:40:04.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.172 7fbafffff700 1 -- 172.21.2.101:0/1082580795 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v103) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbb10007290 con 0x7fbb1809f3a0 2021-02-10T11:40:04.175 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:40:04.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.173 7fbb1d4e1700 1 -- 172.21.2.101:0/1082580795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbaf80186c0 msgr2=0x7fbaf801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:04.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.173 7fbb1d4e1700 1 --2- 172.21.2.101:0/1082580795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbaf80186c0 0x7fbaf801abc0 crc :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:04.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.173 7fbb1d4e1700 1 -- 172.21.2.101:0/1082580795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb1809f3a0 msgr2=0x7fbb180c1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:04.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.173 7fbb1d4e1700 1 --2- 172.21.2.101:0/1082580795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb1809f3a0 0x7fbb180c1780 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 rx=0x7fbb1000a880 tx=0x7fbb1000ac20).stop 2021-02-10T11:40:04.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.174 7fbb1d4e1700 1 -- 172.21.2.101:0/1082580795 shutdown_connections 2021-02-10T11:40:04.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.174 7fbb1d4e1700 1 --2- 172.21.2.101:0/1082580795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb1809f3a0 0x7fbb180c1780 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:04.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.174 7fbb1d4e1700 1 --2- 172.21.2.101:0/1082580795 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb181a7fc0 0x7fbb180c1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:04.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.174 7fbb1d4e1700 1 --2- 172.21.2.101:0/1082580795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbaf80186c0 0x7fbaf801abc0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:04.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.174 7fbb1d4e1700 1 --2- 172.21.2.101:0/1082580795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb1809efa0 0x7fbb181a79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:04.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.174 7fbb1d4e1700 1 -- 172.21.2.101:0/1082580795 >> 172.21.2.101:0/1082580795 conn(0x7fbb180a0770 msgr2=0x7fbb18086fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:04.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.174 7fbb1d4e1700 1 -- 172.21.2.101:0/1082580795 shutdown_connections 2021-02-10T11:40:04.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.174 7fbb1d4e1700 1 -- 172.21.2.101:0/1082580795 wait complete. 2021-02-10T11:40:04.185 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:40:04.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.596 7f4bdfa7d700 1 Processor -- start 2021-02-10T11:40:04.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.596 7f4bdfa7d700 1 -- start start 2021-02-10T11:40:04.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.596 7f4bdfa7d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f4bd80d1870 0x7f4bd81b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:04.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.597 7f4bdfa7d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4bd81b0ae0 0x7f4bd8108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:04.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.597 7f4bdfa7d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd80fe1c0 con 0x7f4bd80d1c70 2021-02-10T11:40:04.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.597 7f4bdfa7d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd8109a60 con 0x7f4bd80d1870 2021-02-10T11:40:04.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.597 7f4bdfa7d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd8109750 con 0x7f4bd81b0ae0 2021-02-10T11:40:04.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.597 7f4bde09a700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f4bd80d1c70 0x7f4bd8108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56690/0 (socket says 172.21.2.101:56690) 2021-02-10T11:40:04.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.597 7f4bde09a700 1 -- 172.21.2.101:0/62038031 learned_addr learned my addr 172.21.2.101:0/62038031 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:04.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.597 7f4bdd899700 1 --2- 172.21.2.101:0/62038031 >> v2:172.21.2.101:3300/0 conn(0x7f4bd80d1870 0x7f4bd81b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:04.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.597 7f4bdd899700 1 -- 172.21.2.101:0/62038031 >> v2:172.21.2.108:3300/0 conn(0x7f4bd81b0ae0 msgr2=0x7f4bd8108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:04.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.597 7f4bdd899700 1 --2- 172.21.2.101:0/62038031 >> v2:172.21.2.108:3300/0 conn(0x7f4bd81b0ae0 0x7f4bd8108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:04.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.597 7f4bdd899700 1 -- 172.21.2.101:0/62038031 >> v1:172.21.2.101:6789/0 conn(0x7f4bd80d1c70 legacy=0x7f4bd8108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:04.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.597 7f4bdc897700 1 -- 172.21.2.101:0/62038031 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1340548366 0 0) 0x7f4bd0001500 con 0x7f4bd80d1c70 2021-02-10T11:40:04.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.597 7f4bdd899700 1 -- 172.21.2.101:0/62038031 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd8088070 con 0x7f4bd80d1870 2021-02-10T11:40:04.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.598 7f4bdd899700 1 --2- 172.21.2.101:0/62038031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bd80d1870 0x7f4bd81b04c0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 rx=0x7f4bd4006650 tx=0x7f4bd4009a00).ready entity=mon.0 client_cookie=f28483a7e9b0d176 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:04.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.598 7f4bdc897700 1 -- 172.21.2.101:0/62038031 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4bd4009df0 con 0x7f4bd80d1870 2021-02-10T11:40:04.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.598 7f4bdc897700 1 -- 172.21.2.101:0/62038031 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd40009b0 con 0x7f4bd80d1870 2021-02-10T11:40:04.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.598 7f4bdc897700 1 -- 172.21.2.101:0/62038031 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4bd400c2b0 con 0x7f4bd80d1870 2021-02-10T11:40:04.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.598 7f4bdfa7d700 1 -- 172.21.2.101:0/62038031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bd80d1870 msgr2=0x7f4bd81b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:04.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.598 7f4bdfa7d700 1 --2- 172.21.2.101:0/62038031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bd80d1870 0x7f4bd81b04c0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 rx=0x7f4bd4006650 tx=0x7f4bd4009a00).stop 2021-02-10T11:40:04.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.599 7f4bdfa7d700 1 -- 172.21.2.101:0/62038031 shutdown_connections 2021-02-10T11:40:04.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.599 7f4bdfa7d700 1 --2- 172.21.2.101:0/62038031 >> v2:172.21.2.108:3300/0 conn(0x7f4bd81b0ae0 0x7f4bd8108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:04.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.599 7f4bdfa7d700 1 --2- 172.21.2.101:0/62038031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bd80d1870 0x7f4bd81b04c0 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:04.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.599 7f4bdfa7d700 1 -- 172.21.2.101:0/62038031 >> 172.21.2.101:0/62038031 conn(0x7f4bd80ce420 msgr2=0x7f4bd80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:04.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.599 7f4bdfa7d700 1 -- 172.21.2.101:0/62038031 shutdown_connections 2021-02-10T11:40:04.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.599 7f4bdfa7d700 1 -- 172.21.2.101:0/62038031 wait complete. 2021-02-10T11:40:04.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.614 7f4bdfa7d700 1 Processor -- start 2021-02-10T11:40:04.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.614 7f4bdfa7d700 1 -- start start 2021-02-10T11:40:04.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.614 7f4bdfa7d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bd80c2370 0x7f4bd818f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:04.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.614 7f4bdfa7d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bd8084cb0 0x7f4bd81a7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:04.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.614 7f4bdfa7d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4bd81a7a90 0x7f4bd8085700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:04.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.614 7f4bdfa7d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd80be840 con 0x7f4bd8084cb0 2021-02-10T11:40:04.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.614 7f4bdfa7d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd80c29a0 con 0x7f4bd81a7a90 2021-02-10T11:40:04.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.614 7f4bdfa7d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd81a7e90 con 0x7f4bd80c2370 2021-02-10T11:40:04.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.614 7f4bdd098700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bd8084cb0 0x7f4bd81a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:04.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.614 7f4bdd098700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bd8084cb0 0x7f4bd81a7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56204/0 (socket says 172.21.2.101:56204) 2021-02-10T11:40:04.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.614 7f4bdd098700 1 -- 172.21.2.101:0/1340188385 learned_addr learned my addr 172.21.2.101:0/1340188385 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:04.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.615 7f4bdd098700 1 -- 172.21.2.101:0/1340188385 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bd80c2370 msgr2=0x7f4bd818f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:04.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.615 7f4bdd098700 1 --2- 172.21.2.101:0/1340188385 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bd80c2370 0x7f4bd818f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:04.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.615 7f4bdd098700 1 -- 172.21.2.101:0/1340188385 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4bd81a7a90 msgr2=0x7f4bd8085700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:04.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.615 7f4bdd098700 1 --2- 172.21.2.101:0/1340188385 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4bd81a7a90 0x7f4bd8085700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:04.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.615 7f4bdd098700 1 -- 172.21.2.101:0/1340188385 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd81a80b0 con 0x7f4bd8084cb0 2021-02-10T11:40:04.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.615 7f4bdd098700 1 --2- 172.21.2.101:0/1340188385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bd8084cb0 0x7f4bd81a7470 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 rx=0x7f4bc8006f00 tx=0x7f4bc800b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:04.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.615 7f4bce7fc700 1 -- 172.21.2.101:0/1340188385 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4bc800bd30 con 0x7f4bd8084cb0 2021-02-10T11:40:04.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.615 7f4bce7fc700 1 -- 172.21.2.101:0/1340188385 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc8000c90 con 0x7f4bd8084cb0 2021-02-10T11:40:04.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.615 7f4bce7fc700 1 -- 172.21.2.101:0/1340188385 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4bc80022a0 con 0x7f4bd8084cb0 2021-02-10T11:40:04.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.615 7f4bdfa7d700 1 -- 172.21.2.101:0/1340188385 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd81a80b0 con 0x7f4bd8084cb0 2021-02-10T11:40:04.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.615 7f4bdfa7d700 1 -- 172.21.2.101:0/1340188385 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd81a80b0 con 0x7f4bd8084cb0 2021-02-10T11:40:04.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.616 7f4bce7fc700 1 -- 172.21.2.101:0/1340188385 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4bc8005310 con 0x7f4bd8084cb0 2021-02-10T11:40:04.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.616 7f4bce7fc700 1 --2- 172.21.2.101:0/1340188385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bc00186c0 0x7f4bc001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:04.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.616 7f4bce7fc700 1 -- 172.21.2.101:0/1340188385 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4bc8032780 con 0x7f4bd8084cb0 2021-02-10T11:40:04.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.616 7f4bdd899700 1 --2- 172.21.2.101:0/1340188385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bc00186c0 0x7f4bc001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:04.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.616 7f4bdd899700 1 --2- 172.21.2.101:0/1340188385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bc00186c0 0x7f4bc001abc0 crc :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:04.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.646 7f4bdfa7d700 1 -- 172.21.2.101:0/1340188385 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd8082a70 con 0x7f4bd8084cb0 2021-02-10T11:40:04.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.649 7f4bce7fc700 1 -- 172.21.2.101:0/1340188385 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4bc8006b90 con 0x7f4bd8084cb0 2021-02-10T11:40:04.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.776 7f4bdfa7d700 1 -- 172.21.2.101:0/1340188385 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f4bd8082a70 con 0x7f4bd8084cb0 2021-02-10T11:40:04.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.777 7f4bce7fc700 1 -- 172.21.2.101:0/1340188385 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v103) v1 ==== 104+0+10 (secure 0 0 0) 0x7f4bc800bd30 con 0x7f4bd8084cb0 2021-02-10T11:40:04.780 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:40:04.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.779 7f4bdfa7d700 1 -- 172.21.2.101:0/1340188385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bc00186c0 msgr2=0x7f4bc001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:04.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.779 7f4bdfa7d700 1 --2- 172.21.2.101:0/1340188385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bc00186c0 0x7f4bc001abc0 crc :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:04.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.779 7f4bdfa7d700 1 -- 172.21.2.101:0/1340188385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bd8084cb0 msgr2=0x7f4bd81a7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:04.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.779 7f4bdfa7d700 1 --2- 172.21.2.101:0/1340188385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bd8084cb0 0x7f4bd81a7470 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 rx=0x7f4bc8006f00 tx=0x7f4bc800b940).stop 2021-02-10T11:40:04.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.779 7f4bdfa7d700 1 -- 172.21.2.101:0/1340188385 shutdown_connections 2021-02-10T11:40:04.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.779 7f4bdfa7d700 1 --2- 172.21.2.101:0/1340188385 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4bd81a7a90 0x7f4bd8085700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:04.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.779 7f4bdfa7d700 1 --2- 172.21.2.101:0/1340188385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4bd8084cb0 0x7f4bd81a7470 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:04.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.779 7f4bdfa7d700 1 --2- 172.21.2.101:0/1340188385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bc00186c0 0x7f4bc001abc0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:04.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.779 7f4bdfa7d700 1 --2- 172.21.2.101:0/1340188385 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4bd80c2370 0x7f4bd818f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:04.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.779 7f4bdfa7d700 1 -- 172.21.2.101:0/1340188385 >> 172.21.2.101:0/1340188385 conn(0x7f4bd80b5dc0 msgr2=0x7f4bd80c2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:04.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.779 7f4bdfa7d700 1 -- 172.21.2.101:0/1340188385 shutdown_connections 2021-02-10T11:40:04.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:04.779 7f4bdfa7d700 1 -- 172.21.2.101:0/1340188385 wait complete. 2021-02-10T11:40:04.790 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:40:05.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.201 7f6db8738700 1 Processor -- start 2021-02-10T11:40:05.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.201 7f6db8738700 1 -- start start 2021-02-10T11:40:05.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.201 7f6db8738700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6db00d1870 0x7f6db01b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:05.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.201 7f6db8738700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6db01b0ae0 0x7f6db0108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:05.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.201 7f6db8738700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db00fe1c0 con 0x7f6db00d1c70 2021-02-10T11:40:05.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.201 7f6db8738700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db0109a60 con 0x7f6db00d1870 2021-02-10T11:40:05.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.201 7f6db8738700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db0109750 con 0x7f6db01b0ae0 2021-02-10T11:40:05.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.201 7f6db6554700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6db00d1870 0x7f6db01b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:05.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.201 7f6db6d55700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f6db00d1c70 0x7f6db0108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56706/0 (socket says 172.21.2.101:56706) 2021-02-10T11:40:05.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.201 7f6db6d55700 1 -- 172.21.2.101:0/2920002077 learned_addr learned my addr 172.21.2.101:0/2920002077 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:05.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.201 7f6db5d53700 1 --2- 172.21.2.101:0/2920002077 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6db01b0ae0 0x7f6db0108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:05.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.202 7f6db5552700 1 -- 172.21.2.101:0/2920002077 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833117185 0 0) 0x7f6dac001180 con 0x7f6db00d1c70 2021-02-10T11:40:05.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.202 7f6db5d53700 1 -- 172.21.2.101:0/2920002077 >> v2:172.21.2.101:3300/0 conn(0x7f6db00d1870 msgr2=0x7f6db01b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:05.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.202 7f6db5d53700 1 --2- 172.21.2.101:0/2920002077 >> v2:172.21.2.101:3300/0 conn(0x7f6db00d1870 0x7f6db01b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:05.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.202 7f6db5d53700 1 -- 172.21.2.101:0/2920002077 >> v1:172.21.2.101:6789/0 conn(0x7f6db00d1c70 legacy=0x7f6db0108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:05.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.202 7f6db5d53700 1 -- 172.21.2.101:0/2920002077 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db0088070 con 0x7f6db01b0ae0 2021-02-10T11:40:05.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.202 7f6db6554700 1 --2- 172.21.2.101:0/2920002077 >> v2:172.21.2.101:3300/0 conn(0x7f6db00d1870 0x7f6db01b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:40:05.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.202 7f6db5d53700 1 --2- 172.21.2.101:0/2920002077 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6db01b0ae0 0x7f6db0108940 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f6da80069d0 tx=0x7f6da8009e40).ready entity=mon.2 client_cookie=1025e14d8d3e1b1e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:05.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.202 7f6db5552700 1 -- 172.21.2.101:0/2920002077 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6da80066c0 con 0x7f6db01b0ae0 2021-02-10T11:40:05.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.202 7f6db5552700 1 -- 172.21.2.101:0/2920002077 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da800b510 con 0x7f6db01b0ae0 2021-02-10T11:40:05.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.202 7f6db8738700 1 -- 172.21.2.101:0/2920002077 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6db01b0ae0 msgr2=0x7f6db0108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:05.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.202 7f6db5552700 1 -- 172.21.2.101:0/2920002077 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6da8000d30 con 0x7f6db01b0ae0 2021-02-10T11:40:05.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.202 7f6db8738700 1 --2- 172.21.2.101:0/2920002077 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6db01b0ae0 0x7f6db0108940 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f6da80069d0 tx=0x7f6da8009e40).stop 2021-02-10T11:40:05.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.203 7f6db8738700 1 -- 172.21.2.101:0/2920002077 shutdown_connections 2021-02-10T11:40:05.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.203 7f6db8738700 1 --2- 172.21.2.101:0/2920002077 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6db01b0ae0 0x7f6db0108940 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:05.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.203 7f6db8738700 1 --2- 172.21.2.101:0/2920002077 >> v2:172.21.2.101:3300/0 conn(0x7f6db00d1870 0x7f6db01b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:05.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.203 7f6db8738700 1 -- 172.21.2.101:0/2920002077 >> 172.21.2.101:0/2920002077 conn(0x7f6db00ce420 msgr2=0x7f6db00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:05.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.203 7f6db8738700 1 -- 172.21.2.101:0/2920002077 shutdown_connections 2021-02-10T11:40:05.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.203 7f6db8738700 1 -- 172.21.2.101:0/2920002077 wait complete. 2021-02-10T11:40:05.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.218 7f6db8738700 1 Processor -- start 2021-02-10T11:40:05.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.218 7f6db8738700 1 -- start start 2021-02-10T11:40:05.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.218 7f6db8738700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6db0191b10 0x7f6db00ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:05.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.218 7f6db8738700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6db0191f10 0x7f6db019ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:05.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.218 7f6db8738700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6db01a05f0 0x7f6db0082ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:05.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.218 7f6db8738700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db0192440 con 0x7f6db0191b10 2021-02-10T11:40:05.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.218 7f6db8738700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db01916e0 con 0x7f6db01a05f0 2021-02-10T11:40:05.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.218 7f6db8738700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db0191900 con 0x7f6db0191f10 2021-02-10T11:40:05.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.218 7f6db6554700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6db0191b10 0x7f6db00ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:05.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.218 7f6db5d53700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6db0191f10 0x7f6db019ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:05.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.218 7f6db5d53700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6db0191f10 0x7f6db019ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:48946/0 (socket says 172.21.2.101:48946) 2021-02-10T11:40:05.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.218 7f6db6554700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6db0191b10 0x7f6db00ae640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56216/0 (socket says 172.21.2.101:56216) 2021-02-10T11:40:05.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.218 7f6db5d53700 1 -- 172.21.2.101:0/1016648526 learned_addr learned my addr 172.21.2.101:0/1016648526 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:05.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.218 7f6db6d55700 1 --2- 172.21.2.101:0/1016648526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6db01a05f0 0x7f6db0082ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:05.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.219 7f6db6554700 1 -- 172.21.2.101:0/1016648526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6db0191f10 msgr2=0x7f6db019ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:05.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.219 7f6db6554700 1 --2- 172.21.2.101:0/1016648526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6db0191f10 0x7f6db019ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:05.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.219 7f6db6554700 1 -- 172.21.2.101:0/1016648526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6db01a05f0 msgr2=0x7f6db0082ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:05.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.219 7f6db6554700 1 --2- 172.21.2.101:0/1016648526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6db01a05f0 0x7f6db0082ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:05.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.219 7f6db6554700 1 -- 172.21.2.101:0/1016648526 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db00834f0 con 0x7f6db0191b10 2021-02-10T11:40:05.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.219 7f6db6d55700 1 --2- 172.21.2.101:0/1016648526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6db01a05f0 0x7f6db0082ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:40:05.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.219 7f6db6554700 1 --2- 172.21.2.101:0/1016648526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6db0191b10 0x7f6db00ae640 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 rx=0x7f6da0009540 tx=0x7f6da000b8e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:05.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.219 7f6da6ffd700 1 -- 172.21.2.101:0/1016648526 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6da000bcd0 con 0x7f6db0191b10 2021-02-10T11:40:05.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.219 7f6da6ffd700 1 -- 172.21.2.101:0/1016648526 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da0006740 con 0x7f6db0191b10 2021-02-10T11:40:05.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.219 7f6db8738700 1 -- 172.21.2.101:0/1016648526 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db00834f0 con 0x7f6db0191b10 2021-02-10T11:40:05.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.219 7f6da6ffd700 1 -- 172.21.2.101:0/1016648526 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6da00009b0 con 0x7f6db0191b10 2021-02-10T11:40:05.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.219 7f6db8738700 1 -- 172.21.2.101:0/1016648526 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db00834f0 con 0x7f6db0191b10 2021-02-10T11:40:05.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.220 7f6da6ffd700 1 -- 172.21.2.101:0/1016648526 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6da0000c90 con 0x7f6db0191b10 2021-02-10T11:40:05.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.220 7f6da6ffd700 1 --2- 172.21.2.101:0/1016648526 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6d980186c0 0x7f6d9801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:05.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.220 7f6da6ffd700 1 -- 172.21.2.101:0/1016648526 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6da00324d0 con 0x7f6db0191b10 2021-02-10T11:40:05.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.220 7f6db5d53700 1 --2- 172.21.2.101:0/1016648526 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6d980186c0 0x7f6d9801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:05.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.220 7f6db5d53700 1 --2- 172.21.2.101:0/1016648526 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6d980186c0 0x7f6d9801abc0 crc :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:05.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.251 7f6db8738700 1 -- 172.21.2.101:0/1016648526 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db0192440 con 0x7f6db0191b10 2021-02-10T11:40:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.254 7f6da6ffd700 1 -- 172.21.2.101:0/1016648526 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6da00009b0 con 0x7f6db0191b10 2021-02-10T11:40:05.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.381 7f6db8738700 1 -- 172.21.2.101:0/1016648526 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f6db0192440 con 0x7f6db0191b10 2021-02-10T11:40:05.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.381 7f6da6ffd700 1 -- 172.21.2.101:0/1016648526 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v103) v1 ==== 100+0+10 (secure 0 0 0) 0x7f6da0000c90 con 0x7f6db0191b10 2021-02-10T11:40:05.385 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:40:05.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.383 7f6db8738700 1 -- 172.21.2.101:0/1016648526 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6d980186c0 msgr2=0x7f6d9801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:05.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.383 7f6db8738700 1 --2- 172.21.2.101:0/1016648526 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6d980186c0 0x7f6d9801abc0 crc :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:05.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.383 7f6db8738700 1 -- 172.21.2.101:0/1016648526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6db0191b10 msgr2=0x7f6db00ae640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:05.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.383 7f6db8738700 1 --2- 172.21.2.101:0/1016648526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6db0191b10 0x7f6db00ae640 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 rx=0x7f6da0009540 tx=0x7f6da000b8e0).stop 2021-02-10T11:40:05.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.384 7f6db8738700 1 -- 172.21.2.101:0/1016648526 shutdown_connections 2021-02-10T11:40:05.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.384 7f6db8738700 1 --2- 172.21.2.101:0/1016648526 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6d980186c0 0x7f6d9801abc0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:05.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.384 7f6db8738700 1 --2- 172.21.2.101:0/1016648526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6db0191f10 0x7f6db019ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:05.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.384 7f6db8738700 1 --2- 172.21.2.101:0/1016648526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6db01a05f0 0x7f6db0082ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:05.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.384 7f6db8738700 1 --2- 172.21.2.101:0/1016648526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6db0191b10 0x7f6db00ae640 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:05.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.384 7f6db8738700 1 -- 172.21.2.101:0/1016648526 >> 172.21.2.101:0/1016648526 conn(0x7f6db00a02d0 msgr2=0x7f6db009f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:05.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.384 7f6db8738700 1 -- 172.21.2.101:0/1016648526 shutdown_connections 2021-02-10T11:40:05.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.384 7f6db8738700 1 -- 172.21.2.101:0/1016648526 wait complete. 2021-02-10T11:40:05.395 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:40:05.396 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:40:05.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.804 7f5bada98700 1 Processor -- start 2021-02-10T11:40:05.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.804 7f5bada98700 1 -- start start 2021-02-10T11:40:05.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.805 7f5bada98700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ba80d1870 0x7f5ba81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:05.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.805 7f5bada98700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f5ba81b0a10 0x7f5ba80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:05.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.805 7f5bada98700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba80f0b10 con 0x7f5ba80d1c70 2021-02-10T11:40:05.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.805 7f5bada98700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba80fe230 con 0x7f5ba81b0a10 2021-02-10T11:40:05.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.805 7f5bada98700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba80e5ed0 con 0x7f5ba80d1870 2021-02-10T11:40:05.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.805 7f5ba7fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f5ba80d1c70 0x7f5ba80cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56716/0 (socket says 172.21.2.101:56716) 2021-02-10T11:40:05.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.805 7f5ba7fff700 1 -- 172.21.2.101:0/3938336236 learned_addr learned my addr 172.21.2.101:0/3938336236 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:05.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.805 7f5ba6ffd700 1 --2- 172.21.2.101:0/3938336236 >> v2:172.21.2.108:3300/0 conn(0x7f5ba81b0a10 0x7f5ba80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:05.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.805 7f5ba77fe700 1 --2- 172.21.2.101:0/3938336236 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ba80d1870 0x7f5ba81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:05.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.805 7f5ba67fc700 1 -- 172.21.2.101:0/3938336236 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3333970735 0 0) 0x7f5ba0000f90 con 0x7f5ba80d1c70 2021-02-10T11:40:05.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.805 7f5ba67fc700 1 -- 172.21.2.101:0/3938336236 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b94002fe0 con 0x7f5ba80d1c70 2021-02-10T11:40:05.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.806 7f5ba6ffd700 1 -- 172.21.2.101:0/3938336236 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ba80d1870 msgr2=0x7f5ba81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:05.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.806 7f5ba6ffd700 1 --2- 172.21.2.101:0/3938336236 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ba80d1870 0x7f5ba81b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:05.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.806 7f5ba6ffd700 1 -- 172.21.2.101:0/3938336236 >> v1:172.21.2.101:6789/0 conn(0x7f5ba80d1c70 legacy=0x7f5ba80cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:05.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.806 7f5ba6ffd700 1 -- 172.21.2.101:0/3938336236 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba80f0650 con 0x7f5ba81b0a10 2021-02-10T11:40:05.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.806 7f5ba6ffd700 1 --2- 172.21.2.101:0/3938336236 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ba81b0a10 0x7f5ba80cf330 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f5b98006650 tx=0x7f5b98009a80).ready entity=mon.1 client_cookie=b6d62e8c73be22d9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:05.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.806 7f5ba67fc700 1 -- 172.21.2.101:0/3938336236 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5b9800c0c0 con 0x7f5ba81b0a10 2021-02-10T11:40:05.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.806 7f5ba67fc700 1 -- 172.21.2.101:0/3938336236 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9800c340 con 0x7f5ba81b0a10 2021-02-10T11:40:05.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.806 7f5ba67fc700 1 -- 172.21.2.101:0/3938336236 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5b9800c860 con 0x7f5ba81b0a10 2021-02-10T11:40:05.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.806 7f5bada98700 1 -- 172.21.2.101:0/3938336236 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ba81b0a10 msgr2=0x7f5ba80cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:05.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.806 7f5bada98700 1 --2- 172.21.2.101:0/3938336236 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ba81b0a10 0x7f5ba80cf330 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f5b98006650 tx=0x7f5b98009a80).stop 2021-02-10T11:40:05.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.807 7f5bada98700 1 -- 172.21.2.101:0/3938336236 shutdown_connections 2021-02-10T11:40:05.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.807 7f5bada98700 1 --2- 172.21.2.101:0/3938336236 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ba81b0a10 0x7f5ba80cf330 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:05.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.807 7f5bada98700 1 --2- 172.21.2.101:0/3938336236 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ba80d1870 0x7f5ba81b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:05.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.807 7f5bada98700 1 -- 172.21.2.101:0/3938336236 >> 172.21.2.101:0/3938336236 conn(0x7f5ba80ce450 msgr2=0x7f5ba80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:05.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.807 7f5bada98700 1 -- 172.21.2.101:0/3938336236 shutdown_connections 2021-02-10T11:40:05.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.807 7f5bada98700 1 -- 172.21.2.101:0/3938336236 wait complete. 2021-02-10T11:40:05.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.821 7f5bada98700 1 Processor -- start 2021-02-10T11:40:05.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.821 7f5bada98700 1 -- start start 2021-02-10T11:40:05.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.822 7f5bada98700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ba80aa700 0x7f5ba8090ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:05.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.822 7f5bada98700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ba80aab00 0x7f5ba80910e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:05.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.822 7f5bada98700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ba808ce20 0x7f5ba808d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:05.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.822 7f5bada98700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba8090540 con 0x7f5ba80aab00 2021-02-10T11:40:05.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.822 7f5bada98700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba80c0170 con 0x7f5ba80aa700 2021-02-10T11:40:05.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.822 7f5bada98700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba808d840 con 0x7f5ba808ce20 2021-02-10T11:40:05.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.822 7f5ba6ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ba80aab00 0x7f5ba80910e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:05.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.822 7f5ba6ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ba80aab00 0x7f5ba80910e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56230/0 (socket says 172.21.2.101:56230) 2021-02-10T11:40:05.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.822 7f5ba6ffd700 1 -- 172.21.2.101:0/388854715 learned_addr learned my addr 172.21.2.101:0/388854715 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:05.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.822 7f5ba7fff700 1 --2- 172.21.2.101:0/388854715 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ba808ce20 0x7f5ba808d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:05.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.822 7f5ba77fe700 1 --2- 172.21.2.101:0/388854715 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ba80aa700 0x7f5ba8090ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:05.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.822 7f5ba6ffd700 1 -- 172.21.2.101:0/388854715 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ba808ce20 msgr2=0x7f5ba808d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:05.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.822 7f5ba6ffd700 1 --2- 172.21.2.101:0/388854715 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ba808ce20 0x7f5ba808d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:05.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.822 7f5ba6ffd700 1 -- 172.21.2.101:0/388854715 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ba80aa700 msgr2=0x7f5ba8090ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:05.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.822 7f5ba6ffd700 1 --2- 172.21.2.101:0/388854715 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ba80aa700 0x7f5ba8090ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:05.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.822 7f5ba6ffd700 1 -- 172.21.2.101:0/388854715 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba808da60 con 0x7f5ba80aab00 2021-02-10T11:40:05.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.822 7f5ba7fff700 1 --2- 172.21.2.101:0/388854715 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ba808ce20 0x7f5ba808d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:40:05.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.822 7f5ba6ffd700 1 --2- 172.21.2.101:0/388854715 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ba80aab00 0x7f5ba80910e0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 rx=0x7f5b9800d7e0 tx=0x7f5b9800db80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:05.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.823 7f5b9ffff700 1 -- 172.21.2.101:0/388854715 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5b98000c90 con 0x7f5ba80aab00 2021-02-10T11:40:05.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.823 7f5b9ffff700 1 -- 172.21.2.101:0/388854715 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b98006740 con 0x7f5ba80aab00 2021-02-10T11:40:05.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.823 7f5b9ffff700 1 -- 172.21.2.101:0/388854715 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5b98002220 con 0x7f5ba80aab00 2021-02-10T11:40:05.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.823 7f5bada98700 1 -- 172.21.2.101:0/388854715 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba808da60 con 0x7f5ba80aab00 2021-02-10T11:40:05.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.823 7f5bada98700 1 -- 172.21.2.101:0/388854715 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba808da60 con 0x7f5ba80aab00 2021-02-10T11:40:05.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.824 7f5b9ffff700 1 -- 172.21.2.101:0/388854715 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5b98006030 con 0x7f5ba80aab00 2021-02-10T11:40:05.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.824 7f5b9ffff700 1 --2- 172.21.2.101:0/388854715 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b8c0186c0 0x7f5b8c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:05.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.824 7f5b9ffff700 1 -- 172.21.2.101:0/388854715 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5b98034790 con 0x7f5ba80aab00 2021-02-10T11:40:05.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.824 7f5ba77fe700 1 --2- 172.21.2.101:0/388854715 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b8c0186c0 0x7f5b8c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:05.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.824 7f5ba77fe700 1 --2- 172.21.2.101:0/388854715 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b8c0186c0 0x7f5b8c01abc0 crc :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:05.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.855 7f5bada98700 1 -- 172.21.2.101:0/388854715 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba80ebc00 con 0x7f5ba80aab00 2021-02-10T11:40:05.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.857 7f5b9ffff700 1 -- 172.21.2.101:0/388854715 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5b98006490 con 0x7f5ba80aab00 2021-02-10T11:40:05.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.985 7f5bada98700 1 -- 172.21.2.101:0/388854715 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ba80ebc00 con 0x7f5ba80aab00 2021-02-10T11:40:05.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.985 7f5b9ffff700 1 -- 172.21.2.101:0/388854715 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+2107 (secure 0 0 0) 0x7f5b98006030 con 0x7f5ba80aab00 2021-02-10T11:40:05.989 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:40:05.989 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":79,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10125,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/3121560299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3121560299},{"type":"v1","addr":"172.21.2.101:6835","nonce":3121560299}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":76}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:39:56.054150","modified":"2021-02-10 11:39:58.060269","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10122},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10122":{"gid":10122,"name":"b","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/1690286235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":1690286235},{"type":"v1","addr":"172.21.2.108:6833","nonce":1690286235}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2021-02-10T11:40:05.989 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 79 2021-02-10T11:40:05.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.987 7f5bada98700 1 -- 172.21.2.101:0/388854715 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b8c0186c0 msgr2=0x7f5b8c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:05.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.987 7f5bada98700 1 --2- 172.21.2.101:0/388854715 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b8c0186c0 0x7f5b8c01abc0 crc :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:05.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.987 7f5bada98700 1 -- 172.21.2.101:0/388854715 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ba80aab00 msgr2=0x7f5ba80910e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:05.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.987 7f5bada98700 1 --2- 172.21.2.101:0/388854715 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ba80aab00 0x7f5ba80910e0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 rx=0x7f5b9800d7e0 tx=0x7f5b9800db80).stop 2021-02-10T11:40:05.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.987 7f5bada98700 1 -- 172.21.2.101:0/388854715 shutdown_connections 2021-02-10T11:40:05.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.987 7f5bada98700 1 --2- 172.21.2.101:0/388854715 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b8c0186c0 0x7f5b8c01abc0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:05.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.987 7f5bada98700 1 --2- 172.21.2.101:0/388854715 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ba808ce20 0x7f5ba808d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:05.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.987 7f5bada98700 1 --2- 172.21.2.101:0/388854715 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ba80aab00 0x7f5ba80910e0 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:05.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.987 7f5bada98700 1 --2- 172.21.2.101:0/388854715 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ba80aa700 0x7f5ba8090ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:05.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.987 7f5bada98700 1 -- 172.21.2.101:0/388854715 >> 172.21.2.101:0/388854715 conn(0x7f5ba80b7e70 msgr2=0x7f5ba80c3850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:05.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.988 7f5bada98700 1 -- 172.21.2.101:0/388854715 shutdown_connections 2021-02-10T11:40:05.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:05.988 7f5bada98700 1 -- 172.21.2.101:0/388854715 wait complete. 2021-02-10T11:40:06.200 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 79, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:39:56.054150', 'modified': '2021-02-10 11:39:58.060269', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10122}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10122': {'gid': 10122, 'name': 'b', 'rank': 0, 'incarnation': 77, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/1690286235', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 1690286235}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 1690286235}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:40:06.201 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:40:06.201 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:40:06.598 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:40:06.598 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:40:06.598 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:40:06.599 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 10122, 2021-02-10T11:40:06.599 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:40:06.599 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:40:06.599 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 10.539751716, 2021-02-10T11:40:06.599 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 79, 2021-02-10T11:40:06.599 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 103, 2021-02-10T11:40:06.599 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 103, 2021-02-10T11:40:06.599 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 16.175551987999999 2021-02-10T11:40:06.599 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:40:06.602 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 10122, "want_state": "up:active", "state": "up:active", "rank_uptime": 10.539751716, "mdsmap_epoch": 79, "osdmap_epoch": 103, "osdmap_epoch_barrier": 103, "uptime": 16.175551987999999 } 2021-02-10T11:40:06.602 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:40:06.602 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-02-10T11:40:06.602 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-02-10T11:40:06.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:06.603 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:40:06.937 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:40:07.282 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:40:07.283 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:40:07.626 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:40:07.626 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:40:07.627 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:40:07.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:07.627 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:40:07.964 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-02-10T11:40:07.964 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:40:07.967 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:40:08.310 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:40:08.311 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:40:08.377 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[143449]: starting ceph client 2021-02-10T11:40:08.377 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:2021-02-10 11:40:08.374 7fd7271c5f80 -1 init, newargv = 0x55d7d16fa1a0 newargc=9 2021-02-10T11:40:08.384 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[143449]: starting fuse 2021-02-10T11:40:08.654 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:40:08.655 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:40:08.655 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:40:08.656 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:08.656 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:40:08.991 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:40:08.992 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2021-02-10T11:40:08.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:08.992 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:40:08.992 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:40:08.992 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:40:08.992 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:40:08.992 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:40:08.992 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:40:08.992 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:40:08.992 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-02-10T11:40:08.992 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:40:08.992 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:40:08.993 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:40:08.993 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:40:08.993 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:40:08.993 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:40:08.993 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:40:08.993 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:40:08.993 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:40:08.993 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:40:08.993 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:40:08.993 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:40:08.993 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:40:08.993 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.0")) 2021-02-10T11:40:08.993 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:40:09.354 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.0.143449.asok 2021-02-10T11:40:09.357 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.143449.asok 2021-02-10T11:40:09.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:09.358 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.143449.asok status 2021-02-10T11:40:09.749 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:40:09.749 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:40:09.749 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:40:09.749 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:40:09.750 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "0", 2021-02-10T11:40:09.750 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:40:09.750 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-02-10T11:40:09.750 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "143449", 2021-02-10T11:40:09.750 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:40:09.750 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:09.750 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:40:09.750 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:40:09.750 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 10284, 2021-02-10T11:40:09.750 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:40:09.750 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:40:09.750 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:40:09.750 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 10284 2021-02-10T11:40:09.751 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:09.751 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:40:09.751 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:09.751 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:40:09.751 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1265047927 2021-02-10T11:40:09.751 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:09.751 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:09.751 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:40:09.751 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:09.751 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:40:09.751 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1265047927 2021-02-10T11:40:09.751 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:09.751 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.10284 172.21.2.101:0/1265047927", 2021-02-10T11:40:09.752 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/1265047927", 2021-02-10T11:40:09.752 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:40:09.752 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 79, 2021-02-10T11:40:09.752 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 103, 2021-02-10T11:40:09.752 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:40:09.752 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:40:09.752 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:40:09.753 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:09.753 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:40:09.754 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:40:10.089 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:40:10.089 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:40:10.089 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:10.089 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:40:10.435 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:40:10.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.843 7ff0e66d1700 1 Processor -- start 2021-02-10T11:40:10.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.843 7ff0e66d1700 1 -- start start 2021-02-10T11:40:10.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.844 7ff0e66d1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff0e01b0470 0x7ff0e01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:10.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.844 7ff0e66d1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0e00ef610 0x7ff0e00efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:10.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.844 7ff0e66d1700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e00f0b50 con 0x7ff0e00f0030 2021-02-10T11:40:10.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.844 7ff0e66d1700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e00e52e0 con 0x7ff0e01b0470 2021-02-10T11:40:10.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.844 7ff0e66d1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00e54d0 con 0x7ff0e00ef610 2021-02-10T11:40:10.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.844 7ff0dffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff0e01b0470 0x7ff0e01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:10.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.844 7ff0dffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff0e01b0470 0x7ff0e01b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56248/0 (socket says 172.21.2.101:56248) 2021-02-10T11:40:10.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.844 7ff0dffff700 1 -- 172.21.2.101:0/2813619970 learned_addr learned my addr 172.21.2.101:0/2813619970 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:10.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.844 7ff0dffff700 1 -- 172.21.2.101:0/2813619970 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0e00ef610 msgr2=0x7ff0e00efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:40:10.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.844 7ff0dffff700 1 --2- 172.21.2.101:0/2813619970 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0e00ef610 0x7ff0e00efa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:10.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.844 7ff0dffff700 1 -- 172.21.2.101:0/2813619970 >> v1:172.21.2.108:6789/0 conn(0x7ff0e00f0030 legacy=0x7ff0e00f10d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:40:10.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.844 7ff0dffff700 1 -- 172.21.2.101:0/2813619970 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e00f0430 con 0x7ff0e01b0470 2021-02-10T11:40:10.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.844 7ff0dffff700 1 --2- 172.21.2.101:0/2813619970 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0e01b0470 0x7ff0e01b0870 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 rx=0x7ff0d0006700 tx=0x7ff0d0009b80).ready entity=mon.0 client_cookie=c012392ed88aa872 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:10.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.845 7ff0deffd700 1 -- 172.21.2.101:0/2813619970 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0d000c0c0 con 0x7ff0e01b0470 2021-02-10T11:40:10.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.845 7ff0deffd700 1 -- 172.21.2.101:0/2813619970 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d000c400 con 0x7ff0e01b0470 2021-02-10T11:40:10.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.845 7ff0deffd700 1 -- 172.21.2.101:0/2813619970 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0d000c920 con 0x7ff0e01b0470 2021-02-10T11:40:10.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.845 7ff0e66d1700 1 -- 172.21.2.101:0/2813619970 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0e01b0470 msgr2=0x7ff0e01b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:10.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.845 7ff0e66d1700 1 --2- 172.21.2.101:0/2813619970 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0e01b0470 0x7ff0e01b0870 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 rx=0x7ff0d0006700 tx=0x7ff0d0009b80).stop 2021-02-10T11:40:10.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.845 7ff0e66d1700 1 -- 172.21.2.101:0/2813619970 shutdown_connections 2021-02-10T11:40:10.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.845 7ff0e66d1700 1 --2- 172.21.2.101:0/2813619970 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0e00ef610 0x7ff0e00efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:10.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.845 7ff0e66d1700 1 --2- 172.21.2.101:0/2813619970 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0e01b0470 0x7ff0e01b0870 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:10.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.845 7ff0e66d1700 1 -- 172.21.2.101:0/2813619970 >> 172.21.2.101:0/2813619970 conn(0x7ff0e00ce3e0 msgr2=0x7ff0e00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:10.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.845 7ff0e66d1700 1 -- 172.21.2.101:0/2813619970 shutdown_connections 2021-02-10T11:40:10.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.845 7ff0e66d1700 1 -- 172.21.2.101:0/2813619970 wait complete. 2021-02-10T11:40:10.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.860 7ff0e66d1700 1 Processor -- start 2021-02-10T11:40:10.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.860 7ff0e66d1700 1 -- start start 2021-02-10T11:40:10.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.860 7ff0e66d1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0e0187e20 0x7ff0e0188220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:10.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.860 7ff0e66d1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0e0188840 0x7ff0e018d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:10.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.860 7ff0e66d1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0e018dcc0 0x7ff0e018e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:10.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.860 7ff0e66d1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00a9330 con 0x7ff0e018dcc0 2021-02-10T11:40:10.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.860 7ff0e66d1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e0191310 con 0x7ff0e0188840 2021-02-10T11:40:10.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.860 7ff0e66d1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e009da00 con 0x7ff0e0187e20 2021-02-10T11:40:10.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.861 7ff0e4cee700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0e018dcc0 0x7ff0e018e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:10.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.861 7ff0e4cee700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0e018dcc0 0x7ff0e018e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56258/0 (socket says 172.21.2.101:56258) 2021-02-10T11:40:10.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.861 7ff0e4cee700 1 -- 172.21.2.101:0/928996690 learned_addr learned my addr 172.21.2.101:0/928996690 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:10.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.861 7ff0e4cee700 1 -- 172.21.2.101:0/928996690 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0e0187e20 msgr2=0x7ff0e0188220 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:10.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.861 7ff0e4cee700 1 --2- 172.21.2.101:0/928996690 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0e0187e20 0x7ff0e0188220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:10.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.861 7ff0e4cee700 1 -- 172.21.2.101:0/928996690 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0e0188840 msgr2=0x7ff0e018d6a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:10.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.861 7ff0e4cee700 1 --2- 172.21.2.101:0/928996690 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0e0188840 0x7ff0e018d6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:10.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.861 7ff0e4cee700 1 -- 172.21.2.101:0/928996690 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e0188c40 con 0x7ff0e018dcc0 2021-02-10T11:40:10.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.861 7ff0e4cee700 1 --2- 172.21.2.101:0/928996690 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0e018dcc0 0x7ff0e018e0c0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 rx=0x7ff0d8009940 tx=0x7ff0d800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:10.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.861 7ff0dcff9700 1 -- 172.21.2.101:0/928996690 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0d8003030 con 0x7ff0e018dcc0 2021-02-10T11:40:10.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.861 7ff0dcff9700 1 -- 172.21.2.101:0/928996690 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d8001270 con 0x7ff0e018dcc0 2021-02-10T11:40:10.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.861 7ff0dcff9700 1 -- 172.21.2.101:0/928996690 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0d8001790 con 0x7ff0e018dcc0 2021-02-10T11:40:10.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.861 7ff0e66d1700 1 -- 172.21.2.101:0/928996690 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e0188c40 con 0x7ff0e018dcc0 2021-02-10T11:40:10.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.861 7ff0e66d1700 1 -- 172.21.2.101:0/928996690 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e018e9d0 con 0x7ff0e018dcc0 2021-02-10T11:40:10.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.862 7ff0dcff9700 1 -- 172.21.2.101:0/928996690 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff0d8001450 con 0x7ff0e018dcc0 2021-02-10T11:40:10.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.862 7ff0dcff9700 1 --2- 172.21.2.101:0/928996690 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0c00186c0 0x7ff0c001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:10.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.862 7ff0dcff9700 1 -- 172.21.2.101:0/928996690 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff0d8032780 con 0x7ff0e018dcc0 2021-02-10T11:40:10.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.862 7ff0dffff700 1 --2- 172.21.2.101:0/928996690 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0c00186c0 0x7ff0c001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:10.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.862 7ff0dffff700 1 --2- 172.21.2.101:0/928996690 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0c00186c0 0x7ff0c001abc0 crc :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:10.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.894 7ff0e66d1700 1 -- 172.21.2.101:0/928996690 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e00ede30 con 0x7ff0e018dcc0 2021-02-10T11:40:10.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:10.896 7ff0dcff9700 1 -- 172.21.2.101:0/928996690 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff0d801c7b0 con 0x7ff0e018dcc0 2021-02-10T11:40:11.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.022 7ff0e66d1700 1 -- 172.21.2.101:0/928996690 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0e00e52e0 con 0x7ff0e018dcc0 2021-02-10T11:40:11.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.022 7ff0dcff9700 1 -- 172.21.2.101:0/928996690 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+14005 (secure 0 0 0) 0x7ff0d801c7b0 con 0x7ff0e018dcc0 2021-02-10T11:40:11.026 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:40:11.026 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":103,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:39:57.056338","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:39:54.022810","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2021-02-10 11:39:55.095134","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:40:11.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.024 7ff0e66d1700 1 -- 172.21.2.101:0/928996690 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0c00186c0 msgr2=0x7ff0c001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:11.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.024 7ff0e66d1700 1 --2- 172.21.2.101:0/928996690 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0c00186c0 0x7ff0c001abc0 crc :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:11.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.024 7ff0e66d1700 1 -- 172.21.2.101:0/928996690 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0e018dcc0 msgr2=0x7ff0e018e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:11.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.024 7ff0e66d1700 1 --2- 172.21.2.101:0/928996690 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0e018dcc0 0x7ff0e018e0c0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 rx=0x7ff0d8009940 tx=0x7ff0d800bcf0).stop 2021-02-10T11:40:11.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.024 7ff0e66d1700 1 -- 172.21.2.101:0/928996690 shutdown_connections 2021-02-10T11:40:11.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.024 7ff0e66d1700 1 --2- 172.21.2.101:0/928996690 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0e018dcc0 0x7ff0e018e0c0 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:11.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.024 7ff0e66d1700 1 --2- 172.21.2.101:0/928996690 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0e0188840 0x7ff0e018d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:11.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.024 7ff0e66d1700 1 --2- 172.21.2.101:0/928996690 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0c00186c0 0x7ff0c001abc0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:11.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.024 7ff0e66d1700 1 --2- 172.21.2.101:0/928996690 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0e0187e20 0x7ff0e0188220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:11.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.024 7ff0e66d1700 1 -- 172.21.2.101:0/928996690 >> 172.21.2.101:0/928996690 conn(0x7ff0e0090d90 msgr2=0x7ff0e01b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:11.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.024 7ff0e66d1700 1 -- 172.21.2.101:0/928996690 shutdown_connections 2021-02-10T11:40:11.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.024 7ff0e66d1700 1 -- 172.21.2.101:0/928996690 wait complete. 2021-02-10T11:40:11.036 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:39:54.022810', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:39:55.095134', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:40:11.036 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:40:11.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.448 7f90ee17f700 1 Processor -- start 2021-02-10T11:40:11.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.448 7f90ee17f700 1 -- start start 2021-02-10T11:40:11.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.449 7f90ee17f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f90e80d1860 0x7f90e81b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:11.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.449 7f90ee17f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f90e80cff60 0x7f90e8109120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:11.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.449 7f90ee17f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e8109a50 con 0x7f90e81b0ad0 2021-02-10T11:40:11.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.449 7f90ee17f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e8109740 con 0x7f90e80cff60 2021-02-10T11:40:11.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.449 7f90ee17f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e8088070 con 0x7f90e80d1860 2021-02-10T11:40:11.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.450 7f90e7fff700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f90e80cff60 0x7f90e8109120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:11.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.450 7f90e77fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f90e80d1860 0x7f90e81b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:11.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.450 7f90e6ffd700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f90e81b0ad0 0x7f90e8108930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:50166/0 (socket says 172.21.2.101:50166) 2021-02-10T11:40:11.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.450 7f90e6ffd700 1 -- 172.21.2.101:0/1232260422 learned_addr learned my addr 172.21.2.101:0/1232260422 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:11.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.450 7f90e7fff700 1 -- 172.21.2.101:0/1232260422 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f90e80d1860 msgr2=0x7f90e81b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:11.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.450 7f90e7fff700 1 --2- 172.21.2.101:0/1232260422 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f90e80d1860 0x7f90e81b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:11.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.450 7f90e7fff700 1 -- 172.21.2.101:0/1232260422 >> v1:172.21.2.108:6789/0 conn(0x7f90e81b0ad0 legacy=0x7f90e8108930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:11.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.450 7f90e67fc700 1 -- 172.21.2.101:0/1232260422 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1496655773 0 0) 0x7f90d00010b0 con 0x7f90e81b0ad0 2021-02-10T11:40:11.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.450 7f90e7fff700 1 -- 172.21.2.101:0/1232260422 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e8088260 con 0x7f90e80cff60 2021-02-10T11:40:11.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.450 7f90e77fe700 1 --2- 172.21.2.101:0/1232260422 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f90e80d1860 0x7f90e81b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:40:11.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.451 7f90e7fff700 1 --2- 172.21.2.101:0/1232260422 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f90e80cff60 0x7f90e8109120 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f90e0006e50 tx=0x7f90e0009e70).ready entity=mon.1 client_cookie=156f9251ca209958 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:11.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.451 7f90e67fc700 1 -- 172.21.2.101:0/1232260422 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f90e000b510 con 0x7f90e80cff60 2021-02-10T11:40:11.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.451 7f90e67fc700 1 -- 172.21.2.101:0/1232260422 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e0006790 con 0x7f90e80cff60 2021-02-10T11:40:11.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.451 7f90e67fc700 1 -- 172.21.2.101:0/1232260422 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f90e000b940 con 0x7f90e80cff60 2021-02-10T11:40:11.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.451 7f90ee17f700 1 -- 172.21.2.101:0/1232260422 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f90e80cff60 msgr2=0x7f90e8109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:11.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.451 7f90ee17f700 1 --2- 172.21.2.101:0/1232260422 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f90e80cff60 0x7f90e8109120 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f90e0006e50 tx=0x7f90e0009e70).stop 2021-02-10T11:40:11.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.451 7f90ee17f700 1 -- 172.21.2.101:0/1232260422 shutdown_connections 2021-02-10T11:40:11.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.451 7f90ee17f700 1 --2- 172.21.2.101:0/1232260422 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f90e80cff60 0x7f90e8109120 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:11.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.451 7f90ee17f700 1 --2- 172.21.2.101:0/1232260422 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f90e80d1860 0x7f90e81b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:11.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.451 7f90ee17f700 1 -- 172.21.2.101:0/1232260422 >> 172.21.2.101:0/1232260422 conn(0x7f90e80ce410 msgr2=0x7f90e80d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:11.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.452 7f90ee17f700 1 -- 172.21.2.101:0/1232260422 shutdown_connections 2021-02-10T11:40:11.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.452 7f90ee17f700 1 -- 172.21.2.101:0/1232260422 wait complete. 2021-02-10T11:40:11.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.466 7f90ee17f700 1 Processor -- start 2021-02-10T11:40:11.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.466 7f90ee17f700 1 -- start start 2021-02-10T11:40:11.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.466 7f90ee17f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f90e80b92c0 0x7f90e80a2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:11.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.466 7f90ee17f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f90e80b96c0 0x7f90e80b8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.466 7f90ee17f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f90e80b8d60 0x7f90e809cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.466 7f90ee17f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e809d3b0 con 0x7f90e80b8d60 2021-02-10T11:40:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.466 7f90ee17f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e809d5d0 con 0x7f90e80b96c0 2021-02-10T11:40:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.466 7f90ee17f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e809d7f0 con 0x7f90e80b92c0 2021-02-10T11:40:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.467 7f90e7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f90e80b8d60 0x7f90e809cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.467 7f90e7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f90e80b8d60 0x7f90e809cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56270/0 (socket says 172.21.2.101:56270) 2021-02-10T11:40:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.467 7f90e7fff700 1 -- 172.21.2.101:0/1893734977 learned_addr learned my addr 172.21.2.101:0/1893734977 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.467 7f90e7fff700 1 -- 172.21.2.101:0/1893734977 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f90e80b92c0 msgr2=0x7f90e80a2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.467 7f90e7fff700 1 --2- 172.21.2.101:0/1893734977 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f90e80b92c0 0x7f90e80a2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.467 7f90e7fff700 1 -- 172.21.2.101:0/1893734977 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f90e80b96c0 msgr2=0x7f90e80b8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.467 7f90e7fff700 1 --2- 172.21.2.101:0/1893734977 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f90e80b96c0 0x7f90e80b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.467 7f90e7fff700 1 -- 172.21.2.101:0/1893734977 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e809da10 con 0x7f90e80b8d60 2021-02-10T11:40:11.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.467 7f90e7fff700 1 --2- 172.21.2.101:0/1893734977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f90e80b8d60 0x7f90e809cd90 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 rx=0x7f90e000b910 tx=0x7f90e000bcb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:11.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.467 7f90dffff700 1 -- 172.21.2.101:0/1893734977 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f90e0002f80 con 0x7f90e80b8d60 2021-02-10T11:40:11.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.467 7f90dffff700 1 -- 172.21.2.101:0/1893734977 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e00093d0 con 0x7f90e80b8d60 2021-02-10T11:40:11.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.467 7f90ee17f700 1 -- 172.21.2.101:0/1893734977 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e809da10 con 0x7f90e80b8d60 2021-02-10T11:40:11.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.468 7f90dffff700 1 -- 172.21.2.101:0/1893734977 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f90e000c4e0 con 0x7f90e80b8d60 2021-02-10T11:40:11.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.468 7f90ee17f700 1 -- 172.21.2.101:0/1893734977 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e809da10 con 0x7f90e80b8d60 2021-02-10T11:40:11.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.468 7f90dffff700 1 -- 172.21.2.101:0/1893734977 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f90e0002ef0 con 0x7f90e80b8d60 2021-02-10T11:40:11.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.468 7f90dffff700 1 --2- 172.21.2.101:0/1893734977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90cc0186c0 0x7f90cc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:11.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.468 7f90dffff700 1 -- 172.21.2.101:0/1893734977 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f90e00328f0 con 0x7f90e80b8d60 2021-02-10T11:40:11.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.468 7f90e77fe700 1 --2- 172.21.2.101:0/1893734977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90cc0186c0 0x7f90cc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:11.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.469 7f90e77fe700 1 --2- 172.21.2.101:0/1893734977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90cc0186c0 0x7f90cc01abc0 crc :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:11.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.498 7f90ee17f700 1 -- 172.21.2.101:0/1893734977 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e809e290 con 0x7f90e80b8d60 2021-02-10T11:40:11.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.501 7f90dffff700 1 -- 172.21.2.101:0/1893734977 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f90e002de90 con 0x7f90e80b8d60 2021-02-10T11:40:11.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.628 7f90ee17f700 1 -- 172.21.2.101:0/1893734977 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f90e809e290 con 0x7f90e80b8d60 2021-02-10T11:40:11.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.628 7f90dffff700 1 -- 172.21.2.101:0/1893734977 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v103) v1 ==== 92+0+10 (secure 0 0 0) 0x7f90e0000df0 con 0x7f90e80b8d60 2021-02-10T11:40:11.632 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:40:11.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.630 7f90ee17f700 1 -- 172.21.2.101:0/1893734977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90cc0186c0 msgr2=0x7f90cc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:11.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.630 7f90ee17f700 1 --2- 172.21.2.101:0/1893734977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90cc0186c0 0x7f90cc01abc0 crc :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:11.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.630 7f90ee17f700 1 -- 172.21.2.101:0/1893734977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f90e80b8d60 msgr2=0x7f90e809cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:11.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.630 7f90ee17f700 1 --2- 172.21.2.101:0/1893734977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f90e80b8d60 0x7f90e809cd90 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 rx=0x7f90e000b910 tx=0x7f90e000bcb0).stop 2021-02-10T11:40:11.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.631 7f90ee17f700 1 -- 172.21.2.101:0/1893734977 shutdown_connections 2021-02-10T11:40:11.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.631 7f90ee17f700 1 --2- 172.21.2.101:0/1893734977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f90e80b8d60 0x7f90e809cd90 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:11.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.631 7f90ee17f700 1 --2- 172.21.2.101:0/1893734977 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f90e80b96c0 0x7f90e80b8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:11.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.631 7f90ee17f700 1 --2- 172.21.2.101:0/1893734977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90cc0186c0 0x7f90cc01abc0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:11.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.631 7f90ee17f700 1 --2- 172.21.2.101:0/1893734977 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f90e80b92c0 0x7f90e80a2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:11.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.631 7f90ee17f700 1 -- 172.21.2.101:0/1893734977 >> 172.21.2.101:0/1893734977 conn(0x7f90e80a28e0 msgr2=0x7f90e8094c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:11.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.631 7f90ee17f700 1 -- 172.21.2.101:0/1893734977 shutdown_connections 2021-02-10T11:40:11.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:11.631 7f90ee17f700 1 -- 172.21.2.101:0/1893734977 wait complete. 2021-02-10T11:40:11.642 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:40:12.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.054 7fca5138a700 1 Processor -- start 2021-02-10T11:40:12.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.054 7fca5138a700 1 -- start start 2021-02-10T11:40:12.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.054 7fca5138a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fca4c0d1870 0x7fca4c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:12.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.054 7fca5138a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fca4c0cff70 0x7fca4c109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:12.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.054 7fca5138a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca4c109a60 con 0x7fca4c1b0ae0 2021-02-10T11:40:12.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.054 7fca5138a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca4c109750 con 0x7fca4c0cff70 2021-02-10T11:40:12.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.054 7fca5138a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c088070 con 0x7fca4c0d1870 2021-02-10T11:40:12.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.054 7fca4a87c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fca4c1b0ae0 0x7fca4c108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56768/0 (socket says 172.21.2.101:56768) 2021-02-10T11:40:12.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.054 7fca4a87c700 1 -- 172.21.2.101:0/4258516533 learned_addr learned my addr 172.21.2.101:0/4258516533 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:12.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.055 7fca4a07b700 1 -- 172.21.2.101:0/4258516533 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2009555215 0 0) 0x7fca340010b0 con 0x7fca4c1b0ae0 2021-02-10T11:40:12.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.055 7fca4a07b700 1 -- 172.21.2.101:0/4258516533 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca38002fe0 con 0x7fca4c1b0ae0 2021-02-10T11:40:12.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.055 7fca4b07d700 1 --2- 172.21.2.101:0/4258516533 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fca4c0d1870 0x7fca4c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:12.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.055 7fca4b87e700 1 --2- 172.21.2.101:0/4258516533 >> v2:172.21.2.108:3300/0 conn(0x7fca4c0cff70 0x7fca4c109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:12.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.055 7fca4a07b700 1 -- 172.21.2.101:0/4258516533 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2820695784 0 0) 0x7fca340010b0 con 0x7fca4c1b0ae0 2021-02-10T11:40:12.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.055 7fca4a07b700 1 -- 172.21.2.101:0/4258516533 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fca4c0d1870 msgr2=0x7fca4c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:12.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.055 7fca4a07b700 1 --2- 172.21.2.101:0/4258516533 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fca4c0d1870 0x7fca4c1b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:12.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.055 7fca4a07b700 1 -- 172.21.2.101:0/4258516533 >> v2:172.21.2.108:3300/0 conn(0x7fca4c0cff70 msgr2=0x7fca4c109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:12.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.055 7fca4a07b700 1 --2- 172.21.2.101:0/4258516533 >> v2:172.21.2.108:3300/0 conn(0x7fca4c0cff70 0x7fca4c109130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:12.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.055 7fca4a07b700 1 -- 172.21.2.101:0/4258516533 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c088260 con 0x7fca4c1b0ae0 2021-02-10T11:40:12.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.055 7fca4a07b700 1 -- 172.21.2.101:0/4258516533 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fca340015b0 con 0x7fca4c1b0ae0 2021-02-10T11:40:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.055 7fca4a07b700 1 -- 172.21.2.101:0/4258516533 >> v1:172.21.2.101:6789/0 conn(0x7fca4c1b0ae0 legacy=0x7fca4c108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.055 7fca4a07b700 1 --2- 172.21.2.101:0/4258516533 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca38003b10 0x7fca38006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.055 7fca4a07b700 1 --2- 172.21.2.101:0/4258516533 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fca3804a7f0 0x7fca3804cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.055 7fca4a07b700 1 --2- 172.21.2.101:0/4258516533 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fca3804d500 0x7fca3804fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.055 7fca4a07b700 1 -- 172.21.2.101:0/4258516533 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca380502d0 con 0x7fca38003b10 2021-02-10T11:40:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.055 7fca4a07b700 1 -- 172.21.2.101:0/4258516533 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca380504f0 con 0x7fca3804d500 2021-02-10T11:40:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.055 7fca4a07b700 1 -- 172.21.2.101:0/4258516533 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca38050710 con 0x7fca3804a7f0 2021-02-10T11:40:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.055 7fca4b07d700 1 --2- 172.21.2.101:0/4258516533 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca38003b10 0x7fca38006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.055 7fca4b07d700 1 -- 172.21.2.101:0/4258516533 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fca3804a7f0 msgr2=0x7fca3804cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.055 7fca4b07d700 1 --2- 172.21.2.101:0/4258516533 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fca3804a7f0 0x7fca3804cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.055 7fca4b87e700 1 --2- 172.21.2.101:0/4258516533 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fca3804d500 0x7fca3804fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.056 7fca4b07d700 1 -- 172.21.2.101:0/4258516533 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fca3804d500 msgr2=0x7fca3804fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.056 7fca4b07d700 1 --2- 172.21.2.101:0/4258516533 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fca3804d500 0x7fca3804fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:12.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.056 7fca4b07d700 1 -- 172.21.2.101:0/4258516533 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca380509b0 con 0x7fca38003b10 2021-02-10T11:40:12.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.056 7fca4b87e700 1 --2- 172.21.2.101:0/4258516533 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fca3804d500 0x7fca3804fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:40:12.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.056 7fca4b07d700 1 --2- 172.21.2.101:0/4258516533 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca38003b10 0x7fca38006010 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 rx=0x7fca3c009310 tx=0x7fca3c009a10).ready entity=mon.0 client_cookie=62fdd7238279088c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:12.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.056 7fca4a07b700 1 -- 172.21.2.101:0/4258516533 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fca3c0009b0 con 0x7fca38003b10 2021-02-10T11:40:12.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.056 7fca4a07b700 1 -- 172.21.2.101:0/4258516533 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3c006400 con 0x7fca38003b10 2021-02-10T11:40:12.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.056 7fca4a07b700 1 -- 172.21.2.101:0/4258516533 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fca3c00c340 con 0x7fca38003b10 2021-02-10T11:40:12.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.056 7fca5138a700 1 -- 172.21.2.101:0/4258516533 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca38003b10 msgr2=0x7fca38006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:12.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.056 7fca5138a700 1 --2- 172.21.2.101:0/4258516533 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca38003b10 0x7fca38006010 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 rx=0x7fca3c009310 tx=0x7fca3c009a10).stop 2021-02-10T11:40:12.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.056 7fca4b87e700 1 -- 172.21.2.101:0/4258516533 reap_dead start 2021-02-10T11:40:12.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.056 7fca5138a700 1 -- 172.21.2.101:0/4258516533 shutdown_connections 2021-02-10T11:40:12.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.056 7fca5138a700 1 -- 172.21.2.101:0/4258516533 >> 172.21.2.101:0/4258516533 conn(0x7fca4c0ce420 msgr2=0x7fca4c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:12.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.056 7fca5138a700 1 -- 172.21.2.101:0/4258516533 shutdown_connections 2021-02-10T11:40:12.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.056 7fca5138a700 1 -- 172.21.2.101:0/4258516533 wait complete. 2021-02-10T11:40:12.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.071 7fca5138a700 1 Processor -- start 2021-02-10T11:40:12.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.071 7fca5138a700 1 -- start start 2021-02-10T11:40:12.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.071 7fca5138a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca4c0abd60 0x7fca4c0ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:12.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.071 7fca5138a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fca4c0ac780 0x7fca4c0acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:12.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.071 7fca5138a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fca4c083910 0x7fca4c083d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:12.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.071 7fca5138a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c087180 con 0x7fca4c0abd60 2021-02-10T11:40:12.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.071 7fca5138a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c18fcd0 con 0x7fca4c0ac780 2021-02-10T11:40:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.071 7fca5138a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c084330 con 0x7fca4c083910 2021-02-10T11:40:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.072 7fca4b07d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca4c0abd60 0x7fca4c0ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.072 7fca4b07d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca4c0abd60 0x7fca4c0ac160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56282/0 (socket says 172.21.2.101:56282) 2021-02-10T11:40:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.072 7fca4b07d700 1 -- 172.21.2.101:0/3604942809 learned_addr learned my addr 172.21.2.101:0/3604942809 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.072 7fca4a87c700 1 --2- 172.21.2.101:0/3604942809 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fca4c0ac780 0x7fca4c0acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.072 7fca4b07d700 1 -- 172.21.2.101:0/3604942809 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fca4c083910 msgr2=0x7fca4c083d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.072 7fca4b07d700 1 --2- 172.21.2.101:0/3604942809 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fca4c083910 0x7fca4c083d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.072 7fca4b07d700 1 -- 172.21.2.101:0/3604942809 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fca4c0ac780 msgr2=0x7fca4c0acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.072 7fca4b07d700 1 --2- 172.21.2.101:0/3604942809 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fca4c0ac780 0x7fca4c0acb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.072 7fca4b07d700 1 -- 172.21.2.101:0/3604942809 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c084550 con 0x7fca4c0abd60 2021-02-10T11:40:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.072 7fca4a87c700 1 --2- 172.21.2.101:0/3604942809 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fca4c0ac780 0x7fca4c0acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:40:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.072 7fca4b07d700 1 --2- 172.21.2.101:0/3604942809 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca4c0abd60 0x7fca4c0ac160 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 rx=0x7fca3c009fc0 tx=0x7fca3c00b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:12.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.072 7fca43fff700 1 -- 172.21.2.101:0/3604942809 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fca3c00bd60 con 0x7fca4c0abd60 2021-02-10T11:40:12.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.072 7fca43fff700 1 -- 172.21.2.101:0/3604942809 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3c000c90 con 0x7fca4c0abd60 2021-02-10T11:40:12.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.072 7fca5138a700 1 -- 172.21.2.101:0/3604942809 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4c084550 con 0x7fca4c0abd60 2021-02-10T11:40:12.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.072 7fca43fff700 1 -- 172.21.2.101:0/3604942809 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fca3c0022b0 con 0x7fca4c0abd60 2021-02-10T11:40:12.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.072 7fca5138a700 1 -- 172.21.2.101:0/3604942809 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4c084ae0 con 0x7fca4c0abd60 2021-02-10T11:40:12.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.073 7fca43fff700 1 -- 172.21.2.101:0/3604942809 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fca3c0052e0 con 0x7fca4c0abd60 2021-02-10T11:40:12.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.073 7fca43fff700 1 --2- 172.21.2.101:0/3604942809 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fca300186c0 0x7fca3001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:12.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.073 7fca43fff700 1 -- 172.21.2.101:0/3604942809 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fca3c032480 con 0x7fca4c0abd60 2021-02-10T11:40:12.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.073 7fca4a87c700 1 --2- 172.21.2.101:0/3604942809 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fca300186c0 0x7fca3001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:12.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.073 7fca4a87c700 1 --2- 172.21.2.101:0/3604942809 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fca300186c0 0x7fca3001abc0 crc :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:12.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.104 7fca5138a700 1 -- 172.21.2.101:0/3604942809 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4c084d90 con 0x7fca4c0abd60 2021-02-10T11:40:12.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.107 7fca43fff700 1 -- 172.21.2.101:0/3604942809 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fca3c030db0 con 0x7fca4c0abd60 2021-02-10T11:40:12.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.233 7fca5138a700 1 -- 172.21.2.101:0/3604942809 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7fca4c084d90 con 0x7fca4c0abd60 2021-02-10T11:40:12.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.234 7fca43fff700 1 -- 172.21.2.101:0/3604942809 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v103) v1 ==== 104+0+10 (secure 0 0 0) 0x7fca3c00bd60 con 0x7fca4c0abd60 2021-02-10T11:40:12.237 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:40:12.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.236 7fca5138a700 1 -- 172.21.2.101:0/3604942809 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fca300186c0 msgr2=0x7fca3001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:12.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.236 7fca5138a700 1 --2- 172.21.2.101:0/3604942809 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fca300186c0 0x7fca3001abc0 crc :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:12.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.236 7fca5138a700 1 -- 172.21.2.101:0/3604942809 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca4c0abd60 msgr2=0x7fca4c0ac160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:12.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.236 7fca5138a700 1 --2- 172.21.2.101:0/3604942809 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca4c0abd60 0x7fca4c0ac160 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 rx=0x7fca3c009fc0 tx=0x7fca3c00b970).stop 2021-02-10T11:40:12.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.236 7fca5138a700 1 -- 172.21.2.101:0/3604942809 shutdown_connections 2021-02-10T11:40:12.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.236 7fca5138a700 1 --2- 172.21.2.101:0/3604942809 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fca300186c0 0x7fca3001abc0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:12.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.236 7fca5138a700 1 --2- 172.21.2.101:0/3604942809 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fca4c083910 0x7fca4c083d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:12.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.236 7fca5138a700 1 --2- 172.21.2.101:0/3604942809 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fca4c0ac780 0x7fca4c0acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:12.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.236 7fca5138a700 1 --2- 172.21.2.101:0/3604942809 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fca4c0abd60 0x7fca4c0ac160 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:12.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.236 7fca5138a700 1 -- 172.21.2.101:0/3604942809 >> 172.21.2.101:0/3604942809 conn(0x7fca4c0a2b30 msgr2=0x7fca4c09ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:12.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.236 7fca5138a700 1 -- 172.21.2.101:0/3604942809 shutdown_connections 2021-02-10T11:40:12.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.236 7fca5138a700 1 -- 172.21.2.101:0/3604942809 wait complete. 2021-02-10T11:40:12.247 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:40:12.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.659 7f11f8075700 1 Processor -- start 2021-02-10T11:40:12.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.659 7f11f8075700 1 -- start start 2021-02-10T11:40:12.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.660 7f11f8075700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f11f00d1870 0x7f11f01b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:12.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.660 7f11f8075700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f11f01b0ae0 0x7f11f0108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:12.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.660 7f11f8075700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f11f00d1c70 0x7f11f0108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:12.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.660 7f11f8075700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f00e5ef0 con 0x7f11f00d1c70 2021-02-10T11:40:12.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.660 7f11f8075700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f0109a60 con 0x7f11f01b0ae0 2021-02-10T11:40:12.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.660 7f11f8075700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f0088070 con 0x7f11f00d1870 2021-02-10T11:40:12.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.660 7f11f6692700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f11f00d1c70 0x7f11f0108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:12.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.660 7f11f6692700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f11f00d1c70 0x7f11f0108f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56290/0 (socket says 172.21.2.101:56290) 2021-02-10T11:40:12.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.660 7f11f6692700 1 -- 172.21.2.101:0/1172580945 learned_addr learned my addr 172.21.2.101:0/1172580945 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:12.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.660 7f11f5e91700 1 --2- 172.21.2.101:0/1172580945 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f11f00d1870 0x7f11f01b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:12.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.660 7f11f6692700 1 -- 172.21.2.101:0/1172580945 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f11f00d1870 msgr2=0x7f11f01b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:12.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.660 7f11f5690700 1 --2- 172.21.2.101:0/1172580945 >> v2:172.21.2.108:3300/0 conn(0x7f11f01b0ae0 0x7f11f0108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:12.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.660 7f11f6692700 1 --2- 172.21.2.101:0/1172580945 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f11f00d1870 0x7f11f01b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:12.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.660 7f11f6692700 1 -- 172.21.2.101:0/1172580945 >> v2:172.21.2.108:3300/0 conn(0x7f11f01b0ae0 msgr2=0x7f11f0108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:12.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.660 7f11f6692700 1 --2- 172.21.2.101:0/1172580945 >> v2:172.21.2.108:3300/0 conn(0x7f11f01b0ae0 0x7f11f0108940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:12.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.660 7f11f6692700 1 -- 172.21.2.101:0/1172580945 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f0109580 con 0x7f11f00d1c70 2021-02-10T11:40:12.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.660 7f11f5690700 1 --2- 172.21.2.101:0/1172580945 >> v2:172.21.2.108:3300/0 conn(0x7f11f01b0ae0 0x7f11f0108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:40:12.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.660 7f11f6692700 1 --2- 172.21.2.101:0/1172580945 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f11f00d1c70 0x7f11f0108f60 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 rx=0x7f11e8006f00 tx=0x7f11e8009f10).ready entity=mon.0 client_cookie=bda943e470ae5d49 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:12.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.660 7f11f4e8f700 1 -- 172.21.2.101:0/1172580945 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f11e800b3b0 con 0x7f11f00d1c70 2021-02-10T11:40:12.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.661 7f11f4e8f700 1 -- 172.21.2.101:0/1172580945 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e800b630 con 0x7f11f00d1c70 2021-02-10T11:40:12.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.661 7f11f4e8f700 1 -- 172.21.2.101:0/1172580945 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f11e800bb50 con 0x7f11f00d1c70 2021-02-10T11:40:12.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.661 7f11f8075700 1 -- 172.21.2.101:0/1172580945 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f11f00d1c70 msgr2=0x7f11f0108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:12.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.661 7f11f8075700 1 --2- 172.21.2.101:0/1172580945 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f11f00d1c70 0x7f11f0108f60 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 rx=0x7f11e8006f00 tx=0x7f11e8009f10).stop 2021-02-10T11:40:12.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.661 7f11f8075700 1 -- 172.21.2.101:0/1172580945 shutdown_connections 2021-02-10T11:40:12.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.661 7f11f8075700 1 --2- 172.21.2.101:0/1172580945 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f11f00d1c70 0x7f11f0108f60 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:12.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.661 7f11f8075700 1 --2- 172.21.2.101:0/1172580945 >> v2:172.21.2.108:3300/0 conn(0x7f11f01b0ae0 0x7f11f0108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:12.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.661 7f11f8075700 1 --2- 172.21.2.101:0/1172580945 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f11f00d1870 0x7f11f01b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:12.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.661 7f11f8075700 1 -- 172.21.2.101:0/1172580945 >> 172.21.2.101:0/1172580945 conn(0x7f11f00ce420 msgr2=0x7f11f00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:12.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.661 7f11f8075700 1 -- 172.21.2.101:0/1172580945 shutdown_connections 2021-02-10T11:40:12.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.661 7f11f8075700 1 -- 172.21.2.101:0/1172580945 wait complete. 2021-02-10T11:40:12.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.676 7f11f8075700 1 Processor -- start 2021-02-10T11:40:12.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.676 7f11f8075700 1 -- start start 2021-02-10T11:40:12.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.676 7f11f8075700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f11f00abd60 0x7f11f00ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:12.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.676 7f11f8075700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f11f00ac780 0x7f11f00acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:12.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.676 7f11f8075700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f11f0083910 0x7f11f0083d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:12.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.676 7f11f8075700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f0087180 con 0x7f11f0083910 2021-02-10T11:40:12.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.676 7f11f8075700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f018fcd0 con 0x7f11f00abd60 2021-02-10T11:40:12.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.676 7f11f8075700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f0084330 con 0x7f11f00ac780 2021-02-10T11:40:12.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.677 7f11f6692700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f11f0083910 0x7f11f0083d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:12.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.677 7f11f6692700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f11f0083910 0x7f11f0083d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56296/0 (socket says 172.21.2.101:56296) 2021-02-10T11:40:12.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.677 7f11f6692700 1 -- 172.21.2.101:0/2079741121 learned_addr learned my addr 172.21.2.101:0/2079741121 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:12.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.677 7f11f6692700 1 -- 172.21.2.101:0/2079741121 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f11f00ac780 msgr2=0x7f11f00acb80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:12.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.677 7f11f6692700 1 --2- 172.21.2.101:0/2079741121 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f11f00ac780 0x7f11f00acb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:12.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.677 7f11f6692700 1 -- 172.21.2.101:0/2079741121 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f11f00abd60 msgr2=0x7f11f00ac160 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:12.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.677 7f11f6692700 1 --2- 172.21.2.101:0/2079741121 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f11f00abd60 0x7f11f00ac160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:12.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.677 7f11f6692700 1 -- 172.21.2.101:0/2079741121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f0084550 con 0x7f11f0083910 2021-02-10T11:40:12.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.677 7f11f6692700 1 --2- 172.21.2.101:0/2079741121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f11f0083910 0x7f11f0083d10 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 rx=0x7f11e8009390 tx=0x7f11e800bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:12.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.677 7f11e27fc700 1 -- 172.21.2.101:0/2079741121 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f11e8003030 con 0x7f11f0083910 2021-02-10T11:40:12.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.677 7f11e27fc700 1 -- 172.21.2.101:0/2079741121 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e800c2f0 con 0x7f11f0083910 2021-02-10T11:40:12.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.677 7f11f8075700 1 -- 172.21.2.101:0/2079741121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f0084550 con 0x7f11f0083910 2021-02-10T11:40:12.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.677 7f11e27fc700 1 -- 172.21.2.101:0/2079741121 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f11e800c850 con 0x7f11f0083910 2021-02-10T11:40:12.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.677 7f11f8075700 1 -- 172.21.2.101:0/2079741121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f0084550 con 0x7f11f0083910 2021-02-10T11:40:12.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.678 7f11e27fc700 1 -- 172.21.2.101:0/2079741121 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f11e8001270 con 0x7f11f0083910 2021-02-10T11:40:12.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.678 7f11e27fc700 1 --2- 172.21.2.101:0/2079741121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f11d8018730 0x7f11d801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:12.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.678 7f11e27fc700 1 -- 172.21.2.101:0/2079741121 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f11e8032b80 con 0x7f11f0083910 2021-02-10T11:40:12.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.678 7f11f5e91700 1 --2- 172.21.2.101:0/2079741121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f11d8018730 0x7f11d801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:12.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.678 7f11f5e91700 1 --2- 172.21.2.101:0/2079741121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f11d8018730 0x7f11d801ac10 crc :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:12.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.710 7f11f8075700 1 -- 172.21.2.101:0/2079741121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f0082980 con 0x7f11f0083910 2021-02-10T11:40:12.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.712 7f11e27fc700 1 -- 172.21.2.101:0/2079741121 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f11e8000cc0 con 0x7f11f0083910 2021-02-10T11:40:12.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.839 7f11f8075700 1 -- 172.21.2.101:0/2079741121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f11f0109a60 con 0x7f11f0083910 2021-02-10T11:40:12.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.840 7f11e27fc700 1 -- 172.21.2.101:0/2079741121 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v103) v1 ==== 100+0+10 (secure 0 0 0) 0x7f11e8001270 con 0x7f11f0083910 2021-02-10T11:40:12.843 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:40:12.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.842 7f11f8075700 1 -- 172.21.2.101:0/2079741121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f11d8018730 msgr2=0x7f11d801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:12.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.842 7f11f8075700 1 --2- 172.21.2.101:0/2079741121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f11d8018730 0x7f11d801ac10 crc :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:12.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.842 7f11f8075700 1 -- 172.21.2.101:0/2079741121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f11f0083910 msgr2=0x7f11f0083d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:12.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.842 7f11f8075700 1 --2- 172.21.2.101:0/2079741121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f11f0083910 0x7f11f0083d10 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 rx=0x7f11e8009390 tx=0x7f11e800bd40).stop 2021-02-10T11:40:12.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.842 7f11f8075700 1 -- 172.21.2.101:0/2079741121 shutdown_connections 2021-02-10T11:40:12.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.842 7f11f8075700 1 --2- 172.21.2.101:0/2079741121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f11d8018730 0x7f11d801ac10 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:12.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.842 7f11f8075700 1 --2- 172.21.2.101:0/2079741121 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f11f00ac780 0x7f11f00acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:12.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.842 7f11f8075700 1 --2- 172.21.2.101:0/2079741121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f11f0083910 0x7f11f0083d10 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:12.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.842 7f11f8075700 1 --2- 172.21.2.101:0/2079741121 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f11f00abd60 0x7f11f00ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:12.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.842 7f11f8075700 1 -- 172.21.2.101:0/2079741121 >> 172.21.2.101:0/2079741121 conn(0x7f11f00a2b30 msgr2=0x7f11f009ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:12.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.842 7f11f8075700 1 -- 172.21.2.101:0/2079741121 shutdown_connections 2021-02-10T11:40:12.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:12.842 7f11f8075700 1 -- 172.21.2.101:0/2079741121 wait complete. 2021-02-10T11:40:12.853 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:40:12.853 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:40:13.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.262 7fae29259700 1 Processor -- start 2021-02-10T11:40:13.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.262 7fae29259700 1 -- start start 2021-02-10T11:40:13.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.263 7fae29259700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae240d1870 0x7fae241b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:13.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.263 7fae29259700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae240f0b10 con 0x7fae240f0650 2021-02-10T11:40:13.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.263 7fae29259700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae240d0310 con 0x7fae241b0a10 2021-02-10T11:40:13.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.263 7fae29259700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae240e5ed0 con 0x7fae240d1870 2021-02-10T11:40:13.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.263 7fae2359e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fae240f0650 0x7fae240cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56800/0 (socket says 172.21.2.101:56800) 2021-02-10T11:40:13.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.263 7fae2359e700 1 -- 172.21.2.101:0/1055518580 learned_addr learned my addr 172.21.2.101:0/1055518580 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:13.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.263 7fae22d9d700 1 --2- 172.21.2.101:0/1055518580 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae240d1870 0x7fae241b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:13.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.263 7fae21d9b700 1 -- 172.21.2.101:0/1055518580 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 456564831 0 0) 0x7fae1c001500 con 0x7fae240f0650 2021-02-10T11:40:13.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.263 7fae21d9b700 1 -- 172.21.2.101:0/1055518580 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae10002fe0 con 0x7fae240f0650 2021-02-10T11:40:13.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.264 7fae22d9d700 1 -- 172.21.2.101:0/1055518580 >> v1:172.21.2.108:6789/0 conn(0x7fae241b0a10 legacy=0x7fae240cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:13.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.264 7fae22d9d700 1 -- 172.21.2.101:0/1055518580 >> v1:172.21.2.101:6789/0 conn(0x7fae240f0650 legacy=0x7fae240cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:13.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.264 7fae22d9d700 1 -- 172.21.2.101:0/1055518580 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae240d0530 con 0x7fae240d1870 2021-02-10T11:40:13.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.264 7fae22d9d700 1 --2- 172.21.2.101:0/1055518580 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae240d1870 0x7fae241b03f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fae14006650 tx=0x7fae14009a80).ready entity=mon.2 client_cookie=1d94cda9b241d123 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:13.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.264 7fae21d9b700 1 -- 172.21.2.101:0/1055518580 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fae1400c0c0 con 0x7fae240d1870 2021-02-10T11:40:13.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.264 7fae21d9b700 1 -- 172.21.2.101:0/1055518580 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1400c340 con 0x7fae240d1870 2021-02-10T11:40:13.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.264 7fae21d9b700 1 -- 172.21.2.101:0/1055518580 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fae1400c860 con 0x7fae240d1870 2021-02-10T11:40:13.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.264 7fae29259700 1 -- 172.21.2.101:0/1055518580 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae240d1870 msgr2=0x7fae241b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:13.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.264 7fae29259700 1 --2- 172.21.2.101:0/1055518580 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae240d1870 0x7fae241b03f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fae14006650 tx=0x7fae14009a80).stop 2021-02-10T11:40:13.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.265 7fae29259700 1 -- 172.21.2.101:0/1055518580 shutdown_connections 2021-02-10T11:40:13.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.265 7fae29259700 1 --2- 172.21.2.101:0/1055518580 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae240d1870 0x7fae241b03f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:13.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.265 7fae29259700 1 -- 172.21.2.101:0/1055518580 >> 172.21.2.101:0/1055518580 conn(0x7fae240ce450 msgr2=0x7fae240d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:13.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.265 7fae29259700 1 -- 172.21.2.101:0/1055518580 shutdown_connections 2021-02-10T11:40:13.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.265 7fae29259700 1 -- 172.21.2.101:0/1055518580 wait complete. 2021-02-10T11:40:13.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.280 7fae29259700 1 Processor -- start 2021-02-10T11:40:13.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.280 7fae29259700 1 -- start start 2021-02-10T11:40:13.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.280 7fae29259700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae240b4f30 0x7fae2418b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:13.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.280 7fae29259700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fae24093f40 0x7fae24094340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:13.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.280 7fae29259700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae24094960 0x7fae24187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:13.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.280 7fae29259700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae24094e90 con 0x7fae240b4f30 2021-02-10T11:40:13.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.280 7fae29259700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2418bff0 con 0x7fae24093f40 2021-02-10T11:40:13.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.280 7fae29259700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2408f780 con 0x7fae24094960 2021-02-10T11:40:13.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.281 7fae22d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae240b4f30 0x7fae2418b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:13.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.281 7fae22d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae240b4f30 0x7fae2418b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56308/0 (socket says 172.21.2.101:56308) 2021-02-10T11:40:13.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.281 7fae22d9d700 1 -- 172.21.2.101:0/2603895459 learned_addr learned my addr 172.21.2.101:0/2603895459 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:13.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.281 7fae2359e700 1 --2- 172.21.2.101:0/2603895459 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae24094960 0x7fae24187e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:13.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.281 7fae2259c700 1 --2- 172.21.2.101:0/2603895459 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fae24093f40 0x7fae24094340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:13.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.281 7fae22d9d700 1 -- 172.21.2.101:0/2603895459 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae24094960 msgr2=0x7fae24187e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:13.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.281 7fae22d9d700 1 --2- 172.21.2.101:0/2603895459 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae24094960 0x7fae24187e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:13.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.281 7fae22d9d700 1 -- 172.21.2.101:0/2603895459 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fae24093f40 msgr2=0x7fae24094340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:13.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.281 7fae22d9d700 1 --2- 172.21.2.101:0/2603895459 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fae24093f40 0x7fae24094340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:13.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.281 7fae22d9d700 1 -- 172.21.2.101:0/2603895459 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae24188470 con 0x7fae240b4f30 2021-02-10T11:40:13.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.281 7fae22d9d700 1 --2- 172.21.2.101:0/2603895459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae240b4f30 0x7fae2418b9d0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 rx=0x7fae14009630 tx=0x7fae1400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:13.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.281 7fae1b7fe700 1 -- 172.21.2.101:0/2603895459 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fae140093a0 con 0x7fae240b4f30 2021-02-10T11:40:13.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.281 7fae1b7fe700 1 -- 172.21.2.101:0/2603895459 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1400bd70 con 0x7fae240b4f30 2021-02-10T11:40:13.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.281 7fae29259700 1 -- 172.21.2.101:0/2603895459 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae24188470 con 0x7fae240b4f30 2021-02-10T11:40:13.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.281 7fae29259700 1 -- 172.21.2.101:0/2603895459 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae24188a00 con 0x7fae240b4f30 2021-02-10T11:40:13.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.281 7fae1b7fe700 1 -- 172.21.2.101:0/2603895459 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fae14005560 con 0x7fae240b4f30 2021-02-10T11:40:13.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.282 7fae1b7fe700 1 -- 172.21.2.101:0/2603895459 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fae1400a000 con 0x7fae240b4f30 2021-02-10T11:40:13.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.282 7fae1b7fe700 1 --2- 172.21.2.101:0/2603895459 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fae08018730 0x7fae0801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:13.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.282 7fae1b7fe700 1 -- 172.21.2.101:0/2603895459 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fae14032800 con 0x7fae240b4f30 2021-02-10T11:40:13.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.282 7fae2259c700 1 --2- 172.21.2.101:0/2603895459 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fae08018730 0x7fae0801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:13.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.282 7fae2259c700 1 --2- 172.21.2.101:0/2603895459 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fae08018730 0x7fae0801ac10 crc :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:13.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.313 7fae29259700 1 -- 172.21.2.101:0/2603895459 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae240ce450 con 0x7fae240b4f30 2021-02-10T11:40:13.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.315 7fae1b7fe700 1 -- 172.21.2.101:0/2603895459 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fae14009340 con 0x7fae240b4f30 2021-02-10T11:40:13.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.443 7fae29259700 1 -- 172.21.2.101:0/2603895459 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae240ce450 con 0x7fae240b4f30 2021-02-10T11:40:13.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.444 7fae1b7fe700 1 -- 172.21.2.101:0/2603895459 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+2107 (secure 0 0 0) 0x7fae1400bd70 con 0x7fae240b4f30 2021-02-10T11:40:13.447 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 79 2021-02-10T11:40:13.447 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:40:13.447 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":79,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10125,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/3121560299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3121560299},{"type":"v1","addr":"172.21.2.101:6835","nonce":3121560299}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":76}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:39:56.054150","modified":"2021-02-10 11:39:58.060269","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10122},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10122":{"gid":10122,"name":"b","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/1690286235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":1690286235},{"type":"v1","addr":"172.21.2.108:6833","nonce":1690286235}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2021-02-10T11:40:13.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.446 7fae29259700 1 -- 172.21.2.101:0/2603895459 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fae08018730 msgr2=0x7fae0801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:13.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.446 7fae29259700 1 --2- 172.21.2.101:0/2603895459 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fae08018730 0x7fae0801ac10 crc :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:13.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.446 7fae29259700 1 -- 172.21.2.101:0/2603895459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae240b4f30 msgr2=0x7fae2418b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:13.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.446 7fae29259700 1 --2- 172.21.2.101:0/2603895459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae240b4f30 0x7fae2418b9d0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 rx=0x7fae14009630 tx=0x7fae1400b980).stop 2021-02-10T11:40:13.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.446 7fae29259700 1 -- 172.21.2.101:0/2603895459 shutdown_connections 2021-02-10T11:40:13.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.446 7fae29259700 1 --2- 172.21.2.101:0/2603895459 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fae08018730 0x7fae0801ac10 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:13.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.446 7fae29259700 1 --2- 172.21.2.101:0/2603895459 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae24094960 0x7fae24187e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:13.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.446 7fae29259700 1 --2- 172.21.2.101:0/2603895459 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fae24093f40 0x7fae24094340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:13.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.446 7fae29259700 1 --2- 172.21.2.101:0/2603895459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae240b4f30 0x7fae2418b9d0 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:13.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.446 7fae29259700 1 -- 172.21.2.101:0/2603895459 >> 172.21.2.101:0/2603895459 conn(0x7fae2419f8f0 msgr2=0x7fae241a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:13.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.446 7fae29259700 1 -- 172.21.2.101:0/2603895459 shutdown_connections 2021-02-10T11:40:13.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:13.446 7fae29259700 1 -- 172.21.2.101:0/2603895459 wait complete. 2021-02-10T11:40:13.659 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 79, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:39:56.054150', 'modified': '2021-02-10 11:39:58.060269', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10122}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10122': {'gid': 10122, 'name': 'b', 'rank': 0, 'incarnation': 77, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/1690286235', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 1690286235}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 1690286235}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:40:13.659 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:40:13.659 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:40:14.059 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:40:14.060 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:40:14.060 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:40:14.060 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 10122, 2021-02-10T11:40:14.060 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:40:14.060 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:40:14.060 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 18.000357284, 2021-02-10T11:40:14.060 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 79, 2021-02-10T11:40:14.060 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 103, 2021-02-10T11:40:14.061 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 103, 2021-02-10T11:40:14.061 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 23.636157559000001 2021-02-10T11:40:14.061 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:40:14.063 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 10122, "want_state": "up:active", "state": "up:active", "rank_uptime": 18.000357284, "mdsmap_epoch": 79, "osdmap_epoch": 103, "osdmap_epoch_barrier": 103, "uptime": 23.636157559000001 } 2021-02-10T11:40:14.063 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:40:14.063 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-02-10T11:40:14.063 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:40:14.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:14.064 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:40:14.429 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:40:14.776 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:40:14.776 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:40:15.119 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:40:15.120 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:40:15.120 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:40:15.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:15.121 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:40:15.456 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:40:15.456 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2021-02-10T11:40:15.456 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:40:15.460 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:40:15.803 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:40:15.804 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:40:15.869 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:2021-02-10 11:40:15.866 7f63c5633f80 -1 init, newargv = 0x55a4a818e9a0 newargc=9 2021-02-10T11:40:15.869 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[144090]: starting ceph client 2021-02-10T11:40:15.874 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[144090]: starting fuse 2021-02-10T11:40:16.145 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:40:16.146 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:40:16.146 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:40:16.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:16.147 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:40:16.483 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:40:16.483 INFO:teuthology.orchestra.run.gibba001.stdout:44 2021-02-10T11:40:16.483 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2021-02-10T11:40:16.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:16.484 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:40:16.484 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:40:16.484 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:40:16.484 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:40:16.484 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:40:16.484 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:40:16.484 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:40:16.484 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-02-10T11:40:16.484 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:40:16.484 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:40:16.484 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:40:16.485 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:40:16.485 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:40:16.485 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:40:16.485 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:40:16.485 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:40:16.485 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:40:16.485 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:40:16.485 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:40:16.485 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:40:16.485 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:40:16.485 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.1")) 2021-02-10T11:40:16.485 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:40:16.847 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.1.144090.asok 2021-02-10T11:40:16.850 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.144090.asok 2021-02-10T11:40:16.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:16.850 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.144090.asok status 2021-02-10T11:40:17.240 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:40:17.241 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:40:17.241 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:40:17.241 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:40:17.241 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "1", 2021-02-10T11:40:17.241 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:40:17.241 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:40:17.241 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "144090", 2021-02-10T11:40:17.241 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:40:17.241 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:17.241 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:40:17.241 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:40:17.241 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 10317, 2021-02-10T11:40:17.241 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:40:17.242 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:40:17.242 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:40:17.242 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 10317 2021-02-10T11:40:17.242 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:17.242 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:40:17.242 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:17.242 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:40:17.242 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 816148985 2021-02-10T11:40:17.242 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:17.242 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:17.242 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:40:17.242 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:17.242 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:40:17.243 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 816148985 2021-02-10T11:40:17.243 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:17.243 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.10317 172.21.2.101:0/816148985", 2021-02-10T11:40:17.243 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/816148985", 2021-02-10T11:40:17.243 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:40:17.243 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 79, 2021-02-10T11:40:17.243 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 103, 2021-02-10T11:40:17.243 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:40:17.243 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:40:17.243 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:40:17.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:17.245 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:40:17.245 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:40:17.580 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:40:17.580 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:40:17.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:17.580 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:40:17.926 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2021-02-10T11:40:18.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.338 7f548d170700 1 Processor -- start 2021-02-10T11:40:18.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.339 7f548d170700 1 -- start start 2021-02-10T11:40:18.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.339 7f548d170700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f54880d1870 0x7f54881b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:18.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.339 7f548d170700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f54881b0ae0 0x7f5488108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:18.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.339 7f548d170700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54880fe1c0 con 0x7f54880d1c70 2021-02-10T11:40:18.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.339 7f548d170700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5488109a60 con 0x7f54881b0ae0 2021-02-10T11:40:18.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.339 7f548d170700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5488109750 con 0x7f54880d1870 2021-02-10T11:40:18.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.339 7f548659c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f54881b0ae0 0x7f5488108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:18.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.339 7f548759e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f54880d1c70 0x7f5488108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56832/0 (socket says 172.21.2.101:56832) 2021-02-10T11:40:18.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.339 7f548759e700 1 -- 172.21.2.101:0/68266067 learned_addr learned my addr 172.21.2.101:0/68266067 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:18.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.340 7f548659c700 1 -- 172.21.2.101:0/68266067 >> v2:172.21.2.108:3300/0 conn(0x7f54880d1870 msgr2=0x7f54881b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:18.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.340 7f548659c700 1 --2- 172.21.2.101:0/68266067 >> v2:172.21.2.108:3300/0 conn(0x7f54880d1870 0x7f54881b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:18.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.340 7f548659c700 1 -- 172.21.2.101:0/68266067 >> v1:172.21.2.101:6789/0 conn(0x7f54880d1c70 legacy=0x7f5488108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:18.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.340 7f548659c700 1 -- 172.21.2.101:0/68266067 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5488088070 con 0x7f54881b0ae0 2021-02-10T11:40:18.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.340 7f5485d9b700 1 -- 172.21.2.101:0/68266067 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 606752930 0 0) 0x7f5480001500 con 0x7f54880d1c70 2021-02-10T11:40:18.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.340 7f548659c700 1 --2- 172.21.2.101:0/68266067 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54881b0ae0 0x7f5488108940 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 rx=0x7f5478006650 tx=0x7f5478009a80).ready entity=mon.0 client_cookie=944d4f006dc87b4f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:18.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.340 7f5485d9b700 1 -- 172.21.2.101:0/68266067 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f547800c0c0 con 0x7f54881b0ae0 2021-02-10T11:40:18.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.340 7f5485d9b700 1 -- 172.21.2.101:0/68266067 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547800c340 con 0x7f54881b0ae0 2021-02-10T11:40:18.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.340 7f5485d9b700 1 -- 172.21.2.101:0/68266067 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f547800c860 con 0x7f54881b0ae0 2021-02-10T11:40:18.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.340 7f548d170700 1 -- 172.21.2.101:0/68266067 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54881b0ae0 msgr2=0x7f5488108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:18.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.340 7f548d170700 1 --2- 172.21.2.101:0/68266067 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54881b0ae0 0x7f5488108940 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 rx=0x7f5478006650 tx=0x7f5478009a80).stop 2021-02-10T11:40:18.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.341 7f548d170700 1 -- 172.21.2.101:0/68266067 shutdown_connections 2021-02-10T11:40:18.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.341 7f548d170700 1 --2- 172.21.2.101:0/68266067 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54881b0ae0 0x7f5488108940 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:18.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.341 7f548d170700 1 --2- 172.21.2.101:0/68266067 >> v2:172.21.2.108:3300/0 conn(0x7f54880d1870 0x7f54881b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:18.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.341 7f548d170700 1 -- 172.21.2.101:0/68266067 >> 172.21.2.101:0/68266067 conn(0x7f54880ce420 msgr2=0x7f54880d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:18.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.341 7f548d170700 1 -- 172.21.2.101:0/68266067 shutdown_connections 2021-02-10T11:40:18.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.341 7f548d170700 1 -- 172.21.2.101:0/68266067 wait complete. 2021-02-10T11:40:18.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.355 7f548d170700 1 Processor -- start 2021-02-10T11:40:18.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.355 7f548d170700 1 -- start start 2021-02-10T11:40:18.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.356 7f548d170700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f548819e5c0 0x7f5488099fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:18.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.356 7f548d170700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f548809a5d0 0x7f548809a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:18.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.356 7f548d170700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54880982a0 0x7f54880986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:18.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.356 7f548d170700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548819feb0 con 0x7f548819e5c0 2021-02-10T11:40:18.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.356 7f548d170700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5488099a30 con 0x7f54880982a0 2021-02-10T11:40:18.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.356 7f548d170700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548819fb30 con 0x7f548809a5d0 2021-02-10T11:40:18.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.356 7f5486d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f548819e5c0 0x7f5488099fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:18.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.356 7f5486d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f548819e5c0 0x7f5488099fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56342/0 (socket says 172.21.2.101:56342) 2021-02-10T11:40:18.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.356 7f5486d9d700 1 -- 172.21.2.101:0/2988903861 learned_addr learned my addr 172.21.2.101:0/2988903861 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:18.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.356 7f5486d9d700 1 -- 172.21.2.101:0/2988903861 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f548809a5d0 msgr2=0x7f548809a9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:18.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.356 7f5486d9d700 1 --2- 172.21.2.101:0/2988903861 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f548809a5d0 0x7f548809a9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:18.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.356 7f5486d9d700 1 -- 172.21.2.101:0/2988903861 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54880982a0 msgr2=0x7f54880986a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:18.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.356 7f5486d9d700 1 --2- 172.21.2.101:0/2988903861 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54880982a0 0x7f54880986a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:18.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.356 7f5486d9d700 1 -- 172.21.2.101:0/2988903861 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548819e9c0 con 0x7f548819e5c0 2021-02-10T11:40:18.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.356 7f5486d9d700 1 --2- 172.21.2.101:0/2988903861 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f548819e5c0 0x7f5488099fb0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 rx=0x7f5470006f00 tx=0x7f547000b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:18.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.357 7f547f7fe700 1 -- 172.21.2.101:0/2988903861 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5470006350 con 0x7f548819e5c0 2021-02-10T11:40:18.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.357 7f547f7fe700 1 -- 172.21.2.101:0/2988903861 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547000be00 con 0x7f548819e5c0 2021-02-10T11:40:18.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.357 7f548d170700 1 -- 172.21.2.101:0/2988903861 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548819e9c0 con 0x7f548819e5c0 2021-02-10T11:40:18.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.357 7f548d170700 1 -- 172.21.2.101:0/2988903861 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5488098fb0 con 0x7f548819e5c0 2021-02-10T11:40:18.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.357 7f547f7fe700 1 -- 172.21.2.101:0/2988903861 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5470002020 con 0x7f548819e5c0 2021-02-10T11:40:18.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.357 7f547f7fe700 1 -- 172.21.2.101:0/2988903861 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5470005310 con 0x7f548819e5c0 2021-02-10T11:40:18.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.357 7f547f7fe700 1 --2- 172.21.2.101:0/2988903861 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f546c018730 0x7f546c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:18.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.358 7f547f7fe700 1 -- 172.21.2.101:0/2988903861 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f54700327d0 con 0x7f548819e5c0 2021-02-10T11:40:18.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.358 7f548659c700 1 --2- 172.21.2.101:0/2988903861 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f546c018730 0x7f546c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:18.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.358 7f548659c700 1 --2- 172.21.2.101:0/2988903861 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f546c018730 0x7f546c01ac10 crc :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:18.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.389 7f548d170700 1 -- 172.21.2.101:0/2988903861 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54880825d0 con 0x7f548819e5c0 2021-02-10T11:40:18.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.392 7f547f7fe700 1 -- 172.21.2.101:0/2988903861 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5470000960 con 0x7f548819e5c0 2021-02-10T11:40:18.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.529 7f548d170700 1 -- 172.21.2.101:0/2988903861 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f54880825d0 con 0x7f548819e5c0 2021-02-10T11:40:18.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.583 7f547f7fe700 1 -- 172.21.2.101:0/2988903861 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v80) v1 ==== 136+0+0 (secure 0 0 0) 0x7f5470005310 con 0x7f548819e5c0 2021-02-10T11:40:18.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.595 7f548d170700 1 -- 172.21.2.101:0/2988903861 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f546c018730 msgr2=0x7f546c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:18.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.595 7f548d170700 1 --2- 172.21.2.101:0/2988903861 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f546c018730 0x7f546c01ac10 crc :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:18.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.595 7f548d170700 1 -- 172.21.2.101:0/2988903861 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f548819e5c0 msgr2=0x7f5488099fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:18.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.595 7f548d170700 1 --2- 172.21.2.101:0/2988903861 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f548819e5c0 0x7f5488099fb0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 rx=0x7f5470006f00 tx=0x7f547000b940).stop 2021-02-10T11:40:18.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.596 7f548d170700 1 -- 172.21.2.101:0/2988903861 shutdown_connections 2021-02-10T11:40:18.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.596 7f548d170700 1 --2- 172.21.2.101:0/2988903861 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f546c018730 0x7f546c01ac10 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:18.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.596 7f548d170700 1 --2- 172.21.2.101:0/2988903861 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f548809a5d0 0x7f548809a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:18.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.596 7f548d170700 1 --2- 172.21.2.101:0/2988903861 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54880982a0 0x7f54880986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:18.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.596 7f548d170700 1 --2- 172.21.2.101:0/2988903861 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f548819e5c0 0x7f5488099fb0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:18.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.596 7f548d170700 1 -- 172.21.2.101:0/2988903861 >> 172.21.2.101:0/2988903861 conn(0x7f54881b3560 msgr2=0x7f54881b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:18.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.596 7f548d170700 1 -- 172.21.2.101:0/2988903861 shutdown_connections 2021-02-10T11:40:18.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:18.596 7f548d170700 1 -- 172.21.2.101:0/2988903861 wait complete. 2021-02-10T11:40:18.607 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs volume ls 2021-02-10T11:40:19.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.014 7f6ac5ff6700 1 Processor -- start 2021-02-10T11:40:19.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.014 7f6ac5ff6700 1 -- start start 2021-02-10T11:40:19.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.014 7f6ac5ff6700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6ac00d1870 0x7f6ac01b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:19.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.014 7f6ac5ff6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ac01b0a10 0x7f6ac00cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:19.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.014 7f6ac5ff6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ac00f0b10 con 0x7f6ac00d1c70 2021-02-10T11:40:19.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.014 7f6ac5ff6700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ac00e5ed0 con 0x7f6ac00d1870 2021-02-10T11:40:19.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.014 7f6ac5ff6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac00f0650 con 0x7f6ac01b0a10 2021-02-10T11:40:19.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.015 7f6abffff700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f6ac00d1c70 0x7f6ac00cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:50252/0 (socket says 172.21.2.101:50252) 2021-02-10T11:40:19.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.015 7f6abf7fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6ac00d1870 0x7f6ac01b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:19.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.015 7f6abffff700 1 -- 172.21.2.101:0/1661660904 learned_addr learned my addr 172.21.2.101:0/1661660904 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:19.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.015 7f6abeffd700 1 --2- 172.21.2.101:0/1661660904 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ac01b0a10 0x7f6ac00cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:19.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.016 7f6abeffd700 1 -- 172.21.2.101:0/1661660904 >> v2:172.21.2.108:3300/0 conn(0x7f6ac00d1870 msgr2=0x7f6ac01b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:19.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.016 7f6abeffd700 1 --2- 172.21.2.101:0/1661660904 >> v2:172.21.2.108:3300/0 conn(0x7f6ac00d1870 0x7f6ac01b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:19.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.016 7f6abeffd700 1 -- 172.21.2.101:0/1661660904 >> v1:172.21.2.108:6789/0 conn(0x7f6ac00d1c70 legacy=0x7f6ac00cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:19.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.016 7f6abeffd700 1 -- 172.21.2.101:0/1661660904 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac00f0870 con 0x7f6ac01b0a10 2021-02-10T11:40:19.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.016 7f6abf7fe700 1 --2- 172.21.2.101:0/1661660904 >> v2:172.21.2.108:3300/0 conn(0x7f6ac00d1870 0x7f6ac01b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:40:19.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.016 7f6abeffd700 1 --2- 172.21.2.101:0/1661660904 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ac01b0a10 0x7f6ac00cf330 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f6aa8006650 tx=0x7f6aa8009a80).ready entity=mon.2 client_cookie=cf9f5cb610855411 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:19.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.016 7f6abe7fc700 1 -- 172.21.2.101:0/1661660904 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6aa800c0c0 con 0x7f6ac01b0a10 2021-02-10T11:40:19.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.016 7f6abe7fc700 1 -- 172.21.2.101:0/1661660904 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa800c340 con 0x7f6ac01b0a10 2021-02-10T11:40:19.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.016 7f6abe7fc700 1 -- 172.21.2.101:0/1661660904 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6aa800c790 con 0x7f6ac01b0a10 2021-02-10T11:40:19.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.017 7f6ac5ff6700 1 -- 172.21.2.101:0/1661660904 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ac01b0a10 msgr2=0x7f6ac00cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:19.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.017 7f6ac5ff6700 1 --2- 172.21.2.101:0/1661660904 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ac01b0a10 0x7f6ac00cf330 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f6aa8006650 tx=0x7f6aa8009a80).stop 2021-02-10T11:40:19.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.017 7f6ac5ff6700 1 -- 172.21.2.101:0/1661660904 shutdown_connections 2021-02-10T11:40:19.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.017 7f6ac5ff6700 1 --2- 172.21.2.101:0/1661660904 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ac01b0a10 0x7f6ac00cf330 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:19.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.017 7f6ac5ff6700 1 --2- 172.21.2.101:0/1661660904 >> v2:172.21.2.108:3300/0 conn(0x7f6ac00d1870 0x7f6ac01b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:19.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.017 7f6ac5ff6700 1 -- 172.21.2.101:0/1661660904 >> 172.21.2.101:0/1661660904 conn(0x7f6ac00ce450 msgr2=0x7f6ac00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:19.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.017 7f6ac5ff6700 1 -- 172.21.2.101:0/1661660904 shutdown_connections 2021-02-10T11:40:19.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.017 7f6ac5ff6700 1 -- 172.21.2.101:0/1661660904 wait complete. 2021-02-10T11:40:19.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.031 7f6ac5ff6700 1 Processor -- start 2021-02-10T11:40:19.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.032 7f6ac5ff6700 1 -- start start 2021-02-10T11:40:19.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.032 7f6ac5ff6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6ac00b7510 0x7f6ac00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:19.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.032 7f6ac5ff6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ac009d530 0x7f6ac009d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:19.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.032 7f6ac5ff6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ac009df50 0x7f6ac009eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:19.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.032 7f6ac5ff6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac00b8140 con 0x7f6ac009df50 2021-02-10T11:40:19.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.032 7f6ac5ff6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac009e750 con 0x7f6ac00b7510 2021-02-10T11:40:19.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.032 7f6ac5ff6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac009e350 con 0x7f6ac009d530 2021-02-10T11:40:19.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.032 7f6abffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ac009df50 0x7f6ac009eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:19.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.032 7f6abffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ac009df50 0x7f6ac009eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56354/0 (socket says 172.21.2.101:56354) 2021-02-10T11:40:19.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.032 7f6abffff700 1 -- 172.21.2.101:0/1028632235 learned_addr learned my addr 172.21.2.101:0/1028632235 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:19.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.032 7f6abeffd700 1 --2- 172.21.2.101:0/1028632235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ac009d530 0x7f6ac009d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:19.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.032 7f6abf7fe700 1 --2- 172.21.2.101:0/1028632235 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6ac00b7510 0x7f6ac00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:19.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.033 7f6abffff700 1 -- 172.21.2.101:0/1028632235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ac009d530 msgr2=0x7f6ac009d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:19.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.033 7f6abffff700 1 --2- 172.21.2.101:0/1028632235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ac009d530 0x7f6ac009d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:19.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.033 7f6abffff700 1 -- 172.21.2.101:0/1028632235 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6ac00b7510 msgr2=0x7f6ac00b7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:19.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.033 7f6abffff700 1 --2- 172.21.2.101:0/1028632235 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6ac00b7510 0x7f6ac00b7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:19.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.033 7f6abffff700 1 -- 172.21.2.101:0/1028632235 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac009f2c0 con 0x7f6ac009df50 2021-02-10T11:40:19.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.033 7f6abffff700 1 --2- 172.21.2.101:0/1028632235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ac009df50 0x7f6ac009eca0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 rx=0x7f6ab8007f90 tx=0x7f6ab800ac70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:19.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.033 7f6ab7fff700 1 -- 172.21.2.101:0/1028632235 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ab80070d0 con 0x7f6ac009df50 2021-02-10T11:40:19.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.033 7f6ab7fff700 1 -- 172.21.2.101:0/1028632235 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab8007390 con 0x7f6ac009df50 2021-02-10T11:40:19.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.033 7f6ab7fff700 1 -- 172.21.2.101:0/1028632235 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ab80048b0 con 0x7f6ac009df50 2021-02-10T11:40:19.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.033 7f6ac5ff6700 1 -- 172.21.2.101:0/1028632235 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac009f2c0 con 0x7f6ac009df50 2021-02-10T11:40:19.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.033 7f6ac5ff6700 1 -- 172.21.2.101:0/1028632235 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac009f850 con 0x7f6ac009df50 2021-02-10T11:40:19.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.034 7f6ab7fff700 1 -- 172.21.2.101:0/1028632235 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6ab8007570 con 0x7f6ac009df50 2021-02-10T11:40:19.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.034 7f6ab7fff700 1 --2- 172.21.2.101:0/1028632235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6aa40186c0 0x7f6aa401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:19.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.034 7f6ab7fff700 1 -- 172.21.2.101:0/1028632235 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6ab8032900 con 0x7f6ac009df50 2021-02-10T11:40:19.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.034 7f6abf7fe700 1 --2- 172.21.2.101:0/1028632235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6aa40186c0 0x7f6aa401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:19.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.034 7f6abf7fe700 1 --2- 172.21.2.101:0/1028632235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6aa40186c0 0x7f6aa401abc0 crc :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:19.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.064 7f6ac5ff6700 1 -- 172.21.2.101:0/1028632235 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac00ebc50 con 0x7f6ac009df50 2021-02-10T11:40:19.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.067 7f6ab7fff700 1 -- 172.21.2.101:0/1028632235 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6ab802cf50 con 0x7f6ac009df50 2021-02-10T11:40:19.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.200 7f6ac5ff6700 1 -- 172.21.2.101:0/1028632235 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs volume ls", "target": ["mgr", ""]}) v1 -- 0x7f6ac009f2c0 con 0x7f6aa40186c0 2021-02-10T11:40:19.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.201 7f6ab7fff700 1 -- 172.21.2.101:0/1028632235 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7f6ab0006a30 con 0x7f6aa40186c0 2021-02-10T11:40:19.204 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:40:19.204 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:19.204 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "cephfs" 2021-02-10T11:40:19.205 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:19.205 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:40:19.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.203 7f6ac5ff6700 1 -- 172.21.2.101:0/1028632235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6aa40186c0 msgr2=0x7f6aa401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:19.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.203 7f6ac5ff6700 1 --2- 172.21.2.101:0/1028632235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6aa40186c0 0x7f6aa401abc0 crc :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:19.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.203 7f6ac5ff6700 1 -- 172.21.2.101:0/1028632235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ac009df50 msgr2=0x7f6ac009eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:19.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.203 7f6ac5ff6700 1 --2- 172.21.2.101:0/1028632235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ac009df50 0x7f6ac009eca0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 rx=0x7f6ab8007f90 tx=0x7f6ab800ac70).stop 2021-02-10T11:40:19.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.203 7f6ac5ff6700 1 -- 172.21.2.101:0/1028632235 shutdown_connections 2021-02-10T11:40:19.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.203 7f6ac5ff6700 1 --2- 172.21.2.101:0/1028632235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6aa40186c0 0x7f6aa401abc0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:19.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.203 7f6ac5ff6700 1 --2- 172.21.2.101:0/1028632235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ac009d530 0x7f6ac009d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:19.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.203 7f6ac5ff6700 1 --2- 172.21.2.101:0/1028632235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ac009df50 0x7f6ac009eca0 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:19.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.203 7f6ac5ff6700 1 --2- 172.21.2.101:0/1028632235 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6ac00b7510 0x7f6ac00b7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:19.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.203 7f6ac5ff6700 1 -- 172.21.2.101:0/1028632235 >> 172.21.2.101:0/1028632235 conn(0x7f6ac00b83b0 msgr2=0x7f6ac00a6850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:19.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.203 7f6ac5ff6700 1 -- 172.21.2.101:0/1028632235 shutdown_connections 2021-02-10T11:40:19.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.203 7f6ac5ff6700 1 -- 172.21.2.101:0/1028632235 wait complete. 2021-02-10T11:40:19.214 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:40:19.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.626 7f08e9699700 1 Processor -- start 2021-02-10T11:40:19.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.626 7f08e9699700 1 -- start start 2021-02-10T11:40:19.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.627 7f08e9699700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08e40d1c70 0x7f08e40cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:19.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.627 7f08e9699700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f08e40cf580 0x7f08e40cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:19.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.627 7f08e9699700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e40fe1c0 con 0x7f08e40d1870 2021-02-10T11:40:19.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.627 7f08e9699700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e41b0cb0 con 0x7f08e40cf580 2021-02-10T11:40:19.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.627 7f08e9699700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e4109a60 con 0x7f08e40d1c70 2021-02-10T11:40:19.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.627 7f08e3ac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f08e40cf580 0x7f08e40cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:19.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.627 7f08e3ac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f08e40cf580 0x7f08e40cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56362/0 (socket says 172.21.2.101:56362) 2021-02-10T11:40:19.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.627 7f08e3ac0700 1 -- 172.21.2.101:0/156558057 learned_addr learned my addr 172.21.2.101:0/156558057 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:19.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.627 7f08e3ac0700 1 -- 172.21.2.101:0/156558057 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08e40d1c70 msgr2=0x7f08e40cef60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:40:19.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.627 7f08e3ac0700 1 --2- 172.21.2.101:0/156558057 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08e40d1c70 0x7f08e40cef60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:19.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.627 7f08e3ac0700 1 -- 172.21.2.101:0/156558057 >> v1:172.21.2.108:6789/0 conn(0x7f08e40d1870 legacy=0x7f08e41b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:40:19.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.627 7f08e3ac0700 1 -- 172.21.2.101:0/156558057 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e4088070 con 0x7f08e40cf580 2021-02-10T11:40:19.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.627 7f08e3ac0700 1 --2- 172.21.2.101:0/156558057 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08e40cf580 0x7f08e40cf980 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 rx=0x7f08dc006f00 tx=0x7f08dc009e90).ready entity=mon.0 client_cookie=ba74aaea99327561 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:19.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.628 7f08e22bd700 1 -- 172.21.2.101:0/156558057 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f08dc00b300 con 0x7f08e40cf580 2021-02-10T11:40:19.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.628 7f08e22bd700 1 -- 172.21.2.101:0/156558057 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc00b640 con 0x7f08e40cf580 2021-02-10T11:40:19.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.628 7f08e22bd700 1 -- 172.21.2.101:0/156558057 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f08dc00bb60 con 0x7f08e40cf580 2021-02-10T11:40:19.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.628 7f08e9699700 1 -- 172.21.2.101:0/156558057 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08e40cf580 msgr2=0x7f08e40cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:19.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.628 7f08e9699700 1 --2- 172.21.2.101:0/156558057 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08e40cf580 0x7f08e40cf980 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 rx=0x7f08dc006f00 tx=0x7f08dc009e90).stop 2021-02-10T11:40:19.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.628 7f08e9699700 1 -- 172.21.2.101:0/156558057 shutdown_connections 2021-02-10T11:40:19.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.628 7f08e9699700 1 --2- 172.21.2.101:0/156558057 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08e40d1c70 0x7f08e40cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:19.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.628 7f08e9699700 1 --2- 172.21.2.101:0/156558057 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08e40cf580 0x7f08e40cf980 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:19.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.628 7f08e9699700 1 -- 172.21.2.101:0/156558057 >> 172.21.2.101:0/156558057 conn(0x7f08e40ce420 msgr2=0x7f08e40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:19.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.628 7f08e9699700 1 -- 172.21.2.101:0/156558057 shutdown_connections 2021-02-10T11:40:19.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.628 7f08e9699700 1 -- 172.21.2.101:0/156558057 wait complete. 2021-02-10T11:40:19.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.643 7f08e9699700 1 Processor -- start 2021-02-10T11:40:19.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.643 7f08e9699700 1 -- start start 2021-02-10T11:40:19.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.644 7f08e9699700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08e40c2370 0x7f08e418f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:19.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.644 7f08e9699700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08e4084cb0 0x7f08e41a7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:19.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.644 7f08e9699700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f08e41a7a90 0x7f08e4085700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:19.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.644 7f08e9699700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e40be840 con 0x7f08e40c2370 2021-02-10T11:40:19.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.644 7f08e9699700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e40c29a0 con 0x7f08e41a7a90 2021-02-10T11:40:19.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.644 7f08e9699700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e41a7e90 con 0x7f08e4084cb0 2021-02-10T11:40:19.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.644 7f08e32bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08e40c2370 0x7f08e418f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:19.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.644 7f08e32bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08e40c2370 0x7f08e418f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56366/0 (socket says 172.21.2.101:56366) 2021-02-10T11:40:19.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.644 7f08e32bf700 1 -- 172.21.2.101:0/4272933436 learned_addr learned my addr 172.21.2.101:0/4272933436 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:19.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.644 7f08e32bf700 1 -- 172.21.2.101:0/4272933436 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08e4084cb0 msgr2=0x7f08e41a7470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:19.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.644 7f08e32bf700 1 --2- 172.21.2.101:0/4272933436 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08e4084cb0 0x7f08e41a7470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:19.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.644 7f08e32bf700 1 -- 172.21.2.101:0/4272933436 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f08e41a7a90 msgr2=0x7f08e4085700 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:19.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.644 7f08e32bf700 1 --2- 172.21.2.101:0/4272933436 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f08e41a7a90 0x7f08e4085700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:19.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.644 7f08e32bf700 1 -- 172.21.2.101:0/4272933436 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e41a80b0 con 0x7f08e40c2370 2021-02-10T11:40:19.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.644 7f08e32bf700 1 --2- 172.21.2.101:0/4272933436 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08e40c2370 0x7f08e418f870 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 rx=0x7f08d400b5b0 tx=0x7f08d400b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:19.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.645 7f08dbfff700 1 -- 172.21.2.101:0/4272933436 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f08d400bd40 con 0x7f08e40c2370 2021-02-10T11:40:19.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.645 7f08dbfff700 1 -- 172.21.2.101:0/4272933436 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d4000e20 con 0x7f08e40c2370 2021-02-10T11:40:19.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.645 7f08dbfff700 1 -- 172.21.2.101:0/4272933436 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f08d40054a0 con 0x7f08e40c2370 2021-02-10T11:40:19.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.645 7f08e9699700 1 -- 172.21.2.101:0/4272933436 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e41a80b0 con 0x7f08e40c2370 2021-02-10T11:40:19.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.645 7f08e9699700 1 -- 172.21.2.101:0/4272933436 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e4086010 con 0x7f08e40c2370 2021-02-10T11:40:19.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.645 7f08dbfff700 1 -- 172.21.2.101:0/4272933436 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f08d4006030 con 0x7f08e40c2370 2021-02-10T11:40:19.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.645 7f08dbfff700 1 --2- 172.21.2.101:0/4272933436 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f08c80186c0 0x7f08c801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:19.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.645 7f08dbfff700 1 -- 172.21.2.101:0/4272933436 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f08d40323f0 con 0x7f08e40c2370 2021-02-10T11:40:19.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.645 7f08e2abe700 1 --2- 172.21.2.101:0/4272933436 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f08c80186c0 0x7f08c801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:19.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.646 7f08e2abe700 1 --2- 172.21.2.101:0/4272933436 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f08c80186c0 0x7f08c801abc0 crc :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:19.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.677 7f08e9699700 1 -- 172.21.2.101:0/4272933436 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e4082a70 con 0x7f08e40c2370 2021-02-10T11:40:19.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.680 7f08dbfff700 1 -- 172.21.2.101:0/4272933436 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f08d4014810 con 0x7f08e40c2370 2021-02-10T11:40:19.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.828 7f08e9699700 1 -- 172.21.2.101:0/4272933436 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f08e4082a70 con 0x7f08e40c2370 2021-02-10T11:40:19.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.829 7f08dbfff700 1 -- 172.21.2.101:0/4272933436 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f08d4014810 con 0x7f08e40c2370 2021-02-10T11:40:19.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.830 7f08e9699700 1 -- 172.21.2.101:0/4272933436 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f08c80186c0 msgr2=0x7f08c801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:19.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.830 7f08e9699700 1 --2- 172.21.2.101:0/4272933436 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f08c80186c0 0x7f08c801abc0 crc :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:19.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.830 7f08e9699700 1 -- 172.21.2.101:0/4272933436 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08e40c2370 msgr2=0x7f08e418f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:19.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.830 7f08e9699700 1 --2- 172.21.2.101:0/4272933436 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08e40c2370 0x7f08e418f870 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 rx=0x7f08d400b5b0 tx=0x7f08d400b950).stop 2021-02-10T11:40:19.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.831 7f08e9699700 1 -- 172.21.2.101:0/4272933436 shutdown_connections 2021-02-10T11:40:19.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.831 7f08e9699700 1 --2- 172.21.2.101:0/4272933436 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f08c80186c0 0x7f08c801abc0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:19.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.831 7f08e9699700 1 --2- 172.21.2.101:0/4272933436 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08e4084cb0 0x7f08e41a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:19.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.831 7f08e9699700 1 --2- 172.21.2.101:0/4272933436 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f08e41a7a90 0x7f08e4085700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:19.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.831 7f08e9699700 1 --2- 172.21.2.101:0/4272933436 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08e40c2370 0x7f08e418f870 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:19.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.831 7f08e9699700 1 -- 172.21.2.101:0/4272933436 >> 172.21.2.101:0/4272933436 conn(0x7f08e40b5dc0 msgr2=0x7f08e40c2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:19.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.831 7f08e9699700 1 -- 172.21.2.101:0/4272933436 shutdown_connections 2021-02-10T11:40:19.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:19.831 7f08e9699700 1 -- 172.21.2.101:0/4272933436 wait complete. 2021-02-10T11:40:19.842 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000580387 2021-02-10T11:40:20.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.254 7f7c8a52f700 1 Processor -- start 2021-02-10T11:40:20.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.254 7f7c8a52f700 1 -- start start 2021-02-10T11:40:20.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.255 7f7c8a52f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c840d1c70 0x7f7c840cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:20.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.255 7f7c8a52f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7c840cf580 0x7f7c840cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:20.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.255 7f7c8a52f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c840fe1c0 con 0x7f7c840d1870 2021-02-10T11:40:20.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.255 7f7c8a52f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c841b0cb0 con 0x7f7c840cf580 2021-02-10T11:40:20.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.255 7f7c8a52f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c84109a60 con 0x7f7c840d1c70 2021-02-10T11:40:20.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.255 7f7c83fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f7c840d1870 0x7f7c841b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56866/0 (socket says 172.21.2.101:56866) 2021-02-10T11:40:20.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.255 7f7c83fff700 1 -- 172.21.2.101:0/1397008720 learned_addr learned my addr 172.21.2.101:0/1397008720 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:20.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.255 7f7c82ffd700 1 -- 172.21.2.101:0/1397008720 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2586236718 0 0) 0x7f7c74000ce0 con 0x7f7c840d1870 2021-02-10T11:40:20.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.255 7f7c82ffd700 1 -- 172.21.2.101:0/1397008720 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c6c002fe0 con 0x7f7c840d1870 2021-02-10T11:40:20.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.255 7f7c837fe700 1 --2- 172.21.2.101:0/1397008720 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c840d1c70 0x7f7c840cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:20.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.255 7f7c88b4c700 1 --2- 172.21.2.101:0/1397008720 >> v2:172.21.2.108:3300/0 conn(0x7f7c840cf580 0x7f7c840cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:20.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.255 7f7c82ffd700 1 -- 172.21.2.101:0/1397008720 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1068114021 0 0) 0x7f7c740010e0 con 0x7f7c840d1870 2021-02-10T11:40:20.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.255 7f7c82ffd700 1 -- 172.21.2.101:0/1397008720 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c840d1c70 msgr2=0x7f7c840cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:20.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.255 7f7c82ffd700 1 --2- 172.21.2.101:0/1397008720 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c840d1c70 0x7f7c840cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:20.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.255 7f7c82ffd700 1 -- 172.21.2.101:0/1397008720 >> v2:172.21.2.108:3300/0 conn(0x7f7c840cf580 msgr2=0x7f7c840cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:20.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.255 7f7c82ffd700 1 --2- 172.21.2.101:0/1397008720 >> v2:172.21.2.108:3300/0 conn(0x7f7c840cf580 0x7f7c840cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:20.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.255 7f7c82ffd700 1 -- 172.21.2.101:0/1397008720 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c84088070 con 0x7f7c840d1870 2021-02-10T11:40:20.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.255 7f7c82ffd700 1 -- 172.21.2.101:0/1397008720 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f7c74001660 con 0x7f7c840d1870 2021-02-10T11:40:20.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.256 7f7c82ffd700 1 -- 172.21.2.101:0/1397008720 >> v1:172.21.2.101:6789/0 conn(0x7f7c840d1870 legacy=0x7f7c841b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:20.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.256 7f7c82ffd700 1 --2- 172.21.2.101:0/1397008720 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c6c003ac0 0x7f7c6c005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:20.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.256 7f7c82ffd700 1 --2- 172.21.2.101:0/1397008720 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c6c006790 0x7f7c6c008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:20.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.256 7f7c82ffd700 1 --2- 172.21.2.101:0/1397008720 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c6c009590 0x7f7c6c00ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:20.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.256 7f7c82ffd700 1 -- 172.21.2.101:0/1397008720 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c0502e0 con 0x7f7c6c003ac0 2021-02-10T11:40:20.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.256 7f7c82ffd700 1 -- 172.21.2.101:0/1397008720 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c0504c0 con 0x7f7c6c006790 2021-02-10T11:40:20.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.256 7f7c82ffd700 1 -- 172.21.2.101:0/1397008720 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c0506e0 con 0x7f7c6c009590 2021-02-10T11:40:20.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.256 7f7c83fff700 1 --2- 172.21.2.101:0/1397008720 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c6c003ac0 0x7f7c6c005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:20.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.256 7f7c88b4c700 1 --2- 172.21.2.101:0/1397008720 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c6c009590 0x7f7c6c00ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:20.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.256 7f7c837fe700 1 --2- 172.21.2.101:0/1397008720 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c6c006790 0x7f7c6c008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:20.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.256 7f7c88b4c700 1 -- 172.21.2.101:0/1397008720 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c6c006790 msgr2=0x7f7c6c008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:20.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.256 7f7c88b4c700 1 --2- 172.21.2.101:0/1397008720 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c6c006790 0x7f7c6c008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:20.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.256 7f7c88b4c700 1 -- 172.21.2.101:0/1397008720 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c6c003ac0 msgr2=0x7f7c6c005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:20.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.256 7f7c88b4c700 1 --2- 172.21.2.101:0/1397008720 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c6c003ac0 0x7f7c6c005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:20.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.256 7f7c88b4c700 1 -- 172.21.2.101:0/1397008720 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c050980 con 0x7f7c6c009590 2021-02-10T11:40:20.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.256 7f7c83fff700 1 --2- 172.21.2.101:0/1397008720 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c6c003ac0 0x7f7c6c005ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:40:20.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.257 7f7c88b4c700 1 --2- 172.21.2.101:0/1397008720 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c6c009590 0x7f7c6c00ba80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f7c7c0068d0 tx=0x7f7c7c00b050).ready entity=mon.2 client_cookie=b54cf782caeadf6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:20.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.257 7f7c82ffd700 1 -- 172.21.2.101:0/1397008720 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7c7c00b650 con 0x7f7c6c009590 2021-02-10T11:40:20.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.257 7f7c82ffd700 1 -- 172.21.2.101:0/1397008720 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7c006c60 con 0x7f7c6c009590 2021-02-10T11:40:20.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.257 7f7c82ffd700 1 -- 172.21.2.101:0/1397008720 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7c7c00bad0 con 0x7f7c6c009590 2021-02-10T11:40:20.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.257 7f7c8a52f700 1 -- 172.21.2.101:0/1397008720 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c6c009590 msgr2=0x7f7c6c00ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:20.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.257 7f7c8a52f700 1 --2- 172.21.2.101:0/1397008720 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c6c009590 0x7f7c6c00ba80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f7c7c0068d0 tx=0x7f7c7c00b050).stop 2021-02-10T11:40:20.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.257 7f7c88b4c700 1 -- 172.21.2.101:0/1397008720 reap_dead start 2021-02-10T11:40:20.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.258 7f7c8a52f700 1 -- 172.21.2.101:0/1397008720 shutdown_connections 2021-02-10T11:40:20.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.258 7f7c8a52f700 1 -- 172.21.2.101:0/1397008720 >> 172.21.2.101:0/1397008720 conn(0x7f7c840ce420 msgr2=0x7f7c840d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:20.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.258 7f7c8a52f700 1 -- 172.21.2.101:0/1397008720 shutdown_connections 2021-02-10T11:40:20.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.258 7f7c8a52f700 1 -- 172.21.2.101:0/1397008720 wait complete. 2021-02-10T11:40:20.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.272 7f7c8a52f700 1 Processor -- start 2021-02-10T11:40:20.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.272 7f7c8a52f700 1 -- start start 2021-02-10T11:40:20.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.273 7f7c8a52f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c84085bc0 0x7f7c840863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:20.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.273 7f7c8a52f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c840954c0 0x7f7c84096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:20.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.273 7f7c8a52f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c840958c0 0x7f7c84096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:20.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.273 7f7c8a52f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c84095d80 con 0x7f7c840954c0 2021-02-10T11:40:20.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.273 7f7c8a52f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c840bfa60 con 0x7f7c840958c0 2021-02-10T11:40:20.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.273 7f7c8a52f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c84085fc0 con 0x7f7c84085bc0 2021-02-10T11:40:20.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.273 7f7c837fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c840954c0 0x7f7c84096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:20.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.273 7f7c837fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c840954c0 0x7f7c84096300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56386/0 (socket says 172.21.2.101:56386) 2021-02-10T11:40:20.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.273 7f7c837fe700 1 -- 172.21.2.101:0/3295680938 learned_addr learned my addr 172.21.2.101:0/3295680938 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:20.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.273 7f7c837fe700 1 -- 172.21.2.101:0/3295680938 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c84085bc0 msgr2=0x7f7c840863e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:20.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.273 7f7c837fe700 1 --2- 172.21.2.101:0/3295680938 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c84085bc0 0x7f7c840863e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:20.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.273 7f7c837fe700 1 -- 172.21.2.101:0/3295680938 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c840958c0 msgr2=0x7f7c84096920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:20.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.273 7f7c837fe700 1 --2- 172.21.2.101:0/3295680938 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c840958c0 0x7f7c84096920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:20.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.273 7f7c837fe700 1 -- 172.21.2.101:0/3295680938 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c84086c10 con 0x7f7c840954c0 2021-02-10T11:40:20.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.273 7f7c837fe700 1 --2- 172.21.2.101:0/3295680938 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c840954c0 0x7f7c84096300 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 rx=0x7f7c7800b5e0 tx=0x7f7c7800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:20.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.274 7f7c80ff9700 1 -- 172.21.2.101:0/3295680938 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7c780009b0 con 0x7f7c840954c0 2021-02-10T11:40:20.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.274 7f7c80ff9700 1 -- 172.21.2.101:0/3295680938 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7800bd70 con 0x7f7c840954c0 2021-02-10T11:40:20.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.274 7f7c80ff9700 1 -- 172.21.2.101:0/3295680938 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7c780022b0 con 0x7f7c840954c0 2021-02-10T11:40:20.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.274 7f7c8a52f700 1 -- 172.21.2.101:0/3295680938 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c84086c10 con 0x7f7c840954c0 2021-02-10T11:40:20.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.274 7f7c8a52f700 1 -- 172.21.2.101:0/3295680938 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c84097230 con 0x7f7c840954c0 2021-02-10T11:40:20.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.274 7f7c80ff9700 1 -- 172.21.2.101:0/3295680938 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7c78000c90 con 0x7f7c840954c0 2021-02-10T11:40:20.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.274 7f7c80ff9700 1 --2- 172.21.2.101:0/3295680938 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c640186c0 0x7f7c6401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:20.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.274 7f7c80ff9700 1 -- 172.21.2.101:0/3295680938 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7c78032400 con 0x7f7c840954c0 2021-02-10T11:40:20.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.275 7f7c83fff700 1 --2- 172.21.2.101:0/3295680938 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c640186c0 0x7f7c6401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:20.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.275 7f7c83fff700 1 --2- 172.21.2.101:0/3295680938 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c640186c0 0x7f7c6401abc0 crc :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:20.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.304 7f7c8a52f700 1 -- 172.21.2.101:0/3295680938 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c840fe1c0 con 0x7f7c840954c0 2021-02-10T11:40:20.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.307 7f7c80ff9700 1 -- 172.21.2.101:0/3295680938 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7c78000940 con 0x7f7c840954c0 2021-02-10T11:40:20.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.436 7f7c8a52f700 1 -- 172.21.2.101:0/3295680938 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000580387", "target": ["mgr", ""]}) v1 -- 0x7f7c84097230 con 0x7f7c640186c0 2021-02-10T11:40:20.443 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:40:20.441 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:40:20.444 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:40:20.441 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:40:20.444 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:40:20.441 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:40:20.444 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:40:20.441 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:40:20.444 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:40:20.441 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:40:20.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.471 7f7c80ff9700 1 -- 172.21.2.101:0/3295680938 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f7c74001340 con 0x7f7c640186c0 2021-02-10T11:40:20.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.501 7f7c8a52f700 1 -- 172.21.2.101:0/3295680938 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c640186c0 msgr2=0x7f7c6401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:20.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.501 7f7c8a52f700 1 --2- 172.21.2.101:0/3295680938 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c640186c0 0x7f7c6401abc0 crc :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:20.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.501 7f7c8a52f700 1 -- 172.21.2.101:0/3295680938 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c840954c0 msgr2=0x7f7c84096300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:20.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.501 7f7c8a52f700 1 --2- 172.21.2.101:0/3295680938 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c840954c0 0x7f7c84096300 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 rx=0x7f7c7800b5e0 tx=0x7f7c7800b980).stop 2021-02-10T11:40:20.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.501 7f7c8a52f700 1 -- 172.21.2.101:0/3295680938 shutdown_connections 2021-02-10T11:40:20.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.501 7f7c8a52f700 1 --2- 172.21.2.101:0/3295680938 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c840958c0 0x7f7c84096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:20.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.501 7f7c8a52f700 1 --2- 172.21.2.101:0/3295680938 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c840954c0 0x7f7c84096300 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:20.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.501 7f7c8a52f700 1 --2- 172.21.2.101:0/3295680938 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c640186c0 0x7f7c6401abc0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:20.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.501 7f7c8a52f700 1 --2- 172.21.2.101:0/3295680938 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c84085bc0 0x7f7c840863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:20.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.501 7f7c8a52f700 1 -- 172.21.2.101:0/3295680938 >> 172.21.2.101:0/3295680938 conn(0x7f7c840bef00 msgr2=0x7f7c8409d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:20.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.501 7f7c8a52f700 1 -- 172.21.2.101:0/3295680938 shutdown_connections 2021-02-10T11:40:20.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.501 7f7c8a52f700 1 -- 172.21.2.101:0/3295680938 wait complete. 2021-02-10T11:40:20.513 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000580387 2021-02-10T11:40:20.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.924 7f3ce6391700 1 Processor -- start 2021-02-10T11:40:20.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.924 7f3ce6391700 1 -- start start 2021-02-10T11:40:20.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.925 7f3ce6391700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ce00d1870 0x7f3ce01b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:20.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.925 7f3ce6391700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3ce00cff70 0x7f3ce0109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:20.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.925 7f3ce6391700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce0109a60 con 0x7f3ce01b0ae0 2021-02-10T11:40:20.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.925 7f3ce6391700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce0109750 con 0x7f3ce00cff70 2021-02-10T11:40:20.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.925 7f3ce6391700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce0088070 con 0x7f3ce00d1870 2021-02-10T11:40:20.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.925 7f3cdffff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ce00d1870 0x7f3ce01b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:20.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.925 7f3ce49ae700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3ce00cff70 0x7f3ce0109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:20.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.925 7f3cd7fff700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f3ce01b0ae0 0x7f3ce0108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:50314/0 (socket says 172.21.2.101:50314) 2021-02-10T11:40:20.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.925 7f3cd7fff700 1 -- 172.21.2.101:0/2209436802 learned_addr learned my addr 172.21.2.101:0/2209436802 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:20.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.925 7f3ce49ae700 1 -- 172.21.2.101:0/2209436802 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ce00d1870 msgr2=0x7f3ce01b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:20.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.925 7f3ce49ae700 1 --2- 172.21.2.101:0/2209436802 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ce00d1870 0x7f3ce01b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:20.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.925 7f3ce49ae700 1 -- 172.21.2.101:0/2209436802 >> v1:172.21.2.108:6789/0 conn(0x7f3ce01b0ae0 legacy=0x7f3ce0108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:20.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.925 7f3ce49ae700 1 -- 172.21.2.101:0/2209436802 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce0088260 con 0x7f3ce00cff70 2021-02-10T11:40:20.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.925 7f3cdffff700 1 --2- 172.21.2.101:0/2209436802 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ce00d1870 0x7f3ce01b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:40:20.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.926 7f3ce49ae700 1 --2- 172.21.2.101:0/2209436802 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ce00cff70 0x7f3ce0109130 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f3cd8006e50 tx=0x7f3cd8009e70).ready entity=mon.1 client_cookie=c44061e4acb79358 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:20.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.926 7f3cdf7fe700 1 -- 172.21.2.101:0/2209436802 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3cd8006a20 con 0x7f3ce00cff70 2021-02-10T11:40:20.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.926 7f3cdf7fe700 1 -- 172.21.2.101:0/2209436802 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd8006790 con 0x7f3ce00cff70 2021-02-10T11:40:20.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.926 7f3cdf7fe700 1 -- 172.21.2.101:0/2209436802 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3cd800b700 con 0x7f3ce00cff70 2021-02-10T11:40:20.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.926 7f3ce6391700 1 -- 172.21.2.101:0/2209436802 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ce00cff70 msgr2=0x7f3ce0109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:20.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.926 7f3ce6391700 1 --2- 172.21.2.101:0/2209436802 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ce00cff70 0x7f3ce0109130 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f3cd8006e50 tx=0x7f3cd8009e70).stop 2021-02-10T11:40:20.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.926 7f3ce6391700 1 -- 172.21.2.101:0/2209436802 shutdown_connections 2021-02-10T11:40:20.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.926 7f3ce6391700 1 --2- 172.21.2.101:0/2209436802 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ce00cff70 0x7f3ce0109130 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:20.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.926 7f3ce6391700 1 --2- 172.21.2.101:0/2209436802 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ce00d1870 0x7f3ce01b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:20.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.926 7f3ce6391700 1 -- 172.21.2.101:0/2209436802 >> 172.21.2.101:0/2209436802 conn(0x7f3ce00ce420 msgr2=0x7f3ce00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:20.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.926 7f3ce6391700 1 -- 172.21.2.101:0/2209436802 shutdown_connections 2021-02-10T11:40:20.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.926 7f3ce6391700 1 -- 172.21.2.101:0/2209436802 wait complete. 2021-02-10T11:40:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.941 7f3ce6391700 1 Processor -- start 2021-02-10T11:40:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.941 7f3ce6391700 1 -- start start 2021-02-10T11:40:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.941 7f3ce6391700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ce019e3d0 0x7f3ce00a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.941 7f3ce6391700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ce00a6d50 0x7f3ce0083890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.941 7f3ce6391700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ce00a7150 0x7f3ce0083eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.941 7f3ce6391700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce00a1430 con 0x7f3ce019e3d0 2021-02-10T11:40:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.941 7f3ce6391700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce00846b0 con 0x7f3ce00a7150 2021-02-10T11:40:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.941 7f3ce6391700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce00a1ac0 con 0x7f3ce00a6d50 2021-02-10T11:40:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.941 7f3cdffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ce019e3d0 0x7f3ce00a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.941 7f3cdffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ce019e3d0 0x7f3ce00a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56416/0 (socket says 172.21.2.101:56416) 2021-02-10T11:40:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.941 7f3cdffff700 1 -- 172.21.2.101:0/238675428 learned_addr learned my addr 172.21.2.101:0/238675428 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.941 7f3cd7fff700 1 --2- 172.21.2.101:0/238675428 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ce00a6d50 0x7f3ce0083890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:20.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.941 7f3ce49ae700 1 --2- 172.21.2.101:0/238675428 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ce00a7150 0x7f3ce0083eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:20.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.942 7f3cdffff700 1 -- 172.21.2.101:0/238675428 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ce00a6d50 msgr2=0x7f3ce0083890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:20.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.942 7f3cdffff700 1 --2- 172.21.2.101:0/238675428 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ce00a6d50 0x7f3ce0083890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:20.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.942 7f3cdffff700 1 -- 172.21.2.101:0/238675428 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ce00a7150 msgr2=0x7f3ce0083eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:20.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.942 7f3cdffff700 1 --2- 172.21.2.101:0/238675428 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ce00a7150 0x7f3ce0083eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:20.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.942 7f3cdffff700 1 -- 172.21.2.101:0/238675428 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce00a1ce0 con 0x7f3ce019e3d0 2021-02-10T11:40:20.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.942 7f3cdffff700 1 --2- 172.21.2.101:0/238675428 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ce019e3d0 0x7f3ce00a0d70 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 rx=0x7f3cc8006f40 tx=0x7f3cc800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:20.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.942 7f3cdd7fa700 1 -- 172.21.2.101:0/238675428 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3cc800bd80 con 0x7f3ce019e3d0 2021-02-10T11:40:20.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.942 7f3cdd7fa700 1 -- 172.21.2.101:0/238675428 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc8000c90 con 0x7f3ce019e3d0 2021-02-10T11:40:20.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.942 7f3ce6391700 1 -- 172.21.2.101:0/238675428 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce00a1ce0 con 0x7f3ce019e3d0 2021-02-10T11:40:20.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.942 7f3cdd7fa700 1 -- 172.21.2.101:0/238675428 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3cc8002220 con 0x7f3ce019e3d0 2021-02-10T11:40:20.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.942 7f3ce6391700 1 -- 172.21.2.101:0/238675428 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce00a2270 con 0x7f3ce019e3d0 2021-02-10T11:40:20.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.943 7f3cdd7fa700 1 -- 172.21.2.101:0/238675428 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3cc800a000 con 0x7f3ce019e3d0 2021-02-10T11:40:20.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.943 7f3cdd7fa700 1 --2- 172.21.2.101:0/238675428 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cc40186c0 0x7f3cc401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:20.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.943 7f3cdd7fa700 1 -- 172.21.2.101:0/238675428 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3cc80325b0 con 0x7f3ce019e3d0 2021-02-10T11:40:20.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.943 7f3cd7fff700 1 --2- 172.21.2.101:0/238675428 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cc40186c0 0x7f3cc401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:20.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.943 7f3cd7fff700 1 --2- 172.21.2.101:0/238675428 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cc40186c0 0x7f3cc401abc0 crc :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:20.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.974 7f3ce6391700 1 -- 172.21.2.101:0/238675428 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce0084660 con 0x7f3ce019e3d0 2021-02-10T11:40:20.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:20.977 7f3cdd7fa700 1 -- 172.21.2.101:0/238675428 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3cc8030de0 con 0x7f3ce019e3d0 2021-02-10T11:40:21.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:21.104 7f3ce6391700 1 -- 172.21.2.101:0/238675428 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000580387", "target": ["mgr", ""]}) v1 -- 0x7f3ce00a2270 con 0x7f3cc40186c0 2021-02-10T11:40:21.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:21.108 7f3cdd7fa700 1 -- 172.21.2.101:0/238675428 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f3cd0001a00 con 0x7f3cc40186c0 2021-02-10T11:40:21.114 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000580387/d7fe75bb-2ea0-406d-b341-ca4515a8290d 2021-02-10T11:40:21.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:21.113 7f3ce6391700 1 -- 172.21.2.101:0/238675428 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cc40186c0 msgr2=0x7f3cc401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:21.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:21.113 7f3ce6391700 1 --2- 172.21.2.101:0/238675428 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cc40186c0 0x7f3cc401abc0 crc :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:21.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:21.113 7f3ce6391700 1 -- 172.21.2.101:0/238675428 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ce019e3d0 msgr2=0x7f3ce00a0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:21.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:21.113 7f3ce6391700 1 --2- 172.21.2.101:0/238675428 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ce019e3d0 0x7f3ce00a0d70 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 rx=0x7f3cc8006f40 tx=0x7f3cc800b990).stop 2021-02-10T11:40:21.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:21.113 7f3ce6391700 1 -- 172.21.2.101:0/238675428 shutdown_connections 2021-02-10T11:40:21.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:21.113 7f3ce6391700 1 --2- 172.21.2.101:0/238675428 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3cc40186c0 0x7f3cc401abc0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:21.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:21.113 7f3ce6391700 1 --2- 172.21.2.101:0/238675428 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ce00a6d50 0x7f3ce0083890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:21.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:21.113 7f3ce6391700 1 --2- 172.21.2.101:0/238675428 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ce00a7150 0x7f3ce0083eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:21.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:21.113 7f3ce6391700 1 --2- 172.21.2.101:0/238675428 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ce019e3d0 0x7f3ce00a0d70 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:21.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:21.113 7f3ce6391700 1 -- 172.21.2.101:0/238675428 >> 172.21.2.101:0/238675428 conn(0x7f3ce00bc070 msgr2=0x7f3ce00a0510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:21.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:21.113 7f3ce6391700 1 -- 172.21.2.101:0/238675428 shutdown_connections 2021-02-10T11:40:21.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:21.113 7f3ce6391700 1 -- 172.21.2.101:0/238675428 wait complete. 2021-02-10T11:40:21.125 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-02-10T11:40:21.125 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:40:21.125 DEBUG:teuthology.orchestra.run.gibba001:> import stat 2021-02-10T11:40:21.125 DEBUG:teuthology.orchestra.run.gibba001:> import json 2021-02-10T11:40:21.125 DEBUG:teuthology.orchestra.run.gibba001:> import sys 2021-02-10T11:40:21.125 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:40:21.126 DEBUG:teuthology.orchestra.run.gibba001:> try: 2021-02-10T11:40:21.126 DEBUG:teuthology.orchestra.run.gibba001:> s = os.stat("/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvolume_0000000000580387/d7fe75bb-2ea0-406d-b341-ca4515a8290d") 2021-02-10T11:40:21.126 DEBUG:teuthology.orchestra.run.gibba001:> except OSError as e: 2021-02-10T11:40:21.126 DEBUG:teuthology.orchestra.run.gibba001:> sys.exit(e.errno) 2021-02-10T11:40:21.126 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:40:21.126 DEBUG:teuthology.orchestra.run.gibba001:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2021-02-10T11:40:21.126 DEBUG:teuthology.orchestra.run.gibba001:> print(json.dumps( 2021-02-10T11:40:21.126 DEBUG:teuthology.orchestra.run.gibba001:> dict([(a, getattr(s, a)) for a in attrs]), 2021-02-10T11:40:21.126 DEBUG:teuthology.orchestra.run.gibba001:> indent=2)) 2021-02-10T11:40:21.126 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:40:21.533 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:40:21.533 INFO:teuthology.orchestra.run.gibba001.stdout: "st_mode": 16877, 2021-02-10T11:40:21.533 INFO:teuthology.orchestra.run.gibba001.stdout: "st_ino": 1099511627779, 2021-02-10T11:40:21.533 INFO:teuthology.orchestra.run.gibba001.stdout: "st_dev": 43, 2021-02-10T11:40:21.533 INFO:teuthology.orchestra.run.gibba001.stdout: "st_nlink": 2, 2021-02-10T11:40:21.533 INFO:teuthology.orchestra.run.gibba001.stdout: "st_uid": 0, 2021-02-10T11:40:21.533 INFO:teuthology.orchestra.run.gibba001.stdout: "st_gid": 0, 2021-02-10T11:40:21.533 INFO:teuthology.orchestra.run.gibba001.stdout: "st_size": 0, 2021-02-10T11:40:21.534 INFO:teuthology.orchestra.run.gibba001.stdout: "st_atime": 1612957220.4524443, 2021-02-10T11:40:21.534 INFO:teuthology.orchestra.run.gibba001.stdout: "st_mtime": 1612957220.4524443, 2021-02-10T11:40:21.534 INFO:teuthology.orchestra.run.gibba001.stdout: "st_ctime": 1612957220.4620464 2021-02-10T11:40:21.534 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:40:21.699 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000580387 2021-02-10T11:40:22.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.150 7f0f23c4a700 1 Processor -- start 2021-02-10T11:40:22.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.150 7f0f23c4a700 1 -- start start 2021-02-10T11:40:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.151 7f0f23c4a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0f1c0d1870 0x7f0f1c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.151 7f0f23c4a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f1c0fe1c0 con 0x7f0f1c1b0ae0 2021-02-10T11:40:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.151 7f0f23c4a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f1c109a60 con 0x7f0f1c0cff70 2021-02-10T11:40:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.151 7f0f23c4a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f1c088070 con 0x7f0f1c0d1870 2021-02-10T11:40:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.151 7f0f21265700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f0f1c1b0ae0 0x7f0f1c108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56914/0 (socket says 172.21.2.101:56914) 2021-02-10T11:40:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.151 7f0f21265700 1 -- 172.21.2.101:0/3800555163 learned_addr learned my addr 172.21.2.101:0/3800555163 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.151 7f0f20a64700 1 -- 172.21.2.101:0/3800555163 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3971206393 0 0) 0x7f0f180009b0 con 0x7f0f1c1b0ae0 2021-02-10T11:40:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.151 7f0f20a64700 1 -- 172.21.2.101:0/3800555163 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f08002fe0 con 0x7f0f1c1b0ae0 2021-02-10T11:40:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.151 7f0f21a66700 1 --2- 172.21.2.101:0/3800555163 >> v2:172.21.2.108:3300/0 conn(0x7f0f1c0d1870 0x7f0f1c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f20a64700 1 -- 172.21.2.101:0/3800555163 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3979895395 0 0) 0x7f0f18001010 con 0x7f0f1c1b0ae0 2021-02-10T11:40:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f20a64700 1 -- 172.21.2.101:0/3800555163 >> v2:172.21.2.108:3300/0 conn(0x7f0f1c0d1870 msgr2=0x7f0f1c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f20a64700 1 --2- 172.21.2.101:0/3800555163 >> v2:172.21.2.108:3300/0 conn(0x7f0f1c0d1870 0x7f0f1c1b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f20a64700 1 -- 172.21.2.101:0/3800555163 >> v1:172.21.2.108:6789/0 conn(0x7f0f1c0cff70 legacy=0x7f0f1c109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f20a64700 1 -- 172.21.2.101:0/3800555163 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1c088260 con 0x7f0f1c1b0ae0 2021-02-10T11:40:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f20a64700 1 -- 172.21.2.101:0/3800555163 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f0f18001590 con 0x7f0f1c1b0ae0 2021-02-10T11:40:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f20a64700 1 -- 172.21.2.101:0/3800555163 >> v1:172.21.2.101:6789/0 conn(0x7f0f1c1b0ae0 legacy=0x7f0f1c108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f20a64700 1 --2- 172.21.2.101:0/3800555163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f08003aa0 0x7f0f08005ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f20a64700 1 --2- 172.21.2.101:0/3800555163 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f08006770 0x7f0f08008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f20a64700 1 --2- 172.21.2.101:0/3800555163 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f08009590 0x7f0f0800bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f20a64700 1 -- 172.21.2.101:0/3800555163 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0800c300 con 0x7f0f08003aa0 2021-02-10T11:40:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f20a64700 1 -- 172.21.2.101:0/3800555163 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0800c520 con 0x7f0f08006770 2021-02-10T11:40:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f20a64700 1 -- 172.21.2.101:0/3800555163 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0800c740 con 0x7f0f08009590 2021-02-10T11:40:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f21a66700 1 --2- 172.21.2.101:0/3800555163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f08003aa0 0x7f0f08005ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f21265700 1 --2- 172.21.2.101:0/3800555163 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f08006770 0x7f0f08008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f21a66700 1 -- 172.21.2.101:0/3800555163 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f08009590 msgr2=0x7f0f0800bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:22.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f21a66700 1 --2- 172.21.2.101:0/3800555163 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f08009590 0x7f0f0800bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:22.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f21a66700 1 -- 172.21.2.101:0/3800555163 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f08006770 msgr2=0x7f0f08008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:22.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f21a66700 1 --2- 172.21.2.101:0/3800555163 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f08006770 0x7f0f08008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:22.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f21a66700 1 -- 172.21.2.101:0/3800555163 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0800c9e0 con 0x7f0f08003aa0 2021-02-10T11:40:22.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f22267700 1 -- 172.21.2.101:0/3800555163 reap_dead start 2021-02-10T11:40:22.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f21a66700 1 --2- 172.21.2.101:0/3800555163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f08003aa0 0x7f0f08005ed0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 rx=0x7f0f10006360 tx=0x7f0f1004dc30).ready entity=mon.0 client_cookie=512ed55aeeb8972d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:22.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f20a64700 1 -- 172.21.2.101:0/3800555163 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0f1004dd70 con 0x7f0f08003aa0 2021-02-10T11:40:22.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f20a64700 1 -- 172.21.2.101:0/3800555163 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f10006620 con 0x7f0f08003aa0 2021-02-10T11:40:22.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f20a64700 1 -- 172.21.2.101:0/3800555163 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0f10002030 con 0x7f0f08003aa0 2021-02-10T11:40:22.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f23c4a700 1 -- 172.21.2.101:0/3800555163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f08003aa0 msgr2=0x7f0f08005ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:22.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.152 7f0f23c4a700 1 --2- 172.21.2.101:0/3800555163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f08003aa0 0x7f0f08005ed0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 rx=0x7f0f10006360 tx=0x7f0f1004dc30).stop 2021-02-10T11:40:22.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.153 7f0f23c4a700 1 -- 172.21.2.101:0/3800555163 shutdown_connections 2021-02-10T11:40:22.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.153 7f0f23c4a700 1 --2- 172.21.2.101:0/3800555163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f08003aa0 0x7f0f08005ed0 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:22.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.153 7f0f23c4a700 1 -- 172.21.2.101:0/3800555163 >> 172.21.2.101:0/3800555163 conn(0x7f0f1c0ce420 msgr2=0x7f0f1c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:22.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.153 7f0f23c4a700 1 -- 172.21.2.101:0/3800555163 shutdown_connections 2021-02-10T11:40:22.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.153 7f0f23c4a700 1 -- 172.21.2.101:0/3800555163 wait complete. 2021-02-10T11:40:22.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.168 7f0f23c4a700 1 Processor -- start 2021-02-10T11:40:22.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.168 7f0f23c4a700 1 -- start start 2021-02-10T11:40:22.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.168 7f0f23c4a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f1c09bcc0 0x7f0f1c09c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:22.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.168 7f0f23c4a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f1c0a0820 0x7f0f1c0a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:22.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.168 7f0f23c4a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f1c0a1240 0x7f0f1c0858d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:22.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.168 7f0f23c4a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c0a66b0 con 0x7f0f1c09bcc0 2021-02-10T11:40:22.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.168 7f0f23c4a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c1a49b0 con 0x7f0f1c0a0820 2021-02-10T11:40:22.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.168 7f0f23c4a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c1a4b90 con 0x7f0f1c0a1240 2021-02-10T11:40:22.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.168 7f0f21a66700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f1c09bcc0 0x7f0f1c09c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:22.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.168 7f0f21a66700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f1c09bcc0 0x7f0f1c09c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56432/0 (socket says 172.21.2.101:56432) 2021-02-10T11:40:22.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.168 7f0f21a66700 1 -- 172.21.2.101:0/1890080929 learned_addr learned my addr 172.21.2.101:0/1890080929 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:22.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.169 7f0f21a66700 1 -- 172.21.2.101:0/1890080929 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f1c0a1240 msgr2=0x7f0f1c0858d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:22.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.169 7f0f21a66700 1 --2- 172.21.2.101:0/1890080929 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f1c0a1240 0x7f0f1c0858d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:22.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.169 7f0f21a66700 1 -- 172.21.2.101:0/1890080929 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f1c0a0820 msgr2=0x7f0f1c0a0c20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:22.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.169 7f0f21a66700 1 --2- 172.21.2.101:0/1890080929 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f1c0a0820 0x7f0f1c0a0c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:22.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.169 7f0f21a66700 1 -- 172.21.2.101:0/1890080929 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1c0a1640 con 0x7f0f1c09bcc0 2021-02-10T11:40:22.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.169 7f0f21a66700 1 --2- 172.21.2.101:0/1890080929 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f1c09bcc0 0x7f0f1c09c0c0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 rx=0x7f0f10006a00 tx=0x7f0f1004f990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:22.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.169 7f0f0e7fc700 1 -- 172.21.2.101:0/1890080929 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0f100009b0 con 0x7f0f1c09bcc0 2021-02-10T11:40:22.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.169 7f0f0e7fc700 1 -- 172.21.2.101:0/1890080929 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1004fd80 con 0x7f0f1c09bcc0 2021-02-10T11:40:22.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.169 7f0f0e7fc700 1 -- 172.21.2.101:0/1890080929 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0f10005560 con 0x7f0f1c09bcc0 2021-02-10T11:40:22.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.169 7f0f23c4a700 1 -- 172.21.2.101:0/1890080929 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f1c0a1640 con 0x7f0f1c09bcc0 2021-02-10T11:40:22.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.169 7f0f23c4a700 1 -- 172.21.2.101:0/1890080929 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f1c0861e0 con 0x7f0f1c09bcc0 2021-02-10T11:40:22.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.170 7f0f0e7fc700 1 -- 172.21.2.101:0/1890080929 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0f1004e000 con 0x7f0f1c09bcc0 2021-02-10T11:40:22.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.170 7f0f0e7fc700 1 --2- 172.21.2.101:0/1890080929 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0f040186c0 0x7f0f0401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:22.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.170 7f0f0e7fc700 1 -- 172.21.2.101:0/1890080929 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0f100765a0 con 0x7f0f1c09bcc0 2021-02-10T11:40:22.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.170 7f0f21265700 1 --2- 172.21.2.101:0/1890080929 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0f040186c0 0x7f0f0401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:22.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.170 7f0f21265700 1 --2- 172.21.2.101:0/1890080929 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0f040186c0 0x7f0f0401abc0 crc :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:22.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.201 7f0f23c4a700 1 -- 172.21.2.101:0/1890080929 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f1c0fe1c0 con 0x7f0f1c09bcc0 2021-02-10T11:40:22.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.203 7f0f0e7fc700 1 -- 172.21.2.101:0/1890080929 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0f10074d50 con 0x7f0f1c09bcc0 2021-02-10T11:40:22.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.331 7f0f23c4a700 1 -- 172.21.2.101:0/1890080929 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000580387", "target": ["mgr", ""]}) v1 -- 0x7f0f1c0861e0 con 0x7f0f040186c0 2021-02-10T11:40:22.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.343 7f0f0e7fc700 1 -- 172.21.2.101:0/1890080929 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f0f18000fa0 con 0x7f0f040186c0 2021-02-10T11:40:22.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.348 7f0f23c4a700 1 -- 172.21.2.101:0/1890080929 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0f040186c0 msgr2=0x7f0f0401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:22.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.348 7f0f23c4a700 1 --2- 172.21.2.101:0/1890080929 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0f040186c0 0x7f0f0401abc0 crc :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:22.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.348 7f0f23c4a700 1 -- 172.21.2.101:0/1890080929 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f1c09bcc0 msgr2=0x7f0f1c09c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:22.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.348 7f0f23c4a700 1 --2- 172.21.2.101:0/1890080929 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f1c09bcc0 0x7f0f1c09c0c0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 rx=0x7f0f10006a00 tx=0x7f0f1004f990).stop 2021-02-10T11:40:22.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.349 7f0f23c4a700 1 -- 172.21.2.101:0/1890080929 shutdown_connections 2021-02-10T11:40:22.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.349 7f0f23c4a700 1 --2- 172.21.2.101:0/1890080929 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0f040186c0 0x7f0f0401abc0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:22.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.349 7f0f23c4a700 1 --2- 172.21.2.101:0/1890080929 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f1c0a1240 0x7f0f1c0858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:22.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.349 7f0f23c4a700 1 --2- 172.21.2.101:0/1890080929 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f1c0a0820 0x7f0f1c0a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:22.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.349 7f0f23c4a700 1 --2- 172.21.2.101:0/1890080929 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f1c09bcc0 0x7f0f1c09c0c0 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:22.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.349 7f0f23c4a700 1 -- 172.21.2.101:0/1890080929 >> 172.21.2.101:0/1890080929 conn(0x7f0f1c0bef00 msgr2=0x7f0f1c09d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:22.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.349 7f0f23c4a700 1 -- 172.21.2.101:0/1890080929 shutdown_connections 2021-02-10T11:40:22.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:22.349 7f0f23c4a700 1 -- 172.21.2.101:0/1890080929 wait complete. 2021-02-10T11:40:22.360 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo getfattr --only-values -n ceph.dir.entries /home/ubuntu/cephtest/mnt.0/./volumes/_deleting 2021-02-10T11:40:22.710 INFO:teuthology.orchestra.run.gibba001.stderr:getfattr: Removing leading '/' from absolute path names 2021-02-10T11:40:22.711 INFO:teuthology.orchestra.run.gibba001.stdout:0 2021-02-10T11:40:22.711 DEBUG:tasks.cephfs.mount:Directory ./volumes/_deleting seen empty from 0 after 0s 2021-02-10T11:40:22.712 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:40:22.712 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:40:23.057 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:40:23.403 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:40:23.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:23.403 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:40:23.404 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:40:23.747 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:40:23.747 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:40:23.747 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:40:23.747 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:40:23.747 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:40:24.099 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[143449]: fuse finished with error 0 and tester_r 0 2021-02-10T11:40:24.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:24.100 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:40:24.100 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:40:24.435 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:40:24.435 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:40:24.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:40:24.435 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:40:24.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:24.770 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:40:24.770 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:40:25.106 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:40:25.106 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:40:25.107 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:40:25.107 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:40:25.107 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:40:25.457 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[144090]: fuse finished with error 0 and tester_r 0 2021-02-10T11:40:25.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:25.458 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:40:25.458 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:40:25.793 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:40:25.793 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:40:25.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:40:25.794 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:40:26.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:26.130 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:40:26.130 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:40:26.464 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:40:26.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:40:26.465 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:40:26.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:40:26.465 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:40:26.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:26.800 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:40:26.800 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:40:27.134 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:40:27.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:40:27.135 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:40:27.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:40:27.135 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:40:27.470 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-02-10T11:40:27.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.882 7f9640f8f700 1 Processor -- start 2021-02-10T11:40:27.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.882 7f9640f8f700 1 -- start start 2021-02-10T11:40:27.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.883 7f9640f8f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f963c0d1870 0x7f963c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:27.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.883 7f9640f8f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f963c1b0a10 0x7f963c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:27.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.883 7f9640f8f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f963c0f0b10 con 0x7f963c0d1c70 2021-02-10T11:40:27.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.883 7f9640f8f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f963c0e5ed0 con 0x7f963c0d1870 2021-02-10T11:40:27.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.883 7f9640f8f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f963c0f0650 con 0x7f963c1b0a10 2021-02-10T11:40:27.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.884 7f963a59c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f963c1b0a10 0x7f963c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:27.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.884 7f963ad9d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f963c0d1870 0x7f963c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:27.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.884 7f963b59e700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f963c0d1c70 0x7f963c0cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:50348/0 (socket says 172.21.2.101:50348) 2021-02-10T11:40:27.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.884 7f963b59e700 1 -- 172.21.2.101:0/1070776298 learned_addr learned my addr 172.21.2.101:0/1070776298 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:27.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.884 7f9639d9b700 1 -- 172.21.2.101:0/1070776298 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183546326 0 0) 0x7f9634001510 con 0x7f963c0d1c70 2021-02-10T11:40:27.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.884 7f963ad9d700 1 -- 172.21.2.101:0/1070776298 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f963c1b0a10 msgr2=0x7f963c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:27.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.884 7f963ad9d700 1 --2- 172.21.2.101:0/1070776298 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f963c1b0a10 0x7f963c0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:27.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.884 7f963ad9d700 1 -- 172.21.2.101:0/1070776298 >> v1:172.21.2.108:6789/0 conn(0x7f963c0d1c70 legacy=0x7f963c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:27.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.884 7f963ad9d700 1 -- 172.21.2.101:0/1070776298 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963c0f0870 con 0x7f963c0d1870 2021-02-10T11:40:27.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.884 7f963a59c700 1 --2- 172.21.2.101:0/1070776298 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f963c1b0a10 0x7f963c0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:40:27.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.885 7f963ad9d700 1 --2- 172.21.2.101:0/1070776298 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f963c0d1870 0x7f963c1b03f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f9624006650 tx=0x7f9624009a80).ready entity=mon.1 client_cookie=4bc245b64fd59cea server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:27.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.885 7f9639d9b700 1 -- 172.21.2.101:0/1070776298 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9624006390 con 0x7f963c0d1870 2021-02-10T11:40:27.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.885 7f9639d9b700 1 -- 172.21.2.101:0/1070776298 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96240009b0 con 0x7f963c0d1870 2021-02-10T11:40:27.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.885 7f9639d9b700 1 -- 172.21.2.101:0/1070776298 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f962400c2b0 con 0x7f963c0d1870 2021-02-10T11:40:27.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.885 7f9640f8f700 1 -- 172.21.2.101:0/1070776298 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f963c0d1870 msgr2=0x7f963c1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:27.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.885 7f9640f8f700 1 --2- 172.21.2.101:0/1070776298 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f963c0d1870 0x7f963c1b03f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f9624006650 tx=0x7f9624009a80).stop 2021-02-10T11:40:27.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.886 7f9640f8f700 1 -- 172.21.2.101:0/1070776298 shutdown_connections 2021-02-10T11:40:27.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.886 7f9640f8f700 1 --2- 172.21.2.101:0/1070776298 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f963c1b0a10 0x7f963c0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:27.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.886 7f9640f8f700 1 --2- 172.21.2.101:0/1070776298 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f963c0d1870 0x7f963c1b03f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:27.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.886 7f9640f8f700 1 -- 172.21.2.101:0/1070776298 >> 172.21.2.101:0/1070776298 conn(0x7f963c0ce450 msgr2=0x7f963c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:27.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.886 7f9640f8f700 1 -- 172.21.2.101:0/1070776298 shutdown_connections 2021-02-10T11:40:27.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.886 7f9640f8f700 1 -- 172.21.2.101:0/1070776298 wait complete. 2021-02-10T11:40:27.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.900 7f9640f8f700 1 Processor -- start 2021-02-10T11:40:27.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.900 7f9640f8f700 1 -- start start 2021-02-10T11:40:27.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.901 7f9640f8f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f963c0ebe70 0x7f963c0ec270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:27.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.901 7f9640f8f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f963c093830 0x7f963c093c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:27.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.901 7f9640f8f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f963c094250 0x7f963c094650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:27.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.901 7f9640f8f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c0ec940 con 0x7f963c094250 2021-02-10T11:40:27.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.901 7f9640f8f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c0ece40 con 0x7f963c093830 2021-02-10T11:40:27.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.901 7f9640f8f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f963c0ed060 con 0x7f963c0ebe70 2021-02-10T11:40:27.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.901 7f963b59e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f963c094250 0x7f963c094650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:27.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.901 7f963b59e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f963c094250 0x7f963c094650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56450/0 (socket says 172.21.2.101:56450) 2021-02-10T11:40:27.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.901 7f963b59e700 1 -- 172.21.2.101:0/2395438522 learned_addr learned my addr 172.21.2.101:0/2395438522 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:27.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.901 7f963b59e700 1 -- 172.21.2.101:0/2395438522 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f963c0ebe70 msgr2=0x7f963c0ec270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:27.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.901 7f963b59e700 1 --2- 172.21.2.101:0/2395438522 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f963c0ebe70 0x7f963c0ec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:27.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.901 7f963b59e700 1 -- 172.21.2.101:0/2395438522 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f963c093830 msgr2=0x7f963c093c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:27.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.901 7f963b59e700 1 --2- 172.21.2.101:0/2395438522 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f963c093830 0x7f963c093c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:27.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.901 7f963b59e700 1 -- 172.21.2.101:0/2395438522 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963c0ed280 con 0x7f963c094250 2021-02-10T11:40:27.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.902 7f963b59e700 1 --2- 172.21.2.101:0/2395438522 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f963c094250 0x7f963c094650 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 rx=0x7f963400a900 tx=0x7f963400aca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:27.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.902 7f96337fe700 1 -- 172.21.2.101:0/2395438522 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9634000f40 con 0x7f963c094250 2021-02-10T11:40:27.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.902 7f96337fe700 1 -- 172.21.2.101:0/2395438522 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9634007020 con 0x7f963c094250 2021-02-10T11:40:27.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.902 7f96337fe700 1 -- 172.21.2.101:0/2395438522 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96340046a0 con 0x7f963c094250 2021-02-10T11:40:27.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.902 7f9640f8f700 1 -- 172.21.2.101:0/2395438522 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f963c0ed280 con 0x7f963c094250 2021-02-10T11:40:27.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.902 7f9640f8f700 1 -- 172.21.2.101:0/2395438522 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f963c0ed810 con 0x7f963c094250 2021-02-10T11:40:27.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.903 7f96337fe700 1 -- 172.21.2.101:0/2395438522 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9634007200 con 0x7f963c094250 2021-02-10T11:40:27.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.903 7f96337fe700 1 --2- 172.21.2.101:0/2395438522 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96200186c0 0x7f962001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:27.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.903 7f96337fe700 1 -- 172.21.2.101:0/2395438522 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9634032880 con 0x7f963c094250 2021-02-10T11:40:27.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.903 7f963ad9d700 1 --2- 172.21.2.101:0/2395438522 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96200186c0 0x7f962001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:27.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.903 7f963ad9d700 1 --2- 172.21.2.101:0/2395438522 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96200186c0 0x7f962001abc0 crc :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:27.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.933 7f9640f8f700 1 -- 172.21.2.101:0/2395438522 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f963c0ebc40 con 0x7f963c094250 2021-02-10T11:40:27.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:27.936 7f96337fe700 1 -- 172.21.2.101:0/2395438522 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9634000f40 con 0x7f963c094250 2021-02-10T11:40:28.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.084 7f9640f8f700 1 -- 172.21.2.101:0/2395438522 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f963c0ebc40 con 0x7f963c094250 2021-02-10T11:40:28.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.088 7f96337fe700 1 -- 172.21.2.101:0/2395438522 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9634007200 con 0x7f963c094250 2021-02-10T11:40:28.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.093 7f9640f8f700 1 -- 172.21.2.101:0/2395438522 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96200186c0 msgr2=0x7f962001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:28.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.093 7f9640f8f700 1 --2- 172.21.2.101:0/2395438522 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96200186c0 0x7f962001abc0 crc :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:28.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.093 7f9640f8f700 1 -- 172.21.2.101:0/2395438522 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f963c094250 msgr2=0x7f963c094650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:28.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.093 7f9640f8f700 1 --2- 172.21.2.101:0/2395438522 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f963c094250 0x7f963c094650 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 rx=0x7f963400a900 tx=0x7f963400aca0).stop 2021-02-10T11:40:28.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.093 7f9640f8f700 1 -- 172.21.2.101:0/2395438522 shutdown_connections 2021-02-10T11:40:28.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.093 7f9640f8f700 1 --2- 172.21.2.101:0/2395438522 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f963c094250 0x7f963c094650 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:28.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.093 7f9640f8f700 1 --2- 172.21.2.101:0/2395438522 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f963c093830 0x7f963c093c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:28.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.093 7f9640f8f700 1 --2- 172.21.2.101:0/2395438522 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96200186c0 0x7f962001abc0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:28.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.093 7f9640f8f700 1 --2- 172.21.2.101:0/2395438522 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f963c0ebe70 0x7f963c0ec270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:28.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.093 7f9640f8f700 1 -- 172.21.2.101:0/2395438522 >> 172.21.2.101:0/2395438522 conn(0x7f963c08fa10 msgr2=0x7f963c0bfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:28.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.094 7f9640f8f700 1 -- 172.21.2.101:0/2395438522 shutdown_connections 2021-02-10T11:40:28.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.094 7f9640f8f700 1 -- 172.21.2.101:0/2395438522 wait complete. 2021-02-10T11:40:28.107 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_default_uid_gid_subvolume' 2021-02-10T11:40:28.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.514 7fe7eb86e700 1 Processor -- start 2021-02-10T11:40:28.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.514 7fe7eb86e700 1 -- start start 2021-02-10T11:40:28.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.515 7fe7eb86e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe7e40d1870 0x7fe7e41b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.515 7fe7eb86e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7e40f0650 0x7fe7e40cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.515 7fe7eb86e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7e40f0b10 con 0x7fe7e41b0a10 2021-02-10T11:40:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.515 7fe7eb86e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7e40e5ed0 con 0x7fe7e40d1870 2021-02-10T11:40:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.515 7fe7eb86e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e40d1c70 con 0x7fe7e40f0650 2021-02-10T11:40:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.515 7fe7e968a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe7e40d1870 0x7fe7e41b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.515 7fe7e968a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe7e40d1870 0x7fe7e41b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56454/0 (socket says 172.21.2.101:56454) 2021-02-10T11:40:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.515 7fe7e968a700 1 -- 172.21.2.101:0/755706928 learned_addr learned my addr 172.21.2.101:0/755706928 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.515 7fe7e968a700 1 -- 172.21.2.101:0/755706928 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7e40f0650 msgr2=0x7fe7e40cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.515 7fe7e968a700 1 --2- 172.21.2.101:0/755706928 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7e40f0650 0x7fe7e40cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.515 7fe7e968a700 1 -- 172.21.2.101:0/755706928 >> v1:172.21.2.101:6789/0 conn(0x7fe7e41b0a10 legacy=0x7fe7e40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.515 7fe7e968a700 1 -- 172.21.2.101:0/755706928 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e40d1e90 con 0x7fe7e40d1870 2021-02-10T11:40:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.515 7fe7e968a700 1 --2- 172.21.2.101:0/755706928 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7e40d1870 0x7fe7e41b03f0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 rx=0x7fe7e0006700 tx=0x7fe7e0009b00).ready entity=mon.0 client_cookie=5ffa641a5f9be835 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.515 7fe7d7fff700 1 -- 172.21.2.101:0/755706928 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe7e000c0c0 con 0x7fe7e40d1870 2021-02-10T11:40:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.516 7fe7d7fff700 1 -- 172.21.2.101:0/755706928 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e000c340 con 0x7fe7e40d1870 2021-02-10T11:40:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.516 7fe7d7fff700 1 -- 172.21.2.101:0/755706928 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe7e000c820 con 0x7fe7e40d1870 2021-02-10T11:40:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.516 7fe7eb86e700 1 -- 172.21.2.101:0/755706928 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7e40d1870 msgr2=0x7fe7e41b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.516 7fe7eb86e700 1 --2- 172.21.2.101:0/755706928 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7e40d1870 0x7fe7e41b03f0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 rx=0x7fe7e0006700 tx=0x7fe7e0009b00).stop 2021-02-10T11:40:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.516 7fe7eb86e700 1 -- 172.21.2.101:0/755706928 shutdown_connections 2021-02-10T11:40:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.516 7fe7eb86e700 1 --2- 172.21.2.101:0/755706928 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7e40f0650 0x7fe7e40cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.516 7fe7eb86e700 1 --2- 172.21.2.101:0/755706928 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7e40d1870 0x7fe7e41b03f0 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.516 7fe7eb86e700 1 -- 172.21.2.101:0/755706928 >> 172.21.2.101:0/755706928 conn(0x7fe7e40ce450 msgr2=0x7fe7e40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.516 7fe7eb86e700 1 -- 172.21.2.101:0/755706928 shutdown_connections 2021-02-10T11:40:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.516 7fe7eb86e700 1 -- 172.21.2.101:0/755706928 wait complete. 2021-02-10T11:40:28.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.531 7fe7eb86e700 1 Processor -- start 2021-02-10T11:40:28.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.531 7fe7eb86e700 1 -- start start 2021-02-10T11:40:28.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.531 7fe7eb86e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7e40b0ad0 0x7fe7e4091f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:28.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.531 7fe7eb86e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7e41a8850 0x7fe7e41a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:28.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.531 7fe7eb86e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe7e41a6fa0 0x7fe7e41a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:28.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.531 7fe7eb86e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e41abfe0 con 0x7fe7e40b0ad0 2021-02-10T11:40:28.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.531 7fe7eb86e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e41a9c40 con 0x7fe7e41a6fa0 2021-02-10T11:40:28.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.531 7fe7eb86e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e41a9730 con 0x7fe7e41a8850 2021-02-10T11:40:28.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.532 7fe7e968a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7e40b0ad0 0x7fe7e4091f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:28.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.532 7fe7e968a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7e40b0ad0 0x7fe7e4091f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56456/0 (socket says 172.21.2.101:56456) 2021-02-10T11:40:28.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.532 7fe7e968a700 1 -- 172.21.2.101:0/559216686 learned_addr learned my addr 172.21.2.101:0/559216686 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:28.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.532 7fe7e968a700 1 -- 172.21.2.101:0/559216686 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7e41a8850 msgr2=0x7fe7e41a8c50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:28.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.532 7fe7e968a700 1 --2- 172.21.2.101:0/559216686 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7e41a8850 0x7fe7e41a8c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:28.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.532 7fe7e968a700 1 -- 172.21.2.101:0/559216686 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe7e41a6fa0 msgr2=0x7fe7e41a73a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:28.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.532 7fe7e968a700 1 --2- 172.21.2.101:0/559216686 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe7e41a6fa0 0x7fe7e41a73a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:28.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.532 7fe7e968a700 1 -- 172.21.2.101:0/559216686 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e41a9270 con 0x7fe7e40b0ad0 2021-02-10T11:40:28.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.532 7fe7e968a700 1 --2- 172.21.2.101:0/559216686 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7e40b0ad0 0x7fe7e4091f30 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 rx=0x7fe7e0006f20 tx=0x7fe7e000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:28.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.532 7fe7d5ffb700 1 -- 172.21.2.101:0/559216686 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe7e0000c90 con 0x7fe7e40b0ad0 2021-02-10T11:40:28.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.532 7fe7d5ffb700 1 -- 172.21.2.101:0/559216686 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e0009100 con 0x7fe7e40b0ad0 2021-02-10T11:40:28.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.532 7fe7d5ffb700 1 -- 172.21.2.101:0/559216686 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe7e0002220 con 0x7fe7e40b0ad0 2021-02-10T11:40:28.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.532 7fe7eb86e700 1 -- 172.21.2.101:0/559216686 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e41a9270 con 0x7fe7e40b0ad0 2021-02-10T11:40:28.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.532 7fe7eb86e700 1 -- 172.21.2.101:0/559216686 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e41a9270 con 0x7fe7e40b0ad0 2021-02-10T11:40:28.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.533 7fe7d5ffb700 1 -- 172.21.2.101:0/559216686 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe7e000a000 con 0x7fe7e40b0ad0 2021-02-10T11:40:28.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.533 7fe7d5ffb700 1 --2- 172.21.2.101:0/559216686 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7c40186c0 0x7fe7c401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:28.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.533 7fe7d5ffb700 1 -- 172.21.2.101:0/559216686 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe7e0032820 con 0x7fe7e40b0ad0 2021-02-10T11:40:28.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.533 7fe7e8e89700 1 --2- 172.21.2.101:0/559216686 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7c40186c0 0x7fe7c401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:28.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.533 7fe7e8e89700 1 --2- 172.21.2.101:0/559216686 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7c40186c0 0x7fe7c401abc0 crc :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:28.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.565 7fe7eb86e700 1 -- 172.21.2.101:0/559216686 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e41abfe0 con 0x7fe7e40b0ad0 2021-02-10T11:40:28.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.567 7fe7d5ffb700 1 -- 172.21.2.101:0/559216686 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe7e000a000 con 0x7fe7e40b0ad0 2021-02-10T11:40:28.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:28.701 7fe7eb86e700 1 -- 172.21.2.101:0/559216686 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_default_uid_gid_subvolume"]} v 0) v1 -- 0x7fe7e41abfe0 con 0x7fe7e40b0ad0 2021-02-10T11:40:29.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.144 7fe7d5ffb700 1 -- 172.21.2.101:0/559216686 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_default_uid_gid_subvolume"]}]=0 v546) v1 ==== 147+0+0 (secure 0 0 0) 0x7fe7e0009100 con 0x7fe7e40b0ad0 2021-02-10T11:40:29.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.169 7fe7eb86e700 1 -- 172.21.2.101:0/559216686 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7c40186c0 msgr2=0x7fe7c401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:29.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.169 7fe7eb86e700 1 --2- 172.21.2.101:0/559216686 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7c40186c0 0x7fe7c401abc0 crc :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:29.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.169 7fe7eb86e700 1 -- 172.21.2.101:0/559216686 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7e40b0ad0 msgr2=0x7fe7e4091f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:29.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.169 7fe7eb86e700 1 --2- 172.21.2.101:0/559216686 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7e40b0ad0 0x7fe7e4091f30 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 rx=0x7fe7e0006f20 tx=0x7fe7e000b990).stop 2021-02-10T11:40:29.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.169 7fe7eb86e700 1 -- 172.21.2.101:0/559216686 shutdown_connections 2021-02-10T11:40:29.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.169 7fe7eb86e700 1 --2- 172.21.2.101:0/559216686 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7c40186c0 0x7fe7c401abc0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:29.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.169 7fe7eb86e700 1 --2- 172.21.2.101:0/559216686 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7e41a8850 0x7fe7e41a8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:29.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.169 7fe7eb86e700 1 --2- 172.21.2.101:0/559216686 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe7e41a6fa0 0x7fe7e41a73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:29.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.169 7fe7eb86e700 1 --2- 172.21.2.101:0/559216686 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7e40b0ad0 0x7fe7e4091f30 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:29.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.169 7fe7eb86e700 1 -- 172.21.2.101:0/559216686 >> 172.21.2.101:0/559216686 conn(0x7fe7e419f8f0 msgr2=0x7fe7e41a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:29.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.169 7fe7eb86e700 1 -- 172.21.2.101:0/559216686 shutdown_connections 2021-02-10T11:40:29.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.169 7fe7eb86e700 1 -- 172.21.2.101:0/559216686 wait complete. 2021-02-10T11:40:29.180 INFO:tasks.cephfs_test_runner:test_default_uid_gid_subvolume (tasks.cephfs.test_volumes.TestVolumes) ... ok 2021-02-10T11:40:29.180 INFO:tasks.cephfs_test_runner:Starting test: test_default_uid_gid_subvolume_group (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:40:29.180 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_default_uid_gid_subvolume_group' 2021-02-10T11:40:29.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.589 7fd7ff0dc700 1 Processor -- start 2021-02-10T11:40:29.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.589 7fd7ff0dc700 1 -- start start 2021-02-10T11:40:29.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.589 7fd7ff0dc700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd7f80d1c70 0x7fd7f80f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:29.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.589 7fd7ff0dc700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f80f0b10 con 0x7fd7f80d0340 2021-02-10T11:40:29.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.589 7fd7ff0dc700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f81b0be0 con 0x7fd7f80d1870 2021-02-10T11:40:29.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.589 7fd7ff0dc700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f80e5ed0 con 0x7fd7f80d1c70 2021-02-10T11:40:29.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.590 7fd7f7fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd7f80d0340 0x7fd7f80ef640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56958/0 (socket says 172.21.2.101:56958) 2021-02-10T11:40:29.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.590 7fd7f7fff700 1 -- 172.21.2.101:0/4258620537 learned_addr learned my addr 172.21.2.101:0/4258620537 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:29.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.590 7fd7f77fe700 1 -- 172.21.2.101:0/4258620537 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3610508064 0 0) 0x7fd7e8000ce0 con 0x7fd7f80d0340 2021-02-10T11:40:29.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.590 7fd7f77fe700 1 -- 172.21.2.101:0/4258620537 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7e4002fe0 con 0x7fd7f80d0340 2021-02-10T11:40:29.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.590 7fd7fd6f9700 1 --2- 172.21.2.101:0/4258620537 >> v2:172.21.2.108:3300/0 conn(0x7fd7f80d1c70 0x7fd7f80f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:29.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.590 7fd7f77fe700 1 -- 172.21.2.101:0/4258620537 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1149477908 0 0) 0x7fd7e80010e0 con 0x7fd7f80d0340 2021-02-10T11:40:29.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.590 7fd7f77fe700 1 -- 172.21.2.101:0/4258620537 >> v2:172.21.2.108:3300/0 conn(0x7fd7f80d1c70 msgr2=0x7fd7f80f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:29.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.590 7fd7f77fe700 1 --2- 172.21.2.101:0/4258620537 >> v2:172.21.2.108:3300/0 conn(0x7fd7f80d1c70 0x7fd7f80f20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:29.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.590 7fd7f77fe700 1 -- 172.21.2.101:0/4258620537 >> v1:172.21.2.108:6789/0 conn(0x7fd7f80d1870 legacy=0x7fd7f81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:29.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.590 7fd7f77fe700 1 -- 172.21.2.101:0/4258620537 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f80efe30 con 0x7fd7f80d0340 2021-02-10T11:40:29.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.590 7fd7f77fe700 1 -- 172.21.2.101:0/4258620537 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fd7e8001660 con 0x7fd7f80d0340 2021-02-10T11:40:29.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.590 7fd7f77fe700 1 -- 172.21.2.101:0/4258620537 >> v1:172.21.2.101:6789/0 conn(0x7fd7f80d0340 legacy=0x7fd7f80ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:29.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.590 7fd7f77fe700 1 --2- 172.21.2.101:0/4258620537 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7e4003a60 0x7fd7e4005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:29.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.590 7fd7f77fe700 1 --2- 172.21.2.101:0/4258620537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7e4006780 0x7fd7e4008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:29.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.590 7fd7f77fe700 1 --2- 172.21.2.101:0/4258620537 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7e4009580 0x7fd7e400baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:29.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.590 7fd7f77fe700 1 -- 172.21.2.101:0/4258620537 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e400c2f0 con 0x7fd7e4006780 2021-02-10T11:40:29.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.590 7fd7f77fe700 1 -- 172.21.2.101:0/4258620537 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e400c510 con 0x7fd7e4009580 2021-02-10T11:40:29.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.590 7fd7f77fe700 1 -- 172.21.2.101:0/4258620537 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e400c730 con 0x7fd7e4003a60 2021-02-10T11:40:29.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.590 7fd7f7fff700 1 --2- 172.21.2.101:0/4258620537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7e4006780 0x7fd7e4008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:29.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.591 7fd7fd6f9700 1 --2- 172.21.2.101:0/4258620537 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7e4009580 0x7fd7e400baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:29.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.591 7fd7f7fff700 1 -- 172.21.2.101:0/4258620537 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7e4003a60 msgr2=0x7fd7e4005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:29.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.591 7fd7f7fff700 1 --2- 172.21.2.101:0/4258620537 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7e4003a60 0x7fd7e4005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:29.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.591 7fd7f7fff700 1 -- 172.21.2.101:0/4258620537 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7e4009580 msgr2=0x7fd7e400baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:29.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.591 7fd7f7fff700 1 --2- 172.21.2.101:0/4258620537 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7e4009580 0x7fd7e400baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:29.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.591 7fd7f7fff700 1 -- 172.21.2.101:0/4258620537 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e400c9d0 con 0x7fd7e4006780 2021-02-10T11:40:29.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.591 7fd7fd6f9700 1 -- 172.21.2.101:0/4258620537 reap_dead start 2021-02-10T11:40:29.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.591 7fd7f7fff700 1 --2- 172.21.2.101:0/4258620537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7e4006780 0x7fd7e4008c80 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 rx=0x7fd7e8001880 tx=0x7fd7e804bd80).ready entity=mon.0 client_cookie=fb6ff77dfa06d878 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:29.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.591 7fd7f77fe700 1 -- 172.21.2.101:0/4258620537 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd7e80010e0 con 0x7fd7e4006780 2021-02-10T11:40:29.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.591 7fd7f77fe700 1 -- 172.21.2.101:0/4258620537 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e804e310 con 0x7fd7e4006780 2021-02-10T11:40:29.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.591 7fd7f77fe700 1 -- 172.21.2.101:0/4258620537 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd7e804e770 con 0x7fd7e4006780 2021-02-10T11:40:29.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.591 7fd7ff0dc700 1 -- 172.21.2.101:0/4258620537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7e4006780 msgr2=0x7fd7e4008c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:29.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.591 7fd7ff0dc700 1 --2- 172.21.2.101:0/4258620537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7e4006780 0x7fd7e4008c80 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 rx=0x7fd7e8001880 tx=0x7fd7e804bd80).stop 2021-02-10T11:40:29.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.591 7fd7ff0dc700 1 -- 172.21.2.101:0/4258620537 shutdown_connections 2021-02-10T11:40:29.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.591 7fd7ff0dc700 1 --2- 172.21.2.101:0/4258620537 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7e4006780 0x7fd7e4008c80 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:29.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.591 7fd7ff0dc700 1 -- 172.21.2.101:0/4258620537 >> 172.21.2.101:0/4258620537 conn(0x7fd7f80ce450 msgr2=0x7fd7f80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:29.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.591 7fd7ff0dc700 1 -- 172.21.2.101:0/4258620537 shutdown_connections 2021-02-10T11:40:29.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.592 7fd7ff0dc700 1 -- 172.21.2.101:0/4258620537 wait complete. 2021-02-10T11:40:29.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.606 7fd7ff0dc700 1 Processor -- start 2021-02-10T11:40:29.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.606 7fd7ff0dc700 1 -- start start 2021-02-10T11:40:29.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.606 7fd7ff0dc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7f81942b0 0x7fd7f80b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:29.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.607 7fd7ff0dc700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7f8195070 0x7fd7f8195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:29.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.607 7fd7ff0dc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7f80bb8e0 0x7fd7f80bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:29.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.607 7fd7ff0dc700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f81946b0 con 0x7fd7f81942b0 2021-02-10T11:40:29.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.607 7fd7ff0dc700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f80b3b70 con 0x7fd7f8195070 2021-02-10T11:40:29.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.607 7fd7ff0dc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f80b3d90 con 0x7fd7f80bb8e0 2021-02-10T11:40:29.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.607 7fd7fcef8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7f81942b0 0x7fd7f80b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:29.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.607 7fd7fcef8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7f81942b0 0x7fd7f80b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56478/0 (socket says 172.21.2.101:56478) 2021-02-10T11:40:29.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.607 7fd7fcef8700 1 -- 172.21.2.101:0/343474513 learned_addr learned my addr 172.21.2.101:0/343474513 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:29.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.607 7fd7fcef8700 1 -- 172.21.2.101:0/343474513 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7f80bb8e0 msgr2=0x7fd7f80bbce0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:29.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.607 7fd7fcef8700 1 --2- 172.21.2.101:0/343474513 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7f80bb8e0 0x7fd7f80bbce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:29.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.607 7fd7fcef8700 1 -- 172.21.2.101:0/343474513 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7f8195070 msgr2=0x7fd7f8195470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:29.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.607 7fd7fcef8700 1 --2- 172.21.2.101:0/343474513 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7f8195070 0x7fd7f8195470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.607 7fd7fcef8700 1 -- 172.21.2.101:0/343474513 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f81a49e0 con 0x7fd7f81942b0 2021-02-10T11:40:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.607 7fd7fcef8700 1 --2- 172.21.2.101:0/343474513 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7f81942b0 0x7fd7f80b3550 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 rx=0x7fd7e000b560 tx=0x7fd7e000b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.607 7fd7f57fa700 1 -- 172.21.2.101:0/343474513 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd7e0006240 con 0x7fd7f81942b0 2021-02-10T11:40:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.607 7fd7f57fa700 1 -- 172.21.2.101:0/343474513 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e00054f0 con 0x7fd7f81942b0 2021-02-10T11:40:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.607 7fd7f57fa700 1 -- 172.21.2.101:0/343474513 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd7e0005a10 con 0x7fd7f81942b0 2021-02-10T11:40:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.607 7fd7ff0dc700 1 -- 172.21.2.101:0/343474513 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f81a49e0 con 0x7fd7f81942b0 2021-02-10T11:40:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.607 7fd7ff0dc700 1 -- 172.21.2.101:0/343474513 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f80bc5f0 con 0x7fd7f81942b0 2021-02-10T11:40:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.608 7fd7f57fa700 1 -- 172.21.2.101:0/343474513 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd7e0006030 con 0x7fd7f81942b0 2021-02-10T11:40:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.608 7fd7f57fa700 1 --2- 172.21.2.101:0/343474513 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7dc0186c0 0x7fd7dc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.608 7fd7f57fa700 1 -- 172.21.2.101:0/343474513 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd7e00325b0 con 0x7fd7f81942b0 2021-02-10T11:40:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.608 7fd7f7fff700 1 --2- 172.21.2.101:0/343474513 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7dc0186c0 0x7fd7dc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:29.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.608 7fd7f7fff700 1 --2- 172.21.2.101:0/343474513 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7dc0186c0 0x7fd7dc01abc0 crc :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:29.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.639 7fd7ff0dc700 1 -- 172.21.2.101:0/343474513 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f80bc5f0 con 0x7fd7f81942b0 2021-02-10T11:40:29.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.642 7fd7f57fa700 1 -- 172.21.2.101:0/343474513 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd7e001f470 con 0x7fd7f81942b0 2021-02-10T11:40:29.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:29.774 7fd7ff0dc700 1 -- 172.21.2.101:0/343474513 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_default_uid_gid_subvolume_group"]} v 0) v1 -- 0x7fd7f80bc5f0 con 0x7fd7f81942b0 2021-02-10T11:40:30.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.148 7fd7f57fa700 1 -- 172.21.2.101:0/343474513 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_default_uid_gid_subvolume_group"]}]=0 v547) v1 ==== 156+0+0 (secure 0 0 0) 0x7fd7e0006030 con 0x7fd7f81942b0 2021-02-10T11:40:30.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.194 7fd7ff0dc700 1 -- 172.21.2.101:0/343474513 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7dc0186c0 msgr2=0x7fd7dc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:30.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.194 7fd7ff0dc700 1 --2- 172.21.2.101:0/343474513 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7dc0186c0 0x7fd7dc01abc0 crc :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:30.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.194 7fd7ff0dc700 1 -- 172.21.2.101:0/343474513 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7f81942b0 msgr2=0x7fd7f80b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:30.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.194 7fd7ff0dc700 1 --2- 172.21.2.101:0/343474513 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7f81942b0 0x7fd7f80b3550 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 rx=0x7fd7e000b560 tx=0x7fd7e000b900).stop 2021-02-10T11:40:30.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.194 7fd7ff0dc700 1 -- 172.21.2.101:0/343474513 shutdown_connections 2021-02-10T11:40:30.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.194 7fd7ff0dc700 1 --2- 172.21.2.101:0/343474513 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7dc0186c0 0x7fd7dc01abc0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:30.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.194 7fd7ff0dc700 1 --2- 172.21.2.101:0/343474513 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7f80bb8e0 0x7fd7f80bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:30.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.194 7fd7ff0dc700 1 --2- 172.21.2.101:0/343474513 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7f8195070 0x7fd7f8195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:30.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.194 7fd7ff0dc700 1 --2- 172.21.2.101:0/343474513 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7f81942b0 0x7fd7f80b3550 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:30.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.194 7fd7ff0dc700 1 -- 172.21.2.101:0/343474513 >> 172.21.2.101:0/343474513 conn(0x7fd7f80b3040 msgr2=0x7fd7f80c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:30.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.195 7fd7ff0dc700 1 -- 172.21.2.101:0/343474513 shutdown_connections 2021-02-10T11:40:30.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.195 7fd7ff0dc700 1 -- 172.21.2.101:0/343474513 wait complete. 2021-02-10T11:40:30.205 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:40:30.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.618 7f330ea68700 1 Processor -- start 2021-02-10T11:40:30.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.618 7f330ea68700 1 -- start start 2021-02-10T11:40:30.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.618 7f330ea68700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33080d1860 0x7f33081b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:30.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.618 7f330ea68700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f33081b0ad0 0x7f3308108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:30.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.618 7f330ea68700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3308109a50 con 0x7f33080d1c60 2021-02-10T11:40:30.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.618 7f330ea68700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33080e5ee0 con 0x7f33081b0ad0 2021-02-10T11:40:30.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.618 7f330ea68700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3308109740 con 0x7f33080d1860 2021-02-10T11:40:30.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.619 7f3307fff700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f33081b0ad0 0x7f3308108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:30.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.619 7f3307fff700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f33081b0ad0 0x7f3308108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:38738/0 (socket says 172.21.2.101:38738) 2021-02-10T11:40:30.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.619 7f3307fff700 1 -- 172.21.2.101:0/2838154617 learned_addr learned my addr 172.21.2.101:0/2838154617 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:30.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.619 7f330c884700 1 --2- 172.21.2.101:0/2838154617 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33080d1860 0x7f33081b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:30.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.620 7f3307fff700 1 -- 172.21.2.101:0/2838154617 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33080d1860 msgr2=0x7f33081b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:30.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.620 7f3307fff700 1 --2- 172.21.2.101:0/2838154617 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33080d1860 0x7f33081b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:30.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.620 7f3307fff700 1 -- 172.21.2.101:0/2838154617 >> v1:172.21.2.108:6789/0 conn(0x7f33080d1c60 legacy=0x7f3308108f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:30.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.620 7f3307fff700 1 -- 172.21.2.101:0/2838154617 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3308088070 con 0x7f33081b0ad0 2021-02-10T11:40:30.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.620 7f3307fff700 1 --2- 172.21.2.101:0/2838154617 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f33081b0ad0 0x7f3308108930 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f32f8006700 tx=0x7f32f8009b80).ready entity=mon.1 client_cookie=c927954666d798b4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:30.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.620 7f33077fe700 1 -- 172.21.2.101:0/2838154617 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f32f800c0c0 con 0x7f33081b0ad0 2021-02-10T11:40:30.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.620 7f33077fe700 1 -- 172.21.2.101:0/2838154617 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f800c0c0 con 0x7f33081b0ad0 2021-02-10T11:40:30.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.620 7f33077fe700 1 -- 172.21.2.101:0/2838154617 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f32f800c400 con 0x7f33081b0ad0 2021-02-10T11:40:30.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.620 7f330ea68700 1 -- 172.21.2.101:0/2838154617 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f33081b0ad0 msgr2=0x7f3308108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:30.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.620 7f330ea68700 1 --2- 172.21.2.101:0/2838154617 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f33081b0ad0 0x7f3308108930 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f32f8006700 tx=0x7f32f8009b80).stop 2021-02-10T11:40:30.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.621 7f330ea68700 1 -- 172.21.2.101:0/2838154617 shutdown_connections 2021-02-10T11:40:30.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.621 7f330ea68700 1 --2- 172.21.2.101:0/2838154617 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f33081b0ad0 0x7f3308108930 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:30.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.621 7f330ea68700 1 --2- 172.21.2.101:0/2838154617 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33080d1860 0x7f33081b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:30.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.621 7f330ea68700 1 -- 172.21.2.101:0/2838154617 >> 172.21.2.101:0/2838154617 conn(0x7f33080ce410 msgr2=0x7f33080d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:30.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.621 7f330ea68700 1 -- 172.21.2.101:0/2838154617 shutdown_connections 2021-02-10T11:40:30.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.621 7f330ea68700 1 -- 172.21.2.101:0/2838154617 wait complete. 2021-02-10T11:40:30.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.636 7f330ea68700 1 Processor -- start 2021-02-10T11:40:30.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.636 7f330ea68700 1 -- start start 2021-02-10T11:40:30.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.636 7f330ea68700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33080b6da0 0x7f33080b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:30.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.636 7f330ea68700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33080a70d0 0x7f33080b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:30.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.636 7f330ea68700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f33080b8780 0x7f33080b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:30.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.636 7f330ea68700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33080a7610 con 0x7f33080a70d0 2021-02-10T11:40:30.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.636 7f330ea68700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33080b7de0 con 0x7f33080b8780 2021-02-10T11:40:30.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.636 7f330ea68700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33080b8000 con 0x7f33080b6da0 2021-02-10T11:40:30.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.636 7f3307fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33080a70d0 0x7f33080b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:30.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.637 7f3307fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33080a70d0 0x7f33080b77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56492/0 (socket says 172.21.2.101:56492) 2021-02-10T11:40:30.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.637 7f3307fff700 1 -- 172.21.2.101:0/1109420679 learned_addr learned my addr 172.21.2.101:0/1109420679 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:30.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.637 7f3307fff700 1 -- 172.21.2.101:0/1109420679 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33080b6da0 msgr2=0x7f33080b71a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:30.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.637 7f3307fff700 1 --2- 172.21.2.101:0/1109420679 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33080b6da0 0x7f33080b71a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:30.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.637 7f3307fff700 1 -- 172.21.2.101:0/1109420679 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f33080b8780 msgr2=0x7f33080b8b80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:30.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.637 7f3307fff700 1 --2- 172.21.2.101:0/1109420679 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f33080b8780 0x7f33080b8b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:30.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.637 7f3307fff700 1 -- 172.21.2.101:0/1109420679 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33080b91a0 con 0x7f33080a70d0 2021-02-10T11:40:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.637 7f3307fff700 1 --2- 172.21.2.101:0/1109420679 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33080a70d0 0x7f33080b77c0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 rx=0x7f32f80066a0 tx=0x7f32f800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.637 7f33057fa700 1 -- 172.21.2.101:0/1109420679 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f32f800bd80 con 0x7f33080a70d0 2021-02-10T11:40:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.637 7f33057fa700 1 -- 172.21.2.101:0/1109420679 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f80052e0 con 0x7f33080a70d0 2021-02-10T11:40:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.637 7f330ea68700 1 -- 172.21.2.101:0/1109420679 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33080b91a0 con 0x7f33080a70d0 2021-02-10T11:40:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.637 7f33057fa700 1 -- 172.21.2.101:0/1109420679 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f32f8005740 con 0x7f33080a70d0 2021-02-10T11:40:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.637 7f330ea68700 1 -- 172.21.2.101:0/1109420679 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33080b9730 con 0x7f33080a70d0 2021-02-10T11:40:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.638 7f33057fa700 1 -- 172.21.2.101:0/1109420679 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f32f8006030 con 0x7f33080a70d0 2021-02-10T11:40:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.638 7f33057fa700 1 --2- 172.21.2.101:0/1109420679 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32ec0186c0 0x7f32ec01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.638 7f33057fa700 1 -- 172.21.2.101:0/1109420679 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f32f80323d0 con 0x7f33080a70d0 2021-02-10T11:40:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.638 7f330c884700 1 --2- 172.21.2.101:0/1109420679 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32ec0186c0 0x7f32ec01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.638 7f330c884700 1 --2- 172.21.2.101:0/1109420679 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32ec0186c0 0x7f32ec01abc0 crc :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:30.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.668 7f330ea68700 1 -- 172.21.2.101:0/1109420679 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3308082920 con 0x7f33080a70d0 2021-02-10T11:40:30.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.671 7f33057fa700 1 -- 172.21.2.101:0/1109420679 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f32f8025f80 con 0x7f33080a70d0 2021-02-10T11:40:30.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.821 7f330ea68700 1 -- 172.21.2.101:0/1109420679 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3308109a50 con 0x7f33080a70d0 2021-02-10T11:40:30.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.825 7f33057fa700 1 -- 172.21.2.101:0/1109420679 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f32f8025f80 con 0x7f33080a70d0 2021-02-10T11:40:30.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.829 7f330ea68700 1 -- 172.21.2.101:0/1109420679 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32ec0186c0 msgr2=0x7f32ec01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:30.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.829 7f330ea68700 1 --2- 172.21.2.101:0/1109420679 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32ec0186c0 0x7f32ec01abc0 crc :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:30.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.829 7f330ea68700 1 -- 172.21.2.101:0/1109420679 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33080a70d0 msgr2=0x7f33080b77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:30.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.829 7f330ea68700 1 --2- 172.21.2.101:0/1109420679 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33080a70d0 0x7f33080b77c0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 rx=0x7f32f80066a0 tx=0x7f32f800b990).stop 2021-02-10T11:40:30.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.830 7f330ea68700 1 -- 172.21.2.101:0/1109420679 shutdown_connections 2021-02-10T11:40:30.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.830 7f330ea68700 1 --2- 172.21.2.101:0/1109420679 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f33080b8780 0x7f33080b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:30.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.830 7f330ea68700 1 --2- 172.21.2.101:0/1109420679 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33080a70d0 0x7f33080b77c0 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:30.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.830 7f330ea68700 1 --2- 172.21.2.101:0/1109420679 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32ec0186c0 0x7f32ec01abc0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:30.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.830 7f330ea68700 1 --2- 172.21.2.101:0/1109420679 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33080b6da0 0x7f33080b71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:30.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.830 7f330ea68700 1 -- 172.21.2.101:0/1109420679 >> 172.21.2.101:0/1109420679 conn(0x7f3308092c70 msgr2=0x7f3308093530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:30.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.830 7f330ea68700 1 -- 172.21.2.101:0/1109420679 shutdown_connections 2021-02-10T11:40:30.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:30.830 7f330ea68700 1 -- 172.21.2.101:0/1109420679 wait complete. 2021-02-10T11:40:30.841 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:40:30.841 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:40:31.187 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:40:31.533 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:40:31.533 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:31.533 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:40:31.533 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:40:31.867 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2021-02-10T11:40:31.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:40:31.867 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-02-10T11:40:31.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:31.868 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:40:31.868 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:40:32.202 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:40:32.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:40:32.202 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:40:32.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:32.203 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:40:32.203 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:40:32.538 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:40:32.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:40:32.539 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:40:32.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:32.539 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:40:32.539 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:40:32.874 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:40:32.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:40:32.874 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:40:32.874 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json-pretty 2021-02-10T11:40:33.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.284 7fb849c38700 1 Processor -- start 2021-02-10T11:40:33.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.284 7fb849c38700 1 -- start start 2021-02-10T11:40:33.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.284 7fb849c38700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb8440d0340 0x7fb8440ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:33.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.284 7fb849c38700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8440f0b10 con 0x7fb8440d1870 2021-02-10T11:40:33.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.284 7fb849c38700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8441b0be0 con 0x7fb8440d1c70 2021-02-10T11:40:33.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.284 7fb849c38700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8440e5ed0 con 0x7fb8440d0340 2021-02-10T11:40:33.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.284 7fb842ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb8440d0340 0x7fb8440ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:33.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.284 7fb8437fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb8440d1870 0x7fb8441b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:56992/0 (socket says 172.21.2.101:56992) 2021-02-10T11:40:33.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.284 7fb842ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb8440d0340 0x7fb8440ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56498/0 (socket says 172.21.2.101:56498) 2021-02-10T11:40:33.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.284 7fb8437fe700 1 -- 172.21.2.101:0/21159319 learned_addr learned my addr 172.21.2.101:0/21159319 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:33.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.285 7fb842ffd700 1 -- 172.21.2.101:0/21159319 >> v1:172.21.2.108:6789/0 conn(0x7fb8440d1c70 legacy=0x7fb8440efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:33.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.285 7fb8427fc700 1 -- 172.21.2.101:0/21159319 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79663307 0 0) 0x7fb834000ce0 con 0x7fb8440d1870 2021-02-10T11:40:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.285 7fb842ffd700 1 -- 172.21.2.101:0/21159319 >> v1:172.21.2.101:6789/0 conn(0x7fb8440d1870 legacy=0x7fb8441b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.285 7fb842ffd700 1 -- 172.21.2.101:0/21159319 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8440f0450 con 0x7fb8440d0340 2021-02-10T11:40:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.285 7fb842ffd700 1 --2- 172.21.2.101:0/21159319 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8440d0340 0x7fb8440ef640 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 rx=0x7fb838006650 tx=0x7fb838009a80).ready entity=mon.0 client_cookie=e8531d9fb032ef0c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.285 7fb8427fc700 1 -- 172.21.2.101:0/21159319 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb8380009b0 con 0x7fb8440d0340 2021-02-10T11:40:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.285 7fb8427fc700 1 -- 172.21.2.101:0/21159319 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83800c0c0 con 0x7fb8440d0340 2021-02-10T11:40:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.285 7fb8427fc700 1 -- 172.21.2.101:0/21159319 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb83800c5e0 con 0x7fb8440d0340 2021-02-10T11:40:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.285 7fb849c38700 1 -- 172.21.2.101:0/21159319 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8440d0340 msgr2=0x7fb8440ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.285 7fb849c38700 1 --2- 172.21.2.101:0/21159319 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8440d0340 0x7fb8440ef640 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 rx=0x7fb838006650 tx=0x7fb838009a80).stop 2021-02-10T11:40:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.286 7fb849c38700 1 -- 172.21.2.101:0/21159319 shutdown_connections 2021-02-10T11:40:33.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.286 7fb849c38700 1 --2- 172.21.2.101:0/21159319 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8440d0340 0x7fb8440ef640 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:33.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.286 7fb849c38700 1 -- 172.21.2.101:0/21159319 >> 172.21.2.101:0/21159319 conn(0x7fb8440ce450 msgr2=0x7fb8440d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:33.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.286 7fb849c38700 1 -- 172.21.2.101:0/21159319 shutdown_connections 2021-02-10T11:40:33.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.286 7fb849c38700 1 -- 172.21.2.101:0/21159319 wait complete. 2021-02-10T11:40:33.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.301 7fb849c38700 1 Processor -- start 2021-02-10T11:40:33.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.301 7fb849c38700 1 -- start start 2021-02-10T11:40:33.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.301 7fb849c38700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8441947b0 0x7fb8440a69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:33.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.301 7fb849c38700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb8440a6fc0 0x7fb8441a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:33.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.301 7fb849c38700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb8441a1bf0 0x7fb8441a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:33.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.301 7fb849c38700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8440a73c0 con 0x7fb8441947b0 2021-02-10T11:40:33.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.301 7fb849c38700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84418f170 con 0x7fb8441a1bf0 2021-02-10T11:40:33.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.301 7fb849c38700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb844194bb0 con 0x7fb8440a6fc0 2021-02-10T11:40:33.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.301 7fb8437fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8441947b0 0x7fb8440a69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:33.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.301 7fb8437fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8441947b0 0x7fb8440a69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56504/0 (socket says 172.21.2.101:56504) 2021-02-10T11:40:33.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.302 7fb8437fe700 1 -- 172.21.2.101:0/536816346 learned_addr learned my addr 172.21.2.101:0/536816346 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:33.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.302 7fb8437fe700 1 -- 172.21.2.101:0/536816346 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb8440a6fc0 msgr2=0x7fb8441a15d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:33.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.302 7fb8437fe700 1 --2- 172.21.2.101:0/536816346 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb8440a6fc0 0x7fb8441a15d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:33.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.302 7fb8437fe700 1 -- 172.21.2.101:0/536816346 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb8441a1bf0 msgr2=0x7fb8441a1ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:33.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.302 7fb8437fe700 1 --2- 172.21.2.101:0/536816346 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb8441a1bf0 0x7fb8441a1ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:33.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.302 7fb8437fe700 1 -- 172.21.2.101:0/536816346 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84418ec70 con 0x7fb8441947b0 2021-02-10T11:40:33.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.302 7fb8437fe700 1 --2- 172.21.2.101:0/536816346 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8441947b0 0x7fb8440a69a0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 rx=0x7fb834001430 tx=0x7fb83400a930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:33.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.302 7fb82bfff700 1 -- 172.21.2.101:0/536816346 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb834000ce0 con 0x7fb8441947b0 2021-02-10T11:40:33.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.302 7fb82bfff700 1 -- 172.21.2.101:0/536816346 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83400ad20 con 0x7fb8441947b0 2021-02-10T11:40:33.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.302 7fb82bfff700 1 -- 172.21.2.101:0/536816346 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb834004620 con 0x7fb8441947b0 2021-02-10T11:40:33.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.302 7fb849c38700 1 -- 172.21.2.101:0/536816346 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb84418ec70 con 0x7fb8441947b0 2021-02-10T11:40:33.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.302 7fb849c38700 1 -- 172.21.2.101:0/536816346 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84418ec70 con 0x7fb8441947b0 2021-02-10T11:40:33.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.303 7fb82bfff700 1 -- 172.21.2.101:0/536816346 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb8340043a0 con 0x7fb8441947b0 2021-02-10T11:40:33.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.303 7fb82bfff700 1 --2- 172.21.2.101:0/536816346 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8240186c0 0x7fb82401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:33.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.303 7fb82bfff700 1 -- 172.21.2.101:0/536816346 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb834032630 con 0x7fb8441947b0 2021-02-10T11:40:33.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.303 7fb842ffd700 1 --2- 172.21.2.101:0/536816346 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8240186c0 0x7fb82401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:33.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.303 7fb842ffd700 1 --2- 172.21.2.101:0/536816346 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8240186c0 0x7fb82401abc0 crc :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:33.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.334 7fb849c38700 1 -- 172.21.2.101:0/536816346 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8440ee0e0 con 0x7fb8441947b0 2021-02-10T11:40:33.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.336 7fb82bfff700 1 -- 172.21.2.101:0/536816346 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb834030de0 con 0x7fb8441947b0 2021-02-10T11:40:33.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.459 7fb849c38700 1 -- 172.21.2.101:0/536816346 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb8440ce450 con 0x7fb8441947b0 2021-02-10T11:40:33.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.460 7fb82bfff700 1 -- 172.21.2.101:0/536816346 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v103) v1 ==== 81+0+30372 (secure 0 0 0) 0x7fb834030de0 con 0x7fb8441947b0 2021-02-10T11:40:33.463 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:40:33.463 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:40:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 103, 2021-02-10T11:40:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:40:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "created": "2021-02-10 11:12:36.788366", 2021-02-10T11:40:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "modified": "2021-02-10 11:39:57.056338", 2021-02-10T11:40:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "last_up_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:40:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "last_in_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:40:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-02-10T11:40:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_num": 5799936, 2021-02-10T11:40:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_set": [ 2021-02-10T11:40:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "pglog_hardlimit", 2021-02-10T11:40:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "purged_snapdirs", 2021-02-10T11:40:33.464 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_deletes", 2021-02-10T11:40:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: "sortbitwise" 2021-02-10T11:40:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:40:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_version": 4, 2021-02-10T11:40:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: "full_ratio": 0.94999998807907104, 2021-02-10T11:40:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-02-10T11:40:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-02-10T11:40:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_snapshot": "", 2021-02-10T11:40:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_max": 21, 2021-02-10T11:40:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: "max_osd": 8, 2021-02-10T11:40:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: "require_min_compat_client": "jewel", 2021-02-10T11:40:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: "min_compat_client": "jewel", 2021-02-10T11:40:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: "require_osd_release": "nautilus", 2021-02-10T11:40:33.465 INFO:teuthology.orchestra.run.gibba001.stdout: "pools": [ 2021-02-10T11:40:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 1, 2021-02-10T11:40:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "rbd", 2021-02-10T11:40:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:13:08.037472", 2021-02-10T11:40:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:40:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:40:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:40:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:40:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:40:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:40:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:40:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:40:33.466 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:40:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:40:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:40:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:40:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:40:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:40:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:40:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:40:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:40:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:40:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:40:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:40:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.467 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "16", 2021-02-10T11:40:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:40:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:40:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:40:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:40:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:40:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:40:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:40:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:40:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:40:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:40:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:40:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:40:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:40:33.468 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:40:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:40:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:40:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:40:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:40:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:40:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:40:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:40:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:40:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:40:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:40:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:40:33.469 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:40:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:40:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:40:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:40:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:40:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:40:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:40:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:40:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:40:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:40:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:40:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:40:33.470 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:40:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:40:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd": {} 2021-02-10T11:40:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 20, 2021-02-10T11:40:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_metadata", 2021-02-10T11:40:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:39:54.022810", 2021-02-10T11:40:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:40:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:40:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:40:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:40:33.471 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:40:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:40:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:40:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:40:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:40:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:40:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:40:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:40:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:40:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:40:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:40:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:40:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:40:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:40:33.472 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:40:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:40:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "102", 2021-02-10T11:40:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:40:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:40:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:40:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:40:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:40:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:40:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:40:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:40:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:40:33.473 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:40:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:40:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:40:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:40:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:40:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:40:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:40:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:40:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:40:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:40:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:40:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:40:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:40:33.474 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:40:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:40:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:40:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:40:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:40:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:40:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:40:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:40:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:40:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:40:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:40:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:40:33.475 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:40:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:40:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:40:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: "options": { 2021-02-10T11:40:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_bias": 4, 2021-02-10T11:40:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_min": 16, 2021-02-10T11:40:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_priority": 5 2021-02-10T11:40:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:40:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:40:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": "cephfs" 2021-02-10T11:40:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.476 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 21, 2021-02-10T11:40:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_data", 2021-02-10T11:40:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:39:55.095134", 2021-02-10T11:40:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:40:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:40:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:40:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:40:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:40:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:40:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:40:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:40:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:40:33.477 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:40:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:40:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:40:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:40:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:40:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:40:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:40:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:40:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:40:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:40:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:40:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "103", 2021-02-10T11:40:33.478 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:40:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:40:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:40:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:40:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:40:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:40:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:40:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:40:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:40:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:40:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:40:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:40:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:40:33.479 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:40:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:40:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:40:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:40:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:40:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:40:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:40:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:40:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:40:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:40:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:40:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:40:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:40:33.480 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:40:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:40:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:40:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:40:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:40:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:40:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:40:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:40:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:40:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:40:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:40:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:40:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:40:33.481 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:40:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: "data": "cephfs" 2021-02-10T11:40:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:40:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: "osds": [ 2021-02-10T11:40:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:40:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5", 2021-02-10T11:40:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:40:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:40:33.482 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:40:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:40:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:40:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:40:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:40:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 99, 2021-02-10T11:40:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:40:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:40:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:40:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6802", 2021-02-10T11:40:33.483 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:40:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6805", 2021-02-10T11:40:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:40:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:40:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.484 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6807", 2021-02-10T11:40:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:40:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6809", 2021-02-10T11:40:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:40:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:40:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.485 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6817", 2021-02-10T11:40:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:40:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6820", 2021-02-10T11:40:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:40:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.486 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:40:33.487 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.487 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.487 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.487 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6812", 2021-02-10T11:40:33.487 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:40:33.487 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.487 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.487 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.488 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6814", 2021-02-10T11:40:33.488 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:40:33.488 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.489 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.489 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.489 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6805/25093", 2021-02-10T11:40:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6809/25093", 2021-02-10T11:40:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6820/25093", 2021-02-10T11:40:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6814/25093", 2021-02-10T11:40:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:40:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:40:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:40:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:40:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "ddc04ce3-b313-4707-8270-585391e38ecc", 2021-02-10T11:40:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:40:33.490 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:40:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:40:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:40:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:40:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:40:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:40:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 101, 2021-02-10T11:40:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:40:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:40:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:40:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.491 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6803", 2021-02-10T11:40:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:40:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6804", 2021-02-10T11:40:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:40:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:40:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.492 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6806", 2021-02-10T11:40:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:40:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6808", 2021-02-10T11:40:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:40:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:40:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.493 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6818", 2021-02-10T11:40:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:40:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6821", 2021-02-10T11:40:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:40:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.494 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:40:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6810", 2021-02-10T11:40:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:40:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6815", 2021-02-10T11:40:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:40:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.495 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6804/25092", 2021-02-10T11:40:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6808/25092", 2021-02-10T11:40:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6821/25092", 2021-02-10T11:40:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6815/25092", 2021-02-10T11:40:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:40:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:40:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:40:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.496 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:40:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3", 2021-02-10T11:40:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:40:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:40:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:40:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:40:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:40:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:40:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:40:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 101, 2021-02-10T11:40:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:40:33.497 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:40:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:40:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6811", 2021-02-10T11:40:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:40:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6813", 2021-02-10T11:40:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:40:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.498 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:40:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6816", 2021-02-10T11:40:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:40:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6819", 2021-02-10T11:40:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:40:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.499 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:40:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6825", 2021-02-10T11:40:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:40:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6827", 2021-02-10T11:40:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:40:33.500 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:40:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6822", 2021-02-10T11:40:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:40:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.501 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6823", 2021-02-10T11:40:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:40:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6813/25094", 2021-02-10T11:40:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6819/25094", 2021-02-10T11:40:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6827/25094", 2021-02-10T11:40:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6823/25094", 2021-02-10T11:40:33.502 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:40:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:40:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:40:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:40:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "0d9ef9b1-98aa-4d37-aa16-6305c7d46d52", 2021-02-10T11:40:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:40:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:40:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:40:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:40:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:40:33.503 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:40:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:40:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 101, 2021-02-10T11:40:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:40:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:40:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:40:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6824", 2021-02-10T11:40:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:40:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.504 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6826", 2021-02-10T11:40:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:40:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:40:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6828", 2021-02-10T11:40:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:40:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.505 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6829", 2021-02-10T11:40:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:40:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:40:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6832", 2021-02-10T11:40:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:40:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.506 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6833", 2021-02-10T11:40:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:40:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:40:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6830", 2021-02-10T11:40:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:40:33.507 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.508 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.508 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.508 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6831", 2021-02-10T11:40:33.508 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:40:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6826/25095", 2021-02-10T11:40:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6829/25095", 2021-02-10T11:40:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6833/25095", 2021-02-10T11:40:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6831/25095", 2021-02-10T11:40:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:40:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:40:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:40:33.509 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:40:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "36742c21-bfeb-4e11-bce9-8e745378b605", 2021-02-10T11:40:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:40:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:40:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:40:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:40:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:40:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:40:33.510 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:40:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 92, 2021-02-10T11:40:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:40:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:40:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:40:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6816", 2021-02-10T11:40:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:40:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6818", 2021-02-10T11:40:33.511 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:40:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:40:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6820", 2021-02-10T11:40:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:40:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6822", 2021-02-10T11:40:33.512 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:40:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:40:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6828", 2021-02-10T11:40:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:40:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.513 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6830", 2021-02-10T11:40:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:40:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:40:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6824", 2021-02-10T11:40:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:40:33.514 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6826", 2021-02-10T11:40:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:40:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6818/36380", 2021-02-10T11:40:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6822/36380", 2021-02-10T11:40:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6830/36380", 2021-02-10T11:40:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6826/36380", 2021-02-10T11:40:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:40:33.515 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:40:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:40:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:40:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "88e694e8-fe98-4d1b-88c8-8c9907ece60b", 2021-02-10T11:40:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:40:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:40:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:40:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:40:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:40:33.516 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:40:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:40:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 101, 2021-02-10T11:40:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:40:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:40:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:40:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6808", 2021-02-10T11:40:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:40:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.517 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6809", 2021-02-10T11:40:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:40:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:40:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6810", 2021-02-10T11:40:33.518 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:40:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6811", 2021-02-10T11:40:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:40:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:40:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.519 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6814", 2021-02-10T11:40:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:40:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6815", 2021-02-10T11:40:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:40:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:40:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.520 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6812", 2021-02-10T11:40:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:40:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6813", 2021-02-10T11:40:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:40:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6809/36383", 2021-02-10T11:40:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6811/36383", 2021-02-10T11:40:33.521 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6815/36383", 2021-02-10T11:40:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6813/36383", 2021-02-10T11:40:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:40:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:40:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:40:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:40:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "87a13c6c-cf85-4aaf-916b-e73913484a49", 2021-02-10T11:40:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:40:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:40:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:40:33.522 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:40:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:40:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:40:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:40:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 101, 2021-02-10T11:40:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:40:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:40:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:40:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6800", 2021-02-10T11:40:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:40:33.523 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6801", 2021-02-10T11:40:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:40:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:40:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6802", 2021-02-10T11:40:33.524 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:40:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6803", 2021-02-10T11:40:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:40:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:40:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6806", 2021-02-10T11:40:33.525 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:40:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6807", 2021-02-10T11:40:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:40:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:40:33.526 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6804", 2021-02-10T11:40:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:40:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6805", 2021-02-10T11:40:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:40:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.527 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6801/36381", 2021-02-10T11:40:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6803/36381", 2021-02-10T11:40:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6807/36381", 2021-02-10T11:40:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6805/36381", 2021-02-10T11:40:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:40:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:40:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:40:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.528 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:40:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "7b90823f-854c-4592-a95b-a7fa98f93254", 2021-02-10T11:40:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:40:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:40:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:40:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:40:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:40:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:40:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:40:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 101, 2021-02-10T11:40:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:40:33.529 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:40:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:40:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6817", 2021-02-10T11:40:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:40:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.530 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6819", 2021-02-10T11:40:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:40:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:40:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6821", 2021-02-10T11:40:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:40:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.531 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6823", 2021-02-10T11:40:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:40:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:40:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6829", 2021-02-10T11:40:33.532 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:40:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6831", 2021-02-10T11:40:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:40:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:40:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:40:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.533 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:40:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6825", 2021-02-10T11:40:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:40:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6827", 2021-02-10T11:40:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:40:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6819/36382", 2021-02-10T11:40:33.534 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6823/36382", 2021-02-10T11:40:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6831/36382", 2021-02-10T11:40:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6827/36382", 2021-02-10T11:40:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:40:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:40:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:40:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:40:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_xinfo": [ 2021-02-10T11:40:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:40:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:40:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:40:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:40:33.535 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:40:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:40:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:40:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:40:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:40:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:40:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:40:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:40:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.536 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:40:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:40:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:40:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:40:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:40:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:40:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:40:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:40:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:40:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:40:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:40:33.537 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:40:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:40:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:40:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:40:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:40:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:40:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:40:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:40:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:40:33.538 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:40:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:40:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:40:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:40:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:40:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:40:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:40:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:40:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:40:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:40:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.539 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:40:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:40:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:40:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:40:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:40:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:40:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:40:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap": [], 2021-02-10T11:40:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap_items": [], 2021-02-10T11:40:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_temp": [], 2021-02-10T11:40:33.540 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_temp": [], 2021-02-10T11:40:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklist": {}, 2021-02-10T11:40:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profiles": { 2021-02-10T11:40:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "default": { 2021-02-10T11:40:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "crush-failure-domain": "osd", 2021-02-10T11:40:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "k": "2", 2021-02-10T11:40:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "m": "1", 2021-02-10T11:40:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "plugin": "jerasure", 2021-02-10T11:40:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "ruleset-failure-domain": "osd", 2021-02-10T11:40:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "technique": "reed_sol_van" 2021-02-10T11:40:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps_queue": [], 2021-02-10T11:40:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "new_removed_snaps": [], 2021-02-10T11:40:33.541 INFO:teuthology.orchestra.run.gibba001.stdout: "new_purged_snaps": [], 2021-02-10T11:40:33.542 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_node_flags": {}, 2021-02-10T11:40:33.542 INFO:teuthology.orchestra.run.gibba001.stdout: "device_class_flags": {} 2021-02-10T11:40:33.542 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:40:33.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.462 7fb849c38700 1 -- 172.21.2.101:0/536816346 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8240186c0 msgr2=0x7fb82401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:33.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.462 7fb849c38700 1 --2- 172.21.2.101:0/536816346 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8240186c0 0x7fb82401abc0 crc :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:33.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.462 7fb849c38700 1 -- 172.21.2.101:0/536816346 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8441947b0 msgr2=0x7fb8440a69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:33.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.462 7fb849c38700 1 --2- 172.21.2.101:0/536816346 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8441947b0 0x7fb8440a69a0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 rx=0x7fb834001430 tx=0x7fb83400a930).stop 2021-02-10T11:40:33.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.462 7fb849c38700 1 -- 172.21.2.101:0/536816346 shutdown_connections 2021-02-10T11:40:33.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.462 7fb849c38700 1 --2- 172.21.2.101:0/536816346 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8240186c0 0x7fb82401abc0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:33.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.462 7fb849c38700 1 --2- 172.21.2.101:0/536816346 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb8440a6fc0 0x7fb8441a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:33.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.462 7fb849c38700 1 --2- 172.21.2.101:0/536816346 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb8441a1bf0 0x7fb8441a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:33.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.462 7fb849c38700 1 --2- 172.21.2.101:0/536816346 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8441947b0 0x7fb8440a69a0 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:33.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.462 7fb849c38700 1 -- 172.21.2.101:0/536816346 >> 172.21.2.101:0/536816346 conn(0x7fb8441a38c0 msgr2=0x7fb8441a2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:33.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.462 7fb849c38700 1 -- 172.21.2.101:0/536816346 shutdown_connections 2021-02-10T11:40:33.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.462 7fb849c38700 1 -- 172.21.2.101:0/536816346 wait complete. 2021-02-10T11:40:33.545 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:40:33.954 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:40:33.957 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:40:33.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.952 7fd296b4c700 1 Processor -- start 2021-02-10T11:40:33.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.952 7fd296b4c700 1 -- start start 2021-02-10T11:40:33.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.952 7fd296b4c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd2900d1870 0x7fd2901b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:33.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.952 7fd296b4c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2900f0b10 con 0x7fd2900f0650 2021-02-10T11:40:33.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.953 7fd296b4c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2900d0310 con 0x7fd2901b0a10 2021-02-10T11:40:33.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.953 7fd296b4c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2900d0530 con 0x7fd2900d1870 2021-02-10T11:40:33.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.953 7fd295169700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd2900f0650 0x7fd2900cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57006/0 (socket says 172.21.2.101:57006) 2021-02-10T11:40:33.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.953 7fd295169700 1 -- 172.21.2.101:0/1630534045 learned_addr learned my addr 172.21.2.101:0/1630534045 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:33.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.953 7fd28f7fe700 1 -- 172.21.2.101:0/1630534045 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1835248882 0 0) 0x7fd288001180 con 0x7fd2900f0650 2021-02-10T11:40:33.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.953 7fd28f7fe700 1 -- 172.21.2.101:0/1630534045 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd27c002fe0 con 0x7fd2900f0650 2021-02-10T11:40:33.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.953 7fd294968700 1 --2- 172.21.2.101:0/1630534045 >> v2:172.21.2.108:3300/0 conn(0x7fd2900d1870 0x7fd2901b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:33.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.953 7fd28f7fe700 1 -- 172.21.2.101:0/1630534045 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2779195194 0 0) 0x7fd288001180 con 0x7fd2900f0650 2021-02-10T11:40:33.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.953 7fd28f7fe700 1 -- 172.21.2.101:0/1630534045 >> v2:172.21.2.108:3300/0 conn(0x7fd2900d1870 msgr2=0x7fd2901b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:33.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.953 7fd28f7fe700 1 --2- 172.21.2.101:0/1630534045 >> v2:172.21.2.108:3300/0 conn(0x7fd2900d1870 0x7fd2901b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:33.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.953 7fd28f7fe700 1 -- 172.21.2.101:0/1630534045 >> v1:172.21.2.108:6789/0 conn(0x7fd2901b0a10 legacy=0x7fd2900cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:33.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.953 7fd28f7fe700 1 -- 172.21.2.101:0/1630534045 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2900d1c70 con 0x7fd2900f0650 2021-02-10T11:40:33.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.953 7fd28f7fe700 1 -- 172.21.2.101:0/1630534045 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fd288001680 con 0x7fd2900f0650 2021-02-10T11:40:33.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.953 7fd28f7fe700 1 -- 172.21.2.101:0/1630534045 >> v1:172.21.2.101:6789/0 conn(0x7fd2900f0650 legacy=0x7fd2900cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:33.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.953 7fd28f7fe700 1 --2- 172.21.2.101:0/1630534045 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd27c003ac0 0x7fd27c005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:33.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.953 7fd28f7fe700 1 --2- 172.21.2.101:0/1630534045 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd27c006790 0x7fd27c008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:33.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.953 7fd28f7fe700 1 --2- 172.21.2.101:0/1630534045 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd27c009590 0x7fd27c00bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:33.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.953 7fd28f7fe700 1 -- 172.21.2.101:0/1630534045 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c00c300 con 0x7fd27c003ac0 2021-02-10T11:40:33.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.953 7fd28f7fe700 1 -- 172.21.2.101:0/1630534045 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c00c520 con 0x7fd27c006790 2021-02-10T11:40:33.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.953 7fd28f7fe700 1 -- 172.21.2.101:0/1630534045 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c00c740 con 0x7fd27c009590 2021-02-10T11:40:33.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.953 7fd294968700 1 --2- 172.21.2.101:0/1630534045 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd27c003ac0 0x7fd27c005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:33.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.954 7fd28ffff700 1 --2- 172.21.2.101:0/1630534045 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd27c006790 0x7fd27c008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:33.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.954 7fd295169700 1 --2- 172.21.2.101:0/1630534045 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd27c009590 0x7fd27c00bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:33.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.954 7fd294968700 1 -- 172.21.2.101:0/1630534045 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd27c009590 msgr2=0x7fd27c00bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:33.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.954 7fd294968700 1 --2- 172.21.2.101:0/1630534045 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd27c009590 0x7fd27c00bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:33.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.954 7fd294968700 1 -- 172.21.2.101:0/1630534045 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd27c006790 msgr2=0x7fd27c008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:33.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.954 7fd294968700 1 --2- 172.21.2.101:0/1630534045 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd27c006790 0x7fd27c008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:33.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.954 7fd294968700 1 -- 172.21.2.101:0/1630534045 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27c00c9e0 con 0x7fd27c003ac0 2021-02-10T11:40:33.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.954 7fd295169700 1 --2- 172.21.2.101:0/1630534045 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd27c009590 0x7fd27c00bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:40:33.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.954 7fd295169700 1 -- 172.21.2.101:0/1630534045 reap_dead start 2021-02-10T11:40:33.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.954 7fd294968700 1 --2- 172.21.2.101:0/1630534045 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd27c003ac0 0x7fd27c005ef0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 rx=0x7fd278006360 tx=0x7fd27804dc30).ready entity=mon.0 client_cookie=8b5dd4384cba5aaf server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:33.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.954 7fd28f7fe700 1 -- 172.21.2.101:0/1630534045 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd278006610 con 0x7fd27c003ac0 2021-02-10T11:40:33.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.954 7fd28f7fe700 1 -- 172.21.2.101:0/1630534045 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd278006610 con 0x7fd27c003ac0 2021-02-10T11:40:33.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.954 7fd28f7fe700 1 -- 172.21.2.101:0/1630534045 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd278000c90 con 0x7fd27c003ac0 2021-02-10T11:40:33.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.954 7fd296b4c700 1 -- 172.21.2.101:0/1630534045 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd27c003ac0 msgr2=0x7fd27c005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:33.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.954 7fd296b4c700 1 --2- 172.21.2.101:0/1630534045 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd27c003ac0 0x7fd27c005ef0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 rx=0x7fd278006360 tx=0x7fd27804dc30).stop 2021-02-10T11:40:33.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.955 7fd296b4c700 1 -- 172.21.2.101:0/1630534045 shutdown_connections 2021-02-10T11:40:33.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.955 7fd296b4c700 1 --2- 172.21.2.101:0/1630534045 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd27c003ac0 0x7fd27c005ef0 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:33.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.955 7fd296b4c700 1 -- 172.21.2.101:0/1630534045 >> 172.21.2.101:0/1630534045 conn(0x7fd2900ce450 msgr2=0x7fd2900d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:33.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.955 7fd296b4c700 1 -- 172.21.2.101:0/1630534045 shutdown_connections 2021-02-10T11:40:33.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.955 7fd296b4c700 1 -- 172.21.2.101:0/1630534045 wait complete. 2021-02-10T11:40:33.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.970 7fd296b4c700 1 Processor -- start 2021-02-10T11:40:33.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.970 7fd296b4c700 1 -- start start 2021-02-10T11:40:33.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd296b4c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2901a4a60 0x7fd2900c1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:33.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd296b4c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2900c2290 0x7fd2900c2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:33.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd296b4c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2900c2cb0 0x7fd2900bfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:33.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd296b4c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2900c31c0 con 0x7fd2900c2290 2021-02-10T11:40:33.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd296b4c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2900f05f0 con 0x7fd2900c2cb0 2021-02-10T11:40:33.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd296b4c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2900c0340 con 0x7fd2901a4a60 2021-02-10T11:40:33.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd28ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2900c2290 0x7fd2900c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:33.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd28ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2900c2290 0x7fd2900c2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56522/0 (socket says 172.21.2.101:56522) 2021-02-10T11:40:33.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd28ffff700 1 -- 172.21.2.101:0/4007510696 learned_addr learned my addr 172.21.2.101:0/4007510696 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:33.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd294968700 1 --2- 172.21.2.101:0/4007510696 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2901a4a60 0x7fd2900c1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:33.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd295169700 1 --2- 172.21.2.101:0/4007510696 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2900c2cb0 0x7fd2900bfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:33.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd28ffff700 1 -- 172.21.2.101:0/4007510696 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2901a4a60 msgr2=0x7fd2900c1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:33.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd28ffff700 1 --2- 172.21.2.101:0/4007510696 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2901a4a60 0x7fd2900c1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:33.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd28ffff700 1 -- 172.21.2.101:0/4007510696 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2900c2cb0 msgr2=0x7fd2900bfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:33.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd28ffff700 1 --2- 172.21.2.101:0/4007510696 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2900c2cb0 0x7fd2900bfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:33.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd28ffff700 1 -- 172.21.2.101:0/4007510696 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2900c0560 con 0x7fd2900c2290 2021-02-10T11:40:33.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd295169700 1 --2- 172.21.2.101:0/4007510696 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2900c2cb0 0x7fd2900bfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:40:33.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd294968700 1 --2- 172.21.2.101:0/4007510696 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2901a4a60 0x7fd2900c1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:40:33.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd28ffff700 1 --2- 172.21.2.101:0/4007510696 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2900c2290 0x7fd2900c2690 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 rx=0x7fd2800099a0 tx=0x7fd28000bcc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:33.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd28d7fa700 1 -- 172.21.2.101:0/4007510696 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd280006220 con 0x7fd2900c2290 2021-02-10T11:40:33.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd28d7fa700 1 -- 172.21.2.101:0/4007510696 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd280006520 con 0x7fd2900c2290 2021-02-10T11:40:33.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd296b4c700 1 -- 172.21.2.101:0/4007510696 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2900c0560 con 0x7fd2900c2290 2021-02-10T11:40:33.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd28d7fa700 1 -- 172.21.2.101:0/4007510696 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2800058a0 con 0x7fd2900c2290 2021-02-10T11:40:33.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.971 7fd296b4c700 1 -- 172.21.2.101:0/4007510696 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2900c0560 con 0x7fd2900c2290 2021-02-10T11:40:33.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.972 7fd28d7fa700 1 -- 172.21.2.101:0/4007510696 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd280006030 con 0x7fd2900c2290 2021-02-10T11:40:33.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.972 7fd28d7fa700 1 --2- 172.21.2.101:0/4007510696 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd274018730 0x7fd27401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:33.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.972 7fd28d7fa700 1 -- 172.21.2.101:0/4007510696 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd2800329f0 con 0x7fd2900c2290 2021-02-10T11:40:33.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.972 7fd294968700 1 --2- 172.21.2.101:0/4007510696 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd274018730 0x7fd27401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:33.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:33.972 7fd294968700 1 --2- 172.21.2.101:0/4007510696 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd274018730 0x7fd27401ac10 crc :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:34.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.002 7fd296b4c700 1 -- 172.21.2.101:0/4007510696 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2900f0b10 con 0x7fd2900c2290 2021-02-10T11:40:34.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.005 7fd28d7fa700 1 -- 172.21.2.101:0/4007510696 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd28000be00 con 0x7fd2900c2290 2021-02-10T11:40:34.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.134 7fd296b4c700 1 -- 172.21.2.101:0/4007510696 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2900f0b10 con 0x7fd2900c2290 2021-02-10T11:40:34.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.134 7fd28d7fa700 1 -- 172.21.2.101:0/4007510696 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+2107 (secure 0 0 0) 0x7fd280006520 con 0x7fd2900c2290 2021-02-10T11:40:34.138 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:40:34.138 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":80,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10125,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/3121560299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3121560299},{"type":"v1","addr":"172.21.2.101:6835","nonce":3121560299}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":76}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:39:56.054150","modified":"2021-02-10 11:39:58.060269","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10122},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10122":{"gid":10122,"name":"b","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/1690286235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":1690286235},{"type":"v1","addr":"172.21.2.108:6833","nonce":1690286235}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2021-02-10T11:40:34.138 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 80 2021-02-10T11:40:34.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.136 7fd296b4c700 1 -- 172.21.2.101:0/4007510696 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd274018730 msgr2=0x7fd27401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:34.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.136 7fd296b4c700 1 --2- 172.21.2.101:0/4007510696 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd274018730 0x7fd27401ac10 crc :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:34.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.136 7fd296b4c700 1 -- 172.21.2.101:0/4007510696 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2900c2290 msgr2=0x7fd2900c2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:34.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.136 7fd296b4c700 1 --2- 172.21.2.101:0/4007510696 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2900c2290 0x7fd2900c2690 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 rx=0x7fd2800099a0 tx=0x7fd28000bcc0).stop 2021-02-10T11:40:34.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.136 7fd296b4c700 1 -- 172.21.2.101:0/4007510696 shutdown_connections 2021-02-10T11:40:34.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.136 7fd296b4c700 1 --2- 172.21.2.101:0/4007510696 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2900c2cb0 0x7fd2900bfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:34.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.136 7fd296b4c700 1 --2- 172.21.2.101:0/4007510696 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2900c2290 0x7fd2900c2690 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:34.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.136 7fd296b4c700 1 --2- 172.21.2.101:0/4007510696 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd274018730 0x7fd27401ac10 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:34.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.136 7fd296b4c700 1 --2- 172.21.2.101:0/4007510696 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2901a4a60 0x7fd2900c1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:34.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.136 7fd296b4c700 1 -- 172.21.2.101:0/4007510696 >> 172.21.2.101:0/4007510696 conn(0x7fd2900ebe30 msgr2=0x7fd2900bf200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:34.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.136 7fd296b4c700 1 -- 172.21.2.101:0/4007510696 shutdown_connections 2021-02-10T11:40:34.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.136 7fd296b4c700 1 -- 172.21.2.101:0/4007510696 wait complete. 2021-02-10T11:40:34.151 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs fail cephfs 2021-02-10T11:40:34.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.598 7efe0664c700 1 Processor -- start 2021-02-10T11:40:34.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.598 7efe0664c700 1 -- start start 2021-02-10T11:40:34.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.599 7efe0664c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efe000d1870 0x7efe001b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:34.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.599 7efe0664c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7efe000f0650 0x7efe000cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:34.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.599 7efe0664c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe000f0b10 con 0x7efe001b0a10 2021-02-10T11:40:34.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.599 7efe0664c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe000fe230 con 0x7efe000f0650 2021-02-10T11:40:34.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.599 7efe0664c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe000e5ed0 con 0x7efe000d1870 2021-02-10T11:40:34.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.599 7efe04c69700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7efe000f0650 0x7efe000cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:34.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.599 7efe04c69700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7efe000f0650 0x7efe000cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56530/0 (socket says 172.21.2.101:56530) 2021-02-10T11:40:34.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.599 7efe04c69700 1 -- 172.21.2.101:0/528555912 learned_addr learned my addr 172.21.2.101:0/528555912 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:34.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.599 7efdff7fe700 1 --1- 172.21.2.101:0/528555912 >> v1:172.21.2.101:6789/0 conn(0x7efe001b0a10 0x7efe000cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57024/0 (socket says 172.21.2.101:57024) 2021-02-10T11:40:34.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.599 7efdfffff700 1 --2- 172.21.2.101:0/528555912 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efe000d1870 0x7efe001b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:34.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.599 7efdfeffd700 1 -- 172.21.2.101:0/528555912 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 324618707 0 0) 0x7efdf40010b0 con 0x7efe001b0a10 2021-02-10T11:40:34.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.599 7efdfeffd700 1 -- 172.21.2.101:0/528555912 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efde8002fe0 con 0x7efe001b0a10 2021-02-10T11:40:34.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.599 7efe04c69700 1 -- 172.21.2.101:0/528555912 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efe000d1870 msgr2=0x7efe001b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:34.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.599 7efe04c69700 1 --2- 172.21.2.101:0/528555912 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efe000d1870 0x7efe001b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:34.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.599 7efe04c69700 1 -- 172.21.2.101:0/528555912 >> v1:172.21.2.101:6789/0 conn(0x7efe001b0a10 legacy=0x7efe000cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:34.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.599 7efe04c69700 1 -- 172.21.2.101:0/528555912 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe000f20a0 con 0x7efe000f0650 2021-02-10T11:40:34.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.599 7efe04c69700 1 --2- 172.21.2.101:0/528555912 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe000f0650 0x7efe000cfb20 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 rx=0x7efdf8006f00 tx=0x7efdf8009f10).ready entity=mon.0 client_cookie=a71e8040ea750e67 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:34.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.599 7efdfeffd700 1 -- 172.21.2.101:0/528555912 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efdf800b3b0 con 0x7efe000f0650 2021-02-10T11:40:34.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.600 7efdfeffd700 1 -- 172.21.2.101:0/528555912 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf800b630 con 0x7efe000f0650 2021-02-10T11:40:34.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.600 7efdfeffd700 1 -- 172.21.2.101:0/528555912 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efdf800bb50 con 0x7efe000f0650 2021-02-10T11:40:34.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.600 7efe0664c700 1 -- 172.21.2.101:0/528555912 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe000f0650 msgr2=0x7efe000cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:34.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.600 7efe0664c700 1 --2- 172.21.2.101:0/528555912 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe000f0650 0x7efe000cfb20 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 rx=0x7efdf8006f00 tx=0x7efdf8009f10).stop 2021-02-10T11:40:34.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.600 7efe0664c700 1 -- 172.21.2.101:0/528555912 shutdown_connections 2021-02-10T11:40:34.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.600 7efe0664c700 1 --2- 172.21.2.101:0/528555912 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe000f0650 0x7efe000cfb20 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:34.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.600 7efe0664c700 1 --2- 172.21.2.101:0/528555912 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efe000d1870 0x7efe001b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:34.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.600 7efe0664c700 1 -- 172.21.2.101:0/528555912 >> 172.21.2.101:0/528555912 conn(0x7efe000ce450 msgr2=0x7efe000d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:34.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.600 7efe0664c700 1 -- 172.21.2.101:0/528555912 shutdown_connections 2021-02-10T11:40:34.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.600 7efe0664c700 1 -- 172.21.2.101:0/528555912 wait complete. 2021-02-10T11:40:34.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.615 7efe0664c700 1 Processor -- start 2021-02-10T11:40:34.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.615 7efe0664c700 1 -- start start 2021-02-10T11:40:34.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.615 7efe0664c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe000eee10 0x7efe000bc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:34.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.615 7efe0664c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efe000ef210 0x7efe000a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:34.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.615 7efe0664c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efe000efd70 0x7efe000f0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:34.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.615 7efe0664c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe000ef7f0 con 0x7efe000eee10 2021-02-10T11:40:34.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.615 7efe0664c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe000f0790 con 0x7efe000efd70 2021-02-10T11:40:34.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.615 7efe0664c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe000bcad0 con 0x7efe000ef210 2021-02-10T11:40:34.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.615 7efdfffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe000eee10 0x7efe000bc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:34.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.615 7efdfffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe000eee10 0x7efe000bc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56538/0 (socket says 172.21.2.101:56538) 2021-02-10T11:40:34.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.615 7efdfffff700 1 -- 172.21.2.101:0/1951656007 learned_addr learned my addr 172.21.2.101:0/1951656007 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:34.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.616 7efdfffff700 1 -- 172.21.2.101:0/1951656007 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efe000ef210 msgr2=0x7efe000a5640 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:34.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.616 7efdfffff700 1 --2- 172.21.2.101:0/1951656007 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efe000ef210 0x7efe000a5640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:34.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.616 7efdfffff700 1 -- 172.21.2.101:0/1951656007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efe000efd70 msgr2=0x7efe000f0170 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:34.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.616 7efdfffff700 1 --2- 172.21.2.101:0/1951656007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efe000efd70 0x7efe000f0170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:34.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.616 7efdfffff700 1 -- 172.21.2.101:0/1951656007 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe000a5c60 con 0x7efe000eee10 2021-02-10T11:40:34.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.616 7efdfffff700 1 --2- 172.21.2.101:0/1951656007 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe000eee10 0x7efe000bc4b0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 rx=0x7efdf0006f60 tx=0x7efdf000b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:34.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.616 7efdfcff9700 1 -- 172.21.2.101:0/1951656007 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efdf000bd60 con 0x7efe000eee10 2021-02-10T11:40:34.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.616 7efdfcff9700 1 -- 172.21.2.101:0/1951656007 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf0000c90 con 0x7efe000eee10 2021-02-10T11:40:34.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.616 7efe0664c700 1 -- 172.21.2.101:0/1951656007 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe000a5c60 con 0x7efe000eee10 2021-02-10T11:40:34.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.616 7efe0664c700 1 -- 172.21.2.101:0/1951656007 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe000a46c0 con 0x7efe000eee10 2021-02-10T11:40:34.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.616 7efdfcff9700 1 -- 172.21.2.101:0/1951656007 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efdf00022b0 con 0x7efe000eee10 2021-02-10T11:40:34.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.616 7efdfcff9700 1 -- 172.21.2.101:0/1951656007 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7efdf0006b00 con 0x7efe000eee10 2021-02-10T11:40:34.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.617 7efdfcff9700 1 --2- 172.21.2.101:0/1951656007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efde00186d0 0x7efde001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:34.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.617 7efdfcff9700 1 -- 172.21.2.101:0/1951656007 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5573+0+0 (secure 0 0 0) 0x7efdf00327b0 con 0x7efe000eee10 2021-02-10T11:40:34.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.617 7efdff7fe700 1 --2- 172.21.2.101:0/1951656007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efde00186d0 0x7efde001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:34.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.617 7efdff7fe700 1 --2- 172.21.2.101:0/1951656007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efde00186d0 0x7efde001abd0 crc :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:34.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.649 7efe0664c700 1 -- 172.21.2.101:0/1951656007 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe000ebc50 con 0x7efe000eee10 2021-02-10T11:40:34.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.651 7efdfcff9700 1 -- 172.21.2.101:0/1951656007 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7efdf0000960 con 0x7efe000eee10 2021-02-10T11:40:34.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:34.778 7efe0664c700 1 -- 172.21.2.101:0/1951656007 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7efe000ebc50 con 0x7efe000eee10 2021-02-10T11:40:35.561 INFO:tasks.ceph.mon.a.gibba001.stderr:2021-02-10 11:40:35.558 7f974d04d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-02-10T11:40:35.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:35.563 7efdfcff9700 1 -- 172.21.2.101:0/1951656007 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v81) v1 ==== 161+0+0 (secure 0 0 0) 0x7efdf0006b00 con 0x7efe000eee10 2021-02-10T11:40:35.600 INFO:teuthology.orchestra.run.gibba001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-02-10T11:40:35.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:35.599 7efe0664c700 1 -- 172.21.2.101:0/1951656007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efde00186d0 msgr2=0x7efde001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:35.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:35.599 7efe0664c700 1 --2- 172.21.2.101:0/1951656007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efde00186d0 0x7efde001abd0 crc :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:35.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:35.599 7efe0664c700 1 -- 172.21.2.101:0/1951656007 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe000eee10 msgr2=0x7efe000bc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:35.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:35.599 7efe0664c700 1 --2- 172.21.2.101:0/1951656007 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe000eee10 0x7efe000bc4b0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 rx=0x7efdf0006f60 tx=0x7efdf000b970).stop 2021-02-10T11:40:35.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:35.599 7efe0664c700 1 -- 172.21.2.101:0/1951656007 shutdown_connections 2021-02-10T11:40:35.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:35.599 7efe0664c700 1 --2- 172.21.2.101:0/1951656007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efde00186d0 0x7efde001abd0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:35.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:35.599 7efe0664c700 1 --2- 172.21.2.101:0/1951656007 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efe000ef210 0x7efe000a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:35.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:35.599 7efe0664c700 1 --2- 172.21.2.101:0/1951656007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efe000efd70 0x7efe000f0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:35.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:35.599 7efe0664c700 1 --2- 172.21.2.101:0/1951656007 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efe000eee10 0x7efe000bc4b0 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:35.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:35.599 7efe0664c700 1 -- 172.21.2.101:0/1951656007 >> 172.21.2.101:0/1951656007 conn(0x7efe000bc090 msgr2=0x7efe00091bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:35.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:35.599 7efe0664c700 1 -- 172.21.2.101:0/1951656007 shutdown_connections 2021-02-10T11:40:35.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:35.599 7efe0664c700 1 -- 172.21.2.101:0/1951656007 wait complete. 2021-02-10T11:40:35.610 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:40:35.616 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:40:36.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.019 7fbed00ee700 1 Processor -- start 2021-02-10T11:40:36.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.019 7fbed00ee700 1 -- start start 2021-02-10T11:40:36.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.019 7fbed00ee700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbec80d1870 0x7fbec81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:36.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.019 7fbed00ee700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbec81b0a10 0x7fbec80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:36.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.019 7fbed00ee700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec80f0b10 con 0x7fbec80d1c70 2021-02-10T11:40:36.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.019 7fbed00ee700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec80e5ed0 con 0x7fbec80d1870 2021-02-10T11:40:36.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.019 7fbed00ee700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec80f0650 con 0x7fbec81b0a10 2021-02-10T11:40:36.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.020 7fbece70b700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fbec80d1c70 0x7fbec80cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57036/0 (socket says 172.21.2.101:57036) 2021-02-10T11:40:36.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.020 7fbece70b700 1 -- 172.21.2.101:0/2785489894 learned_addr learned my addr 172.21.2.101:0/2785489894 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:36.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.020 7fbecdf0a700 1 --2- 172.21.2.101:0/2785489894 >> v2:172.21.2.101:3300/0 conn(0x7fbec80d1870 0x7fbec81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:36.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.020 7fbecd709700 1 --2- 172.21.2.101:0/2785489894 >> v2:172.21.2.108:3300/0 conn(0x7fbec81b0a10 0x7fbec80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:36.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.020 7fbecdf0a700 1 -- 172.21.2.101:0/2785489894 >> v2:172.21.2.108:3300/0 conn(0x7fbec81b0a10 msgr2=0x7fbec80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:36.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.020 7fbecdf0a700 1 --2- 172.21.2.101:0/2785489894 >> v2:172.21.2.108:3300/0 conn(0x7fbec81b0a10 0x7fbec80cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:36.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.020 7fbecdf0a700 1 -- 172.21.2.101:0/2785489894 >> v1:172.21.2.101:6789/0 conn(0x7fbec80d1c70 legacy=0x7fbec80cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:36.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.020 7fbeccf08700 1 -- 172.21.2.101:0/2785489894 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208081932 0 0) 0x7fbec0001500 con 0x7fbec80d1c70 2021-02-10T11:40:36.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.020 7fbecdf0a700 1 -- 172.21.2.101:0/2785489894 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec80f0840 con 0x7fbec80d1870 2021-02-10T11:40:36.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.020 7fbecdf0a700 1 --2- 172.21.2.101:0/2785489894 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbec80d1870 0x7fbec81b03f0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 rx=0x7fbebc006650 tx=0x7fbebc009a00).ready entity=mon.0 client_cookie=8f47a82295c29ffc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:36.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.020 7fbeccf08700 1 -- 172.21.2.101:0/2785489894 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbebc009df0 con 0x7fbec80d1870 2021-02-10T11:40:36.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.021 7fbeccf08700 1 -- 172.21.2.101:0/2785489894 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbebc0009b0 con 0x7fbec80d1870 2021-02-10T11:40:36.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.021 7fbeccf08700 1 -- 172.21.2.101:0/2785489894 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbebc00c2b0 con 0x7fbec80d1870 2021-02-10T11:40:36.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.021 7fbed00ee700 1 -- 172.21.2.101:0/2785489894 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbec80d1870 msgr2=0x7fbec81b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:36.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.021 7fbed00ee700 1 --2- 172.21.2.101:0/2785489894 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbec80d1870 0x7fbec81b03f0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 rx=0x7fbebc006650 tx=0x7fbebc009a00).stop 2021-02-10T11:40:36.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.021 7fbed00ee700 1 -- 172.21.2.101:0/2785489894 shutdown_connections 2021-02-10T11:40:36.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.021 7fbed00ee700 1 --2- 172.21.2.101:0/2785489894 >> v2:172.21.2.108:3300/0 conn(0x7fbec81b0a10 0x7fbec80cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:36.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.021 7fbed00ee700 1 --2- 172.21.2.101:0/2785489894 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbec80d1870 0x7fbec81b03f0 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:36.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.021 7fbed00ee700 1 -- 172.21.2.101:0/2785489894 >> 172.21.2.101:0/2785489894 conn(0x7fbec80ce450 msgr2=0x7fbec80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:36.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.021 7fbed00ee700 1 -- 172.21.2.101:0/2785489894 shutdown_connections 2021-02-10T11:40:36.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.021 7fbed00ee700 1 -- 172.21.2.101:0/2785489894 wait complete. 2021-02-10T11:40:36.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.036 7fbed00ee700 1 Processor -- start 2021-02-10T11:40:36.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.036 7fbed00ee700 1 -- start start 2021-02-10T11:40:36.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.037 7fbed00ee700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbec818f610 0x7fbec818fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:36.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.037 7fbed00ee700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbec818b1d0 0x7fbec818b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:36.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.037 7fbed00ee700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbec818c3e0 0x7fbec818c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:36.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.037 7fbed00ee700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec818be60 con 0x7fbec818c3e0 2021-02-10T11:40:36.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.037 7fbed00ee700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec818fa10 con 0x7fbec818b1d0 2021-02-10T11:40:36.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.037 7fbed00ee700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec81902e0 con 0x7fbec818f610 2021-02-10T11:40:36.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.037 7fbece70b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbec818c3e0 0x7fbec818c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:36.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.037 7fbece70b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbec818c3e0 0x7fbec818c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56550/0 (socket says 172.21.2.101:56550) 2021-02-10T11:40:36.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.037 7fbece70b700 1 -- 172.21.2.101:0/1154576760 learned_addr learned my addr 172.21.2.101:0/1154576760 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:36.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.037 7fbece70b700 1 -- 172.21.2.101:0/1154576760 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbec818f610 msgr2=0x7fbec818fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:36.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.037 7fbece70b700 1 --2- 172.21.2.101:0/1154576760 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbec818f610 0x7fbec818fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:36.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.037 7fbece70b700 1 -- 172.21.2.101:0/1154576760 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbec818b1d0 msgr2=0x7fbec818b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:36.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.037 7fbece70b700 1 --2- 172.21.2.101:0/1154576760 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbec818b1d0 0x7fbec818b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:36.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.037 7fbece70b700 1 -- 172.21.2.101:0/1154576760 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec81906a0 con 0x7fbec818c3e0 2021-02-10T11:40:36.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.037 7fbece70b700 1 --2- 172.21.2.101:0/1154576760 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbec818c3e0 0x7fbec818c7e0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 rx=0x7fbec0001bf0 tx=0x7fbec000abd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:36.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.037 7fbeba7fc700 1 -- 172.21.2.101:0/1154576760 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbec00070d0 con 0x7fbec818c3e0 2021-02-10T11:40:36.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.038 7fbeba7fc700 1 -- 172.21.2.101:0/1154576760 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec00073d0 con 0x7fbec818c3e0 2021-02-10T11:40:36.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.038 7fbed00ee700 1 -- 172.21.2.101:0/1154576760 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec81906a0 con 0x7fbec818c3e0 2021-02-10T11:40:36.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.038 7fbeba7fc700 1 -- 172.21.2.101:0/1154576760 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbec00049a0 con 0x7fbec818c3e0 2021-02-10T11:40:36.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.038 7fbed00ee700 1 -- 172.21.2.101:0/1154576760 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec81906a0 con 0x7fbec818c3e0 2021-02-10T11:40:36.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.038 7fbeba7fc700 1 -- 172.21.2.101:0/1154576760 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbec0004720 con 0x7fbec818c3e0 2021-02-10T11:40:36.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.038 7fbeba7fc700 1 --2- 172.21.2.101:0/1154576760 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbeb0018730 0x7fbeb001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:36.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.038 7fbeba7fc700 1 -- 172.21.2.101:0/1154576760 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fbec0032b70 con 0x7fbec818c3e0 2021-02-10T11:40:36.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.039 7fbecdf0a700 1 --2- 172.21.2.101:0/1154576760 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbeb0018730 0x7fbeb001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:36.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.039 7fbecdf0a700 1 --2- 172.21.2.101:0/1154576760 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbeb0018730 0x7fbeb001ac10 crc :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:36.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.069 7fbed00ee700 1 -- 172.21.2.101:0/1154576760 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec818d750 con 0x7fbec818c3e0 2021-02-10T11:40:36.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.072 7fbeba7fc700 1 -- 172.21.2.101:0/1154576760 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbec0000cc0 con 0x7fbec818c3e0 2021-02-10T11:40:36.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.200 7fbed00ee700 1 -- 172.21.2.101:0/1154576760 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbec818d750 con 0x7fbec818c3e0 2021-02-10T11:40:36.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.200 7fbeba7fc700 1 -- 172.21.2.101:0/1154576760 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+1760 (secure 0 0 0) 0x7fbec00073d0 con 0x7fbec818c3e0 2021-02-10T11:40:36.204 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:40:36.204 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":81,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10125,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/3121560299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3121560299},{"type":"v1","addr":"172.21.2.101:6835","nonce":3121560299}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:39:56.054150","modified":"2021-02-10 11:40:35.559247","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2021-02-10T11:40:36.204 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 81 2021-02-10T11:40:36.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.202 7fbed00ee700 1 -- 172.21.2.101:0/1154576760 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbeb0018730 msgr2=0x7fbeb001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:36.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.202 7fbed00ee700 1 --2- 172.21.2.101:0/1154576760 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbeb0018730 0x7fbeb001ac10 crc :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:36.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.202 7fbed00ee700 1 -- 172.21.2.101:0/1154576760 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbec818c3e0 msgr2=0x7fbec818c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:36.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.202 7fbed00ee700 1 --2- 172.21.2.101:0/1154576760 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbec818c3e0 0x7fbec818c7e0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 rx=0x7fbec0001bf0 tx=0x7fbec000abd0).stop 2021-02-10T11:40:36.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.202 7fbed00ee700 1 -- 172.21.2.101:0/1154576760 shutdown_connections 2021-02-10T11:40:36.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.202 7fbed00ee700 1 --2- 172.21.2.101:0/1154576760 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbec818c3e0 0x7fbec818c7e0 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:36.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.202 7fbed00ee700 1 --2- 172.21.2.101:0/1154576760 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbec818b1d0 0x7fbec818b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:36.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.202 7fbed00ee700 1 --2- 172.21.2.101:0/1154576760 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbeb0018730 0x7fbeb001ac10 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:36.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.202 7fbed00ee700 1 --2- 172.21.2.101:0/1154576760 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbec818f610 0x7fbec818fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:36.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.202 7fbed00ee700 1 -- 172.21.2.101:0/1154576760 >> 172.21.2.101:0/1154576760 conn(0x7fbec8097ac0 msgr2=0x7fbec80c2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:36.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.202 7fbed00ee700 1 -- 172.21.2.101:0/1154576760 shutdown_connections 2021-02-10T11:40:36.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.203 7fbed00ee700 1 -- 172.21.2.101:0/1154576760 wait complete. 2021-02-10T11:40:36.214 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-02-10T11:40:36.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.627 7f0ad80f5700 1 Processor -- start 2021-02-10T11:40:36.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.627 7f0ad80f5700 1 -- start start 2021-02-10T11:40:36.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.627 7f0ad80f5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0ad00d1870 0x7f0ad01b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:36.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.627 7f0ad80f5700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0ad00f0650 0x7f0ad00cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:36.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.627 7f0ad80f5700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ad00f0b10 con 0x7f0ad01b0a10 2021-02-10T11:40:36.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.627 7f0ad80f5700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ad00fe230 con 0x7f0ad00f0650 2021-02-10T11:40:36.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.627 7f0ad80f5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad00e5ed0 con 0x7f0ad00d1870 2021-02-10T11:40:36.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.627 7f0ad6712700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0ad00f0650 0x7f0ad00cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:36.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.628 7f0ad6712700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0ad00f0650 0x7f0ad00cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56558/0 (socket says 172.21.2.101:56558) 2021-02-10T11:40:36.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.628 7f0ad6712700 1 -- 172.21.2.101:0/3388726848 learned_addr learned my addr 172.21.2.101:0/3388726848 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:36.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.628 7f0ad6712700 1 -- 172.21.2.101:0/3388726848 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0ad00d1870 msgr2=0x7f0ad01b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:36.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.628 7f0ad6712700 1 --2- 172.21.2.101:0/3388726848 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0ad00d1870 0x7f0ad01b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:36.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.628 7f0ad6712700 1 -- 172.21.2.101:0/3388726848 >> v1:172.21.2.108:6789/0 conn(0x7f0ad01b0a10 legacy=0x7f0ad00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:36.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.628 7f0ad6712700 1 -- 172.21.2.101:0/3388726848 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad00f20a0 con 0x7f0ad00f0650 2021-02-10T11:40:36.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.628 7f0ad6712700 1 --2- 172.21.2.101:0/3388726848 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0ad00f0650 0x7f0ad00cfb20 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 rx=0x7f0acc006b80 tx=0x7f0acc009b90).ready entity=mon.0 client_cookie=331ff99fa3fb7e58 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:36.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.628 7f0ad4f0f700 1 -- 172.21.2.101:0/3388726848 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0acc0022e0 con 0x7f0ad00f0650 2021-02-10T11:40:36.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.628 7f0ad4f0f700 1 -- 172.21.2.101:0/3388726848 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0acc002560 con 0x7f0ad00f0650 2021-02-10T11:40:36.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.628 7f0ad4f0f700 1 -- 172.21.2.101:0/3388726848 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0acc002a80 con 0x7f0ad00f0650 2021-02-10T11:40:36.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.628 7f0ad80f5700 1 -- 172.21.2.101:0/3388726848 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0ad00f0650 msgr2=0x7f0ad00cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:36.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.628 7f0ad80f5700 1 --2- 172.21.2.101:0/3388726848 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0ad00f0650 0x7f0ad00cfb20 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 rx=0x7f0acc006b80 tx=0x7f0acc009b90).stop 2021-02-10T11:40:36.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.629 7f0ad80f5700 1 -- 172.21.2.101:0/3388726848 shutdown_connections 2021-02-10T11:40:36.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.629 7f0ad80f5700 1 --2- 172.21.2.101:0/3388726848 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0ad00f0650 0x7f0ad00cfb20 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:36.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.629 7f0ad80f5700 1 --2- 172.21.2.101:0/3388726848 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0ad00d1870 0x7f0ad01b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:36.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.629 7f0ad80f5700 1 -- 172.21.2.101:0/3388726848 >> 172.21.2.101:0/3388726848 conn(0x7f0ad00ce450 msgr2=0x7f0ad00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:36.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.629 7f0ad80f5700 1 -- 172.21.2.101:0/3388726848 shutdown_connections 2021-02-10T11:40:36.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.629 7f0ad80f5700 1 -- 172.21.2.101:0/3388726848 wait complete. 2021-02-10T11:40:36.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.644 7f0ad80f5700 1 Processor -- start 2021-02-10T11:40:36.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.644 7f0ad80f5700 1 -- start start 2021-02-10T11:40:36.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.645 7f0ad80f5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0ad00a2fb0 0x7f0ad00a33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:36.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.645 7f0ad80f5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0ad00a39d0 0x7f0ad00a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:36.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.645 7f0ad80f5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0ad00a3dd0 0x7f0ad00a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:36.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.645 7f0ad80f5700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad00a2620 con 0x7f0ad00a2fb0 2021-02-10T11:40:36.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.645 7f0ad80f5700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad00a2840 con 0x7f0ad00a3dd0 2021-02-10T11:40:36.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.645 7f0ad80f5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad00a2b80 con 0x7f0ad00a39d0 2021-02-10T11:40:36.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.645 7f0ad5f11700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0ad00a2fb0 0x7f0ad00a33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:36.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.645 7f0ad5f11700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0ad00a2fb0 0x7f0ad00a33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56564/0 (socket says 172.21.2.101:56564) 2021-02-10T11:40:36.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.645 7f0ad5f11700 1 -- 172.21.2.101:0/3058096642 learned_addr learned my addr 172.21.2.101:0/3058096642 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:36.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.645 7f0ad5f11700 1 -- 172.21.2.101:0/3058096642 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0ad00a39d0 msgr2=0x7f0ad00a4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:36.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.645 7f0ad5f11700 1 --2- 172.21.2.101:0/3058096642 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0ad00a39d0 0x7f0ad00a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:36.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.645 7f0ad5f11700 1 -- 172.21.2.101:0/3058096642 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0ad00a3dd0 msgr2=0x7f0ad00a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:36.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.645 7f0ad5f11700 1 --2- 172.21.2.101:0/3058096642 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0ad00a3dd0 0x7f0ad00a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:36.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.645 7f0ad5f11700 1 -- 172.21.2.101:0/3058096642 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad00a54a0 con 0x7f0ad00a2fb0 2021-02-10T11:40:36.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.645 7f0ad5f11700 1 --2- 172.21.2.101:0/3058096642 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0ad00a2fb0 0x7f0ad00a33b0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 rx=0x7f0ac800b8c0 tx=0x7f0ac800bc60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:36.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.646 7f0ac67fc700 1 -- 172.21.2.101:0/3058096642 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0ac800c2f0 con 0x7f0ad00a2fb0 2021-02-10T11:40:36.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.646 7f0ac67fc700 1 -- 172.21.2.101:0/3058096642 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac800c570 con 0x7f0ad00a2fb0 2021-02-10T11:40:36.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.646 7f0ad80f5700 1 -- 172.21.2.101:0/3058096642 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad00a54a0 con 0x7f0ad00a2fb0 2021-02-10T11:40:36.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.646 7f0ac67fc700 1 -- 172.21.2.101:0/3058096642 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0ac800ca90 con 0x7f0ad00a2fb0 2021-02-10T11:40:36.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.646 7f0ad80f5700 1 -- 172.21.2.101:0/3058096642 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad00a54a0 con 0x7f0ad00a2fb0 2021-02-10T11:40:36.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.646 7f0ac67fc700 1 -- 172.21.2.101:0/3058096642 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0ac8001010 con 0x7f0ad00a2fb0 2021-02-10T11:40:36.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.646 7f0ac67fc700 1 --2- 172.21.2.101:0/3058096642 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0ab8018700 0x7f0ab801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:36.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.646 7f0ac67fc700 1 -- 172.21.2.101:0/3058096642 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f0ac8032ba0 con 0x7f0ad00a2fb0 2021-02-10T11:40:36.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.646 7f0ad5710700 1 --2- 172.21.2.101:0/3058096642 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0ab8018700 0x7f0ab801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:36.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.647 7f0ad5710700 1 --2- 172.21.2.101:0/3058096642 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0ab8018700 0x7f0ab801abc0 crc :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:36.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.677 7f0ad80f5700 1 -- 172.21.2.101:0/3058096642 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad00ee060 con 0x7f0ad00a2fb0 2021-02-10T11:40:36.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.679 7f0ac67fc700 1 -- 172.21.2.101:0/3058096642 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0ac800bda0 con 0x7f0ad00a2fb0 2021-02-10T11:40:36.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:36.810 7f0ad80f5700 1 -- 172.21.2.101:0/3058096642 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0ad00ce450 con 0x7f0ad00a2fb0 2021-02-10T11:40:37.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:37.572 7f0ac67fc700 1 -- 172.21.2.101:0/3058096642 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v82) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0ac8001010 con 0x7f0ad00a2fb0 2021-02-10T11:40:37.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:37.580 7f0ad80f5700 1 -- 172.21.2.101:0/3058096642 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0ab8018700 msgr2=0x7f0ab801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:37.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:37.580 7f0ad80f5700 1 --2- 172.21.2.101:0/3058096642 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0ab8018700 0x7f0ab801abc0 crc :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:37.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:37.580 7f0ad80f5700 1 -- 172.21.2.101:0/3058096642 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0ad00a2fb0 msgr2=0x7f0ad00a33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:37.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:37.580 7f0ad80f5700 1 --2- 172.21.2.101:0/3058096642 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0ad00a2fb0 0x7f0ad00a33b0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 rx=0x7f0ac800b8c0 tx=0x7f0ac800bc60).stop 2021-02-10T11:40:37.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:37.580 7f0ad80f5700 1 -- 172.21.2.101:0/3058096642 shutdown_connections 2021-02-10T11:40:37.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:37.580 7f0ad80f5700 1 --2- 172.21.2.101:0/3058096642 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0ab8018700 0x7f0ab801abc0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:37.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:37.580 7f0ad80f5700 1 --2- 172.21.2.101:0/3058096642 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0ad00a39d0 0x7f0ad00a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:37.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:37.580 7f0ad80f5700 1 --2- 172.21.2.101:0/3058096642 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0ad00a3dd0 0x7f0ad00a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:37.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:37.580 7f0ad80f5700 1 --2- 172.21.2.101:0/3058096642 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0ad00a2fb0 0x7f0ad00a33b0 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:37.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:37.580 7f0ad80f5700 1 -- 172.21.2.101:0/3058096642 >> 172.21.2.101:0/3058096642 conn(0x7f0ad008f030 msgr2=0x7f0ad00b5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:37.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:37.580 7f0ad80f5700 1 -- 172.21.2.101:0/3058096642 shutdown_connections 2021-02-10T11:40:37.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:37.580 7f0ad80f5700 1 -- 172.21.2.101:0/3058096642 wait complete. 2021-02-10T11:40:37.591 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-02-10T11:40:38.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.005 7f146c1f4700 1 Processor -- start 2021-02-10T11:40:38.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.005 7f146c1f4700 1 -- start start 2021-02-10T11:40:38.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.006 7f146c1f4700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f14640d1880 0x7f14641b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:38.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.006 7f146c1f4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14640cff80 0x7f1464109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:38.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.006 7f146c1f4700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1464109a70 con 0x7f14641b0af0 2021-02-10T11:40:38.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.006 7f146c1f4700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1464109760 con 0x7f14640d1880 2021-02-10T11:40:38.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.006 7f146c1f4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1464088070 con 0x7f14640cff80 2021-02-10T11:40:38.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.007 7f146a811700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14640cff80 0x7f1464109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:38.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.007 7f146a010700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f14640d1880 0x7f14641b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:38.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.007 7f146980f700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f14641b0af0 0x7f1464108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:50472/0 (socket says 172.21.2.101:50472) 2021-02-10T11:40:38.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.007 7f146980f700 1 -- 172.21.2.101:0/1220751577 learned_addr learned my addr 172.21.2.101:0/1220751577 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:38.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.008 7f146a811700 1 -- 172.21.2.101:0/1220751577 >> v2:172.21.2.108:3300/0 conn(0x7f14640d1880 msgr2=0x7f14641b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:38.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.008 7f146a811700 1 --2- 172.21.2.101:0/1220751577 >> v2:172.21.2.108:3300/0 conn(0x7f14640d1880 0x7f14641b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:38.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.008 7f146a811700 1 -- 172.21.2.101:0/1220751577 >> v1:172.21.2.108:6789/0 conn(0x7f14641b0af0 legacy=0x7f1464108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:38.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.008 7f146a811700 1 -- 172.21.2.101:0/1220751577 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14640d1c80 con 0x7f14640cff80 2021-02-10T11:40:38.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.008 7f146a010700 1 --2- 172.21.2.101:0/1220751577 >> v2:172.21.2.108:3300/0 conn(0x7f14640d1880 0x7f14641b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:40:38.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.008 7f146a811700 1 --2- 172.21.2.101:0/1220751577 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14640cff80 0x7f1464109140 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f145c006e50 tx=0x7f145c009e70).ready entity=mon.2 client_cookie=227233d9e58bf000 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:38.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.009 7f146900e700 1 -- 172.21.2.101:0/1220751577 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f145c00b510 con 0x7f14640cff80 2021-02-10T11:40:38.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.009 7f146900e700 1 -- 172.21.2.101:0/1220751577 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145c006790 con 0x7f14640cff80 2021-02-10T11:40:38.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.009 7f146900e700 1 -- 172.21.2.101:0/1220751577 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f145c00b940 con 0x7f14640cff80 2021-02-10T11:40:38.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.009 7f146c1f4700 1 -- 172.21.2.101:0/1220751577 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14640cff80 msgr2=0x7f1464109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:38.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.009 7f146c1f4700 1 --2- 172.21.2.101:0/1220751577 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14640cff80 0x7f1464109140 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f145c006e50 tx=0x7f145c009e70).stop 2021-02-10T11:40:38.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.010 7f146c1f4700 1 -- 172.21.2.101:0/1220751577 shutdown_connections 2021-02-10T11:40:38.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.010 7f146c1f4700 1 --2- 172.21.2.101:0/1220751577 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14640cff80 0x7f1464109140 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:38.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.010 7f146c1f4700 1 --2- 172.21.2.101:0/1220751577 >> v2:172.21.2.108:3300/0 conn(0x7f14640d1880 0x7f14641b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:38.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.010 7f146c1f4700 1 -- 172.21.2.101:0/1220751577 >> 172.21.2.101:0/1220751577 conn(0x7f14640ce430 msgr2=0x7f14640d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:38.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.010 7f146c1f4700 1 -- 172.21.2.101:0/1220751577 shutdown_connections 2021-02-10T11:40:38.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.010 7f146c1f4700 1 -- 172.21.2.101:0/1220751577 wait complete. 2021-02-10T11:40:38.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.024 7f146c1f4700 1 Processor -- start 2021-02-10T11:40:38.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.024 7f146c1f4700 1 -- start start 2021-02-10T11:40:38.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.024 7f146c1f4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14641b58d0 0x7f14640aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:38.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.024 7f146c1f4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14640aaaf0 0x7f14641b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:38.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.024 7f146c1f4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14641b86c0 0x7f146419de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:38.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.024 7f146c1f4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14641b8ac0 con 0x7f14641b86c0 2021-02-10T11:40:38.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.024 7f146c1f4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14641b8e20 con 0x7f14641b58d0 2021-02-10T11:40:38.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.024 7f146c1f4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14641b5cd0 con 0x7f14640aaaf0 2021-02-10T11:40:38.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.025 7f146a811700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14641b86c0 0x7f146419de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:38.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.025 7f146a010700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14641b58d0 0x7f14640aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:38.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.025 7f146a811700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14641b86c0 0x7f146419de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56570/0 (socket says 172.21.2.101:56570) 2021-02-10T11:40:38.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.025 7f146a811700 1 -- 172.21.2.101:0/173313543 learned_addr learned my addr 172.21.2.101:0/173313543 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:38.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.025 7f146980f700 1 --2- 172.21.2.101:0/173313543 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14640aaaf0 0x7f14641b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:38.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.025 7f146a811700 1 -- 172.21.2.101:0/173313543 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14640aaaf0 msgr2=0x7f14641b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:38.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.025 7f146a811700 1 --2- 172.21.2.101:0/173313543 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14640aaaf0 0x7f14641b80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:38.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.025 7f146a811700 1 -- 172.21.2.101:0/173313543 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14641b58d0 msgr2=0x7f14640aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:38.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.025 7f146a811700 1 --2- 172.21.2.101:0/173313543 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14641b58d0 0x7f14640aa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:38.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.025 7f146a811700 1 -- 172.21.2.101:0/173313543 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1464095710 con 0x7f14641b86c0 2021-02-10T11:40:38.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.025 7f146a811700 1 --2- 172.21.2.101:0/173313543 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14641b86c0 0x7f146419de70 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 rx=0x7f145c006c50 tx=0x7f145c00bd20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:38.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.025 7f1456ffd700 1 -- 172.21.2.101:0/173313543 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f145c0093d0 con 0x7f14641b86c0 2021-02-10T11:40:38.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.025 7f1456ffd700 1 -- 172.21.2.101:0/173313543 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145c00be60 con 0x7f14641b86c0 2021-02-10T11:40:38.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.025 7f146c1f4700 1 -- 172.21.2.101:0/173313543 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1464095710 con 0x7f14641b86c0 2021-02-10T11:40:38.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.025 7f146c1f4700 1 -- 172.21.2.101:0/173313543 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146419e780 con 0x7f14641b86c0 2021-02-10T11:40:38.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.025 7f1456ffd700 1 -- 172.21.2.101:0/173313543 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f145c00c570 con 0x7f14641b86c0 2021-02-10T11:40:38.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.026 7f1456ffd700 1 -- 172.21.2.101:0/173313543 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f145c0093d0 con 0x7f14641b86c0 2021-02-10T11:40:38.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.026 7f1456ffd700 1 --2- 172.21.2.101:0/173313543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f144c018730 0x7f144c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:38.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.026 7f1456ffd700 1 -- 172.21.2.101:0/173313543 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f145c032a00 con 0x7f14641b86c0 2021-02-10T11:40:38.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.026 7f146a010700 1 --2- 172.21.2.101:0/173313543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f144c018730 0x7f144c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:38.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.026 7f146a010700 1 --2- 172.21.2.101:0/173313543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f144c018730 0x7f144c01ac10 crc :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:38.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.056 7f146c1f4700 1 -- 172.21.2.101:0/173313543 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1464082aa0 con 0x7f14641b86c0 2021-02-10T11:40:38.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.058 7f1456ffd700 1 -- 172.21.2.101:0/173313543 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f145c002ef0 con 0x7f14641b86c0 2021-02-10T11:40:38.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.186 7f146c1f4700 1 -- 172.21.2.101:0/173313543 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1464082aa0 con 0x7f14641b86c0 2021-02-10T11:40:38.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.576 7f1456ffd700 1 -- 172.21.2.101:0/173313543 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) v1 ==== 185+0+0 (secure 0 0 0) 0x7f145c0093d0 con 0x7f14641b86c0 2021-02-10T11:40:38.606 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' removed 2021-02-10T11:40:38.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.605 7f146c1f4700 1 -- 172.21.2.101:0/173313543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f144c018730 msgr2=0x7f144c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:38.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.605 7f146c1f4700 1 --2- 172.21.2.101:0/173313543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f144c018730 0x7f144c01ac10 crc :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:38.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.605 7f146c1f4700 1 -- 172.21.2.101:0/173313543 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14641b86c0 msgr2=0x7f146419de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:38.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.605 7f146c1f4700 1 --2- 172.21.2.101:0/173313543 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14641b86c0 0x7f146419de70 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 rx=0x7f145c006c50 tx=0x7f145c00bd20).stop 2021-02-10T11:40:38.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.605 7f146c1f4700 1 -- 172.21.2.101:0/173313543 shutdown_connections 2021-02-10T11:40:38.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.605 7f146c1f4700 1 --2- 172.21.2.101:0/173313543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f144c018730 0x7f144c01ac10 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:38.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.605 7f146c1f4700 1 --2- 172.21.2.101:0/173313543 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f14640aaaf0 0x7f14641b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:38.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.605 7f146c1f4700 1 --2- 172.21.2.101:0/173313543 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f14641b86c0 0x7f146419de70 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:38.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.605 7f146c1f4700 1 --2- 172.21.2.101:0/173313543 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f14641b58d0 0x7f14640aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:38.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.605 7f146c1f4700 1 -- 172.21.2.101:0/173313543 >> 172.21.2.101:0/173313543 conn(0x7f146419f940 msgr2=0x7f146418c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:38.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.605 7f146c1f4700 1 -- 172.21.2.101:0/173313543 shutdown_connections 2021-02-10T11:40:38.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:38.605 7f146c1f4700 1 -- 172.21.2.101:0/173313543 wait complete. 2021-02-10T11:40:38.616 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-02-10T11:40:39.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.028 7fe09f4aa700 1 Processor -- start 2021-02-10T11:40:39.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.029 7fe09f4aa700 1 -- start start 2021-02-10T11:40:39.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.029 7fe09f4aa700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe0980d1c80 0x7fe0980cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:39.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.029 7fe09f4aa700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0980d0180 con 0x7fe0980cf590 2021-02-10T11:40:39.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.029 7fe09f4aa700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0980fe1d0 con 0x7fe0980d1880 2021-02-10T11:40:39.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.029 7fe09f4aa700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0981b0cc0 con 0x7fe0980d1c80 2021-02-10T11:40:39.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.029 7fe09dac7700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe0980cf590 0x7fe0980cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57072/0 (socket says 172.21.2.101:57072) 2021-02-10T11:40:39.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.029 7fe09dac7700 1 -- 172.21.2.101:0/890367962 learned_addr learned my addr 172.21.2.101:0/890367962 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:39.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.029 7fe09cac5700 1 --2- 172.21.2.101:0/890367962 >> v2:172.21.2.108:3300/0 conn(0x7fe0980d1c80 0x7fe0980cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:39.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.030 7fe08bfff700 1 -- 172.21.2.101:0/890367962 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76261314 0 0) 0x7fe090001500 con 0x7fe0980cf590 2021-02-10T11:40:39.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.030 7fe08bfff700 1 -- 172.21.2.101:0/890367962 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe084002fe0 con 0x7fe0980cf590 2021-02-10T11:40:39.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.030 7fe08bfff700 1 -- 172.21.2.101:0/890367962 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3729931188 0 0) 0x7fe094000ff0 con 0x7fe0980d1880 2021-02-10T11:40:39.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.030 7fe08bfff700 1 -- 172.21.2.101:0/890367962 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe084003210 con 0x7fe0980d1880 2021-02-10T11:40:39.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.030 7fe09cac5700 1 -- 172.21.2.101:0/890367962 >> v1:172.21.2.108:6789/0 conn(0x7fe0980d1880 legacy=0x7fe0981b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:39.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.030 7fe09cac5700 1 -- 172.21.2.101:0/890367962 >> v1:172.21.2.101:6789/0 conn(0x7fe0980cf590 legacy=0x7fe0980cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:39.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.030 7fe09cac5700 1 -- 172.21.2.101:0/890367962 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe098109a70 con 0x7fe0980d1c80 2021-02-10T11:40:39.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.030 7fe09cac5700 1 --2- 172.21.2.101:0/890367962 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe0980d1c80 0x7fe0980cef70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fe08c006650 tx=0x7fe08c009a00).ready entity=mon.1 client_cookie=f1131cc2f741e2c9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:39.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.030 7fe08bfff700 1 -- 172.21.2.101:0/890367962 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe08c0009b0 con 0x7fe0980d1c80 2021-02-10T11:40:39.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.030 7fe08bfff700 1 -- 172.21.2.101:0/890367962 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08c009df0 con 0x7fe0980d1c80 2021-02-10T11:40:39.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.030 7fe08bfff700 1 -- 172.21.2.101:0/890367962 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe08c00c0c0 con 0x7fe0980d1c80 2021-02-10T11:40:39.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.030 7fe09f4aa700 1 -- 172.21.2.101:0/890367962 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe0980d1c80 msgr2=0x7fe0980cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:39.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.030 7fe09f4aa700 1 --2- 172.21.2.101:0/890367962 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe0980d1c80 0x7fe0980cef70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fe08c006650 tx=0x7fe08c009a00).stop 2021-02-10T11:40:39.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.031 7fe09f4aa700 1 -- 172.21.2.101:0/890367962 shutdown_connections 2021-02-10T11:40:39.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.031 7fe09f4aa700 1 --2- 172.21.2.101:0/890367962 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe0980d1c80 0x7fe0980cef70 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:39.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.031 7fe09f4aa700 1 -- 172.21.2.101:0/890367962 >> 172.21.2.101:0/890367962 conn(0x7fe0980ce430 msgr2=0x7fe0980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:39.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.031 7fe09f4aa700 1 -- 172.21.2.101:0/890367962 shutdown_connections 2021-02-10T11:40:39.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.031 7fe09f4aa700 1 -- 172.21.2.101:0/890367962 wait complete. 2021-02-10T11:40:39.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.046 7fe09f4aa700 1 Processor -- start 2021-02-10T11:40:39.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.046 7fe09f4aa700 1 -- start start 2021-02-10T11:40:39.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.046 7fe09f4aa700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0980ad520 0x7fe0980ad920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:39.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.046 7fe09f4aa700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe0980adf40 0x7fe098097c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:39.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.046 7fe09f4aa700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe098098250 0x7fe098098650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:39.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.046 7fe09f4aa700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe098098c70 con 0x7fe098098250 2021-02-10T11:40:39.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.046 7fe09f4aa700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0980910f0 con 0x7fe0980adf40 2021-02-10T11:40:39.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.046 7fe09f4aa700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe098091310 con 0x7fe0980ad520 2021-02-10T11:40:39.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.046 7fe09dac7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe098098250 0x7fe098098650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:39.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.046 7fe09dac7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe098098250 0x7fe098098650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56586/0 (socket says 172.21.2.101:56586) 2021-02-10T11:40:39.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.046 7fe09dac7700 1 -- 172.21.2.101:0/473886489 learned_addr learned my addr 172.21.2.101:0/473886489 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:39.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.046 7fe09dac7700 1 -- 172.21.2.101:0/473886489 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0980ad520 msgr2=0x7fe0980ad920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:39.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.046 7fe09dac7700 1 --2- 172.21.2.101:0/473886489 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0980ad520 0x7fe0980ad920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:39.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.046 7fe09dac7700 1 -- 172.21.2.101:0/473886489 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe0980adf40 msgr2=0x7fe098097c30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:39.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.046 7fe09dac7700 1 --2- 172.21.2.101:0/473886489 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe0980adf40 0x7fe098097c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:39.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.046 7fe09dac7700 1 -- 172.21.2.101:0/473886489 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09819a3b0 con 0x7fe098098250 2021-02-10T11:40:39.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.047 7fe09dac7700 1 --2- 172.21.2.101:0/473886489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe098098250 0x7fe098098650 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 rx=0x7fe09000a900 tx=0x7fe09000aca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:39.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.047 7fe089ffb700 1 -- 172.21.2.101:0/473886489 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe090001270 con 0x7fe098098250 2021-02-10T11:40:39.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.047 7fe089ffb700 1 -- 172.21.2.101:0/473886489 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe090000f90 con 0x7fe098098250 2021-02-10T11:40:39.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.047 7fe09f4aa700 1 -- 172.21.2.101:0/473886489 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe09819a3b0 con 0x7fe098098250 2021-02-10T11:40:39.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.047 7fe089ffb700 1 -- 172.21.2.101:0/473886489 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe0900072c0 con 0x7fe098098250 2021-02-10T11:40:39.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.047 7fe09f4aa700 1 -- 172.21.2.101:0/473886489 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09819a3b0 con 0x7fe098098250 2021-02-10T11:40:39.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.047 7fe089ffb700 1 -- 172.21.2.101:0/473886489 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe090004930 con 0x7fe098098250 2021-02-10T11:40:39.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.047 7fe089ffb700 1 --2- 172.21.2.101:0/473886489 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0780186c0 0x7fe07801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:39.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.048 7fe089ffb700 1 -- 172.21.2.101:0/473886489 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5221+0+0 (secure 0 0 0) 0x7fe0900329e0 con 0x7fe098098250 2021-02-10T11:40:39.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.048 7fe09d2c6700 1 --2- 172.21.2.101:0/473886489 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0780186c0 0x7fe07801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:39.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.048 7fe09d2c6700 1 --2- 172.21.2.101:0/473886489 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0780186c0 0x7fe07801abc0 crc :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:39.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.079 7fe09f4aa700 1 -- 172.21.2.101:0/473886489 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe098098c70 con 0x7fe098098250 2021-02-10T11:40:39.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.081 7fe089ffb700 1 -- 172.21.2.101:0/473886489 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe090004720 con 0x7fe098098250 2021-02-10T11:40:39.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.208 7fe09f4aa700 1 -- 172.21.2.101:0/473886489 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe098098c70 con 0x7fe098098250 2021-02-10T11:40:39.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.579 7fe089ffb700 1 -- 172.21.2.101:0/473886489 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) v1 ==== 173+0+0 (secure 0 0 0) 0x7fe090004930 con 0x7fe098098250 2021-02-10T11:40:39.627 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' removed 2021-02-10T11:40:39.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.626 7fe09f4aa700 1 -- 172.21.2.101:0/473886489 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0780186c0 msgr2=0x7fe07801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:39.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.626 7fe09f4aa700 1 --2- 172.21.2.101:0/473886489 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0780186c0 0x7fe07801abc0 crc :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:39.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.626 7fe09f4aa700 1 -- 172.21.2.101:0/473886489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe098098250 msgr2=0x7fe098098650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:39.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.626 7fe09f4aa700 1 --2- 172.21.2.101:0/473886489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe098098250 0x7fe098098650 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 rx=0x7fe09000a900 tx=0x7fe09000aca0).stop 2021-02-10T11:40:39.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.626 7fe09f4aa700 1 -- 172.21.2.101:0/473886489 shutdown_connections 2021-02-10T11:40:39.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.626 7fe09f4aa700 1 --2- 172.21.2.101:0/473886489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe098098250 0x7fe098098650 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:39.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.626 7fe09f4aa700 1 --2- 172.21.2.101:0/473886489 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe0980adf40 0x7fe098097c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:39.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.626 7fe09f4aa700 1 --2- 172.21.2.101:0/473886489 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0780186c0 0x7fe07801abc0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:39.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.626 7fe09f4aa700 1 --2- 172.21.2.101:0/473886489 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0980ad520 0x7fe0980ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:39.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.626 7fe09f4aa700 1 -- 172.21.2.101:0/473886489 >> 172.21.2.101:0/473886489 conn(0x7fe09818e810 msgr2=0x7fe09818c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:39.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.626 7fe09f4aa700 1 -- 172.21.2.101:0/473886489 shutdown_connections 2021-02-10T11:40:39.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:39.626 7fe09f4aa700 1 -- 172.21.2.101:0/473886489 wait complete. 2021-02-10T11:40:39.637 INFO:tasks.ceph.mds.a:Restarting daemon 2021-02-10T11:40:39.637 INFO:tasks.ceph.mds.a:Stopping old one... 2021-02-10T11:40:39.637 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-02-10T11:40:39.637 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:40:39.638 INFO:tasks.ceph.mds.b:Restarting daemon 2021-02-10T11:40:39.638 INFO:tasks.ceph.mds.b:Stopping old one... 2021-02-10T11:40:39.638 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-02-10T11:40:39.638 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:40:39.643 INFO:tasks.ceph.mds.a:Stopped 2021-02-10T11:40:39.643 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-02-10T11:40:39.644 INFO:tasks.ceph.mds.b:Stopped 2021-02-10T11:40:39.644 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-02-10T11:40:39.685 INFO:tasks.ceph.mds.a:Started 2021-02-10T11:40:39.686 DEBUG:teuthology.parallel:result is None 2021-02-10T11:40:39.686 INFO:tasks.ceph.mds.b:Started 2021-02-10T11:40:39.687 DEBUG:teuthology.parallel:result is None 2021-02-10T11:40:39.687 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd blacklist clear 2021-02-10T11:40:40.091 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:40:40.094 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:40:40.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.098 7f27b11f9700 1 Processor -- start 2021-02-10T11:40:40.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.098 7f27b11f9700 1 -- start start 2021-02-10T11:40:40.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.098 7f27b11f9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27ac0d1870 0x7f27ac1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:40.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.098 7f27b11f9700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f27ac0f0650 0x7f27ac0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:40.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.098 7f27b11f9700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27ac0f0b10 con 0x7f27ac1b0a10 2021-02-10T11:40:40.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.098 7f27b11f9700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27ac0fe230 con 0x7f27ac0f0650 2021-02-10T11:40:40.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.098 7f27b11f9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0e5ed0 con 0x7f27ac0d1870 2021-02-10T11:40:40.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.099 7f27aa59c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f27ac1b0a10 0x7f27ac0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57098/0 (socket says 172.21.2.101:57098) 2021-02-10T11:40:40.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.099 7f27aa59c700 1 -- 172.21.2.101:0/3604300008 learned_addr learned my addr 172.21.2.101:0/3604300008 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:40.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.099 7f27ab59e700 1 --2- 172.21.2.101:0/3604300008 >> v2:172.21.2.108:3300/0 conn(0x7f27ac0f0650 0x7f27ac0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:40.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.099 7f27aad9d700 1 --2- 172.21.2.101:0/3604300008 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27ac0d1870 0x7f27ac1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:40.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.099 7f27a9d9b700 1 -- 172.21.2.101:0/3604300008 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2635150712 0 0) 0x7f27a00010b0 con 0x7f27ac1b0a10 2021-02-10T11:40:40.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.099 7f27a9d9b700 1 -- 172.21.2.101:0/3604300008 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2794002fe0 con 0x7f27ac1b0a10 2021-02-10T11:40:40.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.099 7f27a9d9b700 1 -- 172.21.2.101:0/3604300008 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2596433309 0 0) 0x7f27a00010b0 con 0x7f27ac1b0a10 2021-02-10T11:40:40.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.099 7f27ab59e700 1 -- 172.21.2.101:0/3604300008 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27ac0d1870 msgr2=0x7f27ac1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:40.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.099 7f27ab59e700 1 --2- 172.21.2.101:0/3604300008 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27ac0d1870 0x7f27ac1b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:40.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.099 7f27ab59e700 1 -- 172.21.2.101:0/3604300008 >> v1:172.21.2.101:6789/0 conn(0x7f27ac1b0a10 legacy=0x7f27ac0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:40.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.099 7f27ab59e700 1 -- 172.21.2.101:0/3604300008 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac0f20a0 con 0x7f27ac0f0650 2021-02-10T11:40:40.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.099 7f27ab59e700 1 --2- 172.21.2.101:0/3604300008 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f27ac0f0650 0x7f27ac0cfb20 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f27a4006e50 tx=0x7f27a4009e70).ready entity=mon.1 client_cookie=8552a2d05335d46c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:40.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.099 7f27a9d9b700 1 -- 172.21.2.101:0/3604300008 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f27a400b510 con 0x7f27ac0f0650 2021-02-10T11:40:40.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.099 7f27a9d9b700 1 -- 172.21.2.101:0/3604300008 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a4006790 con 0x7f27ac0f0650 2021-02-10T11:40:40.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.099 7f27a9d9b700 1 -- 172.21.2.101:0/3604300008 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f27a400b940 con 0x7f27ac0f0650 2021-02-10T11:40:40.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.100 7f27b11f9700 1 -- 172.21.2.101:0/3604300008 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f27ac0f0650 msgr2=0x7f27ac0cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:40.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.100 7f27b11f9700 1 --2- 172.21.2.101:0/3604300008 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f27ac0f0650 0x7f27ac0cfb20 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f27a4006e50 tx=0x7f27a4009e70).stop 2021-02-10T11:40:40.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.100 7f27b11f9700 1 -- 172.21.2.101:0/3604300008 shutdown_connections 2021-02-10T11:40:40.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.100 7f27b11f9700 1 --2- 172.21.2.101:0/3604300008 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f27ac0f0650 0x7f27ac0cfb20 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:40.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.100 7f27b11f9700 1 --2- 172.21.2.101:0/3604300008 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27ac0d1870 0x7f27ac1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:40.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.100 7f27b11f9700 1 -- 172.21.2.101:0/3604300008 >> 172.21.2.101:0/3604300008 conn(0x7f27ac0ce450 msgr2=0x7f27ac0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:40.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.100 7f27b11f9700 1 -- 172.21.2.101:0/3604300008 shutdown_connections 2021-02-10T11:40:40.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.100 7f27b11f9700 1 -- 172.21.2.101:0/3604300008 wait complete. 2021-02-10T11:40:40.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.115 7f27b11f9700 1 Processor -- start 2021-02-10T11:40:40.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.115 7f27b11f9700 1 -- start start 2021-02-10T11:40:40.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.116 7f27b11f9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f27ac19ffa0 0x7f27ac0eda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:40.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.116 7f27b11f9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27ac1a03a0 0x7f27ac0ee030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:40.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.116 7f27b11f9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27ac0ee650 0x7f27ac0ebe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:40.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.116 7f27b11f9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0ed490 con 0x7f27ac0ee650 2021-02-10T11:40:40.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.116 7f27b11f9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0eea50 con 0x7f27ac19ffa0 2021-02-10T11:40:40.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.116 7f27b11f9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0eec30 con 0x7f27ac1a03a0 2021-02-10T11:40:40.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.116 7f27ab59e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27ac0ee650 0x7f27ac0ebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:40.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.116 7f27ab59e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27ac0ee650 0x7f27ac0ebe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56612/0 (socket says 172.21.2.101:56612) 2021-02-10T11:40:40.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.116 7f27ab59e700 1 -- 172.21.2.101:0/1643119149 learned_addr learned my addr 172.21.2.101:0/1643119149 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:40.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.116 7f27aad9d700 1 --2- 172.21.2.101:0/1643119149 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f27ac19ffa0 0x7f27ac0eda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:40.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.116 7f27aa59c700 1 --2- 172.21.2.101:0/1643119149 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27ac1a03a0 0x7f27ac0ee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:40.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.116 7f27ab59e700 1 -- 172.21.2.101:0/1643119149 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27ac1a03a0 msgr2=0x7f27ac0ee030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:40.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.116 7f27ab59e700 1 --2- 172.21.2.101:0/1643119149 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27ac1a03a0 0x7f27ac0ee030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:40.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.116 7f27ab59e700 1 -- 172.21.2.101:0/1643119149 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f27ac19ffa0 msgr2=0x7f27ac0eda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:40.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.116 7f27ab59e700 1 --2- 172.21.2.101:0/1643119149 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f27ac19ffa0 0x7f27ac0eda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:40.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.116 7f27ab59e700 1 -- 172.21.2.101:0/1643119149 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac0ec4a0 con 0x7f27ac0ee650 2021-02-10T11:40:40.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.117 7f27ab59e700 1 --2- 172.21.2.101:0/1643119149 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27ac0ee650 0x7f27ac0ebe80 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 rx=0x7f27a4009eb0 tx=0x7f27a400bd60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:40.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.117 7f279b7fe700 1 -- 172.21.2.101:0/1643119149 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f27a4009650 con 0x7f27ac0ee650 2021-02-10T11:40:40.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.117 7f279b7fe700 1 -- 172.21.2.101:0/1643119149 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a400c2f0 con 0x7f27ac0ee650 2021-02-10T11:40:40.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.117 7f279b7fe700 1 -- 172.21.2.101:0/1643119149 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f27a400c7d0 con 0x7f27ac0ee650 2021-02-10T11:40:40.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.117 7f27b11f9700 1 -- 172.21.2.101:0/1643119149 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27ac0ec4a0 con 0x7f27ac0ee650 2021-02-10T11:40:40.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.117 7f27b11f9700 1 -- 172.21.2.101:0/1643119149 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27ac0eca30 con 0x7f27ac0ee650 2021-02-10T11:40:40.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.117 7f279b7fe700 1 -- 172.21.2.101:0/1643119149 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f27a4008020 con 0x7f27ac0ee650 2021-02-10T11:40:40.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.117 7f279b7fe700 1 --2- 172.21.2.101:0/1643119149 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f27900186c0 0x7f279001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:40.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.117 7f279b7fe700 1 -- 172.21.2.101:0/1643119149 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4839+0+0 (secure 0 0 0) 0x7f27a40324a0 con 0x7f27ac0ee650 2021-02-10T11:40:40.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.118 7f27aad9d700 1 --2- 172.21.2.101:0/1643119149 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f27900186c0 0x7f279001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:40.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.118 7f27aad9d700 1 --2- 172.21.2.101:0/1643119149 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f27900186c0 0x7f279001abc0 crc :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:40.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.147 7f27b11f9700 1 -- 172.21.2.101:0/1643119149 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27ac0ebc50 con 0x7f27ac0ee650 2021-02-10T11:40:40.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.149 7f279b7fe700 1 -- 172.21.2.101:0/1643119149 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f27a4009650 con 0x7f27ac0ee650 2021-02-10T11:40:40.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.288 7f27b11f9700 1 -- 172.21.2.101:0/1643119149 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f27ac0ebc50 con 0x7f27ac0ee650 2021-02-10T11:40:40.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.624 7f279b7fe700 1 -- 172.21.2.101:0/1643119149 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v107) v1 ==== 97+0+0 (secure 0 0 0) 0x7f27a4008020 con 0x7f27ac0ee650 2021-02-10T11:40:40.657 INFO:teuthology.orchestra.run.gibba001.stderr: removed all blacklist entries 2021-02-10T11:40:40.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.656 7f27b11f9700 1 -- 172.21.2.101:0/1643119149 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f27900186c0 msgr2=0x7f279001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:40.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.656 7f27b11f9700 1 --2- 172.21.2.101:0/1643119149 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f27900186c0 0x7f279001abc0 crc :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:40.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.656 7f27b11f9700 1 -- 172.21.2.101:0/1643119149 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27ac0ee650 msgr2=0x7f27ac0ebe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:40.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.656 7f27b11f9700 1 --2- 172.21.2.101:0/1643119149 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27ac0ee650 0x7f27ac0ebe80 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 rx=0x7f27a4009eb0 tx=0x7f27a400bd60).stop 2021-02-10T11:40:40.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.656 7f27b11f9700 1 -- 172.21.2.101:0/1643119149 shutdown_connections 2021-02-10T11:40:40.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.656 7f27b11f9700 1 --2- 172.21.2.101:0/1643119149 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f27900186c0 0x7f279001abc0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:40.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.656 7f27b11f9700 1 --2- 172.21.2.101:0/1643119149 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f27ac1a03a0 0x7f27ac0ee030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:40.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.656 7f27b11f9700 1 --2- 172.21.2.101:0/1643119149 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f27ac0ee650 0x7f27ac0ebe80 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:40.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.656 7f27b11f9700 1 --2- 172.21.2.101:0/1643119149 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f27ac19ffa0 0x7f27ac0eda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:40.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.656 7f27b11f9700 1 -- 172.21.2.101:0/1643119149 >> 172.21.2.101:0/1643119149 conn(0x7f27ac094d90 msgr2=0x7f27ac08a420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:40.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.656 7f27b11f9700 1 -- 172.21.2.101:0/1643119149 shutdown_connections 2021-02-10T11:40:40.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:40.656 7f27b11f9700 1 -- 172.21.2.101:0/1643119149 wait complete. 2021-02-10T11:40:40.667 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2021-02-10T11:40:40.667 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:40:41.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.075 7fe023f21700 1 Processor -- start 2021-02-10T11:40:41.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.075 7fe023f21700 1 -- start start 2021-02-10T11:40:41.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.076 7fe023f21700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe01c1b0470 0x7fe01c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:41.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.076 7fe023f21700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe01c0f0b50 con 0x7fe01c0f20e0 2021-02-10T11:40:41.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.076 7fe023f21700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe01c0f24e0 con 0x7fe01c0ef610 2021-02-10T11:40:41.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.076 7fe023f21700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe01c0e5ea0 con 0x7fe01c1b0470 2021-02-10T11:40:41.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.076 7fe02253e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe01c0f20e0 0x7fe01c0f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57110/0 (socket says 172.21.2.101:57110) 2021-02-10T11:40:41.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.076 7fe02253e700 1 -- 172.21.2.101:0/2931927398 learned_addr learned my addr 172.21.2.101:0/2931927398 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:41.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.077 7fe021d3d700 1 --2- 172.21.2.101:0/2931927398 >> v2:172.21.2.108:3300/0 conn(0x7fe01c1b0470 0x7fe01c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:41.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.077 7fe020d3b700 1 -- 172.21.2.101:0/2931927398 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651788612 0 0) 0x7fe014001500 con 0x7fe01c0f20e0 2021-02-10T11:40:41.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.077 7fe020d3b700 1 -- 172.21.2.101:0/2931927398 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe008002fe0 con 0x7fe01c0f20e0 2021-02-10T11:40:41.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.077 7fe021d3d700 1 -- 172.21.2.101:0/2931927398 >> v1:172.21.2.108:6789/0 conn(0x7fe01c0ef610 legacy=0x7fe01c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:41.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.077 7fe021d3d700 1 -- 172.21.2.101:0/2931927398 >> v1:172.21.2.101:6789/0 conn(0x7fe01c0f20e0 legacy=0x7fe01c0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:41.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.077 7fe021d3d700 1 -- 172.21.2.101:0/2931927398 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c0e52e0 con 0x7fe01c1b0470 2021-02-10T11:40:41.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.077 7fe021d3d700 1 --2- 172.21.2.101:0/2931927398 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe01c1b0470 0x7fe01c1b0870 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7fe018006650 tx=0x7fe018009a80).ready entity=mon.1 client_cookie=623c702252db7cd6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:41.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.078 7fe020d3b700 1 -- 172.21.2.101:0/2931927398 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe0180009b0 con 0x7fe01c1b0470 2021-02-10T11:40:41.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.078 7fe020d3b700 1 -- 172.21.2.101:0/2931927398 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01800c0c0 con 0x7fe01c1b0470 2021-02-10T11:40:41.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.078 7fe020d3b700 1 -- 172.21.2.101:0/2931927398 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe01800c5e0 con 0x7fe01c1b0470 2021-02-10T11:40:41.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.078 7fe023f21700 1 -- 172.21.2.101:0/2931927398 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe01c1b0470 msgr2=0x7fe01c1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:41.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.078 7fe023f21700 1 --2- 172.21.2.101:0/2931927398 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe01c1b0470 0x7fe01c1b0870 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7fe018006650 tx=0x7fe018009a80).stop 2021-02-10T11:40:41.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.078 7fe023f21700 1 -- 172.21.2.101:0/2931927398 shutdown_connections 2021-02-10T11:40:41.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.078 7fe023f21700 1 --2- 172.21.2.101:0/2931927398 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe01c1b0470 0x7fe01c1b0870 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:41.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.078 7fe023f21700 1 -- 172.21.2.101:0/2931927398 >> 172.21.2.101:0/2931927398 conn(0x7fe01c0ce3e0 msgr2=0x7fe01c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:41.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.079 7fe023f21700 1 -- 172.21.2.101:0/2931927398 shutdown_connections 2021-02-10T11:40:41.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.079 7fe023f21700 1 -- 172.21.2.101:0/2931927398 wait complete. 2021-02-10T11:40:41.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.093 7fe023f21700 1 Processor -- start 2021-02-10T11:40:41.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.093 7fe023f21700 1 -- start start 2021-02-10T11:40:41.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.094 7fe023f21700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe01c0b0970 0x7fe01c092ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:41.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.094 7fe023f21700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe01c0934f0 0x7fe01c094dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:41.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.094 7fe023f21700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe01c0938f0 0x7fe01c0953e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:41.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.094 7fe023f21700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c195ad0 con 0x7fe01c0b0970 2021-02-10T11:40:41.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.094 7fe023f21700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c095f40 con 0x7fe01c0934f0 2021-02-10T11:40:41.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.094 7fe023f21700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c095a30 con 0x7fe01c0938f0 2021-02-10T11:40:41.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.094 7fe021d3d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe01c0b0970 0x7fe01c092ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:41.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.094 7fe021d3d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe01c0b0970 0x7fe01c092ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56620/0 (socket says 172.21.2.101:56620) 2021-02-10T11:40:41.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.094 7fe021d3d700 1 -- 172.21.2.101:0/1470174795 learned_addr learned my addr 172.21.2.101:0/1470174795 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:41.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.094 7fe021d3d700 1 -- 172.21.2.101:0/1470174795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe01c0938f0 msgr2=0x7fe01c0953e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:41.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.094 7fe021d3d700 1 --2- 172.21.2.101:0/1470174795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe01c0938f0 0x7fe01c0953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:41.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.094 7fe021d3d700 1 -- 172.21.2.101:0/1470174795 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe01c0934f0 msgr2=0x7fe01c094dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:41.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.094 7fe021d3d700 1 --2- 172.21.2.101:0/1470174795 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe01c0934f0 0x7fe01c094dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:41.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.094 7fe021d3d700 1 -- 172.21.2.101:0/1470174795 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c095c50 con 0x7fe01c0b0970 2021-02-10T11:40:41.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.094 7fe021d3d700 1 --2- 172.21.2.101:0/1470174795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe01c0b0970 0x7fe01c092ed0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 rx=0x7fe018009630 tx=0x7fe01800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:41.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.094 7fe0127fc700 1 -- 172.21.2.101:0/1470174795 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe0180093a0 con 0x7fe01c0b0970 2021-02-10T11:40:41.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.094 7fe0127fc700 1 -- 172.21.2.101:0/1470174795 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01800bd70 con 0x7fe01c0b0970 2021-02-10T11:40:41.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.095 7fe023f21700 1 -- 172.21.2.101:0/1470174795 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01c095c50 con 0x7fe01c0b0970 2021-02-10T11:40:41.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.095 7fe0127fc700 1 -- 172.21.2.101:0/1470174795 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe018005560 con 0x7fe01c0b0970 2021-02-10T11:40:41.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.095 7fe023f21700 1 -- 172.21.2.101:0/1470174795 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01c095c50 con 0x7fe01c0b0970 2021-02-10T11:40:41.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.095 7fe0127fc700 1 -- 172.21.2.101:0/1470174795 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe0180052e0 con 0x7fe01c0b0970 2021-02-10T11:40:41.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.095 7fe0127fc700 1 --2- 172.21.2.101:0/1470174795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe004018730 0x7fe00401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:41.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.095 7fe0127fc700 1 -- 172.21.2.101:0/1470174795 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fe0180322f0 con 0x7fe01c0b0970 2021-02-10T11:40:41.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.095 7fe02153c700 1 --2- 172.21.2.101:0/1470174795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe004018730 0x7fe00401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:41.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.096 7fe02153c700 1 --2- 172.21.2.101:0/1470174795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe004018730 0x7fe00401abc0 crc :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:41.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.126 7fe023f21700 1 -- 172.21.2.101:0/1470174795 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe01c0e52e0 con 0x7fe01c0b0970 2021-02-10T11:40:41.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.129 7fe0127fc700 1 -- 172.21.2.101:0/1470174795 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe018006030 con 0x7fe01c0b0970 2021-02-10T11:40:41.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.261 7fe023f21700 1 -- 172.21.2.101:0/1470174795 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe01c0e52e0 con 0x7fe01c0b0970 2021-02-10T11:40:41.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.261 7fe0127fc700 1 -- 172.21.2.101:0/1470174795 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v64) v1 ==== 80+0+4513 (secure 0 0 0) 0x7fe01800bd70 con 0x7fe01c0b0970 2021-02-10T11:40:41.264 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:40:41.264 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:40:41.265 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:40:41.265 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:41.265 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:40:41.265 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:40:41.265 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:40:41.265 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:40:41.265 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:40:41.265 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:40:41.265 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:40:41.265 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:41.265 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:41.265 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:41.265 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:40:41.266 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:40:41.266 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:40:41.266 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:40:41.266 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:40:41.266 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:40:41.266 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:40:41.266 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:41.266 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:41.266 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:41.266 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:40:41.266 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:40:41.266 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:40:41.267 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:40:41.267 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:40:41.267 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:40:41.267 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:41.267 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:41.267 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:41.267 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:40:41.267 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:40:41.267 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:40:41.267 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:40:41.267 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:40:41.267 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:40:41.267 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:41.268 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:41.268 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:41.268 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:40:41.268 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:40:41.268 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:40:41.268 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:40:41.268 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:40:41.268 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:40:41.268 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:41.268 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:41.268 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:41.268 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:40:41.268 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:40:41.268 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:40:41.269 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:40:41.269 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:40:41.269 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:40:41.269 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:41.269 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:41.269 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:41.269 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:40:41.269 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:40:41.269 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:40:41.269 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:40:41.269 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:40:41.269 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:40:41.269 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:41.270 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:41.270 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:41.270 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:40:41.270 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:40:41.270 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:40:41.270 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:40:41.270 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:40:41.270 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:40:41.270 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:41.270 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:41.270 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:41.270 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:40:41.270 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:40:41.271 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:40:41.271 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:40:41.271 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:40:41.271 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:40:41.271 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:41.271 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:41.271 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:41.271 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:40:41.271 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:40:41.271 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:40:41.271 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:40:41.271 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:40:41.271 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:40:41.271 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:41.272 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:41.272 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:41.272 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:40:41.272 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:40:41.272 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:40:41.272 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:40:41.272 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:40:41.272 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:40:41.272 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:41.272 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:41.272 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:41.272 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:40:41.272 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:40:41.272 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:40:41.273 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:40:41.273 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:40:41.273 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:40:41.273 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:41.273 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:41.273 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:41.273 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:40:41.273 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:40:41.273 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:40:41.273 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:40:41.273 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:40:41.273 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:40:41.273 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:41.273 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:41.273 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:41.274 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:40:41.274 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:40:41.274 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:40:41.274 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:40:41.274 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:40:41.274 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:40:41.274 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:41.274 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:41.274 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:41.274 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:40:41.274 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:40:41.274 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:40:41.275 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:40:41.275 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:40:41.275 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:40:41.275 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:40:41.275 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:41.275 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:41.275 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:41.275 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:40:41.275 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:40:41.275 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:40:41.275 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:40:41.275 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:40:41.275 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:40:41.276 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:41.276 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:41.276 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:41.276 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:40:41.276 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:40:41.276 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:40:41.276 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:40:41.276 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:40:41.276 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:40:41.276 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:41.276 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:41.276 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:40:41.276 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:40:41.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.263 7fe023f21700 1 -- 172.21.2.101:0/1470174795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe004018730 msgr2=0x7fe00401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:41.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.263 7fe023f21700 1 --2- 172.21.2.101:0/1470174795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe004018730 0x7fe00401abc0 crc :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:41.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.263 7fe023f21700 1 -- 172.21.2.101:0/1470174795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe01c0b0970 msgr2=0x7fe01c092ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:41.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.263 7fe023f21700 1 --2- 172.21.2.101:0/1470174795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe01c0b0970 0x7fe01c092ed0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 rx=0x7fe018009630 tx=0x7fe01800b980).stop 2021-02-10T11:40:41.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.263 7fe023f21700 1 -- 172.21.2.101:0/1470174795 shutdown_connections 2021-02-10T11:40:41.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.263 7fe023f21700 1 --2- 172.21.2.101:0/1470174795 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe004018730 0x7fe00401abc0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:41.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.263 7fe023f21700 1 --2- 172.21.2.101:0/1470174795 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe01c0938f0 0x7fe01c0953e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:41.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.263 7fe023f21700 1 --2- 172.21.2.101:0/1470174795 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe01c0934f0 0x7fe01c094dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:41.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.263 7fe023f21700 1 --2- 172.21.2.101:0/1470174795 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe01c0b0970 0x7fe01c092ed0 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:41.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.263 7fe023f21700 1 -- 172.21.2.101:0/1470174795 >> 172.21.2.101:0/1470174795 conn(0x7fe01c19f7a0 msgr2=0x7fe01c0940e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:41.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.263 7fe023f21700 1 -- 172.21.2.101:0/1470174795 shutdown_connections 2021-02-10T11:40:41.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.263 7fe023f21700 1 -- 172.21.2.101:0/1470174795 wait complete. 2021-02-10T11:40:41.279 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:40:41.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.686 7f3975dd0700 1 Processor -- start 2021-02-10T11:40:41.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.686 7f3975dd0700 1 -- start start 2021-02-10T11:40:41.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.687 7f3975dd0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f39700d0340 0x7f39700ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:41.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.687 7f3975dd0700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f39700d1c70 0x7f39700efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:41.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.687 7f3975dd0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39700f0b10 con 0x7f39700d1870 2021-02-10T11:40:41.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.687 7f3975dd0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39700e5ed0 con 0x7f39700d0340 2021-02-10T11:40:41.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.687 7f3975dd0700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39701b0be0 con 0x7f39700d1c70 2021-02-10T11:40:41.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.687 7f396effd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f39700d0340 0x7f39700ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:41.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.687 7f396effd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f39700d0340 0x7f39700ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56626/0 (socket says 172.21.2.101:56626) 2021-02-10T11:40:41.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.687 7f396effd700 1 -- 172.21.2.101:0/342605937 learned_addr learned my addr 172.21.2.101:0/342605937 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:41.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.687 7f396effd700 1 -- 172.21.2.101:0/342605937 >> v2:172.21.2.108:3300/0 conn(0x7f39700d1c70 msgr2=0x7f39700efc60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:40:41.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.687 7f396effd700 1 --2- 172.21.2.101:0/342605937 >> v2:172.21.2.108:3300/0 conn(0x7f39700d1c70 0x7f39700efc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:41.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.687 7f396effd700 1 -- 172.21.2.101:0/342605937 >> v1:172.21.2.108:6789/0 conn(0x7f39700d1870 legacy=0x7f39701b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:40:41.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.687 7f396effd700 1 -- 172.21.2.101:0/342605937 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39700f20a0 con 0x7f39700d0340 2021-02-10T11:40:41.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.687 7f396effd700 1 --2- 172.21.2.101:0/342605937 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f39700d0340 0x7f39700ef640 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 rx=0x7f3958006700 tx=0x7f3958009b80).ready entity=mon.0 client_cookie=35afef818c6b3a4d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:41.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.688 7f396e7fc700 1 -- 172.21.2.101:0/342605937 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f395800c0c0 con 0x7f39700d0340 2021-02-10T11:40:41.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.688 7f396e7fc700 1 -- 172.21.2.101:0/342605937 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395800c440 con 0x7f39700d0340 2021-02-10T11:40:41.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.688 7f396e7fc700 1 -- 172.21.2.101:0/342605937 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f395800c960 con 0x7f39700d0340 2021-02-10T11:40:41.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.688 7f3975dd0700 1 -- 172.21.2.101:0/342605937 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f39700d0340 msgr2=0x7f39700ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:41.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.688 7f3975dd0700 1 --2- 172.21.2.101:0/342605937 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f39700d0340 0x7f39700ef640 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 rx=0x7f3958006700 tx=0x7f3958009b80).stop 2021-02-10T11:40:41.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.689 7f3975dd0700 1 -- 172.21.2.101:0/342605937 shutdown_connections 2021-02-10T11:40:41.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.689 7f3975dd0700 1 --2- 172.21.2.101:0/342605937 >> v2:172.21.2.108:3300/0 conn(0x7f39700d1c70 0x7f39700efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:41.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.689 7f3975dd0700 1 --2- 172.21.2.101:0/342605937 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f39700d0340 0x7f39700ef640 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:41.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.689 7f3975dd0700 1 -- 172.21.2.101:0/342605937 >> 172.21.2.101:0/342605937 conn(0x7f39700ce450 msgr2=0x7f39700d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:41.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.689 7f3975dd0700 1 -- 172.21.2.101:0/342605937 shutdown_connections 2021-02-10T11:40:41.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.689 7f3975dd0700 1 -- 172.21.2.101:0/342605937 wait complete. 2021-02-10T11:40:41.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.703 7f3975dd0700 1 Processor -- start 2021-02-10T11:40:41.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.703 7f3975dd0700 1 -- start start 2021-02-10T11:40:41.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.704 7f3975dd0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f39701947b0 0x7f39700a69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:41.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.704 7f3975dd0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f39700a6fc0 0x7f39701a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:41.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.704 7f3975dd0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f39701a1bf0 0x7f39701a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:41.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.704 7f3975dd0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39700a73c0 con 0x7f39701947b0 2021-02-10T11:40:41.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.704 7f3975dd0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397018f170 con 0x7f39701a1bf0 2021-02-10T11:40:41.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.704 7f3975dd0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3970194bb0 con 0x7f39700a6fc0 2021-02-10T11:40:41.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.704 7f396f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f39701947b0 0x7f39700a69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:41.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.704 7f396f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f39701947b0 0x7f39700a69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56632/0 (socket says 172.21.2.101:56632) 2021-02-10T11:40:41.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.704 7f396f7fe700 1 -- 172.21.2.101:0/1266901686 learned_addr learned my addr 172.21.2.101:0/1266901686 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:41.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.704 7f396f7fe700 1 -- 172.21.2.101:0/1266901686 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f39700a6fc0 msgr2=0x7f39701a15d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:41.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.704 7f396f7fe700 1 --2- 172.21.2.101:0/1266901686 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f39700a6fc0 0x7f39701a15d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:41.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.704 7f396f7fe700 1 -- 172.21.2.101:0/1266901686 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f39701a1bf0 msgr2=0x7f39701a1ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:41.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.704 7f396f7fe700 1 --2- 172.21.2.101:0/1266901686 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f39701a1bf0 0x7f39701a1ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:41.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.704 7f396f7fe700 1 -- 172.21.2.101:0/1266901686 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397018ec70 con 0x7f39701947b0 2021-02-10T11:40:41.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.704 7f396f7fe700 1 --2- 172.21.2.101:0/1266901686 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f39701947b0 0x7f39700a69a0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 rx=0x7f3960009940 tx=0x7f396000bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:41.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.704 7f3967fff700 1 -- 172.21.2.101:0/1266901686 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3960000fe0 con 0x7f39701947b0 2021-02-10T11:40:41.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.704 7f3967fff700 1 -- 172.21.2.101:0/1266901686 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f396000c2f0 con 0x7f39701947b0 2021-02-10T11:40:41.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.704 7f3967fff700 1 -- 172.21.2.101:0/1266901686 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f396000c810 con 0x7f39701947b0 2021-02-10T11:40:41.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.704 7f3975dd0700 1 -- 172.21.2.101:0/1266901686 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397018ec70 con 0x7f39701947b0 2021-02-10T11:40:41.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.704 7f3975dd0700 1 -- 172.21.2.101:0/1266901686 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397018ec70 con 0x7f39701947b0 2021-02-10T11:40:41.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.705 7f3967fff700 1 -- 172.21.2.101:0/1266901686 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3960001230 con 0x7f39701947b0 2021-02-10T11:40:41.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.705 7f3967fff700 1 --2- 172.21.2.101:0/1266901686 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3954018730 0x7f395401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:41.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.705 7f3967fff700 1 -- 172.21.2.101:0/1266901686 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f3960032850 con 0x7f39701947b0 2021-02-10T11:40:41.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.705 7f396effd700 1 --2- 172.21.2.101:0/1266901686 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3954018730 0x7f395401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:41.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.706 7f396effd700 1 --2- 172.21.2.101:0/1266901686 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3954018730 0x7f395401abc0 crc :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:41.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.737 7f3975dd0700 1 -- 172.21.2.101:0/1266901686 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39700ee0e0 con 0x7f39701947b0 2021-02-10T11:40:41.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.739 7f3967fff700 1 -- 172.21.2.101:0/1266901686 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f396000be30 con 0x7f39701947b0 2021-02-10T11:40:41.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.864 7f3975dd0700 1 -- 172.21.2.101:0/1266901686 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39700ce450 con 0x7f39701947b0 2021-02-10T11:40:41.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.865 7f3967fff700 1 -- 172.21.2.101:0/1266901686 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+11152 (secure 0 0 0) 0x7f396001f8c0 con 0x7f39701947b0 2021-02-10T11:40:41.868 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:40:41.868 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":107,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:40:40.620777","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:40:41.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.866 7f3975dd0700 1 -- 172.21.2.101:0/1266901686 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3954018730 msgr2=0x7f395401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:41.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.866 7f3975dd0700 1 --2- 172.21.2.101:0/1266901686 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3954018730 0x7f395401abc0 crc :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:41.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.866 7f3975dd0700 1 -- 172.21.2.101:0/1266901686 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f39701947b0 msgr2=0x7f39700a69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:41.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.866 7f3975dd0700 1 --2- 172.21.2.101:0/1266901686 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f39701947b0 0x7f39700a69a0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 rx=0x7f3960009940 tx=0x7f396000bcf0).stop 2021-02-10T11:40:41.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.866 7f3975dd0700 1 -- 172.21.2.101:0/1266901686 shutdown_connections 2021-02-10T11:40:41.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.866 7f3975dd0700 1 --2- 172.21.2.101:0/1266901686 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3954018730 0x7f395401abc0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:41.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.866 7f3975dd0700 1 --2- 172.21.2.101:0/1266901686 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f39700a6fc0 0x7f39701a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:41.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.866 7f3975dd0700 1 --2- 172.21.2.101:0/1266901686 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f39701a1bf0 0x7f39701a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:41.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.866 7f3975dd0700 1 --2- 172.21.2.101:0/1266901686 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f39701947b0 0x7f39700a69a0 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:41.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.866 7f3975dd0700 1 -- 172.21.2.101:0/1266901686 >> 172.21.2.101:0/1266901686 conn(0x7f39701a38c0 msgr2=0x7f39701a2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:41.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.867 7f3975dd0700 1 -- 172.21.2.101:0/1266901686 shutdown_connections 2021-02-10T11:40:41.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:41.867 7f3975dd0700 1 -- 172.21.2.101:0/1266901686 wait complete. 2021-02-10T11:40:41.878 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-10T11:40:41.878 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:40:42.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.290 7f128c85d700 1 Processor -- start 2021-02-10T11:40:42.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.290 7f128c85d700 1 -- start start 2021-02-10T11:40:42.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.291 7f128c85d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f12880d1860 0x7f12881b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:42.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.291 7f128c85d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f12880cff60 0x7f1288109120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:42.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.291 7f128c85d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12880fe1b0 con 0x7f12881b0ad0 2021-02-10T11:40:42.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.291 7f128c85d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1288109ac0 con 0x7f12880d1860 2021-02-10T11:40:42.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.291 7f128c85d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1288109740 con 0x7f12880cff60 2021-02-10T11:40:42.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.291 7f128661c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f12880d1860 0x7f12881b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:42.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.291 7f128661c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f12880d1860 0x7f12881b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56640/0 (socket says 172.21.2.101:56640) 2021-02-10T11:40:42.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.291 7f128661c700 1 -- 172.21.2.101:0/2324370702 learned_addr learned my addr 172.21.2.101:0/2324370702 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:42.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.292 7f128661c700 1 -- 172.21.2.101:0/2324370702 >> v2:172.21.2.108:3300/0 conn(0x7f12880cff60 msgr2=0x7f1288109120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:42.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.292 7f128661c700 1 --2- 172.21.2.101:0/2324370702 >> v2:172.21.2.108:3300/0 conn(0x7f12880cff60 0x7f1288109120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:42.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.292 7f128661c700 1 -- 172.21.2.101:0/2324370702 >> v1:172.21.2.108:6789/0 conn(0x7f12881b0ad0 legacy=0x7f1288108930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:42.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.292 7f128661c700 1 -- 172.21.2.101:0/2324370702 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1288088070 con 0x7f12880d1860 2021-02-10T11:40:42.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.292 7f128661c700 1 --2- 172.21.2.101:0/2324370702 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12880d1860 0x7f12881b04b0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 rx=0x7f1278006700 tx=0x7f1278009b80).ready entity=mon.0 client_cookie=b61e3d5f1e282275 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:42.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.292 7f128561a700 1 -- 172.21.2.101:0/2324370702 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f127800c0c0 con 0x7f12880d1860 2021-02-10T11:40:42.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.292 7f128561a700 1 -- 172.21.2.101:0/2324370702 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127800c340 con 0x7f12880d1860 2021-02-10T11:40:42.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.292 7f128561a700 1 -- 172.21.2.101:0/2324370702 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f127800c860 con 0x7f12880d1860 2021-02-10T11:40:42.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.292 7f128c85d700 1 -- 172.21.2.101:0/2324370702 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12880d1860 msgr2=0x7f12881b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:42.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.292 7f128c85d700 1 --2- 172.21.2.101:0/2324370702 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12880d1860 0x7f12881b04b0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 rx=0x7f1278006700 tx=0x7f1278009b80).stop 2021-02-10T11:40:42.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.293 7f128c85d700 1 -- 172.21.2.101:0/2324370702 shutdown_connections 2021-02-10T11:40:42.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.293 7f128c85d700 1 --2- 172.21.2.101:0/2324370702 >> v2:172.21.2.108:3300/0 conn(0x7f12880cff60 0x7f1288109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:42.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.293 7f128c85d700 1 --2- 172.21.2.101:0/2324370702 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12880d1860 0x7f12881b04b0 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:42.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.293 7f128c85d700 1 -- 172.21.2.101:0/2324370702 >> 172.21.2.101:0/2324370702 conn(0x7f12880ce410 msgr2=0x7f12880d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:42.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.293 7f128c85d700 1 -- 172.21.2.101:0/2324370702 shutdown_connections 2021-02-10T11:40:42.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.293 7f128c85d700 1 -- 172.21.2.101:0/2324370702 wait complete. 2021-02-10T11:40:42.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.308 7f128c85d700 1 Processor -- start 2021-02-10T11:40:42.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.308 7f128c85d700 1 -- start start 2021-02-10T11:40:42.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.308 7f128c85d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f12880bfa10 0x7f12880b82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:42.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.308 7f128c85d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12881a4a70 0x7f1288094c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:42.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.308 7f128c85d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f12880c0410 0x7f12880c0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:42.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.308 7f128c85d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12880c3a50 con 0x7f12881a4a70 2021-02-10T11:40:42.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.308 7f128c85d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12880b88c0 con 0x7f12880bfa10 2021-02-10T11:40:42.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.308 7f128c85d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1288095260 con 0x7f12880c0410 2021-02-10T11:40:42.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.308 7f1285e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12881a4a70 0x7f1288094c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:42.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.308 7f1285e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12881a4a70 0x7f1288094c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56648/0 (socket says 172.21.2.101:56648) 2021-02-10T11:40:42.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.308 7f1285e1b700 1 -- 172.21.2.101:0/1876357450 learned_addr learned my addr 172.21.2.101:0/1876357450 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:42.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.309 7f1286e1d700 1 --2- 172.21.2.101:0/1876357450 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f12880c0410 0x7f12880c0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:42.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.309 7f128661c700 1 --2- 172.21.2.101:0/1876357450 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f12880bfa10 0x7f12880b82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:42.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.309 7f1285e1b700 1 -- 172.21.2.101:0/1876357450 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f12880c0410 msgr2=0x7f12880c0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:42.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.309 7f1285e1b700 1 --2- 172.21.2.101:0/1876357450 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f12880c0410 0x7f12880c0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:42.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.309 7f1285e1b700 1 -- 172.21.2.101:0/1876357450 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f12880bfa10 msgr2=0x7f12880b82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:42.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.309 7f1285e1b700 1 --2- 172.21.2.101:0/1876357450 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f12880bfa10 0x7f12880b82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:42.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.309 7f1285e1b700 1 -- 172.21.2.101:0/1876357450 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12880c3540 con 0x7f12881a4a70 2021-02-10T11:40:42.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.309 7f128661c700 1 --2- 172.21.2.101:0/1876357450 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f12880bfa10 0x7f12880b82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:40:42.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.309 7f1285e1b700 1 --2- 172.21.2.101:0/1876357450 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12881a4a70 0x7f1288094c40 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 rx=0x7f127c00b5b0 tx=0x7f127c00b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:42.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.309 7f1276ffd700 1 -- 172.21.2.101:0/1876357450 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f127c0009b0 con 0x7f12881a4a70 2021-02-10T11:40:42.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.309 7f1276ffd700 1 -- 172.21.2.101:0/1876357450 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127c00bd40 con 0x7f12881a4a70 2021-02-10T11:40:42.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.309 7f128c85d700 1 -- 172.21.2.101:0/1876357450 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12880c3540 con 0x7f12881a4a70 2021-02-10T11:40:42.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.309 7f1276ffd700 1 -- 172.21.2.101:0/1876357450 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f127c006210 con 0x7f12881a4a70 2021-02-10T11:40:42.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.309 7f128c85d700 1 -- 172.21.2.101:0/1876357450 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12880c1120 con 0x7f12881a4a70 2021-02-10T11:40:42.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.310 7f1276ffd700 1 -- 172.21.2.101:0/1876357450 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f127c006020 con 0x7f12881a4a70 2021-02-10T11:40:42.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.310 7f1276ffd700 1 --2- 172.21.2.101:0/1876357450 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f126c018730 0x7f126c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:42.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.310 7f1276ffd700 1 -- 172.21.2.101:0/1876357450 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f127c0324a0 con 0x7f12881a4a70 2021-02-10T11:40:42.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.310 7f128661c700 1 --2- 172.21.2.101:0/1876357450 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f126c018730 0x7f126c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:42.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.310 7f128661c700 1 --2- 172.21.2.101:0/1876357450 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f126c018730 0x7f126c01ac10 crc :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:42.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.341 7f128c85d700 1 -- 172.21.2.101:0/1876357450 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12880bfee0 con 0x7f12881a4a70 2021-02-10T11:40:42.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.344 7f1276ffd700 1 -- 172.21.2.101:0/1876357450 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f127c000940 con 0x7f12881a4a70 2021-02-10T11:40:42.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.470 7f128c85d700 1 -- 172.21.2.101:0/1876357450 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f12880bfee0 con 0x7f12881a4a70 2021-02-10T11:40:42.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.471 7f1276ffd700 1 -- 172.21.2.101:0/1876357450 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v107) v1 ==== 92+0+10 (secure 0 0 0) 0x7f127c00bd40 con 0x7f12881a4a70 2021-02-10T11:40:42.474 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:40:42.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.473 7f128c85d700 1 -- 172.21.2.101:0/1876357450 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f126c018730 msgr2=0x7f126c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:42.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.473 7f128c85d700 1 --2- 172.21.2.101:0/1876357450 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f126c018730 0x7f126c01ac10 crc :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:42.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.473 7f128c85d700 1 -- 172.21.2.101:0/1876357450 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12881a4a70 msgr2=0x7f1288094c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:42.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.473 7f128c85d700 1 --2- 172.21.2.101:0/1876357450 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12881a4a70 0x7f1288094c40 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 rx=0x7f127c00b5b0 tx=0x7f127c00b950).stop 2021-02-10T11:40:42.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.473 7f128c85d700 1 -- 172.21.2.101:0/1876357450 shutdown_connections 2021-02-10T11:40:42.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.473 7f128c85d700 1 --2- 172.21.2.101:0/1876357450 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f126c018730 0x7f126c01ac10 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:42.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.473 7f128c85d700 1 --2- 172.21.2.101:0/1876357450 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f12880c0410 0x7f12880c0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:42.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.473 7f128c85d700 1 --2- 172.21.2.101:0/1876357450 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f12881a4a70 0x7f1288094c40 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:42.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.473 7f128c85d700 1 --2- 172.21.2.101:0/1876357450 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f12880bfa10 0x7f12880b82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:42.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.473 7f128c85d700 1 -- 172.21.2.101:0/1876357450 >> 172.21.2.101:0/1876357450 conn(0x7f12880a6a80 msgr2=0x7f12880ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:42.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.473 7f128c85d700 1 -- 172.21.2.101:0/1876357450 shutdown_connections 2021-02-10T11:40:42.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.473 7f128c85d700 1 -- 172.21.2.101:0/1876357450 wait complete. 2021-02-10T11:40:42.484 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --format=json-pretty osd lspools 2021-02-10T11:40:42.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.892 7f97544b3700 1 Processor -- start 2021-02-10T11:40:42.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.892 7f97544b3700 1 -- start start 2021-02-10T11:40:42.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.893 7f97544b3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f974c0d1870 0x7f974c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:42.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.893 7f97544b3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f974c0f0650 0x7f974c0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:42.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.893 7f97544b3700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f974c0f0b10 con 0x7f974c1b0a10 2021-02-10T11:40:42.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.893 7f97544b3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f974c0fe230 con 0x7f974c0f0650 2021-02-10T11:40:42.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.893 7f97544b3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974c0e5ed0 con 0x7f974c0d1870 2021-02-10T11:40:42.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.893 7f9751ace700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f974c1b0a10 0x7f974c0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57146/0 (socket says 172.21.2.101:57146) 2021-02-10T11:40:42.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.893 7f9751ace700 1 -- 172.21.2.101:0/3361527737 learned_addr learned my addr 172.21.2.101:0/3361527737 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:42.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.893 7f9752ad0700 1 --2- 172.21.2.101:0/3361527737 >> v2:172.21.2.108:3300/0 conn(0x7f974c0f0650 0x7f974c0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:42.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.893 7f97522cf700 1 --2- 172.21.2.101:0/3361527737 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f974c0d1870 0x7f974c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:42.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.893 7f97512cd700 1 -- 172.21.2.101:0/3361527737 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3224846886 0 0) 0x7f973c001180 con 0x7f974c1b0a10 2021-02-10T11:40:42.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.893 7f97512cd700 1 -- 172.21.2.101:0/3361527737 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9738002fe0 con 0x7f974c1b0a10 2021-02-10T11:40:42.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.894 7f97522cf700 1 -- 172.21.2.101:0/3361527737 >> v2:172.21.2.108:3300/0 conn(0x7f974c0f0650 msgr2=0x7f974c0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:42.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.894 7f97522cf700 1 --2- 172.21.2.101:0/3361527737 >> v2:172.21.2.108:3300/0 conn(0x7f974c0f0650 0x7f974c0cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:42.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.894 7f97522cf700 1 -- 172.21.2.101:0/3361527737 >> v1:172.21.2.101:6789/0 conn(0x7f974c1b0a10 legacy=0x7f974c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:42.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.894 7f97522cf700 1 -- 172.21.2.101:0/3361527737 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974c0f20a0 con 0x7f974c0d1870 2021-02-10T11:40:42.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.894 7f9752ad0700 1 --2- 172.21.2.101:0/3361527737 >> v2:172.21.2.108:3300/0 conn(0x7f974c0f0650 0x7f974c0cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:40:42.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.894 7f97522cf700 1 --2- 172.21.2.101:0/3361527737 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f974c0d1870 0x7f974c1b03f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f9748006650 tx=0x7f9748009a80).ready entity=mon.2 client_cookie=6a5b65fe039e2ac3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:42.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.894 7f97512cd700 1 -- 172.21.2.101:0/3361527737 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f97480009b0 con 0x7f974c0d1870 2021-02-10T11:40:42.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.894 7f97512cd700 1 -- 172.21.2.101:0/3361527737 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974800c0c0 con 0x7f974c0d1870 2021-02-10T11:40:42.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.894 7f97544b3700 1 -- 172.21.2.101:0/3361527737 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f974c0d1870 msgr2=0x7f974c1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:42.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.894 7f97544b3700 1 --2- 172.21.2.101:0/3361527737 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f974c0d1870 0x7f974c1b03f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f9748006650 tx=0x7f9748009a80).stop 2021-02-10T11:40:42.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.894 7f97512cd700 1 -- 172.21.2.101:0/3361527737 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f974800c5e0 con 0x7f974c0d1870 2021-02-10T11:40:42.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.895 7f97544b3700 1 -- 172.21.2.101:0/3361527737 shutdown_connections 2021-02-10T11:40:42.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.895 7f97544b3700 1 --2- 172.21.2.101:0/3361527737 >> v2:172.21.2.108:3300/0 conn(0x7f974c0f0650 0x7f974c0cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:42.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.895 7f97544b3700 1 --2- 172.21.2.101:0/3361527737 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f974c0d1870 0x7f974c1b03f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:42.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.895 7f97544b3700 1 -- 172.21.2.101:0/3361527737 >> 172.21.2.101:0/3361527737 conn(0x7f974c0ce450 msgr2=0x7f974c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:42.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.895 7f97544b3700 1 -- 172.21.2.101:0/3361527737 shutdown_connections 2021-02-10T11:40:42.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.895 7f97544b3700 1 -- 172.21.2.101:0/3361527737 wait complete. 2021-02-10T11:40:42.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.910 7f97544b3700 1 Processor -- start 2021-02-10T11:40:42.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.910 7f97544b3700 1 -- start start 2021-02-10T11:40:42.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.910 7f97544b3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f974c0ada90 0x7f974c0ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:42.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.910 7f97544b3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f974c099d20 0x7f974c09a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:42.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.910 7f97544b3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f974c09a740 0x7f974c097bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:42.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.910 7f97544b3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c0997a0 con 0x7f974c0ada90 2021-02-10T11:40:42.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.910 7f97544b3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c18f0c0 con 0x7f974c099d20 2021-02-10T11:40:42.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.910 7f97544b3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974c09ab40 con 0x7f974c09a740 2021-02-10T11:40:42.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.910 7f97522cf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f974c0ada90 0x7f974c0ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:42.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.910 7f97522cf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f974c0ada90 0x7f974c0ade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56662/0 (socket says 172.21.2.101:56662) 2021-02-10T11:40:42.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.910 7f97522cf700 1 -- 172.21.2.101:0/1586095653 learned_addr learned my addr 172.21.2.101:0/1586095653 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:42.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.911 7f9752ad0700 1 --2- 172.21.2.101:0/1586095653 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f974c09a740 0x7f974c097bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:42.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.911 7f97522cf700 1 -- 172.21.2.101:0/1586095653 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f974c09a740 msgr2=0x7f974c097bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:42.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.911 7f9751ace700 1 --2- 172.21.2.101:0/1586095653 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f974c099d20 0x7f974c09a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:42.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.911 7f97522cf700 1 --2- 172.21.2.101:0/1586095653 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f974c09a740 0x7f974c097bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:42.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.911 7f97522cf700 1 -- 172.21.2.101:0/1586095653 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f974c099d20 msgr2=0x7f974c09a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:42.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.911 7f97522cf700 1 --2- 172.21.2.101:0/1586095653 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f974c099d20 0x7f974c09a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:42.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.911 7f97522cf700 1 -- 172.21.2.101:0/1586095653 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974c09ad60 con 0x7f974c0ada90 2021-02-10T11:40:42.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.911 7f9752ad0700 1 --2- 172.21.2.101:0/1586095653 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f974c09a740 0x7f974c097bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:40:42.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.911 7f9751ace700 1 --2- 172.21.2.101:0/1586095653 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f974c099d20 0x7f974c09a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:40:42.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.911 7f97522cf700 1 --2- 172.21.2.101:0/1586095653 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f974c0ada90 0x7f974c0ade90 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 rx=0x7f9748009630 tx=0x7f974800b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:42.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.911 7f9742ffd700 1 -- 172.21.2.101:0/1586095653 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9748009360 con 0x7f974c0ada90 2021-02-10T11:40:42.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.911 7f9742ffd700 1 -- 172.21.2.101:0/1586095653 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9748002030 con 0x7f974c0ada90 2021-02-10T11:40:42.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.911 7f9742ffd700 1 -- 172.21.2.101:0/1586095653 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9748002550 con 0x7f974c0ada90 2021-02-10T11:40:42.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.911 7f97544b3700 1 -- 172.21.2.101:0/1586095653 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974c09ad60 con 0x7f974c0ada90 2021-02-10T11:40:42.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.912 7f97544b3700 1 -- 172.21.2.101:0/1586095653 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974c09ad60 con 0x7f974c0ada90 2021-02-10T11:40:42.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.912 7f9742ffd700 1 -- 172.21.2.101:0/1586095653 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9748000c90 con 0x7f974c0ada90 2021-02-10T11:40:42.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.912 7f9742ffd700 1 --2- 172.21.2.101:0/1586095653 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97340186c0 0x7f973401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:42.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.912 7f9742ffd700 1 -- 172.21.2.101:0/1586095653 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f97480324d0 con 0x7f974c0ada90 2021-02-10T11:40:42.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.912 7f9751ace700 1 --2- 172.21.2.101:0/1586095653 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97340186c0 0x7f973401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:42.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.913 7f9751ace700 1 --2- 172.21.2.101:0/1586095653 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97340186c0 0x7f973401abc0 crc :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:42.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.942 7f97544b3700 1 -- 172.21.2.101:0/1586095653 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f974c0ebc00 con 0x7f974c0ada90 2021-02-10T11:40:42.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:42.945 7f9742ffd700 1 -- 172.21.2.101:0/1586095653 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f974801f540 con 0x7f974c0ada90 2021-02-10T11:40:43.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.071 7f97544b3700 1 -- 172.21.2.101:0/1586095653 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f974c0ebc00 con 0x7f974c0ada90 2021-02-10T11:40:43.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.071 7f9742ffd700 1 -- 172.21.2.101:0/1586095653 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v107) v1 ==== 84+0+64 (secure 0 0 0) 0x7f974800bd60 con 0x7f974c0ada90 2021-02-10T11:40:43.074 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:40:43.075 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:40:43.075 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:40:43.075 INFO:teuthology.orchestra.run.gibba001.stdout: "poolnum": 1, 2021-02-10T11:40:43.075 INFO:teuthology.orchestra.run.gibba001.stdout: "poolname": "rbd" 2021-02-10T11:40:43.075 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:43.075 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:40:43.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.073 7f97544b3700 1 -- 172.21.2.101:0/1586095653 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97340186c0 msgr2=0x7f973401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:43.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.073 7f97544b3700 1 --2- 172.21.2.101:0/1586095653 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97340186c0 0x7f973401abc0 crc :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:43.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.073 7f97544b3700 1 -- 172.21.2.101:0/1586095653 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f974c0ada90 msgr2=0x7f974c0ade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:43.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.073 7f97544b3700 1 --2- 172.21.2.101:0/1586095653 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f974c0ada90 0x7f974c0ade90 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 rx=0x7f9748009630 tx=0x7f974800b970).stop 2021-02-10T11:40:43.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.073 7f97544b3700 1 -- 172.21.2.101:0/1586095653 shutdown_connections 2021-02-10T11:40:43.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.073 7f97544b3700 1 --2- 172.21.2.101:0/1586095653 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97340186c0 0x7f973401abc0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:43.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.073 7f97544b3700 1 --2- 172.21.2.101:0/1586095653 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f974c09a740 0x7f974c097bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:43.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.073 7f97544b3700 1 --2- 172.21.2.101:0/1586095653 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f974c099d20 0x7f974c09a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:43.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.073 7f97544b3700 1 --2- 172.21.2.101:0/1586095653 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f974c0ada90 0x7f974c0ade90 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:43.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.073 7f97544b3700 1 -- 172.21.2.101:0/1586095653 >> 172.21.2.101:0/1586095653 conn(0x7f974c0f0550 msgr2=0x7f974c191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:43.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.074 7f97544b3700 1 -- 172.21.2.101:0/1586095653 shutdown_connections 2021-02-10T11:40:43.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.074 7f97544b3700 1 -- 172.21.2.101:0/1586095653 wait complete. 2021-02-10T11:40:43.084 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-02-10T11:40:43.085 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-02-10T11:40:43.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.497 7f1338621700 1 Processor -- start 2021-02-10T11:40:43.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.497 7f1338621700 1 -- start start 2021-02-10T11:40:43.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.498 7f1338621700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13300d1870 0x7f13301b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:43.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.498 7f1338621700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f13300cff70 0x7f1330109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:43.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.498 7f1338621700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1330109a60 con 0x7f13301b0ae0 2021-02-10T11:40:43.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.498 7f1338621700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1330109750 con 0x7f13300cff70 2021-02-10T11:40:43.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.498 7f1338621700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1330088070 con 0x7f13300d1870 2021-02-10T11:40:43.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.498 7f1336c3e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f13300cff70 0x7f1330109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:43.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.498 7f1336c3e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f13300cff70 0x7f1330109130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56666/0 (socket says 172.21.2.101:56666) 2021-02-10T11:40:43.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.498 7f1336c3e700 1 -- 172.21.2.101:0/4230152373 learned_addr learned my addr 172.21.2.101:0/4230152373 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:43.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.498 7f1336c3e700 1 -- 172.21.2.101:0/4230152373 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13300d1870 msgr2=0x7f13301b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:40:43.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.498 7f1336c3e700 1 --2- 172.21.2.101:0/4230152373 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13300d1870 0x7f13301b04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:43.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.498 7f1336c3e700 1 -- 172.21.2.101:0/4230152373 >> v1:172.21.2.108:6789/0 conn(0x7f13301b0ae0 legacy=0x7f1330108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:43.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.498 7f1336c3e700 1 -- 172.21.2.101:0/4230152373 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1330088260 con 0x7f13300cff70 2021-02-10T11:40:43.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.499 7f1336c3e700 1 --2- 172.21.2.101:0/4230152373 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13300cff70 0x7f1330109130 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 rx=0x7f1328006f00 tx=0x7f1328009e90).ready entity=mon.0 client_cookie=bd66af104b46dbbd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:43.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.499 7f133543b700 1 -- 172.21.2.101:0/4230152373 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f132800b300 con 0x7f13300cff70 2021-02-10T11:40:43.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.499 7f133543b700 1 -- 172.21.2.101:0/4230152373 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132800b580 con 0x7f13300cff70 2021-02-10T11:40:43.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.499 7f133543b700 1 -- 172.21.2.101:0/4230152373 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f132800b9d0 con 0x7f13300cff70 2021-02-10T11:40:43.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.499 7f1338621700 1 -- 172.21.2.101:0/4230152373 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13300cff70 msgr2=0x7f1330109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:43.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.499 7f1338621700 1 --2- 172.21.2.101:0/4230152373 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13300cff70 0x7f1330109130 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 rx=0x7f1328006f00 tx=0x7f1328009e90).stop 2021-02-10T11:40:43.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.499 7f1338621700 1 -- 172.21.2.101:0/4230152373 shutdown_connections 2021-02-10T11:40:43.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.499 7f1338621700 1 --2- 172.21.2.101:0/4230152373 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13300cff70 0x7f1330109130 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:43.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.500 7f1338621700 1 --2- 172.21.2.101:0/4230152373 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13300d1870 0x7f13301b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:43.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.500 7f1338621700 1 -- 172.21.2.101:0/4230152373 >> 172.21.2.101:0/4230152373 conn(0x7f13300ce420 msgr2=0x7f13300d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:43.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.500 7f1338621700 1 -- 172.21.2.101:0/4230152373 shutdown_connections 2021-02-10T11:40:43.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.500 7f1338621700 1 -- 172.21.2.101:0/4230152373 wait complete. 2021-02-10T11:40:43.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.514 7f1338621700 1 Processor -- start 2021-02-10T11:40:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.514 7f1338621700 1 -- start start 2021-02-10T11:40:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.515 7f1338621700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13300c3820 0x7f1330084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.515 7f1338621700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13300a0290 0x7f13300852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.515 7f1338621700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1330086270 0x7f1330086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.515 7f1338621700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13300c0700 con 0x7f13300c3820 2021-02-10T11:40:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.515 7f1338621700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13300858f0 con 0x7f1330086270 2021-02-10T11:40:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.515 7f1338621700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1330085b10 con 0x7f13300a0290 2021-02-10T11:40:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.515 7f133643d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13300c3820 0x7f1330084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.515 7f133643d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13300c3820 0x7f1330084cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56672/0 (socket says 172.21.2.101:56672) 2021-02-10T11:40:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.515 7f133643d700 1 -- 172.21.2.101:0/2141986734 learned_addr learned my addr 172.21.2.101:0/2141986734 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.515 7f1336c3e700 1 --2- 172.21.2.101:0/2141986734 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1330086270 0x7f1330086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.515 7f1335c3c700 1 --2- 172.21.2.101:0/2141986734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13300a0290 0x7f13300852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:43.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.515 7f133643d700 1 -- 172.21.2.101:0/2141986734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13300a0290 msgr2=0x7f13300852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:43.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.515 7f133643d700 1 --2- 172.21.2.101:0/2141986734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13300a0290 0x7f13300852d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:43.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.515 7f133643d700 1 -- 172.21.2.101:0/2141986734 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1330086270 msgr2=0x7f1330086670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:43.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.515 7f133643d700 1 --2- 172.21.2.101:0/2141986734 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1330086270 0x7f1330086670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:43.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.515 7f133643d700 1 -- 172.21.2.101:0/2141986734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1330086c90 con 0x7f13300c3820 2021-02-10T11:40:43.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.515 7f1335c3c700 1 --2- 172.21.2.101:0/2141986734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13300a0290 0x7f13300852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:40:43.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.516 7f133643d700 1 --2- 172.21.2.101:0/2141986734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13300c3820 0x7f1330084cb0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 rx=0x7f132c00b5c0 tx=0x7f132c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:43.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.516 7f1326ffd700 1 -- 172.21.2.101:0/2141986734 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f132c00bd50 con 0x7f13300c3820 2021-02-10T11:40:43.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.516 7f1326ffd700 1 -- 172.21.2.101:0/2141986734 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132c006030 con 0x7f13300c3820 2021-02-10T11:40:43.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.516 7f1326ffd700 1 -- 172.21.2.101:0/2141986734 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f132c0054d0 con 0x7f13300c3820 2021-02-10T11:40:43.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.516 7f1338621700 1 -- 172.21.2.101:0/2141986734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1330086c90 con 0x7f13300c3820 2021-02-10T11:40:43.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.516 7f1338621700 1 -- 172.21.2.101:0/2141986734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1330086c90 con 0x7f13300c3820 2021-02-10T11:40:43.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.517 7f1326ffd700 1 -- 172.21.2.101:0/2141986734 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f132c006210 con 0x7f13300c3820 2021-02-10T11:40:43.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.517 7f1326ffd700 1 --2- 172.21.2.101:0/2141986734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13180186c0 0x7f131801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:43.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.517 7f1326ffd700 1 -- 172.21.2.101:0/2141986734 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f132c032490 con 0x7f13300c3820 2021-02-10T11:40:43.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.517 7f1335c3c700 1 --2- 172.21.2.101:0/2141986734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13180186c0 0x7f131801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:43.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.517 7f1335c3c700 1 --2- 172.21.2.101:0/2141986734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13180186c0 0x7f131801abc0 crc :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:43.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.547 7f1338621700 1 -- 172.21.2.101:0/2141986734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1330082a70 con 0x7f13300c3820 2021-02-10T11:40:43.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.550 7f1326ffd700 1 -- 172.21.2.101:0/2141986734 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f132c006b80 con 0x7f13300c3820 2021-02-10T11:40:43.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:43.680 7f1338621700 1 -- 172.21.2.101:0/2141986734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"} v 0) v1 -- 0x7f1330082a70 con 0x7f13300c3820 2021-02-10T11:40:44.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.097 7f1326ffd700 1 -- 172.21.2.101:0/2141986734 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"}]=0 pool 'cephfs_metadata' created v108) v1 ==== 133+0+0 (secure 0 0 0) 0x7f132c006030 con 0x7f13300c3820 2021-02-10T11:40:44.308 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' created 2021-02-10T11:40:44.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.147 7f1338621700 1 -- 172.21.2.101:0/2141986734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13180186c0 msgr2=0x7f131801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:44.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.147 7f1338621700 1 --2- 172.21.2.101:0/2141986734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13180186c0 0x7f131801abc0 crc :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:44.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.147 7f1338621700 1 -- 172.21.2.101:0/2141986734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13300c3820 msgr2=0x7f1330084cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:44.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.147 7f1338621700 1 --2- 172.21.2.101:0/2141986734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13300c3820 0x7f1330084cb0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 rx=0x7f132c00b5c0 tx=0x7f132c00b960).stop 2021-02-10T11:40:44.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.147 7f1338621700 1 -- 172.21.2.101:0/2141986734 shutdown_connections 2021-02-10T11:40:44.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.147 7f1338621700 1 --2- 172.21.2.101:0/2141986734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13180186c0 0x7f131801abc0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:44.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.147 7f1338621700 1 --2- 172.21.2.101:0/2141986734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13300a0290 0x7f13300852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:44.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.147 7f1338621700 1 --2- 172.21.2.101:0/2141986734 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1330086270 0x7f1330086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:44.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.147 7f1338621700 1 --2- 172.21.2.101:0/2141986734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13300c3820 0x7f1330084cb0 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:44.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.147 7f1338621700 1 -- 172.21.2.101:0/2141986734 >> 172.21.2.101:0/2141986734 conn(0x7f13300872c0 msgr2=0x7f1330095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:44.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.148 7f1338621700 1 -- 172.21.2.101:0/2141986734 shutdown_connections 2021-02-10T11:40:44.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.148 7f1338621700 1 -- 172.21.2.101:0/2141986734 wait complete. 2021-02-10T11:40:44.309 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_data 8 2021-02-10T11:40:44.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.722 7f19b2d4d700 1 Processor -- start 2021-02-10T11:40:44.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.722 7f19b2d4d700 1 -- start start 2021-02-10T11:40:44.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.722 7f19b2d4d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19ac0d1870 0x7f19ac1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:44.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.722 7f19b2d4d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ac109a60 con 0x7f19ac0cff70 2021-02-10T11:40:44.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.722 7f19b2d4d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ac088070 con 0x7f19ac1b0ae0 2021-02-10T11:40:44.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.722 7f19b2d4d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac0e5ef0 con 0x7f19ac0d1870 2021-02-10T11:40:44.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.723 7f19b136a700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f19ac0cff70 0x7f19ac109130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57176/0 (socket says 172.21.2.101:57176) 2021-02-10T11:40:44.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.723 7f19b136a700 1 -- 172.21.2.101:0/2447791089 learned_addr learned my addr 172.21.2.101:0/2447791089 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:44.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.723 7f19ab7fe700 1 -- 172.21.2.101:0/2447791089 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141682152 0 0) 0x7f19a4001500 con 0x7f19ac0cff70 2021-02-10T11:40:44.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.723 7f19ab7fe700 1 -- 172.21.2.101:0/2447791089 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1998002fe0 con 0x7f19ac0cff70 2021-02-10T11:40:44.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.723 7f19b0b69700 1 --2- 172.21.2.101:0/2447791089 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19ac0d1870 0x7f19ac1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:44.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.723 7f19ab7fe700 1 -- 172.21.2.101:0/2447791089 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 447619629 0 0) 0x7f19a4001460 con 0x7f19ac0cff70 2021-02-10T11:40:44.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.723 7f19ab7fe700 1 -- 172.21.2.101:0/2447791089 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19ac0d1870 msgr2=0x7f19ac1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:44.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.723 7f19ab7fe700 1 --2- 172.21.2.101:0/2447791089 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19ac0d1870 0x7f19ac1b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:44.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.723 7f19ab7fe700 1 -- 172.21.2.101:0/2447791089 >> v1:172.21.2.108:6789/0 conn(0x7f19ac1b0ae0 legacy=0x7f19ac108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:44.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.723 7f19ab7fe700 1 -- 172.21.2.101:0/2447791089 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ac0d1c70 con 0x7f19ac0cff70 2021-02-10T11:40:44.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.723 7f19ab7fe700 1 -- 172.21.2.101:0/2447791089 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f19a4001980 con 0x7f19ac0cff70 2021-02-10T11:40:44.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.723 7f19ab7fe700 1 -- 172.21.2.101:0/2447791089 >> v1:172.21.2.101:6789/0 conn(0x7f19ac0cff70 legacy=0x7f19ac109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:44.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.723 7f19ab7fe700 1 --2- 172.21.2.101:0/2447791089 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1998003b90 0x7f19980060b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:44.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.723 7f19ab7fe700 1 --2- 172.21.2.101:0/2447791089 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1998006840 0x7f1998008d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:44.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.723 7f19ab7fe700 1 --2- 172.21.2.101:0/2447791089 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19980095d0 0x7f199800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:44.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.723 7f19ab7fe700 1 -- 172.21.2.101:0/2447791089 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199800c310 con 0x7f19980095d0 2021-02-10T11:40:44.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.723 7f19ab7fe700 1 -- 172.21.2.101:0/2447791089 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199800c530 con 0x7f1998003b90 2021-02-10T11:40:44.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.723 7f19ab7fe700 1 -- 172.21.2.101:0/2447791089 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199800c750 con 0x7f1998006840 2021-02-10T11:40:44.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.723 7f19b136a700 1 --2- 172.21.2.101:0/2447791089 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19980095d0 0x7f199800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:44.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.724 7f19b136a700 1 -- 172.21.2.101:0/2447791089 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1998006840 msgr2=0x7f1998008d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:44.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.724 7f19b136a700 1 --2- 172.21.2.101:0/2447791089 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1998006840 0x7f1998008d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:44.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.724 7f19b136a700 1 -- 172.21.2.101:0/2447791089 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1998003b90 msgr2=0x7f19980060b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:44.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.724 7f19b136a700 1 --2- 172.21.2.101:0/2447791089 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1998003b90 0x7f19980060b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:44.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.724 7f19b136a700 1 -- 172.21.2.101:0/2447791089 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199800c9f0 con 0x7f19980095d0 2021-02-10T11:40:44.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.724 7f19b136a700 1 --2- 172.21.2.101:0/2447791089 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19980095d0 0x7f199800ba60 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 rx=0x7f19a4001b60 tx=0x7f19a404e320).ready entity=mon.0 client_cookie=5da3318593c2d6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:44.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.724 7f19ab7fe700 1 -- 172.21.2.101:0/2447791089 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19a4007a30 con 0x7f19980095d0 2021-02-10T11:40:44.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.724 7f19ab7fe700 1 -- 172.21.2.101:0/2447791089 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a40070d0 con 0x7f19980095d0 2021-02-10T11:40:44.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.724 7f19ab7fe700 1 -- 172.21.2.101:0/2447791089 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19a4007560 con 0x7f19980095d0 2021-02-10T11:40:44.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.724 7f19b2d4d700 1 -- 172.21.2.101:0/2447791089 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19980095d0 msgr2=0x7f199800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:44.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.724 7f19b2d4d700 1 --2- 172.21.2.101:0/2447791089 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19980095d0 0x7f199800ba60 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 rx=0x7f19a4001b60 tx=0x7f19a404e320).stop 2021-02-10T11:40:44.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.724 7f19b136a700 1 -- 172.21.2.101:0/2447791089 reap_dead start 2021-02-10T11:40:44.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.725 7f19b2d4d700 1 -- 172.21.2.101:0/2447791089 shutdown_connections 2021-02-10T11:40:44.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.725 7f19b2d4d700 1 -- 172.21.2.101:0/2447791089 >> 172.21.2.101:0/2447791089 conn(0x7f19ac0ce420 msgr2=0x7f19ac0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:44.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.725 7f19b2d4d700 1 -- 172.21.2.101:0/2447791089 shutdown_connections 2021-02-10T11:40:44.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.725 7f19b2d4d700 1 -- 172.21.2.101:0/2447791089 wait complete. 2021-02-10T11:40:44.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.741 7f19b2d4d700 1 Processor -- start 2021-02-10T11:40:44.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.741 7f19b2d4d700 1 -- start start 2021-02-10T11:40:44.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.742 7f19b2d4d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19ac085bc0 0x7f19ac0863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:44.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.742 7f19b2d4d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19ac0954c0 0x7f19ac096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:44.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.742 7f19b2d4d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19ac0958c0 0x7f19ac096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:44.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.742 7f19b2d4d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac095d80 con 0x7f19ac0958c0 2021-02-10T11:40:44.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.742 7f19b2d4d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac0bfa60 con 0x7f19ac085bc0 2021-02-10T11:40:44.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.742 7f19b2d4d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac085fc0 con 0x7f19ac0954c0 2021-02-10T11:40:44.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.742 7f19b136a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19ac0958c0 0x7f19ac096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:44.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.742 7f19b0b69700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19ac085bc0 0x7f19ac0863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:44.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.742 7f19b136a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19ac0958c0 0x7f19ac096920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56694/0 (socket says 172.21.2.101:56694) 2021-02-10T11:40:44.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.742 7f19b136a700 1 -- 172.21.2.101:0/4206417122 learned_addr learned my addr 172.21.2.101:0/4206417122 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:44.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.742 7f19abfff700 1 --2- 172.21.2.101:0/4206417122 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19ac0954c0 0x7f19ac096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:44.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.743 7f19b136a700 1 -- 172.21.2.101:0/4206417122 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19ac0954c0 msgr2=0x7f19ac096300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:44.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.743 7f19b136a700 1 --2- 172.21.2.101:0/4206417122 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19ac0954c0 0x7f19ac096300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:44.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.743 7f19b136a700 1 -- 172.21.2.101:0/4206417122 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19ac085bc0 msgr2=0x7f19ac0863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:44.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.743 7f19b136a700 1 --2- 172.21.2.101:0/4206417122 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19ac085bc0 0x7f19ac0863e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:44.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.743 7f19b136a700 1 -- 172.21.2.101:0/4206417122 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ac086c10 con 0x7f19ac0958c0 2021-02-10T11:40:44.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.743 7f19b0b69700 1 --2- 172.21.2.101:0/4206417122 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19ac085bc0 0x7f19ac0863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:40:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.743 7f19abfff700 1 --2- 172.21.2.101:0/4206417122 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19ac0954c0 0x7f19ac096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:40:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.743 7f19b136a700 1 --2- 172.21.2.101:0/4206417122 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19ac0958c0 0x7f19ac096920 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 rx=0x7f19a404f940 tx=0x7f19a404fce0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.743 7f19a97fa700 1 -- 172.21.2.101:0/4206417122 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19a4004930 con 0x7f19ac0958c0 2021-02-10T11:40:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.743 7f19a97fa700 1 -- 172.21.2.101:0/4206417122 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a4004cb0 con 0x7f19ac0958c0 2021-02-10T11:40:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.743 7f19b2d4d700 1 -- 172.21.2.101:0/4206417122 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ac086c10 con 0x7f19ac0958c0 2021-02-10T11:40:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.743 7f19a97fa700 1 -- 172.21.2.101:0/4206417122 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19a4007350 con 0x7f19ac0958c0 2021-02-10T11:40:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.743 7f19b2d4d700 1 -- 172.21.2.101:0/4206417122 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ac086c10 con 0x7f19ac0958c0 2021-02-10T11:40:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.744 7f19a97fa700 1 -- 172.21.2.101:0/4206417122 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f19a4001860 con 0x7f19ac0958c0 2021-02-10T11:40:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.744 7f19a97fa700 1 --2- 172.21.2.101:0/4206417122 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1990018730 0x7f199001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.744 7f19a97fa700 1 -- 172.21.2.101:0/4206417122 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5107+0+0 (secure 0 0 0) 0x7f19a4076830 con 0x7f19ac0958c0 2021-02-10T11:40:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.744 7f19b0b69700 1 --2- 172.21.2.101:0/4206417122 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1990018730 0x7f199001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:44.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.744 7f19b0b69700 1 --2- 172.21.2.101:0/4206417122 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1990018730 0x7f199001ac10 crc :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:44.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.773 7f19b2d4d700 1 -- 172.21.2.101:0/4206417122 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ac0fe1c0 con 0x7f19ac0958c0 2021-02-10T11:40:44.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.776 7f19a97fa700 1 -- 172.21.2.101:0/4206417122 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f19a4007070 con 0x7f19ac0958c0 2021-02-10T11:40:44.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:44.906 7f19b2d4d700 1 -- 172.21.2.101:0/4206417122 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"} v 0) v1 -- 0x7f19ac0fe1c0 con 0x7f19ac0958c0 2021-02-10T11:40:45.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.110 7f19a97fa700 1 -- 172.21.2.101:0/4206417122 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"}]=0 pool 'cephfs_data' created v110) v1 ==== 125+0+0 (secure 0 0 0) 0x7f19a4004c90 con 0x7f19ac0958c0 2021-02-10T11:40:45.122 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' created 2021-02-10T11:40:45.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.122 7f19b2d4d700 1 -- 172.21.2.101:0/4206417122 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1990018730 msgr2=0x7f199001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:45.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.122 7f19b2d4d700 1 --2- 172.21.2.101:0/4206417122 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1990018730 0x7f199001ac10 crc :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:45.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.122 7f19b2d4d700 1 -- 172.21.2.101:0/4206417122 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19ac0958c0 msgr2=0x7f19ac096920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:45.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.122 7f19b2d4d700 1 --2- 172.21.2.101:0/4206417122 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19ac0958c0 0x7f19ac096920 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 rx=0x7f19a404f940 tx=0x7f19a404fce0).stop 2021-02-10T11:40:45.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.122 7f19b2d4d700 1 -- 172.21.2.101:0/4206417122 shutdown_connections 2021-02-10T11:40:45.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.122 7f19b2d4d700 1 --2- 172.21.2.101:0/4206417122 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1990018730 0x7f199001ac10 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:45.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.122 7f19b2d4d700 1 --2- 172.21.2.101:0/4206417122 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19ac0954c0 0x7f19ac096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:45.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.122 7f19b2d4d700 1 --2- 172.21.2.101:0/4206417122 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19ac0958c0 0x7f19ac096920 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:45.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.122 7f19b2d4d700 1 --2- 172.21.2.101:0/4206417122 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19ac085bc0 0x7f19ac0863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:45.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.122 7f19b2d4d700 1 -- 172.21.2.101:0/4206417122 >> 172.21.2.101:0/4206417122 conn(0x7f19ac0bef00 msgr2=0x7f19ac09d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:45.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.122 7f19b2d4d700 1 -- 172.21.2.101:0/4206417122 shutdown_connections 2021-02-10T11:40:45.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.122 7f19b2d4d700 1 -- 172.21.2.101:0/4206417122 wait complete. 2021-02-10T11:40:45.133 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-02-10T11:40:45.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.547 7f66fa6ba700 1 Processor -- start 2021-02-10T11:40:45.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.547 7f66fa6ba700 1 -- start start 2021-02-10T11:40:45.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.548 7f66fa6ba700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66f40d1c80 0x7f66f40cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:45.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.548 7f66fa6ba700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f40d0180 con 0x7f66f40d1880 2021-02-10T11:40:45.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.548 7f66fa6ba700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f40fe1d0 con 0x7f66f40cf590 2021-02-10T11:40:45.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.548 7f66fa6ba700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f41b0cc0 con 0x7f66f40d1c80 2021-02-10T11:40:45.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.548 7f66f3fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f66f40d1880 0x7f66f41b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57194/0 (socket says 172.21.2.101:57194) 2021-02-10T11:40:45.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.548 7f66f3fff700 1 -- 172.21.2.101:0/939526742 learned_addr learned my addr 172.21.2.101:0/939526742 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:45.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.548 7f66f2ffd700 1 -- 172.21.2.101:0/939526742 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300402763 0 0) 0x7f66e4001180 con 0x7f66f40d1880 2021-02-10T11:40:45.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.548 7f66f2ffd700 1 -- 172.21.2.101:0/939526742 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66dc002fe0 con 0x7f66f40d1880 2021-02-10T11:40:45.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.548 7f66f37fe700 1 --2- 172.21.2.101:0/939526742 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66f40d1c80 0x7f66f40cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:45.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.548 7f66f2ffd700 1 -- 172.21.2.101:0/939526742 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3635334795 0 0) 0x7f66e4001180 con 0x7f66f40d1880 2021-02-10T11:40:45.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.549 7f66f2ffd700 1 -- 172.21.2.101:0/939526742 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66f40d1c80 msgr2=0x7f66f40cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:45.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.549 7f66f2ffd700 1 --2- 172.21.2.101:0/939526742 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66f40d1c80 0x7f66f40cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:45.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.549 7f66f2ffd700 1 -- 172.21.2.101:0/939526742 >> v1:172.21.2.108:6789/0 conn(0x7f66f40cf590 legacy=0x7f66f40cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:45.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.549 7f66f2ffd700 1 -- 172.21.2.101:0/939526742 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f4109a70 con 0x7f66f40d1880 2021-02-10T11:40:45.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.549 7f66f2ffd700 1 -- 172.21.2.101:0/939526742 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f66e4001680 con 0x7f66f40d1880 2021-02-10T11:40:45.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.549 7f66f2ffd700 1 -- 172.21.2.101:0/939526742 >> v1:172.21.2.101:6789/0 conn(0x7f66f40d1880 legacy=0x7f66f41b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:45.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.549 7f66f2ffd700 1 --2- 172.21.2.101:0/939526742 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f66dc003aa0 0x7f66dc005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:45.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.549 7f66f2ffd700 1 --2- 172.21.2.101:0/939526742 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66dc006800 0x7f66dc008cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:45.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.549 7f66f2ffd700 1 --2- 172.21.2.101:0/939526742 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66dc009590 0x7f66dc00bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:45.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.549 7f66f2ffd700 1 -- 172.21.2.101:0/939526742 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc00c2e0 con 0x7f66dc009590 2021-02-10T11:40:45.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.549 7f66f2ffd700 1 -- 172.21.2.101:0/939526742 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc00c500 con 0x7f66dc003aa0 2021-02-10T11:40:45.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.549 7f66f2ffd700 1 -- 172.21.2.101:0/939526742 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc00c720 con 0x7f66dc006800 2021-02-10T11:40:45.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.549 7f66f8cd7700 1 --2- 172.21.2.101:0/939526742 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66dc009590 0x7f66dc00bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:45.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.549 7f66f37fe700 1 --2- 172.21.2.101:0/939526742 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66dc006800 0x7f66dc008cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:45.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.549 7f66f3fff700 1 --2- 172.21.2.101:0/939526742 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f66dc003aa0 0x7f66dc005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:45.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.549 7f66f8cd7700 1 -- 172.21.2.101:0/939526742 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66dc006800 msgr2=0x7f66dc008cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:45.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.549 7f66f8cd7700 1 --2- 172.21.2.101:0/939526742 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66dc006800 0x7f66dc008cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:45.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.549 7f66f8cd7700 1 -- 172.21.2.101:0/939526742 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f66dc003aa0 msgr2=0x7f66dc005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:45.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.549 7f66f8cd7700 1 --2- 172.21.2.101:0/939526742 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f66dc003aa0 0x7f66dc005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:45.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.549 7f66f8cd7700 1 -- 172.21.2.101:0/939526742 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66dc00c9c0 con 0x7f66dc009590 2021-02-10T11:40:45.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.549 7f66f8cd7700 1 --2- 172.21.2.101:0/939526742 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66dc009590 0x7f66dc00bab0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 rx=0x7f66ec04ae40 tx=0x7f66ec04de20).ready entity=mon.0 client_cookie=fd84f51051da1e74 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:45.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.550 7f66f2ffd700 1 -- 172.21.2.101:0/939526742 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f66ec04aa80 con 0x7f66dc009590 2021-02-10T11:40:45.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.550 7f66f2ffd700 1 -- 172.21.2.101:0/939526742 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ec04aa80 con 0x7f66dc009590 2021-02-10T11:40:45.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.550 7f66f2ffd700 1 -- 172.21.2.101:0/939526742 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f66ec001400 con 0x7f66dc009590 2021-02-10T11:40:45.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.550 7f66fa6ba700 1 -- 172.21.2.101:0/939526742 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66dc009590 msgr2=0x7f66dc00bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:45.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.550 7f66fa6ba700 1 --2- 172.21.2.101:0/939526742 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66dc009590 0x7f66dc00bab0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 rx=0x7f66ec04ae40 tx=0x7f66ec04de20).stop 2021-02-10T11:40:45.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.550 7f66f8cd7700 1 -- 172.21.2.101:0/939526742 reap_dead start 2021-02-10T11:40:45.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.550 7f66fa6ba700 1 -- 172.21.2.101:0/939526742 shutdown_connections 2021-02-10T11:40:45.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.550 7f66fa6ba700 1 -- 172.21.2.101:0/939526742 >> 172.21.2.101:0/939526742 conn(0x7f66f40ce430 msgr2=0x7f66f40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:45.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.550 7f66fa6ba700 1 -- 172.21.2.101:0/939526742 shutdown_connections 2021-02-10T11:40:45.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.550 7f66fa6ba700 1 -- 172.21.2.101:0/939526742 wait complete. 2021-02-10T11:40:45.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.565 7f66fa6ba700 1 Processor -- start 2021-02-10T11:40:45.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.565 7f66fa6ba700 1 -- start start 2021-02-10T11:40:45.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.565 7f66fa6ba700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f66f41a3570 0x7f66f41a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:45.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.565 7f66fa6ba700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66f409c9f0 0x7f66f409cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:45.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.565 7f66fa6ba700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66f409d410 0x7f66f4098170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:45.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.565 7f66fa6ba700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f409c470 con 0x7f66f409c9f0 2021-02-10T11:40:45.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.565 7f66fa6ba700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f40b2840 con 0x7f66f41a3570 2021-02-10T11:40:45.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.565 7f66fa6ba700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f40b2a20 con 0x7f66f409d410 2021-02-10T11:40:45.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.566 7f66f37fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66f409c9f0 0x7f66f409cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:45.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.566 7f66f37fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66f409c9f0 0x7f66f409cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56710/0 (socket says 172.21.2.101:56710) 2021-02-10T11:40:45.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.566 7f66f37fe700 1 -- 172.21.2.101:0/2193193585 learned_addr learned my addr 172.21.2.101:0/2193193585 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:45.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.566 7f66f37fe700 1 -- 172.21.2.101:0/2193193585 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66f409d410 msgr2=0x7f66f4098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:45.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.566 7f66f37fe700 1 --2- 172.21.2.101:0/2193193585 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66f409d410 0x7f66f4098170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:45.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.566 7f66f37fe700 1 -- 172.21.2.101:0/2193193585 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f66f41a3570 msgr2=0x7f66f41a3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:45.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.566 7f66f37fe700 1 --2- 172.21.2.101:0/2193193585 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f66f41a3570 0x7f66f41a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:45.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.566 7f66f37fe700 1 -- 172.21.2.101:0/2193193585 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f409d880 con 0x7f66f409c9f0 2021-02-10T11:40:45.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.566 7f66f37fe700 1 --2- 172.21.2.101:0/2193193585 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66f409c9f0 0x7f66f409cdf0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 rx=0x7f66e800b5e0 tx=0x7f66e800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:45.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.566 7f66f0ff9700 1 -- 172.21.2.101:0/2193193585 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f66e800bd70 con 0x7f66f409c9f0 2021-02-10T11:40:45.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.566 7f66f0ff9700 1 -- 172.21.2.101:0/2193193585 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e8002030 con 0x7f66f409c9f0 2021-02-10T11:40:45.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.566 7f66f0ff9700 1 -- 172.21.2.101:0/2193193585 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f66e8002400 con 0x7f66f409c9f0 2021-02-10T11:40:45.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.566 7f66fa6ba700 1 -- 172.21.2.101:0/2193193585 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f409d880 con 0x7f66f409c9f0 2021-02-10T11:40:45.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.566 7f66fa6ba700 1 -- 172.21.2.101:0/2193193585 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f4098a80 con 0x7f66f409c9f0 2021-02-10T11:40:45.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.567 7f66f0ff9700 1 -- 172.21.2.101:0/2193193585 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f66e8000c90 con 0x7f66f409c9f0 2021-02-10T11:40:45.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.567 7f66f0ff9700 1 --2- 172.21.2.101:0/2193193585 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66d40186c0 0x7f66d401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:45.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.567 7f66f0ff9700 1 -- 172.21.2.101:0/2193193585 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5457+0+0 (secure 0 0 0) 0x7f66e8032370 con 0x7f66f409c9f0 2021-02-10T11:40:45.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.567 7f66f3fff700 1 --2- 172.21.2.101:0/2193193585 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66d40186c0 0x7f66d401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:45.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.568 7f66f3fff700 1 --2- 172.21.2.101:0/2193193585 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66d40186c0 0x7f66d401abc0 crc :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:45.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.598 7f66fa6ba700 1 -- 172.21.2.101:0/2193193585 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f4082a70 con 0x7f66f409c9f0 2021-02-10T11:40:45.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.600 7f66f0ff9700 1 -- 172.21.2.101:0/2193193585 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f66e8000940 con 0x7f66f409c9f0 2021-02-10T11:40:45.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.727 7f66fa6ba700 1 -- 172.21.2.101:0/2193193585 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"} v 0) v1 -- 0x7f66f4082a70 con 0x7f66f409c9f0 2021-02-10T11:40:45.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.732 7f66f0ff9700 1 -- 172.21.2.101:0/2193193585 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"}]=0 new fs with metadata pool 22 and data pool 23 v85) v1 ==== 189+0+0 (secure 0 0 0) 0x7f66e8000c90 con 0x7f66f409c9f0 2021-02-10T11:40:45.736 INFO:teuthology.orchestra.run.gibba001.stderr:new fs with metadata pool 22 and data pool 23 2021-02-10T11:40:45.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.737 7f66fa6ba700 1 -- 172.21.2.101:0/2193193585 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66d40186c0 msgr2=0x7f66d401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:45.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.737 7f66fa6ba700 1 --2- 172.21.2.101:0/2193193585 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66d40186c0 0x7f66d401abc0 crc :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:45.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.737 7f66fa6ba700 1 -- 172.21.2.101:0/2193193585 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66f409c9f0 msgr2=0x7f66f409cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:45.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.737 7f66fa6ba700 1 --2- 172.21.2.101:0/2193193585 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66f409c9f0 0x7f66f409cdf0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 rx=0x7f66e800b5e0 tx=0x7f66e800b980).stop 2021-02-10T11:40:45.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.737 7f66fa6ba700 1 -- 172.21.2.101:0/2193193585 shutdown_connections 2021-02-10T11:40:45.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.737 7f66fa6ba700 1 --2- 172.21.2.101:0/2193193585 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f66d40186c0 0x7f66d401abc0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:45.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.737 7f66fa6ba700 1 --2- 172.21.2.101:0/2193193585 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f66f409d410 0x7f66f4098170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:45.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.737 7f66fa6ba700 1 --2- 172.21.2.101:0/2193193585 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f66f409c9f0 0x7f66f409cdf0 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:45.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.737 7f66fa6ba700 1 --2- 172.21.2.101:0/2193193585 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f66f41a3570 0x7f66f41a3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:45.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.737 7f66fa6ba700 1 -- 172.21.2.101:0/2193193585 >> 172.21.2.101:0/2193193585 conn(0x7f66f40c2cd0 msgr2=0x7f66f40b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:45.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.737 7f66fa6ba700 1 -- 172.21.2.101:0/2193193585 shutdown_connections 2021-02-10T11:40:45.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:45.737 7f66fa6ba700 1 -- 172.21.2.101:0/2193193585 wait complete. 2021-02-10T11:40:45.747 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:40:46.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.155 7fd6ddb75700 1 Processor -- start 2021-02-10T11:40:46.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.156 7fd6ddb75700 1 -- start start 2021-02-10T11:40:46.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.156 7fd6ddb75700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd6d80d1870 0x7fd6d81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:46.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.156 7fd6ddb75700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd6d81b0a10 0x7fd6d80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:46.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.156 7fd6ddb75700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6d80d1c70 0x7fd6d80cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:46.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.156 7fd6ddb75700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d80e5ed0 con 0x7fd6d80d1870 2021-02-10T11:40:46.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.156 7fd6ddb75700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d80f0b10 con 0x7fd6d81b0a10 2021-02-10T11:40:46.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.156 7fd6ddb75700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d80cff70 con 0x7fd6d80d1c70 2021-02-10T11:40:46.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.156 7fd6d77fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd6d80d1870 0x7fd6d81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:46.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.156 7fd6d77fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd6d80d1870 0x7fd6d81b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56718/0 (socket says 172.21.2.101:56718) 2021-02-10T11:40:46.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.156 7fd6d77fe700 1 -- 172.21.2.101:0/714906238 learned_addr learned my addr 172.21.2.101:0/714906238 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:46.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.157 7fd6d77fe700 1 -- 172.21.2.101:0/714906238 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6d80d1c70 msgr2=0x7fd6d80cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:46.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.157 7fd6d77fe700 1 --2- 172.21.2.101:0/714906238 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6d80d1c70 0x7fd6d80cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:46.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.157 7fd6d77fe700 1 -- 172.21.2.101:0/714906238 >> v2:172.21.2.108:3300/0 conn(0x7fd6d81b0a10 msgr2=0x7fd6d80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:46.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.157 7fd6d77fe700 1 --2- 172.21.2.101:0/714906238 >> v2:172.21.2.108:3300/0 conn(0x7fd6d81b0a10 0x7fd6d80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:46.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.157 7fd6d77fe700 1 -- 172.21.2.101:0/714906238 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d80d0150 con 0x7fd6d80d1870 2021-02-10T11:40:46.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.157 7fd6d77fe700 1 --2- 172.21.2.101:0/714906238 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6d80d1870 0x7fd6d81b03f0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 rx=0x7fd6c0006700 tx=0x7fd6c0009b80).ready entity=mon.0 client_cookie=ea8628b571a067aa server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:46.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.157 7fd6d67fc700 1 -- 172.21.2.101:0/714906238 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6c000c0c0 con 0x7fd6d80d1870 2021-02-10T11:40:46.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.157 7fd6d67fc700 1 -- 172.21.2.101:0/714906238 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c000c340 con 0x7fd6d80d1870 2021-02-10T11:40:46.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.157 7fd6d67fc700 1 -- 172.21.2.101:0/714906238 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6c000c860 con 0x7fd6d80d1870 2021-02-10T11:40:46.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.157 7fd6ddb75700 1 -- 172.21.2.101:0/714906238 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6d80d1870 msgr2=0x7fd6d81b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:46.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.157 7fd6ddb75700 1 --2- 172.21.2.101:0/714906238 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6d80d1870 0x7fd6d81b03f0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 rx=0x7fd6c0006700 tx=0x7fd6c0009b80).stop 2021-02-10T11:40:46.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.158 7fd6ddb75700 1 -- 172.21.2.101:0/714906238 shutdown_connections 2021-02-10T11:40:46.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.158 7fd6ddb75700 1 --2- 172.21.2.101:0/714906238 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6d80d1c70 0x7fd6d80cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:46.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.158 7fd6ddb75700 1 --2- 172.21.2.101:0/714906238 >> v2:172.21.2.108:3300/0 conn(0x7fd6d81b0a10 0x7fd6d80cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:46.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.158 7fd6ddb75700 1 --2- 172.21.2.101:0/714906238 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6d80d1870 0x7fd6d81b03f0 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:46.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.158 7fd6ddb75700 1 -- 172.21.2.101:0/714906238 >> 172.21.2.101:0/714906238 conn(0x7fd6d80ce450 msgr2=0x7fd6d80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:46.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.158 7fd6ddb75700 1 -- 172.21.2.101:0/714906238 shutdown_connections 2021-02-10T11:40:46.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.158 7fd6ddb75700 1 -- 172.21.2.101:0/714906238 wait complete. 2021-02-10T11:40:46.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.172 7fd6ddb75700 1 Processor -- start 2021-02-10T11:40:46.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.172 7fd6ddb75700 1 -- start start 2021-02-10T11:40:46.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.173 7fd6ddb75700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6d81ac980 0x7fd6d81ace80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:46.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.173 7fd6ddb75700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6d80aa400 0x7fd6d81b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:46.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.173 7fd6ddb75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6d80aa800 0x7fd6d81b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:46.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.173 7fd6ddb75700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d81b6d50 con 0x7fd6d80aa800 2021-02-10T11:40:46.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.173 7fd6ddb75700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d81ad4a0 con 0x7fd6d80aa400 2021-02-10T11:40:46.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.173 7fd6ddb75700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d818f0e0 con 0x7fd6d81ac980 2021-02-10T11:40:46.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.173 7fd6d7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6d80aa800 0x7fd6d81b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:46.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.173 7fd6d7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6d80aa800 0x7fd6d81b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56720/0 (socket says 172.21.2.101:56720) 2021-02-10T11:40:46.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.173 7fd6d7fff700 1 -- 172.21.2.101:0/4276865222 learned_addr learned my addr 172.21.2.101:0/4276865222 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:46.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.173 7fd6d7fff700 1 -- 172.21.2.101:0/4276865222 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6d81ac980 msgr2=0x7fd6d81ace80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:46.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.173 7fd6d7fff700 1 --2- 172.21.2.101:0/4276865222 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6d81ac980 0x7fd6d81ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:46.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.173 7fd6d7fff700 1 -- 172.21.2.101:0/4276865222 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6d80aa400 msgr2=0x7fd6d81b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:46.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.173 7fd6d7fff700 1 --2- 172.21.2.101:0/4276865222 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6d80aa400 0x7fd6d81b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:46.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.173 7fd6d7fff700 1 -- 172.21.2.101:0/4276865222 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d81ac4c0 con 0x7fd6d80aa800 2021-02-10T11:40:46.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.173 7fd6d7fff700 1 --2- 172.21.2.101:0/4276865222 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6d80aa800 0x7fd6d81b78f0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 rx=0x7fd6d0009980 tx=0x7fd6d000bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:46.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.174 7fd6cffff700 1 -- 172.21.2.101:0/4276865222 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6d0003020 con 0x7fd6d80aa800 2021-02-10T11:40:46.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.174 7fd6cffff700 1 -- 172.21.2.101:0/4276865222 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d0005a80 con 0x7fd6d80aa800 2021-02-10T11:40:46.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.174 7fd6cffff700 1 -- 172.21.2.101:0/4276865222 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6d0006030 con 0x7fd6d80aa800 2021-02-10T11:40:46.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.174 7fd6ddb75700 1 -- 172.21.2.101:0/4276865222 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d81ac4c0 con 0x7fd6d80aa800 2021-02-10T11:40:46.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.174 7fd6ddb75700 1 -- 172.21.2.101:0/4276865222 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d81b8200 con 0x7fd6d80aa800 2021-02-10T11:40:46.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.174 7fd6cffff700 1 -- 172.21.2.101:0/4276865222 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd6d0005c60 con 0x7fd6d80aa800 2021-02-10T11:40:46.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.174 7fd6cffff700 1 --2- 172.21.2.101:0/4276865222 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6bc0186c0 0x7fd6bc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:46.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.174 7fd6cffff700 1 -- 172.21.2.101:0/4276865222 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd6d0032bb0 con 0x7fd6d80aa800 2021-02-10T11:40:46.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.175 7fd6d77fe700 1 --2- 172.21.2.101:0/4276865222 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6bc0186c0 0x7fd6bc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:46.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.175 7fd6d77fe700 1 --2- 172.21.2.101:0/4276865222 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6bc0186c0 0x7fd6bc01abc0 crc :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:46.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.206 7fd6ddb75700 1 -- 172.21.2.101:0/4276865222 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d80ebc60 con 0x7fd6d80aa800 2021-02-10T11:40:46.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.208 7fd6cffff700 1 -- 172.21.2.101:0/4276865222 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd6d0005c70 con 0x7fd6d80aa800 2021-02-10T11:40:46.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.333 7fd6ddb75700 1 -- 172.21.2.101:0/4276865222 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6d80f0b10 con 0x7fd6d80aa800 2021-02-10T11:40:46.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.334 7fd6cffff700 1 -- 172.21.2.101:0/4276865222 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+14020 (secure 0 0 0) 0x7fd6d0005a60 con 0x7fd6d80aa800 2021-02-10T11:40:46.335 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:40:46.336 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":111,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:40:45.727919","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:40:43.698421","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2021-02-10 11:40:44.923352","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:40:46.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.335 7fd6ddb75700 1 -- 172.21.2.101:0/4276865222 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6bc0186c0 msgr2=0x7fd6bc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:46.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.335 7fd6ddb75700 1 --2- 172.21.2.101:0/4276865222 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6bc0186c0 0x7fd6bc01abc0 crc :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:46.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.335 7fd6ddb75700 1 -- 172.21.2.101:0/4276865222 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6d80aa800 msgr2=0x7fd6d81b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:46.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.335 7fd6ddb75700 1 --2- 172.21.2.101:0/4276865222 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6d80aa800 0x7fd6d81b78f0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 rx=0x7fd6d0009980 tx=0x7fd6d000bcd0).stop 2021-02-10T11:40:46.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.335 7fd6ddb75700 1 -- 172.21.2.101:0/4276865222 shutdown_connections 2021-02-10T11:40:46.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.335 7fd6ddb75700 1 --2- 172.21.2.101:0/4276865222 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6d80aa800 0x7fd6d81b78f0 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:46.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.335 7fd6ddb75700 1 --2- 172.21.2.101:0/4276865222 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6d80aa400 0x7fd6d81b72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:46.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.335 7fd6ddb75700 1 --2- 172.21.2.101:0/4276865222 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6bc0186c0 0x7fd6bc01abc0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:46.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.335 7fd6ddb75700 1 --2- 172.21.2.101:0/4276865222 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6d81ac980 0x7fd6d81ace80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:46.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.335 7fd6ddb75700 1 -- 172.21.2.101:0/4276865222 >> 172.21.2.101:0/4276865222 conn(0x7fd6d81ac040 msgr2=0x7fd6d8094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:46.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.336 7fd6ddb75700 1 -- 172.21.2.101:0/4276865222 shutdown_connections 2021-02-10T11:40:46.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.336 7fd6ddb75700 1 -- 172.21.2.101:0/4276865222 wait complete. 2021-02-10T11:40:46.346 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:40:46.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.755 7fc7c86dc700 1 Processor -- start 2021-02-10T11:40:46.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.755 7fc7c86dc700 1 -- start start 2021-02-10T11:40:46.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.755 7fc7c86dc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c00d1870 0x7fc7c01b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:46.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.755 7fc7c86dc700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc7c01b0a10 0x7fc7c00cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:46.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.756 7fc7c86dc700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc7c00d1c70 0x7fc7c00cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:46.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.756 7fc7c86dc700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c00fe230 con 0x7fc7c00d1c70 2021-02-10T11:40:46.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.756 7fc7c86dc700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c00e5ed0 con 0x7fc7c01b0a10 2021-02-10T11:40:46.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.756 7fc7c86dc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c00f0b10 con 0x7fc7c00d1870 2021-02-10T11:40:46.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.756 7fc7c6cf9700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc7c00d1c70 0x7fc7c00cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:46.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.756 7fc7c6cf9700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc7c00d1c70 0x7fc7c00cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56728/0 (socket says 172.21.2.101:56728) 2021-02-10T11:40:46.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.756 7fc7c6cf9700 1 -- 172.21.2.101:0/2223261785 learned_addr learned my addr 172.21.2.101:0/2223261785 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:46.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.756 7fc7c64f8700 1 --2- 172.21.2.101:0/2223261785 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c00d1870 0x7fc7c01b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:46.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.756 7fc7c5cf7700 1 --2- 172.21.2.101:0/2223261785 >> v2:172.21.2.108:3300/0 conn(0x7fc7c01b0a10 0x7fc7c00cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:46.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.756 7fc7c5cf7700 1 -- 172.21.2.101:0/2223261785 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c00d1870 msgr2=0x7fc7c01b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:46.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.756 7fc7c5cf7700 1 --2- 172.21.2.101:0/2223261785 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c00d1870 0x7fc7c01b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:46.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.756 7fc7c5cf7700 1 -- 172.21.2.101:0/2223261785 >> v2:172.21.2.101:3300/0 conn(0x7fc7c00d1c70 msgr2=0x7fc7c00cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:46.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.756 7fc7c5cf7700 1 --2- 172.21.2.101:0/2223261785 >> v2:172.21.2.101:3300/0 conn(0x7fc7c00d1c70 0x7fc7c00cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:46.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.756 7fc7c5cf7700 1 -- 172.21.2.101:0/2223261785 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c00cff70 con 0x7fc7c01b0a10 2021-02-10T11:40:46.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.756 7fc7c6cf9700 1 --2- 172.21.2.101:0/2223261785 >> v2:172.21.2.101:3300/0 conn(0x7fc7c00d1c70 0x7fc7c00cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:40:46.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.756 7fc7c5cf7700 1 --2- 172.21.2.101:0/2223261785 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7c01b0a10 0x7fc7c00cf330 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7fc7bc006650 tx=0x7fc7bc009a00).ready entity=mon.1 client_cookie=64a83ca92fadb358 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:46.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.756 7fc7c54f6700 1 -- 172.21.2.101:0/2223261785 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc7bc009df0 con 0x7fc7c01b0a10 2021-02-10T11:40:46.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.756 7fc7c54f6700 1 -- 172.21.2.101:0/2223261785 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7bc0009b0 con 0x7fc7c01b0a10 2021-02-10T11:40:46.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.756 7fc7c54f6700 1 -- 172.21.2.101:0/2223261785 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc7bc00c2b0 con 0x7fc7c01b0a10 2021-02-10T11:40:46.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.756 7fc7c86dc700 1 -- 172.21.2.101:0/2223261785 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7c01b0a10 msgr2=0x7fc7c00cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:46.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.756 7fc7c86dc700 1 --2- 172.21.2.101:0/2223261785 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7c01b0a10 0x7fc7c00cf330 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7fc7bc006650 tx=0x7fc7bc009a00).stop 2021-02-10T11:40:46.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.757 7fc7c86dc700 1 -- 172.21.2.101:0/2223261785 shutdown_connections 2021-02-10T11:40:46.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.757 7fc7c86dc700 1 --2- 172.21.2.101:0/2223261785 >> v2:172.21.2.101:3300/0 conn(0x7fc7c00d1c70 0x7fc7c00cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:46.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.757 7fc7c86dc700 1 --2- 172.21.2.101:0/2223261785 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7c01b0a10 0x7fc7c00cf330 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:46.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.757 7fc7c86dc700 1 --2- 172.21.2.101:0/2223261785 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c00d1870 0x7fc7c01b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:46.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.757 7fc7c86dc700 1 -- 172.21.2.101:0/2223261785 >> 172.21.2.101:0/2223261785 conn(0x7fc7c00ce450 msgr2=0x7fc7c00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:46.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.757 7fc7c86dc700 1 -- 172.21.2.101:0/2223261785 shutdown_connections 2021-02-10T11:40:46.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.757 7fc7c86dc700 1 -- 172.21.2.101:0/2223261785 wait complete. 2021-02-10T11:40:46.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.771 7fc7c86dc700 1 Processor -- start 2021-02-10T11:40:46.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.771 7fc7c86dc700 1 -- start start 2021-02-10T11:40:46.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.772 7fc7c86dc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c01a4a60 0x7fc7c00c1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:46.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.772 7fc7c86dc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c00c2290 0x7fc7c00c2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:46.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.772 7fc7c86dc700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7c00c2cb0 0x7fc7c00bfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:46.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.772 7fc7c86dc700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c00f05b0 con 0x7fc7c01a4a60 2021-02-10T11:40:46.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.772 7fc7c86dc700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c00f07d0 con 0x7fc7c00c2cb0 2021-02-10T11:40:46.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.772 7fc7c86dc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c00c0340 con 0x7fc7c00c2290 2021-02-10T11:40:46.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.772 7fc7c64f8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c01a4a60 0x7fc7c00c1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:46.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.772 7fc7c64f8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c01a4a60 0x7fc7c00c1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56732/0 (socket says 172.21.2.101:56732) 2021-02-10T11:40:46.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.772 7fc7c64f8700 1 -- 172.21.2.101:0/240547655 learned_addr learned my addr 172.21.2.101:0/240547655 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:46.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.772 7fc7c5cf7700 1 --2- 172.21.2.101:0/240547655 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c00c2290 0x7fc7c00c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:46.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.772 7fc7c6cf9700 1 --2- 172.21.2.101:0/240547655 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7c00c2cb0 0x7fc7c00bfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:46.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.773 7fc7c64f8700 1 -- 172.21.2.101:0/240547655 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c00c2290 msgr2=0x7fc7c00c2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:46.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.773 7fc7c64f8700 1 --2- 172.21.2.101:0/240547655 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c00c2290 0x7fc7c00c2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:46.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.773 7fc7c64f8700 1 -- 172.21.2.101:0/240547655 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7c00c2cb0 msgr2=0x7fc7c00bfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:46.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.773 7fc7c64f8700 1 --2- 172.21.2.101:0/240547655 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7c00c2cb0 0x7fc7c00bfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:46.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.773 7fc7c64f8700 1 -- 172.21.2.101:0/240547655 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c00c0560 con 0x7fc7c01a4a60 2021-02-10T11:40:46.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.773 7fc7c64f8700 1 --2- 172.21.2.101:0/240547655 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c01a4a60 0x7fc7c00c1c70 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 rx=0x7fc7b4006be0 tx=0x7fc7b400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:46.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.773 7fc7b2ffd700 1 -- 172.21.2.101:0/240547655 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc7b40064d0 con 0x7fc7c01a4a60 2021-02-10T11:40:46.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.773 7fc7b2ffd700 1 -- 172.21.2.101:0/240547655 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b400bd80 con 0x7fc7c01a4a60 2021-02-10T11:40:46.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.773 7fc7b2ffd700 1 -- 172.21.2.101:0/240547655 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc7b4002030 con 0x7fc7c01a4a60 2021-02-10T11:40:46.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.773 7fc7c86dc700 1 -- 172.21.2.101:0/240547655 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c00c0560 con 0x7fc7c01a4a60 2021-02-10T11:40:46.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.773 7fc7c86dc700 1 -- 172.21.2.101:0/240547655 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c00c0560 con 0x7fc7c01a4a60 2021-02-10T11:40:46.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.774 7fc7b2ffd700 1 -- 172.21.2.101:0/240547655 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc7b4006c90 con 0x7fc7c01a4a60 2021-02-10T11:40:46.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.774 7fc7b2ffd700 1 --2- 172.21.2.101:0/240547655 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7a80186c0 0x7fc7a801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:46.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.774 7fc7b2ffd700 1 -- 172.21.2.101:0/240547655 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc7b4032580 con 0x7fc7c01a4a60 2021-02-10T11:40:46.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.774 7fc7c5cf7700 1 --2- 172.21.2.101:0/240547655 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7a80186c0 0x7fc7a801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:46.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.774 7fc7c5cf7700 1 --2- 172.21.2.101:0/240547655 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7a80186c0 0x7fc7a801abc0 crc :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:46.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.805 7fc7c86dc700 1 -- 172.21.2.101:0/240547655 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c00f0af0 con 0x7fc7c01a4a60 2021-02-10T11:40:46.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.808 7fc7b2ffd700 1 -- 172.21.2.101:0/240547655 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc7b4006820 con 0x7fc7c01a4a60 2021-02-10T11:40:46.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.933 7fc7c86dc700 1 -- 172.21.2.101:0/240547655 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7c00f0af0 con 0x7fc7c01a4a60 2021-02-10T11:40:46.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.934 7fc7b2ffd700 1 -- 172.21.2.101:0/240547655 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+14007 (secure 0 0 0) 0x7fc7b4006c90 con 0x7fc7c01a4a60 2021-02-10T11:40:46.936 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:40:46.936 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":112,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:40:46.730463","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:40:43.698421","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2021-02-10 11:40:44.923352","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:40:46.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.936 7fc7c86dc700 1 -- 172.21.2.101:0/240547655 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7a80186c0 msgr2=0x7fc7a801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:46.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.936 7fc7c86dc700 1 --2- 172.21.2.101:0/240547655 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7a80186c0 0x7fc7a801abc0 crc :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:46.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.936 7fc7c86dc700 1 -- 172.21.2.101:0/240547655 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c01a4a60 msgr2=0x7fc7c00c1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:46.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.936 7fc7c86dc700 1 --2- 172.21.2.101:0/240547655 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c01a4a60 0x7fc7c00c1c70 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 rx=0x7fc7b4006be0 tx=0x7fc7b400b990).stop 2021-02-10T11:40:46.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.936 7fc7c86dc700 1 -- 172.21.2.101:0/240547655 shutdown_connections 2021-02-10T11:40:46.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.936 7fc7c86dc700 1 --2- 172.21.2.101:0/240547655 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7a80186c0 0x7fc7a801abc0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:46.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.936 7fc7c86dc700 1 --2- 172.21.2.101:0/240547655 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc7c00c2290 0x7fc7c00c2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:46.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.936 7fc7c86dc700 1 --2- 172.21.2.101:0/240547655 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc7c00c2cb0 0x7fc7c00bfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:46.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.936 7fc7c86dc700 1 --2- 172.21.2.101:0/240547655 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc7c01a4a60 0x7fc7c00c1c70 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:46.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.936 7fc7c86dc700 1 -- 172.21.2.101:0/240547655 >> 172.21.2.101:0/240547655 conn(0x7fc7c00ebe30 msgr2=0x7fc7c00bf200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:46.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.936 7fc7c86dc700 1 -- 172.21.2.101:0/240547655 shutdown_connections 2021-02-10T11:40:46.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:46.936 7fc7c86dc700 1 -- 172.21.2.101:0/240547655 wait complete. 2021-02-10T11:40:46.946 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-02-10T11:40:47.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.360 7fdf6f40f700 1 Processor -- start 2021-02-10T11:40:47.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.360 7fdf6f40f700 1 -- start start 2021-02-10T11:40:47.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.361 7fdf6f40f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fdf680d1870 0x7fdf681b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:47.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.361 7fdf6f40f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf680fe1c0 con 0x7fdf680cff70 2021-02-10T11:40:47.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.361 7fdf6f40f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf68109a60 con 0x7fdf681b0ae0 2021-02-10T11:40:47.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.361 7fdf6f40f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf68088070 con 0x7fdf680d1870 2021-02-10T11:40:47.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.361 7fdf6da2c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fdf680cff70 0x7fdf68109130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57234/0 (socket says 172.21.2.101:57234) 2021-02-10T11:40:47.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.361 7fdf6da2c700 1 -- 172.21.2.101:0/2552246092 learned_addr learned my addr 172.21.2.101:0/2552246092 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:47.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.361 7fdf5bfff700 1 -- 172.21.2.101:0/2552246092 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1086412498 0 0) 0x7fdf60001500 con 0x7fdf680cff70 2021-02-10T11:40:47.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.361 7fdf5bfff700 1 -- 172.21.2.101:0/2552246092 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf54002fe0 con 0x7fdf680cff70 2021-02-10T11:40:47.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.361 7fdf5bfff700 1 -- 172.21.2.101:0/2552246092 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4091612638 0 0) 0x7fdf60001460 con 0x7fdf680cff70 2021-02-10T11:40:47.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.361 7fdf5bfff700 1 -- 172.21.2.101:0/2552246092 >> v2:172.21.2.108:3300/0 conn(0x7fdf680d1870 msgr2=0x7fdf681b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:47.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.361 7fdf6d22b700 1 --2- 172.21.2.101:0/2552246092 >> v2:172.21.2.108:3300/0 conn(0x7fdf680d1870 0x7fdf681b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:47.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf5bfff700 1 --2- 172.21.2.101:0/2552246092 >> v2:172.21.2.108:3300/0 conn(0x7fdf680d1870 0x7fdf681b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:47.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf5bfff700 1 -- 172.21.2.101:0/2552246092 >> v1:172.21.2.108:6789/0 conn(0x7fdf681b0ae0 legacy=0x7fdf68108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:47.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf5bfff700 1 -- 172.21.2.101:0/2552246092 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf68088260 con 0x7fdf680cff70 2021-02-10T11:40:47.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf5bfff700 1 -- 172.21.2.101:0/2552246092 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fdf60001980 con 0x7fdf680cff70 2021-02-10T11:40:47.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf5bfff700 1 -- 172.21.2.101:0/2552246092 >> v1:172.21.2.101:6789/0 conn(0x7fdf680cff70 legacy=0x7fdf68109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:47.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf5bfff700 1 --2- 172.21.2.101:0/2552246092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf54003af0 0x7fdf54005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:47.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf5bfff700 1 --2- 172.21.2.101:0/2552246092 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf54006820 0x7fdf54008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:47.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf5bfff700 1 --2- 172.21.2.101:0/2552246092 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf54009640 0x7fdf5400bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:47.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf5bfff700 1 -- 172.21.2.101:0/2552246092 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5400c3b0 con 0x7fdf54003af0 2021-02-10T11:40:47.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf5bfff700 1 -- 172.21.2.101:0/2552246092 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5400c5d0 con 0x7fdf54006820 2021-02-10T11:40:47.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf5bfff700 1 -- 172.21.2.101:0/2552246092 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5400c7f0 con 0x7fdf54009640 2021-02-10T11:40:47.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf6d22b700 1 --2- 172.21.2.101:0/2552246092 >> v2:172.21.2.108:3300/0 conn(0x7fdf680d1870 0x7fdf681b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:40:47.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf6d22b700 1 --2- 172.21.2.101:0/2552246092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf54003af0 0x7fdf54005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:47.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf6da2c700 1 --2- 172.21.2.101:0/2552246092 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf54009640 0x7fdf5400bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:47.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf6ca2a700 1 --2- 172.21.2.101:0/2552246092 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf54006820 0x7fdf54008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:47.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf6d22b700 1 -- 172.21.2.101:0/2552246092 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf54009640 msgr2=0x7fdf5400bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:47.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf6d22b700 1 --2- 172.21.2.101:0/2552246092 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf54009640 0x7fdf5400bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:47.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf6d22b700 1 -- 172.21.2.101:0/2552246092 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf54006820 msgr2=0x7fdf54008d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:47.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf6d22b700 1 --2- 172.21.2.101:0/2552246092 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf54006820 0x7fdf54008d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:47.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf6da2c700 1 -- 172.21.2.101:0/2552246092 reap_dead start 2021-02-10T11:40:47.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf6d22b700 1 -- 172.21.2.101:0/2552246092 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5400ca90 con 0x7fdf54003af0 2021-02-10T11:40:47.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf6d22b700 1 --2- 172.21.2.101:0/2552246092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf54003af0 0x7fdf54005fa0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 rx=0x7fdf64006460 tx=0x7fdf6404db30).ready entity=mon.0 client_cookie=60bbbb0306f84356 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:47.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf5bfff700 1 -- 172.21.2.101:0/2552246092 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdf640065d0 con 0x7fdf54003af0 2021-02-10T11:40:47.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.362 7fdf5bfff700 1 -- 172.21.2.101:0/2552246092 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf640500c0 con 0x7fdf54003af0 2021-02-10T11:40:47.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.363 7fdf5bfff700 1 -- 172.21.2.101:0/2552246092 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdf640065d0 con 0x7fdf54003af0 2021-02-10T11:40:47.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.363 7fdf6f40f700 1 -- 172.21.2.101:0/2552246092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf54003af0 msgr2=0x7fdf54005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:47.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.363 7fdf6f40f700 1 --2- 172.21.2.101:0/2552246092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf54003af0 0x7fdf54005fa0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 rx=0x7fdf64006460 tx=0x7fdf6404db30).stop 2021-02-10T11:40:47.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.363 7fdf6f40f700 1 -- 172.21.2.101:0/2552246092 shutdown_connections 2021-02-10T11:40:47.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.363 7fdf6f40f700 1 --2- 172.21.2.101:0/2552246092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf54003af0 0x7fdf54005fa0 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:47.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.363 7fdf6f40f700 1 -- 172.21.2.101:0/2552246092 >> 172.21.2.101:0/2552246092 conn(0x7fdf680ce420 msgr2=0x7fdf680d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:47.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.363 7fdf6f40f700 1 -- 172.21.2.101:0/2552246092 shutdown_connections 2021-02-10T11:40:47.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.363 7fdf6f40f700 1 -- 172.21.2.101:0/2552246092 wait complete. 2021-02-10T11:40:47.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.378 7fdf6f40f700 1 Processor -- start 2021-02-10T11:40:47.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.378 7fdf6f40f700 1 -- start start 2021-02-10T11:40:47.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.378 7fdf6f40f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf68085bc0 0x7fdf680863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:47.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.378 7fdf6f40f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf680954c0 0x7fdf68096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:47.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.378 7fdf6f40f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf680958c0 0x7fdf68096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:47.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.378 7fdf6f40f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf68095d80 con 0x7fdf680958c0 2021-02-10T11:40:47.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.378 7fdf6f40f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf680bfa60 con 0x7fdf68085bc0 2021-02-10T11:40:47.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.378 7fdf6f40f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf68085fc0 con 0x7fdf680954c0 2021-02-10T11:40:47.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.379 7fdf6d22b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf68085bc0 0x7fdf680863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:47.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.379 7fdf6d22b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf68085bc0 0x7fdf680863e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:39002/0 (socket says 172.21.2.101:39002) 2021-02-10T11:40:47.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.379 7fdf6d22b700 1 -- 172.21.2.101:0/2385040867 learned_addr learned my addr 172.21.2.101:0/2385040867 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:47.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.379 7fdf6da2c700 1 --2- 172.21.2.101:0/2385040867 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf680958c0 0x7fdf68096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:47.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.379 7fdf6ca2a700 1 --2- 172.21.2.101:0/2385040867 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf680954c0 0x7fdf68096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:47.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.379 7fdf6d22b700 1 -- 172.21.2.101:0/2385040867 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf680954c0 msgr2=0x7fdf68096300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:47.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.379 7fdf6d22b700 1 --2- 172.21.2.101:0/2385040867 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf680954c0 0x7fdf68096300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:47.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.379 7fdf6d22b700 1 -- 172.21.2.101:0/2385040867 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf680958c0 msgr2=0x7fdf68096920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:47.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.379 7fdf6d22b700 1 --2- 172.21.2.101:0/2385040867 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf680958c0 0x7fdf68096920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:47.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.379 7fdf6d22b700 1 -- 172.21.2.101:0/2385040867 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf68086c10 con 0x7fdf68085bc0 2021-02-10T11:40:47.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.379 7fdf6d22b700 1 --2- 172.21.2.101:0/2385040867 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf68085bc0 0x7fdf680863e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fdf64050560 tx=0x7fdf64050900).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:47.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.380 7fdf59ffb700 1 -- 172.21.2.101:0/2385040867 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdf640065d0 con 0x7fdf68085bc0 2021-02-10T11:40:47.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.380 7fdf59ffb700 1 -- 172.21.2.101:0/2385040867 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf64050cf0 con 0x7fdf68085bc0 2021-02-10T11:40:47.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.380 7fdf6f40f700 1 -- 172.21.2.101:0/2385040867 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf68086c10 con 0x7fdf68085bc0 2021-02-10T11:40:47.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.380 7fdf59ffb700 1 -- 172.21.2.101:0/2385040867 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdf64002030 con 0x7fdf68085bc0 2021-02-10T11:40:47.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.380 7fdf6f40f700 1 -- 172.21.2.101:0/2385040867 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf68086c10 con 0x7fdf68085bc0 2021-02-10T11:40:47.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.380 7fdf59ffb700 1 -- 172.21.2.101:0/2385040867 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdf64000c90 con 0x7fdf68085bc0 2021-02-10T11:40:47.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.380 7fdf59ffb700 1 --2- 172.21.2.101:0/2385040867 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf48018700 0x7fdf4801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:47.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.380 7fdf59ffb700 1 -- 172.21.2.101:0/2385040867 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fdf640766d0 con 0x7fdf68085bc0 2021-02-10T11:40:47.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.381 7fdf6ca2a700 1 --2- 172.21.2.101:0/2385040867 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf48018700 0x7fdf4801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:47.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.381 7fdf6ca2a700 1 --2- 172.21.2.101:0/2385040867 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf48018700 0x7fdf4801abc0 crc :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:47.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.410 7fdf6f40f700 1 -- 172.21.2.101:0/2385040867 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf680fe1c0 con 0x7fdf68085bc0 2021-02-10T11:40:47.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.413 7fdf59ffb700 1 -- 172.21.2.101:0/2385040867 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdf6404e020 con 0x7fdf68085bc0 2021-02-10T11:40:47.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.548 7fdf6f40f700 1 -- 172.21.2.101:0/2385040867 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"} v 0) v1 -- 0x7fdf680fe1c0 con 0x7fdf68085bc0 2021-02-10T11:40:47.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.742 7fdf59ffb700 1 -- 172.21.2.101:0/2385040867 <== mon.1 v2:172.21.2.108:3300/0 7 ==== mon_command_ack([{"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"}]=0 v88) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdf64000c90 con 0x7fdf68085bc0 2021-02-10T11:40:47.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.766 7fdf6f40f700 1 -- 172.21.2.101:0/2385040867 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf48018700 msgr2=0x7fdf4801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:47.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.766 7fdf6f40f700 1 --2- 172.21.2.101:0/2385040867 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf48018700 0x7fdf4801abc0 crc :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:47.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.766 7fdf6f40f700 1 -- 172.21.2.101:0/2385040867 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf68085bc0 msgr2=0x7fdf680863e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:47.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.766 7fdf6f40f700 1 --2- 172.21.2.101:0/2385040867 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf68085bc0 0x7fdf680863e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fdf64050560 tx=0x7fdf64050900).stop 2021-02-10T11:40:47.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.766 7fdf6f40f700 1 -- 172.21.2.101:0/2385040867 shutdown_connections 2021-02-10T11:40:47.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.766 7fdf6f40f700 1 --2- 172.21.2.101:0/2385040867 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf48018700 0x7fdf4801abc0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:47.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.766 7fdf6f40f700 1 --2- 172.21.2.101:0/2385040867 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf680954c0 0x7fdf68096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:47.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.766 7fdf6f40f700 1 --2- 172.21.2.101:0/2385040867 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf680958c0 0x7fdf68096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:47.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.766 7fdf6f40f700 1 --2- 172.21.2.101:0/2385040867 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf68085bc0 0x7fdf680863e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:47.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.766 7fdf6f40f700 1 -- 172.21.2.101:0/2385040867 >> 172.21.2.101:0/2385040867 conn(0x7fdf680bef00 msgr2=0x7fdf6809d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:47.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.766 7fdf6f40f700 1 -- 172.21.2.101:0/2385040867 shutdown_connections 2021-02-10T11:40:47.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:47.766 7fdf6f40f700 1 -- 172.21.2.101:0/2385040867 wait complete. 2021-02-10T11:40:47.776 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:40:48.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.186 7fa23a545700 1 Processor -- start 2021-02-10T11:40:48.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.186 7fa23a545700 1 -- start start 2021-02-10T11:40:48.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.187 7fa23a545700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa2340d1870 0x7fa2341b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:48.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.187 7fa23a545700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa2341b0a10 0x7fa2340cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:48.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.187 7fa23a545700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2340d1c70 0x7fa2340cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:48.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.187 7fa23a545700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2340e5ed0 con 0x7fa2341b0a10 2021-02-10T11:40:48.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.187 7fa23a545700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2340f0b10 con 0x7fa2340d1870 2021-02-10T11:40:48.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.187 7fa23a545700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2340cff70 con 0x7fa2340d1c70 2021-02-10T11:40:48.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.187 7fa2337fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa2341b0a10 0x7fa2340cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:48.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.187 7fa2337fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa2341b0a10 0x7fa2340cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56758/0 (socket says 172.21.2.101:56758) 2021-02-10T11:40:48.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.187 7fa2337fe700 1 -- 172.21.2.101:0/2597834808 learned_addr learned my addr 172.21.2.101:0/2597834808 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:48.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.187 7fa238b62700 1 --2- 172.21.2.101:0/2597834808 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2340d1c70 0x7fa2340cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:48.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.187 7fa233fff700 1 --2- 172.21.2.101:0/2597834808 >> v2:172.21.2.108:3300/0 conn(0x7fa2340d1870 0x7fa2341b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:48.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.187 7fa2337fe700 1 -- 172.21.2.101:0/2597834808 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2340d1c70 msgr2=0x7fa2340cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:48.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.187 7fa2337fe700 1 --2- 172.21.2.101:0/2597834808 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2340d1c70 0x7fa2340cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:48.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.189 7fa2337fe700 1 -- 172.21.2.101:0/2597834808 >> v2:172.21.2.108:3300/0 conn(0x7fa2340d1870 msgr2=0x7fa2341b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:48.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.189 7fa2337fe700 1 --2- 172.21.2.101:0/2597834808 >> v2:172.21.2.108:3300/0 conn(0x7fa2340d1870 0x7fa2341b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:48.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.189 7fa2337fe700 1 -- 172.21.2.101:0/2597834808 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2340d0150 con 0x7fa2341b0a10 2021-02-10T11:40:48.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.189 7fa233fff700 1 --2- 172.21.2.101:0/2597834808 >> v2:172.21.2.108:3300/0 conn(0x7fa2340d1870 0x7fa2341b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:40:48.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.189 7fa2337fe700 1 --2- 172.21.2.101:0/2597834808 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2341b0a10 0x7fa2340cf330 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 rx=0x7fa228006700 tx=0x7fa228009b80).ready entity=mon.0 client_cookie=36e0aea656526b38 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:48.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.189 7fa232ffd700 1 -- 172.21.2.101:0/2597834808 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa22800c0c0 con 0x7fa2341b0a10 2021-02-10T11:40:48.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.189 7fa232ffd700 1 -- 172.21.2.101:0/2597834808 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22800c440 con 0x7fa2341b0a10 2021-02-10T11:40:48.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.189 7fa232ffd700 1 -- 172.21.2.101:0/2597834808 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa22800c960 con 0x7fa2341b0a10 2021-02-10T11:40:48.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.189 7fa23a545700 1 -- 172.21.2.101:0/2597834808 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2341b0a10 msgr2=0x7fa2340cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:48.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.189 7fa23a545700 1 --2- 172.21.2.101:0/2597834808 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2341b0a10 0x7fa2340cf330 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 rx=0x7fa228006700 tx=0x7fa228009b80).stop 2021-02-10T11:40:48.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.189 7fa23a545700 1 -- 172.21.2.101:0/2597834808 shutdown_connections 2021-02-10T11:40:48.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.189 7fa23a545700 1 --2- 172.21.2.101:0/2597834808 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2340d1c70 0x7fa2340cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:48.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.189 7fa23a545700 1 --2- 172.21.2.101:0/2597834808 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2341b0a10 0x7fa2340cf330 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:48.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.189 7fa23a545700 1 --2- 172.21.2.101:0/2597834808 >> v2:172.21.2.108:3300/0 conn(0x7fa2340d1870 0x7fa2341b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:48.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.189 7fa23a545700 1 -- 172.21.2.101:0/2597834808 >> 172.21.2.101:0/2597834808 conn(0x7fa2340ce450 msgr2=0x7fa2340d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:48.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.189 7fa23a545700 1 -- 172.21.2.101:0/2597834808 shutdown_connections 2021-02-10T11:40:48.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.189 7fa23a545700 1 -- 172.21.2.101:0/2597834808 wait complete. 2021-02-10T11:40:48.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.203 7fa23a545700 1 Processor -- start 2021-02-10T11:40:48.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.203 7fa23a545700 1 -- start start 2021-02-10T11:40:48.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.203 7fa23a545700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2341942b0 0x7fa2340b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:48.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.203 7fa23a545700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa234195070 0x7fa234195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:48.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.203 7fa23a545700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2340bb8e0 0x7fa2340bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:48.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.203 7fa23a545700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2341946b0 con 0x7fa2341942b0 2021-02-10T11:40:48.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.203 7fa23a545700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2340b3b70 con 0x7fa234195070 2021-02-10T11:40:48.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.203 7fa23a545700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2340b3d90 con 0x7fa2340bb8e0 2021-02-10T11:40:48.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.205 7fa233fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2341942b0 0x7fa2340b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:48.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.205 7fa233fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2341942b0 0x7fa2340b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56764/0 (socket says 172.21.2.101:56764) 2021-02-10T11:40:48.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.205 7fa233fff700 1 -- 172.21.2.101:0/3919853466 learned_addr learned my addr 172.21.2.101:0/3919853466 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:48.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.205 7fa233fff700 1 -- 172.21.2.101:0/3919853466 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2340bb8e0 msgr2=0x7fa2340bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:48.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.205 7fa233fff700 1 --2- 172.21.2.101:0/3919853466 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2340bb8e0 0x7fa2340bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:48.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.205 7fa233fff700 1 -- 172.21.2.101:0/3919853466 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa234195070 msgr2=0x7fa234195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:48.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.205 7fa233fff700 1 --2- 172.21.2.101:0/3919853466 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa234195070 0x7fa234195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:48.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.205 7fa233fff700 1 -- 172.21.2.101:0/3919853466 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2341a49e0 con 0x7fa2341942b0 2021-02-10T11:40:48.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.205 7fa233fff700 1 --2- 172.21.2.101:0/3919853466 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2341942b0 0x7fa2340b3550 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 rx=0x7fa22400b580 tx=0x7fa22400b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:48.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.205 7fa230ff9700 1 -- 172.21.2.101:0/3919853466 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa22400bd10 con 0x7fa2341942b0 2021-02-10T11:40:48.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.205 7fa230ff9700 1 -- 172.21.2.101:0/3919853466 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa224000c90 con 0x7fa2341942b0 2021-02-10T11:40:48.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.205 7fa23a545700 1 -- 172.21.2.101:0/3919853466 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2341a49e0 con 0x7fa2341942b0 2021-02-10T11:40:48.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.205 7fa230ff9700 1 -- 172.21.2.101:0/3919853466 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa2240022b0 con 0x7fa2341942b0 2021-02-10T11:40:48.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.205 7fa23a545700 1 -- 172.21.2.101:0/3919853466 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2341a49e0 con 0x7fa2341942b0 2021-02-10T11:40:48.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.205 7fa230ff9700 1 -- 172.21.2.101:0/3919853466 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa22400a000 con 0x7fa2341942b0 2021-02-10T11:40:48.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.205 7fa230ff9700 1 --2- 172.21.2.101:0/3919853466 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2140186d0 0x7fa21401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:48.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.206 7fa230ff9700 1 -- 172.21.2.101:0/3919853466 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa2240325e0 con 0x7fa2341942b0 2021-02-10T11:40:48.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.206 7fa2337fe700 1 --2- 172.21.2.101:0/3919853466 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2140186d0 0x7fa21401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:48.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.206 7fa2337fe700 1 --2- 172.21.2.101:0/3919853466 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2140186d0 0x7fa21401abd0 crc :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:48.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.237 7fa23a545700 1 -- 172.21.2.101:0/3919853466 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2340bc5f0 con 0x7fa2341942b0 2021-02-10T11:40:48.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.239 7fa230ff9700 1 -- 172.21.2.101:0/3919853466 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa224006bb0 con 0x7fa2341942b0 2021-02-10T11:40:48.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.367 7fa23a545700 1 -- 172.21.2.101:0/3919853466 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2340bc5f0 con 0x7fa2341942b0 2021-02-10T11:40:48.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.368 7fa230ff9700 1 -- 172.21.2.101:0/3919853466 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+2104 (secure 0 0 0) 0x7fa224000c90 con 0x7fa2341942b0 2021-02-10T11:40:48.371 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:40:48.371 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":88,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2840404451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2840404451},{"type":"v1","addr":"172.21.2.101:6835","nonce":2840404451}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":84}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:40:45.728009","modified":"2021-02-10 11:40:47.738972","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7450},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7450":{"gid":7450,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2757440614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2757440614},{"type":"v1","addr":"172.21.2.108:6833","nonce":2757440614}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2021-02-10T11:40:48.371 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 88 2021-02-10T11:40:48.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.369 7fa23a545700 1 -- 172.21.2.101:0/3919853466 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2140186d0 msgr2=0x7fa21401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:48.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.369 7fa23a545700 1 --2- 172.21.2.101:0/3919853466 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2140186d0 0x7fa21401abd0 crc :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:48.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.369 7fa23a545700 1 -- 172.21.2.101:0/3919853466 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2341942b0 msgr2=0x7fa2340b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:48.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.369 7fa23a545700 1 --2- 172.21.2.101:0/3919853466 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2341942b0 0x7fa2340b3550 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 rx=0x7fa22400b580 tx=0x7fa22400b920).stop 2021-02-10T11:40:48.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.371 7fa23a545700 1 -- 172.21.2.101:0/3919853466 shutdown_connections 2021-02-10T11:40:48.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.371 7fa23a545700 1 --2- 172.21.2.101:0/3919853466 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2140186d0 0x7fa21401abd0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:48.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.371 7fa23a545700 1 --2- 172.21.2.101:0/3919853466 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2340bb8e0 0x7fa2340bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:48.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.371 7fa23a545700 1 --2- 172.21.2.101:0/3919853466 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa234195070 0x7fa234195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:48.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.371 7fa23a545700 1 --2- 172.21.2.101:0/3919853466 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2341942b0 0x7fa2340b3550 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:48.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.371 7fa23a545700 1 -- 172.21.2.101:0/3919853466 >> 172.21.2.101:0/3919853466 conn(0x7fa2340b3040 msgr2=0x7fa2340c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:48.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.371 7fa23a545700 1 -- 172.21.2.101:0/3919853466 shutdown_connections 2021-02-10T11:40:48.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.371 7fa23a545700 1 -- 172.21.2.101:0/3919853466 wait complete. 2021-02-10T11:40:48.381 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:40:48.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.792 7f00ef21d700 1 Processor -- start 2021-02-10T11:40:48.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.792 7f00ef21d700 1 -- start start 2021-02-10T11:40:48.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.792 7f00ef21d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f00e80d0340 0x7f00e80ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:48.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.792 7f00ef21d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f00e80d1c70 0x7f00e80efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:48.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.792 7f00ef21d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00e80f0b10 con 0x7f00e80d1870 2021-02-10T11:40:48.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.792 7f00ef21d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00e80e5ed0 con 0x7f00e80d0340 2021-02-10T11:40:48.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.792 7f00ef21d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00e81b0be0 con 0x7f00e80d1c70 2021-02-10T11:40:48.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.793 7f00ec838700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f00e80d0340 0x7f00e80ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:48.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.793 7f00ec838700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f00e80d0340 0x7f00e80ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56770/0 (socket says 172.21.2.101:56770) 2021-02-10T11:40:48.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.793 7f00ec838700 1 -- 172.21.2.101:0/3904374820 learned_addr learned my addr 172.21.2.101:0/3904374820 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:48.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.793 7f00ec838700 1 -- 172.21.2.101:0/3904374820 >> v2:172.21.2.108:3300/0 conn(0x7f00e80d1c70 msgr2=0x7f00e80efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:48.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.793 7f00ec838700 1 --2- 172.21.2.101:0/3904374820 >> v2:172.21.2.108:3300/0 conn(0x7f00e80d1c70 0x7f00e80efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:48.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.793 7f00ec838700 1 -- 172.21.2.101:0/3904374820 >> v1:172.21.2.108:6789/0 conn(0x7f00e80d1870 legacy=0x7f00e81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:48.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.793 7f00ec838700 1 -- 172.21.2.101:0/3904374820 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e80f20a0 con 0x7f00e80d0340 2021-02-10T11:40:48.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.793 7f00ec838700 1 --2- 172.21.2.101:0/3904374820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00e80d0340 0x7f00e80ef640 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 rx=0x7f00d8006700 tx=0x7f00d8009b80).ready entity=mon.0 client_cookie=dba5fa9b6d3b753e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:48.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.793 7f00dffff700 1 -- 172.21.2.101:0/3904374820 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f00d800c0c0 con 0x7f00e80d0340 2021-02-10T11:40:48.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.795 7f00dffff700 1 -- 172.21.2.101:0/3904374820 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d800c440 con 0x7f00e80d0340 2021-02-10T11:40:48.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.795 7f00dffff700 1 -- 172.21.2.101:0/3904374820 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f00d800c8a0 con 0x7f00e80d0340 2021-02-10T11:40:48.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.795 7f00ef21d700 1 -- 172.21.2.101:0/3904374820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00e80d0340 msgr2=0x7f00e80ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:48.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.795 7f00ef21d700 1 --2- 172.21.2.101:0/3904374820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00e80d0340 0x7f00e80ef640 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 rx=0x7f00d8006700 tx=0x7f00d8009b80).stop 2021-02-10T11:40:48.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.795 7f00ef21d700 1 -- 172.21.2.101:0/3904374820 shutdown_connections 2021-02-10T11:40:48.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.795 7f00ef21d700 1 --2- 172.21.2.101:0/3904374820 >> v2:172.21.2.108:3300/0 conn(0x7f00e80d1c70 0x7f00e80efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:48.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.795 7f00ef21d700 1 --2- 172.21.2.101:0/3904374820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00e80d0340 0x7f00e80ef640 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:48.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.795 7f00ef21d700 1 -- 172.21.2.101:0/3904374820 >> 172.21.2.101:0/3904374820 conn(0x7f00e80ce450 msgr2=0x7f00e80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:48.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.795 7f00ef21d700 1 -- 172.21.2.101:0/3904374820 shutdown_connections 2021-02-10T11:40:48.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.795 7f00ef21d700 1 -- 172.21.2.101:0/3904374820 wait complete. 2021-02-10T11:40:48.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.809 7f00ef21d700 1 Processor -- start 2021-02-10T11:40:48.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.809 7f00ef21d700 1 -- start start 2021-02-10T11:40:48.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.810 7f00ef21d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00e818f0a0 0x7f00e8196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:48.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.810 7f00ef21d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00e81970d0 0x7f00e81974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:48.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.810 7f00ef21d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f00e8094650 0x7f00e8094a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:48.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.810 7f00ef21d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e8095bd0 con 0x7f00e81970d0 2021-02-10T11:40:48.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.810 7f00ef21d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e80ad7c0 con 0x7f00e8094650 2021-02-10T11:40:48.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.810 7f00ef21d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e818ec70 con 0x7f00e818f0a0 2021-02-10T11:40:48.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.810 7f00ec838700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00e81970d0 0x7f00e81974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:48.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.810 7f00ec838700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00e81970d0 0x7f00e81974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56778/0 (socket says 172.21.2.101:56778) 2021-02-10T11:40:48.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.810 7f00ec838700 1 -- 172.21.2.101:0/1603289191 learned_addr learned my addr 172.21.2.101:0/1603289191 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:48.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.810 7f00ec838700 1 -- 172.21.2.101:0/1603289191 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00e818f0a0 msgr2=0x7f00e8196ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:48.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.810 7f00ec838700 1 --2- 172.21.2.101:0/1603289191 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00e818f0a0 0x7f00e8196ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:48.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.810 7f00ec838700 1 -- 172.21.2.101:0/1603289191 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f00e8094650 msgr2=0x7f00e8094a50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:48.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.810 7f00ec838700 1 --2- 172.21.2.101:0/1603289191 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f00e8094650 0x7f00e8094a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:48.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.810 7f00ec838700 1 -- 172.21.2.101:0/1603289191 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e8095070 con 0x7f00e81970d0 2021-02-10T11:40:48.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.810 7f00ec838700 1 --2- 172.21.2.101:0/1603289191 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00e81970d0 0x7f00e81974d0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 rx=0x7f00d800b5e0 tx=0x7f00d800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:48.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.810 7f00ddffb700 1 -- 172.21.2.101:0/1603289191 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f00d800bd70 con 0x7f00e81970d0 2021-02-10T11:40:48.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.811 7f00ddffb700 1 -- 172.21.2.101:0/1603289191 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d8002030 con 0x7f00e81970d0 2021-02-10T11:40:48.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.811 7f00ef21d700 1 -- 172.21.2.101:0/1603289191 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e8095070 con 0x7f00e81970d0 2021-02-10T11:40:48.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.811 7f00ddffb700 1 -- 172.21.2.101:0/1603289191 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f00d80024d0 con 0x7f00e81970d0 2021-02-10T11:40:48.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.811 7f00ef21d700 1 -- 172.21.2.101:0/1603289191 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e8095600 con 0x7f00e81970d0 2021-02-10T11:40:48.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.811 7f00ddffb700 1 -- 172.21.2.101:0/1603289191 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f00d8002210 con 0x7f00e81970d0 2021-02-10T11:40:48.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.811 7f00ddffb700 1 --2- 172.21.2.101:0/1603289191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f00c80186c0 0x7f00c801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:48.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.811 7f00ddffb700 1 -- 172.21.2.101:0/1603289191 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f00d80323b0 con 0x7f00e81970d0 2021-02-10T11:40:48.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.811 7f00ed039700 1 --2- 172.21.2.101:0/1603289191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f00c80186c0 0x7f00c801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:48.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.812 7f00ed039700 1 --2- 172.21.2.101:0/1603289191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f00c80186c0 0x7f00c801abc0 crc :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:48.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.843 7f00ef21d700 1 -- 172.21.2.101:0/1603289191 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e80ad790 con 0x7f00e81970d0 2021-02-10T11:40:48.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.845 7f00ddffb700 1 -- 172.21.2.101:0/1603289191 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f00d8030ce0 con 0x7f00e81970d0 2021-02-10T11:40:48.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.970 7f00ef21d700 1 -- 172.21.2.101:0/1603289191 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00e80ad790 con 0x7f00e81970d0 2021-02-10T11:40:48.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.970 7f00ddffb700 1 -- 172.21.2.101:0/1603289191 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f00d8002210 con 0x7f00e81970d0 2021-02-10T11:40:48.973 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:40:48.973 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":112,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:40:46.730463","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:40:43.698421","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2021-02-10 11:40:44.923352","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:40:48.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.972 7f00ef21d700 1 -- 172.21.2.101:0/1603289191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f00c80186c0 msgr2=0x7f00c801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:48.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.972 7f00ef21d700 1 --2- 172.21.2.101:0/1603289191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f00c80186c0 0x7f00c801abc0 crc :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:48.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.972 7f00ef21d700 1 -- 172.21.2.101:0/1603289191 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00e81970d0 msgr2=0x7f00e81974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:48.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.972 7f00ef21d700 1 --2- 172.21.2.101:0/1603289191 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00e81970d0 0x7f00e81974d0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 rx=0x7f00d800b5e0 tx=0x7f00d800b980).stop 2021-02-10T11:40:48.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.972 7f00ef21d700 1 -- 172.21.2.101:0/1603289191 shutdown_connections 2021-02-10T11:40:48.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.972 7f00ef21d700 1 --2- 172.21.2.101:0/1603289191 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f00e8094650 0x7f00e8094a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:48.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.972 7f00ef21d700 1 --2- 172.21.2.101:0/1603289191 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f00e81970d0 0x7f00e81974d0 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:48.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.972 7f00ef21d700 1 --2- 172.21.2.101:0/1603289191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f00c80186c0 0x7f00c801abc0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:48.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.972 7f00ef21d700 1 --2- 172.21.2.101:0/1603289191 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f00e818f0a0 0x7f00e8196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:48.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.972 7f00ef21d700 1 -- 172.21.2.101:0/1603289191 >> 172.21.2.101:0/1603289191 conn(0x7f00e81a38c0 msgr2=0x7f00e81a2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:48.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.972 7f00ef21d700 1 -- 172.21.2.101:0/1603289191 shutdown_connections 2021-02-10T11:40:48.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:48.972 7f00ef21d700 1 -- 172.21.2.101:0/1603289191 wait complete. 2021-02-10T11:40:48.990 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:40:49.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.411 7f677dc02700 1 Processor -- start 2021-02-10T11:40:49.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.411 7f677dc02700 1 -- start start 2021-02-10T11:40:49.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.411 7f677dc02700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f67780d1880 0x7f67781b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:49.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.411 7f677dc02700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f67781b0af0 0x7f6778108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:49.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.411 7f677dc02700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67780fe1d0 con 0x7f67780d1c80 2021-02-10T11:40:49.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.411 7f677dc02700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6778109a70 con 0x7f67780d1880 2021-02-10T11:40:49.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.411 7f677dc02700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6778109760 con 0x7f67781b0af0 2021-02-10T11:40:49.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.411 7f67777fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f67780d1880 0x7f67781b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:49.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.411 7f676effd700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f67781b0af0 0x7f6778108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:49.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.411 7f6777fff700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f67780d1c80 0x7f6778108f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:50692/0 (socket says 172.21.2.101:50692) 2021-02-10T11:40:49.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.411 7f6777fff700 1 -- 172.21.2.101:0/964086350 learned_addr learned my addr 172.21.2.101:0/964086350 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:49.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.412 7f6776ffd700 1 -- 172.21.2.101:0/964086350 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3570640424 0 0) 0x7f6770001440 con 0x7f67780d1c80 2021-02-10T11:40:49.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.412 7f6776ffd700 1 -- 172.21.2.101:0/964086350 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6764002fe0 con 0x7f67780d1c80 2021-02-10T11:40:49.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.412 7f67777fe700 1 -- 172.21.2.101:0/964086350 >> v2:172.21.2.108:3300/0 conn(0x7f67781b0af0 msgr2=0x7f6778108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:49.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.412 7f67777fe700 1 --2- 172.21.2.101:0/964086350 >> v2:172.21.2.108:3300/0 conn(0x7f67781b0af0 0x7f6778108950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:49.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.412 7f67777fe700 1 -- 172.21.2.101:0/964086350 >> v1:172.21.2.108:6789/0 conn(0x7f67780d1c80 legacy=0x7f6778108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:49.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.412 7f67777fe700 1 -- 172.21.2.101:0/964086350 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6778088070 con 0x7f67780d1880 2021-02-10T11:40:49.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.412 7f67777fe700 1 --2- 172.21.2.101:0/964086350 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67780d1880 0x7f67781b04d0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 rx=0x7f6760006650 tx=0x7f6760009a80).ready entity=mon.0 client_cookie=33fd1bd90c6ad05a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.412 7f6776ffd700 1 -- 172.21.2.101:0/964086350 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f676000c0c0 con 0x7f67780d1880 2021-02-10T11:40:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.412 7f6776ffd700 1 -- 172.21.2.101:0/964086350 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676000c340 con 0x7f67780d1880 2021-02-10T11:40:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.412 7f6776ffd700 1 -- 172.21.2.101:0/964086350 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f676000c860 con 0x7f67780d1880 2021-02-10T11:40:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.412 7f677dc02700 1 -- 172.21.2.101:0/964086350 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67780d1880 msgr2=0x7f67781b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.412 7f677dc02700 1 --2- 172.21.2.101:0/964086350 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67780d1880 0x7f67781b04d0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 rx=0x7f6760006650 tx=0x7f6760009a80).stop 2021-02-10T11:40:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.413 7f677dc02700 1 -- 172.21.2.101:0/964086350 shutdown_connections 2021-02-10T11:40:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.413 7f677dc02700 1 --2- 172.21.2.101:0/964086350 >> v2:172.21.2.108:3300/0 conn(0x7f67781b0af0 0x7f6778108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.413 7f677dc02700 1 --2- 172.21.2.101:0/964086350 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67780d1880 0x7f67781b04d0 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.413 7f677dc02700 1 -- 172.21.2.101:0/964086350 >> 172.21.2.101:0/964086350 conn(0x7f67780ce430 msgr2=0x7f67780d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:49.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.413 7f677dc02700 1 -- 172.21.2.101:0/964086350 shutdown_connections 2021-02-10T11:40:49.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.413 7f677dc02700 1 -- 172.21.2.101:0/964086350 wait complete. 2021-02-10T11:40:49.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.427 7f677dc02700 1 Processor -- start 2021-02-10T11:40:49.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.428 7f677dc02700 1 -- start start 2021-02-10T11:40:49.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.428 7f677dc02700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67781a34e0 0x7f67781a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:49.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.428 7f677dc02700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6778083220 0x7f6778083620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:49.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.428 7f677dc02700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f67780acda0 0x7f67780ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:49.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.428 7f677dc02700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67780ad7c0 con 0x7f67781a34e0 2021-02-10T11:40:49.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.428 7f677dc02700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67780ad9e0 con 0x7f67780acda0 2021-02-10T11:40:49.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.428 7f677dc02700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67780adc00 con 0x7f6778083220 2021-02-10T11:40:49.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.428 7f67777fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67781a34e0 0x7f67781a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:49.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.428 7f67777fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67781a34e0 0x7f67781a38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56794/0 (socket says 172.21.2.101:56794) 2021-02-10T11:40:49.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.428 7f67777fe700 1 -- 172.21.2.101:0/676748709 learned_addr learned my addr 172.21.2.101:0/676748709 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:49.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.428 7f6777fff700 1 --2- 172.21.2.101:0/676748709 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f67780acda0 0x7f67780ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:49.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.428 7f676effd700 1 --2- 172.21.2.101:0/676748709 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6778083220 0x7f6778083620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:49.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.429 7f67777fe700 1 -- 172.21.2.101:0/676748709 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6778083220 msgr2=0x7f6778083620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:49.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.429 7f67777fe700 1 --2- 172.21.2.101:0/676748709 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6778083220 0x7f6778083620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:49.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.429 7f67777fe700 1 -- 172.21.2.101:0/676748709 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f67780acda0 msgr2=0x7f67780ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:49.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.429 7f67777fe700 1 --2- 172.21.2.101:0/676748709 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f67780acda0 0x7f67780ad1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:49.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.429 7f67777fe700 1 -- 172.21.2.101:0/676748709 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67780ade20 con 0x7f67781a34e0 2021-02-10T11:40:49.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.429 7f67777fe700 1 --2- 172.21.2.101:0/676748709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67781a34e0 0x7f67781a38e0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 rx=0x7f6760009630 tx=0x7f676000b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:49.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.429 7f6774ff9700 1 -- 172.21.2.101:0/676748709 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f67600093a0 con 0x7f67781a34e0 2021-02-10T11:40:49.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.429 7f6774ff9700 1 -- 172.21.2.101:0/676748709 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676000bcf0 con 0x7f67781a34e0 2021-02-10T11:40:49.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.429 7f677dc02700 1 -- 172.21.2.101:0/676748709 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67780ade20 con 0x7f67781a34e0 2021-02-10T11:40:49.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.429 7f6774ff9700 1 -- 172.21.2.101:0/676748709 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f67600054d0 con 0x7f67781a34e0 2021-02-10T11:40:49.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.429 7f677dc02700 1 -- 172.21.2.101:0/676748709 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67780ade20 con 0x7f67781a34e0 2021-02-10T11:40:49.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.429 7f6774ff9700 1 -- 172.21.2.101:0/676748709 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f67600054f0 con 0x7f67781a34e0 2021-02-10T11:40:49.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.429 7f6774ff9700 1 --2- 172.21.2.101:0/676748709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f675c018730 0x7f675c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:49.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.429 7f6774ff9700 1 -- 172.21.2.101:0/676748709 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6760032630 con 0x7f67781a34e0 2021-02-10T11:40:49.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.430 7f676effd700 1 --2- 172.21.2.101:0/676748709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f675c018730 0x7f675c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:49.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.430 7f676effd700 1 --2- 172.21.2.101:0/676748709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f675c018730 0x7f675c01ac10 crc :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:49.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.460 7f677dc02700 1 -- 172.21.2.101:0/676748709 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67780fe1d0 con 0x7f67781a34e0 2021-02-10T11:40:49.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.463 7f6774ff9700 1 -- 172.21.2.101:0/676748709 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f67600052e0 con 0x7f67781a34e0 2021-02-10T11:40:49.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.592 7f677dc02700 1 -- 172.21.2.101:0/676748709 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f67780fe1d0 con 0x7f67781a34e0 2021-02-10T11:40:49.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.596 7f6774ff9700 1 -- 172.21.2.101:0/676748709 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v65) v1 ==== 184+0+0 (secure 0 0 0) 0x7f67600054f0 con 0x7f67781a34e0 2021-02-10T11:40:49.602 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.0 2021-02-10T11:40:49.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.601 7f677dc02700 1 -- 172.21.2.101:0/676748709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f675c018730 msgr2=0x7f675c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:49.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.601 7f677dc02700 1 --2- 172.21.2.101:0/676748709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f675c018730 0x7f675c01ac10 crc :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:49.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.601 7f677dc02700 1 -- 172.21.2.101:0/676748709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67781a34e0 msgr2=0x7f67781a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:49.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.601 7f677dc02700 1 --2- 172.21.2.101:0/676748709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67781a34e0 0x7f67781a38e0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 rx=0x7f6760009630 tx=0x7f676000b900).stop 2021-02-10T11:40:49.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.601 7f677dc02700 1 -- 172.21.2.101:0/676748709 shutdown_connections 2021-02-10T11:40:49.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.601 7f677dc02700 1 --2- 172.21.2.101:0/676748709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f675c018730 0x7f675c01ac10 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:49.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.601 7f677dc02700 1 --2- 172.21.2.101:0/676748709 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6778083220 0x7f6778083620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:49.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.601 7f677dc02700 1 --2- 172.21.2.101:0/676748709 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f67780acda0 0x7f67780ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:49.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.601 7f677dc02700 1 --2- 172.21.2.101:0/676748709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f67781a34e0 0x7f67781a38e0 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:49.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.601 7f677dc02700 1 -- 172.21.2.101:0/676748709 >> 172.21.2.101:0/676748709 conn(0x7f67780a1070 msgr2=0x7f677808eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:49.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.601 7f677dc02700 1 -- 172.21.2.101:0/676748709 shutdown_connections 2021-02-10T11:40:49.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:49.601 7f677dc02700 1 -- 172.21.2.101:0/676748709 wait complete. 2021-02-10T11:40:49.612 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:40:50.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.031 7f543234a700 1 Processor -- start 2021-02-10T11:40:50.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.031 7f543234a700 1 -- start start 2021-02-10T11:40:50.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.032 7f543234a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f542c0d1880 0x7f542c1b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:50.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.032 7f543234a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f542c0cff80 0x7f542c109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:50.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.032 7f543234a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542c109a70 con 0x7f542c1b0af0 2021-02-10T11:40:50.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.032 7f543234a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f542c109760 con 0x7f542c0d1880 2021-02-10T11:40:50.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.032 7f543234a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c088070 con 0x7f542c0cff80 2021-02-10T11:40:50.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.032 7f542b7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f542c1b0af0 0x7f542c108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57292/0 (socket says 172.21.2.101:57292) 2021-02-10T11:40:50.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.032 7f542b7fe700 1 -- 172.21.2.101:0/1124235758 learned_addr learned my addr 172.21.2.101:0/1124235758 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:50.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.032 7f5430967700 1 --2- 172.21.2.101:0/1124235758 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f542c0cff80 0x7f542c109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:50.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.032 7f542bfff700 1 --2- 172.21.2.101:0/1124235758 >> v2:172.21.2.108:3300/0 conn(0x7f542c0d1880 0x7f542c1b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:50.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.032 7f542affd700 1 -- 172.21.2.101:0/1124235758 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3578784760 0 0) 0x7f5420000ce0 con 0x7f542c1b0af0 2021-02-10T11:40:50.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.032 7f542affd700 1 -- 172.21.2.101:0/1124235758 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5414002fe0 con 0x7f542c1b0af0 2021-02-10T11:40:50.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.033 7f5430967700 1 -- 172.21.2.101:0/1124235758 >> v2:172.21.2.108:3300/0 conn(0x7f542c0d1880 msgr2=0x7f542c1b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:50.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.033 7f5430967700 1 --2- 172.21.2.101:0/1124235758 >> v2:172.21.2.108:3300/0 conn(0x7f542c0d1880 0x7f542c1b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:50.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.033 7f5430967700 1 -- 172.21.2.101:0/1124235758 >> v1:172.21.2.101:6789/0 conn(0x7f542c1b0af0 legacy=0x7f542c108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:50.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.033 7f5430967700 1 -- 172.21.2.101:0/1124235758 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c0d1c80 con 0x7f542c0cff80 2021-02-10T11:40:50.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.033 7f5430967700 1 --2- 172.21.2.101:0/1124235758 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f542c0cff80 0x7f542c109140 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f5424006e50 tx=0x7f5424009e70).ready entity=mon.2 client_cookie=74d991fa9c992203 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:50.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.033 7f542affd700 1 -- 172.21.2.101:0/1124235758 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f542400b510 con 0x7f542c0cff80 2021-02-10T11:40:50.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.033 7f542affd700 1 -- 172.21.2.101:0/1124235758 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5424006790 con 0x7f542c0cff80 2021-02-10T11:40:50.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.033 7f542affd700 1 -- 172.21.2.101:0/1124235758 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f542400b940 con 0x7f542c0cff80 2021-02-10T11:40:50.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.033 7f543234a700 1 -- 172.21.2.101:0/1124235758 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f542c0cff80 msgr2=0x7f542c109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:50.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.033 7f543234a700 1 --2- 172.21.2.101:0/1124235758 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f542c0cff80 0x7f542c109140 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f5424006e50 tx=0x7f5424009e70).stop 2021-02-10T11:40:50.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.034 7f543234a700 1 -- 172.21.2.101:0/1124235758 shutdown_connections 2021-02-10T11:40:50.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.034 7f543234a700 1 --2- 172.21.2.101:0/1124235758 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f542c0cff80 0x7f542c109140 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:50.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.034 7f543234a700 1 --2- 172.21.2.101:0/1124235758 >> v2:172.21.2.108:3300/0 conn(0x7f542c0d1880 0x7f542c1b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:50.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.034 7f543234a700 1 -- 172.21.2.101:0/1124235758 >> 172.21.2.101:0/1124235758 conn(0x7f542c0ce430 msgr2=0x7f542c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:50.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.034 7f543234a700 1 -- 172.21.2.101:0/1124235758 shutdown_connections 2021-02-10T11:40:50.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.034 7f543234a700 1 -- 172.21.2.101:0/1124235758 wait complete. 2021-02-10T11:40:50.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.049 7f543234a700 1 Processor -- start 2021-02-10T11:40:50.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.049 7f543234a700 1 -- start start 2021-02-10T11:40:50.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.049 7f543234a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f542c18c290 0x7f542c18d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:50.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.049 7f543234a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f542c18c690 0x7f542c18d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:50.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.049 7f543234a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f542c18dcf0 0x7f542c0c2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:50.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.049 7f543234a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c18e0f0 con 0x7f542c18dcf0 2021-02-10T11:40:50.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.049 7f543234a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c18e310 con 0x7f542c18c290 2021-02-10T11:40:50.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.049 7f543234a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c0c30b0 con 0x7f542c18c690 2021-02-10T11:40:50.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.050 7f5430967700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f542c18dcf0 0x7f542c0c2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:50.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.050 7f5430967700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f542c18dcf0 0x7f542c0c2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56804/0 (socket says 172.21.2.101:56804) 2021-02-10T11:40:50.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.050 7f5430967700 1 -- 172.21.2.101:0/3892716931 learned_addr learned my addr 172.21.2.101:0/3892716931 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:50.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.050 7f5430967700 1 -- 172.21.2.101:0/3892716931 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f542c18c690 msgr2=0x7f542c18d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:50.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.050 7f5430967700 1 --2- 172.21.2.101:0/3892716931 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f542c18c690 0x7f542c18d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:50.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.050 7f5430967700 1 -- 172.21.2.101:0/3892716931 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f542c18c290 msgr2=0x7f542c18d0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:50.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.050 7f5430967700 1 --2- 172.21.2.101:0/3892716931 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f542c18c290 0x7f542c18d0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:50.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.050 7f5430967700 1 -- 172.21.2.101:0/3892716931 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c0c32d0 con 0x7f542c18dcf0 2021-02-10T11:40:50.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.050 7f5430967700 1 --2- 172.21.2.101:0/3892716931 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f542c18dcf0 0x7f542c0c2a90 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 rx=0x7f5424006c50 tx=0x7f542400bd20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:50.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.050 7f5428ff9700 1 -- 172.21.2.101:0/3892716931 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f54240093d0 con 0x7f542c18dcf0 2021-02-10T11:40:50.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.050 7f5428ff9700 1 -- 172.21.2.101:0/3892716931 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542400be60 con 0x7f542c18dcf0 2021-02-10T11:40:50.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.050 7f543234a700 1 -- 172.21.2.101:0/3892716931 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542c0c32d0 con 0x7f542c18dcf0 2021-02-10T11:40:50.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.050 7f5428ff9700 1 -- 172.21.2.101:0/3892716931 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f542400c570 con 0x7f542c18dcf0 2021-02-10T11:40:50.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.050 7f543234a700 1 -- 172.21.2.101:0/3892716931 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542c0c3860 con 0x7f542c18dcf0 2021-02-10T11:40:50.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.051 7f5428ff9700 1 -- 172.21.2.101:0/3892716931 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f54240093d0 con 0x7f542c18dcf0 2021-02-10T11:40:50.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.051 7f5428ff9700 1 --2- 172.21.2.101:0/3892716931 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f540c018730 0x7f540c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:50.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.051 7f5428ff9700 1 -- 172.21.2.101:0/3892716931 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f54240329c0 con 0x7f542c18dcf0 2021-02-10T11:40:50.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.051 7f542bfff700 1 --2- 172.21.2.101:0/3892716931 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f540c018730 0x7f540c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:50.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.051 7f542bfff700 1 --2- 172.21.2.101:0/3892716931 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f540c018730 0x7f540c01abc0 crc :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:50.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.081 7f543234a700 1 -- 172.21.2.101:0/3892716931 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542c082980 con 0x7f542c18dcf0 2021-02-10T11:40:50.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.084 7f5428ff9700 1 -- 172.21.2.101:0/3892716931 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5424002ef0 con 0x7f542c18dcf0 2021-02-10T11:40:50.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.212 7f543234a700 1 -- 172.21.2.101:0/3892716931 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f542c082980 con 0x7f542c18dcf0 2021-02-10T11:40:50.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.216 7f5428ff9700 1 -- 172.21.2.101:0/3892716931 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v66) v1 ==== 184+0+0 (secure 0 0 0) 0x7f54240093d0 con 0x7f542c18dcf0 2021-02-10T11:40:50.222 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.1 2021-02-10T11:40:50.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.221 7f543234a700 1 -- 172.21.2.101:0/3892716931 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f540c018730 msgr2=0x7f540c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:50.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.221 7f543234a700 1 --2- 172.21.2.101:0/3892716931 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f540c018730 0x7f540c01abc0 crc :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:50.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.221 7f543234a700 1 -- 172.21.2.101:0/3892716931 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f542c18dcf0 msgr2=0x7f542c0c2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:50.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.221 7f543234a700 1 --2- 172.21.2.101:0/3892716931 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f542c18dcf0 0x7f542c0c2a90 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 rx=0x7f5424006c50 tx=0x7f542400bd20).stop 2021-02-10T11:40:50.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.221 7f543234a700 1 -- 172.21.2.101:0/3892716931 shutdown_connections 2021-02-10T11:40:50.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.221 7f543234a700 1 --2- 172.21.2.101:0/3892716931 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f540c018730 0x7f540c01abc0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:50.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.221 7f543234a700 1 --2- 172.21.2.101:0/3892716931 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f542c18c690 0x7f542c18d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:50.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.221 7f543234a700 1 --2- 172.21.2.101:0/3892716931 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f542c18dcf0 0x7f542c0c2a90 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:50.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.221 7f543234a700 1 --2- 172.21.2.101:0/3892716931 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f542c18c290 0x7f542c18d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:50.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.221 7f543234a700 1 -- 172.21.2.101:0/3892716931 >> 172.21.2.101:0/3892716931 conn(0x7f542c1909a0 msgr2=0x7f542c08a480 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:50.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.222 7f543234a700 1 -- 172.21.2.101:0/3892716931 shutdown_connections 2021-02-10T11:40:50.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.222 7f543234a700 1 -- 172.21.2.101:0/3892716931 wait complete. 2021-02-10T11:40:50.232 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:40:50.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.653 7ff140e70700 1 Processor -- start 2021-02-10T11:40:50.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.653 7ff140e70700 1 -- start start 2021-02-10T11:40:50.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.653 7ff140e70700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff13c0d1c80 0x7ff13c0cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:50.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.653 7ff140e70700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff13c0cf590 0x7ff13c0cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:50.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.653 7ff140e70700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13c0fe1d0 con 0x7ff13c0d1880 2021-02-10T11:40:50.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.653 7ff140e70700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff13c1b0cc0 con 0x7ff13c0d1c80 2021-02-10T11:40:50.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.653 7ff140e70700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c109a70 con 0x7ff13c0cf590 2021-02-10T11:40:50.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.653 7ff13a312700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff13c0d1c80 0x7ff13c0cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.653 7ff13a312700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff13c0d1c80 0x7ff13c0cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56810/0 (socket says 172.21.2.101:56810) 2021-02-10T11:40:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.653 7ff13a312700 1 -- 172.21.2.101:0/2644515820 learned_addr learned my addr 172.21.2.101:0/2644515820 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.654 7ff13b314700 1 --2- 172.21.2.101:0/2644515820 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff13c0cf590 0x7ff13c0cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.654 7ff13a312700 1 -- 172.21.2.101:0/2644515820 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff13c0cf590 msgr2=0x7ff13c0cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.654 7ff13a312700 1 --2- 172.21.2.101:0/2644515820 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff13c0cf590 0x7ff13c0cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.654 7ff13a312700 1 -- 172.21.2.101:0/2644515820 >> v1:172.21.2.108:6789/0 conn(0x7ff13c0d1880 legacy=0x7ff13c1b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.654 7ff13a312700 1 -- 172.21.2.101:0/2644515820 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c088070 con 0x7ff13c0d1c80 2021-02-10T11:40:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.654 7ff13b314700 1 --2- 172.21.2.101:0/2644515820 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff13c0cf590 0x7ff13c0cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:40:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.654 7ff13a312700 1 --2- 172.21.2.101:0/2644515820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff13c0d1c80 0x7ff13c0cef70 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 rx=0x7ff12c006700 tx=0x7ff12c009b80).ready entity=mon.0 client_cookie=6e5e8cfac97aeb03 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.654 7ff139b11700 1 -- 172.21.2.101:0/2644515820 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff12c000c90 con 0x7ff13c0d1c80 2021-02-10T11:40:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.654 7ff139b11700 1 -- 172.21.2.101:0/2644515820 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c00c0c0 con 0x7ff13c0d1c80 2021-02-10T11:40:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.654 7ff139b11700 1 -- 172.21.2.101:0/2644515820 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff12c00c520 con 0x7ff13c0d1c80 2021-02-10T11:40:50.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.654 7ff140e70700 1 -- 172.21.2.101:0/2644515820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff13c0d1c80 msgr2=0x7ff13c0cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:50.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.654 7ff140e70700 1 --2- 172.21.2.101:0/2644515820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff13c0d1c80 0x7ff13c0cef70 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 rx=0x7ff12c006700 tx=0x7ff12c009b80).stop 2021-02-10T11:40:50.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.655 7ff140e70700 1 -- 172.21.2.101:0/2644515820 shutdown_connections 2021-02-10T11:40:50.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.655 7ff140e70700 1 --2- 172.21.2.101:0/2644515820 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff13c0cf590 0x7ff13c0cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:50.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.655 7ff140e70700 1 --2- 172.21.2.101:0/2644515820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff13c0d1c80 0x7ff13c0cef70 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:50.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.655 7ff140e70700 1 -- 172.21.2.101:0/2644515820 >> 172.21.2.101:0/2644515820 conn(0x7ff13c0ce430 msgr2=0x7ff13c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:50.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.655 7ff140e70700 1 -- 172.21.2.101:0/2644515820 shutdown_connections 2021-02-10T11:40:50.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.655 7ff140e70700 1 -- 172.21.2.101:0/2644515820 wait complete. 2021-02-10T11:40:50.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.670 7ff140e70700 1 Processor -- start 2021-02-10T11:40:50.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.670 7ff140e70700 1 -- start start 2021-02-10T11:40:50.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.670 7ff140e70700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff13c1b9280 0x7ff13c18f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:50.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.670 7ff140e70700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff13c1b9680 0x7ff13c18fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:50.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.670 7ff140e70700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff13c1acd60 0x7ff13c1ad160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:50.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.670 7ff140e70700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c190390 con 0x7ff13c1b9680 2021-02-10T11:40:50.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.670 7ff140e70700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c1907c0 con 0x7ff13c1acd60 2021-02-10T11:40:50.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.670 7ff140e70700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c1ac3d0 con 0x7ff13c1b9280 2021-02-10T11:40:50.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.670 7ff13a312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff13c1b9680 0x7ff13c18fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:50.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.670 7ff13a312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff13c1b9680 0x7ff13c18fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56814/0 (socket says 172.21.2.101:56814) 2021-02-10T11:40:50.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.670 7ff13a312700 1 -- 172.21.2.101:0/248841070 learned_addr learned my addr 172.21.2.101:0/248841070 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.671 7ff13a312700 1 -- 172.21.2.101:0/248841070 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff13c1b9280 msgr2=0x7ff13c18f750 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.671 7ff13a312700 1 --2- 172.21.2.101:0/248841070 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff13c1b9280 0x7ff13c18f750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.671 7ff13a312700 1 -- 172.21.2.101:0/248841070 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff13c1acd60 msgr2=0x7ff13c1ad160 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:40:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.671 7ff13a312700 1 --2- 172.21.2.101:0/248841070 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff13c1acd60 0x7ff13c1ad160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.671 7ff13a312700 1 -- 172.21.2.101:0/248841070 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c1ad780 con 0x7ff13c1b9680 2021-02-10T11:40:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.671 7ff13a312700 1 --2- 172.21.2.101:0/248841070 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff13c1b9680 0x7ff13c18fd70 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 rx=0x7ff12c009fb0 tx=0x7ff12c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.671 7ff1337fe700 1 -- 172.21.2.101:0/248841070 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff12c006340 con 0x7ff13c1b9680 2021-02-10T11:40:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.671 7ff1337fe700 1 -- 172.21.2.101:0/248841070 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c002080 con 0x7ff13c1b9680 2021-02-10T11:40:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.671 7ff140e70700 1 -- 172.21.2.101:0/248841070 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13c1ad780 con 0x7ff13c1b9680 2021-02-10T11:40:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.671 7ff140e70700 1 -- 172.21.2.101:0/248841070 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13c1add10 con 0x7ff13c1b9680 2021-02-10T11:40:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.671 7ff1337fe700 1 -- 172.21.2.101:0/248841070 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff12c0024d0 con 0x7ff13c1b9680 2021-02-10T11:40:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.671 7ff1337fe700 1 -- 172.21.2.101:0/248841070 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff12c000c90 con 0x7ff13c1b9680 2021-02-10T11:40:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.671 7ff1337fe700 1 --2- 172.21.2.101:0/248841070 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff120018730 0x7ff12001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:50.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.672 7ff1337fe700 1 -- 172.21.2.101:0/248841070 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff12c032630 con 0x7ff13c1b9680 2021-02-10T11:40:50.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.672 7ff13ab13700 1 --2- 172.21.2.101:0/248841070 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff120018730 0x7ff12001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:50.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.672 7ff13ab13700 1 --2- 172.21.2.101:0/248841070 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff120018730 0x7ff12001ac10 crc :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:50.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.703 7ff140e70700 1 -- 172.21.2.101:0/248841070 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13c1907a0 con 0x7ff13c1b9680 2021-02-10T11:40:50.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.706 7ff1337fe700 1 -- 172.21.2.101:0/248841070 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff12c01f4f0 con 0x7ff13c1b9680 2021-02-10T11:40:50.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.835 7ff140e70700 1 -- 172.21.2.101:0/248841070 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ff13c1907a0 con 0x7ff13c1b9680 2021-02-10T11:40:50.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.839 7ff1337fe700 1 -- 172.21.2.101:0/248841070 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v67) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff12c01f4f0 con 0x7ff13c1b9680 2021-02-10T11:40:50.845 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.2 2021-02-10T11:40:50.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.844 7ff140e70700 1 -- 172.21.2.101:0/248841070 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff120018730 msgr2=0x7ff12001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:50.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.844 7ff140e70700 1 --2- 172.21.2.101:0/248841070 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff120018730 0x7ff12001ac10 crc :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:50.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.844 7ff140e70700 1 -- 172.21.2.101:0/248841070 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff13c1b9680 msgr2=0x7ff13c18fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:50.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.844 7ff140e70700 1 --2- 172.21.2.101:0/248841070 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff13c1b9680 0x7ff13c18fd70 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 rx=0x7ff12c009fb0 tx=0x7ff12c00b990).stop 2021-02-10T11:40:50.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.844 7ff140e70700 1 -- 172.21.2.101:0/248841070 shutdown_connections 2021-02-10T11:40:50.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.844 7ff140e70700 1 --2- 172.21.2.101:0/248841070 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff13c1acd60 0x7ff13c1ad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:50.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.844 7ff140e70700 1 --2- 172.21.2.101:0/248841070 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff13c1b9680 0x7ff13c18fd70 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:50.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.844 7ff140e70700 1 --2- 172.21.2.101:0/248841070 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff120018730 0x7ff12001ac10 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:50.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.844 7ff140e70700 1 --2- 172.21.2.101:0/248841070 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff13c1b9280 0x7ff13c18f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:50.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.844 7ff140e70700 1 -- 172.21.2.101:0/248841070 >> 172.21.2.101:0/248841070 conn(0x7ff13c0845b0 msgr2=0x7ff13c18c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:50.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.844 7ff140e70700 1 -- 172.21.2.101:0/248841070 shutdown_connections 2021-02-10T11:40:50.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:50.844 7ff140e70700 1 -- 172.21.2.101:0/248841070 wait complete. 2021-02-10T11:40:50.856 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:40:51.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.269 7fbdedf9b700 1 Processor -- start 2021-02-10T11:40:51.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.269 7fbdedf9b700 1 -- start start 2021-02-10T11:40:51.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.270 7fbdedf9b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbde810afe0 0x7fbde80cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:51.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.270 7fbdedf9b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde80fe1b0 con 0x7fbde80d1c60 2021-02-10T11:40:51.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.270 7fbdedf9b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde81b0ca0 con 0x7fbde80d1860 2021-02-10T11:40:51.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.270 7fbdedf9b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde8109a50 con 0x7fbde810afe0 2021-02-10T11:40:51.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.270 7fbde6ffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fbde80d1c60 0x7fbde80cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57316/0 (socket says 172.21.2.101:57316) 2021-02-10T11:40:51.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.270 7fbde6ffd700 1 -- 172.21.2.101:0/1840468418 learned_addr learned my addr 172.21.2.101:0/1840468418 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:51.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.270 7fbde7fff700 1 --2- 172.21.2.101:0/1840468418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbde810afe0 0x7fbde80cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:51.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.270 7fbde67fc700 1 -- 172.21.2.101:0/1840468418 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 713418646 0 0) 0x7fbdd0001060 con 0x7fbde80d1c60 2021-02-10T11:40:51.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.270 7fbde67fc700 1 -- 172.21.2.101:0/1840468418 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdd4002fe0 con 0x7fbde80d1c60 2021-02-10T11:40:51.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.271 7fbde67fc700 1 -- 172.21.2.101:0/1840468418 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2750902415 0 0) 0x7fbdd8001080 con 0x7fbde80d1860 2021-02-10T11:40:51.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.271 7fbde67fc700 1 -- 172.21.2.101:0/1840468418 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdd4003210 con 0x7fbde80d1860 2021-02-10T11:40:51.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.271 7fbde7fff700 1 -- 172.21.2.101:0/1840468418 >> v1:172.21.2.108:6789/0 conn(0x7fbde80d1860 legacy=0x7fbde81b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:51.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.271 7fbde7fff700 1 -- 172.21.2.101:0/1840468418 >> v1:172.21.2.101:6789/0 conn(0x7fbde80d1c60 legacy=0x7fbde80cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:51.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.271 7fbde7fff700 1 -- 172.21.2.101:0/1840468418 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde810b3e0 con 0x7fbde810afe0 2021-02-10T11:40:51.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.271 7fbde7fff700 1 --2- 172.21.2.101:0/1840468418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbde810afe0 0x7fbde80cf740 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fbde0006ad0 tx=0x7fbde0009ad0).ready entity=mon.2 client_cookie=ff3544348750ae3d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:51.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.271 7fbde67fc700 1 -- 172.21.2.101:0/1840468418 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbde00022e0 con 0x7fbde810afe0 2021-02-10T11:40:51.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.271 7fbde67fc700 1 -- 172.21.2.101:0/1840468418 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde0006410 con 0x7fbde810afe0 2021-02-10T11:40:51.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.271 7fbde67fc700 1 -- 172.21.2.101:0/1840468418 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbde0002620 con 0x7fbde810afe0 2021-02-10T11:40:51.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.271 7fbdedf9b700 1 -- 172.21.2.101:0/1840468418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbde810afe0 msgr2=0x7fbde80cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:51.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.271 7fbdedf9b700 1 --2- 172.21.2.101:0/1840468418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbde810afe0 0x7fbde80cf740 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fbde0006ad0 tx=0x7fbde0009ad0).stop 2021-02-10T11:40:51.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.272 7fbdedf9b700 1 -- 172.21.2.101:0/1840468418 shutdown_connections 2021-02-10T11:40:51.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.272 7fbdedf9b700 1 --2- 172.21.2.101:0/1840468418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbde810afe0 0x7fbde80cf740 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:51.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.272 7fbdedf9b700 1 -- 172.21.2.101:0/1840468418 >> 172.21.2.101:0/1840468418 conn(0x7fbde80ce410 msgr2=0x7fbde80d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:51.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.272 7fbdedf9b700 1 -- 172.21.2.101:0/1840468418 shutdown_connections 2021-02-10T11:40:51.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.272 7fbdedf9b700 1 -- 172.21.2.101:0/1840468418 wait complete. 2021-02-10T11:40:51.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.287 7fbdedf9b700 1 Processor -- start 2021-02-10T11:40:51.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.287 7fbdedf9b700 1 -- start start 2021-02-10T11:40:51.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.287 7fbdedf9b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbde819b6f0 0x7fbde8094080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:51.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.287 7fbdedf9b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbde819baf0 0x7fbde80aa450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:51.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.287 7fbdedf9b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbde819c690 0x7fbde819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:51.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.287 7fbdedf9b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde8091bb0 con 0x7fbde819baf0 2021-02-10T11:40:51.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.287 7fbdedf9b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde81b9520 con 0x7fbde819b6f0 2021-02-10T11:40:51.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.287 7fbdedf9b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde81b9740 con 0x7fbde819c690 2021-02-10T11:40:51.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.287 7fbde6ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbde819baf0 0x7fbde80aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:51.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.287 7fbde6ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbde819baf0 0x7fbde80aa450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56830/0 (socket says 172.21.2.101:56830) 2021-02-10T11:40:51.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.287 7fbde6ffd700 1 -- 172.21.2.101:0/3138466709 learned_addr learned my addr 172.21.2.101:0/3138466709 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:51.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.288 7fbde6ffd700 1 -- 172.21.2.101:0/3138466709 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbde819c690 msgr2=0x7fbde819ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:51.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.288 7fbde6ffd700 1 --2- 172.21.2.101:0/3138466709 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbde819c690 0x7fbde819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:51.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.288 7fbde6ffd700 1 -- 172.21.2.101:0/3138466709 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbde819b6f0 msgr2=0x7fbde8094080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:51.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.288 7fbde6ffd700 1 --2- 172.21.2.101:0/3138466709 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbde819b6f0 0x7fbde8094080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:51.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.288 7fbde6ffd700 1 -- 172.21.2.101:0/3138466709 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde80aaa70 con 0x7fbde819baf0 2021-02-10T11:40:51.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.288 7fbde6ffd700 1 --2- 172.21.2.101:0/3138466709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbde819baf0 0x7fbde80aa450 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 rx=0x7fbdd0009960 tx=0x7fbdd000bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:51.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.288 7fbddffff700 1 -- 172.21.2.101:0/3138466709 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbdd00062b0 con 0x7fbde819baf0 2021-02-10T11:40:51.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.288 7fbddffff700 1 -- 172.21.2.101:0/3138466709 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd0004720 con 0x7fbde819baf0 2021-02-10T11:40:51.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.288 7fbdedf9b700 1 -- 172.21.2.101:0/3138466709 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde80aaa70 con 0x7fbde819baf0 2021-02-10T11:40:51.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.288 7fbddffff700 1 -- 172.21.2.101:0/3138466709 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbdd0004b80 con 0x7fbde819baf0 2021-02-10T11:40:51.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.288 7fbdedf9b700 1 -- 172.21.2.101:0/3138466709 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde80aaa70 con 0x7fbde819baf0 2021-02-10T11:40:51.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.289 7fbddffff700 1 -- 172.21.2.101:0/3138466709 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbdd00062b0 con 0x7fbde819baf0 2021-02-10T11:40:51.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.289 7fbddffff700 1 --2- 172.21.2.101:0/3138466709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbdcc0186c0 0x7fbdcc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:51.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.289 7fbddffff700 1 -- 172.21.2.101:0/3138466709 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fbdd00328f0 con 0x7fbde819baf0 2021-02-10T11:40:51.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.289 7fbde77fe700 1 --2- 172.21.2.101:0/3138466709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbdcc0186c0 0x7fbdcc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:51.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.290 7fbde77fe700 1 --2- 172.21.2.101:0/3138466709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbdcc0186c0 0x7fbdcc01abc0 crc :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:51.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.320 7fbdedf9b700 1 -- 172.21.2.101:0/3138466709 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde8082960 con 0x7fbde819baf0 2021-02-10T11:40:51.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.322 7fbddffff700 1 -- 172.21.2.101:0/3138466709 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbdd000be30 con 0x7fbde819baf0 2021-02-10T11:40:51.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.453 7fbdedf9b700 1 -- 172.21.2.101:0/3138466709 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fbde8082960 con 0x7fbde819baf0 2021-02-10T11:40:51.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.456 7fbddffff700 1 -- 172.21.2.101:0/3138466709 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v68) v1 ==== 184+0+0 (secure 0 0 0) 0x7fbdd0009f40 con 0x7fbde819baf0 2021-02-10T11:40:51.462 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.3 2021-02-10T11:40:51.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.461 7fbdedf9b700 1 -- 172.21.2.101:0/3138466709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbdcc0186c0 msgr2=0x7fbdcc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:51.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.461 7fbdedf9b700 1 --2- 172.21.2.101:0/3138466709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbdcc0186c0 0x7fbdcc01abc0 crc :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:51.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.461 7fbdedf9b700 1 -- 172.21.2.101:0/3138466709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbde819baf0 msgr2=0x7fbde80aa450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:51.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.461 7fbdedf9b700 1 --2- 172.21.2.101:0/3138466709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbde819baf0 0x7fbde80aa450 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 rx=0x7fbdd0009960 tx=0x7fbdd000bcf0).stop 2021-02-10T11:40:51.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.461 7fbdedf9b700 1 -- 172.21.2.101:0/3138466709 shutdown_connections 2021-02-10T11:40:51.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.461 7fbdedf9b700 1 --2- 172.21.2.101:0/3138466709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbdcc0186c0 0x7fbdcc01abc0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:51.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.461 7fbdedf9b700 1 --2- 172.21.2.101:0/3138466709 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbde819c690 0x7fbde819ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:51.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.461 7fbdedf9b700 1 --2- 172.21.2.101:0/3138466709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbde819baf0 0x7fbde80aa450 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:51.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.461 7fbdedf9b700 1 --2- 172.21.2.101:0/3138466709 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbde819b6f0 0x7fbde8094080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:51.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.461 7fbdedf9b700 1 -- 172.21.2.101:0/3138466709 >> 172.21.2.101:0/3138466709 conn(0x7fbde819f820 msgr2=0x7fbde818c270 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:51.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.462 7fbdedf9b700 1 -- 172.21.2.101:0/3138466709 shutdown_connections 2021-02-10T11:40:51.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.462 7fbdedf9b700 1 -- 172.21.2.101:0/3138466709 wait complete. 2021-02-10T11:40:51.472 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:40:51.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.882 7fe3d21ad700 1 Processor -- start 2021-02-10T11:40:51.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.882 7fe3d21ad700 1 -- start start 2021-02-10T11:40:51.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.883 7fe3d21ad700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3cc0d1870 0x7fe3cc1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:51.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.883 7fe3d21ad700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe3cc1b0a10 0x7fe3cc0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:51.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.883 7fe3d21ad700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3cc0f0b10 con 0x7fe3cc0d1c70 2021-02-10T11:40:51.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.883 7fe3d21ad700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3cc0fe230 con 0x7fe3cc1b0a10 2021-02-10T11:40:51.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.883 7fe3d21ad700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc0e5ed0 con 0x7fe3cc0d1870 2021-02-10T11:40:51.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.883 7fe3cbfff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe3cc0d1c70 0x7fe3cc0cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57330/0 (socket says 172.21.2.101:57330) 2021-02-10T11:40:51.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.883 7fe3cbfff700 1 -- 172.21.2.101:0/736389135 learned_addr learned my addr 172.21.2.101:0/736389135 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:51.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.883 7fe3ca7fc700 1 -- 172.21.2.101:0/736389135 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225092816 0 0) 0x7fe3c4001500 con 0x7fe3cc0d1c70 2021-02-10T11:40:51.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.883 7fe3ca7fc700 1 -- 172.21.2.101:0/736389135 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3b8002fe0 con 0x7fe3cc0d1c70 2021-02-10T11:40:51.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.883 7fe3cb7fe700 1 --2- 172.21.2.101:0/736389135 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3cc0d1870 0x7fe3cc1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:51.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.883 7fe3caffd700 1 --2- 172.21.2.101:0/736389135 >> v2:172.21.2.108:3300/0 conn(0x7fe3cc1b0a10 0x7fe3cc0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:51.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3ca7fc700 1 -- 172.21.2.101:0/736389135 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3417913111 0 0) 0x7fe3c4001500 con 0x7fe3cc0d1c70 2021-02-10T11:40:51.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3ca7fc700 1 -- 172.21.2.101:0/736389135 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3cc0d1870 msgr2=0x7fe3cc1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:51.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3ca7fc700 1 --2- 172.21.2.101:0/736389135 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3cc0d1870 0x7fe3cc1b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:51.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3ca7fc700 1 -- 172.21.2.101:0/736389135 >> v2:172.21.2.108:3300/0 conn(0x7fe3cc1b0a10 msgr2=0x7fe3cc0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:51.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3ca7fc700 1 --2- 172.21.2.101:0/736389135 >> v2:172.21.2.108:3300/0 conn(0x7fe3cc1b0a10 0x7fe3cc0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:51.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3ca7fc700 1 -- 172.21.2.101:0/736389135 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc0f0650 con 0x7fe3cc0d1c70 2021-02-10T11:40:51.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3cb7fe700 1 --2- 172.21.2.101:0/736389135 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3cc0d1870 0x7fe3cc1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:40:51.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3ca7fc700 1 -- 172.21.2.101:0/736389135 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fe3c4001a00 con 0x7fe3cc0d1c70 2021-02-10T11:40:51.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3ca7fc700 1 -- 172.21.2.101:0/736389135 >> v1:172.21.2.101:6789/0 conn(0x7fe3cc0d1c70 legacy=0x7fe3cc0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:51.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3ca7fc700 1 --2- 172.21.2.101:0/736389135 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe3b8003af0 0x7fe3b8006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:51.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3ca7fc700 1 --2- 172.21.2.101:0/736389135 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3b8006830 0x7fe3b8008d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:51.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3ca7fc700 1 --2- 172.21.2.101:0/736389135 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3b8009630 0x7fe3b800bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:51.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3ca7fc700 1 -- 172.21.2.101:0/736389135 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b8050380 con 0x7fe3b8006830 2021-02-10T11:40:51.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3ca7fc700 1 -- 172.21.2.101:0/736389135 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80505a0 con 0x7fe3b8003af0 2021-02-10T11:40:51.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3ca7fc700 1 -- 172.21.2.101:0/736389135 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80507c0 con 0x7fe3b8009630 2021-02-10T11:40:51.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3caffd700 1 --2- 172.21.2.101:0/736389135 >> v2:172.21.2.108:3300/0 conn(0x7fe3cc1b0a10 0x7fe3cc0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:40:51.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3cb7fe700 1 --2- 172.21.2.101:0/736389135 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe3b8003af0 0x7fe3b8006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:51.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3caffd700 1 --2- 172.21.2.101:0/736389135 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3b8006830 0x7fe3b8008d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:51.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3cbfff700 1 --2- 172.21.2.101:0/736389135 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3b8009630 0x7fe3b800bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:51.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3cb7fe700 1 -- 172.21.2.101:0/736389135 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3b8009630 msgr2=0x7fe3b800bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:51.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3cb7fe700 1 --2- 172.21.2.101:0/736389135 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3b8009630 0x7fe3b800bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:51.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3cb7fe700 1 -- 172.21.2.101:0/736389135 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3b8006830 msgr2=0x7fe3b8008d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:51.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3cb7fe700 1 --2- 172.21.2.101:0/736389135 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3b8006830 0x7fe3b8008d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:51.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3cb7fe700 1 -- 172.21.2.101:0/736389135 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b8050a60 con 0x7fe3b8003af0 2021-02-10T11:40:51.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3caffd700 1 --2- 172.21.2.101:0/736389135 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3b8006830 0x7fe3b8008d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:40:51.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.884 7fe3cbfff700 1 --2- 172.21.2.101:0/736389135 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3b8009630 0x7fe3b800bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:40:51.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.885 7fe3cb7fe700 1 --2- 172.21.2.101:0/736389135 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe3b8003af0 0x7fe3b8006010 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7fe3b4006a90 tx=0x7fe3b4009a90).ready entity=mon.1 client_cookie=9833f63d96090df1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:51.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.885 7fe3ca7fc700 1 -- 172.21.2.101:0/736389135 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe3b400c0c0 con 0x7fe3b8003af0 2021-02-10T11:40:51.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.885 7fe3ca7fc700 1 -- 172.21.2.101:0/736389135 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b400c340 con 0x7fe3b8003af0 2021-02-10T11:40:51.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.885 7fe3ca7fc700 1 -- 172.21.2.101:0/736389135 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe3b40009b0 con 0x7fe3b8003af0 2021-02-10T11:40:51.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.885 7fe3d21ad700 1 -- 172.21.2.101:0/736389135 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe3b8003af0 msgr2=0x7fe3b8006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:51.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.885 7fe3d21ad700 1 --2- 172.21.2.101:0/736389135 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe3b8003af0 0x7fe3b8006010 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7fe3b4006a90 tx=0x7fe3b4009a90).stop 2021-02-10T11:40:51.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.885 7fe3cbfff700 1 -- 172.21.2.101:0/736389135 reap_dead start 2021-02-10T11:40:51.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.886 7fe3d21ad700 1 -- 172.21.2.101:0/736389135 shutdown_connections 2021-02-10T11:40:51.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.886 7fe3d21ad700 1 -- 172.21.2.101:0/736389135 >> 172.21.2.101:0/736389135 conn(0x7fe3cc0ce450 msgr2=0x7fe3cc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:51.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.886 7fe3d21ad700 1 -- 172.21.2.101:0/736389135 shutdown_connections 2021-02-10T11:40:51.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.886 7fe3d21ad700 1 -- 172.21.2.101:0/736389135 wait complete. 2021-02-10T11:40:51.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.900 7fe3d21ad700 1 Processor -- start 2021-02-10T11:40:51.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.900 7fe3d21ad700 1 -- start start 2021-02-10T11:40:51.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.901 7fe3d21ad700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe3cc0b25d0 0x7fe3cc0a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:51.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.901 7fe3d21ad700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3cc0a9340 0x7fe3cc0a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:51.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.901 7fe3d21ad700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3cc0a69d0 0x7fe3cc0a6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:51.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.901 7fe3d21ad700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc0b29d0 con 0x7fe3cc0a9340 2021-02-10T11:40:51.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.901 7fe3d21ad700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc0a73f0 con 0x7fe3cc0b25d0 2021-02-10T11:40:51.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.901 7fe3d21ad700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc0a7870 con 0x7fe3cc0a69d0 2021-02-10T11:40:51.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.901 7fe3caffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3cc0a9340 0x7fe3cc0a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:51.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.901 7fe3caffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3cc0a9340 0x7fe3cc0a63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56846/0 (socket says 172.21.2.101:56846) 2021-02-10T11:40:51.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.901 7fe3caffd700 1 -- 172.21.2.101:0/4118193973 learned_addr learned my addr 172.21.2.101:0/4118193973 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:51.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.901 7fe3caffd700 1 -- 172.21.2.101:0/4118193973 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3cc0a69d0 msgr2=0x7fe3cc0a6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:51.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.901 7fe3caffd700 1 --2- 172.21.2.101:0/4118193973 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3cc0a69d0 0x7fe3cc0a6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:51.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.901 7fe3caffd700 1 -- 172.21.2.101:0/4118193973 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe3cc0b25d0 msgr2=0x7fe3cc0a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:51.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.901 7fe3caffd700 1 --2- 172.21.2.101:0/4118193973 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe3cc0b25d0 0x7fe3cc0a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:51.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.901 7fe3caffd700 1 -- 172.21.2.101:0/4118193973 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc0a7a90 con 0x7fe3cc0a9340 2021-02-10T11:40:51.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.902 7fe3caffd700 1 --2- 172.21.2.101:0/4118193973 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3cc0a9340 0x7fe3cc0a63b0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 rx=0x7fe3bc00b650 tx=0x7fe3bc00b9f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:51.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.902 7fe3c3fff700 1 -- 172.21.2.101:0/4118193973 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe3bc0067d0 con 0x7fe3cc0a9340 2021-02-10T11:40:51.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.902 7fe3c3fff700 1 -- 172.21.2.101:0/4118193973 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3bc002030 con 0x7fe3cc0a9340 2021-02-10T11:40:51.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.902 7fe3c3fff700 1 -- 172.21.2.101:0/4118193973 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe3bc002550 con 0x7fe3cc0a9340 2021-02-10T11:40:51.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.902 7fe3d21ad700 1 -- 172.21.2.101:0/4118193973 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3cc0a7a90 con 0x7fe3cc0a9340 2021-02-10T11:40:51.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.902 7fe3d21ad700 1 -- 172.21.2.101:0/4118193973 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3cc0a8020 con 0x7fe3cc0a9340 2021-02-10T11:40:51.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.902 7fe3c3fff700 1 -- 172.21.2.101:0/4118193973 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe3bc000c90 con 0x7fe3cc0a9340 2021-02-10T11:40:51.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.903 7fe3c3fff700 1 --2- 172.21.2.101:0/4118193973 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3b00186c0 0x7fe3b001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:51.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.903 7fe3c3fff700 1 -- 172.21.2.101:0/4118193973 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe3bc0325d0 con 0x7fe3cc0a9340 2021-02-10T11:40:51.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.903 7fe3cb7fe700 1 --2- 172.21.2.101:0/4118193973 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3b00186c0 0x7fe3b001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:51.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.903 7fe3cb7fe700 1 --2- 172.21.2.101:0/4118193973 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3b00186c0 0x7fe3b001abc0 crc :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:51.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.932 7fe3d21ad700 1 -- 172.21.2.101:0/4118193973 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3cc0ebc80 con 0x7fe3cc0a9340 2021-02-10T11:40:51.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:51.935 7fe3c3fff700 1 -- 172.21.2.101:0/4118193973 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe3bc030d80 con 0x7fe3cc0a9340 2021-02-10T11:40:52.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:52.063 7fe3d21ad700 1 -- 172.21.2.101:0/4118193973 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3cc0f0b10 con 0x7fe3cc0a9340 2021-02-10T11:40:52.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:52.064 7fe3c3fff700 1 -- 172.21.2.101:0/4118193973 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+2104 (secure 0 0 0) 0x7fe3bc00bde0 con 0x7fe3cc0a9340 2021-02-10T11:40:52.067 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:40:52.067 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":88,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2840404451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2840404451},{"type":"v1","addr":"172.21.2.101:6835","nonce":2840404451}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":84}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:40:45.728009","modified":"2021-02-10 11:40:47.738972","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7450},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7450":{"gid":7450,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2757440614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2757440614},{"type":"v1","addr":"172.21.2.108:6833","nonce":2757440614}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2021-02-10T11:40:52.067 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 88 2021-02-10T11:40:52.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:52.066 7fe3d21ad700 1 -- 172.21.2.101:0/4118193973 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3b00186c0 msgr2=0x7fe3b001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:52.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:52.066 7fe3d21ad700 1 --2- 172.21.2.101:0/4118193973 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3b00186c0 0x7fe3b001abc0 crc :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:52.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:52.066 7fe3d21ad700 1 -- 172.21.2.101:0/4118193973 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3cc0a9340 msgr2=0x7fe3cc0a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:52.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:52.066 7fe3d21ad700 1 --2- 172.21.2.101:0/4118193973 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3cc0a9340 0x7fe3cc0a63b0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 rx=0x7fe3bc00b650 tx=0x7fe3bc00b9f0).stop 2021-02-10T11:40:52.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:52.066 7fe3d21ad700 1 -- 172.21.2.101:0/4118193973 shutdown_connections 2021-02-10T11:40:52.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:52.066 7fe3d21ad700 1 --2- 172.21.2.101:0/4118193973 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3b00186c0 0x7fe3b001abc0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:52.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:52.066 7fe3d21ad700 1 --2- 172.21.2.101:0/4118193973 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3cc0a69d0 0x7fe3cc0a6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:52.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:52.066 7fe3d21ad700 1 --2- 172.21.2.101:0/4118193973 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3cc0a9340 0x7fe3cc0a63b0 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:52.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:52.066 7fe3d21ad700 1 --2- 172.21.2.101:0/4118193973 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe3cc0b25d0 0x7fe3cc0a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:52.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:52.066 7fe3d21ad700 1 -- 172.21.2.101:0/4118193973 >> 172.21.2.101:0/4118193973 conn(0x7fe3cc0bbb50 msgr2=0x7fe3cc0ad8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:52.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:52.066 7fe3d21ad700 1 -- 172.21.2.101:0/4118193973 shutdown_connections 2021-02-10T11:40:52.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:52.066 7fe3d21ad700 1 -- 172.21.2.101:0/4118193973 wait complete. 2021-02-10T11:40:52.278 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:40:45.728009', 'modified': '2021-02-10 11:40:47.738972', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7450}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7450': {'gid': 7450, 'name': 'b', 'rank': 0, 'incarnation': 85, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/2757440614', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 2757440614}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 2757440614}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:40:52.279 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:40:52.279 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:40:52.678 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:40:52.678 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:40:52.679 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:40:52.679 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 7450, 2021-02-10T11:40:52.679 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:40:52.679 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:40:52.679 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 6.9451053460000001, 2021-02-10T11:40:52.679 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 88, 2021-02-10T11:40:52.679 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 112, 2021-02-10T11:40:52.679 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 112, 2021-02-10T11:40:52.679 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 12.584576094999999 2021-02-10T11:40:52.679 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:40:52.682 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 7450, "want_state": "up:active", "state": "up:active", "rank_uptime": 6.9451053460000001, "mdsmap_epoch": 88, "osdmap_epoch": 112, "osdmap_epoch_barrier": 112, "uptime": 12.584576094999999 } 2021-02-10T11:40:52.683 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:40:53.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.091 7fea5024e700 1 Processor -- start 2021-02-10T11:40:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.091 7fea5024e700 1 -- start start 2021-02-10T11:40:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.092 7fea5024e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fea480d1c70 0x7fea480f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.092 7fea5024e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea480f0b10 con 0x7fea480d1870 2021-02-10T11:40:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.092 7fea5024e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea481b0be0 con 0x7fea480d0340 2021-02-10T11:40:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.092 7fea5024e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea480e5ed0 con 0x7fea480d1c70 2021-02-10T11:40:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.092 7fea4e06a700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fea480d1870 0x7fea481b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57348/0 (socket says 172.21.2.101:57348) 2021-02-10T11:40:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.092 7fea4e06a700 1 -- 172.21.2.101:0/1883448710 learned_addr learned my addr 172.21.2.101:0/1883448710 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.092 7fea4e86b700 1 --2- 172.21.2.101:0/1883448710 >> v2:172.21.2.108:3300/0 conn(0x7fea480d1c70 0x7fea480f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.092 7fea4d068700 1 -- 172.21.2.101:0/1883448710 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610004435 0 0) 0x7fea440010b0 con 0x7fea480d1870 2021-02-10T11:40:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.092 7fea4d068700 1 -- 172.21.2.101:0/1883448710 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea34002fe0 con 0x7fea480d1870 2021-02-10T11:40:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4d068700 1 -- 172.21.2.101:0/1883448710 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2710016512 0 0) 0x7fea44001010 con 0x7fea480d1870 2021-02-10T11:40:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4d068700 1 -- 172.21.2.101:0/1883448710 >> v2:172.21.2.108:3300/0 conn(0x7fea480d1c70 msgr2=0x7fea480f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:53.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4d068700 1 --2- 172.21.2.101:0/1883448710 >> v2:172.21.2.108:3300/0 conn(0x7fea480d1c70 0x7fea480f20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4d068700 1 -- 172.21.2.101:0/1883448710 >> v1:172.21.2.108:6789/0 conn(0x7fea480d0340 legacy=0x7fea480ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4d068700 1 -- 172.21.2.101:0/1883448710 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea480efe30 con 0x7fea480d1870 2021-02-10T11:40:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4d068700 1 -- 172.21.2.101:0/1883448710 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fea44001530 con 0x7fea480d1870 2021-02-10T11:40:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4d068700 1 -- 172.21.2.101:0/1883448710 >> v1:172.21.2.101:6789/0 conn(0x7fea480d1870 legacy=0x7fea481b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4d068700 1 --2- 172.21.2.101:0/1883448710 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea34003a60 0x7fea34005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4d068700 1 --2- 172.21.2.101:0/1883448710 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea34006790 0x7fea34008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4d068700 1 --2- 172.21.2.101:0/1883448710 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea34009590 0x7fea3400bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4d068700 1 -- 172.21.2.101:0/1883448710 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3400c300 con 0x7fea34009590 2021-02-10T11:40:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4d068700 1 -- 172.21.2.101:0/1883448710 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3400c520 con 0x7fea34003a60 2021-02-10T11:40:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4d068700 1 -- 172.21.2.101:0/1883448710 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea34050750 con 0x7fea34006790 2021-02-10T11:40:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4e86b700 1 --2- 172.21.2.101:0/1883448710 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea34009590 0x7fea3400bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4d869700 1 --2- 172.21.2.101:0/1883448710 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea34006790 0x7fea34008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4e06a700 1 --2- 172.21.2.101:0/1883448710 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea34003a60 0x7fea34005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:53.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4e86b700 1 -- 172.21.2.101:0/1883448710 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea34006790 msgr2=0x7fea34008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:53.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4e86b700 1 --2- 172.21.2.101:0/1883448710 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea34006790 0x7fea34008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:53.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4e86b700 1 -- 172.21.2.101:0/1883448710 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea34003a60 msgr2=0x7fea34005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:53.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4e86b700 1 --2- 172.21.2.101:0/1883448710 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea34003a60 0x7fea34005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:53.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4e86b700 1 -- 172.21.2.101:0/1883448710 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea340509b0 con 0x7fea34009590 2021-02-10T11:40:53.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4e06a700 1 --2- 172.21.2.101:0/1883448710 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea34003a60 0x7fea34005ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:40:53.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4e86b700 1 -- 172.21.2.101:0/1883448710 reap_dead start 2021-02-10T11:40:53.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.093 7fea4e86b700 1 --2- 172.21.2.101:0/1883448710 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea34009590 0x7fea3400bab0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 rx=0x7fea40006a40 tx=0x7fea4000c0c0).ready entity=mon.0 client_cookie=96f4688cb428c625 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:53.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.094 7fea4d068700 1 -- 172.21.2.101:0/1883448710 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fea40006a80 con 0x7fea34009590 2021-02-10T11:40:53.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.094 7fea4d068700 1 -- 172.21.2.101:0/1883448710 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea40007070 con 0x7fea34009590 2021-02-10T11:40:53.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.094 7fea4d068700 1 -- 172.21.2.101:0/1883448710 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fea40007590 con 0x7fea34009590 2021-02-10T11:40:53.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.094 7fea5024e700 1 -- 172.21.2.101:0/1883448710 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea34009590 msgr2=0x7fea3400bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:53.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.094 7fea5024e700 1 --2- 172.21.2.101:0/1883448710 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea34009590 0x7fea3400bab0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 rx=0x7fea40006a40 tx=0x7fea4000c0c0).stop 2021-02-10T11:40:53.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.094 7fea5024e700 1 -- 172.21.2.101:0/1883448710 shutdown_connections 2021-02-10T11:40:53.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.094 7fea5024e700 1 --2- 172.21.2.101:0/1883448710 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea34009590 0x7fea3400bab0 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:53.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.094 7fea5024e700 1 -- 172.21.2.101:0/1883448710 >> 172.21.2.101:0/1883448710 conn(0x7fea480ce450 msgr2=0x7fea480d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:53.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.095 7fea5024e700 1 -- 172.21.2.101:0/1883448710 shutdown_connections 2021-02-10T11:40:53.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.095 7fea5024e700 1 -- 172.21.2.101:0/1883448710 wait complete. 2021-02-10T11:40:53.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.109 7fea5024e700 1 Processor -- start 2021-02-10T11:40:53.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.109 7fea5024e700 1 -- start start 2021-02-10T11:40:53.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.110 7fea5024e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea480beeb0 0x7fea480bf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:53.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.110 7fea5024e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea4818ef70 0x7fea4818f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.110 7fea5024e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea4818f990 0x7fea480c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.110 7fea5024e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea480bfc50 con 0x7fea4818f990 2021-02-10T11:40:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.110 7fea5024e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4818fd90 con 0x7fea4818ef70 2021-02-10T11:40:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.110 7fea5024e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea480bf8d0 con 0x7fea480beeb0 2021-02-10T11:40:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.110 7fea4e86b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea4818f990 0x7fea480c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.110 7fea4e86b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea4818f990 0x7fea480c01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56866/0 (socket says 172.21.2.101:56866) 2021-02-10T11:40:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.110 7fea4e86b700 1 -- 172.21.2.101:0/2245647614 learned_addr learned my addr 172.21.2.101:0/2245647614 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.110 7fea4e86b700 1 -- 172.21.2.101:0/2245647614 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea480beeb0 msgr2=0x7fea480bf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.110 7fea4e86b700 1 --2- 172.21.2.101:0/2245647614 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea480beeb0 0x7fea480bf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.110 7fea4e86b700 1 -- 172.21.2.101:0/2245647614 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea4818ef70 msgr2=0x7fea4818f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.110 7fea4e86b700 1 --2- 172.21.2.101:0/2245647614 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea4818ef70 0x7fea4818f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.110 7fea4e86b700 1 -- 172.21.2.101:0/2245647614 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea480c07f0 con 0x7fea4818f990 2021-02-10T11:40:53.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.110 7fea4e86b700 1 --2- 172.21.2.101:0/2245647614 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea4818f990 0x7fea480c01d0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 rx=0x7fea400099c0 tx=0x7fea4000bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:53.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.111 7fea3affd700 1 -- 172.21.2.101:0/2245647614 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fea4000c2f0 con 0x7fea4818f990 2021-02-10T11:40:53.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.111 7fea3affd700 1 -- 172.21.2.101:0/2245647614 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4000c5b0 con 0x7fea4818f990 2021-02-10T11:40:53.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.111 7fea3affd700 1 -- 172.21.2.101:0/2245647614 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fea4000ca40 con 0x7fea4818f990 2021-02-10T11:40:53.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.111 7fea5024e700 1 -- 172.21.2.101:0/2245647614 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea480c07f0 con 0x7fea4818f990 2021-02-10T11:40:53.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.111 7fea5024e700 1 -- 172.21.2.101:0/2245647614 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea480c0d80 con 0x7fea4818f990 2021-02-10T11:40:53.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.111 7fea3affd700 1 -- 172.21.2.101:0/2245647614 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fea400056b0 con 0x7fea4818f990 2021-02-10T11:40:53.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.111 7fea3affd700 1 --2- 172.21.2.101:0/2245647614 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea300186c0 0x7fea3001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:53.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.112 7fea3affd700 1 -- 172.21.2.101:0/2245647614 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fea400328d0 con 0x7fea4818f990 2021-02-10T11:40:53.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.112 7fea4e06a700 1 --2- 172.21.2.101:0/2245647614 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea300186c0 0x7fea3001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:53.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.112 7fea4e06a700 1 --2- 172.21.2.101:0/2245647614 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea300186c0 0x7fea3001abc0 crc :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:53.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.142 7fea5024e700 1 -- 172.21.2.101:0/2245647614 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea480ebc60 con 0x7fea4818f990 2021-02-10T11:40:53.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.144 7fea3affd700 1 -- 172.21.2.101:0/2245647614 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fea4000be30 con 0x7fea4818f990 2021-02-10T11:40:53.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.269 7fea5024e700 1 -- 172.21.2.101:0/2245647614 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea480f0b10 con 0x7fea4818f990 2021-02-10T11:40:53.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.269 7fea3affd700 1 -- 172.21.2.101:0/2245647614 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+14007 (secure 0 0 0) 0x7fea400056b0 con 0x7fea4818f990 2021-02-10T11:40:53.272 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:40:53.273 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":112,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:40:46.730463","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:40:43.698421","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2021-02-10 11:40:44.923352","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:40:53.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.271 7fea5024e700 1 -- 172.21.2.101:0/2245647614 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea300186c0 msgr2=0x7fea3001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:53.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.271 7fea5024e700 1 --2- 172.21.2.101:0/2245647614 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea300186c0 0x7fea3001abc0 crc :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:53.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.271 7fea5024e700 1 -- 172.21.2.101:0/2245647614 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea4818f990 msgr2=0x7fea480c01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:53.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.271 7fea5024e700 1 --2- 172.21.2.101:0/2245647614 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea4818f990 0x7fea480c01d0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 rx=0x7fea400099c0 tx=0x7fea4000bcf0).stop 2021-02-10T11:40:53.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.271 7fea5024e700 1 -- 172.21.2.101:0/2245647614 shutdown_connections 2021-02-10T11:40:53.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.271 7fea5024e700 1 --2- 172.21.2.101:0/2245647614 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea4818f990 0x7fea480c01d0 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:53.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.271 7fea5024e700 1 --2- 172.21.2.101:0/2245647614 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea4818ef70 0x7fea4818f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:53.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.271 7fea5024e700 1 --2- 172.21.2.101:0/2245647614 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea300186c0 0x7fea3001abc0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:53.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.271 7fea5024e700 1 --2- 172.21.2.101:0/2245647614 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea480beeb0 0x7fea480bf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:53.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.271 7fea5024e700 1 -- 172.21.2.101:0/2245647614 >> 172.21.2.101:0/2245647614 conn(0x7fea480c2130 msgr2=0x7fea4809e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:53.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.271 7fea5024e700 1 -- 172.21.2.101:0/2245647614 shutdown_connections 2021-02-10T11:40:53.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.271 7fea5024e700 1 -- 172.21.2.101:0/2245647614 wait complete. 2021-02-10T11:40:53.282 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:40:43.698421', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:40:44.923352', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:40:53.283 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:40:53.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.695 7f76295dc700 1 Processor -- start 2021-02-10T11:40:53.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.695 7f76295dc700 1 -- start start 2021-02-10T11:40:53.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.696 7f76295dc700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f76240d1c70 0x7f76240cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:53.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.696 7f76295dc700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f76240cf580 0x7f76240cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:53.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.696 7f76295dc700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76240fe1c0 con 0x7f76240d1870 2021-02-10T11:40:53.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.696 7f76295dc700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76241b0cb0 con 0x7f76240cf580 2021-02-10T11:40:53.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.696 7f76295dc700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7624109a60 con 0x7f76240d1c70 2021-02-10T11:40:53.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.696 7f76232bf700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f76240d1870 0x7f76241b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57364/0 (socket says 172.21.2.101:57364) 2021-02-10T11:40:53.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.696 7f76232bf700 1 -- 172.21.2.101:0/2492079607 learned_addr learned my addr 172.21.2.101:0/2492079607 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:53.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.696 7f7623ac0700 1 --2- 172.21.2.101:0/2492079607 >> v2:172.21.2.101:3300/0 conn(0x7f76240cf580 0x7f76240cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:53.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.696 7f7622abe700 1 --2- 172.21.2.101:0/2492079607 >> v2:172.21.2.108:3300/0 conn(0x7f76240d1c70 0x7f76240cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:53.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.697 7f7623ac0700 1 -- 172.21.2.101:0/2492079607 >> v2:172.21.2.108:3300/0 conn(0x7f76240d1c70 msgr2=0x7f76240cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:53.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.697 7f7623ac0700 1 --2- 172.21.2.101:0/2492079607 >> v2:172.21.2.108:3300/0 conn(0x7f76240d1c70 0x7f76240cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:53.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.697 7f7623ac0700 1 -- 172.21.2.101:0/2492079607 >> v1:172.21.2.101:6789/0 conn(0x7f76240d1870 legacy=0x7f76241b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:53.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.697 7f7623ac0700 1 -- 172.21.2.101:0/2492079607 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7624088070 con 0x7f76240cf580 2021-02-10T11:40:53.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.697 7f7623ac0700 1 --2- 172.21.2.101:0/2492079607 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f76240cf580 0x7f76240cf980 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 rx=0x7f761c006e30 tx=0x7f761c009e60).ready entity=mon.0 client_cookie=5b680718a7b7cd2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:53.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.697 7f76222bd700 1 -- 172.21.2.101:0/2492079607 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f761c006a50 con 0x7f76240cf580 2021-02-10T11:40:53.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.697 7f76222bd700 1 -- 172.21.2.101:0/2492079607 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761c00b300 con 0x7f76240cf580 2021-02-10T11:40:53.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.697 7f76222bd700 1 -- 172.21.2.101:0/2492079607 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f761c00b7e0 con 0x7f76240cf580 2021-02-10T11:40:53.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.697 7f76295dc700 1 -- 172.21.2.101:0/2492079607 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f76240cf580 msgr2=0x7f76240cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:53.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.697 7f76295dc700 1 --2- 172.21.2.101:0/2492079607 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f76240cf580 0x7f76240cf980 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 rx=0x7f761c006e30 tx=0x7f761c009e60).stop 2021-02-10T11:40:53.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.698 7f76295dc700 1 -- 172.21.2.101:0/2492079607 shutdown_connections 2021-02-10T11:40:53.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.698 7f76295dc700 1 --2- 172.21.2.101:0/2492079607 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f76240cf580 0x7f76240cf980 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:53.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.698 7f76295dc700 1 --2- 172.21.2.101:0/2492079607 >> v2:172.21.2.108:3300/0 conn(0x7f76240d1c70 0x7f76240cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:53.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.698 7f76295dc700 1 -- 172.21.2.101:0/2492079607 >> 172.21.2.101:0/2492079607 conn(0x7f76240ce420 msgr2=0x7f76240d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:53.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.698 7f76295dc700 1 -- 172.21.2.101:0/2492079607 shutdown_connections 2021-02-10T11:40:53.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.698 7f76295dc700 1 -- 172.21.2.101:0/2492079607 wait complete. 2021-02-10T11:40:53.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.713 7f76295dc700 1 Processor -- start 2021-02-10T11:40:53.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.713 7f76295dc700 1 -- start start 2021-02-10T11:40:53.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.713 7f76295dc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f76240a7950 0x7f76240b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:53.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.713 7f76295dc700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f76241a7490 0x7f76241a7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:53.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.713 7f76295dc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f76241a7eb0 0x7f76240ac630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:53.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.713 7f76295dc700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76240a7fb0 con 0x7f76241a7eb0 2021-02-10T11:40:53.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.713 7f76295dc700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76240acc50 con 0x7f76241a7490 2021-02-10T11:40:53.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.713 7f76295dc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76240ace30 con 0x7f76240a7950 2021-02-10T11:40:53.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.713 7f7623ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f76241a7eb0 0x7f76240ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:53.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.713 7f7623ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f76241a7eb0 0x7f76240ac630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56880/0 (socket says 172.21.2.101:56880) 2021-02-10T11:40:53.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.713 7f7623ac0700 1 -- 172.21.2.101:0/153004432 learned_addr learned my addr 172.21.2.101:0/153004432 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:53.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.714 7f7623ac0700 1 -- 172.21.2.101:0/153004432 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f76240a7950 msgr2=0x7f76240b8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:53.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.714 7f7623ac0700 1 --2- 172.21.2.101:0/153004432 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f76240a7950 0x7f76240b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:53.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.714 7f7623ac0700 1 -- 172.21.2.101:0/153004432 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f76241a7490 msgr2=0x7f76241a7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:53.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.714 7f7623ac0700 1 --2- 172.21.2.101:0/153004432 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f76241a7490 0x7f76241a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:53.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.714 7f7623ac0700 1 -- 172.21.2.101:0/153004432 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76240ad050 con 0x7f76241a7eb0 2021-02-10T11:40:53.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.714 7f7623ac0700 1 --2- 172.21.2.101:0/153004432 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f76241a7eb0 0x7f76240ac630 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 rx=0x7f761c00b910 tx=0x7f761c00bcb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:53.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.714 7f760bfff700 1 -- 172.21.2.101:0/153004432 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f761c00bdf0 con 0x7f76241a7eb0 2021-02-10T11:40:53.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.714 7f760bfff700 1 -- 172.21.2.101:0/153004432 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761c005820 con 0x7f76241a7eb0 2021-02-10T11:40:53.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.714 7f76295dc700 1 -- 172.21.2.101:0/153004432 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76240ad050 con 0x7f76241a7eb0 2021-02-10T11:40:53.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.714 7f76295dc700 1 -- 172.21.2.101:0/153004432 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76240a8530 con 0x7f76241a7eb0 2021-02-10T11:40:53.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.714 7f760bfff700 1 -- 172.21.2.101:0/153004432 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f761c005c70 con 0x7f76241a7eb0 2021-02-10T11:40:53.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.715 7f760bfff700 1 -- 172.21.2.101:0/153004432 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f761c005a00 con 0x7f76241a7eb0 2021-02-10T11:40:53.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.715 7f760bfff700 1 --2- 172.21.2.101:0/153004432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76040186c0 0x7f760401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:53.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.715 7f760bfff700 1 -- 172.21.2.101:0/153004432 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f761c032930 con 0x7f76241a7eb0 2021-02-10T11:40:53.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.715 7f76232bf700 1 --2- 172.21.2.101:0/153004432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76040186c0 0x7f760401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:53.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.715 7f76232bf700 1 --2- 172.21.2.101:0/153004432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76040186c0 0x7f760401abc0 crc :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:53.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.746 7f76295dc700 1 -- 172.21.2.101:0/153004432 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7624082a70 con 0x7f76241a7eb0 2021-02-10T11:40:53.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.748 7f760bfff700 1 -- 172.21.2.101:0/153004432 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f761c0093d0 con 0x7f76241a7eb0 2021-02-10T11:40:53.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.877 7f76295dc700 1 -- 172.21.2.101:0/153004432 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f76240a84f0 con 0x7f76241a7eb0 2021-02-10T11:40:53.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.878 7f760bfff700 1 -- 172.21.2.101:0/153004432 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v112) v1 ==== 92+0+10 (secure 0 0 0) 0x7f761c0059d0 con 0x7f76241a7eb0 2021-02-10T11:40:53.881 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:40:53.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.880 7f76295dc700 1 -- 172.21.2.101:0/153004432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76040186c0 msgr2=0x7f760401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:53.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.880 7f76295dc700 1 --2- 172.21.2.101:0/153004432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76040186c0 0x7f760401abc0 crc :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:53.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.880 7f76295dc700 1 -- 172.21.2.101:0/153004432 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f76241a7eb0 msgr2=0x7f76240ac630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:53.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.880 7f76295dc700 1 --2- 172.21.2.101:0/153004432 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f76241a7eb0 0x7f76240ac630 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 rx=0x7f761c00b910 tx=0x7f761c00bcb0).stop 2021-02-10T11:40:53.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.880 7f76295dc700 1 -- 172.21.2.101:0/153004432 shutdown_connections 2021-02-10T11:40:53.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.880 7f76295dc700 1 --2- 172.21.2.101:0/153004432 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f76241a7eb0 0x7f76240ac630 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:53.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.880 7f76295dc700 1 --2- 172.21.2.101:0/153004432 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f76241a7490 0x7f76241a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:53.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.880 7f76295dc700 1 --2- 172.21.2.101:0/153004432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f76040186c0 0x7f760401abc0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:53.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.880 7f76295dc700 1 --2- 172.21.2.101:0/153004432 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f76240a7950 0x7f76240b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:53.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.880 7f76295dc700 1 -- 172.21.2.101:0/153004432 >> 172.21.2.101:0/153004432 conn(0x7f76240a02d0 msgr2=0x7f762409f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:53.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.880 7f76295dc700 1 -- 172.21.2.101:0/153004432 shutdown_connections 2021-02-10T11:40:53.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:53.880 7f76295dc700 1 -- 172.21.2.101:0/153004432 wait complete. 2021-02-10T11:40:53.891 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:40:54.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.303 7f772111c700 1 Processor -- start 2021-02-10T11:40:54.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.303 7f772111c700 1 -- start start 2021-02-10T11:40:54.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.303 7f772111c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f771c0d1870 0x7f771c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:54.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.303 7f772111c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f771c1b0ae0 0x7f771c108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:54.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.303 7f772111c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f771c0fe1c0 con 0x7f771c0d1c70 2021-02-10T11:40:54.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.303 7f772111c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f771c109a60 con 0x7f771c0d1870 2021-02-10T11:40:54.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.303 7f772111c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771c109750 con 0x7f771c1b0ae0 2021-02-10T11:40:54.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.305 7f771b59e700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f771c0d1c70 0x7f771c108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:50792/0 (socket says 172.21.2.101:50792) 2021-02-10T11:40:54.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.305 7f771ad9d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f771c0d1870 0x7f771c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:54.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.305 7f771a59c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f771c1b0ae0 0x7f771c108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:54.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.305 7f771b59e700 1 -- 172.21.2.101:0/3940372106 learned_addr learned my addr 172.21.2.101:0/3940372106 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:54.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.305 7f771a59c700 1 -- 172.21.2.101:0/3940372106 >> v2:172.21.2.108:3300/0 conn(0x7f771c0d1870 msgr2=0x7f771c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:54.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.305 7f771a59c700 1 --2- 172.21.2.101:0/3940372106 >> v2:172.21.2.108:3300/0 conn(0x7f771c0d1870 0x7f771c1b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:54.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.305 7f771a59c700 1 -- 172.21.2.101:0/3940372106 >> v1:172.21.2.108:6789/0 conn(0x7f771c0d1c70 legacy=0x7f771c108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:54.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.305 7f771a59c700 1 -- 172.21.2.101:0/3940372106 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771c088070 con 0x7f771c1b0ae0 2021-02-10T11:40:54.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.305 7f771a59c700 1 --2- 172.21.2.101:0/3940372106 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f771c1b0ae0 0x7f771c108940 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f770c006650 tx=0x7f770c009a80).ready entity=mon.2 client_cookie=184f2edb8bde3889 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:54.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.306 7f7719d9b700 1 -- 172.21.2.101:0/3940372106 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f770c00c0c0 con 0x7f771c1b0ae0 2021-02-10T11:40:54.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.306 7f7719d9b700 1 -- 172.21.2.101:0/3940372106 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c00c340 con 0x7f771c1b0ae0 2021-02-10T11:40:54.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.306 7f7719d9b700 1 -- 172.21.2.101:0/3940372106 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f770c00c860 con 0x7f771c1b0ae0 2021-02-10T11:40:54.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.306 7f772111c700 1 -- 172.21.2.101:0/3940372106 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f771c1b0ae0 msgr2=0x7f771c108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:54.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.306 7f772111c700 1 --2- 172.21.2.101:0/3940372106 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f771c1b0ae0 0x7f771c108940 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f770c006650 tx=0x7f770c009a80).stop 2021-02-10T11:40:54.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.307 7f772111c700 1 -- 172.21.2.101:0/3940372106 shutdown_connections 2021-02-10T11:40:54.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.307 7f772111c700 1 --2- 172.21.2.101:0/3940372106 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f771c1b0ae0 0x7f771c108940 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:54.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.307 7f772111c700 1 --2- 172.21.2.101:0/3940372106 >> v2:172.21.2.108:3300/0 conn(0x7f771c0d1870 0x7f771c1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:54.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.307 7f772111c700 1 -- 172.21.2.101:0/3940372106 >> 172.21.2.101:0/3940372106 conn(0x7f771c0ce420 msgr2=0x7f771c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:54.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.307 7f772111c700 1 -- 172.21.2.101:0/3940372106 shutdown_connections 2021-02-10T11:40:54.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.307 7f772111c700 1 -- 172.21.2.101:0/3940372106 wait complete. 2021-02-10T11:40:54.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.321 7f772111c700 1 Processor -- start 2021-02-10T11:40:54.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.321 7f772111c700 1 -- start start 2021-02-10T11:40:54.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.322 7f772111c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f771c0a7950 0x7f771c0b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:54.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.322 7f772111c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f771c1a7490 0x7f771c1a7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:54.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.322 7f772111c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f771c1a7eb0 0x7f771c0ac630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:54.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.322 7f772111c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771c0a7fb0 con 0x7f771c1a7490 2021-02-10T11:40:54.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.322 7f772111c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771c0acc50 con 0x7f771c0a7950 2021-02-10T11:40:54.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.322 7f772111c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771c0ace30 con 0x7f771c1a7eb0 2021-02-10T11:40:54.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.322 7f771a59c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f771c1a7490 0x7f771c1a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:54.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.322 7f771a59c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f771c1a7490 0x7f771c1a7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56890/0 (socket says 172.21.2.101:56890) 2021-02-10T11:40:54.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.322 7f771a59c700 1 -- 172.21.2.101:0/930320771 learned_addr learned my addr 172.21.2.101:0/930320771 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:54.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.322 7f771a59c700 1 -- 172.21.2.101:0/930320771 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f771c1a7eb0 msgr2=0x7f771c0ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:54.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.322 7f771b59e700 1 --2- 172.21.2.101:0/930320771 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f771c1a7eb0 0x7f771c0ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:54.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.322 7f771a59c700 1 --2- 172.21.2.101:0/930320771 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f771c1a7eb0 0x7f771c0ac630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:54.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.322 7f771ad9d700 1 --2- 172.21.2.101:0/930320771 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f771c0a7950 0x7f771c0b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:54.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.322 7f771a59c700 1 -- 172.21.2.101:0/930320771 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f771c0a7950 msgr2=0x7f771c0b8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:54.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.322 7f771a59c700 1 --2- 172.21.2.101:0/930320771 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f771c0a7950 0x7f771c0b8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:54.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.322 7f771a59c700 1 -- 172.21.2.101:0/930320771 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771c0ad050 con 0x7f771c1a7490 2021-02-10T11:40:54.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.322 7f771b59e700 1 --2- 172.21.2.101:0/930320771 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f771c1a7eb0 0x7f771c0ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:40:54.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.322 7f771ad9d700 1 --2- 172.21.2.101:0/930320771 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f771c0a7950 0x7f771c0b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:40:54.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.323 7f771a59c700 1 --2- 172.21.2.101:0/930320771 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f771c1a7490 0x7f771c1a7890 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 rx=0x7f770c009630 tx=0x7f770c00b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:54.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.323 7f77137fe700 1 -- 172.21.2.101:0/930320771 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f770c0093a0 con 0x7f771c1a7490 2021-02-10T11:40:54.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.323 7f77137fe700 1 -- 172.21.2.101:0/930320771 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c00bd70 con 0x7f771c1a7490 2021-02-10T11:40:54.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.323 7f772111c700 1 -- 172.21.2.101:0/930320771 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f771c0ad050 con 0x7f771c1a7490 2021-02-10T11:40:54.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.323 7f77137fe700 1 -- 172.21.2.101:0/930320771 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f770c0054d0 con 0x7f771c1a7490 2021-02-10T11:40:54.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.323 7f772111c700 1 -- 172.21.2.101:0/930320771 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f771c0a8530 con 0x7f771c1a7490 2021-02-10T11:40:54.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.324 7f77137fe700 1 -- 172.21.2.101:0/930320771 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f770c0052e0 con 0x7f771c1a7490 2021-02-10T11:40:54.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.324 7f77137fe700 1 --2- 172.21.2.101:0/930320771 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77000186c0 0x7f770001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:54.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.324 7f77137fe700 1 -- 172.21.2.101:0/930320771 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f770c032650 con 0x7f771c1a7490 2021-02-10T11:40:54.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.324 7f771ad9d700 1 --2- 172.21.2.101:0/930320771 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77000186c0 0x7f770001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:54.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.324 7f771ad9d700 1 --2- 172.21.2.101:0/930320771 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77000186c0 0x7f770001abc0 crc :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:54.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.353 7f772111c700 1 -- 172.21.2.101:0/930320771 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f771c082a70 con 0x7f771c1a7490 2021-02-10T11:40:54.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.356 7f77137fe700 1 -- 172.21.2.101:0/930320771 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f770c01f510 con 0x7f771c1a7490 2021-02-10T11:40:54.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.484 7f772111c700 1 -- 172.21.2.101:0/930320771 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f771c082a70 con 0x7f771c1a7490 2021-02-10T11:40:54.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.484 7f77137fe700 1 -- 172.21.2.101:0/930320771 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v112) v1 ==== 104+0+10 (secure 0 0 0) 0x7f770c0052e0 con 0x7f771c1a7490 2021-02-10T11:40:54.487 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:40:54.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.486 7f772111c700 1 -- 172.21.2.101:0/930320771 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77000186c0 msgr2=0x7f770001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:54.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.486 7f772111c700 1 --2- 172.21.2.101:0/930320771 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77000186c0 0x7f770001abc0 crc :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:54.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.486 7f772111c700 1 -- 172.21.2.101:0/930320771 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f771c1a7490 msgr2=0x7f771c1a7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:54.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.486 7f772111c700 1 --2- 172.21.2.101:0/930320771 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f771c1a7490 0x7f771c1a7890 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 rx=0x7f770c009630 tx=0x7f770c00b980).stop 2021-02-10T11:40:54.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.486 7f772111c700 1 -- 172.21.2.101:0/930320771 shutdown_connections 2021-02-10T11:40:54.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.486 7f772111c700 1 --2- 172.21.2.101:0/930320771 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77000186c0 0x7f770001abc0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:54.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.486 7f772111c700 1 --2- 172.21.2.101:0/930320771 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f771c1a7eb0 0x7f771c0ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:54.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.486 7f772111c700 1 --2- 172.21.2.101:0/930320771 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f771c1a7490 0x7f771c1a7890 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:54.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.486 7f772111c700 1 --2- 172.21.2.101:0/930320771 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f771c0a7950 0x7f771c0b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:54.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.486 7f772111c700 1 -- 172.21.2.101:0/930320771 >> 172.21.2.101:0/930320771 conn(0x7f771c0a02d0 msgr2=0x7f771c09f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:54.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.486 7f772111c700 1 -- 172.21.2.101:0/930320771 shutdown_connections 2021-02-10T11:40:54.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.486 7f772111c700 1 -- 172.21.2.101:0/930320771 wait complete. 2021-02-10T11:40:54.497 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:40:54.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.912 7ff7e4466700 1 Processor -- start 2021-02-10T11:40:54.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.912 7ff7e4466700 1 -- start start 2021-02-10T11:40:54.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.913 7ff7e4466700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff7dc0d1870 0x7ff7dc1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:54.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.913 7ff7e4466700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff7dc1b0ae0 0x7ff7dc108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:54.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.913 7ff7e4466700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7dc0fe1c0 con 0x7ff7dc0d1c70 2021-02-10T11:40:54.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.913 7ff7e4466700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7dc109a60 con 0x7ff7dc1b0ae0 2021-02-10T11:40:54.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.913 7ff7e4466700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7dc109750 con 0x7ff7dc0d1870 2021-02-10T11:40:54.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.913 7ff7e2a83700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff7dc0d1c70 0x7ff7dc108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57392/0 (socket says 172.21.2.101:57392) 2021-02-10T11:40:54.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.913 7ff7e2a83700 1 -- 172.21.2.101:0/211961503 learned_addr learned my addr 172.21.2.101:0/211961503 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:54.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.913 7ff7e1a81700 1 --2- 172.21.2.101:0/211961503 >> v2:172.21.2.101:3300/0 conn(0x7ff7dc1b0ae0 0x7ff7dc108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:54.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.913 7ff7e1a81700 1 -- 172.21.2.101:0/211961503 >> v2:172.21.2.108:3300/0 conn(0x7ff7dc0d1870 msgr2=0x7ff7dc1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:54.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.913 7ff7e1a81700 1 --2- 172.21.2.101:0/211961503 >> v2:172.21.2.108:3300/0 conn(0x7ff7dc0d1870 0x7ff7dc1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:54.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.913 7ff7e1a81700 1 -- 172.21.2.101:0/211961503 >> v1:172.21.2.101:6789/0 conn(0x7ff7dc0d1c70 legacy=0x7ff7dc108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:54.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.913 7ff7e1a81700 1 -- 172.21.2.101:0/211961503 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7dc088070 con 0x7ff7dc1b0ae0 2021-02-10T11:40:54.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.913 7ff7e1a81700 1 --2- 172.21.2.101:0/211961503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff7dc1b0ae0 0x7ff7dc108940 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 rx=0x7ff7d0006650 tx=0x7ff7d0009a80).ready entity=mon.0 client_cookie=f30df53c668aa470 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:54.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.914 7ff7e1280700 1 -- 172.21.2.101:0/211961503 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff7d00009b0 con 0x7ff7dc1b0ae0 2021-02-10T11:40:54.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.914 7ff7e1280700 1 -- 172.21.2.101:0/211961503 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d000c0c0 con 0x7ff7dc1b0ae0 2021-02-10T11:40:54.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.914 7ff7e1280700 1 -- 172.21.2.101:0/211961503 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff7d000c510 con 0x7ff7dc1b0ae0 2021-02-10T11:40:54.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.914 7ff7e4466700 1 -- 172.21.2.101:0/211961503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff7dc1b0ae0 msgr2=0x7ff7dc108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:54.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.914 7ff7e4466700 1 --2- 172.21.2.101:0/211961503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff7dc1b0ae0 0x7ff7dc108940 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 rx=0x7ff7d0006650 tx=0x7ff7d0009a80).stop 2021-02-10T11:40:54.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.914 7ff7e4466700 1 -- 172.21.2.101:0/211961503 shutdown_connections 2021-02-10T11:40:54.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.914 7ff7e4466700 1 --2- 172.21.2.101:0/211961503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff7dc1b0ae0 0x7ff7dc108940 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:54.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.914 7ff7e4466700 1 --2- 172.21.2.101:0/211961503 >> v2:172.21.2.108:3300/0 conn(0x7ff7dc0d1870 0x7ff7dc1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:54.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.914 7ff7e4466700 1 -- 172.21.2.101:0/211961503 >> 172.21.2.101:0/211961503 conn(0x7ff7dc0ce420 msgr2=0x7ff7dc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:54.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.914 7ff7e4466700 1 -- 172.21.2.101:0/211961503 shutdown_connections 2021-02-10T11:40:54.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.914 7ff7e4466700 1 -- 172.21.2.101:0/211961503 wait complete. 2021-02-10T11:40:54.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.929 7ff7e4466700 1 Processor -- start 2021-02-10T11:40:54.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.929 7ff7e4466700 1 -- start start 2021-02-10T11:40:54.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.929 7ff7e4466700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff7dc0c3820 0x7ff7dc0a7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:54.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.929 7ff7e4466700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff7dc1a44c0 0x7ff7dc1a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:54.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.929 7ff7e4466700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff7dc0863d0 0x7ff7dc0867d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:54.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.929 7ff7e4466700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc085ed0 con 0x7ff7dc0863d0 2021-02-10T11:40:54.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.929 7ff7e4466700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc0c0740 con 0x7ff7dc0c3820 2021-02-10T11:40:54.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.929 7ff7e4466700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc086df0 con 0x7ff7dc1a44c0 2021-02-10T11:40:54.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.929 7ff7e2a83700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff7dc0863d0 0x7ff7dc0867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:54.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.929 7ff7e2a83700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff7dc0863d0 0x7ff7dc0867d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56906/0 (socket says 172.21.2.101:56906) 2021-02-10T11:40:54.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.929 7ff7e2a83700 1 -- 172.21.2.101:0/2076209960 learned_addr learned my addr 172.21.2.101:0/2076209960 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:54.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.929 7ff7e2a83700 1 -- 172.21.2.101:0/2076209960 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff7dc1a44c0 msgr2=0x7ff7dc1a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:54.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.929 7ff7e2a83700 1 --2- 172.21.2.101:0/2076209960 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff7dc1a44c0 0x7ff7dc1a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:54.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.929 7ff7e2a83700 1 -- 172.21.2.101:0/2076209960 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff7dc0c3820 msgr2=0x7ff7dc0a7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:54.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.929 7ff7e2a83700 1 --2- 172.21.2.101:0/2076209960 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff7dc0c3820 0x7ff7dc0a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:54.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.930 7ff7e2a83700 1 -- 172.21.2.101:0/2076209960 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7dc087010 con 0x7ff7dc0863d0 2021-02-10T11:40:54.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.930 7ff7e2a83700 1 --2- 172.21.2.101:0/2076209960 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff7dc0863d0 0x7ff7dc0867d0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 rx=0x7ff7d400a900 tx=0x7ff7d400aca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:54.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.930 7ff7ceffd700 1 -- 172.21.2.101:0/2076209960 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff7d4007020 con 0x7ff7dc0863d0 2021-02-10T11:40:54.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.930 7ff7ceffd700 1 -- 172.21.2.101:0/2076209960 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d4007320 con 0x7ff7dc0863d0 2021-02-10T11:40:54.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.930 7ff7e4466700 1 -- 172.21.2.101:0/2076209960 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7dc087010 con 0x7ff7dc0863d0 2021-02-10T11:40:54.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.930 7ff7ceffd700 1 -- 172.21.2.101:0/2076209960 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff7d4004720 con 0x7ff7dc0863d0 2021-02-10T11:40:54.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.930 7ff7e4466700 1 -- 172.21.2.101:0/2076209960 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7dc087010 con 0x7ff7dc0863d0 2021-02-10T11:40:54.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.930 7ff7ceffd700 1 -- 172.21.2.101:0/2076209960 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff7d400ade0 con 0x7ff7dc0863d0 2021-02-10T11:40:54.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.930 7ff7ceffd700 1 --2- 172.21.2.101:0/2076209960 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7c40186d0 0x7ff7c401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:54.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.931 7ff7ceffd700 1 -- 172.21.2.101:0/2076209960 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff7d40329b0 con 0x7ff7dc0863d0 2021-02-10T11:40:54.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.931 7ff7e2282700 1 --2- 172.21.2.101:0/2076209960 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7c40186d0 0x7ff7c401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:54.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.931 7ff7e2282700 1 --2- 172.21.2.101:0/2076209960 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7c40186d0 0x7ff7c401abd0 crc :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:54.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.963 7ff7e4466700 1 -- 172.21.2.101:0/2076209960 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7dc0fe1c0 con 0x7ff7dc0863d0 2021-02-10T11:40:54.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:54.966 7ff7ceffd700 1 -- 172.21.2.101:0/2076209960 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff7d4007c90 con 0x7ff7dc0863d0 2021-02-10T11:40:55.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.093 7ff7e4466700 1 -- 172.21.2.101:0/2076209960 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7ff7dc0fe1c0 con 0x7ff7dc0863d0 2021-02-10T11:40:55.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.094 7ff7ceffd700 1 -- 172.21.2.101:0/2076209960 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v112) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff7d400ade0 con 0x7ff7dc0863d0 2021-02-10T11:40:55.097 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:40:55.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.096 7ff7e4466700 1 -- 172.21.2.101:0/2076209960 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7c40186d0 msgr2=0x7ff7c401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:55.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.096 7ff7e4466700 1 --2- 172.21.2.101:0/2076209960 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7c40186d0 0x7ff7c401abd0 crc :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:55.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.096 7ff7e4466700 1 -- 172.21.2.101:0/2076209960 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff7dc0863d0 msgr2=0x7ff7dc0867d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:55.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.096 7ff7e4466700 1 --2- 172.21.2.101:0/2076209960 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff7dc0863d0 0x7ff7dc0867d0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 rx=0x7ff7d400a900 tx=0x7ff7d400aca0).stop 2021-02-10T11:40:55.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.096 7ff7e4466700 1 -- 172.21.2.101:0/2076209960 shutdown_connections 2021-02-10T11:40:55.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.096 7ff7e4466700 1 --2- 172.21.2.101:0/2076209960 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7c40186d0 0x7ff7c401abd0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:55.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.096 7ff7e4466700 1 --2- 172.21.2.101:0/2076209960 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff7dc1a44c0 0x7ff7dc1a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:55.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.096 7ff7e4466700 1 --2- 172.21.2.101:0/2076209960 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff7dc0863d0 0x7ff7dc0867d0 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:55.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.096 7ff7e4466700 1 --2- 172.21.2.101:0/2076209960 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff7dc0c3820 0x7ff7dc0a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:55.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.096 7ff7e4466700 1 -- 172.21.2.101:0/2076209960 >> 172.21.2.101:0/2076209960 conn(0x7ff7dc0872c0 msgr2=0x7ff7dc095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:55.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.096 7ff7e4466700 1 -- 172.21.2.101:0/2076209960 shutdown_connections 2021-02-10T11:40:55.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.096 7ff7e4466700 1 -- 172.21.2.101:0/2076209960 wait complete. 2021-02-10T11:40:55.107 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:40:55.107 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:40:55.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.516 7fb4c4851700 1 Processor -- start 2021-02-10T11:40:55.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.516 7fb4c4851700 1 -- start start 2021-02-10T11:40:55.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.517 7fb4c4851700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4c00d0340 0x7fb4c00ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:55.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.517 7fb4c4851700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb4c00d1c70 0x7fb4c00efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:55.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.517 7fb4c4851700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c00f0b10 con 0x7fb4c00d1870 2021-02-10T11:40:55.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.517 7fb4c4851700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4c00e5ed0 con 0x7fb4c00d1c70 2021-02-10T11:40:55.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.517 7fb4c4851700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c01b0be0 con 0x7fb4c00d0340 2021-02-10T11:40:55.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.517 7fb4be61c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb4c00d1870 0x7fb4c01b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57404/0 (socket says 172.21.2.101:57404) 2021-02-10T11:40:55.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.517 7fb4be61c700 1 -- 172.21.2.101:0/4167386197 learned_addr learned my addr 172.21.2.101:0/4167386197 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:40:55.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.517 7fb4bd61a700 1 -- 172.21.2.101:0/4167386197 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 72317091 0 0) 0x7fb4a80010b0 con 0x7fb4c00d1870 2021-02-10T11:40:55.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.517 7fb4bd61a700 1 -- 172.21.2.101:0/4167386197 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4ac002fe0 con 0x7fb4c00d1870 2021-02-10T11:40:55.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bee1d700 1 --2- 172.21.2.101:0/4167386197 >> v2:172.21.2.108:3300/0 conn(0x7fb4c00d1c70 0x7fb4c00efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:55.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bde1b700 1 --2- 172.21.2.101:0/4167386197 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4c00d0340 0x7fb4c00ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:55.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bd61a700 1 -- 172.21.2.101:0/4167386197 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1983759325 0 0) 0x7fb4a8001010 con 0x7fb4c00d1870 2021-02-10T11:40:55.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bd61a700 1 -- 172.21.2.101:0/4167386197 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4c00d0340 msgr2=0x7fb4c00ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:55.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bd61a700 1 --2- 172.21.2.101:0/4167386197 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4c00d0340 0x7fb4c00ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:55.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bd61a700 1 -- 172.21.2.101:0/4167386197 >> v2:172.21.2.108:3300/0 conn(0x7fb4c00d1c70 msgr2=0x7fb4c00efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:55.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bd61a700 1 --2- 172.21.2.101:0/4167386197 >> v2:172.21.2.108:3300/0 conn(0x7fb4c00d1c70 0x7fb4c00efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:55.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bd61a700 1 -- 172.21.2.101:0/4167386197 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c00f20a0 con 0x7fb4c00d1870 2021-02-10T11:40:55.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bd61a700 1 -- 172.21.2.101:0/4167386197 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fb4a8001530 con 0x7fb4c00d1870 2021-02-10T11:40:55.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bd61a700 1 -- 172.21.2.101:0/4167386197 >> v1:172.21.2.101:6789/0 conn(0x7fb4c00d1870 legacy=0x7fb4c01b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:55.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bd61a700 1 --2- 172.21.2.101:0/4167386197 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4ac003af0 0x7fb4ac005f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:55.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bd61a700 1 --2- 172.21.2.101:0/4167386197 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4ac006780 0x7fb4ac008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:55.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bd61a700 1 --2- 172.21.2.101:0/4167386197 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4ac009580 0x7fb4ac00ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:55.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bd61a700 1 -- 172.21.2.101:0/4167386197 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac00c2c0 con 0x7fb4ac006780 2021-02-10T11:40:55.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bd61a700 1 -- 172.21.2.101:0/4167386197 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac0504b0 con 0x7fb4ac003af0 2021-02-10T11:40:55.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bd61a700 1 -- 172.21.2.101:0/4167386197 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac0506d0 con 0x7fb4ac009580 2021-02-10T11:40:55.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bde1b700 1 --2- 172.21.2.101:0/4167386197 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4ac006780 0x7fb4ac008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:55.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4be61c700 1 --2- 172.21.2.101:0/4167386197 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4ac003af0 0x7fb4ac005f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:55.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bde1b700 1 -- 172.21.2.101:0/4167386197 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4ac009580 msgr2=0x7fb4ac00ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:55.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bde1b700 1 --2- 172.21.2.101:0/4167386197 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4ac009580 0x7fb4ac00ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:55.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bde1b700 1 -- 172.21.2.101:0/4167386197 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4ac003af0 msgr2=0x7fb4ac005f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:40:55.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bde1b700 1 --2- 172.21.2.101:0/4167386197 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4ac003af0 0x7fb4ac005f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:55.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bde1b700 1 -- 172.21.2.101:0/4167386197 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ac050970 con 0x7fb4ac006780 2021-02-10T11:40:55.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4be61c700 1 --2- 172.21.2.101:0/4167386197 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4ac003af0 0x7fb4ac005f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:40:55.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bde1b700 1 --2- 172.21.2.101:0/4167386197 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4ac006780 0x7fb4ac008c80 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 rx=0x7fb4b0006a90 tx=0x7fb4b0009a90).ready entity=mon.0 client_cookie=5821120065c6d6a4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:55.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bd61a700 1 -- 172.21.2.101:0/4167386197 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb4b000c0c0 con 0x7fb4ac006780 2021-02-10T11:40:55.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bd61a700 1 -- 172.21.2.101:0/4167386197 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b000c400 con 0x7fb4ac006780 2021-02-10T11:40:55.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bd61a700 1 -- 172.21.2.101:0/4167386197 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb4b000c8e0 con 0x7fb4ac006780 2021-02-10T11:40:55.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4c4851700 1 -- 172.21.2.101:0/4167386197 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4ac006780 msgr2=0x7fb4ac008c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:55.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4c4851700 1 --2- 172.21.2.101:0/4167386197 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4ac006780 0x7fb4ac008c80 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 rx=0x7fb4b0006a90 tx=0x7fb4b0009a90).stop 2021-02-10T11:40:55.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.518 7fb4bee1d700 1 -- 172.21.2.101:0/4167386197 reap_dead start 2021-02-10T11:40:55.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.519 7fb4c4851700 1 -- 172.21.2.101:0/4167386197 shutdown_connections 2021-02-10T11:40:55.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.519 7fb4c4851700 1 -- 172.21.2.101:0/4167386197 >> 172.21.2.101:0/4167386197 conn(0x7fb4c00ce450 msgr2=0x7fb4c00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:55.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.519 7fb4c4851700 1 -- 172.21.2.101:0/4167386197 shutdown_connections 2021-02-10T11:40:55.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.519 7fb4c4851700 1 -- 172.21.2.101:0/4167386197 wait complete. 2021-02-10T11:40:55.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.534 7fb4c4851700 1 Processor -- start 2021-02-10T11:40:55.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.534 7fb4c4851700 1 -- start start 2021-02-10T11:40:55.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.534 7fb4c4851700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4c00b1b20 0x7fb4c01a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:55.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.534 7fb4c4851700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4c01a13c0 0x7fb4c00c0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:55.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.534 7fb4c4851700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4c01a17c0 0x7fb4c00c15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:55.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.534 7fb4c4851700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c00c1c10 con 0x7fb4c00b1b20 2021-02-10T11:40:55.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.534 7fb4c4851700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c00c1e30 con 0x7fb4c01a13c0 2021-02-10T11:40:55.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.534 7fb4c4851700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c00c2050 con 0x7fb4c01a17c0 2021-02-10T11:40:55.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.535 7fb4be61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4c00b1b20 0x7fb4c01a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:55.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.535 7fb4be61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4c00b1b20 0x7fb4c01a0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56922/0 (socket says 172.21.2.101:56922) 2021-02-10T11:40:55.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.535 7fb4be61c700 1 -- 172.21.2.101:0/2159582697 learned_addr learned my addr 172.21.2.101:0/2159582697 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:40:55.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.535 7fb4be61c700 1 -- 172.21.2.101:0/2159582697 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4c01a17c0 msgr2=0x7fb4c00c15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:55.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.535 7fb4be61c700 1 --2- 172.21.2.101:0/2159582697 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4c01a17c0 0x7fb4c00c15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:55.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.535 7fb4be61c700 1 -- 172.21.2.101:0/2159582697 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4c01a13c0 msgr2=0x7fb4c00c0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:55.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.535 7fb4be61c700 1 --2- 172.21.2.101:0/2159582697 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4c01a13c0 0x7fb4c00c0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:55.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.535 7fb4be61c700 1 -- 172.21.2.101:0/2159582697 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c00bcf80 con 0x7fb4c00b1b20 2021-02-10T11:40:55.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.535 7fb4be61c700 1 --2- 172.21.2.101:0/2159582697 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4c00b1b20 0x7fb4c01a0da0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 rx=0x7fb4a8007650 tx=0x7fb4a800a900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:55.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.535 7fb4b6ffd700 1 -- 172.21.2.101:0/2159582697 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb4a800acf0 con 0x7fb4c00b1b20 2021-02-10T11:40:55.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.535 7fb4b6ffd700 1 -- 172.21.2.101:0/2159582697 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a8004350 con 0x7fb4c00b1b20 2021-02-10T11:40:55.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.535 7fb4c4851700 1 -- 172.21.2.101:0/2159582697 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c00bcf80 con 0x7fb4c00b1b20 2021-02-10T11:40:55.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.535 7fb4b6ffd700 1 -- 172.21.2.101:0/2159582697 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb4a8004870 con 0x7fb4c00b1b20 2021-02-10T11:40:55.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.536 7fb4c4851700 1 -- 172.21.2.101:0/2159582697 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c00bd510 con 0x7fb4c00b1b20 2021-02-10T11:40:55.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.536 7fb4b6ffd700 1 -- 172.21.2.101:0/2159582697 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb4a8004530 con 0x7fb4c00b1b20 2021-02-10T11:40:55.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.536 7fb4b6ffd700 1 --2- 172.21.2.101:0/2159582697 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4a40186c0 0x7fb4a401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:40:55.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.536 7fb4b6ffd700 1 -- 172.21.2.101:0/2159582697 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb4a8032540 con 0x7fb4c00b1b20 2021-02-10T11:40:55.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.536 7fb4bde1b700 1 --2- 172.21.2.101:0/2159582697 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4a40186c0 0x7fb4a401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:40:55.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.537 7fb4bde1b700 1 --2- 172.21.2.101:0/2159582697 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4a40186c0 0x7fb4a401abc0 crc :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:40:55.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.567 7fb4c4851700 1 -- 172.21.2.101:0/2159582697 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c00ebc00 con 0x7fb4c00b1b20 2021-02-10T11:40:55.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.569 7fb4b6ffd700 1 -- 172.21.2.101:0/2159582697 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb4a80018f0 con 0x7fb4c00b1b20 2021-02-10T11:40:55.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.696 7fb4c4851700 1 -- 172.21.2.101:0/2159582697 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4c00f0b10 con 0x7fb4c00b1b20 2021-02-10T11:40:55.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.696 7fb4b6ffd700 1 -- 172.21.2.101:0/2159582697 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+2104 (secure 0 0 0) 0x7fb4a8004510 con 0x7fb4c00b1b20 2021-02-10T11:40:55.699 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:40:55.699 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":88,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2840404451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2840404451},{"type":"v1","addr":"172.21.2.101:6835","nonce":2840404451}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":84}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:40:45.728009","modified":"2021-02-10 11:40:47.738972","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7450},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7450":{"gid":7450,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2757440614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2757440614},{"type":"v1","addr":"172.21.2.108:6833","nonce":2757440614}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2021-02-10T11:40:55.699 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 88 2021-02-10T11:40:55.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.698 7fb4c4851700 1 -- 172.21.2.101:0/2159582697 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4a40186c0 msgr2=0x7fb4a401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:55.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.698 7fb4c4851700 1 --2- 172.21.2.101:0/2159582697 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4a40186c0 0x7fb4a401abc0 crc :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:55.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.698 7fb4c4851700 1 -- 172.21.2.101:0/2159582697 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4c00b1b20 msgr2=0x7fb4c01a0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:40:55.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.698 7fb4c4851700 1 --2- 172.21.2.101:0/2159582697 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4c00b1b20 0x7fb4c01a0da0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 rx=0x7fb4a8007650 tx=0x7fb4a800a900).stop 2021-02-10T11:40:55.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.698 7fb4c4851700 1 -- 172.21.2.101:0/2159582697 shutdown_connections 2021-02-10T11:40:55.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.698 7fb4c4851700 1 --2- 172.21.2.101:0/2159582697 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb4a40186c0 0x7fb4a401abc0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:55.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.698 7fb4c4851700 1 --2- 172.21.2.101:0/2159582697 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4c01a17c0 0x7fb4c00c15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:55.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.698 7fb4c4851700 1 --2- 172.21.2.101:0/2159582697 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4c01a13c0 0x7fb4c00c0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:40:55.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.698 7fb4c4851700 1 --2- 172.21.2.101:0/2159582697 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4c00b1b20 0x7fb4c01a0da0 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:40:55.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.698 7fb4c4851700 1 -- 172.21.2.101:0/2159582697 >> 172.21.2.101:0/2159582697 conn(0x7fb4c009eb60 msgr2=0x7fb4c00ebe30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:40:55.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.698 7fb4c4851700 1 -- 172.21.2.101:0/2159582697 shutdown_connections 2021-02-10T11:40:55.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:40:55.698 7fb4c4851700 1 -- 172.21.2.101:0/2159582697 wait complete. 2021-02-10T11:40:55.911 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:40:45.728009', 'modified': '2021-02-10 11:40:47.738972', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7450}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7450': {'gid': 7450, 'name': 'b', 'rank': 0, 'incarnation': 85, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/2757440614', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 2757440614}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 2757440614}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:40:55.911 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:40:55.911 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:40:56.311 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:40:56.311 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:40:56.311 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:40:56.311 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 7450, 2021-02-10T11:40:56.311 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:40:56.311 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:40:56.311 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 10.577343875, 2021-02-10T11:40:56.311 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 88, 2021-02-10T11:40:56.311 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 112, 2021-02-10T11:40:56.311 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 112, 2021-02-10T11:40:56.312 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 16.216790129 2021-02-10T11:40:56.312 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:40:56.315 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 7450, "want_state": "up:active", "state": "up:active", "rank_uptime": 10.577343875, "mdsmap_epoch": 88, "osdmap_epoch": 112, "osdmap_epoch_barrier": 112, "uptime": 16.216790129 } 2021-02-10T11:40:56.315 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:40:56.315 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-02-10T11:40:56.315 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-02-10T11:40:56.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:56.315 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:40:56.652 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:40:56.997 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:40:56.997 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:40:57.341 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:40:57.341 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:40:57.342 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:40:57.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:57.342 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:40:57.677 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-02-10T11:40:57.677 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:40:57.680 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:40:58.023 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:40:58.023 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:40:58.090 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[147200]: starting ceph client 2021-02-10T11:40:58.090 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:2021-02-10 11:40:58.087 7fd168d73f80 -1 init, newargv = 0x55a7c3775e00 newargc=9 2021-02-10T11:40:58.096 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[147200]: starting fuse 2021-02-10T11:40:58.366 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:40:58.366 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:40:58.367 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:40:58.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:58.367 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:40:58.700 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:40:58.701 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2021-02-10T11:40:58.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:58.701 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:40:58.701 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:40:58.701 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:40:58.701 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:40:58.701 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:40:58.701 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:40:58.701 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:40:58.702 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-02-10T11:40:58.702 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:40:58.702 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:40:58.702 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:40:58.702 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:40:58.702 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:40:58.702 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:40:58.702 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:40:58.702 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:40:58.702 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:40:58.702 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:40:58.702 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:40:58.702 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:40:58.703 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:40:58.703 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.0")) 2021-02-10T11:40:58.703 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:40:59.063 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.0.147200.asok 2021-02-10T11:40:59.066 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.147200.asok 2021-02-10T11:40:59.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:59.067 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.147200.asok status 2021-02-10T11:40:59.455 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:40:59.455 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:40:59.456 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:40:59.456 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:40:59.456 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "0", 2021-02-10T11:40:59.456 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:40:59.456 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-02-10T11:40:59.456 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "147200", 2021-02-10T11:40:59.456 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:40:59.456 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:59.456 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:40:59.456 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:40:59.456 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 10590, 2021-02-10T11:40:59.456 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:40:59.456 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:40:59.456 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:40:59.457 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 10590 2021-02-10T11:40:59.457 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:59.457 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:40:59.457 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:59.457 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:40:59.457 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1211615379 2021-02-10T11:40:59.457 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:40:59.457 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:59.457 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:40:59.457 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:40:59.457 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:40:59.457 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1211615379 2021-02-10T11:40:59.457 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:40:59.457 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.10590 172.21.2.101:0/1211615379", 2021-02-10T11:40:59.458 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/1211615379", 2021-02-10T11:40:59.458 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:40:59.458 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 88, 2021-02-10T11:40:59.458 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 112, 2021-02-10T11:40:59.458 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:40:59.458 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:40:59.458 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:40:59.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:59.460 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:40:59.460 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:40:59.794 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:40:59.794 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:40:59.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:40:59.795 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:41:00.142 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:41:00.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.552 7fa15fde4700 1 Processor -- start 2021-02-10T11:41:00.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.552 7fa15fde4700 1 -- start start 2021-02-10T11:41:00.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.553 7fa15fde4700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa1580d1870 0x7fa1581b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:00.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.553 7fa15fde4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa1580f0650 0x7fa1580cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:00.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.553 7fa15fde4700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1580f0b10 con 0x7fa1581b0a10 2021-02-10T11:41:00.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.553 7fa15fde4700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1580e5ed0 con 0x7fa1580f0650 2021-02-10T11:41:00.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.553 7fa15fde4700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1580d1c70 con 0x7fa1580d1870 2021-02-10T11:41:00.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.553 7fa15e401700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa1580f0650 0x7fa1580cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:00.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.553 7fa15e401700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa1580f0650 0x7fa1580cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56944/0 (socket says 172.21.2.101:56944) 2021-02-10T11:41:00.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.553 7fa15d3ff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa1581b0a10 0x7fa1580cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57440/0 (socket says 172.21.2.101:57440) 2021-02-10T11:41:00.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.553 7fa15e401700 1 -- 172.21.2.101:0/2052837002 learned_addr learned my addr 172.21.2.101:0/2052837002 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:00.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.553 7fa15e401700 1 -- 172.21.2.101:0/2052837002 >> v2:172.21.2.108:3300/0 conn(0x7fa1580d1870 msgr2=0x7fa1581b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:41:00.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.553 7fa15e401700 1 --2- 172.21.2.101:0/2052837002 >> v2:172.21.2.108:3300/0 conn(0x7fa1580d1870 0x7fa1581b03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:00.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.553 7fa15e401700 1 -- 172.21.2.101:0/2052837002 >> v1:172.21.2.101:6789/0 conn(0x7fa1581b0a10 legacy=0x7fa1580cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:00.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.553 7fa15e401700 1 -- 172.21.2.101:0/2052837002 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1580d1e90 con 0x7fa1580f0650 2021-02-10T11:41:00.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.553 7fa15e401700 1 --2- 172.21.2.101:0/2052837002 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1580f0650 0x7fa1580cfb20 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 rx=0x7fa150009b50 tx=0x7fa150009ef0).ready entity=mon.0 client_cookie=8b86dda8d86be884 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:00.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.553 7fa15cbfe700 1 -- 172.21.2.101:0/2052837002 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa150000f90 con 0x7fa1580f0650 2021-02-10T11:41:00.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.553 7fa15cbfe700 1 -- 172.21.2.101:0/2052837002 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15000b3b0 con 0x7fa1580f0650 2021-02-10T11:41:00.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.553 7fa15cbfe700 1 -- 172.21.2.101:0/2052837002 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa15000b7c0 con 0x7fa1580f0650 2021-02-10T11:41:00.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.553 7fa15fde4700 1 -- 172.21.2.101:0/2052837002 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1580f0650 msgr2=0x7fa1580cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:00.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.553 7fa15fde4700 1 --2- 172.21.2.101:0/2052837002 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1580f0650 0x7fa1580cfb20 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 rx=0x7fa150009b50 tx=0x7fa150009ef0).stop 2021-02-10T11:41:00.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.554 7fa15fde4700 1 -- 172.21.2.101:0/2052837002 shutdown_connections 2021-02-10T11:41:00.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.554 7fa15fde4700 1 --2- 172.21.2.101:0/2052837002 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1580f0650 0x7fa1580cfb20 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:00.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.554 7fa15fde4700 1 --2- 172.21.2.101:0/2052837002 >> v2:172.21.2.108:3300/0 conn(0x7fa1580d1870 0x7fa1581b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:00.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.554 7fa15fde4700 1 -- 172.21.2.101:0/2052837002 >> 172.21.2.101:0/2052837002 conn(0x7fa1580ce450 msgr2=0x7fa1580d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:00.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.554 7fa15fde4700 1 -- 172.21.2.101:0/2052837002 shutdown_connections 2021-02-10T11:41:00.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.554 7fa15fde4700 1 -- 172.21.2.101:0/2052837002 wait complete. 2021-02-10T11:41:00.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.569 7fa15fde4700 1 Processor -- start 2021-02-10T11:41:00.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.569 7fa15fde4700 1 -- start start 2021-02-10T11:41:00.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.569 7fa15fde4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1581b7cf0 0x7fa15819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:00.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.569 7fa15fde4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1581b80f0 0x7fa1581b84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:00.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.569 7fa15fde4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa15819da20 0x7fa15819de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:00.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.569 7fa15fde4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1580a77c0 con 0x7fa15819da20 2021-02-10T11:41:00.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.569 7fa15fde4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15819d4a0 con 0x7fa1581b80f0 2021-02-10T11:41:00.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.569 7fa15fde4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1580954f0 con 0x7fa1581b7cf0 2021-02-10T11:41:00.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.569 7fa15e401700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa15819da20 0x7fa15819de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:00.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.569 7fa15e401700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa15819da20 0x7fa15819de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56950/0 (socket says 172.21.2.101:56950) 2021-02-10T11:41:00.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.569 7fa15e401700 1 -- 172.21.2.101:0/746218842 learned_addr learned my addr 172.21.2.101:0/746218842 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:00.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.570 7fa15e401700 1 -- 172.21.2.101:0/746218842 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1581b7cf0 msgr2=0x7fa15819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:00.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.570 7fa15e401700 1 --2- 172.21.2.101:0/746218842 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1581b7cf0 0x7fa15819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:00.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.570 7fa15e401700 1 -- 172.21.2.101:0/746218842 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1581b80f0 msgr2=0x7fa1581b84f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:00.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.570 7fa15e401700 1 --2- 172.21.2.101:0/746218842 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1581b80f0 0x7fa1581b84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:00.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.570 7fa15e401700 1 -- 172.21.2.101:0/746218842 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1581b8d60 con 0x7fa15819da20 2021-02-10T11:41:00.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.570 7fa15e401700 1 --2- 172.21.2.101:0/746218842 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa15819da20 0x7fa15819de20 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 rx=0x7fa1500010f0 tx=0x7fa15000bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:00.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.570 7fa14a7fc700 1 -- 172.21.2.101:0/746218842 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa150003080 con 0x7fa15819da20 2021-02-10T11:41:00.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.570 7fa14a7fc700 1 -- 172.21.2.101:0/746218842 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa150009360 con 0x7fa15819da20 2021-02-10T11:41:00.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.570 7fa14a7fc700 1 -- 172.21.2.101:0/746218842 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa150001670 con 0x7fa15819da20 2021-02-10T11:41:00.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.570 7fa15fde4700 1 -- 172.21.2.101:0/746218842 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1581b8d60 con 0x7fa15819da20 2021-02-10T11:41:00.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.570 7fa15fde4700 1 -- 172.21.2.101:0/746218842 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1581b8d60 con 0x7fa15819da20 2021-02-10T11:41:00.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.571 7fa14a7fc700 1 -- 172.21.2.101:0/746218842 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa150001270 con 0x7fa15819da20 2021-02-10T11:41:00.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.571 7fa14a7fc700 1 --2- 172.21.2.101:0/746218842 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1400186d0 0x7fa14001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:00.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.571 7fa14a7fc700 1 -- 172.21.2.101:0/746218842 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa1500329d0 con 0x7fa15819da20 2021-02-10T11:41:00.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.571 7fa15dc00700 1 --2- 172.21.2.101:0/746218842 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1400186d0 0x7fa14001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:00.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.571 7fa15dc00700 1 --2- 172.21.2.101:0/746218842 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1400186d0 0x7fa14001abd0 crc :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:00.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.602 7fa15fde4700 1 -- 172.21.2.101:0/746218842 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1580ebc00 con 0x7fa15819da20 2021-02-10T11:41:00.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.605 7fa14a7fc700 1 -- 172.21.2.101:0/746218842 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa150002ef0 con 0x7fa15819da20 2021-02-10T11:41:00.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.730 7fa15fde4700 1 -- 172.21.2.101:0/746218842 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1580f0b10 con 0x7fa15819da20 2021-02-10T11:41:00.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.730 7fa14a7fc700 1 -- 172.21.2.101:0/746218842 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+14007 (secure 0 0 0) 0x7fa150001270 con 0x7fa15819da20 2021-02-10T11:41:00.733 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:41:00.733 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":112,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:40:46.730463","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:40:43.698421","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2021-02-10 11:40:44.923352","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:41:00.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.732 7fa15fde4700 1 -- 172.21.2.101:0/746218842 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1400186d0 msgr2=0x7fa14001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:00.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.732 7fa15fde4700 1 --2- 172.21.2.101:0/746218842 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1400186d0 0x7fa14001abd0 crc :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:00.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.732 7fa15fde4700 1 -- 172.21.2.101:0/746218842 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa15819da20 msgr2=0x7fa15819de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:00.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.732 7fa15fde4700 1 --2- 172.21.2.101:0/746218842 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa15819da20 0x7fa15819de20 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 rx=0x7fa1500010f0 tx=0x7fa15000bd10).stop 2021-02-10T11:41:00.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.732 7fa15fde4700 1 -- 172.21.2.101:0/746218842 shutdown_connections 2021-02-10T11:41:00.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.732 7fa15fde4700 1 --2- 172.21.2.101:0/746218842 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa15819da20 0x7fa15819de20 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:00.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.732 7fa15fde4700 1 --2- 172.21.2.101:0/746218842 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1581b80f0 0x7fa1581b84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:00.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.732 7fa15fde4700 1 --2- 172.21.2.101:0/746218842 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1400186d0 0x7fa14001abd0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:00.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.732 7fa15fde4700 1 --2- 172.21.2.101:0/746218842 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1581b7cf0 0x7fa15819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:00.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.732 7fa15fde4700 1 -- 172.21.2.101:0/746218842 >> 172.21.2.101:0/746218842 conn(0x7fa1580ed810 msgr2=0x7fa158094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:00.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.732 7fa15fde4700 1 -- 172.21.2.101:0/746218842 shutdown_connections 2021-02-10T11:41:00.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:00.732 7fa15fde4700 1 -- 172.21.2.101:0/746218842 wait complete. 2021-02-10T11:41:00.743 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:40:43.698421', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:40:44.923352', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:41:00.744 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:41:01.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.155 7f471de1a700 1 Processor -- start 2021-02-10T11:41:01.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.155 7f471de1a700 1 -- start start 2021-02-10T11:41:01.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.156 7f471de1a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f47180d1c60 0x7f47180cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:01.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.156 7f471de1a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47180cf570 0x7f47180cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:01.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.156 7f471de1a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47180fe1b0 con 0x7f47180d1860 2021-02-10T11:41:01.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.156 7f471de1a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47181b0ca0 con 0x7f47180d1c60 2021-02-10T11:41:01.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.156 7f471de1a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4718109a50 con 0x7f47180cf570 2021-02-10T11:41:01.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.156 7f4716ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f47180d1c60 0x7f47180cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:01.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.156 7f4716ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f47180d1c60 0x7f47180cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56956/0 (socket says 172.21.2.101:56956) 2021-02-10T11:41:01.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.156 7f4716ffd700 1 -- 172.21.2.101:0/2012193563 learned_addr learned my addr 172.21.2.101:0/2012193563 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:01.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.156 7f4717fff700 1 --2- 172.21.2.101:0/2012193563 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47180cf570 0x7f47180cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:01.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.156 7f4716ffd700 1 -- 172.21.2.101:0/2012193563 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47180cf570 msgr2=0x7f47180cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:01.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.156 7f4716ffd700 1 --2- 172.21.2.101:0/2012193563 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47180cf570 0x7f47180cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:01.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.156 7f4716ffd700 1 -- 172.21.2.101:0/2012193563 >> v1:172.21.2.108:6789/0 conn(0x7f47180d1860 legacy=0x7f47181b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:01.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.156 7f4716ffd700 1 -- 172.21.2.101:0/2012193563 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4718088070 con 0x7f47180d1c60 2021-02-10T11:41:01.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.156 7f4716ffd700 1 --2- 172.21.2.101:0/2012193563 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47180d1c60 0x7f47180cef50 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 rx=0x7f4708006700 tx=0x7f4708009b80).ready entity=mon.0 client_cookie=b273e005e12497f9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:01.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.156 7f47167fc700 1 -- 172.21.2.101:0/2012193563 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f470800c0c0 con 0x7f47180d1c60 2021-02-10T11:41:01.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.157 7f47167fc700 1 -- 172.21.2.101:0/2012193563 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470800c440 con 0x7f47180d1c60 2021-02-10T11:41:01.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.157 7f47167fc700 1 -- 172.21.2.101:0/2012193563 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f470800c8a0 con 0x7f47180d1c60 2021-02-10T11:41:01.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.157 7f471de1a700 1 -- 172.21.2.101:0/2012193563 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47180d1c60 msgr2=0x7f47180cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:01.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.157 7f471de1a700 1 --2- 172.21.2.101:0/2012193563 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47180d1c60 0x7f47180cef50 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 rx=0x7f4708006700 tx=0x7f4708009b80).stop 2021-02-10T11:41:01.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.157 7f471de1a700 1 -- 172.21.2.101:0/2012193563 shutdown_connections 2021-02-10T11:41:01.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.157 7f471de1a700 1 --2- 172.21.2.101:0/2012193563 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47180cf570 0x7f47180cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:01.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.157 7f471de1a700 1 --2- 172.21.2.101:0/2012193563 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47180d1c60 0x7f47180cef50 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:01.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.157 7f471de1a700 1 -- 172.21.2.101:0/2012193563 >> 172.21.2.101:0/2012193563 conn(0x7f47180ce410 msgr2=0x7f47180d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:01.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.157 7f471de1a700 1 -- 172.21.2.101:0/2012193563 shutdown_connections 2021-02-10T11:41:01.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.157 7f471de1a700 1 -- 172.21.2.101:0/2012193563 wait complete. 2021-02-10T11:41:01.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.172 7f471de1a700 1 Processor -- start 2021-02-10T11:41:01.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.172 7f471de1a700 1 -- start start 2021-02-10T11:41:01.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.172 7f471de1a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47180adf50 0x7f47180ae350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:01.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.172 7f471de1a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47180ae970 0x7f47180aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:01.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.172 7f471de1a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f47180982c0 0x7f47180986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:01.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.172 7f471de1a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471809f450 con 0x7f47180ae970 2021-02-10T11:41:01.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.172 7f471de1a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4718098ce0 con 0x7f47180982c0 2021-02-10T11:41:01.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.172 7f471de1a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4718098ed0 con 0x7f47180adf50 2021-02-10T11:41:01.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.172 7f4716ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47180ae970 0x7f47180aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:01.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.173 7f4717fff700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f47180982c0 0x7f47180986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:01.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.173 7f4716ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47180ae970 0x7f47180aed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56966/0 (socket says 172.21.2.101:56966) 2021-02-10T11:41:01.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.173 7f4716ffd700 1 -- 172.21.2.101:0/2052468152 learned_addr learned my addr 172.21.2.101:0/2052468152 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:01.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.173 7f47177fe700 1 --2- 172.21.2.101:0/2052468152 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47180adf50 0x7f47180ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:01.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.173 7f4716ffd700 1 -- 172.21.2.101:0/2052468152 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47180adf50 msgr2=0x7f47180ae350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:01.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.173 7f4716ffd700 1 --2- 172.21.2.101:0/2052468152 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47180adf50 0x7f47180ae350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:01.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.173 7f4716ffd700 1 -- 172.21.2.101:0/2052468152 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f47180982c0 msgr2=0x7f47180986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:01.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.173 7f4716ffd700 1 --2- 172.21.2.101:0/2052468152 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f47180982c0 0x7f47180986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:01.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.173 7f4716ffd700 1 -- 172.21.2.101:0/2052468152 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47180990f0 con 0x7f47180ae970 2021-02-10T11:41:01.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.173 7f4716ffd700 1 --2- 172.21.2.101:0/2052468152 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47180ae970 0x7f47180aed70 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 rx=0x7f470800b5e0 tx=0x7f470800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:01.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.173 7f470ffff700 1 -- 172.21.2.101:0/2052468152 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f470800bd70 con 0x7f47180ae970 2021-02-10T11:41:01.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.173 7f470ffff700 1 -- 172.21.2.101:0/2052468152 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4708002030 con 0x7f47180ae970 2021-02-10T11:41:01.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.173 7f470ffff700 1 -- 172.21.2.101:0/2052468152 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4708002550 con 0x7f47180ae970 2021-02-10T11:41:01.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.173 7f471de1a700 1 -- 172.21.2.101:0/2052468152 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47180990f0 con 0x7f47180ae970 2021-02-10T11:41:01.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.173 7f471de1a700 1 -- 172.21.2.101:0/2052468152 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4718099680 con 0x7f47180ae970 2021-02-10T11:41:01.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.174 7f470ffff700 1 -- 172.21.2.101:0/2052468152 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4708000c90 con 0x7f47180ae970 2021-02-10T11:41:01.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.174 7f470ffff700 1 --2- 172.21.2.101:0/2052468152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f46fc0186d0 0x7f46fc01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:01.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.174 7f470ffff700 1 -- 172.21.2.101:0/2052468152 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4708032810 con 0x7f47180ae970 2021-02-10T11:41:01.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.174 7f47177fe700 1 --2- 172.21.2.101:0/2052468152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f46fc0186d0 0x7f46fc01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:01.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.174 7f47177fe700 1 --2- 172.21.2.101:0/2052468152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f46fc0186d0 0x7f46fc01abd0 crc :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:01.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.205 7f471de1a700 1 -- 172.21.2.101:0/2052468152 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4718082a20 con 0x7f47180ae970 2021-02-10T11:41:01.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.208 7f470ffff700 1 -- 172.21.2.101:0/2052468152 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4708000940 con 0x7f47180ae970 2021-02-10T11:41:01.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.338 7f471de1a700 1 -- 172.21.2.101:0/2052468152 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f4718082a20 con 0x7f47180ae970 2021-02-10T11:41:01.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.338 7f470ffff700 1 -- 172.21.2.101:0/2052468152 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v112) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4708000c90 con 0x7f47180ae970 2021-02-10T11:41:01.341 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:41:01.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.340 7f471de1a700 1 -- 172.21.2.101:0/2052468152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f46fc0186d0 msgr2=0x7f46fc01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:01.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.340 7f471de1a700 1 --2- 172.21.2.101:0/2052468152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f46fc0186d0 0x7f46fc01abd0 crc :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:01.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.340 7f471de1a700 1 -- 172.21.2.101:0/2052468152 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47180ae970 msgr2=0x7f47180aed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:01.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.340 7f471de1a700 1 --2- 172.21.2.101:0/2052468152 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47180ae970 0x7f47180aed70 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 rx=0x7f470800b5e0 tx=0x7f470800b980).stop 2021-02-10T11:41:01.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.340 7f471de1a700 1 -- 172.21.2.101:0/2052468152 shutdown_connections 2021-02-10T11:41:01.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.340 7f471de1a700 1 --2- 172.21.2.101:0/2052468152 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f47180982c0 0x7f47180986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:01.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.340 7f471de1a700 1 --2- 172.21.2.101:0/2052468152 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47180ae970 0x7f47180aed70 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:01.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.340 7f471de1a700 1 --2- 172.21.2.101:0/2052468152 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f46fc0186d0 0x7f46fc01abd0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:01.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.340 7f471de1a700 1 --2- 172.21.2.101:0/2052468152 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47180adf50 0x7f47180ae350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:01.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.340 7f471de1a700 1 -- 172.21.2.101:0/2052468152 >> 172.21.2.101:0/2052468152 conn(0x7f47180bee90 msgr2=0x7f47180bb880 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:01.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.340 7f471de1a700 1 -- 172.21.2.101:0/2052468152 shutdown_connections 2021-02-10T11:41:01.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.340 7f471de1a700 1 -- 172.21.2.101:0/2052468152 wait complete. 2021-02-10T11:41:01.351 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:41:01.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.764 7fbdb8e73700 1 Processor -- start 2021-02-10T11:41:01.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.764 7fbdb8e73700 1 -- start start 2021-02-10T11:41:01.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.765 7fbdb8e73700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbdb40d1870 0x7fbdb41b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:01.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.765 7fbdb8e73700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbdb41b0ae0 0x7fbdb4108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:01.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.765 7fbdb8e73700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbdb40d1c70 0x7fbdb4108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:01.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.765 7fbdb8e73700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb40fe1c0 con 0x7fbdb40d1870 2021-02-10T11:41:01.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.765 7fbdb8e73700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb4109a60 con 0x7fbdb41b0ae0 2021-02-10T11:41:01.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.765 7fbdb8e73700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb4088070 con 0x7fbdb40d1c70 2021-02-10T11:41:01.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.765 7fbdb2b13700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbdb40d1870 0x7fbdb41b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:01.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.765 7fbdb2b13700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbdb40d1870 0x7fbdb41b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56970/0 (socket says 172.21.2.101:56970) 2021-02-10T11:41:01.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.765 7fbdb2b13700 1 -- 172.21.2.101:0/2562453624 learned_addr learned my addr 172.21.2.101:0/2562453624 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:01.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.765 7fbdb3314700 1 --2- 172.21.2.101:0/2562453624 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbdb40d1c70 0x7fbdb4108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:01.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.765 7fbdb2312700 1 --2- 172.21.2.101:0/2562453624 >> v2:172.21.2.108:3300/0 conn(0x7fbdb41b0ae0 0x7fbdb4108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:01.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.765 7fbdb2b13700 1 -- 172.21.2.101:0/2562453624 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbdb40d1c70 msgr2=0x7fbdb4108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:01.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.765 7fbdb2b13700 1 --2- 172.21.2.101:0/2562453624 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbdb40d1c70 0x7fbdb4108f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:01.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.766 7fbdb2b13700 1 -- 172.21.2.101:0/2562453624 >> v2:172.21.2.108:3300/0 conn(0x7fbdb41b0ae0 msgr2=0x7fbdb4108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:01.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.766 7fbdb2b13700 1 --2- 172.21.2.101:0/2562453624 >> v2:172.21.2.108:3300/0 conn(0x7fbdb41b0ae0 0x7fbdb4108940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:01.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.766 7fbdb2b13700 1 -- 172.21.2.101:0/2562453624 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb4109580 con 0x7fbdb40d1870 2021-02-10T11:41:01.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.766 7fbdb3314700 1 --2- 172.21.2.101:0/2562453624 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbdb40d1c70 0x7fbdb4108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:41:01.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.766 7fbdb2312700 1 --2- 172.21.2.101:0/2562453624 >> v2:172.21.2.108:3300/0 conn(0x7fbdb41b0ae0 0x7fbdb4108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:41:01.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.766 7fbdb2b13700 1 --2- 172.21.2.101:0/2562453624 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbdb40d1870 0x7fbdb41b04c0 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 rx=0x7fbdac006a80 tx=0x7fbdac009f00).ready entity=mon.0 client_cookie=fa79fe66d47611d6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:01.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.766 7fbdb1b11700 1 -- 172.21.2.101:0/2562453624 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbdac0066c0 con 0x7fbdb40d1870 2021-02-10T11:41:01.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.766 7fbdb1b11700 1 -- 172.21.2.101:0/2562453624 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdac00b5c0 con 0x7fbdb40d1870 2021-02-10T11:41:01.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.766 7fbdb1b11700 1 -- 172.21.2.101:0/2562453624 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbdac00ba10 con 0x7fbdb40d1870 2021-02-10T11:41:01.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.766 7fbdb8e73700 1 -- 172.21.2.101:0/2562453624 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbdb40d1870 msgr2=0x7fbdb41b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:01.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.766 7fbdb8e73700 1 --2- 172.21.2.101:0/2562453624 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbdb40d1870 0x7fbdb41b04c0 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 rx=0x7fbdac006a80 tx=0x7fbdac009f00).stop 2021-02-10T11:41:01.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.766 7fbdb8e73700 1 -- 172.21.2.101:0/2562453624 shutdown_connections 2021-02-10T11:41:01.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.766 7fbdb8e73700 1 --2- 172.21.2.101:0/2562453624 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbdb40d1c70 0x7fbdb4108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:01.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.766 7fbdb8e73700 1 --2- 172.21.2.101:0/2562453624 >> v2:172.21.2.108:3300/0 conn(0x7fbdb41b0ae0 0x7fbdb4108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:01.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.766 7fbdb8e73700 1 --2- 172.21.2.101:0/2562453624 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbdb40d1870 0x7fbdb41b04c0 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:01.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.767 7fbdb8e73700 1 -- 172.21.2.101:0/2562453624 >> 172.21.2.101:0/2562453624 conn(0x7fbdb40ce420 msgr2=0x7fbdb40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:01.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.767 7fbdb8e73700 1 -- 172.21.2.101:0/2562453624 shutdown_connections 2021-02-10T11:41:01.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.767 7fbdb8e73700 1 -- 172.21.2.101:0/2562453624 wait complete. 2021-02-10T11:41:01.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.781 7fbdb8e73700 1 Processor -- start 2021-02-10T11:41:01.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.781 7fbdb8e73700 1 -- start start 2021-02-10T11:41:01.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.782 7fbdb8e73700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbdb4085bc0 0x7fbdb40863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:01.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.782 7fbdb8e73700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbdb40954c0 0x7fbdb4096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:01.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.782 7fbdb8e73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbdb40958c0 0x7fbdb4096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:01.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.782 7fbdb8e73700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb4095d80 con 0x7fbdb40958c0 2021-02-10T11:41:01.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.782 7fbdb8e73700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb40bfa60 con 0x7fbdb40954c0 2021-02-10T11:41:01.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.782 7fbdb8e73700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb4085fc0 con 0x7fbdb4085bc0 2021-02-10T11:41:01.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.782 7fbdb3314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbdb40958c0 0x7fbdb4096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:01.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.782 7fbdb3314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbdb40958c0 0x7fbdb4096920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56976/0 (socket says 172.21.2.101:56976) 2021-02-10T11:41:01.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.782 7fbdb3314700 1 -- 172.21.2.101:0/1848321963 learned_addr learned my addr 172.21.2.101:0/1848321963 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:01.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.782 7fbdb2312700 1 --2- 172.21.2.101:0/1848321963 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbdb40954c0 0x7fbdb4096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:01.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.782 7fbdb2b13700 1 --2- 172.21.2.101:0/1848321963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbdb4085bc0 0x7fbdb40863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:01.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.782 7fbdb3314700 1 -- 172.21.2.101:0/1848321963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbdb4085bc0 msgr2=0x7fbdb40863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:01.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.782 7fbdb3314700 1 --2- 172.21.2.101:0/1848321963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbdb4085bc0 0x7fbdb40863e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:01.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.782 7fbdb3314700 1 -- 172.21.2.101:0/1848321963 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbdb40954c0 msgr2=0x7fbdb4096300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:01.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.782 7fbdb3314700 1 --2- 172.21.2.101:0/1848321963 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbdb40954c0 0x7fbdb4096300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:01.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.782 7fbdb3314700 1 -- 172.21.2.101:0/1848321963 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb4086c10 con 0x7fbdb40958c0 2021-02-10T11:41:01.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.782 7fbdb2312700 1 --2- 172.21.2.101:0/1848321963 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbdb40954c0 0x7fbdb4096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:41:01.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.782 7fbdb3314700 1 --2- 172.21.2.101:0/1848321963 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbdb40958c0 0x7fbdb4096920 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 rx=0x7fbda4006f10 tx=0x7fbda400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:01.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.783 7fbda37fe700 1 -- 172.21.2.101:0/1848321963 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbda400bd80 con 0x7fbdb40958c0 2021-02-10T11:41:01.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.783 7fbda37fe700 1 -- 172.21.2.101:0/1848321963 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda4006410 con 0x7fbdb40958c0 2021-02-10T11:41:01.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.783 7fbda37fe700 1 -- 172.21.2.101:0/1848321963 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbda40022b0 con 0x7fbdb40958c0 2021-02-10T11:41:01.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.783 7fbdb8e73700 1 -- 172.21.2.101:0/1848321963 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb4086c10 con 0x7fbdb40958c0 2021-02-10T11:41:01.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.783 7fbdb8e73700 1 -- 172.21.2.101:0/1848321963 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb4097230 con 0x7fbdb40958c0 2021-02-10T11:41:01.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.783 7fbda37fe700 1 -- 172.21.2.101:0/1848321963 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbda400a030 con 0x7fbdb40958c0 2021-02-10T11:41:01.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.783 7fbda37fe700 1 --2- 172.21.2.101:0/1848321963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd980186c0 0x7fbd9801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:01.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.783 7fbda37fe700 1 -- 172.21.2.101:0/1848321963 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fbda4032590 con 0x7fbdb40958c0 2021-02-10T11:41:01.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.784 7fbdb2b13700 1 --2- 172.21.2.101:0/1848321963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd980186c0 0x7fbd9801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:01.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.784 7fbdb2b13700 1 --2- 172.21.2.101:0/1848321963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd980186c0 0x7fbd9801abc0 crc :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:01.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.815 7fbdb8e73700 1 -- 172.21.2.101:0/1848321963 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb40fe1c0 con 0x7fbdb40958c0 2021-02-10T11:41:01.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.817 7fbda37fe700 1 -- 172.21.2.101:0/1848321963 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbda4000bc0 con 0x7fbdb40958c0 2021-02-10T11:41:01.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.946 7fbdb8e73700 1 -- 172.21.2.101:0/1848321963 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7fbdb40fe1c0 con 0x7fbdb40958c0 2021-02-10T11:41:01.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.946 7fbda37fe700 1 -- 172.21.2.101:0/1848321963 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v112) v1 ==== 104+0+10 (secure 0 0 0) 0x7fbda4000bc0 con 0x7fbdb40958c0 2021-02-10T11:41:01.949 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:41:01.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.948 7fbdb8e73700 1 -- 172.21.2.101:0/1848321963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd980186c0 msgr2=0x7fbd9801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:01.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.948 7fbdb8e73700 1 --2- 172.21.2.101:0/1848321963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd980186c0 0x7fbd9801abc0 crc :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:01.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.948 7fbdb8e73700 1 -- 172.21.2.101:0/1848321963 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbdb40958c0 msgr2=0x7fbdb4096920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:01.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.948 7fbdb8e73700 1 --2- 172.21.2.101:0/1848321963 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbdb40958c0 0x7fbdb4096920 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 rx=0x7fbda4006f10 tx=0x7fbda400b990).stop 2021-02-10T11:41:01.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.948 7fbdb8e73700 1 -- 172.21.2.101:0/1848321963 shutdown_connections 2021-02-10T11:41:01.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.948 7fbdb8e73700 1 --2- 172.21.2.101:0/1848321963 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbdb40958c0 0x7fbdb4096920 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:01.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.948 7fbdb8e73700 1 --2- 172.21.2.101:0/1848321963 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbdb40954c0 0x7fbdb4096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:01.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.948 7fbdb8e73700 1 --2- 172.21.2.101:0/1848321963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd980186c0 0x7fbd9801abc0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:01.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.948 7fbdb8e73700 1 --2- 172.21.2.101:0/1848321963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbdb4085bc0 0x7fbdb40863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:01.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.948 7fbdb8e73700 1 -- 172.21.2.101:0/1848321963 >> 172.21.2.101:0/1848321963 conn(0x7fbdb40bef00 msgr2=0x7fbdb409d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:01.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.948 7fbdb8e73700 1 -- 172.21.2.101:0/1848321963 shutdown_connections 2021-02-10T11:41:01.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:01.948 7fbdb8e73700 1 -- 172.21.2.101:0/1848321963 wait complete. 2021-02-10T11:41:01.959 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:41:02.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.371 7f2c63b61700 1 Processor -- start 2021-02-10T11:41:02.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.371 7f2c63b61700 1 -- start start 2021-02-10T11:41:02.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.372 7f2c63b61700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c5c0d1870 0x7f2c5c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:02.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.372 7f2c63b61700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2c5c0cff70 0x7f2c5c109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:02.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.372 7f2c63b61700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c5c109a60 con 0x7f2c5c1b0ae0 2021-02-10T11:41:02.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.372 7f2c63b61700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c5c109750 con 0x7f2c5c0cff70 2021-02-10T11:41:02.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.372 7f2c63b61700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c088070 con 0x7f2c5c0d1870 2021-02-10T11:41:02.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.372 7f2c6217e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2c5c0cff70 0x7f2c5c109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:02.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.372 7f2c6217e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2c5c0cff70 0x7f2c5c109130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56984/0 (socket says 172.21.2.101:56984) 2021-02-10T11:41:02.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.372 7f2c6217e700 1 -- 172.21.2.101:0/2267045697 learned_addr learned my addr 172.21.2.101:0/2267045697 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:02.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.372 7f2c6217e700 1 -- 172.21.2.101:0/2267045697 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c5c0d1870 msgr2=0x7f2c5c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:02.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.372 7f2c6217e700 1 --2- 172.21.2.101:0/2267045697 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c5c0d1870 0x7f2c5c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:02.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.372 7f2c6217e700 1 -- 172.21.2.101:0/2267045697 >> v1:172.21.2.108:6789/0 conn(0x7f2c5c1b0ae0 legacy=0x7f2c5c108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:02.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.372 7f2c6217e700 1 -- 172.21.2.101:0/2267045697 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c088260 con 0x7f2c5c0cff70 2021-02-10T11:41:02.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.373 7f2c6217e700 1 --2- 172.21.2.101:0/2267045697 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c5c0cff70 0x7f2c5c109130 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 rx=0x7f2c54006f00 tx=0x7f2c54009f10).ready entity=mon.0 client_cookie=a7346857a68da87e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:02.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.373 7f2c6097b700 1 -- 172.21.2.101:0/2267045697 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2c5400b3b0 con 0x7f2c5c0cff70 2021-02-10T11:41:02.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.373 7f2c6097b700 1 -- 172.21.2.101:0/2267045697 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5400b630 con 0x7f2c5c0cff70 2021-02-10T11:41:02.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.373 7f2c6097b700 1 -- 172.21.2.101:0/2267045697 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2c5400bb50 con 0x7f2c5c0cff70 2021-02-10T11:41:02.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.373 7f2c63b61700 1 -- 172.21.2.101:0/2267045697 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c5c0cff70 msgr2=0x7f2c5c109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:02.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.373 7f2c63b61700 1 --2- 172.21.2.101:0/2267045697 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c5c0cff70 0x7f2c5c109130 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 rx=0x7f2c54006f00 tx=0x7f2c54009f10).stop 2021-02-10T11:41:02.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.374 7f2c63b61700 1 -- 172.21.2.101:0/2267045697 shutdown_connections 2021-02-10T11:41:02.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.374 7f2c63b61700 1 --2- 172.21.2.101:0/2267045697 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c5c0cff70 0x7f2c5c109130 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:02.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.374 7f2c63b61700 1 --2- 172.21.2.101:0/2267045697 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c5c0d1870 0x7f2c5c1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:02.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.374 7f2c63b61700 1 -- 172.21.2.101:0/2267045697 >> 172.21.2.101:0/2267045697 conn(0x7f2c5c0ce420 msgr2=0x7f2c5c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.374 7f2c63b61700 1 -- 172.21.2.101:0/2267045697 shutdown_connections 2021-02-10T11:41:02.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.374 7f2c63b61700 1 -- 172.21.2.101:0/2267045697 wait complete. 2021-02-10T11:41:02.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.388 7f2c63b61700 1 Processor -- start 2021-02-10T11:41:02.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.389 7f2c63b61700 1 -- start start 2021-02-10T11:41:02.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.389 7f2c63b61700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c5c0c3820 0x7f2c5c084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:02.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.389 7f2c63b61700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2c5c0a0290 0x7f2c5c0852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:02.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.389 7f2c63b61700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c5c086270 0x7f2c5c086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:02.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.389 7f2c63b61700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c0c0700 con 0x7f2c5c0c3820 2021-02-10T11:41:02.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.389 7f2c63b61700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c0858f0 con 0x7f2c5c0a0290 2021-02-10T11:41:02.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.389 7f2c63b61700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c085b10 con 0x7f2c5c086270 2021-02-10T11:41:02.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.389 7f2c6197d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c5c0c3820 0x7f2c5c084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:02.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.389 7f2c6197d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c5c0c3820 0x7f2c5c084cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:56988/0 (socket says 172.21.2.101:56988) 2021-02-10T11:41:02.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.389 7f2c6197d700 1 -- 172.21.2.101:0/2895806518 learned_addr learned my addr 172.21.2.101:0/2895806518 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:02.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.390 7f2c6197d700 1 -- 172.21.2.101:0/2895806518 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c5c086270 msgr2=0x7f2c5c086670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:02.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.390 7f2c6197d700 1 --2- 172.21.2.101:0/2895806518 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c5c086270 0x7f2c5c086670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:02.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.390 7f2c6197d700 1 -- 172.21.2.101:0/2895806518 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2c5c0a0290 msgr2=0x7f2c5c0852d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:02.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.390 7f2c6197d700 1 --2- 172.21.2.101:0/2895806518 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2c5c0a0290 0x7f2c5c0852d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:02.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.390 7f2c6197d700 1 -- 172.21.2.101:0/2895806518 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c086c90 con 0x7f2c5c0c3820 2021-02-10T11:41:02.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.390 7f2c6197d700 1 --2- 172.21.2.101:0/2895806518 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c5c0c3820 0x7f2c5c084cb0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 rx=0x7f2c50006f00 tx=0x7f2c5000b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:02.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.390 7f2c4e7fc700 1 -- 172.21.2.101:0/2895806518 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2c50006b90 con 0x7f2c5c0c3820 2021-02-10T11:41:02.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.390 7f2c4e7fc700 1 -- 172.21.2.101:0/2895806518 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5000bd30 con 0x7f2c5c0c3820 2021-02-10T11:41:02.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.390 7f2c4e7fc700 1 -- 172.21.2.101:0/2895806518 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2c500020d0 con 0x7f2c5c0c3820 2021-02-10T11:41:02.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.390 7f2c63b61700 1 -- 172.21.2.101:0/2895806518 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5c086c90 con 0x7f2c5c0c3820 2021-02-10T11:41:02.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.390 7f2c63b61700 1 -- 172.21.2.101:0/2895806518 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5c086c90 con 0x7f2c5c0c3820 2021-02-10T11:41:02.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.391 7f2c4e7fc700 1 -- 172.21.2.101:0/2895806518 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2c50007000 con 0x7f2c5c0c3820 2021-02-10T11:41:02.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.391 7f2c4e7fc700 1 --2- 172.21.2.101:0/2895806518 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2c440186d0 0x7f2c4401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:02.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.391 7f2c4e7fc700 1 -- 172.21.2.101:0/2895806518 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2c50032790 con 0x7f2c5c0c3820 2021-02-10T11:41:02.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.391 7f2c6117c700 1 --2- 172.21.2.101:0/2895806518 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2c440186d0 0x7f2c4401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:02.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.391 7f2c6117c700 1 --2- 172.21.2.101:0/2895806518 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2c440186d0 0x7f2c4401abd0 crc :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:02.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.422 7f2c63b61700 1 -- 172.21.2.101:0/2895806518 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5c082a70 con 0x7f2c5c0c3820 2021-02-10T11:41:02.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.424 7f2c4e7fc700 1 -- 172.21.2.101:0/2895806518 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2c50006350 con 0x7f2c5c0c3820 2021-02-10T11:41:02.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.550 7f2c63b61700 1 -- 172.21.2.101:0/2895806518 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f2c5c082a70 con 0x7f2c5c0c3820 2021-02-10T11:41:02.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.551 7f2c4e7fc700 1 -- 172.21.2.101:0/2895806518 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v112) v1 ==== 100+0+10 (secure 0 0 0) 0x7f2c5000bd30 con 0x7f2c5c0c3820 2021-02-10T11:41:02.554 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:41:02.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.553 7f2c63b61700 1 -- 172.21.2.101:0/2895806518 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2c440186d0 msgr2=0x7f2c4401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:02.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.553 7f2c63b61700 1 --2- 172.21.2.101:0/2895806518 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2c440186d0 0x7f2c4401abd0 crc :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:02.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.553 7f2c63b61700 1 -- 172.21.2.101:0/2895806518 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c5c0c3820 msgr2=0x7f2c5c084cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:02.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.553 7f2c63b61700 1 --2- 172.21.2.101:0/2895806518 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c5c0c3820 0x7f2c5c084cb0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 rx=0x7f2c50006f00 tx=0x7f2c5000b940).stop 2021-02-10T11:41:02.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.553 7f2c63b61700 1 -- 172.21.2.101:0/2895806518 shutdown_connections 2021-02-10T11:41:02.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.553 7f2c63b61700 1 --2- 172.21.2.101:0/2895806518 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2c440186d0 0x7f2c4401abd0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:02.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.553 7f2c63b61700 1 --2- 172.21.2.101:0/2895806518 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c5c086270 0x7f2c5c086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:02.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.553 7f2c63b61700 1 --2- 172.21.2.101:0/2895806518 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2c5c0a0290 0x7f2c5c0852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:02.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.553 7f2c63b61700 1 --2- 172.21.2.101:0/2895806518 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c5c0c3820 0x7f2c5c084cb0 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:02.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.553 7f2c63b61700 1 -- 172.21.2.101:0/2895806518 >> 172.21.2.101:0/2895806518 conn(0x7f2c5c0872c0 msgr2=0x7f2c5c095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:02.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.553 7f2c63b61700 1 -- 172.21.2.101:0/2895806518 shutdown_connections 2021-02-10T11:41:02.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.553 7f2c63b61700 1 -- 172.21.2.101:0/2895806518 wait complete. 2021-02-10T11:41:02.564 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:41:02.564 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:41:02.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.975 7fc8fbf92700 1 Processor -- start 2021-02-10T11:41:02.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.975 7fc8fbf92700 1 -- start start 2021-02-10T11:41:02.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.976 7fc8fbf92700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc8f40d0340 0x7fc8f40ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:02.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.976 7fc8fbf92700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc8f40d1c70 0x7fc8f40efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:02.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.976 7fc8fbf92700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f40f0b10 con 0x7fc8f40d1870 2021-02-10T11:41:02.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.976 7fc8fbf92700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f40e5ed0 con 0x7fc8f40d0340 2021-02-10T11:41:02.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.976 7fc8fbf92700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f41b0be0 con 0x7fc8f40d1c70 2021-02-10T11:41:02.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.976 7fc8f9dae700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc8f40d1870 0x7fc8f41b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57490/0 (socket says 172.21.2.101:57490) 2021-02-10T11:41:02.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.976 7fc8f9dae700 1 -- 172.21.2.101:0/3521091992 learned_addr learned my addr 172.21.2.101:0/3521091992 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:02.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.976 7fc8f95ad700 1 --2- 172.21.2.101:0/3521091992 >> v2:172.21.2.101:3300/0 conn(0x7fc8f40d0340 0x7fc8f40ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:02.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.976 7fc8fa5af700 1 --2- 172.21.2.101:0/3521091992 >> v2:172.21.2.108:3300/0 conn(0x7fc8f40d1c70 0x7fc8f40efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:02.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.977 7fc8fa5af700 1 -- 172.21.2.101:0/3521091992 >> v2:172.21.2.101:3300/0 conn(0x7fc8f40d0340 msgr2=0x7fc8f40ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:02.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.977 7fc8fa5af700 1 --2- 172.21.2.101:0/3521091992 >> v2:172.21.2.101:3300/0 conn(0x7fc8f40d0340 0x7fc8f40ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:02.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.977 7fc8fa5af700 1 -- 172.21.2.101:0/3521091992 >> v1:172.21.2.101:6789/0 conn(0x7fc8f40d1870 legacy=0x7fc8f41b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:02.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.977 7fc8fa5af700 1 -- 172.21.2.101:0/3521091992 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f40f20a0 con 0x7fc8f40d1c70 2021-02-10T11:41:02.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.977 7fc8f8dac700 1 -- 172.21.2.101:0/3521091992 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997955753 0 0) 0x7fc8e8000ce0 con 0x7fc8f40d1870 2021-02-10T11:41:02.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.977 7fc8fa5af700 1 --2- 172.21.2.101:0/3521091992 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8f40d1c70 0x7fc8f40efc60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fc8ec006e50 tx=0x7fc8ec009e70).ready entity=mon.1 client_cookie=b9e7290b06d54a0f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:02.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.977 7fc8f8dac700 1 -- 172.21.2.101:0/3521091992 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc8ec00b510 con 0x7fc8f40d1c70 2021-02-10T11:41:02.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.977 7fc8f8dac700 1 -- 172.21.2.101:0/3521091992 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8ec006790 con 0x7fc8f40d1c70 2021-02-10T11:41:02.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.977 7fc8f8dac700 1 -- 172.21.2.101:0/3521091992 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc8ec00b940 con 0x7fc8f40d1c70 2021-02-10T11:41:02.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.977 7fc8fbf92700 1 -- 172.21.2.101:0/3521091992 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8f40d1c70 msgr2=0x7fc8f40efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:02.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.977 7fc8fbf92700 1 --2- 172.21.2.101:0/3521091992 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8f40d1c70 0x7fc8f40efc60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fc8ec006e50 tx=0x7fc8ec009e70).stop 2021-02-10T11:41:02.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.978 7fc8fbf92700 1 -- 172.21.2.101:0/3521091992 shutdown_connections 2021-02-10T11:41:02.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.978 7fc8fbf92700 1 --2- 172.21.2.101:0/3521091992 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8f40d1c70 0x7fc8f40efc60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:02.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.978 7fc8fbf92700 1 --2- 172.21.2.101:0/3521091992 >> v2:172.21.2.101:3300/0 conn(0x7fc8f40d0340 0x7fc8f40ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:02.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.978 7fc8fbf92700 1 -- 172.21.2.101:0/3521091992 >> 172.21.2.101:0/3521091992 conn(0x7fc8f40ce450 msgr2=0x7fc8f40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:02.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.978 7fc8fbf92700 1 -- 172.21.2.101:0/3521091992 shutdown_connections 2021-02-10T11:41:02.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.978 7fc8fbf92700 1 -- 172.21.2.101:0/3521091992 wait complete. 2021-02-10T11:41:02.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.993 7fc8fbf92700 1 Processor -- start 2021-02-10T11:41:02.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.993 7fc8fbf92700 1 -- start start 2021-02-10T11:41:03.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.994 7fc8fbf92700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8f40ada90 0x7fc8f40ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:03.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.994 7fc8fbf92700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8f4099d20 0x7fc8f409a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:03.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.994 7fc8fbf92700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8f409a740 0x7fc8f4097bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:03.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.994 7fc8fbf92700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f40997a0 con 0x7fc8f4099d20 2021-02-10T11:41:03.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.994 7fc8fbf92700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f418f0c0 con 0x7fc8f40ada90 2021-02-10T11:41:03.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.994 7fc8fbf92700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f409ab40 con 0x7fc8f409a740 2021-02-10T11:41:03.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.994 7fc8f95ad700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8f4099d20 0x7fc8f409a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:03.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.994 7fc8f95ad700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8f4099d20 0x7fc8f409a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57006/0 (socket says 172.21.2.101:57006) 2021-02-10T11:41:03.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.994 7fc8f95ad700 1 -- 172.21.2.101:0/1106033213 learned_addr learned my addr 172.21.2.101:0/1106033213 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:03.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.994 7fc8f9dae700 1 --2- 172.21.2.101:0/1106033213 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8f40ada90 0x7fc8f40ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:03.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.994 7fc8fa5af700 1 --2- 172.21.2.101:0/1106033213 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8f409a740 0x7fc8f4097bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:03.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.994 7fc8f95ad700 1 -- 172.21.2.101:0/1106033213 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8f409a740 msgr2=0x7fc8f4097bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:03.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.994 7fc8f95ad700 1 --2- 172.21.2.101:0/1106033213 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8f409a740 0x7fc8f4097bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:03.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.994 7fc8f95ad700 1 -- 172.21.2.101:0/1106033213 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8f40ada90 msgr2=0x7fc8f40ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:03.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.994 7fc8f95ad700 1 --2- 172.21.2.101:0/1106033213 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8f40ada90 0x7fc8f40ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:03.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.994 7fc8f95ad700 1 -- 172.21.2.101:0/1106033213 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f409ad60 con 0x7fc8f4099d20 2021-02-10T11:41:03.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.994 7fc8fa5af700 1 --2- 172.21.2.101:0/1106033213 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8f409a740 0x7fc8f4097bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:41:03.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.994 7fc8f95ad700 1 --2- 172.21.2.101:0/1106033213 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8f4099d20 0x7fc8f409a120 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 rx=0x7fc8f0006f60 tx=0x7fc8f000b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:03.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.995 7fc8e67fc700 1 -- 172.21.2.101:0/1106033213 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc8f000bd60 con 0x7fc8f4099d20 2021-02-10T11:41:03.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.995 7fc8e67fc700 1 -- 172.21.2.101:0/1106033213 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f0000c90 con 0x7fc8f4099d20 2021-02-10T11:41:03.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.995 7fc8fbf92700 1 -- 172.21.2.101:0/1106033213 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f409ad60 con 0x7fc8f4099d20 2021-02-10T11:41:03.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.995 7fc8e67fc700 1 -- 172.21.2.101:0/1106033213 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc8f00022b0 con 0x7fc8f4099d20 2021-02-10T11:41:03.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.995 7fc8fbf92700 1 -- 172.21.2.101:0/1106033213 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f409ad60 con 0x7fc8f4099d20 2021-02-10T11:41:03.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.995 7fc8e67fc700 1 -- 172.21.2.101:0/1106033213 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc8f00052e0 con 0x7fc8f4099d20 2021-02-10T11:41:03.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.995 7fc8e67fc700 1 --2- 172.21.2.101:0/1106033213 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc8dc018730 0x7fc8dc01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:03.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.995 7fc8e67fc700 1 -- 172.21.2.101:0/1106033213 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc8f00327e0 con 0x7fc8f4099d20 2021-02-10T11:41:03.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.996 7fc8f9dae700 1 --2- 172.21.2.101:0/1106033213 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc8dc018730 0x7fc8dc01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:03.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:02.996 7fc8f9dae700 1 --2- 172.21.2.101:0/1106033213 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc8dc018730 0x7fc8dc01ac10 crc :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:03.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:03.026 7fc8fbf92700 1 -- 172.21.2.101:0/1106033213 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f40ebc00 con 0x7fc8f4099d20 2021-02-10T11:41:03.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:03.028 7fc8e67fc700 1 -- 172.21.2.101:0/1106033213 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc8f0000c90 con 0x7fc8f4099d20 2021-02-10T11:41:03.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:03.160 7fc8fbf92700 1 -- 172.21.2.101:0/1106033213 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8f40f0b10 con 0x7fc8f4099d20 2021-02-10T11:41:03.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:03.160 7fc8e67fc700 1 -- 172.21.2.101:0/1106033213 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+2104 (secure 0 0 0) 0x7fc8f000bd60 con 0x7fc8f4099d20 2021-02-10T11:41:03.164 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:41:03.164 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":88,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2840404451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2840404451},{"type":"v1","addr":"172.21.2.101:6835","nonce":2840404451}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":84}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:40:45.728009","modified":"2021-02-10 11:40:47.738972","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7450},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7450":{"gid":7450,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2757440614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2757440614},{"type":"v1","addr":"172.21.2.108:6833","nonce":2757440614}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2021-02-10T11:41:03.164 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 88 2021-02-10T11:41:03.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:03.162 7fc8fbf92700 1 -- 172.21.2.101:0/1106033213 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc8dc018730 msgr2=0x7fc8dc01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:03.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:03.162 7fc8fbf92700 1 --2- 172.21.2.101:0/1106033213 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc8dc018730 0x7fc8dc01ac10 crc :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:03.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:03.162 7fc8fbf92700 1 -- 172.21.2.101:0/1106033213 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8f4099d20 msgr2=0x7fc8f409a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:03.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:03.162 7fc8fbf92700 1 --2- 172.21.2.101:0/1106033213 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8f4099d20 0x7fc8f409a120 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 rx=0x7fc8f0006f60 tx=0x7fc8f000b970).stop 2021-02-10T11:41:03.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:03.163 7fc8fbf92700 1 -- 172.21.2.101:0/1106033213 shutdown_connections 2021-02-10T11:41:03.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:03.163 7fc8fbf92700 1 --2- 172.21.2.101:0/1106033213 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc8dc018730 0x7fc8dc01ac10 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:03.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:03.163 7fc8fbf92700 1 --2- 172.21.2.101:0/1106033213 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8f409a740 0x7fc8f4097bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:03.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:03.163 7fc8fbf92700 1 --2- 172.21.2.101:0/1106033213 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8f4099d20 0x7fc8f409a120 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:03.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:03.163 7fc8fbf92700 1 --2- 172.21.2.101:0/1106033213 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8f40ada90 0x7fc8f40ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:03.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:03.163 7fc8fbf92700 1 -- 172.21.2.101:0/1106033213 >> 172.21.2.101:0/1106033213 conn(0x7fc8f40f0550 msgr2=0x7fc8f4191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:03.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:03.163 7fc8fbf92700 1 -- 172.21.2.101:0/1106033213 shutdown_connections 2021-02-10T11:41:03.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:03.163 7fc8fbf92700 1 -- 172.21.2.101:0/1106033213 wait complete. 2021-02-10T11:41:03.375 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:40:45.728009', 'modified': '2021-02-10 11:40:47.738972', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7450}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7450': {'gid': 7450, 'name': 'b', 'rank': 0, 'incarnation': 85, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/2757440614', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 2757440614}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 2757440614}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:41:03.376 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:41:03.376 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:41:03.782 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:41:03.783 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:41:03.783 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:41:03.783 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 7450, 2021-02-10T11:41:03.783 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:41:03.783 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:41:03.783 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 18.048355020999999, 2021-02-10T11:41:03.783 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 88, 2021-02-10T11:41:03.783 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 112, 2021-02-10T11:41:03.783 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 112, 2021-02-10T11:41:03.783 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 23.687801659000002 2021-02-10T11:41:03.783 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:41:03.787 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 7450, "want_state": "up:active", "state": "up:active", "rank_uptime": 18.048355020999999, "mdsmap_epoch": 88, "osdmap_epoch": 112, "osdmap_epoch_barrier": 112, "uptime": 23.687801659000002 } 2021-02-10T11:41:03.787 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:41:03.787 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-02-10T11:41:03.787 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:41:03.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:03.787 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:41:04.124 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:41:04.323 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:41:04.326 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:41:04.467 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:41:04.468 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:41:04.811 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:41:04.812 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:41:04.812 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:41:04.813 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:04.813 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:41:05.148 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:41:05.148 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2021-02-10T11:41:05.149 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:41:05.151 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:41:05.495 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:41:05.495 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:41:05.561 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[147867]: starting ceph client 2021-02-10T11:41:05.561 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:2021-02-10 11:41:05.558 7f93cb59ef80 -1 init, newargv = 0x563bf31271a0 newargc=9 2021-02-10T11:41:05.566 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[147867]: starting fuse 2021-02-10T11:41:05.839 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:41:05.839 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:41:05.840 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:41:05.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:05.841 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:41:06.176 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:41:06.176 INFO:teuthology.orchestra.run.gibba001.stdout:44 2021-02-10T11:41:06.176 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2021-02-10T11:41:06.176 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:06.176 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:41:06.176 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:41:06.177 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:41:06.177 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:41:06.177 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:41:06.177 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:41:06.177 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:41:06.177 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-02-10T11:41:06.177 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:41:06.177 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:41:06.177 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:41:06.177 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:41:06.177 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:41:06.177 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:41:06.177 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:41:06.177 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:41:06.177 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:41:06.178 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:41:06.178 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:41:06.178 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:41:06.178 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:41:06.178 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.1")) 2021-02-10T11:41:06.178 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:41:06.540 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.1.147867.asok 2021-02-10T11:41:06.543 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.147867.asok 2021-02-10T11:41:06.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:06.543 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.147867.asok status 2021-02-10T11:41:06.934 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:41:06.934 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:41:06.934 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:41:06.935 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:41:06.935 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "1", 2021-02-10T11:41:06.935 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:41:06.935 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:41:06.935 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "147867", 2021-02-10T11:41:06.935 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:41:06.935 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:06.935 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:41:06.935 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:41:06.935 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 6533, 2021-02-10T11:41:06.935 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:41:06.935 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:41:06.936 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:41:06.936 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 6533 2021-02-10T11:41:06.936 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:06.936 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:41:06.936 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:06.936 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:41:06.936 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 2433469685 2021-02-10T11:41:06.936 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:06.936 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:06.936 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:41:06.936 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:06.936 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:41:06.937 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 2433469685 2021-02-10T11:41:06.937 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:06.937 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.6533 172.21.2.101:0/2433469685", 2021-02-10T11:41:06.937 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/2433469685", 2021-02-10T11:41:06.937 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:41:06.937 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 88, 2021-02-10T11:41:06.937 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 112, 2021-02-10T11:41:06.937 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:41:06.937 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:41:06.937 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:41:06.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:06.939 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:41:06.939 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:41:07.273 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:41:07.273 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:41:07.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:07.274 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:41:07.619 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2021-02-10T11:41:08.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.032 7fe6ab62f700 1 Processor -- start 2021-02-10T11:41:08.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.032 7fe6ab62f700 1 -- start start 2021-02-10T11:41:08.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.033 7fe6ab62f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe6a40d1870 0x7fe6a41b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:08.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.033 7fe6ab62f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe6a40cff70 0x7fe6a4109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:08.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.033 7fe6ab62f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a40fe1c0 con 0x7fe6a41b0ae0 2021-02-10T11:41:08.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.033 7fe6ab62f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a4109ad0 con 0x7fe6a40cff70 2021-02-10T11:41:08.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.033 7fe6ab62f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a4109750 con 0x7fe6a40d1870 2021-02-10T11:41:08.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.033 7fe6a8c4a700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe6a41b0ae0 0x7fe6a4108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57516/0 (socket says 172.21.2.101:57516) 2021-02-10T11:41:08.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.033 7fe6a8c4a700 1 -- 172.21.2.101:0/20972430 learned_addr learned my addr 172.21.2.101:0/20972430 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:08.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.033 7fe6a9c4c700 1 --2- 172.21.2.101:0/20972430 >> v2:172.21.2.101:3300/0 conn(0x7fe6a40cff70 0x7fe6a4109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:08.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.033 7fe6a944b700 1 --2- 172.21.2.101:0/20972430 >> v2:172.21.2.108:3300/0 conn(0x7fe6a40d1870 0x7fe6a41b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:08.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.033 7fe697fff700 1 -- 172.21.2.101:0/20972430 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3950637594 0 0) 0x7fe6980010b0 con 0x7fe6a41b0ae0 2021-02-10T11:41:08.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.033 7fe697fff700 1 -- 172.21.2.101:0/20972430 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe690002fe0 con 0x7fe6a41b0ae0 2021-02-10T11:41:08.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.033 7fe6a9c4c700 1 -- 172.21.2.101:0/20972430 >> v2:172.21.2.108:3300/0 conn(0x7fe6a40d1870 msgr2=0x7fe6a41b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:08.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.033 7fe6a9c4c700 1 --2- 172.21.2.101:0/20972430 >> v2:172.21.2.108:3300/0 conn(0x7fe6a40d1870 0x7fe6a41b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:08.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.033 7fe6a9c4c700 1 -- 172.21.2.101:0/20972430 >> v1:172.21.2.101:6789/0 conn(0x7fe6a41b0ae0 legacy=0x7fe6a4108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:08.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.033 7fe6a9c4c700 1 -- 172.21.2.101:0/20972430 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a4088070 con 0x7fe6a40cff70 2021-02-10T11:41:08.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.034 7fe6a9c4c700 1 --2- 172.21.2.101:0/20972430 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6a40cff70 0x7fe6a4109130 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 rx=0x7fe69c006e30 tx=0x7fe69c009e60).ready entity=mon.0 client_cookie=27e0542951e9fb0d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:08.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.034 7fe697fff700 1 -- 172.21.2.101:0/20972430 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe69c006a50 con 0x7fe6a40cff70 2021-02-10T11:41:08.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.034 7fe697fff700 1 -- 172.21.2.101:0/20972430 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c00b300 con 0x7fe6a40cff70 2021-02-10T11:41:08.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.034 7fe697fff700 1 -- 172.21.2.101:0/20972430 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe69c00b820 con 0x7fe6a40cff70 2021-02-10T11:41:08.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.034 7fe6ab62f700 1 -- 172.21.2.101:0/20972430 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6a40cff70 msgr2=0x7fe6a4109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:08.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.034 7fe6ab62f700 1 --2- 172.21.2.101:0/20972430 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6a40cff70 0x7fe6a4109130 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 rx=0x7fe69c006e30 tx=0x7fe69c009e60).stop 2021-02-10T11:41:08.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.034 7fe6ab62f700 1 -- 172.21.2.101:0/20972430 shutdown_connections 2021-02-10T11:41:08.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.034 7fe6ab62f700 1 --2- 172.21.2.101:0/20972430 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6a40cff70 0x7fe6a4109130 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:08.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.034 7fe6ab62f700 1 --2- 172.21.2.101:0/20972430 >> v2:172.21.2.108:3300/0 conn(0x7fe6a40d1870 0x7fe6a41b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:08.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.034 7fe6ab62f700 1 -- 172.21.2.101:0/20972430 >> 172.21.2.101:0/20972430 conn(0x7fe6a40ce420 msgr2=0x7fe6a40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:08.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.035 7fe6ab62f700 1 -- 172.21.2.101:0/20972430 shutdown_connections 2021-02-10T11:41:08.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.035 7fe6ab62f700 1 -- 172.21.2.101:0/20972430 wait complete. 2021-02-10T11:41:08.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.049 7fe6ab62f700 1 Processor -- start 2021-02-10T11:41:08.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.049 7fe6ab62f700 1 -- start start 2021-02-10T11:41:08.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.049 7fe6ab62f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6a408cde0 0x7fe6a408d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:08.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.049 7fe6ab62f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6a408d800 0x7fe6a408dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:08.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.049 7fe6ab62f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6a40b4d30 0x7fe6a40b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:08.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.049 7fe6ab62f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a408c8e0 con 0x7fe6a408cde0 2021-02-10T11:41:08.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.050 7fe6ab62f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40b6740 con 0x7fe6a408d800 2021-02-10T11:41:08.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.050 7fe6ab62f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40b5750 con 0x7fe6a40b4d30 2021-02-10T11:41:08.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.050 7fe6a944b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6a408cde0 0x7fe6a408d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:08.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.050 7fe6a944b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6a408cde0 0x7fe6a408d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57028/0 (socket says 172.21.2.101:57028) 2021-02-10T11:41:08.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.050 7fe6a8c4a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6a408d800 0x7fe6a408dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:08.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.050 7fe6a944b700 1 -- 172.21.2.101:0/1990534563 learned_addr learned my addr 172.21.2.101:0/1990534563 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:08.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.050 7fe6a9c4c700 1 --2- 172.21.2.101:0/1990534563 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6a40b4d30 0x7fe6a40b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:08.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.050 7fe6a944b700 1 -- 172.21.2.101:0/1990534563 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6a40b4d30 msgr2=0x7fe6a40b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:08.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.050 7fe6a944b700 1 --2- 172.21.2.101:0/1990534563 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6a40b4d30 0x7fe6a40b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:08.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.050 7fe6a944b700 1 -- 172.21.2.101:0/1990534563 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6a408d800 msgr2=0x7fe6a408dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:08.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.050 7fe6a944b700 1 --2- 172.21.2.101:0/1990534563 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6a408d800 0x7fe6a408dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:08.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.050 7fe6a944b700 1 -- 172.21.2.101:0/1990534563 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a40b5970 con 0x7fe6a408cde0 2021-02-10T11:41:08.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.050 7fe6a8c4a700 1 --2- 172.21.2.101:0/1990534563 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6a408d800 0x7fe6a408dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:41:08.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.050 7fe6a944b700 1 --2- 172.21.2.101:0/1990534563 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6a408cde0 0x7fe6a408d1e0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 rx=0x7fe6a0006f60 tx=0x7fe6a000b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:08.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.050 7fe695ffb700 1 -- 172.21.2.101:0/1990534563 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe6a000bd60 con 0x7fe6a408cde0 2021-02-10T11:41:08.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.050 7fe695ffb700 1 -- 172.21.2.101:0/1990534563 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a0000c90 con 0x7fe6a408cde0 2021-02-10T11:41:08.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.050 7fe695ffb700 1 -- 172.21.2.101:0/1990534563 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe6a0002220 con 0x7fe6a408cde0 2021-02-10T11:41:08.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.050 7fe6ab62f700 1 -- 172.21.2.101:0/1990534563 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a40b5970 con 0x7fe6a408cde0 2021-02-10T11:41:08.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.050 7fe6ab62f700 1 -- 172.21.2.101:0/1990534563 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a40b5970 con 0x7fe6a408cde0 2021-02-10T11:41:08.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.051 7fe695ffb700 1 -- 172.21.2.101:0/1990534563 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe6a00052e0 con 0x7fe6a408cde0 2021-02-10T11:41:08.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.051 7fe695ffb700 1 --2- 172.21.2.101:0/1990534563 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6840186d0 0x7fe68401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:08.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.051 7fe695ffb700 1 -- 172.21.2.101:0/1990534563 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe6a00327d0 con 0x7fe6a408cde0 2021-02-10T11:41:08.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.051 7fe6a8c4a700 1 --2- 172.21.2.101:0/1990534563 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6840186d0 0x7fe68401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:08.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.052 7fe6a8c4a700 1 --2- 172.21.2.101:0/1990534563 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6840186d0 0x7fe68401abd0 crc :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:08.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.083 7fe6ab62f700 1 -- 172.21.2.101:0/1990534563 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a40b6740 con 0x7fe6a408cde0 2021-02-10T11:41:08.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.085 7fe695ffb700 1 -- 172.21.2.101:0/1990534563 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe6a0006bb0 con 0x7fe6a408cde0 2021-02-10T11:41:08.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.220 7fe6ab62f700 1 -- 172.21.2.101:0/1990534563 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe6a40b6740 con 0x7fe6a408cde0 2021-02-10T11:41:08.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.274 7fe695ffb700 1 -- 172.21.2.101:0/1990534563 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v89) v1 ==== 136+0+0 (secure 0 0 0) 0x7fe6a00052e0 con 0x7fe6a408cde0 2021-02-10T11:41:08.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.286 7fe6ab62f700 1 -- 172.21.2.101:0/1990534563 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6840186d0 msgr2=0x7fe68401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:08.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.286 7fe6ab62f700 1 --2- 172.21.2.101:0/1990534563 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6840186d0 0x7fe68401abd0 crc :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:08.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.286 7fe6ab62f700 1 -- 172.21.2.101:0/1990534563 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6a408cde0 msgr2=0x7fe6a408d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:08.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.286 7fe6ab62f700 1 --2- 172.21.2.101:0/1990534563 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6a408cde0 0x7fe6a408d1e0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 rx=0x7fe6a0006f60 tx=0x7fe6a000b970).stop 2021-02-10T11:41:08.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.286 7fe6ab62f700 1 -- 172.21.2.101:0/1990534563 shutdown_connections 2021-02-10T11:41:08.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.286 7fe6ab62f700 1 --2- 172.21.2.101:0/1990534563 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6840186d0 0x7fe68401abd0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:08.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.286 7fe6ab62f700 1 --2- 172.21.2.101:0/1990534563 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6a40b4d30 0x7fe6a40b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:08.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.286 7fe6ab62f700 1 --2- 172.21.2.101:0/1990534563 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6a408d800 0x7fe6a408dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:08.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.286 7fe6ab62f700 1 --2- 172.21.2.101:0/1990534563 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6a408cde0 0x7fe6a408d1e0 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:08.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.286 7fe6ab62f700 1 -- 172.21.2.101:0/1990534563 >> 172.21.2.101:0/1990534563 conn(0x7fe6a40872d0 msgr2=0x7fe6a40ad590 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:08.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.286 7fe6ab62f700 1 -- 172.21.2.101:0/1990534563 shutdown_connections 2021-02-10T11:41:08.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.286 7fe6ab62f700 1 -- 172.21.2.101:0/1990534563 wait complete. 2021-02-10T11:41:08.297 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs volume ls 2021-02-10T11:41:08.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.707 7f8edcb6f700 1 Processor -- start 2021-02-10T11:41:08.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.707 7f8edcb6f700 1 -- start start 2021-02-10T11:41:08.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.707 7f8edcb6f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8ed80d0340 0x7f8ed80ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:08.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.708 7f8edcb6f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed80f0b10 con 0x7f8ed80d1870 2021-02-10T11:41:08.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.708 7f8edcb6f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed81b0be0 con 0x7f8ed80d1c70 2021-02-10T11:41:08.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.708 7f8edcb6f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed80e5ed0 con 0x7f8ed80d0340 2021-02-10T11:41:08.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.708 7f8ed685e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8ed80d1870 0x7f8ed81b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57530/0 (socket says 172.21.2.101:57530) 2021-02-10T11:41:08.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.708 7f8ed685e700 1 -- 172.21.2.101:0/3459529774 learned_addr learned my addr 172.21.2.101:0/3459529774 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:08.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.708 7f8ed605d700 1 --2- 172.21.2.101:0/3459529774 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8ed80d0340 0x7f8ed80ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:08.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.708 7f8ed605d700 1 -- 172.21.2.101:0/3459529774 >> v1:172.21.2.108:6789/0 conn(0x7f8ed80d1c70 legacy=0x7f8ed80efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:08.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.708 7f8ed605d700 1 -- 172.21.2.101:0/3459529774 >> v1:172.21.2.101:6789/0 conn(0x7f8ed80d1870 legacy=0x7f8ed81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:08.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.708 7f8ed585c700 1 -- 172.21.2.101:0/3459529774 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282234260 0 0) 0x7f8ec0000ce0 con 0x7f8ed80d1870 2021-02-10T11:41:08.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.708 7f8ed605d700 1 -- 172.21.2.101:0/3459529774 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed80f0450 con 0x7f8ed80d0340 2021-02-10T11:41:08.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.708 7f8ed605d700 1 --2- 172.21.2.101:0/3459529774 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8ed80d0340 0x7f8ed80ef640 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f8ec8006650 tx=0x7f8ec8009a80).ready entity=mon.2 client_cookie=e5cebaccb75c6279 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:08.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.709 7f8ed585c700 1 -- 172.21.2.101:0/3459529774 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8ec80009b0 con 0x7f8ed80d0340 2021-02-10T11:41:08.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.709 7f8ed585c700 1 -- 172.21.2.101:0/3459529774 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec800c0c0 con 0x7f8ed80d0340 2021-02-10T11:41:08.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.709 7f8edcb6f700 1 -- 172.21.2.101:0/3459529774 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8ed80d0340 msgr2=0x7f8ed80ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:08.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.709 7f8edcb6f700 1 --2- 172.21.2.101:0/3459529774 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8ed80d0340 0x7f8ed80ef640 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f8ec8006650 tx=0x7f8ec8009a80).stop 2021-02-10T11:41:08.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.709 7f8ed585c700 1 -- 172.21.2.101:0/3459529774 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8ec80009b0 con 0x7f8ed80d0340 2021-02-10T11:41:08.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.709 7f8edcb6f700 1 -- 172.21.2.101:0/3459529774 shutdown_connections 2021-02-10T11:41:08.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.709 7f8edcb6f700 1 --2- 172.21.2.101:0/3459529774 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8ed80d0340 0x7f8ed80ef640 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:08.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.709 7f8edcb6f700 1 -- 172.21.2.101:0/3459529774 >> 172.21.2.101:0/3459529774 conn(0x7f8ed80ce450 msgr2=0x7f8ed80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:08.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.709 7f8edcb6f700 1 -- 172.21.2.101:0/3459529774 shutdown_connections 2021-02-10T11:41:08.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.709 7f8edcb6f700 1 -- 172.21.2.101:0/3459529774 wait complete. 2021-02-10T11:41:08.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.724 7f8edcb6f700 1 Processor -- start 2021-02-10T11:41:08.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.724 7f8edcb6f700 1 -- start start 2021-02-10T11:41:08.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.725 7f8edcb6f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8ed80ea610 0x7f8ed80aa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:08.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.725 7f8edcb6f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ed80a2f90 0x7f8ed80a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:08.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.725 7f8edcb6f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8ed80a3390 0x7f8ed8096710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:08.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.725 7f8edcb6f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed80a9340 con 0x7f8ed80a2f90 2021-02-10T11:41:08.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.725 7f8edcb6f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed80a9ee0 con 0x7f8ed80a3390 2021-02-10T11:41:08.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.725 7f8edcb6f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed80aa0c0 con 0x7f8ed80ea610 2021-02-10T11:41:08.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.725 7f8ed605d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ed80a2f90 0x7f8ed80a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:08.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.725 7f8ed605d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ed80a2f90 0x7f8ed80a98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57044/0 (socket says 172.21.2.101:57044) 2021-02-10T11:41:08.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.725 7f8ed605d700 1 -- 172.21.2.101:0/413392008 learned_addr learned my addr 172.21.2.101:0/413392008 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:08.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.725 7f8ed685e700 1 --2- 172.21.2.101:0/413392008 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8ed80ea610 0x7f8ed80aa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:08.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.725 7f8ed705f700 1 --2- 172.21.2.101:0/413392008 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8ed80a3390 0x7f8ed8096710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:08.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.725 7f8ed605d700 1 -- 172.21.2.101:0/413392008 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8ed80ea610 msgr2=0x7f8ed80aa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:08.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.725 7f8ed605d700 1 --2- 172.21.2.101:0/413392008 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8ed80ea610 0x7f8ed80aa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:08.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.725 7f8ed605d700 1 -- 172.21.2.101:0/413392008 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8ed80a3390 msgr2=0x7f8ed8096710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:08.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.725 7f8ed605d700 1 --2- 172.21.2.101:0/413392008 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8ed80a3390 0x7f8ed8096710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:08.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.725 7f8ed605d700 1 -- 172.21.2.101:0/413392008 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed8096d30 con 0x7f8ed80a2f90 2021-02-10T11:41:08.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.725 7f8ed605d700 1 --2- 172.21.2.101:0/413392008 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ed80a2f90 0x7f8ed80a98c0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 rx=0x7f8ec80066e0 tx=0x7f8ec800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:08.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.726 7f8ecf7fe700 1 -- 172.21.2.101:0/413392008 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8ec8006470 con 0x7f8ed80a2f90 2021-02-10T11:41:08.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.726 7f8ecf7fe700 1 -- 172.21.2.101:0/413392008 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec800bd80 con 0x7f8ed80a2f90 2021-02-10T11:41:08.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.726 7f8ecf7fe700 1 -- 172.21.2.101:0/413392008 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8ec80054d0 con 0x7f8ed80a2f90 2021-02-10T11:41:08.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.726 7f8edcb6f700 1 -- 172.21.2.101:0/413392008 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed8096d30 con 0x7f8ed80a2f90 2021-02-10T11:41:08.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.726 7f8edcb6f700 1 -- 172.21.2.101:0/413392008 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed80c19a0 con 0x7f8ed80a2f90 2021-02-10T11:41:08.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.726 7f8ecf7fe700 1 -- 172.21.2.101:0/413392008 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8ec800a000 con 0x7f8ed80a2f90 2021-02-10T11:41:08.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.726 7f8ecf7fe700 1 --2- 172.21.2.101:0/413392008 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ebc0186c0 0x7f8ebc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:08.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.727 7f8ecf7fe700 1 -- 172.21.2.101:0/413392008 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8ec8032400 con 0x7f8ed80a2f90 2021-02-10T11:41:08.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.727 7f8ed685e700 1 --2- 172.21.2.101:0/413392008 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ebc0186c0 0x7f8ebc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:08.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.727 7f8ed685e700 1 --2- 172.21.2.101:0/413392008 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ebc0186c0 0x7f8ebc01abc0 crc :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:08.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.757 7f8edcb6f700 1 -- 172.21.2.101:0/413392008 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed80c2230 con 0x7f8ed80a2f90 2021-02-10T11:41:08.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.760 7f8ecf7fe700 1 -- 172.21.2.101:0/413392008 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8ec80009b0 con 0x7f8ed80a2f90 2021-02-10T11:41:08.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.896 7f8edcb6f700 1 -- 172.21.2.101:0/413392008 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs volume ls", "target": ["mgr", ""]}) v1 -- 0x7f8ed80c19a0 con 0x7f8ebc0186c0 2021-02-10T11:41:08.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.897 7f8ecf7fe700 1 -- 172.21.2.101:0/413392008 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7f8ec0000ce0 con 0x7f8ebc0186c0 2021-02-10T11:41:08.899 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:41:08.900 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:08.900 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "cephfs" 2021-02-10T11:41:08.900 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:08.900 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:41:08.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.898 7f8edcb6f700 1 -- 172.21.2.101:0/413392008 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ebc0186c0 msgr2=0x7f8ebc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:08.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.898 7f8edcb6f700 1 --2- 172.21.2.101:0/413392008 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ebc0186c0 0x7f8ebc01abc0 crc :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:08.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.898 7f8edcb6f700 1 -- 172.21.2.101:0/413392008 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ed80a2f90 msgr2=0x7f8ed80a98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:08.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.898 7f8edcb6f700 1 --2- 172.21.2.101:0/413392008 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ed80a2f90 0x7f8ed80a98c0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 rx=0x7f8ec80066e0 tx=0x7f8ec800b990).stop 2021-02-10T11:41:08.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.898 7f8edcb6f700 1 -- 172.21.2.101:0/413392008 shutdown_connections 2021-02-10T11:41:08.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.898 7f8edcb6f700 1 --2- 172.21.2.101:0/413392008 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8ed80a3390 0x7f8ed8096710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:08.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.898 7f8edcb6f700 1 --2- 172.21.2.101:0/413392008 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ed80a2f90 0x7f8ed80a98c0 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:08.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.898 7f8edcb6f700 1 --2- 172.21.2.101:0/413392008 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ebc0186c0 0x7f8ebc01abc0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:08.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.898 7f8edcb6f700 1 --2- 172.21.2.101:0/413392008 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8ed80ea610 0x7f8ed80aa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:08.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.898 7f8edcb6f700 1 -- 172.21.2.101:0/413392008 >> 172.21.2.101:0/413392008 conn(0x7f8ed8090950 msgr2=0x7f8ed80c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:08.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.899 7f8edcb6f700 1 -- 172.21.2.101:0/413392008 shutdown_connections 2021-02-10T11:41:08.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:08.899 7f8edcb6f700 1 -- 172.21.2.101:0/413392008 wait complete. 2021-02-10T11:41:08.911 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:41:09.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.326 7f71adff6700 1 Processor -- start 2021-02-10T11:41:09.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.326 7f71adff6700 1 -- start start 2021-02-10T11:41:09.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.326 7f71adff6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f71a810aff0 0x7f71a80cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:09.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.326 7f71adff6700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a80fe1c0 con 0x7f71a80d1c70 2021-02-10T11:41:09.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.326 7f71adff6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a81b0cb0 con 0x7f71a80d1870 2021-02-10T11:41:09.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.326 7f71adff6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a8109a60 con 0x7f71a810aff0 2021-02-10T11:41:09.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.326 7f71a7fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f71a810aff0 0x7f71a80cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:09.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.326 7f71a7fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f71a810aff0 0x7f71a80cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57054/0 (socket says 172.21.2.101:57054) 2021-02-10T11:41:09.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.326 7f71a7fff700 1 -- 172.21.2.101:0/3262873167 learned_addr learned my addr 172.21.2.101:0/3262873167 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:09.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.327 7f71a7fff700 1 -- 172.21.2.101:0/3262873167 >> v1:172.21.2.108:6789/0 conn(0x7f71a80d1870 legacy=0x7f71a81b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:09.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.327 7f71a7fff700 1 -- 172.21.2.101:0/3262873167 >> v1:172.21.2.101:6789/0 conn(0x7f71a80d1c70 legacy=0x7f71a80cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:09.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.327 7f71a7fff700 1 -- 172.21.2.101:0/3262873167 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a810b3f0 con 0x7f71a810aff0 2021-02-10T11:41:09.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.327 7f71a7fff700 1 --2- 172.21.2.101:0/3262873167 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71a810aff0 0x7f71a80cf750 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 rx=0x7f71a0006f00 tx=0x7f71a0009f10).ready entity=mon.0 client_cookie=d547c6996164252 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:09.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.327 7f71a67fc700 1 -- 172.21.2.101:0/3262873167 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f71a000b3b0 con 0x7f71a810aff0 2021-02-10T11:41:09.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.327 7f71a67fc700 1 -- 172.21.2.101:0/3262873167 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a000b630 con 0x7f71a810aff0 2021-02-10T11:41:09.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.327 7f71a67fc700 1 -- 172.21.2.101:0/3262873167 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f71a000bb50 con 0x7f71a810aff0 2021-02-10T11:41:09.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.327 7f71adff6700 1 -- 172.21.2.101:0/3262873167 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71a810aff0 msgr2=0x7f71a80cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:09.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.327 7f71adff6700 1 --2- 172.21.2.101:0/3262873167 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71a810aff0 0x7f71a80cf750 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 rx=0x7f71a0006f00 tx=0x7f71a0009f10).stop 2021-02-10T11:41:09.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.328 7f71adff6700 1 -- 172.21.2.101:0/3262873167 shutdown_connections 2021-02-10T11:41:09.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.328 7f71adff6700 1 --2- 172.21.2.101:0/3262873167 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71a810aff0 0x7f71a80cf750 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:09.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.328 7f71adff6700 1 -- 172.21.2.101:0/3262873167 >> 172.21.2.101:0/3262873167 conn(0x7f71a80ce420 msgr2=0x7f71a80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:09.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.328 7f71adff6700 1 -- 172.21.2.101:0/3262873167 shutdown_connections 2021-02-10T11:41:09.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.328 7f71adff6700 1 -- 172.21.2.101:0/3262873167 wait complete. 2021-02-10T11:41:09.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.343 7f71adff6700 1 Processor -- start 2021-02-10T11:41:09.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.343 7f71adff6700 1 -- start start 2021-02-10T11:41:09.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.343 7f71adff6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f71a80c0740 0x7f71a81c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:09.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.343 7f71adff6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71a81c5ba0 0x7f71a80a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:09.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.343 7f71adff6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71a80b6e60 0x7f71a80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:09.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.343 7f71adff6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a80a6880 con 0x7f71a81c5ba0 2021-02-10T11:41:09.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.343 7f71adff6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a80a6aa0 con 0x7f71a80c0740 2021-02-10T11:41:09.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.343 7f71adff6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a80a6cc0 con 0x7f71a80b6e60 2021-02-10T11:41:09.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.343 7f71a6ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71a81c5ba0 0x7f71a80a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:09.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.343 7f71a6ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71a81c5ba0 0x7f71a80a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57060/0 (socket says 172.21.2.101:57060) 2021-02-10T11:41:09.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.343 7f71a6ffd700 1 -- 172.21.2.101:0/3409604629 learned_addr learned my addr 172.21.2.101:0/3409604629 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:09.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.344 7f71a7fff700 1 --2- 172.21.2.101:0/3409604629 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71a80b6e60 0x7f71a80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:09.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.344 7f71a77fe700 1 --2- 172.21.2.101:0/3409604629 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f71a80c0740 0x7f71a81c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:09.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.344 7f71a6ffd700 1 -- 172.21.2.101:0/3409604629 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71a80b6e60 msgr2=0x7f71a80a6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:09.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.344 7f71a6ffd700 1 --2- 172.21.2.101:0/3409604629 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71a80b6e60 0x7f71a80a6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:09.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.344 7f71a6ffd700 1 -- 172.21.2.101:0/3409604629 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f71a80c0740 msgr2=0x7f71a81c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:09.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.344 7f71a6ffd700 1 --2- 172.21.2.101:0/3409604629 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f71a80c0740 0x7f71a81c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:09.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.344 7f71a6ffd700 1 -- 172.21.2.101:0/3409604629 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a80a6ee0 con 0x7f71a81c5ba0 2021-02-10T11:41:09.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.344 7f71a77fe700 1 --2- 172.21.2.101:0/3409604629 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f71a80c0740 0x7f71a81c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:41:09.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.344 7f71a6ffd700 1 --2- 172.21.2.101:0/3409604629 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71a81c5ba0 0x7f71a80a5c40 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 rx=0x7f7198006ed0 tx=0x7f719800b8e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:09.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.344 7f719ffff700 1 -- 172.21.2.101:0/3409604629 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7198006240 con 0x7f71a81c5ba0 2021-02-10T11:41:09.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.344 7f719ffff700 1 -- 172.21.2.101:0/3409604629 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71980009b0 con 0x7f71a81c5ba0 2021-02-10T11:41:09.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.344 7f719ffff700 1 -- 172.21.2.101:0/3409604629 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7198005730 con 0x7f71a81c5ba0 2021-02-10T11:41:09.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.344 7f71adff6700 1 -- 172.21.2.101:0/3409604629 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a80a6ee0 con 0x7f71a81c5ba0 2021-02-10T11:41:09.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.344 7f71adff6700 1 -- 172.21.2.101:0/3409604629 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a80a6ee0 con 0x7f71a81c5ba0 2021-02-10T11:41:09.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.344 7f719ffff700 1 -- 172.21.2.101:0/3409604629 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7198006030 con 0x7f71a81c5ba0 2021-02-10T11:41:09.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.344 7f719ffff700 1 --2- 172.21.2.101:0/3409604629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f718c0186d0 0x7f718c01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:09.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.344 7f719ffff700 1 -- 172.21.2.101:0/3409604629 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f71980327b0 con 0x7f71a81c5ba0 2021-02-10T11:41:09.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.345 7f71a77fe700 1 --2- 172.21.2.101:0/3409604629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f718c0186d0 0x7f718c01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:09.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.345 7f71a77fe700 1 --2- 172.21.2.101:0/3409604629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f718c0186d0 0x7f718c01abd0 crc :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:09.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.376 7f71adff6700 1 -- 172.21.2.101:0/3409604629 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a80807d0 con 0x7f71a81c5ba0 2021-02-10T11:41:09.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.379 7f719ffff700 1 -- 172.21.2.101:0/3409604629 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7198006030 con 0x7f71a81c5ba0 2021-02-10T11:41:09.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.527 7f71adff6700 1 -- 172.21.2.101:0/3409604629 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f71a8088070 con 0x7f71a81c5ba0 2021-02-10T11:41:09.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.527 7f719ffff700 1 -- 172.21.2.101:0/3409604629 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7198000d00 con 0x7f71a81c5ba0 2021-02-10T11:41:09.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.529 7f71adff6700 1 -- 172.21.2.101:0/3409604629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f718c0186d0 msgr2=0x7f718c01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:09.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.529 7f71adff6700 1 --2- 172.21.2.101:0/3409604629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f718c0186d0 0x7f718c01abd0 crc :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:09.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.529 7f71adff6700 1 -- 172.21.2.101:0/3409604629 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71a81c5ba0 msgr2=0x7f71a80a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:09.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.529 7f71adff6700 1 --2- 172.21.2.101:0/3409604629 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71a81c5ba0 0x7f71a80a5c40 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 rx=0x7f7198006ed0 tx=0x7f719800b8e0).stop 2021-02-10T11:41:09.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.530 7f71adff6700 1 -- 172.21.2.101:0/3409604629 shutdown_connections 2021-02-10T11:41:09.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.530 7f71adff6700 1 --2- 172.21.2.101:0/3409604629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f718c0186d0 0x7f718c01abd0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:09.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.530 7f71adff6700 1 --2- 172.21.2.101:0/3409604629 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71a80b6e60 0x7f71a80a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:09.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.530 7f71adff6700 1 --2- 172.21.2.101:0/3409604629 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71a81c5ba0 0x7f71a80a5c40 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:09.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.530 7f71adff6700 1 --2- 172.21.2.101:0/3409604629 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f71a80c0740 0x7f71a81c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:09.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.530 7f71adff6700 1 -- 172.21.2.101:0/3409604629 >> 172.21.2.101:0/3409604629 conn(0x7f71a80c4120 msgr2=0x7f71a80ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:09.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.530 7f71adff6700 1 -- 172.21.2.101:0/3409604629 shutdown_connections 2021-02-10T11:41:09.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.530 7f71adff6700 1 -- 172.21.2.101:0/3409604629 wait complete. 2021-02-10T11:41:09.541 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolumegroup create cephfs group_0000000000831820 2021-02-10T11:41:09.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.953 7f95764dc700 1 Processor -- start 2021-02-10T11:41:09.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.953 7f95764dc700 1 -- start start 2021-02-10T11:41:09.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.954 7f95764dc700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f957010aff0 0x7f95700cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:09.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.954 7f95764dc700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95700fe1c0 con 0x7f95700d1870 2021-02-10T11:41:09.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.954 7f95764dc700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95701b0cb0 con 0x7f95700d1c70 2021-02-10T11:41:09.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.954 7f95764dc700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9570109a60 con 0x7f957010aff0 2021-02-10T11:41:09.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.954 7f956ffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f95700d1870 0x7f95701b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57558/0 (socket says 172.21.2.101:57558) 2021-02-10T11:41:09.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.954 7f956ffff700 1 -- 172.21.2.101:0/4019563353 learned_addr learned my addr 172.21.2.101:0/4019563353 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:09.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.954 7f9574af9700 1 --2- 172.21.2.101:0/4019563353 >> v2:172.21.2.108:3300/0 conn(0x7f957010aff0 0x7f95700cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:09.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.954 7f956effd700 1 -- 172.21.2.101:0/4019563353 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238848960 0 0) 0x7f9560000ce0 con 0x7f95700d1870 2021-02-10T11:41:09.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.954 7f956effd700 1 -- 172.21.2.101:0/4019563353 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9558002fe0 con 0x7f95700d1870 2021-02-10T11:41:09.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.954 7f9574af9700 1 -- 172.21.2.101:0/4019563353 >> v1:172.21.2.108:6789/0 conn(0x7f95700d1c70 legacy=0x7f95700cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:09.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.954 7f9574af9700 1 -- 172.21.2.101:0/4019563353 >> v1:172.21.2.101:6789/0 conn(0x7f95700d1870 legacy=0x7f95701b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:09.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.954 7f9574af9700 1 -- 172.21.2.101:0/4019563353 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957010b3f0 con 0x7f957010aff0 2021-02-10T11:41:09.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.954 7f9574af9700 1 --2- 172.21.2.101:0/4019563353 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f957010aff0 0x7f95700cf750 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f9568006e50 tx=0x7f9568009df0).ready entity=mon.1 client_cookie=1628f06ddade222e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:09.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.955 7f956effd700 1 -- 172.21.2.101:0/4019563353 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f956800b510 con 0x7f957010aff0 2021-02-10T11:41:09.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.955 7f956effd700 1 -- 172.21.2.101:0/4019563353 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9568003030 con 0x7f957010aff0 2021-02-10T11:41:09.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.955 7f956effd700 1 -- 172.21.2.101:0/4019563353 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f956800b7d0 con 0x7f957010aff0 2021-02-10T11:41:09.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.955 7f95764dc700 1 -- 172.21.2.101:0/4019563353 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f957010aff0 msgr2=0x7f95700cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:09.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.955 7f95764dc700 1 --2- 172.21.2.101:0/4019563353 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f957010aff0 0x7f95700cf750 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f9568006e50 tx=0x7f9568009df0).stop 2021-02-10T11:41:09.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.955 7f95764dc700 1 -- 172.21.2.101:0/4019563353 shutdown_connections 2021-02-10T11:41:09.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.955 7f95764dc700 1 --2- 172.21.2.101:0/4019563353 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f957010aff0 0x7f95700cf750 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:09.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.955 7f95764dc700 1 -- 172.21.2.101:0/4019563353 >> 172.21.2.101:0/4019563353 conn(0x7f95700ce420 msgr2=0x7f95700d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:09.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.955 7f95764dc700 1 -- 172.21.2.101:0/4019563353 shutdown_connections 2021-02-10T11:41:09.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.955 7f95764dc700 1 -- 172.21.2.101:0/4019563353 wait complete. 2021-02-10T11:41:09.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.969 7f95764dc700 1 Processor -- start 2021-02-10T11:41:09.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.970 7f95764dc700 1 -- start start 2021-02-10T11:41:09.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.970 7f95764dc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95700b99c0 0x7f95700b9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:09.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.970 7f95764dc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95700ba3e0 0x7f95700ba7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:09.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.970 7f95764dc700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9570085f20 0x7f9570086320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:09.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.970 7f95764dc700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95700b9440 con 0x7f95700ba3e0 2021-02-10T11:41:09.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.970 7f95764dc700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9570086940 con 0x7f9570085f20 2021-02-10T11:41:09.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.970 7f95764dc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9570086b20 con 0x7f95700b99c0 2021-02-10T11:41:09.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.970 7f956f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95700ba3e0 0x7f95700ba7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:09.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.970 7f956f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95700ba3e0 0x7f95700ba7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57070/0 (socket says 172.21.2.101:57070) 2021-02-10T11:41:09.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.970 7f956f7fe700 1 -- 172.21.2.101:0/2871675104 learned_addr learned my addr 172.21.2.101:0/2871675104 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:09.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.970 7f9574af9700 1 --2- 172.21.2.101:0/2871675104 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9570085f20 0x7f9570086320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:09.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.970 7f956ffff700 1 --2- 172.21.2.101:0/2871675104 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95700b99c0 0x7f95700b9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:09.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.970 7f956f7fe700 1 -- 172.21.2.101:0/2871675104 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95700b99c0 msgr2=0x7f95700b9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:09.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.970 7f956f7fe700 1 --2- 172.21.2.101:0/2871675104 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95700b99c0 0x7f95700b9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:09.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.970 7f956f7fe700 1 -- 172.21.2.101:0/2871675104 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9570085f20 msgr2=0x7f9570086320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:09.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.970 7f956f7fe700 1 --2- 172.21.2.101:0/2871675104 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9570085f20 0x7f9570086320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:09.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.971 7f956f7fe700 1 -- 172.21.2.101:0/2871675104 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9570086d40 con 0x7f95700ba3e0 2021-02-10T11:41:09.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.971 7f9574af9700 1 --2- 172.21.2.101:0/2871675104 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9570085f20 0x7f9570086320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:41:09.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.971 7f956ffff700 1 --2- 172.21.2.101:0/2871675104 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95700b99c0 0x7f95700b9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:41:09.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.971 7f956f7fe700 1 --2- 172.21.2.101:0/2871675104 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95700ba3e0 0x7f95700ba7e0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 rx=0x7f9564006ed0 tx=0x7f956400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:09.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.971 7f956cff9700 1 -- 172.21.2.101:0/2871675104 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9564006020 con 0x7f95700ba3e0 2021-02-10T11:41:09.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.971 7f956cff9700 1 -- 172.21.2.101:0/2871675104 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95640063a0 con 0x7f95700ba3e0 2021-02-10T11:41:09.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.971 7f95764dc700 1 -- 172.21.2.101:0/2871675104 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9570086d40 con 0x7f95700ba3e0 2021-02-10T11:41:09.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.971 7f956cff9700 1 -- 172.21.2.101:0/2871675104 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9564005730 con 0x7f95700ba3e0 2021-02-10T11:41:09.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.971 7f95764dc700 1 -- 172.21.2.101:0/2871675104 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95700872d0 con 0x7f95700ba3e0 2021-02-10T11:41:09.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.971 7f956cff9700 1 -- 172.21.2.101:0/2871675104 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9564006020 con 0x7f95700ba3e0 2021-02-10T11:41:09.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.971 7f956cff9700 1 --2- 172.21.2.101:0/2871675104 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95500186d0 0x7f955001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:09.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.972 7f956cff9700 1 -- 172.21.2.101:0/2871675104 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9564032620 con 0x7f95700ba3e0 2021-02-10T11:41:09.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.972 7f956ffff700 1 --2- 172.21.2.101:0/2871675104 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95500186d0 0x7f955001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:09.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:09.972 7f956ffff700 1 --2- 172.21.2.101:0/2871675104 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95500186d0 0x7f955001abd0 crc :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:10.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.003 7f95764dc700 1 -- 172.21.2.101:0/2871675104 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f957019a080 con 0x7f95700ba3e0 2021-02-10T11:41:10.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.006 7f956cff9700 1 -- 172.21.2.101:0/2871675104 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9564006380 con 0x7f95700ba3e0 2021-02-10T11:41:10.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.133 7f95764dc700 1 -- 172.21.2.101:0/2871675104 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "target": ["mgr", ""], "group_name": "group_0000000000831820"}) v1 -- 0x7f95700b9440 con 0x7f95500186d0 2021-02-10T11:41:10.139 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:41:10.137 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:41:10.139 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:41:10.137 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:41:10.139 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:41:10.137 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:41:10.139 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:41:10.137 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:41:10.139 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:41:10.137 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:41:10.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.150 7f956cff9700 1 -- 172.21.2.101:0/2871675104 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f9560006880 con 0x7f95500186d0 2021-02-10T11:41:10.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.166 7f95764dc700 1 -- 172.21.2.101:0/2871675104 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95500186d0 msgr2=0x7f955001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:10.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.166 7f95764dc700 1 --2- 172.21.2.101:0/2871675104 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95500186d0 0x7f955001abd0 crc :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:10.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.166 7f95764dc700 1 -- 172.21.2.101:0/2871675104 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95700ba3e0 msgr2=0x7f95700ba7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:10.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.166 7f95764dc700 1 --2- 172.21.2.101:0/2871675104 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95700ba3e0 0x7f95700ba7e0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 rx=0x7f9564006ed0 tx=0x7f956400b960).stop 2021-02-10T11:41:10.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.166 7f95764dc700 1 -- 172.21.2.101:0/2871675104 shutdown_connections 2021-02-10T11:41:10.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.166 7f95764dc700 1 --2- 172.21.2.101:0/2871675104 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9570085f20 0x7f9570086320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:10.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.166 7f95764dc700 1 --2- 172.21.2.101:0/2871675104 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f95700ba3e0 0x7f95700ba7e0 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:10.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.166 7f95764dc700 1 --2- 172.21.2.101:0/2871675104 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f95500186d0 0x7f955001abd0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:10.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.166 7f95764dc700 1 --2- 172.21.2.101:0/2871675104 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f95700b99c0 0x7f95700b9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:10.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.166 7f95764dc700 1 -- 172.21.2.101:0/2871675104 >> 172.21.2.101:0/2871675104 conn(0x7f95700bc0c0 msgr2=0x7f9570082ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:10.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.167 7f95764dc700 1 -- 172.21.2.101:0/2871675104 shutdown_connections 2021-02-10T11:41:10.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.167 7f95764dc700 1 -- 172.21.2.101:0/2871675104 wait complete. 2021-02-10T11:41:10.177 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolumegroup getpath cephfs group_0000000000831820 2021-02-10T11:41:10.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.590 7faea3450700 1 Processor -- start 2021-02-10T11:41:10.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.590 7faea3450700 1 -- start start 2021-02-10T11:41:10.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.590 7faea3450700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae9c0d1870 0x7fae9c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:10.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.590 7faea3450700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fae9c0cff70 0x7fae9c109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:10.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.590 7faea3450700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9c109a60 con 0x7fae9c1b0ae0 2021-02-10T11:41:10.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.590 7faea3450700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae9c109750 con 0x7fae9c0cff70 2021-02-10T11:41:10.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.590 7faea3450700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c088070 con 0x7fae9c0d1870 2021-02-10T11:41:10.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.591 7faea1a6d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fae9c0cff70 0x7fae9c109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:10.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.591 7faea0a6b700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fae9c1b0ae0 0x7fae9c108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57586/0 (socket says 172.21.2.101:57586) 2021-02-10T11:41:10.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.591 7faea0a6b700 1 -- 172.21.2.101:0/1752400977 learned_addr learned my addr 172.21.2.101:0/1752400977 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:10.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.591 7faea126c700 1 --2- 172.21.2.101:0/1752400977 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae9c0d1870 0x7fae9c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:10.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.591 7faea1a6d700 1 -- 172.21.2.101:0/1752400977 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae9c0d1870 msgr2=0x7fae9c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:10.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.591 7faea1a6d700 1 --2- 172.21.2.101:0/1752400977 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae9c0d1870 0x7fae9c1b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:10.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.591 7faea1a6d700 1 -- 172.21.2.101:0/1752400977 >> v1:172.21.2.101:6789/0 conn(0x7fae9c1b0ae0 legacy=0x7fae9c108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:10.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.591 7faea1a6d700 1 -- 172.21.2.101:0/1752400977 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c088260 con 0x7fae9c0cff70 2021-02-10T11:41:10.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.591 7faea126c700 1 --2- 172.21.2.101:0/1752400977 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae9c0d1870 0x7fae9c1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:41:10.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.591 7faea1a6d700 1 --2- 172.21.2.101:0/1752400977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae9c0cff70 0x7fae9c109130 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 rx=0x7fae94006e30 tx=0x7fae94009e60).ready entity=mon.0 client_cookie=5bcfa782ccbfd9a4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:10.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.592 7fae93fff700 1 -- 172.21.2.101:0/1752400977 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fae9400b300 con 0x7fae9c0cff70 2021-02-10T11:41:10.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.592 7fae93fff700 1 -- 172.21.2.101:0/1752400977 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9400b680 con 0x7fae9c0cff70 2021-02-10T11:41:10.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.592 7fae93fff700 1 -- 172.21.2.101:0/1752400977 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fae9400bb60 con 0x7fae9c0cff70 2021-02-10T11:41:10.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.592 7faea3450700 1 -- 172.21.2.101:0/1752400977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae9c0cff70 msgr2=0x7fae9c109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:10.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.592 7faea3450700 1 --2- 172.21.2.101:0/1752400977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae9c0cff70 0x7fae9c109130 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 rx=0x7fae94006e30 tx=0x7fae94009e60).stop 2021-02-10T11:41:10.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.592 7faea3450700 1 -- 172.21.2.101:0/1752400977 shutdown_connections 2021-02-10T11:41:10.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.592 7faea3450700 1 --2- 172.21.2.101:0/1752400977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae9c0cff70 0x7fae9c109130 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:10.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.592 7faea3450700 1 --2- 172.21.2.101:0/1752400977 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae9c0d1870 0x7fae9c1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:10.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.592 7faea3450700 1 -- 172.21.2.101:0/1752400977 >> 172.21.2.101:0/1752400977 conn(0x7fae9c0ce420 msgr2=0x7fae9c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:10.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.592 7faea3450700 1 -- 172.21.2.101:0/1752400977 shutdown_connections 2021-02-10T11:41:10.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.592 7faea3450700 1 -- 172.21.2.101:0/1752400977 wait complete. 2021-02-10T11:41:10.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.607 7faea3450700 1 Processor -- start 2021-02-10T11:41:10.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.607 7faea3450700 1 -- start start 2021-02-10T11:41:10.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.608 7faea3450700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae9c0abd60 0x7fae9c0ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:10.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.608 7faea3450700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae9c0ac780 0x7fae9c0acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:10.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.608 7faea3450700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fae9c083910 0x7fae9c083d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:10.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.608 7faea3450700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c087180 con 0x7fae9c0abd60 2021-02-10T11:41:10.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.608 7faea3450700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c18fcd0 con 0x7fae9c083910 2021-02-10T11:41:10.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.608 7faea3450700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c084330 con 0x7fae9c0ac780 2021-02-10T11:41:10.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.608 7faea126c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae9c0abd60 0x7fae9c0ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:10.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.608 7faea126c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae9c0abd60 0x7fae9c0ac160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57096/0 (socket says 172.21.2.101:57096) 2021-02-10T11:41:10.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.608 7faea126c700 1 -- 172.21.2.101:0/1232999310 learned_addr learned my addr 172.21.2.101:0/1232999310 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:10.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.608 7faea1a6d700 1 --2- 172.21.2.101:0/1232999310 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fae9c083910 0x7fae9c083d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:10.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.608 7faea0a6b700 1 --2- 172.21.2.101:0/1232999310 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae9c0ac780 0x7fae9c0acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:10.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.608 7faea126c700 1 -- 172.21.2.101:0/1232999310 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae9c0ac780 msgr2=0x7fae9c0acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:10.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.608 7faea126c700 1 --2- 172.21.2.101:0/1232999310 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae9c0ac780 0x7fae9c0acb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:10.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.608 7faea126c700 1 -- 172.21.2.101:0/1232999310 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fae9c083910 msgr2=0x7fae9c083d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:10.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.608 7faea126c700 1 --2- 172.21.2.101:0/1232999310 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fae9c083910 0x7fae9c083d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:10.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.608 7faea126c700 1 -- 172.21.2.101:0/1232999310 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c084550 con 0x7fae9c0abd60 2021-02-10T11:41:10.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.608 7faea0a6b700 1 --2- 172.21.2.101:0/1232999310 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae9c0ac780 0x7fae9c0acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:41:10.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.608 7faea126c700 1 --2- 172.21.2.101:0/1232999310 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae9c0abd60 0x7fae9c0ac160 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 rx=0x7fae98006f40 tx=0x7fae9800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:10.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.609 7fae91ffb700 1 -- 172.21.2.101:0/1232999310 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fae9800bd80 con 0x7fae9c0abd60 2021-02-10T11:41:10.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.609 7fae91ffb700 1 -- 172.21.2.101:0/1232999310 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae98002030 con 0x7fae9c0abd60 2021-02-10T11:41:10.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.609 7fae91ffb700 1 -- 172.21.2.101:0/1232999310 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fae98002550 con 0x7fae9c0abd60 2021-02-10T11:41:10.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.609 7faea3450700 1 -- 172.21.2.101:0/1232999310 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae9c084550 con 0x7fae9c0abd60 2021-02-10T11:41:10.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.609 7faea3450700 1 -- 172.21.2.101:0/1232999310 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9c084550 con 0x7fae9c0abd60 2021-02-10T11:41:10.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.609 7fae91ffb700 1 -- 172.21.2.101:0/1232999310 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fae98000c90 con 0x7fae9c0abd60 2021-02-10T11:41:10.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.609 7fae91ffb700 1 --2- 172.21.2.101:0/1232999310 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fae7c0186c0 0x7fae7c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:10.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.609 7fae91ffb700 1 -- 172.21.2.101:0/1232999310 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fae980327a0 con 0x7fae9c0abd60 2021-02-10T11:41:10.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.609 7faea0a6b700 1 --2- 172.21.2.101:0/1232999310 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fae7c0186c0 0x7fae7c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:10.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.610 7faea0a6b700 1 --2- 172.21.2.101:0/1232999310 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fae7c0186c0 0x7fae7c01abc0 crc :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:10.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.640 7faea3450700 1 -- 172.21.2.101:0/1232999310 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae9c082980 con 0x7fae9c0abd60 2021-02-10T11:41:10.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.643 7fae91ffb700 1 -- 172.21.2.101:0/1232999310 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fae98006bd0 con 0x7fae9c0abd60 2021-02-10T11:41:10.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.772 7faea3450700 1 -- 172.21.2.101:0/1232999310 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolumegroup getpath", "vol_name": "cephfs", "target": ["mgr", ""], "group_name": "group_0000000000831820"}) v1 -- 0x7fae9c084dd0 con 0x7fae7c0186c0 2021-02-10T11:41:10.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.773 7fae91ffb700 1 -- 172.21.2.101:0/1232999310 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+32 (crc 0 0 0) 0x7fae8c001120 con 0x7fae7c0186c0 2021-02-10T11:41:10.776 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/group_0000000000831820 2021-02-10T11:41:10.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.774 7faea3450700 1 -- 172.21.2.101:0/1232999310 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fae7c0186c0 msgr2=0x7fae7c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:10.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.774 7faea3450700 1 --2- 172.21.2.101:0/1232999310 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fae7c0186c0 0x7fae7c01abc0 crc :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:10.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.774 7faea3450700 1 -- 172.21.2.101:0/1232999310 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae9c0abd60 msgr2=0x7fae9c0ac160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:10.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.774 7faea3450700 1 --2- 172.21.2.101:0/1232999310 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae9c0abd60 0x7fae9c0ac160 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 rx=0x7fae98006f40 tx=0x7fae9800b990).stop 2021-02-10T11:41:10.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.775 7faea3450700 1 -- 172.21.2.101:0/1232999310 shutdown_connections 2021-02-10T11:41:10.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.775 7faea3450700 1 --2- 172.21.2.101:0/1232999310 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fae7c0186c0 0x7fae7c01abc0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:10.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.775 7faea3450700 1 --2- 172.21.2.101:0/1232999310 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae9c0ac780 0x7fae9c0acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:10.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.775 7faea3450700 1 --2- 172.21.2.101:0/1232999310 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fae9c083910 0x7fae9c083d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:10.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.775 7faea3450700 1 --2- 172.21.2.101:0/1232999310 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae9c0abd60 0x7fae9c0ac160 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:10.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.775 7faea3450700 1 -- 172.21.2.101:0/1232999310 >> 172.21.2.101:0/1232999310 conn(0x7fae9c0a2b30 msgr2=0x7fae9c09ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:10.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.775 7faea3450700 1 -- 172.21.2.101:0/1232999310 shutdown_connections 2021-02-10T11:41:10.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:10.775 7faea3450700 1 -- 172.21.2.101:0/1232999310 wait complete. 2021-02-10T11:41:10.786 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-02-10T11:41:10.786 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:41:10.786 DEBUG:teuthology.orchestra.run.gibba001:> import stat 2021-02-10T11:41:10.786 DEBUG:teuthology.orchestra.run.gibba001:> import json 2021-02-10T11:41:10.786 DEBUG:teuthology.orchestra.run.gibba001:> import sys 2021-02-10T11:41:10.786 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:41:10.786 DEBUG:teuthology.orchestra.run.gibba001:> try: 2021-02-10T11:41:10.786 DEBUG:teuthology.orchestra.run.gibba001:> s = os.stat("/home/ubuntu/cephtest/mnt.0/volumes/group_0000000000831820") 2021-02-10T11:41:10.786 DEBUG:teuthology.orchestra.run.gibba001:> except OSError as e: 2021-02-10T11:41:10.786 DEBUG:teuthology.orchestra.run.gibba001:> sys.exit(e.errno) 2021-02-10T11:41:10.786 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:41:10.786 DEBUG:teuthology.orchestra.run.gibba001:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2021-02-10T11:41:10.786 DEBUG:teuthology.orchestra.run.gibba001:> print(json.dumps( 2021-02-10T11:41:10.787 DEBUG:teuthology.orchestra.run.gibba001:> dict([(a, getattr(s, a)) for a in attrs]), 2021-02-10T11:41:10.787 DEBUG:teuthology.orchestra.run.gibba001:> indent=2)) 2021-02-10T11:41:10.787 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:41:11.177 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:41:11.177 INFO:teuthology.orchestra.run.gibba001.stdout: "st_mode": 16877, 2021-02-10T11:41:11.177 INFO:teuthology.orchestra.run.gibba001.stdout: "st_ino": 1099511627777, 2021-02-10T11:41:11.177 INFO:teuthology.orchestra.run.gibba001.stdout: "st_dev": 43, 2021-02-10T11:41:11.177 INFO:teuthology.orchestra.run.gibba001.stdout: "st_nlink": 2, 2021-02-10T11:41:11.177 INFO:teuthology.orchestra.run.gibba001.stdout: "st_uid": 0, 2021-02-10T11:41:11.177 INFO:teuthology.orchestra.run.gibba001.stdout: "st_gid": 0, 2021-02-10T11:41:11.178 INFO:teuthology.orchestra.run.gibba001.stdout: "st_size": 0, 2021-02-10T11:41:11.178 INFO:teuthology.orchestra.run.gibba001.stdout: "st_atime": 1612957270.1459336, 2021-02-10T11:41:11.178 INFO:teuthology.orchestra.run.gibba001.stdout: "st_mtime": 1612957270.1459336, 2021-02-10T11:41:11.178 INFO:teuthology.orchestra.run.gibba001.stdout: "st_ctime": 1612957270.1517448 2021-02-10T11:41:11.178 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:41:11.360 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolumegroup rm cephfs group_0000000000831820 2021-02-10T11:41:11.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.811 7f18cbaad700 1 Processor -- start 2021-02-10T11:41:11.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.812 7f18cbaad700 1 -- start start 2021-02-10T11:41:11.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.812 7f18cbaad700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18c40d1c70 0x7f18c40cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:11.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.812 7f18cbaad700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c40d0170 con 0x7f18c40cf580 2021-02-10T11:41:11.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.812 7f18cbaad700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c40fe1c0 con 0x7f18c40d1870 2021-02-10T11:41:11.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.812 7f18cbaad700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c41b0cb0 con 0x7f18c40d1c70 2021-02-10T11:41:11.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.812 7f18ca0ca700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f18c40cf580 0x7f18c40cf980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57600/0 (socket says 172.21.2.101:57600) 2021-02-10T11:41:11.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.812 7f18ca0ca700 1 -- 172.21.2.101:0/4073942368 learned_addr learned my addr 172.21.2.101:0/4073942368 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:11.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.812 7f18c90c8700 1 --2- 172.21.2.101:0/4073942368 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18c40d1c70 0x7f18c40cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:11.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.813 7f18c88c7700 1 -- 172.21.2.101:0/4073942368 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2728441855 0 0) 0x7f18c0000c10 con 0x7f18c40cf580 2021-02-10T11:41:11.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.813 7f18c88c7700 1 -- 172.21.2.101:0/4073942368 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18b0002fe0 con 0x7f18c40cf580 2021-02-10T11:41:11.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.813 7f18c88c7700 1 -- 172.21.2.101:0/4073942368 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 434868741 0 0) 0x7f18b4000c60 con 0x7f18c40d1870 2021-02-10T11:41:11.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.813 7f18c88c7700 1 -- 172.21.2.101:0/4073942368 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18b0003210 con 0x7f18c40d1870 2021-02-10T11:41:11.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.813 7f18c90c8700 1 -- 172.21.2.101:0/4073942368 >> v1:172.21.2.108:6789/0 conn(0x7f18c40d1870 legacy=0x7f18c41b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:11.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.813 7f18c90c8700 1 -- 172.21.2.101:0/4073942368 >> v1:172.21.2.101:6789/0 conn(0x7f18c40cf580 legacy=0x7f18c40cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:11.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.813 7f18c90c8700 1 -- 172.21.2.101:0/4073942368 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c4109a60 con 0x7f18c40d1c70 2021-02-10T11:41:11.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.813 7f18c90c8700 1 --2- 172.21.2.101:0/4073942368 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18c40d1c70 0x7f18c40cef60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f18bc0069d0 tx=0x7f18bc009e40).ready entity=mon.2 client_cookie=777dc479e5e6fc14 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:11.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.813 7f18c88c7700 1 -- 172.21.2.101:0/4073942368 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f18bc00b510 con 0x7f18c40d1c70 2021-02-10T11:41:11.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.813 7f18c88c7700 1 -- 172.21.2.101:0/4073942368 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18bc00b790 con 0x7f18c40d1c70 2021-02-10T11:41:11.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.813 7f18c88c7700 1 -- 172.21.2.101:0/4073942368 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f18bc00bcb0 con 0x7f18c40d1c70 2021-02-10T11:41:11.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.813 7f18cbaad700 1 -- 172.21.2.101:0/4073942368 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18c40d1c70 msgr2=0x7f18c40cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:11.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.813 7f18cbaad700 1 --2- 172.21.2.101:0/4073942368 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18c40d1c70 0x7f18c40cef60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f18bc0069d0 tx=0x7f18bc009e40).stop 2021-02-10T11:41:11.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.814 7f18cbaad700 1 -- 172.21.2.101:0/4073942368 shutdown_connections 2021-02-10T11:41:11.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.814 7f18cbaad700 1 --2- 172.21.2.101:0/4073942368 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18c40d1c70 0x7f18c40cef60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:11.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.814 7f18cbaad700 1 -- 172.21.2.101:0/4073942368 >> 172.21.2.101:0/4073942368 conn(0x7f18c40ce420 msgr2=0x7f18c40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:11.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.814 7f18cbaad700 1 -- 172.21.2.101:0/4073942368 shutdown_connections 2021-02-10T11:41:11.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.814 7f18cbaad700 1 -- 172.21.2.101:0/4073942368 wait complete. 2021-02-10T11:41:11.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.829 7f18cbaad700 1 Processor -- start 2021-02-10T11:41:11.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.829 7f18cbaad700 1 -- start start 2021-02-10T11:41:11.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.829 7f18cbaad700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f18c4083f60 0x7f18c4094f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:11.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.829 7f18cbaad700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18c41a7230 0x7f18c41a7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:11.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.829 7f18cbaad700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18c41a7c50 0x7f18c40c14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:11.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.829 7f18cbaad700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c40c0f60 con 0x7f18c41a7c50 2021-02-10T11:41:11.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.829 7f18cbaad700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c4084360 con 0x7f18c4083f60 2021-02-10T11:41:11.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.829 7f18cbaad700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c4084790 con 0x7f18c41a7230 2021-02-10T11:41:11.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.830 7f18c90c8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18c41a7230 0x7f18c41a7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:11.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.830 7f18c90c8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18c41a7230 0x7f18c41a7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:49840/0 (socket says 172.21.2.101:49840) 2021-02-10T11:41:11.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.830 7f18c90c8700 1 -- 172.21.2.101:0/3241353327 learned_addr learned my addr 172.21.2.101:0/3241353327 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:11.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.830 7f18c98c9700 1 --2- 172.21.2.101:0/3241353327 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f18c4083f60 0x7f18c4094f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:11.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.830 7f18ca0ca700 1 --2- 172.21.2.101:0/3241353327 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18c41a7c50 0x7f18c40c14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:11.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.830 7f18ca0ca700 1 -- 172.21.2.101:0/3241353327 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18c41a7230 msgr2=0x7f18c41a7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:11.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.830 7f18ca0ca700 1 --2- 172.21.2.101:0/3241353327 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18c41a7230 0x7f18c41a7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:11.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.830 7f18ca0ca700 1 -- 172.21.2.101:0/3241353327 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f18c4083f60 msgr2=0x7f18c4094f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:11.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.830 7f18ca0ca700 1 --2- 172.21.2.101:0/3241353327 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f18c4083f60 0x7f18c4094f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:11.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.830 7f18ca0ca700 1 -- 172.21.2.101:0/3241353327 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c41a8050 con 0x7f18c41a7c50 2021-02-10T11:41:11.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.830 7f18ca0ca700 1 --2- 172.21.2.101:0/3241353327 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18c41a7c50 0x7f18c40c14e0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 rx=0x7f18c0006e90 tx=0x7f18c000b810).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:11.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.830 7f18ba7fc700 1 -- 172.21.2.101:0/3241353327 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f18c000bc00 con 0x7f18c41a7c50 2021-02-10T11:41:11.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.830 7f18ba7fc700 1 -- 172.21.2.101:0/3241353327 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c0006020 con 0x7f18c41a7c50 2021-02-10T11:41:11.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.830 7f18ba7fc700 1 -- 172.21.2.101:0/3241353327 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f18c00043a0 con 0x7f18c41a7c50 2021-02-10T11:41:11.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.830 7f18cbaad700 1 -- 172.21.2.101:0/3241353327 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c41a8050 con 0x7f18c41a7c50 2021-02-10T11:41:11.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.831 7f18cbaad700 1 -- 172.21.2.101:0/3241353327 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c41a8050 con 0x7f18c41a7c50 2021-02-10T11:41:11.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.831 7f18ba7fc700 1 -- 172.21.2.101:0/3241353327 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f18c0000ef0 con 0x7f18c41a7c50 2021-02-10T11:41:11.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.831 7f18ba7fc700 1 --2- 172.21.2.101:0/3241353327 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18ac0186c0 0x7f18ac01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:11.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.831 7f18ba7fc700 1 -- 172.21.2.101:0/3241353327 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f18c00323b0 con 0x7f18c41a7c50 2021-02-10T11:41:11.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.831 7f18c98c9700 1 --2- 172.21.2.101:0/3241353327 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18ac0186c0 0x7f18ac01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:11.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.831 7f18c98c9700 1 --2- 172.21.2.101:0/3241353327 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18ac0186c0 0x7f18ac01abc0 crc :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:11.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.861 7f18cbaad700 1 -- 172.21.2.101:0/3241353327 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c40fe1c0 con 0x7f18c41a7c50 2021-02-10T11:41:11.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.864 7f18ba7fc700 1 -- 172.21.2.101:0/3241353327 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f18c00009b0 con 0x7f18c41a7c50 2021-02-10T11:41:11.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.992 7f18cbaad700 1 -- 172.21.2.101:0/3241353327 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "target": ["mgr", ""], "group_name": "group_0000000000831820"}) v1 -- 0x7f18c40fe1c0 con 0x7f18ac0186c0 2021-02-10T11:41:11.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:11.996 7f18ba7fc700 1 -- 172.21.2.101:0/3241353327 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f18b4006550 con 0x7f18ac0186c0 2021-02-10T11:41:12.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:12.001 7f18cbaad700 1 -- 172.21.2.101:0/3241353327 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18ac0186c0 msgr2=0x7f18ac01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:12.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:12.001 7f18cbaad700 1 --2- 172.21.2.101:0/3241353327 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18ac0186c0 0x7f18ac01abc0 crc :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:12.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:12.001 7f18cbaad700 1 -- 172.21.2.101:0/3241353327 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18c41a7c50 msgr2=0x7f18c40c14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:12.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:12.001 7f18cbaad700 1 --2- 172.21.2.101:0/3241353327 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18c41a7c50 0x7f18c40c14e0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 rx=0x7f18c0006e90 tx=0x7f18c000b810).stop 2021-02-10T11:41:12.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:12.002 7f18cbaad700 1 -- 172.21.2.101:0/3241353327 shutdown_connections 2021-02-10T11:41:12.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:12.002 7f18cbaad700 1 --2- 172.21.2.101:0/3241353327 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18ac0186c0 0x7f18ac01abc0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:12.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:12.002 7f18cbaad700 1 --2- 172.21.2.101:0/3241353327 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18c41a7230 0x7f18c41a7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:12.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:12.002 7f18cbaad700 1 --2- 172.21.2.101:0/3241353327 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18c41a7c50 0x7f18c40c14e0 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:12.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:12.002 7f18cbaad700 1 --2- 172.21.2.101:0/3241353327 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f18c4083f60 0x7f18c4094f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:12.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:12.002 7f18cbaad700 1 -- 172.21.2.101:0/3241353327 >> 172.21.2.101:0/3241353327 conn(0x7f18c40b6870 msgr2=0x7f18c40abf70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:12.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:12.002 7f18cbaad700 1 -- 172.21.2.101:0/3241353327 shutdown_connections 2021-02-10T11:41:12.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:12.002 7f18cbaad700 1 -- 172.21.2.101:0/3241353327 wait complete. 2021-02-10T11:41:12.012 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:41:12.013 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:41:12.358 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:41:12.703 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:41:12.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:12.703 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:41:12.703 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:41:13.039 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:41:13.039 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:41:13.039 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:41:13.039 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:41:13.039 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:41:13.389 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[147200]: fuse finished with error 0 and tester_r 0 2021-02-10T11:41:13.391 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:13.391 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:41:13.391 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:41:13.725 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:41:13.726 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:41:13.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:41:13.726 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:41:14.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:14.063 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:41:14.063 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:41:14.399 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:41:14.399 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:41:14.400 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:41:14.400 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:41:14.400 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:41:14.756 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[147867]: fuse finished with error 0 and tester_r 0 2021-02-10T11:41:14.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:14.758 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:41:14.758 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:41:15.091 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:41:15.092 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:41:15.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:41:15.092 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:41:15.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:15.428 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:41:15.428 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:41:15.769 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:41:15.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:41:15.770 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:41:15.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:41:15.770 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:41:16.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:16.106 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:41:16.106 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:41:16.442 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:41:16.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:41:16.442 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:41:16.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:41:16.442 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:41:16.777 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-02-10T11:41:17.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.191 7f7c0c000700 1 Processor -- start 2021-02-10T11:41:17.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.191 7f7c0c000700 1 -- start start 2021-02-10T11:41:17.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.191 7f7c0c000700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7c040d0340 0x7f7c040ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:17.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.191 7f7c0c000700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c040f0b10 con 0x7f7c040d1870 2021-02-10T11:41:17.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.191 7f7c0c000700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c041b0be0 con 0x7f7c040d1c70 2021-02-10T11:41:17.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.191 7f7c0c000700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c040e5ed0 con 0x7f7c040d0340 2021-02-10T11:41:17.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.191 7f7c09e1c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f7c040d1870 0x7f7c041b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57610/0 (socket says 172.21.2.101:57610) 2021-02-10T11:41:17.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.191 7f7c09e1c700 1 -- 172.21.2.101:0/1528022560 learned_addr learned my addr 172.21.2.101:0/1528022560 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:17.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.192 7f7c08e1a700 1 -- 172.21.2.101:0/1528022560 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2519456797 0 0) 0x7f7c000009b0 con 0x7f7c040d1870 2021-02-10T11:41:17.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.192 7f7c08e1a700 1 -- 172.21.2.101:0/1528022560 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bf0002fe0 con 0x7f7c040d1870 2021-02-10T11:41:17.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.192 7f7c0961b700 1 --2- 172.21.2.101:0/1528022560 >> v2:172.21.2.108:3300/0 conn(0x7f7c040d0340 0x7f7c040ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:17.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.192 7f7c08e1a700 1 -- 172.21.2.101:0/1528022560 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2740279425 0 0) 0x7f7c00000e20 con 0x7f7c040d1870 2021-02-10T11:41:17.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.192 7f7c08e1a700 1 -- 172.21.2.101:0/1528022560 >> v2:172.21.2.108:3300/0 conn(0x7f7c040d0340 msgr2=0x7f7c040ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:17.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.192 7f7c08e1a700 1 --2- 172.21.2.101:0/1528022560 >> v2:172.21.2.108:3300/0 conn(0x7f7c040d0340 0x7f7c040ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:17.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.192 7f7c08e1a700 1 -- 172.21.2.101:0/1528022560 >> v1:172.21.2.108:6789/0 conn(0x7f7c040d1c70 legacy=0x7f7c040efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:17.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.192 7f7c08e1a700 1 -- 172.21.2.101:0/1528022560 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c040f0450 con 0x7f7c040d1870 2021-02-10T11:41:17.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.192 7f7c08e1a700 1 -- 172.21.2.101:0/1528022560 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f7c000015b0 con 0x7f7c040d1870 2021-02-10T11:41:17.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.192 7f7c08e1a700 1 -- 172.21.2.101:0/1528022560 >> v1:172.21.2.101:6789/0 conn(0x7f7c040d1870 legacy=0x7f7c041b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:17.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.192 7f7c08e1a700 1 --2- 172.21.2.101:0/1528022560 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7bf0003aa0 0x7f7bf0005ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:17.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.192 7f7c08e1a700 1 --2- 172.21.2.101:0/1528022560 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7bf0006770 0x7f7bf0008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:17.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.192 7f7c08e1a700 1 --2- 172.21.2.101:0/1528022560 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bf0009590 0x7f7bf000bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:17.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.192 7f7c08e1a700 1 -- 172.21.2.101:0/1528022560 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf000c300 con 0x7f7bf0009590 2021-02-10T11:41:17.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.192 7f7c08e1a700 1 -- 172.21.2.101:0/1528022560 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf000c520 con 0x7f7bf0006770 2021-02-10T11:41:17.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.192 7f7c08e1a700 1 -- 172.21.2.101:0/1528022560 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf000c740 con 0x7f7bf0003aa0 2021-02-10T11:41:17.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.192 7f7c0a61d700 1 --2- 172.21.2.101:0/1528022560 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bf0009590 0x7f7bf000bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:17.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.192 7f7c09e1c700 1 --2- 172.21.2.101:0/1528022560 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7bf0003aa0 0x7f7bf0005ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:17.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.193 7f7c0961b700 1 --2- 172.21.2.101:0/1528022560 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7bf0006770 0x7f7bf0008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:17.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.193 7f7c0a61d700 1 -- 172.21.2.101:0/1528022560 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7bf0003aa0 msgr2=0x7f7bf0005ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:17.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.193 7f7c0a61d700 1 --2- 172.21.2.101:0/1528022560 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7bf0003aa0 0x7f7bf0005ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:17.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.193 7f7c0a61d700 1 -- 172.21.2.101:0/1528022560 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7bf0006770 msgr2=0x7f7bf0008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:17.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.193 7f7c0a61d700 1 --2- 172.21.2.101:0/1528022560 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7bf0006770 0x7f7bf0008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:17.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.193 7f7c0a61d700 1 -- 172.21.2.101:0/1528022560 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf000c9e0 con 0x7f7bf0009590 2021-02-10T11:41:17.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.193 7f7c0a61d700 1 -- 172.21.2.101:0/1528022560 reap_dead start 2021-02-10T11:41:17.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.193 7f7c0a61d700 1 --2- 172.21.2.101:0/1528022560 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bf0009590 0x7f7bf000bab0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 rx=0x7f7bfc0069e0 tx=0x7f7bfc04de30).ready entity=mon.0 client_cookie=1177641ba4bd5604 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:17.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.193 7f7c08e1a700 1 -- 172.21.2.101:0/1528022560 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7bfc04f300 con 0x7f7bf0009590 2021-02-10T11:41:17.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.193 7f7c08e1a700 1 -- 172.21.2.101:0/1528022560 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bfc04f540 con 0x7f7bf0009590 2021-02-10T11:41:17.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.193 7f7c08e1a700 1 -- 172.21.2.101:0/1528022560 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7bfc04f9f0 con 0x7f7bf0009590 2021-02-10T11:41:17.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.193 7f7c0c000700 1 -- 172.21.2.101:0/1528022560 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bf0009590 msgr2=0x7f7bf000bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:17.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.193 7f7c0c000700 1 --2- 172.21.2.101:0/1528022560 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bf0009590 0x7f7bf000bab0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 rx=0x7f7bfc0069e0 tx=0x7f7bfc04de30).stop 2021-02-10T11:41:17.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.194 7f7c0c000700 1 -- 172.21.2.101:0/1528022560 shutdown_connections 2021-02-10T11:41:17.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.194 7f7c0c000700 1 --2- 172.21.2.101:0/1528022560 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7bf0009590 0x7f7bf000bab0 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:17.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.194 7f7c0c000700 1 -- 172.21.2.101:0/1528022560 >> 172.21.2.101:0/1528022560 conn(0x7f7c040ce450 msgr2=0x7f7c040d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:17.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.194 7f7c0c000700 1 -- 172.21.2.101:0/1528022560 shutdown_connections 2021-02-10T11:41:17.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.194 7f7c0c000700 1 -- 172.21.2.101:0/1528022560 wait complete. 2021-02-10T11:41:17.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.209 7f7c0c000700 1 Processor -- start 2021-02-10T11:41:17.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.209 7f7c0c000700 1 -- start start 2021-02-10T11:41:17.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.209 7f7c0c000700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c0418f790 0x7f7c040bc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:17.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.209 7f7c0c000700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c0418fb90 0x7f7c040bc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:17.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.209 7f7c0c000700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c040bcd80 0x7f7c040bd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:17.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.209 7f7c0c000700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040ebe70 con 0x7f7c0418fb90 2021-02-10T11:41:17.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.209 7f7c0c000700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c041a49c0 con 0x7f7c0418f790 2021-02-10T11:41:17.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.209 7f7c0c000700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0409fd10 con 0x7f7c040bcd80 2021-02-10T11:41:17.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.209 7f7c0961b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c0418fb90 0x7f7c040bc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:17.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.209 7f7c0961b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c0418fb90 0x7f7c040bc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57128/0 (socket says 172.21.2.101:57128) 2021-02-10T11:41:17.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.209 7f7c0961b700 1 -- 172.21.2.101:0/3228233851 learned_addr learned my addr 172.21.2.101:0/3228233851 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:17.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.210 7f7c0961b700 1 -- 172.21.2.101:0/3228233851 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c040bcd80 msgr2=0x7f7c040bd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:17.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.210 7f7c0961b700 1 --2- 172.21.2.101:0/3228233851 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c040bcd80 0x7f7c040bd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:17.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.210 7f7c0961b700 1 -- 172.21.2.101:0/3228233851 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c0418f790 msgr2=0x7f7c040bc140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:17.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.210 7f7c0961b700 1 --2- 172.21.2.101:0/3228233851 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c0418f790 0x7f7c040bc140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:17.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.210 7f7c0961b700 1 -- 172.21.2.101:0/3228233851 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c040bdd50 con 0x7f7c0418fb90 2021-02-10T11:41:17.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.210 7f7c0961b700 1 --2- 172.21.2.101:0/3228233851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c0418fb90 0x7f7c040bc760 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 rx=0x7f7bf4006950 tx=0x7f7bf400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:17.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.210 7f7bfa7fc700 1 -- 172.21.2.101:0/3228233851 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7bf400bd50 con 0x7f7c0418fb90 2021-02-10T11:41:17.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.210 7f7bfa7fc700 1 -- 172.21.2.101:0/3228233851 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf4000c90 con 0x7f7c0418fb90 2021-02-10T11:41:17.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.210 7f7c0c000700 1 -- 172.21.2.101:0/3228233851 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c040bdd50 con 0x7f7c0418fb90 2021-02-10T11:41:17.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.210 7f7bfa7fc700 1 -- 172.21.2.101:0/3228233851 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7bf40022b0 con 0x7f7c0418fb90 2021-02-10T11:41:17.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.210 7f7c0c000700 1 -- 172.21.2.101:0/3228233851 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c040bdd50 con 0x7f7c0418fb90 2021-02-10T11:41:17.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.211 7f7bfa7fc700 1 -- 172.21.2.101:0/3228233851 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7bf4002020 con 0x7f7c0418fb90 2021-02-10T11:41:17.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.211 7f7bfa7fc700 1 --2- 172.21.2.101:0/3228233851 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7bec018730 0x7f7bec01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:17.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.211 7f7bfa7fc700 1 -- 172.21.2.101:0/3228233851 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7bf4032790 con 0x7f7c0418fb90 2021-02-10T11:41:17.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.211 7f7c09e1c700 1 --2- 172.21.2.101:0/3228233851 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7bec018730 0x7f7bec01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:17.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.211 7f7c09e1c700 1 --2- 172.21.2.101:0/3228233851 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7bec018730 0x7f7bec01ac10 crc :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:17.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.241 7f7c0c000700 1 -- 172.21.2.101:0/3228233851 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c040f0b10 con 0x7f7c0418fb90 2021-02-10T11:41:17.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.244 7f7bfa7fc700 1 -- 172.21.2.101:0/3228233851 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7bf40065e0 con 0x7f7c0418fb90 2021-02-10T11:41:17.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.394 7f7c0c000700 1 -- 172.21.2.101:0/3228233851 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7c040f0b10 con 0x7f7c0418fb90 2021-02-10T11:41:17.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.397 7f7bfa7fc700 1 -- 172.21.2.101:0/3228233851 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7bf4000c90 con 0x7f7c0418fb90 2021-02-10T11:41:17.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.403 7f7c0c000700 1 -- 172.21.2.101:0/3228233851 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7bec018730 msgr2=0x7f7bec01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:17.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.403 7f7c0c000700 1 --2- 172.21.2.101:0/3228233851 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7bec018730 0x7f7bec01ac10 crc :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:17.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.403 7f7c0c000700 1 -- 172.21.2.101:0/3228233851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c0418fb90 msgr2=0x7f7c040bc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:17.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.403 7f7c0c000700 1 --2- 172.21.2.101:0/3228233851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c0418fb90 0x7f7c040bc760 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 rx=0x7f7bf4006950 tx=0x7f7bf400b960).stop 2021-02-10T11:41:17.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.403 7f7c0c000700 1 -- 172.21.2.101:0/3228233851 shutdown_connections 2021-02-10T11:41:17.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.403 7f7c0c000700 1 --2- 172.21.2.101:0/3228233851 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7bec018730 0x7f7bec01ac10 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:17.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.403 7f7c0c000700 1 --2- 172.21.2.101:0/3228233851 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c040bcd80 0x7f7c040bd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:17.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.403 7f7c0c000700 1 --2- 172.21.2.101:0/3228233851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c0418fb90 0x7f7c040bc760 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:17.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.403 7f7c0c000700 1 --2- 172.21.2.101:0/3228233851 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c0418f790 0x7f7c040bc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:17.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.403 7f7c0c000700 1 -- 172.21.2.101:0/3228233851 >> 172.21.2.101:0/3228233851 conn(0x7f7c040c2d70 msgr2=0x7f7c04094c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:17.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.403 7f7c0c000700 1 -- 172.21.2.101:0/3228233851 shutdown_connections 2021-02-10T11:41:17.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.403 7f7c0c000700 1 -- 172.21.2.101:0/3228233851 wait complete. 2021-02-10T11:41:17.413 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_default_uid_gid_subvolume_group' 2021-02-10T11:41:17.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.822 7f3e7904d700 1 Processor -- start 2021-02-10T11:41:17.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.822 7f3e7904d700 1 -- start start 2021-02-10T11:41:17.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.822 7f3e7904d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e740d1870 0x7f3e741b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.822 7f3e7904d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3e740f0650 0x7f3e740cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.822 7f3e7904d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e740f0b10 con 0x7f3e741b0a10 2021-02-10T11:41:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.822 7f3e7904d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e740fe230 con 0x7f3e740f0650 2021-02-10T11:41:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.822 7f3e7904d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e740e5ed0 con 0x7f3e740d1870 2021-02-10T11:41:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.823 7f3e7359e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3e740f0650 0x7f3e740cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.823 7f3e7259c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f3e741b0a10 0x7f3e740cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57628/0 (socket says 172.21.2.101:57628) 2021-02-10T11:41:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.823 7f3e7259c700 1 -- 172.21.2.101:0/1003964944 learned_addr learned my addr 172.21.2.101:0/1003964944 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.823 7f3e7359e700 1 -- 172.21.2.101:0/1003964944 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e740d1870 msgr2=0x7f3e741b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:41:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.823 7f3e7359e700 1 --2- 172.21.2.101:0/1003964944 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e740d1870 0x7f3e741b03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.823 7f3e7359e700 1 -- 172.21.2.101:0/1003964944 >> v1:172.21.2.101:6789/0 conn(0x7f3e741b0a10 legacy=0x7f3e740cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.823 7f3e7359e700 1 -- 172.21.2.101:0/1003964944 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e740f20a0 con 0x7f3e740f0650 2021-02-10T11:41:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.823 7f3e7359e700 1 --2- 172.21.2.101:0/1003964944 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e740f0650 0x7f3e740cfb20 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 rx=0x7f3e64006ab0 tx=0x7f3e64009b40).ready entity=mon.0 client_cookie=8ed2bc301f3953eb server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.823 7f3e71d9b700 1 -- 172.21.2.101:0/1003964944 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3e640022e0 con 0x7f3e740f0650 2021-02-10T11:41:17.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.823 7f3e71d9b700 1 -- 172.21.2.101:0/1003964944 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e64002560 con 0x7f3e740f0650 2021-02-10T11:41:17.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.823 7f3e71d9b700 1 -- 172.21.2.101:0/1003964944 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3e640029f0 con 0x7f3e740f0650 2021-02-10T11:41:17.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.823 7f3e7904d700 1 -- 172.21.2.101:0/1003964944 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e740f0650 msgr2=0x7f3e740cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:17.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.823 7f3e7904d700 1 --2- 172.21.2.101:0/1003964944 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e740f0650 0x7f3e740cfb20 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 rx=0x7f3e64006ab0 tx=0x7f3e64009b40).stop 2021-02-10T11:41:17.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.824 7f3e7904d700 1 -- 172.21.2.101:0/1003964944 shutdown_connections 2021-02-10T11:41:17.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.824 7f3e7904d700 1 --2- 172.21.2.101:0/1003964944 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e740f0650 0x7f3e740cfb20 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:17.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.824 7f3e7904d700 1 --2- 172.21.2.101:0/1003964944 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e740d1870 0x7f3e741b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:17.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.824 7f3e7904d700 1 -- 172.21.2.101:0/1003964944 >> 172.21.2.101:0/1003964944 conn(0x7f3e740ce450 msgr2=0x7f3e740d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:17.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.824 7f3e7904d700 1 -- 172.21.2.101:0/1003964944 shutdown_connections 2021-02-10T11:41:17.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.824 7f3e7904d700 1 -- 172.21.2.101:0/1003964944 wait complete. 2021-02-10T11:41:17.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.838 7f3e7904d700 1 Processor -- start 2021-02-10T11:41:17.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.838 7f3e7904d700 1 -- start start 2021-02-10T11:41:17.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.839 7f3e7904d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e740b4f30 0x7f3e7418b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:17.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.839 7f3e7904d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e74093f40 0x7f3e74094340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:17.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.839 7f3e7904d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3e74094960 0x7f3e74187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:17.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.839 7f3e7904d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e74094e90 con 0x7f3e740b4f30 2021-02-10T11:41:17.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.839 7f3e7904d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7418bff0 con 0x7f3e74094960 2021-02-10T11:41:17.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.839 7f3e7904d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7408f780 con 0x7f3e74093f40 2021-02-10T11:41:17.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.839 7f3e72d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e740b4f30 0x7f3e7418b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:17.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.839 7f3e72d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e740b4f30 0x7f3e7418b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57140/0 (socket says 172.21.2.101:57140) 2021-02-10T11:41:17.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.839 7f3e72d9d700 1 -- 172.21.2.101:0/3113865381 learned_addr learned my addr 172.21.2.101:0/3113865381 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:17.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.839 7f3e72d9d700 1 -- 172.21.2.101:0/3113865381 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e74093f40 msgr2=0x7f3e74094340 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:17.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.839 7f3e72d9d700 1 --2- 172.21.2.101:0/3113865381 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e74093f40 0x7f3e74094340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:17.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.839 7f3e72d9d700 1 -- 172.21.2.101:0/3113865381 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3e74094960 msgr2=0x7f3e74187e50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:17.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.839 7f3e72d9d700 1 --2- 172.21.2.101:0/3113865381 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3e74094960 0x7f3e74187e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:17.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.839 7f3e72d9d700 1 -- 172.21.2.101:0/3113865381 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e74188470 con 0x7f3e740b4f30 2021-02-10T11:41:17.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.839 7f3e72d9d700 1 --2- 172.21.2.101:0/3113865381 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e740b4f30 0x7f3e7418b9d0 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 rx=0x7f3e6800b5c0 tx=0x7f3e6800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:17.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.840 7f3e637fe700 1 -- 172.21.2.101:0/3113865381 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3e6800bd50 con 0x7f3e740b4f30 2021-02-10T11:41:17.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.840 7f3e637fe700 1 -- 172.21.2.101:0/3113865381 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e68006030 con 0x7f3e740b4f30 2021-02-10T11:41:17.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.840 7f3e637fe700 1 -- 172.21.2.101:0/3113865381 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3e680054d0 con 0x7f3e740b4f30 2021-02-10T11:41:17.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.840 7f3e7904d700 1 -- 172.21.2.101:0/3113865381 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e74188470 con 0x7f3e740b4f30 2021-02-10T11:41:17.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.840 7f3e7904d700 1 -- 172.21.2.101:0/3113865381 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e74188470 con 0x7f3e740b4f30 2021-02-10T11:41:17.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.840 7f3e637fe700 1 -- 172.21.2.101:0/3113865381 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3e68006210 con 0x7f3e740b4f30 2021-02-10T11:41:17.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.841 7f3e637fe700 1 --2- 172.21.2.101:0/3113865381 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3e580186c0 0x7f3e5801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:17.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.841 7f3e637fe700 1 -- 172.21.2.101:0/3113865381 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3e68032550 con 0x7f3e740b4f30 2021-02-10T11:41:17.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.841 7f3e7259c700 1 --2- 172.21.2.101:0/3113865381 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3e580186c0 0x7f3e5801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:17.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.841 7f3e7259c700 1 --2- 172.21.2.101:0/3113865381 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3e580186c0 0x7f3e5801abc0 crc :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:17.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.872 7f3e7904d700 1 -- 172.21.2.101:0/3113865381 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e740ce450 con 0x7f3e740b4f30 2021-02-10T11:41:17.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:17.875 7f3e637fe700 1 -- 172.21.2.101:0/3113865381 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3e68006b80 con 0x7f3e740b4f30 2021-02-10T11:41:18.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.007 7f3e7904d700 1 -- 172.21.2.101:0/3113865381 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_default_uid_gid_subvolume_group"]} v 0) v1 -- 0x7f3e740ce450 con 0x7f3e740b4f30 2021-02-10T11:41:18.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.569 7f3e637fe700 1 -- 172.21.2.101:0/3113865381 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_default_uid_gid_subvolume_group"]}]=0 v581) v1 ==== 153+0+0 (secure 0 0 0) 0x7f3e68006210 con 0x7f3e740b4f30 2021-02-10T11:41:18.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.575 7f3e7904d700 1 -- 172.21.2.101:0/3113865381 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3e580186c0 msgr2=0x7f3e5801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:18.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.575 7f3e7904d700 1 --2- 172.21.2.101:0/3113865381 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3e580186c0 0x7f3e5801abc0 crc :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:18.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.575 7f3e7904d700 1 -- 172.21.2.101:0/3113865381 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e740b4f30 msgr2=0x7f3e7418b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:18.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.575 7f3e7904d700 1 --2- 172.21.2.101:0/3113865381 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e740b4f30 0x7f3e7418b9d0 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 rx=0x7f3e6800b5c0 tx=0x7f3e6800b960).stop 2021-02-10T11:41:18.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.575 7f3e7904d700 1 -- 172.21.2.101:0/3113865381 shutdown_connections 2021-02-10T11:41:18.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.575 7f3e7904d700 1 --2- 172.21.2.101:0/3113865381 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3e580186c0 0x7f3e5801abc0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:18.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.575 7f3e7904d700 1 --2- 172.21.2.101:0/3113865381 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e74093f40 0x7f3e74094340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:18.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.575 7f3e7904d700 1 --2- 172.21.2.101:0/3113865381 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3e74094960 0x7f3e74187e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:18.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.575 7f3e7904d700 1 --2- 172.21.2.101:0/3113865381 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e740b4f30 0x7f3e7418b9d0 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:18.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.575 7f3e7904d700 1 -- 172.21.2.101:0/3113865381 >> 172.21.2.101:0/3113865381 conn(0x7f3e7419f8f0 msgr2=0x7f3e741a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:18.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.575 7f3e7904d700 1 -- 172.21.2.101:0/3113865381 shutdown_connections 2021-02-10T11:41:18.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.575 7f3e7904d700 1 -- 172.21.2.101:0/3113865381 wait complete. 2021-02-10T11:41:18.586 INFO:tasks.cephfs_test_runner:test_default_uid_gid_subvolume_group (tasks.cephfs.test_volumes.TestVolumes) ... ok 2021-02-10T11:41:18.586 INFO:tasks.cephfs_test_runner:Starting test: test_multitenant_subvolumes (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:41:18.587 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_multitenant_subvolumes' 2021-02-10T11:41:19.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.995 7f6fe7333700 1 Processor -- start 2021-02-10T11:41:19.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.995 7f6fe7333700 1 -- start start 2021-02-10T11:41:19.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.996 7f6fe7333700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6fe00d1870 0x7f6fe01b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:19.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.996 7f6fe7333700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe00f0b10 con 0x7f6fe00f0650 2021-02-10T11:41:19.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.996 7f6fe7333700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe00d0310 con 0x7f6fe01b0a10 2021-02-10T11:41:19.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.996 7f6fe7333700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe00d0530 con 0x7f6fe00d1870 2021-02-10T11:41:19.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.996 7f6fe5950700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f6fe00f0650 0x7f6fe00cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57640/0 (socket says 172.21.2.101:57640) 2021-02-10T11:41:19.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.996 7f6fe5950700 1 -- 172.21.2.101:0/1510627851 learned_addr learned my addr 172.21.2.101:0/1510627851 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:19.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.996 7f6fdffff700 1 -- 172.21.2.101:0/1510627851 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1542255180 0 0) 0x7f6fd0000c10 con 0x7f6fe00f0650 2021-02-10T11:41:19.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.996 7f6fdffff700 1 -- 172.21.2.101:0/1510627851 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fc8002fe0 con 0x7f6fe00f0650 2021-02-10T11:41:19.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.997 7f6fe514f700 1 --2- 172.21.2.101:0/1510627851 >> v2:172.21.2.108:3300/0 conn(0x7f6fe00d1870 0x7f6fe01b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:19.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.997 7f6fdffff700 1 -- 172.21.2.101:0/1510627851 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 33318451 0 0) 0x7f6fd0001180 con 0x7f6fe00f0650 2021-02-10T11:41:19.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.997 7f6fdffff700 1 -- 172.21.2.101:0/1510627851 >> v2:172.21.2.108:3300/0 conn(0x7f6fe00d1870 msgr2=0x7f6fe01b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:19.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.997 7f6fdffff700 1 --2- 172.21.2.101:0/1510627851 >> v2:172.21.2.108:3300/0 conn(0x7f6fe00d1870 0x7f6fe01b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:19.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.997 7f6fdffff700 1 -- 172.21.2.101:0/1510627851 >> v1:172.21.2.108:6789/0 conn(0x7f6fe01b0a10 legacy=0x7f6fe00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:19.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.997 7f6fdffff700 1 -- 172.21.2.101:0/1510627851 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe00d1c70 con 0x7f6fe00f0650 2021-02-10T11:41:19.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.997 7f6fdffff700 1 -- 172.21.2.101:0/1510627851 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f6fd0001680 con 0x7f6fe00f0650 2021-02-10T11:41:19.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.997 7f6fdffff700 1 -- 172.21.2.101:0/1510627851 >> v1:172.21.2.101:6789/0 conn(0x7f6fe00f0650 legacy=0x7f6fe00cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:19.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.997 7f6fdffff700 1 --2- 172.21.2.101:0/1510627851 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6fc8003a60 0x7f6fc8005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:19.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.997 7f6fdffff700 1 --2- 172.21.2.101:0/1510627851 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6fc8006780 0x7f6fc8008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:19.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.997 7f6fdffff700 1 --2- 172.21.2.101:0/1510627851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6fc8009580 0x7f6fc800baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:19.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.997 7f6fdffff700 1 -- 172.21.2.101:0/1510627851 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc800c2f0 con 0x7f6fc8009580 2021-02-10T11:41:19.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.997 7f6fdffff700 1 -- 172.21.2.101:0/1510627851 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc800c510 con 0x7f6fc8003a60 2021-02-10T11:41:19.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.997 7f6fdffff700 1 -- 172.21.2.101:0/1510627851 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc800c730 con 0x7f6fc8006780 2021-02-10T11:41:19.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.997 7f6fe5950700 1 --2- 172.21.2.101:0/1510627851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6fc8009580 0x7f6fc800baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:19.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.997 7f6fe494e700 1 --2- 172.21.2.101:0/1510627851 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6fc8006780 0x7f6fc8008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:19.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.997 7f6fe514f700 1 --2- 172.21.2.101:0/1510627851 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6fc8003a60 0x7f6fc8005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:19.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.998 7f6fe5950700 1 -- 172.21.2.101:0/1510627851 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6fc8006780 msgr2=0x7f6fc8008c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:19.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.998 7f6fe5950700 1 --2- 172.21.2.101:0/1510627851 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6fc8006780 0x7f6fc8008c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:19.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.998 7f6fe5950700 1 -- 172.21.2.101:0/1510627851 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6fc8003a60 msgr2=0x7f6fc8005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:19.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.998 7f6fe5950700 1 --2- 172.21.2.101:0/1510627851 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6fc8003a60 0x7f6fc8005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:19.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.998 7f6fe5950700 1 -- 172.21.2.101:0/1510627851 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc80509a0 con 0x7f6fc8009580 2021-02-10T11:41:19.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.998 7f6fe5950700 1 -- 172.21.2.101:0/1510627851 reap_dead start 2021-02-10T11:41:19.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.998 7f6fe5950700 1 --2- 172.21.2.101:0/1510627851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6fc8009580 0x7f6fc800baa0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 rx=0x7f6fd0001880 tx=0x7f6fd0007cf0).ready entity=mon.0 client_cookie=f021de9b3eb77241 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:19.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.998 7f6fdffff700 1 -- 172.21.2.101:0/1510627851 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6fd00070c0 con 0x7f6fc8009580 2021-02-10T11:41:19.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.998 7f6fdffff700 1 -- 172.21.2.101:0/1510627851 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd0001450 con 0x7f6fc8009580 2021-02-10T11:41:19.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.998 7f6fdffff700 1 -- 172.21.2.101:0/1510627851 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6fd0004590 con 0x7f6fc8009580 2021-02-10T11:41:19.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.998 7f6fe7333700 1 -- 172.21.2.101:0/1510627851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6fc8009580 msgr2=0x7f6fc800baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:19.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.998 7f6fe7333700 1 --2- 172.21.2.101:0/1510627851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6fc8009580 0x7f6fc800baa0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 rx=0x7f6fd0001880 tx=0x7f6fd0007cf0).stop 2021-02-10T11:41:19.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.998 7f6fe7333700 1 -- 172.21.2.101:0/1510627851 shutdown_connections 2021-02-10T11:41:19.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.998 7f6fe7333700 1 --2- 172.21.2.101:0/1510627851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6fc8009580 0x7f6fc800baa0 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:19.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.998 7f6fe7333700 1 -- 172.21.2.101:0/1510627851 >> 172.21.2.101:0/1510627851 conn(0x7f6fe00ce450 msgr2=0x7f6fe00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:19.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.998 7f6fe7333700 1 -- 172.21.2.101:0/1510627851 shutdown_connections 2021-02-10T11:41:19.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:18.998 7f6fe7333700 1 -- 172.21.2.101:0/1510627851 wait complete. 2021-02-10T11:41:19.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.013 7f6fe7333700 1 Processor -- start 2021-02-10T11:41:19.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.013 7f6fe7333700 1 -- start start 2021-02-10T11:41:19.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.013 7f6fe7333700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6fe00aab60 0x7f6fe009be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:19.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.013 7f6fe7333700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6fe009c9c0 0x7f6fe009ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:19.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.013 7f6fe7333700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6fe009c460 0x7f6fe009d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:19.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.013 7f6fe7333700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe00c34a0 con 0x7f6fe00aab60 2021-02-10T11:41:19.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.013 7f6fe7333700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe00c36c0 con 0x7f6fe009c9c0 2021-02-10T11:41:19.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.013 7f6fe7333700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe00c38e0 con 0x7f6fe009c460 2021-02-10T11:41:19.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.014 7f6fe5950700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6fe009c460 0x7f6fe009d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:19.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.014 7f6fe5950700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6fe009c460 0x7f6fe009d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:49888/0 (socket says 172.21.2.101:49888) 2021-02-10T11:41:19.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.014 7f6fe5950700 1 -- 172.21.2.101:0/3254709963 learned_addr learned my addr 172.21.2.101:0/3254709963 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:19.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.014 7f6fe514f700 1 --2- 172.21.2.101:0/3254709963 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6fe00aab60 0x7f6fe009be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:19.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.014 7f6fe494e700 1 --2- 172.21.2.101:0/3254709963 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6fe009c9c0 0x7f6fe009ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:19.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.014 7f6fe514f700 1 -- 172.21.2.101:0/3254709963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6fe009c460 msgr2=0x7f6fe009d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:19.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.014 7f6fe514f700 1 --2- 172.21.2.101:0/3254709963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6fe009c460 0x7f6fe009d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:19.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.014 7f6fe514f700 1 -- 172.21.2.101:0/3254709963 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6fe009c9c0 msgr2=0x7f6fe009ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:19.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.014 7f6fe514f700 1 --2- 172.21.2.101:0/3254709963 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6fe009c9c0 0x7f6fe009ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:19.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.014 7f6fe514f700 1 -- 172.21.2.101:0/3254709963 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe00c3b00 con 0x7f6fe00aab60 2021-02-10T11:41:19.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.014 7f6fe5950700 1 --2- 172.21.2.101:0/3254709963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6fe009c460 0x7f6fe009d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:41:19.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.014 7f6fe514f700 1 --2- 172.21.2.101:0/3254709963 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6fe00aab60 0x7f6fe009be40 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 rx=0x7f6fd800b8a0 tx=0x7f6fd800bc40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:19.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.014 7f6fddffb700 1 -- 172.21.2.101:0/3254709963 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6fd800bd80 con 0x7f6fe00aab60 2021-02-10T11:41:19.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.014 7f6fddffb700 1 -- 172.21.2.101:0/3254709963 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd800c2f0 con 0x7f6fe00aab60 2021-02-10T11:41:19.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.014 7f6fe7333700 1 -- 172.21.2.101:0/3254709963 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe00c3b00 con 0x7f6fe00aab60 2021-02-10T11:41:19.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.014 7f6fddffb700 1 -- 172.21.2.101:0/3254709963 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6fd800c740 con 0x7f6fe00aab60 2021-02-10T11:41:19.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.014 7f6fe7333700 1 -- 172.21.2.101:0/3254709963 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe00c3b00 con 0x7f6fe00aab60 2021-02-10T11:41:19.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.015 7f6fddffb700 1 -- 172.21.2.101:0/3254709963 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6fd8001010 con 0x7f6fe00aab60 2021-02-10T11:41:19.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.015 7f6fddffb700 1 --2- 172.21.2.101:0/3254709963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6fc0018730 0x7f6fc001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:19.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.015 7f6fddffb700 1 -- 172.21.2.101:0/3254709963 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6fd80328f0 con 0x7f6fe00aab60 2021-02-10T11:41:19.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.015 7f6fe494e700 1 --2- 172.21.2.101:0/3254709963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6fc0018730 0x7f6fc001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:19.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.015 7f6fe494e700 1 --2- 172.21.2.101:0/3254709963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6fc0018730 0x7f6fc001ac10 crc :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:19.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.046 7f6fe7333700 1 -- 172.21.2.101:0/3254709963 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe00c4380 con 0x7f6fe00aab60 2021-02-10T11:41:19.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.048 7f6fddffb700 1 -- 172.21.2.101:0/3254709963 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6fd8006ac0 con 0x7f6fe00aab60 2021-02-10T11:41:19.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.183 7f6fe7333700 1 -- 172.21.2.101:0/3254709963 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_multitenant_subvolumes"]} v 0) v1 -- 0x7f6fe00c4380 con 0x7f6fe00aab60 2021-02-10T11:41:19.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.572 7f6fddffb700 1 -- 172.21.2.101:0/3254709963 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_multitenant_subvolumes"]}]=0 v582) v1 ==== 147+0+0 (secure 0 0 0) 0x7f6fd8001010 con 0x7f6fe00aab60 2021-02-10T11:41:19.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.600 7f6fe7333700 1 -- 172.21.2.101:0/3254709963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6fc0018730 msgr2=0x7f6fc001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:19.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.600 7f6fe7333700 1 --2- 172.21.2.101:0/3254709963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6fc0018730 0x7f6fc001ac10 crc :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:19.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.600 7f6fe7333700 1 -- 172.21.2.101:0/3254709963 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6fe00aab60 msgr2=0x7f6fe009be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:19.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.600 7f6fe7333700 1 --2- 172.21.2.101:0/3254709963 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6fe00aab60 0x7f6fe009be40 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 rx=0x7f6fd800b8a0 tx=0x7f6fd800bc40).stop 2021-02-10T11:41:19.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.600 7f6fe7333700 1 -- 172.21.2.101:0/3254709963 shutdown_connections 2021-02-10T11:41:19.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.600 7f6fe7333700 1 --2- 172.21.2.101:0/3254709963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6fc0018730 0x7f6fc001ac10 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:19.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.600 7f6fe7333700 1 --2- 172.21.2.101:0/3254709963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6fe009c460 0x7f6fe009d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:19.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.600 7f6fe7333700 1 --2- 172.21.2.101:0/3254709963 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6fe009c9c0 0x7f6fe009ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:19.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.600 7f6fe7333700 1 --2- 172.21.2.101:0/3254709963 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6fe00aab60 0x7f6fe009be40 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:19.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.600 7f6fe7333700 1 -- 172.21.2.101:0/3254709963 >> 172.21.2.101:0/3254709963 conn(0x7f6fe009eba0 msgr2=0x7f6fe00ebe90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:19.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.601 7f6fe7333700 1 -- 172.21.2.101:0/3254709963 shutdown_connections 2021-02-10T11:41:19.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:19.601 7f6fe7333700 1 -- 172.21.2.101:0/3254709963 wait complete. 2021-02-10T11:41:19.611 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:41:20.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.024 7f6e6ba8c700 1 Processor -- start 2021-02-10T11:41:20.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.024 7f6e6ba8c700 1 -- start start 2021-02-10T11:41:20.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.024 7f6e6ba8c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6e640d1c70 0x7f6e640cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:20.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.024 7f6e6ba8c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6e640cf580 0x7f6e640cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:20.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.024 7f6e6ba8c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e640fe1c0 con 0x7f6e640d1870 2021-02-10T11:41:20.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.024 7f6e6ba8c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e641b0cb0 con 0x7f6e640cf580 2021-02-10T11:41:20.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.024 7f6e6ba8c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e64109a60 con 0x7f6e640d1c70 2021-02-10T11:41:20.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.025 7f6e698a8700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f6e640d1870 0x7f6e641b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57658/0 (socket says 172.21.2.101:57658) 2021-02-10T11:41:20.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.025 7f6e698a8700 1 -- 172.21.2.101:0/3599050641 learned_addr learned my addr 172.21.2.101:0/3599050641 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:20.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.025 7f6e6a0a9700 1 --2- 172.21.2.101:0/3599050641 >> v2:172.21.2.101:3300/0 conn(0x7f6e640cf580 0x7f6e640cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:20.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.025 7f6e688a6700 1 -- 172.21.2.101:0/3599050641 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1010307725 0 0) 0x7f6e580009b0 con 0x7f6e640d1870 2021-02-10T11:41:20.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.025 7f6e688a6700 1 -- 172.21.2.101:0/3599050641 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e50002fe0 con 0x7f6e640d1870 2021-02-10T11:41:20.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.025 7f6e6a0a9700 1 -- 172.21.2.101:0/3599050641 >> v2:172.21.2.108:3300/0 conn(0x7f6e640d1c70 msgr2=0x7f6e640cef60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:41:20.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.025 7f6e6a0a9700 1 --2- 172.21.2.101:0/3599050641 >> v2:172.21.2.108:3300/0 conn(0x7f6e640d1c70 0x7f6e640cef60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.025 7f6e6a0a9700 1 -- 172.21.2.101:0/3599050641 >> v1:172.21.2.101:6789/0 conn(0x7f6e640d1870 legacy=0x7f6e641b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.025 7f6e6a0a9700 1 -- 172.21.2.101:0/3599050641 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e64088070 con 0x7f6e640cf580 2021-02-10T11:41:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.025 7f6e6a0a9700 1 --2- 172.21.2.101:0/3599050641 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e640cf580 0x7f6e640cf980 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 rx=0x7f6e5c006e30 tx=0x7f6e5c009de0).ready entity=mon.0 client_cookie=45220fedea0070a9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.025 7f6e688a6700 1 -- 172.21.2.101:0/3599050641 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e5c00b300 con 0x7f6e640cf580 2021-02-10T11:41:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.025 7f6e688a6700 1 -- 172.21.2.101:0/3599050641 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5c000d30 con 0x7f6e640cf580 2021-02-10T11:41:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.025 7f6e688a6700 1 -- 172.21.2.101:0/3599050641 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e5c00b900 con 0x7f6e640cf580 2021-02-10T11:41:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.025 7f6e6ba8c700 1 -- 172.21.2.101:0/3599050641 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e640cf580 msgr2=0x7f6e640cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.025 7f6e6ba8c700 1 --2- 172.21.2.101:0/3599050641 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e640cf580 0x7f6e640cf980 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 rx=0x7f6e5c006e30 tx=0x7f6e5c009de0).stop 2021-02-10T11:41:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.026 7f6e6ba8c700 1 -- 172.21.2.101:0/3599050641 shutdown_connections 2021-02-10T11:41:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.026 7f6e6ba8c700 1 --2- 172.21.2.101:0/3599050641 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e640cf580 0x7f6e640cf980 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.026 7f6e6ba8c700 1 --2- 172.21.2.101:0/3599050641 >> v2:172.21.2.108:3300/0 conn(0x7f6e640d1c70 0x7f6e640cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.026 7f6e6ba8c700 1 -- 172.21.2.101:0/3599050641 >> 172.21.2.101:0/3599050641 conn(0x7f6e640ce420 msgr2=0x7f6e640d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.026 7f6e6ba8c700 1 -- 172.21.2.101:0/3599050641 shutdown_connections 2021-02-10T11:41:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.026 7f6e6ba8c700 1 -- 172.21.2.101:0/3599050641 wait complete. 2021-02-10T11:41:20.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.040 7f6e6ba8c700 1 Processor -- start 2021-02-10T11:41:20.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.041 7f6e6ba8c700 1 -- start start 2021-02-10T11:41:20.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.041 7f6e6ba8c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e640a7950 0x7f6e640b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:20.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.041 7f6e6ba8c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e641a7490 0x7f6e641a7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:20.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.041 7f6e6ba8c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e641a7eb0 0x7f6e640ac630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:20.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.041 7f6e6ba8c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e640a7fb0 con 0x7f6e641a7eb0 2021-02-10T11:41:20.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.041 7f6e6ba8c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e640acc50 con 0x7f6e641a7490 2021-02-10T11:41:20.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.041 7f6e6ba8c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e640ace30 con 0x7f6e640a7950 2021-02-10T11:41:20.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.041 7f6e6a0a9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e641a7eb0 0x7f6e640ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:20.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.041 7f6e6a0a9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e641a7eb0 0x7f6e640ac630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57172/0 (socket says 172.21.2.101:57172) 2021-02-10T11:41:20.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.041 7f6e6a0a9700 1 -- 172.21.2.101:0/1973236753 learned_addr learned my addr 172.21.2.101:0/1973236753 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:20.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.041 7f6e6a0a9700 1 -- 172.21.2.101:0/1973236753 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e640a7950 msgr2=0x7f6e640b8df0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:20.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.041 7f6e6a0a9700 1 --2- 172.21.2.101:0/1973236753 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e640a7950 0x7f6e640b8df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:20.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.041 7f6e6a0a9700 1 -- 172.21.2.101:0/1973236753 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e641a7490 msgr2=0x7f6e641a7890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:20.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.041 7f6e6a0a9700 1 --2- 172.21.2.101:0/1973236753 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e641a7490 0x7f6e641a7890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:20.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.041 7f6e6a0a9700 1 -- 172.21.2.101:0/1973236753 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e640ad050 con 0x7f6e641a7eb0 2021-02-10T11:41:20.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.042 7f6e6a0a9700 1 --2- 172.21.2.101:0/1973236753 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e641a7eb0 0x7f6e640ac630 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 rx=0x7f6e5c00b990 tx=0x7f6e5c00bd30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:20.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.042 7f6e567fc700 1 -- 172.21.2.101:0/1973236753 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e5c009750 con 0x7f6e641a7eb0 2021-02-10T11:41:20.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.042 7f6e567fc700 1 -- 172.21.2.101:0/1973236753 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5c009380 con 0x7f6e641a7eb0 2021-02-10T11:41:20.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.042 7f6e567fc700 1 -- 172.21.2.101:0/1973236753 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e5c00c570 con 0x7f6e641a7eb0 2021-02-10T11:41:20.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.042 7f6e6ba8c700 1 -- 172.21.2.101:0/1973236753 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e640ad050 con 0x7f6e641a7eb0 2021-02-10T11:41:20.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.042 7f6e6ba8c700 1 -- 172.21.2.101:0/1973236753 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e640ad050 con 0x7f6e641a7eb0 2021-02-10T11:41:20.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.043 7f6e567fc700 1 -- 172.21.2.101:0/1973236753 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6e5c009720 con 0x7f6e641a7eb0 2021-02-10T11:41:20.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.043 7f6e567fc700 1 --2- 172.21.2.101:0/1973236753 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e4c0186c0 0x7f6e4c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:20.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.043 7f6e567fc700 1 -- 172.21.2.101:0/1973236753 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6e5c032950 con 0x7f6e641a7eb0 2021-02-10T11:41:20.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.043 7f6e698a8700 1 --2- 172.21.2.101:0/1973236753 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e4c0186c0 0x7f6e4c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:20.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.043 7f6e698a8700 1 --2- 172.21.2.101:0/1973236753 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e4c0186c0 0x7f6e4c01abc0 crc :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:20.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.074 7f6e6ba8c700 1 -- 172.21.2.101:0/1973236753 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e64082a70 con 0x7f6e641a7eb0 2021-02-10T11:41:20.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.077 7f6e567fc700 1 -- 172.21.2.101:0/1973236753 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6e5c003070 con 0x7f6e641a7eb0 2021-02-10T11:41:20.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.229 7f6e6ba8c700 1 -- 172.21.2.101:0/1973236753 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6e64082a70 con 0x7f6e641a7eb0 2021-02-10T11:41:20.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.232 7f6e567fc700 1 -- 172.21.2.101:0/1973236753 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6e5c009720 con 0x7f6e641a7eb0 2021-02-10T11:41:20.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.237 7f6e6ba8c700 1 -- 172.21.2.101:0/1973236753 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e4c0186c0 msgr2=0x7f6e4c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:20.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.237 7f6e6ba8c700 1 --2- 172.21.2.101:0/1973236753 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e4c0186c0 0x7f6e4c01abc0 crc :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:20.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.237 7f6e6ba8c700 1 -- 172.21.2.101:0/1973236753 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e641a7eb0 msgr2=0x7f6e640ac630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:20.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.237 7f6e6ba8c700 1 --2- 172.21.2.101:0/1973236753 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e641a7eb0 0x7f6e640ac630 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 rx=0x7f6e5c00b990 tx=0x7f6e5c00bd30).stop 2021-02-10T11:41:20.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.237 7f6e6ba8c700 1 -- 172.21.2.101:0/1973236753 shutdown_connections 2021-02-10T11:41:20.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.237 7f6e6ba8c700 1 --2- 172.21.2.101:0/1973236753 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e641a7eb0 0x7f6e640ac630 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:20.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.237 7f6e6ba8c700 1 --2- 172.21.2.101:0/1973236753 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e641a7490 0x7f6e641a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:20.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.237 7f6e6ba8c700 1 --2- 172.21.2.101:0/1973236753 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e4c0186c0 0x7f6e4c01abc0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:20.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.237 7f6e6ba8c700 1 --2- 172.21.2.101:0/1973236753 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e640a7950 0x7f6e640b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:20.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.237 7f6e6ba8c700 1 -- 172.21.2.101:0/1973236753 >> 172.21.2.101:0/1973236753 conn(0x7f6e640a02d0 msgr2=0x7f6e6409f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:20.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.237 7f6e6ba8c700 1 -- 172.21.2.101:0/1973236753 shutdown_connections 2021-02-10T11:41:20.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:20.237 7f6e6ba8c700 1 -- 172.21.2.101:0/1973236753 wait complete. 2021-02-10T11:41:20.248 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:41:20.248 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:41:20.593 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:41:20.942 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:41:20.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:20.942 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:41:20.943 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:41:21.279 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2021-02-10T11:41:21.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:41:21.279 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-02-10T11:41:21.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:21.279 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:41:21.279 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:41:21.615 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:41:21.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:41:21.616 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:41:21.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:21.616 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:41:21.616 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:41:21.952 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:41:21.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:41:21.953 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:41:21.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:21.953 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:41:21.953 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:41:22.288 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:41:22.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:41:22.289 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:41:22.289 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json-pretty 2021-02-10T11:41:22.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.697 7f5ceac6b700 1 Processor -- start 2021-02-10T11:41:22.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.698 7f5ceac6b700 1 -- start start 2021-02-10T11:41:22.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.698 7f5ceac6b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ce40d1870 0x7f5ce41b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:22.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.698 7f5ceac6b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce40f0b10 con 0x7f5ce40f0650 2021-02-10T11:41:22.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.698 7f5ceac6b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce40d0310 con 0x7f5ce41b0a10 2021-02-10T11:41:22.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.698 7f5ceac6b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce40e5ed0 con 0x7f5ce40d1870 2021-02-10T11:41:22.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.698 7f5ce9288700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f5ce40f0650 0x7f5ce40cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57672/0 (socket says 172.21.2.101:57672) 2021-02-10T11:41:22.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.698 7f5ce9288700 1 -- 172.21.2.101:0/3940175462 learned_addr learned my addr 172.21.2.101:0/3940175462 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:22.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.698 7f5ce37fe700 1 -- 172.21.2.101:0/3940175462 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 842222277 0 0) 0x7f5cdc001500 con 0x7f5ce40f0650 2021-02-10T11:41:22.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce37fe700 1 -- 172.21.2.101:0/3940175462 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cd0002fe0 con 0x7f5ce40f0650 2021-02-10T11:41:22.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce8a87700 1 --2- 172.21.2.101:0/3940175462 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ce40d1870 0x7f5ce41b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:22.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce37fe700 1 -- 172.21.2.101:0/3940175462 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1692429237 0 0) 0x7f5cdc001500 con 0x7f5ce40f0650 2021-02-10T11:41:22.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce37fe700 1 -- 172.21.2.101:0/3940175462 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ce40d1870 msgr2=0x7f5ce41b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:22.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce37fe700 1 --2- 172.21.2.101:0/3940175462 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ce40d1870 0x7f5ce41b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:22.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce37fe700 1 -- 172.21.2.101:0/3940175462 >> v1:172.21.2.108:6789/0 conn(0x7f5ce41b0a10 legacy=0x7f5ce40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:22.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce37fe700 1 -- 172.21.2.101:0/3940175462 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce40d0530 con 0x7f5ce40f0650 2021-02-10T11:41:22.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce37fe700 1 -- 172.21.2.101:0/3940175462 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f5cdc001a00 con 0x7f5ce40f0650 2021-02-10T11:41:22.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce37fe700 1 -- 172.21.2.101:0/3940175462 >> v1:172.21.2.101:6789/0 conn(0x7f5ce40f0650 legacy=0x7f5ce40cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:22.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce37fe700 1 --2- 172.21.2.101:0/3940175462 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5cd0003b10 0x7f5cd0005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:22.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce37fe700 1 --2- 172.21.2.101:0/3940175462 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5cd0006800 0x7f5cd0008d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:22.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce37fe700 1 --2- 172.21.2.101:0/3940175462 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5cd0009570 0x7f5cd000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:22.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce37fe700 1 -- 172.21.2.101:0/3940175462 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd000c2e0 con 0x7f5cd0006800 2021-02-10T11:41:22.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce37fe700 1 -- 172.21.2.101:0/3940175462 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd000c500 con 0x7f5cd0009570 2021-02-10T11:41:22.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce37fe700 1 -- 172.21.2.101:0/3940175462 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd000c720 con 0x7f5cd0003b10 2021-02-10T11:41:22.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce3fff700 1 --2- 172.21.2.101:0/3940175462 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5cd0006800 0x7f5cd0008d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:22.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce9288700 1 --2- 172.21.2.101:0/3940175462 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5cd0009570 0x7f5cd000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:22.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce3fff700 1 -- 172.21.2.101:0/3940175462 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5cd0003b10 msgr2=0x7f5cd0005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:22.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce3fff700 1 --2- 172.21.2.101:0/3940175462 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5cd0003b10 0x7f5cd0005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:22.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce3fff700 1 -- 172.21.2.101:0/3940175462 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5cd0009570 msgr2=0x7f5cd000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:22.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce3fff700 1 --2- 172.21.2.101:0/3940175462 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5cd0009570 0x7f5cd000ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:22.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce3fff700 1 -- 172.21.2.101:0/3940175462 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd0050960 con 0x7f5cd0006800 2021-02-10T11:41:22.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.699 7f5ce3fff700 1 --2- 172.21.2.101:0/3940175462 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5cd0006800 0x7f5cd0008d20 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 rx=0x7f5cd4001040 tx=0x7f5cd4009af0).ready entity=mon.0 client_cookie=a51441629553047e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:22.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.700 7f5ce37fe700 1 -- 172.21.2.101:0/3940175462 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5cd400c0c0 con 0x7f5cd0006800 2021-02-10T11:41:22.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.700 7f5ce37fe700 1 -- 172.21.2.101:0/3940175462 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd400c440 con 0x7f5cd0006800 2021-02-10T11:41:22.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.700 7f5ce37fe700 1 -- 172.21.2.101:0/3940175462 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5cd400c920 con 0x7f5cd0006800 2021-02-10T11:41:22.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.700 7f5ceac6b700 1 -- 172.21.2.101:0/3940175462 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5cd0006800 msgr2=0x7f5cd0008d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:22.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.700 7f5ceac6b700 1 --2- 172.21.2.101:0/3940175462 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5cd0006800 0x7f5cd0008d20 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 rx=0x7f5cd4001040 tx=0x7f5cd4009af0).stop 2021-02-10T11:41:22.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.700 7f5ce9288700 1 -- 172.21.2.101:0/3940175462 reap_dead start 2021-02-10T11:41:22.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.700 7f5ceac6b700 1 -- 172.21.2.101:0/3940175462 shutdown_connections 2021-02-10T11:41:22.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.700 7f5ceac6b700 1 -- 172.21.2.101:0/3940175462 >> 172.21.2.101:0/3940175462 conn(0x7f5ce40ce450 msgr2=0x7f5ce40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:22.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.700 7f5ceac6b700 1 -- 172.21.2.101:0/3940175462 shutdown_connections 2021-02-10T11:41:22.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.700 7f5ceac6b700 1 -- 172.21.2.101:0/3940175462 wait complete. 2021-02-10T11:41:22.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.715 7f5ceac6b700 1 Processor -- start 2021-02-10T11:41:22.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.715 7f5ceac6b700 1 -- start start 2021-02-10T11:41:22.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.716 7f5ceac6b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ce41a05d0 0x7f5ce40a87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:22.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.716 7f5ceac6b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ce40a8de0 0x7f5ce40a2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:22.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.716 7f5ceac6b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ce40a2c50 0x7f5ce40a3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:22.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.716 7f5ceac6b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce40b32d0 con 0x7f5ce40a2c50 2021-02-10T11:41:22.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.716 7f5ceac6b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce40a91e0 con 0x7f5ce41a05d0 2021-02-10T11:41:22.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.716 7f5ceac6b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce40b37d0 con 0x7f5ce40a8de0 2021-02-10T11:41:22.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.716 7f5ce9288700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ce40a2c50 0x7f5ce40a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:22.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.716 7f5ce9288700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ce40a2c50 0x7f5ce40a3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57190/0 (socket says 172.21.2.101:57190) 2021-02-10T11:41:22.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.716 7f5ce9288700 1 -- 172.21.2.101:0/170911535 learned_addr learned my addr 172.21.2.101:0/170911535 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:22.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.716 7f5ce8a87700 1 --2- 172.21.2.101:0/170911535 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ce41a05d0 0x7f5ce40a87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:22.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.716 7f5ce3fff700 1 --2- 172.21.2.101:0/170911535 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ce40a8de0 0x7f5ce40a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:22.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.716 7f5ce9288700 1 -- 172.21.2.101:0/170911535 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ce40a8de0 msgr2=0x7f5ce40a2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:22.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.716 7f5ce9288700 1 --2- 172.21.2.101:0/170911535 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ce40a8de0 0x7f5ce40a2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:22.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.716 7f5ce9288700 1 -- 172.21.2.101:0/170911535 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ce41a05d0 msgr2=0x7f5ce40a87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:22.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.716 7f5ce9288700 1 --2- 172.21.2.101:0/170911535 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ce41a05d0 0x7f5ce40a87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:22.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.716 7f5ce9288700 1 -- 172.21.2.101:0/170911535 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce40b39b0 con 0x7f5ce40a2c50 2021-02-10T11:41:22.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.717 7f5ce9288700 1 --2- 172.21.2.101:0/170911535 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ce40a2c50 0x7f5ce40a3050 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 rx=0x7f5cdc0069f0 tx=0x7f5cdc00bc40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:22.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.717 7f5ce17fa700 1 -- 172.21.2.101:0/170911535 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5cdc0078f0 con 0x7f5ce40a2c50 2021-02-10T11:41:22.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.717 7f5ce17fa700 1 -- 172.21.2.101:0/170911535 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cdc007bb0 con 0x7f5ce40a2c50 2021-02-10T11:41:22.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.717 7f5ce17fa700 1 -- 172.21.2.101:0/170911535 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5cdc0062b0 con 0x7f5ce40a2c50 2021-02-10T11:41:22.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.717 7f5ceac6b700 1 -- 172.21.2.101:0/170911535 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce40b39b0 con 0x7f5ce40a2c50 2021-02-10T11:41:22.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.717 7f5ceac6b700 1 -- 172.21.2.101:0/170911535 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce40b3f40 con 0x7f5ce40a2c50 2021-02-10T11:41:22.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.718 7f5ce17fa700 1 -- 172.21.2.101:0/170911535 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5cdc006240 con 0x7f5ce40a2c50 2021-02-10T11:41:22.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.718 7f5ce17fa700 1 --2- 172.21.2.101:0/170911535 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5cc80186c0 0x7f5cc801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:22.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.718 7f5ce17fa700 1 -- 172.21.2.101:0/170911535 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5cdc032920 con 0x7f5ce40a2c50 2021-02-10T11:41:22.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.718 7f5ce8a87700 1 --2- 172.21.2.101:0/170911535 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5cc80186c0 0x7f5cc801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:22.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.718 7f5ce8a87700 1 --2- 172.21.2.101:0/170911535 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5cc80186c0 0x7f5cc801abc0 crc :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:22.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.748 7f5ceac6b700 1 -- 172.21.2.101:0/170911535 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce40ebc80 con 0x7f5ce40a2c50 2021-02-10T11:41:22.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.750 7f5ce17fa700 1 -- 172.21.2.101:0/170911535 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5cdc007a90 con 0x7f5ce40a2c50 2021-02-10T11:41:22.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.875 7f5ceac6b700 1 -- 172.21.2.101:0/170911535 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5ce40f0b10 con 0x7f5ce40a2c50 2021-02-10T11:41:22.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.875 7f5ce17fa700 1 -- 172.21.2.101:0/170911535 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v112) v1 ==== 81+0+30374 (secure 0 0 0) 0x7f5cdc007a90 con 0x7f5ce40a2c50 2021-02-10T11:41:22.879 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:41:22.879 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:41:22.879 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 112, 2021-02-10T11:41:22.879 INFO:teuthology.orchestra.run.gibba001.stdout: "fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:41:22.879 INFO:teuthology.orchestra.run.gibba001.stdout: "created": "2021-02-10 11:12:36.788366", 2021-02-10T11:41:22.880 INFO:teuthology.orchestra.run.gibba001.stdout: "modified": "2021-02-10 11:40:46.730463", 2021-02-10T11:41:22.880 INFO:teuthology.orchestra.run.gibba001.stdout: "last_up_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:41:22.880 INFO:teuthology.orchestra.run.gibba001.stdout: "last_in_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:41:22.880 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-02-10T11:41:22.880 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_num": 5799936, 2021-02-10T11:41:22.880 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_set": [ 2021-02-10T11:41:22.881 INFO:teuthology.orchestra.run.gibba001.stdout: "pglog_hardlimit", 2021-02-10T11:41:22.881 INFO:teuthology.orchestra.run.gibba001.stdout: "purged_snapdirs", 2021-02-10T11:41:22.881 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_deletes", 2021-02-10T11:41:22.881 INFO:teuthology.orchestra.run.gibba001.stdout: "sortbitwise" 2021-02-10T11:41:22.881 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:41:22.881 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_version": 4, 2021-02-10T11:41:22.881 INFO:teuthology.orchestra.run.gibba001.stdout: "full_ratio": 0.94999998807907104, 2021-02-10T11:41:22.881 INFO:teuthology.orchestra.run.gibba001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-02-10T11:41:22.881 INFO:teuthology.orchestra.run.gibba001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-02-10T11:41:22.881 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_snapshot": "", 2021-02-10T11:41:22.882 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_max": 23, 2021-02-10T11:41:22.882 INFO:teuthology.orchestra.run.gibba001.stdout: "max_osd": 8, 2021-02-10T11:41:22.882 INFO:teuthology.orchestra.run.gibba001.stdout: "require_min_compat_client": "jewel", 2021-02-10T11:41:22.882 INFO:teuthology.orchestra.run.gibba001.stdout: "min_compat_client": "jewel", 2021-02-10T11:41:22.882 INFO:teuthology.orchestra.run.gibba001.stdout: "require_osd_release": "nautilus", 2021-02-10T11:41:22.882 INFO:teuthology.orchestra.run.gibba001.stdout: "pools": [ 2021-02-10T11:41:22.882 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.882 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 1, 2021-02-10T11:41:22.882 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "rbd", 2021-02-10T11:41:22.882 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:13:08.037472", 2021-02-10T11:41:22.882 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:41:22.882 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:41:22.883 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:41:22.883 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:41:22.883 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:41:22.883 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:41:22.883 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:41:22.883 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:41:22.883 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:41:22.883 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:41:22.883 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:41:22.883 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:41:22.883 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:41:22.883 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:41:22.883 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:41:22.883 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:41:22.884 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:41:22.884 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:41:22.884 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:41:22.884 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:41:22.884 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.884 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "16", 2021-02-10T11:41:22.884 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:41:22.884 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:41:22.885 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:41:22.885 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:41:22.885 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:41:22.885 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:41:22.885 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:41:22.885 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:41:22.885 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:41:22.885 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:41:22.885 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:41:22.885 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:41:22.886 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:41:22.886 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:41:22.886 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:41:22.886 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:41:22.886 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:41:22.886 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:41:22.886 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:41:22.886 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:41:22.886 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:41:22.886 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:41:22.886 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:41:22.887 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:41:22.887 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:41:22.887 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:41:22.887 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.887 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:41:22.887 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:41:22.887 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:41:22.887 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:41:22.887 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:41:22.887 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:41:22.887 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:41:22.887 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:41:22.887 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:41:22.888 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:41:22.888 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:41:22.888 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:41:22.888 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:41:22.888 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd": {} 2021-02-10T11:41:22.888 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.888 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.888 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.889 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 22, 2021-02-10T11:41:22.889 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_metadata", 2021-02-10T11:41:22.889 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:40:43.698421", 2021-02-10T11:41:22.889 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:41:22.889 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:41:22.889 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:41:22.889 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:41:22.889 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:41:22.889 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:41:22.889 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:41:22.889 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:41:22.889 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:41:22.890 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:41:22.890 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:41:22.890 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:41:22.890 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:41:22.890 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:41:22.890 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:41:22.890 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:41:22.890 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:41:22.890 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:41:22.890 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:41:22.891 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:41:22.891 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.891 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "111", 2021-02-10T11:41:22.891 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:41:22.891 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:41:22.891 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:41:22.891 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:41:22.891 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:41:22.891 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:41:22.891 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:41:22.891 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:41:22.891 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:41:22.891 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:41:22.892 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:41:22.892 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:41:22.892 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:41:22.892 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:41:22.892 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:41:22.892 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:41:22.892 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:41:22.892 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:41:22.892 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:41:22.892 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:41:22.892 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:41:22.892 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:41:22.892 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:41:22.893 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:41:22.893 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:41:22.893 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:41:22.893 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.893 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:41:22.893 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:41:22.893 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:41:22.893 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:41:22.893 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:41:22.893 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:41:22.893 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:41:22.893 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:41:22.893 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:41:22.894 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:41:22.894 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:41:22.894 INFO:teuthology.orchestra.run.gibba001.stdout: "options": { 2021-02-10T11:41:22.894 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_bias": 4, 2021-02-10T11:41:22.894 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_min": 16, 2021-02-10T11:41:22.894 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_priority": 5 2021-02-10T11:41:22.894 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.894 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:41:22.894 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:41:22.894 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": "cephfs" 2021-02-10T11:41:22.895 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.895 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.895 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.895 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.895 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 23, 2021-02-10T11:41:22.895 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_data", 2021-02-10T11:41:22.895 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:40:44.923352", 2021-02-10T11:41:22.895 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:41:22.896 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:41:22.896 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:41:22.896 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:41:22.896 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:41:22.896 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:41:22.896 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:41:22.896 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:41:22.897 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:41:22.897 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:41:22.897 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:41:22.897 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:41:22.897 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:41:22.897 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:41:22.897 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:41:22.897 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:41:22.898 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:41:22.898 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:41:22.898 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:41:22.898 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:41:22.898 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.898 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "112", 2021-02-10T11:41:22.898 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:41:22.898 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:41:22.899 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:41:22.899 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:41:22.899 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:41:22.899 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:41:22.899 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:41:22.899 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:41:22.899 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:41:22.900 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:41:22.900 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:41:22.900 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:41:22.900 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:41:22.900 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:41:22.900 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:41:22.900 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:41:22.901 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:41:22.901 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:41:22.901 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:41:22.901 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:41:22.901 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:41:22.901 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:41:22.901 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:41:22.901 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:41:22.902 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:41:22.902 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:41:22.902 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.902 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:41:22.902 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:41:22.902 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:41:22.902 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:41:22.902 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:41:22.903 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:41:22.903 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:41:22.903 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:41:22.903 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:41:22.903 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:41:22.903 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:41:22.903 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:41:22.903 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:41:22.904 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:41:22.904 INFO:teuthology.orchestra.run.gibba001.stdout: "data": "cephfs" 2021-02-10T11:41:22.904 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.904 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.904 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.904 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:41:22.904 INFO:teuthology.orchestra.run.gibba001.stdout: "osds": [ 2021-02-10T11:41:22.905 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.905 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:41:22.905 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5", 2021-02-10T11:41:22.905 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:41:22.905 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:41:22.905 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:41:22.905 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:41:22.905 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:41:22.905 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:41:22.906 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:41:22.906 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 110, 2021-02-10T11:41:22.906 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:41:22.906 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:41:22.906 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:41:22.906 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.906 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.906 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.906 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6802", 2021-02-10T11:41:22.906 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:41:22.907 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.907 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.907 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.907 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6805", 2021-02-10T11:41:22.907 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:41:22.907 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.907 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.907 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.907 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:41:22.907 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.907 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.907 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.907 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6807", 2021-02-10T11:41:22.908 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:41:22.908 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.908 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.908 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.908 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6809", 2021-02-10T11:41:22.908 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:41:22.908 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.908 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.908 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.908 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:41:22.908 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.909 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.909 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.909 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6817", 2021-02-10T11:41:22.909 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:41:22.909 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.909 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.909 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.909 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6820", 2021-02-10T11:41:22.909 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:41:22.909 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.909 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.909 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.909 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:41:22.910 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.910 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.910 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.910 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6812", 2021-02-10T11:41:22.910 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:41:22.910 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.910 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.910 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.910 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6814", 2021-02-10T11:41:22.911 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:41:22.911 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.911 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.911 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.911 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6805/25093", 2021-02-10T11:41:22.911 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6809/25093", 2021-02-10T11:41:22.911 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6820/25093", 2021-02-10T11:41:22.911 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6814/25093", 2021-02-10T11:41:22.911 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:41:22.911 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:41:22.911 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:41:22.911 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.912 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.912 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.912 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:41:22.912 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "ddc04ce3-b313-4707-8270-585391e38ecc", 2021-02-10T11:41:22.912 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:41:22.912 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:41:22.912 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:41:22.912 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:41:22.912 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:41:22.912 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:41:22.912 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:41:22.912 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 109, 2021-02-10T11:41:22.912 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:41:22.913 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:41:22.913 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:41:22.913 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.913 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.913 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.913 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6803", 2021-02-10T11:41:22.913 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:41:22.913 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.913 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.913 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.913 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6804", 2021-02-10T11:41:22.913 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:41:22.913 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.914 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.914 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.914 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:41:22.914 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.914 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.914 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.914 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6806", 2021-02-10T11:41:22.914 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:41:22.915 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.915 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.915 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.915 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6808", 2021-02-10T11:41:22.915 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:41:22.915 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.915 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.915 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.915 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:41:22.915 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.915 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.915 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.916 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6818", 2021-02-10T11:41:22.916 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:41:22.916 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.916 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.916 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.916 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6821", 2021-02-10T11:41:22.916 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:41:22.916 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.916 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.916 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.916 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:41:22.916 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.916 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.917 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.917 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6810", 2021-02-10T11:41:22.917 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:41:22.917 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.917 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.917 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.917 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6815", 2021-02-10T11:41:22.917 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:41:22.917 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.917 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.917 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.917 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6804/25092", 2021-02-10T11:41:22.917 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6808/25092", 2021-02-10T11:41:22.918 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6821/25092", 2021-02-10T11:41:22.918 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6815/25092", 2021-02-10T11:41:22.918 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:41:22.918 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:41:22.918 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:41:22.918 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.918 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.918 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.918 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:41:22.918 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3", 2021-02-10T11:41:22.918 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:41:22.919 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:41:22.919 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:41:22.919 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:41:22.919 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:41:22.919 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:41:22.919 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:41:22.919 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 110, 2021-02-10T11:41:22.919 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:41:22.919 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:41:22.919 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:41:22.919 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.919 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.919 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.920 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6811", 2021-02-10T11:41:22.920 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:41:22.920 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.920 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.920 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.920 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6813", 2021-02-10T11:41:22.920 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:41:22.920 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.920 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.920 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.920 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:41:22.920 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.920 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.921 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.921 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6816", 2021-02-10T11:41:22.921 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:41:22.921 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.921 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.921 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.921 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6819", 2021-02-10T11:41:22.921 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:41:22.921 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.921 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.922 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.922 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:41:22.922 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.922 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.922 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.922 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6825", 2021-02-10T11:41:22.922 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:41:22.922 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.922 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.922 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.922 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6827", 2021-02-10T11:41:22.922 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:41:22.923 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.923 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.923 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.923 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:41:22.923 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.923 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.923 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.923 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6822", 2021-02-10T11:41:22.923 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:41:22.923 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.923 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.923 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.923 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6823", 2021-02-10T11:41:22.923 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:41:22.924 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.924 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.924 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.924 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6813/25094", 2021-02-10T11:41:22.924 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6819/25094", 2021-02-10T11:41:22.924 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6827/25094", 2021-02-10T11:41:22.924 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6823/25094", 2021-02-10T11:41:22.924 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:41:22.924 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:41:22.924 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:41:22.924 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.924 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.925 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.925 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:41:22.925 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "0d9ef9b1-98aa-4d37-aa16-6305c7d46d52", 2021-02-10T11:41:22.925 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:41:22.925 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:41:22.925 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:41:22.925 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:41:22.925 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:41:22.925 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:41:22.925 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:41:22.925 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 110, 2021-02-10T11:41:22.925 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:41:22.925 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:41:22.926 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:41:22.926 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.926 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.926 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.926 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6824", 2021-02-10T11:41:22.926 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:41:22.926 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.926 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.926 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.926 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6826", 2021-02-10T11:41:22.926 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:41:22.927 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.927 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.927 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.927 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:41:22.927 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.927 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.927 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.927 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6828", 2021-02-10T11:41:22.927 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:41:22.927 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.927 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.927 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.927 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6829", 2021-02-10T11:41:22.928 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:41:22.928 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.928 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.928 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.928 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:41:22.928 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.928 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.928 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.928 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6832", 2021-02-10T11:41:22.928 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:41:22.928 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.928 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.928 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.929 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6833", 2021-02-10T11:41:22.929 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:41:22.929 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.929 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.929 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.929 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:41:22.929 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.929 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.929 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.929 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6830", 2021-02-10T11:41:22.929 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:41:22.929 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.929 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.930 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.930 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6831", 2021-02-10T11:41:22.930 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:41:22.930 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.930 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.930 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.930 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6826/25095", 2021-02-10T11:41:22.930 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6829/25095", 2021-02-10T11:41:22.930 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6833/25095", 2021-02-10T11:41:22.930 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6831/25095", 2021-02-10T11:41:22.930 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:41:22.930 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:41:22.931 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:41:22.931 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.931 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.931 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.931 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:41:22.931 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "36742c21-bfeb-4e11-bce9-8e745378b605", 2021-02-10T11:41:22.931 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:41:22.931 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:41:22.931 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:41:22.931 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:41:22.932 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:41:22.932 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:41:22.932 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:41:22.932 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 110, 2021-02-10T11:41:22.932 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:41:22.932 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:41:22.932 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:41:22.932 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.932 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.932 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.932 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6816", 2021-02-10T11:41:22.932 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:41:22.932 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.932 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.933 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.933 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6818", 2021-02-10T11:41:22.933 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:41:22.933 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.933 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.933 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.933 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:41:22.933 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.933 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.933 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.933 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6820", 2021-02-10T11:41:22.933 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:41:22.934 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.934 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.934 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.934 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6822", 2021-02-10T11:41:22.934 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:41:22.934 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.934 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.934 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.934 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:41:22.934 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.934 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.934 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.935 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6828", 2021-02-10T11:41:22.935 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:41:22.935 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.935 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.935 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.935 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6830", 2021-02-10T11:41:22.935 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:41:22.935 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.935 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.935 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.935 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:41:22.935 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.936 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.936 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.936 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6824", 2021-02-10T11:41:22.936 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:41:22.936 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.936 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.936 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.936 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6826", 2021-02-10T11:41:22.936 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:41:22.936 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.936 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.936 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.937 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6818/36380", 2021-02-10T11:41:22.937 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6822/36380", 2021-02-10T11:41:22.937 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6830/36380", 2021-02-10T11:41:22.937 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6826/36380", 2021-02-10T11:41:22.937 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:41:22.937 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:41:22.937 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:41:22.937 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.937 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.937 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.937 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:41:22.937 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "88e694e8-fe98-4d1b-88c8-8c9907ece60b", 2021-02-10T11:41:22.937 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:41:22.937 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:41:22.938 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:41:22.938 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:41:22.938 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:41:22.938 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:41:22.938 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:41:22.938 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 110, 2021-02-10T11:41:22.938 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:41:22.938 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:41:22.939 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:41:22.939 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.939 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.939 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.939 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6808", 2021-02-10T11:41:22.939 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:41:22.939 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.939 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.940 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.940 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6809", 2021-02-10T11:41:22.940 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:41:22.940 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.940 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.940 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.940 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:41:22.940 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.940 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.940 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.940 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6810", 2021-02-10T11:41:22.940 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:41:22.940 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.941 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.941 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.941 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6811", 2021-02-10T11:41:22.941 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:41:22.941 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.941 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.941 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.941 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:41:22.941 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.941 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.941 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.942 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6814", 2021-02-10T11:41:22.942 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:41:22.942 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.942 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.942 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.942 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6815", 2021-02-10T11:41:22.942 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:41:22.942 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.942 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.942 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.943 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:41:22.943 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.943 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.943 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.943 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6812", 2021-02-10T11:41:22.943 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:41:22.943 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.943 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.943 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.943 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6813", 2021-02-10T11:41:22.943 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:41:22.943 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.943 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.944 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.944 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6809/36383", 2021-02-10T11:41:22.944 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6811/36383", 2021-02-10T11:41:22.944 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6815/36383", 2021-02-10T11:41:22.944 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6813/36383", 2021-02-10T11:41:22.944 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:41:22.944 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:41:22.944 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:41:22.944 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.944 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.944 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.944 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:41:22.944 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "87a13c6c-cf85-4aaf-916b-e73913484a49", 2021-02-10T11:41:22.945 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:41:22.945 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:41:22.945 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:41:22.945 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:41:22.945 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:41:22.945 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:41:22.945 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:41:22.945 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 108, 2021-02-10T11:41:22.945 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:41:22.945 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:41:22.945 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:41:22.945 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.945 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.946 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.946 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6800", 2021-02-10T11:41:22.946 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:41:22.946 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.946 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.946 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.946 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6801", 2021-02-10T11:41:22.946 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:41:22.946 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.946 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.946 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.946 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:41:22.946 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.947 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.947 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.947 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6802", 2021-02-10T11:41:22.947 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:41:22.947 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.947 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.947 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.947 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6803", 2021-02-10T11:41:22.947 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:41:22.947 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.947 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.947 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.947 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:41:22.948 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.948 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.948 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.948 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6806", 2021-02-10T11:41:22.948 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:41:22.948 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.948 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.948 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.948 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6807", 2021-02-10T11:41:22.948 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:41:22.948 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.948 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.948 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.949 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:41:22.949 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.949 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.949 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.949 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6804", 2021-02-10T11:41:22.949 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:41:22.949 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.949 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.949 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.949 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6805", 2021-02-10T11:41:22.949 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:41:22.949 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.950 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.950 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.950 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6801/36381", 2021-02-10T11:41:22.950 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6803/36381", 2021-02-10T11:41:22.950 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6807/36381", 2021-02-10T11:41:22.950 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6805/36381", 2021-02-10T11:41:22.950 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:41:22.950 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:41:22.950 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:41:22.950 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.950 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.950 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.950 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:41:22.951 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "7b90823f-854c-4592-a95b-a7fa98f93254", 2021-02-10T11:41:22.951 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:41:22.951 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:41:22.951 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:41:22.951 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:41:22.951 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:41:22.951 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:41:22.951 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:41:22.951 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 101, 2021-02-10T11:41:22.951 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:41:22.951 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:41:22.951 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:41:22.951 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.952 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.952 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.952 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6817", 2021-02-10T11:41:22.952 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:41:22.952 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.952 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.952 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.952 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6819", 2021-02-10T11:41:22.952 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:41:22.953 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.953 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.953 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.953 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:41:22.953 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.953 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.953 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.953 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6821", 2021-02-10T11:41:22.953 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:41:22.953 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.953 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.954 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.954 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6823", 2021-02-10T11:41:22.954 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:41:22.954 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.954 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.954 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.954 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:41:22.954 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.954 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.954 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.954 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6829", 2021-02-10T11:41:22.954 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:41:22.954 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.955 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.955 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.955 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6831", 2021-02-10T11:41:22.955 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:41:22.955 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.955 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.955 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.955 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:41:22.955 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:41:22.955 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.955 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:41:22.955 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6825", 2021-02-10T11:41:22.955 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:41:22.955 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.956 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.956 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:22.956 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6827", 2021-02-10T11:41:22.956 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:41:22.956 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.956 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.956 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.956 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6819/36382", 2021-02-10T11:41:22.956 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6823/36382", 2021-02-10T11:41:22.956 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6831/36382", 2021-02-10T11:41:22.956 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6827/36382", 2021-02-10T11:41:22.956 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:41:22.957 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:41:22.957 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:41:22.957 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:22.957 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.957 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:41:22.957 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_xinfo": [ 2021-02-10T11:41:22.957 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.957 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:41:22.957 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:41:22.957 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:41:22.957 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:41:22.957 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:41:22.958 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:41:22.958 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.958 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.958 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:41:22.959 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:41:22.959 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:41:22.959 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:41:22.959 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:41:22.959 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:41:22.959 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.959 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.959 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:41:22.959 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:41:22.959 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:41:22.959 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:41:22.959 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:41:22.959 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:41:22.960 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.960 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.960 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:41:22.960 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:41:22.960 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:41:22.960 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:41:22.960 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:41:22.960 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:41:22.960 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.960 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.960 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:41:22.960 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:41:22.961 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:41:22.961 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:41:22.961 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:41:22.961 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:41:22.961 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.961 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.961 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:41:22.961 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:41:22.961 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:41:22.962 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:41:22.962 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:41:22.962 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:41:22.962 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.962 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.962 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:41:22.962 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:41:22.962 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:41:22.962 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:41:22.963 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:41:22.963 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:41:22.963 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.963 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:22.963 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:41:22.963 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:41:22.963 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:41:22.963 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:41:22.963 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:41:22.964 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:41:22.964 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.964 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:41:22.964 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap": [], 2021-02-10T11:41:22.964 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap_items": [], 2021-02-10T11:41:22.964 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_temp": [], 2021-02-10T11:41:22.964 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_temp": [], 2021-02-10T11:41:22.964 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklist": {}, 2021-02-10T11:41:22.964 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profiles": { 2021-02-10T11:41:22.964 INFO:teuthology.orchestra.run.gibba001.stdout: "default": { 2021-02-10T11:41:22.964 INFO:teuthology.orchestra.run.gibba001.stdout: "crush-failure-domain": "osd", 2021-02-10T11:41:22.964 INFO:teuthology.orchestra.run.gibba001.stdout: "k": "2", 2021-02-10T11:41:22.965 INFO:teuthology.orchestra.run.gibba001.stdout: "m": "1", 2021-02-10T11:41:22.965 INFO:teuthology.orchestra.run.gibba001.stdout: "plugin": "jerasure", 2021-02-10T11:41:22.965 INFO:teuthology.orchestra.run.gibba001.stdout: "ruleset-failure-domain": "osd", 2021-02-10T11:41:22.965 INFO:teuthology.orchestra.run.gibba001.stdout: "technique": "reed_sol_van" 2021-02-10T11:41:22.965 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:22.965 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:22.965 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps_queue": [], 2021-02-10T11:41:22.966 INFO:teuthology.orchestra.run.gibba001.stdout: "new_removed_snaps": [], 2021-02-10T11:41:22.966 INFO:teuthology.orchestra.run.gibba001.stdout: "new_purged_snaps": [], 2021-02-10T11:41:22.966 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_node_flags": {}, 2021-02-10T11:41:22.966 INFO:teuthology.orchestra.run.gibba001.stdout: "device_class_flags": {} 2021-02-10T11:41:22.966 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:41:22.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.877 7f5ceac6b700 1 -- 172.21.2.101:0/170911535 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5cc80186c0 msgr2=0x7f5cc801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:22.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.877 7f5ceac6b700 1 --2- 172.21.2.101:0/170911535 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5cc80186c0 0x7f5cc801abc0 crc :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:22.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.877 7f5ceac6b700 1 -- 172.21.2.101:0/170911535 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ce40a2c50 msgr2=0x7f5ce40a3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:22.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.877 7f5ceac6b700 1 --2- 172.21.2.101:0/170911535 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ce40a2c50 0x7f5ce40a3050 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 rx=0x7f5cdc0069f0 tx=0x7f5cdc00bc40).stop 2021-02-10T11:41:22.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.877 7f5ceac6b700 1 -- 172.21.2.101:0/170911535 shutdown_connections 2021-02-10T11:41:22.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.877 7f5ceac6b700 1 --2- 172.21.2.101:0/170911535 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5cc80186c0 0x7f5cc801abc0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:22.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.877 7f5ceac6b700 1 --2- 172.21.2.101:0/170911535 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ce40a8de0 0x7f5ce40a2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:22.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.877 7f5ceac6b700 1 --2- 172.21.2.101:0/170911535 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ce40a2c50 0x7f5ce40a3050 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:22.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.877 7f5ceac6b700 1 --2- 172.21.2.101:0/170911535 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ce41a05d0 0x7f5ce40a87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:22.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.877 7f5ceac6b700 1 -- 172.21.2.101:0/170911535 >> 172.21.2.101:0/170911535 conn(0x7f5ce40a9bf0 msgr2=0x7f5ce40ebeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:22.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.877 7f5ceac6b700 1 -- 172.21.2.101:0/170911535 shutdown_connections 2021-02-10T11:41:22.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:22.877 7f5ceac6b700 1 -- 172.21.2.101:0/170911535 wait complete. 2021-02-10T11:41:22.969 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:41:23.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.377 7f1848f47700 1 Processor -- start 2021-02-10T11:41:23.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.377 7f1848f47700 1 -- start start 2021-02-10T11:41:23.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.378 7f1848f47700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f18441b0470 0x7f18441b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:23.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.378 7f1848f47700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18440ef610 0x7f18440efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:23.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.378 7f1848f47700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f18440f0030 0x7f18440f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:23.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.378 7f1848f47700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18440e5ea0 con 0x7f18440f0030 2021-02-10T11:41:23.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.378 7f1848f47700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18440f0b50 con 0x7f18441b0470 2021-02-10T11:41:23.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.378 7f1848f47700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18440e52e0 con 0x7f18440ef610 2021-02-10T11:41:23.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.378 7f1843314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f18440f0030 0x7f18440f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:23.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.378 7f1843314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f18440f0030 0x7f18440f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57194/0 (socket says 172.21.2.101:57194) 2021-02-10T11:41:23.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.378 7f1843314700 1 -- 172.21.2.101:0/220976310 learned_addr learned my addr 172.21.2.101:0/220976310 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:23.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.378 7f1843314700 1 -- 172.21.2.101:0/220976310 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18440ef610 msgr2=0x7f18440efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:41:23.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.378 7f1843314700 1 --2- 172.21.2.101:0/220976310 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18440ef610 0x7f18440efa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:23.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.378 7f1843314700 1 -- 172.21.2.101:0/220976310 >> v2:172.21.2.108:3300/0 conn(0x7f18441b0470 msgr2=0x7f18441b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:41:23.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.378 7f1843314700 1 --2- 172.21.2.101:0/220976310 >> v2:172.21.2.108:3300/0 conn(0x7f18441b0470 0x7f18441b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:23.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.378 7f1843314700 1 -- 172.21.2.101:0/220976310 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18440f10d0 con 0x7f18440f0030 2021-02-10T11:41:23.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.379 7f1843314700 1 --2- 172.21.2.101:0/220976310 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18440f0030 0x7f18440f0430 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 rx=0x7f183c006f00 tx=0x7f183c009f10).ready entity=mon.0 client_cookie=bc4a8d7413f24f82 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:23.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.379 7f1841b11700 1 -- 172.21.2.101:0/220976310 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f183c00b3b0 con 0x7f18440f0030 2021-02-10T11:41:23.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.379 7f1841b11700 1 -- 172.21.2.101:0/220976310 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183c00b730 con 0x7f18440f0030 2021-02-10T11:41:23.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.379 7f1841b11700 1 -- 172.21.2.101:0/220976310 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f183c00bc50 con 0x7f18440f0030 2021-02-10T11:41:23.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.379 7f1848f47700 1 -- 172.21.2.101:0/220976310 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18440f0030 msgr2=0x7f18440f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:23.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.379 7f1848f47700 1 --2- 172.21.2.101:0/220976310 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18440f0030 0x7f18440f0430 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 rx=0x7f183c006f00 tx=0x7f183c009f10).stop 2021-02-10T11:41:23.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.380 7f1848f47700 1 -- 172.21.2.101:0/220976310 shutdown_connections 2021-02-10T11:41:23.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.380 7f1848f47700 1 --2- 172.21.2.101:0/220976310 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f18440ef610 0x7f18440efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:23.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.380 7f1848f47700 1 --2- 172.21.2.101:0/220976310 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f18440f0030 0x7f18440f0430 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:23.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.380 7f1848f47700 1 --2- 172.21.2.101:0/220976310 >> v2:172.21.2.108:3300/0 conn(0x7f18441b0470 0x7f18441b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:23.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.380 7f1848f47700 1 -- 172.21.2.101:0/220976310 >> 172.21.2.101:0/220976310 conn(0x7f18440ce3e0 msgr2=0x7f18440e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:23.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.380 7f1848f47700 1 -- 172.21.2.101:0/220976310 shutdown_connections 2021-02-10T11:41:23.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.380 7f1848f47700 1 -- 172.21.2.101:0/220976310 wait complete. 2021-02-10T11:41:23.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.394 7f1848f47700 1 Processor -- start 2021-02-10T11:41:23.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.395 7f1848f47700 1 -- start start 2021-02-10T11:41:23.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.395 7f1848f47700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f18440aa8d0 0x7f184419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:23.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.395 7f1848f47700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f184419e9d0 0x7f184419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:23.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.395 7f1848f47700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f184419ad40 0x7f184419b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:23.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.395 7f1848f47700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184419b1f0 con 0x7f184419ad40 2021-02-10T11:41:23.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.395 7f1848f47700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1844109b30 con 0x7f18440aa8d0 2021-02-10T11:41:23.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.395 7f1848f47700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1844099620 con 0x7f184419e9d0 2021-02-10T11:41:23.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.395 7f1843314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f184419ad40 0x7f184419b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:23.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.395 7f1843314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f184419ad40 0x7f184419b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57200/0 (socket says 172.21.2.101:57200) 2021-02-10T11:41:23.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.395 7f1843314700 1 -- 172.21.2.101:0/197891388 learned_addr learned my addr 172.21.2.101:0/197891388 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:23.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.395 7f1843314700 1 -- 172.21.2.101:0/197891388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f184419e9d0 msgr2=0x7f184419a720 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:23.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.396 7f1843314700 1 --2- 172.21.2.101:0/197891388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f184419e9d0 0x7f184419a720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:23.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.396 7f1843314700 1 -- 172.21.2.101:0/197891388 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f18440aa8d0 msgr2=0x7f184419a100 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:23.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.396 7f1843314700 1 --2- 172.21.2.101:0/197891388 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f18440aa8d0 0x7f184419a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:23.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.396 7f1843314700 1 -- 172.21.2.101:0/197891388 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1844109640 con 0x7f184419ad40 2021-02-10T11:41:23.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.396 7f1843314700 1 --2- 172.21.2.101:0/197891388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f184419ad40 0x7f184419b770 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 rx=0x7f183c00b8e0 tx=0x7f183c00bc80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:23.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.396 7f18337fe700 1 -- 172.21.2.101:0/197891388 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f183c003030 con 0x7f184419ad40 2021-02-10T11:41:23.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.396 7f18337fe700 1 -- 172.21.2.101:0/197891388 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183c0068e0 con 0x7f184419ad40 2021-02-10T11:41:23.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.396 7f18337fe700 1 -- 172.21.2.101:0/197891388 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f183c00c380 con 0x7f184419ad40 2021-02-10T11:41:23.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.396 7f1848f47700 1 -- 172.21.2.101:0/197891388 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1844109640 con 0x7f184419ad40 2021-02-10T11:41:23.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.396 7f1848f47700 1 -- 172.21.2.101:0/197891388 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1844109640 con 0x7f184419ad40 2021-02-10T11:41:23.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.397 7f18337fe700 1 -- 172.21.2.101:0/197891388 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f183c001270 con 0x7f184419ad40 2021-02-10T11:41:23.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.397 7f18337fe700 1 --2- 172.21.2.101:0/197891388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18280186c0 0x7f182801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:23.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.397 7f18337fe700 1 -- 172.21.2.101:0/197891388 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f183c0328d0 con 0x7f184419ad40 2021-02-10T11:41:23.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.397 7f1842b13700 1 --2- 172.21.2.101:0/197891388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18280186c0 0x7f182801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:23.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.397 7f1842b13700 1 --2- 172.21.2.101:0/197891388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18280186c0 0x7f182801abc0 crc :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:23.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.428 7f1848f47700 1 -- 172.21.2.101:0/197891388 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18440f0b50 con 0x7f184419ad40 2021-02-10T11:41:23.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.431 7f18337fe700 1 -- 172.21.2.101:0/197891388 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f183c0272d0 con 0x7f184419ad40 2021-02-10T11:41:23.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.558 7f1848f47700 1 -- 172.21.2.101:0/197891388 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18440f0b50 con 0x7f184419ad40 2021-02-10T11:41:23.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.558 7f18337fe700 1 -- 172.21.2.101:0/197891388 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+2104 (secure 0 0 0) 0x7f183c0272d0 con 0x7f184419ad40 2021-02-10T11:41:23.561 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:41:23.561 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":89,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2840404451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2840404451},{"type":"v1","addr":"172.21.2.101:6835","nonce":2840404451}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":84}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:40:45.728009","modified":"2021-02-10 11:40:47.738972","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7450},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7450":{"gid":7450,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2757440614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2757440614},{"type":"v1","addr":"172.21.2.108:6833","nonce":2757440614}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2021-02-10T11:41:23.561 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 89 2021-02-10T11:41:23.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.560 7f1848f47700 1 -- 172.21.2.101:0/197891388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18280186c0 msgr2=0x7f182801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:23.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.560 7f1848f47700 1 --2- 172.21.2.101:0/197891388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18280186c0 0x7f182801abc0 crc :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:23.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.560 7f1848f47700 1 -- 172.21.2.101:0/197891388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f184419ad40 msgr2=0x7f184419b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:23.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.560 7f1848f47700 1 --2- 172.21.2.101:0/197891388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f184419ad40 0x7f184419b770 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 rx=0x7f183c00b8e0 tx=0x7f183c00bc80).stop 2021-02-10T11:41:23.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.560 7f1848f47700 1 -- 172.21.2.101:0/197891388 shutdown_connections 2021-02-10T11:41:23.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.560 7f1848f47700 1 --2- 172.21.2.101:0/197891388 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f18280186c0 0x7f182801abc0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:23.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.560 7f1848f47700 1 --2- 172.21.2.101:0/197891388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f184419e9d0 0x7f184419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:23.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.560 7f1848f47700 1 --2- 172.21.2.101:0/197891388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f184419ad40 0x7f184419b770 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:23.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.560 7f1848f47700 1 --2- 172.21.2.101:0/197891388 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f18440aa8d0 0x7f184419a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:23.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.560 7f1848f47700 1 -- 172.21.2.101:0/197891388 >> 172.21.2.101:0/197891388 conn(0x7f184419f7b0 msgr2=0x7f184418f380 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:23.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.560 7f1848f47700 1 -- 172.21.2.101:0/197891388 shutdown_connections 2021-02-10T11:41:23.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.561 7f1848f47700 1 -- 172.21.2.101:0/197891388 wait complete. 2021-02-10T11:41:23.572 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs fail cephfs 2021-02-10T11:41:23.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.980 7fdf76912700 1 Processor -- start 2021-02-10T11:41:23.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.980 7fdf76912700 1 -- start start 2021-02-10T11:41:23.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.980 7fdf76912700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fdf700d0340 0x7fdf700ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:23.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.980 7fdf76912700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdf700d1c70 0x7fdf700efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:23.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.980 7fdf76912700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf700f0b10 con 0x7fdf700d1870 2021-02-10T11:41:23.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.980 7fdf76912700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf700e5ed0 con 0x7fdf700d1c70 2021-02-10T11:41:23.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.980 7fdf76912700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf701b0be0 con 0x7fdf700d0340 2021-02-10T11:41:23.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.981 7fdf74f2f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdf700d1c70 0x7fdf700efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:23.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.981 7fdf74f2f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdf700d1c70 0x7fdf700efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57206/0 (socket says 172.21.2.101:57206) 2021-02-10T11:41:23.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.981 7fdf74f2f700 1 -- 172.21.2.101:0/2002934748 learned_addr learned my addr 172.21.2.101:0/2002934748 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:23.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.981 7fdf74f2f700 1 -- 172.21.2.101:0/2002934748 >> v2:172.21.2.108:3300/0 conn(0x7fdf700d0340 msgr2=0x7fdf700ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:23.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.981 7fdf74f2f700 1 --2- 172.21.2.101:0/2002934748 >> v2:172.21.2.108:3300/0 conn(0x7fdf700d0340 0x7fdf700ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:23.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.981 7fdf74f2f700 1 -- 172.21.2.101:0/2002934748 >> v1:172.21.2.108:6789/0 conn(0x7fdf700d1870 legacy=0x7fdf701b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:23.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.981 7fdf74f2f700 1 -- 172.21.2.101:0/2002934748 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf700f20a0 con 0x7fdf700d1c70 2021-02-10T11:41:23.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.981 7fdf74f2f700 1 --2- 172.21.2.101:0/2002934748 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf700d1c70 0x7fdf700efc60 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 rx=0x7fdf68006f00 tx=0x7fdf68009f10).ready entity=mon.0 client_cookie=82fa3b0268896678 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:23.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.981 7fdf6effd700 1 -- 172.21.2.101:0/2002934748 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdf6800b3b0 con 0x7fdf700d1c70 2021-02-10T11:41:24.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.981 7fdf6effd700 1 -- 172.21.2.101:0/2002934748 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6800b730 con 0x7fdf700d1c70 2021-02-10T11:41:24.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.981 7fdf6effd700 1 -- 172.21.2.101:0/2002934748 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdf6800bc10 con 0x7fdf700d1c70 2021-02-10T11:41:24.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.982 7fdf76912700 1 -- 172.21.2.101:0/2002934748 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf700d1c70 msgr2=0x7fdf700efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:24.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.982 7fdf76912700 1 --2- 172.21.2.101:0/2002934748 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf700d1c70 0x7fdf700efc60 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 rx=0x7fdf68006f00 tx=0x7fdf68009f10).stop 2021-02-10T11:41:24.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.982 7fdf76912700 1 -- 172.21.2.101:0/2002934748 shutdown_connections 2021-02-10T11:41:24.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.982 7fdf76912700 1 --2- 172.21.2.101:0/2002934748 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf700d1c70 0x7fdf700efc60 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:24.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.982 7fdf76912700 1 --2- 172.21.2.101:0/2002934748 >> v2:172.21.2.108:3300/0 conn(0x7fdf700d0340 0x7fdf700ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:24.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.982 7fdf76912700 1 -- 172.21.2.101:0/2002934748 >> 172.21.2.101:0/2002934748 conn(0x7fdf700ce450 msgr2=0x7fdf700d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:24.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.982 7fdf76912700 1 -- 172.21.2.101:0/2002934748 shutdown_connections 2021-02-10T11:41:24.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.982 7fdf76912700 1 -- 172.21.2.101:0/2002934748 wait complete. 2021-02-10T11:41:24.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.997 7fdf76912700 1 Processor -- start 2021-02-10T11:41:24.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.997 7fdf76912700 1 -- start start 2021-02-10T11:41:24.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.997 7fdf76912700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf700b3c90 0x7fdf700b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:24.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.997 7fdf76912700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf700b46b0 0x7fdf700b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:24.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.997 7fdf76912700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf700b4ab0 0x7fdf700b2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:24.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.997 7fdf76912700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf700b3790 con 0x7fdf700b3c90 2021-02-10T11:41:24.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.997 7fdf76912700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf700a2bb0 con 0x7fdf700b4ab0 2021-02-10T11:41:24.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.997 7fdf76912700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf700a2590 con 0x7fdf700b46b0 2021-02-10T11:41:24.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.997 7fdf6ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf700b3c90 0x7fdf700b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:24.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.997 7fdf6ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf700b3c90 0x7fdf700b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57210/0 (socket says 172.21.2.101:57210) 2021-02-10T11:41:24.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.997 7fdf6ffff700 1 -- 172.21.2.101:0/663423048 learned_addr learned my addr 172.21.2.101:0/663423048 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:24.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.998 7fdf74f2f700 1 --2- 172.21.2.101:0/663423048 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf700b4ab0 0x7fdf700b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:24.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.998 7fdf6f7fe700 1 --2- 172.21.2.101:0/663423048 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf700b46b0 0x7fdf700b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:24.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.998 7fdf6ffff700 1 -- 172.21.2.101:0/663423048 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf700b46b0 msgr2=0x7fdf700b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:24.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.998 7fdf6ffff700 1 --2- 172.21.2.101:0/663423048 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf700b46b0 0x7fdf700b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:24.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.998 7fdf6ffff700 1 -- 172.21.2.101:0/663423048 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf700b4ab0 msgr2=0x7fdf700b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:24.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.998 7fdf6ffff700 1 --2- 172.21.2.101:0/663423048 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf700b4ab0 0x7fdf700b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:24.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.998 7fdf6ffff700 1 -- 172.21.2.101:0/663423048 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf700b2d40 con 0x7fdf700b3c90 2021-02-10T11:41:24.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.998 7fdf74f2f700 1 --2- 172.21.2.101:0/663423048 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf700b4ab0 0x7fdf700b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:41:24.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.998 7fdf6f7fe700 1 --2- 172.21.2.101:0/663423048 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf700b46b0 0x7fdf700b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:41:24.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.998 7fdf6ffff700 1 --2- 172.21.2.101:0/663423048 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf700b3c90 0x7fdf700b4090 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 rx=0x7fdf6000b5b0 tx=0x7fdf6000b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:24.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.998 7fdf6cff9700 1 -- 172.21.2.101:0/663423048 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdf6000bd40 con 0x7fdf700b3c90 2021-02-10T11:41:24.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.999 7fdf6cff9700 1 -- 172.21.2.101:0/663423048 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf60006240 con 0x7fdf700b3c90 2021-02-10T11:41:24.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.999 7fdf6cff9700 1 -- 172.21.2.101:0/663423048 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdf60005720 con 0x7fdf700b3c90 2021-02-10T11:41:24.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.999 7fdf76912700 1 -- 172.21.2.101:0/663423048 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf700b2d40 con 0x7fdf700b3c90 2021-02-10T11:41:24.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.999 7fdf76912700 1 -- 172.21.2.101:0/663423048 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf700b2d40 con 0x7fdf700b3c90 2021-02-10T11:41:24.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.999 7fdf6cff9700 1 -- 172.21.2.101:0/663423048 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdf600054a0 con 0x7fdf700b3c90 2021-02-10T11:41:24.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.999 7fdf6cff9700 1 --2- 172.21.2.101:0/663423048 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf500186c0 0x7fdf5001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:24.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.999 7fdf6cff9700 1 -- 172.21.2.101:0/663423048 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fdf60032440 con 0x7fdf700b3c90 2021-02-10T11:41:24.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:23.999 7fdf6f7fe700 1 --2- 172.21.2.101:0/663423048 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf500186c0 0x7fdf5001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:24.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.000 7fdf6f7fe700 1 --2- 172.21.2.101:0/663423048 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf500186c0 0x7fdf5001abc0 crc :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:24.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.030 7fdf76912700 1 -- 172.21.2.101:0/663423048 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf700a2bb0 con 0x7fdf700b3c90 2021-02-10T11:41:24.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.033 7fdf6cff9700 1 -- 172.21.2.101:0/663423048 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdf600009b0 con 0x7fdf700b3c90 2021-02-10T11:41:24.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.160 7fdf76912700 1 -- 172.21.2.101:0/663423048 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdf700a2bb0 con 0x7fdf700b3c90 2021-02-10T11:41:24.568 INFO:tasks.ceph.mon.a.gibba001.stderr:2021-02-10 11:41:24.566 7f974d04d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-02-10T11:41:24.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.572 7fdf6cff9700 1 -- 172.21.2.101:0/663423048 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v90) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdf6001f520 con 0x7fdf700b3c90 2021-02-10T11:41:24.578 INFO:teuthology.orchestra.run.gibba001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-02-10T11:41:24.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.578 7fdf76912700 1 -- 172.21.2.101:0/663423048 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf500186c0 msgr2=0x7fdf5001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:24.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.578 7fdf76912700 1 --2- 172.21.2.101:0/663423048 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf500186c0 0x7fdf5001abc0 crc :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:24.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.578 7fdf76912700 1 -- 172.21.2.101:0/663423048 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf700b3c90 msgr2=0x7fdf700b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:24.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.578 7fdf76912700 1 --2- 172.21.2.101:0/663423048 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf700b3c90 0x7fdf700b4090 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 rx=0x7fdf6000b5b0 tx=0x7fdf6000b950).stop 2021-02-10T11:41:24.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.578 7fdf76912700 1 -- 172.21.2.101:0/663423048 shutdown_connections 2021-02-10T11:41:24.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.578 7fdf76912700 1 --2- 172.21.2.101:0/663423048 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf500186c0 0x7fdf5001abc0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:24.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.578 7fdf76912700 1 --2- 172.21.2.101:0/663423048 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf700b46b0 0x7fdf700b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:24.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.578 7fdf76912700 1 --2- 172.21.2.101:0/663423048 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf700b4ab0 0x7fdf700b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:24.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.578 7fdf76912700 1 --2- 172.21.2.101:0/663423048 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf700b3c90 0x7fdf700b4090 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:24.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.578 7fdf76912700 1 -- 172.21.2.101:0/663423048 >> 172.21.2.101:0/663423048 conn(0x7fdf700946f0 msgr2=0x7fdf70095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:24.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.579 7fdf76912700 1 -- 172.21.2.101:0/663423048 shutdown_connections 2021-02-10T11:41:24.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.579 7fdf76912700 1 -- 172.21.2.101:0/663423048 wait complete. 2021-02-10T11:41:24.589 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:41:24.621 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:41:25.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.997 7fa4ec546700 1 Processor -- start 2021-02-10T11:41:25.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.997 7fa4ec546700 1 -- start start 2021-02-10T11:41:25.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.998 7fa4ec546700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4e40d1870 0x7fa4e41b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:25.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.998 7fa4ec546700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa4e40f0650 0x7fa4e40cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:25.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.998 7fa4ec546700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4e40f0b10 con 0x7fa4e41b0a10 2021-02-10T11:41:25.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.998 7fa4ec546700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e40fe230 con 0x7fa4e40f0650 2021-02-10T11:41:25.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.998 7fa4ec546700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e40e5ed0 con 0x7fa4e40d1870 2021-02-10T11:41:25.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.998 7fa4eab63700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa4e40f0650 0x7fa4e40cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:25.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.998 7fa4eab63700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa4e40f0650 0x7fa4e40cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57218/0 (socket says 172.21.2.101:57218) 2021-02-10T11:41:25.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.998 7fa4eab63700 1 -- 172.21.2.101:0/1558270520 learned_addr learned my addr 172.21.2.101:0/1558270520 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:25.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.998 7fa4eab63700 1 -- 172.21.2.101:0/1558270520 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4e40d1870 msgr2=0x7fa4e41b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:25.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.998 7fa4eab63700 1 --2- 172.21.2.101:0/1558270520 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4e40d1870 0x7fa4e41b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:25.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.998 7fa4eab63700 1 -- 172.21.2.101:0/1558270520 >> v1:172.21.2.108:6789/0 conn(0x7fa4e41b0a10 legacy=0x7fa4e40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:25.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.999 7fa4eab63700 1 -- 172.21.2.101:0/1558270520 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e40f20a0 con 0x7fa4e40f0650 2021-02-10T11:41:25.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.999 7fa4eab63700 1 --2- 172.21.2.101:0/1558270520 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4e40f0650 0x7fa4e40cfb20 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 rx=0x7fa4dc006f00 tx=0x7fa4dc009e90).ready entity=mon.0 client_cookie=38cb6b9b82de6b01 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:25.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.999 7fa4e9360700 1 -- 172.21.2.101:0/1558270520 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa4dc00b300 con 0x7fa4e40f0650 2021-02-10T11:41:25.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.999 7fa4e9360700 1 -- 172.21.2.101:0/1558270520 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4dc00b640 con 0x7fa4e40f0650 2021-02-10T11:41:25.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.999 7fa4e9360700 1 -- 172.21.2.101:0/1558270520 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa4dc00ba90 con 0x7fa4e40f0650 2021-02-10T11:41:25.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.999 7fa4ec546700 1 -- 172.21.2.101:0/1558270520 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4e40f0650 msgr2=0x7fa4e40cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:25.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:24.999 7fa4ec546700 1 --2- 172.21.2.101:0/1558270520 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4e40f0650 0x7fa4e40cfb20 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 rx=0x7fa4dc006f00 tx=0x7fa4dc009e90).stop 2021-02-10T11:41:25.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.000 7fa4ec546700 1 -- 172.21.2.101:0/1558270520 shutdown_connections 2021-02-10T11:41:25.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.000 7fa4ec546700 1 --2- 172.21.2.101:0/1558270520 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4e40f0650 0x7fa4e40cfb20 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:25.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.000 7fa4ec546700 1 --2- 172.21.2.101:0/1558270520 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4e40d1870 0x7fa4e41b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:25.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.000 7fa4ec546700 1 -- 172.21.2.101:0/1558270520 >> 172.21.2.101:0/1558270520 conn(0x7fa4e40ce450 msgr2=0x7fa4e40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:25.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.000 7fa4ec546700 1 -- 172.21.2.101:0/1558270520 shutdown_connections 2021-02-10T11:41:25.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.000 7fa4ec546700 1 -- 172.21.2.101:0/1558270520 wait complete. 2021-02-10T11:41:25.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.014 7fa4ec546700 1 Processor -- start 2021-02-10T11:41:25.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.015 7fa4ec546700 1 -- start start 2021-02-10T11:41:25.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.015 7fa4ec546700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4e418a540 0x7fa4e4197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:25.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.015 7fa4ec546700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4e418aa50 0x7fa4e40b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:25.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.015 7fa4ec546700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa4e40b7df0 0x7fa4e40b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:25.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.015 7fa4ec546700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e41a79d0 con 0x7fa4e418a540 2021-02-10T11:41:25.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.015 7fa4ec546700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e41a9e20 con 0x7fa4e40b7df0 2021-02-10T11:41:25.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.015 7fa4ec546700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e41aa780 con 0x7fa4e418aa50 2021-02-10T11:41:25.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.015 7fa4ea362700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4e418a540 0x7fa4e4197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:25.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.015 7fa4ea362700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4e418a540 0x7fa4e4197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57222/0 (socket says 172.21.2.101:57222) 2021-02-10T11:41:25.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.015 7fa4ea362700 1 -- 172.21.2.101:0/1617371852 learned_addr learned my addr 172.21.2.101:0/1617371852 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:25.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.016 7fa4ea362700 1 -- 172.21.2.101:0/1617371852 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4e418aa50 msgr2=0x7fa4e40b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:25.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.016 7fa4ea362700 1 --2- 172.21.2.101:0/1617371852 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4e418aa50 0x7fa4e40b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:25.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.016 7fa4ea362700 1 -- 172.21.2.101:0/1617371852 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa4e40b7df0 msgr2=0x7fa4e40b81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:25.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.016 7fa4ea362700 1 --2- 172.21.2.101:0/1617371852 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa4e40b7df0 0x7fa4e40b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:25.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.016 7fa4ea362700 1 -- 172.21.2.101:0/1617371852 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e41aa9a0 con 0x7fa4e418a540 2021-02-10T11:41:25.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.016 7fa4ea362700 1 --2- 172.21.2.101:0/1617371852 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4e418a540 0x7fa4e4197480 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 rx=0x7fa4e0006f00 tx=0x7fa4e000b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:25.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.016 7fa4daffd700 1 -- 172.21.2.101:0/1617371852 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa4e000bd30 con 0x7fa4e418a540 2021-02-10T11:41:25.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.016 7fa4daffd700 1 -- 172.21.2.101:0/1617371852 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e0000c90 con 0x7fa4e418a540 2021-02-10T11:41:25.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.016 7fa4daffd700 1 -- 172.21.2.101:0/1617371852 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa4e00022a0 con 0x7fa4e418a540 2021-02-10T11:41:25.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.016 7fa4ec546700 1 -- 172.21.2.101:0/1617371852 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e41aa9a0 con 0x7fa4e418a540 2021-02-10T11:41:25.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.016 7fa4ec546700 1 -- 172.21.2.101:0/1617371852 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e41aaf30 con 0x7fa4e418a540 2021-02-10T11:41:25.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.017 7fa4daffd700 1 -- 172.21.2.101:0/1617371852 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa4e0005310 con 0x7fa4e418a540 2021-02-10T11:41:25.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.017 7fa4daffd700 1 --2- 172.21.2.101:0/1617371852 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa4cc0186c0 0x7fa4cc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:25.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.017 7fa4daffd700 1 -- 172.21.2.101:0/1617371852 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fa4e00325d0 con 0x7fa4e418a540 2021-02-10T11:41:25.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.017 7fa4e9b61700 1 --2- 172.21.2.101:0/1617371852 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa4cc0186c0 0x7fa4cc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:25.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.017 7fa4e9b61700 1 --2- 172.21.2.101:0/1617371852 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa4cc0186c0 0x7fa4cc01abc0 crc :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:25.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.048 7fa4ec546700 1 -- 172.21.2.101:0/1617371852 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e41aa250 con 0x7fa4e418a540 2021-02-10T11:41:25.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.050 7fa4daffd700 1 -- 172.21.2.101:0/1617371852 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa4e0000960 con 0x7fa4e418a540 2021-02-10T11:41:25.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.179 7fa4ec546700 1 -- 172.21.2.101:0/1617371852 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4e41aa250 con 0x7fa4e418a540 2021-02-10T11:41:25.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.179 7fa4daffd700 1 -- 172.21.2.101:0/1617371852 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+1760 (secure 0 0 0) 0x7fa4e0000d50 con 0x7fa4e418a540 2021-02-10T11:41:25.182 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:41:25.182 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2840404451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2840404451},{"type":"v1","addr":"172.21.2.101:6835","nonce":2840404451}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":84}],"filesystems":[{"mdsmap":{"epoch":90,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:40:45.728009","modified":"2021-02-10 11:41:24.567045","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2021-02-10T11:41:25.182 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 90 2021-02-10T11:41:25.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.181 7fa4ec546700 1 -- 172.21.2.101:0/1617371852 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa4cc0186c0 msgr2=0x7fa4cc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:25.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.181 7fa4ec546700 1 --2- 172.21.2.101:0/1617371852 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa4cc0186c0 0x7fa4cc01abc0 crc :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:25.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.181 7fa4ec546700 1 -- 172.21.2.101:0/1617371852 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4e418a540 msgr2=0x7fa4e4197480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:25.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.181 7fa4ec546700 1 --2- 172.21.2.101:0/1617371852 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4e418a540 0x7fa4e4197480 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 rx=0x7fa4e0006f00 tx=0x7fa4e000b940).stop 2021-02-10T11:41:25.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.181 7fa4ec546700 1 -- 172.21.2.101:0/1617371852 shutdown_connections 2021-02-10T11:41:25.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.181 7fa4ec546700 1 --2- 172.21.2.101:0/1617371852 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa4cc0186c0 0x7fa4cc01abc0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:25.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.181 7fa4ec546700 1 --2- 172.21.2.101:0/1617371852 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa4e418aa50 0x7fa4e40b77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:25.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.181 7fa4ec546700 1 --2- 172.21.2.101:0/1617371852 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa4e40b7df0 0x7fa4e40b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:25.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.181 7fa4ec546700 1 --2- 172.21.2.101:0/1617371852 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa4e418a540 0x7fa4e4197480 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:25.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.181 7fa4ec546700 1 -- 172.21.2.101:0/1617371852 >> 172.21.2.101:0/1617371852 conn(0x7fa4e40efc70 msgr2=0x7fa4e40ef410 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:25.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.181 7fa4ec546700 1 -- 172.21.2.101:0/1617371852 shutdown_connections 2021-02-10T11:41:25.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.181 7fa4ec546700 1 -- 172.21.2.101:0/1617371852 wait complete. 2021-02-10T11:41:25.192 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-02-10T11:41:25.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.614 7f7c4a95c700 1 Processor -- start 2021-02-10T11:41:25.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.614 7f7c4a95c700 1 -- start start 2021-02-10T11:41:25.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.614 7f7c4a95c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7c440d1870 0x7f7c441b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:25.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.614 7f7c4a95c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7c441b0a10 0x7f7c440cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:25.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.614 7f7c4a95c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c440f0b10 con 0x7f7c440d1c70 2021-02-10T11:41:25.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.614 7f7c4a95c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c440e5ed0 con 0x7f7c440d1870 2021-02-10T11:41:25.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.614 7f7c4a95c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c440f0650 con 0x7f7c441b0a10 2021-02-10T11:41:25.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.614 7f7c48f79700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f7c440d1c70 0x7f7c440cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57724/0 (socket says 172.21.2.101:57724) 2021-02-10T11:41:25.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.614 7f7c48f79700 1 -- 172.21.2.101:0/3828013224 learned_addr learned my addr 172.21.2.101:0/3828013224 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:25.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.614 7f7c43fff700 1 --2- 172.21.2.101:0/3828013224 >> v2:172.21.2.101:3300/0 conn(0x7f7c440d1870 0x7f7c441b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:25.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.615 7f7c42ffd700 1 -- 172.21.2.101:0/3828013224 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398868449 0 0) 0x7f7c3c001500 con 0x7f7c440d1c70 2021-02-10T11:41:25.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.615 7f7c42ffd700 1 -- 172.21.2.101:0/3828013224 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c28002fe0 con 0x7f7c440d1c70 2021-02-10T11:41:25.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.615 7f7c43fff700 1 -- 172.21.2.101:0/3828013224 >> v2:172.21.2.108:3300/0 conn(0x7f7c441b0a10 msgr2=0x7f7c440cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:25.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.615 7f7c43fff700 1 --2- 172.21.2.101:0/3828013224 >> v2:172.21.2.108:3300/0 conn(0x7f7c441b0a10 0x7f7c440cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:25.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.615 7f7c43fff700 1 -- 172.21.2.101:0/3828013224 >> v1:172.21.2.101:6789/0 conn(0x7f7c440d1c70 legacy=0x7f7c440cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:25.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.615 7f7c43fff700 1 -- 172.21.2.101:0/3828013224 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c440f0840 con 0x7f7c440d1870 2021-02-10T11:41:25.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.615 7f7c43fff700 1 --2- 172.21.2.101:0/3828013224 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c440d1870 0x7f7c441b03f0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 rx=0x7f7c34006650 tx=0x7f7c34009a80).ready entity=mon.0 client_cookie=d8818c7960583631 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:25.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.615 7f7c42ffd700 1 -- 172.21.2.101:0/3828013224 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7c340009b0 con 0x7f7c440d1870 2021-02-10T11:41:25.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.615 7f7c42ffd700 1 -- 172.21.2.101:0/3828013224 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3400c0c0 con 0x7f7c440d1870 2021-02-10T11:41:25.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.615 7f7c42ffd700 1 -- 172.21.2.101:0/3828013224 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7c3400c5a0 con 0x7f7c440d1870 2021-02-10T11:41:25.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.615 7f7c4a95c700 1 -- 172.21.2.101:0/3828013224 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c440d1870 msgr2=0x7f7c441b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:25.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.615 7f7c4a95c700 1 --2- 172.21.2.101:0/3828013224 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c440d1870 0x7f7c441b03f0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 rx=0x7f7c34006650 tx=0x7f7c34009a80).stop 2021-02-10T11:41:25.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.616 7f7c4a95c700 1 -- 172.21.2.101:0/3828013224 shutdown_connections 2021-02-10T11:41:25.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.616 7f7c4a95c700 1 --2- 172.21.2.101:0/3828013224 >> v2:172.21.2.108:3300/0 conn(0x7f7c441b0a10 0x7f7c440cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:25.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.616 7f7c4a95c700 1 --2- 172.21.2.101:0/3828013224 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c440d1870 0x7f7c441b03f0 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:25.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.616 7f7c4a95c700 1 -- 172.21.2.101:0/3828013224 >> 172.21.2.101:0/3828013224 conn(0x7f7c440ce450 msgr2=0x7f7c440d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:25.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.616 7f7c4a95c700 1 -- 172.21.2.101:0/3828013224 shutdown_connections 2021-02-10T11:41:25.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.616 7f7c4a95c700 1 -- 172.21.2.101:0/3828013224 wait complete. 2021-02-10T11:41:25.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.631 7f7c4a95c700 1 Processor -- start 2021-02-10T11:41:25.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.631 7f7c4a95c700 1 -- start start 2021-02-10T11:41:25.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.631 7f7c4a95c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c4418f790 0x7f7c440bc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:25.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.631 7f7c4a95c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c4418fb90 0x7f7c440bc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:25.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.631 7f7c4a95c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c440bcd80 0x7f7c440bd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:25.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.631 7f7c4a95c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c440ebe70 con 0x7f7c440bcd80 2021-02-10T11:41:25.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.631 7f7c4a95c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c441a49c0 con 0x7f7c4418f790 2021-02-10T11:41:25.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.631 7f7c4a95c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4409fd10 con 0x7f7c4418fb90 2021-02-10T11:41:25.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.632 7f7c48f79700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c440bcd80 0x7f7c440bd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:25.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.632 7f7c48f79700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c440bcd80 0x7f7c440bd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57236/0 (socket says 172.21.2.101:57236) 2021-02-10T11:41:25.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.632 7f7c48f79700 1 -- 172.21.2.101:0/292536150 learned_addr learned my addr 172.21.2.101:0/292536150 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:25.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.632 7f7c48f79700 1 -- 172.21.2.101:0/292536150 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c4418fb90 msgr2=0x7f7c440bc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:25.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.632 7f7c48f79700 1 --2- 172.21.2.101:0/292536150 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c4418fb90 0x7f7c440bc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:25.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.632 7f7c48f79700 1 -- 172.21.2.101:0/292536150 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c4418f790 msgr2=0x7f7c440bc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:25.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.632 7f7c48f79700 1 --2- 172.21.2.101:0/292536150 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c4418f790 0x7f7c440bc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:25.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.632 7f7c48f79700 1 -- 172.21.2.101:0/292536150 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c440bdd50 con 0x7f7c440bcd80 2021-02-10T11:41:25.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.632 7f7c48f79700 1 --2- 172.21.2.101:0/292536150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c440bcd80 0x7f7c440bd730 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 rx=0x7f7c3c009980 tx=0x7f7c3c00bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:25.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.632 7f7c40ff9700 1 -- 172.21.2.101:0/292536150 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7c3c00be10 con 0x7f7c440bcd80 2021-02-10T11:41:25.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.632 7f7c40ff9700 1 -- 172.21.2.101:0/292536150 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3c006080 con 0x7f7c440bcd80 2021-02-10T11:41:25.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.632 7f7c4a95c700 1 -- 172.21.2.101:0/292536150 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c440bdd50 con 0x7f7c440bcd80 2021-02-10T11:41:25.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.632 7f7c40ff9700 1 -- 172.21.2.101:0/292536150 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7c3c0064e0 con 0x7f7c440bcd80 2021-02-10T11:41:25.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.632 7f7c4a95c700 1 -- 172.21.2.101:0/292536150 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c440bdd50 con 0x7f7c440bcd80 2021-02-10T11:41:25.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.633 7f7c40ff9700 1 -- 172.21.2.101:0/292536150 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7c3c006260 con 0x7f7c440bcd80 2021-02-10T11:41:25.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.633 7f7c40ff9700 1 --2- 172.21.2.101:0/292536150 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c200186d0 0x7f7c2001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:25.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.633 7f7c40ff9700 1 -- 172.21.2.101:0/292536150 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f7c3c032a40 con 0x7f7c440bcd80 2021-02-10T11:41:25.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.633 7f7c43fff700 1 --2- 172.21.2.101:0/292536150 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c200186d0 0x7f7c2001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:25.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.633 7f7c43fff700 1 --2- 172.21.2.101:0/292536150 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c200186d0 0x7f7c2001abd0 crc :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:25.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.664 7f7c4a95c700 1 -- 172.21.2.101:0/292536150 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c440f0b10 con 0x7f7c440bcd80 2021-02-10T11:41:25.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.667 7f7c40ff9700 1 -- 172.21.2.101:0/292536150 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7c3c006290 con 0x7f7c440bcd80 2021-02-10T11:41:25.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:25.795 7f7c4a95c700 1 -- 172.21.2.101:0/292536150 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7c440f0b10 con 0x7f7c440bcd80 2021-02-10T11:41:26.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:26.578 7f7c40ff9700 1 -- 172.21.2.101:0/292536150 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v91) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7c3c006080 con 0x7f7c440bcd80 2021-02-10T11:41:26.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:26.617 7f7c4a95c700 1 -- 172.21.2.101:0/292536150 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c200186d0 msgr2=0x7f7c2001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:26.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:26.617 7f7c4a95c700 1 --2- 172.21.2.101:0/292536150 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c200186d0 0x7f7c2001abd0 crc :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:26.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:26.617 7f7c4a95c700 1 -- 172.21.2.101:0/292536150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c440bcd80 msgr2=0x7f7c440bd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:26.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:26.617 7f7c4a95c700 1 --2- 172.21.2.101:0/292536150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c440bcd80 0x7f7c440bd730 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 rx=0x7f7c3c009980 tx=0x7f7c3c00bcd0).stop 2021-02-10T11:41:26.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:26.618 7f7c4a95c700 1 -- 172.21.2.101:0/292536150 shutdown_connections 2021-02-10T11:41:26.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:26.618 7f7c4a95c700 1 --2- 172.21.2.101:0/292536150 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c200186d0 0x7f7c2001abd0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:26.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:26.618 7f7c4a95c700 1 --2- 172.21.2.101:0/292536150 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c4418fb90 0x7f7c440bc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:26.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:26.618 7f7c4a95c700 1 --2- 172.21.2.101:0/292536150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c440bcd80 0x7f7c440bd730 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:26.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:26.618 7f7c4a95c700 1 --2- 172.21.2.101:0/292536150 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c4418f790 0x7f7c440bc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:26.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:26.618 7f7c4a95c700 1 -- 172.21.2.101:0/292536150 >> 172.21.2.101:0/292536150 conn(0x7f7c440c2d70 msgr2=0x7f7c44094c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:26.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:26.618 7f7c4a95c700 1 -- 172.21.2.101:0/292536150 shutdown_connections 2021-02-10T11:41:26.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:26.618 7f7c4a95c700 1 -- 172.21.2.101:0/292536150 wait complete. 2021-02-10T11:41:26.628 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-02-10T11:41:27.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.040 7fa1927ca700 1 Processor -- start 2021-02-10T11:41:27.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.040 7fa1927ca700 1 -- start start 2021-02-10T11:41:27.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.040 7fa1927ca700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa18c0d1880 0x7fa18c1b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:27.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.040 7fa1927ca700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa18c0cff80 0x7fa18c109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:27.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.040 7fa1927ca700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa18c0fe1d0 con 0x7fa18c1b0af0 2021-02-10T11:41:27.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.040 7fa1927ca700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18c109ae0 con 0x7fa18c0d1880 2021-02-10T11:41:27.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.040 7fa1927ca700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18c109760 con 0x7fa18c0cff80 2021-02-10T11:41:27.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.040 7fa18bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa18c0d1880 0x7fa18c1b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:27.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.040 7fa18bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa18c0d1880 0x7fa18c1b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57242/0 (socket says 172.21.2.101:57242) 2021-02-10T11:41:27.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.040 7fa18bfff700 1 -- 172.21.2.101:0/3642371599 learned_addr learned my addr 172.21.2.101:0/3642371599 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:27.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.040 7fa190de7700 1 --2- 172.21.2.101:0/3642371599 >> v2:172.21.2.108:3300/0 conn(0x7fa18c0cff80 0x7fa18c109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:27.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.041 7fa18bfff700 1 -- 172.21.2.101:0/3642371599 >> v2:172.21.2.108:3300/0 conn(0x7fa18c0cff80 msgr2=0x7fa18c109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:27.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.041 7fa18bfff700 1 --2- 172.21.2.101:0/3642371599 >> v2:172.21.2.108:3300/0 conn(0x7fa18c0cff80 0x7fa18c109140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:27.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.041 7fa18bfff700 1 -- 172.21.2.101:0/3642371599 >> v1:172.21.2.108:6789/0 conn(0x7fa18c1b0af0 legacy=0x7fa18c108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:27.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.041 7fa18bfff700 1 -- 172.21.2.101:0/3642371599 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18c088070 con 0x7fa18c0d1880 2021-02-10T11:41:27.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.041 7fa18bfff700 1 --2- 172.21.2.101:0/3642371599 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa18c0d1880 0x7fa18c1b04d0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 rx=0x7fa17c006700 tx=0x7fa17c009b80).ready entity=mon.0 client_cookie=9d7ecbd468f71605 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:27.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.041 7fa18affd700 1 -- 172.21.2.101:0/3642371599 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa17c00c0c0 con 0x7fa18c0d1880 2021-02-10T11:41:27.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.041 7fa18affd700 1 -- 172.21.2.101:0/3642371599 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17c00c400 con 0x7fa18c0d1880 2021-02-10T11:41:27.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.041 7fa18affd700 1 -- 172.21.2.101:0/3642371599 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa17c00c8e0 con 0x7fa18c0d1880 2021-02-10T11:41:27.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.041 7fa1927ca700 1 -- 172.21.2.101:0/3642371599 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa18c0d1880 msgr2=0x7fa18c1b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:27.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.041 7fa1927ca700 1 --2- 172.21.2.101:0/3642371599 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa18c0d1880 0x7fa18c1b04d0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 rx=0x7fa17c006700 tx=0x7fa17c009b80).stop 2021-02-10T11:41:27.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.041 7fa1927ca700 1 -- 172.21.2.101:0/3642371599 shutdown_connections 2021-02-10T11:41:27.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.041 7fa1927ca700 1 --2- 172.21.2.101:0/3642371599 >> v2:172.21.2.108:3300/0 conn(0x7fa18c0cff80 0x7fa18c109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:27.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.041 7fa1927ca700 1 --2- 172.21.2.101:0/3642371599 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa18c0d1880 0x7fa18c1b04d0 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:27.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.041 7fa1927ca700 1 -- 172.21.2.101:0/3642371599 >> 172.21.2.101:0/3642371599 conn(0x7fa18c0ce430 msgr2=0x7fa18c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:27.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.041 7fa1927ca700 1 -- 172.21.2.101:0/3642371599 shutdown_connections 2021-02-10T11:41:27.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.041 7fa1927ca700 1 -- 172.21.2.101:0/3642371599 wait complete. 2021-02-10T11:41:27.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.056 7fa1927ca700 1 Processor -- start 2021-02-10T11:41:27.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.056 7fa1927ca700 1 -- start start 2021-02-10T11:41:27.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.056 7fa1927ca700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa18c0aef00 0x7fa18c099c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:27.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.056 7fa1927ca700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa18c09a260 0x7fa18c09a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:27.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.056 7fa1927ca700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa18c09ac80 0x7fa18c19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:27.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.056 7fa1927ca700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c19bdd0 con 0x7fa18c09ac80 2021-02-10T11:41:27.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.056 7fa1927ca700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c19de70 con 0x7fa18c0aef00 2021-02-10T11:41:27.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.056 7fa1927ca700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c19a2d0 con 0x7fa18c09a260 2021-02-10T11:41:27.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.057 7fa190de7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa18c09ac80 0x7fa18c19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:27.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.057 7fa18b7fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa18c09a260 0x7fa18c09a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:27.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.057 7fa18bfff700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa18c0aef00 0x7fa18c099c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:27.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.057 7fa190de7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa18c09ac80 0x7fa18c19a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57250/0 (socket says 172.21.2.101:57250) 2021-02-10T11:41:27.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.057 7fa190de7700 1 -- 172.21.2.101:0/2801982603 learned_addr learned my addr 172.21.2.101:0/2801982603 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:27.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.057 7fa190de7700 1 -- 172.21.2.101:0/2801982603 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa18c09a260 msgr2=0x7fa18c09a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:27.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.057 7fa190de7700 1 --2- 172.21.2.101:0/2801982603 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa18c09a260 0x7fa18c09a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:27.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.057 7fa190de7700 1 -- 172.21.2.101:0/2801982603 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa18c0aef00 msgr2=0x7fa18c099c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:27.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.057 7fa190de7700 1 --2- 172.21.2.101:0/2801982603 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa18c0aef00 0x7fa18c099c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:27.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.057 7fa190de7700 1 -- 172.21.2.101:0/2801982603 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18c199f50 con 0x7fa18c09ac80 2021-02-10T11:41:27.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.057 7fa190de7700 1 --2- 172.21.2.101:0/2801982603 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa18c09ac80 0x7fa18c19a850 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 rx=0x7fa1840099a0 tx=0x7fa18400bc70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:27.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.057 7fa188ff9700 1 -- 172.21.2.101:0/2801982603 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa18400bdb0 con 0x7fa18c09ac80 2021-02-10T11:41:27.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.057 7fa188ff9700 1 -- 172.21.2.101:0/2801982603 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa184003030 con 0x7fa18c09ac80 2021-02-10T11:41:27.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.057 7fa188ff9700 1 -- 172.21.2.101:0/2801982603 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa184001470 con 0x7fa18c09ac80 2021-02-10T11:41:27.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.057 7fa1927ca700 1 -- 172.21.2.101:0/2801982603 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18c199f50 con 0x7fa18c09ac80 2021-02-10T11:41:27.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.058 7fa1927ca700 1 -- 172.21.2.101:0/2801982603 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18c19b160 con 0x7fa18c09ac80 2021-02-10T11:41:27.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.058 7fa188ff9700 1 -- 172.21.2.101:0/2801982603 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa1840011f0 con 0x7fa18c09ac80 2021-02-10T11:41:27.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.058 7fa188ff9700 1 --2- 172.21.2.101:0/2801982603 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa16c0186c0 0x7fa16c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:27.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.058 7fa188ff9700 1 -- 172.21.2.101:0/2801982603 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fa1840328f0 con 0x7fa18c09ac80 2021-02-10T11:41:27.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.058 7fa18bfff700 1 --2- 172.21.2.101:0/2801982603 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa16c0186c0 0x7fa16c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:27.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.059 7fa18bfff700 1 --2- 172.21.2.101:0/2801982603 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa16c0186c0 0x7fa16c01abc0 crc :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:27.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.090 7fa1927ca700 1 -- 172.21.2.101:0/2801982603 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa18c082a70 con 0x7fa18c09ac80 2021-02-10T11:41:27.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.093 7fa188ff9700 1 -- 172.21.2.101:0/2801982603 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa184000f90 con 0x7fa18c09ac80 2021-02-10T11:41:27.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.218 7fa1927ca700 1 -- 172.21.2.101:0/2801982603 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa18c199f50 con 0x7fa18c09ac80 2021-02-10T11:41:27.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.582 7fa188ff9700 1 -- 172.21.2.101:0/2801982603 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v114) v1 ==== 185+0+0 (secure 0 0 0) 0x7fa18401f870 con 0x7fa18c09ac80 2021-02-10T11:41:27.586 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' removed 2021-02-10T11:41:27.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.586 7fa1927ca700 1 -- 172.21.2.101:0/2801982603 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa16c0186c0 msgr2=0x7fa16c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:27.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.586 7fa1927ca700 1 --2- 172.21.2.101:0/2801982603 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa16c0186c0 0x7fa16c01abc0 crc :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:27.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.586 7fa1927ca700 1 -- 172.21.2.101:0/2801982603 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa18c09ac80 msgr2=0x7fa18c19a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:27.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.586 7fa1927ca700 1 --2- 172.21.2.101:0/2801982603 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa18c09ac80 0x7fa18c19a850 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 rx=0x7fa1840099a0 tx=0x7fa18400bc70).stop 2021-02-10T11:41:27.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.586 7fa1927ca700 1 -- 172.21.2.101:0/2801982603 shutdown_connections 2021-02-10T11:41:27.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.586 7fa1927ca700 1 --2- 172.21.2.101:0/2801982603 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa16c0186c0 0x7fa16c01abc0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:27.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.586 7fa1927ca700 1 --2- 172.21.2.101:0/2801982603 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa18c09a260 0x7fa18c09a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:27.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.586 7fa1927ca700 1 --2- 172.21.2.101:0/2801982603 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa18c09ac80 0x7fa18c19a850 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:27.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.586 7fa1927ca700 1 --2- 172.21.2.101:0/2801982603 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa18c0aef00 0x7fa18c099c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:27.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.586 7fa1927ca700 1 -- 172.21.2.101:0/2801982603 >> 172.21.2.101:0/2801982603 conn(0x7fa18c18e810 msgr2=0x7fa18c18c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:27.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.586 7fa1927ca700 1 -- 172.21.2.101:0/2801982603 shutdown_connections 2021-02-10T11:41:27.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:27.586 7fa1927ca700 1 -- 172.21.2.101:0/2801982603 wait complete. 2021-02-10T11:41:27.597 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-02-10T11:41:28.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.009 7f87b8cb8700 1 Processor -- start 2021-02-10T11:41:28.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.009 7f87b8cb8700 1 -- start start 2021-02-10T11:41:28.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.009 7f87b8cb8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f87b410b000 0x7f87b40cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:28.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.009 7f87b8cb8700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b40fe1d0 con 0x7f87b40d1880 2021-02-10T11:41:28.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.009 7f87b8cb8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b41b0cc0 con 0x7f87b40d1c80 2021-02-10T11:41:28.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.009 7f87b8cb8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b4109a70 con 0x7f87b410b000 2021-02-10T11:41:28.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.010 7f87b285e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f87b40d1880 0x7f87b41b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57748/0 (socket says 172.21.2.101:57748) 2021-02-10T11:41:28.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.010 7f87b285e700 1 -- 172.21.2.101:0/3094912443 learned_addr learned my addr 172.21.2.101:0/3094912443 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:28.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.010 7f87b185c700 1 -- 172.21.2.101:0/3094912443 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575794510 0 0) 0x7f879c000ce0 con 0x7f87b40d1880 2021-02-10T11:41:28.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.010 7f87b185c700 1 -- 172.21.2.101:0/3094912443 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87a0002fe0 con 0x7f87b40d1880 2021-02-10T11:41:28.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.010 7f87b305f700 1 --2- 172.21.2.101:0/3094912443 >> v2:172.21.2.108:3300/0 conn(0x7f87b410b000 0x7f87b40cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:28.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.010 7f87b185c700 1 -- 172.21.2.101:0/3094912443 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 509796885 0 0) 0x7f87a4000ff0 con 0x7f87b40d1c80 2021-02-10T11:41:28.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.010 7f87b185c700 1 -- 172.21.2.101:0/3094912443 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87a0003210 con 0x7f87b40d1c80 2021-02-10T11:41:28.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.010 7f87b185c700 1 -- 172.21.2.101:0/3094912443 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2261492720 0 0) 0x7f879c0010e0 con 0x7f87b40d1880 2021-02-10T11:41:28.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.010 7f87b185c700 1 -- 172.21.2.101:0/3094912443 >> v2:172.21.2.108:3300/0 conn(0x7f87b410b000 msgr2=0x7f87b40cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:28.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.010 7f87b185c700 1 --2- 172.21.2.101:0/3094912443 >> v2:172.21.2.108:3300/0 conn(0x7f87b410b000 0x7f87b40cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:28.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.010 7f87b185c700 1 -- 172.21.2.101:0/3094912443 >> v1:172.21.2.108:6789/0 conn(0x7f87b40d1c80 legacy=0x7f87b40cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:28.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.010 7f87b185c700 1 -- 172.21.2.101:0/3094912443 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b410b400 con 0x7f87b40d1880 2021-02-10T11:41:28.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.010 7f87b185c700 1 -- 172.21.2.101:0/3094912443 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f879c0015e0 con 0x7f87b40d1880 2021-02-10T11:41:28.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.010 7f87b185c700 1 -- 172.21.2.101:0/3094912443 >> v1:172.21.2.101:6789/0 conn(0x7f87b40d1880 legacy=0x7f87b41b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:28.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.010 7f87b185c700 1 --2- 172.21.2.101:0/3094912443 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87a00037e0 0x7f87a0003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:28.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.010 7f87b185c700 1 --2- 172.21.2.101:0/3094912443 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87a00068e0 0x7f87a0008d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:28.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.010 7f87b185c700 1 --2- 172.21.2.101:0/3094912443 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87a00095b0 0x7f87a000bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:28.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.010 7f87b185c700 1 -- 172.21.2.101:0/3094912443 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a000c300 con 0x7f87a00037e0 2021-02-10T11:41:28.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.011 7f87b185c700 1 -- 172.21.2.101:0/3094912443 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a00504f0 con 0x7f87a00095b0 2021-02-10T11:41:28.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.011 7f87b185c700 1 -- 172.21.2.101:0/3094912443 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a0050710 con 0x7f87a00068e0 2021-02-10T11:41:28.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.011 7f87b285e700 1 --2- 172.21.2.101:0/3094912443 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87a00037e0 0x7f87a0003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:28.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.011 7f87b305f700 1 --2- 172.21.2.101:0/3094912443 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87a00095b0 0x7f87a000bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:28.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.011 7f87b285e700 1 -- 172.21.2.101:0/3094912443 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87a00068e0 msgr2=0x7f87a0008d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:28.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.011 7f87b285e700 1 --2- 172.21.2.101:0/3094912443 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87a00068e0 0x7f87a0008d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:28.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.011 7f87b285e700 1 -- 172.21.2.101:0/3094912443 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87a00095b0 msgr2=0x7f87a000bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:28.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.011 7f87b285e700 1 --2- 172.21.2.101:0/3094912443 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87a00095b0 0x7f87a000bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:28.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.011 7f87b285e700 1 -- 172.21.2.101:0/3094912443 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a00509b0 con 0x7f87a00037e0 2021-02-10T11:41:28.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.011 7f87b305f700 1 -- 172.21.2.101:0/3094912443 reap_dead start 2021-02-10T11:41:28.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.011 7f87b285e700 1 --2- 172.21.2.101:0/3094912443 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87a00037e0 0x7f87a0003be0 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 rx=0x7f879c001890 tx=0x7f879c007d20).ready entity=mon.0 client_cookie=b7160523b101b166 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:28.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.011 7f87b185c700 1 -- 172.21.2.101:0/3094912443 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f879c0014a0 con 0x7f87a00037e0 2021-02-10T11:41:28.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.011 7f87b185c700 1 -- 172.21.2.101:0/3094912443 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879c0070c0 con 0x7f87a00037e0 2021-02-10T11:41:28.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.011 7f87b185c700 1 -- 172.21.2.101:0/3094912443 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f879c0010e0 con 0x7f87a00037e0 2021-02-10T11:41:28.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.011 7f87b8cb8700 1 -- 172.21.2.101:0/3094912443 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87a00037e0 msgr2=0x7f87a0003be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:28.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.011 7f87b8cb8700 1 --2- 172.21.2.101:0/3094912443 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87a00037e0 0x7f87a0003be0 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 rx=0x7f879c001890 tx=0x7f879c007d20).stop 2021-02-10T11:41:28.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.012 7f87b8cb8700 1 -- 172.21.2.101:0/3094912443 shutdown_connections 2021-02-10T11:41:28.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.012 7f87b8cb8700 1 --2- 172.21.2.101:0/3094912443 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87a00037e0 0x7f87a0003be0 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:28.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.012 7f87b8cb8700 1 -- 172.21.2.101:0/3094912443 >> 172.21.2.101:0/3094912443 conn(0x7f87b40ce430 msgr2=0x7f87b40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:28.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.012 7f87b8cb8700 1 -- 172.21.2.101:0/3094912443 shutdown_connections 2021-02-10T11:41:28.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.012 7f87b8cb8700 1 -- 172.21.2.101:0/3094912443 wait complete. 2021-02-10T11:41:28.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.027 7f87b8cb8700 1 Processor -- start 2021-02-10T11:41:28.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.027 7f87b8cb8700 1 -- start start 2021-02-10T11:41:28.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.027 7f87b8cb8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87b41a05e0 0x7f87b409a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:28.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.027 7f87b8cb8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87b409a8b0 0x7f87b409acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:28.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.027 7f87b8cb8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87b409b2d0 0x7f87b40beed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:28.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.027 7f87b8cb8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4099d50 con 0x7f87b41a05e0 2021-02-10T11:41:28.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.027 7f87b8cb8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40bf4f0 con 0x7f87b409a8b0 2021-02-10T11:41:28.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.027 7f87b8cb8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40bf6e0 con 0x7f87b409b2d0 2021-02-10T11:41:28.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.027 7f87b285e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87b41a05e0 0x7f87b409a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:28.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.027 7f87b285e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87b41a05e0 0x7f87b409a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57268/0 (socket says 172.21.2.101:57268) 2021-02-10T11:41:28.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.027 7f87b285e700 1 -- 172.21.2.101:0/2844025500 learned_addr learned my addr 172.21.2.101:0/2844025500 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:28.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.028 7f87b285e700 1 -- 172.21.2.101:0/2844025500 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87b409b2d0 msgr2=0x7f87b40beed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:28.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.028 7f87b285e700 1 --2- 172.21.2.101:0/2844025500 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87b409b2d0 0x7f87b40beed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:28.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.028 7f87b285e700 1 -- 172.21.2.101:0/2844025500 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87b409a8b0 msgr2=0x7f87b409acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:28.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.028 7f87b285e700 1 --2- 172.21.2.101:0/2844025500 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87b409a8b0 0x7f87b409acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:28.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.028 7f87b285e700 1 -- 172.21.2.101:0/2844025500 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b40bf900 con 0x7f87b41a05e0 2021-02-10T11:41:28.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.028 7f87b285e700 1 --2- 172.21.2.101:0/2844025500 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87b41a05e0 0x7f87b409a290 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 rx=0x7f879c007f20 tx=0x7f879c00a950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:28.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.028 7f87ab7fe700 1 -- 172.21.2.101:0/2844025500 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f879c00ad40 con 0x7f87b41a05e0 2021-02-10T11:41:28.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.028 7f87ab7fe700 1 -- 172.21.2.101:0/2844025500 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879c001b90 con 0x7f87b41a05e0 2021-02-10T11:41:28.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.028 7f87ab7fe700 1 -- 172.21.2.101:0/2844025500 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f879c004590 con 0x7f87b41a05e0 2021-02-10T11:41:28.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.028 7f87b8cb8700 1 -- 172.21.2.101:0/2844025500 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b40bf900 con 0x7f87b41a05e0 2021-02-10T11:41:28.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.028 7f87b8cb8700 1 -- 172.21.2.101:0/2844025500 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b40bfe90 con 0x7f87b41a05e0 2021-02-10T11:41:28.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.029 7f87ab7fe700 1 -- 172.21.2.101:0/2844025500 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f879c007070 con 0x7f87b41a05e0 2021-02-10T11:41:28.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.029 7f87ab7fe700 1 --2- 172.21.2.101:0/2844025500 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87980186c0 0x7f879801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:28.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.029 7f87ab7fe700 1 -- 172.21.2.101:0/2844025500 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f879c032390 con 0x7f87b41a05e0 2021-02-10T11:41:28.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.029 7f87b205d700 1 --2- 172.21.2.101:0/2844025500 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87980186c0 0x7f879801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:28.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.029 7f87b205d700 1 --2- 172.21.2.101:0/2844025500 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87980186c0 0x7f879801abc0 crc :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:28.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.059 7f87b8cb8700 1 -- 172.21.2.101:0/2844025500 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b4099d50 con 0x7f87b41a05e0 2021-02-10T11:41:28.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.061 7f87ab7fe700 1 -- 172.21.2.101:0/2844025500 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f879c000cb0 con 0x7f87b41a05e0 2021-02-10T11:41:28.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.187 7f87b8cb8700 1 -- 172.21.2.101:0/2844025500 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f87b4099d50 con 0x7f87b41a05e0 2021-02-10T11:41:28.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.586 7f87ab7fe700 1 -- 172.21.2.101:0/2844025500 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v115) v1 ==== 173+0+0 (secure 0 0 0) 0x7f879c007050 con 0x7f87b41a05e0 2021-02-10T11:41:28.604 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' removed 2021-02-10T11:41:28.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.603 7f87b8cb8700 1 -- 172.21.2.101:0/2844025500 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87980186c0 msgr2=0x7f879801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:28.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.603 7f87b8cb8700 1 --2- 172.21.2.101:0/2844025500 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87980186c0 0x7f879801abc0 crc :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:28.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.603 7f87b8cb8700 1 -- 172.21.2.101:0/2844025500 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87b41a05e0 msgr2=0x7f87b409a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:28.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.603 7f87b8cb8700 1 --2- 172.21.2.101:0/2844025500 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87b41a05e0 0x7f87b409a290 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 rx=0x7f879c007f20 tx=0x7f879c00a950).stop 2021-02-10T11:41:28.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.603 7f87b8cb8700 1 -- 172.21.2.101:0/2844025500 shutdown_connections 2021-02-10T11:41:28.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.603 7f87b8cb8700 1 --2- 172.21.2.101:0/2844025500 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87980186c0 0x7f879801abc0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:28.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.603 7f87b8cb8700 1 --2- 172.21.2.101:0/2844025500 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87b409b2d0 0x7f87b40beed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:28.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.603 7f87b8cb8700 1 --2- 172.21.2.101:0/2844025500 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87b409a8b0 0x7f87b409acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:28.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.603 7f87b8cb8700 1 --2- 172.21.2.101:0/2844025500 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f87b41a05e0 0x7f87b409a290 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:28.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.603 7f87b8cb8700 1 -- 172.21.2.101:0/2844025500 >> 172.21.2.101:0/2844025500 conn(0x7f87b40a1770 msgr2=0x7f87b409e210 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:28.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.604 7f87b8cb8700 1 -- 172.21.2.101:0/2844025500 shutdown_connections 2021-02-10T11:41:28.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:28.604 7f87b8cb8700 1 -- 172.21.2.101:0/2844025500 wait complete. 2021-02-10T11:41:28.614 INFO:tasks.ceph.mds.a:Restarting daemon 2021-02-10T11:41:28.614 INFO:tasks.ceph.mds.a:Stopping old one... 2021-02-10T11:41:28.615 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-02-10T11:41:28.615 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:41:28.615 INFO:tasks.ceph.mds.b:Restarting daemon 2021-02-10T11:41:28.615 INFO:tasks.ceph.mds.b:Stopping old one... 2021-02-10T11:41:28.615 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-02-10T11:41:28.615 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:41:28.620 INFO:tasks.ceph.mds.a:Stopped 2021-02-10T11:41:28.621 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-02-10T11:41:28.622 INFO:tasks.ceph.mds.b:Stopped 2021-02-10T11:41:28.622 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-02-10T11:41:28.663 INFO:tasks.ceph.mds.b:Started 2021-02-10T11:41:28.663 DEBUG:teuthology.parallel:result is None 2021-02-10T11:41:28.664 INFO:tasks.ceph.mds.a:Started 2021-02-10T11:41:28.664 DEBUG:teuthology.parallel:result is None 2021-02-10T11:41:28.664 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd blacklist clear 2021-02-10T11:41:29.069 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:41:29.072 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:41:29.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.074 7fc26721a700 1 Processor -- start 2021-02-10T11:41:29.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.074 7fc26721a700 1 -- start start 2021-02-10T11:41:29.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.074 7fc26721a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2600d1870 0x7fc2601b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:29.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.075 7fc26721a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc2601b0a10 0x7fc2600cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:29.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.075 7fc26721a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc2600d1c70 0x7fc2600cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:29.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.075 7fc26721a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2600fe230 con 0x7fc2600d1c70 2021-02-10T11:41:29.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.075 7fc26721a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2600e5ed0 con 0x7fc2601b0a10 2021-02-10T11:41:29.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.075 7fc26721a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2600f0b10 con 0x7fc2600d1870 2021-02-10T11:41:29.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.075 7fc265837700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc2600d1c70 0x7fc2600cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:29.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.075 7fc264835700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc2601b0a10 0x7fc2600cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:29.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.075 7fc264835700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc2601b0a10 0x7fc2600cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:39540/0 (socket says 172.21.2.101:39540) 2021-02-10T11:41:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.075 7fc265837700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc2600d1c70 0x7fc2600cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57290/0 (socket says 172.21.2.101:57290) 2021-02-10T11:41:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.075 7fc264835700 1 -- 172.21.2.101:0/4282173848 learned_addr learned my addr 172.21.2.101:0/4282173848 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.075 7fc265036700 1 --2- 172.21.2.101:0/4282173848 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2600d1870 0x7fc2601b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.075 7fc265837700 1 -- 172.21.2.101:0/4282173848 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2600d1870 msgr2=0x7fc2601b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.075 7fc265837700 1 --2- 172.21.2.101:0/4282173848 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2600d1870 0x7fc2601b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.075 7fc265837700 1 -- 172.21.2.101:0/4282173848 >> v2:172.21.2.108:3300/0 conn(0x7fc2601b0a10 msgr2=0x7fc2600cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.075 7fc265837700 1 --2- 172.21.2.101:0/4282173848 >> v2:172.21.2.108:3300/0 conn(0x7fc2601b0a10 0x7fc2600cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.075 7fc265837700 1 -- 172.21.2.101:0/4282173848 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2600cff70 con 0x7fc2600d1c70 2021-02-10T11:41:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.075 7fc265837700 1 --2- 172.21.2.101:0/4282173848 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2600d1c70 0x7fc2600cf950 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 rx=0x7fc258006e30 tx=0x7fc258009e60).ready entity=mon.0 client_cookie=4b99883eeb1af35b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.075 7fc253fff700 1 -- 172.21.2.101:0/4282173848 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc25800b300 con 0x7fc2600d1c70 2021-02-10T11:41:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.075 7fc253fff700 1 -- 172.21.2.101:0/4282173848 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25800b580 con 0x7fc2600d1c70 2021-02-10T11:41:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.075 7fc253fff700 1 -- 172.21.2.101:0/4282173848 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc25800b950 con 0x7fc2600d1c70 2021-02-10T11:41:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.075 7fc26721a700 1 -- 172.21.2.101:0/4282173848 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2600d1c70 msgr2=0x7fc2600cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.075 7fc26721a700 1 --2- 172.21.2.101:0/4282173848 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2600d1c70 0x7fc2600cf950 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 rx=0x7fc258006e30 tx=0x7fc258009e60).stop 2021-02-10T11:41:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.076 7fc26721a700 1 -- 172.21.2.101:0/4282173848 shutdown_connections 2021-02-10T11:41:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.076 7fc26721a700 1 --2- 172.21.2.101:0/4282173848 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2600d1c70 0x7fc2600cf950 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.076 7fc26721a700 1 --2- 172.21.2.101:0/4282173848 >> v2:172.21.2.108:3300/0 conn(0x7fc2601b0a10 0x7fc2600cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.076 7fc26721a700 1 --2- 172.21.2.101:0/4282173848 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2600d1870 0x7fc2601b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.076 7fc26721a700 1 -- 172.21.2.101:0/4282173848 >> 172.21.2.101:0/4282173848 conn(0x7fc2600ce450 msgr2=0x7fc2600d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.076 7fc26721a700 1 -- 172.21.2.101:0/4282173848 shutdown_connections 2021-02-10T11:41:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.076 7fc26721a700 1 -- 172.21.2.101:0/4282173848 wait complete. 2021-02-10T11:41:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.091 7fc26721a700 1 Processor -- start 2021-02-10T11:41:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.091 7fc26721a700 1 -- start start 2021-02-10T11:41:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.091 7fc26721a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2600ebe80 0x7fc2600b8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.091 7fc26721a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2600afa30 0x7fc2600afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.091 7fc26721a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc2600b0450 0x7fc2600ec8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.091 7fc26721a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2600b8d50 con 0x7fc2600afa30 2021-02-10T11:41:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.091 7fc26721a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26009df70 con 0x7fc2600b0450 2021-02-10T11:41:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.091 7fc26721a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2600b0850 con 0x7fc2600ebe80 2021-02-10T11:41:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.092 7fc264835700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2600afa30 0x7fc2600afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.092 7fc264835700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2600afa30 0x7fc2600afe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57296/0 (socket says 172.21.2.101:57296) 2021-02-10T11:41:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.092 7fc264835700 1 -- 172.21.2.101:0/1769414840 learned_addr learned my addr 172.21.2.101:0/1769414840 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.092 7fc265036700 1 --2- 172.21.2.101:0/1769414840 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2600ebe80 0x7fc2600b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.092 7fc265837700 1 --2- 172.21.2.101:0/1769414840 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc2600b0450 0x7fc2600ec8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.092 7fc264835700 1 -- 172.21.2.101:0/1769414840 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2600ebe80 msgr2=0x7fc2600b8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.092 7fc264835700 1 --2- 172.21.2.101:0/1769414840 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2600ebe80 0x7fc2600b8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.092 7fc264835700 1 -- 172.21.2.101:0/1769414840 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc2600b0450 msgr2=0x7fc2600ec8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.092 7fc264835700 1 --2- 172.21.2.101:0/1769414840 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc2600b0450 0x7fc2600ec8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.092 7fc264835700 1 -- 172.21.2.101:0/1769414840 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2600ecef0 con 0x7fc2600afa30 2021-02-10T11:41:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.092 7fc264835700 1 --2- 172.21.2.101:0/1769414840 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2600afa30 0x7fc2600afe30 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 rx=0x7fc25400b580 tx=0x7fc25400b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:29.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.092 7fc251ffb700 1 -- 172.21.2.101:0/1769414840 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc25400bd10 con 0x7fc2600afa30 2021-02-10T11:41:29.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.092 7fc251ffb700 1 -- 172.21.2.101:0/1769414840 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc254002030 con 0x7fc2600afa30 2021-02-10T11:41:29.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.092 7fc26721a700 1 -- 172.21.2.101:0/1769414840 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2600ecef0 con 0x7fc2600afa30 2021-02-10T11:41:29.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.093 7fc251ffb700 1 -- 172.21.2.101:0/1769414840 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc254002550 con 0x7fc2600afa30 2021-02-10T11:41:29.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.093 7fc26721a700 1 -- 172.21.2.101:0/1769414840 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2600ecef0 con 0x7fc2600afa30 2021-02-10T11:41:29.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.093 7fc251ffb700 1 -- 172.21.2.101:0/1769414840 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc254002210 con 0x7fc2600afa30 2021-02-10T11:41:29.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.093 7fc251ffb700 1 --2- 172.21.2.101:0/1769414840 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc240018730 0x7fc24001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:29.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.093 7fc251ffb700 1 -- 172.21.2.101:0/1769414840 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 4839+0+0 (secure 0 0 0) 0x7fc254002210 con 0x7fc2600afa30 2021-02-10T11:41:29.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.093 7fc265036700 1 --2- 172.21.2.101:0/1769414840 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc240018730 0x7fc24001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:29.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.094 7fc265036700 1 --2- 172.21.2.101:0/1769414840 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc240018730 0x7fc24001ac10 crc :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:29.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.123 7fc26721a700 1 -- 172.21.2.101:0/1769414840 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2600b8d50 con 0x7fc2600afa30 2021-02-10T11:41:29.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.126 7fc251ffb700 1 -- 172.21.2.101:0/1769414840 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc254000940 con 0x7fc2600afa30 2021-02-10T11:41:29.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.265 7fc26721a700 1 -- 172.21.2.101:0/1769414840 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fc2600b8d50 con 0x7fc2600afa30 2021-02-10T11:41:29.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.592 7fc251ffb700 1 -- 172.21.2.101:0/1769414840 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v116) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc254001f10 con 0x7fc2600afa30 2021-02-10T11:41:29.633 INFO:teuthology.orchestra.run.gibba001.stderr: removed all blacklist entries 2021-02-10T11:41:29.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.632 7fc26721a700 1 -- 172.21.2.101:0/1769414840 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc240018730 msgr2=0x7fc24001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:29.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.632 7fc26721a700 1 --2- 172.21.2.101:0/1769414840 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc240018730 0x7fc24001ac10 crc :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:29.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.632 7fc26721a700 1 -- 172.21.2.101:0/1769414840 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2600afa30 msgr2=0x7fc2600afe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:29.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.632 7fc26721a700 1 --2- 172.21.2.101:0/1769414840 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2600afa30 0x7fc2600afe30 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 rx=0x7fc25400b580 tx=0x7fc25400b920).stop 2021-02-10T11:41:29.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.633 7fc26721a700 1 -- 172.21.2.101:0/1769414840 shutdown_connections 2021-02-10T11:41:29.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.633 7fc26721a700 1 --2- 172.21.2.101:0/1769414840 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc2600b0450 0x7fc2600ec8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:29.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.633 7fc26721a700 1 --2- 172.21.2.101:0/1769414840 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2600afa30 0x7fc2600afe30 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:29.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.633 7fc26721a700 1 --2- 172.21.2.101:0/1769414840 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc240018730 0x7fc24001ac10 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:29.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.633 7fc26721a700 1 --2- 172.21.2.101:0/1769414840 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2600ebe80 0x7fc2600b8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:29.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.633 7fc26721a700 1 -- 172.21.2.101:0/1769414840 >> 172.21.2.101:0/1769414840 conn(0x7fc2600a6b90 msgr2=0x7fc2600af100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:29.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.633 7fc26721a700 1 -- 172.21.2.101:0/1769414840 shutdown_connections 2021-02-10T11:41:29.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:29.633 7fc26721a700 1 -- 172.21.2.101:0/1769414840 wait complete. 2021-02-10T11:41:29.643 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2021-02-10T11:41:29.644 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:41:30.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.051 7fdd4da80700 1 Processor -- start 2021-02-10T11:41:30.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.051 7fdd4da80700 1 -- start start 2021-02-10T11:41:30.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.052 7fdd4da80700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdd480d1870 0x7fdd481b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:30.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.052 7fdd4da80700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fdd480f0650 0x7fdd480cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:30.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.052 7fdd4da80700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd480f0b10 con 0x7fdd481b0a10 2021-02-10T11:41:30.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.052 7fdd4da80700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd480e5ed0 con 0x7fdd480d1870 2021-02-10T11:41:30.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.052 7fdd4da80700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd480d1c70 con 0x7fdd480f0650 2021-02-10T11:41:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.052 7fdd477fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdd480d1870 0x7fdd481b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.052 7fdd477fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdd480d1870 0x7fdd481b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57300/0 (socket says 172.21.2.101:57300) 2021-02-10T11:41:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.052 7fdd477fe700 1 -- 172.21.2.101:0/563222095 learned_addr learned my addr 172.21.2.101:0/563222095 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.052 7fdd477fe700 1 -- 172.21.2.101:0/563222095 >> v2:172.21.2.108:3300/0 conn(0x7fdd480f0650 msgr2=0x7fdd480cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.052 7fdd477fe700 1 --2- 172.21.2.101:0/563222095 >> v2:172.21.2.108:3300/0 conn(0x7fdd480f0650 0x7fdd480cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.052 7fdd477fe700 1 -- 172.21.2.101:0/563222095 >> v1:172.21.2.108:6789/0 conn(0x7fdd481b0a10 legacy=0x7fdd480cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.052 7fdd477fe700 1 -- 172.21.2.101:0/563222095 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd480d1e90 con 0x7fdd480d1870 2021-02-10T11:41:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.053 7fdd477fe700 1 --2- 172.21.2.101:0/563222095 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd480d1870 0x7fdd481b03f0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 rx=0x7fdd30006700 tx=0x7fdd30009b00).ready entity=mon.0 client_cookie=8c832cb8bdcfb281 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.053 7fdd467fc700 1 -- 172.21.2.101:0/563222095 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd3000c0c0 con 0x7fdd480d1870 2021-02-10T11:41:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.053 7fdd467fc700 1 -- 172.21.2.101:0/563222095 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3000c400 con 0x7fdd480d1870 2021-02-10T11:41:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.053 7fdd467fc700 1 -- 172.21.2.101:0/563222095 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd3000c920 con 0x7fdd480d1870 2021-02-10T11:41:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.053 7fdd4da80700 1 -- 172.21.2.101:0/563222095 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd480d1870 msgr2=0x7fdd481b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.053 7fdd4da80700 1 --2- 172.21.2.101:0/563222095 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd480d1870 0x7fdd481b03f0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 rx=0x7fdd30006700 tx=0x7fdd30009b00).stop 2021-02-10T11:41:30.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.053 7fdd4da80700 1 -- 172.21.2.101:0/563222095 shutdown_connections 2021-02-10T11:41:30.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.053 7fdd4da80700 1 --2- 172.21.2.101:0/563222095 >> v2:172.21.2.108:3300/0 conn(0x7fdd480f0650 0x7fdd480cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:30.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.053 7fdd4da80700 1 --2- 172.21.2.101:0/563222095 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd480d1870 0x7fdd481b03f0 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:30.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.053 7fdd4da80700 1 -- 172.21.2.101:0/563222095 >> 172.21.2.101:0/563222095 conn(0x7fdd480ce450 msgr2=0x7fdd480d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:30.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.054 7fdd4da80700 1 -- 172.21.2.101:0/563222095 shutdown_connections 2021-02-10T11:41:30.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.054 7fdd4da80700 1 -- 172.21.2.101:0/563222095 wait complete. 2021-02-10T11:41:30.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.068 7fdd4da80700 1 Processor -- start 2021-02-10T11:41:30.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.068 7fdd4da80700 1 -- start start 2021-02-10T11:41:30.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.069 7fdd4da80700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd480b0ad0 0x7fdd48091f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:30.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.069 7fdd4da80700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd481a8850 0x7fdd481a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.069 7fdd4da80700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd481a6fa0 0x7fdd481a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.069 7fdd4da80700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd481abfe0 con 0x7fdd481a8850 2021-02-10T11:41:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.069 7fdd4da80700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd481a9c40 con 0x7fdd481a6fa0 2021-02-10T11:41:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.069 7fdd4da80700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd481a9730 con 0x7fdd480b0ad0 2021-02-10T11:41:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.069 7fdd46ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd481a8850 0x7fdd481a8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.069 7fdd46ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd481a8850 0x7fdd481a8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57308/0 (socket says 172.21.2.101:57308) 2021-02-10T11:41:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.069 7fdd46ffd700 1 -- 172.21.2.101:0/4274380791 learned_addr learned my addr 172.21.2.101:0/4274380791 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.069 7fdd46ffd700 1 -- 172.21.2.101:0/4274380791 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd480b0ad0 msgr2=0x7fdd48091f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.069 7fdd46ffd700 1 --2- 172.21.2.101:0/4274380791 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd480b0ad0 0x7fdd48091f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.069 7fdd46ffd700 1 -- 172.21.2.101:0/4274380791 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd481a6fa0 msgr2=0x7fdd481a73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.069 7fdd46ffd700 1 --2- 172.21.2.101:0/4274380791 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd481a6fa0 0x7fdd481a73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:30.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.069 7fdd46ffd700 1 -- 172.21.2.101:0/4274380791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd481a9270 con 0x7fdd481a8850 2021-02-10T11:41:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.069 7fdd46ffd700 1 --2- 172.21.2.101:0/4274380791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd481a8850 0x7fdd481a8c50 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 rx=0x7fdd3800b5b0 tx=0x7fdd3800b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.070 7fdd3ffff700 1 -- 172.21.2.101:0/4274380791 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd3800bd40 con 0x7fdd481a8850 2021-02-10T11:41:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.070 7fdd3ffff700 1 -- 172.21.2.101:0/4274380791 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd38000e20 con 0x7fdd481a8850 2021-02-10T11:41:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.070 7fdd4da80700 1 -- 172.21.2.101:0/4274380791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd481a9270 con 0x7fdd481a8850 2021-02-10T11:41:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.070 7fdd4da80700 1 -- 172.21.2.101:0/4274380791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd481a7cb0 con 0x7fdd481a8850 2021-02-10T11:41:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.070 7fdd3ffff700 1 -- 172.21.2.101:0/4274380791 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd38006210 con 0x7fdd481a8850 2021-02-10T11:41:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.070 7fdd3ffff700 1 -- 172.21.2.101:0/4274380791 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdd38006020 con 0x7fdd481a8850 2021-02-10T11:41:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.070 7fdd3ffff700 1 --2- 172.21.2.101:0/4274380791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd2c0186c0 0x7fdd2c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.070 7fdd3ffff700 1 -- 172.21.2.101:0/4274380791 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fdd380322a0 con 0x7fdd481a8850 2021-02-10T11:41:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.070 7fdd477fe700 1 --2- 172.21.2.101:0/4274380791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd2c0186c0 0x7fdd2c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:30.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.071 7fdd477fe700 1 --2- 172.21.2.101:0/4274380791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd2c0186c0 0x7fdd2c01abc0 crc :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:30.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.102 7fdd4da80700 1 -- 172.21.2.101:0/4274380791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd480ce450 con 0x7fdd481a8850 2021-02-10T11:41:30.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.104 7fdd3ffff700 1 -- 172.21.2.101:0/4274380791 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdd38000940 con 0x7fdd481a8850 2021-02-10T11:41:30.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.234 7fdd4da80700 1 -- 172.21.2.101:0/4274380791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fdd480ce450 con 0x7fdd481a8850 2021-02-10T11:41:30.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.235 7fdd3ffff700 1 -- 172.21.2.101:0/4274380791 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v68) v1 ==== 80+0+4513 (secure 0 0 0) 0x7fdd3800bd40 con 0x7fdd481a8850 2021-02-10T11:41:30.238 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:41:30.238 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:41:30.238 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:41:30.238 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:30.238 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:41:30.238 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:41:30.238 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:41:30.238 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:41:30.238 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:41:30.238 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:41:30.238 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:41:30.238 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:30.239 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:30.239 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:30.239 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:41:30.239 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:41:30.239 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:41:30.239 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:41:30.239 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:41:30.239 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:41:30.239 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:41:30.239 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:30.239 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:30.239 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:30.239 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:41:30.240 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:41:30.240 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:41:30.240 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:41:30.240 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:41:30.240 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:41:30.240 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:30.240 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:30.240 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:30.240 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:41:30.240 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:41:30.240 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:41:30.240 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:41:30.240 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:41:30.241 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:41:30.241 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:30.241 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:30.241 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:30.241 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:41:30.241 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:41:30.241 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:41:30.241 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:41:30.241 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:41:30.241 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:41:30.241 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:30.241 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:30.241 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:30.242 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:41:30.242 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:41:30.242 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:41:30.242 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:41:30.242 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:41:30.242 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:41:30.242 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:30.242 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:30.242 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:30.242 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:41:30.242 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:41:30.242 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:41:30.242 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:41:30.243 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:41:30.243 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:41:30.243 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:30.243 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:30.243 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:30.243 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:41:30.243 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:41:30.243 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:41:30.243 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:41:30.243 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:41:30.243 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:41:30.243 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:30.243 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:30.244 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:30.244 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:41:30.244 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:41:30.244 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:41:30.244 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:41:30.244 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:41:30.244 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:41:30.244 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:30.244 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:30.244 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:30.244 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:41:30.244 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:41:30.244 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:41:30.245 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:41:30.245 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:41:30.245 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:41:30.245 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:30.245 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:30.245 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:30.245 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:41:30.245 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:41:30.245 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:41:30.245 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:41:30.245 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:41:30.245 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:41:30.245 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:30.246 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:30.246 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:30.246 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:41:30.246 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:41:30.246 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:41:30.246 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:41:30.246 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:41:30.246 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:41:30.246 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:30.246 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:30.246 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:30.246 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:41:30.246 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:41:30.247 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:41:30.247 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:41:30.247 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:41:30.247 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:41:30.247 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:30.247 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:30.247 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:30.247 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:41:30.247 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:41:30.247 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:41:30.247 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:41:30.247 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:41:30.247 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:41:30.248 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:30.248 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:30.248 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:30.248 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:41:30.248 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:41:30.248 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:41:30.248 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:41:30.248 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:41:30.248 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:41:30.248 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:41:30.248 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:30.248 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:30.248 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:30.249 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:41:30.249 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:41:30.249 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:41:30.249 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:41:30.249 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:41:30.249 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:41:30.249 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:30.249 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:30.249 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:30.249 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:41:30.249 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:41:30.249 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:41:30.249 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:41:30.250 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:41:30.250 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:41:30.250 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:30.250 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:30.250 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:41:30.250 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:41:30.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.236 7fdd4da80700 1 -- 172.21.2.101:0/4274380791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd2c0186c0 msgr2=0x7fdd2c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:30.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.236 7fdd4da80700 1 --2- 172.21.2.101:0/4274380791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd2c0186c0 0x7fdd2c01abc0 crc :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:30.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.237 7fdd4da80700 1 -- 172.21.2.101:0/4274380791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd481a8850 msgr2=0x7fdd481a8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:30.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.237 7fdd4da80700 1 --2- 172.21.2.101:0/4274380791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd481a8850 0x7fdd481a8c50 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 rx=0x7fdd3800b5b0 tx=0x7fdd3800b950).stop 2021-02-10T11:41:30.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.237 7fdd4da80700 1 -- 172.21.2.101:0/4274380791 shutdown_connections 2021-02-10T11:41:30.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.237 7fdd4da80700 1 --2- 172.21.2.101:0/4274380791 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdd481a6fa0 0x7fdd481a73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:30.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.237 7fdd4da80700 1 --2- 172.21.2.101:0/4274380791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdd481a8850 0x7fdd481a8c50 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:30.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.237 7fdd4da80700 1 --2- 172.21.2.101:0/4274380791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd2c0186c0 0x7fdd2c01abc0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:30.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.237 7fdd4da80700 1 --2- 172.21.2.101:0/4274380791 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdd480b0ad0 0x7fdd48091f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:30.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.237 7fdd4da80700 1 -- 172.21.2.101:0/4274380791 >> 172.21.2.101:0/4274380791 conn(0x7fdd4819f8f0 msgr2=0x7fdd481a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:30.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.237 7fdd4da80700 1 -- 172.21.2.101:0/4274380791 shutdown_connections 2021-02-10T11:41:30.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.237 7fdd4da80700 1 -- 172.21.2.101:0/4274380791 wait complete. 2021-02-10T11:41:30.253 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:41:30.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.664 7f7c6bafe700 1 Processor -- start 2021-02-10T11:41:30.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.664 7f7c6bafe700 1 -- start start 2021-02-10T11:41:30.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.664 7f7c6bafe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7c640d0340 0x7f7c640ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:30.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.664 7f7c6bafe700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c640f0b10 con 0x7f7c640d1c70 2021-02-10T11:41:30.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.664 7f7c6bafe700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c641b0be0 con 0x7f7c640d1870 2021-02-10T11:41:30.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.664 7f7c6bafe700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c640e5ed0 con 0x7f7c640d0340 2021-02-10T11:41:30.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.665 7f7c69119700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7c640d0340 0x7f7c640ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:30.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.665 7f7c6a11b700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f7c640d1c70 0x7f7c640efc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57806/0 (socket says 172.21.2.101:57806) 2021-02-10T11:41:30.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.665 7f7c6a11b700 1 -- 172.21.2.101:0/2704813191 learned_addr learned my addr 172.21.2.101:0/2704813191 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:30.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.665 7f7c69119700 1 -- 172.21.2.101:0/2704813191 >> v1:172.21.2.108:6789/0 conn(0x7f7c640d1870 legacy=0x7f7c641b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:41:30.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.665 7f7c69119700 1 -- 172.21.2.101:0/2704813191 >> v1:172.21.2.101:6789/0 conn(0x7f7c640d1c70 legacy=0x7f7c640efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:30.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.665 7f7c69119700 1 -- 172.21.2.101:0/2704813191 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c640f0450 con 0x7f7c640d0340 2021-02-10T11:41:30.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.665 7f7c69119700 1 --2- 172.21.2.101:0/2704813191 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c640d0340 0x7f7c640ef640 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 rx=0x7f7c58006650 tx=0x7f7c58009a00).ready entity=mon.0 client_cookie=64eb64d3a0a50f66 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:30.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.665 7f7c68918700 1 -- 172.21.2.101:0/2704813191 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7c580009b0 con 0x7f7c640d0340 2021-02-10T11:41:30.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.665 7f7c68918700 1 -- 172.21.2.101:0/2704813191 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5800c0c0 con 0x7f7c640d0340 2021-02-10T11:41:30.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.665 7f7c68918700 1 -- 172.21.2.101:0/2704813191 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7c5800c5a0 con 0x7f7c640d0340 2021-02-10T11:41:30.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.665 7f7c6bafe700 1 -- 172.21.2.101:0/2704813191 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c640d0340 msgr2=0x7f7c640ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:30.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.665 7f7c6bafe700 1 --2- 172.21.2.101:0/2704813191 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c640d0340 0x7f7c640ef640 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 rx=0x7f7c58006650 tx=0x7f7c58009a00).stop 2021-02-10T11:41:30.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.666 7f7c6bafe700 1 -- 172.21.2.101:0/2704813191 shutdown_connections 2021-02-10T11:41:30.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.666 7f7c6bafe700 1 --2- 172.21.2.101:0/2704813191 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c640d0340 0x7f7c640ef640 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:30.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.666 7f7c6bafe700 1 -- 172.21.2.101:0/2704813191 >> 172.21.2.101:0/2704813191 conn(0x7f7c640ce450 msgr2=0x7f7c640d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:30.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.666 7f7c6bafe700 1 -- 172.21.2.101:0/2704813191 shutdown_connections 2021-02-10T11:41:30.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.666 7f7c6bafe700 1 -- 172.21.2.101:0/2704813191 wait complete. 2021-02-10T11:41:30.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.681 7f7c6bafe700 1 Processor -- start 2021-02-10T11:41:30.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.681 7f7c6bafe700 1 -- start start 2021-02-10T11:41:30.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.681 7f7c6bafe700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c640c0060 0x7f7c640c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:30.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.681 7f7c6bafe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c641c69b0 0x7f7c640c3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:30.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.681 7f7c6bafe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c640c3830 0x7f7c640b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:30.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.681 7f7c6bafe700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c640b33f0 con 0x7f7c641c69b0 2021-02-10T11:41:30.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.681 7f7c6bafe700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c640c3c30 con 0x7f7c640c0060 2021-02-10T11:41:30.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.681 7f7c6bafe700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c640bfbc0 con 0x7f7c640c3830 2021-02-10T11:41:30.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.681 7f7c69119700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c641c69b0 0x7f7c640c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:30.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.681 7f7c69119700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c641c69b0 0x7f7c640c3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57314/0 (socket says 172.21.2.101:57314) 2021-02-10T11:41:30.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.681 7f7c69119700 1 -- 172.21.2.101:0/2425151177 learned_addr learned my addr 172.21.2.101:0/2425151177 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:30.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.682 7f7c69119700 1 -- 172.21.2.101:0/2425151177 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c640c3830 msgr2=0x7f7c640b3970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:30.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.682 7f7c69119700 1 --2- 172.21.2.101:0/2425151177 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c640c3830 0x7f7c640b3970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:30.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.682 7f7c69119700 1 -- 172.21.2.101:0/2425151177 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c640c0060 msgr2=0x7f7c640c2bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:30.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.682 7f7c69119700 1 --2- 172.21.2.101:0/2425151177 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c640c0060 0x7f7c640c2bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:30.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.682 7f7c69119700 1 -- 172.21.2.101:0/2425151177 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c640b3f90 con 0x7f7c641c69b0 2021-02-10T11:41:30.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.682 7f7c69119700 1 --2- 172.21.2.101:0/2425151177 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c641c69b0 0x7f7c640c3210 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 rx=0x7f7c5800b5e0 tx=0x7f7c5800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:30.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.682 7f7c567fc700 1 -- 172.21.2.101:0/2425151177 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7c580090c0 con 0x7f7c641c69b0 2021-02-10T11:41:30.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.682 7f7c567fc700 1 -- 172.21.2.101:0/2425151177 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c580009b0 con 0x7f7c641c69b0 2021-02-10T11:41:30.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.682 7f7c567fc700 1 -- 172.21.2.101:0/2425151177 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7c580022e0 con 0x7f7c641c69b0 2021-02-10T11:41:30.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.682 7f7c6bafe700 1 -- 172.21.2.101:0/2425151177 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c640b3f90 con 0x7f7c641c69b0 2021-02-10T11:41:30.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.682 7f7c6bafe700 1 -- 172.21.2.101:0/2425151177 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c640b3f90 con 0x7f7c641c69b0 2021-02-10T11:41:30.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.682 7f7c567fc700 1 -- 172.21.2.101:0/2425151177 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7c58000c90 con 0x7f7c641c69b0 2021-02-10T11:41:30.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.683 7f7c567fc700 1 --2- 172.21.2.101:0/2425151177 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c4c018730 0x7f7c4c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:30.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.683 7f7c567fc700 1 -- 172.21.2.101:0/2425151177 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f7c580322e0 con 0x7f7c641c69b0 2021-02-10T11:41:30.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.683 7f7c6991a700 1 --2- 172.21.2.101:0/2425151177 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c4c018730 0x7f7c4c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:30.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.683 7f7c6991a700 1 --2- 172.21.2.101:0/2425151177 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c4c018730 0x7f7c4c01abc0 crc :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:30.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.714 7f7c6bafe700 1 -- 172.21.2.101:0/2425151177 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c640f06f0 con 0x7f7c641c69b0 2021-02-10T11:41:30.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.717 7f7c567fc700 1 -- 172.21.2.101:0/2425151177 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7c58000940 con 0x7f7c641c69b0 2021-02-10T11:41:30.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.842 7f7c6bafe700 1 -- 172.21.2.101:0/2425151177 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c640f06f0 con 0x7f7c641c69b0 2021-02-10T11:41:30.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.843 7f7c567fc700 1 -- 172.21.2.101:0/2425151177 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+11154 (secure 0 0 0) 0x7f7c58000c90 con 0x7f7c641c69b0 2021-02-10T11:41:30.846 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:41:30.846 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":116,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:41:29.586801","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:41:30.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.845 7f7c6bafe700 1 -- 172.21.2.101:0/2425151177 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c4c018730 msgr2=0x7f7c4c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:30.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.845 7f7c6bafe700 1 --2- 172.21.2.101:0/2425151177 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c4c018730 0x7f7c4c01abc0 crc :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:30.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.845 7f7c6bafe700 1 -- 172.21.2.101:0/2425151177 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c641c69b0 msgr2=0x7f7c640c3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:30.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.845 7f7c6bafe700 1 --2- 172.21.2.101:0/2425151177 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c641c69b0 0x7f7c640c3210 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 rx=0x7f7c5800b5e0 tx=0x7f7c5800b980).stop 2021-02-10T11:41:30.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.845 7f7c6bafe700 1 -- 172.21.2.101:0/2425151177 shutdown_connections 2021-02-10T11:41:30.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.845 7f7c6bafe700 1 --2- 172.21.2.101:0/2425151177 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7c4c018730 0x7f7c4c01abc0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:30.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.845 7f7c6bafe700 1 --2- 172.21.2.101:0/2425151177 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c640c3830 0x7f7c640b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:30.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.845 7f7c6bafe700 1 --2- 172.21.2.101:0/2425151177 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c641c69b0 0x7f7c640c3210 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:30.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.845 7f7c6bafe700 1 --2- 172.21.2.101:0/2425151177 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c640c0060 0x7f7c640c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:30.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.845 7f7c6bafe700 1 -- 172.21.2.101:0/2425151177 >> 172.21.2.101:0/2425151177 conn(0x7f7c6410a820 msgr2=0x7f7c64108150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:30.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.845 7f7c6bafe700 1 -- 172.21.2.101:0/2425151177 shutdown_connections 2021-02-10T11:41:30.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:30.845 7f7c6bafe700 1 -- 172.21.2.101:0/2425151177 wait complete. 2021-02-10T11:41:30.856 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-10T11:41:30.856 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:41:31.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.268 7fe6e72e2700 1 Processor -- start 2021-02-10T11:41:31.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.268 7fe6e72e2700 1 -- start start 2021-02-10T11:41:31.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.268 7fe6e72e2700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe6e00d1870 0x7fe6e01b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:31.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.268 7fe6e72e2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6e01b0ae0 0x7fe6e0108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:31.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.268 7fe6e72e2700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6e00fe1c0 con 0x7fe6e00d1c70 2021-02-10T11:41:31.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.268 7fe6e72e2700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6e0109a60 con 0x7fe6e00d1870 2021-02-10T11:41:31.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.268 7fe6e72e2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e0109750 con 0x7fe6e01b0ae0 2021-02-10T11:41:31.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.269 7fe6e58ff700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fe6e00d1c70 0x7fe6e0108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:51226/0 (socket says 172.21.2.101:51226) 2021-02-10T11:41:31.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.269 7fe6e58ff700 1 -- 172.21.2.101:0/1924827015 learned_addr learned my addr 172.21.2.101:0/1924827015 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:31.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.269 7fe6e48fd700 1 --2- 172.21.2.101:0/1924827015 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6e01b0ae0 0x7fe6e0108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:31.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.269 7fe6e50fe700 1 --2- 172.21.2.101:0/1924827015 >> v2:172.21.2.108:3300/0 conn(0x7fe6e00d1870 0x7fe6e01b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:31.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.269 7fe6e50fe700 1 -- 172.21.2.101:0/1924827015 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6e01b0ae0 msgr2=0x7fe6e0108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:31.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.269 7fe6e50fe700 1 --2- 172.21.2.101:0/1924827015 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6e01b0ae0 0x7fe6e0108940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:31.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.269 7fe6e50fe700 1 -- 172.21.2.101:0/1924827015 >> v1:172.21.2.108:6789/0 conn(0x7fe6e00d1c70 legacy=0x7fe6e0108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:31.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.269 7fe6e50fe700 1 -- 172.21.2.101:0/1924827015 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e0088070 con 0x7fe6e00d1870 2021-02-10T11:41:31.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.270 7fe6e50fe700 1 --2- 172.21.2.101:0/1924827015 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6e00d1870 0x7fe6e01b04c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7fe6d4006650 tx=0x7fe6d4009a80).ready entity=mon.1 client_cookie=10ebf5dda306c43 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:31.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.270 7fe6d3fff700 1 -- 172.21.2.101:0/1924827015 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe6d40009b0 con 0x7fe6e00d1870 2021-02-10T11:41:31.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.270 7fe6d3fff700 1 -- 172.21.2.101:0/1924827015 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d400c0c0 con 0x7fe6e00d1870 2021-02-10T11:41:31.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.270 7fe6d3fff700 1 -- 172.21.2.101:0/1924827015 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe6d400c5e0 con 0x7fe6e00d1870 2021-02-10T11:41:31.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.270 7fe6e72e2700 1 -- 172.21.2.101:0/1924827015 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6e00d1870 msgr2=0x7fe6e01b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:31.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.270 7fe6e72e2700 1 --2- 172.21.2.101:0/1924827015 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6e00d1870 0x7fe6e01b04c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7fe6d4006650 tx=0x7fe6d4009a80).stop 2021-02-10T11:41:31.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.271 7fe6e72e2700 1 -- 172.21.2.101:0/1924827015 shutdown_connections 2021-02-10T11:41:31.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.271 7fe6e72e2700 1 --2- 172.21.2.101:0/1924827015 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6e01b0ae0 0x7fe6e0108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:31.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.271 7fe6e72e2700 1 --2- 172.21.2.101:0/1924827015 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6e00d1870 0x7fe6e01b04c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:31.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.271 7fe6e72e2700 1 -- 172.21.2.101:0/1924827015 >> 172.21.2.101:0/1924827015 conn(0x7fe6e00ce420 msgr2=0x7fe6e00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:31.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.271 7fe6e72e2700 1 -- 172.21.2.101:0/1924827015 shutdown_connections 2021-02-10T11:41:31.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.271 7fe6e72e2700 1 -- 172.21.2.101:0/1924827015 wait complete. 2021-02-10T11:41:31.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.286 7fe6e72e2700 1 Processor -- start 2021-02-10T11:41:31.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.286 7fe6e72e2700 1 -- start start 2021-02-10T11:41:31.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.286 7fe6e72e2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6e00a7950 0x7fe6e00b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:31.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.286 7fe6e72e2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6e01a7490 0x7fe6e01a7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:31.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.286 7fe6e72e2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6e01a7eb0 0x7fe6e00ac630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:31.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.286 7fe6e72e2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e00a7fb0 con 0x7fe6e01a7490 2021-02-10T11:41:31.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.286 7fe6e72e2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e00acc50 con 0x7fe6e01a7eb0 2021-02-10T11:41:31.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.286 7fe6e72e2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e00ace30 con 0x7fe6e00a7950 2021-02-10T11:41:31.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.287 7fe6e48fd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6e01a7490 0x7fe6e01a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:31.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.287 7fe6e48fd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6e01a7490 0x7fe6e01a7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57324/0 (socket says 172.21.2.101:57324) 2021-02-10T11:41:31.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.287 7fe6e48fd700 1 -- 172.21.2.101:0/1459344628 learned_addr learned my addr 172.21.2.101:0/1459344628 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:31.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.287 7fe6e50fe700 1 --2- 172.21.2.101:0/1459344628 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6e00a7950 0x7fe6e00b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:31.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.287 7fe6e48fd700 1 -- 172.21.2.101:0/1459344628 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6e00a7950 msgr2=0x7fe6e00b8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:31.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.287 7fe6e48fd700 1 --2- 172.21.2.101:0/1459344628 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6e00a7950 0x7fe6e00b8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:31.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.287 7fe6e48fd700 1 -- 172.21.2.101:0/1459344628 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6e01a7eb0 msgr2=0x7fe6e00ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:31.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.287 7fe6e48fd700 1 --2- 172.21.2.101:0/1459344628 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6e01a7eb0 0x7fe6e00ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:31.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.287 7fe6e48fd700 1 -- 172.21.2.101:0/1459344628 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e00ad050 con 0x7fe6e01a7490 2021-02-10T11:41:31.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.287 7fe6e50fe700 1 --2- 172.21.2.101:0/1459344628 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6e00a7950 0x7fe6e00b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:41:31.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.287 7fe6e48fd700 1 --2- 172.21.2.101:0/1459344628 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6e01a7490 0x7fe6e01a7890 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 rx=0x7fe6dc006f60 tx=0x7fe6dc00b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:31.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.288 7fe6d1ffb700 1 -- 172.21.2.101:0/1459344628 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe6dc00bd60 con 0x7fe6e01a7490 2021-02-10T11:41:31.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.288 7fe6d1ffb700 1 -- 172.21.2.101:0/1459344628 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6dc000c90 con 0x7fe6e01a7490 2021-02-10T11:41:31.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.288 7fe6d1ffb700 1 -- 172.21.2.101:0/1459344628 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe6dc0022b0 con 0x7fe6e01a7490 2021-02-10T11:41:31.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.288 7fe6e72e2700 1 -- 172.21.2.101:0/1459344628 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e00ad050 con 0x7fe6e01a7490 2021-02-10T11:41:31.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.288 7fe6e72e2700 1 -- 172.21.2.101:0/1459344628 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e00a8530 con 0x7fe6e01a7490 2021-02-10T11:41:31.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.288 7fe6d1ffb700 1 -- 172.21.2.101:0/1459344628 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe6dc002020 con 0x7fe6e01a7490 2021-02-10T11:41:31.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.288 7fe6d1ffb700 1 --2- 172.21.2.101:0/1459344628 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6c00186c0 0x7fe6c001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:31.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.288 7fe6d1ffb700 1 -- 172.21.2.101:0/1459344628 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fe6dc032220 con 0x7fe6e01a7490 2021-02-10T11:41:31.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.289 7fe6e50fe700 1 --2- 172.21.2.101:0/1459344628 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6c00186c0 0x7fe6c001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:31.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.289 7fe6e50fe700 1 --2- 172.21.2.101:0/1459344628 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6c00186c0 0x7fe6c001abc0 crc :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:31.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.318 7fe6e72e2700 1 -- 172.21.2.101:0/1459344628 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e0082a70 con 0x7fe6e01a7490 2021-02-10T11:41:31.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.321 7fe6d1ffb700 1 -- 172.21.2.101:0/1459344628 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe6dc0009b0 con 0x7fe6e01a7490 2021-02-10T11:41:31.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.449 7fe6e72e2700 1 -- 172.21.2.101:0/1459344628 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fe6e0082a70 con 0x7fe6e01a7490 2021-02-10T11:41:31.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.450 7fe6d1ffb700 1 -- 172.21.2.101:0/1459344628 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v116) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe6dc00bd60 con 0x7fe6e01a7490 2021-02-10T11:41:31.453 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:41:31.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.452 7fe6e72e2700 1 -- 172.21.2.101:0/1459344628 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6c00186c0 msgr2=0x7fe6c001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:31.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.452 7fe6e72e2700 1 --2- 172.21.2.101:0/1459344628 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6c00186c0 0x7fe6c001abc0 crc :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:31.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.452 7fe6e72e2700 1 -- 172.21.2.101:0/1459344628 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6e01a7490 msgr2=0x7fe6e01a7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:31.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.452 7fe6e72e2700 1 --2- 172.21.2.101:0/1459344628 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6e01a7490 0x7fe6e01a7890 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 rx=0x7fe6dc006f60 tx=0x7fe6dc00b970).stop 2021-02-10T11:41:31.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.452 7fe6e72e2700 1 -- 172.21.2.101:0/1459344628 shutdown_connections 2021-02-10T11:41:31.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.452 7fe6e72e2700 1 --2- 172.21.2.101:0/1459344628 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6e01a7eb0 0x7fe6e00ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:31.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.452 7fe6e72e2700 1 --2- 172.21.2.101:0/1459344628 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6e01a7490 0x7fe6e01a7890 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:31.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.452 7fe6e72e2700 1 --2- 172.21.2.101:0/1459344628 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6c00186c0 0x7fe6c001abc0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:31.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.452 7fe6e72e2700 1 --2- 172.21.2.101:0/1459344628 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6e00a7950 0x7fe6e00b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:31.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.452 7fe6e72e2700 1 -- 172.21.2.101:0/1459344628 >> 172.21.2.101:0/1459344628 conn(0x7fe6e00a02d0 msgr2=0x7fe6e009f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:31.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.452 7fe6e72e2700 1 -- 172.21.2.101:0/1459344628 shutdown_connections 2021-02-10T11:41:31.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.452 7fe6e72e2700 1 -- 172.21.2.101:0/1459344628 wait complete. 2021-02-10T11:41:31.462 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --format=json-pretty osd lspools 2021-02-10T11:41:31.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.870 7f1f7b91f700 1 Processor -- start 2021-02-10T11:41:31.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.870 7f1f7b91f700 1 -- start start 2021-02-10T11:41:31.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.871 7f1f7b91f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1f740d1870 0x7f1f741b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:31.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.871 7f1f7b91f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1f740f0650 0x7f1f740cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:31.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.871 7f1f7b91f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f740f0b10 con 0x7f1f741b0a10 2021-02-10T11:41:31.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.871 7f1f7b91f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f740e5ed0 con 0x7f1f740f0650 2021-02-10T11:41:31.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.871 7f1f7b91f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f740d1c70 con 0x7f1f740d1870 2021-02-10T11:41:31.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.871 7f1f78f3a700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f1f741b0a10 0x7f1f740cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57826/0 (socket says 172.21.2.101:57826) 2021-02-10T11:41:31.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.871 7f1f78f3a700 1 -- 172.21.2.101:0/548233436 learned_addr learned my addr 172.21.2.101:0/548233436 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:31.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.871 7f1f79f3c700 1 --2- 172.21.2.101:0/548233436 >> v2:172.21.2.101:3300/0 conn(0x7f1f740f0650 0x7f1f740cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:31.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.871 7f1f67fff700 1 -- 172.21.2.101:0/548233436 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3416634280 0 0) 0x7f1f680010b0 con 0x7f1f741b0a10 2021-02-10T11:41:31.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.871 7f1f67fff700 1 -- 172.21.2.101:0/548233436 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f60002fe0 con 0x7f1f741b0a10 2021-02-10T11:41:31.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.871 7f1f79f3c700 1 -- 172.21.2.101:0/548233436 >> v2:172.21.2.108:3300/0 conn(0x7f1f740d1870 msgr2=0x7f1f741b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:31.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.871 7f1f79f3c700 1 --2- 172.21.2.101:0/548233436 >> v2:172.21.2.108:3300/0 conn(0x7f1f740d1870 0x7f1f741b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:31.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.871 7f1f79f3c700 1 -- 172.21.2.101:0/548233436 >> v1:172.21.2.101:6789/0 conn(0x7f1f741b0a10 legacy=0x7f1f740cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:31.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.871 7f1f79f3c700 1 -- 172.21.2.101:0/548233436 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f740d1e90 con 0x7f1f740f0650 2021-02-10T11:41:31.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.872 7f1f79f3c700 1 --2- 172.21.2.101:0/548233436 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f740f0650 0x7f1f740cfb20 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 rx=0x7f1f6c006e30 tx=0x7f1f6c009e60).ready entity=mon.0 client_cookie=de3b1f48fe1cda8b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:31.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.872 7f1f67fff700 1 -- 172.21.2.101:0/548233436 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f6c006a50 con 0x7f1f740f0650 2021-02-10T11:41:31.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.872 7f1f67fff700 1 -- 172.21.2.101:0/548233436 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6c00b300 con 0x7f1f740f0650 2021-02-10T11:41:31.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.872 7f1f67fff700 1 -- 172.21.2.101:0/548233436 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f6c00b820 con 0x7f1f740f0650 2021-02-10T11:41:31.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.872 7f1f7b91f700 1 -- 172.21.2.101:0/548233436 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f740f0650 msgr2=0x7f1f740cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:31.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.872 7f1f7b91f700 1 --2- 172.21.2.101:0/548233436 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f740f0650 0x7f1f740cfb20 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 rx=0x7f1f6c006e30 tx=0x7f1f6c009e60).stop 2021-02-10T11:41:31.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.873 7f1f7b91f700 1 -- 172.21.2.101:0/548233436 shutdown_connections 2021-02-10T11:41:31.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.873 7f1f7b91f700 1 --2- 172.21.2.101:0/548233436 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f740f0650 0x7f1f740cfb20 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:31.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.873 7f1f7b91f700 1 --2- 172.21.2.101:0/548233436 >> v2:172.21.2.108:3300/0 conn(0x7f1f740d1870 0x7f1f741b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:31.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.873 7f1f7b91f700 1 -- 172.21.2.101:0/548233436 >> 172.21.2.101:0/548233436 conn(0x7f1f740ce450 msgr2=0x7f1f740d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:31.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.873 7f1f7b91f700 1 -- 172.21.2.101:0/548233436 shutdown_connections 2021-02-10T11:41:31.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.873 7f1f7b91f700 1 -- 172.21.2.101:0/548233436 wait complete. 2021-02-10T11:41:31.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.887 7f1f7b91f700 1 Processor -- start 2021-02-10T11:41:31.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.887 7f1f7b91f700 1 -- start start 2021-02-10T11:41:31.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.888 7f1f7b91f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f74195650 0x7f1f740b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:31.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.888 7f1f7b91f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f740b5b80 0x7f1f741a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:31.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.888 7f1f7b91f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f740acfa0 0x7f1f741a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:31.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.888 7f1f7b91f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740b6010 con 0x7f1f74195650 2021-02-10T11:41:31.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.888 7f1f7b91f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f741a3a60 con 0x7f1f740b5b80 2021-02-10T11:41:31.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.888 7f1f7b91f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f741a3c40 con 0x7f1f740acfa0 2021-02-10T11:41:31.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.888 7f1f7973b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f74195650 0x7f1f740b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:31.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.888 7f1f7973b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f74195650 0x7f1f740b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57334/0 (socket says 172.21.2.101:57334) 2021-02-10T11:41:31.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.888 7f1f7973b700 1 -- 172.21.2.101:0/1494562639 learned_addr learned my addr 172.21.2.101:0/1494562639 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:31.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.889 7f1f7973b700 1 -- 172.21.2.101:0/1494562639 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f740acfa0 msgr2=0x7f1f741a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:31.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.889 7f1f7973b700 1 --2- 172.21.2.101:0/1494562639 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f740acfa0 0x7f1f741a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:31.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.889 7f1f7973b700 1 -- 172.21.2.101:0/1494562639 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f740b5b80 msgr2=0x7f1f741a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:31.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.889 7f1f7973b700 1 --2- 172.21.2.101:0/1494562639 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f740b5b80 0x7f1f741a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:31.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.889 7f1f7973b700 1 -- 172.21.2.101:0/1494562639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f740bcae0 con 0x7f1f74195650 2021-02-10T11:41:31.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.889 7f1f7973b700 1 --2- 172.21.2.101:0/1494562639 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f74195650 0x7f1f740b5560 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 rx=0x7f1f70006f00 tx=0x7f1f7000b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:31.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.889 7f1f65ffb700 1 -- 172.21.2.101:0/1494562639 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f7000bd30 con 0x7f1f74195650 2021-02-10T11:41:31.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.889 7f1f65ffb700 1 -- 172.21.2.101:0/1494562639 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f70000c90 con 0x7f1f74195650 2021-02-10T11:41:31.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.889 7f1f65ffb700 1 -- 172.21.2.101:0/1494562639 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f700022a0 con 0x7f1f74195650 2021-02-10T11:41:31.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.889 7f1f7b91f700 1 -- 172.21.2.101:0/1494562639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f740bcae0 con 0x7f1f74195650 2021-02-10T11:41:31.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.889 7f1f7b91f700 1 -- 172.21.2.101:0/1494562639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f740bd070 con 0x7f1f74195650 2021-02-10T11:41:31.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.890 7f1f65ffb700 1 -- 172.21.2.101:0/1494562639 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1f70005310 con 0x7f1f74195650 2021-02-10T11:41:31.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.890 7f1f65ffb700 1 --2- 172.21.2.101:0/1494562639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f540186c0 0x7f1f5401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:31.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.890 7f1f65ffb700 1 -- 172.21.2.101:0/1494562639 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f1f70032100 con 0x7f1f74195650 2021-02-10T11:41:31.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.890 7f1f78f3a700 1 --2- 172.21.2.101:0/1494562639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f540186c0 0x7f1f5401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:31.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.891 7f1f78f3a700 1 --2- 172.21.2.101:0/1494562639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f540186c0 0x7f1f5401abc0 crc :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:31.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.921 7f1f7b91f700 1 -- 172.21.2.101:0/1494562639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f740f0ab0 con 0x7f1f74195650 2021-02-10T11:41:31.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:31.924 7f1f65ffb700 1 -- 172.21.2.101:0/1494562639 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1f7002cf30 con 0x7f1f74195650 2021-02-10T11:41:32.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.049 7f1f7b91f700 1 -- 172.21.2.101:0/1494562639 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1f740f0ab0 con 0x7f1f74195650 2021-02-10T11:41:32.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.050 7f1f65ffb700 1 -- 172.21.2.101:0/1494562639 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v116) v1 ==== 84+0+64 (secure 0 0 0) 0x7f1f7002cf30 con 0x7f1f74195650 2021-02-10T11:41:32.052 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:41:32.053 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:41:32.053 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:41:32.053 INFO:teuthology.orchestra.run.gibba001.stdout: "poolnum": 1, 2021-02-10T11:41:32.053 INFO:teuthology.orchestra.run.gibba001.stdout: "poolname": "rbd" 2021-02-10T11:41:32.053 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:32.053 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:41:32.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.052 7f1f7b91f700 1 -- 172.21.2.101:0/1494562639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f540186c0 msgr2=0x7f1f5401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:32.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.052 7f1f7b91f700 1 --2- 172.21.2.101:0/1494562639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f540186c0 0x7f1f5401abc0 crc :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:32.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.052 7f1f7b91f700 1 -- 172.21.2.101:0/1494562639 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f74195650 msgr2=0x7f1f740b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:32.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.052 7f1f7b91f700 1 --2- 172.21.2.101:0/1494562639 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f74195650 0x7f1f740b5560 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 rx=0x7f1f70006f00 tx=0x7f1f7000b940).stop 2021-02-10T11:41:32.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.052 7f1f7b91f700 1 -- 172.21.2.101:0/1494562639 shutdown_connections 2021-02-10T11:41:32.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.052 7f1f7b91f700 1 --2- 172.21.2.101:0/1494562639 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f540186c0 0x7f1f5401abc0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:32.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.052 7f1f7b91f700 1 --2- 172.21.2.101:0/1494562639 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f740acfa0 0x7f1f741a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:32.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.052 7f1f7b91f700 1 --2- 172.21.2.101:0/1494562639 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f740b5b80 0x7f1f741a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:32.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.052 7f1f7b91f700 1 --2- 172.21.2.101:0/1494562639 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f74195650 0x7f1f740b5560 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:32.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.052 7f1f7b91f700 1 -- 172.21.2.101:0/1494562639 >> 172.21.2.101:0/1494562639 conn(0x7f1f740b3040 msgr2=0x7f1f740c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:32.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.052 7f1f7b91f700 1 -- 172.21.2.101:0/1494562639 shutdown_connections 2021-02-10T11:41:32.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.052 7f1f7b91f700 1 -- 172.21.2.101:0/1494562639 wait complete. 2021-02-10T11:41:32.062 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-02-10T11:41:32.063 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-02-10T11:41:32.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.475 7ff030c4c700 1 Processor -- start 2021-02-10T11:41:32.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.475 7ff030c4c700 1 -- start start 2021-02-10T11:41:32.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.476 7ff030c4c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff02c0d1870 0x7ff02c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:32.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.476 7ff030c4c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff02c0fe1c0 con 0x7ff02c0cff70 2021-02-10T11:41:32.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.476 7ff030c4c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff02c109a60 con 0x7ff02c1b0ae0 2021-02-10T11:41:32.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.476 7ff030c4c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff02c088070 con 0x7ff02c0d1870 2021-02-10T11:41:32.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.476 7ff02b05f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff02c0cff70 0x7ff02c109130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57836/0 (socket says 172.21.2.101:57836) 2021-02-10T11:41:32.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.476 7ff02b05f700 1 -- 172.21.2.101:0/1800937415 learned_addr learned my addr 172.21.2.101:0/1800937415 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:32.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.477 7ff02985c700 1 -- 172.21.2.101:0/1800937415 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 751154248 0 0) 0x7ff024001500 con 0x7ff02c0cff70 2021-02-10T11:41:32.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.477 7ff02985c700 1 -- 172.21.2.101:0/1800937415 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff014002fe0 con 0x7ff02c0cff70 2021-02-10T11:41:32.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.477 7ff02a85e700 1 --2- 172.21.2.101:0/1800937415 >> v2:172.21.2.108:3300/0 conn(0x7ff02c0d1870 0x7ff02c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:32.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.477 7ff02985c700 1 -- 172.21.2.101:0/1800937415 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1832548277 0 0) 0x7ff024001500 con 0x7ff02c0cff70 2021-02-10T11:41:32.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.477 7ff02985c700 1 -- 172.21.2.101:0/1800937415 >> v2:172.21.2.108:3300/0 conn(0x7ff02c0d1870 msgr2=0x7ff02c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:32.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.477 7ff02985c700 1 --2- 172.21.2.101:0/1800937415 >> v2:172.21.2.108:3300/0 conn(0x7ff02c0d1870 0x7ff02c1b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:32.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.477 7ff02985c700 1 -- 172.21.2.101:0/1800937415 >> v1:172.21.2.108:6789/0 conn(0x7ff02c1b0ae0 legacy=0x7ff02c108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:32.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.477 7ff02985c700 1 -- 172.21.2.101:0/1800937415 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02c088260 con 0x7ff02c0cff70 2021-02-10T11:41:32.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.477 7ff02985c700 1 -- 172.21.2.101:0/1800937415 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7ff024001a00 con 0x7ff02c0cff70 2021-02-10T11:41:32.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.477 7ff02985c700 1 -- 172.21.2.101:0/1800937415 >> v1:172.21.2.101:6789/0 conn(0x7ff02c0cff70 legacy=0x7ff02c109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:32.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.477 7ff02985c700 1 --2- 172.21.2.101:0/1800937415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff014003a60 0x7ff014005ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:32.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.477 7ff02985c700 1 --2- 172.21.2.101:0/1800937415 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff014006770 0x7ff014008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:32.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.477 7ff02985c700 1 --2- 172.21.2.101:0/1800937415 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff014009590 0x7ff01400bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:32.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.477 7ff02985c700 1 -- 172.21.2.101:0/1800937415 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01400c300 con 0x7ff014009590 2021-02-10T11:41:32.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.477 7ff02985c700 1 -- 172.21.2.101:0/1800937415 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01400c520 con 0x7ff014006770 2021-02-10T11:41:32.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.477 7ff02985c700 1 -- 172.21.2.101:0/1800937415 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff01400c740 con 0x7ff014003a60 2021-02-10T11:41:32.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.477 7ff02b05f700 1 --2- 172.21.2.101:0/1800937415 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff014009590 0x7ff01400bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:32.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.477 7ff02a85e700 1 --2- 172.21.2.101:0/1800937415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff014003a60 0x7ff014005ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:32.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.478 7ff02a05d700 1 --2- 172.21.2.101:0/1800937415 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff014006770 0x7ff014008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:32.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.478 7ff02a85e700 1 -- 172.21.2.101:0/1800937415 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff014006770 msgr2=0x7ff014008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:32.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.478 7ff02a85e700 1 --2- 172.21.2.101:0/1800937415 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff014006770 0x7ff014008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:32.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.478 7ff02a85e700 1 -- 172.21.2.101:0/1800937415 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff014009590 msgr2=0x7ff01400bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:32.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.478 7ff02a85e700 1 --2- 172.21.2.101:0/1800937415 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff014009590 0x7ff01400bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:32.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.478 7ff02a85e700 1 -- 172.21.2.101:0/1800937415 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0140509b0 con 0x7ff014003a60 2021-02-10T11:41:32.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.478 7ff02b05f700 1 --2- 172.21.2.101:0/1800937415 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff014009590 0x7ff01400bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:41:32.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.478 7ff02b05f700 1 -- 172.21.2.101:0/1800937415 reap_dead start 2021-02-10T11:41:32.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.478 7ff02a85e700 1 --2- 172.21.2.101:0/1800937415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff014003a60 0x7ff014005ed0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7ff01c00c8f0 tx=0x7ff01c00cc90).ready entity=mon.2 client_cookie=6e27929e1778b35 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:32.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.478 7ff02985c700 1 -- 172.21.2.101:0/1800937415 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff01c009310 con 0x7ff014003a60 2021-02-10T11:41:32.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.478 7ff02985c700 1 -- 172.21.2.101:0/1800937415 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01c009610 con 0x7ff014003a60 2021-02-10T11:41:32.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.478 7ff02985c700 1 -- 172.21.2.101:0/1800937415 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff01c009b30 con 0x7ff014003a60 2021-02-10T11:41:32.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.478 7ff030c4c700 1 -- 172.21.2.101:0/1800937415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff014003a60 msgr2=0x7ff014005ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:32.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.478 7ff030c4c700 1 --2- 172.21.2.101:0/1800937415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff014003a60 0x7ff014005ed0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7ff01c00c8f0 tx=0x7ff01c00cc90).stop 2021-02-10T11:41:32.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.479 7ff030c4c700 1 -- 172.21.2.101:0/1800937415 shutdown_connections 2021-02-10T11:41:32.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.479 7ff030c4c700 1 --2- 172.21.2.101:0/1800937415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff014003a60 0x7ff014005ed0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:32.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.479 7ff030c4c700 1 -- 172.21.2.101:0/1800937415 >> 172.21.2.101:0/1800937415 conn(0x7ff02c0ce420 msgr2=0x7ff02c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:32.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.479 7ff030c4c700 1 -- 172.21.2.101:0/1800937415 shutdown_connections 2021-02-10T11:41:32.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.479 7ff030c4c700 1 -- 172.21.2.101:0/1800937415 wait complete. 2021-02-10T11:41:32.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.494 7ff030c4c700 1 Processor -- start 2021-02-10T11:41:32.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.494 7ff030c4c700 1 -- start start 2021-02-10T11:41:32.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.494 7ff030c4c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff02c085bc0 0x7ff02c0863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:32.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.494 7ff030c4c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff02c0954c0 0x7ff02c096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:32.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.494 7ff030c4c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff02c0958c0 0x7ff02c096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:32.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.494 7ff030c4c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c095d80 con 0x7ff02c0958c0 2021-02-10T11:41:32.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.494 7ff030c4c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c0bfa60 con 0x7ff02c0954c0 2021-02-10T11:41:32.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.494 7ff030c4c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c085fc0 con 0x7ff02c085bc0 2021-02-10T11:41:32.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.495 7ff02b05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff02c0958c0 0x7ff02c096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:32.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.495 7ff02b05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff02c0958c0 0x7ff02c096920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57356/0 (socket says 172.21.2.101:57356) 2021-02-10T11:41:32.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.495 7ff02b05f700 1 -- 172.21.2.101:0/3060009611 learned_addr learned my addr 172.21.2.101:0/3060009611 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:32.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.495 7ff02b05f700 1 -- 172.21.2.101:0/3060009611 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff02c085bc0 msgr2=0x7ff02c0863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:32.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.495 7ff02b05f700 1 --2- 172.21.2.101:0/3060009611 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff02c085bc0 0x7ff02c0863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:32.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.495 7ff02b05f700 1 -- 172.21.2.101:0/3060009611 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff02c0954c0 msgr2=0x7ff02c096300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:32.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.495 7ff02b05f700 1 --2- 172.21.2.101:0/3060009611 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff02c0954c0 0x7ff02c096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:32.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.495 7ff02b05f700 1 -- 172.21.2.101:0/3060009611 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02c086c10 con 0x7ff02c0958c0 2021-02-10T11:41:32.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.495 7ff02b05f700 1 --2- 172.21.2.101:0/3060009611 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff02c0958c0 0x7ff02c096920 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 rx=0x7ff02400a920 tx=0x7ff02400acc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:32.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.496 7ff01b7fe700 1 -- 172.21.2.101:0/3060009611 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff024001270 con 0x7ff02c0958c0 2021-02-10T11:41:32.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.496 7ff01b7fe700 1 -- 172.21.2.101:0/3060009611 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0240070d0 con 0x7ff02c0958c0 2021-02-10T11:41:32.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.496 7ff01b7fe700 1 -- 172.21.2.101:0/3060009611 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff024007520 con 0x7ff02c0958c0 2021-02-10T11:41:32.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.496 7ff030c4c700 1 -- 172.21.2.101:0/3060009611 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff02c086c10 con 0x7ff02c0958c0 2021-02-10T11:41:32.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.496 7ff030c4c700 1 -- 172.21.2.101:0/3060009611 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff02c086c10 con 0x7ff02c0958c0 2021-02-10T11:41:32.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.496 7ff01b7fe700 1 -- 172.21.2.101:0/3060009611 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff024004720 con 0x7ff02c0958c0 2021-02-10T11:41:32.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.496 7ff01b7fe700 1 --2- 172.21.2.101:0/3060009611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0100186c0 0x7ff01001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:32.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.496 7ff01b7fe700 1 -- 172.21.2.101:0/3060009611 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4753+0+0 (secure 0 0 0) 0x7ff0240328e0 con 0x7ff02c0958c0 2021-02-10T11:41:32.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.497 7ff02a85e700 1 --2- 172.21.2.101:0/3060009611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0100186c0 0x7ff01001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:32.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.497 7ff02a85e700 1 --2- 172.21.2.101:0/3060009611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0100186c0 0x7ff01001abc0 crc :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:32.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.526 7ff030c4c700 1 -- 172.21.2.101:0/3060009611 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff02c0fe1c0 con 0x7ff02c0958c0 2021-02-10T11:41:32.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.528 7ff01b7fe700 1 -- 172.21.2.101:0/3060009611 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff0240070b0 con 0x7ff02c0958c0 2021-02-10T11:41:32.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:32.658 7ff030c4c700 1 -- 172.21.2.101:0/3060009611 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"} v 0) v1 -- 0x7ff02c0fe1c0 con 0x7ff02c0958c0 2021-02-10T11:41:33.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.079 7ff01b7fe700 1 -- 172.21.2.101:0/3060009611 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"}]=0 pool 'cephfs_metadata' created v117) v1 ==== 133+0+0 (secure 0 0 0) 0x7ff024004720 con 0x7ff02c0958c0 2021-02-10T11:41:33.126 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' created 2021-02-10T11:41:33.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.126 7ff030c4c700 1 -- 172.21.2.101:0/3060009611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0100186c0 msgr2=0x7ff01001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:33.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.126 7ff030c4c700 1 --2- 172.21.2.101:0/3060009611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0100186c0 0x7ff01001abc0 crc :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:33.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.126 7ff030c4c700 1 -- 172.21.2.101:0/3060009611 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff02c0958c0 msgr2=0x7ff02c096920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:33.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.126 7ff030c4c700 1 --2- 172.21.2.101:0/3060009611 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff02c0958c0 0x7ff02c096920 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 rx=0x7ff02400a920 tx=0x7ff02400acc0).stop 2021-02-10T11:41:33.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.126 7ff030c4c700 1 -- 172.21.2.101:0/3060009611 shutdown_connections 2021-02-10T11:41:33.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.126 7ff030c4c700 1 --2- 172.21.2.101:0/3060009611 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff02c0958c0 0x7ff02c096920 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:33.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.126 7ff030c4c700 1 --2- 172.21.2.101:0/3060009611 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff02c0954c0 0x7ff02c096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:33.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.126 7ff030c4c700 1 --2- 172.21.2.101:0/3060009611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0100186c0 0x7ff01001abc0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:33.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.126 7ff030c4c700 1 --2- 172.21.2.101:0/3060009611 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff02c085bc0 0x7ff02c0863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:33.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.126 7ff030c4c700 1 -- 172.21.2.101:0/3060009611 >> 172.21.2.101:0/3060009611 conn(0x7ff02c0bef00 msgr2=0x7ff02c09d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:33.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.126 7ff030c4c700 1 -- 172.21.2.101:0/3060009611 shutdown_connections 2021-02-10T11:41:33.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.126 7ff030c4c700 1 -- 172.21.2.101:0/3060009611 wait complete. 2021-02-10T11:41:33.137 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_data 8 2021-02-10T11:41:33.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.548 7f3431ad4700 1 Processor -- start 2021-02-10T11:41:33.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.548 7f3431ad4700 1 -- start start 2021-02-10T11:41:33.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.548 7f3431ad4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f342c10afe0 0x7f342c0cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:33.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.548 7f3431ad4700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f342c0fe1b0 con 0x7f342c0d1c60 2021-02-10T11:41:33.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.548 7f3431ad4700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f342c1b0ca0 con 0x7f342c0d1860 2021-02-10T11:41:33.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.548 7f3431ad4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342c109a50 con 0x7f342c10afe0 2021-02-10T11:41:33.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.548 7f342affd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f342c0d1c60 0x7f342c0cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57854/0 (socket says 172.21.2.101:57854) 2021-02-10T11:41:33.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.549 7f342affd700 1 -- 172.21.2.101:0/3222805141 learned_addr learned my addr 172.21.2.101:0/3222805141 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:33.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.549 7f342a7fc700 1 -- 172.21.2.101:0/3222805141 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3760393106 0 0) 0x7f341c0010b0 con 0x7f342c0d1c60 2021-02-10T11:41:33.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.549 7f342a7fc700 1 -- 172.21.2.101:0/3222805141 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3418002fe0 con 0x7f342c0d1c60 2021-02-10T11:41:33.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.549 7f342bfff700 1 --2- 172.21.2.101:0/3222805141 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f342c10afe0 0x7f342c0cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:33.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.549 7f342a7fc700 1 -- 172.21.2.101:0/3222805141 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1020209976 0 0) 0x7f341c0010b0 con 0x7f342c0d1c60 2021-02-10T11:41:33.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.549 7f342a7fc700 1 -- 172.21.2.101:0/3222805141 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f342c10afe0 msgr2=0x7f342c0cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:33.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.549 7f342a7fc700 1 --2- 172.21.2.101:0/3222805141 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f342c10afe0 0x7f342c0cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:33.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.549 7f342a7fc700 1 -- 172.21.2.101:0/3222805141 >> v1:172.21.2.108:6789/0 conn(0x7f342c0d1860 legacy=0x7f342c1b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:33.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.549 7f342a7fc700 1 -- 172.21.2.101:0/3222805141 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c10b3e0 con 0x7f342c0d1c60 2021-02-10T11:41:33.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.549 7f342a7fc700 1 -- 172.21.2.101:0/3222805141 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f341c0015b0 con 0x7f342c0d1c60 2021-02-10T11:41:33.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.549 7f342a7fc700 1 -- 172.21.2.101:0/3222805141 >> v1:172.21.2.101:6789/0 conn(0x7f342c0d1c60 legacy=0x7f342c0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:33.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.549 7f342a7fc700 1 --2- 172.21.2.101:0/3222805141 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3418003b90 0x7f34180060b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:33.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.549 7f342a7fc700 1 --2- 172.21.2.101:0/3222805141 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f34180067f0 0x7f3418008ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:33.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.549 7f342a7fc700 1 --2- 172.21.2.101:0/3222805141 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f34180095c0 0x7f341800ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:33.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.549 7f342a7fc700 1 -- 172.21.2.101:0/3222805141 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341800c2c0 con 0x7f34180067f0 2021-02-10T11:41:33.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.549 7f342a7fc700 1 -- 172.21.2.101:0/3222805141 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341800c4e0 con 0x7f3418003b90 2021-02-10T11:41:33.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.549 7f342a7fc700 1 -- 172.21.2.101:0/3222805141 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341800c700 con 0x7f34180095c0 2021-02-10T11:41:33.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.549 7f342affd700 1 --2- 172.21.2.101:0/3222805141 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f34180067f0 0x7f3418008ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:33.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.550 7f342bfff700 1 --2- 172.21.2.101:0/3222805141 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f34180095c0 0x7f341800ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:33.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.550 7f342b7fe700 1 --2- 172.21.2.101:0/3222805141 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3418003b90 0x7f34180060b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:33.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.550 7f342affd700 1 -- 172.21.2.101:0/3222805141 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f34180095c0 msgr2=0x7f341800ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:33.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.550 7f342affd700 1 --2- 172.21.2.101:0/3222805141 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f34180095c0 0x7f341800ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:33.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.550 7f342affd700 1 -- 172.21.2.101:0/3222805141 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3418003b90 msgr2=0x7f34180060b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:33.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.550 7f342affd700 1 --2- 172.21.2.101:0/3222805141 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3418003b90 0x7f34180060b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:33.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.550 7f342affd700 1 -- 172.21.2.101:0/3222805141 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3418050970 con 0x7f34180067f0 2021-02-10T11:41:33.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.550 7f342affd700 1 --2- 172.21.2.101:0/3222805141 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f34180067f0 0x7f3418008ce0 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 rx=0x7f341c007990 tx=0x7f341c007d30).ready entity=mon.0 client_cookie=844299504fb69455 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:33.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.550 7f342a7fc700 1 -- 172.21.2.101:0/3222805141 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f341c0009b0 con 0x7f34180067f0 2021-02-10T11:41:33.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.550 7f342a7fc700 1 -- 172.21.2.101:0/3222805141 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341c00a310 con 0x7f34180067f0 2021-02-10T11:41:33.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.550 7f342a7fc700 1 -- 172.21.2.101:0/3222805141 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f341c00a760 con 0x7f34180067f0 2021-02-10T11:41:33.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.550 7f3431ad4700 1 -- 172.21.2.101:0/3222805141 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f34180067f0 msgr2=0x7f3418008ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:33.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.550 7f3431ad4700 1 --2- 172.21.2.101:0/3222805141 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f34180067f0 0x7f3418008ce0 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 rx=0x7f341c007990 tx=0x7f341c007d30).stop 2021-02-10T11:41:33.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.550 7f342bfff700 1 -- 172.21.2.101:0/3222805141 reap_dead start 2021-02-10T11:41:33.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.551 7f3431ad4700 1 -- 172.21.2.101:0/3222805141 shutdown_connections 2021-02-10T11:41:33.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.551 7f3431ad4700 1 -- 172.21.2.101:0/3222805141 >> 172.21.2.101:0/3222805141 conn(0x7f342c0ce410 msgr2=0x7f342c0d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:33.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.551 7f3431ad4700 1 -- 172.21.2.101:0/3222805141 shutdown_connections 2021-02-10T11:41:33.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.551 7f3431ad4700 1 -- 172.21.2.101:0/3222805141 wait complete. 2021-02-10T11:41:33.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.566 7f3431ad4700 1 Processor -- start 2021-02-10T11:41:33.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.566 7f3431ad4700 1 -- start start 2021-02-10T11:41:33.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.566 7f3431ad4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f342c0badd0 0x7f342c09eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:33.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.566 7f3431ad4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f342c0adf50 0x7f342c09f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:33.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.566 7f3431ad4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f342c1a7470 0x7f342c1a7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:33.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.566 7f3431ad4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c09e5a0 con 0x7f342c0badd0 2021-02-10T11:41:33.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.566 7f3431ad4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c09f6e0 con 0x7f342c0adf50 2021-02-10T11:41:33.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.566 7f3431ad4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342c0bb220 con 0x7f342c1a7470 2021-02-10T11:41:33.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.566 7f342b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f342c0badd0 0x7f342c09eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:33.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.566 7f342b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f342c0badd0 0x7f342c09eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57372/0 (socket says 172.21.2.101:57372) 2021-02-10T11:41:33.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.566 7f342b7fe700 1 -- 172.21.2.101:0/3062105200 learned_addr learned my addr 172.21.2.101:0/3062105200 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:33.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.567 7f342b7fe700 1 -- 172.21.2.101:0/3062105200 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f342c1a7470 msgr2=0x7f342c1a7870 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:33.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.567 7f342b7fe700 1 --2- 172.21.2.101:0/3062105200 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f342c1a7470 0x7f342c1a7870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:33.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.567 7f342b7fe700 1 -- 172.21.2.101:0/3062105200 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f342c0adf50 msgr2=0x7f342c09f0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:33.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.567 7f342b7fe700 1 --2- 172.21.2.101:0/3062105200 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f342c0adf50 0x7f342c09f0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:33.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.567 7f342b7fe700 1 -- 172.21.2.101:0/3062105200 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c1a7e90 con 0x7f342c0badd0 2021-02-10T11:41:33.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.567 7f342b7fe700 1 --2- 172.21.2.101:0/3062105200 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f342c0badd0 0x7f342c09eaa0 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 rx=0x7f341400b5e0 tx=0x7f341400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:33.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.567 7f3423fff700 1 -- 172.21.2.101:0/3062105200 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3414006780 con 0x7f342c0badd0 2021-02-10T11:41:33.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.567 7f3423fff700 1 -- 172.21.2.101:0/3062105200 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3414002030 con 0x7f342c0badd0 2021-02-10T11:41:33.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.567 7f3423fff700 1 -- 172.21.2.101:0/3062105200 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3414002490 con 0x7f342c0badd0 2021-02-10T11:41:33.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.567 7f3431ad4700 1 -- 172.21.2.101:0/3062105200 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342c1a7e90 con 0x7f342c0badd0 2021-02-10T11:41:33.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.567 7f3431ad4700 1 -- 172.21.2.101:0/3062105200 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342c09d1f0 con 0x7f342c0badd0 2021-02-10T11:41:33.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.568 7f3423fff700 1 -- 172.21.2.101:0/3062105200 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3414002420 con 0x7f342c0badd0 2021-02-10T11:41:33.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.568 7f3423fff700 1 --2- 172.21.2.101:0/3062105200 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f34100186c0 0x7f341001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:33.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.568 7f3423fff700 1 -- 172.21.2.101:0/3062105200 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5107+0+0 (secure 0 0 0) 0x7f3414032360 con 0x7f342c0badd0 2021-02-10T11:41:33.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.568 7f342affd700 1 --2- 172.21.2.101:0/3062105200 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f34100186c0 0x7f341001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:33.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.568 7f342affd700 1 --2- 172.21.2.101:0/3062105200 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f34100186c0 0x7f341001abc0 crc :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:33.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.598 7f3431ad4700 1 -- 172.21.2.101:0/3062105200 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342c082a20 con 0x7f342c0badd0 2021-02-10T11:41:33.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.600 7f3423fff700 1 -- 172.21.2.101:0/3062105200 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3414002210 con 0x7f342c0badd0 2021-02-10T11:41:33.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:33.729 7f3431ad4700 1 -- 172.21.2.101:0/3062105200 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"} v 0) v1 -- 0x7f342c09e520 con 0x7f342c0badd0 2021-02-10T11:41:34.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.082 7f3423fff700 1 -- 172.21.2.101:0/3062105200 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"}]=0 pool 'cephfs_data' created v118) v1 ==== 125+0+0 (secure 0 0 0) 0x7f3414002420 con 0x7f342c0badd0 2021-02-10T11:41:34.096 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' created 2021-02-10T11:41:34.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.095 7f3431ad4700 1 -- 172.21.2.101:0/3062105200 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f34100186c0 msgr2=0x7f341001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:34.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.095 7f3431ad4700 1 --2- 172.21.2.101:0/3062105200 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f34100186c0 0x7f341001abc0 crc :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:34.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.095 7f3431ad4700 1 -- 172.21.2.101:0/3062105200 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f342c0badd0 msgr2=0x7f342c09eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:34.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.095 7f3431ad4700 1 --2- 172.21.2.101:0/3062105200 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f342c0badd0 0x7f342c09eaa0 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 rx=0x7f341400b5e0 tx=0x7f341400b980).stop 2021-02-10T11:41:34.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.095 7f3431ad4700 1 -- 172.21.2.101:0/3062105200 shutdown_connections 2021-02-10T11:41:34.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.095 7f3431ad4700 1 --2- 172.21.2.101:0/3062105200 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f34100186c0 0x7f341001abc0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:34.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.095 7f3431ad4700 1 --2- 172.21.2.101:0/3062105200 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f342c1a7470 0x7f342c1a7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:34.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.095 7f3431ad4700 1 --2- 172.21.2.101:0/3062105200 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f342c0adf50 0x7f342c09f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:34.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.095 7f3431ad4700 1 --2- 172.21.2.101:0/3062105200 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f342c0badd0 0x7f342c09eaa0 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:34.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.095 7f3431ad4700 1 -- 172.21.2.101:0/3062105200 >> 172.21.2.101:0/3062105200 conn(0x7f342c0bee90 msgr2=0x7f342c0bb880 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:34.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.095 7f3431ad4700 1 -- 172.21.2.101:0/3062105200 shutdown_connections 2021-02-10T11:41:34.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.095 7f3431ad4700 1 -- 172.21.2.101:0/3062105200 wait complete. 2021-02-10T11:41:34.106 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-02-10T11:41:34.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.518 7fb727ebb700 1 Processor -- start 2021-02-10T11:41:34.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.518 7fb727ebb700 1 -- start start 2021-02-10T11:41:34.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.518 7fb727ebb700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb7200d1c80 0x7fb7200cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:34.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.518 7fb727ebb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7200cf590 0x7fb7200cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:34.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.518 7fb727ebb700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7200fe1d0 con 0x7fb7200d1880 2021-02-10T11:41:34.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.518 7fb727ebb700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7201b0cc0 con 0x7fb7200d1c80 2021-02-10T11:41:34.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.518 7fb727ebb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb720109a70 con 0x7fb7200cf590 2021-02-10T11:41:34.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.518 7fb7254d6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb7200d1c80 0x7fb7200cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:34.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.519 7fb7254d6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb7200d1c80 0x7fb7200cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57382/0 (socket says 172.21.2.101:57382) 2021-02-10T11:41:34.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.519 7fb7254d6700 1 -- 172.21.2.101:0/1385415591 learned_addr learned my addr 172.21.2.101:0/1385415591 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:34.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.519 7fb7254d6700 1 -- 172.21.2.101:0/1385415591 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7200cf590 msgr2=0x7fb7200cf990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:41:34.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.519 7fb7254d6700 1 --2- 172.21.2.101:0/1385415591 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7200cf590 0x7fb7200cf990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:34.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.519 7fb7254d6700 1 -- 172.21.2.101:0/1385415591 >> v1:172.21.2.108:6789/0 conn(0x7fb7200d1880 legacy=0x7fb7201b04d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:41:34.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.519 7fb7254d6700 1 -- 172.21.2.101:0/1385415591 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb720088070 con 0x7fb7200d1c80 2021-02-10T11:41:34.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.519 7fb7254d6700 1 --2- 172.21.2.101:0/1385415591 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7200d1c80 0x7fb7200cef70 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 rx=0x7fb71c006700 tx=0x7fb71c009b80).ready entity=mon.0 client_cookie=d9148988c2544bc3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:34.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.519 7fb724cd5700 1 -- 172.21.2.101:0/1385415591 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb71c00c0c0 con 0x7fb7200d1c80 2021-02-10T11:41:34.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.519 7fb724cd5700 1 -- 172.21.2.101:0/1385415591 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71c00c340 con 0x7fb7200d1c80 2021-02-10T11:41:34.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.519 7fb724cd5700 1 -- 172.21.2.101:0/1385415591 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb71c00c860 con 0x7fb7200d1c80 2021-02-10T11:41:34.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.519 7fb727ebb700 1 -- 172.21.2.101:0/1385415591 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7200d1c80 msgr2=0x7fb7200cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:34.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.519 7fb727ebb700 1 --2- 172.21.2.101:0/1385415591 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7200d1c80 0x7fb7200cef70 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 rx=0x7fb71c006700 tx=0x7fb71c009b80).stop 2021-02-10T11:41:34.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.520 7fb727ebb700 1 -- 172.21.2.101:0/1385415591 shutdown_connections 2021-02-10T11:41:34.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.520 7fb727ebb700 1 --2- 172.21.2.101:0/1385415591 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7200cf590 0x7fb7200cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:34.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.520 7fb727ebb700 1 --2- 172.21.2.101:0/1385415591 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7200d1c80 0x7fb7200cef70 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:34.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.520 7fb727ebb700 1 -- 172.21.2.101:0/1385415591 >> 172.21.2.101:0/1385415591 conn(0x7fb7200ce430 msgr2=0x7fb7200d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:34.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.520 7fb727ebb700 1 -- 172.21.2.101:0/1385415591 shutdown_connections 2021-02-10T11:41:34.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.520 7fb727ebb700 1 -- 172.21.2.101:0/1385415591 wait complete. 2021-02-10T11:41:34.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.534 7fb727ebb700 1 Processor -- start 2021-02-10T11:41:34.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.535 7fb727ebb700 1 -- start start 2021-02-10T11:41:34.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.535 7fb727ebb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7201add90 0x7fb7201b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:34.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.535 7fb727ebb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb72019b180 0x7fb7200b7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:34.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.535 7fb727ebb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb72019b580 0x7fb7200b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:34.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.535 7fb727ebb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7201ae190 con 0x7fb72019b180 2021-02-10T11:41:34.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.535 7fb727ebb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7200b86e0 con 0x7fb72019b580 2021-02-10T11:41:34.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.535 7fb727ebb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7200b8900 con 0x7fb7201add90 2021-02-10T11:41:34.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.535 7fb7254d6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb72019b180 0x7fb7200b7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:34.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.535 7fb7254d6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb72019b180 0x7fb7200b7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57392/0 (socket says 172.21.2.101:57392) 2021-02-10T11:41:34.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.535 7fb7254d6700 1 -- 172.21.2.101:0/501546461 learned_addr learned my addr 172.21.2.101:0/501546461 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:34.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.535 7fb7254d6700 1 -- 172.21.2.101:0/501546461 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7201add90 msgr2=0x7fb7201b0630 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:34.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.535 7fb7254d6700 1 --2- 172.21.2.101:0/501546461 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7201add90 0x7fb7201b0630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:34.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.535 7fb7254d6700 1 -- 172.21.2.101:0/501546461 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb72019b580 msgr2=0x7fb7200b80c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:34.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.535 7fb7254d6700 1 --2- 172.21.2.101:0/501546461 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb72019b580 0x7fb7200b80c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:34.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.535 7fb7254d6700 1 -- 172.21.2.101:0/501546461 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72019c040 con 0x7fb72019b180 2021-02-10T11:41:34.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.536 7fb7254d6700 1 --2- 172.21.2.101:0/501546461 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb72019b180 0x7fb7200b7aa0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 rx=0x7fb71c00b650 tx=0x7fb71c00b9f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:34.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.536 7fb7167fc700 1 -- 172.21.2.101:0/501546461 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb71c00bde0 con 0x7fb72019b180 2021-02-10T11:41:34.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.536 7fb7167fc700 1 -- 172.21.2.101:0/501546461 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71c000c90 con 0x7fb72019b180 2021-02-10T11:41:34.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.536 7fb7167fc700 1 -- 172.21.2.101:0/501546461 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb71c0022b0 con 0x7fb72019b180 2021-02-10T11:41:34.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.536 7fb727ebb700 1 -- 172.21.2.101:0/501546461 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb72019c040 con 0x7fb72019b180 2021-02-10T11:41:34.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.536 7fb727ebb700 1 -- 172.21.2.101:0/501546461 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb72019c040 con 0x7fb72019b180 2021-02-10T11:41:34.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.537 7fb7167fc700 1 -- 172.21.2.101:0/501546461 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb71c000c90 con 0x7fb72019b180 2021-02-10T11:41:34.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.537 7fb7167fc700 1 --2- 172.21.2.101:0/501546461 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb7080186c0 0x7fb70801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:34.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.537 7fb7167fc700 1 -- 172.21.2.101:0/501546461 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5457+0+0 (secure 0 0 0) 0x7fb71c032540 con 0x7fb72019b180 2021-02-10T11:41:34.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.537 7fb725cd7700 1 --2- 172.21.2.101:0/501546461 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb7080186c0 0x7fb70801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:34.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.537 7fb725cd7700 1 --2- 172.21.2.101:0/501546461 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb7080186c0 0x7fb70801abc0 crc :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:34.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.568 7fb727ebb700 1 -- 172.21.2.101:0/501546461 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7200fe1d0 con 0x7fb72019b180 2021-02-10T11:41:34.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.571 7fb7167fc700 1 -- 172.21.2.101:0/501546461 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb71c030d60 con 0x7fb72019b180 2021-02-10T11:41:34.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.699 7fb727ebb700 1 -- 172.21.2.101:0/501546461 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"} v 0) v1 -- 0x7fb7200fe1d0 con 0x7fb72019b180 2021-02-10T11:41:34.705 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:41:34.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.704 7fb7167fc700 1 -- 172.21.2.101:0/501546461 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"}]=0 new fs with metadata pool 24 and data pool 25 v93) v1 ==== 189+0+0 (secure 0 0 0) 0x7fb71c000c90 con 0x7fb72019b180 2021-02-10T11:41:34.708 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:41:34.710 INFO:teuthology.orchestra.run.gibba001.stderr:new fs with metadata pool 24 and data pool 25 2021-02-10T11:41:34.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.709 7fb727ebb700 1 -- 172.21.2.101:0/501546461 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb7080186c0 msgr2=0x7fb70801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:34.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.709 7fb727ebb700 1 --2- 172.21.2.101:0/501546461 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb7080186c0 0x7fb70801abc0 crc :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:34.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.709 7fb727ebb700 1 -- 172.21.2.101:0/501546461 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb72019b180 msgr2=0x7fb7200b7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:34.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.709 7fb727ebb700 1 --2- 172.21.2.101:0/501546461 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb72019b180 0x7fb7200b7aa0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 rx=0x7fb71c00b650 tx=0x7fb71c00b9f0).stop 2021-02-10T11:41:34.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.709 7fb727ebb700 1 -- 172.21.2.101:0/501546461 shutdown_connections 2021-02-10T11:41:34.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.709 7fb727ebb700 1 --2- 172.21.2.101:0/501546461 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb72019b580 0x7fb7200b80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:34.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.709 7fb727ebb700 1 --2- 172.21.2.101:0/501546461 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb72019b180 0x7fb7200b7aa0 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:34.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.709 7fb727ebb700 1 --2- 172.21.2.101:0/501546461 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb7080186c0 0x7fb70801abc0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:34.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.709 7fb727ebb700 1 --2- 172.21.2.101:0/501546461 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7201add90 0x7fb7201b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:34.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.709 7fb727ebb700 1 -- 172.21.2.101:0/501546461 >> 172.21.2.101:0/501546461 conn(0x7fb72019f940 msgr2=0x7fb7201ab890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:34.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.709 7fb727ebb700 1 -- 172.21.2.101:0/501546461 shutdown_connections 2021-02-10T11:41:34.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:34.709 7fb727ebb700 1 -- 172.21.2.101:0/501546461 wait complete. 2021-02-10T11:41:34.720 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:41:35.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.122 7f91a5ca2700 1 Processor -- start 2021-02-10T11:41:35.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.122 7f91a5ca2700 1 -- start start 2021-02-10T11:41:35.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.122 7f91a5ca2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91a00d1870 0x7f91a01b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:35.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.122 7f91a5ca2700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91a00f0b10 con 0x7f91a00f0650 2021-02-10T11:41:35.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.122 7f91a5ca2700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91a00d0310 con 0x7f91a01b0a10 2021-02-10T11:41:35.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.122 7f91a5ca2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a00e5ed0 con 0x7f91a00d1870 2021-02-10T11:41:35.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.122 7f919ffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f91a00f0650 0x7f91a00cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57890/0 (socket says 172.21.2.101:57890) 2021-02-10T11:41:35.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.122 7f919ffff700 1 -- 172.21.2.101:0/2555202659 learned_addr learned my addr 172.21.2.101:0/2555202659 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:35.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919e7fc700 1 -- 172.21.2.101:0/2555202659 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2948870372 0 0) 0x7f9198001500 con 0x7f91a00f0650 2021-02-10T11:41:35.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919e7fc700 1 -- 172.21.2.101:0/2555202659 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f918c002fe0 con 0x7f91a00f0650 2021-02-10T11:41:35.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919f7fe700 1 --2- 172.21.2.101:0/2555202659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91a00d1870 0x7f91a01b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:35.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919e7fc700 1 -- 172.21.2.101:0/2555202659 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3146998561 0 0) 0x7f9198001460 con 0x7f91a00f0650 2021-02-10T11:41:35.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919e7fc700 1 -- 172.21.2.101:0/2555202659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91a00d1870 msgr2=0x7f91a01b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:35.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919e7fc700 1 --2- 172.21.2.101:0/2555202659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91a00d1870 0x7f91a01b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:35.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919e7fc700 1 -- 172.21.2.101:0/2555202659 >> v1:172.21.2.108:6789/0 conn(0x7f91a01b0a10 legacy=0x7f91a00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:35.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919e7fc700 1 -- 172.21.2.101:0/2555202659 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a00d0530 con 0x7f91a00f0650 2021-02-10T11:41:35.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919e7fc700 1 -- 172.21.2.101:0/2555202659 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f9198001980 con 0x7f91a00f0650 2021-02-10T11:41:35.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919e7fc700 1 -- 172.21.2.101:0/2555202659 >> v1:172.21.2.101:6789/0 conn(0x7f91a00f0650 legacy=0x7f91a00cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:35.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919e7fc700 1 --2- 172.21.2.101:0/2555202659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f918c003ac0 0x7f918c005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:35.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919e7fc700 1 --2- 172.21.2.101:0/2555202659 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f918c006790 0x7f918c008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:35.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919e7fc700 1 --2- 172.21.2.101:0/2555202659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f918c009590 0x7f918c00ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:35.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919e7fc700 1 -- 172.21.2.101:0/2555202659 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918c00c2d0 con 0x7f918c003ac0 2021-02-10T11:41:35.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919e7fc700 1 -- 172.21.2.101:0/2555202659 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918c00c4f0 con 0x7f918c006790 2021-02-10T11:41:35.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919e7fc700 1 -- 172.21.2.101:0/2555202659 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918c00c710 con 0x7f918c009590 2021-02-10T11:41:35.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919f7fe700 1 --2- 172.21.2.101:0/2555202659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f918c003ac0 0x7f918c005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:35.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919ffff700 1 --2- 172.21.2.101:0/2555202659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f918c009590 0x7f918c00ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:35.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919effd700 1 --2- 172.21.2.101:0/2555202659 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f918c006790 0x7f918c008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:35.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919f7fe700 1 -- 172.21.2.101:0/2555202659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f918c009590 msgr2=0x7f918c00ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:35.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919f7fe700 1 --2- 172.21.2.101:0/2555202659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f918c009590 0x7f918c00ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:35.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919f7fe700 1 -- 172.21.2.101:0/2555202659 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f918c006790 msgr2=0x7f918c008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:35.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.123 7f919f7fe700 1 --2- 172.21.2.101:0/2555202659 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f918c006790 0x7f918c008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:35.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.124 7f919f7fe700 1 -- 172.21.2.101:0/2555202659 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918c00c9b0 con 0x7f918c003ac0 2021-02-10T11:41:35.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.124 7f919effd700 1 --2- 172.21.2.101:0/2555202659 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f918c006790 0x7f918c008c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:41:35.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.124 7f919f7fe700 1 --2- 172.21.2.101:0/2555202659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f918c003ac0 0x7f918c005ef0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 rx=0x7f919004aae0 tx=0x7f919004daf0).ready entity=mon.0 client_cookie=e9b22235b50305d0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:35.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.124 7f919e7fc700 1 -- 172.21.2.101:0/2555202659 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f919004a6e0 con 0x7f918c003ac0 2021-02-10T11:41:35.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.124 7f919e7fc700 1 -- 172.21.2.101:0/2555202659 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9190000c90 con 0x7f918c003ac0 2021-02-10T11:41:35.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.124 7f919e7fc700 1 -- 172.21.2.101:0/2555202659 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9190050340 con 0x7f918c003ac0 2021-02-10T11:41:35.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.124 7f91a5ca2700 1 -- 172.21.2.101:0/2555202659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f918c003ac0 msgr2=0x7f918c005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:35.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.124 7f91a5ca2700 1 --2- 172.21.2.101:0/2555202659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f918c003ac0 0x7f918c005ef0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 rx=0x7f919004aae0 tx=0x7f919004daf0).stop 2021-02-10T11:41:35.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.124 7f919ffff700 1 -- 172.21.2.101:0/2555202659 reap_dead start 2021-02-10T11:41:35.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.124 7f91a5ca2700 1 -- 172.21.2.101:0/2555202659 shutdown_connections 2021-02-10T11:41:35.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.124 7f91a5ca2700 1 -- 172.21.2.101:0/2555202659 >> 172.21.2.101:0/2555202659 conn(0x7f91a00ce450 msgr2=0x7f91a00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:35.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.124 7f91a5ca2700 1 -- 172.21.2.101:0/2555202659 shutdown_connections 2021-02-10T11:41:35.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.124 7f91a5ca2700 1 -- 172.21.2.101:0/2555202659 wait complete. 2021-02-10T11:41:35.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.139 7f91a5ca2700 1 Processor -- start 2021-02-10T11:41:35.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.139 7f91a5ca2700 1 -- start start 2021-02-10T11:41:35.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.140 7f91a5ca2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91a01a05d0 0x7f91a00a87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:35.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.140 7f91a5ca2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f91a00a8de0 0x7f91a00a2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:35.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.140 7f91a5ca2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91a00a2c50 0x7f91a00a3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:35.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.140 7f91a5ca2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a00b32d0 con 0x7f91a00a2c50 2021-02-10T11:41:35.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.140 7f91a5ca2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a00a91e0 con 0x7f91a00a8de0 2021-02-10T11:41:35.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.140 7f91a5ca2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a00b37d0 con 0x7f91a01a05d0 2021-02-10T11:41:35.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.140 7f919ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91a00a2c50 0x7f91a00a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:35.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.140 7f919ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91a00a2c50 0x7f91a00a3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57410/0 (socket says 172.21.2.101:57410) 2021-02-10T11:41:35.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.140 7f919ffff700 1 -- 172.21.2.101:0/296488816 learned_addr learned my addr 172.21.2.101:0/296488816 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:35.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.140 7f919ffff700 1 -- 172.21.2.101:0/296488816 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91a01a05d0 msgr2=0x7f91a00a87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:35.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.140 7f919ffff700 1 --2- 172.21.2.101:0/296488816 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91a01a05d0 0x7f91a00a87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:35.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.140 7f919ffff700 1 -- 172.21.2.101:0/296488816 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f91a00a8de0 msgr2=0x7f91a00a2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:35.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.140 7f919ffff700 1 --2- 172.21.2.101:0/296488816 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f91a00a8de0 0x7f91a00a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:35.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.140 7f919ffff700 1 -- 172.21.2.101:0/296488816 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a00b39b0 con 0x7f91a00a2c50 2021-02-10T11:41:35.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.140 7f919ffff700 1 --2- 172.21.2.101:0/296488816 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91a00a2c50 0x7f91a00a3050 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 rx=0x7f9198001c30 tx=0x7f919800ac50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:35.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.141 7f9197fff700 1 -- 172.21.2.101:0/296488816 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9198000f90 con 0x7f91a00a2c50 2021-02-10T11:41:35.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.141 7f9197fff700 1 -- 172.21.2.101:0/296488816 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9198007020 con 0x7f91a00a2c50 2021-02-10T11:41:35.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.141 7f9197fff700 1 -- 172.21.2.101:0/296488816 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9198007540 con 0x7f91a00a2c50 2021-02-10T11:41:35.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.141 7f91a5ca2700 1 -- 172.21.2.101:0/296488816 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a00b39b0 con 0x7f91a00a2c50 2021-02-10T11:41:35.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.141 7f91a5ca2700 1 -- 172.21.2.101:0/296488816 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a00b3f40 con 0x7f91a00a2c50 2021-02-10T11:41:35.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.141 7f9197fff700 1 -- 172.21.2.101:0/296488816 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9198001320 con 0x7f91a00a2c50 2021-02-10T11:41:35.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.141 7f9197fff700 1 --2- 172.21.2.101:0/296488816 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f91840186c0 0x7f918401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:35.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.141 7f9197fff700 1 -- 172.21.2.101:0/296488816 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9198032740 con 0x7f91a00a2c50 2021-02-10T11:41:35.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.142 7f919f7fe700 1 --2- 172.21.2.101:0/296488816 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f91840186c0 0x7f918401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:35.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.142 7f919f7fe700 1 --2- 172.21.2.101:0/296488816 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f91840186c0 0x7f918401abc0 crc :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:35.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.172 7f91a5ca2700 1 -- 172.21.2.101:0/296488816 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a00ebc80 con 0x7f91a00a2c50 2021-02-10T11:41:35.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.175 7f9197fff700 1 -- 172.21.2.101:0/296488816 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9198014670 con 0x7f91a00a2c50 2021-02-10T11:41:35.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.299 7f91a5ca2700 1 -- 172.21.2.101:0/296488816 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91a00f0b10 con 0x7f91a00a2c50 2021-02-10T11:41:35.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.300 7f9197fff700 1 -- 172.21.2.101:0/296488816 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+14020 (secure 0 0 0) 0x7f9198014670 con 0x7f91a00a2c50 2021-02-10T11:41:35.302 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:41:35.302 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":119,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:41:34.701255","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:41:32.676408","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2021-02-10 11:41:33.747692","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:41:35.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.301 7f91a5ca2700 1 -- 172.21.2.101:0/296488816 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f91840186c0 msgr2=0x7f918401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:35.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.301 7f91a5ca2700 1 --2- 172.21.2.101:0/296488816 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f91840186c0 0x7f918401abc0 crc :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:35.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.301 7f91a5ca2700 1 -- 172.21.2.101:0/296488816 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91a00a2c50 msgr2=0x7f91a00a3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:35.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.301 7f91a5ca2700 1 --2- 172.21.2.101:0/296488816 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91a00a2c50 0x7f91a00a3050 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 rx=0x7f9198001c30 tx=0x7f919800ac50).stop 2021-02-10T11:41:35.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.301 7f91a5ca2700 1 -- 172.21.2.101:0/296488816 shutdown_connections 2021-02-10T11:41:35.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.301 7f91a5ca2700 1 --2- 172.21.2.101:0/296488816 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f91a00a2c50 0x7f91a00a3050 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:35.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.301 7f91a5ca2700 1 --2- 172.21.2.101:0/296488816 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f91a00a8de0 0x7f91a00a2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:35.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.301 7f91a5ca2700 1 --2- 172.21.2.101:0/296488816 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f91840186c0 0x7f918401abc0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:35.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.301 7f91a5ca2700 1 --2- 172.21.2.101:0/296488816 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f91a01a05d0 0x7f91a00a87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:35.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.301 7f91a5ca2700 1 -- 172.21.2.101:0/296488816 >> 172.21.2.101:0/296488816 conn(0x7f91a00a9bf0 msgr2=0x7f91a00ebeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:35.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.302 7f91a5ca2700 1 -- 172.21.2.101:0/296488816 shutdown_connections 2021-02-10T11:41:35.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.302 7f91a5ca2700 1 -- 172.21.2.101:0/296488816 wait complete. 2021-02-10T11:41:35.312 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:41:35.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.723 7f2d1f6ab700 1 Processor -- start 2021-02-10T11:41:35.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.723 7f2d1f6ab700 1 -- start start 2021-02-10T11:41:35.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.723 7f2d1f6ab700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2d180d1c70 0x7f2d180f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:35.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.723 7f2d1f6ab700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d180f0b10 con 0x7f2d180d1870 2021-02-10T11:41:35.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.723 7f2d1f6ab700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d181b0be0 con 0x7f2d180d0340 2021-02-10T11:41:35.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.723 7f2d1f6ab700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d180e5ed0 con 0x7f2d180d1c70 2021-02-10T11:41:35.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.723 7f2d1d4c7700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f2d180d1870 0x7f2d181b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57908/0 (socket says 172.21.2.101:57908) 2021-02-10T11:41:35.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.723 7f2d1d4c7700 1 -- 172.21.2.101:0/1987717629 learned_addr learned my addr 172.21.2.101:0/1987717629 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:35.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.723 7f2d1dcc8700 1 --2- 172.21.2.101:0/1987717629 >> v2:172.21.2.108:3300/0 conn(0x7f2d180d1c70 0x7f2d180f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:35.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.724 7f2d0ffff700 1 -- 172.21.2.101:0/1987717629 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1931412051 0 0) 0x7f2d08000ff0 con 0x7f2d180d0340 2021-02-10T11:41:35.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.724 7f2d0ffff700 1 -- 172.21.2.101:0/1987717629 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d04002fe0 con 0x7f2d180d0340 2021-02-10T11:41:35.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.724 7f2d0ffff700 1 -- 172.21.2.101:0/1987717629 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844835688 0 0) 0x7f2d14001100 con 0x7f2d180d1870 2021-02-10T11:41:35.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.724 7f2d0ffff700 1 -- 172.21.2.101:0/1987717629 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d04003210 con 0x7f2d180d1870 2021-02-10T11:41:35.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.724 7f2d0ffff700 1 -- 172.21.2.101:0/1987717629 <== mon.1 v1:172.21.2.108:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3621357942 0 0) 0x7f2d08000de0 con 0x7f2d180d0340 2021-02-10T11:41:35.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.724 7f2d1dcc8700 1 -- 172.21.2.101:0/1987717629 >> v1:172.21.2.108:6789/0 conn(0x7f2d180d0340 legacy=0x7f2d180ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:35.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.724 7f2d1dcc8700 1 -- 172.21.2.101:0/1987717629 >> v1:172.21.2.101:6789/0 conn(0x7f2d180d1870 legacy=0x7f2d181b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:35.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.724 7f2d1dcc8700 1 -- 172.21.2.101:0/1987717629 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d180efe30 con 0x7f2d180d1c70 2021-02-10T11:41:35.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.724 7f2d1dcc8700 1 --2- 172.21.2.101:0/1987717629 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d180d1c70 0x7f2d180f20a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f2d10006e50 tx=0x7f2d10009df0).ready entity=mon.1 client_cookie=1b394c6f30ad844f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:35.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.724 7f2d0ffff700 1 -- 172.21.2.101:0/1987717629 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d1000b510 con 0x7f2d180d1c70 2021-02-10T11:41:35.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.724 7f2d0ffff700 1 -- 172.21.2.101:0/1987717629 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d10003030 con 0x7f2d180d1c70 2021-02-10T11:41:35.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.724 7f2d0ffff700 1 -- 172.21.2.101:0/1987717629 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d1000b7d0 con 0x7f2d180d1c70 2021-02-10T11:41:35.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.724 7f2d1f6ab700 1 -- 172.21.2.101:0/1987717629 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d180d1c70 msgr2=0x7f2d180f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:35.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.724 7f2d1f6ab700 1 --2- 172.21.2.101:0/1987717629 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d180d1c70 0x7f2d180f20a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f2d10006e50 tx=0x7f2d10009df0).stop 2021-02-10T11:41:35.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.725 7f2d1f6ab700 1 -- 172.21.2.101:0/1987717629 shutdown_connections 2021-02-10T11:41:35.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.725 7f2d1f6ab700 1 --2- 172.21.2.101:0/1987717629 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d180d1c70 0x7f2d180f20a0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:35.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.725 7f2d1f6ab700 1 -- 172.21.2.101:0/1987717629 >> 172.21.2.101:0/1987717629 conn(0x7f2d180ce450 msgr2=0x7f2d180d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:35.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.725 7f2d1f6ab700 1 -- 172.21.2.101:0/1987717629 shutdown_connections 2021-02-10T11:41:35.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.725 7f2d1f6ab700 1 -- 172.21.2.101:0/1987717629 wait complete. 2021-02-10T11:41:35.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.740 7f2d1f6ab700 1 Processor -- start 2021-02-10T11:41:35.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.740 7f2d1f6ab700 1 -- start start 2021-02-10T11:41:35.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.740 7f2d1f6ab700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d18195650 0x7f2d180b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:35.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.740 7f2d1f6ab700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d180b5b80 0x7f2d181a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:35.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.740 7f2d1f6ab700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d180acfa0 0x7f2d181a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:35.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.740 7f2d1f6ab700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d180b6010 con 0x7f2d180acfa0 2021-02-10T11:41:35.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.740 7f2d1f6ab700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d181a3a60 con 0x7f2d18195650 2021-02-10T11:41:35.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.740 7f2d1f6ab700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d181a3c40 con 0x7f2d180b5b80 2021-02-10T11:41:35.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.741 7f2d1ccc6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d180b5b80 0x7f2d181a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:35.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.741 7f2d1ccc6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d180b5b80 0x7f2d181a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:50150/0 (socket says 172.21.2.101:50150) 2021-02-10T11:41:35.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.741 7f2d1ccc6700 1 -- 172.21.2.101:0/486354709 learned_addr learned my addr 172.21.2.101:0/486354709 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:35.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.741 7f2d1dcc8700 1 --2- 172.21.2.101:0/486354709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d180acfa0 0x7f2d181a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:35.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.741 7f2d1d4c7700 1 --2- 172.21.2.101:0/486354709 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d18195650 0x7f2d180b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:35.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.741 7f2d1ccc6700 1 -- 172.21.2.101:0/486354709 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d18195650 msgr2=0x7f2d180b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:35.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.741 7f2d1ccc6700 1 --2- 172.21.2.101:0/486354709 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d18195650 0x7f2d180b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:35.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.741 7f2d1ccc6700 1 -- 172.21.2.101:0/486354709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d180acfa0 msgr2=0x7f2d181a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:35.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.741 7f2d1ccc6700 1 --2- 172.21.2.101:0/486354709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d180acfa0 0x7f2d181a3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:35.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.741 7f2d1ccc6700 1 -- 172.21.2.101:0/486354709 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d180bcae0 con 0x7f2d180b5b80 2021-02-10T11:41:35.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.741 7f2d1ccc6700 1 --2- 172.21.2.101:0/486354709 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d180b5b80 0x7f2d181a2e20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f2d08006f10 tx=0x7f2d0800b990).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:35.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.741 7f2d0dffb700 1 -- 172.21.2.101:0/486354709 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d0800bd80 con 0x7f2d180b5b80 2021-02-10T11:41:35.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.741 7f2d0dffb700 1 -- 172.21.2.101:0/486354709 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d08000cf0 con 0x7f2d180b5b80 2021-02-10T11:41:35.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.741 7f2d1f6ab700 1 -- 172.21.2.101:0/486354709 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d180bcae0 con 0x7f2d180b5b80 2021-02-10T11:41:35.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.741 7f2d0dffb700 1 -- 172.21.2.101:0/486354709 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d08002500 con 0x7f2d180b5b80 2021-02-10T11:41:35.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.742 7f2d1f6ab700 1 -- 172.21.2.101:0/486354709 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d180bcae0 con 0x7f2d180b5b80 2021-02-10T11:41:35.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.742 7f2d0dffb700 1 -- 172.21.2.101:0/486354709 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2d08002020 con 0x7f2d180b5b80 2021-02-10T11:41:35.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.742 7f2d0dffb700 1 --2- 172.21.2.101:0/486354709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2cf8018730 0x7f2cf801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:35.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.742 7f2d0dffb700 1 -- 172.21.2.101:0/486354709 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2d08032600 con 0x7f2d180b5b80 2021-02-10T11:41:35.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.742 7f2d1d4c7700 1 --2- 172.21.2.101:0/486354709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2cf8018730 0x7f2cf801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:35.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.743 7f2d1d4c7700 1 --2- 172.21.2.101:0/486354709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2cf8018730 0x7f2cf801ac10 crc :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:35.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.772 7f2d1f6ab700 1 -- 172.21.2.101:0/486354709 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d180f0ab0 con 0x7f2d180b5b80 2021-02-10T11:41:35.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.774 7f2d0dffb700 1 -- 172.21.2.101:0/486354709 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2d08006ba0 con 0x7f2d180b5b80 2021-02-10T11:41:35.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.902 7f2d1f6ab700 1 -- 172.21.2.101:0/486354709 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d180f0ab0 con 0x7f2d180b5b80 2021-02-10T11:41:35.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.903 7f2d0dffb700 1 -- 172.21.2.101:0/486354709 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f2d0800bd80 con 0x7f2d180b5b80 2021-02-10T11:41:35.907 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:41:35.907 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":120,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:41:35.703752","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:41:32.676408","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2021-02-10 11:41:33.747692","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:41:35.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.906 7f2d1f6ab700 1 -- 172.21.2.101:0/486354709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2cf8018730 msgr2=0x7f2cf801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:35.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.906 7f2d1f6ab700 1 --2- 172.21.2.101:0/486354709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2cf8018730 0x7f2cf801ac10 crc :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:35.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.906 7f2d1f6ab700 1 -- 172.21.2.101:0/486354709 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d180b5b80 msgr2=0x7f2d181a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:35.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.906 7f2d1f6ab700 1 --2- 172.21.2.101:0/486354709 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d180b5b80 0x7f2d181a2e20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f2d08006f10 tx=0x7f2d0800b990).stop 2021-02-10T11:41:35.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.907 7f2d1f6ab700 1 -- 172.21.2.101:0/486354709 shutdown_connections 2021-02-10T11:41:35.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.907 7f2d1f6ab700 1 --2- 172.21.2.101:0/486354709 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2cf8018730 0x7f2cf801ac10 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:35.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.907 7f2d1f6ab700 1 --2- 172.21.2.101:0/486354709 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d180b5b80 0x7f2d181a2e20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:35.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.907 7f2d1f6ab700 1 --2- 172.21.2.101:0/486354709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d180acfa0 0x7f2d181a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:35.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.907 7f2d1f6ab700 1 --2- 172.21.2.101:0/486354709 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d18195650 0x7f2d180b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:35.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.907 7f2d1f6ab700 1 -- 172.21.2.101:0/486354709 >> 172.21.2.101:0/486354709 conn(0x7f2d180b3040 msgr2=0x7f2d180c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:35.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.907 7f2d1f6ab700 1 -- 172.21.2.101:0/486354709 shutdown_connections 2021-02-10T11:41:35.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:35.907 7f2d1f6ab700 1 -- 172.21.2.101:0/486354709 wait complete. 2021-02-10T11:41:35.918 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-02-10T11:41:36.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.331 7f1549577700 1 Processor -- start 2021-02-10T11:41:36.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.331 7f1549577700 1 -- start start 2021-02-10T11:41:36.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.331 7f1549577700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f15440d1860 0x7f15441b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:36.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.331 7f1549577700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15440fe1b0 con 0x7f15440cff60 2021-02-10T11:41:36.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.331 7f1549577700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1544109a50 con 0x7f15441b0ad0 2021-02-10T11:41:36.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.331 7f1549577700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1544088070 con 0x7f15440d1860 2021-02-10T11:41:36.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.331 7f1543ac0700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f15440cff60 0x7f1544109120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57920/0 (socket says 172.21.2.101:57920) 2021-02-10T11:41:36.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.331 7f1543ac0700 1 -- 172.21.2.101:0/273315040 learned_addr learned my addr 172.21.2.101:0/273315040 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:36.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.331 7f15432bf700 1 --2- 172.21.2.101:0/273315040 >> v2:172.21.2.101:3300/0 conn(0x7f15440d1860 0x7f15441b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:36.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.332 7f15422bd700 1 -- 172.21.2.101:0/273315040 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1492033562 0 0) 0x7f153c001500 con 0x7f15440cff60 2021-02-10T11:41:36.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.332 7f15422bd700 1 -- 172.21.2.101:0/273315040 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1530002fe0 con 0x7f15440cff60 2021-02-10T11:41:36.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.332 7f15432bf700 1 -- 172.21.2.101:0/273315040 >> v1:172.21.2.108:6789/0 conn(0x7f15441b0ad0 legacy=0x7f1544108930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:36.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.332 7f15432bf700 1 -- 172.21.2.101:0/273315040 >> v1:172.21.2.101:6789/0 conn(0x7f15440cff60 legacy=0x7f1544109120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:36.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.332 7f15432bf700 1 -- 172.21.2.101:0/273315040 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1544088260 con 0x7f15440d1860 2021-02-10T11:41:36.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.332 7f15432bf700 1 --2- 172.21.2.101:0/273315040 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15440d1860 0x7f15441b04b0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 rx=0x7f1534006650 tx=0x7f1534009a80).ready entity=mon.0 client_cookie=b47186634098898b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:36.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.332 7f15422bd700 1 -- 172.21.2.101:0/273315040 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f153400c0c0 con 0x7f15440d1860 2021-02-10T11:41:36.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.332 7f15422bd700 1 -- 172.21.2.101:0/273315040 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153400c340 con 0x7f15440d1860 2021-02-10T11:41:36.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.332 7f15422bd700 1 -- 172.21.2.101:0/273315040 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f153400c860 con 0x7f15440d1860 2021-02-10T11:41:36.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.332 7f1549577700 1 -- 172.21.2.101:0/273315040 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15440d1860 msgr2=0x7f15441b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:36.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.332 7f1549577700 1 --2- 172.21.2.101:0/273315040 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15440d1860 0x7f15441b04b0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 rx=0x7f1534006650 tx=0x7f1534009a80).stop 2021-02-10T11:41:36.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.333 7f1549577700 1 -- 172.21.2.101:0/273315040 shutdown_connections 2021-02-10T11:41:36.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.333 7f1549577700 1 --2- 172.21.2.101:0/273315040 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15440d1860 0x7f15441b04b0 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:36.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.333 7f1549577700 1 -- 172.21.2.101:0/273315040 >> 172.21.2.101:0/273315040 conn(0x7f15440ce410 msgr2=0x7f15440d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:36.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.333 7f1549577700 1 -- 172.21.2.101:0/273315040 shutdown_connections 2021-02-10T11:41:36.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.333 7f1549577700 1 -- 172.21.2.101:0/273315040 wait complete. 2021-02-10T11:41:36.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.347 7f1549577700 1 Processor -- start 2021-02-10T11:41:36.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.348 7f1549577700 1 -- start start 2021-02-10T11:41:36.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.348 7f1549577700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15441a05e0 0x7f154418b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:36.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.348 7f1549577700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15440b8780 0x7f15440bfa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:36.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.348 7f1549577700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15440c0030 0x7f1544189e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:36.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.348 7f1549577700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15440b82a0 con 0x7f15441a05e0 2021-02-10T11:41:36.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.348 7f1549577700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15440c0430 con 0x7f15440b8780 2021-02-10T11:41:36.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.348 7f1549577700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15440c0650 con 0x7f15440c0030 2021-02-10T11:41:36.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.348 7f15432bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15441a05e0 0x7f154418b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:36.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.348 7f15432bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15441a05e0 0x7f154418b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57430/0 (socket says 172.21.2.101:57430) 2021-02-10T11:41:36.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.348 7f15432bf700 1 -- 172.21.2.101:0/1585835183 learned_addr learned my addr 172.21.2.101:0/1585835183 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:36.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.349 7f15432bf700 1 -- 172.21.2.101:0/1585835183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15440c0030 msgr2=0x7f1544189e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:36.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.349 7f15432bf700 1 --2- 172.21.2.101:0/1585835183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15440c0030 0x7f1544189e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:36.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.349 7f15432bf700 1 -- 172.21.2.101:0/1585835183 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15440b8780 msgr2=0x7f15440bfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:36.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.349 7f15432bf700 1 --2- 172.21.2.101:0/1585835183 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15440b8780 0x7f15440bfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:36.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.349 7f15432bf700 1 -- 172.21.2.101:0/1585835183 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154418a490 con 0x7f15441a05e0 2021-02-10T11:41:36.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.349 7f15432bf700 1 --2- 172.21.2.101:0/1585835183 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15441a05e0 0x7f154418b970 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 rx=0x7f1534009630 tx=0x7f153400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:36.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.349 7f153bfff700 1 -- 172.21.2.101:0/1585835183 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f15340093a0 con 0x7f15441a05e0 2021-02-10T11:41:36.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.349 7f153bfff700 1 -- 172.21.2.101:0/1585835183 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153400bd70 con 0x7f15441a05e0 2021-02-10T11:41:36.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.349 7f153bfff700 1 -- 172.21.2.101:0/1585835183 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1534005560 con 0x7f15441a05e0 2021-02-10T11:41:36.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.349 7f1549577700 1 -- 172.21.2.101:0/1585835183 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154418a490 con 0x7f15441a05e0 2021-02-10T11:41:36.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.349 7f1549577700 1 -- 172.21.2.101:0/1585835183 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154418a490 con 0x7f15441a05e0 2021-02-10T11:41:36.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.350 7f153bfff700 1 -- 172.21.2.101:0/1585835183 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f153400a000 con 0x7f15441a05e0 2021-02-10T11:41:36.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.350 7f153bfff700 1 --2- 172.21.2.101:0/1585835183 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15280186c0 0x7f152801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:36.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.350 7f153bfff700 1 -- 172.21.2.101:0/1585835183 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f15340323e0 con 0x7f15441a05e0 2021-02-10T11:41:36.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.350 7f1542abe700 1 --2- 172.21.2.101:0/1585835183 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15280186c0 0x7f152801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:36.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.350 7f1542abe700 1 --2- 172.21.2.101:0/1585835183 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15280186c0 0x7f152801abc0 crc :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:36.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.381 7f1549577700 1 -- 172.21.2.101:0/1585835183 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1544082a20 con 0x7f15441a05e0 2021-02-10T11:41:36.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.384 7f153bfff700 1 -- 172.21.2.101:0/1585835183 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1534030d10 con 0x7f15441a05e0 2021-02-10T11:41:36.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.520 7f1549577700 1 -- 172.21.2.101:0/1585835183 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"} v 0) v1 -- 0x7f1544082a20 con 0x7f15441a05e0 2021-02-10T11:41:36.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.714 7f153bfff700 1 -- 172.21.2.101:0/1585835183 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"}]=0 v96) v1 ==== 118+0+0 (secure 0 0 0) 0x7f153400a000 con 0x7f15441a05e0 2021-02-10T11:41:36.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.738 7f1549577700 1 -- 172.21.2.101:0/1585835183 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15280186c0 msgr2=0x7f152801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:36.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.738 7f1549577700 1 --2- 172.21.2.101:0/1585835183 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15280186c0 0x7f152801abc0 crc :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:36.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.738 7f1549577700 1 -- 172.21.2.101:0/1585835183 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15441a05e0 msgr2=0x7f154418b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:36.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.738 7f1549577700 1 --2- 172.21.2.101:0/1585835183 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15441a05e0 0x7f154418b970 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 rx=0x7f1534009630 tx=0x7f153400b980).stop 2021-02-10T11:41:36.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.738 7f1549577700 1 -- 172.21.2.101:0/1585835183 shutdown_connections 2021-02-10T11:41:36.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.738 7f1549577700 1 --2- 172.21.2.101:0/1585835183 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15280186c0 0x7f152801abc0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:36.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.738 7f1549577700 1 --2- 172.21.2.101:0/1585835183 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15440c0030 0x7f1544189e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:36.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.738 7f1549577700 1 --2- 172.21.2.101:0/1585835183 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15440b8780 0x7f15440bfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:36.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.738 7f1549577700 1 --2- 172.21.2.101:0/1585835183 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15441a05e0 0x7f154418b970 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:36.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.738 7f1549577700 1 -- 172.21.2.101:0/1585835183 >> 172.21.2.101:0/1585835183 conn(0x7f15440a6a80 msgr2=0x7f15440ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:36.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.738 7f1549577700 1 -- 172.21.2.101:0/1585835183 shutdown_connections 2021-02-10T11:41:36.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:36.738 7f1549577700 1 -- 172.21.2.101:0/1585835183 wait complete. 2021-02-10T11:41:36.749 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:41:37.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.160 7f28f0430700 1 Processor -- start 2021-02-10T11:41:37.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.160 7f28f0430700 1 -- start start 2021-02-10T11:41:37.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.160 7f28f0430700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f28e80d0340 0x7f28e80ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:37.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.160 7f28f0430700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28e80f0b10 con 0x7f28e80d1870 2021-02-10T11:41:37.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.160 7f28f0430700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28e81b0be0 con 0x7f28e80d1c70 2021-02-10T11:41:37.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.160 7f28f0430700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28e80e5ed0 con 0x7f28e80d0340 2021-02-10T11:41:37.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.161 7f28eda4b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f28e80d0340 0x7f28e80ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:37.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.161 7f28ee24c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f28e80d1870 0x7f28e81b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57932/0 (socket says 172.21.2.101:57932) 2021-02-10T11:41:37.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.161 7f28ee24c700 1 -- 172.21.2.101:0/1171785321 learned_addr learned my addr 172.21.2.101:0/1171785321 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:37.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.161 7f28eda4b700 1 -- 172.21.2.101:0/1171785321 >> v1:172.21.2.108:6789/0 conn(0x7f28e80d1c70 legacy=0x7f28e80efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:37.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.161 7f28eda4b700 1 -- 172.21.2.101:0/1171785321 >> v1:172.21.2.101:6789/0 conn(0x7f28e80d1870 legacy=0x7f28e81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:37.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.161 7f28eda4b700 1 -- 172.21.2.101:0/1171785321 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e80f0450 con 0x7f28e80d0340 2021-02-10T11:41:37.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.161 7f28eda4b700 1 --2- 172.21.2.101:0/1171785321 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28e80d0340 0x7f28e80ef640 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 rx=0x7f28dc006650 tx=0x7f28dc009a80).ready entity=mon.0 client_cookie=2ea045697dd9c01f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:37.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.161 7f28ed24a700 1 -- 172.21.2.101:0/1171785321 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f28dc00c0c0 con 0x7f28e80d0340 2021-02-10T11:41:37.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.161 7f28ed24a700 1 -- 172.21.2.101:0/1171785321 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc00c340 con 0x7f28e80d0340 2021-02-10T11:41:37.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.161 7f28ed24a700 1 -- 172.21.2.101:0/1171785321 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f28dc00c860 con 0x7f28e80d0340 2021-02-10T11:41:37.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.162 7f28f0430700 1 -- 172.21.2.101:0/1171785321 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28e80d0340 msgr2=0x7f28e80ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:37.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.162 7f28f0430700 1 --2- 172.21.2.101:0/1171785321 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28e80d0340 0x7f28e80ef640 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 rx=0x7f28dc006650 tx=0x7f28dc009a80).stop 2021-02-10T11:41:37.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.162 7f28f0430700 1 -- 172.21.2.101:0/1171785321 shutdown_connections 2021-02-10T11:41:37.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.162 7f28f0430700 1 --2- 172.21.2.101:0/1171785321 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28e80d0340 0x7f28e80ef640 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:37.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.162 7f28f0430700 1 -- 172.21.2.101:0/1171785321 >> 172.21.2.101:0/1171785321 conn(0x7f28e80ce450 msgr2=0x7f28e80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:37.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.162 7f28f0430700 1 -- 172.21.2.101:0/1171785321 shutdown_connections 2021-02-10T11:41:37.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.162 7f28f0430700 1 -- 172.21.2.101:0/1171785321 wait complete. 2021-02-10T11:41:37.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.177 7f28f0430700 1 Processor -- start 2021-02-10T11:41:37.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.177 7f28f0430700 1 -- start start 2021-02-10T11:41:37.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.177 7f28f0430700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f28e80c0060 0x7f28e80c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:37.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.177 7f28f0430700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28e81c69b0 0x7f28e80c3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:37.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.177 7f28f0430700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28e80c3830 0x7f28e80b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:37.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.177 7f28f0430700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e80b33f0 con 0x7f28e81c69b0 2021-02-10T11:41:37.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.177 7f28f0430700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e80c3c30 con 0x7f28e80c0060 2021-02-10T11:41:37.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.177 7f28f0430700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e80bfbc0 con 0x7f28e80c3830 2021-02-10T11:41:37.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.177 7f28eda4b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28e81c69b0 0x7f28e80c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:37.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.177 7f28eda4b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28e81c69b0 0x7f28e80c3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57446/0 (socket says 172.21.2.101:57446) 2021-02-10T11:41:37.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.177 7f28eda4b700 1 -- 172.21.2.101:0/4143594734 learned_addr learned my addr 172.21.2.101:0/4143594734 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:37.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.178 7f28eda4b700 1 -- 172.21.2.101:0/4143594734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28e80c3830 msgr2=0x7f28e80b3970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:37.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.178 7f28eda4b700 1 --2- 172.21.2.101:0/4143594734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28e80c3830 0x7f28e80b3970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:37.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.178 7f28eda4b700 1 -- 172.21.2.101:0/4143594734 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f28e80c0060 msgr2=0x7f28e80c2bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:37.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.178 7f28eda4b700 1 --2- 172.21.2.101:0/4143594734 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f28e80c0060 0x7f28e80c2bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:37.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.178 7f28eda4b700 1 -- 172.21.2.101:0/4143594734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e80b3f90 con 0x7f28e81c69b0 2021-02-10T11:41:37.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.178 7f28eda4b700 1 --2- 172.21.2.101:0/4143594734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28e81c69b0 0x7f28e80c3210 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 rx=0x7f28dc009630 tx=0x7f28dc00b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:37.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.178 7f28daffd700 1 -- 172.21.2.101:0/4143594734 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f28dc0093a0 con 0x7f28e81c69b0 2021-02-10T11:41:37.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.178 7f28daffd700 1 -- 172.21.2.101:0/4143594734 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc00bd70 con 0x7f28e81c69b0 2021-02-10T11:41:37.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.178 7f28f0430700 1 -- 172.21.2.101:0/4143594734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e80b3f90 con 0x7f28e81c69b0 2021-02-10T11:41:37.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.178 7f28f0430700 1 -- 172.21.2.101:0/4143594734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e80b4520 con 0x7f28e81c69b0 2021-02-10T11:41:37.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.178 7f28daffd700 1 -- 172.21.2.101:0/4143594734 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f28dc005560 con 0x7f28e81c69b0 2021-02-10T11:41:37.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.179 7f28daffd700 1 -- 172.21.2.101:0/4143594734 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f28dc00a000 con 0x7f28e81c69b0 2021-02-10T11:41:37.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.179 7f28daffd700 1 --2- 172.21.2.101:0/4143594734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28d0018730 0x7f28d001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:37.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.179 7f28daffd700 1 -- 172.21.2.101:0/4143594734 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f28dc032790 con 0x7f28e81c69b0 2021-02-10T11:41:37.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.179 7f28ee24c700 1 --2- 172.21.2.101:0/4143594734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28d0018730 0x7f28d001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:37.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.179 7f28ee24c700 1 --2- 172.21.2.101:0/4143594734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28d0018730 0x7f28d001ac10 crc :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:37.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.210 7f28f0430700 1 -- 172.21.2.101:0/4143594734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e80f06f0 con 0x7f28e81c69b0 2021-02-10T11:41:37.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.213 7f28daffd700 1 -- 172.21.2.101:0/4143594734 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f28dc009340 con 0x7f28e81c69b0 2021-02-10T11:41:37.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.341 7f28f0430700 1 -- 172.21.2.101:0/4143594734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28e80f06f0 con 0x7f28e81c69b0 2021-02-10T11:41:37.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.341 7f28daffd700 1 -- 172.21.2.101:0/4143594734 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f28dc00bd70 con 0x7f28e81c69b0 2021-02-10T11:41:37.344 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:41:37.344 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":96,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/654364385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":654364385},{"type":"v1","addr":"172.21.2.101:6835","nonce":654364385}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":92}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:41:34.701339","modified":"2021-02-10 11:41:36.711148","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6581":{"gid":6581,"name":"b","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2146451837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2146451837},{"type":"v1","addr":"172.21.2.108:6833","nonce":2146451837}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2021-02-10T11:41:37.344 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 96 2021-02-10T11:41:37.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.343 7f28f0430700 1 -- 172.21.2.101:0/4143594734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28d0018730 msgr2=0x7f28d001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:37.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.343 7f28f0430700 1 --2- 172.21.2.101:0/4143594734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28d0018730 0x7f28d001ac10 crc :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:37.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.343 7f28f0430700 1 -- 172.21.2.101:0/4143594734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28e81c69b0 msgr2=0x7f28e80c3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:37.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.343 7f28f0430700 1 --2- 172.21.2.101:0/4143594734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28e81c69b0 0x7f28e80c3210 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 rx=0x7f28dc009630 tx=0x7f28dc00b980).stop 2021-02-10T11:41:37.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.343 7f28f0430700 1 -- 172.21.2.101:0/4143594734 shutdown_connections 2021-02-10T11:41:37.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.343 7f28f0430700 1 --2- 172.21.2.101:0/4143594734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f28d0018730 0x7f28d001ac10 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:37.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.343 7f28f0430700 1 --2- 172.21.2.101:0/4143594734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28e80c3830 0x7f28e80b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:37.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.343 7f28f0430700 1 --2- 172.21.2.101:0/4143594734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28e81c69b0 0x7f28e80c3210 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:37.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.343 7f28f0430700 1 --2- 172.21.2.101:0/4143594734 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f28e80c0060 0x7f28e80c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:37.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.343 7f28f0430700 1 -- 172.21.2.101:0/4143594734 >> 172.21.2.101:0/4143594734 conn(0x7f28e810a820 msgr2=0x7f28e8108150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:37.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.343 7f28f0430700 1 -- 172.21.2.101:0/4143594734 shutdown_connections 2021-02-10T11:41:37.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.343 7f28f0430700 1 -- 172.21.2.101:0/4143594734 wait complete. 2021-02-10T11:41:37.354 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:41:37.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.764 7f070f1b8700 1 Processor -- start 2021-02-10T11:41:37.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.764 7f070f1b8700 1 -- start start 2021-02-10T11:41:37.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.765 7f070f1b8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f07080d1c70 0x7f07080f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:37.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.765 7f070f1b8700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07080f0b10 con 0x7f07080d1870 2021-02-10T11:41:37.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.765 7f070f1b8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07081b0be0 con 0x7f07080d0340 2021-02-10T11:41:37.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.765 7f070f1b8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07080e5ed0 con 0x7f07080d1c70 2021-02-10T11:41:37.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.765 7f070d7d5700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f07080d1c70 0x7f07080f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:37.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.765 7f070d7d5700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f07080d1c70 0x7f07080f20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57452/0 (socket says 172.21.2.101:57452) 2021-02-10T11:41:37.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.765 7f070d7d5700 1 -- 172.21.2.101:0/1239740018 learned_addr learned my addr 172.21.2.101:0/1239740018 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:37.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.765 7f070d7d5700 1 -- 172.21.2.101:0/1239740018 >> v1:172.21.2.108:6789/0 conn(0x7f07080d0340 legacy=0x7f07080ef640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:41:37.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.765 7f070d7d5700 1 -- 172.21.2.101:0/1239740018 >> v1:172.21.2.101:6789/0 conn(0x7f07080d1870 legacy=0x7f07081b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:37.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.765 7f070d7d5700 1 -- 172.21.2.101:0/1239740018 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07080efe30 con 0x7f07080d1c70 2021-02-10T11:41:37.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.765 7f070d7d5700 1 --2- 172.21.2.101:0/1239740018 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07080d1c70 0x7f07080f20a0 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 rx=0x7f0700006f00 tx=0x7f0700009f10).ready entity=mon.0 client_cookie=42ee7fc664548c7c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:37.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.765 7f07077fe700 1 -- 172.21.2.101:0/1239740018 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f070000b3b0 con 0x7f07080d1c70 2021-02-10T11:41:37.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.765 7f07077fe700 1 -- 172.21.2.101:0/1239740018 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070000b630 con 0x7f07080d1c70 2021-02-10T11:41:37.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.765 7f07077fe700 1 -- 172.21.2.101:0/1239740018 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f070000bb50 con 0x7f07080d1c70 2021-02-10T11:41:37.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.765 7f070f1b8700 1 -- 172.21.2.101:0/1239740018 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07080d1c70 msgr2=0x7f07080f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:37.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.765 7f070f1b8700 1 --2- 172.21.2.101:0/1239740018 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07080d1c70 0x7f07080f20a0 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 rx=0x7f0700006f00 tx=0x7f0700009f10).stop 2021-02-10T11:41:37.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.766 7f070f1b8700 1 -- 172.21.2.101:0/1239740018 shutdown_connections 2021-02-10T11:41:37.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.766 7f070f1b8700 1 --2- 172.21.2.101:0/1239740018 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07080d1c70 0x7f07080f20a0 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:37.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.766 7f070f1b8700 1 -- 172.21.2.101:0/1239740018 >> 172.21.2.101:0/1239740018 conn(0x7f07080ce450 msgr2=0x7f07080d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:37.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.766 7f070f1b8700 1 -- 172.21.2.101:0/1239740018 shutdown_connections 2021-02-10T11:41:37.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.766 7f070f1b8700 1 -- 172.21.2.101:0/1239740018 wait complete. 2021-02-10T11:41:37.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.780 7f070f1b8700 1 Processor -- start 2021-02-10T11:41:37.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.780 7f070f1b8700 1 -- start start 2021-02-10T11:41:37.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.781 7f070f1b8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07080af040 0x7f07080af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:37.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.781 7f070f1b8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07080afa60 0x7f07080ada20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:37.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.781 7f070f1b8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07080afe60 0x7f07080ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:37.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.781 7f070f1b8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07080923c0 con 0x7f07080afe60 2021-02-10T11:41:37.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.781 7f070f1b8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07080b0410 con 0x7f07080af040 2021-02-10T11:41:37.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.781 7f070f1b8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0708091e40 con 0x7f07080afa60 2021-02-10T11:41:37.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.781 7f070d7d5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07080afe60 0x7f07080ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:37.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.781 7f070d7d5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07080afe60 0x7f07080ae040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57462/0 (socket says 172.21.2.101:57462) 2021-02-10T11:41:37.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.781 7f070d7d5700 1 -- 172.21.2.101:0/2588435833 learned_addr learned my addr 172.21.2.101:0/2588435833 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:37.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.781 7f070d7d5700 1 -- 172.21.2.101:0/2588435833 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07080afa60 msgr2=0x7f07080ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:37.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.781 7f070d7d5700 1 --2- 172.21.2.101:0/2588435833 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07080afa60 0x7f07080ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:37.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.781 7f070d7d5700 1 -- 172.21.2.101:0/2588435833 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07080af040 msgr2=0x7f07080af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:37.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.781 7f070d7d5700 1 --2- 172.21.2.101:0/2588435833 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07080af040 0x7f07080af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:37.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.781 7f070d7d5700 1 -- 172.21.2.101:0/2588435833 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070808f6d0 con 0x7f07080afe60 2021-02-10T11:41:37.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.782 7f070d7d5700 1 --2- 172.21.2.101:0/2588435833 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07080afe60 0x7f07080ae040 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 rx=0x7f070000b9b0 tx=0x7f070000bd50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:37.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.782 7f07057fa700 1 -- 172.21.2.101:0/2588435833 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0700003030 con 0x7f07080afe60 2021-02-10T11:41:37.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.782 7f07057fa700 1 -- 172.21.2.101:0/2588435833 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070000c2f0 con 0x7f07080afe60 2021-02-10T11:41:37.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.782 7f07057fa700 1 -- 172.21.2.101:0/2588435833 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f070000c750 con 0x7f07080afe60 2021-02-10T11:41:37.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.782 7f070f1b8700 1 -- 172.21.2.101:0/2588435833 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070808f6d0 con 0x7f07080afe60 2021-02-10T11:41:37.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.782 7f070f1b8700 1 -- 172.21.2.101:0/2588435833 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07080ae950 con 0x7f07080afe60 2021-02-10T11:41:37.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.782 7f07057fa700 1 -- 172.21.2.101:0/2588435833 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0700001270 con 0x7f07080afe60 2021-02-10T11:41:37.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.782 7f07057fa700 1 --2- 172.21.2.101:0/2588435833 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06ec0186c0 0x7f06ec01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:37.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.782 7f07057fa700 1 -- 172.21.2.101:0/2588435833 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0700032920 con 0x7f07080afe60 2021-02-10T11:41:37.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.783 7f070cfd4700 1 --2- 172.21.2.101:0/2588435833 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06ec0186c0 0x7f06ec01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:37.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.783 7f070cfd4700 1 --2- 172.21.2.101:0/2588435833 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06ec0186c0 0x7f06ec01abc0 crc :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:37.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.814 7f070f1b8700 1 -- 172.21.2.101:0/2588435833 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07080aec00 con 0x7f07080afe60 2021-02-10T11:41:37.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.817 7f07057fa700 1 -- 172.21.2.101:0/2588435833 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0700009450 con 0x7f07080afe60 2021-02-10T11:41:37.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.941 7f070f1b8700 1 -- 172.21.2.101:0/2588435833 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07080aec00 con 0x7f07080afe60 2021-02-10T11:41:37.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.942 7f07057fa700 1 -- 172.21.2.101:0/2588435833 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f0700001270 con 0x7f07080afe60 2021-02-10T11:41:37.944 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:41:37.944 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":120,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:41:35.703752","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:41:32.676408","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2021-02-10 11:41:33.747692","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:41:37.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.943 7f070f1b8700 1 -- 172.21.2.101:0/2588435833 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06ec0186c0 msgr2=0x7f06ec01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:37.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.943 7f070f1b8700 1 --2- 172.21.2.101:0/2588435833 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06ec0186c0 0x7f06ec01abc0 crc :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:37.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.943 7f070f1b8700 1 -- 172.21.2.101:0/2588435833 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07080afe60 msgr2=0x7f07080ae040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:37.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.943 7f070f1b8700 1 --2- 172.21.2.101:0/2588435833 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07080afe60 0x7f07080ae040 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 rx=0x7f070000b9b0 tx=0x7f070000bd50).stop 2021-02-10T11:41:37.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.944 7f070f1b8700 1 -- 172.21.2.101:0/2588435833 shutdown_connections 2021-02-10T11:41:37.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.944 7f070f1b8700 1 --2- 172.21.2.101:0/2588435833 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06ec0186c0 0x7f06ec01abc0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:37.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.944 7f070f1b8700 1 --2- 172.21.2.101:0/2588435833 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07080afa60 0x7f07080ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:37.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.944 7f070f1b8700 1 --2- 172.21.2.101:0/2588435833 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07080afe60 0x7f07080ae040 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:37.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.944 7f070f1b8700 1 --2- 172.21.2.101:0/2588435833 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07080af040 0x7f07080af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:37.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.944 7f070f1b8700 1 -- 172.21.2.101:0/2588435833 >> 172.21.2.101:0/2588435833 conn(0x7f07080a0030 msgr2=0x7f07080f22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:37.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.944 7f070f1b8700 1 -- 172.21.2.101:0/2588435833 shutdown_connections 2021-02-10T11:41:37.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:37.944 7f070f1b8700 1 -- 172.21.2.101:0/2588435833 wait complete. 2021-02-10T11:41:37.955 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:41:38.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.367 7f6e44963700 1 Processor -- start 2021-02-10T11:41:38.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.367 7f6e44963700 1 -- start start 2021-02-10T11:41:38.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.367 7f6e44963700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e400d1c80 0x7f6e400cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:38.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.367 7f6e44963700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e400d0180 con 0x7f6e400d1880 2021-02-10T11:41:38.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.367 7f6e44963700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e400fe1d0 con 0x7f6e400cf590 2021-02-10T11:41:38.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.367 7f6e44963700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e401b0cc0 con 0x7f6e400d1c80 2021-02-10T11:41:38.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.368 7f6e3e61c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f6e400d1880 0x7f6e401b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57960/0 (socket says 172.21.2.101:57960) 2021-02-10T11:41:38.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.368 7f6e3e61c700 1 -- 172.21.2.101:0/1715354025 learned_addr learned my addr 172.21.2.101:0/1715354025 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:38.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.368 7f6e3de1b700 1 --2- 172.21.2.101:0/1715354025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e400d1c80 0x7f6e400cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:38.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.368 7f6e3d61a700 1 -- 172.21.2.101:0/1715354025 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 209770048 0 0) 0x7f6e28001180 con 0x7f6e400d1880 2021-02-10T11:41:38.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.368 7f6e3d61a700 1 -- 172.21.2.101:0/1715354025 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e2c002fe0 con 0x7f6e400d1880 2021-02-10T11:41:38.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.368 7f6e3d61a700 1 -- 172.21.2.101:0/1715354025 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 348800045 0 0) 0x7f6e38001400 con 0x7f6e400cf590 2021-02-10T11:41:38.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.368 7f6e3d61a700 1 -- 172.21.2.101:0/1715354025 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e2c003210 con 0x7f6e400cf590 2021-02-10T11:41:38.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.368 7f6e3d61a700 1 -- 172.21.2.101:0/1715354025 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3457996402 0 0) 0x7f6e28001180 con 0x7f6e400d1880 2021-02-10T11:41:38.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.368 7f6e3d61a700 1 -- 172.21.2.101:0/1715354025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e400d1c80 msgr2=0x7f6e400cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:38.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.368 7f6e3d61a700 1 --2- 172.21.2.101:0/1715354025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e400d1c80 0x7f6e400cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:38.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.368 7f6e3d61a700 1 -- 172.21.2.101:0/1715354025 >> v1:172.21.2.108:6789/0 conn(0x7f6e400cf590 legacy=0x7f6e400cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:38.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3d61a700 1 -- 172.21.2.101:0/1715354025 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e40109a70 con 0x7f6e400d1880 2021-02-10T11:41:38.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3d61a700 1 -- 172.21.2.101:0/1715354025 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f6e28001680 con 0x7f6e400d1880 2021-02-10T11:41:38.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3d61a700 1 -- 172.21.2.101:0/1715354025 >> v1:172.21.2.101:6789/0 conn(0x7f6e400d1880 legacy=0x7f6e401b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:38.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3d61a700 1 --2- 172.21.2.101:0/1715354025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e2c003b90 0x7f6e2c005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:38.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3d61a700 1 --2- 172.21.2.101:0/1715354025 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e2c006800 0x7f6e2c008d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:38.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3d61a700 1 --2- 172.21.2.101:0/1715354025 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e2c0095c0 0x7f6e2c00bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:38.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3d61a700 1 -- 172.21.2.101:0/1715354025 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c00c370 con 0x7f6e2c0095c0 2021-02-10T11:41:38.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3d61a700 1 -- 172.21.2.101:0/1715354025 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c050560 con 0x7f6e2c006800 2021-02-10T11:41:38.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3d61a700 1 -- 172.21.2.101:0/1715354025 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c050780 con 0x7f6e2c003b90 2021-02-10T11:41:38.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3de1b700 1 --2- 172.21.2.101:0/1715354025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e400d1c80 0x7f6e400cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:41:38.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3ee1d700 1 --2- 172.21.2.101:0/1715354025 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e2c0095c0 0x7f6e2c00bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:38.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3e61c700 1 --2- 172.21.2.101:0/1715354025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e2c003b90 0x7f6e2c005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:38.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3de1b700 1 --2- 172.21.2.101:0/1715354025 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e2c006800 0x7f6e2c008d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:38.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3ee1d700 1 -- 172.21.2.101:0/1715354025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e2c003b90 msgr2=0x7f6e2c005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:38.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3ee1d700 1 --2- 172.21.2.101:0/1715354025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e2c003b90 0x7f6e2c005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:38.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3ee1d700 1 -- 172.21.2.101:0/1715354025 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e2c006800 msgr2=0x7f6e2c008d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:38.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3ee1d700 1 --2- 172.21.2.101:0/1715354025 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e2c006800 0x7f6e2c008d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:38.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3ee1d700 1 -- 172.21.2.101:0/1715354025 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c050a20 con 0x7f6e2c0095c0 2021-02-10T11:41:38.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3e61c700 1 --2- 172.21.2.101:0/1715354025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e2c003b90 0x7f6e2c005fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:41:38.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3ee1d700 1 --2- 172.21.2.101:0/1715354025 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e2c0095c0 0x7f6e2c00bac0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 rx=0x7f6e380013c0 tx=0x7f6e38009e20).ready entity=mon.0 client_cookie=3e1d76d20852d35d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:38.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3d61a700 1 -- 172.21.2.101:0/1715354025 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e38001400 con 0x7f6e2c0095c0 2021-02-10T11:41:38.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3d61a700 1 -- 172.21.2.101:0/1715354025 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e38001780 con 0x7f6e2c0095c0 2021-02-10T11:41:38.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e3d61a700 1 -- 172.21.2.101:0/1715354025 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e38001c60 con 0x7f6e2c0095c0 2021-02-10T11:41:38.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e44963700 1 -- 172.21.2.101:0/1715354025 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e2c0095c0 msgr2=0x7f6e2c00bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:38.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.369 7f6e44963700 1 --2- 172.21.2.101:0/1715354025 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e2c0095c0 0x7f6e2c00bac0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 rx=0x7f6e380013c0 tx=0x7f6e38009e20).stop 2021-02-10T11:41:38.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.370 7f6e3ee1d700 1 -- 172.21.2.101:0/1715354025 reap_dead start 2021-02-10T11:41:38.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.370 7f6e44963700 1 -- 172.21.2.101:0/1715354025 shutdown_connections 2021-02-10T11:41:38.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.370 7f6e44963700 1 -- 172.21.2.101:0/1715354025 >> 172.21.2.101:0/1715354025 conn(0x7f6e400ce430 msgr2=0x7f6e400d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:38.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.370 7f6e44963700 1 -- 172.21.2.101:0/1715354025 shutdown_connections 2021-02-10T11:41:38.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.370 7f6e44963700 1 -- 172.21.2.101:0/1715354025 wait complete. 2021-02-10T11:41:38.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.385 7f6e44963700 1 Processor -- start 2021-02-10T11:41:38.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.385 7f6e44963700 1 -- start start 2021-02-10T11:41:38.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.385 7f6e44963700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e4008f620 0x7f6e4008fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:38.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.385 7f6e44963700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e40195140 0x7f6e40195540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:38.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.385 7f6e44963700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e40083a40 0x7f6e40083e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:38.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.385 7f6e44963700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e400834c0 con 0x7f6e40195140 2021-02-10T11:41:38.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.385 7f6e44963700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e40083220 con 0x7f6e40083a40 2021-02-10T11:41:38.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.385 7f6e44963700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e40090040 con 0x7f6e4008f620 2021-02-10T11:41:38.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.385 7f6e3de1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e40195140 0x7f6e40195540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:38.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.385 7f6e3de1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e40195140 0x7f6e40195540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57482/0 (socket says 172.21.2.101:57482) 2021-02-10T11:41:38.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.386 7f6e3de1b700 1 -- 172.21.2.101:0/488469717 learned_addr learned my addr 172.21.2.101:0/488469717 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:38.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.386 7f6e3e61c700 1 --2- 172.21.2.101:0/488469717 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e4008f620 0x7f6e4008fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:38.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.386 7f6e3ee1d700 1 --2- 172.21.2.101:0/488469717 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e40083a40 0x7f6e40083e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:38.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.386 7f6e3e61c700 1 -- 172.21.2.101:0/488469717 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e40083a40 msgr2=0x7f6e40083e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:38.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.386 7f6e3e61c700 1 --2- 172.21.2.101:0/488469717 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e40083a40 0x7f6e40083e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:38.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.386 7f6e3e61c700 1 -- 172.21.2.101:0/488469717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e40195140 msgr2=0x7f6e40195540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:38.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.386 7f6e3e61c700 1 --2- 172.21.2.101:0/488469717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e40195140 0x7f6e40195540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:38.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.386 7f6e3e61c700 1 -- 172.21.2.101:0/488469717 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e401a4a00 con 0x7f6e4008f620 2021-02-10T11:41:38.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.386 7f6e3de1b700 1 --2- 172.21.2.101:0/488469717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e40195140 0x7f6e40195540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:41:38.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.386 7f6e3e61c700 1 --2- 172.21.2.101:0/488469717 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e4008f620 0x7f6e4008fa20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f6e28001ec0 tx=0x7f6e2800a870).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:38.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.386 7f6e36ffd700 1 -- 172.21.2.101:0/488469717 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e280070d0 con 0x7f6e4008f620 2021-02-10T11:41:38.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.386 7f6e36ffd700 1 -- 172.21.2.101:0/488469717 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e280074d0 con 0x7f6e4008f620 2021-02-10T11:41:38.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.386 7f6e36ffd700 1 -- 172.21.2.101:0/488469717 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e28000d90 con 0x7f6e4008f620 2021-02-10T11:41:38.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.386 7f6e44963700 1 -- 172.21.2.101:0/488469717 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e401a4a00 con 0x7f6e4008f620 2021-02-10T11:41:38.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.386 7f6e44963700 1 -- 172.21.2.101:0/488469717 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e401a4a00 con 0x7f6e4008f620 2021-02-10T11:41:38.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.387 7f6e36ffd700 1 -- 172.21.2.101:0/488469717 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6e28000d90 con 0x7f6e4008f620 2021-02-10T11:41:38.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.387 7f6e36ffd700 1 --2- 172.21.2.101:0/488469717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e240186c0 0x7f6e2401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:38.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.387 7f6e36ffd700 1 -- 172.21.2.101:0/488469717 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6e280322f0 con 0x7f6e4008f620 2021-02-10T11:41:38.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.387 7f6e3de1b700 1 --2- 172.21.2.101:0/488469717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e240186c0 0x7f6e2401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:38.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.387 7f6e3de1b700 1 --2- 172.21.2.101:0/488469717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e240186c0 0x7f6e2401abc0 crc :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:38.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.417 7f6e44963700 1 -- 172.21.2.101:0/488469717 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e40083220 con 0x7f6e4008f620 2021-02-10T11:41:38.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.421 7f6e36ffd700 1 -- 172.21.2.101:0/488469717 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6e28007820 con 0x7f6e4008f620 2021-02-10T11:41:38.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.554 7f6e44963700 1 -- 172.21.2.101:0/488469717 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f6e40083220 con 0x7f6e4008f620 2021-02-10T11:41:38.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.558 7f6e36ffd700 1 -- 172.21.2.101:0/488469717 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v69) v1 ==== 184+0+0 (secure 0 0 0) 0x7f6e280043a0 con 0x7f6e4008f620 2021-02-10T11:41:38.563 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.0 2021-02-10T11:41:38.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.563 7f6e44963700 1 -- 172.21.2.101:0/488469717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e240186c0 msgr2=0x7f6e2401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:38.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.563 7f6e44963700 1 --2- 172.21.2.101:0/488469717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e240186c0 0x7f6e2401abc0 crc :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:38.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.563 7f6e44963700 1 -- 172.21.2.101:0/488469717 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e4008f620 msgr2=0x7f6e4008fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:38.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.563 7f6e44963700 1 --2- 172.21.2.101:0/488469717 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e4008f620 0x7f6e4008fa20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f6e28001ec0 tx=0x7f6e2800a870).stop 2021-02-10T11:41:38.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.563 7f6e44963700 1 -- 172.21.2.101:0/488469717 shutdown_connections 2021-02-10T11:41:38.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.563 7f6e44963700 1 --2- 172.21.2.101:0/488469717 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e40083a40 0x7f6e40083e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:38.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.563 7f6e44963700 1 --2- 172.21.2.101:0/488469717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e40195140 0x7f6e40195540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:38.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.563 7f6e44963700 1 --2- 172.21.2.101:0/488469717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e240186c0 0x7f6e2401abc0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:38.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.563 7f6e44963700 1 --2- 172.21.2.101:0/488469717 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e4008f620 0x7f6e4008fa20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:38.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.563 7f6e44963700 1 -- 172.21.2.101:0/488469717 >> 172.21.2.101:0/488469717 conn(0x7f6e400a1070 msgr2=0x7f6e4008eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:38.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.563 7f6e44963700 1 -- 172.21.2.101:0/488469717 shutdown_connections 2021-02-10T11:41:38.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.563 7f6e44963700 1 -- 172.21.2.101:0/488469717 wait complete. 2021-02-10T11:41:38.574 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:41:39.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.995 7f832443a700 1 Processor -- start 2021-02-10T11:41:39.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.995 7f832443a700 1 -- start start 2021-02-10T11:41:39.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.995 7f832443a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f831c0d1880 0x7f831c1b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:39.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.996 7f832443a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f831c0cff80 0x7f831c109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:39.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.996 7f832443a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f831c109a70 con 0x7f831c1b0af0 2021-02-10T11:41:39.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.996 7f832443a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f831c109760 con 0x7f831c0cff80 2021-02-10T11:41:39.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.996 7f832443a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831c088070 con 0x7f831c0d1880 2021-02-10T11:41:39.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.996 7f8321a55700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f831c1b0af0 0x7f831c108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57980/0 (socket says 172.21.2.101:57980) 2021-02-10T11:41:39.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.996 7f8321a55700 1 -- 172.21.2.101:0/335915076 learned_addr learned my addr 172.21.2.101:0/335915076 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:39.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.996 7f8322a57700 1 --2- 172.21.2.101:0/335915076 >> v2:172.21.2.101:3300/0 conn(0x7f831c0cff80 0x7f831c109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:39.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.996 7f8322a57700 1 -- 172.21.2.101:0/335915076 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f831c0d1880 msgr2=0x7f831c1b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:39.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.996 7f8322a57700 1 --2- 172.21.2.101:0/335915076 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f831c0d1880 0x7f831c1b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:39.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.996 7f8322a57700 1 -- 172.21.2.101:0/335915076 >> v1:172.21.2.101:6789/0 conn(0x7f831c1b0af0 legacy=0x7f831c108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:39.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.996 7f8322a57700 1 -- 172.21.2.101:0/335915076 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c088260 con 0x7f831c0cff80 2021-02-10T11:41:39.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.996 7f8322a57700 1 --2- 172.21.2.101:0/335915076 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0cff80 0x7f831c109140 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 rx=0x7f8314006e30 tx=0x7f8314009e60).ready entity=mon.0 client_cookie=8e40d10349b392f9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:39.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.997 7f8321254700 1 -- 172.21.2.101:0/335915076 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8314006a50 con 0x7f831c0cff80 2021-02-10T11:41:39.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.997 7f8321254700 1 -- 172.21.2.101:0/335915076 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831400b300 con 0x7f831c0cff80 2021-02-10T11:41:39.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.997 7f8321254700 1 -- 172.21.2.101:0/335915076 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f831400b7e0 con 0x7f831c0cff80 2021-02-10T11:41:39.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.997 7f832443a700 1 -- 172.21.2.101:0/335915076 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0cff80 msgr2=0x7f831c109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:39.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.997 7f832443a700 1 --2- 172.21.2.101:0/335915076 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0cff80 0x7f831c109140 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 rx=0x7f8314006e30 tx=0x7f8314009e60).stop 2021-02-10T11:41:39.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.998 7f832443a700 1 -- 172.21.2.101:0/335915076 shutdown_connections 2021-02-10T11:41:39.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.998 7f832443a700 1 --2- 172.21.2.101:0/335915076 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c0cff80 0x7f831c109140 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:39.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.998 7f832443a700 1 --2- 172.21.2.101:0/335915076 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f831c0d1880 0x7f831c1b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:39.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.998 7f832443a700 1 -- 172.21.2.101:0/335915076 >> 172.21.2.101:0/335915076 conn(0x7f831c0ce430 msgr2=0x7f831c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:39.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.998 7f832443a700 1 -- 172.21.2.101:0/335915076 shutdown_connections 2021-02-10T11:41:39.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:38.998 7f832443a700 1 -- 172.21.2.101:0/335915076 wait complete. 2021-02-10T11:41:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.012 7f832443a700 1 Processor -- start 2021-02-10T11:41:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.012 7f832443a700 1 -- start start 2021-02-10T11:41:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.013 7f832443a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f831c1a68f0 0x7f831c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.013 7f832443a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f831c1a5e00 0x7f831c1a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.013 7f832443a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c1a08e0 0x7f831c1aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.013 7f832443a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831c1a6370 con 0x7f831c1a08e0 2021-02-10T11:41:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.013 7f832443a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831c1a3960 con 0x7f831c1a5e00 2021-02-10T11:41:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.013 7f832443a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831c1ab440 con 0x7f831c1a68f0 2021-02-10T11:41:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.013 7f8322a57700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c1a08e0 0x7f831c1aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.013 7f8322a57700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c1a08e0 0x7f831c1aae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57490/0 (socket says 172.21.2.101:57490) 2021-02-10T11:41:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.013 7f8322a57700 1 -- 172.21.2.101:0/3024258029 learned_addr learned my addr 172.21.2.101:0/3024258029 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.013 7f8322a57700 1 -- 172.21.2.101:0/3024258029 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f831c1a68f0 msgr2=0x7f831c19fca0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.013 7f8322a57700 1 --2- 172.21.2.101:0/3024258029 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f831c1a68f0 0x7f831c19fca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.013 7f8322a57700 1 -- 172.21.2.101:0/3024258029 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f831c1a5e00 msgr2=0x7f831c1a02c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.013 7f8322a57700 1 --2- 172.21.2.101:0/3024258029 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f831c1a5e00 0x7f831c1a02c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.013 7f8322a57700 1 -- 172.21.2.101:0/3024258029 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c1ab660 con 0x7f831c1a08e0 2021-02-10T11:41:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.013 7f8322a57700 1 --2- 172.21.2.101:0/3024258029 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c1a08e0 0x7f831c1aae20 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 rx=0x7f831400b970 tx=0x7f831400bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.014 7f8312ffd700 1 -- 172.21.2.101:0/3024258029 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8314009380 con 0x7f831c1a08e0 2021-02-10T11:41:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.014 7f8312ffd700 1 -- 172.21.2.101:0/3024258029 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8314005820 con 0x7f831c1a08e0 2021-02-10T11:41:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.014 7f832443a700 1 -- 172.21.2.101:0/3024258029 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f831c1ab660 con 0x7f831c1a08e0 2021-02-10T11:41:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.014 7f8312ffd700 1 -- 172.21.2.101:0/3024258029 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8314005d40 con 0x7f831c1a08e0 2021-02-10T11:41:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.014 7f832443a700 1 -- 172.21.2.101:0/3024258029 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f831c1abbf0 con 0x7f831c1a08e0 2021-02-10T11:41:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.015 7f8312ffd700 1 -- 172.21.2.101:0/3024258029 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8314005cd0 con 0x7f831c1a08e0 2021-02-10T11:41:39.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.015 7f8312ffd700 1 --2- 172.21.2.101:0/3024258029 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83040186d0 0x7f830401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:39.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.015 7f8312ffd700 1 -- 172.21.2.101:0/3024258029 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8314032b40 con 0x7f831c1a08e0 2021-02-10T11:41:39.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.015 7f8322256700 1 --2- 172.21.2.101:0/3024258029 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83040186d0 0x7f830401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:39.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.015 7f8322256700 1 --2- 172.21.2.101:0/3024258029 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83040186d0 0x7f830401abd0 crc :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:39.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.045 7f832443a700 1 -- 172.21.2.101:0/3024258029 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f831c0829a0 con 0x7f831c1a08e0 2021-02-10T11:41:39.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.048 7f8312ffd700 1 -- 172.21.2.101:0/3024258029 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8314005ac0 con 0x7f831c1a08e0 2021-02-10T11:41:39.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.177 7f832443a700 1 -- 172.21.2.101:0/3024258029 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f831c0829a0 con 0x7f831c1a08e0 2021-02-10T11:41:39.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.181 7f8312ffd700 1 -- 172.21.2.101:0/3024258029 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v70) v1 ==== 184+0+0 (secure 0 0 0) 0x7f8314006ee0 con 0x7f831c1a08e0 2021-02-10T11:41:39.186 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.1 2021-02-10T11:41:39.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.186 7f832443a700 1 -- 172.21.2.101:0/3024258029 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83040186d0 msgr2=0x7f830401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:39.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.186 7f832443a700 1 --2- 172.21.2.101:0/3024258029 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83040186d0 0x7f830401abd0 crc :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:39.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.186 7f832443a700 1 -- 172.21.2.101:0/3024258029 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c1a08e0 msgr2=0x7f831c1aae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:39.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.186 7f832443a700 1 --2- 172.21.2.101:0/3024258029 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c1a08e0 0x7f831c1aae20 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 rx=0x7f831400b970 tx=0x7f831400bd10).stop 2021-02-10T11:41:39.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.186 7f832443a700 1 -- 172.21.2.101:0/3024258029 shutdown_connections 2021-02-10T11:41:39.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.186 7f832443a700 1 --2- 172.21.2.101:0/3024258029 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f831c1a08e0 0x7f831c1aae20 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:39.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.186 7f832443a700 1 --2- 172.21.2.101:0/3024258029 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f831c1a5e00 0x7f831c1a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:39.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.186 7f832443a700 1 --2- 172.21.2.101:0/3024258029 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83040186d0 0x7f830401abd0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:39.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.186 7f832443a700 1 --2- 172.21.2.101:0/3024258029 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f831c1a68f0 0x7f831c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:39.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.186 7f832443a700 1 -- 172.21.2.101:0/3024258029 >> 172.21.2.101:0/3024258029 conn(0x7f831c19f880 msgr2=0x7f831c1a71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:39.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.186 7f832443a700 1 -- 172.21.2.101:0/3024258029 shutdown_connections 2021-02-10T11:41:39.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.186 7f832443a700 1 -- 172.21.2.101:0/3024258029 wait complete. 2021-02-10T11:41:39.196 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:41:39.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.618 7f7d270fe700 1 Processor -- start 2021-02-10T11:41:39.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.618 7f7d270fe700 1 -- start start 2021-02-10T11:41:39.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.619 7f7d270fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7d200d1880 0x7f7d201b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:39.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.619 7f7d270fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d200cff80 0x7f7d20109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:39.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.619 7f7d270fe700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d20109a70 con 0x7f7d201b0af0 2021-02-10T11:41:39.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.619 7f7d270fe700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d20109760 con 0x7f7d200d1880 2021-02-10T11:41:39.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.619 7f7d270fe700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d20088070 con 0x7f7d200cff80 2021-02-10T11:41:39.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.619 7f7d24f1a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7d200d1880 0x7f7d201b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:39.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.619 7f7d1ffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f7d201b0af0 0x7f7d20108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:57992/0 (socket says 172.21.2.101:57992) 2021-02-10T11:41:39.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.619 7f7d1ffff700 1 -- 172.21.2.101:0/2769873206 learned_addr learned my addr 172.21.2.101:0/2769873206 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:39.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.619 7f7d24f1a700 1 -- 172.21.2.101:0/2769873206 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d200cff80 msgr2=0x7f7d20109140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:41:39.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.619 7f7d24f1a700 1 --2- 172.21.2.101:0/2769873206 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d200cff80 0x7f7d20109140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:39.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.619 7f7d24f1a700 1 -- 172.21.2.101:0/2769873206 >> v1:172.21.2.101:6789/0 conn(0x7f7d201b0af0 legacy=0x7f7d20108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:39.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.619 7f7d24f1a700 1 -- 172.21.2.101:0/2769873206 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d200d1c80 con 0x7f7d200d1880 2021-02-10T11:41:39.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.619 7f7d24f1a700 1 --2- 172.21.2.101:0/2769873206 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d200d1880 0x7f7d201b04d0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 rx=0x7f7d08006650 tx=0x7f7d08009a80).ready entity=mon.0 client_cookie=ad6e27a50023a3c8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:39.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.619 7f7d1f7fe700 1 -- 172.21.2.101:0/2769873206 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7d080009b0 con 0x7f7d200d1880 2021-02-10T11:41:39.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.619 7f7d1f7fe700 1 -- 172.21.2.101:0/2769873206 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0800c0c0 con 0x7f7d200d1880 2021-02-10T11:41:39.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.619 7f7d1f7fe700 1 -- 172.21.2.101:0/2769873206 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7d0800c510 con 0x7f7d200d1880 2021-02-10T11:41:39.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.619 7f7d270fe700 1 -- 172.21.2.101:0/2769873206 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d200d1880 msgr2=0x7f7d201b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:39.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.619 7f7d270fe700 1 --2- 172.21.2.101:0/2769873206 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d200d1880 0x7f7d201b04d0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 rx=0x7f7d08006650 tx=0x7f7d08009a80).stop 2021-02-10T11:41:39.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.620 7f7d270fe700 1 -- 172.21.2.101:0/2769873206 shutdown_connections 2021-02-10T11:41:39.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.620 7f7d270fe700 1 --2- 172.21.2.101:0/2769873206 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d200cff80 0x7f7d20109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:39.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.620 7f7d270fe700 1 --2- 172.21.2.101:0/2769873206 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d200d1880 0x7f7d201b04d0 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:39.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.620 7f7d270fe700 1 -- 172.21.2.101:0/2769873206 >> 172.21.2.101:0/2769873206 conn(0x7f7d200ce430 msgr2=0x7f7d200d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:39.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.620 7f7d270fe700 1 -- 172.21.2.101:0/2769873206 shutdown_connections 2021-02-10T11:41:39.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.620 7f7d270fe700 1 -- 172.21.2.101:0/2769873206 wait complete. 2021-02-10T11:41:39.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.634 7f7d270fe700 1 Processor -- start 2021-02-10T11:41:39.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.634 7f7d270fe700 1 -- start start 2021-02-10T11:41:39.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.635 7f7d270fe700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7d20086520 0x7f7d201a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:39.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.635 7f7d270fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d20086920 0x7f7d20187e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:39.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.635 7f7d270fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d201a22d0 0x7f7d201884a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:39.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.635 7f7d270fe700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d20099600 con 0x7f7d201a22d0 2021-02-10T11:41:39.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.635 7f7d270fe700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d200999f0 con 0x7f7d20086520 2021-02-10T11:41:39.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.635 7f7d270fe700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d20188ac0 con 0x7f7d20086920 2021-02-10T11:41:39.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.635 7f7d2571b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d201a22d0 0x7f7d201884a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:39.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.635 7f7d2571b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d201a22d0 0x7f7d201884a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57504/0 (socket says 172.21.2.101:57504) 2021-02-10T11:41:39.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.635 7f7d2571b700 1 -- 172.21.2.101:0/4181926120 learned_addr learned my addr 172.21.2.101:0/4181926120 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:39.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.635 7f7d2571b700 1 -- 172.21.2.101:0/4181926120 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d20086920 msgr2=0x7f7d20187e80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:39.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.635 7f7d2571b700 1 --2- 172.21.2.101:0/4181926120 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d20086920 0x7f7d20187e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:39.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.635 7f7d2571b700 1 -- 172.21.2.101:0/4181926120 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7d20086520 msgr2=0x7f7d201a1cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:39.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.635 7f7d2571b700 1 --2- 172.21.2.101:0/4181926120 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7d20086520 0x7f7d201a1cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:39.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.635 7f7d2571b700 1 -- 172.21.2.101:0/4181926120 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d20188ce0 con 0x7f7d201a22d0 2021-02-10T11:41:39.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.635 7f7d2571b700 1 --2- 172.21.2.101:0/4181926120 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d201a22d0 0x7f7d201884a0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 rx=0x7f7d18009960 tx=0x7f7d1800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:39.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.636 7f7d1d7fa700 1 -- 172.21.2.101:0/4181926120 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7d18003030 con 0x7f7d201a22d0 2021-02-10T11:41:39.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.636 7f7d1d7fa700 1 -- 172.21.2.101:0/4181926120 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d18001310 con 0x7f7d201a22d0 2021-02-10T11:41:39.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.636 7f7d1d7fa700 1 -- 172.21.2.101:0/4181926120 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7d18001770 con 0x7f7d201a22d0 2021-02-10T11:41:39.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.636 7f7d270fe700 1 -- 172.21.2.101:0/4181926120 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d20188ce0 con 0x7f7d201a22d0 2021-02-10T11:41:39.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.636 7f7d270fe700 1 -- 172.21.2.101:0/4181926120 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d20189270 con 0x7f7d201a22d0 2021-02-10T11:41:39.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.636 7f7d1d7fa700 1 -- 172.21.2.101:0/4181926120 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7d180014f0 con 0x7f7d201a22d0 2021-02-10T11:41:39.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.636 7f7d1d7fa700 1 --2- 172.21.2.101:0/4181926120 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7d040186c0 0x7f7d0401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:39.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.636 7f7d1d7fa700 1 -- 172.21.2.101:0/4181926120 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7d18032940 con 0x7f7d201a22d0 2021-02-10T11:41:39.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.637 7f7d24f1a700 1 --2- 172.21.2.101:0/4181926120 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7d040186c0 0x7f7d0401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:39.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.637 7f7d24f1a700 1 --2- 172.21.2.101:0/4181926120 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7d040186c0 0x7f7d0401abc0 crc :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:39.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.668 7f7d270fe700 1 -- 172.21.2.101:0/4181926120 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d200fe1d0 con 0x7f7d201a22d0 2021-02-10T11:41:39.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.670 7f7d1d7fa700 1 -- 172.21.2.101:0/4181926120 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7d18001ee0 con 0x7f7d201a22d0 2021-02-10T11:41:39.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.801 7f7d270fe700 1 -- 172.21.2.101:0/4181926120 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f7d200999a0 con 0x7f7d201a22d0 2021-02-10T11:41:39.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.804 7f7d1d7fa700 1 -- 172.21.2.101:0/4181926120 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v71) v1 ==== 184+0+0 (secure 0 0 0) 0x7f7d18000f40 con 0x7f7d201a22d0 2021-02-10T11:41:39.806 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.2 2021-02-10T11:41:39.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.806 7f7d270fe700 1 -- 172.21.2.101:0/4181926120 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7d040186c0 msgr2=0x7f7d0401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:39.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.806 7f7d270fe700 1 --2- 172.21.2.101:0/4181926120 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7d040186c0 0x7f7d0401abc0 crc :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:39.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.806 7f7d270fe700 1 -- 172.21.2.101:0/4181926120 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d201a22d0 msgr2=0x7f7d201884a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:39.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.806 7f7d270fe700 1 --2- 172.21.2.101:0/4181926120 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d201a22d0 0x7f7d201884a0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 rx=0x7f7d18009960 tx=0x7f7d1800bcf0).stop 2021-02-10T11:41:39.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.806 7f7d270fe700 1 -- 172.21.2.101:0/4181926120 shutdown_connections 2021-02-10T11:41:39.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.806 7f7d270fe700 1 --2- 172.21.2.101:0/4181926120 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7d040186c0 0x7f7d0401abc0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:39.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.806 7f7d270fe700 1 --2- 172.21.2.101:0/4181926120 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7d20086920 0x7f7d20187e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:39.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.806 7f7d270fe700 1 --2- 172.21.2.101:0/4181926120 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7d201a22d0 0x7f7d201884a0 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:39.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.806 7f7d270fe700 1 --2- 172.21.2.101:0/4181926120 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7d20086520 0x7f7d201a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:39.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.806 7f7d270fe700 1 -- 172.21.2.101:0/4181926120 >> 172.21.2.101:0/4181926120 conn(0x7f7d2019f880 msgr2=0x7f7d2008f890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:39.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.806 7f7d270fe700 1 -- 172.21.2.101:0/4181926120 shutdown_connections 2021-02-10T11:41:39.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:39.806 7f7d270fe700 1 -- 172.21.2.101:0/4181926120 wait complete. 2021-02-10T11:41:39.816 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:41:40.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.236 7f974d621700 1 Processor -- start 2021-02-10T11:41:40.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.236 7f974d621700 1 -- start start 2021-02-10T11:41:40.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.237 7f974d621700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f97480d1c80 0x7f97480cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:40.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.237 7f974d621700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f97480cf590 0x7f97480cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:40.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.237 7f974d621700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97480fe1d0 con 0x7f97480d1880 2021-02-10T11:41:40.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.237 7f974d621700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97481b0cc0 con 0x7f97480d1c80 2021-02-10T11:41:40.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.237 7f974d621700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9748109a70 con 0x7f97480cf590 2021-02-10T11:41:40.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.237 7f9746abe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f97480d1c80 0x7f97480cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:40.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.237 7f97472bf700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f97480d1880 0x7f97481b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58006/0 (socket says 172.21.2.101:58006) 2021-02-10T11:41:40.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.237 7f97472bf700 1 -- 172.21.2.101:0/1506358693 learned_addr learned my addr 172.21.2.101:0/1506358693 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:40.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.237 7f9747ac0700 1 --2- 172.21.2.101:0/1506358693 >> v2:172.21.2.108:3300/0 conn(0x7f97480cf590 0x7f97480cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:40.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.237 7f9747ac0700 1 -- 172.21.2.101:0/1506358693 >> v2:172.21.2.101:3300/0 conn(0x7f97480d1c80 msgr2=0x7f97480cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:40.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.237 7f9747ac0700 1 --2- 172.21.2.101:0/1506358693 >> v2:172.21.2.101:3300/0 conn(0x7f97480d1c80 0x7f97480cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:40.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.237 7f9747ac0700 1 -- 172.21.2.101:0/1506358693 >> v1:172.21.2.101:6789/0 conn(0x7f97480d1880 legacy=0x7f97481b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:40.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.237 7f9747ac0700 1 -- 172.21.2.101:0/1506358693 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9748088070 con 0x7f97480cf590 2021-02-10T11:41:40.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.238 7f9747ac0700 1 --2- 172.21.2.101:0/1506358693 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97480cf590 0x7f97480cf990 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f9740006e50 tx=0x7f9740009e70).ready entity=mon.1 client_cookie=92c8df31b8497c90 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:40.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.238 7f97462bd700 1 -- 172.21.2.101:0/1506358693 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9740006a20 con 0x7f97480cf590 2021-02-10T11:41:40.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.238 7f97462bd700 1 -- 172.21.2.101:0/1506358693 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9740006790 con 0x7f97480cf590 2021-02-10T11:41:40.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.238 7f97462bd700 1 -- 172.21.2.101:0/1506358693 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9740000f90 con 0x7f97480cf590 2021-02-10T11:41:40.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.238 7f974d621700 1 -- 172.21.2.101:0/1506358693 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97480cf590 msgr2=0x7f97480cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:40.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.238 7f974d621700 1 --2- 172.21.2.101:0/1506358693 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97480cf590 0x7f97480cf990 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f9740006e50 tx=0x7f9740009e70).stop 2021-02-10T11:41:40.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.238 7f974d621700 1 -- 172.21.2.101:0/1506358693 shutdown_connections 2021-02-10T11:41:40.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.238 7f974d621700 1 --2- 172.21.2.101:0/1506358693 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97480cf590 0x7f97480cf990 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:40.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.238 7f974d621700 1 --2- 172.21.2.101:0/1506358693 >> v2:172.21.2.101:3300/0 conn(0x7f97480d1c80 0x7f97480cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:40.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.238 7f974d621700 1 -- 172.21.2.101:0/1506358693 >> 172.21.2.101:0/1506358693 conn(0x7f97480ce430 msgr2=0x7f97480d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:40.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.238 7f974d621700 1 -- 172.21.2.101:0/1506358693 shutdown_connections 2021-02-10T11:41:40.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.238 7f974d621700 1 -- 172.21.2.101:0/1506358693 wait complete. 2021-02-10T11:41:40.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.253 7f974d621700 1 Processor -- start 2021-02-10T11:41:40.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.253 7f974d621700 1 -- start start 2021-02-10T11:41:40.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.254 7f974d621700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f974819a070 0x7f974808c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:40.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.254 7f974d621700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f974819a470 0x7f974808cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:40.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.254 7f974d621700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9748095520 0x7f9748095920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:40.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.254 7f974d621700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9748096900 con 0x7f974819a470 2021-02-10T11:41:40.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.254 7f974d621700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974808d4c0 con 0x7f974819a070 2021-02-10T11:41:40.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.254 7f974d621700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974819e9e0 con 0x7f9748095520 2021-02-10T11:41:40.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.254 7f9746abe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f974819a470 0x7f974808cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:40.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.254 7f9746abe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f974819a470 0x7f974808cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57518/0 (socket says 172.21.2.101:57518) 2021-02-10T11:41:40.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.254 7f9746abe700 1 -- 172.21.2.101:0/4267221028 learned_addr learned my addr 172.21.2.101:0/4267221028 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:40.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.254 7f97472bf700 1 --2- 172.21.2.101:0/4267221028 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f974819a070 0x7f974808c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:40.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.254 7f9747ac0700 1 --2- 172.21.2.101:0/4267221028 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9748095520 0x7f9748095920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:40.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.254 7f9746abe700 1 -- 172.21.2.101:0/4267221028 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9748095520 msgr2=0x7f9748095920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:40.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.254 7f9746abe700 1 --2- 172.21.2.101:0/4267221028 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9748095520 0x7f9748095920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:40.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.254 7f9746abe700 1 -- 172.21.2.101:0/4267221028 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f974819a070 msgr2=0x7f974808c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:40.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.255 7f9746abe700 1 --2- 172.21.2.101:0/4267221028 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f974819a070 0x7f974808c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:40.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.255 7f9746abe700 1 -- 172.21.2.101:0/4267221028 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9748095f40 con 0x7f974819a470 2021-02-10T11:41:40.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.255 7f97472bf700 1 --2- 172.21.2.101:0/4267221028 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f974819a070 0x7f974808c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:41:40.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.255 7f9746abe700 1 --2- 172.21.2.101:0/4267221028 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f974819a470 0x7f974808cda0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 rx=0x7f973800b560 tx=0x7f973800b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:40.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.255 7f973ffff700 1 -- 172.21.2.101:0/4267221028 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f973800bcf0 con 0x7f974819a470 2021-02-10T11:41:40.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.255 7f973ffff700 1 -- 172.21.2.101:0/4267221028 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9738000c90 con 0x7f974819a470 2021-02-10T11:41:40.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.255 7f974d621700 1 -- 172.21.2.101:0/4267221028 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9748095f40 con 0x7f974819a470 2021-02-10T11:41:40.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.255 7f973ffff700 1 -- 172.21.2.101:0/4267221028 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9738002220 con 0x7f974819a470 2021-02-10T11:41:40.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.255 7f974d621700 1 -- 172.21.2.101:0/4267221028 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9748095f40 con 0x7f974819a470 2021-02-10T11:41:40.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.256 7f973ffff700 1 -- 172.21.2.101:0/4267221028 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f97380052e0 con 0x7f974819a470 2021-02-10T11:41:40.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.256 7f973ffff700 1 --2- 172.21.2.101:0/4267221028 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f972c0186c0 0x7f972c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:40.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.256 7f973ffff700 1 -- 172.21.2.101:0/4267221028 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f97380323a0 con 0x7f974819a470 2021-02-10T11:41:40.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.256 7f97472bf700 1 --2- 172.21.2.101:0/4267221028 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f972c0186c0 0x7f972c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:40.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.256 7f97472bf700 1 --2- 172.21.2.101:0/4267221028 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f972c0186c0 0x7f972c01abc0 crc :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:40.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.286 7f974d621700 1 -- 172.21.2.101:0/4267221028 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9748082980 con 0x7f974819a470 2021-02-10T11:41:40.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.289 7f973ffff700 1 -- 172.21.2.101:0/4267221028 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f973802b120 con 0x7f974819a470 2021-02-10T11:41:40.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.420 7f974d621700 1 -- 172.21.2.101:0/4267221028 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f9748082980 con 0x7f974819a470 2021-02-10T11:41:40.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.424 7f973ffff700 1 -- 172.21.2.101:0/4267221028 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v72) v1 ==== 184+0+0 (secure 0 0 0) 0x7f973802b120 con 0x7f974819a470 2021-02-10T11:41:40.429 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.3 2021-02-10T11:41:40.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.429 7f974d621700 1 -- 172.21.2.101:0/4267221028 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f972c0186c0 msgr2=0x7f972c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:40.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.429 7f974d621700 1 --2- 172.21.2.101:0/4267221028 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f972c0186c0 0x7f972c01abc0 crc :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:40.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.429 7f974d621700 1 -- 172.21.2.101:0/4267221028 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f974819a470 msgr2=0x7f974808cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:40.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.429 7f974d621700 1 --2- 172.21.2.101:0/4267221028 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f974819a470 0x7f974808cda0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 rx=0x7f973800b560 tx=0x7f973800b900).stop 2021-02-10T11:41:40.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.429 7f974d621700 1 -- 172.21.2.101:0/4267221028 shutdown_connections 2021-02-10T11:41:40.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.429 7f974d621700 1 --2- 172.21.2.101:0/4267221028 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f972c0186c0 0x7f972c01abc0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:40.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.429 7f974d621700 1 --2- 172.21.2.101:0/4267221028 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9748095520 0x7f9748095920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:40.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.429 7f974d621700 1 --2- 172.21.2.101:0/4267221028 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f974819a470 0x7f974808cda0 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:40.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.429 7f974d621700 1 --2- 172.21.2.101:0/4267221028 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f974819a070 0x7f974808c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:40.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.429 7f974d621700 1 -- 172.21.2.101:0/4267221028 >> 172.21.2.101:0/4267221028 conn(0x7f97481a6190 msgr2=0x7f97480949b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:40.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.430 7f974d621700 1 -- 172.21.2.101:0/4267221028 shutdown_connections 2021-02-10T11:41:40.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.430 7f974d621700 1 -- 172.21.2.101:0/4267221028 wait complete. 2021-02-10T11:41:40.439 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:41:40.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.850 7fb79cb95700 1 Processor -- start 2021-02-10T11:41:40.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.850 7fb79cb95700 1 -- start start 2021-02-10T11:41:40.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.850 7fb79cb95700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb7980d1870 0x7fb7981b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:40.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.850 7fb79cb95700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb7981b0a10 0x7fb7980cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:40.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.850 7fb79cb95700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7980d1c70 0x7fb7980cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:40.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.850 7fb79cb95700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7980e5ed0 con 0x7fb7981b0a10 2021-02-10T11:41:40.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.850 7fb79cb95700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7980f0b10 con 0x7fb7980d1870 2021-02-10T11:41:40.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.850 7fb79cb95700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7980cff70 con 0x7fb7980d1c70 2021-02-10T11:41:40.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.850 7fb79605d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb7981b0a10 0x7fb7980cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:40.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.850 7fb79605d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb7981b0a10 0x7fb7980cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57528/0 (socket says 172.21.2.101:57528) 2021-02-10T11:41:40.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.850 7fb79605d700 1 -- 172.21.2.101:0/3963497665 learned_addr learned my addr 172.21.2.101:0/3963497665 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:40.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.851 7fb79605d700 1 -- 172.21.2.101:0/3963497665 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7980d1c70 msgr2=0x7fb7980cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:40.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.851 7fb79605d700 1 --2- 172.21.2.101:0/3963497665 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7980d1c70 0x7fb7980cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:40.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.851 7fb79605d700 1 -- 172.21.2.101:0/3963497665 >> v2:172.21.2.108:3300/0 conn(0x7fb7980d1870 msgr2=0x7fb7981b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:40.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.851 7fb79605d700 1 --2- 172.21.2.101:0/3963497665 >> v2:172.21.2.108:3300/0 conn(0x7fb7980d1870 0x7fb7981b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:40.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.851 7fb79605d700 1 -- 172.21.2.101:0/3963497665 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7980d0150 con 0x7fb7981b0a10 2021-02-10T11:41:40.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.851 7fb79605d700 1 --2- 172.21.2.101:0/3963497665 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7981b0a10 0x7fb7980cf330 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 rx=0x7fb780006700 tx=0x7fb780009b80).ready entity=mon.0 client_cookie=ff9e3a825bb1898f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:40.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.851 7fb79585c700 1 -- 172.21.2.101:0/3963497665 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb78000c0c0 con 0x7fb7981b0a10 2021-02-10T11:41:40.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.851 7fb79585c700 1 -- 172.21.2.101:0/3963497665 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78000c340 con 0x7fb7981b0a10 2021-02-10T11:41:40.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.851 7fb79585c700 1 -- 172.21.2.101:0/3963497665 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb78000c860 con 0x7fb7981b0a10 2021-02-10T11:41:40.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.851 7fb79cb95700 1 -- 172.21.2.101:0/3963497665 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7981b0a10 msgr2=0x7fb7980cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:40.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.851 7fb79cb95700 1 --2- 172.21.2.101:0/3963497665 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7981b0a10 0x7fb7980cf330 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 rx=0x7fb780006700 tx=0x7fb780009b80).stop 2021-02-10T11:41:40.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.852 7fb79cb95700 1 -- 172.21.2.101:0/3963497665 shutdown_connections 2021-02-10T11:41:40.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.852 7fb79cb95700 1 --2- 172.21.2.101:0/3963497665 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7980d1c70 0x7fb7980cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:40.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.852 7fb79cb95700 1 --2- 172.21.2.101:0/3963497665 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7981b0a10 0x7fb7980cf330 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:40.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.852 7fb79cb95700 1 --2- 172.21.2.101:0/3963497665 >> v2:172.21.2.108:3300/0 conn(0x7fb7980d1870 0x7fb7981b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:40.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.852 7fb79cb95700 1 -- 172.21.2.101:0/3963497665 >> 172.21.2.101:0/3963497665 conn(0x7fb7980ce450 msgr2=0x7fb7980d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:40.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.852 7fb79cb95700 1 -- 172.21.2.101:0/3963497665 shutdown_connections 2021-02-10T11:41:40.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.852 7fb79cb95700 1 -- 172.21.2.101:0/3963497665 wait complete. 2021-02-10T11:41:40.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.867 7fb79cb95700 1 Processor -- start 2021-02-10T11:41:40.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.867 7fb79cb95700 1 -- start start 2021-02-10T11:41:40.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.867 7fb79cb95700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7981b8ad0 0x7fb798187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:40.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.867 7fb79cb95700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb7981b8ed0 0x7fb7980b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:40.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.867 7fb79cb95700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7980b2110 0x7fb7981b7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:40.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.867 7fb79cb95700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb798188470 con 0x7fb7981b8ad0 2021-02-10T11:41:40.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.867 7fb79cb95700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79818be10 con 0x7fb7981b8ed0 2021-02-10T11:41:40.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.867 7fb79cb95700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79818c030 con 0x7fb7980b2110 2021-02-10T11:41:40.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.868 7fb79685e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7981b8ad0 0x7fb798187e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:40.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.868 7fb79685e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7981b8ad0 0x7fb798187e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57534/0 (socket says 172.21.2.101:57534) 2021-02-10T11:41:40.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.868 7fb79685e700 1 -- 172.21.2.101:0/1868156866 learned_addr learned my addr 172.21.2.101:0/1868156866 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:40.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.868 7fb79705f700 1 --2- 172.21.2.101:0/1868156866 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7980b2110 0x7fb7981b7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:40.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.868 7fb79605d700 1 --2- 172.21.2.101:0/1868156866 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb7981b8ed0 0x7fb7980b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:40.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.868 7fb79685e700 1 -- 172.21.2.101:0/1868156866 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7980b2110 msgr2=0x7fb7981b7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:40.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.868 7fb79685e700 1 --2- 172.21.2.101:0/1868156866 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7980b2110 0x7fb7981b7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:40.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.868 7fb79685e700 1 -- 172.21.2.101:0/1868156866 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb7981b8ed0 msgr2=0x7fb7980b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:40.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.868 7fb79685e700 1 --2- 172.21.2.101:0/1868156866 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb7981b8ed0 0x7fb7980b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:40.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.868 7fb79685e700 1 -- 172.21.2.101:0/1868156866 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7980b2510 con 0x7fb7981b8ad0 2021-02-10T11:41:40.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.868 7fb79705f700 1 --2- 172.21.2.101:0/1868156866 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7980b2110 0x7fb7981b7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:41:40.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.868 7fb79685e700 1 --2- 172.21.2.101:0/1868156866 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7981b8ad0 0x7fb798187e50 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 rx=0x7fb788006f00 tx=0x7fb78800b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:40.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.869 7fb78f7fe700 1 -- 172.21.2.101:0/1868156866 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb788006b90 con 0x7fb7981b8ad0 2021-02-10T11:41:40.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.869 7fb78f7fe700 1 -- 172.21.2.101:0/1868156866 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78800bd30 con 0x7fb7981b8ad0 2021-02-10T11:41:40.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.869 7fb78f7fe700 1 -- 172.21.2.101:0/1868156866 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb7880020d0 con 0x7fb7981b8ad0 2021-02-10T11:41:40.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.869 7fb79cb95700 1 -- 172.21.2.101:0/1868156866 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7980b2510 con 0x7fb7981b8ad0 2021-02-10T11:41:40.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.869 7fb79cb95700 1 -- 172.21.2.101:0/1868156866 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7981b8020 con 0x7fb7981b8ad0 2021-02-10T11:41:40.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.869 7fb78f7fe700 1 -- 172.21.2.101:0/1868156866 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb788007020 con 0x7fb7981b8ad0 2021-02-10T11:41:40.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.870 7fb78f7fe700 1 --2- 172.21.2.101:0/1868156866 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb77c0186c0 0x7fb77c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:40.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.870 7fb78f7fe700 1 -- 172.21.2.101:0/1868156866 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb788032550 con 0x7fb7981b8ad0 2021-02-10T11:41:40.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.870 7fb79605d700 1 --2- 172.21.2.101:0/1868156866 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb77c0186c0 0x7fb77c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:40.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.870 7fb79605d700 1 --2- 172.21.2.101:0/1868156866 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb77c0186c0 0x7fb77c01abc0 crc :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:40.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.900 7fb79cb95700 1 -- 172.21.2.101:0/1868156866 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7981b88b0 con 0x7fb7981b8ad0 2021-02-10T11:41:40.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:40.903 7fb78f7fe700 1 -- 172.21.2.101:0/1868156866 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb788014de0 con 0x7fb7981b8ad0 2021-02-10T11:41:41.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:41.030 7fb79cb95700 1 -- 172.21.2.101:0/1868156866 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7981b88b0 con 0x7fb7981b8ad0 2021-02-10T11:41:41.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:41.030 7fb78f7fe700 1 -- 172.21.2.101:0/1868156866 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+2101 (secure 0 0 0) 0x7fb788014de0 con 0x7fb7981b8ad0 2021-02-10T11:41:41.032 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:41:41.032 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":96,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/654364385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":654364385},{"type":"v1","addr":"172.21.2.101:6835","nonce":654364385}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":92}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:41:34.701339","modified":"2021-02-10 11:41:36.711148","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6581":{"gid":6581,"name":"b","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2146451837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2146451837},{"type":"v1","addr":"172.21.2.108:6833","nonce":2146451837}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2021-02-10T11:41:41.033 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 96 2021-02-10T11:41:41.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:41.032 7fb79cb95700 1 -- 172.21.2.101:0/1868156866 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb77c0186c0 msgr2=0x7fb77c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:41.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:41.032 7fb79cb95700 1 --2- 172.21.2.101:0/1868156866 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb77c0186c0 0x7fb77c01abc0 crc :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:41.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:41.032 7fb79cb95700 1 -- 172.21.2.101:0/1868156866 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7981b8ad0 msgr2=0x7fb798187e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:41.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:41.032 7fb79cb95700 1 --2- 172.21.2.101:0/1868156866 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7981b8ad0 0x7fb798187e50 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 rx=0x7fb788006f00 tx=0x7fb78800b940).stop 2021-02-10T11:41:41.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:41.032 7fb79cb95700 1 -- 172.21.2.101:0/1868156866 shutdown_connections 2021-02-10T11:41:41.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:41.032 7fb79cb95700 1 --2- 172.21.2.101:0/1868156866 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb77c0186c0 0x7fb77c01abc0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:41.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:41.032 7fb79cb95700 1 --2- 172.21.2.101:0/1868156866 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7980b2110 0x7fb7981b7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:41.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:41.032 7fb79cb95700 1 --2- 172.21.2.101:0/1868156866 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb7981b8ed0 0x7fb7980b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:41.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:41.032 7fb79cb95700 1 --2- 172.21.2.101:0/1868156866 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7981b8ad0 0x7fb798187e50 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:41.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:41.032 7fb79cb95700 1 -- 172.21.2.101:0/1868156866 >> 172.21.2.101:0/1868156866 conn(0x7fb7980ed810 msgr2=0x7fb798094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:41.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:41.032 7fb79cb95700 1 -- 172.21.2.101:0/1868156866 shutdown_connections 2021-02-10T11:41:41.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:41.033 7fb79cb95700 1 -- 172.21.2.101:0/1868156866 wait complete. 2021-02-10T11:41:41.244 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:41:34.701339', 'modified': '2021-02-10 11:41:36.711148', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6581}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6581': {'gid': 6581, 'name': 'b', 'rank': 0, 'incarnation': 93, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/2146451837', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 2146451837}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 2146451837}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:41:41.244 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:41:41.245 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:41:41.642 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:41:41.642 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:41:41.643 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:41:41.643 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 6581, 2021-02-10T11:41:41.643 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:41:41.643 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:41:41.643 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 6.9365744100000004, 2021-02-10T11:41:41.643 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 96, 2021-02-10T11:41:41.643 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 120, 2021-02-10T11:41:41.643 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 120, 2021-02-10T11:41:41.643 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 12.571031548000001 2021-02-10T11:41:41.643 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:41:41.647 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 6581, "want_state": "up:active", "state": "up:active", "rank_uptime": 6.9365744100000004, "mdsmap_epoch": 96, "osdmap_epoch": 120, "osdmap_epoch_barrier": 120, "uptime": 12.571031548000001 } 2021-02-10T11:41:41.647 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:41:42.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.055 7f9fad703700 1 Processor -- start 2021-02-10T11:41:42.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.055 7f9fad703700 1 -- start start 2021-02-10T11:41:42.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.055 7f9fad703700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9fa80d0340 0x7f9fa80ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:42.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.056 7f9fad703700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f9fa80d1c70 0x7f9fa80efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:42.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.056 7f9fad703700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa80f0b10 con 0x7f9fa80d1870 2021-02-10T11:41:42.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.056 7f9fad703700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa80e5ed0 con 0x7f9fa80d0340 2021-02-10T11:41:42.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.056 7f9fad703700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa81b0be0 con 0x7f9fa80d1c70 2021-02-10T11:41:42.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.056 7f9fa72bf700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f9fa80d1870 0x7f9fa81b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58034/0 (socket says 172.21.2.101:58034) 2021-02-10T11:41:42.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.056 7f9fa72bf700 1 -- 172.21.2.101:0/3913175637 learned_addr learned my addr 172.21.2.101:0/3913175637 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:42.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.056 7f9fa6abe700 1 --2- 172.21.2.101:0/3913175637 >> v2:172.21.2.101:3300/0 conn(0x7f9fa80d0340 0x7f9fa80ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:42.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.056 7f9fa6abe700 1 -- 172.21.2.101:0/3913175637 >> v2:172.21.2.108:3300/0 conn(0x7f9fa80d1c70 msgr2=0x7f9fa80efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:42.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.056 7f9fa6abe700 1 --2- 172.21.2.101:0/3913175637 >> v2:172.21.2.108:3300/0 conn(0x7f9fa80d1c70 0x7f9fa80efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:42.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.056 7f9fa6abe700 1 -- 172.21.2.101:0/3913175637 >> v1:172.21.2.101:6789/0 conn(0x7f9fa80d1870 legacy=0x7f9fa81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:42.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.056 7f9fa6abe700 1 -- 172.21.2.101:0/3913175637 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa80f20a0 con 0x7f9fa80d0340 2021-02-10T11:41:42.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.056 7f9fa6abe700 1 --2- 172.21.2.101:0/3913175637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fa80d0340 0x7f9fa80ef640 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 rx=0x7f9f900069d0 tx=0x7f9f90009e40).ready entity=mon.0 client_cookie=b75798e7529372ac server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:42.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.057 7f9fa62bd700 1 -- 172.21.2.101:0/3913175637 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9f9000b510 con 0x7f9fa80d0340 2021-02-10T11:41:42.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.057 7f9fa62bd700 1 -- 172.21.2.101:0/3913175637 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9000b890 con 0x7f9fa80d0340 2021-02-10T11:41:42.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.057 7f9fa62bd700 1 -- 172.21.2.101:0/3913175637 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9f9000bdb0 con 0x7f9fa80d0340 2021-02-10T11:41:42.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.057 7f9fad703700 1 -- 172.21.2.101:0/3913175637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fa80d0340 msgr2=0x7f9fa80ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:42.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.057 7f9fad703700 1 --2- 172.21.2.101:0/3913175637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fa80d0340 0x7f9fa80ef640 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 rx=0x7f9f900069d0 tx=0x7f9f90009e40).stop 2021-02-10T11:41:42.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.057 7f9fad703700 1 -- 172.21.2.101:0/3913175637 shutdown_connections 2021-02-10T11:41:42.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.057 7f9fad703700 1 --2- 172.21.2.101:0/3913175637 >> v2:172.21.2.108:3300/0 conn(0x7f9fa80d1c70 0x7f9fa80efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:42.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.057 7f9fad703700 1 --2- 172.21.2.101:0/3913175637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fa80d0340 0x7f9fa80ef640 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:42.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.057 7f9fad703700 1 -- 172.21.2.101:0/3913175637 >> 172.21.2.101:0/3913175637 conn(0x7f9fa80ce450 msgr2=0x7f9fa80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:42.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.057 7f9fad703700 1 -- 172.21.2.101:0/3913175637 shutdown_connections 2021-02-10T11:41:42.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.058 7f9fad703700 1 -- 172.21.2.101:0/3913175637 wait complete. 2021-02-10T11:41:42.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.072 7f9fad703700 1 Processor -- start 2021-02-10T11:41:42.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.072 7f9fad703700 1 -- start start 2021-02-10T11:41:42.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.072 7f9fad703700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fa818f0a0 0x7f9fa8196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:42.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.072 7f9fad703700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fa81970d0 0x7f9fa81974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:42.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.072 7f9fad703700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9fa8094650 0x7f9fa8094a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:42.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.072 7f9fad703700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa8095bd0 con 0x7f9fa818f0a0 2021-02-10T11:41:42.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.072 7f9fad703700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa80ad7c0 con 0x7f9fa8094650 2021-02-10T11:41:42.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.072 7f9fad703700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa818ec70 con 0x7f9fa81970d0 2021-02-10T11:41:42.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.072 7f9fa72bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fa818f0a0 0x7f9fa8196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:42.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.072 7f9fa72bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fa818f0a0 0x7f9fa8196ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57550/0 (socket says 172.21.2.101:57550) 2021-02-10T11:41:42.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.073 7f9fa72bf700 1 -- 172.21.2.101:0/3628132493 learned_addr learned my addr 172.21.2.101:0/3628132493 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:42.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.073 7f9fa6abe700 1 --2- 172.21.2.101:0/3628132493 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fa81970d0 0x7f9fa81974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:42.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.073 7f9fa72bf700 1 -- 172.21.2.101:0/3628132493 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fa81970d0 msgr2=0x7f9fa81974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:42.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.073 7f9fa72bf700 1 --2- 172.21.2.101:0/3628132493 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fa81970d0 0x7f9fa81974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:42.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.073 7f9fa72bf700 1 -- 172.21.2.101:0/3628132493 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9fa8094650 msgr2=0x7f9fa8094a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:42.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.073 7f9fa72bf700 1 --2- 172.21.2.101:0/3628132493 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9fa8094650 0x7f9fa8094a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:42.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.073 7f9fa72bf700 1 -- 172.21.2.101:0/3628132493 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa8095070 con 0x7f9fa818f0a0 2021-02-10T11:41:42.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.073 7f9fa72bf700 1 --2- 172.21.2.101:0/3628132493 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fa818f0a0 0x7f9fa8196ab0 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 rx=0x7f9f9800a570 tx=0x7f9f9800a910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:42.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.073 7f9f9ffff700 1 -- 172.21.2.101:0/3628132493 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9f980070d0 con 0x7f9fa818f0a0 2021-02-10T11:41:42.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.073 7f9f9ffff700 1 -- 172.21.2.101:0/3628132493 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9800ad00 con 0x7f9fa818f0a0 2021-02-10T11:41:42.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.073 7f9f9ffff700 1 -- 172.21.2.101:0/3628132493 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9f98004620 con 0x7f9fa818f0a0 2021-02-10T11:41:42.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.073 7f9fad703700 1 -- 172.21.2.101:0/3628132493 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa8095070 con 0x7f9fa818f0a0 2021-02-10T11:41:42.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.073 7f9fad703700 1 -- 172.21.2.101:0/3628132493 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa8095070 con 0x7f9fa818f0a0 2021-02-10T11:41:42.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.074 7f9f9ffff700 1 -- 172.21.2.101:0/3628132493 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9f98000c90 con 0x7f9fa818f0a0 2021-02-10T11:41:42.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.074 7f9f9ffff700 1 --2- 172.21.2.101:0/3628132493 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f8c0186c0 0x7f9f8c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:42.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.074 7f9f9ffff700 1 -- 172.21.2.101:0/3628132493 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9f980327a0 con 0x7f9fa818f0a0 2021-02-10T11:41:42.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.074 7f9fa6abe700 1 --2- 172.21.2.101:0/3628132493 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f8c0186c0 0x7f9f8c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:42.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.074 7f9fa6abe700 1 --2- 172.21.2.101:0/3628132493 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f8c0186c0 0x7f9f8c01abc0 crc :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:42.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.106 7f9fad703700 1 -- 172.21.2.101:0/3628132493 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa80ad790 con 0x7f9fa818f0a0 2021-02-10T11:41:42.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.109 7f9f9ffff700 1 -- 172.21.2.101:0/3628132493 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9f98004ef0 con 0x7f9fa818f0a0 2021-02-10T11:41:42.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.232 7f9fad703700 1 -- 172.21.2.101:0/3628132493 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fa80ad790 con 0x7f9fa818f0a0 2021-02-10T11:41:42.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.233 7f9f9ffff700 1 -- 172.21.2.101:0/3628132493 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f9f98004ef0 con 0x7f9fa818f0a0 2021-02-10T11:41:42.235 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:41:42.235 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":120,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:41:35.703752","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:41:32.676408","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2021-02-10 11:41:33.747692","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:41:42.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.234 7f9fad703700 1 -- 172.21.2.101:0/3628132493 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f8c0186c0 msgr2=0x7f9f8c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:42.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.234 7f9fad703700 1 --2- 172.21.2.101:0/3628132493 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f8c0186c0 0x7f9f8c01abc0 crc :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:42.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.235 7f9fad703700 1 -- 172.21.2.101:0/3628132493 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fa818f0a0 msgr2=0x7f9fa8196ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:42.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.235 7f9fad703700 1 --2- 172.21.2.101:0/3628132493 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fa818f0a0 0x7f9fa8196ab0 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 rx=0x7f9f9800a570 tx=0x7f9f9800a910).stop 2021-02-10T11:41:42.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.235 7f9fad703700 1 -- 172.21.2.101:0/3628132493 shutdown_connections 2021-02-10T11:41:42.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.235 7f9fad703700 1 --2- 172.21.2.101:0/3628132493 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f8c0186c0 0x7f9f8c01abc0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:42.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.235 7f9fad703700 1 --2- 172.21.2.101:0/3628132493 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fa81970d0 0x7f9fa81974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:42.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.235 7f9fad703700 1 --2- 172.21.2.101:0/3628132493 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9fa8094650 0x7f9fa8094a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:42.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.235 7f9fad703700 1 --2- 172.21.2.101:0/3628132493 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fa818f0a0 0x7f9fa8196ab0 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:42.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.235 7f9fad703700 1 -- 172.21.2.101:0/3628132493 >> 172.21.2.101:0/3628132493 conn(0x7f9fa81a38c0 msgr2=0x7f9fa81a2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:42.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.235 7f9fad703700 1 -- 172.21.2.101:0/3628132493 shutdown_connections 2021-02-10T11:41:42.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.235 7f9fad703700 1 -- 172.21.2.101:0/3628132493 wait complete. 2021-02-10T11:41:42.245 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:41:32.676408', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:41:33.747692', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:41:42.245 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:41:42.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.658 7f287565d700 1 Processor -- start 2021-02-10T11:41:42.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.658 7f287565d700 1 -- start start 2021-02-10T11:41:42.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.659 7f287565d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f28700d1c70 0x7f28700cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:42.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.659 7f287565d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28700cf580 0x7f28700cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:42.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.659 7f287565d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28700fe1c0 con 0x7f28700d1870 2021-02-10T11:41:42.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.659 7f287565d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28701b0cb0 con 0x7f28700d1c70 2021-02-10T11:41:42.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.659 7f287565d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2870109a60 con 0x7f28700cf580 2021-02-10T11:41:42.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.659 7f286f2bf700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f28700d1870 0x7f28701b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58048/0 (socket says 172.21.2.101:58048) 2021-02-10T11:41:42.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.659 7f286f2bf700 1 -- 172.21.2.101:0/1776224767 learned_addr learned my addr 172.21.2.101:0/1776224767 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:42.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.659 7f286eabe700 1 --2- 172.21.2.101:0/1776224767 >> v2:172.21.2.101:3300/0 conn(0x7f28700d1c70 0x7f28700cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:42.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.659 7f286fac0700 1 --2- 172.21.2.101:0/1776224767 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28700cf580 0x7f28700cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:42.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.659 7f286eabe700 1 -- 172.21.2.101:0/1776224767 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28700cf580 msgr2=0x7f28700cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:42.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.659 7f286eabe700 1 --2- 172.21.2.101:0/1776224767 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28700cf580 0x7f28700cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:42.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.659 7f286eabe700 1 -- 172.21.2.101:0/1776224767 >> v1:172.21.2.101:6789/0 conn(0x7f28700d1870 legacy=0x7f28701b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:42.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.659 7f286eabe700 1 -- 172.21.2.101:0/1776224767 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2870088070 con 0x7f28700d1c70 2021-02-10T11:41:42.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.659 7f286fac0700 1 --2- 172.21.2.101:0/1776224767 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28700cf580 0x7f28700cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:41:42.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.659 7f286eabe700 1 --2- 172.21.2.101:0/1776224767 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28700d1c70 0x7f28700cef60 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 rx=0x7f2858006650 tx=0x7f2858009a80).ready entity=mon.0 client_cookie=817cc60ec76cfc0e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:42.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.660 7f286e2bd700 1 -- 172.21.2.101:0/1776224767 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f28580009b0 con 0x7f28700d1c70 2021-02-10T11:41:42.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.660 7f286e2bd700 1 -- 172.21.2.101:0/1776224767 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285800c0c0 con 0x7f28700d1c70 2021-02-10T11:41:42.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.660 7f286e2bd700 1 -- 172.21.2.101:0/1776224767 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f285800c5e0 con 0x7f28700d1c70 2021-02-10T11:41:42.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.660 7f287565d700 1 -- 172.21.2.101:0/1776224767 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28700d1c70 msgr2=0x7f28700cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:42.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.660 7f287565d700 1 --2- 172.21.2.101:0/1776224767 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28700d1c70 0x7f28700cef60 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 rx=0x7f2858006650 tx=0x7f2858009a80).stop 2021-02-10T11:41:42.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.660 7f287565d700 1 -- 172.21.2.101:0/1776224767 shutdown_connections 2021-02-10T11:41:42.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.660 7f287565d700 1 --2- 172.21.2.101:0/1776224767 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28700cf580 0x7f28700cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:42.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.660 7f287565d700 1 --2- 172.21.2.101:0/1776224767 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f28700d1c70 0x7f28700cef60 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:42.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.660 7f287565d700 1 -- 172.21.2.101:0/1776224767 >> 172.21.2.101:0/1776224767 conn(0x7f28700ce420 msgr2=0x7f28700d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:42.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.660 7f287565d700 1 -- 172.21.2.101:0/1776224767 shutdown_connections 2021-02-10T11:41:42.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.660 7f287565d700 1 -- 172.21.2.101:0/1776224767 wait complete. 2021-02-10T11:41:42.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.675 7f287565d700 1 Processor -- start 2021-02-10T11:41:42.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.675 7f287565d700 1 -- start start 2021-02-10T11:41:42.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.675 7f287565d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f28700b04d0 0x7f28701a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:42.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.675 7f287565d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28701a8090 0x7f28700c23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:42.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.675 7f287565d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f287008d680 0x7f28700c29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:42.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.675 7f287565d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28700b0a00 con 0x7f287008d680 2021-02-10T11:41:42.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.675 7f287565d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28700c2fe0 con 0x7f28700b04d0 2021-02-10T11:41:42.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.675 7f287565d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28700c3200 con 0x7f28701a8090 2021-02-10T11:41:42.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.676 7f286eabe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28701a8090 0x7f28700c23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:42.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.676 7f286fac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f287008d680 0x7f28700c29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:42.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.676 7f286fac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f287008d680 0x7f28700c29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57564/0 (socket says 172.21.2.101:57564) 2021-02-10T11:41:42.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.676 7f286fac0700 1 -- 172.21.2.101:0/55304138 learned_addr learned my addr 172.21.2.101:0/55304138 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:42.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.676 7f286f2bf700 1 --2- 172.21.2.101:0/55304138 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f28700b04d0 0x7f28701a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:42.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.676 7f286fac0700 1 -- 172.21.2.101:0/55304138 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28701a8090 msgr2=0x7f28700c23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:42.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.676 7f286fac0700 1 --2- 172.21.2.101:0/55304138 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28701a8090 0x7f28700c23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:42.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.676 7f286fac0700 1 -- 172.21.2.101:0/55304138 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f28700b04d0 msgr2=0x7f28701a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:42.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.676 7f286fac0700 1 --2- 172.21.2.101:0/55304138 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f28700b04d0 0x7f28701a7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:42.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.676 7f286fac0700 1 -- 172.21.2.101:0/55304138 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287008e080 con 0x7f287008d680 2021-02-10T11:41:42.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.676 7f286fac0700 1 --2- 172.21.2.101:0/55304138 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f287008d680 0x7f28700c29c0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 rx=0x7f2868009980 tx=0x7f286800bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:42.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.676 7f2867fff700 1 -- 172.21.2.101:0/55304138 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f286800c2f0 con 0x7f287008d680 2021-02-10T11:41:42.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.676 7f2867fff700 1 -- 172.21.2.101:0/55304138 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2868006790 con 0x7f287008d680 2021-02-10T11:41:42.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.676 7f287565d700 1 -- 172.21.2.101:0/55304138 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287008e080 con 0x7f287008d680 2021-02-10T11:41:42.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.676 7f2867fff700 1 -- 172.21.2.101:0/55304138 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f286800c7e0 con 0x7f287008d680 2021-02-10T11:41:42.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.676 7f287565d700 1 -- 172.21.2.101:0/55304138 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287008e080 con 0x7f287008d680 2021-02-10T11:41:42.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.677 7f2867fff700 1 -- 172.21.2.101:0/55304138 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f28680011f0 con 0x7f287008d680 2021-02-10T11:41:42.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.677 7f2867fff700 1 --2- 172.21.2.101:0/55304138 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2854018730 0x7f285401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:42.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.677 7f2867fff700 1 -- 172.21.2.101:0/55304138 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2868032a40 con 0x7f287008d680 2021-02-10T11:41:42.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.677 7f286f2bf700 1 --2- 172.21.2.101:0/55304138 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2854018730 0x7f285401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:42.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.677 7f286f2bf700 1 --2- 172.21.2.101:0/55304138 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2854018730 0x7f285401ac10 crc :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:42.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.709 7f287565d700 1 -- 172.21.2.101:0/55304138 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2870082a70 con 0x7f287008d680 2021-02-10T11:41:42.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.711 7f2867fff700 1 -- 172.21.2.101:0/55304138 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f286800be10 con 0x7f287008d680 2021-02-10T11:41:42.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.838 7f287565d700 1 -- 172.21.2.101:0/55304138 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f2870082a70 con 0x7f287008d680 2021-02-10T11:41:42.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.838 7f2867fff700 1 -- 172.21.2.101:0/55304138 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v120) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2868006670 con 0x7f287008d680 2021-02-10T11:41:42.840 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:41:42.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.841 7f287565d700 1 -- 172.21.2.101:0/55304138 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2854018730 msgr2=0x7f285401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:42.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.841 7f287565d700 1 --2- 172.21.2.101:0/55304138 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2854018730 0x7f285401ac10 crc :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:42.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.841 7f287565d700 1 -- 172.21.2.101:0/55304138 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f287008d680 msgr2=0x7f28700c29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:42.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.841 7f287565d700 1 --2- 172.21.2.101:0/55304138 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f287008d680 0x7f28700c29c0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 rx=0x7f2868009980 tx=0x7f286800bcd0).stop 2021-02-10T11:41:42.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.841 7f287565d700 1 -- 172.21.2.101:0/55304138 shutdown_connections 2021-02-10T11:41:42.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.841 7f287565d700 1 --2- 172.21.2.101:0/55304138 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2854018730 0x7f285401ac10 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:42.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.841 7f287565d700 1 --2- 172.21.2.101:0/55304138 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f28701a8090 0x7f28700c23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:42.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.841 7f287565d700 1 --2- 172.21.2.101:0/55304138 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f287008d680 0x7f28700c29c0 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:42.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.841 7f287565d700 1 --2- 172.21.2.101:0/55304138 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f28700b04d0 0x7f28701a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:42.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.841 7f287565d700 1 -- 172.21.2.101:0/55304138 >> 172.21.2.101:0/55304138 conn(0x7f28700be530 msgr2=0x7f287009f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:42.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.841 7f287565d700 1 -- 172.21.2.101:0/55304138 shutdown_connections 2021-02-10T11:41:42.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:42.841 7f287565d700 1 -- 172.21.2.101:0/55304138 wait complete. 2021-02-10T11:41:42.851 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:41:43.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.264 7f3314295700 1 Processor -- start 2021-02-10T11:41:43.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.264 7f3314295700 1 -- start start 2021-02-10T11:41:43.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.264 7f3314295700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f330c0d1870 0x7f330c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:43.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.264 7f3314295700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f330c0fe1c0 con 0x7f330c1b0ae0 2021-02-10T11:41:43.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.264 7f3314295700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f330c109a60 con 0x7f330c0cff70 2021-02-10T11:41:43.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.264 7f3314295700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f330c088070 con 0x7f330c0d1870 2021-02-10T11:41:43.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.265 7f33118b0700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f330c1b0ae0 0x7f330c108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58062/0 (socket says 172.21.2.101:58062) 2021-02-10T11:41:43.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.265 7f33118b0700 1 -- 172.21.2.101:0/639884842 learned_addr learned my addr 172.21.2.101:0/639884842 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:43.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.265 7f33120b1700 1 --2- 172.21.2.101:0/639884842 >> v2:172.21.2.108:3300/0 conn(0x7f330c0d1870 0x7f330c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:43.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.265 7f33110af700 1 -- 172.21.2.101:0/639884842 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 202917525 0 0) 0x7f33080010b0 con 0x7f330c1b0ae0 2021-02-10T11:41:43.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.265 7f33110af700 1 -- 172.21.2.101:0/639884842 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32f8002fe0 con 0x7f330c1b0ae0 2021-02-10T11:41:43.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.265 7f33120b1700 1 -- 172.21.2.101:0/639884842 >> v1:172.21.2.108:6789/0 conn(0x7f330c0cff70 legacy=0x7f330c109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:43.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.265 7f33120b1700 1 -- 172.21.2.101:0/639884842 >> v1:172.21.2.101:6789/0 conn(0x7f330c1b0ae0 legacy=0x7f330c108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:43.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.265 7f33120b1700 1 -- 172.21.2.101:0/639884842 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330c088260 con 0x7f330c0d1870 2021-02-10T11:41:43.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.267 7f33120b1700 1 --2- 172.21.2.101:0/639884842 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f330c0d1870 0x7f330c1b04c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f3300006650 tx=0x7f3300009a80).ready entity=mon.1 client_cookie=dd71a629aae4f9c7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:43.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.267 7f33110af700 1 -- 172.21.2.101:0/639884842 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f330000c0c0 con 0x7f330c0d1870 2021-02-10T11:41:43.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.267 7f33110af700 1 -- 172.21.2.101:0/639884842 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330000c340 con 0x7f330c0d1870 2021-02-10T11:41:43.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.267 7f33110af700 1 -- 172.21.2.101:0/639884842 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f330000c860 con 0x7f330c0d1870 2021-02-10T11:41:43.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.267 7f3314295700 1 -- 172.21.2.101:0/639884842 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f330c0d1870 msgr2=0x7f330c1b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:43.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.267 7f3314295700 1 --2- 172.21.2.101:0/639884842 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f330c0d1870 0x7f330c1b04c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f3300006650 tx=0x7f3300009a80).stop 2021-02-10T11:41:43.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.268 7f3314295700 1 -- 172.21.2.101:0/639884842 shutdown_connections 2021-02-10T11:41:43.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.268 7f3314295700 1 --2- 172.21.2.101:0/639884842 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f330c0d1870 0x7f330c1b04c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:43.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.268 7f3314295700 1 -- 172.21.2.101:0/639884842 >> 172.21.2.101:0/639884842 conn(0x7f330c0ce420 msgr2=0x7f330c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:43.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.268 7f3314295700 1 -- 172.21.2.101:0/639884842 shutdown_connections 2021-02-10T11:41:43.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.268 7f3314295700 1 -- 172.21.2.101:0/639884842 wait complete. 2021-02-10T11:41:43.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.282 7f3314295700 1 Processor -- start 2021-02-10T11:41:43.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.282 7f3314295700 1 -- start start 2021-02-10T11:41:43.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.282 7f3314295700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f330c0c3820 0x7f330c084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:43.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.282 7f3314295700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f330c0a0290 0x7f330c0852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:43.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.282 7f3314295700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f330c086270 0x7f330c086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:43.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.282 7f3314295700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c0c0700 con 0x7f330c0a0290 2021-02-10T11:41:43.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.282 7f3314295700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c0858f0 con 0x7f330c0c3820 2021-02-10T11:41:43.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.282 7f3314295700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330c085b10 con 0x7f330c086270 2021-02-10T11:41:43.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.284 7f33118b0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f330c0a0290 0x7f330c0852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:43.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.284 7f33120b1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f330c0c3820 0x7f330c084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:43.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.284 7f33118b0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f330c0a0290 0x7f330c0852d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57578/0 (socket says 172.21.2.101:57578) 2021-02-10T11:41:43.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.284 7f33120b1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f330c0c3820 0x7f330c084cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:39828/0 (socket says 172.21.2.101:39828) 2021-02-10T11:41:43.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.284 7f33118b0700 1 -- 172.21.2.101:0/969637422 learned_addr learned my addr 172.21.2.101:0/969637422 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:43.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.284 7f33128b2700 1 --2- 172.21.2.101:0/969637422 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f330c086270 0x7f330c086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:43.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.284 7f33118b0700 1 -- 172.21.2.101:0/969637422 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f330c086270 msgr2=0x7f330c086670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:43.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.284 7f33118b0700 1 --2- 172.21.2.101:0/969637422 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f330c086270 0x7f330c086670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:43.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.284 7f33118b0700 1 -- 172.21.2.101:0/969637422 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f330c0c3820 msgr2=0x7f330c084cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:43.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.284 7f33118b0700 1 --2- 172.21.2.101:0/969637422 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f330c0c3820 0x7f330c084cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:43.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.284 7f33118b0700 1 -- 172.21.2.101:0/969637422 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330c086c90 con 0x7f330c0a0290 2021-02-10T11:41:43.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.284 7f33118b0700 1 --2- 172.21.2.101:0/969637422 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f330c0a0290 0x7f330c0852d0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 rx=0x7f3308001c60 tx=0x7f330800a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:43.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.284 7f32feffd700 1 -- 172.21.2.101:0/969637422 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f330800acc0 con 0x7f330c0a0290 2021-02-10T11:41:43.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.284 7f32feffd700 1 -- 172.21.2.101:0/969637422 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3308004350 con 0x7f330c0a0290 2021-02-10T11:41:43.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.284 7f3314295700 1 -- 172.21.2.101:0/969637422 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f330c086c90 con 0x7f330c0a0290 2021-02-10T11:41:43.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.284 7f32feffd700 1 -- 172.21.2.101:0/969637422 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f33080047b0 con 0x7f330c0a0290 2021-02-10T11:41:43.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.284 7f3314295700 1 -- 172.21.2.101:0/969637422 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f330c1a2f00 con 0x7f330c0a0290 2021-02-10T11:41:43.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.284 7f32feffd700 1 -- 172.21.2.101:0/969637422 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3308007f30 con 0x7f330c0a0290 2021-02-10T11:41:43.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.284 7f32feffd700 1 --2- 172.21.2.101:0/969637422 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32f4018730 0x7f32f401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:43.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.284 7f32feffd700 1 -- 172.21.2.101:0/969637422 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3308032630 con 0x7f330c0a0290 2021-02-10T11:41:43.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.286 7f33120b1700 1 --2- 172.21.2.101:0/969637422 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32f4018730 0x7f32f401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:43.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.286 7f33120b1700 1 --2- 172.21.2.101:0/969637422 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32f4018730 0x7f32f401ac10 crc :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:43.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.315 7f3314295700 1 -- 172.21.2.101:0/969637422 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f330c082a70 con 0x7f330c0a0290 2021-02-10T11:41:43.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.317 7f32feffd700 1 -- 172.21.2.101:0/969637422 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3308000940 con 0x7f330c0a0290 2021-02-10T11:41:43.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.445 7f3314295700 1 -- 172.21.2.101:0/969637422 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f330c082a70 con 0x7f330c0a0290 2021-02-10T11:41:43.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.445 7f32feffd700 1 -- 172.21.2.101:0/969637422 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v120) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3308007f30 con 0x7f330c0a0290 2021-02-10T11:41:43.448 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:41:43.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.448 7f3314295700 1 -- 172.21.2.101:0/969637422 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32f4018730 msgr2=0x7f32f401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:43.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.448 7f3314295700 1 --2- 172.21.2.101:0/969637422 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32f4018730 0x7f32f401ac10 crc :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:43.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.448 7f3314295700 1 -- 172.21.2.101:0/969637422 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f330c0a0290 msgr2=0x7f330c0852d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:43.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.448 7f3314295700 1 --2- 172.21.2.101:0/969637422 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f330c0a0290 0x7f330c0852d0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 rx=0x7f3308001c60 tx=0x7f330800a8d0).stop 2021-02-10T11:41:43.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.448 7f3314295700 1 -- 172.21.2.101:0/969637422 shutdown_connections 2021-02-10T11:41:43.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.448 7f3314295700 1 --2- 172.21.2.101:0/969637422 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f32f4018730 0x7f32f401ac10 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:43.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.448 7f3314295700 1 --2- 172.21.2.101:0/969637422 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f330c086270 0x7f330c086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:43.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.448 7f3314295700 1 --2- 172.21.2.101:0/969637422 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f330c0a0290 0x7f330c0852d0 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:43.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.448 7f3314295700 1 --2- 172.21.2.101:0/969637422 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f330c0c3820 0x7f330c084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:43.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.448 7f3314295700 1 -- 172.21.2.101:0/969637422 >> 172.21.2.101:0/969637422 conn(0x7f330c0872c0 msgr2=0x7f330c095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:43.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.448 7f3314295700 1 -- 172.21.2.101:0/969637422 shutdown_connections 2021-02-10T11:41:43.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.448 7f3314295700 1 -- 172.21.2.101:0/969637422 wait complete. 2021-02-10T11:41:43.458 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:41:43.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.871 7f8724244700 1 Processor -- start 2021-02-10T11:41:43.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.871 7f8724244700 1 -- start start 2021-02-10T11:41:43.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.872 7f8724244700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f871c0d1c60 0x7f871c0cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:43.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.872 7f8724244700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871c0d0160 con 0x7f871c0cf570 2021-02-10T11:41:43.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.872 7f8724244700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871c0fe1b0 con 0x7f871c0d1860 2021-02-10T11:41:43.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.872 7f8724244700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f871c1b0ca0 con 0x7f871c0d1c60 2021-02-10T11:41:43.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.872 7f8722861700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f871c0cf570 0x7f871c0cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58078/0 (socket says 172.21.2.101:58078) 2021-02-10T11:41:43.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.872 7f8722861700 1 -- 172.21.2.101:0/490744790 learned_addr learned my addr 172.21.2.101:0/490744790 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:43.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.872 7f872185f700 1 --2- 172.21.2.101:0/490744790 >> v2:172.21.2.101:3300/0 conn(0x7f871c0d1c60 0x7f871c0cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:43.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.872 7f872185f700 1 -- 172.21.2.101:0/490744790 >> v1:172.21.2.108:6789/0 conn(0x7f871c0d1860 legacy=0x7f871c1b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:43.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.872 7f872105e700 1 -- 172.21.2.101:0/490744790 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3506178311 0 0) 0x7f8714001500 con 0x7f871c0cf570 2021-02-10T11:41:43.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.872 7f872185f700 1 -- 172.21.2.101:0/490744790 >> v1:172.21.2.101:6789/0 conn(0x7f871c0cf570 legacy=0x7f871c0cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:43.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.872 7f872185f700 1 -- 172.21.2.101:0/490744790 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871c109a50 con 0x7f871c0d1c60 2021-02-10T11:41:43.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.872 7f872185f700 1 --2- 172.21.2.101:0/490744790 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f871c0d1c60 0x7f871c0cef50 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 rx=0x7f8718006650 tx=0x7f8718009a80).ready entity=mon.0 client_cookie=ce9e46d2b80194c6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:43.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.873 7f872105e700 1 -- 172.21.2.101:0/490744790 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f87180009b0 con 0x7f871c0d1c60 2021-02-10T11:41:43.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.873 7f872105e700 1 -- 172.21.2.101:0/490744790 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8718006740 con 0x7f871c0d1c60 2021-02-10T11:41:43.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.873 7f872105e700 1 -- 172.21.2.101:0/490744790 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f871800c0c0 con 0x7f871c0d1c60 2021-02-10T11:41:43.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.873 7f8724244700 1 -- 172.21.2.101:0/490744790 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f871c0d1c60 msgr2=0x7f871c0cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:43.891 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.873 7f8724244700 1 --2- 172.21.2.101:0/490744790 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f871c0d1c60 0x7f871c0cef50 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 rx=0x7f8718006650 tx=0x7f8718009a80).stop 2021-02-10T11:41:43.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.873 7f8724244700 1 -- 172.21.2.101:0/490744790 shutdown_connections 2021-02-10T11:41:43.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.873 7f8724244700 1 --2- 172.21.2.101:0/490744790 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f871c0d1c60 0x7f871c0cef50 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:43.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.874 7f8724244700 1 -- 172.21.2.101:0/490744790 >> 172.21.2.101:0/490744790 conn(0x7f871c0ce410 msgr2=0x7f871c0d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:43.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.874 7f8724244700 1 -- 172.21.2.101:0/490744790 shutdown_connections 2021-02-10T11:41:43.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.874 7f8724244700 1 -- 172.21.2.101:0/490744790 wait complete. 2021-02-10T11:41:43.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.888 7f8724244700 1 Processor -- start 2021-02-10T11:41:43.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.888 7f8724244700 1 -- start start 2021-02-10T11:41:43.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.889 7f8724244700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f871c1a05e0 0x7f871c18b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:43.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.889 7f8724244700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f871c0b8780 0x7f871c0bfa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:43.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.889 7f8724244700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f871c0c0030 0x7f871c189e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:43.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.889 7f8724244700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c0b82a0 con 0x7f871c0c0030 2021-02-10T11:41:43.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.889 7f8724244700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c0c0430 con 0x7f871c0b8780 2021-02-10T11:41:43.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.889 7f8724244700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871c0c0650 con 0x7f871c1a05e0 2021-02-10T11:41:43.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.889 7f8722861700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f871c0c0030 0x7f871c189e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:43.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.889 7f8722861700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f871c0c0030 0x7f871c189e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57586/0 (socket says 172.21.2.101:57586) 2021-02-10T11:41:43.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.889 7f8722861700 1 -- 172.21.2.101:0/1817011048 learned_addr learned my addr 172.21.2.101:0/1817011048 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:43.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.889 7f8722861700 1 -- 172.21.2.101:0/1817011048 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f871c1a05e0 msgr2=0x7f871c18b970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:43.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.889 7f8722861700 1 --2- 172.21.2.101:0/1817011048 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f871c1a05e0 0x7f871c18b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:43.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.889 7f8722861700 1 -- 172.21.2.101:0/1817011048 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f871c0b8780 msgr2=0x7f871c0bfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:43.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.889 7f8722861700 1 --2- 172.21.2.101:0/1817011048 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f871c0b8780 0x7f871c0bfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:43.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.889 7f8722861700 1 -- 172.21.2.101:0/1817011048 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871c18a490 con 0x7f871c0c0030 2021-02-10T11:41:43.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.889 7f8722861700 1 --2- 172.21.2.101:0/1817011048 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f871c0c0030 0x7f871c189e70 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 rx=0x7f8714001c10 tx=0x7f871400ac60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:43.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.890 7f870effd700 1 -- 172.21.2.101:0/1817011048 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8714000f90 con 0x7f871c0c0030 2021-02-10T11:41:43.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.890 7f870effd700 1 -- 172.21.2.101:0/1817011048 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8714007020 con 0x7f871c0c0030 2021-02-10T11:41:43.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.890 7f8724244700 1 -- 172.21.2.101:0/1817011048 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f871c18a490 con 0x7f871c0c0030 2021-02-10T11:41:43.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.890 7f870effd700 1 -- 172.21.2.101:0/1817011048 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f87140074b0 con 0x7f871c0c0030 2021-02-10T11:41:43.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.890 7f8724244700 1 -- 172.21.2.101:0/1817011048 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871c18aa20 con 0x7f871c0c0030 2021-02-10T11:41:43.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.890 7f870effd700 1 -- 172.21.2.101:0/1817011048 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8714007200 con 0x7f871c0c0030 2021-02-10T11:41:43.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.890 7f870effd700 1 --2- 172.21.2.101:0/1817011048 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87040186d0 0x7f870401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:43.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.890 7f870effd700 1 -- 172.21.2.101:0/1817011048 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8714032980 con 0x7f871c0c0030 2021-02-10T11:41:43.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.891 7f8722060700 1 --2- 172.21.2.101:0/1817011048 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87040186d0 0x7f870401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:43.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.891 7f8722060700 1 --2- 172.21.2.101:0/1817011048 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87040186d0 0x7f870401abd0 crc :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:43.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.921 7f8724244700 1 -- 172.21.2.101:0/1817011048 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f871c082a20 con 0x7f871c0c0030 2021-02-10T11:41:43.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:43.925 7f870effd700 1 -- 172.21.2.101:0/1817011048 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8714000f90 con 0x7f871c0c0030 2021-02-10T11:41:44.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.053 7f8724244700 1 -- 172.21.2.101:0/1817011048 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f871c082a20 con 0x7f871c0c0030 2021-02-10T11:41:44.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.054 7f870effd700 1 -- 172.21.2.101:0/1817011048 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v120) v1 ==== 100+0+10 (secure 0 0 0) 0x7f8714007020 con 0x7f871c0c0030 2021-02-10T11:41:44.057 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:41:44.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.056 7f8724244700 1 -- 172.21.2.101:0/1817011048 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87040186d0 msgr2=0x7f870401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:44.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.056 7f8724244700 1 --2- 172.21.2.101:0/1817011048 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87040186d0 0x7f870401abd0 crc :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:44.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.056 7f8724244700 1 -- 172.21.2.101:0/1817011048 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f871c0c0030 msgr2=0x7f871c189e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:44.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.056 7f8724244700 1 --2- 172.21.2.101:0/1817011048 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f871c0c0030 0x7f871c189e70 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 rx=0x7f8714001c10 tx=0x7f871400ac60).stop 2021-02-10T11:41:44.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.056 7f8724244700 1 -- 172.21.2.101:0/1817011048 shutdown_connections 2021-02-10T11:41:44.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.056 7f8724244700 1 --2- 172.21.2.101:0/1817011048 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f871c0c0030 0x7f871c189e70 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:44.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.056 7f8724244700 1 --2- 172.21.2.101:0/1817011048 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f871c0b8780 0x7f871c0bfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:44.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.056 7f8724244700 1 --2- 172.21.2.101:0/1817011048 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87040186d0 0x7f870401abd0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:44.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.056 7f8724244700 1 --2- 172.21.2.101:0/1817011048 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f871c1a05e0 0x7f871c18b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:44.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.056 7f8724244700 1 -- 172.21.2.101:0/1817011048 >> 172.21.2.101:0/1817011048 conn(0x7f871c0a6a80 msgr2=0x7f871c0ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:44.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.056 7f8724244700 1 -- 172.21.2.101:0/1817011048 shutdown_connections 2021-02-10T11:41:44.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.056 7f8724244700 1 -- 172.21.2.101:0/1817011048 wait complete. 2021-02-10T11:41:44.067 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:41:44.067 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:41:44.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.477 7f1db2722700 1 Processor -- start 2021-02-10T11:41:44.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.477 7f1db2722700 1 -- start start 2021-02-10T11:41:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.478 7f1db2722700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1dac0d0340 0x7f1dac0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.478 7f1db2722700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dac0f0b10 con 0x7f1dac0d1870 2021-02-10T11:41:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.478 7f1db2722700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dac1b0be0 con 0x7f1dac0d1c70 2021-02-10T11:41:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.478 7f1db2722700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dac0e5ed0 con 0x7f1dac0d0340 2021-02-10T11:41:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.478 7f1dab7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1dac0d0340 0x7f1dac0ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.478 7f1dabfff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f1dac0d1870 0x7f1dac1b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58088/0 (socket says 172.21.2.101:58088) 2021-02-10T11:41:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.478 7f1dabfff700 1 -- 172.21.2.101:0/661951236 learned_addr learned my addr 172.21.2.101:0/661951236 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.479 7f1daaffd700 1 -- 172.21.2.101:0/661951236 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1629536041 0 0) 0x7f1d9c000ce0 con 0x7f1dac0d1870 2021-02-10T11:41:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.479 7f1daaffd700 1 -- 172.21.2.101:0/661951236 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d94002fe0 con 0x7f1dac0d1870 2021-02-10T11:41:44.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.479 7f1dab7fe700 1 -- 172.21.2.101:0/661951236 >> v1:172.21.2.108:6789/0 conn(0x7f1dac0d1c70 legacy=0x7f1dac0efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:44.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.479 7f1dab7fe700 1 -- 172.21.2.101:0/661951236 >> v1:172.21.2.101:6789/0 conn(0x7f1dac0d1870 legacy=0x7f1dac1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:44.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.479 7f1dab7fe700 1 -- 172.21.2.101:0/661951236 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dac0f0450 con 0x7f1dac0d0340 2021-02-10T11:41:44.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.479 7f1dab7fe700 1 --2- 172.21.2.101:0/661951236 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dac0d0340 0x7f1dac0ef640 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 rx=0x7f1da0006650 tx=0x7f1da0009a80).ready entity=mon.0 client_cookie=bd241524a342dae6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:44.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.479 7f1daaffd700 1 -- 172.21.2.101:0/661951236 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1da00009b0 con 0x7f1dac0d0340 2021-02-10T11:41:44.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.479 7f1daaffd700 1 -- 172.21.2.101:0/661951236 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da000c0c0 con 0x7f1dac0d0340 2021-02-10T11:41:44.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.479 7f1daaffd700 1 -- 172.21.2.101:0/661951236 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1da000c5e0 con 0x7f1dac0d0340 2021-02-10T11:41:44.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.479 7f1db2722700 1 -- 172.21.2.101:0/661951236 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dac0d0340 msgr2=0x7f1dac0ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:44.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.479 7f1db2722700 1 --2- 172.21.2.101:0/661951236 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dac0d0340 0x7f1dac0ef640 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 rx=0x7f1da0006650 tx=0x7f1da0009a80).stop 2021-02-10T11:41:44.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.480 7f1db2722700 1 -- 172.21.2.101:0/661951236 shutdown_connections 2021-02-10T11:41:44.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.480 7f1db2722700 1 --2- 172.21.2.101:0/661951236 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dac0d0340 0x7f1dac0ef640 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:44.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.480 7f1db2722700 1 -- 172.21.2.101:0/661951236 >> 172.21.2.101:0/661951236 conn(0x7f1dac0ce450 msgr2=0x7f1dac0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:44.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.480 7f1db2722700 1 -- 172.21.2.101:0/661951236 shutdown_connections 2021-02-10T11:41:44.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.480 7f1db2722700 1 -- 172.21.2.101:0/661951236 wait complete. 2021-02-10T11:41:44.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.495 7f1db2722700 1 Processor -- start 2021-02-10T11:41:44.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.495 7f1db2722700 1 -- start start 2021-02-10T11:41:44.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.495 7f1db2722700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1dac18e810 0x7f1dac0ef200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:44.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.495 7f1db2722700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dac0ef820 0x7f1dac18d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:44.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.495 7f1db2722700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1dac18ec10 0x7f1dac18ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:44.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.495 7f1db2722700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac19e010 con 0x7f1dac0ef820 2021-02-10T11:41:44.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.495 7f1db2722700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac0ac6c0 con 0x7f1dac18ec10 2021-02-10T11:41:44.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.495 7f1db2722700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac18e3f0 con 0x7f1dac18e810 2021-02-10T11:41:44.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.496 7f1dab7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dac0ef820 0x7f1dac18d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:44.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.496 7f1dab7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dac0ef820 0x7f1dac18d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57600/0 (socket says 172.21.2.101:57600) 2021-02-10T11:41:44.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.496 7f1dab7fe700 1 -- 172.21.2.101:0/2956453557 learned_addr learned my addr 172.21.2.101:0/2956453557 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:44.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.496 7f1dabfff700 1 --2- 172.21.2.101:0/2956453557 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1dac18e810 0x7f1dac0ef200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:44.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.496 7f1dab7fe700 1 -- 172.21.2.101:0/2956453557 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1dac18e810 msgr2=0x7f1dac0ef200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:44.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.496 7f1dab7fe700 1 --2- 172.21.2.101:0/2956453557 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1dac18e810 0x7f1dac0ef200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:44.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.496 7f1dab7fe700 1 -- 172.21.2.101:0/2956453557 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1dac18ec10 msgr2=0x7f1dac18ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:44.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.496 7f1dab7fe700 1 --2- 172.21.2.101:0/2956453557 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1dac18ec10 0x7f1dac18ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:44.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.496 7f1dab7fe700 1 -- 172.21.2.101:0/2956453557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dac095fe0 con 0x7f1dac0ef820 2021-02-10T11:41:44.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.496 7f1dab7fe700 1 --2- 172.21.2.101:0/2956453557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dac0ef820 0x7f1dac18d7b0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 rx=0x7f1da0009650 tx=0x7f1da000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:44.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.496 7f1da8ff9700 1 -- 172.21.2.101:0/2956453557 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1da00093a0 con 0x7f1dac0ef820 2021-02-10T11:41:44.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.497 7f1da8ff9700 1 -- 172.21.2.101:0/2956453557 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da000bde0 con 0x7f1dac0ef820 2021-02-10T11:41:44.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.497 7f1da8ff9700 1 -- 172.21.2.101:0/2956453557 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1da00052e0 con 0x7f1dac0ef820 2021-02-10T11:41:44.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.497 7f1db2722700 1 -- 172.21.2.101:0/2956453557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dac095fe0 con 0x7f1dac0ef820 2021-02-10T11:41:44.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.497 7f1db2722700 1 -- 172.21.2.101:0/2956453557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dac095fe0 con 0x7f1dac0ef820 2021-02-10T11:41:44.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.497 7f1da8ff9700 1 -- 172.21.2.101:0/2956453557 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1da0006030 con 0x7f1dac0ef820 2021-02-10T11:41:44.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.497 7f1da8ff9700 1 --2- 172.21.2.101:0/2956453557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d8c0186c0 0x7f1d8c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:44.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.497 7f1da8ff9700 1 -- 172.21.2.101:0/2956453557 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1da0032410 con 0x7f1dac0ef820 2021-02-10T11:41:44.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.497 7f1dabfff700 1 --2- 172.21.2.101:0/2956453557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d8c0186c0 0x7f1d8c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:44.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.498 7f1dabfff700 1 --2- 172.21.2.101:0/2956453557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d8c0186c0 0x7f1d8c01abc0 crc :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:44.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.528 7f1db2722700 1 -- 172.21.2.101:0/2956453557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dac19e010 con 0x7f1dac0ef820 2021-02-10T11:41:44.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.530 7f1da8ff9700 1 -- 172.21.2.101:0/2956453557 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1da0000940 con 0x7f1dac0ef820 2021-02-10T11:41:44.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.657 7f1db2722700 1 -- 172.21.2.101:0/2956453557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1dac19e010 con 0x7f1dac0ef820 2021-02-10T11:41:44.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.658 7f1da8ff9700 1 -- 172.21.2.101:0/2956453557 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f1da0006030 con 0x7f1dac0ef820 2021-02-10T11:41:44.661 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 96 2021-02-10T11:41:44.661 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:41:44.661 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":96,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/654364385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":654364385},{"type":"v1","addr":"172.21.2.101:6835","nonce":654364385}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":92}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:41:34.701339","modified":"2021-02-10 11:41:36.711148","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6581":{"gid":6581,"name":"b","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2146451837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2146451837},{"type":"v1","addr":"172.21.2.108:6833","nonce":2146451837}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2021-02-10T11:41:44.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.659 7f1db2722700 1 -- 172.21.2.101:0/2956453557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d8c0186c0 msgr2=0x7f1d8c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:44.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.659 7f1db2722700 1 --2- 172.21.2.101:0/2956453557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d8c0186c0 0x7f1d8c01abc0 crc :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:44.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.659 7f1db2722700 1 -- 172.21.2.101:0/2956453557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dac0ef820 msgr2=0x7f1dac18d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:44.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.659 7f1db2722700 1 --2- 172.21.2.101:0/2956453557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dac0ef820 0x7f1dac18d7b0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 rx=0x7f1da0009650 tx=0x7f1da000b960).stop 2021-02-10T11:41:44.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.660 7f1db2722700 1 -- 172.21.2.101:0/2956453557 shutdown_connections 2021-02-10T11:41:44.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.660 7f1db2722700 1 --2- 172.21.2.101:0/2956453557 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1dac18ec10 0x7f1dac18ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:44.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.660 7f1db2722700 1 --2- 172.21.2.101:0/2956453557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dac0ef820 0x7f1dac18d7b0 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:44.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.660 7f1db2722700 1 --2- 172.21.2.101:0/2956453557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d8c0186c0 0x7f1d8c01abc0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:44.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.660 7f1db2722700 1 --2- 172.21.2.101:0/2956453557 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1dac18e810 0x7f1dac0ef200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:44.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.660 7f1db2722700 1 -- 172.21.2.101:0/2956453557 >> 172.21.2.101:0/2956453557 conn(0x7f1dac0efcf0 msgr2=0x7f1dac0f1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:44.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.660 7f1db2722700 1 -- 172.21.2.101:0/2956453557 shutdown_connections 2021-02-10T11:41:44.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:44.660 7f1db2722700 1 -- 172.21.2.101:0/2956453557 wait complete. 2021-02-10T11:41:44.873 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:41:34.701339', 'modified': '2021-02-10 11:41:36.711148', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6581}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6581': {'gid': 6581, 'name': 'b', 'rank': 0, 'incarnation': 93, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/2146451837', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 2146451837}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 2146451837}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:41:44.873 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:41:44.873 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:41:45.273 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:41:45.273 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:41:45.273 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:41:45.273 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 6581, 2021-02-10T11:41:45.273 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:41:45.273 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:41:45.273 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 10.566877982999999, 2021-02-10T11:41:45.273 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 96, 2021-02-10T11:41:45.274 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 120, 2021-02-10T11:41:45.274 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 120, 2021-02-10T11:41:45.274 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 16.201337287000001 2021-02-10T11:41:45.274 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:41:45.277 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 6581, "want_state": "up:active", "state": "up:active", "rank_uptime": 10.566877982999999, "mdsmap_epoch": 96, "osdmap_epoch": 120, "osdmap_epoch_barrier": 120, "uptime": 16.201337287000001 } 2021-02-10T11:41:45.277 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:41:45.278 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-02-10T11:41:45.278 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-02-10T11:41:45.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:45.278 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:41:45.613 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:41:45.959 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:41:47.918 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:41:48.263 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:41:48.264 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:41:48.264 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:41:48.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:48.265 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:41:48.598 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-02-10T11:41:48.598 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:41:48.601 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:41:48.945 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:41:48.945 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:41:49.011 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:2021-02-10 11:41:49.009 7ffb785cef80 -1 init, newargv = 0x55dd670391f0 newargc=9 2021-02-10T11:41:49.011 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[150948]: starting ceph client 2021-02-10T11:41:49.017 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[150948]: starting fuse 2021-02-10T11:41:49.289 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:41:49.289 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:41:49.290 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:41:49.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:49.290 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:41:49.624 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:41:49.624 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2021-02-10T11:41:49.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:49.625 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:41:49.625 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:41:49.625 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:41:49.625 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:41:49.625 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:41:49.625 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:41:49.625 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:41:49.625 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-02-10T11:41:49.625 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:41:49.625 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:41:49.625 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:41:49.625 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:41:49.625 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:41:49.626 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:41:49.626 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:41:49.626 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:41:49.626 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:41:49.626 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:41:49.626 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:41:49.626 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:41:49.626 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:41:49.626 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.0")) 2021-02-10T11:41:49.626 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:41:49.986 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.0.150948.asok 2021-02-10T11:41:49.988 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.150948.asok 2021-02-10T11:41:49.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:49.989 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.150948.asok status 2021-02-10T11:41:50.379 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:41:50.379 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:41:50.379 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:41:50.379 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:41:50.379 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "0", 2021-02-10T11:41:50.379 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:41:50.380 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-02-10T11:41:50.380 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "150948", 2021-02-10T11:41:50.380 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:41:50.380 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:50.380 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:41:50.380 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:41:50.380 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 10956, 2021-02-10T11:41:50.380 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:41:50.380 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:41:50.380 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:41:50.381 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 10956 2021-02-10T11:41:50.381 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:50.381 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:41:50.381 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:50.381 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:41:50.382 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1839056777 2021-02-10T11:41:50.382 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:50.382 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:50.382 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:41:50.382 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:50.382 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:41:50.382 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1839056777 2021-02-10T11:41:50.382 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:50.382 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.10956 172.21.2.101:0/1839056777", 2021-02-10T11:41:50.382 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/1839056777", 2021-02-10T11:41:50.382 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:41:50.382 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 96, 2021-02-10T11:41:50.382 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 120, 2021-02-10T11:41:50.383 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:41:50.383 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:41:50.383 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:41:50.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:50.384 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:41:50.384 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:41:50.718 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:41:50.719 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:41:50.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:50.719 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:41:51.064 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:41:51.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.473 7fa3fe716700 1 Processor -- start 2021-02-10T11:41:51.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.473 7fa3fe716700 1 -- start start 2021-02-10T11:41:51.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.474 7fa3fe716700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3f81b0470 0x7fa3f81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:51.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.474 7fa3fe716700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa3f80ef610 0x7fa3f80efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:51.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.474 7fa3fe716700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f80f0b50 con 0x7fa3f80f0030 2021-02-10T11:41:51.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.474 7fa3fe716700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f80e52e0 con 0x7fa3f80ef610 2021-02-10T11:41:51.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.474 7fa3fe716700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f80e5490 con 0x7fa3f81b0470 2021-02-10T11:41:51.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.474 7fa3f7fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3f81b0470 0x7fa3f81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:51.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.474 7fa3f7fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3f81b0470 0x7fa3f81b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:50352/0 (socket says 172.21.2.101:50352) 2021-02-10T11:41:51.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.474 7fa3f7fff700 1 -- 172.21.2.101:0/691613097 learned_addr learned my addr 172.21.2.101:0/691613097 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:51.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.474 7fa3fcd33700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fa3f80f0030 0x7fa3f80f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:51524/0 (socket says 172.21.2.101:51524) 2021-02-10T11:41:51.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.474 7fa3f77fe700 1 --2- 172.21.2.101:0/691613097 >> v2:172.21.2.108:3300/0 conn(0x7fa3f80ef610 0x7fa3f80efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:51.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.475 7fa3f77fe700 1 -- 172.21.2.101:0/691613097 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3f81b0470 msgr2=0x7fa3f81b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.475 7fa3f77fe700 1 --2- 172.21.2.101:0/691613097 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3f81b0470 0x7fa3f81b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.475 7fa3f77fe700 1 -- 172.21.2.101:0/691613097 >> v1:172.21.2.108:6789/0 conn(0x7fa3f80f0030 legacy=0x7fa3f80f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.475 7fa3f77fe700 1 -- 172.21.2.101:0/691613097 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f80f0430 con 0x7fa3f80ef610 2021-02-10T11:41:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.475 7fa3f7fff700 1 --2- 172.21.2.101:0/691613097 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3f81b0470 0x7fa3f81b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:41:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.475 7fa3f77fe700 1 --2- 172.21.2.101:0/691613097 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3f80ef610 0x7fa3f80efa10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7fa3ec006650 tx=0x7fa3ec009a80).ready entity=mon.1 client_cookie=86aa586a187e2251 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.475 7fa3f6ffd700 1 -- 172.21.2.101:0/691613097 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa3ec0009b0 con 0x7fa3f80ef610 2021-02-10T11:41:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.475 7fa3f6ffd700 1 -- 172.21.2.101:0/691613097 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ec00c0c0 con 0x7fa3f80ef610 2021-02-10T11:41:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.475 7fa3f6ffd700 1 -- 172.21.2.101:0/691613097 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa3ec00c5e0 con 0x7fa3f80ef610 2021-02-10T11:41:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.475 7fa3fe716700 1 -- 172.21.2.101:0/691613097 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3f80ef610 msgr2=0x7fa3f80efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.475 7fa3fe716700 1 --2- 172.21.2.101:0/691613097 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3f80ef610 0x7fa3f80efa10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7fa3ec006650 tx=0x7fa3ec009a80).stop 2021-02-10T11:41:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.476 7fa3fe716700 1 -- 172.21.2.101:0/691613097 shutdown_connections 2021-02-10T11:41:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.476 7fa3fe716700 1 --2- 172.21.2.101:0/691613097 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3f80ef610 0x7fa3f80efa10 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:51.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.476 7fa3fe716700 1 --2- 172.21.2.101:0/691613097 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3f81b0470 0x7fa3f81b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:51.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.476 7fa3fe716700 1 -- 172.21.2.101:0/691613097 >> 172.21.2.101:0/691613097 conn(0x7fa3f80ce3e0 msgr2=0x7fa3f80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:51.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.476 7fa3fe716700 1 -- 172.21.2.101:0/691613097 shutdown_connections 2021-02-10T11:41:51.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.476 7fa3fe716700 1 -- 172.21.2.101:0/691613097 wait complete. 2021-02-10T11:41:51.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.491 7fa3fe716700 1 Processor -- start 2021-02-10T11:41:51.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.491 7fa3fe716700 1 -- start start 2021-02-10T11:41:51.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.491 7fa3fe716700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa3f819fec0 0x7fa3f81acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:51.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.491 7fa3fe716700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3f81aea70 0x7fa3f81aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:51.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.491 7fa3fe716700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3f81af490 0x7fa3f81a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:51.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.491 7fa3fe716700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f81a02c0 con 0x7fa3f819fec0 2021-02-10T11:41:51.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.491 7fa3fe716700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f81af890 con 0x7fa3f81aea70 2021-02-10T11:41:51.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.491 7fa3fe716700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f81afab0 con 0x7fa3f81af490 2021-02-10T11:41:51.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.492 7fa3f7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa3f819fec0 0x7fa3f81acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:51.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.492 7fa3f77fe700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3f81aea70 0x7fa3f81aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:51.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.492 7fa3fcd33700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3f81af490 0x7fa3f81a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:51.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.492 7fa3f7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa3f819fec0 0x7fa3f81acfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57622/0 (socket says 172.21.2.101:57622) 2021-02-10T11:41:51.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.492 7fa3f77fe700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3f81aea70 0x7fa3f81aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:39880/0 (socket says 172.21.2.101:39880) 2021-02-10T11:41:51.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.492 7fa3f7fff700 1 -- 172.21.2.101:0/2512814816 learned_addr learned my addr 172.21.2.101:0/2512814816 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:51.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.492 7fa3fcd33700 1 -- 172.21.2.101:0/2512814816 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3f81aea70 msgr2=0x7fa3f81aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:51.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.492 7fa3fcd33700 1 --2- 172.21.2.101:0/2512814816 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3f81aea70 0x7fa3f81aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:51.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.492 7fa3fcd33700 1 -- 172.21.2.101:0/2512814816 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa3f819fec0 msgr2=0x7fa3f81acfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:51.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.492 7fa3fcd33700 1 --2- 172.21.2.101:0/2512814816 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa3f819fec0 0x7fa3f81acfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:51.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.492 7fa3fcd33700 1 -- 172.21.2.101:0/2512814816 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f81a1100 con 0x7fa3f81af490 2021-02-10T11:41:51.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.492 7fa3fcd33700 1 --2- 172.21.2.101:0/2512814816 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3f81af490 0x7fa3f81a0ae0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fa3f0001b10 tx=0x7fa3f000aba0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:51.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.493 7fa3f4ff9700 1 -- 172.21.2.101:0/2512814816 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa3f0007280 con 0x7fa3f81af490 2021-02-10T11:41:51.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.493 7fa3f4ff9700 1 -- 172.21.2.101:0/2512814816 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f0007500 con 0x7fa3f81af490 2021-02-10T11:41:51.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.493 7fa3f4ff9700 1 -- 172.21.2.101:0/2512814816 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa3f0004920 con 0x7fa3f81af490 2021-02-10T11:41:51.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.493 7fa3fe716700 1 -- 172.21.2.101:0/2512814816 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f81a1100 con 0x7fa3f81af490 2021-02-10T11:41:51.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.493 7fa3fe716700 1 -- 172.21.2.101:0/2512814816 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f81a1690 con 0x7fa3f81af490 2021-02-10T11:41:51.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.493 7fa3f4ff9700 1 -- 172.21.2.101:0/2512814816 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa3f00048b0 con 0x7fa3f81af490 2021-02-10T11:41:51.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.493 7fa3f4ff9700 1 --2- 172.21.2.101:0/2512814816 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa3d80186c0 0x7fa3d801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:51.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.494 7fa3f4ff9700 1 -- 172.21.2.101:0/2512814816 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa3f0032800 con 0x7fa3f81af490 2021-02-10T11:41:51.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.494 7fa3f7fff700 1 --2- 172.21.2.101:0/2512814816 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa3d80186c0 0x7fa3d801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:51.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.494 7fa3f7fff700 1 --2- 172.21.2.101:0/2512814816 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa3d80186c0 0x7fa3d801abc0 crc :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:51.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.524 7fa3fe716700 1 -- 172.21.2.101:0/2512814816 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f81a1e60 con 0x7fa3f81af490 2021-02-10T11:41:51.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.527 7fa3f4ff9700 1 -- 172.21.2.101:0/2512814816 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa3f00046a0 con 0x7fa3f81af490 2021-02-10T11:41:51.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.651 7fa3fe716700 1 -- 172.21.2.101:0/2512814816 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3f81a1e60 con 0x7fa3f81af490 2021-02-10T11:41:51.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.653 7fa3f4ff9700 1 -- 172.21.2.101:0/2512814816 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+14007 (secure 0 0 0) 0x7fa3f00048b0 con 0x7fa3f81af490 2021-02-10T11:41:51.656 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:41:51.656 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":120,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:41:35.703752","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:41:32.676408","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2021-02-10 11:41:33.747692","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:41:51.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.655 7fa3fe716700 1 -- 172.21.2.101:0/2512814816 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa3d80186c0 msgr2=0x7fa3d801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:51.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.655 7fa3fe716700 1 --2- 172.21.2.101:0/2512814816 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa3d80186c0 0x7fa3d801abc0 crc :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:51.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.655 7fa3fe716700 1 -- 172.21.2.101:0/2512814816 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3f81af490 msgr2=0x7fa3f81a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:51.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.655 7fa3fe716700 1 --2- 172.21.2.101:0/2512814816 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3f81af490 0x7fa3f81a0ae0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fa3f0001b10 tx=0x7fa3f000aba0).stop 2021-02-10T11:41:51.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.656 7fa3fe716700 1 -- 172.21.2.101:0/2512814816 shutdown_connections 2021-02-10T11:41:51.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.656 7fa3fe716700 1 --2- 172.21.2.101:0/2512814816 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa3d80186c0 0x7fa3d801abc0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:51.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.656 7fa3fe716700 1 --2- 172.21.2.101:0/2512814816 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3f81af490 0x7fa3f81a0ae0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:51.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.656 7fa3fe716700 1 --2- 172.21.2.101:0/2512814816 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3f81aea70 0x7fa3f81aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:51.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.656 7fa3fe716700 1 --2- 172.21.2.101:0/2512814816 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa3f819fec0 0x7fa3f81acfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:51.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.656 7fa3fe716700 1 -- 172.21.2.101:0/2512814816 >> 172.21.2.101:0/2512814816 conn(0x7fa3f81a37a0 msgr2=0x7fa3f818c230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:51.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.656 7fa3fe716700 1 -- 172.21.2.101:0/2512814816 shutdown_connections 2021-02-10T11:41:51.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:51.656 7fa3fe716700 1 -- 172.21.2.101:0/2512814816 wait complete. 2021-02-10T11:41:51.667 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:41:32.676408', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:41:33.747692', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:41:51.667 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:41:52.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.080 7fd1757d8700 1 Processor -- start 2021-02-10T11:41:52.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.080 7fd1757d8700 1 -- start start 2021-02-10T11:41:52.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.080 7fd1757d8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1700d1c60 0x7fd1700cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:52.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.080 7fd1757d8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd1700cf570 0x7fd1700cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:52.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.080 7fd1757d8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1700fe1b0 con 0x7fd1700d1860 2021-02-10T11:41:52.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.080 7fd1757d8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1701b0ca0 con 0x7fd1700cf570 2021-02-10T11:41:52.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.080 7fd1757d8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd170109a50 con 0x7fd1700d1c60 2021-02-10T11:41:52.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.081 7fd16ed73700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1700d1c60 0x7fd1700cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:52.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.081 7fd16ed73700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1700d1c60 0x7fd1700cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:50362/0 (socket says 172.21.2.101:50362) 2021-02-10T11:41:52.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.081 7fd16ed73700 1 -- 172.21.2.101:0/4242835555 learned_addr learned my addr 172.21.2.101:0/4242835555 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:52.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.081 7fd16fd75700 1 --2- 172.21.2.101:0/4242835555 >> v2:172.21.2.108:3300/0 conn(0x7fd1700cf570 0x7fd1700cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:52.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.081 7fd16ed73700 1 -- 172.21.2.101:0/4242835555 >> v2:172.21.2.108:3300/0 conn(0x7fd1700cf570 msgr2=0x7fd1700cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:52.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.081 7fd16ed73700 1 --2- 172.21.2.101:0/4242835555 >> v2:172.21.2.108:3300/0 conn(0x7fd1700cf570 0x7fd1700cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:52.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.081 7fd16ed73700 1 -- 172.21.2.101:0/4242835555 >> v1:172.21.2.108:6789/0 conn(0x7fd1700d1860 legacy=0x7fd1701b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:52.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.081 7fd16ed73700 1 -- 172.21.2.101:0/4242835555 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd170088070 con 0x7fd1700d1c60 2021-02-10T11:41:52.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.081 7fd16ed73700 1 --2- 172.21.2.101:0/4242835555 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1700d1c60 0x7fd1700cef50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fd158006700 tx=0x7fd158009b80).ready entity=mon.2 client_cookie=ba1e2a833c7ca25 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:52.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.082 7fd16e572700 1 -- 172.21.2.101:0/4242835555 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd15800c0c0 con 0x7fd1700d1c60 2021-02-10T11:41:52.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.082 7fd16e572700 1 -- 172.21.2.101:0/4242835555 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15800c3c0 con 0x7fd1700d1c60 2021-02-10T11:41:52.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.082 7fd16e572700 1 -- 172.21.2.101:0/4242835555 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd15800c8e0 con 0x7fd1700d1c60 2021-02-10T11:41:52.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.082 7fd1757d8700 1 -- 172.21.2.101:0/4242835555 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1700d1c60 msgr2=0x7fd1700cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:52.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.082 7fd1757d8700 1 --2- 172.21.2.101:0/4242835555 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1700d1c60 0x7fd1700cef50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fd158006700 tx=0x7fd158009b80).stop 2021-02-10T11:41:52.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.082 7fd1757d8700 1 -- 172.21.2.101:0/4242835555 shutdown_connections 2021-02-10T11:41:52.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.082 7fd1757d8700 1 --2- 172.21.2.101:0/4242835555 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1700d1c60 0x7fd1700cef50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:52.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.082 7fd1757d8700 1 --2- 172.21.2.101:0/4242835555 >> v2:172.21.2.108:3300/0 conn(0x7fd1700cf570 0x7fd1700cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:52.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.082 7fd1757d8700 1 -- 172.21.2.101:0/4242835555 >> 172.21.2.101:0/4242835555 conn(0x7fd1700ce410 msgr2=0x7fd1700d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:52.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.083 7fd1757d8700 1 -- 172.21.2.101:0/4242835555 shutdown_connections 2021-02-10T11:41:52.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.083 7fd1757d8700 1 -- 172.21.2.101:0/4242835555 wait complete. 2021-02-10T11:41:52.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.097 7fd1757d8700 1 Processor -- start 2021-02-10T11:41:52.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.097 7fd1757d8700 1 -- start start 2021-02-10T11:41:52.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.097 7fd1757d8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1700c2a30 0x7fd1700c2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:52.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.097 7fd1757d8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1700a7950 0x7fd1700a7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:52.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.097 7fd1757d8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1701a71f0 0x7fd1701a75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:52.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.097 7fd1757d8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700a8440 con 0x7fd1701a71f0 2021-02-10T11:41:52.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.097 7fd1757d8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700c3790 con 0x7fd1700c2a30 2021-02-10T11:41:52.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.097 7fd1757d8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1701a7c10 con 0x7fd1700a7950 2021-02-10T11:41:52.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.098 7fd16fd75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1701a71f0 0x7fd1701a75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:52.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.098 7fd16fd75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1701a71f0 0x7fd1701a75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57640/0 (socket says 172.21.2.101:57640) 2021-02-10T11:41:52.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.098 7fd16fd75700 1 -- 172.21.2.101:0/363654593 learned_addr learned my addr 172.21.2.101:0/363654593 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:52.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.098 7fd16ed73700 1 --2- 172.21.2.101:0/363654593 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1700a7950 0x7fd1700a7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:52.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.098 7fd16f574700 1 --2- 172.21.2.101:0/363654593 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1700c2a30 0x7fd1700c2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:52.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.098 7fd16ed73700 1 -- 172.21.2.101:0/363654593 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1700c2a30 msgr2=0x7fd1700c2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:52.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.098 7fd16ed73700 1 --2- 172.21.2.101:0/363654593 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1700c2a30 0x7fd1700c2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:52.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.098 7fd16ed73700 1 -- 172.21.2.101:0/363654593 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1701a71f0 msgr2=0x7fd1701a75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:52.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.098 7fd16ed73700 1 --2- 172.21.2.101:0/363654593 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1701a71f0 0x7fd1701a75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:52.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.098 7fd16ed73700 1 -- 172.21.2.101:0/363654593 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1701a7e30 con 0x7fd1700a7950 2021-02-10T11:41:52.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.098 7fd16fd75700 1 --2- 172.21.2.101:0/363654593 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1701a71f0 0x7fd1701a75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:41:52.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.098 7fd16ed73700 1 --2- 172.21.2.101:0/363654593 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1700a7950 0x7fd1700a7d50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fd1580096d0 tx=0x7fd15800b8c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:52.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.099 7fd167fff700 1 -- 172.21.2.101:0/363654593 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd15800bcb0 con 0x7fd1700a7950 2021-02-10T11:41:52.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.099 7fd167fff700 1 -- 172.21.2.101:0/363654593 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd158002030 con 0x7fd1700a7950 2021-02-10T11:41:52.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.099 7fd167fff700 1 -- 172.21.2.101:0/363654593 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd158002550 con 0x7fd1700a7950 2021-02-10T11:41:52.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.099 7fd1757d8700 1 -- 172.21.2.101:0/363654593 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1701a7e30 con 0x7fd1700a7950 2021-02-10T11:41:52.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.099 7fd1757d8700 1 -- 172.21.2.101:0/363654593 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1700a8f80 con 0x7fd1700a7950 2021-02-10T11:41:52.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.100 7fd167fff700 1 -- 172.21.2.101:0/363654593 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd158000c90 con 0x7fd1700a7950 2021-02-10T11:41:52.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.100 7fd167fff700 1 --2- 172.21.2.101:0/363654593 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1540186c0 0x7fd15401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:52.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.100 7fd167fff700 1 -- 172.21.2.101:0/363654593 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd158032460 con 0x7fd1700a7950 2021-02-10T11:41:52.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.100 7fd16f574700 1 --2- 172.21.2.101:0/363654593 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1540186c0 0x7fd15401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:52.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.100 7fd16f574700 1 --2- 172.21.2.101:0/363654593 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1540186c0 0x7fd15401abc0 crc :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:52.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.130 7fd1757d8700 1 -- 172.21.2.101:0/363654593 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd170082a20 con 0x7fd1700a7950 2021-02-10T11:41:52.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.134 7fd167fff700 1 -- 172.21.2.101:0/363654593 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd158006390 con 0x7fd1700a7950 2021-02-10T11:41:52.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.264 7fd1757d8700 1 -- 172.21.2.101:0/363654593 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fd170082a20 con 0x7fd1700a7950 2021-02-10T11:41:52.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.266 7fd167fff700 1 -- 172.21.2.101:0/363654593 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v120) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd158000c90 con 0x7fd1700a7950 2021-02-10T11:41:52.270 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:41:52.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.268 7fd1757d8700 1 -- 172.21.2.101:0/363654593 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1540186c0 msgr2=0x7fd15401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:52.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.268 7fd1757d8700 1 --2- 172.21.2.101:0/363654593 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1540186c0 0x7fd15401abc0 crc :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:52.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.268 7fd1757d8700 1 -- 172.21.2.101:0/363654593 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1700a7950 msgr2=0x7fd1700a7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:52.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.268 7fd1757d8700 1 --2- 172.21.2.101:0/363654593 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1700a7950 0x7fd1700a7d50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fd1580096d0 tx=0x7fd15800b8c0).stop 2021-02-10T11:41:52.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.269 7fd1757d8700 1 -- 172.21.2.101:0/363654593 shutdown_connections 2021-02-10T11:41:52.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.269 7fd1757d8700 1 --2- 172.21.2.101:0/363654593 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1540186c0 0x7fd15401abc0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:52.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.269 7fd1757d8700 1 --2- 172.21.2.101:0/363654593 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1700a7950 0x7fd1700a7d50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:52.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.269 7fd1757d8700 1 --2- 172.21.2.101:0/363654593 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1701a71f0 0x7fd1701a75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:52.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.269 7fd1757d8700 1 --2- 172.21.2.101:0/363654593 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1700c2a30 0x7fd1700c2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:52.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.269 7fd1757d8700 1 -- 172.21.2.101:0/363654593 >> 172.21.2.101:0/363654593 conn(0x7fd1700b5f10 msgr2=0x7fd17009f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:52.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.269 7fd1757d8700 1 -- 172.21.2.101:0/363654593 shutdown_connections 2021-02-10T11:41:52.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.269 7fd1757d8700 1 -- 172.21.2.101:0/363654593 wait complete. 2021-02-10T11:41:52.279 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:41:52.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.693 7f2a31b80700 1 Processor -- start 2021-02-10T11:41:52.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.693 7f2a31b80700 1 -- start start 2021-02-10T11:41:52.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.694 7f2a31b80700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2a2c10aff0 0x7f2a2c0cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:52.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.694 7f2a31b80700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a2c0fe1c0 con 0x7f2a2c0d1870 2021-02-10T11:41:52.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.694 7f2a31b80700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a2c1b0cb0 con 0x7f2a2c0d1c70 2021-02-10T11:41:52.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.694 7f2a31b80700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a2c109a60 con 0x7f2a2c10aff0 2021-02-10T11:41:52.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.694 7f2a2b7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f2a2c0d1870 0x7f2a2c1b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58138/0 (socket says 172.21.2.101:58138) 2021-02-10T11:41:52.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.694 7f2a2b7fe700 1 -- 172.21.2.101:0/3832994762 learned_addr learned my addr 172.21.2.101:0/3832994762 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:52.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.694 7f2a2a7fc700 1 -- 172.21.2.101:0/3832994762 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 526560678 0 0) 0x7f2a1c001180 con 0x7f2a2c0d1870 2021-02-10T11:41:52.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.694 7f2a2a7fc700 1 -- 172.21.2.101:0/3832994762 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a14002fe0 con 0x7f2a2c0d1870 2021-02-10T11:41:52.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.694 7f2a2bfff700 1 --2- 172.21.2.101:0/3832994762 >> v2:172.21.2.108:3300/0 conn(0x7f2a2c10aff0 0x7f2a2c0cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:52.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2a7fc700 1 -- 172.21.2.101:0/3832994762 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1480307935 0 0) 0x7f2a1c001180 con 0x7f2a2c0d1870 2021-02-10T11:41:52.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2a7fc700 1 -- 172.21.2.101:0/3832994762 >> v2:172.21.2.108:3300/0 conn(0x7f2a2c10aff0 msgr2=0x7f2a2c0cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:52.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2a7fc700 1 --2- 172.21.2.101:0/3832994762 >> v2:172.21.2.108:3300/0 conn(0x7f2a2c10aff0 0x7f2a2c0cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:52.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2a7fc700 1 -- 172.21.2.101:0/3832994762 >> v1:172.21.2.108:6789/0 conn(0x7f2a2c0d1c70 legacy=0x7f2a2c0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:52.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2a7fc700 1 -- 172.21.2.101:0/3832994762 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c10b3f0 con 0x7f2a2c0d1870 2021-02-10T11:41:52.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2a7fc700 1 -- 172.21.2.101:0/3832994762 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f2a1c001680 con 0x7f2a2c0d1870 2021-02-10T11:41:52.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2a7fc700 1 -- 172.21.2.101:0/3832994762 >> v1:172.21.2.101:6789/0 conn(0x7f2a2c0d1870 legacy=0x7f2a2c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2a7fc700 1 --2- 172.21.2.101:0/3832994762 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2a14003aa0 0x7f2a14005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2a7fc700 1 --2- 172.21.2.101:0/3832994762 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a14006780 0x7f2a14008ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2a7fc700 1 --2- 172.21.2.101:0/3832994762 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a14009580 0x7f2a1400baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2a7fc700 1 -- 172.21.2.101:0/3832994762 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1400c2f0 con 0x7f2a14006780 2021-02-10T11:41:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2a7fc700 1 -- 172.21.2.101:0/3832994762 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1400c510 con 0x7f2a14003aa0 2021-02-10T11:41:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2a7fc700 1 -- 172.21.2.101:0/3832994762 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a14050740 con 0x7f2a14009580 2021-02-10T11:41:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2b7fe700 1 --2- 172.21.2.101:0/3832994762 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2a14003aa0 0x7f2a14005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2affd700 1 --2- 172.21.2.101:0/3832994762 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a14006780 0x7f2a14008ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2bfff700 1 --2- 172.21.2.101:0/3832994762 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a14009580 0x7f2a1400baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2affd700 1 -- 172.21.2.101:0/3832994762 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a14009580 msgr2=0x7f2a1400baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2affd700 1 --2- 172.21.2.101:0/3832994762 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a14009580 0x7f2a1400baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2affd700 1 -- 172.21.2.101:0/3832994762 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2a14003aa0 msgr2=0x7f2a14005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:52.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2affd700 1 --2- 172.21.2.101:0/3832994762 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2a14003aa0 0x7f2a14005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:52.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2affd700 1 -- 172.21.2.101:0/3832994762 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a14050970 con 0x7f2a14006780 2021-02-10T11:41:52.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2bfff700 1 -- 172.21.2.101:0/3832994762 reap_dead start 2021-02-10T11:41:52.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2b7fe700 1 --2- 172.21.2.101:0/3832994762 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2a14003aa0 0x7f2a14005f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:41:52.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2affd700 1 --2- 172.21.2.101:0/3832994762 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a14006780 0x7f2a14008ca0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 rx=0x7f2a20006ab0 tx=0x7f2a20009ac0).ready entity=mon.0 client_cookie=7a972f6cccfa4280 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:52.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2a7fc700 1 -- 172.21.2.101:0/3832994762 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2a200066a0 con 0x7f2a14006780 2021-02-10T11:41:52.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.695 7f2a2a7fc700 1 -- 172.21.2.101:0/3832994762 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a200022e0 con 0x7f2a14006780 2021-02-10T11:41:52.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.696 7f2a2a7fc700 1 -- 172.21.2.101:0/3832994762 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2a200027c0 con 0x7f2a14006780 2021-02-10T11:41:52.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.696 7f2a31b80700 1 -- 172.21.2.101:0/3832994762 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a14006780 msgr2=0x7f2a14008ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:52.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.696 7f2a31b80700 1 --2- 172.21.2.101:0/3832994762 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a14006780 0x7f2a14008ca0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 rx=0x7f2a20006ab0 tx=0x7f2a20009ac0).stop 2021-02-10T11:41:52.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.696 7f2a31b80700 1 -- 172.21.2.101:0/3832994762 shutdown_connections 2021-02-10T11:41:52.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.696 7f2a31b80700 1 --2- 172.21.2.101:0/3832994762 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a14006780 0x7f2a14008ca0 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:52.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.696 7f2a31b80700 1 -- 172.21.2.101:0/3832994762 >> 172.21.2.101:0/3832994762 conn(0x7f2a2c0ce420 msgr2=0x7f2a2c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:52.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.696 7f2a31b80700 1 -- 172.21.2.101:0/3832994762 shutdown_connections 2021-02-10T11:41:52.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.696 7f2a31b80700 1 -- 172.21.2.101:0/3832994762 wait complete. 2021-02-10T11:41:52.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.711 7f2a31b80700 1 Processor -- start 2021-02-10T11:41:52.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.711 7f2a31b80700 1 -- start start 2021-02-10T11:41:52.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.711 7f2a31b80700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c09bcc0 0x7f2a2c09c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:52.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.711 7f2a31b80700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2a2c0a0820 0x7f2a2c0a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:52.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.711 7f2a31b80700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a2c0a1240 0x7f2a2c0858d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:52.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.711 7f2a31b80700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c0a66b0 con 0x7f2a2c09bcc0 2021-02-10T11:41:52.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.711 7f2a31b80700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c1a49b0 con 0x7f2a2c0a0820 2021-02-10T11:41:52.718 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.711 7f2a31b80700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c1a4b90 con 0x7f2a2c0a1240 2021-02-10T11:41:52.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.712 7f2a2b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c09bcc0 0x7f2a2c09c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:52.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.712 7f2a2b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c09bcc0 0x7f2a2c09c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57658/0 (socket says 172.21.2.101:57658) 2021-02-10T11:41:52.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.712 7f2a2b7fe700 1 -- 172.21.2.101:0/1768769437 learned_addr learned my addr 172.21.2.101:0/1768769437 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:52.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.712 7f2a2b7fe700 1 -- 172.21.2.101:0/1768769437 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a2c0a1240 msgr2=0x7f2a2c0858d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:52.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.712 7f2a2b7fe700 1 --2- 172.21.2.101:0/1768769437 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a2c0a1240 0x7f2a2c0858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:52.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.712 7f2a2b7fe700 1 -- 172.21.2.101:0/1768769437 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2a2c0a0820 msgr2=0x7f2a2c0a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:52.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.712 7f2a2b7fe700 1 --2- 172.21.2.101:0/1768769437 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2a2c0a0820 0x7f2a2c0a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:52.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.712 7f2a2b7fe700 1 -- 172.21.2.101:0/1768769437 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c0a1640 con 0x7f2a2c09bcc0 2021-02-10T11:41:52.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.712 7f2a2b7fe700 1 --2- 172.21.2.101:0/1768769437 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c09bcc0 0x7f2a2c09c0c0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 rx=0x7f2a1c00a570 tx=0x7f2a1c00a910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:52.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.712 7f2a13fff700 1 -- 172.21.2.101:0/1768769437 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2a1c001ba0 con 0x7f2a2c09bcc0 2021-02-10T11:41:52.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.712 7f2a13fff700 1 -- 172.21.2.101:0/1768769437 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1c0070d0 con 0x7f2a2c09bcc0 2021-02-10T11:41:52.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.712 7f2a31b80700 1 -- 172.21.2.101:0/1768769437 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2c0a1640 con 0x7f2a2c09bcc0 2021-02-10T11:41:52.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.712 7f2a31b80700 1 -- 172.21.2.101:0/1768769437 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2c0861e0 con 0x7f2a2c09bcc0 2021-02-10T11:41:52.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.712 7f2a13fff700 1 -- 172.21.2.101:0/1768769437 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2a1c0045b0 con 0x7f2a2c09bcc0 2021-02-10T11:41:52.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.713 7f2a13fff700 1 -- 172.21.2.101:0/1768769437 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2a1c007480 con 0x7f2a2c09bcc0 2021-02-10T11:41:52.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.713 7f2a13fff700 1 --2- 172.21.2.101:0/1768769437 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a0c018730 0x7f2a0c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:52.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.713 7f2a13fff700 1 -- 172.21.2.101:0/1768769437 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2a1c0327a0 con 0x7f2a2c09bcc0 2021-02-10T11:41:52.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.713 7f2a2affd700 1 --2- 172.21.2.101:0/1768769437 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a0c018730 0x7f2a0c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:52.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.713 7f2a2affd700 1 --2- 172.21.2.101:0/1768769437 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a0c018730 0x7f2a0c01ac10 crc :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:52.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.744 7f2a31b80700 1 -- 172.21.2.101:0/1768769437 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a2c086a70 con 0x7f2a2c09bcc0 2021-02-10T11:41:52.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.746 7f2a13fff700 1 -- 172.21.2.101:0/1768769437 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2a1c004ef0 con 0x7f2a2c09bcc0 2021-02-10T11:41:52.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.874 7f2a31b80700 1 -- 172.21.2.101:0/1768769437 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f2a2c086a70 con 0x7f2a2c09bcc0 2021-02-10T11:41:52.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.874 7f2a13fff700 1 -- 172.21.2.101:0/1768769437 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v120) v1 ==== 104+0+10 (secure 0 0 0) 0x7f2a1c007480 con 0x7f2a2c09bcc0 2021-02-10T11:41:52.876 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:41:52.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.876 7f2a31b80700 1 -- 172.21.2.101:0/1768769437 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a0c018730 msgr2=0x7f2a0c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:52.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.876 7f2a31b80700 1 --2- 172.21.2.101:0/1768769437 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a0c018730 0x7f2a0c01ac10 crc :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:52.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.876 7f2a31b80700 1 -- 172.21.2.101:0/1768769437 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c09bcc0 msgr2=0x7f2a2c09c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:52.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.876 7f2a31b80700 1 --2- 172.21.2.101:0/1768769437 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c09bcc0 0x7f2a2c09c0c0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 rx=0x7f2a1c00a570 tx=0x7f2a1c00a910).stop 2021-02-10T11:41:52.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.876 7f2a31b80700 1 -- 172.21.2.101:0/1768769437 shutdown_connections 2021-02-10T11:41:52.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.876 7f2a31b80700 1 --2- 172.21.2.101:0/1768769437 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a0c018730 0x7f2a0c01ac10 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:52.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.876 7f2a31b80700 1 --2- 172.21.2.101:0/1768769437 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a2c0a1240 0x7f2a2c0858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:52.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.876 7f2a31b80700 1 --2- 172.21.2.101:0/1768769437 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2a2c0a0820 0x7f2a2c0a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:52.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.876 7f2a31b80700 1 --2- 172.21.2.101:0/1768769437 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a2c09bcc0 0x7f2a2c09c0c0 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:52.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.876 7f2a31b80700 1 -- 172.21.2.101:0/1768769437 >> 172.21.2.101:0/1768769437 conn(0x7f2a2c0bef00 msgr2=0x7f2a2c09d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:52.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.876 7f2a31b80700 1 -- 172.21.2.101:0/1768769437 shutdown_connections 2021-02-10T11:41:52.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:52.876 7f2a31b80700 1 -- 172.21.2.101:0/1768769437 wait complete. 2021-02-10T11:41:52.886 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:41:53.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.298 7ff3ed164700 1 Processor -- start 2021-02-10T11:41:53.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.298 7ff3ed164700 1 -- start start 2021-02-10T11:41:53.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.299 7ff3ed164700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff3e80d1c70 0x7ff3e80cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:53.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.299 7ff3ed164700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3e80cf580 0x7ff3e80cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:53.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.299 7ff3ed164700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e80fe1c0 con 0x7ff3e80d1870 2021-02-10T11:41:53.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.299 7ff3ed164700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e81b0cb0 con 0x7ff3e80d1c70 2021-02-10T11:41:53.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.299 7ff3ed164700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e8109a60 con 0x7ff3e80cf580 2021-02-10T11:41:53.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.299 7ff3e6d9d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff3e80d1870 0x7ff3e81b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58158/0 (socket says 172.21.2.101:58158) 2021-02-10T11:41:53.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.299 7ff3e6d9d700 1 -- 172.21.2.101:0/2604461443 learned_addr learned my addr 172.21.2.101:0/2604461443 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:41:53.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.299 7ff3e759e700 1 --2- 172.21.2.101:0/2604461443 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3e80cf580 0x7ff3e80cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:53.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.299 7ff3e659c700 1 --2- 172.21.2.101:0/2604461443 >> v2:172.21.2.108:3300/0 conn(0x7ff3e80d1c70 0x7ff3e80cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:53.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.300 7ff3e5d9b700 1 -- 172.21.2.101:0/2604461443 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1831898544 0 0) 0x7ff3d8000ce0 con 0x7ff3e80d1870 2021-02-10T11:41:53.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.300 7ff3e5d9b700 1 -- 172.21.2.101:0/2604461443 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3d4002fe0 con 0x7ff3e80d1870 2021-02-10T11:41:53.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.300 7ff3e5d9b700 1 -- 172.21.2.101:0/2604461443 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2112128159 0 0) 0x7ff3d80010e0 con 0x7ff3e80d1870 2021-02-10T11:41:53.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.300 7ff3e659c700 1 -- 172.21.2.101:0/2604461443 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3e80cf580 msgr2=0x7ff3e80cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:53.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.300 7ff3e659c700 1 --2- 172.21.2.101:0/2604461443 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3e80cf580 0x7ff3e80cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:53.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.300 7ff3e659c700 1 -- 172.21.2.101:0/2604461443 >> v1:172.21.2.101:6789/0 conn(0x7ff3e80d1870 legacy=0x7ff3e81b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:53.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.300 7ff3e659c700 1 -- 172.21.2.101:0/2604461443 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e8088070 con 0x7ff3e80d1c70 2021-02-10T11:41:53.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.300 7ff3e659c700 1 --2- 172.21.2.101:0/2604461443 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff3e80d1c70 0x7ff3e80cef60 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7ff3d00069d0 tx=0x7ff3d0009e40).ready entity=mon.1 client_cookie=64adbf900341bf24 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:53.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.300 7ff3e5d9b700 1 -- 172.21.2.101:0/2604461443 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff3d000b510 con 0x7ff3e80d1c70 2021-02-10T11:41:53.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.300 7ff3e5d9b700 1 -- 172.21.2.101:0/2604461443 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d000b790 con 0x7ff3e80d1c70 2021-02-10T11:41:53.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.300 7ff3e5d9b700 1 -- 172.21.2.101:0/2604461443 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff3d000bc70 con 0x7ff3e80d1c70 2021-02-10T11:41:53.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.300 7ff3ed164700 1 -- 172.21.2.101:0/2604461443 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff3e80d1c70 msgr2=0x7ff3e80cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:53.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.300 7ff3ed164700 1 --2- 172.21.2.101:0/2604461443 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff3e80d1c70 0x7ff3e80cef60 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7ff3d00069d0 tx=0x7ff3d0009e40).stop 2021-02-10T11:41:53.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.301 7ff3ed164700 1 -- 172.21.2.101:0/2604461443 shutdown_connections 2021-02-10T11:41:53.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.301 7ff3ed164700 1 --2- 172.21.2.101:0/2604461443 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3e80cf580 0x7ff3e80cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:53.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.301 7ff3ed164700 1 --2- 172.21.2.101:0/2604461443 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff3e80d1c70 0x7ff3e80cef60 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:53.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.301 7ff3ed164700 1 -- 172.21.2.101:0/2604461443 >> 172.21.2.101:0/2604461443 conn(0x7ff3e80ce420 msgr2=0x7ff3e80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:53.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.301 7ff3ed164700 1 -- 172.21.2.101:0/2604461443 shutdown_connections 2021-02-10T11:41:53.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.301 7ff3ed164700 1 -- 172.21.2.101:0/2604461443 wait complete. 2021-02-10T11:41:53.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.316 7ff3ed164700 1 Processor -- start 2021-02-10T11:41:53.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.316 7ff3ed164700 1 -- start start 2021-02-10T11:41:53.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.316 7ff3ed164700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff3e81a79a0 0x7ff3e81a7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:53.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.316 7ff3ed164700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3e80bdb40 0x7ff3e80bdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:53.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.316 7ff3ed164700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3e8082ca0 0x7ff3e80830a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:53.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.316 7ff3ed164700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e819d990 con 0x7ff3e8082ca0 2021-02-10T11:41:53.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.316 7ff3ed164700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e819dbb0 con 0x7ff3e81a79a0 2021-02-10T11:41:53.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.316 7ff3ed164700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80836c0 con 0x7ff3e80bdb40 2021-02-10T11:41:53.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.316 7ff3e759e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3e8082ca0 0x7ff3e80830a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:53.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.316 7ff3e759e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3e8082ca0 0x7ff3e80830a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57672/0 (socket says 172.21.2.101:57672) 2021-02-10T11:41:53.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.316 7ff3e759e700 1 -- 172.21.2.101:0/1011244952 learned_addr learned my addr 172.21.2.101:0/1011244952 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:53.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.317 7ff3e759e700 1 -- 172.21.2.101:0/1011244952 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3e80bdb40 msgr2=0x7ff3e80bdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:53.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.317 7ff3e759e700 1 --2- 172.21.2.101:0/1011244952 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3e80bdb40 0x7ff3e80bdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:53.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.317 7ff3e759e700 1 -- 172.21.2.101:0/1011244952 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff3e81a79a0 msgr2=0x7ff3e81a7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:53.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.317 7ff3e759e700 1 --2- 172.21.2.101:0/1011244952 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff3e81a79a0 0x7ff3e81a7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:53.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.317 7ff3e759e700 1 -- 172.21.2.101:0/1011244952 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e80838e0 con 0x7ff3e8082ca0 2021-02-10T11:41:53.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.317 7ff3e759e700 1 --2- 172.21.2.101:0/1011244952 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3e8082ca0 0x7ff3e80830a0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 rx=0x7ff3e000b5e0 tx=0x7ff3e000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:53.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.317 7ff3df7fe700 1 -- 172.21.2.101:0/1011244952 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff3e0000c10 con 0x7ff3e8082ca0 2021-02-10T11:41:53.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.317 7ff3df7fe700 1 -- 172.21.2.101:0/1011244952 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e000bd70 con 0x7ff3e8082ca0 2021-02-10T11:41:53.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.317 7ff3ed164700 1 -- 172.21.2.101:0/1011244952 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e80838e0 con 0x7ff3e8082ca0 2021-02-10T11:41:53.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.317 7ff3ed164700 1 -- 172.21.2.101:0/1011244952 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e8084420 con 0x7ff3e8082ca0 2021-02-10T11:41:53.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.317 7ff3df7fe700 1 -- 172.21.2.101:0/1011244952 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff3e00024d0 con 0x7ff3e8082ca0 2021-02-10T11:41:53.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.318 7ff3df7fe700 1 -- 172.21.2.101:0/1011244952 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff3e0006720 con 0x7ff3e8082ca0 2021-02-10T11:41:53.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.318 7ff3df7fe700 1 --2- 172.21.2.101:0/1011244952 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff3cc018730 0x7ff3cc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:53.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.318 7ff3df7fe700 1 -- 172.21.2.101:0/1011244952 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff3e00327d0 con 0x7ff3e8082ca0 2021-02-10T11:41:53.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.318 7ff3e6d9d700 1 --2- 172.21.2.101:0/1011244952 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff3cc018730 0x7ff3cc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:53.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.319 7ff3e6d9d700 1 --2- 172.21.2.101:0/1011244952 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff3cc018730 0x7ff3cc01abc0 crc :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:53.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.349 7ff3ed164700 1 -- 172.21.2.101:0/1011244952 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e8082a70 con 0x7ff3e8082ca0 2021-02-10T11:41:53.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.352 7ff3df7fe700 1 -- 172.21.2.101:0/1011244952 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff3e0000bc0 con 0x7ff3e8082ca0 2021-02-10T11:41:53.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.479 7ff3ed164700 1 -- 172.21.2.101:0/1011244952 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7ff3e8082a70 con 0x7ff3e8082ca0 2021-02-10T11:41:53.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.479 7ff3df7fe700 1 -- 172.21.2.101:0/1011244952 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v120) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff3e0006720 con 0x7ff3e8082ca0 2021-02-10T11:41:53.481 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:41:53.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.481 7ff3ed164700 1 -- 172.21.2.101:0/1011244952 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff3cc018730 msgr2=0x7ff3cc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:53.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.481 7ff3ed164700 1 --2- 172.21.2.101:0/1011244952 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff3cc018730 0x7ff3cc01abc0 crc :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:53.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.481 7ff3ed164700 1 -- 172.21.2.101:0/1011244952 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3e8082ca0 msgr2=0x7ff3e80830a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:53.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.481 7ff3ed164700 1 --2- 172.21.2.101:0/1011244952 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3e8082ca0 0x7ff3e80830a0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 rx=0x7ff3e000b5e0 tx=0x7ff3e000b980).stop 2021-02-10T11:41:53.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.481 7ff3ed164700 1 -- 172.21.2.101:0/1011244952 shutdown_connections 2021-02-10T11:41:53.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.481 7ff3ed164700 1 --2- 172.21.2.101:0/1011244952 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff3cc018730 0x7ff3cc01abc0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:53.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.481 7ff3ed164700 1 --2- 172.21.2.101:0/1011244952 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff3e80bdb40 0x7ff3e80bdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:53.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.481 7ff3ed164700 1 --2- 172.21.2.101:0/1011244952 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff3e8082ca0 0x7ff3e80830a0 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:53.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.481 7ff3ed164700 1 --2- 172.21.2.101:0/1011244952 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff3e81a79a0 0x7ff3e81a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:53.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.481 7ff3ed164700 1 -- 172.21.2.101:0/1011244952 >> 172.21.2.101:0/1011244952 conn(0x7ff3e80bef00 msgr2=0x7ff3e809f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:53.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.481 7ff3ed164700 1 -- 172.21.2.101:0/1011244952 shutdown_connections 2021-02-10T11:41:53.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.481 7ff3ed164700 1 -- 172.21.2.101:0/1011244952 wait complete. 2021-02-10T11:41:53.492 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:41:53.493 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:41:53.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.901 7f11b3350700 1 Processor -- start 2021-02-10T11:41:53.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.901 7f11b3350700 1 -- start start 2021-02-10T11:41:53.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.902 7f11b3350700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f11ac0d1870 0x7f11ac1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:53.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.902 7f11b3350700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f11ac1b0a10 0x7f11ac0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:53.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.902 7f11b3350700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11ac0f0b10 con 0x7f11ac0d1c70 2021-02-10T11:41:53.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.902 7f11b3350700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11ac0fe230 con 0x7f11ac1b0a10 2021-02-10T11:41:53.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.902 7f11b3350700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac0e5ed0 con 0x7f11ac0d1870 2021-02-10T11:41:53.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.902 7f11b096b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f11ac1b0a10 0x7f11ac0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:53.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.902 7f11b096b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f11ac1b0a10 0x7f11ac0cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57676/0 (socket says 172.21.2.101:57676) 2021-02-10T11:41:53.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.902 7f11b096b700 1 -- 172.21.2.101:0/3785482645 learned_addr learned my addr 172.21.2.101:0/3785482645 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:53.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.902 7f11b096b700 1 -- 172.21.2.101:0/3785482645 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f11ac0d1870 msgr2=0x7f11ac1b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:41:53.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.902 7f11b096b700 1 --2- 172.21.2.101:0/3785482645 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f11ac0d1870 0x7f11ac1b03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:53.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.902 7f11b096b700 1 -- 172.21.2.101:0/3785482645 >> v1:172.21.2.101:6789/0 conn(0x7f11ac0d1c70 legacy=0x7f11ac0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:53.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.902 7f11b096b700 1 -- 172.21.2.101:0/3785482645 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ac0f0650 con 0x7f11ac1b0a10 2021-02-10T11:41:53.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.902 7f11b096b700 1 --2- 172.21.2.101:0/3785482645 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f11ac1b0a10 0x7f11ac0cf330 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 rx=0x7f11a0006700 tx=0x7f11a0009b00).ready entity=mon.0 client_cookie=5b5c7380b7d5ca9a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:53.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.903 7f119ffff700 1 -- 172.21.2.101:0/3785482645 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f11a0000c90 con 0x7f11ac1b0a10 2021-02-10T11:41:53.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.903 7f119ffff700 1 -- 172.21.2.101:0/3785482645 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a000c0c0 con 0x7f11ac1b0a10 2021-02-10T11:41:53.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.903 7f119ffff700 1 -- 172.21.2.101:0/3785482645 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f11a000c5e0 con 0x7f11ac1b0a10 2021-02-10T11:41:53.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.903 7f11b3350700 1 -- 172.21.2.101:0/3785482645 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f11ac1b0a10 msgr2=0x7f11ac0cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:53.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.903 7f11b3350700 1 --2- 172.21.2.101:0/3785482645 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f11ac1b0a10 0x7f11ac0cf330 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 rx=0x7f11a0006700 tx=0x7f11a0009b00).stop 2021-02-10T11:41:53.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.903 7f11b3350700 1 -- 172.21.2.101:0/3785482645 shutdown_connections 2021-02-10T11:41:53.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.903 7f11b3350700 1 --2- 172.21.2.101:0/3785482645 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f11ac1b0a10 0x7f11ac0cf330 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:53.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.903 7f11b3350700 1 --2- 172.21.2.101:0/3785482645 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f11ac0d1870 0x7f11ac1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:53.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.903 7f11b3350700 1 -- 172.21.2.101:0/3785482645 >> 172.21.2.101:0/3785482645 conn(0x7f11ac0ce450 msgr2=0x7f11ac0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:53.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.903 7f11b3350700 1 -- 172.21.2.101:0/3785482645 shutdown_connections 2021-02-10T11:41:53.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.903 7f11b3350700 1 -- 172.21.2.101:0/3785482645 wait complete. 2021-02-10T11:41:53.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.918 7f11b3350700 1 Processor -- start 2021-02-10T11:41:53.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.918 7f11b3350700 1 -- start start 2021-02-10T11:41:53.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.919 7f11b3350700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f11ac0b4f30 0x7f11ac18b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:53.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.919 7f11b3350700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f11ac093f40 0x7f11ac094340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:53.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.919 7f11b3350700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f11ac094960 0x7f11ac187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:53.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.919 7f11b3350700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac094e90 con 0x7f11ac094960 2021-02-10T11:41:53.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.919 7f11b3350700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac18bff0 con 0x7f11ac093f40 2021-02-10T11:41:53.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.919 7f11b3350700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac08f780 con 0x7f11ac0b4f30 2021-02-10T11:41:53.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.919 7f11b196d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f11ac094960 0x7f11ac187e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:53.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.919 7f11b196d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f11ac094960 0x7f11ac187e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57684/0 (socket says 172.21.2.101:57684) 2021-02-10T11:41:53.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.919 7f11b196d700 1 -- 172.21.2.101:0/654664162 learned_addr learned my addr 172.21.2.101:0/654664162 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:53.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.919 7f11b196d700 1 -- 172.21.2.101:0/654664162 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f11ac0b4f30 msgr2=0x7f11ac18b9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:53.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.919 7f11b196d700 1 --2- 172.21.2.101:0/654664162 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f11ac0b4f30 0x7f11ac18b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:53.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.919 7f11b196d700 1 -- 172.21.2.101:0/654664162 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f11ac093f40 msgr2=0x7f11ac094340 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:41:53.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.919 7f11b196d700 1 --2- 172.21.2.101:0/654664162 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f11ac093f40 0x7f11ac094340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:53.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.919 7f11b196d700 1 -- 172.21.2.101:0/654664162 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ac188470 con 0x7f11ac094960 2021-02-10T11:41:53.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.919 7f11b196d700 1 --2- 172.21.2.101:0/654664162 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f11ac094960 0x7f11ac187e50 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 rx=0x7f11a4009980 tx=0x7f11a400bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:53.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.919 7f119dffb700 1 -- 172.21.2.101:0/654664162 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f11a4005870 con 0x7f11ac094960 2021-02-10T11:41:53.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.920 7f119dffb700 1 -- 172.21.2.101:0/654664162 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a4005a70 con 0x7f11ac094960 2021-02-10T11:41:53.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.920 7f119dffb700 1 -- 172.21.2.101:0/654664162 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f11a4006030 con 0x7f11ac094960 2021-02-10T11:41:53.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.920 7f11b3350700 1 -- 172.21.2.101:0/654664162 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11ac188470 con 0x7f11ac094960 2021-02-10T11:41:53.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.920 7f11b3350700 1 -- 172.21.2.101:0/654664162 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11ac188a00 con 0x7f11ac094960 2021-02-10T11:41:53.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.920 7f119dffb700 1 -- 172.21.2.101:0/654664162 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f11a4005c50 con 0x7f11ac094960 2021-02-10T11:41:53.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.920 7f119dffb700 1 --2- 172.21.2.101:0/654664162 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f118c0186c0 0x7f118c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:53.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.920 7f119dffb700 1 -- 172.21.2.101:0/654664162 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f11a4032940 con 0x7f11ac094960 2021-02-10T11:41:53.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.920 7f11b116c700 1 --2- 172.21.2.101:0/654664162 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f118c0186c0 0x7f118c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:53.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.921 7f11b116c700 1 --2- 172.21.2.101:0/654664162 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f118c0186c0 0x7f118c01abc0 crc :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:53.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.952 7f11b3350700 1 -- 172.21.2.101:0/654664162 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11ac18bff0 con 0x7f11ac094960 2021-02-10T11:41:53.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:53.954 7f119dffb700 1 -- 172.21.2.101:0/654664162 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f11a400be10 con 0x7f11ac094960 2021-02-10T11:41:54.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:54.083 7f11b3350700 1 -- 172.21.2.101:0/654664162 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11ac18bff0 con 0x7f11ac094960 2021-02-10T11:41:54.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:54.083 7f119dffb700 1 -- 172.21.2.101:0/654664162 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+2101 (secure 0 0 0) 0x7f11a400be10 con 0x7f11ac094960 2021-02-10T11:41:54.086 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 96 2021-02-10T11:41:54.086 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:41:54.086 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":96,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/654364385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":654364385},{"type":"v1","addr":"172.21.2.101:6835","nonce":654364385}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":92}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:41:34.701339","modified":"2021-02-10 11:41:36.711148","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6581":{"gid":6581,"name":"b","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2146451837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2146451837},{"type":"v1","addr":"172.21.2.108:6833","nonce":2146451837}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2021-02-10T11:41:54.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:54.085 7f11b3350700 1 -- 172.21.2.101:0/654664162 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f118c0186c0 msgr2=0x7f118c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:54.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:54.085 7f11b3350700 1 --2- 172.21.2.101:0/654664162 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f118c0186c0 0x7f118c01abc0 crc :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:54.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:54.085 7f11b3350700 1 -- 172.21.2.101:0/654664162 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f11ac094960 msgr2=0x7f11ac187e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:54.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:54.085 7f11b3350700 1 --2- 172.21.2.101:0/654664162 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f11ac094960 0x7f11ac187e50 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 rx=0x7f11a4009980 tx=0x7f11a400bcd0).stop 2021-02-10T11:41:54.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:54.085 7f11b3350700 1 -- 172.21.2.101:0/654664162 shutdown_connections 2021-02-10T11:41:54.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:54.085 7f11b3350700 1 --2- 172.21.2.101:0/654664162 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f11ac094960 0x7f11ac187e50 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:54.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:54.085 7f11b3350700 1 --2- 172.21.2.101:0/654664162 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f11ac093f40 0x7f11ac094340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:54.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:54.085 7f11b3350700 1 --2- 172.21.2.101:0/654664162 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f118c0186c0 0x7f118c01abc0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:54.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:54.085 7f11b3350700 1 --2- 172.21.2.101:0/654664162 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f11ac0b4f30 0x7f11ac18b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:54.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:54.085 7f11b3350700 1 -- 172.21.2.101:0/654664162 >> 172.21.2.101:0/654664162 conn(0x7f11ac19f8f0 msgr2=0x7f11ac1a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:54.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:54.085 7f11b3350700 1 -- 172.21.2.101:0/654664162 shutdown_connections 2021-02-10T11:41:54.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:54.086 7f11b3350700 1 -- 172.21.2.101:0/654664162 wait complete. 2021-02-10T11:41:54.298 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:41:34.701339', 'modified': '2021-02-10 11:41:36.711148', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6581}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6581': {'gid': 6581, 'name': 'b', 'rank': 0, 'incarnation': 93, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/2146451837', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 2146451837}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 2146451837}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:41:54.298 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:41:54.299 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:41:54.699 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:41:54.699 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:41:54.699 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:41:54.699 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 6581, 2021-02-10T11:41:54.700 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:41:54.700 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:41:54.700 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 19.992213132, 2021-02-10T11:41:54.700 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 96, 2021-02-10T11:41:54.700 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 120, 2021-02-10T11:41:54.700 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 120, 2021-02-10T11:41:54.700 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 25.626672288000002 2021-02-10T11:41:54.700 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:41:54.704 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 6581, "want_state": "up:active", "state": "up:active", "rank_uptime": 19.992213132, "mdsmap_epoch": 96, "osdmap_epoch": 120, "osdmap_epoch_barrier": 120, "uptime": 25.626672288000002 } 2021-02-10T11:41:54.704 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:41:54.704 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-02-10T11:41:54.705 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:41:54.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:54.705 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:41:55.041 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:41:55.384 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:41:55.385 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:41:55.728 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:41:55.728 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:41:55.729 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:41:55.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:55.729 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:41:56.064 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:41:56.065 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2021-02-10T11:41:56.065 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:41:56.069 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:41:56.413 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:41:56.413 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:41:56.480 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[151589]: starting ceph client 2021-02-10T11:41:56.481 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:2021-02-10 11:41:56.478 7f4a24648f80 -1 init, newargv = 0x557f245461c0 newargc=9 2021-02-10T11:41:56.487 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[151589]: starting fuse 2021-02-10T11:41:56.757 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:41:56.757 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:41:56.758 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:41:56.759 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:56.759 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:41:57.095 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:41:57.095 INFO:teuthology.orchestra.run.gibba001.stdout:44 2021-02-10T11:41:57.095 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2021-02-10T11:41:57.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:57.095 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:41:57.095 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:41:57.095 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:41:57.096 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:41:57.096 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:41:57.096 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:41:57.096 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:41:57.096 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-02-10T11:41:57.096 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:41:57.096 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:41:57.096 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:41:57.096 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:41:57.096 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:41:57.096 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:41:57.096 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:41:57.096 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:41:57.096 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:41:57.096 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:41:57.097 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:41:57.097 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:41:57.097 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:41:57.097 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.1")) 2021-02-10T11:41:57.097 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:41:57.458 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.1.151589.asok 2021-02-10T11:41:57.461 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.151589.asok 2021-02-10T11:41:57.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:57.462 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.151589.asok status 2021-02-10T11:41:57.852 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:41:57.853 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:41:57.853 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:41:57.853 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:41:57.853 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "1", 2021-02-10T11:41:57.853 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:41:57.853 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:41:57.853 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "151589", 2021-02-10T11:41:57.853 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:41:57.853 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:57.853 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:41:57.854 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:41:57.854 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 10986, 2021-02-10T11:41:57.854 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:41:57.854 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:41:57.854 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:41:57.854 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 10986 2021-02-10T11:41:57.854 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:57.854 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:41:57.854 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:57.854 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:41:57.854 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 3108631689 2021-02-10T11:41:57.854 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:41:57.855 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:57.855 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:41:57.855 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:41:57.855 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:41:57.855 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 3108631689 2021-02-10T11:41:57.855 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:41:57.855 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.10986 172.21.2.101:0/3108631689", 2021-02-10T11:41:57.855 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/3108631689", 2021-02-10T11:41:57.855 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:41:57.855 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 96, 2021-02-10T11:41:57.855 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 120, 2021-02-10T11:41:57.855 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:41:57.855 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:41:57.856 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:41:57.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:57.857 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:41:57.857 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:41:58.191 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:41:58.192 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:41:58.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:41:58.192 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:41:58.537 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2021-02-10T11:41:58.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.949 7f73da2ed700 1 Processor -- start 2021-02-10T11:41:58.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.949 7f73da2ed700 1 -- start start 2021-02-10T11:41:58.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.950 7f73da2ed700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f73d40d1880 0x7f73d41b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:58.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.950 7f73da2ed700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73d41b0af0 0x7f73d4108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:58.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.950 7f73da2ed700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d40fe1d0 con 0x7f73d40d1c80 2021-02-10T11:41:58.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.950 7f73da2ed700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d4109a70 con 0x7f73d40d1880 2021-02-10T11:41:58.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.950 7f73da2ed700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d4109760 con 0x7f73d41b0af0 2021-02-10T11:41:58.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.950 7f73d37fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73d41b0af0 0x7f73d4108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:58.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.950 7f73d37fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73d41b0af0 0x7f73d4108950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:50432/0 (socket says 172.21.2.101:50432) 2021-02-10T11:41:58.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.950 7f73d37fe700 1 -- 172.21.2.101:0/2208946474 learned_addr learned my addr 172.21.2.101:0/2208946474 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:58.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.950 7f73d3fff700 1 --2- 172.21.2.101:0/2208946474 >> v2:172.21.2.108:3300/0 conn(0x7f73d40d1880 0x7f73d41b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:58.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.951 7f73d37fe700 1 -- 172.21.2.101:0/2208946474 >> v2:172.21.2.108:3300/0 conn(0x7f73d40d1880 msgr2=0x7f73d41b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:41:58.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.951 7f73d37fe700 1 --2- 172.21.2.101:0/2208946474 >> v2:172.21.2.108:3300/0 conn(0x7f73d40d1880 0x7f73d41b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:58.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.951 7f73d37fe700 1 -- 172.21.2.101:0/2208946474 >> v1:172.21.2.108:6789/0 conn(0x7f73d40d1c80 legacy=0x7f73d4108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:58.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.951 7f73d37fe700 1 -- 172.21.2.101:0/2208946474 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d4088070 con 0x7f73d41b0af0 2021-02-10T11:41:58.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.951 7f73d37fe700 1 --2- 172.21.2.101:0/2208946474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73d41b0af0 0x7f73d4108950 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f73c8006700 tx=0x7f73c8009b00).ready entity=mon.2 client_cookie=8b405a5b2ed909af server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:58.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.951 7f73d2ffd700 1 -- 172.21.2.101:0/2208946474 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73c80009b0 con 0x7f73d41b0af0 2021-02-10T11:41:58.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.951 7f73d2ffd700 1 -- 172.21.2.101:0/2208946474 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c800c0c0 con 0x7f73d41b0af0 2021-02-10T11:41:58.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.951 7f73d2ffd700 1 -- 172.21.2.101:0/2208946474 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73c800c5c0 con 0x7f73d41b0af0 2021-02-10T11:41:58.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.951 7f73da2ed700 1 -- 172.21.2.101:0/2208946474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73d41b0af0 msgr2=0x7f73d4108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:58.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.951 7f73da2ed700 1 --2- 172.21.2.101:0/2208946474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73d41b0af0 0x7f73d4108950 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f73c8006700 tx=0x7f73c8009b00).stop 2021-02-10T11:41:58.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.952 7f73da2ed700 1 -- 172.21.2.101:0/2208946474 shutdown_connections 2021-02-10T11:41:58.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.952 7f73da2ed700 1 --2- 172.21.2.101:0/2208946474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73d41b0af0 0x7f73d4108950 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:58.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.952 7f73da2ed700 1 --2- 172.21.2.101:0/2208946474 >> v2:172.21.2.108:3300/0 conn(0x7f73d40d1880 0x7f73d41b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:58.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.952 7f73da2ed700 1 -- 172.21.2.101:0/2208946474 >> 172.21.2.101:0/2208946474 conn(0x7f73d40ce430 msgr2=0x7f73d40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:58.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.952 7f73da2ed700 1 -- 172.21.2.101:0/2208946474 shutdown_connections 2021-02-10T11:41:58.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.952 7f73da2ed700 1 -- 172.21.2.101:0/2208946474 wait complete. 2021-02-10T11:41:58.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.966 7f73da2ed700 1 Processor -- start 2021-02-10T11:41:58.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.967 7f73da2ed700 1 -- start start 2021-02-10T11:41:58.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.967 7f73da2ed700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73d40aef00 0x7f73d4099c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:58.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.967 7f73da2ed700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73d409a260 0x7f73d409a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:58.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.967 7f73da2ed700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73d409ac80 0x7f73d419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:58.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.967 7f73da2ed700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d419bdd0 con 0x7f73d409a260 2021-02-10T11:41:58.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.967 7f73da2ed700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d419de70 con 0x7f73d409ac80 2021-02-10T11:41:58.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.967 7f73da2ed700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d419a2d0 con 0x7f73d40aef00 2021-02-10T11:41:58.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.967 7f73d37fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73d409a260 0x7f73d409a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:58.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.967 7f73d37fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73d409a260 0x7f73d409a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57708/0 (socket says 172.21.2.101:57708) 2021-02-10T11:41:58.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.967 7f73d37fe700 1 -- 172.21.2.101:0/2000459666 learned_addr learned my addr 172.21.2.101:0/2000459666 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:41:58.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.968 7f73d37fe700 1 -- 172.21.2.101:0/2000459666 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73d40aef00 msgr2=0x7f73d4099c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:58.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.968 7f73d37fe700 1 --2- 172.21.2.101:0/2000459666 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73d40aef00 0x7f73d4099c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:58.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.968 7f73d37fe700 1 -- 172.21.2.101:0/2000459666 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73d409ac80 msgr2=0x7f73d419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:58.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.968 7f73d37fe700 1 --2- 172.21.2.101:0/2000459666 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73d409ac80 0x7f73d419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:58.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.968 7f73d37fe700 1 -- 172.21.2.101:0/2000459666 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d4199f50 con 0x7f73d409a260 2021-02-10T11:41:58.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.968 7f73d37fe700 1 --2- 172.21.2.101:0/2000459666 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73d409a260 0x7f73d409a660 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 rx=0x7f73c8002260 tx=0x7f73c8002600).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:58.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.968 7f73d0ff9700 1 -- 172.21.2.101:0/2000459666 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73c8006020 con 0x7f73d409a260 2021-02-10T11:41:58.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.968 7f73d0ff9700 1 -- 172.21.2.101:0/2000459666 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c80052e0 con 0x7f73d409a260 2021-02-10T11:41:58.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.968 7f73d0ff9700 1 -- 172.21.2.101:0/2000459666 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73c8005800 con 0x7f73d409a260 2021-02-10T11:41:58.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.968 7f73da2ed700 1 -- 172.21.2.101:0/2000459666 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d4199f50 con 0x7f73d409a260 2021-02-10T11:41:58.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.968 7f73da2ed700 1 -- 172.21.2.101:0/2000459666 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d419b160 con 0x7f73d409a260 2021-02-10T11:41:58.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.969 7f73d0ff9700 1 -- 172.21.2.101:0/2000459666 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f73c8005790 con 0x7f73d409a260 2021-02-10T11:41:58.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.969 7f73d0ff9700 1 --2- 172.21.2.101:0/2000459666 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73b40186c0 0x7f73b401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:41:58.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.969 7f73d0ff9700 1 -- 172.21.2.101:0/2000459666 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f73c8032550 con 0x7f73d409a260 2021-02-10T11:41:58.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.969 7f73d3fff700 1 --2- 172.21.2.101:0/2000459666 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73b40186c0 0x7f73b401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:41:58.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.969 7f73d3fff700 1 --2- 172.21.2.101:0/2000459666 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73b40186c0 0x7f73b401abc0 crc :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:41:59.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:58.999 7f73da2ed700 1 -- 172.21.2.101:0/2000459666 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d4082a70 con 0x7f73d409a260 2021-02-10T11:41:59.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:59.002 7f73d0ff9700 1 -- 172.21.2.101:0/2000459666 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f73c80054c0 con 0x7f73d409a260 2021-02-10T11:41:59.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:59.139 7f73da2ed700 1 -- 172.21.2.101:0/2000459666 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f73d4199f50 con 0x7f73d409a260 2021-02-10T11:41:59.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:59.577 7f73d0ff9700 1 -- 172.21.2.101:0/2000459666 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v97) v1 ==== 136+0+0 (secure 0 0 0) 0x7f73c8005790 con 0x7f73d409a260 2021-02-10T11:41:59.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:59.609 7f73da2ed700 1 -- 172.21.2.101:0/2000459666 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73b40186c0 msgr2=0x7f73b401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:59.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:59.609 7f73da2ed700 1 --2- 172.21.2.101:0/2000459666 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73b40186c0 0x7f73b401abc0 crc :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:59.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:59.609 7f73da2ed700 1 -- 172.21.2.101:0/2000459666 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73d409a260 msgr2=0x7f73d409a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:41:59.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:59.609 7f73da2ed700 1 --2- 172.21.2.101:0/2000459666 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73d409a260 0x7f73d409a660 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 rx=0x7f73c8002260 tx=0x7f73c8002600).stop 2021-02-10T11:41:59.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:59.609 7f73da2ed700 1 -- 172.21.2.101:0/2000459666 shutdown_connections 2021-02-10T11:41:59.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:59.609 7f73da2ed700 1 --2- 172.21.2.101:0/2000459666 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73d409ac80 0x7f73d419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:59.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:59.609 7f73da2ed700 1 --2- 172.21.2.101:0/2000459666 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73d409a260 0x7f73d409a660 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:59.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:59.609 7f73da2ed700 1 --2- 172.21.2.101:0/2000459666 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73b40186c0 0x7f73b401abc0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:41:59.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:59.609 7f73da2ed700 1 --2- 172.21.2.101:0/2000459666 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73d40aef00 0x7f73d4099c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:41:59.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:59.609 7f73da2ed700 1 -- 172.21.2.101:0/2000459666 >> 172.21.2.101:0/2000459666 conn(0x7f73d418e810 msgr2=0x7f73d418c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:41:59.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:59.609 7f73da2ed700 1 -- 172.21.2.101:0/2000459666 shutdown_connections 2021-02-10T11:41:59.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:41:59.609 7f73da2ed700 1 -- 172.21.2.101:0/2000459666 wait complete. 2021-02-10T11:41:59.619 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs volume ls 2021-02-10T11:42:00.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.030 7fd3aa08b700 1 Processor -- start 2021-02-10T11:42:00.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.030 7fd3aa08b700 1 -- start start 2021-02-10T11:42:00.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.030 7fd3aa08b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd3a40d0340 0x7fd3a40ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:00.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.030 7fd3aa08b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a40f0b10 con 0x7fd3a40d1870 2021-02-10T11:42:00.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.030 7fd3aa08b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a41b0be0 con 0x7fd3a40d1c70 2021-02-10T11:42:00.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.030 7fd3aa08b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a40e5ed0 con 0x7fd3a40d0340 2021-02-10T11:42:00.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.030 7fd3a37fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd3a40d1870 0x7fd3a41b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58208/0 (socket says 172.21.2.101:58208) 2021-02-10T11:42:00.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.030 7fd3a37fe700 1 -- 172.21.2.101:0/2767624092 learned_addr learned my addr 172.21.2.101:0/2767624092 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:00.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.031 7fd3a27fc700 1 -- 172.21.2.101:0/2767624092 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4052929018 0 0) 0x7fd3940009b0 con 0x7fd3a40d1870 2021-02-10T11:42:00.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.031 7fd3a27fc700 1 -- 172.21.2.101:0/2767624092 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd390002fe0 con 0x7fd3a40d1870 2021-02-10T11:42:00.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.031 7fd3a2ffd700 1 --2- 172.21.2.101:0/2767624092 >> v2:172.21.2.108:3300/0 conn(0x7fd3a40d0340 0x7fd3a40ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:00.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.031 7fd3a27fc700 1 -- 172.21.2.101:0/2767624092 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 830684001 0 0) 0x7fd394001010 con 0x7fd3a40d1870 2021-02-10T11:42:00.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.031 7fd3a27fc700 1 -- 172.21.2.101:0/2767624092 >> v2:172.21.2.108:3300/0 conn(0x7fd3a40d0340 msgr2=0x7fd3a40ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:00.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.031 7fd3a27fc700 1 --2- 172.21.2.101:0/2767624092 >> v2:172.21.2.108:3300/0 conn(0x7fd3a40d0340 0x7fd3a40ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:00.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.031 7fd3a27fc700 1 -- 172.21.2.101:0/2767624092 >> v1:172.21.2.108:6789/0 conn(0x7fd3a40d1c70 legacy=0x7fd3a40efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:00.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.031 7fd3a27fc700 1 -- 172.21.2.101:0/2767624092 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a40f0450 con 0x7fd3a40d1870 2021-02-10T11:42:00.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.031 7fd3a27fc700 1 -- 172.21.2.101:0/2767624092 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fd394001590 con 0x7fd3a40d1870 2021-02-10T11:42:00.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.031 7fd3a27fc700 1 -- 172.21.2.101:0/2767624092 >> v1:172.21.2.101:6789/0 conn(0x7fd3a40d1870 legacy=0x7fd3a41b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:00.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.031 7fd3a27fc700 1 --2- 172.21.2.101:0/2767624092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd390003b10 0x7fd390006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:00.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.031 7fd3a27fc700 1 --2- 172.21.2.101:0/2767624092 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd390006830 0x7fd390008d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:00.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.031 7fd3a27fc700 1 --2- 172.21.2.101:0/2767624092 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd3900095c0 0x7fd39000bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:00.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.031 7fd3a27fc700 1 -- 172.21.2.101:0/2767624092 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39000c310 con 0x7fd390003b10 2021-02-10T11:42:00.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.031 7fd3a27fc700 1 -- 172.21.2.101:0/2767624092 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39000c530 con 0x7fd3900095c0 2021-02-10T11:42:00.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.031 7fd3a27fc700 1 -- 172.21.2.101:0/2767624092 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39000c750 con 0x7fd390006830 2021-02-10T11:42:00.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.031 7fd3a37fe700 1 --2- 172.21.2.101:0/2767624092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd390003b10 0x7fd390006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:00.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.032 7fd3a37fe700 1 -- 172.21.2.101:0/2767624092 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd390006830 msgr2=0x7fd390008d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:00.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.032 7fd3a37fe700 1 --2- 172.21.2.101:0/2767624092 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd390006830 0x7fd390008d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:00.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.032 7fd3a37fe700 1 -- 172.21.2.101:0/2767624092 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd3900095c0 msgr2=0x7fd39000bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:00.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.032 7fd3a37fe700 1 --2- 172.21.2.101:0/2767624092 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd3900095c0 0x7fd39000bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:00.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.032 7fd3a37fe700 1 -- 172.21.2.101:0/2767624092 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd390050990 con 0x7fd390003b10 2021-02-10T11:42:00.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.032 7fd3a3fff700 1 -- 172.21.2.101:0/2767624092 reap_dead start 2021-02-10T11:42:00.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.032 7fd3a37fe700 1 --2- 172.21.2.101:0/2767624092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd390003b10 0x7fd390006010 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 rx=0x7fd394007560 tx=0x7fd394007d10).ready entity=mon.0 client_cookie=db901f3ab7a7a9c3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:00.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.032 7fd3a27fc700 1 -- 172.21.2.101:0/2767624092 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd3940009b0 con 0x7fd390003b10 2021-02-10T11:42:00.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.032 7fd3a27fc700 1 -- 172.21.2.101:0/2767624092 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3940013d0 con 0x7fd390003b10 2021-02-10T11:42:00.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.032 7fd3a27fc700 1 -- 172.21.2.101:0/2767624092 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd39400a7a0 con 0x7fd390003b10 2021-02-10T11:42:00.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.032 7fd3aa08b700 1 -- 172.21.2.101:0/2767624092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd390003b10 msgr2=0x7fd390006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:00.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.032 7fd3aa08b700 1 --2- 172.21.2.101:0/2767624092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd390003b10 0x7fd390006010 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 rx=0x7fd394007560 tx=0x7fd394007d10).stop 2021-02-10T11:42:00.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.032 7fd3aa08b700 1 -- 172.21.2.101:0/2767624092 shutdown_connections 2021-02-10T11:42:00.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.032 7fd3aa08b700 1 --2- 172.21.2.101:0/2767624092 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd390003b10 0x7fd390006010 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:00.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.032 7fd3aa08b700 1 -- 172.21.2.101:0/2767624092 >> 172.21.2.101:0/2767624092 conn(0x7fd3a40ce450 msgr2=0x7fd3a40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:00.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.032 7fd3aa08b700 1 -- 172.21.2.101:0/2767624092 shutdown_connections 2021-02-10T11:42:00.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.032 7fd3aa08b700 1 -- 172.21.2.101:0/2767624092 wait complete. 2021-02-10T11:42:00.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.047 7fd3aa08b700 1 Processor -- start 2021-02-10T11:42:00.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.047 7fd3aa08b700 1 -- start start 2021-02-10T11:42:00.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.047 7fd3aa08b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3a419ffa0 0x7fd3a40eda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:00.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.047 7fd3aa08b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3a41a03a0 0x7fd3a40ee030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:00.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.047 7fd3aa08b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd3a40ee650 0x7fd3a40ebe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:00.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.047 7fd3aa08b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a40ed490 con 0x7fd3a419ffa0 2021-02-10T11:42:00.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.047 7fd3aa08b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a40eea50 con 0x7fd3a40ee650 2021-02-10T11:42:00.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.047 7fd3aa08b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a40eec30 con 0x7fd3a41a03a0 2021-02-10T11:42:00.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.048 7fd3a37fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3a419ffa0 0x7fd3a40eda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:00.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.048 7fd3a37fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3a419ffa0 0x7fd3a40eda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57726/0 (socket says 172.21.2.101:57726) 2021-02-10T11:42:00.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.048 7fd3a37fe700 1 -- 172.21.2.101:0/1418866351 learned_addr learned my addr 172.21.2.101:0/1418866351 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:00.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.048 7fd3a37fe700 1 -- 172.21.2.101:0/1418866351 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3a41a03a0 msgr2=0x7fd3a40ee030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:00.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.048 7fd3a37fe700 1 --2- 172.21.2.101:0/1418866351 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3a41a03a0 0x7fd3a40ee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:00.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.048 7fd3a37fe700 1 -- 172.21.2.101:0/1418866351 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd3a40ee650 msgr2=0x7fd3a40ebe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:00.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.048 7fd3a37fe700 1 --2- 172.21.2.101:0/1418866351 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd3a40ee650 0x7fd3a40ebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:00.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.048 7fd3a37fe700 1 -- 172.21.2.101:0/1418866351 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a40ec4a0 con 0x7fd3a419ffa0 2021-02-10T11:42:00.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.048 7fd3a37fe700 1 --2- 172.21.2.101:0/1418866351 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3a419ffa0 0x7fd3a40eda10 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 rx=0x7fd394007fc0 tx=0x7fd39400c960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:00.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.048 7fd39bfff700 1 -- 172.21.2.101:0/1418866351 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd39400cd50 con 0x7fd3a419ffa0 2021-02-10T11:42:00.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.048 7fd39bfff700 1 -- 172.21.2.101:0/1418866351 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd394007570 con 0x7fd3a419ffa0 2021-02-10T11:42:00.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.048 7fd3aa08b700 1 -- 172.21.2.101:0/1418866351 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a40ec4a0 con 0x7fd3a419ffa0 2021-02-10T11:42:00.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.048 7fd39bfff700 1 -- 172.21.2.101:0/1418866351 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd394004330 con 0x7fd3a419ffa0 2021-02-10T11:42:00.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.048 7fd3aa08b700 1 -- 172.21.2.101:0/1418866351 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a40ec4a0 con 0x7fd3a419ffa0 2021-02-10T11:42:00.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.049 7fd39bfff700 1 -- 172.21.2.101:0/1418866351 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd394004140 con 0x7fd3a419ffa0 2021-02-10T11:42:00.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.049 7fd39bfff700 1 --2- 172.21.2.101:0/1418866351 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3880186c0 0x7fd38801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:00.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.049 7fd39bfff700 1 -- 172.21.2.101:0/1418866351 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd394032430 con 0x7fd3a419ffa0 2021-02-10T11:42:00.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.049 7fd3a2ffd700 1 --2- 172.21.2.101:0/1418866351 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3880186c0 0x7fd38801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:00.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.049 7fd3a2ffd700 1 --2- 172.21.2.101:0/1418866351 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3880186c0 0x7fd38801abc0 crc :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:00.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.080 7fd3aa08b700 1 -- 172.21.2.101:0/1418866351 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a40ebc50 con 0x7fd3a419ffa0 2021-02-10T11:42:00.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.083 7fd39bfff700 1 -- 172.21.2.101:0/1418866351 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd394007b50 con 0x7fd3a419ffa0 2021-02-10T11:42:00.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.218 7fd3aa08b700 1 -- 172.21.2.101:0/1418866351 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs volume ls", "target": ["mgr", ""]}) v1 -- 0x7fd3a40ebc50 con 0x7fd3880186c0 2021-02-10T11:42:00.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.219 7fd39bfff700 1 -- 172.21.2.101:0/1418866351 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7fd38c006e70 con 0x7fd3880186c0 2021-02-10T11:42:00.221 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:42:00.221 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:00.221 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "cephfs" 2021-02-10T11:42:00.221 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:00.221 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:42:00.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.220 7fd3aa08b700 1 -- 172.21.2.101:0/1418866351 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3880186c0 msgr2=0x7fd38801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:00.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.220 7fd3aa08b700 1 --2- 172.21.2.101:0/1418866351 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3880186c0 0x7fd38801abc0 crc :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:00.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.220 7fd3aa08b700 1 -- 172.21.2.101:0/1418866351 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3a419ffa0 msgr2=0x7fd3a40eda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:00.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.220 7fd3aa08b700 1 --2- 172.21.2.101:0/1418866351 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3a419ffa0 0x7fd3a40eda10 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 rx=0x7fd394007fc0 tx=0x7fd39400c960).stop 2021-02-10T11:42:00.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.221 7fd3aa08b700 1 -- 172.21.2.101:0/1418866351 shutdown_connections 2021-02-10T11:42:00.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.221 7fd3aa08b700 1 --2- 172.21.2.101:0/1418866351 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3880186c0 0x7fd38801abc0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:00.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.221 7fd3aa08b700 1 --2- 172.21.2.101:0/1418866351 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3a41a03a0 0x7fd3a40ee030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:00.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.221 7fd3aa08b700 1 --2- 172.21.2.101:0/1418866351 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd3a40ee650 0x7fd3a40ebe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:00.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.221 7fd3aa08b700 1 --2- 172.21.2.101:0/1418866351 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3a419ffa0 0x7fd3a40eda10 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:00.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.221 7fd3aa08b700 1 -- 172.21.2.101:0/1418866351 >> 172.21.2.101:0/1418866351 conn(0x7fd3a4094d90 msgr2=0x7fd3a408a420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:00.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.221 7fd3aa08b700 1 -- 172.21.2.101:0/1418866351 shutdown_connections 2021-02-10T11:42:00.224 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.221 7fd3aa08b700 1 -- 172.21.2.101:0/1418866351 wait complete. 2021-02-10T11:42:00.231 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:42:00.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.648 7f5565d84700 1 Processor -- start 2021-02-10T11:42:00.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.648 7f5565d84700 1 -- start start 2021-02-10T11:42:00.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.648 7f5565d84700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f55600d1c70 0x7f55600cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:00.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.648 7f5565d84700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f55600cf580 0x7f55600cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:00.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.648 7f5565d84700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55600fe1c0 con 0x7f55600d1870 2021-02-10T11:42:00.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.648 7f5565d84700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55601b0cb0 con 0x7f55600cf580 2021-02-10T11:42:00.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.648 7f5565d84700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5560109a60 con 0x7f55600d1c70 2021-02-10T11:42:00.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.648 7f555ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f55600cf580 0x7f55600cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:00.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.648 7f555ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f55600cf580 0x7f55600cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57738/0 (socket says 172.21.2.101:57738) 2021-02-10T11:42:00.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.648 7f555ffff700 1 -- 172.21.2.101:0/561950144 learned_addr learned my addr 172.21.2.101:0/561950144 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:00.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.648 7f555effd700 1 --2- 172.21.2.101:0/561950144 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f55600d1c70 0x7f55600cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:00.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.649 7f555ffff700 1 -- 172.21.2.101:0/561950144 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f55600d1c70 msgr2=0x7f55600cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:00.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.649 7f555ffff700 1 --2- 172.21.2.101:0/561950144 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f55600d1c70 0x7f55600cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:00.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.649 7f555ffff700 1 -- 172.21.2.101:0/561950144 >> v1:172.21.2.108:6789/0 conn(0x7f55600d1870 legacy=0x7f55601b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:00.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.649 7f555ffff700 1 -- 172.21.2.101:0/561950144 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5560088070 con 0x7f55600cf580 2021-02-10T11:42:00.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.649 7f555effd700 1 --2- 172.21.2.101:0/561950144 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f55600d1c70 0x7f55600cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:42:00.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.649 7f555ffff700 1 --2- 172.21.2.101:0/561950144 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f55600cf580 0x7f55600cf980 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 rx=0x7f5558006f00 tx=0x7f5558009f10).ready entity=mon.0 client_cookie=28571447b88d1fd8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:00.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.649 7f555e7fc700 1 -- 172.21.2.101:0/561950144 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5558003030 con 0x7f55600cf580 2021-02-10T11:42:00.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.649 7f555e7fc700 1 -- 172.21.2.101:0/561950144 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555800b3b0 con 0x7f55600cf580 2021-02-10T11:42:00.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.649 7f555e7fc700 1 -- 172.21.2.101:0/561950144 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f555800b8d0 con 0x7f55600cf580 2021-02-10T11:42:00.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.649 7f5565d84700 1 -- 172.21.2.101:0/561950144 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f55600cf580 msgr2=0x7f55600cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:00.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.649 7f5565d84700 1 --2- 172.21.2.101:0/561950144 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f55600cf580 0x7f55600cf980 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 rx=0x7f5558006f00 tx=0x7f5558009f10).stop 2021-02-10T11:42:00.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.650 7f5565d84700 1 -- 172.21.2.101:0/561950144 shutdown_connections 2021-02-10T11:42:00.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.650 7f5565d84700 1 --2- 172.21.2.101:0/561950144 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f55600d1c70 0x7f55600cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:00.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.650 7f5565d84700 1 --2- 172.21.2.101:0/561950144 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f55600cf580 0x7f55600cf980 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:00.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.650 7f5565d84700 1 -- 172.21.2.101:0/561950144 >> 172.21.2.101:0/561950144 conn(0x7f55600ce420 msgr2=0x7f55600d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:00.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.650 7f5565d84700 1 -- 172.21.2.101:0/561950144 shutdown_connections 2021-02-10T11:42:00.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.650 7f5565d84700 1 -- 172.21.2.101:0/561950144 wait complete. 2021-02-10T11:42:00.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.665 7f5565d84700 1 Processor -- start 2021-02-10T11:42:00.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.665 7f5565d84700 1 -- start start 2021-02-10T11:42:00.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.665 7f5565d84700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5560191b10 0x7f55600ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:00.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.665 7f5565d84700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5560191f10 0x7f556019ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:00.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.665 7f5565d84700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f55601a05f0 0x7f5560082ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:00.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.665 7f5565d84700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5560192440 con 0x7f5560191f10 2021-02-10T11:42:00.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.665 7f5565d84700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55601916e0 con 0x7f55601a05f0 2021-02-10T11:42:00.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.665 7f5565d84700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5560191900 con 0x7f5560191b10 2021-02-10T11:42:00.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.666 7f555effd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5560191f10 0x7f556019ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:00.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.666 7f555effd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5560191f10 0x7f556019ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57740/0 (socket says 172.21.2.101:57740) 2021-02-10T11:42:00.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.666 7f555effd700 1 -- 172.21.2.101:0/535761622 learned_addr learned my addr 172.21.2.101:0/535761622 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:00.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.666 7f555effd700 1 -- 172.21.2.101:0/535761622 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5560191b10 msgr2=0x7f55600ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:00.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.666 7f555effd700 1 --2- 172.21.2.101:0/535761622 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5560191b10 0x7f55600ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:00.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.666 7f555effd700 1 -- 172.21.2.101:0/535761622 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f55601a05f0 msgr2=0x7f5560082ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:00.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.666 7f555effd700 1 --2- 172.21.2.101:0/535761622 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f55601a05f0 0x7f5560082ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:00.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.666 7f555effd700 1 -- 172.21.2.101:0/535761622 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55600834f0 con 0x7f5560191f10 2021-02-10T11:42:00.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.666 7f555effd700 1 --2- 172.21.2.101:0/535761622 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5560191f10 0x7f556019ffd0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 rx=0x7f5550006f40 tx=0x7f555000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:00.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.666 7f5557fff700 1 -- 172.21.2.101:0/535761622 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f555000bd80 con 0x7f5560191f10 2021-02-10T11:42:00.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.666 7f5557fff700 1 -- 172.21.2.101:0/535761622 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5550000c90 con 0x7f5560191f10 2021-02-10T11:42:00.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.666 7f5557fff700 1 -- 172.21.2.101:0/535761622 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5550002220 con 0x7f5560191f10 2021-02-10T11:42:00.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.666 7f5565d84700 1 -- 172.21.2.101:0/535761622 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55600834f0 con 0x7f5560191f10 2021-02-10T11:42:00.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.666 7f5565d84700 1 -- 172.21.2.101:0/535761622 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5560083a80 con 0x7f5560191f10 2021-02-10T11:42:00.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.667 7f5557fff700 1 -- 172.21.2.101:0/535761622 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f555000a000 con 0x7f5560191f10 2021-02-10T11:42:00.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.667 7f5557fff700 1 --2- 172.21.2.101:0/535761622 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f55440186c0 0x7f554401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:00.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.667 7f5557fff700 1 -- 172.21.2.101:0/535761622 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5550032590 con 0x7f5560191f10 2021-02-10T11:42:00.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.667 7f555f7fe700 1 --2- 172.21.2.101:0/535761622 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f55440186c0 0x7f554401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:00.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.667 7f555f7fe700 1 --2- 172.21.2.101:0/535761622 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f55440186c0 0x7f554401abc0 crc :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:00.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.698 7f5565d84700 1 -- 172.21.2.101:0/535761622 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55600fe1c0 con 0x7f5560191f10 2021-02-10T11:42:00.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.701 7f5557fff700 1 -- 172.21.2.101:0/535761622 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5550000940 con 0x7f5560191f10 2021-02-10T11:42:00.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.850 7f5565d84700 1 -- 172.21.2.101:0/535761622 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f55600fe1c0 con 0x7f5560191f10 2021-02-10T11:42:00.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.850 7f5557fff700 1 -- 172.21.2.101:0/535761622 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f555000a000 con 0x7f5560191f10 2021-02-10T11:42:00.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.852 7f5565d84700 1 -- 172.21.2.101:0/535761622 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f55440186c0 msgr2=0x7f554401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:00.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.852 7f5565d84700 1 --2- 172.21.2.101:0/535761622 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f55440186c0 0x7f554401abc0 crc :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:00.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.852 7f5565d84700 1 -- 172.21.2.101:0/535761622 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5560191f10 msgr2=0x7f556019ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:00.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.852 7f5565d84700 1 --2- 172.21.2.101:0/535761622 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5560191f10 0x7f556019ffd0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 rx=0x7f5550006f40 tx=0x7f555000b990).stop 2021-02-10T11:42:00.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.853 7f5565d84700 1 -- 172.21.2.101:0/535761622 shutdown_connections 2021-02-10T11:42:00.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.853 7f5565d84700 1 --2- 172.21.2.101:0/535761622 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f55601a05f0 0x7f5560082ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:00.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.853 7f5565d84700 1 --2- 172.21.2.101:0/535761622 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5560191f10 0x7f556019ffd0 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:00.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.853 7f5565d84700 1 --2- 172.21.2.101:0/535761622 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f55440186c0 0x7f554401abc0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:00.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.853 7f5565d84700 1 --2- 172.21.2.101:0/535761622 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5560191b10 0x7f55600ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:00.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.853 7f5565d84700 1 -- 172.21.2.101:0/535761622 >> 172.21.2.101:0/535761622 conn(0x7f55600a02d0 msgr2=0x7f556009f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:00.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.853 7f5565d84700 1 -- 172.21.2.101:0/535761622 shutdown_connections 2021-02-10T11:42:00.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:00.853 7f5565d84700 1 -- 172.21.2.101:0/535761622 wait complete. 2021-02-10T11:42:00.863 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolumegroup create cephfs group_0000000000192983 2021-02-10T11:42:01.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.276 7fedd3772700 1 Processor -- start 2021-02-10T11:42:01.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.276 7fedd3772700 1 -- start start 2021-02-10T11:42:01.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.276 7fedd3772700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fedcc0d1870 0x7fedcc1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:01.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.276 7fedd3772700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedcc0fe1c0 con 0x7fedcc0cff70 2021-02-10T11:42:01.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.276 7fedd3772700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedcc109a60 con 0x7fedcc1b0ae0 2021-02-10T11:42:01.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.276 7fedd3772700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedcc088070 con 0x7fedcc0d1870 2021-02-10T11:42:01.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.277 7fedd1d8f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fedcc0cff70 0x7fedcc109130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58242/0 (socket says 172.21.2.101:58242) 2021-02-10T11:42:01.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.277 7fedd1d8f700 1 -- 172.21.2.101:0/1841344250 learned_addr learned my addr 172.21.2.101:0/1841344250 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:01.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.277 7fedbffff700 1 -- 172.21.2.101:0/1841344250 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4040188296 0 0) 0x7fedc4000f90 con 0x7fedcc0cff70 2021-02-10T11:42:01.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.277 7fedbffff700 1 -- 172.21.2.101:0/1841344250 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedb8002fe0 con 0x7fedcc0cff70 2021-02-10T11:42:01.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.277 7fedd158e700 1 --2- 172.21.2.101:0/1841344250 >> v2:172.21.2.108:3300/0 conn(0x7fedcc0d1870 0x7fedcc1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:01.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.277 7fedbffff700 1 -- 172.21.2.101:0/1841344250 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 14969521 0 0) 0x7fedc4001460 con 0x7fedcc0cff70 2021-02-10T11:42:01.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.277 7fedbffff700 1 -- 172.21.2.101:0/1841344250 >> v2:172.21.2.108:3300/0 conn(0x7fedcc0d1870 msgr2=0x7fedcc1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:01.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.277 7fedbffff700 1 --2- 172.21.2.101:0/1841344250 >> v2:172.21.2.108:3300/0 conn(0x7fedcc0d1870 0x7fedcc1b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:01.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.278 7fedbffff700 1 -- 172.21.2.101:0/1841344250 >> v1:172.21.2.108:6789/0 conn(0x7fedcc1b0ae0 legacy=0x7fedcc108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:01.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.278 7fedbffff700 1 -- 172.21.2.101:0/1841344250 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc088260 con 0x7fedcc0cff70 2021-02-10T11:42:01.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.278 7fedbffff700 1 -- 172.21.2.101:0/1841344250 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fedc4001980 con 0x7fedcc0cff70 2021-02-10T11:42:01.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.278 7fedbffff700 1 -- 172.21.2.101:0/1841344250 >> v1:172.21.2.101:6789/0 conn(0x7fedcc0cff70 legacy=0x7fedcc109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:01.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.278 7fedbffff700 1 --2- 172.21.2.101:0/1841344250 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedb8003b10 0x7fedb8005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:01.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.278 7fedbffff700 1 --2- 172.21.2.101:0/1841344250 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedb8006820 0x7fedb8008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:01.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.278 7fedbffff700 1 --2- 172.21.2.101:0/1841344250 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fedb8009640 0x7fedb800bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:01.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.278 7fedbffff700 1 -- 172.21.2.101:0/1841344250 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb800c390 con 0x7fedb8003b10 2021-02-10T11:42:01.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.278 7fedbffff700 1 -- 172.21.2.101:0/1841344250 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb800c5b0 con 0x7fedb8009640 2021-02-10T11:42:01.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.278 7fedbffff700 1 -- 172.21.2.101:0/1841344250 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb800c7d0 con 0x7fedb8006820 2021-02-10T11:42:01.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.278 7fedd158e700 1 --2- 172.21.2.101:0/1841344250 >> v2:172.21.2.108:3300/0 conn(0x7fedcc0d1870 0x7fedcc1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:42:01.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.278 7fedd158e700 1 --2- 172.21.2.101:0/1841344250 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedb8003b10 0x7fedb8005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:01.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.278 7fedd0d8d700 1 --2- 172.21.2.101:0/1841344250 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedb8006820 0x7fedb8008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:01.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.278 7fedd1d8f700 1 --2- 172.21.2.101:0/1841344250 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fedb8009640 0x7fedb800bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:01.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.278 7fedd158e700 1 -- 172.21.2.101:0/1841344250 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedb8006820 msgr2=0x7fedb8008d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:01.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.278 7fedd158e700 1 --2- 172.21.2.101:0/1841344250 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedb8006820 0x7fedb8008d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:01.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.278 7fedd158e700 1 -- 172.21.2.101:0/1841344250 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fedb8009640 msgr2=0x7fedb800bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:01.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.278 7fedd158e700 1 --2- 172.21.2.101:0/1841344250 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fedb8009640 0x7fedb800bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:01.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.278 7fedd1d8f700 1 -- 172.21.2.101:0/1841344250 reap_dead start 2021-02-10T11:42:01.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.278 7fedd158e700 1 -- 172.21.2.101:0/1841344250 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb800ca70 con 0x7fedb8003b10 2021-02-10T11:42:01.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.278 7fedd158e700 1 --2- 172.21.2.101:0/1841344250 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedb8003b10 0x7fedb8005fa0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 rx=0x7fedc8006460 tx=0x7fedc804db30).ready entity=mon.0 client_cookie=c1cff2d3ef61c2ed server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:01.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.279 7fedbffff700 1 -- 172.21.2.101:0/1841344250 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fedc80065d0 con 0x7fedb8003b10 2021-02-10T11:42:01.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.279 7fedbffff700 1 -- 172.21.2.101:0/1841344250 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc80500c0 con 0x7fedb8003b10 2021-02-10T11:42:01.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.279 7fedbffff700 1 -- 172.21.2.101:0/1841344250 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fedc80505c0 con 0x7fedb8003b10 2021-02-10T11:42:01.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.279 7fedd3772700 1 -- 172.21.2.101:0/1841344250 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedb8003b10 msgr2=0x7fedb8005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:01.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.279 7fedd3772700 1 --2- 172.21.2.101:0/1841344250 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedb8003b10 0x7fedb8005fa0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 rx=0x7fedc8006460 tx=0x7fedc804db30).stop 2021-02-10T11:42:01.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.279 7fedd3772700 1 -- 172.21.2.101:0/1841344250 shutdown_connections 2021-02-10T11:42:01.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.279 7fedd3772700 1 --2- 172.21.2.101:0/1841344250 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedb8003b10 0x7fedb8005fa0 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:01.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.279 7fedd3772700 1 -- 172.21.2.101:0/1841344250 >> 172.21.2.101:0/1841344250 conn(0x7fedcc0ce420 msgr2=0x7fedcc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:01.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.279 7fedd3772700 1 -- 172.21.2.101:0/1841344250 shutdown_connections 2021-02-10T11:42:01.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.279 7fedd3772700 1 -- 172.21.2.101:0/1841344250 wait complete. 2021-02-10T11:42:01.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.294 7fedd3772700 1 Processor -- start 2021-02-10T11:42:01.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.294 7fedd3772700 1 -- start start 2021-02-10T11:42:01.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.295 7fedd3772700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedcc0ac2c0 0x7fedcc1a7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:01.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.295 7fedd3772700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedcc0bfb60 0x7fedcc1a7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:01.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.295 7fedd3772700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fedcc083980 0x7fedcc083d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:01.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.295 7fedd3772700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0bf740 con 0x7fedcc0ac2c0 2021-02-10T11:42:01.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.295 7fedd3772700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc1a80c0 con 0x7fedcc083980 2021-02-10T11:42:01.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.295 7fedd3772700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc1a82e0 con 0x7fedcc0bfb60 2021-02-10T11:42:01.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.295 7fedd158e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedcc0ac2c0 0x7fedcc1a7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:01.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.295 7fedd158e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedcc0ac2c0 0x7fedcc1a7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57760/0 (socket says 172.21.2.101:57760) 2021-02-10T11:42:01.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.295 7fedd158e700 1 -- 172.21.2.101:0/2204189782 learned_addr learned my addr 172.21.2.101:0/2204189782 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:01.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.295 7fedd0d8d700 1 --2- 172.21.2.101:0/2204189782 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedcc0bfb60 0x7fedcc1a7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:01.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.295 7fedd158e700 1 -- 172.21.2.101:0/2204189782 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedcc0bfb60 msgr2=0x7fedcc1a7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:01.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.295 7fedd158e700 1 --2- 172.21.2.101:0/2204189782 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedcc0bfb60 0x7fedcc1a7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:01.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.295 7fedd158e700 1 -- 172.21.2.101:0/2204189782 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fedcc083980 msgr2=0x7fedcc083d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:01.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.295 7fedd158e700 1 --2- 172.21.2.101:0/2204189782 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fedcc083980 0x7fedcc083d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:01.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.295 7fedd158e700 1 -- 172.21.2.101:0/2204189782 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc0843a0 con 0x7fedcc0ac2c0 2021-02-10T11:42:01.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.295 7fedd0d8d700 1 --2- 172.21.2.101:0/2204189782 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedcc0bfb60 0x7fedcc1a7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:42:01.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.296 7fedd158e700 1 --2- 172.21.2.101:0/2204189782 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedcc0ac2c0 0x7fedcc1a7480 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 rx=0x7fedc80505c0 tx=0x7fedc8050960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:01.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.296 7fedbdffb700 1 -- 172.21.2.101:0/2204189782 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fedc8050d50 con 0x7fedcc0ac2c0 2021-02-10T11:42:01.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.296 7fedbdffb700 1 -- 172.21.2.101:0/2204189782 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc8000c90 con 0x7fedcc0ac2c0 2021-02-10T11:42:01.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.296 7fedbdffb700 1 -- 172.21.2.101:0/2204189782 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fedc80022b0 con 0x7fedcc0ac2c0 2021-02-10T11:42:01.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.296 7fedd3772700 1 -- 172.21.2.101:0/2204189782 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedcc0843a0 con 0x7fedcc0ac2c0 2021-02-10T11:42:01.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.296 7fedd3772700 1 -- 172.21.2.101:0/2204189782 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedcc0843a0 con 0x7fedcc0ac2c0 2021-02-10T11:42:01.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.297 7fedbdffb700 1 -- 172.21.2.101:0/2204189782 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fedc8006020 con 0x7fedcc0ac2c0 2021-02-10T11:42:01.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.297 7fedbdffb700 1 --2- 172.21.2.101:0/2204189782 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fedac018730 0x7fedac01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:01.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.297 7fedbdffb700 1 -- 172.21.2.101:0/2204189782 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fedc8076790 con 0x7fedcc0ac2c0 2021-02-10T11:42:01.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.297 7fedd0d8d700 1 --2- 172.21.2.101:0/2204189782 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fedac018730 0x7fedac01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:01.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.297 7fedd0d8d700 1 --2- 172.21.2.101:0/2204189782 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fedac018730 0x7fedac01ac10 crc :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:01.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.326 7fedd3772700 1 -- 172.21.2.101:0/2204189782 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedcc082a70 con 0x7fedcc0ac2c0 2021-02-10T11:42:01.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.329 7fedbdffb700 1 -- 172.21.2.101:0/2204189782 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fedc8000c90 con 0x7fedcc0ac2c0 2021-02-10T11:42:01.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.457 7fedd3772700 1 -- 172.21.2.101:0/2204189782 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "target": ["mgr", ""], "group_name": "group_0000000000192983"}) v1 -- 0x7fedcc082a70 con 0x7fedac018730 2021-02-10T11:42:01.464 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:42:01.461 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:42:01.465 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:42:01.461 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:42:01.465 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:42:01.461 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:42:01.465 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:42:01.461 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:42:01.465 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:42:01.461 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:42:01.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.477 7fedbdffb700 1 -- 172.21.2.101:0/2204189782 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fedc0006b10 con 0x7fedac018730 2021-02-10T11:42:01.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.490 7fedd3772700 1 -- 172.21.2.101:0/2204189782 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fedac018730 msgr2=0x7fedac01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:01.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.490 7fedd3772700 1 --2- 172.21.2.101:0/2204189782 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fedac018730 0x7fedac01ac10 crc :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:01.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.490 7fedd3772700 1 -- 172.21.2.101:0/2204189782 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedcc0ac2c0 msgr2=0x7fedcc1a7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:01.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.490 7fedd3772700 1 --2- 172.21.2.101:0/2204189782 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedcc0ac2c0 0x7fedcc1a7480 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 rx=0x7fedc80505c0 tx=0x7fedc8050960).stop 2021-02-10T11:42:01.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.490 7fedd3772700 1 -- 172.21.2.101:0/2204189782 shutdown_connections 2021-02-10T11:42:01.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.490 7fedd3772700 1 --2- 172.21.2.101:0/2204189782 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fedac018730 0x7fedac01ac10 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:01.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.490 7fedd3772700 1 --2- 172.21.2.101:0/2204189782 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedcc0bfb60 0x7fedcc1a7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:01.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.490 7fedd3772700 1 --2- 172.21.2.101:0/2204189782 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fedcc083980 0x7fedcc083d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:01.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.490 7fedd3772700 1 --2- 172.21.2.101:0/2204189782 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedcc0ac2c0 0x7fedcc1a7480 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:01.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.490 7fedd3772700 1 -- 172.21.2.101:0/2204189782 >> 172.21.2.101:0/2204189782 conn(0x7fedcc09ffd0 msgr2=0x7fedcc082ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:01.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.490 7fedd3772700 1 -- 172.21.2.101:0/2204189782 shutdown_connections 2021-02-10T11:42:01.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.490 7fedd3772700 1 -- 172.21.2.101:0/2204189782 wait complete. 2021-02-10T11:42:01.500 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000228711 --group_name group_0000000000192983 2021-02-10T11:42:01.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.917 7f38de17d700 1 Processor -- start 2021-02-10T11:42:01.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.917 7f38de17d700 1 -- start start 2021-02-10T11:42:01.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.918 7f38de17d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38d80d1880 0x7f38d81b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:01.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.918 7f38de17d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f38d80cff80 0x7f38d8109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:01.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.918 7f38de17d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d8109a70 con 0x7f38d81b0af0 2021-02-10T11:42:01.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.918 7f38de17d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d8109760 con 0x7f38d80cff80 2021-02-10T11:42:01.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.918 7f38de17d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d8088070 con 0x7f38d80d1880 2021-02-10T11:42:01.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.918 7f38d6ffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f38d81b0af0 0x7f38d8108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58274/0 (socket says 172.21.2.101:58274) 2021-02-10T11:42:01.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.918 7f38d6ffd700 1 -- 172.21.2.101:0/2657025800 learned_addr learned my addr 172.21.2.101:0/2657025800 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:01.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.918 7f38d7fff700 1 --2- 172.21.2.101:0/2657025800 >> v2:172.21.2.101:3300/0 conn(0x7f38d80cff80 0x7f38d8109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:01.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.918 7f38d77fe700 1 --2- 172.21.2.101:0/2657025800 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38d80d1880 0x7f38d81b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:01.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.918 7f38d67fc700 1 -- 172.21.2.101:0/2657025800 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899012862 0 0) 0x7f38c00009b0 con 0x7f38d81b0af0 2021-02-10T11:42:01.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.918 7f38d67fc700 1 -- 172.21.2.101:0/2657025800 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38c4002fe0 con 0x7f38d81b0af0 2021-02-10T11:42:01.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.918 7f38d7fff700 1 -- 172.21.2.101:0/2657025800 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38d80d1880 msgr2=0x7f38d81b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:01.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.918 7f38d7fff700 1 --2- 172.21.2.101:0/2657025800 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38d80d1880 0x7f38d81b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:01.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.918 7f38d7fff700 1 -- 172.21.2.101:0/2657025800 >> v1:172.21.2.101:6789/0 conn(0x7f38d81b0af0 legacy=0x7f38d8108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:01.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.918 7f38d7fff700 1 -- 172.21.2.101:0/2657025800 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d8088260 con 0x7f38d80cff80 2021-02-10T11:42:01.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.919 7f38d7fff700 1 --2- 172.21.2.101:0/2657025800 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38d80cff80 0x7f38d8109140 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 rx=0x7f38d0006e30 tx=0x7f38d0009e60).ready entity=mon.0 client_cookie=7cdfbc53960d349e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:01.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.919 7f38d67fc700 1 -- 172.21.2.101:0/2657025800 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f38d0006a50 con 0x7f38d80cff80 2021-02-10T11:42:01.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.919 7f38d67fc700 1 -- 172.21.2.101:0/2657025800 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d000b300 con 0x7f38d80cff80 2021-02-10T11:42:01.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.919 7f38d67fc700 1 -- 172.21.2.101:0/2657025800 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f38d000b820 con 0x7f38d80cff80 2021-02-10T11:42:01.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.919 7f38de17d700 1 -- 172.21.2.101:0/2657025800 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38d80cff80 msgr2=0x7f38d8109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:01.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.919 7f38de17d700 1 --2- 172.21.2.101:0/2657025800 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38d80cff80 0x7f38d8109140 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 rx=0x7f38d0006e30 tx=0x7f38d0009e60).stop 2021-02-10T11:42:01.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.920 7f38de17d700 1 -- 172.21.2.101:0/2657025800 shutdown_connections 2021-02-10T11:42:01.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.920 7f38de17d700 1 --2- 172.21.2.101:0/2657025800 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38d80cff80 0x7f38d8109140 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:01.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.920 7f38de17d700 1 --2- 172.21.2.101:0/2657025800 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38d80d1880 0x7f38d81b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:01.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.920 7f38de17d700 1 -- 172.21.2.101:0/2657025800 >> 172.21.2.101:0/2657025800 conn(0x7f38d80ce430 msgr2=0x7f38d80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:01.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.920 7f38de17d700 1 -- 172.21.2.101:0/2657025800 shutdown_connections 2021-02-10T11:42:01.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.920 7f38de17d700 1 -- 172.21.2.101:0/2657025800 wait complete. 2021-02-10T11:42:01.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.934 7f38de17d700 1 Processor -- start 2021-02-10T11:42:01.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.934 7f38de17d700 1 -- start start 2021-02-10T11:42:01.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.935 7f38de17d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38d818bca0 0x7f38d8187e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:01.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.935 7f38de17d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38d818afd0 0x7f38d818b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:01.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.935 7f38de17d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38d80b1950 0x7f38d80b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:01.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.935 7f38de17d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d818b9f0 con 0x7f38d818bca0 2021-02-10T11:42:01.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.935 7f38de17d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d81884a0 con 0x7f38d80b1950 2021-02-10T11:42:01.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.935 7f38de17d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d80b2370 con 0x7f38d818afd0 2021-02-10T11:42:01.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.935 7f38d77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38d818bca0 0x7f38d8187e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:01.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.935 7f38d77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38d818bca0 0x7f38d8187e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57786/0 (socket says 172.21.2.101:57786) 2021-02-10T11:42:01.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.935 7f38d77fe700 1 -- 172.21.2.101:0/1336276109 learned_addr learned my addr 172.21.2.101:0/1336276109 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:01.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.935 7f38d77fe700 1 -- 172.21.2.101:0/1336276109 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38d818afd0 msgr2=0x7f38d818b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:01.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.935 7f38d77fe700 1 --2- 172.21.2.101:0/1336276109 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38d818afd0 0x7f38d818b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:01.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.935 7f38d77fe700 1 -- 172.21.2.101:0/1336276109 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38d80b1950 msgr2=0x7f38d80b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:01.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.935 7f38d77fe700 1 --2- 172.21.2.101:0/1336276109 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38d80b1950 0x7f38d80b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:01.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.935 7f38d77fe700 1 -- 172.21.2.101:0/1336276109 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d80b2590 con 0x7f38d818bca0 2021-02-10T11:42:01.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.936 7f38d77fe700 1 --2- 172.21.2.101:0/1336276109 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38d818bca0 0x7f38d8187e80 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 rx=0x7f38c8006f00 tx=0x7f38c800b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:01.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.936 7f38cffff700 1 -- 172.21.2.101:0/1336276109 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f38c800bd30 con 0x7f38d818bca0 2021-02-10T11:42:01.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.936 7f38cffff700 1 -- 172.21.2.101:0/1336276109 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c8002030 con 0x7f38d818bca0 2021-02-10T11:42:01.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.936 7f38cffff700 1 -- 172.21.2.101:0/1336276109 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f38c8002550 con 0x7f38d818bca0 2021-02-10T11:42:01.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.936 7f38de17d700 1 -- 172.21.2.101:0/1336276109 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d80b2590 con 0x7f38d818bca0 2021-02-10T11:42:01.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.936 7f38de17d700 1 -- 172.21.2.101:0/1336276109 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d80b2590 con 0x7f38d818bca0 2021-02-10T11:42:01.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.936 7f38cffff700 1 -- 172.21.2.101:0/1336276109 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f38c8002210 con 0x7f38d818bca0 2021-02-10T11:42:01.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.936 7f38cffff700 1 --2- 172.21.2.101:0/1336276109 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38bc0186c0 0x7f38bc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:01.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.936 7f38cffff700 1 -- 172.21.2.101:0/1336276109 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f38c80327d0 con 0x7f38d818bca0 2021-02-10T11:42:01.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.936 7f38d6ffd700 1 --2- 172.21.2.101:0/1336276109 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38bc0186c0 0x7f38bc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:01.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.937 7f38d6ffd700 1 --2- 172.21.2.101:0/1336276109 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38bc0186c0 0x7f38bc01abc0 crc :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:01.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.968 7f38de17d700 1 -- 172.21.2.101:0/1336276109 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d818b9f0 con 0x7f38d818bca0 2021-02-10T11:42:01.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:01.970 7f38cffff700 1 -- 172.21.2.101:0/1336276109 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f38c8002240 con 0x7f38d818bca0 2021-02-10T11:42:02.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.097 7f38de17d700 1 -- 172.21.2.101:0/1336276109 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000228711", "target": ["mgr", ""], "group_name": "group_0000000000192983"}) v1 -- 0x7f38d818b9f0 con 0x7f38bc0186c0 2021-02-10T11:42:02.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.122 7f38cffff700 1 -- 172.21.2.101:0/1336276109 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f38c0006e30 con 0x7f38bc0186c0 2021-02-10T11:42:02.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.130 7f38de17d700 1 -- 172.21.2.101:0/1336276109 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38bc0186c0 msgr2=0x7f38bc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:02.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.130 7f38de17d700 1 --2- 172.21.2.101:0/1336276109 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38bc0186c0 0x7f38bc01abc0 crc :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:02.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.130 7f38de17d700 1 -- 172.21.2.101:0/1336276109 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38d818bca0 msgr2=0x7f38d8187e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:02.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.130 7f38de17d700 1 --2- 172.21.2.101:0/1336276109 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38d818bca0 0x7f38d8187e80 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 rx=0x7f38c8006f00 tx=0x7f38c800b940).stop 2021-02-10T11:42:02.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.130 7f38de17d700 1 -- 172.21.2.101:0/1336276109 shutdown_connections 2021-02-10T11:42:02.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.130 7f38de17d700 1 --2- 172.21.2.101:0/1336276109 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38bc0186c0 0x7f38bc01abc0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:02.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.130 7f38de17d700 1 --2- 172.21.2.101:0/1336276109 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38d818afd0 0x7f38d818b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:02.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.130 7f38de17d700 1 --2- 172.21.2.101:0/1336276109 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38d80b1950 0x7f38d80b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:02.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.130 7f38de17d700 1 --2- 172.21.2.101:0/1336276109 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38d818bca0 0x7f38d8187e80 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:02.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.130 7f38de17d700 1 -- 172.21.2.101:0/1336276109 >> 172.21.2.101:0/1336276109 conn(0x7f38d818e870 msgr2=0x7f38d818ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:02.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.130 7f38de17d700 1 -- 172.21.2.101:0/1336276109 shutdown_connections 2021-02-10T11:42:02.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.130 7f38de17d700 1 -- 172.21.2.101:0/1336276109 wait complete. 2021-02-10T11:42:02.140 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.1 && sudo ls volumes 2021-02-10T11:42:02.491 INFO:teuthology.orchestra.run.gibba001.stdout:group_0000000000192983 2021-02-10T11:42:02.491 INFO:teuthology.orchestra.run.gibba001.stdout:_group_0000000000192983:subvolume_0000000000228711.meta 2021-02-10T11:42:02.492 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume authorize cephfs subvolume_0000000000228711 alice --group_name group_0000000000192983 --tenant_id tenant1 2021-02-10T11:42:02.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.914 7f3c239cf700 1 Processor -- start 2021-02-10T11:42:02.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.914 7f3c239cf700 1 -- start start 2021-02-10T11:42:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.914 7f3c239cf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c1c0e53a0 0x7f3c1c1b0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.914 7f3c239cf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3c1c1b0b70 0x7f3c1c1089d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.914 7f3c239cf700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3c1c108ff0 0x7f3c1c1093f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.914 7f3c239cf700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c1c0fe2b0 con 0x7f3c1c1b0b70 2021-02-10T11:42:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.914 7f3c239cf700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c1c0e5f80 con 0x7f3c1c108ff0 2021-02-10T11:42:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.914 7f3c239cf700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c109ac0 con 0x7f3c1c0e53a0 2021-02-10T11:42:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.914 7f3c20fea700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3c1c1b0b70 0x7f3c1c1089d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.914 7f3c20fea700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3c1c1b0b70 0x7f3c1c1089d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57794/0 (socket says 172.21.2.101:57794) 2021-02-10T11:42:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.914 7f3c20fea700 1 -- 172.21.2.101:0/4241002679 learned_addr learned my addr 172.21.2.101:0/4241002679 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.915 7f3c20fea700 1 -- 172.21.2.101:0/4241002679 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c1c0e53a0 msgr2=0x7f3c1c1b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.915 7f3c20fea700 1 --2- 172.21.2.101:0/4241002679 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c1c0e53a0 0x7f3c1c1b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.915 7f3c20fea700 1 -- 172.21.2.101:0/4241002679 >> v2:172.21.2.108:3300/0 conn(0x7f3c1c108ff0 msgr2=0x7f3c1c1093f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.915 7f3c20fea700 1 --2- 172.21.2.101:0/4241002679 >> v2:172.21.2.108:3300/0 conn(0x7f3c1c108ff0 0x7f3c1c1093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:02.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.915 7f3c20fea700 1 -- 172.21.2.101:0/4241002679 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c1c0e57a0 con 0x7f3c1c1b0b70 2021-02-10T11:42:02.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.915 7f3c20fea700 1 --2- 172.21.2.101:0/4241002679 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c1c1b0b70 0x7f3c1c1089d0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 rx=0x7f3c18006700 tx=0x7f3c18009b80).ready entity=mon.0 client_cookie=ec4f1867c85dc8f2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:02.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.915 7f3c0ffff700 1 -- 172.21.2.101:0/4241002679 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3c1800c0c0 con 0x7f3c1c1b0b70 2021-02-10T11:42:02.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.915 7f3c0ffff700 1 -- 172.21.2.101:0/4241002679 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1800c340 con 0x7f3c1c1b0b70 2021-02-10T11:42:02.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.915 7f3c0ffff700 1 -- 172.21.2.101:0/4241002679 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3c1800c820 con 0x7f3c1c1b0b70 2021-02-10T11:42:02.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.915 7f3c239cf700 1 -- 172.21.2.101:0/4241002679 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c1c1b0b70 msgr2=0x7f3c1c1089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:02.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.915 7f3c239cf700 1 --2- 172.21.2.101:0/4241002679 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c1c1b0b70 0x7f3c1c1089d0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 rx=0x7f3c18006700 tx=0x7f3c18009b80).stop 2021-02-10T11:42:02.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.916 7f3c239cf700 1 -- 172.21.2.101:0/4241002679 shutdown_connections 2021-02-10T11:42:02.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.916 7f3c239cf700 1 --2- 172.21.2.101:0/4241002679 >> v2:172.21.2.108:3300/0 conn(0x7f3c1c108ff0 0x7f3c1c1093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:02.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.916 7f3c239cf700 1 --2- 172.21.2.101:0/4241002679 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c1c1b0b70 0x7f3c1c1089d0 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:02.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.916 7f3c239cf700 1 --2- 172.21.2.101:0/4241002679 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c1c0e53a0 0x7f3c1c1b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:02.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.916 7f3c239cf700 1 -- 172.21.2.101:0/4241002679 >> 172.21.2.101:0/4241002679 conn(0x7f3c1c0ce4b0 msgr2=0x7f3c1c0e45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:02.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.916 7f3c239cf700 1 -- 172.21.2.101:0/4241002679 shutdown_connections 2021-02-10T11:42:02.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.916 7f3c239cf700 1 -- 172.21.2.101:0/4241002679 wait complete. 2021-02-10T11:42:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.931 7f3c239cf700 1 Processor -- start 2021-02-10T11:42:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.931 7f3c239cf700 1 -- start start 2021-02-10T11:42:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.931 7f3c239cf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c1c1a5960 0x7f3c1c1a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.931 7f3c239cf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c1c1a22f0 0x7f3c1c19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.931 7f3c239cf700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c1c19c350 0x7f3c1c19c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.931 7f3c239cf700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c1a26f0 con 0x7f3c1c1a22f0 2021-02-10T11:42:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.931 7f3c239cf700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c1a5d60 con 0x7f3c1c19c350 2021-02-10T11:42:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.931 7f3c239cf700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c08a510 con 0x7f3c1c1a5960 2021-02-10T11:42:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.931 7f3c20fea700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c1c1a22f0 0x7f3c1c19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.931 7f3c20fea700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c1c1a22f0 0x7f3c1c19bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57796/0 (socket says 172.21.2.101:57796) 2021-02-10T11:42:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.931 7f3c20fea700 1 -- 172.21.2.101:0/2924259073 learned_addr learned my addr 172.21.2.101:0/2924259073 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.932 7f3c20fea700 1 -- 172.21.2.101:0/2924259073 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c1c1a5960 msgr2=0x7f3c1c1a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.932 7f3c20fea700 1 --2- 172.21.2.101:0/2924259073 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c1c1a5960 0x7f3c1c1a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.932 7f3c20fea700 1 -- 172.21.2.101:0/2924259073 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c1c19c350 msgr2=0x7f3c1c19c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.932 7f3c20fea700 1 --2- 172.21.2.101:0/2924259073 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c1c19c350 0x7f3c1c19c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.932 7f3c20fea700 1 -- 172.21.2.101:0/2924259073 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c1c19cd70 con 0x7f3c1c1a22f0 2021-02-10T11:42:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.932 7f3c20fea700 1 --2- 172.21.2.101:0/2924259073 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c1c1a22f0 0x7f3c1c19bd30 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 rx=0x7f3c1800b650 tx=0x7f3c1800b9f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.932 7f3c0dffb700 1 -- 172.21.2.101:0/2924259073 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3c1800bde0 con 0x7f3c1c1a22f0 2021-02-10T11:42:02.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.932 7f3c0dffb700 1 -- 172.21.2.101:0/2924259073 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c18002030 con 0x7f3c1c1a22f0 2021-02-10T11:42:02.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.932 7f3c239cf700 1 -- 172.21.2.101:0/2924259073 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c1c19cd70 con 0x7f3c1c1a22f0 2021-02-10T11:42:02.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.932 7f3c239cf700 1 -- 172.21.2.101:0/2924259073 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c1c0b1a90 con 0x7f3c1c1a22f0 2021-02-10T11:42:02.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.932 7f3c0dffb700 1 -- 172.21.2.101:0/2924259073 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3c18002510 con 0x7f3c1c1a22f0 2021-02-10T11:42:02.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.933 7f3c0dffb700 1 -- 172.21.2.101:0/2924259073 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3c18000c90 con 0x7f3c1c1a22f0 2021-02-10T11:42:02.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.933 7f3c0dffb700 1 --2- 172.21.2.101:0/2924259073 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3bfc0186c0 0x7f3bfc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:02.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.933 7f3c0dffb700 1 -- 172.21.2.101:0/2924259073 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3c180325f0 con 0x7f3c1c1a22f0 2021-02-10T11:42:02.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.933 7f3c217eb700 1 --2- 172.21.2.101:0/2924259073 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3bfc0186c0 0x7f3bfc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:02.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.933 7f3c217eb700 1 --2- 172.21.2.101:0/2924259073 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3bfc0186c0 0x7f3bfc01abc0 crc :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:02.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.964 7f3c239cf700 1 -- 172.21.2.101:0/2924259073 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf4002fd0 con 0x7f3c1c1a22f0 2021-02-10T11:42:02.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:02.967 7f3c0dffb700 1 -- 172.21.2.101:0/2924259073 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3c18030da0 con 0x7f3c1c1a22f0 2021-02-10T11:42:03.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:03.092 7f3c239cf700 1 -- 172.21.2.101:0/2924259073 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000228711", "target": ["mgr", ""], "tenant_id": "tenant1", "auth_id": "alice", "group_name": "group_0000000000192983"}) v1 -- 0x7f3bf4002f80 con 0x7f3bfc0186c0 2021-02-10T11:42:03.099 INFO:tasks.ceph.mgr.y.gibba001.stderr:Creating meta for ID alice with tenant tenant1 2021-02-10T11:42:03.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:03.126 7f3c0dffb700 1 -- 172.21.2.101:0/2924259073 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7f3c10006e40 con 0x7f3bfc0186c0 2021-02-10T11:42:03.157 INFO:teuthology.orchestra.run.gibba001.stdout:AQCLxiNgslqJBhAAUx5yPjoT/bCTAlTp28xnoA== 2021-02-10T11:42:03.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:03.157 7f3c239cf700 1 -- 172.21.2.101:0/2924259073 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3bfc0186c0 msgr2=0x7f3bfc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:03.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:03.157 7f3c239cf700 1 --2- 172.21.2.101:0/2924259073 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3bfc0186c0 0x7f3bfc01abc0 crc :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:03.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:03.157 7f3c239cf700 1 -- 172.21.2.101:0/2924259073 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c1c1a22f0 msgr2=0x7f3c1c19bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:03.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:03.157 7f3c239cf700 1 --2- 172.21.2.101:0/2924259073 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c1c1a22f0 0x7f3c1c19bd30 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 rx=0x7f3c1800b650 tx=0x7f3c1800b9f0).stop 2021-02-10T11:42:03.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:03.157 7f3c239cf700 1 -- 172.21.2.101:0/2924259073 shutdown_connections 2021-02-10T11:42:03.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:03.157 7f3c239cf700 1 --2- 172.21.2.101:0/2924259073 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3c1c19c350 0x7f3c1c19c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:03.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:03.157 7f3c239cf700 1 --2- 172.21.2.101:0/2924259073 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3c1c1a22f0 0x7f3c1c19bd30 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:03.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:03.157 7f3c239cf700 1 --2- 172.21.2.101:0/2924259073 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3bfc0186c0 0x7f3bfc01abc0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:03.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:03.157 7f3c239cf700 1 --2- 172.21.2.101:0/2924259073 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3c1c1a5960 0x7f3c1c1a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:03.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:03.157 7f3c239cf700 1 -- 172.21.2.101:0/2924259073 >> 172.21.2.101:0/2924259073 conn(0x7f3c1c191430 msgr2=0x7f3c1c19ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:03.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:03.157 7f3c239cf700 1 -- 172.21.2.101:0/2924259073 shutdown_connections 2021-02-10T11:42:03.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:03.157 7f3c239cf700 1 -- 172.21.2.101:0/2924259073 wait complete. 2021-02-10T11:42:03.168 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.1 && sudo ls volumes 2021-02-10T11:42:03.516 INFO:teuthology.orchestra.run.gibba001.stdout:$alice.meta 2021-02-10T11:42:03.516 INFO:teuthology.orchestra.run.gibba001.stdout:group_0000000000192983 2021-02-10T11:42:03.516 INFO:teuthology.orchestra.run.gibba001.stdout:_group_0000000000192983:subvolume_0000000000228711.meta 2021-02-10T11:42:03.517 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.1 && sudo sudo cat '/home/ubuntu/cephtest/mnt.1/volumes/$alice.meta' 2021-02-10T11:42:03.880 INFO:teuthology.orchestra.run.gibba001.stdout:{"compat_version": 1, "tenant_id": "tenant1", "version": 5, "dirty": false, "subvolumes": {"group_0000000000192983/subvolume_0000000000228711": {"access_level": "rw", "dirty": false}}} 2021-02-10T11:42:03.881 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.1 && sudo sudo cat /home/ubuntu/cephtest/mnt.1/volumes/_group_0000000000192983:subvolume_0000000000228711.meta 2021-02-10T11:42:04.243 INFO:teuthology.orchestra.run.gibba001.stdout:{"auths": {"alice": {"access_level": "rw", "dirty": false}}, "version": 5, "compat_version": 1} 2021-02-10T11:42:04.243 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume authorize cephfs subvolume_0000000000228711 alice --group_name group_0000000000192983 --tenant_id tenant2 2021-02-10T11:42:04.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.667 7fbf052d0700 1 Processor -- start 2021-02-10T11:42:04.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.667 7fbf052d0700 1 -- start start 2021-02-10T11:42:04.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.668 7fbf052d0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf000e53a0 0x7fbf001b0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:04.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.668 7fbf052d0700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbf001b0b70 0x7fbf001089d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:04.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.668 7fbf052d0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf00109ac0 con 0x7fbf00108ff0 2021-02-10T11:42:04.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.668 7fbf052d0700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf000e5f80 con 0x7fbf001b0b70 2021-02-10T11:42:04.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.668 7fbf052d0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000e57a0 con 0x7fbf000e53a0 2021-02-10T11:42:04.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.669 7fbeff07d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf000e53a0 0x7fbf001b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:04.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.669 7fbefe87c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbf001b0b70 0x7fbf001089d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:04.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.669 7fbeff07d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf000e53a0 0x7fbf001b0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:50542/0 (socket says 172.21.2.101:50542) 2021-02-10T11:42:04.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.669 7fbefe87c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbf001b0b70 0x7fbf001089d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:40064/0 (socket says 172.21.2.101:40064) 2021-02-10T11:42:04.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.669 7fbeff07d700 1 -- 172.21.2.101:0/2101058432 learned_addr learned my addr 172.21.2.101:0/2101058432 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:04.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.669 7fbeff07d700 1 -- 172.21.2.101:0/2101058432 >> v2:172.21.2.108:3300/0 conn(0x7fbf001b0b70 msgr2=0x7fbf001089d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:04.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.669 7fbeff07d700 1 --2- 172.21.2.101:0/2101058432 >> v2:172.21.2.108:3300/0 conn(0x7fbf001b0b70 0x7fbf001089d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:04.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.669 7fbeff07d700 1 -- 172.21.2.101:0/2101058432 >> v1:172.21.2.108:6789/0 conn(0x7fbf00108ff0 legacy=0x7fbf0010a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:04.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.669 7fbeff07d700 1 -- 172.21.2.101:0/2101058432 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf00088070 con 0x7fbf000e53a0 2021-02-10T11:42:04.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.669 7fbefe87c700 1 --2- 172.21.2.101:0/2101058432 >> v2:172.21.2.108:3300/0 conn(0x7fbf001b0b70 0x7fbf001089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:42:04.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.669 7fbeff07d700 1 --2- 172.21.2.101:0/2101058432 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf000e53a0 0x7fbf001b0550 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fbee8006700 tx=0x7fbee8009b80).ready entity=mon.2 client_cookie=792ddfc880c80e00 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:04.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.670 7fbefe07b700 1 -- 172.21.2.101:0/2101058432 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbee800c0c0 con 0x7fbf000e53a0 2021-02-10T11:42:04.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.670 7fbefe07b700 1 -- 172.21.2.101:0/2101058432 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee800c340 con 0x7fbf000e53a0 2021-02-10T11:42:04.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.670 7fbefe07b700 1 -- 172.21.2.101:0/2101058432 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbee800c860 con 0x7fbf000e53a0 2021-02-10T11:42:04.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.670 7fbf052d0700 1 -- 172.21.2.101:0/2101058432 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf000e53a0 msgr2=0x7fbf001b0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:04.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.670 7fbf052d0700 1 --2- 172.21.2.101:0/2101058432 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf000e53a0 0x7fbf001b0550 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fbee8006700 tx=0x7fbee8009b80).stop 2021-02-10T11:42:04.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.671 7fbf052d0700 1 -- 172.21.2.101:0/2101058432 shutdown_connections 2021-02-10T11:42:04.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.671 7fbf052d0700 1 --2- 172.21.2.101:0/2101058432 >> v2:172.21.2.108:3300/0 conn(0x7fbf001b0b70 0x7fbf001089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:04.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.671 7fbf052d0700 1 --2- 172.21.2.101:0/2101058432 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf000e53a0 0x7fbf001b0550 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:04.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.671 7fbf052d0700 1 -- 172.21.2.101:0/2101058432 >> 172.21.2.101:0/2101058432 conn(0x7fbf000ce4b0 msgr2=0x7fbf000e45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:04.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.671 7fbf052d0700 1 -- 172.21.2.101:0/2101058432 shutdown_connections 2021-02-10T11:42:04.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.671 7fbf052d0700 1 -- 172.21.2.101:0/2101058432 wait complete. 2021-02-10T11:42:04.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.685 7fbf052d0700 1 Processor -- start 2021-02-10T11:42:04.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.685 7fbf052d0700 1 -- start start 2021-02-10T11:42:04.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.686 7fbf052d0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf000aaa90 0x7fbf00095ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:04.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.686 7fbf052d0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbf000962c0 0x7fbf000966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:04.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.686 7fbf052d0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf00096ce0 0x7fbf000908d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:04.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.686 7fbf052d0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0019eb80 con 0x7fbf00096ce0 2021-02-10T11:42:04.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.686 7fbf052d0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf00090ef0 con 0x7fbf000962c0 2021-02-10T11:42:04.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.686 7fbf052d0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf00091110 con 0x7fbf000aaa90 2021-02-10T11:42:04.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.686 7fbeff87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf00096ce0 0x7fbf000908d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:04.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.686 7fbeff87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf00096ce0 0x7fbf000908d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57818/0 (socket says 172.21.2.101:57818) 2021-02-10T11:42:04.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.686 7fbeff87e700 1 -- 172.21.2.101:0/1993550684 learned_addr learned my addr 172.21.2.101:0/1993550684 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:04.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.686 7fbeff87e700 1 -- 172.21.2.101:0/1993550684 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf000aaa90 msgr2=0x7fbf00095ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:04.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.686 7fbeff87e700 1 --2- 172.21.2.101:0/1993550684 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf000aaa90 0x7fbf00095ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:04.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.686 7fbeff87e700 1 -- 172.21.2.101:0/1993550684 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbf000962c0 msgr2=0x7fbf000966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:04.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.686 7fbeff87e700 1 --2- 172.21.2.101:0/1993550684 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbf000962c0 0x7fbf000966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:04.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.686 7fbeff87e700 1 -- 172.21.2.101:0/1993550684 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf00091330 con 0x7fbf00096ce0 2021-02-10T11:42:04.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.687 7fbeff87e700 1 --2- 172.21.2.101:0/1993550684 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf00096ce0 0x7fbf000908d0 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 rx=0x7fbef8009940 tx=0x7fbef800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:04.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.687 7fbef7fff700 1 -- 172.21.2.101:0/1993550684 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbef8000ce0 con 0x7fbf00096ce0 2021-02-10T11:42:04.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.687 7fbef7fff700 1 -- 172.21.2.101:0/1993550684 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef800c2f0 con 0x7fbf00096ce0 2021-02-10T11:42:04.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.687 7fbf052d0700 1 -- 172.21.2.101:0/1993550684 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf00091330 con 0x7fbf00096ce0 2021-02-10T11:42:04.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.687 7fbf052d0700 1 -- 172.21.2.101:0/1993550684 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf000918c0 con 0x7fbf00096ce0 2021-02-10T11:42:04.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.687 7fbef7fff700 1 -- 172.21.2.101:0/1993550684 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbef800c810 con 0x7fbf00096ce0 2021-02-10T11:42:04.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.688 7fbef7fff700 1 -- 172.21.2.101:0/1993550684 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbef8001220 con 0x7fbf00096ce0 2021-02-10T11:42:04.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.688 7fbef7fff700 1 --2- 172.21.2.101:0/1993550684 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbee40186c0 0x7fbee401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:04.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.688 7fbef7fff700 1 -- 172.21.2.101:0/1993550684 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fbef8032950 con 0x7fbf00096ce0 2021-02-10T11:42:04.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.688 7fbeff07d700 1 --2- 172.21.2.101:0/1993550684 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbee40186c0 0x7fbee401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:04.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.688 7fbeff07d700 1 --2- 172.21.2.101:0/1993550684 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbee40186c0 0x7fbee401abc0 crc :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:04.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.718 7fbf052d0700 1 -- 172.21.2.101:0/1993550684 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf00103750 con 0x7fbf00096ce0 2021-02-10T11:42:04.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.721 7fbef7fff700 1 -- 172.21.2.101:0/1993550684 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbef8002ef0 con 0x7fbf00096ce0 2021-02-10T11:42:04.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.846 7fbf052d0700 1 -- 172.21.2.101:0/1993550684 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000228711", "target": ["mgr", ""], "tenant_id": "tenant2", "auth_id": "alice", "group_name": "group_0000000000192983"}) v1 -- 0x7fbf0019eb80 con 0x7fbee40186c0 2021-02-10T11:42:04.854 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:42:04.852 7f2cc39e1700 -1 mgr.server reply reply (1) Operation not permitted auth ID: alice is already in use 2021-02-10T11:42:04.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.852 7fbef7fff700 1 -- 172.21.2.101:0/1993550684 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: -1 auth ID: alice is already in use) v1 ==== 40+0+0 (crc 0 0 0) 0x7fbee8006e10 con 0x7fbee40186c0 2021-02-10T11:42:04.855 INFO:teuthology.orchestra.run.gibba001.stderr:Error EPERM: auth ID: alice is already in use 2021-02-10T11:42:04.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.855 7fbf052d0700 1 -- 172.21.2.101:0/1993550684 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbee40186c0 msgr2=0x7fbee401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:04.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.855 7fbf052d0700 1 --2- 172.21.2.101:0/1993550684 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbee40186c0 0x7fbee401abc0 crc :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:04.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.855 7fbf052d0700 1 -- 172.21.2.101:0/1993550684 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf00096ce0 msgr2=0x7fbf000908d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:04.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.855 7fbf052d0700 1 --2- 172.21.2.101:0/1993550684 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf00096ce0 0x7fbf000908d0 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 rx=0x7fbef8009940 tx=0x7fbef800bcf0).stop 2021-02-10T11:42:04.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.855 7fbf052d0700 1 -- 172.21.2.101:0/1993550684 shutdown_connections 2021-02-10T11:42:04.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.855 7fbf052d0700 1 --2- 172.21.2.101:0/1993550684 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf00096ce0 0x7fbf000908d0 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:04.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.855 7fbf052d0700 1 --2- 172.21.2.101:0/1993550684 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbf000962c0 0x7fbf000966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:04.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.855 7fbf052d0700 1 --2- 172.21.2.101:0/1993550684 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbee40186c0 0x7fbee401abc0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:04.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.855 7fbf052d0700 1 --2- 172.21.2.101:0/1993550684 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf000aaa90 0x7fbf00095ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:04.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.855 7fbf052d0700 1 -- 172.21.2.101:0/1993550684 >> 172.21.2.101:0/1993550684 conn(0x7fbf0019b000 msgr2=0x7fbf0018c310 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:04.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.855 7fbf052d0700 1 -- 172.21.2.101:0/1993550684 shutdown_connections 2021-02-10T11:42:04.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:04.855 7fbf052d0700 1 -- 172.21.2.101:0/1993550684 wait complete. 2021-02-10T11:42:04.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:42:04.865 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume deauthorize cephfs subvolume_0000000000228711 alice --group_name group_0000000000192983 2021-02-10T11:42:05.077 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:42:05.080 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:42:05.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.280 7f37a0e18700 1 Processor -- start 2021-02-10T11:42:05.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.280 7f37a0e18700 1 -- start start 2021-02-10T11:42:05.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.280 7f37a0e18700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f379c0eb220 0x7f379c0eb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:05.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.281 7f37a0e18700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f379c10a550 con 0x7f379c10aad0 2021-02-10T11:42:05.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.281 7f37a0e18700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f379c0881d0 con 0x7f379c0dce80 2021-02-10T11:42:05.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.281 7f37a0e18700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379c0d1f30 con 0x7f379c0eb220 2021-02-10T11:42:05.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.281 7f379b314700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f379c10aad0 0x7f379c10aed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58320/0 (socket says 172.21.2.101:58320) 2021-02-10T11:42:05.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.281 7f379ab13700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f379c0dce80 0x7f379c0d14f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:51726/0 (socket says 172.21.2.101:51726) 2021-02-10T11:42:05.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.281 7f379b314700 1 -- 172.21.2.101:0/1162346656 learned_addr learned my addr 172.21.2.101:0/1162346656 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:05.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.281 7f379a312700 1 --2- 172.21.2.101:0/1162346656 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f379c0eb220 0x7f379c0eb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:05.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.281 7f3799b11700 1 -- 172.21.2.101:0/1162346656 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3656410974 0 0) 0x7f3794000f90 con 0x7f379c10aad0 2021-02-10T11:42:05.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.281 7f379a312700 1 -- 172.21.2.101:0/1162346656 >> v1:172.21.2.108:6789/0 conn(0x7f379c0dce80 legacy=0x7f379c0d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:05.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.281 7f379a312700 1 -- 172.21.2.101:0/1162346656 >> v1:172.21.2.101:6789/0 conn(0x7f379c10aad0 legacy=0x7f379c10aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:05.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.281 7f379a312700 1 -- 172.21.2.101:0/1162346656 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c0dd280 con 0x7f379c0eb220 2021-02-10T11:42:05.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.282 7f379a312700 1 --2- 172.21.2.101:0/1162346656 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f379c0eb220 0x7f379c0eb620 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f378c006650 tx=0x7f378c009a00).ready entity=mon.2 client_cookie=8eb7fead6884c497 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:05.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.282 7f3799b11700 1 -- 172.21.2.101:0/1162346656 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f378c009df0 con 0x7f379c0eb220 2021-02-10T11:42:05.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.282 7f3799b11700 1 -- 172.21.2.101:0/1162346656 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378c0009b0 con 0x7f379c0eb220 2021-02-10T11:42:05.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.282 7f3799b11700 1 -- 172.21.2.101:0/1162346656 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f378c00c2b0 con 0x7f379c0eb220 2021-02-10T11:42:05.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.282 7f37a0e18700 1 -- 172.21.2.101:0/1162346656 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f379c0eb220 msgr2=0x7f379c0eb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:05.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.282 7f37a0e18700 1 --2- 172.21.2.101:0/1162346656 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f379c0eb220 0x7f379c0eb620 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f378c006650 tx=0x7f378c009a00).stop 2021-02-10T11:42:05.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.282 7f37a0e18700 1 -- 172.21.2.101:0/1162346656 shutdown_connections 2021-02-10T11:42:05.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.282 7f37a0e18700 1 --2- 172.21.2.101:0/1162346656 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f379c0eb220 0x7f379c0eb620 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:05.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.282 7f37a0e18700 1 -- 172.21.2.101:0/1162346656 >> 172.21.2.101:0/1162346656 conn(0x7f379c0ea630 msgr2=0x7f379c0d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:05.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.282 7f37a0e18700 1 -- 172.21.2.101:0/1162346656 shutdown_connections 2021-02-10T11:42:05.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.282 7f37a0e18700 1 -- 172.21.2.101:0/1162346656 wait complete. 2021-02-10T11:42:05.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.297 7f37a0e18700 1 Processor -- start 2021-02-10T11:42:05.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.297 7f37a0e18700 1 -- start start 2021-02-10T11:42:05.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.297 7f37a0e18700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f379c0ac9c0 0x7f379c104f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:05.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.297 7f37a0e18700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f379c092490 0x7f379c092890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:05.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.297 7f37a0e18700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f379c092eb0 0x7f379c0932b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:05.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.297 7f37a0e18700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c091f90 con 0x7f379c092490 2021-02-10T11:42:05.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.297 7f37a0e18700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c105560 con 0x7f379c0ac9c0 2021-02-10T11:42:05.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.297 7f37a0e18700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379c090900 con 0x7f379c092eb0 2021-02-10T11:42:05.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.298 7f379a312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f379c092490 0x7f379c092890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:05.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.298 7f379a312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f379c092490 0x7f379c092890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57830/0 (socket says 172.21.2.101:57830) 2021-02-10T11:42:05.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.298 7f379a312700 1 -- 172.21.2.101:0/3953269212 learned_addr learned my addr 172.21.2.101:0/3953269212 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:05.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.298 7f379b314700 1 --2- 172.21.2.101:0/3953269212 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f379c092eb0 0x7f379c0932b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:05.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.298 7f379ab13700 1 --2- 172.21.2.101:0/3953269212 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f379c0ac9c0 0x7f379c104f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:05.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.298 7f379a312700 1 -- 172.21.2.101:0/3953269212 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f379c092eb0 msgr2=0x7f379c0932b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:05.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.298 7f379a312700 1 --2- 172.21.2.101:0/3953269212 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f379c092eb0 0x7f379c0932b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:05.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.298 7f379a312700 1 -- 172.21.2.101:0/3953269212 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f379c0ac9c0 msgr2=0x7f379c104f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:05.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.298 7f379a312700 1 --2- 172.21.2.101:0/3953269212 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f379c0ac9c0 0x7f379c104f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:05.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.298 7f379a312700 1 -- 172.21.2.101:0/3953269212 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c090b20 con 0x7f379c092490 2021-02-10T11:42:05.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.298 7f379ab13700 1 --2- 172.21.2.101:0/3953269212 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f379c0ac9c0 0x7f379c104f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:42:05.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.298 7f379a312700 1 --2- 172.21.2.101:0/3953269212 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f379c092490 0x7f379c092890 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 rx=0x7f378c0096a0 tx=0x7f378c00b9b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:05.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.299 7f37937fe700 1 -- 172.21.2.101:0/3953269212 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f378c0093a0 con 0x7f379c092490 2021-02-10T11:42:05.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.299 7f37937fe700 1 -- 172.21.2.101:0/3953269212 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378c0067f0 con 0x7f379c092490 2021-02-10T11:42:05.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.299 7f37937fe700 1 -- 172.21.2.101:0/3953269212 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f378c002220 con 0x7f379c092490 2021-02-10T11:42:05.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.299 7f37a0e18700 1 -- 172.21.2.101:0/3953269212 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379c090b20 con 0x7f379c092490 2021-02-10T11:42:05.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.299 7f37a0e18700 1 -- 172.21.2.101:0/3953269212 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379c090b20 con 0x7f379c092490 2021-02-10T11:42:05.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.299 7f37937fe700 1 -- 172.21.2.101:0/3953269212 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f378c0052e0 con 0x7f379c092490 2021-02-10T11:42:05.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.299 7f37937fe700 1 --2- 172.21.2.101:0/3953269212 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37800186c0 0x7f378001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:05.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.299 7f37937fe700 1 -- 172.21.2.101:0/3953269212 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f378c032570 con 0x7f379c092490 2021-02-10T11:42:05.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.299 7f379ab13700 1 --2- 172.21.2.101:0/3953269212 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37800186c0 0x7f378001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:05.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.300 7f379ab13700 1 --2- 172.21.2.101:0/3953269212 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37800186c0 0x7f378001abc0 crc :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:05.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.331 7f37a0e18700 1 -- 172.21.2.101:0/3953269212 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f379c10a550 con 0x7f379c092490 2021-02-10T11:42:05.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.333 7f37937fe700 1 -- 172.21.2.101:0/3953269212 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f378c02ebd0 con 0x7f379c092490 2021-02-10T11:42:05.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.463 7f37a0e18700 1 -- 172.21.2.101:0/3953269212 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume deauthorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000228711", "target": ["mgr", ""], "auth_id": "alice", "group_name": "group_0000000000192983"}) v1 -- 0x7f379c10a550 con 0x7f37800186c0 2021-02-10T11:42:05.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.500 7f37937fe700 1 -- 172.21.2.101:0/3953269212 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f37840079c0 con 0x7f37800186c0 2021-02-10T11:42:05.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.528 7f37a0e18700 1 -- 172.21.2.101:0/3953269212 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37800186c0 msgr2=0x7f378001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:05.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.528 7f37a0e18700 1 --2- 172.21.2.101:0/3953269212 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37800186c0 0x7f378001abc0 crc :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:05.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.528 7f37a0e18700 1 -- 172.21.2.101:0/3953269212 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f379c092490 msgr2=0x7f379c092890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:05.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.528 7f37a0e18700 1 --2- 172.21.2.101:0/3953269212 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f379c092490 0x7f379c092890 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 rx=0x7f378c0096a0 tx=0x7f378c00b9b0).stop 2021-02-10T11:42:05.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.528 7f37a0e18700 1 -- 172.21.2.101:0/3953269212 shutdown_connections 2021-02-10T11:42:05.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.528 7f37a0e18700 1 --2- 172.21.2.101:0/3953269212 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37800186c0 0x7f378001abc0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:05.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.528 7f37a0e18700 1 --2- 172.21.2.101:0/3953269212 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f379c092eb0 0x7f379c0932b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:05.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.528 7f37a0e18700 1 --2- 172.21.2.101:0/3953269212 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f379c092490 0x7f379c092890 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:05.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.528 7f37a0e18700 1 --2- 172.21.2.101:0/3953269212 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f379c0ac9c0 0x7f379c104f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:05.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.528 7f37a0e18700 1 -- 172.21.2.101:0/3953269212 >> 172.21.2.101:0/3953269212 conn(0x7f379c1a8260 msgr2=0x7f379c0fac60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:05.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.528 7f37a0e18700 1 -- 172.21.2.101:0/3953269212 shutdown_connections 2021-02-10T11:42:05.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:05.528 7f37a0e18700 1 -- 172.21.2.101:0/3953269212 wait complete. 2021-02-10T11:42:05.539 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.1 && sudo ls volumes 2021-02-10T11:42:05.927 INFO:teuthology.orchestra.run.gibba001.stdout:group_0000000000192983 2021-02-10T11:42:05.927 INFO:teuthology.orchestra.run.gibba001.stdout:_group_0000000000192983:subvolume_0000000000228711.meta 2021-02-10T11:42:05.928 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000228711 --group_name group_0000000000192983 2021-02-10T11:42:06.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.340 7fea837ad700 1 Processor -- start 2021-02-10T11:42:06.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.340 7fea837ad700 1 -- start start 2021-02-10T11:42:06.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.341 7fea837ad700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fea7c0d1880 0x7fea7c1b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:06.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.341 7fea837ad700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea7c0fe1d0 con 0x7fea7c0cff80 2021-02-10T11:42:06.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.341 7fea837ad700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea7c109a70 con 0x7fea7c1b0af0 2021-02-10T11:42:06.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.341 7fea837ad700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea7c088070 con 0x7fea7c0d1880 2021-02-10T11:42:06.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.341 7fea81dca700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fea7c0cff80 0x7fea7c109140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58330/0 (socket says 172.21.2.101:58330) 2021-02-10T11:42:06.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.341 7fea81dca700 1 -- 172.21.2.101:0/2381200937 learned_addr learned my addr 172.21.2.101:0/2381200937 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:06.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.341 7fea815c9700 1 --2- 172.21.2.101:0/2381200937 >> v2:172.21.2.108:3300/0 conn(0x7fea7c0d1880 0x7fea7c1b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:06.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.341 7fea73fff700 1 -- 172.21.2.101:0/2381200937 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 769939021 0 0) 0x7fea74001500 con 0x7fea7c0cff80 2021-02-10T11:42:06.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.341 7fea73fff700 1 -- 172.21.2.101:0/2381200937 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea68002fe0 con 0x7fea7c0cff80 2021-02-10T11:42:06.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.341 7fea73fff700 1 -- 172.21.2.101:0/2381200937 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 459574974 0 0) 0x7fea6c000c60 con 0x7fea7c1b0af0 2021-02-10T11:42:06.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.341 7fea73fff700 1 -- 172.21.2.101:0/2381200937 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea68003210 con 0x7fea7c1b0af0 2021-02-10T11:42:06.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.342 7fea815c9700 1 -- 172.21.2.101:0/2381200937 >> v1:172.21.2.108:6789/0 conn(0x7fea7c1b0af0 legacy=0x7fea7c108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:06.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.342 7fea815c9700 1 -- 172.21.2.101:0/2381200937 >> v1:172.21.2.101:6789/0 conn(0x7fea7c0cff80 legacy=0x7fea7c109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:06.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.342 7fea815c9700 1 -- 172.21.2.101:0/2381200937 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7c088260 con 0x7fea7c0d1880 2021-02-10T11:42:06.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.342 7fea815c9700 1 --2- 172.21.2.101:0/2381200937 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea7c0d1880 0x7fea7c1b04d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7fea78006650 tx=0x7fea78009a00).ready entity=mon.1 client_cookie=789b1617b9246ee5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:06.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.342 7fea73fff700 1 -- 172.21.2.101:0/2381200937 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fea780009b0 con 0x7fea7c0d1880 2021-02-10T11:42:06.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.342 7fea73fff700 1 -- 172.21.2.101:0/2381200937 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea78009df0 con 0x7fea7c0d1880 2021-02-10T11:42:06.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.342 7fea73fff700 1 -- 172.21.2.101:0/2381200937 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fea7800c0c0 con 0x7fea7c0d1880 2021-02-10T11:42:06.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.342 7fea837ad700 1 -- 172.21.2.101:0/2381200937 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea7c0d1880 msgr2=0x7fea7c1b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:06.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.342 7fea837ad700 1 --2- 172.21.2.101:0/2381200937 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea7c0d1880 0x7fea7c1b04d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7fea78006650 tx=0x7fea78009a00).stop 2021-02-10T11:42:06.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.343 7fea837ad700 1 -- 172.21.2.101:0/2381200937 shutdown_connections 2021-02-10T11:42:06.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.343 7fea837ad700 1 --2- 172.21.2.101:0/2381200937 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea7c0d1880 0x7fea7c1b04d0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:06.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.343 7fea837ad700 1 -- 172.21.2.101:0/2381200937 >> 172.21.2.101:0/2381200937 conn(0x7fea7c0ce430 msgr2=0x7fea7c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:06.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.343 7fea837ad700 1 -- 172.21.2.101:0/2381200937 shutdown_connections 2021-02-10T11:42:06.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.343 7fea837ad700 1 -- 172.21.2.101:0/2381200937 wait complete. 2021-02-10T11:42:06.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.385 7fea837ad700 1 Processor -- start 2021-02-10T11:42:06.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.385 7fea837ad700 1 -- start start 2021-02-10T11:42:06.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.386 7fea837ad700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea7c1b8e50 0x7fea7c091bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:06.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.386 7fea837ad700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea7c1b9250 0x7fea7c0aa460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:06.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.386 7fea837ad700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea7c0aaa80 0x7fea7c1acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:06.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.386 7fea837ad700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c1b9770 con 0x7fea7c1b8e50 2021-02-10T11:42:06.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.386 7fea837ad700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c1ac570 con 0x7fea7c0aaa80 2021-02-10T11:42:06.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.386 7fea837ad700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c1ac160 con 0x7fea7c1b9250 2021-02-10T11:42:06.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.386 7fea80dc8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea7c1b9250 0x7fea7c0aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:06.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.386 7fea815c9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea7c1b8e50 0x7fea7c091bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:06.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.386 7fea815c9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea7c1b8e50 0x7fea7c091bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57840/0 (socket says 172.21.2.101:57840) 2021-02-10T11:42:06.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.386 7fea815c9700 1 -- 172.21.2.101:0/1458931856 learned_addr learned my addr 172.21.2.101:0/1458931856 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:06.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.386 7fea81dca700 1 --2- 172.21.2.101:0/1458931856 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea7c0aaa80 0x7fea7c1acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:06.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.387 7fea815c9700 1 -- 172.21.2.101:0/1458931856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea7c1b9250 msgr2=0x7fea7c0aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:06.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.387 7fea815c9700 1 --2- 172.21.2.101:0/1458931856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea7c1b9250 0x7fea7c0aa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:06.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.387 7fea815c9700 1 -- 172.21.2.101:0/1458931856 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea7c0aaa80 msgr2=0x7fea7c1acaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:06.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.387 7fea815c9700 1 --2- 172.21.2.101:0/1458931856 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea7c0aaa80 0x7fea7c1acaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:06.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.387 7fea815c9700 1 -- 172.21.2.101:0/1458931856 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7c1ad110 con 0x7fea7c1b8e50 2021-02-10T11:42:06.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.387 7fea80dc8700 1 --2- 172.21.2.101:0/1458931856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea7c1b9250 0x7fea7c0aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:42:06.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.387 7fea815c9700 1 --2- 172.21.2.101:0/1458931856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea7c1b8e50 0x7fea7c091bf0 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 rx=0x7fea7800c610 tx=0x7fea7800c9b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:06.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.387 7fea71ffb700 1 -- 172.21.2.101:0/1458931856 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fea780009b0 con 0x7fea7c1b8e50 2021-02-10T11:42:06.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.387 7fea71ffb700 1 -- 172.21.2.101:0/1458931856 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea78006360 con 0x7fea7c1b8e50 2021-02-10T11:42:06.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.387 7fea837ad700 1 -- 172.21.2.101:0/1458931856 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea7c1ad110 con 0x7fea7c1b8e50 2021-02-10T11:42:06.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.387 7fea71ffb700 1 -- 172.21.2.101:0/1458931856 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fea780020d0 con 0x7fea7c1b8e50 2021-02-10T11:42:06.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.387 7fea837ad700 1 -- 172.21.2.101:0/1458931856 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea7c1ad110 con 0x7fea7c1b8e50 2021-02-10T11:42:06.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.387 7fea71ffb700 1 -- 172.21.2.101:0/1458931856 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fea780080c0 con 0x7fea7c1b8e50 2021-02-10T11:42:06.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.387 7fea71ffb700 1 --2- 172.21.2.101:0/1458931856 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea5c018730 0x7fea5c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:06.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.387 7fea71ffb700 1 -- 172.21.2.101:0/1458931856 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fea78032650 con 0x7fea7c1b8e50 2021-02-10T11:42:06.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.388 7fea80dc8700 1 --2- 172.21.2.101:0/1458931856 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea5c018730 0x7fea5c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:06.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.388 7fea80dc8700 1 --2- 172.21.2.101:0/1458931856 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea5c018730 0x7fea5c01ac10 crc :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:06.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.390 7fea837ad700 1 -- 172.21.2.101:0/1458931856 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea7c082ad0 con 0x7fea7c1b8e50 2021-02-10T11:42:06.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.393 7fea71ffb700 1 -- 172.21.2.101:0/1458931856 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fea78006360 con 0x7fea7c1b8e50 2021-02-10T11:42:06.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.520 7fea837ad700 1 -- 172.21.2.101:0/1458931856 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000228711", "target": ["mgr", ""], "group_name": "group_0000000000192983"}) v1 -- 0x7fea7c082ad0 con 0x7fea5c018730 2021-02-10T11:42:06.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.532 7fea71ffb700 1 -- 172.21.2.101:0/1458931856 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fea6c0068d0 con 0x7fea5c018730 2021-02-10T11:42:06.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.537 7fea837ad700 1 -- 172.21.2.101:0/1458931856 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea5c018730 msgr2=0x7fea5c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:06.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.537 7fea837ad700 1 --2- 172.21.2.101:0/1458931856 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea5c018730 0x7fea5c01ac10 crc :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:06.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.537 7fea837ad700 1 -- 172.21.2.101:0/1458931856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea7c1b8e50 msgr2=0x7fea7c091bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:06.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.537 7fea837ad700 1 --2- 172.21.2.101:0/1458931856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea7c1b8e50 0x7fea7c091bf0 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 rx=0x7fea7800c610 tx=0x7fea7800c9b0).stop 2021-02-10T11:42:06.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.537 7fea837ad700 1 -- 172.21.2.101:0/1458931856 shutdown_connections 2021-02-10T11:42:06.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.537 7fea837ad700 1 --2- 172.21.2.101:0/1458931856 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea5c018730 0x7fea5c01ac10 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:06.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.537 7fea837ad700 1 --2- 172.21.2.101:0/1458931856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea7c1b9250 0x7fea7c0aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:06.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.537 7fea837ad700 1 --2- 172.21.2.101:0/1458931856 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea7c0aaa80 0x7fea7c1acaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:06.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.537 7fea837ad700 1 --2- 172.21.2.101:0/1458931856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea7c1b8e50 0x7fea7c091bf0 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:06.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.537 7fea837ad700 1 -- 172.21.2.101:0/1458931856 >> 172.21.2.101:0/1458931856 conn(0x7fea7c0846e0 msgr2=0x7fea7c1ab3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:06.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.537 7fea837ad700 1 -- 172.21.2.101:0/1458931856 shutdown_connections 2021-02-10T11:42:06.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:06.537 7fea837ad700 1 -- 172.21.2.101:0/1458931856 wait complete. 2021-02-10T11:42:06.549 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.1 && sudo ls volumes 2021-02-10T11:42:06.896 INFO:teuthology.orchestra.run.gibba001.stdout:_deleting 2021-02-10T11:42:06.896 INFO:teuthology.orchestra.run.gibba001.stdout:group_0000000000192983 2021-02-10T11:42:06.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:06.897 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:42:06.897 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:42:07.234 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:42:07.235 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:42:07.235 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:42:07.235 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:42:07.235 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:42:07.591 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[151589]: fuse finished with error 0 and tester_r 0 2021-02-10T11:42:07.592 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:07.592 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:42:07.592 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:42:07.926 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:42:07.926 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:42:07.927 INFO:teuthology.orchestra.run:waiting for 900 2021-02-10T11:42:07.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:42:07.927 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:42:08.263 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolumegroup rm cephfs group_0000000000192983 2021-02-10T11:42:08.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.677 7f34488cf700 1 Processor -- start 2021-02-10T11:42:08.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.677 7f34488cf700 1 -- start start 2021-02-10T11:42:08.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.678 7f34488cf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f34440d1870 0x7f34441b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:08.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.678 7f34488cf700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f34440cff70 0x7f3444109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:08.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.678 7f34488cf700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34440fe1c0 con 0x7f34441b0ae0 2021-02-10T11:42:08.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.678 7f34488cf700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3444109ad0 con 0x7f34440d1870 2021-02-10T11:42:08.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.678 7f34488cf700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3444109750 con 0x7f34440cff70 2021-02-10T11:42:08.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.678 7f344261c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f34440d1870 0x7f34441b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:08.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.678 7f3442e1d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f34440cff70 0x7f3444109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:08.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.678 7f344261c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f34440d1870 0x7f34441b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57846/0 (socket says 172.21.2.101:57846) 2021-02-10T11:42:08.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.678 7f344261c700 1 -- 172.21.2.101:0/3230977020 learned_addr learned my addr 172.21.2.101:0/3230977020 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:08.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.678 7f3442e1d700 1 -- 172.21.2.101:0/3230977020 >> v2:172.21.2.101:3300/0 conn(0x7f34440d1870 msgr2=0x7f34441b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:08.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.678 7f3442e1d700 1 --2- 172.21.2.101:0/3230977020 >> v2:172.21.2.101:3300/0 conn(0x7f34440d1870 0x7f34441b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:08.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.678 7f3442e1d700 1 -- 172.21.2.101:0/3230977020 >> v1:172.21.2.108:6789/0 conn(0x7f34441b0ae0 legacy=0x7f3444108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:08.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.679 7f3442e1d700 1 -- 172.21.2.101:0/3230977020 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3444088070 con 0x7f34440cff70 2021-02-10T11:42:08.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.679 7f3442e1d700 1 --2- 172.21.2.101:0/3230977020 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f34440cff70 0x7f3444109130 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f343c006e50 tx=0x7f343c009e70).ready entity=mon.1 client_cookie=86ebe227b4ebebdb server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:08.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.679 7f344161a700 1 -- 172.21.2.101:0/3230977020 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f343c00b510 con 0x7f34440cff70 2021-02-10T11:42:08.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.679 7f344161a700 1 -- 172.21.2.101:0/3230977020 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343c006790 con 0x7f34440cff70 2021-02-10T11:42:08.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.679 7f344161a700 1 -- 172.21.2.101:0/3230977020 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f343c00b940 con 0x7f34440cff70 2021-02-10T11:42:08.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.679 7f34488cf700 1 -- 172.21.2.101:0/3230977020 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f34440cff70 msgr2=0x7f3444109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:08.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.679 7f34488cf700 1 --2- 172.21.2.101:0/3230977020 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f34440cff70 0x7f3444109130 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f343c006e50 tx=0x7f343c009e70).stop 2021-02-10T11:42:08.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.680 7f34488cf700 1 -- 172.21.2.101:0/3230977020 shutdown_connections 2021-02-10T11:42:08.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.680 7f34488cf700 1 --2- 172.21.2.101:0/3230977020 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f34440cff70 0x7f3444109130 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:08.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.680 7f34488cf700 1 --2- 172.21.2.101:0/3230977020 >> v2:172.21.2.101:3300/0 conn(0x7f34440d1870 0x7f34441b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:08.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.680 7f34488cf700 1 -- 172.21.2.101:0/3230977020 >> 172.21.2.101:0/3230977020 conn(0x7f34440ce420 msgr2=0x7f34440d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:08.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.680 7f34488cf700 1 -- 172.21.2.101:0/3230977020 shutdown_connections 2021-02-10T11:42:08.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.680 7f34488cf700 1 -- 172.21.2.101:0/3230977020 wait complete. 2021-02-10T11:42:08.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.694 7f34488cf700 1 Processor -- start 2021-02-10T11:42:08.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.694 7f34488cf700 1 -- start start 2021-02-10T11:42:08.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.695 7f34488cf700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3444085bc0 0x7f34440863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:08.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.695 7f34488cf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f34440954c0 0x7f3444096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:08.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.695 7f34488cf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f34440958c0 0x7f3444096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:08.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.695 7f34488cf700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3444095d80 con 0x7f34440954c0 2021-02-10T11:42:08.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.695 7f34488cf700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34440bfa60 con 0x7f3444085bc0 2021-02-10T11:42:08.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.695 7f34488cf700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3444085fc0 con 0x7f34440958c0 2021-02-10T11:42:08.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.695 7f3442e1d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f34440958c0 0x7f3444096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:08.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.695 7f3441e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f34440954c0 0x7f3444096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:08.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.695 7f344261c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3444085bc0 0x7f34440863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:08.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.695 7f3442e1d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f34440958c0 0x7f3444096920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:50584/0 (socket says 172.21.2.101:50584) 2021-02-10T11:42:08.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.695 7f3442e1d700 1 -- 172.21.2.101:0/2532800267 learned_addr learned my addr 172.21.2.101:0/2532800267 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:08.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.696 7f3442e1d700 1 -- 172.21.2.101:0/2532800267 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3444085bc0 msgr2=0x7f34440863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:08.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.696 7f3442e1d700 1 --2- 172.21.2.101:0/2532800267 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3444085bc0 0x7f34440863e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:08.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.696 7f3442e1d700 1 -- 172.21.2.101:0/2532800267 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f34440954c0 msgr2=0x7f3444096300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:08.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.696 7f3442e1d700 1 --2- 172.21.2.101:0/2532800267 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f34440954c0 0x7f3444096300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:08.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.696 7f3442e1d700 1 -- 172.21.2.101:0/2532800267 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3444086c10 con 0x7f34440958c0 2021-02-10T11:42:08.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.696 7f3442e1d700 1 --2- 172.21.2.101:0/2532800267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f34440958c0 0x7f3444096920 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f343c009930 tx=0x7f343c00bcc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:08.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.696 7f343affd700 1 -- 172.21.2.101:0/2532800267 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f343c00be00 con 0x7f34440958c0 2021-02-10T11:42:08.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.696 7f343affd700 1 -- 172.21.2.101:0/2532800267 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343c009020 con 0x7f34440958c0 2021-02-10T11:42:08.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.696 7f343affd700 1 -- 172.21.2.101:0/2532800267 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f343c00c4e0 con 0x7f34440958c0 2021-02-10T11:42:08.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.696 7f34488cf700 1 -- 172.21.2.101:0/2532800267 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3444086c10 con 0x7f34440958c0 2021-02-10T11:42:08.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.696 7f34488cf700 1 -- 172.21.2.101:0/2532800267 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3444097230 con 0x7f34440958c0 2021-02-10T11:42:08.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.697 7f343affd700 1 -- 172.21.2.101:0/2532800267 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f343c002020 con 0x7f34440958c0 2021-02-10T11:42:08.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.697 7f343affd700 1 --2- 172.21.2.101:0/2532800267 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f34280186c0 0x7f342801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:08.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.697 7f343affd700 1 -- 172.21.2.101:0/2532800267 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f343c0328d0 con 0x7f34440958c0 2021-02-10T11:42:08.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.697 7f344261c700 1 --2- 172.21.2.101:0/2532800267 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f34280186c0 0x7f342801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:08.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.697 7f344261c700 1 --2- 172.21.2.101:0/2532800267 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f34280186c0 0x7f342801abc0 crc :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:08.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.728 7f34488cf700 1 -- 172.21.2.101:0/2532800267 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34440fe1c0 con 0x7f34440958c0 2021-02-10T11:42:08.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.731 7f343affd700 1 -- 172.21.2.101:0/2532800267 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f343c0093d0 con 0x7f34440958c0 2021-02-10T11:42:08.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.862 7f34488cf700 1 -- 172.21.2.101:0/2532800267 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "target": ["mgr", ""], "group_name": "group_0000000000192983"}) v1 -- 0x7f3444086c10 con 0x7f34280186c0 2021-02-10T11:42:08.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.864 7f343affd700 1 -- 172.21.2.101:0/2532800267 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f342c006a80 con 0x7f34280186c0 2021-02-10T11:42:08.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.866 7f34488cf700 1 -- 172.21.2.101:0/2532800267 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f34280186c0 msgr2=0x7f342801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:08.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.866 7f34488cf700 1 --2- 172.21.2.101:0/2532800267 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f34280186c0 0x7f342801abc0 crc :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:08.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.866 7f34488cf700 1 -- 172.21.2.101:0/2532800267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f34440958c0 msgr2=0x7f3444096920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:08.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.866 7f34488cf700 1 --2- 172.21.2.101:0/2532800267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f34440958c0 0x7f3444096920 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f343c009930 tx=0x7f343c00bcc0).stop 2021-02-10T11:42:08.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.866 7f34488cf700 1 -- 172.21.2.101:0/2532800267 shutdown_connections 2021-02-10T11:42:08.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.866 7f34488cf700 1 --2- 172.21.2.101:0/2532800267 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f34280186c0 0x7f342801abc0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:08.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.866 7f34488cf700 1 --2- 172.21.2.101:0/2532800267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f34440958c0 0x7f3444096920 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:08.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.866 7f34488cf700 1 --2- 172.21.2.101:0/2532800267 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f34440954c0 0x7f3444096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:08.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.866 7f34488cf700 1 --2- 172.21.2.101:0/2532800267 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3444085bc0 0x7f34440863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:08.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.866 7f34488cf700 1 -- 172.21.2.101:0/2532800267 >> 172.21.2.101:0/2532800267 conn(0x7f34440bef00 msgr2=0x7f344409d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:08.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.867 7f34488cf700 1 -- 172.21.2.101:0/2532800267 shutdown_connections 2021-02-10T11:42:08.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:08.867 7f34488cf700 1 -- 172.21.2.101:0/2532800267 wait complete. 2021-02-10T11:42:08.877 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:42:08.877 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:42:09.223 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:42:09.569 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:42:09.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:09.569 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:42:09.569 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:42:09.906 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:42:09.906 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:42:09.906 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:42:09.906 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:42:09.906 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:42:10.263 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[150948]: fuse finished with error 0 and tester_r 0 2021-02-10T11:42:10.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:10.264 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:42:10.264 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:42:10.599 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:42:10.599 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:42:10.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:42:10.599 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:42:10.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:10.936 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:42:10.936 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:42:11.271 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:42:11.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:42:11.271 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:42:11.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:42:11.272 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:42:11.606 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:11.606 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:42:11.606 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:42:11.942 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:42:11.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:42:11.943 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:42:11.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:42:11.943 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:42:12.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:12.277 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:42:12.277 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:42:12.612 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:42:12.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:42:12.613 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:42:12.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:42:12.613 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:42:12.947 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-02-10T11:42:13.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.359 7f98a4444700 1 Processor -- start 2021-02-10T11:42:13.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.359 7f98a4444700 1 -- start start 2021-02-10T11:42:13.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.360 7f98a4444700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f989c0d1860 0x7f989c1b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:13.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.360 7f98a4444700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f989c0f0640 0x7f989c0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:13.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.360 7f98a4444700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0f0b00 con 0x7f989c1b0a00 2021-02-10T11:42:13.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.360 7f98a4444700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f989c0fe220 con 0x7f989c0f0640 2021-02-10T11:42:13.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.360 7f98a4444700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989c0e5ec0 con 0x7f989c0d1860 2021-02-10T11:42:13.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.360 7f98a1a5f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f989c1b0a00 0x7f989c0cf320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58352/0 (socket says 172.21.2.101:58352) 2021-02-10T11:42:13.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.360 7f98a1a5f700 1 -- 172.21.2.101:0/2355726828 learned_addr learned my addr 172.21.2.101:0/2355726828 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:13.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.360 7f98a2a61700 1 --2- 172.21.2.101:0/2355726828 >> v2:172.21.2.108:3300/0 conn(0x7f989c0f0640 0x7f989c0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:13.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.360 7f98a2260700 1 --2- 172.21.2.101:0/2355726828 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f989c0d1860 0x7f989c1b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:13.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.360 7f98a125e700 1 -- 172.21.2.101:0/2355726828 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1788844354 0 0) 0x7f9898000ce0 con 0x7f989c1b0a00 2021-02-10T11:42:13.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.360 7f98a125e700 1 -- 172.21.2.101:0/2355726828 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9888002fe0 con 0x7f989c1b0a00 2021-02-10T11:42:13.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.361 7f98a2a61700 1 -- 172.21.2.101:0/2355726828 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f989c0d1860 msgr2=0x7f989c1b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:13.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.361 7f98a2a61700 1 --2- 172.21.2.101:0/2355726828 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f989c0d1860 0x7f989c1b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:13.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.361 7f98a2a61700 1 -- 172.21.2.101:0/2355726828 >> v1:172.21.2.101:6789/0 conn(0x7f989c1b0a00 legacy=0x7f989c0cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:13.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.361 7f98a2a61700 1 -- 172.21.2.101:0/2355726828 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c0f2090 con 0x7f989c0f0640 2021-02-10T11:42:13.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.361 7f98a2a61700 1 --2- 172.21.2.101:0/2355726828 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f989c0f0640 0x7f989c0cfb10 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f9894006e50 tx=0x7f9894009e70).ready entity=mon.1 client_cookie=3f4e391753d44b87 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:13.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.361 7f98a125e700 1 -- 172.21.2.101:0/2355726828 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9894006a20 con 0x7f989c0f0640 2021-02-10T11:42:13.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.361 7f98a125e700 1 -- 172.21.2.101:0/2355726828 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9894006790 con 0x7f989c0f0640 2021-02-10T11:42:13.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.361 7f98a125e700 1 -- 172.21.2.101:0/2355726828 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9894000f90 con 0x7f989c0f0640 2021-02-10T11:42:13.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.361 7f98a4444700 1 -- 172.21.2.101:0/2355726828 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f989c0f0640 msgr2=0x7f989c0cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:13.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.361 7f98a4444700 1 --2- 172.21.2.101:0/2355726828 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f989c0f0640 0x7f989c0cfb10 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f9894006e50 tx=0x7f9894009e70).stop 2021-02-10T11:42:13.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.362 7f98a4444700 1 -- 172.21.2.101:0/2355726828 shutdown_connections 2021-02-10T11:42:13.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.362 7f98a4444700 1 --2- 172.21.2.101:0/2355726828 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f989c0f0640 0x7f989c0cfb10 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:13.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.362 7f98a4444700 1 --2- 172.21.2.101:0/2355726828 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f989c0d1860 0x7f989c1b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:13.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.362 7f98a4444700 1 -- 172.21.2.101:0/2355726828 >> 172.21.2.101:0/2355726828 conn(0x7f989c0ce440 msgr2=0x7f989c0d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:13.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.362 7f98a4444700 1 -- 172.21.2.101:0/2355726828 shutdown_connections 2021-02-10T11:42:13.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.362 7f98a4444700 1 -- 172.21.2.101:0/2355726828 wait complete. 2021-02-10T11:42:13.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.376 7f98a4444700 1 Processor -- start 2021-02-10T11:42:13.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.376 7f98a4444700 1 -- start start 2021-02-10T11:42:13.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.377 7f98a4444700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f989c09eaf0 0x7f989c0ebdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:13.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.377 7f98a4444700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f989c0b72e0 0x7f989c09f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:13.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.377 7f98a4444700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f989c0b7dd0 0x7f989c0b81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:13.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.377 7f98a4444700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989c0b7850 con 0x7f989c0b72e0 2021-02-10T11:42:13.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.377 7f98a4444700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989c09eef0 con 0x7f989c09eaf0 2021-02-10T11:42:13.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.377 7f98a4444700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989c0ec3f0 con 0x7f989c0b7dd0 2021-02-10T11:42:13.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.377 7f98a2260700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f989c09eaf0 0x7f989c0ebdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:13.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.377 7f98a1a5f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f989c0b72e0 0x7f989c09f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:13.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.377 7f98a1a5f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f989c0b72e0 0x7f989c09f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57866/0 (socket says 172.21.2.101:57866) 2021-02-10T11:42:13.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.377 7f98a1a5f700 1 -- 172.21.2.101:0/3404161810 learned_addr learned my addr 172.21.2.101:0/3404161810 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:13.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.377 7f98a2260700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f989c09eaf0 0x7f989c0ebdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:40122/0 (socket says 172.21.2.101:40122) 2021-02-10T11:42:13.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.377 7f98a2a61700 1 --2- 172.21.2.101:0/3404161810 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f989c0b7dd0 0x7f989c0b81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:13.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.377 7f98a1a5f700 1 -- 172.21.2.101:0/3404161810 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f989c0b7dd0 msgr2=0x7f989c0b81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:13.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.377 7f98a1a5f700 1 --2- 172.21.2.101:0/3404161810 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f989c0b7dd0 0x7f989c0b81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:13.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.377 7f98a1a5f700 1 -- 172.21.2.101:0/3404161810 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f989c09eaf0 msgr2=0x7f989c0ebdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:13.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.378 7f98a1a5f700 1 --2- 172.21.2.101:0/3404161810 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f989c09eaf0 0x7f989c0ebdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:13.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.378 7f98a1a5f700 1 -- 172.21.2.101:0/3404161810 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c09f7c0 con 0x7f989c0b72e0 2021-02-10T11:42:13.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.378 7f98a1a5f700 1 --2- 172.21.2.101:0/3404161810 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f989c0b72e0 0x7f989c09f1a0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 rx=0x7f989800b5c0 tx=0x7f989800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:13.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.378 7f9892ffd700 1 -- 172.21.2.101:0/3404161810 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9898006030 con 0x7f989c0b72e0 2021-02-10T11:42:13.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.378 7f9892ffd700 1 -- 172.21.2.101:0/3404161810 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98980043a0 con 0x7f989c0b72e0 2021-02-10T11:42:13.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.378 7f9892ffd700 1 -- 172.21.2.101:0/3404161810 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f98980048c0 con 0x7f989c0b72e0 2021-02-10T11:42:13.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.378 7f98a4444700 1 -- 172.21.2.101:0/3404161810 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f989c09f7c0 con 0x7f989c0b72e0 2021-02-10T11:42:13.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.378 7f98a4444700 1 -- 172.21.2.101:0/3404161810 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f989c09f7c0 con 0x7f989c0b72e0 2021-02-10T11:42:13.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.379 7f9892ffd700 1 -- 172.21.2.101:0/3404161810 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9898004580 con 0x7f989c0b72e0 2021-02-10T11:42:13.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.379 7f9892ffd700 1 --2- 172.21.2.101:0/3404161810 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98840186c0 0x7f988401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:13.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.379 7f9892ffd700 1 -- 172.21.2.101:0/3404161810 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9898032370 con 0x7f989c0b72e0 2021-02-10T11:42:13.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.379 7f98a2260700 1 --2- 172.21.2.101:0/3404161810 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98840186c0 0x7f988401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:13.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.379 7f98a2260700 1 --2- 172.21.2.101:0/3404161810 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98840186c0 0x7f988401abc0 crc :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:13.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.410 7f98a4444700 1 -- 172.21.2.101:0/3404161810 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f989c0ebba0 con 0x7f989c0b72e0 2021-02-10T11:42:13.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.412 7f9892ffd700 1 -- 172.21.2.101:0/3404161810 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9898024fe0 con 0x7f989c0b72e0 2021-02-10T11:42:13.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.562 7f98a4444700 1 -- 172.21.2.101:0/3404161810 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f989c0f0b00 con 0x7f989c0b72e0 2021-02-10T11:42:13.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.566 7f9892ffd700 1 -- 172.21.2.101:0/3404161810 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9898024fe0 con 0x7f989c0b72e0 2021-02-10T11:42:13.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.571 7f98a4444700 1 -- 172.21.2.101:0/3404161810 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98840186c0 msgr2=0x7f988401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:13.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.571 7f98a4444700 1 --2- 172.21.2.101:0/3404161810 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98840186c0 0x7f988401abc0 crc :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:13.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.571 7f98a4444700 1 -- 172.21.2.101:0/3404161810 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f989c0b72e0 msgr2=0x7f989c09f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:13.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.571 7f98a4444700 1 --2- 172.21.2.101:0/3404161810 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f989c0b72e0 0x7f989c09f1a0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 rx=0x7f989800b5c0 tx=0x7f989800b960).stop 2021-02-10T11:42:13.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.572 7f98a4444700 1 -- 172.21.2.101:0/3404161810 shutdown_connections 2021-02-10T11:42:13.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.572 7f98a4444700 1 --2- 172.21.2.101:0/3404161810 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98840186c0 0x7f988401abc0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:13.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.572 7f98a4444700 1 --2- 172.21.2.101:0/3404161810 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f989c0b7dd0 0x7f989c0b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:13.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.572 7f98a4444700 1 --2- 172.21.2.101:0/3404161810 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f989c0b72e0 0x7f989c09f1a0 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:13.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.572 7f98a4444700 1 --2- 172.21.2.101:0/3404161810 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f989c09eaf0 0x7f989c0ebdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:13.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.572 7f98a4444700 1 -- 172.21.2.101:0/3404161810 >> 172.21.2.101:0/3404161810 conn(0x7f989c0b8df0 msgr2=0x7f989c0add50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:13.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.572 7f98a4444700 1 -- 172.21.2.101:0/3404161810 shutdown_connections 2021-02-10T11:42:13.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.572 7f98a4444700 1 -- 172.21.2.101:0/3404161810 wait complete. 2021-02-10T11:42:13.583 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_multitenant_subvolumes' 2021-02-10T11:42:14.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.992 7fbfadda8700 1 Processor -- start 2021-02-10T11:42:14.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.992 7fbfadda8700 1 -- start start 2021-02-10T11:42:14.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.992 7fbfadda8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbfa80d1870 0x7fbfa81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:14.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.992 7fbfadda8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbfa81b0a10 0x7fbfa80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:14.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.992 7fbfadda8700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa80f0b10 con 0x7fbfa80d1c70 2021-02-10T11:42:14.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.992 7fbfadda8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfa80fe230 con 0x7fbfa81b0a10 2021-02-10T11:42:14.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.992 7fbfadda8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa80e5ed0 con 0x7fbfa80d1870 2021-02-10T11:42:14.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.993 7fbfa7fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fbfa80d1c70 0x7fbfa80cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58366/0 (socket says 172.21.2.101:58366) 2021-02-10T11:42:14.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.993 7fbfa7fff700 1 -- 172.21.2.101:0/1527114998 learned_addr learned my addr 172.21.2.101:0/1527114998 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:14.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.993 7fbfa77fe700 1 --2- 172.21.2.101:0/1527114998 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbfa80d1870 0x7fbfa81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:14.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.993 7fbfa67fc700 1 -- 172.21.2.101:0/1527114998 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306632877 0 0) 0x7fbfa0001500 con 0x7fbfa80d1c70 2021-02-10T11:42:14.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.993 7fbfa6ffd700 1 --2- 172.21.2.101:0/1527114998 >> v2:172.21.2.108:3300/0 conn(0x7fbfa81b0a10 0x7fbfa80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:14.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.993 7fbfa67fc700 1 -- 172.21.2.101:0/1527114998 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf90002fe0 con 0x7fbfa80d1c70 2021-02-10T11:42:14.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa67fc700 1 -- 172.21.2.101:0/1527114998 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1121215189 0 0) 0x7fbfa0001270 con 0x7fbfa80d1c70 2021-02-10T11:42:14.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa67fc700 1 -- 172.21.2.101:0/1527114998 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbfa80d1870 msgr2=0x7fbfa81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:14.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa67fc700 1 --2- 172.21.2.101:0/1527114998 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbfa80d1870 0x7fbfa81b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:14.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa67fc700 1 -- 172.21.2.101:0/1527114998 >> v2:172.21.2.108:3300/0 conn(0x7fbfa81b0a10 msgr2=0x7fbfa80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:14.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa67fc700 1 --2- 172.21.2.101:0/1527114998 >> v2:172.21.2.108:3300/0 conn(0x7fbfa81b0a10 0x7fbfa80cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:14.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa67fc700 1 -- 172.21.2.101:0/1527114998 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa80f0650 con 0x7fbfa80d1c70 2021-02-10T11:42:14.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa67fc700 1 -- 172.21.2.101:0/1527114998 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fbfa0001b50 con 0x7fbfa80d1c70 2021-02-10T11:42:14.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa67fc700 1 -- 172.21.2.101:0/1527114998 >> v1:172.21.2.101:6789/0 conn(0x7fbfa80d1c70 legacy=0x7fbfa80cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:14.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa67fc700 1 --2- 172.21.2.101:0/1527114998 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf90003a60 0x7fbf90005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:14.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa67fc700 1 --2- 172.21.2.101:0/1527114998 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbf9004a7a0 0x7fbf9004cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:14.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa67fc700 1 --2- 172.21.2.101:0/1527114998 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf9004d5a0 0x7fbf9004fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:14.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa67fc700 1 -- 172.21.2.101:0/1527114998 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf900502e0 con 0x7fbf90003a60 2021-02-10T11:42:14.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa67fc700 1 -- 172.21.2.101:0/1527114998 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf90050500 con 0x7fbf9004a7a0 2021-02-10T11:42:14.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa67fc700 1 -- 172.21.2.101:0/1527114998 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf90050720 con 0x7fbf9004d5a0 2021-02-10T11:42:14.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa77fe700 1 --2- 172.21.2.101:0/1527114998 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf90003a60 0x7fbf90005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:14.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa6ffd700 1 --2- 172.21.2.101:0/1527114998 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbf9004a7a0 0x7fbf9004cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:14.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa7fff700 1 --2- 172.21.2.101:0/1527114998 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf9004d5a0 0x7fbf9004fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:14.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa77fe700 1 -- 172.21.2.101:0/1527114998 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf9004d5a0 msgr2=0x7fbf9004fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:14.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa77fe700 1 --2- 172.21.2.101:0/1527114998 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf9004d5a0 0x7fbf9004fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:14.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa77fe700 1 -- 172.21.2.101:0/1527114998 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbf9004a7a0 msgr2=0x7fbf9004cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:14.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa77fe700 1 --2- 172.21.2.101:0/1527114998 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbf9004a7a0 0x7fbf9004cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:14.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa77fe700 1 -- 172.21.2.101:0/1527114998 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf900509c0 con 0x7fbf90003a60 2021-02-10T11:42:14.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa6ffd700 1 --2- 172.21.2.101:0/1527114998 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbf9004a7a0 0x7fbf9004cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:42:14.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa77fe700 1 --2- 172.21.2.101:0/1527114998 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf90003a60 0x7fbf90005ef0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 rx=0x7fbf98006ac0 tx=0x7fbf98009ab0).ready entity=mon.0 client_cookie=7529a95e2fb8113e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:14.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa67fc700 1 -- 172.21.2.101:0/1527114998 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbf9800c0c0 con 0x7fbf90003a60 2021-02-10T11:42:14.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa67fc700 1 -- 172.21.2.101:0/1527114998 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9800c3c0 con 0x7fbf90003a60 2021-02-10T11:42:14.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa67fc700 1 -- 172.21.2.101:0/1527114998 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbf980009b0 con 0x7fbf90003a60 2021-02-10T11:42:14.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfadda8700 1 -- 172.21.2.101:0/1527114998 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf90003a60 msgr2=0x7fbf90005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:14.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfadda8700 1 --2- 172.21.2.101:0/1527114998 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf90003a60 0x7fbf90005ef0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 rx=0x7fbf98006ac0 tx=0x7fbf98009ab0).stop 2021-02-10T11:42:14.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.994 7fbfa7fff700 1 -- 172.21.2.101:0/1527114998 reap_dead start 2021-02-10T11:42:14.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.995 7fbfadda8700 1 -- 172.21.2.101:0/1527114998 shutdown_connections 2021-02-10T11:42:14.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.995 7fbfadda8700 1 -- 172.21.2.101:0/1527114998 >> 172.21.2.101:0/1527114998 conn(0x7fbfa80ce450 msgr2=0x7fbfa80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:14.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.995 7fbfadda8700 1 -- 172.21.2.101:0/1527114998 shutdown_connections 2021-02-10T11:42:14.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:13.995 7fbfadda8700 1 -- 172.21.2.101:0/1527114998 wait complete. 2021-02-10T11:42:14.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.009 7fbfadda8700 1 Processor -- start 2021-02-10T11:42:14.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.010 7fbfadda8700 1 -- start start 2021-02-10T11:42:14.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.010 7fbfadda8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbfa8195650 0x7fbfa80b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:14.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.010 7fbfadda8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbfa80b5b80 0x7fbfa81a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:14.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.010 7fbfadda8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbfa80acfa0 0x7fbfa81a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:14.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.010 7fbfadda8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa80b6010 con 0x7fbfa8195650 2021-02-10T11:42:14.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.010 7fbfadda8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa81a3a60 con 0x7fbfa80acfa0 2021-02-10T11:42:14.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.010 7fbfadda8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa81a3c40 con 0x7fbfa80b5b80 2021-02-10T11:42:14.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.010 7fbfa77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbfa8195650 0x7fbfa80b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:14.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.010 7fbfa77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbfa8195650 0x7fbfa80b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57884/0 (socket says 172.21.2.101:57884) 2021-02-10T11:42:14.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.010 7fbfa77fe700 1 -- 172.21.2.101:0/3872815535 learned_addr learned my addr 172.21.2.101:0/3872815535 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:14.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.010 7fbfa77fe700 1 -- 172.21.2.101:0/3872815535 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbfa80b5b80 msgr2=0x7fbfa81a2e20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:42:14.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.010 7fbfa77fe700 1 --2- 172.21.2.101:0/3872815535 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbfa80b5b80 0x7fbfa81a2e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:14.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.010 7fbfa77fe700 1 -- 172.21.2.101:0/3872815535 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbfa80acfa0 msgr2=0x7fbfa81a3440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:42:14.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.010 7fbfa77fe700 1 --2- 172.21.2.101:0/3872815535 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbfa80acfa0 0x7fbfa81a3440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:14.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.010 7fbfa77fe700 1 -- 172.21.2.101:0/3872815535 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa80bcae0 con 0x7fbfa8195650 2021-02-10T11:42:14.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.010 7fbfa77fe700 1 --2- 172.21.2.101:0/3872815535 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbfa8195650 0x7fbfa80b5560 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 rx=0x7fbf980097a0 tx=0x7fbf9800b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:14.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.010 7fbf8ffff700 1 -- 172.21.2.101:0/3872815535 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbf9800bd60 con 0x7fbfa8195650 2021-02-10T11:42:14.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.011 7fbf8ffff700 1 -- 172.21.2.101:0/3872815535 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf98009140 con 0x7fbfa8195650 2021-02-10T11:42:14.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.011 7fbfadda8700 1 -- 172.21.2.101:0/3872815535 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa80bcae0 con 0x7fbfa8195650 2021-02-10T11:42:14.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.011 7fbf8ffff700 1 -- 172.21.2.101:0/3872815535 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbf98002030 con 0x7fbfa8195650 2021-02-10T11:42:14.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.011 7fbfadda8700 1 -- 172.21.2.101:0/3872815535 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa80bcae0 con 0x7fbfa8195650 2021-02-10T11:42:14.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.011 7fbf8ffff700 1 -- 172.21.2.101:0/3872815535 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbf98000940 con 0x7fbfa8195650 2021-02-10T11:42:14.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.011 7fbf8ffff700 1 --2- 172.21.2.101:0/3872815535 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf88018730 0x7fbf8801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:14.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.011 7fbf8ffff700 1 -- 172.21.2.101:0/3872815535 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fbf98032600 con 0x7fbfa8195650 2021-02-10T11:42:14.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.011 7fbfa6ffd700 1 --2- 172.21.2.101:0/3872815535 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf88018730 0x7fbf8801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:14.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.012 7fbfa6ffd700 1 --2- 172.21.2.101:0/3872815535 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf88018730 0x7fbf8801ac10 crc :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:14.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.042 7fbfadda8700 1 -- 172.21.2.101:0/3872815535 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa80b6010 con 0x7fbfa8195650 2021-02-10T11:42:14.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.045 7fbf8ffff700 1 -- 172.21.2.101:0/3872815535 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbf98000940 con 0x7fbfa8195650 2021-02-10T11:42:14.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.177 7fbfadda8700 1 -- 172.21.2.101:0/3872815535 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_multitenant_subvolumes"]} v 0) v1 -- 0x7fbfa80b6010 con 0x7fbfa8195650 2021-02-10T11:42:14.583 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.581 7fbf8ffff700 1 -- 172.21.2.101:0/3872815535 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_multitenant_subvolumes"]}]=0 v617) v1 ==== 144+0+0 (secure 0 0 0) 0x7fbf98009140 con 0x7fbfa8195650 2021-02-10T11:42:14.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.594 7fbfadda8700 1 -- 172.21.2.101:0/3872815535 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf88018730 msgr2=0x7fbf8801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:14.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.594 7fbfadda8700 1 --2- 172.21.2.101:0/3872815535 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf88018730 0x7fbf8801ac10 crc :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:14.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.594 7fbfadda8700 1 -- 172.21.2.101:0/3872815535 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbfa8195650 msgr2=0x7fbfa80b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:14.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.594 7fbfadda8700 1 --2- 172.21.2.101:0/3872815535 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbfa8195650 0x7fbfa80b5560 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 rx=0x7fbf980097a0 tx=0x7fbf9800b970).stop 2021-02-10T11:42:14.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.594 7fbfadda8700 1 -- 172.21.2.101:0/3872815535 shutdown_connections 2021-02-10T11:42:14.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.594 7fbfadda8700 1 --2- 172.21.2.101:0/3872815535 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf88018730 0x7fbf8801ac10 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:14.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.594 7fbfadda8700 1 --2- 172.21.2.101:0/3872815535 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbfa80b5b80 0x7fbfa81a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:14.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.594 7fbfadda8700 1 --2- 172.21.2.101:0/3872815535 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbfa80acfa0 0x7fbfa81a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:14.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.594 7fbfadda8700 1 --2- 172.21.2.101:0/3872815535 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbfa8195650 0x7fbfa80b5560 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:14.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.594 7fbfadda8700 1 -- 172.21.2.101:0/3872815535 >> 172.21.2.101:0/3872815535 conn(0x7fbfa80b3040 msgr2=0x7fbfa80c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:14.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.595 7fbfadda8700 1 -- 172.21.2.101:0/3872815535 shutdown_connections 2021-02-10T11:42:14.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:14.595 7fbfadda8700 1 -- 172.21.2.101:0/3872815535 wait complete. 2021-02-10T11:42:14.605 INFO:tasks.cephfs_test_runner:test_multitenant_subvolumes (tasks.cephfs.test_volumes.TestVolumes) ... ok 2021-02-10T11:42:14.605 INFO:tasks.cephfs_test_runner:Starting test: test_non_clone_status (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:42:14.605 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_non_clone_status' 2021-02-10T11:42:15.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.015 7f499754f700 1 Processor -- start 2021-02-10T11:42:15.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.015 7f499754f700 1 -- start start 2021-02-10T11:42:15.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.015 7f499754f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f49900d1870 0x7f49901b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:15.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.015 7f499754f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f49900f0650 0x7f49900cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:15.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.015 7f499754f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49900f0b10 con 0x7f49901b0a10 2021-02-10T11:42:15.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.015 7f499754f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49900e5ed0 con 0x7f49900d1870 2021-02-10T11:42:15.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.015 7f499754f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49900d1c70 con 0x7f49900f0650 2021-02-10T11:42:15.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.016 7f499536b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f49900d1870 0x7f49901b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:15.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.016 7f499536b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f49900d1870 0x7f49901b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57892/0 (socket says 172.21.2.101:57892) 2021-02-10T11:42:15.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.016 7f499536b700 1 -- 172.21.2.101:0/3041670362 learned_addr learned my addr 172.21.2.101:0/3041670362 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:15.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.016 7f499536b700 1 -- 172.21.2.101:0/3041670362 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f49900f0650 msgr2=0x7f49900cfb20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:42:15.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.016 7f499536b700 1 --2- 172.21.2.101:0/3041670362 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f49900f0650 0x7f49900cfb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:15.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.016 7f499536b700 1 -- 172.21.2.101:0/3041670362 >> v1:172.21.2.108:6789/0 conn(0x7f49901b0a10 legacy=0x7f49900cf330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:42:15.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.016 7f499536b700 1 -- 172.21.2.101:0/3041670362 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49900d1e90 con 0x7f49900d1870 2021-02-10T11:42:15.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.016 7f499536b700 1 --2- 172.21.2.101:0/3041670362 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f49900d1870 0x7f49901b03f0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 rx=0x7f4984006700 tx=0x7f4984009b80).ready entity=mon.0 client_cookie=c6e4af4683cf1a62 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:15.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.016 7f4983fff700 1 -- 172.21.2.101:0/3041670362 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f498400c0c0 con 0x7f49900d1870 2021-02-10T11:42:15.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.016 7f4983fff700 1 -- 172.21.2.101:0/3041670362 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498400c440 con 0x7f49900d1870 2021-02-10T11:42:15.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.016 7f4983fff700 1 -- 172.21.2.101:0/3041670362 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f498400c960 con 0x7f49900d1870 2021-02-10T11:42:15.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.016 7f499754f700 1 -- 172.21.2.101:0/3041670362 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f49900d1870 msgr2=0x7f49901b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:15.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.016 7f499754f700 1 --2- 172.21.2.101:0/3041670362 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f49900d1870 0x7f49901b03f0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 rx=0x7f4984006700 tx=0x7f4984009b80).stop 2021-02-10T11:42:15.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.017 7f499754f700 1 -- 172.21.2.101:0/3041670362 shutdown_connections 2021-02-10T11:42:15.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.017 7f499754f700 1 --2- 172.21.2.101:0/3041670362 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f49900f0650 0x7f49900cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:15.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.017 7f499754f700 1 --2- 172.21.2.101:0/3041670362 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f49900d1870 0x7f49901b03f0 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:15.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.017 7f499754f700 1 -- 172.21.2.101:0/3041670362 >> 172.21.2.101:0/3041670362 conn(0x7f49900ce450 msgr2=0x7f49900d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:15.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.017 7f499754f700 1 -- 172.21.2.101:0/3041670362 shutdown_connections 2021-02-10T11:42:15.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.017 7f499754f700 1 -- 172.21.2.101:0/3041670362 wait complete. 2021-02-10T11:42:15.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.032 7f499754f700 1 Processor -- start 2021-02-10T11:42:15.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.032 7f499754f700 1 -- start start 2021-02-10T11:42:15.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.032 7f499754f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f49900b0ad0 0x7f4990091f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:15.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.032 7f499754f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f49901a8850 0x7f49901a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:15.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.032 7f499754f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f49901a6fa0 0x7f49901a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:15.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.032 7f499754f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49901abfe0 con 0x7f49900b0ad0 2021-02-10T11:42:15.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.032 7f499754f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49901a9c40 con 0x7f49901a8850 2021-02-10T11:42:15.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.032 7f499754f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49901a9730 con 0x7f49901a6fa0 2021-02-10T11:42:15.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.032 7f499536b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f49900b0ad0 0x7f4990091f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:15.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.032 7f499536b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f49900b0ad0 0x7f4990091f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57894/0 (socket says 172.21.2.101:57894) 2021-02-10T11:42:15.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.033 7f499536b700 1 -- 172.21.2.101:0/3962645871 learned_addr learned my addr 172.21.2.101:0/3962645871 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:15.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.033 7f499536b700 1 -- 172.21.2.101:0/3962645871 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f49901a6fa0 msgr2=0x7f49901a73a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:42:15.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.033 7f499536b700 1 --2- 172.21.2.101:0/3962645871 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f49901a6fa0 0x7f49901a73a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:15.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.033 7f499536b700 1 -- 172.21.2.101:0/3962645871 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f49901a8850 msgr2=0x7f49901a8c50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:42:15.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.033 7f499536b700 1 --2- 172.21.2.101:0/3962645871 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f49901a8850 0x7f49901a8c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:15.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.033 7f499536b700 1 -- 172.21.2.101:0/3962645871 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49901a9270 con 0x7f49900b0ad0 2021-02-10T11:42:15.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.033 7f499536b700 1 --2- 172.21.2.101:0/3962645871 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f49900b0ad0 0x7f4990091f30 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 rx=0x7f498400b5e0 tx=0x7f498400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:15.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.033 7f4981ffb700 1 -- 172.21.2.101:0/3962645871 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f498400bd70 con 0x7f49900b0ad0 2021-02-10T11:42:15.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.033 7f4981ffb700 1 -- 172.21.2.101:0/3962645871 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4984002030 con 0x7f49900b0ad0 2021-02-10T11:42:15.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.033 7f4981ffb700 1 -- 172.21.2.101:0/3962645871 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4984002550 con 0x7f49900b0ad0 2021-02-10T11:42:15.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.033 7f499754f700 1 -- 172.21.2.101:0/3962645871 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49901a9270 con 0x7f49900b0ad0 2021-02-10T11:42:15.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.033 7f499754f700 1 -- 172.21.2.101:0/3962645871 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49901a7cb0 con 0x7f49900b0ad0 2021-02-10T11:42:15.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.034 7f4981ffb700 1 -- 172.21.2.101:0/3962645871 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4984000c90 con 0x7f49900b0ad0 2021-02-10T11:42:15.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.034 7f4981ffb700 1 --2- 172.21.2.101:0/3962645871 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f49700186c0 0x7f497001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:15.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.034 7f4981ffb700 1 -- 172.21.2.101:0/3962645871 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f49840325f0 con 0x7f49900b0ad0 2021-02-10T11:42:15.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.034 7f4994b6a700 1 --2- 172.21.2.101:0/3962645871 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f49700186c0 0x7f497001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:15.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.034 7f4994b6a700 1 --2- 172.21.2.101:0/3962645871 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f49700186c0 0x7f497001abc0 crc :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:15.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.065 7f499754f700 1 -- 172.21.2.101:0/3962645871 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49900ce450 con 0x7f49900b0ad0 2021-02-10T11:42:15.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.068 7f4981ffb700 1 -- 172.21.2.101:0/3962645871 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f49840009b0 con 0x7f49900b0ad0 2021-02-10T11:42:15.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.201 7f499754f700 1 -- 172.21.2.101:0/3962645871 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_non_clone_status"]} v 0) v1 -- 0x7f49900ce450 con 0x7f49900b0ad0 2021-02-10T11:42:15.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.583 7f4981ffb700 1 -- 172.21.2.101:0/3962645871 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_non_clone_status"]}]=0 v618) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4984000c90 con 0x7f49900b0ad0 2021-02-10T11:42:15.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.618 7f499754f700 1 -- 172.21.2.101:0/3962645871 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f49700186c0 msgr2=0x7f497001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:15.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.618 7f499754f700 1 --2- 172.21.2.101:0/3962645871 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f49700186c0 0x7f497001abc0 crc :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:15.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.618 7f499754f700 1 -- 172.21.2.101:0/3962645871 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f49900b0ad0 msgr2=0x7f4990091f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:15.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.618 7f499754f700 1 --2- 172.21.2.101:0/3962645871 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f49900b0ad0 0x7f4990091f30 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 rx=0x7f498400b5e0 tx=0x7f498400b980).stop 2021-02-10T11:42:15.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.619 7f499754f700 1 -- 172.21.2.101:0/3962645871 shutdown_connections 2021-02-10T11:42:15.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.619 7f499754f700 1 --2- 172.21.2.101:0/3962645871 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f49700186c0 0x7f497001abc0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:15.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.619 7f499754f700 1 --2- 172.21.2.101:0/3962645871 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f49901a6fa0 0x7f49901a73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:15.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.619 7f499754f700 1 --2- 172.21.2.101:0/3962645871 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f49901a8850 0x7f49901a8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:15.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.619 7f499754f700 1 --2- 172.21.2.101:0/3962645871 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f49900b0ad0 0x7f4990091f30 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:15.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.619 7f499754f700 1 -- 172.21.2.101:0/3962645871 >> 172.21.2.101:0/3962645871 conn(0x7f499019f8f0 msgr2=0x7f49901a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:15.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.619 7f499754f700 1 -- 172.21.2.101:0/3962645871 shutdown_connections 2021-02-10T11:42:15.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:15.619 7f499754f700 1 -- 172.21.2.101:0/3962645871 wait complete. 2021-02-10T11:42:15.629 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:42:16.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.042 7f1fe8783700 1 Processor -- start 2021-02-10T11:42:16.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.042 7f1fe8783700 1 -- start start 2021-02-10T11:42:16.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.043 7f1fe8783700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fe010aff0 0x7f1fe00cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:16.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.043 7f1fe8783700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe00fe1c0 con 0x7f1fe00d1870 2021-02-10T11:42:16.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.043 7f1fe8783700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe01b0cb0 con 0x7f1fe00d1c70 2021-02-10T11:42:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.043 7f1fe8783700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe0109a60 con 0x7f1fe010aff0 2021-02-10T11:42:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.043 7f1fe659f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f1fe00d1870 0x7f1fe01b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58396/0 (socket says 172.21.2.101:58396) 2021-02-10T11:42:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.043 7f1fe659f700 1 -- 172.21.2.101:0/1380571548 learned_addr learned my addr 172.21.2.101:0/1380571548 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.043 7f1fe6da0700 1 --2- 172.21.2.101:0/1380571548 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fe010aff0 0x7f1fe00cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.043 7f1fe559d700 1 -- 172.21.2.101:0/1380571548 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580899974 0 0) 0x7f1fd4001180 con 0x7f1fe00d1870 2021-02-10T11:42:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.044 7f1fe559d700 1 -- 172.21.2.101:0/1380571548 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fcc002fe0 con 0x7f1fe00d1870 2021-02-10T11:42:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.044 7f1fe559d700 1 -- 172.21.2.101:0/1380571548 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3773105759 0 0) 0x7f1fd4001180 con 0x7f1fe00d1870 2021-02-10T11:42:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.044 7f1fe559d700 1 -- 172.21.2.101:0/1380571548 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fe010aff0 msgr2=0x7f1fe00cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.044 7f1fe559d700 1 --2- 172.21.2.101:0/1380571548 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fe010aff0 0x7f1fe00cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.044 7f1fe559d700 1 -- 172.21.2.101:0/1380571548 >> v1:172.21.2.108:6789/0 conn(0x7f1fe00d1c70 legacy=0x7f1fe00cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.044 7f1fe559d700 1 -- 172.21.2.101:0/1380571548 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe010b3f0 con 0x7f1fe00d1870 2021-02-10T11:42:16.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.044 7f1fe559d700 1 -- 172.21.2.101:0/1380571548 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f1fd4001680 con 0x7f1fe00d1870 2021-02-10T11:42:16.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.044 7f1fe559d700 1 -- 172.21.2.101:0/1380571548 >> v1:172.21.2.101:6789/0 conn(0x7f1fe00d1870 legacy=0x7f1fe01b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:16.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.044 7f1fe559d700 1 --2- 172.21.2.101:0/1380571548 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fcc003ac0 0x7f1fcc005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:16.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.044 7f1fe559d700 1 --2- 172.21.2.101:0/1380571548 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1fcc006790 0x7f1fcc008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:16.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.044 7f1fe559d700 1 --2- 172.21.2.101:0/1380571548 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fcc009590 0x7f1fcc00ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:16.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.044 7f1fe559d700 1 -- 172.21.2.101:0/1380571548 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc00c2d0 con 0x7f1fcc003ac0 2021-02-10T11:42:16.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.044 7f1fe559d700 1 -- 172.21.2.101:0/1380571548 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc00c4f0 con 0x7f1fcc006790 2021-02-10T11:42:16.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.044 7f1fe559d700 1 -- 172.21.2.101:0/1380571548 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc00c710 con 0x7f1fcc009590 2021-02-10T11:42:16.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.044 7f1fe659f700 1 --2- 172.21.2.101:0/1380571548 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fcc003ac0 0x7f1fcc005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:16.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.044 7f1fe5d9e700 1 --2- 172.21.2.101:0/1380571548 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1fcc006790 0x7f1fcc008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:16.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.044 7f1fe6da0700 1 --2- 172.21.2.101:0/1380571548 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fcc009590 0x7f1fcc00ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:16.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.044 7f1fe659f700 1 -- 172.21.2.101:0/1380571548 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fcc009590 msgr2=0x7f1fcc00ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:16.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.044 7f1fe659f700 1 --2- 172.21.2.101:0/1380571548 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fcc009590 0x7f1fcc00ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:16.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.044 7f1fe659f700 1 -- 172.21.2.101:0/1380571548 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1fcc006790 msgr2=0x7f1fcc008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:16.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.044 7f1fe659f700 1 --2- 172.21.2.101:0/1380571548 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1fcc006790 0x7f1fcc008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:16.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.044 7f1fe659f700 1 -- 172.21.2.101:0/1380571548 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc050980 con 0x7f1fcc003ac0 2021-02-10T11:42:16.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.045 7f1fe659f700 1 --2- 172.21.2.101:0/1380571548 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fcc003ac0 0x7f1fcc005ef0 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 rx=0x7f1fd4007970 tx=0x7f1fd4007d10).ready entity=mon.0 client_cookie=32d8004aba972e22 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:16.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.045 7f1fe559d700 1 -- 172.21.2.101:0/1380571548 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1fd40070c0 con 0x7f1fcc003ac0 2021-02-10T11:42:16.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.045 7f1fe559d700 1 -- 172.21.2.101:0/1380571548 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd40043a0 con 0x7f1fcc003ac0 2021-02-10T11:42:16.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.045 7f1fe559d700 1 -- 172.21.2.101:0/1380571548 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1fd40048c0 con 0x7f1fcc003ac0 2021-02-10T11:42:16.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.045 7f1fe8783700 1 -- 172.21.2.101:0/1380571548 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fcc003ac0 msgr2=0x7f1fcc005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:16.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.045 7f1fe8783700 1 --2- 172.21.2.101:0/1380571548 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fcc003ac0 0x7f1fcc005ef0 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 rx=0x7f1fd4007970 tx=0x7f1fd4007d10).stop 2021-02-10T11:42:16.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.045 7f1fe6da0700 1 -- 172.21.2.101:0/1380571548 reap_dead start 2021-02-10T11:42:16.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.045 7f1fe8783700 1 -- 172.21.2.101:0/1380571548 shutdown_connections 2021-02-10T11:42:16.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.045 7f1fe8783700 1 -- 172.21.2.101:0/1380571548 >> 172.21.2.101:0/1380571548 conn(0x7f1fe00ce420 msgr2=0x7f1fe00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:16.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.046 7f1fe8783700 1 -- 172.21.2.101:0/1380571548 shutdown_connections 2021-02-10T11:42:16.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.046 7f1fe8783700 1 -- 172.21.2.101:0/1380571548 wait complete. 2021-02-10T11:42:16.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.060 7f1fe8783700 1 Processor -- start 2021-02-10T11:42:16.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.060 7f1fe8783700 1 -- start start 2021-02-10T11:42:16.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.061 7f1fe8783700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1fe0085bc0 0x7f1fe00863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:16.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.061 7f1fe8783700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fe00954c0 0x7f1fe0096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:16.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.061 7f1fe8783700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fe00958c0 0x7f1fe0096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:16.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.061 7f1fe8783700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe0095d80 con 0x7f1fe00954c0 2021-02-10T11:42:16.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.061 7f1fe8783700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe00bfa60 con 0x7f1fe0085bc0 2021-02-10T11:42:16.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.061 7f1fe8783700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe0085fc0 con 0x7f1fe00958c0 2021-02-10T11:42:16.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.061 7f1fe5d9e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fe00954c0 0x7f1fe0096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:16.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.061 7f1fe5d9e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fe00954c0 0x7f1fe0096300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57916/0 (socket says 172.21.2.101:57916) 2021-02-10T11:42:16.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.061 7f1fe5d9e700 1 -- 172.21.2.101:0/12360808 learned_addr learned my addr 172.21.2.101:0/12360808 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:16.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.061 7f1fe6da0700 1 --2- 172.21.2.101:0/12360808 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fe00958c0 0x7f1fe0096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:16.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.061 7f1fe659f700 1 --2- 172.21.2.101:0/12360808 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1fe0085bc0 0x7f1fe00863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:16.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.061 7f1fe5d9e700 1 -- 172.21.2.101:0/12360808 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fe00958c0 msgr2=0x7f1fe0096920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:16.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.061 7f1fe5d9e700 1 --2- 172.21.2.101:0/12360808 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fe00958c0 0x7f1fe0096920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:16.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.061 7f1fe5d9e700 1 -- 172.21.2.101:0/12360808 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1fe0085bc0 msgr2=0x7f1fe00863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:16.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.061 7f1fe5d9e700 1 --2- 172.21.2.101:0/12360808 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1fe0085bc0 0x7f1fe00863e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:16.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.061 7f1fe5d9e700 1 -- 172.21.2.101:0/12360808 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe0086c10 con 0x7f1fe00954c0 2021-02-10T11:42:16.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.061 7f1fe6da0700 1 --2- 172.21.2.101:0/12360808 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fe00958c0 0x7f1fe0096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:42:16.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.061 7f1fe5d9e700 1 --2- 172.21.2.101:0/12360808 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fe00954c0 0x7f1fe0096300 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 rx=0x7f1fdc00b610 tx=0x7f1fdc00b9b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:16.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.062 7f1fd2ffd700 1 -- 172.21.2.101:0/12360808 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1fdc0063c0 con 0x7f1fe00954c0 2021-02-10T11:42:16.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.062 7f1fd2ffd700 1 -- 172.21.2.101:0/12360808 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fdc000960 con 0x7f1fe00954c0 2021-02-10T11:42:16.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.062 7f1fd2ffd700 1 -- 172.21.2.101:0/12360808 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1fdc002030 con 0x7f1fe00954c0 2021-02-10T11:42:16.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.062 7f1fe8783700 1 -- 172.21.2.101:0/12360808 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe0086c10 con 0x7f1fe00954c0 2021-02-10T11:42:16.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.062 7f1fe8783700 1 -- 172.21.2.101:0/12360808 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe0097230 con 0x7f1fe00954c0 2021-02-10T11:42:16.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.063 7f1fd2ffd700 1 -- 172.21.2.101:0/12360808 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1fdc000ca0 con 0x7f1fe00954c0 2021-02-10T11:42:16.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.063 7f1fd2ffd700 1 --2- 172.21.2.101:0/12360808 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1fc80186c0 0x7f1fc801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:16.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.063 7f1fd2ffd700 1 -- 172.21.2.101:0/12360808 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1fdc032570 con 0x7f1fe00954c0 2021-02-10T11:42:16.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.063 7f1fe659f700 1 --2- 172.21.2.101:0/12360808 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1fc80186c0 0x7f1fc801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:16.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.063 7f1fe659f700 1 --2- 172.21.2.101:0/12360808 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1fc80186c0 0x7f1fc801abc0 crc :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:16.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.093 7f1fe8783700 1 -- 172.21.2.101:0/12360808 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe00fe1c0 con 0x7f1fe00954c0 2021-02-10T11:42:16.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.096 7f1fd2ffd700 1 -- 172.21.2.101:0/12360808 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1fdc030d20 con 0x7f1fe00954c0 2021-02-10T11:42:16.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.245 7f1fe8783700 1 -- 172.21.2.101:0/12360808 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1fe00fe1c0 con 0x7f1fe00954c0 2021-02-10T11:42:16.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.249 7f1fd2ffd700 1 -- 172.21.2.101:0/12360808 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1fdc000ca0 con 0x7f1fe00954c0 2021-02-10T11:42:16.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.254 7f1fe8783700 1 -- 172.21.2.101:0/12360808 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1fc80186c0 msgr2=0x7f1fc801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:16.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.254 7f1fe8783700 1 --2- 172.21.2.101:0/12360808 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1fc80186c0 0x7f1fc801abc0 crc :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:16.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.254 7f1fe8783700 1 -- 172.21.2.101:0/12360808 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fe00954c0 msgr2=0x7f1fe0096300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:16.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.254 7f1fe8783700 1 --2- 172.21.2.101:0/12360808 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fe00954c0 0x7f1fe0096300 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 rx=0x7f1fdc00b610 tx=0x7f1fdc00b9b0).stop 2021-02-10T11:42:16.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.254 7f1fe8783700 1 -- 172.21.2.101:0/12360808 shutdown_connections 2021-02-10T11:42:16.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.254 7f1fe8783700 1 --2- 172.21.2.101:0/12360808 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1fc80186c0 0x7f1fc801abc0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:16.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.254 7f1fe8783700 1 --2- 172.21.2.101:0/12360808 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fe00958c0 0x7f1fe0096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:16.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.254 7f1fe8783700 1 --2- 172.21.2.101:0/12360808 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fe00954c0 0x7f1fe0096300 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:16.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.254 7f1fe8783700 1 --2- 172.21.2.101:0/12360808 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1fe0085bc0 0x7f1fe00863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:16.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.254 7f1fe8783700 1 -- 172.21.2.101:0/12360808 >> 172.21.2.101:0/12360808 conn(0x7f1fe00bef00 msgr2=0x7f1fe009d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:16.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.254 7f1fe8783700 1 -- 172.21.2.101:0/12360808 shutdown_connections 2021-02-10T11:42:16.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:16.254 7f1fe8783700 1 -- 172.21.2.101:0/12360808 wait complete. 2021-02-10T11:42:16.264 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:42:16.265 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:42:16.610 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:42:16.955 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:42:16.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:16.971 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:42:16.971 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:42:17.308 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2021-02-10T11:42:17.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:42:17.308 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-02-10T11:42:17.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:17.308 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:42:17.308 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:42:17.643 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:42:17.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:42:17.643 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:42:17.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:17.644 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:42:17.644 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:42:17.979 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:42:17.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:42:17.979 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:42:17.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:17.979 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:42:17.980 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:42:18.315 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:42:18.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:42:18.315 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:42:18.316 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json-pretty 2021-02-10T11:42:18.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.727 7efd72efd700 1 Processor -- start 2021-02-10T11:42:18.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.727 7efd72efd700 1 -- start start 2021-02-10T11:42:18.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.727 7efd72efd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd6c0d0340 0x7efd6c0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:18.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.727 7efd72efd700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7efd6c0d1c70 0x7efd6c0efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:18.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.727 7efd72efd700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd6c0f0b10 con 0x7efd6c0d1870 2021-02-10T11:42:18.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.727 7efd72efd700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd6c0e5ed0 con 0x7efd6c0d1c70 2021-02-10T11:42:18.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.728 7efd72efd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c1b0be0 con 0x7efd6c0d0340 2021-02-10T11:42:18.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.728 7efd70d19700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7efd6c0d1870 0x7efd6c1b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58418/0 (socket says 172.21.2.101:58418) 2021-02-10T11:42:18.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.728 7efd70d19700 1 -- 172.21.2.101:0/1350014645 learned_addr learned my addr 172.21.2.101:0/1350014645 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:18.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.728 7efd6b7fe700 1 -- 172.21.2.101:0/1350014645 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146782212 0 0) 0x7efd540009b0 con 0x7efd6c0d1870 2021-02-10T11:42:18.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.728 7efd6b7fe700 1 -- 172.21.2.101:0/1350014645 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd58002fe0 con 0x7efd6c0d1870 2021-02-10T11:42:18.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.728 7efd6bfff700 1 --2- 172.21.2.101:0/1350014645 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd6c0d0340 0x7efd6c0ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:18.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.728 7efd7151a700 1 --2- 172.21.2.101:0/1350014645 >> v2:172.21.2.108:3300/0 conn(0x7efd6c0d1c70 0x7efd6c0efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:18.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.728 7efd6b7fe700 1 -- 172.21.2.101:0/1350014645 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1383782980 0 0) 0x7efd54001010 con 0x7efd6c0d1870 2021-02-10T11:42:18.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.728 7efd6b7fe700 1 -- 172.21.2.101:0/1350014645 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd6c0d0340 msgr2=0x7efd6c0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:18.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.728 7efd6b7fe700 1 --2- 172.21.2.101:0/1350014645 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd6c0d0340 0x7efd6c0ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:18.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.728 7efd6b7fe700 1 -- 172.21.2.101:0/1350014645 >> v2:172.21.2.108:3300/0 conn(0x7efd6c0d1c70 msgr2=0x7efd6c0efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:18.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.728 7efd6b7fe700 1 --2- 172.21.2.101:0/1350014645 >> v2:172.21.2.108:3300/0 conn(0x7efd6c0d1c70 0x7efd6c0efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:18.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.728 7efd6b7fe700 1 -- 172.21.2.101:0/1350014645 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6c0f20a0 con 0x7efd6c0d1870 2021-02-10T11:42:18.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.728 7efd6b7fe700 1 -- 172.21.2.101:0/1350014645 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7efd54001590 con 0x7efd6c0d1870 2021-02-10T11:42:18.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.728 7efd6b7fe700 1 -- 172.21.2.101:0/1350014645 >> v1:172.21.2.101:6789/0 conn(0x7efd6c0d1870 legacy=0x7efd6c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:18.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.728 7efd6b7fe700 1 --2- 172.21.2.101:0/1350014645 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd58003ae0 0x7efd58005f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:18.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.728 7efd6b7fe700 1 --2- 172.21.2.101:0/1350014645 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd58006770 0x7efd58008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:18.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.728 7efd6b7fe700 1 --2- 172.21.2.101:0/1350014645 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd5804d500 0x7efd5804fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:18.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.728 7efd6b7fe700 1 -- 172.21.2.101:0/1350014645 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd580502d0 con 0x7efd5804d500 2021-02-10T11:42:18.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.728 7efd6b7fe700 1 -- 172.21.2.101:0/1350014645 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd580504f0 con 0x7efd58006770 2021-02-10T11:42:18.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.728 7efd6b7fe700 1 -- 172.21.2.101:0/1350014645 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd58050710 con 0x7efd58003ae0 2021-02-10T11:42:18.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.729 7efd7151a700 1 --2- 172.21.2.101:0/1350014645 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd5804d500 0x7efd5804fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:18.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.729 7efd6bfff700 1 --2- 172.21.2.101:0/1350014645 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd58006770 0x7efd58008c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:18.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.729 7efd70d19700 1 --2- 172.21.2.101:0/1350014645 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd58003ae0 0x7efd58005f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:18.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.729 7efd7151a700 1 -- 172.21.2.101:0/1350014645 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd58003ae0 msgr2=0x7efd58005f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:18.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.729 7efd7151a700 1 --2- 172.21.2.101:0/1350014645 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd58003ae0 0x7efd58005f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:18.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.729 7efd7151a700 1 -- 172.21.2.101:0/1350014645 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd58006770 msgr2=0x7efd58008c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:18.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.729 7efd7151a700 1 --2- 172.21.2.101:0/1350014645 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd58006770 0x7efd58008c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:18.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.729 7efd7151a700 1 -- 172.21.2.101:0/1350014645 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd580509b0 con 0x7efd5804d500 2021-02-10T11:42:18.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.729 7efd7151a700 1 --2- 172.21.2.101:0/1350014645 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd5804d500 0x7efd5804fa20 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 rx=0x7efd640090f0 tx=0x7efd6400b050).ready entity=mon.0 client_cookie=93d38c86fc121733 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:18.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.729 7efd6b7fe700 1 -- 172.21.2.101:0/1350014645 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd64000f90 con 0x7efd5804d500 2021-02-10T11:42:18.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.729 7efd6b7fe700 1 -- 172.21.2.101:0/1350014645 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd64006240 con 0x7efd5804d500 2021-02-10T11:42:18.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.729 7efd6b7fe700 1 -- 172.21.2.101:0/1350014645 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd640056b0 con 0x7efd5804d500 2021-02-10T11:42:18.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.729 7efd72efd700 1 -- 172.21.2.101:0/1350014645 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd5804d500 msgr2=0x7efd5804fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:18.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.729 7efd72efd700 1 --2- 172.21.2.101:0/1350014645 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd5804d500 0x7efd5804fa20 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 rx=0x7efd640090f0 tx=0x7efd6400b050).stop 2021-02-10T11:42:18.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.729 7efd7151a700 1 -- 172.21.2.101:0/1350014645 reap_dead start 2021-02-10T11:42:18.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.730 7efd72efd700 1 -- 172.21.2.101:0/1350014645 shutdown_connections 2021-02-10T11:42:18.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.730 7efd72efd700 1 -- 172.21.2.101:0/1350014645 >> 172.21.2.101:0/1350014645 conn(0x7efd6c0ce450 msgr2=0x7efd6c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:18.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.730 7efd72efd700 1 -- 172.21.2.101:0/1350014645 shutdown_connections 2021-02-10T11:42:18.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.730 7efd72efd700 1 -- 172.21.2.101:0/1350014645 wait complete. 2021-02-10T11:42:18.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.745 7efd72efd700 1 Processor -- start 2021-02-10T11:42:18.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.745 7efd72efd700 1 -- start start 2021-02-10T11:42:18.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.745 7efd72efd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd6c1942b0 0x7efd6c0b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:18.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.745 7efd72efd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd6c195070 0x7efd6c195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:18.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.745 7efd72efd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd6c0bb8e0 0x7efd6c0bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:18.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.745 7efd72efd700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c1946b0 con 0x7efd6c195070 2021-02-10T11:42:18.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.745 7efd72efd700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c0b3b70 con 0x7efd6c1942b0 2021-02-10T11:42:18.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.745 7efd72efd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c0b3d90 con 0x7efd6c0bb8e0 2021-02-10T11:42:18.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.745 7efd6bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd6c195070 0x7efd6c195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:18.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.745 7efd6bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd6c195070 0x7efd6c195470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57936/0 (socket says 172.21.2.101:57936) 2021-02-10T11:42:18.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.745 7efd6bfff700 1 -- 172.21.2.101:0/578975490 learned_addr learned my addr 172.21.2.101:0/578975490 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:18.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.746 7efd7151a700 1 --2- 172.21.2.101:0/578975490 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd6c0bb8e0 0x7efd6c0bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:18.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.746 7efd70d19700 1 --2- 172.21.2.101:0/578975490 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd6c1942b0 0x7efd6c0b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:18.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.746 7efd6bfff700 1 -- 172.21.2.101:0/578975490 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd6c0bb8e0 msgr2=0x7efd6c0bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:18.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.746 7efd6bfff700 1 --2- 172.21.2.101:0/578975490 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd6c0bb8e0 0x7efd6c0bbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:18.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.746 7efd6bfff700 1 -- 172.21.2.101:0/578975490 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd6c1942b0 msgr2=0x7efd6c0b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:18.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.746 7efd6bfff700 1 --2- 172.21.2.101:0/578975490 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd6c1942b0 0x7efd6c0b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:18.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.746 7efd6bfff700 1 -- 172.21.2.101:0/578975490 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6c1a49e0 con 0x7efd6c195070 2021-02-10T11:42:18.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.746 7efd6bfff700 1 --2- 172.21.2.101:0/578975490 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd6c195070 0x7efd6c195470 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 rx=0x7efd5c006f60 tx=0x7efd5c00b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:18.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.746 7efd697fa700 1 -- 172.21.2.101:0/578975490 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd5c00bd60 con 0x7efd6c195070 2021-02-10T11:42:18.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.746 7efd697fa700 1 -- 172.21.2.101:0/578975490 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5c000c90 con 0x7efd6c195070 2021-02-10T11:42:18.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.746 7efd697fa700 1 -- 172.21.2.101:0/578975490 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd5c002220 con 0x7efd6c195070 2021-02-10T11:42:18.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.746 7efd72efd700 1 -- 172.21.2.101:0/578975490 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd6c1a49e0 con 0x7efd6c195070 2021-02-10T11:42:18.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.746 7efd72efd700 1 -- 172.21.2.101:0/578975490 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6c1a49e0 con 0x7efd6c195070 2021-02-10T11:42:18.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.747 7efd697fa700 1 -- 172.21.2.101:0/578975490 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7efd5c006b00 con 0x7efd6c195070 2021-02-10T11:42:18.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.747 7efd697fa700 1 --2- 172.21.2.101:0/578975490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd500186d0 0x7efd5001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:18.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.747 7efd697fa700 1 -- 172.21.2.101:0/578975490 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7efd5c0327f0 con 0x7efd6c195070 2021-02-10T11:42:18.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.747 7efd70d19700 1 --2- 172.21.2.101:0/578975490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd500186d0 0x7efd5001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:18.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.747 7efd70d19700 1 --2- 172.21.2.101:0/578975490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd500186d0 0x7efd5001abd0 crc :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:18.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.777 7efd72efd700 1 -- 172.21.2.101:0/578975490 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd6c0bc5f0 con 0x7efd6c195070 2021-02-10T11:42:18.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.780 7efd697fa700 1 -- 172.21.2.101:0/578975490 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7efd5c01f530 con 0x7efd6c195070 2021-02-10T11:42:18.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.903 7efd72efd700 1 -- 172.21.2.101:0/578975490 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7efd6c0bc5f0 con 0x7efd6c195070 2021-02-10T11:42:18.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.904 7efd697fa700 1 -- 172.21.2.101:0/578975490 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v120) v1 ==== 81+0+30374 (secure 0 0 0) 0x7efd5c006b00 con 0x7efd6c195070 2021-02-10T11:42:18.906 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:42:18.906 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:42:18.907 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 120, 2021-02-10T11:42:18.907 INFO:teuthology.orchestra.run.gibba001.stdout: "fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:42:18.907 INFO:teuthology.orchestra.run.gibba001.stdout: "created": "2021-02-10 11:12:36.788366", 2021-02-10T11:42:18.907 INFO:teuthology.orchestra.run.gibba001.stdout: "modified": "2021-02-10 11:41:35.703752", 2021-02-10T11:42:18.907 INFO:teuthology.orchestra.run.gibba001.stdout: "last_up_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:42:18.907 INFO:teuthology.orchestra.run.gibba001.stdout: "last_in_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:42:18.907 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-02-10T11:42:18.907 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_num": 5799936, 2021-02-10T11:42:18.907 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_set": [ 2021-02-10T11:42:18.907 INFO:teuthology.orchestra.run.gibba001.stdout: "pglog_hardlimit", 2021-02-10T11:42:18.907 INFO:teuthology.orchestra.run.gibba001.stdout: "purged_snapdirs", 2021-02-10T11:42:18.907 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_deletes", 2021-02-10T11:42:18.908 INFO:teuthology.orchestra.run.gibba001.stdout: "sortbitwise" 2021-02-10T11:42:18.908 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:42:18.908 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_version": 4, 2021-02-10T11:42:18.908 INFO:teuthology.orchestra.run.gibba001.stdout: "full_ratio": 0.94999998807907104, 2021-02-10T11:42:18.908 INFO:teuthology.orchestra.run.gibba001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-02-10T11:42:18.908 INFO:teuthology.orchestra.run.gibba001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-02-10T11:42:18.908 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_snapshot": "", 2021-02-10T11:42:18.908 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_max": 25, 2021-02-10T11:42:18.908 INFO:teuthology.orchestra.run.gibba001.stdout: "max_osd": 8, 2021-02-10T11:42:18.908 INFO:teuthology.orchestra.run.gibba001.stdout: "require_min_compat_client": "jewel", 2021-02-10T11:42:18.909 INFO:teuthology.orchestra.run.gibba001.stdout: "min_compat_client": "jewel", 2021-02-10T11:42:18.909 INFO:teuthology.orchestra.run.gibba001.stdout: "require_osd_release": "nautilus", 2021-02-10T11:42:18.909 INFO:teuthology.orchestra.run.gibba001.stdout: "pools": [ 2021-02-10T11:42:18.909 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.909 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 1, 2021-02-10T11:42:18.909 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "rbd", 2021-02-10T11:42:18.909 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:13:08.037472", 2021-02-10T11:42:18.909 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:42:18.909 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:42:18.909 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:42:18.909 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:42:18.909 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:42:18.909 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:42:18.909 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:42:18.910 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:42:18.910 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:42:18.910 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:42:18.910 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:42:18.910 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:42:18.910 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:42:18.910 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:42:18.910 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:42:18.910 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:42:18.910 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:42:18.910 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:42:18.911 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:42:18.911 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:42:18.911 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.911 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "16", 2021-02-10T11:42:18.911 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:42:18.911 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:42:18.911 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:42:18.911 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:42:18.911 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:42:18.911 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:42:18.911 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:42:18.911 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:42:18.912 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:42:18.912 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:42:18.912 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:42:18.912 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:42:18.912 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:42:18.912 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:42:18.912 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:42:18.912 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:42:18.912 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:42:18.912 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:42:18.912 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:42:18.912 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:42:18.912 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:42:18.913 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:42:18.913 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:42:18.913 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:42:18.913 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:42:18.913 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:42:18.913 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.913 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:42:18.913 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:42:18.913 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:42:18.913 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:42:18.913 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:42:18.913 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:42:18.913 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:42:18.914 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:42:18.914 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:42:18.914 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:42:18.914 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:42:18.914 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:42:18.914 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:42:18.914 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd": {} 2021-02-10T11:42:18.914 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.914 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.914 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.914 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 24, 2021-02-10T11:42:18.914 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_metadata", 2021-02-10T11:42:18.914 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:41:32.676408", 2021-02-10T11:42:18.915 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:42:18.915 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:42:18.915 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:42:18.915 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:42:18.915 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:42:18.915 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:42:18.915 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:42:18.915 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:42:18.915 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:42:18.915 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:42:18.915 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:42:18.915 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:42:18.915 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:42:18.916 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:42:18.916 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:42:18.916 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:42:18.916 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:42:18.916 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:42:18.916 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:42:18.916 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:42:18.916 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.916 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "119", 2021-02-10T11:42:18.916 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:42:18.916 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:42:18.916 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:42:18.916 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:42:18.917 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:42:18.917 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:42:18.917 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:42:18.917 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:42:18.917 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:42:18.917 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:42:18.917 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:42:18.917 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:42:18.917 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:42:18.917 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:42:18.917 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:42:18.917 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:42:18.917 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:42:18.918 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:42:18.918 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:42:18.918 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:42:18.918 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:42:18.918 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:42:18.918 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:42:18.918 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:42:18.918 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:42:18.918 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:42:18.918 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.918 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:42:18.918 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:42:18.919 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:42:18.919 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:42:18.919 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:42:18.919 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:42:18.919 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:42:18.919 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:42:18.919 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:42:18.919 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:42:18.919 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:42:18.919 INFO:teuthology.orchestra.run.gibba001.stdout: "options": { 2021-02-10T11:42:18.919 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_bias": 4, 2021-02-10T11:42:18.919 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_min": 16, 2021-02-10T11:42:18.919 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_priority": 5 2021-02-10T11:42:18.920 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.920 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:42:18.920 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:42:18.920 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": "cephfs" 2021-02-10T11:42:18.920 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.920 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.920 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.920 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.920 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 25, 2021-02-10T11:42:18.920 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_data", 2021-02-10T11:42:18.920 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:41:33.747692", 2021-02-10T11:42:18.920 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:42:18.920 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:42:18.920 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:42:18.921 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:42:18.921 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:42:18.921 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:42:18.921 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:42:18.921 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:42:18.921 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:42:18.921 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:42:18.921 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:42:18.921 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:42:18.921 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:42:18.921 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:42:18.921 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:42:18.921 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:42:18.922 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:42:18.922 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:42:18.922 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:42:18.922 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:42:18.922 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.922 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "120", 2021-02-10T11:42:18.922 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:42:18.922 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:42:18.922 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:42:18.922 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:42:18.922 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:42:18.922 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:42:18.922 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:42:18.923 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:42:18.923 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:42:18.923 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:42:18.923 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:42:18.923 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:42:18.923 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:42:18.923 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:42:18.923 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:42:18.923 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:42:18.923 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:42:18.923 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:42:18.923 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:42:18.923 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:42:18.923 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:42:18.924 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:42:18.924 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:42:18.924 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:42:18.924 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:42:18.924 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:42:18.924 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.924 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:42:18.924 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:42:18.924 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:42:18.924 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:42:18.924 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:42:18.924 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:42:18.924 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:42:18.925 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:42:18.925 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:42:18.925 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:42:18.925 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:42:18.925 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:42:18.925 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:42:18.925 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:42:18.925 INFO:teuthology.orchestra.run.gibba001.stdout: "data": "cephfs" 2021-02-10T11:42:18.925 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.925 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.925 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.925 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:42:18.925 INFO:teuthology.orchestra.run.gibba001.stdout: "osds": [ 2021-02-10T11:42:18.926 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.926 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:42:18.926 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5", 2021-02-10T11:42:18.926 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:42:18.926 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:42:18.926 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:42:18.926 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:42:18.926 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:42:18.926 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:42:18.926 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:42:18.926 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 118, 2021-02-10T11:42:18.926 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:42:18.927 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:42:18.927 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:42:18.927 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.927 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.927 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.927 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6802", 2021-02-10T11:42:18.927 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:42:18.927 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.927 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.927 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.927 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6805", 2021-02-10T11:42:18.927 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:42:18.928 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.928 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.928 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.928 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:42:18.928 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.928 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.928 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.928 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6807", 2021-02-10T11:42:18.928 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:42:18.928 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.928 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.928 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.928 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6809", 2021-02-10T11:42:18.928 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:42:18.929 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.929 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.929 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.929 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:42:18.929 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.929 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.929 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.929 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6817", 2021-02-10T11:42:18.929 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:42:18.929 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.929 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.929 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.929 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6820", 2021-02-10T11:42:18.930 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:42:18.930 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.930 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.930 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.930 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:42:18.930 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.930 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.930 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.930 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6812", 2021-02-10T11:42:18.930 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:42:18.930 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.930 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.930 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.931 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6814", 2021-02-10T11:42:18.931 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:42:18.931 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.931 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.931 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.931 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6805/25093", 2021-02-10T11:42:18.931 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6809/25093", 2021-02-10T11:42:18.931 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6820/25093", 2021-02-10T11:42:18.931 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6814/25093", 2021-02-10T11:42:18.931 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:42:18.931 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:42:18.931 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:42:18.931 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.932 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.932 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.932 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:42:18.932 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "ddc04ce3-b313-4707-8270-585391e38ecc", 2021-02-10T11:42:18.932 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:42:18.932 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:42:18.932 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:42:18.932 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:42:18.932 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:42:18.932 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:42:18.932 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:42:18.932 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 117, 2021-02-10T11:42:18.932 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:42:18.932 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:42:18.933 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:42:18.933 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.933 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.933 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.933 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6803", 2021-02-10T11:42:18.933 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:42:18.933 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.933 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.933 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.933 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6804", 2021-02-10T11:42:18.933 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:42:18.933 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.933 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.934 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.934 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:42:18.934 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.934 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.934 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.934 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6806", 2021-02-10T11:42:18.934 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:42:18.934 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.934 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.934 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.934 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6808", 2021-02-10T11:42:18.934 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:42:18.935 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.935 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.935 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.935 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:42:18.935 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.935 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.935 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.935 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6818", 2021-02-10T11:42:18.935 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:42:18.935 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.935 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.935 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.936 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6821", 2021-02-10T11:42:18.936 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:42:18.936 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.936 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.936 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.936 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:42:18.936 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.936 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.936 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.936 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6810", 2021-02-10T11:42:18.936 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:42:18.936 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.936 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.937 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.937 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6815", 2021-02-10T11:42:18.937 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:42:18.937 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.937 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.937 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.937 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6804/25092", 2021-02-10T11:42:18.937 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6808/25092", 2021-02-10T11:42:18.937 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6821/25092", 2021-02-10T11:42:18.937 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6815/25092", 2021-02-10T11:42:18.937 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:42:18.937 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:42:18.937 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:42:18.938 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.938 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.938 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.938 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:42:18.938 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3", 2021-02-10T11:42:18.938 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:42:18.938 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:42:18.938 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:42:18.938 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:42:18.938 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:42:18.938 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:42:18.938 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:42:18.939 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 118, 2021-02-10T11:42:18.939 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:42:18.939 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:42:18.939 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:42:18.939 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.939 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.939 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.939 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6811", 2021-02-10T11:42:18.939 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:42:18.939 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.939 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.939 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.940 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6813", 2021-02-10T11:42:18.940 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:42:18.940 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.940 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.940 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.940 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:42:18.940 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.940 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.940 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.940 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6816", 2021-02-10T11:42:18.940 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:42:18.940 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.940 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.941 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.941 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6819", 2021-02-10T11:42:18.941 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:42:18.941 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.941 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.941 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.941 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:42:18.941 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.941 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.941 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.941 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6825", 2021-02-10T11:42:18.941 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:42:18.941 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.942 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.942 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.942 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6827", 2021-02-10T11:42:18.942 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:42:18.942 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.942 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.942 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.942 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:42:18.942 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.942 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.942 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.942 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6822", 2021-02-10T11:42:18.943 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:42:18.943 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.943 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.943 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.943 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6823", 2021-02-10T11:42:18.943 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:42:18.943 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.943 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.943 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.943 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6813/25094", 2021-02-10T11:42:18.943 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6819/25094", 2021-02-10T11:42:18.943 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6827/25094", 2021-02-10T11:42:18.943 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6823/25094", 2021-02-10T11:42:18.944 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:42:18.944 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:42:18.944 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:42:18.944 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.944 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.944 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.944 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:42:18.944 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "0d9ef9b1-98aa-4d37-aa16-6305c7d46d52", 2021-02-10T11:42:18.945 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:42:18.945 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:42:18.945 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:42:18.945 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:42:18.945 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:42:18.945 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:42:18.945 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:42:18.945 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 118, 2021-02-10T11:42:18.945 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:42:18.945 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:42:18.945 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:42:18.945 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.946 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.946 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.946 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6824", 2021-02-10T11:42:18.946 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:42:18.946 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.946 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.946 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.946 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6826", 2021-02-10T11:42:18.946 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:42:18.946 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.946 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.946 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.946 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:42:18.947 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.947 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.947 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.947 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6828", 2021-02-10T11:42:18.947 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:42:18.947 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.947 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.947 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.947 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6829", 2021-02-10T11:42:18.947 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:42:18.947 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.947 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.947 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.948 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:42:18.948 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.948 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.948 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.948 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6832", 2021-02-10T11:42:18.948 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:42:18.948 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.948 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.948 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.948 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6833", 2021-02-10T11:42:18.948 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:42:18.948 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.948 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.949 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.949 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:42:18.949 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.949 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.949 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.949 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6830", 2021-02-10T11:42:18.949 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:42:18.949 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.949 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.949 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.949 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6831", 2021-02-10T11:42:18.949 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:42:18.949 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.950 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.950 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.950 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6826/25095", 2021-02-10T11:42:18.950 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6829/25095", 2021-02-10T11:42:18.950 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6833/25095", 2021-02-10T11:42:18.950 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6831/25095", 2021-02-10T11:42:18.950 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:42:18.950 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:42:18.950 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:42:18.950 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.950 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.950 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.950 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:42:18.951 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "36742c21-bfeb-4e11-bce9-8e745378b605", 2021-02-10T11:42:18.951 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:42:18.951 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:42:18.951 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:42:18.951 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:42:18.951 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:42:18.951 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:42:18.951 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:42:18.951 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 110, 2021-02-10T11:42:18.951 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:42:18.951 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:42:18.951 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:42:18.951 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.952 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.952 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.952 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6816", 2021-02-10T11:42:18.952 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:42:18.952 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.952 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.952 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.952 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6818", 2021-02-10T11:42:18.952 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:42:18.952 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.952 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.952 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.952 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:42:18.952 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.953 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.953 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.953 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6820", 2021-02-10T11:42:18.953 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:42:18.953 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.953 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.953 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.953 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6822", 2021-02-10T11:42:18.954 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:42:18.954 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.954 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.954 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.954 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:42:18.954 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.954 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.954 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.954 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6828", 2021-02-10T11:42:18.954 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:42:18.955 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.955 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.955 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.955 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6830", 2021-02-10T11:42:18.955 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:42:18.955 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.955 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.955 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.955 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:42:18.955 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.955 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.955 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.956 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6824", 2021-02-10T11:42:18.956 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:42:18.956 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.956 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.956 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.956 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6826", 2021-02-10T11:42:18.956 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:42:18.956 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.956 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.956 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.956 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6818/36380", 2021-02-10T11:42:18.956 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6822/36380", 2021-02-10T11:42:18.956 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6830/36380", 2021-02-10T11:42:18.957 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6826/36380", 2021-02-10T11:42:18.957 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:42:18.957 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:42:18.957 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:42:18.957 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.957 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.957 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.957 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:42:18.957 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "88e694e8-fe98-4d1b-88c8-8c9907ece60b", 2021-02-10T11:42:18.957 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:42:18.957 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:42:18.957 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:42:18.957 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:42:18.958 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:42:18.958 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:42:18.958 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:42:18.958 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 117, 2021-02-10T11:42:18.958 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:42:18.958 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:42:18.958 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:42:18.958 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.958 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.958 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.958 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6808", 2021-02-10T11:42:18.958 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:42:18.958 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.958 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.959 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.959 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6809", 2021-02-10T11:42:18.959 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:42:18.959 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.959 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.959 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.959 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:42:18.959 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.959 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.959 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.959 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6810", 2021-02-10T11:42:18.959 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:42:18.959 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.960 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.960 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.960 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6811", 2021-02-10T11:42:18.960 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:42:18.960 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.960 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.960 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.960 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:42:18.960 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.960 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.960 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.960 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6814", 2021-02-10T11:42:18.960 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:42:18.960 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.961 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.961 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.961 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6815", 2021-02-10T11:42:18.961 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:42:18.961 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.961 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.961 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.961 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:42:18.961 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.961 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.962 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.962 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6812", 2021-02-10T11:42:18.962 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:42:18.962 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.962 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.962 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.962 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6813", 2021-02-10T11:42:18.962 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:42:18.962 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.962 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.962 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.963 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6809/36383", 2021-02-10T11:42:18.963 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6811/36383", 2021-02-10T11:42:18.963 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6815/36383", 2021-02-10T11:42:18.963 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6813/36383", 2021-02-10T11:42:18.963 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:42:18.963 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:42:18.963 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:42:18.963 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.963 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.963 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.963 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:42:18.963 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "87a13c6c-cf85-4aaf-916b-e73913484a49", 2021-02-10T11:42:18.963 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:42:18.964 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:42:18.964 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:42:18.964 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:42:18.964 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:42:18.964 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:42:18.964 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:42:18.964 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 118, 2021-02-10T11:42:18.964 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:42:18.964 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:42:18.964 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:42:18.964 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.964 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.964 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.964 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6800", 2021-02-10T11:42:18.965 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:42:18.965 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.965 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.965 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.965 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6801", 2021-02-10T11:42:18.965 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:42:18.965 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.965 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.965 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.965 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:42:18.965 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.965 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.965 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.966 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6802", 2021-02-10T11:42:18.966 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:42:18.966 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.966 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.966 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.966 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6803", 2021-02-10T11:42:18.966 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:42:18.966 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.966 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.966 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.966 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:42:18.966 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.967 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.967 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.967 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6806", 2021-02-10T11:42:18.967 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:42:18.967 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.967 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.967 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.967 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6807", 2021-02-10T11:42:18.967 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:42:18.967 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.967 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.967 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.967 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:42:18.967 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.968 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.968 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.968 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6804", 2021-02-10T11:42:18.968 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:42:18.968 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.968 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.968 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.968 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6805", 2021-02-10T11:42:18.968 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:42:18.968 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.968 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.968 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.968 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6801/36381", 2021-02-10T11:42:18.969 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6803/36381", 2021-02-10T11:42:18.969 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6807/36381", 2021-02-10T11:42:18.969 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6805/36381", 2021-02-10T11:42:18.969 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:42:18.969 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:42:18.969 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:42:18.969 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.969 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.969 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.969 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:42:18.969 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "7b90823f-854c-4592-a95b-a7fa98f93254", 2021-02-10T11:42:18.969 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:42:18.969 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:42:18.970 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:42:18.970 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:42:18.970 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:42:18.970 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:42:18.970 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:42:18.970 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 118, 2021-02-10T11:42:18.970 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:42:18.970 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:42:18.970 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:42:18.971 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.971 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.971 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.971 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6817", 2021-02-10T11:42:18.971 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:42:18.971 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.971 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.971 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.971 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6819", 2021-02-10T11:42:18.971 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:42:18.971 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.971 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.972 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.972 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:42:18.972 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.972 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.972 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.972 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6821", 2021-02-10T11:42:18.972 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:42:18.972 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.972 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.972 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.972 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6823", 2021-02-10T11:42:18.972 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:42:18.972 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.973 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.973 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.973 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:42:18.973 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.973 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.973 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.973 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6829", 2021-02-10T11:42:18.973 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:42:18.973 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.973 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.973 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.973 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6831", 2021-02-10T11:42:18.973 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:42:18.974 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.974 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.974 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.974 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:42:18.974 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:42:18.974 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.974 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:42:18.974 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6825", 2021-02-10T11:42:18.974 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:42:18.974 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.974 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.974 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:18.974 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6827", 2021-02-10T11:42:18.975 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:42:18.975 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.975 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.975 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.975 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6819/36382", 2021-02-10T11:42:18.975 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6823/36382", 2021-02-10T11:42:18.975 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6831/36382", 2021-02-10T11:42:18.975 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6827/36382", 2021-02-10T11:42:18.975 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:42:18.975 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:42:18.975 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:42:18.976 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:18.976 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.976 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:42:18.976 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_xinfo": [ 2021-02-10T11:42:18.976 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.976 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:42:18.976 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:42:18.976 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:42:18.976 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:42:18.976 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:42:18.976 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:42:18.976 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.976 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.977 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:42:18.977 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:42:18.977 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:42:18.977 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:42:18.977 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:42:18.977 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:42:18.977 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.977 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.977 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:42:18.977 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:42:18.977 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:42:18.977 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:42:18.977 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:42:18.977 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:42:18.978 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.978 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.978 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:42:18.978 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:42:18.978 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:42:18.978 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:42:18.978 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:42:18.978 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:42:18.978 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.978 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.978 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:42:18.978 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:42:18.978 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:42:18.979 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:42:18.979 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:42:18.979 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:42:18.979 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.979 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.979 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:42:18.983 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:42:18.983 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:42:18.983 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:42:18.983 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:42:18.983 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:42:18.983 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.984 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.984 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:42:18.984 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:42:18.984 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:42:18.984 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:42:18.984 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:42:18.984 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:42:18.984 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.984 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:18.984 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:42:18.984 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:42:18.984 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:42:18.984 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:42:18.985 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:42:18.985 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:42:18.985 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.985 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:42:18.985 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap": [], 2021-02-10T11:42:18.985 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap_items": [], 2021-02-10T11:42:18.985 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_temp": [], 2021-02-10T11:42:18.985 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_temp": [], 2021-02-10T11:42:18.985 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklist": {}, 2021-02-10T11:42:18.985 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profiles": { 2021-02-10T11:42:18.985 INFO:teuthology.orchestra.run.gibba001.stdout: "default": { 2021-02-10T11:42:18.985 INFO:teuthology.orchestra.run.gibba001.stdout: "crush-failure-domain": "osd", 2021-02-10T11:42:18.985 INFO:teuthology.orchestra.run.gibba001.stdout: "k": "2", 2021-02-10T11:42:18.985 INFO:teuthology.orchestra.run.gibba001.stdout: "m": "1", 2021-02-10T11:42:18.986 INFO:teuthology.orchestra.run.gibba001.stdout: "plugin": "jerasure", 2021-02-10T11:42:18.986 INFO:teuthology.orchestra.run.gibba001.stdout: "ruleset-failure-domain": "osd", 2021-02-10T11:42:18.986 INFO:teuthology.orchestra.run.gibba001.stdout: "technique": "reed_sol_van" 2021-02-10T11:42:18.986 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:18.986 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:18.986 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps_queue": [], 2021-02-10T11:42:18.986 INFO:teuthology.orchestra.run.gibba001.stdout: "new_removed_snaps": [], 2021-02-10T11:42:18.986 INFO:teuthology.orchestra.run.gibba001.stdout: "new_purged_snaps": [], 2021-02-10T11:42:18.986 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_node_flags": {}, 2021-02-10T11:42:18.986 INFO:teuthology.orchestra.run.gibba001.stdout: "device_class_flags": {} 2021-02-10T11:42:18.986 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:42:18.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.905 7efd72efd700 1 -- 172.21.2.101:0/578975490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd500186d0 msgr2=0x7efd5001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:18.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.905 7efd72efd700 1 --2- 172.21.2.101:0/578975490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd500186d0 0x7efd5001abd0 crc :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:18.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.905 7efd72efd700 1 -- 172.21.2.101:0/578975490 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd6c195070 msgr2=0x7efd6c195470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:18.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.905 7efd72efd700 1 --2- 172.21.2.101:0/578975490 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd6c195070 0x7efd6c195470 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 rx=0x7efd5c006f60 tx=0x7efd5c00b970).stop 2021-02-10T11:42:18.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.906 7efd72efd700 1 -- 172.21.2.101:0/578975490 shutdown_connections 2021-02-10T11:42:18.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.906 7efd72efd700 1 --2- 172.21.2.101:0/578975490 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd500186d0 0x7efd5001abd0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:18.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.906 7efd72efd700 1 --2- 172.21.2.101:0/578975490 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd6c0bb8e0 0x7efd6c0bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:18.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.906 7efd72efd700 1 --2- 172.21.2.101:0/578975490 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd6c195070 0x7efd6c195470 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:18.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.906 7efd72efd700 1 --2- 172.21.2.101:0/578975490 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd6c1942b0 0x7efd6c0b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:18.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.906 7efd72efd700 1 -- 172.21.2.101:0/578975490 >> 172.21.2.101:0/578975490 conn(0x7efd6c0b3040 msgr2=0x7efd6c0c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:18.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.906 7efd72efd700 1 -- 172.21.2.101:0/578975490 shutdown_connections 2021-02-10T11:42:18.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:18.906 7efd72efd700 1 -- 172.21.2.101:0/578975490 wait complete. 2021-02-10T11:42:18.989 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:42:19.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.399 7ff53c08a700 1 Processor -- start 2021-02-10T11:42:19.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.400 7ff53c08a700 1 -- start start 2021-02-10T11:42:19.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.400 7ff53c08a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5340d1c70 0x7ff5340f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:19.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.400 7ff53c08a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5340f0b10 con 0x7ff5340d0340 2021-02-10T11:42:19.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.400 7ff53c08a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5341b0be0 con 0x7ff5340d1870 2021-02-10T11:42:19.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.400 7ff53c08a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340e5ed0 con 0x7ff5340d1c70 2021-02-10T11:42:19.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.400 7ff5396a5700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff5340d0340 0x7ff5340ef640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58438/0 (socket says 172.21.2.101:58438) 2021-02-10T11:42:19.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.400 7ff5396a5700 1 -- 172.21.2.101:0/3569783056 learned_addr learned my addr 172.21.2.101:0/3569783056 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:19.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.400 7ff53a6a7700 1 --2- 172.21.2.101:0/3569783056 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5340d1c70 0x7ff5340f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:19.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.401 7ff538ea4700 1 -- 172.21.2.101:0/3569783056 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3368342525 0 0) 0x7ff5240009b0 con 0x7ff5340d0340 2021-02-10T11:42:19.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.401 7ff538ea4700 1 -- 172.21.2.101:0/3569783056 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff520002fe0 con 0x7ff5340d0340 2021-02-10T11:42:19.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.401 7ff538ea4700 1 -- 172.21.2.101:0/3569783056 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2065508767 0 0) 0x7ff52c000fe0 con 0x7ff5340d1870 2021-02-10T11:42:19.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.401 7ff538ea4700 1 -- 172.21.2.101:0/3569783056 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff520003210 con 0x7ff5340d1870 2021-02-10T11:42:19.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.401 7ff53a6a7700 1 -- 172.21.2.101:0/3569783056 >> v1:172.21.2.108:6789/0 conn(0x7ff5340d1870 legacy=0x7ff5341b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:19.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.401 7ff53a6a7700 1 -- 172.21.2.101:0/3569783056 >> v1:172.21.2.101:6789/0 conn(0x7ff5340d0340 legacy=0x7ff5340ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:19.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.401 7ff53a6a7700 1 -- 172.21.2.101:0/3569783056 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5340d2fe0 con 0x7ff5340d1c70 2021-02-10T11:42:19.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.401 7ff53a6a7700 1 --2- 172.21.2.101:0/3569783056 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5340d1c70 0x7ff5340f20a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7ff530006ad0 tx=0x7ff530009ad0).ready entity=mon.2 client_cookie=a02ffb78db900bd5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:19.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.401 7ff538ea4700 1 -- 172.21.2.101:0/3569783056 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff5300066a0 con 0x7ff5340d1c70 2021-02-10T11:42:19.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.401 7ff538ea4700 1 -- 172.21.2.101:0/3569783056 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff530006410 con 0x7ff5340d1c70 2021-02-10T11:42:19.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.401 7ff538ea4700 1 -- 172.21.2.101:0/3569783056 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff5300024d0 con 0x7ff5340d1c70 2021-02-10T11:42:19.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.401 7ff53c08a700 1 -- 172.21.2.101:0/3569783056 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5340d1c70 msgr2=0x7ff5340f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:19.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.401 7ff53c08a700 1 --2- 172.21.2.101:0/3569783056 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5340d1c70 0x7ff5340f20a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7ff530006ad0 tx=0x7ff530009ad0).stop 2021-02-10T11:42:19.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.402 7ff53c08a700 1 -- 172.21.2.101:0/3569783056 shutdown_connections 2021-02-10T11:42:19.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.402 7ff53c08a700 1 --2- 172.21.2.101:0/3569783056 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5340d1c70 0x7ff5340f20a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:19.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.402 7ff53c08a700 1 -- 172.21.2.101:0/3569783056 >> 172.21.2.101:0/3569783056 conn(0x7ff5340ce450 msgr2=0x7ff5340d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:19.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.402 7ff53c08a700 1 -- 172.21.2.101:0/3569783056 shutdown_connections 2021-02-10T11:42:19.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.402 7ff53c08a700 1 -- 172.21.2.101:0/3569783056 wait complete. 2021-02-10T11:42:19.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.417 7ff53c08a700 1 Processor -- start 2021-02-10T11:42:19.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.417 7ff53c08a700 1 -- start start 2021-02-10T11:42:19.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.417 7ff53c08a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5341b7cf0 0x7ff53419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:19.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.417 7ff53c08a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5341b80f0 0x7ff5341b84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:19.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.417 7ff53c08a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff53419da20 0x7ff53419de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:19.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.417 7ff53c08a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340a77c0 con 0x7ff5341b80f0 2021-02-10T11:42:19.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.417 7ff53c08a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53419d4a0 con 0x7ff5341b7cf0 2021-02-10T11:42:19.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.417 7ff53c08a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340954f0 con 0x7ff53419da20 2021-02-10T11:42:19.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.418 7ff5396a5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5341b80f0 0x7ff5341b84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:19.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.418 7ff5396a5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5341b80f0 0x7ff5341b84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57948/0 (socket says 172.21.2.101:57948) 2021-02-10T11:42:19.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.418 7ff5396a5700 1 -- 172.21.2.101:0/3471999664 learned_addr learned my addr 172.21.2.101:0/3471999664 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:19.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.418 7ff539ea6700 1 --2- 172.21.2.101:0/3471999664 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5341b7cf0 0x7ff53419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:19.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.418 7ff53a6a7700 1 --2- 172.21.2.101:0/3471999664 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff53419da20 0x7ff53419de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:19.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.418 7ff5396a5700 1 -- 172.21.2.101:0/3471999664 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff53419da20 msgr2=0x7ff53419de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:19.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.418 7ff5396a5700 1 --2- 172.21.2.101:0/3471999664 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff53419da20 0x7ff53419de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:19.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.418 7ff5396a5700 1 -- 172.21.2.101:0/3471999664 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5341b7cf0 msgr2=0x7ff53419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:19.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.418 7ff5396a5700 1 --2- 172.21.2.101:0/3471999664 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5341b7cf0 0x7ff53419cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:19.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.418 7ff5396a5700 1 -- 172.21.2.101:0/3471999664 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5341b8d60 con 0x7ff5341b80f0 2021-02-10T11:42:19.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.418 7ff5396a5700 1 --2- 172.21.2.101:0/3471999664 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5341b80f0 0x7ff5341b84f0 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 rx=0x7ff524001c60 tx=0x7ff52400a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:19.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.418 7ff52a7fc700 1 -- 172.21.2.101:0/3471999664 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff52400acc0 con 0x7ff5341b80f0 2021-02-10T11:42:19.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.418 7ff52a7fc700 1 -- 172.21.2.101:0/3471999664 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff524004140 con 0x7ff5341b80f0 2021-02-10T11:42:19.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.418 7ff53c08a700 1 -- 172.21.2.101:0/3471999664 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5341b8d60 con 0x7ff5341b80f0 2021-02-10T11:42:19.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.418 7ff52a7fc700 1 -- 172.21.2.101:0/3471999664 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff5240045a0 con 0x7ff5341b80f0 2021-02-10T11:42:19.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.419 7ff53c08a700 1 -- 172.21.2.101:0/3471999664 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5341b8d60 con 0x7ff5341b80f0 2021-02-10T11:42:19.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.419 7ff52a7fc700 1 -- 172.21.2.101:0/3471999664 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff524007f30 con 0x7ff5341b80f0 2021-02-10T11:42:19.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.419 7ff52a7fc700 1 --2- 172.21.2.101:0/3471999664 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff51c018730 0x7ff51c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:19.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.419 7ff52a7fc700 1 -- 172.21.2.101:0/3471999664 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff524007f30 con 0x7ff5341b80f0 2021-02-10T11:42:19.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.419 7ff539ea6700 1 --2- 172.21.2.101:0/3471999664 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff51c018730 0x7ff51c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:19.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.419 7ff539ea6700 1 --2- 172.21.2.101:0/3471999664 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff51c018730 0x7ff51c01ac10 crc :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:19.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.450 7ff53c08a700 1 -- 172.21.2.101:0/3471999664 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5340ebc00 con 0x7ff5341b80f0 2021-02-10T11:42:19.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.452 7ff52a7fc700 1 -- 172.21.2.101:0/3471999664 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff524000940 con 0x7ff5341b80f0 2021-02-10T11:42:19.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.582 7ff53c08a700 1 -- 172.21.2.101:0/3471999664 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5340f0b10 con 0x7ff5341b80f0 2021-02-10T11:42:19.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.583 7ff52a7fc700 1 -- 172.21.2.101:0/3471999664 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+2101 (secure 0 0 0) 0x7ff524007f30 con 0x7ff5341b80f0 2021-02-10T11:42:19.585 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:42:19.585 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":97,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/654364385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":654364385},{"type":"v1","addr":"172.21.2.101:6835","nonce":654364385}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":92}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:41:34.701339","modified":"2021-02-10 11:41:36.711148","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6581":{"gid":6581,"name":"b","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2146451837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2146451837},{"type":"v1","addr":"172.21.2.108:6833","nonce":2146451837}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2021-02-10T11:42:19.585 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 97 2021-02-10T11:42:19.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.585 7ff53c08a700 1 -- 172.21.2.101:0/3471999664 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff51c018730 msgr2=0x7ff51c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:19.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.585 7ff53c08a700 1 --2- 172.21.2.101:0/3471999664 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff51c018730 0x7ff51c01ac10 crc :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:19.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.585 7ff53c08a700 1 -- 172.21.2.101:0/3471999664 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5341b80f0 msgr2=0x7ff5341b84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:19.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.585 7ff53c08a700 1 --2- 172.21.2.101:0/3471999664 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5341b80f0 0x7ff5341b84f0 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 rx=0x7ff524001c60 tx=0x7ff52400a8d0).stop 2021-02-10T11:42:19.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.585 7ff53c08a700 1 -- 172.21.2.101:0/3471999664 shutdown_connections 2021-02-10T11:42:19.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.585 7ff53c08a700 1 --2- 172.21.2.101:0/3471999664 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff51c018730 0x7ff51c01ac10 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:19.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.585 7ff53c08a700 1 --2- 172.21.2.101:0/3471999664 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff53419da20 0x7ff53419de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:19.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.585 7ff53c08a700 1 --2- 172.21.2.101:0/3471999664 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5341b80f0 0x7ff5341b84f0 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:19.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.585 7ff53c08a700 1 --2- 172.21.2.101:0/3471999664 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5341b7cf0 0x7ff53419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:19.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.585 7ff53c08a700 1 -- 172.21.2.101:0/3471999664 >> 172.21.2.101:0/3471999664 conn(0x7ff5340ed810 msgr2=0x7ff534094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:19.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.585 7ff53c08a700 1 -- 172.21.2.101:0/3471999664 shutdown_connections 2021-02-10T11:42:19.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:19.585 7ff53c08a700 1 -- 172.21.2.101:0/3471999664 wait complete. 2021-02-10T11:42:19.596 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs fail cephfs 2021-02-10T11:42:20.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.004 7f9373490700 1 Processor -- start 2021-02-10T11:42:20.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.004 7f9373490700 1 -- start start 2021-02-10T11:42:20.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.004 7f9373490700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f936c0d1c60 0x7f936c0f2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:20.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.004 7f9373490700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936c0f0b00 con 0x7f936c0d0330 2021-02-10T11:42:20.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.004 7f9373490700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936c1b0bd0 con 0x7f936c0d1860 2021-02-10T11:42:20.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.004 7f9373490700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936c0e5ec0 con 0x7f936c0d1c60 2021-02-10T11:42:20.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.005 7f9370aab700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f936c0d0330 0x7f936c0ef630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58450/0 (socket says 172.21.2.101:58450) 2021-02-10T11:42:20.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.005 7f9370aab700 1 -- 172.21.2.101:0/669787222 learned_addr learned my addr 172.21.2.101:0/669787222 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:20.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.005 7f935ffff700 1 -- 172.21.2.101:0/669787222 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 387935397 0 0) 0x7f93680010b0 con 0x7f936c0d0330 2021-02-10T11:42:20.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.005 7f935ffff700 1 -- 172.21.2.101:0/669787222 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9358002fe0 con 0x7f936c0d0330 2021-02-10T11:42:20.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.005 7f9371aad700 1 --2- 172.21.2.101:0/669787222 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f936c0d1c60 0x7f936c0f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:20.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.005 7f935ffff700 1 -- 172.21.2.101:0/669787222 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1693515806 0 0) 0x7f93680010b0 con 0x7f936c0d0330 2021-02-10T11:42:20.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.005 7f935ffff700 1 -- 172.21.2.101:0/669787222 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f936c0d1c60 msgr2=0x7f936c0f2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:20.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.005 7f935ffff700 1 --2- 172.21.2.101:0/669787222 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f936c0d1c60 0x7f936c0f2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:20.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.005 7f935ffff700 1 -- 172.21.2.101:0/669787222 >> v1:172.21.2.108:6789/0 conn(0x7f936c0d1860 legacy=0x7f936c1b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:20.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.005 7f935ffff700 1 -- 172.21.2.101:0/669787222 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c0efe20 con 0x7f936c0d0330 2021-02-10T11:42:20.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.005 7f935ffff700 1 -- 172.21.2.101:0/669787222 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f93680015b0 con 0x7f936c0d0330 2021-02-10T11:42:20.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.005 7f935ffff700 1 -- 172.21.2.101:0/669787222 >> v1:172.21.2.101:6789/0 conn(0x7f936c0d0330 legacy=0x7f936c0ef630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:20.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.005 7f935ffff700 1 --2- 172.21.2.101:0/669787222 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9358003aa0 0x7f9358005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:20.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.005 7f935ffff700 1 --2- 172.21.2.101:0/669787222 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9358006800 0x7f9358008d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:20.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.006 7f935ffff700 1 --2- 172.21.2.101:0/669787222 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9358009620 0x7f935800bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:20.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.006 7f935ffff700 1 -- 172.21.2.101:0/669787222 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935800c2e0 con 0x7f9358003aa0 2021-02-10T11:42:20.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.006 7f935ffff700 1 -- 172.21.2.101:0/669787222 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935800c500 con 0x7f9358006800 2021-02-10T11:42:20.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.006 7f935ffff700 1 -- 172.21.2.101:0/669787222 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935800c720 con 0x7f9358009620 2021-02-10T11:42:20.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.006 7f93712ac700 1 --2- 172.21.2.101:0/669787222 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9358003aa0 0x7f9358005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:20.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.006 7f9371aad700 1 --2- 172.21.2.101:0/669787222 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9358009620 0x7f935800bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:20.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.006 7f9370aab700 1 --2- 172.21.2.101:0/669787222 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9358006800 0x7f9358008d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:20.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.006 7f93712ac700 1 -- 172.21.2.101:0/669787222 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9358009620 msgr2=0x7f935800bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:20.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.006 7f93712ac700 1 --2- 172.21.2.101:0/669787222 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9358009620 0x7f935800bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:20.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.006 7f93712ac700 1 -- 172.21.2.101:0/669787222 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9358006800 msgr2=0x7f9358008d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:20.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.006 7f93712ac700 1 --2- 172.21.2.101:0/669787222 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9358006800 0x7f9358008d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:20.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.006 7f93712ac700 1 -- 172.21.2.101:0/669787222 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9358050960 con 0x7f9358003aa0 2021-02-10T11:42:20.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.006 7f9371aad700 1 --2- 172.21.2.101:0/669787222 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9358009620 0x7f935800bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:42:20.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.006 7f93712ac700 1 --2- 172.21.2.101:0/669787222 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9358003aa0 0x7f9358005f60 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 rx=0x7f9360006ab0 tx=0x7f9360009b40).ready entity=mon.0 client_cookie=48fa18ea130c4318 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:20.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.006 7f935ffff700 1 -- 172.21.2.101:0/669787222 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f936000b2e0 con 0x7f9358003aa0 2021-02-10T11:42:20.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.006 7f935ffff700 1 -- 172.21.2.101:0/669787222 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936000b5e0 con 0x7f9358003aa0 2021-02-10T11:42:20.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.006 7f935ffff700 1 -- 172.21.2.101:0/669787222 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9360000960 con 0x7f9358003aa0 2021-02-10T11:42:20.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.006 7f9373490700 1 -- 172.21.2.101:0/669787222 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9358003aa0 msgr2=0x7f9358005f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:20.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.006 7f9373490700 1 --2- 172.21.2.101:0/669787222 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9358003aa0 0x7f9358005f60 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 rx=0x7f9360006ab0 tx=0x7f9360009b40).stop 2021-02-10T11:42:20.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.006 7f9371aad700 1 -- 172.21.2.101:0/669787222 reap_dead start 2021-02-10T11:42:20.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.007 7f9373490700 1 -- 172.21.2.101:0/669787222 shutdown_connections 2021-02-10T11:42:20.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.007 7f9373490700 1 -- 172.21.2.101:0/669787222 >> 172.21.2.101:0/669787222 conn(0x7f936c0ce440 msgr2=0x7f936c0d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:20.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.007 7f9373490700 1 -- 172.21.2.101:0/669787222 shutdown_connections 2021-02-10T11:42:20.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.007 7f9373490700 1 -- 172.21.2.101:0/669787222 wait complete. 2021-02-10T11:42:20.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.021 7f9373490700 1 Processor -- start 2021-02-10T11:42:20.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.021 7f9373490700 1 -- start start 2021-02-10T11:42:20.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.022 7f9373490700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f936c1a3550 0x7f936c1a3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:20.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.022 7f9373490700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f936c0bfb30 0x7f936c0bff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:20.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.022 7f9373490700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f936c0c0550 0x7f936c0c0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:20.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.022 7f9373490700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c08e180 con 0x7f936c0bfb30 2021-02-10T11:42:20.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.022 7f9373490700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c090c10 con 0x7f936c1a3550 2021-02-10T11:42:20.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.022 7f9373490700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936c090e30 con 0x7f936c0c0550 2021-02-10T11:42:20.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.022 7f9370aab700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f936c0bfb30 0x7f936c0bff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:20.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.022 7f9370aab700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f936c0bfb30 0x7f936c0bff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57968/0 (socket says 172.21.2.101:57968) 2021-02-10T11:42:20.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.022 7f9370aab700 1 -- 172.21.2.101:0/3417159556 learned_addr learned my addr 172.21.2.101:0/3417159556 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:20.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.022 7f9370aab700 1 -- 172.21.2.101:0/3417159556 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f936c0c0550 msgr2=0x7f936c0c0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:20.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.022 7f9371aad700 1 --2- 172.21.2.101:0/3417159556 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f936c0c0550 0x7f936c0c0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:20.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.023 7f93712ac700 1 --2- 172.21.2.101:0/3417159556 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f936c1a3550 0x7f936c1a3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:20.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.023 7f9370aab700 1 --2- 172.21.2.101:0/3417159556 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f936c0c0550 0x7f936c0c0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:20.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.023 7f9370aab700 1 -- 172.21.2.101:0/3417159556 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f936c1a3550 msgr2=0x7f936c1a3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:20.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.023 7f9370aab700 1 --2- 172.21.2.101:0/3417159556 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f936c1a3550 0x7f936c1a3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:20.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.023 7f9370aab700 1 -- 172.21.2.101:0/3417159556 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c091050 con 0x7f936c0bfb30 2021-02-10T11:42:20.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.023 7f9371aad700 1 --2- 172.21.2.101:0/3417159556 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f936c0c0550 0x7f936c0c0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:42:20.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.023 7f93712ac700 1 --2- 172.21.2.101:0/3417159556 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f936c1a3550 0x7f936c1a3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:42:20.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.023 7f9370aab700 1 --2- 172.21.2.101:0/3417159556 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f936c0bfb30 0x7f936c0bff30 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 rx=0x7f936800a5c0 tx=0x7f936800a960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:20.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.023 7f935dffb700 1 -- 172.21.2.101:0/3417159556 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9368000940 con 0x7f936c0bfb30 2021-02-10T11:42:20.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.023 7f935dffb700 1 -- 172.21.2.101:0/3417159556 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936800ad50 con 0x7f936c0bfb30 2021-02-10T11:42:20.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.023 7f935dffb700 1 -- 172.21.2.101:0/3417159556 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f93680045d0 con 0x7f936c0bfb30 2021-02-10T11:42:20.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.023 7f9373490700 1 -- 172.21.2.101:0/3417159556 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f936c091050 con 0x7f936c0bfb30 2021-02-10T11:42:20.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.023 7f9373490700 1 -- 172.21.2.101:0/3417159556 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936c0915e0 con 0x7f936c0bfb30 2021-02-10T11:42:20.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.024 7f935dffb700 1 -- 172.21.2.101:0/3417159556 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9368004140 con 0x7f936c0bfb30 2021-02-10T11:42:20.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.024 7f935dffb700 1 --2- 172.21.2.101:0/3417159556 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f934c0186c0 0x7f934c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:20.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.024 7f935dffb700 1 -- 172.21.2.101:0/3417159556 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f93680323c0 con 0x7f936c0bfb30 2021-02-10T11:42:20.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.024 7f93712ac700 1 --2- 172.21.2.101:0/3417159556 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f934c0186c0 0x7f934c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:20.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.024 7f93712ac700 1 --2- 172.21.2.101:0/3417159556 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f934c0186c0 0x7f934c01abc0 crc :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:20.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.055 7f9373490700 1 -- 172.21.2.101:0/3417159556 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f936c0ebbe0 con 0x7f936c0bfb30 2021-02-10T11:42:20.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.057 7f935dffb700 1 -- 172.21.2.101:0/3417159556 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9368025fe0 con 0x7f936c0bfb30 2021-02-10T11:42:20.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.185 7f9373490700 1 -- 172.21.2.101:0/3417159556 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f936c0f0b00 con 0x7f936c0bfb30 2021-02-10T11:42:20.581 INFO:tasks.ceph.mon.a.gibba001.stderr:2021-02-10 11:42:20.579 7f974d04d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-02-10T11:42:20.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.585 7f935dffb700 1 -- 172.21.2.101:0/3417159556 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v98) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9368025fe0 con 0x7f936c0bfb30 2021-02-10T11:42:20.604 INFO:teuthology.orchestra.run.gibba001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-02-10T11:42:20.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.603 7f9373490700 1 -- 172.21.2.101:0/3417159556 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f934c0186c0 msgr2=0x7f934c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:20.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.603 7f9373490700 1 --2- 172.21.2.101:0/3417159556 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f934c0186c0 0x7f934c01abc0 crc :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:20.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.603 7f9373490700 1 -- 172.21.2.101:0/3417159556 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f936c0bfb30 msgr2=0x7f936c0bff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:20.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.603 7f9373490700 1 --2- 172.21.2.101:0/3417159556 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f936c0bfb30 0x7f936c0bff30 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 rx=0x7f936800a5c0 tx=0x7f936800a960).stop 2021-02-10T11:42:20.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.604 7f9373490700 1 -- 172.21.2.101:0/3417159556 shutdown_connections 2021-02-10T11:42:20.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.604 7f9373490700 1 --2- 172.21.2.101:0/3417159556 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f934c0186c0 0x7f934c01abc0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:20.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.604 7f9373490700 1 --2- 172.21.2.101:0/3417159556 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f936c0c0550 0x7f936c0c0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:20.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.604 7f9373490700 1 --2- 172.21.2.101:0/3417159556 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f936c0bfb30 0x7f936c0bff30 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:20.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.604 7f9373490700 1 --2- 172.21.2.101:0/3417159556 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f936c1a3550 0x7f936c1a3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:20.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.604 7f9373490700 1 -- 172.21.2.101:0/3417159556 >> 172.21.2.101:0/3417159556 conn(0x7f936c0a0290 msgr2=0x7f936c08e560 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:20.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.604 7f9373490700 1 -- 172.21.2.101:0/3417159556 shutdown_connections 2021-02-10T11:42:20.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:20.604 7f9373490700 1 -- 172.21.2.101:0/3417159556 wait complete. 2021-02-10T11:42:20.613 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:42:20.638 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:42:21.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.023 7ff461c5e700 1 Processor -- start 2021-02-10T11:42:21.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.023 7ff461c5e700 1 -- start start 2021-02-10T11:42:21.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.023 7ff461c5e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff45c1b0470 0x7ff45c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:21.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.023 7ff461c5e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff45c0ef610 0x7ff45c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:21.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.023 7ff461c5e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff45c0f0b50 con 0x7ff45c0f0030 2021-02-10T11:42:21.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.023 7ff461c5e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff45c0e52e0 con 0x7ff45c1b0470 2021-02-10T11:42:21.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.023 7ff461c5e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c0e54d0 con 0x7ff45c0ef610 2021-02-10T11:42:21.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.024 7ff45bfff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff45c0f0030 0x7ff45c0f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58468/0 (socket says 172.21.2.101:58468) 2021-02-10T11:42:21.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.024 7ff45bfff700 1 -- 172.21.2.101:0/4222425825 learned_addr learned my addr 172.21.2.101:0/4222425825 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:21.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.024 7ff45a7fc700 1 -- 172.21.2.101:0/4222425825 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4149944027 0 0) 0x7ff454001500 con 0x7ff45c0f0030 2021-02-10T11:42:21.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.024 7ff45a7fc700 1 -- 172.21.2.101:0/4222425825 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff444002fe0 con 0x7ff45c0f0030 2021-02-10T11:42:21.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.024 7ff45affd700 1 --2- 172.21.2.101:0/4222425825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff45c0ef610 0x7ff45c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:21.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.024 7ff45a7fc700 1 -- 172.21.2.101:0/4222425825 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4169553812 0 0) 0x7ff454001460 con 0x7ff45c0f0030 2021-02-10T11:42:21.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.024 7ff45b7fe700 1 --2- 172.21.2.101:0/4222425825 >> v2:172.21.2.108:3300/0 conn(0x7ff45c1b0470 0x7ff45c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:21.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.024 7ff45a7fc700 1 -- 172.21.2.101:0/4222425825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff45c0ef610 msgr2=0x7ff45c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:21.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.024 7ff45a7fc700 1 --2- 172.21.2.101:0/4222425825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff45c0ef610 0x7ff45c0efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:21.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.024 7ff45a7fc700 1 -- 172.21.2.101:0/4222425825 >> v2:172.21.2.108:3300/0 conn(0x7ff45c1b0470 msgr2=0x7ff45c1b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:21.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.024 7ff45a7fc700 1 --2- 172.21.2.101:0/4222425825 >> v2:172.21.2.108:3300/0 conn(0x7ff45c1b0470 0x7ff45c1b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:21.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.024 7ff45a7fc700 1 -- 172.21.2.101:0/4222425825 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c0f0430 con 0x7ff45c0f0030 2021-02-10T11:42:21.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.024 7ff45a7fc700 1 -- 172.21.2.101:0/4222425825 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7ff454001980 con 0x7ff45c0f0030 2021-02-10T11:42:21.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.024 7ff45a7fc700 1 -- 172.21.2.101:0/4222425825 >> v1:172.21.2.101:6789/0 conn(0x7ff45c0f0030 legacy=0x7ff45c0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:21.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.024 7ff45a7fc700 1 --2- 172.21.2.101:0/4222425825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff444003b90 0x7ff444005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:21.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.024 7ff45a7fc700 1 --2- 172.21.2.101:0/4222425825 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff444006880 0x7ff444008da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:21.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.024 7ff45a7fc700 1 --2- 172.21.2.101:0/4222425825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff444009640 0x7ff44400bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:21.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.024 7ff45a7fc700 1 -- 172.21.2.101:0/4222425825 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44400c410 con 0x7ff444009640 2021-02-10T11:42:21.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.024 7ff45a7fc700 1 -- 172.21.2.101:0/4222425825 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44400c630 con 0x7ff444006880 2021-02-10T11:42:21.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.024 7ff45a7fc700 1 -- 172.21.2.101:0/4222425825 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44400c850 con 0x7ff444003b90 2021-02-10T11:42:21.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.024 7ff45bfff700 1 --2- 172.21.2.101:0/4222425825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff444009640 0x7ff44400bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:21.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.024 7ff45b7fe700 1 --2- 172.21.2.101:0/4222425825 >> v2:172.21.2.108:3300/0 conn(0x7ff45c1b0470 0x7ff45c1b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:42:21.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.025 7ff45affd700 1 --2- 172.21.2.101:0/4222425825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff45c0ef610 0x7ff45c0efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:42:21.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.025 7ff45bfff700 1 -- 172.21.2.101:0/4222425825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff444003b90 msgr2=0x7ff444005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:21.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.025 7ff45bfff700 1 --2- 172.21.2.101:0/4222425825 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff444003b90 0x7ff444005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:21.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.025 7ff45bfff700 1 -- 172.21.2.101:0/4222425825 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff444006880 msgr2=0x7ff444008da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:21.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.025 7ff45bfff700 1 --2- 172.21.2.101:0/4222425825 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff444006880 0x7ff444008da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:21.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.025 7ff45bfff700 1 -- 172.21.2.101:0/4222425825 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44400caf0 con 0x7ff444009640 2021-02-10T11:42:21.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.025 7ff45bfff700 1 --2- 172.21.2.101:0/4222425825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff444009640 0x7ff44400bb60 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 rx=0x7ff454001ae0 tx=0x7ff45400a310).ready entity=mon.0 client_cookie=208b57a3362062cc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:21.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.025 7ff45a7fc700 1 -- 172.21.2.101:0/4222425825 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff4540070c0 con 0x7ff444009640 2021-02-10T11:42:21.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.025 7ff45a7fc700 1 -- 172.21.2.101:0/4222425825 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff454007340 con 0x7ff444009640 2021-02-10T11:42:21.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.025 7ff45a7fc700 1 -- 172.21.2.101:0/4222425825 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff4540049a0 con 0x7ff444009640 2021-02-10T11:42:21.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.025 7ff461c5e700 1 -- 172.21.2.101:0/4222425825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff444009640 msgr2=0x7ff44400bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:21.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.025 7ff461c5e700 1 --2- 172.21.2.101:0/4222425825 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff444009640 0x7ff44400bb60 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 rx=0x7ff454001ae0 tx=0x7ff45400a310).stop 2021-02-10T11:42:21.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.025 7ff45bfff700 1 -- 172.21.2.101:0/4222425825 reap_dead start 2021-02-10T11:42:21.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.026 7ff461c5e700 1 -- 172.21.2.101:0/4222425825 shutdown_connections 2021-02-10T11:42:21.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.026 7ff461c5e700 1 -- 172.21.2.101:0/4222425825 >> 172.21.2.101:0/4222425825 conn(0x7ff45c0ce3e0 msgr2=0x7ff45c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:21.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.026 7ff461c5e700 1 -- 172.21.2.101:0/4222425825 shutdown_connections 2021-02-10T11:42:21.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.026 7ff461c5e700 1 -- 172.21.2.101:0/4222425825 wait complete. 2021-02-10T11:42:21.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.041 7ff461c5e700 1 Processor -- start 2021-02-10T11:42:21.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.041 7ff461c5e700 1 -- start start 2021-02-10T11:42:21.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.041 7ff461c5e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff45c0938f0 0x7ff45c1a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:21.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.041 7ff461c5e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff45c093cf0 0x7ff45c092690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:21.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.041 7ff461c5e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff45c092cb0 0x7ff45c0930b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:21.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.041 7ff461c5e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c0b8300 con 0x7ff45c092cb0 2021-02-10T11:42:21.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.041 7ff461c5e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c0b7f40 con 0x7ff45c0938f0 2021-02-10T11:42:21.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.041 7ff461c5e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c1a4930 con 0x7ff45c093cf0 2021-02-10T11:42:21.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.041 7ff45bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff45c092cb0 0x7ff45c0930b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:21.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.041 7ff45bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff45c092cb0 0x7ff45c0930b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:57988/0 (socket says 172.21.2.101:57988) 2021-02-10T11:42:21.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.041 7ff45bfff700 1 -- 172.21.2.101:0/3504898278 learned_addr learned my addr 172.21.2.101:0/3504898278 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:21.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.042 7ff45bfff700 1 -- 172.21.2.101:0/3504898278 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff45c093cf0 msgr2=0x7ff45c092690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:21.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.042 7ff45bfff700 1 --2- 172.21.2.101:0/3504898278 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff45c093cf0 0x7ff45c092690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:21.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.042 7ff45bfff700 1 -- 172.21.2.101:0/3504898278 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff45c0938f0 msgr2=0x7ff45c1a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:21.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.042 7ff45bfff700 1 --2- 172.21.2.101:0/3504898278 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff45c0938f0 0x7ff45c1a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:21.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.042 7ff45bfff700 1 -- 172.21.2.101:0/3504898278 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c0ad530 con 0x7ff45c092cb0 2021-02-10T11:42:21.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.042 7ff45bfff700 1 --2- 172.21.2.101:0/3504898278 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff45c092cb0 0x7ff45c0930b0 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 rx=0x7ff454004e30 tx=0x7ff4540078b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:21.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.042 7ff443fff700 1 -- 172.21.2.101:0/3504898278 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff45400b310 con 0x7ff45c092cb0 2021-02-10T11:42:21.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.042 7ff443fff700 1 -- 172.21.2.101:0/3504898278 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff454000fe0 con 0x7ff45c092cb0 2021-02-10T11:42:21.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.042 7ff443fff700 1 -- 172.21.2.101:0/3504898278 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff45400b690 con 0x7ff45c092cb0 2021-02-10T11:42:21.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.042 7ff461c5e700 1 -- 172.21.2.101:0/3504898278 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff45c0ad530 con 0x7ff45c092cb0 2021-02-10T11:42:21.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.042 7ff461c5e700 1 -- 172.21.2.101:0/3504898278 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff45c0ad530 con 0x7ff45c092cb0 2021-02-10T11:42:21.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.043 7ff443fff700 1 -- 172.21.2.101:0/3504898278 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff454009020 con 0x7ff45c092cb0 2021-02-10T11:42:21.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.043 7ff443fff700 1 --2- 172.21.2.101:0/3504898278 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff43c0186c0 0x7ff43c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:21.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.043 7ff443fff700 1 -- 172.21.2.101:0/3504898278 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5659+0+0 (secure 0 0 0) 0x7ff454032930 con 0x7ff45c092cb0 2021-02-10T11:42:21.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.043 7ff45b7fe700 1 --2- 172.21.2.101:0/3504898278 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff43c0186c0 0x7ff43c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:21.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.043 7ff45b7fe700 1 --2- 172.21.2.101:0/3504898278 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff43c0186c0 0x7ff43c01abc0 crc :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:21.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.074 7ff461c5e700 1 -- 172.21.2.101:0/3504898278 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff45c105a80 con 0x7ff45c092cb0 2021-02-10T11:42:21.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.076 7ff443fff700 1 -- 172.21.2.101:0/3504898278 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff454008dc0 con 0x7ff45c092cb0 2021-02-10T11:42:21.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.207 7ff461c5e700 1 -- 172.21.2.101:0/3504898278 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff45c105a80 con 0x7ff45c092cb0 2021-02-10T11:42:21.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.207 7ff443fff700 1 -- 172.21.2.101:0/3504898278 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+1757 (secure 0 0 0) 0x7ff454009000 con 0x7ff45c092cb0 2021-02-10T11:42:21.210 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:42:21.210 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":98,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/654364385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":654364385},{"type":"v1","addr":"172.21.2.101:6835","nonce":654364385}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":92}],"filesystems":[{"mdsmap":{"epoch":98,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:41:34.701339","modified":"2021-02-10 11:42:20.580015","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2021-02-10T11:42:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 98 2021-02-10T11:42:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.209 7ff461c5e700 1 -- 172.21.2.101:0/3504898278 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff43c0186c0 msgr2=0x7ff43c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.209 7ff461c5e700 1 --2- 172.21.2.101:0/3504898278 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff43c0186c0 0x7ff43c01abc0 crc :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.209 7ff461c5e700 1 -- 172.21.2.101:0/3504898278 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff45c092cb0 msgr2=0x7ff45c0930b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.209 7ff461c5e700 1 --2- 172.21.2.101:0/3504898278 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff45c092cb0 0x7ff45c0930b0 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 rx=0x7ff454004e30 tx=0x7ff4540078b0).stop 2021-02-10T11:42:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.209 7ff461c5e700 1 -- 172.21.2.101:0/3504898278 shutdown_connections 2021-02-10T11:42:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.209 7ff461c5e700 1 --2- 172.21.2.101:0/3504898278 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff43c0186c0 0x7ff43c01abc0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.209 7ff461c5e700 1 --2- 172.21.2.101:0/3504898278 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff45c093cf0 0x7ff45c092690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.209 7ff461c5e700 1 --2- 172.21.2.101:0/3504898278 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff45c092cb0 0x7ff45c0930b0 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.209 7ff461c5e700 1 --2- 172.21.2.101:0/3504898278 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff45c0938f0 0x7ff45c1a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.209 7ff461c5e700 1 -- 172.21.2.101:0/3504898278 >> 172.21.2.101:0/3504898278 conn(0x7ff45c0a1dc0 msgr2=0x7ff45c08e860 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:21.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.209 7ff461c5e700 1 -- 172.21.2.101:0/3504898278 shutdown_connections 2021-02-10T11:42:21.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.209 7ff461c5e700 1 -- 172.21.2.101:0/3504898278 wait complete. 2021-02-10T11:42:21.220 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-02-10T11:42:21.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.635 7feacc9f1700 1 Processor -- start 2021-02-10T11:42:21.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.635 7feacc9f1700 1 -- start start 2021-02-10T11:42:21.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.636 7feacc9f1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7feac80d0340 0x7feac80ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:21.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.636 7feacc9f1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feac80d1c70 0x7feac80efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:21.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.636 7feacc9f1700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac80f0b10 con 0x7feac80d1870 2021-02-10T11:42:21.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.636 7feacc9f1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac80e5ed0 con 0x7feac80d0340 2021-02-10T11:42:21.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.636 7feacc9f1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac81b0be0 con 0x7feac80d1c70 2021-02-10T11:42:21.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.636 7feac661c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7feac80d1870 0x7feac81b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58486/0 (socket says 172.21.2.101:58486) 2021-02-10T11:42:21.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.636 7feac661c700 1 -- 172.21.2.101:0/2974768373 learned_addr learned my addr 172.21.2.101:0/2974768373 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:21.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.636 7feac5e1b700 1 --2- 172.21.2.101:0/2974768373 >> v2:172.21.2.108:3300/0 conn(0x7feac80d0340 0x7feac80ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:21.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.636 7feac6e1d700 1 --2- 172.21.2.101:0/2974768373 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feac80d1c70 0x7feac80efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:21.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.636 7feac561a700 1 -- 172.21.2.101:0/2974768373 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 551850462 0 0) 0x7feab00010b0 con 0x7feac80d1870 2021-02-10T11:42:21.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.636 7feac561a700 1 -- 172.21.2.101:0/2974768373 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feab4002fe0 con 0x7feac80d1870 2021-02-10T11:42:21.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.636 7feac5e1b700 1 -- 172.21.2.101:0/2974768373 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feac80d1c70 msgr2=0x7feac80efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:21.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.636 7feac5e1b700 1 --2- 172.21.2.101:0/2974768373 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feac80d1c70 0x7feac80efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:21.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.636 7feac5e1b700 1 -- 172.21.2.101:0/2974768373 >> v1:172.21.2.101:6789/0 conn(0x7feac80d1870 legacy=0x7feac81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:21.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.637 7feac561a700 1 -- 172.21.2.101:0/2974768373 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2950484991 0 0) 0x7feab00010b0 con 0x7feac80d1870 2021-02-10T11:42:21.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.637 7feac5e1b700 1 -- 172.21.2.101:0/2974768373 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac80f20a0 con 0x7feac80d0340 2021-02-10T11:42:21.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.637 7feac5e1b700 1 --2- 172.21.2.101:0/2974768373 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feac80d0340 0x7feac80ef640 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7feab8006650 tx=0x7feab8009a00).ready entity=mon.1 client_cookie=484172606073c21c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:21.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.637 7feac561a700 1 -- 172.21.2.101:0/2974768373 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feab80009b0 con 0x7feac80d0340 2021-02-10T11:42:21.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.637 7feac561a700 1 -- 172.21.2.101:0/2974768373 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab800c0c0 con 0x7feac80d0340 2021-02-10T11:42:21.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.637 7feac561a700 1 -- 172.21.2.101:0/2974768373 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feab800c5a0 con 0x7feac80d0340 2021-02-10T11:42:21.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.637 7feacc9f1700 1 -- 172.21.2.101:0/2974768373 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feac80d0340 msgr2=0x7feac80ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:21.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.637 7feacc9f1700 1 --2- 172.21.2.101:0/2974768373 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feac80d0340 0x7feac80ef640 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7feab8006650 tx=0x7feab8009a00).stop 2021-02-10T11:42:21.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.638 7feacc9f1700 1 -- 172.21.2.101:0/2974768373 shutdown_connections 2021-02-10T11:42:21.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.638 7feacc9f1700 1 --2- 172.21.2.101:0/2974768373 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feac80d1c70 0x7feac80efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:21.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.638 7feacc9f1700 1 --2- 172.21.2.101:0/2974768373 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feac80d0340 0x7feac80ef640 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:21.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.638 7feacc9f1700 1 -- 172.21.2.101:0/2974768373 >> 172.21.2.101:0/2974768373 conn(0x7feac80ce450 msgr2=0x7feac80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:21.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.638 7feacc9f1700 1 -- 172.21.2.101:0/2974768373 shutdown_connections 2021-02-10T11:42:21.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.638 7feacc9f1700 1 -- 172.21.2.101:0/2974768373 wait complete. 2021-02-10T11:42:21.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.653 7feacc9f1700 1 Processor -- start 2021-02-10T11:42:21.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.653 7feacc9f1700 1 -- start start 2021-02-10T11:42:21.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.653 7feacc9f1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feac8094310 0x7feac80ebe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:21.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.653 7feacc9f1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feac8094710 0x7feac80bca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:21.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.653 7feacc9f1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feac80ec490 0x7feac80bd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:21.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.653 7feacc9f1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac8093d90 con 0x7feac80ec490 2021-02-10T11:42:21.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.653 7feacc9f1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac80bd670 con 0x7feac8094310 2021-02-10T11:42:21.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.653 7feacc9f1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac80939f0 con 0x7feac8094710 2021-02-10T11:42:21.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.653 7feac6e1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feac80ec490 0x7feac80bd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:21.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.653 7feac6e1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feac80ec490 0x7feac80bd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58000/0 (socket says 172.21.2.101:58000) 2021-02-10T11:42:21.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.653 7feac6e1d700 1 -- 172.21.2.101:0/70426310 learned_addr learned my addr 172.21.2.101:0/70426310 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:21.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.654 7feac6e1d700 1 -- 172.21.2.101:0/70426310 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feac8094710 msgr2=0x7feac80bca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:21.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.654 7feac6e1d700 1 --2- 172.21.2.101:0/70426310 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feac8094710 0x7feac80bca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:21.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.654 7feac6e1d700 1 -- 172.21.2.101:0/70426310 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feac8094310 msgr2=0x7feac80ebe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:21.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.654 7feac6e1d700 1 --2- 172.21.2.101:0/70426310 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feac8094310 0x7feac80ebe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:21.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.654 7feac6e1d700 1 -- 172.21.2.101:0/70426310 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac80bda80 con 0x7feac80ec490 2021-02-10T11:42:21.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.654 7feac6e1d700 1 --2- 172.21.2.101:0/70426310 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feac80ec490 0x7feac80bd050 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 rx=0x7feac000b910 tx=0x7feac000bcb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:21.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.654 7feabeffd700 1 -- 172.21.2.101:0/70426310 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feac0006aa0 con 0x7feac80ec490 2021-02-10T11:42:21.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.654 7feabeffd700 1 -- 172.21.2.101:0/70426310 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac00011f0 con 0x7feac80ec490 2021-02-10T11:42:21.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.654 7feabeffd700 1 -- 172.21.2.101:0/70426310 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feac0001710 con 0x7feac80ec490 2021-02-10T11:42:21.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.654 7feacc9f1700 1 -- 172.21.2.101:0/70426310 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac80bda80 con 0x7feac80ec490 2021-02-10T11:42:21.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.654 7feacc9f1700 1 -- 172.21.2.101:0/70426310 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac81a76d0 con 0x7feac80ec490 2021-02-10T11:42:21.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.655 7feabeffd700 1 -- 172.21.2.101:0/70426310 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7feac00013d0 con 0x7feac80ec490 2021-02-10T11:42:21.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.655 7feabeffd700 1 --2- 172.21.2.101:0/70426310 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feaac0186c0 0x7feaac01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:21.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.655 7feabeffd700 1 -- 172.21.2.101:0/70426310 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5659+0+0 (secure 0 0 0) 0x7feac0032970 con 0x7feac80ec490 2021-02-10T11:42:21.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.655 7feac661c700 1 --2- 172.21.2.101:0/70426310 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feaac0186c0 0x7feaac01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:21.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.656 7feac661c700 1 --2- 172.21.2.101:0/70426310 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feaac0186c0 0x7feaac01abc0 crc :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:21.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.686 7feacc9f1700 1 -- 172.21.2.101:0/70426310 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac80ebc40 con 0x7feac80ec490 2021-02-10T11:42:21.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.688 7feabeffd700 1 -- 172.21.2.101:0/70426310 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7feac0000f40 con 0x7feac80ec490 2021-02-10T11:42:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:21.815 7feacc9f1700 1 -- 172.21.2.101:0/70426310 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7feac80f0b10 con 0x7feac80ec490 2021-02-10T11:42:22.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:22.591 7feabeffd700 1 -- 172.21.2.101:0/70426310 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v99) v1 ==== 104+0+0 (secure 0 0 0) 0x7feac00013d0 con 0x7feac80ec490 2021-02-10T11:42:22.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:22.637 7feacc9f1700 1 -- 172.21.2.101:0/70426310 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feaac0186c0 msgr2=0x7feaac01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:22.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:22.637 7feacc9f1700 1 --2- 172.21.2.101:0/70426310 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feaac0186c0 0x7feaac01abc0 crc :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:22.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:22.637 7feacc9f1700 1 -- 172.21.2.101:0/70426310 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feac80ec490 msgr2=0x7feac80bd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:22.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:22.637 7feacc9f1700 1 --2- 172.21.2.101:0/70426310 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feac80ec490 0x7feac80bd050 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 rx=0x7feac000b910 tx=0x7feac000bcb0).stop 2021-02-10T11:42:22.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:22.637 7feacc9f1700 1 -- 172.21.2.101:0/70426310 shutdown_connections 2021-02-10T11:42:22.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:22.637 7feacc9f1700 1 --2- 172.21.2.101:0/70426310 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feaac0186c0 0x7feaac01abc0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:22.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:22.637 7feacc9f1700 1 --2- 172.21.2.101:0/70426310 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feac8094710 0x7feac80bca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:22.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:22.637 7feacc9f1700 1 --2- 172.21.2.101:0/70426310 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feac80ec490 0x7feac80bd050 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:22.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:22.637 7feacc9f1700 1 --2- 172.21.2.101:0/70426310 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feac8094310 0x7feac80ebe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:22.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:22.637 7feacc9f1700 1 -- 172.21.2.101:0/70426310 >> 172.21.2.101:0/70426310 conn(0x7feac808fa10 msgr2=0x7feac80bfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:22.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:22.637 7feacc9f1700 1 -- 172.21.2.101:0/70426310 shutdown_connections 2021-02-10T11:42:22.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:22.637 7feacc9f1700 1 -- 172.21.2.101:0/70426310 wait complete. 2021-02-10T11:42:22.647 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-02-10T11:42:23.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.061 7f204d83c700 1 Processor -- start 2021-02-10T11:42:23.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.061 7f204d83c700 1 -- start start 2021-02-10T11:42:23.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.061 7f204d83c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f20480d1c80 0x7f20480cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:23.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.061 7f204d83c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20480cf590 0x7f20480cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:23.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.061 7f204d83c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20480fe1d0 con 0x7f20480d1880 2021-02-10T11:42:23.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.061 7f204d83c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20481b0cc0 con 0x7f20480d1c80 2021-02-10T11:42:23.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.061 7f204d83c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2048109a70 con 0x7f20480cf590 2021-02-10T11:42:23.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.061 7f2046d73700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f20480d1c80 0x7f20480cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:23.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.061 7f2047574700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f20480d1880 0x7f20481b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58500/0 (socket says 172.21.2.101:58500) 2021-02-10T11:42:23.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.061 7f2047574700 1 -- 172.21.2.101:0/1742193385 learned_addr learned my addr 172.21.2.101:0/1742193385 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:23.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.062 7f2046572700 1 -- 172.21.2.101:0/1742193385 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3659862755 0 0) 0x7f2030000ce0 con 0x7f20480d1880 2021-02-10T11:42:23.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.062 7f2046572700 1 -- 172.21.2.101:0/1742193385 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2034002fe0 con 0x7f20480d1880 2021-02-10T11:42:23.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.062 7f2046d73700 1 -- 172.21.2.101:0/1742193385 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20480cf590 msgr2=0x7f20480cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:23.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.062 7f2046d73700 1 --2- 172.21.2.101:0/1742193385 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20480cf590 0x7f20480cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:23.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.062 7f2046d73700 1 -- 172.21.2.101:0/1742193385 >> v1:172.21.2.101:6789/0 conn(0x7f20480d1880 legacy=0x7f20481b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:23.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.062 7f2046d73700 1 -- 172.21.2.101:0/1742193385 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2048088070 con 0x7f20480d1c80 2021-02-10T11:42:23.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.062 7f2046d73700 1 --2- 172.21.2.101:0/1742193385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20480d1c80 0x7f20480cef70 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 rx=0x7f2038006650 tx=0x7f2038009a80).ready entity=mon.0 client_cookie=3157759004716189 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:23.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.062 7f2046572700 1 -- 172.21.2.101:0/1742193385 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f20380009b0 con 0x7f20480d1c80 2021-02-10T11:42:23.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.062 7f2046572700 1 -- 172.21.2.101:0/1742193385 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203800c0c0 con 0x7f20480d1c80 2021-02-10T11:42:23.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.062 7f2046572700 1 -- 172.21.2.101:0/1742193385 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f203800c5e0 con 0x7f20480d1c80 2021-02-10T11:42:23.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.062 7f204d83c700 1 -- 172.21.2.101:0/1742193385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20480d1c80 msgr2=0x7f20480cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:23.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.062 7f204d83c700 1 --2- 172.21.2.101:0/1742193385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20480d1c80 0x7f20480cef70 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 rx=0x7f2038006650 tx=0x7f2038009a80).stop 2021-02-10T11:42:23.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.063 7f204d83c700 1 -- 172.21.2.101:0/1742193385 shutdown_connections 2021-02-10T11:42:23.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.063 7f204d83c700 1 --2- 172.21.2.101:0/1742193385 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20480cf590 0x7f20480cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:23.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.063 7f204d83c700 1 --2- 172.21.2.101:0/1742193385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20480d1c80 0x7f20480cef70 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:23.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.063 7f204d83c700 1 -- 172.21.2.101:0/1742193385 >> 172.21.2.101:0/1742193385 conn(0x7f20480ce430 msgr2=0x7f20480d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:23.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.063 7f204d83c700 1 -- 172.21.2.101:0/1742193385 shutdown_connections 2021-02-10T11:42:23.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.063 7f204d83c700 1 -- 172.21.2.101:0/1742193385 wait complete. 2021-02-10T11:42:23.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.077 7f204d83c700 1 Processor -- start 2021-02-10T11:42:23.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.078 7f204d83c700 1 -- start start 2021-02-10T11:42:23.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.078 7f204d83c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20480ad520 0x7f20480ad920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:23.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.078 7f204d83c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20480adf40 0x7f2048097c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:23.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.078 7f204d83c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2048098250 0x7f2048098650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:23.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.078 7f204d83c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2048098c70 con 0x7f20480adf40 2021-02-10T11:42:23.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.078 7f204d83c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20480910f0 con 0x7f2048098250 2021-02-10T11:42:23.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.078 7f204d83c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2048091310 con 0x7f20480ad520 2021-02-10T11:42:23.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.078 7f2046d73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20480adf40 0x7f2048097c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:23.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.078 7f2046d73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20480adf40 0x7f2048097c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58016/0 (socket says 172.21.2.101:58016) 2021-02-10T11:42:23.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.078 7f2046d73700 1 -- 172.21.2.101:0/3098662882 learned_addr learned my addr 172.21.2.101:0/3098662882 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:23.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.079 7f2046d73700 1 -- 172.21.2.101:0/3098662882 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20480ad520 msgr2=0x7f20480ad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:23.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.079 7f2046d73700 1 --2- 172.21.2.101:0/3098662882 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20480ad520 0x7f20480ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:23.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.079 7f2046d73700 1 -- 172.21.2.101:0/3098662882 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2048098250 msgr2=0x7f2048098650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:23.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.079 7f2046d73700 1 --2- 172.21.2.101:0/3098662882 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2048098250 0x7f2048098650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:23.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.079 7f2046d73700 1 -- 172.21.2.101:0/3098662882 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204819a3b0 con 0x7f20480adf40 2021-02-10T11:42:23.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.079 7f2046d73700 1 --2- 172.21.2.101:0/3098662882 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20480adf40 0x7f2048097c30 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 rx=0x7f2038009650 tx=0x7f203800b8e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:23.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.079 7f203ffff700 1 -- 172.21.2.101:0/3098662882 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2038006030 con 0x7f20480adf40 2021-02-10T11:42:23.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.079 7f203ffff700 1 -- 172.21.2.101:0/3098662882 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203800bcd0 con 0x7f20480adf40 2021-02-10T11:42:23.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.079 7f203ffff700 1 -- 172.21.2.101:0/3098662882 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f20380054d0 con 0x7f20480adf40 2021-02-10T11:42:23.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.079 7f204d83c700 1 -- 172.21.2.101:0/3098662882 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204819a3b0 con 0x7f20480adf40 2021-02-10T11:42:23.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.079 7f204d83c700 1 -- 172.21.2.101:0/3098662882 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204819a3b0 con 0x7f20480adf40 2021-02-10T11:42:23.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.080 7f203ffff700 1 -- 172.21.2.101:0/3098662882 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2038009300 con 0x7f20480adf40 2021-02-10T11:42:23.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.080 7f203ffff700 1 --2- 172.21.2.101:0/3098662882 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f202c0186c0 0x7f202c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:23.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.080 7f203ffff700 1 -- 172.21.2.101:0/3098662882 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f20380325b0 con 0x7f20480adf40 2021-02-10T11:42:23.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.080 7f2047574700 1 --2- 172.21.2.101:0/3098662882 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f202c0186c0 0x7f202c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:23.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.081 7f2047574700 1 --2- 172.21.2.101:0/3098662882 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f202c0186c0 0x7f202c01abc0 crc :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:23.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.111 7f204d83c700 1 -- 172.21.2.101:0/3098662882 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2048098c70 con 0x7f20480adf40 2021-02-10T11:42:23.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.114 7f203ffff700 1 -- 172.21.2.101:0/3098662882 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2038000940 con 0x7f20480adf40 2021-02-10T11:42:23.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.242 7f204d83c700 1 -- 172.21.2.101:0/3098662882 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2048098c70 con 0x7f20480adf40 2021-02-10T11:42:23.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.594 7f203ffff700 1 -- 172.21.2.101:0/3098662882 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v122) v1 ==== 185+0+0 (secure 0 0 0) 0x7f203801f530 con 0x7f20480adf40 2021-02-10T11:42:23.610 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' removed 2021-02-10T11:42:23.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.611 7f204d83c700 1 -- 172.21.2.101:0/3098662882 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f202c0186c0 msgr2=0x7f202c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:23.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.611 7f204d83c700 1 --2- 172.21.2.101:0/3098662882 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f202c0186c0 0x7f202c01abc0 crc :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:23.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.611 7f204d83c700 1 -- 172.21.2.101:0/3098662882 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20480adf40 msgr2=0x7f2048097c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:23.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.611 7f204d83c700 1 --2- 172.21.2.101:0/3098662882 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20480adf40 0x7f2048097c30 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 rx=0x7f2038009650 tx=0x7f203800b8e0).stop 2021-02-10T11:42:23.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.611 7f204d83c700 1 -- 172.21.2.101:0/3098662882 shutdown_connections 2021-02-10T11:42:23.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.611 7f204d83c700 1 --2- 172.21.2.101:0/3098662882 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2048098250 0x7f2048098650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:23.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.611 7f204d83c700 1 --2- 172.21.2.101:0/3098662882 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20480adf40 0x7f2048097c30 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:23.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.611 7f204d83c700 1 --2- 172.21.2.101:0/3098662882 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f202c0186c0 0x7f202c01abc0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:23.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.611 7f204d83c700 1 --2- 172.21.2.101:0/3098662882 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20480ad520 0x7f20480ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:23.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.611 7f204d83c700 1 -- 172.21.2.101:0/3098662882 >> 172.21.2.101:0/3098662882 conn(0x7f204818e810 msgr2=0x7f204818c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:23.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.611 7f204d83c700 1 -- 172.21.2.101:0/3098662882 shutdown_connections 2021-02-10T11:42:23.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:23.611 7f204d83c700 1 -- 172.21.2.101:0/3098662882 wait complete. 2021-02-10T11:42:23.622 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-02-10T11:42:24.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.034 7f576a875700 1 Processor -- start 2021-02-10T11:42:24.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.034 7f576a875700 1 -- start start 2021-02-10T11:42:24.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.035 7f576a875700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f57640d1c80 0x7f57640cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:24.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.035 7f576a875700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f57640cf590 0x7f57640cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:24.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.035 7f576a875700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57640fe1d0 con 0x7f57640d1880 2021-02-10T11:42:24.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.035 7f576a875700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57641b0cc0 con 0x7f57640d1c80 2021-02-10T11:42:24.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.035 7f576a875700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5764109a70 con 0x7f57640cf590 2021-02-10T11:42:24.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.035 7f57637fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f57640d1c80 0x7f57640cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:24.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.035 7f57637fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f57640d1c80 0x7f57640cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58024/0 (socket says 172.21.2.101:58024) 2021-02-10T11:42:24.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.035 7f57637fe700 1 -- 172.21.2.101:0/867512181 learned_addr learned my addr 172.21.2.101:0/867512181 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:24.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.035 7f57637fe700 1 -- 172.21.2.101:0/867512181 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f57640cf590 msgr2=0x7f57640cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:24.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.035 7f57637fe700 1 --2- 172.21.2.101:0/867512181 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f57640cf590 0x7f57640cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:24.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.035 7f57637fe700 1 -- 172.21.2.101:0/867512181 >> v1:172.21.2.108:6789/0 conn(0x7f57640d1880 legacy=0x7f57641b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:24.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.035 7f57637fe700 1 -- 172.21.2.101:0/867512181 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5764088070 con 0x7f57640d1c80 2021-02-10T11:42:24.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.036 7f57637fe700 1 --2- 172.21.2.101:0/867512181 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f57640d1c80 0x7f57640cef70 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 rx=0x7f5758006700 tx=0x7f5758009b80).ready entity=mon.0 client_cookie=f903ed737da88f73 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:24.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.036 7f5762ffd700 1 -- 172.21.2.101:0/867512181 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f575800c0c0 con 0x7f57640d1c80 2021-02-10T11:42:24.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.036 7f5762ffd700 1 -- 172.21.2.101:0/867512181 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575800c400 con 0x7f57640d1c80 2021-02-10T11:42:24.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.036 7f5762ffd700 1 -- 172.21.2.101:0/867512181 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f575800c920 con 0x7f57640d1c80 2021-02-10T11:42:24.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.036 7f576a875700 1 -- 172.21.2.101:0/867512181 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f57640d1c80 msgr2=0x7f57640cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:24.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.036 7f576a875700 1 --2- 172.21.2.101:0/867512181 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f57640d1c80 0x7f57640cef70 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 rx=0x7f5758006700 tx=0x7f5758009b80).stop 2021-02-10T11:42:24.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.037 7f576a875700 1 -- 172.21.2.101:0/867512181 shutdown_connections 2021-02-10T11:42:24.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.037 7f576a875700 1 --2- 172.21.2.101:0/867512181 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f57640cf590 0x7f57640cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:24.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.037 7f576a875700 1 --2- 172.21.2.101:0/867512181 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f57640d1c80 0x7f57640cef70 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:24.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.037 7f576a875700 1 -- 172.21.2.101:0/867512181 >> 172.21.2.101:0/867512181 conn(0x7f57640ce430 msgr2=0x7f57640d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:24.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.037 7f576a875700 1 -- 172.21.2.101:0/867512181 shutdown_connections 2021-02-10T11:42:24.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.037 7f576a875700 1 -- 172.21.2.101:0/867512181 wait complete. 2021-02-10T11:42:24.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.051 7f576a875700 1 Processor -- start 2021-02-10T11:42:24.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.052 7f576a875700 1 -- start start 2021-02-10T11:42:24.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.052 7f576a875700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f576419e5b0 0x7f57641afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:24.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.052 7f576a875700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f576419e9b0 0x7f57641b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:24.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.052 7f576a875700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f57641b0250 0x7f57641b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:24.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.052 7f576a875700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5764195bd0 con 0x7f576419e9b0 2021-02-10T11:42:24.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.052 7f576a875700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57641b0840 con 0x7f576419e5b0 2021-02-10T11:42:24.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.052 7f576a875700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576419eed0 con 0x7f57641b0250 2021-02-10T11:42:24.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.052 7f57637fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f576419e9b0 0x7f57641b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:24.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.052 7f57637fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f576419e9b0 0x7f57641b0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58026/0 (socket says 172.21.2.101:58026) 2021-02-10T11:42:24.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.052 7f57637fe700 1 -- 172.21.2.101:0/2646421640 learned_addr learned my addr 172.21.2.101:0/2646421640 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:24.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.052 7f57637fe700 1 -- 172.21.2.101:0/2646421640 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f57641b0250 msgr2=0x7f57641b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:24.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.052 7f57637fe700 1 --2- 172.21.2.101:0/2646421640 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f57641b0250 0x7f57641b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:24.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.052 7f57637fe700 1 -- 172.21.2.101:0/2646421640 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f576419e5b0 msgr2=0x7f57641afc30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:42:24.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.052 7f57637fe700 1 --2- 172.21.2.101:0/2646421640 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f576419e5b0 0x7f57641afc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:24.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.052 7f57637fe700 1 -- 172.21.2.101:0/2646421640 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576419f0b0 con 0x7f576419e9b0 2021-02-10T11:42:24.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.052 7f57637fe700 1 --2- 172.21.2.101:0/2646421640 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f576419e9b0 0x7f57641b0dc0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 rx=0x7f57580066a0 tx=0x7f575800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:24.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.053 7f5760ff9700 1 -- 172.21.2.101:0/2646421640 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f575800bd80 con 0x7f576419e9b0 2021-02-10T11:42:24.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.053 7f5760ff9700 1 -- 172.21.2.101:0/2646421640 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57580052e0 con 0x7f576419e9b0 2021-02-10T11:42:24.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.053 7f576a875700 1 -- 172.21.2.101:0/2646421640 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576419f0b0 con 0x7f576419e9b0 2021-02-10T11:42:24.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.053 7f576a875700 1 -- 172.21.2.101:0/2646421640 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57641b1cf0 con 0x7f576419e9b0 2021-02-10T11:42:24.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.053 7f5760ff9700 1 -- 172.21.2.101:0/2646421640 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f57580057c0 con 0x7f576419e9b0 2021-02-10T11:42:24.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.053 7f5760ff9700 1 -- 172.21.2.101:0/2646421640 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5758006030 con 0x7f576419e9b0 2021-02-10T11:42:24.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.053 7f5760ff9700 1 --2- 172.21.2.101:0/2646421640 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f57440186c0 0x7f574401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:24.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.053 7f5760ff9700 1 -- 172.21.2.101:0/2646421640 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f5758032700 con 0x7f576419e9b0 2021-02-10T11:42:24.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.054 7f5763fff700 1 --2- 172.21.2.101:0/2646421640 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f57440186c0 0x7f574401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:24.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.054 7f5763fff700 1 --2- 172.21.2.101:0/2646421640 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f57440186c0 0x7f574401abc0 crc :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:24.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.085 7f576a875700 1 -- 172.21.2.101:0/2646421640 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57641b0840 con 0x7f576419e9b0 2021-02-10T11:42:24.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.087 7f5760ff9700 1 -- 172.21.2.101:0/2646421640 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5758005a80 con 0x7f576419e9b0 2021-02-10T11:42:24.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.218 7f576a875700 1 -- 172.21.2.101:0/2646421640 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f57641b0840 con 0x7f576419e9b0 2021-02-10T11:42:24.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.600 7f5760ff9700 1 -- 172.21.2.101:0/2646421640 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v123) v1 ==== 173+0+0 (secure 0 0 0) 0x7f5758006030 con 0x7f576419e9b0 2021-02-10T11:42:24.636 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' removed 2021-02-10T11:42:24.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.636 7f576a875700 1 -- 172.21.2.101:0/2646421640 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f57440186c0 msgr2=0x7f574401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:24.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.636 7f576a875700 1 --2- 172.21.2.101:0/2646421640 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f57440186c0 0x7f574401abc0 crc :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:24.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.636 7f576a875700 1 -- 172.21.2.101:0/2646421640 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f576419e9b0 msgr2=0x7f57641b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:24.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.636 7f576a875700 1 --2- 172.21.2.101:0/2646421640 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f576419e9b0 0x7f57641b0dc0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 rx=0x7f57580066a0 tx=0x7f575800b990).stop 2021-02-10T11:42:24.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.637 7f576a875700 1 -- 172.21.2.101:0/2646421640 shutdown_connections 2021-02-10T11:42:24.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.637 7f576a875700 1 --2- 172.21.2.101:0/2646421640 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f57440186c0 0x7f574401abc0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:24.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.637 7f576a875700 1 --2- 172.21.2.101:0/2646421640 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f57641b0250 0x7f57641b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:24.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.637 7f576a875700 1 --2- 172.21.2.101:0/2646421640 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f576419e9b0 0x7f57641b0dc0 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:24.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.637 7f576a875700 1 --2- 172.21.2.101:0/2646421640 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f576419e5b0 0x7f57641afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:24.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.637 7f576a875700 1 -- 172.21.2.101:0/2646421640 >> 172.21.2.101:0/2646421640 conn(0x7f576419f940 msgr2=0x7f576418c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:24.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.637 7f576a875700 1 -- 172.21.2.101:0/2646421640 shutdown_connections 2021-02-10T11:42:24.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:24.637 7f576a875700 1 -- 172.21.2.101:0/2646421640 wait complete. 2021-02-10T11:42:24.647 INFO:tasks.ceph.mds.a:Restarting daemon 2021-02-10T11:42:24.647 INFO:tasks.ceph.mds.a:Stopping old one... 2021-02-10T11:42:24.647 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-02-10T11:42:24.648 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:42:24.648 INFO:tasks.ceph.mds.b:Restarting daemon 2021-02-10T11:42:24.648 INFO:tasks.ceph.mds.b:Stopping old one... 2021-02-10T11:42:24.648 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-02-10T11:42:24.648 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:42:24.653 INFO:tasks.ceph.mds.a:Stopped 2021-02-10T11:42:24.654 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-02-10T11:42:24.654 INFO:tasks.ceph.mds.b:Stopped 2021-02-10T11:42:24.655 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-02-10T11:42:24.696 INFO:tasks.ceph.mds.a:Started 2021-02-10T11:42:24.696 DEBUG:teuthology.parallel:result is None 2021-02-10T11:42:24.696 INFO:tasks.ceph.mds.b:Started 2021-02-10T11:42:24.697 DEBUG:teuthology.parallel:result is None 2021-02-10T11:42:24.697 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd blacklist clear 2021-02-10T11:42:25.104 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:42:25.107 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:42:25.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.111 7f7b140c6700 1 Processor -- start 2021-02-10T11:42:25.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.111 7f7b140c6700 1 -- start start 2021-02-10T11:42:25.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.112 7f7b140c6700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7b0c0d1c70 0x7f7b0c0f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:25.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.112 7f7b140c6700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0c0f0b10 con 0x7f7b0c0d1870 2021-02-10T11:42:25.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.112 7f7b140c6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0c1b0be0 con 0x7f7b0c0d0340 2021-02-10T11:42:25.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.112 7f7b140c6700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0c0e5ed0 con 0x7f7b0c0d1c70 2021-02-10T11:42:25.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.112 7f7b11ee2700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f7b0c0d1870 0x7f7b0c1b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58540/0 (socket says 172.21.2.101:58540) 2021-02-10T11:42:25.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.112 7f7b11ee2700 1 -- 172.21.2.101:0/528755750 learned_addr learned my addr 172.21.2.101:0/528755750 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:25.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.112 7f7b126e3700 1 --2- 172.21.2.101:0/528755750 >> v2:172.21.2.108:3300/0 conn(0x7f7b0c0d1c70 0x7f7b0c0f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:25.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.112 7f7b10ee0700 1 -- 172.21.2.101:0/528755750 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3756637764 0 0) 0x7f7b080009b0 con 0x7f7b0c0d1870 2021-02-10T11:42:25.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.112 7f7b10ee0700 1 -- 172.21.2.101:0/528755750 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7af8002fe0 con 0x7f7b0c0d1870 2021-02-10T11:42:25.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.113 7f7b126e3700 1 -- 172.21.2.101:0/528755750 >> v1:172.21.2.108:6789/0 conn(0x7f7b0c0d0340 legacy=0x7f7b0c0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:25.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.113 7f7b126e3700 1 -- 172.21.2.101:0/528755750 >> v1:172.21.2.101:6789/0 conn(0x7f7b0c0d1870 legacy=0x7f7b0c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:25.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.113 7f7b126e3700 1 -- 172.21.2.101:0/528755750 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c0efe30 con 0x7f7b0c0d1c70 2021-02-10T11:42:25.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.113 7f7b126e3700 1 --2- 172.21.2.101:0/528755750 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7b0c0d1c70 0x7f7b0c0f20a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f7b04006e50 tx=0x7f7b04009e70).ready entity=mon.1 client_cookie=3608d990a5e48ffe server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:25.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.113 7f7b10ee0700 1 -- 172.21.2.101:0/528755750 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7b0400b510 con 0x7f7b0c0d1c70 2021-02-10T11:42:25.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.113 7f7b10ee0700 1 -- 172.21.2.101:0/528755750 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b04006790 con 0x7f7b0c0d1c70 2021-02-10T11:42:25.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.113 7f7b10ee0700 1 -- 172.21.2.101:0/528755750 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7b0400b940 con 0x7f7b0c0d1c70 2021-02-10T11:42:25.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.113 7f7b140c6700 1 -- 172.21.2.101:0/528755750 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7b0c0d1c70 msgr2=0x7f7b0c0f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:25.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.113 7f7b140c6700 1 --2- 172.21.2.101:0/528755750 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7b0c0d1c70 0x7f7b0c0f20a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f7b04006e50 tx=0x7f7b04009e70).stop 2021-02-10T11:42:25.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.113 7f7b140c6700 1 -- 172.21.2.101:0/528755750 shutdown_connections 2021-02-10T11:42:25.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.113 7f7b140c6700 1 --2- 172.21.2.101:0/528755750 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7b0c0d1c70 0x7f7b0c0f20a0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:25.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.113 7f7b140c6700 1 -- 172.21.2.101:0/528755750 >> 172.21.2.101:0/528755750 conn(0x7f7b0c0ce450 msgr2=0x7f7b0c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:25.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.114 7f7b140c6700 1 -- 172.21.2.101:0/528755750 shutdown_connections 2021-02-10T11:42:25.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.114 7f7b140c6700 1 -- 172.21.2.101:0/528755750 wait complete. 2021-02-10T11:42:25.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.128 7f7b140c6700 1 Processor -- start 2021-02-10T11:42:25.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.129 7f7b140c6700 1 -- start start 2021-02-10T11:42:25.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.129 7f7b140c6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b0c1a4990 0x7f7b0c0b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:25.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.129 7f7b140c6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7b0c0b2b50 0x7f7b0c09b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:25.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.129 7f7b140c6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7b0c09bfb0 0x7f7b0c189aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:25.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.129 7f7b140c6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c09c3b0 con 0x7f7b0c1a4990 2021-02-10T11:42:25.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.129 7f7b140c6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c18a0c0 con 0x7f7b0c09bfb0 2021-02-10T11:42:25.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.129 7f7b140c6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c18a2e0 con 0x7f7b0c0b2b50 2021-02-10T11:42:25.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.129 7f7b11ee2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b0c1a4990 0x7f7b0c0b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:25.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.129 7f7b11ee2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b0c1a4990 0x7f7b0c0b2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58052/0 (socket says 172.21.2.101:58052) 2021-02-10T11:42:25.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.129 7f7b11ee2700 1 -- 172.21.2.101:0/135720602 learned_addr learned my addr 172.21.2.101:0/135720602 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:25.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.129 7f7b11ee2700 1 -- 172.21.2.101:0/135720602 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7b0c0b2b50 msgr2=0x7f7b0c09b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:25.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.129 7f7b11ee2700 1 --2- 172.21.2.101:0/135720602 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7b0c0b2b50 0x7f7b0c09b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:25.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.129 7f7b11ee2700 1 -- 172.21.2.101:0/135720602 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7b0c09bfb0 msgr2=0x7f7b0c189aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:25.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.129 7f7b11ee2700 1 --2- 172.21.2.101:0/135720602 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7b0c09bfb0 0x7f7b0c189aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:25.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.129 7f7b11ee2700 1 -- 172.21.2.101:0/135720602 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c18a500 con 0x7f7b0c1a4990 2021-02-10T11:42:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.129 7f7b11ee2700 1 --2- 172.21.2.101:0/135720602 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b0c1a4990 0x7f7b0c0b2530 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 rx=0x7f7b08001c60 tx=0x7f7b0800a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.130 7f7afe7fc700 1 -- 172.21.2.101:0/135720602 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7b0800acc0 con 0x7f7b0c1a4990 2021-02-10T11:42:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.130 7f7afe7fc700 1 -- 172.21.2.101:0/135720602 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b08004140 con 0x7f7b0c1a4990 2021-02-10T11:42:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.130 7f7afe7fc700 1 -- 172.21.2.101:0/135720602 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7b080045e0 con 0x7f7b0c1a4990 2021-02-10T11:42:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.130 7f7b140c6700 1 -- 172.21.2.101:0/135720602 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0c18a500 con 0x7f7b0c1a4990 2021-02-10T11:42:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.130 7f7b140c6700 1 -- 172.21.2.101:0/135720602 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0c18a500 con 0x7f7b0c1a4990 2021-02-10T11:42:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.130 7f7afe7fc700 1 -- 172.21.2.101:0/135720602 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7b08004320 con 0x7f7b0c1a4990 2021-02-10T11:42:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.130 7f7afe7fc700 1 --2- 172.21.2.101:0/135720602 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7af40186c0 0x7f7af401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.130 7f7afe7fc700 1 -- 172.21.2.101:0/135720602 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 4839+0+0 (secure 0 0 0) 0x7f7b080320d0 con 0x7f7b0c1a4990 2021-02-10T11:42:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.130 7f7b116e1700 1 --2- 172.21.2.101:0/135720602 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7af40186c0 0x7f7af401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:25.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.131 7f7b116e1700 1 --2- 172.21.2.101:0/135720602 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7af40186c0 0x7f7af401abc0 crc :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:25.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.161 7f7b140c6700 1 -- 172.21.2.101:0/135720602 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0c0ee0a0 con 0x7f7b0c1a4990 2021-02-10T11:42:25.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.164 7f7afe7fc700 1 -- 172.21.2.101:0/135720602 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7b08000e20 con 0x7f7b0c1a4990 2021-02-10T11:42:25.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.303 7f7b140c6700 1 -- 172.21.2.101:0/135720602 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f7b0c0ce450 con 0x7f7b0c1a4990 2021-02-10T11:42:25.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.645 7f7afe7fc700 1 -- 172.21.2.101:0/135720602 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v124) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7b08004320 con 0x7f7b0c1a4990 2021-02-10T11:42:25.671 INFO:teuthology.orchestra.run.gibba001.stderr: removed all blacklist entries 2021-02-10T11:42:25.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.671 7f7b140c6700 1 -- 172.21.2.101:0/135720602 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7af40186c0 msgr2=0x7f7af401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:25.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.671 7f7b140c6700 1 --2- 172.21.2.101:0/135720602 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7af40186c0 0x7f7af401abc0 crc :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:25.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.671 7f7b140c6700 1 -- 172.21.2.101:0/135720602 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b0c1a4990 msgr2=0x7f7b0c0b2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:25.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.671 7f7b140c6700 1 --2- 172.21.2.101:0/135720602 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b0c1a4990 0x7f7b0c0b2530 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 rx=0x7f7b08001c60 tx=0x7f7b0800a8d0).stop 2021-02-10T11:42:25.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.671 7f7b140c6700 1 -- 172.21.2.101:0/135720602 shutdown_connections 2021-02-10T11:42:25.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.671 7f7b140c6700 1 --2- 172.21.2.101:0/135720602 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7af40186c0 0x7f7af401abc0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:25.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.671 7f7b140c6700 1 --2- 172.21.2.101:0/135720602 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7b0c0b2b50 0x7f7b0c09b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:25.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.671 7f7b140c6700 1 --2- 172.21.2.101:0/135720602 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7b0c09bfb0 0x7f7b0c189aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:25.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.671 7f7b140c6700 1 --2- 172.21.2.101:0/135720602 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7b0c1a4990 0x7f7b0c0b2530 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:25.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.671 7f7b140c6700 1 -- 172.21.2.101:0/135720602 >> 172.21.2.101:0/135720602 conn(0x7f7b0c0a6870 msgr2=0x7f7b0c08a420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:25.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.671 7f7b140c6700 1 -- 172.21.2.101:0/135720602 shutdown_connections 2021-02-10T11:42:25.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:25.672 7f7b140c6700 1 -- 172.21.2.101:0/135720602 wait complete. 2021-02-10T11:42:25.682 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2021-02-10T11:42:25.682 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:42:26.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.090 7fa8c5865700 1 Processor -- start 2021-02-10T11:42:26.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.090 7fa8c5865700 1 -- start start 2021-02-10T11:42:26.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.090 7fa8c5865700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa8c01b0470 0x7fa8c01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:26.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.090 7fa8c5865700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8c00ef610 0x7fa8c00efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:26.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.090 7fa8c5865700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa8c00f0030 0x7fa8c00f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:26.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.090 7fa8c5865700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c00e5ea0 con 0x7fa8c01b0470 2021-02-10T11:42:26.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.090 7fa8c5865700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c00f0b50 con 0x7fa8c00f0030 2021-02-10T11:42:26.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.090 7fa8c5865700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c00e52e0 con 0x7fa8c00ef610 2021-02-10T11:42:26.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.090 7fa8bf574700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa8c01b0470 0x7fa8c01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:26.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.091 7fa8bf574700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa8c01b0470 0x7fa8c01b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58060/0 (socket says 172.21.2.101:58060) 2021-02-10T11:42:26.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.091 7fa8bf574700 1 -- 172.21.2.101:0/2536407207 learned_addr learned my addr 172.21.2.101:0/2536407207 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:26.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.091 7fa8bf574700 1 -- 172.21.2.101:0/2536407207 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8c00ef610 msgr2=0x7fa8c00efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:26.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.091 7fa8bf574700 1 --2- 172.21.2.101:0/2536407207 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8c00ef610 0x7fa8c00efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:26.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.091 7fa8bf574700 1 -- 172.21.2.101:0/2536407207 >> v2:172.21.2.108:3300/0 conn(0x7fa8c00f0030 msgr2=0x7fa8c00f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:26.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.091 7fa8bf574700 1 --2- 172.21.2.101:0/2536407207 >> v2:172.21.2.108:3300/0 conn(0x7fa8c00f0030 0x7fa8c00f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:26.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.091 7fa8bf574700 1 -- 172.21.2.101:0/2536407207 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c00f10d0 con 0x7fa8c01b0470 2021-02-10T11:42:26.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.091 7fa8bf574700 1 --2- 172.21.2.101:0/2536407207 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8c01b0470 0x7fa8c01b0870 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 rx=0x7fa8b0006700 tx=0x7fa8b0009b80).ready entity=mon.0 client_cookie=859c640c26696410 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:26.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.092 7fa8be572700 1 -- 172.21.2.101:0/2536407207 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa8b000c0c0 con 0x7fa8c01b0470 2021-02-10T11:42:26.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.092 7fa8be572700 1 -- 172.21.2.101:0/2536407207 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b000c440 con 0x7fa8c01b0470 2021-02-10T11:42:26.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.092 7fa8be572700 1 -- 172.21.2.101:0/2536407207 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa8b000c960 con 0x7fa8c01b0470 2021-02-10T11:42:26.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.092 7fa8c5865700 1 -- 172.21.2.101:0/2536407207 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8c01b0470 msgr2=0x7fa8c01b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:26.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.092 7fa8c5865700 1 --2- 172.21.2.101:0/2536407207 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8c01b0470 0x7fa8c01b0870 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 rx=0x7fa8b0006700 tx=0x7fa8b0009b80).stop 2021-02-10T11:42:26.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.092 7fa8c5865700 1 -- 172.21.2.101:0/2536407207 shutdown_connections 2021-02-10T11:42:26.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.092 7fa8c5865700 1 --2- 172.21.2.101:0/2536407207 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8c00ef610 0x7fa8c00efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:26.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.092 7fa8c5865700 1 --2- 172.21.2.101:0/2536407207 >> v2:172.21.2.108:3300/0 conn(0x7fa8c00f0030 0x7fa8c00f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:26.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.092 7fa8c5865700 1 --2- 172.21.2.101:0/2536407207 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8c01b0470 0x7fa8c01b0870 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:26.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.092 7fa8c5865700 1 -- 172.21.2.101:0/2536407207 >> 172.21.2.101:0/2536407207 conn(0x7fa8c00ce3e0 msgr2=0x7fa8c00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:26.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.093 7fa8c5865700 1 -- 172.21.2.101:0/2536407207 shutdown_connections 2021-02-10T11:42:26.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.093 7fa8c5865700 1 -- 172.21.2.101:0/2536407207 wait complete. 2021-02-10T11:42:26.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.107 7fa8c5865700 1 Processor -- start 2021-02-10T11:42:26.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.107 7fa8c5865700 1 -- start start 2021-02-10T11:42:26.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.108 7fa8c5865700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8c01095b0 0x7fa8c019e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:26.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.108 7fa8c5865700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8c01099b0 0x7fa8c00933c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:26.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.108 7fa8c5865700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8c00939e0 0x7fa8c01ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:26.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.108 7fa8c5865700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c0093de0 con 0x7fa8c01099b0 2021-02-10T11:42:26.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.108 7fa8c5865700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c00aed40 con 0x7fa8c00939e0 2021-02-10T11:42:26.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.108 7fa8c5865700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c01ad420 con 0x7fa8c01095b0 2021-02-10T11:42:26.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.108 7fa8bed73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8c01099b0 0x7fa8c00933c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:26.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.108 7fa8bed73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8c01099b0 0x7fa8c00933c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58066/0 (socket says 172.21.2.101:58066) 2021-02-10T11:42:26.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.108 7fa8bed73700 1 -- 172.21.2.101:0/3427863591 learned_addr learned my addr 172.21.2.101:0/3427863591 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:26.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.108 7fa8bf574700 1 --2- 172.21.2.101:0/3427863591 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8c01095b0 0x7fa8c019e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:26.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.108 7fa8bed73700 1 -- 172.21.2.101:0/3427863591 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8c01095b0 msgr2=0x7fa8c019e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:26.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.108 7fa8bed73700 1 --2- 172.21.2.101:0/3427863591 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8c01095b0 0x7fa8c019e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:26.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.108 7fa8bed73700 1 -- 172.21.2.101:0/3427863591 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8c00939e0 msgr2=0x7fa8c01ace00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:26.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.108 7fa8bfd75700 1 --2- 172.21.2.101:0/3427863591 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8c00939e0 0x7fa8c01ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:26.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.108 7fa8bed73700 1 --2- 172.21.2.101:0/3427863591 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8c00939e0 0x7fa8c01ace00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:26.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.108 7fa8bed73700 1 -- 172.21.2.101:0/3427863591 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c01ad640 con 0x7fa8c01099b0 2021-02-10T11:42:26.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.108 7fa8bf574700 1 --2- 172.21.2.101:0/3427863591 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8c01095b0 0x7fa8c019e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:42:26.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.108 7fa8bfd75700 1 --2- 172.21.2.101:0/3427863591 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8c00939e0 0x7fa8c01ace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:42:26.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.108 7fa8bed73700 1 --2- 172.21.2.101:0/3427863591 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8c01099b0 0x7fa8c00933c0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 rx=0x7fa8b400b5b0 tx=0x7fa8b400b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:26.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.109 7fa8a7fff700 1 -- 172.21.2.101:0/3427863591 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa8b400bd40 con 0x7fa8c01099b0 2021-02-10T11:42:26.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.109 7fa8a7fff700 1 -- 172.21.2.101:0/3427863591 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b40009b0 con 0x7fa8c01099b0 2021-02-10T11:42:26.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.109 7fa8a7fff700 1 -- 172.21.2.101:0/3427863591 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa8b4002300 con 0x7fa8c01099b0 2021-02-10T11:42:26.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.109 7fa8c5865700 1 -- 172.21.2.101:0/3427863591 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c01ad640 con 0x7fa8c01099b0 2021-02-10T11:42:26.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.109 7fa8c5865700 1 -- 172.21.2.101:0/3427863591 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c01adbd0 con 0x7fa8c01099b0 2021-02-10T11:42:26.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.109 7fa8a7fff700 1 -- 172.21.2.101:0/3427863591 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa8b4002020 con 0x7fa8c01099b0 2021-02-10T11:42:26.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.109 7fa8a7fff700 1 --2- 172.21.2.101:0/3427863591 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8a00186c0 0x7fa8a001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:26.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.109 7fa8a7fff700 1 -- 172.21.2.101:0/3427863591 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fa8b4032080 con 0x7fa8c01099b0 2021-02-10T11:42:26.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.110 7fa8bf574700 1 --2- 172.21.2.101:0/3427863591 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8a00186c0 0x7fa8a001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:26.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.110 7fa8bf574700 1 --2- 172.21.2.101:0/3427863591 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8a00186c0 0x7fa8a001abc0 crc :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:26.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.140 7fa8c5865700 1 -- 172.21.2.101:0/3427863591 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c00f0b50 con 0x7fa8c01099b0 2021-02-10T11:42:26.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.143 7fa8a7fff700 1 -- 172.21.2.101:0/3427863591 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa8b4002da0 con 0x7fa8c01099b0 2021-02-10T11:42:26.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.274 7fa8c5865700 1 -- 172.21.2.101:0/3427863591 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa8c00f0b50 con 0x7fa8c01099b0 2021-02-10T11:42:26.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.275 7fa8a7fff700 1 -- 172.21.2.101:0/3427863591 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v74) v1 ==== 80+0+4513 (secure 0 0 0) 0x7fa8b400bd40 con 0x7fa8c01099b0 2021-02-10T11:42:26.277 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:42:26.278 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:42:26.278 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:42:26.278 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:26.278 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:42:26.278 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:42:26.278 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:42:26.278 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:42:26.278 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:42:26.278 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:42:26.278 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:42:26.278 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:26.278 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:26.279 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:26.279 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:42:26.279 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:42:26.279 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:42:26.279 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:42:26.279 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:42:26.279 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:42:26.279 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:42:26.280 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:26.280 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:26.280 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:26.280 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:42:26.280 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:42:26.280 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:42:26.280 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:42:26.280 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:42:26.280 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:42:26.280 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:26.280 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:26.280 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:26.281 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:42:26.281 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:42:26.281 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:42:26.281 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:42:26.281 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:42:26.281 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:42:26.281 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:26.281 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:26.281 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:26.281 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:42:26.281 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:42:26.281 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:42:26.281 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:42:26.282 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:42:26.282 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:42:26.282 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:26.282 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:26.282 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:26.282 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:42:26.282 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:42:26.283 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:42:26.283 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:42:26.283 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:42:26.283 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:42:26.283 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:26.283 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:26.283 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:26.283 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:42:26.283 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:42:26.283 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:42:26.283 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:42:26.283 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:42:26.283 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:42:26.284 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:26.284 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:26.284 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:26.284 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:42:26.284 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:42:26.284 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:42:26.284 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:42:26.284 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:42:26.284 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:42:26.284 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:26.284 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:26.284 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:26.285 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:42:26.285 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:42:26.285 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:42:26.285 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:42:26.285 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:42:26.285 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:42:26.285 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:26.285 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:26.285 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:26.286 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:42:26.286 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:42:26.286 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:42:26.286 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:42:26.286 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:42:26.286 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:42:26.286 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:26.286 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:26.286 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:26.287 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:42:26.287 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:42:26.287 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:42:26.287 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:42:26.287 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:42:26.287 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:42:26.287 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:26.288 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:26.288 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:26.288 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:42:26.288 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:42:26.288 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:42:26.288 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:42:26.288 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:42:26.289 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:42:26.289 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:26.289 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:26.289 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:26.289 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:42:26.289 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:42:26.290 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:42:26.290 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:42:26.290 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:42:26.290 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:42:26.290 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:26.290 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:26.291 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:26.291 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:42:26.291 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:42:26.291 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:42:26.291 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:42:26.291 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:42:26.292 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:42:26.292 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:26.293 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:26.293 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:26.293 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:42:26.293 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:42:26.293 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:42:26.293 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:42:26.294 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:42:26.294 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:42:26.294 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:42:26.294 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:26.294 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:26.295 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:26.295 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:42:26.295 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:42:26.295 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:42:26.295 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:42:26.295 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:42:26.296 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:42:26.296 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:26.296 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:26.296 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:26.296 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:42:26.296 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:42:26.297 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:42:26.297 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:42:26.297 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:42:26.297 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:42:26.297 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:26.298 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:26.298 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:42:26.298 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:42:26.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.277 7fa8c5865700 1 -- 172.21.2.101:0/3427863591 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8a00186c0 msgr2=0x7fa8a001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:26.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.277 7fa8c5865700 1 --2- 172.21.2.101:0/3427863591 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8a00186c0 0x7fa8a001abc0 crc :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:26.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.277 7fa8c5865700 1 -- 172.21.2.101:0/3427863591 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8c01099b0 msgr2=0x7fa8c00933c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:26.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.277 7fa8c5865700 1 --2- 172.21.2.101:0/3427863591 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8c01099b0 0x7fa8c00933c0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 rx=0x7fa8b400b5b0 tx=0x7fa8b400b950).stop 2021-02-10T11:42:26.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.277 7fa8c5865700 1 -- 172.21.2.101:0/3427863591 shutdown_connections 2021-02-10T11:42:26.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.277 7fa8c5865700 1 --2- 172.21.2.101:0/3427863591 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8c00939e0 0x7fa8c01ace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:26.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.277 7fa8c5865700 1 --2- 172.21.2.101:0/3427863591 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8c01099b0 0x7fa8c00933c0 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:26.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.277 7fa8c5865700 1 --2- 172.21.2.101:0/3427863591 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8a00186c0 0x7fa8a001abc0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:26.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.277 7fa8c5865700 1 --2- 172.21.2.101:0/3427863591 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8c01095b0 0x7fa8c019e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:26.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.277 7fa8c5865700 1 -- 172.21.2.101:0/3427863591 >> 172.21.2.101:0/3427863591 conn(0x7fa8c019f7b0 msgr2=0x7fa8c018f380 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:26.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.277 7fa8c5865700 1 -- 172.21.2.101:0/3427863591 shutdown_connections 2021-02-10T11:42:26.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.277 7fa8c5865700 1 -- 172.21.2.101:0/3427863591 wait complete. 2021-02-10T11:42:26.302 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:42:26.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.712 7fb6fb9ce700 1 Processor -- start 2021-02-10T11:42:26.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.713 7fb6fb9ce700 1 -- start start 2021-02-10T11:42:26.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.713 7fb6fb9ce700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6f40d1870 0x7fb6f41b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:26.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.713 7fb6fb9ce700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb6f41b0a10 0x7fb6f40cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:26.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.713 7fb6fb9ce700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f40f0b10 con 0x7fb6f40d1c70 2021-02-10T11:42:26.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.713 7fb6fb9ce700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f40fe230 con 0x7fb6f41b0a10 2021-02-10T11:42:26.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.713 7fb6fb9ce700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f40e5ed0 con 0x7fb6f40d1870 2021-02-10T11:42:26.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.713 7fb6f9feb700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb6f40d1c70 0x7fb6f40cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58566/0 (socket says 172.21.2.101:58566) 2021-02-10T11:42:26.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.713 7fb6f9feb700 1 -- 172.21.2.101:0/2200609944 learned_addr learned my addr 172.21.2.101:0/2200609944 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:26.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.713 7fb6f8fe9700 1 --2- 172.21.2.101:0/2200609944 >> v2:172.21.2.101:3300/0 conn(0x7fb6f41b0a10 0x7fb6f40cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:26.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.714 7fb6f8fe9700 1 -- 172.21.2.101:0/2200609944 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6f40d1870 msgr2=0x7fb6f41b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:26.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.714 7fb6f8fe9700 1 --2- 172.21.2.101:0/2200609944 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6f40d1870 0x7fb6f41b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:26.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.714 7fb6f8fe9700 1 -- 172.21.2.101:0/2200609944 >> v1:172.21.2.101:6789/0 conn(0x7fb6f40d1c70 legacy=0x7fb6f40cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:26.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.714 7fb6f8fe9700 1 -- 172.21.2.101:0/2200609944 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f40f0650 con 0x7fb6f41b0a10 2021-02-10T11:42:26.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.714 7fb6f8fe9700 1 --2- 172.21.2.101:0/2200609944 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6f41b0a10 0x7fb6f40cf330 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 rx=0x7fb6f0006650 tx=0x7fb6f0009a80).ready entity=mon.0 client_cookie=72166e5bf4f6b7dd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:26.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.714 7fb6e7fff700 1 -- 172.21.2.101:0/2200609944 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb6f00009b0 con 0x7fb6f41b0a10 2021-02-10T11:42:26.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.714 7fb6e7fff700 1 -- 172.21.2.101:0/2200609944 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f000c0c0 con 0x7fb6f41b0a10 2021-02-10T11:42:26.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.714 7fb6e7fff700 1 -- 172.21.2.101:0/2200609944 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb6f000c520 con 0x7fb6f41b0a10 2021-02-10T11:42:26.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.714 7fb6fb9ce700 1 -- 172.21.2.101:0/2200609944 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6f41b0a10 msgr2=0x7fb6f40cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:26.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.714 7fb6fb9ce700 1 --2- 172.21.2.101:0/2200609944 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6f41b0a10 0x7fb6f40cf330 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 rx=0x7fb6f0006650 tx=0x7fb6f0009a80).stop 2021-02-10T11:42:26.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.715 7fb6fb9ce700 1 -- 172.21.2.101:0/2200609944 shutdown_connections 2021-02-10T11:42:26.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.715 7fb6fb9ce700 1 --2- 172.21.2.101:0/2200609944 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6f41b0a10 0x7fb6f40cf330 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:26.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.715 7fb6fb9ce700 1 --2- 172.21.2.101:0/2200609944 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6f40d1870 0x7fb6f41b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:26.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.715 7fb6fb9ce700 1 -- 172.21.2.101:0/2200609944 >> 172.21.2.101:0/2200609944 conn(0x7fb6f40ce450 msgr2=0x7fb6f40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:26.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.715 7fb6fb9ce700 1 -- 172.21.2.101:0/2200609944 shutdown_connections 2021-02-10T11:42:26.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.715 7fb6fb9ce700 1 -- 172.21.2.101:0/2200609944 wait complete. 2021-02-10T11:42:26.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.730 7fb6fb9ce700 1 Processor -- start 2021-02-10T11:42:26.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.730 7fb6fb9ce700 1 -- start start 2021-02-10T11:42:26.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.730 7fb6fb9ce700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6f40b0ad0 0x7fb6f4091f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:26.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.730 7fb6fb9ce700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb6f41a8850 0x7fb6f41a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:26.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.730 7fb6fb9ce700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6f41a6fa0 0x7fb6f41a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:26.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.730 7fb6fb9ce700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f41abfe0 con 0x7fb6f40b0ad0 2021-02-10T11:42:26.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.730 7fb6fb9ce700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f41a9c40 con 0x7fb6f41a8850 2021-02-10T11:42:26.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.730 7fb6fb9ce700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f41a9730 con 0x7fb6f41a6fa0 2021-02-10T11:42:26.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.731 7fb6f97ea700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6f40b0ad0 0x7fb6f4091f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:26.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.731 7fb6f97ea700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6f40b0ad0 0x7fb6f4091f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58078/0 (socket says 172.21.2.101:58078) 2021-02-10T11:42:26.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.731 7fb6f97ea700 1 -- 172.21.2.101:0/966486792 learned_addr learned my addr 172.21.2.101:0/966486792 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:26.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.731 7fb6f9feb700 1 --2- 172.21.2.101:0/966486792 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6f41a6fa0 0x7fb6f41a73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:26.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.731 7fb6f8fe9700 1 --2- 172.21.2.101:0/966486792 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb6f41a8850 0x7fb6f41a8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:26.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.731 7fb6f97ea700 1 -- 172.21.2.101:0/966486792 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6f41a6fa0 msgr2=0x7fb6f41a73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:26.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.731 7fb6f97ea700 1 --2- 172.21.2.101:0/966486792 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6f41a6fa0 0x7fb6f41a73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:26.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.731 7fb6f97ea700 1 -- 172.21.2.101:0/966486792 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb6f41a8850 msgr2=0x7fb6f41a8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:26.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.731 7fb6f97ea700 1 --2- 172.21.2.101:0/966486792 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb6f41a8850 0x7fb6f41a8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:26.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.731 7fb6f97ea700 1 -- 172.21.2.101:0/966486792 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f41a9270 con 0x7fb6f40b0ad0 2021-02-10T11:42:26.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.731 7fb6f8fe9700 1 --2- 172.21.2.101:0/966486792 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb6f41a8850 0x7fb6f41a8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:42:26.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.731 7fb6f97ea700 1 --2- 172.21.2.101:0/966486792 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6f40b0ad0 0x7fb6f4091f30 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 rx=0x7fb6e8006f00 tx=0x7fb6e800b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:26.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.731 7fb6e5ffb700 1 -- 172.21.2.101:0/966486792 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb6e800bd30 con 0x7fb6f40b0ad0 2021-02-10T11:42:26.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.731 7fb6e5ffb700 1 -- 172.21.2.101:0/966486792 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e8000c90 con 0x7fb6f40b0ad0 2021-02-10T11:42:26.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.731 7fb6fb9ce700 1 -- 172.21.2.101:0/966486792 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f41a9270 con 0x7fb6f40b0ad0 2021-02-10T11:42:26.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.731 7fb6e5ffb700 1 -- 172.21.2.101:0/966486792 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb6e80022a0 con 0x7fb6f40b0ad0 2021-02-10T11:42:26.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.731 7fb6fb9ce700 1 -- 172.21.2.101:0/966486792 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f41a9270 con 0x7fb6f40b0ad0 2021-02-10T11:42:26.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.732 7fb6e5ffb700 1 -- 172.21.2.101:0/966486792 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb6e8005310 con 0x7fb6f40b0ad0 2021-02-10T11:42:26.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.732 7fb6e5ffb700 1 --2- 172.21.2.101:0/966486792 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb6d40186c0 0x7fb6d401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:26.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.732 7fb6e5ffb700 1 -- 172.21.2.101:0/966486792 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fb6e80320a0 con 0x7fb6f40b0ad0 2021-02-10T11:42:26.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.732 7fb6f8fe9700 1 --2- 172.21.2.101:0/966486792 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb6d40186c0 0x7fb6d401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:26.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.732 7fb6f8fe9700 1 --2- 172.21.2.101:0/966486792 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb6d40186c0 0x7fb6d401abc0 crc :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:26.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.763 7fb6fb9ce700 1 -- 172.21.2.101:0/966486792 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f41abfe0 con 0x7fb6f40b0ad0 2021-02-10T11:42:26.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.765 7fb6e5ffb700 1 -- 172.21.2.101:0/966486792 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb6e8006b90 con 0x7fb6f40b0ad0 2021-02-10T11:42:26.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.891 7fb6fb9ce700 1 -- 172.21.2.101:0/966486792 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6f41abfe0 con 0x7fb6f40b0ad0 2021-02-10T11:42:26.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.891 7fb6e5ffb700 1 -- 172.21.2.101:0/966486792 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+11154 (secure 0 0 0) 0x7fb6e8005310 con 0x7fb6f40b0ad0 2021-02-10T11:42:26.893 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:42:26.894 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":124,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:42:25.642736","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:42:26.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.893 7fb6fb9ce700 1 -- 172.21.2.101:0/966486792 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb6d40186c0 msgr2=0x7fb6d401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:26.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.893 7fb6fb9ce700 1 --2- 172.21.2.101:0/966486792 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb6d40186c0 0x7fb6d401abc0 crc :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:26.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.893 7fb6fb9ce700 1 -- 172.21.2.101:0/966486792 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6f40b0ad0 msgr2=0x7fb6f4091f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:26.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.893 7fb6fb9ce700 1 --2- 172.21.2.101:0/966486792 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6f40b0ad0 0x7fb6f4091f30 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 rx=0x7fb6e8006f00 tx=0x7fb6e800b940).stop 2021-02-10T11:42:26.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.893 7fb6fb9ce700 1 -- 172.21.2.101:0/966486792 shutdown_connections 2021-02-10T11:42:26.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.893 7fb6fb9ce700 1 --2- 172.21.2.101:0/966486792 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb6d40186c0 0x7fb6d401abc0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:26.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.893 7fb6fb9ce700 1 --2- 172.21.2.101:0/966486792 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6f41a6fa0 0x7fb6f41a73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:26.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.893 7fb6fb9ce700 1 --2- 172.21.2.101:0/966486792 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb6f41a8850 0x7fb6f41a8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:26.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.893 7fb6fb9ce700 1 --2- 172.21.2.101:0/966486792 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6f40b0ad0 0x7fb6f4091f30 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:26.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.893 7fb6fb9ce700 1 -- 172.21.2.101:0/966486792 >> 172.21.2.101:0/966486792 conn(0x7fb6f419f8f0 msgr2=0x7fb6f41a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:26.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.893 7fb6fb9ce700 1 -- 172.21.2.101:0/966486792 shutdown_connections 2021-02-10T11:42:26.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:26.893 7fb6fb9ce700 1 -- 172.21.2.101:0/966486792 wait complete. 2021-02-10T11:42:26.904 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-10T11:42:26.904 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:42:27.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.317 7f98981a7700 1 Processor -- start 2021-02-10T11:42:27.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.317 7f98981a7700 1 -- start start 2021-02-10T11:42:27.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.318 7f98981a7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f98900d1870 0x7f98901b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:27.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.318 7f98981a7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98900cff70 0x7f9890109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:27.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.318 7f98981a7700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9890109a60 con 0x7f98901b0ae0 2021-02-10T11:42:27.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.318 7f98981a7700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9890109750 con 0x7f98900d1870 2021-02-10T11:42:27.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.318 7f98981a7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9890088070 con 0x7f98900cff70 2021-02-10T11:42:27.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.318 7f9895fc3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f98900d1870 0x7f98901b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:27.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.318 7f9895fc3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f98900d1870 0x7f98901b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58084/0 (socket says 172.21.2.101:58084) 2021-02-10T11:42:27.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.318 7f9895fc3700 1 -- 172.21.2.101:0/3320301116 learned_addr learned my addr 172.21.2.101:0/3320301116 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:27.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.318 7f9895fc3700 1 -- 172.21.2.101:0/3320301116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98900cff70 msgr2=0x7f9890109130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:42:27.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.318 7f9895fc3700 1 --2- 172.21.2.101:0/3320301116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98900cff70 0x7f9890109130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:27.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.318 7f9895fc3700 1 -- 172.21.2.101:0/3320301116 >> v1:172.21.2.108:6789/0 conn(0x7f98901b0ae0 legacy=0x7f9890108940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:42:27.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.318 7f9895fc3700 1 -- 172.21.2.101:0/3320301116 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98900d1c70 con 0x7f98900d1870 2021-02-10T11:42:27.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.318 7f9895fc3700 1 --2- 172.21.2.101:0/3320301116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98900d1870 0x7f98901b04c0 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 rx=0x7f988c006700 tx=0x7f988c009b80).ready entity=mon.0 client_cookie=77e144a66d40ced7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:27.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.319 7f9894fc1700 1 -- 172.21.2.101:0/3320301116 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f988c00c0c0 con 0x7f98900d1870 2021-02-10T11:42:27.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.319 7f9894fc1700 1 -- 172.21.2.101:0/3320301116 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988c00c340 con 0x7f98900d1870 2021-02-10T11:42:27.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.319 7f9894fc1700 1 -- 172.21.2.101:0/3320301116 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f988c00c860 con 0x7f98900d1870 2021-02-10T11:42:27.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.319 7f98981a7700 1 -- 172.21.2.101:0/3320301116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98900d1870 msgr2=0x7f98901b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:27.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.319 7f98981a7700 1 --2- 172.21.2.101:0/3320301116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98900d1870 0x7f98901b04c0 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 rx=0x7f988c006700 tx=0x7f988c009b80).stop 2021-02-10T11:42:27.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.319 7f98981a7700 1 -- 172.21.2.101:0/3320301116 shutdown_connections 2021-02-10T11:42:27.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.319 7f98981a7700 1 --2- 172.21.2.101:0/3320301116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98900cff70 0x7f9890109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:27.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.319 7f98981a7700 1 --2- 172.21.2.101:0/3320301116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98900d1870 0x7f98901b04c0 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:27.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.319 7f98981a7700 1 -- 172.21.2.101:0/3320301116 >> 172.21.2.101:0/3320301116 conn(0x7f98900ce420 msgr2=0x7f98900d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:27.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.320 7f98981a7700 1 -- 172.21.2.101:0/3320301116 shutdown_connections 2021-02-10T11:42:27.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.320 7f98981a7700 1 -- 172.21.2.101:0/3320301116 wait complete. 2021-02-10T11:42:27.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.334 7f98981a7700 1 Processor -- start 2021-02-10T11:42:27.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.334 7f98981a7700 1 -- start start 2021-02-10T11:42:27.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.334 7f98981a7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98900c3820 0x7f9890084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:27.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.334 7f98981a7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98900a0290 0x7f98900852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:27.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.334 7f98981a7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9890086270 0x7f9890086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:27.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.334 7f98981a7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98900c0700 con 0x7f9890086270 2021-02-10T11:42:27.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.334 7f98981a7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98900858f0 con 0x7f98900a0290 2021-02-10T11:42:27.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.334 7f98981a7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9890085b10 con 0x7f98900c3820 2021-02-10T11:42:27.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.334 7f98967c4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9890086270 0x7f9890086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:27.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.335 7f98967c4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9890086270 0x7f9890086670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58088/0 (socket says 172.21.2.101:58088) 2021-02-10T11:42:27.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.335 7f98967c4700 1 -- 172.21.2.101:0/2685366332 learned_addr learned my addr 172.21.2.101:0/2685366332 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:27.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.335 7f98967c4700 1 -- 172.21.2.101:0/2685366332 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98900c3820 msgr2=0x7f9890084cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:27.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.335 7f98967c4700 1 --2- 172.21.2.101:0/2685366332 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98900c3820 0x7f9890084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:27.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.335 7f98967c4700 1 -- 172.21.2.101:0/2685366332 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98900a0290 msgr2=0x7f98900852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:27.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.335 7f98967c4700 1 --2- 172.21.2.101:0/2685366332 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98900a0290 0x7f98900852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:27.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.335 7f98967c4700 1 -- 172.21.2.101:0/2685366332 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9890086c90 con 0x7f9890086270 2021-02-10T11:42:27.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.335 7f98967c4700 1 --2- 172.21.2.101:0/2685366332 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9890086270 0x7f9890086670 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 rx=0x7f9888009960 tx=0x7f988800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:27.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.335 7f98827fc700 1 -- 172.21.2.101:0/2685366332 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9888000d10 con 0x7f9890086270 2021-02-10T11:42:27.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.335 7f98827fc700 1 -- 172.21.2.101:0/2685366332 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9888003020 con 0x7f9890086270 2021-02-10T11:42:27.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.335 7f98827fc700 1 -- 172.21.2.101:0/2685366332 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9888005ad0 con 0x7f9890086270 2021-02-10T11:42:27.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.335 7f98981a7700 1 -- 172.21.2.101:0/2685366332 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9890086c90 con 0x7f9890086270 2021-02-10T11:42:27.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.335 7f98981a7700 1 -- 172.21.2.101:0/2685366332 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9890086c90 con 0x7f9890086270 2021-02-10T11:42:27.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.336 7f98827fc700 1 -- 172.21.2.101:0/2685366332 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9888000d10 con 0x7f9890086270 2021-02-10T11:42:27.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.336 7f98827fc700 1 --2- 172.21.2.101:0/2685366332 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98780186c0 0x7f987801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:27.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.336 7f98827fc700 1 -- 172.21.2.101:0/2685366332 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f9888032610 con 0x7f9890086270 2021-02-10T11:42:27.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.336 7f9895fc3700 1 --2- 172.21.2.101:0/2685366332 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98780186c0 0x7f987801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:27.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.336 7f9895fc3700 1 --2- 172.21.2.101:0/2685366332 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98780186c0 0x7f987801abc0 crc :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:27.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.367 7f98981a7700 1 -- 172.21.2.101:0/2685366332 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9890082a70 con 0x7f9890086270 2021-02-10T11:42:27.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.370 7f98827fc700 1 -- 172.21.2.101:0/2685366332 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9888026fe0 con 0x7f9890086270 2021-02-10T11:42:27.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.498 7f98981a7700 1 -- 172.21.2.101:0/2685366332 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f9890082a70 con 0x7f9890086270 2021-02-10T11:42:27.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.499 7f98827fc700 1 -- 172.21.2.101:0/2685366332 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v124) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9888026fe0 con 0x7f9890086270 2021-02-10T11:42:27.501 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:42:27.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.501 7f98981a7700 1 -- 172.21.2.101:0/2685366332 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98780186c0 msgr2=0x7f987801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:27.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.501 7f98981a7700 1 --2- 172.21.2.101:0/2685366332 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98780186c0 0x7f987801abc0 crc :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:27.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.501 7f98981a7700 1 -- 172.21.2.101:0/2685366332 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9890086270 msgr2=0x7f9890086670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:27.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.501 7f98981a7700 1 --2- 172.21.2.101:0/2685366332 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9890086270 0x7f9890086670 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 rx=0x7f9888009960 tx=0x7f988800bcf0).stop 2021-02-10T11:42:27.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.501 7f98981a7700 1 -- 172.21.2.101:0/2685366332 shutdown_connections 2021-02-10T11:42:27.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.501 7f98981a7700 1 --2- 172.21.2.101:0/2685366332 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9890086270 0x7f9890086670 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:27.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.501 7f98981a7700 1 --2- 172.21.2.101:0/2685366332 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98900a0290 0x7f98900852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:27.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.501 7f98981a7700 1 --2- 172.21.2.101:0/2685366332 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f98780186c0 0x7f987801abc0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:27.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.501 7f98981a7700 1 --2- 172.21.2.101:0/2685366332 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98900c3820 0x7f9890084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:27.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.501 7f98981a7700 1 -- 172.21.2.101:0/2685366332 >> 172.21.2.101:0/2685366332 conn(0x7f98900872c0 msgr2=0x7f9890095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:27.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.501 7f98981a7700 1 -- 172.21.2.101:0/2685366332 shutdown_connections 2021-02-10T11:42:27.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.501 7f98981a7700 1 -- 172.21.2.101:0/2685366332 wait complete. 2021-02-10T11:42:27.511 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --format=json-pretty osd lspools 2021-02-10T11:42:27.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.921 7f9b9a6e6700 1 Processor -- start 2021-02-10T11:42:27.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.922 7f9b9a6e6700 1 -- start start 2021-02-10T11:42:27.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.922 7f9b9a6e6700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f9b940d1870 0x7f9b941b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:27.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.922 7f9b9a6e6700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b940f0b10 con 0x7f9b941b0a10 2021-02-10T11:42:27.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.922 7f9b9a6e6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b940d0310 con 0x7f9b940f0650 2021-02-10T11:42:27.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.922 7f9b9a6e6700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b940d0530 con 0x7f9b940d1870 2021-02-10T11:42:27.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.922 7f9b937fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f9b941b0a10 0x7f9b940cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58590/0 (socket says 172.21.2.101:58590) 2021-02-10T11:42:27.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.922 7f9b937fe700 1 -- 172.21.2.101:0/315899692 learned_addr learned my addr 172.21.2.101:0/315899692 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:27.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b92ffd700 1 -- 172.21.2.101:0/315899692 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 625394361 0 0) 0x7f9b880010b0 con 0x7f9b941b0a10 2021-02-10T11:42:27.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b92ffd700 1 -- 172.21.2.101:0/315899692 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b7c002fe0 con 0x7f9b941b0a10 2021-02-10T11:42:27.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b93fff700 1 --2- 172.21.2.101:0/315899692 >> v2:172.21.2.108:3300/0 conn(0x7f9b940d1870 0x7f9b941b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:27.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b92ffd700 1 -- 172.21.2.101:0/315899692 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3237238795 0 0) 0x7f9b880010b0 con 0x7f9b941b0a10 2021-02-10T11:42:27.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b92ffd700 1 -- 172.21.2.101:0/315899692 >> v2:172.21.2.108:3300/0 conn(0x7f9b940d1870 msgr2=0x7f9b941b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:27.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b92ffd700 1 --2- 172.21.2.101:0/315899692 >> v2:172.21.2.108:3300/0 conn(0x7f9b940d1870 0x7f9b941b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:27.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b92ffd700 1 -- 172.21.2.101:0/315899692 >> v1:172.21.2.108:6789/0 conn(0x7f9b940f0650 legacy=0x7f9b940cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:27.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b92ffd700 1 -- 172.21.2.101:0/315899692 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b940d1c70 con 0x7f9b941b0a10 2021-02-10T11:42:27.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b92ffd700 1 -- 172.21.2.101:0/315899692 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f9b880015b0 con 0x7f9b941b0a10 2021-02-10T11:42:27.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b92ffd700 1 -- 172.21.2.101:0/315899692 >> v1:172.21.2.101:6789/0 conn(0x7f9b941b0a10 legacy=0x7f9b940cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:27.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b92ffd700 1 --2- 172.21.2.101:0/315899692 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b7c003b90 0x7f9b7c005fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:27.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b92ffd700 1 --2- 172.21.2.101:0/315899692 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b7c006860 0x7f9b7c008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:27.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b92ffd700 1 --2- 172.21.2.101:0/315899692 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b7c009640 0x7f9b7c00bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:27.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b92ffd700 1 -- 172.21.2.101:0/315899692 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c00c3b0 con 0x7f9b7c003b90 2021-02-10T11:42:27.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b92ffd700 1 -- 172.21.2.101:0/315899692 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c00c5d0 con 0x7f9b7c009640 2021-02-10T11:42:27.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b92ffd700 1 -- 172.21.2.101:0/315899692 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c00c7f0 con 0x7f9b7c006860 2021-02-10T11:42:27.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b93fff700 1 --2- 172.21.2.101:0/315899692 >> v2:172.21.2.108:3300/0 conn(0x7f9b940d1870 0x7f9b941b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:42:27.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b93fff700 1 --2- 172.21.2.101:0/315899692 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b7c003b90 0x7f9b7c005fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:27.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b937fe700 1 --2- 172.21.2.101:0/315899692 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b7c006860 0x7f9b7c008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:27.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b98d03700 1 --2- 172.21.2.101:0/315899692 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b7c009640 0x7f9b7c00bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:27.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b93fff700 1 -- 172.21.2.101:0/315899692 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b7c006860 msgr2=0x7f9b7c008d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:27.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b93fff700 1 --2- 172.21.2.101:0/315899692 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b7c006860 0x7f9b7c008d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:27.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b93fff700 1 -- 172.21.2.101:0/315899692 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b7c009640 msgr2=0x7f9b7c00bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:27.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b93fff700 1 --2- 172.21.2.101:0/315899692 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b7c009640 0x7f9b7c00bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:27.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b93fff700 1 -- 172.21.2.101:0/315899692 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7c00ca90 con 0x7f9b7c003b90 2021-02-10T11:42:27.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b98d03700 1 --2- 172.21.2.101:0/315899692 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b7c009640 0x7f9b7c00bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:42:27.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.923 7f9b98d03700 1 -- 172.21.2.101:0/315899692 reap_dead start 2021-02-10T11:42:27.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.924 7f9b93fff700 1 --2- 172.21.2.101:0/315899692 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b7c003b90 0x7f9b7c005fc0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 rx=0x7f9b84006530 tx=0x7f9b84050cc0).ready entity=mon.0 client_cookie=c6f99517bb737be1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:27.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.924 7f9b92ffd700 1 -- 172.21.2.101:0/315899692 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9b84006940 con 0x7f9b7c003b90 2021-02-10T11:42:27.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.924 7f9b92ffd700 1 -- 172.21.2.101:0/315899692 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b84006cc0 con 0x7f9b7c003b90 2021-02-10T11:42:27.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.924 7f9b92ffd700 1 -- 172.21.2.101:0/315899692 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9b84002220 con 0x7f9b7c003b90 2021-02-10T11:42:27.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.924 7f9b9a6e6700 1 -- 172.21.2.101:0/315899692 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b7c003b90 msgr2=0x7f9b7c005fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:27.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.924 7f9b9a6e6700 1 --2- 172.21.2.101:0/315899692 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b7c003b90 0x7f9b7c005fc0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 rx=0x7f9b84006530 tx=0x7f9b84050cc0).stop 2021-02-10T11:42:27.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.924 7f9b9a6e6700 1 -- 172.21.2.101:0/315899692 shutdown_connections 2021-02-10T11:42:27.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.924 7f9b9a6e6700 1 --2- 172.21.2.101:0/315899692 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b7c003b90 0x7f9b7c005fc0 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:27.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.924 7f9b9a6e6700 1 -- 172.21.2.101:0/315899692 >> 172.21.2.101:0/315899692 conn(0x7f9b940ce450 msgr2=0x7f9b940d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:27.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.924 7f9b9a6e6700 1 -- 172.21.2.101:0/315899692 shutdown_connections 2021-02-10T11:42:27.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.924 7f9b9a6e6700 1 -- 172.21.2.101:0/315899692 wait complete. 2021-02-10T11:42:27.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.939 7f9b9a6e6700 1 Processor -- start 2021-02-10T11:42:27.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.939 7f9b9a6e6700 1 -- start start 2021-02-10T11:42:27.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.940 7f9b9a6e6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b94195650 0x7f9b940b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:27.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.940 7f9b9a6e6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b940b5b80 0x7f9b941a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:27.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.940 7f9b9a6e6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b940acfa0 0x7f9b941a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:27.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.940 7f9b9a6e6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b940b6010 con 0x7f9b940acfa0 2021-02-10T11:42:27.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.940 7f9b9a6e6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b941a3a60 con 0x7f9b94195650 2021-02-10T11:42:27.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.940 7f9b9a6e6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b941a3c40 con 0x7f9b940b5b80 2021-02-10T11:42:27.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.940 7f9b98d03700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b940acfa0 0x7f9b941a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:27.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.940 7f9b98d03700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b940acfa0 0x7f9b941a3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58108/0 (socket says 172.21.2.101:58108) 2021-02-10T11:42:27.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.940 7f9b98d03700 1 -- 172.21.2.101:0/4244495526 learned_addr learned my addr 172.21.2.101:0/4244495526 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:27.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.940 7f9b98d03700 1 -- 172.21.2.101:0/4244495526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b940b5b80 msgr2=0x7f9b941a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:27.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.940 7f9b98d03700 1 --2- 172.21.2.101:0/4244495526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b940b5b80 0x7f9b941a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:27.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.940 7f9b98d03700 1 -- 172.21.2.101:0/4244495526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b94195650 msgr2=0x7f9b940b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:27.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.940 7f9b98d03700 1 --2- 172.21.2.101:0/4244495526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b94195650 0x7f9b940b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:27.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.940 7f9b98d03700 1 -- 172.21.2.101:0/4244495526 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b940bcae0 con 0x7f9b940acfa0 2021-02-10T11:42:27.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.940 7f9b98d03700 1 --2- 172.21.2.101:0/4244495526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b940acfa0 0x7f9b941a3440 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 rx=0x7f9b8c0099a0 tx=0x7f9b8c00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:27.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.940 7f9b90ff9700 1 -- 172.21.2.101:0/4244495526 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9b8c000ce0 con 0x7f9b940acfa0 2021-02-10T11:42:27.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.940 7f9b90ff9700 1 -- 172.21.2.101:0/4244495526 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8c00c2f0 con 0x7f9b940acfa0 2021-02-10T11:42:27.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.940 7f9b9a6e6700 1 -- 172.21.2.101:0/4244495526 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b940bcae0 con 0x7f9b940acfa0 2021-02-10T11:42:27.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.941 7f9b90ff9700 1 -- 172.21.2.101:0/4244495526 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9b8c00c700 con 0x7f9b940acfa0 2021-02-10T11:42:27.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.941 7f9b9a6e6700 1 -- 172.21.2.101:0/4244495526 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b940bcae0 con 0x7f9b940acfa0 2021-02-10T11:42:27.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.941 7f9b90ff9700 1 -- 172.21.2.101:0/4244495526 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9b8c00a000 con 0x7f9b940acfa0 2021-02-10T11:42:27.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.941 7f9b90ff9700 1 --2- 172.21.2.101:0/4244495526 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b740186c0 0x7f9b7401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:27.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.941 7f9b90ff9700 1 -- 172.21.2.101:0/4244495526 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f9b8c0325f0 con 0x7f9b940acfa0 2021-02-10T11:42:27.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.941 7f9b93fff700 1 --2- 172.21.2.101:0/4244495526 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b740186c0 0x7f9b7401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:27.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.942 7f9b93fff700 1 --2- 172.21.2.101:0/4244495526 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b740186c0 0x7f9b7401abc0 crc :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:27.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.972 7f9b9a6e6700 1 -- 172.21.2.101:0/4244495526 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b940b6010 con 0x7f9b940acfa0 2021-02-10T11:42:27.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:27.975 7f9b90ff9700 1 -- 172.21.2.101:0/4244495526 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9b8c00be30 con 0x7f9b940acfa0 2021-02-10T11:42:28.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.098 7f9b9a6e6700 1 -- 172.21.2.101:0/4244495526 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9b940b6010 con 0x7f9b940acfa0 2021-02-10T11:42:28.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.099 7f9b90ff9700 1 -- 172.21.2.101:0/4244495526 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v124) v1 ==== 84+0+64 (secure 0 0 0) 0x7f9b8c00a000 con 0x7f9b940acfa0 2021-02-10T11:42:28.101 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:42:28.101 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:42:28.101 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:28.101 INFO:teuthology.orchestra.run.gibba001.stdout: "poolnum": 1, 2021-02-10T11:42:28.101 INFO:teuthology.orchestra.run.gibba001.stdout: "poolname": "rbd" 2021-02-10T11:42:28.102 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:28.102 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:42:28.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.101 7f9b9a6e6700 1 -- 172.21.2.101:0/4244495526 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b740186c0 msgr2=0x7f9b7401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:28.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.101 7f9b9a6e6700 1 --2- 172.21.2.101:0/4244495526 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b740186c0 0x7f9b7401abc0 crc :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:28.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.101 7f9b9a6e6700 1 -- 172.21.2.101:0/4244495526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b940acfa0 msgr2=0x7f9b941a3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:28.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.101 7f9b9a6e6700 1 --2- 172.21.2.101:0/4244495526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b940acfa0 0x7f9b941a3440 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 rx=0x7f9b8c0099a0 tx=0x7f9b8c00bcf0).stop 2021-02-10T11:42:28.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.101 7f9b9a6e6700 1 -- 172.21.2.101:0/4244495526 shutdown_connections 2021-02-10T11:42:28.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.101 7f9b9a6e6700 1 --2- 172.21.2.101:0/4244495526 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b740186c0 0x7f9b7401abc0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:28.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.101 7f9b9a6e6700 1 --2- 172.21.2.101:0/4244495526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b940b5b80 0x7f9b941a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:28.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.101 7f9b9a6e6700 1 --2- 172.21.2.101:0/4244495526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b940acfa0 0x7f9b941a3440 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:28.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.101 7f9b9a6e6700 1 --2- 172.21.2.101:0/4244495526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b94195650 0x7f9b940b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:28.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.101 7f9b9a6e6700 1 -- 172.21.2.101:0/4244495526 >> 172.21.2.101:0/4244495526 conn(0x7f9b940b3040 msgr2=0x7f9b940c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:28.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.101 7f9b9a6e6700 1 -- 172.21.2.101:0/4244495526 shutdown_connections 2021-02-10T11:42:28.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.101 7f9b9a6e6700 1 -- 172.21.2.101:0/4244495526 wait complete. 2021-02-10T11:42:28.111 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-02-10T11:42:28.112 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-02-10T11:42:28.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.527 7fbf8cd3e700 1 Processor -- start 2021-02-10T11:42:28.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.527 7fbf8cd3e700 1 -- start start 2021-02-10T11:42:28.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.528 7fbf8cd3e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf880d1870 0x7fbf881b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:28.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.528 7fbf8cd3e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbf881b0ae0 0x7fbf88108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:28.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.528 7fbf8cd3e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf88109a60 con 0x7fbf880d1c70 2021-02-10T11:42:28.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.528 7fbf8cd3e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf880e5ef0 con 0x7fbf881b0ae0 2021-02-10T11:42:28.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.528 7fbf8cd3e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf88109750 con 0x7fbf880d1870 2021-02-10T11:42:28.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.528 7fbf87314700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fbf880d1c70 0x7fbf88108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58610/0 (socket says 172.21.2.101:58610) 2021-02-10T11:42:28.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.528 7fbf87314700 1 -- 172.21.2.101:0/1742411947 learned_addr learned my addr 172.21.2.101:0/1742411947 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:28.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.528 7fbf86312700 1 --2- 172.21.2.101:0/1742411947 >> v2:172.21.2.101:3300/0 conn(0x7fbf881b0ae0 0x7fbf88108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:28.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.528 7fbf86b13700 1 --2- 172.21.2.101:0/1742411947 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf880d1870 0x7fbf881b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:28.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.528 7fbf86312700 1 -- 172.21.2.101:0/1742411947 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf880d1870 msgr2=0x7fbf881b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:28.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.528 7fbf86312700 1 --2- 172.21.2.101:0/1742411947 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf880d1870 0x7fbf881b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:28.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.528 7fbf86312700 1 -- 172.21.2.101:0/1742411947 >> v1:172.21.2.101:6789/0 conn(0x7fbf880d1c70 legacy=0x7fbf88108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:28.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.528 7fbf86312700 1 -- 172.21.2.101:0/1742411947 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf88088070 con 0x7fbf881b0ae0 2021-02-10T11:42:28.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.529 7fbf86312700 1 --2- 172.21.2.101:0/1742411947 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf881b0ae0 0x7fbf88108940 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 rx=0x7fbf70006650 tx=0x7fbf70009a80).ready entity=mon.0 client_cookie=6221c91ba7afe80 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:28.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.529 7fbf85b11700 1 -- 172.21.2.101:0/1742411947 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbf700009b0 con 0x7fbf881b0ae0 2021-02-10T11:42:28.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.529 7fbf85b11700 1 -- 172.21.2.101:0/1742411947 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7000c0c0 con 0x7fbf881b0ae0 2021-02-10T11:42:28.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.529 7fbf85b11700 1 -- 172.21.2.101:0/1742411947 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbf7000c5e0 con 0x7fbf881b0ae0 2021-02-10T11:42:28.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.529 7fbf8cd3e700 1 -- 172.21.2.101:0/1742411947 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf881b0ae0 msgr2=0x7fbf88108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:28.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.529 7fbf8cd3e700 1 --2- 172.21.2.101:0/1742411947 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf881b0ae0 0x7fbf88108940 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 rx=0x7fbf70006650 tx=0x7fbf70009a80).stop 2021-02-10T11:42:28.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.530 7fbf8cd3e700 1 -- 172.21.2.101:0/1742411947 shutdown_connections 2021-02-10T11:42:28.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.530 7fbf8cd3e700 1 --2- 172.21.2.101:0/1742411947 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf881b0ae0 0x7fbf88108940 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:28.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.530 7fbf8cd3e700 1 --2- 172.21.2.101:0/1742411947 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf880d1870 0x7fbf881b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:28.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.530 7fbf8cd3e700 1 -- 172.21.2.101:0/1742411947 >> 172.21.2.101:0/1742411947 conn(0x7fbf880ce420 msgr2=0x7fbf880d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:28.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.530 7fbf8cd3e700 1 -- 172.21.2.101:0/1742411947 shutdown_connections 2021-02-10T11:42:28.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.530 7fbf8cd3e700 1 -- 172.21.2.101:0/1742411947 wait complete. 2021-02-10T11:42:28.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.545 7fbf8cd3e700 1 Processor -- start 2021-02-10T11:42:28.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.545 7fbf8cd3e700 1 -- start start 2021-02-10T11:42:28.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.545 7fbf8cd3e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf8808aa00 0x7fbf881a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:28.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.545 7fbf8cd3e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf8809f580 0x7fbf8809f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:28.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.545 7fbf8cd3e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbf880a0830 0x7fbf880a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:28.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.545 7fbf8cd3e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf880a0070 con 0x7fbf8809f580 2021-02-10T11:42:28.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.545 7fbf8cd3e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8808ae30 con 0x7fbf880a0830 2021-02-10T11:42:28.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.545 7fbf8cd3e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8808d2a0 con 0x7fbf8808aa00 2021-02-10T11:42:28.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.545 7fbf86312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf8809f580 0x7fbf8809f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:28.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.545 7fbf86312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf8809f580 0x7fbf8809f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58118/0 (socket says 172.21.2.101:58118) 2021-02-10T11:42:28.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.545 7fbf86312700 1 -- 172.21.2.101:0/2883199076 learned_addr learned my addr 172.21.2.101:0/2883199076 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:28.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.546 7fbf86312700 1 -- 172.21.2.101:0/2883199076 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf8808aa00 msgr2=0x7fbf881a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:28.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.546 7fbf86312700 1 --2- 172.21.2.101:0/2883199076 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf8808aa00 0x7fbf881a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:28.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.546 7fbf86312700 1 -- 172.21.2.101:0/2883199076 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbf880a0830 msgr2=0x7fbf880a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:28.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.546 7fbf86312700 1 --2- 172.21.2.101:0/2883199076 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbf880a0830 0x7fbf880a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:28.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.546 7fbf86312700 1 -- 172.21.2.101:0/2883199076 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf881a49d0 con 0x7fbf8809f580 2021-02-10T11:42:28.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.546 7fbf86312700 1 --2- 172.21.2.101:0/2883199076 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf8809f580 0x7fbf8809f980 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 rx=0x7fbf70009630 tx=0x7fbf7000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:28.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.546 7fbf7f7fe700 1 -- 172.21.2.101:0/2883199076 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbf700093a0 con 0x7fbf8809f580 2021-02-10T11:42:28.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.546 7fbf7f7fe700 1 -- 172.21.2.101:0/2883199076 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7000bd70 con 0x7fbf8809f580 2021-02-10T11:42:28.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.546 7fbf7f7fe700 1 -- 172.21.2.101:0/2883199076 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbf70005560 con 0x7fbf8809f580 2021-02-10T11:42:28.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.546 7fbf8cd3e700 1 -- 172.21.2.101:0/2883199076 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf881a49d0 con 0x7fbf8809f580 2021-02-10T11:42:28.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.546 7fbf8cd3e700 1 -- 172.21.2.101:0/2883199076 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf880a1540 con 0x7fbf8809f580 2021-02-10T11:42:28.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.547 7fbf7f7fe700 1 -- 172.21.2.101:0/2883199076 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbf7000a000 con 0x7fbf8809f580 2021-02-10T11:42:28.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.547 7fbf7f7fe700 1 --2- 172.21.2.101:0/2883199076 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf6c0186c0 0x7fbf6c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:28.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.547 7fbf7f7fe700 1 -- 172.21.2.101:0/2883199076 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fbf70032050 con 0x7fbf8809f580 2021-02-10T11:42:28.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.547 7fbf86b13700 1 --2- 172.21.2.101:0/2883199076 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf6c0186c0 0x7fbf6c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:28.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.547 7fbf86b13700 1 --2- 172.21.2.101:0/2883199076 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf6c0186c0 0x7fbf6c01abc0 crc :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:28.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.578 7fbf8cd3e700 1 -- 172.21.2.101:0/2883199076 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf8808ae00 con 0x7fbf8809f580 2021-02-10T11:42:28.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.580 7fbf7f7fe700 1 -- 172.21.2.101:0/2883199076 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbf70025f60 con 0x7fbf8809f580 2021-02-10T11:42:28.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:28.708 7fbf8cd3e700 1 -- 172.21.2.101:0/2883199076 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"} v 0) v1 -- 0x7fbf8808ae00 con 0x7fbf8809f580 2021-02-10T11:42:29.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.111 7fbf7f7fe700 1 -- 172.21.2.101:0/2883199076 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"}]=0 pool 'cephfs_metadata' created v125) v1 ==== 133+0+0 (secure 0 0 0) 0x7fbf70025f60 con 0x7fbf8809f580 2021-02-10T11:42:29.126 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' created 2021-02-10T11:42:29.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.126 7fbf8cd3e700 1 -- 172.21.2.101:0/2883199076 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf6c0186c0 msgr2=0x7fbf6c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:29.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.126 7fbf8cd3e700 1 --2- 172.21.2.101:0/2883199076 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf6c0186c0 0x7fbf6c01abc0 crc :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:29.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.126 7fbf8cd3e700 1 -- 172.21.2.101:0/2883199076 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf8809f580 msgr2=0x7fbf8809f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:29.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.126 7fbf8cd3e700 1 --2- 172.21.2.101:0/2883199076 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf8809f580 0x7fbf8809f980 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 rx=0x7fbf70009630 tx=0x7fbf7000b980).stop 2021-02-10T11:42:29.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.126 7fbf8cd3e700 1 -- 172.21.2.101:0/2883199076 shutdown_connections 2021-02-10T11:42:29.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.126 7fbf8cd3e700 1 --2- 172.21.2.101:0/2883199076 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbf880a0830 0x7fbf880a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:29.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.126 7fbf8cd3e700 1 --2- 172.21.2.101:0/2883199076 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbf8809f580 0x7fbf8809f980 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:29.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.126 7fbf8cd3e700 1 --2- 172.21.2.101:0/2883199076 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbf6c0186c0 0x7fbf6c01abc0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:29.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.126 7fbf8cd3e700 1 --2- 172.21.2.101:0/2883199076 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbf8808aa00 0x7fbf881a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:29.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.126 7fbf8cd3e700 1 -- 172.21.2.101:0/2883199076 >> 172.21.2.101:0/2883199076 conn(0x7fbf880bfb90 msgr2=0x7fbf880a6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:29.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.126 7fbf8cd3e700 1 -- 172.21.2.101:0/2883199076 shutdown_connections 2021-02-10T11:42:29.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.126 7fbf8cd3e700 1 -- 172.21.2.101:0/2883199076 wait complete. 2021-02-10T11:42:29.136 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_data 8 2021-02-10T11:42:29.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.550 7f5b8541a700 1 Processor -- start 2021-02-10T11:42:29.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.550 7f5b8541a700 1 -- start start 2021-02-10T11:42:29.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.551 7f5b8541a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b800d1c70 0x7f5b800cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:29.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.551 7f5b8541a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f5b800cf580 0x7f5b800cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:29.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.551 7f5b8541a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b800fe1c0 con 0x7f5b800d1870 2021-02-10T11:42:29.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.551 7f5b8541a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b801b0cb0 con 0x7f5b800cf580 2021-02-10T11:42:29.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.551 7f5b8541a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b80109a60 con 0x7f5b800d1c70 2021-02-10T11:42:29.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.551 7f5b7f07d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f5b800d1870 0x7f5b801b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58620/0 (socket says 172.21.2.101:58620) 2021-02-10T11:42:29.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.551 7f5b7f07d700 1 -- 172.21.2.101:0/1313385415 learned_addr learned my addr 172.21.2.101:0/1313385415 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:29.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.552 7f5b7e07b700 1 -- 172.21.2.101:0/1313385415 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1133722015 0 0) 0x7f5b68001180 con 0x7f5b800d1870 2021-02-10T11:42:29.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.552 7f5b7e87c700 1 --2- 172.21.2.101:0/1313385415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b800d1c70 0x7f5b800cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:29.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.552 7f5b7f87e700 1 --2- 172.21.2.101:0/1313385415 >> v2:172.21.2.108:3300/0 conn(0x7f5b800cf580 0x7f5b800cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:29.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.552 7f5b7e07b700 1 -- 172.21.2.101:0/1313385415 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b6c002fe0 con 0x7f5b800d1870 2021-02-10T11:42:29.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.552 7f5b7e07b700 1 -- 172.21.2.101:0/1313385415 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1951936838 0 0) 0x7f5b680010e0 con 0x7f5b800d1870 2021-02-10T11:42:29.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.552 7f5b7e07b700 1 -- 172.21.2.101:0/1313385415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b800d1c70 msgr2=0x7f5b800cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:29.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.552 7f5b7e07b700 1 --2- 172.21.2.101:0/1313385415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b800d1c70 0x7f5b800cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:29.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.552 7f5b7e07b700 1 -- 172.21.2.101:0/1313385415 >> v2:172.21.2.108:3300/0 conn(0x7f5b800cf580 msgr2=0x7f5b800cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:29.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.552 7f5b7e07b700 1 --2- 172.21.2.101:0/1313385415 >> v2:172.21.2.108:3300/0 conn(0x7f5b800cf580 0x7f5b800cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:29.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.552 7f5b7e07b700 1 -- 172.21.2.101:0/1313385415 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b80088070 con 0x7f5b800d1870 2021-02-10T11:42:29.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.552 7f5b7e07b700 1 -- 172.21.2.101:0/1313385415 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f5b68001600 con 0x7f5b800d1870 2021-02-10T11:42:29.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.552 7f5b7e07b700 1 -- 172.21.2.101:0/1313385415 >> v1:172.21.2.101:6789/0 conn(0x7f5b800d1870 legacy=0x7f5b801b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:29.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.552 7f5b7e07b700 1 --2- 172.21.2.101:0/1313385415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b6c003ac0 0x7f5b6c005f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:29.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.552 7f5b7e07b700 1 --2- 172.21.2.101:0/1313385415 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5b6c04a760 0x7f5b6c04cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:29.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.552 7f5b7e07b700 1 --2- 172.21.2.101:0/1313385415 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b6c04d500 0x7f5b6c04fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:29.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.552 7f5b7e07b700 1 -- 172.21.2.101:0/1313385415 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c0502d0 con 0x7f5b6c04d500 2021-02-10T11:42:29.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.552 7f5b7e07b700 1 -- 172.21.2.101:0/1313385415 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c0504f0 con 0x7f5b6c04a760 2021-02-10T11:42:29.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.552 7f5b7e07b700 1 -- 172.21.2.101:0/1313385415 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c050710 con 0x7f5b6c003ac0 2021-02-10T11:42:29.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.552 7f5b7f87e700 1 --2- 172.21.2.101:0/1313385415 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b6c04d500 0x7f5b6c04fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:29.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.552 7f5b7f07d700 1 --2- 172.21.2.101:0/1313385415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b6c003ac0 0x7f5b6c005f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:29.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.553 7f5b7e87c700 1 --2- 172.21.2.101:0/1313385415 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5b6c04a760 0x7f5b6c04cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:29.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.553 7f5b7f87e700 1 -- 172.21.2.101:0/1313385415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b6c003ac0 msgr2=0x7f5b6c005f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:29.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.553 7f5b7f87e700 1 --2- 172.21.2.101:0/1313385415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b6c003ac0 0x7f5b6c005f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:29.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.553 7f5b7f87e700 1 -- 172.21.2.101:0/1313385415 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5b6c04a760 msgr2=0x7f5b6c04cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:29.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.553 7f5b7f87e700 1 --2- 172.21.2.101:0/1313385415 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5b6c04a760 0x7f5b6c04cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:29.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.553 7f5b7f87e700 1 -- 172.21.2.101:0/1313385415 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b6c0509b0 con 0x7f5b6c04d500 2021-02-10T11:42:29.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.553 7f5b7f07d700 1 --2- 172.21.2.101:0/1313385415 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b6c003ac0 0x7f5b6c005f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:42:29.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.553 7f5b7f87e700 1 --2- 172.21.2.101:0/1313385415 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b6c04d500 0x7f5b6c04fa20 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 rx=0x7f5b78006a40 tx=0x7f5b7800c0c0).ready entity=mon.0 client_cookie=55ff8dc2a16ae088 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:29.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.553 7f5b7e07b700 1 -- 172.21.2.101:0/1313385415 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5b7800c4b0 con 0x7f5b6c04d500 2021-02-10T11:42:29.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.553 7f5b7e07b700 1 -- 172.21.2.101:0/1313385415 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7800c6f0 con 0x7f5b6c04d500 2021-02-10T11:42:29.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.553 7f5b7e07b700 1 -- 172.21.2.101:0/1313385415 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5b7800cbd0 con 0x7f5b6c04d500 2021-02-10T11:42:29.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.553 7f5b8541a700 1 -- 172.21.2.101:0/1313385415 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b6c04d500 msgr2=0x7f5b6c04fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:29.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.553 7f5b8541a700 1 --2- 172.21.2.101:0/1313385415 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b6c04d500 0x7f5b6c04fa20 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 rx=0x7f5b78006a40 tx=0x7f5b7800c0c0).stop 2021-02-10T11:42:29.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.553 7f5b7f87e700 1 -- 172.21.2.101:0/1313385415 reap_dead start 2021-02-10T11:42:29.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.553 7f5b8541a700 1 -- 172.21.2.101:0/1313385415 shutdown_connections 2021-02-10T11:42:29.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.553 7f5b8541a700 1 -- 172.21.2.101:0/1313385415 >> 172.21.2.101:0/1313385415 conn(0x7f5b800ce420 msgr2=0x7f5b800d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:29.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.554 7f5b8541a700 1 -- 172.21.2.101:0/1313385415 shutdown_connections 2021-02-10T11:42:29.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.554 7f5b8541a700 1 -- 172.21.2.101:0/1313385415 wait complete. 2021-02-10T11:42:29.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.568 7f5b8541a700 1 Processor -- start 2021-02-10T11:42:29.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.568 7f5b8541a700 1 -- start start 2021-02-10T11:42:29.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.569 7f5b8541a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b80085bc0 0x7f5b800863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:29.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.569 7f5b8541a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5b800954c0 0x7f5b80096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:29.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.569 7f5b8541a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b800958c0 0x7f5b80096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:29.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.569 7f5b8541a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b80095d80 con 0x7f5b80085bc0 2021-02-10T11:42:29.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.569 7f5b8541a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b800bfa60 con 0x7f5b800954c0 2021-02-10T11:42:29.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.569 7f5b8541a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b80085fc0 con 0x7f5b800958c0 2021-02-10T11:42:29.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.569 7f5b7f07d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b80085bc0 0x7f5b800863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:29.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.569 7f5b7f07d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b80085bc0 0x7f5b800863e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58138/0 (socket says 172.21.2.101:58138) 2021-02-10T11:42:29.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.569 7f5b7f07d700 1 -- 172.21.2.101:0/2338304573 learned_addr learned my addr 172.21.2.101:0/2338304573 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:29.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.569 7f5b7f07d700 1 -- 172.21.2.101:0/2338304573 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b800958c0 msgr2=0x7f5b80096920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:42:29.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.570 7f5b7f07d700 1 --2- 172.21.2.101:0/2338304573 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b800958c0 0x7f5b80096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:29.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.570 7f5b7f07d700 1 -- 172.21.2.101:0/2338304573 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5b800954c0 msgr2=0x7f5b80096300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:29.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.570 7f5b7f07d700 1 --2- 172.21.2.101:0/2338304573 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5b800954c0 0x7f5b80096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:29.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.570 7f5b7f07d700 1 -- 172.21.2.101:0/2338304573 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b80086c10 con 0x7f5b80085bc0 2021-02-10T11:42:29.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.570 7f5b7f07d700 1 --2- 172.21.2.101:0/2338304573 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b80085bc0 0x7f5b800863e0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 rx=0x7f5b6800a590 tx=0x7f5b6800a930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:29.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.570 7f5b77fff700 1 -- 172.21.2.101:0/2338304573 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5b68000ef0 con 0x7f5b80085bc0 2021-02-10T11:42:29.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.570 7f5b77fff700 1 -- 172.21.2.101:0/2338304573 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6800ad20 con 0x7f5b80085bc0 2021-02-10T11:42:29.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.570 7f5b77fff700 1 -- 172.21.2.101:0/2338304573 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5b68004620 con 0x7f5b80085bc0 2021-02-10T11:42:29.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.570 7f5b8541a700 1 -- 172.21.2.101:0/2338304573 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b80086c10 con 0x7f5b80085bc0 2021-02-10T11:42:29.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.570 7f5b8541a700 1 -- 172.21.2.101:0/2338304573 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b80097230 con 0x7f5b80085bc0 2021-02-10T11:42:29.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.571 7f5b77fff700 1 -- 172.21.2.101:0/2338304573 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5b680045b0 con 0x7f5b80085bc0 2021-02-10T11:42:29.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.571 7f5b77fff700 1 --2- 172.21.2.101:0/2338304573 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b640186c0 0x7f5b6401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:29.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.571 7f5b77fff700 1 -- 172.21.2.101:0/2338304573 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5107+0+0 (secure 0 0 0) 0x7f5b68032250 con 0x7f5b80085bc0 2021-02-10T11:42:29.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.571 7f5b7e87c700 1 --2- 172.21.2.101:0/2338304573 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b640186c0 0x7f5b6401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:29.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.571 7f5b7e87c700 1 --2- 172.21.2.101:0/2338304573 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b640186c0 0x7f5b6401abc0 crc :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:29.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.601 7f5b8541a700 1 -- 172.21.2.101:0/2338304573 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b800fe1c0 con 0x7f5b80085bc0 2021-02-10T11:42:29.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.604 7f5b77fff700 1 -- 172.21.2.101:0/2338304573 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5b680150f0 con 0x7f5b80085bc0 2021-02-10T11:42:29.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:29.735 7f5b8541a700 1 -- 172.21.2.101:0/2338304573 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"} v 0) v1 -- 0x7f5b800fe1c0 con 0x7f5b80085bc0 2021-02-10T11:42:30.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.118 7f5b77fff700 1 -- 172.21.2.101:0/2338304573 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"}]=0 pool 'cephfs_data' created v127) v1 ==== 125+0+0 (secure 0 0 0) 0x7f5b680150f0 con 0x7f5b80085bc0 2021-02-10T11:42:30.153 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' created 2021-02-10T11:42:30.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.152 7f5b8541a700 1 -- 172.21.2.101:0/2338304573 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b640186c0 msgr2=0x7f5b6401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:30.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.152 7f5b8541a700 1 --2- 172.21.2.101:0/2338304573 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b640186c0 0x7f5b6401abc0 crc :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:30.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.153 7f5b8541a700 1 -- 172.21.2.101:0/2338304573 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b80085bc0 msgr2=0x7f5b800863e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:30.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.153 7f5b8541a700 1 --2- 172.21.2.101:0/2338304573 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b80085bc0 0x7f5b800863e0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 rx=0x7f5b6800a590 tx=0x7f5b6800a930).stop 2021-02-10T11:42:30.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.153 7f5b8541a700 1 -- 172.21.2.101:0/2338304573 shutdown_connections 2021-02-10T11:42:30.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.153 7f5b8541a700 1 --2- 172.21.2.101:0/2338304573 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b640186c0 0x7f5b6401abc0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:30.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.153 7f5b8541a700 1 --2- 172.21.2.101:0/2338304573 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b800958c0 0x7f5b80096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:30.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.153 7f5b8541a700 1 --2- 172.21.2.101:0/2338304573 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5b800954c0 0x7f5b80096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:30.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.153 7f5b8541a700 1 --2- 172.21.2.101:0/2338304573 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b80085bc0 0x7f5b800863e0 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:30.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.153 7f5b8541a700 1 -- 172.21.2.101:0/2338304573 >> 172.21.2.101:0/2338304573 conn(0x7f5b800bef00 msgr2=0x7f5b8009d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:30.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.153 7f5b8541a700 1 -- 172.21.2.101:0/2338304573 shutdown_connections 2021-02-10T11:42:30.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.153 7f5b8541a700 1 -- 172.21.2.101:0/2338304573 wait complete. 2021-02-10T11:42:30.163 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-02-10T11:42:30.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.578 7f7dc5109700 1 Processor -- start 2021-02-10T11:42:30.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.578 7f7dc5109700 1 -- start start 2021-02-10T11:42:30.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.579 7f7dc5109700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7dc00d1c80 0x7f7dc00cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:30.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.579 7f7dc5109700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7dc00cf590 0x7f7dc00cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:30.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.579 7f7dc5109700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc00fe1d0 con 0x7f7dc00d1880 2021-02-10T11:42:30.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.579 7f7dc5109700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc01b0cc0 con 0x7f7dc00d1c80 2021-02-10T11:42:30.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.579 7f7dc5109700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc0109a70 con 0x7f7dc00cf590 2021-02-10T11:42:30.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.579 7f7dbed9d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f7dc00d1880 0x7f7dc01b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58642/0 (socket says 172.21.2.101:58642) 2021-02-10T11:42:30.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.579 7f7dbed9d700 1 -- 172.21.2.101:0/4022744508 learned_addr learned my addr 172.21.2.101:0/4022744508 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:30.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.579 7f7dbdd9b700 1 -- 172.21.2.101:0/4022744508 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801387040 0 0) 0x7f7da8001180 con 0x7f7dc00d1880 2021-02-10T11:42:30.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.580 7f7dbdd9b700 1 -- 172.21.2.101:0/4022744508 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dac002fe0 con 0x7f7dc00d1880 2021-02-10T11:42:30.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.580 7f7dbe59c700 1 --2- 172.21.2.101:0/4022744508 >> v2:172.21.2.108:3300/0 conn(0x7f7dc00d1c80 0x7f7dc00cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:30.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.580 7f7dbf59e700 1 --2- 172.21.2.101:0/4022744508 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7dc00cf590 0x7f7dc00cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:30.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.580 7f7dbdd9b700 1 -- 172.21.2.101:0/4022744508 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3163466604 0 0) 0x7f7da8001180 con 0x7f7dc00d1880 2021-02-10T11:42:30.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.580 7f7dbe59c700 1 -- 172.21.2.101:0/4022744508 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7dc00cf590 msgr2=0x7f7dc00cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:30.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.580 7f7dbe59c700 1 --2- 172.21.2.101:0/4022744508 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7dc00cf590 0x7f7dc00cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:30.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.580 7f7dbe59c700 1 -- 172.21.2.101:0/4022744508 >> v1:172.21.2.101:6789/0 conn(0x7f7dc00d1880 legacy=0x7f7dc01b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:30.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.580 7f7dbe59c700 1 -- 172.21.2.101:0/4022744508 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc0088070 con 0x7f7dc00d1c80 2021-02-10T11:42:30.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.580 7f7dbe59c700 1 --2- 172.21.2.101:0/4022744508 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7dc00d1c80 0x7f7dc00cef70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f7db80069d0 tx=0x7f7db8009e40).ready entity=mon.1 client_cookie=3ce8679b88334739 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:30.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.580 7f7dbdd9b700 1 -- 172.21.2.101:0/4022744508 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7db800b300 con 0x7f7dc00d1c80 2021-02-10T11:42:30.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.580 7f7dbdd9b700 1 -- 172.21.2.101:0/4022744508 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db800b640 con 0x7f7dc00d1c80 2021-02-10T11:42:30.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.580 7f7dbdd9b700 1 -- 172.21.2.101:0/4022744508 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7db800baa0 con 0x7f7dc00d1c80 2021-02-10T11:42:30.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.581 7f7dc5109700 1 -- 172.21.2.101:0/4022744508 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7dc00d1c80 msgr2=0x7f7dc00cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:30.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.581 7f7dc5109700 1 --2- 172.21.2.101:0/4022744508 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7dc00d1c80 0x7f7dc00cef70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f7db80069d0 tx=0x7f7db8009e40).stop 2021-02-10T11:42:30.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.581 7f7dc5109700 1 -- 172.21.2.101:0/4022744508 shutdown_connections 2021-02-10T11:42:30.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.581 7f7dc5109700 1 --2- 172.21.2.101:0/4022744508 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7dc00cf590 0x7f7dc00cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:30.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.581 7f7dc5109700 1 --2- 172.21.2.101:0/4022744508 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7dc00d1c80 0x7f7dc00cef70 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:30.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.581 7f7dc5109700 1 -- 172.21.2.101:0/4022744508 >> 172.21.2.101:0/4022744508 conn(0x7f7dc00ce430 msgr2=0x7f7dc00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:30.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.581 7f7dc5109700 1 -- 172.21.2.101:0/4022744508 shutdown_connections 2021-02-10T11:42:30.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.581 7f7dc5109700 1 -- 172.21.2.101:0/4022744508 wait complete. 2021-02-10T11:42:30.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.596 7f7dc5109700 1 Processor -- start 2021-02-10T11:42:30.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.596 7f7dc5109700 1 -- start start 2021-02-10T11:42:30.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.596 7f7dc5109700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7dc0188400 0x7f7dc0188800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:30.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.596 7f7dc5109700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7dc0188e20 0x7f7dc0189220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:30.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.596 7f7dc5109700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7dc009cbc0 0x7f7dc009cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:30.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.596 7f7dc5109700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc0187e80 con 0x7f7dc0188e20 2021-02-10T11:42:30.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.596 7f7dc5109700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc00b27c0 con 0x7f7dc0188400 2021-02-10T11:42:30.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.596 7f7dc5109700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc00b29a0 con 0x7f7dc009cbc0 2021-02-10T11:42:30.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.597 7f7dbe59c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7dc0188e20 0x7f7dc0189220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:30.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.597 7f7dbe59c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7dc0188e20 0x7f7dc0189220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58158/0 (socket says 172.21.2.101:58158) 2021-02-10T11:42:30.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.597 7f7dbe59c700 1 -- 172.21.2.101:0/3311438016 learned_addr learned my addr 172.21.2.101:0/3311438016 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:30.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.597 7f7dbe59c700 1 -- 172.21.2.101:0/3311438016 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7dc009cbc0 msgr2=0x7f7dc009cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:30.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.597 7f7dbe59c700 1 --2- 172.21.2.101:0/3311438016 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7dc009cbc0 0x7f7dc009cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:30.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.597 7f7dbe59c700 1 -- 172.21.2.101:0/3311438016 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7dc0188400 msgr2=0x7f7dc0188800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:30.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.597 7f7dbe59c700 1 --2- 172.21.2.101:0/3311438016 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7dc0188400 0x7f7dc0188800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:30.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.597 7f7dbe59c700 1 -- 172.21.2.101:0/3311438016 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc009d5e0 con 0x7f7dc0188e20 2021-02-10T11:42:30.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.597 7f7dbe59c700 1 --2- 172.21.2.101:0/3311438016 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7dc0188e20 0x7f7dc0189220 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 rx=0x7f7db8009310 tx=0x7f7db800bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:30.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.597 7f7db77fe700 1 -- 172.21.2.101:0/3311438016 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7db800c2f0 con 0x7f7dc0188e20 2021-02-10T11:42:30.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.597 7f7db77fe700 1 -- 172.21.2.101:0/3311438016 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db80068b0 con 0x7f7dc0188e20 2021-02-10T11:42:30.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.598 7f7db77fe700 1 -- 172.21.2.101:0/3311438016 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7db800c830 con 0x7f7dc0188e20 2021-02-10T11:42:30.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.598 7f7dc5109700 1 -- 172.21.2.101:0/3311438016 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc009d5e0 con 0x7f7dc0188e20 2021-02-10T11:42:30.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.598 7f7dc5109700 1 -- 172.21.2.101:0/3311438016 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc009db70 con 0x7f7dc0188e20 2021-02-10T11:42:30.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.598 7f7db77fe700 1 -- 172.21.2.101:0/3311438016 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7db80066c0 con 0x7f7dc0188e20 2021-02-10T11:42:30.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.598 7f7db77fe700 1 --2- 172.21.2.101:0/3311438016 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7da40186c0 0x7f7da401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:30.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.598 7f7db77fe700 1 -- 172.21.2.101:0/3311438016 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5457+0+0 (secure 0 0 0) 0x7f7db80328b0 con 0x7f7dc0188e20 2021-02-10T11:42:30.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.598 7f7dbed9d700 1 --2- 172.21.2.101:0/3311438016 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7da40186c0 0x7f7da401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:30.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.599 7f7dbed9d700 1 --2- 172.21.2.101:0/3311438016 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7da40186c0 0x7f7da401abc0 crc :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:30.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.629 7f7dc5109700 1 -- 172.21.2.101:0/3311438016 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc0082a70 con 0x7f7dc0188e20 2021-02-10T11:42:30.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.631 7f7db77fe700 1 -- 172.21.2.101:0/3311438016 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7db801f660 con 0x7f7dc0188e20 2021-02-10T11:42:30.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.760 7f7dc5109700 1 -- 172.21.2.101:0/3311438016 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"} v 0) v1 -- 0x7f7dc0082a70 con 0x7f7dc0188e20 2021-02-10T11:42:30.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.765 7f7db77fe700 1 -- 172.21.2.101:0/3311438016 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"}]=0 new fs with metadata pool 26 and data pool 27 v102) v1 ==== 189+0+0 (secure 0 0 0) 0x7f7db801f870 con 0x7f7dc0188e20 2021-02-10T11:42:30.769 INFO:teuthology.orchestra.run.gibba001.stderr:new fs with metadata pool 26 and data pool 27 2021-02-10T11:42:30.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.770 7f7dc5109700 1 -- 172.21.2.101:0/3311438016 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7da40186c0 msgr2=0x7f7da401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:30.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.770 7f7dc5109700 1 --2- 172.21.2.101:0/3311438016 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7da40186c0 0x7f7da401abc0 crc :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:30.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.770 7f7dc5109700 1 -- 172.21.2.101:0/3311438016 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7dc0188e20 msgr2=0x7f7dc0189220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:30.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.770 7f7dc5109700 1 --2- 172.21.2.101:0/3311438016 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7dc0188e20 0x7f7dc0189220 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 rx=0x7f7db8009310 tx=0x7f7db800bcd0).stop 2021-02-10T11:42:30.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.770 7f7dc5109700 1 -- 172.21.2.101:0/3311438016 shutdown_connections 2021-02-10T11:42:30.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.770 7f7dc5109700 1 --2- 172.21.2.101:0/3311438016 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7da40186c0 0x7f7da401abc0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:30.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.770 7f7dc5109700 1 --2- 172.21.2.101:0/3311438016 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7dc009cbc0 0x7f7dc009cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:30.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.770 7f7dc5109700 1 --2- 172.21.2.101:0/3311438016 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7dc0188e20 0x7f7dc0189220 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:30.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.770 7f7dc5109700 1 --2- 172.21.2.101:0/3311438016 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7dc0188400 0x7f7dc0188800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:30.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.770 7f7dc5109700 1 -- 172.21.2.101:0/3311438016 >> 172.21.2.101:0/3311438016 conn(0x7f7dc00c2cd0 msgr2=0x7f7dc00b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:30.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.770 7f7dc5109700 1 -- 172.21.2.101:0/3311438016 shutdown_connections 2021-02-10T11:42:30.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:30.770 7f7dc5109700 1 -- 172.21.2.101:0/3311438016 wait complete. 2021-02-10T11:42:30.780 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:42:31.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.191 7f1660982700 1 Processor -- start 2021-02-10T11:42:31.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.191 7f1660982700 1 -- start start 2021-02-10T11:42:31.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.192 7f1660982700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f165c1b0870 0x7f165c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:31.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.192 7f1660982700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f165c0efe00 0x7f165c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:31.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.192 7f1660982700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f165c1b0c70 con 0x7f165c1b0470 2021-02-10T11:42:31.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.192 7f1660982700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f165c0f0b50 con 0x7f165c1b0870 2021-02-10T11:42:31.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.192 7f1660982700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f165c0f0820 con 0x7f165c0efe00 2021-02-10T11:42:31.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.192 7f1659e1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f165c1b0870 0x7f165c0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:31.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.192 7f165a61c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f165c1b0470 0x7f165c0ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58656/0 (socket says 172.21.2.101:58656) 2021-02-10T11:42:31.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.192 7f165a61c700 1 -- 172.21.2.101:0/1205387660 learned_addr learned my addr 172.21.2.101:0/1205387660 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:31.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.192 7f165ae1d700 1 --2- 172.21.2.101:0/1205387660 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f165c0efe00 0x7f165c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:31.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.192 7f165961a700 1 -- 172.21.2.101:0/1205387660 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1974913558 0 0) 0x7f164c000ce0 con 0x7f165c1b0470 2021-02-10T11:42:31.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.192 7f165961a700 1 -- 172.21.2.101:0/1205387660 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1648002fe0 con 0x7f165c1b0470 2021-02-10T11:42:31.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.192 7f1659e1b700 1 -- 172.21.2.101:0/1205387660 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f165c0efe00 msgr2=0x7f165c0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:31.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.192 7f1659e1b700 1 --2- 172.21.2.101:0/1205387660 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f165c0efe00 0x7f165c0f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:31.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.192 7f1659e1b700 1 -- 172.21.2.101:0/1205387660 >> v1:172.21.2.101:6789/0 conn(0x7f165c1b0470 legacy=0x7f165c0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:31.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.192 7f1659e1b700 1 -- 172.21.2.101:0/1205387660 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f165c0e52e0 con 0x7f165c1b0870 2021-02-10T11:42:31.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.192 7f1659e1b700 1 --2- 172.21.2.101:0/1205387660 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f165c1b0870 0x7f165c0f20e0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 rx=0x7f1644006650 tx=0x7f1644009a80).ready entity=mon.0 client_cookie=bdd469db9194fd81 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:31.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.193 7f165961a700 1 -- 172.21.2.101:0/1205387660 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f164400c0c0 con 0x7f165c1b0870 2021-02-10T11:42:31.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.193 7f165961a700 1 -- 172.21.2.101:0/1205387660 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f164400c340 con 0x7f165c1b0870 2021-02-10T11:42:31.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.193 7f165961a700 1 -- 172.21.2.101:0/1205387660 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f164400c860 con 0x7f165c1b0870 2021-02-10T11:42:31.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.193 7f1660982700 1 -- 172.21.2.101:0/1205387660 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f165c1b0870 msgr2=0x7f165c0f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:31.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.193 7f1660982700 1 --2- 172.21.2.101:0/1205387660 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f165c1b0870 0x7f165c0f20e0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 rx=0x7f1644006650 tx=0x7f1644009a80).stop 2021-02-10T11:42:31.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.194 7f1660982700 1 -- 172.21.2.101:0/1205387660 shutdown_connections 2021-02-10T11:42:31.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.194 7f1660982700 1 --2- 172.21.2.101:0/1205387660 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f165c0efe00 0x7f165c0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:31.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.194 7f1660982700 1 --2- 172.21.2.101:0/1205387660 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f165c1b0870 0x7f165c0f20e0 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:31.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.194 7f1660982700 1 -- 172.21.2.101:0/1205387660 >> 172.21.2.101:0/1205387660 conn(0x7f165c0ce3e0 msgr2=0x7f165c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:31.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.194 7f1660982700 1 -- 172.21.2.101:0/1205387660 shutdown_connections 2021-02-10T11:42:31.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.194 7f1660982700 1 -- 172.21.2.101:0/1205387660 wait complete. 2021-02-10T11:42:31.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.208 7f1660982700 1 Processor -- start 2021-02-10T11:42:31.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.208 7f1660982700 1 -- start start 2021-02-10T11:42:31.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.208 7f1660982700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f165c0aed10 0x7f165c095420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:31.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.208 7f1660982700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f165c095a40 0x7f165c095e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:31.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.208 7f1660982700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f165c08c460 0x7f165c08c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:31.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.208 7f1660982700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165c096530 con 0x7f165c0aed10 2021-02-10T11:42:31.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.208 7f1660982700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165c1b6440 con 0x7f165c08c460 2021-02-10T11:42:31.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.208 7f1660982700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f165c08e270 con 0x7f165c095a40 2021-02-10T11:42:31.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.209 7f165a61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f165c0aed10 0x7f165c095420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:31.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.209 7f165a61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f165c0aed10 0x7f165c095420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58168/0 (socket says 172.21.2.101:58168) 2021-02-10T11:42:31.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.209 7f165a61c700 1 -- 172.21.2.101:0/1964904408 learned_addr learned my addr 172.21.2.101:0/1964904408 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:31.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.209 7f165a61c700 1 -- 172.21.2.101:0/1964904408 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f165c095a40 msgr2=0x7f165c095e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:31.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.209 7f165a61c700 1 --2- 172.21.2.101:0/1964904408 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f165c095a40 0x7f165c095e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:31.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.209 7f165a61c700 1 -- 172.21.2.101:0/1964904408 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f165c08c460 msgr2=0x7f165c08c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:31.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.209 7f165a61c700 1 --2- 172.21.2.101:0/1964904408 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f165c08c460 0x7f165c08c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:31.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.209 7f165a61c700 1 -- 172.21.2.101:0/1964904408 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f165c08ce80 con 0x7f165c0aed10 2021-02-10T11:42:31.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.209 7f165a61c700 1 --2- 172.21.2.101:0/1964904408 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f165c0aed10 0x7f165c095420 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 rx=0x7f164c00b560 tx=0x7f164c00b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:31.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.209 7f1652ffd700 1 -- 172.21.2.101:0/1964904408 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f164c00bcf0 con 0x7f165c0aed10 2021-02-10T11:42:31.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.209 7f1652ffd700 1 -- 172.21.2.101:0/1964904408 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f164c0060d0 con 0x7f165c0aed10 2021-02-10T11:42:31.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.209 7f1660982700 1 -- 172.21.2.101:0/1964904408 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f165c08ce80 con 0x7f165c0aed10 2021-02-10T11:42:31.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.209 7f1652ffd700 1 -- 172.21.2.101:0/1964904408 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f164c0043a0 con 0x7f165c0aed10 2021-02-10T11:42:31.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.210 7f1660982700 1 -- 172.21.2.101:0/1964904408 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f165c08d410 con 0x7f165c0aed10 2021-02-10T11:42:31.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.210 7f1652ffd700 1 -- 172.21.2.101:0/1964904408 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f164c007000 con 0x7f165c0aed10 2021-02-10T11:42:31.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.210 7f1652ffd700 1 --2- 172.21.2.101:0/1964904408 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16400186c0 0x7f164001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:31.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.210 7f1652ffd700 1 -- 172.21.2.101:0/1964904408 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f164c0323f0 con 0x7f165c0aed10 2021-02-10T11:42:31.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.210 7f1659e1b700 1 --2- 172.21.2.101:0/1964904408 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16400186c0 0x7f164001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:31.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.211 7f1659e1b700 1 --2- 172.21.2.101:0/1964904408 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16400186c0 0x7f164001abc0 crc :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:31.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.241 7f1660982700 1 -- 172.21.2.101:0/1964904408 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f165c105ab0 con 0x7f165c0aed10 2021-02-10T11:42:31.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.244 7f1652ffd700 1 -- 172.21.2.101:0/1964904408 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f164c0062b0 con 0x7f165c0aed10 2021-02-10T11:42:31.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.370 7f1660982700 1 -- 172.21.2.101:0/1964904408 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f165c0f0b50 con 0x7f165c0aed10 2021-02-10T11:42:31.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.370 7f1652ffd700 1 -- 172.21.2.101:0/1964904408 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+14020 (secure 0 0 0) 0x7f164c005fc0 con 0x7f165c0aed10 2021-02-10T11:42:31.373 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:42:31.373 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":128,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:42:30.762270","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:42:28.726448","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2021-02-10 11:42:29.754869","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:42:31.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.372 7f1660982700 1 -- 172.21.2.101:0/1964904408 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16400186c0 msgr2=0x7f164001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:31.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.372 7f1660982700 1 --2- 172.21.2.101:0/1964904408 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16400186c0 0x7f164001abc0 crc :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:31.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.372 7f1660982700 1 -- 172.21.2.101:0/1964904408 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f165c0aed10 msgr2=0x7f165c095420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:31.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.372 7f1660982700 1 --2- 172.21.2.101:0/1964904408 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f165c0aed10 0x7f165c095420 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 rx=0x7f164c00b560 tx=0x7f164c00b900).stop 2021-02-10T11:42:31.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.372 7f1660982700 1 -- 172.21.2.101:0/1964904408 shutdown_connections 2021-02-10T11:42:31.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.372 7f1660982700 1 --2- 172.21.2.101:0/1964904408 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f16400186c0 0x7f164001abc0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:31.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.372 7f1660982700 1 --2- 172.21.2.101:0/1964904408 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f165c095a40 0x7f165c095e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:31.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.372 7f1660982700 1 --2- 172.21.2.101:0/1964904408 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f165c08c460 0x7f165c08c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:31.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.373 7f1660982700 1 --2- 172.21.2.101:0/1964904408 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f165c0aed10 0x7f165c095420 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:31.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.373 7f1660982700 1 -- 172.21.2.101:0/1964904408 >> 172.21.2.101:0/1964904408 conn(0x7f165c19f7c0 msgr2=0x7f165c1b6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:31.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.373 7f1660982700 1 -- 172.21.2.101:0/1964904408 shutdown_connections 2021-02-10T11:42:31.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.373 7f1660982700 1 -- 172.21.2.101:0/1964904408 wait complete. 2021-02-10T11:42:31.383 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:42:31.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.795 7f73b2ecb700 1 Processor -- start 2021-02-10T11:42:31.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.795 7f73b2ecb700 1 -- start start 2021-02-10T11:42:31.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.795 7f73b2ecb700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f73ac0d1870 0x7f73ac1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:31.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.795 7f73b2ecb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73ac0f0650 0x7f73ac0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:31.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.795 7f73b2ecb700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73ac0f0b10 con 0x7f73ac1b0a10 2021-02-10T11:42:31.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.795 7f73b2ecb700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73ac0e5ed0 con 0x7f73ac0d1870 2021-02-10T11:42:31.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.795 7f73b2ecb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac0d1c70 con 0x7f73ac0f0650 2021-02-10T11:42:31.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.795 7f73b14e8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73ac0f0650 0x7f73ac0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:31.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.795 7f73b0ce7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f73ac0d1870 0x7f73ac1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:31.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.796 7f73b0ce7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f73ac0d1870 0x7f73ac1b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58176/0 (socket says 172.21.2.101:58176) 2021-02-10T11:42:31.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.796 7f73b14e8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73ac0f0650 0x7f73ac0cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:50910/0 (socket says 172.21.2.101:50910) 2021-02-10T11:42:31.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.796 7f73b14e8700 1 -- 172.21.2.101:0/2695613335 learned_addr learned my addr 172.21.2.101:0/2695613335 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:31.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.796 7f73abfff700 1 --1- 172.21.2.101:0/2695613335 >> v1:172.21.2.108:6789/0 conn(0x7f73ac1b0a10 0x7f73ac0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:52080/0 (socket says 172.21.2.101:52080) 2021-02-10T11:42:31.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.796 7f73b14e8700 1 -- 172.21.2.101:0/2695613335 >> v2:172.21.2.101:3300/0 conn(0x7f73ac0d1870 msgr2=0x7f73ac1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:31.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.796 7f73b14e8700 1 --2- 172.21.2.101:0/2695613335 >> v2:172.21.2.101:3300/0 conn(0x7f73ac0d1870 0x7f73ac1b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:31.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.796 7f73b14e8700 1 -- 172.21.2.101:0/2695613335 >> v1:172.21.2.108:6789/0 conn(0x7f73ac1b0a10 legacy=0x7f73ac0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:31.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.796 7f73b14e8700 1 -- 172.21.2.101:0/2695613335 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac0d1e90 con 0x7f73ac0f0650 2021-02-10T11:42:31.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.796 7f73b14e8700 1 --2- 172.21.2.101:0/2695613335 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73ac0f0650 0x7f73ac0cfb20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f73a4006720 tx=0x7f73a4009ba0).ready entity=mon.2 client_cookie=fbdda21b3808c2f0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:31.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.796 7f73ab7fe700 1 -- 172.21.2.101:0/2695613335 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73a400c0c0 con 0x7f73ac0f0650 2021-02-10T11:42:31.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.796 7f73ab7fe700 1 -- 172.21.2.101:0/2695613335 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a400c400 con 0x7f73ac0f0650 2021-02-10T11:42:31.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.796 7f73ab7fe700 1 -- 172.21.2.101:0/2695613335 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73a400c860 con 0x7f73ac0f0650 2021-02-10T11:42:31.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.797 7f73b2ecb700 1 -- 172.21.2.101:0/2695613335 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73ac0f0650 msgr2=0x7f73ac0cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:31.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.797 7f73b2ecb700 1 --2- 172.21.2.101:0/2695613335 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73ac0f0650 0x7f73ac0cfb20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f73a4006720 tx=0x7f73a4009ba0).stop 2021-02-10T11:42:31.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.797 7f73b2ecb700 1 -- 172.21.2.101:0/2695613335 shutdown_connections 2021-02-10T11:42:31.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.797 7f73b2ecb700 1 --2- 172.21.2.101:0/2695613335 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73ac0f0650 0x7f73ac0cfb20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:31.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.797 7f73b2ecb700 1 --2- 172.21.2.101:0/2695613335 >> v2:172.21.2.101:3300/0 conn(0x7f73ac0d1870 0x7f73ac1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:31.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.797 7f73b2ecb700 1 -- 172.21.2.101:0/2695613335 >> 172.21.2.101:0/2695613335 conn(0x7f73ac0ce450 msgr2=0x7f73ac0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:31.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.797 7f73b2ecb700 1 -- 172.21.2.101:0/2695613335 shutdown_connections 2021-02-10T11:42:31.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.797 7f73b2ecb700 1 -- 172.21.2.101:0/2695613335 wait complete. 2021-02-10T11:42:31.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.812 7f73b2ecb700 1 Processor -- start 2021-02-10T11:42:31.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.812 7f73b2ecb700 1 -- start start 2021-02-10T11:42:31.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.813 7f73b2ecb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73ac0970b0 0x7f73ac0ebe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:31.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.813 7f73b2ecb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73ac098810 0x7f73ac098c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:31.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.813 7f73b2ecb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73ac099230 0x7f73ac0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:31.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.813 7f73b2ecb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac0ec6d0 con 0x7f73ac098810 2021-02-10T11:42:31.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.813 7f73b2ecb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac0974b0 con 0x7f73ac099230 2021-02-10T11:42:31.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.813 7f73b2ecb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac0b1ee0 con 0x7f73ac0970b0 2021-02-10T11:42:31.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.813 7f73abfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73ac098810 0x7f73ac098c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:31.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.813 7f73abfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73ac098810 0x7f73ac098c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58180/0 (socket says 172.21.2.101:58180) 2021-02-10T11:42:31.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.813 7f73abfff700 1 -- 172.21.2.101:0/417136880 learned_addr learned my addr 172.21.2.101:0/417136880 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:31.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.813 7f73abfff700 1 -- 172.21.2.101:0/417136880 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73ac0970b0 msgr2=0x7f73ac0ebe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:31.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.813 7f73abfff700 1 --2- 172.21.2.101:0/417136880 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73ac0970b0 0x7f73ac0ebe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:31.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.813 7f73abfff700 1 -- 172.21.2.101:0/417136880 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73ac099230 msgr2=0x7f73ac0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:31.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.813 7f73abfff700 1 --2- 172.21.2.101:0/417136880 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73ac099230 0x7f73ac0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:31.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.813 7f73abfff700 1 -- 172.21.2.101:0/417136880 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac0b2100 con 0x7f73ac098810 2021-02-10T11:42:31.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.813 7f73abfff700 1 --2- 172.21.2.101:0/417136880 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73ac098810 0x7f73ac098c10 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 rx=0x7f739c00a950 tx=0x7f739c00acf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:31.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.814 7f73a97fa700 1 -- 172.21.2.101:0/417136880 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f739c0011f0 con 0x7f73ac098810 2021-02-10T11:42:31.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.814 7f73a97fa700 1 -- 172.21.2.101:0/417136880 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739c0070d0 con 0x7f73ac098810 2021-02-10T11:42:31.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.814 7f73a97fa700 1 -- 172.21.2.101:0/417136880 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f739c007560 con 0x7f73ac098810 2021-02-10T11:42:31.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.814 7f73b2ecb700 1 -- 172.21.2.101:0/417136880 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73ac0b2100 con 0x7f73ac098810 2021-02-10T11:42:31.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.814 7f73b2ecb700 1 -- 172.21.2.101:0/417136880 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73ac0b2690 con 0x7f73ac098810 2021-02-10T11:42:31.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.814 7f73a97fa700 1 -- 172.21.2.101:0/417136880 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f739c0072b0 con 0x7f73ac098810 2021-02-10T11:42:31.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.814 7f73a97fa700 1 --2- 172.21.2.101:0/417136880 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73900186c0 0x7f739001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:31.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.814 7f73a97fa700 1 -- 172.21.2.101:0/417136880 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f739c0328e0 con 0x7f73ac098810 2021-02-10T11:42:31.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.815 7f73b0ce7700 1 --2- 172.21.2.101:0/417136880 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73900186c0 0x7f739001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:31.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.815 7f73b0ce7700 1 --2- 172.21.2.101:0/417136880 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73900186c0 0x7f739001abc0 crc :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:31.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.845 7f73b2ecb700 1 -- 172.21.2.101:0/417136880 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73ac0ebc00 con 0x7f73ac098810 2021-02-10T11:42:31.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.848 7f73a97fa700 1 -- 172.21.2.101:0/417136880 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f739c0011a0 con 0x7f73ac098810 2021-02-10T11:42:31.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.975 7f73b2ecb700 1 -- 172.21.2.101:0/417136880 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73ac0f0b10 con 0x7f73ac098810 2021-02-10T11:42:31.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.975 7f73a97fa700 1 -- 172.21.2.101:0/417136880 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f739c0072b0 con 0x7f73ac098810 2021-02-10T11:42:31.978 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:42:31.978 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":129,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:42:31.764498","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:42:28.726448","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2021-02-10 11:42:29.754869","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:42:31.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.977 7f73b2ecb700 1 -- 172.21.2.101:0/417136880 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73900186c0 msgr2=0x7f739001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:31.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.977 7f73b2ecb700 1 --2- 172.21.2.101:0/417136880 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73900186c0 0x7f739001abc0 crc :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:31.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.977 7f73b2ecb700 1 -- 172.21.2.101:0/417136880 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73ac098810 msgr2=0x7f73ac098c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:31.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.977 7f73b2ecb700 1 --2- 172.21.2.101:0/417136880 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73ac098810 0x7f73ac098c10 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 rx=0x7f739c00a950 tx=0x7f739c00acf0).stop 2021-02-10T11:42:31.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.978 7f73b2ecb700 1 -- 172.21.2.101:0/417136880 shutdown_connections 2021-02-10T11:42:31.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.978 7f73b2ecb700 1 --2- 172.21.2.101:0/417136880 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73ac099230 0x7f73ac0b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:31.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.978 7f73b2ecb700 1 --2- 172.21.2.101:0/417136880 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73ac098810 0x7f73ac098c10 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:31.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.978 7f73b2ecb700 1 --2- 172.21.2.101:0/417136880 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73900186c0 0x7f739001abc0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:31.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.978 7f73b2ecb700 1 --2- 172.21.2.101:0/417136880 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73ac0970b0 0x7f73ac0ebe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:31.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.978 7f73b2ecb700 1 -- 172.21.2.101:0/417136880 >> 172.21.2.101:0/417136880 conn(0x7f73ac19a730 msgr2=0x7f73ac19e480 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:31.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.978 7f73b2ecb700 1 -- 172.21.2.101:0/417136880 shutdown_connections 2021-02-10T11:42:31.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:31.978 7f73b2ecb700 1 -- 172.21.2.101:0/417136880 wait complete. 2021-02-10T11:42:31.988 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-02-10T11:42:32.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.401 7f3fd55a0700 1 Processor -- start 2021-02-10T11:42:32.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.402 7f3fd55a0700 1 -- start start 2021-02-10T11:42:32.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.402 7f3fd55a0700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3fd00d1870 0x7f3fd01b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:32.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.402 7f3fd55a0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3fd01b0ae0 0x7f3fd0108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:32.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.402 7f3fd55a0700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd00fe1c0 con 0x7f3fd00d1c70 2021-02-10T11:42:32.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.402 7f3fd55a0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd0109a60 con 0x7f3fd01b0ae0 2021-02-10T11:42:32.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.402 7f3fd55a0700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd0109750 con 0x7f3fd00d1870 2021-02-10T11:42:32.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.402 7f3fcfac0700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f3fd00d1c70 0x7f3fd0108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58682/0 (socket says 172.21.2.101:58682) 2021-02-10T11:42:32.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.402 7f3fcfac0700 1 -- 172.21.2.101:0/421085273 learned_addr learned my addr 172.21.2.101:0/421085273 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:32.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.402 7f3fceabe700 1 --2- 172.21.2.101:0/421085273 >> v2:172.21.2.101:3300/0 conn(0x7f3fd01b0ae0 0x7f3fd0108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:32.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.403 7f3fceabe700 1 -- 172.21.2.101:0/421085273 >> v2:172.21.2.108:3300/0 conn(0x7f3fd00d1870 msgr2=0x7f3fd01b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:32.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.403 7f3fceabe700 1 --2- 172.21.2.101:0/421085273 >> v2:172.21.2.108:3300/0 conn(0x7f3fd00d1870 0x7f3fd01b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:32.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.403 7f3fceabe700 1 -- 172.21.2.101:0/421085273 >> v1:172.21.2.101:6789/0 conn(0x7f3fd00d1c70 legacy=0x7f3fd0108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:32.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.403 7f3fce2bd700 1 -- 172.21.2.101:0/421085273 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302495223 0 0) 0x7f3fc8001500 con 0x7f3fd00d1c70 2021-02-10T11:42:32.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.403 7f3fceabe700 1 -- 172.21.2.101:0/421085273 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd0088070 con 0x7f3fd01b0ae0 2021-02-10T11:42:32.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.403 7f3fceabe700 1 --2- 172.21.2.101:0/421085273 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fd01b0ae0 0x7f3fd0108940 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 rx=0x7f3fc0006650 tx=0x7f3fc0009a80).ready entity=mon.0 client_cookie=1ae9b213ddae130a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:32.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.403 7f3fce2bd700 1 -- 172.21.2.101:0/421085273 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3fc00009b0 con 0x7f3fd01b0ae0 2021-02-10T11:42:32.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.403 7f3fce2bd700 1 -- 172.21.2.101:0/421085273 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc000c0c0 con 0x7f3fd01b0ae0 2021-02-10T11:42:32.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.403 7f3fce2bd700 1 -- 172.21.2.101:0/421085273 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3fc000c520 con 0x7f3fd01b0ae0 2021-02-10T11:42:32.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.403 7f3fd55a0700 1 -- 172.21.2.101:0/421085273 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fd01b0ae0 msgr2=0x7f3fd0108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:32.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.403 7f3fd55a0700 1 --2- 172.21.2.101:0/421085273 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fd01b0ae0 0x7f3fd0108940 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 rx=0x7f3fc0006650 tx=0x7f3fc0009a80).stop 2021-02-10T11:42:32.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.404 7f3fd55a0700 1 -- 172.21.2.101:0/421085273 shutdown_connections 2021-02-10T11:42:32.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.404 7f3fd55a0700 1 --2- 172.21.2.101:0/421085273 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fd01b0ae0 0x7f3fd0108940 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:32.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.404 7f3fd55a0700 1 --2- 172.21.2.101:0/421085273 >> v2:172.21.2.108:3300/0 conn(0x7f3fd00d1870 0x7f3fd01b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:32.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.404 7f3fd55a0700 1 -- 172.21.2.101:0/421085273 >> 172.21.2.101:0/421085273 conn(0x7f3fd00ce420 msgr2=0x7f3fd00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:32.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.404 7f3fd55a0700 1 -- 172.21.2.101:0/421085273 shutdown_connections 2021-02-10T11:42:32.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.404 7f3fd55a0700 1 -- 172.21.2.101:0/421085273 wait complete. 2021-02-10T11:42:32.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.418 7f3fd55a0700 1 Processor -- start 2021-02-10T11:42:32.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.418 7f3fd55a0700 1 -- start start 2021-02-10T11:42:32.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.419 7f3fd55a0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3fd019e410 0x7f3fd008a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:32.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.419 7f3fd55a0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3fd0083980 0x7f3fd0083d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:32.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.419 7f3fd55a0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fd00baa30 0x7f3fd00bae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:32.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.419 7f3fd55a0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd00ba4b0 con 0x7f3fd00baa30 2021-02-10T11:42:32.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.419 7f3fd55a0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd00843a0 con 0x7f3fd019e410 2021-02-10T11:42:32.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.419 7f3fd55a0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd0084710 con 0x7f3fd0083980 2021-02-10T11:42:32.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.419 7f3fcfac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fd00baa30 0x7f3fd00bae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:32.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.419 7f3fcfac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fd00baa30 0x7f3fd00bae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58194/0 (socket says 172.21.2.101:58194) 2021-02-10T11:42:32.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.419 7f3fcfac0700 1 -- 172.21.2.101:0/327314670 learned_addr learned my addr 172.21.2.101:0/327314670 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:32.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.419 7f3fcf2bf700 1 --2- 172.21.2.101:0/327314670 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3fd019e410 0x7f3fd008a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:32.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.419 7f3fceabe700 1 --2- 172.21.2.101:0/327314670 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3fd0083980 0x7f3fd0083d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:32.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.419 7f3fcfac0700 1 -- 172.21.2.101:0/327314670 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3fd0083980 msgr2=0x7f3fd0083d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:32.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.419 7f3fcfac0700 1 --2- 172.21.2.101:0/327314670 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3fd0083980 0x7f3fd0083d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:32.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.419 7f3fcfac0700 1 -- 172.21.2.101:0/327314670 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3fd019e410 msgr2=0x7f3fd008a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:32.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.419 7f3fcfac0700 1 --2- 172.21.2.101:0/327314670 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3fd019e410 0x7f3fd008a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:32.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.420 7f3fcfac0700 1 -- 172.21.2.101:0/327314670 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd00bb450 con 0x7f3fd00baa30 2021-02-10T11:42:32.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.420 7f3fcfac0700 1 --2- 172.21.2.101:0/327314670 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fd00baa30 0x7f3fd00bae30 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 rx=0x7f3fc8001c10 tx=0x7f3fc800ac60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:32.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.420 7f3fc7fff700 1 -- 172.21.2.101:0/327314670 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3fc8000f90 con 0x7f3fd00baa30 2021-02-10T11:42:32.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.420 7f3fc7fff700 1 -- 172.21.2.101:0/327314670 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc8007020 con 0x7f3fd00baa30 2021-02-10T11:42:32.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.420 7f3fc7fff700 1 -- 172.21.2.101:0/327314670 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3fc80074b0 con 0x7f3fd00baa30 2021-02-10T11:42:32.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.420 7f3fd55a0700 1 -- 172.21.2.101:0/327314670 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd00bb450 con 0x7f3fd00baa30 2021-02-10T11:42:32.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.420 7f3fd55a0700 1 -- 172.21.2.101:0/327314670 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd00bb450 con 0x7f3fd00baa30 2021-02-10T11:42:32.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.420 7f3fc7fff700 1 -- 172.21.2.101:0/327314670 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3fc8007200 con 0x7f3fd00baa30 2021-02-10T11:42:32.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.420 7f3fc7fff700 1 --2- 172.21.2.101:0/327314670 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3fb40186c0 0x7f3fb401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:32.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.420 7f3fc7fff700 1 -- 172.21.2.101:0/327314670 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3fc8032750 con 0x7f3fd00baa30 2021-02-10T11:42:32.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.421 7f3fcf2bf700 1 --2- 172.21.2.101:0/327314670 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3fb40186c0 0x7f3fb401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:32.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.421 7f3fcf2bf700 1 --2- 172.21.2.101:0/327314670 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3fb40186c0 0x7f3fb401abc0 crc :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:32.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.452 7f3fd55a0700 1 -- 172.21.2.101:0/327314670 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd0082a70 con 0x7f3fd00baa30 2021-02-10T11:42:32.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.455 7f3fc7fff700 1 -- 172.21.2.101:0/327314670 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3fc8000cc0 con 0x7f3fd00baa30 2021-02-10T11:42:32.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.590 7f3fd55a0700 1 -- 172.21.2.101:0/327314670 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"} v 0) v1 -- 0x7f3fd0082a70 con 0x7f3fd00baa30 2021-02-10T11:42:32.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.775 7f3fc7fff700 1 -- 172.21.2.101:0/327314670 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"}]=0 v105) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3fc8007200 con 0x7f3fd00baa30 2021-02-10T11:42:32.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.806 7f3fd55a0700 1 -- 172.21.2.101:0/327314670 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3fb40186c0 msgr2=0x7f3fb401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:32.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.806 7f3fd55a0700 1 --2- 172.21.2.101:0/327314670 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3fb40186c0 0x7f3fb401abc0 crc :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:32.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.806 7f3fd55a0700 1 -- 172.21.2.101:0/327314670 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fd00baa30 msgr2=0x7f3fd00bae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:32.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.806 7f3fd55a0700 1 --2- 172.21.2.101:0/327314670 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fd00baa30 0x7f3fd00bae30 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 rx=0x7f3fc8001c10 tx=0x7f3fc800ac60).stop 2021-02-10T11:42:32.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.807 7f3fd55a0700 1 -- 172.21.2.101:0/327314670 shutdown_connections 2021-02-10T11:42:32.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.807 7f3fd55a0700 1 --2- 172.21.2.101:0/327314670 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3fb40186c0 0x7f3fb401abc0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:32.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.807 7f3fd55a0700 1 --2- 172.21.2.101:0/327314670 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3fd0083980 0x7f3fd0083d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:32.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.807 7f3fd55a0700 1 --2- 172.21.2.101:0/327314670 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3fd00baa30 0x7f3fd00bae30 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:32.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.807 7f3fd55a0700 1 --2- 172.21.2.101:0/327314670 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3fd019e410 0x7f3fd008a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:32.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.807 7f3fd55a0700 1 -- 172.21.2.101:0/327314670 >> 172.21.2.101:0/327314670 conn(0x7f3fd00bc0c0 msgr2=0x7f3fd0082ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:32.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.807 7f3fd55a0700 1 -- 172.21.2.101:0/327314670 shutdown_connections 2021-02-10T11:42:32.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:32.807 7f3fd55a0700 1 -- 172.21.2.101:0/327314670 wait complete. 2021-02-10T11:42:32.818 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:42:33.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.228 7f15a7962700 1 Processor -- start 2021-02-10T11:42:33.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.228 7f15a7962700 1 -- start start 2021-02-10T11:42:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.229 7f15a7962700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f15a00f05f0 0x7f15a00f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.229 7f15a7962700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a01b0c70 con 0x7f15a01b0870 2021-02-10T11:42:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.229 7f15a7962700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a00f0b50 con 0x7f15a01b0470 2021-02-10T11:42:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.229 7f15a7962700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a00e5ea0 con 0x7f15a00f05f0 2021-02-10T11:42:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.229 7f15a4f7d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f15a01b0870 0x7f15a00efe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58692/0 (socket says 172.21.2.101:58692) 2021-02-10T11:42:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.229 7f15a4f7d700 1 -- 172.21.2.101:0/1106504776 learned_addr learned my addr 172.21.2.101:0/1106504776 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.229 7f15a5f7f700 1 --2- 172.21.2.101:0/1106504776 >> v2:172.21.2.108:3300/0 conn(0x7f15a00f05f0 0x7f15a00f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.229 7f1593fff700 1 -- 172.21.2.101:0/1106504776 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 21002021 0 0) 0x7f1594001180 con 0x7f15a01b0870 2021-02-10T11:42:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.229 7f1593fff700 1 -- 172.21.2.101:0/1106504776 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f158c002fe0 con 0x7f15a01b0870 2021-02-10T11:42:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.230 7f15a5f7f700 1 -- 172.21.2.101:0/1106504776 >> v1:172.21.2.108:6789/0 conn(0x7f15a01b0470 legacy=0x7f15a00ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.230 7f15a5f7f700 1 -- 172.21.2.101:0/1106504776 >> v1:172.21.2.101:6789/0 conn(0x7f15a01b0870 legacy=0x7f15a00efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.230 7f15a5f7f700 1 -- 172.21.2.101:0/1106504776 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a00e52e0 con 0x7f15a00f05f0 2021-02-10T11:42:33.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.230 7f15a5f7f700 1 --2- 172.21.2.101:0/1106504776 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15a00f05f0 0x7f15a00f20e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f1598006e50 tx=0x7f1598009e70).ready entity=mon.1 client_cookie=25d1ad68ba5943cc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.230 7f1593fff700 1 -- 172.21.2.101:0/1106504776 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f159800b510 con 0x7f15a00f05f0 2021-02-10T11:42:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.230 7f1593fff700 1 -- 172.21.2.101:0/1106504776 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1598006790 con 0x7f15a00f05f0 2021-02-10T11:42:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.230 7f1593fff700 1 -- 172.21.2.101:0/1106504776 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f159800b940 con 0x7f15a00f05f0 2021-02-10T11:42:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.230 7f15a7962700 1 -- 172.21.2.101:0/1106504776 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15a00f05f0 msgr2=0x7f15a00f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.230 7f15a7962700 1 --2- 172.21.2.101:0/1106504776 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15a00f05f0 0x7f15a00f20e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f1598006e50 tx=0x7f1598009e70).stop 2021-02-10T11:42:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.231 7f15a7962700 1 -- 172.21.2.101:0/1106504776 shutdown_connections 2021-02-10T11:42:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.231 7f15a7962700 1 --2- 172.21.2.101:0/1106504776 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15a00f05f0 0x7f15a00f20e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.231 7f15a7962700 1 -- 172.21.2.101:0/1106504776 >> 172.21.2.101:0/1106504776 conn(0x7f15a00ce3e0 msgr2=0x7f15a00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.231 7f15a7962700 1 -- 172.21.2.101:0/1106504776 shutdown_connections 2021-02-10T11:42:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.231 7f15a7962700 1 -- 172.21.2.101:0/1106504776 wait complete. 2021-02-10T11:42:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.246 7f15a7962700 1 Processor -- start 2021-02-10T11:42:33.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.246 7f15a7962700 1 -- start start 2021-02-10T11:42:33.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.246 7f15a7962700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15a0092e40 0x7f15a0093240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:33.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.246 7f15a7962700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15a0093860 0x7f15a01a83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:33.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.246 7f15a7962700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15a01a89f0 0x7f15a01a8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:33.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.246 7f15a7962700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a0195ad0 con 0x7f15a01a89f0 2021-02-10T11:42:33.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.246 7f15a7962700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a01a9450 con 0x7f15a0092e40 2021-02-10T11:42:33.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.246 7f15a7962700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a00b1b00 con 0x7f15a0093860 2021-02-10T11:42:33.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.246 7f15a5f7f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15a01a89f0 0x7f15a01a8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:33.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.246 7f15a5f7f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15a01a89f0 0x7f15a01a8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58202/0 (socket says 172.21.2.101:58202) 2021-02-10T11:42:33.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.246 7f15a5f7f700 1 -- 172.21.2.101:0/3857866 learned_addr learned my addr 172.21.2.101:0/3857866 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:33.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.247 7f15a5f7f700 1 -- 172.21.2.101:0/3857866 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15a0093860 msgr2=0x7f15a01a83d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:42:33.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.247 7f15a5f7f700 1 --2- 172.21.2.101:0/3857866 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15a0093860 0x7f15a01a83d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:33.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.247 7f15a5f7f700 1 -- 172.21.2.101:0/3857866 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15a0092e40 msgr2=0x7f15a0093240 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:42:33.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.247 7f15a5f7f700 1 --2- 172.21.2.101:0/3857866 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15a0092e40 0x7f15a0093240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:33.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.247 7f15a5f7f700 1 -- 172.21.2.101:0/3857866 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a01a99d0 con 0x7f15a01a89f0 2021-02-10T11:42:33.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.247 7f15a5f7f700 1 --2- 172.21.2.101:0/3857866 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15a01a89f0 0x7f15a01a8df0 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 rx=0x7f1598009a90 tx=0x7f159800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:33.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.247 7f1591ffb700 1 -- 172.21.2.101:0/3857866 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1598000df0 con 0x7f15a01a89f0 2021-02-10T11:42:33.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.247 7f1591ffb700 1 -- 172.21.2.101:0/3857866 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1598002ef0 con 0x7f15a01a89f0 2021-02-10T11:42:33.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.247 7f15a7962700 1 -- 172.21.2.101:0/3857866 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a01a99d0 con 0x7f15a01a89f0 2021-02-10T11:42:33.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.247 7f15a7962700 1 -- 172.21.2.101:0/3857866 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a01a99d0 con 0x7f15a01a89f0 2021-02-10T11:42:33.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.247 7f1591ffb700 1 -- 172.21.2.101:0/3857866 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f159800c4e0 con 0x7f15a01a89f0 2021-02-10T11:42:33.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.247 7f1591ffb700 1 -- 172.21.2.101:0/3857866 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1598009170 con 0x7f15a01a89f0 2021-02-10T11:42:33.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.247 7f1591ffb700 1 --2- 172.21.2.101:0/3857866 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1580018730 0x7f158001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:33.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.247 7f1591ffb700 1 -- 172.21.2.101:0/3857866 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1598032b30 con 0x7f15a01a89f0 2021-02-10T11:42:33.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.248 7f15a577e700 1 --2- 172.21.2.101:0/3857866 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1580018730 0x7f158001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:33.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.248 7f15a577e700 1 --2- 172.21.2.101:0/3857866 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1580018730 0x7f158001ac10 crc :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:33.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.279 7f15a7962700 1 -- 172.21.2.101:0/3857866 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a00b1b00 con 0x7f15a01a89f0 2021-02-10T11:42:33.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.281 7f1591ffb700 1 -- 172.21.2.101:0/3857866 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f159800be30 con 0x7f15a01a89f0 2021-02-10T11:42:33.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.410 7f15a7962700 1 -- 172.21.2.101:0/3857866 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15a00b1b00 con 0x7f15a01a89f0 2021-02-10T11:42:33.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.411 7f1591ffb700 1 -- 172.21.2.101:0/3857866 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+2105 (secure 0 0 0) 0x7f1598002ef0 con 0x7f15a01a89f0 2021-02-10T11:42:33.413 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 105 2021-02-10T11:42:33.413 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:42:33.413 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":105,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11139,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1647083555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1647083555},{"type":"v1","addr":"172.21.2.101:6835","nonce":1647083555}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":101}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:42:30.762357","modified":"2021-02-10 11:42:32.772467","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7816":{"gid":7816,"name":"b","rank":0,"incarnation":102,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/381011050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":381011050},{"type":"v1","addr":"172.21.2.108:6833","nonce":381011050}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-02-10T11:42:33.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.412 7f15a7962700 1 -- 172.21.2.101:0/3857866 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1580018730 msgr2=0x7f158001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:33.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.412 7f15a7962700 1 --2- 172.21.2.101:0/3857866 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1580018730 0x7f158001ac10 crc :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:33.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.412 7f15a7962700 1 -- 172.21.2.101:0/3857866 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15a01a89f0 msgr2=0x7f15a01a8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:33.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.412 7f15a7962700 1 --2- 172.21.2.101:0/3857866 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15a01a89f0 0x7f15a01a8df0 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 rx=0x7f1598009a90 tx=0x7f159800bcf0).stop 2021-02-10T11:42:33.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.413 7f15a7962700 1 -- 172.21.2.101:0/3857866 shutdown_connections 2021-02-10T11:42:33.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.413 7f15a7962700 1 --2- 172.21.2.101:0/3857866 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1580018730 0x7f158001ac10 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:33.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.413 7f15a7962700 1 --2- 172.21.2.101:0/3857866 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15a0093860 0x7f15a01a83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:33.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.413 7f15a7962700 1 --2- 172.21.2.101:0/3857866 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15a01a89f0 0x7f15a01a8df0 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:33.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.413 7f15a7962700 1 --2- 172.21.2.101:0/3857866 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15a0092e40 0x7f15a0093240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:33.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.413 7f15a7962700 1 -- 172.21.2.101:0/3857866 >> 172.21.2.101:0/3857866 conn(0x7f15a019f7a0 msgr2=0x7f15a00940e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:33.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.413 7f15a7962700 1 -- 172.21.2.101:0/3857866 shutdown_connections 2021-02-10T11:42:33.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.413 7f15a7962700 1 -- 172.21.2.101:0/3857866 wait complete. 2021-02-10T11:42:33.423 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:42:33.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.833 7f42fd189700 1 Processor -- start 2021-02-10T11:42:33.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.833 7f42fd189700 1 -- start start 2021-02-10T11:42:33.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.834 7f42fd189700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f42f80d1870 0x7f42f81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:33.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.834 7f42fd189700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f42f80f0650 0x7f42f80cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:33.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.834 7f42fd189700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f80f0b10 con 0x7f42f81b0a10 2021-02-10T11:42:33.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.834 7f42fd189700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f80e5ed0 con 0x7f42f80d1870 2021-02-10T11:42:33.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.834 7f42fd189700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80d1c70 con 0x7f42f80f0650 2021-02-10T11:42:33.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.834 7f42f659c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f42f81b0a10 0x7f42f80cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58706/0 (socket says 172.21.2.101:58706) 2021-02-10T11:42:33.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.834 7f42f659c700 1 -- 172.21.2.101:0/3387078429 learned_addr learned my addr 172.21.2.101:0/3387078429 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:33.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.834 7f42f6d9d700 1 --2- 172.21.2.101:0/3387078429 >> v2:172.21.2.101:3300/0 conn(0x7f42f80d1870 0x7f42f81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:33.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.834 7f42f5d9b700 1 -- 172.21.2.101:0/3387078429 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909597614 0 0) 0x7f42e00010b0 con 0x7f42f81b0a10 2021-02-10T11:42:33.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.834 7f42f5d9b700 1 -- 172.21.2.101:0/3387078429 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42e4002fe0 con 0x7f42f81b0a10 2021-02-10T11:42:33.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.834 7f42f6d9d700 1 -- 172.21.2.101:0/3387078429 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f42f80f0650 msgr2=0x7f42f80cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:33.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.834 7f42f6d9d700 1 --2- 172.21.2.101:0/3387078429 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f42f80f0650 0x7f42f80cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:33.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.834 7f42f6d9d700 1 -- 172.21.2.101:0/3387078429 >> v1:172.21.2.101:6789/0 conn(0x7f42f81b0a10 legacy=0x7f42f80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:33.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.834 7f42f6d9d700 1 -- 172.21.2.101:0/3387078429 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f80d1e90 con 0x7f42f80d1870 2021-02-10T11:42:33.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.834 7f42f6d9d700 1 --2- 172.21.2.101:0/3387078429 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42f80d1870 0x7f42f81b03f0 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 rx=0x7f42e8006650 tx=0x7f42e8009a80).ready entity=mon.0 client_cookie=7eadf54822cecaa5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:33.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.835 7f42f5d9b700 1 -- 172.21.2.101:0/3387078429 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f42e80009b0 con 0x7f42f80d1870 2021-02-10T11:42:33.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.835 7f42f5d9b700 1 -- 172.21.2.101:0/3387078429 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e800c0c0 con 0x7f42f80d1870 2021-02-10T11:42:33.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.835 7f42f5d9b700 1 -- 172.21.2.101:0/3387078429 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f42e800c5e0 con 0x7f42f80d1870 2021-02-10T11:42:33.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.835 7f42fd189700 1 -- 172.21.2.101:0/3387078429 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42f80d1870 msgr2=0x7f42f81b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:33.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.835 7f42fd189700 1 --2- 172.21.2.101:0/3387078429 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42f80d1870 0x7f42f81b03f0 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 rx=0x7f42e8006650 tx=0x7f42e8009a80).stop 2021-02-10T11:42:33.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.836 7f42fd189700 1 -- 172.21.2.101:0/3387078429 shutdown_connections 2021-02-10T11:42:33.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.836 7f42fd189700 1 --2- 172.21.2.101:0/3387078429 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f42f80f0650 0x7f42f80cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:33.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.836 7f42fd189700 1 --2- 172.21.2.101:0/3387078429 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42f80d1870 0x7f42f81b03f0 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:33.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.836 7f42fd189700 1 -- 172.21.2.101:0/3387078429 >> 172.21.2.101:0/3387078429 conn(0x7f42f80ce450 msgr2=0x7f42f80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:33.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.836 7f42fd189700 1 -- 172.21.2.101:0/3387078429 shutdown_connections 2021-02-10T11:42:33.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.836 7f42fd189700 1 -- 172.21.2.101:0/3387078429 wait complete. 2021-02-10T11:42:33.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.850 7f42fd189700 1 Processor -- start 2021-02-10T11:42:33.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.850 7f42fd189700 1 -- start start 2021-02-10T11:42:33.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.851 7f42fd189700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f42f81a4a60 0x7f42f80c1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:33.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.851 7f42fd189700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42f80c2290 0x7f42f80c2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:33.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.851 7f42fd189700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f42f80c2cb0 0x7f42f80bfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:33.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.851 7f42fd189700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80f05b0 con 0x7f42f80c2290 2021-02-10T11:42:33.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.851 7f42fd189700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80f07d0 con 0x7f42f81a4a60 2021-02-10T11:42:33.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.851 7f42fd189700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80c0340 con 0x7f42f80c2cb0 2021-02-10T11:42:33.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.851 7f42f659c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42f80c2290 0x7f42f80c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:33.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.851 7f42f659c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42f80c2290 0x7f42f80c2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58218/0 (socket says 172.21.2.101:58218) 2021-02-10T11:42:33.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.851 7f42f659c700 1 -- 172.21.2.101:0/2337189685 learned_addr learned my addr 172.21.2.101:0/2337189685 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:33.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.851 7f42f759e700 1 --2- 172.21.2.101:0/2337189685 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f42f80c2cb0 0x7f42f80bfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:33.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.851 7f42f6d9d700 1 --2- 172.21.2.101:0/2337189685 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f42f81a4a60 0x7f42f80c1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:33.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.851 7f42f659c700 1 -- 172.21.2.101:0/2337189685 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f42f80c2cb0 msgr2=0x7f42f80bfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:33.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.851 7f42f659c700 1 --2- 172.21.2.101:0/2337189685 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f42f80c2cb0 0x7f42f80bfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:33.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.851 7f42f659c700 1 -- 172.21.2.101:0/2337189685 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f42f81a4a60 msgr2=0x7f42f80c1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:33.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.851 7f42f659c700 1 --2- 172.21.2.101:0/2337189685 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f42f81a4a60 0x7f42f80c1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:33.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.851 7f42f659c700 1 -- 172.21.2.101:0/2337189685 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f80c0560 con 0x7f42f80c2290 2021-02-10T11:42:33.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.852 7f42f659c700 1 --2- 172.21.2.101:0/2337189685 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42f80c2290 0x7f42f80c2690 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 rx=0x7f42e0001c60 tx=0x7f42e000a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:33.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.852 7f42ef7fe700 1 -- 172.21.2.101:0/2337189685 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f42e000acc0 con 0x7f42f80c2290 2021-02-10T11:42:33.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.852 7f42ef7fe700 1 -- 172.21.2.101:0/2337189685 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e0004350 con 0x7f42f80c2290 2021-02-10T11:42:33.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.852 7f42ef7fe700 1 -- 172.21.2.101:0/2337189685 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f42e0004830 con 0x7f42f80c2290 2021-02-10T11:42:33.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.852 7f42fd189700 1 -- 172.21.2.101:0/2337189685 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f80c0560 con 0x7f42f80c2290 2021-02-10T11:42:33.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.852 7f42fd189700 1 -- 172.21.2.101:0/2337189685 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f80c0af0 con 0x7f42f80c2290 2021-02-10T11:42:33.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.852 7f42ef7fe700 1 -- 172.21.2.101:0/2337189685 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f42e0004530 con 0x7f42f80c2290 2021-02-10T11:42:33.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.852 7f42ef7fe700 1 --2- 172.21.2.101:0/2337189685 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f42dc0186c0 0x7f42dc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:33.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.852 7f42ef7fe700 1 -- 172.21.2.101:0/2337189685 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f42e0032560 con 0x7f42f80c2290 2021-02-10T11:42:33.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.852 7f42f6d9d700 1 --2- 172.21.2.101:0/2337189685 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f42dc0186c0 0x7f42dc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:33.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.853 7f42f6d9d700 1 --2- 172.21.2.101:0/2337189685 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f42dc0186c0 0x7f42dc01abc0 crc :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:33.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.883 7f42fd189700 1 -- 172.21.2.101:0/2337189685 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f80f0af0 con 0x7f42f80c2290 2021-02-10T11:42:33.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:33.886 7f42ef7fe700 1 -- 172.21.2.101:0/2337189685 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f42e0000940 con 0x7f42f80c2290 2021-02-10T11:42:34.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.009 7f42fd189700 1 -- 172.21.2.101:0/2337189685 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42f80f0af0 con 0x7f42f80c2290 2021-02-10T11:42:34.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.009 7f42ef7fe700 1 -- 172.21.2.101:0/2337189685 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f42e0004510 con 0x7f42f80c2290 2021-02-10T11:42:34.011 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:42:34.012 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":129,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:42:31.764498","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:42:28.726448","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2021-02-10 11:42:29.754869","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:42:34.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.011 7f42fd189700 1 -- 172.21.2.101:0/2337189685 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f42dc0186c0 msgr2=0x7f42dc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:34.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.011 7f42fd189700 1 --2- 172.21.2.101:0/2337189685 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f42dc0186c0 0x7f42dc01abc0 crc :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:34.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.011 7f42fd189700 1 -- 172.21.2.101:0/2337189685 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42f80c2290 msgr2=0x7f42f80c2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:34.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.011 7f42fd189700 1 --2- 172.21.2.101:0/2337189685 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42f80c2290 0x7f42f80c2690 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 rx=0x7f42e0001c60 tx=0x7f42e000a8d0).stop 2021-02-10T11:42:34.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.011 7f42fd189700 1 -- 172.21.2.101:0/2337189685 shutdown_connections 2021-02-10T11:42:34.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.011 7f42fd189700 1 --2- 172.21.2.101:0/2337189685 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f42dc0186c0 0x7f42dc01abc0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:34.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.011 7f42fd189700 1 --2- 172.21.2.101:0/2337189685 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f42f80c2cb0 0x7f42f80bfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:34.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.011 7f42fd189700 1 --2- 172.21.2.101:0/2337189685 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42f80c2290 0x7f42f80c2690 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:34.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.011 7f42fd189700 1 --2- 172.21.2.101:0/2337189685 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f42f81a4a60 0x7f42f80c1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:34.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.011 7f42fd189700 1 -- 172.21.2.101:0/2337189685 >> 172.21.2.101:0/2337189685 conn(0x7f42f80ebe30 msgr2=0x7f42f80bf200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:34.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.011 7f42fd189700 1 -- 172.21.2.101:0/2337189685 shutdown_connections 2021-02-10T11:42:34.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.012 7f42fd189700 1 -- 172.21.2.101:0/2337189685 wait complete. 2021-02-10T11:42:34.022 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:42:34.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.445 7f349a98a700 1 Processor -- start 2021-02-10T11:42:34.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.445 7f349a98a700 1 -- start start 2021-02-10T11:42:34.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.445 7f349a98a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f34940d1860 0x7f34941b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:34.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.445 7f349a98a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34940fe1b0 con 0x7f34940cff60 2021-02-10T11:42:34.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.445 7f349a98a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3494109a50 con 0x7f34941b0ad0 2021-02-10T11:42:34.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.445 7f349a98a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3494088070 con 0x7f34940d1860 2021-02-10T11:42:34.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.446 7f3498fa7700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f34940cff60 0x7f3494109120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58718/0 (socket says 172.21.2.101:58718) 2021-02-10T11:42:34.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.446 7f3493fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f34940d1860 0x7f34941b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:34.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.446 7f3498fa7700 1 -- 172.21.2.101:0/1513861538 learned_addr learned my addr 172.21.2.101:0/1513861538 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:34.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.446 7f34937fe700 1 --1- 172.21.2.101:0/1513861538 >> v1:172.21.2.108:6789/0 conn(0x7f34941b0ad0 0x7f3494108930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:52128/0 (socket says 172.21.2.101:52128) 2021-02-10T11:42:34.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.446 7f3493fff700 1 -- 172.21.2.101:0/1513861538 >> v1:172.21.2.108:6789/0 conn(0x7f34941b0ad0 legacy=0x7f3494108930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:34.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.446 7f3493fff700 1 -- 172.21.2.101:0/1513861538 >> v1:172.21.2.101:6789/0 conn(0x7f34940cff60 legacy=0x7f3494109120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:34.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.446 7f3492ffd700 1 -- 172.21.2.101:0/1513861538 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312038416 0 0) 0x7f348c000d30 con 0x7f34940cff60 2021-02-10T11:42:34.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.446 7f3493fff700 1 -- 172.21.2.101:0/1513861538 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3494088260 con 0x7f34940d1860 2021-02-10T11:42:34.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.446 7f3493fff700 1 --2- 172.21.2.101:0/1513861538 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f34940d1860 0x7f34941b04b0 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 rx=0x7f3484006650 tx=0x7f3484009a80).ready entity=mon.0 client_cookie=93a60010a006d943 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:34.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.446 7f3492ffd700 1 -- 172.21.2.101:0/1513861538 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f34840009b0 con 0x7f34940d1860 2021-02-10T11:42:34.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.446 7f3492ffd700 1 -- 172.21.2.101:0/1513861538 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348400c0c0 con 0x7f34940d1860 2021-02-10T11:42:34.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.446 7f3492ffd700 1 -- 172.21.2.101:0/1513861538 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f348400c5e0 con 0x7f34940d1860 2021-02-10T11:42:34.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.446 7f349a98a700 1 -- 172.21.2.101:0/1513861538 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f34940d1860 msgr2=0x7f34941b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:34.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.446 7f349a98a700 1 --2- 172.21.2.101:0/1513861538 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f34940d1860 0x7f34941b04b0 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 rx=0x7f3484006650 tx=0x7f3484009a80).stop 2021-02-10T11:42:34.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.447 7f349a98a700 1 -- 172.21.2.101:0/1513861538 shutdown_connections 2021-02-10T11:42:34.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.447 7f349a98a700 1 --2- 172.21.2.101:0/1513861538 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f34940d1860 0x7f34941b04b0 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:34.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.447 7f349a98a700 1 -- 172.21.2.101:0/1513861538 >> 172.21.2.101:0/1513861538 conn(0x7f34940ce410 msgr2=0x7f34940d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:34.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.447 7f349a98a700 1 -- 172.21.2.101:0/1513861538 shutdown_connections 2021-02-10T11:42:34.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.447 7f349a98a700 1 -- 172.21.2.101:0/1513861538 wait complete. 2021-02-10T11:42:34.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.462 7f349a98a700 1 Processor -- start 2021-02-10T11:42:34.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.462 7f349a98a700 1 -- start start 2021-02-10T11:42:34.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.463 7f349a98a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3494195500 0x7f34940b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:34.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.463 7f349a98a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f34940b0a30 0x7f349418f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:34.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.463 7f349a98a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f349418f990 0x7f3494190680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:34.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.463 7f349a98a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3494195a60 con 0x7f3494195500 2021-02-10T11:42:34.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.463 7f349a98a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34941a5970 con 0x7f34940b0a30 2021-02-10T11:42:34.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.463 7f349a98a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349418fd90 con 0x7f349418f990 2021-02-10T11:42:34.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.463 7f3493fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3494195500 0x7f34940b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:34.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.463 7f3493fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3494195500 0x7f34940b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58228/0 (socket says 172.21.2.101:58228) 2021-02-10T11:42:34.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.463 7f3493fff700 1 -- 172.21.2.101:0/490043348 learned_addr learned my addr 172.21.2.101:0/490043348 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:34.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.463 7f34937fe700 1 --2- 172.21.2.101:0/490043348 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f34940b0a30 0x7f349418f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:34.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.463 7f3498fa7700 1 --2- 172.21.2.101:0/490043348 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f349418f990 0x7f3494190680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:34.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.463 7f3493fff700 1 -- 172.21.2.101:0/490043348 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f349418f990 msgr2=0x7f3494190680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:34.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.463 7f3493fff700 1 --2- 172.21.2.101:0/490043348 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f349418f990 0x7f3494190680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:34.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.463 7f3493fff700 1 -- 172.21.2.101:0/490043348 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f34940b0a30 msgr2=0x7f349418f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:34.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.463 7f3493fff700 1 --2- 172.21.2.101:0/490043348 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f34940b0a30 0x7f349418f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:34.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.464 7f3493fff700 1 -- 172.21.2.101:0/490043348 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3494190ca0 con 0x7f3494195500 2021-02-10T11:42:34.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.464 7f34937fe700 1 --2- 172.21.2.101:0/490043348 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f34940b0a30 0x7f349418f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:42:34.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.464 7f3493fff700 1 --2- 172.21.2.101:0/490043348 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3494195500 0x7f34940b0410 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 rx=0x7f3484009630 tx=0x7f348400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:34.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.464 7f3490ff9700 1 -- 172.21.2.101:0/490043348 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f34840093a0 con 0x7f3494195500 2021-02-10T11:42:34.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.464 7f3490ff9700 1 -- 172.21.2.101:0/490043348 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348400bd70 con 0x7f3494195500 2021-02-10T11:42:34.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.464 7f3490ff9700 1 -- 172.21.2.101:0/490043348 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3484005560 con 0x7f3494195500 2021-02-10T11:42:34.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.464 7f349a98a700 1 -- 172.21.2.101:0/490043348 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3494190ca0 con 0x7f3494195500 2021-02-10T11:42:34.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.464 7f349a98a700 1 -- 172.21.2.101:0/490043348 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3494190ca0 con 0x7f3494195500 2021-02-10T11:42:34.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.464 7f3490ff9700 1 -- 172.21.2.101:0/490043348 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3484005770 con 0x7f3494195500 2021-02-10T11:42:34.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.465 7f3490ff9700 1 --2- 172.21.2.101:0/490043348 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3470018700 0x7f347001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:34.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.465 7f3490ff9700 1 -- 172.21.2.101:0/490043348 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3484032680 con 0x7f3494195500 2021-02-10T11:42:34.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.465 7f34937fe700 1 --2- 172.21.2.101:0/490043348 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3470018700 0x7f347001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:34.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.465 7f34937fe700 1 --2- 172.21.2.101:0/490043348 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3470018700 0x7f347001abc0 crc :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:34.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.495 7f349a98a700 1 -- 172.21.2.101:0/490043348 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3494082980 con 0x7f3494195500 2021-02-10T11:42:34.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.497 7f3490ff9700 1 -- 172.21.2.101:0/490043348 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3484005560 con 0x7f3494195500 2021-02-10T11:42:34.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.628 7f349a98a700 1 -- 172.21.2.101:0/490043348 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f3494082980 con 0x7f3494195500 2021-02-10T11:42:34.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.632 7f3490ff9700 1 -- 172.21.2.101:0/490043348 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v75) v1 ==== 184+0+0 (secure 0 0 0) 0x7f3484005a70 con 0x7f3494195500 2021-02-10T11:42:34.637 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.0 2021-02-10T11:42:34.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.637 7f349a98a700 1 -- 172.21.2.101:0/490043348 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3470018700 msgr2=0x7f347001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:34.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.637 7f349a98a700 1 --2- 172.21.2.101:0/490043348 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3470018700 0x7f347001abc0 crc :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:34.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.637 7f349a98a700 1 -- 172.21.2.101:0/490043348 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3494195500 msgr2=0x7f34940b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:34.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.637 7f349a98a700 1 --2- 172.21.2.101:0/490043348 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3494195500 0x7f34940b0410 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 rx=0x7f3484009630 tx=0x7f348400b980).stop 2021-02-10T11:42:34.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.637 7f349a98a700 1 -- 172.21.2.101:0/490043348 shutdown_connections 2021-02-10T11:42:34.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.637 7f349a98a700 1 --2- 172.21.2.101:0/490043348 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3470018700 0x7f347001abc0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:34.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.637 7f349a98a700 1 --2- 172.21.2.101:0/490043348 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f349418f990 0x7f3494190680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:34.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.637 7f349a98a700 1 --2- 172.21.2.101:0/490043348 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f34940b0a30 0x7f349418f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:34.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.637 7f349a98a700 1 --2- 172.21.2.101:0/490043348 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3494195500 0x7f34940b0410 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:34.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.637 7f349a98a700 1 -- 172.21.2.101:0/490043348 >> 172.21.2.101:0/490043348 conn(0x7f349419f820 msgr2=0x7f34941a5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:34.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.637 7f349a98a700 1 -- 172.21.2.101:0/490043348 shutdown_connections 2021-02-10T11:42:34.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:34.637 7f349a98a700 1 -- 172.21.2.101:0/490043348 wait complete. 2021-02-10T11:42:34.647 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:42:35.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.068 7fc4f49f9700 1 Processor -- start 2021-02-10T11:42:35.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.068 7fc4f49f9700 1 -- start start 2021-02-10T11:42:35.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.068 7fc4f49f9700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc4f00d1860 0x7fc4f01b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:35.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.068 7fc4f49f9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc4f01b0ad0 0x7fc4f0108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:35.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.068 7fc4f49f9700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4f00fe1b0 con 0x7fc4f00d1c60 2021-02-10T11:42:35.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.068 7fc4f49f9700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f0109a50 con 0x7fc4f00d1860 2021-02-10T11:42:35.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.068 7fc4f49f9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f0109740 con 0x7fc4f01b0ad0 2021-02-10T11:42:35.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.069 7fc4ee61c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc4f00d1860 0x7fc4f01b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:35.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.069 7fc4ee61c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc4f00d1860 0x7fc4f01b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58236/0 (socket says 172.21.2.101:58236) 2021-02-10T11:42:35.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.069 7fc4ee61c700 1 -- 172.21.2.101:0/3532013552 learned_addr learned my addr 172.21.2.101:0/3532013552 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:35.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.069 7fc4eee1d700 1 --1- 172.21.2.101:0/3532013552 >> v1:172.21.2.101:6789/0 conn(0x7fc4f00d1c60 0x7fc4f0108f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58734/0 (socket says 172.21.2.101:58734) 2021-02-10T11:42:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.069 7fc4ee61c700 1 -- 172.21.2.101:0/3532013552 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc4f01b0ad0 msgr2=0x7fc4f0108930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:42:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.069 7fc4ee61c700 1 --2- 172.21.2.101:0/3532013552 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc4f01b0ad0 0x7fc4f0108930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.069 7fc4ee61c700 1 -- 172.21.2.101:0/3532013552 >> v1:172.21.2.101:6789/0 conn(0x7fc4f00d1c60 legacy=0x7fc4f0108f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.069 7fc4ee61c700 1 -- 172.21.2.101:0/3532013552 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f0088070 con 0x7fc4f00d1860 2021-02-10T11:42:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.069 7fc4ee61c700 1 --2- 172.21.2.101:0/3532013552 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc4f00d1860 0x7fc4f01b04b0 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 rx=0x7fc4d8006700 tx=0x7fc4d8009b00).ready entity=mon.0 client_cookie=b601fda721628f28 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.069 7fc4ed61a700 1 -- 172.21.2.101:0/3532013552 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc4d800c0c0 con 0x7fc4f00d1860 2021-02-10T11:42:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.069 7fc4ed61a700 1 -- 172.21.2.101:0/3532013552 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d800c340 con 0x7fc4f00d1860 2021-02-10T11:42:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.069 7fc4ed61a700 1 -- 172.21.2.101:0/3532013552 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc4d800c820 con 0x7fc4f00d1860 2021-02-10T11:42:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.069 7fc4f49f9700 1 -- 172.21.2.101:0/3532013552 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc4f00d1860 msgr2=0x7fc4f01b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.069 7fc4f49f9700 1 --2- 172.21.2.101:0/3532013552 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc4f00d1860 0x7fc4f01b04b0 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 rx=0x7fc4d8006700 tx=0x7fc4d8009b00).stop 2021-02-10T11:42:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.070 7fc4f49f9700 1 -- 172.21.2.101:0/3532013552 shutdown_connections 2021-02-10T11:42:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.070 7fc4f49f9700 1 --2- 172.21.2.101:0/3532013552 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc4f01b0ad0 0x7fc4f0108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.070 7fc4f49f9700 1 --2- 172.21.2.101:0/3532013552 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc4f00d1860 0x7fc4f01b04b0 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.070 7fc4f49f9700 1 -- 172.21.2.101:0/3532013552 >> 172.21.2.101:0/3532013552 conn(0x7fc4f00ce410 msgr2=0x7fc4f00d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:35.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.070 7fc4f49f9700 1 -- 172.21.2.101:0/3532013552 shutdown_connections 2021-02-10T11:42:35.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.070 7fc4f49f9700 1 -- 172.21.2.101:0/3532013552 wait complete. 2021-02-10T11:42:35.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.084 7fc4f49f9700 1 Processor -- start 2021-02-10T11:42:35.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.084 7fc4f49f9700 1 -- start start 2021-02-10T11:42:35.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.085 7fc4f49f9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc4f0093cc0 0x7fc4f019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:35.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.085 7fc4f49f9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc4f019e5b0 0x7fc4f00aa420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:35.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.085 7fc4f49f9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc4f00aaa40 0x7fc4f0095660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:35.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.085 7fc4f49f9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f019dd20 con 0x7fc4f0093cc0 2021-02-10T11:42:35.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.085 7fc4f49f9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f00940c0 con 0x7fc4f00aaa40 2021-02-10T11:42:35.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.085 7fc4f49f9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f00943f0 con 0x7fc4f019e5b0 2021-02-10T11:42:35.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.085 7fc4ee61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc4f0093cc0 0x7fc4f019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:35.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.085 7fc4ee61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc4f0093cc0 0x7fc4f019d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58246/0 (socket says 172.21.2.101:58246) 2021-02-10T11:42:35.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.085 7fc4ee61c700 1 -- 172.21.2.101:0/2958608067 learned_addr learned my addr 172.21.2.101:0/2958608067 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:35.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.085 7fc4ee61c700 1 -- 172.21.2.101:0/2958608067 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc4f019e5b0 msgr2=0x7fc4f00aa420 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:42:35.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.085 7fc4ee61c700 1 --2- 172.21.2.101:0/2958608067 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc4f019e5b0 0x7fc4f00aa420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:35.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.085 7fc4ee61c700 1 -- 172.21.2.101:0/2958608067 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc4f00aaa40 msgr2=0x7fc4f0095660 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:42:35.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.085 7fc4ee61c700 1 --2- 172.21.2.101:0/2958608067 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc4f00aaa40 0x7fc4f0095660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:35.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.085 7fc4ee61c700 1 -- 172.21.2.101:0/2958608067 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f01b9520 con 0x7fc4f0093cc0 2021-02-10T11:42:35.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.085 7fc4ee61c700 1 --2- 172.21.2.101:0/2958608067 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc4f0093cc0 0x7fc4f019d700 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 rx=0x7fc4d8006f20 tx=0x7fc4d800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:35.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.085 7fc4e6ffd700 1 -- 172.21.2.101:0/2958608067 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc4d8000c90 con 0x7fc4f0093cc0 2021-02-10T11:42:35.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.085 7fc4e6ffd700 1 -- 172.21.2.101:0/2958608067 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d8009100 con 0x7fc4f0093cc0 2021-02-10T11:42:35.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.086 7fc4f49f9700 1 -- 172.21.2.101:0/2958608067 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f01b9520 con 0x7fc4f0093cc0 2021-02-10T11:42:35.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.086 7fc4e6ffd700 1 -- 172.21.2.101:0/2958608067 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc4d8002220 con 0x7fc4f0093cc0 2021-02-10T11:42:35.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.086 7fc4f49f9700 1 -- 172.21.2.101:0/2958608067 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f01b9520 con 0x7fc4f0093cc0 2021-02-10T11:42:35.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.086 7fc4e6ffd700 1 -- 172.21.2.101:0/2958608067 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc4d8007000 con 0x7fc4f0093cc0 2021-02-10T11:42:35.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.086 7fc4e6ffd700 1 --2- 172.21.2.101:0/2958608067 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4d4018730 0x7fc4d401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:35.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.086 7fc4e6ffd700 1 -- 172.21.2.101:0/2958608067 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc4d80327f0 con 0x7fc4f0093cc0 2021-02-10T11:42:35.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.086 7fc4ede1b700 1 --2- 172.21.2.101:0/2958608067 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4d4018730 0x7fc4d401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:35.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.087 7fc4ede1b700 1 --2- 172.21.2.101:0/2958608067 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4d4018730 0x7fc4d401ac10 crc :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:35.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.118 7fc4f49f9700 1 -- 172.21.2.101:0/2958608067 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f019eca0 con 0x7fc4f0093cc0 2021-02-10T11:42:35.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.121 7fc4e6ffd700 1 -- 172.21.2.101:0/2958608067 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc4d8007000 con 0x7fc4f0093cc0 2021-02-10T11:42:35.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.249 7fc4f49f9700 1 -- 172.21.2.101:0/2958608067 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fc4f019eca0 con 0x7fc4f0093cc0 2021-02-10T11:42:35.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.253 7fc4e6ffd700 1 -- 172.21.2.101:0/2958608067 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v76) v1 ==== 184+0+0 (secure 0 0 0) 0x7fc4d8007f00 con 0x7fc4f0093cc0 2021-02-10T11:42:35.258 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.1 2021-02-10T11:42:35.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.258 7fc4f49f9700 1 -- 172.21.2.101:0/2958608067 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4d4018730 msgr2=0x7fc4d401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:35.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.258 7fc4f49f9700 1 --2- 172.21.2.101:0/2958608067 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4d4018730 0x7fc4d401ac10 crc :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:35.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.258 7fc4f49f9700 1 -- 172.21.2.101:0/2958608067 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc4f0093cc0 msgr2=0x7fc4f019d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:35.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.258 7fc4f49f9700 1 --2- 172.21.2.101:0/2958608067 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc4f0093cc0 0x7fc4f019d700 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 rx=0x7fc4d8006f20 tx=0x7fc4d800b990).stop 2021-02-10T11:42:35.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.258 7fc4f49f9700 1 -- 172.21.2.101:0/2958608067 shutdown_connections 2021-02-10T11:42:35.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.258 7fc4f49f9700 1 --2- 172.21.2.101:0/2958608067 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4d4018730 0x7fc4d401ac10 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:35.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.258 7fc4f49f9700 1 --2- 172.21.2.101:0/2958608067 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc4f019e5b0 0x7fc4f00aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:35.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.258 7fc4f49f9700 1 --2- 172.21.2.101:0/2958608067 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc4f00aaa40 0x7fc4f0095660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:35.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.258 7fc4f49f9700 1 --2- 172.21.2.101:0/2958608067 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc4f0093cc0 0x7fc4f019d700 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:35.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.258 7fc4f49f9700 1 -- 172.21.2.101:0/2958608067 >> 172.21.2.101:0/2958608067 conn(0x7fc4f019f820 msgr2=0x7fc4f018c270 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:35.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.258 7fc4f49f9700 1 -- 172.21.2.101:0/2958608067 shutdown_connections 2021-02-10T11:42:35.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.258 7fc4f49f9700 1 -- 172.21.2.101:0/2958608067 wait complete. 2021-02-10T11:42:35.268 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:42:35.438 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:42:35.442 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:42:35.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.691 7f6f6fb6d700 1 Processor -- start 2021-02-10T11:42:35.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.691 7f6f6fb6d700 1 -- start start 2021-02-10T11:42:35.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.692 7f6f6fb6d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6f680d1880 0x7f6f681b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:35.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.692 7f6f6fb6d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6f680cff80 0x7f6f68109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:35.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.692 7f6f6fb6d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f68109a70 con 0x7f6f681b0af0 2021-02-10T11:42:35.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.692 7f6f6fb6d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f68109760 con 0x7f6f680cff80 2021-02-10T11:42:35.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.692 7f6f6fb6d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f68088070 con 0x7f6f680d1880 2021-02-10T11:42:35.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.692 7f6f6e18a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6f680cff80 0x7f6f68109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:35.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.692 7f6f6e18a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6f680cff80 0x7f6f68109140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58250/0 (socket says 172.21.2.101:58250) 2021-02-10T11:42:35.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.692 7f6f6e18a700 1 -- 172.21.2.101:0/2280711270 learned_addr learned my addr 172.21.2.101:0/2280711270 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:35.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.692 7f6f6e18a700 1 -- 172.21.2.101:0/2280711270 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6f680d1880 msgr2=0x7f6f681b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:35.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.692 7f6f6e18a700 1 --2- 172.21.2.101:0/2280711270 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6f680d1880 0x7f6f681b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:35.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.692 7f6f6e18a700 1 -- 172.21.2.101:0/2280711270 >> v1:172.21.2.108:6789/0 conn(0x7f6f681b0af0 legacy=0x7f6f68108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:35.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.692 7f6f6e18a700 1 -- 172.21.2.101:0/2280711270 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f68088260 con 0x7f6f680cff80 2021-02-10T11:42:35.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.692 7f6f6e18a700 1 --2- 172.21.2.101:0/2280711270 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6f680cff80 0x7f6f68109140 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 rx=0x7f6f60006f00 tx=0x7f6f60009f10).ready entity=mon.0 client_cookie=4aeb09cd6df65159 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:35.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.693 7f6f6c987700 1 -- 172.21.2.101:0/2280711270 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6f6000b3b0 con 0x7f6f680cff80 2021-02-10T11:42:35.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.693 7f6f6c987700 1 -- 172.21.2.101:0/2280711270 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6000b730 con 0x7f6f680cff80 2021-02-10T11:42:35.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.693 7f6f6c987700 1 -- 172.21.2.101:0/2280711270 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6f6000bc50 con 0x7f6f680cff80 2021-02-10T11:42:35.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.693 7f6f6fb6d700 1 -- 172.21.2.101:0/2280711270 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6f680cff80 msgr2=0x7f6f68109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:35.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.693 7f6f6fb6d700 1 --2- 172.21.2.101:0/2280711270 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6f680cff80 0x7f6f68109140 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 rx=0x7f6f60006f00 tx=0x7f6f60009f10).stop 2021-02-10T11:42:35.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.694 7f6f6fb6d700 1 -- 172.21.2.101:0/2280711270 shutdown_connections 2021-02-10T11:42:35.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.694 7f6f6fb6d700 1 --2- 172.21.2.101:0/2280711270 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6f680cff80 0x7f6f68109140 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:35.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.694 7f6f6fb6d700 1 --2- 172.21.2.101:0/2280711270 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6f680d1880 0x7f6f681b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:35.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.694 7f6f6fb6d700 1 -- 172.21.2.101:0/2280711270 >> 172.21.2.101:0/2280711270 conn(0x7f6f680ce430 msgr2=0x7f6f680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:35.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.694 7f6f6fb6d700 1 -- 172.21.2.101:0/2280711270 shutdown_connections 2021-02-10T11:42:35.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.694 7f6f6fb6d700 1 -- 172.21.2.101:0/2280711270 wait complete. 2021-02-10T11:42:35.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.708 7f6f6fb6d700 1 Processor -- start 2021-02-10T11:42:35.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.708 7f6f6fb6d700 1 -- start start 2021-02-10T11:42:35.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.709 7f6f6fb6d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6f681a68f0 0x7f6f6819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:35.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.709 7f6f6fb6d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6f681a5e00 0x7f6f681a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:35.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.709 7f6f6fb6d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6f681a08e0 0x7f6f681aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:35.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.709 7f6f6fb6d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f681a6370 con 0x7f6f681a5e00 2021-02-10T11:42:35.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.709 7f6f6fb6d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f681a3960 con 0x7f6f681a08e0 2021-02-10T11:42:35.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.709 7f6f6fb6d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f681ab440 con 0x7f6f681a68f0 2021-02-10T11:42:35.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.709 7f6f6d188700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6f681a5e00 0x7f6f681a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:35.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.709 7f6f6d188700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6f681a5e00 0x7f6f681a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58258/0 (socket says 172.21.2.101:58258) 2021-02-10T11:42:35.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.709 7f6f6d188700 1 -- 172.21.2.101:0/1434763015 learned_addr learned my addr 172.21.2.101:0/1434763015 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:35.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.709 7f6f6d188700 1 -- 172.21.2.101:0/1434763015 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6f681a68f0 msgr2=0x7f6f6819fca0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:42:35.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.709 7f6f6d188700 1 --2- 172.21.2.101:0/1434763015 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6f681a68f0 0x7f6f6819fca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:35.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.709 7f6f6d188700 1 -- 172.21.2.101:0/1434763015 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6f681a08e0 msgr2=0x7f6f681aae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:35.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.709 7f6f6d188700 1 --2- 172.21.2.101:0/1434763015 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6f681a08e0 0x7f6f681aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:35.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.709 7f6f6d188700 1 -- 172.21.2.101:0/1434763015 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f681ab660 con 0x7f6f681a5e00 2021-02-10T11:42:35.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.710 7f6f6d188700 1 --2- 172.21.2.101:0/1434763015 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6f681a5e00 0x7f6f681a02c0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 rx=0x7f6f5c00b5b0 tx=0x7f6f5c00b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:35.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.710 7f6f5a7fc700 1 -- 172.21.2.101:0/1434763015 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6f5c00bd40 con 0x7f6f681a5e00 2021-02-10T11:42:35.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.710 7f6f5a7fc700 1 -- 172.21.2.101:0/1434763015 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5c000e20 con 0x7f6f681a5e00 2021-02-10T11:42:35.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.710 7f6f6fb6d700 1 -- 172.21.2.101:0/1434763015 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f681ab660 con 0x7f6f681a5e00 2021-02-10T11:42:35.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.710 7f6f5a7fc700 1 -- 172.21.2.101:0/1434763015 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6f5c0054a0 con 0x7f6f681a5e00 2021-02-10T11:42:35.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.710 7f6f6fb6d700 1 -- 172.21.2.101:0/1434763015 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f681ab660 con 0x7f6f681a5e00 2021-02-10T11:42:35.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.711 7f6f5a7fc700 1 -- 172.21.2.101:0/1434763015 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6f5c006030 con 0x7f6f681a5e00 2021-02-10T11:42:35.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.711 7f6f5a7fc700 1 --2- 172.21.2.101:0/1434763015 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6f50018730 0x7f6f5001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:35.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.711 7f6f5a7fc700 1 -- 172.21.2.101:0/1434763015 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6f5c006030 con 0x7f6f681a5e00 2021-02-10T11:42:35.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.711 7f6f6d989700 1 --2- 172.21.2.101:0/1434763015 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6f50018730 0x7f6f5001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:35.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.711 7f6f6d989700 1 --2- 172.21.2.101:0/1434763015 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6f50018730 0x7f6f5001ac10 crc :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:35.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.741 7f6f6fb6d700 1 -- 172.21.2.101:0/1434763015 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f680829a0 con 0x7f6f681a5e00 2021-02-10T11:42:35.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.744 7f6f5a7fc700 1 -- 172.21.2.101:0/1434763015 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6f5c000ce0 con 0x7f6f681a5e00 2021-02-10T11:42:35.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.878 7f6f6fb6d700 1 -- 172.21.2.101:0/1434763015 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f6f680829a0 con 0x7f6f681a5e00 2021-02-10T11:42:35.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.881 7f6f5a7fc700 1 -- 172.21.2.101:0/1434763015 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v77) v1 ==== 184+0+0 (secure 0 0 0) 0x7f6f5c006030 con 0x7f6f681a5e00 2021-02-10T11:42:35.886 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.2 2021-02-10T11:42:35.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.886 7f6f6fb6d700 1 -- 172.21.2.101:0/1434763015 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6f50018730 msgr2=0x7f6f5001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:35.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.886 7f6f6fb6d700 1 --2- 172.21.2.101:0/1434763015 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6f50018730 0x7f6f5001ac10 crc :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:35.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.886 7f6f6fb6d700 1 -- 172.21.2.101:0/1434763015 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6f681a5e00 msgr2=0x7f6f681a02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:35.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.886 7f6f6fb6d700 1 --2- 172.21.2.101:0/1434763015 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6f681a5e00 0x7f6f681a02c0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 rx=0x7f6f5c00b5b0 tx=0x7f6f5c00b950).stop 2021-02-10T11:42:35.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.886 7f6f6fb6d700 1 -- 172.21.2.101:0/1434763015 shutdown_connections 2021-02-10T11:42:35.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.886 7f6f6fb6d700 1 --2- 172.21.2.101:0/1434763015 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6f681a08e0 0x7f6f681aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:35.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.886 7f6f6fb6d700 1 --2- 172.21.2.101:0/1434763015 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6f681a5e00 0x7f6f681a02c0 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:35.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.886 7f6f6fb6d700 1 --2- 172.21.2.101:0/1434763015 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6f50018730 0x7f6f5001ac10 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:35.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.886 7f6f6fb6d700 1 --2- 172.21.2.101:0/1434763015 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6f681a68f0 0x7f6f6819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:35.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.886 7f6f6fb6d700 1 -- 172.21.2.101:0/1434763015 >> 172.21.2.101:0/1434763015 conn(0x7f6f6819f880 msgr2=0x7f6f681a71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:35.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.887 7f6f6fb6d700 1 -- 172.21.2.101:0/1434763015 shutdown_connections 2021-02-10T11:42:35.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:35.887 7f6f6fb6d700 1 -- 172.21.2.101:0/1434763015 wait complete. 2021-02-10T11:42:35.897 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:42:36.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.311 7f05e62f0700 1 Processor -- start 2021-02-10T11:42:36.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.311 7f05e62f0700 1 -- start start 2021-02-10T11:42:36.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.311 7f05e62f0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05e00d1c80 0x7f05e00cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:36.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.311 7f05e62f0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f05e00cf590 0x7f05e00cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:36.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.311 7f05e62f0700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e00fe1d0 con 0x7f05e00d1880 2021-02-10T11:42:36.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.311 7f05e62f0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e01b0cc0 con 0x7f05e00cf590 2021-02-10T11:42:36.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.311 7f05e62f0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e0109a70 con 0x7f05e00d1c80 2021-02-10T11:42:36.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.311 7f05dffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f05e00d1880 0x7f05e01b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58756/0 (socket says 172.21.2.101:58756) 2021-02-10T11:42:36.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.311 7f05e490d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f05e00cf590 0x7f05e00cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:36.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.311 7f05dffff700 1 -- 172.21.2.101:0/1602077409 learned_addr learned my addr 172.21.2.101:0/1602077409 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:36.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.311 7f05df7fe700 1 --2- 172.21.2.101:0/1602077409 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05e00d1c80 0x7f05e00cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:36.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.312 7f05e490d700 1 -- 172.21.2.101:0/1602077409 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05e00d1c80 msgr2=0x7f05e00cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:36.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.312 7f05e490d700 1 --2- 172.21.2.101:0/1602077409 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05e00d1c80 0x7f05e00cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:36.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.312 7f05e490d700 1 -- 172.21.2.101:0/1602077409 >> v1:172.21.2.101:6789/0 conn(0x7f05e00d1880 legacy=0x7f05e01b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:36.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.312 7f05deffd700 1 -- 172.21.2.101:0/1602077409 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1539864889 0 0) 0x7f05d0000ce0 con 0x7f05e00d1880 2021-02-10T11:42:36.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.312 7f05e490d700 1 -- 172.21.2.101:0/1602077409 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e0088070 con 0x7f05e00cf590 2021-02-10T11:42:36.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.312 7f05df7fe700 1 --2- 172.21.2.101:0/1602077409 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05e00d1c80 0x7f05e00cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:42:36.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.312 7f05e490d700 1 --2- 172.21.2.101:0/1602077409 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05e00cf590 0x7f05e00cf990 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 rx=0x7f05d8006e30 tx=0x7f05d8009e60).ready entity=mon.0 client_cookie=2b53a1d4d1fd34aa server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:36.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.312 7f05deffd700 1 -- 172.21.2.101:0/1602077409 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f05d8006a50 con 0x7f05e00cf590 2021-02-10T11:42:36.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.312 7f05deffd700 1 -- 172.21.2.101:0/1602077409 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d800b300 con 0x7f05e00cf590 2021-02-10T11:42:36.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.312 7f05deffd700 1 -- 172.21.2.101:0/1602077409 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f05d800b760 con 0x7f05e00cf590 2021-02-10T11:42:36.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.313 7f05e62f0700 1 -- 172.21.2.101:0/1602077409 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05e00cf590 msgr2=0x7f05e00cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:36.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.313 7f05e62f0700 1 --2- 172.21.2.101:0/1602077409 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05e00cf590 0x7f05e00cf990 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 rx=0x7f05d8006e30 tx=0x7f05d8009e60).stop 2021-02-10T11:42:36.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.313 7f05e62f0700 1 -- 172.21.2.101:0/1602077409 shutdown_connections 2021-02-10T11:42:36.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.313 7f05e62f0700 1 --2- 172.21.2.101:0/1602077409 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05e00d1c80 0x7f05e00cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:36.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.313 7f05e62f0700 1 --2- 172.21.2.101:0/1602077409 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05e00cf590 0x7f05e00cf990 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:36.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.313 7f05e62f0700 1 -- 172.21.2.101:0/1602077409 >> 172.21.2.101:0/1602077409 conn(0x7f05e00ce430 msgr2=0x7f05e00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:36.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.313 7f05e62f0700 1 -- 172.21.2.101:0/1602077409 shutdown_connections 2021-02-10T11:42:36.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.313 7f05e62f0700 1 -- 172.21.2.101:0/1602077409 wait complete. 2021-02-10T11:42:36.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.328 7f05e62f0700 1 Processor -- start 2021-02-10T11:42:36.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.328 7f05e62f0700 1 -- start start 2021-02-10T11:42:36.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.328 7f05e62f0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f05e01a34e0 0x7f05e01a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:36.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.328 7f05e62f0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05e0083220 0x7f05e0083620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:36.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.328 7f05e62f0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05e00acda0 0x7f05e00ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:36.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.328 7f05e62f0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e00ad7c0 con 0x7f05e0083220 2021-02-10T11:42:36.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.328 7f05e62f0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e00ad9e0 con 0x7f05e01a34e0 2021-02-10T11:42:36.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.328 7f05e62f0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e00adc00 con 0x7f05e00acda0 2021-02-10T11:42:36.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.329 7f05df7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05e0083220 0x7f05e0083620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:36.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.329 7f05df7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05e0083220 0x7f05e0083620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58270/0 (socket says 172.21.2.101:58270) 2021-02-10T11:42:36.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.329 7f05df7fe700 1 -- 172.21.2.101:0/2690315562 learned_addr learned my addr 172.21.2.101:0/2690315562 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:36.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.329 7f05dffff700 1 --2- 172.21.2.101:0/2690315562 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f05e01a34e0 0x7f05e01a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:36.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.329 7f05e490d700 1 --2- 172.21.2.101:0/2690315562 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05e00acda0 0x7f05e00ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:36.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.329 7f05dffff700 1 -- 172.21.2.101:0/2690315562 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05e00acda0 msgr2=0x7f05e00ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:36.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.329 7f05dffff700 1 --2- 172.21.2.101:0/2690315562 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05e00acda0 0x7f05e00ad1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:36.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.329 7f05dffff700 1 -- 172.21.2.101:0/2690315562 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05e0083220 msgr2=0x7f05e0083620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:36.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.329 7f05dffff700 1 --2- 172.21.2.101:0/2690315562 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05e0083220 0x7f05e0083620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:36.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.329 7f05dffff700 1 -- 172.21.2.101:0/2690315562 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e00ade20 con 0x7f05e01a34e0 2021-02-10T11:42:36.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.329 7f05df7fe700 1 --2- 172.21.2.101:0/2690315562 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05e0083220 0x7f05e0083620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:42:36.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.329 7f05dffff700 1 --2- 172.21.2.101:0/2690315562 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f05e01a34e0 0x7f05e01a38e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f05d000a4d0 tx=0x7f05d000a870).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:36.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.330 7f05dcff9700 1 -- 172.21.2.101:0/2690315562 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f05d0000d10 con 0x7f05e01a34e0 2021-02-10T11:42:36.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.330 7f05dcff9700 1 -- 172.21.2.101:0/2690315562 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d00043a0 con 0x7f05e01a34e0 2021-02-10T11:42:36.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.330 7f05dcff9700 1 -- 172.21.2.101:0/2690315562 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f05d0004800 con 0x7f05e01a34e0 2021-02-10T11:42:36.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.330 7f05e62f0700 1 -- 172.21.2.101:0/2690315562 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e00ade20 con 0x7f05e01a34e0 2021-02-10T11:42:36.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.330 7f05e62f0700 1 -- 172.21.2.101:0/2690315562 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e00ade20 con 0x7f05e01a34e0 2021-02-10T11:42:36.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.331 7f05dcff9700 1 -- 172.21.2.101:0/2690315562 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f05d00043a0 con 0x7f05e01a34e0 2021-02-10T11:42:36.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.331 7f05dcff9700 1 --2- 172.21.2.101:0/2690315562 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f05c00186c0 0x7f05c001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:36.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.331 7f05dcff9700 1 -- 172.21.2.101:0/2690315562 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f05d0032310 con 0x7f05e01a34e0 2021-02-10T11:42:36.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.331 7f05df7fe700 1 --2- 172.21.2.101:0/2690315562 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f05c00186c0 0x7f05c001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:36.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.331 7f05df7fe700 1 --2- 172.21.2.101:0/2690315562 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f05c00186c0 0x7f05c001abc0 crc :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:36.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.361 7f05e62f0700 1 -- 172.21.2.101:0/2690315562 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e00fe1d0 con 0x7f05e01a34e0 2021-02-10T11:42:36.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.364 7f05dcff9700 1 -- 172.21.2.101:0/2690315562 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f05d00043a0 con 0x7f05e01a34e0 2021-02-10T11:42:36.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.494 7f05e62f0700 1 -- 172.21.2.101:0/2690315562 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f05e00fe1d0 con 0x7f05e01a34e0 2021-02-10T11:42:36.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.500 7f05dcff9700 1 -- 172.21.2.101:0/2690315562 <== mon.1 v2:172.21.2.108:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v78) v1 ==== 184+0+0 (secure 0 0 0) 0x7f05d0000c90 con 0x7f05e01a34e0 2021-02-10T11:42:36.503 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.3 2021-02-10T11:42:36.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.503 7f05e62f0700 1 -- 172.21.2.101:0/2690315562 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f05c00186c0 msgr2=0x7f05c001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:36.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.503 7f05e62f0700 1 --2- 172.21.2.101:0/2690315562 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f05c00186c0 0x7f05c001abc0 crc :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:36.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.503 7f05e62f0700 1 -- 172.21.2.101:0/2690315562 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f05e01a34e0 msgr2=0x7f05e01a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:36.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.503 7f05e62f0700 1 --2- 172.21.2.101:0/2690315562 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f05e01a34e0 0x7f05e01a38e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f05d000a4d0 tx=0x7f05d000a870).stop 2021-02-10T11:42:36.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.503 7f05e62f0700 1 -- 172.21.2.101:0/2690315562 shutdown_connections 2021-02-10T11:42:36.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.503 7f05e62f0700 1 --2- 172.21.2.101:0/2690315562 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f05c00186c0 0x7f05c001abc0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:36.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.503 7f05e62f0700 1 --2- 172.21.2.101:0/2690315562 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f05e00acda0 0x7f05e00ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:36.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.503 7f05e62f0700 1 --2- 172.21.2.101:0/2690315562 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f05e0083220 0x7f05e0083620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:36.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.503 7f05e62f0700 1 --2- 172.21.2.101:0/2690315562 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f05e01a34e0 0x7f05e01a38e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:36.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.503 7f05e62f0700 1 -- 172.21.2.101:0/2690315562 >> 172.21.2.101:0/2690315562 conn(0x7f05e00a1070 msgr2=0x7f05e008eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:36.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.503 7f05e62f0700 1 -- 172.21.2.101:0/2690315562 shutdown_connections 2021-02-10T11:42:36.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.504 7f05e62f0700 1 -- 172.21.2.101:0/2690315562 wait complete. 2021-02-10T11:42:36.514 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:42:36.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.924 7fd4d1926700 1 Processor -- start 2021-02-10T11:42:36.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.924 7fd4d1926700 1 -- start start 2021-02-10T11:42:36.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.925 7fd4d1926700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4cc0d0340 0x7fd4cc0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:36.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.925 7fd4d1926700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd4cc0d1c70 0x7fd4cc0efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:36.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.925 7fd4d1926700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4cc0f0b10 con 0x7fd4cc0d1870 2021-02-10T11:42:36.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.925 7fd4d1926700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4cc0e5ed0 con 0x7fd4cc0d1c70 2021-02-10T11:42:36.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.925 7fd4d1926700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc1b0be0 con 0x7fd4cc0d0340 2021-02-10T11:42:36.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.925 7fd4cbd75700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd4cc0d1c70 0x7fd4cc0efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:36.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.925 7fd4cbd75700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd4cc0d1c70 0x7fd4cc0efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58276/0 (socket says 172.21.2.101:58276) 2021-02-10T11:42:36.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.925 7fd4cbd75700 1 -- 172.21.2.101:0/3192102608 learned_addr learned my addr 172.21.2.101:0/3192102608 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:36.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.925 7fd4cb574700 1 --1- 172.21.2.101:0/3192102608 >> v1:172.21.2.101:6789/0 conn(0x7fd4cc0d1870 0x7fd4cc1b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58770/0 (socket says 172.21.2.101:58770) 2021-02-10T11:42:36.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.925 7fd4cad73700 1 --2- 172.21.2.101:0/3192102608 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4cc0d0340 0x7fd4cc0ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:36.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.925 7fd4cbd75700 1 -- 172.21.2.101:0/3192102608 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4cc0d0340 msgr2=0x7fd4cc0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:36.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.925 7fd4cbd75700 1 --2- 172.21.2.101:0/3192102608 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4cc0d0340 0x7fd4cc0ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:36.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.925 7fd4cbd75700 1 -- 172.21.2.101:0/3192102608 >> v1:172.21.2.101:6789/0 conn(0x7fd4cc0d1870 legacy=0x7fd4cc1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:36.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.925 7fd4cbd75700 1 -- 172.21.2.101:0/3192102608 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc0f20a0 con 0x7fd4cc0d1c70 2021-02-10T11:42:36.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.925 7fd4cad73700 1 --2- 172.21.2.101:0/3192102608 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4cc0d0340 0x7fd4cc0ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:42:36.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.925 7fd4cbd75700 1 --2- 172.21.2.101:0/3192102608 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4cc0d1c70 0x7fd4cc0efc60 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 rx=0x7fd4bc006b80 tx=0x7fd4bc009b90).ready entity=mon.0 client_cookie=d41e45cbce7afb0a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:36.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.926 7fd4ca572700 1 -- 172.21.2.101:0/3192102608 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd4bc0022e0 con 0x7fd4cc0d1c70 2021-02-10T11:42:36.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.926 7fd4ca572700 1 -- 172.21.2.101:0/3192102608 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4bc002620 con 0x7fd4cc0d1c70 2021-02-10T11:42:36.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.926 7fd4ca572700 1 -- 172.21.2.101:0/3192102608 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd4bc002b00 con 0x7fd4cc0d1c70 2021-02-10T11:42:36.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.926 7fd4d1926700 1 -- 172.21.2.101:0/3192102608 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4cc0d1c70 msgr2=0x7fd4cc0efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:36.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.926 7fd4d1926700 1 --2- 172.21.2.101:0/3192102608 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4cc0d1c70 0x7fd4cc0efc60 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 rx=0x7fd4bc006b80 tx=0x7fd4bc009b90).stop 2021-02-10T11:42:36.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.926 7fd4d1926700 1 -- 172.21.2.101:0/3192102608 shutdown_connections 2021-02-10T11:42:36.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.926 7fd4d1926700 1 --2- 172.21.2.101:0/3192102608 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4cc0d0340 0x7fd4cc0ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:36.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.926 7fd4d1926700 1 --2- 172.21.2.101:0/3192102608 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4cc0d1c70 0x7fd4cc0efc60 unknown :-1 s=CLOSED pgs=2442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:36.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.926 7fd4d1926700 1 -- 172.21.2.101:0/3192102608 >> 172.21.2.101:0/3192102608 conn(0x7fd4cc0ce450 msgr2=0x7fd4cc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:36.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.926 7fd4d1926700 1 -- 172.21.2.101:0/3192102608 shutdown_connections 2021-02-10T11:42:36.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.926 7fd4d1926700 1 -- 172.21.2.101:0/3192102608 wait complete. 2021-02-10T11:42:36.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.941 7fd4d1926700 1 Processor -- start 2021-02-10T11:42:36.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.941 7fd4d1926700 1 -- start start 2021-02-10T11:42:36.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.941 7fd4d1926700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4cc094040 0x7fd4cc093370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:36.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.941 7fd4d1926700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4cc093990 0x7fd4cc0ad700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:36.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.941 7fd4d1926700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4cc0add20 0x7fd4cc1a2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:36.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.941 7fd4d1926700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc093d90 con 0x7fd4cc094040 2021-02-10T11:42:36.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.941 7fd4d1926700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc1a1e50 con 0x7fd4cc093990 2021-02-10T11:42:36.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.941 7fd4d1926700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc094590 con 0x7fd4cc0add20 2021-02-10T11:42:36.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.942 7fd4cb574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4cc094040 0x7fd4cc093370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:36.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.942 7fd4cb574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4cc094040 0x7fd4cc093370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58282/0 (socket says 172.21.2.101:58282) 2021-02-10T11:42:36.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.942 7fd4cb574700 1 -- 172.21.2.101:0/137039351 learned_addr learned my addr 172.21.2.101:0/137039351 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:36.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.942 7fd4cb574700 1 -- 172.21.2.101:0/137039351 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4cc0add20 msgr2=0x7fd4cc1a2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:36.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.942 7fd4cb574700 1 --2- 172.21.2.101:0/137039351 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4cc0add20 0x7fd4cc1a2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:36.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.942 7fd4cb574700 1 -- 172.21.2.101:0/137039351 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4cc093990 msgr2=0x7fd4cc0ad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:36.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.942 7fd4cb574700 1 --2- 172.21.2.101:0/137039351 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4cc093990 0x7fd4cc0ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:36.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.942 7fd4cb574700 1 -- 172.21.2.101:0/137039351 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc0ae120 con 0x7fd4cc094040 2021-02-10T11:42:36.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.942 7fd4cb574700 1 --2- 172.21.2.101:0/137039351 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4cc094040 0x7fd4cc093370 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 rx=0x7fd4c0001c60 tx=0x7fd4c000a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:36.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.942 7fd4b3fff700 1 -- 172.21.2.101:0/137039351 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd4c000acc0 con 0x7fd4cc094040 2021-02-10T11:42:36.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.942 7fd4b3fff700 1 -- 172.21.2.101:0/137039351 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c0004140 con 0x7fd4cc094040 2021-02-10T11:42:36.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.942 7fd4b3fff700 1 -- 172.21.2.101:0/137039351 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd4c0004660 con 0x7fd4cc094040 2021-02-10T11:42:36.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.942 7fd4d1926700 1 -- 172.21.2.101:0/137039351 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4cc0ae120 con 0x7fd4cc094040 2021-02-10T11:42:36.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.942 7fd4d1926700 1 -- 172.21.2.101:0/137039351 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4cc1a2d90 con 0x7fd4cc094040 2021-02-10T11:42:36.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.943 7fd4b3fff700 1 -- 172.21.2.101:0/137039351 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd4c0004320 con 0x7fd4cc094040 2021-02-10T11:42:36.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.943 7fd4b3fff700 1 --2- 172.21.2.101:0/137039351 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4ac0186c0 0x7fd4ac01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:36.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.943 7fd4b3fff700 1 -- 172.21.2.101:0/137039351 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd4c0032540 con 0x7fd4cc094040 2021-02-10T11:42:36.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.943 7fd4cad73700 1 --2- 172.21.2.101:0/137039351 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4ac0186c0 0x7fd4ac01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:36.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.943 7fd4cad73700 1 --2- 172.21.2.101:0/137039351 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4ac0186c0 0x7fd4ac01abc0 crc :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:36.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.975 7fd4d1926700 1 -- 172.21.2.101:0/137039351 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4cc0ebc60 con 0x7fd4cc094040 2021-02-10T11:42:36.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:36.977 7fd4b3fff700 1 -- 172.21.2.101:0/137039351 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd4c0000dd0 con 0x7fd4cc094040 2021-02-10T11:42:37.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:37.105 7fd4d1926700 1 -- 172.21.2.101:0/137039351 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4cc0f0b10 con 0x7fd4cc094040 2021-02-10T11:42:37.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:37.105 7fd4b3fff700 1 -- 172.21.2.101:0/137039351 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+2105 (secure 0 0 0) 0x7fd4c000acc0 con 0x7fd4cc094040 2021-02-10T11:42:37.107 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 105 2021-02-10T11:42:37.108 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:42:37.108 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":105,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11139,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1647083555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1647083555},{"type":"v1","addr":"172.21.2.101:6835","nonce":1647083555}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":101}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:42:30.762357","modified":"2021-02-10 11:42:32.772467","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7816":{"gid":7816,"name":"b","rank":0,"incarnation":102,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/381011050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":381011050},{"type":"v1","addr":"172.21.2.108:6833","nonce":381011050}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-02-10T11:42:37.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:37.107 7fd4d1926700 1 -- 172.21.2.101:0/137039351 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4ac0186c0 msgr2=0x7fd4ac01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:37.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:37.107 7fd4d1926700 1 --2- 172.21.2.101:0/137039351 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4ac0186c0 0x7fd4ac01abc0 crc :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:37.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:37.107 7fd4d1926700 1 -- 172.21.2.101:0/137039351 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4cc094040 msgr2=0x7fd4cc093370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:37.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:37.107 7fd4d1926700 1 --2- 172.21.2.101:0/137039351 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4cc094040 0x7fd4cc093370 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 rx=0x7fd4c0001c60 tx=0x7fd4c000a8d0).stop 2021-02-10T11:42:37.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:37.107 7fd4d1926700 1 -- 172.21.2.101:0/137039351 shutdown_connections 2021-02-10T11:42:37.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:37.107 7fd4d1926700 1 --2- 172.21.2.101:0/137039351 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd4ac0186c0 0x7fd4ac01abc0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:37.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:37.107 7fd4d1926700 1 --2- 172.21.2.101:0/137039351 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4cc0add20 0x7fd4cc1a2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:37.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:37.107 7fd4d1926700 1 --2- 172.21.2.101:0/137039351 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd4cc093990 0x7fd4cc0ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:37.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:37.107 7fd4d1926700 1 --2- 172.21.2.101:0/137039351 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4cc094040 0x7fd4cc093370 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:37.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:37.107 7fd4d1926700 1 -- 172.21.2.101:0/137039351 >> 172.21.2.101:0/137039351 conn(0x7fd4cc19f8a0 msgr2=0x7fd4cc094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:37.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:37.107 7fd4d1926700 1 -- 172.21.2.101:0/137039351 shutdown_connections 2021-02-10T11:42:37.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:37.108 7fd4d1926700 1 -- 172.21.2.101:0/137039351 wait complete. 2021-02-10T11:42:37.319 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:42:30.762357', 'modified': '2021-02-10 11:42:32.772467', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7816}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7816': {'gid': 7816, 'name': 'b', 'rank': 0, 'incarnation': 102, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/381011050', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 381011050}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 381011050}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:42:37.320 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:42:37.320 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:42:37.718 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:42:37.718 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:42:37.719 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:42:37.719 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 7816, 2021-02-10T11:42:37.719 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:42:37.719 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:42:37.719 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 6.9519090989999999, 2021-02-10T11:42:37.719 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 105, 2021-02-10T11:42:37.719 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 129, 2021-02-10T11:42:37.719 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 129, 2021-02-10T11:42:37.719 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 12.612631365 2021-02-10T11:42:37.719 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:42:37.722 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 7816, "want_state": "up:active", "state": "up:active", "rank_uptime": 6.9519090989999999, "mdsmap_epoch": 105, "osdmap_epoch": 129, "osdmap_epoch_barrier": 129, "uptime": 12.612631365 } 2021-02-10T11:42:37.723 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:42:38.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.132 7f04c6d6b700 1 Processor -- start 2021-02-10T11:42:38.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.132 7f04c6d6b700 1 -- start start 2021-02-10T11:42:38.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.133 7f04c6d6b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f04c00d1870 0x7f04c01b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:38.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.133 7f04c6d6b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f04c00f0650 0x7f04c00cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:38.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.133 7f04c6d6b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c00f0b10 con 0x7f04c01b0a10 2021-02-10T11:42:38.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.133 7f04c6d6b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c00e5ed0 con 0x7f04c00d1870 2021-02-10T11:42:38.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.133 7f04c6d6b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c00d1c70 con 0x7f04c00f0650 2021-02-10T11:42:38.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.133 7f04c4b87700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f04c00d1870 0x7f04c01b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:38.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.133 7f04bffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f04c01b0a10 0x7f04c00cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58782/0 (socket says 172.21.2.101:58782) 2021-02-10T11:42:38.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.133 7f04bffff700 1 -- 172.21.2.101:0/204346911 learned_addr learned my addr 172.21.2.101:0/204346911 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:38.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.133 7f04c5388700 1 --2- 172.21.2.101:0/204346911 >> v2:172.21.2.108:3300/0 conn(0x7f04c00f0650 0x7f04c00cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:38.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.134 7f04c4b87700 1 -- 172.21.2.101:0/204346911 >> v2:172.21.2.108:3300/0 conn(0x7f04c00f0650 msgr2=0x7f04c00cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:38.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.134 7f04c4b87700 1 --2- 172.21.2.101:0/204346911 >> v2:172.21.2.108:3300/0 conn(0x7f04c00f0650 0x7f04c00cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:38.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.134 7f04c4b87700 1 -- 172.21.2.101:0/204346911 >> v1:172.21.2.101:6789/0 conn(0x7f04c01b0a10 legacy=0x7f04c00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:38.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.134 7f04c4b87700 1 -- 172.21.2.101:0/204346911 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c00d1e90 con 0x7f04c00d1870 2021-02-10T11:42:38.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.134 7f04c5388700 1 --2- 172.21.2.101:0/204346911 >> v2:172.21.2.108:3300/0 conn(0x7f04c00f0650 0x7f04c00cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:42:38.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.134 7f04c4b87700 1 --2- 172.21.2.101:0/204346911 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04c00d1870 0x7f04c01b03f0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 rx=0x7f04a8006650 tx=0x7f04a8009a80).ready entity=mon.0 client_cookie=418d029d1b1a6238 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:38.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.134 7f04bf7fe700 1 -- 172.21.2.101:0/204346911 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f04a80009b0 con 0x7f04c00d1870 2021-02-10T11:42:38.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.134 7f04bf7fe700 1 -- 172.21.2.101:0/204346911 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a800c0c0 con 0x7f04c00d1870 2021-02-10T11:42:38.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.134 7f04bf7fe700 1 -- 172.21.2.101:0/204346911 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f04a800c510 con 0x7f04c00d1870 2021-02-10T11:42:38.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.134 7f04c6d6b700 1 -- 172.21.2.101:0/204346911 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04c00d1870 msgr2=0x7f04c01b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:38.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.134 7f04c6d6b700 1 --2- 172.21.2.101:0/204346911 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04c00d1870 0x7f04c01b03f0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 rx=0x7f04a8006650 tx=0x7f04a8009a80).stop 2021-02-10T11:42:38.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.134 7f04c6d6b700 1 -- 172.21.2.101:0/204346911 shutdown_connections 2021-02-10T11:42:38.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.134 7f04c6d6b700 1 --2- 172.21.2.101:0/204346911 >> v2:172.21.2.108:3300/0 conn(0x7f04c00f0650 0x7f04c00cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:38.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.134 7f04c6d6b700 1 --2- 172.21.2.101:0/204346911 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04c00d1870 0x7f04c01b03f0 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:38.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.134 7f04c6d6b700 1 -- 172.21.2.101:0/204346911 >> 172.21.2.101:0/204346911 conn(0x7f04c00ce450 msgr2=0x7f04c00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:38.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.134 7f04c6d6b700 1 -- 172.21.2.101:0/204346911 shutdown_connections 2021-02-10T11:42:38.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.135 7f04c6d6b700 1 -- 172.21.2.101:0/204346911 wait complete. 2021-02-10T11:42:38.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.149 7f04c6d6b700 1 Processor -- start 2021-02-10T11:42:38.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.149 7f04c6d6b700 1 -- start start 2021-02-10T11:42:38.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.150 7f04c6d6b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04c01b7cf0 0x7f04c019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:38.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.150 7f04c6d6b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f04c01b80f0 0x7f04c01b84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:38.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.150 7f04c6d6b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f04c019da20 0x7f04c019de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:38.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.150 7f04c6d6b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c00a77c0 con 0x7f04c01b7cf0 2021-02-10T11:42:38.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.150 7f04c6d6b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c019d4a0 con 0x7f04c01b80f0 2021-02-10T11:42:38.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.150 7f04c6d6b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c00954f0 con 0x7f04c019da20 2021-02-10T11:42:38.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.150 7f04c4b87700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04c01b7cf0 0x7f04c019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:38.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.150 7f04c4b87700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04c01b7cf0 0x7f04c019cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58296/0 (socket says 172.21.2.101:58296) 2021-02-10T11:42:38.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.150 7f04c4b87700 1 -- 172.21.2.101:0/1509349427 learned_addr learned my addr 172.21.2.101:0/1509349427 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:38.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.150 7f04c4b87700 1 -- 172.21.2.101:0/1509349427 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f04c019da20 msgr2=0x7f04c019de20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:42:38.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.150 7f04c4b87700 1 --2- 172.21.2.101:0/1509349427 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f04c019da20 0x7f04c019de20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:38.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.150 7f04c4b87700 1 -- 172.21.2.101:0/1509349427 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f04c01b80f0 msgr2=0x7f04c01b84f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:42:38.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.150 7f04c4b87700 1 --2- 172.21.2.101:0/1509349427 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f04c01b80f0 0x7f04c01b84f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:38.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.150 7f04c4b87700 1 -- 172.21.2.101:0/1509349427 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c01b8d60 con 0x7f04c01b7cf0 2021-02-10T11:42:38.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.150 7f04c4b87700 1 --2- 172.21.2.101:0/1509349427 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04c01b7cf0 0x7f04c019cdb0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 rx=0x7f04a8009650 tx=0x7f04a800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:38.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.151 7f04bd7fa700 1 -- 172.21.2.101:0/1509349427 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f04a8009300 con 0x7f04c01b7cf0 2021-02-10T11:42:38.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.151 7f04bd7fa700 1 -- 172.21.2.101:0/1509349427 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a8006030 con 0x7f04c01b7cf0 2021-02-10T11:42:38.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.151 7f04bd7fa700 1 -- 172.21.2.101:0/1509349427 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f04a8005560 con 0x7f04c01b7cf0 2021-02-10T11:42:38.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.151 7f04c6d6b700 1 -- 172.21.2.101:0/1509349427 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c01b8d60 con 0x7f04c01b7cf0 2021-02-10T11:42:38.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.151 7f04c6d6b700 1 -- 172.21.2.101:0/1509349427 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c019e730 con 0x7f04c01b7cf0 2021-02-10T11:42:38.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.151 7f04bd7fa700 1 -- 172.21.2.101:0/1509349427 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f04a8009300 con 0x7f04c01b7cf0 2021-02-10T11:42:38.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.151 7f04bd7fa700 1 --2- 172.21.2.101:0/1509349427 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04a40186c0 0x7f04a401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:38.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.151 7f04bd7fa700 1 -- 172.21.2.101:0/1509349427 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f04a8032400 con 0x7f04c01b7cf0 2021-02-10T11:42:38.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.152 7f04bffff700 1 --2- 172.21.2.101:0/1509349427 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04a40186c0 0x7f04a401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:38.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.152 7f04bffff700 1 --2- 172.21.2.101:0/1509349427 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04a40186c0 0x7f04a401abc0 crc :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:38.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.182 7f04c6d6b700 1 -- 172.21.2.101:0/1509349427 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c00ebc00 con 0x7f04c01b7cf0 2021-02-10T11:42:38.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.185 7f04bd7fa700 1 -- 172.21.2.101:0/1509349427 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f04a8030d30 con 0x7f04c01b7cf0 2021-02-10T11:42:38.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.309 7f04c6d6b700 1 -- 172.21.2.101:0/1509349427 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04c00f0b10 con 0x7f04c01b7cf0 2021-02-10T11:42:38.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.310 7f04bd7fa700 1 -- 172.21.2.101:0/1509349427 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f04a8009300 con 0x7f04c01b7cf0 2021-02-10T11:42:38.312 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:42:38.312 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":129,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:42:31.764498","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:42:28.726448","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2021-02-10 11:42:29.754869","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:42:38.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.312 7f04c6d6b700 1 -- 172.21.2.101:0/1509349427 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04a40186c0 msgr2=0x7f04a401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:38.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.312 7f04c6d6b700 1 --2- 172.21.2.101:0/1509349427 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04a40186c0 0x7f04a401abc0 crc :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:38.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.312 7f04c6d6b700 1 -- 172.21.2.101:0/1509349427 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04c01b7cf0 msgr2=0x7f04c019cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:38.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.312 7f04c6d6b700 1 --2- 172.21.2.101:0/1509349427 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04c01b7cf0 0x7f04c019cdb0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 rx=0x7f04a8009650 tx=0x7f04a800b960).stop 2021-02-10T11:42:38.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.312 7f04c6d6b700 1 -- 172.21.2.101:0/1509349427 shutdown_connections 2021-02-10T11:42:38.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.312 7f04c6d6b700 1 --2- 172.21.2.101:0/1509349427 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04a40186c0 0x7f04a401abc0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:38.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.312 7f04c6d6b700 1 --2- 172.21.2.101:0/1509349427 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f04c019da20 0x7f04c019de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:38.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.312 7f04c6d6b700 1 --2- 172.21.2.101:0/1509349427 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f04c01b80f0 0x7f04c01b84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:38.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.312 7f04c6d6b700 1 --2- 172.21.2.101:0/1509349427 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04c01b7cf0 0x7f04c019cdb0 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:38.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.312 7f04c6d6b700 1 -- 172.21.2.101:0/1509349427 >> 172.21.2.101:0/1509349427 conn(0x7f04c00ed810 msgr2=0x7f04c0094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:38.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.312 7f04c6d6b700 1 -- 172.21.2.101:0/1509349427 shutdown_connections 2021-02-10T11:42:38.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.312 7f04c6d6b700 1 -- 172.21.2.101:0/1509349427 wait complete. 2021-02-10T11:42:38.323 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:42:28.726448', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:42:29.754869', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:42:38.323 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:42:38.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.737 7f230080f700 1 Processor -- start 2021-02-10T11:42:38.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.737 7f230080f700 1 -- start start 2021-02-10T11:42:38.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.738 7f230080f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22fc0d1870 0x7f22fc1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:38.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.738 7f230080f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f22fc1b0ae0 0x7f22fc108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:38.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.738 7f230080f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f22fc0d1c70 0x7f22fc108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:38.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.738 7f230080f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22fc0e5ef0 con 0x7f22fc0d1c70 2021-02-10T11:42:38.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.738 7f230080f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22fc109a60 con 0x7f22fc1b0ae0 2021-02-10T11:42:38.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.738 7f230080f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc088070 con 0x7f22fc0d1870 2021-02-10T11:42:38.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.738 7f22fae1d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f22fc0d1c70 0x7f22fc108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:38.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.738 7f22fae1d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f22fc0d1c70 0x7f22fc108f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58302/0 (socket says 172.21.2.101:58302) 2021-02-10T11:42:38.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.738 7f22fae1d700 1 -- 172.21.2.101:0/2485460124 learned_addr learned my addr 172.21.2.101:0/2485460124 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:38.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.738 7f22fae1d700 1 -- 172.21.2.101:0/2485460124 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22fc0d1870 msgr2=0x7f22fc1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:38.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.738 7f22fae1d700 1 --2- 172.21.2.101:0/2485460124 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22fc0d1870 0x7f22fc1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:38.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.738 7f22fae1d700 1 -- 172.21.2.101:0/2485460124 >> v2:172.21.2.108:3300/0 conn(0x7f22fc1b0ae0 msgr2=0x7f22fc108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:38.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.738 7f22fae1d700 1 --2- 172.21.2.101:0/2485460124 >> v2:172.21.2.108:3300/0 conn(0x7f22fc1b0ae0 0x7f22fc108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:38.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.738 7f22fae1d700 1 -- 172.21.2.101:0/2485460124 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22fc109580 con 0x7f22fc0d1c70 2021-02-10T11:42:38.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.739 7f22fae1d700 1 --2- 172.21.2.101:0/2485460124 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22fc0d1c70 0x7f22fc108f60 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 rx=0x7f22f4006f00 tx=0x7f22f4009f10).ready entity=mon.0 client_cookie=a9f0de4ce16464c4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:38.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.739 7f22f961a700 1 -- 172.21.2.101:0/2485460124 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f22f400b3b0 con 0x7f22fc0d1c70 2021-02-10T11:42:38.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.739 7f22f961a700 1 -- 172.21.2.101:0/2485460124 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f400b630 con 0x7f22fc0d1c70 2021-02-10T11:42:38.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.739 7f22f961a700 1 -- 172.21.2.101:0/2485460124 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f22f400bb50 con 0x7f22fc0d1c70 2021-02-10T11:42:38.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.739 7f230080f700 1 -- 172.21.2.101:0/2485460124 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22fc0d1c70 msgr2=0x7f22fc108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:38.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.739 7f230080f700 1 --2- 172.21.2.101:0/2485460124 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22fc0d1c70 0x7f22fc108f60 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 rx=0x7f22f4006f00 tx=0x7f22f4009f10).stop 2021-02-10T11:42:38.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.739 7f230080f700 1 -- 172.21.2.101:0/2485460124 shutdown_connections 2021-02-10T11:42:38.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.740 7f230080f700 1 --2- 172.21.2.101:0/2485460124 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22fc0d1c70 0x7f22fc108f60 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:38.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.740 7f230080f700 1 --2- 172.21.2.101:0/2485460124 >> v2:172.21.2.108:3300/0 conn(0x7f22fc1b0ae0 0x7f22fc108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:38.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.740 7f230080f700 1 --2- 172.21.2.101:0/2485460124 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22fc0d1870 0x7f22fc1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:38.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.740 7f230080f700 1 -- 172.21.2.101:0/2485460124 >> 172.21.2.101:0/2485460124 conn(0x7f22fc0ce420 msgr2=0x7f22fc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:38.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.740 7f230080f700 1 -- 172.21.2.101:0/2485460124 shutdown_connections 2021-02-10T11:42:38.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.740 7f230080f700 1 -- 172.21.2.101:0/2485460124 wait complete. 2021-02-10T11:42:38.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.754 7f230080f700 1 Processor -- start 2021-02-10T11:42:38.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.754 7f230080f700 1 -- start start 2021-02-10T11:42:38.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.754 7f230080f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22fc0988a0 0x7f22fc0a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:38.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.754 7f230080f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22fc0a1010 0x7f22fc0a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:38.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.754 7f230080f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22fc09ec50 0x7f22fc09f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:38.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.754 7f230080f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc0a0470 con 0x7f22fc09ec50 2021-02-10T11:42:38.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.754 7f230080f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc098ca0 con 0x7f22fc0a1010 2021-02-10T11:42:38.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.754 7f230080f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc0a1c80 con 0x7f22fc0988a0 2021-02-10T11:42:38.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.755 7f22fae1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22fc09ec50 0x7f22fc09f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:38.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.755 7f22fae1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22fc09ec50 0x7f22fc09f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58306/0 (socket says 172.21.2.101:58306) 2021-02-10T11:42:38.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.755 7f22fae1d700 1 -- 172.21.2.101:0/2618754791 learned_addr learned my addr 172.21.2.101:0/2618754791 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:38.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.755 7f22fae1d700 1 -- 172.21.2.101:0/2618754791 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22fc0988a0 msgr2=0x7f22fc0a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:38.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.755 7f22fae1d700 1 --2- 172.21.2.101:0/2618754791 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22fc0988a0 0x7f22fc0a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:38.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.755 7f22fae1d700 1 -- 172.21.2.101:0/2618754791 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22fc0a1010 msgr2=0x7f22fc0a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:38.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.755 7f22fae1d700 1 --2- 172.21.2.101:0/2618754791 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22fc0a1010 0x7f22fc0a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:38.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.755 7f22fae1d700 1 -- 172.21.2.101:0/2618754791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22fc09f670 con 0x7f22fc09ec50 2021-02-10T11:42:38.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.755 7f22fae1d700 1 --2- 172.21.2.101:0/2618754791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22fc09ec50 0x7f22fc09f050 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 rx=0x7f22f4009390 tx=0x7f22f400bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:38.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.755 7f22f2ffd700 1 -- 172.21.2.101:0/2618754791 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f22f4003030 con 0x7f22fc09ec50 2021-02-10T11:42:38.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.755 7f22f2ffd700 1 -- 172.21.2.101:0/2618754791 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f40068e0 con 0x7f22fc09ec50 2021-02-10T11:42:38.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.755 7f22f2ffd700 1 -- 172.21.2.101:0/2618754791 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f22f400c570 con 0x7f22fc09ec50 2021-02-10T11:42:38.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.755 7f230080f700 1 -- 172.21.2.101:0/2618754791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22fc09f670 con 0x7f22fc09ec50 2021-02-10T11:42:38.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.755 7f230080f700 1 -- 172.21.2.101:0/2618754791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22fc09f670 con 0x7f22fc09ec50 2021-02-10T11:42:38.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.756 7f22f2ffd700 1 -- 172.21.2.101:0/2618754791 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f22f4005820 con 0x7f22fc09ec50 2021-02-10T11:42:38.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.756 7f22f2ffd700 1 --2- 172.21.2.101:0/2618754791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22e00186c0 0x7f22e001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:38.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.756 7f22f2ffd700 1 -- 172.21.2.101:0/2618754791 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f22f4032920 con 0x7f22fc09ec50 2021-02-10T11:42:38.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.756 7f22fa61c700 1 --2- 172.21.2.101:0/2618754791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22e00186c0 0x7f22e001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:38.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.757 7f22fa61c700 1 --2- 172.21.2.101:0/2618754791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22e00186c0 0x7f22e001abc0 crc :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:38.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.788 7f230080f700 1 -- 172.21.2.101:0/2618754791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22fc109a60 con 0x7f22fc09ec50 2021-02-10T11:42:38.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.790 7f22f2ffd700 1 -- 172.21.2.101:0/2618754791 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f22f4006c80 con 0x7f22fc09ec50 2021-02-10T11:42:38.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.920 7f230080f700 1 -- 172.21.2.101:0/2618754791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f22fc109a60 con 0x7f22fc09ec50 2021-02-10T11:42:38.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.920 7f22f2ffd700 1 -- 172.21.2.101:0/2618754791 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v129) v1 ==== 92+0+10 (secure 0 0 0) 0x7f22f4006c80 con 0x7f22fc09ec50 2021-02-10T11:42:38.921 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:42:38.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.922 7f230080f700 1 -- 172.21.2.101:0/2618754791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22e00186c0 msgr2=0x7f22e001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:38.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.922 7f230080f700 1 --2- 172.21.2.101:0/2618754791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22e00186c0 0x7f22e001abc0 crc :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:38.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.922 7f230080f700 1 -- 172.21.2.101:0/2618754791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22fc09ec50 msgr2=0x7f22fc09f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:38.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.922 7f230080f700 1 --2- 172.21.2.101:0/2618754791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22fc09ec50 0x7f22fc09f050 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 rx=0x7f22f4009390 tx=0x7f22f400bd40).stop 2021-02-10T11:42:38.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.922 7f230080f700 1 -- 172.21.2.101:0/2618754791 shutdown_connections 2021-02-10T11:42:38.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.922 7f230080f700 1 --2- 172.21.2.101:0/2618754791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22fc09ec50 0x7f22fc09f050 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:38.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.922 7f230080f700 1 --2- 172.21.2.101:0/2618754791 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22fc0a1010 0x7f22fc0a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:38.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.922 7f230080f700 1 --2- 172.21.2.101:0/2618754791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22e00186c0 0x7f22e001abc0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:38.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.922 7f230080f700 1 --2- 172.21.2.101:0/2618754791 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22fc0988a0 0x7f22fc0a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:38.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.922 7f230080f700 1 -- 172.21.2.101:0/2618754791 >> 172.21.2.101:0/2618754791 conn(0x7f22fc0affb0 msgr2=0x7f22fc093b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:38.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.922 7f230080f700 1 -- 172.21.2.101:0/2618754791 shutdown_connections 2021-02-10T11:42:38.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:38.923 7f230080f700 1 -- 172.21.2.101:0/2618754791 wait complete. 2021-02-10T11:42:38.932 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:42:39.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.347 7fabae77b700 1 Processor -- start 2021-02-10T11:42:39.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.347 7fabae77b700 1 -- start start 2021-02-10T11:42:39.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.347 7fabae77b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faba80d1870 0x7faba81b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:39.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.347 7fabae77b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7faba80cff70 0x7faba8109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:39.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.347 7fabae77b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba8109a60 con 0x7faba81b0ae0 2021-02-10T11:42:39.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.347 7fabae77b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba8109750 con 0x7faba80cff70 2021-02-10T11:42:39.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.347 7fabae77b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba8088070 con 0x7faba80d1870 2021-02-10T11:42:39.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.347 7fabacd98700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7faba80cff70 0x7faba8109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:39.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.348 7fabacd98700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7faba80cff70 0x7faba8109130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58322/0 (socket says 172.21.2.101:58322) 2021-02-10T11:42:39.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.348 7fabacd98700 1 -- 172.21.2.101:0/1928465624 learned_addr learned my addr 172.21.2.101:0/1928465624 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:39.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.348 7faba7fff700 1 --2- 172.21.2.101:0/1928465624 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faba80d1870 0x7faba81b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:39.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.348 7fabacd98700 1 -- 172.21.2.101:0/1928465624 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faba80d1870 msgr2=0x7faba81b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:39.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.348 7fabacd98700 1 --2- 172.21.2.101:0/1928465624 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faba80d1870 0x7faba81b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:39.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.348 7fabacd98700 1 -- 172.21.2.101:0/1928465624 >> v1:172.21.2.108:6789/0 conn(0x7faba81b0ae0 legacy=0x7faba8108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:39.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.348 7fabacd98700 1 -- 172.21.2.101:0/1928465624 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba8088260 con 0x7faba80cff70 2021-02-10T11:42:39.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.348 7faba7fff700 1 --2- 172.21.2.101:0/1928465624 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faba80d1870 0x7faba81b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:42:39.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.348 7fabacd98700 1 --2- 172.21.2.101:0/1928465624 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faba80cff70 0x7faba8109130 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 rx=0x7faba0006f00 tx=0x7faba0009f10).ready entity=mon.0 client_cookie=7fe890dcfb5cc254 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:39.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.348 7faba6ffd700 1 -- 172.21.2.101:0/1928465624 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faba000b3b0 con 0x7faba80cff70 2021-02-10T11:42:39.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.348 7faba6ffd700 1 -- 172.21.2.101:0/1928465624 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba000b730 con 0x7faba80cff70 2021-02-10T11:42:39.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.348 7faba6ffd700 1 -- 172.21.2.101:0/1928465624 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faba000bc50 con 0x7faba80cff70 2021-02-10T11:42:39.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.348 7fabae77b700 1 -- 172.21.2.101:0/1928465624 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faba80cff70 msgr2=0x7faba8109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:39.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.348 7fabae77b700 1 --2- 172.21.2.101:0/1928465624 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faba80cff70 0x7faba8109130 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 rx=0x7faba0006f00 tx=0x7faba0009f10).stop 2021-02-10T11:42:39.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.349 7fabae77b700 1 -- 172.21.2.101:0/1928465624 shutdown_connections 2021-02-10T11:42:39.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.349 7fabae77b700 1 --2- 172.21.2.101:0/1928465624 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faba80cff70 0x7faba8109130 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:39.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.349 7fabae77b700 1 --2- 172.21.2.101:0/1928465624 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faba80d1870 0x7faba81b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:39.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.349 7fabae77b700 1 -- 172.21.2.101:0/1928465624 >> 172.21.2.101:0/1928465624 conn(0x7faba80ce420 msgr2=0x7faba80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:39.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.349 7fabae77b700 1 -- 172.21.2.101:0/1928465624 shutdown_connections 2021-02-10T11:42:39.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.349 7fabae77b700 1 -- 172.21.2.101:0/1928465624 wait complete. 2021-02-10T11:42:39.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.363 7fabae77b700 1 Processor -- start 2021-02-10T11:42:39.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.364 7fabae77b700 1 -- start start 2021-02-10T11:42:39.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.364 7fabae77b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faba80a58c0 0x7faba80a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:39.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.364 7fabae77b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faba80b1ba0 0x7faba80b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:39.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.364 7fabae77b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faba80b25c0 0x7faba80b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:39.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.364 7fabae77b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba80b2a40 con 0x7faba80b25c0 2021-02-10T11:42:39.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.364 7fabae77b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba80a5340 con 0x7faba80a58c0 2021-02-10T11:42:39.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.364 7fabae77b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba80a5560 con 0x7faba80b1ba0 2021-02-10T11:42:39.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.364 7fabacd98700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faba80b25c0 0x7faba80b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:39.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.364 7fabacd98700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faba80b25c0 0x7faba80b3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58328/0 (socket says 172.21.2.101:58328) 2021-02-10T11:42:39.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.364 7fabacd98700 1 -- 172.21.2.101:0/1831604391 learned_addr learned my addr 172.21.2.101:0/1831604391 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:39.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.365 7fabacd98700 1 -- 172.21.2.101:0/1831604391 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faba80b1ba0 msgr2=0x7faba80b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:39.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.365 7faba77fe700 1 --2- 172.21.2.101:0/1831604391 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faba80b1ba0 0x7faba80b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:39.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.365 7faba7fff700 1 --2- 172.21.2.101:0/1831604391 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faba80a58c0 0x7faba80a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:39.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.365 7fabacd98700 1 --2- 172.21.2.101:0/1831604391 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faba80b1ba0 0x7faba80b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:39.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.365 7fabacd98700 1 -- 172.21.2.101:0/1831604391 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faba80a58c0 msgr2=0x7faba80a4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:39.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.365 7fabacd98700 1 --2- 172.21.2.101:0/1831604391 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faba80a58c0 0x7faba80a4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:39.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.365 7fabacd98700 1 -- 172.21.2.101:0/1831604391 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba80b3890 con 0x7faba80b25c0 2021-02-10T11:42:39.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.365 7faba7fff700 1 --2- 172.21.2.101:0/1831604391 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faba80a58c0 0x7faba80a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:42:39.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.365 7fabacd98700 1 --2- 172.21.2.101:0/1831604391 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faba80b25c0 0x7faba80b3270 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 rx=0x7faba000b8e0 tx=0x7faba000bc80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:39.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.365 7faba4ff9700 1 -- 172.21.2.101:0/1831604391 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faba0003030 con 0x7faba80b25c0 2021-02-10T11:42:39.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.365 7faba4ff9700 1 -- 172.21.2.101:0/1831604391 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba00068e0 con 0x7faba80b25c0 2021-02-10T11:42:39.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.365 7faba4ff9700 1 -- 172.21.2.101:0/1831604391 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faba000c4e0 con 0x7faba80b25c0 2021-02-10T11:42:39.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.365 7fabae77b700 1 -- 172.21.2.101:0/1831604391 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba80b3890 con 0x7faba80b25c0 2021-02-10T11:42:39.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.365 7fabae77b700 1 -- 172.21.2.101:0/1831604391 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba80b3de0 con 0x7faba80b25c0 2021-02-10T11:42:39.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.366 7faba4ff9700 1 -- 172.21.2.101:0/1831604391 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7faba0001270 con 0x7faba80b25c0 2021-02-10T11:42:39.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.366 7faba4ff9700 1 --2- 172.21.2.101:0/1831604391 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab880186c0 0x7fab8801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:39.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.366 7faba4ff9700 1 -- 172.21.2.101:0/1831604391 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7faba00328c0 con 0x7faba80b25c0 2021-02-10T11:42:39.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.366 7faba7fff700 1 --2- 172.21.2.101:0/1831604391 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab880186c0 0x7fab8801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:39.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.366 7faba7fff700 1 --2- 172.21.2.101:0/1831604391 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab880186c0 0x7fab8801abc0 crc :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:39.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.397 7fabae77b700 1 -- 172.21.2.101:0/1831604391 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba8082980 con 0x7faba80b25c0 2021-02-10T11:42:39.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.399 7faba4ff9700 1 -- 172.21.2.101:0/1831604391 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7faba0009430 con 0x7faba80b25c0 2021-02-10T11:42:39.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.527 7fabae77b700 1 -- 172.21.2.101:0/1831604391 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7faba8109a60 con 0x7faba80b25c0 2021-02-10T11:42:39.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.527 7faba4ff9700 1 -- 172.21.2.101:0/1831604391 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v129) v1 ==== 104+0+10 (secure 0 0 0) 0x7faba0001270 con 0x7faba80b25c0 2021-02-10T11:42:39.528 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:42:39.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.529 7fabae77b700 1 -- 172.21.2.101:0/1831604391 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab880186c0 msgr2=0x7fab8801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:39.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.529 7fabae77b700 1 --2- 172.21.2.101:0/1831604391 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab880186c0 0x7fab8801abc0 crc :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:39.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.529 7fabae77b700 1 -- 172.21.2.101:0/1831604391 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faba80b25c0 msgr2=0x7faba80b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:39.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.529 7fabae77b700 1 --2- 172.21.2.101:0/1831604391 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faba80b25c0 0x7faba80b3270 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 rx=0x7faba000b8e0 tx=0x7faba000bc80).stop 2021-02-10T11:42:39.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.529 7fabae77b700 1 -- 172.21.2.101:0/1831604391 shutdown_connections 2021-02-10T11:42:39.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.529 7fabae77b700 1 --2- 172.21.2.101:0/1831604391 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab880186c0 0x7fab8801abc0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:39.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.529 7fabae77b700 1 --2- 172.21.2.101:0/1831604391 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faba80b1ba0 0x7faba80b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:39.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.529 7fabae77b700 1 --2- 172.21.2.101:0/1831604391 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faba80b25c0 0x7faba80b3270 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:39.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.529 7fabae77b700 1 --2- 172.21.2.101:0/1831604391 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faba80a58c0 0x7faba80a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:39.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.529 7fabae77b700 1 -- 172.21.2.101:0/1831604391 >> 172.21.2.101:0/1831604391 conn(0x7faba80b5dc0 msgr2=0x7faba8095960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:39.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.530 7fabae77b700 1 -- 172.21.2.101:0/1831604391 shutdown_connections 2021-02-10T11:42:39.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.530 7fabae77b700 1 -- 172.21.2.101:0/1831604391 wait complete. 2021-02-10T11:42:39.539 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:42:39.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.952 7fa35cfca700 1 Processor -- start 2021-02-10T11:42:39.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.952 7fa35cfca700 1 -- start start 2021-02-10T11:42:39.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.952 7fa35cfca700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa3580d1c70 0x7fa3580cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:39.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.952 7fa35cfca700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3580cf580 0x7fa3580cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:39.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.952 7fa35cfca700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3580fe1c0 con 0x7fa3580d1870 2021-02-10T11:42:39.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.952 7fa35cfca700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3581b0cb0 con 0x7fa3580d1c70 2021-02-10T11:42:39.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.952 7fa35cfca700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa358109a60 con 0x7fa3580cf580 2021-02-10T11:42:39.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.952 7fa35659c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa3580d1c70 0x7fa3580cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:39.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.952 7fa35659c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa3580d1c70 0x7fa3580cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58336/0 (socket says 172.21.2.101:58336) 2021-02-10T11:42:39.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.952 7fa35659c700 1 -- 172.21.2.101:0/3968791801 learned_addr learned my addr 172.21.2.101:0/3968791801 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:39.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.953 7fa35659c700 1 -- 172.21.2.101:0/3968791801 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3580cf580 msgr2=0x7fa3580cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:39.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.953 7fa35659c700 1 --2- 172.21.2.101:0/3968791801 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3580cf580 0x7fa3580cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:39.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.953 7fa35659c700 1 -- 172.21.2.101:0/3968791801 >> v1:172.21.2.108:6789/0 conn(0x7fa3580d1870 legacy=0x7fa3581b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:39.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.953 7fa35659c700 1 -- 172.21.2.101:0/3968791801 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa358088070 con 0x7fa3580d1c70 2021-02-10T11:42:39.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.953 7fa35659c700 1 --2- 172.21.2.101:0/3968791801 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa3580d1c70 0x7fa3580cef60 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 rx=0x7fa34c006700 tx=0x7fa34c009b80).ready entity=mon.0 client_cookie=14f52a21036d67b0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:39.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.953 7fa355d9b700 1 -- 172.21.2.101:0/3968791801 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa34c00c0c0 con 0x7fa3580d1c70 2021-02-10T11:42:39.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.953 7fa355d9b700 1 -- 172.21.2.101:0/3968791801 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34c00c340 con 0x7fa3580d1c70 2021-02-10T11:42:39.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.953 7fa355d9b700 1 -- 172.21.2.101:0/3968791801 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa34c00c860 con 0x7fa3580d1c70 2021-02-10T11:42:39.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.953 7fa35cfca700 1 -- 172.21.2.101:0/3968791801 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa3580d1c70 msgr2=0x7fa3580cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:39.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.953 7fa35cfca700 1 --2- 172.21.2.101:0/3968791801 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa3580d1c70 0x7fa3580cef60 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 rx=0x7fa34c006700 tx=0x7fa34c009b80).stop 2021-02-10T11:42:39.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.954 7fa35cfca700 1 -- 172.21.2.101:0/3968791801 shutdown_connections 2021-02-10T11:42:39.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.954 7fa35cfca700 1 --2- 172.21.2.101:0/3968791801 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3580cf580 0x7fa3580cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:39.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.954 7fa35cfca700 1 --2- 172.21.2.101:0/3968791801 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa3580d1c70 0x7fa3580cef60 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:39.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.954 7fa35cfca700 1 -- 172.21.2.101:0/3968791801 >> 172.21.2.101:0/3968791801 conn(0x7fa3580ce420 msgr2=0x7fa3580d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:39.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.954 7fa35cfca700 1 -- 172.21.2.101:0/3968791801 shutdown_connections 2021-02-10T11:42:39.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.954 7fa35cfca700 1 -- 172.21.2.101:0/3968791801 wait complete. 2021-02-10T11:42:39.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.969 7fa35cfca700 1 Processor -- start 2021-02-10T11:42:39.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.969 7fa35cfca700 1 -- start start 2021-02-10T11:42:39.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.969 7fa35cfca700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3580c3820 0x7fa358084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:39.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.969 7fa35cfca700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3580a0290 0x7fa3580852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:39.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.969 7fa35cfca700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa358086270 0x7fa358086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:39.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.969 7fa35cfca700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3580c0700 con 0x7fa358086270 2021-02-10T11:42:39.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.969 7fa35cfca700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3580858f0 con 0x7fa3580c3820 2021-02-10T11:42:39.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.969 7fa35cfca700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa358085b10 con 0x7fa3580a0290 2021-02-10T11:42:39.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.969 7fa35759e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa358086270 0x7fa358086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:39.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.969 7fa35759e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa358086270 0x7fa358086670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58344/0 (socket says 172.21.2.101:58344) 2021-02-10T11:42:39.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.969 7fa35759e700 1 -- 172.21.2.101:0/1067707203 learned_addr learned my addr 172.21.2.101:0/1067707203 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:39.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.970 7fa35659c700 1 --2- 172.21.2.101:0/1067707203 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3580a0290 0x7fa3580852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:39.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.970 7fa356d9d700 1 --2- 172.21.2.101:0/1067707203 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3580c3820 0x7fa358084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:39.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.970 7fa35759e700 1 -- 172.21.2.101:0/1067707203 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3580a0290 msgr2=0x7fa3580852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:39.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.970 7fa35759e700 1 --2- 172.21.2.101:0/1067707203 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3580a0290 0x7fa3580852d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:39.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.970 7fa35759e700 1 -- 172.21.2.101:0/1067707203 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3580c3820 msgr2=0x7fa358084cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:39.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.970 7fa35759e700 1 --2- 172.21.2.101:0/1067707203 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3580c3820 0x7fa358084cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:39.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.970 7fa35759e700 1 -- 172.21.2.101:0/1067707203 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa358086c90 con 0x7fa358086270 2021-02-10T11:42:39.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.970 7fa35759e700 1 --2- 172.21.2.101:0/1067707203 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa358086270 0x7fa358086670 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 rx=0x7fa350009980 tx=0x7fa35000bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:39.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.970 7fa3477fe700 1 -- 172.21.2.101:0/1067707203 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa350005870 con 0x7fa358086270 2021-02-10T11:42:39.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.970 7fa3477fe700 1 -- 172.21.2.101:0/1067707203 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa350005b30 con 0x7fa358086270 2021-02-10T11:42:39.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.970 7fa35cfca700 1 -- 172.21.2.101:0/1067707203 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa358086c90 con 0x7fa358086270 2021-02-10T11:42:39.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.970 7fa35cfca700 1 -- 172.21.2.101:0/1067707203 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3581a2f00 con 0x7fa358086270 2021-02-10T11:42:39.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.970 7fa3477fe700 1 -- 172.21.2.101:0/1067707203 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa350006030 con 0x7fa358086270 2021-02-10T11:42:39.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.971 7fa3477fe700 1 -- 172.21.2.101:0/1067707203 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa350005d10 con 0x7fa358086270 2021-02-10T11:42:39.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.971 7fa3477fe700 1 --2- 172.21.2.101:0/1067707203 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa33c018730 0x7fa33c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:39.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.971 7fa3477fe700 1 -- 172.21.2.101:0/1067707203 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa350032bb0 con 0x7fa358086270 2021-02-10T11:42:39.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.971 7fa356d9d700 1 --2- 172.21.2.101:0/1067707203 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa33c018730 0x7fa33c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:39.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:39.972 7fa356d9d700 1 --2- 172.21.2.101:0/1067707203 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa33c018730 0x7fa33c01ac10 crc :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:40.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.002 7fa35cfca700 1 -- 172.21.2.101:0/1067707203 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa358082a70 con 0x7fa358086270 2021-02-10T11:42:40.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.005 7fa3477fe700 1 -- 172.21.2.101:0/1067707203 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa350005d40 con 0x7fa358086270 2021-02-10T11:42:40.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.133 7fa35cfca700 1 -- 172.21.2.101:0/1067707203 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7fa358082a70 con 0x7fa358086270 2021-02-10T11:42:40.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.133 7fa3477fe700 1 -- 172.21.2.101:0/1067707203 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v129) v1 ==== 100+0+10 (secure 0 0 0) 0x7fa350006f20 con 0x7fa358086270 2021-02-10T11:42:40.134 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:42:40.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.135 7fa35cfca700 1 -- 172.21.2.101:0/1067707203 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa33c018730 msgr2=0x7fa33c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:40.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.135 7fa35cfca700 1 --2- 172.21.2.101:0/1067707203 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa33c018730 0x7fa33c01ac10 crc :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:40.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.135 7fa35cfca700 1 -- 172.21.2.101:0/1067707203 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa358086270 msgr2=0x7fa358086670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:40.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.135 7fa35cfca700 1 --2- 172.21.2.101:0/1067707203 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa358086270 0x7fa358086670 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 rx=0x7fa350009980 tx=0x7fa35000bcd0).stop 2021-02-10T11:42:40.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.135 7fa35cfca700 1 -- 172.21.2.101:0/1067707203 shutdown_connections 2021-02-10T11:42:40.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.135 7fa35cfca700 1 --2- 172.21.2.101:0/1067707203 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa33c018730 0x7fa33c01ac10 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:40.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.135 7fa35cfca700 1 --2- 172.21.2.101:0/1067707203 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3580a0290 0x7fa3580852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:40.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.135 7fa35cfca700 1 --2- 172.21.2.101:0/1067707203 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa358086270 0x7fa358086670 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:40.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.135 7fa35cfca700 1 --2- 172.21.2.101:0/1067707203 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3580c3820 0x7fa358084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:40.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.135 7fa35cfca700 1 -- 172.21.2.101:0/1067707203 >> 172.21.2.101:0/1067707203 conn(0x7fa3580872c0 msgr2=0x7fa358095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:40.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.135 7fa35cfca700 1 -- 172.21.2.101:0/1067707203 shutdown_connections 2021-02-10T11:42:40.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.136 7fa35cfca700 1 -- 172.21.2.101:0/1067707203 wait complete. 2021-02-10T11:42:40.145 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:42:40.145 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:42:40.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.557 7fd6f0f66700 1 Processor -- start 2021-02-10T11:42:40.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.557 7fd6f0f66700 1 -- start start 2021-02-10T11:42:40.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.557 7fd6f0f66700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd6ec0d0340 0x7fd6ec0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:40.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.557 7fd6f0f66700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ec0f0b10 con 0x7fd6ec0d1c70 2021-02-10T11:42:40.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.557 7fd6f0f66700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ec1b0be0 con 0x7fd6ec0d1870 2021-02-10T11:42:40.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.557 7fd6f0f66700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6ec0e5ed0 con 0x7fd6ec0d0340 2021-02-10T11:42:40.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.557 7fd6eb314700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd6ec0d1c70 0x7fd6ec0efc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58842/0 (socket says 172.21.2.101:58842) 2021-02-10T11:42:40.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.557 7fd6eb314700 1 -- 172.21.2.101:0/909845079 learned_addr learned my addr 172.21.2.101:0/909845079 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:40.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.558 7fd6e9b11700 1 -- 172.21.2.101:0/909845079 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2030322752 0 0) 0x7fd6e4000f90 con 0x7fd6ec0d1c70 2021-02-10T11:42:40.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.558 7fd6e9b11700 1 -- 172.21.2.101:0/909845079 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6d8002fe0 con 0x7fd6ec0d1c70 2021-02-10T11:42:40.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.558 7fd6ea312700 1 --2- 172.21.2.101:0/909845079 >> v2:172.21.2.108:3300/0 conn(0x7fd6ec0d0340 0x7fd6ec0ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:40.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.558 7fd6e9b11700 1 -- 172.21.2.101:0/909845079 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2195598843 0 0) 0x7fd6e4001460 con 0x7fd6ec0d1c70 2021-02-10T11:42:40.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.558 7fd6e9b11700 1 -- 172.21.2.101:0/909845079 >> v2:172.21.2.108:3300/0 conn(0x7fd6ec0d0340 msgr2=0x7fd6ec0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:40.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.558 7fd6e9b11700 1 --2- 172.21.2.101:0/909845079 >> v2:172.21.2.108:3300/0 conn(0x7fd6ec0d0340 0x7fd6ec0ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:40.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.558 7fd6e9b11700 1 -- 172.21.2.101:0/909845079 >> v1:172.21.2.108:6789/0 conn(0x7fd6ec0d1870 legacy=0x7fd6ec1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:40.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.558 7fd6e9b11700 1 -- 172.21.2.101:0/909845079 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec0f0450 con 0x7fd6ec0d1c70 2021-02-10T11:42:40.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.558 7fd6e9b11700 1 -- 172.21.2.101:0/909845079 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fd6e40019e0 con 0x7fd6ec0d1c70 2021-02-10T11:42:40.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.558 7fd6e9b11700 1 -- 172.21.2.101:0/909845079 >> v1:172.21.2.101:6789/0 conn(0x7fd6ec0d1c70 legacy=0x7fd6ec0efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:40.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.558 7fd6e9b11700 1 --2- 172.21.2.101:0/909845079 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6d8003a60 0x7fd6d8005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:40.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.558 7fd6e9b11700 1 --2- 172.21.2.101:0/909845079 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6d8006790 0x7fd6d8008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:40.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.558 7fd6e9b11700 1 --2- 172.21.2.101:0/909845079 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6d8009590 0x7fd6d800bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:40.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.558 7fd6e9b11700 1 -- 172.21.2.101:0/909845079 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d800c300 con 0x7fd6d8006790 2021-02-10T11:42:40.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.558 7fd6e9b11700 1 -- 172.21.2.101:0/909845079 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d800c520 con 0x7fd6d8003a60 2021-02-10T11:42:40.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.558 7fd6e9b11700 1 -- 172.21.2.101:0/909845079 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d800c740 con 0x7fd6d8009590 2021-02-10T11:42:40.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.559 7fd6ea312700 1 --2- 172.21.2.101:0/909845079 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6d8006790 0x7fd6d8008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:40.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.559 7fd6ea312700 1 -- 172.21.2.101:0/909845079 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6d8009590 msgr2=0x7fd6d800bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:40.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.559 7fd6eb314700 1 --2- 172.21.2.101:0/909845079 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6d8009590 0x7fd6d800bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:40.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.559 7fd6ea312700 1 --2- 172.21.2.101:0/909845079 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6d8009590 0x7fd6d800bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:40.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.559 7fd6ea312700 1 -- 172.21.2.101:0/909845079 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6d8003a60 msgr2=0x7fd6d8005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:40.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.559 7fd6ea312700 1 --2- 172.21.2.101:0/909845079 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6d8003a60 0x7fd6d8005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:40.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.559 7fd6ea312700 1 -- 172.21.2.101:0/909845079 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d80509b0 con 0x7fd6d8006790 2021-02-10T11:42:40.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.559 7fd6eb314700 1 --2- 172.21.2.101:0/909845079 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6d8009590 0x7fd6d800bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:42:40.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.559 7fd6eb314700 1 -- 172.21.2.101:0/909845079 reap_dead start 2021-02-10T11:42:40.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.559 7fd6ea312700 1 --2- 172.21.2.101:0/909845079 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6d8006790 0x7fd6d8008c90 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 rx=0x7fd6dc00c8f0 tx=0x7fd6dc00cc90).ready entity=mon.0 client_cookie=fd235652c16742c9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:40.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.559 7fd6e9b11700 1 -- 172.21.2.101:0/909845079 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6dc0009b0 con 0x7fd6d8006790 2021-02-10T11:42:40.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.559 7fd6e9b11700 1 -- 172.21.2.101:0/909845079 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc00cdd0 con 0x7fd6d8006790 2021-02-10T11:42:40.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.559 7fd6e9b11700 1 -- 172.21.2.101:0/909845079 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6dc009590 con 0x7fd6d8006790 2021-02-10T11:42:40.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.559 7fd6f0f66700 1 -- 172.21.2.101:0/909845079 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6d8006790 msgr2=0x7fd6d8008c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:40.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.559 7fd6f0f66700 1 --2- 172.21.2.101:0/909845079 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6d8006790 0x7fd6d8008c90 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 rx=0x7fd6dc00c8f0 tx=0x7fd6dc00cc90).stop 2021-02-10T11:42:40.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.560 7fd6f0f66700 1 -- 172.21.2.101:0/909845079 shutdown_connections 2021-02-10T11:42:40.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.560 7fd6f0f66700 1 --2- 172.21.2.101:0/909845079 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6d8006790 0x7fd6d8008c90 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:40.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.560 7fd6f0f66700 1 -- 172.21.2.101:0/909845079 >> 172.21.2.101:0/909845079 conn(0x7fd6ec0ce450 msgr2=0x7fd6ec0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:40.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.560 7fd6f0f66700 1 -- 172.21.2.101:0/909845079 shutdown_connections 2021-02-10T11:42:40.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.560 7fd6f0f66700 1 -- 172.21.2.101:0/909845079 wait complete. 2021-02-10T11:42:40.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.575 7fd6f0f66700 1 Processor -- start 2021-02-10T11:42:40.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.575 7fd6f0f66700 1 -- start start 2021-02-10T11:42:40.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.575 7fd6f0f66700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6ec195650 0x7fd6ec0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:40.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.575 7fd6f0f66700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0b5b80 0x7fd6ec1a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:40.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.575 7fd6f0f66700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6ec0acfa0 0x7fd6ec1a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:40.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.575 7fd6f0f66700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec0b6010 con 0x7fd6ec0b5b80 2021-02-10T11:42:40.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.575 7fd6f0f66700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec1a3a60 con 0x7fd6ec0acfa0 2021-02-10T11:42:40.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.575 7fd6f0f66700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec1a3c40 con 0x7fd6ec195650 2021-02-10T11:42:40.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.576 7fd6ea312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0b5b80 0x7fd6ec1a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:40.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.576 7fd6ea312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0b5b80 0x7fd6ec1a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58358/0 (socket says 172.21.2.101:58358) 2021-02-10T11:42:40.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.576 7fd6ea312700 1 -- 172.21.2.101:0/665194107 learned_addr learned my addr 172.21.2.101:0/665194107 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:40.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.576 7fd6ea312700 1 -- 172.21.2.101:0/665194107 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6ec195650 msgr2=0x7fd6ec0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:40.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.576 7fd6ea312700 1 --2- 172.21.2.101:0/665194107 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6ec195650 0x7fd6ec0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:40.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.576 7fd6ea312700 1 -- 172.21.2.101:0/665194107 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6ec0acfa0 msgr2=0x7fd6ec1a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:40.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.576 7fd6ea312700 1 --2- 172.21.2.101:0/665194107 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6ec0acfa0 0x7fd6ec1a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:40.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.576 7fd6ea312700 1 -- 172.21.2.101:0/665194107 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec0bcae0 con 0x7fd6ec0b5b80 2021-02-10T11:42:40.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.576 7fd6ea312700 1 --2- 172.21.2.101:0/665194107 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0b5b80 0x7fd6ec1a2e20 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 rx=0x7fd6dc006db0 tx=0x7fd6dc00b300).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:40.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.577 7fd6e37fe700 1 -- 172.21.2.101:0/665194107 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6dc006740 con 0x7fd6ec0b5b80 2021-02-10T11:42:40.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.577 7fd6e37fe700 1 -- 172.21.2.101:0/665194107 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc00c7c0 con 0x7fd6ec0b5b80 2021-02-10T11:42:40.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.577 7fd6f0f66700 1 -- 172.21.2.101:0/665194107 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ec0bcae0 con 0x7fd6ec0b5b80 2021-02-10T11:42:40.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.577 7fd6f0f66700 1 -- 172.21.2.101:0/665194107 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ec0bd070 con 0x7fd6ec0b5b80 2021-02-10T11:42:40.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.577 7fd6e37fe700 1 -- 172.21.2.101:0/665194107 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6dc00cce0 con 0x7fd6ec0b5b80 2021-02-10T11:42:40.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.577 7fd6e37fe700 1 -- 172.21.2.101:0/665194107 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd6dc00b6f0 con 0x7fd6ec0b5b80 2021-02-10T11:42:40.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.577 7fd6e37fe700 1 --2- 172.21.2.101:0/665194107 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6d00186d0 0x7fd6d001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:40.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.578 7fd6e37fe700 1 -- 172.21.2.101:0/665194107 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd6dc0327e0 con 0x7fd6ec0b5b80 2021-02-10T11:42:40.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.578 7fd6eab13700 1 --2- 172.21.2.101:0/665194107 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6d00186d0 0x7fd6d001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:40.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.578 7fd6eab13700 1 --2- 172.21.2.101:0/665194107 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6d00186d0 0x7fd6d001abd0 crc :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:40.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.607 7fd6f0f66700 1 -- 172.21.2.101:0/665194107 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ec0f0ab0 con 0x7fd6ec0b5b80 2021-02-10T11:42:40.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.610 7fd6e37fe700 1 -- 172.21.2.101:0/665194107 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd6dc0009b0 con 0x7fd6ec0b5b80 2021-02-10T11:42:40.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.737 7fd6f0f66700 1 -- 172.21.2.101:0/665194107 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6ec0f0ab0 con 0x7fd6ec0b5b80 2021-02-10T11:42:40.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.738 7fd6e37fe700 1 -- 172.21.2.101:0/665194107 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+2105 (secure 0 0 0) 0x7fd6dc00b6f0 con 0x7fd6ec0b5b80 2021-02-10T11:42:40.740 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 105 2021-02-10T11:42:40.740 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:42:40.740 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":105,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11139,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1647083555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1647083555},{"type":"v1","addr":"172.21.2.101:6835","nonce":1647083555}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":101}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:42:30.762357","modified":"2021-02-10 11:42:32.772467","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7816":{"gid":7816,"name":"b","rank":0,"incarnation":102,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/381011050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":381011050},{"type":"v1","addr":"172.21.2.108:6833","nonce":381011050}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-02-10T11:42:40.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.740 7fd6f0f66700 1 -- 172.21.2.101:0/665194107 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6d00186d0 msgr2=0x7fd6d001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:40.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.740 7fd6f0f66700 1 --2- 172.21.2.101:0/665194107 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6d00186d0 0x7fd6d001abd0 crc :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:40.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.740 7fd6f0f66700 1 -- 172.21.2.101:0/665194107 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0b5b80 msgr2=0x7fd6ec1a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:40.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.740 7fd6f0f66700 1 --2- 172.21.2.101:0/665194107 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0b5b80 0x7fd6ec1a2e20 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 rx=0x7fd6dc006db0 tx=0x7fd6dc00b300).stop 2021-02-10T11:42:40.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.740 7fd6f0f66700 1 -- 172.21.2.101:0/665194107 shutdown_connections 2021-02-10T11:42:40.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.740 7fd6f0f66700 1 --2- 172.21.2.101:0/665194107 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6ec0acfa0 0x7fd6ec1a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:40.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.740 7fd6f0f66700 1 --2- 172.21.2.101:0/665194107 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0b5b80 0x7fd6ec1a2e20 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:40.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.740 7fd6f0f66700 1 --2- 172.21.2.101:0/665194107 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6d00186d0 0x7fd6d001abd0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:40.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.740 7fd6f0f66700 1 --2- 172.21.2.101:0/665194107 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6ec195650 0x7fd6ec0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:40.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.740 7fd6f0f66700 1 -- 172.21.2.101:0/665194107 >> 172.21.2.101:0/665194107 conn(0x7fd6ec0b3040 msgr2=0x7fd6ec0c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:40.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.740 7fd6f0f66700 1 -- 172.21.2.101:0/665194107 shutdown_connections 2021-02-10T11:42:40.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:40.740 7fd6f0f66700 1 -- 172.21.2.101:0/665194107 wait complete. 2021-02-10T11:42:40.951 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:42:30.762357', 'modified': '2021-02-10 11:42:32.772467', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7816}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7816': {'gid': 7816, 'name': 'b', 'rank': 0, 'incarnation': 102, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/381011050', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 381011050}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 381011050}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:42:40.951 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:42:40.951 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:42:41.351 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:42:41.351 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:42:41.351 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:42:41.351 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 7816, 2021-02-10T11:42:41.351 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:42:41.352 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:42:41.352 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 10.584169254000001, 2021-02-10T11:42:41.352 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 105, 2021-02-10T11:42:41.352 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 129, 2021-02-10T11:42:41.352 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 129, 2021-02-10T11:42:41.352 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 16.24489239 2021-02-10T11:42:41.352 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:42:41.355 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 7816, "want_state": "up:active", "state": "up:active", "rank_uptime": 10.584169254000001, "mdsmap_epoch": 105, "osdmap_epoch": 129, "osdmap_epoch_barrier": 129, "uptime": 16.24489239 } 2021-02-10T11:42:41.355 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:42:41.355 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-02-10T11:42:41.355 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-02-10T11:42:41.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:41.356 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:42:41.691 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:42:42.036 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:42:42.036 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:42:42.379 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:42:42.379 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:42:42.380 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:42:42.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:42.381 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:42:42.715 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-02-10T11:42:42.716 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:42:42.718 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:42:43.063 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:42:43.063 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:42:43.129 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[155120]: starting ceph client 2021-02-10T11:42:43.130 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:2021-02-10 11:42:43.128 7f3b646e3f80 -1 init, newargv = 0x55570739d220 newargc=9 2021-02-10T11:42:43.135 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[155120]: starting fuse 2021-02-10T11:42:43.408 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:42:43.408 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:42:43.408 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:42:43.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:43.409 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:42:43.745 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:42:43.745 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2021-02-10T11:42:43.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:43.746 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:42:43.746 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:42:43.746 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:42:43.746 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:42:43.746 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:42:43.746 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:42:43.746 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:42:43.746 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-02-10T11:42:43.746 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:42:43.746 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:42:43.746 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:42:43.747 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:42:43.747 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:42:43.747 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:42:43.747 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:42:43.747 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:42:43.747 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:42:43.747 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:42:43.747 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:42:43.747 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:42:43.747 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:42:43.747 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.0")) 2021-02-10T11:42:43.747 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:42:44.108 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.0.155120.asok 2021-02-10T11:42:44.111 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.155120.asok 2021-02-10T11:42:44.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:44.111 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.155120.asok status 2021-02-10T11:42:44.502 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:42:44.503 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:42:44.503 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:42:44.503 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:42:44.503 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "0", 2021-02-10T11:42:44.503 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:42:44.503 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-02-10T11:42:44.503 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "155120", 2021-02-10T11:42:44.503 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:42:44.503 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:44.503 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:42:44.503 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:42:44.503 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 11316, 2021-02-10T11:42:44.504 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:42:44.504 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:42:44.504 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:42:44.504 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 11316 2021-02-10T11:42:44.504 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:44.504 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:42:44.504 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:44.504 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:42:44.504 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 663941955 2021-02-10T11:42:44.504 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:44.505 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:44.505 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:42:44.505 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:44.505 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:42:44.505 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 663941955 2021-02-10T11:42:44.505 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:44.505 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.11316 172.21.2.101:0/663941955", 2021-02-10T11:42:44.505 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/663941955", 2021-02-10T11:42:44.505 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:42:44.505 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 105, 2021-02-10T11:42:44.505 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 129, 2021-02-10T11:42:44.505 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:42:44.505 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:42:44.506 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:42:44.507 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:44.507 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:42:44.507 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:42:44.843 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:42:44.844 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:42:44.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:44.844 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:42:45.192 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:42:45.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.602 7f3f4d96a700 1 Processor -- start 2021-02-10T11:42:45.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.602 7f3f4d96a700 1 -- start start 2021-02-10T11:42:45.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.602 7f3f4d96a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3f480d1870 0x7f3f481b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:45.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.602 7f3f4d96a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f481b0a10 0x7f3f480cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:45.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.602 7f3f4d96a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f480f0b10 con 0x7f3f480d1c70 2021-02-10T11:42:45.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.602 7f3f4d96a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f480e5ed0 con 0x7f3f480d1870 2021-02-10T11:42:45.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.602 7f3f4d96a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f480f0650 con 0x7f3f481b0a10 2021-02-10T11:42:45.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.602 7f3f47d75700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f3f480d1c70 0x7f3f480cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58870/0 (socket says 172.21.2.101:58870) 2021-02-10T11:42:45.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.602 7f3f47d75700 1 -- 172.21.2.101:0/1645013526 learned_addr learned my addr 172.21.2.101:0/1645013526 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:45.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.603 7f3f46572700 1 -- 172.21.2.101:0/1645013526 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2818919876 0 0) 0x7f3f40000f90 con 0x7f3f480d1c70 2021-02-10T11:42:45.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.603 7f3f46572700 1 -- 172.21.2.101:0/1645013526 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f30002fe0 con 0x7f3f480d1c70 2021-02-10T11:42:45.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.603 7f3f47574700 1 --2- 172.21.2.101:0/1645013526 >> v2:172.21.2.108:3300/0 conn(0x7f3f480d1870 0x7f3f481b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:45.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.603 7f3f46d73700 1 --2- 172.21.2.101:0/1645013526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f481b0a10 0x7f3f480cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:45.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.603 7f3f46572700 1 -- 172.21.2.101:0/1645013526 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 80613646 0 0) 0x7f3f40001460 con 0x7f3f480d1c70 2021-02-10T11:42:45.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.603 7f3f46572700 1 -- 172.21.2.101:0/1645013526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f481b0a10 msgr2=0x7f3f480cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:45.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.603 7f3f46572700 1 --2- 172.21.2.101:0/1645013526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f481b0a10 0x7f3f480cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:45.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.603 7f3f46572700 1 -- 172.21.2.101:0/1645013526 >> v2:172.21.2.108:3300/0 conn(0x7f3f480d1870 msgr2=0x7f3f481b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:45.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.603 7f3f46572700 1 --2- 172.21.2.101:0/1645013526 >> v2:172.21.2.108:3300/0 conn(0x7f3f480d1870 0x7f3f481b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:45.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.603 7f3f46572700 1 -- 172.21.2.101:0/1645013526 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f480f0870 con 0x7f3f480d1c70 2021-02-10T11:42:45.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.603 7f3f46572700 1 -- 172.21.2.101:0/1645013526 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f3f40001980 con 0x7f3f480d1c70 2021-02-10T11:42:45.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.603 7f3f46572700 1 -- 172.21.2.101:0/1645013526 >> v1:172.21.2.101:6789/0 conn(0x7f3f480d1c70 legacy=0x7f3f480cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:45.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.603 7f3f46572700 1 --2- 172.21.2.101:0/1645013526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f30003af0 0x7f3f30005f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:45.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.603 7f3f46572700 1 --2- 172.21.2.101:0/1645013526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3f3004a7b0 0x7f3f3004cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:45.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.603 7f3f46572700 1 --2- 172.21.2.101:0/1645013526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f3004d590 0x7f3f3004fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:45.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.603 7f3f46572700 1 -- 172.21.2.101:0/1645013526 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f300502d0 con 0x7f3f30003af0 2021-02-10T11:42:45.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.603 7f3f46572700 1 -- 172.21.2.101:0/1645013526 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f300504f0 con 0x7f3f3004a7b0 2021-02-10T11:42:45.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.603 7f3f46572700 1 -- 172.21.2.101:0/1645013526 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f30050710 con 0x7f3f3004d590 2021-02-10T11:42:45.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.603 7f3f47574700 1 --2- 172.21.2.101:0/1645013526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f30003af0 0x7f3f30005f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:45.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.604 7f3f47d75700 1 --2- 172.21.2.101:0/1645013526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f3004d590 0x7f3f3004fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:45.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.604 7f3f46d73700 1 --2- 172.21.2.101:0/1645013526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3f3004a7b0 0x7f3f3004cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:45.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.604 7f3f47574700 1 -- 172.21.2.101:0/1645013526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f3004d590 msgr2=0x7f3f3004fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:45.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.604 7f3f47574700 1 --2- 172.21.2.101:0/1645013526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f3004d590 0x7f3f3004fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:45.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.604 7f3f47574700 1 -- 172.21.2.101:0/1645013526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3f3004a7b0 msgr2=0x7f3f3004cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:45.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.604 7f3f47574700 1 --2- 172.21.2.101:0/1645013526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3f3004a7b0 0x7f3f3004cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:45.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.604 7f3f47574700 1 -- 172.21.2.101:0/1645013526 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f300509b0 con 0x7f3f30003af0 2021-02-10T11:42:45.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.604 7f3f47574700 1 --2- 172.21.2.101:0/1645013526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f30003af0 0x7f3f30005f00 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 rx=0x7f3f3c00c8f0 tx=0x7f3f3c00cc90).ready entity=mon.0 client_cookie=cb22fbc69ef6bbbc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:45.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.604 7f3f46572700 1 -- 172.21.2.101:0/1645013526 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3f3c00cdd0 con 0x7f3f30003af0 2021-02-10T11:42:45.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.604 7f3f46572700 1 -- 172.21.2.101:0/1645013526 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3c009310 con 0x7f3f30003af0 2021-02-10T11:42:45.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.604 7f3f46572700 1 -- 172.21.2.101:0/1645013526 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3f3c0097f0 con 0x7f3f30003af0 2021-02-10T11:42:45.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.604 7f3f4d96a700 1 -- 172.21.2.101:0/1645013526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f30003af0 msgr2=0x7f3f30005f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:45.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.604 7f3f4d96a700 1 --2- 172.21.2.101:0/1645013526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f30003af0 0x7f3f30005f00 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 rx=0x7f3f3c00c8f0 tx=0x7f3f3c00cc90).stop 2021-02-10T11:42:45.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.604 7f3f47d75700 1 -- 172.21.2.101:0/1645013526 reap_dead start 2021-02-10T11:42:45.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.605 7f3f4d96a700 1 -- 172.21.2.101:0/1645013526 shutdown_connections 2021-02-10T11:42:45.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.605 7f3f4d96a700 1 -- 172.21.2.101:0/1645013526 >> 172.21.2.101:0/1645013526 conn(0x7f3f480ce450 msgr2=0x7f3f480d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:45.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.605 7f3f4d96a700 1 -- 172.21.2.101:0/1645013526 shutdown_connections 2021-02-10T11:42:45.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.605 7f3f4d96a700 1 -- 172.21.2.101:0/1645013526 wait complete. 2021-02-10T11:42:45.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.620 7f3f4d96a700 1 Processor -- start 2021-02-10T11:42:45.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.620 7f3f4d96a700 1 -- start start 2021-02-10T11:42:45.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.620 7f3f4d96a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3f481942b0 0x7f3f480b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:45.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.620 7f3f4d96a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f48195070 0x7f3f48195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:45.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.620 7f3f4d96a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f480bb8e0 0x7f3f480bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:45.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.620 7f3f4d96a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f481946b0 con 0x7f3f480bb8e0 2021-02-10T11:42:45.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.620 7f3f4d96a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f480b3b70 con 0x7f3f481942b0 2021-02-10T11:42:45.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.620 7f3f4d96a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f480b3d90 con 0x7f3f48195070 2021-02-10T11:42:45.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.620 7f3f47d75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f480bb8e0 0x7f3f480bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:45.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.620 7f3f47d75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f480bb8e0 0x7f3f480bbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58388/0 (socket says 172.21.2.101:58388) 2021-02-10T11:42:45.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.620 7f3f47d75700 1 -- 172.21.2.101:0/2197522044 learned_addr learned my addr 172.21.2.101:0/2197522044 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:45.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.620 7f3f47574700 1 --2- 172.21.2.101:0/2197522044 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3f481942b0 0x7f3f480b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:45.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.621 7f3f46d73700 1 --2- 172.21.2.101:0/2197522044 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f48195070 0x7f3f48195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:45.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.621 7f3f47d75700 1 -- 172.21.2.101:0/2197522044 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f48195070 msgr2=0x7f3f48195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:45.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.621 7f3f47d75700 1 --2- 172.21.2.101:0/2197522044 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f48195070 0x7f3f48195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:45.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.621 7f3f47d75700 1 -- 172.21.2.101:0/2197522044 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3f481942b0 msgr2=0x7f3f480b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:45.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.621 7f3f47d75700 1 --2- 172.21.2.101:0/2197522044 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3f481942b0 0x7f3f480b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:45.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.621 7f3f47d75700 1 -- 172.21.2.101:0/2197522044 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f481a49e0 con 0x7f3f480bb8e0 2021-02-10T11:42:45.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.621 7f3f47d75700 1 --2- 172.21.2.101:0/2197522044 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f480bb8e0 0x7f3f480bbce0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 rx=0x7f3f4000a8f0 tx=0x7f3f4000ac90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:45.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.621 7f3f2ffff700 1 -- 172.21.2.101:0/2197522044 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3f40000dc0 con 0x7f3f480bb8e0 2021-02-10T11:42:45.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.621 7f3f2ffff700 1 -- 172.21.2.101:0/2197522044 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f400070d0 con 0x7f3f480bb8e0 2021-02-10T11:42:45.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.621 7f3f2ffff700 1 -- 172.21.2.101:0/2197522044 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3f40007530 con 0x7f3f480bb8e0 2021-02-10T11:42:45.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.621 7f3f4d96a700 1 -- 172.21.2.101:0/2197522044 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f481a49e0 con 0x7f3f480bb8e0 2021-02-10T11:42:45.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.621 7f3f4d96a700 1 -- 172.21.2.101:0/2197522044 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f481a49e0 con 0x7f3f480bb8e0 2021-02-10T11:42:45.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.622 7f3f2ffff700 1 -- 172.21.2.101:0/2197522044 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3f40001750 con 0x7f3f480bb8e0 2021-02-10T11:42:45.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.622 7f3f2ffff700 1 --2- 172.21.2.101:0/2197522044 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f28018730 0x7f3f2801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:45.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.622 7f3f2ffff700 1 -- 172.21.2.101:0/2197522044 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3f40032b50 con 0x7f3f480bb8e0 2021-02-10T11:42:45.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.622 7f3f47574700 1 --2- 172.21.2.101:0/2197522044 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f28018730 0x7f3f2801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:45.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.622 7f3f47574700 1 --2- 172.21.2.101:0/2197522044 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f28018730 0x7f3f2801abc0 crc :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:45.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.652 7f3f4d96a700 1 -- 172.21.2.101:0/2197522044 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f480bc5f0 con 0x7f3f480bb8e0 2021-02-10T11:42:45.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.655 7f3f2ffff700 1 -- 172.21.2.101:0/2197522044 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3f40001bc0 con 0x7f3f480bb8e0 2021-02-10T11:42:45.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.780 7f3f4d96a700 1 -- 172.21.2.101:0/2197522044 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f480bc5f0 con 0x7f3f480bb8e0 2021-02-10T11:42:45.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.780 7f3f2ffff700 1 -- 172.21.2.101:0/2197522044 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f3f40001750 con 0x7f3f480bb8e0 2021-02-10T11:42:45.783 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:42:45.783 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":129,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:42:31.764498","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:42:28.726448","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2021-02-10 11:42:29.754869","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:42:45.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.782 7f3f4d96a700 1 -- 172.21.2.101:0/2197522044 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f28018730 msgr2=0x7f3f2801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:45.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.782 7f3f4d96a700 1 --2- 172.21.2.101:0/2197522044 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f28018730 0x7f3f2801abc0 crc :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:45.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.782 7f3f4d96a700 1 -- 172.21.2.101:0/2197522044 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f480bb8e0 msgr2=0x7f3f480bbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:45.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.782 7f3f4d96a700 1 --2- 172.21.2.101:0/2197522044 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f480bb8e0 0x7f3f480bbce0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 rx=0x7f3f4000a8f0 tx=0x7f3f4000ac90).stop 2021-02-10T11:42:45.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.782 7f3f4d96a700 1 -- 172.21.2.101:0/2197522044 shutdown_connections 2021-02-10T11:42:45.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.782 7f3f4d96a700 1 --2- 172.21.2.101:0/2197522044 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f28018730 0x7f3f2801abc0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:45.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.782 7f3f4d96a700 1 --2- 172.21.2.101:0/2197522044 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f48195070 0x7f3f48195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:45.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.782 7f3f4d96a700 1 --2- 172.21.2.101:0/2197522044 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f480bb8e0 0x7f3f480bbce0 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:45.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.782 7f3f4d96a700 1 --2- 172.21.2.101:0/2197522044 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3f481942b0 0x7f3f480b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:45.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.782 7f3f4d96a700 1 -- 172.21.2.101:0/2197522044 >> 172.21.2.101:0/2197522044 conn(0x7f3f480b3040 msgr2=0x7f3f480c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:45.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.782 7f3f4d96a700 1 -- 172.21.2.101:0/2197522044 shutdown_connections 2021-02-10T11:42:45.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:45.782 7f3f4d96a700 1 -- 172.21.2.101:0/2197522044 wait complete. 2021-02-10T11:42:45.792 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:42:28.726448', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:42:29.754869', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:42:45.793 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:42:46.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.210 7fd7ea80b700 1 Processor -- start 2021-02-10T11:42:46.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.210 7fd7ea80b700 1 -- start start 2021-02-10T11:42:46.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.211 7fd7ea80b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd7e40d1c70 0x7fd7e40cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:46.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.211 7fd7ea80b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e40d0170 con 0x7fd7e40cf580 2021-02-10T11:42:46.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.211 7fd7ea80b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e40fe1c0 con 0x7fd7e40d1870 2021-02-10T11:42:46.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.211 7fd7ea80b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e41b0cb0 con 0x7fd7e40d1c70 2021-02-10T11:42:46.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.211 7fd7e8e28700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd7e40cf580 0x7fd7e40cf980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58890/0 (socket says 172.21.2.101:58890) 2021-02-10T11:42:46.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.211 7fd7e8e28700 1 -- 172.21.2.101:0/569127021 learned_addr learned my addr 172.21.2.101:0/569127021 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:46.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.211 7fd7e2ffd700 1 -- 172.21.2.101:0/569127021 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 462918508 0 0) 0x7fd7dc001500 con 0x7fd7e40cf580 2021-02-10T11:42:46.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.211 7fd7e2ffd700 1 -- 172.21.2.101:0/569127021 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7cc002fe0 con 0x7fd7e40cf580 2021-02-10T11:42:46.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.211 7fd7e2ffd700 1 -- 172.21.2.101:0/569127021 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1891481226 0 0) 0x7fd7dc001270 con 0x7fd7e40cf580 2021-02-10T11:42:46.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.211 7fd7e2ffd700 1 -- 172.21.2.101:0/569127021 >> v2:172.21.2.108:3300/0 conn(0x7fd7e40d1c70 msgr2=0x7fd7e40cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:46.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.211 7fd7e2ffd700 1 --2- 172.21.2.101:0/569127021 >> v2:172.21.2.108:3300/0 conn(0x7fd7e40d1c70 0x7fd7e40cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:46.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.211 7fd7e2ffd700 1 -- 172.21.2.101:0/569127021 >> v1:172.21.2.108:6789/0 conn(0x7fd7e40d1870 legacy=0x7fd7e41b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:46.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.211 7fd7e2ffd700 1 -- 172.21.2.101:0/569127021 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e4109a60 con 0x7fd7e40cf580 2021-02-10T11:42:46.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.211 7fd7e2ffd700 1 -- 172.21.2.101:0/569127021 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fd7dc001b50 con 0x7fd7e40cf580 2021-02-10T11:42:46.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.212 7fd7e2ffd700 1 -- 172.21.2.101:0/569127021 >> v1:172.21.2.101:6789/0 conn(0x7fd7e40cf580 legacy=0x7fd7e40cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:46.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.212 7fd7e2ffd700 1 --2- 172.21.2.101:0/569127021 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7cc003ac0 0x7fd7cc005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:46.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.212 7fd7e2ffd700 1 --2- 172.21.2.101:0/569127021 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7cc006800 0x7fd7cc008cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:46.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.212 7fd7e2ffd700 1 --2- 172.21.2.101:0/569127021 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7cc0095b0 0x7fd7cc00bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:46.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.212 7fd7e2ffd700 1 -- 172.21.2.101:0/569127021 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc00c300 con 0x7fd7cc0095b0 2021-02-10T11:42:46.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.212 7fd7e2ffd700 1 -- 172.21.2.101:0/569127021 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc00c520 con 0x7fd7cc003ac0 2021-02-10T11:42:46.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.212 7fd7e2ffd700 1 -- 172.21.2.101:0/569127021 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc00c740 con 0x7fd7cc006800 2021-02-10T11:42:46.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.212 7fd7e8e28700 1 --2- 172.21.2.101:0/569127021 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7cc0095b0 0x7fd7cc00bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:46.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.212 7fd7e3fff700 1 --2- 172.21.2.101:0/569127021 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7cc003ac0 0x7fd7cc005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:46.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.212 7fd7e37fe700 1 --2- 172.21.2.101:0/569127021 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7cc006800 0x7fd7cc008cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:46.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.212 7fd7e8e28700 1 -- 172.21.2.101:0/569127021 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7cc006800 msgr2=0x7fd7cc008cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:46.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.212 7fd7e8e28700 1 --2- 172.21.2.101:0/569127021 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7cc006800 0x7fd7cc008cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:46.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.212 7fd7e8e28700 1 -- 172.21.2.101:0/569127021 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7cc003ac0 msgr2=0x7fd7cc005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:46.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.212 7fd7e8e28700 1 --2- 172.21.2.101:0/569127021 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7cc003ac0 0x7fd7cc005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:46.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.212 7fd7e8e28700 1 -- 172.21.2.101:0/569127021 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc00c9e0 con 0x7fd7cc0095b0 2021-02-10T11:42:46.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.212 7fd7e8e28700 1 -- 172.21.2.101:0/569127021 reap_dead start 2021-02-10T11:42:46.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.212 7fd7e8e28700 1 --2- 172.21.2.101:0/569127021 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7cc0095b0 0x7fd7cc00bad0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 rx=0x7fd7dc007cb0 tx=0x7fd7dc050310).ready entity=mon.0 client_cookie=5e284f22d827304c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:46.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.212 7fd7e2ffd700 1 -- 172.21.2.101:0/569127021 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd7dc007d40 con 0x7fd7cc0095b0 2021-02-10T11:42:46.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.212 7fd7e2ffd700 1 -- 172.21.2.101:0/569127021 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7dc007020 con 0x7fd7cc0095b0 2021-02-10T11:42:46.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.212 7fd7e2ffd700 1 -- 172.21.2.101:0/569127021 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd7dc0074c0 con 0x7fd7cc0095b0 2021-02-10T11:42:46.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.212 7fd7ea80b700 1 -- 172.21.2.101:0/569127021 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7cc0095b0 msgr2=0x7fd7cc00bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:46.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.212 7fd7ea80b700 1 --2- 172.21.2.101:0/569127021 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7cc0095b0 0x7fd7cc00bad0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 rx=0x7fd7dc007cb0 tx=0x7fd7dc050310).stop 2021-02-10T11:42:46.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.213 7fd7ea80b700 1 -- 172.21.2.101:0/569127021 shutdown_connections 2021-02-10T11:42:46.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.213 7fd7ea80b700 1 --2- 172.21.2.101:0/569127021 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7cc0095b0 0x7fd7cc00bad0 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:46.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.213 7fd7ea80b700 1 -- 172.21.2.101:0/569127021 >> 172.21.2.101:0/569127021 conn(0x7fd7e40ce420 msgr2=0x7fd7e40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:46.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.213 7fd7ea80b700 1 -- 172.21.2.101:0/569127021 shutdown_connections 2021-02-10T11:42:46.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.213 7fd7ea80b700 1 -- 172.21.2.101:0/569127021 wait complete. 2021-02-10T11:42:46.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.228 7fd7ea80b700 1 Processor -- start 2021-02-10T11:42:46.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.228 7fd7ea80b700 1 -- start start 2021-02-10T11:42:46.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.228 7fd7ea80b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7e409bcc0 0x7fd7e409c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:46.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.228 7fd7ea80b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7e40a0820 0x7fd7e40a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:46.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.228 7fd7ea80b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7e40a1240 0x7fd7e40858d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:46.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.228 7fd7ea80b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e40a66b0 con 0x7fd7e40a1240 2021-02-10T11:42:46.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.228 7fd7ea80b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e41a49b0 con 0x7fd7e40a0820 2021-02-10T11:42:46.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.228 7fd7ea80b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e41a4b90 con 0x7fd7e409bcc0 2021-02-10T11:42:46.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.228 7fd7e8e28700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7e40a1240 0x7fd7e40858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:46.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.228 7fd7e8e28700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7e40a1240 0x7fd7e40858d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58410/0 (socket says 172.21.2.101:58410) 2021-02-10T11:42:46.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.228 7fd7e8e28700 1 -- 172.21.2.101:0/3353067654 learned_addr learned my addr 172.21.2.101:0/3353067654 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:46.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.229 7fd7e8e28700 1 -- 172.21.2.101:0/3353067654 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7e409bcc0 msgr2=0x7fd7e409c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:46.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.229 7fd7e8e28700 1 --2- 172.21.2.101:0/3353067654 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7e409bcc0 0x7fd7e409c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:46.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.229 7fd7e8e28700 1 -- 172.21.2.101:0/3353067654 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7e40a0820 msgr2=0x7fd7e40a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:46.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.229 7fd7e8e28700 1 --2- 172.21.2.101:0/3353067654 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7e40a0820 0x7fd7e40a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:46.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.229 7fd7e8e28700 1 -- 172.21.2.101:0/3353067654 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e40a1640 con 0x7fd7e40a1240 2021-02-10T11:42:46.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.229 7fd7e8e28700 1 --2- 172.21.2.101:0/3353067654 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7e40a1240 0x7fd7e40858d0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 rx=0x7fd7dc007ce0 tx=0x7fd7dc051c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:46.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.229 7fd7e0ff9700 1 -- 172.21.2.101:0/3353067654 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd7dc007280 con 0x7fd7e40a1240 2021-02-10T11:42:46.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.229 7fd7e0ff9700 1 -- 172.21.2.101:0/3353067654 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7dc007500 con 0x7fd7e40a1240 2021-02-10T11:42:46.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.229 7fd7ea80b700 1 -- 172.21.2.101:0/3353067654 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e40a1640 con 0x7fd7e40a1240 2021-02-10T11:42:46.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.229 7fd7ea80b700 1 -- 172.21.2.101:0/3353067654 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e40861e0 con 0x7fd7e40a1240 2021-02-10T11:42:46.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.229 7fd7e0ff9700 1 -- 172.21.2.101:0/3353067654 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd7dc009350 con 0x7fd7e40a1240 2021-02-10T11:42:46.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.230 7fd7e0ff9700 1 -- 172.21.2.101:0/3353067654 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd7dc007020 con 0x7fd7e40a1240 2021-02-10T11:42:46.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.230 7fd7e0ff9700 1 --2- 172.21.2.101:0/3353067654 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7c40186c0 0x7fd7c401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:46.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.230 7fd7e0ff9700 1 -- 172.21.2.101:0/3353067654 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd7dc076b70 con 0x7fd7e40a1240 2021-02-10T11:42:46.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.230 7fd7e3fff700 1 --2- 172.21.2.101:0/3353067654 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7c40186c0 0x7fd7c401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:46.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.230 7fd7e3fff700 1 --2- 172.21.2.101:0/3353067654 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7c40186c0 0x7fd7c401abc0 crc :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:46.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.261 7fd7ea80b700 1 -- 172.21.2.101:0/3353067654 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e4086a70 con 0x7fd7e40a1240 2021-02-10T11:42:46.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.263 7fd7e0ff9700 1 -- 172.21.2.101:0/3353067654 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd7dc0015d0 con 0x7fd7e40a1240 2021-02-10T11:42:46.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.391 7fd7ea80b700 1 -- 172.21.2.101:0/3353067654 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fd7e4086a70 con 0x7fd7e40a1240 2021-02-10T11:42:46.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.392 7fd7e0ff9700 1 -- 172.21.2.101:0/3353067654 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v129) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd7dc051d80 con 0x7fd7e40a1240 2021-02-10T11:42:46.394 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:42:46.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.393 7fd7ea80b700 1 -- 172.21.2.101:0/3353067654 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7c40186c0 msgr2=0x7fd7c401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:46.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.394 7fd7ea80b700 1 --2- 172.21.2.101:0/3353067654 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7c40186c0 0x7fd7c401abc0 crc :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:46.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.394 7fd7ea80b700 1 -- 172.21.2.101:0/3353067654 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7e40a1240 msgr2=0x7fd7e40858d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:46.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.394 7fd7ea80b700 1 --2- 172.21.2.101:0/3353067654 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7e40a1240 0x7fd7e40858d0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 rx=0x7fd7dc007ce0 tx=0x7fd7dc051c40).stop 2021-02-10T11:42:46.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.394 7fd7ea80b700 1 -- 172.21.2.101:0/3353067654 shutdown_connections 2021-02-10T11:42:46.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.394 7fd7ea80b700 1 --2- 172.21.2.101:0/3353067654 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7e40a1240 0x7fd7e40858d0 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:46.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.394 7fd7ea80b700 1 --2- 172.21.2.101:0/3353067654 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7e40a0820 0x7fd7e40a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:46.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.394 7fd7ea80b700 1 --2- 172.21.2.101:0/3353067654 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7c40186c0 0x7fd7c401abc0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:46.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.394 7fd7ea80b700 1 --2- 172.21.2.101:0/3353067654 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7e409bcc0 0x7fd7e409c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:46.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.394 7fd7ea80b700 1 -- 172.21.2.101:0/3353067654 >> 172.21.2.101:0/3353067654 conn(0x7fd7e40bef00 msgr2=0x7fd7e409d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:46.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.394 7fd7ea80b700 1 -- 172.21.2.101:0/3353067654 shutdown_connections 2021-02-10T11:42:46.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.394 7fd7ea80b700 1 -- 172.21.2.101:0/3353067654 wait complete. 2021-02-10T11:42:46.404 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:42:46.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.819 7f602d241700 1 Processor -- start 2021-02-10T11:42:46.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.819 7f602d241700 1 -- start start 2021-02-10T11:42:46.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.819 7f602d241700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f60280d1870 0x7f60281b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:46.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.819 7f602d241700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f60281b0ae0 0x7f6028108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:46.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.819 7f602d241700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f60280d1c70 0x7f6028108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:46.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.819 7f602d241700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60280e5ef0 con 0x7f60280d1c70 2021-02-10T11:42:46.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.819 7f602d241700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6028109a60 con 0x7f60281b0ae0 2021-02-10T11:42:46.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.819 7f602d241700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6028088070 con 0x7f60280d1870 2021-02-10T11:42:46.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.820 7f602759e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f60280d1c70 0x7f6028108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:46.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.820 7f602759e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f60280d1c70 0x7f6028108f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58418/0 (socket says 172.21.2.101:58418) 2021-02-10T11:42:46.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.820 7f602759e700 1 -- 172.21.2.101:0/1777441650 learned_addr learned my addr 172.21.2.101:0/1777441650 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:46.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.820 7f602759e700 1 -- 172.21.2.101:0/1777441650 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f60280d1870 msgr2=0x7f60281b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:46.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.820 7f602759e700 1 --2- 172.21.2.101:0/1777441650 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f60280d1870 0x7f60281b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:46.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.820 7f602759e700 1 -- 172.21.2.101:0/1777441650 >> v2:172.21.2.108:3300/0 conn(0x7f60281b0ae0 msgr2=0x7f6028108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:46.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.820 7f602759e700 1 --2- 172.21.2.101:0/1777441650 >> v2:172.21.2.108:3300/0 conn(0x7f60281b0ae0 0x7f6028108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:46.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.820 7f602759e700 1 -- 172.21.2.101:0/1777441650 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6028109580 con 0x7f60280d1c70 2021-02-10T11:42:46.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.820 7f602759e700 1 --2- 172.21.2.101:0/1777441650 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60280d1c70 0x7f6028108f60 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 rx=0x7f6020006f00 tx=0x7f6020009f10).ready entity=mon.0 client_cookie=d1993a32ff16533a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:46.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.820 7f6025d9b700 1 -- 172.21.2.101:0/1777441650 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f602000b3b0 con 0x7f60280d1c70 2021-02-10T11:42:46.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.820 7f6025d9b700 1 -- 172.21.2.101:0/1777441650 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6020000d30 con 0x7f60280d1c70 2021-02-10T11:42:46.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.820 7f6025d9b700 1 -- 172.21.2.101:0/1777441650 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f602000b820 con 0x7f60280d1c70 2021-02-10T11:42:46.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.820 7f602d241700 1 -- 172.21.2.101:0/1777441650 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60280d1c70 msgr2=0x7f6028108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:46.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.820 7f602d241700 1 --2- 172.21.2.101:0/1777441650 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60280d1c70 0x7f6028108f60 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 rx=0x7f6020006f00 tx=0x7f6020009f10).stop 2021-02-10T11:42:46.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.821 7f602d241700 1 -- 172.21.2.101:0/1777441650 shutdown_connections 2021-02-10T11:42:46.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.821 7f602d241700 1 --2- 172.21.2.101:0/1777441650 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60280d1c70 0x7f6028108f60 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:46.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.821 7f602d241700 1 --2- 172.21.2.101:0/1777441650 >> v2:172.21.2.108:3300/0 conn(0x7f60281b0ae0 0x7f6028108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:46.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.821 7f602d241700 1 --2- 172.21.2.101:0/1777441650 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f60280d1870 0x7f60281b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:46.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.821 7f602d241700 1 -- 172.21.2.101:0/1777441650 >> 172.21.2.101:0/1777441650 conn(0x7f60280ce420 msgr2=0x7f60280d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:46.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.821 7f602d241700 1 -- 172.21.2.101:0/1777441650 shutdown_connections 2021-02-10T11:42:46.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.821 7f602d241700 1 -- 172.21.2.101:0/1777441650 wait complete. 2021-02-10T11:42:46.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.835 7f602d241700 1 Processor -- start 2021-02-10T11:42:46.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.835 7f602d241700 1 -- start start 2021-02-10T11:42:46.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.836 7f602d241700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f60281a7bf0 0x7f60281a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:46.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.836 7f602d241700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6028195560 0x7f6028195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:46.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.836 7f602d241700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6028195f80 0x7f6028196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:46.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.836 7f602d241700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6028194fe0 con 0x7f6028195560 2021-02-10T11:42:46.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.836 7f602d241700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6028193020 con 0x7f60281a7bf0 2021-02-10T11:42:46.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.836 7f602d241700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6028193440 con 0x7f6028195f80 2021-02-10T11:42:46.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.836 7f602659c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6028195560 0x7f6028195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:46.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.836 7f602659c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6028195560 0x7f6028195960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58424/0 (socket says 172.21.2.101:58424) 2021-02-10T11:42:46.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.836 7f602659c700 1 -- 172.21.2.101:0/2881674441 learned_addr learned my addr 172.21.2.101:0/2881674441 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:46.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.836 7f602759e700 1 --2- 172.21.2.101:0/2881674441 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6028195f80 0x7f6028196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:46.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.836 7f6026d9d700 1 --2- 172.21.2.101:0/2881674441 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f60281a7bf0 0x7f60281a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:46.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.837 7f602659c700 1 -- 172.21.2.101:0/2881674441 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6028195f80 msgr2=0x7f6028196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:46.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.837 7f602659c700 1 --2- 172.21.2.101:0/2881674441 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6028195f80 0x7f6028196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:46.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.837 7f602659c700 1 -- 172.21.2.101:0/2881674441 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f60281a7bf0 msgr2=0x7f60281a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:46.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.837 7f602659c700 1 --2- 172.21.2.101:0/2881674441 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f60281a7bf0 0x7f60281a7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:46.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.837 7f602659c700 1 -- 172.21.2.101:0/2881674441 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6028193660 con 0x7f6028195560 2021-02-10T11:42:46.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.837 7f602659c700 1 --2- 172.21.2.101:0/2881674441 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6028195560 0x7f6028195960 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 rx=0x7f601000b530 tx=0x7f601000b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:46.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.837 7f601f7fe700 1 -- 172.21.2.101:0/2881674441 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f601000bcc0 con 0x7f6028195560 2021-02-10T11:42:46.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.837 7f601f7fe700 1 -- 172.21.2.101:0/2881674441 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6010000c90 con 0x7f6028195560 2021-02-10T11:42:46.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.837 7f602d241700 1 -- 172.21.2.101:0/2881674441 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6028193660 con 0x7f6028195560 2021-02-10T11:42:46.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.837 7f601f7fe700 1 -- 172.21.2.101:0/2881674441 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f60100022a0 con 0x7f6028195560 2021-02-10T11:42:46.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.837 7f602d241700 1 -- 172.21.2.101:0/2881674441 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6028193660 con 0x7f6028195560 2021-02-10T11:42:46.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.838 7f601f7fe700 1 -- 172.21.2.101:0/2881674441 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6010005310 con 0x7f6028195560 2021-02-10T11:42:46.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.838 7f601f7fe700 1 --2- 172.21.2.101:0/2881674441 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f600c018730 0x7f600c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:46.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.838 7f601f7fe700 1 -- 172.21.2.101:0/2881674441 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6010032610 con 0x7f6028195560 2021-02-10T11:42:46.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.838 7f6026d9d700 1 --2- 172.21.2.101:0/2881674441 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f600c018730 0x7f600c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:46.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.838 7f6026d9d700 1 --2- 172.21.2.101:0/2881674441 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f600c018730 0x7f600c01ac10 crc :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:46.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.869 7f602d241700 1 -- 172.21.2.101:0/2881674441 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6028082980 con 0x7f6028195560 2021-02-10T11:42:46.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:46.872 7f601f7fe700 1 -- 172.21.2.101:0/2881674441 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6010006b90 con 0x7f6028195560 2021-02-10T11:42:47.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.002 7f602d241700 1 -- 172.21.2.101:0/2881674441 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f6028109a60 con 0x7f6028195560 2021-02-10T11:42:47.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.002 7f601f7fe700 1 -- 172.21.2.101:0/2881674441 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v129) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6010005310 con 0x7f6028195560 2021-02-10T11:42:47.004 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:42:47.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.004 7f602d241700 1 -- 172.21.2.101:0/2881674441 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f600c018730 msgr2=0x7f600c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:47.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.004 7f602d241700 1 --2- 172.21.2.101:0/2881674441 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f600c018730 0x7f600c01ac10 crc :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:47.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.004 7f602d241700 1 -- 172.21.2.101:0/2881674441 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6028195560 msgr2=0x7f6028195960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:47.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.004 7f602d241700 1 --2- 172.21.2.101:0/2881674441 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6028195560 0x7f6028195960 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 rx=0x7f601000b530 tx=0x7f601000b8d0).stop 2021-02-10T11:42:47.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.004 7f602d241700 1 -- 172.21.2.101:0/2881674441 shutdown_connections 2021-02-10T11:42:47.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.004 7f602d241700 1 --2- 172.21.2.101:0/2881674441 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f600c018730 0x7f600c01ac10 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.004 7f602d241700 1 --2- 172.21.2.101:0/2881674441 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6028195f80 0x7f6028196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.004 7f602d241700 1 --2- 172.21.2.101:0/2881674441 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6028195560 0x7f6028195960 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.004 7f602d241700 1 --2- 172.21.2.101:0/2881674441 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f60281a7bf0 0x7f60281a7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.004 7f602d241700 1 -- 172.21.2.101:0/2881674441 >> 172.21.2.101:0/2881674441 conn(0x7f60280a3270 msgr2=0x7f6028095970 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.004 7f602d241700 1 -- 172.21.2.101:0/2881674441 shutdown_connections 2021-02-10T11:42:47.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.004 7f602d241700 1 -- 172.21.2.101:0/2881674441 wait complete. 2021-02-10T11:42:47.014 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:42:47.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.428 7f71c3f04700 1 Processor -- start 2021-02-10T11:42:47.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.428 7f71c3f04700 1 -- start start 2021-02-10T11:42:47.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.428 7f71c3f04700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f71bc10aff0 0x7f71bc0cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:47.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.428 7f71c3f04700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71bc0fe1c0 con 0x7f71bc0d1870 2021-02-10T11:42:47.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.428 7f71c3f04700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71bc1b0cb0 con 0x7f71bc0d1c70 2021-02-10T11:42:47.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.428 7f71c3f04700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71bc109a60 con 0x7f71bc10aff0 2021-02-10T11:42:47.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.429 7f71c2521700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f71bc10aff0 0x7f71bc0cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:47.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.429 7f71c2521700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f71bc10aff0 0x7f71bc0cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58432/0 (socket says 172.21.2.101:58432) 2021-02-10T11:42:47.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.429 7f71c2521700 1 -- 172.21.2.101:0/3684496348 learned_addr learned my addr 172.21.2.101:0/3684496348 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:47.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.429 7f71c2521700 1 -- 172.21.2.101:0/3684496348 >> v1:172.21.2.108:6789/0 conn(0x7f71bc0d1c70 legacy=0x7f71bc0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:47.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.429 7f71c2521700 1 -- 172.21.2.101:0/3684496348 >> v1:172.21.2.101:6789/0 conn(0x7f71bc0d1870 legacy=0x7f71bc1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:47.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.429 7f71c0d1e700 1 -- 172.21.2.101:0/3684496348 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695807176 0 0) 0x7f71b8000ff0 con 0x7f71bc0d1870 2021-02-10T11:42:47.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.429 7f71c2521700 1 -- 172.21.2.101:0/3684496348 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc10b3f0 con 0x7f71bc10aff0 2021-02-10T11:42:47.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.429 7f71c2521700 1 --2- 172.21.2.101:0/3684496348 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71bc10aff0 0x7f71bc0cf750 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 rx=0x7f71b4006f00 tx=0x7f71b4009f10).ready entity=mon.0 client_cookie=f36d636f5786228 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:47.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.430 7f71c0d1e700 1 -- 172.21.2.101:0/3684496348 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f71b400b3b0 con 0x7f71bc10aff0 2021-02-10T11:42:47.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.430 7f71c0d1e700 1 -- 172.21.2.101:0/3684496348 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b400b3b0 con 0x7f71bc10aff0 2021-02-10T11:42:47.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.430 7f71c0d1e700 1 -- 172.21.2.101:0/3684496348 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f71b400b630 con 0x7f71bc10aff0 2021-02-10T11:42:47.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.430 7f71c3f04700 1 -- 172.21.2.101:0/3684496348 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71bc10aff0 msgr2=0x7f71bc0cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:47.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.430 7f71c3f04700 1 --2- 172.21.2.101:0/3684496348 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71bc10aff0 0x7f71bc0cf750 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 rx=0x7f71b4006f00 tx=0x7f71b4009f10).stop 2021-02-10T11:42:47.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.430 7f71c3f04700 1 -- 172.21.2.101:0/3684496348 shutdown_connections 2021-02-10T11:42:47.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.430 7f71c3f04700 1 --2- 172.21.2.101:0/3684496348 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71bc10aff0 0x7f71bc0cf750 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:47.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.430 7f71c3f04700 1 -- 172.21.2.101:0/3684496348 >> 172.21.2.101:0/3684496348 conn(0x7f71bc0ce420 msgr2=0x7f71bc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:47.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.431 7f71c3f04700 1 -- 172.21.2.101:0/3684496348 shutdown_connections 2021-02-10T11:42:47.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.431 7f71c3f04700 1 -- 172.21.2.101:0/3684496348 wait complete. 2021-02-10T11:42:47.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.445 7f71c3f04700 1 Processor -- start 2021-02-10T11:42:47.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.445 7f71c3f04700 1 -- start start 2021-02-10T11:42:47.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.446 7f71c3f04700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f71bc0c3820 0x7f71bc084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:47.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.446 7f71c3f04700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71bc0a0290 0x7f71bc0852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:47.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.446 7f71c3f04700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71bc086270 0x7f71bc086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:47.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.446 7f71c3f04700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc0c0700 con 0x7f71bc0a0290 2021-02-10T11:42:47.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.446 7f71c3f04700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc0858f0 con 0x7f71bc0c3820 2021-02-10T11:42:47.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.446 7f71c3f04700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc085b10 con 0x7f71bc086270 2021-02-10T11:42:47.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.446 7f71c151f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71bc0a0290 0x7f71bc0852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:47.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.446 7f71c151f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71bc0a0290 0x7f71bc0852d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58434/0 (socket says 172.21.2.101:58434) 2021-02-10T11:42:47.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.446 7f71c151f700 1 -- 172.21.2.101:0/2106433975 learned_addr learned my addr 172.21.2.101:0/2106433975 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:47.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.446 7f71c2521700 1 --2- 172.21.2.101:0/2106433975 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71bc086270 0x7f71bc086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:47.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.446 7f71c1d20700 1 --2- 172.21.2.101:0/2106433975 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f71bc0c3820 0x7f71bc084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:47.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.446 7f71c151f700 1 -- 172.21.2.101:0/2106433975 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71bc086270 msgr2=0x7f71bc086670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:47.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.446 7f71c151f700 1 --2- 172.21.2.101:0/2106433975 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71bc086270 0x7f71bc086670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:47.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.446 7f71c151f700 1 -- 172.21.2.101:0/2106433975 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f71bc0c3820 msgr2=0x7f71bc084cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:47.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.446 7f71c151f700 1 --2- 172.21.2.101:0/2106433975 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f71bc0c3820 0x7f71bc084cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:47.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.446 7f71c151f700 1 -- 172.21.2.101:0/2106433975 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc086c90 con 0x7f71bc0a0290 2021-02-10T11:42:47.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.446 7f71c1d20700 1 --2- 172.21.2.101:0/2106433975 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f71bc0c3820 0x7f71bc084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:42:47.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.446 7f71c2521700 1 --2- 172.21.2.101:0/2106433975 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71bc086270 0x7f71bc086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:42:47.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.446 7f71c151f700 1 --2- 172.21.2.101:0/2106433975 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71bc0a0290 0x7f71bc0852d0 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 rx=0x7f71b0006ed0 tx=0x7f71b000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:47.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.446 7f71ae7fc700 1 -- 172.21.2.101:0/2106433975 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f71b0006020 con 0x7f71bc0a0290 2021-02-10T11:42:47.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.447 7f71ae7fc700 1 -- 172.21.2.101:0/2106433975 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b00009b0 con 0x7f71bc0a0290 2021-02-10T11:42:47.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.447 7f71ae7fc700 1 -- 172.21.2.101:0/2106433975 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f71b00054b0 con 0x7f71bc0a0290 2021-02-10T11:42:47.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.447 7f71c3f04700 1 -- 172.21.2.101:0/2106433975 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71bc086c90 con 0x7f71bc0a0290 2021-02-10T11:42:47.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.447 7f71c3f04700 1 -- 172.21.2.101:0/2106433975 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71bc086c90 con 0x7f71bc0a0290 2021-02-10T11:42:47.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.447 7f71ae7fc700 1 -- 172.21.2.101:0/2106433975 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f71b0006230 con 0x7f71bc0a0290 2021-02-10T11:42:47.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.447 7f71ae7fc700 1 --2- 172.21.2.101:0/2106433975 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f71a40186c0 0x7f71a401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:47.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.447 7f71ae7fc700 1 -- 172.21.2.101:0/2106433975 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f71b0032570 con 0x7f71bc0a0290 2021-02-10T11:42:47.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.447 7f71c1d20700 1 --2- 172.21.2.101:0/2106433975 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f71a40186c0 0x7f71a401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:47.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.448 7f71c1d20700 1 --2- 172.21.2.101:0/2106433975 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f71a40186c0 0x7f71a401abc0 crc :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:47.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.478 7f71c3f04700 1 -- 172.21.2.101:0/2106433975 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71bc082a70 con 0x7f71bc0a0290 2021-02-10T11:42:47.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.481 7f71ae7fc700 1 -- 172.21.2.101:0/2106433975 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f71b0006000 con 0x7f71bc0a0290 2021-02-10T11:42:47.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.608 7f71c3f04700 1 -- 172.21.2.101:0/2106433975 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f71bc082a70 con 0x7f71bc0a0290 2021-02-10T11:42:47.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.608 7f71ae7fc700 1 -- 172.21.2.101:0/2106433975 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v129) v1 ==== 100+0+10 (secure 0 0 0) 0x7f71b0006210 con 0x7f71bc0a0290 2021-02-10T11:42:47.610 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:42:47.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.610 7f71c3f04700 1 -- 172.21.2.101:0/2106433975 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f71a40186c0 msgr2=0x7f71a401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:47.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.610 7f71c3f04700 1 --2- 172.21.2.101:0/2106433975 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f71a40186c0 0x7f71a401abc0 crc :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:47.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.610 7f71c3f04700 1 -- 172.21.2.101:0/2106433975 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71bc0a0290 msgr2=0x7f71bc0852d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:47.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.610 7f71c3f04700 1 --2- 172.21.2.101:0/2106433975 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71bc0a0290 0x7f71bc0852d0 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 rx=0x7f71b0006ed0 tx=0x7f71b000b960).stop 2021-02-10T11:42:47.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.610 7f71c3f04700 1 -- 172.21.2.101:0/2106433975 shutdown_connections 2021-02-10T11:42:47.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.610 7f71c3f04700 1 --2- 172.21.2.101:0/2106433975 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f71a40186c0 0x7f71a401abc0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:47.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.610 7f71c3f04700 1 --2- 172.21.2.101:0/2106433975 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71bc086270 0x7f71bc086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:47.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.610 7f71c3f04700 1 --2- 172.21.2.101:0/2106433975 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71bc0a0290 0x7f71bc0852d0 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:47.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.610 7f71c3f04700 1 --2- 172.21.2.101:0/2106433975 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f71bc0c3820 0x7f71bc084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:47.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.610 7f71c3f04700 1 -- 172.21.2.101:0/2106433975 >> 172.21.2.101:0/2106433975 conn(0x7f71bc0872c0 msgr2=0x7f71bc095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:47.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.611 7f71c3f04700 1 -- 172.21.2.101:0/2106433975 shutdown_connections 2021-02-10T11:42:47.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:47.611 7f71c3f04700 1 -- 172.21.2.101:0/2106433975 wait complete. 2021-02-10T11:42:47.621 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:42:47.621 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:42:48.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.031 7f0249d00700 1 Processor -- start 2021-02-10T11:42:48.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.031 7f0249d00700 1 -- start start 2021-02-10T11:42:48.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.032 7f0249d00700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f02440f05f0 0x7f02440f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:48.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.032 7f0249d00700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02441b0c70 con 0x7f02441b0870 2021-02-10T11:42:48.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.032 7f0249d00700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02440f0b50 con 0x7f02441b0470 2021-02-10T11:42:48.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.032 7f0249d00700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02440e5ea0 con 0x7f02440f05f0 2021-02-10T11:42:48.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.032 7f0242ffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f02441b0870 0x7f02440efe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58936/0 (socket says 172.21.2.101:58936) 2021-02-10T11:42:48.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.032 7f0242ffd700 1 -- 172.21.2.101:0/3199699213 learned_addr learned my addr 172.21.2.101:0/3199699213 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:48.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.032 7f02427fc700 1 -- 172.21.2.101:0/3199699213 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2091402849 0 0) 0x7f0234001180 con 0x7f02441b0870 2021-02-10T11:42:48.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.032 7f02427fc700 1 -- 172.21.2.101:0/3199699213 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0230002fe0 con 0x7f02441b0870 2021-02-10T11:42:48.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.032 7f0243fff700 1 --2- 172.21.2.101:0/3199699213 >> v2:172.21.2.108:3300/0 conn(0x7f02440f05f0 0x7f02440f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:48.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f02427fc700 1 -- 172.21.2.101:0/3199699213 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3461679416 0 0) 0x7f0234001180 con 0x7f02441b0870 2021-02-10T11:42:48.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f02427fc700 1 -- 172.21.2.101:0/3199699213 >> v2:172.21.2.108:3300/0 conn(0x7f02440f05f0 msgr2=0x7f02440f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:48.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f02427fc700 1 --2- 172.21.2.101:0/3199699213 >> v2:172.21.2.108:3300/0 conn(0x7f02440f05f0 0x7f02440f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:48.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f02427fc700 1 -- 172.21.2.101:0/3199699213 >> v1:172.21.2.108:6789/0 conn(0x7f02441b0470 legacy=0x7f02440ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:48.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f02427fc700 1 -- 172.21.2.101:0/3199699213 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02440e52e0 con 0x7f02441b0870 2021-02-10T11:42:48.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f02427fc700 1 -- 172.21.2.101:0/3199699213 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f0234001680 con 0x7f02441b0870 2021-02-10T11:42:48.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f02427fc700 1 -- 172.21.2.101:0/3199699213 >> v1:172.21.2.101:6789/0 conn(0x7f02441b0870 legacy=0x7f02440efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:48.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f02427fc700 1 --2- 172.21.2.101:0/3199699213 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0230003af0 0x7f0230005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:48.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f02427fc700 1 --2- 172.21.2.101:0/3199699213 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0230006790 0x7f0230008cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:48.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f02427fc700 1 --2- 172.21.2.101:0/3199699213 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02300095b0 0x7f023000bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:48.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f02427fc700 1 -- 172.21.2.101:0/3199699213 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023000c300 con 0x7f0230006790 2021-02-10T11:42:48.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f02427fc700 1 -- 172.21.2.101:0/3199699213 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023000c520 con 0x7f02300095b0 2021-02-10T11:42:48.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f02427fc700 1 -- 172.21.2.101:0/3199699213 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f023000c740 con 0x7f0230003af0 2021-02-10T11:42:48.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f0242ffd700 1 --2- 172.21.2.101:0/3199699213 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0230006790 0x7f0230008cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:48.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f02437fe700 1 --2- 172.21.2.101:0/3199699213 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0230003af0 0x7f0230005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:48.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f0243fff700 1 --2- 172.21.2.101:0/3199699213 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02300095b0 0x7f023000bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:48.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f0243fff700 1 -- 172.21.2.101:0/3199699213 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0230003af0 msgr2=0x7f0230005f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:48.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f0243fff700 1 --2- 172.21.2.101:0/3199699213 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0230003af0 0x7f0230005f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:48.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f0243fff700 1 -- 172.21.2.101:0/3199699213 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0230006790 msgr2=0x7f0230008cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:48.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f0243fff700 1 --2- 172.21.2.101:0/3199699213 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0230006790 0x7f0230008cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:48.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f0243fff700 1 -- 172.21.2.101:0/3199699213 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023000c9e0 con 0x7f02300095b0 2021-02-10T11:42:48.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f0242ffd700 1 --2- 172.21.2.101:0/3199699213 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0230006790 0x7f0230008cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:42:48.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f0243fff700 1 -- 172.21.2.101:0/3199699213 reap_dead start 2021-02-10T11:42:48.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.033 7f0243fff700 1 --2- 172.21.2.101:0/3199699213 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02300095b0 0x7f023000bab0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f023c0068d0 tx=0x7f023c00b050).ready entity=mon.1 client_cookie=ca00e6cada5ff760 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:48.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.034 7f02427fc700 1 -- 172.21.2.101:0/3199699213 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f023c00b440 con 0x7f02300095b0 2021-02-10T11:42:48.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.034 7f02427fc700 1 -- 172.21.2.101:0/3199699213 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023c003030 con 0x7f02300095b0 2021-02-10T11:42:48.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.034 7f02427fc700 1 -- 172.21.2.101:0/3199699213 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f023c006b40 con 0x7f02300095b0 2021-02-10T11:42:48.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.034 7f0249d00700 1 -- 172.21.2.101:0/3199699213 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02300095b0 msgr2=0x7f023000bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:48.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.034 7f0249d00700 1 --2- 172.21.2.101:0/3199699213 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02300095b0 0x7f023000bab0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f023c0068d0 tx=0x7f023c00b050).stop 2021-02-10T11:42:48.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.034 7f0249d00700 1 -- 172.21.2.101:0/3199699213 shutdown_connections 2021-02-10T11:42:48.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.034 7f0249d00700 1 --2- 172.21.2.101:0/3199699213 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02300095b0 0x7f023000bab0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:48.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.034 7f0249d00700 1 -- 172.21.2.101:0/3199699213 >> 172.21.2.101:0/3199699213 conn(0x7f02440ce3e0 msgr2=0x7f02440e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:48.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.034 7f0249d00700 1 -- 172.21.2.101:0/3199699213 shutdown_connections 2021-02-10T11:42:48.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.034 7f0249d00700 1 -- 172.21.2.101:0/3199699213 wait complete. 2021-02-10T11:42:48.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.049 7f0249d00700 1 Processor -- start 2021-02-10T11:42:48.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.049 7f0249d00700 1 -- start start 2021-02-10T11:42:48.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.049 7f0249d00700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02440bba80 0x7f0244108e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:48.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.049 7f0249d00700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02440bbe80 0x7f02440bd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:48.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.049 7f0249d00700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02441094b0 0x7f02440bdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:48.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.049 7f0249d00700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02441089f0 con 0x7f02441094b0 2021-02-10T11:42:48.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.049 7f0249d00700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02440c2ec0 con 0x7f02440bbe80 2021-02-10T11:42:48.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.049 7f0249d00700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02440c30e0 con 0x7f02440bba80 2021-02-10T11:42:48.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.050 7f0243fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02441094b0 0x7f02440bdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:48.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.050 7f0243fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02441094b0 0x7f02440bdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58452/0 (socket says 172.21.2.101:58452) 2021-02-10T11:42:48.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.050 7f0243fff700 1 -- 172.21.2.101:0/3171451090 learned_addr learned my addr 172.21.2.101:0/3171451090 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:48.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.050 7f0243fff700 1 -- 172.21.2.101:0/3171451090 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02440bba80 msgr2=0x7f0244108e90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:42:48.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.050 7f0243fff700 1 --2- 172.21.2.101:0/3171451090 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02440bba80 0x7f0244108e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:48.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.050 7f0243fff700 1 -- 172.21.2.101:0/3171451090 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02440bbe80 msgr2=0x7f02440bd5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:42:48.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.050 7f0243fff700 1 --2- 172.21.2.101:0/3171451090 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02440bbe80 0x7f02440bd5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:48.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.050 7f0243fff700 1 -- 172.21.2.101:0/3171451090 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02440be200 con 0x7f02441094b0 2021-02-10T11:42:48.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.050 7f0243fff700 1 --2- 172.21.2.101:0/3171451090 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02441094b0 0x7f02440bdbe0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 rx=0x7f023c006e00 tx=0x7f023c00bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:48.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.050 7f023bfff700 1 -- 172.21.2.101:0/3171451090 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f023c009900 con 0x7f02441094b0 2021-02-10T11:42:48.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.050 7f023bfff700 1 -- 172.21.2.101:0/3171451090 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023c0022e0 con 0x7f02441094b0 2021-02-10T11:42:48.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.050 7f0249d00700 1 -- 172.21.2.101:0/3171451090 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02440be200 con 0x7f02441094b0 2021-02-10T11:42:48.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.050 7f0249d00700 1 -- 172.21.2.101:0/3171451090 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02440be200 con 0x7f02441094b0 2021-02-10T11:42:48.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.050 7f023bfff700 1 -- 172.21.2.101:0/3171451090 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f023c002730 con 0x7f02441094b0 2021-02-10T11:42:48.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.051 7f023bfff700 1 -- 172.21.2.101:0/3171451090 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f023c009900 con 0x7f02441094b0 2021-02-10T11:42:48.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.051 7f023bfff700 1 --2- 172.21.2.101:0/3171451090 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0228018730 0x7f022801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:48.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.051 7f023bfff700 1 -- 172.21.2.101:0/3171451090 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f023c032bd0 con 0x7f02441094b0 2021-02-10T11:42:48.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.051 7f02437fe700 1 --2- 172.21.2.101:0/3171451090 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0228018730 0x7f022801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:48.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.051 7f02437fe700 1 --2- 172.21.2.101:0/3171451090 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0228018730 0x7f022801ac10 crc :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:48.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.081 7f0249d00700 1 -- 172.21.2.101:0/3171451090 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02440f0b50 con 0x7f02441094b0 2021-02-10T11:42:48.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.084 7f023bfff700 1 -- 172.21.2.101:0/3171451090 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f023c01f910 con 0x7f02441094b0 2021-02-10T11:42:48.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.210 7f0249d00700 1 -- 172.21.2.101:0/3171451090 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02440f0b50 con 0x7f02441094b0 2021-02-10T11:42:48.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.211 7f023bfff700 1 -- 172.21.2.101:0/3171451090 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+2105 (secure 0 0 0) 0x7f023c0098c0 con 0x7f02441094b0 2021-02-10T11:42:48.213 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:42:48.213 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":105,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11139,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1647083555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1647083555},{"type":"v1","addr":"172.21.2.101:6835","nonce":1647083555}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":101}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:42:30.762357","modified":"2021-02-10 11:42:32.772467","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7816":{"gid":7816,"name":"b","rank":0,"incarnation":102,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/381011050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":381011050},{"type":"v1","addr":"172.21.2.108:6833","nonce":381011050}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-02-10T11:42:48.213 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 105 2021-02-10T11:42:48.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.213 7f0249d00700 1 -- 172.21.2.101:0/3171451090 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0228018730 msgr2=0x7f022801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:48.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.213 7f0249d00700 1 --2- 172.21.2.101:0/3171451090 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0228018730 0x7f022801ac10 crc :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:48.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.213 7f0249d00700 1 -- 172.21.2.101:0/3171451090 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02441094b0 msgr2=0x7f02440bdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:48.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.213 7f0249d00700 1 --2- 172.21.2.101:0/3171451090 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02441094b0 0x7f02440bdbe0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 rx=0x7f023c006e00 tx=0x7f023c00bd40).stop 2021-02-10T11:42:48.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.213 7f0249d00700 1 -- 172.21.2.101:0/3171451090 shutdown_connections 2021-02-10T11:42:48.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.213 7f0249d00700 1 --2- 172.21.2.101:0/3171451090 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02441094b0 0x7f02440bdbe0 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:48.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.213 7f0249d00700 1 --2- 172.21.2.101:0/3171451090 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02440bbe80 0x7f02440bd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:48.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.213 7f0249d00700 1 --2- 172.21.2.101:0/3171451090 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0228018730 0x7f022801ac10 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:48.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.213 7f0249d00700 1 --2- 172.21.2.101:0/3171451090 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02440bba80 0x7f0244108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:48.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.213 7f0249d00700 1 -- 172.21.2.101:0/3171451090 >> 172.21.2.101:0/3171451090 conn(0x7f02440bc6c0 msgr2=0x7f02440b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:48.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.213 7f0249d00700 1 -- 172.21.2.101:0/3171451090 shutdown_connections 2021-02-10T11:42:48.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:48.213 7f0249d00700 1 -- 172.21.2.101:0/3171451090 wait complete. 2021-02-10T11:42:48.424 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:42:30.762357', 'modified': '2021-02-10 11:42:32.772467', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7816}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7816': {'gid': 7816, 'name': 'b', 'rank': 0, 'incarnation': 102, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/381011050', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 381011050}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 381011050}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:42:48.424 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:42:48.425 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:42:48.822 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:42:48.822 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:42:48.822 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:42:48.823 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 7816, 2021-02-10T11:42:48.823 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:42:48.823 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:42:48.823 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 18.055094374999999, 2021-02-10T11:42:48.823 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 105, 2021-02-10T11:42:48.823 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 129, 2021-02-10T11:42:48.823 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 129, 2021-02-10T11:42:48.823 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 23.715817421000001 2021-02-10T11:42:48.823 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:42:48.826 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 7816, "want_state": "up:active", "state": "up:active", "rank_uptime": 18.055094374999999, "mdsmap_epoch": 105, "osdmap_epoch": 129, "osdmap_epoch_barrier": 129, "uptime": 23.715817421000001 } 2021-02-10T11:42:48.826 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:42:48.827 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-02-10T11:42:48.827 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:42:48.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:48.827 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:42:49.163 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:42:49.509 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:42:49.509 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:42:49.853 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:42:49.853 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:42:49.854 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:42:49.854 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:49.854 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:42:50.189 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:42:50.189 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2021-02-10T11:42:50.190 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:42:50.193 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:42:50.537 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:42:50.537 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:42:50.604 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:2021-02-10 11:42:50.603 7f096348af80 -1 init, newargv = 0x561f487ee1f0 newargc=9 2021-02-10T11:42:50.604 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[155760]: starting ceph client 2021-02-10T11:42:50.609 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[155760]: starting fuse 2021-02-10T11:42:50.882 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:42:50.882 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:42:50.883 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:42:50.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:50.883 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:42:51.219 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:42:51.219 INFO:teuthology.orchestra.run.gibba001.stdout:44 2021-02-10T11:42:51.219 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2021-02-10T11:42:51.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:51.219 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:42:51.220 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:42:51.220 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:42:51.220 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:42:51.220 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:42:51.220 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:42:51.220 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:42:51.220 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-02-10T11:42:51.220 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:42:51.220 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:42:51.220 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:42:51.220 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:42:51.220 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:42:51.220 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:42:51.220 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:42:51.221 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:42:51.221 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:42:51.221 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:42:51.221 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:42:51.221 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:42:51.221 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:42:51.221 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.1")) 2021-02-10T11:42:51.221 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:42:51.583 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.1.155760.asok 2021-02-10T11:42:51.586 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.155760.asok 2021-02-10T11:42:51.586 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:51.586 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.155760.asok status 2021-02-10T11:42:51.976 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:42:51.977 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:42:51.977 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:42:51.977 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:42:51.977 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "1", 2021-02-10T11:42:51.977 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:42:51.977 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:42:51.977 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "155760", 2021-02-10T11:42:51.977 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:42:51.977 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:51.977 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:42:51.977 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:42:51.977 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 11355, 2021-02-10T11:42:51.978 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:42:51.978 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:42:51.978 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:42:51.978 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 11355 2021-02-10T11:42:51.978 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:51.978 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:42:51.978 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:51.978 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:42:51.978 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 492675513 2021-02-10T11:42:51.978 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:51.978 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:51.978 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:42:51.978 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:42:51.979 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:42:51.979 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 492675513 2021-02-10T11:42:51.979 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:42:51.979 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.11355 172.21.2.101:0/492675513", 2021-02-10T11:42:51.979 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/492675513", 2021-02-10T11:42:51.979 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:42:51.979 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 105, 2021-02-10T11:42:51.979 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 129, 2021-02-10T11:42:51.979 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:42:51.979 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:42:51.979 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:42:51.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:51.981 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:42:51.981 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:42:52.317 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:42:52.317 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:42:52.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:52.317 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:42:52.665 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2021-02-10T11:42:53.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.080 7f9b67474700 1 Processor -- start 2021-02-10T11:42:53.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.080 7f9b67474700 1 -- start start 2021-02-10T11:42:53.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.081 7f9b67474700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f9b600d1c80 0x7f9b600cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:53.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.081 7f9b67474700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b600cf590 0x7f9b600cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:53.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.081 7f9b67474700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b600fe1d0 con 0x7f9b600d1880 2021-02-10T11:42:53.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.081 7f9b67474700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b601b0cc0 con 0x7f9b600d1c80 2021-02-10T11:42:53.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.081 7f9b67474700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b60109a70 con 0x7f9b600cf590 2021-02-10T11:42:53.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.081 7f9b65a91700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b600cf590 0x7f9b600cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:53.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.081 7f9b65290700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f9b600d1880 0x7f9b601b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:52378/0 (socket says 172.21.2.101:52378) 2021-02-10T11:42:53.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.081 7f9b65290700 1 -- 172.21.2.101:0/705613844 learned_addr learned my addr 172.21.2.101:0/705613844 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:53.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.081 7f9b64a8f700 1 --2- 172.21.2.101:0/705613844 >> v2:172.21.2.108:3300/0 conn(0x7f9b600d1c80 0x7f9b600cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:53.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.082 7f9b57fff700 1 -- 172.21.2.101:0/705613844 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2553806442 0 0) 0x7f9b5c0010b0 con 0x7f9b600d1880 2021-02-10T11:42:53.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.082 7f9b65a91700 1 -- 172.21.2.101:0/705613844 >> v2:172.21.2.108:3300/0 conn(0x7f9b600d1c80 msgr2=0x7f9b600cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:53.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.082 7f9b65a91700 1 --2- 172.21.2.101:0/705613844 >> v2:172.21.2.108:3300/0 conn(0x7f9b600d1c80 0x7f9b600cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:53.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.082 7f9b65a91700 1 -- 172.21.2.101:0/705613844 >> v1:172.21.2.108:6789/0 conn(0x7f9b600d1880 legacy=0x7f9b601b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:53.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.082 7f9b65a91700 1 -- 172.21.2.101:0/705613844 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b60088070 con 0x7f9b600cf590 2021-02-10T11:42:53.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.082 7f9b64a8f700 1 --2- 172.21.2.101:0/705613844 >> v2:172.21.2.108:3300/0 conn(0x7f9b600d1c80 0x7f9b600cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:42:53.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.082 7f9b65a91700 1 --2- 172.21.2.101:0/705613844 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b600cf590 0x7f9b600cf990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f9b58006e50 tx=0x7f9b58009e70).ready entity=mon.2 client_cookie=1ca5203c473aa606 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:53.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.082 7f9b57fff700 1 -- 172.21.2.101:0/705613844 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9b5800b510 con 0x7f9b600cf590 2021-02-10T11:42:53.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.082 7f9b57fff700 1 -- 172.21.2.101:0/705613844 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b58006790 con 0x7f9b600cf590 2021-02-10T11:42:53.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.082 7f9b57fff700 1 -- 172.21.2.101:0/705613844 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9b58000f90 con 0x7f9b600cf590 2021-02-10T11:42:53.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.083 7f9b67474700 1 -- 172.21.2.101:0/705613844 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b600cf590 msgr2=0x7f9b600cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:53.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.083 7f9b67474700 1 --2- 172.21.2.101:0/705613844 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b600cf590 0x7f9b600cf990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f9b58006e50 tx=0x7f9b58009e70).stop 2021-02-10T11:42:53.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.083 7f9b67474700 1 -- 172.21.2.101:0/705613844 shutdown_connections 2021-02-10T11:42:53.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.083 7f9b67474700 1 --2- 172.21.2.101:0/705613844 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b600cf590 0x7f9b600cf990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:53.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.083 7f9b67474700 1 --2- 172.21.2.101:0/705613844 >> v2:172.21.2.108:3300/0 conn(0x7f9b600d1c80 0x7f9b600cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:53.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.083 7f9b67474700 1 -- 172.21.2.101:0/705613844 >> 172.21.2.101:0/705613844 conn(0x7f9b600ce430 msgr2=0x7f9b600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:53.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.084 7f9b67474700 1 -- 172.21.2.101:0/705613844 shutdown_connections 2021-02-10T11:42:53.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.084 7f9b67474700 1 -- 172.21.2.101:0/705613844 wait complete. 2021-02-10T11:42:53.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.098 7f9b67474700 1 Processor -- start 2021-02-10T11:42:53.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.098 7f9b67474700 1 -- start start 2021-02-10T11:42:53.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.099 7f9b67474700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b601a05e0 0x7f9b6009a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:53.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.099 7f9b67474700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b6009a8b0 0x7f9b6009acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:53.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.099 7f9b67474700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b6009b2d0 0x7f9b600beed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:53.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.099 7f9b67474700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b60099d50 con 0x7f9b6009a8b0 2021-02-10T11:42:53.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.099 7f9b67474700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b600bf4f0 con 0x7f9b6009b2d0 2021-02-10T11:42:53.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.099 7f9b67474700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b600bf6e0 con 0x7f9b601a05e0 2021-02-10T11:42:53.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.099 7f9b64a8f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b6009a8b0 0x7f9b6009acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:53.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.099 7f9b64a8f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b6009a8b0 0x7f9b6009acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58482/0 (socket says 172.21.2.101:58482) 2021-02-10T11:42:53.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.099 7f9b64a8f700 1 -- 172.21.2.101:0/3070225750 learned_addr learned my addr 172.21.2.101:0/3070225750 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:53.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.099 7f9b65290700 1 --2- 172.21.2.101:0/3070225750 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b601a05e0 0x7f9b6009a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:53.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.099 7f9b65a91700 1 --2- 172.21.2.101:0/3070225750 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b6009b2d0 0x7f9b600beed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:53.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.099 7f9b64a8f700 1 -- 172.21.2.101:0/3070225750 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b601a05e0 msgr2=0x7f9b6009a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:53.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.099 7f9b64a8f700 1 --2- 172.21.2.101:0/3070225750 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b601a05e0 0x7f9b6009a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:53.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.099 7f9b64a8f700 1 -- 172.21.2.101:0/3070225750 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b6009b2d0 msgr2=0x7f9b600beed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:53.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.099 7f9b64a8f700 1 --2- 172.21.2.101:0/3070225750 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b6009b2d0 0x7f9b600beed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:53.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.099 7f9b64a8f700 1 -- 172.21.2.101:0/3070225750 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b600bf900 con 0x7f9b6009a8b0 2021-02-10T11:42:53.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.099 7f9b64a8f700 1 --2- 172.21.2.101:0/3070225750 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b6009a8b0 0x7f9b6009acb0 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 rx=0x7f9b50006f40 tx=0x7f9b5000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:53.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.100 7f9b55ffb700 1 -- 172.21.2.101:0/3070225750 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9b50006bd0 con 0x7f9b6009a8b0 2021-02-10T11:42:53.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.100 7f9b55ffb700 1 -- 172.21.2.101:0/3070225750 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5000bd80 con 0x7f9b6009a8b0 2021-02-10T11:42:53.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.100 7f9b67474700 1 -- 172.21.2.101:0/3070225750 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b600bf900 con 0x7f9b6009a8b0 2021-02-10T11:42:53.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.100 7f9b55ffb700 1 -- 172.21.2.101:0/3070225750 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9b50002220 con 0x7f9b6009a8b0 2021-02-10T11:42:53.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.100 7f9b67474700 1 -- 172.21.2.101:0/3070225750 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b600bfe90 con 0x7f9b6009a8b0 2021-02-10T11:42:53.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.100 7f9b55ffb700 1 -- 172.21.2.101:0/3070225750 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9b50000c90 con 0x7f9b6009a8b0 2021-02-10T11:42:53.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.100 7f9b55ffb700 1 --2- 172.21.2.101:0/3070225750 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b400186d0 0x7f9b4001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:53.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.100 7f9b55ffb700 1 -- 172.21.2.101:0/3070225750 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9b50032650 con 0x7f9b6009a8b0 2021-02-10T11:42:53.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.100 7f9b65290700 1 --2- 172.21.2.101:0/3070225750 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b400186d0 0x7f9b4001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:53.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.101 7f9b65290700 1 --2- 172.21.2.101:0/3070225750 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b400186d0 0x7f9b4001abd0 crc :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:53.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.130 7f9b67474700 1 -- 172.21.2.101:0/3070225750 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b60099d50 con 0x7f9b6009a8b0 2021-02-10T11:42:53.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.133 7f9b55ffb700 1 -- 172.21.2.101:0/3070225750 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9b500009b0 con 0x7f9b6009a8b0 2021-02-10T11:42:53.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.271 7f9b67474700 1 -- 172.21.2.101:0/3070225750 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9b60099d50 con 0x7f9b6009a8b0 2021-02-10T11:42:53.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.590 7f9b55ffb700 1 -- 172.21.2.101:0/3070225750 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v106) v1 ==== 136+0+0 (secure 0 0 0) 0x7f9b50000c90 con 0x7f9b6009a8b0 2021-02-10T11:42:53.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.640 7f9b67474700 1 -- 172.21.2.101:0/3070225750 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b400186d0 msgr2=0x7f9b4001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:53.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.640 7f9b67474700 1 --2- 172.21.2.101:0/3070225750 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b400186d0 0x7f9b4001abd0 crc :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:53.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.640 7f9b67474700 1 -- 172.21.2.101:0/3070225750 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b6009a8b0 msgr2=0x7f9b6009acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:53.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.640 7f9b67474700 1 --2- 172.21.2.101:0/3070225750 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b6009a8b0 0x7f9b6009acb0 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 rx=0x7f9b50006f40 tx=0x7f9b5000b990).stop 2021-02-10T11:42:53.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.640 7f9b67474700 1 -- 172.21.2.101:0/3070225750 shutdown_connections 2021-02-10T11:42:53.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.640 7f9b67474700 1 --2- 172.21.2.101:0/3070225750 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b6009b2d0 0x7f9b600beed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:53.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.640 7f9b67474700 1 --2- 172.21.2.101:0/3070225750 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b6009a8b0 0x7f9b6009acb0 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:53.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.640 7f9b67474700 1 --2- 172.21.2.101:0/3070225750 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b400186d0 0x7f9b4001abd0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:53.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.640 7f9b67474700 1 --2- 172.21.2.101:0/3070225750 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b601a05e0 0x7f9b6009a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:53.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.640 7f9b67474700 1 -- 172.21.2.101:0/3070225750 >> 172.21.2.101:0/3070225750 conn(0x7f9b600a1770 msgr2=0x7f9b6009e210 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:53.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.640 7f9b67474700 1 -- 172.21.2.101:0/3070225750 shutdown_connections 2021-02-10T11:42:53.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:53.640 7f9b67474700 1 -- 172.21.2.101:0/3070225750 wait complete. 2021-02-10T11:42:53.650 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs volume ls 2021-02-10T11:42:54.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.059 7faa0b531700 1 Processor -- start 2021-02-10T11:42:54.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.059 7faa0b531700 1 -- start start 2021-02-10T11:42:54.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.059 7faa0b531700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7faa040d1870 0x7faa041b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:54.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.059 7faa0b531700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa040f0b10 con 0x7faa040f0650 2021-02-10T11:42:54.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.059 7faa0b531700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa040d0310 con 0x7faa041b0a10 2021-02-10T11:42:54.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.059 7faa0b531700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa040d0530 con 0x7faa040d1870 2021-02-10T11:42:54.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.059 7faa09b4e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7faa040f0650 0x7faa040cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58980/0 (socket says 172.21.2.101:58980) 2021-02-10T11:42:54.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.059 7faa09b4e700 1 -- 172.21.2.101:0/4208912173 learned_addr learned my addr 172.21.2.101:0/4208912173 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:54.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.060 7faa0934d700 1 --2- 172.21.2.101:0/4208912173 >> v2:172.21.2.101:3300/0 conn(0x7faa040d1870 0x7faa041b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:54.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.060 7faa0934d700 1 -- 172.21.2.101:0/4208912173 >> v1:172.21.2.108:6789/0 conn(0x7faa041b0a10 legacy=0x7faa040cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:54.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.060 7faa0934d700 1 -- 172.21.2.101:0/4208912173 >> v1:172.21.2.101:6789/0 conn(0x7faa040f0650 legacy=0x7faa040cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:54.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.060 7faa0934d700 1 -- 172.21.2.101:0/4208912173 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa040d1c70 con 0x7faa040d1870 2021-02-10T11:42:54.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.060 7faa0934d700 1 --2- 172.21.2.101:0/4208912173 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faa040d1870 0x7faa041b03f0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 rx=0x7faa00006650 tx=0x7faa00009a80).ready entity=mon.0 client_cookie=bd0ea639254e483d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:54.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.061 7fa9f7fff700 1 -- 172.21.2.101:0/4208912173 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faa0000c0c0 con 0x7faa040d1870 2021-02-10T11:42:54.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.061 7fa9f7fff700 1 -- 172.21.2.101:0/4208912173 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0000c340 con 0x7faa040d1870 2021-02-10T11:42:54.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.061 7fa9f7fff700 1 -- 172.21.2.101:0/4208912173 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faa0000c860 con 0x7faa040d1870 2021-02-10T11:42:54.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.061 7faa0b531700 1 -- 172.21.2.101:0/4208912173 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faa040d1870 msgr2=0x7faa041b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:54.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.061 7faa0b531700 1 --2- 172.21.2.101:0/4208912173 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faa040d1870 0x7faa041b03f0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 rx=0x7faa00006650 tx=0x7faa00009a80).stop 2021-02-10T11:42:54.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.061 7faa0b531700 1 -- 172.21.2.101:0/4208912173 shutdown_connections 2021-02-10T11:42:54.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.061 7faa0b531700 1 --2- 172.21.2.101:0/4208912173 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faa040d1870 0x7faa041b03f0 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:54.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.061 7faa0b531700 1 -- 172.21.2.101:0/4208912173 >> 172.21.2.101:0/4208912173 conn(0x7faa040ce450 msgr2=0x7faa040d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:54.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.061 7faa0b531700 1 -- 172.21.2.101:0/4208912173 shutdown_connections 2021-02-10T11:42:54.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.061 7faa0b531700 1 -- 172.21.2.101:0/4208912173 wait complete. 2021-02-10T11:42:54.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.076 7faa0b531700 1 Processor -- start 2021-02-10T11:42:54.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.076 7faa0b531700 1 -- start start 2021-02-10T11:42:54.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.076 7faa0b531700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faa040c3290 0x7faa04092e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:54.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.076 7faa0b531700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faa040c3690 0x7faa040bef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:54.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.076 7faa0b531700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faa04093440 0x7faa040bf550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:54.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.076 7faa0b531700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa040c2d10 con 0x7faa040c3290 2021-02-10T11:42:54.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.076 7faa0b531700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa041c24b0 con 0x7faa040c3690 2021-02-10T11:42:54.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.076 7faa0b531700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa041c2690 con 0x7faa04093440 2021-02-10T11:42:54.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.077 7faa0934d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faa040c3290 0x7faa04092e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:54.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.077 7faa0934d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faa040c3290 0x7faa04092e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58492/0 (socket says 172.21.2.101:58492) 2021-02-10T11:42:54.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.077 7faa0934d700 1 -- 172.21.2.101:0/3323963547 learned_addr learned my addr 172.21.2.101:0/3323963547 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:54.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.077 7faa0934d700 1 -- 172.21.2.101:0/3323963547 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faa04093440 msgr2=0x7faa040bf550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:42:54.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.077 7faa0934d700 1 --2- 172.21.2.101:0/3323963547 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faa04093440 0x7faa040bf550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:54.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.077 7faa0934d700 1 -- 172.21.2.101:0/3323963547 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faa040c3690 msgr2=0x7faa040bef30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:42:54.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.077 7faa0934d700 1 --2- 172.21.2.101:0/3323963547 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faa040c3690 0x7faa040bef30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:54.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.077 7faa0934d700 1 -- 172.21.2.101:0/3323963547 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa040bfb70 con 0x7faa040c3290 2021-02-10T11:42:54.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.077 7faa0934d700 1 --2- 172.21.2.101:0/3323963547 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faa040c3290 0x7faa04092e20 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 rx=0x7faa000093a0 tx=0x7faa00002670).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:54.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.077 7fa9f5ffb700 1 -- 172.21.2.101:0/3323963547 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faa00006070 con 0x7faa040c3290 2021-02-10T11:42:54.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.077 7fa9f5ffb700 1 -- 172.21.2.101:0/3323963547 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa00006ae0 con 0x7faa040c3290 2021-02-10T11:42:54.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.077 7fa9f5ffb700 1 -- 172.21.2.101:0/3323963547 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faa00007030 con 0x7faa040c3290 2021-02-10T11:42:54.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.077 7faa0b531700 1 -- 172.21.2.101:0/3323963547 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa040bfb70 con 0x7faa040c3290 2021-02-10T11:42:54.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.078 7faa0b531700 1 -- 172.21.2.101:0/3323963547 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa040c0100 con 0x7faa040c3290 2021-02-10T11:42:54.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.078 7fa9f5ffb700 1 -- 172.21.2.101:0/3323963547 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7faa00006cc0 con 0x7faa040c3290 2021-02-10T11:42:54.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.078 7fa9f5ffb700 1 --2- 172.21.2.101:0/3323963547 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa9e40186c0 0x7fa9e401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:54.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.078 7fa9f5ffb700 1 -- 172.21.2.101:0/3323963547 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7faa00032440 con 0x7faa040c3290 2021-02-10T11:42:54.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.078 7faa08b4c700 1 --2- 172.21.2.101:0/3323963547 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa9e40186c0 0x7fa9e401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:54.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.079 7faa08b4c700 1 --2- 172.21.2.101:0/3323963547 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa9e40186c0 0x7fa9e401abc0 crc :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:54.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.109 7faa0b531700 1 -- 172.21.2.101:0/3323963547 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa040c0990 con 0x7faa040c3290 2021-02-10T11:42:54.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.112 7fa9f5ffb700 1 -- 172.21.2.101:0/3323963547 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7faa000009b0 con 0x7faa040c3290 2021-02-10T11:42:54.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.245 7faa0b531700 1 -- 172.21.2.101:0/3323963547 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs volume ls", "target": ["mgr", ""]}) v1 -- 0x7faa040c0100 con 0x7fa9e40186c0 2021-02-10T11:42:54.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.246 7fa9f5ffb700 1 -- 172.21.2.101:0/3323963547 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7fa9f8006970 con 0x7fa9e40186c0 2021-02-10T11:42:54.247 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:42:54.247 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:42:54.247 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "cephfs" 2021-02-10T11:42:54.248 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:42:54.248 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:42:54.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.247 7faa0b531700 1 -- 172.21.2.101:0/3323963547 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa9e40186c0 msgr2=0x7fa9e401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:54.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.247 7faa0b531700 1 --2- 172.21.2.101:0/3323963547 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa9e40186c0 0x7fa9e401abc0 crc :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:54.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.247 7faa0b531700 1 -- 172.21.2.101:0/3323963547 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faa040c3290 msgr2=0x7faa04092e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:54.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.247 7faa0b531700 1 --2- 172.21.2.101:0/3323963547 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faa040c3290 0x7faa04092e20 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 rx=0x7faa000093a0 tx=0x7faa00002670).stop 2021-02-10T11:42:54.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.247 7faa0b531700 1 -- 172.21.2.101:0/3323963547 shutdown_connections 2021-02-10T11:42:54.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.247 7faa0b531700 1 --2- 172.21.2.101:0/3323963547 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa9e40186c0 0x7fa9e401abc0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:54.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.247 7faa0b531700 1 --2- 172.21.2.101:0/3323963547 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faa04093440 0x7faa040bf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:54.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.247 7faa0b531700 1 --2- 172.21.2.101:0/3323963547 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faa040c3690 0x7faa040bef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:54.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.247 7faa0b531700 1 --2- 172.21.2.101:0/3323963547 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faa040c3290 0x7faa04092e20 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:54.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.247 7faa0b531700 1 -- 172.21.2.101:0/3323963547 >> 172.21.2.101:0/3323963547 conn(0x7faa040f1dc0 msgr2=0x7faa040e7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:54.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.248 7faa0b531700 1 -- 172.21.2.101:0/3323963547 shutdown_connections 2021-02-10T11:42:54.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.248 7faa0b531700 1 -- 172.21.2.101:0/3323963547 wait complete. 2021-02-10T11:42:54.257 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:42:54.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.672 7f65582ee700 1 Processor -- start 2021-02-10T11:42:54.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.672 7f65582ee700 1 -- start start 2021-02-10T11:42:54.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.673 7f65582ee700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f65500d1c70 0x7f65500cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:54.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.673 7f65582ee700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65500cf580 0x7f65500cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:54.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.673 7f65582ee700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65500fe1c0 con 0x7f65500d1870 2021-02-10T11:42:54.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.673 7f65582ee700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65501b0cb0 con 0x7f65500d1c70 2021-02-10T11:42:54.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.673 7f65582ee700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6550109a60 con 0x7f65500cf580 2021-02-10T11:42:54.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.673 7f655610a700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f65500d1870 0x7f65501b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:58994/0 (socket says 172.21.2.101:58994) 2021-02-10T11:42:54.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.673 7f655610a700 1 -- 172.21.2.101:0/1561822434 learned_addr learned my addr 172.21.2.101:0/1561822434 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:54.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.673 7f6555909700 1 --2- 172.21.2.101:0/1561822434 >> v2:172.21.2.108:3300/0 conn(0x7f65500d1c70 0x7f65500cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:54.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.673 7f655690b700 1 --2- 172.21.2.101:0/1561822434 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65500cf580 0x7f65500cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:54.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.674 7f6555108700 1 -- 172.21.2.101:0/1561822434 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3843050537 0 0) 0x7f654c0010b0 con 0x7f65500d1870 2021-02-10T11:42:54.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.674 7f6555108700 1 -- 172.21.2.101:0/1561822434 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f653c002fe0 con 0x7f65500d1870 2021-02-10T11:42:54.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.674 7f6555909700 1 -- 172.21.2.101:0/1561822434 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65500cf580 msgr2=0x7f65500cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:54.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.674 7f6555909700 1 --2- 172.21.2.101:0/1561822434 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65500cf580 0x7f65500cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:54.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.674 7f6555909700 1 -- 172.21.2.101:0/1561822434 >> v1:172.21.2.101:6789/0 conn(0x7f65500d1870 legacy=0x7f65501b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:54.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.674 7f6555909700 1 -- 172.21.2.101:0/1561822434 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6550088070 con 0x7f65500d1c70 2021-02-10T11:42:54.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.674 7f6555909700 1 --2- 172.21.2.101:0/1561822434 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65500d1c70 0x7f65500cef60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f6544006650 tx=0x7f6544009a80).ready entity=mon.1 client_cookie=d94d9d80880bb808 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:54.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.674 7f6555108700 1 -- 172.21.2.101:0/1561822434 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f654400c0c0 con 0x7f65500d1c70 2021-02-10T11:42:54.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.674 7f6555108700 1 -- 172.21.2.101:0/1561822434 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654400c340 con 0x7f65500d1c70 2021-02-10T11:42:54.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.674 7f6555108700 1 -- 172.21.2.101:0/1561822434 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f654400c860 con 0x7f65500d1c70 2021-02-10T11:42:54.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.674 7f65582ee700 1 -- 172.21.2.101:0/1561822434 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65500d1c70 msgr2=0x7f65500cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:54.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.674 7f65582ee700 1 --2- 172.21.2.101:0/1561822434 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65500d1c70 0x7f65500cef60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f6544006650 tx=0x7f6544009a80).stop 2021-02-10T11:42:54.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.675 7f65582ee700 1 -- 172.21.2.101:0/1561822434 shutdown_connections 2021-02-10T11:42:54.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.675 7f65582ee700 1 --2- 172.21.2.101:0/1561822434 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65500cf580 0x7f65500cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:54.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.675 7f65582ee700 1 --2- 172.21.2.101:0/1561822434 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65500d1c70 0x7f65500cef60 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:54.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.675 7f65582ee700 1 -- 172.21.2.101:0/1561822434 >> 172.21.2.101:0/1561822434 conn(0x7f65500ce420 msgr2=0x7f65500d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:54.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.675 7f65582ee700 1 -- 172.21.2.101:0/1561822434 shutdown_connections 2021-02-10T11:42:54.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.675 7f65582ee700 1 -- 172.21.2.101:0/1561822434 wait complete. 2021-02-10T11:42:54.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.690 7f65582ee700 1 Processor -- start 2021-02-10T11:42:54.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.690 7f65582ee700 1 -- start start 2021-02-10T11:42:54.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.690 7f65582ee700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65501a79a0 0x7f65501a7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:54.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.690 7f65582ee700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65500bdb40 0x7f65500bdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:54.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.690 7f65582ee700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6550082ca0 0x7f65500830a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:54.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.690 7f65582ee700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655019d990 con 0x7f65501a79a0 2021-02-10T11:42:54.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.690 7f65582ee700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655019dbb0 con 0x7f65500bdb40 2021-02-10T11:42:54.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.690 7f65582ee700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65500836c0 con 0x7f6550082ca0 2021-02-10T11:42:54.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.690 7f655610a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65501a79a0 0x7f65501a7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:54.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.690 7f655610a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65501a79a0 0x7f65501a7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58504/0 (socket says 172.21.2.101:58504) 2021-02-10T11:42:54.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.690 7f655610a700 1 -- 172.21.2.101:0/2453161787 learned_addr learned my addr 172.21.2.101:0/2453161787 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:54.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.690 7f6555909700 1 --2- 172.21.2.101:0/2453161787 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65500bdb40 0x7f65500bdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:54.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.690 7f655690b700 1 --2- 172.21.2.101:0/2453161787 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6550082ca0 0x7f65500830a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:54.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.691 7f6555909700 1 -- 172.21.2.101:0/2453161787 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6550082ca0 msgr2=0x7f65500830a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:54.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.691 7f6555909700 1 --2- 172.21.2.101:0/2453161787 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6550082ca0 0x7f65500830a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:54.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.691 7f6555909700 1 -- 172.21.2.101:0/2453161787 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65501a79a0 msgr2=0x7f65501a7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:54.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.691 7f6555909700 1 --2- 172.21.2.101:0/2453161787 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65501a79a0 0x7f65501a7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:54.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.691 7f6555909700 1 -- 172.21.2.101:0/2453161787 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65500838e0 con 0x7f65500bdb40 2021-02-10T11:42:54.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.691 7f6555909700 1 --2- 172.21.2.101:0/2453161787 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65500bdb40 0x7f65500bdf40 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f654400b4b0 tx=0x7f654400b850).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:54.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.691 7f6542ffd700 1 -- 172.21.2.101:0/2453161787 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6544009280 con 0x7f65500bdb40 2021-02-10T11:42:54.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.691 7f6542ffd700 1 -- 172.21.2.101:0/2453161787 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654400bc40 con 0x7f65500bdb40 2021-02-10T11:42:54.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.691 7f65582ee700 1 -- 172.21.2.101:0/2453161787 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65500838e0 con 0x7f65500bdb40 2021-02-10T11:42:54.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.691 7f6542ffd700 1 -- 172.21.2.101:0/2453161787 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6544009280 con 0x7f65500bdb40 2021-02-10T11:42:54.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.691 7f65582ee700 1 -- 172.21.2.101:0/2453161787 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65500838e0 con 0x7f65500bdb40 2021-02-10T11:42:54.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.691 7f6542ffd700 1 -- 172.21.2.101:0/2453161787 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f65440052e0 con 0x7f65500bdb40 2021-02-10T11:42:54.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.691 7f6542ffd700 1 --2- 172.21.2.101:0/2453161787 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6538018730 0x7f653801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:54.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.691 7f6542ffd700 1 -- 172.21.2.101:0/2453161787 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6544032530 con 0x7f65500bdb40 2021-02-10T11:42:54.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.692 7f655610a700 1 --2- 172.21.2.101:0/2453161787 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6538018730 0x7f653801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:54.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.692 7f655610a700 1 --2- 172.21.2.101:0/2453161787 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6538018730 0x7f653801ac10 crc :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:54.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.723 7f65582ee700 1 -- 172.21.2.101:0/2453161787 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6550082a70 con 0x7f65500bdb40 2021-02-10T11:42:54.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.726 7f6542ffd700 1 -- 172.21.2.101:0/2453161787 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6544000c90 con 0x7f65500bdb40 2021-02-10T11:42:54.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.880 7f65582ee700 1 -- 172.21.2.101:0/2453161787 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6550082a70 con 0x7f65500bdb40 2021-02-10T11:42:54.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.881 7f6542ffd700 1 -- 172.21.2.101:0/2453161787 <== mon.1 v2:172.21.2.108:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f65440052e0 con 0x7f65500bdb40 2021-02-10T11:42:54.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.884 7f65582ee700 1 -- 172.21.2.101:0/2453161787 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6538018730 msgr2=0x7f653801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:54.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.884 7f65582ee700 1 --2- 172.21.2.101:0/2453161787 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6538018730 0x7f653801ac10 crc :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:54.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.884 7f65582ee700 1 -- 172.21.2.101:0/2453161787 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65500bdb40 msgr2=0x7f65500bdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:54.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.884 7f65582ee700 1 --2- 172.21.2.101:0/2453161787 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65500bdb40 0x7f65500bdf40 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f654400b4b0 tx=0x7f654400b850).stop 2021-02-10T11:42:54.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.884 7f65582ee700 1 -- 172.21.2.101:0/2453161787 shutdown_connections 2021-02-10T11:42:54.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.884 7f65582ee700 1 --2- 172.21.2.101:0/2453161787 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6538018730 0x7f653801ac10 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:54.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.884 7f65582ee700 1 --2- 172.21.2.101:0/2453161787 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6550082ca0 0x7f65500830a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:54.886 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.884 7f65582ee700 1 --2- 172.21.2.101:0/2453161787 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65500bdb40 0x7f65500bdf40 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:54.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.884 7f65582ee700 1 --2- 172.21.2.101:0/2453161787 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65501a79a0 0x7f65501a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:54.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.885 7f65582ee700 1 -- 172.21.2.101:0/2453161787 >> 172.21.2.101:0/2453161787 conn(0x7f65500bef00 msgr2=0x7f655009f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:54.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.885 7f65582ee700 1 -- 172.21.2.101:0/2453161787 shutdown_connections 2021-02-10T11:42:54.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:54.885 7f65582ee700 1 -- 172.21.2.101:0/2453161787 wait complete. 2021-02-10T11:42:54.895 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000812466 2021-02-10T11:42:55.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.308 7fc8e43ec700 1 Processor -- start 2021-02-10T11:42:55.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.308 7fc8e43ec700 1 -- start start 2021-02-10T11:42:55.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.309 7fc8e43ec700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc8dc0d1c70 0x7fc8dc0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:55.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.309 7fc8e43ec700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8dc0cf580 0x7fc8dc0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:55.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.309 7fc8e43ec700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8dc0fe1c0 con 0x7fc8dc0d1870 2021-02-10T11:42:55.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.309 7fc8e43ec700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8dc1b0cb0 con 0x7fc8dc0d1c70 2021-02-10T11:42:55.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.309 7fc8e43ec700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc109a60 con 0x7fc8dc0cf580 2021-02-10T11:42:55.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.309 7fc8e1a07700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc8dc0d1c70 0x7fc8dc0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:55.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.309 7fc8e2208700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc8dc0d1870 0x7fc8dc1b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59006/0 (socket says 172.21.2.101:59006) 2021-02-10T11:42:55.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.309 7fc8e2208700 1 -- 172.21.2.101:0/2470287688 learned_addr learned my addr 172.21.2.101:0/2470287688 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:55.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.309 7fc8e1a07700 1 -- 172.21.2.101:0/2470287688 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8dc0cf580 msgr2=0x7fc8dc0cf980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:42:55.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.309 7fc8e1a07700 1 --2- 172.21.2.101:0/2470287688 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8dc0cf580 0x7fc8dc0cf980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:55.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.309 7fc8e1a07700 1 -- 172.21.2.101:0/2470287688 >> v1:172.21.2.101:6789/0 conn(0x7fc8dc0d1870 legacy=0x7fc8dc1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:55.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.309 7fc8e1a07700 1 -- 172.21.2.101:0/2470287688 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8dc088070 con 0x7fc8dc0d1c70 2021-02-10T11:42:55.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.310 7fc8e1a07700 1 --2- 172.21.2.101:0/2470287688 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8dc0d1c70 0x7fc8dc0cef60 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 rx=0x7fc8d8006650 tx=0x7fc8d8009a00).ready entity=mon.0 client_cookie=78d85a4024badc59 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:55.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.310 7fc8e1206700 1 -- 172.21.2.101:0/2470287688 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc8d80009b0 con 0x7fc8dc0d1c70 2021-02-10T11:42:55.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.310 7fc8e1206700 1 -- 172.21.2.101:0/2470287688 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d800c0c0 con 0x7fc8dc0d1c70 2021-02-10T11:42:55.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.310 7fc8e1206700 1 -- 172.21.2.101:0/2470287688 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc8d800c5a0 con 0x7fc8dc0d1c70 2021-02-10T11:42:55.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.310 7fc8e43ec700 1 -- 172.21.2.101:0/2470287688 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8dc0d1c70 msgr2=0x7fc8dc0cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:55.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.310 7fc8e43ec700 1 --2- 172.21.2.101:0/2470287688 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8dc0d1c70 0x7fc8dc0cef60 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 rx=0x7fc8d8006650 tx=0x7fc8d8009a00).stop 2021-02-10T11:42:55.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.310 7fc8e43ec700 1 -- 172.21.2.101:0/2470287688 shutdown_connections 2021-02-10T11:42:55.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.310 7fc8e43ec700 1 --2- 172.21.2.101:0/2470287688 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8dc0cf580 0x7fc8dc0cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:55.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.310 7fc8e43ec700 1 --2- 172.21.2.101:0/2470287688 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8dc0d1c70 0x7fc8dc0cef60 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:55.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.310 7fc8e43ec700 1 -- 172.21.2.101:0/2470287688 >> 172.21.2.101:0/2470287688 conn(0x7fc8dc0ce420 msgr2=0x7fc8dc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:55.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.311 7fc8e43ec700 1 -- 172.21.2.101:0/2470287688 shutdown_connections 2021-02-10T11:42:55.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.311 7fc8e43ec700 1 -- 172.21.2.101:0/2470287688 wait complete. 2021-02-10T11:42:55.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.325 7fc8e43ec700 1 Processor -- start 2021-02-10T11:42:55.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.325 7fc8e43ec700 1 -- start start 2021-02-10T11:42:55.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.325 7fc8e43ec700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8dc0c3820 0x7fc8dc084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:55.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.325 7fc8e43ec700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8dc0a0290 0x7fc8dc0852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:55.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.325 7fc8e43ec700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8dc086270 0x7fc8dc086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:55.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.325 7fc8e43ec700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc0c0700 con 0x7fc8dc0a0290 2021-02-10T11:42:55.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.325 7fc8e43ec700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc0858f0 con 0x7fc8dc0c3820 2021-02-10T11:42:55.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.325 7fc8e43ec700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc085b10 con 0x7fc8dc086270 2021-02-10T11:42:55.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.326 7fc8e1a07700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8dc0a0290 0x7fc8dc0852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:55.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.326 7fc8e1a07700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8dc0a0290 0x7fc8dc0852d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58520/0 (socket says 172.21.2.101:58520) 2021-02-10T11:42:55.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.326 7fc8e1a07700 1 -- 172.21.2.101:0/4192856989 learned_addr learned my addr 172.21.2.101:0/4192856989 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:55.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.326 7fc8e1a07700 1 -- 172.21.2.101:0/4192856989 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8dc086270 msgr2=0x7fc8dc086670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:55.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.326 7fc8e1a07700 1 --2- 172.21.2.101:0/4192856989 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8dc086270 0x7fc8dc086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:55.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.326 7fc8e1a07700 1 -- 172.21.2.101:0/4192856989 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8dc0c3820 msgr2=0x7fc8dc084cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:55.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.326 7fc8e1a07700 1 --2- 172.21.2.101:0/4192856989 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8dc0c3820 0x7fc8dc084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:55.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.326 7fc8e1a07700 1 -- 172.21.2.101:0/4192856989 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8dc086c90 con 0x7fc8dc0a0290 2021-02-10T11:42:55.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.326 7fc8e1a07700 1 --2- 172.21.2.101:0/4192856989 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8dc0a0290 0x7fc8dc0852d0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 rx=0x7fc8d800b640 tx=0x7fc8d800b9e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:55.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.326 7fc8ceffd700 1 -- 172.21.2.101:0/4192856989 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc8d800bdd0 con 0x7fc8dc0a0290 2021-02-10T11:42:55.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.326 7fc8ceffd700 1 -- 172.21.2.101:0/4192856989 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d8000c90 con 0x7fc8dc0a0290 2021-02-10T11:42:55.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.326 7fc8ceffd700 1 -- 172.21.2.101:0/4192856989 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc8d80022b0 con 0x7fc8dc0a0290 2021-02-10T11:42:55.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.326 7fc8e43ec700 1 -- 172.21.2.101:0/4192856989 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8dc086c90 con 0x7fc8dc0a0290 2021-02-10T11:42:55.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.326 7fc8e43ec700 1 -- 172.21.2.101:0/4192856989 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8dc086c90 con 0x7fc8dc0a0290 2021-02-10T11:42:55.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.327 7fc8ceffd700 1 -- 172.21.2.101:0/4192856989 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc8d8006a10 con 0x7fc8dc0a0290 2021-02-10T11:42:55.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.327 7fc8ceffd700 1 --2- 172.21.2.101:0/4192856989 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc8c40186c0 0x7fc8c401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:55.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.327 7fc8ceffd700 1 -- 172.21.2.101:0/4192856989 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc8d80326b0 con 0x7fc8dc0a0290 2021-02-10T11:42:55.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.327 7fc8e2208700 1 --2- 172.21.2.101:0/4192856989 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc8c40186c0 0x7fc8c401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:55.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.327 7fc8e2208700 1 --2- 172.21.2.101:0/4192856989 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc8c40186c0 0x7fc8c401abc0 crc :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:55.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.359 7fc8e43ec700 1 -- 172.21.2.101:0/4192856989 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8dc082a70 con 0x7fc8dc0a0290 2021-02-10T11:42:55.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.361 7fc8ceffd700 1 -- 172.21.2.101:0/4192856989 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc8d801f570 con 0x7fc8dc0a0290 2021-02-10T11:42:55.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.488 7fc8e43ec700 1 -- 172.21.2.101:0/4192856989 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000812466", "target": ["mgr", ""]}) v1 -- 0x7fc8dc0858f0 con 0x7fc8c40186c0 2021-02-10T11:42:55.493 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:42:55.492 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:42:55.493 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:42:55.492 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:42:55.493 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:42:55.492 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:42:55.493 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:42:55.492 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:42:55.493 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:42:55.492 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:42:55.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.524 7fc8ceffd700 1 -- 172.21.2.101:0/4192856989 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fc8d0001140 con 0x7fc8c40186c0 2021-02-10T11:42:55.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.554 7fc8e43ec700 1 -- 172.21.2.101:0/4192856989 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc8c40186c0 msgr2=0x7fc8c401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:55.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.554 7fc8e43ec700 1 --2- 172.21.2.101:0/4192856989 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc8c40186c0 0x7fc8c401abc0 crc :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:55.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.554 7fc8e43ec700 1 -- 172.21.2.101:0/4192856989 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8dc0a0290 msgr2=0x7fc8dc0852d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:55.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.554 7fc8e43ec700 1 --2- 172.21.2.101:0/4192856989 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8dc0a0290 0x7fc8dc0852d0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 rx=0x7fc8d800b640 tx=0x7fc8d800b9e0).stop 2021-02-10T11:42:55.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.554 7fc8e43ec700 1 -- 172.21.2.101:0/4192856989 shutdown_connections 2021-02-10T11:42:55.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.554 7fc8e43ec700 1 --2- 172.21.2.101:0/4192856989 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc8c40186c0 0x7fc8c401abc0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:55.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.554 7fc8e43ec700 1 --2- 172.21.2.101:0/4192856989 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8dc086270 0x7fc8dc086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:55.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.554 7fc8e43ec700 1 --2- 172.21.2.101:0/4192856989 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8dc0a0290 0x7fc8dc0852d0 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:55.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.554 7fc8e43ec700 1 --2- 172.21.2.101:0/4192856989 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8dc0c3820 0x7fc8dc084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:55.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.554 7fc8e43ec700 1 -- 172.21.2.101:0/4192856989 >> 172.21.2.101:0/4192856989 conn(0x7fc8dc0872c0 msgr2=0x7fc8dc095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:55.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.554 7fc8e43ec700 1 -- 172.21.2.101:0/4192856989 shutdown_connections 2021-02-10T11:42:55.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.554 7fc8e43ec700 1 -- 172.21.2.101:0/4192856989 wait complete. 2021-02-10T11:42:55.564 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs clone status cephfs subvolume_0000000000812466 2021-02-10T11:42:55.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.978 7fcf303f2700 1 Processor -- start 2021-02-10T11:42:55.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.978 7fcf303f2700 1 -- start start 2021-02-10T11:42:55.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.978 7fcf303f2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcf280d1870 0x7fcf281b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:55.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.978 7fcf303f2700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fcf281b0ae0 0x7fcf28108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:55.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.978 7fcf303f2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcf280d1c70 0x7fcf28108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:55.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.978 7fcf303f2700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf280fe1c0 con 0x7fcf280d1870 2021-02-10T11:42:55.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.978 7fcf303f2700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf28109a60 con 0x7fcf281b0ae0 2021-02-10T11:42:55.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.978 7fcf303f2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf28088070 con 0x7fcf280d1c70 2021-02-10T11:42:55.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.978 7fcf2e20e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcf280d1870 0x7fcf281b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:55.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.978 7fcf2e20e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcf280d1870 0x7fcf281b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58542/0 (socket says 172.21.2.101:58542) 2021-02-10T11:42:55.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.979 7fcf2e20e700 1 -- 172.21.2.101:0/1446276370 learned_addr learned my addr 172.21.2.101:0/1446276370 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:55.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.979 7fcf2e20e700 1 -- 172.21.2.101:0/1446276370 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcf280d1c70 msgr2=0x7fcf28108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:55.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.979 7fcf2e20e700 1 --2- 172.21.2.101:0/1446276370 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcf280d1c70 0x7fcf28108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:55.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.979 7fcf2e20e700 1 -- 172.21.2.101:0/1446276370 >> v2:172.21.2.108:3300/0 conn(0x7fcf281b0ae0 msgr2=0x7fcf28108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:42:55.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.979 7fcf2e20e700 1 --2- 172.21.2.101:0/1446276370 >> v2:172.21.2.108:3300/0 conn(0x7fcf281b0ae0 0x7fcf28108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:55.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.979 7fcf2e20e700 1 -- 172.21.2.101:0/1446276370 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf28109580 con 0x7fcf280d1870 2021-02-10T11:42:55.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.979 7fcf2e20e700 1 --2- 172.21.2.101:0/1446276370 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcf280d1870 0x7fcf281b04c0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 rx=0x7fcf24006700 tx=0x7fcf24009b80).ready entity=mon.0 client_cookie=49fc89551d47840f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:55.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.979 7fcf2d20c700 1 -- 172.21.2.101:0/1446276370 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcf2400c0c0 con 0x7fcf280d1870 2021-02-10T11:42:55.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.979 7fcf2d20c700 1 -- 172.21.2.101:0/1446276370 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2400c440 con 0x7fcf280d1870 2021-02-10T11:42:55.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.979 7fcf2d20c700 1 -- 172.21.2.101:0/1446276370 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcf2400c8a0 con 0x7fcf280d1870 2021-02-10T11:42:55.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.979 7fcf303f2700 1 -- 172.21.2.101:0/1446276370 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcf280d1870 msgr2=0x7fcf281b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:55.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.979 7fcf303f2700 1 --2- 172.21.2.101:0/1446276370 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcf280d1870 0x7fcf281b04c0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 rx=0x7fcf24006700 tx=0x7fcf24009b80).stop 2021-02-10T11:42:55.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.980 7fcf303f2700 1 -- 172.21.2.101:0/1446276370 shutdown_connections 2021-02-10T11:42:55.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.980 7fcf303f2700 1 --2- 172.21.2.101:0/1446276370 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcf280d1c70 0x7fcf28108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:55.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.980 7fcf303f2700 1 --2- 172.21.2.101:0/1446276370 >> v2:172.21.2.108:3300/0 conn(0x7fcf281b0ae0 0x7fcf28108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:55.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.980 7fcf303f2700 1 --2- 172.21.2.101:0/1446276370 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcf280d1870 0x7fcf281b04c0 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:55.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.980 7fcf303f2700 1 -- 172.21.2.101:0/1446276370 >> 172.21.2.101:0/1446276370 conn(0x7fcf280ce420 msgr2=0x7fcf280d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:55.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.980 7fcf303f2700 1 -- 172.21.2.101:0/1446276370 shutdown_connections 2021-02-10T11:42:55.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.980 7fcf303f2700 1 -- 172.21.2.101:0/1446276370 wait complete. 2021-02-10T11:42:55.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.995 7fcf303f2700 1 Processor -- start 2021-02-10T11:42:55.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.995 7fcf303f2700 1 -- start start 2021-02-10T11:42:55.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.995 7fcf303f2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcf2809bd90 0x7fcf2819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:56.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.995 7fcf303f2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcf2819f740 0x7fcf28085350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:56.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.995 7fcf303f2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcf28085970 0x7fcf28085d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:56.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.995 7fcf303f2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf28086410 con 0x7fcf2809bd90 2021-02-10T11:42:56.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.995 7fcf303f2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf28086600 con 0x7fcf28085970 2021-02-10T11:42:56.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.995 7fcf303f2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf28086820 con 0x7fcf2819f740 2021-02-10T11:42:56.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.996 7fcf2e20e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcf2809bd90 0x7fcf2819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:56.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.996 7fcf2e20e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcf2809bd90 0x7fcf2819f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58546/0 (socket says 172.21.2.101:58546) 2021-02-10T11:42:56.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.996 7fcf2e20e700 1 -- 172.21.2.101:0/483730454 learned_addr learned my addr 172.21.2.101:0/483730454 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:56.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.996 7fcf2e20e700 1 -- 172.21.2.101:0/483730454 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcf2819f740 msgr2=0x7fcf28085350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:42:56.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.996 7fcf2e20e700 1 --2- 172.21.2.101:0/483730454 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcf2819f740 0x7fcf28085350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:56.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.996 7fcf2e20e700 1 -- 172.21.2.101:0/483730454 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcf28085970 msgr2=0x7fcf28085d70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:42:56.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.996 7fcf2e20e700 1 --2- 172.21.2.101:0/483730454 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcf28085970 0x7fcf28085d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:56.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.996 7fcf2e20e700 1 -- 172.21.2.101:0/483730454 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf280832b0 con 0x7fcf2809bd90 2021-02-10T11:42:56.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.996 7fcf2e20e700 1 --2- 172.21.2.101:0/483730454 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcf2809bd90 0x7fcf2819f120 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 rx=0x7fcf2400b5e0 tx=0x7fcf2400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:56.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.996 7fcf1effd700 1 -- 172.21.2.101:0/483730454 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcf2400bd70 con 0x7fcf2809bd90 2021-02-10T11:42:56.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.996 7fcf1effd700 1 -- 172.21.2.101:0/483730454 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf24002030 con 0x7fcf2809bd90 2021-02-10T11:42:56.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.996 7fcf1effd700 1 -- 172.21.2.101:0/483730454 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcf240024d0 con 0x7fcf2809bd90 2021-02-10T11:42:56.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.996 7fcf303f2700 1 -- 172.21.2.101:0/483730454 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf280832b0 con 0x7fcf2809bd90 2021-02-10T11:42:56.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.996 7fcf303f2700 1 -- 172.21.2.101:0/483730454 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf280832b0 con 0x7fcf2809bd90 2021-02-10T11:42:56.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.997 7fcf1effd700 1 -- 172.21.2.101:0/483730454 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fcf24000c90 con 0x7fcf2809bd90 2021-02-10T11:42:56.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.997 7fcf1effd700 1 --2- 172.21.2.101:0/483730454 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcf100186c0 0x7fcf1001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:56.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.997 7fcf1effd700 1 -- 172.21.2.101:0/483730454 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fcf24032610 con 0x7fcf2809bd90 2021-02-10T11:42:56.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.997 7fcf2da0d700 1 --2- 172.21.2.101:0/483730454 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcf100186c0 0x7fcf1001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:56.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:55.998 7fcf2da0d700 1 --2- 172.21.2.101:0/483730454 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcf100186c0 0x7fcf1001abc0 crc :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:56.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.028 7fcf303f2700 1 -- 172.21.2.101:0/483730454 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf28082a70 con 0x7fcf2809bd90 2021-02-10T11:42:56.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.031 7fcf1effd700 1 -- 172.21.2.101:0/483730454 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fcf24020f00 con 0x7fcf2809bd90 2021-02-10T11:42:56.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.162 7fcf303f2700 1 -- 172.21.2.101:0/483730454 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvolume_0000000000812466", "target": ["mgr", ""]}) v1 -- 0x7fcf28082a70 con 0x7fcf100186c0 2021-02-10T11:42:56.165 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:42:56.164 7f2cc39e1700 -1 mgr.server reply reply (95) Operation not supported operation 'clone-status' is not allowed on subvolume 'subvolume_0000000000812466' of type subvolume 2021-02-10T11:42:56.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.164 7fcf1effd700 1 -- 172.21.2.101:0/483730454 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: -95 operation 'clone-status' is not allowed on subvolume 'subvolume_0000000000812466' of type subvolume) v1 ==== 107+0+0 (crc 0 0 0) 0x7fcf180009b0 con 0x7fcf100186c0 2021-02-10T11:42:56.166 INFO:teuthology.orchestra.run.gibba001.stderr:Error ENOTSUP: operation 'clone-status' is not allowed on subvolume 'subvolume_0000000000812466' of type subvolume 2021-02-10T11:42:56.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.166 7fcf303f2700 1 -- 172.21.2.101:0/483730454 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcf100186c0 msgr2=0x7fcf1001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:56.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.166 7fcf303f2700 1 --2- 172.21.2.101:0/483730454 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcf100186c0 0x7fcf1001abc0 crc :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:56.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.166 7fcf303f2700 1 -- 172.21.2.101:0/483730454 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcf2809bd90 msgr2=0x7fcf2819f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:56.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.166 7fcf303f2700 1 --2- 172.21.2.101:0/483730454 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcf2809bd90 0x7fcf2819f120 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 rx=0x7fcf2400b5e0 tx=0x7fcf2400b980).stop 2021-02-10T11:42:56.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.167 7fcf303f2700 1 -- 172.21.2.101:0/483730454 shutdown_connections 2021-02-10T11:42:56.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.167 7fcf303f2700 1 --2- 172.21.2.101:0/483730454 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcf100186c0 0x7fcf1001abc0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:56.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.167 7fcf303f2700 1 --2- 172.21.2.101:0/483730454 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcf2819f740 0x7fcf28085350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:56.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.167 7fcf303f2700 1 --2- 172.21.2.101:0/483730454 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcf28085970 0x7fcf28085d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:56.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.167 7fcf303f2700 1 --2- 172.21.2.101:0/483730454 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcf2809bd90 0x7fcf2819f120 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:56.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.167 7fcf303f2700 1 -- 172.21.2.101:0/483730454 >> 172.21.2.101:0/483730454 conn(0x7fcf280946f0 msgr2=0x7fcf28095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:56.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.167 7fcf303f2700 1 -- 172.21.2.101:0/483730454 shutdown_connections 2021-02-10T11:42:56.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.167 7fcf303f2700 1 -- 172.21.2.101:0/483730454 wait complete. 2021-02-10T11:42:56.176 DEBUG:teuthology.orchestra.run:got remote process result: 95 2021-02-10T11:42:56.176 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000812466 2021-02-10T11:42:56.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.592 7f022fb78700 1 Processor -- start 2021-02-10T11:42:56.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.592 7f022fb78700 1 -- start start 2021-02-10T11:42:56.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.593 7f022fb78700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f022810aff0 0x7f02280cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:56.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.593 7f022fb78700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02280fe1c0 con 0x7f02280d1870 2021-02-10T11:42:56.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.593 7f022fb78700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02281b0cb0 con 0x7f02280d1c70 2021-02-10T11:42:56.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.593 7f022fb78700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0228109a60 con 0x7f022810aff0 2021-02-10T11:42:56.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.593 7f022d994700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f02280d1870 0x7f02281b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59048/0 (socket says 172.21.2.101:59048) 2021-02-10T11:42:56.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.593 7f022d994700 1 -- 172.21.2.101:0/1158850466 learned_addr learned my addr 172.21.2.101:0/1158850466 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:42:56.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.593 7f022e195700 1 --2- 172.21.2.101:0/1158850466 >> v2:172.21.2.108:3300/0 conn(0x7f022810aff0 0x7f02280cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:56.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.593 7f022c992700 1 -- 172.21.2.101:0/1158850466 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970883467 0 0) 0x7f02240010b0 con 0x7f02280d1870 2021-02-10T11:42:56.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.593 7f022e195700 1 -- 172.21.2.101:0/1158850466 >> v1:172.21.2.108:6789/0 conn(0x7f02280d1c70 legacy=0x7f02280cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:56.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.593 7f022e195700 1 -- 172.21.2.101:0/1158850466 >> v1:172.21.2.101:6789/0 conn(0x7f02280d1870 legacy=0x7f02281b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:56.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.593 7f022e195700 1 -- 172.21.2.101:0/1158850466 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022810b3f0 con 0x7f022810aff0 2021-02-10T11:42:56.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.594 7f022e195700 1 --2- 172.21.2.101:0/1158850466 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f022810aff0 0x7f02280cf750 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f0220006e50 tx=0x7f0220009e70).ready entity=mon.1 client_cookie=1e69f4e602f2dd55 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:56.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.594 7f022c992700 1 -- 172.21.2.101:0/1158850466 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0220006a20 con 0x7f022810aff0 2021-02-10T11:42:56.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.594 7f022c992700 1 -- 172.21.2.101:0/1158850466 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0220006790 con 0x7f022810aff0 2021-02-10T11:42:56.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.594 7f022c992700 1 -- 172.21.2.101:0/1158850466 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f022000b700 con 0x7f022810aff0 2021-02-10T11:42:56.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.594 7f022fb78700 1 -- 172.21.2.101:0/1158850466 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f022810aff0 msgr2=0x7f02280cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:56.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.594 7f022fb78700 1 --2- 172.21.2.101:0/1158850466 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f022810aff0 0x7f02280cf750 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f0220006e50 tx=0x7f0220009e70).stop 2021-02-10T11:42:56.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.595 7f022fb78700 1 -- 172.21.2.101:0/1158850466 shutdown_connections 2021-02-10T11:42:56.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.595 7f022fb78700 1 --2- 172.21.2.101:0/1158850466 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f022810aff0 0x7f02280cf750 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:56.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.595 7f022fb78700 1 -- 172.21.2.101:0/1158850466 >> 172.21.2.101:0/1158850466 conn(0x7f02280ce420 msgr2=0x7f02280d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:56.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.595 7f022fb78700 1 -- 172.21.2.101:0/1158850466 shutdown_connections 2021-02-10T11:42:56.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.595 7f022fb78700 1 -- 172.21.2.101:0/1158850466 wait complete. 2021-02-10T11:42:56.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.609 7f022fb78700 1 Processor -- start 2021-02-10T11:42:56.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.609 7f022fb78700 1 -- start start 2021-02-10T11:42:56.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.610 7f022fb78700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02280c3820 0x7f02280a7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:56.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.610 7f022fb78700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02281a44c0 0x7f02281a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:56.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.610 7f022fb78700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02280863d0 0x7f02280867d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:56.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.610 7f022fb78700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0228085ed0 con 0x7f02280c3820 2021-02-10T11:42:56.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.610 7f022fb78700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02280c0740 con 0x7f02281a44c0 2021-02-10T11:42:56.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.610 7f022fb78700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0228086df0 con 0x7f02280863d0 2021-02-10T11:42:56.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.610 7f022d994700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02280c3820 0x7f02280a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:56.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.610 7f022d994700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02280c3820 0x7f02280a7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58560/0 (socket says 172.21.2.101:58560) 2021-02-10T11:42:56.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.610 7f022d994700 1 -- 172.21.2.101:0/599342209 learned_addr learned my addr 172.21.2.101:0/599342209 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:42:56.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.610 7f022d994700 1 -- 172.21.2.101:0/599342209 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02280863d0 msgr2=0x7f02280867d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:56.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.610 7f022d994700 1 --2- 172.21.2.101:0/599342209 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02280863d0 0x7f02280867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:56.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.610 7f022d994700 1 -- 172.21.2.101:0/599342209 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02281a44c0 msgr2=0x7f02281a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:56.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.610 7f022d994700 1 --2- 172.21.2.101:0/599342209 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02281a44c0 0x7f02281a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:56.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.610 7f022d994700 1 -- 172.21.2.101:0/599342209 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0228087010 con 0x7f02280c3820 2021-02-10T11:42:56.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.610 7f022d994700 1 --2- 172.21.2.101:0/599342209 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02280c3820 0x7f02280a7950 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 rx=0x7f0224001c10 tx=0x7f022400a8f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:56.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.611 7f021e7fc700 1 -- 172.21.2.101:0/599342209 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f02240009b0 con 0x7f02280c3820 2021-02-10T11:42:56.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.611 7f021e7fc700 1 -- 172.21.2.101:0/599342209 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02240012c0 con 0x7f02280c3820 2021-02-10T11:42:56.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.611 7f022fb78700 1 -- 172.21.2.101:0/599342209 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0228087010 con 0x7f02280c3820 2021-02-10T11:42:56.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.611 7f021e7fc700 1 -- 172.21.2.101:0/599342209 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0224004350 con 0x7f02280c3820 2021-02-10T11:42:56.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.611 7f022fb78700 1 -- 172.21.2.101:0/599342209 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0228087010 con 0x7f02280c3820 2021-02-10T11:42:56.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.611 7f021e7fc700 1 -- 172.21.2.101:0/599342209 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0224004140 con 0x7f02280c3820 2021-02-10T11:42:56.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.611 7f021e7fc700 1 --2- 172.21.2.101:0/599342209 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0210018730 0x7f021001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:42:56.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.612 7f021e7fc700 1 -- 172.21.2.101:0/599342209 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f02240327b0 con 0x7f02280c3820 2021-02-10T11:42:56.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.612 7f022d193700 1 --2- 172.21.2.101:0/599342209 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0210018730 0x7f021001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:42:56.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.612 7f022d193700 1 --2- 172.21.2.101:0/599342209 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0210018730 0x7f021001ac10 crc :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:42:56.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.642 7f022fb78700 1 -- 172.21.2.101:0/599342209 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02280fe1c0 con 0x7f02280c3820 2021-02-10T11:42:56.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.644 7f021e7fc700 1 -- 172.21.2.101:0/599342209 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f022401f4f0 con 0x7f02280c3820 2021-02-10T11:42:56.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.771 7f022fb78700 1 -- 172.21.2.101:0/599342209 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000812466", "target": ["mgr", ""]}) v1 -- 0x7f02280fe1c0 con 0x7f0210018730 2021-02-10T11:42:56.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.780 7f021e7fc700 1 -- 172.21.2.101:0/599342209 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f0218006e20 con 0x7f0210018730 2021-02-10T11:42:56.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.788 7f022fb78700 1 -- 172.21.2.101:0/599342209 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0210018730 msgr2=0x7f021001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:56.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.788 7f022fb78700 1 --2- 172.21.2.101:0/599342209 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0210018730 0x7f021001ac10 crc :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:56.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.788 7f022fb78700 1 -- 172.21.2.101:0/599342209 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02280c3820 msgr2=0x7f02280a7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:42:56.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.788 7f022fb78700 1 --2- 172.21.2.101:0/599342209 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02280c3820 0x7f02280a7950 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 rx=0x7f0224001c10 tx=0x7f022400a8f0).stop 2021-02-10T11:42:56.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.788 7f022fb78700 1 -- 172.21.2.101:0/599342209 shutdown_connections 2021-02-10T11:42:56.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.788 7f022fb78700 1 --2- 172.21.2.101:0/599342209 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0210018730 0x7f021001ac10 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:56.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.788 7f022fb78700 1 --2- 172.21.2.101:0/599342209 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f02280863d0 0x7f02280867d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:56.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.788 7f022fb78700 1 --2- 172.21.2.101:0/599342209 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f02281a44c0 0x7f02281a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:42:56.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.788 7f022fb78700 1 --2- 172.21.2.101:0/599342209 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f02280c3820 0x7f02280a7950 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:42:56.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.788 7f022fb78700 1 -- 172.21.2.101:0/599342209 >> 172.21.2.101:0/599342209 conn(0x7f02280872c0 msgr2=0x7f0228095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:42:56.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.788 7f022fb78700 1 -- 172.21.2.101:0/599342209 shutdown_connections 2021-02-10T11:42:56.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:42:56.788 7f022fb78700 1 -- 172.21.2.101:0/599342209 wait complete. 2021-02-10T11:42:56.798 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo getfattr --only-values -n ceph.dir.entries /home/ubuntu/cephtest/mnt.0/./volumes/_deleting 2021-02-10T11:42:57.147 INFO:teuthology.orchestra.run.gibba001.stderr:getfattr: Removing leading '/' from absolute path names 2021-02-10T11:42:57.148 INFO:teuthology.orchestra.run.gibba001.stdout:0 2021-02-10T11:42:57.148 DEBUG:tasks.cephfs.mount:Directory ./volumes/_deleting seen empty from 0 after 0s 2021-02-10T11:42:57.149 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:42:57.149 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:42:57.495 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:42:57.841 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:42:57.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:57.841 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:42:57.841 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:42:58.176 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:42:58.176 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:42:58.177 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:42:58.177 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:42:58.177 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:42:58.529 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[155120]: fuse finished with error 0 and tester_r 0 2021-02-10T11:42:58.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:58.530 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:42:58.531 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:42:58.866 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:42:58.867 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:42:58.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:42:58.867 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:42:59.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:59.202 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:42:59.202 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:42:59.536 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:42:59.537 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:42:59.537 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:42:59.537 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:42:59.537 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:42:59.886 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[155760]: fuse finished with error 0 and tester_r 0 2021-02-10T11:42:59.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:42:59.888 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:42:59.888 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:43:00.223 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:43:00.224 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:43:00.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:43:00.224 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:43:00.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:00.560 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:43:00.560 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:43:00.894 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:43:00.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:43:00.895 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:43:00.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:43:00.895 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:43:01.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:01.231 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:43:01.231 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:43:01.566 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:43:01.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:43:01.567 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:43:01.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:43:01.567 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:43:01.902 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-02-10T11:43:02.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.316 7f06caeeb700 1 Processor -- start 2021-02-10T11:43:02.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.316 7f06caeeb700 1 -- start start 2021-02-10T11:43:02.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.317 7f06caeeb700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f06c40d1870 0x7f06c41b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:02.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.317 7f06caeeb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06c40f0650 0x7f06c40cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:02.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.317 7f06caeeb700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c40f0b10 con 0x7f06c41b0a10 2021-02-10T11:43:02.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.317 7f06caeeb700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c40e5ed0 con 0x7f06c40d1870 2021-02-10T11:43:02.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.317 7f06caeeb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c40d1c70 con 0x7f06c40f0650 2021-02-10T11:43:02.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.317 7f06c9508700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06c40f0650 0x7f06c40cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:02.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.317 7f06c3fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f06c41b0a10 0x7f06c40cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59064/0 (socket says 172.21.2.101:59064) 2021-02-10T11:43:02.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.317 7f06c3fff700 1 -- 172.21.2.101:0/1809584259 learned_addr learned my addr 172.21.2.101:0/1809584259 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:02.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.317 7f06c8d07700 1 --2- 172.21.2.101:0/1809584259 >> v2:172.21.2.108:3300/0 conn(0x7f06c40d1870 0x7f06c41b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:02.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.317 7f06c37fe700 1 -- 172.21.2.101:0/1809584259 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76292520 0 0) 0x7f06b40009b0 con 0x7f06c41b0a10 2021-02-10T11:43:02.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.317 7f06c9508700 1 -- 172.21.2.101:0/1809584259 >> v2:172.21.2.108:3300/0 conn(0x7f06c40d1870 msgr2=0x7f06c41b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:02.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.317 7f06c9508700 1 --2- 172.21.2.101:0/1809584259 >> v2:172.21.2.108:3300/0 conn(0x7f06c40d1870 0x7f06c41b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:02.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.317 7f06c9508700 1 -- 172.21.2.101:0/1809584259 >> v1:172.21.2.101:6789/0 conn(0x7f06c41b0a10 legacy=0x7f06c40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:02.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.317 7f06c9508700 1 -- 172.21.2.101:0/1809584259 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c40d1e90 con 0x7f06c40f0650 2021-02-10T11:43:02.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.318 7f06c9508700 1 --2- 172.21.2.101:0/1809584259 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06c40f0650 0x7f06c40cfb20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f06bc006e50 tx=0x7f06bc009e70).ready entity=mon.2 client_cookie=ceca72f05f59afa0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:02.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.318 7f06c37fe700 1 -- 172.21.2.101:0/1809584259 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f06bc00b510 con 0x7f06c40f0650 2021-02-10T11:43:02.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.318 7f06c37fe700 1 -- 172.21.2.101:0/1809584259 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06bc006790 con 0x7f06c40f0650 2021-02-10T11:43:02.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.318 7f06c37fe700 1 -- 172.21.2.101:0/1809584259 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f06bc00b940 con 0x7f06c40f0650 2021-02-10T11:43:02.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.318 7f06caeeb700 1 -- 172.21.2.101:0/1809584259 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06c40f0650 msgr2=0x7f06c40cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:02.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.318 7f06caeeb700 1 --2- 172.21.2.101:0/1809584259 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06c40f0650 0x7f06c40cfb20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f06bc006e50 tx=0x7f06bc009e70).stop 2021-02-10T11:43:02.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.319 7f06caeeb700 1 -- 172.21.2.101:0/1809584259 shutdown_connections 2021-02-10T11:43:02.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.319 7f06caeeb700 1 --2- 172.21.2.101:0/1809584259 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06c40f0650 0x7f06c40cfb20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:02.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.319 7f06caeeb700 1 --2- 172.21.2.101:0/1809584259 >> v2:172.21.2.108:3300/0 conn(0x7f06c40d1870 0x7f06c41b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:02.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.319 7f06caeeb700 1 -- 172.21.2.101:0/1809584259 >> 172.21.2.101:0/1809584259 conn(0x7f06c40ce450 msgr2=0x7f06c40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:02.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.319 7f06caeeb700 1 -- 172.21.2.101:0/1809584259 shutdown_connections 2021-02-10T11:43:02.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.319 7f06caeeb700 1 -- 172.21.2.101:0/1809584259 wait complete. 2021-02-10T11:43:02.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.333 7f06caeeb700 1 Processor -- start 2021-02-10T11:43:02.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.333 7f06caeeb700 1 -- start start 2021-02-10T11:43:02.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.334 7f06caeeb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06c40ebe70 0x7f06c40ec270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:02.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.334 7f06caeeb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f06c4093830 0x7f06c4093c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:02.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.334 7f06caeeb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06c4094250 0x7f06c4094650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:02.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.334 7f06caeeb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c40ec940 con 0x7f06c40ebe70 2021-02-10T11:43:02.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.334 7f06caeeb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c40ece40 con 0x7f06c4093830 2021-02-10T11:43:02.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.334 7f06caeeb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c40ed060 con 0x7f06c4094250 2021-02-10T11:43:02.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.334 7f06c8d07700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06c40ebe70 0x7f06c40ec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:02.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.334 7f06c3fff700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f06c4093830 0x7f06c4093c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:02.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.334 7f06c8d07700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06c40ebe70 0x7f06c40ec270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58574/0 (socket says 172.21.2.101:58574) 2021-02-10T11:43:02.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.334 7f06c8d07700 1 -- 172.21.2.101:0/3589599895 learned_addr learned my addr 172.21.2.101:0/3589599895 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:02.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.334 7f06c9508700 1 --2- 172.21.2.101:0/3589599895 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06c4094250 0x7f06c4094650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:02.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.334 7f06c8d07700 1 -- 172.21.2.101:0/3589599895 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06c4094250 msgr2=0x7f06c4094650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:02.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.334 7f06c8d07700 1 --2- 172.21.2.101:0/3589599895 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06c4094250 0x7f06c4094650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:02.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.334 7f06c8d07700 1 -- 172.21.2.101:0/3589599895 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f06c4093830 msgr2=0x7f06c4093c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:02.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.334 7f06c8d07700 1 --2- 172.21.2.101:0/3589599895 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f06c4093830 0x7f06c4093c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:02.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.334 7f06c8d07700 1 -- 172.21.2.101:0/3589599895 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c40ed280 con 0x7f06c40ebe70 2021-02-10T11:43:02.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.335 7f06c8d07700 1 --2- 172.21.2.101:0/3589599895 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06c40ebe70 0x7f06c40ec270 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 rx=0x7f06ac006f60 tx=0x7f06ac00b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:02.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.335 7f06c17fa700 1 -- 172.21.2.101:0/3589599895 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f06ac00bd60 con 0x7f06c40ebe70 2021-02-10T11:43:02.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.335 7f06c17fa700 1 -- 172.21.2.101:0/3589599895 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06ac000c90 con 0x7f06c40ebe70 2021-02-10T11:43:02.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.335 7f06c17fa700 1 -- 172.21.2.101:0/3589599895 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f06ac0022b0 con 0x7f06c40ebe70 2021-02-10T11:43:02.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.335 7f06caeeb700 1 -- 172.21.2.101:0/3589599895 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c40ed280 con 0x7f06c40ebe70 2021-02-10T11:43:02.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.335 7f06caeeb700 1 -- 172.21.2.101:0/3589599895 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c40ed280 con 0x7f06c40ebe70 2021-02-10T11:43:02.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.336 7f06c17fa700 1 -- 172.21.2.101:0/3589599895 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f06ac006b00 con 0x7f06c40ebe70 2021-02-10T11:43:02.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.336 7f06c17fa700 1 --2- 172.21.2.101:0/3589599895 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06a80186c0 0x7f06a801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:02.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.336 7f06c17fa700 1 -- 172.21.2.101:0/3589599895 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f06ac032560 con 0x7f06c40ebe70 2021-02-10T11:43:02.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.336 7f06c3fff700 1 --2- 172.21.2.101:0/3589599895 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06a80186c0 0x7f06a801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:02.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.336 7f06c3fff700 1 --2- 172.21.2.101:0/3589599895 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06a80186c0 0x7f06a801abc0 crc :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:02.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.366 7f06caeeb700 1 -- 172.21.2.101:0/3589599895 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c40ebc40 con 0x7f06c40ebe70 2021-02-10T11:43:02.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.369 7f06c17fa700 1 -- 172.21.2.101:0/3589599895 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f06ac0195d0 con 0x7f06c40ebe70 2021-02-10T11:43:02.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.517 7f06caeeb700 1 -- 172.21.2.101:0/3589599895 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f06c40ebc40 con 0x7f06c40ebe70 2021-02-10T11:43:02.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.521 7f06c17fa700 1 -- 172.21.2.101:0/3589599895 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7f06ac0195d0 con 0x7f06c40ebe70 2021-02-10T11:43:02.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.526 7f06caeeb700 1 -- 172.21.2.101:0/3589599895 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06a80186c0 msgr2=0x7f06a801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:02.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.526 7f06caeeb700 1 --2- 172.21.2.101:0/3589599895 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06a80186c0 0x7f06a801abc0 crc :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:02.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.526 7f06caeeb700 1 -- 172.21.2.101:0/3589599895 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06c40ebe70 msgr2=0x7f06c40ec270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:02.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.526 7f06caeeb700 1 --2- 172.21.2.101:0/3589599895 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06c40ebe70 0x7f06c40ec270 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 rx=0x7f06ac006f60 tx=0x7f06ac00b970).stop 2021-02-10T11:43:02.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.526 7f06caeeb700 1 -- 172.21.2.101:0/3589599895 shutdown_connections 2021-02-10T11:43:02.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.526 7f06caeeb700 1 --2- 172.21.2.101:0/3589599895 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06a80186c0 0x7f06a801abc0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:02.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.526 7f06caeeb700 1 --2- 172.21.2.101:0/3589599895 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06c4094250 0x7f06c4094650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:02.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.526 7f06caeeb700 1 --2- 172.21.2.101:0/3589599895 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f06c4093830 0x7f06c4093c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:02.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.526 7f06caeeb700 1 --2- 172.21.2.101:0/3589599895 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06c40ebe70 0x7f06c40ec270 unknown :-1 s=CLOSED pgs=2489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:02.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.526 7f06caeeb700 1 -- 172.21.2.101:0/3589599895 >> 172.21.2.101:0/3589599895 conn(0x7f06c408fa10 msgr2=0x7f06c40bfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:02.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.526 7f06caeeb700 1 -- 172.21.2.101:0/3589599895 shutdown_connections 2021-02-10T11:43:02.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.526 7f06caeeb700 1 -- 172.21.2.101:0/3589599895 wait complete. 2021-02-10T11:43:02.536 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_non_clone_status' 2021-02-10T11:43:02.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.946 7f8956a1f700 1 Processor -- start 2021-02-10T11:43:02.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.946 7f8956a1f700 1 -- start start 2021-02-10T11:43:02.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.947 7f8956a1f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f89500d0340 0x7f89500ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:02.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.947 7f8956a1f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89500d1c70 0x7f89500efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:02.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.947 7f8956a1f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89500f0b10 con 0x7f89500d1870 2021-02-10T11:43:02.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.947 7f8956a1f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89500e5ed0 con 0x7f89500d0340 2021-02-10T11:43:02.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.947 7f8956a1f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89501b0be0 con 0x7f89500d1c70 2021-02-10T11:43:02.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.947 7f895483b700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f89500d1870 0x7f89501b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59074/0 (socket says 172.21.2.101:59074) 2021-02-10T11:43:02.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.947 7f895483b700 1 -- 172.21.2.101:0/4227623175 learned_addr learned my addr 172.21.2.101:0/4227623175 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:02.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.947 7f894f7fe700 1 -- 172.21.2.101:0/4227623175 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2858468707 0 0) 0x7f89380009b0 con 0x7f89500d1870 2021-02-10T11:43:02.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.947 7f894f7fe700 1 -- 172.21.2.101:0/4227623175 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f893c002fe0 con 0x7f89500d1870 2021-02-10T11:43:02.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f894f7fe700 1 -- 172.21.2.101:0/4227623175 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 13095437 0 0) 0x7f8938001010 con 0x7f89500d1870 2021-02-10T11:43:02.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f894f7fe700 1 -- 172.21.2.101:0/4227623175 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89500d1c70 msgr2=0x7f89500efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:02.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f894f7fe700 1 --2- 172.21.2.101:0/4227623175 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89500d1c70 0x7f89500efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:02.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f894f7fe700 1 -- 172.21.2.101:0/4227623175 >> v2:172.21.2.108:3300/0 conn(0x7f89500d0340 msgr2=0x7f89500ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:02.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f894f7fe700 1 --2- 172.21.2.101:0/4227623175 >> v2:172.21.2.108:3300/0 conn(0x7f89500d0340 0x7f89500ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:02.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f894f7fe700 1 -- 172.21.2.101:0/4227623175 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89500f20a0 con 0x7f89500d1870 2021-02-10T11:43:02.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f894f7fe700 1 -- 172.21.2.101:0/4227623175 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f8938001530 con 0x7f89500d1870 2021-02-10T11:43:02.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f894f7fe700 1 -- 172.21.2.101:0/4227623175 >> v1:172.21.2.101:6789/0 conn(0x7f89500d1870 legacy=0x7f89501b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:02.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f894f7fe700 1 --2- 172.21.2.101:0/4227623175 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f893c003b10 0x7f893c006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:02.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f894f7fe700 1 --2- 172.21.2.101:0/4227623175 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f893c006820 0x7f893c008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:02.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f894f7fe700 1 --2- 172.21.2.101:0/4227623175 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f893c0095a0 0x7f893c00ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:02.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f894f7fe700 1 -- 172.21.2.101:0/4227623175 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c00c2e0 con 0x7f893c006820 2021-02-10T11:43:02.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f894f7fe700 1 -- 172.21.2.101:0/4227623175 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c00c500 con 0x7f893c003b10 2021-02-10T11:43:02.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f894f7fe700 1 -- 172.21.2.101:0/4227623175 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893c00c720 con 0x7f893c0095a0 2021-02-10T11:43:02.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f894ffff700 1 --2- 172.21.2.101:0/4227623175 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f893c006820 0x7f893c008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:02.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f895503c700 1 --2- 172.21.2.101:0/4227623175 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f893c0095a0 0x7f893c00ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:02.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f895483b700 1 --2- 172.21.2.101:0/4227623175 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f893c003b10 0x7f893c006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:02.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f894ffff700 1 -- 172.21.2.101:0/4227623175 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f893c0095a0 msgr2=0x7f893c00ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:02.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f894ffff700 1 --2- 172.21.2.101:0/4227623175 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f893c0095a0 0x7f893c00ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:02.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f894ffff700 1 -- 172.21.2.101:0/4227623175 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f893c003b10 msgr2=0x7f893c006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:02.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f894ffff700 1 --2- 172.21.2.101:0/4227623175 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f893c003b10 0x7f893c006030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:02.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f894ffff700 1 -- 172.21.2.101:0/4227623175 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893c00c9c0 con 0x7f893c006820 2021-02-10T11:43:02.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.948 7f894ffff700 1 --2- 172.21.2.101:0/4227623175 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f893c006820 0x7f893c008d40 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 rx=0x7f8940006f70 tx=0x7f894004f050).ready entity=mon.0 client_cookie=20270fa7c84de132 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:02.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.949 7f894f7fe700 1 -- 172.21.2.101:0/4227623175 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8940006b90 con 0x7f893c006820 2021-02-10T11:43:02.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.949 7f894f7fe700 1 -- 172.21.2.101:0/4227623175 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894004f650 con 0x7f893c006820 2021-02-10T11:43:02.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.949 7f894f7fe700 1 -- 172.21.2.101:0/4227623175 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f894004faa0 con 0x7f893c006820 2021-02-10T11:43:02.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.949 7f8956a1f700 1 -- 172.21.2.101:0/4227623175 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f893c006820 msgr2=0x7f893c008d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:02.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.949 7f8956a1f700 1 --2- 172.21.2.101:0/4227623175 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f893c006820 0x7f893c008d40 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 rx=0x7f8940006f70 tx=0x7f894004f050).stop 2021-02-10T11:43:02.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.949 7f895503c700 1 -- 172.21.2.101:0/4227623175 reap_dead start 2021-02-10T11:43:02.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.949 7f8956a1f700 1 -- 172.21.2.101:0/4227623175 shutdown_connections 2021-02-10T11:43:02.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.949 7f8956a1f700 1 -- 172.21.2.101:0/4227623175 >> 172.21.2.101:0/4227623175 conn(0x7f89500ce450 msgr2=0x7f89500d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:02.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.949 7f8956a1f700 1 -- 172.21.2.101:0/4227623175 shutdown_connections 2021-02-10T11:43:02.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.949 7f8956a1f700 1 -- 172.21.2.101:0/4227623175 wait complete. 2021-02-10T11:43:02.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.964 7f8956a1f700 1 Processor -- start 2021-02-10T11:43:02.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.964 7f8956a1f700 1 -- start start 2021-02-10T11:43:02.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.964 7f8956a1f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89501942b0 0x7f89500b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:02.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.964 7f8956a1f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8950195070 0x7f8950195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:02.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.964 7f8956a1f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89500bb8e0 0x7f89500bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:02.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.964 7f8956a1f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89501946b0 con 0x7f89501942b0 2021-02-10T11:43:02.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.964 7f8956a1f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89500b3b70 con 0x7f89500bb8e0 2021-02-10T11:43:02.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.964 7f8956a1f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89500b3d90 con 0x7f8950195070 2021-02-10T11:43:02.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.965 7f895483b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89501942b0 0x7f89500b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:02.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.965 7f895483b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89501942b0 0x7f89500b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58590/0 (socket says 172.21.2.101:58590) 2021-02-10T11:43:02.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.965 7f895483b700 1 -- 172.21.2.101:0/424773299 learned_addr learned my addr 172.21.2.101:0/424773299 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:02.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.965 7f895483b700 1 -- 172.21.2.101:0/424773299 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8950195070 msgr2=0x7f8950195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:02.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.965 7f895483b700 1 --2- 172.21.2.101:0/424773299 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8950195070 0x7f8950195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:02.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.965 7f895483b700 1 -- 172.21.2.101:0/424773299 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89500bb8e0 msgr2=0x7f89500bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:02.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.965 7f895483b700 1 --2- 172.21.2.101:0/424773299 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89500bb8e0 0x7f89500bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:02.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.965 7f895483b700 1 -- 172.21.2.101:0/424773299 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89501a49e0 con 0x7f89501942b0 2021-02-10T11:43:02.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.965 7f895483b700 1 --2- 172.21.2.101:0/424773299 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89501942b0 0x7f89500b3550 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 rx=0x7f8938001660 tx=0x7f893800a930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:02.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.965 7f894d7fa700 1 -- 172.21.2.101:0/424773299 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f893800ad20 con 0x7f89501942b0 2021-02-10T11:43:02.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.965 7f894d7fa700 1 -- 172.21.2.101:0/424773299 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89380016a0 con 0x7f89501942b0 2021-02-10T11:43:02.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.965 7f894d7fa700 1 -- 172.21.2.101:0/424773299 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8938004330 con 0x7f89501942b0 2021-02-10T11:43:02.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.965 7f8956a1f700 1 -- 172.21.2.101:0/424773299 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89501a49e0 con 0x7f89501942b0 2021-02-10T11:43:02.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.966 7f8956a1f700 1 -- 172.21.2.101:0/424773299 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89500bc5f0 con 0x7f89501942b0 2021-02-10T11:43:02.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.966 7f894d7fa700 1 -- 172.21.2.101:0/424773299 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8938004140 con 0x7f89501942b0 2021-02-10T11:43:02.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.966 7f894d7fa700 1 --2- 172.21.2.101:0/424773299 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8934018700 0x7f893401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:02.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.966 7f894d7fa700 1 -- 172.21.2.101:0/424773299 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8938032610 con 0x7f89501942b0 2021-02-10T11:43:02.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.966 7f894ffff700 1 --2- 172.21.2.101:0/424773299 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8934018700 0x7f893401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:02.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.966 7f894ffff700 1 --2- 172.21.2.101:0/424773299 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8934018700 0x7f893401abc0 crc :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:02.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.996 7f8956a1f700 1 -- 172.21.2.101:0/424773299 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89500bc5f0 con 0x7f89501942b0 2021-02-10T11:43:03.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:02.999 7f894d7fa700 1 -- 172.21.2.101:0/424773299 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f89380044f0 con 0x7f89501942b0 2021-02-10T11:43:03.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:03.132 7f8956a1f700 1 -- 172.21.2.101:0/424773299 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_non_clone_status"]} v 0) v1 -- 0x7f89500bc5f0 con 0x7f89501942b0 2021-02-10T11:43:03.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:03.575 7f894d7fa700 1 -- 172.21.2.101:0/424773299 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_non_clone_status"]}]=0 v651) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8938004140 con 0x7f89501942b0 2021-02-10T11:43:03.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:03.598 7f8956a1f700 1 -- 172.21.2.101:0/424773299 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8934018700 msgr2=0x7f893401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:03.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:03.598 7f8956a1f700 1 --2- 172.21.2.101:0/424773299 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8934018700 0x7f893401abc0 crc :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:03.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:03.598 7f8956a1f700 1 -- 172.21.2.101:0/424773299 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89501942b0 msgr2=0x7f89500b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:03.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:03.598 7f8956a1f700 1 --2- 172.21.2.101:0/424773299 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89501942b0 0x7f89500b3550 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 rx=0x7f8938001660 tx=0x7f893800a930).stop 2021-02-10T11:43:03.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:03.599 7f8956a1f700 1 -- 172.21.2.101:0/424773299 shutdown_connections 2021-02-10T11:43:03.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:03.599 7f8956a1f700 1 --2- 172.21.2.101:0/424773299 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8934018700 0x7f893401abc0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:03.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:03.599 7f8956a1f700 1 --2- 172.21.2.101:0/424773299 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8950195070 0x7f8950195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:03.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:03.599 7f8956a1f700 1 --2- 172.21.2.101:0/424773299 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89500bb8e0 0x7f89500bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:03.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:03.599 7f8956a1f700 1 --2- 172.21.2.101:0/424773299 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89501942b0 0x7f89500b3550 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:03.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:03.599 7f8956a1f700 1 -- 172.21.2.101:0/424773299 >> 172.21.2.101:0/424773299 conn(0x7f89500b3040 msgr2=0x7f89500c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:03.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:03.599 7f8956a1f700 1 -- 172.21.2.101:0/424773299 shutdown_connections 2021-02-10T11:43:03.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:03.599 7f8956a1f700 1 -- 172.21.2.101:0/424773299 wait complete. 2021-02-10T11:43:03.609 INFO:tasks.cephfs_test_runner:test_non_clone_status (tasks.cephfs.test_volumes.TestVolumes) ... ok 2021-02-10T11:43:03.609 INFO:tasks.cephfs_test_runner:Starting test: test_nonexistent_subvolume_group_create (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:43:03.609 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_group_create' 2021-02-10T11:43:04.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.019 7f5c10b0a700 1 Processor -- start 2021-02-10T11:43:04.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.019 7f5c10b0a700 1 -- start start 2021-02-10T11:43:04.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.020 7f5c10b0a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5c0c1b0470 0x7f5c0c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:04.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.020 7f5c10b0a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c0c0ef610 0x7f5c0c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:04.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.020 7f5c10b0a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c0c0f0b50 con 0x7f5c0c0f0030 2021-02-10T11:43:04.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.020 7f5c10b0a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0c0e52e0 con 0x7f5c0c1b0470 2021-02-10T11:43:04.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.020 7f5c10b0a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c0e54d0 con 0x7f5c0c0ef610 2021-02-10T11:43:04.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.020 7f5c0a85e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5c0c1b0470 0x7f5c0c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:04.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.020 7f5c0b05f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f5c0c0f0030 0x7f5c0c0f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59096/0 (socket says 172.21.2.101:59096) 2021-02-10T11:43:04.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.020 7f5c0a85e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5c0c1b0470 0x7f5c0c1b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58598/0 (socket says 172.21.2.101:58598) 2021-02-10T11:43:04.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.020 7f5c0b05f700 1 -- 172.21.2.101:0/4168344408 learned_addr learned my addr 172.21.2.101:0/4168344408 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:04.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.020 7f5c0a85e700 1 -- 172.21.2.101:0/4168344408 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c0c0ef610 msgr2=0x7f5c0c0efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:43:04.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.020 7f5c0a85e700 1 --2- 172.21.2.101:0/4168344408 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c0c0ef610 0x7f5c0c0efa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:04.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.020 7f5c0a85e700 1 -- 172.21.2.101:0/4168344408 >> v1:172.21.2.101:6789/0 conn(0x7f5c0c0f0030 legacy=0x7f5c0c0f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:04.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.020 7f5c0985c700 1 -- 172.21.2.101:0/4168344408 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1339915449 0 0) 0x7f5c04001500 con 0x7f5c0c0f0030 2021-02-10T11:43:04.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.020 7f5c0a85e700 1 -- 172.21.2.101:0/4168344408 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c0f0430 con 0x7f5c0c1b0470 2021-02-10T11:43:04.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.020 7f5c0a85e700 1 --2- 172.21.2.101:0/4168344408 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c0c1b0470 0x7f5c0c1b0870 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 rx=0x7f5bf4006650 tx=0x7f5bf4009a00).ready entity=mon.0 client_cookie=e400b7ee4659ffb server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:04.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.020 7f5c0985c700 1 -- 172.21.2.101:0/4168344408 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5bf40009b0 con 0x7f5c0c1b0470 2021-02-10T11:43:04.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.021 7f5c0985c700 1 -- 172.21.2.101:0/4168344408 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf400c0c0 con 0x7f5c0c1b0470 2021-02-10T11:43:04.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.021 7f5c0985c700 1 -- 172.21.2.101:0/4168344408 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5bf400c5a0 con 0x7f5c0c1b0470 2021-02-10T11:43:04.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.021 7f5c10b0a700 1 -- 172.21.2.101:0/4168344408 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c0c1b0470 msgr2=0x7f5c0c1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:04.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.021 7f5c10b0a700 1 --2- 172.21.2.101:0/4168344408 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c0c1b0470 0x7f5c0c1b0870 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 rx=0x7f5bf4006650 tx=0x7f5bf4009a00).stop 2021-02-10T11:43:04.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.021 7f5c10b0a700 1 -- 172.21.2.101:0/4168344408 shutdown_connections 2021-02-10T11:43:04.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.021 7f5c10b0a700 1 --2- 172.21.2.101:0/4168344408 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c0c0ef610 0x7f5c0c0efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:04.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.021 7f5c10b0a700 1 --2- 172.21.2.101:0/4168344408 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c0c1b0470 0x7f5c0c1b0870 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:04.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.021 7f5c10b0a700 1 -- 172.21.2.101:0/4168344408 >> 172.21.2.101:0/4168344408 conn(0x7f5c0c0ce3e0 msgr2=0x7f5c0c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:04.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.021 7f5c10b0a700 1 -- 172.21.2.101:0/4168344408 shutdown_connections 2021-02-10T11:43:04.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.021 7f5c10b0a700 1 -- 172.21.2.101:0/4168344408 wait complete. 2021-02-10T11:43:04.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.036 7f5c10b0a700 1 Processor -- start 2021-02-10T11:43:04.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.036 7f5c10b0a700 1 -- start start 2021-02-10T11:43:04.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.036 7f5c10b0a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c0c0aa8d0 0x7f5c0c19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:04.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.036 7f5c10b0a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c0c19e9d0 0x7f5c0c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:04.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.036 7f5c10b0a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5c0c19ad40 0x7f5c0c19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:04.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.036 7f5c10b0a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c19b1f0 con 0x7f5c0c0aa8d0 2021-02-10T11:43:04.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.036 7f5c10b0a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c109b30 con 0x7f5c0c19ad40 2021-02-10T11:43:04.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.036 7f5c10b0a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c099620 con 0x7f5c0c19e9d0 2021-02-10T11:43:04.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.037 7f5c0a85e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c0c0aa8d0 0x7f5c0c19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:04.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.037 7f5c0a85e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c0c0aa8d0 0x7f5c0c19a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58604/0 (socket says 172.21.2.101:58604) 2021-02-10T11:43:04.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.037 7f5c0a85e700 1 -- 172.21.2.101:0/2871228337 learned_addr learned my addr 172.21.2.101:0/2871228337 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:04.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.037 7f5c0a85e700 1 -- 172.21.2.101:0/2871228337 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c0c19e9d0 msgr2=0x7f5c0c19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:04.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.037 7f5c0a85e700 1 --2- 172.21.2.101:0/2871228337 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c0c19e9d0 0x7f5c0c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:04.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.037 7f5c0a85e700 1 -- 172.21.2.101:0/2871228337 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5c0c19ad40 msgr2=0x7f5c0c19b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:04.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.037 7f5c0a85e700 1 --2- 172.21.2.101:0/2871228337 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5c0c19ad40 0x7f5c0c19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:04.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.037 7f5c0a85e700 1 -- 172.21.2.101:0/2871228337 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c109640 con 0x7f5c0c0aa8d0 2021-02-10T11:43:04.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.037 7f5c0a85e700 1 --2- 172.21.2.101:0/2871228337 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c0c0aa8d0 0x7f5c0c19a100 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 rx=0x7f5bf400b640 tx=0x7f5bf400b9e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:04.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.037 7f5c037fe700 1 -- 172.21.2.101:0/2871228337 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5bf400bdd0 con 0x7f5c0c0aa8d0 2021-02-10T11:43:04.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.037 7f5c037fe700 1 -- 172.21.2.101:0/2871228337 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf4000c90 con 0x7f5c0c0aa8d0 2021-02-10T11:43:04.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.037 7f5c10b0a700 1 -- 172.21.2.101:0/2871228337 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c0c109640 con 0x7f5c0c0aa8d0 2021-02-10T11:43:04.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.038 7f5c037fe700 1 -- 172.21.2.101:0/2871228337 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5bf40022b0 con 0x7f5c0c0aa8d0 2021-02-10T11:43:04.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.038 7f5c10b0a700 1 -- 172.21.2.101:0/2871228337 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c0c109640 con 0x7f5c0c0aa8d0 2021-02-10T11:43:04.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.038 7f5c037fe700 1 -- 172.21.2.101:0/2871228337 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5bf4006a10 con 0x7f5c0c0aa8d0 2021-02-10T11:43:04.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.038 7f5c037fe700 1 --2- 172.21.2.101:0/2871228337 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5bf00186c0 0x7f5bf001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:04.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.038 7f5c037fe700 1 -- 172.21.2.101:0/2871228337 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5bf40325e0 con 0x7f5c0c0aa8d0 2021-02-10T11:43:04.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.038 7f5c0a05d700 1 --2- 172.21.2.101:0/2871228337 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5bf00186c0 0x7f5bf001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:04.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.039 7f5c0a05d700 1 --2- 172.21.2.101:0/2871228337 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5bf00186c0 0x7f5bf001abc0 crc :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:04.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.069 7f5c10b0a700 1 -- 172.21.2.101:0/2871228337 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c0c0f0b50 con 0x7f5c0c0aa8d0 2021-02-10T11:43:04.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.072 7f5c037fe700 1 -- 172.21.2.101:0/2871228337 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5bf4030d90 con 0x7f5c0c0aa8d0 2021-02-10T11:43:04.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.204 7f5c10b0a700 1 -- 172.21.2.101:0/2871228337 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_group_create"]} v 0) v1 -- 0x7f5c0c109ae0 con 0x7f5c0c0aa8d0 2021-02-10T11:43:04.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.580 7f5c037fe700 1 -- 172.21.2.101:0/2871228337 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_group_create"]}]=0 v652) v1 ==== 159+0+0 (secure 0 0 0) 0x7f5bf4006a10 con 0x7f5c0c0aa8d0 2021-02-10T11:43:04.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.621 7f5c10b0a700 1 -- 172.21.2.101:0/2871228337 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5bf00186c0 msgr2=0x7f5bf001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:04.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.621 7f5c10b0a700 1 --2- 172.21.2.101:0/2871228337 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5bf00186c0 0x7f5bf001abc0 crc :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:04.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.621 7f5c10b0a700 1 -- 172.21.2.101:0/2871228337 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c0c0aa8d0 msgr2=0x7f5c0c19a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:04.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.621 7f5c10b0a700 1 --2- 172.21.2.101:0/2871228337 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c0c0aa8d0 0x7f5c0c19a100 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 rx=0x7f5bf400b640 tx=0x7f5bf400b9e0).stop 2021-02-10T11:43:04.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.621 7f5c10b0a700 1 -- 172.21.2.101:0/2871228337 shutdown_connections 2021-02-10T11:43:04.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.622 7f5c10b0a700 1 --2- 172.21.2.101:0/2871228337 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5bf00186c0 0x7f5bf001abc0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:04.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.622 7f5c10b0a700 1 --2- 172.21.2.101:0/2871228337 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c0c19e9d0 0x7f5c0c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:04.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.622 7f5c10b0a700 1 --2- 172.21.2.101:0/2871228337 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5c0c19ad40 0x7f5c0c19b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:04.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.622 7f5c10b0a700 1 --2- 172.21.2.101:0/2871228337 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c0c0aa8d0 0x7f5c0c19a100 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:04.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.622 7f5c10b0a700 1 -- 172.21.2.101:0/2871228337 >> 172.21.2.101:0/2871228337 conn(0x7f5c0c19f7b0 msgr2=0x7f5c0c18f380 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:04.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.622 7f5c10b0a700 1 -- 172.21.2.101:0/2871228337 shutdown_connections 2021-02-10T11:43:04.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:04.622 7f5c10b0a700 1 -- 172.21.2.101:0/2871228337 wait complete. 2021-02-10T11:43:04.631 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:43:05.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.045 7ff46a64f700 1 Processor -- start 2021-02-10T11:43:05.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.045 7ff46a64f700 1 -- start start 2021-02-10T11:43:05.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.046 7ff46a64f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff4640d1860 0x7ff4641b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:05.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.046 7ff46a64f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff4641b0ad0 0x7ff464108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:05.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.046 7ff46a64f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4640fe1b0 con 0x7ff4640d1c60 2021-02-10T11:43:05.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.046 7ff46a64f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff464109a50 con 0x7ff4640d1860 2021-02-10T11:43:05.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.046 7ff46a64f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff464109740 con 0x7ff4641b0ad0 2021-02-10T11:43:05.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.046 7ff463fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff4640d1860 0x7ff4641b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:05.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.046 7ff463fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff4640d1860 0x7ff4641b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58620/0 (socket says 172.21.2.101:58620) 2021-02-10T11:43:05.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.046 7ff463fff700 1 -- 172.21.2.101:0/2835961665 learned_addr learned my addr 172.21.2.101:0/2835961665 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:05.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.046 7ff463fff700 1 -- 172.21.2.101:0/2835961665 >> v2:172.21.2.108:3300/0 conn(0x7ff4641b0ad0 msgr2=0x7ff464108930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:43:05.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.046 7ff463fff700 1 --2- 172.21.2.101:0/2835961665 >> v2:172.21.2.108:3300/0 conn(0x7ff4641b0ad0 0x7ff464108930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:05.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.046 7ff463fff700 1 -- 172.21.2.101:0/2835961665 >> v1:172.21.2.108:6789/0 conn(0x7ff4640d1c60 legacy=0x7ff464108f50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:43:05.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.046 7ff463fff700 1 -- 172.21.2.101:0/2835961665 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff464088070 con 0x7ff4640d1860 2021-02-10T11:43:05.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.046 7ff463fff700 1 --2- 172.21.2.101:0/2835961665 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4640d1860 0x7ff4641b04b0 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 rx=0x7ff454006700 tx=0x7ff454009b80).ready entity=mon.0 client_cookie=3e4e3a548a771ec8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:05.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.047 7ff462ffd700 1 -- 172.21.2.101:0/2835961665 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff45400c0c0 con 0x7ff4640d1860 2021-02-10T11:43:05.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.047 7ff462ffd700 1 -- 172.21.2.101:0/2835961665 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45400c440 con 0x7ff4640d1860 2021-02-10T11:43:05.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.047 7ff462ffd700 1 -- 172.21.2.101:0/2835961665 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff45400c920 con 0x7ff4640d1860 2021-02-10T11:43:05.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.047 7ff46a64f700 1 -- 172.21.2.101:0/2835961665 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4640d1860 msgr2=0x7ff4641b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:05.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.047 7ff46a64f700 1 --2- 172.21.2.101:0/2835961665 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4640d1860 0x7ff4641b04b0 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 rx=0x7ff454006700 tx=0x7ff454009b80).stop 2021-02-10T11:43:05.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.047 7ff46a64f700 1 -- 172.21.2.101:0/2835961665 shutdown_connections 2021-02-10T11:43:05.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.047 7ff46a64f700 1 --2- 172.21.2.101:0/2835961665 >> v2:172.21.2.108:3300/0 conn(0x7ff4641b0ad0 0x7ff464108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:05.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.047 7ff46a64f700 1 --2- 172.21.2.101:0/2835961665 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4640d1860 0x7ff4641b04b0 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:05.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.047 7ff46a64f700 1 -- 172.21.2.101:0/2835961665 >> 172.21.2.101:0/2835961665 conn(0x7ff4640ce410 msgr2=0x7ff4640d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:05.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.047 7ff46a64f700 1 -- 172.21.2.101:0/2835961665 shutdown_connections 2021-02-10T11:43:05.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.047 7ff46a64f700 1 -- 172.21.2.101:0/2835961665 wait complete. 2021-02-10T11:43:05.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.062 7ff46a64f700 1 Processor -- start 2021-02-10T11:43:05.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.062 7ff46a64f700 1 -- start start 2021-02-10T11:43:05.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.062 7ff46a64f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4641a05e0 0x7ff46418b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:05.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.062 7ff46a64f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff4640b8780 0x7ff4640bfa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:05.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.062 7ff46a64f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4640c0030 0x7ff464189e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:05.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.062 7ff46a64f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4640b82a0 con 0x7ff4640c0030 2021-02-10T11:43:05.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.062 7ff46a64f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4640c0430 con 0x7ff4640b8780 2021-02-10T11:43:05.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.062 7ff46a64f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4640c0650 con 0x7ff4641a05e0 2021-02-10T11:43:05.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.062 7ff468c6c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4640c0030 0x7ff464189e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:05.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.062 7ff468c6c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4640c0030 0x7ff464189e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58626/0 (socket says 172.21.2.101:58626) 2021-02-10T11:43:05.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.062 7ff468c6c700 1 -- 172.21.2.101:0/78626637 learned_addr learned my addr 172.21.2.101:0/78626637 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:05.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.063 7ff468c6c700 1 -- 172.21.2.101:0/78626637 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4641a05e0 msgr2=0x7ff46418b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:05.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.063 7ff468c6c700 1 --2- 172.21.2.101:0/78626637 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4641a05e0 0x7ff46418b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:05.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.063 7ff468c6c700 1 -- 172.21.2.101:0/78626637 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff4640b8780 msgr2=0x7ff4640bfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:05.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.063 7ff468c6c700 1 --2- 172.21.2.101:0/78626637 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff4640b8780 0x7ff4640bfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:05.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.063 7ff468c6c700 1 -- 172.21.2.101:0/78626637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46418a490 con 0x7ff4640c0030 2021-02-10T11:43:05.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.063 7ff468c6c700 1 --2- 172.21.2.101:0/78626637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4640c0030 0x7ff464189e70 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 rx=0x7ff45c009940 tx=0x7ff45c00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:05.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.063 7ff460ff9700 1 -- 172.21.2.101:0/78626637 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff45c003030 con 0x7ff4640c0030 2021-02-10T11:43:05.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.063 7ff460ff9700 1 -- 172.21.2.101:0/78626637 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45c001270 con 0x7ff4640c0030 2021-02-10T11:43:05.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.063 7ff46a64f700 1 -- 172.21.2.101:0/78626637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff46418a490 con 0x7ff4640c0030 2021-02-10T11:43:05.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.063 7ff460ff9700 1 -- 172.21.2.101:0/78626637 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff45c001790 con 0x7ff4640c0030 2021-02-10T11:43:05.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.063 7ff46a64f700 1 -- 172.21.2.101:0/78626637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff46418aa20 con 0x7ff4640c0030 2021-02-10T11:43:05.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.064 7ff460ff9700 1 -- 172.21.2.101:0/78626637 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff45c00a000 con 0x7ff4640c0030 2021-02-10T11:43:05.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.064 7ff460ff9700 1 --2- 172.21.2.101:0/78626637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4400186c0 0x7ff44001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:05.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.064 7ff460ff9700 1 -- 172.21.2.101:0/78626637 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff45c032990 con 0x7ff4640c0030 2021-02-10T11:43:05.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.064 7ff463fff700 1 --2- 172.21.2.101:0/78626637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4400186c0 0x7ff44001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:05.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.064 7ff463fff700 1 --2- 172.21.2.101:0/78626637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4400186c0 0x7ff44001abc0 crc :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:05.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.096 7ff46a64f700 1 -- 172.21.2.101:0/78626637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff464082a20 con 0x7ff4640c0030 2021-02-10T11:43:05.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.098 7ff460ff9700 1 -- 172.21.2.101:0/78626637 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff45c000f20 con 0x7ff4640c0030 2021-02-10T11:43:05.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.252 7ff46a64f700 1 -- 172.21.2.101:0/78626637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff464082a20 con 0x7ff4640c0030 2021-02-10T11:43:05.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.255 7ff460ff9700 1 -- 172.21.2.101:0/78626637 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff45c00a000 con 0x7ff4640c0030 2021-02-10T11:43:05.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.260 7ff46a64f700 1 -- 172.21.2.101:0/78626637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4400186c0 msgr2=0x7ff44001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:05.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.260 7ff46a64f700 1 --2- 172.21.2.101:0/78626637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4400186c0 0x7ff44001abc0 crc :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:05.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.260 7ff46a64f700 1 -- 172.21.2.101:0/78626637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4640c0030 msgr2=0x7ff464189e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:05.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.260 7ff46a64f700 1 --2- 172.21.2.101:0/78626637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4640c0030 0x7ff464189e70 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 rx=0x7ff45c009940 tx=0x7ff45c00bcf0).stop 2021-02-10T11:43:05.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.261 7ff46a64f700 1 -- 172.21.2.101:0/78626637 shutdown_connections 2021-02-10T11:43:05.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.261 7ff46a64f700 1 --2- 172.21.2.101:0/78626637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff4640c0030 0x7ff464189e70 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:05.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.261 7ff46a64f700 1 --2- 172.21.2.101:0/78626637 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff4640b8780 0x7ff4640bfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:05.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.261 7ff46a64f700 1 --2- 172.21.2.101:0/78626637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4400186c0 0x7ff44001abc0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:05.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.261 7ff46a64f700 1 --2- 172.21.2.101:0/78626637 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4641a05e0 0x7ff46418b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:05.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.261 7ff46a64f700 1 -- 172.21.2.101:0/78626637 >> 172.21.2.101:0/78626637 conn(0x7ff4640a6a80 msgr2=0x7ff4640ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:05.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.261 7ff46a64f700 1 -- 172.21.2.101:0/78626637 shutdown_connections 2021-02-10T11:43:05.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:05.261 7ff46a64f700 1 -- 172.21.2.101:0/78626637 wait complete. 2021-02-10T11:43:05.270 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:43:05.270 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:43:05.616 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:43:05.817 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:43:05.820 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:43:05.960 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:43:05.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:05.961 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:43:05.961 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:43:06.295 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2021-02-10T11:43:06.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:43:06.295 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-02-10T11:43:06.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:06.296 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:43:06.296 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:43:06.630 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:43:06.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:43:06.630 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:43:06.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:06.631 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:43:06.631 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:43:06.965 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:43:06.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:43:06.966 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:43:06.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:06.966 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:43:06.966 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:43:07.301 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:43:07.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:43:07.302 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:43:07.302 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json-pretty 2021-02-10T11:43:07.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.715 7f72a2f28700 1 Processor -- start 2021-02-10T11:43:07.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.715 7f72a2f28700 1 -- start start 2021-02-10T11:43:07.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.715 7f72a2f28700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f729c0d0340 0x7f729c0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:07.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.715 7f72a2f28700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f729c0d1c70 0x7f729c0efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:07.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.715 7f72a2f28700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729c0f0b10 con 0x7f729c0d1870 2021-02-10T11:43:07.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.715 7f72a2f28700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729c0e5ed0 con 0x7f729c0d1c70 2021-02-10T11:43:07.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.715 7f72a2f28700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f729c1b0be0 con 0x7f729c0d0340 2021-02-10T11:43:07.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.716 7f72a1545700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f729c0d1c70 0x7f729c0efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:07.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.716 7f729bfff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f729c0d0340 0x7f729c0ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:07.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.716 7f72a1545700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f729c0d1c70 0x7f729c0efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58634/0 (socket says 172.21.2.101:58634) 2021-02-10T11:43:07.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.716 7f72a1545700 1 -- 172.21.2.101:0/247226693 learned_addr learned my addr 172.21.2.101:0/247226693 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:07.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.716 7f72a1545700 1 -- 172.21.2.101:0/247226693 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f729c0d0340 msgr2=0x7f729c0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:07.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.716 7f72a1545700 1 --2- 172.21.2.101:0/247226693 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f729c0d0340 0x7f729c0ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:07.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.716 7f72a1545700 1 -- 172.21.2.101:0/247226693 >> v1:172.21.2.108:6789/0 conn(0x7f729c0d1870 legacy=0x7f729c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:07.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.716 7f72a1545700 1 -- 172.21.2.101:0/247226693 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729c0f20a0 con 0x7f729c0d1c70 2021-02-10T11:43:07.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.716 7f72a1545700 1 --2- 172.21.2.101:0/247226693 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f729c0d1c70 0x7f729c0efc60 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 rx=0x7f7294006f00 tx=0x7f7294009f10).ready entity=mon.0 client_cookie=887b31eb7e2a481d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:07.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.716 7f729b7fe700 1 -- 172.21.2.101:0/247226693 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f729400b3b0 con 0x7f729c0d1c70 2021-02-10T11:43:07.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.716 7f729b7fe700 1 -- 172.21.2.101:0/247226693 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729400b730 con 0x7f729c0d1c70 2021-02-10T11:43:07.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.716 7f729b7fe700 1 -- 172.21.2.101:0/247226693 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f729400bc50 con 0x7f729c0d1c70 2021-02-10T11:43:07.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.717 7f72a2f28700 1 -- 172.21.2.101:0/247226693 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f729c0d1c70 msgr2=0x7f729c0efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:07.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.717 7f72a2f28700 1 --2- 172.21.2.101:0/247226693 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f729c0d1c70 0x7f729c0efc60 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 rx=0x7f7294006f00 tx=0x7f7294009f10).stop 2021-02-10T11:43:07.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.717 7f72a2f28700 1 -- 172.21.2.101:0/247226693 shutdown_connections 2021-02-10T11:43:07.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.717 7f72a2f28700 1 --2- 172.21.2.101:0/247226693 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f729c0d0340 0x7f729c0ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:07.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.717 7f72a2f28700 1 --2- 172.21.2.101:0/247226693 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f729c0d1c70 0x7f729c0efc60 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:07.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.717 7f72a2f28700 1 -- 172.21.2.101:0/247226693 >> 172.21.2.101:0/247226693 conn(0x7f729c0ce450 msgr2=0x7f729c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:07.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.717 7f72a2f28700 1 -- 172.21.2.101:0/247226693 shutdown_connections 2021-02-10T11:43:07.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.717 7f72a2f28700 1 -- 172.21.2.101:0/247226693 wait complete. 2021-02-10T11:43:07.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.732 7f72a2f28700 1 Processor -- start 2021-02-10T11:43:07.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.732 7f72a2f28700 1 -- start start 2021-02-10T11:43:07.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.732 7f72a2f28700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f729c0ada90 0x7f729c0ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:07.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.732 7f72a2f28700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f729c099d20 0x7f729c09a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:07.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.732 7f72a2f28700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f729c09a740 0x7f729c097bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:07.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.732 7f72a2f28700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c0997a0 con 0x7f729c09a740 2021-02-10T11:43:07.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.732 7f72a2f28700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c18f0c0 con 0x7f729c099d20 2021-02-10T11:43:07.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.732 7f72a2f28700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f729c09ab40 con 0x7f729c0ada90 2021-02-10T11:43:07.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.732 7f72a1545700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f729c09a740 0x7f729c097bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:07.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.732 7f72a1545700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f729c09a740 0x7f729c097bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58638/0 (socket says 172.21.2.101:58638) 2021-02-10T11:43:07.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.732 7f72a1545700 1 -- 172.21.2.101:0/1686666199 learned_addr learned my addr 172.21.2.101:0/1686666199 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:07.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.733 7f729bfff700 1 --2- 172.21.2.101:0/1686666199 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f729c099d20 0x7f729c09a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:07.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.733 7f72a0d44700 1 --2- 172.21.2.101:0/1686666199 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f729c0ada90 0x7f729c0ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:07.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.733 7f72a1545700 1 -- 172.21.2.101:0/1686666199 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f729c0ada90 msgr2=0x7f729c0ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:07.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.733 7f72a1545700 1 --2- 172.21.2.101:0/1686666199 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f729c0ada90 0x7f729c0ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:07.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.733 7f72a1545700 1 -- 172.21.2.101:0/1686666199 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f729c099d20 msgr2=0x7f729c09a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:07.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.733 7f72a1545700 1 --2- 172.21.2.101:0/1686666199 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f729c099d20 0x7f729c09a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:07.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.733 7f72a1545700 1 -- 172.21.2.101:0/1686666199 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729c09ad60 con 0x7f729c09a740 2021-02-10T11:43:07.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.733 7f72a1545700 1 --2- 172.21.2.101:0/1686666199 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f729c09a740 0x7f729c097bc0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 rx=0x7f7294009390 tx=0x7f729400bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:07.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.733 7f72997fa700 1 -- 172.21.2.101:0/1686666199 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7294003030 con 0x7f729c09a740 2021-02-10T11:43:07.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.733 7f72997fa700 1 -- 172.21.2.101:0/1686666199 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7294006c80 con 0x7f729c09a740 2021-02-10T11:43:07.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.733 7f72a2f28700 1 -- 172.21.2.101:0/1686666199 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729c09ad60 con 0x7f729c09a740 2021-02-10T11:43:07.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.733 7f72997fa700 1 -- 172.21.2.101:0/1686666199 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7294001480 con 0x7f729c09a740 2021-02-10T11:43:07.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.733 7f72a2f28700 1 -- 172.21.2.101:0/1686666199 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729c09ad60 con 0x7f729c09a740 2021-02-10T11:43:07.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.734 7f72997fa700 1 -- 172.21.2.101:0/1686666199 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7294001270 con 0x7f729c09a740 2021-02-10T11:43:07.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.734 7f72997fa700 1 --2- 172.21.2.101:0/1686666199 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7280018700 0x7f728001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:07.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.734 7f72997fa700 1 -- 172.21.2.101:0/1686666199 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f72940329a0 con 0x7f729c09a740 2021-02-10T11:43:07.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.734 7f72a0d44700 1 --2- 172.21.2.101:0/1686666199 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7280018700 0x7f728001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:07.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.734 7f72a0d44700 1 --2- 172.21.2.101:0/1686666199 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7280018700 0x7f728001abc0 crc :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:07.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.765 7f72a2f28700 1 -- 172.21.2.101:0/1686666199 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729c0ebc00 con 0x7f729c09a740 2021-02-10T11:43:07.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.768 7f72997fa700 1 -- 172.21.2.101:0/1686666199 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7294001ef0 con 0x7f729c09a740 2021-02-10T11:43:07.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.893 7f72a2f28700 1 -- 172.21.2.101:0/1686666199 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f729c0ebc00 con 0x7f729c09a740 2021-02-10T11:43:07.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.894 7f72997fa700 1 -- 172.21.2.101:0/1686666199 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v129) v1 ==== 81+0+30374 (secure 0 0 0) 0x7f7294006c80 con 0x7f729c09a740 2021-02-10T11:43:07.895 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:43:07.895 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:43:07.895 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 129, 2021-02-10T11:43:07.895 INFO:teuthology.orchestra.run.gibba001.stdout: "fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:43:07.895 INFO:teuthology.orchestra.run.gibba001.stdout: "created": "2021-02-10 11:12:36.788366", 2021-02-10T11:43:07.896 INFO:teuthology.orchestra.run.gibba001.stdout: "modified": "2021-02-10 11:42:31.764498", 2021-02-10T11:43:07.896 INFO:teuthology.orchestra.run.gibba001.stdout: "last_up_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:43:07.896 INFO:teuthology.orchestra.run.gibba001.stdout: "last_in_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:43:07.896 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-02-10T11:43:07.896 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_num": 5799936, 2021-02-10T11:43:07.896 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_set": [ 2021-02-10T11:43:07.896 INFO:teuthology.orchestra.run.gibba001.stdout: "pglog_hardlimit", 2021-02-10T11:43:07.896 INFO:teuthology.orchestra.run.gibba001.stdout: "purged_snapdirs", 2021-02-10T11:43:07.896 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_deletes", 2021-02-10T11:43:07.896 INFO:teuthology.orchestra.run.gibba001.stdout: "sortbitwise" 2021-02-10T11:43:07.896 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:43:07.897 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_version": 4, 2021-02-10T11:43:07.897 INFO:teuthology.orchestra.run.gibba001.stdout: "full_ratio": 0.94999998807907104, 2021-02-10T11:43:07.897 INFO:teuthology.orchestra.run.gibba001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-02-10T11:43:07.897 INFO:teuthology.orchestra.run.gibba001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-02-10T11:43:07.897 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_snapshot": "", 2021-02-10T11:43:07.897 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_max": 27, 2021-02-10T11:43:07.897 INFO:teuthology.orchestra.run.gibba001.stdout: "max_osd": 8, 2021-02-10T11:43:07.897 INFO:teuthology.orchestra.run.gibba001.stdout: "require_min_compat_client": "jewel", 2021-02-10T11:43:07.897 INFO:teuthology.orchestra.run.gibba001.stdout: "min_compat_client": "jewel", 2021-02-10T11:43:07.897 INFO:teuthology.orchestra.run.gibba001.stdout: "require_osd_release": "nautilus", 2021-02-10T11:43:07.898 INFO:teuthology.orchestra.run.gibba001.stdout: "pools": [ 2021-02-10T11:43:07.898 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.898 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 1, 2021-02-10T11:43:07.898 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "rbd", 2021-02-10T11:43:07.898 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:13:08.037472", 2021-02-10T11:43:07.898 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:43:07.898 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:43:07.898 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:43:07.898 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:43:07.899 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:43:07.899 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:43:07.899 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:43:07.899 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:43:07.899 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:43:07.899 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:43:07.899 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:43:07.899 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:43:07.899 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:43:07.899 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:43:07.899 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:43:07.899 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:43:07.900 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:43:07.900 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:43:07.900 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:43:07.900 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:43:07.900 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.900 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "16", 2021-02-10T11:43:07.900 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:43:07.900 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:43:07.900 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:43:07.900 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:43:07.900 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:43:07.900 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:43:07.900 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:43:07.901 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:43:07.901 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:43:07.901 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:43:07.901 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:43:07.901 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:43:07.901 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:43:07.901 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:43:07.901 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:43:07.901 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:43:07.901 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:43:07.901 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:43:07.902 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:43:07.902 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:43:07.902 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:43:07.902 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:43:07.902 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:43:07.902 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:43:07.902 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:43:07.902 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:43:07.902 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.903 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:43:07.903 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:43:07.903 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:43:07.903 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:43:07.903 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:43:07.903 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:43:07.903 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:43:07.903 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:43:07.903 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:43:07.903 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:43:07.903 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:43:07.904 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:43:07.904 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:43:07.904 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd": {} 2021-02-10T11:43:07.904 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.904 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.904 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.904 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 26, 2021-02-10T11:43:07.904 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_metadata", 2021-02-10T11:43:07.904 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:42:28.726448", 2021-02-10T11:43:07.904 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:43:07.904 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:43:07.905 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:43:07.905 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:43:07.905 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:43:07.905 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:43:07.905 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:43:07.905 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:43:07.905 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:43:07.905 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:43:07.905 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:43:07.905 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:43:07.906 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:43:07.906 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:43:07.906 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:43:07.906 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:43:07.906 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:43:07.906 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:43:07.906 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:43:07.906 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:43:07.906 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.907 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "128", 2021-02-10T11:43:07.907 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:43:07.907 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:43:07.907 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:43:07.907 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:43:07.907 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:43:07.907 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:43:07.907 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:43:07.907 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:43:07.907 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:43:07.907 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:43:07.907 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:43:07.907 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:43:07.908 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:43:07.908 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:43:07.908 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:43:07.908 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:43:07.908 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:43:07.908 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:43:07.908 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:43:07.908 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:43:07.908 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:43:07.908 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:43:07.908 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:43:07.909 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:43:07.909 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:43:07.909 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:43:07.909 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.909 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:43:07.909 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:43:07.909 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:43:07.909 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:43:07.909 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:43:07.909 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:43:07.909 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:43:07.909 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:43:07.909 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:43:07.910 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:43:07.910 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:43:07.910 INFO:teuthology.orchestra.run.gibba001.stdout: "options": { 2021-02-10T11:43:07.910 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_bias": 4, 2021-02-10T11:43:07.910 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_min": 16, 2021-02-10T11:43:07.910 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_priority": 5 2021-02-10T11:43:07.910 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.910 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:43:07.911 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:43:07.911 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": "cephfs" 2021-02-10T11:43:07.911 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.911 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.911 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.911 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.911 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 27, 2021-02-10T11:43:07.911 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_data", 2021-02-10T11:43:07.911 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:42:29.754869", 2021-02-10T11:43:07.911 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:43:07.911 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:43:07.912 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:43:07.912 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:43:07.912 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:43:07.912 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:43:07.912 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:43:07.912 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:43:07.912 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:43:07.912 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:43:07.912 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:43:07.912 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:43:07.912 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:43:07.913 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:43:07.913 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:43:07.913 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:43:07.913 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:43:07.913 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:43:07.913 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:43:07.913 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:43:07.913 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.913 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "129", 2021-02-10T11:43:07.914 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:43:07.914 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:43:07.914 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:43:07.914 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:43:07.914 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:43:07.914 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:43:07.914 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:43:07.914 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:43:07.914 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:43:07.915 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:43:07.915 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:43:07.915 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:43:07.915 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:43:07.915 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:43:07.915 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:43:07.915 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:43:07.915 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:43:07.915 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:43:07.915 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:43:07.915 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:43:07.916 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:43:07.916 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:43:07.916 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:43:07.916 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:43:07.916 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:43:07.916 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:43:07.916 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.916 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:43:07.916 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:43:07.916 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:43:07.917 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:43:07.917 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:43:07.917 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:43:07.917 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:43:07.917 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:43:07.917 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:43:07.917 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:43:07.917 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:43:07.917 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:43:07.918 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:43:07.918 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:43:07.918 INFO:teuthology.orchestra.run.gibba001.stdout: "data": "cephfs" 2021-02-10T11:43:07.918 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.918 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.918 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.918 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:43:07.918 INFO:teuthology.orchestra.run.gibba001.stdout: "osds": [ 2021-02-10T11:43:07.918 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.918 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:43:07.919 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5", 2021-02-10T11:43:07.919 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:43:07.919 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:43:07.919 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:43:07.919 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:43:07.919 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:43:07.919 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:43:07.919 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:43:07.920 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 127, 2021-02-10T11:43:07.920 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:43:07.920 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:43:07.920 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:43:07.920 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.920 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.920 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.920 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6802", 2021-02-10T11:43:07.920 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:43:07.921 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.921 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.921 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.921 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6805", 2021-02-10T11:43:07.921 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:43:07.921 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.921 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.921 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.921 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:43:07.922 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.922 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.922 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.922 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6807", 2021-02-10T11:43:07.922 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:43:07.922 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.922 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.922 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.922 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6809", 2021-02-10T11:43:07.922 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:43:07.922 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.922 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.923 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.923 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:43:07.923 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.923 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.923 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.923 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6817", 2021-02-10T11:43:07.923 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:43:07.923 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.923 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.923 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.923 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6820", 2021-02-10T11:43:07.924 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:43:07.924 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.924 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.924 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.924 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:43:07.924 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.924 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.924 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.924 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6812", 2021-02-10T11:43:07.924 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:43:07.924 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.925 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.925 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.925 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6814", 2021-02-10T11:43:07.925 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:43:07.925 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.925 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.925 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.925 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6805/25093", 2021-02-10T11:43:07.925 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6809/25093", 2021-02-10T11:43:07.926 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6820/25093", 2021-02-10T11:43:07.926 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6814/25093", 2021-02-10T11:43:07.926 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:43:07.926 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:43:07.926 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:43:07.926 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.926 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.926 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.926 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:43:07.926 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "ddc04ce3-b313-4707-8270-585391e38ecc", 2021-02-10T11:43:07.927 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:43:07.927 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:43:07.927 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:43:07.927 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:43:07.927 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:43:07.927 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:43:07.927 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:43:07.927 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 126, 2021-02-10T11:43:07.927 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:43:07.927 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:43:07.927 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:43:07.928 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.928 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.928 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.928 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6803", 2021-02-10T11:43:07.928 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:43:07.928 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.928 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.928 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.928 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6804", 2021-02-10T11:43:07.928 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:43:07.928 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.929 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.929 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.929 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:43:07.929 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.929 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.929 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.929 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6806", 2021-02-10T11:43:07.929 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:43:07.929 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.929 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.929 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.930 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6808", 2021-02-10T11:43:07.930 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:43:07.930 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.930 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.930 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.930 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:43:07.930 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.930 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.930 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.930 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6818", 2021-02-10T11:43:07.931 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:43:07.931 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.931 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.931 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.931 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6821", 2021-02-10T11:43:07.931 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:43:07.931 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.931 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.931 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.932 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:43:07.932 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.932 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.932 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.932 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6810", 2021-02-10T11:43:07.932 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:43:07.932 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.932 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.932 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.932 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6815", 2021-02-10T11:43:07.932 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:43:07.933 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.933 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.933 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.933 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6804/25092", 2021-02-10T11:43:07.933 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6808/25092", 2021-02-10T11:43:07.933 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6821/25092", 2021-02-10T11:43:07.933 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6815/25092", 2021-02-10T11:43:07.933 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:43:07.933 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:43:07.933 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:43:07.933 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.934 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.934 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.934 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:43:07.934 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3", 2021-02-10T11:43:07.934 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:43:07.934 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:43:07.934 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:43:07.934 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:43:07.934 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:43:07.934 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:43:07.934 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:43:07.934 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 127, 2021-02-10T11:43:07.934 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:43:07.935 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:43:07.935 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:43:07.935 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.935 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.935 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.935 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6811", 2021-02-10T11:43:07.935 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:43:07.935 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.935 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.935 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.936 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6813", 2021-02-10T11:43:07.936 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:43:07.936 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.936 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.936 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.936 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:43:07.936 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.936 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.936 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.936 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6816", 2021-02-10T11:43:07.937 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:43:07.937 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.937 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.937 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.937 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6819", 2021-02-10T11:43:07.937 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:43:07.937 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.937 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.937 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.937 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:43:07.937 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.938 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.938 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.938 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6825", 2021-02-10T11:43:07.938 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:43:07.938 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.938 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.938 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.938 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6827", 2021-02-10T11:43:07.938 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:43:07.938 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.939 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.939 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.939 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:43:07.939 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.939 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.939 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.939 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6822", 2021-02-10T11:43:07.939 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:43:07.939 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.939 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.939 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.940 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6823", 2021-02-10T11:43:07.940 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:43:07.940 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.940 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.940 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.940 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6813/25094", 2021-02-10T11:43:07.940 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6819/25094", 2021-02-10T11:43:07.940 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6827/25094", 2021-02-10T11:43:07.940 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6823/25094", 2021-02-10T11:43:07.940 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:43:07.940 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:43:07.940 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:43:07.941 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.941 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.941 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.941 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:43:07.941 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "0d9ef9b1-98aa-4d37-aa16-6305c7d46d52", 2021-02-10T11:43:07.941 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:43:07.941 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:43:07.941 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:43:07.941 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:43:07.941 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:43:07.941 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:43:07.941 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:43:07.941 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 118, 2021-02-10T11:43:07.941 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:43:07.942 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:43:07.942 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:43:07.942 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.942 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.942 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.942 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6824", 2021-02-10T11:43:07.942 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:43:07.942 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.942 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.942 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.942 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6826", 2021-02-10T11:43:07.942 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:43:07.942 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.942 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.943 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.943 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:43:07.943 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.943 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.943 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.943 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6828", 2021-02-10T11:43:07.943 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:43:07.943 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.943 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.943 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.943 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6829", 2021-02-10T11:43:07.943 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:43:07.943 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.944 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.944 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.944 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:43:07.944 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.944 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.944 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.944 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6832", 2021-02-10T11:43:07.944 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:43:07.944 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.945 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.945 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.945 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6833", 2021-02-10T11:43:07.945 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:43:07.945 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.945 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.945 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.945 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:43:07.945 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.945 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.945 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.945 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6830", 2021-02-10T11:43:07.945 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:43:07.945 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.946 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.946 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.946 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6831", 2021-02-10T11:43:07.946 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:43:07.946 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.946 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.946 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.946 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6826/25095", 2021-02-10T11:43:07.946 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6829/25095", 2021-02-10T11:43:07.946 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6833/25095", 2021-02-10T11:43:07.946 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6831/25095", 2021-02-10T11:43:07.946 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:43:07.946 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:43:07.947 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:43:07.947 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.947 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.947 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.947 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:43:07.947 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "36742c21-bfeb-4e11-bce9-8e745378b605", 2021-02-10T11:43:07.947 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:43:07.947 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:43:07.947 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:43:07.947 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:43:07.947 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:43:07.947 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:43:07.947 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:43:07.947 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 127, 2021-02-10T11:43:07.948 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:43:07.948 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:43:07.948 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:43:07.948 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.948 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.948 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.948 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6816", 2021-02-10T11:43:07.948 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:43:07.948 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.948 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.948 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.948 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6818", 2021-02-10T11:43:07.948 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:43:07.948 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.949 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.949 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.949 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:43:07.949 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.949 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.949 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.949 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6820", 2021-02-10T11:43:07.949 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:43:07.949 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.949 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.949 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.949 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6822", 2021-02-10T11:43:07.949 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:43:07.949 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.950 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.950 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.950 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:43:07.950 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.950 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.950 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.950 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6828", 2021-02-10T11:43:07.950 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:43:07.950 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.950 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.950 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.950 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6830", 2021-02-10T11:43:07.950 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:43:07.951 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.951 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.951 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.951 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:43:07.951 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.951 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.951 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.951 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6824", 2021-02-10T11:43:07.951 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:43:07.951 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.951 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.951 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.951 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6826", 2021-02-10T11:43:07.951 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:43:07.952 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.952 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.952 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.952 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6818/36380", 2021-02-10T11:43:07.952 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6822/36380", 2021-02-10T11:43:07.952 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6830/36380", 2021-02-10T11:43:07.952 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6826/36380", 2021-02-10T11:43:07.952 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:43:07.952 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:43:07.952 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:43:07.952 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.953 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.953 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.953 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:43:07.953 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "88e694e8-fe98-4d1b-88c8-8c9907ece60b", 2021-02-10T11:43:07.953 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:43:07.953 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:43:07.953 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:43:07.953 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:43:07.953 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:43:07.953 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:43:07.953 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:43:07.954 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 127, 2021-02-10T11:43:07.954 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:43:07.954 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:43:07.954 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:43:07.954 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.954 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.954 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.954 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6808", 2021-02-10T11:43:07.954 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:43:07.954 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.954 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.954 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.954 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6809", 2021-02-10T11:43:07.954 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:43:07.955 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.955 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.955 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.955 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:43:07.955 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.955 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.955 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.955 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6810", 2021-02-10T11:43:07.955 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:43:07.955 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.955 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.955 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.955 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6811", 2021-02-10T11:43:07.955 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:43:07.956 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.956 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.956 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.956 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:43:07.956 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.956 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.956 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.956 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6814", 2021-02-10T11:43:07.956 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:43:07.956 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.956 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.956 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.956 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6815", 2021-02-10T11:43:07.956 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:43:07.957 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.957 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.957 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.957 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:43:07.957 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.957 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.957 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.957 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6812", 2021-02-10T11:43:07.957 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:43:07.957 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.957 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.957 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.957 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6813", 2021-02-10T11:43:07.957 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:43:07.958 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.958 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.958 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.958 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6809/36383", 2021-02-10T11:43:07.958 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6811/36383", 2021-02-10T11:43:07.958 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6815/36383", 2021-02-10T11:43:07.958 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6813/36383", 2021-02-10T11:43:07.958 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:43:07.958 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:43:07.958 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:43:07.958 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.958 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.958 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.958 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:43:07.959 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "87a13c6c-cf85-4aaf-916b-e73913484a49", 2021-02-10T11:43:07.959 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:43:07.959 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:43:07.959 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:43:07.959 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:43:07.959 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:43:07.959 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:43:07.959 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:43:07.959 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 127, 2021-02-10T11:43:07.959 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:43:07.959 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:43:07.959 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:43:07.959 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.959 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.960 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.960 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6800", 2021-02-10T11:43:07.960 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:43:07.960 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.960 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.960 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.960 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6801", 2021-02-10T11:43:07.960 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:43:07.960 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.960 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.960 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.960 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:43:07.960 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.960 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.961 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.961 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6802", 2021-02-10T11:43:07.961 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:43:07.961 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.961 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.961 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.961 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6803", 2021-02-10T11:43:07.961 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:43:07.962 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.962 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.962 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.962 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:43:07.962 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.962 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.962 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.962 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6806", 2021-02-10T11:43:07.962 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:43:07.962 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.962 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.962 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.963 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6807", 2021-02-10T11:43:07.963 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:43:07.963 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.963 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.963 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.963 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:43:07.963 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.963 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.963 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.963 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6804", 2021-02-10T11:43:07.963 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:43:07.964 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.964 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.964 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.964 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6805", 2021-02-10T11:43:07.964 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:43:07.964 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.964 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.964 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.964 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6801/36381", 2021-02-10T11:43:07.964 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6803/36381", 2021-02-10T11:43:07.964 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6807/36381", 2021-02-10T11:43:07.965 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6805/36381", 2021-02-10T11:43:07.965 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:43:07.965 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:43:07.965 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:43:07.965 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.965 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.965 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.965 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:43:07.965 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "7b90823f-854c-4592-a95b-a7fa98f93254", 2021-02-10T11:43:07.965 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:43:07.965 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:43:07.966 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:43:07.966 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:43:07.966 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:43:07.966 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:43:07.966 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:43:07.966 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 126, 2021-02-10T11:43:07.966 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:43:07.966 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:43:07.966 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:43:07.966 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.966 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.967 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.967 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6817", 2021-02-10T11:43:07.967 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:43:07.967 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.967 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.967 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.967 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6819", 2021-02-10T11:43:07.967 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:43:07.967 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.967 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.967 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.967 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:43:07.967 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.968 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.968 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.968 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6821", 2021-02-10T11:43:07.968 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:43:07.968 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.968 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.968 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.968 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6823", 2021-02-10T11:43:07.968 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:43:07.969 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.969 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.969 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.969 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:43:07.969 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.969 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.969 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.969 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6829", 2021-02-10T11:43:07.969 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:43:07.970 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.970 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.970 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.970 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6831", 2021-02-10T11:43:07.970 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:43:07.970 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.970 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.970 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.970 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:43:07.971 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:07.971 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.971 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:07.971 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6825", 2021-02-10T11:43:07.971 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:43:07.971 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.971 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.971 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:07.971 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6827", 2021-02-10T11:43:07.971 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:43:07.972 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.972 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.972 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.972 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6819/36382", 2021-02-10T11:43:07.972 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6823/36382", 2021-02-10T11:43:07.972 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6831/36382", 2021-02-10T11:43:07.972 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6827/36382", 2021-02-10T11:43:07.972 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:43:07.972 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:43:07.972 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:43:07.972 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:07.972 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.972 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:43:07.972 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_xinfo": [ 2021-02-10T11:43:07.973 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.973 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:43:07.973 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:43:07.973 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:43:07.973 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:43:07.973 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:43:07.973 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:43:07.973 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.973 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.973 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:43:07.973 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:43:07.974 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:43:07.974 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:43:07.974 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:43:07.974 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:43:07.974 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.974 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.974 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:43:07.974 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:43:07.974 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:43:07.974 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:43:07.974 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:43:07.975 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:43:07.975 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.975 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.975 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:43:07.975 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:43:07.975 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:43:07.975 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:43:07.975 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:43:07.975 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:43:07.975 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.975 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.975 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:43:07.976 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:43:07.976 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:43:07.976 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:43:07.976 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:43:07.976 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:43:07.976 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.976 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.976 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:43:07.976 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:43:07.976 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:43:07.976 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:43:07.976 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:43:07.977 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:43:07.977 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.977 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.977 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:43:07.977 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:43:07.977 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:43:07.977 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:43:07.977 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:43:07.977 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:43:07.977 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.977 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:07.977 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:43:07.977 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:43:07.978 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:43:07.978 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:43:07.978 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:43:07.978 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:43:07.978 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.978 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:43:07.978 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap": [], 2021-02-10T11:43:07.978 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap_items": [], 2021-02-10T11:43:07.978 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_temp": [], 2021-02-10T11:43:07.979 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_temp": [], 2021-02-10T11:43:07.979 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklist": {}, 2021-02-10T11:43:07.979 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profiles": { 2021-02-10T11:43:07.979 INFO:teuthology.orchestra.run.gibba001.stdout: "default": { 2021-02-10T11:43:07.979 INFO:teuthology.orchestra.run.gibba001.stdout: "crush-failure-domain": "osd", 2021-02-10T11:43:07.979 INFO:teuthology.orchestra.run.gibba001.stdout: "k": "2", 2021-02-10T11:43:07.979 INFO:teuthology.orchestra.run.gibba001.stdout: "m": "1", 2021-02-10T11:43:07.979 INFO:teuthology.orchestra.run.gibba001.stdout: "plugin": "jerasure", 2021-02-10T11:43:07.979 INFO:teuthology.orchestra.run.gibba001.stdout: "ruleset-failure-domain": "osd", 2021-02-10T11:43:07.979 INFO:teuthology.orchestra.run.gibba001.stdout: "technique": "reed_sol_van" 2021-02-10T11:43:07.979 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:07.979 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:07.980 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps_queue": [], 2021-02-10T11:43:07.980 INFO:teuthology.orchestra.run.gibba001.stdout: "new_removed_snaps": [], 2021-02-10T11:43:07.980 INFO:teuthology.orchestra.run.gibba001.stdout: "new_purged_snaps": [], 2021-02-10T11:43:07.980 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_node_flags": {}, 2021-02-10T11:43:07.980 INFO:teuthology.orchestra.run.gibba001.stdout: "device_class_flags": {} 2021-02-10T11:43:07.980 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:43:07.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.896 7f72a2f28700 1 -- 172.21.2.101:0/1686666199 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7280018700 msgr2=0x7f728001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:07.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.896 7f72a2f28700 1 --2- 172.21.2.101:0/1686666199 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7280018700 0x7f728001abc0 crc :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:07.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.896 7f72a2f28700 1 -- 172.21.2.101:0/1686666199 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f729c09a740 msgr2=0x7f729c097bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:07.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.896 7f72a2f28700 1 --2- 172.21.2.101:0/1686666199 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f729c09a740 0x7f729c097bc0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 rx=0x7f7294009390 tx=0x7f729400bd40).stop 2021-02-10T11:43:07.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.896 7f72a2f28700 1 -- 172.21.2.101:0/1686666199 shutdown_connections 2021-02-10T11:43:07.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.896 7f72a2f28700 1 --2- 172.21.2.101:0/1686666199 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f729c09a740 0x7f729c097bc0 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:07.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.896 7f72a2f28700 1 --2- 172.21.2.101:0/1686666199 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f729c099d20 0x7f729c09a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:07.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.896 7f72a2f28700 1 --2- 172.21.2.101:0/1686666199 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7280018700 0x7f728001abc0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:07.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.896 7f72a2f28700 1 --2- 172.21.2.101:0/1686666199 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f729c0ada90 0x7f729c0ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:07.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.896 7f72a2f28700 1 -- 172.21.2.101:0/1686666199 >> 172.21.2.101:0/1686666199 conn(0x7f729c0f0550 msgr2=0x7f729c191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:07.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.896 7f72a2f28700 1 -- 172.21.2.101:0/1686666199 shutdown_connections 2021-02-10T11:43:07.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:07.896 7f72a2f28700 1 -- 172.21.2.101:0/1686666199 wait complete. 2021-02-10T11:43:07.984 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:43:08.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.395 7fe09183c700 1 Processor -- start 2021-02-10T11:43:08.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.395 7fe09183c700 1 -- start start 2021-02-10T11:43:08.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.395 7fe09183c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe08c0d1870 0x7fe08c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:08.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.395 7fe09183c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe08c0f0b10 con 0x7fe08c0f0650 2021-02-10T11:43:08.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.395 7fe09183c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe08c0d0310 con 0x7fe08c1b0a10 2021-02-10T11:43:08.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.395 7fe09183c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe08c0d0530 con 0x7fe08c0d1870 2021-02-10T11:43:08.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.396 7fe08bd75700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe08c0f0650 0x7fe08c0cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59138/0 (socket says 172.21.2.101:59138) 2021-02-10T11:43:08.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.396 7fe08bd75700 1 -- 172.21.2.101:0/2236673913 learned_addr learned my addr 172.21.2.101:0/2236673913 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:08.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.396 7fe08a572700 1 -- 172.21.2.101:0/2236673913 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 320453863 0 0) 0x7fe084001500 con 0x7fe08c0f0650 2021-02-10T11:43:08.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.396 7fe08a572700 1 -- 172.21.2.101:0/2236673913 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe074002fe0 con 0x7fe08c0f0650 2021-02-10T11:43:08.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.396 7fe08b574700 1 --2- 172.21.2.101:0/2236673913 >> v2:172.21.2.108:3300/0 conn(0x7fe08c0d1870 0x7fe08c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:08.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.397 7fe08a572700 1 -- 172.21.2.101:0/2236673913 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1378163438 0 0) 0x7fe084001500 con 0x7fe08c0f0650 2021-02-10T11:43:08.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.397 7fe08a572700 1 -- 172.21.2.101:0/2236673913 >> v2:172.21.2.108:3300/0 conn(0x7fe08c0d1870 msgr2=0x7fe08c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:08.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.397 7fe08a572700 1 --2- 172.21.2.101:0/2236673913 >> v2:172.21.2.108:3300/0 conn(0x7fe08c0d1870 0x7fe08c1b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:08.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.397 7fe08a572700 1 -- 172.21.2.101:0/2236673913 >> v1:172.21.2.108:6789/0 conn(0x7fe08c1b0a10 legacy=0x7fe08c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:08.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.397 7fe08a572700 1 -- 172.21.2.101:0/2236673913 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08c0d1c70 con 0x7fe08c0f0650 2021-02-10T11:43:08.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.397 7fe08a572700 1 -- 172.21.2.101:0/2236673913 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fe084001a00 con 0x7fe08c0f0650 2021-02-10T11:43:08.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.397 7fe08a572700 1 -- 172.21.2.101:0/2236673913 >> v1:172.21.2.101:6789/0 conn(0x7fe08c0f0650 legacy=0x7fe08c0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:08.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.397 7fe08a572700 1 --2- 172.21.2.101:0/2236673913 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe074003a60 0x7fe074005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:08.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.397 7fe08a572700 1 --2- 172.21.2.101:0/2236673913 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe074006790 0x7fe074008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:08.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.397 7fe08a572700 1 --2- 172.21.2.101:0/2236673913 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe074009590 0x7fe07400bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:08.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.397 7fe08a572700 1 -- 172.21.2.101:0/2236673913 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07400c300 con 0x7fe074003a60 2021-02-10T11:43:08.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.397 7fe08a572700 1 -- 172.21.2.101:0/2236673913 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07400c520 con 0x7fe074009590 2021-02-10T11:43:08.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.397 7fe08a572700 1 -- 172.21.2.101:0/2236673913 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe07400c740 con 0x7fe074006790 2021-02-10T11:43:08.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.397 7fe08b574700 1 --2- 172.21.2.101:0/2236673913 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe074003a60 0x7fe074005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:08.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.397 7fe08bd75700 1 --2- 172.21.2.101:0/2236673913 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe074009590 0x7fe07400bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:08.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.397 7fe08ad73700 1 --2- 172.21.2.101:0/2236673913 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe074006790 0x7fe074008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:08.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.397 7fe08b574700 1 -- 172.21.2.101:0/2236673913 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe074006790 msgr2=0x7fe074008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:08.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.397 7fe08b574700 1 --2- 172.21.2.101:0/2236673913 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe074006790 0x7fe074008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:08.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.397 7fe08b574700 1 -- 172.21.2.101:0/2236673913 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe074009590 msgr2=0x7fe07400bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:08.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.397 7fe08b574700 1 --2- 172.21.2.101:0/2236673913 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe074009590 0x7fe07400bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:08.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.397 7fe08bd75700 1 -- 172.21.2.101:0/2236673913 reap_dead start 2021-02-10T11:43:08.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.397 7fe08b574700 1 -- 172.21.2.101:0/2236673913 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe074050980 con 0x7fe074003a60 2021-02-10T11:43:08.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.398 7fe08b574700 1 --2- 172.21.2.101:0/2236673913 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe074003a60 0x7fe074005ef0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 rx=0x7fe07c00c470 tx=0x7fe07c006c20).ready entity=mon.0 client_cookie=8783cfb99f684eab server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:08.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.398 7fe08a572700 1 -- 172.21.2.101:0/2236673913 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe07c0090d0 con 0x7fe074003a60 2021-02-10T11:43:08.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.398 7fe08a572700 1 -- 172.21.2.101:0/2236673913 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c009390 con 0x7fe074003a60 2021-02-10T11:43:08.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.398 7fe08a572700 1 -- 172.21.2.101:0/2236673913 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe07c0098b0 con 0x7fe074003a60 2021-02-10T11:43:08.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.398 7fe09183c700 1 -- 172.21.2.101:0/2236673913 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe074003a60 msgr2=0x7fe074005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:08.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.398 7fe09183c700 1 --2- 172.21.2.101:0/2236673913 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe074003a60 0x7fe074005ef0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 rx=0x7fe07c00c470 tx=0x7fe07c006c20).stop 2021-02-10T11:43:08.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.399 7fe09183c700 1 -- 172.21.2.101:0/2236673913 shutdown_connections 2021-02-10T11:43:08.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.399 7fe09183c700 1 --2- 172.21.2.101:0/2236673913 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe074003a60 0x7fe074005ef0 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:08.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.399 7fe09183c700 1 -- 172.21.2.101:0/2236673913 >> 172.21.2.101:0/2236673913 conn(0x7fe08c0ce450 msgr2=0x7fe08c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:08.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.399 7fe09183c700 1 -- 172.21.2.101:0/2236673913 shutdown_connections 2021-02-10T11:43:08.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.399 7fe09183c700 1 -- 172.21.2.101:0/2236673913 wait complete. 2021-02-10T11:43:08.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.414 7fe09183c700 1 Processor -- start 2021-02-10T11:43:08.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.414 7fe09183c700 1 -- start start 2021-02-10T11:43:08.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.414 7fe09183c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe08c1a4a60 0x7fe08c0c1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:08.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.414 7fe09183c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe08c0c2290 0x7fe08c0c2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:08.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.414 7fe09183c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe08c0c2cb0 0x7fe08c0bfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:08.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.414 7fe09183c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c0f05b0 con 0x7fe08c1a4a60 2021-02-10T11:43:08.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.414 7fe09183c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c0f07d0 con 0x7fe08c0c2290 2021-02-10T11:43:08.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.414 7fe09183c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c0c0340 con 0x7fe08c0c2cb0 2021-02-10T11:43:08.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.414 7fe08b574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe08c1a4a60 0x7fe08c0c1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:08.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.414 7fe08b574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe08c1a4a60 0x7fe08c0c1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58654/0 (socket says 172.21.2.101:58654) 2021-02-10T11:43:08.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.414 7fe08b574700 1 -- 172.21.2.101:0/2360334094 learned_addr learned my addr 172.21.2.101:0/2360334094 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:08.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.414 7fe08b574700 1 -- 172.21.2.101:0/2360334094 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe08c0c2cb0 msgr2=0x7fe08c0bfd20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:08.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.414 7fe08b574700 1 --2- 172.21.2.101:0/2360334094 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe08c0c2cb0 0x7fe08c0bfd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:08.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.414 7fe08b574700 1 -- 172.21.2.101:0/2360334094 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe08c0c2290 msgr2=0x7fe08c0c2690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:08.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.414 7fe08b574700 1 --2- 172.21.2.101:0/2360334094 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe08c0c2290 0x7fe08c0c2690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:08.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.414 7fe08b574700 1 -- 172.21.2.101:0/2360334094 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08c0c0560 con 0x7fe08c1a4a60 2021-02-10T11:43:08.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.415 7fe08b574700 1 --2- 172.21.2.101:0/2360334094 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe08c1a4a60 0x7fe08c0c1c70 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 rx=0x7fe07c006f80 tx=0x7fe07c00b300).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:08.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.415 7fe073fff700 1 -- 172.21.2.101:0/2360334094 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe07c00c100 con 0x7fe08c1a4a60 2021-02-10T11:43:08.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.415 7fe073fff700 1 -- 172.21.2.101:0/2360334094 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c00c4d0 con 0x7fe08c1a4a60 2021-02-10T11:43:08.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.415 7fe073fff700 1 -- 172.21.2.101:0/2360334094 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe07c00c9f0 con 0x7fe08c1a4a60 2021-02-10T11:43:08.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.415 7fe09183c700 1 -- 172.21.2.101:0/2360334094 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe08c0c0560 con 0x7fe08c1a4a60 2021-02-10T11:43:08.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.415 7fe09183c700 1 -- 172.21.2.101:0/2360334094 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08c0c0560 con 0x7fe08c1a4a60 2021-02-10T11:43:08.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.415 7fe073fff700 1 -- 172.21.2.101:0/2360334094 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe07c00b640 con 0x7fe08c1a4a60 2021-02-10T11:43:08.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.416 7fe073fff700 1 --2- 172.21.2.101:0/2360334094 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe06c0186c0 0x7fe06c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:08.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.416 7fe073fff700 1 -- 172.21.2.101:0/2360334094 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe07c032380 con 0x7fe08c1a4a60 2021-02-10T11:43:08.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.416 7fe08ad73700 1 --2- 172.21.2.101:0/2360334094 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe06c0186c0 0x7fe06c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:08.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.416 7fe08ad73700 1 --2- 172.21.2.101:0/2360334094 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe06c0186c0 0x7fe06c01abc0 crc :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:08.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.445 7fe09183c700 1 -- 172.21.2.101:0/2360334094 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe08c0f0af0 con 0x7fe08c1a4a60 2021-02-10T11:43:08.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.448 7fe073fff700 1 -- 172.21.2.101:0/2360334094 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe07c028e50 con 0x7fe08c1a4a60 2021-02-10T11:43:08.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.576 7fe09183c700 1 -- 172.21.2.101:0/2360334094 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe08c0f0af0 con 0x7fe08c1a4a60 2021-02-10T11:43:08.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.577 7fe073fff700 1 -- 172.21.2.101:0/2360334094 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+2105 (secure 0 0 0) 0x7fe07c028e50 con 0x7fe08c1a4a60 2021-02-10T11:43:08.577 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:43:08.577 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":106,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11139,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1647083555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1647083555},{"type":"v1","addr":"172.21.2.101:6835","nonce":1647083555}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":101}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:42:30.762357","modified":"2021-02-10 11:42:32.772467","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7816":{"gid":7816,"name":"b","rank":0,"incarnation":102,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/381011050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":381011050},{"type":"v1","addr":"172.21.2.108:6833","nonce":381011050}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-02-10T11:43:08.577 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 106 2021-02-10T11:43:08.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.578 7fe09183c700 1 -- 172.21.2.101:0/2360334094 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe06c0186c0 msgr2=0x7fe06c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:08.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.578 7fe09183c700 1 --2- 172.21.2.101:0/2360334094 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe06c0186c0 0x7fe06c01abc0 crc :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:08.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.578 7fe09183c700 1 -- 172.21.2.101:0/2360334094 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe08c1a4a60 msgr2=0x7fe08c0c1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:08.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.578 7fe09183c700 1 --2- 172.21.2.101:0/2360334094 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe08c1a4a60 0x7fe08c0c1c70 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 rx=0x7fe07c006f80 tx=0x7fe07c00b300).stop 2021-02-10T11:43:08.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.579 7fe09183c700 1 -- 172.21.2.101:0/2360334094 shutdown_connections 2021-02-10T11:43:08.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.579 7fe09183c700 1 --2- 172.21.2.101:0/2360334094 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe06c0186c0 0x7fe06c01abc0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:08.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.579 7fe09183c700 1 --2- 172.21.2.101:0/2360334094 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe08c0c2cb0 0x7fe08c0bfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:08.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.579 7fe09183c700 1 --2- 172.21.2.101:0/2360334094 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe08c0c2290 0x7fe08c0c2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:08.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.579 7fe09183c700 1 --2- 172.21.2.101:0/2360334094 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe08c1a4a60 0x7fe08c0c1c70 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:08.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.579 7fe09183c700 1 -- 172.21.2.101:0/2360334094 >> 172.21.2.101:0/2360334094 conn(0x7fe08c0ebe30 msgr2=0x7fe08c0bf200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:08.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.579 7fe09183c700 1 -- 172.21.2.101:0/2360334094 shutdown_connections 2021-02-10T11:43:08.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.579 7fe09183c700 1 -- 172.21.2.101:0/2360334094 wait complete. 2021-02-10T11:43:08.587 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs fail cephfs 2021-02-10T11:43:09.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.999 7fd56a9b3700 1 Processor -- start 2021-02-10T11:43:09.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.999 7fd56a9b3700 1 -- start start 2021-02-10T11:43:09.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.999 7fd56a9b3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd5640d1870 0x7fd5641b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:09.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.999 7fd56a9b3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd5640f0650 0x7fd5640cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:09.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.999 7fd56a9b3700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5640f0b10 con 0x7fd5641b0a10 2021-02-10T11:43:09.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.999 7fd56a9b3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5640e5ed0 con 0x7fd5640f0650 2021-02-10T11:43:09.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.999 7fd56a9b3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5640d1c70 con 0x7fd5640d1870 2021-02-10T11:43:09.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.999 7fd568fd0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd5640f0650 0x7fd5640cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:09.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.999 7fd568fd0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd5640f0650 0x7fd5640cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58662/0 (socket says 172.21.2.101:58662) 2021-02-10T11:43:09.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.999 7fd568fd0700 1 -- 172.21.2.101:0/1174191748 learned_addr learned my addr 172.21.2.101:0/1174191748 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:09.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.999 7fd5637fe700 1 --1- 172.21.2.101:0/1174191748 >> v1:172.21.2.101:6789/0 conn(0x7fd5641b0a10 0x7fd5640cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59156/0 (socket says 172.21.2.101:59156) 2021-02-10T11:43:09.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:08.999 7fd563fff700 1 --2- 172.21.2.101:0/1174191748 >> v2:172.21.2.108:3300/0 conn(0x7fd5640d1870 0x7fd5641b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:09.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.000 7fd568fd0700 1 -- 172.21.2.101:0/1174191748 >> v2:172.21.2.108:3300/0 conn(0x7fd5640d1870 msgr2=0x7fd5641b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:09.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.000 7fd568fd0700 1 --2- 172.21.2.101:0/1174191748 >> v2:172.21.2.108:3300/0 conn(0x7fd5640d1870 0x7fd5641b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:09.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.000 7fd568fd0700 1 -- 172.21.2.101:0/1174191748 >> v1:172.21.2.101:6789/0 conn(0x7fd5641b0a10 legacy=0x7fd5640cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:09.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.000 7fd568fd0700 1 -- 172.21.2.101:0/1174191748 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5640d1e90 con 0x7fd5640f0650 2021-02-10T11:43:09.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.000 7fd563fff700 1 --2- 172.21.2.101:0/1174191748 >> v2:172.21.2.108:3300/0 conn(0x7fd5640d1870 0x7fd5641b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:43:09.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.000 7fd568fd0700 1 --2- 172.21.2.101:0/1174191748 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd5640f0650 0x7fd5640cfb20 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 rx=0x7fd55c006f00 tx=0x7fd55c009f10).ready entity=mon.0 client_cookie=f800497cd5b405a8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:09.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.000 7fd562ffd700 1 -- 172.21.2.101:0/1174191748 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd55c00b3b0 con 0x7fd5640f0650 2021-02-10T11:43:09.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.000 7fd562ffd700 1 -- 172.21.2.101:0/1174191748 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55c00b6b0 con 0x7fd5640f0650 2021-02-10T11:43:09.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.000 7fd562ffd700 1 -- 172.21.2.101:0/1174191748 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd55c00bbd0 con 0x7fd5640f0650 2021-02-10T11:43:09.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.000 7fd56a9b3700 1 -- 172.21.2.101:0/1174191748 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd5640f0650 msgr2=0x7fd5640cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:09.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.000 7fd56a9b3700 1 --2- 172.21.2.101:0/1174191748 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd5640f0650 0x7fd5640cfb20 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 rx=0x7fd55c006f00 tx=0x7fd55c009f10).stop 2021-02-10T11:43:09.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.001 7fd56a9b3700 1 -- 172.21.2.101:0/1174191748 shutdown_connections 2021-02-10T11:43:09.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.001 7fd56a9b3700 1 --2- 172.21.2.101:0/1174191748 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd5640f0650 0x7fd5640cfb20 unknown :-1 s=CLOSED pgs=2504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:09.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.001 7fd56a9b3700 1 --2- 172.21.2.101:0/1174191748 >> v2:172.21.2.108:3300/0 conn(0x7fd5640d1870 0x7fd5641b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:09.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.001 7fd56a9b3700 1 -- 172.21.2.101:0/1174191748 >> 172.21.2.101:0/1174191748 conn(0x7fd5640ce450 msgr2=0x7fd5640d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:09.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.001 7fd56a9b3700 1 -- 172.21.2.101:0/1174191748 shutdown_connections 2021-02-10T11:43:09.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.001 7fd56a9b3700 1 -- 172.21.2.101:0/1174191748 wait complete. 2021-02-10T11:43:09.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.016 7fd56a9b3700 1 Processor -- start 2021-02-10T11:43:09.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.016 7fd56a9b3700 1 -- start start 2021-02-10T11:43:09.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.016 7fd56a9b3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5640b7510 0x7fd5640b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:09.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.016 7fd56a9b3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd56409d530 0x7fd56409d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:09.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.016 7fd56a9b3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd56409df50 0x7fd56409eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:09.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.016 7fd56a9b3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5640b8140 con 0x7fd56409df50 2021-02-10T11:43:09.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.016 7fd56a9b3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56409e750 con 0x7fd56409d530 2021-02-10T11:43:09.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.016 7fd56a9b3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56409e350 con 0x7fd5640b7510 2021-02-10T11:43:09.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.016 7fd568fd0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd56409df50 0x7fd56409eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:09.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.017 7fd568fd0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd56409df50 0x7fd56409eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58666/0 (socket says 172.21.2.101:58666) 2021-02-10T11:43:09.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.017 7fd568fd0700 1 -- 172.21.2.101:0/3243207214 learned_addr learned my addr 172.21.2.101:0/3243207214 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:09.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.017 7fd5637fe700 1 --2- 172.21.2.101:0/3243207214 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd56409d530 0x7fd56409d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:09.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.017 7fd563fff700 1 --2- 172.21.2.101:0/3243207214 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5640b7510 0x7fd5640b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:09.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.017 7fd568fd0700 1 -- 172.21.2.101:0/3243207214 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5640b7510 msgr2=0x7fd5640b7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:09.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.017 7fd568fd0700 1 --2- 172.21.2.101:0/3243207214 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5640b7510 0x7fd5640b7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:09.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.017 7fd568fd0700 1 -- 172.21.2.101:0/3243207214 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd56409d530 msgr2=0x7fd56409d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:09.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.017 7fd568fd0700 1 --2- 172.21.2.101:0/3243207214 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd56409d530 0x7fd56409d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:09.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.017 7fd568fd0700 1 -- 172.21.2.101:0/3243207214 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56409f2c0 con 0x7fd56409df50 2021-02-10T11:43:09.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.017 7fd568fd0700 1 --2- 172.21.2.101:0/3243207214 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd56409df50 0x7fd56409eca0 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 rx=0x7fd55c0098f0 tx=0x7fd55c00bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:09.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.017 7fd560ff9700 1 -- 172.21.2.101:0/3243207214 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd55c003030 con 0x7fd56409df50 2021-02-10T11:43:09.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.017 7fd560ff9700 1 -- 172.21.2.101:0/3243207214 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55c00c2f0 con 0x7fd56409df50 2021-02-10T11:43:09.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.017 7fd560ff9700 1 -- 172.21.2.101:0/3243207214 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd55c00c810 con 0x7fd56409df50 2021-02-10T11:43:09.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.017 7fd56a9b3700 1 -- 172.21.2.101:0/3243207214 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56409f2c0 con 0x7fd56409df50 2021-02-10T11:43:09.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.017 7fd56a9b3700 1 -- 172.21.2.101:0/3243207214 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56409f850 con 0x7fd56409df50 2021-02-10T11:43:09.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.018 7fd560ff9700 1 -- 172.21.2.101:0/3243207214 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd55c001270 con 0x7fd56409df50 2021-02-10T11:43:09.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.018 7fd560ff9700 1 --2- 172.21.2.101:0/3243207214 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd5440186c0 0x7fd54401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:09.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.018 7fd560ff9700 1 -- 172.21.2.101:0/3243207214 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd55c032960 con 0x7fd56409df50 2021-02-10T11:43:09.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.018 7fd563fff700 1 --2- 172.21.2.101:0/3243207214 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd5440186c0 0x7fd54401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:09.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.018 7fd563fff700 1 --2- 172.21.2.101:0/3243207214 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd5440186c0 0x7fd54401abc0 crc :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:09.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.049 7fd56a9b3700 1 -- 172.21.2.101:0/3243207214 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5640ebc50 con 0x7fd56409df50 2021-02-10T11:43:09.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.051 7fd560ff9700 1 -- 172.21.2.101:0/3243207214 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd55c001600 con 0x7fd56409df50 2021-02-10T11:43:09.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.180 7fd56a9b3700 1 -- 172.21.2.101:0/3243207214 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd5640f0b10 con 0x7fd56409df50 2021-02-10T11:43:09.953 INFO:tasks.ceph.mon.a.gibba001.stderr:2021-02-10 11:43:09.952 7f974d04d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-02-10T11:43:09.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.958 7fd560ff9700 1 -- 172.21.2.101:0/3243207214 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v107) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd55c001270 con 0x7fd56409df50 2021-02-10T11:43:09.997 INFO:teuthology.orchestra.run.gibba001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-02-10T11:43:09.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.997 7fd56a9b3700 1 -- 172.21.2.101:0/3243207214 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd5440186c0 msgr2=0x7fd54401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:09.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.997 7fd56a9b3700 1 --2- 172.21.2.101:0/3243207214 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd5440186c0 0x7fd54401abc0 crc :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:09.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.997 7fd56a9b3700 1 -- 172.21.2.101:0/3243207214 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd56409df50 msgr2=0x7fd56409eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:09.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.997 7fd56a9b3700 1 --2- 172.21.2.101:0/3243207214 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd56409df50 0x7fd56409eca0 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 rx=0x7fd55c0098f0 tx=0x7fd55c00bcd0).stop 2021-02-10T11:43:09.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.997 7fd56a9b3700 1 -- 172.21.2.101:0/3243207214 shutdown_connections 2021-02-10T11:43:10.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.997 7fd56a9b3700 1 --2- 172.21.2.101:0/3243207214 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd56409df50 0x7fd56409eca0 unknown :-1 s=CLOSED pgs=2505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:10.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.997 7fd56a9b3700 1 --2- 172.21.2.101:0/3243207214 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd56409d530 0x7fd56409d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:10.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.997 7fd56a9b3700 1 --2- 172.21.2.101:0/3243207214 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd5440186c0 0x7fd54401abc0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:10.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.997 7fd56a9b3700 1 --2- 172.21.2.101:0/3243207214 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5640b7510 0x7fd5640b7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:10.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.997 7fd56a9b3700 1 -- 172.21.2.101:0/3243207214 >> 172.21.2.101:0/3243207214 conn(0x7fd5640b83b0 msgr2=0x7fd5640a6850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:10.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.998 7fd56a9b3700 1 -- 172.21.2.101:0/3243207214 shutdown_connections 2021-02-10T11:43:10.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:09.998 7fd56a9b3700 1 -- 172.21.2.101:0/3243207214 wait complete. 2021-02-10T11:43:10.007 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:43:10.010 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:43:10.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.418 7f5f621c7700 1 Processor -- start 2021-02-10T11:43:10.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.418 7f5f621c7700 1 -- start start 2021-02-10T11:43:10.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.419 7f5f621c7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f5f5c0d1870 0x7f5f5c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:10.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.419 7f5f621c7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5f5c1b0a10 0x7f5f5c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:10.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.419 7f5f621c7700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f5c0f0b10 con 0x7f5f5c0d1c70 2021-02-10T11:43:10.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.419 7f5f621c7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f5c0e5ed0 con 0x7f5f5c0d1870 2021-02-10T11:43:10.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.419 7f5f621c7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c0f0650 con 0x7f5f5c1b0a10 2021-02-10T11:43:10.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.420 7f5f5bfff700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f5f5c0d1c70 0x7f5f5c0cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:52578/0 (socket says 172.21.2.101:52578) 2021-02-10T11:43:10.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.420 7f5f5b7fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f5f5c0d1870 0x7f5f5c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:10.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.420 7f5f5bfff700 1 -- 172.21.2.101:0/1605340796 learned_addr learned my addr 172.21.2.101:0/1605340796 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:10.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.420 7f5f5affd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5f5c1b0a10 0x7f5f5c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:10.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.421 7f5f5b7fe700 1 -- 172.21.2.101:0/1605340796 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5f5c1b0a10 msgr2=0x7f5f5c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:10.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.421 7f5f5b7fe700 1 --2- 172.21.2.101:0/1605340796 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5f5c1b0a10 0x7f5f5c0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:10.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.421 7f5f5b7fe700 1 -- 172.21.2.101:0/1605340796 >> v1:172.21.2.108:6789/0 conn(0x7f5f5c0d1c70 legacy=0x7f5f5c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:10.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.421 7f5f5b7fe700 1 -- 172.21.2.101:0/1605340796 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5c0f0870 con 0x7f5f5c0d1870 2021-02-10T11:43:10.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.421 7f5f5a7fc700 1 -- 172.21.2.101:0/1605340796 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1112947120 0 0) 0x7f5f54001510 con 0x7f5f5c0d1c70 2021-02-10T11:43:10.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.421 7f5f5affd700 1 --2- 172.21.2.101:0/1605340796 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5f5c1b0a10 0x7f5f5c0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:43:10.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.421 7f5f5b7fe700 1 --2- 172.21.2.101:0/1605340796 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5f5c0d1870 0x7f5f5c1b03f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f5f4c006650 tx=0x7f5f4c009a80).ready entity=mon.1 client_cookie=c43772c997c7a481 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:10.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.421 7f5f5a7fc700 1 -- 172.21.2.101:0/1605340796 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5f4c00c0c0 con 0x7f5f5c0d1870 2021-02-10T11:43:10.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.421 7f5f5a7fc700 1 -- 172.21.2.101:0/1605340796 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4c00c340 con 0x7f5f5c0d1870 2021-02-10T11:43:10.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.421 7f5f5a7fc700 1 -- 172.21.2.101:0/1605340796 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5f4c00c860 con 0x7f5f5c0d1870 2021-02-10T11:43:10.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.422 7f5f621c7700 1 -- 172.21.2.101:0/1605340796 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5f5c0d1870 msgr2=0x7f5f5c1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:10.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.422 7f5f621c7700 1 --2- 172.21.2.101:0/1605340796 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5f5c0d1870 0x7f5f5c1b03f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f5f4c006650 tx=0x7f5f4c009a80).stop 2021-02-10T11:43:10.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.422 7f5f621c7700 1 -- 172.21.2.101:0/1605340796 shutdown_connections 2021-02-10T11:43:10.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.422 7f5f621c7700 1 --2- 172.21.2.101:0/1605340796 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5f5c1b0a10 0x7f5f5c0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:10.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.422 7f5f621c7700 1 --2- 172.21.2.101:0/1605340796 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5f5c0d1870 0x7f5f5c1b03f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:10.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.422 7f5f621c7700 1 -- 172.21.2.101:0/1605340796 >> 172.21.2.101:0/1605340796 conn(0x7f5f5c0ce450 msgr2=0x7f5f5c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:10.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.422 7f5f621c7700 1 -- 172.21.2.101:0/1605340796 shutdown_connections 2021-02-10T11:43:10.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.422 7f5f621c7700 1 -- 172.21.2.101:0/1605340796 wait complete. 2021-02-10T11:43:10.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.437 7f5f621c7700 1 Processor -- start 2021-02-10T11:43:10.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.437 7f5f621c7700 1 -- start start 2021-02-10T11:43:10.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.437 7f5f621c7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5f5c1942b0 0x7f5f5c0b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:10.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.437 7f5f621c7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f5c195070 0x7f5f5c195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:10.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.437 7f5f621c7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5f5c0bb8e0 0x7f5f5c0bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:10.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.437 7f5f621c7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c1946b0 con 0x7f5f5c195070 2021-02-10T11:43:10.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.437 7f5f621c7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c0b3b70 con 0x7f5f5c0bb8e0 2021-02-10T11:43:10.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.437 7f5f621c7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c0b3d90 con 0x7f5f5c1942b0 2021-02-10T11:43:10.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.437 7f5f5affd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f5c195070 0x7f5f5c195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:10.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.437 7f5f5affd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f5c195070 0x7f5f5c195470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58682/0 (socket says 172.21.2.101:58682) 2021-02-10T11:43:10.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.437 7f5f5affd700 1 -- 172.21.2.101:0/129560432 learned_addr learned my addr 172.21.2.101:0/129560432 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:10.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.438 7f5f5affd700 1 -- 172.21.2.101:0/129560432 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5f5c1942b0 msgr2=0x7f5f5c0b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:10.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.438 7f5f5affd700 1 --2- 172.21.2.101:0/129560432 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5f5c1942b0 0x7f5f5c0b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:10.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.438 7f5f5affd700 1 -- 172.21.2.101:0/129560432 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5f5c0bb8e0 msgr2=0x7f5f5c0bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:10.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.438 7f5f5affd700 1 --2- 172.21.2.101:0/129560432 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5f5c0bb8e0 0x7f5f5c0bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:10.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.438 7f5f5affd700 1 -- 172.21.2.101:0/129560432 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5c1a49e0 con 0x7f5f5c195070 2021-02-10T11:43:10.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.438 7f5f5affd700 1 --2- 172.21.2.101:0/129560432 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f5c195070 0x7f5f5c195470 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 rx=0x7f5f50006f40 tx=0x7f5f5000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:10.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.438 7f5f43fff700 1 -- 172.21.2.101:0/129560432 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5f5000bd80 con 0x7f5f5c195070 2021-02-10T11:43:10.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.438 7f5f43fff700 1 -- 172.21.2.101:0/129560432 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f50000c90 con 0x7f5f5c195070 2021-02-10T11:43:10.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.438 7f5f621c7700 1 -- 172.21.2.101:0/129560432 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f5c1a49e0 con 0x7f5f5c195070 2021-02-10T11:43:10.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.438 7f5f43fff700 1 -- 172.21.2.101:0/129560432 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5f50002220 con 0x7f5f5c195070 2021-02-10T11:43:10.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.438 7f5f621c7700 1 -- 172.21.2.101:0/129560432 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f5c0bc5f0 con 0x7f5f5c195070 2021-02-10T11:43:10.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.439 7f5f43fff700 1 -- 172.21.2.101:0/129560432 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5f5000a000 con 0x7f5f5c195070 2021-02-10T11:43:10.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.439 7f5f43fff700 1 --2- 172.21.2.101:0/129560432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5f3c0186c0 0x7f5f3c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:10.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.439 7f5f43fff700 1 -- 172.21.2.101:0/129560432 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f5f50032430 con 0x7f5f5c195070 2021-02-10T11:43:10.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.439 7f5f5b7fe700 1 --2- 172.21.2.101:0/129560432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5f3c0186c0 0x7f5f3c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:10.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.440 7f5f5b7fe700 1 --2- 172.21.2.101:0/129560432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5f3c0186c0 0x7f5f3c01abc0 crc :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:10.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.468 7f5f621c7700 1 -- 172.21.2.101:0/129560432 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f5c0bc5f0 con 0x7f5f5c195070 2021-02-10T11:43:10.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.471 7f5f43fff700 1 -- 172.21.2.101:0/129560432 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5f50029030 con 0x7f5f5c195070 2021-02-10T11:43:10.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.599 7f5f621c7700 1 -- 172.21.2.101:0/129560432 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f5c0bc5f0 con 0x7f5f5c195070 2021-02-10T11:43:10.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.600 7f5f43fff700 1 -- 172.21.2.101:0/129560432 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+1763 (secure 0 0 0) 0x7f5f50029030 con 0x7f5f5c195070 2021-02-10T11:43:10.601 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:43:10.602 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":107,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11139,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1647083555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1647083555},{"type":"v1","addr":"172.21.2.101:6835","nonce":1647083555}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":101}],"filesystems":[{"mdsmap":{"epoch":107,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:42:30.762357","modified":"2021-02-10 11:43:09.953347","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-02-10T11:43:10.602 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 107 2021-02-10T11:43:10.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.602 7f5f621c7700 1 -- 172.21.2.101:0/129560432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5f3c0186c0 msgr2=0x7f5f3c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:10.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.602 7f5f621c7700 1 --2- 172.21.2.101:0/129560432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5f3c0186c0 0x7f5f3c01abc0 crc :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:10.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.602 7f5f621c7700 1 -- 172.21.2.101:0/129560432 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f5c195070 msgr2=0x7f5f5c195470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:10.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.602 7f5f621c7700 1 --2- 172.21.2.101:0/129560432 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f5c195070 0x7f5f5c195470 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 rx=0x7f5f50006f40 tx=0x7f5f5000b990).stop 2021-02-10T11:43:10.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.602 7f5f621c7700 1 -- 172.21.2.101:0/129560432 shutdown_connections 2021-02-10T11:43:10.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.602 7f5f621c7700 1 --2- 172.21.2.101:0/129560432 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5f5c0bb8e0 0x7f5f5c0bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:10.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.602 7f5f621c7700 1 --2- 172.21.2.101:0/129560432 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5f5c195070 0x7f5f5c195470 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:10.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.602 7f5f621c7700 1 --2- 172.21.2.101:0/129560432 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5f3c0186c0 0x7f5f3c01abc0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:10.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.602 7f5f621c7700 1 --2- 172.21.2.101:0/129560432 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5f5c1942b0 0x7f5f5c0b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:10.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.602 7f5f621c7700 1 -- 172.21.2.101:0/129560432 >> 172.21.2.101:0/129560432 conn(0x7f5f5c0b3040 msgr2=0x7f5f5c0c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:10.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.602 7f5f621c7700 1 -- 172.21.2.101:0/129560432 shutdown_connections 2021-02-10T11:43:10.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:10.602 7f5f621c7700 1 -- 172.21.2.101:0/129560432 wait complete. 2021-02-10T11:43:10.611 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-02-10T11:43:11.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.028 7fd91474e700 1 Processor -- start 2021-02-10T11:43:11.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.028 7fd91474e700 1 -- start start 2021-02-10T11:43:11.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.029 7fd91474e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd90c0d0340 0x7fd90c0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:11.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.029 7fd91474e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd90c0d1c70 0x7fd90c0efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:11.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.029 7fd91474e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd90c0f0b10 con 0x7fd90c0d1870 2021-02-10T11:43:11.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.029 7fd91474e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd90c0e5ed0 con 0x7fd90c0d1c70 2021-02-10T11:43:11.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.029 7fd91474e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd90c1b0be0 con 0x7fd90c0d0340 2021-02-10T11:43:11.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.029 7fd912d6b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd90c0d1c70 0x7fd90c0efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:11.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.029 7fd912d6b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd90c0d1c70 0x7fd90c0efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58692/0 (socket says 172.21.2.101:58692) 2021-02-10T11:43:11.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.029 7fd912d6b700 1 -- 172.21.2.101:0/2410238594 learned_addr learned my addr 172.21.2.101:0/2410238594 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:11.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.029 7fd912d6b700 1 -- 172.21.2.101:0/2410238594 >> v2:172.21.2.108:3300/0 conn(0x7fd90c0d0340 msgr2=0x7fd90c0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:11.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.029 7fd912d6b700 1 --2- 172.21.2.101:0/2410238594 >> v2:172.21.2.108:3300/0 conn(0x7fd90c0d0340 0x7fd90c0ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:11.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.029 7fd912d6b700 1 -- 172.21.2.101:0/2410238594 >> v1:172.21.2.108:6789/0 conn(0x7fd90c0d1870 legacy=0x7fd90c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:11.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.029 7fd912d6b700 1 -- 172.21.2.101:0/2410238594 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c0f20a0 con 0x7fd90c0d1c70 2021-02-10T11:43:11.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.030 7fd912d6b700 1 --2- 172.21.2.101:0/2410238594 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd90c0d1c70 0x7fd90c0efc60 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 rx=0x7fd904006f00 tx=0x7fd904009e90).ready entity=mon.0 client_cookie=e09a189dc7276e8c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:11.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.030 7fd911568700 1 -- 172.21.2.101:0/2410238594 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd90400b300 con 0x7fd90c0d1c70 2021-02-10T11:43:11.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.030 7fd911568700 1 -- 172.21.2.101:0/2410238594 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90400b580 con 0x7fd90c0d1c70 2021-02-10T11:43:11.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.030 7fd911568700 1 -- 172.21.2.101:0/2410238594 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd90400b9d0 con 0x7fd90c0d1c70 2021-02-10T11:43:11.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.030 7fd91474e700 1 -- 172.21.2.101:0/2410238594 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd90c0d1c70 msgr2=0x7fd90c0efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:11.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.030 7fd91474e700 1 --2- 172.21.2.101:0/2410238594 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd90c0d1c70 0x7fd90c0efc60 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 rx=0x7fd904006f00 tx=0x7fd904009e90).stop 2021-02-10T11:43:11.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.031 7fd91474e700 1 -- 172.21.2.101:0/2410238594 shutdown_connections 2021-02-10T11:43:11.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.031 7fd91474e700 1 --2- 172.21.2.101:0/2410238594 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd90c0d1c70 0x7fd90c0efc60 unknown :-1 s=CLOSED pgs=2509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:11.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.031 7fd91474e700 1 --2- 172.21.2.101:0/2410238594 >> v2:172.21.2.108:3300/0 conn(0x7fd90c0d0340 0x7fd90c0ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:11.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.031 7fd91474e700 1 -- 172.21.2.101:0/2410238594 >> 172.21.2.101:0/2410238594 conn(0x7fd90c0ce450 msgr2=0x7fd90c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:11.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.031 7fd91474e700 1 -- 172.21.2.101:0/2410238594 shutdown_connections 2021-02-10T11:43:11.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.031 7fd91474e700 1 -- 172.21.2.101:0/2410238594 wait complete. 2021-02-10T11:43:11.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.045 7fd91474e700 1 Processor -- start 2021-02-10T11:43:11.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.045 7fd91474e700 1 -- start start 2021-02-10T11:43:11.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.045 7fd91474e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd90c09bf50 0x7fd90c09c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:11.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.046 7fd91474e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd90c09c350 0x7fd90c0ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:11.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.046 7fd91474e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd90c09cfc0 0x7fd90c0acbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:11.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.046 7fd91474e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c0ad210 con 0x7fd90c09bf50 2021-02-10T11:43:11.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.046 7fd91474e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c0ad400 con 0x7fd90c09c350 2021-02-10T11:43:11.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.046 7fd91474e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c0ad620 con 0x7fd90c09cfc0 2021-02-10T11:43:11.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.046 7fd91256a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd90c09bf50 0x7fd90c09c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:11.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.046 7fd91256a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd90c09bf50 0x7fd90c09c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58694/0 (socket says 172.21.2.101:58694) 2021-02-10T11:43:11.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.046 7fd91256a700 1 -- 172.21.2.101:0/3978518290 learned_addr learned my addr 172.21.2.101:0/3978518290 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:11.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.046 7fd91256a700 1 -- 172.21.2.101:0/3978518290 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd90c09cfc0 msgr2=0x7fd90c0acbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:11.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.046 7fd91256a700 1 --2- 172.21.2.101:0/3978518290 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd90c09cfc0 0x7fd90c0acbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:11.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.046 7fd91256a700 1 -- 172.21.2.101:0/3978518290 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd90c09c350 msgr2=0x7fd90c0ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:11.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.046 7fd91256a700 1 --2- 172.21.2.101:0/3978518290 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd90c09c350 0x7fd90c0ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:11.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.046 7fd91256a700 1 -- 172.21.2.101:0/3978518290 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c0ada80 con 0x7fd90c09bf50 2021-02-10T11:43:11.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.046 7fd91256a700 1 --2- 172.21.2.101:0/3978518290 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd90c09bf50 0x7fd90c09c9a0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 rx=0x7fd90000b5b0 tx=0x7fd90000b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:11.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.046 7fd8feffd700 1 -- 172.21.2.101:0/3978518290 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd90000bd40 con 0x7fd90c09bf50 2021-02-10T11:43:11.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.046 7fd8feffd700 1 -- 172.21.2.101:0/3978518290 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd900000e20 con 0x7fd90c09bf50 2021-02-10T11:43:11.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.046 7fd8feffd700 1 -- 172.21.2.101:0/3978518290 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd9000054a0 con 0x7fd90c09bf50 2021-02-10T11:43:11.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.046 7fd91474e700 1 -- 172.21.2.101:0/3978518290 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90c0ada80 con 0x7fd90c09bf50 2021-02-10T11:43:11.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.047 7fd91474e700 1 -- 172.21.2.101:0/3978518290 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90c0ada80 con 0x7fd90c09bf50 2021-02-10T11:43:11.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.047 7fd8feffd700 1 -- 172.21.2.101:0/3978518290 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd900006030 con 0x7fd90c09bf50 2021-02-10T11:43:11.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.047 7fd8feffd700 1 --2- 172.21.2.101:0/3978518290 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8f40186d0 0x7fd8f401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:11.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.047 7fd8feffd700 1 -- 172.21.2.101:0/3978518290 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fd900032830 con 0x7fd90c09bf50 2021-02-10T11:43:11.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.047 7fd911d69700 1 --2- 172.21.2.101:0/3978518290 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8f40186d0 0x7fd8f401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:11.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.048 7fd911d69700 1 --2- 172.21.2.101:0/3978518290 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8f40186d0 0x7fd8f401abd0 crc :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:11.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.079 7fd91474e700 1 -- 172.21.2.101:0/3978518290 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd90c0ce450 con 0x7fd90c09bf50 2021-02-10T11:43:11.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.081 7fd8feffd700 1 -- 172.21.2.101:0/3978518290 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd90000bd40 con 0x7fd90c09bf50 2021-02-10T11:43:11.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.208 7fd91474e700 1 -- 172.21.2.101:0/3978518290 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd90c0ce450 con 0x7fd90c09bf50 2021-02-10T11:43:11.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.966 7fd8feffd700 1 -- 172.21.2.101:0/3978518290 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v108) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd900006030 con 0x7fd90c09bf50 2021-02-10T11:43:11.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.979 7fd91474e700 1 -- 172.21.2.101:0/3978518290 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8f40186d0 msgr2=0x7fd8f401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:11.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.979 7fd91474e700 1 --2- 172.21.2.101:0/3978518290 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8f40186d0 0x7fd8f401abd0 crc :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:11.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.979 7fd91474e700 1 -- 172.21.2.101:0/3978518290 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd90c09bf50 msgr2=0x7fd90c09c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:11.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.979 7fd91474e700 1 --2- 172.21.2.101:0/3978518290 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd90c09bf50 0x7fd90c09c9a0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 rx=0x7fd90000b5b0 tx=0x7fd90000b950).stop 2021-02-10T11:43:11.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.979 7fd91474e700 1 -- 172.21.2.101:0/3978518290 shutdown_connections 2021-02-10T11:43:11.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.979 7fd91474e700 1 --2- 172.21.2.101:0/3978518290 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8f40186d0 0x7fd8f401abd0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:11.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.979 7fd91474e700 1 --2- 172.21.2.101:0/3978518290 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd90c09cfc0 0x7fd90c0acbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:11.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.979 7fd91474e700 1 --2- 172.21.2.101:0/3978518290 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd90c09c350 0x7fd90c0ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:11.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.979 7fd91474e700 1 --2- 172.21.2.101:0/3978518290 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd90c09bf50 0x7fd90c09c9a0 unknown :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:11.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.979 7fd91474e700 1 -- 172.21.2.101:0/3978518290 >> 172.21.2.101:0/3978518290 conn(0x7fd90c0a6620 msgr2=0x7fd90c0b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:11.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.980 7fd91474e700 1 -- 172.21.2.101:0/3978518290 shutdown_connections 2021-02-10T11:43:11.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:11.980 7fd91474e700 1 -- 172.21.2.101:0/3978518290 wait complete. 2021-02-10T11:43:11.989 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-02-10T11:43:12.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.405 7f0564601700 1 Processor -- start 2021-02-10T11:43:12.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.405 7f0564601700 1 -- start start 2021-02-10T11:43:12.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.406 7f0564601700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f055c0d1c80 0x7f055c0cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:12.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.406 7f0564601700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f055c0d0180 con 0x7f055c0d1880 2021-02-10T11:43:12.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.406 7f0564601700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f055c0fe1d0 con 0x7f055c0cf590 2021-02-10T11:43:12.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.406 7f0564601700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f055c1b0cc0 con 0x7f055c0d1c80 2021-02-10T11:43:12.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.406 7f056241d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f055c0d1880 0x7f055c1b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59196/0 (socket says 172.21.2.101:59196) 2021-02-10T11:43:12.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.406 7f056241d700 1 -- 172.21.2.101:0/340212625 learned_addr learned my addr 172.21.2.101:0/340212625 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:12.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.406 7f0561c1c700 1 --2- 172.21.2.101:0/340212625 >> v2:172.21.2.108:3300/0 conn(0x7f055c0d1c80 0x7f055c0cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:12.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.406 7f056141b700 1 -- 172.21.2.101:0/340212625 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2593696959 0 0) 0x7f0558001180 con 0x7f055c0d1880 2021-02-10T11:43:12.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.406 7f056141b700 1 -- 172.21.2.101:0/340212625 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0548002fe0 con 0x7f055c0d1880 2021-02-10T11:43:12.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.407 7f056141b700 1 -- 172.21.2.101:0/340212625 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1001125424 0 0) 0x7f0554001400 con 0x7f055c0cf590 2021-02-10T11:43:12.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.407 7f056141b700 1 -- 172.21.2.101:0/340212625 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0548003210 con 0x7f055c0cf590 2021-02-10T11:43:12.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.407 7f0561c1c700 1 -- 172.21.2.101:0/340212625 >> v1:172.21.2.108:6789/0 conn(0x7f055c0cf590 legacy=0x7f055c0cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:12.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.407 7f0561c1c700 1 -- 172.21.2.101:0/340212625 >> v1:172.21.2.101:6789/0 conn(0x7f055c0d1880 legacy=0x7f055c1b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:12.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.407 7f0561c1c700 1 -- 172.21.2.101:0/340212625 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055c109a70 con 0x7f055c0d1c80 2021-02-10T11:43:12.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.407 7f0561c1c700 1 --2- 172.21.2.101:0/340212625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f055c0d1c80 0x7f055c0cef70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f0550006650 tx=0x7f0550009a00).ready entity=mon.1 client_cookie=aaa1c94e9cca1868 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:12.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.407 7f056141b700 1 -- 172.21.2.101:0/340212625 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0550009df0 con 0x7f055c0d1c80 2021-02-10T11:43:12.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.407 7f056141b700 1 -- 172.21.2.101:0/340212625 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05500009b0 con 0x7f055c0d1c80 2021-02-10T11:43:12.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.407 7f056141b700 1 -- 172.21.2.101:0/340212625 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f055000c2b0 con 0x7f055c0d1c80 2021-02-10T11:43:12.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.407 7f0564601700 1 -- 172.21.2.101:0/340212625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f055c0d1c80 msgr2=0x7f055c0cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:12.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.407 7f0564601700 1 --2- 172.21.2.101:0/340212625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f055c0d1c80 0x7f055c0cef70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f0550006650 tx=0x7f0550009a00).stop 2021-02-10T11:43:12.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.408 7f0564601700 1 -- 172.21.2.101:0/340212625 shutdown_connections 2021-02-10T11:43:12.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.408 7f0564601700 1 --2- 172.21.2.101:0/340212625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f055c0d1c80 0x7f055c0cef70 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:12.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.408 7f0564601700 1 -- 172.21.2.101:0/340212625 >> 172.21.2.101:0/340212625 conn(0x7f055c0ce430 msgr2=0x7f055c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:12.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.408 7f0564601700 1 -- 172.21.2.101:0/340212625 shutdown_connections 2021-02-10T11:43:12.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.408 7f0564601700 1 -- 172.21.2.101:0/340212625 wait complete. 2021-02-10T11:43:12.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.423 7f0564601700 1 Processor -- start 2021-02-10T11:43:12.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.423 7f0564601700 1 -- start start 2021-02-10T11:43:12.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.423 7f0564601700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f055c0ad520 0x7f055c0ad920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:12.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.423 7f0564601700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f055c0adf40 0x7f055c097c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:12.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.423 7f0564601700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f055c098250 0x7f055c098650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:12.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.423 7f0564601700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c098c70 con 0x7f055c0adf40 2021-02-10T11:43:12.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.423 7f0564601700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c0910f0 con 0x7f055c0ad520 2021-02-10T11:43:12.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.423 7f0564601700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f055c091310 con 0x7f055c098250 2021-02-10T11:43:12.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.423 7f0561c1c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f055c0adf40 0x7f055c097c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:12.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.423 7f0561c1c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f055c0adf40 0x7f055c097c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58708/0 (socket says 172.21.2.101:58708) 2021-02-10T11:43:12.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.423 7f0561c1c700 1 -- 172.21.2.101:0/655141058 learned_addr learned my addr 172.21.2.101:0/655141058 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:12.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.423 7f056241d700 1 --2- 172.21.2.101:0/655141058 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f055c0ad520 0x7f055c0ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:12.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.423 7f0562c1e700 1 --2- 172.21.2.101:0/655141058 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f055c098250 0x7f055c098650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:12.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.424 7f0561c1c700 1 -- 172.21.2.101:0/655141058 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f055c098250 msgr2=0x7f055c098650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:12.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.424 7f0561c1c700 1 --2- 172.21.2.101:0/655141058 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f055c098250 0x7f055c098650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:12.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.424 7f0561c1c700 1 -- 172.21.2.101:0/655141058 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f055c0ad520 msgr2=0x7f055c0ad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:12.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.424 7f0561c1c700 1 --2- 172.21.2.101:0/655141058 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f055c0ad520 0x7f055c0ad920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:12.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.424 7f0561c1c700 1 -- 172.21.2.101:0/655141058 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055c19a3b0 con 0x7f055c0adf40 2021-02-10T11:43:12.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.424 7f0561c1c700 1 --2- 172.21.2.101:0/655141058 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f055c0adf40 0x7f055c097c30 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 rx=0x7f05500096a0 tx=0x7f055000b9b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:12.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.424 7f054effd700 1 -- 172.21.2.101:0/655141058 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f05500093a0 con 0x7f055c0adf40 2021-02-10T11:43:12.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.424 7f054effd700 1 -- 172.21.2.101:0/655141058 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05500067f0 con 0x7f055c0adf40 2021-02-10T11:43:12.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.424 7f0564601700 1 -- 172.21.2.101:0/655141058 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055c19a3b0 con 0x7f055c0adf40 2021-02-10T11:43:12.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.424 7f054effd700 1 -- 172.21.2.101:0/655141058 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0550002220 con 0x7f055c0adf40 2021-02-10T11:43:12.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.424 7f0564601700 1 -- 172.21.2.101:0/655141058 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f055c19a940 con 0x7f055c0adf40 2021-02-10T11:43:12.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.425 7f054effd700 1 -- 172.21.2.101:0/655141058 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f05500052e0 con 0x7f055c0adf40 2021-02-10T11:43:12.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.425 7f054effd700 1 --2- 172.21.2.101:0/655141058 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0544018730 0x7f054401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:12.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.425 7f054effd700 1 -- 172.21.2.101:0/655141058 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f0550032650 con 0x7f055c0adf40 2021-02-10T11:43:12.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.425 7f056241d700 1 --2- 172.21.2.101:0/655141058 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0544018730 0x7f054401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:12.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.426 7f056241d700 1 --2- 172.21.2.101:0/655141058 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0544018730 0x7f054401ac10 crc :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:12.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.456 7f0564601700 1 -- 172.21.2.101:0/655141058 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f055c082a70 con 0x7f055c0adf40 2021-02-10T11:43:12.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.458 7f054effd700 1 -- 172.21.2.101:0/655141058 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f05500093a0 con 0x7f055c0adf40 2021-02-10T11:43:12.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.586 7f0564601700 1 -- 172.21.2.101:0/655141058 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f055c082a70 con 0x7f055c0adf40 2021-02-10T11:43:12.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:12.971 7f054effd700 1 -- 172.21.2.101:0/655141058 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v131) v1 ==== 185+0+0 (secure 0 0 0) 0x7f05500052e0 con 0x7f055c0adf40 2021-02-10T11:43:13.004 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' removed 2021-02-10T11:43:13.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.005 7f0564601700 1 -- 172.21.2.101:0/655141058 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0544018730 msgr2=0x7f054401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:13.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.005 7f0564601700 1 --2- 172.21.2.101:0/655141058 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0544018730 0x7f054401ac10 crc :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:13.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.005 7f0564601700 1 -- 172.21.2.101:0/655141058 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f055c0adf40 msgr2=0x7f055c097c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:13.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.005 7f0564601700 1 --2- 172.21.2.101:0/655141058 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f055c0adf40 0x7f055c097c30 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 rx=0x7f05500096a0 tx=0x7f055000b9b0).stop 2021-02-10T11:43:13.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.005 7f0564601700 1 -- 172.21.2.101:0/655141058 shutdown_connections 2021-02-10T11:43:13.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.005 7f0564601700 1 --2- 172.21.2.101:0/655141058 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0544018730 0x7f054401ac10 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:13.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.005 7f0564601700 1 --2- 172.21.2.101:0/655141058 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f055c098250 0x7f055c098650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:13.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.005 7f0564601700 1 --2- 172.21.2.101:0/655141058 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f055c0adf40 0x7f055c097c30 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:13.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.005 7f0564601700 1 --2- 172.21.2.101:0/655141058 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f055c0ad520 0x7f055c0ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:13.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.005 7f0564601700 1 -- 172.21.2.101:0/655141058 >> 172.21.2.101:0/655141058 conn(0x7f055c18e810 msgr2=0x7f055c18c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:13.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.005 7f0564601700 1 -- 172.21.2.101:0/655141058 shutdown_connections 2021-02-10T11:43:13.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.005 7f0564601700 1 -- 172.21.2.101:0/655141058 wait complete. 2021-02-10T11:43:13.014 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-02-10T11:43:13.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.430 7f074d249700 1 Processor -- start 2021-02-10T11:43:13.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.430 7f074d249700 1 -- start start 2021-02-10T11:43:13.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.430 7f074d249700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f074810aff0 0x7f07480cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:13.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.430 7f074d249700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07480fe1c0 con 0x7f07480d1870 2021-02-10T11:43:13.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.430 7f074d249700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07481b0cb0 con 0x7f07480d1c70 2021-02-10T11:43:13.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.430 7f074d249700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0748109a60 con 0x7f074810aff0 2021-02-10T11:43:13.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f0746d9d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f07480d1870 0x7f07481b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59210/0 (socket says 172.21.2.101:59210) 2021-02-10T11:43:13.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f0746d9d700 1 -- 172.21.2.101:0/2499053388 learned_addr learned my addr 172.21.2.101:0/2499053388 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:13.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f074659c700 1 --1- 172.21.2.101:0/2499053388 >> v1:172.21.2.108:6789/0 conn(0x7f07480d1c70 0x7f07480cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:52620/0 (socket says 172.21.2.101:52620) 2021-02-10T11:43:13.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f074759e700 1 --2- 172.21.2.101:0/2499053388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f074810aff0 0x7f07480cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:13.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f0745d9b700 1 -- 172.21.2.101:0/2499053388 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2745906418 0 0) 0x7f07300009b0 con 0x7f07480d1c70 2021-02-10T11:43:13.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f0745d9b700 1 -- 172.21.2.101:0/2499053388 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0734002fe0 con 0x7f07480d1c70 2021-02-10T11:43:13.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f0745d9b700 1 -- 172.21.2.101:0/2499053388 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1877023739 0 0) 0x7f0738001180 con 0x7f07480d1870 2021-02-10T11:43:13.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f0745d9b700 1 -- 172.21.2.101:0/2499053388 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0734003210 con 0x7f07480d1870 2021-02-10T11:43:13.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f0745d9b700 1 -- 172.21.2.101:0/2499053388 <== mon.1 v1:172.21.2.108:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2000786623 0 0) 0x7f0730001010 con 0x7f07480d1c70 2021-02-10T11:43:13.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f0745d9b700 1 -- 172.21.2.101:0/2499053388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f074810aff0 msgr2=0x7f07480cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:13.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f0745d9b700 1 --2- 172.21.2.101:0/2499053388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f074810aff0 0x7f07480cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:13.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f0745d9b700 1 -- 172.21.2.101:0/2499053388 >> v1:172.21.2.101:6789/0 conn(0x7f07480d1870 legacy=0x7f07481b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:13.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f0745d9b700 1 -- 172.21.2.101:0/2499053388 --> v1:172.21.2.108:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074810b3f0 con 0x7f07480d1c70 2021-02-10T11:43:13.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f0745d9b700 1 -- 172.21.2.101:0/2499053388 <== mon.1 v1:172.21.2.108:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f0730001530 con 0x7f07480d1c70 2021-02-10T11:43:13.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f0745d9b700 1 -- 172.21.2.101:0/2499053388 >> v1:172.21.2.108:6789/0 conn(0x7f07480d1c70 legacy=0x7f07480cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:13.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f0745d9b700 1 --2- 172.21.2.101:0/2499053388 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0734003b90 0x7f0734005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:13.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f0745d9b700 1 --2- 172.21.2.101:0/2499053388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07340067a0 0x7f0734008ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:13.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f0745d9b700 1 --2- 172.21.2.101:0/2499053388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07340095a0 0x7f073400ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:13.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f0745d9b700 1 -- 172.21.2.101:0/2499053388 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07340502f0 con 0x7f07340067a0 2021-02-10T11:43:13.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f0745d9b700 1 -- 172.21.2.101:0/2499053388 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07340504d0 con 0x7f0734003b90 2021-02-10T11:43:13.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f0745d9b700 1 -- 172.21.2.101:0/2499053388 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07340506f0 con 0x7f07340095a0 2021-02-10T11:43:13.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f074659c700 1 --2- 172.21.2.101:0/2499053388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07340067a0 0x7f0734008ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:13.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f074759e700 1 --2- 172.21.2.101:0/2499053388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07340095a0 0x7f073400ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:13.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.431 7f0746d9d700 1 --2- 172.21.2.101:0/2499053388 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0734003b90 0x7f0734005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:13.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.432 7f074659c700 1 -- 172.21.2.101:0/2499053388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07340095a0 msgr2=0x7f073400ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:13.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.432 7f074659c700 1 --2- 172.21.2.101:0/2499053388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07340095a0 0x7f073400ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:13.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.432 7f074659c700 1 -- 172.21.2.101:0/2499053388 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0734003b90 msgr2=0x7f0734005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:13.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.432 7f074659c700 1 --2- 172.21.2.101:0/2499053388 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0734003b90 0x7f0734005fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:13.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.432 7f074659c700 1 -- 172.21.2.101:0/2499053388 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0734050990 con 0x7f07340067a0 2021-02-10T11:43:13.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.432 7f074659c700 1 --2- 172.21.2.101:0/2499053388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07340067a0 0x7f0734008ca0 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 rx=0x7f0730001380 tx=0x7f0730007bf0).ready entity=mon.0 client_cookie=10f246015fdb53f8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:13.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.432 7f0745d9b700 1 -- 172.21.2.101:0/2499053388 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f073000a0d0 con 0x7f07340067a0 2021-02-10T11:43:13.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.432 7f0745d9b700 1 -- 172.21.2.101:0/2499053388 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f073000a410 con 0x7f07340067a0 2021-02-10T11:43:13.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.432 7f0745d9b700 1 -- 172.21.2.101:0/2499053388 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f073000a8f0 con 0x7f07340067a0 2021-02-10T11:43:13.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.432 7f074d249700 1 -- 172.21.2.101:0/2499053388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07340067a0 msgr2=0x7f0734008ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:13.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.432 7f074d249700 1 --2- 172.21.2.101:0/2499053388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07340067a0 0x7f0734008ca0 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 rx=0x7f0730001380 tx=0x7f0730007bf0).stop 2021-02-10T11:43:13.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.432 7f074759e700 1 -- 172.21.2.101:0/2499053388 reap_dead start 2021-02-10T11:43:13.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.433 7f074d249700 1 -- 172.21.2.101:0/2499053388 shutdown_connections 2021-02-10T11:43:13.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.433 7f074d249700 1 -- 172.21.2.101:0/2499053388 >> 172.21.2.101:0/2499053388 conn(0x7f07480ce420 msgr2=0x7f07480d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:13.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.433 7f074d249700 1 -- 172.21.2.101:0/2499053388 shutdown_connections 2021-02-10T11:43:13.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.433 7f074d249700 1 -- 172.21.2.101:0/2499053388 wait complete. 2021-02-10T11:43:13.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.448 7f074d249700 1 Processor -- start 2021-02-10T11:43:13.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.448 7f074d249700 1 -- start start 2021-02-10T11:43:13.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.448 7f074d249700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07481a05a0 0x7f07480a8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:13.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.448 7f074d249700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07480b5090 0x7f07480b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:13.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.448 7f074d249700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07480b5ab0 0x7f07480b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:13.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.448 7f074d249700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07480b6090 con 0x7f07480b5ab0 2021-02-10T11:43:13.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.448 7f074d249700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07480a9240 con 0x7f07480b5090 2021-02-10T11:43:13.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.448 7f074d249700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07480b2a30 con 0x7f07481a05a0 2021-02-10T11:43:13.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.448 7f074759e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07480b5ab0 0x7f07480b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:13.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.448 7f0746d9d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07481a05a0 0x7f07480a8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:13.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.448 7f074759e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07480b5ab0 0x7f07480b68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58732/0 (socket says 172.21.2.101:58732) 2021-02-10T11:43:13.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.448 7f074759e700 1 -- 172.21.2.101:0/2433296822 learned_addr learned my addr 172.21.2.101:0/2433296822 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:13.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.448 7f074659c700 1 --2- 172.21.2.101:0/2433296822 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07480b5090 0x7f07480b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:13.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.449 7f0746d9d700 1 -- 172.21.2.101:0/2433296822 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07480b5090 msgr2=0x7f07480b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:13.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.449 7f0746d9d700 1 --2- 172.21.2.101:0/2433296822 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07480b5090 0x7f07480b5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:13.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.449 7f0746d9d700 1 -- 172.21.2.101:0/2433296822 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07480b5ab0 msgr2=0x7f07480b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:13.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.449 7f0746d9d700 1 --2- 172.21.2.101:0/2433296822 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07480b5ab0 0x7f07480b68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:13.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.449 7f0746d9d700 1 -- 172.21.2.101:0/2433296822 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07480b6ef0 con 0x7f07481a05a0 2021-02-10T11:43:13.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.449 7f074759e700 1 --2- 172.21.2.101:0/2433296822 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07480b5ab0 0x7f07480b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:43:13.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.449 7f0746d9d700 1 --2- 172.21.2.101:0/2433296822 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07481a05a0 0x7f07480a8c20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f073800a4c0 tx=0x7f073800a860).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:13.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.449 7f073f7fe700 1 -- 172.21.2.101:0/2433296822 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0738007280 con 0x7f07481a05a0 2021-02-10T11:43:13.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.449 7f073f7fe700 1 -- 172.21.2.101:0/2433296822 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0738000940 con 0x7f07481a05a0 2021-02-10T11:43:13.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.449 7f073f7fe700 1 -- 172.21.2.101:0/2433296822 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f07380043a0 con 0x7f07481a05a0 2021-02-10T11:43:13.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.449 7f074d249700 1 -- 172.21.2.101:0/2433296822 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07480b6ef0 con 0x7f07481a05a0 2021-02-10T11:43:13.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.449 7f074d249700 1 -- 172.21.2.101:0/2433296822 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07480b6ef0 con 0x7f07481a05a0 2021-02-10T11:43:13.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.450 7f073f7fe700 1 -- 172.21.2.101:0/2433296822 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f07380043a0 con 0x7f07481a05a0 2021-02-10T11:43:13.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.450 7f073f7fe700 1 --2- 172.21.2.101:0/2433296822 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f072c0186c0 0x7f072c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:13.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.450 7f073f7fe700 1 -- 172.21.2.101:0/2433296822 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f0738032310 con 0x7f07481a05a0 2021-02-10T11:43:13.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.450 7f074659c700 1 --2- 172.21.2.101:0/2433296822 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f072c0186c0 0x7f072c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:13.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.450 7f074659c700 1 --2- 172.21.2.101:0/2433296822 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f072c0186c0 0x7f072c01abc0 crc :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:13.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.480 7f074d249700 1 -- 172.21.2.101:0/2433296822 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07480825c0 con 0x7f07481a05a0 2021-02-10T11:43:13.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.483 7f073f7fe700 1 -- 172.21.2.101:0/2433296822 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0738030c20 con 0x7f07481a05a0 2021-02-10T11:43:13.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.608 7f074d249700 1 -- 172.21.2.101:0/2433296822 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f07480825c0 con 0x7f07481a05a0 2021-02-10T11:43:13.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:13.975 7f073f7fe700 1 -- 172.21.2.101:0/2433296822 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v132) v1 ==== 173+0+0 (secure 0 0 0) 0x7f07380043a0 con 0x7f07481a05a0 2021-02-10T11:43:14.025 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' removed 2021-02-10T11:43:14.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.026 7f074d249700 1 -- 172.21.2.101:0/2433296822 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f072c0186c0 msgr2=0x7f072c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:14.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.026 7f074d249700 1 --2- 172.21.2.101:0/2433296822 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f072c0186c0 0x7f072c01abc0 crc :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:14.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.026 7f074d249700 1 -- 172.21.2.101:0/2433296822 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07481a05a0 msgr2=0x7f07480a8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:14.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.026 7f074d249700 1 --2- 172.21.2.101:0/2433296822 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07481a05a0 0x7f07480a8c20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f073800a4c0 tx=0x7f073800a860).stop 2021-02-10T11:43:14.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.026 7f074d249700 1 -- 172.21.2.101:0/2433296822 shutdown_connections 2021-02-10T11:43:14.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.026 7f074d249700 1 --2- 172.21.2.101:0/2433296822 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07480b5ab0 0x7f07480b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:14.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.026 7f074d249700 1 --2- 172.21.2.101:0/2433296822 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07480b5090 0x7f07480b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:14.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.026 7f074d249700 1 --2- 172.21.2.101:0/2433296822 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f072c0186c0 0x7f072c01abc0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:14.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.026 7f074d249700 1 --2- 172.21.2.101:0/2433296822 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07481a05a0 0x7f07480a8c20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:14.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.026 7f074d249700 1 -- 172.21.2.101:0/2433296822 >> 172.21.2.101:0/2433296822 conn(0x7f07480bc130 msgr2=0x7f07480a9630 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:14.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.026 7f074d249700 1 -- 172.21.2.101:0/2433296822 shutdown_connections 2021-02-10T11:43:14.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.026 7f074d249700 1 -- 172.21.2.101:0/2433296822 wait complete. 2021-02-10T11:43:14.035 INFO:tasks.ceph.mds.a:Restarting daemon 2021-02-10T11:43:14.035 INFO:tasks.ceph.mds.a:Stopping old one... 2021-02-10T11:43:14.035 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-02-10T11:43:14.035 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:43:14.036 INFO:tasks.ceph.mds.b:Restarting daemon 2021-02-10T11:43:14.036 INFO:tasks.ceph.mds.b:Stopping old one... 2021-02-10T11:43:14.036 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-02-10T11:43:14.036 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:43:14.053 INFO:tasks.ceph.mds.b:Stopped 2021-02-10T11:43:14.054 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-02-10T11:43:14.057 INFO:tasks.ceph.mds.a:Stopped 2021-02-10T11:43:14.058 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-02-10T11:43:14.107 INFO:tasks.ceph.mds.a:Started 2021-02-10T11:43:14.107 DEBUG:teuthology.parallel:result is None 2021-02-10T11:43:14.124 INFO:tasks.ceph.mds.b:Started 2021-02-10T11:43:14.125 DEBUG:teuthology.parallel:result is None 2021-02-10T11:43:14.125 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd blacklist clear 2021-02-10T11:43:14.515 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:43:14.528 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:43:14.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.520 7f1f3a827700 1 Processor -- start 2021-02-10T11:43:14.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.520 7f1f3a827700 1 -- start start 2021-02-10T11:43:14.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.520 7f1f3a827700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1f340d0340 0x7f1f340ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:14.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.520 7f1f3a827700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f340f0b10 con 0x7f1f340d1870 2021-02-10T11:43:14.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.520 7f1f3a827700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f341b0be0 con 0x7f1f340d1c70 2021-02-10T11:43:14.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.520 7f1f3a827700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f340e5ed0 con 0x7f1f340d0340 2021-02-10T11:43:14.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.521 7f1f33fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f1f340d1870 0x7f1f341b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59242/0 (socket says 172.21.2.101:59242) 2021-02-10T11:43:14.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.521 7f1f33fff700 1 -- 172.21.2.101:0/1436892581 learned_addr learned my addr 172.21.2.101:0/1436892581 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:14.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.521 7f1f337fe700 1 --2- 172.21.2.101:0/1436892581 >> v2:172.21.2.101:3300/0 conn(0x7f1f340d0340 0x7f1f340ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:14.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.521 7f1f337fe700 1 -- 172.21.2.101:0/1436892581 >> v1:172.21.2.108:6789/0 conn(0x7f1f340d1c70 legacy=0x7f1f340efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:14.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.521 7f1f337fe700 1 -- 172.21.2.101:0/1436892581 >> v1:172.21.2.101:6789/0 conn(0x7f1f340d1870 legacy=0x7f1f341b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:14.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.521 7f1f337fe700 1 -- 172.21.2.101:0/1436892581 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f340f0450 con 0x7f1f340d0340 2021-02-10T11:43:14.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.521 7f1f337fe700 1 --2- 172.21.2.101:0/1436892581 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f340d0340 0x7f1f340ef640 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 rx=0x7f1f28006650 tx=0x7f1f28009a80).ready entity=mon.0 client_cookie=5d5e0ad91cbdd5be server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:14.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.521 7f1f32ffd700 1 -- 172.21.2.101:0/1436892581 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f280009b0 con 0x7f1f340d0340 2021-02-10T11:43:14.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.521 7f1f32ffd700 1 -- 172.21.2.101:0/1436892581 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2800c0c0 con 0x7f1f340d0340 2021-02-10T11:43:14.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.521 7f1f32ffd700 1 -- 172.21.2.101:0/1436892581 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f2800c5e0 con 0x7f1f340d0340 2021-02-10T11:43:14.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.521 7f1f3a827700 1 -- 172.21.2.101:0/1436892581 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f340d0340 msgr2=0x7f1f340ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:14.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.521 7f1f3a827700 1 --2- 172.21.2.101:0/1436892581 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f340d0340 0x7f1f340ef640 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 rx=0x7f1f28006650 tx=0x7f1f28009a80).stop 2021-02-10T11:43:14.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.522 7f1f3a827700 1 -- 172.21.2.101:0/1436892581 shutdown_connections 2021-02-10T11:43:14.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.522 7f1f3a827700 1 --2- 172.21.2.101:0/1436892581 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f340d0340 0x7f1f340ef640 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:14.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.522 7f1f3a827700 1 -- 172.21.2.101:0/1436892581 >> 172.21.2.101:0/1436892581 conn(0x7f1f340ce450 msgr2=0x7f1f340d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:14.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.522 7f1f3a827700 1 -- 172.21.2.101:0/1436892581 shutdown_connections 2021-02-10T11:43:14.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.522 7f1f3a827700 1 -- 172.21.2.101:0/1436892581 wait complete. 2021-02-10T11:43:14.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.537 7f1f3a827700 1 Processor -- start 2021-02-10T11:43:14.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.537 7f1f3a827700 1 -- start start 2021-02-10T11:43:14.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.537 7f1f3a827700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f340b1130 0x7f1f340b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:14.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.537 7f1f3a827700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f340b1b50 0x7f1f340bb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:14.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.537 7f1f3a827700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f340bb650 0x7f1f340bba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:14.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.537 7f1f3a827700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3408baa0 con 0x7f1f340bb650 2021-02-10T11:43:14.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.537 7f1f3a827700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f340bc070 con 0x7f1f340b1130 2021-02-10T11:43:14.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.537 7f1f3a827700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3408c020 con 0x7f1f340b1b50 2021-02-10T11:43:14.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.537 7f1f38e44700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f340bb650 0x7f1f340bba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:14.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.537 7f1f38e44700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f340bb650 0x7f1f340bba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58756/0 (socket says 172.21.2.101:58756) 2021-02-10T11:43:14.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.537 7f1f38e44700 1 -- 172.21.2.101:0/2960671859 learned_addr learned my addr 172.21.2.101:0/2960671859 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:14.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.538 7f1f337fe700 1 --2- 172.21.2.101:0/2960671859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f340b1b50 0x7f1f340bb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:14.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.538 7f1f38e44700 1 -- 172.21.2.101:0/2960671859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f340b1b50 msgr2=0x7f1f340bb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:14.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.538 7f1f38e44700 1 --2- 172.21.2.101:0/2960671859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f340b1b50 0x7f1f340bb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:14.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.538 7f1f38e44700 1 -- 172.21.2.101:0/2960671859 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f340b1130 msgr2=0x7f1f340b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:14.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.538 7f1f38e44700 1 --2- 172.21.2.101:0/2960671859 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f340b1130 0x7f1f340b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:14.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.538 7f1f38e44700 1 -- 172.21.2.101:0/2960671859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3408c240 con 0x7f1f340bb650 2021-02-10T11:43:14.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.538 7f1f337fe700 1 --2- 172.21.2.101:0/2960671859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f340b1b50 0x7f1f340bb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:43:14.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.538 7f1f38e44700 1 --2- 172.21.2.101:0/2960671859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f340bb650 0x7f1f340bba50 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 rx=0x7f1f2c009940 tx=0x7f1f2c00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:14.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.538 7f1f30ff9700 1 -- 172.21.2.101:0/2960671859 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f2c000f40 con 0x7f1f340bb650 2021-02-10T11:43:14.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.538 7f1f30ff9700 1 -- 172.21.2.101:0/2960671859 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2c00c2f0 con 0x7f1f340bb650 2021-02-10T11:43:14.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.538 7f1f30ff9700 1 -- 172.21.2.101:0/2960671859 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f2c00c810 con 0x7f1f340bb650 2021-02-10T11:43:14.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.538 7f1f3a827700 1 -- 172.21.2.101:0/2960671859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3408c240 con 0x7f1f340bb650 2021-02-10T11:43:14.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.538 7f1f3a827700 1 -- 172.21.2.101:0/2960671859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3408c7d0 con 0x7f1f340bb650 2021-02-10T11:43:14.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.538 7f1f30ff9700 1 -- 172.21.2.101:0/2960671859 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1f2c001220 con 0x7f1f340bb650 2021-02-10T11:43:14.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.539 7f1f30ff9700 1 --2- 172.21.2.101:0/2960671859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f140186c0 0x7f1f1401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:14.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.539 7f1f30ff9700 1 -- 172.21.2.101:0/2960671859 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 4839+0+0 (secure 0 0 0) 0x7f1f2c032610 con 0x7f1f340bb650 2021-02-10T11:43:14.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.539 7f1f33fff700 1 --2- 172.21.2.101:0/2960671859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f140186c0 0x7f1f1401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:14.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.539 7f1f33fff700 1 --2- 172.21.2.101:0/2960671859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f140186c0 0x7f1f1401abc0 crc :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:14.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.569 7f1f3a827700 1 -- 172.21.2.101:0/2960671859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f340ffa00 con 0x7f1f340bb650 2021-02-10T11:43:14.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.572 7f1f30ff9700 1 -- 172.21.2.101:0/2960671859 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1f2c000f40 con 0x7f1f340bb650 2021-02-10T11:43:14.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:14.712 7f1f3a827700 1 -- 172.21.2.101:0/2960671859 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f1f340f06f0 con 0x7f1f340bb650 2021-02-10T11:43:15.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.019 7f1f30ff9700 1 -- 172.21.2.101:0/2960671859 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v133) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1f2c01f860 con 0x7f1f340bb650 2021-02-10T11:43:15.029 INFO:teuthology.orchestra.run.gibba001.stderr: removed all blacklist entries 2021-02-10T11:43:15.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.030 7f1f3a827700 1 -- 172.21.2.101:0/2960671859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f140186c0 msgr2=0x7f1f1401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:15.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.030 7f1f3a827700 1 --2- 172.21.2.101:0/2960671859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f140186c0 0x7f1f1401abc0 crc :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:15.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.030 7f1f3a827700 1 -- 172.21.2.101:0/2960671859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f340bb650 msgr2=0x7f1f340bba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:15.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.030 7f1f3a827700 1 --2- 172.21.2.101:0/2960671859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f340bb650 0x7f1f340bba50 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 rx=0x7f1f2c009940 tx=0x7f1f2c00bcf0).stop 2021-02-10T11:43:15.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.030 7f1f3a827700 1 -- 172.21.2.101:0/2960671859 shutdown_connections 2021-02-10T11:43:15.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.030 7f1f3a827700 1 --2- 172.21.2.101:0/2960671859 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f140186c0 0x7f1f1401abc0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:15.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.030 7f1f3a827700 1 --2- 172.21.2.101:0/2960671859 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f340b1b50 0x7f1f340bb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:15.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.030 7f1f3a827700 1 --2- 172.21.2.101:0/2960671859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f340bb650 0x7f1f340bba50 unknown :-1 s=CLOSED pgs=2519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:15.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.030 7f1f3a827700 1 --2- 172.21.2.101:0/2960671859 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f340b1130 0x7f1f340b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:15.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.030 7f1f3a827700 1 -- 172.21.2.101:0/2960671859 >> 172.21.2.101:0/2960671859 conn(0x7f1f340a74c0 msgr2=0x7f1f340bc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:15.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.030 7f1f3a827700 1 -- 172.21.2.101:0/2960671859 shutdown_connections 2021-02-10T11:43:15.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.030 7f1f3a827700 1 -- 172.21.2.101:0/2960671859 wait complete. 2021-02-10T11:43:15.040 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2021-02-10T11:43:15.040 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:43:15.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.450 7f9eaa7bb700 1 Processor -- start 2021-02-10T11:43:15.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.450 7f9eaa7bb700 1 -- start start 2021-02-10T11:43:15.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.451 7f9eaa7bb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ea40d0340 0x7f9ea40ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:15.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.451 7f9eaa7bb700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9ea40d1c70 0x7f9ea40efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:15.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.451 7f9eaa7bb700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea40f0b10 con 0x7f9ea40d1870 2021-02-10T11:43:15.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.451 7f9eaa7bb700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea40e5ed0 con 0x7f9ea40d1c70 2021-02-10T11:43:15.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.451 7f9eaa7bb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea41b0be0 con 0x7f9ea40d0340 2021-02-10T11:43:15.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.451 7f9ea8dd8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9ea40d1c70 0x7f9ea40efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:15.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.451 7f9ea8dd8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9ea40d1c70 0x7f9ea40efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58764/0 (socket says 172.21.2.101:58764) 2021-02-10T11:43:15.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.451 7f9ea8dd8700 1 -- 172.21.2.101:0/570298841 learned_addr learned my addr 172.21.2.101:0/570298841 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:15.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.451 7f9ea8dd8700 1 -- 172.21.2.101:0/570298841 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ea40d0340 msgr2=0x7f9ea40ef640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:43:15.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.451 7f9ea8dd8700 1 --2- 172.21.2.101:0/570298841 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ea40d0340 0x7f9ea40ef640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:15.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.451 7f9ea8dd8700 1 -- 172.21.2.101:0/570298841 >> v1:172.21.2.101:6789/0 conn(0x7f9ea40d1870 legacy=0x7f9ea41b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:15.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.451 7f9ea8dd8700 1 -- 172.21.2.101:0/570298841 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea40f20a0 con 0x7f9ea40d1c70 2021-02-10T11:43:15.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.451 7f9ea8dd8700 1 --2- 172.21.2.101:0/570298841 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ea40d1c70 0x7f9ea40efc60 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 rx=0x7f9e9c006f00 tx=0x7f9e9c009f10).ready entity=mon.0 client_cookie=56151d843036b4b3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:15.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.451 7f9ea2ffd700 1 -- 172.21.2.101:0/570298841 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9e9c00b3b0 con 0x7f9ea40d1c70 2021-02-10T11:43:15.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.452 7f9ea2ffd700 1 -- 172.21.2.101:0/570298841 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c000d30 con 0x7f9ea40d1c70 2021-02-10T11:43:15.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.452 7f9ea2ffd700 1 -- 172.21.2.101:0/570298841 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9e9c00b8b0 con 0x7f9ea40d1c70 2021-02-10T11:43:15.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.452 7f9eaa7bb700 1 -- 172.21.2.101:0/570298841 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ea40d1c70 msgr2=0x7f9ea40efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:15.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.452 7f9eaa7bb700 1 --2- 172.21.2.101:0/570298841 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ea40d1c70 0x7f9ea40efc60 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 rx=0x7f9e9c006f00 tx=0x7f9e9c009f10).stop 2021-02-10T11:43:15.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.452 7f9eaa7bb700 1 -- 172.21.2.101:0/570298841 shutdown_connections 2021-02-10T11:43:15.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.452 7f9eaa7bb700 1 --2- 172.21.2.101:0/570298841 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ea40d0340 0x7f9ea40ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:15.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.452 7f9eaa7bb700 1 --2- 172.21.2.101:0/570298841 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ea40d1c70 0x7f9ea40efc60 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:15.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.452 7f9eaa7bb700 1 -- 172.21.2.101:0/570298841 >> 172.21.2.101:0/570298841 conn(0x7f9ea40ce450 msgr2=0x7f9ea40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:15.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.452 7f9eaa7bb700 1 -- 172.21.2.101:0/570298841 shutdown_connections 2021-02-10T11:43:15.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.452 7f9eaa7bb700 1 -- 172.21.2.101:0/570298841 wait complete. 2021-02-10T11:43:15.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.467 7f9eaa7bb700 1 Processor -- start 2021-02-10T11:43:15.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.467 7f9eaa7bb700 1 -- start start 2021-02-10T11:43:15.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.467 7f9eaa7bb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ea4099b00 0x7f9ea4187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:15.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.467 7f9eaa7bb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ea4099f00 0x7f9ea4188470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:15.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.467 7f9eaa7bb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9ea4188a90 0x7f9ea4189870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:15.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.467 7f9eaa7bb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea408fb30 con 0x7f9ea4099f00 2021-02-10T11:43:15.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.467 7f9eaa7bb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea4188e90 con 0x7f9ea4188a90 2021-02-10T11:43:15.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.467 7f9eaa7bb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea41890b0 con 0x7f9ea4099b00 2021-02-10T11:43:15.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.467 7f9ea37fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ea4099f00 0x7f9ea4188470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:15.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.467 7f9ea37fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ea4099f00 0x7f9ea4188470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58768/0 (socket says 172.21.2.101:58768) 2021-02-10T11:43:15.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.467 7f9ea37fe700 1 -- 172.21.2.101:0/1484468632 learned_addr learned my addr 172.21.2.101:0/1484468632 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:15.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.468 7f9ea37fe700 1 -- 172.21.2.101:0/1484468632 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ea4099b00 msgr2=0x7f9ea4187e50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:15.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.468 7f9ea37fe700 1 --2- 172.21.2.101:0/1484468632 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ea4099b00 0x7f9ea4187e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:15.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.468 7f9ea37fe700 1 -- 172.21.2.101:0/1484468632 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9ea4188a90 msgr2=0x7f9ea4189870 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:15.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.468 7f9ea37fe700 1 --2- 172.21.2.101:0/1484468632 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9ea4188a90 0x7f9ea4189870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:15.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.468 7f9ea37fe700 1 -- 172.21.2.101:0/1484468632 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea4189e90 con 0x7f9ea4099f00 2021-02-10T11:43:15.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.468 7f9ea37fe700 1 --2- 172.21.2.101:0/1484468632 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ea4099f00 0x7f9ea4188470 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 rx=0x7f9e9800b5c0 tx=0x7f9e9800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:15.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.468 7f9ea0ff9700 1 -- 172.21.2.101:0/1484468632 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9e9800bd50 con 0x7f9ea4099f00 2021-02-10T11:43:15.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.468 7f9ea0ff9700 1 -- 172.21.2.101:0/1484468632 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e98006030 con 0x7f9ea4099f00 2021-02-10T11:43:15.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.468 7f9ea0ff9700 1 -- 172.21.2.101:0/1484468632 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9e980054d0 con 0x7f9ea4099f00 2021-02-10T11:43:15.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.468 7f9eaa7bb700 1 -- 172.21.2.101:0/1484468632 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea4189e90 con 0x7f9ea4099f00 2021-02-10T11:43:15.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.468 7f9eaa7bb700 1 -- 172.21.2.101:0/1484468632 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea418a420 con 0x7f9ea4099f00 2021-02-10T11:43:15.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.469 7f9ea0ff9700 1 -- 172.21.2.101:0/1484468632 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9e98006210 con 0x7f9ea4099f00 2021-02-10T11:43:15.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.469 7f9ea0ff9700 1 --2- 172.21.2.101:0/1484468632 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9e840186c0 0x7f9e8401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:15.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.469 7f9ea0ff9700 1 -- 172.21.2.101:0/1484468632 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f9e98032250 con 0x7f9ea4099f00 2021-02-10T11:43:15.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.469 7f9ea3fff700 1 --2- 172.21.2.101:0/1484468632 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9e840186c0 0x7f9e8401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:15.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.469 7f9ea3fff700 1 --2- 172.21.2.101:0/1484468632 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9e840186c0 0x7f9e8401abc0 crc :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:15.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.501 7f9eaa7bb700 1 -- 172.21.2.101:0/1484468632 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea41890b0 con 0x7f9ea4099f00 2021-02-10T11:43:15.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.504 7f9ea0ff9700 1 -- 172.21.2.101:0/1484468632 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9e98006b80 con 0x7f9ea4099f00 2021-02-10T11:43:15.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.634 7f9eaa7bb700 1 -- 172.21.2.101:0/1484468632 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9ea4189340 con 0x7f9ea4099f00 2021-02-10T11:43:15.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.635 7f9ea0ff9700 1 -- 172.21.2.101:0/1484468632 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v78) v1 ==== 80+0+4513 (secure 0 0 0) 0x7f9e9800bd50 con 0x7f9ea4099f00 2021-02-10T11:43:15.636 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:43:15.636 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:43:15.636 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:43:15.636 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:15.637 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:43:15.637 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:43:15.637 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:43:15.637 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:43:15.637 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:43:15.637 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:43:15.637 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:43:15.637 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:15.637 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:15.637 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:15.637 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:43:15.637 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:43:15.637 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:43:15.638 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:43:15.638 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:43:15.638 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:43:15.638 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:43:15.638 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:15.638 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:15.638 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:15.638 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:43:15.638 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:43:15.638 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:43:15.638 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:43:15.638 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:43:15.638 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:43:15.639 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:15.639 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:15.639 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:15.639 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:43:15.639 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:43:15.639 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:43:15.639 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:43:15.639 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:43:15.639 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:43:15.639 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:15.639 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:15.639 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:15.639 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:43:15.639 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:43:15.640 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:43:15.640 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:43:15.640 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:43:15.640 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:43:15.640 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:15.640 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:15.640 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:15.640 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:43:15.640 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:43:15.640 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:43:15.640 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:43:15.640 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:43:15.641 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:43:15.641 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:15.641 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:15.641 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:15.641 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:43:15.641 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:43:15.641 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:43:15.641 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:43:15.641 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:43:15.641 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:43:15.641 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:15.641 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:15.642 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:15.642 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:43:15.642 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:43:15.642 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:43:15.643 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:43:15.643 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:43:15.643 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:43:15.643 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:15.643 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:15.643 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:15.643 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:43:15.643 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:43:15.643 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:43:15.643 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:43:15.643 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:43:15.643 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:43:15.643 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:15.644 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:15.644 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:15.644 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:43:15.644 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:43:15.644 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:43:15.644 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:43:15.644 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:43:15.644 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:43:15.644 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:15.644 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:15.644 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:15.644 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:43:15.645 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:43:15.645 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:43:15.645 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:43:15.645 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:43:15.645 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:43:15.645 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:15.645 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:15.645 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:15.646 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:43:15.646 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:43:15.646 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:43:15.647 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:43:15.647 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:43:15.647 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:43:15.647 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:15.647 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:15.647 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:15.647 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:43:15.647 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:43:15.647 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:43:15.647 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:43:15.647 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:43:15.647 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:43:15.648 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:15.648 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:15.648 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:15.648 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:43:15.648 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:43:15.648 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:43:15.648 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:43:15.648 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:43:15.648 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:43:15.648 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:15.648 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:15.648 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:15.648 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:43:15.649 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:43:15.649 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:43:15.649 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:43:15.649 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:43:15.649 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:43:15.649 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:43:15.649 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:15.649 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:15.649 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:15.649 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:43:15.649 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:43:15.649 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:43:15.650 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:43:15.650 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:43:15.650 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:43:15.650 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:15.650 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:15.650 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:15.650 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:43:15.650 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:43:15.650 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:43:15.650 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:43:15.650 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:43:15.650 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:43:15.651 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:15.651 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:15.651 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:15.651 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:43:15.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.637 7f9eaa7bb700 1 -- 172.21.2.101:0/1484468632 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9e840186c0 msgr2=0x7f9e8401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:15.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.637 7f9eaa7bb700 1 --2- 172.21.2.101:0/1484468632 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9e840186c0 0x7f9e8401abc0 crc :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:15.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.637 7f9eaa7bb700 1 -- 172.21.2.101:0/1484468632 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ea4099f00 msgr2=0x7f9ea4188470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:15.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.637 7f9eaa7bb700 1 --2- 172.21.2.101:0/1484468632 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ea4099f00 0x7f9ea4188470 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 rx=0x7f9e9800b5c0 tx=0x7f9e9800b960).stop 2021-02-10T11:43:15.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.637 7f9eaa7bb700 1 -- 172.21.2.101:0/1484468632 shutdown_connections 2021-02-10T11:43:15.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.637 7f9eaa7bb700 1 --2- 172.21.2.101:0/1484468632 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9ea4188a90 0x7f9ea4189870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:15.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.637 7f9eaa7bb700 1 --2- 172.21.2.101:0/1484468632 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ea4099f00 0x7f9ea4188470 unknown :-1 s=CLOSED pgs=2522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:15.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.637 7f9eaa7bb700 1 --2- 172.21.2.101:0/1484468632 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9e840186c0 0x7f9e8401abc0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:15.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.637 7f9eaa7bb700 1 --2- 172.21.2.101:0/1484468632 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ea4099b00 0x7f9ea4187e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:15.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.637 7f9eaa7bb700 1 -- 172.21.2.101:0/1484468632 >> 172.21.2.101:0/1484468632 conn(0x7f9ea419f8c0 msgr2=0x7f9ea41a5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:15.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.637 7f9eaa7bb700 1 -- 172.21.2.101:0/1484468632 shutdown_connections 2021-02-10T11:43:15.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:15.637 7f9eaa7bb700 1 -- 172.21.2.101:0/1484468632 wait complete. 2021-02-10T11:43:15.654 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:43:16.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.065 7fd38f57a700 1 Processor -- start 2021-02-10T11:43:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.065 7fd38f57a700 1 -- start start 2021-02-10T11:43:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.066 7fd38f57a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3881b0470 0x7fd3881b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.066 7fd38f57a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd3880f20e0 0x7fd3880f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.066 7fd38f57a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3880f0b50 con 0x7fd3880ef610 2021-02-10T11:43:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.066 7fd38f57a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3880f0820 con 0x7fd3880f20e0 2021-02-10T11:43:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.066 7fd38f57a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3880f24e0 con 0x7fd3881b0470 2021-02-10T11:43:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.066 7fd38cb95700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd3880ef610 0x7fd3880efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59270/0 (socket says 172.21.2.101:59270) 2021-02-10T11:43:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.066 7fd38cb95700 1 -- 172.21.2.101:0/442800227 learned_addr learned my addr 172.21.2.101:0/442800227 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.066 7fd37bfff700 1 -- 172.21.2.101:0/442800227 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 332737282 0 0) 0x7fd3840010b0 con 0x7fd3880ef610 2021-02-10T11:43:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.066 7fd37bfff700 1 -- 172.21.2.101:0/442800227 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd374002fe0 con 0x7fd3880ef610 2021-02-10T11:43:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.066 7fd38db97700 1 --2- 172.21.2.101:0/442800227 >> v2:172.21.2.108:3300/0 conn(0x7fd3880f20e0 0x7fd3880f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:16.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.066 7fd38d396700 1 --2- 172.21.2.101:0/442800227 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3881b0470 0x7fd3881b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:16.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd37bfff700 1 -- 172.21.2.101:0/442800227 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2708110455 0 0) 0x7fd3840010b0 con 0x7fd3880ef610 2021-02-10T11:43:16.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd37bfff700 1 -- 172.21.2.101:0/442800227 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3881b0470 msgr2=0x7fd3881b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:16.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd37bfff700 1 --2- 172.21.2.101:0/442800227 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3881b0470 0x7fd3881b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:16.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd37bfff700 1 -- 172.21.2.101:0/442800227 >> v2:172.21.2.108:3300/0 conn(0x7fd3880f20e0 msgr2=0x7fd3880f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:16.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd37bfff700 1 --2- 172.21.2.101:0/442800227 >> v2:172.21.2.108:3300/0 conn(0x7fd3880f20e0 0x7fd3880f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:16.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd37bfff700 1 -- 172.21.2.101:0/442800227 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3880e52e0 con 0x7fd3880ef610 2021-02-10T11:43:16.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd37bfff700 1 -- 172.21.2.101:0/442800227 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fd3840015b0 con 0x7fd3880ef610 2021-02-10T11:43:16.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd37bfff700 1 -- 172.21.2.101:0/442800227 >> v1:172.21.2.101:6789/0 conn(0x7fd3880ef610 legacy=0x7fd3880efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:16.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd37bfff700 1 --2- 172.21.2.101:0/442800227 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd374003ac0 0x7fd374005f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:16.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd37bfff700 1 --2- 172.21.2.101:0/442800227 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd374006750 0x7fd374008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:16.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd37bfff700 1 --2- 172.21.2.101:0/442800227 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd374009510 0x7fd37400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:16.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd37bfff700 1 -- 172.21.2.101:0/442800227 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3740502d0 con 0x7fd374009510 2021-02-10T11:43:16.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd37bfff700 1 -- 172.21.2.101:0/442800227 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3740504f0 con 0x7fd374006750 2021-02-10T11:43:16.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd37bfff700 1 -- 172.21.2.101:0/442800227 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd374050710 con 0x7fd374003ac0 2021-02-10T11:43:16.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd38db97700 1 --2- 172.21.2.101:0/442800227 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd374009510 0x7fd37400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:16.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd38d396700 1 --2- 172.21.2.101:0/442800227 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd374003ac0 0x7fd374005f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:16.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd38cb95700 1 --2- 172.21.2.101:0/442800227 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd374006750 0x7fd374008c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:16.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd38db97700 1 -- 172.21.2.101:0/442800227 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd374003ac0 msgr2=0x7fd374005f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:16.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd38db97700 1 --2- 172.21.2.101:0/442800227 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd374003ac0 0x7fd374005f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:16.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd38db97700 1 -- 172.21.2.101:0/442800227 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd374006750 msgr2=0x7fd374008c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:16.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd38db97700 1 --2- 172.21.2.101:0/442800227 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd374006750 0x7fd374008c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:16.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd38db97700 1 -- 172.21.2.101:0/442800227 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3740509b0 con 0x7fd374009510 2021-02-10T11:43:16.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd38d396700 1 --2- 172.21.2.101:0/442800227 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd374003ac0 0x7fd374005f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:43:16.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd38db97700 1 --2- 172.21.2.101:0/442800227 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd374009510 0x7fd37400ba10 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 rx=0x7fd380006a40 tx=0x7fd38000c0c0).ready entity=mon.0 client_cookie=ff99fe7c2fc11dcf server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:16.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.067 7fd37bfff700 1 -- 172.21.2.101:0/442800227 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd38000c4b0 con 0x7fd374009510 2021-02-10T11:43:16.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.068 7fd37bfff700 1 -- 172.21.2.101:0/442800227 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38000c770 con 0x7fd374009510 2021-02-10T11:43:16.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.068 7fd37bfff700 1 -- 172.21.2.101:0/442800227 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd38000cc50 con 0x7fd374009510 2021-02-10T11:43:16.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.068 7fd38f57a700 1 -- 172.21.2.101:0/442800227 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd374009510 msgr2=0x7fd37400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:16.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.068 7fd38f57a700 1 --2- 172.21.2.101:0/442800227 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd374009510 0x7fd37400ba10 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 rx=0x7fd380006a40 tx=0x7fd38000c0c0).stop 2021-02-10T11:43:16.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.068 7fd38db97700 1 -- 172.21.2.101:0/442800227 reap_dead start 2021-02-10T11:43:16.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.068 7fd38f57a700 1 -- 172.21.2.101:0/442800227 shutdown_connections 2021-02-10T11:43:16.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.068 7fd38f57a700 1 -- 172.21.2.101:0/442800227 >> 172.21.2.101:0/442800227 conn(0x7fd3880ce3e0 msgr2=0x7fd3880e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:16.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.068 7fd38f57a700 1 -- 172.21.2.101:0/442800227 shutdown_connections 2021-02-10T11:43:16.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.068 7fd38f57a700 1 -- 172.21.2.101:0/442800227 wait complete. 2021-02-10T11:43:16.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.083 7fd38f57a700 1 Processor -- start 2021-02-10T11:43:16.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.083 7fd38f57a700 1 -- start start 2021-02-10T11:43:16.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.083 7fd38f57a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd388108ce0 0x7fd38809da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:16.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.083 7fd38f57a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3881090e0 0x7fd38809e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:16.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.083 7fd38f57a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd38809eee0 0x7fd38809f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:16.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.083 7fd38f57a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3881094e0 con 0x7fd3881090e0 2021-02-10T11:43:16.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.083 7fd38f57a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38809e960 con 0x7fd38809eee0 2021-02-10T11:43:16.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.083 7fd38f57a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38809e690 con 0x7fd388108ce0 2021-02-10T11:43:16.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.083 7fd38cb95700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3881090e0 0x7fd38809e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:16.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.083 7fd38cb95700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3881090e0 0x7fd38809e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58786/0 (socket says 172.21.2.101:58786) 2021-02-10T11:43:16.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.083 7fd38cb95700 1 -- 172.21.2.101:0/3032981880 learned_addr learned my addr 172.21.2.101:0/3032981880 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:16.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.084 7fd38cb95700 1 -- 172.21.2.101:0/3032981880 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd388108ce0 msgr2=0x7fd38809da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:16.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.084 7fd38cb95700 1 --2- 172.21.2.101:0/3032981880 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd388108ce0 0x7fd38809da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:16.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.084 7fd38cb95700 1 -- 172.21.2.101:0/3032981880 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd38809eee0 msgr2=0x7fd38809f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:16.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.084 7fd38cb95700 1 --2- 172.21.2.101:0/3032981880 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd38809eee0 0x7fd38809f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:16.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.084 7fd38cb95700 1 -- 172.21.2.101:0/3032981880 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38809f900 con 0x7fd3881090e0 2021-02-10T11:43:16.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.084 7fd38cb95700 1 --2- 172.21.2.101:0/3032981880 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3881090e0 0x7fd38809e070 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 rx=0x7fd384001c00 tx=0x7fd38400a8e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:16.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.084 7fd379ffb700 1 -- 172.21.2.101:0/3032981880 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd3840009b0 con 0x7fd3881090e0 2021-02-10T11:43:16.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.084 7fd379ffb700 1 -- 172.21.2.101:0/3032981880 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38400acd0 con 0x7fd3881090e0 2021-02-10T11:43:16.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.084 7fd379ffb700 1 -- 172.21.2.101:0/3032981880 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd384004540 con 0x7fd3881090e0 2021-02-10T11:43:16.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.084 7fd38f57a700 1 -- 172.21.2.101:0/3032981880 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38809f900 con 0x7fd3881090e0 2021-02-10T11:43:16.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.084 7fd38f57a700 1 -- 172.21.2.101:0/3032981880 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38809fe90 con 0x7fd3881090e0 2021-02-10T11:43:16.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.085 7fd379ffb700 1 -- 172.21.2.101:0/3032981880 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd384004140 con 0x7fd3881090e0 2021-02-10T11:43:16.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.085 7fd379ffb700 1 --2- 172.21.2.101:0/3032981880 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3680186c0 0x7fd36801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:16.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.085 7fd379ffb700 1 -- 172.21.2.101:0/3032981880 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fd384032280 con 0x7fd3881090e0 2021-02-10T11:43:16.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.085 7fd38d396700 1 --2- 172.21.2.101:0/3032981880 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3680186c0 0x7fd36801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:16.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.085 7fd38d396700 1 --2- 172.21.2.101:0/3032981880 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3680186c0 0x7fd36801abc0 crc :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:16.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.116 7fd38f57a700 1 -- 172.21.2.101:0/3032981880 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38809e960 con 0x7fd3881090e0 2021-02-10T11:43:16.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.118 7fd379ffb700 1 -- 172.21.2.101:0/3032981880 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd3840253d0 con 0x7fd3881090e0 2021-02-10T11:43:16.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.243 7fd38f57a700 1 -- 172.21.2.101:0/3032981880 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd38809e960 con 0x7fd3881090e0 2021-02-10T11:43:16.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.244 7fd379ffb700 1 -- 172.21.2.101:0/3032981880 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+11154 (secure 0 0 0) 0x7fd3840253d0 con 0x7fd3881090e0 2021-02-10T11:43:16.245 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:43:16.245 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":133,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:43:15.015348","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:43:16.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.245 7fd38f57a700 1 -- 172.21.2.101:0/3032981880 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3680186c0 msgr2=0x7fd36801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:16.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.245 7fd38f57a700 1 --2- 172.21.2.101:0/3032981880 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3680186c0 0x7fd36801abc0 crc :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:16.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.245 7fd38f57a700 1 -- 172.21.2.101:0/3032981880 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3881090e0 msgr2=0x7fd38809e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:16.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.245 7fd38f57a700 1 --2- 172.21.2.101:0/3032981880 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3881090e0 0x7fd38809e070 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 rx=0x7fd384001c00 tx=0x7fd38400a8e0).stop 2021-02-10T11:43:16.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.246 7fd38f57a700 1 -- 172.21.2.101:0/3032981880 shutdown_connections 2021-02-10T11:43:16.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.246 7fd38f57a700 1 --2- 172.21.2.101:0/3032981880 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd38809eee0 0x7fd38809f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:16.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.246 7fd38f57a700 1 --2- 172.21.2.101:0/3032981880 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3881090e0 0x7fd38809e070 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:16.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.246 7fd38f57a700 1 --2- 172.21.2.101:0/3032981880 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3680186c0 0x7fd36801abc0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:16.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.246 7fd38f57a700 1 --2- 172.21.2.101:0/3032981880 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd388108ce0 0x7fd38809da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:16.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.246 7fd38f57a700 1 -- 172.21.2.101:0/3032981880 >> 172.21.2.101:0/3032981880 conn(0x7fd3880bc6c0 msgr2=0x7fd3880b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:16.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.246 7fd38f57a700 1 -- 172.21.2.101:0/3032981880 shutdown_connections 2021-02-10T11:43:16.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.246 7fd38f57a700 1 -- 172.21.2.101:0/3032981880 wait complete. 2021-02-10T11:43:16.255 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-10T11:43:16.255 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:43:16.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.669 7f77f7278700 1 Processor -- start 2021-02-10T11:43:16.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.669 7f77f7278700 1 -- start start 2021-02-10T11:43:16.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.669 7f77f7278700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77f00d1870 0x7f77f01b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:16.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.669 7f77f7278700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f77f01b0ae0 0x7f77f0108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:16.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.669 7f77f7278700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f77f00d1c70 0x7f77f0108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:16.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.669 7f77f7278700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f00e5ef0 con 0x7f77f00d1c70 2021-02-10T11:43:16.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.669 7f77f7278700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f0109a60 con 0x7f77f01b0ae0 2021-02-10T11:43:16.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.669 7f77f7278700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f0088070 con 0x7f77f00d1870 2021-02-10T11:43:16.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.670 7f77f5895700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f77f00d1c70 0x7f77f0108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:16.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.670 7f77f5895700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f77f00d1c70 0x7f77f0108f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58794/0 (socket says 172.21.2.101:58794) 2021-02-10T11:43:16.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.670 7f77f5895700 1 -- 172.21.2.101:0/691371815 learned_addr learned my addr 172.21.2.101:0/691371815 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:16.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.670 7f77f5895700 1 -- 172.21.2.101:0/691371815 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77f00d1870 msgr2=0x7f77f01b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:43:16.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.670 7f77f5895700 1 --2- 172.21.2.101:0/691371815 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77f00d1870 0x7f77f01b04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:16.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.670 7f77f5895700 1 -- 172.21.2.101:0/691371815 >> v2:172.21.2.108:3300/0 conn(0x7f77f01b0ae0 msgr2=0x7f77f0108940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:43:16.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.670 7f77f5895700 1 --2- 172.21.2.101:0/691371815 >> v2:172.21.2.108:3300/0 conn(0x7f77f01b0ae0 0x7f77f0108940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:16.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.670 7f77f5895700 1 -- 172.21.2.101:0/691371815 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f0109580 con 0x7f77f00d1c70 2021-02-10T11:43:16.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.670 7f77f5895700 1 --2- 172.21.2.101:0/691371815 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77f00d1c70 0x7f77f0108f60 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 rx=0x7f77e8006f00 tx=0x7f77e8009f10).ready entity=mon.0 client_cookie=2c973f6b492cc6f7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:16.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.670 7f77e3fff700 1 -- 172.21.2.101:0/691371815 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f77e800b3b0 con 0x7f77f00d1c70 2021-02-10T11:43:16.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.670 7f77e3fff700 1 -- 172.21.2.101:0/691371815 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e800b630 con 0x7f77f00d1c70 2021-02-10T11:43:16.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.670 7f77e3fff700 1 -- 172.21.2.101:0/691371815 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f77e800bb50 con 0x7f77f00d1c70 2021-02-10T11:43:16.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.670 7f77f7278700 1 -- 172.21.2.101:0/691371815 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77f00d1c70 msgr2=0x7f77f0108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:16.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.670 7f77f7278700 1 --2- 172.21.2.101:0/691371815 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77f00d1c70 0x7f77f0108f60 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 rx=0x7f77e8006f00 tx=0x7f77e8009f10).stop 2021-02-10T11:43:16.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.671 7f77f7278700 1 -- 172.21.2.101:0/691371815 shutdown_connections 2021-02-10T11:43:16.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.671 7f77f7278700 1 --2- 172.21.2.101:0/691371815 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77f00d1c70 0x7f77f0108f60 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:16.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.671 7f77f7278700 1 --2- 172.21.2.101:0/691371815 >> v2:172.21.2.108:3300/0 conn(0x7f77f01b0ae0 0x7f77f0108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:16.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.671 7f77f7278700 1 --2- 172.21.2.101:0/691371815 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77f00d1870 0x7f77f01b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:16.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.671 7f77f7278700 1 -- 172.21.2.101:0/691371815 >> 172.21.2.101:0/691371815 conn(0x7f77f00ce420 msgr2=0x7f77f00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:16.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.671 7f77f7278700 1 -- 172.21.2.101:0/691371815 shutdown_connections 2021-02-10T11:43:16.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.671 7f77f7278700 1 -- 172.21.2.101:0/691371815 wait complete. 2021-02-10T11:43:16.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.686 7f77f7278700 1 Processor -- start 2021-02-10T11:43:16.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.686 7f77f7278700 1 -- start start 2021-02-10T11:43:16.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.686 7f77f7278700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77f00872c0 0x7f77f01a7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:16.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.686 7f77f7278700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77f01a7ba0 0x7f77f00a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:16.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.686 7f77f7278700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77f01a7fa0 0x7f77f00a6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:16.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.686 7f77f7278700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f0086e20 con 0x7f77f01a7ba0 2021-02-10T11:43:16.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.686 7f77f7278700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f0190c60 con 0x7f77f00872c0 2021-02-10T11:43:16.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.686 7f77f7278700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f00a74a0 con 0x7f77f01a7fa0 2021-02-10T11:43:16.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.686 7f77f4893700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77f01a7ba0 0x7f77f00a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:16.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.687 7f77f4893700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77f01a7ba0 0x7f77f00a6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58796/0 (socket says 172.21.2.101:58796) 2021-02-10T11:43:16.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.687 7f77f4893700 1 -- 172.21.2.101:0/2687819091 learned_addr learned my addr 172.21.2.101:0/2687819091 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:16.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.687 7f77f4893700 1 -- 172.21.2.101:0/2687819091 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77f01a7fa0 msgr2=0x7f77f00a6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:16.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.687 7f77f4893700 1 --2- 172.21.2.101:0/2687819091 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77f01a7fa0 0x7f77f00a6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:16.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.687 7f77f4893700 1 -- 172.21.2.101:0/2687819091 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77f00872c0 msgr2=0x7f77f01a7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:16.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.687 7f77f5094700 1 --2- 172.21.2.101:0/2687819091 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77f00872c0 0x7f77f01a7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:16.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.687 7f77f4893700 1 --2- 172.21.2.101:0/2687819091 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77f00872c0 0x7f77f01a7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:16.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.687 7f77f4893700 1 -- 172.21.2.101:0/2687819091 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f00a76c0 con 0x7f77f01a7ba0 2021-02-10T11:43:16.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.687 7f77f5094700 1 --2- 172.21.2.101:0/2687819091 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77f00872c0 0x7f77f01a7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:43:16.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.687 7f77f4893700 1 --2- 172.21.2.101:0/2687819091 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77f01a7ba0 0x7f77f00a6860 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 rx=0x7f77ec00b560 tx=0x7f77ec00b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:16.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.687 7f77e1ffb700 1 -- 172.21.2.101:0/2687819091 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f77ec00bcf0 con 0x7f77f01a7ba0 2021-02-10T11:43:16.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.687 7f77e1ffb700 1 -- 172.21.2.101:0/2687819091 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ec006030 con 0x7f77f01a7ba0 2021-02-10T11:43:16.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.687 7f77e1ffb700 1 -- 172.21.2.101:0/2687819091 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f77ec0054d0 con 0x7f77f01a7ba0 2021-02-10T11:43:16.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.688 7f77f7278700 1 -- 172.21.2.101:0/2687819091 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f00a76c0 con 0x7f77f01a7ba0 2021-02-10T11:43:16.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.688 7f77f7278700 1 -- 172.21.2.101:0/2687819091 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f00b0090 con 0x7f77f01a7ba0 2021-02-10T11:43:16.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.688 7f77e1ffb700 1 -- 172.21.2.101:0/2687819091 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f77ec006210 con 0x7f77f01a7ba0 2021-02-10T11:43:16.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.688 7f77e1ffb700 1 --2- 172.21.2.101:0/2687819091 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77d00186c0 0x7f77d001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:16.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.688 7f77e1ffb700 1 -- 172.21.2.101:0/2687819091 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f77ec032090 con 0x7f77f01a7ba0 2021-02-10T11:43:16.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.688 7f77f5094700 1 --2- 172.21.2.101:0/2687819091 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77d00186c0 0x7f77d001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:16.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.688 7f77f5094700 1 --2- 172.21.2.101:0/2687819091 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77d00186c0 0x7f77d001abc0 crc :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:16.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.720 7f77f7278700 1 -- 172.21.2.101:0/2687819091 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f0082980 con 0x7f77f01a7ba0 2021-02-10T11:43:16.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.722 7f77e1ffb700 1 -- 172.21.2.101:0/2687819091 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f77ec0009b0 con 0x7f77f01a7ba0 2021-02-10T11:43:16.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.848 7f77f7278700 1 -- 172.21.2.101:0/2687819091 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f77f0109a60 con 0x7f77f01a7ba0 2021-02-10T11:43:16.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.848 7f77e1ffb700 1 -- 172.21.2.101:0/2687819091 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v133) v1 ==== 92+0+10 (secure 0 0 0) 0x7f77ec000c40 con 0x7f77f01a7ba0 2021-02-10T11:43:16.850 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:43:16.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.850 7f77f7278700 1 -- 172.21.2.101:0/2687819091 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77d00186c0 msgr2=0x7f77d001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:16.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.851 7f77f7278700 1 --2- 172.21.2.101:0/2687819091 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77d00186c0 0x7f77d001abc0 crc :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:16.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.851 7f77f7278700 1 -- 172.21.2.101:0/2687819091 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77f01a7ba0 msgr2=0x7f77f00a6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:16.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.851 7f77f7278700 1 --2- 172.21.2.101:0/2687819091 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77f01a7ba0 0x7f77f00a6860 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 rx=0x7f77ec00b560 tx=0x7f77ec00b900).stop 2021-02-10T11:43:16.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.851 7f77f7278700 1 -- 172.21.2.101:0/2687819091 shutdown_connections 2021-02-10T11:43:16.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.851 7f77f7278700 1 --2- 172.21.2.101:0/2687819091 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f77d00186c0 0x7f77d001abc0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:16.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.851 7f77f7278700 1 --2- 172.21.2.101:0/2687819091 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77f01a7fa0 0x7f77f00a6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:16.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.851 7f77f7278700 1 --2- 172.21.2.101:0/2687819091 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77f01a7ba0 0x7f77f00a6860 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:16.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.851 7f77f7278700 1 --2- 172.21.2.101:0/2687819091 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f77f00872c0 0x7f77f01a7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:16.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.851 7f77f7278700 1 -- 172.21.2.101:0/2687819091 >> 172.21.2.101:0/2687819091 conn(0x7f77f00be980 msgr2=0x7f77f0092cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:16.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.851 7f77f7278700 1 -- 172.21.2.101:0/2687819091 shutdown_connections 2021-02-10T11:43:16.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:16.851 7f77f7278700 1 -- 172.21.2.101:0/2687819091 wait complete. 2021-02-10T11:43:16.860 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --format=json-pretty osd lspools 2021-02-10T11:43:17.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.270 7feee1c7c700 1 Processor -- start 2021-02-10T11:43:17.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.270 7feee1c7c700 1 -- start start 2021-02-10T11:43:17.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.271 7feee1c7c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7feedc0d1870 0x7feedc1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:17.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.271 7feee1c7c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7feedc0f0650 0x7feedc0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:17.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.271 7feee1c7c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feedc0f0b10 con 0x7feedc1b0a10 2021-02-10T11:43:17.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.271 7feee1c7c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feedc0e5ed0 con 0x7feedc0f0650 2021-02-10T11:43:17.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.271 7feee1c7c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feedc0d1c70 con 0x7feedc0d1870 2021-02-10T11:43:17.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.271 7feedbfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7feedc0f0650 0x7feedc0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:17.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.271 7feedbfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7feedc0f0650 0x7feedc0cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58804/0 (socket says 172.21.2.101:58804) 2021-02-10T11:43:17.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.271 7feedbfff700 1 -- 172.21.2.101:0/3144463177 learned_addr learned my addr 172.21.2.101:0/3144463177 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:17.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.272 7feedbfff700 1 -- 172.21.2.101:0/3144463177 >> v2:172.21.2.108:3300/0 conn(0x7feedc0d1870 msgr2=0x7feedc1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:17.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.272 7feedbfff700 1 --2- 172.21.2.101:0/3144463177 >> v2:172.21.2.108:3300/0 conn(0x7feedc0d1870 0x7feedc1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:17.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.272 7feedbfff700 1 -- 172.21.2.101:0/3144463177 >> v1:172.21.2.101:6789/0 conn(0x7feedc1b0a10 legacy=0x7feedc0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:17.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.272 7feedbfff700 1 -- 172.21.2.101:0/3144463177 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc0d1e90 con 0x7feedc0f0650 2021-02-10T11:43:17.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.272 7feedbfff700 1 --2- 172.21.2.101:0/3144463177 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feedc0f0650 0x7feedc0cfb20 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 rx=0x7feed4006f00 tx=0x7feed4009f10).ready entity=mon.0 client_cookie=e580e5704cae5fec server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:17.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.272 7feeda7fc700 1 -- 172.21.2.101:0/3144463177 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feed400b3b0 con 0x7feedc0f0650 2021-02-10T11:43:17.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.272 7feeda7fc700 1 -- 172.21.2.101:0/3144463177 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed400b670 con 0x7feedc0f0650 2021-02-10T11:43:17.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.272 7feeda7fc700 1 -- 172.21.2.101:0/3144463177 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feed400bb90 con 0x7feedc0f0650 2021-02-10T11:43:17.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.272 7feee1c7c700 1 -- 172.21.2.101:0/3144463177 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feedc0f0650 msgr2=0x7feedc0cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:17.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.272 7feee1c7c700 1 --2- 172.21.2.101:0/3144463177 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feedc0f0650 0x7feedc0cfb20 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 rx=0x7feed4006f00 tx=0x7feed4009f10).stop 2021-02-10T11:43:17.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.273 7feee1c7c700 1 -- 172.21.2.101:0/3144463177 shutdown_connections 2021-02-10T11:43:17.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.273 7feee1c7c700 1 --2- 172.21.2.101:0/3144463177 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feedc0f0650 0x7feedc0cfb20 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:17.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.273 7feee1c7c700 1 --2- 172.21.2.101:0/3144463177 >> v2:172.21.2.108:3300/0 conn(0x7feedc0d1870 0x7feedc1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:17.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.273 7feee1c7c700 1 -- 172.21.2.101:0/3144463177 >> 172.21.2.101:0/3144463177 conn(0x7feedc0ce450 msgr2=0x7feedc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:17.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.273 7feee1c7c700 1 -- 172.21.2.101:0/3144463177 shutdown_connections 2021-02-10T11:43:17.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.273 7feee1c7c700 1 -- 172.21.2.101:0/3144463177 wait complete. 2021-02-10T11:43:17.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.288 7feee1c7c700 1 Processor -- start 2021-02-10T11:43:17.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.288 7feee1c7c700 1 -- start start 2021-02-10T11:43:17.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.288 7feee1c7c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feedc0b0ad0 0x7feedc091f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:17.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.288 7feee1c7c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feedc1a8850 0x7feedc1a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:17.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.288 7feee1c7c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feedc1a6fa0 0x7feedc1a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:17.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.288 7feee1c7c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc1abfe0 con 0x7feedc1a8850 2021-02-10T11:43:17.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.288 7feee1c7c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc1a9c40 con 0x7feedc0b0ad0 2021-02-10T11:43:17.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.288 7feee1c7c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feedc1a9730 con 0x7feedc1a6fa0 2021-02-10T11:43:17.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.289 7feedaffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feedc1a8850 0x7feedc1a8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:17.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.289 7feedaffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feedc1a8850 0x7feedc1a8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58810/0 (socket says 172.21.2.101:58810) 2021-02-10T11:43:17.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.289 7feedaffd700 1 -- 172.21.2.101:0/4075532355 learned_addr learned my addr 172.21.2.101:0/4075532355 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:17.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.289 7feedaffd700 1 -- 172.21.2.101:0/4075532355 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feedc1a6fa0 msgr2=0x7feedc1a73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:17.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.289 7feedaffd700 1 --2- 172.21.2.101:0/4075532355 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feedc1a6fa0 0x7feedc1a73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:17.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.289 7feedaffd700 1 -- 172.21.2.101:0/4075532355 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feedc0b0ad0 msgr2=0x7feedc091f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:17.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.289 7feedaffd700 1 --2- 172.21.2.101:0/4075532355 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feedc0b0ad0 0x7feedc091f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:17.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.289 7feedaffd700 1 -- 172.21.2.101:0/4075532355 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc1a9270 con 0x7feedc1a8850 2021-02-10T11:43:17.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.289 7feedaffd700 1 --2- 172.21.2.101:0/4075532355 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feedc1a8850 0x7feedc1a8c50 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 rx=0x7feecc006ed0 tx=0x7feecc00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:17.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.289 7feed3fff700 1 -- 172.21.2.101:0/4075532355 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feecc006240 con 0x7feedc1a8850 2021-02-10T11:43:17.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.289 7feed3fff700 1 -- 172.21.2.101:0/4075532355 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feecc0054b0 con 0x7feedc1a8850 2021-02-10T11:43:17.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.289 7feed3fff700 1 -- 172.21.2.101:0/4075532355 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feecc005990 con 0x7feedc1a8850 2021-02-10T11:43:17.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.289 7feee1c7c700 1 -- 172.21.2.101:0/4075532355 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feedc1a9270 con 0x7feedc1a8850 2021-02-10T11:43:17.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.289 7feee1c7c700 1 -- 172.21.2.101:0/4075532355 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feedc1a7cb0 con 0x7feedc1a8850 2021-02-10T11:43:17.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.290 7feed3fff700 1 -- 172.21.2.101:0/4075532355 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7feecc006030 con 0x7feedc1a8850 2021-02-10T11:43:17.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.290 7feed3fff700 1 --2- 172.21.2.101:0/4075532355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feec00186c0 0x7feec001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:17.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.290 7feed3fff700 1 -- 172.21.2.101:0/4075532355 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 4753+0+0 (secure 0 0 0) 0x7feecc0324c0 con 0x7feedc1a8850 2021-02-10T11:43:17.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.290 7feedb7fe700 1 --2- 172.21.2.101:0/4075532355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feec00186c0 0x7feec001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:17.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.290 7feedb7fe700 1 --2- 172.21.2.101:0/4075532355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feec00186c0 0x7feec001abc0 crc :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:17.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.321 7feee1c7c700 1 -- 172.21.2.101:0/4075532355 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feedc1a9710 con 0x7feedc1a8850 2021-02-10T11:43:17.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.324 7feed3fff700 1 -- 172.21.2.101:0/4075532355 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7feecc0054b0 con 0x7feedc1a8850 2021-02-10T11:43:17.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.449 7feee1c7c700 1 -- 172.21.2.101:0/4075532355 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7feedc1a9710 con 0x7feedc1a8850 2021-02-10T11:43:17.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.450 7feed3fff700 1 -- 172.21.2.101:0/4075532355 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v133) v1 ==== 84+0+64 (secure 0 0 0) 0x7feecc006030 con 0x7feedc1a8850 2021-02-10T11:43:17.451 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:43:17.451 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:43:17.451 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:17.451 INFO:teuthology.orchestra.run.gibba001.stdout: "poolnum": 1, 2021-02-10T11:43:17.451 INFO:teuthology.orchestra.run.gibba001.stdout: "poolname": "rbd" 2021-02-10T11:43:17.451 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:17.451 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:43:17.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.451 7feee1c7c700 1 -- 172.21.2.101:0/4075532355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feec00186c0 msgr2=0x7feec001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:17.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.451 7feee1c7c700 1 --2- 172.21.2.101:0/4075532355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feec00186c0 0x7feec001abc0 crc :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:17.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.451 7feee1c7c700 1 -- 172.21.2.101:0/4075532355 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feedc1a8850 msgr2=0x7feedc1a8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:17.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.451 7feee1c7c700 1 --2- 172.21.2.101:0/4075532355 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feedc1a8850 0x7feedc1a8c50 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 rx=0x7feecc006ed0 tx=0x7feecc00b960).stop 2021-02-10T11:43:17.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.451 7feee1c7c700 1 -- 172.21.2.101:0/4075532355 shutdown_connections 2021-02-10T11:43:17.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.451 7feee1c7c700 1 --2- 172.21.2.101:0/4075532355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feec00186c0 0x7feec001abc0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:17.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.452 7feee1c7c700 1 --2- 172.21.2.101:0/4075532355 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feedc1a6fa0 0x7feedc1a73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:17.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.452 7feee1c7c700 1 --2- 172.21.2.101:0/4075532355 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feedc1a8850 0x7feedc1a8c50 unknown :-1 s=CLOSED pgs=2530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:17.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.452 7feee1c7c700 1 --2- 172.21.2.101:0/4075532355 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feedc0b0ad0 0x7feedc091f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:17.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.452 7feee1c7c700 1 -- 172.21.2.101:0/4075532355 >> 172.21.2.101:0/4075532355 conn(0x7feedc19f8f0 msgr2=0x7feedc1a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:17.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.452 7feee1c7c700 1 -- 172.21.2.101:0/4075532355 shutdown_connections 2021-02-10T11:43:17.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.452 7feee1c7c700 1 -- 172.21.2.101:0/4075532355 wait complete. 2021-02-10T11:43:17.461 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-02-10T11:43:17.461 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-02-10T11:43:17.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.876 7f8c35f86700 1 Processor -- start 2021-02-10T11:43:17.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.876 7f8c35f86700 1 -- start start 2021-02-10T11:43:17.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.876 7f8c35f86700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8c300d1870 0x7f8c301b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:17.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.876 7f8c35f86700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c301b0ae0 0x7f8c30108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:17.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.876 7f8c35f86700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c300fe1c0 con 0x7f8c300d1c70 2021-02-10T11:43:17.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.876 7f8c35f86700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c30109a60 con 0x7f8c300d1870 2021-02-10T11:43:17.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.876 7f8c35f86700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c30109750 con 0x7f8c301b0ae0 2021-02-10T11:43:17.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.877 7f8c2f7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8c300d1870 0x7f8c301b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:17.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.877 7f8c2f7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8c300d1870 0x7f8c301b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58816/0 (socket says 172.21.2.101:58816) 2021-02-10T11:43:17.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.877 7f8c2f7fe700 1 -- 172.21.2.101:0/2717395802 learned_addr learned my addr 172.21.2.101:0/2717395802 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:17.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.877 7f8c2f7fe700 1 -- 172.21.2.101:0/2717395802 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c301b0ae0 msgr2=0x7f8c30108940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:43:17.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.877 7f8c2f7fe700 1 --2- 172.21.2.101:0/2717395802 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c301b0ae0 0x7f8c30108940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:17.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.877 7f8c2f7fe700 1 -- 172.21.2.101:0/2717395802 >> v1:172.21.2.108:6789/0 conn(0x7f8c300d1c70 legacy=0x7f8c30108f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:43:17.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.877 7f8c2f7fe700 1 -- 172.21.2.101:0/2717395802 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c30088070 con 0x7f8c300d1870 2021-02-10T11:43:17.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.877 7f8c2f7fe700 1 --2- 172.21.2.101:0/2717395802 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c300d1870 0x7f8c301b04c0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 rx=0x7f8c20006700 tx=0x7f8c20009b00).ready entity=mon.0 client_cookie=916222563df2912d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:17.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.877 7f8c2e7fc700 1 -- 172.21.2.101:0/2717395802 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c2000c0c0 con 0x7f8c300d1870 2021-02-10T11:43:17.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.877 7f8c2e7fc700 1 -- 172.21.2.101:0/2717395802 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2000c400 con 0x7f8c300d1870 2021-02-10T11:43:17.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.877 7f8c2e7fc700 1 -- 172.21.2.101:0/2717395802 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c2000c8e0 con 0x7f8c300d1870 2021-02-10T11:43:17.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.877 7f8c35f86700 1 -- 172.21.2.101:0/2717395802 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c300d1870 msgr2=0x7f8c301b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:17.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.877 7f8c35f86700 1 --2- 172.21.2.101:0/2717395802 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c300d1870 0x7f8c301b04c0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 rx=0x7f8c20006700 tx=0x7f8c20009b00).stop 2021-02-10T11:43:17.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.878 7f8c35f86700 1 -- 172.21.2.101:0/2717395802 shutdown_connections 2021-02-10T11:43:17.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.878 7f8c35f86700 1 --2- 172.21.2.101:0/2717395802 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c301b0ae0 0x7f8c30108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:17.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.878 7f8c35f86700 1 --2- 172.21.2.101:0/2717395802 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c300d1870 0x7f8c301b04c0 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:17.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.878 7f8c35f86700 1 -- 172.21.2.101:0/2717395802 >> 172.21.2.101:0/2717395802 conn(0x7f8c300ce420 msgr2=0x7f8c300d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:17.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.878 7f8c35f86700 1 -- 172.21.2.101:0/2717395802 shutdown_connections 2021-02-10T11:43:17.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.878 7f8c35f86700 1 -- 172.21.2.101:0/2717395802 wait complete. 2021-02-10T11:43:17.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.893 7f8c35f86700 1 Processor -- start 2021-02-10T11:43:17.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.893 7f8c35f86700 1 -- start start 2021-02-10T11:43:17.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.893 7f8c35f86700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c300c3820 0x7f8c30084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:17.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.893 7f8c35f86700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c300a0290 0x7f8c300852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:17.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.893 7f8c35f86700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c30086270 0x7f8c30086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:17.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.893 7f8c35f86700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c300c0700 con 0x7f8c300a0290 2021-02-10T11:43:17.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.893 7f8c35f86700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c300858f0 con 0x7f8c30086270 2021-02-10T11:43:17.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.893 7f8c35f86700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c30085b10 con 0x7f8c300c3820 2021-02-10T11:43:17.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.894 7f8c2effd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c300a0290 0x7f8c300852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:17.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.894 7f8c2effd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c300a0290 0x7f8c300852d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58820/0 (socket says 172.21.2.101:58820) 2021-02-10T11:43:17.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.894 7f8c2effd700 1 -- 172.21.2.101:0/3841127289 learned_addr learned my addr 172.21.2.101:0/3841127289 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:17.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.894 7f8c2effd700 1 -- 172.21.2.101:0/3841127289 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c300c3820 msgr2=0x7f8c30084cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:17.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.894 7f8c2effd700 1 --2- 172.21.2.101:0/3841127289 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c300c3820 0x7f8c30084cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:17.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.894 7f8c2effd700 1 -- 172.21.2.101:0/3841127289 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c30086270 msgr2=0x7f8c30086670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:17.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.894 7f8c2effd700 1 --2- 172.21.2.101:0/3841127289 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c30086270 0x7f8c30086670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:17.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.894 7f8c2effd700 1 -- 172.21.2.101:0/3841127289 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c30086c90 con 0x7f8c300a0290 2021-02-10T11:43:17.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.894 7f8c2effd700 1 --2- 172.21.2.101:0/3841127289 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c300a0290 0x7f8c300852d0 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 rx=0x7f8c1800b5c0 tx=0x7f8c1800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:17.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.894 7f8c27fff700 1 -- 172.21.2.101:0/3841127289 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c1800bd50 con 0x7f8c300a0290 2021-02-10T11:43:17.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.894 7f8c27fff700 1 -- 172.21.2.101:0/3841127289 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c18006030 con 0x7f8c300a0290 2021-02-10T11:43:17.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.894 7f8c27fff700 1 -- 172.21.2.101:0/3841127289 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c180054d0 con 0x7f8c300a0290 2021-02-10T11:43:17.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.894 7f8c35f86700 1 -- 172.21.2.101:0/3841127289 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c30086c90 con 0x7f8c300a0290 2021-02-10T11:43:17.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.895 7f8c35f86700 1 -- 172.21.2.101:0/3841127289 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c30086c90 con 0x7f8c300a0290 2021-02-10T11:43:17.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.895 7f8c27fff700 1 -- 172.21.2.101:0/3841127289 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8c18006210 con 0x7f8c300a0290 2021-02-10T11:43:17.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.895 7f8c27fff700 1 --2- 172.21.2.101:0/3841127289 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c140186c0 0x7f8c1401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:17.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.895 7f8c27fff700 1 -- 172.21.2.101:0/3841127289 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f8c18032460 con 0x7f8c300a0290 2021-02-10T11:43:17.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.895 7f8c2f7fe700 1 --2- 172.21.2.101:0/3841127289 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c140186c0 0x7f8c1401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:17.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.896 7f8c2f7fe700 1 --2- 172.21.2.101:0/3841127289 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c140186c0 0x7f8c1401abc0 crc :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:17.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.926 7f8c35f86700 1 -- 172.21.2.101:0/3841127289 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c30082a70 con 0x7f8c300a0290 2021-02-10T11:43:17.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:17.929 7f8c27fff700 1 -- 172.21.2.101:0/3841127289 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8c18006b80 con 0x7f8c300a0290 2021-02-10T11:43:18.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.057 7f8c35f86700 1 -- 172.21.2.101:0/3841127289 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"} v 0) v1 -- 0x7f8c30082a70 con 0x7f8c300a0290 2021-02-10T11:43:18.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.522 7f8c27fff700 1 -- 172.21.2.101:0/3841127289 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"}]=0 pool 'cephfs_metadata' created v134) v1 ==== 133+0+0 (secure 0 0 0) 0x7f8c18006030 con 0x7f8c300a0290 2021-02-10T11:43:18.524 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' created 2021-02-10T11:43:18.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.525 7f8c35f86700 1 -- 172.21.2.101:0/3841127289 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c140186c0 msgr2=0x7f8c1401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:18.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.525 7f8c35f86700 1 --2- 172.21.2.101:0/3841127289 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c140186c0 0x7f8c1401abc0 crc :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:18.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.525 7f8c35f86700 1 -- 172.21.2.101:0/3841127289 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c300a0290 msgr2=0x7f8c300852d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:18.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.525 7f8c35f86700 1 --2- 172.21.2.101:0/3841127289 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c300a0290 0x7f8c300852d0 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 rx=0x7f8c1800b5c0 tx=0x7f8c1800b960).stop 2021-02-10T11:43:18.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.525 7f8c35f86700 1 -- 172.21.2.101:0/3841127289 shutdown_connections 2021-02-10T11:43:18.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.525 7f8c35f86700 1 --2- 172.21.2.101:0/3841127289 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c30086270 0x7f8c30086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:18.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.525 7f8c35f86700 1 --2- 172.21.2.101:0/3841127289 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c300a0290 0x7f8c300852d0 unknown :-1 s=CLOSED pgs=2532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:18.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.525 7f8c35f86700 1 --2- 172.21.2.101:0/3841127289 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c140186c0 0x7f8c1401abc0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:18.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.525 7f8c35f86700 1 --2- 172.21.2.101:0/3841127289 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c300c3820 0x7f8c30084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:18.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.525 7f8c35f86700 1 -- 172.21.2.101:0/3841127289 >> 172.21.2.101:0/3841127289 conn(0x7f8c300872c0 msgr2=0x7f8c30095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:18.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.526 7f8c35f86700 1 -- 172.21.2.101:0/3841127289 shutdown_connections 2021-02-10T11:43:18.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.526 7f8c35f86700 1 -- 172.21.2.101:0/3841127289 wait complete. 2021-02-10T11:43:18.536 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_data 8 2021-02-10T11:43:18.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.950 7f36b122a700 1 Processor -- start 2021-02-10T11:43:18.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.950 7f36b122a700 1 -- start start 2021-02-10T11:43:18.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.951 7f36b122a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f36ac0d1c70 0x7f36ac0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:18.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.951 7f36b122a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ac0d0170 con 0x7f36ac0cf580 2021-02-10T11:43:18.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.951 7f36b122a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ac0fe1c0 con 0x7f36ac0d1870 2021-02-10T11:43:18.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.951 7f36b122a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac1b0cb0 con 0x7f36ac0d1c70 2021-02-10T11:43:18.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.951 7f36ab59e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f36ac0cf580 0x7f36ac0cf980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59326/0 (socket says 172.21.2.101:59326) 2021-02-10T11:43:18.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.951 7f36ab59e700 1 -- 172.21.2.101:0/383019247 learned_addr learned my addr 172.21.2.101:0/383019247 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:18.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.951 7f36a9d9b700 1 -- 172.21.2.101:0/383019247 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2050717480 0 0) 0x7f36a4001500 con 0x7f36ac0cf580 2021-02-10T11:43:18.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.951 7f36a9d9b700 1 -- 172.21.2.101:0/383019247 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3698002fe0 con 0x7f36ac0cf580 2021-02-10T11:43:18.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.951 7f36a9d9b700 1 -- 172.21.2.101:0/383019247 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3447120670 0 0) 0x7f36a4001460 con 0x7f36ac0cf580 2021-02-10T11:43:18.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.951 7f36a9d9b700 1 -- 172.21.2.101:0/383019247 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f36ac0d1c70 msgr2=0x7f36ac0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:18.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.951 7f36a9d9b700 1 --2- 172.21.2.101:0/383019247 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f36ac0d1c70 0x7f36ac0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:18.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.951 7f36a9d9b700 1 -- 172.21.2.101:0/383019247 >> v1:172.21.2.108:6789/0 conn(0x7f36ac0d1870 legacy=0x7f36ac1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:18.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.951 7f36a9d9b700 1 -- 172.21.2.101:0/383019247 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac109a60 con 0x7f36ac0cf580 2021-02-10T11:43:18.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.951 7f36a9d9b700 1 -- 172.21.2.101:0/383019247 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f36a4001980 con 0x7f36ac0cf580 2021-02-10T11:43:18.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.951 7f36a9d9b700 1 -- 172.21.2.101:0/383019247 >> v1:172.21.2.101:6789/0 conn(0x7f36ac0cf580 legacy=0x7f36ac0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:18.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.951 7f36a9d9b700 1 --2- 172.21.2.101:0/383019247 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3698003a60 0x7f3698005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:18.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.951 7f36a9d9b700 1 --2- 172.21.2.101:0/383019247 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3698006790 0x7f3698008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:18.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.951 7f36a9d9b700 1 --2- 172.21.2.101:0/383019247 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3698009590 0x7f369800ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:18.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.951 7f36a9d9b700 1 -- 172.21.2.101:0/383019247 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369800c2d0 con 0x7f3698006790 2021-02-10T11:43:18.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.951 7f36a9d9b700 1 -- 172.21.2.101:0/383019247 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369800c4f0 con 0x7f3698003a60 2021-02-10T11:43:18.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.951 7f36a9d9b700 1 -- 172.21.2.101:0/383019247 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369800c710 con 0x7f3698009590 2021-02-10T11:43:18.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.951 7f36aa59c700 1 --2- 172.21.2.101:0/383019247 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3698006790 0x7f3698008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:18.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.952 7f36aad9d700 1 --2- 172.21.2.101:0/383019247 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3698003a60 0x7f3698005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:18.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.952 7f36ab59e700 1 --2- 172.21.2.101:0/383019247 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3698009590 0x7f369800ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:18.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.952 7f36aa59c700 1 -- 172.21.2.101:0/383019247 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3698009590 msgr2=0x7f369800ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:18.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.952 7f36aa59c700 1 --2- 172.21.2.101:0/383019247 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3698009590 0x7f369800ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:18.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.952 7f36aa59c700 1 -- 172.21.2.101:0/383019247 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3698003a60 msgr2=0x7f3698005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:18.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.952 7f36aa59c700 1 --2- 172.21.2.101:0/383019247 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3698003a60 0x7f3698005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:18.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.952 7f36aa59c700 1 -- 172.21.2.101:0/383019247 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369800c9b0 con 0x7f3698006790 2021-02-10T11:43:18.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.952 7f36aa59c700 1 --2- 172.21.2.101:0/383019247 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3698006790 0x7f3698008c90 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 rx=0x7f3694006ab0 tx=0x7f369404fbc0).ready entity=mon.0 client_cookie=ffc17141eae39d4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:18.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.952 7f36a9d9b700 1 -- 172.21.2.101:0/383019247 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3694002080 con 0x7f3698006790 2021-02-10T11:43:18.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.952 7f36a9d9b700 1 -- 172.21.2.101:0/383019247 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36940063f0 con 0x7f3698006790 2021-02-10T11:43:18.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.952 7f36a9d9b700 1 -- 172.21.2.101:0/383019247 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3694002300 con 0x7f3698006790 2021-02-10T11:43:18.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.952 7f36b122a700 1 -- 172.21.2.101:0/383019247 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3698006790 msgr2=0x7f3698008c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:18.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.952 7f36b122a700 1 --2- 172.21.2.101:0/383019247 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3698006790 0x7f3698008c90 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 rx=0x7f3694006ab0 tx=0x7f369404fbc0).stop 2021-02-10T11:43:18.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.952 7f36ab59e700 1 -- 172.21.2.101:0/383019247 reap_dead start 2021-02-10T11:43:18.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.953 7f36b122a700 1 -- 172.21.2.101:0/383019247 shutdown_connections 2021-02-10T11:43:18.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.953 7f36b122a700 1 -- 172.21.2.101:0/383019247 >> 172.21.2.101:0/383019247 conn(0x7f36ac0ce420 msgr2=0x7f36ac0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:18.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.953 7f36b122a700 1 -- 172.21.2.101:0/383019247 shutdown_connections 2021-02-10T11:43:18.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.953 7f36b122a700 1 -- 172.21.2.101:0/383019247 wait complete. 2021-02-10T11:43:18.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.968 7f36b122a700 1 Processor -- start 2021-02-10T11:43:18.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.968 7f36b122a700 1 -- start start 2021-02-10T11:43:18.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.968 7f36b122a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f36ac09bcc0 0x7f36ac09c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:18.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.968 7f36b122a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36ac0a0820 0x7f36ac0a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:18.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.968 7f36b122a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f36ac0a1240 0x7f36ac0858d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:18.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.968 7f36b122a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac0a66b0 con 0x7f36ac0a0820 2021-02-10T11:43:18.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.968 7f36b122a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac1a49b0 con 0x7f36ac0a1240 2021-02-10T11:43:18.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.968 7f36b122a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac1a4b90 con 0x7f36ac09bcc0 2021-02-10T11:43:18.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.968 7f36aa59c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36ac0a0820 0x7f36ac0a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:18.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.968 7f36aa59c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36ac0a0820 0x7f36ac0a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58842/0 (socket says 172.21.2.101:58842) 2021-02-10T11:43:18.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.968 7f36aa59c700 1 -- 172.21.2.101:0/4252620625 learned_addr learned my addr 172.21.2.101:0/4252620625 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:18.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.969 7f36aa59c700 1 -- 172.21.2.101:0/4252620625 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f36ac09bcc0 msgr2=0x7f36ac09c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:18.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.969 7f36aa59c700 1 --2- 172.21.2.101:0/4252620625 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f36ac09bcc0 0x7f36ac09c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:18.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.969 7f36aa59c700 1 -- 172.21.2.101:0/4252620625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f36ac0a1240 msgr2=0x7f36ac0858d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:18.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.969 7f36aa59c700 1 --2- 172.21.2.101:0/4252620625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f36ac0a1240 0x7f36ac0858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:18.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.969 7f36aa59c700 1 -- 172.21.2.101:0/4252620625 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac0a1640 con 0x7f36ac0a0820 2021-02-10T11:43:18.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.969 7f36aa59c700 1 --2- 172.21.2.101:0/4252620625 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36ac0a0820 0x7f36ac0a0c20 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 rx=0x7f3694006710 tx=0x7f3694050960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:18.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.969 7f36a37fe700 1 -- 172.21.2.101:0/4252620625 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3694050d50 con 0x7f36ac0a0820 2021-02-10T11:43:18.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.969 7f36a37fe700 1 -- 172.21.2.101:0/4252620625 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369404e050 con 0x7f36ac0a0820 2021-02-10T11:43:18.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.969 7f36b122a700 1 -- 172.21.2.101:0/4252620625 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ac0a1640 con 0x7f36ac0a0820 2021-02-10T11:43:18.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.969 7f36b122a700 1 -- 172.21.2.101:0/4252620625 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ac0861e0 con 0x7f36ac0a0820 2021-02-10T11:43:18.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.969 7f36a37fe700 1 -- 172.21.2.101:0/4252620625 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f369404e570 con 0x7f36ac0a0820 2021-02-10T11:43:18.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.970 7f36a37fe700 1 -- 172.21.2.101:0/4252620625 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3694005310 con 0x7f36ac0a0820 2021-02-10T11:43:18.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.970 7f36a37fe700 1 --2- 172.21.2.101:0/4252620625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3690018730 0x7f369001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:18.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.970 7f36a37fe700 1 -- 172.21.2.101:0/4252620625 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5107+0+0 (secure 0 0 0) 0x7f3694076470 con 0x7f36ac0a0820 2021-02-10T11:43:18.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.970 7f36aad9d700 1 --2- 172.21.2.101:0/4252620625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3690018730 0x7f369001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:18.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:18.970 7f36aad9d700 1 --2- 172.21.2.101:0/4252620625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3690018730 0x7f369001ac10 crc :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:19.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.000 7f36b122a700 1 -- 172.21.2.101:0/4252620625 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ac086a70 con 0x7f36ac0a0820 2021-02-10T11:43:19.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.003 7f36a37fe700 1 -- 172.21.2.101:0/4252620625 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3694000990 con 0x7f36ac0a0820 2021-02-10T11:43:19.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.133 7f36b122a700 1 -- 172.21.2.101:0/4252620625 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"} v 0) v1 -- 0x7f36ac086a70 con 0x7f36ac0a0820 2021-02-10T11:43:19.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.540 7f36a37fe700 1 -- 172.21.2.101:0/4252620625 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"}]=0 pool 'cephfs_data' created v136) v1 ==== 125+0+0 (secure 0 0 0) 0x7f3694005310 con 0x7f36ac0a0820 2021-02-10T11:43:19.550 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' created 2021-02-10T11:43:19.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.551 7f36b122a700 1 -- 172.21.2.101:0/4252620625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3690018730 msgr2=0x7f369001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:19.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.551 7f36b122a700 1 --2- 172.21.2.101:0/4252620625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3690018730 0x7f369001ac10 crc :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:19.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.551 7f36b122a700 1 -- 172.21.2.101:0/4252620625 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36ac0a0820 msgr2=0x7f36ac0a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:19.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.551 7f36b122a700 1 --2- 172.21.2.101:0/4252620625 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36ac0a0820 0x7f36ac0a0c20 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 rx=0x7f3694006710 tx=0x7f3694050960).stop 2021-02-10T11:43:19.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.551 7f36b122a700 1 -- 172.21.2.101:0/4252620625 shutdown_connections 2021-02-10T11:43:19.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.551 7f36b122a700 1 --2- 172.21.2.101:0/4252620625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f36ac0a1240 0x7f36ac0858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:19.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.551 7f36b122a700 1 --2- 172.21.2.101:0/4252620625 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f36ac0a0820 0x7f36ac0a0c20 unknown :-1 s=CLOSED pgs=2535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:19.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.551 7f36b122a700 1 --2- 172.21.2.101:0/4252620625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3690018730 0x7f369001ac10 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:19.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.551 7f36b122a700 1 --2- 172.21.2.101:0/4252620625 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f36ac09bcc0 0x7f36ac09c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:19.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.551 7f36b122a700 1 -- 172.21.2.101:0/4252620625 >> 172.21.2.101:0/4252620625 conn(0x7f36ac0bef00 msgr2=0x7f36ac09d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:19.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.551 7f36b122a700 1 -- 172.21.2.101:0/4252620625 shutdown_connections 2021-02-10T11:43:19.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.551 7f36b122a700 1 -- 172.21.2.101:0/4252620625 wait complete. 2021-02-10T11:43:19.560 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-02-10T11:43:19.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.976 7f4c3ba0a700 1 Processor -- start 2021-02-10T11:43:19.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.976 7f4c3ba0a700 1 -- start start 2021-02-10T11:43:19.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.976 7f4c3ba0a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f4c340d1880 0x7f4c341b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:19.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.976 7f4c3ba0a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c340cff80 0x7f4c34109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:19.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.976 7f4c3ba0a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c34109a70 con 0x7f4c341b0af0 2021-02-10T11:43:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.976 7f4c3ba0a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c34109760 con 0x7f4c340d1880 2021-02-10T11:43:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.976 7f4c3ba0a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c34088070 con 0x7f4c340cff80 2021-02-10T11:43:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.976 7f4c39826700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f4c340d1880 0x7f4c341b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.976 7f4c39826700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f4c340d1880 0x7f4c341b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58850/0 (socket says 172.21.2.101:58850) 2021-02-10T11:43:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.976 7f4c39826700 1 -- 172.21.2.101:0/3555689505 learned_addr learned my addr 172.21.2.101:0/3555689505 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.977 7f4c39826700 1 -- 172.21.2.101:0/3555689505 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c340cff80 msgr2=0x7f4c34109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.977 7f4c39826700 1 --2- 172.21.2.101:0/3555689505 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c340cff80 0x7f4c34109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.977 7f4c39826700 1 -- 172.21.2.101:0/3555689505 >> v1:172.21.2.108:6789/0 conn(0x7f4c341b0af0 legacy=0x7f4c34108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.977 7f4c39826700 1 -- 172.21.2.101:0/3555689505 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c340d1c80 con 0x7f4c340d1880 2021-02-10T11:43:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.977 7f4c39826700 1 --2- 172.21.2.101:0/3555689505 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c340d1880 0x7f4c341b04d0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 rx=0x7f4c24006700 tx=0x7f4c24009b80).ready entity=mon.0 client_cookie=74698be9a91cada6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.977 7f4c38824700 1 -- 172.21.2.101:0/3555689505 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4c2400c0c0 con 0x7f4c340d1880 2021-02-10T11:43:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.977 7f4c38824700 1 -- 172.21.2.101:0/3555689505 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2400c440 con 0x7f4c340d1880 2021-02-10T11:43:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.977 7f4c38824700 1 -- 172.21.2.101:0/3555689505 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4c2400c8a0 con 0x7f4c340d1880 2021-02-10T11:43:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.977 7f4c3ba0a700 1 -- 172.21.2.101:0/3555689505 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c340d1880 msgr2=0x7f4c341b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.977 7f4c3ba0a700 1 --2- 172.21.2.101:0/3555689505 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c340d1880 0x7f4c341b04d0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 rx=0x7f4c24006700 tx=0x7f4c24009b80).stop 2021-02-10T11:43:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.978 7f4c3ba0a700 1 -- 172.21.2.101:0/3555689505 shutdown_connections 2021-02-10T11:43:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.978 7f4c3ba0a700 1 --2- 172.21.2.101:0/3555689505 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c340cff80 0x7f4c34109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.978 7f4c3ba0a700 1 --2- 172.21.2.101:0/3555689505 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c340d1880 0x7f4c341b04d0 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.978 7f4c3ba0a700 1 -- 172.21.2.101:0/3555689505 >> 172.21.2.101:0/3555689505 conn(0x7f4c340ce430 msgr2=0x7f4c340d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:19.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.978 7f4c3ba0a700 1 -- 172.21.2.101:0/3555689505 shutdown_connections 2021-02-10T11:43:19.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.978 7f4c3ba0a700 1 -- 172.21.2.101:0/3555689505 wait complete. 2021-02-10T11:43:19.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.993 7f4c3ba0a700 1 Processor -- start 2021-02-10T11:43:19.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.993 7f4c3ba0a700 1 -- start start 2021-02-10T11:43:19.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.993 7f4c3ba0a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c3409ef20 0x7f4c3418d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:19.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.993 7f4c3ba0a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c3408eae0 0x7f4c3408eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:19.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.993 7f4c3ba0a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c3408b0e0 0x7f4c3408b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:19.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.993 7f4c3ba0a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3408f500 con 0x7f4c3408b0e0 2021-02-10T11:43:19.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.993 7f4c3ba0a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3409f320 con 0x7f4c3408eae0 2021-02-10T11:43:19.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.993 7f4c3ba0a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3409f540 con 0x7f4c3409ef20 2021-02-10T11:43:19.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.993 7f4c3a027700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c3408b0e0 0x7f4c3408b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:19.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.993 7f4c3a027700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c3408b0e0 0x7f4c3408b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58854/0 (socket says 172.21.2.101:58854) 2021-02-10T11:43:19.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.993 7f4c3a027700 1 -- 172.21.2.101:0/3783406540 learned_addr learned my addr 172.21.2.101:0/3783406540 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:19.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.994 7f4c3a027700 1 -- 172.21.2.101:0/3783406540 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c3409ef20 msgr2=0x7f4c3418d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:19.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.994 7f4c3a027700 1 --2- 172.21.2.101:0/3783406540 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c3409ef20 0x7f4c3418d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:19.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.994 7f4c3a027700 1 -- 172.21.2.101:0/3783406540 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c3408eae0 msgr2=0x7f4c3408eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:19.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.994 7f4c3a027700 1 --2- 172.21.2.101:0/3783406540 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c3408eae0 0x7f4c3408eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:19.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.994 7f4c3a027700 1 -- 172.21.2.101:0/3783406540 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3408bb00 con 0x7f4c3408b0e0 2021-02-10T11:43:19.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.994 7f4c3a027700 1 --2- 172.21.2.101:0/3783406540 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c3408b0e0 0x7f4c3408b4e0 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 rx=0x7f4c2c009980 tx=0x7f4c2c00bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:19.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.994 7f4c327fc700 1 -- 172.21.2.101:0/3783406540 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4c2c005870 con 0x7f4c3408b0e0 2021-02-10T11:43:19.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.994 7f4c327fc700 1 -- 172.21.2.101:0/3783406540 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2c005af0 con 0x7f4c3408b0e0 2021-02-10T11:43:20.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.994 7f4c327fc700 1 -- 172.21.2.101:0/3783406540 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4c2c006030 con 0x7f4c3408b0e0 2021-02-10T11:43:20.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.994 7f4c3ba0a700 1 -- 172.21.2.101:0/3783406540 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3408bb00 con 0x7f4c3408b0e0 2021-02-10T11:43:20.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.994 7f4c3ba0a700 1 -- 172.21.2.101:0/3783406540 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3408bb00 con 0x7f4c3408b0e0 2021-02-10T11:43:20.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.995 7f4c327fc700 1 -- 172.21.2.101:0/3783406540 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4c2c005d30 con 0x7f4c3408b0e0 2021-02-10T11:43:20.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.995 7f4c327fc700 1 --2- 172.21.2.101:0/3783406540 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4c180186c0 0x7f4c1801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:20.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.995 7f4c327fc700 1 -- 172.21.2.101:0/3783406540 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5457+0+0 (secure 0 0 0) 0x7f4c2c0328c0 con 0x7f4c3408b0e0 2021-02-10T11:43:20.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.995 7f4c39826700 1 --2- 172.21.2.101:0/3783406540 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4c180186c0 0x7f4c1801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:20.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:19.995 7f4c39826700 1 --2- 172.21.2.101:0/3783406540 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4c180186c0 0x7f4c1801abc0 crc :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:20.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.026 7f4c3ba0a700 1 -- 172.21.2.101:0/3783406540 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c3408e5b0 con 0x7f4c3408b0e0 2021-02-10T11:43:20.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.029 7f4c327fc700 1 -- 172.21.2.101:0/3783406540 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4c2c00be10 con 0x7f4c3408b0e0 2021-02-10T11:43:20.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.157 7f4c3ba0a700 1 -- 172.21.2.101:0/3783406540 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"} v 0) v1 -- 0x7f4c3408e5b0 con 0x7f4c3408b0e0 2021-02-10T11:43:20.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.162 7f4c327fc700 1 -- 172.21.2.101:0/3783406540 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"}]=0 new fs with metadata pool 28 and data pool 29 v111) v1 ==== 189+0+0 (secure 0 0 0) 0x7f4c2c005d30 con 0x7f4c3408b0e0 2021-02-10T11:43:20.165 INFO:teuthology.orchestra.run.gibba001.stderr:new fs with metadata pool 28 and data pool 29 2021-02-10T11:43:20.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.167 7f4c3ba0a700 1 -- 172.21.2.101:0/3783406540 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4c180186c0 msgr2=0x7f4c1801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:20.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.167 7f4c3ba0a700 1 --2- 172.21.2.101:0/3783406540 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4c180186c0 0x7f4c1801abc0 crc :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:20.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.167 7f4c3ba0a700 1 -- 172.21.2.101:0/3783406540 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c3408b0e0 msgr2=0x7f4c3408b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:20.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.167 7f4c3ba0a700 1 --2- 172.21.2.101:0/3783406540 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c3408b0e0 0x7f4c3408b4e0 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 rx=0x7f4c2c009980 tx=0x7f4c2c00bcd0).stop 2021-02-10T11:43:20.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.167 7f4c3ba0a700 1 -- 172.21.2.101:0/3783406540 shutdown_connections 2021-02-10T11:43:20.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.167 7f4c3ba0a700 1 --2- 172.21.2.101:0/3783406540 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c3408b0e0 0x7f4c3408b4e0 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:20.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.167 7f4c3ba0a700 1 --2- 172.21.2.101:0/3783406540 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c3408eae0 0x7f4c3408eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:20.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.167 7f4c3ba0a700 1 --2- 172.21.2.101:0/3783406540 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4c180186c0 0x7f4c1801abc0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:20.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.167 7f4c3ba0a700 1 --2- 172.21.2.101:0/3783406540 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c3409ef20 0x7f4c3418d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:20.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.167 7f4c3ba0a700 1 -- 172.21.2.101:0/3783406540 >> 172.21.2.101:0/3783406540 conn(0x7f4c3419f970 msgr2=0x7f4c341a61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:20.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.167 7f4c3ba0a700 1 -- 172.21.2.101:0/3783406540 shutdown_connections 2021-02-10T11:43:20.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.167 7f4c3ba0a700 1 -- 172.21.2.101:0/3783406540 wait complete. 2021-02-10T11:43:20.176 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:43:20.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.586 7f9ad4258700 1 Processor -- start 2021-02-10T11:43:20.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.586 7f9ad4258700 1 -- start start 2021-02-10T11:43:20.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.587 7f9ad4258700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9acc0d1c70 0x7f9acc0f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:20.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.587 7f9ad4258700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9acc0f0b10 con 0x7f9acc0d1870 2021-02-10T11:43:20.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.587 7f9ad4258700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9acc1b0be0 con 0x7f9acc0d0340 2021-02-10T11:43:20.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.587 7f9ad4258700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9acc0e5ed0 con 0x7f9acc0d1c70 2021-02-10T11:43:20.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.587 7f9ad2875700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9acc0d1c70 0x7f9acc0f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:20.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.587 7f9ad2074700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f9acc0d1870 0x7f9acc1b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59354/0 (socket says 172.21.2.101:59354) 2021-02-10T11:43:20.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.587 7f9ad2875700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9acc0d1c70 0x7f9acc0f20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58864/0 (socket says 172.21.2.101:58864) 2021-02-10T11:43:20.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.587 7f9ad2074700 1 -- 172.21.2.101:0/1021830579 learned_addr learned my addr 172.21.2.101:0/1021830579 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:20.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.587 7f9ad1072700 1 -- 172.21.2.101:0/1021830579 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737473090 0 0) 0x7f9ac8001180 con 0x7f9acc0d1870 2021-02-10T11:43:20.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.588 7f9ad1072700 1 -- 172.21.2.101:0/1021830579 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ab8002fe0 con 0x7f9acc0d1870 2021-02-10T11:43:20.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.588 7f9ad1072700 1 -- 172.21.2.101:0/1021830579 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243910379 0 0) 0x7f9abc000c60 con 0x7f9acc0d0340 2021-02-10T11:43:20.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.588 7f9ad2875700 1 -- 172.21.2.101:0/1021830579 >> v1:172.21.2.108:6789/0 conn(0x7f9acc0d0340 legacy=0x7f9acc0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:20.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.588 7f9ad2875700 1 -- 172.21.2.101:0/1021830579 >> v1:172.21.2.101:6789/0 conn(0x7f9acc0d1870 legacy=0x7f9acc1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:20.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.588 7f9ad2875700 1 -- 172.21.2.101:0/1021830579 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc0efe30 con 0x7f9acc0d1c70 2021-02-10T11:43:20.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.588 7f9ad2875700 1 --2- 172.21.2.101:0/1021830579 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9acc0d1c70 0x7f9acc0f20a0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 rx=0x7f9ac4006e30 tx=0x7f9ac4009e60).ready entity=mon.0 client_cookie=65bd882ee57ad98b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:20.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.588 7f9ad1072700 1 -- 172.21.2.101:0/1021830579 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9ac4006a50 con 0x7f9acc0d1c70 2021-02-10T11:43:20.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.588 7f9ad1072700 1 -- 172.21.2.101:0/1021830579 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac400b300 con 0x7f9acc0d1c70 2021-02-10T11:43:20.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.588 7f9ad1072700 1 -- 172.21.2.101:0/1021830579 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9ac400b7e0 con 0x7f9acc0d1c70 2021-02-10T11:43:20.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.588 7f9ad4258700 1 -- 172.21.2.101:0/1021830579 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9acc0d1c70 msgr2=0x7f9acc0f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:20.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.588 7f9ad4258700 1 --2- 172.21.2.101:0/1021830579 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9acc0d1c70 0x7f9acc0f20a0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 rx=0x7f9ac4006e30 tx=0x7f9ac4009e60).stop 2021-02-10T11:43:20.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.589 7f9ad4258700 1 -- 172.21.2.101:0/1021830579 shutdown_connections 2021-02-10T11:43:20.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.589 7f9ad4258700 1 --2- 172.21.2.101:0/1021830579 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9acc0d1c70 0x7f9acc0f20a0 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:20.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.589 7f9ad4258700 1 -- 172.21.2.101:0/1021830579 >> 172.21.2.101:0/1021830579 conn(0x7f9acc0ce450 msgr2=0x7f9acc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:20.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.589 7f9ad4258700 1 -- 172.21.2.101:0/1021830579 shutdown_connections 2021-02-10T11:43:20.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.589 7f9ad4258700 1 -- 172.21.2.101:0/1021830579 wait complete. 2021-02-10T11:43:20.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.604 7f9ad4258700 1 Processor -- start 2021-02-10T11:43:20.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.604 7f9ad4258700 1 -- start start 2021-02-10T11:43:20.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.604 7f9ad4258700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9acc195650 0x7f9acc0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:20.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.604 7f9ad4258700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9acc0b5b80 0x7f9acc1a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:20.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.604 7f9ad4258700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9acc0acfa0 0x7f9acc1a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:20.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.604 7f9ad4258700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0b6010 con 0x7f9acc0b5b80 2021-02-10T11:43:20.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.604 7f9ad4258700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc1a3a60 con 0x7f9acc195650 2021-02-10T11:43:20.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.604 7f9ad4258700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc1a3c40 con 0x7f9acc0acfa0 2021-02-10T11:43:20.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.604 7f9ad1873700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9acc0b5b80 0x7f9acc1a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:20.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.604 7f9ad2074700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9acc195650 0x7f9acc0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:20.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.604 7f9ad1873700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9acc0b5b80 0x7f9acc1a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58866/0 (socket says 172.21.2.101:58866) 2021-02-10T11:43:20.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.604 7f9ad1873700 1 -- 172.21.2.101:0/2401803775 learned_addr learned my addr 172.21.2.101:0/2401803775 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:20.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.605 7f9ad2875700 1 --2- 172.21.2.101:0/2401803775 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9acc0acfa0 0x7f9acc1a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:20.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.605 7f9ad1873700 1 -- 172.21.2.101:0/2401803775 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9acc0acfa0 msgr2=0x7f9acc1a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:20.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.605 7f9ad1873700 1 --2- 172.21.2.101:0/2401803775 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9acc0acfa0 0x7f9acc1a3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:20.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.605 7f9ad1873700 1 -- 172.21.2.101:0/2401803775 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9acc195650 msgr2=0x7f9acc0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:20.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.605 7f9ad1873700 1 --2- 172.21.2.101:0/2401803775 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9acc195650 0x7f9acc0b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:20.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.605 7f9ad1873700 1 -- 172.21.2.101:0/2401803775 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc0bcae0 con 0x7f9acc0b5b80 2021-02-10T11:43:20.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.605 7f9ad2875700 1 --2- 172.21.2.101:0/2401803775 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9acc0acfa0 0x7f9acc1a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:43:20.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.605 7f9ad1873700 1 --2- 172.21.2.101:0/2401803775 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9acc0b5b80 0x7f9acc1a2e20 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 rx=0x7f9abc006f30 tx=0x7f9abc00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:20.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.605 7f9ac2ffd700 1 -- 172.21.2.101:0/2401803775 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9abc00bd80 con 0x7f9acc0b5b80 2021-02-10T11:43:20.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.605 7f9ac2ffd700 1 -- 172.21.2.101:0/2401803775 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9abc000e20 con 0x7f9acc0b5b80 2021-02-10T11:43:20.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.605 7f9ad4258700 1 -- 172.21.2.101:0/2401803775 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9acc0bcae0 con 0x7f9acc0b5b80 2021-02-10T11:43:20.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.605 7f9ad4258700 1 -- 172.21.2.101:0/2401803775 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc0bd070 con 0x7f9acc0b5b80 2021-02-10T11:43:20.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.605 7f9ac2ffd700 1 -- 172.21.2.101:0/2401803775 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9abc005540 con 0x7f9acc0b5b80 2021-02-10T11:43:20.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.606 7f9ac2ffd700 1 -- 172.21.2.101:0/2401803775 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9abc006030 con 0x7f9acc0b5b80 2021-02-10T11:43:20.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.606 7f9ac2ffd700 1 --2- 172.21.2.101:0/2401803775 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ab4018730 0x7f9ab401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:20.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.606 7f9ac2ffd700 1 -- 172.21.2.101:0/2401803775 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9abc032640 con 0x7f9acc0b5b80 2021-02-10T11:43:20.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.606 7f9ad2074700 1 --2- 172.21.2.101:0/2401803775 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ab4018730 0x7f9ab401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:20.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.606 7f9ad2074700 1 --2- 172.21.2.101:0/2401803775 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ab4018730 0x7f9ab401ac10 crc :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:20.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.637 7f9ad4258700 1 -- 172.21.2.101:0/2401803775 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9acc0f0ab0 con 0x7f9acc0b5b80 2021-02-10T11:43:20.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.639 7f9ac2ffd700 1 -- 172.21.2.101:0/2401803775 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9abc000bc0 con 0x7f9acc0b5b80 2021-02-10T11:43:20.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.766 7f9ad4258700 1 -- 172.21.2.101:0/2401803775 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9acc0f0ab0 con 0x7f9acc0b5b80 2021-02-10T11:43:20.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.767 7f9ac2ffd700 1 -- 172.21.2.101:0/2401803775 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+14020 (secure 0 0 0) 0x7f9abc006030 con 0x7f9acc0b5b80 2021-02-10T11:43:20.767 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:43:20.768 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":137,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:43:20.159044","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:43:18.076350","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2021-02-10 11:43:19.152025","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:43:20.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.768 7f9ad4258700 1 -- 172.21.2.101:0/2401803775 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ab4018730 msgr2=0x7f9ab401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:20.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.768 7f9ad4258700 1 --2- 172.21.2.101:0/2401803775 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ab4018730 0x7f9ab401ac10 crc :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:20.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.768 7f9ad4258700 1 -- 172.21.2.101:0/2401803775 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9acc0b5b80 msgr2=0x7f9acc1a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:20.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.768 7f9ad4258700 1 --2- 172.21.2.101:0/2401803775 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9acc0b5b80 0x7f9acc1a2e20 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 rx=0x7f9abc006f30 tx=0x7f9abc00b990).stop 2021-02-10T11:43:20.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.768 7f9ad4258700 1 -- 172.21.2.101:0/2401803775 shutdown_connections 2021-02-10T11:43:20.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.768 7f9ad4258700 1 --2- 172.21.2.101:0/2401803775 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ab4018730 0x7f9ab401ac10 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:20.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.768 7f9ad4258700 1 --2- 172.21.2.101:0/2401803775 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9acc0acfa0 0x7f9acc1a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:20.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.768 7f9ad4258700 1 --2- 172.21.2.101:0/2401803775 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9acc0b5b80 0x7f9acc1a2e20 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:20.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.768 7f9ad4258700 1 --2- 172.21.2.101:0/2401803775 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9acc195650 0x7f9acc0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:20.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.769 7f9ad4258700 1 -- 172.21.2.101:0/2401803775 >> 172.21.2.101:0/2401803775 conn(0x7f9acc0b3040 msgr2=0x7f9acc0c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:20.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.769 7f9ad4258700 1 -- 172.21.2.101:0/2401803775 shutdown_connections 2021-02-10T11:43:20.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:20.769 7f9ad4258700 1 -- 172.21.2.101:0/2401803775 wait complete. 2021-02-10T11:43:20.778 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:43:21.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.188 7f94f4a7f700 1 Processor -- start 2021-02-10T11:43:21.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.188 7f94f4a7f700 1 -- start start 2021-02-10T11:43:21.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.189 7f94f4a7f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94f00d1870 0x7f94f01b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:21.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.189 7f94f4a7f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94f00f0b10 con 0x7f94f00f0650 2021-02-10T11:43:21.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.189 7f94f4a7f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94f00d0310 con 0x7f94f01b0a10 2021-02-10T11:43:21.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.189 7f94f4a7f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f00e5ed0 con 0x7f94f00d1870 2021-02-10T11:43:21.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.189 7f94ef05f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f94f00f0650 0x7f94f00cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59372/0 (socket says 172.21.2.101:59372) 2021-02-10T11:43:21.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.189 7f94ef05f700 1 -- 172.21.2.101:0/3697115087 learned_addr learned my addr 172.21.2.101:0/3697115087 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:21.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.189 7f94ee85e700 1 --2- 172.21.2.101:0/3697115087 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94f00d1870 0x7f94f01b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:21.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.189 7f94ed85c700 1 -- 172.21.2.101:0/3697115087 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440953134 0 0) 0x7f94e8000f90 con 0x7f94f00f0650 2021-02-10T11:43:21.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.189 7f94ed85c700 1 -- 172.21.2.101:0/3697115087 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94dc002fe0 con 0x7f94f00f0650 2021-02-10T11:43:21.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.189 7f94ed85c700 1 -- 172.21.2.101:0/3697115087 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1039826123 0 0) 0x7f94e0001080 con 0x7f94f01b0a10 2021-02-10T11:43:21.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.189 7f94ed85c700 1 -- 172.21.2.101:0/3697115087 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94dc004370 con 0x7f94f01b0a10 2021-02-10T11:43:21.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.189 7f94ee85e700 1 -- 172.21.2.101:0/3697115087 >> v1:172.21.2.108:6789/0 conn(0x7f94f01b0a10 legacy=0x7f94f00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:21.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.189 7f94ee85e700 1 -- 172.21.2.101:0/3697115087 >> v1:172.21.2.101:6789/0 conn(0x7f94f00f0650 legacy=0x7f94f00cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:21.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.190 7f94ee85e700 1 -- 172.21.2.101:0/3697115087 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f00d0530 con 0x7f94f00d1870 2021-02-10T11:43:21.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.190 7f94ee85e700 1 --2- 172.21.2.101:0/3697115087 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94f00d1870 0x7f94f01b03f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f94d8006650 tx=0x7f94d8009a80).ready entity=mon.2 client_cookie=f18ddfeb6c9c067c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:21.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.190 7f94ed85c700 1 -- 172.21.2.101:0/3697115087 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f94d800c0c0 con 0x7f94f00d1870 2021-02-10T11:43:21.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.190 7f94ed85c700 1 -- 172.21.2.101:0/3697115087 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d800c340 con 0x7f94f00d1870 2021-02-10T11:43:21.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.190 7f94ed85c700 1 -- 172.21.2.101:0/3697115087 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f94d800c7a0 con 0x7f94f00d1870 2021-02-10T11:43:21.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.190 7f94f4a7f700 1 -- 172.21.2.101:0/3697115087 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94f00d1870 msgr2=0x7f94f01b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:21.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.190 7f94f4a7f700 1 --2- 172.21.2.101:0/3697115087 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94f00d1870 0x7f94f01b03f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f94d8006650 tx=0x7f94d8009a80).stop 2021-02-10T11:43:21.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.191 7f94f4a7f700 1 -- 172.21.2.101:0/3697115087 shutdown_connections 2021-02-10T11:43:21.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.191 7f94f4a7f700 1 --2- 172.21.2.101:0/3697115087 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94f00d1870 0x7f94f01b03f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:21.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.191 7f94f4a7f700 1 -- 172.21.2.101:0/3697115087 >> 172.21.2.101:0/3697115087 conn(0x7f94f00ce450 msgr2=0x7f94f00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:21.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.191 7f94f4a7f700 1 -- 172.21.2.101:0/3697115087 shutdown_connections 2021-02-10T11:43:21.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.191 7f94f4a7f700 1 -- 172.21.2.101:0/3697115087 wait complete. 2021-02-10T11:43:21.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.206 7f94f4a7f700 1 Processor -- start 2021-02-10T11:43:21.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.206 7f94f4a7f700 1 -- start start 2021-02-10T11:43:21.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.206 7f94f4a7f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94f018d870 0x7f94f00b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:21.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.206 7f94f4a7f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94f00b1fd0 0x7f94f0099ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:21.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.206 7f94f4a7f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94f00b23d0 0x7f94f009a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.206 7f94f4a7f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f018dc70 con 0x7f94f00b23d0 2021-02-10T11:43:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.206 7f94f4a7f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f0199e60 con 0x7f94f018d870 2021-02-10T11:43:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.206 7f94f4a7f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f009ab30 con 0x7f94f00b1fd0 2021-02-10T11:43:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.206 7f94ef05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94f00b23d0 0x7f94f009a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.206 7f94ef05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94f00b23d0 0x7f94f009a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58884/0 (socket says 172.21.2.101:58884) 2021-02-10T11:43:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.206 7f94ef05f700 1 -- 172.21.2.101:0/2658450195 learned_addr learned my addr 172.21.2.101:0/2658450195 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.206 7f94ee85e700 1 --2- 172.21.2.101:0/2658450195 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94f018d870 0x7f94f00b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.206 7f94ef05f700 1 -- 172.21.2.101:0/2658450195 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94f00b1fd0 msgr2=0x7f94f0099ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.207 7f94ef05f700 1 --2- 172.21.2.101:0/2658450195 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94f00b1fd0 0x7f94f0099ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.207 7f94ef05f700 1 -- 172.21.2.101:0/2658450195 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94f018d870 msgr2=0x7f94f00b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.207 7f94ef05f700 1 --2- 172.21.2.101:0/2658450195 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94f018d870 0x7f94f00b19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.207 7f94ef05f700 1 -- 172.21.2.101:0/2658450195 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f009ad50 con 0x7f94f00b23d0 2021-02-10T11:43:21.210 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.207 7f94ef05f700 1 --2- 172.21.2.101:0/2658450195 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94f00b23d0 0x7f94f009a510 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 rx=0x7f94e8001c10 tx=0x7f94e800aca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.207 7f94e77fe700 1 -- 172.21.2.101:0/2658450195 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f94e8007ce0 con 0x7f94f00b23d0 2021-02-10T11:43:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.207 7f94e77fe700 1 -- 172.21.2.101:0/2658450195 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e80070d0 con 0x7f94f00b23d0 2021-02-10T11:43:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.207 7f94f4a7f700 1 -- 172.21.2.101:0/2658450195 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f009ad50 con 0x7f94f00b23d0 2021-02-10T11:43:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.207 7f94f4a7f700 1 -- 172.21.2.101:0/2658450195 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f009ad50 con 0x7f94f00b23d0 2021-02-10T11:43:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.207 7f94e77fe700 1 -- 172.21.2.101:0/2658450195 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f94e8007530 con 0x7f94f00b23d0 2021-02-10T11:43:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.207 7f94e77fe700 1 -- 172.21.2.101:0/2658450195 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f94e8000ea0 con 0x7f94f00b23d0 2021-02-10T11:43:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.207 7f94e77fe700 1 --2- 172.21.2.101:0/2658450195 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94d40186d0 0x7f94d401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.207 7f94e77fe700 1 -- 172.21.2.101:0/2658450195 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f94e8032b50 con 0x7f94f00b23d0 2021-02-10T11:43:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.207 7f94ee85e700 1 --2- 172.21.2.101:0/2658450195 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94d40186d0 0x7f94d401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:21.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.208 7f94ee85e700 1 --2- 172.21.2.101:0/2658450195 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94d40186d0 0x7f94d401abd0 crc :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:21.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.239 7f94f4a7f700 1 -- 172.21.2.101:0/2658450195 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f00999c0 con 0x7f94f00b23d0 2021-02-10T11:43:21.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.241 7f94e77fe700 1 -- 172.21.2.101:0/2658450195 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f94e8007ce0 con 0x7f94f00b23d0 2021-02-10T11:43:21.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.369 7f94f4a7f700 1 -- 172.21.2.101:0/2658450195 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94f00999c0 con 0x7f94f00b23d0 2021-02-10T11:43:21.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.370 7f94e77fe700 1 -- 172.21.2.101:0/2658450195 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f94e80070d0 con 0x7f94f00b23d0 2021-02-10T11:43:21.371 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:43:21.371 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":138,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:43:21.161685","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:43:18.076350","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2021-02-10 11:43:19.152025","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:43:21.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.372 7f94f4a7f700 1 -- 172.21.2.101:0/2658450195 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94d40186d0 msgr2=0x7f94d401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:21.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.372 7f94f4a7f700 1 --2- 172.21.2.101:0/2658450195 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94d40186d0 0x7f94d401abd0 crc :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:21.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.372 7f94f4a7f700 1 -- 172.21.2.101:0/2658450195 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94f00b23d0 msgr2=0x7f94f009a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:21.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.372 7f94f4a7f700 1 --2- 172.21.2.101:0/2658450195 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94f00b23d0 0x7f94f009a510 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 rx=0x7f94e8001c10 tx=0x7f94e800aca0).stop 2021-02-10T11:43:21.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.372 7f94f4a7f700 1 -- 172.21.2.101:0/2658450195 shutdown_connections 2021-02-10T11:43:21.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.372 7f94f4a7f700 1 --2- 172.21.2.101:0/2658450195 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94d40186d0 0x7f94d401abd0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:21.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.372 7f94f4a7f700 1 --2- 172.21.2.101:0/2658450195 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94f00b1fd0 0x7f94f0099ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:21.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.372 7f94f4a7f700 1 --2- 172.21.2.101:0/2658450195 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94f00b23d0 0x7f94f009a510 unknown :-1 s=CLOSED pgs=2542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:21.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.372 7f94f4a7f700 1 --2- 172.21.2.101:0/2658450195 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94f018d870 0x7f94f00b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:21.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.372 7f94f4a7f700 1 -- 172.21.2.101:0/2658450195 >> 172.21.2.101:0/2658450195 conn(0x7f94f00f0550 msgr2=0x7f94f0191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:21.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.372 7f94f4a7f700 1 -- 172.21.2.101:0/2658450195 shutdown_connections 2021-02-10T11:43:21.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.372 7f94f4a7f700 1 -- 172.21.2.101:0/2658450195 wait complete. 2021-02-10T11:43:21.381 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-02-10T11:43:21.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.796 7fa2ddb32700 1 Processor -- start 2021-02-10T11:43:21.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.796 7fa2ddb32700 1 -- start start 2021-02-10T11:43:21.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.796 7fa2ddb32700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2d810aff0 0x7fa2d80cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:21.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.796 7fa2ddb32700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d80fe1c0 con 0x7fa2d80d1c70 2021-02-10T11:43:21.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.796 7fa2ddb32700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d81b0cb0 con 0x7fa2d80d1870 2021-02-10T11:43:21.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.796 7fa2ddb32700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d8109a60 con 0x7fa2d810aff0 2021-02-10T11:43:21.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.796 7fa2d6ffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa2d80d1c70 0x7fa2d80cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59382/0 (socket says 172.21.2.101:59382) 2021-02-10T11:43:21.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.796 7fa2d6ffd700 1 -- 172.21.2.101:0/2766879256 learned_addr learned my addr 172.21.2.101:0/2766879256 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:21.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.797 7fa2d67fc700 1 -- 172.21.2.101:0/2766879256 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 209001779 0 0) 0x7fa2cc0009b0 con 0x7fa2d80d1c70 2021-02-10T11:43:21.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.797 7fa2d67fc700 1 -- 172.21.2.101:0/2766879256 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2c0002fe0 con 0x7fa2d80d1c70 2021-02-10T11:43:21.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.797 7fa2d7fff700 1 --2- 172.21.2.101:0/2766879256 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2d810aff0 0x7fa2d80cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:21.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.797 7fa2d67fc700 1 -- 172.21.2.101:0/2766879256 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3950163486 0 0) 0x7fa2cc0010b0 con 0x7fa2d80d1c70 2021-02-10T11:43:21.815 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.797 7fa2d67fc700 1 -- 172.21.2.101:0/2766879256 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2d810aff0 msgr2=0x7fa2d80cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:21.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.797 7fa2d67fc700 1 --2- 172.21.2.101:0/2766879256 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2d810aff0 0x7fa2d80cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:21.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.797 7fa2d67fc700 1 -- 172.21.2.101:0/2766879256 >> v1:172.21.2.108:6789/0 conn(0x7fa2d80d1870 legacy=0x7fa2d81b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:21.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.797 7fa2d67fc700 1 -- 172.21.2.101:0/2766879256 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d810b3f0 con 0x7fa2d80d1c70 2021-02-10T11:43:21.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.797 7fa2d67fc700 1 -- 172.21.2.101:0/2766879256 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fa2cc0015b0 con 0x7fa2d80d1c70 2021-02-10T11:43:21.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.797 7fa2d67fc700 1 -- 172.21.2.101:0/2766879256 >> v1:172.21.2.101:6789/0 conn(0x7fa2d80d1c70 legacy=0x7fa2d80cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:21.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.797 7fa2d67fc700 1 --2- 172.21.2.101:0/2766879256 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2c0003a40 0x7fa2c0005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:21.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.797 7fa2d67fc700 1 --2- 172.21.2.101:0/2766879256 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2c0006800 0x7fa2c0008d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:21.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.797 7fa2d67fc700 1 --2- 172.21.2.101:0/2766879256 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2c0009620 0x7fa2c000ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:21.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.797 7fa2d67fc700 1 -- 172.21.2.101:0/2766879256 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c000c2e0 con 0x7fa2c0003a40 2021-02-10T11:43:21.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.797 7fa2d67fc700 1 -- 172.21.2.101:0/2766879256 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c000c500 con 0x7fa2c0006800 2021-02-10T11:43:21.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.797 7fa2d67fc700 1 -- 172.21.2.101:0/2766879256 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c000c720 con 0x7fa2c0009620 2021-02-10T11:43:21.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.797 7fa2d77fe700 1 --2- 172.21.2.101:0/2766879256 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2c0003a40 0x7fa2c0005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:21.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.797 7fa2d6ffd700 1 --2- 172.21.2.101:0/2766879256 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2c0006800 0x7fa2c0008d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.797 7fa2d7fff700 1 --2- 172.21.2.101:0/2766879256 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2c0009620 0x7fa2c000ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.798 7fa2d77fe700 1 -- 172.21.2.101:0/2766879256 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2c0009620 msgr2=0x7fa2c000ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.798 7fa2d77fe700 1 --2- 172.21.2.101:0/2766879256 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2c0009620 0x7fa2c000ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.798 7fa2d77fe700 1 -- 172.21.2.101:0/2766879256 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2c0006800 msgr2=0x7fa2c0008d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.798 7fa2d77fe700 1 --2- 172.21.2.101:0/2766879256 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2c0006800 0x7fa2c0008d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.798 7fa2d77fe700 1 -- 172.21.2.101:0/2766879256 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c000c9c0 con 0x7fa2c0003a40 2021-02-10T11:43:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.798 7fa2d77fe700 1 --2- 172.21.2.101:0/2766879256 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2c0003a40 0x7fa2c0005f60 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 rx=0x7fa2c804aad0 tx=0x7fa2c804da50).ready entity=mon.0 client_cookie=4d75d84131bb1bd7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.798 7fa2d67fc700 1 -- 172.21.2.101:0/2766879256 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa2c804b280 con 0x7fa2c0003a40 2021-02-10T11:43:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.798 7fa2d67fc700 1 -- 172.21.2.101:0/2766879256 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c804b540 con 0x7fa2c0003a40 2021-02-10T11:43:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.798 7fa2d67fc700 1 -- 172.21.2.101:0/2766879256 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa2c804ba60 con 0x7fa2c0003a40 2021-02-10T11:43:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.798 7fa2ddb32700 1 -- 172.21.2.101:0/2766879256 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2c0003a40 msgr2=0x7fa2c0005f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.798 7fa2ddb32700 1 --2- 172.21.2.101:0/2766879256 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2c0003a40 0x7fa2c0005f60 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 rx=0x7fa2c804aad0 tx=0x7fa2c804da50).stop 2021-02-10T11:43:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.798 7fa2d7fff700 1 -- 172.21.2.101:0/2766879256 reap_dead start 2021-02-10T11:43:21.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.799 7fa2ddb32700 1 -- 172.21.2.101:0/2766879256 shutdown_connections 2021-02-10T11:43:21.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.799 7fa2ddb32700 1 -- 172.21.2.101:0/2766879256 >> 172.21.2.101:0/2766879256 conn(0x7fa2d80ce420 msgr2=0x7fa2d80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:21.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.799 7fa2ddb32700 1 -- 172.21.2.101:0/2766879256 shutdown_connections 2021-02-10T11:43:21.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.799 7fa2ddb32700 1 -- 172.21.2.101:0/2766879256 wait complete. 2021-02-10T11:43:21.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.814 7fa2ddb32700 1 Processor -- start 2021-02-10T11:43:21.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.814 7fa2ddb32700 1 -- start start 2021-02-10T11:43:21.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.814 7fa2ddb32700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2d8085bc0 0x7fa2d80863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:21.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.814 7fa2ddb32700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2d80954c0 0x7fa2d8096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:21.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.814 7fa2ddb32700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2d80958c0 0x7fa2d8096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:21.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.814 7fa2ddb32700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d8095d80 con 0x7fa2d80958c0 2021-02-10T11:43:21.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.814 7fa2ddb32700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d80bfa60 con 0x7fa2d80954c0 2021-02-10T11:43:21.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.814 7fa2ddb32700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d8085fc0 con 0x7fa2d8085bc0 2021-02-10T11:43:21.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.815 7fa2d7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2d80958c0 0x7fa2d8096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:21.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.815 7fa2d7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2d80958c0 0x7fa2d8096920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58900/0 (socket says 172.21.2.101:58900) 2021-02-10T11:43:21.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.815 7fa2d7fff700 1 -- 172.21.2.101:0/2466558069 learned_addr learned my addr 172.21.2.101:0/2466558069 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:21.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.815 7fa2d77fe700 1 --2- 172.21.2.101:0/2466558069 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2d8085bc0 0x7fa2d80863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:21.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.815 7fa2d6ffd700 1 --2- 172.21.2.101:0/2466558069 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2d80954c0 0x7fa2d8096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:21.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.815 7fa2d7fff700 1 -- 172.21.2.101:0/2466558069 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2d8085bc0 msgr2=0x7fa2d80863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:21.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.815 7fa2d7fff700 1 --2- 172.21.2.101:0/2466558069 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2d8085bc0 0x7fa2d80863e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:21.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.815 7fa2d7fff700 1 -- 172.21.2.101:0/2466558069 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2d80954c0 msgr2=0x7fa2d8096300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:21.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.815 7fa2d7fff700 1 --2- 172.21.2.101:0/2466558069 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2d80954c0 0x7fa2d8096300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:21.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.815 7fa2d7fff700 1 -- 172.21.2.101:0/2466558069 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d8086c10 con 0x7fa2d80958c0 2021-02-10T11:43:21.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.815 7fa2d6ffd700 1 --2- 172.21.2.101:0/2466558069 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2d80954c0 0x7fa2d8096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:43:21.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.815 7fa2d7fff700 1 --2- 172.21.2.101:0/2466558069 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2d80958c0 0x7fa2d8096920 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 rx=0x7fa2d000b970 tx=0x7fa2d000bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:21.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.815 7fa2bffff700 1 -- 172.21.2.101:0/2466558069 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa2d000c2f0 con 0x7fa2d80958c0 2021-02-10T11:43:21.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.815 7fa2bffff700 1 -- 172.21.2.101:0/2466558069 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d000c530 con 0x7fa2d80958c0 2021-02-10T11:43:21.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.815 7fa2ddb32700 1 -- 172.21.2.101:0/2466558069 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d8086c10 con 0x7fa2d80958c0 2021-02-10T11:43:21.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.815 7fa2bffff700 1 -- 172.21.2.101:0/2466558069 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa2d000ca50 con 0x7fa2d80958c0 2021-02-10T11:43:21.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.816 7fa2ddb32700 1 -- 172.21.2.101:0/2466558069 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d8097230 con 0x7fa2d80958c0 2021-02-10T11:43:21.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.816 7fa2bffff700 1 -- 172.21.2.101:0/2466558069 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa2d000c710 con 0x7fa2d80958c0 2021-02-10T11:43:21.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.816 7fa2bffff700 1 --2- 172.21.2.101:0/2466558069 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2b80186c0 0x7fa2b801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:21.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.816 7fa2bffff700 1 -- 172.21.2.101:0/2466558069 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa2d0032790 con 0x7fa2d80958c0 2021-02-10T11:43:21.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.816 7fa2d77fe700 1 --2- 172.21.2.101:0/2466558069 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2b80186c0 0x7fa2b801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:21.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.816 7fa2d77fe700 1 --2- 172.21.2.101:0/2466558069 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2b80186c0 0x7fa2b801abc0 crc :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:21.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.846 7fa2ddb32700 1 -- 172.21.2.101:0/2466558069 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d80fe1c0 con 0x7fa2d80958c0 2021-02-10T11:43:21.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.849 7fa2bffff700 1 -- 172.21.2.101:0/2466558069 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa2d001f870 con 0x7fa2d80958c0 2021-02-10T11:43:21.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:21.984 7fa2ddb32700 1 -- 172.21.2.101:0/2466558069 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"} v 0) v1 -- 0x7fa2d80fe1c0 con 0x7fa2d80958c0 2021-02-10T11:43:22.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.171 7fa2bffff700 1 -- 172.21.2.101:0/2466558069 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"}]=0 v114) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa2d000c710 con 0x7fa2d80958c0 2021-02-10T11:43:22.202 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.201 7fa2ddb32700 1 -- 172.21.2.101:0/2466558069 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2b80186c0 msgr2=0x7fa2b801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:22.202 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.201 7fa2ddb32700 1 --2- 172.21.2.101:0/2466558069 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2b80186c0 0x7fa2b801abc0 crc :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:22.202 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.201 7fa2ddb32700 1 -- 172.21.2.101:0/2466558069 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2d80958c0 msgr2=0x7fa2d8096920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:22.202 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.201 7fa2ddb32700 1 --2- 172.21.2.101:0/2466558069 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2d80958c0 0x7fa2d8096920 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 rx=0x7fa2d000b970 tx=0x7fa2d000bd10).stop 2021-02-10T11:43:22.202 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.202 7fa2ddb32700 1 -- 172.21.2.101:0/2466558069 shutdown_connections 2021-02-10T11:43:22.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.202 7fa2ddb32700 1 --2- 172.21.2.101:0/2466558069 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2d80958c0 0x7fa2d8096920 unknown :-1 s=CLOSED pgs=2545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:22.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.202 7fa2ddb32700 1 --2- 172.21.2.101:0/2466558069 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2d80954c0 0x7fa2d8096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:22.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.202 7fa2ddb32700 1 --2- 172.21.2.101:0/2466558069 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2b80186c0 0x7fa2b801abc0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:22.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.202 7fa2ddb32700 1 --2- 172.21.2.101:0/2466558069 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2d8085bc0 0x7fa2d80863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:22.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.202 7fa2ddb32700 1 -- 172.21.2.101:0/2466558069 >> 172.21.2.101:0/2466558069 conn(0x7fa2d80bef00 msgr2=0x7fa2d809d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:22.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.202 7fa2ddb32700 1 -- 172.21.2.101:0/2466558069 shutdown_connections 2021-02-10T11:43:22.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.202 7fa2ddb32700 1 -- 172.21.2.101:0/2466558069 wait complete. 2021-02-10T11:43:22.211 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:43:22.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.623 7efc99f26700 1 Processor -- start 2021-02-10T11:43:22.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.623 7efc99f26700 1 -- start start 2021-02-10T11:43:22.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.623 7efc99f26700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7efc941b0470 0x7efc941b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:22.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.623 7efc99f26700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7efc940f20e0 0x7efc940f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:22.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.623 7efc99f26700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc940f0b50 con 0x7efc940ef610 2021-02-10T11:43:22.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.623 7efc99f26700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc940f0820 con 0x7efc941b0470 2021-02-10T11:43:22.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.623 7efc99f26700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc940f24e0 con 0x7efc940f20e0 2021-02-10T11:43:22.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.623 7efc937fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7efc941b0470 0x7efc941b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:22.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.623 7efc92ffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7efc940ef610 0x7efc940efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59402/0 (socket says 172.21.2.101:59402) 2021-02-10T11:43:22.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.623 7efc92ffd700 1 -- 172.21.2.101:0/2638031543 learned_addr learned my addr 172.21.2.101:0/2638031543 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:22.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.623 7efc93fff700 1 --2- 172.21.2.101:0/2638031543 >> v2:172.21.2.108:3300/0 conn(0x7efc940f20e0 0x7efc940f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:22.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.624 7efc927fc700 1 -- 172.21.2.101:0/2638031543 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1127767393 0 0) 0x7efc880009b0 con 0x7efc940ef610 2021-02-10T11:43:22.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.624 7efc927fc700 1 -- 172.21.2.101:0/2638031543 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc7c002fe0 con 0x7efc940ef610 2021-02-10T11:43:22.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.624 7efc937fe700 1 -- 172.21.2.101:0/2638031543 >> v2:172.21.2.108:3300/0 conn(0x7efc940f20e0 msgr2=0x7efc940f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:22.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.624 7efc937fe700 1 --2- 172.21.2.101:0/2638031543 >> v2:172.21.2.108:3300/0 conn(0x7efc940f20e0 0x7efc940f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:22.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.624 7efc937fe700 1 -- 172.21.2.101:0/2638031543 >> v1:172.21.2.101:6789/0 conn(0x7efc940ef610 legacy=0x7efc940efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:22.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.624 7efc937fe700 1 -- 172.21.2.101:0/2638031543 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc940e52e0 con 0x7efc941b0470 2021-02-10T11:43:22.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.624 7efc937fe700 1 --2- 172.21.2.101:0/2638031543 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc941b0470 0x7efc941b0870 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 rx=0x7efc84006650 tx=0x7efc84009a80).ready entity=mon.0 client_cookie=2875396b2e4506ba server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:22.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.624 7efc927fc700 1 -- 172.21.2.101:0/2638031543 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efc840009b0 con 0x7efc941b0470 2021-02-10T11:43:22.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.624 7efc927fc700 1 -- 172.21.2.101:0/2638031543 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8400c0c0 con 0x7efc941b0470 2021-02-10T11:43:22.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.624 7efc927fc700 1 -- 172.21.2.101:0/2638031543 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efc8400c5e0 con 0x7efc941b0470 2021-02-10T11:43:22.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.624 7efc99f26700 1 -- 172.21.2.101:0/2638031543 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc941b0470 msgr2=0x7efc941b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:22.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.624 7efc99f26700 1 --2- 172.21.2.101:0/2638031543 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc941b0470 0x7efc941b0870 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 rx=0x7efc84006650 tx=0x7efc84009a80).stop 2021-02-10T11:43:22.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.625 7efc99f26700 1 -- 172.21.2.101:0/2638031543 shutdown_connections 2021-02-10T11:43:22.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.625 7efc99f26700 1 --2- 172.21.2.101:0/2638031543 >> v2:172.21.2.108:3300/0 conn(0x7efc940f20e0 0x7efc940f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:22.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.625 7efc99f26700 1 --2- 172.21.2.101:0/2638031543 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc941b0470 0x7efc941b0870 unknown :-1 s=CLOSED pgs=2547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:22.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.625 7efc99f26700 1 -- 172.21.2.101:0/2638031543 >> 172.21.2.101:0/2638031543 conn(0x7efc940ce3e0 msgr2=0x7efc940e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:22.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.625 7efc99f26700 1 -- 172.21.2.101:0/2638031543 shutdown_connections 2021-02-10T11:43:22.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.625 7efc99f26700 1 -- 172.21.2.101:0/2638031543 wait complete. 2021-02-10T11:43:22.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.640 7efc99f26700 1 Processor -- start 2021-02-10T11:43:22.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.640 7efc99f26700 1 -- start start 2021-02-10T11:43:22.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.640 7efc99f26700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efc941a4a20 0x7efc940b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:22.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.640 7efc99f26700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efc940b7fb0 0x7efc9408b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:22.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.640 7efc99f26700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc94106880 0x7efc9408bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:22.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.640 7efc99f26700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9409a260 con 0x7efc94106880 2021-02-10T11:43:22.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.640 7efc99f26700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9409a7e0 con 0x7efc941a4a20 2021-02-10T11:43:22.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.640 7efc99f26700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc9409aa00 con 0x7efc940b7fb0 2021-02-10T11:43:22.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.640 7efc93fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc94106880 0x7efc9408bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:22.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.640 7efc93fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc94106880 0x7efc9408bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58914/0 (socket says 172.21.2.101:58914) 2021-02-10T11:43:22.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.640 7efc93fff700 1 -- 172.21.2.101:0/1594205289 learned_addr learned my addr 172.21.2.101:0/1594205289 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:22.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.641 7efc93fff700 1 -- 172.21.2.101:0/1594205289 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efc940b7fb0 msgr2=0x7efc9408b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:22.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.641 7efc93fff700 1 --2- 172.21.2.101:0/1594205289 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efc940b7fb0 0x7efc9408b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:22.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.641 7efc93fff700 1 -- 172.21.2.101:0/1594205289 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efc941a4a20 msgr2=0x7efc940b7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:22.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.641 7efc93fff700 1 --2- 172.21.2.101:0/1594205289 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efc941a4a20 0x7efc940b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:22.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.641 7efc93fff700 1 -- 172.21.2.101:0/1594205289 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc9409ac20 con 0x7efc94106880 2021-02-10T11:43:22.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.641 7efc93fff700 1 --2- 172.21.2.101:0/1594205289 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc94106880 0x7efc9408bf40 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 rx=0x7efc8c00b970 tx=0x7efc8c00bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:22.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.641 7efc7bfff700 1 -- 172.21.2.101:0/1594205289 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efc8c000f90 con 0x7efc94106880 2021-02-10T11:43:22.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.641 7efc7bfff700 1 -- 172.21.2.101:0/1594205289 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8c003030 con 0x7efc94106880 2021-02-10T11:43:22.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.641 7efc7bfff700 1 -- 172.21.2.101:0/1594205289 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efc8c0015f0 con 0x7efc94106880 2021-02-10T11:43:22.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.641 7efc99f26700 1 -- 172.21.2.101:0/1594205289 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc9409ac20 con 0x7efc94106880 2021-02-10T11:43:22.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.641 7efc99f26700 1 -- 172.21.2.101:0/1594205289 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc9409b1b0 con 0x7efc94106880 2021-02-10T11:43:22.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.642 7efc7bfff700 1 -- 172.21.2.101:0/1594205289 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7efc8c0056b0 con 0x7efc94106880 2021-02-10T11:43:22.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.642 7efc7bfff700 1 --2- 172.21.2.101:0/1594205289 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efc740186c0 0x7efc7401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:22.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.642 7efc7bfff700 1 -- 172.21.2.101:0/1594205289 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7efc8c0328d0 con 0x7efc94106880 2021-02-10T11:43:22.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.642 7efc937fe700 1 --2- 172.21.2.101:0/1594205289 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efc740186c0 0x7efc7401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:22.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.642 7efc937fe700 1 --2- 172.21.2.101:0/1594205289 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efc740186c0 0x7efc7401abc0 crc :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:22.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.673 7efc99f26700 1 -- 172.21.2.101:0/1594205289 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc94105ab0 con 0x7efc94106880 2021-02-10T11:43:22.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.676 7efc7bfff700 1 -- 172.21.2.101:0/1594205289 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7efc8c000f20 con 0x7efc94106880 2021-02-10T11:43:22.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.803 7efc99f26700 1 -- 172.21.2.101:0/1594205289 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc940f0b50 con 0x7efc94106880 2021-02-10T11:43:22.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.803 7efc7bfff700 1 -- 172.21.2.101:0/1594205289 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+2102 (secure 0 0 0) 0x7efc8c0056b0 con 0x7efc94106880 2021-02-10T11:43:22.804 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:43:22.804 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":114,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/371124351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":371124351},{"type":"v1","addr":"172.21.2.101:6835","nonce":371124351}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:43:20.159141","modified":"2021-02-10 11:43:22.169386","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7990":{"gid":7990,"name":"b","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/797061093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":797061093},{"type":"v1","addr":"172.21.2.108:6833","nonce":797061093}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-02-10T11:43:22.804 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 114 2021-02-10T11:43:22.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.805 7efc99f26700 1 -- 172.21.2.101:0/1594205289 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efc740186c0 msgr2=0x7efc7401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:22.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.805 7efc99f26700 1 --2- 172.21.2.101:0/1594205289 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efc740186c0 0x7efc7401abc0 crc :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:22.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.805 7efc99f26700 1 -- 172.21.2.101:0/1594205289 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc94106880 msgr2=0x7efc9408bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:22.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.805 7efc99f26700 1 --2- 172.21.2.101:0/1594205289 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc94106880 0x7efc9408bf40 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 rx=0x7efc8c00b970 tx=0x7efc8c00bd10).stop 2021-02-10T11:43:22.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.805 7efc99f26700 1 -- 172.21.2.101:0/1594205289 shutdown_connections 2021-02-10T11:43:22.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.805 7efc99f26700 1 --2- 172.21.2.101:0/1594205289 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efc740186c0 0x7efc7401abc0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:22.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.805 7efc99f26700 1 --2- 172.21.2.101:0/1594205289 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efc940b7fb0 0x7efc9408b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:22.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.805 7efc99f26700 1 --2- 172.21.2.101:0/1594205289 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc94106880 0x7efc9408bf40 unknown :-1 s=CLOSED pgs=2548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:22.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.805 7efc99f26700 1 --2- 172.21.2.101:0/1594205289 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efc941a4a20 0x7efc940b7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:22.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.805 7efc99f26700 1 -- 172.21.2.101:0/1594205289 >> 172.21.2.101:0/1594205289 conn(0x7efc940b9290 msgr2=0x7efc940aa950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:22.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.805 7efc99f26700 1 -- 172.21.2.101:0/1594205289 shutdown_connections 2021-02-10T11:43:22.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:22.805 7efc99f26700 1 -- 172.21.2.101:0/1594205289 wait complete. 2021-02-10T11:43:22.814 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:43:23.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.227 7fea7ddd8700 1 Processor -- start 2021-02-10T11:43:23.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.227 7fea7ddd8700 1 -- start start 2021-02-10T11:43:23.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.227 7fea7ddd8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fea780d1870 0x7fea781b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:23.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.227 7fea7ddd8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea781b0a10 0x7fea780cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:23.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.227 7fea7ddd8700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea780f0b10 con 0x7fea780d1c70 2021-02-10T11:43:23.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.227 7fea7ddd8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea780e5ed0 con 0x7fea780d1870 2021-02-10T11:43:23.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.227 7fea7ddd8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea780f0650 con 0x7fea781b0a10 2021-02-10T11:43:23.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.227 7fea777fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fea780d1870 0x7fea781b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:23.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.227 7fea777fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fea780d1870 0x7fea781b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58918/0 (socket says 172.21.2.101:58918) 2021-02-10T11:43:23.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.227 7fea777fe700 1 -- 172.21.2.101:0/890752510 learned_addr learned my addr 172.21.2.101:0/890752510 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:23.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.227 7fea76ffd700 1 --2- 172.21.2.101:0/890752510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea781b0a10 0x7fea780cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:23.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.228 7fea777fe700 1 -- 172.21.2.101:0/890752510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea781b0a10 msgr2=0x7fea780cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:23.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.228 7fea777fe700 1 --2- 172.21.2.101:0/890752510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea781b0a10 0x7fea780cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:23.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.228 7fea777fe700 1 -- 172.21.2.101:0/890752510 >> v1:172.21.2.101:6789/0 conn(0x7fea780d1c70 legacy=0x7fea780cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:23.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.228 7fea777fe700 1 -- 172.21.2.101:0/890752510 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea780f0870 con 0x7fea780d1870 2021-02-10T11:43:23.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.228 7fea777fe700 1 --2- 172.21.2.101:0/890752510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea780d1870 0x7fea781b03f0 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 rx=0x7fea68006700 tx=0x7fea68009b80).ready entity=mon.0 client_cookie=4c92b85836bfa694 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:23.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.228 7fea767fc700 1 -- 172.21.2.101:0/890752510 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fea6800c0c0 con 0x7fea780d1870 2021-02-10T11:43:23.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.228 7fea767fc700 1 -- 172.21.2.101:0/890752510 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6800c3c0 con 0x7fea780d1870 2021-02-10T11:43:23.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.228 7fea767fc700 1 -- 172.21.2.101:0/890752510 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fea6800c8e0 con 0x7fea780d1870 2021-02-10T11:43:23.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.228 7fea7ddd8700 1 -- 172.21.2.101:0/890752510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea780d1870 msgr2=0x7fea781b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:23.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.228 7fea7ddd8700 1 --2- 172.21.2.101:0/890752510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea780d1870 0x7fea781b03f0 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 rx=0x7fea68006700 tx=0x7fea68009b80).stop 2021-02-10T11:43:23.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.229 7fea7ddd8700 1 -- 172.21.2.101:0/890752510 shutdown_connections 2021-02-10T11:43:23.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.229 7fea7ddd8700 1 --2- 172.21.2.101:0/890752510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea781b0a10 0x7fea780cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:23.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.229 7fea7ddd8700 1 --2- 172.21.2.101:0/890752510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea780d1870 0x7fea781b03f0 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:23.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.229 7fea7ddd8700 1 -- 172.21.2.101:0/890752510 >> 172.21.2.101:0/890752510 conn(0x7fea780ce450 msgr2=0x7fea780d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:23.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.229 7fea7ddd8700 1 -- 172.21.2.101:0/890752510 shutdown_connections 2021-02-10T11:43:23.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.229 7fea7ddd8700 1 -- 172.21.2.101:0/890752510 wait complete. 2021-02-10T11:43:23.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.244 7fea7ddd8700 1 Processor -- start 2021-02-10T11:43:23.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.244 7fea7ddd8700 1 -- start start 2021-02-10T11:43:23.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.244 7fea7ddd8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea781b7cf0 0x7fea7819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:23.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.244 7fea7ddd8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea781b80f0 0x7fea781b84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:23.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.244 7fea7ddd8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea7819da20 0x7fea7819de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:23.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.244 7fea7ddd8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea780a77c0 con 0x7fea781b7cf0 2021-02-10T11:43:23.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.244 7fea7ddd8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7819d4a0 con 0x7fea7819da20 2021-02-10T11:43:23.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.244 7fea7ddd8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea780954f0 con 0x7fea781b80f0 2021-02-10T11:43:23.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.244 7fea777fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea781b7cf0 0x7fea7819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:23.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.244 7fea777fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea781b7cf0 0x7fea7819cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58924/0 (socket says 172.21.2.101:58924) 2021-02-10T11:43:23.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.244 7fea777fe700 1 -- 172.21.2.101:0/3824566646 learned_addr learned my addr 172.21.2.101:0/3824566646 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:23.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.245 7fea777fe700 1 -- 172.21.2.101:0/3824566646 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea781b80f0 msgr2=0x7fea781b84f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:23.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.245 7fea777fe700 1 --2- 172.21.2.101:0/3824566646 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea781b80f0 0x7fea781b84f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:23.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.245 7fea777fe700 1 -- 172.21.2.101:0/3824566646 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea7819da20 msgr2=0x7fea7819de20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:23.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.245 7fea777fe700 1 --2- 172.21.2.101:0/3824566646 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea7819da20 0x7fea7819de20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:23.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.245 7fea777fe700 1 -- 172.21.2.101:0/3824566646 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea781b8d60 con 0x7fea781b7cf0 2021-02-10T11:43:23.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.245 7fea777fe700 1 --2- 172.21.2.101:0/3824566646 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea781b7cf0 0x7fea7819cdb0 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 rx=0x7fea6800c620 tx=0x7fea6800c9c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:23.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.245 7fea6ffff700 1 -- 172.21.2.101:0/3824566646 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fea6800cdb0 con 0x7fea781b7cf0 2021-02-10T11:43:23.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.245 7fea6ffff700 1 -- 172.21.2.101:0/3824566646 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea68009390 con 0x7fea781b7cf0 2021-02-10T11:43:23.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.245 7fea6ffff700 1 -- 172.21.2.101:0/3824566646 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fea68002220 con 0x7fea781b7cf0 2021-02-10T11:43:23.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.245 7fea7ddd8700 1 -- 172.21.2.101:0/3824566646 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea781b8d60 con 0x7fea781b7cf0 2021-02-10T11:43:23.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.245 7fea7ddd8700 1 -- 172.21.2.101:0/3824566646 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea7819e730 con 0x7fea781b7cf0 2021-02-10T11:43:23.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.246 7fea6ffff700 1 -- 172.21.2.101:0/3824566646 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fea68000c90 con 0x7fea781b7cf0 2021-02-10T11:43:23.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.246 7fea6ffff700 1 --2- 172.21.2.101:0/3824566646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea5c0186c0 0x7fea5c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:23.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.246 7fea6ffff700 1 -- 172.21.2.101:0/3824566646 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fea68032590 con 0x7fea781b7cf0 2021-02-10T11:43:23.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.246 7fea76ffd700 1 --2- 172.21.2.101:0/3824566646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea5c0186c0 0x7fea5c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:23.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.246 7fea76ffd700 1 --2- 172.21.2.101:0/3824566646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea5c0186c0 0x7fea5c01abc0 crc :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:23.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.277 7fea7ddd8700 1 -- 172.21.2.101:0/3824566646 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea780ebc00 con 0x7fea781b7cf0 2021-02-10T11:43:23.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.279 7fea6ffff700 1 -- 172.21.2.101:0/3824566646 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fea68000940 con 0x7fea781b7cf0 2021-02-10T11:43:23.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.404 7fea7ddd8700 1 -- 172.21.2.101:0/3824566646 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea780f0b10 con 0x7fea781b7cf0 2021-02-10T11:43:23.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.405 7fea6ffff700 1 -- 172.21.2.101:0/3824566646 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+14007 (secure 0 0 0) 0x7fea68000c90 con 0x7fea781b7cf0 2021-02-10T11:43:23.406 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:43:23.406 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":138,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:43:21.161685","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:43:18.076350","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2021-02-10 11:43:19.152025","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:43:23.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.406 7fea7ddd8700 1 -- 172.21.2.101:0/3824566646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea5c0186c0 msgr2=0x7fea5c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:23.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.406 7fea7ddd8700 1 --2- 172.21.2.101:0/3824566646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea5c0186c0 0x7fea5c01abc0 crc :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:23.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.406 7fea7ddd8700 1 -- 172.21.2.101:0/3824566646 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea781b7cf0 msgr2=0x7fea7819cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:23.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.406 7fea7ddd8700 1 --2- 172.21.2.101:0/3824566646 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea781b7cf0 0x7fea7819cdb0 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 rx=0x7fea6800c620 tx=0x7fea6800c9c0).stop 2021-02-10T11:43:23.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.407 7fea7ddd8700 1 -- 172.21.2.101:0/3824566646 shutdown_connections 2021-02-10T11:43:23.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.407 7fea7ddd8700 1 --2- 172.21.2.101:0/3824566646 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fea5c0186c0 0x7fea5c01abc0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:23.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.407 7fea7ddd8700 1 --2- 172.21.2.101:0/3824566646 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fea781b80f0 0x7fea781b84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:23.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.407 7fea7ddd8700 1 --2- 172.21.2.101:0/3824566646 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fea7819da20 0x7fea7819de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:23.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.407 7fea7ddd8700 1 --2- 172.21.2.101:0/3824566646 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fea781b7cf0 0x7fea7819cdb0 unknown :-1 s=CLOSED pgs=2551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:23.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.407 7fea7ddd8700 1 -- 172.21.2.101:0/3824566646 >> 172.21.2.101:0/3824566646 conn(0x7fea780ed810 msgr2=0x7fea78094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:23.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.407 7fea7ddd8700 1 -- 172.21.2.101:0/3824566646 shutdown_connections 2021-02-10T11:43:23.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.407 7fea7ddd8700 1 -- 172.21.2.101:0/3824566646 wait complete. 2021-02-10T11:43:23.416 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:43:23.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.840 7f298b02f700 1 Processor -- start 2021-02-10T11:43:23.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.840 7f298b02f700 1 -- start start 2021-02-10T11:43:23.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.840 7f298b02f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f29840d1c80 0x7f29840cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:23.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.840 7f298b02f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f29840cf590 0x7f29840cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:23.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.840 7f298b02f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29840fe1d0 con 0x7f29840d1880 2021-02-10T11:43:23.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.840 7f298b02f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29841b0cc0 con 0x7f29840d1c80 2021-02-10T11:43:23.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.840 7f298b02f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2984109a70 con 0x7f29840cf590 2021-02-10T11:43:23.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.840 7f2983fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f29840d1c80 0x7f29840cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:23.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.840 7f2983fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f29840d1c80 0x7f29840cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58930/0 (socket says 172.21.2.101:58930) 2021-02-10T11:43:23.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.840 7f2983fff700 1 -- 172.21.2.101:0/3244929698 learned_addr learned my addr 172.21.2.101:0/3244929698 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:23.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.840 7f2983fff700 1 -- 172.21.2.101:0/3244929698 >> v2:172.21.2.108:3300/0 conn(0x7f29840cf590 msgr2=0x7f29840cf990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:43:23.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.840 7f2983fff700 1 --2- 172.21.2.101:0/3244929698 >> v2:172.21.2.108:3300/0 conn(0x7f29840cf590 0x7f29840cf990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:23.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.841 7f2983fff700 1 -- 172.21.2.101:0/3244929698 >> v1:172.21.2.108:6789/0 conn(0x7f29840d1880 legacy=0x7f29841b04d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:43:23.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.841 7f2983fff700 1 -- 172.21.2.101:0/3244929698 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2984088070 con 0x7f29840d1c80 2021-02-10T11:43:23.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.841 7f2983fff700 1 --2- 172.21.2.101:0/3244929698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29840d1c80 0x7f29840cef70 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 rx=0x7f2974006a80 tx=0x7f2974009e80).ready entity=mon.0 client_cookie=cd4dcbdd27d267f6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:23.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.841 7f29837fe700 1 -- 172.21.2.101:0/3244929698 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f297400b510 con 0x7f29840d1c80 2021-02-10T11:43:23.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.841 7f29837fe700 1 -- 172.21.2.101:0/3244929698 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297400b7d0 con 0x7f29840d1c80 2021-02-10T11:43:23.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.841 7f29837fe700 1 -- 172.21.2.101:0/3244929698 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f297400bcb0 con 0x7f29840d1c80 2021-02-10T11:43:23.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.841 7f298b02f700 1 -- 172.21.2.101:0/3244929698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29840d1c80 msgr2=0x7f29840cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:23.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.841 7f298b02f700 1 --2- 172.21.2.101:0/3244929698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29840d1c80 0x7f29840cef70 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 rx=0x7f2974006a80 tx=0x7f2974009e80).stop 2021-02-10T11:43:23.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.841 7f298b02f700 1 -- 172.21.2.101:0/3244929698 shutdown_connections 2021-02-10T11:43:23.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.841 7f298b02f700 1 --2- 172.21.2.101:0/3244929698 >> v2:172.21.2.108:3300/0 conn(0x7f29840cf590 0x7f29840cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:23.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.841 7f298b02f700 1 --2- 172.21.2.101:0/3244929698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29840d1c80 0x7f29840cef70 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:23.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.841 7f298b02f700 1 -- 172.21.2.101:0/3244929698 >> 172.21.2.101:0/3244929698 conn(0x7f29840ce430 msgr2=0x7f29840d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:23.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.842 7f298b02f700 1 -- 172.21.2.101:0/3244929698 shutdown_connections 2021-02-10T11:43:23.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.842 7f298b02f700 1 -- 172.21.2.101:0/3244929698 wait complete. 2021-02-10T11:43:23.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.856 7f298b02f700 1 Processor -- start 2021-02-10T11:43:23.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.856 7f298b02f700 1 -- start start 2021-02-10T11:43:23.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.856 7f298b02f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2984086520 0x7f29841a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:23.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.856 7f298b02f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2984086920 0x7f2984187e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:23.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.856 7f298b02f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29841a22d0 0x7f29841884a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:23.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.856 7f298b02f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2984099600 con 0x7f2984086520 2021-02-10T11:43:23.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.856 7f298b02f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29840999f0 con 0x7f2984086920 2021-02-10T11:43:23.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.856 7f298b02f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2984188ac0 con 0x7f29841a22d0 2021-02-10T11:43:23.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.857 7f2988e4b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2984086520 0x7f29841a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:23.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.857 7f2988e4b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2984086520 0x7f29841a1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58938/0 (socket says 172.21.2.101:58938) 2021-02-10T11:43:23.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.857 7f2988e4b700 1 -- 172.21.2.101:0/1959963851 learned_addr learned my addr 172.21.2.101:0/1959963851 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:23.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.857 7f2988e4b700 1 -- 172.21.2.101:0/1959963851 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29841a22d0 msgr2=0x7f29841884a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:23.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.857 7f2988e4b700 1 --2- 172.21.2.101:0/1959963851 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29841a22d0 0x7f29841884a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:23.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.857 7f2988e4b700 1 -- 172.21.2.101:0/1959963851 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2984086920 msgr2=0x7f2984187e80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:23.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.857 7f2988e4b700 1 --2- 172.21.2.101:0/1959963851 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2984086920 0x7f2984187e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:23.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.857 7f2988e4b700 1 -- 172.21.2.101:0/1959963851 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2984188ce0 con 0x7f2984086520 2021-02-10T11:43:23.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.857 7f2988e4b700 1 --2- 172.21.2.101:0/1959963851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2984086520 0x7f29841a1cb0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 rx=0x7f296c00b530 tx=0x7f296c00b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:23.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.857 7f29817fa700 1 -- 172.21.2.101:0/1959963851 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f296c00bcc0 con 0x7f2984086520 2021-02-10T11:43:23.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.857 7f29817fa700 1 -- 172.21.2.101:0/1959963851 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296c006020 con 0x7f2984086520 2021-02-10T11:43:23.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.857 7f29817fa700 1 -- 172.21.2.101:0/1959963851 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f296c0054a0 con 0x7f2984086520 2021-02-10T11:43:23.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.857 7f298b02f700 1 -- 172.21.2.101:0/1959963851 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2984188ce0 con 0x7f2984086520 2021-02-10T11:43:23.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.857 7f298b02f700 1 -- 172.21.2.101:0/1959963851 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2984189270 con 0x7f2984086520 2021-02-10T11:43:23.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.858 7f29817fa700 1 -- 172.21.2.101:0/1959963851 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f296c006200 con 0x7f2984086520 2021-02-10T11:43:23.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.858 7f29817fa700 1 --2- 172.21.2.101:0/1959963851 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29680186c0 0x7f296801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:23.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.858 7f29817fa700 1 -- 172.21.2.101:0/1959963851 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f296c0325b0 con 0x7f2984086520 2021-02-10T11:43:23.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.858 7f2983fff700 1 --2- 172.21.2.101:0/1959963851 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29680186c0 0x7f296801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:23.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.858 7f2983fff700 1 --2- 172.21.2.101:0/1959963851 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29680186c0 0x7f296801abc0 crc :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:23.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.890 7f298b02f700 1 -- 172.21.2.101:0/1959963851 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29840fe1d0 con 0x7f2984086520 2021-02-10T11:43:23.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:23.893 7f29817fa700 1 -- 172.21.2.101:0/1959963851 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f296c030d60 con 0x7f2984086520 2021-02-10T11:43:24.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.022 7f298b02f700 1 -- 172.21.2.101:0/1959963851 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f29840fe1d0 con 0x7f2984086520 2021-02-10T11:43:24.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.026 7f29817fa700 1 -- 172.21.2.101:0/1959963851 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v79) v1 ==== 184+0+0 (secure 0 0 0) 0x7f296c01f730 con 0x7f2984086520 2021-02-10T11:43:24.030 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.0 2021-02-10T11:43:24.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.031 7f298b02f700 1 -- 172.21.2.101:0/1959963851 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29680186c0 msgr2=0x7f296801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:24.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.031 7f298b02f700 1 --2- 172.21.2.101:0/1959963851 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29680186c0 0x7f296801abc0 crc :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:24.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.031 7f298b02f700 1 -- 172.21.2.101:0/1959963851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2984086520 msgr2=0x7f29841a1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:24.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.031 7f298b02f700 1 --2- 172.21.2.101:0/1959963851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2984086520 0x7f29841a1cb0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 rx=0x7f296c00b530 tx=0x7f296c00b8d0).stop 2021-02-10T11:43:24.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.031 7f298b02f700 1 -- 172.21.2.101:0/1959963851 shutdown_connections 2021-02-10T11:43:24.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.031 7f298b02f700 1 --2- 172.21.2.101:0/1959963851 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29680186c0 0x7f296801abc0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:24.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.031 7f298b02f700 1 --2- 172.21.2.101:0/1959963851 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29841a22d0 0x7f29841884a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:24.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.031 7f298b02f700 1 --2- 172.21.2.101:0/1959963851 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2984086920 0x7f2984187e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:24.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.032 7f298b02f700 1 --2- 172.21.2.101:0/1959963851 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2984086520 0x7f29841a1cb0 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:24.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.032 7f298b02f700 1 -- 172.21.2.101:0/1959963851 >> 172.21.2.101:0/1959963851 conn(0x7f298419f880 msgr2=0x7f298408f890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:24.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.032 7f298b02f700 1 -- 172.21.2.101:0/1959963851 shutdown_connections 2021-02-10T11:43:24.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.032 7f298b02f700 1 -- 172.21.2.101:0/1959963851 wait complete. 2021-02-10T11:43:24.040 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:43:24.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.465 7fa2b21aa700 1 Processor -- start 2021-02-10T11:43:24.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.465 7fa2b21aa700 1 -- start start 2021-02-10T11:43:24.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.465 7fa2b21aa700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa2ac10b000 0x7fa2ac0cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:24.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.465 7fa2b21aa700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ac0fe1d0 con 0x7fa2ac0d1c80 2021-02-10T11:43:24.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.465 7fa2b21aa700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ac1b0cc0 con 0x7fa2ac0d1880 2021-02-10T11:43:24.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.465 7fa2b21aa700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2ac109a70 con 0x7fa2ac10b000 2021-02-10T11:43:24.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.465 7fa2aaffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa2ac0d1c80 0x7fa2ac0cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59436/0 (socket says 172.21.2.101:59436) 2021-02-10T11:43:24.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.465 7fa2aaffd700 1 -- 172.21.2.101:0/2977533968 learned_addr learned my addr 172.21.2.101:0/2977533968 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:24.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.466 7fa2aa7fc700 1 -- 172.21.2.101:0/2977533968 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843256331 0 0) 0x7fa2940010b0 con 0x7fa2ac0d1c80 2021-02-10T11:43:24.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.466 7fa2aa7fc700 1 -- 172.21.2.101:0/2977533968 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa298002fe0 con 0x7fa2ac0d1c80 2021-02-10T11:43:24.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.466 7fa2aa7fc700 1 -- 172.21.2.101:0/2977533968 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3972705100 0 0) 0x7fa2940010b0 con 0x7fa2ac0d1c80 2021-02-10T11:43:24.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.466 7fa2aa7fc700 1 -- 172.21.2.101:0/2977533968 >> v2:172.21.2.108:3300/0 conn(0x7fa2ac10b000 msgr2=0x7fa2ac0cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:24.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.466 7fa2abfff700 1 --2- 172.21.2.101:0/2977533968 >> v2:172.21.2.108:3300/0 conn(0x7fa2ac10b000 0x7fa2ac0cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:24.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.466 7fa2aa7fc700 1 --2- 172.21.2.101:0/2977533968 >> v2:172.21.2.108:3300/0 conn(0x7fa2ac10b000 0x7fa2ac0cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:24.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.466 7fa2aa7fc700 1 -- 172.21.2.101:0/2977533968 >> v1:172.21.2.108:6789/0 conn(0x7fa2ac0d1880 legacy=0x7fa2ac1b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:24.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.466 7fa2aa7fc700 1 -- 172.21.2.101:0/2977533968 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ac10b400 con 0x7fa2ac0d1c80 2021-02-10T11:43:24.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.466 7fa2aa7fc700 1 -- 172.21.2.101:0/2977533968 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fa2940015b0 con 0x7fa2ac0d1c80 2021-02-10T11:43:24.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.466 7fa2aa7fc700 1 -- 172.21.2.101:0/2977533968 >> v1:172.21.2.101:6789/0 conn(0x7fa2ac0d1c80 legacy=0x7fa2ac0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:24.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.466 7fa2aa7fc700 1 --2- 172.21.2.101:0/2977533968 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa298003ae0 0x7fa298005f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:24.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.466 7fa2aa7fc700 1 --2- 172.21.2.101:0/2977533968 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa298006830 0x7fa298008d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:24.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.466 7fa2aa7fc700 1 --2- 172.21.2.101:0/2977533968 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa298009630 0x7fa29800bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:24.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.466 7fa2aa7fc700 1 -- 172.21.2.101:0/2977533968 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29800c3a0 con 0x7fa298006830 2021-02-10T11:43:24.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.466 7fa2aa7fc700 1 -- 172.21.2.101:0/2977533968 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29800c5c0 con 0x7fa298009630 2021-02-10T11:43:24.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.466 7fa2aa7fc700 1 -- 172.21.2.101:0/2977533968 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa29800c7e0 con 0x7fa298003ae0 2021-02-10T11:43:24.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.466 7fa2aaffd700 1 --2- 172.21.2.101:0/2977533968 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa298006830 0x7fa298008d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:24.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.466 7fa2abfff700 1 --2- 172.21.2.101:0/2977533968 >> v2:172.21.2.108:3300/0 conn(0x7fa2ac10b000 0x7fa2ac0cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:43:24.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.466 7fa2aaffd700 1 -- 172.21.2.101:0/2977533968 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa298003ae0 msgr2=0x7fa298005f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:24.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.467 7fa2aaffd700 1 --2- 172.21.2.101:0/2977533968 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa298003ae0 0x7fa298005f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:24.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.467 7fa2aaffd700 1 -- 172.21.2.101:0/2977533968 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa298009630 msgr2=0x7fa29800bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:24.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.467 7fa2aaffd700 1 --2- 172.21.2.101:0/2977533968 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa298009630 0x7fa29800bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:24.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.467 7fa2aaffd700 1 -- 172.21.2.101:0/2977533968 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29800ca80 con 0x7fa298006830 2021-02-10T11:43:24.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.467 7fa2abfff700 1 -- 172.21.2.101:0/2977533968 reap_dead start 2021-02-10T11:43:24.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.467 7fa2aaffd700 1 --2- 172.21.2.101:0/2977533968 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa298006830 0x7fa298008d30 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 rx=0x7fa294001c60 tx=0x7fa294007d40).ready entity=mon.0 client_cookie=fc935b376a4999c2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:24.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.467 7fa2aa7fc700 1 -- 172.21.2.101:0/2977533968 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa294001010 con 0x7fa298006830 2021-02-10T11:43:24.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.467 7fa2aa7fc700 1 -- 172.21.2.101:0/2977533968 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29404e320 con 0x7fa298006830 2021-02-10T11:43:24.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.467 7fa2aa7fc700 1 -- 172.21.2.101:0/2977533968 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa29404e6f0 con 0x7fa298006830 2021-02-10T11:43:24.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.467 7fa2b21aa700 1 -- 172.21.2.101:0/2977533968 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa298006830 msgr2=0x7fa298008d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:24.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.467 7fa2b21aa700 1 --2- 172.21.2.101:0/2977533968 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa298006830 0x7fa298008d30 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 rx=0x7fa294001c60 tx=0x7fa294007d40).stop 2021-02-10T11:43:24.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.467 7fa2b21aa700 1 -- 172.21.2.101:0/2977533968 shutdown_connections 2021-02-10T11:43:24.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.467 7fa2b21aa700 1 --2- 172.21.2.101:0/2977533968 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa298006830 0x7fa298008d30 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:24.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.467 7fa2b21aa700 1 -- 172.21.2.101:0/2977533968 >> 172.21.2.101:0/2977533968 conn(0x7fa2ac0ce430 msgr2=0x7fa2ac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:24.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.467 7fa2b21aa700 1 -- 172.21.2.101:0/2977533968 shutdown_connections 2021-02-10T11:43:24.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.467 7fa2b21aa700 1 -- 172.21.2.101:0/2977533968 wait complete. 2021-02-10T11:43:24.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.482 7fa2b21aa700 1 Processor -- start 2021-02-10T11:43:24.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.482 7fa2b21aa700 1 -- start start 2021-02-10T11:43:24.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.482 7fa2b21aa700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2ac1a0580 0x7fa2ac086fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:24.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.482 7fa2b21aa700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2ac0b66b0 0x7fa2ac0b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:24.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.482 7fa2b21aa700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2ac082bb0 0x7fa2ac082fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:24.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.482 7fa2b21aa700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac0875d0 con 0x7fa2ac082bb0 2021-02-10T11:43:24.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.482 7fa2b21aa700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac0835d0 con 0x7fa2ac1a0580 2021-02-10T11:43:24.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.482 7fa2b21aa700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac0837f0 con 0x7fa2ac0b66b0 2021-02-10T11:43:24.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.483 7fa2abfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2ac082bb0 0x7fa2ac082fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:24.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.483 7fa2abfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2ac082bb0 0x7fa2ac082fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58958/0 (socket says 172.21.2.101:58958) 2021-02-10T11:43:24.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.483 7fa2abfff700 1 -- 172.21.2.101:0/4183423782 learned_addr learned my addr 172.21.2.101:0/4183423782 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:24.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.483 7fa2abfff700 1 -- 172.21.2.101:0/4183423782 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2ac0b66b0 msgr2=0x7fa2ac0b6ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:24.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.483 7fa2abfff700 1 --2- 172.21.2.101:0/4183423782 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2ac0b66b0 0x7fa2ac0b6ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:24.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.483 7fa2abfff700 1 -- 172.21.2.101:0/4183423782 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2ac1a0580 msgr2=0x7fa2ac086fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:24.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.483 7fa2abfff700 1 --2- 172.21.2.101:0/4183423782 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2ac1a0580 0x7fa2ac086fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:24.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.483 7fa2abfff700 1 -- 172.21.2.101:0/4183423782 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ac083a10 con 0x7fa2ac082bb0 2021-02-10T11:43:24.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.483 7fa2abfff700 1 --2- 172.21.2.101:0/4183423782 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2ac082bb0 0x7fa2ac082fb0 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 rx=0x7fa2a4009980 tx=0x7fa2a400bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:24.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.483 7fa2a3fff700 1 -- 172.21.2.101:0/4183423782 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa2a4000e40 con 0x7fa2ac082bb0 2021-02-10T11:43:24.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.483 7fa2a3fff700 1 -- 172.21.2.101:0/4183423782 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a400c2f0 con 0x7fa2ac082bb0 2021-02-10T11:43:24.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.483 7fa2b21aa700 1 -- 172.21.2.101:0/4183423782 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ac083a10 con 0x7fa2ac082bb0 2021-02-10T11:43:24.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.483 7fa2b21aa700 1 -- 172.21.2.101:0/4183423782 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ac083fa0 con 0x7fa2ac082bb0 2021-02-10T11:43:24.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.483 7fa2a3fff700 1 -- 172.21.2.101:0/4183423782 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa2a400c810 con 0x7fa2ac082bb0 2021-02-10T11:43:24.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.484 7fa2a3fff700 1 -- 172.21.2.101:0/4183423782 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa2a4000e40 con 0x7fa2ac082bb0 2021-02-10T11:43:24.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.484 7fa2a3fff700 1 --2- 172.21.2.101:0/4183423782 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa290018730 0x7fa29001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:24.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.484 7fa2a3fff700 1 -- 172.21.2.101:0/4183423782 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa2a4032b80 con 0x7fa2ac082bb0 2021-02-10T11:43:24.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.484 7fa2ab7fe700 1 --2- 172.21.2.101:0/4183423782 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa290018730 0x7fa29001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:24.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.484 7fa2ab7fe700 1 --2- 172.21.2.101:0/4183423782 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa290018730 0x7fa29001abc0 crc :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:24.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.515 7fa2b21aa700 1 -- 172.21.2.101:0/4183423782 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ac082980 con 0x7fa2ac082bb0 2021-02-10T11:43:24.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.518 7fa2a3fff700 1 -- 172.21.2.101:0/4183423782 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa2a4006930 con 0x7fa2ac082bb0 2021-02-10T11:43:24.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.648 7fa2b21aa700 1 -- 172.21.2.101:0/4183423782 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fa2ac082980 con 0x7fa2ac082bb0 2021-02-10T11:43:24.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.652 7fa2a3fff700 1 -- 172.21.2.101:0/4183423782 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v80) v1 ==== 184+0+0 (secure 0 0 0) 0x7fa2a4006b40 con 0x7fa2ac082bb0 2021-02-10T11:43:24.655 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.1 2021-02-10T11:43:24.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.656 7fa2b21aa700 1 -- 172.21.2.101:0/4183423782 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa290018730 msgr2=0x7fa29001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:24.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.656 7fa2b21aa700 1 --2- 172.21.2.101:0/4183423782 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa290018730 0x7fa29001abc0 crc :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:24.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.656 7fa2b21aa700 1 -- 172.21.2.101:0/4183423782 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2ac082bb0 msgr2=0x7fa2ac082fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:24.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.656 7fa2b21aa700 1 --2- 172.21.2.101:0/4183423782 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2ac082bb0 0x7fa2ac082fb0 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 rx=0x7fa2a4009980 tx=0x7fa2a400bcd0).stop 2021-02-10T11:43:24.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.656 7fa2b21aa700 1 -- 172.21.2.101:0/4183423782 shutdown_connections 2021-02-10T11:43:24.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.656 7fa2b21aa700 1 --2- 172.21.2.101:0/4183423782 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa290018730 0x7fa29001abc0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:24.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.657 7fa2b21aa700 1 --2- 172.21.2.101:0/4183423782 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa2ac0b66b0 0x7fa2ac0b6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:24.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.657 7fa2b21aa700 1 --2- 172.21.2.101:0/4183423782 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa2ac082bb0 0x7fa2ac082fb0 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:24.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.657 7fa2b21aa700 1 --2- 172.21.2.101:0/4183423782 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa2ac1a0580 0x7fa2ac086fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:24.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.657 7fa2b21aa700 1 -- 172.21.2.101:0/4183423782 >> 172.21.2.101:0/4183423782 conn(0x7fa2ac0afc60 msgr2=0x7fa2ac0b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:24.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.657 7fa2b21aa700 1 -- 172.21.2.101:0/4183423782 shutdown_connections 2021-02-10T11:43:24.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:24.657 7fa2b21aa700 1 -- 172.21.2.101:0/4183423782 wait complete. 2021-02-10T11:43:24.666 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:43:25.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.087 7fa0cc664700 1 Processor -- start 2021-02-10T11:43:25.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.088 7fa0cc664700 1 -- start start 2021-02-10T11:43:25.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.088 7fa0cc664700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa0c40d1880 0x7fa0c41b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:25.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.088 7fa0cc664700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c4109a70 con 0x7fa0c41b0af0 2021-02-10T11:43:25.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.088 7fa0cc664700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c4088070 con 0x7fa0c40cff80 2021-02-10T11:43:25.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.088 7fa0cc664700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40e5f00 con 0x7fa0c40d1880 2021-02-10T11:43:25.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.088 7fa0c9c7f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa0c41b0af0 0x7fa0c4108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59456/0 (socket says 172.21.2.101:59456) 2021-02-10T11:43:25.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.088 7fa0c9c7f700 1 -- 172.21.2.101:0/3332736267 learned_addr learned my addr 172.21.2.101:0/3332736267 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:25.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.089 7fa0c947e700 1 -- 172.21.2.101:0/3332736267 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2462916744 0 0) 0x7fa0b40010b0 con 0x7fa0c41b0af0 2021-02-10T11:43:25.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.089 7fa0c947e700 1 -- 172.21.2.101:0/3332736267 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0b0002fe0 con 0x7fa0c41b0af0 2021-02-10T11:43:25.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.089 7fa0ca480700 1 --2- 172.21.2.101:0/3332736267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa0c40d1880 0x7fa0c41b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:25.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.089 7fa0c947e700 1 -- 172.21.2.101:0/3332736267 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 58579970 0 0) 0x7fa0b40010b0 con 0x7fa0c41b0af0 2021-02-10T11:43:25.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.089 7fa0c947e700 1 -- 172.21.2.101:0/3332736267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa0c40d1880 msgr2=0x7fa0c41b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:25.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.089 7fa0c947e700 1 --2- 172.21.2.101:0/3332736267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa0c40d1880 0x7fa0c41b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:25.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.089 7fa0c947e700 1 -- 172.21.2.101:0/3332736267 >> v1:172.21.2.108:6789/0 conn(0x7fa0c40cff80 legacy=0x7fa0c4109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:25.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.089 7fa0c947e700 1 -- 172.21.2.101:0/3332736267 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c40d1c80 con 0x7fa0c41b0af0 2021-02-10T11:43:25.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.089 7fa0c947e700 1 -- 172.21.2.101:0/3332736267 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fa0b40015b0 con 0x7fa0c41b0af0 2021-02-10T11:43:25.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.089 7fa0c947e700 1 -- 172.21.2.101:0/3332736267 >> v1:172.21.2.101:6789/0 conn(0x7fa0c41b0af0 legacy=0x7fa0c4108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:25.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.089 7fa0c947e700 1 --2- 172.21.2.101:0/3332736267 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0b0003ac0 0x7fa0b0005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:25.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.089 7fa0c947e700 1 --2- 172.21.2.101:0/3332736267 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa0b0006800 0x7fa0b0008d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:25.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.089 7fa0c947e700 1 --2- 172.21.2.101:0/3332736267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa0b0009620 0x7fa0b000bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:25.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.089 7fa0c947e700 1 -- 172.21.2.101:0/3332736267 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b000c360 con 0x7fa0b0003ac0 2021-02-10T11:43:25.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.090 7fa0c947e700 1 -- 172.21.2.101:0/3332736267 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b000c580 con 0x7fa0b0006800 2021-02-10T11:43:25.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.090 7fa0c947e700 1 -- 172.21.2.101:0/3332736267 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b000c7a0 con 0x7fa0b0009620 2021-02-10T11:43:25.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.090 7fa0ca480700 1 --2- 172.21.2.101:0/3332736267 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0b0003ac0 0x7fa0b0005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:25.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.090 7fa0cac81700 1 --2- 172.21.2.101:0/3332736267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa0b0009620 0x7fa0b000bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:25.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.090 7fa0c9c7f700 1 --2- 172.21.2.101:0/3332736267 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa0b0006800 0x7fa0b0008d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:25.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.090 7fa0ca480700 1 -- 172.21.2.101:0/3332736267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa0b0009620 msgr2=0x7fa0b000bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:25.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.090 7fa0ca480700 1 --2- 172.21.2.101:0/3332736267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa0b0009620 0x7fa0b000bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:25.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.090 7fa0ca480700 1 -- 172.21.2.101:0/3332736267 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa0b0006800 msgr2=0x7fa0b0008d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:25.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.090 7fa0ca480700 1 --2- 172.21.2.101:0/3332736267 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa0b0006800 0x7fa0b0008d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:25.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.090 7fa0ca480700 1 -- 172.21.2.101:0/3332736267 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b00509a0 con 0x7fa0b0003ac0 2021-02-10T11:43:25.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.090 7fa0c9c7f700 1 --2- 172.21.2.101:0/3332736267 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa0b0006800 0x7fa0b0008d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:43:25.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.090 7fa0ca480700 1 --2- 172.21.2.101:0/3332736267 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0b0003ac0 0x7fa0b0005f60 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 rx=0x7fa0c0006ac0 tx=0x7fa0c0009a30).ready entity=mon.0 client_cookie=c95d28bae0c8d618 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:25.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.090 7fa0c947e700 1 -- 172.21.2.101:0/3332736267 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa0c000c0c0 con 0x7fa0b0003ac0 2021-02-10T11:43:25.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.090 7fa0c947e700 1 -- 172.21.2.101:0/3332736267 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c0009e20 con 0x7fa0b0003ac0 2021-02-10T11:43:25.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.090 7fa0c947e700 1 -- 172.21.2.101:0/3332736267 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa0c000c6c0 con 0x7fa0b0003ac0 2021-02-10T11:43:25.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.090 7fa0cc664700 1 -- 172.21.2.101:0/3332736267 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0b0003ac0 msgr2=0x7fa0b0005f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:25.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.090 7fa0cc664700 1 --2- 172.21.2.101:0/3332736267 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0b0003ac0 0x7fa0b0005f60 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 rx=0x7fa0c0006ac0 tx=0x7fa0c0009a30).stop 2021-02-10T11:43:25.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.090 7fa0cac81700 1 -- 172.21.2.101:0/3332736267 reap_dead start 2021-02-10T11:43:25.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.091 7fa0cc664700 1 -- 172.21.2.101:0/3332736267 shutdown_connections 2021-02-10T11:43:25.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.091 7fa0cc664700 1 -- 172.21.2.101:0/3332736267 >> 172.21.2.101:0/3332736267 conn(0x7fa0c40ce430 msgr2=0x7fa0c40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:25.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.091 7fa0cc664700 1 -- 172.21.2.101:0/3332736267 shutdown_connections 2021-02-10T11:43:25.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.091 7fa0cc664700 1 -- 172.21.2.101:0/3332736267 wait complete. 2021-02-10T11:43:25.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.106 7fa0cc664700 1 Processor -- start 2021-02-10T11:43:25.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.106 7fa0cc664700 1 -- start start 2021-02-10T11:43:25.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.106 7fa0cc664700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa0c41a0580 0x7fa0c4086fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:25.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.106 7fa0cc664700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa0c40b66b0 0x7fa0c40b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:25.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.106 7fa0cc664700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0c4082bb0 0x7fa0c4082fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:25.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.106 7fa0cc664700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40875d0 con 0x7fa0c4082bb0 2021-02-10T11:43:25.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.106 7fa0cc664700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40835d0 con 0x7fa0c40b66b0 2021-02-10T11:43:25.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.106 7fa0cc664700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40837f0 con 0x7fa0c41a0580 2021-02-10T11:43:25.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.106 7fa0cac81700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0c4082bb0 0x7fa0c4082fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:25.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.106 7fa0cac81700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0c4082bb0 0x7fa0c4082fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58972/0 (socket says 172.21.2.101:58972) 2021-02-10T11:43:25.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.106 7fa0cac81700 1 -- 172.21.2.101:0/2713485024 learned_addr learned my addr 172.21.2.101:0/2713485024 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:25.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.106 7fa0cac81700 1 -- 172.21.2.101:0/2713485024 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa0c41a0580 msgr2=0x7fa0c4086fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:25.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.106 7fa0cac81700 1 --2- 172.21.2.101:0/2713485024 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa0c41a0580 0x7fa0c4086fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:25.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.107 7fa0cac81700 1 -- 172.21.2.101:0/2713485024 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa0c40b66b0 msgr2=0x7fa0c40b6ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:25.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.107 7fa0cac81700 1 --2- 172.21.2.101:0/2713485024 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa0c40b66b0 0x7fa0c40b6ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:25.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.107 7fa0cac81700 1 -- 172.21.2.101:0/2713485024 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c4083a10 con 0x7fa0c4082bb0 2021-02-10T11:43:25.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.107 7fa0cac81700 1 --2- 172.21.2.101:0/2713485024 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0c4082bb0 0x7fa0c4082fb0 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 rx=0x7fa0bc006900 tx=0x7fa0bc00bd20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:25.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.107 7fa0baffd700 1 -- 172.21.2.101:0/2713485024 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa0bc006240 con 0x7fa0c4082bb0 2021-02-10T11:43:25.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.107 7fa0baffd700 1 -- 172.21.2.101:0/2713485024 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc00be60 con 0x7fa0c4082bb0 2021-02-10T11:43:25.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.107 7fa0baffd700 1 -- 172.21.2.101:0/2713485024 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa0bc0056a0 con 0x7fa0c4082bb0 2021-02-10T11:43:25.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.107 7fa0cc664700 1 -- 172.21.2.101:0/2713485024 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c4083a10 con 0x7fa0c4082bb0 2021-02-10T11:43:25.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.107 7fa0cc664700 1 -- 172.21.2.101:0/2713485024 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c4083fa0 con 0x7fa0c4082bb0 2021-02-10T11:43:25.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.107 7fa0baffd700 1 -- 172.21.2.101:0/2713485024 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa0bc006030 con 0x7fa0c4082bb0 2021-02-10T11:43:25.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.108 7fa0baffd700 1 --2- 172.21.2.101:0/2713485024 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa0ac0186c0 0x7fa0ac01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:25.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.108 7fa0baffd700 1 -- 172.21.2.101:0/2713485024 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa0bc0328a0 con 0x7fa0c4082bb0 2021-02-10T11:43:25.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.108 7fa0ca480700 1 --2- 172.21.2.101:0/2713485024 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa0ac0186c0 0x7fa0ac01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:25.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.108 7fa0ca480700 1 --2- 172.21.2.101:0/2713485024 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa0ac0186c0 0x7fa0ac01abc0 crc :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:25.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.138 7fa0cc664700 1 -- 172.21.2.101:0/2713485024 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c4082980 con 0x7fa0c4082bb0 2021-02-10T11:43:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.141 7fa0baffd700 1 -- 172.21.2.101:0/2713485024 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa0bc006a30 con 0x7fa0c4082bb0 2021-02-10T11:43:25.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.272 7fa0cc664700 1 -- 172.21.2.101:0/2713485024 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fa0c4082980 con 0x7fa0c4082bb0 2021-02-10T11:43:25.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.276 7fa0baffd700 1 -- 172.21.2.101:0/2713485024 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v81) v1 ==== 184+0+0 (secure 0 0 0) 0x7fa0bc01f8b0 con 0x7fa0c4082bb0 2021-02-10T11:43:25.280 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.2 2021-02-10T11:43:25.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.281 7fa0cc664700 1 -- 172.21.2.101:0/2713485024 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa0ac0186c0 msgr2=0x7fa0ac01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:25.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.281 7fa0cc664700 1 --2- 172.21.2.101:0/2713485024 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa0ac0186c0 0x7fa0ac01abc0 crc :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:25.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.281 7fa0cc664700 1 -- 172.21.2.101:0/2713485024 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0c4082bb0 msgr2=0x7fa0c4082fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:25.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.281 7fa0cc664700 1 --2- 172.21.2.101:0/2713485024 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0c4082bb0 0x7fa0c4082fb0 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 rx=0x7fa0bc006900 tx=0x7fa0bc00bd20).stop 2021-02-10T11:43:25.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.281 7fa0cc664700 1 -- 172.21.2.101:0/2713485024 shutdown_connections 2021-02-10T11:43:25.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.281 7fa0cc664700 1 --2- 172.21.2.101:0/2713485024 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa0c4082bb0 0x7fa0c4082fb0 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:25.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.281 7fa0cc664700 1 --2- 172.21.2.101:0/2713485024 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa0c40b66b0 0x7fa0c40b6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:25.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.281 7fa0cc664700 1 --2- 172.21.2.101:0/2713485024 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa0ac0186c0 0x7fa0ac01abc0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:25.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.281 7fa0cc664700 1 --2- 172.21.2.101:0/2713485024 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa0c41a0580 0x7fa0c4086fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:25.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.281 7fa0cc664700 1 -- 172.21.2.101:0/2713485024 >> 172.21.2.101:0/2713485024 conn(0x7fa0c40afc60 msgr2=0x7fa0c40b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:25.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.281 7fa0cc664700 1 -- 172.21.2.101:0/2713485024 shutdown_connections 2021-02-10T11:43:25.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.281 7fa0cc664700 1 -- 172.21.2.101:0/2713485024 wait complete. 2021-02-10T11:43:25.290 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:43:25.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.706 7fa166589700 1 Processor -- start 2021-02-10T11:43:25.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.706 7fa166589700 1 -- start start 2021-02-10T11:43:25.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.706 7fa166589700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa1600d1c80 0x7fa1600cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:25.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.707 7fa166589700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1600d0180 con 0x7fa1600d1880 2021-02-10T11:43:25.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.707 7fa166589700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1600fe1d0 con 0x7fa1600cf590 2021-02-10T11:43:25.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.707 7fa166589700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1601b0cc0 con 0x7fa1600d1c80 2021-02-10T11:43:25.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.707 7fa15ffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa1600d1880 0x7fa1601b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59474/0 (socket says 172.21.2.101:59474) 2021-02-10T11:43:25.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.707 7fa15ffff700 1 -- 172.21.2.101:0/96223555 learned_addr learned my addr 172.21.2.101:0/96223555 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:25.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.707 7fa15effd700 1 -- 172.21.2.101:0/96223555 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596529375 0 0) 0x7fa1500010b0 con 0x7fa1600d1880 2021-02-10T11:43:25.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.707 7fa15effd700 1 -- 172.21.2.101:0/96223555 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa148002fe0 con 0x7fa1600d1880 2021-02-10T11:43:25.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.707 7fa15f7fe700 1 --2- 172.21.2.101:0/96223555 >> v2:172.21.2.108:3300/0 conn(0x7fa1600d1c80 0x7fa1600cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:25.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.707 7fa15effd700 1 -- 172.21.2.101:0/96223555 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2530652406 0 0) 0x7fa1500010b0 con 0x7fa1600d1880 2021-02-10T11:43:25.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.707 7fa15effd700 1 -- 172.21.2.101:0/96223555 >> v2:172.21.2.108:3300/0 conn(0x7fa1600d1c80 msgr2=0x7fa1600cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:25.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.707 7fa15effd700 1 --2- 172.21.2.101:0/96223555 >> v2:172.21.2.108:3300/0 conn(0x7fa1600d1c80 0x7fa1600cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:25.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.707 7fa15effd700 1 -- 172.21.2.101:0/96223555 >> v1:172.21.2.108:6789/0 conn(0x7fa1600cf590 legacy=0x7fa1600cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:25.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.707 7fa15effd700 1 -- 172.21.2.101:0/96223555 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa160109a70 con 0x7fa1600d1880 2021-02-10T11:43:25.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.707 7fa15effd700 1 -- 172.21.2.101:0/96223555 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fa1500015b0 con 0x7fa1600d1880 2021-02-10T11:43:25.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.707 7fa15effd700 1 -- 172.21.2.101:0/96223555 >> v1:172.21.2.101:6789/0 conn(0x7fa1600d1880 legacy=0x7fa1601b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:25.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.707 7fa15effd700 1 --2- 172.21.2.101:0/96223555 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa148003a40 0x7fa148005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:25.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.707 7fa15effd700 1 --2- 172.21.2.101:0/96223555 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa148006790 0x7fa148008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:25.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.707 7fa15effd700 1 --2- 172.21.2.101:0/96223555 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa148009590 0x7fa14800bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:25.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.707 7fa15effd700 1 -- 172.21.2.101:0/96223555 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14800c300 con 0x7fa148006790 2021-02-10T11:43:25.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.707 7fa15effd700 1 -- 172.21.2.101:0/96223555 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14800c520 con 0x7fa148003a40 2021-02-10T11:43:25.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.708 7fa15effd700 1 -- 172.21.2.101:0/96223555 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa148050750 con 0x7fa148009590 2021-02-10T11:43:25.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.708 7fa15f7fe700 1 --2- 172.21.2.101:0/96223555 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa148006790 0x7fa148008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:25.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.708 7fa15ffff700 1 --2- 172.21.2.101:0/96223555 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa148003a40 0x7fa148005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:25.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.708 7fa164ba6700 1 --2- 172.21.2.101:0/96223555 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa148009590 0x7fa14800bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:25.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.708 7fa15f7fe700 1 -- 172.21.2.101:0/96223555 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa148009590 msgr2=0x7fa14800bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:25.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.708 7fa15f7fe700 1 --2- 172.21.2.101:0/96223555 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa148009590 0x7fa14800bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:25.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.708 7fa15f7fe700 1 -- 172.21.2.101:0/96223555 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa148003a40 msgr2=0x7fa148005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:25.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.708 7fa15f7fe700 1 --2- 172.21.2.101:0/96223555 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa148003a40 0x7fa148005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:25.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.708 7fa15f7fe700 1 -- 172.21.2.101:0/96223555 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1480509b0 con 0x7fa148006790 2021-02-10T11:43:25.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.708 7fa164ba6700 1 -- 172.21.2.101:0/96223555 reap_dead start 2021-02-10T11:43:25.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.708 7fa15f7fe700 1 --2- 172.21.2.101:0/96223555 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa148006790 0x7fa148008c90 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 rx=0x7fa15400c8f0 tx=0x7fa15400cc90).ready entity=mon.0 client_cookie=c15bd70ee39bec84 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:25.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.708 7fa15effd700 1 -- 172.21.2.101:0/96223555 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa154000c90 con 0x7fa148006790 2021-02-10T11:43:25.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.708 7fa15effd700 1 -- 172.21.2.101:0/96223555 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa154009310 con 0x7fa148006790 2021-02-10T11:43:25.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.708 7fa15effd700 1 -- 172.21.2.101:0/96223555 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa154009830 con 0x7fa148006790 2021-02-10T11:43:25.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.708 7fa166589700 1 -- 172.21.2.101:0/96223555 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa148006790 msgr2=0x7fa148008c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:25.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.708 7fa166589700 1 --2- 172.21.2.101:0/96223555 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa148006790 0x7fa148008c90 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 rx=0x7fa15400c8f0 tx=0x7fa15400cc90).stop 2021-02-10T11:43:25.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.709 7fa166589700 1 -- 172.21.2.101:0/96223555 shutdown_connections 2021-02-10T11:43:25.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.709 7fa166589700 1 --2- 172.21.2.101:0/96223555 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa148006790 0x7fa148008c90 unknown :-1 s=CLOSED pgs=2561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:25.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.709 7fa166589700 1 -- 172.21.2.101:0/96223555 >> 172.21.2.101:0/96223555 conn(0x7fa1600ce430 msgr2=0x7fa1600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:25.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.709 7fa166589700 1 -- 172.21.2.101:0/96223555 shutdown_connections 2021-02-10T11:43:25.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.709 7fa166589700 1 -- 172.21.2.101:0/96223555 wait complete. 2021-02-10T11:43:25.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.724 7fa166589700 1 Processor -- start 2021-02-10T11:43:25.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.724 7fa166589700 1 -- start start 2021-02-10T11:43:25.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.724 7fa166589700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa16018a3b0 0x7fa1601898c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:25.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.724 7fa166589700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa160189ee0 0x7fa160099c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:25.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.724 7fa166589700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa16009a290 0x7fa16009ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:25.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.724 7fa166589700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16009a810 con 0x7fa160189ee0 2021-02-10T11:43:25.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.724 7fa166589700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16009b3b0 con 0x7fa16018a3b0 2021-02-10T11:43:25.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.724 7fa166589700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16009b590 con 0x7fa16009a290 2021-02-10T11:43:25.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.724 7fa15f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa160189ee0 0x7fa160099c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:25.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.724 7fa15f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa160189ee0 0x7fa160099c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:58994/0 (socket says 172.21.2.101:58994) 2021-02-10T11:43:25.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.724 7fa15f7fe700 1 -- 172.21.2.101:0/1551499561 learned_addr learned my addr 172.21.2.101:0/1551499561 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:25.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.725 7fa15f7fe700 1 -- 172.21.2.101:0/1551499561 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa16009a290 msgr2=0x7fa16009ad90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:25.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.725 7fa15f7fe700 1 --2- 172.21.2.101:0/1551499561 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa16009a290 0x7fa16009ad90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:25.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.725 7fa15f7fe700 1 -- 172.21.2.101:0/1551499561 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa16018a3b0 msgr2=0x7fa1601898c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:25.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.725 7fa15f7fe700 1 --2- 172.21.2.101:0/1551499561 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa16018a3b0 0x7fa1601898c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:25.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.725 7fa15f7fe700 1 -- 172.21.2.101:0/1551499561 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16009b7b0 con 0x7fa160189ee0 2021-02-10T11:43:25.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.725 7fa15f7fe700 1 --2- 172.21.2.101:0/1551499561 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa160189ee0 0x7fa160099c70 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 rx=0x7fa154009df0 tx=0x7fa15400b300).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:25.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.725 7fa15cff9700 1 -- 172.21.2.101:0/1551499561 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa15400c4f0 con 0x7fa160189ee0 2021-02-10T11:43:25.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.725 7fa15cff9700 1 -- 172.21.2.101:0/1551499561 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15400c770 con 0x7fa160189ee0 2021-02-10T11:43:25.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.725 7fa166589700 1 -- 172.21.2.101:0/1551499561 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16009b7b0 con 0x7fa160189ee0 2021-02-10T11:43:25.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.725 7fa166589700 1 -- 172.21.2.101:0/1551499561 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16009bd40 con 0x7fa160189ee0 2021-02-10T11:43:25.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.725 7fa15cff9700 1 -- 172.21.2.101:0/1551499561 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa15400cc90 con 0x7fa160189ee0 2021-02-10T11:43:25.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.726 7fa15cff9700 1 -- 172.21.2.101:0/1551499561 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa154000c90 con 0x7fa160189ee0 2021-02-10T11:43:25.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.726 7fa15cff9700 1 --2- 172.21.2.101:0/1551499561 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1400186c0 0x7fa14001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:25.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.726 7fa15cff9700 1 -- 172.21.2.101:0/1551499561 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa1540325f0 con 0x7fa160189ee0 2021-02-10T11:43:25.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.726 7fa15ffff700 1 --2- 172.21.2.101:0/1551499561 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1400186c0 0x7fa14001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:25.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.726 7fa15ffff700 1 --2- 172.21.2.101:0/1551499561 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1400186c0 0x7fa14001abc0 crc :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:25.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.756 7fa166589700 1 -- 172.21.2.101:0/1551499561 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa160082990 con 0x7fa160189ee0 2021-02-10T11:43:25.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.759 7fa15cff9700 1 -- 172.21.2.101:0/1551499561 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa15400c140 con 0x7fa160189ee0 2021-02-10T11:43:25.892 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.892 7fa166589700 1 -- 172.21.2.101:0/1551499561 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fa160082990 con 0x7fa160189ee0 2021-02-10T11:43:25.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.896 7fa15cff9700 1 -- 172.21.2.101:0/1551499561 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v82) v1 ==== 184+0+0 (secure 0 0 0) 0x7fa154000c90 con 0x7fa160189ee0 2021-02-10T11:43:25.899 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.3 2021-02-10T11:43:25.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.901 7fa166589700 1 -- 172.21.2.101:0/1551499561 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1400186c0 msgr2=0x7fa14001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:25.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.901 7fa166589700 1 --2- 172.21.2.101:0/1551499561 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1400186c0 0x7fa14001abc0 crc :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:25.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.901 7fa166589700 1 -- 172.21.2.101:0/1551499561 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa160189ee0 msgr2=0x7fa160099c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:25.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.901 7fa166589700 1 --2- 172.21.2.101:0/1551499561 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa160189ee0 0x7fa160099c70 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 rx=0x7fa154009df0 tx=0x7fa15400b300).stop 2021-02-10T11:43:25.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.901 7fa166589700 1 -- 172.21.2.101:0/1551499561 shutdown_connections 2021-02-10T11:43:25.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.901 7fa166589700 1 --2- 172.21.2.101:0/1551499561 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1400186c0 0x7fa14001abc0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:25.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.901 7fa166589700 1 --2- 172.21.2.101:0/1551499561 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa16009a290 0x7fa16009ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:25.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.901 7fa166589700 1 --2- 172.21.2.101:0/1551499561 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa160189ee0 0x7fa160099c70 unknown :-1 s=CLOSED pgs=2562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:25.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.901 7fa166589700 1 --2- 172.21.2.101:0/1551499561 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa16018a3b0 0x7fa1601898c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:25.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.901 7fa166589700 1 -- 172.21.2.101:0/1551499561 >> 172.21.2.101:0/1551499561 conn(0x7fa160097440 msgr2=0x7fa1600aba10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:25.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.901 7fa166589700 1 -- 172.21.2.101:0/1551499561 shutdown_connections 2021-02-10T11:43:25.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:25.901 7fa166589700 1 -- 172.21.2.101:0/1551499561 wait complete. 2021-02-10T11:43:25.910 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:43:26.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.321 7fe878fe9700 1 Processor -- start 2021-02-10T11:43:26.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.321 7fe878fe9700 1 -- start start 2021-02-10T11:43:26.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.322 7fe878fe9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8740d1870 0x7fe8741b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:26.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.322 7fe878fe9700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe8740f0650 0x7fe8740cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:26.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.322 7fe878fe9700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8740f0b10 con 0x7fe8741b0a10 2021-02-10T11:43:26.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.322 7fe878fe9700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8740fe230 con 0x7fe8740f0650 2021-02-10T11:43:26.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.322 7fe878fe9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8740e5ed0 con 0x7fe8740d1870 2021-02-10T11:43:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.322 7fe87359e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe8740f0650 0x7fe8740cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.322 7fe87359e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe8740f0650 0x7fe8740cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59000/0 (socket says 172.21.2.101:59000) 2021-02-10T11:43:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.322 7fe87359e700 1 -- 172.21.2.101:0/3872235380 learned_addr learned my addr 172.21.2.101:0/3872235380 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.322 7fe87359e700 1 -- 172.21.2.101:0/3872235380 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8740d1870 msgr2=0x7fe8741b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:43:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.322 7fe87359e700 1 --2- 172.21.2.101:0/3872235380 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8740d1870 0x7fe8741b03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.322 7fe87359e700 1 -- 172.21.2.101:0/3872235380 >> v1:172.21.2.108:6789/0 conn(0x7fe8741b0a10 legacy=0x7fe8740cf330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:43:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.322 7fe87359e700 1 -- 172.21.2.101:0/3872235380 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8740f20a0 con 0x7fe8740f0650 2021-02-10T11:43:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.322 7fe87359e700 1 --2- 172.21.2.101:0/3872235380 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8740f0650 0x7fe8740cfb20 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 rx=0x7fe86c006f00 tx=0x7fe86c009f10).ready entity=mon.0 client_cookie=15063e55c5b45e14 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.323 7fe871d9b700 1 -- 172.21.2.101:0/3872235380 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe86c00b3b0 con 0x7fe8740f0650 2021-02-10T11:43:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.323 7fe871d9b700 1 -- 172.21.2.101:0/3872235380 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86c00b730 con 0x7fe8740f0650 2021-02-10T11:43:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.323 7fe871d9b700 1 -- 172.21.2.101:0/3872235380 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe86c00bc50 con 0x7fe8740f0650 2021-02-10T11:43:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.323 7fe878fe9700 1 -- 172.21.2.101:0/3872235380 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8740f0650 msgr2=0x7fe8740cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:26.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.323 7fe878fe9700 1 --2- 172.21.2.101:0/3872235380 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8740f0650 0x7fe8740cfb20 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 rx=0x7fe86c006f00 tx=0x7fe86c009f10).stop 2021-02-10T11:43:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.323 7fe878fe9700 1 -- 172.21.2.101:0/3872235380 shutdown_connections 2021-02-10T11:43:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.323 7fe878fe9700 1 --2- 172.21.2.101:0/3872235380 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8740f0650 0x7fe8740cfb20 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.323 7fe878fe9700 1 --2- 172.21.2.101:0/3872235380 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8740d1870 0x7fe8741b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.324 7fe878fe9700 1 -- 172.21.2.101:0/3872235380 >> 172.21.2.101:0/3872235380 conn(0x7fe8740ce450 msgr2=0x7fe8740d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.324 7fe878fe9700 1 -- 172.21.2.101:0/3872235380 shutdown_connections 2021-02-10T11:43:26.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.324 7fe878fe9700 1 -- 172.21.2.101:0/3872235380 wait complete. 2021-02-10T11:43:26.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.338 7fe878fe9700 1 Processor -- start 2021-02-10T11:43:26.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.338 7fe878fe9700 1 -- start start 2021-02-10T11:43:26.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.338 7fe878fe9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8740b4f30 0x7fe87418b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:26.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.338 7fe878fe9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe874093f40 0x7fe874094340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:26.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.338 7fe878fe9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe874094960 0x7fe874187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:26.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.338 7fe878fe9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe874094e90 con 0x7fe874093f40 2021-02-10T11:43:26.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.338 7fe878fe9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87418bff0 con 0x7fe874094960 2021-02-10T11:43:26.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.338 7fe878fe9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87408f780 con 0x7fe8740b4f30 2021-02-10T11:43:26.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.338 7fe87259c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe874093f40 0x7fe874094340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:26.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.339 7fe87259c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe874093f40 0x7fe874094340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59004/0 (socket says 172.21.2.101:59004) 2021-02-10T11:43:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.339 7fe87259c700 1 -- 172.21.2.101:0/1927335733 learned_addr learned my addr 172.21.2.101:0/1927335733 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.339 7fe87259c700 1 -- 172.21.2.101:0/1927335733 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8740b4f30 msgr2=0x7fe87418b9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.339 7fe87259c700 1 --2- 172.21.2.101:0/1927335733 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8740b4f30 0x7fe87418b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.339 7fe87259c700 1 -- 172.21.2.101:0/1927335733 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe874094960 msgr2=0x7fe874187e50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.339 7fe87259c700 1 --2- 172.21.2.101:0/1927335733 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe874094960 0x7fe874187e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.339 7fe87259c700 1 -- 172.21.2.101:0/1927335733 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe874188470 con 0x7fe874093f40 2021-02-10T11:43:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.339 7fe87259c700 1 --2- 172.21.2.101:0/1927335733 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe874093f40 0x7fe874094340 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 rx=0x7fe85c00b5b0 tx=0x7fe85c00b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.339 7fe86b7fe700 1 -- 172.21.2.101:0/1927335733 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe85c00bd40 con 0x7fe874093f40 2021-02-10T11:43:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.339 7fe86b7fe700 1 -- 172.21.2.101:0/1927335733 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85c006020 con 0x7fe874093f40 2021-02-10T11:43:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.339 7fe86b7fe700 1 -- 172.21.2.101:0/1927335733 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe85c0054a0 con 0x7fe874093f40 2021-02-10T11:43:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.339 7fe878fe9700 1 -- 172.21.2.101:0/1927335733 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe874188470 con 0x7fe874093f40 2021-02-10T11:43:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.339 7fe878fe9700 1 -- 172.21.2.101:0/1927335733 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe874188470 con 0x7fe874093f40 2021-02-10T11:43:26.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.340 7fe86b7fe700 1 -- 172.21.2.101:0/1927335733 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe85c006200 con 0x7fe874093f40 2021-02-10T11:43:26.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.340 7fe86b7fe700 1 --2- 172.21.2.101:0/1927335733 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8580186c0 0x7fe85801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:26.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.340 7fe86b7fe700 1 -- 172.21.2.101:0/1927335733 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe85c032450 con 0x7fe874093f40 2021-02-10T11:43:26.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.340 7fe872d9d700 1 --2- 172.21.2.101:0/1927335733 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8580186c0 0x7fe85801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:26.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.340 7fe872d9d700 1 --2- 172.21.2.101:0/1927335733 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8580186c0 0x7fe85801abc0 crc :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:26.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.372 7fe878fe9700 1 -- 172.21.2.101:0/1927335733 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe87418bff0 con 0x7fe874093f40 2021-02-10T11:43:26.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.374 7fe86b7fe700 1 -- 172.21.2.101:0/1927335733 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe85c025ed0 con 0x7fe874093f40 2021-02-10T11:43:26.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.503 7fe878fe9700 1 -- 172.21.2.101:0/1927335733 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe87418bff0 con 0x7fe874093f40 2021-02-10T11:43:26.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.503 7fe86b7fe700 1 -- 172.21.2.101:0/1927335733 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+2102 (secure 0 0 0) 0x7fe85c025ed0 con 0x7fe874093f40 2021-02-10T11:43:26.504 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 114 2021-02-10T11:43:26.504 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:43:26.504 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":114,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/371124351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":371124351},{"type":"v1","addr":"172.21.2.101:6835","nonce":371124351}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:43:20.159141","modified":"2021-02-10 11:43:22.169386","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7990":{"gid":7990,"name":"b","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/797061093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":797061093},{"type":"v1","addr":"172.21.2.108:6833","nonce":797061093}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-02-10T11:43:26.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.505 7fe878fe9700 1 -- 172.21.2.101:0/1927335733 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8580186c0 msgr2=0x7fe85801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:26.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.505 7fe878fe9700 1 --2- 172.21.2.101:0/1927335733 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8580186c0 0x7fe85801abc0 crc :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:26.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.505 7fe878fe9700 1 -- 172.21.2.101:0/1927335733 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe874093f40 msgr2=0x7fe874094340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:26.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.505 7fe878fe9700 1 --2- 172.21.2.101:0/1927335733 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe874093f40 0x7fe874094340 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 rx=0x7fe85c00b5b0 tx=0x7fe85c00b950).stop 2021-02-10T11:43:26.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.505 7fe878fe9700 1 -- 172.21.2.101:0/1927335733 shutdown_connections 2021-02-10T11:43:26.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.505 7fe878fe9700 1 --2- 172.21.2.101:0/1927335733 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe874094960 0x7fe874187e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:26.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.505 7fe878fe9700 1 --2- 172.21.2.101:0/1927335733 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe874093f40 0x7fe874094340 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:26.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.505 7fe878fe9700 1 --2- 172.21.2.101:0/1927335733 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8580186c0 0x7fe85801abc0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:26.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.505 7fe878fe9700 1 --2- 172.21.2.101:0/1927335733 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8740b4f30 0x7fe87418b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:26.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.505 7fe878fe9700 1 -- 172.21.2.101:0/1927335733 >> 172.21.2.101:0/1927335733 conn(0x7fe87419f8f0 msgr2=0x7fe8741a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:26.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.505 7fe878fe9700 1 -- 172.21.2.101:0/1927335733 shutdown_connections 2021-02-10T11:43:26.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:26.506 7fe878fe9700 1 -- 172.21.2.101:0/1927335733 wait complete. 2021-02-10T11:43:26.715 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 114, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:43:20.159141', 'modified': '2021-02-10 11:43:22.169386', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7990}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7990': {'gid': 7990, 'name': 'b', 'rank': 0, 'incarnation': 111, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/797061093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 797061093}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 797061093}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [29], 'metadata_pool': 28, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:43:26.716 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:43:26.716 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:43:27.115 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:43:27.115 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:43:27.115 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:43:27.116 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 7990, 2021-02-10T11:43:27.116 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:43:27.116 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:43:27.116 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 6.9533360310000001, 2021-02-10T11:43:27.116 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 114, 2021-02-10T11:43:27.116 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 138, 2021-02-10T11:43:27.116 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 138, 2021-02-10T11:43:27.116 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 12.586011134 2021-02-10T11:43:27.116 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:43:27.119 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 7990, "want_state": "up:active", "state": "up:active", "rank_uptime": 6.9533360310000001, "mdsmap_epoch": 114, "osdmap_epoch": 138, "osdmap_epoch_barrier": 138, "uptime": 12.586011134 } 2021-02-10T11:43:27.120 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:43:27.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.532 7fb5246fe700 1 Processor -- start 2021-02-10T11:43:27.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.532 7fb5246fe700 1 -- start start 2021-02-10T11:43:27.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.532 7fb5246fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb51c0d1870 0x7fb51c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:27.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.532 7fb5246fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb51c1b0a10 0x7fb51c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:27.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.532 7fb5246fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb51c0d1c70 0x7fb51c0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:27.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.532 7fb5246fe700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb51c0fe200 con 0x7fb51c0d1870 2021-02-10T11:43:27.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.532 7fb5246fe700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb51c0e5f10 con 0x7fb51c0d1c70 2021-02-10T11:43:27.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.532 7fb5246fe700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c0f0b50 con 0x7fb51c1b0a10 2021-02-10T11:43:27.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.533 7fb52251a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb51c0d1870 0x7fb51c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:27.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.533 7fb52251a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb51c0d1870 0x7fb51c1b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59012/0 (socket says 172.21.2.101:59012) 2021-02-10T11:43:27.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.533 7fb52251a700 1 -- 172.21.2.101:0/2941016929 learned_addr learned my addr 172.21.2.101:0/2941016929 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:27.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.533 7fb52251a700 1 -- 172.21.2.101:0/2941016929 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb51c1b0a10 msgr2=0x7fb51c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:27.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.533 7fb52251a700 1 --2- 172.21.2.101:0/2941016929 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb51c1b0a10 0x7fb51c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:27.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.533 7fb52251a700 1 -- 172.21.2.101:0/2941016929 >> v2:172.21.2.108:3300/0 conn(0x7fb51c0d1c70 msgr2=0x7fb51c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:27.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.533 7fb52251a700 1 --2- 172.21.2.101:0/2941016929 >> v2:172.21.2.108:3300/0 conn(0x7fb51c0d1c70 0x7fb51c0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:27.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.533 7fb52251a700 1 -- 172.21.2.101:0/2941016929 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c0cff70 con 0x7fb51c0d1870 2021-02-10T11:43:27.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.533 7fb52251a700 1 --2- 172.21.2.101:0/2941016929 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb51c0d1870 0x7fb51c1b03f0 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 rx=0x7fb518006700 tx=0x7fb518009b80).ready entity=mon.0 client_cookie=2c3da22853cc49e3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:27.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.533 7fb521518700 1 -- 172.21.2.101:0/2941016929 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb51800c0c0 con 0x7fb51c0d1870 2021-02-10T11:43:27.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.533 7fb521518700 1 -- 172.21.2.101:0/2941016929 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51800c340 con 0x7fb51c0d1870 2021-02-10T11:43:27.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.533 7fb521518700 1 -- 172.21.2.101:0/2941016929 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb51800c860 con 0x7fb51c0d1870 2021-02-10T11:43:27.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.533 7fb5246fe700 1 -- 172.21.2.101:0/2941016929 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb51c0d1870 msgr2=0x7fb51c1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:27.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.533 7fb5246fe700 1 --2- 172.21.2.101:0/2941016929 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb51c0d1870 0x7fb51c1b03f0 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 rx=0x7fb518006700 tx=0x7fb518009b80).stop 2021-02-10T11:43:27.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.534 7fb5246fe700 1 -- 172.21.2.101:0/2941016929 shutdown_connections 2021-02-10T11:43:27.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.534 7fb5246fe700 1 --2- 172.21.2.101:0/2941016929 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb51c1b0a10 0x7fb51c0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:27.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.534 7fb5246fe700 1 --2- 172.21.2.101:0/2941016929 >> v2:172.21.2.108:3300/0 conn(0x7fb51c0d1c70 0x7fb51c0cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:27.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.534 7fb5246fe700 1 --2- 172.21.2.101:0/2941016929 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb51c0d1870 0x7fb51c1b03f0 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:27.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.534 7fb5246fe700 1 -- 172.21.2.101:0/2941016929 >> 172.21.2.101:0/2941016929 conn(0x7fb51c0ce450 msgr2=0x7fb51c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:27.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.534 7fb5246fe700 1 -- 172.21.2.101:0/2941016929 shutdown_connections 2021-02-10T11:43:27.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.534 7fb5246fe700 1 -- 172.21.2.101:0/2941016929 wait complete. 2021-02-10T11:43:27.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.549 7fb5246fe700 1 Processor -- start 2021-02-10T11:43:27.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.549 7fb5246fe700 1 -- start start 2021-02-10T11:43:27.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.549 7fb5246fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb51c1971e0 0x7fb51c18eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:27.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.549 7fb5246fe700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb51c1b8180 0x7fb51c1b8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:27.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.549 7fb5246fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb51c1b8ba0 0x7fb51c0b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:27.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.549 7fb5246fe700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c1b9750 con 0x7fb51c1971e0 2021-02-10T11:43:27.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.549 7fb5246fe700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c18e930 con 0x7fb51c1b8180 2021-02-10T11:43:27.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.549 7fb5246fe700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c0b1dd0 con 0x7fb51c1b8ba0 2021-02-10T11:43:27.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.549 7fb52251a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb51c1971e0 0x7fb51c18eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:27.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.549 7fb52251a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb51c1971e0 0x7fb51c18eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59016/0 (socket says 172.21.2.101:59016) 2021-02-10T11:43:27.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.549 7fb52251a700 1 -- 172.21.2.101:0/4190333734 learned_addr learned my addr 172.21.2.101:0/4190333734 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:27.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.550 7fb52251a700 1 -- 172.21.2.101:0/4190333734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb51c1b8ba0 msgr2=0x7fb51c0b17b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:27.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.550 7fb52251a700 1 --2- 172.21.2.101:0/4190333734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb51c1b8ba0 0x7fb51c0b17b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:27.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.550 7fb52251a700 1 -- 172.21.2.101:0/4190333734 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb51c1b8180 msgr2=0x7fb51c1b8580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:27.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.550 7fb52251a700 1 --2- 172.21.2.101:0/4190333734 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb51c1b8180 0x7fb51c1b8580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:27.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.550 7fb52251a700 1 -- 172.21.2.101:0/4190333734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c0b1ff0 con 0x7fb51c1971e0 2021-02-10T11:43:27.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.550 7fb52251a700 1 --2- 172.21.2.101:0/4190333734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb51c1971e0 0x7fb51c18eda0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 rx=0x7fb51800b650 tx=0x7fb51800b9f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:27.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.550 7fb50effd700 1 -- 172.21.2.101:0/4190333734 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb51800bde0 con 0x7fb51c1971e0 2021-02-10T11:43:27.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.550 7fb50effd700 1 -- 172.21.2.101:0/4190333734 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb518002030 con 0x7fb51c1971e0 2021-02-10T11:43:27.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.550 7fb50effd700 1 -- 172.21.2.101:0/4190333734 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb518002550 con 0x7fb51c1971e0 2021-02-10T11:43:27.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.550 7fb5246fe700 1 -- 172.21.2.101:0/4190333734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51c0b1ff0 con 0x7fb51c1971e0 2021-02-10T11:43:27.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.550 7fb5246fe700 1 -- 172.21.2.101:0/4190333734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51c0b1ff0 con 0x7fb51c1971e0 2021-02-10T11:43:27.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.551 7fb50effd700 1 -- 172.21.2.101:0/4190333734 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb518000c90 con 0x7fb51c1971e0 2021-02-10T11:43:27.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.551 7fb50effd700 1 --2- 172.21.2.101:0/4190333734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5040186c0 0x7fb50401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:27.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.551 7fb50effd700 1 -- 172.21.2.101:0/4190333734 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb518032560 con 0x7fb51c1971e0 2021-02-10T11:43:27.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.551 7fb521d19700 1 --2- 172.21.2.101:0/4190333734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5040186c0 0x7fb50401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:27.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.551 7fb521d19700 1 --2- 172.21.2.101:0/4190333734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5040186c0 0x7fb50401abc0 crc :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:27.582 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.582 7fb5246fe700 1 -- 172.21.2.101:0/4190333734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb51c0ee060 con 0x7fb51c1971e0 2021-02-10T11:43:27.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.585 7fb50effd700 1 -- 172.21.2.101:0/4190333734 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb518030d10 con 0x7fb51c1971e0 2021-02-10T11:43:27.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.709 7fb5246fe700 1 -- 172.21.2.101:0/4190333734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb51c0ee060 con 0x7fb51c1971e0 2021-02-10T11:43:27.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.710 7fb50effd700 1 -- 172.21.2.101:0/4190333734 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+14007 (secure 0 0 0) 0x7fb518000c90 con 0x7fb51c1971e0 2021-02-10T11:43:27.711 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:43:27.711 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":138,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:43:21.161685","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:43:18.076350","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2021-02-10 11:43:19.152025","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:43:27.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.712 7fb5246fe700 1 -- 172.21.2.101:0/4190333734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5040186c0 msgr2=0x7fb50401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:27.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.712 7fb5246fe700 1 --2- 172.21.2.101:0/4190333734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5040186c0 0x7fb50401abc0 crc :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:27.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.712 7fb5246fe700 1 -- 172.21.2.101:0/4190333734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb51c1971e0 msgr2=0x7fb51c18eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:27.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.712 7fb5246fe700 1 --2- 172.21.2.101:0/4190333734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb51c1971e0 0x7fb51c18eda0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 rx=0x7fb51800b650 tx=0x7fb51800b9f0).stop 2021-02-10T11:43:27.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.712 7fb5246fe700 1 -- 172.21.2.101:0/4190333734 shutdown_connections 2021-02-10T11:43:27.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.712 7fb5246fe700 1 --2- 172.21.2.101:0/4190333734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5040186c0 0x7fb50401abc0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:27.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.712 7fb5246fe700 1 --2- 172.21.2.101:0/4190333734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb51c1b8ba0 0x7fb51c0b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:27.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.712 7fb5246fe700 1 --2- 172.21.2.101:0/4190333734 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb51c1b8180 0x7fb51c1b8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:27.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.712 7fb5246fe700 1 --2- 172.21.2.101:0/4190333734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb51c1971e0 0x7fb51c18eda0 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:27.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.712 7fb5246fe700 1 -- 172.21.2.101:0/4190333734 >> 172.21.2.101:0/4190333734 conn(0x7fb51c19f8f0 msgr2=0x7fb51c18c260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:27.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.712 7fb5246fe700 1 -- 172.21.2.101:0/4190333734 shutdown_connections 2021-02-10T11:43:27.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:27.712 7fb5246fe700 1 -- 172.21.2.101:0/4190333734 wait complete. 2021-02-10T11:43:27.721 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 28, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:43:18.076350', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 29, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:43:19.152025', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:43:27.721 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:43:28.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.136 7f785434a700 1 Processor -- start 2021-02-10T11:43:28.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.136 7f785434a700 1 -- start start 2021-02-10T11:43:28.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.137 7f785434a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f784c0d1860 0x7f784c1b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:28.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.137 7f785434a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f784c1b0ad0 0x7f784c108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:28.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.137 7f785434a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f784c0fe1b0 con 0x7f784c0d1c60 2021-02-10T11:43:28.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.137 7f785434a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f784c109a50 con 0x7f784c1b0ad0 2021-02-10T11:43:28.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.137 7f785434a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f784c109740 con 0x7f784c0d1860 2021-02-10T11:43:28.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.137 7f7851965700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f784c1b0ad0 0x7f784c108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:28.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.137 7f7851965700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f784c1b0ad0 0x7f784c108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59024/0 (socket says 172.21.2.101:59024) 2021-02-10T11:43:28.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.137 7f7851965700 1 -- 172.21.2.101:0/519009502 learned_addr learned my addr 172.21.2.101:0/519009502 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:28.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.137 7f7851965700 1 -- 172.21.2.101:0/519009502 >> v2:172.21.2.108:3300/0 conn(0x7f784c0d1860 msgr2=0x7f784c1b04b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:43:28.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.137 7f7851965700 1 --2- 172.21.2.101:0/519009502 >> v2:172.21.2.108:3300/0 conn(0x7f784c0d1860 0x7f784c1b04b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:28.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.137 7f7851965700 1 -- 172.21.2.101:0/519009502 >> v1:172.21.2.108:6789/0 conn(0x7f784c0d1c60 legacy=0x7f784c108f50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:43:28.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.137 7f7851965700 1 -- 172.21.2.101:0/519009502 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784c088070 con 0x7f784c1b0ad0 2021-02-10T11:43:28.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.137 7f7851965700 1 --2- 172.21.2.101:0/519009502 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f784c1b0ad0 0x7f784c108930 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 rx=0x7f7848006700 tx=0x7f7848009b00).ready entity=mon.0 client_cookie=a11b0a0be7f77c44 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:28.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.137 7f7851164700 1 -- 172.21.2.101:0/519009502 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f784800c0c0 con 0x7f784c1b0ad0 2021-02-10T11:43:28.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.138 7f7851164700 1 -- 172.21.2.101:0/519009502 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784800c340 con 0x7f784c1b0ad0 2021-02-10T11:43:28.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.138 7f7851164700 1 -- 172.21.2.101:0/519009502 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f784800c820 con 0x7f784c1b0ad0 2021-02-10T11:43:28.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.138 7f785434a700 1 -- 172.21.2.101:0/519009502 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f784c1b0ad0 msgr2=0x7f784c108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:28.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.138 7f785434a700 1 --2- 172.21.2.101:0/519009502 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f784c1b0ad0 0x7f784c108930 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 rx=0x7f7848006700 tx=0x7f7848009b00).stop 2021-02-10T11:43:28.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.138 7f785434a700 1 -- 172.21.2.101:0/519009502 shutdown_connections 2021-02-10T11:43:28.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.138 7f785434a700 1 --2- 172.21.2.101:0/519009502 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f784c1b0ad0 0x7f784c108930 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:28.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.138 7f785434a700 1 --2- 172.21.2.101:0/519009502 >> v2:172.21.2.108:3300/0 conn(0x7f784c0d1860 0x7f784c1b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:28.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.138 7f785434a700 1 -- 172.21.2.101:0/519009502 >> 172.21.2.101:0/519009502 conn(0x7f784c0ce410 msgr2=0x7f784c0d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:28.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.138 7f785434a700 1 -- 172.21.2.101:0/519009502 shutdown_connections 2021-02-10T11:43:28.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.138 7f785434a700 1 -- 172.21.2.101:0/519009502 wait complete. 2021-02-10T11:43:28.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.152 7f785434a700 1 Processor -- start 2021-02-10T11:43:28.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.153 7f785434a700 1 -- start start 2021-02-10T11:43:28.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.153 7f785434a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f784c0bfa10 0x7f784c0b82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:28.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.153 7f785434a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f784c1a4a70 0x7f784c094c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:28.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.153 7f785434a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f784c0c0410 0x7f784c0c0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:28.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.153 7f785434a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c0c3a50 con 0x7f784c1a4a70 2021-02-10T11:43:28.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.153 7f785434a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c0b88c0 con 0x7f784c0bfa10 2021-02-10T11:43:28.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.153 7f785434a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784c095260 con 0x7f784c0c0410 2021-02-10T11:43:28.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.153 7f7851965700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f784c1a4a70 0x7f784c094c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:28.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.153 7f7851965700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f784c1a4a70 0x7f784c094c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59028/0 (socket says 172.21.2.101:59028) 2021-02-10T11:43:28.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.153 7f7851965700 1 -- 172.21.2.101:0/3681168748 learned_addr learned my addr 172.21.2.101:0/3681168748 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:28.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.153 7f7852967700 1 --2- 172.21.2.101:0/3681168748 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f784c0c0410 0x7f784c0c0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:28.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.153 7f7852166700 1 --2- 172.21.2.101:0/3681168748 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f784c0bfa10 0x7f784c0b82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:28.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.154 7f7851965700 1 -- 172.21.2.101:0/3681168748 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f784c0c0410 msgr2=0x7f784c0c0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:28.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.154 7f7851965700 1 --2- 172.21.2.101:0/3681168748 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f784c0c0410 0x7f784c0c0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:28.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.154 7f7851965700 1 -- 172.21.2.101:0/3681168748 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f784c0bfa10 msgr2=0x7f784c0b82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:28.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.154 7f7851965700 1 --2- 172.21.2.101:0/3681168748 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f784c0bfa10 0x7f784c0b82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:28.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.154 7f7851965700 1 -- 172.21.2.101:0/3681168748 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784c0c3540 con 0x7f784c1a4a70 2021-02-10T11:43:28.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.154 7f7852967700 1 --2- 172.21.2.101:0/3681168748 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f784c0c0410 0x7f784c0c0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:43:28.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.154 7f7852166700 1 --2- 172.21.2.101:0/3681168748 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f784c0bfa10 0x7f784c0b82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:43:28.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.154 7f7851965700 1 --2- 172.21.2.101:0/3681168748 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f784c1a4a70 0x7f784c094c40 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 rx=0x7f7848006f20 tx=0x7f784800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:28.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.154 7f783effd700 1 -- 172.21.2.101:0/3681168748 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f784800bd80 con 0x7f784c1a4a70 2021-02-10T11:43:28.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.154 7f783effd700 1 -- 172.21.2.101:0/3681168748 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7848000c90 con 0x7f784c1a4a70 2021-02-10T11:43:28.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.154 7f783effd700 1 -- 172.21.2.101:0/3681168748 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f78480022b0 con 0x7f784c1a4a70 2021-02-10T11:43:28.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.154 7f785434a700 1 -- 172.21.2.101:0/3681168748 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f784c0c3540 con 0x7f784c1a4a70 2021-02-10T11:43:28.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.154 7f785434a700 1 -- 172.21.2.101:0/3681168748 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f784c0c1120 con 0x7f784c1a4a70 2021-02-10T11:43:28.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.155 7f783effd700 1 -- 172.21.2.101:0/3681168748 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7848002020 con 0x7f784c1a4a70 2021-02-10T11:43:28.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.155 7f783effd700 1 --2- 172.21.2.101:0/3681168748 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f78340186c0 0x7f783401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:28.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.155 7f783effd700 1 -- 172.21.2.101:0/3681168748 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f78480325e0 con 0x7f784c1a4a70 2021-02-10T11:43:28.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.155 7f7852166700 1 --2- 172.21.2.101:0/3681168748 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f78340186c0 0x7f783401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:28.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.155 7f7852166700 1 --2- 172.21.2.101:0/3681168748 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f78340186c0 0x7f783401abc0 crc :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:28.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.187 7f785434a700 1 -- 172.21.2.101:0/3681168748 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f784c0bfee0 con 0x7f784c1a4a70 2021-02-10T11:43:28.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.189 7f783effd700 1 -- 172.21.2.101:0/3681168748 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7848006390 con 0x7f784c1a4a70 2021-02-10T11:43:28.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.318 7f785434a700 1 -- 172.21.2.101:0/3681168748 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f784c0bfee0 con 0x7f784c1a4a70 2021-02-10T11:43:28.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.318 7f783effd700 1 -- 172.21.2.101:0/3681168748 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v138) v1 ==== 92+0+10 (secure 0 0 0) 0x7f784800bd80 con 0x7f784c1a4a70 2021-02-10T11:43:28.319 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:43:28.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.320 7f785434a700 1 -- 172.21.2.101:0/3681168748 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f78340186c0 msgr2=0x7f783401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:28.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.320 7f785434a700 1 --2- 172.21.2.101:0/3681168748 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f78340186c0 0x7f783401abc0 crc :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:28.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.320 7f785434a700 1 -- 172.21.2.101:0/3681168748 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f784c1a4a70 msgr2=0x7f784c094c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:28.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.320 7f785434a700 1 --2- 172.21.2.101:0/3681168748 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f784c1a4a70 0x7f784c094c40 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 rx=0x7f7848006f20 tx=0x7f784800b990).stop 2021-02-10T11:43:28.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.320 7f785434a700 1 -- 172.21.2.101:0/3681168748 shutdown_connections 2021-02-10T11:43:28.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.320 7f785434a700 1 --2- 172.21.2.101:0/3681168748 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f78340186c0 0x7f783401abc0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:28.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.321 7f785434a700 1 --2- 172.21.2.101:0/3681168748 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f784c0c0410 0x7f784c0c0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:28.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.321 7f785434a700 1 --2- 172.21.2.101:0/3681168748 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f784c1a4a70 0x7f784c094c40 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:28.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.321 7f785434a700 1 --2- 172.21.2.101:0/3681168748 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f784c0bfa10 0x7f784c0b82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:28.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.321 7f785434a700 1 -- 172.21.2.101:0/3681168748 >> 172.21.2.101:0/3681168748 conn(0x7f784c0a6a80 msgr2=0x7f784c0ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:28.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.321 7f785434a700 1 -- 172.21.2.101:0/3681168748 shutdown_connections 2021-02-10T11:43:28.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.321 7f785434a700 1 -- 172.21.2.101:0/3681168748 wait complete. 2021-02-10T11:43:28.329 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:43:28.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.744 7f0e8d610700 1 Processor -- start 2021-02-10T11:43:28.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.744 7f0e8d610700 1 -- start start 2021-02-10T11:43:28.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.745 7f0e8d610700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0e880d1870 0x7f0e881b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:28.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.745 7f0e8d610700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0e880cff70 0x7f0e88109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:28.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.745 7f0e8d610700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e880fe1c0 con 0x7f0e881b0ae0 2021-02-10T11:43:28.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.745 7f0e8d610700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e88109ad0 con 0x7f0e880cff70 2021-02-10T11:43:28.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.745 7f0e8d610700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e88109750 con 0x7f0e880d1870 2021-02-10T11:43:28.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.745 7f0e87ac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0e880cff70 0x7f0e88109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:28.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.745 7f0e86abe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f0e881b0ae0 0x7f0e88108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59530/0 (socket says 172.21.2.101:59530) 2021-02-10T11:43:28.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.745 7f0e86abe700 1 -- 172.21.2.101:0/1940195820 learned_addr learned my addr 172.21.2.101:0/1940195820 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:28.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.745 7f0e87ac0700 1 -- 172.21.2.101:0/1940195820 >> v2:172.21.2.108:3300/0 conn(0x7f0e880d1870 msgr2=0x7f0e881b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:43:28.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.745 7f0e87ac0700 1 --2- 172.21.2.101:0/1940195820 >> v2:172.21.2.108:3300/0 conn(0x7f0e880d1870 0x7f0e881b04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:28.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.745 7f0e87ac0700 1 -- 172.21.2.101:0/1940195820 >> v1:172.21.2.101:6789/0 conn(0x7f0e881b0ae0 legacy=0x7f0e88108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:28.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.745 7f0e87ac0700 1 -- 172.21.2.101:0/1940195820 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e88088070 con 0x7f0e880cff70 2021-02-10T11:43:28.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.745 7f0e87ac0700 1 --2- 172.21.2.101:0/1940195820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e880cff70 0x7f0e88109130 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 rx=0x7f0e80006e30 tx=0x7f0e80009e60).ready entity=mon.0 client_cookie=cf41d78c1a58578d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:28.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.745 7f0e862bd700 1 -- 172.21.2.101:0/1940195820 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e80006a50 con 0x7f0e880cff70 2021-02-10T11:43:28.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.745 7f0e862bd700 1 -- 172.21.2.101:0/1940195820 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8000b300 con 0x7f0e880cff70 2021-02-10T11:43:28.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.745 7f0e862bd700 1 -- 172.21.2.101:0/1940195820 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e8000b790 con 0x7f0e880cff70 2021-02-10T11:43:28.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.745 7f0e8d610700 1 -- 172.21.2.101:0/1940195820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e880cff70 msgr2=0x7f0e88109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:28.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.745 7f0e8d610700 1 --2- 172.21.2.101:0/1940195820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e880cff70 0x7f0e88109130 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 rx=0x7f0e80006e30 tx=0x7f0e80009e60).stop 2021-02-10T11:43:28.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.746 7f0e8d610700 1 -- 172.21.2.101:0/1940195820 shutdown_connections 2021-02-10T11:43:28.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.746 7f0e8d610700 1 --2- 172.21.2.101:0/1940195820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e880cff70 0x7f0e88109130 unknown :-1 s=CLOSED pgs=2570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:28.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.746 7f0e8d610700 1 --2- 172.21.2.101:0/1940195820 >> v2:172.21.2.108:3300/0 conn(0x7f0e880d1870 0x7f0e881b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:28.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.746 7f0e8d610700 1 -- 172.21.2.101:0/1940195820 >> 172.21.2.101:0/1940195820 conn(0x7f0e880ce420 msgr2=0x7f0e880d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:28.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.746 7f0e8d610700 1 -- 172.21.2.101:0/1940195820 shutdown_connections 2021-02-10T11:43:28.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.746 7f0e8d610700 1 -- 172.21.2.101:0/1940195820 wait complete. 2021-02-10T11:43:28.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.760 7f0e8d610700 1 Processor -- start 2021-02-10T11:43:28.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.761 7f0e8d610700 1 -- start start 2021-02-10T11:43:28.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.761 7f0e8d610700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e880c3820 0x7f0e88084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:28.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.761 7f0e8d610700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e880a0290 0x7f0e880852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:28.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.761 7f0e8d610700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e88086270 0x7f0e88086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:28.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.761 7f0e8d610700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e880c0700 con 0x7f0e880a0290 2021-02-10T11:43:28.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.761 7f0e8d610700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e880858f0 con 0x7f0e88086270 2021-02-10T11:43:28.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.761 7f0e8d610700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e88085b10 con 0x7f0e880c3820 2021-02-10T11:43:28.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.761 7f0e86abe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e880a0290 0x7f0e880852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:28.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.761 7f0e86abe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e880a0290 0x7f0e880852d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59040/0 (socket says 172.21.2.101:59040) 2021-02-10T11:43:28.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.761 7f0e86abe700 1 -- 172.21.2.101:0/2536921334 learned_addr learned my addr 172.21.2.101:0/2536921334 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:28.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.761 7f0e86abe700 1 -- 172.21.2.101:0/2536921334 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e880c3820 msgr2=0x7f0e88084cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:28.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.761 7f0e86abe700 1 --2- 172.21.2.101:0/2536921334 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e880c3820 0x7f0e88084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:28.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.761 7f0e86abe700 1 -- 172.21.2.101:0/2536921334 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e88086270 msgr2=0x7f0e88086670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:28.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.761 7f0e86abe700 1 --2- 172.21.2.101:0/2536921334 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e88086270 0x7f0e88086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:28.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.762 7f0e86abe700 1 -- 172.21.2.101:0/2536921334 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e88086c90 con 0x7f0e880a0290 2021-02-10T11:43:28.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.762 7f0e86abe700 1 --2- 172.21.2.101:0/2536921334 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e880a0290 0x7f0e880852d0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 rx=0x7f0e70001c60 tx=0x7f0e7000a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:28.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.762 7f0e7ffff700 1 -- 172.21.2.101:0/2536921334 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e7000acc0 con 0x7f0e880a0290 2021-02-10T11:43:28.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.762 7f0e7ffff700 1 -- 172.21.2.101:0/2536921334 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e70004140 con 0x7f0e880a0290 2021-02-10T11:43:28.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.762 7f0e8d610700 1 -- 172.21.2.101:0/2536921334 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e88086c90 con 0x7f0e880a0290 2021-02-10T11:43:28.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.762 7f0e8d610700 1 -- 172.21.2.101:0/2536921334 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e88086c90 con 0x7f0e880a0290 2021-02-10T11:43:28.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.762 7f0e7ffff700 1 -- 172.21.2.101:0/2536921334 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0e70004620 con 0x7f0e880a0290 2021-02-10T11:43:28.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.762 7f0e7ffff700 1 -- 172.21.2.101:0/2536921334 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0e70004320 con 0x7f0e880a0290 2021-02-10T11:43:28.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.762 7f0e7ffff700 1 --2- 172.21.2.101:0/2536921334 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e6c0186d0 0x7f0e6c01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:28.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.762 7f0e7ffff700 1 -- 172.21.2.101:0/2536921334 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0e700325e0 con 0x7f0e880a0290 2021-02-10T11:43:28.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.763 7f0e872bf700 1 --2- 172.21.2.101:0/2536921334 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e6c0186d0 0x7f0e6c01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:28.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.763 7f0e872bf700 1 --2- 172.21.2.101:0/2536921334 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e6c0186d0 0x7f0e6c01abd0 crc :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:28.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.795 7f0e8d610700 1 -- 172.21.2.101:0/2536921334 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e88082a70 con 0x7f0e880a0290 2021-02-10T11:43:28.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.797 7f0e7ffff700 1 -- 172.21.2.101:0/2536921334 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0e70000e20 con 0x7f0e880a0290 2021-02-10T11:43:28.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.924 7f0e8d610700 1 -- 172.21.2.101:0/2536921334 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f0e88082a70 con 0x7f0e880a0290 2021-02-10T11:43:28.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.924 7f0e7ffff700 1 -- 172.21.2.101:0/2536921334 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v138) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0e70004140 con 0x7f0e880a0290 2021-02-10T11:43:28.925 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:43:28.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.926 7f0e8d610700 1 -- 172.21.2.101:0/2536921334 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e6c0186d0 msgr2=0x7f0e6c01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:28.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.926 7f0e8d610700 1 --2- 172.21.2.101:0/2536921334 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e6c0186d0 0x7f0e6c01abd0 crc :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:28.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.926 7f0e8d610700 1 -- 172.21.2.101:0/2536921334 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e880a0290 msgr2=0x7f0e880852d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:28.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.926 7f0e8d610700 1 --2- 172.21.2.101:0/2536921334 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e880a0290 0x7f0e880852d0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 rx=0x7f0e70001c60 tx=0x7f0e7000a8d0).stop 2021-02-10T11:43:28.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.926 7f0e8d610700 1 -- 172.21.2.101:0/2536921334 shutdown_connections 2021-02-10T11:43:28.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.926 7f0e8d610700 1 --2- 172.21.2.101:0/2536921334 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0e88086270 0x7f0e88086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:28.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.926 7f0e8d610700 1 --2- 172.21.2.101:0/2536921334 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0e880a0290 0x7f0e880852d0 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:28.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.926 7f0e8d610700 1 --2- 172.21.2.101:0/2536921334 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0e6c0186d0 0x7f0e6c01abd0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:28.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.926 7f0e8d610700 1 --2- 172.21.2.101:0/2536921334 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0e880c3820 0x7f0e88084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:28.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.926 7f0e8d610700 1 -- 172.21.2.101:0/2536921334 >> 172.21.2.101:0/2536921334 conn(0x7f0e880872c0 msgr2=0x7f0e88095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:28.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.927 7f0e8d610700 1 -- 172.21.2.101:0/2536921334 shutdown_connections 2021-02-10T11:43:28.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:28.927 7f0e8d610700 1 -- 172.21.2.101:0/2536921334 wait complete. 2021-02-10T11:43:28.935 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:43:29.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.350 7f7c0f986700 1 Processor -- start 2021-02-10T11:43:29.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.350 7f7c0f986700 1 -- start start 2021-02-10T11:43:29.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.351 7f7c0f986700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c080d1c70 0x7f7c080cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:29.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.351 7f7c0f986700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c080d0170 con 0x7f7c080d1870 2021-02-10T11:43:29.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.351 7f7c0f986700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c080fe1c0 con 0x7f7c080cf580 2021-02-10T11:43:29.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.351 7f7c0f986700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c081b0cb0 con 0x7f7c080d1c70 2021-02-10T11:43:29.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.351 7f7c0d7a2700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f7c080d1870 0x7f7c081b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59540/0 (socket says 172.21.2.101:59540) 2021-02-10T11:43:29.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.351 7f7c0d7a2700 1 -- 172.21.2.101:0/196955096 learned_addr learned my addr 172.21.2.101:0/196955096 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:29.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.351 7f7c0cfa1700 1 --2- 172.21.2.101:0/196955096 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c080d1c70 0x7f7c080cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:29.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.351 7f7bfbfff700 1 -- 172.21.2.101:0/196955096 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172702029 0 0) 0x7f7bfc0010b0 con 0x7f7c080d1870 2021-02-10T11:43:29.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.352 7f7bfbfff700 1 -- 172.21.2.101:0/196955096 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bf4002fe0 con 0x7f7c080d1870 2021-02-10T11:43:29.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.352 7f7c0cfa1700 1 -- 172.21.2.101:0/196955096 >> v1:172.21.2.108:6789/0 conn(0x7f7c080cf580 legacy=0x7f7c080cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:29.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.352 7f7c0cfa1700 1 -- 172.21.2.101:0/196955096 >> v1:172.21.2.101:6789/0 conn(0x7f7c080d1870 legacy=0x7f7c081b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:29.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.352 7f7c0cfa1700 1 -- 172.21.2.101:0/196955096 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c08109a60 con 0x7f7c080d1c70 2021-02-10T11:43:29.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.352 7f7c0cfa1700 1 --2- 172.21.2.101:0/196955096 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c080d1c70 0x7f7c080cef60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f7c04006650 tx=0x7f7c04009a80).ready entity=mon.2 client_cookie=9432ebc1655e8dfe server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:29.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.353 7f7bfbfff700 1 -- 172.21.2.101:0/196955096 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7c0400c0c0 con 0x7f7c080d1c70 2021-02-10T11:43:29.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.353 7f7bfbfff700 1 -- 172.21.2.101:0/196955096 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0400c340 con 0x7f7c080d1c70 2021-02-10T11:43:29.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.353 7f7bfbfff700 1 -- 172.21.2.101:0/196955096 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7c0400c860 con 0x7f7c080d1c70 2021-02-10T11:43:29.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.353 7f7c0f986700 1 -- 172.21.2.101:0/196955096 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c080d1c70 msgr2=0x7f7c080cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:29.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.353 7f7c0f986700 1 --2- 172.21.2.101:0/196955096 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c080d1c70 0x7f7c080cef60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f7c04006650 tx=0x7f7c04009a80).stop 2021-02-10T11:43:29.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.353 7f7c0f986700 1 -- 172.21.2.101:0/196955096 shutdown_connections 2021-02-10T11:43:29.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.353 7f7c0f986700 1 --2- 172.21.2.101:0/196955096 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c080d1c70 0x7f7c080cef60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:29.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.353 7f7c0f986700 1 -- 172.21.2.101:0/196955096 >> 172.21.2.101:0/196955096 conn(0x7f7c080ce420 msgr2=0x7f7c080d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:29.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.354 7f7c0f986700 1 -- 172.21.2.101:0/196955096 shutdown_connections 2021-02-10T11:43:29.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.354 7f7c0f986700 1 -- 172.21.2.101:0/196955096 wait complete. 2021-02-10T11:43:29.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.368 7f7c0f986700 1 Processor -- start 2021-02-10T11:43:29.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.368 7f7c0f986700 1 -- start start 2021-02-10T11:43:29.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.369 7f7c0f986700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c080c3820 0x7f7c080a7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:29.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.369 7f7c0f986700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c081a44c0 0x7f7c081a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:29.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.369 7f7c0f986700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c080863d0 0x7f7c080867d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:29.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.369 7f7c0f986700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c08085ed0 con 0x7f7c080c3820 2021-02-10T11:43:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.369 7f7c0f986700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c080c0740 con 0x7f7c081a44c0 2021-02-10T11:43:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.369 7f7c0f986700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c08086df0 con 0x7f7c080863d0 2021-02-10T11:43:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.369 7f7c0d7a2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c080c3820 0x7f7c080a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.369 7f7c0d7a2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c080c3820 0x7f7c080a7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59052/0 (socket says 172.21.2.101:59052) 2021-02-10T11:43:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.369 7f7c0d7a2700 1 -- 172.21.2.101:0/4279950140 learned_addr learned my addr 172.21.2.101:0/4279950140 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.369 7f7c0d7a2700 1 -- 172.21.2.101:0/4279950140 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c080863d0 msgr2=0x7f7c080867d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.369 7f7c0d7a2700 1 --2- 172.21.2.101:0/4279950140 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c080863d0 0x7f7c080867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.369 7f7c0d7a2700 1 -- 172.21.2.101:0/4279950140 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c081a44c0 msgr2=0x7f7c081a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.369 7f7c0d7a2700 1 --2- 172.21.2.101:0/4279950140 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c081a44c0 0x7f7c081a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.369 7f7c0d7a2700 1 -- 172.21.2.101:0/4279950140 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c08087010 con 0x7f7c080c3820 2021-02-10T11:43:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.369 7f7c0d7a2700 1 --2- 172.21.2.101:0/4279950140 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c080c3820 0x7f7c080a7950 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 rx=0x7f7bfc001c60 tx=0x7f7bfc00a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.369 7f7bf9ffb700 1 -- 172.21.2.101:0/4279950140 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7bfc000e00 con 0x7f7c080c3820 2021-02-10T11:43:29.372 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.370 7f7bf9ffb700 1 -- 172.21.2.101:0/4279950140 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bfc00acc0 con 0x7f7c080c3820 2021-02-10T11:43:29.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.370 7f7bf9ffb700 1 -- 172.21.2.101:0/4279950140 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7bfc0045d0 con 0x7f7c080c3820 2021-02-10T11:43:29.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.370 7f7c0f986700 1 -- 172.21.2.101:0/4279950140 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c08087010 con 0x7f7c080c3820 2021-02-10T11:43:29.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.370 7f7c0f986700 1 -- 172.21.2.101:0/4279950140 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c08083000 con 0x7f7c080c3820 2021-02-10T11:43:29.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.370 7f7bf9ffb700 1 -- 172.21.2.101:0/4279950140 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7bfc004140 con 0x7f7c080c3820 2021-02-10T11:43:29.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.370 7f7bf9ffb700 1 --2- 172.21.2.101:0/4279950140 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7be80186c0 0x7f7be801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:29.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.370 7f7bf9ffb700 1 -- 172.21.2.101:0/4279950140 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7bfc032410 con 0x7f7c080c3820 2021-02-10T11:43:29.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.370 7f7c0cfa1700 1 --2- 172.21.2.101:0/4279950140 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7be80186c0 0x7f7be801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:29.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.371 7f7c0cfa1700 1 --2- 172.21.2.101:0/4279950140 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7be80186c0 0x7f7be801abc0 crc :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:29.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.401 7f7c0f986700 1 -- 172.21.2.101:0/4279950140 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c080fe1c0 con 0x7f7c080c3820 2021-02-10T11:43:29.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.403 7f7bf9ffb700 1 -- 172.21.2.101:0/4279950140 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7bfc020eb0 con 0x7f7c080c3820 2021-02-10T11:43:29.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.531 7f7c0f986700 1 -- 172.21.2.101:0/4279950140 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f7c080fe1c0 con 0x7f7c080c3820 2021-02-10T11:43:29.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.531 7f7bf9ffb700 1 -- 172.21.2.101:0/4279950140 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v138) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7bfc020eb0 con 0x7f7c080c3820 2021-02-10T11:43:29.533 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:43:29.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.533 7f7c0f986700 1 -- 172.21.2.101:0/4279950140 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7be80186c0 msgr2=0x7f7be801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:29.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.533 7f7c0f986700 1 --2- 172.21.2.101:0/4279950140 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7be80186c0 0x7f7be801abc0 crc :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:29.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.533 7f7c0f986700 1 -- 172.21.2.101:0/4279950140 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c080c3820 msgr2=0x7f7c080a7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:29.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.533 7f7c0f986700 1 --2- 172.21.2.101:0/4279950140 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c080c3820 0x7f7c080a7950 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 rx=0x7f7bfc001c60 tx=0x7f7bfc00a8d0).stop 2021-02-10T11:43:29.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.534 7f7c0f986700 1 -- 172.21.2.101:0/4279950140 shutdown_connections 2021-02-10T11:43:29.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.534 7f7c0f986700 1 --2- 172.21.2.101:0/4279950140 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7be80186c0 0x7f7be801abc0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:29.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.534 7f7c0f986700 1 --2- 172.21.2.101:0/4279950140 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7c080863d0 0x7f7c080867d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:29.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.534 7f7c0f986700 1 --2- 172.21.2.101:0/4279950140 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7c081a44c0 0x7f7c081a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:29.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.534 7f7c0f986700 1 --2- 172.21.2.101:0/4279950140 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7c080c3820 0x7f7c080a7950 unknown :-1 s=CLOSED pgs=2573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:29.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.534 7f7c0f986700 1 -- 172.21.2.101:0/4279950140 >> 172.21.2.101:0/4279950140 conn(0x7f7c080872c0 msgr2=0x7f7c08095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:29.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.534 7f7c0f986700 1 -- 172.21.2.101:0/4279950140 shutdown_connections 2021-02-10T11:43:29.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.534 7f7c0f986700 1 -- 172.21.2.101:0/4279950140 wait complete. 2021-02-10T11:43:29.544 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:43:29.544 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:43:29.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.955 7fd29093b700 1 Processor -- start 2021-02-10T11:43:29.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.955 7fd29093b700 1 -- start start 2021-02-10T11:43:29.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.956 7fd29093b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd28c0d1870 0x7fd28c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:29.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.956 7fd29093b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd28c0f0b10 con 0x7fd28c1b0a10 2021-02-10T11:43:29.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.956 7fd29093b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd28c0d0310 con 0x7fd28c0f0650 2021-02-10T11:43:29.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.956 7fd29093b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd28c0d0530 con 0x7fd28c0d1870 2021-02-10T11:43:29.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.956 7fd28a61c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd28c0d1870 0x7fd28c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:29.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.956 7fd289e1b700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd28c1b0a10 0x7fd28c0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59554/0 (socket says 172.21.2.101:59554) 2021-02-10T11:43:29.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.956 7fd28a61c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd28c0d1870 0x7fd28c1b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59060/0 (socket says 172.21.2.101:59060) 2021-02-10T11:43:29.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.956 7fd289e1b700 1 -- 172.21.2.101:0/468575557 learned_addr learned my addr 172.21.2.101:0/468575557 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:29.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.957 7fd28a61c700 1 -- 172.21.2.101:0/468575557 >> v1:172.21.2.108:6789/0 conn(0x7fd28c0f0650 legacy=0x7fd28c0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:29.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.957 7fd28a61c700 1 -- 172.21.2.101:0/468575557 >> v1:172.21.2.101:6789/0 conn(0x7fd28c1b0a10 legacy=0x7fd28c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:29.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.957 7fd28a61c700 1 -- 172.21.2.101:0/468575557 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c0d1c70 con 0x7fd28c0d1870 2021-02-10T11:43:29.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.957 7fd28a61c700 1 --2- 172.21.2.101:0/468575557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd28c0d1870 0x7fd28c1b03f0 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 rx=0x7fd27c006650 tx=0x7fd27c009a80).ready entity=mon.0 client_cookie=6dc544e4ad489064 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:29.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.957 7fd28961a700 1 -- 172.21.2.101:0/468575557 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd27c00c0c0 con 0x7fd28c0d1870 2021-02-10T11:43:29.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.957 7fd28961a700 1 -- 172.21.2.101:0/468575557 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27c00c340 con 0x7fd28c0d1870 2021-02-10T11:43:29.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.957 7fd28961a700 1 -- 172.21.2.101:0/468575557 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd27c00c860 con 0x7fd28c0d1870 2021-02-10T11:43:29.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.957 7fd29093b700 1 -- 172.21.2.101:0/468575557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd28c0d1870 msgr2=0x7fd28c1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:29.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.957 7fd29093b700 1 --2- 172.21.2.101:0/468575557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd28c0d1870 0x7fd28c1b03f0 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 rx=0x7fd27c006650 tx=0x7fd27c009a80).stop 2021-02-10T11:43:29.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.958 7fd29093b700 1 -- 172.21.2.101:0/468575557 shutdown_connections 2021-02-10T11:43:29.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.958 7fd29093b700 1 --2- 172.21.2.101:0/468575557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd28c0d1870 0x7fd28c1b03f0 unknown :-1 s=CLOSED pgs=2575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:29.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.958 7fd29093b700 1 -- 172.21.2.101:0/468575557 >> 172.21.2.101:0/468575557 conn(0x7fd28c0ce450 msgr2=0x7fd28c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:29.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.958 7fd29093b700 1 -- 172.21.2.101:0/468575557 shutdown_connections 2021-02-10T11:43:29.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.958 7fd29093b700 1 -- 172.21.2.101:0/468575557 wait complete. 2021-02-10T11:43:29.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.972 7fd29093b700 1 Processor -- start 2021-02-10T11:43:29.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.972 7fd29093b700 1 -- start start 2021-02-10T11:43:29.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.973 7fd29093b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd28c0c33c0 0x7fd28c0ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:29.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.973 7fd29093b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd28c0adf00 0x7fd28c0ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:29.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.973 7fd29093b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd28c0ae920 0x7fd28c0ee210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:29.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.973 7fd29093b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c0edc90 con 0x7fd28c0ae920 2021-02-10T11:43:29.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.973 7fd29093b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c0ed800 con 0x7fd28c0adf00 2021-02-10T11:43:29.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.973 7fd29093b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c0c37c0 con 0x7fd28c0c33c0 2021-02-10T11:43:29.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.973 7fd28ae1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd28c0ae920 0x7fd28c0ee210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:29.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.973 7fd28ae1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd28c0ae920 0x7fd28c0ee210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59066/0 (socket says 172.21.2.101:59066) 2021-02-10T11:43:29.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.973 7fd28ae1d700 1 -- 172.21.2.101:0/2498917061 learned_addr learned my addr 172.21.2.101:0/2498917061 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:29.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.973 7fd289e1b700 1 --2- 172.21.2.101:0/2498917061 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd28c0adf00 0x7fd28c0ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:29.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.973 7fd28a61c700 1 --2- 172.21.2.101:0/2498917061 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd28c0c33c0 0x7fd28c0ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:29.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.973 7fd28ae1d700 1 -- 172.21.2.101:0/2498917061 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd28c0c33c0 msgr2=0x7fd28c0ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:29.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.973 7fd28ae1d700 1 --2- 172.21.2.101:0/2498917061 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd28c0c33c0 0x7fd28c0ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:29.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.973 7fd28ae1d700 1 -- 172.21.2.101:0/2498917061 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd28c0adf00 msgr2=0x7fd28c0ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:29.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.973 7fd28ae1d700 1 --2- 172.21.2.101:0/2498917061 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd28c0adf00 0x7fd28c0ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:29.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.974 7fd28ae1d700 1 -- 172.21.2.101:0/2498917061 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c0ee830 con 0x7fd28c0ae920 2021-02-10T11:43:29.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.974 7fd289e1b700 1 --2- 172.21.2.101:0/2498917061 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd28c0adf00 0x7fd28c0ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:43:29.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.974 7fd28ae1d700 1 --2- 172.21.2.101:0/2498917061 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd28c0ae920 0x7fd28c0ee210 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 rx=0x7fd28400b8e0 tx=0x7fd28400bc80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:29.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.974 7fd282ffd700 1 -- 172.21.2.101:0/2498917061 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd284000f90 con 0x7fd28c0ae920 2021-02-10T11:43:29.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.974 7fd282ffd700 1 -- 172.21.2.101:0/2498917061 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28400c2f0 con 0x7fd28c0ae920 2021-02-10T11:43:29.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.974 7fd282ffd700 1 -- 172.21.2.101:0/2498917061 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd28400c810 con 0x7fd28c0ae920 2021-02-10T11:43:29.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.974 7fd29093b700 1 -- 172.21.2.101:0/2498917061 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd28c0ee830 con 0x7fd28c0ae920 2021-02-10T11:43:29.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.974 7fd29093b700 1 -- 172.21.2.101:0/2498917061 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd28c0eedc0 con 0x7fd28c0ae920 2021-02-10T11:43:29.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.975 7fd282ffd700 1 -- 172.21.2.101:0/2498917061 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd284001220 con 0x7fd28c0ae920 2021-02-10T11:43:29.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.975 7fd282ffd700 1 --2- 172.21.2.101:0/2498917061 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2700186c0 0x7fd27001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:29.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.975 7fd282ffd700 1 -- 172.21.2.101:0/2498917061 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd2840329a0 con 0x7fd28c0ae920 2021-02-10T11:43:29.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.975 7fd28a61c700 1 --2- 172.21.2.101:0/2498917061 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2700186c0 0x7fd27001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:29.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:29.975 7fd28a61c700 1 --2- 172.21.2.101:0/2498917061 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2700186c0 0x7fd27001abc0 crc :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:30.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:30.006 7fd29093b700 1 -- 172.21.2.101:0/2498917061 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd28c0e7de0 con 0x7fd28c0ae920 2021-02-10T11:43:30.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:30.008 7fd282ffd700 1 -- 172.21.2.101:0/2498917061 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd284015230 con 0x7fd28c0ae920 2021-02-10T11:43:30.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:30.136 7fd29093b700 1 -- 172.21.2.101:0/2498917061 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd28c0d1ee0 con 0x7fd28c0ae920 2021-02-10T11:43:30.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:30.136 7fd282ffd700 1 -- 172.21.2.101:0/2498917061 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+2102 (secure 0 0 0) 0x7fd284015230 con 0x7fd28c0ae920 2021-02-10T11:43:30.137 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:43:30.137 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":114,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/371124351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":371124351},{"type":"v1","addr":"172.21.2.101:6835","nonce":371124351}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:43:20.159141","modified":"2021-02-10 11:43:22.169386","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7990":{"gid":7990,"name":"b","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/797061093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":797061093},{"type":"v1","addr":"172.21.2.108:6833","nonce":797061093}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-02-10T11:43:30.138 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 114 2021-02-10T11:43:30.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:30.138 7fd29093b700 1 -- 172.21.2.101:0/2498917061 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2700186c0 msgr2=0x7fd27001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:30.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:30.138 7fd29093b700 1 --2- 172.21.2.101:0/2498917061 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2700186c0 0x7fd27001abc0 crc :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:30.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:30.138 7fd29093b700 1 -- 172.21.2.101:0/2498917061 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd28c0ae920 msgr2=0x7fd28c0ee210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:30.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:30.138 7fd29093b700 1 --2- 172.21.2.101:0/2498917061 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd28c0ae920 0x7fd28c0ee210 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 rx=0x7fd28400b8e0 tx=0x7fd28400bc80).stop 2021-02-10T11:43:30.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:30.138 7fd29093b700 1 -- 172.21.2.101:0/2498917061 shutdown_connections 2021-02-10T11:43:30.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:30.138 7fd29093b700 1 --2- 172.21.2.101:0/2498917061 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd28c0ae920 0x7fd28c0ee210 unknown :-1 s=CLOSED pgs=2576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:30.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:30.139 7fd29093b700 1 --2- 172.21.2.101:0/2498917061 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd28c0adf00 0x7fd28c0ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:30.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:30.139 7fd29093b700 1 --2- 172.21.2.101:0/2498917061 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2700186c0 0x7fd27001abc0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:30.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:30.139 7fd29093b700 1 --2- 172.21.2.101:0/2498917061 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd28c0c33c0 0x7fd28c0ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:30.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:30.139 7fd29093b700 1 -- 172.21.2.101:0/2498917061 >> 172.21.2.101:0/2498917061 conn(0x7fd28c0a0030 msgr2=0x7fd28c0eb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:30.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:30.139 7fd29093b700 1 -- 172.21.2.101:0/2498917061 shutdown_connections 2021-02-10T11:43:30.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:30.139 7fd29093b700 1 -- 172.21.2.101:0/2498917061 wait complete. 2021-02-10T11:43:30.348 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 114, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:43:20.159141', 'modified': '2021-02-10 11:43:22.169386', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7990}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7990': {'gid': 7990, 'name': 'b', 'rank': 0, 'incarnation': 111, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/797061093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 797061093}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 797061093}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [29], 'metadata_pool': 28, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:43:30.348 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:43:30.349 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:43:30.748 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:43:30.748 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:43:30.748 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:43:30.748 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 7990, 2021-02-10T11:43:30.749 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:43:30.749 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:43:30.749 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 10.585997782, 2021-02-10T11:43:30.749 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 114, 2021-02-10T11:43:30.749 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 138, 2021-02-10T11:43:30.749 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 138, 2021-02-10T11:43:30.749 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 16.218673587000001 2021-02-10T11:43:30.749 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:43:30.752 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 7990, "want_state": "up:active", "state": "up:active", "rank_uptime": 10.585997782, "mdsmap_epoch": 114, "osdmap_epoch": 138, "osdmap_epoch_barrier": 138, "uptime": 16.218673587000001 } 2021-02-10T11:43:30.752 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:43:30.752 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-02-10T11:43:30.753 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-02-10T11:43:30.753 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:30.753 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:43:31.089 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:43:31.433 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:43:31.433 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:43:31.775 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:43:31.776 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:43:31.776 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:43:31.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:31.777 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:43:32.111 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-02-10T11:43:32.112 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:43:32.114 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:43:32.458 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:43:32.458 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:43:32.524 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[158840]: starting ceph client 2021-02-10T11:43:32.524 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:2021-02-10 11:43:32.524 7f53306c3f80 -1 init, newargv = 0x562e0b095d80 newargc=9 2021-02-10T11:43:32.530 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[158840]: starting fuse 2021-02-10T11:43:32.801 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:43:32.801 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:43:32.802 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:43:32.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:32.802 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:43:33.137 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:43:33.138 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2021-02-10T11:43:33.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:33.138 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:43:33.138 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:43:33.138 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:43:33.138 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:43:33.138 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:43:33.138 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:43:33.138 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:43:33.138 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-02-10T11:43:33.138 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:43:33.139 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:43:33.139 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:43:33.139 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:43:33.139 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:43:33.139 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:43:33.139 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:43:33.139 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:43:33.139 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:43:33.139 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:43:33.139 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:43:33.139 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:43:33.139 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:43:33.139 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.0")) 2021-02-10T11:43:33.139 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:43:33.501 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.0.158840.asok 2021-02-10T11:43:33.504 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.158840.asok 2021-02-10T11:43:33.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:33.504 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.158840.asok status 2021-02-10T11:43:33.894 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:43:33.894 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:43:33.894 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:43:33.894 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:43:33.894 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "0", 2021-02-10T11:43:33.894 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:43:33.894 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-02-10T11:43:33.894 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "158840", 2021-02-10T11:43:33.895 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:43:33.895 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:33.895 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:43:33.895 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:43:33.895 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 11661, 2021-02-10T11:43:33.895 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:43:33.895 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:43:33.895 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:43:33.895 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 11661 2021-02-10T11:43:33.895 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:33.895 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:43:33.895 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:33.895 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:43:33.896 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 3647209867 2021-02-10T11:43:33.896 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:33.896 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:33.896 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:43:33.896 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:33.896 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:43:33.896 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 3647209867 2021-02-10T11:43:33.896 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:33.896 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.11661 172.21.2.101:0/3647209867", 2021-02-10T11:43:33.896 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/3647209867", 2021-02-10T11:43:33.896 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:43:33.896 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 114, 2021-02-10T11:43:33.896 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 138, 2021-02-10T11:43:33.897 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:43:33.897 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:43:33.897 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:43:33.898 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:33.898 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:43:33.898 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:43:34.233 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:43:34.234 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:43:34.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:34.234 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:43:34.582 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:43:35.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.996 7f1f49030700 1 Processor -- start 2021-02-10T11:43:35.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.996 7f1f49030700 1 -- start start 2021-02-10T11:43:35.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.997 7f1f49030700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1f440d0340 0x7f1f440ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:35.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.997 7f1f49030700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f440d1c70 0x7f1f440efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:35.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.997 7f1f49030700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f440f0b10 con 0x7f1f440d1870 2021-02-10T11:43:35.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.997 7f1f49030700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f440e5ed0 con 0x7f1f440d0340 2021-02-10T11:43:35.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.997 7f1f49030700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f441b0be0 con 0x7f1f440d1c70 2021-02-10T11:43:35.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.997 7f1f42d9d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f1f440d1870 0x7f1f441b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59576/0 (socket says 172.21.2.101:59576) 2021-02-10T11:43:35.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.997 7f1f42d9d700 1 -- 172.21.2.101:0/1850365488 learned_addr learned my addr 172.21.2.101:0/1850365488 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:35.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.997 7f1f4259c700 1 --2- 172.21.2.101:0/1850365488 >> v2:172.21.2.101:3300/0 conn(0x7f1f440d0340 0x7f1f440ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:35.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.997 7f1f41d9b700 1 -- 172.21.2.101:0/1850365488 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844968595 0 0) 0x7f1f34001180 con 0x7f1f440d1870 2021-02-10T11:43:35.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.997 7f1f41d9b700 1 -- 172.21.2.101:0/1850365488 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f2c002fe0 con 0x7f1f440d1870 2021-02-10T11:43:35.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.997 7f1f4259c700 1 -- 172.21.2.101:0/1850365488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f440d1c70 msgr2=0x7f1f440efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:35.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.997 7f1f4259c700 1 --2- 172.21.2.101:0/1850365488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f440d1c70 0x7f1f440efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:35.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.997 7f1f4259c700 1 -- 172.21.2.101:0/1850365488 >> v1:172.21.2.101:6789/0 conn(0x7f1f440d1870 legacy=0x7f1f441b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:35.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.997 7f1f4259c700 1 -- 172.21.2.101:0/1850365488 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f440f20a0 con 0x7f1f440d0340 2021-02-10T11:43:35.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.998 7f1f4259c700 1 --2- 172.21.2.101:0/1850365488 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f440d0340 0x7f1f440ef640 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 rx=0x7f1f38006650 tx=0x7f1f38009a00).ready entity=mon.0 client_cookie=84c5d95f1f6ec632 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:35.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.998 7f1f41d9b700 1 -- 172.21.2.101:0/1850365488 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f38009df0 con 0x7f1f440d0340 2021-02-10T11:43:35.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.998 7f1f41d9b700 1 -- 172.21.2.101:0/1850365488 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f380009b0 con 0x7f1f440d0340 2021-02-10T11:43:35.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.998 7f1f41d9b700 1 -- 172.21.2.101:0/1850365488 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f3800c2b0 con 0x7f1f440d0340 2021-02-10T11:43:35.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.998 7f1f49030700 1 -- 172.21.2.101:0/1850365488 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f440d0340 msgr2=0x7f1f440ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:35.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.998 7f1f49030700 1 --2- 172.21.2.101:0/1850365488 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f440d0340 0x7f1f440ef640 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 rx=0x7f1f38006650 tx=0x7f1f38009a00).stop 2021-02-10T11:43:35.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.999 7f1f49030700 1 -- 172.21.2.101:0/1850365488 shutdown_connections 2021-02-10T11:43:35.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.999 7f1f49030700 1 --2- 172.21.2.101:0/1850365488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f440d1c70 0x7f1f440efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:35.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.999 7f1f49030700 1 --2- 172.21.2.101:0/1850365488 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f440d0340 0x7f1f440ef640 unknown :-1 s=CLOSED pgs=2581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:35.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.999 7f1f49030700 1 -- 172.21.2.101:0/1850365488 >> 172.21.2.101:0/1850365488 conn(0x7f1f440ce450 msgr2=0x7f1f440d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:35.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.999 7f1f49030700 1 -- 172.21.2.101:0/1850365488 shutdown_connections 2021-02-10T11:43:35.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:34.999 7f1f49030700 1 -- 172.21.2.101:0/1850365488 wait complete. 2021-02-10T11:43:35.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.013 7f1f49030700 1 Processor -- start 2021-02-10T11:43:35.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.013 7f1f49030700 1 -- start start 2021-02-10T11:43:35.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.014 7f1f49030700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f440ada90 0x7f1f440ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:35.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.014 7f1f49030700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f44099d20 0x7f1f4409a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:35.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.014 7f1f49030700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f4409a740 0x7f1f44097bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:35.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.014 7f1f49030700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f440997a0 con 0x7f1f440ada90 2021-02-10T11:43:35.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.014 7f1f49030700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4418f0c0 con 0x7f1f4409a740 2021-02-10T11:43:35.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.014 7f1f49030700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4409ab40 con 0x7f1f44099d20 2021-02-10T11:43:35.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.014 7f1f42d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f440ada90 0x7f1f440ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:35.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.014 7f1f42d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f440ada90 0x7f1f440ade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59090/0 (socket says 172.21.2.101:59090) 2021-02-10T11:43:35.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.014 7f1f42d9d700 1 -- 172.21.2.101:0/1013599235 learned_addr learned my addr 172.21.2.101:0/1013599235 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:35.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.014 7f1f4259c700 1 --2- 172.21.2.101:0/1013599235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f44099d20 0x7f1f4409a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:35.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.014 7f1f4359e700 1 --2- 172.21.2.101:0/1013599235 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f4409a740 0x7f1f44097bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:35.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.015 7f1f42d9d700 1 -- 172.21.2.101:0/1013599235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f44099d20 msgr2=0x7f1f4409a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:35.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.015 7f1f42d9d700 1 --2- 172.21.2.101:0/1013599235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f44099d20 0x7f1f4409a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:35.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.015 7f1f42d9d700 1 -- 172.21.2.101:0/1013599235 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f4409a740 msgr2=0x7f1f44097bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:35.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.015 7f1f42d9d700 1 --2- 172.21.2.101:0/1013599235 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f4409a740 0x7f1f44097bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:35.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.015 7f1f42d9d700 1 -- 172.21.2.101:0/1013599235 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4409ad60 con 0x7f1f440ada90 2021-02-10T11:43:35.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.015 7f1f42d9d700 1 --2- 172.21.2.101:0/1013599235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f440ada90 0x7f1f440ade90 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 rx=0x7f1f3400b530 tx=0x7f1f3400b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:35.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.015 7f1f337fe700 1 -- 172.21.2.101:0/1013599235 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f3400bcc0 con 0x7f1f440ada90 2021-02-10T11:43:35.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.015 7f1f337fe700 1 -- 172.21.2.101:0/1013599235 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f34006030 con 0x7f1f440ada90 2021-02-10T11:43:35.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.015 7f1f337fe700 1 -- 172.21.2.101:0/1013599235 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f34004620 con 0x7f1f440ada90 2021-02-10T11:43:35.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.015 7f1f49030700 1 -- 172.21.2.101:0/1013599235 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f4409ad60 con 0x7f1f440ada90 2021-02-10T11:43:35.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.015 7f1f49030700 1 -- 172.21.2.101:0/1013599235 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f4409ad60 con 0x7f1f440ada90 2021-02-10T11:43:35.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.016 7f1f337fe700 1 -- 172.21.2.101:0/1013599235 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1f340043a0 con 0x7f1f440ada90 2021-02-10T11:43:35.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.016 7f1f337fe700 1 --2- 172.21.2.101:0/1013599235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f280186c0 0x7f1f2801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:35.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.016 7f1f337fe700 1 -- 172.21.2.101:0/1013599235 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1f34032540 con 0x7f1f440ada90 2021-02-10T11:43:35.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.016 7f1f4259c700 1 --2- 172.21.2.101:0/1013599235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f280186c0 0x7f1f2801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:35.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.016 7f1f4259c700 1 --2- 172.21.2.101:0/1013599235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f280186c0 0x7f1f2801abc0 crc :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:35.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.047 7f1f49030700 1 -- 172.21.2.101:0/1013599235 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f440ebc00 con 0x7f1f440ada90 2021-02-10T11:43:35.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.049 7f1f337fe700 1 -- 172.21.2.101:0/1013599235 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1f34025f70 con 0x7f1f440ada90 2021-02-10T11:43:35.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.176 7f1f49030700 1 -- 172.21.2.101:0/1013599235 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f440ebc00 con 0x7f1f440ada90 2021-02-10T11:43:35.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.176 7f1f337fe700 1 -- 172.21.2.101:0/1013599235 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f1f34025f70 con 0x7f1f440ada90 2021-02-10T11:43:35.176 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:43:35.176 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":138,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:43:21.161685","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:43:18.076350","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2021-02-10 11:43:19.152025","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:43:35.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.178 7f1f49030700 1 -- 172.21.2.101:0/1013599235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f280186c0 msgr2=0x7f1f2801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:35.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.178 7f1f49030700 1 --2- 172.21.2.101:0/1013599235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f280186c0 0x7f1f2801abc0 crc :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:35.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.178 7f1f49030700 1 -- 172.21.2.101:0/1013599235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f440ada90 msgr2=0x7f1f440ade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:35.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.178 7f1f49030700 1 --2- 172.21.2.101:0/1013599235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f440ada90 0x7f1f440ade90 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 rx=0x7f1f3400b530 tx=0x7f1f3400b8d0).stop 2021-02-10T11:43:35.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.178 7f1f49030700 1 -- 172.21.2.101:0/1013599235 shutdown_connections 2021-02-10T11:43:35.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.178 7f1f49030700 1 --2- 172.21.2.101:0/1013599235 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f280186c0 0x7f1f2801abc0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:35.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.178 7f1f49030700 1 --2- 172.21.2.101:0/1013599235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f44099d20 0x7f1f4409a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:35.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.178 7f1f49030700 1 --2- 172.21.2.101:0/1013599235 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f4409a740 0x7f1f44097bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:35.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.178 7f1f49030700 1 --2- 172.21.2.101:0/1013599235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f440ada90 0x7f1f440ade90 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:35.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.178 7f1f49030700 1 -- 172.21.2.101:0/1013599235 >> 172.21.2.101:0/1013599235 conn(0x7f1f440f0550 msgr2=0x7f1f44191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:35.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.178 7f1f49030700 1 -- 172.21.2.101:0/1013599235 shutdown_connections 2021-02-10T11:43:35.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.178 7f1f49030700 1 -- 172.21.2.101:0/1013599235 wait complete. 2021-02-10T11:43:35.186 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 28, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:43:18.076350', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 29, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:43:19.152025', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:43:35.186 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:43:35.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.602 7fb36b6f3700 1 Processor -- start 2021-02-10T11:43:35.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.602 7fb36b6f3700 1 -- start start 2021-02-10T11:43:35.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.603 7fb36b6f3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb3640d1c70 0x7fb3640cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:35.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.603 7fb36b6f3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb3640cf580 0x7fb3640cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:35.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.603 7fb36b6f3700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3640fe1c0 con 0x7fb3640d1870 2021-02-10T11:43:35.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.603 7fb36b6f3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3641b0cb0 con 0x7fb3640d1c70 2021-02-10T11:43:35.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.603 7fb36b6f3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb364109a60 con 0x7fb3640cf580 2021-02-10T11:43:35.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.603 7fb368d0e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb3640d1c70 0x7fb3640cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:35.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.603 7fb368d0e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb3640d1c70 0x7fb3640cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59094/0 (socket says 172.21.2.101:59094) 2021-02-10T11:43:35.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.603 7fb368d0e700 1 -- 172.21.2.101:0/3354821998 learned_addr learned my addr 172.21.2.101:0/3354821998 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:35.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.603 7fb368d0e700 1 -- 172.21.2.101:0/3354821998 >> v2:172.21.2.108:3300/0 conn(0x7fb3640cf580 msgr2=0x7fb3640cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:35.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.603 7fb368d0e700 1 --2- 172.21.2.101:0/3354821998 >> v2:172.21.2.108:3300/0 conn(0x7fb3640cf580 0x7fb3640cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:35.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.603 7fb368d0e700 1 -- 172.21.2.101:0/3354821998 >> v1:172.21.2.108:6789/0 conn(0x7fb3640d1870 legacy=0x7fb3641b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:35.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.603 7fb368d0e700 1 -- 172.21.2.101:0/3354821998 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb364088070 con 0x7fb3640d1c70 2021-02-10T11:43:35.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.603 7fb368d0e700 1 --2- 172.21.2.101:0/3354821998 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3640d1c70 0x7fb3640cef60 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 rx=0x7fb358006700 tx=0x7fb358009b80).ready entity=mon.0 client_cookie=ef2881f05c376879 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:35.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.604 7fb357fff700 1 -- 172.21.2.101:0/3354821998 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb35800c0c0 con 0x7fb3640d1c70 2021-02-10T11:43:35.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.604 7fb357fff700 1 -- 172.21.2.101:0/3354821998 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35800c440 con 0x7fb3640d1c70 2021-02-10T11:43:35.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.604 7fb357fff700 1 -- 172.21.2.101:0/3354821998 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb35800c960 con 0x7fb3640d1c70 2021-02-10T11:43:35.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.604 7fb36b6f3700 1 -- 172.21.2.101:0/3354821998 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3640d1c70 msgr2=0x7fb3640cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:35.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.604 7fb36b6f3700 1 --2- 172.21.2.101:0/3354821998 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3640d1c70 0x7fb3640cef60 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 rx=0x7fb358006700 tx=0x7fb358009b80).stop 2021-02-10T11:43:35.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.604 7fb36b6f3700 1 -- 172.21.2.101:0/3354821998 shutdown_connections 2021-02-10T11:43:35.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.604 7fb36b6f3700 1 --2- 172.21.2.101:0/3354821998 >> v2:172.21.2.108:3300/0 conn(0x7fb3640cf580 0x7fb3640cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:35.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.604 7fb36b6f3700 1 --2- 172.21.2.101:0/3354821998 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3640d1c70 0x7fb3640cef60 unknown :-1 s=CLOSED pgs=2583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:35.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.604 7fb36b6f3700 1 -- 172.21.2.101:0/3354821998 >> 172.21.2.101:0/3354821998 conn(0x7fb3640ce420 msgr2=0x7fb3640d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:35.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.605 7fb36b6f3700 1 -- 172.21.2.101:0/3354821998 shutdown_connections 2021-02-10T11:43:35.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.605 7fb36b6f3700 1 -- 172.21.2.101:0/3354821998 wait complete. 2021-02-10T11:43:35.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.619 7fb36b6f3700 1 Processor -- start 2021-02-10T11:43:35.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.619 7fb36b6f3700 1 -- start start 2021-02-10T11:43:35.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.620 7fb36b6f3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb3640c2370 0x7fb36418f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:35.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.620 7fb36b6f3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb364084cb0 0x7fb3641a7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:35.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.620 7fb36b6f3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3641a7a90 0x7fb364085700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:35.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.620 7fb36b6f3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3640be840 con 0x7fb3641a7a90 2021-02-10T11:43:35.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.620 7fb36b6f3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3640c29a0 con 0x7fb364084cb0 2021-02-10T11:43:35.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.620 7fb36b6f3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3641a7e90 con 0x7fb3640c2370 2021-02-10T11:43:35.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.620 7fb369d10700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3641a7a90 0x7fb364085700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:35.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.620 7fb369d10700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3641a7a90 0x7fb364085700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59102/0 (socket says 172.21.2.101:59102) 2021-02-10T11:43:35.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.620 7fb369d10700 1 -- 172.21.2.101:0/894508260 learned_addr learned my addr 172.21.2.101:0/894508260 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:35.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.620 7fb369d10700 1 -- 172.21.2.101:0/894508260 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb3640c2370 msgr2=0x7fb36418f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:35.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.620 7fb369d10700 1 --2- 172.21.2.101:0/894508260 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb3640c2370 0x7fb36418f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:35.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.620 7fb369d10700 1 -- 172.21.2.101:0/894508260 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb364084cb0 msgr2=0x7fb3641a7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:35.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.620 7fb369d10700 1 --2- 172.21.2.101:0/894508260 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb364084cb0 0x7fb3641a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:35.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.620 7fb369d10700 1 -- 172.21.2.101:0/894508260 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3641a80b0 con 0x7fb3641a7a90 2021-02-10T11:43:35.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.620 7fb369d10700 1 --2- 172.21.2.101:0/894508260 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3641a7a90 0x7fb364085700 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 rx=0x7fb35c009980 tx=0x7fb35c00bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:35.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.621 7fb355ffb700 1 -- 172.21.2.101:0/894508260 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb35c005870 con 0x7fb3641a7a90 2021-02-10T11:43:35.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.621 7fb36b6f3700 1 -- 172.21.2.101:0/894508260 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3641a80b0 con 0x7fb3641a7a90 2021-02-10T11:43:35.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.621 7fb355ffb700 1 -- 172.21.2.101:0/894508260 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35c00be10 con 0x7fb3641a7a90 2021-02-10T11:43:35.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.621 7fb355ffb700 1 -- 172.21.2.101:0/894508260 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb35c005af0 con 0x7fb3641a7a90 2021-02-10T11:43:35.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.621 7fb36b6f3700 1 -- 172.21.2.101:0/894508260 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3641a80b0 con 0x7fb3641a7a90 2021-02-10T11:43:35.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.621 7fb355ffb700 1 -- 172.21.2.101:0/894508260 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb35c005870 con 0x7fb3641a7a90 2021-02-10T11:43:35.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.621 7fb355ffb700 1 --2- 172.21.2.101:0/894508260 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb3440186d0 0x7fb34401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:35.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.621 7fb355ffb700 1 -- 172.21.2.101:0/894508260 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb35c032b80 con 0x7fb3641a7a90 2021-02-10T11:43:35.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.621 7fb36950f700 1 --2- 172.21.2.101:0/894508260 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb3440186d0 0x7fb34401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:35.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.622 7fb36950f700 1 --2- 172.21.2.101:0/894508260 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb3440186d0 0x7fb34401abd0 crc :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:35.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.652 7fb36b6f3700 1 -- 172.21.2.101:0/894508260 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3640be840 con 0x7fb3641a7a90 2021-02-10T11:43:35.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.655 7fb355ffb700 1 -- 172.21.2.101:0/894508260 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb35c003020 con 0x7fb3641a7a90 2021-02-10T11:43:35.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.784 7fb36b6f3700 1 -- 172.21.2.101:0/894508260 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fb3640be840 con 0x7fb3641a7a90 2021-02-10T11:43:35.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.785 7fb355ffb700 1 -- 172.21.2.101:0/894508260 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v138) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb35c00be10 con 0x7fb3641a7a90 2021-02-10T11:43:35.785 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:43:35.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.786 7fb36b6f3700 1 -- 172.21.2.101:0/894508260 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb3440186d0 msgr2=0x7fb34401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:35.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.787 7fb36b6f3700 1 --2- 172.21.2.101:0/894508260 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb3440186d0 0x7fb34401abd0 crc :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:35.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.787 7fb36b6f3700 1 -- 172.21.2.101:0/894508260 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3641a7a90 msgr2=0x7fb364085700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:35.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.787 7fb36b6f3700 1 --2- 172.21.2.101:0/894508260 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3641a7a90 0x7fb364085700 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 rx=0x7fb35c009980 tx=0x7fb35c00bcd0).stop 2021-02-10T11:43:35.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.787 7fb36b6f3700 1 -- 172.21.2.101:0/894508260 shutdown_connections 2021-02-10T11:43:35.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.787 7fb36b6f3700 1 --2- 172.21.2.101:0/894508260 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb3641a7a90 0x7fb364085700 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:35.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.787 7fb36b6f3700 1 --2- 172.21.2.101:0/894508260 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb364084cb0 0x7fb3641a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:35.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.787 7fb36b6f3700 1 --2- 172.21.2.101:0/894508260 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb3440186d0 0x7fb34401abd0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:35.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.787 7fb36b6f3700 1 --2- 172.21.2.101:0/894508260 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb3640c2370 0x7fb36418f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:35.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.787 7fb36b6f3700 1 -- 172.21.2.101:0/894508260 >> 172.21.2.101:0/894508260 conn(0x7fb3640b5dc0 msgr2=0x7fb3640c2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:35.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.787 7fb36b6f3700 1 -- 172.21.2.101:0/894508260 shutdown_connections 2021-02-10T11:43:35.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:35.787 7fb36b6f3700 1 -- 172.21.2.101:0/894508260 wait complete. 2021-02-10T11:43:35.795 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:43:36.193 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:43:36.197 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:43:36.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.211 7f8542ff0700 1 Processor -- start 2021-02-10T11:43:36.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.211 7f8542ff0700 1 -- start start 2021-02-10T11:43:36.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.212 7f8542ff0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f853c0d1870 0x7f853c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:36.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.212 7f8542ff0700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f853c1b0ae0 0x7f853c108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:36.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.212 7f8542ff0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f853c0d1c70 0x7f853c108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:36.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.212 7f8542ff0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f853c0e5ef0 con 0x7f853c0d1c70 2021-02-10T11:43:36.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.212 7f8542ff0700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f853c109a60 con 0x7f853c1b0ae0 2021-02-10T11:43:36.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.212 7f8542ff0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853c088070 con 0x7f853c0d1870 2021-02-10T11:43:36.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.212 7f854160d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f853c0d1c70 0x7f853c108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:36.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.212 7f854160d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f853c0d1c70 0x7f853c108f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59110/0 (socket says 172.21.2.101:59110) 2021-02-10T11:43:36.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.212 7f854160d700 1 -- 172.21.2.101:0/355753832 learned_addr learned my addr 172.21.2.101:0/355753832 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:36.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.212 7f853bfff700 1 --2- 172.21.2.101:0/355753832 >> v2:172.21.2.108:3300/0 conn(0x7f853c1b0ae0 0x7f853c108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:36.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.212 7f8540e0c700 1 --2- 172.21.2.101:0/355753832 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f853c0d1870 0x7f853c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:36.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.212 7f854160d700 1 -- 172.21.2.101:0/355753832 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f853c0d1870 msgr2=0x7f853c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:36.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.212 7f854160d700 1 --2- 172.21.2.101:0/355753832 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f853c0d1870 0x7f853c1b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:36.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.212 7f854160d700 1 -- 172.21.2.101:0/355753832 >> v2:172.21.2.108:3300/0 conn(0x7f853c1b0ae0 msgr2=0x7f853c108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:36.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.212 7f854160d700 1 --2- 172.21.2.101:0/355753832 >> v2:172.21.2.108:3300/0 conn(0x7f853c1b0ae0 0x7f853c108940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:36.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.212 7f854160d700 1 -- 172.21.2.101:0/355753832 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853c109580 con 0x7f853c0d1c70 2021-02-10T11:43:36.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.212 7f853bfff700 1 --2- 172.21.2.101:0/355753832 >> v2:172.21.2.108:3300/0 conn(0x7f853c1b0ae0 0x7f853c108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:43:36.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.213 7f854160d700 1 --2- 172.21.2.101:0/355753832 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f853c0d1c70 0x7f853c108f60 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 rx=0x7f8534006b80 tx=0x7f8534009b10).ready entity=mon.0 client_cookie=84ce8c1615661421 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:36.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.213 7f853b7fe700 1 -- 172.21.2.101:0/355753832 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f85340022e0 con 0x7f853c0d1c70 2021-02-10T11:43:36.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.213 7f853b7fe700 1 -- 172.21.2.101:0/355753832 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8534002620 con 0x7f853c0d1c70 2021-02-10T11:43:36.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.213 7f853b7fe700 1 -- 172.21.2.101:0/355753832 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8534002a80 con 0x7f853c0d1c70 2021-02-10T11:43:36.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.213 7f8542ff0700 1 -- 172.21.2.101:0/355753832 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f853c0d1c70 msgr2=0x7f853c108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:36.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.213 7f8542ff0700 1 --2- 172.21.2.101:0/355753832 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f853c0d1c70 0x7f853c108f60 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 rx=0x7f8534006b80 tx=0x7f8534009b10).stop 2021-02-10T11:43:36.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.213 7f8542ff0700 1 -- 172.21.2.101:0/355753832 shutdown_connections 2021-02-10T11:43:36.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.213 7f8542ff0700 1 --2- 172.21.2.101:0/355753832 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f853c0d1c70 0x7f853c108f60 unknown :-1 s=CLOSED pgs=2585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:36.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.213 7f8542ff0700 1 --2- 172.21.2.101:0/355753832 >> v2:172.21.2.108:3300/0 conn(0x7f853c1b0ae0 0x7f853c108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:36.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.213 7f8542ff0700 1 --2- 172.21.2.101:0/355753832 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f853c0d1870 0x7f853c1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:36.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.213 7f8542ff0700 1 -- 172.21.2.101:0/355753832 >> 172.21.2.101:0/355753832 conn(0x7f853c0ce420 msgr2=0x7f853c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:36.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.213 7f8542ff0700 1 -- 172.21.2.101:0/355753832 shutdown_connections 2021-02-10T11:43:36.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.213 7f8542ff0700 1 -- 172.21.2.101:0/355753832 wait complete. 2021-02-10T11:43:36.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.229 7f8542ff0700 1 Processor -- start 2021-02-10T11:43:36.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.229 7f8542ff0700 1 -- start start 2021-02-10T11:43:36.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.229 7f8542ff0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f853c0a2580 0x7f853c0a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:36.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.229 7f8542ff0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f853c094c40 0x7f853c1a75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:36.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.229 7f8542ff0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f853c095040 0x7f853c1a7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:36.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.229 7f8542ff0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c188180 con 0x7f853c094c40 2021-02-10T11:43:36.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.229 7f8542ff0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c187e70 con 0x7f853c0a2580 2021-02-10T11:43:36.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.229 7f8542ff0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853c1a8220 con 0x7f853c095040 2021-02-10T11:43:36.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.229 7f853bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f853c094c40 0x7f853c1a75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:36.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.229 7f853bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f853c094c40 0x7f853c1a75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59118/0 (socket says 172.21.2.101:59118) 2021-02-10T11:43:36.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.229 7f853bfff700 1 -- 172.21.2.101:0/164990193 learned_addr learned my addr 172.21.2.101:0/164990193 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:36.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.230 7f853bfff700 1 -- 172.21.2.101:0/164990193 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f853c095040 msgr2=0x7f853c1a7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:36.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.230 7f853bfff700 1 --2- 172.21.2.101:0/164990193 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f853c095040 0x7f853c1a7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:36.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.230 7f853bfff700 1 -- 172.21.2.101:0/164990193 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f853c0a2580 msgr2=0x7f853c0a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:36.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.230 7f853bfff700 1 --2- 172.21.2.101:0/164990193 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f853c0a2580 0x7f853c0a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:36.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.230 7f853bfff700 1 -- 172.21.2.101:0/164990193 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853c188700 con 0x7f853c094c40 2021-02-10T11:43:36.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.230 7f853bfff700 1 --2- 172.21.2.101:0/164990193 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f853c094c40 0x7f853c1a75e0 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 rx=0x7f852c00b970 tx=0x7f852c00bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:36.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.230 7f85397fa700 1 -- 172.21.2.101:0/164990193 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f852c000d30 con 0x7f853c094c40 2021-02-10T11:43:36.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.230 7f85397fa700 1 -- 172.21.2.101:0/164990193 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852c006240 con 0x7f853c094c40 2021-02-10T11:43:36.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.230 7f85397fa700 1 -- 172.21.2.101:0/164990193 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f852c005660 con 0x7f853c094c40 2021-02-10T11:43:36.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.230 7f8542ff0700 1 -- 172.21.2.101:0/164990193 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f853c188700 con 0x7f853c094c40 2021-02-10T11:43:36.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.230 7f8542ff0700 1 -- 172.21.2.101:0/164990193 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853c188c90 con 0x7f853c094c40 2021-02-10T11:43:36.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.231 7f85397fa700 1 -- 172.21.2.101:0/164990193 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f852c005660 con 0x7f853c094c40 2021-02-10T11:43:36.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.231 7f85397fa700 1 --2- 172.21.2.101:0/164990193 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8520018730 0x7f852001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:36.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.231 7f85397fa700 1 -- 172.21.2.101:0/164990193 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f852c0329b0 con 0x7f853c094c40 2021-02-10T11:43:36.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.231 7f8540e0c700 1 --2- 172.21.2.101:0/164990193 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8520018730 0x7f852001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:36.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.231 7f8540e0c700 1 --2- 172.21.2.101:0/164990193 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8520018730 0x7f852001ac10 crc :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:36.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.261 7f8542ff0700 1 -- 172.21.2.101:0/164990193 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f853c082980 con 0x7f853c094c40 2021-02-10T11:43:36.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.263 7f85397fa700 1 -- 172.21.2.101:0/164990193 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f852c006220 con 0x7f853c094c40 2021-02-10T11:43:36.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.392 7f8542ff0700 1 -- 172.21.2.101:0/164990193 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f853c109a60 con 0x7f853c094c40 2021-02-10T11:43:36.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.393 7f85397fa700 1 -- 172.21.2.101:0/164990193 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v138) v1 ==== 104+0+10 (secure 0 0 0) 0x7f852c005660 con 0x7f853c094c40 2021-02-10T11:43:36.393 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:43:36.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.394 7f8542ff0700 1 -- 172.21.2.101:0/164990193 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8520018730 msgr2=0x7f852001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:36.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.394 7f8542ff0700 1 --2- 172.21.2.101:0/164990193 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8520018730 0x7f852001ac10 crc :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:36.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.394 7f8542ff0700 1 -- 172.21.2.101:0/164990193 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f853c094c40 msgr2=0x7f853c1a75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:36.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.394 7f8542ff0700 1 --2- 172.21.2.101:0/164990193 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f853c094c40 0x7f853c1a75e0 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 rx=0x7f852c00b970 tx=0x7f852c00bd10).stop 2021-02-10T11:43:36.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.395 7f8542ff0700 1 -- 172.21.2.101:0/164990193 shutdown_connections 2021-02-10T11:43:36.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.395 7f8542ff0700 1 --2- 172.21.2.101:0/164990193 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8520018730 0x7f852001ac10 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:36.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.395 7f8542ff0700 1 --2- 172.21.2.101:0/164990193 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f853c095040 0x7f853c1a7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:36.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.395 7f8542ff0700 1 --2- 172.21.2.101:0/164990193 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f853c094c40 0x7f853c1a75e0 unknown :-1 s=CLOSED pgs=2586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:36.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.395 7f8542ff0700 1 --2- 172.21.2.101:0/164990193 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f853c0a2580 0x7f853c0a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:36.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.395 7f8542ff0700 1 -- 172.21.2.101:0/164990193 >> 172.21.2.101:0/164990193 conn(0x7f853c0a2b30 msgr2=0x7f853c09ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:36.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.395 7f8542ff0700 1 -- 172.21.2.101:0/164990193 shutdown_connections 2021-02-10T11:43:36.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.395 7f8542ff0700 1 -- 172.21.2.101:0/164990193 wait complete. 2021-02-10T11:43:36.403 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:43:36.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.857 7fb104e84700 1 Processor -- start 2021-02-10T11:43:36.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.857 7fb104e84700 1 -- start start 2021-02-10T11:43:36.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.857 7fb104e84700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb1000d1c70 0x7fb1000cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:36.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.857 7fb104e84700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1000cf580 0x7fb1000cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:36.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.857 7fb104e84700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1000fe1c0 con 0x7fb1000d1870 2021-02-10T11:43:36.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.857 7fb104e84700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1001b0cb0 con 0x7fb1000cf580 2021-02-10T11:43:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.857 7fb104e84700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb100109a60 con 0x7fb1000d1c70 2021-02-10T11:43:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.857 7fb0ff314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1000cf580 0x7fb1000cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.857 7fb0ff314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1000cf580 0x7fb1000cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59126/0 (socket says 172.21.2.101:59126) 2021-02-10T11:43:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.857 7fb0ff314700 1 -- 172.21.2.101:0/3005387071 learned_addr learned my addr 172.21.2.101:0/3005387071 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.858 7fb0ff314700 1 -- 172.21.2.101:0/3005387071 >> v2:172.21.2.108:3300/0 conn(0x7fb1000d1c70 msgr2=0x7fb1000cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.858 7fb0ff314700 1 --2- 172.21.2.101:0/3005387071 >> v2:172.21.2.108:3300/0 conn(0x7fb1000d1c70 0x7fb1000cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.858 7fb0ff314700 1 -- 172.21.2.101:0/3005387071 >> v1:172.21.2.108:6789/0 conn(0x7fb1000d1870 legacy=0x7fb1001b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.858 7fb0ff314700 1 -- 172.21.2.101:0/3005387071 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb100088070 con 0x7fb1000cf580 2021-02-10T11:43:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.858 7fb0ff314700 1 --2- 172.21.2.101:0/3005387071 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1000cf580 0x7fb1000cf980 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 rx=0x7fb0f8006f00 tx=0x7fb0f8009f10).ready entity=mon.0 client_cookie=9e48fefb9b9716c0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.858 7fb0fdb11700 1 -- 172.21.2.101:0/3005387071 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0f800b3b0 con 0x7fb1000cf580 2021-02-10T11:43:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.858 7fb0fdb11700 1 -- 172.21.2.101:0/3005387071 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f800b630 con 0x7fb1000cf580 2021-02-10T11:43:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.858 7fb0fdb11700 1 -- 172.21.2.101:0/3005387071 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0f800bb50 con 0x7fb1000cf580 2021-02-10T11:43:36.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.858 7fb104e84700 1 -- 172.21.2.101:0/3005387071 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1000cf580 msgr2=0x7fb1000cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.858 7fb104e84700 1 --2- 172.21.2.101:0/3005387071 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1000cf580 0x7fb1000cf980 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 rx=0x7fb0f8006f00 tx=0x7fb0f8009f10).stop 2021-02-10T11:43:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.859 7fb104e84700 1 -- 172.21.2.101:0/3005387071 shutdown_connections 2021-02-10T11:43:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.859 7fb104e84700 1 --2- 172.21.2.101:0/3005387071 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1000cf580 0x7fb1000cf980 unknown :-1 s=CLOSED pgs=2587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.859 7fb104e84700 1 --2- 172.21.2.101:0/3005387071 >> v2:172.21.2.108:3300/0 conn(0x7fb1000d1c70 0x7fb1000cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.859 7fb104e84700 1 -- 172.21.2.101:0/3005387071 >> 172.21.2.101:0/3005387071 conn(0x7fb1000ce420 msgr2=0x7fb1000d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.859 7fb104e84700 1 -- 172.21.2.101:0/3005387071 shutdown_connections 2021-02-10T11:43:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.859 7fb104e84700 1 -- 172.21.2.101:0/3005387071 wait complete. 2021-02-10T11:43:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.874 7fb104e84700 1 Processor -- start 2021-02-10T11:43:36.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.874 7fb104e84700 1 -- start start 2021-02-10T11:43:36.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.874 7fb104e84700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1000c3820 0x7fb100084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:36.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.874 7fb104e84700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1000a0290 0x7fb1000852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:36.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.874 7fb104e84700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb100086270 0x7fb100086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:36.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.874 7fb104e84700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1000c0700 con 0x7fb1000c3820 2021-02-10T11:43:36.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.874 7fb104e84700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1000858f0 con 0x7fb100086270 2021-02-10T11:43:36.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.874 7fb104e84700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb100085b10 con 0x7fb1000a0290 2021-02-10T11:43:36.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.875 7fb0feb13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1000c3820 0x7fb100084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.875 7fb0feb13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1000c3820 0x7fb100084cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59128/0 (socket says 172.21.2.101:59128) 2021-02-10T11:43:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.875 7fb0feb13700 1 -- 172.21.2.101:0/3944769066 learned_addr learned my addr 172.21.2.101:0/3944769066 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.875 7fb0ff314700 1 --2- 172.21.2.101:0/3944769066 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb100086270 0x7fb100086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.875 7fb0feb13700 1 -- 172.21.2.101:0/3944769066 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1000a0290 msgr2=0x7fb1000852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.875 7fb0fe312700 1 --2- 172.21.2.101:0/3944769066 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1000a0290 0x7fb1000852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.875 7fb0feb13700 1 --2- 172.21.2.101:0/3944769066 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1000a0290 0x7fb1000852d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.875 7fb0feb13700 1 -- 172.21.2.101:0/3944769066 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb100086270 msgr2=0x7fb100086670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.875 7fb0feb13700 1 --2- 172.21.2.101:0/3944769066 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb100086270 0x7fb100086670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.875 7fb0feb13700 1 -- 172.21.2.101:0/3944769066 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb100086c90 con 0x7fb1000c3820 2021-02-10T11:43:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.875 7fb0fe312700 1 --2- 172.21.2.101:0/3944769066 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1000a0290 0x7fb1000852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:43:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.875 7fb0feb13700 1 --2- 172.21.2.101:0/3944769066 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1000c3820 0x7fb100084cb0 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 rx=0x7fb0f000b5b0 tx=0x7fb0f000b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.876 7fb0f77fe700 1 -- 172.21.2.101:0/3944769066 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0f000bd40 con 0x7fb1000c3820 2021-02-10T11:43:36.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.876 7fb0f77fe700 1 -- 172.21.2.101:0/3944769066 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f0006240 con 0x7fb1000c3820 2021-02-10T11:43:36.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.876 7fb104e84700 1 -- 172.21.2.101:0/3944769066 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb100086c90 con 0x7fb1000c3820 2021-02-10T11:43:36.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.876 7fb104e84700 1 -- 172.21.2.101:0/3944769066 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb100086c90 con 0x7fb1000c3820 2021-02-10T11:43:36.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.876 7fb0f77fe700 1 -- 172.21.2.101:0/3944769066 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0f0005720 con 0x7fb1000c3820 2021-02-10T11:43:36.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.876 7fb0f77fe700 1 -- 172.21.2.101:0/3944769066 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb0f00054a0 con 0x7fb1000c3820 2021-02-10T11:43:36.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.876 7fb0f77fe700 1 --2- 172.21.2.101:0/3944769066 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0e4018730 0x7fb0e401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:36.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.876 7fb0f77fe700 1 -- 172.21.2.101:0/3944769066 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb0f00327f0 con 0x7fb1000c3820 2021-02-10T11:43:36.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.876 7fb0fe312700 1 --2- 172.21.2.101:0/3944769066 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0e4018730 0x7fb0e401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:36.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.877 7fb0fe312700 1 --2- 172.21.2.101:0/3944769066 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0e4018730 0x7fb0e401ac10 crc :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:36.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.907 7fb104e84700 1 -- 172.21.2.101:0/3944769066 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb100082a70 con 0x7fb1000c3820 2021-02-10T11:43:36.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:36.910 7fb0f77fe700 1 -- 172.21.2.101:0/3944769066 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb0f0006030 con 0x7fb1000c3820 2021-02-10T11:43:37.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.037 7fb104e84700 1 -- 172.21.2.101:0/3944769066 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7fb100082a70 con 0x7fb1000c3820 2021-02-10T11:43:37.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.037 7fb0f77fe700 1 -- 172.21.2.101:0/3944769066 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v138) v1 ==== 100+0+10 (secure 0 0 0) 0x7fb0f000bd40 con 0x7fb1000c3820 2021-02-10T11:43:37.038 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:43:37.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.039 7fb104e84700 1 -- 172.21.2.101:0/3944769066 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0e4018730 msgr2=0x7fb0e401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:37.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.039 7fb104e84700 1 --2- 172.21.2.101:0/3944769066 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0e4018730 0x7fb0e401ac10 crc :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:37.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.039 7fb104e84700 1 -- 172.21.2.101:0/3944769066 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1000c3820 msgr2=0x7fb100084cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:37.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.039 7fb104e84700 1 --2- 172.21.2.101:0/3944769066 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1000c3820 0x7fb100084cb0 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 rx=0x7fb0f000b5b0 tx=0x7fb0f000b950).stop 2021-02-10T11:43:37.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.039 7fb104e84700 1 -- 172.21.2.101:0/3944769066 shutdown_connections 2021-02-10T11:43:37.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.039 7fb104e84700 1 --2- 172.21.2.101:0/3944769066 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0e4018730 0x7fb0e401ac10 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:37.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.039 7fb104e84700 1 --2- 172.21.2.101:0/3944769066 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1000a0290 0x7fb1000852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:37.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.039 7fb104e84700 1 --2- 172.21.2.101:0/3944769066 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb100086270 0x7fb100086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:37.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.039 7fb104e84700 1 --2- 172.21.2.101:0/3944769066 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1000c3820 0x7fb100084cb0 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:37.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.039 7fb104e84700 1 -- 172.21.2.101:0/3944769066 >> 172.21.2.101:0/3944769066 conn(0x7fb1000872c0 msgr2=0x7fb100095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:37.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.039 7fb104e84700 1 -- 172.21.2.101:0/3944769066 shutdown_connections 2021-02-10T11:43:37.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.039 7fb104e84700 1 -- 172.21.2.101:0/3944769066 wait complete. 2021-02-10T11:43:37.048 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:43:37.048 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:43:37.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.459 7fa682233700 1 Processor -- start 2021-02-10T11:43:37.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.460 7fa682233700 1 -- start start 2021-02-10T11:43:37.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.460 7fa682233700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa67c0d1870 0x7fa67c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:37.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.460 7fa682233700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa67c1b0a10 0x7fa67c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:37.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.460 7fa682233700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa67c0d1c70 0x7fa67c0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:37.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.460 7fa682233700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa67c0e5ed0 con 0x7fa67c0d1870 2021-02-10T11:43:37.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.460 7fa682233700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa67c0f0b10 con 0x7fa67c1b0a10 2021-02-10T11:43:37.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.460 7fa682233700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c0cff70 con 0x7fa67c0d1c70 2021-02-10T11:43:37.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.460 7fa67bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa67c0d1870 0x7fa67c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:37.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.460 7fa67b7fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa67c1b0a10 0x7fa67c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:37.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.460 7fa67b7fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa67c1b0a10 0x7fa67c0cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:41390/0 (socket says 172.21.2.101:41390) 2021-02-10T11:43:37.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.460 7fa67bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa67c0d1870 0x7fa67c1b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59136/0 (socket says 172.21.2.101:59136) 2021-02-10T11:43:37.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.460 7fa67b7fe700 1 -- 172.21.2.101:0/3336418012 learned_addr learned my addr 172.21.2.101:0/3336418012 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:37.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.460 7fa680850700 1 --2- 172.21.2.101:0/3336418012 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa67c0d1c70 0x7fa67c0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:37.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.461 7fa67b7fe700 1 -- 172.21.2.101:0/3336418012 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa67c0d1c70 msgr2=0x7fa67c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:37.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.461 7fa67b7fe700 1 --2- 172.21.2.101:0/3336418012 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa67c0d1c70 0x7fa67c0cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:37.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.461 7fa67b7fe700 1 -- 172.21.2.101:0/3336418012 >> v2:172.21.2.101:3300/0 conn(0x7fa67c0d1870 msgr2=0x7fa67c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:37.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.461 7fa67b7fe700 1 --2- 172.21.2.101:0/3336418012 >> v2:172.21.2.101:3300/0 conn(0x7fa67c0d1870 0x7fa67c1b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:37.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.461 7fa67b7fe700 1 -- 172.21.2.101:0/3336418012 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67c0d0150 con 0x7fa67c1b0a10 2021-02-10T11:43:37.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.461 7fa67bfff700 1 --2- 172.21.2.101:0/3336418012 >> v2:172.21.2.101:3300/0 conn(0x7fa67c0d1870 0x7fa67c1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:43:37.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.461 7fa67b7fe700 1 --2- 172.21.2.101:0/3336418012 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa67c1b0a10 0x7fa67c0cf330 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7fa670006700 tx=0x7fa670009b80).ready entity=mon.1 client_cookie=3c50b5223c05c4a4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:37.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.461 7fa67affd700 1 -- 172.21.2.101:0/3336418012 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa67000c0c0 con 0x7fa67c1b0a10 2021-02-10T11:43:37.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.461 7fa67affd700 1 -- 172.21.2.101:0/3336418012 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67000c380 con 0x7fa67c1b0a10 2021-02-10T11:43:37.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.461 7fa67affd700 1 -- 172.21.2.101:0/3336418012 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa67000c7d0 con 0x7fa67c1b0a10 2021-02-10T11:43:37.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.461 7fa682233700 1 -- 172.21.2.101:0/3336418012 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa67c1b0a10 msgr2=0x7fa67c0cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:37.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.461 7fa682233700 1 --2- 172.21.2.101:0/3336418012 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa67c1b0a10 0x7fa67c0cf330 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7fa670006700 tx=0x7fa670009b80).stop 2021-02-10T11:43:37.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.462 7fa682233700 1 -- 172.21.2.101:0/3336418012 shutdown_connections 2021-02-10T11:43:37.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.462 7fa682233700 1 --2- 172.21.2.101:0/3336418012 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa67c0d1c70 0x7fa67c0cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:37.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.462 7fa682233700 1 --2- 172.21.2.101:0/3336418012 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa67c1b0a10 0x7fa67c0cf330 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:37.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.462 7fa682233700 1 --2- 172.21.2.101:0/3336418012 >> v2:172.21.2.101:3300/0 conn(0x7fa67c0d1870 0x7fa67c1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:37.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.462 7fa682233700 1 -- 172.21.2.101:0/3336418012 >> 172.21.2.101:0/3336418012 conn(0x7fa67c0ce450 msgr2=0x7fa67c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:37.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.462 7fa682233700 1 -- 172.21.2.101:0/3336418012 shutdown_connections 2021-02-10T11:43:37.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.462 7fa682233700 1 -- 172.21.2.101:0/3336418012 wait complete. 2021-02-10T11:43:37.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.476 7fa682233700 1 Processor -- start 2021-02-10T11:43:37.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.476 7fa682233700 1 -- start start 2021-02-10T11:43:37.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.477 7fa682233700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa67c1942b0 0x7fa67c0b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:37.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.477 7fa682233700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa67c195070 0x7fa67c195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:37.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.477 7fa682233700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa67c0bb8e0 0x7fa67c0bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:37.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.477 7fa682233700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c1946b0 con 0x7fa67c1942b0 2021-02-10T11:43:37.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.477 7fa682233700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c0b3b70 con 0x7fa67c0bb8e0 2021-02-10T11:43:37.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.477 7fa682233700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c0b3d90 con 0x7fa67c195070 2021-02-10T11:43:37.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.477 7fa67bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa67c1942b0 0x7fa67c0b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:37.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.477 7fa67bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa67c1942b0 0x7fa67c0b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59140/0 (socket says 172.21.2.101:59140) 2021-02-10T11:43:37.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.477 7fa67bfff700 1 -- 172.21.2.101:0/1337696974 learned_addr learned my addr 172.21.2.101:0/1337696974 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:37.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.477 7fa680850700 1 --2- 172.21.2.101:0/1337696974 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa67c0bb8e0 0x7fa67c0bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:37.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.477 7fa67b7fe700 1 --2- 172.21.2.101:0/1337696974 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa67c195070 0x7fa67c195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:37.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.478 7fa67bfff700 1 -- 172.21.2.101:0/1337696974 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa67c195070 msgr2=0x7fa67c195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:37.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.478 7fa67bfff700 1 --2- 172.21.2.101:0/1337696974 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa67c195070 0x7fa67c195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:37.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.478 7fa67bfff700 1 -- 172.21.2.101:0/1337696974 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa67c0bb8e0 msgr2=0x7fa67c0bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:37.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.478 7fa67bfff700 1 --2- 172.21.2.101:0/1337696974 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa67c0bb8e0 0x7fa67c0bbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:37.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.478 7fa67bfff700 1 -- 172.21.2.101:0/1337696974 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67c1a49e0 con 0x7fa67c1942b0 2021-02-10T11:43:37.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.478 7fa67bfff700 1 --2- 172.21.2.101:0/1337696974 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa67c1942b0 0x7fa67c0b3550 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 rx=0x7fa67400b970 tx=0x7fa67400bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:37.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.478 7fa678ff9700 1 -- 172.21.2.101:0/1337696974 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa674000d30 con 0x7fa67c1942b0 2021-02-10T11:43:37.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.478 7fa678ff9700 1 -- 172.21.2.101:0/1337696974 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa674006240 con 0x7fa67c1942b0 2021-02-10T11:43:37.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.478 7fa678ff9700 1 -- 172.21.2.101:0/1337696974 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa674005660 con 0x7fa67c1942b0 2021-02-10T11:43:37.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.478 7fa682233700 1 -- 172.21.2.101:0/1337696974 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa67c1a49e0 con 0x7fa67c1942b0 2021-02-10T11:43:37.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.478 7fa682233700 1 -- 172.21.2.101:0/1337696974 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67c1a49e0 con 0x7fa67c1942b0 2021-02-10T11:43:37.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.479 7fa678ff9700 1 -- 172.21.2.101:0/1337696974 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa674005660 con 0x7fa67c1942b0 2021-02-10T11:43:37.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.479 7fa678ff9700 1 --2- 172.21.2.101:0/1337696974 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa65c0186d0 0x7fa65c01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:37.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.479 7fa678ff9700 1 -- 172.21.2.101:0/1337696974 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa674032a50 con 0x7fa67c1942b0 2021-02-10T11:43:37.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.479 7fa67b7fe700 1 --2- 172.21.2.101:0/1337696974 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa65c0186d0 0x7fa65c01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:37.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.479 7fa67b7fe700 1 --2- 172.21.2.101:0/1337696974 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa65c0186d0 0x7fa65c01abd0 crc :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:37.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.510 7fa682233700 1 -- 172.21.2.101:0/1337696974 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa67c0bc5f0 con 0x7fa67c1942b0 2021-02-10T11:43:37.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.512 7fa678ff9700 1 -- 172.21.2.101:0/1337696974 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa674006240 con 0x7fa67c1942b0 2021-02-10T11:43:37.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.640 7fa682233700 1 -- 172.21.2.101:0/1337696974 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa67c0bc5f0 con 0x7fa67c1942b0 2021-02-10T11:43:37.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.641 7fa678ff9700 1 -- 172.21.2.101:0/1337696974 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+2102 (secure 0 0 0) 0x7fa6740064e0 con 0x7fa67c1942b0 2021-02-10T11:43:37.641 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:43:37.642 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":114,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/371124351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":371124351},{"type":"v1","addr":"172.21.2.101:6835","nonce":371124351}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:43:20.159141","modified":"2021-02-10 11:43:22.169386","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7990":{"gid":7990,"name":"b","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/797061093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":797061093},{"type":"v1","addr":"172.21.2.108:6833","nonce":797061093}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-02-10T11:43:37.642 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 114 2021-02-10T11:43:37.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.643 7fa682233700 1 -- 172.21.2.101:0/1337696974 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa65c0186d0 msgr2=0x7fa65c01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:37.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.643 7fa682233700 1 --2- 172.21.2.101:0/1337696974 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa65c0186d0 0x7fa65c01abd0 crc :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:37.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.643 7fa682233700 1 -- 172.21.2.101:0/1337696974 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa67c1942b0 msgr2=0x7fa67c0b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:37.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.643 7fa682233700 1 --2- 172.21.2.101:0/1337696974 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa67c1942b0 0x7fa67c0b3550 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 rx=0x7fa67400b970 tx=0x7fa67400bd10).stop 2021-02-10T11:43:37.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.643 7fa682233700 1 -- 172.21.2.101:0/1337696974 shutdown_connections 2021-02-10T11:43:37.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.643 7fa682233700 1 --2- 172.21.2.101:0/1337696974 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa65c0186d0 0x7fa65c01abd0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:37.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.643 7fa682233700 1 --2- 172.21.2.101:0/1337696974 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa67c195070 0x7fa67c195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:37.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.643 7fa682233700 1 --2- 172.21.2.101:0/1337696974 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa67c0bb8e0 0x7fa67c0bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:37.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.643 7fa682233700 1 --2- 172.21.2.101:0/1337696974 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa67c1942b0 0x7fa67c0b3550 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:37.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.643 7fa682233700 1 -- 172.21.2.101:0/1337696974 >> 172.21.2.101:0/1337696974 conn(0x7fa67c0b3040 msgr2=0x7fa67c0c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:37.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.643 7fa682233700 1 -- 172.21.2.101:0/1337696974 shutdown_connections 2021-02-10T11:43:37.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:37.643 7fa682233700 1 -- 172.21.2.101:0/1337696974 wait complete. 2021-02-10T11:43:37.853 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 114, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:43:20.159141', 'modified': '2021-02-10 11:43:22.169386', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7990}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7990': {'gid': 7990, 'name': 'b', 'rank': 0, 'incarnation': 111, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/797061093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 797061093}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 797061093}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [29], 'metadata_pool': 28, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:43:37.853 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:43:37.853 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:43:38.251 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:43:38.251 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:43:38.251 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:43:38.251 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 7990, 2021-02-10T11:43:38.252 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:43:38.252 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:43:38.252 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 18.089174434, 2021-02-10T11:43:38.252 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 114, 2021-02-10T11:43:38.252 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 138, 2021-02-10T11:43:38.252 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 138, 2021-02-10T11:43:38.252 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 23.721865770000001 2021-02-10T11:43:38.252 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:43:38.255 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 7990, "want_state": "up:active", "state": "up:active", "rank_uptime": 18.089174434, "mdsmap_epoch": 114, "osdmap_epoch": 138, "osdmap_epoch_barrier": 138, "uptime": 23.721865770000001 } 2021-02-10T11:43:38.255 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:43:38.255 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-02-10T11:43:38.255 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:43:38.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:38.256 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:43:38.593 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:43:38.939 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:43:38.939 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:43:39.283 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:43:39.283 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:43:39.284 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:43:39.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:39.284 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:43:39.619 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:43:39.619 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2021-02-10T11:43:39.620 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:43:39.627 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:43:39.968 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:43:39.969 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:43:40.034 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:2021-02-10 11:43:40.034 7fab72e41f80 -1 init, newargv = 0x55da7e6b21a0 newargc=9 2021-02-10T11:43:40.035 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[159507]: starting ceph client 2021-02-10T11:43:40.039 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[159507]: starting fuse 2021-02-10T11:43:40.311 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:43:40.311 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:43:40.312 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:43:40.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:40.312 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:43:40.646 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:43:40.646 INFO:teuthology.orchestra.run.gibba001.stdout:44 2021-02-10T11:43:40.647 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2021-02-10T11:43:40.647 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:40.647 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:43:40.647 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:43:40.647 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:43:40.647 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:43:40.647 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:43:40.647 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:43:40.647 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:43:40.648 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-02-10T11:43:40.648 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:43:40.648 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:43:40.648 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:43:40.648 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:43:40.648 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:43:40.648 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:43:40.648 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:43:40.648 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:43:40.648 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:43:40.648 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:43:40.648 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:43:40.648 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:43:40.648 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:43:40.649 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.1")) 2021-02-10T11:43:40.649 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:43:41.011 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.1.159507.asok 2021-02-10T11:43:41.013 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.159507.asok 2021-02-10T11:43:41.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:41.014 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.159507.asok status 2021-02-10T11:43:41.405 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:43:41.405 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:43:41.405 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:43:41.405 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:43:41.405 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "1", 2021-02-10T11:43:41.405 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:43:41.405 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:43:41.406 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "159507", 2021-02-10T11:43:41.406 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:43:41.406 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:41.406 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:43:41.406 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:43:41.406 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 11700, 2021-02-10T11:43:41.406 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:43:41.406 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:43:41.406 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:43:41.406 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 11700 2021-02-10T11:43:41.406 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:41.406 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:43:41.406 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:41.406 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:43:41.407 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 91385305 2021-02-10T11:43:41.407 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:41.407 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:41.407 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:43:41.407 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:41.407 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:43:41.407 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 91385305 2021-02-10T11:43:41.407 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:41.407 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.11700 172.21.2.101:0/91385305", 2021-02-10T11:43:41.407 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/91385305", 2021-02-10T11:43:41.407 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:43:41.407 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 114, 2021-02-10T11:43:41.407 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 138, 2021-02-10T11:43:41.408 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:43:41.408 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:43:41.408 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:43:41.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:41.409 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:43:41.409 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:43:41.744 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:43:41.744 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:43:41.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:41.745 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:43:42.091 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2021-02-10T11:43:42.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.508 7f6c8681c700 1 Processor -- start 2021-02-10T11:43:42.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.508 7f6c8681c700 1 -- start start 2021-02-10T11:43:42.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.509 7f6c8681c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c8010b000 0x7f6c800cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:42.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.509 7f6c8681c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c800fe1d0 con 0x7f6c800d1880 2021-02-10T11:43:42.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.509 7f6c8681c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c801b0cc0 con 0x7f6c800d1c80 2021-02-10T11:43:42.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.509 7f6c8681c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c80109a70 con 0x7f6c8010b000 2021-02-10T11:43:42.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.509 7f6c7ffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f6c800d1880 0x7f6c801b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59654/0 (socket says 172.21.2.101:59654) 2021-02-10T11:43:42.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.509 7f6c7ffff700 1 -- 172.21.2.101:0/2333914342 learned_addr learned my addr 172.21.2.101:0/2333914342 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:42.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.509 7f6c7effd700 1 -- 172.21.2.101:0/2333914342 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3410498142 0 0) 0x7f6c70001180 con 0x7f6c800d1880 2021-02-10T11:43:42.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7effd700 1 -- 172.21.2.101:0/2333914342 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c68002fe0 con 0x7f6c800d1880 2021-02-10T11:43:42.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c84e39700 1 --2- 172.21.2.101:0/2333914342 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c8010b000 0x7f6c800cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:42.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7effd700 1 -- 172.21.2.101:0/2333914342 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3122672109 0 0) 0x7f6c70001180 con 0x7f6c800d1880 2021-02-10T11:43:42.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7effd700 1 -- 172.21.2.101:0/2333914342 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c8010b000 msgr2=0x7f6c800cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:42.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7effd700 1 --2- 172.21.2.101:0/2333914342 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c8010b000 0x7f6c800cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:42.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7effd700 1 -- 172.21.2.101:0/2333914342 >> v1:172.21.2.108:6789/0 conn(0x7f6c800d1c80 legacy=0x7f6c800cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:42.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7effd700 1 -- 172.21.2.101:0/2333914342 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8010b400 con 0x7f6c800d1880 2021-02-10T11:43:42.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7effd700 1 -- 172.21.2.101:0/2333914342 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f6c70001680 con 0x7f6c800d1880 2021-02-10T11:43:42.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7effd700 1 -- 172.21.2.101:0/2333914342 >> v1:172.21.2.101:6789/0 conn(0x7f6c800d1880 legacy=0x7f6c801b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:42.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7effd700 1 --2- 172.21.2.101:0/2333914342 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c68003a90 0x7f6c68005f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:42.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7effd700 1 --2- 172.21.2.101:0/2333914342 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c680067a0 0x7f6c68008ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:42.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7effd700 1 --2- 172.21.2.101:0/2333914342 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c680095a0 0x7f6c6800ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:42.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7effd700 1 -- 172.21.2.101:0/2333914342 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6800c2e0 con 0x7f6c68003a90 2021-02-10T11:43:42.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7effd700 1 -- 172.21.2.101:0/2333914342 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6800c500 con 0x7f6c680067a0 2021-02-10T11:43:42.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7effd700 1 -- 172.21.2.101:0/2333914342 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6800c720 con 0x7f6c680095a0 2021-02-10T11:43:42.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7ffff700 1 --2- 172.21.2.101:0/2333914342 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c68003a90 0x7f6c68005f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:42.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7f7fe700 1 --2- 172.21.2.101:0/2333914342 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c680067a0 0x7f6c68008ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:42.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7ffff700 1 -- 172.21.2.101:0/2333914342 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c680095a0 msgr2=0x7f6c6800ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:42.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7ffff700 1 --2- 172.21.2.101:0/2333914342 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c680095a0 0x7f6c6800ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:42.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7ffff700 1 -- 172.21.2.101:0/2333914342 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c680067a0 msgr2=0x7f6c68008ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:42.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7ffff700 1 --2- 172.21.2.101:0/2333914342 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c680067a0 0x7f6c68008ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:42.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7ffff700 1 -- 172.21.2.101:0/2333914342 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6800c9c0 con 0x7f6c68003a90 2021-02-10T11:43:42.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7f7fe700 1 --2- 172.21.2.101:0/2333914342 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c680067a0 0x7f6c68008ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:43:42.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.510 7f6c7ffff700 1 --2- 172.21.2.101:0/2333914342 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c68003a90 0x7f6c68005f00 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 rx=0x7f6c700014e0 tx=0x7f6c7004bd30).ready entity=mon.0 client_cookie=eb4580d193063486 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:42.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.511 7f6c7effd700 1 -- 172.21.2.101:0/2333914342 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6c7004e310 con 0x7f6c68003a90 2021-02-10T11:43:42.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.511 7f6c7effd700 1 -- 172.21.2.101:0/2333914342 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7004e610 con 0x7f6c68003a90 2021-02-10T11:43:42.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.511 7f6c7effd700 1 -- 172.21.2.101:0/2333914342 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6c7004ea60 con 0x7f6c68003a90 2021-02-10T11:43:42.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.511 7f6c8681c700 1 -- 172.21.2.101:0/2333914342 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c68003a90 msgr2=0x7f6c68005f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:42.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.511 7f6c8681c700 1 --2- 172.21.2.101:0/2333914342 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c68003a90 0x7f6c68005f00 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 rx=0x7f6c700014e0 tx=0x7f6c7004bd30).stop 2021-02-10T11:43:42.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.511 7f6c84e39700 1 -- 172.21.2.101:0/2333914342 reap_dead start 2021-02-10T11:43:42.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.511 7f6c8681c700 1 -- 172.21.2.101:0/2333914342 shutdown_connections 2021-02-10T11:43:42.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.511 7f6c8681c700 1 -- 172.21.2.101:0/2333914342 >> 172.21.2.101:0/2333914342 conn(0x7f6c800ce430 msgr2=0x7f6c800d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:42.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.511 7f6c8681c700 1 -- 172.21.2.101:0/2333914342 shutdown_connections 2021-02-10T11:43:42.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.511 7f6c8681c700 1 -- 172.21.2.101:0/2333914342 wait complete. 2021-02-10T11:43:42.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.526 7f6c8681c700 1 Processor -- start 2021-02-10T11:43:42.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.526 7f6c8681c700 1 -- start start 2021-02-10T11:43:42.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.526 7f6c8681c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c800b56c0 0x7f6c800b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:42.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.526 7f6c8681c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c800b51f0 0x7f6c8009c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:42.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.526 7f6c8681c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c8009c9e0 0x7f6c800ac950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:42.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.526 7f6c8681c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8009cf40 con 0x7f6c800b56c0 2021-02-10T11:43:42.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.526 7f6c8681c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8009d4c0 con 0x7f6c800b51f0 2021-02-10T11:43:42.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.527 7f6c8681c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8009d6e0 con 0x7f6c8009c9e0 2021-02-10T11:43:42.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.527 7f6c7ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c800b56c0 0x7f6c800b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:42.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.527 7f6c7ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c800b56c0 0x7f6c800b4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59172/0 (socket says 172.21.2.101:59172) 2021-02-10T11:43:42.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.527 7f6c7ffff700 1 -- 172.21.2.101:0/3831206313 learned_addr learned my addr 172.21.2.101:0/3831206313 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:42.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.527 7f6c7ffff700 1 -- 172.21.2.101:0/3831206313 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c8009c9e0 msgr2=0x7f6c800ac950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:42.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.527 7f6c7ffff700 1 --2- 172.21.2.101:0/3831206313 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c8009c9e0 0x7f6c800ac950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:42.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.527 7f6c7ffff700 1 -- 172.21.2.101:0/3831206313 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c800b51f0 msgr2=0x7f6c8009c3c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:42.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.527 7f6c7ffff700 1 --2- 172.21.2.101:0/3831206313 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c800b51f0 0x7f6c8009c3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:42.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.527 7f6c7ffff700 1 -- 172.21.2.101:0/3831206313 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c800acf70 con 0x7f6c800b56c0 2021-02-10T11:43:42.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.527 7f6c7ffff700 1 --2- 172.21.2.101:0/3831206313 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c800b56c0 0x7f6c800b4bd0 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 rx=0x7f6c7004e590 tx=0x7f6c7004e930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:42.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.527 7f6c7cff9700 1 -- 172.21.2.101:0/3831206313 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6c7004ed20 con 0x7f6c800b56c0 2021-02-10T11:43:42.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.527 7f6c7cff9700 1 -- 172.21.2.101:0/3831206313 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c700043a0 con 0x7f6c800b56c0 2021-02-10T11:43:42.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.527 7f6c7cff9700 1 -- 172.21.2.101:0/3831206313 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6c70004880 con 0x7f6c800b56c0 2021-02-10T11:43:42.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.527 7f6c8681c700 1 -- 172.21.2.101:0/3831206313 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c800acf70 con 0x7f6c800b56c0 2021-02-10T11:43:42.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.527 7f6c8681c700 1 -- 172.21.2.101:0/3831206313 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c800acf70 con 0x7f6c800b56c0 2021-02-10T11:43:42.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.528 7f6c7cff9700 1 -- 172.21.2.101:0/3831206313 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6c70004580 con 0x7f6c800b56c0 2021-02-10T11:43:42.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.528 7f6c7cff9700 1 --2- 172.21.2.101:0/3831206313 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c60018730 0x7f6c6001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:42.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.528 7f6c7cff9700 1 -- 172.21.2.101:0/3831206313 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6c70076760 con 0x7f6c800b56c0 2021-02-10T11:43:42.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.528 7f6c7f7fe700 1 --2- 172.21.2.101:0/3831206313 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c60018730 0x7f6c6001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:42.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.528 7f6c7f7fe700 1 --2- 172.21.2.101:0/3831206313 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c60018730 0x7f6c6001abc0 crc :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:42.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.559 7f6c8681c700 1 -- 172.21.2.101:0/3831206313 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c80082aa0 con 0x7f6c800b56c0 2021-02-10T11:43:42.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.561 7f6c7cff9700 1 -- 172.21.2.101:0/3831206313 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6c700045b0 con 0x7f6c800b56c0 2021-02-10T11:43:42.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:42.697 7f6c8681c700 1 -- 172.21.2.101:0/3831206313 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6c80082aa0 con 0x7f6c800b56c0 2021-02-10T11:43:43.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:43.600 7f6c7cff9700 1 -- 172.21.2.101:0/3831206313 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v115) v1 ==== 136+0+0 (secure 0 0 0) 0x7f6c700043a0 con 0x7f6c800b56c0 2021-02-10T11:43:43.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:43.619 7f6c8681c700 1 -- 172.21.2.101:0/3831206313 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c60018730 msgr2=0x7f6c6001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:43.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:43.619 7f6c8681c700 1 --2- 172.21.2.101:0/3831206313 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c60018730 0x7f6c6001abc0 crc :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:43.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:43.619 7f6c8681c700 1 -- 172.21.2.101:0/3831206313 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c800b56c0 msgr2=0x7f6c800b4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:43.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:43.619 7f6c8681c700 1 --2- 172.21.2.101:0/3831206313 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c800b56c0 0x7f6c800b4bd0 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 rx=0x7f6c7004e590 tx=0x7f6c7004e930).stop 2021-02-10T11:43:43.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:43.619 7f6c8681c700 1 -- 172.21.2.101:0/3831206313 shutdown_connections 2021-02-10T11:43:43.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:43.619 7f6c8681c700 1 --2- 172.21.2.101:0/3831206313 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6c60018730 0x7f6c6001abc0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:43.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:43.619 7f6c8681c700 1 --2- 172.21.2.101:0/3831206313 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6c8009c9e0 0x7f6c800ac950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:43.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:43.619 7f6c8681c700 1 --2- 172.21.2.101:0/3831206313 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6c800b51f0 0x7f6c8009c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:43.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:43.619 7f6c8681c700 1 --2- 172.21.2.101:0/3831206313 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6c800b56c0 0x7f6c800b4bd0 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:43.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:43.619 7f6c8681c700 1 -- 172.21.2.101:0/3831206313 >> 172.21.2.101:0/3831206313 conn(0x7f6c80097a30 msgr2=0x7f6c800c0400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:43.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:43.620 7f6c8681c700 1 -- 172.21.2.101:0/3831206313 shutdown_connections 2021-02-10T11:43:43.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:43.620 7f6c8681c700 1 -- 172.21.2.101:0/3831206313 wait complete. 2021-02-10T11:43:43.628 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs volume ls 2021-02-10T11:43:44.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.038 7f5159ec0700 1 Processor -- start 2021-02-10T11:43:44.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.038 7f5159ec0700 1 -- start start 2021-02-10T11:43:44.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.038 7f5159ec0700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f51540d0330 0x7f51540ef630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:44.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.038 7f5159ec0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f51540d1c60 0x7f51540efc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:44.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.038 7f5159ec0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51540f0b00 con 0x7f51540d1860 2021-02-10T11:43:44.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.038 7f5159ec0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51540e5ec0 con 0x7f51540d1c60 2021-02-10T11:43:44.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.038 7f5159ec0700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51541b0bd0 con 0x7f51540d0330 2021-02-10T11:43:44.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.038 7f5153fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f51540d1c60 0x7f51540efc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:44.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.038 7f5153fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f51540d1c60 0x7f51540efc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59180/0 (socket says 172.21.2.101:59180) 2021-02-10T11:43:44.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.038 7f5153fff700 1 -- 172.21.2.101:0/4124401300 learned_addr learned my addr 172.21.2.101:0/4124401300 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:44.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.039 7f5153fff700 1 -- 172.21.2.101:0/4124401300 >> v2:172.21.2.108:3300/0 conn(0x7f51540d0330 msgr2=0x7f51540ef630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:43:44.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.039 7f5153fff700 1 --2- 172.21.2.101:0/4124401300 >> v2:172.21.2.108:3300/0 conn(0x7f51540d0330 0x7f51540ef630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:44.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.039 7f5153fff700 1 -- 172.21.2.101:0/4124401300 >> v1:172.21.2.108:6789/0 conn(0x7f51540d1860 legacy=0x7f51541b03e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:43:44.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.039 7f5153fff700 1 -- 172.21.2.101:0/4124401300 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51540f2090 con 0x7f51540d1c60 2021-02-10T11:43:44.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.039 7f5153fff700 1 --2- 172.21.2.101:0/4124401300 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51540d1c60 0x7f51540efc50 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 rx=0x7f514c006f00 tx=0x7f514c009f10).ready entity=mon.0 client_cookie=2a4a8a005e2e29d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:44.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.039 7f51527fc700 1 -- 172.21.2.101:0/4124401300 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f514c00b3b0 con 0x7f51540d1c60 2021-02-10T11:43:44.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.039 7f51527fc700 1 -- 172.21.2.101:0/4124401300 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514c00b730 con 0x7f51540d1c60 2021-02-10T11:43:44.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.039 7f51527fc700 1 -- 172.21.2.101:0/4124401300 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f514c00bc50 con 0x7f51540d1c60 2021-02-10T11:43:44.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.039 7f5159ec0700 1 -- 172.21.2.101:0/4124401300 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51540d1c60 msgr2=0x7f51540efc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:44.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.039 7f5159ec0700 1 --2- 172.21.2.101:0/4124401300 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51540d1c60 0x7f51540efc50 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 rx=0x7f514c006f00 tx=0x7f514c009f10).stop 2021-02-10T11:43:44.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.039 7f5159ec0700 1 -- 172.21.2.101:0/4124401300 shutdown_connections 2021-02-10T11:43:44.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.039 7f5159ec0700 1 --2- 172.21.2.101:0/4124401300 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f51540d1c60 0x7f51540efc50 unknown :-1 s=CLOSED pgs=2595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:44.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.039 7f5159ec0700 1 --2- 172.21.2.101:0/4124401300 >> v2:172.21.2.108:3300/0 conn(0x7f51540d0330 0x7f51540ef630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:44.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.039 7f5159ec0700 1 -- 172.21.2.101:0/4124401300 >> 172.21.2.101:0/4124401300 conn(0x7f51540ce440 msgr2=0x7f51540d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:44.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.040 7f5159ec0700 1 -- 172.21.2.101:0/4124401300 shutdown_connections 2021-02-10T11:43:44.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.040 7f5159ec0700 1 -- 172.21.2.101:0/4124401300 wait complete. 2021-02-10T11:43:44.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.054 7f5159ec0700 1 Processor -- start 2021-02-10T11:43:44.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.054 7f5159ec0700 1 -- start start 2021-02-10T11:43:44.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.055 7f5159ec0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51541a78d0 0x7f51541a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:44.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.055 7f5159ec0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5154193090 0x7f5154193490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:44.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.055 7f5159ec0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5154193ab0 0x7f51540a80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:44.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.055 7f5159ec0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51540a0790 con 0x7f5154193ab0 2021-02-10T11:43:44.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.055 7f5159ec0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51540a7b50 con 0x7f5154193090 2021-02-10T11:43:44.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.055 7f5159ec0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51540a02f0 con 0x7f51541a78d0 2021-02-10T11:43:44.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.055 7f5153fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5154193ab0 0x7f51540a80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:44.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.055 7f5153fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5154193ab0 0x7f51540a80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59186/0 (socket says 172.21.2.101:59186) 2021-02-10T11:43:44.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.055 7f5153fff700 1 -- 172.21.2.101:0/1117365385 learned_addr learned my addr 172.21.2.101:0/1117365385 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:44.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.055 7f5153fff700 1 -- 172.21.2.101:0/1117365385 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51541a78d0 msgr2=0x7f51541a7cd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:44.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.055 7f5153fff700 1 --2- 172.21.2.101:0/1117365385 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51541a78d0 0x7f51541a7cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:44.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.055 7f5153fff700 1 -- 172.21.2.101:0/1117365385 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5154193090 msgr2=0x7f5154193490 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:44.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.055 7f5153fff700 1 --2- 172.21.2.101:0/1117365385 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5154193090 0x7f5154193490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:44.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.055 7f5153fff700 1 -- 172.21.2.101:0/1117365385 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5154193eb0 con 0x7f5154193ab0 2021-02-10T11:43:44.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.055 7f5153fff700 1 --2- 172.21.2.101:0/1117365385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5154193ab0 0x7f51540a80d0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 rx=0x7f514c009390 tx=0x7f514c00bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:44.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.055 7f514bfff700 1 -- 172.21.2.101:0/1117365385 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f514c003030 con 0x7f5154193ab0 2021-02-10T11:43:44.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.055 7f514bfff700 1 -- 172.21.2.101:0/1117365385 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514c0068e0 con 0x7f5154193ab0 2021-02-10T11:43:44.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.055 7f5159ec0700 1 -- 172.21.2.101:0/1117365385 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5154193eb0 con 0x7f5154193ab0 2021-02-10T11:43:44.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.055 7f5159ec0700 1 -- 172.21.2.101:0/1117365385 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51540a86f0 con 0x7f5154193ab0 2021-02-10T11:43:44.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.055 7f514bfff700 1 -- 172.21.2.101:0/1117365385 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f514c00c570 con 0x7f5154193ab0 2021-02-10T11:43:44.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.056 7f514bfff700 1 -- 172.21.2.101:0/1117365385 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f514c001270 con 0x7f5154193ab0 2021-02-10T11:43:44.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.056 7f514bfff700 1 --2- 172.21.2.101:0/1117365385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51380186d0 0x7f513801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:44.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.056 7f514bfff700 1 -- 172.21.2.101:0/1117365385 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f514c032b40 con 0x7f5154193ab0 2021-02-10T11:43:44.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.056 7f51537fe700 1 --2- 172.21.2.101:0/1117365385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51380186d0 0x7f513801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:44.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.056 7f51537fe700 1 --2- 172.21.2.101:0/1117365385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51380186d0 0x7f513801abd0 crc :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:44.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.088 7f5159ec0700 1 -- 172.21.2.101:0/1117365385 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51540ee0a0 con 0x7f5154193ab0 2021-02-10T11:43:44.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.091 7f514bfff700 1 -- 172.21.2.101:0/1117365385 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f514c009430 con 0x7f5154193ab0 2021-02-10T11:43:44.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.225 7f5159ec0700 1 -- 172.21.2.101:0/1117365385 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs volume ls", "target": ["mgr", ""]}) v1 -- 0x7f51540a86f0 con 0x7f51380186d0 2021-02-10T11:43:44.226 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.226 7f514bfff700 1 -- 172.21.2.101:0/1117365385 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7f513c006970 con 0x7f51380186d0 2021-02-10T11:43:44.226 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:43:44.227 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:44.227 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "cephfs" 2021-02-10T11:43:44.227 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:44.227 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:43:44.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.228 7f5159ec0700 1 -- 172.21.2.101:0/1117365385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51380186d0 msgr2=0x7f513801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:44.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.228 7f5159ec0700 1 --2- 172.21.2.101:0/1117365385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51380186d0 0x7f513801abd0 crc :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:44.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.228 7f5159ec0700 1 -- 172.21.2.101:0/1117365385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5154193ab0 msgr2=0x7f51540a80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:44.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.228 7f5159ec0700 1 --2- 172.21.2.101:0/1117365385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5154193ab0 0x7f51540a80d0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 rx=0x7f514c009390 tx=0x7f514c00bd40).stop 2021-02-10T11:43:44.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.228 7f5159ec0700 1 -- 172.21.2.101:0/1117365385 shutdown_connections 2021-02-10T11:43:44.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.228 7f5159ec0700 1 --2- 172.21.2.101:0/1117365385 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5154193ab0 0x7f51540a80d0 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:44.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.228 7f5159ec0700 1 --2- 172.21.2.101:0/1117365385 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5154193090 0x7f5154193490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:44.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.228 7f5159ec0700 1 --2- 172.21.2.101:0/1117365385 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f51380186d0 0x7f513801abd0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:44.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.228 7f5159ec0700 1 --2- 172.21.2.101:0/1117365385 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f51541a78d0 0x7f51541a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:44.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.228 7f5159ec0700 1 -- 172.21.2.101:0/1117365385 >> 172.21.2.101:0/1117365385 conn(0x7f5154190ae0 msgr2=0x7f51540bf070 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:44.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.228 7f5159ec0700 1 -- 172.21.2.101:0/1117365385 shutdown_connections 2021-02-10T11:43:44.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.228 7f5159ec0700 1 -- 172.21.2.101:0/1117365385 wait complete. 2021-02-10T11:43:44.236 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:43:44.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.653 7ff683ccb700 1 Processor -- start 2021-02-10T11:43:44.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.653 7ff683ccb700 1 -- start start 2021-02-10T11:43:44.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.654 7ff683ccb700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff67c0d1870 0x7ff67c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:44.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.654 7ff683ccb700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff67c0fe1c0 con 0x7ff67c1b0ae0 2021-02-10T11:43:44.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.654 7ff683ccb700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff67c109a60 con 0x7ff67c0cff70 2021-02-10T11:43:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.654 7ff683ccb700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff67c088070 con 0x7ff67c0d1870 2021-02-10T11:43:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.654 7ff6812e6700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff67c1b0ae0 0x7ff67c108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59688/0 (socket says 172.21.2.101:59688) 2021-02-10T11:43:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.654 7ff6812e6700 1 -- 172.21.2.101:0/2337602773 learned_addr learned my addr 172.21.2.101:0/2337602773 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.654 7ff680ae5700 1 -- 172.21.2.101:0/2337602773 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259407131 0 0) 0x7ff6700009b0 con 0x7ff67c1b0ae0 2021-02-10T11:43:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.654 7ff680ae5700 1 -- 172.21.2.101:0/2337602773 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff668002fe0 con 0x7ff67c1b0ae0 2021-02-10T11:43:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff681ae7700 1 --2- 172.21.2.101:0/2337602773 >> v2:172.21.2.108:3300/0 conn(0x7ff67c0d1870 0x7ff67c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff680ae5700 1 -- 172.21.2.101:0/2337602773 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 196953261 0 0) 0x7ff670001010 con 0x7ff67c1b0ae0 2021-02-10T11:43:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff680ae5700 1 -- 172.21.2.101:0/2337602773 >> v2:172.21.2.108:3300/0 conn(0x7ff67c0d1870 msgr2=0x7ff67c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff680ae5700 1 --2- 172.21.2.101:0/2337602773 >> v2:172.21.2.108:3300/0 conn(0x7ff67c0d1870 0x7ff67c1b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff680ae5700 1 -- 172.21.2.101:0/2337602773 >> v1:172.21.2.108:6789/0 conn(0x7ff67c0cff70 legacy=0x7ff67c109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff680ae5700 1 -- 172.21.2.101:0/2337602773 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c088260 con 0x7ff67c1b0ae0 2021-02-10T11:43:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff680ae5700 1 -- 172.21.2.101:0/2337602773 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7ff670001590 con 0x7ff67c1b0ae0 2021-02-10T11:43:44.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff680ae5700 1 -- 172.21.2.101:0/2337602773 >> v1:172.21.2.101:6789/0 conn(0x7ff67c1b0ae0 legacy=0x7ff67c108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff680ae5700 1 --2- 172.21.2.101:0/2337602773 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff668003aa0 0x7ff668005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff680ae5700 1 --2- 172.21.2.101:0/2337602773 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff668006790 0x7ff668008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff680ae5700 1 --2- 172.21.2.101:0/2337602773 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff668009590 0x7ff66800bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff680ae5700 1 -- 172.21.2.101:0/2337602773 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66800c300 con 0x7ff668003aa0 2021-02-10T11:43:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff680ae5700 1 -- 172.21.2.101:0/2337602773 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66800c520 con 0x7ff668006790 2021-02-10T11:43:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff680ae5700 1 -- 172.21.2.101:0/2337602773 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66800c740 con 0x7ff668009590 2021-02-10T11:43:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff681ae7700 1 --2- 172.21.2.101:0/2337602773 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff668003aa0 0x7ff668005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff6822e8700 1 --2- 172.21.2.101:0/2337602773 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff668009590 0x7ff66800bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff6812e6700 1 --2- 172.21.2.101:0/2337602773 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff668006790 0x7ff668008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff681ae7700 1 -- 172.21.2.101:0/2337602773 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff668009590 msgr2=0x7ff66800bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff681ae7700 1 --2- 172.21.2.101:0/2337602773 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff668009590 0x7ff66800bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff681ae7700 1 -- 172.21.2.101:0/2337602773 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff668006790 msgr2=0x7ff668008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff681ae7700 1 --2- 172.21.2.101:0/2337602773 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff668006790 0x7ff668008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:44.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff681ae7700 1 -- 172.21.2.101:0/2337602773 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66800c9e0 con 0x7ff668003aa0 2021-02-10T11:43:44.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff6822e8700 1 --2- 172.21.2.101:0/2337602773 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff668009590 0x7ff66800bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:43:44.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff6822e8700 1 -- 172.21.2.101:0/2337602773 reap_dead start 2021-02-10T11:43:44.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.655 7ff681ae7700 1 --2- 172.21.2.101:0/2337602773 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff668003aa0 0x7ff668005ef0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 rx=0x7ff678006a40 tx=0x7ff67804dbe0).ready entity=mon.0 client_cookie=e195ae0f4d59b6dd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:44.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.656 7ff680ae5700 1 -- 172.21.2.101:0/2337602773 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff678006540 con 0x7ff668003aa0 2021-02-10T11:43:44.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.656 7ff680ae5700 1 -- 172.21.2.101:0/2337602773 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff678002030 con 0x7ff668003aa0 2021-02-10T11:43:44.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.656 7ff680ae5700 1 -- 172.21.2.101:0/2337602773 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff678002490 con 0x7ff668003aa0 2021-02-10T11:43:44.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.656 7ff683ccb700 1 -- 172.21.2.101:0/2337602773 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff668003aa0 msgr2=0x7ff668005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:44.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.656 7ff683ccb700 1 --2- 172.21.2.101:0/2337602773 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff668003aa0 0x7ff668005ef0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 rx=0x7ff678006a40 tx=0x7ff67804dbe0).stop 2021-02-10T11:43:44.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.656 7ff683ccb700 1 -- 172.21.2.101:0/2337602773 shutdown_connections 2021-02-10T11:43:44.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.656 7ff683ccb700 1 --2- 172.21.2.101:0/2337602773 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff668003aa0 0x7ff668005ef0 unknown :-1 s=CLOSED pgs=2598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:44.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.656 7ff683ccb700 1 -- 172.21.2.101:0/2337602773 >> 172.21.2.101:0/2337602773 conn(0x7ff67c0ce420 msgr2=0x7ff67c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:44.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.657 7ff683ccb700 1 -- 172.21.2.101:0/2337602773 shutdown_connections 2021-02-10T11:43:44.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.657 7ff683ccb700 1 -- 172.21.2.101:0/2337602773 wait complete. 2021-02-10T11:43:44.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.671 7ff683ccb700 1 Processor -- start 2021-02-10T11:43:44.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.671 7ff683ccb700 1 -- start start 2021-02-10T11:43:44.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.672 7ff683ccb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff67c085bc0 0x7ff67c0863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:44.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.672 7ff683ccb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff67c0954c0 0x7ff67c096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:44.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.672 7ff683ccb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff67c0958c0 0x7ff67c096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:44.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.672 7ff683ccb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c095d80 con 0x7ff67c0958c0 2021-02-10T11:43:44.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.672 7ff683ccb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c0bfa60 con 0x7ff67c0954c0 2021-02-10T11:43:44.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.672 7ff683ccb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c085fc0 con 0x7ff67c085bc0 2021-02-10T11:43:44.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.672 7ff6822e8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff67c0958c0 0x7ff67c096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:44.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.672 7ff6822e8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff67c0958c0 0x7ff67c096920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59204/0 (socket says 172.21.2.101:59204) 2021-02-10T11:43:44.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.672 7ff6822e8700 1 -- 172.21.2.101:0/2555188041 learned_addr learned my addr 172.21.2.101:0/2555188041 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:44.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.672 7ff6822e8700 1 -- 172.21.2.101:0/2555188041 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff67c085bc0 msgr2=0x7ff67c0863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:44.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.672 7ff681ae7700 1 --2- 172.21.2.101:0/2555188041 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff67c085bc0 0x7ff67c0863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:44.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.672 7ff6812e6700 1 --2- 172.21.2.101:0/2555188041 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff67c0954c0 0x7ff67c096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:44.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.672 7ff6822e8700 1 --2- 172.21.2.101:0/2555188041 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff67c085bc0 0x7ff67c0863e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:44.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.672 7ff6822e8700 1 -- 172.21.2.101:0/2555188041 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff67c0954c0 msgr2=0x7ff67c096300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:44.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.672 7ff6822e8700 1 --2- 172.21.2.101:0/2555188041 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff67c0954c0 0x7ff67c096300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:44.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.672 7ff6822e8700 1 -- 172.21.2.101:0/2555188041 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c086c10 con 0x7ff67c0958c0 2021-02-10T11:43:44.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.672 7ff681ae7700 1 --2- 172.21.2.101:0/2555188041 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff67c085bc0 0x7ff67c0863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:43:44.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.673 7ff6812e6700 1 --2- 172.21.2.101:0/2555188041 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff67c0954c0 0x7ff67c096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:43:44.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.673 7ff6822e8700 1 --2- 172.21.2.101:0/2555188041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff67c0958c0 0x7ff67c096920 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 rx=0x7ff6740099c0 tx=0x7ff67400bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:44.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.673 7ff66e7fc700 1 -- 172.21.2.101:0/2555188041 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff67400c2f0 con 0x7ff67c0958c0 2021-02-10T11:43:44.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.673 7ff66e7fc700 1 -- 172.21.2.101:0/2555188041 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67400c5b0 con 0x7ff67c0958c0 2021-02-10T11:43:44.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.673 7ff66e7fc700 1 -- 172.21.2.101:0/2555188041 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff67400ca10 con 0x7ff67c0958c0 2021-02-10T11:43:44.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.673 7ff683ccb700 1 -- 172.21.2.101:0/2555188041 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67c086c10 con 0x7ff67c0958c0 2021-02-10T11:43:44.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.673 7ff683ccb700 1 -- 172.21.2.101:0/2555188041 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67c086c10 con 0x7ff67c0958c0 2021-02-10T11:43:44.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.674 7ff66e7fc700 1 -- 172.21.2.101:0/2555188041 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff674001110 con 0x7ff67c0958c0 2021-02-10T11:43:44.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.674 7ff66e7fc700 1 --2- 172.21.2.101:0/2555188041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6640186c0 0x7ff66401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:44.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.674 7ff66e7fc700 1 -- 172.21.2.101:0/2555188041 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff674032b90 con 0x7ff67c0958c0 2021-02-10T11:43:44.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.674 7ff681ae7700 1 --2- 172.21.2.101:0/2555188041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6640186c0 0x7ff66401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:44.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.674 7ff681ae7700 1 --2- 172.21.2.101:0/2555188041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6640186c0 0x7ff66401abc0 crc :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:44.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.704 7ff683ccb700 1 -- 172.21.2.101:0/2555188041 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff67c0fe1c0 con 0x7ff67c0958c0 2021-02-10T11:43:44.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.706 7ff66e7fc700 1 -- 172.21.2.101:0/2555188041 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff67400c2f0 con 0x7ff67c0958c0 2021-02-10T11:43:44.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.855 7ff683ccb700 1 -- 172.21.2.101:0/2555188041 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff67c0fe1c0 con 0x7ff67c0958c0 2021-02-10T11:43:44.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.855 7ff66e7fc700 1 -- 172.21.2.101:0/2555188041 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff674001110 con 0x7ff67c0958c0 2021-02-10T11:43:44.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.857 7ff683ccb700 1 -- 172.21.2.101:0/2555188041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6640186c0 msgr2=0x7ff66401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:44.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.857 7ff683ccb700 1 --2- 172.21.2.101:0/2555188041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6640186c0 0x7ff66401abc0 crc :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:44.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.857 7ff683ccb700 1 -- 172.21.2.101:0/2555188041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff67c0958c0 msgr2=0x7ff67c096920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:44.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.857 7ff683ccb700 1 --2- 172.21.2.101:0/2555188041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff67c0958c0 0x7ff67c096920 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 rx=0x7ff6740099c0 tx=0x7ff67400bd40).stop 2021-02-10T11:43:44.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.857 7ff683ccb700 1 -- 172.21.2.101:0/2555188041 shutdown_connections 2021-02-10T11:43:44.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.857 7ff683ccb700 1 --2- 172.21.2.101:0/2555188041 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff67c0958c0 0x7ff67c096920 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:44.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.857 7ff683ccb700 1 --2- 172.21.2.101:0/2555188041 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff67c0954c0 0x7ff67c096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:44.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.857 7ff683ccb700 1 --2- 172.21.2.101:0/2555188041 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6640186c0 0x7ff66401abc0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:44.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.857 7ff683ccb700 1 --2- 172.21.2.101:0/2555188041 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff67c085bc0 0x7ff67c0863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:44.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.857 7ff683ccb700 1 -- 172.21.2.101:0/2555188041 >> 172.21.2.101:0/2555188041 conn(0x7ff67c0bef00 msgr2=0x7ff67c09d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:44.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.857 7ff683ccb700 1 -- 172.21.2.101:0/2555188041 shutdown_connections 2021-02-10T11:43:44.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:44.857 7ff683ccb700 1 -- 172.21.2.101:0/2555188041 wait complete. 2021-02-10T11:43:44.869 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000634810 --group_name non_existent_group 2021-02-10T11:43:45.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.321 7f8d56fa2700 1 Processor -- start 2021-02-10T11:43:45.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.321 7f8d56fa2700 1 -- start start 2021-02-10T11:43:45.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.321 7f8d56fa2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d500d1880 0x7f8d501b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:45.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.321 7f8d56fa2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8d501b0af0 0x7f8d50108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:45.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.321 7f8d56fa2700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8d500d1c80 0x7f8d50108f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:45.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.321 7f8d56fa2700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d500e5f00 con 0x7f8d501b0af0 2021-02-10T11:43:45.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.321 7f8d56fa2700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d50109a70 con 0x7f8d500d1c80 2021-02-10T11:43:45.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.321 7f8d56fa2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d50088070 con 0x7f8d500d1880 2021-02-10T11:43:45.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.321 7f8d4ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8d501b0af0 0x7f8d50108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:45.337 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.321 7f8d4ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8d501b0af0 0x7f8d50108950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59212/0 (socket says 172.21.2.101:59212) 2021-02-10T11:43:45.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.321 7f8d4ffff700 1 -- 172.21.2.101:0/3472720676 learned_addr learned my addr 172.21.2.101:0/3472720676 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:45.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.321 7f8d4ffff700 1 -- 172.21.2.101:0/3472720676 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d500d1880 msgr2=0x7f8d501b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:45.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.321 7f8d4ffff700 1 --2- 172.21.2.101:0/3472720676 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d500d1880 0x7f8d501b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:45.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.321 7f8d4ffff700 1 -- 172.21.2.101:0/3472720676 >> v2:172.21.2.108:3300/0 conn(0x7f8d500d1c80 msgr2=0x7f8d50108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:45.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.321 7f8d4ffff700 1 --2- 172.21.2.101:0/3472720676 >> v2:172.21.2.108:3300/0 conn(0x7f8d500d1c80 0x7f8d50108f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:45.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.322 7f8d4ffff700 1 -- 172.21.2.101:0/3472720676 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d50109590 con 0x7f8d501b0af0 2021-02-10T11:43:45.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.322 7f8d4ffff700 1 --2- 172.21.2.101:0/3472720676 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d501b0af0 0x7f8d50108950 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 rx=0x7f8d44006700 tx=0x7f8d44009b80).ready entity=mon.0 client_cookie=71c35769d8343f94 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:45.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.322 7f8d4f7fe700 1 -- 172.21.2.101:0/3472720676 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d4400c0c0 con 0x7f8d501b0af0 2021-02-10T11:43:45.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.322 7f8d4f7fe700 1 -- 172.21.2.101:0/3472720676 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4400c340 con 0x7f8d501b0af0 2021-02-10T11:43:45.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.322 7f8d4f7fe700 1 -- 172.21.2.101:0/3472720676 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d4400c860 con 0x7f8d501b0af0 2021-02-10T11:43:45.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.322 7f8d56fa2700 1 -- 172.21.2.101:0/3472720676 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d501b0af0 msgr2=0x7f8d50108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:45.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.322 7f8d56fa2700 1 --2- 172.21.2.101:0/3472720676 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d501b0af0 0x7f8d50108950 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 rx=0x7f8d44006700 tx=0x7f8d44009b80).stop 2021-02-10T11:43:45.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.322 7f8d56fa2700 1 -- 172.21.2.101:0/3472720676 shutdown_connections 2021-02-10T11:43:45.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.323 7f8d56fa2700 1 --2- 172.21.2.101:0/3472720676 >> v2:172.21.2.108:3300/0 conn(0x7f8d500d1c80 0x7f8d50108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:45.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.323 7f8d56fa2700 1 --2- 172.21.2.101:0/3472720676 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d501b0af0 0x7f8d50108950 unknown :-1 s=CLOSED pgs=2600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:45.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.323 7f8d56fa2700 1 --2- 172.21.2.101:0/3472720676 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d500d1880 0x7f8d501b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:45.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.323 7f8d56fa2700 1 -- 172.21.2.101:0/3472720676 >> 172.21.2.101:0/3472720676 conn(0x7f8d500ce430 msgr2=0x7f8d500d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:45.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.323 7f8d56fa2700 1 -- 172.21.2.101:0/3472720676 shutdown_connections 2021-02-10T11:43:45.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.323 7f8d56fa2700 1 -- 172.21.2.101:0/3472720676 wait complete. 2021-02-10T11:43:45.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.337 7f8d56fa2700 1 Processor -- start 2021-02-10T11:43:45.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.337 7f8d56fa2700 1 -- start start 2021-02-10T11:43:45.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.338 7f8d56fa2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d500912c0 0x7f8d500b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:45.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.338 7f8d56fa2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d500b2570 0x7f8d5008e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:45.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.338 7f8d56fa2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d500916c0 0x7f8d5008e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:45.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.338 7f8d56fa2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5008da80 con 0x7f8d500912c0 2021-02-10T11:43:45.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.338 7f8d56fa2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5008ec40 con 0x7f8d500916c0 2021-02-10T11:43:45.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.338 7f8d56fa2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5008ee20 con 0x7f8d500b2570 2021-02-10T11:43:45.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.338 7f8d54dbe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d500912c0 0x7f8d500b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:45.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.338 7f8d54dbe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d500912c0 0x7f8d500b1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59218/0 (socket says 172.21.2.101:59218) 2021-02-10T11:43:45.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.338 7f8d54dbe700 1 -- 172.21.2.101:0/273310512 learned_addr learned my addr 172.21.2.101:0/273310512 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:45.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.338 7f8d54dbe700 1 -- 172.21.2.101:0/273310512 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d500b2570 msgr2=0x7f8d5008e000 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:45.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.338 7f8d54dbe700 1 --2- 172.21.2.101:0/273310512 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d500b2570 0x7f8d5008e000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:45.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.338 7f8d54dbe700 1 -- 172.21.2.101:0/273310512 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d500916c0 msgr2=0x7f8d5008e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:45.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.338 7f8d54dbe700 1 --2- 172.21.2.101:0/273310512 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d500916c0 0x7f8d5008e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:45.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.338 7f8d54dbe700 1 -- 172.21.2.101:0/273310512 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5008f040 con 0x7f8d500912c0 2021-02-10T11:43:45.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.339 7f8d54dbe700 1 --2- 172.21.2.101:0/273310512 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d500912c0 0x7f8d500b1f50 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 rx=0x7f8d40006f00 tx=0x7f8d4000b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:45.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.339 7f8d4d7fa700 1 -- 172.21.2.101:0/273310512 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d4000bd30 con 0x7f8d500912c0 2021-02-10T11:43:45.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.339 7f8d4d7fa700 1 -- 172.21.2.101:0/273310512 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d40000c90 con 0x7f8d500912c0 2021-02-10T11:43:45.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.339 7f8d56fa2700 1 -- 172.21.2.101:0/273310512 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5008f040 con 0x7f8d500912c0 2021-02-10T11:43:45.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.339 7f8d4d7fa700 1 -- 172.21.2.101:0/273310512 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8d400022a0 con 0x7f8d500912c0 2021-02-10T11:43:45.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.339 7f8d56fa2700 1 -- 172.21.2.101:0/273310512 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5008f040 con 0x7f8d500912c0 2021-02-10T11:43:45.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.339 7f8d4d7fa700 1 -- 172.21.2.101:0/273310512 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8d40005310 con 0x7f8d500912c0 2021-02-10T11:43:45.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.339 7f8d4d7fa700 1 --2- 172.21.2.101:0/273310512 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d30018730 0x7f8d3001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:45.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.339 7f8d4d7fa700 1 -- 172.21.2.101:0/273310512 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8d40032600 con 0x7f8d500912c0 2021-02-10T11:43:45.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.339 7f8d4ffff700 1 --2- 172.21.2.101:0/273310512 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d30018730 0x7f8d3001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:45.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.340 7f8d4ffff700 1 --2- 172.21.2.101:0/273310512 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d30018730 0x7f8d3001abc0 crc :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:45.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.371 7f8d56fa2700 1 -- 172.21.2.101:0/273310512 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d50082a30 con 0x7f8d500912c0 2021-02-10T11:43:45.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.374 7f8d4d7fa700 1 -- 172.21.2.101:0/273310512 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8d40006b90 con 0x7f8d500912c0 2021-02-10T11:43:45.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.499 7f8d56fa2700 1 -- 172.21.2.101:0/273310512 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000634810", "target": ["mgr", ""], "group_name": "non_existent_group"}) v1 -- 0x7f8d50082a30 con 0x7f8d30018730 2021-02-10T11:43:45.503 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:43:45.504 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:43:45.503 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:43:45.504 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:43:45.503 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:43:45.504 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:43:45.504 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:43:45.504 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:43:45.504 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:43:45.504 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:43:45.508 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:43:45.508 7f2cc39e1700 -1 mgr.server reply reply (2) No such file or directory subvolume group 'non_existent_group' does not exist 2021-02-10T11:43:45.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.509 7f8d4d7fa700 1 -- 172.21.2.101:0/273310512 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: -2 subvolume group 'non_existent_group' does not exist) v1 ==== 59+0+0 (crc 0 0 0) 0x7f8d440090b0 con 0x7f8d30018730 2021-02-10T11:43:45.514 INFO:teuthology.orchestra.run.gibba001.stderr:Error ENOENT: subvolume group 'non_existent_group' does not exist 2021-02-10T11:43:45.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.516 7f8d56fa2700 1 -- 172.21.2.101:0/273310512 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d30018730 msgr2=0x7f8d3001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:45.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.516 7f8d56fa2700 1 --2- 172.21.2.101:0/273310512 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d30018730 0x7f8d3001abc0 crc :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:45.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.516 7f8d56fa2700 1 -- 172.21.2.101:0/273310512 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d500912c0 msgr2=0x7f8d500b1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:45.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.516 7f8d56fa2700 1 --2- 172.21.2.101:0/273310512 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d500912c0 0x7f8d500b1f50 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 rx=0x7f8d40006f00 tx=0x7f8d4000b940).stop 2021-02-10T11:43:45.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.516 7f8d56fa2700 1 -- 172.21.2.101:0/273310512 shutdown_connections 2021-02-10T11:43:45.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.516 7f8d56fa2700 1 --2- 172.21.2.101:0/273310512 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d30018730 0x7f8d3001abc0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:45.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.516 7f8d56fa2700 1 --2- 172.21.2.101:0/273310512 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d500b2570 0x7f8d5008e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:45.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.516 7f8d56fa2700 1 --2- 172.21.2.101:0/273310512 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d500916c0 0x7f8d5008e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:45.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.516 7f8d56fa2700 1 --2- 172.21.2.101:0/273310512 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d500912c0 0x7f8d500b1f50 unknown :-1 s=CLOSED pgs=2601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:45.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.516 7f8d56fa2700 1 -- 172.21.2.101:0/273310512 >> 172.21.2.101:0/273310512 conn(0x7f8d5008b660 msgr2=0x7f8d501a7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:45.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.516 7f8d56fa2700 1 -- 172.21.2.101:0/273310512 shutdown_connections 2021-02-10T11:43:45.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:45.516 7f8d56fa2700 1 -- 172.21.2.101:0/273310512 wait complete. 2021-02-10T11:43:45.524 DEBUG:teuthology.orchestra.run:got remote process result: 2 2021-02-10T11:43:45.524 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:43:45.524 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:43:45.870 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:43:46.216 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:43:46.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:46.216 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:43:46.216 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:43:46.553 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:43:46.553 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:43:46.553 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:43:46.553 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:43:46.553 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:43:46.912 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[158840]: fuse finished with error 0 and tester_r 0 2021-02-10T11:43:46.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:46.913 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:43:46.914 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:43:47.248 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:43:47.248 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:43:47.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:43:47.249 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:43:47.583 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:47.583 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:43:47.583 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:43:47.920 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:43:47.920 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:43:47.921 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:43:47.921 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:43:47.921 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:43:48.274 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[159507]: fuse finished with error 0 and tester_r 0 2021-02-10T11:43:48.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:48.275 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:43:48.275 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:43:48.610 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:43:48.610 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:43:48.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:43:48.610 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:43:48.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:48.945 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:43:48.945 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:43:49.280 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:43:49.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:43:49.281 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:43:49.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:43:49.281 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:43:49.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:49.617 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:43:49.617 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:43:49.951 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:43:49.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:43:49.952 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:43:49.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:43:49.952 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:43:50.286 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-02-10T11:43:50.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.702 7f5c65a97700 1 Processor -- start 2021-02-10T11:43:50.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.702 7f5c65a97700 1 -- start start 2021-02-10T11:43:50.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.702 7f5c65a97700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f5c600d0340 0x7f5c600ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:50.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.702 7f5c65a97700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c600f0b10 con 0x7f5c600d1870 2021-02-10T11:43:50.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.702 7f5c65a97700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c601b0be0 con 0x7f5c600d1c70 2021-02-10T11:43:50.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.702 7f5c65a97700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c600e5ed0 con 0x7f5c600d0340 2021-02-10T11:43:50.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.702 7f5c5f7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f5c600d1870 0x7f5c601b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59732/0 (socket says 172.21.2.101:59732) 2021-02-10T11:43:50.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.702 7f5c5f7fe700 1 -- 172.21.2.101:0/52122875 learned_addr learned my addr 172.21.2.101:0/52122875 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:50.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5e7fc700 1 -- 172.21.2.101:0/52122875 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1711077801 0 0) 0x7f5c500009b0 con 0x7f5c600d1870 2021-02-10T11:43:50.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5e7fc700 1 -- 172.21.2.101:0/52122875 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c4c002fe0 con 0x7f5c600d1870 2021-02-10T11:43:50.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5effd700 1 --2- 172.21.2.101:0/52122875 >> v2:172.21.2.108:3300/0 conn(0x7f5c600d0340 0x7f5c600ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:50.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5e7fc700 1 -- 172.21.2.101:0/52122875 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2505867236 0 0) 0x7f5c50001010 con 0x7f5c600d1870 2021-02-10T11:43:50.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5e7fc700 1 -- 172.21.2.101:0/52122875 >> v2:172.21.2.108:3300/0 conn(0x7f5c600d0340 msgr2=0x7f5c600ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:50.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5e7fc700 1 --2- 172.21.2.101:0/52122875 >> v2:172.21.2.108:3300/0 conn(0x7f5c600d0340 0x7f5c600ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:50.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5e7fc700 1 -- 172.21.2.101:0/52122875 >> v1:172.21.2.108:6789/0 conn(0x7f5c600d1c70 legacy=0x7f5c600efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:50.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5e7fc700 1 -- 172.21.2.101:0/52122875 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c600f0450 con 0x7f5c600d1870 2021-02-10T11:43:50.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5e7fc700 1 -- 172.21.2.101:0/52122875 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f5c50001590 con 0x7f5c600d1870 2021-02-10T11:43:50.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5e7fc700 1 -- 172.21.2.101:0/52122875 >> v1:172.21.2.101:6789/0 conn(0x7f5c600d1870 legacy=0x7f5c601b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:50.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5e7fc700 1 --2- 172.21.2.101:0/52122875 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c4c003aa0 0x7f5c4c005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:50.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5e7fc700 1 --2- 172.21.2.101:0/52122875 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5c4c006770 0x7f5c4c008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:50.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5e7fc700 1 --2- 172.21.2.101:0/52122875 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c4c009590 0x7f5c4c00bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:50.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5e7fc700 1 -- 172.21.2.101:0/52122875 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c00c300 con 0x7f5c4c003aa0 2021-02-10T11:43:50.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5e7fc700 1 -- 172.21.2.101:0/52122875 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c00c520 con 0x7f5c4c006770 2021-02-10T11:43:50.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5e7fc700 1 -- 172.21.2.101:0/52122875 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c00c740 con 0x7f5c4c009590 2021-02-10T11:43:50.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5f7fe700 1 --2- 172.21.2.101:0/52122875 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c4c003aa0 0x7f5c4c005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:50.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5f7fe700 1 -- 172.21.2.101:0/52122875 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c4c009590 msgr2=0x7f5c4c00bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:50.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5f7fe700 1 --2- 172.21.2.101:0/52122875 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c4c009590 0x7f5c4c00bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:50.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5f7fe700 1 -- 172.21.2.101:0/52122875 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5c4c006770 msgr2=0x7f5c4c008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:50.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5f7fe700 1 --2- 172.21.2.101:0/52122875 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5c4c006770 0x7f5c4c008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:50.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5f7fe700 1 -- 172.21.2.101:0/52122875 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4c0509b0 con 0x7f5c4c003aa0 2021-02-10T11:43:50.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.703 7f5c5ffff700 1 -- 172.21.2.101:0/52122875 reap_dead start 2021-02-10T11:43:50.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.704 7f5c5f7fe700 1 --2- 172.21.2.101:0/52122875 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c4c003aa0 0x7f5c4c005ef0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 rx=0x7f5c50007560 tx=0x7f5c50007d10).ready entity=mon.0 client_cookie=d734d6d4bf3885cf server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:50.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.704 7f5c5e7fc700 1 -- 172.21.2.101:0/52122875 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5c500009b0 con 0x7f5c4c003aa0 2021-02-10T11:43:50.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.704 7f5c5e7fc700 1 -- 172.21.2.101:0/52122875 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c500013d0 con 0x7f5c4c003aa0 2021-02-10T11:43:50.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.704 7f5c5e7fc700 1 -- 172.21.2.101:0/52122875 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5c5000a710 con 0x7f5c4c003aa0 2021-02-10T11:43:50.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.704 7f5c65a97700 1 -- 172.21.2.101:0/52122875 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c4c003aa0 msgr2=0x7f5c4c005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:50.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.704 7f5c65a97700 1 --2- 172.21.2.101:0/52122875 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c4c003aa0 0x7f5c4c005ef0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 rx=0x7f5c50007560 tx=0x7f5c50007d10).stop 2021-02-10T11:43:50.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.704 7f5c65a97700 1 -- 172.21.2.101:0/52122875 shutdown_connections 2021-02-10T11:43:50.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.704 7f5c65a97700 1 --2- 172.21.2.101:0/52122875 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c4c003aa0 0x7f5c4c005ef0 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:50.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.704 7f5c65a97700 1 -- 172.21.2.101:0/52122875 >> 172.21.2.101:0/52122875 conn(0x7f5c600ce450 msgr2=0x7f5c600d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:50.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.705 7f5c65a97700 1 -- 172.21.2.101:0/52122875 shutdown_connections 2021-02-10T11:43:50.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.705 7f5c65a97700 1 -- 172.21.2.101:0/52122875 wait complete. 2021-02-10T11:43:50.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.719 7f5c65a97700 1 Processor -- start 2021-02-10T11:43:50.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.719 7f5c65a97700 1 -- start start 2021-02-10T11:43:50.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.720 7f5c65a97700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5c600bfa90 0x7f5c6018a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:50.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.720 7f5c65a97700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c600bfe90 0x7f5c6018a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:50.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.720 7f5c65a97700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c60192510 0x7f5c60192910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:50.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.720 7f5c65a97700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c60189b50 con 0x7f5c600bfe90 2021-02-10T11:43:50.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.720 7f5c65a97700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c60190d30 con 0x7f5c600bfa90 2021-02-10T11:43:50.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.720 7f5c65a97700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c60193d70 con 0x7f5c60192510 2021-02-10T11:43:50.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.720 7f5c5effd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c600bfe90 0x7f5c6018a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:50.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.720 7f5c5effd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c600bfe90 0x7f5c6018a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59252/0 (socket says 172.21.2.101:59252) 2021-02-10T11:43:50.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.720 7f5c5effd700 1 -- 172.21.2.101:0/385756637 learned_addr learned my addr 172.21.2.101:0/385756637 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:50.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.720 7f5c5effd700 1 -- 172.21.2.101:0/385756637 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c60192510 msgr2=0x7f5c60192910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:43:50.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.720 7f5c5effd700 1 --2- 172.21.2.101:0/385756637 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c60192510 0x7f5c60192910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:50.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.720 7f5c5effd700 1 -- 172.21.2.101:0/385756637 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5c600bfa90 msgr2=0x7f5c6018a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:50.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.720 7f5c5effd700 1 --2- 172.21.2.101:0/385756637 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5c600bfa90 0x7f5c6018a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:50.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.720 7f5c5effd700 1 -- 172.21.2.101:0/385756637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c60192f30 con 0x7f5c600bfe90 2021-02-10T11:43:50.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.720 7f5c5effd700 1 --2- 172.21.2.101:0/385756637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c600bfe90 0x7f5c6018a670 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 rx=0x7f5c48006f40 tx=0x7f5c4800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:50.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.720 7f5c57fff700 1 -- 172.21.2.101:0/385756637 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5c4800bd80 con 0x7f5c600bfe90 2021-02-10T11:43:50.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.720 7f5c57fff700 1 -- 172.21.2.101:0/385756637 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c48000c90 con 0x7f5c600bfe90 2021-02-10T11:43:50.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.720 7f5c57fff700 1 -- 172.21.2.101:0/385756637 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5c480022b0 con 0x7f5c600bfe90 2021-02-10T11:43:50.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.720 7f5c65a97700 1 -- 172.21.2.101:0/385756637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c60192f30 con 0x7f5c600bfe90 2021-02-10T11:43:50.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.720 7f5c65a97700 1 -- 172.21.2.101:0/385756637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c601934c0 con 0x7f5c600bfe90 2021-02-10T11:43:50.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.721 7f5c57fff700 1 -- 172.21.2.101:0/385756637 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5c4800a000 con 0x7f5c600bfe90 2021-02-10T11:43:50.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.721 7f5c57fff700 1 --2- 172.21.2.101:0/385756637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5c44018730 0x7f5c4401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:50.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.721 7f5c57fff700 1 -- 172.21.2.101:0/385756637 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5c48032780 con 0x7f5c600bfe90 2021-02-10T11:43:50.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.721 7f5c5f7fe700 1 --2- 172.21.2.101:0/385756637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5c44018730 0x7f5c4401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:50.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.722 7f5c5f7fe700 1 --2- 172.21.2.101:0/385756637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5c44018730 0x7f5c4401abc0 crc :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:50.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.752 7f5c65a97700 1 -- 172.21.2.101:0/385756637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c600ebc50 con 0x7f5c600bfe90 2021-02-10T11:43:50.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.755 7f5c57fff700 1 -- 172.21.2.101:0/385756637 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5c48000940 con 0x7f5c600bfe90 2021-02-10T11:43:50.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.905 7f5c65a97700 1 -- 172.21.2.101:0/385756637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5c600f0b10 con 0x7f5c600bfe90 2021-02-10T11:43:50.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.909 7f5c57fff700 1 -- 172.21.2.101:0/385756637 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v35) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5c4800a000 con 0x7f5c600bfe90 2021-02-10T11:43:50.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.913 7f5c65a97700 1 -- 172.21.2.101:0/385756637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5c44018730 msgr2=0x7f5c4401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:50.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.913 7f5c65a97700 1 --2- 172.21.2.101:0/385756637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5c44018730 0x7f5c4401abc0 crc :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:50.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.913 7f5c65a97700 1 -- 172.21.2.101:0/385756637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c600bfe90 msgr2=0x7f5c6018a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:50.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.913 7f5c65a97700 1 --2- 172.21.2.101:0/385756637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c600bfe90 0x7f5c6018a670 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 rx=0x7f5c48006f40 tx=0x7f5c4800b990).stop 2021-02-10T11:43:50.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.913 7f5c65a97700 1 -- 172.21.2.101:0/385756637 shutdown_connections 2021-02-10T11:43:50.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.913 7f5c65a97700 1 --2- 172.21.2.101:0/385756637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5c44018730 0x7f5c4401abc0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:50.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.913 7f5c65a97700 1 --2- 172.21.2.101:0/385756637 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5c60192510 0x7f5c60192910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:50.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.913 7f5c65a97700 1 --2- 172.21.2.101:0/385756637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5c600bfe90 0x7f5c6018a670 unknown :-1 s=CLOSED pgs=2606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:50.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.913 7f5c65a97700 1 --2- 172.21.2.101:0/385756637 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5c600bfa90 0x7f5c6018a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:50.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.913 7f5c65a97700 1 -- 172.21.2.101:0/385756637 >> 172.21.2.101:0/385756637 conn(0x7f5c600ed860 msgr2=0x7f5c600bf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:50.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.914 7f5c65a97700 1 -- 172.21.2.101:0/385756637 shutdown_connections 2021-02-10T11:43:50.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:50.914 7f5c65a97700 1 -- 172.21.2.101:0/385756637 wait complete. 2021-02-10T11:43:50.922 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_group_create' 2021-02-10T11:43:51.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.334 7f25f9dc6700 1 Processor -- start 2021-02-10T11:43:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.334 7f25f9dc6700 1 -- start start 2021-02-10T11:43:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.335 7f25f9dc6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f25f40d0340 0x7f25f40ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.335 7f25f9dc6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25f40d1c70 0x7f25f40efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.335 7f25f9dc6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25f40f0b10 con 0x7f25f40d1870 2021-02-10T11:43:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.335 7f25f9dc6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25f40e5ed0 con 0x7f25f40d0340 2021-02-10T11:43:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.335 7f25f9dc6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25f41b0be0 con 0x7f25f40d1c70 2021-02-10T11:43:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.335 7f25f2ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f25f40d0340 0x7f25f40ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.335 7f25f2ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f25f40d0340 0x7f25f40ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59258/0 (socket says 172.21.2.101:59258) 2021-02-10T11:43:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.335 7f25f2ffd700 1 -- 172.21.2.101:0/3764081916 learned_addr learned my addr 172.21.2.101:0/3764081916 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.335 7f25f3fff700 1 --2- 172.21.2.101:0/3764081916 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25f40d1c70 0x7f25f40efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.335 7f25f2ffd700 1 -- 172.21.2.101:0/3764081916 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25f40d1c70 msgr2=0x7f25f40efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:51.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.335 7f25f2ffd700 1 --2- 172.21.2.101:0/3764081916 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25f40d1c70 0x7f25f40efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.335 7f25f2ffd700 1 -- 172.21.2.101:0/3764081916 >> v1:172.21.2.108:6789/0 conn(0x7f25f40d1870 legacy=0x7f25f41b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.335 7f25f2ffd700 1 -- 172.21.2.101:0/3764081916 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f40f20a0 con 0x7f25f40d0340 2021-02-10T11:43:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.335 7f25f3fff700 1 --2- 172.21.2.101:0/3764081916 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25f40d1c70 0x7f25f40efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:43:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.336 7f25f2ffd700 1 --2- 172.21.2.101:0/3764081916 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25f40d0340 0x7f25f40ef640 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 rx=0x7f25dc006700 tx=0x7f25dc009b80).ready entity=mon.0 client_cookie=3c877e62d4da004 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.336 7f25f27fc700 1 -- 172.21.2.101:0/3764081916 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f25dc00c0c0 con 0x7f25f40d0340 2021-02-10T11:43:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.336 7f25f27fc700 1 -- 172.21.2.101:0/3764081916 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25dc00c340 con 0x7f25f40d0340 2021-02-10T11:43:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.336 7f25f27fc700 1 -- 172.21.2.101:0/3764081916 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f25dc00c860 con 0x7f25f40d0340 2021-02-10T11:43:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.336 7f25f9dc6700 1 -- 172.21.2.101:0/3764081916 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25f40d0340 msgr2=0x7f25f40ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.336 7f25f9dc6700 1 --2- 172.21.2.101:0/3764081916 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25f40d0340 0x7f25f40ef640 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 rx=0x7f25dc006700 tx=0x7f25dc009b80).stop 2021-02-10T11:43:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.336 7f25f9dc6700 1 -- 172.21.2.101:0/3764081916 shutdown_connections 2021-02-10T11:43:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.336 7f25f9dc6700 1 --2- 172.21.2.101:0/3764081916 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25f40d1c70 0x7f25f40efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.336 7f25f9dc6700 1 --2- 172.21.2.101:0/3764081916 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25f40d0340 0x7f25f40ef640 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.336 7f25f9dc6700 1 -- 172.21.2.101:0/3764081916 >> 172.21.2.101:0/3764081916 conn(0x7f25f40ce450 msgr2=0x7f25f40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:51.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.337 7f25f9dc6700 1 -- 172.21.2.101:0/3764081916 shutdown_connections 2021-02-10T11:43:51.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.337 7f25f9dc6700 1 -- 172.21.2.101:0/3764081916 wait complete. 2021-02-10T11:43:51.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.351 7f25f9dc6700 1 Processor -- start 2021-02-10T11:43:51.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.351 7f25f9dc6700 1 -- start start 2021-02-10T11:43:51.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.352 7f25f9dc6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25f41953e0 0x7f25f40ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:51.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.352 7f25f9dc6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25f40aef20 0x7f25f41a8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:51.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.352 7f25f9dc6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25f41a9570 0x7f25f418b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:51.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.352 7f25f9dc6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f41a9be0 con 0x7f25f40aef20 2021-02-10T11:43:51.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.352 7f25f9dc6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f4090e10 con 0x7f25f41a9570 2021-02-10T11:43:51.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.352 7f25f9dc6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25f4194fb0 con 0x7f25f41953e0 2021-02-10T11:43:51.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.352 7f25f2ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25f40aef20 0x7f25f41a8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:51.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.352 7f25f2ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25f40aef20 0x7f25f41a8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59262/0 (socket says 172.21.2.101:59262) 2021-02-10T11:43:51.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.352 7f25f2ffd700 1 -- 172.21.2.101:0/1279942039 learned_addr learned my addr 172.21.2.101:0/1279942039 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:51.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.352 7f25f2ffd700 1 -- 172.21.2.101:0/1279942039 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25f41953e0 msgr2=0x7f25f40ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:51.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.352 7f25f2ffd700 1 --2- 172.21.2.101:0/1279942039 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25f41953e0 0x7f25f40ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:51.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.352 7f25f2ffd700 1 -- 172.21.2.101:0/1279942039 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25f41a9570 msgr2=0x7f25f418b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:51.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.352 7f25f2ffd700 1 --2- 172.21.2.101:0/1279942039 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25f41a9570 0x7f25f418b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:51.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.352 7f25f2ffd700 1 -- 172.21.2.101:0/1279942039 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f418b840 con 0x7f25f40aef20 2021-02-10T11:43:51.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.352 7f25f2ffd700 1 --2- 172.21.2.101:0/1279942039 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25f40aef20 0x7f25f41a8f50 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 rx=0x7f25dc009720 tx=0x7f25dc00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.352 7f25ebfff700 1 -- 172.21.2.101:0/1279942039 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f25dc0090c0 con 0x7f25f40aef20 2021-02-10T11:43:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.352 7f25ebfff700 1 -- 172.21.2.101:0/1279942039 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25dc002030 con 0x7f25f40aef20 2021-02-10T11:43:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.352 7f25f9dc6700 1 -- 172.21.2.101:0/1279942039 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f418b840 con 0x7f25f40aef20 2021-02-10T11:43:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.352 7f25ebfff700 1 -- 172.21.2.101:0/1279942039 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f25dc002440 con 0x7f25f40aef20 2021-02-10T11:43:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.353 7f25f9dc6700 1 -- 172.21.2.101:0/1279942039 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f418b840 con 0x7f25f40aef20 2021-02-10T11:43:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.353 7f25ebfff700 1 -- 172.21.2.101:0/1279942039 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f25dc000c90 con 0x7f25f40aef20 2021-02-10T11:43:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.353 7f25ebfff700 1 --2- 172.21.2.101:0/1279942039 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25d8018730 0x7f25d801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.353 7f25ebfff700 1 -- 172.21.2.101:0/1279942039 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f25dc032830 con 0x7f25f40aef20 2021-02-10T11:43:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.353 7f25f37fe700 1 --2- 172.21.2.101:0/1279942039 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25d8018730 0x7f25d801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:51.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.354 7f25f37fe700 1 --2- 172.21.2.101:0/1279942039 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25d8018730 0x7f25d801abc0 crc :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:51.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.385 7f25f9dc6700 1 -- 172.21.2.101:0/1279942039 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25f40ebc60 con 0x7f25f40aef20 2021-02-10T11:43:51.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.387 7f25ebfff700 1 -- 172.21.2.101:0/1279942039 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f25dc009030 con 0x7f25f40aef20 2021-02-10T11:43:51.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.519 7f25f9dc6700 1 -- 172.21.2.101:0/1279942039 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_group_create"]} v 0) v1 -- 0x7f25f40ebc60 con 0x7f25f40aef20 2021-02-10T11:43:51.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.940 7f25ebfff700 1 -- 172.21.2.101:0/1279942039 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_group_create"]}]=0 v686) v1 ==== 156+0+0 (secure 0 0 0) 0x7f25dc000c90 con 0x7f25f40aef20 2021-02-10T11:43:51.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.989 7f25f9dc6700 1 -- 172.21.2.101:0/1279942039 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25d8018730 msgr2=0x7f25d801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:51.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.989 7f25f9dc6700 1 --2- 172.21.2.101:0/1279942039 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25d8018730 0x7f25d801abc0 crc :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:51.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.989 7f25f9dc6700 1 -- 172.21.2.101:0/1279942039 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25f40aef20 msgr2=0x7f25f41a8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:51.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.989 7f25f9dc6700 1 --2- 172.21.2.101:0/1279942039 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25f40aef20 0x7f25f41a8f50 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 rx=0x7f25dc009720 tx=0x7f25dc00b960).stop 2021-02-10T11:43:51.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.989 7f25f9dc6700 1 -- 172.21.2.101:0/1279942039 shutdown_connections 2021-02-10T11:43:51.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.989 7f25f9dc6700 1 --2- 172.21.2.101:0/1279942039 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25f41a9570 0x7f25f418b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:51.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.989 7f25f9dc6700 1 --2- 172.21.2.101:0/1279942039 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25f40aef20 0x7f25f41a8f50 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:51.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.989 7f25f9dc6700 1 --2- 172.21.2.101:0/1279942039 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25d8018730 0x7f25d801abc0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:51.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.989 7f25f9dc6700 1 --2- 172.21.2.101:0/1279942039 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25f41953e0 0x7f25f40ae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:51.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.989 7f25f9dc6700 1 -- 172.21.2.101:0/1279942039 >> 172.21.2.101:0/1279942039 conn(0x7f25f4196190 msgr2=0x7f25f40ebe90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:51.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.989 7f25f9dc6700 1 -- 172.21.2.101:0/1279942039 shutdown_connections 2021-02-10T11:43:51.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:51.989 7f25f9dc6700 1 -- 172.21.2.101:0/1279942039 wait complete. 2021-02-10T11:43:51.997 INFO:tasks.cephfs_test_runner:test_nonexistent_subvolume_group_create (tasks.cephfs.test_volumes.TestVolumes) ... ok 2021-02-10T11:43:51.997 INFO:tasks.cephfs_test_runner:Starting test: test_nonexistent_subvolume_group_rm (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:43:51.998 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_group_rm' 2021-02-10T11:43:52.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.410 7fb1d4248700 1 Processor -- start 2021-02-10T11:43:52.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.410 7fb1d4248700 1 -- start start 2021-02-10T11:43:52.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.410 7fb1d4248700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1cc0d1c70 0x7fb1cc0f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:52.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.410 7fb1d4248700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1cc0f0b10 con 0x7fb1cc0d1870 2021-02-10T11:43:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.410 7fb1d4248700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1cc1b0be0 con 0x7fb1cc0d0340 2021-02-10T11:43:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.410 7fb1d4248700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1cc0e5ed0 con 0x7fb1cc0d1c70 2021-02-10T11:43:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.410 7fb1d2865700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1cc0d1c70 0x7fb1cc0f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.411 7fb1d2865700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb1cc0d1c70 0x7fb1cc0f20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59272/0 (socket says 172.21.2.101:59272) 2021-02-10T11:43:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.411 7fb1d2865700 1 -- 172.21.2.101:0/1484355598 learned_addr learned my addr 172.21.2.101:0/1484355598 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.411 7fb1d2865700 1 -- 172.21.2.101:0/1484355598 >> v1:172.21.2.108:6789/0 conn(0x7fb1cc0d0340 legacy=0x7fb1cc0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.411 7fb1d2865700 1 -- 172.21.2.101:0/1484355598 >> v1:172.21.2.101:6789/0 conn(0x7fb1cc0d1870 legacy=0x7fb1cc1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.411 7fb1d2865700 1 -- 172.21.2.101:0/1484355598 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc0efe30 con 0x7fb1cc0d1c70 2021-02-10T11:43:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.411 7fb1d2865700 1 --2- 172.21.2.101:0/1484355598 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1cc0d1c70 0x7fb1cc0f20a0 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 rx=0x7fb1c4006f00 tx=0x7fb1c4009f10).ready entity=mon.0 client_cookie=de4ebe540e2bb673 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.411 7fb1d1062700 1 -- 172.21.2.101:0/1484355598 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1c400b3b0 con 0x7fb1cc0d1c70 2021-02-10T11:43:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.411 7fb1d1062700 1 -- 172.21.2.101:0/1484355598 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c400b630 con 0x7fb1cc0d1c70 2021-02-10T11:43:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.411 7fb1d1062700 1 -- 172.21.2.101:0/1484355598 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1c400bb10 con 0x7fb1cc0d1c70 2021-02-10T11:43:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.411 7fb1d4248700 1 -- 172.21.2.101:0/1484355598 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1cc0d1c70 msgr2=0x7fb1cc0f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.411 7fb1d4248700 1 --2- 172.21.2.101:0/1484355598 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1cc0d1c70 0x7fb1cc0f20a0 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 rx=0x7fb1c4006f00 tx=0x7fb1c4009f10).stop 2021-02-10T11:43:52.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.412 7fb1d4248700 1 -- 172.21.2.101:0/1484355598 shutdown_connections 2021-02-10T11:43:52.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.412 7fb1d4248700 1 --2- 172.21.2.101:0/1484355598 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1cc0d1c70 0x7fb1cc0f20a0 unknown :-1 s=CLOSED pgs=2610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:52.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.412 7fb1d4248700 1 -- 172.21.2.101:0/1484355598 >> 172.21.2.101:0/1484355598 conn(0x7fb1cc0ce450 msgr2=0x7fb1cc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:52.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.412 7fb1d4248700 1 -- 172.21.2.101:0/1484355598 shutdown_connections 2021-02-10T11:43:52.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.412 7fb1d4248700 1 -- 172.21.2.101:0/1484355598 wait complete. 2021-02-10T11:43:52.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.427 7fb1d4248700 1 Processor -- start 2021-02-10T11:43:52.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.427 7fb1d4248700 1 -- start start 2021-02-10T11:43:52.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.427 7fb1d4248700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1cc0af040 0x7fb1cc0af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:52.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.427 7fb1d4248700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1cc0afa60 0x7fb1cc0ada20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:52.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.427 7fb1d4248700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1cc0afe60 0x7fb1cc0ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:52.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.427 7fb1d4248700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc0923c0 con 0x7fb1cc0afe60 2021-02-10T11:43:52.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.427 7fb1d4248700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc0b0410 con 0x7fb1cc0af040 2021-02-10T11:43:52.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.427 7fb1d4248700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc091e40 con 0x7fb1cc0afa60 2021-02-10T11:43:52.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.428 7fb1d2865700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1cc0afe60 0x7fb1cc0ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:52.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.428 7fb1d2865700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1cc0afe60 0x7fb1cc0ae040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59274/0 (socket says 172.21.2.101:59274) 2021-02-10T11:43:52.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.428 7fb1d2865700 1 -- 172.21.2.101:0/1217908154 learned_addr learned my addr 172.21.2.101:0/1217908154 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:52.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.428 7fb1d2865700 1 -- 172.21.2.101:0/1217908154 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1cc0afa60 msgr2=0x7fb1cc0ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:52.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.428 7fb1d2865700 1 --2- 172.21.2.101:0/1217908154 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1cc0afa60 0x7fb1cc0ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:52.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.428 7fb1d2865700 1 -- 172.21.2.101:0/1217908154 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1cc0af040 msgr2=0x7fb1cc0af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:52.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.428 7fb1d2865700 1 --2- 172.21.2.101:0/1217908154 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1cc0af040 0x7fb1cc0af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:52.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.428 7fb1d2865700 1 -- 172.21.2.101:0/1217908154 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc08f6d0 con 0x7fb1cc0afe60 2021-02-10T11:43:52.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.428 7fb1d2865700 1 --2- 172.21.2.101:0/1217908154 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1cc0afe60 0x7fb1cc0ae040 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 rx=0x7fb1c4009970 tx=0x7fb1c400bcc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.428 7fb1beffd700 1 -- 172.21.2.101:0/1217908154 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1c4009450 con 0x7fb1cc0afe60 2021-02-10T11:43:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.428 7fb1beffd700 1 -- 172.21.2.101:0/1217908154 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c4000cc0 con 0x7fb1cc0afe60 2021-02-10T11:43:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.428 7fb1beffd700 1 -- 172.21.2.101:0/1217908154 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1c400c4e0 con 0x7fb1cc0afe60 2021-02-10T11:43:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.428 7fb1d4248700 1 -- 172.21.2.101:0/1217908154 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1cc08f6d0 con 0x7fb1cc0afe60 2021-02-10T11:43:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.428 7fb1d4248700 1 -- 172.21.2.101:0/1217908154 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1cc0ae950 con 0x7fb1cc0afe60 2021-02-10T11:43:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.429 7fb1beffd700 1 -- 172.21.2.101:0/1217908154 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb1c4001270 con 0x7fb1cc0afe60 2021-02-10T11:43:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.429 7fb1beffd700 1 --2- 172.21.2.101:0/1217908154 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1b40186c0 0x7fb1b401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.429 7fb1beffd700 1 -- 172.21.2.101:0/1217908154 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb1c40327a0 con 0x7fb1cc0afe60 2021-02-10T11:43:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.429 7fb1d2064700 1 --2- 172.21.2.101:0/1217908154 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1b40186c0 0x7fb1b401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.429 7fb1d2064700 1 --2- 172.21.2.101:0/1217908154 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1b40186c0 0x7fb1b401abc0 crc :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:52.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.460 7fb1d4248700 1 -- 172.21.2.101:0/1217908154 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1cc0e7df0 con 0x7fb1cc0afe60 2021-02-10T11:43:52.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.463 7fb1beffd700 1 -- 172.21.2.101:0/1217908154 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb1c400be00 con 0x7fb1cc0afe60 2021-02-10T11:43:52.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.594 7fb1d4248700 1 -- 172.21.2.101:0/1217908154 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_group_rm"]} v 0) v1 -- 0x7fb1cc0ce450 con 0x7fb1cc0afe60 2021-02-10T11:43:52.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.946 7fb1beffd700 1 -- 172.21.2.101:0/1217908154 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_group_rm"]}]=0 v687) v1 ==== 155+0+0 (secure 0 0 0) 0x7fb1c402e020 con 0x7fb1cc0afe60 2021-02-10T11:43:52.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.962 7fb1d4248700 1 -- 172.21.2.101:0/1217908154 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1b40186c0 msgr2=0x7fb1b401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:52.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.962 7fb1d4248700 1 --2- 172.21.2.101:0/1217908154 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1b40186c0 0x7fb1b401abc0 crc :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:52.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.962 7fb1d4248700 1 -- 172.21.2.101:0/1217908154 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1cc0afe60 msgr2=0x7fb1cc0ae040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:52.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.962 7fb1d4248700 1 --2- 172.21.2.101:0/1217908154 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1cc0afe60 0x7fb1cc0ae040 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 rx=0x7fb1c4009970 tx=0x7fb1c400bcc0).stop 2021-02-10T11:43:52.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.962 7fb1d4248700 1 -- 172.21.2.101:0/1217908154 shutdown_connections 2021-02-10T11:43:52.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.962 7fb1d4248700 1 --2- 172.21.2.101:0/1217908154 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1b40186c0 0x7fb1b401abc0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:52.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.962 7fb1d4248700 1 --2- 172.21.2.101:0/1217908154 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1cc0afa60 0x7fb1cc0ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:52.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.962 7fb1d4248700 1 --2- 172.21.2.101:0/1217908154 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb1cc0afe60 0x7fb1cc0ae040 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:52.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.962 7fb1d4248700 1 --2- 172.21.2.101:0/1217908154 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1cc0af040 0x7fb1cc0af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:52.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.962 7fb1d4248700 1 -- 172.21.2.101:0/1217908154 >> 172.21.2.101:0/1217908154 conn(0x7fb1cc0a0030 msgr2=0x7fb1cc0f22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:52.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.963 7fb1d4248700 1 -- 172.21.2.101:0/1217908154 shutdown_connections 2021-02-10T11:43:52.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:52.963 7fb1d4248700 1 -- 172.21.2.101:0/1217908154 wait complete. 2021-02-10T11:43:52.971 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:43:53.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.387 7f197eecc700 1 Processor -- start 2021-02-10T11:43:53.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.387 7f197eecc700 1 -- start start 2021-02-10T11:43:53.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.387 7f197eecc700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f19780d1870 0x7f19781b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:53.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.388 7f197eecc700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19780fe1c0 con 0x7f19781b0ae0 2021-02-10T11:43:53.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.388 7f197eecc700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1978109a60 con 0x7f19780cff70 2021-02-10T11:43:53.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.388 7f197eecc700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1978088070 con 0x7f19780d1870 2021-02-10T11:43:53.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.388 7f1977fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f19781b0ae0 0x7f1978108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59776/0 (socket says 172.21.2.101:59776) 2021-02-10T11:43:53.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.388 7f1977fff700 1 -- 172.21.2.101:0/1320138792 learned_addr learned my addr 172.21.2.101:0/1320138792 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:53.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.388 7f19777fe700 1 -- 172.21.2.101:0/1320138792 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924317116 0 0) 0x7f1968001060 con 0x7f19781b0ae0 2021-02-10T11:43:53.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.388 7f19777fe700 1 -- 172.21.2.101:0/1320138792 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1964002fe0 con 0x7f19781b0ae0 2021-02-10T11:43:53.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.388 7f197cce8700 1 --2- 172.21.2.101:0/1320138792 >> v2:172.21.2.108:3300/0 conn(0x7f19780d1870 0x7f19781b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:53.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.388 7f19777fe700 1 -- 172.21.2.101:0/1320138792 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1677206124 0 0) 0x7f1968001460 con 0x7f19781b0ae0 2021-02-10T11:43:53.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.388 7f19777fe700 1 -- 172.21.2.101:0/1320138792 >> v2:172.21.2.108:3300/0 conn(0x7f19780d1870 msgr2=0x7f19781b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:53.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f19777fe700 1 --2- 172.21.2.101:0/1320138792 >> v2:172.21.2.108:3300/0 conn(0x7f19780d1870 0x7f19781b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:53.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f19777fe700 1 -- 172.21.2.101:0/1320138792 >> v1:172.21.2.108:6789/0 conn(0x7f19780cff70 legacy=0x7f1978109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:53.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f19777fe700 1 -- 172.21.2.101:0/1320138792 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1978088260 con 0x7f19781b0ae0 2021-02-10T11:43:53.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f19777fe700 1 -- 172.21.2.101:0/1320138792 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f19680019e0 con 0x7f19781b0ae0 2021-02-10T11:43:53.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f19777fe700 1 -- 172.21.2.101:0/1320138792 >> v1:172.21.2.101:6789/0 conn(0x7f19781b0ae0 legacy=0x7f1978108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:53.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f19777fe700 1 --2- 172.21.2.101:0/1320138792 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1964003a60 0x7f1964005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:53.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f19777fe700 1 --2- 172.21.2.101:0/1320138792 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1964006780 0x7f1964008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:53.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f19777fe700 1 --2- 172.21.2.101:0/1320138792 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1964009580 0x7f196400baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:53.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f19777fe700 1 -- 172.21.2.101:0/1320138792 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196400c2f0 con 0x7f1964006780 2021-02-10T11:43:53.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f19777fe700 1 -- 172.21.2.101:0/1320138792 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196400c510 con 0x7f1964009580 2021-02-10T11:43:53.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f19777fe700 1 -- 172.21.2.101:0/1320138792 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196400c730 con 0x7f1964003a60 2021-02-10T11:43:53.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f1977fff700 1 --2- 172.21.2.101:0/1320138792 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1964006780 0x7f1964008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:53.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f197d4e9700 1 --2- 172.21.2.101:0/1320138792 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1964009580 0x7f196400baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:53.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f1977fff700 1 -- 172.21.2.101:0/1320138792 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1964003a60 msgr2=0x7f1964005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:53.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f1977fff700 1 --2- 172.21.2.101:0/1320138792 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1964003a60 0x7f1964005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:53.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f1977fff700 1 -- 172.21.2.101:0/1320138792 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1964009580 msgr2=0x7f196400baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:53.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f1977fff700 1 --2- 172.21.2.101:0/1320138792 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1964009580 0x7f196400baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:53.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f1977fff700 1 -- 172.21.2.101:0/1320138792 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19640509a0 con 0x7f1964006780 2021-02-10T11:43:53.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f197d4e9700 1 -- 172.21.2.101:0/1320138792 reap_dead start 2021-02-10T11:43:53.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f1977fff700 1 --2- 172.21.2.101:0/1320138792 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1964006780 0x7f1964008c80 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 rx=0x7f1968007cb0 tx=0x7f196800a310).ready entity=mon.0 client_cookie=b1eb0698b252b545 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:53.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f19777fe700 1 -- 172.21.2.101:0/1320138792 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1968002060 con 0x7f1964006780 2021-02-10T11:43:53.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f19777fe700 1 -- 172.21.2.101:0/1320138792 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19680070c0 con 0x7f1964006780 2021-02-10T11:43:53.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f19777fe700 1 -- 172.21.2.101:0/1320138792 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1968007520 con 0x7f1964006780 2021-02-10T11:43:53.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f197eecc700 1 -- 172.21.2.101:0/1320138792 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1964006780 msgr2=0x7f1964008c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:53.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.389 7f197eecc700 1 --2- 172.21.2.101:0/1320138792 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1964006780 0x7f1964008c80 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 rx=0x7f1968007cb0 tx=0x7f196800a310).stop 2021-02-10T11:43:53.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.390 7f197eecc700 1 -- 172.21.2.101:0/1320138792 shutdown_connections 2021-02-10T11:43:53.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.390 7f197eecc700 1 --2- 172.21.2.101:0/1320138792 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1964006780 0x7f1964008c80 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:53.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.390 7f197eecc700 1 -- 172.21.2.101:0/1320138792 >> 172.21.2.101:0/1320138792 conn(0x7f19780ce420 msgr2=0x7f19780d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:53.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.390 7f197eecc700 1 -- 172.21.2.101:0/1320138792 shutdown_connections 2021-02-10T11:43:53.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.390 7f197eecc700 1 -- 172.21.2.101:0/1320138792 wait complete. 2021-02-10T11:43:53.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.405 7f197eecc700 1 Processor -- start 2021-02-10T11:43:53.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.405 7f197eecc700 1 -- start start 2021-02-10T11:43:53.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.405 7f197eecc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f197809bcc0 0x7f197809c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:53.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.405 7f197eecc700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19780a0820 0x7f19780a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:53.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.405 7f197eecc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19780a1240 0x7f19780858d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:53.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.405 7f197eecc700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19780a66b0 con 0x7f197809bcc0 2021-02-10T11:43:53.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.405 7f197eecc700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19781a49b0 con 0x7f19780a0820 2021-02-10T11:43:53.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.405 7f197eecc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19781a4b90 con 0x7f19780a1240 2021-02-10T11:43:53.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.405 7f197cce8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f197809bcc0 0x7f197809c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:53.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.405 7f197cce8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f197809bcc0 0x7f197809c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59294/0 (socket says 172.21.2.101:59294) 2021-02-10T11:43:53.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.405 7f197cce8700 1 -- 172.21.2.101:0/3611446969 learned_addr learned my addr 172.21.2.101:0/3611446969 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:53.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.406 7f197cce8700 1 -- 172.21.2.101:0/3611446969 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19780a1240 msgr2=0x7f19780858d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:53.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.406 7f197cce8700 1 --2- 172.21.2.101:0/3611446969 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19780a1240 0x7f19780858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:53.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.406 7f197cce8700 1 -- 172.21.2.101:0/3611446969 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19780a0820 msgr2=0x7f19780a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:53.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.406 7f197cce8700 1 --2- 172.21.2.101:0/3611446969 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19780a0820 0x7f19780a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:53.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.406 7f197cce8700 1 -- 172.21.2.101:0/3611446969 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19780a1640 con 0x7f197809bcc0 2021-02-10T11:43:53.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.406 7f197cce8700 1 --2- 172.21.2.101:0/3611446969 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f197809bcc0 0x7f197809c0c0 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 rx=0x7f1960006f40 tx=0x7f196000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:53.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.406 7f19757fa700 1 -- 172.21.2.101:0/3611446969 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19600009b0 con 0x7f197809bcc0 2021-02-10T11:43:53.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.406 7f19757fa700 1 -- 172.21.2.101:0/3611446969 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196000bd80 con 0x7f197809bcc0 2021-02-10T11:43:53.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.406 7f19757fa700 1 -- 172.21.2.101:0/3611446969 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19600022b0 con 0x7f197809bcc0 2021-02-10T11:43:53.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.406 7f197eecc700 1 -- 172.21.2.101:0/3611446969 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19780a1640 con 0x7f197809bcc0 2021-02-10T11:43:53.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.406 7f197eecc700 1 -- 172.21.2.101:0/3611446969 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19780861e0 con 0x7f197809bcc0 2021-02-10T11:43:53.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.407 7f19757fa700 1 -- 172.21.2.101:0/3611446969 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1960000c90 con 0x7f197809bcc0 2021-02-10T11:43:53.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.407 7f19757fa700 1 --2- 172.21.2.101:0/3611446969 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f195c0186c0 0x7f195c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:53.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.407 7f19757fa700 1 -- 172.21.2.101:0/3611446969 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f19600327e0 con 0x7f197809bcc0 2021-02-10T11:43:53.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.407 7f1977fff700 1 --2- 172.21.2.101:0/3611446969 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f195c0186c0 0x7f195c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:53.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.407 7f1977fff700 1 --2- 172.21.2.101:0/3611446969 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f195c0186c0 0x7f195c01abc0 crc :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:53.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.437 7f197eecc700 1 -- 172.21.2.101:0/3611446969 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19780fe1c0 con 0x7f197809bcc0 2021-02-10T11:43:53.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.440 7f19757fa700 1 -- 172.21.2.101:0/3611446969 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1960006bd0 con 0x7f197809bcc0 2021-02-10T11:43:53.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.589 7f197eecc700 1 -- 172.21.2.101:0/3611446969 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f19780fe1c0 con 0x7f197809bcc0 2021-02-10T11:43:53.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.593 7f19757fa700 1 -- 172.21.2.101:0/3611446969 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1960000c90 con 0x7f197809bcc0 2021-02-10T11:43:53.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.598 7f197eecc700 1 -- 172.21.2.101:0/3611446969 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f195c0186c0 msgr2=0x7f195c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:53.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.598 7f197eecc700 1 --2- 172.21.2.101:0/3611446969 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f195c0186c0 0x7f195c01abc0 crc :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:53.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.598 7f197eecc700 1 -- 172.21.2.101:0/3611446969 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f197809bcc0 msgr2=0x7f197809c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:53.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.598 7f197eecc700 1 --2- 172.21.2.101:0/3611446969 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f197809bcc0 0x7f197809c0c0 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 rx=0x7f1960006f40 tx=0x7f196000b990).stop 2021-02-10T11:43:53.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.599 7f197eecc700 1 -- 172.21.2.101:0/3611446969 shutdown_connections 2021-02-10T11:43:53.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.599 7f197eecc700 1 --2- 172.21.2.101:0/3611446969 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f195c0186c0 0x7f195c01abc0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:53.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.599 7f197eecc700 1 --2- 172.21.2.101:0/3611446969 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19780a1240 0x7f19780858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:53.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.599 7f197eecc700 1 --2- 172.21.2.101:0/3611446969 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19780a0820 0x7f19780a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:53.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.599 7f197eecc700 1 --2- 172.21.2.101:0/3611446969 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f197809bcc0 0x7f197809c0c0 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:53.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.599 7f197eecc700 1 -- 172.21.2.101:0/3611446969 >> 172.21.2.101:0/3611446969 conn(0x7f19780bef00 msgr2=0x7f197809d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:53.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.599 7f197eecc700 1 -- 172.21.2.101:0/3611446969 shutdown_connections 2021-02-10T11:43:53.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:53.599 7f197eecc700 1 -- 172.21.2.101:0/3611446969 wait complete. 2021-02-10T11:43:53.607 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:43:53.607 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:43:53.954 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:43:54.298 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:43:54.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:54.298 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:43:54.299 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:43:54.633 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2021-02-10T11:43:54.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:43:54.633 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-02-10T11:43:54.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:54.634 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:43:54.634 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:43:54.969 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:43:54.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:43:54.969 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:43:54.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:54.969 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:43:54.970 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:43:55.304 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:43:55.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:43:55.304 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:43:55.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:43:55.305 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:43:55.305 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:43:55.639 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:43:55.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:43:55.639 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:43:55.640 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json-pretty 2021-02-10T11:43:56.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.052 7f317a747700 1 Processor -- start 2021-02-10T11:43:56.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.052 7f317a747700 1 -- start start 2021-02-10T11:43:56.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.052 7f317a747700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f31740d1870 0x7f31741b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:56.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.052 7f317a747700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f31740f0650 0x7f31740cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:56.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.052 7f317a747700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31740f0b10 con 0x7f31741b0a10 2021-02-10T11:43:56.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.052 7f317a747700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31740e5ed0 con 0x7f31740f0650 2021-02-10T11:43:56.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.052 7f317a747700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31740d1c70 con 0x7f31740d1870 2021-02-10T11:43:56.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.053 7f3178d64700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f31740f0650 0x7f31740cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:56.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.053 7f3178d64700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f31740f0650 0x7f31740cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59300/0 (socket says 172.21.2.101:59300) 2021-02-10T11:43:56.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.053 7f3178d64700 1 -- 172.21.2.101:0/2142641482 learned_addr learned my addr 172.21.2.101:0/2142641482 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:56.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.053 7f3178d64700 1 -- 172.21.2.101:0/2142641482 >> v2:172.21.2.108:3300/0 conn(0x7f31740d1870 msgr2=0x7f31741b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:43:56.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.053 7f3178d64700 1 --2- 172.21.2.101:0/2142641482 >> v2:172.21.2.108:3300/0 conn(0x7f31740d1870 0x7f31741b03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:56.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.053 7f3178d64700 1 -- 172.21.2.101:0/2142641482 >> v1:172.21.2.108:6789/0 conn(0x7f31741b0a10 legacy=0x7f31740cf330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:43:56.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.053 7f3178d64700 1 -- 172.21.2.101:0/2142641482 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31740d1e90 con 0x7f31740f0650 2021-02-10T11:43:56.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.053 7f3178d64700 1 --2- 172.21.2.101:0/2142641482 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f31740f0650 0x7f31740cfb20 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 rx=0x7f316c006f00 tx=0x7f316c009f10).ready entity=mon.0 client_cookie=14de0f48aa02db83 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:56.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.053 7f3172ffd700 1 -- 172.21.2.101:0/2142641482 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f316c00b3b0 con 0x7f31740f0650 2021-02-10T11:43:56.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.053 7f3172ffd700 1 -- 172.21.2.101:0/2142641482 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316c00b630 con 0x7f31740f0650 2021-02-10T11:43:56.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.053 7f3172ffd700 1 -- 172.21.2.101:0/2142641482 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f316c00bb50 con 0x7f31740f0650 2021-02-10T11:43:56.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.053 7f317a747700 1 -- 172.21.2.101:0/2142641482 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f31740f0650 msgr2=0x7f31740cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:56.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.053 7f317a747700 1 --2- 172.21.2.101:0/2142641482 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f31740f0650 0x7f31740cfb20 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 rx=0x7f316c006f00 tx=0x7f316c009f10).stop 2021-02-10T11:43:56.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.054 7f317a747700 1 -- 172.21.2.101:0/2142641482 shutdown_connections 2021-02-10T11:43:56.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.054 7f317a747700 1 --2- 172.21.2.101:0/2142641482 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f31740f0650 0x7f31740cfb20 unknown :-1 s=CLOSED pgs=2615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:56.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.054 7f317a747700 1 --2- 172.21.2.101:0/2142641482 >> v2:172.21.2.108:3300/0 conn(0x7f31740d1870 0x7f31741b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:56.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.054 7f317a747700 1 -- 172.21.2.101:0/2142641482 >> 172.21.2.101:0/2142641482 conn(0x7f31740ce450 msgr2=0x7f31740d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:56.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.054 7f317a747700 1 -- 172.21.2.101:0/2142641482 shutdown_connections 2021-02-10T11:43:56.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.054 7f317a747700 1 -- 172.21.2.101:0/2142641482 wait complete. 2021-02-10T11:43:56.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.068 7f317a747700 1 Processor -- start 2021-02-10T11:43:56.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.068 7f317a747700 1 -- start start 2021-02-10T11:43:56.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.068 7f317a747700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f317419af00 0x7f317419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:56.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.068 7f317a747700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f317419ba80 0x7f317419be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:56.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.068 7f317a747700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f317419c4a0 0x7f317408a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:56.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.068 7f317a747700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317408fb50 con 0x7f317419af00 2021-02-10T11:43:56.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.068 7f317a747700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317408afb0 con 0x7f317419c4a0 2021-02-10T11:43:56.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.068 7f317a747700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317408b1a0 con 0x7f317419ba80 2021-02-10T11:43:56.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.069 7f3173fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f317419af00 0x7f317419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:56.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.069 7f3173fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f317419af00 0x7f317419b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59306/0 (socket says 172.21.2.101:59306) 2021-02-10T11:43:56.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.069 7f3173fff700 1 -- 172.21.2.101:0/3863323665 learned_addr learned my addr 172.21.2.101:0/3863323665 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:56.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.069 7f3173fff700 1 -- 172.21.2.101:0/3863323665 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f317419ba80 msgr2=0x7f317419be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:56.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.069 7f3173fff700 1 --2- 172.21.2.101:0/3863323665 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f317419ba80 0x7f317419be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:56.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.069 7f3173fff700 1 -- 172.21.2.101:0/3863323665 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f317419c4a0 msgr2=0x7f317408a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:56.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.069 7f3173fff700 1 --2- 172.21.2.101:0/3863323665 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f317419c4a0 0x7f317408a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:56.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.069 7f3173fff700 1 -- 172.21.2.101:0/3863323665 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317408b3c0 con 0x7f317419af00 2021-02-10T11:43:56.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.069 7f3173fff700 1 --2- 172.21.2.101:0/3863323665 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f317419af00 0x7f317419b300 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 rx=0x7f316400b5c0 tx=0x7f316400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:56.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.069 7f3170ff9700 1 -- 172.21.2.101:0/3863323665 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f316400bd50 con 0x7f317419af00 2021-02-10T11:43:56.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.069 7f3170ff9700 1 -- 172.21.2.101:0/3863323665 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3164006030 con 0x7f317419af00 2021-02-10T11:43:56.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.069 7f317a747700 1 -- 172.21.2.101:0/3863323665 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317408b3c0 con 0x7f317419af00 2021-02-10T11:43:56.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.069 7f3170ff9700 1 -- 172.21.2.101:0/3863323665 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f31640054d0 con 0x7f317419af00 2021-02-10T11:43:56.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.069 7f317a747700 1 -- 172.21.2.101:0/3863323665 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317408b950 con 0x7f317419af00 2021-02-10T11:43:56.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.070 7f3170ff9700 1 -- 172.21.2.101:0/3863323665 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3164006210 con 0x7f317419af00 2021-02-10T11:43:56.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.070 7f3170ff9700 1 --2- 172.21.2.101:0/3863323665 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3154018730 0x7f315401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:56.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.070 7f3170ff9700 1 -- 172.21.2.101:0/3863323665 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3164032680 con 0x7f317419af00 2021-02-10T11:43:56.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.070 7f31737fe700 1 --2- 172.21.2.101:0/3863323665 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3154018730 0x7f315401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:56.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.070 7f31737fe700 1 --2- 172.21.2.101:0/3863323665 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3154018730 0x7f315401ac10 crc :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:56.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.102 7f317a747700 1 -- 172.21.2.101:0/3863323665 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31740ee030 con 0x7f317419af00 2021-02-10T11:43:56.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.105 7f3170ff9700 1 -- 172.21.2.101:0/3863323665 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3164000940 con 0x7f317419af00 2021-02-10T11:43:56.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.230 7f317a747700 1 -- 172.21.2.101:0/3863323665 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f31740ce450 con 0x7f317419af00 2021-02-10T11:43:56.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.231 7f3170ff9700 1 -- 172.21.2.101:0/3863323665 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v138) v1 ==== 81+0+30374 (secure 0 0 0) 0x7f3164006030 con 0x7f317419af00 2021-02-10T11:43:56.231 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:43:56.231 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:43:56.231 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 138, 2021-02-10T11:43:56.231 INFO:teuthology.orchestra.run.gibba001.stdout: "fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:43:56.231 INFO:teuthology.orchestra.run.gibba001.stdout: "created": "2021-02-10 11:12:36.788366", 2021-02-10T11:43:56.231 INFO:teuthology.orchestra.run.gibba001.stdout: "modified": "2021-02-10 11:43:21.161685", 2021-02-10T11:43:56.231 INFO:teuthology.orchestra.run.gibba001.stdout: "last_up_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:43:56.231 INFO:teuthology.orchestra.run.gibba001.stdout: "last_in_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:43:56.232 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-02-10T11:43:56.232 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_num": 5799936, 2021-02-10T11:43:56.232 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_set": [ 2021-02-10T11:43:56.232 INFO:teuthology.orchestra.run.gibba001.stdout: "pglog_hardlimit", 2021-02-10T11:43:56.232 INFO:teuthology.orchestra.run.gibba001.stdout: "purged_snapdirs", 2021-02-10T11:43:56.232 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_deletes", 2021-02-10T11:43:56.232 INFO:teuthology.orchestra.run.gibba001.stdout: "sortbitwise" 2021-02-10T11:43:56.232 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:43:56.232 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_version": 4, 2021-02-10T11:43:56.232 INFO:teuthology.orchestra.run.gibba001.stdout: "full_ratio": 0.94999998807907104, 2021-02-10T11:43:56.232 INFO:teuthology.orchestra.run.gibba001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-02-10T11:43:56.232 INFO:teuthology.orchestra.run.gibba001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-02-10T11:43:56.232 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_snapshot": "", 2021-02-10T11:43:56.233 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_max": 29, 2021-02-10T11:43:56.233 INFO:teuthology.orchestra.run.gibba001.stdout: "max_osd": 8, 2021-02-10T11:43:56.233 INFO:teuthology.orchestra.run.gibba001.stdout: "require_min_compat_client": "jewel", 2021-02-10T11:43:56.233 INFO:teuthology.orchestra.run.gibba001.stdout: "min_compat_client": "jewel", 2021-02-10T11:43:56.233 INFO:teuthology.orchestra.run.gibba001.stdout: "require_osd_release": "nautilus", 2021-02-10T11:43:56.233 INFO:teuthology.orchestra.run.gibba001.stdout: "pools": [ 2021-02-10T11:43:56.233 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.233 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 1, 2021-02-10T11:43:56.233 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "rbd", 2021-02-10T11:43:56.233 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:13:08.037472", 2021-02-10T11:43:56.233 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:43:56.233 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:43:56.233 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:43:56.234 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:43:56.234 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:43:56.234 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:43:56.234 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:43:56.234 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:43:56.234 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:43:56.234 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:43:56.234 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:43:56.234 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:43:56.234 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:43:56.234 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:43:56.234 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:43:56.234 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:43:56.235 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:43:56.235 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:43:56.235 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:43:56.235 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:43:56.235 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.235 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "16", 2021-02-10T11:43:56.235 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:43:56.235 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:43:56.235 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:43:56.235 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:43:56.235 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:43:56.235 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:43:56.236 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:43:56.236 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:43:56.236 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:43:56.236 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:43:56.236 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:43:56.236 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:43:56.236 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:43:56.236 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:43:56.236 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:43:56.236 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:43:56.236 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:43:56.236 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:43:56.236 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:43:56.237 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:43:56.237 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:43:56.237 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:43:56.237 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:43:56.237 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:43:56.237 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:43:56.237 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:43:56.237 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.237 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:43:56.237 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:43:56.237 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:43:56.237 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:43:56.237 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:43:56.238 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:43:56.238 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:43:56.238 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:43:56.238 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:43:56.238 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:43:56.238 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:43:56.238 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:43:56.238 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:43:56.238 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd": {} 2021-02-10T11:43:56.238 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.238 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.238 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.238 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 28, 2021-02-10T11:43:56.239 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_metadata", 2021-02-10T11:43:56.239 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:43:18.076350", 2021-02-10T11:43:56.239 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:43:56.239 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:43:56.239 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:43:56.239 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:43:56.239 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:43:56.239 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:43:56.239 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:43:56.239 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:43:56.239 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:43:56.239 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:43:56.240 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:43:56.240 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:43:56.240 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:43:56.240 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:43:56.240 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:43:56.240 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:43:56.240 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:43:56.240 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:43:56.240 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:43:56.240 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:43:56.240 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.240 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "137", 2021-02-10T11:43:56.240 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:43:56.241 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:43:56.241 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:43:56.241 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:43:56.241 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:43:56.241 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:43:56.241 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:43:56.241 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:43:56.241 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:43:56.241 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:43:56.241 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:43:56.241 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:43:56.241 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:43:56.242 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:43:56.242 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:43:56.242 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:43:56.242 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:43:56.242 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:43:56.242 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:43:56.242 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:43:56.242 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:43:56.242 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:43:56.242 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:43:56.242 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:43:56.242 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:43:56.243 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:43:56.243 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.243 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:43:56.243 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:43:56.243 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:43:56.243 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:43:56.243 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:43:56.243 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:43:56.243 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:43:56.243 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:43:56.243 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:43:56.243 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:43:56.243 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:43:56.243 INFO:teuthology.orchestra.run.gibba001.stdout: "options": { 2021-02-10T11:43:56.244 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_bias": 4, 2021-02-10T11:43:56.244 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_min": 16, 2021-02-10T11:43:56.244 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_priority": 5 2021-02-10T11:43:56.244 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.244 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:43:56.244 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:43:56.244 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": "cephfs" 2021-02-10T11:43:56.244 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.244 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.244 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.244 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.244 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 29, 2021-02-10T11:43:56.244 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_data", 2021-02-10T11:43:56.245 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:43:19.152025", 2021-02-10T11:43:56.245 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:43:56.245 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:43:56.245 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:43:56.245 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:43:56.245 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:43:56.245 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:43:56.245 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:43:56.245 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:43:56.245 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:43:56.245 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:43:56.246 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:43:56.246 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:43:56.246 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:43:56.246 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:43:56.246 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:43:56.246 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:43:56.246 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:43:56.246 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:43:56.246 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:43:56.246 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:43:56.247 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.247 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "138", 2021-02-10T11:43:56.247 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:43:56.247 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:43:56.247 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:43:56.247 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:43:56.247 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:43:56.247 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:43:56.247 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:43:56.247 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:43:56.247 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:43:56.247 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:43:56.247 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:43:56.248 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:43:56.248 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:43:56.248 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:43:56.248 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:43:56.248 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:43:56.248 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:43:56.248 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:43:56.248 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:43:56.248 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:43:56.248 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:43:56.248 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:43:56.248 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:43:56.249 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:43:56.249 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:43:56.249 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:43:56.249 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.249 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:43:56.249 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:43:56.249 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:43:56.249 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:43:56.249 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:43:56.249 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:43:56.249 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:43:56.249 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:43:56.249 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:43:56.249 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:43:56.250 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:43:56.250 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:43:56.250 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:43:56.250 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:43:56.250 INFO:teuthology.orchestra.run.gibba001.stdout: "data": "cephfs" 2021-02-10T11:43:56.250 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.250 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.250 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.250 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:43:56.250 INFO:teuthology.orchestra.run.gibba001.stdout: "osds": [ 2021-02-10T11:43:56.250 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.250 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:43:56.251 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5", 2021-02-10T11:43:56.251 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:43:56.251 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:43:56.251 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:43:56.251 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:43:56.251 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:43:56.251 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:43:56.251 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:43:56.251 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 135, 2021-02-10T11:43:56.251 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:43:56.251 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:43:56.251 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:43:56.252 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.252 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.252 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.252 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6802", 2021-02-10T11:43:56.252 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:43:56.252 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.252 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.252 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.252 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6805", 2021-02-10T11:43:56.252 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:43:56.252 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.252 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.253 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.253 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:43:56.253 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.253 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.253 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.253 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6807", 2021-02-10T11:43:56.253 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:43:56.253 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.253 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.253 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.253 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6809", 2021-02-10T11:43:56.253 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:43:56.253 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.254 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.254 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.254 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:43:56.254 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.254 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.254 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.254 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6817", 2021-02-10T11:43:56.254 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:43:56.254 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.254 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.254 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.255 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6820", 2021-02-10T11:43:56.255 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:43:56.255 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.255 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.255 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.255 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:43:56.255 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.255 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.255 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.255 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6812", 2021-02-10T11:43:56.255 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:43:56.255 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.255 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.256 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.256 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6814", 2021-02-10T11:43:56.256 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:43:56.256 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.256 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.256 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.256 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6805/25093", 2021-02-10T11:43:56.256 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6809/25093", 2021-02-10T11:43:56.256 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6820/25093", 2021-02-10T11:43:56.256 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6814/25093", 2021-02-10T11:43:56.256 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:43:56.256 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:43:56.256 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:43:56.257 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.257 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.257 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.257 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:43:56.257 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "ddc04ce3-b313-4707-8270-585391e38ecc", 2021-02-10T11:43:56.257 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:43:56.257 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:43:56.257 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:43:56.257 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:43:56.257 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:43:56.257 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:43:56.257 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:43:56.257 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 136, 2021-02-10T11:43:56.258 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:43:56.258 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:43:56.258 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:43:56.258 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.258 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.258 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.258 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6803", 2021-02-10T11:43:56.258 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:43:56.258 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.258 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.258 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.258 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6804", 2021-02-10T11:43:56.259 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:43:56.259 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.259 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.259 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.259 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:43:56.259 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.259 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.259 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.259 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6806", 2021-02-10T11:43:56.259 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:43:56.259 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.259 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.260 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.260 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6808", 2021-02-10T11:43:56.260 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:43:56.260 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.260 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.260 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.260 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:43:56.260 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.260 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.260 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.260 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6818", 2021-02-10T11:43:56.260 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:43:56.260 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.261 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.261 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.261 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6821", 2021-02-10T11:43:56.261 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:43:56.261 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.261 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.261 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.261 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:43:56.261 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.261 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.261 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.261 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6810", 2021-02-10T11:43:56.261 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:43:56.262 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.262 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.262 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.262 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6815", 2021-02-10T11:43:56.262 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:43:56.262 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.262 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.262 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.262 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6804/25092", 2021-02-10T11:43:56.262 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6808/25092", 2021-02-10T11:43:56.262 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6821/25092", 2021-02-10T11:43:56.262 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6815/25092", 2021-02-10T11:43:56.262 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:43:56.263 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:43:56.263 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:43:56.263 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.263 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.263 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.263 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:43:56.263 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3", 2021-02-10T11:43:56.263 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:43:56.263 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:43:56.263 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:43:56.263 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:43:56.263 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:43:56.263 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:43:56.263 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:43:56.264 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 127, 2021-02-10T11:43:56.264 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:43:56.264 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:43:56.264 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:43:56.264 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.264 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.264 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.264 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6811", 2021-02-10T11:43:56.264 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:43:56.264 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.264 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.264 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.264 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6813", 2021-02-10T11:43:56.264 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:43:56.265 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.265 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.265 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.265 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:43:56.265 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.265 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.265 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.265 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6816", 2021-02-10T11:43:56.265 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:43:56.265 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.265 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.265 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.265 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6819", 2021-02-10T11:43:56.266 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:43:56.266 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.266 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.266 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.266 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:43:56.266 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.266 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.266 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.266 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6825", 2021-02-10T11:43:56.266 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:43:56.266 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.266 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.267 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.267 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6827", 2021-02-10T11:43:56.267 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:43:56.267 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.267 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.267 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.267 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:43:56.267 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.267 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.267 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.267 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6822", 2021-02-10T11:43:56.267 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:43:56.267 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.268 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.268 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.268 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6823", 2021-02-10T11:43:56.268 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:43:56.268 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.268 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.268 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.268 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6813/25094", 2021-02-10T11:43:56.268 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6819/25094", 2021-02-10T11:43:56.268 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6827/25094", 2021-02-10T11:43:56.268 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6823/25094", 2021-02-10T11:43:56.268 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:43:56.268 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:43:56.269 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:43:56.269 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.269 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.269 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.269 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:43:56.269 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "0d9ef9b1-98aa-4d37-aa16-6305c7d46d52", 2021-02-10T11:43:56.269 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:43:56.269 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:43:56.269 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:43:56.269 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:43:56.269 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:43:56.269 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:43:56.269 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:43:56.269 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 118, 2021-02-10T11:43:56.270 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:43:56.270 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:43:56.270 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:43:56.270 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.270 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.270 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.270 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6824", 2021-02-10T11:43:56.270 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:43:56.270 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.270 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.270 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.270 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6826", 2021-02-10T11:43:56.270 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:43:56.271 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.271 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.271 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.271 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:43:56.271 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.271 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.271 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.271 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6828", 2021-02-10T11:43:56.271 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:43:56.271 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.271 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.271 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.271 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6829", 2021-02-10T11:43:56.272 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:43:56.272 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.272 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.272 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.272 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:43:56.272 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.272 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.272 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.272 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6832", 2021-02-10T11:43:56.272 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:43:56.272 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.272 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.272 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.272 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6833", 2021-02-10T11:43:56.273 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:43:56.273 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.273 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.273 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.273 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:43:56.273 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.273 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.273 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.273 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6830", 2021-02-10T11:43:56.273 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:43:56.273 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.273 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.273 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.274 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6831", 2021-02-10T11:43:56.274 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:43:56.274 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.274 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.274 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.274 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6826/25095", 2021-02-10T11:43:56.274 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6829/25095", 2021-02-10T11:43:56.274 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6833/25095", 2021-02-10T11:43:56.274 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6831/25095", 2021-02-10T11:43:56.274 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:43:56.274 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:43:56.274 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:43:56.274 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.275 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.275 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.275 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:43:56.275 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "36742c21-bfeb-4e11-bce9-8e745378b605", 2021-02-10T11:43:56.275 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:43:56.275 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:43:56.275 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:43:56.275 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:43:56.275 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:43:56.275 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:43:56.275 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:43:56.275 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 136, 2021-02-10T11:43:56.275 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:43:56.275 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:43:56.276 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:43:56.276 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.276 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.276 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.276 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6816", 2021-02-10T11:43:56.276 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:43:56.276 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.276 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.276 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.276 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6818", 2021-02-10T11:43:56.276 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:43:56.276 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.276 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.277 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.277 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:43:56.277 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.277 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.277 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.277 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6820", 2021-02-10T11:43:56.277 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:43:56.277 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.277 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.277 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.277 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6822", 2021-02-10T11:43:56.277 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:43:56.277 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.278 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.278 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.278 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:43:56.278 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.278 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.278 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.278 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6828", 2021-02-10T11:43:56.278 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:43:56.278 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.278 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.278 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.278 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6830", 2021-02-10T11:43:56.278 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:43:56.279 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.279 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.279 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.279 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:43:56.279 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.279 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.279 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.279 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6824", 2021-02-10T11:43:56.279 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:43:56.279 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.279 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.279 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.279 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6826", 2021-02-10T11:43:56.280 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:43:56.280 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.280 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.280 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.280 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6818/36380", 2021-02-10T11:43:56.280 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6822/36380", 2021-02-10T11:43:56.280 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6830/36380", 2021-02-10T11:43:56.280 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6826/36380", 2021-02-10T11:43:56.280 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:43:56.280 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:43:56.280 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:43:56.280 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.280 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.281 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.281 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:43:56.281 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "88e694e8-fe98-4d1b-88c8-8c9907ece60b", 2021-02-10T11:43:56.281 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:43:56.281 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:43:56.281 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:43:56.281 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:43:56.281 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:43:56.281 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:43:56.281 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:43:56.281 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 134, 2021-02-10T11:43:56.281 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:43:56.282 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:43:56.282 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:43:56.282 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.282 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.282 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.282 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6808", 2021-02-10T11:43:56.282 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:43:56.282 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.282 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.282 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.282 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6809", 2021-02-10T11:43:56.282 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:43:56.282 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.283 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.283 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.283 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:43:56.283 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.283 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.283 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.283 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6810", 2021-02-10T11:43:56.283 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:43:56.283 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.283 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.283 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.283 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6811", 2021-02-10T11:43:56.283 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:43:56.283 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.284 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.284 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.284 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:43:56.284 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.284 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.284 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.284 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6814", 2021-02-10T11:43:56.284 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:43:56.284 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.284 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.284 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.284 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6815", 2021-02-10T11:43:56.284 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:43:56.285 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.285 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.285 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.285 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:43:56.285 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.285 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.285 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.285 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6812", 2021-02-10T11:43:56.285 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:43:56.285 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.285 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.285 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.285 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6813", 2021-02-10T11:43:56.285 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:43:56.286 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.286 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.286 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.286 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6809/36383", 2021-02-10T11:43:56.286 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6811/36383", 2021-02-10T11:43:56.286 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6815/36383", 2021-02-10T11:43:56.286 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6813/36383", 2021-02-10T11:43:56.286 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:43:56.286 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:43:56.286 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:43:56.286 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.286 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.286 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.287 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:43:56.287 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "87a13c6c-cf85-4aaf-916b-e73913484a49", 2021-02-10T11:43:56.287 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:43:56.287 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:43:56.287 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:43:56.287 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:43:56.287 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:43:56.287 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:43:56.287 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:43:56.287 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 136, 2021-02-10T11:43:56.287 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:43:56.287 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:43:56.287 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:43:56.288 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.288 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.288 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.288 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6800", 2021-02-10T11:43:56.288 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:43:56.288 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.288 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.288 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.288 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6801", 2021-02-10T11:43:56.288 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:43:56.288 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.288 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.288 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.288 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:43:56.289 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.289 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.289 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.289 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6802", 2021-02-10T11:43:56.289 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:43:56.289 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.289 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.289 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.289 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6803", 2021-02-10T11:43:56.289 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:43:56.289 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.289 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.290 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.290 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:43:56.290 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.290 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.290 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.290 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6806", 2021-02-10T11:43:56.290 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:43:56.290 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.290 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.290 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.290 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6807", 2021-02-10T11:43:56.291 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:43:56.291 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.291 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.291 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.291 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:43:56.291 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.291 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.291 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.291 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6804", 2021-02-10T11:43:56.291 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:43:56.291 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.291 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.291 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.292 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6805", 2021-02-10T11:43:56.292 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:43:56.292 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.292 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.292 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.292 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6801/36381", 2021-02-10T11:43:56.292 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6803/36381", 2021-02-10T11:43:56.292 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6807/36381", 2021-02-10T11:43:56.292 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6805/36381", 2021-02-10T11:43:56.292 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:43:56.292 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:43:56.292 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:43:56.292 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.293 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.293 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.293 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:43:56.293 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "7b90823f-854c-4592-a95b-a7fa98f93254", 2021-02-10T11:43:56.293 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:43:56.293 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:43:56.293 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:43:56.293 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:43:56.293 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:43:56.293 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:43:56.293 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:43:56.293 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 136, 2021-02-10T11:43:56.294 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:43:56.294 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:43:56.294 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:43:56.294 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.294 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.294 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.294 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6817", 2021-02-10T11:43:56.294 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:43:56.294 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.294 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.294 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.294 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6819", 2021-02-10T11:43:56.294 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:43:56.295 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.295 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.295 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.295 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:43:56.295 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.295 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.295 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.295 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6821", 2021-02-10T11:43:56.295 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:43:56.295 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.295 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.295 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.296 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6823", 2021-02-10T11:43:56.296 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:43:56.296 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.296 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.296 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.296 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:43:56.296 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.296 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.296 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.296 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6829", 2021-02-10T11:43:56.296 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:43:56.296 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.296 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.297 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.297 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6831", 2021-02-10T11:43:56.297 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:43:56.297 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.297 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.298 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.298 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:43:56.298 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:43:56.298 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.298 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:43:56.298 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6825", 2021-02-10T11:43:56.298 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:43:56.298 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.298 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.298 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:43:56.298 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6827", 2021-02-10T11:43:56.298 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:43:56.298 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.299 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.299 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.299 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6819/36382", 2021-02-10T11:43:56.299 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6823/36382", 2021-02-10T11:43:56.299 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6831/36382", 2021-02-10T11:43:56.299 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6827/36382", 2021-02-10T11:43:56.299 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:43:56.299 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:43:56.299 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:43:56.299 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:43:56.300 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.300 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:43:56.300 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_xinfo": [ 2021-02-10T11:43:56.300 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.300 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:43:56.300 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:43:56.300 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:43:56.300 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:43:56.300 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:43:56.300 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:43:56.300 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.300 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.300 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:43:56.301 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:43:56.301 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:43:56.301 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:43:56.301 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:43:56.301 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:43:56.301 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.301 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.301 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:43:56.301 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:43:56.301 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:43:56.301 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:43:56.301 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:43:56.301 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:43:56.302 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.302 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.302 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:43:56.302 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:43:56.302 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:43:56.302 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:43:56.302 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:43:56.302 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:43:56.302 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.302 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.302 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:43:56.302 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:43:56.302 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:43:56.303 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:43:56.303 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:43:56.303 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:43:56.303 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.303 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.303 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:43:56.303 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:43:56.303 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:43:56.303 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:43:56.303 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:43:56.303 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:43:56.303 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.303 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.303 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:43:56.304 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:43:56.304 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:43:56.304 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:43:56.304 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:43:56.304 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:43:56.304 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.304 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:43:56.304 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:43:56.304 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:43:56.304 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:43:56.304 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:43:56.304 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:43:56.305 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:43:56.305 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.305 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:43:56.305 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap": [], 2021-02-10T11:43:56.305 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap_items": [], 2021-02-10T11:43:56.305 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_temp": [], 2021-02-10T11:43:56.305 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_temp": [], 2021-02-10T11:43:56.305 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklist": {}, 2021-02-10T11:43:56.305 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profiles": { 2021-02-10T11:43:56.305 INFO:teuthology.orchestra.run.gibba001.stdout: "default": { 2021-02-10T11:43:56.305 INFO:teuthology.orchestra.run.gibba001.stdout: "crush-failure-domain": "osd", 2021-02-10T11:43:56.305 INFO:teuthology.orchestra.run.gibba001.stdout: "k": "2", 2021-02-10T11:43:56.305 INFO:teuthology.orchestra.run.gibba001.stdout: "m": "1", 2021-02-10T11:43:56.306 INFO:teuthology.orchestra.run.gibba001.stdout: "plugin": "jerasure", 2021-02-10T11:43:56.306 INFO:teuthology.orchestra.run.gibba001.stdout: "ruleset-failure-domain": "osd", 2021-02-10T11:43:56.306 INFO:teuthology.orchestra.run.gibba001.stdout: "technique": "reed_sol_van" 2021-02-10T11:43:56.306 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:43:56.306 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:43:56.306 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps_queue": [], 2021-02-10T11:43:56.306 INFO:teuthology.orchestra.run.gibba001.stdout: "new_removed_snaps": [], 2021-02-10T11:43:56.306 INFO:teuthology.orchestra.run.gibba001.stdout: "new_purged_snaps": [], 2021-02-10T11:43:56.306 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_node_flags": {}, 2021-02-10T11:43:56.306 INFO:teuthology.orchestra.run.gibba001.stdout: "device_class_flags": {} 2021-02-10T11:43:56.306 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:43:56.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.232 7f317a747700 1 -- 172.21.2.101:0/3863323665 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3154018730 msgr2=0x7f315401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:56.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.232 7f317a747700 1 --2- 172.21.2.101:0/3863323665 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3154018730 0x7f315401ac10 crc :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:56.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.232 7f317a747700 1 -- 172.21.2.101:0/3863323665 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f317419af00 msgr2=0x7f317419b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:56.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.232 7f317a747700 1 --2- 172.21.2.101:0/3863323665 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f317419af00 0x7f317419b300 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 rx=0x7f316400b5c0 tx=0x7f316400b960).stop 2021-02-10T11:43:56.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.232 7f317a747700 1 -- 172.21.2.101:0/3863323665 shutdown_connections 2021-02-10T11:43:56.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.232 7f317a747700 1 --2- 172.21.2.101:0/3863323665 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3154018730 0x7f315401ac10 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:56.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.232 7f317a747700 1 --2- 172.21.2.101:0/3863323665 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f317419ba80 0x7f317419be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:56.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.232 7f317a747700 1 --2- 172.21.2.101:0/3863323665 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f317419c4a0 0x7f317408a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:56.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.232 7f317a747700 1 --2- 172.21.2.101:0/3863323665 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f317419af00 0x7f317419b300 unknown :-1 s=CLOSED pgs=2616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:56.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.232 7f317a747700 1 -- 172.21.2.101:0/3863323665 >> 172.21.2.101:0/3863323665 conn(0x7f317419f8c0 msgr2=0x7f31741a5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:56.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.233 7f317a747700 1 -- 172.21.2.101:0/3863323665 shutdown_connections 2021-02-10T11:43:56.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.233 7f317a747700 1 -- 172.21.2.101:0/3863323665 wait complete. 2021-02-10T11:43:56.309 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:43:56.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.722 7f9e6f69d700 1 Processor -- start 2021-02-10T11:43:56.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.722 7f9e6f69d700 1 -- start start 2021-02-10T11:43:56.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.722 7f9e6f69d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9e680d1870 0x7f9e681b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:56.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.722 7f9e6f69d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9e681b0a10 0x7f9e680cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:56.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.722 7f9e6f69d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f9e680d1c70 0x7f9e680cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:56.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.722 7f9e6f69d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e680fe230 con 0x7f9e681b0a10 2021-02-10T11:43:56.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.722 7f9e6f69d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e680e5ed0 con 0x7f9e680d1c70 2021-02-10T11:43:56.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.722 7f9e6f69d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e680f0b10 con 0x7f9e680d1870 2021-02-10T11:43:56.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.723 7f9e6ccb8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9e681b0a10 0x7f9e680cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:56.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.723 7f9e6ccb8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9e681b0a10 0x7f9e680cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59312/0 (socket says 172.21.2.101:59312) 2021-02-10T11:43:56.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.723 7f9e6ccb8700 1 -- 172.21.2.101:0/2155967846 learned_addr learned my addr 172.21.2.101:0/2155967846 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:56.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.723 7f9e6d4b9700 1 --2- 172.21.2.101:0/2155967846 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9e680d1870 0x7f9e681b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:56.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.723 7f9e6ccb8700 1 -- 172.21.2.101:0/2155967846 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9e680d1870 msgr2=0x7f9e681b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:56.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.723 7f9e6ccb8700 1 --2- 172.21.2.101:0/2155967846 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9e680d1870 0x7f9e681b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:56.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.723 7f9e6ccb8700 1 -- 172.21.2.101:0/2155967846 >> v2:172.21.2.108:3300/0 conn(0x7f9e680d1c70 msgr2=0x7f9e680cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:56.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.723 7f9e6ccb8700 1 --2- 172.21.2.101:0/2155967846 >> v2:172.21.2.108:3300/0 conn(0x7f9e680d1c70 0x7f9e680cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:56.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.723 7f9e6ccb8700 1 -- 172.21.2.101:0/2155967846 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e680cff70 con 0x7f9e681b0a10 2021-02-10T11:43:56.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.723 7f9e6d4b9700 1 --2- 172.21.2.101:0/2155967846 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9e680d1870 0x7f9e681b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:43:56.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.723 7f9e6ccb8700 1 --2- 172.21.2.101:0/2155967846 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9e681b0a10 0x7f9e680cf330 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 rx=0x7f9e64006700 tx=0x7f9e64009b80).ready entity=mon.0 client_cookie=4f272a8d6c1a0d31 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:56.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.723 7f9e5bfff700 1 -- 172.21.2.101:0/2155967846 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9e6400c0c0 con 0x7f9e681b0a10 2021-02-10T11:43:56.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.723 7f9e5bfff700 1 -- 172.21.2.101:0/2155967846 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6400c340 con 0x7f9e681b0a10 2021-02-10T11:43:56.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.723 7f9e5bfff700 1 -- 172.21.2.101:0/2155967846 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9e6400c880 con 0x7f9e681b0a10 2021-02-10T11:43:56.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.723 7f9e6f69d700 1 -- 172.21.2.101:0/2155967846 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9e681b0a10 msgr2=0x7f9e680cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:56.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.724 7f9e6f69d700 1 --2- 172.21.2.101:0/2155967846 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9e681b0a10 0x7f9e680cf330 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 rx=0x7f9e64006700 tx=0x7f9e64009b80).stop 2021-02-10T11:43:56.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.724 7f9e6f69d700 1 -- 172.21.2.101:0/2155967846 shutdown_connections 2021-02-10T11:43:56.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.724 7f9e6f69d700 1 --2- 172.21.2.101:0/2155967846 >> v2:172.21.2.108:3300/0 conn(0x7f9e680d1c70 0x7f9e680cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:56.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.724 7f9e6f69d700 1 --2- 172.21.2.101:0/2155967846 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9e681b0a10 0x7f9e680cf330 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:56.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.724 7f9e6f69d700 1 --2- 172.21.2.101:0/2155967846 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9e680d1870 0x7f9e681b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:56.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.724 7f9e6f69d700 1 -- 172.21.2.101:0/2155967846 >> 172.21.2.101:0/2155967846 conn(0x7f9e680ce450 msgr2=0x7f9e680d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:56.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.724 7f9e6f69d700 1 -- 172.21.2.101:0/2155967846 shutdown_connections 2021-02-10T11:43:56.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.724 7f9e6f69d700 1 -- 172.21.2.101:0/2155967846 wait complete. 2021-02-10T11:43:56.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.739 7f9e6f69d700 1 Processor -- start 2021-02-10T11:43:56.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.739 7f9e6f69d700 1 -- start start 2021-02-10T11:43:56.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.739 7f9e6f69d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9e68094040 0x7f9e68093370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:56.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.739 7f9e6f69d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9e68093990 0x7f9e680ad700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:56.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.739 7f9e6f69d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9e680add20 0x7f9e681a2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:56.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.739 7f9e6f69d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e68093d90 con 0x7f9e680add20 2021-02-10T11:43:56.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.739 7f9e6f69d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e681a1e50 con 0x7f9e68094040 2021-02-10T11:43:56.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.739 7f9e6f69d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e68094590 con 0x7f9e68093990 2021-02-10T11:43:56.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.740 7f9e6dcba700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9e680add20 0x7f9e681a2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:56.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.740 7f9e6dcba700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9e680add20 0x7f9e681a2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59320/0 (socket says 172.21.2.101:59320) 2021-02-10T11:43:56.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.740 7f9e6dcba700 1 -- 172.21.2.101:0/1256024425 learned_addr learned my addr 172.21.2.101:0/1256024425 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:56.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.740 7f9e6ccb8700 1 --2- 172.21.2.101:0/1256024425 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9e68093990 0x7f9e680ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:56.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.740 7f9e6d4b9700 1 --2- 172.21.2.101:0/1256024425 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9e68094040 0x7f9e68093370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:56.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.740 7f9e6dcba700 1 -- 172.21.2.101:0/1256024425 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9e68093990 msgr2=0x7f9e680ad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:56.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.740 7f9e6dcba700 1 --2- 172.21.2.101:0/1256024425 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9e68093990 0x7f9e680ad700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:56.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.740 7f9e6dcba700 1 -- 172.21.2.101:0/1256024425 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9e68094040 msgr2=0x7f9e68093370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:56.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.740 7f9e6dcba700 1 --2- 172.21.2.101:0/1256024425 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9e68094040 0x7f9e68093370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:56.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.740 7f9e6dcba700 1 -- 172.21.2.101:0/1256024425 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e680ae120 con 0x7f9e680add20 2021-02-10T11:43:56.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.740 7f9e6dcba700 1 --2- 172.21.2.101:0/1256024425 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9e680add20 0x7f9e681a2770 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 rx=0x7f9e60009980 tx=0x7f9e6000bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:56.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.740 7f9e59ffb700 1 -- 172.21.2.101:0/1256024425 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9e60003020 con 0x7f9e680add20 2021-02-10T11:43:56.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.740 7f9e59ffb700 1 -- 172.21.2.101:0/1256024425 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e60005a80 con 0x7f9e680add20 2021-02-10T11:43:56.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.740 7f9e59ffb700 1 -- 172.21.2.101:0/1256024425 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9e60006030 con 0x7f9e680add20 2021-02-10T11:43:56.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.741 7f9e6f69d700 1 -- 172.21.2.101:0/1256024425 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e680ae120 con 0x7f9e680add20 2021-02-10T11:43:56.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.741 7f9e6f69d700 1 -- 172.21.2.101:0/1256024425 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e680ae120 con 0x7f9e680add20 2021-02-10T11:43:56.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.741 7f9e59ffb700 1 -- 172.21.2.101:0/1256024425 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9e60005c60 con 0x7f9e680add20 2021-02-10T11:43:56.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.741 7f9e59ffb700 1 --2- 172.21.2.101:0/1256024425 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9e480186d0 0x7f9e4801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:56.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.741 7f9e59ffb700 1 -- 172.21.2.101:0/1256024425 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9e60032b10 con 0x7f9e680add20 2021-02-10T11:43:56.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.741 7f9e6d4b9700 1 --2- 172.21.2.101:0/1256024425 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9e480186d0 0x7f9e4801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:56.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.742 7f9e6d4b9700 1 --2- 172.21.2.101:0/1256024425 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9e480186d0 0x7f9e4801abd0 crc :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:56.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.772 7f9e6f69d700 1 -- 172.21.2.101:0/1256024425 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e68093d90 con 0x7f9e680add20 2021-02-10T11:43:56.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.775 7f9e59ffb700 1 -- 172.21.2.101:0/1256024425 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9e60005870 con 0x7f9e680add20 2021-02-10T11:43:56.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.902 7f9e6f69d700 1 -- 172.21.2.101:0/1256024425 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e68093d90 con 0x7f9e680add20 2021-02-10T11:43:56.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.902 7f9e59ffb700 1 -- 172.21.2.101:0/1256024425 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+2102 (secure 0 0 0) 0x7f9e60005a80 con 0x7f9e680add20 2021-02-10T11:43:56.903 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 115 2021-02-10T11:43:56.903 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:43:56.903 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":115,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/371124351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":371124351},{"type":"v1","addr":"172.21.2.101:6835","nonce":371124351}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:43:20.159141","modified":"2021-02-10 11:43:22.169386","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7990":{"gid":7990,"name":"b","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/797061093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":797061093},{"type":"v1","addr":"172.21.2.108:6833","nonce":797061093}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-02-10T11:43:56.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.904 7f9e6f69d700 1 -- 172.21.2.101:0/1256024425 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9e480186d0 msgr2=0x7f9e4801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:56.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.904 7f9e6f69d700 1 --2- 172.21.2.101:0/1256024425 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9e480186d0 0x7f9e4801abd0 crc :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:56.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.904 7f9e6f69d700 1 -- 172.21.2.101:0/1256024425 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9e680add20 msgr2=0x7f9e681a2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:56.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.904 7f9e6f69d700 1 --2- 172.21.2.101:0/1256024425 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9e680add20 0x7f9e681a2770 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 rx=0x7f9e60009980 tx=0x7f9e6000bcd0).stop 2021-02-10T11:43:56.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.905 7f9e6f69d700 1 -- 172.21.2.101:0/1256024425 shutdown_connections 2021-02-10T11:43:56.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.905 7f9e6f69d700 1 --2- 172.21.2.101:0/1256024425 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9e480186d0 0x7f9e4801abd0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:56.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.905 7f9e6f69d700 1 --2- 172.21.2.101:0/1256024425 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9e68093990 0x7f9e680ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:56.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.905 7f9e6f69d700 1 --2- 172.21.2.101:0/1256024425 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9e680add20 0x7f9e681a2770 unknown :-1 s=CLOSED pgs=2618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:56.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.905 7f9e6f69d700 1 --2- 172.21.2.101:0/1256024425 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9e68094040 0x7f9e68093370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:56.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.905 7f9e6f69d700 1 -- 172.21.2.101:0/1256024425 >> 172.21.2.101:0/1256024425 conn(0x7f9e6819f8a0 msgr2=0x7f9e68094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:56.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.905 7f9e6f69d700 1 -- 172.21.2.101:0/1256024425 shutdown_connections 2021-02-10T11:43:56.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:56.905 7f9e6f69d700 1 -- 172.21.2.101:0/1256024425 wait complete. 2021-02-10T11:43:56.913 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs fail cephfs 2021-02-10T11:43:57.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.327 7f71024b8700 1 Processor -- start 2021-02-10T11:43:57.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.327 7f71024b8700 1 -- start start 2021-02-10T11:43:57.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.328 7f71024b8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70fc0d1870 0x7f70fc1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:57.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.328 7f71024b8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f70fc0f0650 0x7f70fc0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:57.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.328 7f71024b8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70fc0f0b10 con 0x7f70fc1b0a10 2021-02-10T11:43:57.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.328 7f71024b8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70fc0fe230 con 0x7f70fc0f0650 2021-02-10T11:43:57.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.328 7f71024b8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc0e5ed0 con 0x7f70fc0d1870 2021-02-10T11:43:57.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.328 7f70fb7fe700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f70fc1b0a10 0x7f70fc0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:53228/0 (socket says 172.21.2.101:53228) 2021-02-10T11:43:57.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.328 7f70fb7fe700 1 -- 172.21.2.101:0/2420762787 learned_addr learned my addr 172.21.2.101:0/2420762787 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:43:57.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.328 7f70fbfff700 1 --2- 172.21.2.101:0/2420762787 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70fc0d1870 0x7f70fc1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:57.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.328 7f7100ad5700 1 --2- 172.21.2.101:0/2420762787 >> v2:172.21.2.108:3300/0 conn(0x7f70fc0f0650 0x7f70fc0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:57.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.329 7f70fbfff700 1 -- 172.21.2.101:0/2420762787 >> v2:172.21.2.108:3300/0 conn(0x7f70fc0f0650 msgr2=0x7f70fc0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:57.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.329 7f70fbfff700 1 --2- 172.21.2.101:0/2420762787 >> v2:172.21.2.108:3300/0 conn(0x7f70fc0f0650 0x7f70fc0cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:57.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.329 7f70fbfff700 1 -- 172.21.2.101:0/2420762787 >> v1:172.21.2.108:6789/0 conn(0x7f70fc1b0a10 legacy=0x7f70fc0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:57.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.329 7f70fbfff700 1 -- 172.21.2.101:0/2420762787 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc0f20a0 con 0x7f70fc0d1870 2021-02-10T11:43:57.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.329 7f70fbfff700 1 --2- 172.21.2.101:0/2420762787 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70fc0d1870 0x7f70fc1b03f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f70ec006650 tx=0x7f70ec009a80).ready entity=mon.2 client_cookie=49eceae39aafa2b0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:57.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.329 7f70faffd700 1 -- 172.21.2.101:0/2420762787 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f70ec0009b0 con 0x7f70fc0d1870 2021-02-10T11:43:57.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.329 7f70faffd700 1 -- 172.21.2.101:0/2420762787 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec00c0c0 con 0x7f70fc0d1870 2021-02-10T11:43:57.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.329 7f70faffd700 1 -- 172.21.2.101:0/2420762787 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f70ec00c5e0 con 0x7f70fc0d1870 2021-02-10T11:43:57.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.329 7f71024b8700 1 -- 172.21.2.101:0/2420762787 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70fc0d1870 msgr2=0x7f70fc1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:57.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.329 7f71024b8700 1 --2- 172.21.2.101:0/2420762787 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70fc0d1870 0x7f70fc1b03f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f70ec006650 tx=0x7f70ec009a80).stop 2021-02-10T11:43:57.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.330 7f71024b8700 1 -- 172.21.2.101:0/2420762787 shutdown_connections 2021-02-10T11:43:57.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.330 7f71024b8700 1 --2- 172.21.2.101:0/2420762787 >> v2:172.21.2.108:3300/0 conn(0x7f70fc0f0650 0x7f70fc0cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:57.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.330 7f71024b8700 1 --2- 172.21.2.101:0/2420762787 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70fc0d1870 0x7f70fc1b03f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:57.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.330 7f71024b8700 1 -- 172.21.2.101:0/2420762787 >> 172.21.2.101:0/2420762787 conn(0x7f70fc0ce450 msgr2=0x7f70fc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:57.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.330 7f71024b8700 1 -- 172.21.2.101:0/2420762787 shutdown_connections 2021-02-10T11:43:57.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.330 7f71024b8700 1 -- 172.21.2.101:0/2420762787 wait complete. 2021-02-10T11:43:57.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.345 7f71024b8700 1 Processor -- start 2021-02-10T11:43:57.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.345 7f71024b8700 1 -- start start 2021-02-10T11:43:57.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.345 7f71024b8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f70fc0b7510 0x7f70fc0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:57.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.345 7f71024b8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70fc09d530 0x7f70fc09d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:57.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.345 7f71024b8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70fc09df50 0x7f70fc09eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:57.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.345 7f71024b8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc0b8140 con 0x7f70fc09d530 2021-02-10T11:43:57.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.345 7f71024b8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc09e750 con 0x7f70fc0b7510 2021-02-10T11:43:57.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.345 7f71024b8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc09e350 con 0x7f70fc09df50 2021-02-10T11:43:57.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.345 7f70fb7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70fc09d530 0x7f70fc09d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:57.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.345 7f70fb7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70fc09d530 0x7f70fc09d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59330/0 (socket says 172.21.2.101:59330) 2021-02-10T11:43:57.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.345 7f70fb7fe700 1 -- 172.21.2.101:0/2400180091 learned_addr learned my addr 172.21.2.101:0/2400180091 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:57.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.345 7f7100ad5700 1 --2- 172.21.2.101:0/2400180091 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70fc09df50 0x7f70fc09eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:57.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.345 7f70fbfff700 1 --2- 172.21.2.101:0/2400180091 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f70fc0b7510 0x7f70fc0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:57.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.346 7f70fb7fe700 1 -- 172.21.2.101:0/2400180091 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70fc09df50 msgr2=0x7f70fc09eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:57.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.346 7f70fb7fe700 1 --2- 172.21.2.101:0/2400180091 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70fc09df50 0x7f70fc09eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:57.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.346 7f70fb7fe700 1 -- 172.21.2.101:0/2400180091 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f70fc0b7510 msgr2=0x7f70fc0b7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:57.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.346 7f70fb7fe700 1 --2- 172.21.2.101:0/2400180091 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f70fc0b7510 0x7f70fc0b7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:57.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.346 7f70fb7fe700 1 -- 172.21.2.101:0/2400180091 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc09f2c0 con 0x7f70fc09d530 2021-02-10T11:43:57.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.346 7f70fb7fe700 1 --2- 172.21.2.101:0/2400180091 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70fc09d530 0x7f70fc09d930 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 rx=0x7f70f0001c60 tx=0x7f70f000a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:57.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.346 7f70f8ff9700 1 -- 172.21.2.101:0/2400180091 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f70f00010b0 con 0x7f70fc09d530 2021-02-10T11:43:57.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.346 7f70f8ff9700 1 -- 172.21.2.101:0/2400180091 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f0004140 con 0x7f70fc09d530 2021-02-10T11:43:57.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.346 7f71024b8700 1 -- 172.21.2.101:0/2400180091 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70fc09f2c0 con 0x7f70fc09d530 2021-02-10T11:43:57.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.346 7f70f8ff9700 1 -- 172.21.2.101:0/2400180091 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f70f0004660 con 0x7f70fc09d530 2021-02-10T11:43:57.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.346 7f71024b8700 1 -- 172.21.2.101:0/2400180091 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70fc09f2c0 con 0x7f70fc09d530 2021-02-10T11:43:57.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.347 7f70f8ff9700 1 -- 172.21.2.101:0/2400180091 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f70f0004320 con 0x7f70fc09d530 2021-02-10T11:43:57.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.347 7f70f8ff9700 1 --2- 172.21.2.101:0/2400180091 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70dc0186c0 0x7f70dc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:57.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.347 7f70f8ff9700 1 -- 172.21.2.101:0/2400180091 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f70f00325e0 con 0x7f70fc09d530 2021-02-10T11:43:57.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.347 7f70fbfff700 1 --2- 172.21.2.101:0/2400180091 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70dc0186c0 0x7f70dc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:57.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.347 7f70fbfff700 1 --2- 172.21.2.101:0/2400180091 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70dc0186c0 0x7f70dc01abc0 crc :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:57.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.377 7f71024b8700 1 -- 172.21.2.101:0/2400180091 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70fc0ebc50 con 0x7f70fc09d530 2021-02-10T11:43:57.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.380 7f70f8ff9700 1 -- 172.21.2.101:0/2400180091 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f70f00009b0 con 0x7f70fc09d530 2021-02-10T11:43:57.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:57.507 7f71024b8700 1 -- 172.21.2.101:0/2400180091 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f70fc0f0b10 con 0x7f70fc09d530 2021-02-10T11:43:58.288 INFO:tasks.ceph.mon.a.gibba001.stderr:2021-02-10 11:43:58.288 7f974d04d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-02-10T11:43:58.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.294 7f70f8ff9700 1 -- 172.21.2.101:0/2400180091 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v116) v1 ==== 161+0+0 (secure 0 0 0) 0x7f70f0004140 con 0x7f70fc09d530 2021-02-10T11:43:58.326 INFO:teuthology.orchestra.run.gibba001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-02-10T11:43:58.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.328 7f71024b8700 1 -- 172.21.2.101:0/2400180091 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70dc0186c0 msgr2=0x7f70dc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:58.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.328 7f71024b8700 1 --2- 172.21.2.101:0/2400180091 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70dc0186c0 0x7f70dc01abc0 crc :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:58.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.328 7f71024b8700 1 -- 172.21.2.101:0/2400180091 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70fc09d530 msgr2=0x7f70fc09d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:58.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.328 7f71024b8700 1 --2- 172.21.2.101:0/2400180091 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70fc09d530 0x7f70fc09d930 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 rx=0x7f70f0001c60 tx=0x7f70f000a8d0).stop 2021-02-10T11:43:58.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.328 7f71024b8700 1 -- 172.21.2.101:0/2400180091 shutdown_connections 2021-02-10T11:43:58.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.328 7f71024b8700 1 --2- 172.21.2.101:0/2400180091 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70dc0186c0 0x7f70dc01abc0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:58.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.328 7f71024b8700 1 --2- 172.21.2.101:0/2400180091 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70fc09df50 0x7f70fc09eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:58.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.328 7f71024b8700 1 --2- 172.21.2.101:0/2400180091 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70fc09d530 0x7f70fc09d930 unknown :-1 s=CLOSED pgs=2619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:58.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.328 7f71024b8700 1 --2- 172.21.2.101:0/2400180091 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f70fc0b7510 0x7f70fc0b7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:58.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.328 7f71024b8700 1 -- 172.21.2.101:0/2400180091 >> 172.21.2.101:0/2400180091 conn(0x7f70fc0b83b0 msgr2=0x7f70fc0a6850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:58.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.328 7f71024b8700 1 -- 172.21.2.101:0/2400180091 shutdown_connections 2021-02-10T11:43:58.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.328 7f71024b8700 1 -- 172.21.2.101:0/2400180091 wait complete. 2021-02-10T11:43:58.336 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:43:58.337 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:43:58.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.748 7f4cded46700 1 Processor -- start 2021-02-10T11:43:58.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.749 7f4cded46700 1 -- start start 2021-02-10T11:43:58.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.749 7f4cded46700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f4cd81b0470 0x7f4cd81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:58.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.749 7f4cded46700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4cd80f20e0 0x7f4cd80f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:58.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.749 7f4cded46700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cd80f0b50 con 0x7f4cd80ef610 2021-02-10T11:43:58.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.749 7f4cded46700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd80f0820 con 0x7f4cd81b0470 2021-02-10T11:43:58.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.749 7f4cded46700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd80f24e0 con 0x7f4cd80f20e0 2021-02-10T11:43:58.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.749 7f4cdcb62700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f4cd81b0470 0x7f4cd81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:58.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.749 7f4cdcb62700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f4cd81b0470 0x7f4cd81b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59340/0 (socket says 172.21.2.101:59340) 2021-02-10T11:43:58.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.749 7f4cdcb62700 1 -- 172.21.2.101:0/1897886976 learned_addr learned my addr 172.21.2.101:0/1897886976 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:58.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.749 7f4cdd363700 1 --2- 172.21.2.101:0/1897886976 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4cd80f20e0 0x7f4cd80f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:58.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.749 7f4cdd363700 1 -- 172.21.2.101:0/1897886976 >> v2:172.21.2.101:3300/0 conn(0x7f4cd81b0470 msgr2=0x7f4cd81b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:58.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.749 7f4cdd363700 1 --2- 172.21.2.101:0/1897886976 >> v2:172.21.2.101:3300/0 conn(0x7f4cd81b0470 0x7f4cd81b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:58.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.749 7f4cdd363700 1 -- 172.21.2.101:0/1897886976 >> v1:172.21.2.108:6789/0 conn(0x7f4cd80ef610 legacy=0x7f4cd80efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:43:58.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.750 7f4cdd363700 1 -- 172.21.2.101:0/1897886976 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd80e52e0 con 0x7f4cd80f20e0 2021-02-10T11:43:58.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.750 7f4cdcb62700 1 --2- 172.21.2.101:0/1897886976 >> v2:172.21.2.101:3300/0 conn(0x7f4cd81b0470 0x7f4cd81b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:43:58.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.750 7f4cdd363700 1 --2- 172.21.2.101:0/1897886976 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4cd80f20e0 0x7f4cd80f0200 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f4cd0006ad0 tx=0x7f4cd0009ad0).ready entity=mon.2 client_cookie=94b7a0aef45063ad server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:58.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.750 7f4cd77fe700 1 -- 172.21.2.101:0/1897886976 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4cd0000c10 con 0x7f4cd80f20e0 2021-02-10T11:43:58.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.750 7f4cd77fe700 1 -- 172.21.2.101:0/1897886976 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd0006370 con 0x7f4cd80f20e0 2021-02-10T11:43:58.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.750 7f4cd77fe700 1 -- 172.21.2.101:0/1897886976 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4cd0002560 con 0x7f4cd80f20e0 2021-02-10T11:43:58.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.750 7f4cded46700 1 -- 172.21.2.101:0/1897886976 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4cd80f20e0 msgr2=0x7f4cd80f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:58.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.750 7f4cded46700 1 --2- 172.21.2.101:0/1897886976 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4cd80f20e0 0x7f4cd80f0200 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f4cd0006ad0 tx=0x7f4cd0009ad0).stop 2021-02-10T11:43:58.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.751 7f4cded46700 1 -- 172.21.2.101:0/1897886976 shutdown_connections 2021-02-10T11:43:58.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.751 7f4cded46700 1 --2- 172.21.2.101:0/1897886976 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4cd80f20e0 0x7f4cd80f0200 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:58.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.751 7f4cded46700 1 --2- 172.21.2.101:0/1897886976 >> v2:172.21.2.101:3300/0 conn(0x7f4cd81b0470 0x7f4cd81b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:58.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.751 7f4cded46700 1 -- 172.21.2.101:0/1897886976 >> 172.21.2.101:0/1897886976 conn(0x7f4cd80ce3e0 msgr2=0x7f4cd80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:58.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.751 7f4cded46700 1 -- 172.21.2.101:0/1897886976 shutdown_connections 2021-02-10T11:43:58.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.751 7f4cded46700 1 -- 172.21.2.101:0/1897886976 wait complete. 2021-02-10T11:43:58.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.765 7f4cded46700 1 Processor -- start 2021-02-10T11:43:58.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.766 7f4cded46700 1 -- start start 2021-02-10T11:43:58.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.766 7f4cded46700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4cd81af8a0 0x7f4cd8198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:58.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.766 7f4cded46700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4cd81af050 0x7f4cd80aa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:58.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.766 7f4cded46700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4cd8197d00 0x7f4cd8198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:58.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.766 7f4cded46700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd81988e0 con 0x7f4cd8197d00 2021-02-10T11:43:58.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.766 7f4cded46700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd81af450 con 0x7f4cd81af8a0 2021-02-10T11:43:58.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.766 7f4cded46700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd80aaa10 con 0x7f4cd81af050 2021-02-10T11:43:58.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.766 7f4cdcb62700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4cd81af8a0 0x7f4cd8198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:58.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.766 7f4cd7fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4cd81af050 0x7f4cd80aa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:58.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.766 7f4cdd363700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4cd8197d00 0x7f4cd8198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:58.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.766 7f4cdcb62700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4cd81af8a0 0x7f4cd8198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:41596/0 (socket says 172.21.2.101:41596) 2021-02-10T11:43:58.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.766 7f4cdcb62700 1 -- 172.21.2.101:0/1028701901 learned_addr learned my addr 172.21.2.101:0/1028701901 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:58.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.767 7f4cd7fff700 1 -- 172.21.2.101:0/1028701901 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4cd81af8a0 msgr2=0x7f4cd8198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:58.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.767 7f4cd7fff700 1 --2- 172.21.2.101:0/1028701901 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4cd81af8a0 0x7f4cd8198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:58.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.767 7f4cd7fff700 1 -- 172.21.2.101:0/1028701901 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4cd8197d00 msgr2=0x7f4cd8198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:58.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.767 7f4cd7fff700 1 --2- 172.21.2.101:0/1028701901 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4cd8197d00 0x7f4cd8198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:58.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.767 7f4cd7fff700 1 -- 172.21.2.101:0/1028701901 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd819fec0 con 0x7f4cd81af050 2021-02-10T11:43:58.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.767 7f4cd7fff700 1 --2- 172.21.2.101:0/1028701901 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4cd81af050 0x7f4cd80aa3f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f4cc800b890 tx=0x7f4cc800bc30).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:58.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.767 7f4cd57fa700 1 -- 172.21.2.101:0/1028701901 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4cc80072f0 con 0x7f4cd81af050 2021-02-10T11:43:58.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.767 7f4cd57fa700 1 -- 172.21.2.101:0/1028701901 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc8003030 con 0x7f4cd81af050 2021-02-10T11:43:58.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.767 7f4cded46700 1 -- 172.21.2.101:0/1028701901 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd819fec0 con 0x7f4cd81af050 2021-02-10T11:43:58.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.767 7f4cded46700 1 -- 172.21.2.101:0/1028701901 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd81a0450 con 0x7f4cd81af050 2021-02-10T11:43:58.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.767 7f4cd57fa700 1 -- 172.21.2.101:0/1028701901 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4cc80077e0 con 0x7f4cd81af050 2021-02-10T11:43:58.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.768 7f4cd57fa700 1 -- 172.21.2.101:0/1028701901 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4cc800bd70 con 0x7f4cd81af050 2021-02-10T11:43:58.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.768 7f4cd57fa700 1 --2- 172.21.2.101:0/1028701901 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cbc0186c0 0x7f4cbc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:58.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.768 7f4cd57fa700 1 -- 172.21.2.101:0/1028701901 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f4cc8032b30 con 0x7f4cd81af050 2021-02-10T11:43:58.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.768 7f4cdcb62700 1 --2- 172.21.2.101:0/1028701901 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cbc0186c0 0x7f4cbc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:58.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.768 7f4cdcb62700 1 --2- 172.21.2.101:0/1028701901 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cbc0186c0 0x7f4cbc01abc0 crc :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:58.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.799 7f4cded46700 1 -- 172.21.2.101:0/1028701901 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd8105a20 con 0x7f4cd81af050 2021-02-10T11:43:58.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.802 7f4cd57fa700 1 -- 172.21.2.101:0/1028701901 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4cc8007aa0 con 0x7f4cd81af050 2021-02-10T11:43:58.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.929 7f4cded46700 1 -- 172.21.2.101:0/1028701901 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4cd80f0b50 con 0x7f4cd81af050 2021-02-10T11:43:58.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.930 7f4cd57fa700 1 -- 172.21.2.101:0/1028701901 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+1760 (secure 0 0 0) 0x7f4cc800bd70 con 0x7f4cd81af050 2021-02-10T11:43:58.930 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 116 2021-02-10T11:43:58.930 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:43:58.930 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":116,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/371124351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":371124351},{"type":"v1","addr":"172.21.2.101:6835","nonce":371124351}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":116,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:43:20.159141","modified":"2021-02-10 11:43:58.289801","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":139,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-02-10T11:43:58.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.932 7f4cded46700 1 -- 172.21.2.101:0/1028701901 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cbc0186c0 msgr2=0x7f4cbc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:58.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.932 7f4cded46700 1 --2- 172.21.2.101:0/1028701901 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cbc0186c0 0x7f4cbc01abc0 crc :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:58.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.932 7f4cded46700 1 -- 172.21.2.101:0/1028701901 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4cd81af050 msgr2=0x7f4cd80aa3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:58.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.932 7f4cded46700 1 --2- 172.21.2.101:0/1028701901 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4cd81af050 0x7f4cd80aa3f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f4cc800b890 tx=0x7f4cc800bc30).stop 2021-02-10T11:43:58.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.932 7f4cded46700 1 -- 172.21.2.101:0/1028701901 shutdown_connections 2021-02-10T11:43:58.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.932 7f4cded46700 1 --2- 172.21.2.101:0/1028701901 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cbc0186c0 0x7f4cbc01abc0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:58.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.932 7f4cded46700 1 --2- 172.21.2.101:0/1028701901 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4cd81af050 0x7f4cd80aa3f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:58.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.932 7f4cded46700 1 --2- 172.21.2.101:0/1028701901 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4cd8197d00 0x7f4cd8198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:58.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.932 7f4cded46700 1 --2- 172.21.2.101:0/1028701901 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4cd81af8a0 0x7f4cd8198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:58.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.932 7f4cded46700 1 -- 172.21.2.101:0/1028701901 >> 172.21.2.101:0/1028701901 conn(0x7f4cd81a37a0 msgr2=0x7f4cd818c230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:58.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.932 7f4cded46700 1 -- 172.21.2.101:0/1028701901 shutdown_connections 2021-02-10T11:43:58.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:58.932 7f4cded46700 1 -- 172.21.2.101:0/1028701901 wait complete. 2021-02-10T11:43:58.940 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-02-10T11:43:59.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.358 7ff8479bb700 1 Processor -- start 2021-02-10T11:43:59.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.358 7ff8479bb700 1 -- start start 2021-02-10T11:43:59.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.358 7ff8479bb700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff8400d1870 0x7ff8401b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:59.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.358 7ff8479bb700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff8401b0a10 0x7ff8400cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:59.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.358 7ff8479bb700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8400f0b10 con 0x7ff8400d1c70 2021-02-10T11:43:59.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.358 7ff8479bb700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8400e5ed0 con 0x7ff8401b0a10 2021-02-10T11:43:59.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.358 7ff8479bb700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8400f0650 con 0x7ff8400d1870 2021-02-10T11:43:59.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.358 7ff844fd6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff8401b0a10 0x7ff8400cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:59.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.358 7ff844fd6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff8401b0a10 0x7ff8400cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59350/0 (socket says 172.21.2.101:59350) 2021-02-10T11:43:59.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.358 7ff844fd6700 1 -- 172.21.2.101:0/296198581 learned_addr learned my addr 172.21.2.101:0/296198581 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:59.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.358 7ff845fd8700 1 --1- 172.21.2.101:0/296198581 >> v1:172.21.2.101:6789/0 conn(0x7ff8400d1c70 0x7ff8400cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59844/0 (socket says 172.21.2.101:59844) 2021-02-10T11:43:59.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.359 7ff844fd6700 1 -- 172.21.2.101:0/296198581 >> v2:172.21.2.108:3300/0 conn(0x7ff8400d1870 msgr2=0x7ff8401b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:43:59.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.359 7ff844fd6700 1 --2- 172.21.2.101:0/296198581 >> v2:172.21.2.108:3300/0 conn(0x7ff8400d1870 0x7ff8401b03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:59.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.359 7ff844fd6700 1 -- 172.21.2.101:0/296198581 >> v1:172.21.2.101:6789/0 conn(0x7ff8400d1c70 legacy=0x7ff8400cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:59.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.359 7ff844fd6700 1 -- 172.21.2.101:0/296198581 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8400f0840 con 0x7ff8401b0a10 2021-02-10T11:43:59.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.359 7ff844fd6700 1 --2- 172.21.2.101:0/296198581 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8401b0a10 0x7ff8400cf330 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 rx=0x7ff830006700 tx=0x7ff830009b80).ready entity=mon.0 client_cookie=125687e7948f13f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:59.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.359 7ff837fff700 1 -- 172.21.2.101:0/296198581 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff83000c0c0 con 0x7ff8401b0a10 2021-02-10T11:43:59.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.359 7ff837fff700 1 -- 172.21.2.101:0/296198581 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83000c440 con 0x7ff8401b0a10 2021-02-10T11:43:59.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.359 7ff837fff700 1 -- 172.21.2.101:0/296198581 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff83000c8a0 con 0x7ff8401b0a10 2021-02-10T11:43:59.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.359 7ff8479bb700 1 -- 172.21.2.101:0/296198581 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8401b0a10 msgr2=0x7ff8400cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:59.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.359 7ff8479bb700 1 --2- 172.21.2.101:0/296198581 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8401b0a10 0x7ff8400cf330 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 rx=0x7ff830006700 tx=0x7ff830009b80).stop 2021-02-10T11:43:59.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.360 7ff8479bb700 1 -- 172.21.2.101:0/296198581 shutdown_connections 2021-02-10T11:43:59.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.360 7ff8479bb700 1 --2- 172.21.2.101:0/296198581 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8401b0a10 0x7ff8400cf330 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:43:59.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.360 7ff8479bb700 1 --2- 172.21.2.101:0/296198581 >> v2:172.21.2.108:3300/0 conn(0x7ff8400d1870 0x7ff8401b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:59.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.360 7ff8479bb700 1 -- 172.21.2.101:0/296198581 >> 172.21.2.101:0/296198581 conn(0x7ff8400ce450 msgr2=0x7ff8400d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:43:59.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.360 7ff8479bb700 1 -- 172.21.2.101:0/296198581 shutdown_connections 2021-02-10T11:43:59.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.360 7ff8479bb700 1 -- 172.21.2.101:0/296198581 wait complete. 2021-02-10T11:43:59.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.375 7ff8479bb700 1 Processor -- start 2021-02-10T11:43:59.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.375 7ff8479bb700 1 -- start start 2021-02-10T11:43:59.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.375 7ff8479bb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff8400be730 0x7ff8400ede90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:59.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.375 7ff8479bb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8401a74f0 0x7ff8401a78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:59.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.375 7ff8479bb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8401a7f10 0x7ff8400eee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:59.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.375 7ff8479bb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8401a8310 con 0x7ff8401a74f0 2021-02-10T11:43:59.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.375 7ff8479bb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8400ee8f0 con 0x7ff8400be730 2021-02-10T11:43:59.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.375 7ff8479bb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8400be2d0 con 0x7ff8401a7f10 2021-02-10T11:43:59.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.375 7ff844fd6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8401a74f0 0x7ff8401a78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:59.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.375 7ff844fd6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8401a74f0 0x7ff8401a78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59356/0 (socket says 172.21.2.101:59356) 2021-02-10T11:43:59.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.375 7ff844fd6700 1 -- 172.21.2.101:0/3547089243 learned_addr learned my addr 172.21.2.101:0/3547089243 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:43:59.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.376 7ff844fd6700 1 -- 172.21.2.101:0/3547089243 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8401a7f10 msgr2=0x7ff8400eee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:59.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.376 7ff844fd6700 1 --2- 172.21.2.101:0/3547089243 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8401a7f10 0x7ff8400eee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:59.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.376 7ff844fd6700 1 -- 172.21.2.101:0/3547089243 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff8400be730 msgr2=0x7ff8400ede90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:43:59.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.376 7ff844fd6700 1 --2- 172.21.2.101:0/3547089243 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff8400be730 0x7ff8400ede90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:43:59.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.376 7ff844fd6700 1 -- 172.21.2.101:0/3547089243 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff84009fd00 con 0x7ff8401a74f0 2021-02-10T11:43:59.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.376 7ff844fd6700 1 --2- 172.21.2.101:0/3547089243 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8401a74f0 0x7ff8401a78f0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 rx=0x7ff83000b5e0 tx=0x7ff83000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:59.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.376 7ff835ffb700 1 -- 172.21.2.101:0/3547089243 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff83000bd70 con 0x7ff8401a74f0 2021-02-10T11:43:59.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.376 7ff835ffb700 1 -- 172.21.2.101:0/3547089243 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff830002030 con 0x7ff8401a74f0 2021-02-10T11:43:59.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.376 7ff835ffb700 1 -- 172.21.2.101:0/3547089243 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff830002550 con 0x7ff8401a74f0 2021-02-10T11:43:59.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.376 7ff8479bb700 1 -- 172.21.2.101:0/3547089243 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff84009fd00 con 0x7ff8401a74f0 2021-02-10T11:43:59.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.376 7ff8479bb700 1 -- 172.21.2.101:0/3547089243 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff84009fd00 con 0x7ff8401a74f0 2021-02-10T11:43:59.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.376 7ff835ffb700 1 -- 172.21.2.101:0/3547089243 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff830002210 con 0x7ff8401a74f0 2021-02-10T11:43:59.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.376 7ff835ffb700 1 --2- 172.21.2.101:0/3547089243 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff820018730 0x7ff82001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:43:59.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.377 7ff835ffb700 1 -- 172.21.2.101:0/3547089243 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5659+0+0 (secure 0 0 0) 0x7ff830032670 con 0x7ff8401a74f0 2021-02-10T11:43:59.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.377 7ff8457d7700 1 --2- 172.21.2.101:0/3547089243 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff820018730 0x7ff82001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:43:59.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.377 7ff8457d7700 1 --2- 172.21.2.101:0/3547089243 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff820018730 0x7ff82001abc0 crc :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:43:59.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.408 7ff8479bb700 1 -- 172.21.2.101:0/3547089243 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8400ebc50 con 0x7ff8401a74f0 2021-02-10T11:43:59.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.410 7ff835ffb700 1 -- 172.21.2.101:0/3547089243 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff830002240 con 0x7ff8401a74f0 2021-02-10T11:43:59.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:43:59.537 7ff8479bb700 1 -- 172.21.2.101:0/3547089243 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff8400f0b10 con 0x7ff8401a74f0 2021-02-10T11:44:00.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.303 7ff835ffb700 1 -- 172.21.2.101:0/3547089243 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v117) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff830002030 con 0x7ff8401a74f0 2021-02-10T11:44:00.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.308 7ff8479bb700 1 -- 172.21.2.101:0/3547089243 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff820018730 msgr2=0x7ff82001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:00.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.308 7ff8479bb700 1 --2- 172.21.2.101:0/3547089243 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff820018730 0x7ff82001abc0 crc :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:00.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.308 7ff8479bb700 1 -- 172.21.2.101:0/3547089243 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8401a74f0 msgr2=0x7ff8401a78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:00.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.308 7ff8479bb700 1 --2- 172.21.2.101:0/3547089243 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8401a74f0 0x7ff8401a78f0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 rx=0x7ff83000b5e0 tx=0x7ff83000b980).stop 2021-02-10T11:44:00.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.308 7ff8479bb700 1 -- 172.21.2.101:0/3547089243 shutdown_connections 2021-02-10T11:44:00.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.308 7ff8479bb700 1 --2- 172.21.2.101:0/3547089243 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff820018730 0x7ff82001abc0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:00.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.308 7ff8479bb700 1 --2- 172.21.2.101:0/3547089243 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8401a7f10 0x7ff8400eee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:00.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.308 7ff8479bb700 1 --2- 172.21.2.101:0/3547089243 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8401a74f0 0x7ff8401a78f0 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:00.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.308 7ff8479bb700 1 --2- 172.21.2.101:0/3547089243 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff8400be730 0x7ff8400ede90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:00.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.308 7ff8479bb700 1 -- 172.21.2.101:0/3547089243 >> 172.21.2.101:0/3547089243 conn(0x7ff84009e9b0 msgr2=0x7ff840095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:00.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.308 7ff8479bb700 1 -- 172.21.2.101:0/3547089243 shutdown_connections 2021-02-10T11:44:00.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.308 7ff8479bb700 1 -- 172.21.2.101:0/3547089243 wait complete. 2021-02-10T11:44:00.316 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-02-10T11:44:00.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.731 7f99d1337700 1 Processor -- start 2021-02-10T11:44:00.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.731 7f99d1337700 1 -- start start 2021-02-10T11:44:00.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.732 7f99d1337700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99cc0d1880 0x7f99cc1b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:00.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.732 7f99d1337700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f99cc0cff80 0x7f99cc109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:00.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.732 7f99d1337700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99cc109a70 con 0x7f99cc1b0af0 2021-02-10T11:44:00.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.732 7f99d1337700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99cc109760 con 0x7f99cc0cff80 2021-02-10T11:44:00.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.732 7f99d1337700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc088070 con 0x7f99cc0d1880 2021-02-10T11:44:00.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.732 7f99ca87c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f99cc1b0af0 0x7f99cc108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59856/0 (socket says 172.21.2.101:59856) 2021-02-10T11:44:00.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.732 7f99ca87c700 1 -- 172.21.2.101:0/4246522716 learned_addr learned my addr 172.21.2.101:0/4246522716 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:00.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.732 7f99cb07d700 1 --2- 172.21.2.101:0/4246522716 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99cc0d1880 0x7f99cc1b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:00.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.732 7f99cb87e700 1 --2- 172.21.2.101:0/4246522716 >> v2:172.21.2.108:3300/0 conn(0x7f99cc0cff80 0x7f99cc109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:00.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.732 7f99cb07d700 1 -- 172.21.2.101:0/4246522716 >> v2:172.21.2.108:3300/0 conn(0x7f99cc0cff80 msgr2=0x7f99cc109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:00.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.732 7f99cb07d700 1 --2- 172.21.2.101:0/4246522716 >> v2:172.21.2.108:3300/0 conn(0x7f99cc0cff80 0x7f99cc109140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:00.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.732 7f99cb07d700 1 -- 172.21.2.101:0/4246522716 >> v1:172.21.2.101:6789/0 conn(0x7f99cc1b0af0 legacy=0x7f99cc108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:00.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.732 7f99cb07d700 1 -- 172.21.2.101:0/4246522716 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99cc088260 con 0x7f99cc0d1880 2021-02-10T11:44:00.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.733 7f99cb07d700 1 --2- 172.21.2.101:0/4246522716 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99cc0d1880 0x7f99cc1b04d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f99b4006650 tx=0x7f99b4009a80).ready entity=mon.2 client_cookie=9aa2a6236e0a9aec server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:00.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.733 7f99ca07b700 1 -- 172.21.2.101:0/4246522716 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f99b400c0c0 con 0x7f99cc0d1880 2021-02-10T11:44:00.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.733 7f99ca07b700 1 -- 172.21.2.101:0/4246522716 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b400c340 con 0x7f99cc0d1880 2021-02-10T11:44:00.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.733 7f99ca07b700 1 -- 172.21.2.101:0/4246522716 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f99b400c860 con 0x7f99cc0d1880 2021-02-10T11:44:00.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.733 7f99d1337700 1 -- 172.21.2.101:0/4246522716 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99cc0d1880 msgr2=0x7f99cc1b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:00.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.733 7f99d1337700 1 --2- 172.21.2.101:0/4246522716 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99cc0d1880 0x7f99cc1b04d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f99b4006650 tx=0x7f99b4009a80).stop 2021-02-10T11:44:00.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.734 7f99d1337700 1 -- 172.21.2.101:0/4246522716 shutdown_connections 2021-02-10T11:44:00.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.734 7f99d1337700 1 --2- 172.21.2.101:0/4246522716 >> v2:172.21.2.108:3300/0 conn(0x7f99cc0cff80 0x7f99cc109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:00.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.734 7f99d1337700 1 --2- 172.21.2.101:0/4246522716 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99cc0d1880 0x7f99cc1b04d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:00.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.734 7f99d1337700 1 -- 172.21.2.101:0/4246522716 >> 172.21.2.101:0/4246522716 conn(0x7f99cc0ce430 msgr2=0x7f99cc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:00.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.734 7f99d1337700 1 -- 172.21.2.101:0/4246522716 shutdown_connections 2021-02-10T11:44:00.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.734 7f99d1337700 1 -- 172.21.2.101:0/4246522716 wait complete. 2021-02-10T11:44:00.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.748 7f99d1337700 1 Processor -- start 2021-02-10T11:44:00.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.748 7f99d1337700 1 -- start start 2021-02-10T11:44:00.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.748 7f99d1337700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99cc09a690 0x7f99cc09aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:00.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.749 7f99d1337700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99cc1b4160 0x7f99cc1b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:00.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.749 7f99d1337700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99cc1b7dd0 0x7f99cc1b81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:00.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.749 7f99d1337700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc1b4b80 con 0x7f99cc1b4160 2021-02-10T11:44:00.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.749 7f99d1337700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc1b87f0 con 0x7f99cc1b7dd0 2021-02-10T11:44:00.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.749 7f99d1337700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc1b8a10 con 0x7f99cc09a690 2021-02-10T11:44:00.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.749 7f99ca87c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99cc1b4160 0x7f99cc1b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:00.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.749 7f99ca87c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99cc1b4160 0x7f99cc1b4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59370/0 (socket says 172.21.2.101:59370) 2021-02-10T11:44:00.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.749 7f99ca87c700 1 -- 172.21.2.101:0/1498144187 learned_addr learned my addr 172.21.2.101:0/1498144187 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:00.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.749 7f99cb07d700 1 --2- 172.21.2.101:0/1498144187 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99cc09a690 0x7f99cc09aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:00.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.749 7f99cb87e700 1 --2- 172.21.2.101:0/1498144187 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99cc1b7dd0 0x7f99cc1b81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:00.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.749 7f99cb87e700 1 -- 172.21.2.101:0/1498144187 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99cc09a690 msgr2=0x7f99cc09aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:00.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.749 7f99cb87e700 1 --2- 172.21.2.101:0/1498144187 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99cc09a690 0x7f99cc09aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:00.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.749 7f99cb87e700 1 -- 172.21.2.101:0/1498144187 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99cc1b4160 msgr2=0x7f99cc1b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:00.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.749 7f99cb87e700 1 --2- 172.21.2.101:0/1498144187 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99cc1b4160 0x7f99cc1b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:00.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.749 7f99cb87e700 1 -- 172.21.2.101:0/1498144187 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99cc1b8c30 con 0x7f99cc1b7dd0 2021-02-10T11:44:00.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.749 7f99cb87e700 1 --2- 172.21.2.101:0/1498144187 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99cc1b7dd0 0x7f99cc1b81d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f99c400b8a0 tx=0x7f99c400bc40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:00.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.749 7f99c3fff700 1 -- 172.21.2.101:0/1498144187 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f99c4000f90 con 0x7f99cc1b7dd0 2021-02-10T11:44:00.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.749 7f99c3fff700 1 -- 172.21.2.101:0/1498144187 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c400bd80 con 0x7f99cc1b7dd0 2021-02-10T11:44:00.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.749 7f99d1337700 1 -- 172.21.2.101:0/1498144187 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99cc1b8c30 con 0x7f99cc1b7dd0 2021-02-10T11:44:00.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.749 7f99c3fff700 1 -- 172.21.2.101:0/1498144187 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f99c4000f90 con 0x7f99cc1b7dd0 2021-02-10T11:44:00.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.750 7f99d1337700 1 -- 172.21.2.101:0/1498144187 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99cc1b8c30 con 0x7f99cc1b7dd0 2021-02-10T11:44:00.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.750 7f99c3fff700 1 -- 172.21.2.101:0/1498144187 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f99c40056b0 con 0x7f99cc1b7dd0 2021-02-10T11:44:00.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.750 7f99c3fff700 1 --2- 172.21.2.101:0/1498144187 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f99b0018730 0x7f99b001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:00.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.750 7f99c3fff700 1 -- 172.21.2.101:0/1498144187 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f99c4032aa0 con 0x7f99cc1b7dd0 2021-02-10T11:44:00.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.750 7f99cb07d700 1 --2- 172.21.2.101:0/1498144187 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f99b0018730 0x7f99b001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:00.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.751 7f99cb07d700 1 --2- 172.21.2.101:0/1498144187 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f99b0018730 0x7f99b001abc0 crc :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:00.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.782 7f99d1337700 1 -- 172.21.2.101:0/1498144187 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99cc09a160 con 0x7f99cc1b7dd0 2021-02-10T11:44:00.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.784 7f99c3fff700 1 -- 172.21.2.101:0/1498144187 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f99c4000f70 con 0x7f99cc1b7dd0 2021-02-10T11:44:00.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:00.911 7f99d1337700 1 -- 172.21.2.101:0/1498144187 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f99cc09a160 con 0x7f99cc1b7dd0 2021-02-10T11:44:01.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.307 7f99c3fff700 1 -- 172.21.2.101:0/1498144187 <== mon.1 v2:172.21.2.108:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v140) v1 ==== 185+0+0 (secure 0 0 0) 0x7f99c40056b0 con 0x7f99cc1b7dd0 2021-02-10T11:44:01.327 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' removed 2021-02-10T11:44:01.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.330 7f99d1337700 1 -- 172.21.2.101:0/1498144187 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f99b0018730 msgr2=0x7f99b001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:01.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.330 7f99d1337700 1 --2- 172.21.2.101:0/1498144187 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f99b0018730 0x7f99b001abc0 crc :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:01.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.330 7f99d1337700 1 -- 172.21.2.101:0/1498144187 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99cc1b7dd0 msgr2=0x7f99cc1b81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:01.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.330 7f99d1337700 1 --2- 172.21.2.101:0/1498144187 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99cc1b7dd0 0x7f99cc1b81d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f99c400b8a0 tx=0x7f99c400bc40).stop 2021-02-10T11:44:01.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.330 7f99d1337700 1 -- 172.21.2.101:0/1498144187 shutdown_connections 2021-02-10T11:44:01.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.330 7f99d1337700 1 --2- 172.21.2.101:0/1498144187 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f99cc1b7dd0 0x7f99cc1b81d0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:01.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.330 7f99d1337700 1 --2- 172.21.2.101:0/1498144187 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f99cc1b4160 0x7f99cc1b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:01.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.330 7f99d1337700 1 --2- 172.21.2.101:0/1498144187 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f99b0018730 0x7f99b001abc0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:01.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.330 7f99d1337700 1 --2- 172.21.2.101:0/1498144187 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f99cc09a690 0x7f99cc09aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:01.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.330 7f99d1337700 1 -- 172.21.2.101:0/1498144187 >> 172.21.2.101:0/1498144187 conn(0x7f99cc18e870 msgr2=0x7f99cc18ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:01.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.330 7f99d1337700 1 -- 172.21.2.101:0/1498144187 shutdown_connections 2021-02-10T11:44:01.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.331 7f99d1337700 1 -- 172.21.2.101:0/1498144187 wait complete. 2021-02-10T11:44:01.338 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-02-10T11:44:01.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.754 7f8f7cccb700 1 Processor -- start 2021-02-10T11:44:01.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.754 7f8f7cccb700 1 -- start start 2021-02-10T11:44:01.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.755 7f8f7cccb700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8f780d1870 0x7f8f781b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:01.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.755 7f8f7cccb700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8f780cff70 0x7f8f78109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:01.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.755 7f8f7cccb700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f780fe1c0 con 0x7f8f781b0ae0 2021-02-10T11:44:01.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.755 7f8f7cccb700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f78109ad0 con 0x7f8f780cff70 2021-02-10T11:44:01.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.755 7f8f7cccb700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f78109750 con 0x7f8f780d1870 2021-02-10T11:44:01.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.755 7f8f7705f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8f780cff70 0x7f8f78109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:01.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.755 7f8f7705f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8f780cff70 0x7f8f78109130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59378/0 (socket says 172.21.2.101:59378) 2021-02-10T11:44:01.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.755 7f8f7705f700 1 -- 172.21.2.101:0/857908838 learned_addr learned my addr 172.21.2.101:0/857908838 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:01.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.755 7f8f7705f700 1 -- 172.21.2.101:0/857908838 >> v2:172.21.2.108:3300/0 conn(0x7f8f780d1870 msgr2=0x7f8f781b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:01.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.755 7f8f7705f700 1 --2- 172.21.2.101:0/857908838 >> v2:172.21.2.108:3300/0 conn(0x7f8f780d1870 0x7f8f781b04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:01.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.755 7f8f7705f700 1 -- 172.21.2.101:0/857908838 >> v1:172.21.2.108:6789/0 conn(0x7f8f781b0ae0 legacy=0x7f8f78108940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:01.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.755 7f8f7705f700 1 -- 172.21.2.101:0/857908838 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f78088070 con 0x7f8f780cff70 2021-02-10T11:44:01.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.755 7f8f7705f700 1 --2- 172.21.2.101:0/857908838 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f780cff70 0x7f8f78109130 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 rx=0x7f8f70006f00 tx=0x7f8f70009f10).ready entity=mon.0 client_cookie=afaf4c77fcd04416 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:01.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.755 7f8f7585c700 1 -- 172.21.2.101:0/857908838 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8f7000b3b0 con 0x7f8f780cff70 2021-02-10T11:44:01.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.755 7f8f7585c700 1 -- 172.21.2.101:0/857908838 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7000b630 con 0x7f8f780cff70 2021-02-10T11:44:01.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.755 7f8f7585c700 1 -- 172.21.2.101:0/857908838 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8f7000bb50 con 0x7f8f780cff70 2021-02-10T11:44:01.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.755 7f8f7cccb700 1 -- 172.21.2.101:0/857908838 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f780cff70 msgr2=0x7f8f78109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:01.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.755 7f8f7cccb700 1 --2- 172.21.2.101:0/857908838 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f780cff70 0x7f8f78109130 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 rx=0x7f8f70006f00 tx=0x7f8f70009f10).stop 2021-02-10T11:44:01.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.756 7f8f7cccb700 1 -- 172.21.2.101:0/857908838 shutdown_connections 2021-02-10T11:44:01.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.756 7f8f7cccb700 1 --2- 172.21.2.101:0/857908838 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f780cff70 0x7f8f78109130 unknown :-1 s=CLOSED pgs=2624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:01.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.756 7f8f7cccb700 1 --2- 172.21.2.101:0/857908838 >> v2:172.21.2.108:3300/0 conn(0x7f8f780d1870 0x7f8f781b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:01.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.756 7f8f7cccb700 1 -- 172.21.2.101:0/857908838 >> 172.21.2.101:0/857908838 conn(0x7f8f780ce420 msgr2=0x7f8f780d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:01.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.756 7f8f7cccb700 1 -- 172.21.2.101:0/857908838 shutdown_connections 2021-02-10T11:44:01.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.756 7f8f7cccb700 1 -- 172.21.2.101:0/857908838 wait complete. 2021-02-10T11:44:01.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.770 7f8f7cccb700 1 Processor -- start 2021-02-10T11:44:01.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.770 7f8f7cccb700 1 -- start start 2021-02-10T11:44:01.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.771 7f8f7cccb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8f781b3330 0x7f8f781b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:01.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.771 7f8f7cccb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f781a29d0 0x7f8f78093130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:01.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.771 7f8f7cccb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f781a2dd0 0x7f8f78187e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:01.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.771 7f8f7cccb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f780937d0 con 0x7f8f781a2dd0 2021-02-10T11:44:01.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.771 7f8f7cccb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f781b3730 con 0x7f8f781b3330 2021-02-10T11:44:01.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.771 7f8f7cccb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f78188490 con 0x7f8f781a29d0 2021-02-10T11:44:01.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.771 7f8f7705f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f781a2dd0 0x7f8f78187e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:01.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.771 7f8f7705f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f781a2dd0 0x7f8f78187e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59384/0 (socket says 172.21.2.101:59384) 2021-02-10T11:44:01.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.771 7f8f7705f700 1 -- 172.21.2.101:0/2995532121 learned_addr learned my addr 172.21.2.101:0/2995532121 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:01.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.771 7f8f7705f700 1 -- 172.21.2.101:0/2995532121 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f781a29d0 msgr2=0x7f8f78093130 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:01.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.771 7f8f7705f700 1 --2- 172.21.2.101:0/2995532121 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f781a29d0 0x7f8f78093130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:01.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.771 7f8f7705f700 1 -- 172.21.2.101:0/2995532121 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8f781b3330 msgr2=0x7f8f781b26f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:01.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.771 7f8f7705f700 1 --2- 172.21.2.101:0/2995532121 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8f781b3330 0x7f8f781b26f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:01.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.771 7f8f7705f700 1 -- 172.21.2.101:0/2995532121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f781886b0 con 0x7f8f781a2dd0 2021-02-10T11:44:01.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.771 7f8f7705f700 1 --2- 172.21.2.101:0/2995532121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f781a2dd0 0x7f8f78187e70 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 rx=0x7f8f70009390 tx=0x7f8f7000bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:01.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.772 7f8f6f7fe700 1 -- 172.21.2.101:0/2995532121 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8f70009450 con 0x7f8f781a2dd0 2021-02-10T11:44:01.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.772 7f8f6f7fe700 1 -- 172.21.2.101:0/2995532121 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f700068e0 con 0x7f8f781a2dd0 2021-02-10T11:44:01.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.772 7f8f7cccb700 1 -- 172.21.2.101:0/2995532121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f781886b0 con 0x7f8f781a2dd0 2021-02-10T11:44:01.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.772 7f8f6f7fe700 1 -- 172.21.2.101:0/2995532121 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8f7000c4e0 con 0x7f8f781a2dd0 2021-02-10T11:44:01.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.772 7f8f7cccb700 1 -- 172.21.2.101:0/2995532121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f780b7a00 con 0x7f8f781a2dd0 2021-02-10T11:44:01.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.772 7f8f6f7fe700 1 -- 172.21.2.101:0/2995532121 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8f70001270 con 0x7f8f781a2dd0 2021-02-10T11:44:01.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.772 7f8f6f7fe700 1 --2- 172.21.2.101:0/2995532121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8f5c0186c0 0x7f8f5c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:01.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.772 7f8f6f7fe700 1 -- 172.21.2.101:0/2995532121 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f8f700327a0 con 0x7f8f781a2dd0 2021-02-10T11:44:01.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.772 7f8f7685e700 1 --2- 172.21.2.101:0/2995532121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8f5c0186c0 0x7f8f5c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:01.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.773 7f8f7685e700 1 --2- 172.21.2.101:0/2995532121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8f5c0186c0 0x7f8f5c01abc0 crc :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:01.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.805 7f8f7cccb700 1 -- 172.21.2.101:0/2995532121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f780825c0 con 0x7f8f781a2dd0 2021-02-10T11:44:01.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.807 7f8f6f7fe700 1 -- 172.21.2.101:0/2995532121 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8f70000cc0 con 0x7f8f781a2dd0 2021-02-10T11:44:01.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:01.937 7f8f7cccb700 1 -- 172.21.2.101:0/2995532121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8f780825c0 con 0x7f8f781a2dd0 2021-02-10T11:44:02.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.313 7f8f6f7fe700 1 -- 172.21.2.101:0/2995532121 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v141) v1 ==== 173+0+0 (secure 0 0 0) 0x7f8f70003000 con 0x7f8f781a2dd0 2021-02-10T11:44:02.353 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' removed 2021-02-10T11:44:02.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.356 7f8f7cccb700 1 -- 172.21.2.101:0/2995532121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8f5c0186c0 msgr2=0x7f8f5c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:02.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.356 7f8f7cccb700 1 --2- 172.21.2.101:0/2995532121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8f5c0186c0 0x7f8f5c01abc0 crc :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:02.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.356 7f8f7cccb700 1 -- 172.21.2.101:0/2995532121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f781a2dd0 msgr2=0x7f8f78187e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:02.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.356 7f8f7cccb700 1 --2- 172.21.2.101:0/2995532121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f781a2dd0 0x7f8f78187e70 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 rx=0x7f8f70009390 tx=0x7f8f7000bd40).stop 2021-02-10T11:44:02.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.356 7f8f7cccb700 1 -- 172.21.2.101:0/2995532121 shutdown_connections 2021-02-10T11:44:02.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.356 7f8f7cccb700 1 --2- 172.21.2.101:0/2995532121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8f5c0186c0 0x7f8f5c01abc0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:02.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.356 7f8f7cccb700 1 --2- 172.21.2.101:0/2995532121 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8f781a29d0 0x7f8f78093130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:02.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.356 7f8f7cccb700 1 --2- 172.21.2.101:0/2995532121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8f781a2dd0 0x7f8f78187e70 unknown :-1 s=CLOSED pgs=2625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:02.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.356 7f8f7cccb700 1 --2- 172.21.2.101:0/2995532121 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8f781b3330 0x7f8f781b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:02.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.356 7f8f7cccb700 1 -- 172.21.2.101:0/2995532121 >> 172.21.2.101:0/2995532121 conn(0x7f8f780841d0 msgr2=0x7f8f780827f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:02.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.356 7f8f7cccb700 1 -- 172.21.2.101:0/2995532121 shutdown_connections 2021-02-10T11:44:02.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.356 7f8f7cccb700 1 -- 172.21.2.101:0/2995532121 wait complete. 2021-02-10T11:44:02.364 INFO:tasks.ceph.mds.a:Restarting daemon 2021-02-10T11:44:02.364 INFO:tasks.ceph.mds.a:Stopping old one... 2021-02-10T11:44:02.364 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-02-10T11:44:02.365 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:44:02.365 INFO:tasks.ceph.mds.b:Restarting daemon 2021-02-10T11:44:02.365 INFO:tasks.ceph.mds.b:Stopping old one... 2021-02-10T11:44:02.365 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-02-10T11:44:02.365 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:44:02.371 INFO:tasks.ceph.mds.a:Stopped 2021-02-10T11:44:02.371 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-02-10T11:44:02.372 INFO:tasks.ceph.mds.b:Stopped 2021-02-10T11:44:02.372 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-02-10T11:44:02.413 INFO:tasks.ceph.mds.a:Started 2021-02-10T11:44:02.413 DEBUG:teuthology.parallel:result is None 2021-02-10T11:44:02.414 INFO:tasks.ceph.mds.b:Started 2021-02-10T11:44:02.414 DEBUG:teuthology.parallel:result is None 2021-02-10T11:44:02.414 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd blacklist clear 2021-02-10T11:44:02.818 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:44:02.821 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:44:02.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.825 7f15073c8700 1 Processor -- start 2021-02-10T11:44:02.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.825 7f15073c8700 1 -- start start 2021-02-10T11:44:02.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.826 7f15073c8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f15000d0340 0x7f15000ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:02.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.826 7f15073c8700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15000f0b10 con 0x7f15000d1870 2021-02-10T11:44:02.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.826 7f15073c8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15001b0be0 con 0x7f15000d1c70 2021-02-10T11:44:02.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.826 7f15073c8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15000e5ed0 con 0x7f15000d0340 2021-02-10T11:44:02.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.826 7f15051e4700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f15000d1870 0x7f15001b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59892/0 (socket says 172.21.2.101:59892) 2021-02-10T11:44:02.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.826 7f15051e4700 1 -- 172.21.2.101:0/2797628442 learned_addr learned my addr 172.21.2.101:0/2797628442 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:02.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.826 7f15049e3700 1 --2- 172.21.2.101:0/2797628442 >> v2:172.21.2.101:3300/0 conn(0x7f15000d0340 0x7f15000ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:02.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.826 7f14f3fff700 1 -- 172.21.2.101:0/2797628442 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3463524266 0 0) 0x7f14fc0010b0 con 0x7f15000d1870 2021-02-10T11:44:02.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.826 7f14f3fff700 1 -- 172.21.2.101:0/2797628442 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14ec002fe0 con 0x7f15000d1870 2021-02-10T11:44:02.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.826 7f15049e3700 1 -- 172.21.2.101:0/2797628442 >> v1:172.21.2.108:6789/0 conn(0x7f15000d1c70 legacy=0x7f15000efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:02.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.826 7f15049e3700 1 -- 172.21.2.101:0/2797628442 >> v1:172.21.2.101:6789/0 conn(0x7f15000d1870 legacy=0x7f15001b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:02.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.826 7f15049e3700 1 -- 172.21.2.101:0/2797628442 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15000f0450 con 0x7f15000d0340 2021-02-10T11:44:02.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.827 7f15049e3700 1 --2- 172.21.2.101:0/2797628442 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15000d0340 0x7f15000ef640 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 rx=0x7f14f4006650 tx=0x7f14f4009a00).ready entity=mon.0 client_cookie=40654451ef3cbcb5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:02.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.827 7f14f3fff700 1 -- 172.21.2.101:0/2797628442 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f14f4009df0 con 0x7f15000d0340 2021-02-10T11:44:02.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.827 7f14f3fff700 1 -- 172.21.2.101:0/2797628442 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f40009b0 con 0x7f15000d0340 2021-02-10T11:44:02.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.827 7f14f3fff700 1 -- 172.21.2.101:0/2797628442 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f14f400c2b0 con 0x7f15000d0340 2021-02-10T11:44:02.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.827 7f15073c8700 1 -- 172.21.2.101:0/2797628442 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15000d0340 msgr2=0x7f15000ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:02.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.827 7f15073c8700 1 --2- 172.21.2.101:0/2797628442 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15000d0340 0x7f15000ef640 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 rx=0x7f14f4006650 tx=0x7f14f4009a00).stop 2021-02-10T11:44:02.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.827 7f15073c8700 1 -- 172.21.2.101:0/2797628442 shutdown_connections 2021-02-10T11:44:02.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.827 7f15073c8700 1 --2- 172.21.2.101:0/2797628442 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15000d0340 0x7f15000ef640 unknown :-1 s=CLOSED pgs=2631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:02.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.827 7f15073c8700 1 -- 172.21.2.101:0/2797628442 >> 172.21.2.101:0/2797628442 conn(0x7f15000ce450 msgr2=0x7f15000d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:02.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.828 7f15073c8700 1 -- 172.21.2.101:0/2797628442 shutdown_connections 2021-02-10T11:44:02.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.828 7f15073c8700 1 -- 172.21.2.101:0/2797628442 wait complete. 2021-02-10T11:44:02.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.842 7f15073c8700 1 Processor -- start 2021-02-10T11:44:02.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.842 7f15073c8700 1 -- start start 2021-02-10T11:44:02.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.843 7f15073c8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f150018b550 0x7f150018b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:02.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.843 7f15073c8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15001a3530 0x7f15001a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:02.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.843 7f15073c8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15000c1540 0x7f15000c1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:02.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.843 7f15073c8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15000b5560 con 0x7f15000c1540 2021-02-10T11:44:02.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.843 7f15073c8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15000c1f60 con 0x7f15001a3530 2021-02-10T11:44:02.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.843 7f15073c8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15000c2180 con 0x7f150018b550 2021-02-10T11:44:02.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.843 7f15059e5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15000c1540 0x7f15000c1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:02.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.843 7f15051e4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f150018b550 0x7f150018b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:02.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.843 7f15051e4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f150018b550 0x7f150018b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:52138/0 (socket says 172.21.2.101:52138) 2021-02-10T11:44:02.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.843 7f15059e5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15000c1540 0x7f15000c1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59408/0 (socket says 172.21.2.101:59408) 2021-02-10T11:44:02.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.843 7f15051e4700 1 -- 172.21.2.101:0/1548173191 learned_addr learned my addr 172.21.2.101:0/1548173191 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:02.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.843 7f15049e3700 1 --2- 172.21.2.101:0/1548173191 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15001a3530 0x7f15001a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:02.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.843 7f15049e3700 1 -- 172.21.2.101:0/1548173191 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f150018b550 msgr2=0x7f150018b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:02.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.843 7f15049e3700 1 --2- 172.21.2.101:0/1548173191 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f150018b550 0x7f150018b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:02.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.843 7f15049e3700 1 -- 172.21.2.101:0/1548173191 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15000c1540 msgr2=0x7f15000c1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:02.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.843 7f15049e3700 1 --2- 172.21.2.101:0/1548173191 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15000c1540 0x7f15000c1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:02.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.843 7f15049e3700 1 -- 172.21.2.101:0/1548173191 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15000c23a0 con 0x7f15001a3530 2021-02-10T11:44:02.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.843 7f15059e5700 1 --2- 172.21.2.101:0/1548173191 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15000c1540 0x7f15000c1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:44:02.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.843 7f15049e3700 1 --2- 172.21.2.101:0/1548173191 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15001a3530 0x7f15001a3930 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f14f4006620 tx=0x7f14f400b860).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:02.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.844 7f14f1ffb700 1 -- 172.21.2.101:0/1548173191 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f14f4000c90 con 0x7f15001a3530 2021-02-10T11:44:02.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.844 7f14f1ffb700 1 -- 172.21.2.101:0/1548173191 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f400bc50 con 0x7f15001a3530 2021-02-10T11:44:02.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.844 7f14f1ffb700 1 -- 172.21.2.101:0/1548173191 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f14f40022b0 con 0x7f15001a3530 2021-02-10T11:44:02.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.844 7f15073c8700 1 -- 172.21.2.101:0/1548173191 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15000c23a0 con 0x7f15001a3530 2021-02-10T11:44:02.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.844 7f15073c8700 1 -- 172.21.2.101:0/1548173191 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15000bfa50 con 0x7f15001a3530 2021-02-10T11:44:02.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.844 7f14f1ffb700 1 -- 172.21.2.101:0/1548173191 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f14f4000940 con 0x7f15001a3530 2021-02-10T11:44:02.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.844 7f14f1ffb700 1 --2- 172.21.2.101:0/1548173191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14e00186c0 0x7f14e001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:02.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.844 7f14f1ffb700 1 -- 172.21.2.101:0/1548173191 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 4839+0+0 (secure 0 0 0) 0x7f14f40321e0 con 0x7f15001a3530 2021-02-10T11:44:02.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.844 7f15051e4700 1 --2- 172.21.2.101:0/1548173191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14e00186c0 0x7f14e001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:02.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.845 7f15051e4700 1 --2- 172.21.2.101:0/1548173191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14e00186c0 0x7f14e001abc0 crc :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:02.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.874 7f15073c8700 1 -- 172.21.2.101:0/1548173191 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15000ee0a0 con 0x7f15001a3530 2021-02-10T11:44:02.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:02.878 7f14f1ffb700 1 -- 172.21.2.101:0/1548173191 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f14f40093a0 con 0x7f15001a3530 2021-02-10T11:44:03.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.020 7f15073c8700 1 -- 172.21.2.101:0/1548173191 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f15000ce450 con 0x7f15001a3530 2021-02-10T11:44:03.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.350 7f14f1ffb700 1 -- 172.21.2.101:0/1548173191 <== mon.1 v2:172.21.2.108:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v142) v1 ==== 97+0+0 (secure 0 0 0) 0x7f14f4000940 con 0x7f15001a3530 2021-02-10T11:44:03.385 INFO:teuthology.orchestra.run.gibba001.stderr: removed all blacklist entries 2021-02-10T11:44:03.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.388 7f15073c8700 1 -- 172.21.2.101:0/1548173191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14e00186c0 msgr2=0x7f14e001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:03.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.388 7f15073c8700 1 --2- 172.21.2.101:0/1548173191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14e00186c0 0x7f14e001abc0 crc :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:03.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.388 7f15073c8700 1 -- 172.21.2.101:0/1548173191 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15001a3530 msgr2=0x7f15001a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:03.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.388 7f15073c8700 1 --2- 172.21.2.101:0/1548173191 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15001a3530 0x7f15001a3930 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f14f4006620 tx=0x7f14f400b860).stop 2021-02-10T11:44:03.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.388 7f15073c8700 1 -- 172.21.2.101:0/1548173191 shutdown_connections 2021-02-10T11:44:03.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.388 7f15073c8700 1 --2- 172.21.2.101:0/1548173191 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15000c1540 0x7f15000c1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:03.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.388 7f15073c8700 1 --2- 172.21.2.101:0/1548173191 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15001a3530 0x7f15001a3930 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:03.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.388 7f15073c8700 1 --2- 172.21.2.101:0/1548173191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f14e00186c0 0x7f14e001abc0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:03.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.388 7f15073c8700 1 --2- 172.21.2.101:0/1548173191 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f150018b550 0x7f150018b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:03.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.388 7f15073c8700 1 -- 172.21.2.101:0/1548173191 >> 172.21.2.101:0/1548173191 conn(0x7f15000a6870 msgr2=0x7f150008a420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:03.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.388 7f15073c8700 1 -- 172.21.2.101:0/1548173191 shutdown_connections 2021-02-10T11:44:03.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.388 7f15073c8700 1 -- 172.21.2.101:0/1548173191 wait complete. 2021-02-10T11:44:03.395 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2021-02-10T11:44:03.396 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:44:03.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.808 7fc20be10700 1 Processor -- start 2021-02-10T11:44:03.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.809 7fc20be10700 1 -- start start 2021-02-10T11:44:03.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.809 7fc20be10700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc2040d1870 0x7fc2041b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:03.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.809 7fc20be10700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2041b0a10 0x7fc2040cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:03.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.809 7fc20be10700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2040f0b10 con 0x7fc2040d1c70 2021-02-10T11:44:03.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.809 7fc20be10700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2040e5ed0 con 0x7fc2040d1870 2021-02-10T11:44:03.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.809 7fc20be10700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2040f0650 con 0x7fc2041b0a10 2021-02-10T11:44:03.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.809 7fc20a42d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc2040d1c70 0x7fc2040cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59906/0 (socket says 172.21.2.101:59906) 2021-02-10T11:44:03.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.809 7fc20a42d700 1 -- 172.21.2.101:0/4057109793 learned_addr learned my addr 172.21.2.101:0/4057109793 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:03.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.809 7fc209c2c700 1 --2- 172.21.2.101:0/4057109793 >> v2:172.21.2.101:3300/0 conn(0x7fc2040d1870 0x7fc2041b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:03.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.809 7fc208c2a700 1 -- 172.21.2.101:0/4057109793 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1011016940 0 0) 0x7fc1fc001500 con 0x7fc2040d1c70 2021-02-10T11:44:03.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.809 7fc208c2a700 1 -- 172.21.2.101:0/4057109793 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1f0002fe0 con 0x7fc2040d1c70 2021-02-10T11:44:03.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.810 7fc209c2c700 1 -- 172.21.2.101:0/4057109793 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2041b0a10 msgr2=0x7fc2040cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:03.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.810 7fc209c2c700 1 --2- 172.21.2.101:0/4057109793 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2041b0a10 0x7fc2040cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:03.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.810 7fc209c2c700 1 -- 172.21.2.101:0/4057109793 >> v1:172.21.2.101:6789/0 conn(0x7fc2040d1c70 legacy=0x7fc2040cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:03.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.810 7fc209c2c700 1 -- 172.21.2.101:0/4057109793 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2040f0870 con 0x7fc2040d1870 2021-02-10T11:44:03.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.810 7fc209c2c700 1 --2- 172.21.2.101:0/4057109793 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2040d1870 0x7fc2041b03f0 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 rx=0x7fc200006650 tx=0x7fc200009a80).ready entity=mon.0 client_cookie=e012eab8f397facf server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:03.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.810 7fc208c2a700 1 -- 172.21.2.101:0/4057109793 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc20000c0c0 con 0x7fc2040d1870 2021-02-10T11:44:03.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.810 7fc208c2a700 1 -- 172.21.2.101:0/4057109793 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20000c340 con 0x7fc2040d1870 2021-02-10T11:44:03.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.810 7fc208c2a700 1 -- 172.21.2.101:0/4057109793 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc20000c860 con 0x7fc2040d1870 2021-02-10T11:44:03.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.810 7fc20be10700 1 -- 172.21.2.101:0/4057109793 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2040d1870 msgr2=0x7fc2041b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:03.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.810 7fc20be10700 1 --2- 172.21.2.101:0/4057109793 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2040d1870 0x7fc2041b03f0 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 rx=0x7fc200006650 tx=0x7fc200009a80).stop 2021-02-10T11:44:03.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.810 7fc20be10700 1 -- 172.21.2.101:0/4057109793 shutdown_connections 2021-02-10T11:44:03.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.811 7fc20be10700 1 --2- 172.21.2.101:0/4057109793 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2041b0a10 0x7fc2040cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:03.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.811 7fc20be10700 1 --2- 172.21.2.101:0/4057109793 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2040d1870 0x7fc2041b03f0 unknown :-1 s=CLOSED pgs=2633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:03.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.811 7fc20be10700 1 -- 172.21.2.101:0/4057109793 >> 172.21.2.101:0/4057109793 conn(0x7fc2040ce450 msgr2=0x7fc2040d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:03.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.811 7fc20be10700 1 -- 172.21.2.101:0/4057109793 shutdown_connections 2021-02-10T11:44:03.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.811 7fc20be10700 1 -- 172.21.2.101:0/4057109793 wait complete. 2021-02-10T11:44:03.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.825 7fc20be10700 1 Processor -- start 2021-02-10T11:44:03.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.825 7fc20be10700 1 -- start start 2021-02-10T11:44:03.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.826 7fc20be10700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc20418d870 0x7fc2040b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:03.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.826 7fc20be10700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2040b1fd0 0x7fc204099ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:03.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.826 7fc20be10700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2040b23d0 0x7fc20409a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:03.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.826 7fc20be10700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20418dc70 con 0x7fc2040b1fd0 2021-02-10T11:44:03.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.826 7fc20be10700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc204199e60 con 0x7fc20418d870 2021-02-10T11:44:03.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.826 7fc20be10700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20409ab30 con 0x7fc2040b23d0 2021-02-10T11:44:03.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.826 7fc20942b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2040b1fd0 0x7fc204099ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:03.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.826 7fc20942b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2040b1fd0 0x7fc204099ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59416/0 (socket says 172.21.2.101:59416) 2021-02-10T11:44:03.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.826 7fc20942b700 1 -- 172.21.2.101:0/2739216305 learned_addr learned my addr 172.21.2.101:0/2739216305 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:03.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.826 7fc209c2c700 1 --2- 172.21.2.101:0/2739216305 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc20418d870 0x7fc2040b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:03.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.826 7fc20a42d700 1 --2- 172.21.2.101:0/2739216305 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2040b23d0 0x7fc20409a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:03.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.826 7fc20942b700 1 -- 172.21.2.101:0/2739216305 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2040b23d0 msgr2=0x7fc20409a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:03.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.826 7fc20942b700 1 --2- 172.21.2.101:0/2739216305 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2040b23d0 0x7fc20409a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:03.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.826 7fc20942b700 1 -- 172.21.2.101:0/2739216305 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc20418d870 msgr2=0x7fc2040b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:03.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.826 7fc20942b700 1 --2- 172.21.2.101:0/2739216305 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc20418d870 0x7fc2040b19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:03.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.826 7fc20942b700 1 -- 172.21.2.101:0/2739216305 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20409ad50 con 0x7fc2040b1fd0 2021-02-10T11:44:03.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.826 7fc20a42d700 1 --2- 172.21.2.101:0/2739216305 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2040b23d0 0x7fc20409a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:44:03.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.827 7fc20942b700 1 --2- 172.21.2.101:0/2739216305 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2040b1fd0 0x7fc204099ef0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 rx=0x7fc1f8006f00 tx=0x7fc1f800b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:03.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.827 7fc1f67fc700 1 -- 172.21.2.101:0/2739216305 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc1f800bd30 con 0x7fc2040b1fd0 2021-02-10T11:44:03.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.827 7fc1f67fc700 1 -- 172.21.2.101:0/2739216305 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f8000c90 con 0x7fc2040b1fd0 2021-02-10T11:44:03.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.827 7fc20be10700 1 -- 172.21.2.101:0/2739216305 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20409ad50 con 0x7fc2040b1fd0 2021-02-10T11:44:03.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.827 7fc1f67fc700 1 -- 172.21.2.101:0/2739216305 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc1f80022a0 con 0x7fc2040b1fd0 2021-02-10T11:44:03.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.827 7fc20be10700 1 -- 172.21.2.101:0/2739216305 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20409ad50 con 0x7fc2040b1fd0 2021-02-10T11:44:03.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.827 7fc1f67fc700 1 -- 172.21.2.101:0/2739216305 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc1f8005310 con 0x7fc2040b1fd0 2021-02-10T11:44:03.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.828 7fc1f67fc700 1 --2- 172.21.2.101:0/2739216305 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1ec018730 0x7fc1ec01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:03.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.828 7fc1f67fc700 1 -- 172.21.2.101:0/2739216305 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fc1f8032450 con 0x7fc2040b1fd0 2021-02-10T11:44:03.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.828 7fc209c2c700 1 --2- 172.21.2.101:0/2739216305 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1ec018730 0x7fc1ec01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:03.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.828 7fc209c2c700 1 --2- 172.21.2.101:0/2739216305 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1ec018730 0x7fc1ec01ac10 crc :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:03.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.859 7fc20be10700 1 -- 172.21.2.101:0/2739216305 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2040999c0 con 0x7fc2040b1fd0 2021-02-10T11:44:03.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.861 7fc1f67fc700 1 -- 172.21.2.101:0/2739216305 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc1f8006b90 con 0x7fc2040b1fd0 2021-02-10T11:44:03.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.994 7fc20be10700 1 -- 172.21.2.101:0/2739216305 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc2040999c0 con 0x7fc2040b1fd0 2021-02-10T11:44:03.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.994 7fc1f67fc700 1 -- 172.21.2.101:0/2739216305 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v82) v1 ==== 80+0+4513 (secure 0 0 0) 0x7fc1f800bd30 con 0x7fc2040b1fd0 2021-02-10T11:44:03.994 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:44:03.994 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:44:03.994 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:44:03.994 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:03.994 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:44:03.995 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:44:03.995 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:03.995 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:44:03.995 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:44:03.995 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:44:03.995 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:03.995 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:03.995 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:03.995 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:03.995 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:44:03.995 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:44:03.995 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:03.995 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:44:03.995 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:44:03.996 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:44:03.996 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:03.996 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:03.996 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:03.996 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:03.996 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:44:03.996 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:44:03.996 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:03.996 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:44:03.996 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:44:03.996 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:03.996 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:03.996 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:03.997 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:03.997 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:44:03.997 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:44:03.997 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:03.997 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:44:03.997 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:44:03.997 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:03.997 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:03.997 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:03.997 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:03.997 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:44:03.997 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:44:03.998 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:03.998 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:44:03.998 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:44:03.998 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:03.998 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:03.998 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:03.998 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:03.998 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:44:03.998 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:44:03.998 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:03.998 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:44:03.998 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:44:03.998 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:03.998 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:03.999 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:03.999 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:03.999 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:44:03.999 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:44:03.999 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:03.999 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:44:03.999 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:44:03.999 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:03.999 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:03.999 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:03.999 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:03.999 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:44:03.999 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:44:04.000 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:04.000 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:44:04.000 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:44:04.000 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:04.000 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:04.000 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:04.000 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:04.000 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:44:04.000 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:44:04.000 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:04.000 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:44:04.000 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:44:04.000 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:04.001 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:04.001 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:04.001 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:04.001 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:44:04.001 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:44:04.001 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:04.001 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:44:04.001 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:44:04.001 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:04.001 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:04.001 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:04.001 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:04.001 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:44:04.001 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:44:04.002 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:04.002 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:44:04.002 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:44:04.002 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:44:04.002 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:04.002 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:04.002 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:04.002 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:44:04.002 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:44:04.002 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:04.002 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:44:04.002 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:44:04.002 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:44:04.003 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:04.003 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:04.003 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:04.003 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:44:04.003 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:44:04.003 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:04.003 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:44:04.003 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:44:04.003 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:44:04.003 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:04.003 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:04.003 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:04.003 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:44:04.004 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:44:04.004 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:04.004 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:44:04.004 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:44:04.004 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:44:04.004 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:04.004 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:04.004 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:04.004 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:44:04.004 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:44:04.004 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:04.004 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:44:04.004 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:44:04.004 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:44:04.004 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:04.005 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:04.005 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:04.005 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:04.005 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:44:04.005 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:44:04.005 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:04.005 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:44:04.005 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:44:04.005 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:04.005 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:04.005 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:04.005 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:04.005 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:44:04.005 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:44:04.005 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:04.005 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:44:04.006 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:44:04.006 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:04.006 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:04.006 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:04.006 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:04.006 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:44:04.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.996 7fc20be10700 1 -- 172.21.2.101:0/2739216305 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1ec018730 msgr2=0x7fc1ec01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:04.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.996 7fc20be10700 1 --2- 172.21.2.101:0/2739216305 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1ec018730 0x7fc1ec01ac10 crc :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:04.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.996 7fc20be10700 1 -- 172.21.2.101:0/2739216305 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2040b1fd0 msgr2=0x7fc204099ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:04.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.996 7fc20be10700 1 --2- 172.21.2.101:0/2739216305 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2040b1fd0 0x7fc204099ef0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 rx=0x7fc1f8006f00 tx=0x7fc1f800b940).stop 2021-02-10T11:44:04.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.996 7fc20be10700 1 -- 172.21.2.101:0/2739216305 shutdown_connections 2021-02-10T11:44:04.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.996 7fc20be10700 1 --2- 172.21.2.101:0/2739216305 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1ec018730 0x7fc1ec01ac10 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:04.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.996 7fc20be10700 1 --2- 172.21.2.101:0/2739216305 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc2040b23d0 0x7fc20409a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:04.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.996 7fc20be10700 1 --2- 172.21.2.101:0/2739216305 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc2040b1fd0 0x7fc204099ef0 unknown :-1 s=CLOSED pgs=2634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:04.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.996 7fc20be10700 1 --2- 172.21.2.101:0/2739216305 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc20418d870 0x7fc2040b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:04.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.996 7fc20be10700 1 -- 172.21.2.101:0/2739216305 >> 172.21.2.101:0/2739216305 conn(0x7fc2040f0550 msgr2=0x7fc204191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:04.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.996 7fc20be10700 1 -- 172.21.2.101:0/2739216305 shutdown_connections 2021-02-10T11:44:04.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:03.996 7fc20be10700 1 -- 172.21.2.101:0/2739216305 wait complete. 2021-02-10T11:44:04.008 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:44:04.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.420 7fc4a37b6700 1 Processor -- start 2021-02-10T11:44:04.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.420 7fc4a37b6700 1 -- start start 2021-02-10T11:44:04.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.420 7fc4a37b6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc49c0d1870 0x7fc49c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:04.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.421 7fc4a37b6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc49c0f0650 0x7fc49c0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:04.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.421 7fc4a37b6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc49c0f0b10 con 0x7fc49c1b0a10 2021-02-10T11:44:04.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.421 7fc4a37b6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49c0e5ed0 con 0x7fc49c0d1870 2021-02-10T11:44:04.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.421 7fc4a37b6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0d1c70 con 0x7fc49c0f0650 2021-02-10T11:44:04.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.421 7fc4a15d2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc49c0d1870 0x7fc49c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:04.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.421 7fc4a15d2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc49c0d1870 0x7fc49c1b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59422/0 (socket says 172.21.2.101:59422) 2021-02-10T11:44:04.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.421 7fc4a15d2700 1 -- 172.21.2.101:0/2708184879 learned_addr learned my addr 172.21.2.101:0/2708184879 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:04.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.421 7fc4a15d2700 1 -- 172.21.2.101:0/2708184879 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc49c0f0650 msgr2=0x7fc49c0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:04.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.421 7fc4a15d2700 1 --2- 172.21.2.101:0/2708184879 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc49c0f0650 0x7fc49c0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:04.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.421 7fc4a15d2700 1 -- 172.21.2.101:0/2708184879 >> v1:172.21.2.108:6789/0 conn(0x7fc49c1b0a10 legacy=0x7fc49c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:04.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.421 7fc4a15d2700 1 -- 172.21.2.101:0/2708184879 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c0d1e90 con 0x7fc49c0d1870 2021-02-10T11:44:04.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.421 7fc4a15d2700 1 --2- 172.21.2.101:0/2708184879 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc49c0d1870 0x7fc49c1b03f0 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 rx=0x7fc498006700 tx=0x7fc498009b80).ready entity=mon.0 client_cookie=3778835db40fcf5f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:04.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.421 7fc493fff700 1 -- 172.21.2.101:0/2708184879 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc49800c0c0 con 0x7fc49c0d1870 2021-02-10T11:44:04.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.421 7fc493fff700 1 -- 172.21.2.101:0/2708184879 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49800c340 con 0x7fc49c0d1870 2021-02-10T11:44:04.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.421 7fc493fff700 1 -- 172.21.2.101:0/2708184879 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc49800c860 con 0x7fc49c0d1870 2021-02-10T11:44:04.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.421 7fc4a37b6700 1 -- 172.21.2.101:0/2708184879 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc49c0d1870 msgr2=0x7fc49c1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:04.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.421 7fc4a37b6700 1 --2- 172.21.2.101:0/2708184879 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc49c0d1870 0x7fc49c1b03f0 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 rx=0x7fc498006700 tx=0x7fc498009b80).stop 2021-02-10T11:44:04.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.422 7fc4a37b6700 1 -- 172.21.2.101:0/2708184879 shutdown_connections 2021-02-10T11:44:04.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.422 7fc4a37b6700 1 --2- 172.21.2.101:0/2708184879 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc49c0f0650 0x7fc49c0cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:04.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.422 7fc4a37b6700 1 --2- 172.21.2.101:0/2708184879 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc49c0d1870 0x7fc49c1b03f0 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:04.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.422 7fc4a37b6700 1 -- 172.21.2.101:0/2708184879 >> 172.21.2.101:0/2708184879 conn(0x7fc49c0ce450 msgr2=0x7fc49c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:04.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.422 7fc4a37b6700 1 -- 172.21.2.101:0/2708184879 shutdown_connections 2021-02-10T11:44:04.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.422 7fc4a37b6700 1 -- 172.21.2.101:0/2708184879 wait complete. 2021-02-10T11:44:04.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.437 7fc4a37b6700 1 Processor -- start 2021-02-10T11:44:04.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.437 7fc4a37b6700 1 -- start start 2021-02-10T11:44:04.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.437 7fc4a37b6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc49c0b0ad0 0x7fc49c091f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:04.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.437 7fc4a37b6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc49c1a8850 0x7fc49c1a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:04.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.437 7fc4a37b6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc49c1a6fa0 0x7fc49c1a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:04.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.437 7fc4a37b6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c1abfe0 con 0x7fc49c1a8850 2021-02-10T11:44:04.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.437 7fc4a37b6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c1a9c40 con 0x7fc49c1a6fa0 2021-02-10T11:44:04.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.437 7fc4a37b6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c1a9730 con 0x7fc49c0b0ad0 2021-02-10T11:44:04.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.438 7fc4a0dd1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc49c1a8850 0x7fc49c1a8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:04.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.438 7fc4a0dd1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc49c1a8850 0x7fc49c1a8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59430/0 (socket says 172.21.2.101:59430) 2021-02-10T11:44:04.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.438 7fc4a0dd1700 1 -- 172.21.2.101:0/81799354 learned_addr learned my addr 172.21.2.101:0/81799354 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:04.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.438 7fc4a0dd1700 1 -- 172.21.2.101:0/81799354 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc49c0b0ad0 msgr2=0x7fc49c091f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:04.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.438 7fc4a0dd1700 1 --2- 172.21.2.101:0/81799354 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc49c0b0ad0 0x7fc49c091f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:04.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.438 7fc4a0dd1700 1 -- 172.21.2.101:0/81799354 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc49c1a6fa0 msgr2=0x7fc49c1a73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:04.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.438 7fc4a0dd1700 1 --2- 172.21.2.101:0/81799354 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc49c1a6fa0 0x7fc49c1a73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:04.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.438 7fc4a0dd1700 1 -- 172.21.2.101:0/81799354 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c1a9270 con 0x7fc49c1a8850 2021-02-10T11:44:04.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.438 7fc4a0dd1700 1 --2- 172.21.2.101:0/81799354 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc49c1a8850 0x7fc49c1a8c50 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 rx=0x7fc48c00b5b0 tx=0x7fc48c00b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:04.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.438 7fc491ffb700 1 -- 172.21.2.101:0/81799354 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc48c00bd40 con 0x7fc49c1a8850 2021-02-10T11:44:04.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.438 7fc491ffb700 1 -- 172.21.2.101:0/81799354 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48c000e20 con 0x7fc49c1a8850 2021-02-10T11:44:04.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.438 7fc4a37b6700 1 -- 172.21.2.101:0/81799354 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49c1a9270 con 0x7fc49c1a8850 2021-02-10T11:44:04.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.438 7fc4a37b6700 1 -- 172.21.2.101:0/81799354 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49c1a9270 con 0x7fc49c1a8850 2021-02-10T11:44:04.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.438 7fc491ffb700 1 -- 172.21.2.101:0/81799354 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc48c0054a0 con 0x7fc49c1a8850 2021-02-10T11:44:04.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.439 7fc491ffb700 1 -- 172.21.2.101:0/81799354 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc48c006240 con 0x7fc49c1a8850 2021-02-10T11:44:04.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.439 7fc491ffb700 1 --2- 172.21.2.101:0/81799354 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc47c018730 0x7fc47c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:04.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.439 7fc491ffb700 1 -- 172.21.2.101:0/81799354 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fc48c0324c0 con 0x7fc49c1a8850 2021-02-10T11:44:04.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.439 7fc4a15d2700 1 --2- 172.21.2.101:0/81799354 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc47c018730 0x7fc47c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:04.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.439 7fc4a15d2700 1 --2- 172.21.2.101:0/81799354 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc47c018730 0x7fc47c01ac10 crc :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:04.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.470 7fc4a37b6700 1 -- 172.21.2.101:0/81799354 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc49c1abfe0 con 0x7fc49c1a8850 2021-02-10T11:44:04.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.473 7fc491ffb700 1 -- 172.21.2.101:0/81799354 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc48c006030 con 0x7fc49c1a8850 2021-02-10T11:44:04.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.597 7fc4a37b6700 1 -- 172.21.2.101:0/81799354 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc49c1abfe0 con 0x7fc49c1a8850 2021-02-10T11:44:04.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.597 7fc491ffb700 1 -- 172.21.2.101:0/81799354 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+11154 (secure 0 0 0) 0x7fc48c006240 con 0x7fc49c1a8850 2021-02-10T11:44:04.597 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:44:04.598 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":142,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:44:03.344800","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:44:04.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.599 7fc4a37b6700 1 -- 172.21.2.101:0/81799354 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc47c018730 msgr2=0x7fc47c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:04.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.599 7fc4a37b6700 1 --2- 172.21.2.101:0/81799354 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc47c018730 0x7fc47c01ac10 crc :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:04.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.599 7fc4a37b6700 1 -- 172.21.2.101:0/81799354 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc49c1a8850 msgr2=0x7fc49c1a8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:04.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.599 7fc4a37b6700 1 --2- 172.21.2.101:0/81799354 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc49c1a8850 0x7fc49c1a8c50 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 rx=0x7fc48c00b5b0 tx=0x7fc48c00b950).stop 2021-02-10T11:44:04.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.599 7fc4a37b6700 1 -- 172.21.2.101:0/81799354 shutdown_connections 2021-02-10T11:44:04.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.599 7fc4a37b6700 1 --2- 172.21.2.101:0/81799354 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc49c1a6fa0 0x7fc49c1a73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:04.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.599 7fc4a37b6700 1 --2- 172.21.2.101:0/81799354 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc49c1a8850 0x7fc49c1a8c50 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:04.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.599 7fc4a37b6700 1 --2- 172.21.2.101:0/81799354 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc47c018730 0x7fc47c01ac10 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:04.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.599 7fc4a37b6700 1 --2- 172.21.2.101:0/81799354 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc49c0b0ad0 0x7fc49c091f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:04.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.599 7fc4a37b6700 1 -- 172.21.2.101:0/81799354 >> 172.21.2.101:0/81799354 conn(0x7fc49c19f8f0 msgr2=0x7fc49c1a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:04.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.599 7fc4a37b6700 1 -- 172.21.2.101:0/81799354 shutdown_connections 2021-02-10T11:44:04.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:04.599 7fc4a37b6700 1 -- 172.21.2.101:0/81799354 wait complete. 2021-02-10T11:44:04.607 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-10T11:44:04.607 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:44:05.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.025 7f926caed700 1 Processor -- start 2021-02-10T11:44:05.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.025 7f926caed700 1 -- start start 2021-02-10T11:44:05.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.026 7f926caed700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f92680d1870 0x7f92681b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:05.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.026 7f926caed700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f92681b0ae0 0x7f9268108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:05.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.026 7f926caed700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f92680d1c70 0x7f9268108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:05.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.026 7f926caed700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92680fe1c0 con 0x7f92680d1870 2021-02-10T11:44:05.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.026 7f926caed700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9268109a60 con 0x7f92681b0ae0 2021-02-10T11:44:05.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.026 7f926caed700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9268088070 con 0x7f92680d1c70 2021-02-10T11:44:05.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.026 7f926685e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f92680d1870 0x7f92681b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:05.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.026 7f926685e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f92680d1870 0x7f92681b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59436/0 (socket says 172.21.2.101:59436) 2021-02-10T11:44:05.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.026 7f926685e700 1 -- 172.21.2.101:0/3289162747 learned_addr learned my addr 172.21.2.101:0/3289162747 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:05.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.026 7f926685e700 1 -- 172.21.2.101:0/3289162747 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f92680d1c70 msgr2=0x7f9268108f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:05.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.026 7f926685e700 1 --2- 172.21.2.101:0/3289162747 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f92680d1c70 0x7f9268108f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:05.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.026 7f926685e700 1 -- 172.21.2.101:0/3289162747 >> v2:172.21.2.108:3300/0 conn(0x7f92681b0ae0 msgr2=0x7f9268108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:05.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.026 7f926685e700 1 --2- 172.21.2.101:0/3289162747 >> v2:172.21.2.108:3300/0 conn(0x7f92681b0ae0 0x7f9268108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:05.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.026 7f926685e700 1 -- 172.21.2.101:0/3289162747 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9268109580 con 0x7f92680d1870 2021-02-10T11:44:05.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.026 7f926685e700 1 --2- 172.21.2.101:0/3289162747 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f92680d1870 0x7f92681b04c0 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 rx=0x7f9250006700 tx=0x7f9250009b80).ready entity=mon.0 client_cookie=254742a6bdb5212a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:05.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.027 7f926585c700 1 -- 172.21.2.101:0/3289162747 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f925000c0c0 con 0x7f92680d1870 2021-02-10T11:44:05.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.027 7f926585c700 1 -- 172.21.2.101:0/3289162747 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925000c340 con 0x7f92680d1870 2021-02-10T11:44:05.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.027 7f926585c700 1 -- 172.21.2.101:0/3289162747 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f925000c860 con 0x7f92680d1870 2021-02-10T11:44:05.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.027 7f926caed700 1 -- 172.21.2.101:0/3289162747 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f92680d1870 msgr2=0x7f92681b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:05.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.027 7f926caed700 1 --2- 172.21.2.101:0/3289162747 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f92680d1870 0x7f92681b04c0 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 rx=0x7f9250006700 tx=0x7f9250009b80).stop 2021-02-10T11:44:05.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.027 7f926caed700 1 -- 172.21.2.101:0/3289162747 shutdown_connections 2021-02-10T11:44:05.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.027 7f926caed700 1 --2- 172.21.2.101:0/3289162747 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f92680d1c70 0x7f9268108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:05.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.027 7f926caed700 1 --2- 172.21.2.101:0/3289162747 >> v2:172.21.2.108:3300/0 conn(0x7f92681b0ae0 0x7f9268108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:05.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.027 7f926caed700 1 --2- 172.21.2.101:0/3289162747 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f92680d1870 0x7f92681b04c0 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:05.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.027 7f926caed700 1 -- 172.21.2.101:0/3289162747 >> 172.21.2.101:0/3289162747 conn(0x7f92680ce420 msgr2=0x7f92680d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:05.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.028 7f926caed700 1 -- 172.21.2.101:0/3289162747 shutdown_connections 2021-02-10T11:44:05.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.028 7f926caed700 1 -- 172.21.2.101:0/3289162747 wait complete. 2021-02-10T11:44:05.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.042 7f926caed700 1 Processor -- start 2021-02-10T11:44:05.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.042 7f926caed700 1 -- start start 2021-02-10T11:44:05.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.043 7f926caed700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f92680c3820 0x7f92680a7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:05.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.043 7f926caed700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f92681a44c0 0x7f92681a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:05.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.043 7f926caed700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f92680863d0 0x7f92680867d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:05.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.043 7f926caed700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9268085ed0 con 0x7f92680863d0 2021-02-10T11:44:05.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.043 7f926caed700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92680c0740 con 0x7f92680c3820 2021-02-10T11:44:05.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.043 7f926caed700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9268086df0 con 0x7f92681a44c0 2021-02-10T11:44:05.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.043 7f926705f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f92680863d0 0x7f92680867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:05.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.043 7f926705f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f92680863d0 0x7f92680867d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59446/0 (socket says 172.21.2.101:59446) 2021-02-10T11:44:05.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.043 7f926705f700 1 -- 172.21.2.101:0/4100831145 learned_addr learned my addr 172.21.2.101:0/4100831145 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:05.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.043 7f926705f700 1 -- 172.21.2.101:0/4100831145 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f92681a44c0 msgr2=0x7f92681a48c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:05.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.043 7f926705f700 1 --2- 172.21.2.101:0/4100831145 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f92681a44c0 0x7f92681a48c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:05.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.043 7f926705f700 1 -- 172.21.2.101:0/4100831145 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f92680c3820 msgr2=0x7f92680a7950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:05.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.043 7f926705f700 1 --2- 172.21.2.101:0/4100831145 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f92680c3820 0x7f92680a7950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:05.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.043 7f926705f700 1 -- 172.21.2.101:0/4100831145 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9268087010 con 0x7f92680863d0 2021-02-10T11:44:05.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.043 7f926705f700 1 --2- 172.21.2.101:0/4100831145 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f92680863d0 0x7f92680867d0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 rx=0x7f9260009960 tx=0x7f926000bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:05.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.044 7f925f7fe700 1 -- 172.21.2.101:0/4100831145 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9260003030 con 0x7f92680863d0 2021-02-10T11:44:05.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.044 7f925f7fe700 1 -- 172.21.2.101:0/4100831145 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9260001310 con 0x7f92680863d0 2021-02-10T11:44:05.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.044 7f925f7fe700 1 -- 172.21.2.101:0/4100831145 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9260001830 con 0x7f92680863d0 2021-02-10T11:44:05.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.044 7f926caed700 1 -- 172.21.2.101:0/4100831145 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9268087010 con 0x7f92680863d0 2021-02-10T11:44:05.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.044 7f926caed700 1 -- 172.21.2.101:0/4100831145 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9268083000 con 0x7f92680863d0 2021-02-10T11:44:05.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.044 7f925f7fe700 1 -- 172.21.2.101:0/4100831145 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f926000a030 con 0x7f92680863d0 2021-02-10T11:44:05.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.045 7f925f7fe700 1 --2- 172.21.2.101:0/4100831145 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f924c0186c0 0x7f924c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:05.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.045 7f925f7fe700 1 -- 172.21.2.101:0/4100831145 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f92600325b0 con 0x7f92680863d0 2021-02-10T11:44:05.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.045 7f926685e700 1 --2- 172.21.2.101:0/4100831145 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f924c0186c0 0x7f924c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:05.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.045 7f926685e700 1 --2- 172.21.2.101:0/4100831145 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f924c0186c0 0x7f924c01abc0 crc :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:05.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.076 7f926caed700 1 -- 172.21.2.101:0/4100831145 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9268085e50 con 0x7f92680863d0 2021-02-10T11:44:05.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.078 7f925f7fe700 1 -- 172.21.2.101:0/4100831145 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9260000cc0 con 0x7f92680863d0 2021-02-10T11:44:05.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.207 7f926caed700 1 -- 172.21.2.101:0/4100831145 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f92680fe1c0 con 0x7f92680863d0 2021-02-10T11:44:05.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.208 7f925f7fe700 1 -- 172.21.2.101:0/4100831145 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v142) v1 ==== 92+0+10 (secure 0 0 0) 0x7f926000a030 con 0x7f92680863d0 2021-02-10T11:44:05.206 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:44:05.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.209 7f926caed700 1 -- 172.21.2.101:0/4100831145 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f924c0186c0 msgr2=0x7f924c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:05.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.209 7f926caed700 1 --2- 172.21.2.101:0/4100831145 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f924c0186c0 0x7f924c01abc0 crc :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:05.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.209 7f926caed700 1 -- 172.21.2.101:0/4100831145 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f92680863d0 msgr2=0x7f92680867d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:05.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.209 7f926caed700 1 --2- 172.21.2.101:0/4100831145 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f92680863d0 0x7f92680867d0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 rx=0x7f9260009960 tx=0x7f926000bcf0).stop 2021-02-10T11:44:05.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.210 7f926caed700 1 -- 172.21.2.101:0/4100831145 shutdown_connections 2021-02-10T11:44:05.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.210 7f926caed700 1 --2- 172.21.2.101:0/4100831145 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f924c0186c0 0x7f924c01abc0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:05.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.210 7f926caed700 1 --2- 172.21.2.101:0/4100831145 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f92681a44c0 0x7f92681a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:05.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.210 7f926caed700 1 --2- 172.21.2.101:0/4100831145 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f92680863d0 0x7f92680867d0 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:05.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.210 7f926caed700 1 --2- 172.21.2.101:0/4100831145 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f92680c3820 0x7f92680a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:05.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.210 7f926caed700 1 -- 172.21.2.101:0/4100831145 >> 172.21.2.101:0/4100831145 conn(0x7f92680872c0 msgr2=0x7f9268095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:05.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.210 7f926caed700 1 -- 172.21.2.101:0/4100831145 shutdown_connections 2021-02-10T11:44:05.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.210 7f926caed700 1 -- 172.21.2.101:0/4100831145 wait complete. 2021-02-10T11:44:05.217 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --format=json-pretty osd lspools 2021-02-10T11:44:05.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.631 7fd41ca0e700 1 Processor -- start 2021-02-10T11:44:05.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.631 7fd41ca0e700 1 -- start start 2021-02-10T11:44:05.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.632 7fd41ca0e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4180d1870 0x7fd4181b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:05.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.632 7fd41ca0e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4180f0b10 con 0x7fd4181b0a10 2021-02-10T11:44:05.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.632 7fd41ca0e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4180d0310 con 0x7fd4180f0650 2021-02-10T11:44:05.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.632 7fd41ca0e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4180e5ed0 con 0x7fd4180d1870 2021-02-10T11:44:05.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.632 7fd415e1b700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd4181b0a10 0x7fd4180cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59944/0 (socket says 172.21.2.101:59944) 2021-02-10T11:44:05.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.632 7fd415e1b700 1 -- 172.21.2.101:0/3950938920 learned_addr learned my addr 172.21.2.101:0/3950938920 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:05.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.632 7fd41661c700 1 --2- 172.21.2.101:0/3950938920 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4180d1870 0x7fd4181b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:05.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.632 7fd41561a700 1 -- 172.21.2.101:0/3950938920 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2467543687 0 0) 0x7fd40c0009b0 con 0x7fd4181b0a10 2021-02-10T11:44:05.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.633 7fd41561a700 1 -- 172.21.2.101:0/3950938920 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd400002fe0 con 0x7fd4181b0a10 2021-02-10T11:44:05.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.633 7fd41561a700 1 -- 172.21.2.101:0/3950938920 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3693775162 0 0) 0x7fd410001400 con 0x7fd4180f0650 2021-02-10T11:44:05.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.633 7fd41561a700 1 -- 172.21.2.101:0/3950938920 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd400003210 con 0x7fd4180f0650 2021-02-10T11:44:05.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.633 7fd41661c700 1 -- 172.21.2.101:0/3950938920 >> v1:172.21.2.108:6789/0 conn(0x7fd4180f0650 legacy=0x7fd4180cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:05.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.633 7fd41561a700 1 -- 172.21.2.101:0/3950938920 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3044803768 0 0) 0x7fd40c001010 con 0x7fd4181b0a10 2021-02-10T11:44:05.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.633 7fd41661c700 1 -- 172.21.2.101:0/3950938920 >> v1:172.21.2.101:6789/0 conn(0x7fd4181b0a10 legacy=0x7fd4180cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:05.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.633 7fd41661c700 1 -- 172.21.2.101:0/3950938920 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4180d0530 con 0x7fd4180d1870 2021-02-10T11:44:05.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.633 7fd41661c700 1 --2- 172.21.2.101:0/3950938920 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4180d1870 0x7fd4181b03f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fd408006650 tx=0x7fd408009a00).ready entity=mon.2 client_cookie=c2755fbdc5eefd3b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:05.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.633 7fd41561a700 1 -- 172.21.2.101:0/3950938920 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd408009df0 con 0x7fd4180d1870 2021-02-10T11:44:05.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.633 7fd41561a700 1 -- 172.21.2.101:0/3950938920 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4080009b0 con 0x7fd4180d1870 2021-02-10T11:44:05.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.633 7fd41561a700 1 -- 172.21.2.101:0/3950938920 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd408009df0 con 0x7fd4180d1870 2021-02-10T11:44:05.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.633 7fd41ca0e700 1 -- 172.21.2.101:0/3950938920 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4180d1870 msgr2=0x7fd4181b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:05.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.633 7fd41ca0e700 1 --2- 172.21.2.101:0/3950938920 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4180d1870 0x7fd4181b03f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fd408006650 tx=0x7fd408009a00).stop 2021-02-10T11:44:05.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.634 7fd41ca0e700 1 -- 172.21.2.101:0/3950938920 shutdown_connections 2021-02-10T11:44:05.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.634 7fd41ca0e700 1 --2- 172.21.2.101:0/3950938920 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4180d1870 0x7fd4181b03f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:05.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.634 7fd41ca0e700 1 -- 172.21.2.101:0/3950938920 >> 172.21.2.101:0/3950938920 conn(0x7fd4180ce450 msgr2=0x7fd4180d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:05.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.634 7fd41ca0e700 1 -- 172.21.2.101:0/3950938920 shutdown_connections 2021-02-10T11:44:05.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.634 7fd41ca0e700 1 -- 172.21.2.101:0/3950938920 wait complete. 2021-02-10T11:44:05.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.649 7fd41ca0e700 1 Processor -- start 2021-02-10T11:44:05.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.649 7fd41ca0e700 1 -- start start 2021-02-10T11:44:05.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.649 7fd41ca0e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd41818d870 0x7fd4180b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:05.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.649 7fd41ca0e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4180b1fd0 0x7fd418099ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:05.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.649 7fd41ca0e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4180b23d0 0x7fd41809a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:05.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.649 7fd41ca0e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41818dc70 con 0x7fd4180b23d0 2021-02-10T11:44:05.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.649 7fd41ca0e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd418199e60 con 0x7fd41818d870 2021-02-10T11:44:05.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.649 7fd41ca0e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41809ab30 con 0x7fd4180b1fd0 2021-02-10T11:44:05.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.650 7fd416e1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4180b23d0 0x7fd41809a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:05.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.650 7fd416e1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4180b23d0 0x7fd41809a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59456/0 (socket says 172.21.2.101:59456) 2021-02-10T11:44:05.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.650 7fd416e1d700 1 -- 172.21.2.101:0/1290875824 learned_addr learned my addr 172.21.2.101:0/1290875824 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:05.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.650 7fd416e1d700 1 -- 172.21.2.101:0/1290875824 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4180b1fd0 msgr2=0x7fd418099ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:05.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.650 7fd416e1d700 1 --2- 172.21.2.101:0/1290875824 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4180b1fd0 0x7fd418099ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:05.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.650 7fd416e1d700 1 -- 172.21.2.101:0/1290875824 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd41818d870 msgr2=0x7fd4180b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:05.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.650 7fd416e1d700 1 --2- 172.21.2.101:0/1290875824 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd41818d870 0x7fd4180b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:05.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.650 7fd416e1d700 1 -- 172.21.2.101:0/1290875824 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41809ad50 con 0x7fd4180b23d0 2021-02-10T11:44:05.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.650 7fd416e1d700 1 --2- 172.21.2.101:0/1290875824 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4180b23d0 0x7fd41809a510 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 rx=0x7fd410009940 tx=0x7fd41000bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:05.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.650 7fd406ffd700 1 -- 172.21.2.101:0/1290875824 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd410000f40 con 0x7fd4180b23d0 2021-02-10T11:44:05.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.650 7fd406ffd700 1 -- 172.21.2.101:0/1290875824 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41000c2f0 con 0x7fd4180b23d0 2021-02-10T11:44:05.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.650 7fd41ca0e700 1 -- 172.21.2.101:0/1290875824 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41809ad50 con 0x7fd4180b23d0 2021-02-10T11:44:05.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.650 7fd406ffd700 1 -- 172.21.2.101:0/1290875824 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd41000c7d0 con 0x7fd4180b23d0 2021-02-10T11:44:05.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.650 7fd41ca0e700 1 -- 172.21.2.101:0/1290875824 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41818e3f0 con 0x7fd4180b23d0 2021-02-10T11:44:05.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.651 7fd406ffd700 1 -- 172.21.2.101:0/1290875824 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd410001250 con 0x7fd4180b23d0 2021-02-10T11:44:05.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.651 7fd406ffd700 1 --2- 172.21.2.101:0/1290875824 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3fc0186c0 0x7fd3fc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:05.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.651 7fd406ffd700 1 -- 172.21.2.101:0/1290875824 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fd4100324a0 con 0x7fd4180b23d0 2021-02-10T11:44:05.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.651 7fd41661c700 1 --2- 172.21.2.101:0/1290875824 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3fc0186c0 0x7fd3fc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:05.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.652 7fd41661c700 1 --2- 172.21.2.101:0/1290875824 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3fc0186c0 0x7fd3fc01abc0 crc :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:05.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.681 7fd41ca0e700 1 -- 172.21.2.101:0/1290875824 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4180999c0 con 0x7fd4180b23d0 2021-02-10T11:44:05.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.684 7fd406ffd700 1 -- 172.21.2.101:0/1290875824 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd410003000 con 0x7fd4180b23d0 2021-02-10T11:44:05.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.806 7fd41ca0e700 1 -- 172.21.2.101:0/1290875824 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd4180999c0 con 0x7fd4180b23d0 2021-02-10T11:44:05.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.807 7fd406ffd700 1 -- 172.21.2.101:0/1290875824 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v142) v1 ==== 84+0+64 (secure 0 0 0) 0x7fd410001230 con 0x7fd4180b23d0 2021-02-10T11:44:05.806 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:44:05.806 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:44:05.806 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:05.806 INFO:teuthology.orchestra.run.gibba001.stdout: "poolnum": 1, 2021-02-10T11:44:05.806 INFO:teuthology.orchestra.run.gibba001.stdout: "poolname": "rbd" 2021-02-10T11:44:05.806 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:05.806 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:44:05.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.809 7fd41ca0e700 1 -- 172.21.2.101:0/1290875824 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3fc0186c0 msgr2=0x7fd3fc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:05.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.809 7fd41ca0e700 1 --2- 172.21.2.101:0/1290875824 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3fc0186c0 0x7fd3fc01abc0 crc :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:05.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.809 7fd41ca0e700 1 -- 172.21.2.101:0/1290875824 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4180b23d0 msgr2=0x7fd41809a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:05.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.809 7fd41ca0e700 1 --2- 172.21.2.101:0/1290875824 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4180b23d0 0x7fd41809a510 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 rx=0x7fd410009940 tx=0x7fd41000bcf0).stop 2021-02-10T11:44:05.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.809 7fd41ca0e700 1 -- 172.21.2.101:0/1290875824 shutdown_connections 2021-02-10T11:44:05.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.809 7fd41ca0e700 1 --2- 172.21.2.101:0/1290875824 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd3fc0186c0 0x7fd3fc01abc0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:05.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.809 7fd41ca0e700 1 --2- 172.21.2.101:0/1290875824 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd4180b1fd0 0x7fd418099ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:05.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.809 7fd41ca0e700 1 --2- 172.21.2.101:0/1290875824 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd4180b23d0 0x7fd41809a510 unknown :-1 s=CLOSED pgs=2640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:05.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.809 7fd41ca0e700 1 --2- 172.21.2.101:0/1290875824 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd41818d870 0x7fd4180b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:05.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.809 7fd41ca0e700 1 -- 172.21.2.101:0/1290875824 >> 172.21.2.101:0/1290875824 conn(0x7fd4180f0550 msgr2=0x7fd418191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:05.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.809 7fd41ca0e700 1 -- 172.21.2.101:0/1290875824 shutdown_connections 2021-02-10T11:44:05.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:05.809 7fd41ca0e700 1 -- 172.21.2.101:0/1290875824 wait complete. 2021-02-10T11:44:05.816 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-02-10T11:44:05.816 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-02-10T11:44:06.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.234 7f84826f1700 1 Processor -- start 2021-02-10T11:44:06.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.234 7f84826f1700 1 -- start start 2021-02-10T11:44:06.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.234 7f84826f1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f847c0d1870 0x7f847c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:06.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.234 7f84826f1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f847c1b0ae0 0x7f847c108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:06.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.234 7f84826f1700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f847c109a60 con 0x7f847c0d1c70 2021-02-10T11:44:06.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.234 7f84826f1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f847c0e5ef0 con 0x7f847c1b0ae0 2021-02-10T11:44:06.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.234 7f84826f1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847c109750 con 0x7f847c0d1870 2021-02-10T11:44:06.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.235 7f8480d0e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f847c0d1c70 0x7f847c108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59956/0 (socket says 172.21.2.101:59956) 2021-02-10T11:44:06.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.235 7f8480d0e700 1 -- 172.21.2.101:0/1265382526 learned_addr learned my addr 172.21.2.101:0/1265382526 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:06.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.235 7f847affd700 1 -- 172.21.2.101:0/1265382526 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3334085916 0 0) 0x7f8474001500 con 0x7f847c0d1c70 2021-02-10T11:44:06.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.235 7f847affd700 1 -- 172.21.2.101:0/1265382526 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8464002fe0 con 0x7f847c0d1c70 2021-02-10T11:44:06.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.235 7f847affd700 1 -- 172.21.2.101:0/1265382526 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3386453337 0 0) 0x7f8474001460 con 0x7f847c0d1c70 2021-02-10T11:44:06.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.235 7f847affd700 1 -- 172.21.2.101:0/1265382526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f847c0d1870 msgr2=0x7f847c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:06.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.235 7f847affd700 1 --2- 172.21.2.101:0/1265382526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f847c0d1870 0x7f847c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:06.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.235 7f847affd700 1 -- 172.21.2.101:0/1265382526 >> v2:172.21.2.108:3300/0 conn(0x7f847c1b0ae0 msgr2=0x7f847c108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:06.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.235 7f847b7fe700 1 --2- 172.21.2.101:0/1265382526 >> v2:172.21.2.108:3300/0 conn(0x7f847c1b0ae0 0x7f847c108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:06.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.235 7f847affd700 1 --2- 172.21.2.101:0/1265382526 >> v2:172.21.2.108:3300/0 conn(0x7f847c1b0ae0 0x7f847c108940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:06.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.235 7f847affd700 1 -- 172.21.2.101:0/1265382526 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c088070 con 0x7f847c0d1c70 2021-02-10T11:44:06.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.235 7f847affd700 1 -- 172.21.2.101:0/1265382526 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f8474001980 con 0x7f847c0d1c70 2021-02-10T11:44:06.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.235 7f847affd700 1 -- 172.21.2.101:0/1265382526 >> v1:172.21.2.101:6789/0 conn(0x7f847c0d1c70 legacy=0x7f847c108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:06.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.235 7f847affd700 1 --2- 172.21.2.101:0/1265382526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8464003ae0 0x7f8464005f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:06.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.235 7f847affd700 1 --2- 172.21.2.101:0/1265382526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8464006830 0x7f8464008d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:06.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.235 7f847affd700 1 --2- 172.21.2.101:0/1265382526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84640095a0 0x7f846400bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:06.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.235 7f847affd700 1 -- 172.21.2.101:0/1265382526 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846400c370 con 0x7f84640095a0 2021-02-10T11:44:06.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.235 7f847affd700 1 -- 172.21.2.101:0/1265382526 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846400c590 con 0x7f8464003ae0 2021-02-10T11:44:06.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.235 7f847affd700 1 -- 172.21.2.101:0/1265382526 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846400c7b0 con 0x7f8464006830 2021-02-10T11:44:06.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.236 7f8480d0e700 1 --2- 172.21.2.101:0/1265382526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84640095a0 0x7f846400bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:06.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.236 7f847b7fe700 1 --2- 172.21.2.101:0/1265382526 >> v2:172.21.2.108:3300/0 conn(0x7f847c1b0ae0 0x7f847c108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:44:06.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.236 7f8480d0e700 1 -- 172.21.2.101:0/1265382526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8464006830 msgr2=0x7f8464008d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:06.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.236 7f8480d0e700 1 --2- 172.21.2.101:0/1265382526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8464006830 0x7f8464008d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:06.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.236 7f8480d0e700 1 -- 172.21.2.101:0/1265382526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8464003ae0 msgr2=0x7f8464005f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:06.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.236 7f8480d0e700 1 --2- 172.21.2.101:0/1265382526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8464003ae0 0x7f8464005f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:06.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.236 7f8480d0e700 1 -- 172.21.2.101:0/1265382526 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846400ca50 con 0x7f84640095a0 2021-02-10T11:44:06.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.236 7f8480d0e700 1 --2- 172.21.2.101:0/1265382526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84640095a0 0x7f846400bac0 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 rx=0x7f8474001ae0 tx=0x7f847400a310).ready entity=mon.0 client_cookie=26da013949a530a0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:06.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.236 7f847affd700 1 -- 172.21.2.101:0/1265382526 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f84740070c0 con 0x7f84640095a0 2021-02-10T11:44:06.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.236 7f847affd700 1 -- 172.21.2.101:0/1265382526 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8474007300 con 0x7f84640095a0 2021-02-10T11:44:06.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.236 7f847affd700 1 -- 172.21.2.101:0/1265382526 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8474004720 con 0x7f84640095a0 2021-02-10T11:44:06.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.236 7f84826f1700 1 -- 172.21.2.101:0/1265382526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84640095a0 msgr2=0x7f846400bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:06.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.236 7f84826f1700 1 --2- 172.21.2.101:0/1265382526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84640095a0 0x7f846400bac0 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 rx=0x7f8474001ae0 tx=0x7f847400a310).stop 2021-02-10T11:44:06.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.236 7f8480d0e700 1 -- 172.21.2.101:0/1265382526 reap_dead start 2021-02-10T11:44:06.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.237 7f84826f1700 1 -- 172.21.2.101:0/1265382526 shutdown_connections 2021-02-10T11:44:06.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.237 7f84826f1700 1 -- 172.21.2.101:0/1265382526 >> 172.21.2.101:0/1265382526 conn(0x7f847c0ce420 msgr2=0x7f847c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:06.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.237 7f84826f1700 1 -- 172.21.2.101:0/1265382526 shutdown_connections 2021-02-10T11:44:06.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.237 7f84826f1700 1 -- 172.21.2.101:0/1265382526 wait complete. 2021-02-10T11:44:06.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.251 7f84826f1700 1 Processor -- start 2021-02-10T11:44:06.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.251 7f84826f1700 1 -- start start 2021-02-10T11:44:06.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.252 7f84826f1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f847c085bc0 0x7f847c0863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:06.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.252 7f84826f1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f847c0954c0 0x7f847c096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:06.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.252 7f84826f1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f847c0958c0 0x7f847c096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:06.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.252 7f84826f1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c095d80 con 0x7f847c085bc0 2021-02-10T11:44:06.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.252 7f84826f1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c0bfa60 con 0x7f847c0958c0 2021-02-10T11:44:06.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.252 7f84826f1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847c085fc0 con 0x7f847c0954c0 2021-02-10T11:44:06.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.252 7f847bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f847c085bc0 0x7f847c0863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:06.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.252 7f847bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f847c085bc0 0x7f847c0863e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59476/0 (socket says 172.21.2.101:59476) 2021-02-10T11:44:06.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.252 7f847bfff700 1 -- 172.21.2.101:0/1555987364 learned_addr learned my addr 172.21.2.101:0/1555987364 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:06.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.252 7f847bfff700 1 -- 172.21.2.101:0/1555987364 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f847c0954c0 msgr2=0x7f847c096300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:06.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.252 7f847bfff700 1 --2- 172.21.2.101:0/1555987364 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f847c0954c0 0x7f847c096300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:06.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.252 7f847bfff700 1 -- 172.21.2.101:0/1555987364 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f847c0958c0 msgr2=0x7f847c096920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:06.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.252 7f847bfff700 1 --2- 172.21.2.101:0/1555987364 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f847c0958c0 0x7f847c096920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:06.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.252 7f847bfff700 1 -- 172.21.2.101:0/1555987364 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c086c10 con 0x7f847c085bc0 2021-02-10T11:44:06.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.253 7f847bfff700 1 --2- 172.21.2.101:0/1555987364 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f847c085bc0 0x7f847c0863e0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 rx=0x7f846c006f00 tx=0x7f846c00b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:06.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.253 7f8478ff9700 1 -- 172.21.2.101:0/1555987364 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f846c00bd30 con 0x7f847c085bc0 2021-02-10T11:44:06.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.253 7f8478ff9700 1 -- 172.21.2.101:0/1555987364 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846c000c90 con 0x7f847c085bc0 2021-02-10T11:44:06.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.253 7f8478ff9700 1 -- 172.21.2.101:0/1555987364 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f846c0022a0 con 0x7f847c085bc0 2021-02-10T11:44:06.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.253 7f84826f1700 1 -- 172.21.2.101:0/1555987364 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847c086c10 con 0x7f847c085bc0 2021-02-10T11:44:06.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.253 7f84826f1700 1 -- 172.21.2.101:0/1555987364 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847c097230 con 0x7f847c085bc0 2021-02-10T11:44:06.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.253 7f8478ff9700 1 -- 172.21.2.101:0/1555987364 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f846c005310 con 0x7f847c085bc0 2021-02-10T11:44:06.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.253 7f8478ff9700 1 --2- 172.21.2.101:0/1555987364 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f845c0186c0 0x7f845c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:06.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.254 7f8478ff9700 1 -- 172.21.2.101:0/1555987364 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f846c032220 con 0x7f847c085bc0 2021-02-10T11:44:06.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.254 7f847b7fe700 1 --2- 172.21.2.101:0/1555987364 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f845c0186c0 0x7f845c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:06.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.254 7f847b7fe700 1 --2- 172.21.2.101:0/1555987364 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f845c0186c0 0x7f845c01abc0 crc :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:06.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.284 7f84826f1700 1 -- 172.21.2.101:0/1555987364 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f847c0fe1c0 con 0x7f847c085bc0 2021-02-10T11:44:06.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.287 7f8478ff9700 1 -- 172.21.2.101:0/1555987364 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f846c000940 con 0x7f847c085bc0 2021-02-10T11:44:06.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.414 7f84826f1700 1 -- 172.21.2.101:0/1555987364 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"} v 0) v1 -- 0x7f847c0fe1c0 con 0x7f847c085bc0 2021-02-10T11:44:06.570 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:44:06.574 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:44:06.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.828 7f8478ff9700 1 -- 172.21.2.101:0/1555987364 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"}]=0 pool 'cephfs_metadata' created v143) v1 ==== 133+0+0 (secure 0 0 0) 0x7f846c005310 con 0x7f847c085bc0 2021-02-10T11:44:06.828 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' created 2021-02-10T11:44:06.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.831 7f84826f1700 1 -- 172.21.2.101:0/1555987364 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f845c0186c0 msgr2=0x7f845c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:06.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.831 7f84826f1700 1 --2- 172.21.2.101:0/1555987364 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f845c0186c0 0x7f845c01abc0 crc :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:06.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.831 7f84826f1700 1 -- 172.21.2.101:0/1555987364 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f847c085bc0 msgr2=0x7f847c0863e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:06.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.831 7f84826f1700 1 --2- 172.21.2.101:0/1555987364 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f847c085bc0 0x7f847c0863e0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 rx=0x7f846c006f00 tx=0x7f846c00b940).stop 2021-02-10T11:44:06.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.832 7f84826f1700 1 -- 172.21.2.101:0/1555987364 shutdown_connections 2021-02-10T11:44:06.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.832 7f84826f1700 1 --2- 172.21.2.101:0/1555987364 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f845c0186c0 0x7f845c01abc0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:06.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.832 7f84826f1700 1 --2- 172.21.2.101:0/1555987364 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f847c0954c0 0x7f847c096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:06.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.832 7f84826f1700 1 --2- 172.21.2.101:0/1555987364 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f847c0958c0 0x7f847c096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:06.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.832 7f84826f1700 1 --2- 172.21.2.101:0/1555987364 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f847c085bc0 0x7f847c0863e0 unknown :-1 s=CLOSED pgs=2643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:06.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.832 7f84826f1700 1 -- 172.21.2.101:0/1555987364 >> 172.21.2.101:0/1555987364 conn(0x7f847c0bef00 msgr2=0x7f847c09d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:06.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.832 7f84826f1700 1 -- 172.21.2.101:0/1555987364 shutdown_connections 2021-02-10T11:44:06.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:06.832 7f84826f1700 1 -- 172.21.2.101:0/1555987364 wait complete. 2021-02-10T11:44:06.839 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_data 8 2021-02-10T11:44:07.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.254 7fe022fee700 1 Processor -- start 2021-02-10T11:44:07.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.254 7fe022fee700 1 -- start start 2021-02-10T11:44:07.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.255 7fe022fee700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe01c0d1c70 0x7fe01c0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:07.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.255 7fe022fee700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe01c0cf580 0x7fe01c0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:07.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.255 7fe022fee700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe01c0fe1c0 con 0x7fe01c0d1870 2021-02-10T11:44:07.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.255 7fe022fee700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe01c1b0cb0 con 0x7fe01c0cf580 2021-02-10T11:44:07.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.255 7fe022fee700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c109a60 con 0x7fe01c0d1c70 2021-02-10T11:44:07.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.255 7fe020e0a700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe01c0d1870 0x7fe01c1b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:59976/0 (socket says 172.21.2.101:59976) 2021-02-10T11:44:07.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.255 7fe020e0a700 1 -- 172.21.2.101:0/4260993900 learned_addr learned my addr 172.21.2.101:0/4260993900 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:07.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.255 7fe02160b700 1 --2- 172.21.2.101:0/4260993900 >> v2:172.21.2.101:3300/0 conn(0x7fe01c0cf580 0x7fe01c0cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:07.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.256 7fe02160b700 1 -- 172.21.2.101:0/4260993900 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe01c0d1c70 msgr2=0x7fe01c0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:07.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.256 7fe02160b700 1 --2- 172.21.2.101:0/4260993900 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe01c0d1c70 0x7fe01c0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:07.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.256 7fe02160b700 1 -- 172.21.2.101:0/4260993900 >> v1:172.21.2.101:6789/0 conn(0x7fe01c0d1870 legacy=0x7fe01c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:07.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.256 7fe02160b700 1 -- 172.21.2.101:0/4260993900 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c088070 con 0x7fe01c0cf580 2021-02-10T11:44:07.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.256 7fe02160b700 1 --2- 172.21.2.101:0/4260993900 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe01c0cf580 0x7fe01c0cf980 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 rx=0x7fe014006ab0 tx=0x7fe014009b40).ready entity=mon.0 client_cookie=e708cc274906b764 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:07.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.256 7fe01b7fe700 1 -- 172.21.2.101:0/4260993900 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe0140022e0 con 0x7fe01c0cf580 2021-02-10T11:44:07.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.256 7fe01b7fe700 1 -- 172.21.2.101:0/4260993900 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe014002560 con 0x7fe01c0cf580 2021-02-10T11:44:07.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.256 7fe01b7fe700 1 -- 172.21.2.101:0/4260993900 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe0140029f0 con 0x7fe01c0cf580 2021-02-10T11:44:07.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.256 7fe022fee700 1 -- 172.21.2.101:0/4260993900 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe01c0cf580 msgr2=0x7fe01c0cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:07.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.256 7fe022fee700 1 --2- 172.21.2.101:0/4260993900 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe01c0cf580 0x7fe01c0cf980 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 rx=0x7fe014006ab0 tx=0x7fe014009b40).stop 2021-02-10T11:44:07.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.258 7fe022fee700 1 -- 172.21.2.101:0/4260993900 shutdown_connections 2021-02-10T11:44:07.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.258 7fe022fee700 1 --2- 172.21.2.101:0/4260993900 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe01c0d1c70 0x7fe01c0cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:07.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.258 7fe022fee700 1 --2- 172.21.2.101:0/4260993900 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe01c0cf580 0x7fe01c0cf980 unknown :-1 s=CLOSED pgs=2645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:07.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.258 7fe022fee700 1 -- 172.21.2.101:0/4260993900 >> 172.21.2.101:0/4260993900 conn(0x7fe01c0ce420 msgr2=0x7fe01c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:07.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.258 7fe022fee700 1 -- 172.21.2.101:0/4260993900 shutdown_connections 2021-02-10T11:44:07.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.258 7fe022fee700 1 -- 172.21.2.101:0/4260993900 wait complete. 2021-02-10T11:44:07.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.272 7fe022fee700 1 Processor -- start 2021-02-10T11:44:07.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.272 7fe022fee700 1 -- start start 2021-02-10T11:44:07.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.272 7fe022fee700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe01c0c3820 0x7fe01c084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:07.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.272 7fe022fee700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe01c0a0290 0x7fe01c0852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:07.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.272 7fe022fee700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe01c086270 0x7fe01c086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:07.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.272 7fe022fee700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c0c0700 con 0x7fe01c086270 2021-02-10T11:44:07.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.272 7fe022fee700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c0858f0 con 0x7fe01c0c3820 2021-02-10T11:44:07.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.272 7fe022fee700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c085b10 con 0x7fe01c0a0290 2021-02-10T11:44:07.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.272 7fe02160b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe01c086270 0x7fe01c086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:07.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.272 7fe02160b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe01c086270 0x7fe01c086670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59488/0 (socket says 172.21.2.101:59488) 2021-02-10T11:44:07.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.272 7fe02160b700 1 -- 172.21.2.101:0/2987866885 learned_addr learned my addr 172.21.2.101:0/2987866885 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:07.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.274 7fe02160b700 1 -- 172.21.2.101:0/2987866885 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe01c0a0290 msgr2=0x7fe01c0852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:07.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.274 7fe02160b700 1 --2- 172.21.2.101:0/2987866885 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe01c0a0290 0x7fe01c0852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:07.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.274 7fe02160b700 1 -- 172.21.2.101:0/2987866885 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe01c0c3820 msgr2=0x7fe01c084cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:07.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.274 7fe02160b700 1 --2- 172.21.2.101:0/2987866885 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe01c0c3820 0x7fe01c084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:07.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.274 7fe02160b700 1 -- 172.21.2.101:0/2987866885 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c086c90 con 0x7fe01c086270 2021-02-10T11:44:07.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.274 7fe02160b700 1 --2- 172.21.2.101:0/2987866885 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe01c086270 0x7fe01c086670 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 rx=0x7fe01400b5e0 tx=0x7fe01400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:07.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.274 7fe0197fa700 1 -- 172.21.2.101:0/2987866885 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe014009070 con 0x7fe01c086270 2021-02-10T11:44:07.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.274 7fe0197fa700 1 -- 172.21.2.101:0/2987866885 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe014002280 con 0x7fe01c086270 2021-02-10T11:44:07.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.274 7fe0197fa700 1 -- 172.21.2.101:0/2987866885 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe014002760 con 0x7fe01c086270 2021-02-10T11:44:07.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.274 7fe022fee700 1 -- 172.21.2.101:0/2987866885 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01c086c90 con 0x7fe01c086270 2021-02-10T11:44:07.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.274 7fe022fee700 1 -- 172.21.2.101:0/2987866885 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01c1a2f00 con 0x7fe01c086270 2021-02-10T11:44:07.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.274 7fe0197fa700 1 -- 172.21.2.101:0/2987866885 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe014002460 con 0x7fe01c086270 2021-02-10T11:44:07.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.274 7fe0197fa700 1 --2- 172.21.2.101:0/2987866885 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0000186c0 0x7fe00001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:07.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.274 7fe0197fa700 1 -- 172.21.2.101:0/2987866885 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5107+0+0 (secure 0 0 0) 0x7fe014032420 con 0x7fe01c086270 2021-02-10T11:44:07.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.274 7fe020e0a700 1 --2- 172.21.2.101:0/2987866885 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0000186c0 0x7fe00001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:07.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.275 7fe020e0a700 1 --2- 172.21.2.101:0/2987866885 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0000186c0 0x7fe00001abc0 crc :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:07.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.306 7fe022fee700 1 -- 172.21.2.101:0/2987866885 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe01c082a70 con 0x7fe01c086270 2021-02-10T11:44:07.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.308 7fe0197fa700 1 -- 172.21.2.101:0/2987866885 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe0140093e0 con 0x7fe01c086270 2021-02-10T11:44:07.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.438 7fe022fee700 1 -- 172.21.2.101:0/2987866885 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"} v 0) v1 -- 0x7fe01c082a70 con 0x7fe01c086270 2021-02-10T11:44:07.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.835 7fe0197fa700 1 -- 172.21.2.101:0/2987866885 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"}]=0 pool 'cephfs_data' created v145) v1 ==== 125+0+0 (secure 0 0 0) 0x7fe014002460 con 0x7fe01c086270 2021-02-10T11:44:07.851 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' created 2021-02-10T11:44:07.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.853 7fe022fee700 1 -- 172.21.2.101:0/2987866885 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0000186c0 msgr2=0x7fe00001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:07.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.853 7fe022fee700 1 --2- 172.21.2.101:0/2987866885 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0000186c0 0x7fe00001abc0 crc :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:07.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.853 7fe022fee700 1 -- 172.21.2.101:0/2987866885 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe01c086270 msgr2=0x7fe01c086670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:07.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.853 7fe022fee700 1 --2- 172.21.2.101:0/2987866885 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe01c086270 0x7fe01c086670 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 rx=0x7fe01400b5e0 tx=0x7fe01400b980).stop 2021-02-10T11:44:07.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.854 7fe022fee700 1 -- 172.21.2.101:0/2987866885 shutdown_connections 2021-02-10T11:44:07.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.854 7fe022fee700 1 --2- 172.21.2.101:0/2987866885 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0000186c0 0x7fe00001abc0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:07.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.854 7fe022fee700 1 --2- 172.21.2.101:0/2987866885 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe01c0a0290 0x7fe01c0852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:07.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.854 7fe022fee700 1 --2- 172.21.2.101:0/2987866885 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe01c086270 0x7fe01c086670 unknown :-1 s=CLOSED pgs=2646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:07.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.854 7fe022fee700 1 --2- 172.21.2.101:0/2987866885 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe01c0c3820 0x7fe01c084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:07.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.854 7fe022fee700 1 -- 172.21.2.101:0/2987866885 >> 172.21.2.101:0/2987866885 conn(0x7fe01c0872c0 msgr2=0x7fe01c095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:07.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.854 7fe022fee700 1 -- 172.21.2.101:0/2987866885 shutdown_connections 2021-02-10T11:44:07.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:07.854 7fe022fee700 1 -- 172.21.2.101:0/2987866885 wait complete. 2021-02-10T11:44:07.862 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-02-10T11:44:08.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.277 7f6d7fc20700 1 Processor -- start 2021-02-10T11:44:08.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.277 7f6d7fc20700 1 -- start start 2021-02-10T11:44:08.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.277 7f6d7fc20700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6d780d1c80 0x7f6d780cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:08.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.277 7f6d7fc20700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6d780cf590 0x7f6d780cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:08.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.277 7f6d7fc20700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d780fe1d0 con 0x7f6d780d1880 2021-02-10T11:44:08.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.277 7f6d7fc20700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d781b0cc0 con 0x7f6d780cf590 2021-02-10T11:44:08.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.277 7f6d7fc20700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d78109a70 con 0x7f6d780d1c80 2021-02-10T11:44:08.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.278 7f6d7e23d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6d780cf590 0x7f6d780cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:08.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.278 7f6d7e23d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6d780cf590 0x7f6d780cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59498/0 (socket says 172.21.2.101:59498) 2021-02-10T11:44:08.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.278 7f6d7e23d700 1 -- 172.21.2.101:0/376833395 learned_addr learned my addr 172.21.2.101:0/376833395 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:08.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.278 7f6d7d23b700 1 --2- 172.21.2.101:0/376833395 >> v2:172.21.2.108:3300/0 conn(0x7f6d780d1c80 0x7f6d780cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:08.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.278 7f6d7e23d700 1 -- 172.21.2.101:0/376833395 >> v2:172.21.2.108:3300/0 conn(0x7f6d780d1c80 msgr2=0x7f6d780cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:08.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.278 7f6d7e23d700 1 --2- 172.21.2.101:0/376833395 >> v2:172.21.2.108:3300/0 conn(0x7f6d780d1c80 0x7f6d780cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:08.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.278 7f6d7e23d700 1 -- 172.21.2.101:0/376833395 >> v1:172.21.2.108:6789/0 conn(0x7f6d780d1880 legacy=0x7f6d781b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:08.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.278 7f6d7e23d700 1 -- 172.21.2.101:0/376833395 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d78088070 con 0x7f6d780cf590 2021-02-10T11:44:08.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.278 7f6d7d23b700 1 --2- 172.21.2.101:0/376833395 >> v2:172.21.2.108:3300/0 conn(0x7f6d780d1c80 0x7f6d780cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:44:08.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.278 7f6d7e23d700 1 --2- 172.21.2.101:0/376833395 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6d780cf590 0x7f6d780cf990 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 rx=0x7f6d70006f00 tx=0x7f6d70009f10).ready entity=mon.0 client_cookie=3f0717ee594cba64 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:08.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.278 7f6d7ca3a700 1 -- 172.21.2.101:0/376833395 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6d7000b3b0 con 0x7f6d780cf590 2021-02-10T11:44:08.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.279 7f6d7ca3a700 1 -- 172.21.2.101:0/376833395 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7000b6b0 con 0x7f6d780cf590 2021-02-10T11:44:08.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.279 7f6d7ca3a700 1 -- 172.21.2.101:0/376833395 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6d7000bb90 con 0x7f6d780cf590 2021-02-10T11:44:08.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.279 7f6d7fc20700 1 -- 172.21.2.101:0/376833395 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6d780cf590 msgr2=0x7f6d780cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:08.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.279 7f6d7fc20700 1 --2- 172.21.2.101:0/376833395 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6d780cf590 0x7f6d780cf990 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 rx=0x7f6d70006f00 tx=0x7f6d70009f10).stop 2021-02-10T11:44:08.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.279 7f6d7fc20700 1 -- 172.21.2.101:0/376833395 shutdown_connections 2021-02-10T11:44:08.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.279 7f6d7fc20700 1 --2- 172.21.2.101:0/376833395 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6d780cf590 0x7f6d780cf990 unknown :-1 s=CLOSED pgs=2647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:08.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.279 7f6d7fc20700 1 --2- 172.21.2.101:0/376833395 >> v2:172.21.2.108:3300/0 conn(0x7f6d780d1c80 0x7f6d780cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:08.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.279 7f6d7fc20700 1 -- 172.21.2.101:0/376833395 >> 172.21.2.101:0/376833395 conn(0x7f6d780ce430 msgr2=0x7f6d780d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:08.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.279 7f6d7fc20700 1 -- 172.21.2.101:0/376833395 shutdown_connections 2021-02-10T11:44:08.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.279 7f6d7fc20700 1 -- 172.21.2.101:0/376833395 wait complete. 2021-02-10T11:44:08.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.294 7f6d7fc20700 1 Processor -- start 2021-02-10T11:44:08.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.294 7f6d7fc20700 1 -- start start 2021-02-10T11:44:08.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.294 7f6d7fc20700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6d780919b0 0x7f6d7808cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:08.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.294 7f6d7fc20700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6d78083980 0x7f6d78083d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:08.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.294 7f6d7fc20700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6d780843a0 0x7f6d781aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:08.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.294 7f6d7fc20700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d780847a0 con 0x7f6d780919b0 2021-02-10T11:44:08.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.294 7f6d7fc20700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d781aa9b0 con 0x7f6d780843a0 2021-02-10T11:44:08.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.294 7f6d7fc20700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d78091db0 con 0x7f6d78083980 2021-02-10T11:44:08.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.294 7f6d7da3c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6d780919b0 0x7f6d7808cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:08.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.295 7f6d7da3c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6d780919b0 0x7f6d7808cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59502/0 (socket says 172.21.2.101:59502) 2021-02-10T11:44:08.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.295 7f6d7da3c700 1 -- 172.21.2.101:0/3288151121 learned_addr learned my addr 172.21.2.101:0/3288151121 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:08.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.295 7f6d7e23d700 1 --2- 172.21.2.101:0/3288151121 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6d780843a0 0x7f6d781aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:08.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.295 7f6d7d23b700 1 --2- 172.21.2.101:0/3288151121 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6d78083980 0x7f6d78083d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:08.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.295 7f6d7da3c700 1 -- 172.21.2.101:0/3288151121 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6d78083980 msgr2=0x7f6d78083d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:08.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.295 7f6d7da3c700 1 --2- 172.21.2.101:0/3288151121 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6d78083980 0x7f6d78083d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:08.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.295 7f6d7da3c700 1 -- 172.21.2.101:0/3288151121 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6d780843a0 msgr2=0x7f6d781aaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:08.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.295 7f6d7da3c700 1 --2- 172.21.2.101:0/3288151121 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6d780843a0 0x7f6d781aaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:08.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.295 7f6d7da3c700 1 -- 172.21.2.101:0/3288151121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7808d160 con 0x7f6d780919b0 2021-02-10T11:44:08.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.295 7f6d7da3c700 1 --2- 172.21.2.101:0/3288151121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6d780919b0 0x7f6d7808cb40 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 rx=0x7f6d7400b530 tx=0x7f6d7400b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:08.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.295 7f6d6e7fc700 1 -- 172.21.2.101:0/3288151121 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6d7400bcc0 con 0x7f6d780919b0 2021-02-10T11:44:08.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.295 7f6d6e7fc700 1 -- 172.21.2.101:0/3288151121 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d74006020 con 0x7f6d780919b0 2021-02-10T11:44:08.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.295 7f6d6e7fc700 1 -- 172.21.2.101:0/3288151121 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6d740054a0 con 0x7f6d780919b0 2021-02-10T11:44:08.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.295 7f6d7fc20700 1 -- 172.21.2.101:0/3288151121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d7808d160 con 0x7f6d780919b0 2021-02-10T11:44:08.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.296 7f6d7fc20700 1 -- 172.21.2.101:0/3288151121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7808d160 con 0x7f6d780919b0 2021-02-10T11:44:08.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.296 7f6d6e7fc700 1 -- 172.21.2.101:0/3288151121 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6d74006200 con 0x7f6d780919b0 2021-02-10T11:44:08.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.296 7f6d6e7fc700 1 --2- 172.21.2.101:0/3288151121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6d600186c0 0x7f6d6001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:08.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.296 7f6d6e7fc700 1 -- 172.21.2.101:0/3288151121 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5457+0+0 (secure 0 0 0) 0x7f6d74032530 con 0x7f6d780919b0 2021-02-10T11:44:08.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.296 7f6d7d23b700 1 --2- 172.21.2.101:0/3288151121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6d600186c0 0x7f6d6001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:08.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.297 7f6d7d23b700 1 --2- 172.21.2.101:0/3288151121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6d600186c0 0x7f6d6001abc0 crc :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:08.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.328 7f6d7fc20700 1 -- 172.21.2.101:0/3288151121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d780fe1d0 con 0x7f6d780919b0 2021-02-10T11:44:08.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.331 7f6d6e7fc700 1 -- 172.21.2.101:0/3288151121 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6d74030d50 con 0x7f6d780919b0 2021-02-10T11:44:08.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.461 7f6d7fc20700 1 -- 172.21.2.101:0/3288151121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"} v 0) v1 -- 0x7f6d780fe1d0 con 0x7f6d780919b0 2021-02-10T11:44:08.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.465 7f6d6e7fc700 1 -- 172.21.2.101:0/3288151121 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"}]=0 new fs with metadata pool 30 and data pool 31 v120) v1 ==== 189+0+0 (secure 0 0 0) 0x7f6d7401f730 con 0x7f6d780919b0 2021-02-10T11:44:08.468 INFO:teuthology.orchestra.run.gibba001.stderr:new fs with metadata pool 30 and data pool 31 2021-02-10T11:44:08.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.470 7f6d7fc20700 1 -- 172.21.2.101:0/3288151121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6d600186c0 msgr2=0x7f6d6001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:08.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.470 7f6d7fc20700 1 --2- 172.21.2.101:0/3288151121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6d600186c0 0x7f6d6001abc0 crc :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:08.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.470 7f6d7fc20700 1 -- 172.21.2.101:0/3288151121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6d780919b0 msgr2=0x7f6d7808cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:08.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.470 7f6d7fc20700 1 --2- 172.21.2.101:0/3288151121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6d780919b0 0x7f6d7808cb40 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 rx=0x7f6d7400b530 tx=0x7f6d7400b8d0).stop 2021-02-10T11:44:08.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.470 7f6d7fc20700 1 -- 172.21.2.101:0/3288151121 shutdown_connections 2021-02-10T11:44:08.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.470 7f6d7fc20700 1 --2- 172.21.2.101:0/3288151121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6d600186c0 0x7f6d6001abc0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:08.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.470 7f6d7fc20700 1 --2- 172.21.2.101:0/3288151121 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6d78083980 0x7f6d78083d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:08.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.470 7f6d7fc20700 1 --2- 172.21.2.101:0/3288151121 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6d780843a0 0x7f6d781aaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:08.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.470 7f6d7fc20700 1 --2- 172.21.2.101:0/3288151121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6d780919b0 0x7f6d7808cb40 unknown :-1 s=CLOSED pgs=2648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:08.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.470 7f6d7fc20700 1 -- 172.21.2.101:0/3288151121 >> 172.21.2.101:0/3288151121 conn(0x7f6d781b9cf0 msgr2=0x7f6d781a8230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:08.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.470 7f6d7fc20700 1 -- 172.21.2.101:0/3288151121 shutdown_connections 2021-02-10T11:44:08.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.471 7f6d7fc20700 1 -- 172.21.2.101:0/3288151121 wait complete. 2021-02-10T11:44:08.478 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:44:08.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.889 7f24e9483700 1 Processor -- start 2021-02-10T11:44:08.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.889 7f24e9483700 1 -- start start 2021-02-10T11:44:08.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.890 7f24e9483700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f24e40d1870 0x7f24e41b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:08.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.890 7f24e9483700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24e41b0a10 0x7f24e40cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:08.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.890 7f24e9483700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24e40f0b10 con 0x7f24e40d1c70 2021-02-10T11:44:08.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.890 7f24e9483700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e40e5ed0 con 0x7f24e40d1870 2021-02-10T11:44:08.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.890 7f24e9483700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e40f0650 con 0x7f24e41b0a10 2021-02-10T11:44:08.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.890 7f24e387e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f24e40d1c70 0x7f24e40cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60004/0 (socket says 172.21.2.101:60004) 2021-02-10T11:44:08.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.890 7f24e387e700 1 -- 172.21.2.101:0/2452006740 learned_addr learned my addr 172.21.2.101:0/2452006740 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:08.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.890 7f24e207b700 1 -- 172.21.2.101:0/2452006740 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1754002739 0 0) 0x7f24dc000c10 con 0x7f24e40d1c70 2021-02-10T11:44:08.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.890 7f24e207b700 1 -- 172.21.2.101:0/2452006740 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24d0002fe0 con 0x7f24e40d1c70 2021-02-10T11:44:08.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.890 7f24e307d700 1 --2- 172.21.2.101:0/2452006740 >> v2:172.21.2.108:3300/0 conn(0x7f24e40d1870 0x7f24e41b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:08.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.890 7f24e287c700 1 --2- 172.21.2.101:0/2452006740 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24e41b0a10 0x7f24e40cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:08.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e207b700 1 -- 172.21.2.101:0/2452006740 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 125721743 0 0) 0x7f24dc0010e0 con 0x7f24e40d1c70 2021-02-10T11:44:08.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e207b700 1 -- 172.21.2.101:0/2452006740 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24e41b0a10 msgr2=0x7f24e40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:08.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e207b700 1 --2- 172.21.2.101:0/2452006740 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24e41b0a10 0x7f24e40cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e207b700 1 -- 172.21.2.101:0/2452006740 >> v2:172.21.2.108:3300/0 conn(0x7f24e40d1870 msgr2=0x7f24e41b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e207b700 1 --2- 172.21.2.101:0/2452006740 >> v2:172.21.2.108:3300/0 conn(0x7f24e40d1870 0x7f24e41b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e207b700 1 -- 172.21.2.101:0/2452006740 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e40f0870 con 0x7f24e40d1c70 2021-02-10T11:44:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e207b700 1 -- 172.21.2.101:0/2452006740 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f24dc001660 con 0x7f24e40d1c70 2021-02-10T11:44:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e207b700 1 -- 172.21.2.101:0/2452006740 >> v1:172.21.2.101:6789/0 conn(0x7f24e40d1c70 legacy=0x7f24e40cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e207b700 1 --2- 172.21.2.101:0/2452006740 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24d0003af0 0x7f24d0005f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e207b700 1 --2- 172.21.2.101:0/2452006740 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24d004a790 0x7f24d004cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e207b700 1 --2- 172.21.2.101:0/2452006740 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24d004d510 0x7f24d004fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e207b700 1 -- 172.21.2.101:0/2452006740 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d00502e0 con 0x7f24d004d510 2021-02-10T11:44:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e207b700 1 -- 172.21.2.101:0/2452006740 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d0050500 con 0x7f24d004a790 2021-02-10T11:44:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e207b700 1 -- 172.21.2.101:0/2452006740 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d0050720 con 0x7f24d0003af0 2021-02-10T11:44:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e307d700 1 --2- 172.21.2.101:0/2452006740 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24d0003af0 0x7f24d0005f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:08.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e287c700 1 --2- 172.21.2.101:0/2452006740 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24d004a790 0x7f24d004cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e387e700 1 --2- 172.21.2.101:0/2452006740 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24d004d510 0x7f24d004fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e307d700 1 -- 172.21.2.101:0/2452006740 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24d004a790 msgr2=0x7f24d004cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e307d700 1 --2- 172.21.2.101:0/2452006740 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24d004a790 0x7f24d004cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e307d700 1 -- 172.21.2.101:0/2452006740 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24d004d510 msgr2=0x7f24d004fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e307d700 1 --2- 172.21.2.101:0/2452006740 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24d004d510 0x7f24d004fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e307d700 1 -- 172.21.2.101:0/2452006740 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d00509c0 con 0x7f24d0003af0 2021-02-10T11:44:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.891 7f24e287c700 1 --2- 172.21.2.101:0/2452006740 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24d004a790 0x7f24d004cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:44:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.892 7f24e307d700 1 --2- 172.21.2.101:0/2452006740 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24d0003af0 0x7f24d0005f00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f24cc006fc0 tx=0x7f24cc00c0c0).ready entity=mon.2 client_cookie=5ef5c4fb36d90b34 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.892 7f24e207b700 1 -- 172.21.2.101:0/2452006740 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f24cc00c4b0 con 0x7f24d0003af0 2021-02-10T11:44:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.892 7f24e207b700 1 -- 172.21.2.101:0/2452006740 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24cc00c7b0 con 0x7f24d0003af0 2021-02-10T11:44:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.892 7f24e207b700 1 -- 172.21.2.101:0/2452006740 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f24cc00cc40 con 0x7f24d0003af0 2021-02-10T11:44:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.892 7f24e9483700 1 -- 172.21.2.101:0/2452006740 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24d0003af0 msgr2=0x7f24d0005f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:08.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.892 7f24e9483700 1 --2- 172.21.2.101:0/2452006740 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24d0003af0 0x7f24d0005f00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f24cc006fc0 tx=0x7f24cc00c0c0).stop 2021-02-10T11:44:08.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.892 7f24e387e700 1 -- 172.21.2.101:0/2452006740 reap_dead start 2021-02-10T11:44:08.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.892 7f24e9483700 1 -- 172.21.2.101:0/2452006740 shutdown_connections 2021-02-10T11:44:08.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.892 7f24e9483700 1 -- 172.21.2.101:0/2452006740 >> 172.21.2.101:0/2452006740 conn(0x7f24e40ce450 msgr2=0x7f24e40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:08.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.893 7f24e9483700 1 -- 172.21.2.101:0/2452006740 shutdown_connections 2021-02-10T11:44:08.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.893 7f24e9483700 1 -- 172.21.2.101:0/2452006740 wait complete. 2021-02-10T11:44:08.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.908 7f24e9483700 1 Processor -- start 2021-02-10T11:44:08.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.908 7f24e9483700 1 -- start start 2021-02-10T11:44:08.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.908 7f24e9483700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24e4195650 0x7f24e40b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:08.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.908 7f24e9483700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24e40b5b80 0x7f24e41a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:08.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.908 7f24e9483700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24e40acfa0 0x7f24e41a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:08.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.908 7f24e9483700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e40b6010 con 0x7f24e40b5b80 2021-02-10T11:44:08.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.908 7f24e9483700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e41a3a60 con 0x7f24e40acfa0 2021-02-10T11:44:08.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.908 7f24e9483700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e41a3c40 con 0x7f24e4195650 2021-02-10T11:44:08.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.908 7f24e287c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24e40b5b80 0x7f24e41a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.908 7f24e287c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24e40b5b80 0x7f24e41a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59522/0 (socket says 172.21.2.101:59522) 2021-02-10T11:44:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.908 7f24e287c700 1 -- 172.21.2.101:0/2135265596 learned_addr learned my addr 172.21.2.101:0/2135265596 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.908 7f24e287c700 1 -- 172.21.2.101:0/2135265596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24e4195650 msgr2=0x7f24e40b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.908 7f24e287c700 1 --2- 172.21.2.101:0/2135265596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24e4195650 0x7f24e40b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.908 7f24e287c700 1 -- 172.21.2.101:0/2135265596 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24e40acfa0 msgr2=0x7f24e41a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.908 7f24e287c700 1 --2- 172.21.2.101:0/2135265596 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24e40acfa0 0x7f24e41a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.908 7f24e287c700 1 -- 172.21.2.101:0/2135265596 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e40bcae0 con 0x7f24e40b5b80 2021-02-10T11:44:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.909 7f24e287c700 1 --2- 172.21.2.101:0/2135265596 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24e40b5b80 0x7f24e41a2e20 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 rx=0x7f24d400b580 tx=0x7f24d400b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.909 7f24dbfff700 1 -- 172.21.2.101:0/2135265596 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f24d400bd10 con 0x7f24e40b5b80 2021-02-10T11:44:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.909 7f24dbfff700 1 -- 172.21.2.101:0/2135265596 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d4000c90 con 0x7f24e40b5b80 2021-02-10T11:44:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.909 7f24dbfff700 1 -- 172.21.2.101:0/2135265596 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f24d4002220 con 0x7f24e40b5b80 2021-02-10T11:44:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.909 7f24e9483700 1 -- 172.21.2.101:0/2135265596 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e40bcae0 con 0x7f24e40b5b80 2021-02-10T11:44:08.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.909 7f24e9483700 1 -- 172.21.2.101:0/2135265596 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e40bcae0 con 0x7f24e40b5b80 2021-02-10T11:44:08.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.910 7f24dbfff700 1 -- 172.21.2.101:0/2135265596 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f24d4006b00 con 0x7f24e40b5b80 2021-02-10T11:44:08.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.910 7f24dbfff700 1 --2- 172.21.2.101:0/2135265596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24c80186c0 0x7f24c801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:08.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.910 7f24dbfff700 1 -- 172.21.2.101:0/2135265596 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f24d4032520 con 0x7f24e40b5b80 2021-02-10T11:44:08.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.910 7f24e307d700 1 --2- 172.21.2.101:0/2135265596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24c80186c0 0x7f24c801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:08.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.910 7f24e307d700 1 --2- 172.21.2.101:0/2135265596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24c80186c0 0x7f24c801abc0 crc :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:08.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.940 7f24e9483700 1 -- 172.21.2.101:0/2135265596 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e41a3c40 con 0x7f24e40b5b80 2021-02-10T11:44:08.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:08.942 7f24dbfff700 1 -- 172.21.2.101:0/2135265596 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f24d4030d10 con 0x7f24e40b5b80 2021-02-10T11:44:09.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.067 7f24e9483700 1 -- 172.21.2.101:0/2135265596 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24e41a3c40 con 0x7f24e40b5b80 2021-02-10T11:44:09.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.067 7f24dbfff700 1 -- 172.21.2.101:0/2135265596 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+14020 (secure 0 0 0) 0x7f24d4006b00 con 0x7f24e40b5b80 2021-02-10T11:44:09.067 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:44:09.067 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":146,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:44:08.462868","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:44:06.431537","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":31,"pool_name":"cephfs_data","create_time":"2021-02-10 11:44:07.456920","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:44:09.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.069 7f24e9483700 1 -- 172.21.2.101:0/2135265596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24c80186c0 msgr2=0x7f24c801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:09.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.069 7f24e9483700 1 --2- 172.21.2.101:0/2135265596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24c80186c0 0x7f24c801abc0 crc :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:09.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.069 7f24e9483700 1 -- 172.21.2.101:0/2135265596 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24e40b5b80 msgr2=0x7f24e41a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:09.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.069 7f24e9483700 1 --2- 172.21.2.101:0/2135265596 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24e40b5b80 0x7f24e41a2e20 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 rx=0x7f24d400b580 tx=0x7f24d400b920).stop 2021-02-10T11:44:09.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.069 7f24e9483700 1 -- 172.21.2.101:0/2135265596 shutdown_connections 2021-02-10T11:44:09.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.069 7f24e9483700 1 --2- 172.21.2.101:0/2135265596 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24e40acfa0 0x7f24e41a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:09.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.069 7f24e9483700 1 --2- 172.21.2.101:0/2135265596 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24e40b5b80 0x7f24e41a2e20 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:09.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.069 7f24e9483700 1 --2- 172.21.2.101:0/2135265596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24c80186c0 0x7f24c801abc0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:09.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.069 7f24e9483700 1 --2- 172.21.2.101:0/2135265596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24e4195650 0x7f24e40b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:09.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.069 7f24e9483700 1 -- 172.21.2.101:0/2135265596 >> 172.21.2.101:0/2135265596 conn(0x7f24e40b3040 msgr2=0x7f24e40c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:09.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.069 7f24e9483700 1 -- 172.21.2.101:0/2135265596 shutdown_connections 2021-02-10T11:44:09.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.069 7f24e9483700 1 -- 172.21.2.101:0/2135265596 wait complete. 2021-02-10T11:44:09.077 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:44:09.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.489 7f4eb9bb1700 1 Processor -- start 2021-02-10T11:44:09.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.489 7f4eb9bb1700 1 -- start start 2021-02-10T11:44:09.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.489 7f4eb9bb1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4eb40d0340 0x7f4eb40ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:09.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.489 7f4eb9bb1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4eb40d1c70 0x7f4eb40efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:09.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.489 7f4eb9bb1700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb40f0b10 con 0x7f4eb40d1870 2021-02-10T11:44:09.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.489 7f4eb9bb1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eb40e5ed0 con 0x7f4eb40d0340 2021-02-10T11:44:09.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.489 7f4eb9bb1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb41b0be0 con 0x7f4eb40d1c70 2021-02-10T11:44:09.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb37fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f4eb40d1870 0x7f4eb41b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60020/0 (socket says 172.21.2.101:60020) 2021-02-10T11:44:09.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb37fe700 1 -- 172.21.2.101:0/2492241133 learned_addr learned my addr 172.21.2.101:0/2492241133 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:09.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb2ffd700 1 --2- 172.21.2.101:0/2492241133 >> v2:172.21.2.108:3300/0 conn(0x7f4eb40d0340 0x7f4eb40ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:09.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb3fff700 1 --2- 172.21.2.101:0/2492241133 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4eb40d1c70 0x7f4eb40efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:09.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb27fc700 1 -- 172.21.2.101:0/2492241133 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74542775 0 0) 0x7f4ea4001060 con 0x7f4eb40d1870 2021-02-10T11:44:09.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb27fc700 1 -- 172.21.2.101:0/2492241133 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ea0002fe0 con 0x7f4eb40d1870 2021-02-10T11:44:09.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb27fc700 1 -- 172.21.2.101:0/2492241133 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 730579912 0 0) 0x7f4ea4001250 con 0x7f4eb40d1870 2021-02-10T11:44:09.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb27fc700 1 -- 172.21.2.101:0/2492241133 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4eb40d1c70 msgr2=0x7f4eb40efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:09.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb27fc700 1 --2- 172.21.2.101:0/2492241133 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4eb40d1c70 0x7f4eb40efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:09.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb27fc700 1 -- 172.21.2.101:0/2492241133 >> v2:172.21.2.108:3300/0 conn(0x7f4eb40d0340 msgr2=0x7f4eb40ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:09.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb27fc700 1 --2- 172.21.2.101:0/2492241133 >> v2:172.21.2.108:3300/0 conn(0x7f4eb40d0340 0x7f4eb40ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:09.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb27fc700 1 -- 172.21.2.101:0/2492241133 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb40f20a0 con 0x7f4eb40d1870 2021-02-10T11:44:09.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb27fc700 1 -- 172.21.2.101:0/2492241133 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f4ea4001980 con 0x7f4eb40d1870 2021-02-10T11:44:09.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb27fc700 1 -- 172.21.2.101:0/2492241133 >> v1:172.21.2.101:6789/0 conn(0x7f4eb40d1870 legacy=0x7f4eb41b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:09.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb27fc700 1 --2- 172.21.2.101:0/2492241133 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ea0003a60 0x7f4ea0005ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:09.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb27fc700 1 --2- 172.21.2.101:0/2492241133 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea004a780 0x7f4ea004cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:09.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb27fc700 1 --2- 172.21.2.101:0/2492241133 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ea004d510 0x7f4ea004fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:09.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb27fc700 1 -- 172.21.2.101:0/2492241133 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea00502e0 con 0x7f4ea004d510 2021-02-10T11:44:09.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb27fc700 1 -- 172.21.2.101:0/2492241133 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea0050500 con 0x7f4ea0003a60 2021-02-10T11:44:09.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb27fc700 1 -- 172.21.2.101:0/2492241133 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea0050720 con 0x7f4ea004a780 2021-02-10T11:44:09.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb2ffd700 1 --2- 172.21.2.101:0/2492241133 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea004a780 0x7f4ea004cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:09.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb3fff700 1 --2- 172.21.2.101:0/2492241133 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ea004d510 0x7f4ea004fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:09.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.490 7f4eb37fe700 1 --2- 172.21.2.101:0/2492241133 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ea0003a60 0x7f4ea0005ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:09.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.491 7f4eb2ffd700 1 -- 172.21.2.101:0/2492241133 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ea0003a60 msgr2=0x7f4ea0005ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:09.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.491 7f4eb2ffd700 1 --2- 172.21.2.101:0/2492241133 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ea0003a60 0x7f4ea0005ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:09.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.491 7f4eb2ffd700 1 -- 172.21.2.101:0/2492241133 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ea004d510 msgr2=0x7f4ea004fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:09.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.491 7f4eb2ffd700 1 --2- 172.21.2.101:0/2492241133 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ea004d510 0x7f4ea004fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:09.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.491 7f4eb2ffd700 1 -- 172.21.2.101:0/2492241133 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea00509c0 con 0x7f4ea004a780 2021-02-10T11:44:09.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.491 7f4eb37fe700 1 --2- 172.21.2.101:0/2492241133 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ea0003a60 0x7f4ea0005ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:44:09.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.491 7f4eb2ffd700 1 --2- 172.21.2.101:0/2492241133 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea004a780 0x7f4ea004cc70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f4e9c00c8f0 tx=0x7f4e9c00cc90).ready entity=mon.2 client_cookie=f7f71c62a0f8ae1b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:09.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.491 7f4eb27fc700 1 -- 172.21.2.101:0/2492241133 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e9c00cdd0 con 0x7f4ea004a780 2021-02-10T11:44:09.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.491 7f4eb27fc700 1 -- 172.21.2.101:0/2492241133 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c009310 con 0x7f4ea004a780 2021-02-10T11:44:09.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.491 7f4eb27fc700 1 -- 172.21.2.101:0/2492241133 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e9c009830 con 0x7f4ea004a780 2021-02-10T11:44:09.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.491 7f4eb9bb1700 1 -- 172.21.2.101:0/2492241133 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea004a780 msgr2=0x7f4ea004cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:09.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.491 7f4eb9bb1700 1 --2- 172.21.2.101:0/2492241133 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea004a780 0x7f4ea004cc70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f4e9c00c8f0 tx=0x7f4e9c00cc90).stop 2021-02-10T11:44:09.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.491 7f4eb3fff700 1 -- 172.21.2.101:0/2492241133 reap_dead start 2021-02-10T11:44:09.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.492 7f4eb9bb1700 1 -- 172.21.2.101:0/2492241133 shutdown_connections 2021-02-10T11:44:09.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.492 7f4eb9bb1700 1 -- 172.21.2.101:0/2492241133 >> 172.21.2.101:0/2492241133 conn(0x7f4eb40ce450 msgr2=0x7f4eb40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:09.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.492 7f4eb9bb1700 1 -- 172.21.2.101:0/2492241133 shutdown_connections 2021-02-10T11:44:09.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.492 7f4eb9bb1700 1 -- 172.21.2.101:0/2492241133 wait complete. 2021-02-10T11:44:09.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.506 7f4eb9bb1700 1 Processor -- start 2021-02-10T11:44:09.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.507 7f4eb9bb1700 1 -- start start 2021-02-10T11:44:09.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.507 7f4eb9bb1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4eb4195650 0x7f4eb40b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:09.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.507 7f4eb9bb1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4eb40b5b80 0x7f4eb41a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:09.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.507 7f4eb9bb1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4eb40acfa0 0x7f4eb41a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:09.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.507 7f4eb9bb1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb40b6010 con 0x7f4eb40b5b80 2021-02-10T11:44:09.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.507 7f4eb9bb1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb41a3a60 con 0x7f4eb40acfa0 2021-02-10T11:44:09.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.507 7f4eb9bb1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb41a3c40 con 0x7f4eb4195650 2021-02-10T11:44:09.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.507 7f4eb2ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4eb40b5b80 0x7f4eb41a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:09.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.507 7f4eb2ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4eb40b5b80 0x7f4eb41a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59540/0 (socket says 172.21.2.101:59540) 2021-02-10T11:44:09.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.507 7f4eb2ffd700 1 -- 172.21.2.101:0/2007131171 learned_addr learned my addr 172.21.2.101:0/2007131171 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:09.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.507 7f4eb3fff700 1 --2- 172.21.2.101:0/2007131171 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4eb40acfa0 0x7f4eb41a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:09.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.508 7f4eb2ffd700 1 -- 172.21.2.101:0/2007131171 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4eb4195650 msgr2=0x7f4eb40b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:09.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.508 7f4eb2ffd700 1 --2- 172.21.2.101:0/2007131171 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4eb4195650 0x7f4eb40b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:09.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.508 7f4eb2ffd700 1 -- 172.21.2.101:0/2007131171 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4eb40acfa0 msgr2=0x7f4eb41a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:09.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.508 7f4eb2ffd700 1 --2- 172.21.2.101:0/2007131171 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4eb40acfa0 0x7f4eb41a3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:09.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.508 7f4eb2ffd700 1 -- 172.21.2.101:0/2007131171 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb40bcae0 con 0x7f4eb40b5b80 2021-02-10T11:44:09.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.508 7f4eb2ffd700 1 --2- 172.21.2.101:0/2007131171 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4eb40b5b80 0x7f4eb41a2e20 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 rx=0x7f4e9c006fc0 tx=0x7f4e9c00b300).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:09.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.508 7f4eabfff700 1 -- 172.21.2.101:0/2007131171 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e9c006030 con 0x7f4eb40b5b80 2021-02-10T11:44:09.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.508 7f4eabfff700 1 -- 172.21.2.101:0/2007131171 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c00c7c0 con 0x7f4eb40b5b80 2021-02-10T11:44:09.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.508 7f4eabfff700 1 -- 172.21.2.101:0/2007131171 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e9c00cce0 con 0x7f4eb40b5b80 2021-02-10T11:44:09.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.508 7f4eb9bb1700 1 -- 172.21.2.101:0/2007131171 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb40bcae0 con 0x7f4eb40b5b80 2021-02-10T11:44:09.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.508 7f4eb9bb1700 1 -- 172.21.2.101:0/2007131171 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb40bd070 con 0x7f4eb40b5b80 2021-02-10T11:44:09.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.509 7f4eabfff700 1 -- 172.21.2.101:0/2007131171 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4e9c006030 con 0x7f4eb40b5b80 2021-02-10T11:44:09.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.509 7f4eabfff700 1 --2- 172.21.2.101:0/2007131171 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4e980186c0 0x7f4e9801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:09.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.509 7f4eabfff700 1 -- 172.21.2.101:0/2007131171 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4e9c0323d0 con 0x7f4eb40b5b80 2021-02-10T11:44:09.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.509 7f4eb37fe700 1 --2- 172.21.2.101:0/2007131171 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4e980186c0 0x7f4e9801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:09.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.509 7f4eb37fe700 1 --2- 172.21.2.101:0/2007131171 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4e980186c0 0x7f4e9801abc0 crc :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:09.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.539 7f4eb9bb1700 1 -- 172.21.2.101:0/2007131171 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb40f0ab0 con 0x7f4eb40b5b80 2021-02-10T11:44:09.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.542 7f4eabfff700 1 -- 172.21.2.101:0/2007131171 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4e9c000970 con 0x7f4eb40b5b80 2021-02-10T11:44:09.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.668 7f4eb9bb1700 1 -- 172.21.2.101:0/2007131171 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4eb40f0ab0 con 0x7f4eb40b5b80 2021-02-10T11:44:09.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.669 7f4eabfff700 1 -- 172.21.2.101:0/2007131171 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f4e9c006030 con 0x7f4eb40b5b80 2021-02-10T11:44:09.669 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:44:09.669 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":147,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:44:09.465154","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:44:06.431537","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":31,"pool_name":"cephfs_data","create_time":"2021-02-10 11:44:07.456920","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:44:09.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.671 7f4eb9bb1700 1 -- 172.21.2.101:0/2007131171 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4e980186c0 msgr2=0x7f4e9801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:09.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.671 7f4eb9bb1700 1 --2- 172.21.2.101:0/2007131171 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4e980186c0 0x7f4e9801abc0 crc :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:09.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.671 7f4eb9bb1700 1 -- 172.21.2.101:0/2007131171 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4eb40b5b80 msgr2=0x7f4eb41a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:09.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.671 7f4eb9bb1700 1 --2- 172.21.2.101:0/2007131171 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4eb40b5b80 0x7f4eb41a2e20 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 rx=0x7f4e9c006fc0 tx=0x7f4e9c00b300).stop 2021-02-10T11:44:09.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.671 7f4eb9bb1700 1 -- 172.21.2.101:0/2007131171 shutdown_connections 2021-02-10T11:44:09.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.671 7f4eb9bb1700 1 --2- 172.21.2.101:0/2007131171 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4eb40acfa0 0x7f4eb41a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:09.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.671 7f4eb9bb1700 1 --2- 172.21.2.101:0/2007131171 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4eb40b5b80 0x7f4eb41a2e20 unknown :-1 s=CLOSED pgs=2652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:09.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.671 7f4eb9bb1700 1 --2- 172.21.2.101:0/2007131171 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4e980186c0 0x7f4e9801abc0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:09.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.671 7f4eb9bb1700 1 --2- 172.21.2.101:0/2007131171 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4eb4195650 0x7f4eb40b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:09.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.671 7f4eb9bb1700 1 -- 172.21.2.101:0/2007131171 >> 172.21.2.101:0/2007131171 conn(0x7f4eb40b3040 msgr2=0x7f4eb40c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:09.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.671 7f4eb9bb1700 1 -- 172.21.2.101:0/2007131171 shutdown_connections 2021-02-10T11:44:09.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:09.671 7f4eb9bb1700 1 -- 172.21.2.101:0/2007131171 wait complete. 2021-02-10T11:44:09.679 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-02-10T11:44:10.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.095 7f3ec7c9e700 1 Processor -- start 2021-02-10T11:44:10.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.096 7f3ec7c9e700 1 -- start start 2021-02-10T11:44:10.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.096 7f3ec7c9e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ec00d1c70 0x7f3ec00cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:10.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.096 7f3ec7c9e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3ec00cf580 0x7f3ec00cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:10.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.096 7f3ec7c9e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ec00fe1c0 con 0x7f3ec00d1870 2021-02-10T11:44:10.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.096 7f3ec7c9e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ec01b0cb0 con 0x7f3ec00cf580 2021-02-10T11:44:10.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.096 7f3ec7c9e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec0109a60 con 0x7f3ec00d1c70 2021-02-10T11:44:10.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.096 7f3ec62bb700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3ec00cf580 0x7f3ec00cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:10.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.096 7f3ec5aba700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f3ec00d1870 0x7f3ec01b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60040/0 (socket says 172.21.2.101:60040) 2021-02-10T11:44:10.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.096 7f3ec5aba700 1 -- 172.21.2.101:0/4110182943 learned_addr learned my addr 172.21.2.101:0/4110182943 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:10.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.097 7f3ec52b9700 1 --2- 172.21.2.101:0/4110182943 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ec00d1c70 0x7f3ec00cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:10.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.097 7f3ec62bb700 1 -- 172.21.2.101:0/4110182943 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ec00d1c70 msgr2=0x7f3ec00cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:10.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.097 7f3ec62bb700 1 --2- 172.21.2.101:0/4110182943 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ec00d1c70 0x7f3ec00cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:10.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.097 7f3ec62bb700 1 -- 172.21.2.101:0/4110182943 >> v1:172.21.2.101:6789/0 conn(0x7f3ec00d1870 legacy=0x7f3ec01b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:10.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.097 7f3ec62bb700 1 -- 172.21.2.101:0/4110182943 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec0088070 con 0x7f3ec00cf580 2021-02-10T11:44:10.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.097 7f3ec62bb700 1 --2- 172.21.2.101:0/4110182943 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ec00cf580 0x7f3ec00cf980 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 rx=0x7f3eb8006e30 tx=0x7f3eb8009e60).ready entity=mon.0 client_cookie=951b7aa3a9b58240 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:10.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.097 7f3ec4ab8700 1 -- 172.21.2.101:0/4110182943 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3eb8006a50 con 0x7f3ec00cf580 2021-02-10T11:44:10.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.097 7f3ec4ab8700 1 -- 172.21.2.101:0/4110182943 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb800b300 con 0x7f3ec00cf580 2021-02-10T11:44:10.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.097 7f3ec4ab8700 1 -- 172.21.2.101:0/4110182943 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3eb800b820 con 0x7f3ec00cf580 2021-02-10T11:44:10.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.097 7f3ec7c9e700 1 -- 172.21.2.101:0/4110182943 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ec00cf580 msgr2=0x7f3ec00cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:10.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.097 7f3ec7c9e700 1 --2- 172.21.2.101:0/4110182943 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ec00cf580 0x7f3ec00cf980 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 rx=0x7f3eb8006e30 tx=0x7f3eb8009e60).stop 2021-02-10T11:44:10.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.098 7f3ec7c9e700 1 -- 172.21.2.101:0/4110182943 shutdown_connections 2021-02-10T11:44:10.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.098 7f3ec7c9e700 1 --2- 172.21.2.101:0/4110182943 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ec00d1c70 0x7f3ec00cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:10.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.098 7f3ec7c9e700 1 --2- 172.21.2.101:0/4110182943 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ec00cf580 0x7f3ec00cf980 unknown :-1 s=CLOSED pgs=2654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:10.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.098 7f3ec7c9e700 1 -- 172.21.2.101:0/4110182943 >> 172.21.2.101:0/4110182943 conn(0x7f3ec00ce420 msgr2=0x7f3ec00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:10.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.098 7f3ec7c9e700 1 -- 172.21.2.101:0/4110182943 shutdown_connections 2021-02-10T11:44:10.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.098 7f3ec7c9e700 1 -- 172.21.2.101:0/4110182943 wait complete. 2021-02-10T11:44:10.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.113 7f3ec7c9e700 1 Processor -- start 2021-02-10T11:44:10.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.113 7f3ec7c9e700 1 -- start start 2021-02-10T11:44:10.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.113 7f3ec7c9e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ec00a7950 0x7f3ec00b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:10.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.113 7f3ec7c9e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ec01a7490 0x7f3ec01a7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:10.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.113 7f3ec7c9e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ec01a7eb0 0x7f3ec00ac630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:10.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.113 7f3ec7c9e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec00a7fb0 con 0x7f3ec01a7490 2021-02-10T11:44:10.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.113 7f3ec7c9e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec00acc50 con 0x7f3ec01a7eb0 2021-02-10T11:44:10.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.113 7f3ec7c9e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec00ace30 con 0x7f3ec00a7950 2021-02-10T11:44:10.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.114 7f3ec52b9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ec01a7490 0x7f3ec01a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:10.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.114 7f3ec52b9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ec01a7490 0x7f3ec01a7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59550/0 (socket says 172.21.2.101:59550) 2021-02-10T11:44:10.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.114 7f3ec52b9700 1 -- 172.21.2.101:0/3022631658 learned_addr learned my addr 172.21.2.101:0/3022631658 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:10.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.114 7f3ec52b9700 1 -- 172.21.2.101:0/3022631658 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ec00a7950 msgr2=0x7f3ec00b8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:10.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.114 7f3ec52b9700 1 --2- 172.21.2.101:0/3022631658 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ec00a7950 0x7f3ec00b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:10.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.114 7f3ec52b9700 1 -- 172.21.2.101:0/3022631658 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ec01a7eb0 msgr2=0x7f3ec00ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:10.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.114 7f3ec52b9700 1 --2- 172.21.2.101:0/3022631658 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ec01a7eb0 0x7f3ec00ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:10.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.114 7f3ec52b9700 1 -- 172.21.2.101:0/3022631658 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec00ad050 con 0x7f3ec01a7490 2021-02-10T11:44:10.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.114 7f3ec52b9700 1 --2- 172.21.2.101:0/3022631658 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ec01a7490 0x7f3ec01a7890 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 rx=0x7f3eb0006f00 tx=0x7f3eb000b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:10.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.114 7f3eb67fc700 1 -- 172.21.2.101:0/3022631658 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3eb000bd30 con 0x7f3ec01a7490 2021-02-10T11:44:10.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.115 7f3eb67fc700 1 -- 172.21.2.101:0/3022631658 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb0002030 con 0x7f3ec01a7490 2021-02-10T11:44:10.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.115 7f3eb67fc700 1 -- 172.21.2.101:0/3022631658 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3eb0002550 con 0x7f3ec01a7490 2021-02-10T11:44:10.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.115 7f3ec7c9e700 1 -- 172.21.2.101:0/3022631658 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec00ad050 con 0x7f3ec01a7490 2021-02-10T11:44:10.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.115 7f3ec7c9e700 1 -- 172.21.2.101:0/3022631658 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec00ad050 con 0x7f3ec01a7490 2021-02-10T11:44:10.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.115 7f3eb67fc700 1 -- 172.21.2.101:0/3022631658 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3eb0000c90 con 0x7f3ec01a7490 2021-02-10T11:44:10.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.115 7f3eb67fc700 1 --2- 172.21.2.101:0/3022631658 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3ea8018730 0x7f3ea801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:10.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.116 7f3eb67fc700 1 -- 172.21.2.101:0/3022631658 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3eb0000940 con 0x7f3ec01a7490 2021-02-10T11:44:10.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.116 7f3ec5aba700 1 --2- 172.21.2.101:0/3022631658 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3ea8018730 0x7f3ea801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:10.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.116 7f3ec5aba700 1 --2- 172.21.2.101:0/3022631658 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3ea8018730 0x7f3ea801ac10 crc :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:10.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.146 7f3ec7c9e700 1 -- 172.21.2.101:0/3022631658 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec0082a70 con 0x7f3ec01a7490 2021-02-10T11:44:10.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.149 7f3eb67fc700 1 -- 172.21.2.101:0/3022631658 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3eb0000940 con 0x7f3ec01a7490 2021-02-10T11:44:10.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.286 7f3ec7c9e700 1 -- 172.21.2.101:0/3022631658 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"} v 0) v1 -- 0x7f3ec0082a70 con 0x7f3ec01a7490 2021-02-10T11:44:10.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.473 7f3eb67fc700 1 -- 172.21.2.101:0/3022631658 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"}]=0 v123) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3eb0000c90 con 0x7f3ec01a7490 2021-02-10T11:44:10.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.504 7f3ec7c9e700 1 -- 172.21.2.101:0/3022631658 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3ea8018730 msgr2=0x7f3ea801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:10.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.504 7f3ec7c9e700 1 --2- 172.21.2.101:0/3022631658 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3ea8018730 0x7f3ea801ac10 crc :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:10.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.504 7f3ec7c9e700 1 -- 172.21.2.101:0/3022631658 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ec01a7490 msgr2=0x7f3ec01a7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:10.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.504 7f3ec7c9e700 1 --2- 172.21.2.101:0/3022631658 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ec01a7490 0x7f3ec01a7890 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 rx=0x7f3eb0006f00 tx=0x7f3eb000b940).stop 2021-02-10T11:44:10.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.504 7f3ec7c9e700 1 -- 172.21.2.101:0/3022631658 shutdown_connections 2021-02-10T11:44:10.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.504 7f3ec7c9e700 1 --2- 172.21.2.101:0/3022631658 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ec01a7eb0 0x7f3ec00ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:10.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.504 7f3ec7c9e700 1 --2- 172.21.2.101:0/3022631658 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ec01a7490 0x7f3ec01a7890 unknown :-1 s=CLOSED pgs=2655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:10.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.504 7f3ec7c9e700 1 --2- 172.21.2.101:0/3022631658 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3ea8018730 0x7f3ea801ac10 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:10.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.504 7f3ec7c9e700 1 --2- 172.21.2.101:0/3022631658 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ec00a7950 0x7f3ec00b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:10.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.504 7f3ec7c9e700 1 -- 172.21.2.101:0/3022631658 >> 172.21.2.101:0/3022631658 conn(0x7f3ec00a02d0 msgr2=0x7f3ec009f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:10.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.504 7f3ec7c9e700 1 -- 172.21.2.101:0/3022631658 shutdown_connections 2021-02-10T11:44:10.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.504 7f3ec7c9e700 1 -- 172.21.2.101:0/3022631658 wait complete. 2021-02-10T11:44:10.512 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:44:10.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.924 7f7accf62700 1 Processor -- start 2021-02-10T11:44:10.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.924 7f7accf62700 1 -- start start 2021-02-10T11:44:10.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.925 7f7accf62700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7ac80d0340 0x7f7ac80ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:10.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.925 7f7accf62700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7ac80d1c70 0x7f7ac80efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:10.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.925 7f7accf62700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac80f0b10 con 0x7f7ac80d1870 2021-02-10T11:44:10.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.925 7f7accf62700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ac80e5ed0 con 0x7f7ac80d1c70 2021-02-10T11:44:10.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.925 7f7accf62700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ac81b0be0 con 0x7f7ac80d0340 2021-02-10T11:44:10.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.925 7f7ac7314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7ac80d1c70 0x7f7ac80efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:10.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.925 7f7ac7314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7ac80d1c70 0x7f7ac80efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59560/0 (socket says 172.21.2.101:59560) 2021-02-10T11:44:10.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.925 7f7ac7314700 1 -- 172.21.2.101:0/368518527 learned_addr learned my addr 172.21.2.101:0/368518527 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:10.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.925 7f7ac7314700 1 -- 172.21.2.101:0/368518527 >> v2:172.21.2.108:3300/0 conn(0x7f7ac80d0340 msgr2=0x7f7ac80ef640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:10.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.925 7f7ac7314700 1 --2- 172.21.2.101:0/368518527 >> v2:172.21.2.108:3300/0 conn(0x7f7ac80d0340 0x7f7ac80ef640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:10.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.925 7f7ac7314700 1 -- 172.21.2.101:0/368518527 >> v1:172.21.2.108:6789/0 conn(0x7f7ac80d1870 legacy=0x7f7ac81b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:10.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.925 7f7ac7314700 1 -- 172.21.2.101:0/368518527 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac80f20a0 con 0x7f7ac80d1c70 2021-02-10T11:44:10.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.926 7f7ac7314700 1 --2- 172.21.2.101:0/368518527 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ac80d1c70 0x7f7ac80efc60 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 rx=0x7f7ac0006f00 tx=0x7f7ac0009f10).ready entity=mon.0 client_cookie=6a96023f128498c9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:10.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.926 7f7ac5b11700 1 -- 172.21.2.101:0/368518527 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ac000b3b0 con 0x7f7ac80d1c70 2021-02-10T11:44:10.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.926 7f7ac5b11700 1 -- 172.21.2.101:0/368518527 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac000b630 con 0x7f7ac80d1c70 2021-02-10T11:44:10.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.926 7f7ac5b11700 1 -- 172.21.2.101:0/368518527 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ac000bb50 con 0x7f7ac80d1c70 2021-02-10T11:44:10.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.926 7f7accf62700 1 -- 172.21.2.101:0/368518527 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ac80d1c70 msgr2=0x7f7ac80efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:10.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.926 7f7accf62700 1 --2- 172.21.2.101:0/368518527 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ac80d1c70 0x7f7ac80efc60 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 rx=0x7f7ac0006f00 tx=0x7f7ac0009f10).stop 2021-02-10T11:44:10.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.927 7f7accf62700 1 -- 172.21.2.101:0/368518527 shutdown_connections 2021-02-10T11:44:10.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.927 7f7accf62700 1 --2- 172.21.2.101:0/368518527 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ac80d1c70 0x7f7ac80efc60 unknown :-1 s=CLOSED pgs=2656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:10.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.927 7f7accf62700 1 --2- 172.21.2.101:0/368518527 >> v2:172.21.2.108:3300/0 conn(0x7f7ac80d0340 0x7f7ac80ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:10.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.927 7f7accf62700 1 -- 172.21.2.101:0/368518527 >> 172.21.2.101:0/368518527 conn(0x7f7ac80ce450 msgr2=0x7f7ac80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:10.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.927 7f7accf62700 1 -- 172.21.2.101:0/368518527 shutdown_connections 2021-02-10T11:44:10.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.927 7f7accf62700 1 -- 172.21.2.101:0/368518527 wait complete. 2021-02-10T11:44:10.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.941 7f7accf62700 1 Processor -- start 2021-02-10T11:44:10.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.941 7f7accf62700 1 -- start start 2021-02-10T11:44:10.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.941 7f7accf62700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ac80b4e00 0x7f7ac80a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:10.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.941 7f7accf62700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ac80a3a60 0x7f7ac80a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:10.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.941 7f7accf62700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ac8094750 0x7f7ac8094b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:10.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.941 7f7accf62700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac80a4550 con 0x7f7ac80a3a60 2021-02-10T11:44:10.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.941 7f7accf62700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac818bf30 con 0x7f7ac8094750 2021-02-10T11:44:10.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.942 7f7accf62700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac818ba60 con 0x7f7ac80b4e00 2021-02-10T11:44:10.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.942 7f7ac6312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ac80a3a60 0x7f7ac80a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:10.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.942 7f7ac6312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ac80a3a60 0x7f7ac80a3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59564/0 (socket says 172.21.2.101:59564) 2021-02-10T11:44:10.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.942 7f7ac6312700 1 -- 172.21.2.101:0/3758524900 learned_addr learned my addr 172.21.2.101:0/3758524900 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:10.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.942 7f7ac6312700 1 -- 172.21.2.101:0/3758524900 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ac80b4e00 msgr2=0x7f7ac80a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:10.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.942 7f7ac6312700 1 --2- 172.21.2.101:0/3758524900 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ac80b4e00 0x7f7ac80a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:10.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.942 7f7ac6312700 1 -- 172.21.2.101:0/3758524900 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ac8094750 msgr2=0x7f7ac8094b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:10.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.942 7f7ac6312700 1 --2- 172.21.2.101:0/3758524900 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ac8094750 0x7f7ac8094b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:10.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.942 7f7ac6312700 1 -- 172.21.2.101:0/3758524900 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac80a15c0 con 0x7f7ac80a3a60 2021-02-10T11:44:10.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.942 7f7ac6312700 1 --2- 172.21.2.101:0/3758524900 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ac80a3a60 0x7f7ac80a3e60 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 rx=0x7f7abc00b5c0 tx=0x7f7abc00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:10.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.943 7f7ab77fe700 1 -- 172.21.2.101:0/3758524900 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7abc00bd50 con 0x7f7ac80a3a60 2021-02-10T11:44:10.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.943 7f7ab77fe700 1 -- 172.21.2.101:0/3758524900 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7abc006030 con 0x7f7ac80a3a60 2021-02-10T11:44:10.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.943 7f7ab77fe700 1 -- 172.21.2.101:0/3758524900 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7abc0054d0 con 0x7f7ac80a3a60 2021-02-10T11:44:10.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.943 7f7accf62700 1 -- 172.21.2.101:0/3758524900 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac80a15c0 con 0x7f7ac80a3a60 2021-02-10T11:44:10.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.943 7f7accf62700 1 -- 172.21.2.101:0/3758524900 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac8095460 con 0x7f7ac80a3a60 2021-02-10T11:44:10.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.943 7f7ab77fe700 1 -- 172.21.2.101:0/3758524900 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7abc006210 con 0x7f7ac80a3a60 2021-02-10T11:44:10.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.944 7f7ab77fe700 1 --2- 172.21.2.101:0/3758524900 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7aac0186c0 0x7f7aac01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:10.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.944 7f7ab77fe700 1 -- 172.21.2.101:0/3758524900 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7abc0325b0 con 0x7f7ac80a3a60 2021-02-10T11:44:10.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.944 7f7ac6b13700 1 --2- 172.21.2.101:0/3758524900 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7aac0186c0 0x7f7aac01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:10.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.944 7f7ac6b13700 1 --2- 172.21.2.101:0/3758524900 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7aac0186c0 0x7f7aac01abc0 crc :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:10.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.975 7f7accf62700 1 -- 172.21.2.101:0/3758524900 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac8095460 con 0x7f7ac80a3a60 2021-02-10T11:44:10.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:10.977 7f7ab77fe700 1 -- 172.21.2.101:0/3758524900 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7abc000940 con 0x7f7ac80a3a60 2021-02-10T11:44:11.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.105 7f7accf62700 1 -- 172.21.2.101:0/3758524900 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ac8095460 con 0x7f7ac80a3a60 2021-02-10T11:44:11.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.105 7f7ab77fe700 1 -- 172.21.2.101:0/3758524900 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+2111 (secure 0 0 0) 0x7f7abc00bd50 con 0x7f7ac80a3a60 2021-02-10T11:44:11.105 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 123 2021-02-10T11:44:11.106 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:44:11.106 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":123,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/4208244473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":4208244473},{"type":"v1","addr":"172.21.2.101:6835","nonce":4208244473}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":119}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:44:08.462960","modified":"2021-02-10 11:44:10.472350","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11814":{"gid":11814,"name":"b","rank":0,"incarnation":120,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/3445995728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3445995728},{"type":"v1","addr":"172.21.2.108:6833","nonce":3445995728}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-02-10T11:44:11.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.107 7f7accf62700 1 -- 172.21.2.101:0/3758524900 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7aac0186c0 msgr2=0x7f7aac01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:11.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.107 7f7accf62700 1 --2- 172.21.2.101:0/3758524900 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7aac0186c0 0x7f7aac01abc0 crc :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:11.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.107 7f7accf62700 1 -- 172.21.2.101:0/3758524900 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ac80a3a60 msgr2=0x7f7ac80a3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:11.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.107 7f7accf62700 1 --2- 172.21.2.101:0/3758524900 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ac80a3a60 0x7f7ac80a3e60 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 rx=0x7f7abc00b5c0 tx=0x7f7abc00b960).stop 2021-02-10T11:44:11.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.108 7f7accf62700 1 -- 172.21.2.101:0/3758524900 shutdown_connections 2021-02-10T11:44:11.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.108 7f7accf62700 1 --2- 172.21.2.101:0/3758524900 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ac8094750 0x7f7ac8094b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:11.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.108 7f7accf62700 1 --2- 172.21.2.101:0/3758524900 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ac80a3a60 0x7f7ac80a3e60 unknown :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:11.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.108 7f7accf62700 1 --2- 172.21.2.101:0/3758524900 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7aac0186c0 0x7f7aac01abc0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:11.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.108 7f7accf62700 1 --2- 172.21.2.101:0/3758524900 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ac80b4e00 0x7f7ac80a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:11.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.108 7f7accf62700 1 -- 172.21.2.101:0/3758524900 >> 172.21.2.101:0/3758524900 conn(0x7f7ac818cab0 msgr2=0x7f7ac8092c60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:11.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.108 7f7accf62700 1 -- 172.21.2.101:0/3758524900 shutdown_connections 2021-02-10T11:44:11.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.108 7f7accf62700 1 -- 172.21.2.101:0/3758524900 wait complete. 2021-02-10T11:44:11.116 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:44:11.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.531 7f6e95ab8700 1 Processor -- start 2021-02-10T11:44:11.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.531 7f6e95ab8700 1 -- start start 2021-02-10T11:44:11.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.532 7f6e95ab8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e900d0340 0x7f6e900ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:11.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.532 7f6e95ab8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6e900d1c70 0x7f6e900efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:11.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.532 7f6e95ab8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e900f0b10 con 0x7f6e900d1870 2021-02-10T11:44:11.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.532 7f6e95ab8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e900e5ed0 con 0x7f6e900d1c70 2021-02-10T11:44:11.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.532 7f6e95ab8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e901b0be0 con 0x7f6e900d0340 2021-02-10T11:44:11.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.532 7f6e8ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6e900d1c70 0x7f6e900efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:11.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.532 7f6e8ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6e900d1c70 0x7f6e900efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59574/0 (socket says 172.21.2.101:59574) 2021-02-10T11:44:11.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.532 7f6e8ffff700 1 -- 172.21.2.101:0/1185179525 learned_addr learned my addr 172.21.2.101:0/1185179525 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:11.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.532 7f6e8effd700 1 --2- 172.21.2.101:0/1185179525 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e900d0340 0x7f6e900ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:11.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.532 7f6e8ffff700 1 -- 172.21.2.101:0/1185179525 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e900d0340 msgr2=0x7f6e900ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:11.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.532 7f6e8ffff700 1 --2- 172.21.2.101:0/1185179525 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e900d0340 0x7f6e900ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:11.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.532 7f6e8ffff700 1 -- 172.21.2.101:0/1185179525 >> v1:172.21.2.108:6789/0 conn(0x7f6e900d1870 legacy=0x7f6e901b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:11.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.532 7f6e8ffff700 1 -- 172.21.2.101:0/1185179525 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e900f20a0 con 0x7f6e900d1c70 2021-02-10T11:44:11.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.532 7f6e8ffff700 1 --2- 172.21.2.101:0/1185179525 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e900d1c70 0x7f6e900efc60 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 rx=0x7f6e88006f00 tx=0x7f6e88009f10).ready entity=mon.0 client_cookie=8001b8bae2092441 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:11.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.533 7f6e8e7fc700 1 -- 172.21.2.101:0/1185179525 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e8800b3b0 con 0x7f6e900d1c70 2021-02-10T11:44:11.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.533 7f6e8e7fc700 1 -- 172.21.2.101:0/1185179525 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8800b730 con 0x7f6e900d1c70 2021-02-10T11:44:11.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.533 7f6e8e7fc700 1 -- 172.21.2.101:0/1185179525 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e8800bc50 con 0x7f6e900d1c70 2021-02-10T11:44:11.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.533 7f6e95ab8700 1 -- 172.21.2.101:0/1185179525 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e900d1c70 msgr2=0x7f6e900efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:11.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.533 7f6e95ab8700 1 --2- 172.21.2.101:0/1185179525 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e900d1c70 0x7f6e900efc60 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 rx=0x7f6e88006f00 tx=0x7f6e88009f10).stop 2021-02-10T11:44:11.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.533 7f6e95ab8700 1 -- 172.21.2.101:0/1185179525 shutdown_connections 2021-02-10T11:44:11.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.533 7f6e95ab8700 1 --2- 172.21.2.101:0/1185179525 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e900d0340 0x7f6e900ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:11.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.533 7f6e95ab8700 1 --2- 172.21.2.101:0/1185179525 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e900d1c70 0x7f6e900efc60 unknown :-1 s=CLOSED pgs=2658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:11.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.533 7f6e95ab8700 1 -- 172.21.2.101:0/1185179525 >> 172.21.2.101:0/1185179525 conn(0x7f6e900ce450 msgr2=0x7f6e900d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:11.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.533 7f6e95ab8700 1 -- 172.21.2.101:0/1185179525 shutdown_connections 2021-02-10T11:44:11.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.533 7f6e95ab8700 1 -- 172.21.2.101:0/1185179525 wait complete. 2021-02-10T11:44:11.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.548 7f6e95ab8700 1 Processor -- start 2021-02-10T11:44:11.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.548 7f6e95ab8700 1 -- start start 2021-02-10T11:44:11.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.548 7f6e95ab8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e9018d870 0x7f6e900b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:11.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.548 7f6e95ab8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e900b1fd0 0x7f6e90099ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:11.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.548 7f6e95ab8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e900b23d0 0x7f6e9009a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:11.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.548 7f6e95ab8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9018dc70 con 0x7f6e900b1fd0 2021-02-10T11:44:11.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.548 7f6e95ab8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e90199e60 con 0x7f6e900b23d0 2021-02-10T11:44:11.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.548 7f6e95ab8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9009ab30 con 0x7f6e9018d870 2021-02-10T11:44:11.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.549 7f6e8effd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e900b1fd0 0x7f6e90099ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:11.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.549 7f6e8effd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e900b1fd0 0x7f6e90099ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59578/0 (socket says 172.21.2.101:59578) 2021-02-10T11:44:11.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.549 7f6e8effd700 1 -- 172.21.2.101:0/3392520529 learned_addr learned my addr 172.21.2.101:0/3392520529 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:11.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.549 7f6e8ffff700 1 --2- 172.21.2.101:0/3392520529 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e900b23d0 0x7f6e9009a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:11.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.549 7f6e8f7fe700 1 --2- 172.21.2.101:0/3392520529 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e9018d870 0x7f6e900b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:11.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.549 7f6e8effd700 1 -- 172.21.2.101:0/3392520529 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e9018d870 msgr2=0x7f6e900b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:11.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.549 7f6e8effd700 1 --2- 172.21.2.101:0/3392520529 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e9018d870 0x7f6e900b19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:11.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.549 7f6e8effd700 1 -- 172.21.2.101:0/3392520529 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e900b23d0 msgr2=0x7f6e9009a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:11.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.549 7f6e8effd700 1 --2- 172.21.2.101:0/3392520529 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e900b23d0 0x7f6e9009a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:11.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.549 7f6e8effd700 1 -- 172.21.2.101:0/3392520529 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9009ad50 con 0x7f6e900b1fd0 2021-02-10T11:44:11.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.549 7f6e8ffff700 1 --2- 172.21.2.101:0/3392520529 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e900b23d0 0x7f6e9009a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:44:11.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.549 7f6e8effd700 1 --2- 172.21.2.101:0/3392520529 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e900b1fd0 0x7f6e90099ef0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 rx=0x7f6e8000b560 tx=0x7f6e8000b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:11.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.550 7f6e87fff700 1 -- 172.21.2.101:0/3392520529 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e8000bcf0 con 0x7f6e900b1fd0 2021-02-10T11:44:11.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.550 7f6e87fff700 1 -- 172.21.2.101:0/3392520529 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e80000c90 con 0x7f6e900b1fd0 2021-02-10T11:44:11.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.550 7f6e87fff700 1 -- 172.21.2.101:0/3392520529 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e800022b0 con 0x7f6e900b1fd0 2021-02-10T11:44:11.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.550 7f6e95ab8700 1 -- 172.21.2.101:0/3392520529 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e9009ad50 con 0x7f6e900b1fd0 2021-02-10T11:44:11.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.550 7f6e95ab8700 1 -- 172.21.2.101:0/3392520529 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9018e3f0 con 0x7f6e900b1fd0 2021-02-10T11:44:11.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.550 7f6e87fff700 1 -- 172.21.2.101:0/3392520529 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6e80006b00 con 0x7f6e900b1fd0 2021-02-10T11:44:11.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.550 7f6e87fff700 1 --2- 172.21.2.101:0/3392520529 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e740186c0 0x7f6e7401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:11.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.550 7f6e87fff700 1 -- 172.21.2.101:0/3392520529 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6e800325c0 con 0x7f6e900b1fd0 2021-02-10T11:44:11.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.551 7f6e8f7fe700 1 --2- 172.21.2.101:0/3392520529 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e740186c0 0x7f6e7401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:11.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.551 7f6e8f7fe700 1 --2- 172.21.2.101:0/3392520529 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e740186c0 0x7f6e7401abc0 crc :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:11.581 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.582 7f6e95ab8700 1 -- 172.21.2.101:0/3392520529 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e900999c0 con 0x7f6e900b1fd0 2021-02-10T11:44:11.584 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.584 7f6e87fff700 1 -- 172.21.2.101:0/3392520529 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6e800009b0 con 0x7f6e900b1fd0 2021-02-10T11:44:11.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.709 7f6e95ab8700 1 -- 172.21.2.101:0/3392520529 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e900999c0 con 0x7f6e900b1fd0 2021-02-10T11:44:11.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.710 7f6e87fff700 1 -- 172.21.2.101:0/3392520529 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f6e80006b00 con 0x7f6e900b1fd0 2021-02-10T11:44:11.710 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:44:11.710 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":147,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:44:09.465154","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:44:06.431537","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":31,"pool_name":"cephfs_data","create_time":"2021-02-10 11:44:07.456920","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:44:11.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.711 7f6e95ab8700 1 -- 172.21.2.101:0/3392520529 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e740186c0 msgr2=0x7f6e7401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:11.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.711 7f6e95ab8700 1 --2- 172.21.2.101:0/3392520529 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e740186c0 0x7f6e7401abc0 crc :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:11.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.711 7f6e95ab8700 1 -- 172.21.2.101:0/3392520529 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e900b1fd0 msgr2=0x7f6e90099ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:11.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.711 7f6e95ab8700 1 --2- 172.21.2.101:0/3392520529 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e900b1fd0 0x7f6e90099ef0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 rx=0x7f6e8000b560 tx=0x7f6e8000b900).stop 2021-02-10T11:44:11.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.712 7f6e95ab8700 1 -- 172.21.2.101:0/3392520529 shutdown_connections 2021-02-10T11:44:11.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.712 7f6e95ab8700 1 --2- 172.21.2.101:0/3392520529 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e900b23d0 0x7f6e9009a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:11.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.712 7f6e95ab8700 1 --2- 172.21.2.101:0/3392520529 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e900b1fd0 0x7f6e90099ef0 unknown :-1 s=CLOSED pgs=2659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:11.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.712 7f6e95ab8700 1 --2- 172.21.2.101:0/3392520529 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e740186c0 0x7f6e7401abc0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:11.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.712 7f6e95ab8700 1 --2- 172.21.2.101:0/3392520529 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e9018d870 0x7f6e900b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:11.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.712 7f6e95ab8700 1 -- 172.21.2.101:0/3392520529 >> 172.21.2.101:0/3392520529 conn(0x7f6e900f0550 msgr2=0x7f6e90191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:11.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.712 7f6e95ab8700 1 -- 172.21.2.101:0/3392520529 shutdown_connections 2021-02-10T11:44:11.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:11.712 7f6e95ab8700 1 -- 172.21.2.101:0/3392520529 wait complete. 2021-02-10T11:44:11.720 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:44:12.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.144 7f099690b700 1 Processor -- start 2021-02-10T11:44:12.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.144 7f099690b700 1 -- start start 2021-02-10T11:44:12.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.144 7f099690b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f09900d1880 0x7f09901b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:12.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.144 7f099690b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f09901b0af0 0x7f0990108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:12.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.144 7f099690b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09900d1c80 0x7f0990108f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:12.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.144 7f099690b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09900fe1d0 con 0x7f09901b0af0 2021-02-10T11:44:12.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.144 7f099690b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0990109a70 con 0x7f09900d1880 2021-02-10T11:44:12.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.144 7f099690b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0990088070 con 0x7f09900d1c80 2021-02-10T11:44:12.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.145 7f098f7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f09901b0af0 0x7f0990108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:12.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.145 7f098f7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f09901b0af0 0x7f0990108950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59586/0 (socket says 172.21.2.101:59586) 2021-02-10T11:44:12.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.145 7f098f7fe700 1 -- 172.21.2.101:0/1287052655 learned_addr learned my addr 172.21.2.101:0/1287052655 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:12.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.145 7f098f7fe700 1 -- 172.21.2.101:0/1287052655 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09900d1c80 msgr2=0x7f0990108f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:12.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.145 7f098f7fe700 1 --2- 172.21.2.101:0/1287052655 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09900d1c80 0x7f0990108f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:12.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.145 7f098f7fe700 1 -- 172.21.2.101:0/1287052655 >> v2:172.21.2.108:3300/0 conn(0x7f09900d1880 msgr2=0x7f09901b04d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:12.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.145 7f098f7fe700 1 --2- 172.21.2.101:0/1287052655 >> v2:172.21.2.108:3300/0 conn(0x7f09900d1880 0x7f09901b04d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:12.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.145 7f098f7fe700 1 -- 172.21.2.101:0/1287052655 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0990109590 con 0x7f09901b0af0 2021-02-10T11:44:12.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.145 7f098f7fe700 1 --2- 172.21.2.101:0/1287052655 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09901b0af0 0x7f0990108950 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 rx=0x7f0984006700 tx=0x7f0984009b80).ready entity=mon.0 client_cookie=cade9bf70e2bdafb server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:12.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.145 7f098effd700 1 -- 172.21.2.101:0/1287052655 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f098400c0c0 con 0x7f09901b0af0 2021-02-10T11:44:12.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.145 7f098effd700 1 -- 172.21.2.101:0/1287052655 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098400c340 con 0x7f09901b0af0 2021-02-10T11:44:12.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.145 7f098effd700 1 -- 172.21.2.101:0/1287052655 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f098400c860 con 0x7f09901b0af0 2021-02-10T11:44:12.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.145 7f099690b700 1 -- 172.21.2.101:0/1287052655 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09901b0af0 msgr2=0x7f0990108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:12.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.145 7f099690b700 1 --2- 172.21.2.101:0/1287052655 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09901b0af0 0x7f0990108950 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 rx=0x7f0984006700 tx=0x7f0984009b80).stop 2021-02-10T11:44:12.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.146 7f099690b700 1 -- 172.21.2.101:0/1287052655 shutdown_connections 2021-02-10T11:44:12.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.146 7f099690b700 1 --2- 172.21.2.101:0/1287052655 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09900d1c80 0x7f0990108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:12.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.146 7f099690b700 1 --2- 172.21.2.101:0/1287052655 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09901b0af0 0x7f0990108950 unknown :-1 s=CLOSED pgs=2660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:12.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.146 7f099690b700 1 --2- 172.21.2.101:0/1287052655 >> v2:172.21.2.108:3300/0 conn(0x7f09900d1880 0x7f09901b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:12.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.146 7f099690b700 1 -- 172.21.2.101:0/1287052655 >> 172.21.2.101:0/1287052655 conn(0x7f09900ce430 msgr2=0x7f09900d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:12.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.146 7f099690b700 1 -- 172.21.2.101:0/1287052655 shutdown_connections 2021-02-10T11:44:12.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.146 7f099690b700 1 -- 172.21.2.101:0/1287052655 wait complete. 2021-02-10T11:44:12.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.161 7f099690b700 1 Processor -- start 2021-02-10T11:44:12.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.161 7f099690b700 1 -- start start 2021-02-10T11:44:12.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.161 7f099690b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09901b9470 0x7f09901ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:12.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.161 7f099690b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09901aecf0 0x7f09900957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:12.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.161 7f099690b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09901af0f0 0x7f0990095e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:12.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.161 7f099690b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0990096d70 con 0x7f09901b9470 2021-02-10T11:44:12.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.161 7f099690b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09900a7780 con 0x7f09901af0f0 2021-02-10T11:44:12.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.161 7f099690b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09901b9870 con 0x7f09901aecf0 2021-02-10T11:44:12.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.162 7f098ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09901b9470 0x7f09901ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:12.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.162 7f098ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09901b9470 0x7f09901ae6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59590/0 (socket says 172.21.2.101:59590) 2021-02-10T11:44:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.162 7f098ffff700 1 -- 172.21.2.101:0/3761509601 learned_addr learned my addr 172.21.2.101:0/3761509601 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.162 7f098ffff700 1 -- 172.21.2.101:0/3761509601 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09901aecf0 msgr2=0x7f09900957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.162 7f098ffff700 1 --2- 172.21.2.101:0/3761509601 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09901aecf0 0x7f09900957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.162 7f098ffff700 1 -- 172.21.2.101:0/3761509601 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09901af0f0 msgr2=0x7f0990095e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.162 7f098ffff700 1 --2- 172.21.2.101:0/3761509601 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09901af0f0 0x7f0990095e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.162 7f098ffff700 1 -- 172.21.2.101:0/3761509601 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099019e6b0 con 0x7f09901b9470 2021-02-10T11:44:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.162 7f098ffff700 1 --2- 172.21.2.101:0/3761509601 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09901b9470 0x7f09901ae6d0 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 rx=0x7f098000b5c0 tx=0x7f098000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.163 7f098cff9700 1 -- 172.21.2.101:0/3761509601 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f098000bd50 con 0x7f09901b9470 2021-02-10T11:44:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.163 7f098cff9700 1 -- 172.21.2.101:0/3761509601 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0980006030 con 0x7f09901b9470 2021-02-10T11:44:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.163 7f098cff9700 1 -- 172.21.2.101:0/3761509601 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f09800054d0 con 0x7f09901b9470 2021-02-10T11:44:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.163 7f099690b700 1 -- 172.21.2.101:0/3761509601 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f099019e6b0 con 0x7f09901b9470 2021-02-10T11:44:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.163 7f099690b700 1 -- 172.21.2.101:0/3761509601 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f099019e6b0 con 0x7f09901b9470 2021-02-10T11:44:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.163 7f098cff9700 1 -- 172.21.2.101:0/3761509601 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0980006210 con 0x7f09901b9470 2021-02-10T11:44:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.163 7f098cff9700 1 --2- 172.21.2.101:0/3761509601 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09700186c0 0x7f097001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:12.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.163 7f098cff9700 1 -- 172.21.2.101:0/3761509601 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0980032420 con 0x7f09901b9470 2021-02-10T11:44:12.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.164 7f098f7fe700 1 --2- 172.21.2.101:0/3761509601 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09700186c0 0x7f097001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:12.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.164 7f098f7fe700 1 --2- 172.21.2.101:0/3761509601 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09700186c0 0x7f097001abc0 crc :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:12.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.194 7f099690b700 1 -- 172.21.2.101:0/3761509601 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09900829a0 con 0x7f09901b9470 2021-02-10T11:44:12.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.196 7f098cff9700 1 -- 172.21.2.101:0/3761509601 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0980006b80 con 0x7f09901b9470 2021-02-10T11:44:12.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.326 7f099690b700 1 -- 172.21.2.101:0/3761509601 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f09900a7780 con 0x7f09901b9470 2021-02-10T11:44:12.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.330 7f098cff9700 1 -- 172.21.2.101:0/3761509601 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v83) v1 ==== 184+0+0 (secure 0 0 0) 0x7f0980006b80 con 0x7f09901b9470 2021-02-10T11:44:12.332 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.0 2021-02-10T11:44:12.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.335 7f099690b700 1 -- 172.21.2.101:0/3761509601 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09700186c0 msgr2=0x7f097001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:12.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.335 7f099690b700 1 --2- 172.21.2.101:0/3761509601 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09700186c0 0x7f097001abc0 crc :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:12.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.335 7f099690b700 1 -- 172.21.2.101:0/3761509601 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09901b9470 msgr2=0x7f09901ae6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:12.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.335 7f099690b700 1 --2- 172.21.2.101:0/3761509601 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09901b9470 0x7f09901ae6d0 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 rx=0x7f098000b5c0 tx=0x7f098000b960).stop 2021-02-10T11:44:12.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.335 7f099690b700 1 -- 172.21.2.101:0/3761509601 shutdown_connections 2021-02-10T11:44:12.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.335 7f099690b700 1 --2- 172.21.2.101:0/3761509601 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09700186c0 0x7f097001abc0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:12.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.335 7f099690b700 1 --2- 172.21.2.101:0/3761509601 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09901aecf0 0x7f09900957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:12.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.335 7f099690b700 1 --2- 172.21.2.101:0/3761509601 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09901af0f0 0x7f0990095e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:12.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.335 7f099690b700 1 --2- 172.21.2.101:0/3761509601 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09901b9470 0x7f09901ae6d0 unknown :-1 s=CLOSED pgs=2661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:12.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.335 7f099690b700 1 -- 172.21.2.101:0/3761509601 >> 172.21.2.101:0/3761509601 conn(0x7f09900845b0 msgr2=0x7f099018c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:12.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.335 7f099690b700 1 -- 172.21.2.101:0/3761509601 shutdown_connections 2021-02-10T11:44:12.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.335 7f099690b700 1 -- 172.21.2.101:0/3761509601 wait complete. 2021-02-10T11:44:12.343 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:44:12.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.768 7f715b942700 1 Processor -- start 2021-02-10T11:44:12.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.768 7f715b942700 1 -- start start 2021-02-10T11:44:12.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.768 7f715b942700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71540d1880 0x7f71541b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:12.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.768 7f715b942700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f71541b0af0 0x7f7154108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:12.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.768 7f715b942700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7154109a70 con 0x7f71540d1c80 2021-02-10T11:44:12.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.768 7f715b942700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71540e5f00 con 0x7f71541b0af0 2021-02-10T11:44:12.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.768 7f715b942700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7154109760 con 0x7f71540d1880 2021-02-10T11:44:12.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.768 7f715975e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71540d1880 0x7f71541b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:12.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.769 7f7159f5f700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f71540d1c80 0x7f7154108f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:53506/0 (socket says 172.21.2.101:53506) 2021-02-10T11:44:12.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.769 7f7159f5f700 1 -- 172.21.2.101:0/3722243766 learned_addr learned my addr 172.21.2.101:0/3722243766 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:12.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.769 7f7158f5d700 1 --2- 172.21.2.101:0/3722243766 >> v2:172.21.2.108:3300/0 conn(0x7f71541b0af0 0x7f7154108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:12.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.769 7f715975e700 1 -- 172.21.2.101:0/3722243766 >> v2:172.21.2.108:3300/0 conn(0x7f71541b0af0 msgr2=0x7f7154108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:12.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.769 7f715975e700 1 --2- 172.21.2.101:0/3722243766 >> v2:172.21.2.108:3300/0 conn(0x7f71541b0af0 0x7f7154108950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:12.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.769 7f715975e700 1 -- 172.21.2.101:0/3722243766 >> v1:172.21.2.108:6789/0 conn(0x7f71540d1c80 legacy=0x7f7154108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:12.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.769 7f715975e700 1 -- 172.21.2.101:0/3722243766 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7154088070 con 0x7f71540d1880 2021-02-10T11:44:12.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.769 7f715975e700 1 --2- 172.21.2.101:0/3722243766 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71540d1880 0x7f71541b04d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f714c0069d0 tx=0x7f714c009e40).ready entity=mon.2 client_cookie=b3bb76aee1eea123 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:12.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.769 7f714bfff700 1 -- 172.21.2.101:0/3722243766 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f714c00b300 con 0x7f71540d1880 2021-02-10T11:44:12.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.769 7f714bfff700 1 -- 172.21.2.101:0/3722243766 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714c00b640 con 0x7f71540d1880 2021-02-10T11:44:12.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.769 7f714bfff700 1 -- 172.21.2.101:0/3722243766 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f714c00bb60 con 0x7f71540d1880 2021-02-10T11:44:12.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.769 7f715b942700 1 -- 172.21.2.101:0/3722243766 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71540d1880 msgr2=0x7f71541b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:12.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.769 7f715b942700 1 --2- 172.21.2.101:0/3722243766 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71540d1880 0x7f71541b04d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f714c0069d0 tx=0x7f714c009e40).stop 2021-02-10T11:44:12.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.770 7f715b942700 1 -- 172.21.2.101:0/3722243766 shutdown_connections 2021-02-10T11:44:12.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.770 7f715b942700 1 --2- 172.21.2.101:0/3722243766 >> v2:172.21.2.108:3300/0 conn(0x7f71541b0af0 0x7f7154108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:12.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.770 7f715b942700 1 --2- 172.21.2.101:0/3722243766 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71540d1880 0x7f71541b04d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:12.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.770 7f715b942700 1 -- 172.21.2.101:0/3722243766 >> 172.21.2.101:0/3722243766 conn(0x7f71540ce430 msgr2=0x7f71540d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:12.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.771 7f715b942700 1 -- 172.21.2.101:0/3722243766 shutdown_connections 2021-02-10T11:44:12.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.771 7f715b942700 1 -- 172.21.2.101:0/3722243766 wait complete. 2021-02-10T11:44:12.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.785 7f715b942700 1 Processor -- start 2021-02-10T11:44:12.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.785 7f715b942700 1 -- start start 2021-02-10T11:44:12.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.786 7f715b942700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71541b9470 0x7f71541ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:12.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.786 7f715b942700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71541aecf0 0x7f71540957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:12.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.786 7f715b942700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f71541af0f0 0x7f7154095e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:12.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.786 7f715b942700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7154096d70 con 0x7f71541b9470 2021-02-10T11:44:12.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.786 7f715b942700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71540a7780 con 0x7f71541af0f0 2021-02-10T11:44:12.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.786 7f715b942700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71541b9870 con 0x7f71541aecf0 2021-02-10T11:44:12.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.786 7f715975e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71541b9470 0x7f71541ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:12.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.786 7f7158f5d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71541aecf0 0x7f71540957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:12.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.786 7f7158f5d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71541aecf0 0x7f71540957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:52336/0 (socket says 172.21.2.101:52336) 2021-02-10T11:44:12.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.786 7f7158f5d700 1 -- 172.21.2.101:0/3881891548 learned_addr learned my addr 172.21.2.101:0/3881891548 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:12.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.786 7f715975e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71541b9470 0x7f71541ae6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59604/0 (socket says 172.21.2.101:59604) 2021-02-10T11:44:12.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.786 7f7159f5f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f71541af0f0 0x7f7154095e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:12.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.787 7f7158f5d700 1 -- 172.21.2.101:0/3881891548 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f71541af0f0 msgr2=0x7f7154095e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:12.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.787 7f7158f5d700 1 --2- 172.21.2.101:0/3881891548 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f71541af0f0 0x7f7154095e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:12.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.787 7f7158f5d700 1 -- 172.21.2.101:0/3881891548 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71541b9470 msgr2=0x7f71541ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:12.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.787 7f7158f5d700 1 --2- 172.21.2.101:0/3881891548 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71541b9470 0x7f71541ae6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:12.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.787 7f7158f5d700 1 -- 172.21.2.101:0/3881891548 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715419e6b0 con 0x7f71541aecf0 2021-02-10T11:44:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.787 7f7158f5d700 1 --2- 172.21.2.101:0/3881891548 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71541aecf0 0x7f71540957f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f7144006f60 tx=0x7f714400b970).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.787 7f7149ffb700 1 -- 172.21.2.101:0/3881891548 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f714400bd60 con 0x7f71541aecf0 2021-02-10T11:44:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.787 7f7149ffb700 1 -- 172.21.2.101:0/3881891548 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7144000c90 con 0x7f71541aecf0 2021-02-10T11:44:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.787 7f7149ffb700 1 -- 172.21.2.101:0/3881891548 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f71440009b0 con 0x7f71541aecf0 2021-02-10T11:44:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.787 7f715b942700 1 -- 172.21.2.101:0/3881891548 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715419e6b0 con 0x7f71541aecf0 2021-02-10T11:44:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.787 7f715b942700 1 -- 172.21.2.101:0/3881891548 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715419e6b0 con 0x7f71541aecf0 2021-02-10T11:44:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.788 7f7149ffb700 1 -- 172.21.2.101:0/3881891548 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f71440052e0 con 0x7f71541aecf0 2021-02-10T11:44:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.788 7f7149ffb700 1 --2- 172.21.2.101:0/3881891548 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f71340186c0 0x7f713401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.788 7f7149ffb700 1 -- 172.21.2.101:0/3881891548 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7144032540 con 0x7f71541aecf0 2021-02-10T11:44:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.788 7f715975e700 1 --2- 172.21.2.101:0/3881891548 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f71340186c0 0x7f713401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:12.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.788 7f715975e700 1 --2- 172.21.2.101:0/3881891548 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f71340186c0 0x7f713401abc0 crc :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:12.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.818 7f715b942700 1 -- 172.21.2.101:0/3881891548 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71540829a0 con 0x7f71541aecf0 2021-02-10T11:44:12.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.821 7f7149ffb700 1 -- 172.21.2.101:0/3881891548 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7144030cf0 con 0x7f71541aecf0 2021-02-10T11:44:12.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.951 7f715b942700 1 -- 172.21.2.101:0/3881891548 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f7154109a70 con 0x7f71541aecf0 2021-02-10T11:44:12.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.956 7f7149ffb700 1 -- 172.21.2.101:0/3881891548 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v84) v1 ==== 184+0+0 (secure 0 0 0) 0x7f71440052e0 con 0x7f71541aecf0 2021-02-10T11:44:12.958 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.1 2021-02-10T11:44:12.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.960 7f715b942700 1 -- 172.21.2.101:0/3881891548 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f71340186c0 msgr2=0x7f713401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:12.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.960 7f715b942700 1 --2- 172.21.2.101:0/3881891548 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f71340186c0 0x7f713401abc0 crc :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:12.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.960 7f715b942700 1 -- 172.21.2.101:0/3881891548 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71541aecf0 msgr2=0x7f71540957f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:12.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.960 7f715b942700 1 --2- 172.21.2.101:0/3881891548 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71541aecf0 0x7f71540957f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f7144006f60 tx=0x7f714400b970).stop 2021-02-10T11:44:12.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.960 7f715b942700 1 -- 172.21.2.101:0/3881891548 shutdown_connections 2021-02-10T11:44:12.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.960 7f715b942700 1 --2- 172.21.2.101:0/3881891548 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f71340186c0 0x7f713401abc0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:12.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.960 7f715b942700 1 --2- 172.21.2.101:0/3881891548 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f71541aecf0 0x7f71540957f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:12.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.960 7f715b942700 1 --2- 172.21.2.101:0/3881891548 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f71541af0f0 0x7f7154095e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:12.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.960 7f715b942700 1 --2- 172.21.2.101:0/3881891548 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f71541b9470 0x7f71541ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:12.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.960 7f715b942700 1 -- 172.21.2.101:0/3881891548 >> 172.21.2.101:0/3881891548 conn(0x7f71540845b0 msgr2=0x7f715418c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:12.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.960 7f715b942700 1 -- 172.21.2.101:0/3881891548 shutdown_connections 2021-02-10T11:44:12.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:12.960 7f715b942700 1 -- 172.21.2.101:0/3881891548 wait complete. 2021-02-10T11:44:12.968 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:44:13.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.381 7efdfba33700 1 Processor -- start 2021-02-10T11:44:13.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.382 7efdfba33700 1 -- start start 2021-02-10T11:44:13.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.382 7efdfba33700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7efdf40d1880 0x7efdf41b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:13.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.382 7efdfba33700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7efdf41b0af0 0x7efdf4108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:13.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.382 7efdfba33700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efdf40d1c80 0x7efdf4108f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:13.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.382 7efdfba33700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf40fe1d0 con 0x7efdf41b0af0 2021-02-10T11:44:13.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.382 7efdfba33700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf4109a70 con 0x7efdf40d1880 2021-02-10T11:44:13.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.382 7efdfba33700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf4088070 con 0x7efdf40d1c80 2021-02-10T11:44:13.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.382 7efdf904e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7efdf41b0af0 0x7efdf4108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:13.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.382 7efdf904e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7efdf41b0af0 0x7efdf4108950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59612/0 (socket says 172.21.2.101:59612) 2021-02-10T11:44:13.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.382 7efdf904e700 1 -- 172.21.2.101:0/244136527 learned_addr learned my addr 172.21.2.101:0/244136527 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:13.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.383 7efdf904e700 1 -- 172.21.2.101:0/244136527 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efdf40d1c80 msgr2=0x7efdf4108f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:13.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.383 7efdf904e700 1 --2- 172.21.2.101:0/244136527 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efdf40d1c80 0x7efdf4108f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:13.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.383 7efdf904e700 1 -- 172.21.2.101:0/244136527 >> v2:172.21.2.108:3300/0 conn(0x7efdf40d1880 msgr2=0x7efdf41b04d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:13.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.383 7efdf904e700 1 --2- 172.21.2.101:0/244136527 >> v2:172.21.2.108:3300/0 conn(0x7efdf40d1880 0x7efdf41b04d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:13.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.383 7efdf904e700 1 -- 172.21.2.101:0/244136527 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf4109590 con 0x7efdf41b0af0 2021-02-10T11:44:13.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.383 7efdf904e700 1 --2- 172.21.2.101:0/244136527 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efdf41b0af0 0x7efdf4108950 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 rx=0x7efde8006a80 tx=0x7efde8009f00).ready entity=mon.0 client_cookie=da6bcb54cc38b61b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:13.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.383 7efdf884d700 1 -- 172.21.2.101:0/244136527 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efde800b5c0 con 0x7efdf41b0af0 2021-02-10T11:44:13.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.383 7efdf884d700 1 -- 172.21.2.101:0/244136527 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde800b8c0 con 0x7efdf41b0af0 2021-02-10T11:44:13.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.383 7efdf884d700 1 -- 172.21.2.101:0/244136527 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efde800bda0 con 0x7efdf41b0af0 2021-02-10T11:44:13.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.383 7efdfba33700 1 -- 172.21.2.101:0/244136527 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efdf41b0af0 msgr2=0x7efdf4108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:13.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.383 7efdfba33700 1 --2- 172.21.2.101:0/244136527 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efdf41b0af0 0x7efdf4108950 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 rx=0x7efde8006a80 tx=0x7efde8009f00).stop 2021-02-10T11:44:13.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.384 7efdfba33700 1 -- 172.21.2.101:0/244136527 shutdown_connections 2021-02-10T11:44:13.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.384 7efdfba33700 1 --2- 172.21.2.101:0/244136527 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efdf40d1c80 0x7efdf4108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:13.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.384 7efdfba33700 1 --2- 172.21.2.101:0/244136527 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efdf41b0af0 0x7efdf4108950 unknown :-1 s=CLOSED pgs=2662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:13.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.384 7efdfba33700 1 --2- 172.21.2.101:0/244136527 >> v2:172.21.2.108:3300/0 conn(0x7efdf40d1880 0x7efdf41b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:13.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.384 7efdfba33700 1 -- 172.21.2.101:0/244136527 >> 172.21.2.101:0/244136527 conn(0x7efdf40ce430 msgr2=0x7efdf40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:13.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.384 7efdfba33700 1 -- 172.21.2.101:0/244136527 shutdown_connections 2021-02-10T11:44:13.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.384 7efdfba33700 1 -- 172.21.2.101:0/244136527 wait complete. 2021-02-10T11:44:13.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.398 7efdfba33700 1 Processor -- start 2021-02-10T11:44:13.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.398 7efdfba33700 1 -- start start 2021-02-10T11:44:13.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.398 7efdfba33700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efdf4090250 0x7efdf41b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:13.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.398 7efdfba33700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efdf41b3a80 0x7efdf41b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:13.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.398 7efdfba33700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efdf40b1700 0x7efdf40b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:13.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.399 7efdfba33700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf40b2120 con 0x7efdf4090250 2021-02-10T11:44:13.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.399 7efdfba33700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf40b2340 con 0x7efdf40b1700 2021-02-10T11:44:13.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.399 7efdfba33700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf40b2560 con 0x7efdf41b3a80 2021-02-10T11:44:13.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.399 7efdf984f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efdf4090250 0x7efdf41b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:13.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.399 7efdf984f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efdf4090250 0x7efdf41b3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59618/0 (socket says 172.21.2.101:59618) 2021-02-10T11:44:13.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.399 7efdf984f700 1 -- 172.21.2.101:0/1213192799 learned_addr learned my addr 172.21.2.101:0/1213192799 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:13.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.399 7efdf984f700 1 -- 172.21.2.101:0/1213192799 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efdf41b3a80 msgr2=0x7efdf41b3e80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:13.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.399 7efdf984f700 1 --2- 172.21.2.101:0/1213192799 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efdf41b3a80 0x7efdf41b3e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:13.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.399 7efdf984f700 1 -- 172.21.2.101:0/1213192799 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efdf40b1700 msgr2=0x7efdf40b1b00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:13.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.399 7efdf984f700 1 --2- 172.21.2.101:0/1213192799 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efdf40b1700 0x7efdf40b1b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:13.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.399 7efdf984f700 1 -- 172.21.2.101:0/1213192799 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf41b6c00 con 0x7efdf4090250 2021-02-10T11:44:13.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.399 7efdf984f700 1 --2- 172.21.2.101:0/1213192799 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efdf4090250 0x7efdf41b3460 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 rx=0x7efde400b560 tx=0x7efde400b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:13.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.399 7efdf27fc700 1 -- 172.21.2.101:0/1213192799 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efde400bcf0 con 0x7efdf4090250 2021-02-10T11:44:13.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.399 7efdf27fc700 1 -- 172.21.2.101:0/1213192799 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde4006030 con 0x7efdf4090250 2021-02-10T11:44:13.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.399 7efdfba33700 1 -- 172.21.2.101:0/1213192799 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf41b6c00 con 0x7efdf4090250 2021-02-10T11:44:13.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.399 7efdf27fc700 1 -- 172.21.2.101:0/1213192799 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efde40054d0 con 0x7efdf4090250 2021-02-10T11:44:13.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.400 7efdfba33700 1 -- 172.21.2.101:0/1213192799 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf41b6c00 con 0x7efdf4090250 2021-02-10T11:44:13.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.400 7efdf27fc700 1 -- 172.21.2.101:0/1213192799 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7efde4006210 con 0x7efdf4090250 2021-02-10T11:44:13.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.400 7efdf27fc700 1 --2- 172.21.2.101:0/1213192799 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efdd40186d0 0x7efdd401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:13.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.400 7efdf27fc700 1 -- 172.21.2.101:0/1213192799 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7efde40327f0 con 0x7efdf4090250 2021-02-10T11:44:13.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.400 7efdf904e700 1 --2- 172.21.2.101:0/1213192799 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efdd40186d0 0x7efdd401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:13.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.401 7efdf904e700 1 --2- 172.21.2.101:0/1213192799 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efdd40186d0 0x7efdd401abd0 crc :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.433 7efdfba33700 1 -- 172.21.2.101:0/1213192799 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf40fe1d0 con 0x7efdf4090250 2021-02-10T11:44:13.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.435 7efdf27fc700 1 -- 172.21.2.101:0/1213192799 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7efde4006b80 con 0x7efdf4090250 2021-02-10T11:44:13.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.566 7efdfba33700 1 -- 172.21.2.101:0/1213192799 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7efdf40fe1d0 con 0x7efdf4090250 2021-02-10T11:44:13.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.569 7efdf27fc700 1 -- 172.21.2.101:0/1213192799 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v85) v1 ==== 184+0+0 (secure 0 0 0) 0x7efde4006160 con 0x7efdf4090250 2021-02-10T11:44:13.568 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.2 2021-02-10T11:44:13.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.570 7efdfba33700 1 -- 172.21.2.101:0/1213192799 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efdd40186d0 msgr2=0x7efdd401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:13.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.570 7efdfba33700 1 --2- 172.21.2.101:0/1213192799 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efdd40186d0 0x7efdd401abd0 crc :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:13.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.570 7efdfba33700 1 -- 172.21.2.101:0/1213192799 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efdf4090250 msgr2=0x7efdf41b3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:13.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.570 7efdfba33700 1 --2- 172.21.2.101:0/1213192799 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efdf4090250 0x7efdf41b3460 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 rx=0x7efde400b560 tx=0x7efde400b900).stop 2021-02-10T11:44:13.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.571 7efdfba33700 1 -- 172.21.2.101:0/1213192799 shutdown_connections 2021-02-10T11:44:13.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.571 7efdfba33700 1 --2- 172.21.2.101:0/1213192799 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efdd40186d0 0x7efdd401abd0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:13.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.571 7efdfba33700 1 --2- 172.21.2.101:0/1213192799 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efdf41b3a80 0x7efdf41b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:13.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.571 7efdfba33700 1 --2- 172.21.2.101:0/1213192799 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efdf40b1700 0x7efdf40b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:13.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.571 7efdfba33700 1 --2- 172.21.2.101:0/1213192799 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efdf4090250 0x7efdf41b3460 unknown :-1 s=CLOSED pgs=2663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:13.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.571 7efdfba33700 1 -- 172.21.2.101:0/1213192799 >> 172.21.2.101:0/1213192799 conn(0x7efdf409aa90 msgr2=0x7efdf41b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:13.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.571 7efdfba33700 1 -- 172.21.2.101:0/1213192799 shutdown_connections 2021-02-10T11:44:13.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:13.571 7efdfba33700 1 -- 172.21.2.101:0/1213192799 wait complete. 2021-02-10T11:44:13.579 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:44:14.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.004 7fc86c842700 1 Processor -- start 2021-02-10T11:44:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.004 7fc86c842700 1 -- start start 2021-02-10T11:44:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.004 7fc86c842700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc8680d1880 0x7fc8681b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.004 7fc86c842700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8680cff80 0x7fc868109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.004 7fc86c842700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc868109a70 con 0x7fc8681b0af0 2021-02-10T11:44:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.004 7fc86c842700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc868109760 con 0x7fc8680d1880 2021-02-10T11:44:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.004 7fc86c842700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc868088070 con 0x7fc8680cff80 2021-02-10T11:44:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.004 7fc86661c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc8680d1880 0x7fc8681b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.004 7fc86661c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc8680d1880 0x7fc8681b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59626/0 (socket says 172.21.2.101:59626) 2021-02-10T11:44:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.004 7fc86661c700 1 -- 172.21.2.101:0/1220457047 learned_addr learned my addr 172.21.2.101:0/1220457047 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.005 7fc86661c700 1 -- 172.21.2.101:0/1220457047 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8680cff80 msgr2=0x7fc868109140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.005 7fc86661c700 1 --2- 172.21.2.101:0/1220457047 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8680cff80 0x7fc868109140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.005 7fc86661c700 1 -- 172.21.2.101:0/1220457047 >> v1:172.21.2.108:6789/0 conn(0x7fc8681b0af0 legacy=0x7fc868108950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:14.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.005 7fc86661c700 1 -- 172.21.2.101:0/1220457047 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8680d1c80 con 0x7fc8680d1880 2021-02-10T11:44:14.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.005 7fc86661c700 1 --2- 172.21.2.101:0/1220457047 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8680d1880 0x7fc8681b04d0 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 rx=0x7fc850006700 tx=0x7fc850009b80).ready entity=mon.0 client_cookie=e0bda5970291db50 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:14.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.005 7fc86561a700 1 -- 172.21.2.101:0/1220457047 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc85000c0c0 con 0x7fc8680d1880 2021-02-10T11:44:14.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.005 7fc86561a700 1 -- 172.21.2.101:0/1220457047 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85000c340 con 0x7fc8680d1880 2021-02-10T11:44:14.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.005 7fc86561a700 1 -- 172.21.2.101:0/1220457047 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc85000c860 con 0x7fc8680d1880 2021-02-10T11:44:14.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.005 7fc86c842700 1 -- 172.21.2.101:0/1220457047 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8680d1880 msgr2=0x7fc8681b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:14.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.005 7fc86c842700 1 --2- 172.21.2.101:0/1220457047 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8680d1880 0x7fc8681b04d0 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 rx=0x7fc850006700 tx=0x7fc850009b80).stop 2021-02-10T11:44:14.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.006 7fc86c842700 1 -- 172.21.2.101:0/1220457047 shutdown_connections 2021-02-10T11:44:14.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.006 7fc86c842700 1 --2- 172.21.2.101:0/1220457047 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8680cff80 0x7fc868109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:14.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.006 7fc86c842700 1 --2- 172.21.2.101:0/1220457047 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8680d1880 0x7fc8681b04d0 unknown :-1 s=CLOSED pgs=2664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:14.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.006 7fc86c842700 1 -- 172.21.2.101:0/1220457047 >> 172.21.2.101:0/1220457047 conn(0x7fc8680ce430 msgr2=0x7fc8680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:14.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.006 7fc86c842700 1 -- 172.21.2.101:0/1220457047 shutdown_connections 2021-02-10T11:44:14.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.006 7fc86c842700 1 -- 172.21.2.101:0/1220457047 wait complete. 2021-02-10T11:44:14.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.021 7fc86c842700 1 Processor -- start 2021-02-10T11:44:14.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.021 7fc86c842700 1 -- start start 2021-02-10T11:44:14.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.021 7fc86c842700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc868086810 0x7fc86819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:14.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.021 7fc86c842700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8680b04d0 0x7fc868097450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:14.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.021 7fc86c842700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8680b08d0 0x7fc868097a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:14.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.021 7fc86c842700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8681a02c0 con 0x7fc868086810 2021-02-10T11:44:14.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.021 7fc86c842700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc868098090 con 0x7fc8680b08d0 2021-02-10T11:44:14.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.021 7fc86c842700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8680982b0 con 0x7fc8680b04d0 2021-02-10T11:44:14.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.021 7fc86661c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc868086810 0x7fc86819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:14.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.021 7fc86661c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc868086810 0x7fc86819fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59634/0 (socket says 172.21.2.101:59634) 2021-02-10T11:44:14.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.021 7fc86661c700 1 -- 172.21.2.101:0/1529227953 learned_addr learned my addr 172.21.2.101:0/1529227953 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:14.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.021 7fc86661c700 1 -- 172.21.2.101:0/1529227953 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8680b04d0 msgr2=0x7fc868097450 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.021 7fc86661c700 1 --2- 172.21.2.101:0/1529227953 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8680b04d0 0x7fc868097450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.021 7fc86661c700 1 -- 172.21.2.101:0/1529227953 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8680b08d0 msgr2=0x7fc868097a70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.022 7fc86661c700 1 --2- 172.21.2.101:0/1529227953 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8680b08d0 0x7fc868097a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.022 7fc86661c700 1 -- 172.21.2.101:0/1529227953 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8680984d0 con 0x7fc868086810 2021-02-10T11:44:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.022 7fc86661c700 1 --2- 172.21.2.101:0/1529227953 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc868086810 0x7fc86819fca0 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 rx=0x7fc85000b5e0 tx=0x7fc85000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.022 7fc85effd700 1 -- 172.21.2.101:0/1529227953 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc85000bd70 con 0x7fc868086810 2021-02-10T11:44:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.022 7fc85effd700 1 -- 172.21.2.101:0/1529227953 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc850002030 con 0x7fc868086810 2021-02-10T11:44:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.022 7fc86c842700 1 -- 172.21.2.101:0/1529227953 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8680984d0 con 0x7fc868086810 2021-02-10T11:44:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.022 7fc86c842700 1 -- 172.21.2.101:0/1529227953 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc868098b80 con 0x7fc868086810 2021-02-10T11:44:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.022 7fc85effd700 1 -- 172.21.2.101:0/1529227953 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc850002550 con 0x7fc868086810 2021-02-10T11:44:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.023 7fc85effd700 1 -- 172.21.2.101:0/1529227953 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc850000c90 con 0x7fc868086810 2021-02-10T11:44:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.023 7fc85effd700 1 --2- 172.21.2.101:0/1529227953 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc84c0186c0 0x7fc84c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.023 7fc85effd700 1 -- 172.21.2.101:0/1529227953 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc8500325c0 con 0x7fc868086810 2021-02-10T11:44:14.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.023 7fc865e1b700 1 --2- 172.21.2.101:0/1529227953 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc84c0186c0 0x7fc84c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:14.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.023 7fc865e1b700 1 --2- 172.21.2.101:0/1529227953 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc84c0186c0 0x7fc84c01abc0 crc :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:14.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.054 7fc86c842700 1 -- 172.21.2.101:0/1529227953 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8680fe1d0 con 0x7fc868086810 2021-02-10T11:44:14.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.057 7fc85effd700 1 -- 172.21.2.101:0/1529227953 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc850028ed0 con 0x7fc868086810 2021-02-10T11:44:14.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.187 7fc86c842700 1 -- 172.21.2.101:0/1529227953 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fc8680fe1d0 con 0x7fc868086810 2021-02-10T11:44:14.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.190 7fc85effd700 1 -- 172.21.2.101:0/1529227953 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v86) v1 ==== 184+0+0 (secure 0 0 0) 0x7fc850028ed0 con 0x7fc868086810 2021-02-10T11:44:14.193 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.3 2021-02-10T11:44:14.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.196 7fc86c842700 1 -- 172.21.2.101:0/1529227953 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc84c0186c0 msgr2=0x7fc84c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:14.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.196 7fc86c842700 1 --2- 172.21.2.101:0/1529227953 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc84c0186c0 0x7fc84c01abc0 crc :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:14.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.196 7fc86c842700 1 -- 172.21.2.101:0/1529227953 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc868086810 msgr2=0x7fc86819fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:14.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.196 7fc86c842700 1 --2- 172.21.2.101:0/1529227953 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc868086810 0x7fc86819fca0 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 rx=0x7fc85000b5e0 tx=0x7fc85000b980).stop 2021-02-10T11:44:14.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.196 7fc86c842700 1 -- 172.21.2.101:0/1529227953 shutdown_connections 2021-02-10T11:44:14.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.196 7fc86c842700 1 --2- 172.21.2.101:0/1529227953 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc84c0186c0 0x7fc84c01abc0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:14.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.196 7fc86c842700 1 --2- 172.21.2.101:0/1529227953 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8680b04d0 0x7fc868097450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:14.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.196 7fc86c842700 1 --2- 172.21.2.101:0/1529227953 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8680b08d0 0x7fc868097a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:14.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.196 7fc86c842700 1 --2- 172.21.2.101:0/1529227953 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc868086810 0x7fc86819fca0 unknown :-1 s=CLOSED pgs=2665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:14.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.196 7fc86c842700 1 -- 172.21.2.101:0/1529227953 >> 172.21.2.101:0/1529227953 conn(0x7fc86819f880 msgr2=0x7fc86808f890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:14.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.196 7fc86c842700 1 -- 172.21.2.101:0/1529227953 shutdown_connections 2021-02-10T11:44:14.196 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.196 7fc86c842700 1 -- 172.21.2.101:0/1529227953 wait complete. 2021-02-10T11:44:14.203 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:44:14.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.616 7fe338069700 1 Processor -- start 2021-02-10T11:44:14.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.616 7fe338069700 1 -- start start 2021-02-10T11:44:14.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.616 7fe338069700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe3300d1870 0x7fe3301b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:14.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.616 7fe338069700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3300f0650 0x7fe3300cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:14.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.616 7fe338069700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3300f0b10 con 0x7fe3301b0a10 2021-02-10T11:44:14.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.616 7fe338069700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3300e5ed0 con 0x7fe3300d1870 2021-02-10T11:44:14.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.616 7fe338069700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3300d1c70 con 0x7fe3300f0650 2021-02-10T11:44:14.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.617 7fe335684700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe3301b0a10 0x7fe3300cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60134/0 (socket says 172.21.2.101:60134) 2021-02-10T11:44:14.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.617 7fe335684700 1 -- 172.21.2.101:0/2312673570 learned_addr learned my addr 172.21.2.101:0/2312673570 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:14.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.617 7fe334e83700 1 -- 172.21.2.101:0/2312673570 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853180561 0 0) 0x7fe3240010b0 con 0x7fe3301b0a10 2021-02-10T11:44:14.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.617 7fe334e83700 1 -- 172.21.2.101:0/2312673570 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe31c002fe0 con 0x7fe3301b0a10 2021-02-10T11:44:14.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.617 7fe334e83700 1 -- 172.21.2.101:0/2312673570 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 137684424 0 0) 0x7fe3240010b0 con 0x7fe3301b0a10 2021-02-10T11:44:14.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.617 7fe334e83700 1 -- 172.21.2.101:0/2312673570 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3300f0650 msgr2=0x7fe3300cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:14.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.617 7fe334e83700 1 --2- 172.21.2.101:0/2312673570 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3300f0650 0x7fe3300cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:14.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.617 7fe334e83700 1 -- 172.21.2.101:0/2312673570 >> v2:172.21.2.108:3300/0 conn(0x7fe3300d1870 msgr2=0x7fe3301b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:14.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.617 7fe334e83700 1 --2- 172.21.2.101:0/2312673570 >> v2:172.21.2.108:3300/0 conn(0x7fe3300d1870 0x7fe3301b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:14.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.617 7fe334e83700 1 -- 172.21.2.101:0/2312673570 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3300d1e90 con 0x7fe3301b0a10 2021-02-10T11:44:14.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.617 7fe334e83700 1 -- 172.21.2.101:0/2312673570 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fe3240015b0 con 0x7fe3301b0a10 2021-02-10T11:44:14.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.617 7fe334e83700 1 -- 172.21.2.101:0/2312673570 >> v1:172.21.2.101:6789/0 conn(0x7fe3301b0a10 legacy=0x7fe3300cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:14.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.617 7fe334e83700 1 --2- 172.21.2.101:0/2312673570 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe31c003af0 0x7fe31c005f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:14.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.617 7fe334e83700 1 --2- 172.21.2.101:0/2312673570 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe31c006760 0x7fe31c008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:14.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.617 7fe334e83700 1 --2- 172.21.2.101:0/2312673570 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe31c009520 0x7fe31c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:14.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.617 7fe334e83700 1 -- 172.21.2.101:0/2312673570 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c00c2d0 con 0x7fe31c006760 2021-02-10T11:44:14.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.617 7fe334e83700 1 -- 172.21.2.101:0/2312673570 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c00c4f0 con 0x7fe31c009520 2021-02-10T11:44:14.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.617 7fe334e83700 1 -- 172.21.2.101:0/2312673570 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c00c710 con 0x7fe31c003af0 2021-02-10T11:44:14.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.617 7fe335684700 1 --2- 172.21.2.101:0/2312673570 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe31c006760 0x7fe31c008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:14.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.618 7fe336686700 1 --2- 172.21.2.101:0/2312673570 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe31c009520 0x7fe31c00ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:14.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.618 7fe335e85700 1 --2- 172.21.2.101:0/2312673570 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe31c003af0 0x7fe31c005f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:14.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.618 7fe335684700 1 -- 172.21.2.101:0/2312673570 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe31c003af0 msgr2=0x7fe31c005f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:14.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.618 7fe335684700 1 --2- 172.21.2.101:0/2312673570 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe31c003af0 0x7fe31c005f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:14.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.618 7fe335684700 1 -- 172.21.2.101:0/2312673570 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe31c009520 msgr2=0x7fe31c00ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:14.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.618 7fe335684700 1 --2- 172.21.2.101:0/2312673570 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe31c009520 0x7fe31c00ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:14.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.618 7fe335684700 1 -- 172.21.2.101:0/2312673570 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c00c9b0 con 0x7fe31c006760 2021-02-10T11:44:14.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.618 7fe335684700 1 --2- 172.21.2.101:0/2312673570 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe31c006760 0x7fe31c008c80 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 rx=0x7fe3240017d0 tx=0x7fe324007d30).ready entity=mon.0 client_cookie=3a1ecb5b0ef71838 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:14.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.618 7fe334e83700 1 -- 172.21.2.101:0/2312673570 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe32404e320 con 0x7fe31c006760 2021-02-10T11:44:14.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.618 7fe334e83700 1 -- 172.21.2.101:0/2312673570 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3240013a0 con 0x7fe31c006760 2021-02-10T11:44:14.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.618 7fe334e83700 1 -- 172.21.2.101:0/2312673570 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe32404e920 con 0x7fe31c006760 2021-02-10T11:44:14.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.618 7fe338069700 1 -- 172.21.2.101:0/2312673570 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe31c006760 msgr2=0x7fe31c008c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:14.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.618 7fe338069700 1 --2- 172.21.2.101:0/2312673570 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe31c006760 0x7fe31c008c80 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 rx=0x7fe3240017d0 tx=0x7fe324007d30).stop 2021-02-10T11:44:14.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.618 7fe336686700 1 -- 172.21.2.101:0/2312673570 reap_dead start 2021-02-10T11:44:14.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.619 7fe338069700 1 -- 172.21.2.101:0/2312673570 shutdown_connections 2021-02-10T11:44:14.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.619 7fe338069700 1 -- 172.21.2.101:0/2312673570 >> 172.21.2.101:0/2312673570 conn(0x7fe3300ce450 msgr2=0x7fe3300d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:14.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.619 7fe338069700 1 -- 172.21.2.101:0/2312673570 shutdown_connections 2021-02-10T11:44:14.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.619 7fe338069700 1 -- 172.21.2.101:0/2312673570 wait complete. 2021-02-10T11:44:14.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.634 7fe338069700 1 Processor -- start 2021-02-10T11:44:14.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.634 7fe338069700 1 -- start start 2021-02-10T11:44:14.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.634 7fe338069700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3300f0280 0x7fe3300a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:14.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.634 7fe338069700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe3300f0680 0x7fe3300a2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:14.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.634 7fe338069700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3300ec180 0x7fe3300ec580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:14.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.634 7fe338069700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3300efd00 con 0x7fe3300ec180 2021-02-10T11:44:14.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.634 7fe338069700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3300ecba0 con 0x7fe3300f0680 2021-02-10T11:44:14.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.634 7fe338069700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3300ecd80 con 0x7fe3300f0280 2021-02-10T11:44:14.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.635 7fe335e85700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3300f0280 0x7fe3300a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:14.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.635 7fe336686700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3300ec180 0x7fe3300ec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:14.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.635 7fe336686700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3300ec180 0x7fe3300ec580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59654/0 (socket says 172.21.2.101:59654) 2021-02-10T11:44:14.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.635 7fe336686700 1 -- 172.21.2.101:0/983882095 learned_addr learned my addr 172.21.2.101:0/983882095 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:14.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.635 7fe335e85700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3300f0280 0x7fe3300a2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:52384/0 (socket says 172.21.2.101:52384) 2021-02-10T11:44:14.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.635 7fe335684700 1 --2- 172.21.2.101:0/983882095 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe3300f0680 0x7fe3300a2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:14.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.635 7fe335e85700 1 -- 172.21.2.101:0/983882095 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe3300f0680 msgr2=0x7fe3300a2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:14.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.635 7fe335e85700 1 --2- 172.21.2.101:0/983882095 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe3300f0680 0x7fe3300a2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:14.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.635 7fe335e85700 1 -- 172.21.2.101:0/983882095 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3300ec180 msgr2=0x7fe3300ec580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:14.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.635 7fe335e85700 1 --2- 172.21.2.101:0/983882095 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3300ec180 0x7fe3300ec580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:14.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.635 7fe335e85700 1 -- 172.21.2.101:0/983882095 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3300ecfa0 con 0x7fe3300f0280 2021-02-10T11:44:14.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.635 7fe336686700 1 --2- 172.21.2.101:0/983882095 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3300ec180 0x7fe3300ec580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:44:14.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.635 7fe335e85700 1 --2- 172.21.2.101:0/983882095 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3300f0280 0x7fe3300a2310 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fe32c006ec0 tx=0x7fe32c00b910).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:14.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.636 7fe3227fc700 1 -- 172.21.2.101:0/983882095 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe32c00bd00 con 0x7fe3300f0280 2021-02-10T11:44:14.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.636 7fe3227fc700 1 -- 172.21.2.101:0/983882095 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32c002de0 con 0x7fe3300f0280 2021-02-10T11:44:14.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.636 7fe338069700 1 -- 172.21.2.101:0/983882095 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3300ecfa0 con 0x7fe3300f0280 2021-02-10T11:44:14.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.636 7fe3227fc700 1 -- 172.21.2.101:0/983882095 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe32c002080 con 0x7fe3300f0280 2021-02-10T11:44:14.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.636 7fe338069700 1 -- 172.21.2.101:0/983882095 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3300ecfa0 con 0x7fe3300f0280 2021-02-10T11:44:14.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.636 7fe3227fc700 1 -- 172.21.2.101:0/983882095 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe32c000c90 con 0x7fe3300f0280 2021-02-10T11:44:14.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.636 7fe3227fc700 1 --2- 172.21.2.101:0/983882095 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3180186c0 0x7fe31801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:14.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.636 7fe3227fc700 1 -- 172.21.2.101:0/983882095 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe32c032510 con 0x7fe3300f0280 2021-02-10T11:44:14.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.637 7fe335684700 1 --2- 172.21.2.101:0/983882095 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3180186c0 0x7fe31801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:14.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.637 7fe335684700 1 --2- 172.21.2.101:0/983882095 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3180186c0 0x7fe31801abc0 crc :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:14.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.666 7fe338069700 1 -- 172.21.2.101:0/983882095 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3300ebc60 con 0x7fe3300f0280 2021-02-10T11:44:14.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.669 7fe3227fc700 1 -- 172.21.2.101:0/983882095 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe32c01b3a0 con 0x7fe3300f0280 2021-02-10T11:44:14.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.800 7fe338069700 1 -- 172.21.2.101:0/983882095 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3300ebc60 con 0x7fe3300f0280 2021-02-10T11:44:14.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.802 7fe3227fc700 1 -- 172.21.2.101:0/983882095 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+2111 (secure 0 0 0) 0x7fe32c01b3a0 con 0x7fe3300f0280 2021-02-10T11:44:14.803 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:44:14.803 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":123,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/4208244473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":4208244473},{"type":"v1","addr":"172.21.2.101:6835","nonce":4208244473}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":119}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:44:08.462960","modified":"2021-02-10 11:44:10.472350","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11814":{"gid":11814,"name":"b","rank":0,"incarnation":120,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/3445995728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3445995728},{"type":"v1","addr":"172.21.2.108:6833","nonce":3445995728}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-02-10T11:44:14.803 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 123 2021-02-10T11:44:14.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.805 7fe338069700 1 -- 172.21.2.101:0/983882095 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3180186c0 msgr2=0x7fe31801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:14.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.805 7fe338069700 1 --2- 172.21.2.101:0/983882095 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3180186c0 0x7fe31801abc0 crc :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:14.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.805 7fe338069700 1 -- 172.21.2.101:0/983882095 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3300f0280 msgr2=0x7fe3300a2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:14.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.805 7fe338069700 1 --2- 172.21.2.101:0/983882095 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3300f0280 0x7fe3300a2310 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fe32c006ec0 tx=0x7fe32c00b910).stop 2021-02-10T11:44:14.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.805 7fe338069700 1 -- 172.21.2.101:0/983882095 shutdown_connections 2021-02-10T11:44:14.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.805 7fe338069700 1 --2- 172.21.2.101:0/983882095 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3300ec180 0x7fe3300ec580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:14.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.805 7fe338069700 1 --2- 172.21.2.101:0/983882095 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe3300f0680 0x7fe3300a2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:14.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.805 7fe338069700 1 --2- 172.21.2.101:0/983882095 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3180186c0 0x7fe31801abc0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:14.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.805 7fe338069700 1 --2- 172.21.2.101:0/983882095 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3300f0280 0x7fe3300a2310 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:14.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.805 7fe338069700 1 -- 172.21.2.101:0/983882095 >> 172.21.2.101:0/983882095 conn(0x7fe3300c2130 msgr2=0x7fe33009e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:14.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.805 7fe338069700 1 -- 172.21.2.101:0/983882095 shutdown_connections 2021-02-10T11:44:14.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:14.805 7fe338069700 1 -- 172.21.2.101:0/983882095 wait complete. 2021-02-10T11:44:15.014 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 123, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:44:08.462960', 'modified': '2021-02-10 11:44:10.472350', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11814}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11814': {'gid': 11814, 'name': 'b', 'rank': 0, 'incarnation': 120, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/3445995728', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 3445995728}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 3445995728}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [31], 'metadata_pool': 30, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:44:15.014 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:44:15.015 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:44:15.412 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:44:15.412 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:44:15.412 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:44:15.412 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 11814, 2021-02-10T11:44:15.412 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:44:15.412 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:44:15.412 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 6.9483842950000003, 2021-02-10T11:44:15.412 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 123, 2021-02-10T11:44:15.413 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 147, 2021-02-10T11:44:15.413 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 147, 2021-02-10T11:44:15.413 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 12.592738688000001 2021-02-10T11:44:15.413 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:44:15.416 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 11814, "want_state": "up:active", "state": "up:active", "rank_uptime": 6.9483842950000003, "mdsmap_epoch": 123, "osdmap_epoch": 147, "osdmap_epoch_barrier": 147, "uptime": 12.592738688000001 } 2021-02-10T11:44:15.416 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:44:15.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.831 7f54a3e14700 1 Processor -- start 2021-02-10T11:44:15.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.831 7f54a3e14700 1 -- start start 2021-02-10T11:44:15.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.831 7f54a3e14700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f549c0d0340 0x7f549c0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:15.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.831 7f54a3e14700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f549c0d1c70 0x7f549c0efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:15.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.831 7f54a3e14700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549c0f0b10 con 0x7f549c0d1870 2021-02-10T11:44:15.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.831 7f54a3e14700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549c0e5ed0 con 0x7f549c0d1c70 2021-02-10T11:44:15.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.831 7f54a3e14700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549c1b0be0 con 0x7f549c0d0340 2021-02-10T11:44:15.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.832 7f54a1c30700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f549c0d1870 0x7f549c1b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60152/0 (socket says 172.21.2.101:60152) 2021-02-10T11:44:15.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.832 7f54a1c30700 1 -- 172.21.2.101:0/2226359082 learned_addr learned my addr 172.21.2.101:0/2226359082 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:15.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.832 7f54a0c2e700 1 -- 172.21.2.101:0/2226359082 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4014363801 0 0) 0x7f54980010b0 con 0x7f549c0d1870 2021-02-10T11:44:15.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.832 7f54a0c2e700 1 -- 172.21.2.101:0/2226359082 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5488002fe0 con 0x7f549c0d1870 2021-02-10T11:44:15.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.832 7f54a2431700 1 --2- 172.21.2.101:0/2226359082 >> v2:172.21.2.108:3300/0 conn(0x7f549c0d1c70 0x7f549c0efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:15.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.832 7f54a142f700 1 --2- 172.21.2.101:0/2226359082 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f549c0d0340 0x7f549c0ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:15.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.832 7f54a0c2e700 1 -- 172.21.2.101:0/2226359082 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3387365494 0 0) 0x7f54980010b0 con 0x7f549c0d1870 2021-02-10T11:44:15.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.832 7f54a0c2e700 1 -- 172.21.2.101:0/2226359082 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f549c0d0340 msgr2=0x7f549c0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:15.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.832 7f54a0c2e700 1 --2- 172.21.2.101:0/2226359082 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f549c0d0340 0x7f549c0ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:15.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.832 7f54a0c2e700 1 -- 172.21.2.101:0/2226359082 >> v2:172.21.2.108:3300/0 conn(0x7f549c0d1c70 msgr2=0x7f549c0efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:15.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.832 7f54a0c2e700 1 --2- 172.21.2.101:0/2226359082 >> v2:172.21.2.108:3300/0 conn(0x7f549c0d1c70 0x7f549c0efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:15.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.832 7f54a0c2e700 1 -- 172.21.2.101:0/2226359082 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c0f20a0 con 0x7f549c0d1870 2021-02-10T11:44:15.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.832 7f54a0c2e700 1 -- 172.21.2.101:0/2226359082 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f54980015b0 con 0x7f549c0d1870 2021-02-10T11:44:15.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.832 7f54a0c2e700 1 -- 172.21.2.101:0/2226359082 >> v1:172.21.2.101:6789/0 conn(0x7f549c0d1870 legacy=0x7f549c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:15.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.832 7f54a0c2e700 1 --2- 172.21.2.101:0/2226359082 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5488003a70 0x7f5488005f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:15.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.832 7f54a0c2e700 1 --2- 172.21.2.101:0/2226359082 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5488006810 0x7f5488008d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:15.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.832 7f54a0c2e700 1 --2- 172.21.2.101:0/2226359082 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5488009630 0x7f548800baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:15.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.832 7f54a0c2e700 1 -- 172.21.2.101:0/2226359082 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548800c2f0 con 0x7f5488006810 2021-02-10T11:44:15.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.833 7f54a0c2e700 1 -- 172.21.2.101:0/2226359082 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54880504b0 con 0x7f5488003a70 2021-02-10T11:44:15.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.833 7f54a0c2e700 1 -- 172.21.2.101:0/2226359082 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54880506d0 con 0x7f5488009630 2021-02-10T11:44:15.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.833 7f54a142f700 1 --2- 172.21.2.101:0/2226359082 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5488006810 0x7f5488008d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:15.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.833 7f54a2431700 1 --2- 172.21.2.101:0/2226359082 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5488009630 0x7f548800baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:15.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.833 7f54a1c30700 1 --2- 172.21.2.101:0/2226359082 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5488003a70 0x7f5488005f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:15.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.833 7f54a2431700 1 -- 172.21.2.101:0/2226359082 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5488003a70 msgr2=0x7f5488005f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:15.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.833 7f54a2431700 1 --2- 172.21.2.101:0/2226359082 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5488003a70 0x7f5488005f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:15.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.833 7f54a2431700 1 -- 172.21.2.101:0/2226359082 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5488006810 msgr2=0x7f5488008d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:15.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.833 7f54a2431700 1 --2- 172.21.2.101:0/2226359082 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5488006810 0x7f5488008d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:15.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.833 7f54a2431700 1 -- 172.21.2.101:0/2226359082 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5488050970 con 0x7f5488009630 2021-02-10T11:44:15.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.833 7f54a1c30700 1 --2- 172.21.2.101:0/2226359082 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5488003a70 0x7f5488005f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:44:15.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.833 7f54a142f700 1 --2- 172.21.2.101:0/2226359082 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5488006810 0x7f5488008d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:44:15.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.833 7f54a2431700 1 --2- 172.21.2.101:0/2226359082 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5488009630 0x7f548800baa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f54940068d0 tx=0x7f549400b050).ready entity=mon.2 client_cookie=386c7b8bd29517a5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:15.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.833 7f54a0c2e700 1 -- 172.21.2.101:0/2226359082 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f549400b650 con 0x7f5488009630 2021-02-10T11:44:15.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.833 7f54a0c2e700 1 -- 172.21.2.101:0/2226359082 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5494006c60 con 0x7f5488009630 2021-02-10T11:44:15.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.833 7f54a0c2e700 1 -- 172.21.2.101:0/2226359082 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5494000f90 con 0x7f5488009630 2021-02-10T11:44:15.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.833 7f54a3e14700 1 -- 172.21.2.101:0/2226359082 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5488009630 msgr2=0x7f548800baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:15.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.833 7f54a3e14700 1 --2- 172.21.2.101:0/2226359082 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5488009630 0x7f548800baa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f54940068d0 tx=0x7f549400b050).stop 2021-02-10T11:44:15.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.833 7f54a2431700 1 -- 172.21.2.101:0/2226359082 reap_dead start 2021-02-10T11:44:15.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.834 7f54a3e14700 1 -- 172.21.2.101:0/2226359082 shutdown_connections 2021-02-10T11:44:15.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.834 7f54a3e14700 1 -- 172.21.2.101:0/2226359082 >> 172.21.2.101:0/2226359082 conn(0x7f549c0ce450 msgr2=0x7f549c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:15.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.834 7f54a3e14700 1 -- 172.21.2.101:0/2226359082 shutdown_connections 2021-02-10T11:44:15.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.834 7f54a3e14700 1 -- 172.21.2.101:0/2226359082 wait complete. 2021-02-10T11:44:15.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.849 7f54a3e14700 1 Processor -- start 2021-02-10T11:44:15.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.849 7f54a3e14700 1 -- start start 2021-02-10T11:44:15.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.850 7f54a3e14700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f549c195650 0x7f549c0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:15.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.850 7f54a3e14700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f549c0b5b80 0x7f549c1a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:15.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.850 7f54a3e14700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f549c0acfa0 0x7f549c1a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:15.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.850 7f54a3e14700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c0b6010 con 0x7f549c0acfa0 2021-02-10T11:44:15.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.850 7f54a3e14700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c1a3a60 con 0x7f549c0b5b80 2021-02-10T11:44:15.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.850 7f54a3e14700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549c1a3c40 con 0x7f549c195650 2021-02-10T11:44:15.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.850 7f54a2431700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f549c0acfa0 0x7f549c1a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:15.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.850 7f54a2431700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f549c0acfa0 0x7f549c1a3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59674/0 (socket says 172.21.2.101:59674) 2021-02-10T11:44:15.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.850 7f54a2431700 1 -- 172.21.2.101:0/2549516526 learned_addr learned my addr 172.21.2.101:0/2549516526 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:15.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.850 7f54a1c30700 1 --2- 172.21.2.101:0/2549516526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f549c195650 0x7f549c0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:15.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.850 7f54a142f700 1 --2- 172.21.2.101:0/2549516526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f549c0b5b80 0x7f549c1a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:15.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.850 7f54a2431700 1 -- 172.21.2.101:0/2549516526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f549c195650 msgr2=0x7f549c0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:15.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.850 7f54a2431700 1 --2- 172.21.2.101:0/2549516526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f549c195650 0x7f549c0b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:15.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.850 7f54a2431700 1 -- 172.21.2.101:0/2549516526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f549c0b5b80 msgr2=0x7f549c1a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:15.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.850 7f54a2431700 1 --2- 172.21.2.101:0/2549516526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f549c0b5b80 0x7f549c1a2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:15.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.850 7f54a2431700 1 -- 172.21.2.101:0/2549516526 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c0bcae0 con 0x7f549c0acfa0 2021-02-10T11:44:15.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.850 7f54a142f700 1 --2- 172.21.2.101:0/2549516526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f549c0b5b80 0x7f549c1a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:44:15.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.850 7f54a2431700 1 --2- 172.21.2.101:0/2549516526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f549c0acfa0 0x7f549c1a3440 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 rx=0x7f5494006e00 tx=0x7f549400cc50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:15.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.850 7f54927fc700 1 -- 172.21.2.101:0/2549516526 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f549400cd90 con 0x7f549c0acfa0 2021-02-10T11:44:15.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.851 7f54927fc700 1 -- 172.21.2.101:0/2549516526 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549400a030 con 0x7f549c0acfa0 2021-02-10T11:44:15.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.851 7f54927fc700 1 -- 172.21.2.101:0/2549516526 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f549400a4c0 con 0x7f549c0acfa0 2021-02-10T11:44:15.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.851 7f54a3e14700 1 -- 172.21.2.101:0/2549516526 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549c0bcae0 con 0x7f549c0acfa0 2021-02-10T11:44:15.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.851 7f54a3e14700 1 -- 172.21.2.101:0/2549516526 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549c0bcae0 con 0x7f549c0acfa0 2021-02-10T11:44:15.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.851 7f54927fc700 1 -- 172.21.2.101:0/2549516526 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5494009ad0 con 0x7f549c0acfa0 2021-02-10T11:44:15.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.851 7f54927fc700 1 --2- 172.21.2.101:0/2549516526 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5484018730 0x7f548401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:15.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.851 7f54927fc700 1 -- 172.21.2.101:0/2549516526 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f54940329e0 con 0x7f549c0acfa0 2021-02-10T11:44:15.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.851 7f54a1c30700 1 --2- 172.21.2.101:0/2549516526 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5484018730 0x7f548401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:15.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.852 7f54a1c30700 1 --2- 172.21.2.101:0/2549516526 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5484018730 0x7f548401ac10 crc :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:15.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.881 7f54a3e14700 1 -- 172.21.2.101:0/2549516526 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549c0b6010 con 0x7f549c0acfa0 2021-02-10T11:44:15.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:15.884 7f54927fc700 1 -- 172.21.2.101:0/2549516526 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5494009ce0 con 0x7f549c0acfa0 2021-02-10T11:44:16.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.008 7f54a3e14700 1 -- 172.21.2.101:0/2549516526 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f549c0b6010 con 0x7f549c0acfa0 2021-02-10T11:44:16.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.009 7f54927fc700 1 -- 172.21.2.101:0/2549516526 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f5494009ad0 con 0x7f549c0acfa0 2021-02-10T11:44:16.009 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:44:16.009 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":147,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:44:09.465154","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:44:06.431537","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":31,"pool_name":"cephfs_data","create_time":"2021-02-10 11:44:07.456920","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:44:16.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.011 7f54a3e14700 1 -- 172.21.2.101:0/2549516526 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5484018730 msgr2=0x7f548401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:16.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.011 7f54a3e14700 1 --2- 172.21.2.101:0/2549516526 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5484018730 0x7f548401ac10 crc :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:16.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.011 7f54a3e14700 1 -- 172.21.2.101:0/2549516526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f549c0acfa0 msgr2=0x7f549c1a3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:16.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.011 7f54a3e14700 1 --2- 172.21.2.101:0/2549516526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f549c0acfa0 0x7f549c1a3440 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 rx=0x7f5494006e00 tx=0x7f549400cc50).stop 2021-02-10T11:44:16.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.011 7f54a3e14700 1 -- 172.21.2.101:0/2549516526 shutdown_connections 2021-02-10T11:44:16.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.011 7f54a3e14700 1 --2- 172.21.2.101:0/2549516526 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f549c0acfa0 0x7f549c1a3440 unknown :-1 s=CLOSED pgs=2669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:16.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.011 7f54a3e14700 1 --2- 172.21.2.101:0/2549516526 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f549c0b5b80 0x7f549c1a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:16.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.011 7f54a3e14700 1 --2- 172.21.2.101:0/2549516526 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5484018730 0x7f548401ac10 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:16.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.011 7f54a3e14700 1 --2- 172.21.2.101:0/2549516526 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f549c195650 0x7f549c0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:16.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.011 7f54a3e14700 1 -- 172.21.2.101:0/2549516526 >> 172.21.2.101:0/2549516526 conn(0x7f549c0b3040 msgr2=0x7f549c0c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:16.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.011 7f54a3e14700 1 -- 172.21.2.101:0/2549516526 shutdown_connections 2021-02-10T11:44:16.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.011 7f54a3e14700 1 -- 172.21.2.101:0/2549516526 wait complete. 2021-02-10T11:44:16.019 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 30, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:44:06.431537', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '146', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 31, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:44:07.456920', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:44:16.019 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:44:16.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.438 7f9ba2f36700 1 Processor -- start 2021-02-10T11:44:16.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.438 7f9ba2f36700 1 -- start start 2021-02-10T11:44:16.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.438 7f9ba2f36700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b9c10afe0 0x7f9b9c0cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:16.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.439 7f9ba2f36700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9c0fe1b0 con 0x7f9b9c0d1860 2021-02-10T11:44:16.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.439 7f9ba2f36700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9c1b0ca0 con 0x7f9b9c0d1c60 2021-02-10T11:44:16.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.439 7f9ba2f36700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c109a50 con 0x7f9b9c10afe0 2021-02-10T11:44:16.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.439 7f9ba0d52700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f9b9c0d1860 0x7f9b9c1b04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60176/0 (socket says 172.21.2.101:60176) 2021-02-10T11:44:16.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.439 7f9ba0d52700 1 -- 172.21.2.101:0/3399418083 learned_addr learned my addr 172.21.2.101:0/3399418083 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:16.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.439 7f9b9b7fe700 1 -- 172.21.2.101:0/3399418083 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2481918734 0 0) 0x7f9b840009b0 con 0x7f9b9c0d1860 2021-02-10T11:44:16.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.439 7f9b9b7fe700 1 -- 172.21.2.101:0/3399418083 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b88002fe0 con 0x7f9b9c0d1860 2021-02-10T11:44:16.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.439 7f9ba1553700 1 --2- 172.21.2.101:0/3399418083 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b9c10afe0 0x7f9b9c0cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:16.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9b9b7fe700 1 -- 172.21.2.101:0/3399418083 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3040740610 0 0) 0x7f9b84001010 con 0x7f9b9c0d1860 2021-02-10T11:44:16.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9b9b7fe700 1 -- 172.21.2.101:0/3399418083 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b9c10afe0 msgr2=0x7f9b9c0cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:16.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9b9b7fe700 1 --2- 172.21.2.101:0/3399418083 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b9c10afe0 0x7f9b9c0cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:16.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9b9b7fe700 1 -- 172.21.2.101:0/3399418083 >> v1:172.21.2.108:6789/0 conn(0x7f9b9c0d1c60 legacy=0x7f9b9c0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:16.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9b9b7fe700 1 -- 172.21.2.101:0/3399418083 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c10b3e0 con 0x7f9b9c0d1860 2021-02-10T11:44:16.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9b9b7fe700 1 -- 172.21.2.101:0/3399418083 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f9b84001590 con 0x7f9b9c0d1860 2021-02-10T11:44:16.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9b9b7fe700 1 -- 172.21.2.101:0/3399418083 >> v1:172.21.2.101:6789/0 conn(0x7f9b9c0d1860 legacy=0x7f9b9c1b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:16.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9b9b7fe700 1 --2- 172.21.2.101:0/3399418083 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b88003af0 0x7f9b88006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:16.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9b9b7fe700 1 --2- 172.21.2.101:0/3399418083 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b880067a0 0x7f9b88008cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:16.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9b9b7fe700 1 --2- 172.21.2.101:0/3399418083 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b880095c0 0x7f9b8800ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:16.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9b9b7fe700 1 -- 172.21.2.101:0/3399418083 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8800c2c0 con 0x7f9b88003af0 2021-02-10T11:44:16.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9b9b7fe700 1 -- 172.21.2.101:0/3399418083 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8800c4e0 con 0x7f9b880067a0 2021-02-10T11:44:16.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9b9b7fe700 1 -- 172.21.2.101:0/3399418083 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8800c700 con 0x7f9b880095c0 2021-02-10T11:44:16.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9ba0d52700 1 --2- 172.21.2.101:0/3399418083 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b88003af0 0x7f9b88006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:16.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9b9bfff700 1 --2- 172.21.2.101:0/3399418083 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b880067a0 0x7f9b88008cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:16.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9ba1553700 1 --2- 172.21.2.101:0/3399418083 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b880095c0 0x7f9b8800ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:16.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9ba0d52700 1 -- 172.21.2.101:0/3399418083 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b880095c0 msgr2=0x7f9b8800ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:16.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9ba0d52700 1 --2- 172.21.2.101:0/3399418083 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b880095c0 0x7f9b8800ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:16.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9ba0d52700 1 -- 172.21.2.101:0/3399418083 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b880067a0 msgr2=0x7f9b88008cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:16.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9ba0d52700 1 --2- 172.21.2.101:0/3399418083 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b880067a0 0x7f9b88008cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:16.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9ba0d52700 1 -- 172.21.2.101:0/3399418083 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b88050970 con 0x7f9b88003af0 2021-02-10T11:44:16.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9ba0d52700 1 --2- 172.21.2.101:0/3399418083 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b88003af0 0x7f9b88006010 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 rx=0x7f9b84007560 tx=0x7f9b84007d10).ready entity=mon.0 client_cookie=7b231dc29062e846 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:16.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9b9b7fe700 1 -- 172.21.2.101:0/3399418083 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9b840009b0 con 0x7f9b88003af0 2021-02-10T11:44:16.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9b9b7fe700 1 -- 172.21.2.101:0/3399418083 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b840013d0 con 0x7f9b88003af0 2021-02-10T11:44:16.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9b9b7fe700 1 -- 172.21.2.101:0/3399418083 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9b8400a710 con 0x7f9b88003af0 2021-02-10T11:44:16.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9ba2f36700 1 -- 172.21.2.101:0/3399418083 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b88003af0 msgr2=0x7f9b88006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:16.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9ba2f36700 1 --2- 172.21.2.101:0/3399418083 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b88003af0 0x7f9b88006010 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 rx=0x7f9b84007560 tx=0x7f9b84007d10).stop 2021-02-10T11:44:16.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.440 7f9ba1553700 1 -- 172.21.2.101:0/3399418083 reap_dead start 2021-02-10T11:44:16.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.441 7f9ba2f36700 1 -- 172.21.2.101:0/3399418083 shutdown_connections 2021-02-10T11:44:16.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.441 7f9ba2f36700 1 -- 172.21.2.101:0/3399418083 >> 172.21.2.101:0/3399418083 conn(0x7f9b9c0ce410 msgr2=0x7f9b9c0d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:16.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.441 7f9ba2f36700 1 -- 172.21.2.101:0/3399418083 shutdown_connections 2021-02-10T11:44:16.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.441 7f9ba2f36700 1 -- 172.21.2.101:0/3399418083 wait complete. 2021-02-10T11:44:16.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.456 7f9ba2f36700 1 Processor -- start 2021-02-10T11:44:16.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.456 7f9ba2f36700 1 -- start start 2021-02-10T11:44:16.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.456 7f9ba2f36700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b9c0adf50 0x7f9b9c0ae350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:16.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.456 7f9ba2f36700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b9c0ae970 0x7f9b9c0aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:16.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.456 7f9ba2f36700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b9c0982c0 0x7f9b9c0986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:16.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.456 7f9ba2f36700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c09f450 con 0x7f9b9c0adf50 2021-02-10T11:44:16.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.456 7f9ba2f36700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c098ce0 con 0x7f9b9c0ae970 2021-02-10T11:44:16.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.456 7f9ba2f36700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c098ed0 con 0x7f9b9c0982c0 2021-02-10T11:44:16.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.456 7f9ba0d52700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b9c0adf50 0x7f9b9c0ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:16.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.456 7f9b9bfff700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b9c0ae970 0x7f9b9c0aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:16.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.456 7f9ba0d52700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b9c0adf50 0x7f9b9c0ae350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59692/0 (socket says 172.21.2.101:59692) 2021-02-10T11:44:16.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.456 7f9ba0d52700 1 -- 172.21.2.101:0/3894619095 learned_addr learned my addr 172.21.2.101:0/3894619095 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:16.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.456 7f9ba1553700 1 --2- 172.21.2.101:0/3894619095 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b9c0982c0 0x7f9b9c0986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:16.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.457 7f9b9bfff700 1 -- 172.21.2.101:0/3894619095 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b9c0982c0 msgr2=0x7f9b9c0986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:16.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.457 7f9b9bfff700 1 --2- 172.21.2.101:0/3894619095 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b9c0982c0 0x7f9b9c0986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:16.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.457 7f9b9bfff700 1 -- 172.21.2.101:0/3894619095 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b9c0adf50 msgr2=0x7f9b9c0ae350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:16.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.457 7f9b9bfff700 1 --2- 172.21.2.101:0/3894619095 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b9c0adf50 0x7f9b9c0ae350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:16.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.457 7f9b9bfff700 1 -- 172.21.2.101:0/3894619095 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c0990f0 con 0x7f9b9c0ae970 2021-02-10T11:44:16.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.457 7f9b9bfff700 1 --2- 172.21.2.101:0/3894619095 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b9c0ae970 0x7f9b9c0aed70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f9b8c00b510 tx=0x7f9b8c00b8b0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:16.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.457 7f9b997fa700 1 -- 172.21.2.101:0/3894619095 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9b8c0063c0 con 0x7f9b9c0ae970 2021-02-10T11:44:16.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.457 7f9b997fa700 1 -- 172.21.2.101:0/3894619095 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8c00bca0 con 0x7f9b9c0ae970 2021-02-10T11:44:16.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.457 7f9ba2f36700 1 -- 172.21.2.101:0/3894619095 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b9c0990f0 con 0x7f9b9c0ae970 2021-02-10T11:44:16.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.457 7f9b997fa700 1 -- 172.21.2.101:0/3894619095 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9b8c0022b0 con 0x7f9b9c0ae970 2021-02-10T11:44:16.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.457 7f9ba2f36700 1 -- 172.21.2.101:0/3894619095 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9c0990f0 con 0x7f9b9c0ae970 2021-02-10T11:44:16.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.458 7f9b997fa700 1 -- 172.21.2.101:0/3894619095 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9b8c000ca0 con 0x7f9b9c0ae970 2021-02-10T11:44:16.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.458 7f9b997fa700 1 --2- 172.21.2.101:0/3894619095 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b80018730 0x7f9b8001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:16.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.458 7f9b997fa700 1 -- 172.21.2.101:0/3894619095 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9b8c032600 con 0x7f9b9c0ae970 2021-02-10T11:44:16.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.458 7f9ba0d52700 1 --2- 172.21.2.101:0/3894619095 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b80018730 0x7f9b8001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:16.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.458 7f9ba0d52700 1 --2- 172.21.2.101:0/3894619095 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b80018730 0x7f9b8001ac10 crc :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:16.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.488 7f9ba2f36700 1 -- 172.21.2.101:0/3894619095 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b9c082a20 con 0x7f9b9c0ae970 2021-02-10T11:44:16.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.491 7f9b997fa700 1 -- 172.21.2.101:0/3894619095 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9b8c006320 con 0x7f9b9c0ae970 2021-02-10T11:44:16.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.622 7f9ba2f36700 1 -- 172.21.2.101:0/3894619095 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f9b9c082a20 con 0x7f9b9c0ae970 2021-02-10T11:44:16.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.623 7f9b997fa700 1 -- 172.21.2.101:0/3894619095 <== mon.1 v2:172.21.2.108:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v147) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9b8c000ca0 con 0x7f9b9c0ae970 2021-02-10T11:44:16.624 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:44:16.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.626 7f9ba2f36700 1 -- 172.21.2.101:0/3894619095 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b80018730 msgr2=0x7f9b8001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:16.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.626 7f9ba2f36700 1 --2- 172.21.2.101:0/3894619095 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b80018730 0x7f9b8001ac10 crc :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:16.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.626 7f9ba2f36700 1 -- 172.21.2.101:0/3894619095 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b9c0ae970 msgr2=0x7f9b9c0aed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:16.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.626 7f9ba2f36700 1 --2- 172.21.2.101:0/3894619095 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b9c0ae970 0x7f9b9c0aed70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f9b8c00b510 tx=0x7f9b8c00b8b0).stop 2021-02-10T11:44:16.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.627 7f9ba2f36700 1 -- 172.21.2.101:0/3894619095 shutdown_connections 2021-02-10T11:44:16.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.627 7f9ba2f36700 1 --2- 172.21.2.101:0/3894619095 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9b80018730 0x7f9b8001ac10 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:16.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.627 7f9ba2f36700 1 --2- 172.21.2.101:0/3894619095 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9b9c0982c0 0x7f9b9c0986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:16.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.627 7f9ba2f36700 1 --2- 172.21.2.101:0/3894619095 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9b9c0ae970 0x7f9b9c0aed70 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:16.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.627 7f9ba2f36700 1 --2- 172.21.2.101:0/3894619095 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9b9c0adf50 0x7f9b9c0ae350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:16.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.627 7f9ba2f36700 1 -- 172.21.2.101:0/3894619095 >> 172.21.2.101:0/3894619095 conn(0x7f9b9c0bee90 msgr2=0x7f9b9c0bb880 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:16.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.627 7f9ba2f36700 1 -- 172.21.2.101:0/3894619095 shutdown_connections 2021-02-10T11:44:16.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:16.627 7f9ba2f36700 1 -- 172.21.2.101:0/3894619095 wait complete. 2021-02-10T11:44:16.635 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:44:17.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.052 7f07d6f87700 1 Processor -- start 2021-02-10T11:44:17.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.052 7f07d6f87700 1 -- start start 2021-02-10T11:44:17.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.052 7f07d6f87700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f07d00d1c70 0x7f07d00cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:17.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.052 7f07d6f87700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07d00cf580 0x7f07d00cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:17.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.052 7f07d6f87700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d00fe1c0 con 0x7f07d00d1870 2021-02-10T11:44:17.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.052 7f07d6f87700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d01b0cb0 con 0x7f07d00d1c70 2021-02-10T11:44:17.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.052 7f07d6f87700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d0109a60 con 0x7f07d00cf580 2021-02-10T11:44:17.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.053 7f07cffff700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f07d00d1c70 0x7f07d00cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:17.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.053 7f07d4da3700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f07d00d1870 0x7f07d01b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:53602/0 (socket says 172.21.2.101:53602) 2021-02-10T11:44:17.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.053 7f07d4da3700 1 -- 172.21.2.101:0/3730504157 learned_addr learned my addr 172.21.2.101:0/3730504157 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:17.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.053 7f07d55a4700 1 --2- 172.21.2.101:0/3730504157 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07d00cf580 0x7f07d00cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:17.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.053 7f07d55a4700 1 -- 172.21.2.101:0/3730504157 >> v2:172.21.2.108:3300/0 conn(0x7f07d00d1c70 msgr2=0x7f07d00cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:17.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.053 7f07d55a4700 1 --2- 172.21.2.101:0/3730504157 >> v2:172.21.2.108:3300/0 conn(0x7f07d00d1c70 0x7f07d00cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:17.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.053 7f07d55a4700 1 -- 172.21.2.101:0/3730504157 >> v1:172.21.2.108:6789/0 conn(0x7f07d00d1870 legacy=0x7f07d01b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:17.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.053 7f07d55a4700 1 -- 172.21.2.101:0/3730504157 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d0088070 con 0x7f07d00cf580 2021-02-10T11:44:17.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.053 7f07d55a4700 1 --2- 172.21.2.101:0/3730504157 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07d00cf580 0x7f07d00cf980 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f07c8006e50 tx=0x7f07c8009e70).ready entity=mon.2 client_cookie=587fa2e96543f55f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:17.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.054 7f07cf7fe700 1 -- 172.21.2.101:0/3730504157 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f07c8006a20 con 0x7f07d00cf580 2021-02-10T11:44:17.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.054 7f07cf7fe700 1 -- 172.21.2.101:0/3730504157 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c8006790 con 0x7f07d00cf580 2021-02-10T11:44:17.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.054 7f07cf7fe700 1 -- 172.21.2.101:0/3730504157 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f07c800b700 con 0x7f07d00cf580 2021-02-10T11:44:17.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.054 7f07d6f87700 1 -- 172.21.2.101:0/3730504157 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07d00cf580 msgr2=0x7f07d00cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:17.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.054 7f07d6f87700 1 --2- 172.21.2.101:0/3730504157 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07d00cf580 0x7f07d00cf980 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f07c8006e50 tx=0x7f07c8009e70).stop 2021-02-10T11:44:17.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.054 7f07d6f87700 1 -- 172.21.2.101:0/3730504157 shutdown_connections 2021-02-10T11:44:17.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.055 7f07d6f87700 1 --2- 172.21.2.101:0/3730504157 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07d00cf580 0x7f07d00cf980 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:17.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.055 7f07d6f87700 1 --2- 172.21.2.101:0/3730504157 >> v2:172.21.2.108:3300/0 conn(0x7f07d00d1c70 0x7f07d00cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:17.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.055 7f07d6f87700 1 -- 172.21.2.101:0/3730504157 >> 172.21.2.101:0/3730504157 conn(0x7f07d00ce420 msgr2=0x7f07d00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:17.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.055 7f07d6f87700 1 -- 172.21.2.101:0/3730504157 shutdown_connections 2021-02-10T11:44:17.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.055 7f07d6f87700 1 -- 172.21.2.101:0/3730504157 wait complete. 2021-02-10T11:44:17.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.069 7f07d6f87700 1 Processor -- start 2021-02-10T11:44:17.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.069 7f07d6f87700 1 -- start start 2021-02-10T11:44:17.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.070 7f07d6f87700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07d0191b10 0x7f07d00ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:17.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.070 7f07d6f87700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07d0191f10 0x7f07d019ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:17.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.070 7f07d6f87700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07d01a05f0 0x7f07d0082ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:17.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.070 7f07d6f87700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d0192440 con 0x7f07d0191f10 2021-02-10T11:44:17.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.070 7f07d6f87700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d01916e0 con 0x7f07d01a05f0 2021-02-10T11:44:17.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.070 7f07d6f87700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d0191900 con 0x7f07d0191b10 2021-02-10T11:44:17.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.070 7f07cffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07d0191f10 0x7f07d019ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:17.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.070 7f07cffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07d0191f10 0x7f07d019ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59708/0 (socket says 172.21.2.101:59708) 2021-02-10T11:44:17.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.070 7f07cffff700 1 -- 172.21.2.101:0/2551720118 learned_addr learned my addr 172.21.2.101:0/2551720118 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:17.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.070 7f07d55a4700 1 --2- 172.21.2.101:0/2551720118 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07d01a05f0 0x7f07d0082ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:17.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.070 7f07cffff700 1 -- 172.21.2.101:0/2551720118 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07d0191b10 msgr2=0x7f07d00ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:17.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.070 7f07d4da3700 1 --2- 172.21.2.101:0/2551720118 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07d0191b10 0x7f07d00ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:17.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.070 7f07cffff700 1 --2- 172.21.2.101:0/2551720118 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07d0191b10 0x7f07d00ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:17.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.070 7f07cffff700 1 -- 172.21.2.101:0/2551720118 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07d01a05f0 msgr2=0x7f07d0082ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:17.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.070 7f07cffff700 1 --2- 172.21.2.101:0/2551720118 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07d01a05f0 0x7f07d0082ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:17.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.070 7f07cffff700 1 -- 172.21.2.101:0/2551720118 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d00834f0 con 0x7f07d0191f10 2021-02-10T11:44:17.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.070 7f07d55a4700 1 --2- 172.21.2.101:0/2551720118 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07d01a05f0 0x7f07d0082ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:44:17.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.070 7f07d4da3700 1 --2- 172.21.2.101:0/2551720118 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07d0191b10 0x7f07d00ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:44:17.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.071 7f07cffff700 1 --2- 172.21.2.101:0/2551720118 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07d0191f10 0x7f07d019ffd0 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 rx=0x7f07c0006f60 tx=0x7f07c000b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:17.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.071 7f07cd7fa700 1 -- 172.21.2.101:0/2551720118 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f07c000bd60 con 0x7f07d0191f10 2021-02-10T11:44:17.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.071 7f07cd7fa700 1 -- 172.21.2.101:0/2551720118 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c0000c90 con 0x7f07d0191f10 2021-02-10T11:44:17.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.071 7f07cd7fa700 1 -- 172.21.2.101:0/2551720118 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f07c0002220 con 0x7f07d0191f10 2021-02-10T11:44:17.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.071 7f07d6f87700 1 -- 172.21.2.101:0/2551720118 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d00834f0 con 0x7f07d0191f10 2021-02-10T11:44:17.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.071 7f07d6f87700 1 -- 172.21.2.101:0/2551720118 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d0083a80 con 0x7f07d0191f10 2021-02-10T11:44:17.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.072 7f07cd7fa700 1 -- 172.21.2.101:0/2551720118 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f07c00052e0 con 0x7f07d0191f10 2021-02-10T11:44:17.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.072 7f07cd7fa700 1 --2- 172.21.2.101:0/2551720118 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07b40186c0 0x7f07b401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:17.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.072 7f07cd7fa700 1 -- 172.21.2.101:0/2551720118 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f07c00325b0 con 0x7f07d0191f10 2021-02-10T11:44:17.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.072 7f07d4da3700 1 --2- 172.21.2.101:0/2551720118 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07b40186c0 0x7f07b401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:17.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.072 7f07d4da3700 1 --2- 172.21.2.101:0/2551720118 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07b40186c0 0x7f07b401abc0 crc :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:17.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.102 7f07d6f87700 1 -- 172.21.2.101:0/2551720118 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d00fe1c0 con 0x7f07d0191f10 2021-02-10T11:44:17.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.105 7f07cd7fa700 1 -- 172.21.2.101:0/2551720118 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f07c0000960 con 0x7f07d0191f10 2021-02-10T11:44:17.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.231 7f07d6f87700 1 -- 172.21.2.101:0/2551720118 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f07d00fe1c0 con 0x7f07d0191f10 2021-02-10T11:44:17.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.231 7f07cd7fa700 1 -- 172.21.2.101:0/2551720118 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v147) v1 ==== 104+0+10 (secure 0 0 0) 0x7f07c00052e0 con 0x7f07d0191f10 2021-02-10T11:44:17.231 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:44:17.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.233 7f07d6f87700 1 -- 172.21.2.101:0/2551720118 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07b40186c0 msgr2=0x7f07b401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:17.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.233 7f07d6f87700 1 --2- 172.21.2.101:0/2551720118 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07b40186c0 0x7f07b401abc0 crc :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:17.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.233 7f07d6f87700 1 -- 172.21.2.101:0/2551720118 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07d0191f10 msgr2=0x7f07d019ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:17.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.233 7f07d6f87700 1 --2- 172.21.2.101:0/2551720118 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07d0191f10 0x7f07d019ffd0 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 rx=0x7f07c0006f60 tx=0x7f07c000b970).stop 2021-02-10T11:44:17.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.233 7f07d6f87700 1 -- 172.21.2.101:0/2551720118 shutdown_connections 2021-02-10T11:44:17.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.233 7f07d6f87700 1 --2- 172.21.2.101:0/2551720118 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f07d01a05f0 0x7f07d0082ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:17.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.233 7f07d6f87700 1 --2- 172.21.2.101:0/2551720118 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f07d0191f10 0x7f07d019ffd0 unknown :-1 s=CLOSED pgs=2672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:17.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.233 7f07d6f87700 1 --2- 172.21.2.101:0/2551720118 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07b40186c0 0x7f07b401abc0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:17.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.233 7f07d6f87700 1 --2- 172.21.2.101:0/2551720118 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f07d0191b10 0x7f07d00ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:17.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.233 7f07d6f87700 1 -- 172.21.2.101:0/2551720118 >> 172.21.2.101:0/2551720118 conn(0x7f07d00a02d0 msgr2=0x7f07d009f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:17.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.234 7f07d6f87700 1 -- 172.21.2.101:0/2551720118 shutdown_connections 2021-02-10T11:44:17.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.234 7f07d6f87700 1 -- 172.21.2.101:0/2551720118 wait complete. 2021-02-10T11:44:17.241 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:44:17.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.658 7fc572b1d700 1 Processor -- start 2021-02-10T11:44:17.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.658 7fc572b1d700 1 -- start start 2021-02-10T11:44:17.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.658 7fc572b1d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc56c0d1870 0x7fc56c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:17.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.658 7fc572b1d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc56c1b0ae0 0x7fc56c108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:17.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.658 7fc572b1d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc56c0d1c70 0x7fc56c108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:17.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.658 7fc572b1d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc56c0e5ef0 con 0x7fc56c1b0ae0 2021-02-10T11:44:17.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.658 7fc572b1d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc56c109a60 con 0x7fc56c0d1c70 2021-02-10T11:44:17.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.658 7fc572b1d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c088070 con 0x7fc56c0d1870 2021-02-10T11:44:17.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.659 7fc56bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc56c1b0ae0 0x7fc56c108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:17.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.659 7fc56bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc56c1b0ae0 0x7fc56c108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59714/0 (socket says 172.21.2.101:59714) 2021-02-10T11:44:17.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.659 7fc56bfff700 1 -- 172.21.2.101:0/472690680 learned_addr learned my addr 172.21.2.101:0/472690680 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:17.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.659 7fc56bfff700 1 -- 172.21.2.101:0/472690680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc56c0d1870 msgr2=0x7fc56c1b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:17.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.659 7fc56bfff700 1 --2- 172.21.2.101:0/472690680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc56c0d1870 0x7fc56c1b04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:17.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.659 7fc56bfff700 1 -- 172.21.2.101:0/472690680 >> v2:172.21.2.108:3300/0 conn(0x7fc56c0d1c70 msgr2=0x7fc56c108f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:17.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.659 7fc56bfff700 1 --2- 172.21.2.101:0/472690680 >> v2:172.21.2.108:3300/0 conn(0x7fc56c0d1c70 0x7fc56c108f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:17.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.659 7fc56bfff700 1 -- 172.21.2.101:0/472690680 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56c109580 con 0x7fc56c1b0ae0 2021-02-10T11:44:17.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.659 7fc56bfff700 1 --2- 172.21.2.101:0/472690680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc56c1b0ae0 0x7fc56c108940 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 rx=0x7fc55c006a80 tx=0x7fc55c009e80).ready entity=mon.0 client_cookie=4d4003ec96847db3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:17.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.659 7fc56b7fe700 1 -- 172.21.2.101:0/472690680 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc55c00b510 con 0x7fc56c1b0ae0 2021-02-10T11:44:17.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.659 7fc56b7fe700 1 -- 172.21.2.101:0/472690680 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55c00b7d0 con 0x7fc56c1b0ae0 2021-02-10T11:44:17.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.659 7fc56b7fe700 1 -- 172.21.2.101:0/472690680 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc55c00bc20 con 0x7fc56c1b0ae0 2021-02-10T11:44:17.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.659 7fc572b1d700 1 -- 172.21.2.101:0/472690680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc56c1b0ae0 msgr2=0x7fc56c108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:17.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.659 7fc572b1d700 1 --2- 172.21.2.101:0/472690680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc56c1b0ae0 0x7fc56c108940 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 rx=0x7fc55c006a80 tx=0x7fc55c009e80).stop 2021-02-10T11:44:17.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.660 7fc572b1d700 1 -- 172.21.2.101:0/472690680 shutdown_connections 2021-02-10T11:44:17.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.660 7fc572b1d700 1 --2- 172.21.2.101:0/472690680 >> v2:172.21.2.108:3300/0 conn(0x7fc56c0d1c70 0x7fc56c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:17.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.660 7fc572b1d700 1 --2- 172.21.2.101:0/472690680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc56c1b0ae0 0x7fc56c108940 unknown :-1 s=CLOSED pgs=2673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:17.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.660 7fc572b1d700 1 --2- 172.21.2.101:0/472690680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc56c0d1870 0x7fc56c1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:17.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.660 7fc572b1d700 1 -- 172.21.2.101:0/472690680 >> 172.21.2.101:0/472690680 conn(0x7fc56c0ce420 msgr2=0x7fc56c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:17.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.660 7fc572b1d700 1 -- 172.21.2.101:0/472690680 shutdown_connections 2021-02-10T11:44:17.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.660 7fc572b1d700 1 -- 172.21.2.101:0/472690680 wait complete. 2021-02-10T11:44:17.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.675 7fc572b1d700 1 Processor -- start 2021-02-10T11:44:17.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.675 7fc572b1d700 1 -- start start 2021-02-10T11:44:17.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.675 7fc572b1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc56c0872c0 0x7fc56c1a7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:17.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.675 7fc572b1d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc56c1a7ba0 0x7fc56c0a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:17.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.675 7fc572b1d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc56c1a7fa0 0x7fc56c0a6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:17.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.675 7fc572b1d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c086e20 con 0x7fc56c0872c0 2021-02-10T11:44:17.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.675 7fc572b1d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c190c60 con 0x7fc56c1a7ba0 2021-02-10T11:44:17.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.675 7fc572b1d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c0a74a0 con 0x7fc56c1a7fa0 2021-02-10T11:44:17.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.675 7fc570939700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc56c0872c0 0x7fc56c1a7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:17.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.675 7fc570939700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc56c0872c0 0x7fc56c1a7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59716/0 (socket says 172.21.2.101:59716) 2021-02-10T11:44:17.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.675 7fc570939700 1 -- 172.21.2.101:0/56602132 learned_addr learned my addr 172.21.2.101:0/56602132 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:17.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.675 7fc56bfff700 1 --2- 172.21.2.101:0/56602132 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc56c1a7ba0 0x7fc56c0a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:17.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.675 7fc57113a700 1 --2- 172.21.2.101:0/56602132 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc56c1a7fa0 0x7fc56c0a6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:17.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.676 7fc570939700 1 -- 172.21.2.101:0/56602132 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc56c1a7fa0 msgr2=0x7fc56c0a6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:17.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.676 7fc570939700 1 --2- 172.21.2.101:0/56602132 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc56c1a7fa0 0x7fc56c0a6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:17.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.676 7fc570939700 1 -- 172.21.2.101:0/56602132 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc56c1a7ba0 msgr2=0x7fc56c0a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:17.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.676 7fc570939700 1 --2- 172.21.2.101:0/56602132 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc56c1a7ba0 0x7fc56c0a6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:17.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.676 7fc570939700 1 -- 172.21.2.101:0/56602132 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56c0a76c0 con 0x7fc56c0872c0 2021-02-10T11:44:17.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.676 7fc570939700 1 --2- 172.21.2.101:0/56602132 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc56c0872c0 0x7fc56c1a7580 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 rx=0x7fc55400b5c0 tx=0x7fc55400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:17.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.676 7fc5697fa700 1 -- 172.21.2.101:0/56602132 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc55400bd50 con 0x7fc56c0872c0 2021-02-10T11:44:17.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.676 7fc5697fa700 1 -- 172.21.2.101:0/56602132 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc554006030 con 0x7fc56c0872c0 2021-02-10T11:44:17.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.676 7fc572b1d700 1 -- 172.21.2.101:0/56602132 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56c0a76c0 con 0x7fc56c0872c0 2021-02-10T11:44:17.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.676 7fc5697fa700 1 -- 172.21.2.101:0/56602132 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc5540054d0 con 0x7fc56c0872c0 2021-02-10T11:44:17.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.676 7fc572b1d700 1 -- 172.21.2.101:0/56602132 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc56c0b0090 con 0x7fc56c0872c0 2021-02-10T11:44:17.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.677 7fc5697fa700 1 -- 172.21.2.101:0/56602132 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc554006210 con 0x7fc56c0872c0 2021-02-10T11:44:17.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.677 7fc5697fa700 1 --2- 172.21.2.101:0/56602132 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5500186c0 0x7fc55001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:17.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.677 7fc5697fa700 1 -- 172.21.2.101:0/56602132 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc5540326d0 con 0x7fc56c0872c0 2021-02-10T11:44:17.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.677 7fc56bfff700 1 --2- 172.21.2.101:0/56602132 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5500186c0 0x7fc55001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:17.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.677 7fc56bfff700 1 --2- 172.21.2.101:0/56602132 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5500186c0 0x7fc55001abc0 crc :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:17.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.708 7fc572b1d700 1 -- 172.21.2.101:0/56602132 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc56c0a76c0 con 0x7fc56c0872c0 2021-02-10T11:44:17.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.711 7fc5697fa700 1 -- 172.21.2.101:0/56602132 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc554000940 con 0x7fc56c0872c0 2021-02-10T11:44:17.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.838 7fc572b1d700 1 -- 172.21.2.101:0/56602132 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7fc56c0a76c0 con 0x7fc56c0872c0 2021-02-10T11:44:17.838 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.839 7fc5697fa700 1 -- 172.21.2.101:0/56602132 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v147) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc554006030 con 0x7fc56c0872c0 2021-02-10T11:44:17.838 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:44:17.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.840 7fc572b1d700 1 -- 172.21.2.101:0/56602132 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5500186c0 msgr2=0x7fc55001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:17.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.840 7fc572b1d700 1 --2- 172.21.2.101:0/56602132 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5500186c0 0x7fc55001abc0 crc :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:17.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.840 7fc572b1d700 1 -- 172.21.2.101:0/56602132 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc56c0872c0 msgr2=0x7fc56c1a7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:17.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.840 7fc572b1d700 1 --2- 172.21.2.101:0/56602132 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc56c0872c0 0x7fc56c1a7580 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 rx=0x7fc55400b5c0 tx=0x7fc55400b960).stop 2021-02-10T11:44:17.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.841 7fc572b1d700 1 -- 172.21.2.101:0/56602132 shutdown_connections 2021-02-10T11:44:17.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.841 7fc572b1d700 1 --2- 172.21.2.101:0/56602132 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5500186c0 0x7fc55001abc0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.841 7fc572b1d700 1 --2- 172.21.2.101:0/56602132 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc56c1a7fa0 0x7fc56c0a6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.841 7fc572b1d700 1 --2- 172.21.2.101:0/56602132 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc56c1a7ba0 0x7fc56c0a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.841 7fc572b1d700 1 --2- 172.21.2.101:0/56602132 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc56c0872c0 0x7fc56c1a7580 unknown :-1 s=CLOSED pgs=2674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.841 7fc572b1d700 1 -- 172.21.2.101:0/56602132 >> 172.21.2.101:0/56602132 conn(0x7fc56c0be980 msgr2=0x7fc56c092cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.841 7fc572b1d700 1 -- 172.21.2.101:0/56602132 shutdown_connections 2021-02-10T11:44:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:17.841 7fc572b1d700 1 -- 172.21.2.101:0/56602132 wait complete. 2021-02-10T11:44:17.848 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:44:17.848 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:44:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.261 7fcb89494700 1 Processor -- start 2021-02-10T11:44:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.261 7fcb89494700 1 -- start start 2021-02-10T11:44:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.262 7fcb89494700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcb840d0340 0x7fcb840ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.262 7fcb89494700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb840f0b10 con 0x7fcb840d1c70 2021-02-10T11:44:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.262 7fcb89494700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb841b0be0 con 0x7fcb840d1870 2021-02-10T11:44:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.262 7fcb89494700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb840e5ed0 con 0x7fcb840d0340 2021-02-10T11:44:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.262 7fcb8387e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fcb840d1c70 0x7fcb840efc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60222/0 (socket says 172.21.2.101:60222) 2021-02-10T11:44:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.262 7fcb8387e700 1 -- 172.21.2.101:0/2170105725 learned_addr learned my addr 172.21.2.101:0/2170105725 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.262 7fcb8287c700 1 --2- 172.21.2.101:0/2170105725 >> v2:172.21.2.101:3300/0 conn(0x7fcb840d0340 0x7fcb840ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.262 7fcb8207b700 1 -- 172.21.2.101:0/2170105725 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3816632951 0 0) 0x7fcb7c001500 con 0x7fcb840d1c70 2021-02-10T11:44:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.262 7fcb8207b700 1 -- 172.21.2.101:0/2170105725 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb70002fe0 con 0x7fcb840d1c70 2021-02-10T11:44:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.262 7fcb8287c700 1 -- 172.21.2.101:0/2170105725 >> v1:172.21.2.108:6789/0 conn(0x7fcb840d1870 legacy=0x7fcb841b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.262 7fcb8287c700 1 -- 172.21.2.101:0/2170105725 >> v1:172.21.2.101:6789/0 conn(0x7fcb840d1c70 legacy=0x7fcb840efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.262 7fcb8287c700 1 -- 172.21.2.101:0/2170105725 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb840f0450 con 0x7fcb840d0340 2021-02-10T11:44:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.263 7fcb8287c700 1 --2- 172.21.2.101:0/2170105725 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb840d0340 0x7fcb840ef640 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 rx=0x7fcb6c006650 tx=0x7fcb6c009a80).ready entity=mon.0 client_cookie=69a5d8cf67e48f18 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.263 7fcb8207b700 1 -- 172.21.2.101:0/2170105725 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcb6c00c0c0 con 0x7fcb840d0340 2021-02-10T11:44:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.263 7fcb8207b700 1 -- 172.21.2.101:0/2170105725 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6c00c340 con 0x7fcb840d0340 2021-02-10T11:44:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.263 7fcb8207b700 1 -- 172.21.2.101:0/2170105725 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcb6c00c860 con 0x7fcb840d0340 2021-02-10T11:44:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.263 7fcb89494700 1 -- 172.21.2.101:0/2170105725 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb840d0340 msgr2=0x7fcb840ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.263 7fcb89494700 1 --2- 172.21.2.101:0/2170105725 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb840d0340 0x7fcb840ef640 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 rx=0x7fcb6c006650 tx=0x7fcb6c009a80).stop 2021-02-10T11:44:18.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.263 7fcb89494700 1 -- 172.21.2.101:0/2170105725 shutdown_connections 2021-02-10T11:44:18.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.263 7fcb89494700 1 --2- 172.21.2.101:0/2170105725 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb840d0340 0x7fcb840ef640 unknown :-1 s=CLOSED pgs=2676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:18.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.263 7fcb89494700 1 -- 172.21.2.101:0/2170105725 >> 172.21.2.101:0/2170105725 conn(0x7fcb840ce450 msgr2=0x7fcb840d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:18.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.263 7fcb89494700 1 -- 172.21.2.101:0/2170105725 shutdown_connections 2021-02-10T11:44:18.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.263 7fcb89494700 1 -- 172.21.2.101:0/2170105725 wait complete. 2021-02-10T11:44:18.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.278 7fcb89494700 1 Processor -- start 2021-02-10T11:44:18.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.278 7fcb89494700 1 -- start start 2021-02-10T11:44:18.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.278 7fcb89494700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb840b7eb0 0x7fcb840b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:18.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.278 7fcb89494700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb84091e70 0x7fcb84092270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:18.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.278 7fcb89494700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcb84092890 0x7fcb841aa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:18.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.278 7fcb89494700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb841ae790 con 0x7fcb84091e70 2021-02-10T11:44:18.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.278 7fcb89494700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb84092c90 con 0x7fcb84092890 2021-02-10T11:44:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.278 7fcb89494700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb841aa9c0 con 0x7fcb840b7eb0 2021-02-10T11:44:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.278 7fcb8287c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb84091e70 0x7fcb84092270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.278 7fcb8287c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb84091e70 0x7fcb84092270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59730/0 (socket says 172.21.2.101:59730) 2021-02-10T11:44:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.278 7fcb8287c700 1 -- 172.21.2.101:0/2725361216 learned_addr learned my addr 172.21.2.101:0/2725361216 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.279 7fcb8287c700 1 -- 172.21.2.101:0/2725361216 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb840b7eb0 msgr2=0x7fcb840b82b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.279 7fcb8287c700 1 --2- 172.21.2.101:0/2725361216 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb840b7eb0 0x7fcb840b82b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.279 7fcb8287c700 1 -- 172.21.2.101:0/2725361216 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcb84092890 msgr2=0x7fcb841aa3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.279 7fcb8287c700 1 --2- 172.21.2.101:0/2725361216 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcb84092890 0x7fcb841aa3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.279 7fcb8287c700 1 -- 172.21.2.101:0/2725361216 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb841aabe0 con 0x7fcb84091e70 2021-02-10T11:44:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.279 7fcb8287c700 1 --2- 172.21.2.101:0/2725361216 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb84091e70 0x7fcb84092270 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 rx=0x7fcb6c009650 tx=0x7fcb6c00b8e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.279 7fcb7bfff700 1 -- 172.21.2.101:0/2725361216 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcb6c0093a0 con 0x7fcb84091e70 2021-02-10T11:44:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.279 7fcb7bfff700 1 -- 172.21.2.101:0/2725361216 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6c006100 con 0x7fcb84091e70 2021-02-10T11:44:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.279 7fcb89494700 1 -- 172.21.2.101:0/2725361216 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb841aabe0 con 0x7fcb84091e70 2021-02-10T11:44:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.279 7fcb89494700 1 -- 172.21.2.101:0/2725361216 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb841ab170 con 0x7fcb84091e70 2021-02-10T11:44:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.279 7fcb7bfff700 1 -- 172.21.2.101:0/2725361216 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcb6c0052e0 con 0x7fcb84091e70 2021-02-10T11:44:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.280 7fcb7bfff700 1 -- 172.21.2.101:0/2725361216 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fcb6c0052e0 con 0x7fcb84091e70 2021-02-10T11:44:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.280 7fcb7bfff700 1 --2- 172.21.2.101:0/2725361216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcb68018700 0x7fcb6801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.280 7fcb7bfff700 1 -- 172.21.2.101:0/2725361216 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fcb6c032630 con 0x7fcb84091e70 2021-02-10T11:44:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.280 7fcb8307d700 1 --2- 172.21.2.101:0/2725361216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcb68018700 0x7fcb6801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.280 7fcb8307d700 1 --2- 172.21.2.101:0/2725361216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcb68018700 0x7fcb6801abc0 crc :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:18.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.312 7fcb89494700 1 -- 172.21.2.101:0/2725361216 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb840ee080 con 0x7fcb84091e70 2021-02-10T11:44:18.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.314 7fcb7bfff700 1 -- 172.21.2.101:0/2725361216 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fcb6c0093a0 con 0x7fcb84091e70 2021-02-10T11:44:18.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.443 7fcb89494700 1 -- 172.21.2.101:0/2725361216 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb840ce450 con 0x7fcb84091e70 2021-02-10T11:44:18.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.443 7fcb7bfff700 1 -- 172.21.2.101:0/2725361216 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+2111 (secure 0 0 0) 0x7fcb6c009300 con 0x7fcb84091e70 2021-02-10T11:44:18.443 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:44:18.443 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":123,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/4208244473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":4208244473},{"type":"v1","addr":"172.21.2.101:6835","nonce":4208244473}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":119}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:44:08.462960","modified":"2021-02-10 11:44:10.472350","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11814":{"gid":11814,"name":"b","rank":0,"incarnation":120,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/3445995728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3445995728},{"type":"v1","addr":"172.21.2.108:6833","nonce":3445995728}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-02-10T11:44:18.443 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 123 2021-02-10T11:44:18.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.445 7fcb89494700 1 -- 172.21.2.101:0/2725361216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcb68018700 msgr2=0x7fcb6801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:18.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.445 7fcb89494700 1 --2- 172.21.2.101:0/2725361216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcb68018700 0x7fcb6801abc0 crc :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:18.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.445 7fcb89494700 1 -- 172.21.2.101:0/2725361216 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb84091e70 msgr2=0x7fcb84092270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:18.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.445 7fcb89494700 1 --2- 172.21.2.101:0/2725361216 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb84091e70 0x7fcb84092270 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 rx=0x7fcb6c009650 tx=0x7fcb6c00b8e0).stop 2021-02-10T11:44:18.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.445 7fcb89494700 1 -- 172.21.2.101:0/2725361216 shutdown_connections 2021-02-10T11:44:18.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.445 7fcb89494700 1 --2- 172.21.2.101:0/2725361216 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcb84092890 0x7fcb841aa3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:18.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.445 7fcb89494700 1 --2- 172.21.2.101:0/2725361216 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb84091e70 0x7fcb84092270 unknown :-1 s=CLOSED pgs=2677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:18.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.445 7fcb89494700 1 --2- 172.21.2.101:0/2725361216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcb68018700 0x7fcb6801abc0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:18.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.445 7fcb89494700 1 --2- 172.21.2.101:0/2725361216 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb840b7eb0 0x7fcb840b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:18.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.445 7fcb89494700 1 -- 172.21.2.101:0/2725361216 >> 172.21.2.101:0/2725361216 conn(0x7fcb840ee2b0 msgr2=0x7fcb840ee6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:18.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.445 7fcb89494700 1 -- 172.21.2.101:0/2725361216 shutdown_connections 2021-02-10T11:44:18.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:18.445 7fcb89494700 1 -- 172.21.2.101:0/2725361216 wait complete. 2021-02-10T11:44:18.654 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 123, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:44:08.462960', 'modified': '2021-02-10 11:44:10.472350', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11814}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11814': {'gid': 11814, 'name': 'b', 'rank': 0, 'incarnation': 120, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/3445995728', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 3445995728}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 3445995728}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [31], 'metadata_pool': 30, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:44:18.655 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:44:18.655 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:44:19.052 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:44:19.052 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:44:19.052 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:44:19.052 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 11814, 2021-02-10T11:44:19.053 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:44:19.053 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:44:19.053 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 10.588908083, 2021-02-10T11:44:19.053 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 123, 2021-02-10T11:44:19.053 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 147, 2021-02-10T11:44:19.053 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 147, 2021-02-10T11:44:19.053 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 16.233263008000002 2021-02-10T11:44:19.053 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:44:19.056 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 11814, "want_state": "up:active", "state": "up:active", "rank_uptime": 10.588908083, "mdsmap_epoch": 123, "osdmap_epoch": 147, "osdmap_epoch_barrier": 147, "uptime": 16.233263008000002 } 2021-02-10T11:44:19.057 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:44:19.057 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-02-10T11:44:19.057 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-02-10T11:44:19.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:19.057 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:44:19.393 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:44:19.738 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:44:19.738 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:44:20.082 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:44:20.082 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:44:20.083 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:44:20.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:20.083 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:44:20.418 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-02-10T11:44:20.418 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:44:20.420 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:44:20.764 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:44:20.764 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:44:20.831 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[162446]: starting ceph client 2021-02-10T11:44:20.832 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:2021-02-10 11:44:20.832 7f3550d70f80 -1 init, newargv = 0x55e1bf7d0180 newargc=9 2021-02-10T11:44:20.837 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[162446]: starting fuse 2021-02-10T11:44:21.107 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:44:21.107 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:44:21.108 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:44:21.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:21.108 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:44:21.443 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:44:21.444 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2021-02-10T11:44:21.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:21.444 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:44:21.444 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:44:21.444 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:44:21.444 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:44:21.444 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:44:21.444 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:44:21.444 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:44:21.444 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-02-10T11:44:21.445 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:44:21.445 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:44:21.445 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:44:21.445 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:44:21.445 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:44:21.445 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:44:21.445 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:44:21.445 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:44:21.445 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:44:21.445 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:44:21.445 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:44:21.445 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:44:21.445 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:44:21.445 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.0")) 2021-02-10T11:44:21.445 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:44:21.808 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.0.162446.asok 2021-02-10T11:44:21.811 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.162446.asok 2021-02-10T11:44:21.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:21.811 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.162446.asok status 2021-02-10T11:44:22.200 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:44:22.201 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:44:22.201 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:44:22.201 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:44:22.201 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "0", 2021-02-10T11:44:22.201 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:44:22.201 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-02-10T11:44:22.201 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "162446", 2021-02-10T11:44:22.201 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:44:22.201 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:22.201 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:44:22.201 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:44:22.201 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 11970, 2021-02-10T11:44:22.202 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:44:22.202 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:44:22.202 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:44:22.202 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 11970 2021-02-10T11:44:22.202 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:22.202 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:44:22.202 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:22.202 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:44:22.202 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1816311599 2021-02-10T11:44:22.202 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:22.202 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:22.202 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:44:22.203 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:22.203 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:44:22.203 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1816311599 2021-02-10T11:44:22.203 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:22.203 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.11970 172.21.2.101:0/1816311599", 2021-02-10T11:44:22.203 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/1816311599", 2021-02-10T11:44:22.203 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:44:22.203 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 123, 2021-02-10T11:44:22.203 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 147, 2021-02-10T11:44:22.203 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:44:22.203 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:44:22.203 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:44:22.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:22.205 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:44:22.205 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:44:22.540 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:44:22.540 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:44:22.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:22.540 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:44:22.885 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:44:23.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.296 7f6b44e6d700 1 Processor -- start 2021-02-10T11:44:23.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.296 7f6b44e6d700 1 -- start start 2021-02-10T11:44:23.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.297 7f6b44e6d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6b400d1870 0x7f6b401b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:23.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.297 7f6b44e6d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6b400f0650 0x7f6b400cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:23.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.297 7f6b44e6d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b400f0b10 con 0x7f6b401b0a10 2021-02-10T11:44:23.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.297 7f6b44e6d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b400e5ed0 con 0x7f6b400d1870 2021-02-10T11:44:23.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.297 7f6b44e6d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b400d1c70 con 0x7f6b400f0650 2021-02-10T11:44:23.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.297 7f6b3eb13700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6b400d1870 0x7f6b401b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:23.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.297 7f6b3f314700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6b400f0650 0x7f6b400cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:23.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.297 7f6b3eb13700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6b400d1870 0x7f6b401b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59758/0 (socket says 172.21.2.101:59758) 2021-02-10T11:44:23.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.297 7f6b3f314700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6b400f0650 0x7f6b400cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:42014/0 (socket says 172.21.2.101:42014) 2021-02-10T11:44:23.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.297 7f6b3eb13700 1 -- 172.21.2.101:0/4058884190 learned_addr learned my addr 172.21.2.101:0/4058884190 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:23.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.297 7f6b3e312700 1 --1- 172.21.2.101:0/4058884190 >> v1:172.21.2.108:6789/0 conn(0x7f6b401b0a10 0x7f6b400cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:53662/0 (socket says 172.21.2.101:53662) 2021-02-10T11:44:23.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.298 7f6b3f314700 1 -- 172.21.2.101:0/4058884190 >> v2:172.21.2.101:3300/0 conn(0x7f6b400d1870 msgr2=0x7f6b401b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:23.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.298 7f6b3f314700 1 --2- 172.21.2.101:0/4058884190 >> v2:172.21.2.101:3300/0 conn(0x7f6b400d1870 0x7f6b401b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:23.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.298 7f6b3f314700 1 -- 172.21.2.101:0/4058884190 >> v1:172.21.2.108:6789/0 conn(0x7f6b401b0a10 legacy=0x7f6b400cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:23.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.298 7f6b3f314700 1 -- 172.21.2.101:0/4058884190 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b400d1e90 con 0x7f6b400f0650 2021-02-10T11:44:23.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.298 7f6b3eb13700 1 --2- 172.21.2.101:0/4058884190 >> v2:172.21.2.101:3300/0 conn(0x7f6b400d1870 0x7f6b401b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:44:23.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.298 7f6b3f314700 1 --2- 172.21.2.101:0/4058884190 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b400f0650 0x7f6b400cfb20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f6b38006e50 tx=0x7f6b38009e70).ready entity=mon.1 client_cookie=531a3a3df163ebc3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:23.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.298 7f6b3db11700 1 -- 172.21.2.101:0/4058884190 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6b38006a20 con 0x7f6b400f0650 2021-02-10T11:44:23.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.298 7f6b3db11700 1 -- 172.21.2.101:0/4058884190 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b38006790 con 0x7f6b400f0650 2021-02-10T11:44:23.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.298 7f6b3db11700 1 -- 172.21.2.101:0/4058884190 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6b3800b700 con 0x7f6b400f0650 2021-02-10T11:44:23.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.298 7f6b44e6d700 1 -- 172.21.2.101:0/4058884190 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b400f0650 msgr2=0x7f6b400cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:23.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.298 7f6b44e6d700 1 --2- 172.21.2.101:0/4058884190 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b400f0650 0x7f6b400cfb20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f6b38006e50 tx=0x7f6b38009e70).stop 2021-02-10T11:44:23.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.299 7f6b44e6d700 1 -- 172.21.2.101:0/4058884190 shutdown_connections 2021-02-10T11:44:23.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.299 7f6b44e6d700 1 --2- 172.21.2.101:0/4058884190 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b400f0650 0x7f6b400cfb20 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:23.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.299 7f6b44e6d700 1 --2- 172.21.2.101:0/4058884190 >> v2:172.21.2.101:3300/0 conn(0x7f6b400d1870 0x7f6b401b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:23.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.299 7f6b44e6d700 1 -- 172.21.2.101:0/4058884190 >> 172.21.2.101:0/4058884190 conn(0x7f6b400ce450 msgr2=0x7f6b400d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:23.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.299 7f6b44e6d700 1 -- 172.21.2.101:0/4058884190 shutdown_connections 2021-02-10T11:44:23.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.299 7f6b44e6d700 1 -- 172.21.2.101:0/4058884190 wait complete. 2021-02-10T11:44:23.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.314 7f6b44e6d700 1 Processor -- start 2021-02-10T11:44:23.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.314 7f6b44e6d700 1 -- start start 2021-02-10T11:44:23.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.314 7f6b44e6d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b40195650 0x7f6b400b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:23.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.314 7f6b44e6d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b400b5b80 0x7f6b401a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:23.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.314 7f6b44e6d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b400acfa0 0x7f6b401a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:23.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.314 7f6b44e6d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b400b6010 con 0x7f6b400acfa0 2021-02-10T11:44:23.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.314 7f6b44e6d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b401a3a60 con 0x7f6b40195650 2021-02-10T11:44:23.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.314 7f6b44e6d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b401a3c40 con 0x7f6b400b5b80 2021-02-10T11:44:23.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.314 7f6b3f314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b400acfa0 0x7f6b401a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:23.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.314 7f6b3e312700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b400b5b80 0x7f6b401a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:23.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.314 7f6b3eb13700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b40195650 0x7f6b400b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:23.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.314 7f6b3eb13700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b40195650 0x7f6b400b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:42016/0 (socket says 172.21.2.101:42016) 2021-02-10T11:44:23.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.314 7f6b3eb13700 1 -- 172.21.2.101:0/3872335276 learned_addr learned my addr 172.21.2.101:0/3872335276 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:23.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.314 7f6b3f314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b400acfa0 0x7f6b401a3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59766/0 (socket says 172.21.2.101:59766) 2021-02-10T11:44:23.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.315 7f6b3eb13700 1 -- 172.21.2.101:0/3872335276 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b400b5b80 msgr2=0x7f6b401a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:23.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.315 7f6b3eb13700 1 --2- 172.21.2.101:0/3872335276 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b400b5b80 0x7f6b401a2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:23.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.315 7f6b3eb13700 1 -- 172.21.2.101:0/3872335276 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b400acfa0 msgr2=0x7f6b401a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:23.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.315 7f6b3eb13700 1 --2- 172.21.2.101:0/3872335276 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b400acfa0 0x7f6b401a3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:23.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.315 7f6b3eb13700 1 -- 172.21.2.101:0/3872335276 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b400bcae0 con 0x7f6b40195650 2021-02-10T11:44:23.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.315 7f6b3eb13700 1 --2- 172.21.2.101:0/3872335276 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b40195650 0x7f6b400b5560 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f6b3000b5e0 tx=0x7f6b3000b980).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:23.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.315 7f6b377fe700 1 -- 172.21.2.101:0/3872335276 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6b3000bd70 con 0x7f6b40195650 2021-02-10T11:44:23.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.315 7f6b377fe700 1 -- 172.21.2.101:0/3872335276 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b30006030 con 0x7f6b40195650 2021-02-10T11:44:23.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.315 7f6b377fe700 1 -- 172.21.2.101:0/3872335276 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6b30005560 con 0x7f6b40195650 2021-02-10T11:44:23.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.315 7f6b44e6d700 1 -- 172.21.2.101:0/3872335276 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b400bcae0 con 0x7f6b40195650 2021-02-10T11:44:23.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.315 7f6b44e6d700 1 -- 172.21.2.101:0/3872335276 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b400bcae0 con 0x7f6b40195650 2021-02-10T11:44:23.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.316 7f6b377fe700 1 -- 172.21.2.101:0/3872335276 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6b300052e0 con 0x7f6b40195650 2021-02-10T11:44:23.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.316 7f6b377fe700 1 --2- 172.21.2.101:0/3872335276 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b24018730 0x7f6b2401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:23.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.316 7f6b377fe700 1 -- 172.21.2.101:0/3872335276 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6b300327b0 con 0x7f6b40195650 2021-02-10T11:44:23.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.316 7f6b3e312700 1 --2- 172.21.2.101:0/3872335276 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b24018730 0x7f6b2401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:23.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.316 7f6b3e312700 1 --2- 172.21.2.101:0/3872335276 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b24018730 0x7f6b2401abc0 crc :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:23.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.347 7f6b44e6d700 1 -- 172.21.2.101:0/3872335276 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b400b6010 con 0x7f6b40195650 2021-02-10T11:44:23.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.351 7f6b377fe700 1 -- 172.21.2.101:0/3872335276 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6b30006030 con 0x7f6b40195650 2021-02-10T11:44:23.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.479 7f6b44e6d700 1 -- 172.21.2.101:0/3872335276 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b400b6010 con 0x7f6b40195650 2021-02-10T11:44:23.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.480 7f6b377fe700 1 -- 172.21.2.101:0/3872335276 <== mon.1 v2:172.21.2.108:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f6b30006240 con 0x7f6b40195650 2021-02-10T11:44:23.481 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:44:23.482 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":147,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:44:09.465154","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:44:06.431537","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":31,"pool_name":"cephfs_data","create_time":"2021-02-10 11:44:07.456920","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:44:23.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.483 7f6b44e6d700 1 -- 172.21.2.101:0/3872335276 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b24018730 msgr2=0x7f6b2401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:23.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.483 7f6b44e6d700 1 --2- 172.21.2.101:0/3872335276 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b24018730 0x7f6b2401abc0 crc :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:23.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.483 7f6b44e6d700 1 -- 172.21.2.101:0/3872335276 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b40195650 msgr2=0x7f6b400b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:23.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.483 7f6b44e6d700 1 --2- 172.21.2.101:0/3872335276 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b40195650 0x7f6b400b5560 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f6b3000b5e0 tx=0x7f6b3000b980).stop 2021-02-10T11:44:23.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.484 7f6b44e6d700 1 -- 172.21.2.101:0/3872335276 shutdown_connections 2021-02-10T11:44:23.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.484 7f6b44e6d700 1 --2- 172.21.2.101:0/3872335276 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6b24018730 0x7f6b2401abc0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:23.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.484 7f6b44e6d700 1 --2- 172.21.2.101:0/3872335276 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6b400b5b80 0x7f6b401a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:23.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.484 7f6b44e6d700 1 --2- 172.21.2.101:0/3872335276 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6b400acfa0 0x7f6b401a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:23.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.484 7f6b44e6d700 1 --2- 172.21.2.101:0/3872335276 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6b40195650 0x7f6b400b5560 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:23.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.484 7f6b44e6d700 1 -- 172.21.2.101:0/3872335276 >> 172.21.2.101:0/3872335276 conn(0x7f6b400b3040 msgr2=0x7f6b400c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:23.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.484 7f6b44e6d700 1 -- 172.21.2.101:0/3872335276 shutdown_connections 2021-02-10T11:44:23.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.484 7f6b44e6d700 1 -- 172.21.2.101:0/3872335276 wait complete. 2021-02-10T11:44:23.491 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 30, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:44:06.431537', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '146', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 31, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:44:07.456920', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:44:23.491 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:44:23.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.908 7f1da15c8700 1 Processor -- start 2021-02-10T11:44:23.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.908 7f1da15c8700 1 -- start start 2021-02-10T11:44:23.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.909 7f1da15c8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1d9c0d1870 0x7f1d9c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:23.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.909 7f1da15c8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d9c1b0ae0 0x7f1d9c108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:23.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.909 7f1da15c8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d9c0fe1c0 con 0x7f1d9c0d1c70 2021-02-10T11:44:23.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.909 7f1da15c8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d9c109a60 con 0x7f1d9c0d1870 2021-02-10T11:44:23.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.909 7f1da15c8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c109750 con 0x7f1d9c1b0ae0 2021-02-10T11:44:23.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.909 7f1d9aabe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d9c1b0ae0 0x7f1d9c108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:23.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.909 7f1d9b2bf700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1d9c0d1870 0x7f1d9c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:23.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.909 7f1d9bac0700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f1d9c0d1c70 0x7f1d9c108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:53674/0 (socket says 172.21.2.101:53674) 2021-02-10T11:44:23.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.909 7f1d9bac0700 1 -- 172.21.2.101:0/962647690 learned_addr learned my addr 172.21.2.101:0/962647690 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:23.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.909 7f1d9b2bf700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1d9c0d1870 0x7f1d9c1b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:42024/0 (socket says 172.21.2.101:42024) 2021-02-10T11:44:23.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.909 7f1d9aabe700 1 -- 172.21.2.101:0/962647690 >> v2:172.21.2.108:3300/0 conn(0x7f1d9c0d1870 msgr2=0x7f1d9c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:23.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.909 7f1d9aabe700 1 --2- 172.21.2.101:0/962647690 >> v2:172.21.2.108:3300/0 conn(0x7f1d9c0d1870 0x7f1d9c1b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:23.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.909 7f1d9aabe700 1 -- 172.21.2.101:0/962647690 >> v1:172.21.2.108:6789/0 conn(0x7f1d9c0d1c70 legacy=0x7f1d9c108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:23.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.909 7f1d9aabe700 1 -- 172.21.2.101:0/962647690 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c088070 con 0x7f1d9c1b0ae0 2021-02-10T11:44:23.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.909 7f1d9b2bf700 1 --2- 172.21.2.101:0/962647690 >> v2:172.21.2.108:3300/0 conn(0x7f1d9c0d1870 0x7f1d9c1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:44:23.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.910 7f1d9aabe700 1 --2- 172.21.2.101:0/962647690 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d9c1b0ae0 0x7f1d9c108940 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f1d8c006650 tx=0x7f1d8c009a80).ready entity=mon.2 client_cookie=4480a421a12de30c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:23.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.910 7f1d9a2bd700 1 -- 172.21.2.101:0/962647690 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1d8c00c0c0 con 0x7f1d9c1b0ae0 2021-02-10T11:44:23.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.910 7f1d9a2bd700 1 -- 172.21.2.101:0/962647690 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c00c340 con 0x7f1d9c1b0ae0 2021-02-10T11:44:23.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.910 7f1d9a2bd700 1 -- 172.21.2.101:0/962647690 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1d8c00c860 con 0x7f1d9c1b0ae0 2021-02-10T11:44:23.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.910 7f1da15c8700 1 -- 172.21.2.101:0/962647690 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d9c1b0ae0 msgr2=0x7f1d9c108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:23.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.910 7f1da15c8700 1 --2- 172.21.2.101:0/962647690 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d9c1b0ae0 0x7f1d9c108940 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f1d8c006650 tx=0x7f1d8c009a80).stop 2021-02-10T11:44:23.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.910 7f1da15c8700 1 -- 172.21.2.101:0/962647690 shutdown_connections 2021-02-10T11:44:23.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.910 7f1da15c8700 1 --2- 172.21.2.101:0/962647690 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d9c1b0ae0 0x7f1d9c108940 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:23.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.910 7f1da15c8700 1 --2- 172.21.2.101:0/962647690 >> v2:172.21.2.108:3300/0 conn(0x7f1d9c0d1870 0x7f1d9c1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:23.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.910 7f1da15c8700 1 -- 172.21.2.101:0/962647690 >> 172.21.2.101:0/962647690 conn(0x7f1d9c0ce420 msgr2=0x7f1d9c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:23.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.911 7f1da15c8700 1 -- 172.21.2.101:0/962647690 shutdown_connections 2021-02-10T11:44:23.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.911 7f1da15c8700 1 -- 172.21.2.101:0/962647690 wait complete. 2021-02-10T11:44:23.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.925 7f1da15c8700 1 Processor -- start 2021-02-10T11:44:23.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.925 7f1da15c8700 1 -- start start 2021-02-10T11:44:23.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.925 7f1da15c8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d9c191b10 0x7f1d9c0ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:23.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.925 7f1da15c8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d9c191f10 0x7f1d9c19ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:23.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.925 7f1da15c8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1d9c1a05f0 0x7f1d9c082ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:23.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.925 7f1da15c8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c192440 con 0x7f1d9c191b10 2021-02-10T11:44:23.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.925 7f1da15c8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c1916e0 con 0x7f1d9c1a05f0 2021-02-10T11:44:23.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.925 7f1da15c8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c191900 con 0x7f1d9c191f10 2021-02-10T11:44:23.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.926 7f1d9aabe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d9c191f10 0x7f1d9c19ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:23.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.926 7f1d9b2bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d9c191b10 0x7f1d9c0ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:23.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.926 7f1d9b2bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d9c191b10 0x7f1d9c0ae640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59774/0 (socket says 172.21.2.101:59774) 2021-02-10T11:44:23.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.926 7f1d9aabe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d9c191f10 0x7f1d9c19ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:52506/0 (socket says 172.21.2.101:52506) 2021-02-10T11:44:23.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.926 7f1d9b2bf700 1 -- 172.21.2.101:0/444849399 learned_addr learned my addr 172.21.2.101:0/444849399 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:23.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.926 7f1d9bac0700 1 --2- 172.21.2.101:0/444849399 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1d9c1a05f0 0x7f1d9c082ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:23.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.926 7f1d9b2bf700 1 -- 172.21.2.101:0/444849399 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d9c191f10 msgr2=0x7f1d9c19ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:23.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.926 7f1d9b2bf700 1 --2- 172.21.2.101:0/444849399 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d9c191f10 0x7f1d9c19ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:23.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.926 7f1d9b2bf700 1 -- 172.21.2.101:0/444849399 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1d9c1a05f0 msgr2=0x7f1d9c082ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:23.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.926 7f1d9b2bf700 1 --2- 172.21.2.101:0/444849399 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1d9c1a05f0 0x7f1d9c082ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:23.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.926 7f1d9b2bf700 1 -- 172.21.2.101:0/444849399 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c0834f0 con 0x7f1d9c191b10 2021-02-10T11:44:23.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.926 7f1d9b2bf700 1 --2- 172.21.2.101:0/444849399 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d9c191b10 0x7f1d9c0ae640 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 rx=0x7f1d84006f40 tx=0x7f1d8400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:23.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.926 7f1d93fff700 1 -- 172.21.2.101:0/444849399 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1d840009b0 con 0x7f1d9c191b10 2021-02-10T11:44:23.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.926 7f1d93fff700 1 -- 172.21.2.101:0/444849399 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8400bd80 con 0x7f1d9c191b10 2021-02-10T11:44:23.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.926 7f1d93fff700 1 -- 172.21.2.101:0/444849399 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1d840022b0 con 0x7f1d9c191b10 2021-02-10T11:44:23.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.926 7f1da15c8700 1 -- 172.21.2.101:0/444849399 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9c0834f0 con 0x7f1d9c191b10 2021-02-10T11:44:23.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.926 7f1da15c8700 1 -- 172.21.2.101:0/444849399 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9c0834f0 con 0x7f1d9c191b10 2021-02-10T11:44:23.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.927 7f1d93fff700 1 -- 172.21.2.101:0/444849399 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1d8400a000 con 0x7f1d9c191b10 2021-02-10T11:44:23.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.927 7f1d93fff700 1 --2- 172.21.2.101:0/444849399 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d80018730 0x7f1d8001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:23.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.927 7f1d93fff700 1 -- 172.21.2.101:0/444849399 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1d84032620 con 0x7f1d9c191b10 2021-02-10T11:44:23.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.927 7f1d9aabe700 1 --2- 172.21.2.101:0/444849399 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d80018730 0x7f1d8001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:23.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.927 7f1d9aabe700 1 --2- 172.21.2.101:0/444849399 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d80018730 0x7f1d8001abc0 crc :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:23.958 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.959 7f1da15c8700 1 -- 172.21.2.101:0/444849399 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9c192440 con 0x7f1d9c191b10 2021-02-10T11:44:23.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:23.961 7f1d93fff700 1 -- 172.21.2.101:0/444849399 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1d84000c90 con 0x7f1d9c191b10 2021-02-10T11:44:24.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.090 7f1da15c8700 1 -- 172.21.2.101:0/444849399 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f1d9c192440 con 0x7f1d9c191b10 2021-02-10T11:44:24.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.090 7f1d93fff700 1 -- 172.21.2.101:0/444849399 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v147) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1d8400bd80 con 0x7f1d9c191b10 2021-02-10T11:44:24.090 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:44:24.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.092 7f1da15c8700 1 -- 172.21.2.101:0/444849399 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d80018730 msgr2=0x7f1d8001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:24.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.092 7f1da15c8700 1 --2- 172.21.2.101:0/444849399 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d80018730 0x7f1d8001abc0 crc :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:24.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.092 7f1da15c8700 1 -- 172.21.2.101:0/444849399 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d9c191b10 msgr2=0x7f1d9c0ae640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:24.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.092 7f1da15c8700 1 --2- 172.21.2.101:0/444849399 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d9c191b10 0x7f1d9c0ae640 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 rx=0x7f1d84006f40 tx=0x7f1d8400b990).stop 2021-02-10T11:44:24.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.093 7f1da15c8700 1 -- 172.21.2.101:0/444849399 shutdown_connections 2021-02-10T11:44:24.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.093 7f1da15c8700 1 --2- 172.21.2.101:0/444849399 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d80018730 0x7f1d8001abc0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:24.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.093 7f1da15c8700 1 --2- 172.21.2.101:0/444849399 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d9c191f10 0x7f1d9c19ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:24.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.093 7f1da15c8700 1 --2- 172.21.2.101:0/444849399 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1d9c1a05f0 0x7f1d9c082ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:24.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.093 7f1da15c8700 1 --2- 172.21.2.101:0/444849399 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d9c191b10 0x7f1d9c0ae640 unknown :-1 s=CLOSED pgs=2681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:24.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.093 7f1da15c8700 1 -- 172.21.2.101:0/444849399 >> 172.21.2.101:0/444849399 conn(0x7f1d9c0a02d0 msgr2=0x7f1d9c09f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:24.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.093 7f1da15c8700 1 -- 172.21.2.101:0/444849399 shutdown_connections 2021-02-10T11:44:24.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.093 7f1da15c8700 1 -- 172.21.2.101:0/444849399 wait complete. 2021-02-10T11:44:24.100 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:44:24.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.517 7f23abf00700 1 Processor -- start 2021-02-10T11:44:24.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.517 7f23abf00700 1 -- start start 2021-02-10T11:44:24.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.517 7f23abf00700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f23a40d1870 0x7f23a41b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:24.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.517 7f23abf00700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f23a40cff70 0x7f23a4109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:24.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.517 7f23abf00700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a4109a60 con 0x7f23a41b0ae0 2021-02-10T11:44:24.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.517 7f23abf00700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a4109750 con 0x7f23a40cff70 2021-02-10T11:44:24.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.517 7f23abf00700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a4088070 con 0x7f23a40d1870 2021-02-10T11:44:24.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.519 7f23a951b700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f23a41b0ae0 0x7f23a4108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:53688/0 (socket says 172.21.2.101:53688) 2021-02-10T11:44:24.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.519 7f23aa51d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f23a40cff70 0x7f23a4109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:24.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.519 7f23a951b700 1 -- 172.21.2.101:0/192128081 learned_addr learned my addr 172.21.2.101:0/192128081 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:24.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.519 7f23a9d1c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f23a40d1870 0x7f23a41b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:24.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.519 7f23aa51d700 1 -- 172.21.2.101:0/192128081 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f23a40d1870 msgr2=0x7f23a41b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:24.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.519 7f23aa51d700 1 --2- 172.21.2.101:0/192128081 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f23a40d1870 0x7f23a41b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:24.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.519 7f23aa51d700 1 -- 172.21.2.101:0/192128081 >> v1:172.21.2.108:6789/0 conn(0x7f23a41b0ae0 legacy=0x7f23a4108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:24.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.519 7f23a8d1a700 1 -- 172.21.2.101:0/192128081 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2015662326 0 0) 0x7f23980009b0 con 0x7f23a41b0ae0 2021-02-10T11:44:24.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.519 7f23aa51d700 1 -- 172.21.2.101:0/192128081 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a4088260 con 0x7f23a40cff70 2021-02-10T11:44:24.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.519 7f23a9d1c700 1 --2- 172.21.2.101:0/192128081 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f23a40d1870 0x7f23a41b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:44:24.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.520 7f23aa51d700 1 --2- 172.21.2.101:0/192128081 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f23a40cff70 0x7f23a4109130 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f239c006e50 tx=0x7f239c009e70).ready entity=mon.1 client_cookie=db063448fc8fe941 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:24.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.520 7f23a8d1a700 1 -- 172.21.2.101:0/192128081 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f239c006a20 con 0x7f23a40cff70 2021-02-10T11:44:24.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.520 7f23a8d1a700 1 -- 172.21.2.101:0/192128081 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239c006790 con 0x7f23a40cff70 2021-02-10T11:44:24.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.520 7f23a8d1a700 1 -- 172.21.2.101:0/192128081 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f239c00b700 con 0x7f23a40cff70 2021-02-10T11:44:24.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.520 7f23abf00700 1 -- 172.21.2.101:0/192128081 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f23a40cff70 msgr2=0x7f23a4109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:24.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.520 7f23abf00700 1 --2- 172.21.2.101:0/192128081 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f23a40cff70 0x7f23a4109130 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f239c006e50 tx=0x7f239c009e70).stop 2021-02-10T11:44:24.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.521 7f23abf00700 1 -- 172.21.2.101:0/192128081 shutdown_connections 2021-02-10T11:44:24.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.521 7f23abf00700 1 --2- 172.21.2.101:0/192128081 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f23a40cff70 0x7f23a4109130 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:24.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.521 7f23abf00700 1 --2- 172.21.2.101:0/192128081 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f23a40d1870 0x7f23a41b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:24.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.521 7f23abf00700 1 -- 172.21.2.101:0/192128081 >> 172.21.2.101:0/192128081 conn(0x7f23a40ce420 msgr2=0x7f23a40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:24.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.521 7f23abf00700 1 -- 172.21.2.101:0/192128081 shutdown_connections 2021-02-10T11:44:24.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.521 7f23abf00700 1 -- 172.21.2.101:0/192128081 wait complete. 2021-02-10T11:44:24.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.535 7f23abf00700 1 Processor -- start 2021-02-10T11:44:24.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.535 7f23abf00700 1 -- start start 2021-02-10T11:44:24.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.536 7f23abf00700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f23a40abd60 0x7f23a40ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:24.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.536 7f23abf00700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f23a40ac780 0x7f23a40acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:24.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.536 7f23abf00700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f23a4083910 0x7f23a4083d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:24.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.536 7f23abf00700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a4087180 con 0x7f23a4083910 2021-02-10T11:44:24.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.536 7f23abf00700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a418fcd0 con 0x7f23a40abd60 2021-02-10T11:44:24.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.536 7f23abf00700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a4084330 con 0x7f23a40ac780 2021-02-10T11:44:24.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.536 7f23aa51d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f23a4083910 0x7f23a4083d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:24.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.536 7f23aa51d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f23a4083910 0x7f23a4083d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59788/0 (socket says 172.21.2.101:59788) 2021-02-10T11:44:24.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.536 7f23aa51d700 1 -- 172.21.2.101:0/1041379584 learned_addr learned my addr 172.21.2.101:0/1041379584 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:24.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.536 7f23aa51d700 1 -- 172.21.2.101:0/1041379584 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f23a40ac780 msgr2=0x7f23a40acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:24.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.536 7f23aa51d700 1 --2- 172.21.2.101:0/1041379584 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f23a40ac780 0x7f23a40acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:24.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.536 7f23aa51d700 1 -- 172.21.2.101:0/1041379584 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f23a40abd60 msgr2=0x7f23a40ac160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:24.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.536 7f23aa51d700 1 --2- 172.21.2.101:0/1041379584 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f23a40abd60 0x7f23a40ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:24.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.536 7f23aa51d700 1 -- 172.21.2.101:0/1041379584 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a4084550 con 0x7f23a4083910 2021-02-10T11:44:24.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.537 7f23aa51d700 1 --2- 172.21.2.101:0/1041379584 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f23a4083910 0x7f23a4083d10 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 rx=0x7f239c006e60 tx=0x7f239c00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:24.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.537 7f23967fc700 1 -- 172.21.2.101:0/1041379584 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f239c009130 con 0x7f23a4083910 2021-02-10T11:44:24.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.537 7f23967fc700 1 -- 172.21.2.101:0/1041379584 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239c009430 con 0x7f23a4083910 2021-02-10T11:44:24.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.537 7f23967fc700 1 -- 172.21.2.101:0/1041379584 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f239c001420 con 0x7f23a4083910 2021-02-10T11:44:24.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.537 7f23abf00700 1 -- 172.21.2.101:0/1041379584 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a4084550 con 0x7f23a4083910 2021-02-10T11:44:24.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.537 7f23abf00700 1 -- 172.21.2.101:0/1041379584 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a4084550 con 0x7f23a4083910 2021-02-10T11:44:24.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.537 7f23967fc700 1 -- 172.21.2.101:0/1041379584 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f239c009130 con 0x7f23a4083910 2021-02-10T11:44:24.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.537 7f23967fc700 1 --2- 172.21.2.101:0/1041379584 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f238c018730 0x7f238c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:24.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.537 7f23967fc700 1 -- 172.21.2.101:0/1041379584 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f239c0329c0 con 0x7f23a4083910 2021-02-10T11:44:24.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.537 7f23a9d1c700 1 --2- 172.21.2.101:0/1041379584 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f238c018730 0x7f238c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:24.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.538 7f23a9d1c700 1 --2- 172.21.2.101:0/1041379584 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f238c018730 0x7f238c01ac10 crc :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:24.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.567 7f23abf00700 1 -- 172.21.2.101:0/1041379584 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a4082980 con 0x7f23a4083910 2021-02-10T11:44:24.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.570 7f23967fc700 1 -- 172.21.2.101:0/1041379584 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f239c009430 con 0x7f23a4083910 2021-02-10T11:44:24.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.698 7f23abf00700 1 -- 172.21.2.101:0/1041379584 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f23a4109a60 con 0x7f23a4083910 2021-02-10T11:44:24.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.699 7f23967fc700 1 -- 172.21.2.101:0/1041379584 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v147) v1 ==== 104+0+10 (secure 0 0 0) 0x7f239c009430 con 0x7f23a4083910 2021-02-10T11:44:24.698 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:44:24.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.701 7f23abf00700 1 -- 172.21.2.101:0/1041379584 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f238c018730 msgr2=0x7f238c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:24.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.701 7f23abf00700 1 --2- 172.21.2.101:0/1041379584 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f238c018730 0x7f238c01ac10 crc :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:24.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.701 7f23abf00700 1 -- 172.21.2.101:0/1041379584 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f23a4083910 msgr2=0x7f23a4083d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:24.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.701 7f23abf00700 1 --2- 172.21.2.101:0/1041379584 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f23a4083910 0x7f23a4083d10 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 rx=0x7f239c006e60 tx=0x7f239c00bcf0).stop 2021-02-10T11:44:24.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.701 7f23abf00700 1 -- 172.21.2.101:0/1041379584 shutdown_connections 2021-02-10T11:44:24.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.701 7f23abf00700 1 --2- 172.21.2.101:0/1041379584 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f238c018730 0x7f238c01ac10 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:24.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.701 7f23abf00700 1 --2- 172.21.2.101:0/1041379584 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f23a40ac780 0x7f23a40acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:24.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.701 7f23abf00700 1 --2- 172.21.2.101:0/1041379584 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f23a4083910 0x7f23a4083d10 unknown :-1 s=CLOSED pgs=2682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:24.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.701 7f23abf00700 1 --2- 172.21.2.101:0/1041379584 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f23a40abd60 0x7f23a40ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:24.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.701 7f23abf00700 1 -- 172.21.2.101:0/1041379584 >> 172.21.2.101:0/1041379584 conn(0x7f23a40a2b30 msgr2=0x7f23a409ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:24.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.701 7f23abf00700 1 -- 172.21.2.101:0/1041379584 shutdown_connections 2021-02-10T11:44:24.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:24.701 7f23abf00700 1 -- 172.21.2.101:0/1041379584 wait complete. 2021-02-10T11:44:24.709 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:44:25.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.123 7fbd83087700 1 Processor -- start 2021-02-10T11:44:25.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.123 7fbd83087700 1 -- start start 2021-02-10T11:44:25.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.124 7fbd83087700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd7c0d1c70 0x7fbd7c0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:25.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.124 7fbd83087700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbd7c0cf580 0x7fbd7c0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.124 7fbd83087700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd7c0fe1c0 con 0x7fbd7c0d1870 2021-02-10T11:44:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.124 7fbd83087700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd7c1b0cb0 con 0x7fbd7c0cf580 2021-02-10T11:44:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.124 7fbd83087700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c109a60 con 0x7fbd7c0d1c70 2021-02-10T11:44:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.124 7fbd80ea3700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fbd7c0d1870 0x7fbd7c1b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60290/0 (socket says 172.21.2.101:60290) 2021-02-10T11:44:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.124 7fbd80ea3700 1 -- 172.21.2.101:0/195108279 learned_addr learned my addr 172.21.2.101:0/195108279 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.124 7fbd7b7fe700 1 -- 172.21.2.101:0/195108279 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 732207190 0 0) 0x7fbd640010b0 con 0x7fbd7c0d1870 2021-02-10T11:44:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.124 7fbd7b7fe700 1 -- 172.21.2.101:0/195108279 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd68002fe0 con 0x7fbd7c0d1870 2021-02-10T11:44:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.125 7fbd7bfff700 1 --2- 172.21.2.101:0/195108279 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd7c0d1c70 0x7fbd7c0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.125 7fbd816a4700 1 --2- 172.21.2.101:0/195108279 >> v2:172.21.2.108:3300/0 conn(0x7fbd7c0cf580 0x7fbd7c0cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.125 7fbd7b7fe700 1 -- 172.21.2.101:0/195108279 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 452594080 0 0) 0x7fbd640010b0 con 0x7fbd7c0d1870 2021-02-10T11:44:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.125 7fbd7b7fe700 1 -- 172.21.2.101:0/195108279 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd7c0d1c70 msgr2=0x7fbd7c0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:25.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.125 7fbd7b7fe700 1 --2- 172.21.2.101:0/195108279 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd7c0d1c70 0x7fbd7c0cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.125 7fbd7b7fe700 1 -- 172.21.2.101:0/195108279 >> v2:172.21.2.108:3300/0 conn(0x7fbd7c0cf580 msgr2=0x7fbd7c0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.125 7fbd7b7fe700 1 --2- 172.21.2.101:0/195108279 >> v2:172.21.2.108:3300/0 conn(0x7fbd7c0cf580 0x7fbd7c0cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.125 7fbd7b7fe700 1 -- 172.21.2.101:0/195108279 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c088070 con 0x7fbd7c0d1870 2021-02-10T11:44:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.125 7fbd7b7fe700 1 -- 172.21.2.101:0/195108279 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fbd640015b0 con 0x7fbd7c0d1870 2021-02-10T11:44:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.125 7fbd7b7fe700 1 -- 172.21.2.101:0/195108279 >> v1:172.21.2.101:6789/0 conn(0x7fbd7c0d1870 legacy=0x7fbd7c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.125 7fbd7b7fe700 1 --2- 172.21.2.101:0/195108279 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbd68003af0 0x7fbd68006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.125 7fbd7b7fe700 1 --2- 172.21.2.101:0/195108279 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd68006830 0x7fbd68008d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.125 7fbd7b7fe700 1 --2- 172.21.2.101:0/195108279 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd680095a0 0x7fbd6800bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.125 7fbd7b7fe700 1 -- 172.21.2.101:0/195108279 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6800c370 con 0x7fbd680095a0 2021-02-10T11:44:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.125 7fbd7b7fe700 1 -- 172.21.2.101:0/195108279 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd68050560 con 0x7fbd68003af0 2021-02-10T11:44:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.125 7fbd7b7fe700 1 -- 172.21.2.101:0/195108279 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd68050780 con 0x7fbd68006830 2021-02-10T11:44:25.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.125 7fbd816a4700 1 --2- 172.21.2.101:0/195108279 >> v2:172.21.2.108:3300/0 conn(0x7fbd7c0cf580 0x7fbd7c0cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:44:25.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.125 7fbd7bfff700 1 --2- 172.21.2.101:0/195108279 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd68006830 0x7fbd68008d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:25.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.125 7fbd80ea3700 1 --2- 172.21.2.101:0/195108279 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbd68003af0 0x7fbd68006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:25.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.125 7fbd816a4700 1 --2- 172.21.2.101:0/195108279 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd680095a0 0x7fbd6800bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:25.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.126 7fbd816a4700 1 -- 172.21.2.101:0/195108279 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd68006830 msgr2=0x7fbd68008d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:25.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.126 7fbd816a4700 1 --2- 172.21.2.101:0/195108279 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd68006830 0x7fbd68008d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:25.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.126 7fbd816a4700 1 -- 172.21.2.101:0/195108279 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbd68003af0 msgr2=0x7fbd68006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:25.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.126 7fbd816a4700 1 --2- 172.21.2.101:0/195108279 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbd68003af0 0x7fbd68006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:25.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.126 7fbd816a4700 1 -- 172.21.2.101:0/195108279 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd68050a20 con 0x7fbd680095a0 2021-02-10T11:44:25.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.126 7fbd80ea3700 1 --2- 172.21.2.101:0/195108279 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbd68003af0 0x7fbd68006010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:44:25.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.126 7fbd816a4700 1 --2- 172.21.2.101:0/195108279 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd680095a0 0x7fbd6800bac0 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 rx=0x7fbd740068c0 tx=0x7fbd7400b300).ready entity=mon.0 client_cookie=ca9828ee6cf140be server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:25.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.126 7fbd7b7fe700 1 -- 172.21.2.101:0/195108279 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbd7400b6f0 con 0x7fbd680095a0 2021-02-10T11:44:25.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.126 7fbd7b7fe700 1 -- 172.21.2.101:0/195108279 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd74003030 con 0x7fbd680095a0 2021-02-10T11:44:25.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.126 7fbd7b7fe700 1 -- 172.21.2.101:0/195108279 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbd7400b970 con 0x7fbd680095a0 2021-02-10T11:44:25.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.126 7fbd83087700 1 -- 172.21.2.101:0/195108279 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd680095a0 msgr2=0x7fbd6800bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:25.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.126 7fbd83087700 1 --2- 172.21.2.101:0/195108279 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd680095a0 0x7fbd6800bac0 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 rx=0x7fbd740068c0 tx=0x7fbd7400b300).stop 2021-02-10T11:44:25.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.126 7fbd816a4700 1 -- 172.21.2.101:0/195108279 reap_dead start 2021-02-10T11:44:25.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.126 7fbd83087700 1 -- 172.21.2.101:0/195108279 shutdown_connections 2021-02-10T11:44:25.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.126 7fbd83087700 1 -- 172.21.2.101:0/195108279 >> 172.21.2.101:0/195108279 conn(0x7fbd7c0ce420 msgr2=0x7fbd7c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:25.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.126 7fbd83087700 1 -- 172.21.2.101:0/195108279 shutdown_connections 2021-02-10T11:44:25.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.126 7fbd83087700 1 -- 172.21.2.101:0/195108279 wait complete. 2021-02-10T11:44:25.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.141 7fbd83087700 1 Processor -- start 2021-02-10T11:44:25.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.141 7fbd83087700 1 -- start start 2021-02-10T11:44:25.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.142 7fbd83087700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd7c085bc0 0x7fbd7c0863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:25.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.142 7fbd83087700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd7c0954c0 0x7fbd7c096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:25.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.142 7fbd83087700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbd7c0958c0 0x7fbd7c096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:25.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.142 7fbd83087700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c095d80 con 0x7fbd7c085bc0 2021-02-10T11:44:25.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.142 7fbd83087700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c0bfa60 con 0x7fbd7c0958c0 2021-02-10T11:44:25.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.142 7fbd83087700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c085fc0 con 0x7fbd7c0954c0 2021-02-10T11:44:25.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.142 7fbd80ea3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd7c085bc0 0x7fbd7c0863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:25.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.142 7fbd80ea3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd7c085bc0 0x7fbd7c0863e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59808/0 (socket says 172.21.2.101:59808) 2021-02-10T11:44:25.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.142 7fbd80ea3700 1 -- 172.21.2.101:0/464426710 learned_addr learned my addr 172.21.2.101:0/464426710 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:25.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.142 7fbd80ea3700 1 -- 172.21.2.101:0/464426710 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd7c0954c0 msgr2=0x7fbd7c096300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:25.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.142 7fbd80ea3700 1 --2- 172.21.2.101:0/464426710 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd7c0954c0 0x7fbd7c096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:25.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.142 7fbd80ea3700 1 -- 172.21.2.101:0/464426710 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbd7c0958c0 msgr2=0x7fbd7c096920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:25.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.142 7fbd80ea3700 1 --2- 172.21.2.101:0/464426710 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbd7c0958c0 0x7fbd7c096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:25.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.142 7fbd80ea3700 1 -- 172.21.2.101:0/464426710 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c086c10 con 0x7fbd7c085bc0 2021-02-10T11:44:25.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.142 7fbd80ea3700 1 --2- 172.21.2.101:0/464426710 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd7c085bc0 0x7fbd7c0863e0 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 rx=0x7fbd64001be0 tx=0x7fbd6400a900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:25.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.142 7fbd797fa700 1 -- 172.21.2.101:0/464426710 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbd6400acf0 con 0x7fbd7c085bc0 2021-02-10T11:44:25.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.142 7fbd797fa700 1 -- 172.21.2.101:0/464426710 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd64004140 con 0x7fbd7c085bc0 2021-02-10T11:44:25.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.142 7fbd83087700 1 -- 172.21.2.101:0/464426710 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd7c086c10 con 0x7fbd7c085bc0 2021-02-10T11:44:25.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.142 7fbd797fa700 1 -- 172.21.2.101:0/464426710 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbd64004660 con 0x7fbd7c085bc0 2021-02-10T11:44:25.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.143 7fbd83087700 1 -- 172.21.2.101:0/464426710 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7c086c10 con 0x7fbd7c085bc0 2021-02-10T11:44:25.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.143 7fbd797fa700 1 -- 172.21.2.101:0/464426710 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbd64004320 con 0x7fbd7c085bc0 2021-02-10T11:44:25.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.143 7fbd797fa700 1 --2- 172.21.2.101:0/464426710 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd600186c0 0x7fbd6001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:25.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.143 7fbd797fa700 1 -- 172.21.2.101:0/464426710 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fbd64032780 con 0x7fbd7c085bc0 2021-02-10T11:44:25.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.143 7fbd7bfff700 1 --2- 172.21.2.101:0/464426710 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd600186c0 0x7fbd6001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:25.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.143 7fbd7bfff700 1 --2- 172.21.2.101:0/464426710 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd600186c0 0x7fbd6001abc0 crc :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:25.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.174 7fbd83087700 1 -- 172.21.2.101:0/464426710 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd7c0fe1c0 con 0x7fbd7c085bc0 2021-02-10T11:44:25.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.176 7fbd797fa700 1 -- 172.21.2.101:0/464426710 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbd64001120 con 0x7fbd7c085bc0 2021-02-10T11:44:25.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.303 7fbd83087700 1 -- 172.21.2.101:0/464426710 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7fbd7c0fe1c0 con 0x7fbd7c085bc0 2021-02-10T11:44:25.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.303 7fbd797fa700 1 -- 172.21.2.101:0/464426710 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v147) v1 ==== 100+0+10 (secure 0 0 0) 0x7fbd64004280 con 0x7fbd7c085bc0 2021-02-10T11:44:25.303 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:44:25.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.305 7fbd83087700 1 -- 172.21.2.101:0/464426710 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd600186c0 msgr2=0x7fbd6001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:25.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.305 7fbd83087700 1 --2- 172.21.2.101:0/464426710 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd600186c0 0x7fbd6001abc0 crc :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:25.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.305 7fbd83087700 1 -- 172.21.2.101:0/464426710 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd7c085bc0 msgr2=0x7fbd7c0863e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:25.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.305 7fbd83087700 1 --2- 172.21.2.101:0/464426710 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd7c085bc0 0x7fbd7c0863e0 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 rx=0x7fbd64001be0 tx=0x7fbd6400a900).stop 2021-02-10T11:44:25.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.305 7fbd83087700 1 -- 172.21.2.101:0/464426710 shutdown_connections 2021-02-10T11:44:25.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.305 7fbd83087700 1 --2- 172.21.2.101:0/464426710 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd600186c0 0x7fbd6001abc0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:25.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.305 7fbd83087700 1 --2- 172.21.2.101:0/464426710 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd7c0954c0 0x7fbd7c096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:25.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.305 7fbd83087700 1 --2- 172.21.2.101:0/464426710 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbd7c0958c0 0x7fbd7c096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:25.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.305 7fbd83087700 1 --2- 172.21.2.101:0/464426710 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd7c085bc0 0x7fbd7c0863e0 unknown :-1 s=CLOSED pgs=2685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:25.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.305 7fbd83087700 1 -- 172.21.2.101:0/464426710 >> 172.21.2.101:0/464426710 conn(0x7fbd7c0bef00 msgr2=0x7fbd7c09d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:25.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.305 7fbd83087700 1 -- 172.21.2.101:0/464426710 shutdown_connections 2021-02-10T11:44:25.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.305 7fbd83087700 1 -- 172.21.2.101:0/464426710 wait complete. 2021-02-10T11:44:25.313 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:44:25.313 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:44:25.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.726 7f19af04a700 1 Processor -- start 2021-02-10T11:44:25.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.726 7f19af04a700 1 -- start start 2021-02-10T11:44:25.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.727 7f19af04a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f19a81b0870 0x7f19a80f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:25.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.727 7f19af04a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19a80efe00 0x7f19a80f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:25.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.727 7f19af04a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19a81b0c70 con 0x7f19a81b0470 2021-02-10T11:44:25.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.727 7f19af04a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19a80f0b50 con 0x7f19a81b0870 2021-02-10T11:44:25.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.727 7f19af04a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a80f0820 con 0x7f19a80efe00 2021-02-10T11:44:25.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.727 7f19a7fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f19a81b0870 0x7f19a80f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:25.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.727 7f19a7fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f19a81b0870 0x7f19a80f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59816/0 (socket says 172.21.2.101:59816) 2021-02-10T11:44:25.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.727 7f19a7fff700 1 -- 172.21.2.101:0/1786841747 learned_addr learned my addr 172.21.2.101:0/1786841747 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:25.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.727 7f19ace66700 1 --1- 172.21.2.101:0/1786841747 >> v1:172.21.2.101:6789/0 conn(0x7f19a81b0470 0x7f19a80ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60310/0 (socket says 172.21.2.101:60310) 2021-02-10T11:44:25.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.727 7f19a7fff700 1 -- 172.21.2.101:0/1786841747 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19a80efe00 msgr2=0x7f19a80f0200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:25.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.727 7f19a7fff700 1 --2- 172.21.2.101:0/1786841747 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19a80efe00 0x7f19a80f0200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:25.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.727 7f19a7fff700 1 -- 172.21.2.101:0/1786841747 >> v1:172.21.2.101:6789/0 conn(0x7f19a81b0470 legacy=0x7f19a80ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:25.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.727 7f19a7fff700 1 -- 172.21.2.101:0/1786841747 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a80e52e0 con 0x7f19a81b0870 2021-02-10T11:44:25.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.728 7f19a7fff700 1 --2- 172.21.2.101:0/1786841747 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19a81b0870 0x7f19a80f20e0 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 rx=0x7f1998006700 tx=0x7f1998009b80).ready entity=mon.0 client_cookie=391787d58b504213 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:25.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.728 7f19a77fe700 1 -- 172.21.2.101:0/1786841747 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f199800c0c0 con 0x7f19a81b0870 2021-02-10T11:44:25.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.728 7f19a77fe700 1 -- 172.21.2.101:0/1786841747 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199800c440 con 0x7f19a81b0870 2021-02-10T11:44:25.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.728 7f19a77fe700 1 -- 172.21.2.101:0/1786841747 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f199800c960 con 0x7f19a81b0870 2021-02-10T11:44:25.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.728 7f19af04a700 1 -- 172.21.2.101:0/1786841747 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19a81b0870 msgr2=0x7f19a80f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:25.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.728 7f19af04a700 1 --2- 172.21.2.101:0/1786841747 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19a81b0870 0x7f19a80f20e0 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 rx=0x7f1998006700 tx=0x7f1998009b80).stop 2021-02-10T11:44:25.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.728 7f19af04a700 1 -- 172.21.2.101:0/1786841747 shutdown_connections 2021-02-10T11:44:25.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.728 7f19af04a700 1 --2- 172.21.2.101:0/1786841747 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19a80efe00 0x7f19a80f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:25.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.728 7f19af04a700 1 --2- 172.21.2.101:0/1786841747 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19a81b0870 0x7f19a80f20e0 unknown :-1 s=CLOSED pgs=2687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:25.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.728 7f19af04a700 1 -- 172.21.2.101:0/1786841747 >> 172.21.2.101:0/1786841747 conn(0x7f19a80ce3e0 msgr2=0x7f19a80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:25.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.729 7f19af04a700 1 -- 172.21.2.101:0/1786841747 shutdown_connections 2021-02-10T11:44:25.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.729 7f19af04a700 1 -- 172.21.2.101:0/1786841747 wait complete. 2021-02-10T11:44:25.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.743 7f19af04a700 1 Processor -- start 2021-02-10T11:44:25.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.743 7f19af04a700 1 -- start start 2021-02-10T11:44:25.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.744 7f19af04a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19a80aa8d0 0x7f19a819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:25.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.744 7f19af04a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19a819e9d0 0x7f19a819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:25.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.744 7f19af04a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19a819ad40 0x7f19a819b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:25.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.744 7f19af04a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a819b1f0 con 0x7f19a819ad40 2021-02-10T11:44:25.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.744 7f19af04a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a8109b30 con 0x7f19a819e9d0 2021-02-10T11:44:25.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.744 7f19af04a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a8099620 con 0x7f19a80aa8d0 2021-02-10T11:44:25.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.744 7f19ad667700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19a819ad40 0x7f19a819b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:25.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.744 7f19ad667700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19a819ad40 0x7f19a819b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59822/0 (socket says 172.21.2.101:59822) 2021-02-10T11:44:25.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.744 7f19ad667700 1 -- 172.21.2.101:0/901597637 learned_addr learned my addr 172.21.2.101:0/901597637 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:25.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.744 7f19ad667700 1 -- 172.21.2.101:0/901597637 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19a80aa8d0 msgr2=0x7f19a819a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:25.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.744 7f19ad667700 1 --2- 172.21.2.101:0/901597637 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19a80aa8d0 0x7f19a819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:25.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.744 7f19ad667700 1 -- 172.21.2.101:0/901597637 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19a819e9d0 msgr2=0x7f19a819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:25.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.744 7f19ad667700 1 --2- 172.21.2.101:0/901597637 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19a819e9d0 0x7f19a819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:25.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.744 7f19ad667700 1 -- 172.21.2.101:0/901597637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a8109640 con 0x7f19a819ad40 2021-02-10T11:44:25.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.745 7f19ad667700 1 --2- 172.21.2.101:0/901597637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19a819ad40 0x7f19a819b770 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 rx=0x7f19a0009960 tx=0x7f19a000bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:25.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.745 7f19a57fa700 1 -- 172.21.2.101:0/901597637 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19a0003030 con 0x7f19a819ad40 2021-02-10T11:44:25.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.745 7f19a57fa700 1 -- 172.21.2.101:0/901597637 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a0001310 con 0x7f19a819ad40 2021-02-10T11:44:25.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.745 7f19af04a700 1 -- 172.21.2.101:0/901597637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a8109640 con 0x7f19a819ad40 2021-02-10T11:44:25.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.745 7f19af04a700 1 -- 172.21.2.101:0/901597637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a819c080 con 0x7f19a819ad40 2021-02-10T11:44:25.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.745 7f19a57fa700 1 -- 172.21.2.101:0/901597637 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19a0001830 con 0x7f19a819ad40 2021-02-10T11:44:25.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.745 7f19a57fa700 1 -- 172.21.2.101:0/901597637 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f19a000a030 con 0x7f19a819ad40 2021-02-10T11:44:25.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.746 7f19a57fa700 1 --2- 172.21.2.101:0/901597637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f198c0186c0 0x7f198c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:25.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.746 7f19a57fa700 1 -- 172.21.2.101:0/901597637 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f19a00328f0 con 0x7f19a819ad40 2021-02-10T11:44:25.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.746 7f19ace66700 1 --2- 172.21.2.101:0/901597637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f198c0186c0 0x7f198c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:25.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.746 7f19ace66700 1 --2- 172.21.2.101:0/901597637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f198c0186c0 0x7f198c01abc0 crc :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:25.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.776 7f19af04a700 1 -- 172.21.2.101:0/901597637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19a80f0b50 con 0x7f19a819ad40 2021-02-10T11:44:25.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.779 7f19a57fa700 1 -- 172.21.2.101:0/901597637 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f19a0000f40 con 0x7f19a819ad40 2021-02-10T11:44:25.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.907 7f19af04a700 1 -- 172.21.2.101:0/901597637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19a8109ab0 con 0x7f19a819ad40 2021-02-10T11:44:25.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.908 7f19a57fa700 1 -- 172.21.2.101:0/901597637 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+2111 (secure 0 0 0) 0x7f19a000a030 con 0x7f19a819ad40 2021-02-10T11:44:25.908 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 123 2021-02-10T11:44:25.908 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:44:25.908 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":123,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/4208244473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":4208244473},{"type":"v1","addr":"172.21.2.101:6835","nonce":4208244473}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":119}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:44:08.462960","modified":"2021-02-10 11:44:10.472350","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11814":{"gid":11814,"name":"b","rank":0,"incarnation":120,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/3445995728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3445995728},{"type":"v1","addr":"172.21.2.108:6833","nonce":3445995728}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-02-10T11:44:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.910 7f19af04a700 1 -- 172.21.2.101:0/901597637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f198c0186c0 msgr2=0x7f198c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.910 7f19af04a700 1 --2- 172.21.2.101:0/901597637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f198c0186c0 0x7f198c01abc0 crc :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.910 7f19af04a700 1 -- 172.21.2.101:0/901597637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19a819ad40 msgr2=0x7f19a819b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:25.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.910 7f19af04a700 1 --2- 172.21.2.101:0/901597637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19a819ad40 0x7f19a819b770 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 rx=0x7f19a0009960 tx=0x7f19a000bcf0).stop 2021-02-10T11:44:25.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.910 7f19af04a700 1 -- 172.21.2.101:0/901597637 shutdown_connections 2021-02-10T11:44:25.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.910 7f19af04a700 1 --2- 172.21.2.101:0/901597637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19a819ad40 0x7f19a819b770 unknown :-1 s=CLOSED pgs=2688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:25.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.910 7f19af04a700 1 --2- 172.21.2.101:0/901597637 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19a819e9d0 0x7f19a819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:25.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.910 7f19af04a700 1 --2- 172.21.2.101:0/901597637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f198c0186c0 0x7f198c01abc0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:25.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.910 7f19af04a700 1 --2- 172.21.2.101:0/901597637 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19a80aa8d0 0x7f19a819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:25.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.910 7f19af04a700 1 -- 172.21.2.101:0/901597637 >> 172.21.2.101:0/901597637 conn(0x7f19a819f7b0 msgr2=0x7f19a818f380 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:25.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.911 7f19af04a700 1 -- 172.21.2.101:0/901597637 shutdown_connections 2021-02-10T11:44:25.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:25.911 7f19af04a700 1 -- 172.21.2.101:0/901597637 wait complete. 2021-02-10T11:44:26.120 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 123, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:44:08.462960', 'modified': '2021-02-10 11:44:10.472350', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11814}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11814': {'gid': 11814, 'name': 'b', 'rank': 0, 'incarnation': 120, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/3445995728', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 3445995728}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 3445995728}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [31], 'metadata_pool': 30, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:44:26.120 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:44:26.120 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:44:26.523 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:44:26.523 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:44:26.523 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:44:26.523 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 11814, 2021-02-10T11:44:26.523 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:44:26.523 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:44:26.523 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 18.059402355, 2021-02-10T11:44:26.523 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 123, 2021-02-10T11:44:26.523 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 147, 2021-02-10T11:44:26.524 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 147, 2021-02-10T11:44:26.524 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 23.703757728999999 2021-02-10T11:44:26.524 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:44:26.527 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 11814, "want_state": "up:active", "state": "up:active", "rank_uptime": 18.059402355, "mdsmap_epoch": 123, "osdmap_epoch": 147, "osdmap_epoch_barrier": 147, "uptime": 23.703757728999999 } 2021-02-10T11:44:26.527 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:44:26.527 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-02-10T11:44:26.527 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:44:26.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:26.528 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:44:26.873 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:44:27.217 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:44:27.218 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:44:27.562 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:44:27.562 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:44:27.563 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:44:27.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:27.563 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:44:27.898 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:44:27.898 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2021-02-10T11:44:27.898 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:44:27.901 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:44:28.243 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:44:28.244 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:44:28.311 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[163086]: starting ceph client 2021-02-10T11:44:28.311 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:2021-02-10 11:44:28.312 7f029c322f80 -1 init, newargv = 0x5573ded2a1c0 newargc=9 2021-02-10T11:44:28.316 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[163086]: starting fuse 2021-02-10T11:44:28.589 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:44:28.589 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:44:28.590 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:44:28.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:28.591 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:44:28.925 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:44:28.925 INFO:teuthology.orchestra.run.gibba001.stdout:44 2021-02-10T11:44:28.925 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2021-02-10T11:44:28.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:28.926 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:44:28.926 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:44:28.926 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:44:28.926 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:44:28.926 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:44:28.926 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:44:28.926 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:44:28.926 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-02-10T11:44:28.926 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:44:28.927 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:44:28.927 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:44:28.927 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:44:28.927 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:44:28.927 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:44:28.927 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:44:28.927 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:44:28.927 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:44:28.927 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:44:28.927 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:44:28.927 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:44:28.927 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:44:28.928 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.1")) 2021-02-10T11:44:28.928 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:44:29.288 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.1.163086.asok 2021-02-10T11:44:29.291 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.163086.asok 2021-02-10T11:44:29.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:29.291 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.163086.asok status 2021-02-10T11:44:29.681 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:44:29.681 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:44:29.681 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:44:29.681 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:44:29.681 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "1", 2021-02-10T11:44:29.681 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:44:29.681 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:44:29.681 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "163086", 2021-02-10T11:44:29.681 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:44:29.681 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:29.681 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:44:29.682 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:44:29.682 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 12003, 2021-02-10T11:44:29.682 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:44:29.682 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:44:29.682 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:44:29.682 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 12003 2021-02-10T11:44:29.682 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:29.682 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:44:29.682 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:29.682 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:44:29.682 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 622108315 2021-02-10T11:44:29.682 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:29.682 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:29.682 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:44:29.683 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:29.683 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:44:29.683 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 622108315 2021-02-10T11:44:29.683 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:29.683 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.12003 172.21.2.101:0/622108315", 2021-02-10T11:44:29.683 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/622108315", 2021-02-10T11:44:29.683 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:44:29.683 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 123, 2021-02-10T11:44:29.683 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 147, 2021-02-10T11:44:29.683 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:44:29.683 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:44:29.683 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:44:29.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:29.685 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:44:29.685 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:44:30.020 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:44:30.021 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:44:30.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:30.021 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:44:30.369 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2021-02-10T11:44:30.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.785 7fce958cb700 1 Processor -- start 2021-02-10T11:44:30.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.785 7fce958cb700 1 -- start start 2021-02-10T11:44:30.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.786 7fce958cb700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fce900d1c80 0x7fce900cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:30.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.786 7fce958cb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fce900cf590 0x7fce900cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:30.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.786 7fce958cb700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce900fe1d0 con 0x7fce900d1880 2021-02-10T11:44:30.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.786 7fce958cb700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce901b0cc0 con 0x7fce900d1c80 2021-02-10T11:44:30.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.786 7fce958cb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce90109a70 con 0x7fce900cf590 2021-02-10T11:44:30.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.787 7fce8ed73700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fce900d1c80 0x7fce900cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:30.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.787 7fce8f574700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fce900d1880 0x7fce901b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:53752/0 (socket says 172.21.2.101:53752) 2021-02-10T11:44:30.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.787 7fce8f574700 1 -- 172.21.2.101:0/1733341583 learned_addr learned my addr 172.21.2.101:0/1733341583 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:30.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.787 7fce8fd75700 1 --2- 172.21.2.101:0/1733341583 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fce900cf590 0x7fce900cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:30.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.787 7fce8fd75700 1 -- 172.21.2.101:0/1733341583 >> v2:172.21.2.108:3300/0 conn(0x7fce900d1c80 msgr2=0x7fce900cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:30.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.787 7fce8fd75700 1 --2- 172.21.2.101:0/1733341583 >> v2:172.21.2.108:3300/0 conn(0x7fce900d1c80 0x7fce900cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:30.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.787 7fce8fd75700 1 -- 172.21.2.101:0/1733341583 >> v1:172.21.2.108:6789/0 conn(0x7fce900d1880 legacy=0x7fce901b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:30.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.787 7fce8fd75700 1 -- 172.21.2.101:0/1733341583 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce90088070 con 0x7fce900cf590 2021-02-10T11:44:30.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.787 7fce8e572700 1 -- 172.21.2.101:0/1733341583 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3018125952 0 0) 0x7fce78000c60 con 0x7fce900d1880 2021-02-10T11:44:30.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.787 7fce8ed73700 1 --2- 172.21.2.101:0/1733341583 >> v2:172.21.2.108:3300/0 conn(0x7fce900d1c80 0x7fce900cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:44:30.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.788 7fce8fd75700 1 --2- 172.21.2.101:0/1733341583 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fce900cf590 0x7fce900cf990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fce88006e50 tx=0x7fce88009e70).ready entity=mon.2 client_cookie=1a503ed52b59c6dc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:30.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.788 7fce8e572700 1 -- 172.21.2.101:0/1733341583 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fce8800b510 con 0x7fce900cf590 2021-02-10T11:44:30.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.788 7fce8e572700 1 -- 172.21.2.101:0/1733341583 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce88006790 con 0x7fce900cf590 2021-02-10T11:44:30.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.788 7fce8e572700 1 -- 172.21.2.101:0/1733341583 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fce8800b940 con 0x7fce900cf590 2021-02-10T11:44:30.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.788 7fce958cb700 1 -- 172.21.2.101:0/1733341583 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fce900cf590 msgr2=0x7fce900cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:30.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.788 7fce958cb700 1 --2- 172.21.2.101:0/1733341583 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fce900cf590 0x7fce900cf990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fce88006e50 tx=0x7fce88009e70).stop 2021-02-10T11:44:30.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.789 7fce958cb700 1 -- 172.21.2.101:0/1733341583 shutdown_connections 2021-02-10T11:44:30.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.789 7fce958cb700 1 --2- 172.21.2.101:0/1733341583 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fce900cf590 0x7fce900cf990 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:30.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.789 7fce958cb700 1 --2- 172.21.2.101:0/1733341583 >> v2:172.21.2.108:3300/0 conn(0x7fce900d1c80 0x7fce900cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:30.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.789 7fce958cb700 1 -- 172.21.2.101:0/1733341583 >> 172.21.2.101:0/1733341583 conn(0x7fce900ce430 msgr2=0x7fce900d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:30.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.789 7fce958cb700 1 -- 172.21.2.101:0/1733341583 shutdown_connections 2021-02-10T11:44:30.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.789 7fce958cb700 1 -- 172.21.2.101:0/1733341583 wait complete. 2021-02-10T11:44:30.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.803 7fce958cb700 1 Processor -- start 2021-02-10T11:44:30.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.803 7fce958cb700 1 -- start start 2021-02-10T11:44:30.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.804 7fce958cb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fce901a3570 0x7fce901a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:30.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.804 7fce958cb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fce9009c9f0 0x7fce9009cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:30.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.804 7fce958cb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fce9009d410 0x7fce90098170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:30.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.804 7fce958cb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9009c470 con 0x7fce9009d410 2021-02-10T11:44:30.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.804 7fce958cb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce900b2840 con 0x7fce9009c9f0 2021-02-10T11:44:30.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.804 7fce958cb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce900b2a20 con 0x7fce901a3570 2021-02-10T11:44:30.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.804 7fce8fd75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fce9009d410 0x7fce90098170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:30.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.804 7fce8fd75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fce9009d410 0x7fce90098170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59858/0 (socket says 172.21.2.101:59858) 2021-02-10T11:44:30.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.804 7fce8fd75700 1 -- 172.21.2.101:0/2163647066 learned_addr learned my addr 172.21.2.101:0/2163647066 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:30.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.804 7fce8f574700 1 --2- 172.21.2.101:0/2163647066 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fce901a3570 0x7fce901a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:30.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.804 7fce8fd75700 1 -- 172.21.2.101:0/2163647066 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fce901a3570 msgr2=0x7fce901a3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:30.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.804 7fce8fd75700 1 --2- 172.21.2.101:0/2163647066 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fce901a3570 0x7fce901a3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:30.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.804 7fce8fd75700 1 -- 172.21.2.101:0/2163647066 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fce9009c9f0 msgr2=0x7fce9009cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:30.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.804 7fce8fd75700 1 --2- 172.21.2.101:0/2163647066 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fce9009c9f0 0x7fce9009cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:30.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.804 7fce8fd75700 1 -- 172.21.2.101:0/2163647066 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9009d880 con 0x7fce9009d410 2021-02-10T11:44:30.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.805 7fce8f574700 1 --2- 172.21.2.101:0/2163647066 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fce901a3570 0x7fce901a3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:44:30.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.805 7fce8fd75700 1 --2- 172.21.2.101:0/2163647066 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fce9009d410 0x7fce90098170 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 rx=0x7fce88006c50 tx=0x7fce8800bd20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:30.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.805 7fce87fff700 1 -- 172.21.2.101:0/2163647066 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fce880093d0 con 0x7fce9009d410 2021-02-10T11:44:30.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.805 7fce87fff700 1 -- 172.21.2.101:0/2163647066 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8800be60 con 0x7fce9009d410 2021-02-10T11:44:30.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.805 7fce958cb700 1 -- 172.21.2.101:0/2163647066 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce9009d880 con 0x7fce9009d410 2021-02-10T11:44:30.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.805 7fce958cb700 1 -- 172.21.2.101:0/2163647066 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce90098a80 con 0x7fce9009d410 2021-02-10T11:44:30.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.805 7fce87fff700 1 -- 172.21.2.101:0/2163647066 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fce8800c570 con 0x7fce9009d410 2021-02-10T11:44:30.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.806 7fce87fff700 1 -- 172.21.2.101:0/2163647066 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fce880093d0 con 0x7fce9009d410 2021-02-10T11:44:30.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.806 7fce87fff700 1 --2- 172.21.2.101:0/2163647066 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fce740186c0 0x7fce7401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:30.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.806 7fce87fff700 1 -- 172.21.2.101:0/2163647066 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fce88032940 con 0x7fce9009d410 2021-02-10T11:44:30.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.806 7fce8f574700 1 --2- 172.21.2.101:0/2163647066 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fce740186c0 0x7fce7401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:30.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.806 7fce8f574700 1 --2- 172.21.2.101:0/2163647066 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fce740186c0 0x7fce7401abc0 crc :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:30.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.836 7fce958cb700 1 -- 172.21.2.101:0/2163647066 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce90082a70 con 0x7fce9009d410 2021-02-10T11:44:30.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.838 7fce87fff700 1 -- 172.21.2.101:0/2163647066 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fce88002ef0 con 0x7fce9009d410 2021-02-10T11:44:30.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:30.974 7fce958cb700 1 -- 172.21.2.101:0/2163647066 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fce90082a70 con 0x7fce9009d410 2021-02-10T11:44:31.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:31.943 7fce87fff700 1 -- 172.21.2.101:0/2163647066 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v124) v1 ==== 136+0+0 (secure 0 0 0) 0x7fce880093d0 con 0x7fce9009d410 2021-02-10T11:44:31.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:31.946 7fce958cb700 1 -- 172.21.2.101:0/2163647066 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fce740186c0 msgr2=0x7fce7401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:31.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:31.946 7fce958cb700 1 --2- 172.21.2.101:0/2163647066 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fce740186c0 0x7fce7401abc0 crc :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:31.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:31.946 7fce958cb700 1 -- 172.21.2.101:0/2163647066 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fce9009d410 msgr2=0x7fce90098170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:31.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:31.946 7fce958cb700 1 --2- 172.21.2.101:0/2163647066 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fce9009d410 0x7fce90098170 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 rx=0x7fce88006c50 tx=0x7fce8800bd20).stop 2021-02-10T11:44:31.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:31.946 7fce958cb700 1 -- 172.21.2.101:0/2163647066 shutdown_connections 2021-02-10T11:44:31.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:31.946 7fce958cb700 1 --2- 172.21.2.101:0/2163647066 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fce9009d410 0x7fce90098170 unknown :-1 s=CLOSED pgs=2692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:31.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:31.946 7fce958cb700 1 --2- 172.21.2.101:0/2163647066 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fce9009c9f0 0x7fce9009cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:31.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:31.946 7fce958cb700 1 --2- 172.21.2.101:0/2163647066 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fce740186c0 0x7fce7401abc0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:31.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:31.946 7fce958cb700 1 --2- 172.21.2.101:0/2163647066 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fce901a3570 0x7fce901a3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:31.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:31.946 7fce958cb700 1 -- 172.21.2.101:0/2163647066 >> 172.21.2.101:0/2163647066 conn(0x7fce900c2cd0 msgr2=0x7fce900b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:31.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:31.947 7fce958cb700 1 -- 172.21.2.101:0/2163647066 shutdown_connections 2021-02-10T11:44:31.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:31.947 7fce958cb700 1 -- 172.21.2.101:0/2163647066 wait complete. 2021-02-10T11:44:31.954 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs volume ls 2021-02-10T11:44:32.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.368 7efefddc3700 1 Processor -- start 2021-02-10T11:44:32.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.368 7efefddc3700 1 -- start start 2021-02-10T11:44:32.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.368 7efefddc3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7efef80d0340 0x7efef80ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:32.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.368 7efefddc3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7efef80d1c70 0x7efef80efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:32.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.368 7efefddc3700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef80f0b10 con 0x7efef80d1870 2021-02-10T11:44:32.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.368 7efefddc3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef80e5ed0 con 0x7efef80d1c70 2021-02-10T11:44:32.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.368 7efefddc3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef81b0be0 con 0x7efef80d0340 2021-02-10T11:44:32.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.368 7efef77fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7efef80d1870 0x7efef81b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60356/0 (socket says 172.21.2.101:60356) 2021-02-10T11:44:32.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.368 7efef77fe700 1 -- 172.21.2.101:0/3018829386 learned_addr learned my addr 172.21.2.101:0/3018829386 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:32.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.368 7efef7fff700 1 --2- 172.21.2.101:0/3018829386 >> v2:172.21.2.101:3300/0 conn(0x7efef80d1c70 0x7efef80efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:32.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.369 7efef67fc700 1 -- 172.21.2.101:0/3018829386 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2251614767 0 0) 0x7efee0001180 con 0x7efef80d1870 2021-02-10T11:44:32.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.369 7efef67fc700 1 -- 172.21.2.101:0/3018829386 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efee4002fe0 con 0x7efef80d1870 2021-02-10T11:44:32.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.369 7efef6ffd700 1 --2- 172.21.2.101:0/3018829386 >> v2:172.21.2.108:3300/0 conn(0x7efef80d0340 0x7efef80ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:32.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.369 7efef7fff700 1 -- 172.21.2.101:0/3018829386 >> v2:172.21.2.108:3300/0 conn(0x7efef80d0340 msgr2=0x7efef80ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:32.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.369 7efef7fff700 1 --2- 172.21.2.101:0/3018829386 >> v2:172.21.2.108:3300/0 conn(0x7efef80d0340 0x7efef80ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:32.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.369 7efef7fff700 1 -- 172.21.2.101:0/3018829386 >> v1:172.21.2.101:6789/0 conn(0x7efef80d1870 legacy=0x7efef81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:32.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.369 7efef7fff700 1 -- 172.21.2.101:0/3018829386 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef80f20a0 con 0x7efef80d1c70 2021-02-10T11:44:32.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.369 7efef7fff700 1 --2- 172.21.2.101:0/3018829386 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efef80d1c70 0x7efef80efc60 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 rx=0x7efef0006e30 tx=0x7efef0009e60).ready entity=mon.0 client_cookie=3eb7b840a465496c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:32.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.370 7efef67fc700 1 -- 172.21.2.101:0/3018829386 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efef0006a50 con 0x7efef80d1c70 2021-02-10T11:44:32.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.370 7efef67fc700 1 -- 172.21.2.101:0/3018829386 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef000b300 con 0x7efef80d1c70 2021-02-10T11:44:32.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.370 7efef67fc700 1 -- 172.21.2.101:0/3018829386 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efef000b820 con 0x7efef80d1c70 2021-02-10T11:44:32.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.370 7efefddc3700 1 -- 172.21.2.101:0/3018829386 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efef80d1c70 msgr2=0x7efef80efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:32.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.370 7efefddc3700 1 --2- 172.21.2.101:0/3018829386 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efef80d1c70 0x7efef80efc60 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 rx=0x7efef0006e30 tx=0x7efef0009e60).stop 2021-02-10T11:44:32.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.370 7efefddc3700 1 -- 172.21.2.101:0/3018829386 shutdown_connections 2021-02-10T11:44:32.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.370 7efefddc3700 1 --2- 172.21.2.101:0/3018829386 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efef80d1c70 0x7efef80efc60 unknown :-1 s=CLOSED pgs=2694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:32.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.370 7efefddc3700 1 --2- 172.21.2.101:0/3018829386 >> v2:172.21.2.108:3300/0 conn(0x7efef80d0340 0x7efef80ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:32.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.370 7efefddc3700 1 -- 172.21.2.101:0/3018829386 >> 172.21.2.101:0/3018829386 conn(0x7efef80ce450 msgr2=0x7efef80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:32.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.370 7efefddc3700 1 -- 172.21.2.101:0/3018829386 shutdown_connections 2021-02-10T11:44:32.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.370 7efefddc3700 1 -- 172.21.2.101:0/3018829386 wait complete. 2021-02-10T11:44:32.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.385 7efefddc3700 1 Processor -- start 2021-02-10T11:44:32.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.385 7efefddc3700 1 -- start start 2021-02-10T11:44:32.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.386 7efefddc3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efef81a06a0 0x7efef809f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:32.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.386 7efefddc3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efef809f9a0 0x7efef808b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:32.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.386 7efefddc3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efef808bb80 0x7efef808bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:32.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.386 7efefddc3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef808c780 con 0x7efef81a06a0 2021-02-10T11:44:32.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.386 7efefddc3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef809fda0 con 0x7efef808bb80 2021-02-10T11:44:32.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.386 7efefddc3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef808cd00 con 0x7efef809f9a0 2021-02-10T11:44:32.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.386 7efef77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efef81a06a0 0x7efef809f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:32.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.386 7efef77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efef81a06a0 0x7efef809f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59868/0 (socket says 172.21.2.101:59868) 2021-02-10T11:44:32.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.386 7efef77fe700 1 -- 172.21.2.101:0/2752837431 learned_addr learned my addr 172.21.2.101:0/2752837431 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:32.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.386 7efef77fe700 1 -- 172.21.2.101:0/2752837431 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efef809f9a0 msgr2=0x7efef808b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:32.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.386 7efef77fe700 1 --2- 172.21.2.101:0/2752837431 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efef809f9a0 0x7efef808b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:32.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.386 7efef77fe700 1 -- 172.21.2.101:0/2752837431 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efef808bb80 msgr2=0x7efef808bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:32.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.386 7efef77fe700 1 --2- 172.21.2.101:0/2752837431 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efef808bb80 0x7efef808bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:32.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.386 7efef77fe700 1 -- 172.21.2.101:0/2752837431 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef808cf20 con 0x7efef81a06a0 2021-02-10T11:44:32.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.386 7efef77fe700 1 --2- 172.21.2.101:0/2752837431 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efef81a06a0 0x7efef809f380 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 rx=0x7efee000a5d0 tx=0x7efee000a970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:32.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.387 7efeeffff700 1 -- 172.21.2.101:0/2752837431 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efee0000ce0 con 0x7efef81a06a0 2021-02-10T11:44:32.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.387 7efeeffff700 1 -- 172.21.2.101:0/2752837431 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee00043a0 con 0x7efef81a06a0 2021-02-10T11:44:32.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.387 7efeeffff700 1 -- 172.21.2.101:0/2752837431 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efee00048c0 con 0x7efef81a06a0 2021-02-10T11:44:32.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.387 7efefddc3700 1 -- 172.21.2.101:0/2752837431 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef808cf20 con 0x7efef81a06a0 2021-02-10T11:44:32.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.387 7efefddc3700 1 -- 172.21.2.101:0/2752837431 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef808d4b0 con 0x7efef81a06a0 2021-02-10T11:44:32.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.387 7efeeffff700 1 -- 172.21.2.101:0/2752837431 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7efee0000ce0 con 0x7efef81a06a0 2021-02-10T11:44:32.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.387 7efeeffff700 1 --2- 172.21.2.101:0/2752837431 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efedc0186c0 0x7efedc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:32.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.387 7efeeffff700 1 -- 172.21.2.101:0/2752837431 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7efee00323f0 con 0x7efef81a06a0 2021-02-10T11:44:32.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.388 7efef6ffd700 1 --2- 172.21.2.101:0/2752837431 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efedc0186c0 0x7efedc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:32.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.388 7efef6ffd700 1 --2- 172.21.2.101:0/2752837431 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efedc0186c0 0x7efedc01abc0 crc :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:32.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.418 7efefddc3700 1 -- 172.21.2.101:0/2752837431 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef80ebc50 con 0x7efef81a06a0 2021-02-10T11:44:32.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.420 7efeeffff700 1 -- 172.21.2.101:0/2752837431 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7efee0026040 con 0x7efef81a06a0 2021-02-10T11:44:32.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.554 7efefddc3700 1 -- 172.21.2.101:0/2752837431 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs volume ls", "target": ["mgr", ""]}) v1 -- 0x7efef808d4b0 con 0x7efedc0186c0 2021-02-10T11:44:32.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.555 7efeeffff700 1 -- 172.21.2.101:0/2752837431 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7efee80052e0 con 0x7efedc0186c0 2021-02-10T11:44:32.554 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:44:32.555 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:32.555 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "cephfs" 2021-02-10T11:44:32.555 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:32.555 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:44:32.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.556 7efefddc3700 1 -- 172.21.2.101:0/2752837431 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efedc0186c0 msgr2=0x7efedc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:32.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.556 7efefddc3700 1 --2- 172.21.2.101:0/2752837431 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efedc0186c0 0x7efedc01abc0 crc :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:32.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.557 7efefddc3700 1 -- 172.21.2.101:0/2752837431 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efef81a06a0 msgr2=0x7efef809f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:32.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.557 7efefddc3700 1 --2- 172.21.2.101:0/2752837431 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efef81a06a0 0x7efef809f380 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 rx=0x7efee000a5d0 tx=0x7efee000a970).stop 2021-02-10T11:44:32.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.557 7efefddc3700 1 -- 172.21.2.101:0/2752837431 shutdown_connections 2021-02-10T11:44:32.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.557 7efefddc3700 1 --2- 172.21.2.101:0/2752837431 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efedc0186c0 0x7efedc01abc0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:32.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.557 7efefddc3700 1 --2- 172.21.2.101:0/2752837431 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efef809f9a0 0x7efef808b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:32.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.557 7efefddc3700 1 --2- 172.21.2.101:0/2752837431 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efef808bb80 0x7efef808bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:32.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.557 7efefddc3700 1 --2- 172.21.2.101:0/2752837431 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efef81a06a0 0x7efef809f380 unknown :-1 s=CLOSED pgs=2695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:32.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.557 7efefddc3700 1 -- 172.21.2.101:0/2752837431 >> 172.21.2.101:0/2752837431 conn(0x7efef80b83b0 msgr2=0x7efef80a6850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:32.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.557 7efefddc3700 1 -- 172.21.2.101:0/2752837431 shutdown_connections 2021-02-10T11:44:32.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.557 7efefddc3700 1 -- 172.21.2.101:0/2752837431 wait complete. 2021-02-10T11:44:32.564 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:44:32.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.980 7fe0fd38b700 1 Processor -- start 2021-02-10T11:44:32.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.980 7fe0fd38b700 1 -- start start 2021-02-10T11:44:32.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.981 7fe0fd38b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe0f80d1c70 0x7fe0f80cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:32.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.981 7fe0fd38b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0f80cf580 0x7fe0f80cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.981 7fe0fd38b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f80fe1c0 con 0x7fe0f80d1870 2021-02-10T11:44:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.981 7fe0fd38b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f81b0cb0 con 0x7fe0f80d1c70 2021-02-10T11:44:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.981 7fe0fd38b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f8109a60 con 0x7fe0f80cf580 2021-02-10T11:44:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.981 7fe0ee87c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe0f80d1c70 0x7fe0f80cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.981 7fe0f707d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe0f80d1870 0x7fe0f81b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60370/0 (socket says 172.21.2.101:60370) 2021-02-10T11:44:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.981 7fe0ee87c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe0f80d1c70 0x7fe0f80cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59878/0 (socket says 172.21.2.101:59878) 2021-02-10T11:44:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.981 7fe0f707d700 1 -- 172.21.2.101:0/731086388 learned_addr learned my addr 172.21.2.101:0/731086388 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.981 7fe0f787e700 1 --2- 172.21.2.101:0/731086388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0f80cf580 0x7fe0f80cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.982 7fe0ee87c700 1 -- 172.21.2.101:0/731086388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0f80cf580 msgr2=0x7fe0f80cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.982 7fe0ee87c700 1 --2- 172.21.2.101:0/731086388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0f80cf580 0x7fe0f80cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.982 7fe0ee87c700 1 -- 172.21.2.101:0/731086388 >> v1:172.21.2.101:6789/0 conn(0x7fe0f80d1870 legacy=0x7fe0f81b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.982 7fe0f687c700 1 -- 172.21.2.101:0/731086388 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1967741654 0 0) 0x7fe0e8001180 con 0x7fe0f80d1870 2021-02-10T11:44:32.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.982 7fe0ee87c700 1 -- 172.21.2.101:0/731086388 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f8088070 con 0x7fe0f80d1c70 2021-02-10T11:44:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.982 7fe0f787e700 1 --2- 172.21.2.101:0/731086388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0f80cf580 0x7fe0f80cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:44:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.982 7fe0ee87c700 1 --2- 172.21.2.101:0/731086388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0f80d1c70 0x7fe0f80cef60 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 rx=0x7fe0e0006650 tx=0x7fe0e0009a80).ready entity=mon.0 client_cookie=240cb3db6bb5b694 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.982 7fe0f687c700 1 -- 172.21.2.101:0/731086388 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe0e0006390 con 0x7fe0f80d1c70 2021-02-10T11:44:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.982 7fe0f687c700 1 -- 172.21.2.101:0/731086388 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e00009b0 con 0x7fe0f80d1c70 2021-02-10T11:44:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.982 7fe0f687c700 1 -- 172.21.2.101:0/731086388 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe0e000c2b0 con 0x7fe0f80d1c70 2021-02-10T11:44:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.982 7fe0fd38b700 1 -- 172.21.2.101:0/731086388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0f80d1c70 msgr2=0x7fe0f80cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.982 7fe0fd38b700 1 --2- 172.21.2.101:0/731086388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0f80d1c70 0x7fe0f80cef60 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 rx=0x7fe0e0006650 tx=0x7fe0e0009a80).stop 2021-02-10T11:44:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.983 7fe0fd38b700 1 -- 172.21.2.101:0/731086388 shutdown_connections 2021-02-10T11:44:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.983 7fe0fd38b700 1 --2- 172.21.2.101:0/731086388 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0f80cf580 0x7fe0f80cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.983 7fe0fd38b700 1 --2- 172.21.2.101:0/731086388 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0f80d1c70 0x7fe0f80cef60 unknown :-1 s=CLOSED pgs=2697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.983 7fe0fd38b700 1 -- 172.21.2.101:0/731086388 >> 172.21.2.101:0/731086388 conn(0x7fe0f80ce420 msgr2=0x7fe0f80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.983 7fe0fd38b700 1 -- 172.21.2.101:0/731086388 shutdown_connections 2021-02-10T11:44:32.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.983 7fe0fd38b700 1 -- 172.21.2.101:0/731086388 wait complete. 2021-02-10T11:44:33.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.997 7fe0fd38b700 1 Processor -- start 2021-02-10T11:44:33.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.998 7fe0fd38b700 1 -- start start 2021-02-10T11:44:33.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.998 7fe0fd38b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0f809bcc0 0x7fe0f809c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:33.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.998 7fe0fd38b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe0f80a0820 0x7fe0f80a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:33.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.998 7fe0fd38b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0f80a1240 0x7fe0f80858d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.998 7fe0fd38b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f80a66b0 con 0x7fe0f80a1240 2021-02-10T11:44:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.998 7fe0fd38b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f81a49b0 con 0x7fe0f80a0820 2021-02-10T11:44:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.998 7fe0fd38b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f81a4b90 con 0x7fe0f809bcc0 2021-02-10T11:44:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.998 7fe0f787e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0f80a1240 0x7fe0f80858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.998 7fe0f787e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0f80a1240 0x7fe0f80858d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59882/0 (socket says 172.21.2.101:59882) 2021-02-10T11:44:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.998 7fe0f787e700 1 -- 172.21.2.101:0/1528222438 learned_addr learned my addr 172.21.2.101:0/1528222438 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.999 7fe0ee87c700 1 --2- 172.21.2.101:0/1528222438 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe0f80a0820 0x7fe0f80a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.999 7fe0f707d700 1 --2- 172.21.2.101:0/1528222438 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0f809bcc0 0x7fe0f809c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.999 7fe0f787e700 1 -- 172.21.2.101:0/1528222438 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0f809bcc0 msgr2=0x7fe0f809c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.999 7fe0f787e700 1 --2- 172.21.2.101:0/1528222438 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0f809bcc0 0x7fe0f809c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.999 7fe0f787e700 1 -- 172.21.2.101:0/1528222438 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe0f80a0820 msgr2=0x7fe0f80a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:33.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.999 7fe0f787e700 1 --2- 172.21.2.101:0/1528222438 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe0f80a0820 0x7fe0f80a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.999 7fe0f787e700 1 -- 172.21.2.101:0/1528222438 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f80a1640 con 0x7fe0f80a1240 2021-02-10T11:44:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.999 7fe0ee87c700 1 --2- 172.21.2.101:0/1528222438 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe0f80a0820 0x7fe0f80a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:44:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.999 7fe0f707d700 1 --2- 172.21.2.101:0/1528222438 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0f809bcc0 0x7fe0f809c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:44:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.999 7fe0f787e700 1 --2- 172.21.2.101:0/1528222438 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0f80a1240 0x7fe0f80858d0 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 rx=0x7fe0f000b970 tx=0x7fe0f000bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.999 7fe0f4878700 1 -- 172.21.2.101:0/1528222438 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe0f0000f90 con 0x7fe0f80a1240 2021-02-10T11:44:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.999 7fe0f4878700 1 -- 172.21.2.101:0/1528222438 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f0006790 con 0x7fe0f80a1240 2021-02-10T11:44:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.999 7fe0fd38b700 1 -- 172.21.2.101:0/1528222438 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f80a1640 con 0x7fe0f80a1240 2021-02-10T11:44:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.999 7fe0f4878700 1 -- 172.21.2.101:0/1528222438 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe0f0001680 con 0x7fe0f80a1240 2021-02-10T11:44:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:32.999 7fe0fd38b700 1 -- 172.21.2.101:0/1528222438 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f80a1640 con 0x7fe0f80a1240 2021-02-10T11:44:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.000 7fe0f4878700 1 -- 172.21.2.101:0/1528222438 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe0f0000df0 con 0x7fe0f80a1240 2021-02-10T11:44:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.000 7fe0f4878700 1 --2- 172.21.2.101:0/1528222438 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0dc0186d0 0x7fe0dc01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.000 7fe0f4878700 1 -- 172.21.2.101:0/1528222438 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe0f0032b10 con 0x7fe0f80a1240 2021-02-10T11:44:33.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.000 7fe0f707d700 1 --2- 172.21.2.101:0/1528222438 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0dc0186d0 0x7fe0dc01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:33.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.000 7fe0f707d700 1 --2- 172.21.2.101:0/1528222438 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0dc0186d0 0x7fe0dc01abd0 crc :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:33.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.031 7fe0fd38b700 1 -- 172.21.2.101:0/1528222438 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f80fe1c0 con 0x7fe0f80a1240 2021-02-10T11:44:33.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.034 7fe0f4878700 1 -- 172.21.2.101:0/1528222438 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe0f0000df0 con 0x7fe0f80a1240 2021-02-10T11:44:33.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.184 7fe0fd38b700 1 -- 172.21.2.101:0/1528222438 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe0f80fe1c0 con 0x7fe0f80a1240 2021-02-10T11:44:33.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.185 7fe0f4878700 1 -- 172.21.2.101:0/1528222438 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe0f00066f0 con 0x7fe0f80a1240 2021-02-10T11:44:33.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.186 7fe0fd38b700 1 -- 172.21.2.101:0/1528222438 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0dc0186d0 msgr2=0x7fe0dc01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:33.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.186 7fe0fd38b700 1 --2- 172.21.2.101:0/1528222438 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0dc0186d0 0x7fe0dc01abd0 crc :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:33.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.186 7fe0fd38b700 1 -- 172.21.2.101:0/1528222438 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0f80a1240 msgr2=0x7fe0f80858d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:33.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.186 7fe0fd38b700 1 --2- 172.21.2.101:0/1528222438 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0f80a1240 0x7fe0f80858d0 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 rx=0x7fe0f000b970 tx=0x7fe0f000bd10).stop 2021-02-10T11:44:33.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.186 7fe0fd38b700 1 -- 172.21.2.101:0/1528222438 shutdown_connections 2021-02-10T11:44:33.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.186 7fe0fd38b700 1 --2- 172.21.2.101:0/1528222438 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe0f80a1240 0x7fe0f80858d0 unknown :-1 s=CLOSED pgs=2698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:33.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.186 7fe0fd38b700 1 --2- 172.21.2.101:0/1528222438 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe0f80a0820 0x7fe0f80a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:33.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.186 7fe0fd38b700 1 --2- 172.21.2.101:0/1528222438 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe0dc0186d0 0x7fe0dc01abd0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:33.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.186 7fe0fd38b700 1 --2- 172.21.2.101:0/1528222438 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe0f809bcc0 0x7fe0f809c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:33.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.187 7fe0fd38b700 1 -- 172.21.2.101:0/1528222438 >> 172.21.2.101:0/1528222438 conn(0x7fe0f80bef00 msgr2=0x7fe0f809d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:33.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.187 7fe0fd38b700 1 -- 172.21.2.101:0/1528222438 shutdown_connections 2021-02-10T11:44:33.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.187 7fe0fd38b700 1 -- 172.21.2.101:0/1528222438 wait complete. 2021-02-10T11:44:33.194 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolumegroup rm cephfs non_existent_group 2021-02-10T11:44:33.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.612 7ff9b6504700 1 Processor -- start 2021-02-10T11:44:33.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.612 7ff9b6504700 1 -- start start 2021-02-10T11:44:33.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.613 7ff9b6504700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9b00d1860 0x7ff9b01b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:33.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.613 7ff9b6504700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff9b01b0ad0 0x7ff9b0108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:33.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.613 7ff9b6504700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b0109a50 con 0x7ff9b00d1c60 2021-02-10T11:44:33.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.613 7ff9b6504700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b00e5ee0 con 0x7ff9b01b0ad0 2021-02-10T11:44:33.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.613 7ff9b6504700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b0109740 con 0x7ff9b00d1860 2021-02-10T11:44:33.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.613 7ff9b4b21700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff9b00d1c60 0x7ff9b0108f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60382/0 (socket says 172.21.2.101:60382) 2021-02-10T11:44:33.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.613 7ff9b4b21700 1 -- 172.21.2.101:0/2368862110 learned_addr learned my addr 172.21.2.101:0/2368862110 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:33.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.613 7ff9af7fe700 1 --2- 172.21.2.101:0/2368862110 >> v2:172.21.2.101:3300/0 conn(0x7ff9b01b0ad0 0x7ff9b0108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:33.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.613 7ff9aeffd700 1 -- 172.21.2.101:0/2368862110 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 431833862 0 0) 0x7ff9a8000d30 con 0x7ff9b00d1c60 2021-02-10T11:44:33.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.613 7ff9aeffd700 1 -- 172.21.2.101:0/2368862110 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff998002fe0 con 0x7ff9b00d1c60 2021-02-10T11:44:33.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.613 7ff9af7fe700 1 -- 172.21.2.101:0/2368862110 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9b00d1860 msgr2=0x7ff9b01b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:33.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.613 7ff9af7fe700 1 --2- 172.21.2.101:0/2368862110 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9b00d1860 0x7ff9b01b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:33.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.613 7ff9af7fe700 1 -- 172.21.2.101:0/2368862110 >> v1:172.21.2.101:6789/0 conn(0x7ff9b00d1c60 legacy=0x7ff9b0108f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:33.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.614 7ff9af7fe700 1 -- 172.21.2.101:0/2368862110 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b0088070 con 0x7ff9b01b0ad0 2021-02-10T11:44:33.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.614 7ff9af7fe700 1 --2- 172.21.2.101:0/2368862110 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9b01b0ad0 0x7ff9b0108930 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 rx=0x7ff9a4006650 tx=0x7ff9a4009a80).ready entity=mon.0 client_cookie=647830c30bb390ab server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:33.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.614 7ff9aeffd700 1 -- 172.21.2.101:0/2368862110 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff9a40009b0 con 0x7ff9b01b0ad0 2021-02-10T11:44:33.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.614 7ff9aeffd700 1 -- 172.21.2.101:0/2368862110 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a400c0c0 con 0x7ff9b01b0ad0 2021-02-10T11:44:33.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.614 7ff9aeffd700 1 -- 172.21.2.101:0/2368862110 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff9a400c5e0 con 0x7ff9b01b0ad0 2021-02-10T11:44:33.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.614 7ff9b6504700 1 -- 172.21.2.101:0/2368862110 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9b01b0ad0 msgr2=0x7ff9b0108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:33.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.614 7ff9b6504700 1 --2- 172.21.2.101:0/2368862110 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9b01b0ad0 0x7ff9b0108930 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 rx=0x7ff9a4006650 tx=0x7ff9a4009a80).stop 2021-02-10T11:44:33.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.615 7ff9b6504700 1 -- 172.21.2.101:0/2368862110 shutdown_connections 2021-02-10T11:44:33.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.615 7ff9b6504700 1 --2- 172.21.2.101:0/2368862110 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9b01b0ad0 0x7ff9b0108930 unknown :-1 s=CLOSED pgs=2700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:33.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.615 7ff9b6504700 1 --2- 172.21.2.101:0/2368862110 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9b00d1860 0x7ff9b01b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:33.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.615 7ff9b6504700 1 -- 172.21.2.101:0/2368862110 >> 172.21.2.101:0/2368862110 conn(0x7ff9b00ce410 msgr2=0x7ff9b00d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:33.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.615 7ff9b6504700 1 -- 172.21.2.101:0/2368862110 shutdown_connections 2021-02-10T11:44:33.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.615 7ff9b6504700 1 -- 172.21.2.101:0/2368862110 wait complete. 2021-02-10T11:44:33.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.629 7ff9b6504700 1 Processor -- start 2021-02-10T11:44:33.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.630 7ff9b6504700 1 -- start start 2021-02-10T11:44:33.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.630 7ff9b6504700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9b00b6da0 0x7ff9b00b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:33.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.630 7ff9b6504700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9b00a70d0 0x7ff9b00b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:33.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.630 7ff9b6504700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff9b00b8780 0x7ff9b00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:33.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.630 7ff9b6504700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b00a7610 con 0x7ff9b00b6da0 2021-02-10T11:44:33.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.630 7ff9b6504700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b00b7de0 con 0x7ff9b00b8780 2021-02-10T11:44:33.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.630 7ff9b6504700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b00b8000 con 0x7ff9b00a70d0 2021-02-10T11:44:33.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.630 7ff9affff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9b00b6da0 0x7ff9b00b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:33.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.630 7ff9affff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9b00b6da0 0x7ff9b00b71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59892/0 (socket says 172.21.2.101:59892) 2021-02-10T11:44:33.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.630 7ff9affff700 1 -- 172.21.2.101:0/657351244 learned_addr learned my addr 172.21.2.101:0/657351244 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:33.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.631 7ff9affff700 1 -- 172.21.2.101:0/657351244 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9b00a70d0 msgr2=0x7ff9b00b77c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:33.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.631 7ff9affff700 1 --2- 172.21.2.101:0/657351244 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9b00a70d0 0x7ff9b00b77c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:33.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.631 7ff9affff700 1 -- 172.21.2.101:0/657351244 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff9b00b8780 msgr2=0x7ff9b00b8b80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:33.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.631 7ff9affff700 1 --2- 172.21.2.101:0/657351244 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff9b00b8780 0x7ff9b00b8b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:33.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.631 7ff9affff700 1 -- 172.21.2.101:0/657351244 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b00b91a0 con 0x7ff9b00b6da0 2021-02-10T11:44:33.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.631 7ff9affff700 1 --2- 172.21.2.101:0/657351244 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9b00b6da0 0x7ff9b00b71a0 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 rx=0x7ff9a0006f00 tx=0x7ff9a000b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:33.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.631 7ff9acff9700 1 -- 172.21.2.101:0/657351244 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff9a000bd30 con 0x7ff9b00b6da0 2021-02-10T11:44:33.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.631 7ff9acff9700 1 -- 172.21.2.101:0/657351244 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a0000c90 con 0x7ff9b00b6da0 2021-02-10T11:44:33.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.631 7ff9acff9700 1 -- 172.21.2.101:0/657351244 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff9a00022a0 con 0x7ff9b00b6da0 2021-02-10T11:44:33.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.631 7ff9b6504700 1 -- 172.21.2.101:0/657351244 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b00b91a0 con 0x7ff9b00b6da0 2021-02-10T11:44:33.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.631 7ff9b6504700 1 -- 172.21.2.101:0/657351244 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b00b9730 con 0x7ff9b00b6da0 2021-02-10T11:44:33.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.632 7ff9acff9700 1 -- 172.21.2.101:0/657351244 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff9a0005310 con 0x7ff9b00b6da0 2021-02-10T11:44:33.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.632 7ff9acff9700 1 --2- 172.21.2.101:0/657351244 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff9900186c0 0x7ff99001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:33.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.632 7ff9acff9700 1 -- 172.21.2.101:0/657351244 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff9a0032580 con 0x7ff9b00b6da0 2021-02-10T11:44:33.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.632 7ff9af7fe700 1 --2- 172.21.2.101:0/657351244 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff9900186c0 0x7ff99001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:33.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.632 7ff9af7fe700 1 --2- 172.21.2.101:0/657351244 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff9900186c0 0x7ff99001abc0 crc :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:33.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.663 7ff9b6504700 1 -- 172.21.2.101:0/657351244 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b0082920 con 0x7ff9b00b6da0 2021-02-10T11:44:33.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.666 7ff9acff9700 1 -- 172.21.2.101:0/657351244 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff9a001c010 con 0x7ff9b00b6da0 2021-02-10T11:44:33.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.793 7ff9b6504700 1 -- 172.21.2.101:0/657351244 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "target": ["mgr", ""], "group_name": "non_existent_group"}) v1 -- 0x7ff9b00b9730 con 0x7ff9900186c0 2021-02-10T11:44:33.796 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:44:33.797 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:44:33.796 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:44:33.797 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:44:33.796 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:44:33.797 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:44:33.796 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:44:33.797 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:44:33.797 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:44:33.797 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:44:33.801 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:44:33.802 7f2cc39e1700 -1 mgr.server reply reply (2) No such file or directory subvolume group 'non_existent_group' does not exist 2021-02-10T11:44:33.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.802 7ff9acff9700 1 -- 172.21.2.101:0/657351244 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: -2 subvolume group 'non_existent_group' does not exist) v1 ==== 59+0+0 (crc 0 0 0) 0x7ff9a4000dc0 con 0x7ff9900186c0 2021-02-10T11:44:33.808 INFO:teuthology.orchestra.run.gibba001.stderr:Error ENOENT: subvolume group 'non_existent_group' does not exist 2021-02-10T11:44:33.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.810 7ff9b6504700 1 -- 172.21.2.101:0/657351244 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff9900186c0 msgr2=0x7ff99001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:33.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.810 7ff9b6504700 1 --2- 172.21.2.101:0/657351244 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff9900186c0 0x7ff99001abc0 crc :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:33.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.810 7ff9b6504700 1 -- 172.21.2.101:0/657351244 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9b00b6da0 msgr2=0x7ff9b00b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:33.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.810 7ff9b6504700 1 --2- 172.21.2.101:0/657351244 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9b00b6da0 0x7ff9b00b71a0 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 rx=0x7ff9a0006f00 tx=0x7ff9a000b940).stop 2021-02-10T11:44:33.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.810 7ff9b6504700 1 -- 172.21.2.101:0/657351244 shutdown_connections 2021-02-10T11:44:33.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.810 7ff9b6504700 1 --2- 172.21.2.101:0/657351244 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff9900186c0 0x7ff99001abc0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:33.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.810 7ff9b6504700 1 --2- 172.21.2.101:0/657351244 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9b00a70d0 0x7ff9b00b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:33.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.810 7ff9b6504700 1 --2- 172.21.2.101:0/657351244 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff9b00b8780 0x7ff9b00b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:33.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.810 7ff9b6504700 1 --2- 172.21.2.101:0/657351244 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9b00b6da0 0x7ff9b00b71a0 unknown :-1 s=CLOSED pgs=2701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:33.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.810 7ff9b6504700 1 -- 172.21.2.101:0/657351244 >> 172.21.2.101:0/657351244 conn(0x7ff9b0092c70 msgr2=0x7ff9b0093530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:33.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.811 7ff9b6504700 1 -- 172.21.2.101:0/657351244 shutdown_connections 2021-02-10T11:44:33.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:33.811 7ff9b6504700 1 -- 172.21.2.101:0/657351244 wait complete. 2021-02-10T11:44:33.817 DEBUG:teuthology.orchestra.run:got remote process result: 2 2021-02-10T11:44:33.818 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:44:33.818 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:44:34.164 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:44:34.511 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:44:34.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:34.511 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:44:34.511 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:44:34.847 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:44:34.847 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:44:34.847 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:44:34.847 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:44:34.847 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:44:35.203 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[162446]: fuse finished with error 0 and tester_r 0 2021-02-10T11:44:35.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:35.205 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:44:35.205 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:44:35.540 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:44:35.541 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:44:35.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:44:35.541 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:44:35.875 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:35.875 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:44:35.875 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:44:36.212 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:44:36.212 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:44:36.212 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:44:36.212 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:44:36.212 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:44:36.564 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[163086]: fuse finished with error 0 and tester_r 0 2021-02-10T11:44:36.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:36.567 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:44:36.567 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:44:36.907 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:44:36.907 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:44:36.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:44:36.908 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:44:36.939 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:44:36.953 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:44:37.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:37.244 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:44:37.244 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:44:37.578 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:44:37.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:44:37.578 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:44:37.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:44:37.579 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:44:37.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:37.914 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:44:37.914 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:44:38.248 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:44:38.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:44:38.249 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:44:38.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:44:38.249 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:44:38.584 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-02-10T11:44:39.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.001 7fb241c5c700 1 Processor -- start 2021-02-10T11:44:39.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.001 7fb241c5c700 1 -- start start 2021-02-10T11:44:39.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.001 7fb241c5c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb23c0d1870 0x7fb23c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:39.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.001 7fb241c5c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb23c1b0a10 0x7fb23c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:39.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.001 7fb241c5c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb23c0d1c70 0x7fb23c0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:39.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.001 7fb241c5c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb23c0fe200 con 0x7fb23c0d1c70 2021-02-10T11:44:39.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.001 7fb241c5c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb23c0e5f10 con 0x7fb23c0d1870 2021-02-10T11:44:39.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.001 7fb241c5c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c0f0b50 con 0x7fb23c1b0a10 2021-02-10T11:44:39.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.001 7fb23b7fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb23c0d1870 0x7fb23c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:39.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.001 7fb23bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb23c0d1c70 0x7fb23c0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:39.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.002 7fb23bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb23c0d1c70 0x7fb23c0cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59914/0 (socket says 172.21.2.101:59914) 2021-02-10T11:44:39.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.002 7fb23affd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb23c1b0a10 0x7fb23c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.002 7fb23b7fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb23c0d1870 0x7fb23c1b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:42168/0 (socket says 172.21.2.101:42168) 2021-02-10T11:44:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.002 7fb23bfff700 1 -- 172.21.2.101:0/2133460244 learned_addr learned my addr 172.21.2.101:0/2133460244 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.002 7fb23bfff700 1 -- 172.21.2.101:0/2133460244 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb23c1b0a10 msgr2=0x7fb23c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.002 7fb23bfff700 1 --2- 172.21.2.101:0/2133460244 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb23c1b0a10 0x7fb23c0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.002 7fb23bfff700 1 -- 172.21.2.101:0/2133460244 >> v2:172.21.2.108:3300/0 conn(0x7fb23c0d1870 msgr2=0x7fb23c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.002 7fb23bfff700 1 --2- 172.21.2.101:0/2133460244 >> v2:172.21.2.108:3300/0 conn(0x7fb23c0d1870 0x7fb23c1b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.002 7fb23bfff700 1 -- 172.21.2.101:0/2133460244 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23c0cff70 con 0x7fb23c0d1c70 2021-02-10T11:44:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.002 7fb23b7fe700 1 --2- 172.21.2.101:0/2133460244 >> v2:172.21.2.108:3300/0 conn(0x7fb23c0d1870 0x7fb23c1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:44:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.002 7fb23bfff700 1 --2- 172.21.2.101:0/2133460244 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb23c0d1c70 0x7fb23c0cf950 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 rx=0x7fb234006b80 tx=0x7fb234009b90).ready entity=mon.0 client_cookie=bdb14cf80fa6391a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.003 7fb23a7fc700 1 -- 172.21.2.101:0/2133460244 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb2340022e0 con 0x7fb23c0d1c70 2021-02-10T11:44:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.003 7fb23a7fc700 1 -- 172.21.2.101:0/2133460244 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2340025a0 con 0x7fb23c0d1c70 2021-02-10T11:44:39.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.003 7fb23a7fc700 1 -- 172.21.2.101:0/2133460244 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb234002ac0 con 0x7fb23c0d1c70 2021-02-10T11:44:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.003 7fb241c5c700 1 -- 172.21.2.101:0/2133460244 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb23c0d1c70 msgr2=0x7fb23c0cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.003 7fb241c5c700 1 --2- 172.21.2.101:0/2133460244 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb23c0d1c70 0x7fb23c0cf950 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 rx=0x7fb234006b80 tx=0x7fb234009b90).stop 2021-02-10T11:44:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.003 7fb241c5c700 1 -- 172.21.2.101:0/2133460244 shutdown_connections 2021-02-10T11:44:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.003 7fb241c5c700 1 --2- 172.21.2.101:0/2133460244 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb23c1b0a10 0x7fb23c0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.003 7fb241c5c700 1 --2- 172.21.2.101:0/2133460244 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb23c0d1c70 0x7fb23c0cf950 unknown :-1 s=CLOSED pgs=2704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.003 7fb241c5c700 1 --2- 172.21.2.101:0/2133460244 >> v2:172.21.2.108:3300/0 conn(0x7fb23c0d1870 0x7fb23c1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.003 7fb241c5c700 1 -- 172.21.2.101:0/2133460244 >> 172.21.2.101:0/2133460244 conn(0x7fb23c0ce450 msgr2=0x7fb23c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.003 7fb241c5c700 1 -- 172.21.2.101:0/2133460244 shutdown_connections 2021-02-10T11:44:39.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.003 7fb241c5c700 1 -- 172.21.2.101:0/2133460244 wait complete. 2021-02-10T11:44:39.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.018 7fb241c5c700 1 Processor -- start 2021-02-10T11:44:39.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.018 7fb241c5c700 1 -- start start 2021-02-10T11:44:39.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.018 7fb241c5c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb23c0ebe70 0x7fb23c0ec270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:39.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.018 7fb241c5c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb23c093830 0x7fb23c093c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:39.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.018 7fb241c5c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb23c094250 0x7fb23c094650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:39.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.018 7fb241c5c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c0ec940 con 0x7fb23c093830 2021-02-10T11:44:39.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.018 7fb241c5c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c0ece40 con 0x7fb23c0ebe70 2021-02-10T11:44:39.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.018 7fb241c5c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c0ed060 con 0x7fb23c094250 2021-02-10T11:44:39.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.019 7fb23affd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb23c093830 0x7fb23c093c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:39.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.019 7fb23affd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb23c093830 0x7fb23c093c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59918/0 (socket says 172.21.2.101:59918) 2021-02-10T11:44:39.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.019 7fb23affd700 1 -- 172.21.2.101:0/4288310324 learned_addr learned my addr 172.21.2.101:0/4288310324 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:39.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.019 7fb23affd700 1 -- 172.21.2.101:0/4288310324 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb23c094250 msgr2=0x7fb23c094650 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:39.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.019 7fb23affd700 1 --2- 172.21.2.101:0/4288310324 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb23c094250 0x7fb23c094650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:39.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.019 7fb23affd700 1 -- 172.21.2.101:0/4288310324 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb23c0ebe70 msgr2=0x7fb23c0ec270 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:39.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.019 7fb23affd700 1 --2- 172.21.2.101:0/4288310324 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb23c0ebe70 0x7fb23c0ec270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:39.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.019 7fb23affd700 1 -- 172.21.2.101:0/4288310324 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23c0ed280 con 0x7fb23c093830 2021-02-10T11:44:39.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.019 7fb23affd700 1 --2- 172.21.2.101:0/4288310324 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb23c093830 0x7fb23c093c30 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 rx=0x7fb22400b560 tx=0x7fb22400b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:39.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.019 7fb233fff700 1 -- 172.21.2.101:0/4288310324 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb22400bcf0 con 0x7fb23c093830 2021-02-10T11:44:39.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.019 7fb233fff700 1 -- 172.21.2.101:0/4288310324 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb224000c90 con 0x7fb23c093830 2021-02-10T11:44:39.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.019 7fb241c5c700 1 -- 172.21.2.101:0/4288310324 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb23c0ed280 con 0x7fb23c093830 2021-02-10T11:44:39.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.019 7fb233fff700 1 -- 172.21.2.101:0/4288310324 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb224002220 con 0x7fb23c093830 2021-02-10T11:44:39.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.019 7fb241c5c700 1 -- 172.21.2.101:0/4288310324 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb23c0ed810 con 0x7fb23c093830 2021-02-10T11:44:39.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.020 7fb233fff700 1 -- 172.21.2.101:0/4288310324 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb2240052e0 con 0x7fb23c093830 2021-02-10T11:44:39.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.020 7fb233fff700 1 --2- 172.21.2.101:0/4288310324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2200186d0 0x7fb22001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:39.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.020 7fb233fff700 1 -- 172.21.2.101:0/4288310324 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb224032650 con 0x7fb23c093830 2021-02-10T11:44:39.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.020 7fb23b7fe700 1 --2- 172.21.2.101:0/4288310324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2200186d0 0x7fb22001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:39.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.020 7fb23b7fe700 1 --2- 172.21.2.101:0/4288310324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2200186d0 0x7fb22001abd0 crc :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:39.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.051 7fb241c5c700 1 -- 172.21.2.101:0/4288310324 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb23c0ebc40 con 0x7fb23c093830 2021-02-10T11:44:39.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.054 7fb233fff700 1 -- 172.21.2.101:0/4288310324 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb224000c90 con 0x7fb23c093830 2021-02-10T11:44:39.202 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.203 7fb241c5c700 1 -- 172.21.2.101:0/4288310324 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb23c0f0b10 con 0x7fb23c093830 2021-02-10T11:44:39.206 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.207 7fb233fff700 1 -- 172.21.2.101:0/4288310324 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb2240052e0 con 0x7fb23c093830 2021-02-10T11:44:39.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.212 7fb241c5c700 1 -- 172.21.2.101:0/4288310324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2200186d0 msgr2=0x7fb22001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:39.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.212 7fb241c5c700 1 --2- 172.21.2.101:0/4288310324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2200186d0 0x7fb22001abd0 crc :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:39.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.212 7fb241c5c700 1 -- 172.21.2.101:0/4288310324 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb23c093830 msgr2=0x7fb23c093c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:39.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.212 7fb241c5c700 1 --2- 172.21.2.101:0/4288310324 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb23c093830 0x7fb23c093c30 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 rx=0x7fb22400b560 tx=0x7fb22400b900).stop 2021-02-10T11:44:39.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.212 7fb241c5c700 1 -- 172.21.2.101:0/4288310324 shutdown_connections 2021-02-10T11:44:39.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.212 7fb241c5c700 1 --2- 172.21.2.101:0/4288310324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2200186d0 0x7fb22001abd0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:39.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.212 7fb241c5c700 1 --2- 172.21.2.101:0/4288310324 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb23c094250 0x7fb23c094650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:39.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.212 7fb241c5c700 1 --2- 172.21.2.101:0/4288310324 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb23c093830 0x7fb23c093c30 unknown :-1 s=CLOSED pgs=2705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:39.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.212 7fb241c5c700 1 --2- 172.21.2.101:0/4288310324 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb23c0ebe70 0x7fb23c0ec270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:39.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.212 7fb241c5c700 1 -- 172.21.2.101:0/4288310324 >> 172.21.2.101:0/4288310324 conn(0x7fb23c08fa10 msgr2=0x7fb23c0bfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:39.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.213 7fb241c5c700 1 -- 172.21.2.101:0/4288310324 shutdown_connections 2021-02-10T11:44:39.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.213 7fb241c5c700 1 -- 172.21.2.101:0/4288310324 wait complete. 2021-02-10T11:44:39.220 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_group_rm' 2021-02-10T11:44:39.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.633 7f1d47682700 1 Processor -- start 2021-02-10T11:44:39.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.633 7f1d47682700 1 -- start start 2021-02-10T11:44:39.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.633 7f1d47682700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d400d0340 0x7f1d400ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:39.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.634 7f1d47682700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1d400d1c70 0x7f1d400efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:39.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.634 7f1d47682700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d400f0b10 con 0x7f1d400d1870 2021-02-10T11:44:39.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.634 7f1d47682700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d400e5ed0 con 0x7f1d400d1c70 2021-02-10T11:44:39.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.634 7f1d47682700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d401b0be0 con 0x7f1d400d0340 2021-02-10T11:44:39.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.634 7f1d4549e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f1d400d1870 0x7f1d401b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60420/0 (socket says 172.21.2.101:60420) 2021-02-10T11:44:39.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.634 7f1d4549e700 1 -- 172.21.2.101:0/1328178257 learned_addr learned my addr 172.21.2.101:0/1328178257 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:39.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.634 7f1d45c9f700 1 --2- 172.21.2.101:0/1328178257 >> v2:172.21.2.101:3300/0 conn(0x7f1d400d1c70 0x7f1d400efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:39.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.634 7f1d44c9d700 1 --2- 172.21.2.101:0/1328178257 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d400d0340 0x7f1d400ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:39.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.634 7f1d33fff700 1 -- 172.21.2.101:0/1328178257 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1653834271 0 0) 0x7f1d3c000ce0 con 0x7f1d400d1870 2021-02-10T11:44:39.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.634 7f1d33fff700 1 -- 172.21.2.101:0/1328178257 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d2c002fe0 con 0x7f1d400d1870 2021-02-10T11:44:39.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.634 7f1d45c9f700 1 -- 172.21.2.101:0/1328178257 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d400d0340 msgr2=0x7f1d400ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:39.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.634 7f1d45c9f700 1 --2- 172.21.2.101:0/1328178257 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d400d0340 0x7f1d400ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:39.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.634 7f1d45c9f700 1 -- 172.21.2.101:0/1328178257 >> v1:172.21.2.101:6789/0 conn(0x7f1d400d1870 legacy=0x7f1d401b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:39.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.634 7f1d45c9f700 1 -- 172.21.2.101:0/1328178257 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d400f20a0 con 0x7f1d400d1c70 2021-02-10T11:44:39.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.634 7f1d44c9d700 1 --2- 172.21.2.101:0/1328178257 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d400d0340 0x7f1d400ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:44:39.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.634 7f1d45c9f700 1 --2- 172.21.2.101:0/1328178257 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d400d1c70 0x7f1d400efc60 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 rx=0x7f1d38006e30 tx=0x7f1d38009e60).ready entity=mon.0 client_cookie=aace7c67095b672b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:39.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.635 7f1d33fff700 1 -- 172.21.2.101:0/1328178257 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1d38006a50 con 0x7f1d400d1c70 2021-02-10T11:44:39.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.635 7f1d33fff700 1 -- 172.21.2.101:0/1328178257 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3800b300 con 0x7f1d400d1c70 2021-02-10T11:44:39.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.635 7f1d33fff700 1 -- 172.21.2.101:0/1328178257 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1d3800b7e0 con 0x7f1d400d1c70 2021-02-10T11:44:39.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.635 7f1d47682700 1 -- 172.21.2.101:0/1328178257 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d400d1c70 msgr2=0x7f1d400efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:39.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.635 7f1d47682700 1 --2- 172.21.2.101:0/1328178257 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d400d1c70 0x7f1d400efc60 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 rx=0x7f1d38006e30 tx=0x7f1d38009e60).stop 2021-02-10T11:44:39.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.635 7f1d47682700 1 -- 172.21.2.101:0/1328178257 shutdown_connections 2021-02-10T11:44:39.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.635 7f1d47682700 1 --2- 172.21.2.101:0/1328178257 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d400d0340 0x7f1d400ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:39.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.635 7f1d47682700 1 --2- 172.21.2.101:0/1328178257 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d400d1c70 0x7f1d400efc60 unknown :-1 s=CLOSED pgs=2707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:39.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.635 7f1d47682700 1 -- 172.21.2.101:0/1328178257 >> 172.21.2.101:0/1328178257 conn(0x7f1d400ce450 msgr2=0x7f1d400d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:39.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.635 7f1d47682700 1 -- 172.21.2.101:0/1328178257 shutdown_connections 2021-02-10T11:44:39.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.635 7f1d47682700 1 -- 172.21.2.101:0/1328178257 wait complete. 2021-02-10T11:44:39.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.650 7f1d47682700 1 Processor -- start 2021-02-10T11:44:39.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.650 7f1d47682700 1 -- start start 2021-02-10T11:44:39.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.651 7f1d47682700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d40195650 0x7f1d400b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:39.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.651 7f1d47682700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d400b5b80 0x7f1d401a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:39.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.651 7f1d47682700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1d400acfa0 0x7f1d401a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:39.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.651 7f1d47682700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d400b6010 con 0x7f1d400b5b80 2021-02-10T11:44:39.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.651 7f1d47682700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d401a3a60 con 0x7f1d400acfa0 2021-02-10T11:44:39.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.651 7f1d47682700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d401a3c40 con 0x7f1d40195650 2021-02-10T11:44:39.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.651 7f1d44c9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d400b5b80 0x7f1d401a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:39.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.651 7f1d44c9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d400b5b80 0x7f1d401a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59932/0 (socket says 172.21.2.101:59932) 2021-02-10T11:44:39.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.651 7f1d44c9d700 1 -- 172.21.2.101:0/1028307112 learned_addr learned my addr 172.21.2.101:0/1028307112 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:39.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.651 7f1d44c9d700 1 -- 172.21.2.101:0/1028307112 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d40195650 msgr2=0x7f1d400b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:39.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.651 7f1d44c9d700 1 --2- 172.21.2.101:0/1028307112 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d40195650 0x7f1d400b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:39.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.651 7f1d44c9d700 1 -- 172.21.2.101:0/1028307112 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1d400acfa0 msgr2=0x7f1d401a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:39.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.651 7f1d44c9d700 1 --2- 172.21.2.101:0/1028307112 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1d400acfa0 0x7f1d401a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:39.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.651 7f1d44c9d700 1 -- 172.21.2.101:0/1028307112 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d400bcae0 con 0x7f1d400b5b80 2021-02-10T11:44:39.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.651 7f1d44c9d700 1 --2- 172.21.2.101:0/1028307112 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d400b5b80 0x7f1d401a2e20 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 rx=0x7f1d34006f40 tx=0x7f1d3400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:39.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.652 7f1d31ffb700 1 -- 172.21.2.101:0/1028307112 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1d340009b0 con 0x7f1d400b5b80 2021-02-10T11:44:39.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.652 7f1d31ffb700 1 -- 172.21.2.101:0/1028307112 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3400bd80 con 0x7f1d400b5b80 2021-02-10T11:44:39.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.652 7f1d31ffb700 1 -- 172.21.2.101:0/1028307112 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1d34002220 con 0x7f1d400b5b80 2021-02-10T11:44:39.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.652 7f1d47682700 1 -- 172.21.2.101:0/1028307112 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d400bcae0 con 0x7f1d400b5b80 2021-02-10T11:44:39.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.652 7f1d47682700 1 -- 172.21.2.101:0/1028307112 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d400bd070 con 0x7f1d400b5b80 2021-02-10T11:44:39.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.652 7f1d31ffb700 1 -- 172.21.2.101:0/1028307112 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1d34006b50 con 0x7f1d400b5b80 2021-02-10T11:44:39.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.652 7f1d31ffb700 1 --2- 172.21.2.101:0/1028307112 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d200186c0 0x7f1d2001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:39.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.652 7f1d31ffb700 1 -- 172.21.2.101:0/1028307112 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1d34032560 con 0x7f1d400b5b80 2021-02-10T11:44:39.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.653 7f1d4549e700 1 --2- 172.21.2.101:0/1028307112 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d200186c0 0x7f1d2001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:39.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.653 7f1d4549e700 1 --2- 172.21.2.101:0/1028307112 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d200186c0 0x7f1d2001abc0 crc :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:39.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.684 7f1d47682700 1 -- 172.21.2.101:0/1028307112 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d400f0ab0 con 0x7f1d400b5b80 2021-02-10T11:44:39.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.686 7f1d31ffb700 1 -- 172.21.2.101:0/1028307112 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1d34000990 con 0x7f1d400b5b80 2021-02-10T11:44:39.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:39.818 7f1d47682700 1 -- 172.21.2.101:0/1028307112 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_group_rm"]} v 0) v1 -- 0x7f1d400f0ab0 con 0x7f1d400b5b80 2021-02-10T11:44:40.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:40.612 7f1d31ffb700 1 -- 172.21.2.101:0/1028307112 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_group_rm"]}]=0 v721) v1 ==== 152+0+0 (secure 0 0 0) 0x7f1d34006b50 con 0x7f1d400b5b80 2021-02-10T11:44:40.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:40.641 7f1d47682700 1 -- 172.21.2.101:0/1028307112 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d200186c0 msgr2=0x7f1d2001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:40.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:40.642 7f1d47682700 1 --2- 172.21.2.101:0/1028307112 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d200186c0 0x7f1d2001abc0 crc :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:40.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:40.642 7f1d47682700 1 -- 172.21.2.101:0/1028307112 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d400b5b80 msgr2=0x7f1d401a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:40.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:40.642 7f1d47682700 1 --2- 172.21.2.101:0/1028307112 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d400b5b80 0x7f1d401a2e20 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 rx=0x7f1d34006f40 tx=0x7f1d3400b990).stop 2021-02-10T11:44:40.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:40.642 7f1d47682700 1 -- 172.21.2.101:0/1028307112 shutdown_connections 2021-02-10T11:44:40.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:40.642 7f1d47682700 1 --2- 172.21.2.101:0/1028307112 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1d400acfa0 0x7f1d401a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:40.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:40.642 7f1d47682700 1 --2- 172.21.2.101:0/1028307112 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d400b5b80 0x7f1d401a2e20 unknown :-1 s=CLOSED pgs=2708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:40.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:40.642 7f1d47682700 1 --2- 172.21.2.101:0/1028307112 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1d200186c0 0x7f1d2001abc0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:40.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:40.642 7f1d47682700 1 --2- 172.21.2.101:0/1028307112 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d40195650 0x7f1d400b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:40.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:40.642 7f1d47682700 1 -- 172.21.2.101:0/1028307112 >> 172.21.2.101:0/1028307112 conn(0x7f1d400b3040 msgr2=0x7f1d400c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:40.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:40.642 7f1d47682700 1 -- 172.21.2.101:0/1028307112 shutdown_connections 2021-02-10T11:44:40.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:40.642 7f1d47682700 1 -- 172.21.2.101:0/1028307112 wait complete. 2021-02-10T11:44:40.649 INFO:tasks.cephfs_test_runner:test_nonexistent_subvolume_group_rm (tasks.cephfs.test_volumes.TestVolumes) ... ok 2021-02-10T11:44:40.650 INFO:tasks.cephfs_test_runner:Starting test: test_nonexistent_subvolume_group_snapshot_rm (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:44:40.650 INFO:tasks.cephfs_test_runner:test_nonexistent_subvolume_group_snapshot_rm (tasks.cephfs.test_volumes.TestVolumes) ... skipped 'skipping subvolumegroup snapshot tests' 2021-02-10T11:44:40.650 INFO:tasks.cephfs_test_runner:Starting test: test_nonexistent_subvolume_rm (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:44:40.650 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_rm' 2021-02-10T11:44:41.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.063 7fe233de0700 1 Processor -- start 2021-02-10T11:44:41.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.063 7fe233de0700 1 -- start start 2021-02-10T11:44:41.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.064 7fe233de0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe22c0d0340 0x7fe22c0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:41.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.064 7fe233de0700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe22c0f0b10 con 0x7fe22c0d1c70 2021-02-10T11:44:41.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.064 7fe233de0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe22c1b0be0 con 0x7fe22c0d1870 2021-02-10T11:44:41.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.064 7fe233de0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c0e5ed0 con 0x7fe22c0d0340 2021-02-10T11:44:41.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.064 7fe2323fd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe22c0d1c70 0x7fe22c0efc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60436/0 (socket says 172.21.2.101:60436) 2021-02-10T11:44:41.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.064 7fe2323fd700 1 -- 172.21.2.101:0/440489170 learned_addr learned my addr 172.21.2.101:0/440489170 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:41.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.064 7fe2313fb700 1 --2- 172.21.2.101:0/440489170 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe22c0d0340 0x7fe22c0ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:41.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.064 7fe230bfa700 1 -- 172.21.2.101:0/440489170 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4084991606 0 0) 0x7fe224001500 con 0x7fe22c0d1c70 2021-02-10T11:44:41.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.064 7fe230bfa700 1 -- 172.21.2.101:0/440489170 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe218002fe0 con 0x7fe22c0d1c70 2021-02-10T11:44:41.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.064 7fe2313fb700 1 -- 172.21.2.101:0/440489170 >> v1:172.21.2.108:6789/0 conn(0x7fe22c0d1870 legacy=0x7fe22c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:41.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.065 7fe2313fb700 1 -- 172.21.2.101:0/440489170 >> v1:172.21.2.101:6789/0 conn(0x7fe22c0d1c70 legacy=0x7fe22c0efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:41.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.065 7fe230bfa700 1 -- 172.21.2.101:0/440489170 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3544943094 0 0) 0x7fe220001080 con 0x7fe22c0d1870 2021-02-10T11:44:41.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.065 7fe2313fb700 1 -- 172.21.2.101:0/440489170 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c0f0450 con 0x7fe22c0d0340 2021-02-10T11:44:41.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.065 7fe2313fb700 1 --2- 172.21.2.101:0/440489170 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe22c0d0340 0x7fe22c0ef640 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fe228006650 tx=0x7fe228009a00).ready entity=mon.2 client_cookie=984bbbaac6ef2831 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:41.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.065 7fe230bfa700 1 -- 172.21.2.101:0/440489170 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe2280009b0 con 0x7fe22c0d0340 2021-02-10T11:44:41.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.065 7fe230bfa700 1 -- 172.21.2.101:0/440489170 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22800c0c0 con 0x7fe22c0d0340 2021-02-10T11:44:41.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.065 7fe230bfa700 1 -- 172.21.2.101:0/440489170 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe22800c5e0 con 0x7fe22c0d0340 2021-02-10T11:44:41.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.065 7fe233de0700 1 -- 172.21.2.101:0/440489170 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe22c0d0340 msgr2=0x7fe22c0ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:41.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.065 7fe233de0700 1 --2- 172.21.2.101:0/440489170 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe22c0d0340 0x7fe22c0ef640 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fe228006650 tx=0x7fe228009a00).stop 2021-02-10T11:44:41.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.066 7fe233de0700 1 -- 172.21.2.101:0/440489170 shutdown_connections 2021-02-10T11:44:41.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.066 7fe233de0700 1 --2- 172.21.2.101:0/440489170 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe22c0d0340 0x7fe22c0ef640 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:41.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.066 7fe233de0700 1 -- 172.21.2.101:0/440489170 >> 172.21.2.101:0/440489170 conn(0x7fe22c0ce450 msgr2=0x7fe22c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:41.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.066 7fe233de0700 1 -- 172.21.2.101:0/440489170 shutdown_connections 2021-02-10T11:44:41.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.066 7fe233de0700 1 -- 172.21.2.101:0/440489170 wait complete. 2021-02-10T11:44:41.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.080 7fe233de0700 1 Processor -- start 2021-02-10T11:44:41.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.081 7fe233de0700 1 -- start start 2021-02-10T11:44:41.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.081 7fe233de0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe22c1ac980 0x7fe22c1ace80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:41.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.081 7fe233de0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe22c0aa400 0x7fe22c1b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:41.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.081 7fe233de0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe22c0aa800 0x7fe22c1b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:41.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.081 7fe233de0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c1b6d50 con 0x7fe22c0aa400 2021-02-10T11:44:41.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.081 7fe233de0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c1ad4a0 con 0x7fe22c0aa800 2021-02-10T11:44:41.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.081 7fe233de0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c18f0e0 con 0x7fe22c1ac980 2021-02-10T11:44:41.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.081 7fe2313fb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe22c0aa400 0x7fe22c1b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:41.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.081 7fe2313fb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe22c0aa400 0x7fe22c1b72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59944/0 (socket says 172.21.2.101:59944) 2021-02-10T11:44:41.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.081 7fe2313fb700 1 -- 172.21.2.101:0/3624181095 learned_addr learned my addr 172.21.2.101:0/3624181095 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:41.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.081 7fe2313fb700 1 -- 172.21.2.101:0/3624181095 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe22c1ac980 msgr2=0x7fe22c1ace80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:41.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.081 7fe2313fb700 1 --2- 172.21.2.101:0/3624181095 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe22c1ac980 0x7fe22c1ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:41.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.081 7fe2313fb700 1 -- 172.21.2.101:0/3624181095 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe22c0aa800 msgr2=0x7fe22c1b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:41.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.081 7fe2313fb700 1 --2- 172.21.2.101:0/3624181095 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe22c0aa800 0x7fe22c1b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:41.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.081 7fe2313fb700 1 -- 172.21.2.101:0/3624181095 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c1ac4c0 con 0x7fe22c0aa400 2021-02-10T11:44:41.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.082 7fe2313fb700 1 --2- 172.21.2.101:0/3624181095 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe22c0aa400 0x7fe22c1b72d0 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 rx=0x7fe22800b580 tx=0x7fe22800b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:41.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.082 7fe21e7fc700 1 -- 172.21.2.101:0/3624181095 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe228006860 con 0x7fe22c0aa400 2021-02-10T11:44:41.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.082 7fe21e7fc700 1 -- 172.21.2.101:0/3624181095 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe228002030 con 0x7fe22c0aa400 2021-02-10T11:44:41.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.082 7fe233de0700 1 -- 172.21.2.101:0/3624181095 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe22c1ac4c0 con 0x7fe22c0aa400 2021-02-10T11:44:41.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.082 7fe21e7fc700 1 -- 172.21.2.101:0/3624181095 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe2280024d0 con 0x7fe22c0aa400 2021-02-10T11:44:41.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.082 7fe233de0700 1 -- 172.21.2.101:0/3624181095 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe22c1ac4c0 con 0x7fe22c0aa400 2021-02-10T11:44:41.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.083 7fe21e7fc700 1 -- 172.21.2.101:0/3624181095 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe228002420 con 0x7fe22c0aa400 2021-02-10T11:44:41.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.083 7fe21e7fc700 1 --2- 172.21.2.101:0/3624181095 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe214018730 0x7fe21401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:41.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.083 7fe21e7fc700 1 -- 172.21.2.101:0/3624181095 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe2280327c0 con 0x7fe22c0aa400 2021-02-10T11:44:41.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.083 7fe231bfc700 1 --2- 172.21.2.101:0/3624181095 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe214018730 0x7fe21401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:41.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.083 7fe231bfc700 1 --2- 172.21.2.101:0/3624181095 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe214018730 0x7fe21401ac10 crc :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:41.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.114 7fe233de0700 1 -- 172.21.2.101:0/3624181095 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe22c0ebc60 con 0x7fe22c0aa400 2021-02-10T11:44:41.115 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.116 7fe21e7fc700 1 -- 172.21.2.101:0/3624181095 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe228002030 con 0x7fe22c0aa400 2021-02-10T11:44:41.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.247 7fe233de0700 1 -- 172.21.2.101:0/3624181095 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_rm"]} v 0) v1 -- 0x7fe22c0f0b10 con 0x7fe22c0aa400 2021-02-10T11:44:41.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.616 7fe21e7fc700 1 -- 172.21.2.101:0/3624181095 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_rm"]}]=0 v722) v1 ==== 149+0+0 (secure 0 0 0) 0x7fe2280023e0 con 0x7fe22c0aa400 2021-02-10T11:44:41.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.665 7fe233de0700 1 -- 172.21.2.101:0/3624181095 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe214018730 msgr2=0x7fe21401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:41.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.665 7fe233de0700 1 --2- 172.21.2.101:0/3624181095 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe214018730 0x7fe21401ac10 crc :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:41.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.665 7fe233de0700 1 -- 172.21.2.101:0/3624181095 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe22c0aa400 msgr2=0x7fe22c1b72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:41.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.665 7fe233de0700 1 --2- 172.21.2.101:0/3624181095 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe22c0aa400 0x7fe22c1b72d0 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 rx=0x7fe22800b580 tx=0x7fe22800b920).stop 2021-02-10T11:44:41.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.665 7fe233de0700 1 -- 172.21.2.101:0/3624181095 shutdown_connections 2021-02-10T11:44:41.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.665 7fe233de0700 1 --2- 172.21.2.101:0/3624181095 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe22c0aa800 0x7fe22c1b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:41.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.665 7fe233de0700 1 --2- 172.21.2.101:0/3624181095 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe22c0aa400 0x7fe22c1b72d0 unknown :-1 s=CLOSED pgs=2710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:41.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.665 7fe233de0700 1 --2- 172.21.2.101:0/3624181095 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe214018730 0x7fe21401ac10 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:41.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.665 7fe233de0700 1 --2- 172.21.2.101:0/3624181095 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe22c1ac980 0x7fe22c1ace80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:41.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.665 7fe233de0700 1 -- 172.21.2.101:0/3624181095 >> 172.21.2.101:0/3624181095 conn(0x7fe22c1ac040 msgr2=0x7fe22c094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:41.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.665 7fe233de0700 1 -- 172.21.2.101:0/3624181095 shutdown_connections 2021-02-10T11:44:41.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:41.665 7fe233de0700 1 -- 172.21.2.101:0/3624181095 wait complete. 2021-02-10T11:44:41.673 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:44:42.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.089 7fb66748e700 1 Processor -- start 2021-02-10T11:44:42.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.089 7fb66748e700 1 -- start start 2021-02-10T11:44:42.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.090 7fb66748e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb6600d1870 0x7fb6601b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:42.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.090 7fb66748e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6600cff70 0x7fb660109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:42.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.090 7fb66748e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb660109a60 con 0x7fb6601b0ae0 2021-02-10T11:44:42.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.090 7fb66748e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb660109750 con 0x7fb6600d1870 2021-02-10T11:44:42.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.090 7fb66748e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb660088070 con 0x7fb6600cff70 2021-02-10T11:44:42.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.090 7fb6652aa700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb6600d1870 0x7fb6601b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:42.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.090 7fb6652aa700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb6600d1870 0x7fb6601b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59952/0 (socket says 172.21.2.101:59952) 2021-02-10T11:44:42.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.090 7fb6652aa700 1 -- 172.21.2.101:0/345534074 learned_addr learned my addr 172.21.2.101:0/345534074 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:42.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.090 7fb6652aa700 1 -- 172.21.2.101:0/345534074 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6600cff70 msgr2=0x7fb660109130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:42.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.090 7fb6652aa700 1 --2- 172.21.2.101:0/345534074 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6600cff70 0x7fb660109130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:42.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.090 7fb6652aa700 1 -- 172.21.2.101:0/345534074 >> v1:172.21.2.108:6789/0 conn(0x7fb6601b0ae0 legacy=0x7fb660108940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:42.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.090 7fb6652aa700 1 -- 172.21.2.101:0/345534074 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6600d1c70 con 0x7fb6600d1870 2021-02-10T11:44:42.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.090 7fb6652aa700 1 --2- 172.21.2.101:0/345534074 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6600d1870 0x7fb6601b04c0 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 rx=0x7fb65c006700 tx=0x7fb65c009b80).ready entity=mon.0 client_cookie=65afca0e59374298 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:42.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.090 7fb657fff700 1 -- 172.21.2.101:0/345534074 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb65c00c0c0 con 0x7fb6600d1870 2021-02-10T11:44:42.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.091 7fb657fff700 1 -- 172.21.2.101:0/345534074 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65c00c440 con 0x7fb6600d1870 2021-02-10T11:44:42.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.091 7fb657fff700 1 -- 172.21.2.101:0/345534074 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb65c00c920 con 0x7fb6600d1870 2021-02-10T11:44:42.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.091 7fb66748e700 1 -- 172.21.2.101:0/345534074 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6600d1870 msgr2=0x7fb6601b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:42.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.091 7fb66748e700 1 --2- 172.21.2.101:0/345534074 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6600d1870 0x7fb6601b04c0 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 rx=0x7fb65c006700 tx=0x7fb65c009b80).stop 2021-02-10T11:44:42.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.091 7fb66748e700 1 -- 172.21.2.101:0/345534074 shutdown_connections 2021-02-10T11:44:42.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.091 7fb66748e700 1 --2- 172.21.2.101:0/345534074 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6600cff70 0x7fb660109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:42.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.091 7fb66748e700 1 --2- 172.21.2.101:0/345534074 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6600d1870 0x7fb6601b04c0 unknown :-1 s=CLOSED pgs=2711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:42.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.091 7fb66748e700 1 -- 172.21.2.101:0/345534074 >> 172.21.2.101:0/345534074 conn(0x7fb6600ce420 msgr2=0x7fb6600d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:42.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.091 7fb66748e700 1 -- 172.21.2.101:0/345534074 shutdown_connections 2021-02-10T11:44:42.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.091 7fb66748e700 1 -- 172.21.2.101:0/345534074 wait complete. 2021-02-10T11:44:42.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.106 7fb66748e700 1 Processor -- start 2021-02-10T11:44:42.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.106 7fb66748e700 1 -- start start 2021-02-10T11:44:42.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.106 7fb66748e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb6600c3820 0x7fb660084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:42.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.106 7fb66748e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6600a0290 0x7fb6600852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:42.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.106 7fb66748e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb660086270 0x7fb660086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:42.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.106 7fb66748e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6600c0700 con 0x7fb6600a0290 2021-02-10T11:44:42.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.106 7fb66748e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6600858f0 con 0x7fb6600c3820 2021-02-10T11:44:42.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.106 7fb66748e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb660085b10 con 0x7fb660086270 2021-02-10T11:44:42.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.106 7fb664aa9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6600a0290 0x7fb6600852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:42.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.106 7fb664aa9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6600a0290 0x7fb6600852d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59958/0 (socket says 172.21.2.101:59958) 2021-02-10T11:44:42.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.106 7fb664aa9700 1 -- 172.21.2.101:0/53462037 learned_addr learned my addr 172.21.2.101:0/53462037 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:42.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.107 7fb664aa9700 1 -- 172.21.2.101:0/53462037 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb660086270 msgr2=0x7fb660086670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:42.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.107 7fb664aa9700 1 --2- 172.21.2.101:0/53462037 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb660086270 0x7fb660086670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:42.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.107 7fb664aa9700 1 -- 172.21.2.101:0/53462037 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb6600c3820 msgr2=0x7fb660084cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:42.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.107 7fb664aa9700 1 --2- 172.21.2.101:0/53462037 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb6600c3820 0x7fb660084cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:42.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.107 7fb664aa9700 1 -- 172.21.2.101:0/53462037 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb660086c90 con 0x7fb6600a0290 2021-02-10T11:44:42.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.107 7fb664aa9700 1 --2- 172.21.2.101:0/53462037 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6600a0290 0x7fb6600852d0 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 rx=0x7fb65000b5b0 tx=0x7fb65000b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:42.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.107 7fb655ffb700 1 -- 172.21.2.101:0/53462037 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb65000bd40 con 0x7fb6600a0290 2021-02-10T11:44:42.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.107 7fb655ffb700 1 -- 172.21.2.101:0/53462037 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb650006020 con 0x7fb6600a0290 2021-02-10T11:44:42.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.107 7fb66748e700 1 -- 172.21.2.101:0/53462037 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb660086c90 con 0x7fb6600a0290 2021-02-10T11:44:42.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.107 7fb66748e700 1 -- 172.21.2.101:0/53462037 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6601a2f00 con 0x7fb6600a0290 2021-02-10T11:44:42.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.107 7fb655ffb700 1 -- 172.21.2.101:0/53462037 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb6500054a0 con 0x7fb6600a0290 2021-02-10T11:44:42.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.107 7fb655ffb700 1 -- 172.21.2.101:0/53462037 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb650006200 con 0x7fb6600a0290 2021-02-10T11:44:42.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.107 7fb655ffb700 1 --2- 172.21.2.101:0/53462037 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb640018700 0x7fb64001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:42.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.108 7fb655ffb700 1 -- 172.21.2.101:0/53462037 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb650032820 con 0x7fb6600a0290 2021-02-10T11:44:42.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.108 7fb6652aa700 1 --2- 172.21.2.101:0/53462037 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb640018700 0x7fb64001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:42.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.108 7fb6652aa700 1 --2- 172.21.2.101:0/53462037 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb640018700 0x7fb64001abc0 crc :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:42.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.140 7fb66748e700 1 -- 172.21.2.101:0/53462037 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb660082a70 con 0x7fb6600a0290 2021-02-10T11:44:42.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.142 7fb655ffb700 1 -- 172.21.2.101:0/53462037 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb650006230 con 0x7fb6600a0290 2021-02-10T11:44:42.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.292 7fb66748e700 1 -- 172.21.2.101:0/53462037 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb660082a70 con 0x7fb6600a0290 2021-02-10T11:44:42.295 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.296 7fb655ffb700 1 -- 172.21.2.101:0/53462037 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb650006020 con 0x7fb6600a0290 2021-02-10T11:44:42.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.301 7fb66748e700 1 -- 172.21.2.101:0/53462037 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb640018700 msgr2=0x7fb64001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:42.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.301 7fb66748e700 1 --2- 172.21.2.101:0/53462037 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb640018700 0x7fb64001abc0 crc :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:42.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.301 7fb66748e700 1 -- 172.21.2.101:0/53462037 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6600a0290 msgr2=0x7fb6600852d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:42.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.301 7fb66748e700 1 --2- 172.21.2.101:0/53462037 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6600a0290 0x7fb6600852d0 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 rx=0x7fb65000b5b0 tx=0x7fb65000b950).stop 2021-02-10T11:44:42.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.301 7fb66748e700 1 -- 172.21.2.101:0/53462037 shutdown_connections 2021-02-10T11:44:42.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.301 7fb66748e700 1 --2- 172.21.2.101:0/53462037 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb640018700 0x7fb64001abc0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:42.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.301 7fb66748e700 1 --2- 172.21.2.101:0/53462037 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb660086270 0x7fb660086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:42.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.301 7fb66748e700 1 --2- 172.21.2.101:0/53462037 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6600a0290 0x7fb6600852d0 unknown :-1 s=CLOSED pgs=2712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:42.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.301 7fb66748e700 1 --2- 172.21.2.101:0/53462037 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb6600c3820 0x7fb660084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:42.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.301 7fb66748e700 1 -- 172.21.2.101:0/53462037 >> 172.21.2.101:0/53462037 conn(0x7fb6600872c0 msgr2=0x7fb660095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:42.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.301 7fb66748e700 1 -- 172.21.2.101:0/53462037 shutdown_connections 2021-02-10T11:44:42.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:42.302 7fb66748e700 1 -- 172.21.2.101:0/53462037 wait complete. 2021-02-10T11:44:42.308 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:44:42.309 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:44:42.654 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:44:43.000 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:44:43.000 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:43.000 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:44:43.000 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:44:43.334 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2021-02-10T11:44:43.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:44:43.334 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-02-10T11:44:43.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:43.335 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:44:43.335 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:44:43.671 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:44:43.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:44:43.671 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:44:43.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:43.671 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:44:43.671 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:44:44.006 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:44:44.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:44:44.007 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:44:44.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:44:44.007 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:44:44.007 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:44:44.342 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:44:44.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:44:44.342 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:44:44.343 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json-pretty 2021-02-10T11:44:44.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.755 7f570e32c700 1 Processor -- start 2021-02-10T11:44:44.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.755 7f570e32c700 1 -- start start 2021-02-10T11:44:44.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.756 7f570e32c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f57081b0470 0x7f57081b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:44.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.756 7f570e32c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f57080f20e0 0x7f57080f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:44.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.756 7f570e32c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57080f0b50 con 0x7f57080ef610 2021-02-10T11:44:44.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.756 7f570e32c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57080f0820 con 0x7f57080f20e0 2021-02-10T11:44:44.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.756 7f570e32c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57080f24e0 con 0x7f57081b0470 2021-02-10T11:44:44.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.756 7f570c949700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f57080f20e0 0x7f57080f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:44.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.756 7f57077fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f57080ef610 0x7f57080efa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60458/0 (socket says 172.21.2.101:60458) 2021-02-10T11:44:44.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.756 7f57077fe700 1 -- 172.21.2.101:0/3586558389 learned_addr learned my addr 172.21.2.101:0/3586558389 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:44.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.756 7f570c949700 1 -- 172.21.2.101:0/3586558389 >> v2:172.21.2.108:3300/0 conn(0x7f57081b0470 msgr2=0x7f57081b0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:44.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.756 7f570c949700 1 --2- 172.21.2.101:0/3586558389 >> v2:172.21.2.108:3300/0 conn(0x7f57081b0470 0x7f57081b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:44.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.756 7f570c949700 1 -- 172.21.2.101:0/3586558389 >> v1:172.21.2.101:6789/0 conn(0x7f57080ef610 legacy=0x7f57080efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:44.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.756 7f570c949700 1 -- 172.21.2.101:0/3586558389 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57080e52e0 con 0x7f57080f20e0 2021-02-10T11:44:44.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.756 7f570c949700 1 --2- 172.21.2.101:0/3586558389 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f57080f20e0 0x7f57080f0200 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 rx=0x7f5700006e30 tx=0x7f5700009de0).ready entity=mon.0 client_cookie=340c84b873897f85 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:44.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.756 7f5706ffd700 1 -- 172.21.2.101:0/3586558389 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f570000b300 con 0x7f57080f20e0 2021-02-10T11:44:44.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.756 7f5706ffd700 1 -- 172.21.2.101:0/3586558389 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570000b580 con 0x7f57080f20e0 2021-02-10T11:44:44.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.756 7f5706ffd700 1 -- 172.21.2.101:0/3586558389 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f570000baa0 con 0x7f57080f20e0 2021-02-10T11:44:44.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.756 7f570e32c700 1 -- 172.21.2.101:0/3586558389 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f57080f20e0 msgr2=0x7f57080f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:44.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.756 7f570e32c700 1 --2- 172.21.2.101:0/3586558389 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f57080f20e0 0x7f57080f0200 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 rx=0x7f5700006e30 tx=0x7f5700009de0).stop 2021-02-10T11:44:44.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.757 7f570e32c700 1 -- 172.21.2.101:0/3586558389 shutdown_connections 2021-02-10T11:44:44.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.757 7f570e32c700 1 --2- 172.21.2.101:0/3586558389 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f57080f20e0 0x7f57080f0200 unknown :-1 s=CLOSED pgs=2714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:44.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.757 7f570e32c700 1 --2- 172.21.2.101:0/3586558389 >> v2:172.21.2.108:3300/0 conn(0x7f57081b0470 0x7f57081b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:44.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.757 7f570e32c700 1 -- 172.21.2.101:0/3586558389 >> 172.21.2.101:0/3586558389 conn(0x7f57080ce3e0 msgr2=0x7f57080e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:44.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.757 7f570e32c700 1 -- 172.21.2.101:0/3586558389 shutdown_connections 2021-02-10T11:44:44.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.757 7f570e32c700 1 -- 172.21.2.101:0/3586558389 wait complete. 2021-02-10T11:44:44.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.772 7f570e32c700 1 Processor -- start 2021-02-10T11:44:44.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.772 7f570e32c700 1 -- start start 2021-02-10T11:44:44.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.772 7f570e32c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5708187e20 0x7f5708188220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:44.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.772 7f570e32c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5708188840 0x7f570818d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:44.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.772 7f570e32c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f570818dcc0 0x7f570818e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:44.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.772 7f570e32c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57080a9330 con 0x7f5708187e20 2021-02-10T11:44:44.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.772 7f570e32c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5708191310 con 0x7f5708188840 2021-02-10T11:44:44.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.772 7f570e32c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570809da00 con 0x7f570818dcc0 2021-02-10T11:44:44.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.773 7f5707fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5708187e20 0x7f5708188220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:44.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.773 7f5707fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5708187e20 0x7f5708188220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59970/0 (socket says 172.21.2.101:59970) 2021-02-10T11:44:44.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.773 7f5707fff700 1 -- 172.21.2.101:0/2151690926 learned_addr learned my addr 172.21.2.101:0/2151690926 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:44.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.773 7f57077fe700 1 --2- 172.21.2.101:0/2151690926 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5708188840 0x7f570818d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:44.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.773 7f570c949700 1 --2- 172.21.2.101:0/2151690926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f570818dcc0 0x7f570818e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:44.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.773 7f5707fff700 1 -- 172.21.2.101:0/2151690926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f570818dcc0 msgr2=0x7f570818e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:44.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.773 7f5707fff700 1 --2- 172.21.2.101:0/2151690926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f570818dcc0 0x7f570818e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:44.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.773 7f5707fff700 1 -- 172.21.2.101:0/2151690926 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5708188840 msgr2=0x7f570818d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:44.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.773 7f5707fff700 1 --2- 172.21.2.101:0/2151690926 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5708188840 0x7f570818d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:44.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.773 7f5707fff700 1 -- 172.21.2.101:0/2151690926 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5708188c40 con 0x7f5708187e20 2021-02-10T11:44:44.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.773 7f57077fe700 1 --2- 172.21.2.101:0/2151690926 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5708188840 0x7f570818d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:44:44.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.773 7f570c949700 1 --2- 172.21.2.101:0/2151690926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f570818dcc0 0x7f570818e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:44:44.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.773 7f5707fff700 1 --2- 172.21.2.101:0/2151690926 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5708187e20 0x7f5708188220 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 rx=0x7f56f800b5c0 tx=0x7f56f800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:44.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.773 7f5704ff9700 1 -- 172.21.2.101:0/2151690926 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f56f8006b80 con 0x7f5708187e20 2021-02-10T11:44:44.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.773 7f5704ff9700 1 -- 172.21.2.101:0/2151690926 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f800bd50 con 0x7f5708187e20 2021-02-10T11:44:44.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.773 7f570e32c700 1 -- 172.21.2.101:0/2151690926 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5708188c40 con 0x7f5708187e20 2021-02-10T11:44:44.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.773 7f5704ff9700 1 -- 172.21.2.101:0/2151690926 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f56f80054d0 con 0x7f5708187e20 2021-02-10T11:44:44.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.773 7f570e32c700 1 -- 172.21.2.101:0/2151690926 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5708188c40 con 0x7f5708187e20 2021-02-10T11:44:44.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.774 7f5704ff9700 1 -- 172.21.2.101:0/2151690926 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f56f8006030 con 0x7f5708187e20 2021-02-10T11:44:44.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.774 7f5704ff9700 1 --2- 172.21.2.101:0/2151690926 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56e8018730 0x7f56e801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:44.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.774 7f5704ff9700 1 -- 172.21.2.101:0/2151690926 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f56f8032690 con 0x7f5708187e20 2021-02-10T11:44:44.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.774 7f57077fe700 1 --2- 172.21.2.101:0/2151690926 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56e8018730 0x7f56e801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:44.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.774 7f57077fe700 1 --2- 172.21.2.101:0/2151690926 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56e8018730 0x7f56e801ac10 crc :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:44.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.805 7f570e32c700 1 -- 172.21.2.101:0/2151690926 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57080a9330 con 0x7f5708187e20 2021-02-10T11:44:44.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.808 7f5704ff9700 1 -- 172.21.2.101:0/2151690926 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f56f8006b80 con 0x7f5708187e20 2021-02-10T11:44:44.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.933 7f570e32c700 1 -- 172.21.2.101:0/2151690926 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f57080a9330 con 0x7f5708187e20 2021-02-10T11:44:44.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.934 7f5704ff9700 1 -- 172.21.2.101:0/2151690926 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v147) v1 ==== 81+0+30374 (secure 0 0 0) 0x7f56f8006030 con 0x7f5708187e20 2021-02-10T11:44:44.934 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:44:44.934 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:44:44.934 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 147, 2021-02-10T11:44:44.934 INFO:teuthology.orchestra.run.gibba001.stdout: "fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:44:44.935 INFO:teuthology.orchestra.run.gibba001.stdout: "created": "2021-02-10 11:12:36.788366", 2021-02-10T11:44:44.935 INFO:teuthology.orchestra.run.gibba001.stdout: "modified": "2021-02-10 11:44:09.465154", 2021-02-10T11:44:44.935 INFO:teuthology.orchestra.run.gibba001.stdout: "last_up_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:44:44.935 INFO:teuthology.orchestra.run.gibba001.stdout: "last_in_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:44:44.935 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-02-10T11:44:44.935 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_num": 5799936, 2021-02-10T11:44:44.935 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_set": [ 2021-02-10T11:44:44.935 INFO:teuthology.orchestra.run.gibba001.stdout: "pglog_hardlimit", 2021-02-10T11:44:44.935 INFO:teuthology.orchestra.run.gibba001.stdout: "purged_snapdirs", 2021-02-10T11:44:44.935 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_deletes", 2021-02-10T11:44:44.935 INFO:teuthology.orchestra.run.gibba001.stdout: "sortbitwise" 2021-02-10T11:44:44.935 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:44:44.935 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_version": 4, 2021-02-10T11:44:44.936 INFO:teuthology.orchestra.run.gibba001.stdout: "full_ratio": 0.94999998807907104, 2021-02-10T11:44:44.936 INFO:teuthology.orchestra.run.gibba001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-02-10T11:44:44.936 INFO:teuthology.orchestra.run.gibba001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-02-10T11:44:44.936 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_snapshot": "", 2021-02-10T11:44:44.936 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_max": 31, 2021-02-10T11:44:44.936 INFO:teuthology.orchestra.run.gibba001.stdout: "max_osd": 8, 2021-02-10T11:44:44.936 INFO:teuthology.orchestra.run.gibba001.stdout: "require_min_compat_client": "jewel", 2021-02-10T11:44:44.936 INFO:teuthology.orchestra.run.gibba001.stdout: "min_compat_client": "jewel", 2021-02-10T11:44:44.936 INFO:teuthology.orchestra.run.gibba001.stdout: "require_osd_release": "nautilus", 2021-02-10T11:44:44.936 INFO:teuthology.orchestra.run.gibba001.stdout: "pools": [ 2021-02-10T11:44:44.936 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.936 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 1, 2021-02-10T11:44:44.936 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "rbd", 2021-02-10T11:44:44.937 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:13:08.037472", 2021-02-10T11:44:44.937 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:44:44.937 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:44:44.937 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:44:44.937 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:44:44.937 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:44:44.937 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:44:44.937 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:44:44.937 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:44:44.937 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:44:44.937 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:44:44.937 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:44:44.938 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:44:44.938 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:44:44.938 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:44:44.938 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:44:44.938 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:44:44.938 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:44:44.938 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:44:44.938 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:44:44.938 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:44:44.938 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.938 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "16", 2021-02-10T11:44:44.938 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:44:44.939 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:44:44.939 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:44:44.939 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:44:44.939 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:44:44.939 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:44:44.939 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:44:44.939 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:44:44.939 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:44:44.939 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:44:44.939 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:44:44.939 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:44:44.939 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:44:44.939 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:44:44.940 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:44:44.940 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:44:44.940 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:44:44.940 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:44:44.940 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:44:44.940 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:44:44.940 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:44:44.940 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:44:44.940 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:44:44.940 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:44:44.940 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:44:44.940 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:44:44.940 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.941 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:44:44.941 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:44:44.941 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:44:44.941 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:44:44.941 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:44:44.941 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:44:44.941 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:44:44.941 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:44:44.941 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:44:44.941 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:44:44.941 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:44:44.941 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:44:44.941 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:44:44.942 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd": {} 2021-02-10T11:44:44.942 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.942 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.942 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.942 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 30, 2021-02-10T11:44:44.942 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_metadata", 2021-02-10T11:44:44.942 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:44:06.431537", 2021-02-10T11:44:44.942 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:44:44.942 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:44:44.942 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:44:44.942 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:44:44.943 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:44:44.943 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:44:44.943 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:44:44.943 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:44:44.943 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:44:44.943 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:44:44.943 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:44:44.943 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:44:44.943 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:44:44.943 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:44:44.943 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:44:44.943 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:44:44.943 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:44:44.944 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:44:44.944 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:44:44.944 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:44:44.944 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.944 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "146", 2021-02-10T11:44:44.944 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:44:44.944 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:44:44.944 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:44:44.944 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:44:44.944 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:44:44.944 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:44:44.944 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:44:44.944 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:44:44.945 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:44:44.945 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:44:44.945 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:44:44.945 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:44:44.945 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:44:44.945 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:44:44.945 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:44:44.945 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:44:44.945 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:44:44.945 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:44:44.945 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:44:44.945 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:44:44.945 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:44:44.946 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:44:44.946 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:44:44.946 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:44:44.946 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:44:44.946 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:44:44.946 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.946 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:44:44.946 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:44:44.946 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:44:44.946 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:44:44.946 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:44:44.946 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:44:44.946 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:44:44.947 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:44:44.947 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:44:44.947 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:44:44.947 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:44:44.947 INFO:teuthology.orchestra.run.gibba001.stdout: "options": { 2021-02-10T11:44:44.947 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_bias": 4, 2021-02-10T11:44:44.947 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_min": 16, 2021-02-10T11:44:44.947 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_priority": 5 2021-02-10T11:44:44.947 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.947 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:44:44.947 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:44:44.947 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": "cephfs" 2021-02-10T11:44:44.947 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.948 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.948 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.948 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.948 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 31, 2021-02-10T11:44:44.948 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_data", 2021-02-10T11:44:44.948 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:44:07.456920", 2021-02-10T11:44:44.948 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:44:44.948 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:44:44.948 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:44:44.948 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:44:44.948 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:44:44.948 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:44:44.948 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:44:44.949 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:44:44.949 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:44:44.949 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:44:44.949 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:44:44.949 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:44:44.949 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:44:44.949 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:44:44.949 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:44:44.949 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:44:44.949 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:44:44.949 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:44:44.949 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:44:44.949 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:44:44.949 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.950 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "147", 2021-02-10T11:44:44.950 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:44:44.950 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:44:44.950 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:44:44.950 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:44:44.950 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:44:44.950 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:44:44.950 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:44:44.950 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:44:44.950 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:44:44.950 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:44:44.950 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:44:44.950 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:44:44.951 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:44:44.951 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:44:44.951 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:44:44.951 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:44:44.951 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:44:44.951 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:44:44.951 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:44:44.951 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:44:44.951 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:44:44.951 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:44:44.951 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:44:44.951 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:44:44.951 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:44:44.952 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:44:44.952 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.952 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:44:44.952 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:44:44.952 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:44:44.952 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:44:44.952 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:44:44.952 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:44:44.952 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:44:44.952 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:44:44.952 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:44:44.952 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:44:44.952 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:44:44.952 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:44:44.953 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:44:44.953 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:44:44.953 INFO:teuthology.orchestra.run.gibba001.stdout: "data": "cephfs" 2021-02-10T11:44:44.953 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.953 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.953 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.953 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:44:44.953 INFO:teuthology.orchestra.run.gibba001.stdout: "osds": [ 2021-02-10T11:44:44.953 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.953 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:44:44.953 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5", 2021-02-10T11:44:44.954 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:44:44.954 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:44:44.954 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:44:44.954 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:44:44.954 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:44:44.954 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:44:44.954 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:44:44.954 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 145, 2021-02-10T11:44:44.954 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:44:44.954 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:44:44.954 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:44:44.954 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.955 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.955 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.955 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6802", 2021-02-10T11:44:44.955 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:44:44.955 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.955 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.955 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.955 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6805", 2021-02-10T11:44:44.955 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:44:44.955 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.955 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.955 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.955 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:44:44.955 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.956 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.956 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.956 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6807", 2021-02-10T11:44:44.956 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:44:44.956 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.956 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.956 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.956 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6809", 2021-02-10T11:44:44.956 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:44:44.956 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.956 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.956 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.956 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:44:44.957 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.957 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.957 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.957 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6817", 2021-02-10T11:44:44.957 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:44:44.957 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.957 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.957 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.957 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6820", 2021-02-10T11:44:44.957 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:44:44.957 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.957 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.957 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.958 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:44:44.958 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.958 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.958 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.958 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6812", 2021-02-10T11:44:44.958 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:44:44.958 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.958 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.958 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.958 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6814", 2021-02-10T11:44:44.958 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:44:44.958 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.958 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.959 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.959 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6805/25093", 2021-02-10T11:44:44.959 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6809/25093", 2021-02-10T11:44:44.959 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6820/25093", 2021-02-10T11:44:44.959 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6814/25093", 2021-02-10T11:44:44.959 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:44:44.959 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:44:44.959 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:44:44.959 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.959 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.959 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.959 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:44:44.960 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "ddc04ce3-b313-4707-8270-585391e38ecc", 2021-02-10T11:44:44.960 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:44:44.960 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:44:44.960 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:44:44.960 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:44:44.960 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:44:44.960 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:44:44.960 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:44:44.960 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 145, 2021-02-10T11:44:44.960 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:44:44.960 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:44:44.960 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:44:44.960 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.960 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.961 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.961 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6803", 2021-02-10T11:44:44.961 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:44:44.961 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.961 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.961 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.961 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6804", 2021-02-10T11:44:44.961 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:44:44.961 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.961 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.961 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.962 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:44:44.962 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.962 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.962 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.962 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6806", 2021-02-10T11:44:44.962 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:44:44.962 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.962 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.962 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.962 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6808", 2021-02-10T11:44:44.962 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:44:44.963 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.963 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.963 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.963 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:44:44.963 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.963 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.963 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.963 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6818", 2021-02-10T11:44:44.963 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:44:44.963 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.963 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.963 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.964 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6821", 2021-02-10T11:44:44.964 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:44:44.964 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.964 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.964 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.964 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:44:44.964 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.964 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.964 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.964 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6810", 2021-02-10T11:44:44.964 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:44:44.964 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.964 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.964 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.965 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6815", 2021-02-10T11:44:44.965 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:44:44.965 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.965 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.965 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.965 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6804/25092", 2021-02-10T11:44:44.965 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6808/25092", 2021-02-10T11:44:44.965 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6821/25092", 2021-02-10T11:44:44.965 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6815/25092", 2021-02-10T11:44:44.965 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:44:44.965 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:44:44.965 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:44:44.965 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.966 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.966 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.966 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:44:44.966 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3", 2021-02-10T11:44:44.966 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:44:44.966 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:44:44.966 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:44:44.966 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:44:44.966 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:44:44.966 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:44:44.966 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:44:44.966 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 143, 2021-02-10T11:44:44.966 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:44:44.967 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:44:44.967 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:44:44.967 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.967 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.967 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.967 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6811", 2021-02-10T11:44:44.967 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:44:44.967 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.967 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.967 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.967 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6813", 2021-02-10T11:44:44.967 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:44:44.967 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.968 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.968 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.968 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:44:44.968 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.968 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.968 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.968 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6816", 2021-02-10T11:44:44.968 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:44:44.968 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.968 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.968 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.968 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6819", 2021-02-10T11:44:44.968 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:44:44.969 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.969 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.969 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.969 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:44:44.969 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.969 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.969 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.969 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6825", 2021-02-10T11:44:44.969 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:44:44.969 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.969 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.969 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.969 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6827", 2021-02-10T11:44:44.970 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:44:44.970 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.970 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.970 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.970 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:44:44.970 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.970 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.970 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.970 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6822", 2021-02-10T11:44:44.970 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:44:44.970 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.970 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.970 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.971 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6823", 2021-02-10T11:44:44.971 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:44:44.971 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.971 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.971 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.971 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6813/25094", 2021-02-10T11:44:44.971 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6819/25094", 2021-02-10T11:44:44.971 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6827/25094", 2021-02-10T11:44:44.971 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6823/25094", 2021-02-10T11:44:44.971 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:44:44.971 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:44:44.971 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:44:44.971 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.972 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.972 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.972 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:44:44.972 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "0d9ef9b1-98aa-4d37-aa16-6305c7d46d52", 2021-02-10T11:44:44.972 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:44:44.972 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:44:44.972 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:44:44.972 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:44:44.972 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:44:44.972 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:44:44.972 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:44:44.972 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 144, 2021-02-10T11:44:44.972 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:44:44.973 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:44:44.973 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:44:44.973 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.973 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.973 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.973 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6824", 2021-02-10T11:44:44.973 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:44:44.973 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.973 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.973 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.973 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6826", 2021-02-10T11:44:44.973 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:44:44.973 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.974 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.974 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.974 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:44:44.974 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.974 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.974 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.974 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6828", 2021-02-10T11:44:44.974 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:44:44.974 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.974 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.974 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.974 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6829", 2021-02-10T11:44:44.975 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:44:44.975 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.975 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.975 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.975 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:44:44.975 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.975 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.975 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.975 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6832", 2021-02-10T11:44:44.975 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:44:44.975 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.975 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.976 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.976 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6833", 2021-02-10T11:44:44.976 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:44:44.976 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.976 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.976 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.976 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:44:44.976 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.976 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.976 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.976 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6830", 2021-02-10T11:44:44.976 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:44:44.977 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.977 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.977 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.977 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6831", 2021-02-10T11:44:44.977 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:44:44.977 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.977 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.977 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.977 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6826/25095", 2021-02-10T11:44:44.977 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6829/25095", 2021-02-10T11:44:44.977 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6833/25095", 2021-02-10T11:44:44.977 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6831/25095", 2021-02-10T11:44:44.977 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:44:44.978 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:44:44.978 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:44:44.978 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.978 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.978 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.978 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:44:44.978 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "36742c21-bfeb-4e11-bce9-8e745378b605", 2021-02-10T11:44:44.978 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:44:44.978 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:44:44.978 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:44:44.978 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:44:44.978 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:44:44.978 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:44:44.979 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:44:44.979 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 145, 2021-02-10T11:44:44.979 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:44:44.979 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:44:44.979 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:44:44.979 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.979 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.979 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.979 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6816", 2021-02-10T11:44:44.979 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:44:44.979 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.979 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.979 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.980 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6818", 2021-02-10T11:44:44.980 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:44:44.980 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.980 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.980 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.980 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:44:44.980 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.980 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.980 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.980 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6820", 2021-02-10T11:44:44.980 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:44:44.980 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.980 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.981 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.981 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6822", 2021-02-10T11:44:44.981 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:44:44.981 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.981 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.981 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.981 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:44:44.981 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.981 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.981 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.981 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6828", 2021-02-10T11:44:44.981 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:44:44.981 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.981 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.982 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.982 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6830", 2021-02-10T11:44:44.982 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:44:44.982 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.982 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.982 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.982 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:44:44.982 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.982 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.982 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.982 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6824", 2021-02-10T11:44:44.982 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:44:44.982 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.983 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.983 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.983 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6826", 2021-02-10T11:44:44.983 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:44:44.983 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.983 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.983 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.983 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6818/36380", 2021-02-10T11:44:44.983 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6822/36380", 2021-02-10T11:44:44.983 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6830/36380", 2021-02-10T11:44:44.983 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6826/36380", 2021-02-10T11:44:44.983 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:44:44.983 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:44:44.984 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:44:44.984 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.984 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.984 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.984 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:44:44.984 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "88e694e8-fe98-4d1b-88c8-8c9907ece60b", 2021-02-10T11:44:44.984 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:44:44.984 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:44:44.984 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:44:44.984 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:44:44.984 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:44:44.984 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:44:44.984 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:44:44.985 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 143, 2021-02-10T11:44:44.985 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:44:44.985 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:44:44.985 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:44:44.985 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.985 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.985 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.985 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6808", 2021-02-10T11:44:44.985 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:44:44.985 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.985 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.985 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.985 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6809", 2021-02-10T11:44:44.986 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:44:44.986 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.986 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.986 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.986 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:44:44.986 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.986 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.986 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.986 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6810", 2021-02-10T11:44:44.986 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:44:44.986 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.986 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.987 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.987 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6811", 2021-02-10T11:44:44.987 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:44:44.987 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.987 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.987 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.987 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:44:44.987 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.987 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.987 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.987 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6814", 2021-02-10T11:44:44.987 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:44:44.987 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.988 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.988 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.988 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6815", 2021-02-10T11:44:44.988 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:44:44.988 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.988 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.988 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.988 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:44:44.988 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.988 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.988 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.988 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6812", 2021-02-10T11:44:44.989 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:44:44.989 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.989 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.989 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.989 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6813", 2021-02-10T11:44:44.989 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:44:44.989 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.989 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.989 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.989 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6809/36383", 2021-02-10T11:44:44.989 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6811/36383", 2021-02-10T11:44:44.989 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6815/36383", 2021-02-10T11:44:44.989 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6813/36383", 2021-02-10T11:44:44.989 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:44:44.990 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:44:44.990 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:44:44.990 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.990 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.990 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.990 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:44:44.990 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "87a13c6c-cf85-4aaf-916b-e73913484a49", 2021-02-10T11:44:44.990 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:44:44.990 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:44:44.990 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:44:44.990 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:44:44.990 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:44:44.991 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:44:44.991 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:44:44.991 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 145, 2021-02-10T11:44:44.991 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:44:44.991 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:44:44.991 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:44:44.991 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.991 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.991 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.991 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6800", 2021-02-10T11:44:44.991 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:44:44.991 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.991 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.992 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.992 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6801", 2021-02-10T11:44:44.992 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:44:44.992 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.992 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.992 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.992 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:44:44.992 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.992 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.992 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.992 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6802", 2021-02-10T11:44:44.992 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:44:44.992 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.992 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.993 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.993 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6803", 2021-02-10T11:44:44.993 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:44:44.993 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.993 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.993 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.993 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:44:44.993 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.993 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.993 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.993 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6806", 2021-02-10T11:44:44.993 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:44:44.993 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.994 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.994 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.994 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6807", 2021-02-10T11:44:44.994 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:44:44.994 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.994 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.994 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.994 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:44:44.994 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.994 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.994 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.994 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6804", 2021-02-10T11:44:44.994 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:44:44.995 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.995 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.995 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.995 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6805", 2021-02-10T11:44:44.995 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:44:44.995 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.995 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.995 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.995 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6801/36381", 2021-02-10T11:44:44.995 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6803/36381", 2021-02-10T11:44:44.995 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6807/36381", 2021-02-10T11:44:44.995 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6805/36381", 2021-02-10T11:44:44.995 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:44:44.996 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:44:44.996 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:44:44.996 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.996 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.996 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.996 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:44:44.996 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "7b90823f-854c-4592-a95b-a7fa98f93254", 2021-02-10T11:44:44.996 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:44:44.996 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:44:44.996 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:44:44.996 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:44:44.996 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:44:44.996 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:44:44.996 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:44:44.997 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 144, 2021-02-10T11:44:44.997 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:44:44.997 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:44:44.997 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:44:44.997 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.997 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.997 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.997 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6817", 2021-02-10T11:44:44.997 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:44:44.997 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.997 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.997 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.997 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6819", 2021-02-10T11:44:44.998 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:44:44.998 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.998 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.998 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.998 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:44:44.998 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.998 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.998 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.998 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6821", 2021-02-10T11:44:44.998 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:44:44.998 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.998 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.998 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:44.999 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6823", 2021-02-10T11:44:44.999 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:44:44.999 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:44.999 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:44.999 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.999 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:44:44.999 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:44.999 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:44.999 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:44.999 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6829", 2021-02-10T11:44:44.999 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:44:44.999 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:44.999 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:45.000 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:45.000 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6831", 2021-02-10T11:44:45.000 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:44:45.000 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:45.000 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:45.000 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:45.000 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:44:45.000 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:44:45.000 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:45.000 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:44:45.000 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6825", 2021-02-10T11:44:45.000 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:44:45.000 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:45.000 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:45.001 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:44:45.001 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6827", 2021-02-10T11:44:45.001 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:44:45.001 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:45.001 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:45.001 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:45.001 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6819/36382", 2021-02-10T11:44:45.001 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6823/36382", 2021-02-10T11:44:45.001 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6831/36382", 2021-02-10T11:44:45.001 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6827/36382", 2021-02-10T11:44:45.001 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:44:45.001 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:44:45.001 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:44:45.002 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:45.002 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:45.002 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:44:45.002 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_xinfo": [ 2021-02-10T11:44:45.002 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:45.002 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:44:45.002 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:44:45.002 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:44:45.002 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:44:45.002 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:44:45.002 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:44:45.002 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:45.002 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:45.003 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:44:45.003 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:44:45.003 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:44:45.003 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:44:45.003 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:44:45.003 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:44:45.003 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:45.003 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:45.003 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:44:45.003 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:44:45.003 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:44:45.003 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:44:45.003 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:44:45.004 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:44:45.004 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:45.004 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:45.004 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:44:45.004 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:44:45.004 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:44:45.004 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:44:45.004 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:44:45.004 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:44:45.004 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:45.004 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:45.004 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:44:45.004 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:44:45.005 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:44:45.005 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:44:45.005 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:44:45.005 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:44:45.005 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:45.005 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:45.005 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:44:45.005 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:44:45.005 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:44:45.005 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:44:45.005 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:44:45.005 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:44:45.005 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:45.006 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:45.006 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:44:45.006 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:44:45.006 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:44:45.006 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:44:45.006 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:44:45.006 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:44:45.006 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:45.006 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:45.006 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:44:45.006 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:44:45.006 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:44:45.007 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:44:45.007 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:44:45.007 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:44:45.007 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:45.007 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:44:45.007 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap": [], 2021-02-10T11:44:45.007 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap_items": [], 2021-02-10T11:44:45.007 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_temp": [], 2021-02-10T11:44:45.007 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_temp": [], 2021-02-10T11:44:45.007 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklist": {}, 2021-02-10T11:44:45.007 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profiles": { 2021-02-10T11:44:45.007 INFO:teuthology.orchestra.run.gibba001.stdout: "default": { 2021-02-10T11:44:45.008 INFO:teuthology.orchestra.run.gibba001.stdout: "crush-failure-domain": "osd", 2021-02-10T11:44:45.008 INFO:teuthology.orchestra.run.gibba001.stdout: "k": "2", 2021-02-10T11:44:45.008 INFO:teuthology.orchestra.run.gibba001.stdout: "m": "1", 2021-02-10T11:44:45.008 INFO:teuthology.orchestra.run.gibba001.stdout: "plugin": "jerasure", 2021-02-10T11:44:45.008 INFO:teuthology.orchestra.run.gibba001.stdout: "ruleset-failure-domain": "osd", 2021-02-10T11:44:45.008 INFO:teuthology.orchestra.run.gibba001.stdout: "technique": "reed_sol_van" 2021-02-10T11:44:45.008 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:45.008 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:45.008 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps_queue": [], 2021-02-10T11:44:45.008 INFO:teuthology.orchestra.run.gibba001.stdout: "new_removed_snaps": [], 2021-02-10T11:44:45.008 INFO:teuthology.orchestra.run.gibba001.stdout: "new_purged_snaps": [], 2021-02-10T11:44:45.008 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_node_flags": {}, 2021-02-10T11:44:45.008 INFO:teuthology.orchestra.run.gibba001.stdout: "device_class_flags": {} 2021-02-10T11:44:45.008 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:44:45.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.935 7f570e32c700 1 -- 172.21.2.101:0/2151690926 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56e8018730 msgr2=0x7f56e801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:45.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.935 7f570e32c700 1 --2- 172.21.2.101:0/2151690926 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56e8018730 0x7f56e801ac10 crc :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:45.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.936 7f570e32c700 1 -- 172.21.2.101:0/2151690926 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5708187e20 msgr2=0x7f5708188220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:45.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.936 7f570e32c700 1 --2- 172.21.2.101:0/2151690926 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5708187e20 0x7f5708188220 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 rx=0x7f56f800b5c0 tx=0x7f56f800b960).stop 2021-02-10T11:44:45.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.936 7f570e32c700 1 -- 172.21.2.101:0/2151690926 shutdown_connections 2021-02-10T11:44:45.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.936 7f570e32c700 1 --2- 172.21.2.101:0/2151690926 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56e8018730 0x7f56e801ac10 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:45.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.936 7f570e32c700 1 --2- 172.21.2.101:0/2151690926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f570818dcc0 0x7f570818e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:45.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.936 7f570e32c700 1 --2- 172.21.2.101:0/2151690926 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5708188840 0x7f570818d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:45.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.936 7f570e32c700 1 --2- 172.21.2.101:0/2151690926 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5708187e20 0x7f5708188220 unknown :-1 s=CLOSED pgs=2715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:45.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.936 7f570e32c700 1 -- 172.21.2.101:0/2151690926 >> 172.21.2.101:0/2151690926 conn(0x7f5708090d90 msgr2=0x7f57081b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:45.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.936 7f570e32c700 1 -- 172.21.2.101:0/2151690926 shutdown_connections 2021-02-10T11:44:45.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:44.936 7f570e32c700 1 -- 172.21.2.101:0/2151690926 wait complete. 2021-02-10T11:44:45.011 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:44:45.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.425 7f2036e4f700 1 Processor -- start 2021-02-10T11:44:45.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.425 7f2036e4f700 1 -- start start 2021-02-10T11:44:45.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.426 7f2036e4f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20300d0340 0x7f20300ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:45.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.426 7f2036e4f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f20300d1c70 0x7f20300efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:45.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.426 7f2036e4f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20300f0b10 con 0x7f20300d1870 2021-02-10T11:44:45.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.426 7f2036e4f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20300e5ed0 con 0x7f20300d1c70 2021-02-10T11:44:45.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.426 7f2036e4f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20301b0be0 con 0x7f20300d0340 2021-02-10T11:44:45.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.426 7f203546c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f20300d1c70 0x7f20300efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:45.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.426 7f202ffff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20300d0340 0x7f20300ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:45.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.426 7f2034c6b700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f20300d1870 0x7f20301b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:53882/0 (socket says 172.21.2.101:53882) 2021-02-10T11:44:45.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.426 7f2034c6b700 1 -- 172.21.2.101:0/2817522838 learned_addr learned my addr 172.21.2.101:0/2817522838 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:45.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.427 7f202ffff700 1 -- 172.21.2.101:0/2817522838 >> v2:172.21.2.108:3300/0 conn(0x7f20300d1c70 msgr2=0x7f20300efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:45.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.427 7f202ffff700 1 --2- 172.21.2.101:0/2817522838 >> v2:172.21.2.108:3300/0 conn(0x7f20300d1c70 0x7f20300efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:45.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.427 7f202ffff700 1 -- 172.21.2.101:0/2817522838 >> v1:172.21.2.108:6789/0 conn(0x7f20300d1870 legacy=0x7f20301b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:45.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.427 7f202ffff700 1 -- 172.21.2.101:0/2817522838 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20300f20a0 con 0x7f20300d0340 2021-02-10T11:44:45.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.427 7f202ffff700 1 --2- 172.21.2.101:0/2817522838 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20300d0340 0x7f20300ef640 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f2020006650 tx=0x7f2020009a80).ready entity=mon.2 client_cookie=84f10bd746bec50e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:45.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.427 7f202f7fe700 1 -- 172.21.2.101:0/2817522838 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f20200009b0 con 0x7f20300d0340 2021-02-10T11:44:45.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.427 7f202f7fe700 1 -- 172.21.2.101:0/2817522838 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202000c0c0 con 0x7f20300d0340 2021-02-10T11:44:45.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.427 7f202f7fe700 1 -- 172.21.2.101:0/2817522838 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f202000c5e0 con 0x7f20300d0340 2021-02-10T11:44:45.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.427 7f2036e4f700 1 -- 172.21.2.101:0/2817522838 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20300d0340 msgr2=0x7f20300ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:45.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.427 7f2036e4f700 1 --2- 172.21.2.101:0/2817522838 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20300d0340 0x7f20300ef640 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f2020006650 tx=0x7f2020009a80).stop 2021-02-10T11:44:45.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.428 7f2036e4f700 1 -- 172.21.2.101:0/2817522838 shutdown_connections 2021-02-10T11:44:45.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.428 7f2036e4f700 1 --2- 172.21.2.101:0/2817522838 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20300d0340 0x7f20300ef640 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:45.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.428 7f2036e4f700 1 --2- 172.21.2.101:0/2817522838 >> v2:172.21.2.108:3300/0 conn(0x7f20300d1c70 0x7f20300efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:45.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.428 7f2036e4f700 1 -- 172.21.2.101:0/2817522838 >> 172.21.2.101:0/2817522838 conn(0x7f20300ce450 msgr2=0x7f20300d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:45.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.428 7f2036e4f700 1 -- 172.21.2.101:0/2817522838 shutdown_connections 2021-02-10T11:44:45.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.428 7f2036e4f700 1 -- 172.21.2.101:0/2817522838 wait complete. 2021-02-10T11:44:45.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.442 7f2036e4f700 1 Processor -- start 2021-02-10T11:44:45.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.442 7f2036e4f700 1 -- start start 2021-02-10T11:44:45.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.443 7f2036e4f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20300936a0 0x7f203019fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:45.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.443 7f2036e4f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2030092e10 0x7f20300ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:45.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.443 7f2036e4f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f20300aee90 0x7f2030093b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:45.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.443 7f2036e4f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2030093210 con 0x7f2030092e10 2021-02-10T11:44:45.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.443 7f2036e4f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2030094190 con 0x7f20300aee90 2021-02-10T11:44:45.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.443 7f2036e4f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20300943b0 con 0x7f20300936a0 2021-02-10T11:44:45.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.443 7f202ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2030092e10 0x7f20300ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:45.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.443 7f202ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2030092e10 0x7f20300ae870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:59984/0 (socket says 172.21.2.101:59984) 2021-02-10T11:44:45.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.443 7f202ffff700 1 -- 172.21.2.101:0/1046245163 learned_addr learned my addr 172.21.2.101:0/1046245163 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:45.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.444 7f202ffff700 1 -- 172.21.2.101:0/1046245163 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20300936a0 msgr2=0x7f203019fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:45.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.444 7f202ffff700 1 --2- 172.21.2.101:0/1046245163 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20300936a0 0x7f203019fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:45.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.444 7f202ffff700 1 -- 172.21.2.101:0/1046245163 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f20300aee90 msgr2=0x7f2030093b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:45.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.444 7f202ffff700 1 --2- 172.21.2.101:0/1046245163 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f20300aee90 0x7f2030093b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:45.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.444 7f202ffff700 1 -- 172.21.2.101:0/1046245163 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2030191aa0 con 0x7f2030092e10 2021-02-10T11:44:45.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.444 7f202ffff700 1 --2- 172.21.2.101:0/1046245163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2030092e10 0x7f20300ae870 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 rx=0x7f2020009630 tx=0x7f202000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:45.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.444 7f202d7fa700 1 -- 172.21.2.101:0/1046245163 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f20200093a0 con 0x7f2030092e10 2021-02-10T11:44:45.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.444 7f202d7fa700 1 -- 172.21.2.101:0/1046245163 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202000bd70 con 0x7f2030092e10 2021-02-10T11:44:45.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.444 7f2036e4f700 1 -- 172.21.2.101:0/1046245163 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2030191aa0 con 0x7f2030092e10 2021-02-10T11:44:45.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.444 7f2036e4f700 1 -- 172.21.2.101:0/1046245163 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2030192030 con 0x7f2030092e10 2021-02-10T11:44:45.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.444 7f202d7fa700 1 -- 172.21.2.101:0/1046245163 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f20200054d0 con 0x7f2030092e10 2021-02-10T11:44:45.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.445 7f202d7fa700 1 -- 172.21.2.101:0/1046245163 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f202000a000 con 0x7f2030092e10 2021-02-10T11:44:45.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.445 7f202d7fa700 1 --2- 172.21.2.101:0/1046245163 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f20140186c0 0x7f201401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:45.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.445 7f202d7fa700 1 -- 172.21.2.101:0/1046245163 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f20200325b0 con 0x7f2030092e10 2021-02-10T11:44:45.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.445 7f2034c6b700 1 --2- 172.21.2.101:0/1046245163 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f20140186c0 0x7f201401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:45.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.445 7f2034c6b700 1 --2- 172.21.2.101:0/1046245163 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f20140186c0 0x7f201401abc0 crc :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:45.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.475 7f2036e4f700 1 -- 172.21.2.101:0/1046245163 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20300ebc60 con 0x7f2030092e10 2021-02-10T11:44:45.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.478 7f202d7fa700 1 -- 172.21.2.101:0/1046245163 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f202002f250 con 0x7f2030092e10 2021-02-10T11:44:45.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.605 7f2036e4f700 1 -- 172.21.2.101:0/1046245163 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20300f0b10 con 0x7f2030092e10 2021-02-10T11:44:45.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.606 7f202d7fa700 1 -- 172.21.2.101:0/1046245163 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+2111 (secure 0 0 0) 0x7f202002f250 con 0x7f2030092e10 2021-02-10T11:44:45.604 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 124 2021-02-10T11:44:45.604 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:44:45.604 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":124,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/4208244473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":4208244473},{"type":"v1","addr":"172.21.2.101:6835","nonce":4208244473}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":119}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:44:08.462960","modified":"2021-02-10 11:44:10.472350","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11814":{"gid":11814,"name":"b","rank":0,"incarnation":120,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/3445995728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":3445995728},{"type":"v1","addr":"172.21.2.108:6833","nonce":3445995728}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-02-10T11:44:45.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.607 7f2036e4f700 1 -- 172.21.2.101:0/1046245163 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f20140186c0 msgr2=0x7f201401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:45.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.607 7f2036e4f700 1 --2- 172.21.2.101:0/1046245163 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f20140186c0 0x7f201401abc0 crc :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:45.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.607 7f2036e4f700 1 -- 172.21.2.101:0/1046245163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2030092e10 msgr2=0x7f20300ae870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:45.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.607 7f2036e4f700 1 --2- 172.21.2.101:0/1046245163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2030092e10 0x7f20300ae870 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 rx=0x7f2020009630 tx=0x7f202000b980).stop 2021-02-10T11:44:45.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.608 7f2036e4f700 1 -- 172.21.2.101:0/1046245163 shutdown_connections 2021-02-10T11:44:45.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.608 7f2036e4f700 1 --2- 172.21.2.101:0/1046245163 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f20300aee90 0x7f2030093b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:45.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.608 7f2036e4f700 1 --2- 172.21.2.101:0/1046245163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2030092e10 0x7f20300ae870 unknown :-1 s=CLOSED pgs=2716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:45.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.608 7f2036e4f700 1 --2- 172.21.2.101:0/1046245163 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f20140186c0 0x7f201401abc0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:45.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.608 7f2036e4f700 1 --2- 172.21.2.101:0/1046245163 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20300936a0 0x7f203019fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:45.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.608 7f2036e4f700 1 -- 172.21.2.101:0/1046245163 >> 172.21.2.101:0/1046245163 conn(0x7f20301aeec0 msgr2=0x7f20301a37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:45.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.608 7f2036e4f700 1 -- 172.21.2.101:0/1046245163 shutdown_connections 2021-02-10T11:44:45.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:45.608 7f2036e4f700 1 -- 172.21.2.101:0/1046245163 wait complete. 2021-02-10T11:44:45.614 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs fail cephfs 2021-02-10T11:44:46.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.029 7f0760da4700 1 Processor -- start 2021-02-10T11:44:46.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.029 7f0760da4700 1 -- start start 2021-02-10T11:44:46.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.029 7f0760da4700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f075c0d0340 0x7f075c0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:46.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.029 7f0760da4700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f075c0f0b10 con 0x7f075c0d1870 2021-02-10T11:44:46.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.029 7f0760da4700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f075c1b0be0 con 0x7f075c0d1c70 2021-02-10T11:44:46.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.029 7f0760da4700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f075c0e5ed0 con 0x7f075c0d0340 2021-02-10T11:44:46.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.029 7f075ab13700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f075c0d1870 0x7f075c1b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60486/0 (socket says 172.21.2.101:60486) 2021-02-10T11:44:46.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.029 7f075ab13700 1 -- 172.21.2.101:0/739905799 learned_addr learned my addr 172.21.2.101:0/739905799 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:46.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.030 7f0759b11700 1 -- 172.21.2.101:0/739905799 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1973499391 0 0) 0x7f074c001180 con 0x7f075c0d1870 2021-02-10T11:44:46.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.030 7f0759b11700 1 -- 172.21.2.101:0/739905799 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0748002fe0 con 0x7f075c0d1870 2021-02-10T11:44:46.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.030 7f0759b11700 1 -- 172.21.2.101:0/739905799 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1350956416 0 0) 0x7f074c001180 con 0x7f075c0d1870 2021-02-10T11:44:46.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.030 7f075a312700 1 --2- 172.21.2.101:0/739905799 >> v2:172.21.2.108:3300/0 conn(0x7f075c0d0340 0x7f075c0ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:46.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.030 7f0759b11700 1 -- 172.21.2.101:0/739905799 >> v2:172.21.2.108:3300/0 conn(0x7f075c0d0340 msgr2=0x7f075c0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:46.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.030 7f0759b11700 1 --2- 172.21.2.101:0/739905799 >> v2:172.21.2.108:3300/0 conn(0x7f075c0d0340 0x7f075c0ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:46.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.030 7f0759b11700 1 -- 172.21.2.101:0/739905799 >> v1:172.21.2.108:6789/0 conn(0x7f075c0d1c70 legacy=0x7f075c0efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:46.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.030 7f0759b11700 1 -- 172.21.2.101:0/739905799 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075c0f0450 con 0x7f075c0d1870 2021-02-10T11:44:46.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.030 7f0759b11700 1 -- 172.21.2.101:0/739905799 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f074c001680 con 0x7f075c0d1870 2021-02-10T11:44:46.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.030 7f0759b11700 1 -- 172.21.2.101:0/739905799 >> v1:172.21.2.101:6789/0 conn(0x7f075c0d1870 legacy=0x7f075c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:46.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.030 7f0759b11700 1 --2- 172.21.2.101:0/739905799 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0748003af0 0x7f0748005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:46.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.030 7f0759b11700 1 --2- 172.21.2.101:0/739905799 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0748006820 0x7f0748008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:46.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.030 7f0759b11700 1 --2- 172.21.2.101:0/739905799 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0748009620 0x7f074800bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:46.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.030 7f0759b11700 1 -- 172.21.2.101:0/739905799 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074800c390 con 0x7f0748003af0 2021-02-10T11:44:46.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.030 7f0759b11700 1 -- 172.21.2.101:0/739905799 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074800c5b0 con 0x7f0748009620 2021-02-10T11:44:46.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.030 7f0759b11700 1 -- 172.21.2.101:0/739905799 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f074800c7d0 con 0x7f0748006820 2021-02-10T11:44:46.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.030 7f075a312700 1 --2- 172.21.2.101:0/739905799 >> v2:172.21.2.108:3300/0 conn(0x7f075c0d0340 0x7f075c0ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:44:46.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.030 7f075ab13700 1 --2- 172.21.2.101:0/739905799 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0748003af0 0x7f0748005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:46.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.030 7f075a312700 1 --2- 172.21.2.101:0/739905799 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0748006820 0x7f0748008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:46.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.031 7f075b314700 1 --2- 172.21.2.101:0/739905799 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0748009620 0x7f074800bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:46.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.031 7f075ab13700 1 -- 172.21.2.101:0/739905799 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0748006820 msgr2=0x7f0748008d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:46.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.031 7f075ab13700 1 --2- 172.21.2.101:0/739905799 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0748006820 0x7f0748008d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:46.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.031 7f075ab13700 1 -- 172.21.2.101:0/739905799 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0748009620 msgr2=0x7f074800bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:46.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.031 7f075ab13700 1 --2- 172.21.2.101:0/739905799 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0748009620 0x7f074800bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:46.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.031 7f075ab13700 1 -- 172.21.2.101:0/739905799 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074800ca70 con 0x7f0748003af0 2021-02-10T11:44:46.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.031 7f075b314700 1 --2- 172.21.2.101:0/739905799 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0748009620 0x7f074800bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:44:46.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.031 7f075b314700 1 -- 172.21.2.101:0/739905799 reap_dead start 2021-02-10T11:44:46.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.031 7f075ab13700 1 --2- 172.21.2.101:0/739905799 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0748003af0 0x7f0748005f80 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 rx=0x7f074c000eb0 tx=0x7f074c007d00).ready entity=mon.0 client_cookie=ef26f0738e506355 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:46.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.031 7f0759b11700 1 -- 172.21.2.101:0/739905799 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f074c000c90 con 0x7f0748003af0 2021-02-10T11:44:46.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.031 7f0759b11700 1 -- 172.21.2.101:0/739905799 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074c0070c0 con 0x7f0748003af0 2021-02-10T11:44:46.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.031 7f0759b11700 1 -- 172.21.2.101:0/739905799 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f074c0045b0 con 0x7f0748003af0 2021-02-10T11:44:46.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.031 7f0760da4700 1 -- 172.21.2.101:0/739905799 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0748003af0 msgr2=0x7f0748005f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:46.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.031 7f0760da4700 1 --2- 172.21.2.101:0/739905799 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0748003af0 0x7f0748005f80 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 rx=0x7f074c000eb0 tx=0x7f074c007d00).stop 2021-02-10T11:44:46.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.031 7f0760da4700 1 -- 172.21.2.101:0/739905799 shutdown_connections 2021-02-10T11:44:46.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.031 7f0760da4700 1 --2- 172.21.2.101:0/739905799 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0748003af0 0x7f0748005f80 unknown :-1 s=CLOSED pgs=2718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:46.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.031 7f0760da4700 1 -- 172.21.2.101:0/739905799 >> 172.21.2.101:0/739905799 conn(0x7f075c0ce450 msgr2=0x7f075c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:46.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.032 7f0760da4700 1 -- 172.21.2.101:0/739905799 shutdown_connections 2021-02-10T11:44:46.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.032 7f0760da4700 1 -- 172.21.2.101:0/739905799 wait complete. 2021-02-10T11:44:46.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.046 7f0760da4700 1 Processor -- start 2021-02-10T11:44:46.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.046 7f0760da4700 1 -- start start 2021-02-10T11:44:46.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.046 7f0760da4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f075c0a60d0 0x7f075c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:46.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.046 7f0760da4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f075c0a6560 0x7f075c0a6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:46.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.046 7f0760da4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f075c0ebec0 0x7f075c0ec2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:46.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.046 7f0760da4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075c08dba0 con 0x7f075c0ebec0 2021-02-10T11:44:46.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.046 7f0760da4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075c08e120 con 0x7f075c0a6560 2021-02-10T11:44:46.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.046 7f0760da4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f075c0a5bc0 con 0x7f075c0a60d0 2021-02-10T11:44:46.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.047 7f075b314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f075c0ebec0 0x7f075c0ec2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:46.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.047 7f075b314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f075c0ebec0 0x7f075c0ec2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60006/0 (socket says 172.21.2.101:60006) 2021-02-10T11:44:46.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.047 7f075b314700 1 -- 172.21.2.101:0/1915046834 learned_addr learned my addr 172.21.2.101:0/1915046834 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:46.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.047 7f075b314700 1 -- 172.21.2.101:0/1915046834 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f075c0a60d0 msgr2=0x7f075c0a54f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:46.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.047 7f075b314700 1 --2- 172.21.2.101:0/1915046834 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f075c0a60d0 0x7f075c0a54f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:46.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.047 7f075b314700 1 -- 172.21.2.101:0/1915046834 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f075c0a6560 msgr2=0x7f075c0a6960 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:46.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.047 7f075b314700 1 --2- 172.21.2.101:0/1915046834 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f075c0a6560 0x7f075c0a6960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:46.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.047 7f075b314700 1 -- 172.21.2.101:0/1915046834 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075c0ec8e0 con 0x7f075c0ebec0 2021-02-10T11:44:46.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.047 7f075b314700 1 --2- 172.21.2.101:0/1915046834 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f075c0ebec0 0x7f075c0ec2c0 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 rx=0x7f075400b8c0 tx=0x7f075400bc60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:46.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.047 7f07537fe700 1 -- 172.21.2.101:0/1915046834 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f075400c2f0 con 0x7f075c0ebec0 2021-02-10T11:44:46.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.047 7f07537fe700 1 -- 172.21.2.101:0/1915046834 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075400c570 con 0x7f075c0ebec0 2021-02-10T11:44:46.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.047 7f07537fe700 1 -- 172.21.2.101:0/1915046834 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f075400ca90 con 0x7f075c0ebec0 2021-02-10T11:44:46.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.047 7f0760da4700 1 -- 172.21.2.101:0/1915046834 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f075c0ec8e0 con 0x7f075c0ebec0 2021-02-10T11:44:46.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.047 7f0760da4700 1 -- 172.21.2.101:0/1915046834 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f075c0ec8e0 con 0x7f075c0ebec0 2021-02-10T11:44:46.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.048 7f07537fe700 1 -- 172.21.2.101:0/1915046834 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0754000dd0 con 0x7f075c0ebec0 2021-02-10T11:44:46.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.048 7f07537fe700 1 --2- 172.21.2.101:0/1915046834 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07400186d0 0x7f074001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:46.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.048 7f07537fe700 1 -- 172.21.2.101:0/1915046834 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0754032a10 con 0x7f075c0ebec0 2021-02-10T11:44:46.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.048 7f075ab13700 1 --2- 172.21.2.101:0/1915046834 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07400186d0 0x7f074001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:46.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.048 7f075ab13700 1 --2- 172.21.2.101:0/1915046834 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07400186d0 0x7f074001abd0 crc :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:46.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.079 7f0760da4700 1 -- 172.21.2.101:0/1915046834 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f075c0ebc90 con 0x7f075c0ebec0 2021-02-10T11:44:46.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.081 7f07537fe700 1 -- 172.21.2.101:0/1915046834 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f07540066f0 con 0x7f075c0ebec0 2021-02-10T11:44:46.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.210 7f0760da4700 1 -- 172.21.2.101:0/1915046834 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f075c0ebc90 con 0x7f075c0ebec0 2021-02-10T11:44:46.609 INFO:tasks.ceph.mon.a.gibba001.stderr:2021-02-10 11:44:46.611 7f974d04d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-02-10T11:44:46.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.618 7f07537fe700 1 -- 172.21.2.101:0/1915046834 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v125) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0754000dd0 con 0x7f075c0ebec0 2021-02-10T11:44:46.623 INFO:teuthology.orchestra.run.gibba001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-02-10T11:44:46.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.627 7f0760da4700 1 -- 172.21.2.101:0/1915046834 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07400186d0 msgr2=0x7f074001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:46.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.627 7f0760da4700 1 --2- 172.21.2.101:0/1915046834 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07400186d0 0x7f074001abd0 crc :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:46.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.627 7f0760da4700 1 -- 172.21.2.101:0/1915046834 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f075c0ebec0 msgr2=0x7f075c0ec2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:46.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.627 7f0760da4700 1 --2- 172.21.2.101:0/1915046834 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f075c0ebec0 0x7f075c0ec2c0 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 rx=0x7f075400b8c0 tx=0x7f075400bc60).stop 2021-02-10T11:44:46.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.627 7f0760da4700 1 -- 172.21.2.101:0/1915046834 shutdown_connections 2021-02-10T11:44:46.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.627 7f0760da4700 1 --2- 172.21.2.101:0/1915046834 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f075c0ebec0 0x7f075c0ec2c0 unknown :-1 s=CLOSED pgs=2719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:46.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.627 7f0760da4700 1 --2- 172.21.2.101:0/1915046834 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f075c0a6560 0x7f075c0a6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:46.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.627 7f0760da4700 1 --2- 172.21.2.101:0/1915046834 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f07400186d0 0x7f074001abd0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:46.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.627 7f0760da4700 1 --2- 172.21.2.101:0/1915046834 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f075c0a60d0 0x7f075c0a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:46.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.627 7f0760da4700 1 -- 172.21.2.101:0/1915046834 >> 172.21.2.101:0/1915046834 conn(0x7f075c0f0190 msgr2=0x7f075c094d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:46.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.627 7f0760da4700 1 -- 172.21.2.101:0/1915046834 shutdown_connections 2021-02-10T11:44:46.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:46.627 7f0760da4700 1 -- 172.21.2.101:0/1915046834 wait complete. 2021-02-10T11:44:46.633 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:44:46.659 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:44:47.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.049 7f432be5b700 1 Processor -- start 2021-02-10T11:44:47.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.049 7f432be5b700 1 -- start start 2021-02-10T11:44:47.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.049 7f432be5b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f43240d1870 0x7f43241b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:47.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.049 7f432be5b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f43241b0a10 0x7f43240cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:47.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.049 7f432be5b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43240f0b10 con 0x7f43240d1c70 2021-02-10T11:44:47.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.049 7f432be5b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43240e5ed0 con 0x7f43241b0a10 2021-02-10T11:44:47.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.049 7f432be5b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43240f0650 con 0x7f43240d1870 2021-02-10T11:44:47.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.050 7f4329476700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f43241b0a10 0x7f43240cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:47.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.050 7f4329476700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f43241b0a10 0x7f43240cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60012/0 (socket says 172.21.2.101:60012) 2021-02-10T11:44:47.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.050 7f432a478700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f43240d1c70 0x7f43240cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60506/0 (socket says 172.21.2.101:60506) 2021-02-10T11:44:47.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.050 7f4329476700 1 -- 172.21.2.101:0/3475850126 learned_addr learned my addr 172.21.2.101:0/3475850126 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:47.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.050 7f4329476700 1 -- 172.21.2.101:0/3475850126 >> v2:172.21.2.108:3300/0 conn(0x7f43240d1870 msgr2=0x7f43241b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:47.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.050 7f4329476700 1 --2- 172.21.2.101:0/3475850126 >> v2:172.21.2.108:3300/0 conn(0x7f43240d1870 0x7f43241b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:47.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.050 7f4329476700 1 -- 172.21.2.101:0/3475850126 >> v1:172.21.2.101:6789/0 conn(0x7f43240d1c70 legacy=0x7f43240cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:47.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.050 7f4329476700 1 -- 172.21.2.101:0/3475850126 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43240f0840 con 0x7f43241b0a10 2021-02-10T11:44:47.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.050 7f4329476700 1 --2- 172.21.2.101:0/3475850126 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43241b0a10 0x7f43240cf330 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 rx=0x7f4320006700 tx=0x7f4320009b80).ready entity=mon.0 client_cookie=2d973d56921bb091 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:47.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.050 7f4328c75700 1 -- 172.21.2.101:0/3475850126 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f432000c0c0 con 0x7f43241b0a10 2021-02-10T11:44:47.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.050 7f4328c75700 1 -- 172.21.2.101:0/3475850126 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432000c440 con 0x7f43241b0a10 2021-02-10T11:44:47.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.050 7f4328c75700 1 -- 172.21.2.101:0/3475850126 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f432000c960 con 0x7f43241b0a10 2021-02-10T11:44:47.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.050 7f432be5b700 1 -- 172.21.2.101:0/3475850126 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43241b0a10 msgr2=0x7f43240cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:47.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.050 7f432be5b700 1 --2- 172.21.2.101:0/3475850126 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43241b0a10 0x7f43240cf330 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 rx=0x7f4320006700 tx=0x7f4320009b80).stop 2021-02-10T11:44:47.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.051 7f432be5b700 1 -- 172.21.2.101:0/3475850126 shutdown_connections 2021-02-10T11:44:47.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.051 7f432be5b700 1 --2- 172.21.2.101:0/3475850126 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43241b0a10 0x7f43240cf330 unknown :-1 s=CLOSED pgs=2722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:47.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.051 7f432be5b700 1 --2- 172.21.2.101:0/3475850126 >> v2:172.21.2.108:3300/0 conn(0x7f43240d1870 0x7f43241b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:47.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.051 7f432be5b700 1 -- 172.21.2.101:0/3475850126 >> 172.21.2.101:0/3475850126 conn(0x7f43240ce450 msgr2=0x7f43240d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:47.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.051 7f432be5b700 1 -- 172.21.2.101:0/3475850126 shutdown_connections 2021-02-10T11:44:47.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.052 7f432be5b700 1 -- 172.21.2.101:0/3475850126 wait complete. 2021-02-10T11:44:47.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.066 7f432be5b700 1 Processor -- start 2021-02-10T11:44:47.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.066 7f432be5b700 1 -- start start 2021-02-10T11:44:47.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.066 7f432be5b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43241ac980 0x7f43241ace80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:47.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.066 7f432be5b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43240aa400 0x7f43241b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:47.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.067 7f432be5b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f43240aa800 0x7f43241b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:47.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.067 7f432be5b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43241b6d50 con 0x7f43240aa400 2021-02-10T11:44:47.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.067 7f432be5b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43241ad4a0 con 0x7f43240aa800 2021-02-10T11:44:47.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.067 7f432be5b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432418f0e0 con 0x7f43241ac980 2021-02-10T11:44:47.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.067 7f4329476700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43240aa400 0x7f43241b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:47.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.067 7f4329476700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43240aa400 0x7f43241b72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60018/0 (socket says 172.21.2.101:60018) 2021-02-10T11:44:47.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.067 7f4329476700 1 -- 172.21.2.101:0/1780304708 learned_addr learned my addr 172.21.2.101:0/1780304708 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:47.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.067 7f4329476700 1 -- 172.21.2.101:0/1780304708 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43241ac980 msgr2=0x7f43241ace80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:47.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.067 7f4329476700 1 --2- 172.21.2.101:0/1780304708 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43241ac980 0x7f43241ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:47.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.067 7f4329476700 1 -- 172.21.2.101:0/1780304708 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f43240aa800 msgr2=0x7f43241b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:47.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.067 7f4329476700 1 --2- 172.21.2.101:0/1780304708 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f43240aa800 0x7f43241b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:47.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.067 7f4329476700 1 -- 172.21.2.101:0/1780304708 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43241ac4c0 con 0x7f43240aa400 2021-02-10T11:44:47.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.067 7f4329476700 1 --2- 172.21.2.101:0/1780304708 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43240aa400 0x7f43241b72d0 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 rx=0x7f432000b5e0 tx=0x7f432000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:47.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.068 7f43167fc700 1 -- 172.21.2.101:0/1780304708 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f432000bd70 con 0x7f43240aa400 2021-02-10T11:44:47.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.068 7f43167fc700 1 -- 172.21.2.101:0/1780304708 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4320002030 con 0x7f43240aa400 2021-02-10T11:44:47.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.068 7f43167fc700 1 -- 172.21.2.101:0/1780304708 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f43200024d0 con 0x7f43240aa400 2021-02-10T11:44:47.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.068 7f432be5b700 1 -- 172.21.2.101:0/1780304708 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43241ac4c0 con 0x7f43240aa400 2021-02-10T11:44:47.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.068 7f432be5b700 1 -- 172.21.2.101:0/1780304708 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43241ac4c0 con 0x7f43240aa400 2021-02-10T11:44:47.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.068 7f43167fc700 1 -- 172.21.2.101:0/1780304708 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4320000c90 con 0x7f43240aa400 2021-02-10T11:44:47.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.068 7f43167fc700 1 --2- 172.21.2.101:0/1780304708 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f430c018730 0x7f430c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:47.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.068 7f43167fc700 1 -- 172.21.2.101:0/1780304708 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f4320032700 con 0x7f43240aa400 2021-02-10T11:44:47.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.068 7f4329c77700 1 --2- 172.21.2.101:0/1780304708 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f430c018730 0x7f430c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:47.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.069 7f4329c77700 1 --2- 172.21.2.101:0/1780304708 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f430c018730 0x7f430c01abc0 crc :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:47.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.099 7f432be5b700 1 -- 172.21.2.101:0/1780304708 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43240ebc60 con 0x7f43240aa400 2021-02-10T11:44:47.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.102 7f43167fc700 1 -- 172.21.2.101:0/1780304708 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f432001f570 con 0x7f43240aa400 2021-02-10T11:44:47.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.230 7f432be5b700 1 -- 172.21.2.101:0/1780304708 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43240f0b10 con 0x7f43240aa400 2021-02-10T11:44:47.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.230 7f43167fc700 1 -- 172.21.2.101:0/1780304708 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+1763 (secure 0 0 0) 0x7f432000bd70 con 0x7f43240aa400 2021-02-10T11:44:47.229 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 125 2021-02-10T11:44:47.229 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:44:47.229 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":125,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/4208244473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":4208244473},{"type":"v1","addr":"172.21.2.101:6835","nonce":4208244473}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":119}],"filesystems":[{"mdsmap":{"epoch":125,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:44:08.462960","modified":"2021-02-10 11:44:46.611787","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":148,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-02-10T11:44:47.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.232 7f432be5b700 1 -- 172.21.2.101:0/1780304708 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f430c018730 msgr2=0x7f430c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:47.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.232 7f432be5b700 1 --2- 172.21.2.101:0/1780304708 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f430c018730 0x7f430c01abc0 crc :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:47.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.232 7f432be5b700 1 -- 172.21.2.101:0/1780304708 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43240aa400 msgr2=0x7f43241b72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:47.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.232 7f432be5b700 1 --2- 172.21.2.101:0/1780304708 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43240aa400 0x7f43241b72d0 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 rx=0x7f432000b5e0 tx=0x7f432000b980).stop 2021-02-10T11:44:47.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.232 7f432be5b700 1 -- 172.21.2.101:0/1780304708 shutdown_connections 2021-02-10T11:44:47.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.232 7f432be5b700 1 --2- 172.21.2.101:0/1780304708 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f43240aa800 0x7f43241b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:47.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.232 7f432be5b700 1 --2- 172.21.2.101:0/1780304708 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43240aa400 0x7f43241b72d0 unknown :-1 s=CLOSED pgs=2723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:47.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.232 7f432be5b700 1 --2- 172.21.2.101:0/1780304708 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f430c018730 0x7f430c01abc0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:47.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.232 7f432be5b700 1 --2- 172.21.2.101:0/1780304708 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43241ac980 0x7f43241ace80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:47.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.232 7f432be5b700 1 -- 172.21.2.101:0/1780304708 >> 172.21.2.101:0/1780304708 conn(0x7f43241ac040 msgr2=0x7f4324094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:47.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.233 7f432be5b700 1 -- 172.21.2.101:0/1780304708 shutdown_connections 2021-02-10T11:44:47.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.233 7f432be5b700 1 -- 172.21.2.101:0/1780304708 wait complete. 2021-02-10T11:44:47.239 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-02-10T11:44:47.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.657 7f06ce73c700 1 Processor -- start 2021-02-10T11:44:47.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.659 7f06ce73c700 1 -- start start 2021-02-10T11:44:47.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.659 7f06ce73c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f06c80d1870 0x7f06c81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:47.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.659 7f06ce73c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06c81b0a10 0x7f06c80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:47.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.659 7f06ce73c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c80f0b10 con 0x7f06c80d1c70 2021-02-10T11:44:47.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.659 7f06ce73c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c80e5ed0 con 0x7f06c80d1870 2021-02-10T11:44:47.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.659 7f06ce73c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c80f0650 con 0x7f06c81b0a10 2021-02-10T11:44:47.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.659 7f06c7fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f06c80d1870 0x7f06c81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:47.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.659 7f06c77fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06c81b0a10 0x7f06c80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:47.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.659 7f06ccd59700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f06c80d1c70 0x7f06c80cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60520/0 (socket says 172.21.2.101:60520) 2021-02-10T11:44:47.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.659 7f06ccd59700 1 -- 172.21.2.101:0/563206820 learned_addr learned my addr 172.21.2.101:0/563206820 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:47.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.660 7f06c7fff700 1 -- 172.21.2.101:0/563206820 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06c81b0a10 msgr2=0x7f06c80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:47.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.660 7f06c7fff700 1 --2- 172.21.2.101:0/563206820 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06c81b0a10 0x7f06c80cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:47.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.660 7f06c7fff700 1 -- 172.21.2.101:0/563206820 >> v1:172.21.2.101:6789/0 conn(0x7f06c80d1c70 legacy=0x7f06c80cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:47.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.660 7f06c7fff700 1 -- 172.21.2.101:0/563206820 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c80f0870 con 0x7f06c80d1870 2021-02-10T11:44:47.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.660 7f06c7fff700 1 --2- 172.21.2.101:0/563206820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06c80d1870 0x7f06c81b03f0 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 rx=0x7f06b8006650 tx=0x7f06b8009a00).ready entity=mon.0 client_cookie=44539a980847c49f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:47.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.660 7f06c6ffd700 1 -- 172.21.2.101:0/563206820 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f06b8009df0 con 0x7f06c80d1870 2021-02-10T11:44:47.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.660 7f06c6ffd700 1 -- 172.21.2.101:0/563206820 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b80009b0 con 0x7f06c80d1870 2021-02-10T11:44:47.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.660 7f06c6ffd700 1 -- 172.21.2.101:0/563206820 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f06b800c2b0 con 0x7f06c80d1870 2021-02-10T11:44:47.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.660 7f06ce73c700 1 -- 172.21.2.101:0/563206820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06c80d1870 msgr2=0x7f06c81b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:47.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.660 7f06ce73c700 1 --2- 172.21.2.101:0/563206820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06c80d1870 0x7f06c81b03f0 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 rx=0x7f06b8006650 tx=0x7f06b8009a00).stop 2021-02-10T11:44:47.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.661 7f06ce73c700 1 -- 172.21.2.101:0/563206820 shutdown_connections 2021-02-10T11:44:47.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.661 7f06ce73c700 1 --2- 172.21.2.101:0/563206820 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06c81b0a10 0x7f06c80cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:47.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.661 7f06ce73c700 1 --2- 172.21.2.101:0/563206820 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06c80d1870 0x7f06c81b03f0 unknown :-1 s=CLOSED pgs=2725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:47.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.661 7f06ce73c700 1 -- 172.21.2.101:0/563206820 >> 172.21.2.101:0/563206820 conn(0x7f06c80ce450 msgr2=0x7f06c80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:47.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.661 7f06ce73c700 1 -- 172.21.2.101:0/563206820 shutdown_connections 2021-02-10T11:44:47.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.661 7f06ce73c700 1 -- 172.21.2.101:0/563206820 wait complete. 2021-02-10T11:44:47.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.674 7f06ce73c700 1 Processor -- start 2021-02-10T11:44:47.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.675 7f06ce73c700 1 -- start start 2021-02-10T11:44:47.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.675 7f06ce73c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06c81a06c0 0x7f06c80ebe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:47.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.675 7f06ce73c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f06c80ec490 0x7f06c80c30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:47.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.675 7f06ce73c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06c80c3710 0x7f06c80bc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:47.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.675 7f06ce73c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c80c3b10 con 0x7f06c80c3710 2021-02-10T11:44:47.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.675 7f06ce73c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c80eca90 con 0x7f06c80ec490 2021-02-10T11:44:47.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.675 7f06ce73c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c80bc690 con 0x7f06c81a06c0 2021-02-10T11:44:47.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.675 7f06ccd59700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06c80c3710 0x7f06c80bc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:47.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.675 7f06ccd59700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06c80c3710 0x7f06c80bc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60032/0 (socket says 172.21.2.101:60032) 2021-02-10T11:44:47.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.675 7f06ccd59700 1 -- 172.21.2.101:0/2528295728 learned_addr learned my addr 172.21.2.101:0/2528295728 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:47.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.675 7f06c7fff700 1 --2- 172.21.2.101:0/2528295728 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06c81a06c0 0x7f06c80ebe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:47.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.675 7f06c77fe700 1 --2- 172.21.2.101:0/2528295728 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f06c80ec490 0x7f06c80c30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:47.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.676 7f06c7fff700 1 -- 172.21.2.101:0/2528295728 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f06c80ec490 msgr2=0x7f06c80c30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:47.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.676 7f06c7fff700 1 --2- 172.21.2.101:0/2528295728 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f06c80ec490 0x7f06c80c30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:47.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.676 7f06c7fff700 1 -- 172.21.2.101:0/2528295728 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06c80c3710 msgr2=0x7f06c80bc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:47.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.676 7f06c7fff700 1 --2- 172.21.2.101:0/2528295728 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06c80c3710 0x7f06c80bc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:47.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.676 7f06c7fff700 1 -- 172.21.2.101:0/2528295728 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c80bc870 con 0x7f06c81a06c0 2021-02-10T11:44:47.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.676 7f06ccd59700 1 --2- 172.21.2.101:0/2528295728 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06c80c3710 0x7f06c80bc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:44:47.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.676 7f06c7fff700 1 --2- 172.21.2.101:0/2528295728 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06c81a06c0 0x7f06c80ebe70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f06b800b510 tx=0x7f06b800b8b0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:47.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.676 7f06c4ff9700 1 -- 172.21.2.101:0/2528295728 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f06b8000940 con 0x7f06c81a06c0 2021-02-10T11:44:47.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.676 7f06c4ff9700 1 -- 172.21.2.101:0/2528295728 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b8000c90 con 0x7f06c81a06c0 2021-02-10T11:44:47.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.676 7f06c4ff9700 1 -- 172.21.2.101:0/2528295728 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f06b8002030 con 0x7f06c81a06c0 2021-02-10T11:44:47.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.676 7f06ce73c700 1 -- 172.21.2.101:0/2528295728 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c80bc870 con 0x7f06c81a06c0 2021-02-10T11:44:47.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.676 7f06ce73c700 1 -- 172.21.2.101:0/2528295728 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c80bce00 con 0x7f06c81a06c0 2021-02-10T11:44:47.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.677 7f06c4ff9700 1 -- 172.21.2.101:0/2528295728 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f06b80052e0 con 0x7f06c81a06c0 2021-02-10T11:44:47.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.677 7f06c4ff9700 1 --2- 172.21.2.101:0/2528295728 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06a80186c0 0x7f06a801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:47.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.677 7f06c4ff9700 1 -- 172.21.2.101:0/2528295728 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f06b80324d0 con 0x7f06c81a06c0 2021-02-10T11:44:47.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.677 7f06c77fe700 1 --2- 172.21.2.101:0/2528295728 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06a80186c0 0x7f06a801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:47.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.677 7f06c77fe700 1 --2- 172.21.2.101:0/2528295728 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06a80186c0 0x7f06a801abc0 crc :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:47.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.708 7f06ce73c700 1 -- 172.21.2.101:0/2528295728 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c80ebc40 con 0x7f06c81a06c0 2021-02-10T11:44:47.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.712 7f06c4ff9700 1 -- 172.21.2.101:0/2528295728 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f06b80018d0 con 0x7f06c81a06c0 2021-02-10T11:44:47.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:47.841 7f06ce73c700 1 -- 172.21.2.101:0/2528295728 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f06c80f0b10 con 0x7f06c81a06c0 2021-02-10T11:44:48.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:48.619 7f06c4ff9700 1 -- 172.21.2.101:0/2528295728 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v126) v1 ==== 104+0+0 (secure 0 0 0) 0x7f06b80018d0 con 0x7f06c81a06c0 2021-02-10T11:44:48.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:48.662 7f06ce73c700 1 -- 172.21.2.101:0/2528295728 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06a80186c0 msgr2=0x7f06a801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:48.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:48.662 7f06ce73c700 1 --2- 172.21.2.101:0/2528295728 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06a80186c0 0x7f06a801abc0 crc :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:48.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:48.662 7f06ce73c700 1 -- 172.21.2.101:0/2528295728 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06c81a06c0 msgr2=0x7f06c80ebe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:48.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:48.662 7f06ce73c700 1 --2- 172.21.2.101:0/2528295728 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06c81a06c0 0x7f06c80ebe70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f06b800b510 tx=0x7f06b800b8b0).stop 2021-02-10T11:44:48.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:48.662 7f06ce73c700 1 -- 172.21.2.101:0/2528295728 shutdown_connections 2021-02-10T11:44:48.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:48.662 7f06ce73c700 1 --2- 172.21.2.101:0/2528295728 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f06c80c3710 0x7f06c80bc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:48.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:48.662 7f06ce73c700 1 --2- 172.21.2.101:0/2528295728 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f06c80ec490 0x7f06c80c30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:48.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:48.662 7f06ce73c700 1 --2- 172.21.2.101:0/2528295728 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f06a80186c0 0x7f06a801abc0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:48.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:48.662 7f06ce73c700 1 --2- 172.21.2.101:0/2528295728 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f06c81a06c0 0x7f06c80ebe70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:48.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:48.662 7f06ce73c700 1 -- 172.21.2.101:0/2528295728 >> 172.21.2.101:0/2528295728 conn(0x7f06c80a6bb0 msgr2=0x7f06c8095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:48.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:48.662 7f06ce73c700 1 -- 172.21.2.101:0/2528295728 shutdown_connections 2021-02-10T11:44:48.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:48.662 7f06ce73c700 1 -- 172.21.2.101:0/2528295728 wait complete. 2021-02-10T11:44:48.669 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-02-10T11:44:49.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.086 7fe218960700 1 Processor -- start 2021-02-10T11:44:49.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.086 7fe218960700 1 -- start start 2021-02-10T11:44:49.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.087 7fe218960700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe2140d1c80 0x7fe2140cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:49.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.087 7fe218960700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe2140cf590 0x7fe2140cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:49.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.087 7fe218960700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2140fe1d0 con 0x7fe2140d1880 2021-02-10T11:44:49.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.087 7fe218960700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2141b0cc0 con 0x7fe2140cf590 2021-02-10T11:44:49.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.087 7fe218960700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe214109a70 con 0x7fe2140d1c80 2021-02-10T11:44:49.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.087 7fe212e1d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe2140cf590 0x7fe2140cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:49.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.087 7fe212e1d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe2140cf590 0x7fe2140cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60040/0 (socket says 172.21.2.101:60040) 2021-02-10T11:44:49.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.087 7fe212e1d700 1 -- 172.21.2.101:0/4211131292 learned_addr learned my addr 172.21.2.101:0/4211131292 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:49.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.087 7fe212e1d700 1 -- 172.21.2.101:0/4211131292 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe2140d1c80 msgr2=0x7fe2140cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:49.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.087 7fe212e1d700 1 --2- 172.21.2.101:0/4211131292 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe2140d1c80 0x7fe2140cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:49.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.087 7fe212e1d700 1 -- 172.21.2.101:0/4211131292 >> v1:172.21.2.108:6789/0 conn(0x7fe2140d1880 legacy=0x7fe2141b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:49.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.087 7fe212e1d700 1 -- 172.21.2.101:0/4211131292 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe214088070 con 0x7fe2140cf590 2021-02-10T11:44:49.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.088 7fe212e1d700 1 --2- 172.21.2.101:0/4211131292 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe2140cf590 0x7fe2140cf990 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 rx=0x7fe20c006f00 tx=0x7fe20c009f10).ready entity=mon.0 client_cookie=3ddb8004b3f65332 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:49.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.088 7fe21161a700 1 -- 172.21.2.101:0/4211131292 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe20c00b3b0 con 0x7fe2140cf590 2021-02-10T11:44:49.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.088 7fe21161a700 1 -- 172.21.2.101:0/4211131292 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20c00b730 con 0x7fe2140cf590 2021-02-10T11:44:49.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.088 7fe21161a700 1 -- 172.21.2.101:0/4211131292 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe20c00bc50 con 0x7fe2140cf590 2021-02-10T11:44:49.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.088 7fe218960700 1 -- 172.21.2.101:0/4211131292 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe2140cf590 msgr2=0x7fe2140cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:49.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.088 7fe218960700 1 --2- 172.21.2.101:0/4211131292 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe2140cf590 0x7fe2140cf990 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 rx=0x7fe20c006f00 tx=0x7fe20c009f10).stop 2021-02-10T11:44:49.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.089 7fe218960700 1 -- 172.21.2.101:0/4211131292 shutdown_connections 2021-02-10T11:44:49.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.089 7fe218960700 1 --2- 172.21.2.101:0/4211131292 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe2140d1c80 0x7fe2140cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:49.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.089 7fe218960700 1 --2- 172.21.2.101:0/4211131292 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe2140cf590 0x7fe2140cf990 unknown :-1 s=CLOSED pgs=2726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:49.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.089 7fe218960700 1 -- 172.21.2.101:0/4211131292 >> 172.21.2.101:0/4211131292 conn(0x7fe2140ce430 msgr2=0x7fe2140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:49.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.089 7fe218960700 1 -- 172.21.2.101:0/4211131292 shutdown_connections 2021-02-10T11:44:49.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.089 7fe218960700 1 -- 172.21.2.101:0/4211131292 wait complete. 2021-02-10T11:44:49.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.103 7fe218960700 1 Processor -- start 2021-02-10T11:44:49.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.104 7fe218960700 1 -- start start 2021-02-10T11:44:49.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.104 7fe218960700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe21419e5b0 0x7fe2141afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:49.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.104 7fe218960700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe21419e9b0 0x7fe2141b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:49.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.104 7fe218960700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe2141b0250 0x7fe2141b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:49.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.104 7fe218960700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe214195bd0 con 0x7fe21419e9b0 2021-02-10T11:44:49.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.104 7fe218960700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2141b0840 con 0x7fe21419e5b0 2021-02-10T11:44:49.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.104 7fe218960700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21419eed0 con 0x7fe2141b0250 2021-02-10T11:44:49.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.104 7fe211e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe21419e9b0 0x7fe2141b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:49.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.104 7fe211e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe21419e9b0 0x7fe2141b0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60042/0 (socket says 172.21.2.101:60042) 2021-02-10T11:44:49.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.104 7fe211e1b700 1 -- 172.21.2.101:0/876193294 learned_addr learned my addr 172.21.2.101:0/876193294 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:49.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.104 7fe211e1b700 1 -- 172.21.2.101:0/876193294 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe2141b0250 msgr2=0x7fe2141b13e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:49.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.104 7fe211e1b700 1 --2- 172.21.2.101:0/876193294 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe2141b0250 0x7fe2141b13e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:49.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.104 7fe211e1b700 1 -- 172.21.2.101:0/876193294 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe21419e5b0 msgr2=0x7fe2141afc30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:49.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.104 7fe211e1b700 1 --2- 172.21.2.101:0/876193294 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe21419e5b0 0x7fe2141afc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:49.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.104 7fe211e1b700 1 -- 172.21.2.101:0/876193294 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21419f0b0 con 0x7fe21419e9b0 2021-02-10T11:44:49.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.105 7fe211e1b700 1 --2- 172.21.2.101:0/876193294 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe21419e9b0 0x7fe2141b0dc0 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 rx=0x7fe208006f00 tx=0x7fe20800b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:49.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.105 7fe202ffd700 1 -- 172.21.2.101:0/876193294 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe208006350 con 0x7fe21419e9b0 2021-02-10T11:44:49.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.105 7fe202ffd700 1 -- 172.21.2.101:0/876193294 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20800bd30 con 0x7fe21419e9b0 2021-02-10T11:44:49.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.105 7fe202ffd700 1 -- 172.21.2.101:0/876193294 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe2080020d0 con 0x7fe21419e9b0 2021-02-10T11:44:49.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.105 7fe218960700 1 -- 172.21.2.101:0/876193294 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21419f0b0 con 0x7fe21419e9b0 2021-02-10T11:44:49.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.105 7fe218960700 1 -- 172.21.2.101:0/876193294 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2141b1cf0 con 0x7fe21419e9b0 2021-02-10T11:44:49.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.106 7fe202ffd700 1 -- 172.21.2.101:0/876193294 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe208000c90 con 0x7fe21419e9b0 2021-02-10T11:44:49.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.106 7fe202ffd700 1 --2- 172.21.2.101:0/876193294 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe1f80186c0 0x7fe1f801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:49.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.106 7fe202ffd700 1 -- 172.21.2.101:0/876193294 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fe2080325e0 con 0x7fe21419e9b0 2021-02-10T11:44:49.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.106 7fe21261c700 1 --2- 172.21.2.101:0/876193294 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe1f80186c0 0x7fe1f801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:49.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.106 7fe21261c700 1 --2- 172.21.2.101:0/876193294 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe1f80186c0 0x7fe1f801abc0 crc :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:49.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.137 7fe218960700 1 -- 172.21.2.101:0/876193294 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2141b1cf0 con 0x7fe21419e9b0 2021-02-10T11:44:49.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.139 7fe202ffd700 1 -- 172.21.2.101:0/876193294 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe208000940 con 0x7fe21419e9b0 2021-02-10T11:44:49.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.266 7fe218960700 1 -- 172.21.2.101:0/876193294 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe2141b1cf0 con 0x7fe21419e9b0 2021-02-10T11:44:49.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.625 7fe202ffd700 1 -- 172.21.2.101:0/876193294 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v149) v1 ==== 185+0+0 (secure 0 0 0) 0x7fe20801f2d0 con 0x7fe21419e9b0 2021-02-10T11:44:49.631 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' removed 2021-02-10T11:44:49.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.634 7fe218960700 1 -- 172.21.2.101:0/876193294 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe1f80186c0 msgr2=0x7fe1f801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:49.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.634 7fe218960700 1 --2- 172.21.2.101:0/876193294 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe1f80186c0 0x7fe1f801abc0 crc :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:49.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.634 7fe218960700 1 -- 172.21.2.101:0/876193294 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe21419e9b0 msgr2=0x7fe2141b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:49.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.634 7fe218960700 1 --2- 172.21.2.101:0/876193294 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe21419e9b0 0x7fe2141b0dc0 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 rx=0x7fe208006f00 tx=0x7fe20800b940).stop 2021-02-10T11:44:49.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.634 7fe218960700 1 -- 172.21.2.101:0/876193294 shutdown_connections 2021-02-10T11:44:49.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.634 7fe218960700 1 --2- 172.21.2.101:0/876193294 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe1f80186c0 0x7fe1f801abc0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:49.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.634 7fe218960700 1 --2- 172.21.2.101:0/876193294 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe2141b0250 0x7fe2141b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:49.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.634 7fe218960700 1 --2- 172.21.2.101:0/876193294 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe21419e9b0 0x7fe2141b0dc0 unknown :-1 s=CLOSED pgs=2727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:49.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.634 7fe218960700 1 --2- 172.21.2.101:0/876193294 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe21419e5b0 0x7fe2141afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:49.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.634 7fe218960700 1 -- 172.21.2.101:0/876193294 >> 172.21.2.101:0/876193294 conn(0x7fe21419f940 msgr2=0x7fe21418c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:49.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.634 7fe218960700 1 -- 172.21.2.101:0/876193294 shutdown_connections 2021-02-10T11:44:49.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:49.634 7fe218960700 1 -- 172.21.2.101:0/876193294 wait complete. 2021-02-10T11:44:49.641 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-02-10T11:44:50.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.059 7ff44154d700 1 Processor -- start 2021-02-10T11:44:50.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.059 7ff44154d700 1 -- start start 2021-02-10T11:44:50.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.060 7ff44154d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff43c0d1870 0x7ff43c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:50.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.060 7ff44154d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff43c109a60 con 0x7ff43c0cff70 2021-02-10T11:44:50.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.060 7ff44154d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff43c088070 con 0x7ff43c1b0ae0 2021-02-10T11:44:50.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.060 7ff44154d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0e5ef0 con 0x7ff43c0d1870 2021-02-10T11:44:50.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.060 7ff43bac0700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff43c0cff70 0x7ff43c109130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60544/0 (socket says 172.21.2.101:60544) 2021-02-10T11:44:50.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.060 7ff43bac0700 1 -- 172.21.2.101:0/2848143857 learned_addr learned my addr 172.21.2.101:0/2848143857 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:50.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.060 7ff43a2bd700 1 -- 172.21.2.101:0/2848143857 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507073271 0 0) 0x7ff434001500 con 0x7ff43c0cff70 2021-02-10T11:44:50.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.060 7ff43a2bd700 1 -- 172.21.2.101:0/2848143857 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff428002fe0 con 0x7ff43c0cff70 2021-02-10T11:44:50.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43b2bf700 1 --2- 172.21.2.101:0/2848143857 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff43c0d1870 0x7ff43c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:50.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43a2bd700 1 -- 172.21.2.101:0/2848143857 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1661627575 0 0) 0x7ff434001460 con 0x7ff43c0cff70 2021-02-10T11:44:50.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43a2bd700 1 -- 172.21.2.101:0/2848143857 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff43c0d1870 msgr2=0x7ff43c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:50.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43a2bd700 1 --2- 172.21.2.101:0/2848143857 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff43c0d1870 0x7ff43c1b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:50.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43a2bd700 1 -- 172.21.2.101:0/2848143857 >> v1:172.21.2.108:6789/0 conn(0x7ff43c1b0ae0 legacy=0x7ff43c108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:50.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43a2bd700 1 -- 172.21.2.101:0/2848143857 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c0d1c70 con 0x7ff43c0cff70 2021-02-10T11:44:50.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43a2bd700 1 -- 172.21.2.101:0/2848143857 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7ff434001980 con 0x7ff43c0cff70 2021-02-10T11:44:50.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43a2bd700 1 -- 172.21.2.101:0/2848143857 >> v1:172.21.2.101:6789/0 conn(0x7ff43c0cff70 legacy=0x7ff43c109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:50.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43a2bd700 1 --2- 172.21.2.101:0/2848143857 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff428003b10 0x7ff428005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:50.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43a2bd700 1 --2- 172.21.2.101:0/2848143857 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4280067a0 0x7ff428008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:50.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43a2bd700 1 --2- 172.21.2.101:0/2848143857 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff428009530 0x7ff42800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:50.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43a2bd700 1 -- 172.21.2.101:0/2848143857 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42800c2e0 con 0x7ff428009530 2021-02-10T11:44:50.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43a2bd700 1 -- 172.21.2.101:0/2848143857 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42800c500 con 0x7ff428003b10 2021-02-10T11:44:50.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43a2bd700 1 -- 172.21.2.101:0/2848143857 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42800c720 con 0x7ff4280067a0 2021-02-10T11:44:50.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43bac0700 1 --2- 172.21.2.101:0/2848143857 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff428009530 0x7ff42800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:50.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43b2bf700 1 --2- 172.21.2.101:0/2848143857 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff428003b10 0x7ff428005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:50.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43aabe700 1 --2- 172.21.2.101:0/2848143857 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4280067a0 0x7ff428008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:50.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43bac0700 1 -- 172.21.2.101:0/2848143857 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4280067a0 msgr2=0x7ff428008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:50.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43bac0700 1 --2- 172.21.2.101:0/2848143857 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff4280067a0 0x7ff428008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:50.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43bac0700 1 -- 172.21.2.101:0/2848143857 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff428003b10 msgr2=0x7ff428005f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:50.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43bac0700 1 --2- 172.21.2.101:0/2848143857 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff428003b10 0x7ff428005f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:50.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43bac0700 1 -- 172.21.2.101:0/2848143857 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42800c9c0 con 0x7ff428009530 2021-02-10T11:44:50.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43bac0700 1 --2- 172.21.2.101:0/2848143857 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff428009530 0x7ff42800ba30 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 rx=0x7ff434001b60 tx=0x7ff43404e320).ready entity=mon.0 client_cookie=ef645a670d489e77 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:50.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43a2bd700 1 -- 172.21.2.101:0/2848143857 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff434007a30 con 0x7ff428009530 2021-02-10T11:44:50.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43a2bd700 1 -- 172.21.2.101:0/2848143857 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4340070d0 con 0x7ff428009530 2021-02-10T11:44:50.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.061 7ff43a2bd700 1 -- 172.21.2.101:0/2848143857 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff4340075b0 con 0x7ff428009530 2021-02-10T11:44:50.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.062 7ff44154d700 1 -- 172.21.2.101:0/2848143857 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff428009530 msgr2=0x7ff42800ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:50.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.062 7ff44154d700 1 --2- 172.21.2.101:0/2848143857 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff428009530 0x7ff42800ba30 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 rx=0x7ff434001b60 tx=0x7ff43404e320).stop 2021-02-10T11:44:50.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.062 7ff43bac0700 1 -- 172.21.2.101:0/2848143857 reap_dead start 2021-02-10T11:44:50.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.062 7ff44154d700 1 -- 172.21.2.101:0/2848143857 shutdown_connections 2021-02-10T11:44:50.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.062 7ff44154d700 1 -- 172.21.2.101:0/2848143857 >> 172.21.2.101:0/2848143857 conn(0x7ff43c0ce420 msgr2=0x7ff43c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:50.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.062 7ff44154d700 1 -- 172.21.2.101:0/2848143857 shutdown_connections 2021-02-10T11:44:50.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.062 7ff44154d700 1 -- 172.21.2.101:0/2848143857 wait complete. 2021-02-10T11:44:50.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.077 7ff44154d700 1 Processor -- start 2021-02-10T11:44:50.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.077 7ff44154d700 1 -- start start 2021-02-10T11:44:50.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.077 7ff44154d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff43c1a05f0 0x7ff43c0a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:50.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.077 7ff44154d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff43c0aba50 0x7ff43c0b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:50.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.077 7ff44154d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff43c0b36b0 0x7ff43c0b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:50.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.077 7ff44154d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0b3d20 con 0x7ff43c0aba50 2021-02-10T11:44:50.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.077 7ff44154d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0abe50 con 0x7ff43c1a05f0 2021-02-10T11:44:50.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.077 7ff44154d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0b23c0 con 0x7ff43c0b36b0 2021-02-10T11:44:50.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.077 7ff43aabe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff43c0aba50 0x7ff43c0b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:50.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.077 7ff43aabe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff43c0aba50 0x7ff43c0b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60062/0 (socket says 172.21.2.101:60062) 2021-02-10T11:44:50.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.077 7ff43aabe700 1 -- 172.21.2.101:0/461878643 learned_addr learned my addr 172.21.2.101:0/461878643 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:50.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.078 7ff43aabe700 1 -- 172.21.2.101:0/461878643 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff43c0b36b0 msgr2=0x7ff43c0b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:50.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.078 7ff43aabe700 1 --2- 172.21.2.101:0/461878643 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff43c0b36b0 0x7ff43c0b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:50.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.078 7ff43aabe700 1 -- 172.21.2.101:0/461878643 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff43c1a05f0 msgr2=0x7ff43c0a8f60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:50.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.078 7ff43aabe700 1 --2- 172.21.2.101:0/461878643 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff43c1a05f0 0x7ff43c0a8f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:50.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.078 7ff43aabe700 1 -- 172.21.2.101:0/461878643 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c0b48c0 con 0x7ff43c0aba50 2021-02-10T11:44:50.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.078 7ff43aabe700 1 --2- 172.21.2.101:0/461878643 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff43c0aba50 0x7ff43c0b3090 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 rx=0x7ff42c006f10 tx=0x7ff42c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:50.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.078 7ff433fff700 1 -- 172.21.2.101:0/461878643 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff42c00bd80 con 0x7ff43c0aba50 2021-02-10T11:44:50.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.078 7ff433fff700 1 -- 172.21.2.101:0/461878643 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42c002030 con 0x7ff43c0aba50 2021-02-10T11:44:50.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.078 7ff433fff700 1 -- 172.21.2.101:0/461878643 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff42c002550 con 0x7ff43c0aba50 2021-02-10T11:44:50.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.078 7ff44154d700 1 -- 172.21.2.101:0/461878643 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff43c0b48c0 con 0x7ff43c0aba50 2021-02-10T11:44:50.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.078 7ff44154d700 1 -- 172.21.2.101:0/461878643 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff43c0b4e50 con 0x7ff43c0aba50 2021-02-10T11:44:50.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.079 7ff433fff700 1 -- 172.21.2.101:0/461878643 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff42c000c50 con 0x7ff43c0aba50 2021-02-10T11:44:50.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.079 7ff433fff700 1 --2- 172.21.2.101:0/461878643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4200186c0 0x7ff42001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:50.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.079 7ff433fff700 1 -- 172.21.2.101:0/461878643 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5221+0+0 (secure 0 0 0) 0x7ff42c0323d0 con 0x7ff43c0aba50 2021-02-10T11:44:50.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.079 7ff43b2bf700 1 --2- 172.21.2.101:0/461878643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4200186c0 0x7ff42001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:50.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.079 7ff43b2bf700 1 --2- 172.21.2.101:0/461878643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4200186c0 0x7ff42001abc0 crc :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:50.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.110 7ff44154d700 1 -- 172.21.2.101:0/461878643 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff43c0825d0 con 0x7ff43c0aba50 2021-02-10T11:44:50.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.112 7ff433fff700 1 -- 172.21.2.101:0/461878643 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff42c030ce0 con 0x7ff43c0aba50 2021-02-10T11:44:50.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.242 7ff44154d700 1 -- 172.21.2.101:0/461878643 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff43c0825d0 con 0x7ff43c0aba50 2021-02-10T11:44:50.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.635 7ff433fff700 1 -- 172.21.2.101:0/461878643 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v150) v1 ==== 173+0+0 (secure 0 0 0) 0x7ff42c000c50 con 0x7ff43c0aba50 2021-02-10T11:44:50.657 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' removed 2021-02-10T11:44:50.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.661 7ff44154d700 1 -- 172.21.2.101:0/461878643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4200186c0 msgr2=0x7ff42001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:50.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.661 7ff44154d700 1 --2- 172.21.2.101:0/461878643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4200186c0 0x7ff42001abc0 crc :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:50.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.661 7ff44154d700 1 -- 172.21.2.101:0/461878643 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff43c0aba50 msgr2=0x7ff43c0b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:50.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.661 7ff44154d700 1 --2- 172.21.2.101:0/461878643 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff43c0aba50 0x7ff43c0b3090 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 rx=0x7ff42c006f10 tx=0x7ff42c00b990).stop 2021-02-10T11:44:50.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.661 7ff44154d700 1 -- 172.21.2.101:0/461878643 shutdown_connections 2021-02-10T11:44:50.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.661 7ff44154d700 1 --2- 172.21.2.101:0/461878643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4200186c0 0x7ff42001abc0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:50.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.661 7ff44154d700 1 --2- 172.21.2.101:0/461878643 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff43c0b36b0 0x7ff43c0b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:50.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.661 7ff44154d700 1 --2- 172.21.2.101:0/461878643 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff43c0aba50 0x7ff43c0b3090 unknown :-1 s=CLOSED pgs=2730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:50.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.661 7ff44154d700 1 --2- 172.21.2.101:0/461878643 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff43c1a05f0 0x7ff43c0a8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:50.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.661 7ff44154d700 1 -- 172.21.2.101:0/461878643 >> 172.21.2.101:0/461878643 conn(0x7ff43c0bc070 msgr2=0x7ff43c0ba770 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:50.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.661 7ff44154d700 1 -- 172.21.2.101:0/461878643 shutdown_connections 2021-02-10T11:44:50.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:50.661 7ff44154d700 1 -- 172.21.2.101:0/461878643 wait complete. 2021-02-10T11:44:50.669 INFO:tasks.ceph.mds.a:Restarting daemon 2021-02-10T11:44:50.669 INFO:tasks.ceph.mds.a:Stopping old one... 2021-02-10T11:44:50.669 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-02-10T11:44:50.669 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:44:50.670 INFO:tasks.ceph.mds.b:Restarting daemon 2021-02-10T11:44:50.670 INFO:tasks.ceph.mds.b:Stopping old one... 2021-02-10T11:44:50.670 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-02-10T11:44:50.670 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:44:50.676 INFO:tasks.ceph.mds.a:Stopped 2021-02-10T11:44:50.676 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-02-10T11:44:50.677 INFO:tasks.ceph.mds.b:Stopped 2021-02-10T11:44:50.677 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-02-10T11:44:50.718 INFO:tasks.ceph.mds.a:Started 2021-02-10T11:44:50.718 DEBUG:teuthology.parallel:result is None 2021-02-10T11:44:50.719 INFO:tasks.ceph.mds.b:Started 2021-02-10T11:44:50.719 DEBUG:teuthology.parallel:result is None 2021-02-10T11:44:50.719 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd blacklist clear 2021-02-10T11:44:51.123 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:44:51.126 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:44:51.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.131 7fd05c6b1700 1 Processor -- start 2021-02-10T11:44:51.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.131 7fd05c6b1700 1 -- start start 2021-02-10T11:44:51.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.131 7fd05c6b1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0540d1870 0x7fd0541b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:51.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.131 7fd05c6b1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd0541b0a10 0x7fd0540cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:51.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.131 7fd05c6b1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd0540d1c70 0x7fd0540cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:51.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.131 7fd05c6b1700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0540fe230 con 0x7fd0541b0a10 2021-02-10T11:44:51.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.131 7fd05c6b1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0540e5ed0 con 0x7fd0540d1c70 2021-02-10T11:44:51.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.131 7fd05c6b1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540f0b10 con 0x7fd0540d1870 2021-02-10T11:44:51.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.131 7fd059ccc700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd0541b0a10 0x7fd0540cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:51.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.131 7fd059ccc700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd0541b0a10 0x7fd0540cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60082/0 (socket says 172.21.2.101:60082) 2021-02-10T11:44:51.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.132 7fd059ccc700 1 -- 172.21.2.101:0/2388649686 learned_addr learned my addr 172.21.2.101:0/2388649686 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:51.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.132 7fd059ccc700 1 -- 172.21.2.101:0/2388649686 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0540d1870 msgr2=0x7fd0541b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.132 7fd059ccc700 1 --2- 172.21.2.101:0/2388649686 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0540d1870 0x7fd0541b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.132 7fd059ccc700 1 -- 172.21.2.101:0/2388649686 >> v2:172.21.2.108:3300/0 conn(0x7fd0540d1c70 msgr2=0x7fd0540cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.132 7fd059ccc700 1 --2- 172.21.2.101:0/2388649686 >> v2:172.21.2.108:3300/0 conn(0x7fd0540d1c70 0x7fd0540cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.132 7fd059ccc700 1 -- 172.21.2.101:0/2388649686 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0540cff70 con 0x7fd0541b0a10 2021-02-10T11:44:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.132 7fd059ccc700 1 --2- 172.21.2.101:0/2388649686 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0541b0a10 0x7fd0540cf330 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 rx=0x7fd04c006a80 tx=0x7fd04c009f00).ready entity=mon.0 client_cookie=6e7cdef25630432d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.132 7fd0594cb700 1 -- 172.21.2.101:0/2388649686 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd04c00b5c0 con 0x7fd0541b0a10 2021-02-10T11:44:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.132 7fd0594cb700 1 -- 172.21.2.101:0/2388649686 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04c00b8c0 con 0x7fd0541b0a10 2021-02-10T11:44:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.132 7fd0594cb700 1 -- 172.21.2.101:0/2388649686 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd04c00bd20 con 0x7fd0541b0a10 2021-02-10T11:44:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.132 7fd05c6b1700 1 -- 172.21.2.101:0/2388649686 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0541b0a10 msgr2=0x7fd0540cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.132 7fd05c6b1700 1 --2- 172.21.2.101:0/2388649686 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0541b0a10 0x7fd0540cf330 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 rx=0x7fd04c006a80 tx=0x7fd04c009f00).stop 2021-02-10T11:44:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.133 7fd05c6b1700 1 -- 172.21.2.101:0/2388649686 shutdown_connections 2021-02-10T11:44:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.133 7fd05c6b1700 1 --2- 172.21.2.101:0/2388649686 >> v2:172.21.2.108:3300/0 conn(0x7fd0540d1c70 0x7fd0540cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.133 7fd05c6b1700 1 --2- 172.21.2.101:0/2388649686 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0541b0a10 0x7fd0540cf330 unknown :-1 s=CLOSED pgs=2735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:51.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.133 7fd05c6b1700 1 --2- 172.21.2.101:0/2388649686 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0540d1870 0x7fd0541b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:51.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.133 7fd05c6b1700 1 -- 172.21.2.101:0/2388649686 >> 172.21.2.101:0/2388649686 conn(0x7fd0540ce450 msgr2=0x7fd0540d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:51.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.133 7fd05c6b1700 1 -- 172.21.2.101:0/2388649686 shutdown_connections 2021-02-10T11:44:51.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.133 7fd05c6b1700 1 -- 172.21.2.101:0/2388649686 wait complete. 2021-02-10T11:44:51.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.148 7fd05c6b1700 1 Processor -- start 2021-02-10T11:44:51.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.148 7fd05c6b1700 1 -- start start 2021-02-10T11:44:51.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.148 7fd05c6b1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0541908b0 0x7fd0541a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:51.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.148 7fd05c6b1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0540bd920 0x7fd0540bdd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:51.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.148 7fd05c6b1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd05408d510 0x7fd05408d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:51.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.148 7fd05c6b1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05408cf90 con 0x7fd0540bd920 2021-02-10T11:44:51.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.148 7fd05c6b1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540be340 con 0x7fd05408d510 2021-02-10T11:44:51.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.148 7fd05c6b1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540be680 con 0x7fd0541908b0 2021-02-10T11:44:51.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.148 7fd059ccc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0540bd920 0x7fd0540bdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:51.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.148 7fd059ccc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0540bd920 0x7fd0540bdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60088/0 (socket says 172.21.2.101:60088) 2021-02-10T11:44:51.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.148 7fd059ccc700 1 -- 172.21.2.101:0/144875737 learned_addr learned my addr 172.21.2.101:0/144875737 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:51.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.149 7fd059ccc700 1 -- 172.21.2.101:0/144875737 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0541908b0 msgr2=0x7fd0541a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:51.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.149 7fd059ccc700 1 --2- 172.21.2.101:0/144875737 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0541908b0 0x7fd0541a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:51.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.149 7fd059ccc700 1 -- 172.21.2.101:0/144875737 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd05408d510 msgr2=0x7fd05408d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:51.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.149 7fd059ccc700 1 --2- 172.21.2.101:0/144875737 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd05408d510 0x7fd05408d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:51.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.149 7fd059ccc700 1 -- 172.21.2.101:0/144875737 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0541a4a60 con 0x7fd0540bd920 2021-02-10T11:44:51.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.149 7fd059ccc700 1 --2- 172.21.2.101:0/144875737 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0540bd920 0x7fd0540bdd20 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 rx=0x7fd04c006a90 tx=0x7fd04c00bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:51.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.149 7fd04affd700 1 -- 172.21.2.101:0/144875737 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd04c000d30 con 0x7fd0540bd920 2021-02-10T11:44:51.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.149 7fd04affd700 1 -- 172.21.2.101:0/144875737 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04c006710 con 0x7fd0540bd920 2021-02-10T11:44:51.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.149 7fd04affd700 1 -- 172.21.2.101:0/144875737 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd04c00c570 con 0x7fd0540bd920 2021-02-10T11:44:51.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.149 7fd05c6b1700 1 -- 172.21.2.101:0/144875737 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0541a4a60 con 0x7fd0540bd920 2021-02-10T11:44:51.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.149 7fd05c6b1700 1 -- 172.21.2.101:0/144875737 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0541a4a60 con 0x7fd0540bd920 2021-02-10T11:44:51.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.149 7fd04affd700 1 -- 172.21.2.101:0/144875737 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd04c007000 con 0x7fd0540bd920 2021-02-10T11:44:51.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.149 7fd04affd700 1 --2- 172.21.2.101:0/144875737 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd03c0186c0 0x7fd03c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:51.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.149 7fd04affd700 1 -- 172.21.2.101:0/144875737 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 4839+0+0 (secure 0 0 0) 0x7fd04c0324d0 con 0x7fd0540bd920 2021-02-10T11:44:51.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.149 7fd05a4cd700 1 --2- 172.21.2.101:0/144875737 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd03c0186c0 0x7fd03c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:51.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.150 7fd05a4cd700 1 --2- 172.21.2.101:0/144875737 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd03c0186c0 0x7fd03c01abc0 crc :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:51.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.180 7fd05c6b1700 1 -- 172.21.2.101:0/144875737 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0540be680 con 0x7fd0540bd920 2021-02-10T11:44:51.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.183 7fd04affd700 1 -- 172.21.2.101:0/144875737 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd04c007330 con 0x7fd0540bd920 2021-02-10T11:44:51.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.322 7fd05c6b1700 1 -- 172.21.2.101:0/144875737 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fd0540be680 con 0x7fd0540bd920 2021-02-10T11:44:51.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.668 7fd04affd700 1 -- 172.21.2.101:0/144875737 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v151) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd04c007000 con 0x7fd0540bd920 2021-02-10T11:44:51.687 INFO:teuthology.orchestra.run.gibba001.stderr: removed all blacklist entries 2021-02-10T11:44:51.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.691 7fd05c6b1700 1 -- 172.21.2.101:0/144875737 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd03c0186c0 msgr2=0x7fd03c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:51.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.691 7fd05c6b1700 1 --2- 172.21.2.101:0/144875737 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd03c0186c0 0x7fd03c01abc0 crc :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:51.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.691 7fd05c6b1700 1 -- 172.21.2.101:0/144875737 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0540bd920 msgr2=0x7fd0540bdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:51.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.691 7fd05c6b1700 1 --2- 172.21.2.101:0/144875737 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0540bd920 0x7fd0540bdd20 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 rx=0x7fd04c006a90 tx=0x7fd04c00bd10).stop 2021-02-10T11:44:51.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.691 7fd05c6b1700 1 -- 172.21.2.101:0/144875737 shutdown_connections 2021-02-10T11:44:51.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.691 7fd05c6b1700 1 --2- 172.21.2.101:0/144875737 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd05408d510 0x7fd05408d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:51.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.691 7fd05c6b1700 1 --2- 172.21.2.101:0/144875737 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd0540bd920 0x7fd0540bdd20 unknown :-1 s=CLOSED pgs=2736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:51.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.691 7fd05c6b1700 1 --2- 172.21.2.101:0/144875737 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd03c0186c0 0x7fd03c01abc0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:51.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.691 7fd05c6b1700 1 --2- 172.21.2.101:0/144875737 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0541908b0 0x7fd0541a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:51.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.691 7fd05c6b1700 1 -- 172.21.2.101:0/144875737 >> 172.21.2.101:0/144875737 conn(0x7fd05408f030 msgr2=0x7fd0540b5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:51.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.691 7fd05c6b1700 1 -- 172.21.2.101:0/144875737 shutdown_connections 2021-02-10T11:44:51.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:51.691 7fd05c6b1700 1 -- 172.21.2.101:0/144875737 wait complete. 2021-02-10T11:44:51.698 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2021-02-10T11:44:51.698 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:44:52.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.110 7f6bc1fab700 1 Processor -- start 2021-02-10T11:44:52.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.110 7f6bc1fab700 1 -- start start 2021-02-10T11:44:52.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.110 7f6bc1fab700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6bbc0d0340 0x7f6bbc0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:52.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.110 7f6bc1fab700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6bbc0d1c70 0x7f6bbc0efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:52.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.110 7f6bc1fab700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bbc0f0b10 con 0x7f6bbc0d1870 2021-02-10T11:44:52.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.110 7f6bc1fab700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bbc0e5ed0 con 0x7f6bbc0d1c70 2021-02-10T11:44:52.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.110 7f6bc1fab700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bbc1b0be0 con 0x7f6bbc0d0340 2021-02-10T11:44:52.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.111 7f6bbbfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6bbc0d1c70 0x7f6bbc0efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:52.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.111 7f6bbbfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6bbc0d1c70 0x7f6bbc0efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60094/0 (socket says 172.21.2.101:60094) 2021-02-10T11:44:52.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.111 7f6bbbfff700 1 -- 172.21.2.101:0/3841820709 learned_addr learned my addr 172.21.2.101:0/3841820709 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:52.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.111 7f6bbbfff700 1 -- 172.21.2.101:0/3841820709 >> v2:172.21.2.108:3300/0 conn(0x7f6bbc0d0340 msgr2=0x7f6bbc0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:52.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.111 7f6bbbfff700 1 --2- 172.21.2.101:0/3841820709 >> v2:172.21.2.108:3300/0 conn(0x7f6bbc0d0340 0x7f6bbc0ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:52.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.111 7f6bbbfff700 1 -- 172.21.2.101:0/3841820709 >> v1:172.21.2.101:6789/0 conn(0x7f6bbc0d1870 legacy=0x7f6bbc1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:52.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.111 7f6bbbfff700 1 -- 172.21.2.101:0/3841820709 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bbc0f20a0 con 0x7f6bbc0d1c70 2021-02-10T11:44:52.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.111 7f6bbbfff700 1 --2- 172.21.2.101:0/3841820709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6bbc0d1c70 0x7f6bbc0efc60 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 rx=0x7f6bb4006f00 tx=0x7f6bb4009f10).ready entity=mon.0 client_cookie=7aa25bae07da5055 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:52.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.111 7f6bba7fc700 1 -- 172.21.2.101:0/3841820709 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6bb400b3b0 con 0x7f6bbc0d1c70 2021-02-10T11:44:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.111 7f6bba7fc700 1 -- 172.21.2.101:0/3841820709 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb400b630 con 0x7f6bbc0d1c70 2021-02-10T11:44:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.111 7f6bba7fc700 1 -- 172.21.2.101:0/3841820709 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6bb400bb50 con 0x7f6bbc0d1c70 2021-02-10T11:44:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.111 7f6bc1fab700 1 -- 172.21.2.101:0/3841820709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6bbc0d1c70 msgr2=0x7f6bbc0efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.111 7f6bc1fab700 1 --2- 172.21.2.101:0/3841820709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6bbc0d1c70 0x7f6bbc0efc60 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 rx=0x7f6bb4006f00 tx=0x7f6bb4009f10).stop 2021-02-10T11:44:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.112 7f6bc1fab700 1 -- 172.21.2.101:0/3841820709 shutdown_connections 2021-02-10T11:44:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.112 7f6bc1fab700 1 --2- 172.21.2.101:0/3841820709 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6bbc0d1c70 0x7f6bbc0efc60 unknown :-1 s=CLOSED pgs=2738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.112 7f6bc1fab700 1 --2- 172.21.2.101:0/3841820709 >> v2:172.21.2.108:3300/0 conn(0x7f6bbc0d0340 0x7f6bbc0ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.112 7f6bc1fab700 1 -- 172.21.2.101:0/3841820709 >> 172.21.2.101:0/3841820709 conn(0x7f6bbc0ce450 msgr2=0x7f6bbc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.112 7f6bc1fab700 1 -- 172.21.2.101:0/3841820709 shutdown_connections 2021-02-10T11:44:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.112 7f6bc1fab700 1 -- 172.21.2.101:0/3841820709 wait complete. 2021-02-10T11:44:52.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.127 7f6bc1fab700 1 Processor -- start 2021-02-10T11:44:52.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.127 7f6bc1fab700 1 -- start start 2021-02-10T11:44:52.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.127 7f6bc1fab700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6bbc0b4e00 0x7f6bbc0a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:52.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.127 7f6bc1fab700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6bbc0a3a60 0x7f6bbc0a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:52.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.127 7f6bc1fab700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6bbc094750 0x7f6bbc094b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:52.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.127 7f6bc1fab700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc0a4550 con 0x7f6bbc0a3a60 2021-02-10T11:44:52.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.127 7f6bc1fab700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc18bf30 con 0x7f6bbc094750 2021-02-10T11:44:52.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.127 7f6bc1fab700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc18ba60 con 0x7f6bbc0b4e00 2021-02-10T11:44:52.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.127 7f6bbaffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6bbc0a3a60 0x7f6bbc0a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:52.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.127 7f6bbaffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6bbc0a3a60 0x7f6bbc0a3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60100/0 (socket says 172.21.2.101:60100) 2021-02-10T11:44:52.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.127 7f6bbaffd700 1 -- 172.21.2.101:0/1664906963 learned_addr learned my addr 172.21.2.101:0/1664906963 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:52.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.128 7f6bbaffd700 1 -- 172.21.2.101:0/1664906963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6bbc0b4e00 msgr2=0x7f6bbc0a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:52.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.128 7f6bbaffd700 1 --2- 172.21.2.101:0/1664906963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6bbc0b4e00 0x7f6bbc0a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:52.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.128 7f6bbaffd700 1 -- 172.21.2.101:0/1664906963 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6bbc094750 msgr2=0x7f6bbc094b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:52.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.128 7f6bbaffd700 1 --2- 172.21.2.101:0/1664906963 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6bbc094750 0x7f6bbc094b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:52.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.128 7f6bbaffd700 1 -- 172.21.2.101:0/1664906963 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bbc0a15c0 con 0x7f6bbc0a3a60 2021-02-10T11:44:52.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.128 7f6bbaffd700 1 --2- 172.21.2.101:0/1664906963 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6bbc0a3a60 0x7f6bbc0a3e60 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 rx=0x7f6bac00b530 tx=0x7f6bac00b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:52.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.128 7f6bb3fff700 1 -- 172.21.2.101:0/1664906963 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6bac00bcc0 con 0x7f6bbc0a3a60 2021-02-10T11:44:52.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.128 7f6bb3fff700 1 -- 172.21.2.101:0/1664906963 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bac000c90 con 0x7f6bbc0a3a60 2021-02-10T11:44:52.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.128 7f6bb3fff700 1 -- 172.21.2.101:0/1664906963 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6bac0022a0 con 0x7f6bbc0a3a60 2021-02-10T11:44:52.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.128 7f6bc1fab700 1 -- 172.21.2.101:0/1664906963 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bbc0a15c0 con 0x7f6bbc0a3a60 2021-02-10T11:44:52.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.128 7f6bc1fab700 1 -- 172.21.2.101:0/1664906963 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bbc095460 con 0x7f6bbc0a3a60 2021-02-10T11:44:52.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.129 7f6bb3fff700 1 -- 172.21.2.101:0/1664906963 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6bac005310 con 0x7f6bbc0a3a60 2021-02-10T11:44:52.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.129 7f6bb3fff700 1 --2- 172.21.2.101:0/1664906963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ba00186d0 0x7f6ba001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:52.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.129 7f6bb3fff700 1 -- 172.21.2.101:0/1664906963 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f6bac032460 con 0x7f6bbc0a3a60 2021-02-10T11:44:52.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.129 7f6bbb7fe700 1 --2- 172.21.2.101:0/1664906963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ba00186d0 0x7f6ba001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:52.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.129 7f6bbb7fe700 1 --2- 172.21.2.101:0/1664906963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ba00186d0 0x7f6ba001abd0 crc :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:52.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.160 7f6bc1fab700 1 -- 172.21.2.101:0/1664906963 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bbc0ee0e0 con 0x7f6bbc0a3a60 2021-02-10T11:44:52.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.163 7f6bb3fff700 1 -- 172.21.2.101:0/1664906963 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6bac006b90 con 0x7f6bbc0a3a60 2021-02-10T11:44:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.294 7f6bc1fab700 1 -- 172.21.2.101:0/1664906963 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6bbc0ce450 con 0x7f6bbc0a3a60 2021-02-10T11:44:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.295 7f6bb3fff700 1 -- 172.21.2.101:0/1664906963 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v86) v1 ==== 80+0+4513 (secure 0 0 0) 0x7f6bac005310 con 0x7f6bbc0a3a60 2021-02-10T11:44:52.294 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:44:52.294 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:44:52.294 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:44:52.295 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:52.295 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:44:52.295 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:44:52.295 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:52.295 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:44:52.295 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:44:52.295 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:44:52.295 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:52.295 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:52.295 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:52.295 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:52.295 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:44:52.295 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:44:52.296 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:52.296 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:44:52.296 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:44:52.296 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:44:52.296 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:52.296 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:52.296 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:52.296 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:52.296 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:44:52.296 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:44:52.296 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:52.296 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:44:52.296 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:44:52.297 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:52.297 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:52.297 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:52.297 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:52.297 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:44:52.297 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:44:52.297 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:52.297 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:44:52.297 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:44:52.297 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:52.297 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:52.298 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:52.298 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:52.298 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:44:52.298 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:44:52.298 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:52.298 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:44:52.298 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:44:52.298 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:52.298 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:52.298 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:52.298 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:52.299 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:44:52.299 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:44:52.299 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:52.299 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:44:52.299 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:44:52.299 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:52.299 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:52.299 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:52.299 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:52.299 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:44:52.300 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:44:52.300 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:52.300 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:44:52.300 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:44:52.300 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:52.300 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:52.300 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:52.300 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:52.300 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:44:52.300 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:44:52.300 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:52.300 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:44:52.300 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:44:52.301 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:52.301 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:52.301 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:52.301 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:52.301 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:44:52.301 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:44:52.301 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:52.301 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:44:52.301 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:44:52.301 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:52.301 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:52.301 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:52.302 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:52.302 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:44:52.302 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:44:52.302 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:52.302 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:44:52.302 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:44:52.302 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:52.302 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:52.302 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:52.302 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:52.302 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:44:52.303 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:44:52.303 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:52.303 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:44:52.303 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:44:52.303 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:44:52.303 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:52.303 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:52.303 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:52.303 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:44:52.304 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:44:52.304 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:52.304 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:44:52.304 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:44:52.304 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:44:52.304 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:52.304 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:52.304 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:52.305 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:44:52.305 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:44:52.305 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:52.305 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:44:52.305 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:44:52.305 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:44:52.305 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:52.305 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:52.305 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:52.305 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:44:52.305 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:44:52.305 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:52.305 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:44:52.306 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:44:52.306 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:44:52.306 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:52.306 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:52.306 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:52.306 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:44:52.306 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:44:52.306 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:52.306 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:44:52.306 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:44:52.306 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:44:52.306 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:52.307 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:52.307 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:52.307 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:52.307 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:44:52.307 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:44:52.308 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:52.308 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:44:52.308 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:44:52.308 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:52.309 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:52.309 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:44:52.309 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:52.309 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:44:52.309 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:44:52.309 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:44:52.309 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:44:52.309 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:44:52.309 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:44:52.309 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:52.309 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:52.310 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:44:52.310 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:44:52.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.297 7f6bc1fab700 1 -- 172.21.2.101:0/1664906963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ba00186d0 msgr2=0x7f6ba001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:52.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.297 7f6bc1fab700 1 --2- 172.21.2.101:0/1664906963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ba00186d0 0x7f6ba001abd0 crc :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:52.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.297 7f6bc1fab700 1 -- 172.21.2.101:0/1664906963 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6bbc0a3a60 msgr2=0x7f6bbc0a3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:52.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.297 7f6bc1fab700 1 --2- 172.21.2.101:0/1664906963 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6bbc0a3a60 0x7f6bbc0a3e60 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 rx=0x7f6bac00b530 tx=0x7f6bac00b8d0).stop 2021-02-10T11:44:52.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.297 7f6bc1fab700 1 -- 172.21.2.101:0/1664906963 shutdown_connections 2021-02-10T11:44:52.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.297 7f6bc1fab700 1 --2- 172.21.2.101:0/1664906963 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6bbc094750 0x7f6bbc094b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:52.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.297 7f6bc1fab700 1 --2- 172.21.2.101:0/1664906963 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6bbc0a3a60 0x7f6bbc0a3e60 unknown :-1 s=CLOSED pgs=2739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:52.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.297 7f6bc1fab700 1 --2- 172.21.2.101:0/1664906963 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ba00186d0 0x7f6ba001abd0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:52.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.297 7f6bc1fab700 1 --2- 172.21.2.101:0/1664906963 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6bbc0b4e00 0x7f6bbc0a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:52.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.297 7f6bc1fab700 1 -- 172.21.2.101:0/1664906963 >> 172.21.2.101:0/1664906963 conn(0x7f6bbc18cab0 msgr2=0x7f6bbc092c60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:52.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.297 7f6bc1fab700 1 -- 172.21.2.101:0/1664906963 shutdown_connections 2021-02-10T11:44:52.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.297 7f6bc1fab700 1 -- 172.21.2.101:0/1664906963 wait complete. 2021-02-10T11:44:52.312 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:44:52.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.727 7fef1bf45700 1 Processor -- start 2021-02-10T11:44:52.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.727 7fef1bf45700 1 -- start start 2021-02-10T11:44:52.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.727 7fef1bf45700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fef140d0340 0x7fef140ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:52.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.727 7fef1bf45700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fef140d1c70 0x7fef140efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:52.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.727 7fef1bf45700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef140f0b10 con 0x7fef140d1870 2021-02-10T11:44:52.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.727 7fef1bf45700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef140e5ed0 con 0x7fef140d0340 2021-02-10T11:44:52.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.727 7fef1bf45700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef141b0be0 con 0x7fef140d1c70 2021-02-10T11:44:52.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.728 7fef19560700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fef140d0340 0x7fef140ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:52.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.728 7fef19d61700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fef140d1870 0x7fef141b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60600/0 (socket says 172.21.2.101:60600) 2021-02-10T11:44:52.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.728 7fef19d61700 1 -- 172.21.2.101:0/3746548872 learned_addr learned my addr 172.21.2.101:0/3746548872 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:52.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.728 7fef19560700 1 -- 172.21.2.101:0/3746548872 >> v2:172.21.2.108:3300/0 conn(0x7fef140d1c70 msgr2=0x7fef140efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:52.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.728 7fef19560700 1 --2- 172.21.2.101:0/3746548872 >> v2:172.21.2.108:3300/0 conn(0x7fef140d1c70 0x7fef140efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:52.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.728 7fef19560700 1 -- 172.21.2.101:0/3746548872 >> v1:172.21.2.101:6789/0 conn(0x7fef140d1870 legacy=0x7fef141b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:52.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.728 7fef19560700 1 -- 172.21.2.101:0/3746548872 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef140f20a0 con 0x7fef140d0340 2021-02-10T11:44:52.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.728 7fef19560700 1 --2- 172.21.2.101:0/3746548872 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef140d0340 0x7fef140ef640 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 rx=0x7fef08006650 tx=0x7fef08009a80).ready entity=mon.0 client_cookie=70f98b7bda8481f9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:52.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.728 7fef18d5f700 1 -- 172.21.2.101:0/3746548872 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fef0800c0c0 con 0x7fef140d0340 2021-02-10T11:44:52.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.728 7fef18d5f700 1 -- 172.21.2.101:0/3746548872 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0800c340 con 0x7fef140d0340 2021-02-10T11:44:52.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.728 7fef18d5f700 1 -- 172.21.2.101:0/3746548872 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fef0800c860 con 0x7fef140d0340 2021-02-10T11:44:52.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.728 7fef1bf45700 1 -- 172.21.2.101:0/3746548872 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef140d0340 msgr2=0x7fef140ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:52.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.728 7fef1bf45700 1 --2- 172.21.2.101:0/3746548872 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef140d0340 0x7fef140ef640 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 rx=0x7fef08006650 tx=0x7fef08009a80).stop 2021-02-10T11:44:52.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.729 7fef1bf45700 1 -- 172.21.2.101:0/3746548872 shutdown_connections 2021-02-10T11:44:52.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.729 7fef1bf45700 1 --2- 172.21.2.101:0/3746548872 >> v2:172.21.2.108:3300/0 conn(0x7fef140d1c70 0x7fef140efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:52.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.729 7fef1bf45700 1 --2- 172.21.2.101:0/3746548872 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef140d0340 0x7fef140ef640 unknown :-1 s=CLOSED pgs=2741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:52.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.729 7fef1bf45700 1 -- 172.21.2.101:0/3746548872 >> 172.21.2.101:0/3746548872 conn(0x7fef140ce450 msgr2=0x7fef140d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:52.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.729 7fef1bf45700 1 -- 172.21.2.101:0/3746548872 shutdown_connections 2021-02-10T11:44:52.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.729 7fef1bf45700 1 -- 172.21.2.101:0/3746548872 wait complete. 2021-02-10T11:44:52.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.744 7fef1bf45700 1 Processor -- start 2021-02-10T11:44:52.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.744 7fef1bf45700 1 -- start start 2021-02-10T11:44:52.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.744 7fef1bf45700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef141947b0 0x7fef140a69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:52.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.744 7fef1bf45700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef140a6fc0 0x7fef141a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:52.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.744 7fef1bf45700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef141a1bf0 0x7fef141a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:52.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.744 7fef1bf45700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef140a73c0 con 0x7fef141a1bf0 2021-02-10T11:44:52.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.744 7fef1bf45700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1418f170 con 0x7fef141947b0 2021-02-10T11:44:52.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.744 7fef1bf45700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef14194bb0 con 0x7fef140a6fc0 2021-02-10T11:44:52.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.744 7fef19d61700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef141947b0 0x7fef140a69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:52.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.744 7fef1a562700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef141a1bf0 0x7fef141a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:52.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.744 7fef19d61700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef141947b0 0x7fef140a69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:42366/0 (socket says 172.21.2.101:42366) 2021-02-10T11:44:52.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.744 7fef1a562700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef141a1bf0 0x7fef141a1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60116/0 (socket says 172.21.2.101:60116) 2021-02-10T11:44:52.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.744 7fef19d61700 1 -- 172.21.2.101:0/1126520563 learned_addr learned my addr 172.21.2.101:0/1126520563 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:52.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.744 7fef19560700 1 --2- 172.21.2.101:0/1126520563 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef140a6fc0 0x7fef141a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:52.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.745 7fef1a562700 1 -- 172.21.2.101:0/1126520563 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef140a6fc0 msgr2=0x7fef141a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:52.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.745 7fef1a562700 1 --2- 172.21.2.101:0/1126520563 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef140a6fc0 0x7fef141a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:52.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.745 7fef1a562700 1 -- 172.21.2.101:0/1126520563 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef141947b0 msgr2=0x7fef140a69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:52.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.745 7fef1a562700 1 --2- 172.21.2.101:0/1126520563 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef141947b0 0x7fef140a69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:52.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.745 7fef1a562700 1 -- 172.21.2.101:0/1126520563 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1418ec70 con 0x7fef141a1bf0 2021-02-10T11:44:52.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.745 7fef1a562700 1 --2- 172.21.2.101:0/1126520563 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef141a1bf0 0x7fef141a1ff0 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 rx=0x7fef0c00b890 tx=0x7fef0c00bc30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:52.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.745 7fef067fc700 1 -- 172.21.2.101:0/1126520563 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fef0c005870 con 0x7fef141a1bf0 2021-02-10T11:44:52.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.745 7fef067fc700 1 -- 172.21.2.101:0/1126520563 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0c005af0 con 0x7fef141a1bf0 2021-02-10T11:44:52.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.745 7fef067fc700 1 -- 172.21.2.101:0/1126520563 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fef0c006030 con 0x7fef141a1bf0 2021-02-10T11:44:52.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.745 7fef1bf45700 1 -- 172.21.2.101:0/1126520563 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef1418ec70 con 0x7fef141a1bf0 2021-02-10T11:44:52.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.745 7fef1bf45700 1 -- 172.21.2.101:0/1126520563 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1418ec70 con 0x7fef141a1bf0 2021-02-10T11:44:52.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.746 7fef067fc700 1 -- 172.21.2.101:0/1126520563 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fef0c00bd70 con 0x7fef141a1bf0 2021-02-10T11:44:52.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.746 7fef067fc700 1 --2- 172.21.2.101:0/1126520563 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feefc0186c0 0x7feefc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:52.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.746 7fef067fc700 1 -- 172.21.2.101:0/1126520563 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fef0c0324e0 con 0x7fef141a1bf0 2021-02-10T11:44:52.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.746 7fef19d61700 1 --2- 172.21.2.101:0/1126520563 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feefc0186c0 0x7feefc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:52.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.746 7fef19d61700 1 --2- 172.21.2.101:0/1126520563 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feefc0186c0 0x7feefc01abc0 crc :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:52.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.777 7fef1bf45700 1 -- 172.21.2.101:0/1126520563 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef140ee0e0 con 0x7fef141a1bf0 2021-02-10T11:44:52.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.780 7fef067fc700 1 -- 172.21.2.101:0/1126520563 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fef0c005870 con 0x7fef141a1bf0 2021-02-10T11:44:52.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.906 7fef1bf45700 1 -- 172.21.2.101:0/1126520563 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef140ce450 con 0x7fef141a1bf0 2021-02-10T11:44:52.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.906 7fef067fc700 1 -- 172.21.2.101:0/1126520563 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+11154 (secure 0 0 0) 0x7fef0c00bd70 con 0x7fef141a1bf0 2021-02-10T11:44:52.905 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:44:52.906 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":151,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:44:51.666180","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:44:52.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.908 7fef1bf45700 1 -- 172.21.2.101:0/1126520563 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feefc0186c0 msgr2=0x7feefc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:52.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.908 7fef1bf45700 1 --2- 172.21.2.101:0/1126520563 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feefc0186c0 0x7feefc01abc0 crc :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:52.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.908 7fef1bf45700 1 -- 172.21.2.101:0/1126520563 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef141a1bf0 msgr2=0x7fef141a1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:52.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.908 7fef1bf45700 1 --2- 172.21.2.101:0/1126520563 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef141a1bf0 0x7fef141a1ff0 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 rx=0x7fef0c00b890 tx=0x7fef0c00bc30).stop 2021-02-10T11:44:52.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.908 7fef1bf45700 1 -- 172.21.2.101:0/1126520563 shutdown_connections 2021-02-10T11:44:52.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.908 7fef1bf45700 1 --2- 172.21.2.101:0/1126520563 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feefc0186c0 0x7feefc01abc0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:52.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.908 7fef1bf45700 1 --2- 172.21.2.101:0/1126520563 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fef140a6fc0 0x7fef141a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:52.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.908 7fef1bf45700 1 --2- 172.21.2.101:0/1126520563 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fef141a1bf0 0x7fef141a1ff0 unknown :-1 s=CLOSED pgs=2742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:52.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.908 7fef1bf45700 1 --2- 172.21.2.101:0/1126520563 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fef141947b0 0x7fef140a69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:52.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.908 7fef1bf45700 1 -- 172.21.2.101:0/1126520563 >> 172.21.2.101:0/1126520563 conn(0x7fef141a38c0 msgr2=0x7fef141a2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:52.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.908 7fef1bf45700 1 -- 172.21.2.101:0/1126520563 shutdown_connections 2021-02-10T11:44:52.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:52.908 7fef1bf45700 1 -- 172.21.2.101:0/1126520563 wait complete. 2021-02-10T11:44:52.916 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-10T11:44:52.916 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:44:53.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.334 7f5297442700 1 Processor -- start 2021-02-10T11:44:53.349 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.334 7f5297442700 1 -- start start 2021-02-10T11:44:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.334 7f5297442700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f52900d1870 0x7f52901b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.334 7f5297442700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f52900cff70 0x7f5290109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.334 7f5297442700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52900fe1c0 con 0x7f52901b0ae0 2021-02-10T11:44:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.334 7f5297442700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5290109ad0 con 0x7f52900cff70 2021-02-10T11:44:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.334 7f5297442700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5290109750 con 0x7f52900d1870 2021-02-10T11:44:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.334 7f5295a5f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f52900cff70 0x7f5290109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.334 7f5295a5f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f52900cff70 0x7f5290109130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60120/0 (socket says 172.21.2.101:60120) 2021-02-10T11:44:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.334 7f5295a5f700 1 -- 172.21.2.101:0/2519527005 learned_addr learned my addr 172.21.2.101:0/2519527005 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.335 7f5295a5f700 1 -- 172.21.2.101:0/2519527005 >> v2:172.21.2.108:3300/0 conn(0x7f52900d1870 msgr2=0x7f52901b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.335 7f5295a5f700 1 --2- 172.21.2.101:0/2519527005 >> v2:172.21.2.108:3300/0 conn(0x7f52900d1870 0x7f52901b04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.335 7f5295a5f700 1 -- 172.21.2.101:0/2519527005 >> v1:172.21.2.108:6789/0 conn(0x7f52901b0ae0 legacy=0x7f5290108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:53.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.335 7f5295a5f700 1 -- 172.21.2.101:0/2519527005 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5290088070 con 0x7f52900cff70 2021-02-10T11:44:53.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.335 7f5295a5f700 1 --2- 172.21.2.101:0/2519527005 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52900cff70 0x7f5290109130 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 rx=0x7f5288006f00 tx=0x7f5288009e90).ready entity=mon.0 client_cookie=52df261024ffa01 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:53.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.335 7f5287fff700 1 -- 172.21.2.101:0/2519527005 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f528800b300 con 0x7f52900cff70 2021-02-10T11:44:53.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.335 7f5287fff700 1 -- 172.21.2.101:0/2519527005 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528800b640 con 0x7f52900cff70 2021-02-10T11:44:53.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.335 7f5287fff700 1 -- 172.21.2.101:0/2519527005 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f528800ba90 con 0x7f52900cff70 2021-02-10T11:44:53.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.335 7f5297442700 1 -- 172.21.2.101:0/2519527005 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52900cff70 msgr2=0x7f5290109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:53.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.335 7f5297442700 1 --2- 172.21.2.101:0/2519527005 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52900cff70 0x7f5290109130 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 rx=0x7f5288006f00 tx=0x7f5288009e90).stop 2021-02-10T11:44:53.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.336 7f5297442700 1 -- 172.21.2.101:0/2519527005 shutdown_connections 2021-02-10T11:44:53.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.336 7f5297442700 1 --2- 172.21.2.101:0/2519527005 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52900cff70 0x7f5290109130 unknown :-1 s=CLOSED pgs=2743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:53.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.336 7f5297442700 1 --2- 172.21.2.101:0/2519527005 >> v2:172.21.2.108:3300/0 conn(0x7f52900d1870 0x7f52901b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:53.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.336 7f5297442700 1 -- 172.21.2.101:0/2519527005 >> 172.21.2.101:0/2519527005 conn(0x7f52900ce420 msgr2=0x7f52900d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:53.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.336 7f5297442700 1 -- 172.21.2.101:0/2519527005 shutdown_connections 2021-02-10T11:44:53.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.336 7f5297442700 1 -- 172.21.2.101:0/2519527005 wait complete. 2021-02-10T11:44:53.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.351 7f5297442700 1 Processor -- start 2021-02-10T11:44:53.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.351 7f5297442700 1 -- start start 2021-02-10T11:44:53.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.351 7f5297442700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f52900c3820 0x7f5290084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:53.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.351 7f5297442700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52900a0290 0x7f52900852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:53.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.351 7f5297442700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5290086270 0x7f5290086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:53.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.351 7f5297442700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52900c0700 con 0x7f52900a0290 2021-02-10T11:44:53.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.351 7f5297442700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52900858f0 con 0x7f52900c3820 2021-02-10T11:44:53.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.351 7f5297442700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5290085b10 con 0x7f5290086270 2021-02-10T11:44:53.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.352 7f5294a5d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52900a0290 0x7f52900852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:53.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.352 7f5294a5d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52900a0290 0x7f52900852d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60124/0 (socket says 172.21.2.101:60124) 2021-02-10T11:44:53.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.352 7f5294a5d700 1 -- 172.21.2.101:0/2272106992 learned_addr learned my addr 172.21.2.101:0/2272106992 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:53.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.352 7f5294a5d700 1 -- 172.21.2.101:0/2272106992 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5290086270 msgr2=0x7f5290086670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:53.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.352 7f5294a5d700 1 --2- 172.21.2.101:0/2272106992 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5290086270 0x7f5290086670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:53.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.352 7f5294a5d700 1 -- 172.21.2.101:0/2272106992 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f52900c3820 msgr2=0x7f5290084cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:53.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.352 7f5294a5d700 1 --2- 172.21.2.101:0/2272106992 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f52900c3820 0x7f5290084cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:53.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.352 7f5294a5d700 1 -- 172.21.2.101:0/2272106992 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5290086c90 con 0x7f52900a0290 2021-02-10T11:44:53.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.352 7f5294a5d700 1 --2- 172.21.2.101:0/2272106992 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52900a0290 0x7f52900852d0 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 rx=0x7f5280006ed0 tx=0x7f528000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:53.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.352 7f5285ffb700 1 -- 172.21.2.101:0/2272106992 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5280006240 con 0x7f52900a0290 2021-02-10T11:44:53.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.353 7f5285ffb700 1 -- 172.21.2.101:0/2272106992 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5280000990 con 0x7f52900a0290 2021-02-10T11:44:53.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.353 7f5285ffb700 1 -- 172.21.2.101:0/2272106992 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5280005740 con 0x7f52900a0290 2021-02-10T11:44:53.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.353 7f5297442700 1 -- 172.21.2.101:0/2272106992 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5290086c90 con 0x7f52900a0290 2021-02-10T11:44:53.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.353 7f5297442700 1 -- 172.21.2.101:0/2272106992 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52901a2f00 con 0x7f52900a0290 2021-02-10T11:44:53.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.353 7f5285ffb700 1 -- 172.21.2.101:0/2272106992 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5280005c10 con 0x7f52900a0290 2021-02-10T11:44:53.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.353 7f5285ffb700 1 --2- 172.21.2.101:0/2272106992 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52700186c0 0x7f527001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:53.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.353 7f5285ffb700 1 -- 172.21.2.101:0/2272106992 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f5280032290 con 0x7f52900a0290 2021-02-10T11:44:53.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.353 7f529525e700 1 --2- 172.21.2.101:0/2272106992 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52700186c0 0x7f527001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:53.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.354 7f529525e700 1 --2- 172.21.2.101:0/2272106992 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52700186c0 0x7f527001abc0 crc :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:53.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.384 7f5297442700 1 -- 172.21.2.101:0/2272106992 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5290082a70 con 0x7f52900a0290 2021-02-10T11:44:53.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.386 7f5285ffb700 1 -- 172.21.2.101:0/2272106992 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5280005a00 con 0x7f52900a0290 2021-02-10T11:44:53.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.514 7f5297442700 1 -- 172.21.2.101:0/2272106992 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f5290082a70 con 0x7f52900a0290 2021-02-10T11:44:53.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.514 7f5285ffb700 1 -- 172.21.2.101:0/2272106992 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v151) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5280005c10 con 0x7f52900a0290 2021-02-10T11:44:53.513 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:44:53.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.516 7f5297442700 1 -- 172.21.2.101:0/2272106992 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52700186c0 msgr2=0x7f527001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:53.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.516 7f5297442700 1 --2- 172.21.2.101:0/2272106992 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52700186c0 0x7f527001abc0 crc :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:53.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.516 7f5297442700 1 -- 172.21.2.101:0/2272106992 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52900a0290 msgr2=0x7f52900852d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:53.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.516 7f5297442700 1 --2- 172.21.2.101:0/2272106992 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52900a0290 0x7f52900852d0 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 rx=0x7f5280006ed0 tx=0x7f528000b960).stop 2021-02-10T11:44:53.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.516 7f5297442700 1 -- 172.21.2.101:0/2272106992 shutdown_connections 2021-02-10T11:44:53.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.516 7f5297442700 1 --2- 172.21.2.101:0/2272106992 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52700186c0 0x7f527001abc0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:53.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.516 7f5297442700 1 --2- 172.21.2.101:0/2272106992 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5290086270 0x7f5290086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:53.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.516 7f5297442700 1 --2- 172.21.2.101:0/2272106992 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f52900a0290 0x7f52900852d0 unknown :-1 s=CLOSED pgs=2744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:53.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.516 7f5297442700 1 --2- 172.21.2.101:0/2272106992 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f52900c3820 0x7f5290084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:53.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.516 7f5297442700 1 -- 172.21.2.101:0/2272106992 >> 172.21.2.101:0/2272106992 conn(0x7f52900872c0 msgr2=0x7f5290095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:53.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.517 7f5297442700 1 -- 172.21.2.101:0/2272106992 shutdown_connections 2021-02-10T11:44:53.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.517 7f5297442700 1 -- 172.21.2.101:0/2272106992 wait complete. 2021-02-10T11:44:53.523 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --format=json-pretty osd lspools 2021-02-10T11:44:53.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.936 7fd6f35ca700 1 Processor -- start 2021-02-10T11:44:53.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.936 7fd6f35ca700 1 -- start start 2021-02-10T11:44:53.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.937 7fd6f35ca700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd6ec0d1870 0x7fd6ec1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:53.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.937 7fd6f35ca700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ec0f0b10 con 0x7fd6ec1b0a10 2021-02-10T11:44:53.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.937 7fd6f35ca700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ec0d0310 con 0x7fd6ec0f0650 2021-02-10T11:44:53.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.937 7fd6f35ca700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6ec0d0530 con 0x7fd6ec0d1870 2021-02-10T11:44:53.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.937 7fd6f13e6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd6ec0d1870 0x7fd6ec1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:53.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.937 7fd6f0be5700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd6ec1b0a10 0x7fd6ec0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60626/0 (socket says 172.21.2.101:60626) 2021-02-10T11:44:53.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.937 7fd6f0be5700 1 -- 172.21.2.101:0/2033010329 learned_addr learned my addr 172.21.2.101:0/2033010329 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:53.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.937 7fd6f13e6700 1 -- 172.21.2.101:0/2033010329 >> v1:172.21.2.108:6789/0 conn(0x7fd6ec0f0650 legacy=0x7fd6ec0cfb20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:53.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.937 7fd6f13e6700 1 -- 172.21.2.101:0/2033010329 >> v1:172.21.2.101:6789/0 conn(0x7fd6ec1b0a10 legacy=0x7fd6ec0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:53.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.937 7fd6f13e6700 1 -- 172.21.2.101:0/2033010329 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec0d1c70 con 0x7fd6ec0d1870 2021-02-10T11:44:53.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.937 7fd6f13e6700 1 --2- 172.21.2.101:0/2033010329 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0d1870 0x7fd6ec1b03f0 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 rx=0x7fd6e8006650 tx=0x7fd6e8009a80).ready entity=mon.0 client_cookie=7af892a73ee7e14d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:53.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.937 7fd6dffff700 1 -- 172.21.2.101:0/2033010329 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6e80009b0 con 0x7fd6ec0d1870 2021-02-10T11:44:53.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.937 7fd6dffff700 1 -- 172.21.2.101:0/2033010329 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e800c0c0 con 0x7fd6ec0d1870 2021-02-10T11:44:53.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.937 7fd6dffff700 1 -- 172.21.2.101:0/2033010329 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6e800c510 con 0x7fd6ec0d1870 2021-02-10T11:44:53.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.938 7fd6f35ca700 1 -- 172.21.2.101:0/2033010329 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0d1870 msgr2=0x7fd6ec1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:53.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.938 7fd6f35ca700 1 --2- 172.21.2.101:0/2033010329 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0d1870 0x7fd6ec1b03f0 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 rx=0x7fd6e8006650 tx=0x7fd6e8009a80).stop 2021-02-10T11:44:53.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.938 7fd6f35ca700 1 -- 172.21.2.101:0/2033010329 shutdown_connections 2021-02-10T11:44:53.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.938 7fd6f35ca700 1 --2- 172.21.2.101:0/2033010329 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0d1870 0x7fd6ec1b03f0 unknown :-1 s=CLOSED pgs=2746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:53.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.938 7fd6f35ca700 1 -- 172.21.2.101:0/2033010329 >> 172.21.2.101:0/2033010329 conn(0x7fd6ec0ce450 msgr2=0x7fd6ec0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:53.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.938 7fd6f35ca700 1 -- 172.21.2.101:0/2033010329 shutdown_connections 2021-02-10T11:44:53.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.938 7fd6f35ca700 1 -- 172.21.2.101:0/2033010329 wait complete. 2021-02-10T11:44:53.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.953 7fd6f35ca700 1 Processor -- start 2021-02-10T11:44:53.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.953 7fd6f35ca700 1 -- start start 2021-02-10T11:44:53.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.953 7fd6f35ca700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0a08f0 0x7fd6ec0a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:53.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.953 7fd6f35ca700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6ec0b5230 0x7fd6ec0b5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:53.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.953 7fd6f35ca700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6ec0b5c50 0x7fd6ec0a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:53.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.953 7fd6f35ca700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec0a1600 con 0x7fd6ec0a08f0 2021-02-10T11:44:53.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.953 7fd6f35ca700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec0b6050 con 0x7fd6ec0b5230 2021-02-10T11:44:53.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.953 7fd6f35ca700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec0b6240 con 0x7fd6ec0b5c50 2021-02-10T11:44:53.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.953 7fd6f13e6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0a08f0 0x7fd6ec0a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:53.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.953 7fd6f13e6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0a08f0 0x7fd6ec0a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60136/0 (socket says 172.21.2.101:60136) 2021-02-10T11:44:53.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.953 7fd6f13e6700 1 -- 172.21.2.101:0/2964898067 learned_addr learned my addr 172.21.2.101:0/2964898067 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:53.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.954 7fd6f13e6700 1 -- 172.21.2.101:0/2964898067 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6ec0b5c50 msgr2=0x7fd6ec0a1b00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:53.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.954 7fd6f13e6700 1 --2- 172.21.2.101:0/2964898067 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6ec0b5c50 0x7fd6ec0a1b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:53.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.954 7fd6f13e6700 1 -- 172.21.2.101:0/2964898067 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6ec0b5230 msgr2=0x7fd6ec0b5630 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:44:53.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.954 7fd6f13e6700 1 --2- 172.21.2.101:0/2964898067 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6ec0b5230 0x7fd6ec0b5630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:53.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.954 7fd6f13e6700 1 -- 172.21.2.101:0/2964898067 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec0a2120 con 0x7fd6ec0a08f0 2021-02-10T11:44:53.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.954 7fd6f13e6700 1 --2- 172.21.2.101:0/2964898067 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0a08f0 0x7fd6ec0a0cf0 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 rx=0x7fd6e8009630 tx=0x7fd6e800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:53.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.954 7fd6ddffb700 1 -- 172.21.2.101:0/2964898067 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6e80093a0 con 0x7fd6ec0a08f0 2021-02-10T11:44:53.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.954 7fd6ddffb700 1 -- 172.21.2.101:0/2964898067 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e800bd70 con 0x7fd6ec0a08f0 2021-02-10T11:44:53.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.954 7fd6ddffb700 1 -- 172.21.2.101:0/2964898067 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6e8005560 con 0x7fd6ec0a08f0 2021-02-10T11:44:53.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.954 7fd6f35ca700 1 -- 172.21.2.101:0/2964898067 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ec0a2120 con 0x7fd6ec0a08f0 2021-02-10T11:44:53.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.954 7fd6f35ca700 1 -- 172.21.2.101:0/2964898067 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ec0a2120 con 0x7fd6ec0a08f0 2021-02-10T11:44:53.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.954 7fd6ddffb700 1 -- 172.21.2.101:0/2964898067 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd6e800a000 con 0x7fd6ec0a08f0 2021-02-10T11:44:53.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.955 7fd6ddffb700 1 --2- 172.21.2.101:0/2964898067 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6cc018730 0x7fd6cc01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:53.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.955 7fd6ddffb700 1 -- 172.21.2.101:0/2964898067 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fd6e80324a0 con 0x7fd6ec0a08f0 2021-02-10T11:44:53.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.955 7fd6f0be5700 1 --2- 172.21.2.101:0/2964898067 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6cc018730 0x7fd6cc01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:53.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.955 7fd6f0be5700 1 --2- 172.21.2.101:0/2964898067 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6cc018730 0x7fd6cc01ac10 crc :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:53.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.987 7fd6f35ca700 1 -- 172.21.2.101:0/2964898067 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ec0e7de0 con 0x7fd6ec0a08f0 2021-02-10T11:44:53.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:53.989 7fd6ddffb700 1 -- 172.21.2.101:0/2964898067 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd6e800bd70 con 0x7fd6ec0a08f0 2021-02-10T11:44:54.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.112 7fd6f35ca700 1 -- 172.21.2.101:0/2964898067 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd6ec0e7de0 con 0x7fd6ec0a08f0 2021-02-10T11:44:54.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.112 7fd6ddffb700 1 -- 172.21.2.101:0/2964898067 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v151) v1 ==== 84+0+64 (secure 0 0 0) 0x7fd6e800bd70 con 0x7fd6ec0a08f0 2021-02-10T11:44:54.112 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:44:54.112 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:44:54.112 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:44:54.112 INFO:teuthology.orchestra.run.gibba001.stdout: "poolnum": 1, 2021-02-10T11:44:54.112 INFO:teuthology.orchestra.run.gibba001.stdout: "poolname": "rbd" 2021-02-10T11:44:54.112 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:44:54.112 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:44:54.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.114 7fd6f35ca700 1 -- 172.21.2.101:0/2964898067 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6cc018730 msgr2=0x7fd6cc01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:54.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.114 7fd6f35ca700 1 --2- 172.21.2.101:0/2964898067 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6cc018730 0x7fd6cc01ac10 crc :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:54.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.114 7fd6f35ca700 1 -- 172.21.2.101:0/2964898067 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0a08f0 msgr2=0x7fd6ec0a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:54.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.114 7fd6f35ca700 1 --2- 172.21.2.101:0/2964898067 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0a08f0 0x7fd6ec0a0cf0 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 rx=0x7fd6e8009630 tx=0x7fd6e800b980).stop 2021-02-10T11:44:54.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.115 7fd6f35ca700 1 -- 172.21.2.101:0/2964898067 shutdown_connections 2021-02-10T11:44:54.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.115 7fd6f35ca700 1 --2- 172.21.2.101:0/2964898067 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6cc018730 0x7fd6cc01ac10 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:54.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.115 7fd6f35ca700 1 --2- 172.21.2.101:0/2964898067 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6ec0b5c50 0x7fd6ec0a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:54.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.115 7fd6f35ca700 1 --2- 172.21.2.101:0/2964898067 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6ec0b5230 0x7fd6ec0b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:54.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.115 7fd6f35ca700 1 --2- 172.21.2.101:0/2964898067 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6ec0a08f0 0x7fd6ec0a0cf0 unknown :-1 s=CLOSED pgs=2747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:54.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.115 7fd6f35ca700 1 -- 172.21.2.101:0/2964898067 >> 172.21.2.101:0/2964898067 conn(0x7fd6ec0a0030 msgr2=0x7fd6ec09b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:54.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.115 7fd6f35ca700 1 -- 172.21.2.101:0/2964898067 shutdown_connections 2021-02-10T11:44:54.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.115 7fd6f35ca700 1 -- 172.21.2.101:0/2964898067 wait complete. 2021-02-10T11:44:54.122 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-02-10T11:44:54.122 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-02-10T11:44:54.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.538 7f4da1fc0700 1 Processor -- start 2021-02-10T11:44:54.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.538 7f4da1fc0700 1 -- start start 2021-02-10T11:44:54.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.539 7f4da1fc0700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4d9c0d1c70 0x7f4d9c0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:54.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.539 7f4da1fc0700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d9c0d0170 con 0x7f4d9c0cf580 2021-02-10T11:44:54.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.539 7f4da1fc0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d9c0fe1c0 con 0x7f4d9c0d1870 2021-02-10T11:44:54.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.539 7f4da1fc0700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d9c1b0cb0 con 0x7f4d9c0d1c70 2021-02-10T11:44:54.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.539 7f4d9bfff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f4d9c0cf580 0x7f4d9c0cf980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60642/0 (socket says 172.21.2.101:60642) 2021-02-10T11:44:54.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.539 7f4d9bfff700 1 -- 172.21.2.101:0/1731290885 learned_addr learned my addr 172.21.2.101:0/1731290885 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:54.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.539 7f4d9a7fc700 1 -- 172.21.2.101:0/1731290885 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2225955358 0 0) 0x7f4d94001500 con 0x7f4d9c0cf580 2021-02-10T11:44:54.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.539 7f4d9a7fc700 1 -- 172.21.2.101:0/1731290885 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d88002fe0 con 0x7f4d9c0cf580 2021-02-10T11:44:54.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.539 7f4d9a7fc700 1 -- 172.21.2.101:0/1731290885 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 990242473 0 0) 0x7f4d94001500 con 0x7f4d9c0cf580 2021-02-10T11:44:54.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.539 7f4d9a7fc700 1 -- 172.21.2.101:0/1731290885 >> v2:172.21.2.108:3300/0 conn(0x7f4d9c0d1c70 msgr2=0x7f4d9c0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:54.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.539 7f4d9a7fc700 1 --2- 172.21.2.101:0/1731290885 >> v2:172.21.2.108:3300/0 conn(0x7f4d9c0d1c70 0x7f4d9c0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:54.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.539 7f4d9a7fc700 1 -- 172.21.2.101:0/1731290885 >> v1:172.21.2.108:6789/0 conn(0x7f4d9c0d1870 legacy=0x7f4d9c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:54.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.539 7f4d9a7fc700 1 -- 172.21.2.101:0/1731290885 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9c109a60 con 0x7f4d9c0cf580 2021-02-10T11:44:54.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.539 7f4d9a7fc700 1 -- 172.21.2.101:0/1731290885 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f4d94001a00 con 0x7f4d9c0cf580 2021-02-10T11:44:54.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.539 7f4d9a7fc700 1 -- 172.21.2.101:0/1731290885 >> v1:172.21.2.101:6789/0 conn(0x7f4d9c0cf580 legacy=0x7f4d9c0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:54.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.539 7f4d9a7fc700 1 --2- 172.21.2.101:0/1731290885 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d88003a70 0x7f4d88005f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:54.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.539 7f4d9a7fc700 1 --2- 172.21.2.101:0/1731290885 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d88006790 0x7f4d88008cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:54.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.539 7f4d9a7fc700 1 --2- 172.21.2.101:0/1731290885 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d880095b0 0x7f4d8800bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:54.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.539 7f4d9a7fc700 1 -- 172.21.2.101:0/1731290885 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8800c300 con 0x7f4d88006790 2021-02-10T11:44:54.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.539 7f4d9a7fc700 1 -- 172.21.2.101:0/1731290885 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8800c520 con 0x7f4d88003a70 2021-02-10T11:44:54.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.539 7f4d9a7fc700 1 -- 172.21.2.101:0/1731290885 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8800c740 con 0x7f4d880095b0 2021-02-10T11:44:54.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.540 7f4d9affd700 1 --2- 172.21.2.101:0/1731290885 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d88006790 0x7f4d88008cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:54.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.540 7f4d9b7fe700 1 --2- 172.21.2.101:0/1731290885 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d88003a70 0x7f4d88005f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:54.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.540 7f4d9affd700 1 -- 172.21.2.101:0/1731290885 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d880095b0 msgr2=0x7f4d8800bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:54.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.540 7f4d9affd700 1 --2- 172.21.2.101:0/1731290885 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d880095b0 0x7f4d8800bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:54.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.540 7f4d9affd700 1 -- 172.21.2.101:0/1731290885 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d88003a70 msgr2=0x7f4d88005f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:54.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.540 7f4d9affd700 1 --2- 172.21.2.101:0/1731290885 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d88003a70 0x7f4d88005f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:54.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.540 7f4d9affd700 1 -- 172.21.2.101:0/1731290885 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8800c9e0 con 0x7f4d88006790 2021-02-10T11:44:54.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.540 7f4d9bfff700 1 -- 172.21.2.101:0/1731290885 reap_dead start 2021-02-10T11:44:54.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.540 7f4d9affd700 1 --2- 172.21.2.101:0/1731290885 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d88006790 0x7f4d88008cb0 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 rx=0x7f4d84006bf0 tx=0x7f4d8404fcd0).ready entity=mon.0 client_cookie=6a1b788dbaa2f67c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:54.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.540 7f4d9a7fc700 1 -- 172.21.2.101:0/1731290885 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d840009b0 con 0x7f4d88006790 2021-02-10T11:44:54.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.540 7f4d9a7fc700 1 -- 172.21.2.101:0/1731290885 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d84002030 con 0x7f4d88006790 2021-02-10T11:44:54.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.540 7f4d9a7fc700 1 -- 172.21.2.101:0/1731290885 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d84002510 con 0x7f4d88006790 2021-02-10T11:44:54.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.540 7f4da1fc0700 1 -- 172.21.2.101:0/1731290885 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d88006790 msgr2=0x7f4d88008cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:54.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.540 7f4da1fc0700 1 --2- 172.21.2.101:0/1731290885 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d88006790 0x7f4d88008cb0 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 rx=0x7f4d84006bf0 tx=0x7f4d8404fcd0).stop 2021-02-10T11:44:54.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.541 7f4da1fc0700 1 -- 172.21.2.101:0/1731290885 shutdown_connections 2021-02-10T11:44:54.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.541 7f4da1fc0700 1 --2- 172.21.2.101:0/1731290885 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d88006790 0x7f4d88008cb0 unknown :-1 s=CLOSED pgs=2749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:54.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.541 7f4da1fc0700 1 -- 172.21.2.101:0/1731290885 >> 172.21.2.101:0/1731290885 conn(0x7f4d9c0ce420 msgr2=0x7f4d9c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:54.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.541 7f4da1fc0700 1 -- 172.21.2.101:0/1731290885 shutdown_connections 2021-02-10T11:44:54.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.541 7f4da1fc0700 1 -- 172.21.2.101:0/1731290885 wait complete. 2021-02-10T11:44:54.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.555 7f4da1fc0700 1 Processor -- start 2021-02-10T11:44:54.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.556 7f4da1fc0700 1 -- start start 2021-02-10T11:44:54.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.556 7f4da1fc0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d9c085bc0 0x7f4d9c0863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:54.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.556 7f4da1fc0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d9c0954c0 0x7f4d9c096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:54.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.556 7f4da1fc0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d9c0958c0 0x7f4d9c096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:54.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.556 7f4da1fc0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c095d80 con 0x7f4d9c0954c0 2021-02-10T11:44:54.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.556 7f4da1fc0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c0bfa60 con 0x7f4d9c085bc0 2021-02-10T11:44:54.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.556 7f4da1fc0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c085fc0 con 0x7f4d9c0958c0 2021-02-10T11:44:54.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.556 7f4d9affd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d9c0954c0 0x7f4d9c096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:54.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.556 7f4d9affd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d9c0954c0 0x7f4d9c096300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60156/0 (socket says 172.21.2.101:60156) 2021-02-10T11:44:54.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.556 7f4d9affd700 1 -- 172.21.2.101:0/896995336 learned_addr learned my addr 172.21.2.101:0/896995336 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:54.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.557 7f4d9affd700 1 -- 172.21.2.101:0/896995336 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d9c0958c0 msgr2=0x7f4d9c096920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:54.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.557 7f4d9affd700 1 --2- 172.21.2.101:0/896995336 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d9c0958c0 0x7f4d9c096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:54.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.557 7f4d9affd700 1 -- 172.21.2.101:0/896995336 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d9c085bc0 msgr2=0x7f4d9c0863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:54.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.557 7f4d9affd700 1 --2- 172.21.2.101:0/896995336 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d9c085bc0 0x7f4d9c0863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:54.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.557 7f4d9affd700 1 -- 172.21.2.101:0/896995336 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9c086c10 con 0x7f4d9c0954c0 2021-02-10T11:44:54.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.557 7f4d9affd700 1 --2- 172.21.2.101:0/896995336 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d9c0954c0 0x7f4d9c096300 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 rx=0x7f4d8404f540 tx=0x7f4d840099d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:54.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.557 7f4d93fff700 1 -- 172.21.2.101:0/896995336 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d84000c90 con 0x7f4d9c0954c0 2021-02-10T11:44:54.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.557 7f4d93fff700 1 -- 172.21.2.101:0/896995336 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d84006020 con 0x7f4d9c0954c0 2021-02-10T11:44:54.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.557 7f4d93fff700 1 -- 172.21.2.101:0/896995336 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d840055e0 con 0x7f4d9c0954c0 2021-02-10T11:44:54.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.557 7f4da1fc0700 1 -- 172.21.2.101:0/896995336 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9c086c10 con 0x7f4d9c0954c0 2021-02-10T11:44:54.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.557 7f4da1fc0700 1 -- 172.21.2.101:0/896995336 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9c097230 con 0x7f4d9c0954c0 2021-02-10T11:44:54.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.558 7f4d93fff700 1 -- 172.21.2.101:0/896995336 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4d840068b0 con 0x7f4d9c0954c0 2021-02-10T11:44:54.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.558 7f4d93fff700 1 --2- 172.21.2.101:0/896995336 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4d800186c0 0x7f4d8001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:54.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.558 7f4d93fff700 1 -- 172.21.2.101:0/896995336 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f4d84076250 con 0x7f4d9c0954c0 2021-02-10T11:44:54.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.558 7f4d9b7fe700 1 --2- 172.21.2.101:0/896995336 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4d800186c0 0x7f4d8001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:54.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.558 7f4d9b7fe700 1 --2- 172.21.2.101:0/896995336 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4d800186c0 0x7f4d8001abc0 crc :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:54.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.589 7f4da1fc0700 1 -- 172.21.2.101:0/896995336 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d9c0fe1c0 con 0x7f4d9c0954c0 2021-02-10T11:44:54.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.592 7f4d93fff700 1 -- 172.21.2.101:0/896995336 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4d84059fe0 con 0x7f4d9c0954c0 2021-02-10T11:44:54.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:54.718 7f4da1fc0700 1 -- 172.21.2.101:0/896995336 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"} v 0) v1 -- 0x7f4d9c0fe1c0 con 0x7f4d9c0954c0 2021-02-10T11:44:55.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.134 7f4d93fff700 1 -- 172.21.2.101:0/896995336 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"}]=0 pool 'cephfs_metadata' created v152) v1 ==== 133+0+0 (secure 0 0 0) 0x7f4d84059fe0 con 0x7f4d9c0954c0 2021-02-10T11:44:55.182 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' created 2021-02-10T11:44:55.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.185 7f4da1fc0700 1 -- 172.21.2.101:0/896995336 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4d800186c0 msgr2=0x7f4d8001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:55.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.185 7f4da1fc0700 1 --2- 172.21.2.101:0/896995336 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4d800186c0 0x7f4d8001abc0 crc :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:55.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.185 7f4da1fc0700 1 -- 172.21.2.101:0/896995336 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d9c0954c0 msgr2=0x7f4d9c096300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:55.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.185 7f4da1fc0700 1 --2- 172.21.2.101:0/896995336 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d9c0954c0 0x7f4d9c096300 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 rx=0x7f4d8404f540 tx=0x7f4d840099d0).stop 2021-02-10T11:44:55.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.185 7f4da1fc0700 1 -- 172.21.2.101:0/896995336 shutdown_connections 2021-02-10T11:44:55.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.185 7f4da1fc0700 1 --2- 172.21.2.101:0/896995336 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4d800186c0 0x7f4d8001abc0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:55.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.185 7f4da1fc0700 1 --2- 172.21.2.101:0/896995336 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d9c0958c0 0x7f4d9c096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:55.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.185 7f4da1fc0700 1 --2- 172.21.2.101:0/896995336 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d9c0954c0 0x7f4d9c096300 unknown :-1 s=CLOSED pgs=2750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:55.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.185 7f4da1fc0700 1 --2- 172.21.2.101:0/896995336 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d9c085bc0 0x7f4d9c0863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:55.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.185 7f4da1fc0700 1 -- 172.21.2.101:0/896995336 >> 172.21.2.101:0/896995336 conn(0x7f4d9c0bef00 msgr2=0x7f4d9c09d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:55.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.186 7f4da1fc0700 1 -- 172.21.2.101:0/896995336 shutdown_connections 2021-02-10T11:44:55.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.186 7f4da1fc0700 1 -- 172.21.2.101:0/896995336 wait complete. 2021-02-10T11:44:55.193 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_data 8 2021-02-10T11:44:55.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.608 7fd36259c700 1 Processor -- start 2021-02-10T11:44:55.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.609 7fd36259c700 1 -- start start 2021-02-10T11:44:55.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.609 7fd36259c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd35c0d1870 0x7fd35c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:55.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.609 7fd36259c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd35c0cff70 0x7fd35c109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:55.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.609 7fd36259c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd35c109a60 con 0x7fd35c1b0ae0 2021-02-10T11:44:55.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.609 7fd36259c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd35c109750 con 0x7fd35c0d1870 2021-02-10T11:44:55.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.609 7fd36259c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c088070 con 0x7fd35c0cff70 2021-02-10T11:44:55.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.609 7fd35b7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd35c1b0ae0 0x7fd35c108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60658/0 (socket says 172.21.2.101:60658) 2021-02-10T11:44:55.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.609 7fd35b7fe700 1 -- 172.21.2.101:0/774419215 learned_addr learned my addr 172.21.2.101:0/774419215 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:55.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.609 7fd35bfff700 1 --2- 172.21.2.101:0/774419215 >> v2:172.21.2.108:3300/0 conn(0x7fd35c0d1870 0x7fd35c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:55.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.609 7fd360bb9700 1 --2- 172.21.2.101:0/774419215 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd35c0cff70 0x7fd35c109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:55.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.609 7fd35affd700 1 -- 172.21.2.101:0/774419215 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564466800 0 0) 0x7fd3500010b0 con 0x7fd35c1b0ae0 2021-02-10T11:44:55.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.609 7fd35affd700 1 -- 172.21.2.101:0/774419215 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd344002fe0 con 0x7fd35c1b0ae0 2021-02-10T11:44:55.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35affd700 1 -- 172.21.2.101:0/774419215 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1225741823 0 0) 0x7fd3500010b0 con 0x7fd35c1b0ae0 2021-02-10T11:44:55.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35affd700 1 -- 172.21.2.101:0/774419215 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd35c0cff70 msgr2=0x7fd35c109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:55.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35affd700 1 --2- 172.21.2.101:0/774419215 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd35c0cff70 0x7fd35c109130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:55.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35affd700 1 -- 172.21.2.101:0/774419215 >> v2:172.21.2.108:3300/0 conn(0x7fd35c0d1870 msgr2=0x7fd35c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:55.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35affd700 1 --2- 172.21.2.101:0/774419215 >> v2:172.21.2.108:3300/0 conn(0x7fd35c0d1870 0x7fd35c1b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:55.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35affd700 1 -- 172.21.2.101:0/774419215 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35c0d1c70 con 0x7fd35c1b0ae0 2021-02-10T11:44:55.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35affd700 1 -- 172.21.2.101:0/774419215 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fd3500015b0 con 0x7fd35c1b0ae0 2021-02-10T11:44:55.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35affd700 1 -- 172.21.2.101:0/774419215 >> v1:172.21.2.101:6789/0 conn(0x7fd35c1b0ae0 legacy=0x7fd35c108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:55.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35affd700 1 --2- 172.21.2.101:0/774419215 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd344003af0 0x7fd344005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:55.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35affd700 1 --2- 172.21.2.101:0/774419215 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd34404a830 0x7fd34404cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:55.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35affd700 1 --2- 172.21.2.101:0/774419215 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd34404d650 0x7fd34404fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:55.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35affd700 1 -- 172.21.2.101:0/774419215 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd344050390 con 0x7fd34404a830 2021-02-10T11:44:55.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35affd700 1 -- 172.21.2.101:0/774419215 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3440505b0 con 0x7fd344003af0 2021-02-10T11:44:55.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35affd700 1 -- 172.21.2.101:0/774419215 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3440507d0 con 0x7fd34404d650 2021-02-10T11:44:55.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35b7fe700 1 --2- 172.21.2.101:0/774419215 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd34404a830 0x7fd34404cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:55.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35bfff700 1 --2- 172.21.2.101:0/774419215 >> v2:172.21.2.108:3300/0 conn(0x7fd35c0d1870 0x7fd35c1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:44:55.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35b7fe700 1 -- 172.21.2.101:0/774419215 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd34404d650 msgr2=0x7fd34404fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:55.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35b7fe700 1 --2- 172.21.2.101:0/774419215 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd34404d650 0x7fd34404fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:55.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35b7fe700 1 -- 172.21.2.101:0/774419215 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd344003af0 msgr2=0x7fd344005f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:55.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35b7fe700 1 --2- 172.21.2.101:0/774419215 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd344003af0 0x7fd344005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:55.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35b7fe700 1 -- 172.21.2.101:0/774419215 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd344050a70 con 0x7fd34404a830 2021-02-10T11:44:55.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35b7fe700 1 --2- 172.21.2.101:0/774419215 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd34404a830 0x7fd34404cd50 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 rx=0x7fd350001370 tx=0x7fd350007d30).ready entity=mon.0 client_cookie=209292bd6b75c657 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:55.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35affd700 1 -- 172.21.2.101:0/774419215 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd3500070c0 con 0x7fd34404a830 2021-02-10T11:44:55.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35affd700 1 -- 172.21.2.101:0/774419215 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3500010b0 con 0x7fd34404a830 2021-02-10T11:44:55.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd35affd700 1 -- 172.21.2.101:0/774419215 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd35000a7a0 con 0x7fd34404a830 2021-02-10T11:44:55.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd36259c700 1 -- 172.21.2.101:0/774419215 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd34404a830 msgr2=0x7fd34404cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:55.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd36259c700 1 --2- 172.21.2.101:0/774419215 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd34404a830 0x7fd34404cd50 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 rx=0x7fd350001370 tx=0x7fd350007d30).stop 2021-02-10T11:44:55.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.610 7fd360bb9700 1 -- 172.21.2.101:0/774419215 reap_dead start 2021-02-10T11:44:55.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.611 7fd36259c700 1 -- 172.21.2.101:0/774419215 shutdown_connections 2021-02-10T11:44:55.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.611 7fd36259c700 1 -- 172.21.2.101:0/774419215 >> 172.21.2.101:0/774419215 conn(0x7fd35c0ce420 msgr2=0x7fd35c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:55.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.611 7fd36259c700 1 -- 172.21.2.101:0/774419215 shutdown_connections 2021-02-10T11:44:55.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.611 7fd36259c700 1 -- 172.21.2.101:0/774419215 wait complete. 2021-02-10T11:44:55.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.626 7fd36259c700 1 Processor -- start 2021-02-10T11:44:55.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.626 7fd36259c700 1 -- start start 2021-02-10T11:44:55.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.626 7fd36259c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd35c1a79a0 0x7fd35c1a7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:55.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.626 7fd36259c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd35c0bdb40 0x7fd35c0bdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:55.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.626 7fd36259c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd35c082ca0 0x7fd35c0830a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:55.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.626 7fd36259c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c19d990 con 0x7fd35c0bdb40 2021-02-10T11:44:55.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.626 7fd36259c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c19dbb0 con 0x7fd35c1a79a0 2021-02-10T11:44:55.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.626 7fd36259c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c0836c0 con 0x7fd35c082ca0 2021-02-10T11:44:55.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.626 7fd35b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd35c0bdb40 0x7fd35c0bdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:55.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.626 7fd35b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd35c0bdb40 0x7fd35c0bdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60174/0 (socket says 172.21.2.101:60174) 2021-02-10T11:44:55.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.626 7fd35b7fe700 1 -- 172.21.2.101:0/3668963770 learned_addr learned my addr 172.21.2.101:0/3668963770 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:55.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.626 7fd35bfff700 1 --2- 172.21.2.101:0/3668963770 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd35c1a79a0 0x7fd35c1a7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:55.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.627 7fd360bb9700 1 --2- 172.21.2.101:0/3668963770 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd35c082ca0 0x7fd35c0830a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:55.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.627 7fd35b7fe700 1 -- 172.21.2.101:0/3668963770 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd35c082ca0 msgr2=0x7fd35c0830a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:55.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.627 7fd35b7fe700 1 --2- 172.21.2.101:0/3668963770 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd35c082ca0 0x7fd35c0830a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:55.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.627 7fd35b7fe700 1 -- 172.21.2.101:0/3668963770 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd35c1a79a0 msgr2=0x7fd35c1a7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:55.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.627 7fd35b7fe700 1 --2- 172.21.2.101:0/3668963770 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd35c1a79a0 0x7fd35c1a7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:55.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.627 7fd35b7fe700 1 -- 172.21.2.101:0/3668963770 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35c0838e0 con 0x7fd35c0bdb40 2021-02-10T11:44:55.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.627 7fd35bfff700 1 --2- 172.21.2.101:0/3668963770 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd35c1a79a0 0x7fd35c1a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:44:55.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.627 7fd35b7fe700 1 --2- 172.21.2.101:0/3668963770 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd35c0bdb40 0x7fd35c0bdf40 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 rx=0x7fd35000a590 tx=0x7fd35000a930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:55.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.627 7fd358ff9700 1 -- 172.21.2.101:0/3668963770 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd350000da0 con 0x7fd35c0bdb40 2021-02-10T11:44:55.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.627 7fd358ff9700 1 -- 172.21.2.101:0/3668963770 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35000ad20 con 0x7fd35c0bdb40 2021-02-10T11:44:55.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.627 7fd36259c700 1 -- 172.21.2.101:0/3668963770 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35c0838e0 con 0x7fd35c0bdb40 2021-02-10T11:44:55.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.627 7fd358ff9700 1 -- 172.21.2.101:0/3668963770 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd3500045d0 con 0x7fd35c0bdb40 2021-02-10T11:44:55.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.627 7fd36259c700 1 -- 172.21.2.101:0/3668963770 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35c0838e0 con 0x7fd35c0bdb40 2021-02-10T11:44:55.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.628 7fd358ff9700 1 -- 172.21.2.101:0/3668963770 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd350004140 con 0x7fd35c0bdb40 2021-02-10T11:44:55.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.628 7fd358ff9700 1 --2- 172.21.2.101:0/3668963770 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd33c018730 0x7fd33c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:55.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.628 7fd358ff9700 1 -- 172.21.2.101:0/3668963770 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5107+0+0 (secure 0 0 0) 0x7fd350032590 con 0x7fd35c0bdb40 2021-02-10T11:44:55.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.628 7fd35bfff700 1 --2- 172.21.2.101:0/3668963770 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd33c018730 0x7fd33c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:55.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.628 7fd35bfff700 1 --2- 172.21.2.101:0/3668963770 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd33c018730 0x7fd33c01ac10 crc :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:55.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.659 7fd36259c700 1 -- 172.21.2.101:0/3668963770 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd35c082a70 con 0x7fd35c0bdb40 2021-02-10T11:44:55.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.662 7fd358ff9700 1 -- 172.21.2.101:0/3668963770 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd350000da0 con 0x7fd35c0bdb40 2021-02-10T11:44:55.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:55.792 7fd36259c700 1 -- 172.21.2.101:0/3668963770 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"} v 0) v1 -- 0x7fd35c082a70 con 0x7fd35c0bdb40 2021-02-10T11:44:56.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.141 7fd358ff9700 1 -- 172.21.2.101:0/3668963770 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"}]=0 pool 'cephfs_data' created v154) v1 ==== 125+0+0 (secure 0 0 0) 0x7fd350004140 con 0x7fd35c0bdb40 2021-02-10T11:44:56.156 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' created 2021-02-10T11:44:56.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.159 7fd36259c700 1 -- 172.21.2.101:0/3668963770 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd33c018730 msgr2=0x7fd33c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:56.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.159 7fd36259c700 1 --2- 172.21.2.101:0/3668963770 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd33c018730 0x7fd33c01ac10 crc :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:56.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.159 7fd36259c700 1 -- 172.21.2.101:0/3668963770 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd35c0bdb40 msgr2=0x7fd35c0bdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:56.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.159 7fd36259c700 1 --2- 172.21.2.101:0/3668963770 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd35c0bdb40 0x7fd35c0bdf40 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 rx=0x7fd35000a590 tx=0x7fd35000a930).stop 2021-02-10T11:44:56.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.159 7fd36259c700 1 -- 172.21.2.101:0/3668963770 shutdown_connections 2021-02-10T11:44:56.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.159 7fd36259c700 1 --2- 172.21.2.101:0/3668963770 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd33c018730 0x7fd33c01ac10 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:56.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.159 7fd36259c700 1 --2- 172.21.2.101:0/3668963770 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd35c082ca0 0x7fd35c0830a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:56.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.159 7fd36259c700 1 --2- 172.21.2.101:0/3668963770 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd35c0bdb40 0x7fd35c0bdf40 unknown :-1 s=CLOSED pgs=2753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:56.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.159 7fd36259c700 1 --2- 172.21.2.101:0/3668963770 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd35c1a79a0 0x7fd35c1a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:56.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.159 7fd36259c700 1 -- 172.21.2.101:0/3668963770 >> 172.21.2.101:0/3668963770 conn(0x7fd35c0bef00 msgr2=0x7fd35c09f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:56.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.160 7fd36259c700 1 -- 172.21.2.101:0/3668963770 shutdown_connections 2021-02-10T11:44:56.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.160 7fd36259c700 1 -- 172.21.2.101:0/3668963770 wait complete. 2021-02-10T11:44:56.166 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-02-10T11:44:56.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.584 7f754c17a700 1 Processor -- start 2021-02-10T11:44:56.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.584 7f754c17a700 1 -- start start 2021-02-10T11:44:56.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.584 7f754c17a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75440d1c80 0x7f75440cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:56.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.585 7f754c17a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f75440cf590 0x7f75440cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:56.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.585 7f754c17a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75440fe1d0 con 0x7f75440d1880 2021-02-10T11:44:56.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.585 7f754c17a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75440d2ff0 con 0x7f75440cf590 2021-02-10T11:44:56.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.585 7f754c17a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75441b0cc0 con 0x7f75440d1c80 2021-02-10T11:44:56.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.585 7f7549f96700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f75440d1880 0x7f75441b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:54088/0 (socket says 172.21.2.101:54088) 2021-02-10T11:44:56.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.585 7f7549f96700 1 -- 172.21.2.101:0/581215488 learned_addr learned my addr 172.21.2.101:0/581215488 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:56.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.585 7f754a797700 1 --2- 172.21.2.101:0/581215488 >> v2:172.21.2.108:3300/0 conn(0x7f75440cf590 0x7f75440cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:56.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.585 7f7549795700 1 --2- 172.21.2.101:0/581215488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75440d1c80 0x7f75440cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:56.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.585 7f754a797700 1 -- 172.21.2.101:0/581215488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75440d1c80 msgr2=0x7f75440cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:56.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.585 7f754a797700 1 --2- 172.21.2.101:0/581215488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75440d1c80 0x7f75440cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:56.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.585 7f754a797700 1 -- 172.21.2.101:0/581215488 >> v1:172.21.2.108:6789/0 conn(0x7f75440d1880 legacy=0x7f75441b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:56.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.585 7f754a797700 1 -- 172.21.2.101:0/581215488 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7544109a70 con 0x7f75440cf590 2021-02-10T11:44:56.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.586 7f754a797700 1 --2- 172.21.2.101:0/581215488 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75440cf590 0x7f75440cf990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7f7540006ad0 tx=0x7f7540009ad0).ready entity=mon.1 client_cookie=6255f2dd6820f7fe server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:56.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.586 7f7548f94700 1 -- 172.21.2.101:0/581215488 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f75400066a0 con 0x7f75440cf590 2021-02-10T11:44:56.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.586 7f7548f94700 1 -- 172.21.2.101:0/581215488 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7540006410 con 0x7f75440cf590 2021-02-10T11:44:56.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.586 7f7548f94700 1 -- 172.21.2.101:0/581215488 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f75400024d0 con 0x7f75440cf590 2021-02-10T11:44:56.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.586 7f754c17a700 1 -- 172.21.2.101:0/581215488 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75440cf590 msgr2=0x7f75440cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:56.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.586 7f754c17a700 1 --2- 172.21.2.101:0/581215488 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75440cf590 0x7f75440cf990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7f7540006ad0 tx=0x7f7540009ad0).stop 2021-02-10T11:44:56.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.587 7f754c17a700 1 -- 172.21.2.101:0/581215488 shutdown_connections 2021-02-10T11:44:56.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.587 7f754c17a700 1 --2- 172.21.2.101:0/581215488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f75440d1c80 0x7f75440cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:56.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.587 7f754c17a700 1 --2- 172.21.2.101:0/581215488 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f75440cf590 0x7f75440cf990 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:56.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.587 7f754c17a700 1 -- 172.21.2.101:0/581215488 >> 172.21.2.101:0/581215488 conn(0x7f75440ce430 msgr2=0x7f75440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:56.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.587 7f754c17a700 1 -- 172.21.2.101:0/581215488 shutdown_connections 2021-02-10T11:44:56.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.587 7f754c17a700 1 -- 172.21.2.101:0/581215488 wait complete. 2021-02-10T11:44:56.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.601 7f754c17a700 1 Processor -- start 2021-02-10T11:44:56.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.601 7f754c17a700 1 -- start start 2021-02-10T11:44:56.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.602 7f754c17a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7544190df0 0x7f75441b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:56.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.602 7f754c17a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f754419ba20 0x7f75441b87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:56.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.602 7f754c17a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f754419be20 0x7f754418f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:56.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.602 7f754c17a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75441b8240 con 0x7f754419ba20 2021-02-10T11:44:56.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.602 7f754c17a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75441911f0 con 0x7f7544190df0 2021-02-10T11:44:56.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.602 7f754c17a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75441908a0 con 0x7f754419be20 2021-02-10T11:44:56.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.602 7f7549795700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f754419ba20 0x7f75441b87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:56.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.602 7f7549795700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f754419ba20 0x7f75441b87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60194/0 (socket says 172.21.2.101:60194) 2021-02-10T11:44:56.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.602 7f7549795700 1 -- 172.21.2.101:0/3261877869 learned_addr learned my addr 172.21.2.101:0/3261877869 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:56.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.603 7f7549795700 1 -- 172.21.2.101:0/3261877869 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f754419be20 msgr2=0x7f754418f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:56.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.603 7f7549795700 1 --2- 172.21.2.101:0/3261877869 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f754419be20 0x7f754418f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:56.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.603 7f7549795700 1 -- 172.21.2.101:0/3261877869 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7544190df0 msgr2=0x7f75441b7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:56.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.603 7f7549795700 1 --2- 172.21.2.101:0/3261877869 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7544190df0 0x7f75441b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:56.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.603 7f7549795700 1 -- 172.21.2.101:0/3261877869 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75441b8de0 con 0x7f754419ba20 2021-02-10T11:44:56.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.603 7f7549795700 1 --2- 172.21.2.101:0/3261877869 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f754419ba20 0x7f75441b87c0 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 rx=0x7f753400b560 tx=0x7f753400b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:56.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.603 7f753a7fc700 1 -- 172.21.2.101:0/3261877869 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f753400bcf0 con 0x7f754419ba20 2021-02-10T11:44:56.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.603 7f753a7fc700 1 -- 172.21.2.101:0/3261877869 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7534000c90 con 0x7f754419ba20 2021-02-10T11:44:56.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.603 7f753a7fc700 1 -- 172.21.2.101:0/3261877869 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7534002220 con 0x7f754419ba20 2021-02-10T11:44:56.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.603 7f754c17a700 1 -- 172.21.2.101:0/3261877869 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75441b8de0 con 0x7f754419ba20 2021-02-10T11:44:56.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.603 7f754c17a700 1 -- 172.21.2.101:0/3261877869 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75441b8de0 con 0x7f754419ba20 2021-02-10T11:44:56.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.604 7f753a7fc700 1 -- 172.21.2.101:0/3261877869 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7534006b00 con 0x7f754419ba20 2021-02-10T11:44:56.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.604 7f753a7fc700 1 --2- 172.21.2.101:0/3261877869 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f752c0186c0 0x7f752c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:56.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.604 7f753a7fc700 1 -- 172.21.2.101:0/3261877869 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5457+0+0 (secure 0 0 0) 0x7f7534032510 con 0x7f754419ba20 2021-02-10T11:44:56.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.604 7f7549f96700 1 --2- 172.21.2.101:0/3261877869 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f752c0186c0 0x7f752c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:56.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.604 7f7549f96700 1 --2- 172.21.2.101:0/3261877869 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f752c0186c0 0x7f752c01abc0 crc :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:56.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.634 7f754c17a700 1 -- 172.21.2.101:0/3261877869 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7544082aa0 con 0x7f754419ba20 2021-02-10T11:44:56.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.637 7f753a7fc700 1 -- 172.21.2.101:0/3261877869 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7534030d30 con 0x7f754419ba20 2021-02-10T11:44:56.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.767 7f754c17a700 1 -- 172.21.2.101:0/3261877869 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"} v 0) v1 -- 0x7f7544082aa0 con 0x7f754419ba20 2021-02-10T11:44:56.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.771 7f753a7fc700 1 -- 172.21.2.101:0/3261877869 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"}]=0 new fs with metadata pool 32 and data pool 33 v129) v1 ==== 189+0+0 (secure 0 0 0) 0x7f753401f700 con 0x7f754419ba20 2021-02-10T11:44:56.773 INFO:teuthology.orchestra.run.gibba001.stderr:new fs with metadata pool 32 and data pool 33 2021-02-10T11:44:56.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.777 7f754c17a700 1 -- 172.21.2.101:0/3261877869 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f752c0186c0 msgr2=0x7f752c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:56.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.777 7f754c17a700 1 --2- 172.21.2.101:0/3261877869 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f752c0186c0 0x7f752c01abc0 crc :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:56.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.777 7f754c17a700 1 -- 172.21.2.101:0/3261877869 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f754419ba20 msgr2=0x7f75441b87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:56.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.777 7f754c17a700 1 --2- 172.21.2.101:0/3261877869 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f754419ba20 0x7f75441b87c0 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 rx=0x7f753400b560 tx=0x7f753400b900).stop 2021-02-10T11:44:56.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.777 7f754c17a700 1 -- 172.21.2.101:0/3261877869 shutdown_connections 2021-02-10T11:44:56.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.777 7f754c17a700 1 --2- 172.21.2.101:0/3261877869 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f752c0186c0 0x7f752c01abc0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:56.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.777 7f754c17a700 1 --2- 172.21.2.101:0/3261877869 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f754419be20 0x7f754418f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:56.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.777 7f754c17a700 1 --2- 172.21.2.101:0/3261877869 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f754419ba20 0x7f75441b87c0 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:56.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.777 7f754c17a700 1 --2- 172.21.2.101:0/3261877869 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7544190df0 0x7f75441b7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:56.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.777 7f754c17a700 1 -- 172.21.2.101:0/3261877869 >> 172.21.2.101:0/3261877869 conn(0x7f754419f940 msgr2=0x7f754418c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:56.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.777 7f754c17a700 1 -- 172.21.2.101:0/3261877869 shutdown_connections 2021-02-10T11:44:56.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:56.777 7f754c17a700 1 -- 172.21.2.101:0/3261877869 wait complete. 2021-02-10T11:44:56.783 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:44:57.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.196 7fabeafb4700 1 Processor -- start 2021-02-10T11:44:57.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.196 7fabeafb4700 1 -- start start 2021-02-10T11:44:57.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.196 7fabeafb4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fabe41b0470 0x7fabe41b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:57.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.196 7fabeafb4700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fabe40f20e0 0x7fabe40f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:57.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.196 7fabeafb4700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe40f0b50 con 0x7fabe40ef610 2021-02-10T11:44:57.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.196 7fabeafb4700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabe40f0820 con 0x7fabe41b0470 2021-02-10T11:44:57.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.196 7fabeafb4700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabe40f24e0 con 0x7fabe40f20e0 2021-02-10T11:44:57.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.197 7fabe8dd0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fabe41b0470 0x7fabe41b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:57.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.197 7fabe8dd0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fabe41b0470 0x7fabe41b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60198/0 (socket says 172.21.2.101:60198) 2021-02-10T11:44:57.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.197 7fabe8dd0700 1 -- 172.21.2.101:0/3310623940 learned_addr learned my addr 172.21.2.101:0/3310623940 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:57.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.197 7fabe8dd0700 1 -- 172.21.2.101:0/3310623940 >> v2:172.21.2.108:3300/0 conn(0x7fabe40f20e0 msgr2=0x7fabe40f0200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:57.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.197 7fabe8dd0700 1 --2- 172.21.2.101:0/3310623940 >> v2:172.21.2.108:3300/0 conn(0x7fabe40f20e0 0x7fabe40f0200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:57.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.197 7fabe8dd0700 1 -- 172.21.2.101:0/3310623940 >> v1:172.21.2.108:6789/0 conn(0x7fabe40ef610 legacy=0x7fabe40efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:57.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.197 7fabe8dd0700 1 -- 172.21.2.101:0/3310623940 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe40e52e0 con 0x7fabe41b0470 2021-02-10T11:44:57.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.197 7fabe8dd0700 1 --2- 172.21.2.101:0/3310623940 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabe41b0470 0x7fabe41b0870 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 rx=0x7fabd4006700 tx=0x7fabd4009b00).ready entity=mon.0 client_cookie=ad4f1db29513ac0f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:57.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.197 7fabe37fe700 1 -- 172.21.2.101:0/3310623940 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fabd400c0c0 con 0x7fabe41b0470 2021-02-10T11:44:57.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.197 7fabe37fe700 1 -- 172.21.2.101:0/3310623940 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd400c380 con 0x7fabe41b0470 2021-02-10T11:44:57.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.197 7fabe37fe700 1 -- 172.21.2.101:0/3310623940 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fabd400c860 con 0x7fabe41b0470 2021-02-10T11:44:57.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.197 7fabeafb4700 1 -- 172.21.2.101:0/3310623940 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabe41b0470 msgr2=0x7fabe41b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:57.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.197 7fabeafb4700 1 --2- 172.21.2.101:0/3310623940 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabe41b0470 0x7fabe41b0870 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 rx=0x7fabd4006700 tx=0x7fabd4009b00).stop 2021-02-10T11:44:57.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.198 7fabeafb4700 1 -- 172.21.2.101:0/3310623940 shutdown_connections 2021-02-10T11:44:57.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.198 7fabeafb4700 1 --2- 172.21.2.101:0/3310623940 >> v2:172.21.2.108:3300/0 conn(0x7fabe40f20e0 0x7fabe40f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:57.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.198 7fabeafb4700 1 --2- 172.21.2.101:0/3310623940 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabe41b0470 0x7fabe41b0870 unknown :-1 s=CLOSED pgs=2755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:57.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.198 7fabeafb4700 1 -- 172.21.2.101:0/3310623940 >> 172.21.2.101:0/3310623940 conn(0x7fabe40ce3e0 msgr2=0x7fabe40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:57.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.198 7fabeafb4700 1 -- 172.21.2.101:0/3310623940 shutdown_connections 2021-02-10T11:44:57.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.198 7fabeafb4700 1 -- 172.21.2.101:0/3310623940 wait complete. 2021-02-10T11:44:57.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.213 7fabeafb4700 1 Processor -- start 2021-02-10T11:44:57.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.213 7fabeafb4700 1 -- start start 2021-02-10T11:44:57.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.213 7fabeafb4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabe4092e40 0x7fabe4093240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:57.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.213 7fabeafb4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabe4093860 0x7fabe41a83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:57.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.213 7fabeafb4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabe41a89f0 0x7fabe41a8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:57.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.213 7fabeafb4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe4195ad0 con 0x7fabe41a89f0 2021-02-10T11:44:57.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.213 7fabeafb4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe41a9450 con 0x7fabe4093860 2021-02-10T11:44:57.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.213 7fabeafb4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe40b1b00 con 0x7fabe4092e40 2021-02-10T11:44:57.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.213 7fabe95d1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabe41a89f0 0x7fabe41a8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:57.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.213 7fabe95d1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabe41a89f0 0x7fabe41a8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60204/0 (socket says 172.21.2.101:60204) 2021-02-10T11:44:57.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.213 7fabe95d1700 1 -- 172.21.2.101:0/1633387222 learned_addr learned my addr 172.21.2.101:0/1633387222 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:57.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.214 7fabe95d1700 1 -- 172.21.2.101:0/1633387222 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabe4092e40 msgr2=0x7fabe4093240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:57.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.214 7fabe95d1700 1 --2- 172.21.2.101:0/1633387222 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabe4092e40 0x7fabe4093240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:57.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.214 7fabe95d1700 1 -- 172.21.2.101:0/1633387222 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabe4093860 msgr2=0x7fabe41a83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:57.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.214 7fabe95d1700 1 --2- 172.21.2.101:0/1633387222 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabe4093860 0x7fabe41a83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:57.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.214 7fabe95d1700 1 -- 172.21.2.101:0/1633387222 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe41a99d0 con 0x7fabe41a89f0 2021-02-10T11:44:57.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.214 7fabe95d1700 1 --2- 172.21.2.101:0/1633387222 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabe41a89f0 0x7fabe41a8df0 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 rx=0x7fabdc009fc0 tx=0x7fabdc00bc40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:57.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.214 7fabe17fa700 1 -- 172.21.2.101:0/1633387222 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fabdc000d10 con 0x7fabe41a89f0 2021-02-10T11:44:57.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.214 7fabe17fa700 1 -- 172.21.2.101:0/1633387222 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabdc003020 con 0x7fabe41a89f0 2021-02-10T11:44:57.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.214 7fabeafb4700 1 -- 172.21.2.101:0/1633387222 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe41a99d0 con 0x7fabe41a89f0 2021-02-10T11:44:57.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.214 7fabeafb4700 1 -- 172.21.2.101:0/1633387222 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe41a9f60 con 0x7fabe41a89f0 2021-02-10T11:44:57.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.214 7fabe17fa700 1 -- 172.21.2.101:0/1633387222 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fabdc005ad0 con 0x7fabe41a89f0 2021-02-10T11:44:57.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.215 7fabe17fa700 1 -- 172.21.2.101:0/1633387222 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fabdc005850 con 0x7fabe41a89f0 2021-02-10T11:44:57.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.215 7fabe17fa700 1 --2- 172.21.2.101:0/1633387222 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fabc80186d0 0x7fabc801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:57.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.215 7fabe17fa700 1 -- 172.21.2.101:0/1633387222 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fabdc032b40 con 0x7fabe41a89f0 2021-02-10T11:44:57.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.215 7fabe8dd0700 1 --2- 172.21.2.101:0/1633387222 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fabc80186d0 0x7fabc801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:57.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.215 7fabe8dd0700 1 --2- 172.21.2.101:0/1633387222 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fabc80186d0 0x7fabc801abd0 crc :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:57.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.246 7fabeafb4700 1 -- 172.21.2.101:0/1633387222 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe40e52e0 con 0x7fabe41a89f0 2021-02-10T11:44:57.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.249 7fabe17fa700 1 -- 172.21.2.101:0/1633387222 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fabdc005a60 con 0x7fabe41a89f0 2021-02-10T11:44:57.370 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.372 7fabeafb4700 1 -- 172.21.2.101:0/1633387222 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabe40e52e0 con 0x7fabe41a89f0 2021-02-10T11:44:57.371 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.373 7fabe17fa700 1 -- 172.21.2.101:0/1633387222 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+14020 (secure 0 0 0) 0x7fabdc005850 con 0x7fabe41a89f0 2021-02-10T11:44:57.373 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:44:57.373 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":155,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:44:56.768376","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:44:54.736504","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":33,"pool_name":"cephfs_data","create_time":"2021-02-10 11:44:55.811077","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:44:57.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.374 7fabeafb4700 1 -- 172.21.2.101:0/1633387222 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fabc80186d0 msgr2=0x7fabc801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:57.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.374 7fabeafb4700 1 --2- 172.21.2.101:0/1633387222 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fabc80186d0 0x7fabc801abd0 crc :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:57.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.374 7fabeafb4700 1 -- 172.21.2.101:0/1633387222 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabe41a89f0 msgr2=0x7fabe41a8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:57.373 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.374 7fabeafb4700 1 --2- 172.21.2.101:0/1633387222 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabe41a89f0 0x7fabe41a8df0 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 rx=0x7fabdc009fc0 tx=0x7fabdc00bc40).stop 2021-02-10T11:44:57.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.374 7fabeafb4700 1 -- 172.21.2.101:0/1633387222 shutdown_connections 2021-02-10T11:44:57.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.374 7fabeafb4700 1 --2- 172.21.2.101:0/1633387222 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabe41a89f0 0x7fabe41a8df0 unknown :-1 s=CLOSED pgs=2756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:57.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.374 7fabeafb4700 1 --2- 172.21.2.101:0/1633387222 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabe4093860 0x7fabe41a83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:57.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.374 7fabeafb4700 1 --2- 172.21.2.101:0/1633387222 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fabc80186d0 0x7fabc801abd0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:57.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.374 7fabeafb4700 1 --2- 172.21.2.101:0/1633387222 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabe4092e40 0x7fabe4093240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:57.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.374 7fabeafb4700 1 -- 172.21.2.101:0/1633387222 >> 172.21.2.101:0/1633387222 conn(0x7fabe419f7a0 msgr2=0x7fabe40940e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:57.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.374 7fabeafb4700 1 -- 172.21.2.101:0/1633387222 shutdown_connections 2021-02-10T11:44:57.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.375 7fabeafb4700 1 -- 172.21.2.101:0/1633387222 wait complete. 2021-02-10T11:44:57.382 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:44:57.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.802 7fe36e624700 1 Processor -- start 2021-02-10T11:44:57.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.802 7fe36e624700 1 -- start start 2021-02-10T11:44:57.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.802 7fe36e624700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3681b0870 0x7fe3680f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:57.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.802 7fe36e624700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe3680efe00 0x7fe3680f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:57.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.802 7fe36e624700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3681b0c70 con 0x7fe3681b0470 2021-02-10T11:44:57.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.802 7fe36e624700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3680f0b50 con 0x7fe3680efe00 2021-02-10T11:44:57.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.802 7fe36e624700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3680f0820 con 0x7fe3681b0870 2021-02-10T11:44:57.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.802 7fe36cc41700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe3680efe00 0x7fe3680f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:57.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.803 7fe36cc41700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe3680efe00 0x7fe3680f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60216/0 (socket says 172.21.2.101:60216) 2021-02-10T11:44:57.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.803 7fe367fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe3681b0470 0x7fe3680ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60706/0 (socket says 172.21.2.101:60706) 2021-02-10T11:44:57.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.803 7fe36cc41700 1 -- 172.21.2.101:0/943926155 learned_addr learned my addr 172.21.2.101:0/943926155 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:57.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.803 7fe3677fe700 1 --2- 172.21.2.101:0/943926155 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3681b0870 0x7fe3680f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:57.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.803 7fe36cc41700 1 -- 172.21.2.101:0/943926155 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3681b0870 msgr2=0x7fe3680f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:57.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.803 7fe36cc41700 1 --2- 172.21.2.101:0/943926155 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3681b0870 0x7fe3680f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:57.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.803 7fe36cc41700 1 -- 172.21.2.101:0/943926155 >> v1:172.21.2.101:6789/0 conn(0x7fe3681b0470 legacy=0x7fe3680ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:57.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.803 7fe36cc41700 1 -- 172.21.2.101:0/943926155 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3680e52e0 con 0x7fe3680efe00 2021-02-10T11:44:57.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.803 7fe36cc41700 1 --2- 172.21.2.101:0/943926155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3680efe00 0x7fe3680f0200 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 rx=0x7fe358006b80 tx=0x7fe358009b10).ready entity=mon.0 client_cookie=beab908ee893c71b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:57.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.803 7fe366ffd700 1 -- 172.21.2.101:0/943926155 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe3580022e0 con 0x7fe3680efe00 2021-02-10T11:44:57.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.803 7fe366ffd700 1 -- 172.21.2.101:0/943926155 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe358002560 con 0x7fe3680efe00 2021-02-10T11:44:57.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.803 7fe366ffd700 1 -- 172.21.2.101:0/943926155 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe358002a80 con 0x7fe3680efe00 2021-02-10T11:44:57.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.803 7fe36e624700 1 -- 172.21.2.101:0/943926155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3680efe00 msgr2=0x7fe3680f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:57.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.803 7fe36e624700 1 --2- 172.21.2.101:0/943926155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3680efe00 0x7fe3680f0200 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 rx=0x7fe358006b80 tx=0x7fe358009b10).stop 2021-02-10T11:44:57.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.804 7fe36e624700 1 -- 172.21.2.101:0/943926155 shutdown_connections 2021-02-10T11:44:57.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.804 7fe36e624700 1 --2- 172.21.2.101:0/943926155 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3681b0870 0x7fe3680f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:57.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.804 7fe36e624700 1 --2- 172.21.2.101:0/943926155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3680efe00 0x7fe3680f0200 unknown :-1 s=CLOSED pgs=2758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:57.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.804 7fe36e624700 1 -- 172.21.2.101:0/943926155 >> 172.21.2.101:0/943926155 conn(0x7fe3680ce3e0 msgr2=0x7fe3680e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:57.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.804 7fe36e624700 1 -- 172.21.2.101:0/943926155 shutdown_connections 2021-02-10T11:44:57.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.804 7fe36e624700 1 -- 172.21.2.101:0/943926155 wait complete. 2021-02-10T11:44:57.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.819 7fe36e624700 1 Processor -- start 2021-02-10T11:44:57.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.819 7fe36e624700 1 -- start start 2021-02-10T11:44:57.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.819 7fe36e624700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3681b6440 0x7fe3681a9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:57.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.819 7fe36e624700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3681b6840 0x7fe36818e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:57.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.819 7fe36e624700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe36818ea20 0x7fe368197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:57.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.819 7fe36e624700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe368197960 con 0x7fe3681b6840 2021-02-10T11:44:57.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.819 7fe36e624700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36818ee20 con 0x7fe36818ea20 2021-02-10T11:44:57.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.819 7fe36e624700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36818f040 con 0x7fe3681b6440 2021-02-10T11:44:57.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.819 7fe3677fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3681b6840 0x7fe36818e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:57.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.819 7fe3677fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3681b6840 0x7fe36818e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60220/0 (socket says 172.21.2.101:60220) 2021-02-10T11:44:57.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.819 7fe3677fe700 1 -- 172.21.2.101:0/3552420464 learned_addr learned my addr 172.21.2.101:0/3552420464 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:57.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.819 7fe367fff700 1 --2- 172.21.2.101:0/3552420464 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3681b6440 0x7fe3681a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:57.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.819 7fe36cc41700 1 --2- 172.21.2.101:0/3552420464 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe36818ea20 0x7fe368197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:57.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.820 7fe3677fe700 1 -- 172.21.2.101:0/3552420464 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3681b6440 msgr2=0x7fe3681a9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:57.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.820 7fe3677fe700 1 --2- 172.21.2.101:0/3552420464 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3681b6440 0x7fe3681a9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:57.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.820 7fe3677fe700 1 -- 172.21.2.101:0/3552420464 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe36818ea20 msgr2=0x7fe368197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:57.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.820 7fe3677fe700 1 --2- 172.21.2.101:0/3552420464 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe36818ea20 0x7fe368197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:57.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.820 7fe3677fe700 1 -- 172.21.2.101:0/3552420464 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe368198420 con 0x7fe3681b6840 2021-02-10T11:44:57.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.820 7fe36cc41700 1 --2- 172.21.2.101:0/3552420464 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe36818ea20 0x7fe368197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:44:57.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.820 7fe3677fe700 1 --2- 172.21.2.101:0/3552420464 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3681b6840 0x7fe36818e400 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 rx=0x7fe35c006be0 tx=0x7fe35c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:57.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.820 7fe364ff9700 1 -- 172.21.2.101:0/3552420464 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe35c0064d0 con 0x7fe3681b6840 2021-02-10T11:44:57.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.820 7fe364ff9700 1 -- 172.21.2.101:0/3552420464 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35c006860 con 0x7fe3681b6840 2021-02-10T11:44:57.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.820 7fe36e624700 1 -- 172.21.2.101:0/3552420464 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe368198420 con 0x7fe3681b6840 2021-02-10T11:44:57.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.820 7fe364ff9700 1 -- 172.21.2.101:0/3552420464 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe35c002030 con 0x7fe3681b6840 2021-02-10T11:44:57.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.820 7fe36e624700 1 -- 172.21.2.101:0/3552420464 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3681989b0 con 0x7fe3681b6840 2021-02-10T11:44:57.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.820 7fe364ff9700 1 -- 172.21.2.101:0/3552420464 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe35c000c90 con 0x7fe3681b6840 2021-02-10T11:44:57.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.821 7fe364ff9700 1 --2- 172.21.2.101:0/3552420464 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3480186c0 0x7fe34801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:57.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.821 7fe364ff9700 1 -- 172.21.2.101:0/3552420464 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe35c032460 con 0x7fe3681b6840 2021-02-10T11:44:57.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.821 7fe367fff700 1 --2- 172.21.2.101:0/3552420464 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3480186c0 0x7fe34801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:57.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.821 7fe367fff700 1 --2- 172.21.2.101:0/3552420464 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3480186c0 0x7fe34801abc0 crc :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:57.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.852 7fe36e624700 1 -- 172.21.2.101:0/3552420464 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe368105a40 con 0x7fe3681b6840 2021-02-10T11:44:57.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.855 7fe364ff9700 1 -- 172.21.2.101:0/3552420464 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe35c030d90 con 0x7fe3681b6840 2021-02-10T11:44:57.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.980 7fe36e624700 1 -- 172.21.2.101:0/3552420464 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3680f0b50 con 0x7fe3681b6840 2021-02-10T11:44:57.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.981 7fe364ff9700 1 -- 172.21.2.101:0/3552420464 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+14007 (secure 0 0 0) 0x7fe35c000c90 con 0x7fe3681b6840 2021-02-10T11:44:57.980 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:44:57.980 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":156,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:44:57.770382","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:44:54.736504","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":33,"pool_name":"cephfs_data","create_time":"2021-02-10 11:44:55.811077","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:44:57.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.983 7fe36e624700 1 -- 172.21.2.101:0/3552420464 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3480186c0 msgr2=0x7fe34801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:57.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.983 7fe36e624700 1 --2- 172.21.2.101:0/3552420464 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3480186c0 0x7fe34801abc0 crc :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:57.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.983 7fe36e624700 1 -- 172.21.2.101:0/3552420464 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3681b6840 msgr2=0x7fe36818e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:57.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.983 7fe36e624700 1 --2- 172.21.2.101:0/3552420464 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3681b6840 0x7fe36818e400 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 rx=0x7fe35c006be0 tx=0x7fe35c00b990).stop 2021-02-10T11:44:57.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.983 7fe36e624700 1 -- 172.21.2.101:0/3552420464 shutdown_connections 2021-02-10T11:44:57.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.983 7fe36e624700 1 --2- 172.21.2.101:0/3552420464 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe36818ea20 0x7fe368197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:57.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.983 7fe36e624700 1 --2- 172.21.2.101:0/3552420464 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe3681b6840 0x7fe36818e400 unknown :-1 s=CLOSED pgs=2759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:57.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.983 7fe36e624700 1 --2- 172.21.2.101:0/3552420464 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3480186c0 0x7fe34801abc0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:57.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.983 7fe36e624700 1 --2- 172.21.2.101:0/3552420464 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe3681b6440 0x7fe3681a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:57.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.983 7fe36e624700 1 -- 172.21.2.101:0/3552420464 >> 172.21.2.101:0/3552420464 conn(0x7fe3681a37c0 msgr2=0x7fe3681b9230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:57.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.983 7fe36e624700 1 -- 172.21.2.101:0/3552420464 shutdown_connections 2021-02-10T11:44:57.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:57.983 7fe36e624700 1 -- 172.21.2.101:0/3552420464 wait complete. 2021-02-10T11:44:57.991 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-02-10T11:44:58.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.407 7f9fd8de4700 1 Processor -- start 2021-02-10T11:44:58.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.407 7f9fd8de4700 1 -- start start 2021-02-10T11:44:58.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.408 7f9fd8de4700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f9fd40d1c70 0x7f9fd40cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:58.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.408 7f9fd8de4700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd40d0170 con 0x7f9fd40cf580 2021-02-10T11:44:58.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.408 7f9fd8de4700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd40fe1c0 con 0x7f9fd40d1870 2021-02-10T11:44:58.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.408 7f9fd8de4700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd41b0cb0 con 0x7f9fd40d1c70 2021-02-10T11:44:58.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.408 7f9fd3314700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f9fd40cf580 0x7f9fd40cf980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60724/0 (socket says 172.21.2.101:60724) 2021-02-10T11:44:58.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.408 7f9fd3314700 1 -- 172.21.2.101:0/4080044378 learned_addr learned my addr 172.21.2.101:0/4080044378 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:58.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.408 7f9fd2312700 1 --2- 172.21.2.101:0/4080044378 >> v2:172.21.2.108:3300/0 conn(0x7f9fd40d1c70 0x7f9fd40cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:58.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd1b11700 1 -- 172.21.2.101:0/4080044378 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3832696871 0 0) 0x7f9fcc001500 con 0x7f9fd40cf580 2021-02-10T11:44:58.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd1b11700 1 -- 172.21.2.101:0/4080044378 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fc0002fe0 con 0x7f9fd40cf580 2021-02-10T11:44:58.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd1b11700 1 -- 172.21.2.101:0/4080044378 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3778278733 0 0) 0x7f9fcc001460 con 0x7f9fd40cf580 2021-02-10T11:44:58.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd1b11700 1 -- 172.21.2.101:0/4080044378 >> v2:172.21.2.108:3300/0 conn(0x7f9fd40d1c70 msgr2=0x7f9fd40cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:58.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd1b11700 1 --2- 172.21.2.101:0/4080044378 >> v2:172.21.2.108:3300/0 conn(0x7f9fd40d1c70 0x7f9fd40cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:58.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd1b11700 1 -- 172.21.2.101:0/4080044378 >> v1:172.21.2.108:6789/0 conn(0x7f9fd40d1870 legacy=0x7f9fd41b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:58.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd1b11700 1 -- 172.21.2.101:0/4080044378 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd4109a60 con 0x7f9fd40cf580 2021-02-10T11:44:58.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd1b11700 1 -- 172.21.2.101:0/4080044378 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f9fcc001980 con 0x7f9fd40cf580 2021-02-10T11:44:58.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd1b11700 1 -- 172.21.2.101:0/4080044378 >> v1:172.21.2.101:6789/0 conn(0x7f9fd40cf580 legacy=0x7f9fd40cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:58.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd1b11700 1 --2- 172.21.2.101:0/4080044378 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fc0003b10 0x7f9fc0005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:58.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd1b11700 1 --2- 172.21.2.101:0/4080044378 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9fc0006790 0x7f9fc0008cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:58.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd1b11700 1 --2- 172.21.2.101:0/4080044378 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fc0009590 0x7f9fc000bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:58.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd1b11700 1 -- 172.21.2.101:0/4080044378 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc000c300 con 0x7f9fc0009590 2021-02-10T11:44:58.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd1b11700 1 -- 172.21.2.101:0/4080044378 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc000c520 con 0x7f9fc0006790 2021-02-10T11:44:58.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd1b11700 1 -- 172.21.2.101:0/4080044378 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc000c740 con 0x7f9fc0003b10 2021-02-10T11:44:58.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd3314700 1 --2- 172.21.2.101:0/4080044378 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fc0009590 0x7f9fc000bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:58.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd2b13700 1 --2- 172.21.2.101:0/4080044378 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fc0003b10 0x7f9fc0005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:58.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd2312700 1 --2- 172.21.2.101:0/4080044378 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9fc0006790 0x7f9fc0008cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:58.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd3314700 1 -- 172.21.2.101:0/4080044378 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fc0003b10 msgr2=0x7f9fc0005f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:58.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd3314700 1 --2- 172.21.2.101:0/4080044378 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fc0003b10 0x7f9fc0005f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:58.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd3314700 1 -- 172.21.2.101:0/4080044378 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9fc0006790 msgr2=0x7f9fc0008cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:44:58.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd3314700 1 --2- 172.21.2.101:0/4080044378 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9fc0006790 0x7f9fc0008cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:58.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd3314700 1 -- 172.21.2.101:0/4080044378 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc00509b0 con 0x7f9fc0009590 2021-02-10T11:44:58.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd3314700 1 -- 172.21.2.101:0/4080044378 reap_dead start 2021-02-10T11:44:58.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.409 7f9fd3314700 1 --2- 172.21.2.101:0/4080044378 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fc0009590 0x7f9fc000bab0 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 rx=0x7f9fcc001ae0 tx=0x7f9fcc00a060).ready entity=mon.0 client_cookie=770f403d3849cca6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:58.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.410 7f9fd1b11700 1 -- 172.21.2.101:0/4080044378 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9fcc0070c0 con 0x7f9fc0009590 2021-02-10T11:44:58.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.410 7f9fd1b11700 1 -- 172.21.2.101:0/4080044378 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc0073c0 con 0x7f9fc0009590 2021-02-10T11:44:58.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.410 7f9fd1b11700 1 -- 172.21.2.101:0/4080044378 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9fcc0049a0 con 0x7f9fc0009590 2021-02-10T11:44:58.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.410 7f9fd8de4700 1 -- 172.21.2.101:0/4080044378 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fc0009590 msgr2=0x7f9fc000bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:58.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.410 7f9fd8de4700 1 --2- 172.21.2.101:0/4080044378 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fc0009590 0x7f9fc000bab0 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 rx=0x7f9fcc001ae0 tx=0x7f9fcc00a060).stop 2021-02-10T11:44:58.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.410 7f9fd8de4700 1 -- 172.21.2.101:0/4080044378 shutdown_connections 2021-02-10T11:44:58.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.410 7f9fd8de4700 1 --2- 172.21.2.101:0/4080044378 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fc0009590 0x7f9fc000bab0 unknown :-1 s=CLOSED pgs=2761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:58.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.410 7f9fd8de4700 1 -- 172.21.2.101:0/4080044378 >> 172.21.2.101:0/4080044378 conn(0x7f9fd40ce420 msgr2=0x7f9fd40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:58.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.410 7f9fd8de4700 1 -- 172.21.2.101:0/4080044378 shutdown_connections 2021-02-10T11:44:58.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.410 7f9fd8de4700 1 -- 172.21.2.101:0/4080044378 wait complete. 2021-02-10T11:44:58.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.425 7f9fd8de4700 1 Processor -- start 2021-02-10T11:44:58.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.425 7f9fd8de4700 1 -- start start 2021-02-10T11:44:58.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.425 7f9fd8de4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fd409bcc0 0x7f9fd409c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:58.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.425 7f9fd8de4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9fd40a0820 0x7f9fd40a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:58.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.425 7f9fd8de4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fd40a1240 0x7f9fd40858d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:58.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.425 7f9fd8de4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd40a66b0 con 0x7f9fd40a1240 2021-02-10T11:44:58.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.425 7f9fd8de4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd41a49b0 con 0x7f9fd40a0820 2021-02-10T11:44:58.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.425 7f9fd8de4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd41a4b90 con 0x7f9fd409bcc0 2021-02-10T11:44:58.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.426 7f9fd3314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fd40a1240 0x7f9fd40858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:58.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.426 7f9fd3314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fd40a1240 0x7f9fd40858d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60240/0 (socket says 172.21.2.101:60240) 2021-02-10T11:44:58.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.426 7f9fd3314700 1 -- 172.21.2.101:0/3214923463 learned_addr learned my addr 172.21.2.101:0/3214923463 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:58.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.426 7f9fd3314700 1 -- 172.21.2.101:0/3214923463 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fd409bcc0 msgr2=0x7f9fd409c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:58.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.426 7f9fd3314700 1 --2- 172.21.2.101:0/3214923463 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fd409bcc0 0x7f9fd409c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:58.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.426 7f9fd3314700 1 -- 172.21.2.101:0/3214923463 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9fd40a0820 msgr2=0x7f9fd40a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:58.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.426 7f9fd3314700 1 --2- 172.21.2.101:0/3214923463 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9fd40a0820 0x7f9fd40a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:58.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.426 7f9fd3314700 1 -- 172.21.2.101:0/3214923463 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd40a1640 con 0x7f9fd40a1240 2021-02-10T11:44:58.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.426 7f9fd3314700 1 --2- 172.21.2.101:0/3214923463 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fd40a1240 0x7f9fd40858d0 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 rx=0x7f9fcc007400 tx=0x7f9fcc0077a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:58.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.426 7f9fcb7fe700 1 -- 172.21.2.101:0/3214923463 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9fcc00b310 con 0x7f9fd40a1240 2021-02-10T11:44:58.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.426 7f9fcb7fe700 1 -- 172.21.2.101:0/3214923463 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc00b610 con 0x7f9fd40a1240 2021-02-10T11:44:58.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.426 7f9fcb7fe700 1 -- 172.21.2.101:0/3214923463 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9fcc00baf0 con 0x7f9fd40a1240 2021-02-10T11:44:58.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.426 7f9fd8de4700 1 -- 172.21.2.101:0/3214923463 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd40a1640 con 0x7f9fd40a1240 2021-02-10T11:44:58.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.427 7f9fd8de4700 1 -- 172.21.2.101:0/3214923463 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd40a1640 con 0x7f9fd40a1240 2021-02-10T11:44:58.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.427 7f9fcb7fe700 1 -- 172.21.2.101:0/3214923463 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9fcc004c60 con 0x7f9fd40a1240 2021-02-10T11:44:58.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.427 7f9fcb7fe700 1 --2- 172.21.2.101:0/3214923463 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9fb8018730 0x7f9fb801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:58.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.427 7f9fcb7fe700 1 -- 172.21.2.101:0/3214923463 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9fcc032990 con 0x7f9fd40a1240 2021-02-10T11:44:58.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.427 7f9fd2b13700 1 --2- 172.21.2.101:0/3214923463 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9fb8018730 0x7f9fb801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:58.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.427 7f9fd2b13700 1 --2- 172.21.2.101:0/3214923463 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9fb8018730 0x7f9fb801ac10 crc :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:58.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.458 7f9fd8de4700 1 -- 172.21.2.101:0/3214923463 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd40fe1c0 con 0x7f9fd40a1240 2021-02-10T11:44:58.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.460 7f9fcb7fe700 1 -- 172.21.2.101:0/3214923463 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9fcc000d90 con 0x7f9fd40a1240 2021-02-10T11:44:58.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.596 7f9fd8de4700 1 -- 172.21.2.101:0/3214923463 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"} v 0) v1 -- 0x7f9fd40fe1c0 con 0x7f9fd40a1240 2021-02-10T11:44:58.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.776 7f9fcb7fe700 1 -- 172.21.2.101:0/3214923463 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"}]=0 v131) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9fcc004c60 con 0x7f9fd40a1240 2021-02-10T11:44:58.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.813 7f9fd8de4700 1 -- 172.21.2.101:0/3214923463 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9fb8018730 msgr2=0x7f9fb801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:58.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.813 7f9fd8de4700 1 --2- 172.21.2.101:0/3214923463 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9fb8018730 0x7f9fb801ac10 crc :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:58.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.813 7f9fd8de4700 1 -- 172.21.2.101:0/3214923463 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fd40a1240 msgr2=0x7f9fd40858d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:58.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.813 7f9fd8de4700 1 --2- 172.21.2.101:0/3214923463 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fd40a1240 0x7f9fd40858d0 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 rx=0x7f9fcc007400 tx=0x7f9fcc0077a0).stop 2021-02-10T11:44:58.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.814 7f9fd8de4700 1 -- 172.21.2.101:0/3214923463 shutdown_connections 2021-02-10T11:44:58.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.814 7f9fd8de4700 1 --2- 172.21.2.101:0/3214923463 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9fd40a1240 0x7f9fd40858d0 unknown :-1 s=CLOSED pgs=2762 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:58.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.814 7f9fd8de4700 1 --2- 172.21.2.101:0/3214923463 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9fd40a0820 0x7f9fd40a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:58.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.814 7f9fd8de4700 1 --2- 172.21.2.101:0/3214923463 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9fb8018730 0x7f9fb801ac10 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:58.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.814 7f9fd8de4700 1 --2- 172.21.2.101:0/3214923463 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9fd409bcc0 0x7f9fd409c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:58.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.814 7f9fd8de4700 1 -- 172.21.2.101:0/3214923463 >> 172.21.2.101:0/3214923463 conn(0x7f9fd40bef00 msgr2=0x7f9fd409d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:58.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.814 7f9fd8de4700 1 -- 172.21.2.101:0/3214923463 shutdown_connections 2021-02-10T11:44:58.814 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:58.814 7f9fd8de4700 1 -- 172.21.2.101:0/3214923463 wait complete. 2021-02-10T11:44:58.820 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:44:59.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.233 7f70b9504700 1 Processor -- start 2021-02-10T11:44:59.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.233 7f70b9504700 1 -- start start 2021-02-10T11:44:59.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.233 7f70b9504700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f70b40d1870 0x7f70b41b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:59.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.233 7f70b9504700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70b41b0a10 0x7f70b40cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:59.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.233 7f70b9504700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b40f0b10 con 0x7f70b40d1c70 2021-02-10T11:44:59.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.233 7f70b9504700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b40e5ed0 con 0x7f70b40d1870 2021-02-10T11:44:59.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.233 7f70b9504700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b40f0650 con 0x7f70b41b0a10 2021-02-10T11:44:59.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.233 7f70b32bf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f70b40d1870 0x7f70b41b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:59.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.233 7f70b32bf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f70b40d1870 0x7f70b41b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60246/0 (socket says 172.21.2.101:60246) 2021-02-10T11:44:59.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.233 7f70b32bf700 1 -- 172.21.2.101:0/3132574179 learned_addr learned my addr 172.21.2.101:0/3132574179 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:59.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.234 7f70b32bf700 1 -- 172.21.2.101:0/3132574179 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70b41b0a10 msgr2=0x7f70b40cf330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:59.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.234 7f70b32bf700 1 --2- 172.21.2.101:0/3132574179 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70b41b0a10 0x7f70b40cf330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:59.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.234 7f70b32bf700 1 -- 172.21.2.101:0/3132574179 >> v1:172.21.2.108:6789/0 conn(0x7f70b40d1c70 legacy=0x7f70b40cf950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:44:59.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.234 7f70b32bf700 1 -- 172.21.2.101:0/3132574179 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b40f0870 con 0x7f70b40d1870 2021-02-10T11:44:59.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.234 7f70b32bf700 1 --2- 172.21.2.101:0/3132574179 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70b40d1870 0x7f70b41b03f0 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 rx=0x7f70a4006700 tx=0x7f70a4009b80).ready entity=mon.0 client_cookie=49bfb7fdec937bab server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:59.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.234 7f70b22bd700 1 -- 172.21.2.101:0/3132574179 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f70a400c0c0 con 0x7f70b40d1870 2021-02-10T11:44:59.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.234 7f70b22bd700 1 -- 172.21.2.101:0/3132574179 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a400c340 con 0x7f70b40d1870 2021-02-10T11:44:59.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.234 7f70b22bd700 1 -- 172.21.2.101:0/3132574179 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f70a400c860 con 0x7f70b40d1870 2021-02-10T11:44:59.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.234 7f70b9504700 1 -- 172.21.2.101:0/3132574179 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70b40d1870 msgr2=0x7f70b41b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:59.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.234 7f70b9504700 1 --2- 172.21.2.101:0/3132574179 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70b40d1870 0x7f70b41b03f0 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 rx=0x7f70a4006700 tx=0x7f70a4009b80).stop 2021-02-10T11:44:59.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.234 7f70b9504700 1 -- 172.21.2.101:0/3132574179 shutdown_connections 2021-02-10T11:44:59.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.234 7f70b9504700 1 --2- 172.21.2.101:0/3132574179 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70b41b0a10 0x7f70b40cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:59.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.234 7f70b9504700 1 --2- 172.21.2.101:0/3132574179 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70b40d1870 0x7f70b41b03f0 unknown :-1 s=CLOSED pgs=2763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:59.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.234 7f70b9504700 1 -- 172.21.2.101:0/3132574179 >> 172.21.2.101:0/3132574179 conn(0x7f70b40ce450 msgr2=0x7f70b40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:59.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.235 7f70b9504700 1 -- 172.21.2.101:0/3132574179 shutdown_connections 2021-02-10T11:44:59.249 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.235 7f70b9504700 1 -- 172.21.2.101:0/3132574179 wait complete. 2021-02-10T11:44:59.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.249 7f70b9504700 1 Processor -- start 2021-02-10T11:44:59.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.249 7f70b9504700 1 -- start start 2021-02-10T11:44:59.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.250 7f70b9504700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f70b4197460 0x7f70b418a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:59.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.250 7f70b9504700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70b418ab60 0x7f70b40b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:59.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.250 7f70b9504700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70b40b7df0 0x7f70b40b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:59.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.250 7f70b9504700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b41aa280 con 0x7f70b40b7df0 2021-02-10T11:44:59.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.250 7f70b9504700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b41a7940 con 0x7f70b4197460 2021-02-10T11:44:59.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.250 7f70b9504700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b41a9e20 con 0x7f70b418ab60 2021-02-10T11:44:59.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.250 7f70b3ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70b40b7df0 0x7f70b40b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:59.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.250 7f70b3ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70b40b7df0 0x7f70b40b81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60254/0 (socket says 172.21.2.101:60254) 2021-02-10T11:44:59.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.250 7f70b3ac0700 1 -- 172.21.2.101:0/525408034 learned_addr learned my addr 172.21.2.101:0/525408034 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:59.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.250 7f70b3ac0700 1 -- 172.21.2.101:0/525408034 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70b418ab60 msgr2=0x7f70b40b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:59.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.250 7f70b3ac0700 1 --2- 172.21.2.101:0/525408034 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70b418ab60 0x7f70b40b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:59.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.250 7f70b3ac0700 1 -- 172.21.2.101:0/525408034 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f70b4197460 msgr2=0x7f70b418a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:59.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.250 7f70b3ac0700 1 --2- 172.21.2.101:0/525408034 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f70b4197460 0x7f70b418a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:59.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.250 7f70b3ac0700 1 -- 172.21.2.101:0/525408034 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b41aa780 con 0x7f70b40b7df0 2021-02-10T11:44:59.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.250 7f70b3ac0700 1 --2- 172.21.2.101:0/525408034 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70b40b7df0 0x7f70b40b81f0 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 rx=0x7f70ac009940 tx=0x7f70ac00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:59.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.251 7f70abfff700 1 -- 172.21.2.101:0/525408034 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f70ac003030 con 0x7f70b40b7df0 2021-02-10T11:44:59.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.251 7f70abfff700 1 -- 172.21.2.101:0/525408034 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ac001270 con 0x7f70b40b7df0 2021-02-10T11:44:59.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.251 7f70abfff700 1 -- 172.21.2.101:0/525408034 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f70ac001790 con 0x7f70b40b7df0 2021-02-10T11:44:59.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.251 7f70b9504700 1 -- 172.21.2.101:0/525408034 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b41aa780 con 0x7f70b40b7df0 2021-02-10T11:44:59.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.251 7f70b9504700 1 -- 172.21.2.101:0/525408034 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b41aad10 con 0x7f70b40b7df0 2021-02-10T11:44:59.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.251 7f70abfff700 1 -- 172.21.2.101:0/525408034 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f70ac001450 con 0x7f70b40b7df0 2021-02-10T11:44:59.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.251 7f70abfff700 1 --2- 172.21.2.101:0/525408034 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70980186c0 0x7f709801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:59.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.252 7f70abfff700 1 -- 172.21.2.101:0/525408034 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f70ac032960 con 0x7f70b40b7df0 2021-02-10T11:44:59.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.252 7f70b32bf700 1 --2- 172.21.2.101:0/525408034 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70980186c0 0x7f709801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:59.253 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.252 7f70b32bf700 1 --2- 172.21.2.101:0/525408034 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70980186c0 0x7f709801abc0 crc :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:59.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.283 7f70b9504700 1 -- 172.21.2.101:0/525408034 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b40ce450 con 0x7f70b40b7df0 2021-02-10T11:44:59.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.285 7f70abfff700 1 -- 172.21.2.101:0/525408034 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f70ac000f40 con 0x7f70b40b7df0 2021-02-10T11:44:59.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.415 7f70b9504700 1 -- 172.21.2.101:0/525408034 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70b41a7900 con 0x7f70b40b7df0 2021-02-10T11:44:59.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.415 7f70abfff700 1 -- 172.21.2.101:0/525408034 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+2105 (secure 0 0 0) 0x7f70ac01f700 con 0x7f70b40b7df0 2021-02-10T11:44:59.415 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:44:59.415 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":131,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":12135,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/3467749863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3467749863},{"type":"v1","addr":"172.21.2.101:6835","nonce":3467749863}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":128}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:44:56.768458","modified":"2021-02-10 11:44:58.773048","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8242":{"gid":8242,"name":"b","rank":0,"incarnation":129,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/485291097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":485291097},{"type":"v1","addr":"172.21.2.108:6833","nonce":485291097}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2021-02-10T11:44:59.415 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 131 2021-02-10T11:44:59.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.417 7f70b9504700 1 -- 172.21.2.101:0/525408034 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70980186c0 msgr2=0x7f709801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:59.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.417 7f70b9504700 1 --2- 172.21.2.101:0/525408034 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70980186c0 0x7f709801abc0 crc :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:59.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.417 7f70b9504700 1 -- 172.21.2.101:0/525408034 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70b40b7df0 msgr2=0x7f70b40b81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:59.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.417 7f70b9504700 1 --2- 172.21.2.101:0/525408034 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70b40b7df0 0x7f70b40b81f0 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 rx=0x7f70ac009940 tx=0x7f70ac00bcf0).stop 2021-02-10T11:44:59.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.417 7f70b9504700 1 -- 172.21.2.101:0/525408034 shutdown_connections 2021-02-10T11:44:59.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.417 7f70b9504700 1 --2- 172.21.2.101:0/525408034 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f70980186c0 0x7f709801abc0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:59.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.417 7f70b9504700 1 --2- 172.21.2.101:0/525408034 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f70b418ab60 0x7f70b40b77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:59.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.417 7f70b9504700 1 --2- 172.21.2.101:0/525408034 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f70b40b7df0 0x7f70b40b81f0 unknown :-1 s=CLOSED pgs=2764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:59.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.417 7f70b9504700 1 --2- 172.21.2.101:0/525408034 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f70b4197460 0x7f70b418a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:44:59.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.417 7f70b9504700 1 -- 172.21.2.101:0/525408034 >> 172.21.2.101:0/525408034 conn(0x7f70b40efc70 msgr2=0x7f70b40ef410 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:59.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.417 7f70b9504700 1 -- 172.21.2.101:0/525408034 shutdown_connections 2021-02-10T11:44:59.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.418 7f70b9504700 1 -- 172.21.2.101:0/525408034 wait complete. 2021-02-10T11:44:59.424 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:44:59.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.838 7fa5aa633700 1 Processor -- start 2021-02-10T11:44:59.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.838 7fa5aa633700 1 -- start start 2021-02-10T11:44:59.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.839 7fa5aa633700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5a40d0340 0x7fa5a40ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:59.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.839 7fa5aa633700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a40f0b10 con 0x7fa5a40d1870 2021-02-10T11:44:59.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.839 7fa5aa633700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a41b0be0 con 0x7fa5a40d1c70 2021-02-10T11:44:59.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.839 7fa5aa633700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a40e5ed0 con 0x7fa5a40d0340 2021-02-10T11:44:59.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.839 7fa5a3fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa5a40d1870 0x7fa5a41b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60752/0 (socket says 172.21.2.101:60752) 2021-02-10T11:44:59.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.839 7fa5a3fff700 1 -- 172.21.2.101:0/2382144680 learned_addr learned my addr 172.21.2.101:0/2382144680 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:44:59.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.839 7fa5a37fe700 1 --2- 172.21.2.101:0/2382144680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5a40d0340 0x7fa5a40ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:59.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.839 7fa5a2ffd700 1 -- 172.21.2.101:0/2382144680 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466498057 0 0) 0x7fa5940009b0 con 0x7fa5a40d1870 2021-02-10T11:44:59.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.839 7fa5a2ffd700 1 -- 172.21.2.101:0/2382144680 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa58c002fe0 con 0x7fa5a40d1870 2021-02-10T11:44:59.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.839 7fa5a37fe700 1 -- 172.21.2.101:0/2382144680 >> v1:172.21.2.108:6789/0 conn(0x7fa5a40d1c70 legacy=0x7fa5a40efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:59.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.839 7fa5a37fe700 1 -- 172.21.2.101:0/2382144680 >> v1:172.21.2.101:6789/0 conn(0x7fa5a40d1870 legacy=0x7fa5a41b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:59.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.839 7fa5a2ffd700 1 -- 172.21.2.101:0/2382144680 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4272348084 0 0) 0x7fa594001010 con 0x7fa5a40d1870 2021-02-10T11:44:59.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.840 7fa5a37fe700 1 -- 172.21.2.101:0/2382144680 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a40f0450 con 0x7fa5a40d0340 2021-02-10T11:44:59.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.840 7fa5a37fe700 1 --2- 172.21.2.101:0/2382144680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5a40d0340 0x7fa5a40ef640 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fa598006650 tx=0x7fa598009a80).ready entity=mon.2 client_cookie=f91cad7f2f163705 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:59.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.840 7fa5a2ffd700 1 -- 172.21.2.101:0/2382144680 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa59800c0c0 con 0x7fa5a40d0340 2021-02-10T11:44:59.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.840 7fa5a2ffd700 1 -- 172.21.2.101:0/2382144680 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59800c340 con 0x7fa5a40d0340 2021-02-10T11:44:59.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.840 7fa5a2ffd700 1 -- 172.21.2.101:0/2382144680 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa59800c860 con 0x7fa5a40d0340 2021-02-10T11:44:59.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.840 7fa5aa633700 1 -- 172.21.2.101:0/2382144680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5a40d0340 msgr2=0x7fa5a40ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:59.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.840 7fa5aa633700 1 --2- 172.21.2.101:0/2382144680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5a40d0340 0x7fa5a40ef640 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fa598006650 tx=0x7fa598009a80).stop 2021-02-10T11:44:59.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.840 7fa5aa633700 1 -- 172.21.2.101:0/2382144680 shutdown_connections 2021-02-10T11:44:59.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.840 7fa5aa633700 1 --2- 172.21.2.101:0/2382144680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5a40d0340 0x7fa5a40ef640 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:59.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.840 7fa5aa633700 1 -- 172.21.2.101:0/2382144680 >> 172.21.2.101:0/2382144680 conn(0x7fa5a40ce450 msgr2=0x7fa5a40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:44:59.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.840 7fa5aa633700 1 -- 172.21.2.101:0/2382144680 shutdown_connections 2021-02-10T11:44:59.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.841 7fa5aa633700 1 -- 172.21.2.101:0/2382144680 wait complete. 2021-02-10T11:44:59.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.855 7fa5aa633700 1 Processor -- start 2021-02-10T11:44:59.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.855 7fa5aa633700 1 -- start start 2021-02-10T11:44:59.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.855 7fa5aa633700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa5a41952e0 0x7fa5a418ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:59.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.855 7fa5aa633700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5a41b8bc0 0x7fa5a41ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:59.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.855 7fa5aa633700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5a41acae0 0x7fa5a4194240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:59.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.855 7fa5aa633700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a41b8640 con 0x7fa5a41acae0 2021-02-10T11:44:59.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.855 7fa5aa633700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a419e8b0 con 0x7fa5a41952e0 2021-02-10T11:44:59.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.855 7fa5aa633700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a419ea90 con 0x7fa5a41b8bc0 2021-02-10T11:44:59.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.856 7fa5a8c50700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5a41acae0 0x7fa5a4194240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:59.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.856 7fa5a8c50700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5a41acae0 0x7fa5a4194240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60268/0 (socket says 172.21.2.101:60268) 2021-02-10T11:44:59.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.856 7fa5a8c50700 1 -- 172.21.2.101:0/2533533270 learned_addr learned my addr 172.21.2.101:0/2533533270 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:44:59.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.856 7fa5a3fff700 1 --2- 172.21.2.101:0/2533533270 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa5a41952e0 0x7fa5a418ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:59.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.856 7fa5a37fe700 1 --2- 172.21.2.101:0/2533533270 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5a41b8bc0 0x7fa5a41ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:59.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.856 7fa5a8c50700 1 -- 172.21.2.101:0/2533533270 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5a41b8bc0 msgr2=0x7fa5a41ac4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:59.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.856 7fa5a8c50700 1 --2- 172.21.2.101:0/2533533270 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5a41b8bc0 0x7fa5a41ac4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:59.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.856 7fa5a8c50700 1 -- 172.21.2.101:0/2533533270 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa5a41952e0 msgr2=0x7fa5a418ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:44:59.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.856 7fa5a8c50700 1 --2- 172.21.2.101:0/2533533270 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa5a41952e0 0x7fa5a418ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:44:59.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.856 7fa5a8c50700 1 -- 172.21.2.101:0/2533533270 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a41acee0 con 0x7fa5a41acae0 2021-02-10T11:44:59.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.856 7fa5a8c50700 1 --2- 172.21.2.101:0/2533533270 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5a41acae0 0x7fa5a4194240 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 rx=0x7fa59c00b8e0 tx=0x7fa59c00bc80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:59.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.856 7fa5a0ff9700 1 -- 172.21.2.101:0/2533533270 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa59c000ce0 con 0x7fa5a41acae0 2021-02-10T11:44:59.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.856 7fa5a0ff9700 1 -- 172.21.2.101:0/2533533270 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59c00bdc0 con 0x7fa5a41acae0 2021-02-10T11:44:59.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.856 7fa5a0ff9700 1 -- 172.21.2.101:0/2533533270 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa59c00c570 con 0x7fa5a41acae0 2021-02-10T11:44:59.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.856 7fa5aa633700 1 -- 172.21.2.101:0/2533533270 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a41acee0 con 0x7fa5a41acae0 2021-02-10T11:44:59.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.857 7fa5aa633700 1 -- 172.21.2.101:0/2533533270 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a41acee0 con 0x7fa5a41acae0 2021-02-10T11:44:59.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.857 7fa5a0ff9700 1 -- 172.21.2.101:0/2533533270 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa59c001220 con 0x7fa5a41acae0 2021-02-10T11:44:59.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.857 7fa5a0ff9700 1 --2- 172.21.2.101:0/2533533270 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5840186c0 0x7fa58401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:44:59.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.857 7fa5a0ff9700 1 -- 172.21.2.101:0/2533533270 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa59c032b60 con 0x7fa5a41acae0 2021-02-10T11:44:59.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.857 7fa5a3fff700 1 --2- 172.21.2.101:0/2533533270 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5840186c0 0x7fa58401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:44:59.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.857 7fa5a3fff700 1 --2- 172.21.2.101:0/2533533270 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5840186c0 0x7fa58401abc0 crc :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:44:59.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.889 7fa5aa633700 1 -- 172.21.2.101:0/2533533270 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a40ebc60 con 0x7fa5a41acae0 2021-02-10T11:44:59.890 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:44:59.891 7fa5a0ff9700 1 -- 172.21.2.101:0/2533533270 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa59c000f90 con 0x7fa5a41acae0 2021-02-10T11:45:00.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.014 7fa5aa633700 1 -- 172.21.2.101:0/2533533270 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5a40f0b10 con 0x7fa5a41acae0 2021-02-10T11:45:00.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.015 7fa5a0ff9700 1 -- 172.21.2.101:0/2533533270 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+14007 (secure 0 0 0) 0x7fa59c00bdc0 con 0x7fa5a41acae0 2021-02-10T11:45:00.014 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:45:00.014 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":156,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:44:57.770382","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:44:54.736504","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":33,"pool_name":"cephfs_data","create_time":"2021-02-10 11:44:55.811077","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:45:00.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.016 7fa5aa633700 1 -- 172.21.2.101:0/2533533270 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5840186c0 msgr2=0x7fa58401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:00.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.016 7fa5aa633700 1 --2- 172.21.2.101:0/2533533270 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5840186c0 0x7fa58401abc0 crc :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:00.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.016 7fa5aa633700 1 -- 172.21.2.101:0/2533533270 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5a41acae0 msgr2=0x7fa5a4194240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:00.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.016 7fa5aa633700 1 --2- 172.21.2.101:0/2533533270 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5a41acae0 0x7fa5a4194240 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 rx=0x7fa59c00b8e0 tx=0x7fa59c00bc80).stop 2021-02-10T11:45:00.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.017 7fa5aa633700 1 -- 172.21.2.101:0/2533533270 shutdown_connections 2021-02-10T11:45:00.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.017 7fa5aa633700 1 --2- 172.21.2.101:0/2533533270 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5840186c0 0x7fa58401abc0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:00.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.017 7fa5aa633700 1 --2- 172.21.2.101:0/2533533270 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5a41b8bc0 0x7fa5a41ac4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:00.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.017 7fa5aa633700 1 --2- 172.21.2.101:0/2533533270 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5a41acae0 0x7fa5a4194240 unknown :-1 s=CLOSED pgs=2766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:00.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.017 7fa5aa633700 1 --2- 172.21.2.101:0/2533533270 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa5a41952e0 0x7fa5a418ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:00.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.017 7fa5aa633700 1 -- 172.21.2.101:0/2533533270 >> 172.21.2.101:0/2533533270 conn(0x7fa5a41ac040 msgr2=0x7fa5a4094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:00.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.017 7fa5aa633700 1 -- 172.21.2.101:0/2533533270 shutdown_connections 2021-02-10T11:45:00.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.017 7fa5aa633700 1 -- 172.21.2.101:0/2533533270 wait complete. 2021-02-10T11:45:00.025 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:45:00.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.450 7f160e815700 1 Processor -- start 2021-02-10T11:45:00.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.450 7f160e815700 1 -- start start 2021-02-10T11:45:00.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.450 7f160e815700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f16080d1880 0x7f16081b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:00.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.450 7f160e815700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f16081b0af0 0x7f1608108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:00.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.450 7f160e815700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16080fe1d0 con 0x7f16080d1c80 2021-02-10T11:45:00.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.450 7f160e815700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1608109a70 con 0x7f16081b0af0 2021-02-10T11:45:00.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.450 7f160e815700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1608109760 con 0x7f16080d1880 2021-02-10T11:45:00.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.450 7f16077fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f16081b0af0 0x7f1608108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:00.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.451 7f16077fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f16081b0af0 0x7f1608108950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60272/0 (socket says 172.21.2.101:60272) 2021-02-10T11:45:00.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.451 7f16077fe700 1 -- 172.21.2.101:0/1315635231 learned_addr learned my addr 172.21.2.101:0/1315635231 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:00.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.451 7f16077fe700 1 -- 172.21.2.101:0/1315635231 >> v2:172.21.2.108:3300/0 conn(0x7f16080d1880 msgr2=0x7f16081b04d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:45:00.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.451 7f16077fe700 1 --2- 172.21.2.101:0/1315635231 >> v2:172.21.2.108:3300/0 conn(0x7f16080d1880 0x7f16081b04d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:00.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.451 7f16077fe700 1 -- 172.21.2.101:0/1315635231 >> v1:172.21.2.101:6789/0 conn(0x7f16080d1c80 legacy=0x7f1608108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:00.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.451 7f16077fe700 1 -- 172.21.2.101:0/1315635231 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1608088070 con 0x7f16081b0af0 2021-02-10T11:45:00.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.451 7f16077fe700 1 --2- 172.21.2.101:0/1315635231 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16081b0af0 0x7f1608108950 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 rx=0x7f15fc006700 tx=0x7f15fc009b80).ready entity=mon.0 client_cookie=80ddcc4fbfac0d35 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:00.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.451 7f1606ffd700 1 -- 172.21.2.101:0/1315635231 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f15fc00c0c0 con 0x7f16081b0af0 2021-02-10T11:45:00.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.451 7f1606ffd700 1 -- 172.21.2.101:0/1315635231 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15fc00c340 con 0x7f16081b0af0 2021-02-10T11:45:00.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.451 7f1606ffd700 1 -- 172.21.2.101:0/1315635231 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f15fc00c860 con 0x7f16081b0af0 2021-02-10T11:45:00.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.451 7f160e815700 1 -- 172.21.2.101:0/1315635231 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16081b0af0 msgr2=0x7f1608108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:00.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.451 7f160e815700 1 --2- 172.21.2.101:0/1315635231 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16081b0af0 0x7f1608108950 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 rx=0x7f15fc006700 tx=0x7f15fc009b80).stop 2021-02-10T11:45:00.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.452 7f160e815700 1 -- 172.21.2.101:0/1315635231 shutdown_connections 2021-02-10T11:45:00.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.452 7f160e815700 1 --2- 172.21.2.101:0/1315635231 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f16081b0af0 0x7f1608108950 unknown :-1 s=CLOSED pgs=2768 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:00.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.452 7f160e815700 1 --2- 172.21.2.101:0/1315635231 >> v2:172.21.2.108:3300/0 conn(0x7f16080d1880 0x7f16081b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:00.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.452 7f160e815700 1 -- 172.21.2.101:0/1315635231 >> 172.21.2.101:0/1315635231 conn(0x7f16080ce430 msgr2=0x7f16080d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:00.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.452 7f160e815700 1 -- 172.21.2.101:0/1315635231 shutdown_connections 2021-02-10T11:45:00.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.452 7f160e815700 1 -- 172.21.2.101:0/1315635231 wait complete. 2021-02-10T11:45:00.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.467 7f160e815700 1 Processor -- start 2021-02-10T11:45:00.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.467 7f160e815700 1 -- start start 2021-02-10T11:45:00.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.467 7f160e815700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1608194890 0x7f1608090d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:00.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.467 7f160e815700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1608194c90 0x7f1608091390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:00.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.467 7f160e815700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16081a2bc0 0x7f16081a2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:00.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.467 7f160e815700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16081a4710 con 0x7f1608194c90 2021-02-10T11:45:00.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.467 7f160e815700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16081a4930 con 0x7f1608194890 2021-02-10T11:45:00.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.467 7f160e815700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16081a35e0 con 0x7f16081a2bc0 2021-02-10T11:45:00.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.467 7f16077fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1608194c90 0x7f1608091390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:00.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.467 7f16077fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1608194c90 0x7f1608091390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60278/0 (socket says 172.21.2.101:60278) 2021-02-10T11:45:00.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.467 7f16077fe700 1 -- 172.21.2.101:0/1010353554 learned_addr learned my addr 172.21.2.101:0/1010353554 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:00.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.468 7f16077fe700 1 -- 172.21.2.101:0/1010353554 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16081a2bc0 msgr2=0x7f16081a2fc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:00.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.468 7f16077fe700 1 --2- 172.21.2.101:0/1010353554 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16081a2bc0 0x7f16081a2fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:00.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.468 7f16077fe700 1 -- 172.21.2.101:0/1010353554 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1608194890 msgr2=0x7f1608090d70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:00.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.468 7f16077fe700 1 --2- 172.21.2.101:0/1010353554 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1608194890 0x7f1608090d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:00.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.468 7f16077fe700 1 -- 172.21.2.101:0/1010353554 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16081a3800 con 0x7f1608194c90 2021-02-10T11:45:00.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.468 7f16077fe700 1 --2- 172.21.2.101:0/1010353554 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1608194c90 0x7f1608091390 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 rx=0x7f15fc00b5e0 tx=0x7f15fc00b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:00.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.468 7f1604ff9700 1 -- 172.21.2.101:0/1010353554 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f15fc00bd70 con 0x7f1608194c90 2021-02-10T11:45:00.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.468 7f1604ff9700 1 -- 172.21.2.101:0/1010353554 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15fc002030 con 0x7f1608194c90 2021-02-10T11:45:00.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.468 7f1604ff9700 1 -- 172.21.2.101:0/1010353554 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f15fc002550 con 0x7f1608194c90 2021-02-10T11:45:00.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.468 7f160e815700 1 -- 172.21.2.101:0/1010353554 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16081a3800 con 0x7f1608194c90 2021-02-10T11:45:00.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.468 7f160e815700 1 -- 172.21.2.101:0/1010353554 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16081a3d90 con 0x7f1608194c90 2021-02-10T11:45:00.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.469 7f1604ff9700 1 -- 172.21.2.101:0/1010353554 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f15fc000c90 con 0x7f1608194c90 2021-02-10T11:45:00.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.469 7f1604ff9700 1 --2- 172.21.2.101:0/1010353554 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15e80186c0 0x7f15e801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:00.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.469 7f1604ff9700 1 -- 172.21.2.101:0/1010353554 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f15fc032420 con 0x7f1608194c90 2021-02-10T11:45:00.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.469 7f1607fff700 1 --2- 172.21.2.101:0/1010353554 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15e80186c0 0x7f15e801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:00.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.469 7f1607fff700 1 --2- 172.21.2.101:0/1010353554 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15e80186c0 0x7f15e801abc0 crc :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:00.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.500 7f160e815700 1 -- 172.21.2.101:0/1010353554 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16080829a0 con 0x7f1608194c90 2021-02-10T11:45:00.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.502 7f1604ff9700 1 -- 172.21.2.101:0/1010353554 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f15fc01dd90 con 0x7f1608194c90 2021-02-10T11:45:00.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.632 7f160e815700 1 -- 172.21.2.101:0/1010353554 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f16080829a0 con 0x7f1608194c90 2021-02-10T11:45:00.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.636 7f1604ff9700 1 -- 172.21.2.101:0/1010353554 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v87) v1 ==== 184+0+0 (secure 0 0 0) 0x7f15fc000c90 con 0x7f1608194c90 2021-02-10T11:45:00.638 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.0 2021-02-10T11:45:00.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.641 7f160e815700 1 -- 172.21.2.101:0/1010353554 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15e80186c0 msgr2=0x7f15e801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:00.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.641 7f160e815700 1 --2- 172.21.2.101:0/1010353554 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15e80186c0 0x7f15e801abc0 crc :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:00.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.641 7f160e815700 1 -- 172.21.2.101:0/1010353554 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1608194c90 msgr2=0x7f1608091390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:00.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.641 7f160e815700 1 --2- 172.21.2.101:0/1010353554 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1608194c90 0x7f1608091390 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 rx=0x7f15fc00b5e0 tx=0x7f15fc00b980).stop 2021-02-10T11:45:00.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.641 7f160e815700 1 -- 172.21.2.101:0/1010353554 shutdown_connections 2021-02-10T11:45:00.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.641 7f160e815700 1 --2- 172.21.2.101:0/1010353554 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15e80186c0 0x7f15e801abc0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:00.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.641 7f160e815700 1 --2- 172.21.2.101:0/1010353554 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f16081a2bc0 0x7f16081a2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:00.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.641 7f160e815700 1 --2- 172.21.2.101:0/1010353554 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1608194c90 0x7f1608091390 unknown :-1 s=CLOSED pgs=2769 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:00.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.641 7f160e815700 1 --2- 172.21.2.101:0/1010353554 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1608194890 0x7f1608090d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:00.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.641 7f160e815700 1 -- 172.21.2.101:0/1010353554 >> 172.21.2.101:0/1010353554 conn(0x7f160819f880 msgr2=0x7f16080990d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:00.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.641 7f160e815700 1 -- 172.21.2.101:0/1010353554 shutdown_connections 2021-02-10T11:45:00.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:00.642 7f160e815700 1 -- 172.21.2.101:0/1010353554 wait complete. 2021-02-10T11:45:00.648 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:45:01.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.073 7ff517846700 1 Processor -- start 2021-02-10T11:45:01.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.073 7ff517846700 1 -- start start 2021-02-10T11:45:01.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.074 7ff517846700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff5100d1880 0x7ff5101b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:01.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.074 7ff517846700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5100fe1d0 con 0x7ff5100cff80 2021-02-10T11:45:01.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.074 7ff517846700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff510109a70 con 0x7ff5101b0af0 2021-02-10T11:45:01.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.074 7ff517846700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff510088070 con 0x7ff5100d1880 2021-02-10T11:45:01.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.074 7ff515e63700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff5100cff80 0x7ff510109140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60780/0 (socket says 172.21.2.101:60780) 2021-02-10T11:45:01.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.074 7ff515e63700 1 -- 172.21.2.101:0/3965239342 learned_addr learned my addr 172.21.2.101:0/3965239342 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:01.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.074 7ff515662700 1 --2- 172.21.2.101:0/3965239342 >> v2:172.21.2.108:3300/0 conn(0x7ff5100d1880 0x7ff5101b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:01.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.075 7ff503fff700 1 -- 172.21.2.101:0/3965239342 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912156860 0 0) 0x7ff508001480 con 0x7ff5100cff80 2021-02-10T11:45:01.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.075 7ff515662700 1 -- 172.21.2.101:0/3965239342 >> v1:172.21.2.108:6789/0 conn(0x7ff5101b0af0 legacy=0x7ff510108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:01.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.075 7ff515662700 1 -- 172.21.2.101:0/3965239342 >> v1:172.21.2.101:6789/0 conn(0x7ff5100cff80 legacy=0x7ff510109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:01.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.075 7ff515662700 1 -- 172.21.2.101:0/3965239342 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff510088260 con 0x7ff5100d1880 2021-02-10T11:45:01.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.075 7ff515662700 1 --2- 172.21.2.101:0/3965239342 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5100d1880 0x7ff5101b04d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7ff504006650 tx=0x7ff504009a00).ready entity=mon.1 client_cookie=ed002b4398334e0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:01.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.075 7ff503fff700 1 -- 172.21.2.101:0/3965239342 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff5040009b0 con 0x7ff5100d1880 2021-02-10T11:45:01.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.075 7ff503fff700 1 -- 172.21.2.101:0/3965239342 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50400c0c0 con 0x7ff5100d1880 2021-02-10T11:45:01.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.075 7ff503fff700 1 -- 172.21.2.101:0/3965239342 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff50400c5a0 con 0x7ff5100d1880 2021-02-10T11:45:01.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.075 7ff517846700 1 -- 172.21.2.101:0/3965239342 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5100d1880 msgr2=0x7ff5101b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:01.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.075 7ff517846700 1 --2- 172.21.2.101:0/3965239342 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5100d1880 0x7ff5101b04d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7ff504006650 tx=0x7ff504009a00).stop 2021-02-10T11:45:01.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.076 7ff517846700 1 -- 172.21.2.101:0/3965239342 shutdown_connections 2021-02-10T11:45:01.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.076 7ff517846700 1 --2- 172.21.2.101:0/3965239342 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5100d1880 0x7ff5101b04d0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:01.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.076 7ff517846700 1 -- 172.21.2.101:0/3965239342 >> 172.21.2.101:0/3965239342 conn(0x7ff5100ce430 msgr2=0x7ff5100d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:01.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.076 7ff517846700 1 -- 172.21.2.101:0/3965239342 shutdown_connections 2021-02-10T11:45:01.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.076 7ff517846700 1 -- 172.21.2.101:0/3965239342 wait complete. 2021-02-10T11:45:01.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.091 7ff517846700 1 Processor -- start 2021-02-10T11:45:01.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.091 7ff517846700 1 -- start start 2021-02-10T11:45:01.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.091 7ff517846700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff510086810 0x7ff51019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:01.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.091 7ff517846700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5100b04d0 0x7ff510097450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:01.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.091 7ff517846700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5100b08d0 0x7ff510097a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:01.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.091 7ff517846700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5101a02c0 con 0x7ff510086810 2021-02-10T11:45:01.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.091 7ff517846700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff510098090 con 0x7ff5100b08d0 2021-02-10T11:45:01.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.091 7ff517846700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100982b0 con 0x7ff5100b04d0 2021-02-10T11:45:01.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.091 7ff515662700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff510086810 0x7ff51019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:01.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.091 7ff515662700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff510086810 0x7ff51019fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60288/0 (socket says 172.21.2.101:60288) 2021-02-10T11:45:01.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.091 7ff515662700 1 -- 172.21.2.101:0/3275057917 learned_addr learned my addr 172.21.2.101:0/3275057917 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:01.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.091 7ff515e63700 1 --2- 172.21.2.101:0/3275057917 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5100b08d0 0x7ff510097a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:01.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.092 7ff514e61700 1 --2- 172.21.2.101:0/3275057917 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5100b04d0 0x7ff510097450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:01.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.092 7ff515662700 1 -- 172.21.2.101:0/3275057917 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5100b04d0 msgr2=0x7ff510097450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:01.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.092 7ff515662700 1 --2- 172.21.2.101:0/3275057917 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5100b04d0 0x7ff510097450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:01.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.092 7ff515662700 1 -- 172.21.2.101:0/3275057917 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5100b08d0 msgr2=0x7ff510097a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:01.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.092 7ff515662700 1 --2- 172.21.2.101:0/3275057917 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5100b08d0 0x7ff510097a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:01.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.092 7ff515662700 1 -- 172.21.2.101:0/3275057917 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5100984d0 con 0x7ff510086810 2021-02-10T11:45:01.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.092 7ff514e61700 1 --2- 172.21.2.101:0/3275057917 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5100b04d0 0x7ff510097450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:45:01.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.092 7ff515662700 1 --2- 172.21.2.101:0/3275057917 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff510086810 0x7ff51019fca0 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 rx=0x7ff50400b5e0 tx=0x7ff50400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:01.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.092 7ff501ffb700 1 -- 172.21.2.101:0/3275057917 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff5040009b0 con 0x7ff510086810 2021-02-10T11:45:01.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.092 7ff501ffb700 1 -- 172.21.2.101:0/3275057917 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5040063b0 con 0x7ff510086810 2021-02-10T11:45:01.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.092 7ff501ffb700 1 -- 172.21.2.101:0/3275057917 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff504002220 con 0x7ff510086810 2021-02-10T11:45:01.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.092 7ff517846700 1 -- 172.21.2.101:0/3275057917 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5100984d0 con 0x7ff510086810 2021-02-10T11:45:01.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.092 7ff517846700 1 -- 172.21.2.101:0/3275057917 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff510098b80 con 0x7ff510086810 2021-02-10T11:45:01.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.093 7ff501ffb700 1 -- 172.21.2.101:0/3275057917 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff504009fb0 con 0x7ff510086810 2021-02-10T11:45:01.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.093 7ff501ffb700 1 --2- 172.21.2.101:0/3275057917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4f00186c0 0x7ff4f001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:01.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.093 7ff501ffb700 1 -- 172.21.2.101:0/3275057917 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff504032480 con 0x7ff510086810 2021-02-10T11:45:01.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.093 7ff514e61700 1 --2- 172.21.2.101:0/3275057917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4f00186c0 0x7ff4f001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:01.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.093 7ff514e61700 1 --2- 172.21.2.101:0/3275057917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4f00186c0 0x7ff4f001abc0 crc :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:01.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.124 7ff517846700 1 -- 172.21.2.101:0/3275057917 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5100fe1d0 con 0x7ff510086810 2021-02-10T11:45:01.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.126 7ff501ffb700 1 -- 172.21.2.101:0/3275057917 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff5040090c0 con 0x7ff510086810 2021-02-10T11:45:01.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.255 7ff517846700 1 -- 172.21.2.101:0/3275057917 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ff5100fe1d0 con 0x7ff510086810 2021-02-10T11:45:01.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.259 7ff501ffb700 1 -- 172.21.2.101:0/3275057917 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v88) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff50401f530 con 0x7ff510086810 2021-02-10T11:45:01.261 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.1 2021-02-10T11:45:01.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.264 7ff517846700 1 -- 172.21.2.101:0/3275057917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4f00186c0 msgr2=0x7ff4f001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:01.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.264 7ff517846700 1 --2- 172.21.2.101:0/3275057917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4f00186c0 0x7ff4f001abc0 crc :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:01.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.264 7ff517846700 1 -- 172.21.2.101:0/3275057917 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff510086810 msgr2=0x7ff51019fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:01.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.264 7ff517846700 1 --2- 172.21.2.101:0/3275057917 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff510086810 0x7ff51019fca0 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 rx=0x7ff50400b5e0 tx=0x7ff50400b980).stop 2021-02-10T11:45:01.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.264 7ff517846700 1 -- 172.21.2.101:0/3275057917 shutdown_connections 2021-02-10T11:45:01.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.264 7ff517846700 1 --2- 172.21.2.101:0/3275057917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff4f00186c0 0x7ff4f001abc0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:01.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.264 7ff517846700 1 --2- 172.21.2.101:0/3275057917 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5100b04d0 0x7ff510097450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:01.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.264 7ff517846700 1 --2- 172.21.2.101:0/3275057917 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5100b08d0 0x7ff510097a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:01.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.264 7ff517846700 1 --2- 172.21.2.101:0/3275057917 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff510086810 0x7ff51019fca0 unknown :-1 s=CLOSED pgs=2771 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:01.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.264 7ff517846700 1 -- 172.21.2.101:0/3275057917 >> 172.21.2.101:0/3275057917 conn(0x7ff51019f880 msgr2=0x7ff51008f890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:01.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.265 7ff517846700 1 -- 172.21.2.101:0/3275057917 shutdown_connections 2021-02-10T11:45:01.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.265 7ff517846700 1 -- 172.21.2.101:0/3275057917 wait complete. 2021-02-10T11:45:01.272 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:45:01.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.687 7f24f67fa700 1 Processor -- start 2021-02-10T11:45:01.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.688 7f24f67fa700 1 -- start start 2021-02-10T11:45:01.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.688 7f24f67fa700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f24f00d1880 0x7f24f01b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:01.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.688 7f24f67fa700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f24f00cff80 0x7f24f0109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:01.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.688 7f24f67fa700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f00fe1d0 con 0x7f24f01b0af0 2021-02-10T11:45:01.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.688 7f24f67fa700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f0109ae0 con 0x7f24f00d1880 2021-02-10T11:45:01.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.688 7f24f67fa700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f0109760 con 0x7f24f00cff80 2021-02-10T11:45:01.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.688 7f24effff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f24f00d1880 0x7f24f01b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:01.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.688 7f24ef7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f24f01b0af0 0x7f24f0108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60790/0 (socket says 172.21.2.101:60790) 2021-02-10T11:45:01.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.688 7f24effff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f24f00d1880 0x7f24f01b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60296/0 (socket says 172.21.2.101:60296) 2021-02-10T11:45:01.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.688 7f24ef7fe700 1 -- 172.21.2.101:0/2335522226 learned_addr learned my addr 172.21.2.101:0/2335522226 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:01.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.689 7f24effff700 1 -- 172.21.2.101:0/2335522226 >> v2:172.21.2.108:3300/0 conn(0x7f24f00cff80 msgr2=0x7f24f0109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:01.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.689 7f24f4e17700 1 --2- 172.21.2.101:0/2335522226 >> v2:172.21.2.108:3300/0 conn(0x7f24f00cff80 0x7f24f0109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:01.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.689 7f24effff700 1 --2- 172.21.2.101:0/2335522226 >> v2:172.21.2.108:3300/0 conn(0x7f24f00cff80 0x7f24f0109140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:01.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.689 7f24effff700 1 -- 172.21.2.101:0/2335522226 >> v1:172.21.2.101:6789/0 conn(0x7f24f01b0af0 legacy=0x7f24f0108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:01.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.689 7f24effff700 1 -- 172.21.2.101:0/2335522226 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f0088070 con 0x7f24f00d1880 2021-02-10T11:45:01.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.689 7f24f4e17700 1 --2- 172.21.2.101:0/2335522226 >> v2:172.21.2.108:3300/0 conn(0x7f24f00cff80 0x7f24f0109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:01.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.689 7f24effff700 1 --2- 172.21.2.101:0/2335522226 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24f00d1880 0x7f24f01b04d0 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 rx=0x7f24e0006650 tx=0x7f24e0009a80).ready entity=mon.0 client_cookie=41e89c26952620d5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:01.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.689 7f24eeffd700 1 -- 172.21.2.101:0/2335522226 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f24e00009b0 con 0x7f24f00d1880 2021-02-10T11:45:01.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.689 7f24eeffd700 1 -- 172.21.2.101:0/2335522226 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e000c0c0 con 0x7f24f00d1880 2021-02-10T11:45:01.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.689 7f24eeffd700 1 -- 172.21.2.101:0/2335522226 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f24e000c5e0 con 0x7f24f00d1880 2021-02-10T11:45:01.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.689 7f24f67fa700 1 -- 172.21.2.101:0/2335522226 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24f00d1880 msgr2=0x7f24f01b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:01.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.689 7f24f67fa700 1 --2- 172.21.2.101:0/2335522226 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24f00d1880 0x7f24f01b04d0 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 rx=0x7f24e0006650 tx=0x7f24e0009a80).stop 2021-02-10T11:45:01.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.690 7f24f67fa700 1 -- 172.21.2.101:0/2335522226 shutdown_connections 2021-02-10T11:45:01.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.690 7f24f67fa700 1 --2- 172.21.2.101:0/2335522226 >> v2:172.21.2.108:3300/0 conn(0x7f24f00cff80 0x7f24f0109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:01.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.690 7f24f67fa700 1 --2- 172.21.2.101:0/2335522226 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24f00d1880 0x7f24f01b04d0 unknown :-1 s=CLOSED pgs=2773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:01.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.690 7f24f67fa700 1 -- 172.21.2.101:0/2335522226 >> 172.21.2.101:0/2335522226 conn(0x7f24f00ce430 msgr2=0x7f24f00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:01.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.690 7f24f67fa700 1 -- 172.21.2.101:0/2335522226 shutdown_connections 2021-02-10T11:45:01.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.690 7f24f67fa700 1 -- 172.21.2.101:0/2335522226 wait complete. 2021-02-10T11:45:01.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.705 7f24f67fa700 1 Processor -- start 2021-02-10T11:45:01.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.705 7f24f67fa700 1 -- start start 2021-02-10T11:45:01.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.705 7f24f67fa700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24f01b9470 0x7f24f01ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:01.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.705 7f24f67fa700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24f01aecf0 0x7f24f00957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:01.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.705 7f24f67fa700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24f01af0f0 0x7f24f0095e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:01.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.705 7f24f67fa700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f0096d70 con 0x7f24f01af0f0 2021-02-10T11:45:01.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.705 7f24f67fa700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f00a7780 con 0x7f24f01aecf0 2021-02-10T11:45:01.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.705 7f24f67fa700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f01b9870 con 0x7f24f01b9470 2021-02-10T11:45:01.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.705 7f24f4e17700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24f01af0f0 0x7f24f0095e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:01.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.705 7f24f4e17700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24f01af0f0 0x7f24f0095e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60302/0 (socket says 172.21.2.101:60302) 2021-02-10T11:45:01.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.705 7f24f4e17700 1 -- 172.21.2.101:0/1200831121 learned_addr learned my addr 172.21.2.101:0/1200831121 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:01.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.706 7f24f4e17700 1 -- 172.21.2.101:0/1200831121 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24f01b9470 msgr2=0x7f24f01ae6d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:01.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.706 7f24f4e17700 1 --2- 172.21.2.101:0/1200831121 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24f01b9470 0x7f24f01ae6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:01.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.706 7f24f4e17700 1 -- 172.21.2.101:0/1200831121 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24f01aecf0 msgr2=0x7f24f00957f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:01.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.706 7f24f4e17700 1 --2- 172.21.2.101:0/1200831121 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24f01aecf0 0x7f24f00957f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:01.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.706 7f24f4e17700 1 -- 172.21.2.101:0/1200831121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f019e6b0 con 0x7f24f01af0f0 2021-02-10T11:45:01.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.706 7f24f4e17700 1 --2- 172.21.2.101:0/1200831121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24f01af0f0 0x7f24f0095e10 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 rx=0x7f24e8009980 tx=0x7f24e800bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:01.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.706 7f24ecff9700 1 -- 172.21.2.101:0/1200831121 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f24e800c2f0 con 0x7f24f01af0f0 2021-02-10T11:45:01.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.706 7f24ecff9700 1 -- 172.21.2.101:0/1200831121 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e8006790 con 0x7f24f01af0f0 2021-02-10T11:45:01.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.706 7f24ecff9700 1 -- 172.21.2.101:0/1200831121 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f24e800c870 con 0x7f24f01af0f0 2021-02-10T11:45:01.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.706 7f24f67fa700 1 -- 172.21.2.101:0/1200831121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f019e6b0 con 0x7f24f01af0f0 2021-02-10T11:45:01.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.706 7f24f67fa700 1 -- 172.21.2.101:0/1200831121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f0096720 con 0x7f24f01af0f0 2021-02-10T11:45:01.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.707 7f24ecff9700 1 -- 172.21.2.101:0/1200831121 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f24e80011f0 con 0x7f24f01af0f0 2021-02-10T11:45:01.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.707 7f24ecff9700 1 --2- 172.21.2.101:0/1200831121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24d00186c0 0x7f24d001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:01.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.707 7f24ecff9700 1 -- 172.21.2.101:0/1200831121 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f24e80328e0 con 0x7f24f01af0f0 2021-02-10T11:45:01.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.707 7f24effff700 1 --2- 172.21.2.101:0/1200831121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24d00186c0 0x7f24d001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:01.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.707 7f24effff700 1 --2- 172.21.2.101:0/1200831121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24d00186c0 0x7f24d001abc0 crc :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:01.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.738 7f24f67fa700 1 -- 172.21.2.101:0/1200831121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f00829a0 con 0x7f24f01af0f0 2021-02-10T11:45:01.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.740 7f24ecff9700 1 -- 172.21.2.101:0/1200831121 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f24e8000f40 con 0x7f24f01af0f0 2021-02-10T11:45:01.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.870 7f24f67fa700 1 -- 172.21.2.101:0/1200831121 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f24f00a7780 con 0x7f24f01af0f0 2021-02-10T11:45:01.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.873 7f24ecff9700 1 -- 172.21.2.101:0/1200831121 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v89) v1 ==== 184+0+0 (secure 0 0 0) 0x7f24e80011f0 con 0x7f24f01af0f0 2021-02-10T11:45:01.875 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.2 2021-02-10T11:45:01.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.879 7f24f67fa700 1 -- 172.21.2.101:0/1200831121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24d00186c0 msgr2=0x7f24d001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:01.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.879 7f24f67fa700 1 --2- 172.21.2.101:0/1200831121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24d00186c0 0x7f24d001abc0 crc :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:01.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.879 7f24f67fa700 1 -- 172.21.2.101:0/1200831121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24f01af0f0 msgr2=0x7f24f0095e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:01.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.879 7f24f67fa700 1 --2- 172.21.2.101:0/1200831121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24f01af0f0 0x7f24f0095e10 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 rx=0x7f24e8009980 tx=0x7f24e800bcd0).stop 2021-02-10T11:45:01.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.879 7f24f67fa700 1 -- 172.21.2.101:0/1200831121 shutdown_connections 2021-02-10T11:45:01.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.879 7f24f67fa700 1 --2- 172.21.2.101:0/1200831121 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24f01af0f0 0x7f24f0095e10 unknown :-1 s=CLOSED pgs=2774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:01.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.879 7f24f67fa700 1 --2- 172.21.2.101:0/1200831121 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24f01aecf0 0x7f24f00957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:01.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.879 7f24f67fa700 1 --2- 172.21.2.101:0/1200831121 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24d00186c0 0x7f24d001abc0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:01.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.879 7f24f67fa700 1 --2- 172.21.2.101:0/1200831121 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24f01b9470 0x7f24f01ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:01.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.879 7f24f67fa700 1 -- 172.21.2.101:0/1200831121 >> 172.21.2.101:0/1200831121 conn(0x7f24f00845b0 msgr2=0x7f24f018c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:01.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.879 7f24f67fa700 1 -- 172.21.2.101:0/1200831121 shutdown_connections 2021-02-10T11:45:01.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:01.879 7f24f67fa700 1 -- 172.21.2.101:0/1200831121 wait complete. 2021-02-10T11:45:01.886 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:45:02.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.311 7ff67e646700 1 Processor -- start 2021-02-10T11:45:02.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.311 7ff67e646700 1 -- start start 2021-02-10T11:45:02.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.312 7ff67e646700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff6780d1880 0x7ff6781b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:02.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.312 7ff67e646700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6781b0af0 0x7ff678108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:02.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.312 7ff67e646700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6780fe1d0 con 0x7ff6780d1c80 2021-02-10T11:45:02.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.312 7ff67e646700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff678109a70 con 0x7ff6780d1880 2021-02-10T11:45:02.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.312 7ff67e646700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff678109760 con 0x7ff6781b0af0 2021-02-10T11:45:02.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.312 7ff67cc63700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff6780d1c80 0x7ff678108f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60802/0 (socket says 172.21.2.101:60802) 2021-02-10T11:45:02.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.312 7ff67cc63700 1 -- 172.21.2.101:0/2402353510 learned_addr learned my addr 172.21.2.101:0/2402353510 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:02.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.312 7ff677fff700 1 --2- 172.21.2.101:0/2402353510 >> v2:172.21.2.101:3300/0 conn(0x7ff6780d1880 0x7ff6781b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:02.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.312 7ff676ffd700 1 -- 172.21.2.101:0/2402353510 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029108451 0 0) 0x7ff670001500 con 0x7ff6780d1c80 2021-02-10T11:45:02.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.312 7ff676ffd700 1 -- 172.21.2.101:0/2402353510 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff65c002fe0 con 0x7ff6780d1c80 2021-02-10T11:45:02.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.312 7ff677fff700 1 -- 172.21.2.101:0/2402353510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6781b0af0 msgr2=0x7ff678108950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:45:02.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.312 7ff677fff700 1 --2- 172.21.2.101:0/2402353510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6781b0af0 0x7ff678108950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:02.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.312 7ff677fff700 1 -- 172.21.2.101:0/2402353510 >> v1:172.21.2.101:6789/0 conn(0x7ff6780d1c80 legacy=0x7ff678108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:02.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.312 7ff676ffd700 1 -- 172.21.2.101:0/2402353510 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 19538036 0 0) 0x7ff670001270 con 0x7ff6780d1c80 2021-02-10T11:45:02.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.312 7ff677fff700 1 -- 172.21.2.101:0/2402353510 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff678088070 con 0x7ff6780d1880 2021-02-10T11:45:02.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.313 7ff677fff700 1 --2- 172.21.2.101:0/2402353510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6780d1880 0x7ff6781b04d0 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 rx=0x7ff668006650 tx=0x7ff668009a00).ready entity=mon.0 client_cookie=71cf176cdedb497e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:02.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.313 7ff676ffd700 1 -- 172.21.2.101:0/2402353510 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff668009df0 con 0x7ff6780d1880 2021-02-10T11:45:02.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.313 7ff676ffd700 1 -- 172.21.2.101:0/2402353510 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6680009b0 con 0x7ff6780d1880 2021-02-10T11:45:02.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.313 7ff676ffd700 1 -- 172.21.2.101:0/2402353510 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff66800c2b0 con 0x7ff6780d1880 2021-02-10T11:45:02.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.313 7ff67e646700 1 -- 172.21.2.101:0/2402353510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6780d1880 msgr2=0x7ff6781b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:02.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.313 7ff67e646700 1 --2- 172.21.2.101:0/2402353510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6780d1880 0x7ff6781b04d0 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 rx=0x7ff668006650 tx=0x7ff668009a00).stop 2021-02-10T11:45:02.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.313 7ff67e646700 1 -- 172.21.2.101:0/2402353510 shutdown_connections 2021-02-10T11:45:02.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.313 7ff67e646700 1 --2- 172.21.2.101:0/2402353510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6781b0af0 0x7ff678108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:02.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.313 7ff67e646700 1 --2- 172.21.2.101:0/2402353510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6780d1880 0x7ff6781b04d0 unknown :-1 s=CLOSED pgs=2776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:02.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.313 7ff67e646700 1 -- 172.21.2.101:0/2402353510 >> 172.21.2.101:0/2402353510 conn(0x7ff6780ce430 msgr2=0x7ff6780d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:02.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.313 7ff67e646700 1 -- 172.21.2.101:0/2402353510 shutdown_connections 2021-02-10T11:45:02.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.313 7ff67e646700 1 -- 172.21.2.101:0/2402353510 wait complete. 2021-02-10T11:45:02.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.328 7ff67e646700 1 Processor -- start 2021-02-10T11:45:02.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.328 7ff67e646700 1 -- start start 2021-02-10T11:45:02.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.328 7ff67e646700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6781b40d0 0x7ff67818de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:02.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.328 7ff67e646700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6780a8320 0x7ff678083e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:02.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.328 7ff67e646700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6780844a0 0x7ff6781b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:02.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.328 7ff67e646700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6780876a0 con 0x7ff6780844a0 2021-02-10T11:45:02.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.328 7ff67e646700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6781b6340 con 0x7ff6781b40d0 2021-02-10T11:45:02.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.328 7ff67e646700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6780a8720 con 0x7ff6780a8320 2021-02-10T11:45:02.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.329 7ff67cc63700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6780844a0 0x7ff6781b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:02.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.329 7ff67cc63700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6780844a0 0x7ff6781b68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60316/0 (socket says 172.21.2.101:60316) 2021-02-10T11:45:02.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.329 7ff67cc63700 1 -- 172.21.2.101:0/3041453279 learned_addr learned my addr 172.21.2.101:0/3041453279 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:02.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.329 7ff67cc63700 1 -- 172.21.2.101:0/3041453279 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6780a8320 msgr2=0x7ff678083e80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:02.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.329 7ff67cc63700 1 --2- 172.21.2.101:0/3041453279 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6780a8320 0x7ff678083e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:02.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.329 7ff67cc63700 1 -- 172.21.2.101:0/3041453279 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6781b40d0 msgr2=0x7ff67818de40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:02.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.329 7ff67cc63700 1 --2- 172.21.2.101:0/3041453279 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6781b40d0 0x7ff67818de40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:02.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.329 7ff67cc63700 1 -- 172.21.2.101:0/3041453279 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6780a8940 con 0x7ff6780844a0 2021-02-10T11:45:02.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.329 7ff67cc63700 1 --2- 172.21.2.101:0/3041453279 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6780844a0 0x7ff6781b68c0 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 rx=0x7ff67000a900 tx=0x7ff67000aca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:02.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.329 7ff674ff9700 1 -- 172.21.2.101:0/3041453279 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff670007020 con 0x7ff6780844a0 2021-02-10T11:45:02.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.329 7ff674ff9700 1 -- 172.21.2.101:0/3041453279 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff670007320 con 0x7ff6780844a0 2021-02-10T11:45:02.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.329 7ff674ff9700 1 -- 172.21.2.101:0/3041453279 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff670004720 con 0x7ff6780844a0 2021-02-10T11:45:02.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.330 7ff67e646700 1 -- 172.21.2.101:0/3041453279 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6780a8940 con 0x7ff6780844a0 2021-02-10T11:45:02.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.330 7ff67e646700 1 -- 172.21.2.101:0/3041453279 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6781b71d0 con 0x7ff6780844a0 2021-02-10T11:45:02.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.330 7ff674ff9700 1 -- 172.21.2.101:0/3041453279 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff670007500 con 0x7ff6780844a0 2021-02-10T11:45:02.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.330 7ff674ff9700 1 --2- 172.21.2.101:0/3041453279 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6540186c0 0x7ff65401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:02.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.330 7ff674ff9700 1 -- 172.21.2.101:0/3041453279 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff6700327a0 con 0x7ff6780844a0 2021-02-10T11:45:02.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.330 7ff677fff700 1 --2- 172.21.2.101:0/3041453279 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6540186c0 0x7ff65401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:02.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.330 7ff677fff700 1 --2- 172.21.2.101:0/3041453279 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6540186c0 0x7ff65401abc0 crc :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:02.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.362 7ff67e646700 1 -- 172.21.2.101:0/3041453279 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff678082990 con 0x7ff6780844a0 2021-02-10T11:45:02.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.364 7ff674ff9700 1 -- 172.21.2.101:0/3041453279 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff670001270 con 0x7ff6780844a0 2021-02-10T11:45:02.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.493 7ff67e646700 1 -- 172.21.2.101:0/3041453279 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ff678082990 con 0x7ff6780844a0 2021-02-10T11:45:02.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.496 7ff674ff9700 1 -- 172.21.2.101:0/3041453279 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v90) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff670007500 con 0x7ff6780844a0 2021-02-10T11:45:02.495 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.3 2021-02-10T11:45:02.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.497 7ff67e646700 1 -- 172.21.2.101:0/3041453279 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6540186c0 msgr2=0x7ff65401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:02.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.497 7ff67e646700 1 --2- 172.21.2.101:0/3041453279 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6540186c0 0x7ff65401abc0 crc :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:02.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.497 7ff67e646700 1 -- 172.21.2.101:0/3041453279 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6780844a0 msgr2=0x7ff6781b68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:02.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.497 7ff67e646700 1 --2- 172.21.2.101:0/3041453279 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6780844a0 0x7ff6781b68c0 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 rx=0x7ff67000a900 tx=0x7ff67000aca0).stop 2021-02-10T11:45:02.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.498 7ff67e646700 1 -- 172.21.2.101:0/3041453279 shutdown_connections 2021-02-10T11:45:02.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.498 7ff67e646700 1 --2- 172.21.2.101:0/3041453279 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6540186c0 0x7ff65401abc0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:02.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.498 7ff67e646700 1 --2- 172.21.2.101:0/3041453279 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6780a8320 0x7ff678083e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:02.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.498 7ff67e646700 1 --2- 172.21.2.101:0/3041453279 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6780844a0 0x7ff6781b68c0 unknown :-1 s=CLOSED pgs=2777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:02.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.498 7ff67e646700 1 --2- 172.21.2.101:0/3041453279 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6781b40d0 0x7ff67818de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:02.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.498 7ff67e646700 1 -- 172.21.2.101:0/3041453279 >> 172.21.2.101:0/3041453279 conn(0x7ff67819c5a0 msgr2=0x7ff678082bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:02.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.498 7ff67e646700 1 -- 172.21.2.101:0/3041453279 shutdown_connections 2021-02-10T11:45:02.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.498 7ff67e646700 1 -- 172.21.2.101:0/3041453279 wait complete. 2021-02-10T11:45:02.505 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:45:02.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.916 7ff89b26e700 1 Processor -- start 2021-02-10T11:45:02.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.916 7ff89b26e700 1 -- start start 2021-02-10T11:45:02.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.917 7ff89b26e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8940d0340 0x7ff8940ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:02.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.917 7ff89b26e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff8940d1c70 0x7ff8940efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:02.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.917 7ff89b26e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8940f0b10 con 0x7ff8940d1870 2021-02-10T11:45:02.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.917 7ff89b26e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8940e5ed0 con 0x7ff8940d1c70 2021-02-10T11:45:02.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.917 7ff89b26e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8941b0be0 con 0x7ff8940d0340 2021-02-10T11:45:02.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.917 7ff89988b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff8940d1c70 0x7ff8940efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:02.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.917 7ff89908a700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7ff8940d1870 0x7ff8941b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:54226/0 (socket says 172.21.2.101:54226) 2021-02-10T11:45:02.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.917 7ff89908a700 1 -- 172.21.2.101:0/4096472733 learned_addr learned my addr 172.21.2.101:0/4096472733 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:02.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.917 7ff898889700 1 --2- 172.21.2.101:0/4096472733 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8940d0340 0x7ff8940ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:02.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.917 7ff89988b700 1 -- 172.21.2.101:0/4096472733 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8940d0340 msgr2=0x7ff8940ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:02.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.917 7ff89988b700 1 --2- 172.21.2.101:0/4096472733 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8940d0340 0x7ff8940ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.917 7ff89988b700 1 -- 172.21.2.101:0/4096472733 >> v1:172.21.2.108:6789/0 conn(0x7ff8940d1870 legacy=0x7ff8941b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.917 7ff89988b700 1 -- 172.21.2.101:0/4096472733 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8940f20a0 con 0x7ff8940d1c70 2021-02-10T11:45:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.918 7ff89988b700 1 --2- 172.21.2.101:0/4096472733 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8940d1c70 0x7ff8940efc60 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 rx=0x7ff88c006e30 tx=0x7ff88c009e60).ready entity=mon.0 client_cookie=b7fbb38b7ba512bc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.918 7ff887fff700 1 -- 172.21.2.101:0/4096472733 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff88c006a50 con 0x7ff8940d1c70 2021-02-10T11:45:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.918 7ff887fff700 1 -- 172.21.2.101:0/4096472733 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88c00b300 con 0x7ff8940d1c70 2021-02-10T11:45:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.918 7ff887fff700 1 -- 172.21.2.101:0/4096472733 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff88c00b820 con 0x7ff8940d1c70 2021-02-10T11:45:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.918 7ff89b26e700 1 -- 172.21.2.101:0/4096472733 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8940d1c70 msgr2=0x7ff8940efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.918 7ff89b26e700 1 --2- 172.21.2.101:0/4096472733 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8940d1c70 0x7ff8940efc60 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 rx=0x7ff88c006e30 tx=0x7ff88c009e60).stop 2021-02-10T11:45:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.918 7ff89b26e700 1 -- 172.21.2.101:0/4096472733 shutdown_connections 2021-02-10T11:45:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.918 7ff89b26e700 1 --2- 172.21.2.101:0/4096472733 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8940d0340 0x7ff8940ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.918 7ff89b26e700 1 --2- 172.21.2.101:0/4096472733 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff8940d1c70 0x7ff8940efc60 unknown :-1 s=CLOSED pgs=2778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.918 7ff89b26e700 1 -- 172.21.2.101:0/4096472733 >> 172.21.2.101:0/4096472733 conn(0x7ff8940ce450 msgr2=0x7ff8940d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:02.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.919 7ff89b26e700 1 -- 172.21.2.101:0/4096472733 shutdown_connections 2021-02-10T11:45:02.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.919 7ff89b26e700 1 -- 172.21.2.101:0/4096472733 wait complete. 2021-02-10T11:45:02.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.933 7ff89b26e700 1 Processor -- start 2021-02-10T11:45:02.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.933 7ff89b26e700 1 -- start start 2021-02-10T11:45:02.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.934 7ff89b26e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff894093f10 0x7ff894094310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.934 7ff89b26e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8940ae9e0 0x7ff8940aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.934 7ff89b26e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff894092d90 0x7ff894093190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.934 7ff89b26e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89419ebc0 con 0x7ff894092d90 2021-02-10T11:45:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.934 7ff89b26e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8941b8760 con 0x7ff894093f10 2021-02-10T11:45:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.934 7ff89b26e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8940937b0 con 0x7ff8940ae9e0 2021-02-10T11:45:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.934 7ff89988b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff894092d90 0x7ff894093190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.934 7ff89988b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff894092d90 0x7ff894093190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60332/0 (socket says 172.21.2.101:60332) 2021-02-10T11:45:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.934 7ff89988b700 1 -- 172.21.2.101:0/2931998163 learned_addr learned my addr 172.21.2.101:0/2931998163 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.934 7ff89988b700 1 -- 172.21.2.101:0/2931998163 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8940ae9e0 msgr2=0x7ff8940aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.934 7ff89988b700 1 --2- 172.21.2.101:0/2931998163 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8940ae9e0 0x7ff8940aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.934 7ff89988b700 1 -- 172.21.2.101:0/2931998163 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff894093f10 msgr2=0x7ff894094310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.934 7ff89988b700 1 --2- 172.21.2.101:0/2931998163 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff894093f10 0x7ff894094310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.934 7ff89988b700 1 -- 172.21.2.101:0/2931998163 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8940939d0 con 0x7ff894092d90 2021-02-10T11:45:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.934 7ff89988b700 1 --2- 172.21.2.101:0/2931998163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff894092d90 0x7ff894093190 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 rx=0x7ff88c00b970 tx=0x7ff88c00bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.935 7ff885ffb700 1 -- 172.21.2.101:0/2931998163 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff88c009380 con 0x7ff894092d90 2021-02-10T11:45:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.935 7ff885ffb700 1 -- 172.21.2.101:0/2931998163 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88c005820 con 0x7ff894092d90 2021-02-10T11:45:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.935 7ff89b26e700 1 -- 172.21.2.101:0/2931998163 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8940939d0 con 0x7ff894092d90 2021-02-10T11:45:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.935 7ff89b26e700 1 -- 172.21.2.101:0/2931998163 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8940939d0 con 0x7ff894092d90 2021-02-10T11:45:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.935 7ff885ffb700 1 -- 172.21.2.101:0/2931998163 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff88c005d00 con 0x7ff894092d90 2021-02-10T11:45:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.935 7ff885ffb700 1 -- 172.21.2.101:0/2931998163 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff88c005a00 con 0x7ff894092d90 2021-02-10T11:45:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.935 7ff885ffb700 1 --2- 172.21.2.101:0/2931998163 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff8740186c0 0x7ff87401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.935 7ff885ffb700 1 -- 172.21.2.101:0/2931998163 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff88c032ba0 con 0x7ff894092d90 2021-02-10T11:45:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.935 7ff89908a700 1 --2- 172.21.2.101:0/2931998163 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff8740186c0 0x7ff87401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.936 7ff89908a700 1 --2- 172.21.2.101:0/2931998163 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff8740186c0 0x7ff87401abc0 crc :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:02.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.967 7ff89b26e700 1 -- 172.21.2.101:0/2931998163 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8940ebc60 con 0x7ff894092d90 2021-02-10T11:45:02.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:02.969 7ff885ffb700 1 -- 172.21.2.101:0/2931998163 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff88c000fa0 con 0x7ff894092d90 2021-02-10T11:45:03.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:03.098 7ff89b26e700 1 -- 172.21.2.101:0/2931998163 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8940f0b10 con 0x7ff894092d90 2021-02-10T11:45:03.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:03.098 7ff885ffb700 1 -- 172.21.2.101:0/2931998163 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+2105 (secure 0 0 0) 0x7ff88c005820 con 0x7ff894092d90 2021-02-10T11:45:03.097 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:45:03.098 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":131,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":12135,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/3467749863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3467749863},{"type":"v1","addr":"172.21.2.101:6835","nonce":3467749863}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":128}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:44:56.768458","modified":"2021-02-10 11:44:58.773048","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8242":{"gid":8242,"name":"b","rank":0,"incarnation":129,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/485291097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":485291097},{"type":"v1","addr":"172.21.2.108:6833","nonce":485291097}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2021-02-10T11:45:03.098 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 131 2021-02-10T11:45:03.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:03.100 7ff89b26e700 1 -- 172.21.2.101:0/2931998163 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff8740186c0 msgr2=0x7ff87401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:03.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:03.100 7ff89b26e700 1 --2- 172.21.2.101:0/2931998163 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff8740186c0 0x7ff87401abc0 crc :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:03.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:03.100 7ff89b26e700 1 -- 172.21.2.101:0/2931998163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff894092d90 msgr2=0x7ff894093190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:03.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:03.100 7ff89b26e700 1 --2- 172.21.2.101:0/2931998163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff894092d90 0x7ff894093190 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 rx=0x7ff88c00b970 tx=0x7ff88c00bd10).stop 2021-02-10T11:45:03.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:03.100 7ff89b26e700 1 -- 172.21.2.101:0/2931998163 shutdown_connections 2021-02-10T11:45:03.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:03.100 7ff89b26e700 1 --2- 172.21.2.101:0/2931998163 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff8740186c0 0x7ff87401abc0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:03.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:03.100 7ff89b26e700 1 --2- 172.21.2.101:0/2931998163 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff8940ae9e0 0x7ff8940aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:03.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:03.100 7ff89b26e700 1 --2- 172.21.2.101:0/2931998163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff894092d90 0x7ff894093190 unknown :-1 s=CLOSED pgs=2779 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:03.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:03.100 7ff89b26e700 1 --2- 172.21.2.101:0/2931998163 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff894093f10 0x7ff894094310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:03.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:03.100 7ff89b26e700 1 -- 172.21.2.101:0/2931998163 >> 172.21.2.101:0/2931998163 conn(0x7ff89419f8a0 msgr2=0x7ff894094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:03.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:03.100 7ff89b26e700 1 -- 172.21.2.101:0/2931998163 shutdown_connections 2021-02-10T11:45:03.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:03.100 7ff89b26e700 1 -- 172.21.2.101:0/2931998163 wait complete. 2021-02-10T11:45:03.309 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 131, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:44:56.768458', 'modified': '2021-02-10 11:44:58.773048', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8242}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8242': {'gid': 8242, 'name': 'b', 'rank': 0, 'incarnation': 129, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/485291097', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 485291097}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 485291097}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [33], 'metadata_pool': 32, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:45:03.309 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:45:03.309 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:45:03.707 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:45:03.707 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:45:03.707 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:45:03.707 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 8242, 2021-02-10T11:45:03.707 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:45:03.707 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:45:03.707 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 6.9376191929999997, 2021-02-10T11:45:03.707 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 131, 2021-02-10T11:45:03.707 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 156, 2021-02-10T11:45:03.707 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 156, 2021-02-10T11:45:03.707 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 12.581797111 2021-02-10T11:45:03.708 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:45:03.711 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 8242, "want_state": "up:active", "state": "up:active", "rank_uptime": 6.9376191929999997, "mdsmap_epoch": 131, "osdmap_epoch": 156, "osdmap_epoch_barrier": 156, "uptime": 12.581797111 } 2021-02-10T11:45:03.711 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:45:04.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.123 7f433276c700 1 Processor -- start 2021-02-10T11:45:04.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.123 7f433276c700 1 -- start start 2021-02-10T11:45:04.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.123 7f433276c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f432c0d1870 0x7f432c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:04.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.123 7f433276c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f432c1b0a10 0x7f432c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:04.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.123 7f433276c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432c0f0b10 con 0x7f432c0d1c70 2021-02-10T11:45:04.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.124 7f433276c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432c0e5ed0 con 0x7f432c0d1870 2021-02-10T11:45:04.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.124 7f433276c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432c0f0650 con 0x7f432c1b0a10 2021-02-10T11:45:04.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.124 7f4330d89700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f432c0d1c70 0x7f432c0cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60830/0 (socket says 172.21.2.101:60830) 2021-02-10T11:45:04.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.124 7f4330d89700 1 -- 172.21.2.101:0/256975204 learned_addr learned my addr 172.21.2.101:0/256975204 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:04.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.124 7f432affd700 1 -- 172.21.2.101:0/256975204 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2545454965 0 0) 0x7f4324001500 con 0x7f432c0d1c70 2021-02-10T11:45:04.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.124 7f432affd700 1 -- 172.21.2.101:0/256975204 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4314002fe0 con 0x7f432c0d1c70 2021-02-10T11:45:04.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.124 7f432affd700 1 -- 172.21.2.101:0/256975204 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1283958669 0 0) 0x7f4324001270 con 0x7f432c0d1c70 2021-02-10T11:45:04.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.124 7f432affd700 1 -- 172.21.2.101:0/256975204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f432c1b0a10 msgr2=0x7f432c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.124 7f432bfff700 1 --2- 172.21.2.101:0/256975204 >> v2:172.21.2.108:3300/0 conn(0x7f432c0d1870 0x7f432c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.124 7f432b7fe700 1 --2- 172.21.2.101:0/256975204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f432c1b0a10 0x7f432c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.124 7f432affd700 1 --2- 172.21.2.101:0/256975204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f432c1b0a10 0x7f432c0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.124 7f432affd700 1 -- 172.21.2.101:0/256975204 >> v2:172.21.2.108:3300/0 conn(0x7f432c0d1870 msgr2=0x7f432c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.124 7f432affd700 1 --2- 172.21.2.101:0/256975204 >> v2:172.21.2.108:3300/0 conn(0x7f432c0d1870 0x7f432c1b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.124 7f432affd700 1 -- 172.21.2.101:0/256975204 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432c0f0870 con 0x7f432c0d1c70 2021-02-10T11:45:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.124 7f432affd700 1 -- 172.21.2.101:0/256975204 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f4324001b50 con 0x7f432c0d1c70 2021-02-10T11:45:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.124 7f432affd700 1 -- 172.21.2.101:0/256975204 >> v1:172.21.2.101:6789/0 conn(0x7f432c0d1c70 legacy=0x7f432c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f432affd700 1 --2- 172.21.2.101:0/256975204 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4314003b90 0x7f4314006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f432affd700 1 --2- 172.21.2.101:0/256975204 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43140068c0 0x7f4314008de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f432affd700 1 --2- 172.21.2.101:0/256975204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43140096e0 0x7f431400bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f432affd700 1 -- 172.21.2.101:0/256975204 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431400c420 con 0x7f43140068c0 2021-02-10T11:45:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f432affd700 1 -- 172.21.2.101:0/256975204 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431400c640 con 0x7f4314003b90 2021-02-10T11:45:04.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f432affd700 1 -- 172.21.2.101:0/256975204 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431400c860 con 0x7f43140096e0 2021-02-10T11:45:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f432bfff700 1 --2- 172.21.2.101:0/256975204 >> v2:172.21.2.108:3300/0 conn(0x7f432c0d1870 0x7f432c1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f432b7fe700 1 --2- 172.21.2.101:0/256975204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f432c1b0a10 0x7f432c0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f432b7fe700 1 --2- 172.21.2.101:0/256975204 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43140068c0 0x7f4314008de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f432bfff700 1 --2- 172.21.2.101:0/256975204 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4314003b90 0x7f4314006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f432b7fe700 1 -- 172.21.2.101:0/256975204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43140096e0 msgr2=0x7f431400bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f4330d89700 1 --2- 172.21.2.101:0/256975204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43140096e0 0x7f431400bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f432b7fe700 1 --2- 172.21.2.101:0/256975204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43140096e0 0x7f431400bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f432b7fe700 1 -- 172.21.2.101:0/256975204 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4314003b90 msgr2=0x7f4314006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f432b7fe700 1 --2- 172.21.2.101:0/256975204 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4314003b90 0x7f4314006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f432b7fe700 1 -- 172.21.2.101:0/256975204 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4314050b10 con 0x7f43140068c0 2021-02-10T11:45:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f432bfff700 1 --2- 172.21.2.101:0/256975204 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4314003b90 0x7f4314006020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:04.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f4330d89700 1 --2- 172.21.2.101:0/256975204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f43140096e0 0x7f431400bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:04.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f432b7fe700 1 --2- 172.21.2.101:0/256975204 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43140068c0 0x7f4314008de0 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 rx=0x7f4320002e00 tx=0x7f4320009ab0).ready entity=mon.0 client_cookie=68834d1066656bb8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:04.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f432affd700 1 -- 172.21.2.101:0/256975204 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4320000c90 con 0x7f43140068c0 2021-02-10T11:45:04.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f432affd700 1 -- 172.21.2.101:0/256975204 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432000c0c0 con 0x7f43140068c0 2021-02-10T11:45:04.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f432affd700 1 -- 172.21.2.101:0/256975204 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4320000c90 con 0x7f43140068c0 2021-02-10T11:45:04.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f433276c700 1 -- 172.21.2.101:0/256975204 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43140068c0 msgr2=0x7f4314008de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:04.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.125 7f433276c700 1 --2- 172.21.2.101:0/256975204 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f43140068c0 0x7f4314008de0 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 rx=0x7f4320002e00 tx=0x7f4320009ab0).stop 2021-02-10T11:45:04.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.126 7f4330d89700 1 -- 172.21.2.101:0/256975204 reap_dead start 2021-02-10T11:45:04.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.126 7f433276c700 1 -- 172.21.2.101:0/256975204 shutdown_connections 2021-02-10T11:45:04.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.126 7f433276c700 1 -- 172.21.2.101:0/256975204 >> 172.21.2.101:0/256975204 conn(0x7f432c0ce450 msgr2=0x7f432c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:04.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.126 7f433276c700 1 -- 172.21.2.101:0/256975204 shutdown_connections 2021-02-10T11:45:04.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.126 7f433276c700 1 -- 172.21.2.101:0/256975204 wait complete. 2021-02-10T11:45:04.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.141 7f433276c700 1 Processor -- start 2021-02-10T11:45:04.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.141 7f433276c700 1 -- start start 2021-02-10T11:45:04.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.141 7f433276c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f432c195650 0x7f432c0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:04.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.141 7f433276c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f432c0b5b80 0x7f432c1a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:04.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.141 7f433276c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f432c0acfa0 0x7f432c1a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:04.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.141 7f433276c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c0b6010 con 0x7f432c0b5b80 2021-02-10T11:45:04.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.141 7f433276c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c1a3a60 con 0x7f432c195650 2021-02-10T11:45:04.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.141 7f433276c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432c1a3c40 con 0x7f432c0acfa0 2021-02-10T11:45:04.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.141 7f432b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f432c0b5b80 0x7f432c1a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:04.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.142 7f432b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f432c0b5b80 0x7f432c1a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60350/0 (socket says 172.21.2.101:60350) 2021-02-10T11:45:04.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.142 7f432b7fe700 1 -- 172.21.2.101:0/2798909386 learned_addr learned my addr 172.21.2.101:0/2798909386 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:04.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.142 7f432b7fe700 1 -- 172.21.2.101:0/2798909386 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f432c0acfa0 msgr2=0x7f432c1a3440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:04.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.142 7f432b7fe700 1 --2- 172.21.2.101:0/2798909386 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f432c0acfa0 0x7f432c1a3440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:04.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.142 7f432b7fe700 1 -- 172.21.2.101:0/2798909386 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f432c195650 msgr2=0x7f432c0b5560 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:04.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.142 7f432b7fe700 1 --2- 172.21.2.101:0/2798909386 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f432c195650 0x7f432c0b5560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:04.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.142 7f432b7fe700 1 -- 172.21.2.101:0/2798909386 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432c0bcae0 con 0x7f432c0b5b80 2021-02-10T11:45:04.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.142 7f432b7fe700 1 --2- 172.21.2.101:0/2798909386 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f432c0b5b80 0x7f432c1a2e20 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 rx=0x7f432000b5e0 tx=0x7f432000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:04.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.142 7f4328ff9700 1 -- 172.21.2.101:0/2798909386 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f432000bd70 con 0x7f432c0b5b80 2021-02-10T11:45:04.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.142 7f4328ff9700 1 -- 172.21.2.101:0/2798909386 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4320006030 con 0x7f432c0b5b80 2021-02-10T11:45:04.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.142 7f4328ff9700 1 -- 172.21.2.101:0/2798909386 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4320005560 con 0x7f432c0b5b80 2021-02-10T11:45:04.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.142 7f433276c700 1 -- 172.21.2.101:0/2798909386 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432c0bcae0 con 0x7f432c0b5b80 2021-02-10T11:45:04.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.142 7f433276c700 1 -- 172.21.2.101:0/2798909386 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432c0bd070 con 0x7f432c0b5b80 2021-02-10T11:45:04.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.143 7f4328ff9700 1 -- 172.21.2.101:0/2798909386 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f43200052e0 con 0x7f432c0b5b80 2021-02-10T11:45:04.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.143 7f4328ff9700 1 --2- 172.21.2.101:0/2798909386 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f430c0186d0 0x7f430c01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:04.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.143 7f4328ff9700 1 -- 172.21.2.101:0/2798909386 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4320032630 con 0x7f432c0b5b80 2021-02-10T11:45:04.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.143 7f432bfff700 1 --2- 172.21.2.101:0/2798909386 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f430c0186d0 0x7f430c01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:04.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.143 7f432bfff700 1 --2- 172.21.2.101:0/2798909386 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f430c0186d0 0x7f430c01abd0 crc :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:04.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.173 7f433276c700 1 -- 172.21.2.101:0/2798909386 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f432c1a3c40 con 0x7f432c0b5b80 2021-02-10T11:45:04.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.176 7f4328ff9700 1 -- 172.21.2.101:0/2798909386 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4320006030 con 0x7f432c0b5b80 2021-02-10T11:45:04.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.301 7f433276c700 1 -- 172.21.2.101:0/2798909386 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f432c1a3c40 con 0x7f432c0b5b80 2021-02-10T11:45:04.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.302 7f4328ff9700 1 -- 172.21.2.101:0/2798909386 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f432000bd70 con 0x7f432c0b5b80 2021-02-10T11:45:04.301 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:45:04.301 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":156,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:44:57.770382","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:44:54.736504","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":33,"pool_name":"cephfs_data","create_time":"2021-02-10 11:44:55.811077","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:45:04.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.303 7f433276c700 1 -- 172.21.2.101:0/2798909386 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f430c0186d0 msgr2=0x7f430c01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:04.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.303 7f433276c700 1 --2- 172.21.2.101:0/2798909386 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f430c0186d0 0x7f430c01abd0 crc :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:04.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.303 7f433276c700 1 -- 172.21.2.101:0/2798909386 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f432c0b5b80 msgr2=0x7f432c1a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:04.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.303 7f433276c700 1 --2- 172.21.2.101:0/2798909386 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f432c0b5b80 0x7f432c1a2e20 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 rx=0x7f432000b5e0 tx=0x7f432000b980).stop 2021-02-10T11:45:04.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.303 7f433276c700 1 -- 172.21.2.101:0/2798909386 shutdown_connections 2021-02-10T11:45:04.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.303 7f433276c700 1 --2- 172.21.2.101:0/2798909386 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f430c0186d0 0x7f430c01abd0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:04.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.303 7f433276c700 1 --2- 172.21.2.101:0/2798909386 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f432c0acfa0 0x7f432c1a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:04.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.303 7f433276c700 1 --2- 172.21.2.101:0/2798909386 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f432c0b5b80 0x7f432c1a2e20 unknown :-1 s=CLOSED pgs=2782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:04.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.303 7f433276c700 1 --2- 172.21.2.101:0/2798909386 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f432c195650 0x7f432c0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:04.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.303 7f433276c700 1 -- 172.21.2.101:0/2798909386 >> 172.21.2.101:0/2798909386 conn(0x7f432c0b3040 msgr2=0x7f432c0c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:04.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.304 7f433276c700 1 -- 172.21.2.101:0/2798909386 shutdown_connections 2021-02-10T11:45:04.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.304 7f433276c700 1 -- 172.21.2.101:0/2798909386 wait complete. 2021-02-10T11:45:04.311 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 32, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:44:54.736504', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 33, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:44:55.811077', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '156', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:45:04.311 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:45:04.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.730 7f6e3ed64700 1 Processor -- start 2021-02-10T11:45:04.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.730 7f6e3ed64700 1 -- start start 2021-02-10T11:45:04.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.730 7f6e3ed64700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6e3810afe0 0x7f6e380cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:04.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.730 7f6e3ed64700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e380fe1b0 con 0x7f6e380d1c60 2021-02-10T11:45:04.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.730 7f6e3ed64700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e381b0ca0 con 0x7f6e380d1860 2021-02-10T11:45:04.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.730 7f6e3ed64700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e38109a50 con 0x7f6e3810afe0 2021-02-10T11:45:04.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.730 7f6e37fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f6e380d1c60 0x7f6e380cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60850/0 (socket says 172.21.2.101:60850) 2021-02-10T11:45:04.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.730 7f6e37fff700 1 -- 172.21.2.101:0/572402445 learned_addr learned my addr 172.21.2.101:0/572402445 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:04.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.730 7f6e377fe700 1 -- 172.21.2.101:0/572402445 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3272548926 0 0) 0x7f6e280009b0 con 0x7f6e380d1c60 2021-02-10T11:45:04.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e377fe700 1 -- 172.21.2.101:0/572402445 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e24002fe0 con 0x7f6e380d1c60 2021-02-10T11:45:04.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e3d381700 1 --2- 172.21.2.101:0/572402445 >> v2:172.21.2.108:3300/0 conn(0x7f6e3810afe0 0x7f6e380cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e377fe700 1 -- 172.21.2.101:0/572402445 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3158530940 0 0) 0x7f6e28001010 con 0x7f6e380d1c60 2021-02-10T11:45:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e377fe700 1 -- 172.21.2.101:0/572402445 >> v2:172.21.2.108:3300/0 conn(0x7f6e3810afe0 msgr2=0x7f6e380cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e377fe700 1 --2- 172.21.2.101:0/572402445 >> v2:172.21.2.108:3300/0 conn(0x7f6e3810afe0 0x7f6e380cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e377fe700 1 -- 172.21.2.101:0/572402445 >> v1:172.21.2.108:6789/0 conn(0x7f6e380d1860 legacy=0x7f6e381b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e377fe700 1 -- 172.21.2.101:0/572402445 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3810b3e0 con 0x7f6e380d1c60 2021-02-10T11:45:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e377fe700 1 -- 172.21.2.101:0/572402445 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f6e28001590 con 0x7f6e380d1c60 2021-02-10T11:45:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e377fe700 1 -- 172.21.2.101:0/572402445 >> v1:172.21.2.101:6789/0 conn(0x7f6e380d1c60 legacy=0x7f6e380cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e377fe700 1 --2- 172.21.2.101:0/572402445 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e24003b20 0x7f6e24006000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e377fe700 1 --2- 172.21.2.101:0/572402445 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e24006820 0x7f6e24008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e377fe700 1 --2- 172.21.2.101:0/572402445 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e24009640 0x7f6e2400bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e377fe700 1 -- 172.21.2.101:0/572402445 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2400c390 con 0x7f6e24009640 2021-02-10T11:45:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e377fe700 1 -- 172.21.2.101:0/572402445 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2400c5b0 con 0x7f6e24006820 2021-02-10T11:45:04.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e377fe700 1 -- 172.21.2.101:0/572402445 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2400c7d0 con 0x7f6e24003b20 2021-02-10T11:45:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e3d381700 1 --2- 172.21.2.101:0/572402445 >> v2:172.21.2.108:3300/0 conn(0x7f6e3810afe0 0x7f6e380cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:45:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e3cb80700 1 --2- 172.21.2.101:0/572402445 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e24003b20 0x7f6e24006000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e3d381700 1 --2- 172.21.2.101:0/572402445 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e24009640 0x7f6e2400bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e37fff700 1 --2- 172.21.2.101:0/572402445 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e24006820 0x7f6e24008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e3d381700 1 -- 172.21.2.101:0/572402445 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e24003b20 msgr2=0x7f6e24006000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e3d381700 1 --2- 172.21.2.101:0/572402445 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e24003b20 0x7f6e24006000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e3d381700 1 -- 172.21.2.101:0/572402445 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e24006820 msgr2=0x7f6e24008d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e3d381700 1 --2- 172.21.2.101:0/572402445 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e24006820 0x7f6e24008d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e3d381700 1 -- 172.21.2.101:0/572402445 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2400ca70 con 0x7f6e24009640 2021-02-10T11:45:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e3d381700 1 -- 172.21.2.101:0/572402445 reap_dead start 2021-02-10T11:45:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.731 7f6e3d381700 1 --2- 172.21.2.101:0/572402445 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e24009640 0x7f6e2400bb40 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 rx=0x7f6e300068c0 tx=0x7f6e3000b300).ready entity=mon.0 client_cookie=83ccc290e0c28201 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.732 7f6e377fe700 1 -- 172.21.2.101:0/572402445 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e3000b970 con 0x7f6e24009640 2021-02-10T11:45:04.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.732 7f6e377fe700 1 -- 172.21.2.101:0/572402445 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e30003030 con 0x7f6e24009640 2021-02-10T11:45:04.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.732 7f6e377fe700 1 -- 172.21.2.101:0/572402445 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e3000bcb0 con 0x7f6e24009640 2021-02-10T11:45:04.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.732 7f6e3ed64700 1 -- 172.21.2.101:0/572402445 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e24009640 msgr2=0x7f6e2400bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:04.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.732 7f6e3ed64700 1 --2- 172.21.2.101:0/572402445 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e24009640 0x7f6e2400bb40 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 rx=0x7f6e300068c0 tx=0x7f6e3000b300).stop 2021-02-10T11:45:04.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.732 7f6e3ed64700 1 -- 172.21.2.101:0/572402445 shutdown_connections 2021-02-10T11:45:04.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.732 7f6e3ed64700 1 --2- 172.21.2.101:0/572402445 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e24009640 0x7f6e2400bb40 unknown :-1 s=CLOSED pgs=2784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:04.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.732 7f6e3ed64700 1 -- 172.21.2.101:0/572402445 >> 172.21.2.101:0/572402445 conn(0x7f6e380ce410 msgr2=0x7f6e380d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:04.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.732 7f6e3ed64700 1 -- 172.21.2.101:0/572402445 shutdown_connections 2021-02-10T11:45:04.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.732 7f6e3ed64700 1 -- 172.21.2.101:0/572402445 wait complete. 2021-02-10T11:45:04.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.747 7f6e3ed64700 1 Processor -- start 2021-02-10T11:45:04.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.747 7f6e3ed64700 1 -- start start 2021-02-10T11:45:04.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.748 7f6e3ed64700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e380badd0 0x7f6e3809eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:04.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.748 7f6e3ed64700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e380adf50 0x7f6e3809f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:04.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.748 7f6e3ed64700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e381a7470 0x7f6e381a7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:04.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.748 7f6e3ed64700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3809e5a0 con 0x7f6e380adf50 2021-02-10T11:45:04.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.748 7f6e3ed64700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3809f6e0 con 0x7f6e381a7470 2021-02-10T11:45:04.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.748 7f6e3ed64700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e380bb220 con 0x7f6e380badd0 2021-02-10T11:45:04.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.748 7f6e37fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e380adf50 0x7f6e3809f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:04.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.748 7f6e37fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e380adf50 0x7f6e3809f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60366/0 (socket says 172.21.2.101:60366) 2021-02-10T11:45:04.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.748 7f6e37fff700 1 -- 172.21.2.101:0/2268989557 learned_addr learned my addr 172.21.2.101:0/2268989557 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:04.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.748 7f6e37fff700 1 -- 172.21.2.101:0/2268989557 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e380badd0 msgr2=0x7f6e3809eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:04.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.748 7f6e37fff700 1 --2- 172.21.2.101:0/2268989557 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e380badd0 0x7f6e3809eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:04.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.748 7f6e37fff700 1 -- 172.21.2.101:0/2268989557 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e381a7470 msgr2=0x7f6e381a7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:04.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.748 7f6e37fff700 1 --2- 172.21.2.101:0/2268989557 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e381a7470 0x7f6e381a7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:04.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.748 7f6e37fff700 1 -- 172.21.2.101:0/2268989557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e381a7e90 con 0x7f6e380adf50 2021-02-10T11:45:04.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.748 7f6e37fff700 1 --2- 172.21.2.101:0/2268989557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e380adf50 0x7f6e3809f0c0 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 rx=0x7f6e28001660 tx=0x7f6e2800a930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:04.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.748 7f6e357fa700 1 -- 172.21.2.101:0/2268989557 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e2800ad20 con 0x7f6e380adf50 2021-02-10T11:45:04.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.749 7f6e357fa700 1 -- 172.21.2.101:0/2268989557 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e280016a0 con 0x7f6e380adf50 2021-02-10T11:45:04.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.749 7f6e357fa700 1 -- 172.21.2.101:0/2268989557 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e28004330 con 0x7f6e380adf50 2021-02-10T11:45:04.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.749 7f6e3ed64700 1 -- 172.21.2.101:0/2268989557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e381a7e90 con 0x7f6e380adf50 2021-02-10T11:45:04.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.749 7f6e3ed64700 1 -- 172.21.2.101:0/2268989557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e3809d1f0 con 0x7f6e380adf50 2021-02-10T11:45:04.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.749 7f6e357fa700 1 -- 172.21.2.101:0/2268989557 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6e28004140 con 0x7f6e380adf50 2021-02-10T11:45:04.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.749 7f6e357fa700 1 --2- 172.21.2.101:0/2268989557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e1c0186c0 0x7f6e1c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:04.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.749 7f6e357fa700 1 -- 172.21.2.101:0/2268989557 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6e28032520 con 0x7f6e380adf50 2021-02-10T11:45:04.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.749 7f6e3cb80700 1 --2- 172.21.2.101:0/2268989557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e1c0186c0 0x7f6e1c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:04.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.750 7f6e3cb80700 1 --2- 172.21.2.101:0/2268989557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e1c0186c0 0x7f6e1c01abc0 crc :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:04.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.780 7f6e3ed64700 1 -- 172.21.2.101:0/2268989557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e38082a20 con 0x7f6e380adf50 2021-02-10T11:45:04.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.782 7f6e357fa700 1 -- 172.21.2.101:0/2268989557 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6e28000960 con 0x7f6e380adf50 2021-02-10T11:45:04.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.911 7f6e3ed64700 1 -- 172.21.2.101:0/2268989557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f6e3809e520 con 0x7f6e380adf50 2021-02-10T11:45:04.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.912 7f6e357fa700 1 -- 172.21.2.101:0/2268989557 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v156) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6e2800ad20 con 0x7f6e380adf50 2021-02-10T11:45:04.911 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:45:04.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.914 7f6e3ed64700 1 -- 172.21.2.101:0/2268989557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e1c0186c0 msgr2=0x7f6e1c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:04.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.914 7f6e3ed64700 1 --2- 172.21.2.101:0/2268989557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e1c0186c0 0x7f6e1c01abc0 crc :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:04.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.914 7f6e3ed64700 1 -- 172.21.2.101:0/2268989557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e380adf50 msgr2=0x7f6e3809f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:04.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.914 7f6e3ed64700 1 --2- 172.21.2.101:0/2268989557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e380adf50 0x7f6e3809f0c0 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 rx=0x7f6e28001660 tx=0x7f6e2800a930).stop 2021-02-10T11:45:04.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.914 7f6e3ed64700 1 -- 172.21.2.101:0/2268989557 shutdown_connections 2021-02-10T11:45:04.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.914 7f6e3ed64700 1 --2- 172.21.2.101:0/2268989557 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6e381a7470 0x7f6e381a7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:04.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.914 7f6e3ed64700 1 --2- 172.21.2.101:0/2268989557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6e380adf50 0x7f6e3809f0c0 unknown :-1 s=CLOSED pgs=2785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:04.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.914 7f6e3ed64700 1 --2- 172.21.2.101:0/2268989557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6e1c0186c0 0x7f6e1c01abc0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:04.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.914 7f6e3ed64700 1 --2- 172.21.2.101:0/2268989557 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6e380badd0 0x7f6e3809eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:04.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.914 7f6e3ed64700 1 -- 172.21.2.101:0/2268989557 >> 172.21.2.101:0/2268989557 conn(0x7f6e380bee90 msgr2=0x7f6e380bb880 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:04.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.914 7f6e3ed64700 1 -- 172.21.2.101:0/2268989557 shutdown_connections 2021-02-10T11:45:04.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:04.914 7f6e3ed64700 1 -- 172.21.2.101:0/2268989557 wait complete. 2021-02-10T11:45:04.921 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:45:05.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.340 7f7ad63b3700 1 Processor -- start 2021-02-10T11:45:05.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.340 7f7ad63b3700 1 -- start start 2021-02-10T11:45:05.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.340 7f7ad63b3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ad010aff0 0x7f7ad00cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:05.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.340 7f7ad63b3700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad00fe1c0 con 0x7f7ad00d1c70 2021-02-10T11:45:05.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.340 7f7ad63b3700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad01b0cb0 con 0x7f7ad00d1870 2021-02-10T11:45:05.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.340 7f7ad63b3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad0109a60 con 0x7f7ad010aff0 2021-02-10T11:45:05.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.340 7f7acf7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f7ad00d1c70 0x7f7ad00cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60868/0 (socket says 172.21.2.101:60868) 2021-02-10T11:45:05.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.340 7f7acf7fe700 1 -- 172.21.2.101:0/978459604 learned_addr learned my addr 172.21.2.101:0/978459604 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:05.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7aceffd700 1 -- 172.21.2.101:0/978459604 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3177272783 0 0) 0x7f7ac40010b0 con 0x7f7ad00d1c70 2021-02-10T11:45:05.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7aceffd700 1 -- 172.21.2.101:0/978459604 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ab8002fe0 con 0x7f7ad00d1c70 2021-02-10T11:45:05.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7ad49d0700 1 --2- 172.21.2.101:0/978459604 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ad010aff0 0x7f7ad00cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:05.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7aceffd700 1 -- 172.21.2.101:0/978459604 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4051482132 0 0) 0x7f7ac40010b0 con 0x7f7ad00d1c70 2021-02-10T11:45:05.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7aceffd700 1 -- 172.21.2.101:0/978459604 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ad010aff0 msgr2=0x7f7ad00cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:05.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7aceffd700 1 --2- 172.21.2.101:0/978459604 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ad010aff0 0x7f7ad00cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:05.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7aceffd700 1 -- 172.21.2.101:0/978459604 >> v1:172.21.2.108:6789/0 conn(0x7f7ad00d1870 legacy=0x7f7ad01b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:05.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7aceffd700 1 -- 172.21.2.101:0/978459604 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad010b3f0 con 0x7f7ad00d1c70 2021-02-10T11:45:05.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7aceffd700 1 -- 172.21.2.101:0/978459604 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f7ac40015b0 con 0x7f7ad00d1c70 2021-02-10T11:45:05.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7aceffd700 1 -- 172.21.2.101:0/978459604 >> v1:172.21.2.101:6789/0 conn(0x7f7ad00d1c70 legacy=0x7f7ad00cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:05.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7aceffd700 1 --2- 172.21.2.101:0/978459604 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ab8003a60 0x7f7ab8005f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:05.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7aceffd700 1 --2- 172.21.2.101:0/978459604 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ab80067b0 0x7f7ab8008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:05.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7aceffd700 1 --2- 172.21.2.101:0/978459604 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ab8009590 0x7f7ab800ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:05.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7aceffd700 1 -- 172.21.2.101:0/978459604 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab800c2d0 con 0x7f7ab80067b0 2021-02-10T11:45:05.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7aceffd700 1 -- 172.21.2.101:0/978459604 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab800c4f0 con 0x7f7ab8003a60 2021-02-10T11:45:05.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7aceffd700 1 -- 172.21.2.101:0/978459604 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab800c710 con 0x7f7ab8009590 2021-02-10T11:45:05.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7acf7fe700 1 --2- 172.21.2.101:0/978459604 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ab80067b0 0x7f7ab8008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:05.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7acffff700 1 --2- 172.21.2.101:0/978459604 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ab8003a60 0x7f7ab8005f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:05.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7acf7fe700 1 -- 172.21.2.101:0/978459604 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ab8009590 msgr2=0x7f7ab800ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:05.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7acf7fe700 1 --2- 172.21.2.101:0/978459604 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ab8009590 0x7f7ab800ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:05.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7acf7fe700 1 -- 172.21.2.101:0/978459604 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ab8003a60 msgr2=0x7f7ab8005f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:05.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7acf7fe700 1 --2- 172.21.2.101:0/978459604 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ab8003a60 0x7f7ab8005f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:05.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7acf7fe700 1 -- 172.21.2.101:0/978459604 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab8050980 con 0x7f7ab80067b0 2021-02-10T11:45:05.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7acf7fe700 1 --2- 172.21.2.101:0/978459604 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ab80067b0 0x7f7ab8008c90 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 rx=0x7f7ac4007990 tx=0x7f7ac4007d30).ready entity=mon.0 client_cookie=e8f52b274bf72f5d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:05.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.341 7f7aceffd700 1 -- 172.21.2.101:0/978459604 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ac40009b0 con 0x7f7ab80067b0 2021-02-10T11:45:05.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.342 7f7aceffd700 1 -- 172.21.2.101:0/978459604 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac40070c0 con 0x7f7ab80067b0 2021-02-10T11:45:05.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.342 7f7aceffd700 1 -- 172.21.2.101:0/978459604 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ac400a310 con 0x7f7ab80067b0 2021-02-10T11:45:05.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.342 7f7ad63b3700 1 -- 172.21.2.101:0/978459604 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ab80067b0 msgr2=0x7f7ab8008c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:05.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.342 7f7ad63b3700 1 --2- 172.21.2.101:0/978459604 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ab80067b0 0x7f7ab8008c90 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 rx=0x7f7ac4007990 tx=0x7f7ac4007d30).stop 2021-02-10T11:45:05.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.342 7f7ad49d0700 1 -- 172.21.2.101:0/978459604 reap_dead start 2021-02-10T11:45:05.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.342 7f7ad63b3700 1 -- 172.21.2.101:0/978459604 shutdown_connections 2021-02-10T11:45:05.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.342 7f7ad63b3700 1 -- 172.21.2.101:0/978459604 >> 172.21.2.101:0/978459604 conn(0x7f7ad00ce420 msgr2=0x7f7ad00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:05.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.342 7f7ad63b3700 1 -- 172.21.2.101:0/978459604 shutdown_connections 2021-02-10T11:45:05.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.342 7f7ad63b3700 1 -- 172.21.2.101:0/978459604 wait complete. 2021-02-10T11:45:05.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.357 7f7ad63b3700 1 Processor -- start 2021-02-10T11:45:05.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.357 7f7ad63b3700 1 -- start start 2021-02-10T11:45:05.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.357 7f7ad63b3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ad0085bc0 0x7f7ad00863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:05.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.357 7f7ad63b3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ad00954c0 0x7f7ad0096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:05.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.357 7f7ad63b3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ad00958c0 0x7f7ad0096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:05.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.357 7f7ad63b3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad0095d80 con 0x7f7ad00958c0 2021-02-10T11:45:05.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.357 7f7ad63b3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad00bfa60 con 0x7f7ad00954c0 2021-02-10T11:45:05.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.357 7f7ad63b3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad0085fc0 con 0x7f7ad0085bc0 2021-02-10T11:45:05.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.358 7f7ad49d0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ad00958c0 0x7f7ad0096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:05.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.358 7f7ad49d0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ad00958c0 0x7f7ad0096920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60386/0 (socket says 172.21.2.101:60386) 2021-02-10T11:45:05.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.358 7f7ad49d0700 1 -- 172.21.2.101:0/1148467219 learned_addr learned my addr 172.21.2.101:0/1148467219 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:05.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.358 7f7ad49d0700 1 -- 172.21.2.101:0/1148467219 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ad0085bc0 msgr2=0x7f7ad00863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:05.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.358 7f7ad49d0700 1 --2- 172.21.2.101:0/1148467219 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ad0085bc0 0x7f7ad00863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:05.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.358 7f7ad49d0700 1 -- 172.21.2.101:0/1148467219 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ad00954c0 msgr2=0x7f7ad0096300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:05.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.358 7f7ad49d0700 1 --2- 172.21.2.101:0/1148467219 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ad00954c0 0x7f7ad0096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:05.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.358 7f7ad49d0700 1 -- 172.21.2.101:0/1148467219 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad0086c10 con 0x7f7ad00958c0 2021-02-10T11:45:05.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.358 7f7ad49d0700 1 --2- 172.21.2.101:0/1148467219 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ad00958c0 0x7f7ad0096920 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 rx=0x7f7ac8009960 tx=0x7f7ac800bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:05.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.358 7f7accff9700 1 -- 172.21.2.101:0/1148467219 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ac80011f0 con 0x7f7ad00958c0 2021-02-10T11:45:05.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.358 7f7accff9700 1 -- 172.21.2.101:0/1148467219 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac8003030 con 0x7f7ad00958c0 2021-02-10T11:45:05.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.358 7f7ad63b3700 1 -- 172.21.2.101:0/1148467219 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad0086c10 con 0x7f7ad00958c0 2021-02-10T11:45:05.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.358 7f7ad63b3700 1 -- 172.21.2.101:0/1148467219 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad0086c10 con 0x7f7ad00958c0 2021-02-10T11:45:05.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.359 7f7accff9700 1 -- 172.21.2.101:0/1148467219 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ac8001770 con 0x7f7ad00958c0 2021-02-10T11:45:05.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.359 7f7accff9700 1 -- 172.21.2.101:0/1148467219 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7ac8001470 con 0x7f7ad00958c0 2021-02-10T11:45:05.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.359 7f7accff9700 1 --2- 172.21.2.101:0/1148467219 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ab0018730 0x7f7ab001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:05.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.359 7f7accff9700 1 -- 172.21.2.101:0/1148467219 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7ac8032b70 con 0x7f7ad00958c0 2021-02-10T11:45:05.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.359 7f7acffff700 1 --2- 172.21.2.101:0/1148467219 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ab0018730 0x7f7ab001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:05.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.360 7f7acffff700 1 --2- 172.21.2.101:0/1148467219 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ab0018730 0x7f7ab001ac10 crc :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:05.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.390 7f7ad63b3700 1 -- 172.21.2.101:0/1148467219 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ad00fe1c0 con 0x7f7ad00958c0 2021-02-10T11:45:05.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.392 7f7accff9700 1 -- 172.21.2.101:0/1148467219 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7ac8000f60 con 0x7f7ad00958c0 2021-02-10T11:45:05.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.521 7f7ad63b3700 1 -- 172.21.2.101:0/1148467219 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f7ad00fe1c0 con 0x7f7ad00958c0 2021-02-10T11:45:05.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.522 7f7accff9700 1 -- 172.21.2.101:0/1148467219 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v156) v1 ==== 104+0+10 (secure 0 0 0) 0x7f7ac8001530 con 0x7f7ad00958c0 2021-02-10T11:45:05.521 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:45:05.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.524 7f7ad63b3700 1 -- 172.21.2.101:0/1148467219 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ab0018730 msgr2=0x7f7ab001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:05.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.524 7f7ad63b3700 1 --2- 172.21.2.101:0/1148467219 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ab0018730 0x7f7ab001ac10 crc :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:05.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.524 7f7ad63b3700 1 -- 172.21.2.101:0/1148467219 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ad00958c0 msgr2=0x7f7ad0096920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:05.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.524 7f7ad63b3700 1 --2- 172.21.2.101:0/1148467219 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ad00958c0 0x7f7ad0096920 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 rx=0x7f7ac8009960 tx=0x7f7ac800bd40).stop 2021-02-10T11:45:05.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.524 7f7ad63b3700 1 -- 172.21.2.101:0/1148467219 shutdown_connections 2021-02-10T11:45:05.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.524 7f7ad63b3700 1 --2- 172.21.2.101:0/1148467219 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ad00958c0 0x7f7ad0096920 unknown :-1 s=CLOSED pgs=2788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:05.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.524 7f7ad63b3700 1 --2- 172.21.2.101:0/1148467219 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ad00954c0 0x7f7ad0096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:05.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.524 7f7ad63b3700 1 --2- 172.21.2.101:0/1148467219 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ab0018730 0x7f7ab001ac10 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:05.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.524 7f7ad63b3700 1 --2- 172.21.2.101:0/1148467219 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ad0085bc0 0x7f7ad00863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:05.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.524 7f7ad63b3700 1 -- 172.21.2.101:0/1148467219 >> 172.21.2.101:0/1148467219 conn(0x7f7ad00bef00 msgr2=0x7f7ad009d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:05.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.524 7f7ad63b3700 1 -- 172.21.2.101:0/1148467219 shutdown_connections 2021-02-10T11:45:05.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.524 7f7ad63b3700 1 -- 172.21.2.101:0/1148467219 wait complete. 2021-02-10T11:45:05.531 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:45:05.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.948 7f2ab73ac700 1 Processor -- start 2021-02-10T11:45:05.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.948 7f2ab73ac700 1 -- start start 2021-02-10T11:45:05.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.948 7f2ab73ac700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2ab00d1c70 0x7f2ab00cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:05.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.948 7f2ab73ac700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab00d0170 con 0x7f2ab00cf580 2021-02-10T11:45:05.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.948 7f2ab73ac700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab00fe1c0 con 0x7f2ab00d1870 2021-02-10T11:45:05.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.948 7f2ab73ac700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab01b0cb0 con 0x7f2ab00d1c70 2021-02-10T11:45:05.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.948 7f2ab49c7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2ab00d1c70 0x7f2ab00cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:05.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.948 7f2ab49c7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2ab00d1c70 0x7f2ab00cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60392/0 (socket says 172.21.2.101:60392) 2021-02-10T11:45:05.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.948 7f2ab49c7700 1 -- 172.21.2.101:0/210551002 learned_addr learned my addr 172.21.2.101:0/210551002 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:05.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.949 7f2ab49c7700 1 -- 172.21.2.101:0/210551002 >> v1:172.21.2.108:6789/0 conn(0x7f2ab00d1870 legacy=0x7f2ab01b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:05.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.949 7f2ab49c7700 1 -- 172.21.2.101:0/210551002 >> v1:172.21.2.101:6789/0 conn(0x7f2ab00cf580 legacy=0x7f2ab00cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:05.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.949 7f2ab49c7700 1 -- 172.21.2.101:0/210551002 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab0109a60 con 0x7f2ab00d1c70 2021-02-10T11:45:05.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.949 7f2ab49c7700 1 --2- 172.21.2.101:0/210551002 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ab00d1c70 0x7f2ab00cef60 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 rx=0x7f2aa4006700 tx=0x7f2aa4009b80).ready entity=mon.0 client_cookie=14ba3360c61dd3f4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:05.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.949 7f2aa3fff700 1 -- 172.21.2.101:0/210551002 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2aa400c0c0 con 0x7f2ab00d1c70 2021-02-10T11:45:05.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.949 7f2aa3fff700 1 -- 172.21.2.101:0/210551002 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa400c340 con 0x7f2ab00d1c70 2021-02-10T11:45:05.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.949 7f2aa3fff700 1 -- 172.21.2.101:0/210551002 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2aa400c860 con 0x7f2ab00d1c70 2021-02-10T11:45:05.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.949 7f2ab73ac700 1 -- 172.21.2.101:0/210551002 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ab00d1c70 msgr2=0x7f2ab00cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:05.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.949 7f2ab73ac700 1 --2- 172.21.2.101:0/210551002 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ab00d1c70 0x7f2ab00cef60 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 rx=0x7f2aa4006700 tx=0x7f2aa4009b80).stop 2021-02-10T11:45:05.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.950 7f2ab73ac700 1 -- 172.21.2.101:0/210551002 shutdown_connections 2021-02-10T11:45:05.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.950 7f2ab73ac700 1 --2- 172.21.2.101:0/210551002 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ab00d1c70 0x7f2ab00cef60 unknown :-1 s=CLOSED pgs=2790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:05.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.950 7f2ab73ac700 1 -- 172.21.2.101:0/210551002 >> 172.21.2.101:0/210551002 conn(0x7f2ab00ce420 msgr2=0x7f2ab00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:05.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.950 7f2ab73ac700 1 -- 172.21.2.101:0/210551002 shutdown_connections 2021-02-10T11:45:05.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.950 7f2ab73ac700 1 -- 172.21.2.101:0/210551002 wait complete. 2021-02-10T11:45:05.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.965 7f2ab73ac700 1 Processor -- start 2021-02-10T11:45:05.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.965 7f2ab73ac700 1 -- start start 2021-02-10T11:45:05.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.965 7f2ab73ac700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ab00c0740 0x7f2ab01c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:05.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.965 7f2ab73ac700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ab01c5ba0 0x7f2ab00a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:05.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.965 7f2ab73ac700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ab00b6e60 0x7f2ab00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:05.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.965 7f2ab73ac700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab00a6880 con 0x7f2ab00c0740 2021-02-10T11:45:05.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.965 7f2ab73ac700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab00a6aa0 con 0x7f2ab01c5ba0 2021-02-10T11:45:05.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.965 7f2ab73ac700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab00a6cc0 con 0x7f2ab00b6e60 2021-02-10T11:45:05.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.965 7f2ab51c8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ab00c0740 0x7f2ab01c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:05.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.965 7f2ab51c8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ab00c0740 0x7f2ab01c5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60400/0 (socket says 172.21.2.101:60400) 2021-02-10T11:45:05.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.965 7f2ab51c8700 1 -- 172.21.2.101:0/2514843279 learned_addr learned my addr 172.21.2.101:0/2514843279 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:05.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.966 7f2ab51c8700 1 -- 172.21.2.101:0/2514843279 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ab00b6e60 msgr2=0x7f2ab00a6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:05.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.966 7f2ab51c8700 1 --2- 172.21.2.101:0/2514843279 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ab00b6e60 0x7f2ab00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:05.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.966 7f2ab51c8700 1 -- 172.21.2.101:0/2514843279 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ab01c5ba0 msgr2=0x7f2ab00a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:05.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.966 7f2ab51c8700 1 --2- 172.21.2.101:0/2514843279 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ab01c5ba0 0x7f2ab00a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:05.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.966 7f2ab51c8700 1 -- 172.21.2.101:0/2514843279 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab00a6ee0 con 0x7f2ab00c0740 2021-02-10T11:45:05.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.966 7f2ab51c8700 1 --2- 172.21.2.101:0/2514843279 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ab00c0740 0x7f2ab01c5580 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 rx=0x7f2aac00b5b0 tx=0x7f2aac00b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:05.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.966 7f2aa1ffb700 1 -- 172.21.2.101:0/2514843279 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2aac000960 con 0x7f2ab00c0740 2021-02-10T11:45:05.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.966 7f2aa1ffb700 1 -- 172.21.2.101:0/2514843279 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aac00bdd0 con 0x7f2ab00c0740 2021-02-10T11:45:05.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.966 7f2aa1ffb700 1 -- 172.21.2.101:0/2514843279 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2aac0054a0 con 0x7f2ab00c0740 2021-02-10T11:45:05.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.966 7f2ab73ac700 1 -- 172.21.2.101:0/2514843279 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab00a6ee0 con 0x7f2ab00c0740 2021-02-10T11:45:05.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.966 7f2ab73ac700 1 -- 172.21.2.101:0/2514843279 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab00b5650 con 0x7f2ab00c0740 2021-02-10T11:45:05.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.967 7f2aa1ffb700 1 -- 172.21.2.101:0/2514843279 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2aac006030 con 0x7f2ab00c0740 2021-02-10T11:45:05.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.967 7f2aa1ffb700 1 --2- 172.21.2.101:0/2514843279 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a90018730 0x7f2a9001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:05.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.967 7f2aa1ffb700 1 -- 172.21.2.101:0/2514843279 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2aac032620 con 0x7f2ab00c0740 2021-02-10T11:45:05.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.967 7f2ab49c7700 1 --2- 172.21.2.101:0/2514843279 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a90018730 0x7f2a9001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:05.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.967 7f2ab49c7700 1 --2- 172.21.2.101:0/2514843279 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a90018730 0x7f2a9001abc0 crc :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:05.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:05.998 7f2ab73ac700 1 -- 172.21.2.101:0/2514843279 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab00807d0 con 0x7f2ab00c0740 2021-02-10T11:45:05.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.000 7f2aa1ffb700 1 -- 172.21.2.101:0/2514843279 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2aac000c60 con 0x7f2ab00c0740 2021-02-10T11:45:06.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.130 7f2ab73ac700 1 -- 172.21.2.101:0/2514843279 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f2ab00807d0 con 0x7f2ab00c0740 2021-02-10T11:45:06.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.130 7f2aa1ffb700 1 -- 172.21.2.101:0/2514843279 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v156) v1 ==== 100+0+10 (secure 0 0 0) 0x7f2aac00bd40 con 0x7f2ab00c0740 2021-02-10T11:45:06.129 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:45:06.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.132 7f2ab73ac700 1 -- 172.21.2.101:0/2514843279 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a90018730 msgr2=0x7f2a9001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:06.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.132 7f2ab73ac700 1 --2- 172.21.2.101:0/2514843279 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a90018730 0x7f2a9001abc0 crc :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:06.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.132 7f2ab73ac700 1 -- 172.21.2.101:0/2514843279 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ab00c0740 msgr2=0x7f2ab01c5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:06.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.132 7f2ab73ac700 1 --2- 172.21.2.101:0/2514843279 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ab00c0740 0x7f2ab01c5580 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 rx=0x7f2aac00b5b0 tx=0x7f2aac00b950).stop 2021-02-10T11:45:06.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.133 7f2ab73ac700 1 -- 172.21.2.101:0/2514843279 shutdown_connections 2021-02-10T11:45:06.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.133 7f2ab73ac700 1 --2- 172.21.2.101:0/2514843279 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a90018730 0x7f2a9001abc0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:06.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.133 7f2ab73ac700 1 --2- 172.21.2.101:0/2514843279 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ab00b6e60 0x7f2ab00a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:06.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.133 7f2ab73ac700 1 --2- 172.21.2.101:0/2514843279 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ab01c5ba0 0x7f2ab00a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:06.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.133 7f2ab73ac700 1 --2- 172.21.2.101:0/2514843279 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ab00c0740 0x7f2ab01c5580 unknown :-1 s=CLOSED pgs=2791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:06.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.133 7f2ab73ac700 1 -- 172.21.2.101:0/2514843279 >> 172.21.2.101:0/2514843279 conn(0x7f2ab00c4120 msgr2=0x7f2ab00ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:06.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.133 7f2ab73ac700 1 -- 172.21.2.101:0/2514843279 shutdown_connections 2021-02-10T11:45:06.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.133 7f2ab73ac700 1 -- 172.21.2.101:0/2514843279 wait complete. 2021-02-10T11:45:06.140 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:45:06.140 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:45:06.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.554 7f25e8ae3700 1 Processor -- start 2021-02-10T11:45:06.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.554 7f25e8ae3700 1 -- start start 2021-02-10T11:45:06.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.555 7f25e8ae3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f25e40d0340 0x7f25e40ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:06.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.555 7f25e8ae3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f25e40d1c70 0x7f25e40efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:06.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.555 7f25e8ae3700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e40f0b10 con 0x7f25e40d1870 2021-02-10T11:45:06.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.555 7f25e8ae3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e40e5ed0 con 0x7f25e40d1c70 2021-02-10T11:45:06.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.555 7f25e8ae3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e41b0be0 con 0x7f25e40d0340 2021-02-10T11:45:06.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.555 7f25e305f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f25e40d1c70 0x7f25e40efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:06.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.555 7f25e305f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f25e40d1c70 0x7f25e40efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60410/0 (socket says 172.21.2.101:60410) 2021-02-10T11:45:06.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.555 7f25e305f700 1 -- 172.21.2.101:0/1182311160 learned_addr learned my addr 172.21.2.101:0/1182311160 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:06.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.555 7f25e305f700 1 -- 172.21.2.101:0/1182311160 >> v2:172.21.2.108:3300/0 conn(0x7f25e40d0340 msgr2=0x7f25e40ef640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:45:06.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.555 7f25e305f700 1 --2- 172.21.2.101:0/1182311160 >> v2:172.21.2.108:3300/0 conn(0x7f25e40d0340 0x7f25e40ef640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:06.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.555 7f25e305f700 1 -- 172.21.2.101:0/1182311160 >> v1:172.21.2.101:6789/0 conn(0x7f25e40d1870 legacy=0x7f25e41b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:06.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.555 7f25e305f700 1 -- 172.21.2.101:0/1182311160 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e40f20a0 con 0x7f25e40d1c70 2021-02-10T11:45:06.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.555 7f25e305f700 1 --2- 172.21.2.101:0/1182311160 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25e40d1c70 0x7f25e40efc60 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 rx=0x7f25dc006f00 tx=0x7f25dc009f10).ready entity=mon.0 client_cookie=7bc15b1eb786fb26 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:06.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.555 7f25e185c700 1 -- 172.21.2.101:0/1182311160 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f25dc00b3b0 con 0x7f25e40d1c70 2021-02-10T11:45:06.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.555 7f25e185c700 1 -- 172.21.2.101:0/1182311160 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25dc00b730 con 0x7f25e40d1c70 2021-02-10T11:45:06.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.556 7f25e185c700 1 -- 172.21.2.101:0/1182311160 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f25dc00bc10 con 0x7f25e40d1c70 2021-02-10T11:45:06.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.556 7f25e8ae3700 1 -- 172.21.2.101:0/1182311160 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25e40d1c70 msgr2=0x7f25e40efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:06.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.556 7f25e8ae3700 1 --2- 172.21.2.101:0/1182311160 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25e40d1c70 0x7f25e40efc60 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 rx=0x7f25dc006f00 tx=0x7f25dc009f10).stop 2021-02-10T11:45:06.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.556 7f25e8ae3700 1 -- 172.21.2.101:0/1182311160 shutdown_connections 2021-02-10T11:45:06.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.556 7f25e8ae3700 1 --2- 172.21.2.101:0/1182311160 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25e40d1c70 0x7f25e40efc60 unknown :-1 s=CLOSED pgs=2793 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:06.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.556 7f25e8ae3700 1 --2- 172.21.2.101:0/1182311160 >> v2:172.21.2.108:3300/0 conn(0x7f25e40d0340 0x7f25e40ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:06.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.556 7f25e8ae3700 1 -- 172.21.2.101:0/1182311160 >> 172.21.2.101:0/1182311160 conn(0x7f25e40ce450 msgr2=0x7f25e40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:06.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.556 7f25e8ae3700 1 -- 172.21.2.101:0/1182311160 shutdown_connections 2021-02-10T11:45:06.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.556 7f25e8ae3700 1 -- 172.21.2.101:0/1182311160 wait complete. 2021-02-10T11:45:06.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.571 7f25e8ae3700 1 Processor -- start 2021-02-10T11:45:06.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.571 7f25e8ae3700 1 -- start start 2021-02-10T11:45:06.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.571 7f25e8ae3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25e418f0a0 0x7f25e4196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:06.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.571 7f25e8ae3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25e41970d0 0x7f25e41974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:06.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.571 7f25e8ae3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25e4094650 0x7f25e4094a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:06.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.571 7f25e8ae3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e4095bd0 con 0x7f25e4094650 2021-02-10T11:45:06.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.571 7f25e8ae3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e40ad7c0 con 0x7f25e41970d0 2021-02-10T11:45:06.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.571 7f25e8ae3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e418ec70 con 0x7f25e418f0a0 2021-02-10T11:45:06.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.571 7f25e305f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25e4094650 0x7f25e4094a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:06.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.571 7f25e305f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25e4094650 0x7f25e4094a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60416/0 (socket says 172.21.2.101:60416) 2021-02-10T11:45:06.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.571 7f25e305f700 1 -- 172.21.2.101:0/1401240920 learned_addr learned my addr 172.21.2.101:0/1401240920 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:06.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.572 7f25e305f700 1 -- 172.21.2.101:0/1401240920 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25e418f0a0 msgr2=0x7f25e4196ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:06.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.572 7f25e305f700 1 --2- 172.21.2.101:0/1401240920 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25e418f0a0 0x7f25e4196ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:06.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.572 7f25e305f700 1 -- 172.21.2.101:0/1401240920 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25e41970d0 msgr2=0x7f25e41974d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:06.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.572 7f25e305f700 1 --2- 172.21.2.101:0/1401240920 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25e41970d0 0x7f25e41974d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:06.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.572 7f25e305f700 1 -- 172.21.2.101:0/1401240920 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e4095070 con 0x7f25e4094650 2021-02-10T11:45:06.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.572 7f25e305f700 1 --2- 172.21.2.101:0/1401240920 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25e4094650 0x7f25e4094a50 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 rx=0x7f25dc009390 tx=0x7f25dc00bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:06.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.572 7f25db7fe700 1 -- 172.21.2.101:0/1401240920 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f25dc003030 con 0x7f25e4094650 2021-02-10T11:45:06.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.572 7f25db7fe700 1 -- 172.21.2.101:0/1401240920 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25dc006c80 con 0x7f25e4094650 2021-02-10T11:45:06.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.572 7f25db7fe700 1 -- 172.21.2.101:0/1401240920 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f25dc001480 con 0x7f25e4094650 2021-02-10T11:45:06.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.572 7f25e8ae3700 1 -- 172.21.2.101:0/1401240920 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e4095070 con 0x7f25e4094650 2021-02-10T11:45:06.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.572 7f25e8ae3700 1 -- 172.21.2.101:0/1401240920 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e4095600 con 0x7f25e4094650 2021-02-10T11:45:06.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.573 7f25db7fe700 1 -- 172.21.2.101:0/1401240920 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f25dc001270 con 0x7f25e4094650 2021-02-10T11:45:06.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.573 7f25db7fe700 1 --2- 172.21.2.101:0/1401240920 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25c80186c0 0x7f25c801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:06.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.573 7f25db7fe700 1 -- 172.21.2.101:0/1401240920 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f25dc0328f0 con 0x7f25e4094650 2021-02-10T11:45:06.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.573 7f25e285e700 1 --2- 172.21.2.101:0/1401240920 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25c80186c0 0x7f25c801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:06.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.573 7f25e285e700 1 --2- 172.21.2.101:0/1401240920 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25c80186c0 0x7f25c801abc0 crc :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:06.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.605 7f25e8ae3700 1 -- 172.21.2.101:0/1401240920 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e40ad790 con 0x7f25e4094650 2021-02-10T11:45:06.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.607 7f25db7fe700 1 -- 172.21.2.101:0/1401240920 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f25dc009450 con 0x7f25e4094650 2021-02-10T11:45:06.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.734 7f25e8ae3700 1 -- 172.21.2.101:0/1401240920 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25e40ad790 con 0x7f25e4094650 2021-02-10T11:45:06.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.734 7f25db7fe700 1 -- 172.21.2.101:0/1401240920 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+2105 (secure 0 0 0) 0x7f25dc001270 con 0x7f25e4094650 2021-02-10T11:45:06.733 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 131 2021-02-10T11:45:06.733 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:45:06.734 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":131,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":12135,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/3467749863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3467749863},{"type":"v1","addr":"172.21.2.101:6835","nonce":3467749863}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":128}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:44:56.768458","modified":"2021-02-10 11:44:58.773048","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8242":{"gid":8242,"name":"b","rank":0,"incarnation":129,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/485291097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":485291097},{"type":"v1","addr":"172.21.2.108:6833","nonce":485291097}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2021-02-10T11:45:06.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.736 7f25e8ae3700 1 -- 172.21.2.101:0/1401240920 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25c80186c0 msgr2=0x7f25c801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:06.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.736 7f25e8ae3700 1 --2- 172.21.2.101:0/1401240920 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25c80186c0 0x7f25c801abc0 crc :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:06.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.736 7f25e8ae3700 1 -- 172.21.2.101:0/1401240920 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25e4094650 msgr2=0x7f25e4094a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:06.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.736 7f25e8ae3700 1 --2- 172.21.2.101:0/1401240920 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25e4094650 0x7f25e4094a50 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 rx=0x7f25dc009390 tx=0x7f25dc00bd40).stop 2021-02-10T11:45:06.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.736 7f25e8ae3700 1 -- 172.21.2.101:0/1401240920 shutdown_connections 2021-02-10T11:45:06.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.736 7f25e8ae3700 1 --2- 172.21.2.101:0/1401240920 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25e4094650 0x7f25e4094a50 unknown :-1 s=CLOSED pgs=2794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:06.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.736 7f25e8ae3700 1 --2- 172.21.2.101:0/1401240920 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25e41970d0 0x7f25e41974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:06.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.736 7f25e8ae3700 1 --2- 172.21.2.101:0/1401240920 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25c80186c0 0x7f25c801abc0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:06.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.736 7f25e8ae3700 1 --2- 172.21.2.101:0/1401240920 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25e418f0a0 0x7f25e4196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:06.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.736 7f25e8ae3700 1 -- 172.21.2.101:0/1401240920 >> 172.21.2.101:0/1401240920 conn(0x7f25e41a38c0 msgr2=0x7f25e41a2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:06.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.736 7f25e8ae3700 1 -- 172.21.2.101:0/1401240920 shutdown_connections 2021-02-10T11:45:06.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:06.736 7f25e8ae3700 1 -- 172.21.2.101:0/1401240920 wait complete. 2021-02-10T11:45:06.945 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 131, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:44:56.768458', 'modified': '2021-02-10 11:44:58.773048', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8242}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8242': {'gid': 8242, 'name': 'b', 'rank': 0, 'incarnation': 129, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/485291097', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 485291097}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 485291097}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [33], 'metadata_pool': 32, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:45:06.945 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:45:06.945 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:45:07.342 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:45:07.345 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:45:07.346 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:45:07.346 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:45:07.346 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:45:07.346 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 8242, 2021-02-10T11:45:07.346 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:45:07.346 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:45:07.346 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 10.575901934999999, 2021-02-10T11:45:07.346 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 131, 2021-02-10T11:45:07.346 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 156, 2021-02-10T11:45:07.346 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 156, 2021-02-10T11:45:07.347 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 16.220080048 2021-02-10T11:45:07.347 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:45:07.349 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 8242, "want_state": "up:active", "state": "up:active", "rank_uptime": 10.575901934999999, "mdsmap_epoch": 131, "osdmap_epoch": 156, "osdmap_epoch_barrier": 156, "uptime": 16.220080048 } 2021-02-10T11:45:07.349 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:45:07.349 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-02-10T11:45:07.349 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-02-10T11:45:07.350 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:07.350 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:45:07.677 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:45:08.022 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:45:08.022 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:45:08.366 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:45:08.366 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:45:08.367 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:45:08.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:08.367 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:45:08.702 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-02-10T11:45:08.702 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:45:08.705 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:45:09.050 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:45:09.051 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:45:09.117 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[166055]: starting ceph client 2021-02-10T11:45:09.117 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:2021-02-10 11:45:09.118 7f42ba839f80 -1 init, newargv = 0x562aadfe11a0 newargc=9 2021-02-10T11:45:09.124 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[166055]: starting fuse 2021-02-10T11:45:09.394 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:45:09.394 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:45:09.395 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:45:09.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:09.395 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:45:09.730 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:45:09.731 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2021-02-10T11:45:09.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:09.731 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:45:09.731 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:45:09.731 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:45:09.731 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:45:09.731 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:45:09.732 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:45:09.732 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:45:09.732 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-02-10T11:45:09.732 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:45:09.732 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:45:09.732 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:45:09.732 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:45:09.732 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:45:09.732 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:45:09.732 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:45:09.732 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:45:09.732 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:45:09.732 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:45:09.732 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:45:09.732 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:45:09.733 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:45:09.733 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.0")) 2021-02-10T11:45:09.733 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:45:10.095 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.0.166055.asok 2021-02-10T11:45:10.097 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.166055.asok 2021-02-10T11:45:10.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:10.098 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.166055.asok status 2021-02-10T11:45:10.490 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:45:10.491 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:45:10.491 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:45:10.491 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:45:10.491 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "0", 2021-02-10T11:45:10.491 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:45:10.491 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-02-10T11:45:10.491 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "166055", 2021-02-10T11:45:10.491 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:45:10.491 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:10.491 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:45:10.491 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:45:10.492 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 12306, 2021-02-10T11:45:10.492 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:45:10.492 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:45:10.492 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:45:10.492 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 12306 2021-02-10T11:45:10.492 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:10.492 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:45:10.492 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:10.492 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:45:10.492 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1232084342 2021-02-10T11:45:10.492 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:10.492 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:10.492 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:45:10.492 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:10.493 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:45:10.493 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1232084342 2021-02-10T11:45:10.493 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:10.493 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.12306 172.21.2.101:0/1232084342", 2021-02-10T11:45:10.493 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/1232084342", 2021-02-10T11:45:10.493 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:45:10.493 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 131, 2021-02-10T11:45:10.493 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 156, 2021-02-10T11:45:10.493 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:45:10.493 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:45:10.493 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:45:10.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:10.495 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:45:10.495 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:45:10.830 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:45:10.830 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:45:10.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:10.830 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:45:11.177 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:45:11.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.591 7f89a0528700 1 Processor -- start 2021-02-10T11:45:11.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.591 7f89a0528700 1 -- start start 2021-02-10T11:45:11.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.592 7f89a0528700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f89980d0340 0x7f89980ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:11.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.592 7f89a0528700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f89980d1c70 0x7f89980efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:11.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.592 7f89a0528700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89980f0b10 con 0x7f89980d1870 2021-02-10T11:45:11.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.592 7f89a0528700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89980e5ed0 con 0x7f89980d0340 2021-02-10T11:45:11.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.592 7f89a0528700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89981b0be0 con 0x7f89980d1c70 2021-02-10T11:45:11.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.592 7f899db43700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f89980d0340 0x7f89980ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:11.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.592 7f899e344700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f89980d1870 0x7f89981b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60928/0 (socket says 172.21.2.101:60928) 2021-02-10T11:45:11.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.592 7f899e344700 1 -- 172.21.2.101:0/2273927301 learned_addr learned my addr 172.21.2.101:0/2273927301 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:11.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.593 7f899db43700 1 -- 172.21.2.101:0/2273927301 >> v2:172.21.2.108:3300/0 conn(0x7f89980d1c70 msgr2=0x7f89980efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:11.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.593 7f899eb45700 1 --2- 172.21.2.101:0/2273927301 >> v2:172.21.2.108:3300/0 conn(0x7f89980d1c70 0x7f89980efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:11.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.593 7f899db43700 1 --2- 172.21.2.101:0/2273927301 >> v2:172.21.2.108:3300/0 conn(0x7f89980d1c70 0x7f89980efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:11.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.593 7f899db43700 1 -- 172.21.2.101:0/2273927301 >> v1:172.21.2.101:6789/0 conn(0x7f89980d1870 legacy=0x7f89981b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:11.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.593 7f899db43700 1 -- 172.21.2.101:0/2273927301 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89980f20a0 con 0x7f89980d0340 2021-02-10T11:45:11.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.593 7f899eb45700 1 --2- 172.21.2.101:0/2273927301 >> v2:172.21.2.108:3300/0 conn(0x7f89980d1c70 0x7f89980efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:11.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.593 7f899db43700 1 --2- 172.21.2.101:0/2273927301 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89980d0340 0x7f89980ef640 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 rx=0x7f8988006650 tx=0x7f8988009a80).ready entity=mon.0 client_cookie=a5b3eda7f8a261b9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:11.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.593 7f899d342700 1 -- 172.21.2.101:0/2273927301 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f89880009b0 con 0x7f89980d0340 2021-02-10T11:45:11.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.593 7f899d342700 1 -- 172.21.2.101:0/2273927301 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898800c0c0 con 0x7f89980d0340 2021-02-10T11:45:11.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.593 7f899d342700 1 -- 172.21.2.101:0/2273927301 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f898800c5e0 con 0x7f89980d0340 2021-02-10T11:45:11.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.593 7f89a0528700 1 -- 172.21.2.101:0/2273927301 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89980d0340 msgr2=0x7f89980ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:11.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.593 7f89a0528700 1 --2- 172.21.2.101:0/2273927301 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89980d0340 0x7f89980ef640 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 rx=0x7f8988006650 tx=0x7f8988009a80).stop 2021-02-10T11:45:11.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.594 7f89a0528700 1 -- 172.21.2.101:0/2273927301 shutdown_connections 2021-02-10T11:45:11.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.594 7f89a0528700 1 --2- 172.21.2.101:0/2273927301 >> v2:172.21.2.108:3300/0 conn(0x7f89980d1c70 0x7f89980efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:11.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.594 7f89a0528700 1 --2- 172.21.2.101:0/2273927301 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89980d0340 0x7f89980ef640 unknown :-1 s=CLOSED pgs=2799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:11.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.594 7f89a0528700 1 -- 172.21.2.101:0/2273927301 >> 172.21.2.101:0/2273927301 conn(0x7f89980ce450 msgr2=0x7f89980d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:11.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.594 7f89a0528700 1 -- 172.21.2.101:0/2273927301 shutdown_connections 2021-02-10T11:45:11.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.594 7f89a0528700 1 -- 172.21.2.101:0/2273927301 wait complete. 2021-02-10T11:45:11.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.609 7f89a0528700 1 Processor -- start 2021-02-10T11:45:11.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.609 7f89a0528700 1 -- start start 2021-02-10T11:45:11.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.609 7f89a0528700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89981952e0 0x7f899818ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:11.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.609 7f89a0528700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89981b8bc0 0x7f89981ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:11.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.609 7f89a0528700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89981acae0 0x7f8998194240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:11.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.609 7f89a0528700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89981b8640 con 0x7f89981acae0 2021-02-10T11:45:11.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.609 7f89a0528700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899819e8b0 con 0x7f89981952e0 2021-02-10T11:45:11.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.609 7f89a0528700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899819ea90 con 0x7f89981b8bc0 2021-02-10T11:45:11.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.609 7f899eb45700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89981acae0 0x7f8998194240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:11.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.609 7f899eb45700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89981acae0 0x7f8998194240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60440/0 (socket says 172.21.2.101:60440) 2021-02-10T11:45:11.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.609 7f899eb45700 1 -- 172.21.2.101:0/1306805558 learned_addr learned my addr 172.21.2.101:0/1306805558 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:11.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.610 7f899e344700 1 --2- 172.21.2.101:0/1306805558 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89981952e0 0x7f899818ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:11.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.610 7f899db43700 1 --2- 172.21.2.101:0/1306805558 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89981b8bc0 0x7f89981ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:11.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.610 7f899eb45700 1 -- 172.21.2.101:0/1306805558 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89981b8bc0 msgr2=0x7f89981ac4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:11.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.610 7f899eb45700 1 --2- 172.21.2.101:0/1306805558 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89981b8bc0 0x7f89981ac4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:11.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.610 7f899eb45700 1 -- 172.21.2.101:0/1306805558 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89981952e0 msgr2=0x7f899818ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:11.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.610 7f899eb45700 1 --2- 172.21.2.101:0/1306805558 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89981952e0 0x7f899818ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:11.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.610 7f899eb45700 1 -- 172.21.2.101:0/1306805558 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89981acee0 con 0x7f89981acae0 2021-02-10T11:45:11.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.610 7f899e344700 1 --2- 172.21.2.101:0/1306805558 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89981952e0 0x7f899818ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:11.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.610 7f899eb45700 1 --2- 172.21.2.101:0/1306805558 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89981acae0 0x7f8998194240 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 rx=0x7f8990009980 tx=0x7f899000bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:11.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.610 7f898effd700 1 -- 172.21.2.101:0/1306805558 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f899000c2f0 con 0x7f89981acae0 2021-02-10T11:45:11.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.610 7f898effd700 1 -- 172.21.2.101:0/1306805558 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8990006790 con 0x7f89981acae0 2021-02-10T11:45:11.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.610 7f89a0528700 1 -- 172.21.2.101:0/1306805558 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89981acee0 con 0x7f89981acae0 2021-02-10T11:45:11.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.610 7f898effd700 1 -- 172.21.2.101:0/1306805558 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f899000c870 con 0x7f89981acae0 2021-02-10T11:45:11.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.610 7f89a0528700 1 -- 172.21.2.101:0/1306805558 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8998194b50 con 0x7f89981acae0 2021-02-10T11:45:11.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.611 7f898effd700 1 -- 172.21.2.101:0/1306805558 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f89900011f0 con 0x7f89981acae0 2021-02-10T11:45:11.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.611 7f898effd700 1 --2- 172.21.2.101:0/1306805558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8980018730 0x7f898001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:11.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.611 7f898effd700 1 -- 172.21.2.101:0/1306805558 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8990032a80 con 0x7f89981acae0 2021-02-10T11:45:11.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.611 7f899e344700 1 --2- 172.21.2.101:0/1306805558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8980018730 0x7f898001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:11.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.611 7f899e344700 1 --2- 172.21.2.101:0/1306805558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8980018730 0x7f898001ac10 crc :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:11.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.642 7f89a0528700 1 -- 172.21.2.101:0/1306805558 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89981b8fc0 con 0x7f89981acae0 2021-02-10T11:45:11.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.644 7f898effd700 1 -- 172.21.2.101:0/1306805558 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8990000f40 con 0x7f89981acae0 2021-02-10T11:45:11.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.768 7f89a0528700 1 -- 172.21.2.101:0/1306805558 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89981b8fc0 con 0x7f89981acae0 2021-02-10T11:45:11.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.768 7f898effd700 1 -- 172.21.2.101:0/1306805558 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f89900011f0 con 0x7f89981acae0 2021-02-10T11:45:11.767 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:45:11.768 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":156,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:44:57.770382","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:44:54.736504","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":33,"pool_name":"cephfs_data","create_time":"2021-02-10 11:44:55.811077","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:45:11.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.770 7f89a0528700 1 -- 172.21.2.101:0/1306805558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8980018730 msgr2=0x7f898001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:11.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.770 7f89a0528700 1 --2- 172.21.2.101:0/1306805558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8980018730 0x7f898001ac10 crc :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:11.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.770 7f89a0528700 1 -- 172.21.2.101:0/1306805558 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89981acae0 msgr2=0x7f8998194240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:11.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.770 7f89a0528700 1 --2- 172.21.2.101:0/1306805558 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89981acae0 0x7f8998194240 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 rx=0x7f8990009980 tx=0x7f899000bcd0).stop 2021-02-10T11:45:11.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.770 7f89a0528700 1 -- 172.21.2.101:0/1306805558 shutdown_connections 2021-02-10T11:45:11.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.770 7f89a0528700 1 --2- 172.21.2.101:0/1306805558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8980018730 0x7f898001ac10 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:11.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.770 7f89a0528700 1 --2- 172.21.2.101:0/1306805558 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89981b8bc0 0x7f89981ac4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:11.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.770 7f89a0528700 1 --2- 172.21.2.101:0/1306805558 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89981acae0 0x7f8998194240 unknown :-1 s=CLOSED pgs=2800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:11.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.770 7f89a0528700 1 --2- 172.21.2.101:0/1306805558 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89981952e0 0x7f899818ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:11.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.770 7f89a0528700 1 -- 172.21.2.101:0/1306805558 >> 172.21.2.101:0/1306805558 conn(0x7f89981ac040 msgr2=0x7f8998094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:11.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.770 7f89a0528700 1 -- 172.21.2.101:0/1306805558 shutdown_connections 2021-02-10T11:45:11.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:11.770 7f89a0528700 1 -- 172.21.2.101:0/1306805558 wait complete. 2021-02-10T11:45:11.777 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 32, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:44:54.736504', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 33, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:44:55.811077', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '156', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:45:11.778 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:45:12.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.195 7f32bc3f8700 1 Processor -- start 2021-02-10T11:45:12.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.195 7f32bc3f8700 1 -- start start 2021-02-10T11:45:12.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.196 7f32bc3f8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f32b40d1870 0x7f32b41b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:12.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.196 7f32bc3f8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32b40cff70 0x7f32b4109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:12.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.196 7f32bc3f8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b4109a60 con 0x7f32b41b0ae0 2021-02-10T11:45:12.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.196 7f32bc3f8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b4109750 con 0x7f32b40d1870 2021-02-10T11:45:12.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.196 7f32bc3f8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b4088070 con 0x7f32b40cff70 2021-02-10T11:45:12.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.196 7f32ba214700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f32b40d1870 0x7f32b41b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:12.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.196 7f32b9a13700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f32b41b0ae0 0x7f32b4108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:54350/0 (socket says 172.21.2.101:54350) 2021-02-10T11:45:12.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.196 7f32b9a13700 1 -- 172.21.2.101:0/4276946674 learned_addr learned my addr 172.21.2.101:0/4276946674 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:12.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.196 7f32baa15700 1 --2- 172.21.2.101:0/4276946674 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32b40cff70 0x7f32b4109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:12.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.197 7f32baa15700 1 -- 172.21.2.101:0/4276946674 >> v2:172.21.2.108:3300/0 conn(0x7f32b40d1870 msgr2=0x7f32b41b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:12.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.197 7f32baa15700 1 --2- 172.21.2.101:0/4276946674 >> v2:172.21.2.108:3300/0 conn(0x7f32b40d1870 0x7f32b41b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:12.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.197 7f32baa15700 1 -- 172.21.2.101:0/4276946674 >> v1:172.21.2.108:6789/0 conn(0x7f32b41b0ae0 legacy=0x7f32b4108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:12.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.197 7f32baa15700 1 -- 172.21.2.101:0/4276946674 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b40d1c70 con 0x7f32b40cff70 2021-02-10T11:45:12.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.197 7f32ba214700 1 --2- 172.21.2.101:0/4276946674 >> v2:172.21.2.108:3300/0 conn(0x7f32b40d1870 0x7f32b41b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:45:12.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.197 7f32baa15700 1 --2- 172.21.2.101:0/4276946674 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32b40cff70 0x7f32b4109130 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f32ac006e50 tx=0x7f32ac009e70).ready entity=mon.2 client_cookie=993e7760c3278dc6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:12.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.198 7f32b9212700 1 -- 172.21.2.101:0/4276946674 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f32ac006a20 con 0x7f32b40cff70 2021-02-10T11:45:12.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.198 7f32b9212700 1 -- 172.21.2.101:0/4276946674 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac006790 con 0x7f32b40cff70 2021-02-10T11:45:12.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.198 7f32b9212700 1 -- 172.21.2.101:0/4276946674 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f32ac000f90 con 0x7f32b40cff70 2021-02-10T11:45:12.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.198 7f32bc3f8700 1 -- 172.21.2.101:0/4276946674 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32b40cff70 msgr2=0x7f32b4109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:12.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.198 7f32bc3f8700 1 --2- 172.21.2.101:0/4276946674 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32b40cff70 0x7f32b4109130 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f32ac006e50 tx=0x7f32ac009e70).stop 2021-02-10T11:45:12.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.198 7f32bc3f8700 1 -- 172.21.2.101:0/4276946674 shutdown_connections 2021-02-10T11:45:12.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.198 7f32bc3f8700 1 --2- 172.21.2.101:0/4276946674 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32b40cff70 0x7f32b4109130 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:12.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.198 7f32bc3f8700 1 --2- 172.21.2.101:0/4276946674 >> v2:172.21.2.108:3300/0 conn(0x7f32b40d1870 0x7f32b41b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:12.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.198 7f32bc3f8700 1 -- 172.21.2.101:0/4276946674 >> 172.21.2.101:0/4276946674 conn(0x7f32b40ce420 msgr2=0x7f32b40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:12.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.199 7f32bc3f8700 1 -- 172.21.2.101:0/4276946674 shutdown_connections 2021-02-10T11:45:12.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.199 7f32bc3f8700 1 -- 172.21.2.101:0/4276946674 wait complete. 2021-02-10T11:45:12.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.213 7f32bc3f8700 1 Processor -- start 2021-02-10T11:45:12.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.213 7f32bc3f8700 1 -- start start 2021-02-10T11:45:12.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.213 7f32bc3f8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f32b40a7950 0x7f32b40b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:12.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.213 7f32bc3f8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32b41a7490 0x7f32b41a7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:12.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.213 7f32bc3f8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32b41a7eb0 0x7f32b40ac630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:12.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.213 7f32bc3f8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b40a7fb0 con 0x7f32b41a7eb0 2021-02-10T11:45:12.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.213 7f32bc3f8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b40acc50 con 0x7f32b40a7950 2021-02-10T11:45:12.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.213 7f32bc3f8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b40ace30 con 0x7f32b41a7490 2021-02-10T11:45:12.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.214 7f32baa15700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32b41a7eb0 0x7f32b40ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:12.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.214 7f32baa15700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32b41a7eb0 0x7f32b40ac630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60450/0 (socket says 172.21.2.101:60450) 2021-02-10T11:45:12.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.214 7f32baa15700 1 -- 172.21.2.101:0/1280597022 learned_addr learned my addr 172.21.2.101:0/1280597022 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:12.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.214 7f32ba214700 1 --2- 172.21.2.101:0/1280597022 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f32b40a7950 0x7f32b40b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:12.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.214 7f32b9a13700 1 --2- 172.21.2.101:0/1280597022 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32b41a7490 0x7f32b41a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:12.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.214 7f32baa15700 1 -- 172.21.2.101:0/1280597022 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32b41a7490 msgr2=0x7f32b41a7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:12.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.214 7f32baa15700 1 --2- 172.21.2.101:0/1280597022 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32b41a7490 0x7f32b41a7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:12.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.214 7f32baa15700 1 -- 172.21.2.101:0/1280597022 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f32b40a7950 msgr2=0x7f32b40b8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:12.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.214 7f32baa15700 1 --2- 172.21.2.101:0/1280597022 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f32b40a7950 0x7f32b40b8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:12.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.214 7f32baa15700 1 -- 172.21.2.101:0/1280597022 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b40ad050 con 0x7f32b41a7eb0 2021-02-10T11:45:12.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.214 7f32ba214700 1 --2- 172.21.2.101:0/1280597022 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f32b40a7950 0x7f32b40b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:45:12.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.214 7f32baa15700 1 --2- 172.21.2.101:0/1280597022 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32b41a7eb0 0x7f32b40ac630 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 rx=0x7f32ac0099f0 tx=0x7f32ac00bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:12.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.215 7f32a6ffd700 1 -- 172.21.2.101:0/1280597022 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f32ac0093d0 con 0x7f32b41a7eb0 2021-02-10T11:45:12.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.215 7f32a6ffd700 1 -- 172.21.2.101:0/1280597022 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac0090c0 con 0x7f32b41a7eb0 2021-02-10T11:45:12.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.215 7f32a6ffd700 1 -- 172.21.2.101:0/1280597022 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f32ac001390 con 0x7f32b41a7eb0 2021-02-10T11:45:12.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.215 7f32bc3f8700 1 -- 172.21.2.101:0/1280597022 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b40ad050 con 0x7f32b41a7eb0 2021-02-10T11:45:12.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.215 7f32bc3f8700 1 -- 172.21.2.101:0/1280597022 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b40a8530 con 0x7f32b41a7eb0 2021-02-10T11:45:12.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.215 7f32a6ffd700 1 -- 172.21.2.101:0/1280597022 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f32ac000df0 con 0x7f32b41a7eb0 2021-02-10T11:45:12.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.215 7f32a6ffd700 1 --2- 172.21.2.101:0/1280597022 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f329c0186c0 0x7f329c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:12.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.215 7f32a6ffd700 1 -- 172.21.2.101:0/1280597022 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f32ac032950 con 0x7f32b41a7eb0 2021-02-10T11:45:12.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.215 7f32ba214700 1 --2- 172.21.2.101:0/1280597022 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f329c0186c0 0x7f329c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:12.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.216 7f32ba214700 1 --2- 172.21.2.101:0/1280597022 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f329c0186c0 0x7f329c01abc0 crc :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:12.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.246 7f32bc3f8700 1 -- 172.21.2.101:0/1280597022 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b4082a70 con 0x7f32b41a7eb0 2021-02-10T11:45:12.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.249 7f32a6ffd700 1 -- 172.21.2.101:0/1280597022 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f32ac0093d0 con 0x7f32b41a7eb0 2021-02-10T11:45:12.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.376 7f32bc3f8700 1 -- 172.21.2.101:0/1280597022 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f32b40a84f0 con 0x7f32b41a7eb0 2021-02-10T11:45:12.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.376 7f32a6ffd700 1 -- 172.21.2.101:0/1280597022 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v156) v1 ==== 92+0+10 (secure 0 0 0) 0x7f32ac000df0 con 0x7f32b41a7eb0 2021-02-10T11:45:12.375 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:45:12.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.378 7f32bc3f8700 1 -- 172.21.2.101:0/1280597022 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f329c0186c0 msgr2=0x7f329c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:12.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.378 7f32bc3f8700 1 --2- 172.21.2.101:0/1280597022 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f329c0186c0 0x7f329c01abc0 crc :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:12.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.378 7f32bc3f8700 1 -- 172.21.2.101:0/1280597022 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32b41a7eb0 msgr2=0x7f32b40ac630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:12.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.378 7f32bc3f8700 1 --2- 172.21.2.101:0/1280597022 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32b41a7eb0 0x7f32b40ac630 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 rx=0x7f32ac0099f0 tx=0x7f32ac00bd10).stop 2021-02-10T11:45:12.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.378 7f32bc3f8700 1 -- 172.21.2.101:0/1280597022 shutdown_connections 2021-02-10T11:45:12.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.378 7f32bc3f8700 1 --2- 172.21.2.101:0/1280597022 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f329c0186c0 0x7f329c01abc0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:12.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.378 7f32bc3f8700 1 --2- 172.21.2.101:0/1280597022 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f32b41a7490 0x7f32b41a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:12.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.378 7f32bc3f8700 1 --2- 172.21.2.101:0/1280597022 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f32b41a7eb0 0x7f32b40ac630 unknown :-1 s=CLOSED pgs=2801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:12.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.378 7f32bc3f8700 1 --2- 172.21.2.101:0/1280597022 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f32b40a7950 0x7f32b40b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:12.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.378 7f32bc3f8700 1 -- 172.21.2.101:0/1280597022 >> 172.21.2.101:0/1280597022 conn(0x7f32b40a02d0 msgr2=0x7f32b409f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:12.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.379 7f32bc3f8700 1 -- 172.21.2.101:0/1280597022 shutdown_connections 2021-02-10T11:45:12.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.379 7f32bc3f8700 1 -- 172.21.2.101:0/1280597022 wait complete. 2021-02-10T11:45:12.386 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:45:12.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.803 7f21fb73c700 1 Processor -- start 2021-02-10T11:45:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.803 7f21fb73c700 1 -- start start 2021-02-10T11:45:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.803 7f21fb73c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21f40d1870 0x7f21f41b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.803 7f21fb73c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f21f41b0ae0 0x7f21f4108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.803 7f21fb73c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f4109a60 con 0x7f21f40d1c70 2021-02-10T11:45:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.803 7f21fb73c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21f40e5ef0 con 0x7f21f41b0ae0 2021-02-10T11:45:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.803 7f21fb73c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f4109750 con 0x7f21f40d1870 2021-02-10T11:45:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.803 7f21f9d59700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f21f40d1c70 0x7f21f4108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60950/0 (socket says 172.21.2.101:60950) 2021-02-10T11:45:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.803 7f21f9d59700 1 -- 172.21.2.101:0/3245255983 learned_addr learned my addr 172.21.2.101:0/3245255983 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21ebfff700 1 -- 172.21.2.101:0/3245255983 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3325636 0 0) 0x7f21f0001180 con 0x7f21f40d1c70 2021-02-10T11:45:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21ebfff700 1 -- 172.21.2.101:0/3245255983 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21e0002fe0 con 0x7f21f40d1c70 2021-02-10T11:45:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21f8d57700 1 --2- 172.21.2.101:0/3245255983 >> v2:172.21.2.108:3300/0 conn(0x7f21f41b0ae0 0x7f21f4108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21f9558700 1 --2- 172.21.2.101:0/3245255983 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21f40d1870 0x7f21f41b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:12.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21ebfff700 1 -- 172.21.2.101:0/3245255983 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2841422824 0 0) 0x7f21f00010e0 con 0x7f21f40d1c70 2021-02-10T11:45:12.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21ebfff700 1 -- 172.21.2.101:0/3245255983 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21f40d1870 msgr2=0x7f21f41b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:12.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21ebfff700 1 --2- 172.21.2.101:0/3245255983 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21f40d1870 0x7f21f41b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:12.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21ebfff700 1 -- 172.21.2.101:0/3245255983 >> v2:172.21.2.108:3300/0 conn(0x7f21f41b0ae0 msgr2=0x7f21f4108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:12.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21ebfff700 1 --2- 172.21.2.101:0/3245255983 >> v2:172.21.2.108:3300/0 conn(0x7f21f41b0ae0 0x7f21f4108940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:12.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21ebfff700 1 -- 172.21.2.101:0/3245255983 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f4088070 con 0x7f21f40d1c70 2021-02-10T11:45:12.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21ebfff700 1 -- 172.21.2.101:0/3245255983 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f21f0001600 con 0x7f21f40d1c70 2021-02-10T11:45:12.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21f8d57700 1 --2- 172.21.2.101:0/3245255983 >> v2:172.21.2.108:3300/0 conn(0x7f21f41b0ae0 0x7f21f4108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:45:12.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21ebfff700 1 -- 172.21.2.101:0/3245255983 >> v1:172.21.2.101:6789/0 conn(0x7f21f40d1c70 legacy=0x7f21f4108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:12.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21ebfff700 1 --2- 172.21.2.101:0/3245255983 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21e0003a60 0x7f21e0005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:12.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21ebfff700 1 --2- 172.21.2.101:0/3245255983 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21e0006790 0x7f21e0008c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:12.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21ebfff700 1 --2- 172.21.2.101:0/3245255983 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21e0009500 0x7f21e000ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:12.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21ebfff700 1 -- 172.21.2.101:0/3245255983 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e00502e0 con 0x7f21e0003a60 2021-02-10T11:45:12.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21ebfff700 1 -- 172.21.2.101:0/3245255983 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e00504c0 con 0x7f21e0009500 2021-02-10T11:45:12.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21ebfff700 1 -- 172.21.2.101:0/3245255983 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e00506e0 con 0x7f21e0006790 2021-02-10T11:45:12.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21f9558700 1 --2- 172.21.2.101:0/3245255983 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21e0003a60 0x7f21e0005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:12.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21f9d59700 1 --2- 172.21.2.101:0/3245255983 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21e0009500 0x7f21e000ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:12.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21f8d57700 1 --2- 172.21.2.101:0/3245255983 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21e0006790 0x7f21e0008c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:12.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21f9558700 1 -- 172.21.2.101:0/3245255983 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21e0006790 msgr2=0x7f21e0008c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:12.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21f9558700 1 --2- 172.21.2.101:0/3245255983 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21e0006790 0x7f21e0008c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:12.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21f9558700 1 -- 172.21.2.101:0/3245255983 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21e0009500 msgr2=0x7f21e000ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:12.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21f9558700 1 --2- 172.21.2.101:0/3245255983 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21e0009500 0x7f21e000ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:12.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21f9558700 1 -- 172.21.2.101:0/3245255983 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e0050980 con 0x7f21e0003a60 2021-02-10T11:45:12.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.804 7f21f8d57700 1 --2- 172.21.2.101:0/3245255983 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21e0006790 0x7f21e0008c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:12.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.805 7f21f9558700 1 --2- 172.21.2.101:0/3245255983 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21e0003a60 0x7f21e0005ef0 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 rx=0x7f21ec006e10 tx=0x7f21ec009e50).ready entity=mon.0 client_cookie=350a38e090100c19 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:12.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.805 7f21ebfff700 1 -- 172.21.2.101:0/3245255983 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21ec00b510 con 0x7f21e0003a60 2021-02-10T11:45:12.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.805 7f21ebfff700 1 -- 172.21.2.101:0/3245255983 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21ec00b790 con 0x7f21e0003a60 2021-02-10T11:45:12.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.805 7f21ebfff700 1 -- 172.21.2.101:0/3245255983 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21ec00bc70 con 0x7f21e0003a60 2021-02-10T11:45:12.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.805 7f21fb73c700 1 -- 172.21.2.101:0/3245255983 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21e0003a60 msgr2=0x7f21e0005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:12.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.805 7f21fb73c700 1 --2- 172.21.2.101:0/3245255983 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21e0003a60 0x7f21e0005ef0 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 rx=0x7f21ec006e10 tx=0x7f21ec009e50).stop 2021-02-10T11:45:12.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.805 7f21f9d59700 1 -- 172.21.2.101:0/3245255983 reap_dead start 2021-02-10T11:45:12.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.805 7f21fb73c700 1 -- 172.21.2.101:0/3245255983 shutdown_connections 2021-02-10T11:45:12.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.805 7f21fb73c700 1 -- 172.21.2.101:0/3245255983 >> 172.21.2.101:0/3245255983 conn(0x7f21f40ce420 msgr2=0x7f21f40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:12.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.806 7f21fb73c700 1 -- 172.21.2.101:0/3245255983 shutdown_connections 2021-02-10T11:45:12.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.806 7f21fb73c700 1 -- 172.21.2.101:0/3245255983 wait complete. 2021-02-10T11:45:12.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.820 7f21fb73c700 1 Processor -- start 2021-02-10T11:45:12.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.820 7f21fb73c700 1 -- start start 2021-02-10T11:45:12.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.821 7f21fb73c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21f409bcc0 0x7f21f409c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:12.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.821 7f21fb73c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21f40a0820 0x7f21f40a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:12.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.821 7f21fb73c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21f40a1240 0x7f21f40858d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:12.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.821 7f21fb73c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f40a66b0 con 0x7f21f409bcc0 2021-02-10T11:45:12.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.821 7f21fb73c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f41a49b0 con 0x7f21f40a1240 2021-02-10T11:45:12.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.821 7f21fb73c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f41a4b90 con 0x7f21f40a0820 2021-02-10T11:45:12.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.821 7f21f9558700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21f409bcc0 0x7f21f409c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:12.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.821 7f21f9558700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21f409bcc0 0x7f21f409c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60466/0 (socket says 172.21.2.101:60466) 2021-02-10T11:45:12.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.821 7f21f9558700 1 -- 172.21.2.101:0/1278250335 learned_addr learned my addr 172.21.2.101:0/1278250335 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:12.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.821 7f21f9558700 1 -- 172.21.2.101:0/1278250335 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21f40a0820 msgr2=0x7f21f40a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:12.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.821 7f21f9558700 1 --2- 172.21.2.101:0/1278250335 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21f40a0820 0x7f21f40a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:12.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.821 7f21f9558700 1 -- 172.21.2.101:0/1278250335 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21f40a1240 msgr2=0x7f21f40858d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:12.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.821 7f21f9558700 1 --2- 172.21.2.101:0/1278250335 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21f40a1240 0x7f21f40858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:12.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.821 7f21f9558700 1 -- 172.21.2.101:0/1278250335 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f40a1640 con 0x7f21f409bcc0 2021-02-10T11:45:12.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.822 7f21f9558700 1 --2- 172.21.2.101:0/1278250335 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21f409bcc0 0x7f21f409c0c0 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 rx=0x7f21ec006a50 tx=0x7f21ec00bce0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:12.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.822 7f21e9ffb700 1 -- 172.21.2.101:0/1278250335 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21ec00c2f0 con 0x7f21f409bcc0 2021-02-10T11:45:12.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.822 7f21e9ffb700 1 -- 172.21.2.101:0/1278250335 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21ec00c530 con 0x7f21f409bcc0 2021-02-10T11:45:12.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.822 7f21e9ffb700 1 -- 172.21.2.101:0/1278250335 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21ec00ca50 con 0x7f21f409bcc0 2021-02-10T11:45:12.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.822 7f21fb73c700 1 -- 172.21.2.101:0/1278250335 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f40a1640 con 0x7f21f409bcc0 2021-02-10T11:45:12.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.822 7f21fb73c700 1 -- 172.21.2.101:0/1278250335 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f40a1640 con 0x7f21f409bcc0 2021-02-10T11:45:12.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.822 7f21e9ffb700 1 -- 172.21.2.101:0/1278250335 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f21ec001010 con 0x7f21f409bcc0 2021-02-10T11:45:12.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.822 7f21e9ffb700 1 --2- 172.21.2.101:0/1278250335 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21d40186c0 0x7f21d401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:12.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.822 7f21e9ffb700 1 -- 172.21.2.101:0/1278250335 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f21ec0328b0 con 0x7f21f409bcc0 2021-02-10T11:45:12.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.823 7f21f8d57700 1 --2- 172.21.2.101:0/1278250335 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21d40186c0 0x7f21d401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:12.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.823 7f21f8d57700 1 --2- 172.21.2.101:0/1278250335 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21d40186c0 0x7f21d401abc0 crc :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:12.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.853 7f21fb73c700 1 -- 172.21.2.101:0/1278250335 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f40fe1c0 con 0x7f21f409bcc0 2021-02-10T11:45:12.854 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.856 7f21e9ffb700 1 -- 172.21.2.101:0/1278250335 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f21ec00be20 con 0x7f21f409bcc0 2021-02-10T11:45:12.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.985 7f21fb73c700 1 -- 172.21.2.101:0/1278250335 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f21f40fe1c0 con 0x7f21f409bcc0 2021-02-10T11:45:12.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.986 7f21e9ffb700 1 -- 172.21.2.101:0/1278250335 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v156) v1 ==== 104+0+10 (secure 0 0 0) 0x7f21ec001010 con 0x7f21f409bcc0 2021-02-10T11:45:12.985 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:45:12.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.988 7f21fb73c700 1 -- 172.21.2.101:0/1278250335 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21d40186c0 msgr2=0x7f21d401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:12.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.988 7f21fb73c700 1 --2- 172.21.2.101:0/1278250335 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21d40186c0 0x7f21d401abc0 crc :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:12.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.988 7f21fb73c700 1 -- 172.21.2.101:0/1278250335 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21f409bcc0 msgr2=0x7f21f409c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:12.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.988 7f21fb73c700 1 --2- 172.21.2.101:0/1278250335 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21f409bcc0 0x7f21f409c0c0 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 rx=0x7f21ec006a50 tx=0x7f21ec00bce0).stop 2021-02-10T11:45:12.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.988 7f21fb73c700 1 -- 172.21.2.101:0/1278250335 shutdown_connections 2021-02-10T11:45:12.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.988 7f21fb73c700 1 --2- 172.21.2.101:0/1278250335 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21d40186c0 0x7f21d401abc0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:12.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.988 7f21fb73c700 1 --2- 172.21.2.101:0/1278250335 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21f40a0820 0x7f21f40a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:12.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.988 7f21fb73c700 1 --2- 172.21.2.101:0/1278250335 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21f40a1240 0x7f21f40858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:12.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.988 7f21fb73c700 1 --2- 172.21.2.101:0/1278250335 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21f409bcc0 0x7f21f409c0c0 unknown :-1 s=CLOSED pgs=2804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:12.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.988 7f21fb73c700 1 -- 172.21.2.101:0/1278250335 >> 172.21.2.101:0/1278250335 conn(0x7f21f40bef00 msgr2=0x7f21f409d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:12.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.988 7f21fb73c700 1 -- 172.21.2.101:0/1278250335 shutdown_connections 2021-02-10T11:45:12.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:12.988 7f21fb73c700 1 -- 172.21.2.101:0/1278250335 wait complete. 2021-02-10T11:45:12.995 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:45:13.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.412 7f04b3431700 1 Processor -- start 2021-02-10T11:45:13.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.412 7f04b3431700 1 -- start start 2021-02-10T11:45:13.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.413 7f04b3431700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f04ac10afe0 0x7f04ac0cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:13.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.413 7f04b3431700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04ac0fe1b0 con 0x7f04ac0d1860 2021-02-10T11:45:13.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.413 7f04b3431700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04ac1b0ca0 con 0x7f04ac0d1c60 2021-02-10T11:45:13.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.413 7f04b3431700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac109a50 con 0x7f04ac10afe0 2021-02-10T11:45:13.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.413 7f04b124d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f04ac0d1860 0x7f04ac1b04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60968/0 (socket says 172.21.2.101:60968) 2021-02-10T11:45:13.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.413 7f04b124d700 1 -- 172.21.2.101:0/1324209025 learned_addr learned my addr 172.21.2.101:0/1324209025 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:13.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.413 7f049ffff700 1 -- 172.21.2.101:0/1324209025 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711864437 0 0) 0x7f04a80010b0 con 0x7f04ac0d1860 2021-02-10T11:45:13.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.413 7f049ffff700 1 -- 172.21.2.101:0/1324209025 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0498002fe0 con 0x7f04ac0d1860 2021-02-10T11:45:13.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f04b1a4e700 1 --2- 172.21.2.101:0/1324209025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f04ac10afe0 0x7f04ac0cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:13.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f049ffff700 1 -- 172.21.2.101:0/1324209025 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 669560995 0 0) 0x7f04a80010b0 con 0x7f04ac0d1860 2021-02-10T11:45:13.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f049ffff700 1 -- 172.21.2.101:0/1324209025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f04ac10afe0 msgr2=0x7f04ac0cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:13.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f049ffff700 1 --2- 172.21.2.101:0/1324209025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f04ac10afe0 0x7f04ac0cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:13.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f049ffff700 1 -- 172.21.2.101:0/1324209025 >> v1:172.21.2.108:6789/0 conn(0x7f04ac0d1c60 legacy=0x7f04ac0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:13.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f049ffff700 1 -- 172.21.2.101:0/1324209025 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ac10b3e0 con 0x7f04ac0d1860 2021-02-10T11:45:13.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f049ffff700 1 -- 172.21.2.101:0/1324209025 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f04a80015b0 con 0x7f04ac0d1860 2021-02-10T11:45:13.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f049ffff700 1 -- 172.21.2.101:0/1324209025 >> v1:172.21.2.101:6789/0 conn(0x7f04ac0d1860 legacy=0x7f04ac1b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:13.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f049ffff700 1 --2- 172.21.2.101:0/1324209025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0498003b90 0x7f0498006000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:13.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f049ffff700 1 --2- 172.21.2.101:0/1324209025 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f04980068a0 0x7f0498008dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:13.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f049ffff700 1 --2- 172.21.2.101:0/1324209025 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0498009660 0x7f049800bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:13.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f049ffff700 1 -- 172.21.2.101:0/1324209025 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049800c410 con 0x7f0498009660 2021-02-10T11:45:13.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f049ffff700 1 -- 172.21.2.101:0/1324209025 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049800c630 con 0x7f04980068a0 2021-02-10T11:45:13.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f049ffff700 1 -- 172.21.2.101:0/1324209025 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049800c850 con 0x7f0498003b90 2021-02-10T11:45:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f04b1a4e700 1 --2- 172.21.2.101:0/1324209025 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0498009660 0x7f049800bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f04b124d700 1 --2- 172.21.2.101:0/1324209025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0498003b90 0x7f0498006000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f04b0a4c700 1 --2- 172.21.2.101:0/1324209025 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f04980068a0 0x7f0498008dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f04b1a4e700 1 -- 172.21.2.101:0/1324209025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0498003b90 msgr2=0x7f0498006000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f04b1a4e700 1 --2- 172.21.2.101:0/1324209025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0498003b90 0x7f0498006000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f04b1a4e700 1 -- 172.21.2.101:0/1324209025 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f04980068a0 msgr2=0x7f0498008dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f04b1a4e700 1 --2- 172.21.2.101:0/1324209025 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f04980068a0 0x7f0498008dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f04b1a4e700 1 -- 172.21.2.101:0/1324209025 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049800caf0 con 0x7f0498009660 2021-02-10T11:45:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f04b124d700 1 --2- 172.21.2.101:0/1324209025 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0498003b90 0x7f0498006000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:45:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.414 7f04b1a4e700 1 --2- 172.21.2.101:0/1324209025 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0498009660 0x7f049800bb60 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 rx=0x7f04a4001120 tx=0x7f04a404de70).ready entity=mon.0 client_cookie=15c9183cb38890a2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.415 7f049ffff700 1 -- 172.21.2.101:0/1324209025 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f04a40069c0 con 0x7f0498009660 2021-02-10T11:45:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.415 7f049ffff700 1 -- 172.21.2.101:0/1324209025 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a4003030 con 0x7f0498009660 2021-02-10T11:45:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.415 7f049ffff700 1 -- 172.21.2.101:0/1324209025 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f04a404f700 con 0x7f0498009660 2021-02-10T11:45:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.415 7f04b3431700 1 -- 172.21.2.101:0/1324209025 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0498009660 msgr2=0x7f049800bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.415 7f04b3431700 1 --2- 172.21.2.101:0/1324209025 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0498009660 0x7f049800bb60 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 rx=0x7f04a4001120 tx=0x7f04a404de70).stop 2021-02-10T11:45:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.415 7f04b1a4e700 1 -- 172.21.2.101:0/1324209025 reap_dead start 2021-02-10T11:45:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.415 7f04b3431700 1 -- 172.21.2.101:0/1324209025 shutdown_connections 2021-02-10T11:45:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.415 7f04b3431700 1 -- 172.21.2.101:0/1324209025 >> 172.21.2.101:0/1324209025 conn(0x7f04ac0ce410 msgr2=0x7f04ac0d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.416 7f04b3431700 1 -- 172.21.2.101:0/1324209025 shutdown_connections 2021-02-10T11:45:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.416 7f04b3431700 1 -- 172.21.2.101:0/1324209025 wait complete. 2021-02-10T11:45:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.430 7f04b3431700 1 Processor -- start 2021-02-10T11:45:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.430 7f04b3431700 1 -- start start 2021-02-10T11:45:13.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.431 7f04b3431700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f04ac0b8100 0x7f04ac083440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:13.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.431 7f04b3431700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04ac0b8500 0x7f04ac083a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:13.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.431 7f04b3431700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f04ac0846f0 0x7f04ac084af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:13.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.431 7f04b3431700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac1a1720 con 0x7f04ac0b8500 2021-02-10T11:45:13.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.431 7f04b3431700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac189e70 con 0x7f04ac0b8100 2021-02-10T11:45:13.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.431 7f04b3431700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac084170 con 0x7f04ac0846f0 2021-02-10T11:45:13.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.431 7f04b0a4c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04ac0b8500 0x7f04ac083a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:13.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.431 7f04b0a4c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04ac0b8500 0x7f04ac083a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60484/0 (socket says 172.21.2.101:60484) 2021-02-10T11:45:13.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.431 7f04b0a4c700 1 -- 172.21.2.101:0/929092888 learned_addr learned my addr 172.21.2.101:0/929092888 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:13.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.431 7f04b0a4c700 1 -- 172.21.2.101:0/929092888 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f04ac0846f0 msgr2=0x7f04ac084af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:13.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.431 7f04b0a4c700 1 --2- 172.21.2.101:0/929092888 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f04ac0846f0 0x7f04ac084af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:13.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.431 7f04b0a4c700 1 -- 172.21.2.101:0/929092888 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f04ac0b8100 msgr2=0x7f04ac083440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:13.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.431 7f04b0a4c700 1 --2- 172.21.2.101:0/929092888 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f04ac0b8100 0x7f04ac083440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:13.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.431 7f04b0a4c700 1 -- 172.21.2.101:0/929092888 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ac085110 con 0x7f04ac0b8500 2021-02-10T11:45:13.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.431 7f04b0a4c700 1 --2- 172.21.2.101:0/929092888 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04ac0b8500 0x7f04ac083a60 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 rx=0x7f04a000b5e0 tx=0x7f04a000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:13.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.431 7f049dffb700 1 -- 172.21.2.101:0/929092888 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f04a000bd70 con 0x7f04ac0b8500 2021-02-10T11:45:13.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.431 7f049dffb700 1 -- 172.21.2.101:0/929092888 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a00063c0 con 0x7f04ac0b8500 2021-02-10T11:45:13.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.431 7f049dffb700 1 -- 172.21.2.101:0/929092888 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f04a0005560 con 0x7f04ac0b8500 2021-02-10T11:45:13.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.431 7f04b3431700 1 -- 172.21.2.101:0/929092888 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04ac085110 con 0x7f04ac0b8500 2021-02-10T11:45:13.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.431 7f04b3431700 1 -- 172.21.2.101:0/929092888 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04ac085110 con 0x7f04ac0b8500 2021-02-10T11:45:13.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.432 7f049dffb700 1 -- 172.21.2.101:0/929092888 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f04a0006030 con 0x7f04ac0b8500 2021-02-10T11:45:13.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.432 7f049dffb700 1 --2- 172.21.2.101:0/929092888 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f048c018730 0x7f048c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:13.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.432 7f049dffb700 1 -- 172.21.2.101:0/929092888 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f04a0032640 con 0x7f04ac0b8500 2021-02-10T11:45:13.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.432 7f04b124d700 1 --2- 172.21.2.101:0/929092888 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f048c018730 0x7f048c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:13.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.432 7f04b124d700 1 --2- 172.21.2.101:0/929092888 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f048c018730 0x7f048c01ac10 crc :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:13.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.462 7f04b3431700 1 -- 172.21.2.101:0/929092888 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04ac1a1720 con 0x7f04ac0b8500 2021-02-10T11:45:13.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.465 7f049dffb700 1 -- 172.21.2.101:0/929092888 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f04a001f500 con 0x7f04ac0b8500 2021-02-10T11:45:13.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.592 7f04b3431700 1 -- 172.21.2.101:0/929092888 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f04ac189e70 con 0x7f04ac0b8500 2021-02-10T11:45:13.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.593 7f049dffb700 1 -- 172.21.2.101:0/929092888 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v156) v1 ==== 100+0+10 (secure 0 0 0) 0x7f04a000bd70 con 0x7f04ac0b8500 2021-02-10T11:45:13.592 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:45:13.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.594 7f04b3431700 1 -- 172.21.2.101:0/929092888 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f048c018730 msgr2=0x7f048c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:13.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.594 7f04b3431700 1 --2- 172.21.2.101:0/929092888 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f048c018730 0x7f048c01ac10 crc :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:13.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.594 7f04b3431700 1 -- 172.21.2.101:0/929092888 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04ac0b8500 msgr2=0x7f04ac083a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:13.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.594 7f04b3431700 1 --2- 172.21.2.101:0/929092888 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04ac0b8500 0x7f04ac083a60 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 rx=0x7f04a000b5e0 tx=0x7f04a000b980).stop 2021-02-10T11:45:13.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.595 7f04b3431700 1 -- 172.21.2.101:0/929092888 shutdown_connections 2021-02-10T11:45:13.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.595 7f04b3431700 1 --2- 172.21.2.101:0/929092888 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f048c018730 0x7f048c01ac10 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:13.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.595 7f04b3431700 1 --2- 172.21.2.101:0/929092888 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f04ac0846f0 0x7f04ac084af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:13.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.595 7f04b3431700 1 --2- 172.21.2.101:0/929092888 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04ac0b8500 0x7f04ac083a60 unknown :-1 s=CLOSED pgs=2807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:13.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.595 7f04b3431700 1 --2- 172.21.2.101:0/929092888 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f04ac0b8100 0x7f04ac083440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:13.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.595 7f04b3431700 1 -- 172.21.2.101:0/929092888 >> 172.21.2.101:0/929092888 conn(0x7f04ac0bee90 msgr2=0x7f04ac09f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:13.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.595 7f04b3431700 1 -- 172.21.2.101:0/929092888 shutdown_connections 2021-02-10T11:45:13.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:13.595 7f04b3431700 1 -- 172.21.2.101:0/929092888 wait complete. 2021-02-10T11:45:13.602 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:45:13.602 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:45:14.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.017 7f8ee89aa700 1 Processor -- start 2021-02-10T11:45:14.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.017 7f8ee89aa700 1 -- start start 2021-02-10T11:45:14.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.017 7f8ee89aa700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8ee40d0340 0x7f8ee40ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:14.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.017 7f8ee89aa700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee40f0b10 con 0x7f8ee40d1c70 2021-02-10T11:45:14.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.017 7f8ee89aa700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee41b0be0 con 0x7f8ee40d1870 2021-02-10T11:45:14.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.017 7f8ee89aa700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ee40e5ed0 con 0x7f8ee40d0340 2021-02-10T11:45:14.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.017 7f8ee1e1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8ee40d0340 0x7f8ee40ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:14.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.017 7f8ee2e1d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8ee40d1c70 0x7f8ee40efc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:60990/0 (socket says 172.21.2.101:60990) 2021-02-10T11:45:14.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.017 7f8ee1e1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8ee40d0340 0x7f8ee40ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60492/0 (socket says 172.21.2.101:60492) 2021-02-10T11:45:14.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.017 7f8ee1e1b700 1 -- 172.21.2.101:0/1964809229 learned_addr learned my addr 172.21.2.101:0/1964809229 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:14.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.018 7f8ee1e1b700 1 -- 172.21.2.101:0/1964809229 >> v1:172.21.2.108:6789/0 conn(0x7f8ee40d1870 legacy=0x7f8ee41b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:14.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.018 7f8ee1e1b700 1 -- 172.21.2.101:0/1964809229 >> v1:172.21.2.101:6789/0 conn(0x7f8ee40d1c70 legacy=0x7f8ee40efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:14.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.018 7f8ee1e1b700 1 -- 172.21.2.101:0/1964809229 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee40f0450 con 0x7f8ee40d0340 2021-02-10T11:45:14.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.018 7f8ee1e1b700 1 --2- 172.21.2.101:0/1964809229 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ee40d0340 0x7f8ee40ef640 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 rx=0x7f8ed8006700 tx=0x7f8ed8009b80).ready entity=mon.0 client_cookie=ad090ba125937e38 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:14.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.018 7f8ee161a700 1 -- 172.21.2.101:0/1964809229 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8ed800c0c0 con 0x7f8ee40d0340 2021-02-10T11:45:14.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.018 7f8ee161a700 1 -- 172.21.2.101:0/1964809229 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed800c440 con 0x7f8ee40d0340 2021-02-10T11:45:14.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.018 7f8ee161a700 1 -- 172.21.2.101:0/1964809229 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8ed800c8a0 con 0x7f8ee40d0340 2021-02-10T11:45:14.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.018 7f8ee89aa700 1 -- 172.21.2.101:0/1964809229 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ee40d0340 msgr2=0x7f8ee40ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:14.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.018 7f8ee89aa700 1 --2- 172.21.2.101:0/1964809229 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ee40d0340 0x7f8ee40ef640 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 rx=0x7f8ed8006700 tx=0x7f8ed8009b80).stop 2021-02-10T11:45:14.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.019 7f8ee89aa700 1 -- 172.21.2.101:0/1964809229 shutdown_connections 2021-02-10T11:45:14.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.019 7f8ee89aa700 1 --2- 172.21.2.101:0/1964809229 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ee40d0340 0x7f8ee40ef640 unknown :-1 s=CLOSED pgs=2809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:14.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.019 7f8ee89aa700 1 -- 172.21.2.101:0/1964809229 >> 172.21.2.101:0/1964809229 conn(0x7f8ee40ce450 msgr2=0x7f8ee40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:14.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.019 7f8ee89aa700 1 -- 172.21.2.101:0/1964809229 shutdown_connections 2021-02-10T11:45:14.034 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.019 7f8ee89aa700 1 -- 172.21.2.101:0/1964809229 wait complete. 2021-02-10T11:45:14.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.034 7f8ee89aa700 1 Processor -- start 2021-02-10T11:45:14.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.034 7f8ee89aa700 1 -- start start 2021-02-10T11:45:14.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.034 7f8ee89aa700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ee4103520 0x7f8ee40c3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:14.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.034 7f8ee89aa700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8ee4103920 0x7f8ee40b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:14.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.034 7f8ee89aa700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8ee408a450 0x7f8ee408a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:14.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.034 7f8ee89aa700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee40c3c60 con 0x7f8ee4103520 2021-02-10T11:45:14.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.034 7f8ee89aa700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee40b38f0 con 0x7f8ee4103920 2021-02-10T11:45:14.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.034 7f8ee89aa700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee40c0be0 con 0x7f8ee408a450 2021-02-10T11:45:14.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.034 7f8ee261c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ee4103520 0x7f8ee40c3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:14.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.034 7f8ee261c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ee4103520 0x7f8ee40c3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60502/0 (socket says 172.21.2.101:60502) 2021-02-10T11:45:14.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.034 7f8ee261c700 1 -- 172.21.2.101:0/2617055178 learned_addr learned my addr 172.21.2.101:0/2617055178 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:14.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.035 7f8ee261c700 1 -- 172.21.2.101:0/2617055178 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8ee408a450 msgr2=0x7f8ee408a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:14.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.035 7f8ee261c700 1 --2- 172.21.2.101:0/2617055178 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8ee408a450 0x7f8ee408a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:14.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.035 7f8ee261c700 1 -- 172.21.2.101:0/2617055178 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8ee4103920 msgr2=0x7f8ee40b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:14.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.035 7f8ee261c700 1 --2- 172.21.2.101:0/2617055178 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8ee4103920 0x7f8ee40b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:14.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.035 7f8ee261c700 1 -- 172.21.2.101:0/2617055178 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee408ae70 con 0x7f8ee4103520 2021-02-10T11:45:14.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.035 7f8ee261c700 1 --2- 172.21.2.101:0/2617055178 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ee4103520 0x7f8ee40c3640 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 rx=0x7f8ed400b530 tx=0x7f8ed400b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:14.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.035 7f8ed2ffd700 1 -- 172.21.2.101:0/2617055178 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8ed400bcc0 con 0x7f8ee4103520 2021-02-10T11:45:14.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.035 7f8ed2ffd700 1 -- 172.21.2.101:0/2617055178 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed4000e20 con 0x7f8ee4103520 2021-02-10T11:45:14.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.035 7f8ed2ffd700 1 -- 172.21.2.101:0/2617055178 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8ed4006210 con 0x7f8ee4103520 2021-02-10T11:45:14.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.035 7f8ee89aa700 1 -- 172.21.2.101:0/2617055178 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ee408ae70 con 0x7f8ee4103520 2021-02-10T11:45:14.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.035 7f8ee89aa700 1 -- 172.21.2.101:0/2617055178 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee408b400 con 0x7f8ee4103520 2021-02-10T11:45:14.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.036 7f8ed2ffd700 1 -- 172.21.2.101:0/2617055178 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8ed4006020 con 0x7f8ee4103520 2021-02-10T11:45:14.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.036 7f8ed2ffd700 1 --2- 172.21.2.101:0/2617055178 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ec80186c0 0x7f8ec801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:14.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.036 7f8ed2ffd700 1 -- 172.21.2.101:0/2617055178 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8ed40325b0 con 0x7f8ee4103520 2021-02-10T11:45:14.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.036 7f8ee1e1b700 1 --2- 172.21.2.101:0/2617055178 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ec80186c0 0x7f8ec801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:14.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.036 7f8ee1e1b700 1 --2- 172.21.2.101:0/2617055178 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ec80186c0 0x7f8ec801abc0 crc :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:14.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.067 7f8ee89aa700 1 -- 172.21.2.101:0/2617055178 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ee40b38f0 con 0x7f8ee4103520 2021-02-10T11:45:14.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.070 7f8ed2ffd700 1 -- 172.21.2.101:0/2617055178 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8ed4000940 con 0x7f8ee4103520 2021-02-10T11:45:14.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.199 7f8ee89aa700 1 -- 172.21.2.101:0/2617055178 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ee40b38f0 con 0x7f8ee4103520 2021-02-10T11:45:14.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.199 7f8ed2ffd700 1 -- 172.21.2.101:0/2617055178 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+2105 (secure 0 0 0) 0x7f8ed4006000 con 0x7f8ee4103520 2021-02-10T11:45:14.198 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:45:14.198 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":131,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":12135,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/3467749863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3467749863},{"type":"v1","addr":"172.21.2.101:6835","nonce":3467749863}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":128}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:44:56.768458","modified":"2021-02-10 11:44:58.773048","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8242":{"gid":8242,"name":"b","rank":0,"incarnation":129,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/485291097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":485291097},{"type":"v1","addr":"172.21.2.108:6833","nonce":485291097}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2021-02-10T11:45:14.198 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 131 2021-02-10T11:45:14.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.201 7f8ee89aa700 1 -- 172.21.2.101:0/2617055178 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ec80186c0 msgr2=0x7f8ec801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:14.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.201 7f8ee89aa700 1 --2- 172.21.2.101:0/2617055178 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ec80186c0 0x7f8ec801abc0 crc :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:14.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.201 7f8ee89aa700 1 -- 172.21.2.101:0/2617055178 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ee4103520 msgr2=0x7f8ee40c3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:14.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.201 7f8ee89aa700 1 --2- 172.21.2.101:0/2617055178 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ee4103520 0x7f8ee40c3640 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 rx=0x7f8ed400b530 tx=0x7f8ed400b8d0).stop 2021-02-10T11:45:14.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.201 7f8ee89aa700 1 -- 172.21.2.101:0/2617055178 shutdown_connections 2021-02-10T11:45:14.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.201 7f8ee89aa700 1 --2- 172.21.2.101:0/2617055178 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ec80186c0 0x7f8ec801abc0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:14.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.201 7f8ee89aa700 1 --2- 172.21.2.101:0/2617055178 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8ee408a450 0x7f8ee408a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:14.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.201 7f8ee89aa700 1 --2- 172.21.2.101:0/2617055178 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8ee4103920 0x7f8ee40b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:14.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.201 7f8ee89aa700 1 --2- 172.21.2.101:0/2617055178 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ee4103520 0x7f8ee40c3640 unknown :-1 s=CLOSED pgs=2810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:14.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.201 7f8ee89aa700 1 -- 172.21.2.101:0/2617055178 >> 172.21.2.101:0/2617055178 conn(0x7f8ee410a820 msgr2=0x7f8ee4108150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:14.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.201 7f8ee89aa700 1 -- 172.21.2.101:0/2617055178 shutdown_connections 2021-02-10T11:45:14.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:14.201 7f8ee89aa700 1 -- 172.21.2.101:0/2617055178 wait complete. 2021-02-10T11:45:14.409 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 131, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:44:56.768458', 'modified': '2021-02-10 11:44:58.773048', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8242}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8242': {'gid': 8242, 'name': 'b', 'rank': 0, 'incarnation': 129, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/485291097', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 485291097}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 485291097}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [33], 'metadata_pool': 32, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:45:14.409 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:45:14.410 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:45:14.808 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:45:14.809 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:45:14.809 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:45:14.809 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 8242, 2021-02-10T11:45:14.809 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:45:14.809 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:45:14.809 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 18.039418082000001, 2021-02-10T11:45:14.809 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 131, 2021-02-10T11:45:14.809 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 156, 2021-02-10T11:45:14.809 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 156, 2021-02-10T11:45:14.809 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 23.683595923999999 2021-02-10T11:45:14.809 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:45:14.812 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 8242, "want_state": "up:active", "state": "up:active", "rank_uptime": 18.039418082000001, "mdsmap_epoch": 131, "osdmap_epoch": 156, "osdmap_epoch_barrier": 156, "uptime": 23.683595923999999 } 2021-02-10T11:45:14.813 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:45:14.813 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-02-10T11:45:14.813 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:45:14.813 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:14.813 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:45:15.149 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:45:15.494 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:45:15.495 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:45:15.837 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:45:15.837 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:45:15.838 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:45:15.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:15.838 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:45:16.173 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:45:16.174 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2021-02-10T11:45:16.174 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:45:16.177 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:45:16.521 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:45:16.521 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:45:16.587 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[166696]: starting ceph client 2021-02-10T11:45:16.587 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:2021-02-10 11:45:16.588 7fc1fb9dcf80 -1 init, newargv = 0x556a4a7e11f0 newargc=9 2021-02-10T11:45:16.592 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[166696]: starting fuse 2021-02-10T11:45:16.864 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:45:16.864 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:45:16.865 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:45:16.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:16.866 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:45:17.199 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:45:17.199 INFO:teuthology.orchestra.run.gibba001.stdout:44 2021-02-10T11:45:17.199 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2021-02-10T11:45:17.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:17.200 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:45:17.200 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:45:17.200 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:45:17.200 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:45:17.200 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:45:17.200 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:45:17.200 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:45:17.200 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-02-10T11:45:17.200 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:45:17.200 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:45:17.200 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:45:17.200 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:45:17.200 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:45:17.200 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:45:17.201 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:45:17.201 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:45:17.201 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:45:17.201 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:45:17.201 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:45:17.201 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:45:17.201 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:45:17.201 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.1")) 2021-02-10T11:45:17.201 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:45:17.563 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.1.166696.asok 2021-02-10T11:45:17.566 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.166696.asok 2021-02-10T11:45:17.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:17.566 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.166696.asok status 2021-02-10T11:45:17.957 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:45:17.957 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:45:17.957 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:45:17.958 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:45:17.958 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "1", 2021-02-10T11:45:17.958 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:45:17.958 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:45:17.958 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "166696", 2021-02-10T11:45:17.958 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:45:17.958 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:17.958 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:45:17.958 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:45:17.958 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 12348, 2021-02-10T11:45:17.958 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:45:17.958 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:45:17.959 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:45:17.959 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 12348 2021-02-10T11:45:17.959 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:17.959 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:45:17.959 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:17.959 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:45:17.959 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 793048344 2021-02-10T11:45:17.959 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:17.959 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:17.959 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:45:17.959 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:17.959 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:45:17.959 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 793048344 2021-02-10T11:45:17.960 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:17.960 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.12348 172.21.2.101:0/793048344", 2021-02-10T11:45:17.960 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/793048344", 2021-02-10T11:45:17.960 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:45:17.960 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 131, 2021-02-10T11:45:17.960 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 156, 2021-02-10T11:45:17.960 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:45:17.960 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:45:17.960 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:45:17.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:17.962 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:45:17.962 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:45:18.298 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:45:18.298 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:45:18.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:18.298 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:45:18.645 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2021-02-10T11:45:19.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.063 7f8c6b67b700 1 Processor -- start 2021-02-10T11:45:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.064 7f8c6b67b700 1 -- start start 2021-02-10T11:45:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.064 7f8c6b67b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8c640d1c80 0x7f8c640cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.064 7f8c6b67b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c640cf590 0x7f8c640cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.064 7f8c6b67b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c640fe1d0 con 0x7f8c640d1880 2021-02-10T11:45:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.064 7f8c6b67b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c641b0cc0 con 0x7f8c640d1c80 2021-02-10T11:45:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.064 7f8c6b67b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c64109a70 con 0x7f8c640cf590 2021-02-10T11:45:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.064 7f8c68c96700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8c640d1c80 0x7f8c640cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.064 7f8c68c96700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8c640d1c80 0x7f8c640cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60520/0 (socket says 172.21.2.101:60520) 2021-02-10T11:45:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.064 7f8c68c96700 1 -- 172.21.2.101:0/239805361 learned_addr learned my addr 172.21.2.101:0/239805361 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.064 7f8c68c96700 1 -- 172.21.2.101:0/239805361 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c640cf590 msgr2=0x7f8c640cf990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:45:19.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.064 7f8c68c96700 1 --2- 172.21.2.101:0/239805361 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c640cf590 0x7f8c640cf990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.064 7f8c68c96700 1 -- 172.21.2.101:0/239805361 >> v1:172.21.2.108:6789/0 conn(0x7f8c640d1880 legacy=0x7f8c641b04d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:45:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.065 7f8c68c96700 1 -- 172.21.2.101:0/239805361 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c64088070 con 0x7f8c640d1c80 2021-02-10T11:45:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.065 7f8c68c96700 1 --2- 172.21.2.101:0/239805361 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c640d1c80 0x7f8c640cef70 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 rx=0x7f8c60006700 tx=0x7f8c60009b80).ready entity=mon.0 client_cookie=d0a4873180c0cf50 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.065 7f8c57fff700 1 -- 172.21.2.101:0/239805361 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c6000c0c0 con 0x7f8c640d1c80 2021-02-10T11:45:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.065 7f8c57fff700 1 -- 172.21.2.101:0/239805361 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6000c340 con 0x7f8c640d1c80 2021-02-10T11:45:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.065 7f8c57fff700 1 -- 172.21.2.101:0/239805361 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c6000c860 con 0x7f8c640d1c80 2021-02-10T11:45:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.065 7f8c6b67b700 1 -- 172.21.2.101:0/239805361 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c640d1c80 msgr2=0x7f8c640cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.065 7f8c6b67b700 1 --2- 172.21.2.101:0/239805361 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c640d1c80 0x7f8c640cef70 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 rx=0x7f8c60006700 tx=0x7f8c60009b80).stop 2021-02-10T11:45:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.066 7f8c6b67b700 1 -- 172.21.2.101:0/239805361 shutdown_connections 2021-02-10T11:45:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.066 7f8c6b67b700 1 --2- 172.21.2.101:0/239805361 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c640cf590 0x7f8c640cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.066 7f8c6b67b700 1 --2- 172.21.2.101:0/239805361 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c640d1c80 0x7f8c640cef70 unknown :-1 s=CLOSED pgs=2814 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.066 7f8c6b67b700 1 -- 172.21.2.101:0/239805361 >> 172.21.2.101:0/239805361 conn(0x7f8c640ce430 msgr2=0x7f8c640d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:19.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.066 7f8c6b67b700 1 -- 172.21.2.101:0/239805361 shutdown_connections 2021-02-10T11:45:19.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.066 7f8c6b67b700 1 -- 172.21.2.101:0/239805361 wait complete. 2021-02-10T11:45:19.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.080 7f8c6b67b700 1 Processor -- start 2021-02-10T11:45:19.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.080 7f8c6b67b700 1 -- start start 2021-02-10T11:45:19.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.081 7f8c6b67b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c6419e5b0 0x7f8c641afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:19.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.081 7f8c6b67b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c6419e9b0 0x7f8c641b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:19.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.081 7f8c6b67b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c641b0250 0x7f8c641b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:19.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.081 7f8c6b67b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c64195bd0 con 0x7f8c6419e9b0 2021-02-10T11:45:19.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.081 7f8c6b67b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c641b0840 con 0x7f8c6419e5b0 2021-02-10T11:45:19.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.081 7f8c6b67b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6419eed0 con 0x7f8c641b0250 2021-02-10T11:45:19.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.081 7f8c68c96700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c6419e9b0 0x7f8c641b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:19.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.081 7f8c68c96700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c6419e9b0 0x7f8c641b0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60526/0 (socket says 172.21.2.101:60526) 2021-02-10T11:45:19.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.081 7f8c68c96700 1 -- 172.21.2.101:0/1936370577 learned_addr learned my addr 172.21.2.101:0/1936370577 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:19.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.081 7f8c69c98700 1 --2- 172.21.2.101:0/1936370577 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c641b0250 0x7f8c641b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:19.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.081 7f8c69497700 1 --2- 172.21.2.101:0/1936370577 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c6419e5b0 0x7f8c641afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:19.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.081 7f8c68c96700 1 -- 172.21.2.101:0/1936370577 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c641b0250 msgr2=0x7f8c641b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:19.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.081 7f8c68c96700 1 --2- 172.21.2.101:0/1936370577 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c641b0250 0x7f8c641b13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:19.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.081 7f8c68c96700 1 -- 172.21.2.101:0/1936370577 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c6419e5b0 msgr2=0x7f8c641afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:19.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.081 7f8c68c96700 1 --2- 172.21.2.101:0/1936370577 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c6419e5b0 0x7f8c641afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:19.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.081 7f8c68c96700 1 -- 172.21.2.101:0/1936370577 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6419f0b0 con 0x7f8c6419e9b0 2021-02-10T11:45:19.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.081 7f8c68c96700 1 --2- 172.21.2.101:0/1936370577 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c6419e9b0 0x7f8c641b0dc0 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 rx=0x7f8c6000b5e0 tx=0x7f8c6000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:19.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.081 7f8c55ffb700 1 -- 172.21.2.101:0/1936370577 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c6000bd70 con 0x7f8c6419e9b0 2021-02-10T11:45:19.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.081 7f8c55ffb700 1 -- 172.21.2.101:0/1936370577 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c60002030 con 0x7f8c6419e9b0 2021-02-10T11:45:19.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.081 7f8c6b67b700 1 -- 172.21.2.101:0/1936370577 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6419f0b0 con 0x7f8c6419e9b0 2021-02-10T11:45:19.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.082 7f8c55ffb700 1 -- 172.21.2.101:0/1936370577 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c600024d0 con 0x7f8c6419e9b0 2021-02-10T11:45:19.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.082 7f8c6b67b700 1 -- 172.21.2.101:0/1936370577 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c6419f0b0 con 0x7f8c6419e9b0 2021-02-10T11:45:19.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.082 7f8c55ffb700 1 -- 172.21.2.101:0/1936370577 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8c60002210 con 0x7f8c6419e9b0 2021-02-10T11:45:19.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.082 7f8c55ffb700 1 --2- 172.21.2.101:0/1936370577 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c44018730 0x7f8c4401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:19.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.082 7f8c55ffb700 1 -- 172.21.2.101:0/1936370577 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8c60032680 con 0x7f8c6419e9b0 2021-02-10T11:45:19.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.082 7f8c69497700 1 --2- 172.21.2.101:0/1936370577 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c44018730 0x7f8c4401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:19.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.083 7f8c69497700 1 --2- 172.21.2.101:0/1936370577 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c44018730 0x7f8c4401abc0 crc :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:19.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.114 7f8c6b67b700 1 -- 172.21.2.101:0/1936370577 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c6419f0b0 con 0x7f8c6419e9b0 2021-02-10T11:45:19.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.116 7f8c55ffb700 1 -- 172.21.2.101:0/1936370577 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8c60002240 con 0x7f8c6419e9b0 2021-02-10T11:45:19.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.252 7f8c6b67b700 1 -- 172.21.2.101:0/1936370577 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8c6419f0b0 con 0x7f8c6419e9b0 2021-02-10T11:45:19.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.622 7f8c55ffb700 1 -- 172.21.2.101:0/1936370577 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v132) v1 ==== 136+0+0 (secure 0 0 0) 0x7f8c60002030 con 0x7f8c6419e9b0 2021-02-10T11:45:19.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.670 7f8c6b67b700 1 -- 172.21.2.101:0/1936370577 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c44018730 msgr2=0x7f8c4401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:19.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.670 7f8c6b67b700 1 --2- 172.21.2.101:0/1936370577 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c44018730 0x7f8c4401abc0 crc :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:19.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.670 7f8c6b67b700 1 -- 172.21.2.101:0/1936370577 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c6419e9b0 msgr2=0x7f8c641b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:19.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.670 7f8c6b67b700 1 --2- 172.21.2.101:0/1936370577 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c6419e9b0 0x7f8c641b0dc0 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 rx=0x7f8c6000b5e0 tx=0x7f8c6000b980).stop 2021-02-10T11:45:19.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.670 7f8c6b67b700 1 -- 172.21.2.101:0/1936370577 shutdown_connections 2021-02-10T11:45:19.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.670 7f8c6b67b700 1 --2- 172.21.2.101:0/1936370577 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c44018730 0x7f8c4401abc0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:19.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.670 7f8c6b67b700 1 --2- 172.21.2.101:0/1936370577 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c641b0250 0x7f8c641b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:19.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.670 7f8c6b67b700 1 --2- 172.21.2.101:0/1936370577 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c6419e9b0 0x7f8c641b0dc0 unknown :-1 s=CLOSED pgs=2815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:19.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.670 7f8c6b67b700 1 --2- 172.21.2.101:0/1936370577 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c6419e5b0 0x7f8c641afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:19.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.670 7f8c6b67b700 1 -- 172.21.2.101:0/1936370577 >> 172.21.2.101:0/1936370577 conn(0x7f8c6419f940 msgr2=0x7f8c6418c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:19.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.670 7f8c6b67b700 1 -- 172.21.2.101:0/1936370577 shutdown_connections 2021-02-10T11:45:19.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:19.670 7f8c6b67b700 1 -- 172.21.2.101:0/1936370577 wait complete. 2021-02-10T11:45:19.676 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs volume ls 2021-02-10T11:45:20.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.090 7f15cc1cb700 1 Processor -- start 2021-02-10T11:45:20.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.090 7f15cc1cb700 1 -- start start 2021-02-10T11:45:20.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.090 7f15cc1cb700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f15c40d1870 0x7f15c41b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:20.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.090 7f15cc1cb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15c40f0650 0x7f15c40cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:20.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.090 7f15cc1cb700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15c40f0b10 con 0x7f15c41b0a10 2021-02-10T11:45:20.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.090 7f15cc1cb700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15c40e5ed0 con 0x7f15c40d1870 2021-02-10T11:45:20.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.090 7f15cc1cb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c40d1c70 con 0x7f15c40f0650 2021-02-10T11:45:20.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.091 7f15c97e6700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f15c41b0a10 0x7f15c40cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:54438/0 (socket says 172.21.2.101:54438) 2021-02-10T11:45:20.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.091 7f15c97e6700 1 -- 172.21.2.101:0/917371488 learned_addr learned my addr 172.21.2.101:0/917371488 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:20.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.091 7f15c9fe7700 1 --2- 172.21.2.101:0/917371488 >> v2:172.21.2.108:3300/0 conn(0x7f15c40d1870 0x7f15c41b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:20.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.091 7f15ca7e8700 1 --2- 172.21.2.101:0/917371488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15c40f0650 0x7f15c40cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:20.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.092 7f15c9fe7700 1 -- 172.21.2.101:0/917371488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15c40f0650 msgr2=0x7f15c40cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:20.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.092 7f15c9fe7700 1 --2- 172.21.2.101:0/917371488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15c40f0650 0x7f15c40cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:20.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.092 7f15c9fe7700 1 -- 172.21.2.101:0/917371488 >> v1:172.21.2.108:6789/0 conn(0x7f15c41b0a10 legacy=0x7f15c40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:20.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.092 7f15c9fe7700 1 -- 172.21.2.101:0/917371488 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c40d1e90 con 0x7f15c40d1870 2021-02-10T11:45:20.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.092 7f15c8fe5700 1 -- 172.21.2.101:0/917371488 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901036928 0 0) 0x7f15b40010b0 con 0x7f15c41b0a10 2021-02-10T11:45:20.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.092 7f15c9fe7700 1 --2- 172.21.2.101:0/917371488 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15c40d1870 0x7f15c41b03f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f15bc0069d0 tx=0x7f15bc009dc0).ready entity=mon.1 client_cookie=60ca309ef154d4d7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:20.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.092 7f15c8fe5700 1 -- 172.21.2.101:0/917371488 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f15bc00b510 con 0x7f15c40d1870 2021-02-10T11:45:20.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.092 7f15c8fe5700 1 -- 172.21.2.101:0/917371488 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15bc006ac0 con 0x7f15c40d1870 2021-02-10T11:45:20.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.092 7f15c8fe5700 1 -- 172.21.2.101:0/917371488 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f15bc00bb10 con 0x7f15c40d1870 2021-02-10T11:45:20.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.092 7f15cc1cb700 1 -- 172.21.2.101:0/917371488 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15c40d1870 msgr2=0x7f15c41b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:20.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.092 7f15cc1cb700 1 --2- 172.21.2.101:0/917371488 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15c40d1870 0x7f15c41b03f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f15bc0069d0 tx=0x7f15bc009dc0).stop 2021-02-10T11:45:20.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.093 7f15cc1cb700 1 -- 172.21.2.101:0/917371488 shutdown_connections 2021-02-10T11:45:20.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.093 7f15cc1cb700 1 --2- 172.21.2.101:0/917371488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15c40f0650 0x7f15c40cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:20.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.093 7f15cc1cb700 1 --2- 172.21.2.101:0/917371488 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15c40d1870 0x7f15c41b03f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:20.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.093 7f15cc1cb700 1 -- 172.21.2.101:0/917371488 >> 172.21.2.101:0/917371488 conn(0x7f15c40ce450 msgr2=0x7f15c40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:20.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.093 7f15cc1cb700 1 -- 172.21.2.101:0/917371488 shutdown_connections 2021-02-10T11:45:20.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.093 7f15cc1cb700 1 -- 172.21.2.101:0/917371488 wait complete. 2021-02-10T11:45:20.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.108 7f15cc1cb700 1 Processor -- start 2021-02-10T11:45:20.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.108 7f15cc1cb700 1 -- start start 2021-02-10T11:45:20.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.108 7f15cc1cb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15c419ffa0 0x7f15c40eda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:20.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.108 7f15cc1cb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15c41a03a0 0x7f15c40ee030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:20.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.108 7f15cc1cb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15c40ee650 0x7f15c40ebe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:20.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.108 7f15cc1cb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c40ed490 con 0x7f15c40ee650 2021-02-10T11:45:20.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.108 7f15cc1cb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c40eea50 con 0x7f15c419ffa0 2021-02-10T11:45:20.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.108 7f15cc1cb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c40eec30 con 0x7f15c41a03a0 2021-02-10T11:45:20.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.108 7f15ca7e8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15c40ee650 0x7f15c40ebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:20.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.108 7f15ca7e8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15c40ee650 0x7f15c40ebe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60540/0 (socket says 172.21.2.101:60540) 2021-02-10T11:45:20.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.108 7f15ca7e8700 1 -- 172.21.2.101:0/395786711 learned_addr learned my addr 172.21.2.101:0/395786711 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:20.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.109 7f15ca7e8700 1 -- 172.21.2.101:0/395786711 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15c41a03a0 msgr2=0x7f15c40ee030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:20.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.109 7f15ca7e8700 1 --2- 172.21.2.101:0/395786711 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15c41a03a0 0x7f15c40ee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:20.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.109 7f15ca7e8700 1 -- 172.21.2.101:0/395786711 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15c419ffa0 msgr2=0x7f15c40eda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:20.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.109 7f15ca7e8700 1 --2- 172.21.2.101:0/395786711 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15c419ffa0 0x7f15c40eda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:20.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.109 7f15ca7e8700 1 -- 172.21.2.101:0/395786711 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c40ec4a0 con 0x7f15c40ee650 2021-02-10T11:45:20.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.109 7f15ca7e8700 1 --2- 172.21.2.101:0/395786711 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15c40ee650 0x7f15c40ebe80 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 rx=0x7f15c000b5e0 tx=0x7f15c000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:20.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.109 7f15ba7fc700 1 -- 172.21.2.101:0/395786711 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f15c0000c10 con 0x7f15c40ee650 2021-02-10T11:45:20.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.109 7f15ba7fc700 1 -- 172.21.2.101:0/395786711 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c000bd70 con 0x7f15c40ee650 2021-02-10T11:45:20.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.109 7f15ba7fc700 1 -- 172.21.2.101:0/395786711 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f15c0002560 con 0x7f15c40ee650 2021-02-10T11:45:20.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.109 7f15cc1cb700 1 -- 172.21.2.101:0/395786711 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c40ec4a0 con 0x7f15c40ee650 2021-02-10T11:45:20.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.109 7f15cc1cb700 1 -- 172.21.2.101:0/395786711 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c40ec4a0 con 0x7f15c40ee650 2021-02-10T11:45:20.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.110 7f15ba7fc700 1 -- 172.21.2.101:0/395786711 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f15c0006720 con 0x7f15c40ee650 2021-02-10T11:45:20.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.110 7f15ba7fc700 1 --2- 172.21.2.101:0/395786711 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15ac018730 0x7f15ac01abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:20.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.110 7f15ba7fc700 1 -- 172.21.2.101:0/395786711 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f15c0032800 con 0x7f15c40ee650 2021-02-10T11:45:20.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.110 7f15c9fe7700 1 --2- 172.21.2.101:0/395786711 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15ac018730 0x7f15ac01abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:20.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.110 7f15c9fe7700 1 --2- 172.21.2.101:0/395786711 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15ac018730 0x7f15ac01abf0 crc :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:20.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.140 7f15cc1cb700 1 -- 172.21.2.101:0/395786711 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15c40ebc50 con 0x7f15c40ee650 2021-02-10T11:45:20.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.143 7f15ba7fc700 1 -- 172.21.2.101:0/395786711 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f15c0006370 con 0x7f15c40ee650 2021-02-10T11:45:20.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.279 7f15cc1cb700 1 -- 172.21.2.101:0/395786711 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs volume ls", "target": ["mgr", ""]}) v1 -- 0x7f15c40ebc50 con 0x7f15ac018730 2021-02-10T11:45:20.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.280 7f15ba7fc700 1 -- 172.21.2.101:0/395786711 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7f15bc000d30 con 0x7f15ac018730 2021-02-10T11:45:20.279 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:45:20.279 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:20.279 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "cephfs" 2021-02-10T11:45:20.279 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:20.279 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:45:20.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.282 7f15cc1cb700 1 -- 172.21.2.101:0/395786711 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15ac018730 msgr2=0x7f15ac01abf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:20.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.282 7f15cc1cb700 1 --2- 172.21.2.101:0/395786711 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15ac018730 0x7f15ac01abf0 crc :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:20.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.282 7f15cc1cb700 1 -- 172.21.2.101:0/395786711 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15c40ee650 msgr2=0x7f15c40ebe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:20.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.282 7f15cc1cb700 1 --2- 172.21.2.101:0/395786711 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15c40ee650 0x7f15c40ebe80 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 rx=0x7f15c000b5e0 tx=0x7f15c000b980).stop 2021-02-10T11:45:20.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.282 7f15cc1cb700 1 -- 172.21.2.101:0/395786711 shutdown_connections 2021-02-10T11:45:20.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.282 7f15cc1cb700 1 --2- 172.21.2.101:0/395786711 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f15ac018730 0x7f15ac01abf0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:20.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.282 7f15cc1cb700 1 --2- 172.21.2.101:0/395786711 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f15c41a03a0 0x7f15c40ee030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:20.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.282 7f15cc1cb700 1 --2- 172.21.2.101:0/395786711 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f15c40ee650 0x7f15c40ebe80 unknown :-1 s=CLOSED pgs=2816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:20.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.282 7f15cc1cb700 1 --2- 172.21.2.101:0/395786711 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f15c419ffa0 0x7f15c40eda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:20.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.282 7f15cc1cb700 1 -- 172.21.2.101:0/395786711 >> 172.21.2.101:0/395786711 conn(0x7f15c4094d90 msgr2=0x7f15c408a420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:20.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.282 7f15cc1cb700 1 -- 172.21.2.101:0/395786711 shutdown_connections 2021-02-10T11:45:20.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.282 7f15cc1cb700 1 -- 172.21.2.101:0/395786711 wait complete. 2021-02-10T11:45:20.289 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:45:20.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.707 7f8331181700 1 Processor -- start 2021-02-10T11:45:20.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.707 7f8331181700 1 -- start start 2021-02-10T11:45:20.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.708 7f8331181700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f832c10afe0 0x7f832c0cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:20.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.708 7f8331181700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832c0fe1b0 con 0x7f832c0d1c60 2021-02-10T11:45:20.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.708 7f8331181700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832c1b0ca0 con 0x7f832c0d1860 2021-02-10T11:45:20.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.708 7f8331181700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f832c109a50 con 0x7f832c10afe0 2021-02-10T11:45:20.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.708 7f832a59c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f832c0d1c60 0x7f832c0cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:32810/0 (socket says 172.21.2.101:32810) 2021-02-10T11:45:20.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.708 7f832a59c700 1 -- 172.21.2.101:0/3425784801 learned_addr learned my addr 172.21.2.101:0/3425784801 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:20.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.708 7f832b59e700 1 --2- 172.21.2.101:0/3425784801 >> v2:172.21.2.108:3300/0 conn(0x7f832c10afe0 0x7f832c0cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:20.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.708 7f8329d9b700 1 -- 172.21.2.101:0/3425784801 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706004732 0 0) 0x7f83140010b0 con 0x7f832c0d1c60 2021-02-10T11:45:20.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.708 7f8329d9b700 1 -- 172.21.2.101:0/3425784801 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8318002fe0 con 0x7f832c0d1c60 2021-02-10T11:45:20.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.709 7f832b59e700 1 -- 172.21.2.101:0/3425784801 >> v1:172.21.2.108:6789/0 conn(0x7f832c0d1860 legacy=0x7f832c1b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:20.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.709 7f832b59e700 1 -- 172.21.2.101:0/3425784801 >> v1:172.21.2.101:6789/0 conn(0x7f832c0d1c60 legacy=0x7f832c0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:20.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.709 7f832b59e700 1 -- 172.21.2.101:0/3425784801 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c10b3e0 con 0x7f832c10afe0 2021-02-10T11:45:20.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.709 7f832b59e700 1 --2- 172.21.2.101:0/3425784801 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f832c10afe0 0x7f832c0cf740 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f8324006ad0 tx=0x7f8324009a50).ready entity=mon.1 client_cookie=34bf7d6a3c8879b1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:20.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.709 7f8329d9b700 1 -- 172.21.2.101:0/3425784801 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f83240066a0 con 0x7f832c10afe0 2021-02-10T11:45:20.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.709 7f8329d9b700 1 -- 172.21.2.101:0/3425784801 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8324006410 con 0x7f832c10afe0 2021-02-10T11:45:20.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.709 7f8329d9b700 1 -- 172.21.2.101:0/3425784801 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f83240024d0 con 0x7f832c10afe0 2021-02-10T11:45:20.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.709 7f8331181700 1 -- 172.21.2.101:0/3425784801 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f832c10afe0 msgr2=0x7f832c0cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:20.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.709 7f8331181700 1 --2- 172.21.2.101:0/3425784801 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f832c10afe0 0x7f832c0cf740 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f8324006ad0 tx=0x7f8324009a50).stop 2021-02-10T11:45:20.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.710 7f8331181700 1 -- 172.21.2.101:0/3425784801 shutdown_connections 2021-02-10T11:45:20.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.710 7f8331181700 1 --2- 172.21.2.101:0/3425784801 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f832c10afe0 0x7f832c0cf740 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:20.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.710 7f8331181700 1 -- 172.21.2.101:0/3425784801 >> 172.21.2.101:0/3425784801 conn(0x7f832c0ce410 msgr2=0x7f832c0d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:20.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.710 7f8331181700 1 -- 172.21.2.101:0/3425784801 shutdown_connections 2021-02-10T11:45:20.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.710 7f8331181700 1 -- 172.21.2.101:0/3425784801 wait complete. 2021-02-10T11:45:20.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.725 7f8331181700 1 Processor -- start 2021-02-10T11:45:20.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.725 7f8331181700 1 -- start start 2021-02-10T11:45:20.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.725 7f8331181700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f832c0bfa10 0x7f832c0b82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:20.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.725 7f8331181700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f832c1a4a70 0x7f832c094c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:20.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.725 7f8331181700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f832c0c0410 0x7f832c0c0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:20.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.725 7f8331181700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0c3a50 con 0x7f832c0bfa10 2021-02-10T11:45:20.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.725 7f8331181700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0b88c0 con 0x7f832c1a4a70 2021-02-10T11:45:20.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.725 7f8331181700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c095260 con 0x7f832c0c0410 2021-02-10T11:45:20.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.725 7f832ad9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f832c0bfa10 0x7f832c0b82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:20.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.725 7f832ad9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f832c0bfa10 0x7f832c0b82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60554/0 (socket says 172.21.2.101:60554) 2021-02-10T11:45:20.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.725 7f832ad9d700 1 -- 172.21.2.101:0/2808976268 learned_addr learned my addr 172.21.2.101:0/2808976268 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:20.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.726 7f832ad9d700 1 -- 172.21.2.101:0/2808976268 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f832c0c0410 msgr2=0x7f832c0c0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:20.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.726 7f832ad9d700 1 --2- 172.21.2.101:0/2808976268 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f832c0c0410 0x7f832c0c0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:20.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.726 7f832ad9d700 1 -- 172.21.2.101:0/2808976268 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f832c1a4a70 msgr2=0x7f832c094c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:20.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.726 7f832ad9d700 1 --2- 172.21.2.101:0/2808976268 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f832c1a4a70 0x7f832c094c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:20.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.726 7f832ad9d700 1 -- 172.21.2.101:0/2808976268 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c0c3540 con 0x7f832c0bfa10 2021-02-10T11:45:20.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.726 7f832ad9d700 1 --2- 172.21.2.101:0/2808976268 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f832c0bfa10 0x7f832c0b82a0 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 rx=0x7f831c00b8c0 tx=0x7f831c00bc60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:20.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.726 7f83237fe700 1 -- 172.21.2.101:0/2808976268 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f831c0058a0 con 0x7f832c0bfa10 2021-02-10T11:45:20.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.726 7f83237fe700 1 -- 172.21.2.101:0/2808976268 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c003210 con 0x7f832c0bfa10 2021-02-10T11:45:20.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.726 7f83237fe700 1 -- 172.21.2.101:0/2808976268 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f831c005de0 con 0x7f832c0bfa10 2021-02-10T11:45:20.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.726 7f8331181700 1 -- 172.21.2.101:0/2808976268 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832c0c3540 con 0x7f832c0bfa10 2021-02-10T11:45:20.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.726 7f8331181700 1 -- 172.21.2.101:0/2808976268 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832c0c1120 con 0x7f832c0bfa10 2021-02-10T11:45:20.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.727 7f83237fe700 1 -- 172.21.2.101:0/2808976268 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f831c006240 con 0x7f832c0bfa10 2021-02-10T11:45:20.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.727 7f83237fe700 1 --2- 172.21.2.101:0/2808976268 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83100186c0 0x7f831001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:20.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.727 7f83237fe700 1 -- 172.21.2.101:0/2808976268 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f831c0328b0 con 0x7f832c0bfa10 2021-02-10T11:45:20.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.727 7f832a59c700 1 --2- 172.21.2.101:0/2808976268 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83100186c0 0x7f831001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:20.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.727 7f832a59c700 1 --2- 172.21.2.101:0/2808976268 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83100186c0 0x7f831001abc0 crc :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:20.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.758 7f8331181700 1 -- 172.21.2.101:0/2808976268 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f832c0bfee0 con 0x7f832c0bfa10 2021-02-10T11:45:20.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.760 7f83237fe700 1 -- 172.21.2.101:0/2808976268 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f831c006030 con 0x7f832c0bfa10 2021-02-10T11:45:20.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.909 7f8331181700 1 -- 172.21.2.101:0/2808976268 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f832c0bfee0 con 0x7f832c0bfa10 2021-02-10T11:45:20.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.909 7f83237fe700 1 -- 172.21.2.101:0/2808976268 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) v1 ==== 122+0+0 (secure 0 0 0) 0x7f831c001ee0 con 0x7f832c0bfa10 2021-02-10T11:45:20.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.911 7f8331181700 1 -- 172.21.2.101:0/2808976268 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83100186c0 msgr2=0x7f831001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:20.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.911 7f8331181700 1 --2- 172.21.2.101:0/2808976268 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83100186c0 0x7f831001abc0 crc :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:20.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.911 7f8331181700 1 -- 172.21.2.101:0/2808976268 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f832c0bfa10 msgr2=0x7f832c0b82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:20.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.911 7f8331181700 1 --2- 172.21.2.101:0/2808976268 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f832c0bfa10 0x7f832c0b82a0 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 rx=0x7f831c00b8c0 tx=0x7f831c00bc60).stop 2021-02-10T11:45:20.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.911 7f8331181700 1 -- 172.21.2.101:0/2808976268 shutdown_connections 2021-02-10T11:45:20.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.911 7f8331181700 1 --2- 172.21.2.101:0/2808976268 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f83100186c0 0x7f831001abc0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:20.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.911 7f8331181700 1 --2- 172.21.2.101:0/2808976268 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f832c0c0410 0x7f832c0c0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:20.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.911 7f8331181700 1 --2- 172.21.2.101:0/2808976268 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f832c1a4a70 0x7f832c094c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:20.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.911 7f8331181700 1 --2- 172.21.2.101:0/2808976268 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f832c0bfa10 0x7f832c0b82a0 unknown :-1 s=CLOSED pgs=2818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:20.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.911 7f8331181700 1 -- 172.21.2.101:0/2808976268 >> 172.21.2.101:0/2808976268 conn(0x7f832c0a6a80 msgr2=0x7f832c0ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:20.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.911 7f8331181700 1 -- 172.21.2.101:0/2808976268 shutdown_connections 2021-02-10T11:45:20.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:20.911 7f8331181700 1 -- 172.21.2.101:0/2808976268 wait complete. 2021-02-10T11:45:20.918 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs non_existent_subvolume 2021-02-10T11:45:21.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.335 7f593432d700 1 Processor -- start 2021-02-10T11:45:21.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.335 7f593432d700 1 -- start start 2021-02-10T11:45:21.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.335 7f593432d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f592c0d1870 0x7f592c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:21.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.335 7f593432d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f592c1b0ae0 0x7f592c108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:21.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.335 7f593432d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f592c0d1c70 0x7f592c108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:21.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.335 7f593432d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f592c0fe1c0 con 0x7f592c1b0ae0 2021-02-10T11:45:21.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.335 7f593432d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f592c109a60 con 0x7f592c0d1870 2021-02-10T11:45:21.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.335 7f593432d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592c088070 con 0x7f592c0d1c70 2021-02-10T11:45:21.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.335 7f5931948700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f592c1b0ae0 0x7f592c108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:21.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.335 7f5931948700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f592c1b0ae0 0x7f592c108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60560/0 (socket says 172.21.2.101:60560) 2021-02-10T11:45:21.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.335 7f5931948700 1 -- 172.21.2.101:0/2478739307 learned_addr learned my addr 172.21.2.101:0/2478739307 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:21.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.336 7f593294a700 1 --2- 172.21.2.101:0/2478739307 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f592c0d1c70 0x7f592c108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:21.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.336 7f5932149700 1 --2- 172.21.2.101:0/2478739307 >> v2:172.21.2.108:3300/0 conn(0x7f592c0d1870 0x7f592c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:21.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.336 7f5931948700 1 -- 172.21.2.101:0/2478739307 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f592c0d1c70 msgr2=0x7f592c108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:21.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.336 7f5931948700 1 --2- 172.21.2.101:0/2478739307 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f592c0d1c70 0x7f592c108f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:21.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.336 7f5931948700 1 -- 172.21.2.101:0/2478739307 >> v2:172.21.2.108:3300/0 conn(0x7f592c0d1870 msgr2=0x7f592c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:21.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.336 7f5931948700 1 --2- 172.21.2.101:0/2478739307 >> v2:172.21.2.108:3300/0 conn(0x7f592c0d1870 0x7f592c1b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:21.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.336 7f5931948700 1 -- 172.21.2.101:0/2478739307 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592c109580 con 0x7f592c1b0ae0 2021-02-10T11:45:21.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.336 7f5931948700 1 --2- 172.21.2.101:0/2478739307 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f592c1b0ae0 0x7f592c108940 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 rx=0x7f5920006700 tx=0x7f5920009b80).ready entity=mon.0 client_cookie=89c94669d338afdf server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:21.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.336 7f5931147700 1 -- 172.21.2.101:0/2478739307 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f592000c0c0 con 0x7f592c1b0ae0 2021-02-10T11:45:21.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.336 7f5931147700 1 -- 172.21.2.101:0/2478739307 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592000c400 con 0x7f592c1b0ae0 2021-02-10T11:45:21.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.336 7f5931147700 1 -- 172.21.2.101:0/2478739307 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f592000c8e0 con 0x7f592c1b0ae0 2021-02-10T11:45:21.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.336 7f593432d700 1 -- 172.21.2.101:0/2478739307 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f592c1b0ae0 msgr2=0x7f592c108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:21.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.336 7f593432d700 1 --2- 172.21.2.101:0/2478739307 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f592c1b0ae0 0x7f592c108940 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 rx=0x7f5920006700 tx=0x7f5920009b80).stop 2021-02-10T11:45:21.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.337 7f593432d700 1 -- 172.21.2.101:0/2478739307 shutdown_connections 2021-02-10T11:45:21.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.337 7f593432d700 1 --2- 172.21.2.101:0/2478739307 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f592c0d1c70 0x7f592c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:21.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.337 7f593432d700 1 --2- 172.21.2.101:0/2478739307 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f592c1b0ae0 0x7f592c108940 unknown :-1 s=CLOSED pgs=2819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:21.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.337 7f593432d700 1 --2- 172.21.2.101:0/2478739307 >> v2:172.21.2.108:3300/0 conn(0x7f592c0d1870 0x7f592c1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:21.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.337 7f593432d700 1 -- 172.21.2.101:0/2478739307 >> 172.21.2.101:0/2478739307 conn(0x7f592c0ce420 msgr2=0x7f592c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:21.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.337 7f593432d700 1 -- 172.21.2.101:0/2478739307 shutdown_connections 2021-02-10T11:45:21.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.337 7f593432d700 1 -- 172.21.2.101:0/2478739307 wait complete. 2021-02-10T11:45:21.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.352 7f593432d700 1 Processor -- start 2021-02-10T11:45:21.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.352 7f593432d700 1 -- start start 2021-02-10T11:45:21.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.352 7f593432d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f592c09bcc0 0x7f592c09c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:21.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.352 7f593432d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f592c0a0820 0x7f592c0a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:21.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.352 7f593432d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f592c0a1240 0x7f592c0858d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:21.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.352 7f593432d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c0a66b0 con 0x7f592c09bcc0 2021-02-10T11:45:21.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.352 7f593432d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c1a49b0 con 0x7f592c0a0820 2021-02-10T11:45:21.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.352 7f593432d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592c1a4b90 con 0x7f592c0a1240 2021-02-10T11:45:21.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.352 7f5932149700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f592c09bcc0 0x7f592c09c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:21.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.352 7f5932149700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f592c09bcc0 0x7f592c09c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60566/0 (socket says 172.21.2.101:60566) 2021-02-10T11:45:21.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.352 7f5932149700 1 -- 172.21.2.101:0/2511689246 learned_addr learned my addr 172.21.2.101:0/2511689246 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:21.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.353 7f593294a700 1 --2- 172.21.2.101:0/2511689246 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f592c0a1240 0x7f592c0858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:21.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.353 7f5931948700 1 --2- 172.21.2.101:0/2511689246 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f592c0a0820 0x7f592c0a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:21.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.353 7f5932149700 1 -- 172.21.2.101:0/2511689246 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f592c0a1240 msgr2=0x7f592c0858d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:21.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.353 7f5932149700 1 --2- 172.21.2.101:0/2511689246 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f592c0a1240 0x7f592c0858d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:21.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.353 7f5932149700 1 -- 172.21.2.101:0/2511689246 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f592c0a0820 msgr2=0x7f592c0a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:21.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.353 7f5932149700 1 --2- 172.21.2.101:0/2511689246 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f592c0a0820 0x7f592c0a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:21.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.353 7f5932149700 1 -- 172.21.2.101:0/2511689246 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592c0a1640 con 0x7f592c09bcc0 2021-02-10T11:45:21.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.353 7f593294a700 1 --2- 172.21.2.101:0/2511689246 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f592c0a1240 0x7f592c0858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:21.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.353 7f5931948700 1 --2- 172.21.2.101:0/2511689246 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f592c0a0820 0x7f592c0a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:21.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.353 7f5932149700 1 --2- 172.21.2.101:0/2511689246 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f592c09bcc0 0x7f592c09c0c0 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 rx=0x7f5928006f00 tx=0x7f592800b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:21.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.353 7f591effd700 1 -- 172.21.2.101:0/2511689246 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f592800bd30 con 0x7f592c09bcc0 2021-02-10T11:45:21.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.353 7f591effd700 1 -- 172.21.2.101:0/2511689246 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5928002030 con 0x7f592c09bcc0 2021-02-10T11:45:21.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.353 7f593432d700 1 -- 172.21.2.101:0/2511689246 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c0a1640 con 0x7f592c09bcc0 2021-02-10T11:45:21.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.353 7f593432d700 1 -- 172.21.2.101:0/2511689246 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c0861e0 con 0x7f592c09bcc0 2021-02-10T11:45:21.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.353 7f591effd700 1 -- 172.21.2.101:0/2511689246 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5928002480 con 0x7f592c09bcc0 2021-02-10T11:45:21.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.354 7f591effd700 1 -- 172.21.2.101:0/2511689246 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5928000c90 con 0x7f592c09bcc0 2021-02-10T11:45:21.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.354 7f591effd700 1 --2- 172.21.2.101:0/2511689246 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5914018730 0x7f591401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:21.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.354 7f591effd700 1 -- 172.21.2.101:0/2511689246 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f59280327e0 con 0x7f592c09bcc0 2021-02-10T11:45:21.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.354 7f5931948700 1 --2- 172.21.2.101:0/2511689246 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5914018730 0x7f591401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:21.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.354 7f5931948700 1 --2- 172.21.2.101:0/2511689246 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5914018730 0x7f591401ac10 crc :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:21.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.385 7f593432d700 1 -- 172.21.2.101:0/2511689246 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592c086a70 con 0x7f592c09bcc0 2021-02-10T11:45:21.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.388 7f591effd700 1 -- 172.21.2.101:0/2511689246 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f59280009b0 con 0x7f592c09bcc0 2021-02-10T11:45:21.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.516 7f593432d700 1 -- 172.21.2.101:0/2511689246 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "non_existent_subvolume", "target": ["mgr", ""]}) v1 -- 0x7f592c0861e0 con 0x7f5914018730 2021-02-10T11:45:21.519 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:45:21.521 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:45:21.520 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:45:21.521 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:45:21.520 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:45:21.521 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:45:21.520 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:45:21.521 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:45:21.520 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:45:21.521 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:45:21.527 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:45:21.527 7f2cc39e1700 -1 mgr.server reply reply (2) No such file or directory subvolume 'non_existent_subvolume' does not exist 2021-02-10T11:45:21.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.527 7f591effd700 1 -- 172.21.2.101:0/2511689246 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: -2 subvolume 'non_existent_subvolume' does not exist) v1 ==== 57+0+0 (crc 0 0 0) 0x7f5920006810 con 0x7f5914018730 2021-02-10T11:45:21.530 INFO:teuthology.orchestra.run.gibba001.stderr:Error ENOENT: subvolume 'non_existent_subvolume' does not exist 2021-02-10T11:45:21.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.534 7f593432d700 1 -- 172.21.2.101:0/2511689246 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5914018730 msgr2=0x7f591401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:21.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.534 7f593432d700 1 --2- 172.21.2.101:0/2511689246 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5914018730 0x7f591401ac10 crc :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:21.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.534 7f593432d700 1 -- 172.21.2.101:0/2511689246 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f592c09bcc0 msgr2=0x7f592c09c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:21.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.534 7f593432d700 1 --2- 172.21.2.101:0/2511689246 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f592c09bcc0 0x7f592c09c0c0 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 rx=0x7f5928006f00 tx=0x7f592800b940).stop 2021-02-10T11:45:21.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.534 7f593432d700 1 -- 172.21.2.101:0/2511689246 shutdown_connections 2021-02-10T11:45:21.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.534 7f593432d700 1 --2- 172.21.2.101:0/2511689246 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5914018730 0x7f591401ac10 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:21.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.534 7f593432d700 1 --2- 172.21.2.101:0/2511689246 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f592c0a1240 0x7f592c0858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:21.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.534 7f593432d700 1 --2- 172.21.2.101:0/2511689246 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f592c0a0820 0x7f592c0a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:21.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.534 7f593432d700 1 --2- 172.21.2.101:0/2511689246 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f592c09bcc0 0x7f592c09c0c0 unknown :-1 s=CLOSED pgs=2820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:21.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.534 7f593432d700 1 -- 172.21.2.101:0/2511689246 >> 172.21.2.101:0/2511689246 conn(0x7f592c0bef00 msgr2=0x7f592c09d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:21.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.534 7f593432d700 1 -- 172.21.2.101:0/2511689246 shutdown_connections 2021-02-10T11:45:21.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:21.534 7f593432d700 1 -- 172.21.2.101:0/2511689246 wait complete. 2021-02-10T11:45:21.540 DEBUG:teuthology.orchestra.run:got remote process result: 2 2021-02-10T11:45:21.541 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:45:21.541 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:45:21.887 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:45:22.233 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:45:22.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:22.233 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:45:22.233 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:45:22.569 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:45:22.569 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:45:22.570 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:45:22.570 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:45:22.570 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:45:22.926 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[166055]: fuse finished with error 0 and tester_r 0 2021-02-10T11:45:22.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:22.956 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:45:22.956 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:45:23.291 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:45:23.292 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:45:23.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:45:23.292 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:45:23.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:23.628 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:45:23.628 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:45:23.964 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:45:23.964 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:45:23.964 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:45:23.964 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:45:23.964 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:45:24.316 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[166696]: fuse finished with error 0 and tester_r 0 2021-02-10T11:45:24.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:24.318 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:45:24.318 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:45:24.653 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:45:24.654 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:45:24.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:45:24.654 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:45:24.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:24.989 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:45:24.989 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:45:25.324 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:45:25.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:45:25.325 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:45:25.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:45:25.325 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:45:25.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:25.661 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:45:25.661 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:45:25.995 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:45:25.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:45:25.995 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:45:25.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:45:25.995 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:45:26.328 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-02-10T11:45:26.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.747 7f50fd3b5700 1 Processor -- start 2021-02-10T11:45:26.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.747 7f50fd3b5700 1 -- start start 2021-02-10T11:45:26.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.747 7f50fd3b5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50f80d1870 0x7f50f81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:26.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.747 7f50fd3b5700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f50f80f0650 0x7f50f80cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:26.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.747 7f50fd3b5700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f80f0b10 con 0x7f50f81b0a10 2021-02-10T11:45:26.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.747 7f50fd3b5700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f80fe230 con 0x7f50f80f0650 2021-02-10T11:45:26.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.747 7f50fd3b5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f80e5ed0 con 0x7f50f80d1870 2021-02-10T11:45:26.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.747 7f50f687c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f50f81b0a10 0x7f50f80cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:32846/0 (socket says 172.21.2.101:32846) 2021-02-10T11:45:26.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.747 7f50f687c700 1 -- 172.21.2.101:0/1120446815 learned_addr learned my addr 172.21.2.101:0/1120446815 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:26.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.748 7f50f707d700 1 --2- 172.21.2.101:0/1120446815 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50f80d1870 0x7f50f81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:26.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.748 7f50f607b700 1 -- 172.21.2.101:0/1120446815 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1719578427 0 0) 0x7f50e80009b0 con 0x7f50f81b0a10 2021-02-10T11:45:26.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.748 7f50f787e700 1 --2- 172.21.2.101:0/1120446815 >> v2:172.21.2.108:3300/0 conn(0x7f50f80f0650 0x7f50f80cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:26.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.748 7f50f607b700 1 -- 172.21.2.101:0/1120446815 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50e4002fe0 con 0x7f50f81b0a10 2021-02-10T11:45:26.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.748 7f50f607b700 1 -- 172.21.2.101:0/1120446815 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3326385118 0 0) 0x7f50e8001070 con 0x7f50f81b0a10 2021-02-10T11:45:26.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.748 7f50f607b700 1 -- 172.21.2.101:0/1120446815 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50f80d1870 msgr2=0x7f50f81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:26.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.748 7f50f607b700 1 --2- 172.21.2.101:0/1120446815 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50f80d1870 0x7f50f81b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:26.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.748 7f50f607b700 1 -- 172.21.2.101:0/1120446815 >> v2:172.21.2.108:3300/0 conn(0x7f50f80f0650 msgr2=0x7f50f80cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:26.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.748 7f50f607b700 1 --2- 172.21.2.101:0/1120446815 >> v2:172.21.2.108:3300/0 conn(0x7f50f80f0650 0x7f50f80cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:26.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.748 7f50f607b700 1 -- 172.21.2.101:0/1120446815 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f80f20a0 con 0x7f50f81b0a10 2021-02-10T11:45:26.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.748 7f50f607b700 1 -- 172.21.2.101:0/1120446815 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f50e8001570 con 0x7f50f81b0a10 2021-02-10T11:45:26.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.748 7f50f607b700 1 -- 172.21.2.101:0/1120446815 >> v1:172.21.2.101:6789/0 conn(0x7f50f81b0a10 legacy=0x7f50f80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:26.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.748 7f50f607b700 1 --2- 172.21.2.101:0/1120446815 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50e4003b10 0x7f50e4006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:26.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.748 7f50f607b700 1 --2- 172.21.2.101:0/1120446815 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50e404a7f0 0x7f50e404cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:26.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.748 7f50f607b700 1 --2- 172.21.2.101:0/1120446815 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f50e404d500 0x7f50e404fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:26.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.748 7f50f607b700 1 -- 172.21.2.101:0/1120446815 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e40502d0 con 0x7f50e4003b10 2021-02-10T11:45:26.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.748 7f50f607b700 1 -- 172.21.2.101:0/1120446815 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e40504f0 con 0x7f50e404d500 2021-02-10T11:45:26.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.748 7f50f607b700 1 -- 172.21.2.101:0/1120446815 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e4050710 con 0x7f50e404a7f0 2021-02-10T11:45:26.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.748 7f50f707d700 1 --2- 172.21.2.101:0/1120446815 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50e4003b10 0x7f50e4006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:26.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.749 7f50f787e700 1 --2- 172.21.2.101:0/1120446815 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f50e404d500 0x7f50e404fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:26.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.749 7f50f707d700 1 -- 172.21.2.101:0/1120446815 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50e404a7f0 msgr2=0x7f50e404cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:26.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.749 7f50f687c700 1 --2- 172.21.2.101:0/1120446815 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50e404a7f0 0x7f50e404cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:26.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.749 7f50f707d700 1 --2- 172.21.2.101:0/1120446815 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50e404a7f0 0x7f50e404cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:26.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.749 7f50f707d700 1 -- 172.21.2.101:0/1120446815 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f50e404d500 msgr2=0x7f50e404fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:26.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.749 7f50f707d700 1 --2- 172.21.2.101:0/1120446815 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f50e404d500 0x7f50e404fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:26.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.749 7f50f707d700 1 -- 172.21.2.101:0/1120446815 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e40509b0 con 0x7f50e4003b10 2021-02-10T11:45:26.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.749 7f50f787e700 1 --2- 172.21.2.101:0/1120446815 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f50e404d500 0x7f50e404fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:26.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.749 7f50f707d700 1 --2- 172.21.2.101:0/1120446815 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50e4003b10 0x7f50e4006010 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 rx=0x7f50e0009310 tx=0x7f50e0009a10).ready entity=mon.0 client_cookie=8819b520ff2095b0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:26.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.749 7f50f607b700 1 -- 172.21.2.101:0/1120446815 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f50e00009b0 con 0x7f50e4003b10 2021-02-10T11:45:26.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.749 7f50f607b700 1 -- 172.21.2.101:0/1120446815 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e0006400 con 0x7f50e4003b10 2021-02-10T11:45:26.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.749 7f50f607b700 1 -- 172.21.2.101:0/1120446815 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f50e000c340 con 0x7f50e4003b10 2021-02-10T11:45:26.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.749 7f50fd3b5700 1 -- 172.21.2.101:0/1120446815 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50e4003b10 msgr2=0x7f50e4006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:26.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.749 7f50fd3b5700 1 --2- 172.21.2.101:0/1120446815 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50e4003b10 0x7f50e4006010 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 rx=0x7f50e0009310 tx=0x7f50e0009a10).stop 2021-02-10T11:45:26.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.749 7f50f787e700 1 -- 172.21.2.101:0/1120446815 reap_dead start 2021-02-10T11:45:26.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.750 7f50fd3b5700 1 -- 172.21.2.101:0/1120446815 shutdown_connections 2021-02-10T11:45:26.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.750 7f50fd3b5700 1 -- 172.21.2.101:0/1120446815 >> 172.21.2.101:0/1120446815 conn(0x7f50f80ce450 msgr2=0x7f50f80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:26.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.750 7f50fd3b5700 1 -- 172.21.2.101:0/1120446815 shutdown_connections 2021-02-10T11:45:26.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.750 7f50fd3b5700 1 -- 172.21.2.101:0/1120446815 wait complete. 2021-02-10T11:45:26.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.765 7f50fd3b5700 1 Processor -- start 2021-02-10T11:45:26.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.765 7f50fd3b5700 1 -- start start 2021-02-10T11:45:26.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.765 7f50fd3b5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50f80bfa90 0x7f50f818a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:26.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.765 7f50fd3b5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50f80bfe90 0x7f50f818a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:26.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.765 7f50fd3b5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f50f8192510 0x7f50f8192910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:26.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.765 7f50fd3b5700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f8189b50 con 0x7f50f80bfe90 2021-02-10T11:45:26.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.765 7f50fd3b5700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f8190d30 con 0x7f50f8192510 2021-02-10T11:45:26.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.765 7f50fd3b5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f8193d70 con 0x7f50f80bfa90 2021-02-10T11:45:26.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.765 7f50f687c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50f80bfe90 0x7f50f818a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:26.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.765 7f50f687c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50f80bfe90 0x7f50f818a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60598/0 (socket says 172.21.2.101:60598) 2021-02-10T11:45:26.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.765 7f50f687c700 1 -- 172.21.2.101:0/775749558 learned_addr learned my addr 172.21.2.101:0/775749558 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:26.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.766 7f50f687c700 1 -- 172.21.2.101:0/775749558 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50f80bfa90 msgr2=0x7f50f818a050 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:26.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.766 7f50f687c700 1 --2- 172.21.2.101:0/775749558 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50f80bfa90 0x7f50f818a050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:26.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.766 7f50f687c700 1 -- 172.21.2.101:0/775749558 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f50f8192510 msgr2=0x7f50f8192910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:26.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.766 7f50f687c700 1 --2- 172.21.2.101:0/775749558 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f50f8192510 0x7f50f8192910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:26.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.766 7f50f687c700 1 -- 172.21.2.101:0/775749558 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f8192f30 con 0x7f50f80bfe90 2021-02-10T11:45:26.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.766 7f50f687c700 1 --2- 172.21.2.101:0/775749558 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50f80bfe90 0x7f50f818a670 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 rx=0x7f50e800a540 tx=0x7f50e800a8e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:26.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.766 7f50effff700 1 -- 172.21.2.101:0/775749558 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f50e800acd0 con 0x7f50f80bfe90 2021-02-10T11:45:26.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.766 7f50effff700 1 -- 172.21.2.101:0/775749558 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e8004350 con 0x7f50f80bfe90 2021-02-10T11:45:26.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.766 7f50effff700 1 -- 172.21.2.101:0/775749558 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f50e8004830 con 0x7f50f80bfe90 2021-02-10T11:45:26.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.766 7f50fd3b5700 1 -- 172.21.2.101:0/775749558 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f8192f30 con 0x7f50f80bfe90 2021-02-10T11:45:26.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.766 7f50fd3b5700 1 -- 172.21.2.101:0/775749558 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f8192f30 con 0x7f50f80bfe90 2021-02-10T11:45:26.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.767 7f50effff700 1 -- 172.21.2.101:0/775749558 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f50e8004530 con 0x7f50f80bfe90 2021-02-10T11:45:26.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.767 7f50effff700 1 --2- 172.21.2.101:0/775749558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50dc0186c0 0x7f50dc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:26.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.767 7f50effff700 1 -- 172.21.2.101:0/775749558 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f50e8032520 con 0x7f50f80bfe90 2021-02-10T11:45:26.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.767 7f50f707d700 1 --2- 172.21.2.101:0/775749558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50dc0186c0 0x7f50dc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:26.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.767 7f50f707d700 1 --2- 172.21.2.101:0/775749558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50dc0186c0 0x7f50dc01abc0 crc :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:26.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.797 7f50fd3b5700 1 -- 172.21.2.101:0/775749558 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f8190d10 con 0x7f50f80bfe90 2021-02-10T11:45:26.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.799 7f50effff700 1 -- 172.21.2.101:0/775749558 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f50e802bff0 con 0x7f50f80bfe90 2021-02-10T11:45:26.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.949 7f50fd3b5700 1 -- 172.21.2.101:0/775749558 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f50f8190d10 con 0x7f50f80bfe90 2021-02-10T11:45:26.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.953 7f50effff700 1 -- 172.21.2.101:0/775749558 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v39) v1 ==== 104+0+0 (secure 0 0 0) 0x7f50e802bff0 con 0x7f50f80bfe90 2021-02-10T11:45:26.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.958 7f50fd3b5700 1 -- 172.21.2.101:0/775749558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50dc0186c0 msgr2=0x7f50dc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:26.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.958 7f50fd3b5700 1 --2- 172.21.2.101:0/775749558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50dc0186c0 0x7f50dc01abc0 crc :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:26.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.958 7f50fd3b5700 1 -- 172.21.2.101:0/775749558 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50f80bfe90 msgr2=0x7f50f818a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:26.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.958 7f50fd3b5700 1 --2- 172.21.2.101:0/775749558 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50f80bfe90 0x7f50f818a670 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 rx=0x7f50e800a540 tx=0x7f50e800a8e0).stop 2021-02-10T11:45:26.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.958 7f50fd3b5700 1 -- 172.21.2.101:0/775749558 shutdown_connections 2021-02-10T11:45:26.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.958 7f50fd3b5700 1 --2- 172.21.2.101:0/775749558 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f50f8192510 0x7f50f8192910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:26.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.958 7f50fd3b5700 1 --2- 172.21.2.101:0/775749558 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f50f80bfe90 0x7f50f818a670 unknown :-1 s=CLOSED pgs=2825 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:26.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.958 7f50fd3b5700 1 --2- 172.21.2.101:0/775749558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f50dc0186c0 0x7f50dc01abc0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:26.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.958 7f50fd3b5700 1 --2- 172.21.2.101:0/775749558 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f50f80bfa90 0x7f50f818a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:26.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.958 7f50fd3b5700 1 -- 172.21.2.101:0/775749558 >> 172.21.2.101:0/775749558 conn(0x7f50f80ed860 msgr2=0x7f50f80bf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:26.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.958 7f50fd3b5700 1 -- 172.21.2.101:0/775749558 shutdown_connections 2021-02-10T11:45:26.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:26.958 7f50fd3b5700 1 -- 172.21.2.101:0/775749558 wait complete. 2021-02-10T11:45:26.965 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_rm' 2021-02-10T11:45:27.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.380 7fbc49d0a700 1 Processor -- start 2021-02-10T11:45:27.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.380 7fbc49d0a700 1 -- start start 2021-02-10T11:45:27.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.380 7fbc49d0a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbc440d0340 0x7fbc440ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:27.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.380 7fbc49d0a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc440f0b10 con 0x7fbc440d1c70 2021-02-10T11:45:27.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.380 7fbc49d0a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc441b0be0 con 0x7fbc440d1870 2021-02-10T11:45:27.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.380 7fbc49d0a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc440e5ed0 con 0x7fbc440d0340 2021-02-10T11:45:27.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.380 7fbc43fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fbc440d1c70 0x7fbc440efc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:32866/0 (socket says 172.21.2.101:32866) 2021-02-10T11:45:27.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.380 7fbc43fff700 1 -- 172.21.2.101:0/203584163 learned_addr learned my addr 172.21.2.101:0/203584163 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:27.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc427fc700 1 -- 172.21.2.101:0/203584163 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 886292123 0 0) 0x7fbc3c001500 con 0x7fbc440d1c70 2021-02-10T11:45:27.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc427fc700 1 -- 172.21.2.101:0/203584163 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc2c002fe0 con 0x7fbc440d1c70 2021-02-10T11:45:27.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc42ffd700 1 --2- 172.21.2.101:0/203584163 >> v2:172.21.2.108:3300/0 conn(0x7fbc440d0340 0x7fbc440ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:27.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc427fc700 1 -- 172.21.2.101:0/203584163 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1849112948 0 0) 0x7fbc3c001460 con 0x7fbc440d1c70 2021-02-10T11:45:27.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc427fc700 1 -- 172.21.2.101:0/203584163 >> v2:172.21.2.108:3300/0 conn(0x7fbc440d0340 msgr2=0x7fbc440ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:27.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc427fc700 1 --2- 172.21.2.101:0/203584163 >> v2:172.21.2.108:3300/0 conn(0x7fbc440d0340 0x7fbc440ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:27.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc427fc700 1 -- 172.21.2.101:0/203584163 >> v1:172.21.2.108:6789/0 conn(0x7fbc440d1870 legacy=0x7fbc441b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:27.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc427fc700 1 -- 172.21.2.101:0/203584163 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc440f0450 con 0x7fbc440d1c70 2021-02-10T11:45:27.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc427fc700 1 -- 172.21.2.101:0/203584163 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fbc3c001980 con 0x7fbc440d1c70 2021-02-10T11:45:27.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc427fc700 1 -- 172.21.2.101:0/203584163 >> v1:172.21.2.101:6789/0 conn(0x7fbc440d1c70 legacy=0x7fbc440efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:27.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc427fc700 1 --2- 172.21.2.101:0/203584163 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbc2c003a70 0x7fbc2c005f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:27.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc427fc700 1 --2- 172.21.2.101:0/203584163 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbc2c0067a0 0x7fbc2c008ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:27.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc427fc700 1 --2- 172.21.2.101:0/203584163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbc2c0095a0 0x7fbc2c00bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:27.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc427fc700 1 -- 172.21.2.101:0/203584163 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c00c310 con 0x7fbc2c0095a0 2021-02-10T11:45:27.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc427fc700 1 -- 172.21.2.101:0/203584163 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c00c530 con 0x7fbc2c0067a0 2021-02-10T11:45:27.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc427fc700 1 -- 172.21.2.101:0/203584163 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c00c750 con 0x7fbc2c003a70 2021-02-10T11:45:27.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc43fff700 1 --2- 172.21.2.101:0/203584163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbc2c0095a0 0x7fbc2c00bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:27.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc43fff700 1 -- 172.21.2.101:0/203584163 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbc2c003a70 msgr2=0x7fbc2c005f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:27.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc42ffd700 1 --2- 172.21.2.101:0/203584163 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbc2c0067a0 0x7fbc2c008ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:27.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc437fe700 1 --2- 172.21.2.101:0/203584163 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbc2c003a70 0x7fbc2c005f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:27.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc43fff700 1 --2- 172.21.2.101:0/203584163 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbc2c003a70 0x7fbc2c005f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:27.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc43fff700 1 -- 172.21.2.101:0/203584163 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbc2c0067a0 msgr2=0x7fbc2c008ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:27.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc43fff700 1 --2- 172.21.2.101:0/203584163 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbc2c0067a0 0x7fbc2c008ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:27.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc43fff700 1 -- 172.21.2.101:0/203584163 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c00c9f0 con 0x7fbc2c0095a0 2021-02-10T11:45:27.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.381 7fbc42ffd700 1 --2- 172.21.2.101:0/203584163 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbc2c0067a0 0x7fbc2c008ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:27.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.382 7fbc43fff700 1 -- 172.21.2.101:0/203584163 reap_dead start 2021-02-10T11:45:27.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.382 7fbc43fff700 1 --2- 172.21.2.101:0/203584163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbc2c0095a0 0x7fbc2c00bac0 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 rx=0x7fbc3c001ae0 tx=0x7fbc3c04e320).ready entity=mon.0 client_cookie=77339f985f98ee19 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:27.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.382 7fbc427fc700 1 -- 172.21.2.101:0/203584163 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbc3c001460 con 0x7fbc2c0095a0 2021-02-10T11:45:27.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.382 7fbc427fc700 1 -- 172.21.2.101:0/203584163 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3c0070c0 con 0x7fbc2c0095a0 2021-02-10T11:45:27.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.382 7fbc427fc700 1 -- 172.21.2.101:0/203584163 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbc3c004720 con 0x7fbc2c0095a0 2021-02-10T11:45:27.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.382 7fbc49d0a700 1 -- 172.21.2.101:0/203584163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbc2c0095a0 msgr2=0x7fbc2c00bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:27.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.382 7fbc49d0a700 1 --2- 172.21.2.101:0/203584163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbc2c0095a0 0x7fbc2c00bac0 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 rx=0x7fbc3c001ae0 tx=0x7fbc3c04e320).stop 2021-02-10T11:45:27.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.382 7fbc49d0a700 1 -- 172.21.2.101:0/203584163 shutdown_connections 2021-02-10T11:45:27.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.382 7fbc49d0a700 1 --2- 172.21.2.101:0/203584163 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbc2c0095a0 0x7fbc2c00bac0 unknown :-1 s=CLOSED pgs=2827 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:27.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.382 7fbc49d0a700 1 -- 172.21.2.101:0/203584163 >> 172.21.2.101:0/203584163 conn(0x7fbc440ce450 msgr2=0x7fbc440d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:27.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.382 7fbc49d0a700 1 -- 172.21.2.101:0/203584163 shutdown_connections 2021-02-10T11:45:27.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.382 7fbc49d0a700 1 -- 172.21.2.101:0/203584163 wait complete. 2021-02-10T11:45:27.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.397 7fbc49d0a700 1 Processor -- start 2021-02-10T11:45:27.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.397 7fbc49d0a700 1 -- start start 2021-02-10T11:45:27.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.397 7fbc49d0a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbc440aa700 0x7fbc44090ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:27.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.397 7fbc49d0a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbc440aab00 0x7fbc440910e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:27.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.397 7fbc49d0a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbc4408ce20 0x7fbc4408d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:27.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.397 7fbc49d0a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc44090540 con 0x7fbc440aa700 2021-02-10T11:45:27.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.397 7fbc49d0a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc440c0170 con 0x7fbc4408ce20 2021-02-10T11:45:27.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.397 7fbc49d0a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4408d840 con 0x7fbc440aab00 2021-02-10T11:45:27.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.397 7fbc437fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbc440aa700 0x7fbc44090ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:27.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.397 7fbc437fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbc440aa700 0x7fbc44090ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60614/0 (socket says 172.21.2.101:60614) 2021-02-10T11:45:27.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.397 7fbc437fe700 1 -- 172.21.2.101:0/1749496898 learned_addr learned my addr 172.21.2.101:0/1749496898 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:27.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.398 7fbc43fff700 1 --2- 172.21.2.101:0/1749496898 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbc4408ce20 0x7fbc4408d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:27.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.398 7fbc42ffd700 1 --2- 172.21.2.101:0/1749496898 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbc440aab00 0x7fbc440910e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:27.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.398 7fbc437fe700 1 -- 172.21.2.101:0/1749496898 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbc440aab00 msgr2=0x7fbc440910e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:27.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.398 7fbc437fe700 1 --2- 172.21.2.101:0/1749496898 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbc440aab00 0x7fbc440910e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:27.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.398 7fbc437fe700 1 -- 172.21.2.101:0/1749496898 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbc4408ce20 msgr2=0x7fbc4408d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:27.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.398 7fbc437fe700 1 --2- 172.21.2.101:0/1749496898 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbc4408ce20 0x7fbc4408d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:27.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.398 7fbc437fe700 1 -- 172.21.2.101:0/1749496898 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4408da60 con 0x7fbc440aa700 2021-02-10T11:45:27.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.398 7fbc437fe700 1 --2- 172.21.2.101:0/1749496898 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbc440aa700 0x7fbc44090ac0 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 rx=0x7fbc34006f10 tx=0x7fbc3400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:27.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.398 7fbc2bfff700 1 -- 172.21.2.101:0/1749496898 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbc3400bd80 con 0x7fbc440aa700 2021-02-10T11:45:27.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.398 7fbc2bfff700 1 -- 172.21.2.101:0/1749496898 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc34006030 con 0x7fbc440aa700 2021-02-10T11:45:27.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.398 7fbc2bfff700 1 -- 172.21.2.101:0/1749496898 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbc34005560 con 0x7fbc440aa700 2021-02-10T11:45:27.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.398 7fbc49d0a700 1 -- 172.21.2.101:0/1749496898 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4408da60 con 0x7fbc440aa700 2021-02-10T11:45:27.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.398 7fbc49d0a700 1 -- 172.21.2.101:0/1749496898 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4408dff0 con 0x7fbc440aa700 2021-02-10T11:45:27.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.399 7fbc2bfff700 1 -- 172.21.2.101:0/1749496898 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbc340052e0 con 0x7fbc440aa700 2021-02-10T11:45:27.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.399 7fbc2bfff700 1 --2- 172.21.2.101:0/1749496898 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbc240186c0 0x7fbc2401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:27.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.399 7fbc2bfff700 1 -- 172.21.2.101:0/1749496898 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fbc34032510 con 0x7fbc440aa700 2021-02-10T11:45:27.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.399 7fbc42ffd700 1 --2- 172.21.2.101:0/1749496898 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbc240186c0 0x7fbc2401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:27.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.400 7fbc42ffd700 1 --2- 172.21.2.101:0/1749496898 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbc240186c0 0x7fbc2401abc0 crc :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:27.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.430 7fbc49d0a700 1 -- 172.21.2.101:0/1749496898 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc440ebc00 con 0x7fbc440aa700 2021-02-10T11:45:27.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.433 7fbc2bfff700 1 -- 172.21.2.101:0/1749496898 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbc340009b0 con 0x7fbc440aa700 2021-02-10T11:45:27.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.566 7fbc49d0a700 1 -- 172.21.2.101:0/1749496898 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_rm"]} v 0) v1 -- 0x7fbc440f0b10 con 0x7fbc440aa700 2021-02-10T11:45:27.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.621 7fbc2bfff700 1 -- 172.21.2.101:0/1749496898 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_rm"]}]=0 v753) v1 ==== 146+0+0 (secure 0 0 0) 0x7fbc34005640 con 0x7fbc440aa700 2021-02-10T11:45:27.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.632 7fbc49d0a700 1 -- 172.21.2.101:0/1749496898 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbc240186c0 msgr2=0x7fbc2401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:27.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.632 7fbc49d0a700 1 --2- 172.21.2.101:0/1749496898 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbc240186c0 0x7fbc2401abc0 crc :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:27.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.632 7fbc49d0a700 1 -- 172.21.2.101:0/1749496898 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbc440aa700 msgr2=0x7fbc44090ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:27.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.632 7fbc49d0a700 1 --2- 172.21.2.101:0/1749496898 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbc440aa700 0x7fbc44090ac0 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 rx=0x7fbc34006f10 tx=0x7fbc3400b990).stop 2021-02-10T11:45:27.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.632 7fbc49d0a700 1 -- 172.21.2.101:0/1749496898 shutdown_connections 2021-02-10T11:45:27.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.632 7fbc49d0a700 1 --2- 172.21.2.101:0/1749496898 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbc240186c0 0x7fbc2401abc0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:27.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.632 7fbc49d0a700 1 --2- 172.21.2.101:0/1749496898 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbc440aab00 0x7fbc440910e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:27.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.632 7fbc49d0a700 1 --2- 172.21.2.101:0/1749496898 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbc4408ce20 0x7fbc4408d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:27.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.632 7fbc49d0a700 1 --2- 172.21.2.101:0/1749496898 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbc440aa700 0x7fbc44090ac0 unknown :-1 s=CLOSED pgs=2828 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:27.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.632 7fbc49d0a700 1 -- 172.21.2.101:0/1749496898 >> 172.21.2.101:0/1749496898 conn(0x7fbc440b7e70 msgr2=0x7fbc440c3850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:27.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.632 7fbc49d0a700 1 -- 172.21.2.101:0/1749496898 shutdown_connections 2021-02-10T11:45:27.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:27.633 7fbc49d0a700 1 -- 172.21.2.101:0/1749496898 wait complete. 2021-02-10T11:45:27.639 INFO:tasks.cephfs_test_runner:test_nonexistent_subvolume_rm (tasks.cephfs.test_volumes.TestVolumes) ... ok 2021-02-10T11:45:27.639 INFO:tasks.cephfs_test_runner:Starting test: test_nonexistent_subvolume_snapshot_rm (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:45:27.640 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_snapshot_rm' 2021-02-10T11:45:28.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.053 7f972a803700 1 Processor -- start 2021-02-10T11:45:28.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.053 7f972a803700 1 -- start start 2021-02-10T11:45:28.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.053 7f972a803700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f97240d1870 0x7f97241b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:28.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.053 7f972a803700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97241b0a10 0x7f97240cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:28.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.053 7f972a803700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97240f0b10 con 0x7f97240d1c70 2021-02-10T11:45:28.067 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.053 7f972a803700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97240e5ed0 con 0x7f97240d1870 2021-02-10T11:45:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.053 7f972a803700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97240f0650 con 0x7f97241b0a10 2021-02-10T11:45:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.054 7f9723fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f97240d1870 0x7f97241b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.054 7f9723fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f97240d1870 0x7f97241b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60620/0 (socket says 172.21.2.101:60620) 2021-02-10T11:45:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.054 7f9723fff700 1 -- 172.21.2.101:0/2412272600 learned_addr learned my addr 172.21.2.101:0/2412272600 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.054 7f9723fff700 1 -- 172.21.2.101:0/2412272600 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97241b0a10 msgr2=0x7f97240cf330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:45:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.054 7f9723fff700 1 --2- 172.21.2.101:0/2412272600 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97241b0a10 0x7f97240cf330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.054 7f9723fff700 1 -- 172.21.2.101:0/2412272600 >> v1:172.21.2.108:6789/0 conn(0x7f97240d1c70 legacy=0x7f97240cf950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:45:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.054 7f9723fff700 1 -- 172.21.2.101:0/2412272600 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97240f0870 con 0x7f97240d1870 2021-02-10T11:45:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.054 7f9723fff700 1 --2- 172.21.2.101:0/2412272600 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f97240d1870 0x7f97241b03f0 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 rx=0x7f9714006700 tx=0x7f9714009b80).ready entity=mon.0 client_cookie=ad9a6f50359ee9a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.054 7f9722ffd700 1 -- 172.21.2.101:0/2412272600 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f971400c0c0 con 0x7f97240d1870 2021-02-10T11:45:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.054 7f9722ffd700 1 -- 172.21.2.101:0/2412272600 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971400c440 con 0x7f97240d1870 2021-02-10T11:45:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.054 7f9722ffd700 1 -- 172.21.2.101:0/2412272600 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f971400c8a0 con 0x7f97240d1870 2021-02-10T11:45:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.054 7f972a803700 1 -- 172.21.2.101:0/2412272600 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f97240d1870 msgr2=0x7f97241b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.054 7f972a803700 1 --2- 172.21.2.101:0/2412272600 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f97240d1870 0x7f97241b03f0 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 rx=0x7f9714006700 tx=0x7f9714009b80).stop 2021-02-10T11:45:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.055 7f972a803700 1 -- 172.21.2.101:0/2412272600 shutdown_connections 2021-02-10T11:45:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.055 7f972a803700 1 --2- 172.21.2.101:0/2412272600 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97241b0a10 0x7f97240cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:28.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.055 7f972a803700 1 --2- 172.21.2.101:0/2412272600 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f97240d1870 0x7f97241b03f0 unknown :-1 s=CLOSED pgs=2829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:28.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.055 7f972a803700 1 -- 172.21.2.101:0/2412272600 >> 172.21.2.101:0/2412272600 conn(0x7f97240ce450 msgr2=0x7f97240d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:28.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.055 7f972a803700 1 -- 172.21.2.101:0/2412272600 shutdown_connections 2021-02-10T11:45:28.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.055 7f972a803700 1 -- 172.21.2.101:0/2412272600 wait complete. 2021-02-10T11:45:28.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.069 7f972a803700 1 Processor -- start 2021-02-10T11:45:28.072 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.070 7f972a803700 1 -- start start 2021-02-10T11:45:28.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.070 7f972a803700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97240b4f30 0x7f972418b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:28.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.070 7f972a803700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9724093f40 0x7f9724094340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:28.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.070 7f972a803700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9724094960 0x7f9724187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:28.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.070 7f972a803700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9724094e90 con 0x7f9724094960 2021-02-10T11:45:28.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.070 7f972a803700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972418bff0 con 0x7f97240b4f30 2021-02-10T11:45:28.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.070 7f972a803700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f972408f780 con 0x7f9724093f40 2021-02-10T11:45:28.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.070 7f9728e20700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9724094960 0x7f9724187e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:28.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.070 7f9728e20700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9724094960 0x7f9724187e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60626/0 (socket says 172.21.2.101:60626) 2021-02-10T11:45:28.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.070 7f9728e20700 1 -- 172.21.2.101:0/1134578746 learned_addr learned my addr 172.21.2.101:0/1134578746 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:28.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.070 7f9728e20700 1 -- 172.21.2.101:0/1134578746 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9724093f40 msgr2=0x7f9724094340 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:28.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.070 7f9728e20700 1 --2- 172.21.2.101:0/1134578746 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9724093f40 0x7f9724094340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:28.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.070 7f9728e20700 1 -- 172.21.2.101:0/1134578746 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97240b4f30 msgr2=0x7f972418b9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:28.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.070 7f9728e20700 1 --2- 172.21.2.101:0/1134578746 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97240b4f30 0x7f972418b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:28.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.070 7f9728e20700 1 -- 172.21.2.101:0/1134578746 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9724188470 con 0x7f9724094960 2021-02-10T11:45:28.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.070 7f9728e20700 1 --2- 172.21.2.101:0/1134578746 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9724094960 0x7f9724187e50 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 rx=0x7f971c009940 tx=0x7f971c00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:28.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.071 7f9720ff9700 1 -- 172.21.2.101:0/1134578746 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f971c003030 con 0x7f9724094960 2021-02-10T11:45:28.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.071 7f9720ff9700 1 -- 172.21.2.101:0/1134578746 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971c001270 con 0x7f9724094960 2021-02-10T11:45:28.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.071 7f9720ff9700 1 -- 172.21.2.101:0/1134578746 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f971c0016d0 con 0x7f9724094960 2021-02-10T11:45:28.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.071 7f972a803700 1 -- 172.21.2.101:0/1134578746 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9724188470 con 0x7f9724094960 2021-02-10T11:45:28.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.071 7f972a803700 1 -- 172.21.2.101:0/1134578746 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9724188a00 con 0x7f9724094960 2021-02-10T11:45:28.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.072 7f9720ff9700 1 -- 172.21.2.101:0/1134578746 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f971c001270 con 0x7f9724094960 2021-02-10T11:45:28.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.072 7f9720ff9700 1 --2- 172.21.2.101:0/1134578746 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97040186c0 0x7f970401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:28.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.072 7f9720ff9700 1 -- 172.21.2.101:0/1134578746 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f971c032820 con 0x7f9724094960 2021-02-10T11:45:28.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.072 7f9723fff700 1 --2- 172.21.2.101:0/1134578746 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97040186c0 0x7f970401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:28.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.072 7f9723fff700 1 --2- 172.21.2.101:0/1134578746 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97040186c0 0x7f970401abc0 crc :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:28.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.103 7f972a803700 1 -- 172.21.2.101:0/1134578746 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f972418bff0 con 0x7f9724094960 2021-02-10T11:45:28.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.106 7f9720ff9700 1 -- 172.21.2.101:0/1134578746 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f971c000cc0 con 0x7f9724094960 2021-02-10T11:45:28.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.236 7f972a803700 1 -- 172.21.2.101:0/1134578746 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_snapshot_rm"]} v 0) v1 -- 0x7f972418bff0 con 0x7f9724094960 2021-02-10T11:45:28.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.627 7f9720ff9700 1 -- 172.21.2.101:0/1134578746 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_snapshot_rm"]}]=0 v754) v1 ==== 158+0+0 (secure 0 0 0) 0x7f971c001270 con 0x7f9724094960 2021-02-10T11:45:28.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.655 7f972a803700 1 -- 172.21.2.101:0/1134578746 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97040186c0 msgr2=0x7f970401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:28.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.655 7f972a803700 1 --2- 172.21.2.101:0/1134578746 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97040186c0 0x7f970401abc0 crc :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:28.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.655 7f972a803700 1 -- 172.21.2.101:0/1134578746 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9724094960 msgr2=0x7f9724187e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:28.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.655 7f972a803700 1 --2- 172.21.2.101:0/1134578746 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9724094960 0x7f9724187e50 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 rx=0x7f971c009940 tx=0x7f971c00bcf0).stop 2021-02-10T11:45:28.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.656 7f972a803700 1 -- 172.21.2.101:0/1134578746 shutdown_connections 2021-02-10T11:45:28.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.656 7f972a803700 1 --2- 172.21.2.101:0/1134578746 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97040186c0 0x7f970401abc0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:28.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.656 7f972a803700 1 --2- 172.21.2.101:0/1134578746 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9724093f40 0x7f9724094340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:28.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.656 7f972a803700 1 --2- 172.21.2.101:0/1134578746 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9724094960 0x7f9724187e50 unknown :-1 s=CLOSED pgs=2830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:28.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.656 7f972a803700 1 --2- 172.21.2.101:0/1134578746 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97240b4f30 0x7f972418b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:28.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.656 7f972a803700 1 -- 172.21.2.101:0/1134578746 >> 172.21.2.101:0/1134578746 conn(0x7f972419f8f0 msgr2=0x7f97241a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:28.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.656 7f972a803700 1 -- 172.21.2.101:0/1134578746 shutdown_connections 2021-02-10T11:45:28.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:28.656 7f972a803700 1 -- 172.21.2.101:0/1134578746 wait complete. 2021-02-10T11:45:28.662 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:45:29.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.079 7f53071c1700 1 Processor -- start 2021-02-10T11:45:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.080 7f53071c1700 1 -- start start 2021-02-10T11:45:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.080 7f53071c1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f53000d1870 0x7f53001b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.080 7f53071c1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f53000cff70 0x7f5300109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.080 7f53071c1700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53000fe1c0 con 0x7f53001b0ae0 2021-02-10T11:45:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.080 7f53071c1700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5300109ad0 con 0x7f53000cff70 2021-02-10T11:45:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.080 7f53071c1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5300109750 con 0x7f53000d1870 2021-02-10T11:45:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.080 7f53057de700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f53000cff70 0x7f5300109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.080 7f53057de700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f53000cff70 0x7f5300109130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60636/0 (socket says 172.21.2.101:60636) 2021-02-10T11:45:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.080 7f53057de700 1 -- 172.21.2.101:0/4283804751 learned_addr learned my addr 172.21.2.101:0/4283804751 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.081 7f53057de700 1 -- 172.21.2.101:0/4283804751 >> v2:172.21.2.108:3300/0 conn(0x7f53000d1870 msgr2=0x7f53001b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:29.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.081 7f53057de700 1 --2- 172.21.2.101:0/4283804751 >> v2:172.21.2.108:3300/0 conn(0x7f53000d1870 0x7f53001b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.081 7f53057de700 1 -- 172.21.2.101:0/4283804751 >> v1:172.21.2.108:6789/0 conn(0x7f53001b0ae0 legacy=0x7f5300108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.081 7f53057de700 1 -- 172.21.2.101:0/4283804751 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5300088070 con 0x7f53000cff70 2021-02-10T11:45:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.081 7f53057de700 1 --2- 172.21.2.101:0/4283804751 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53000cff70 0x7f5300109130 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 rx=0x7f52f8006b80 tx=0x7f52f8009b90).ready entity=mon.0 client_cookie=a826bf9c6f42f0d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.081 7f52ff7fe700 1 -- 172.21.2.101:0/4283804751 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f52f80022e0 con 0x7f53000cff70 2021-02-10T11:45:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.081 7f52ff7fe700 1 -- 172.21.2.101:0/4283804751 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f8002620 con 0x7f53000cff70 2021-02-10T11:45:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.081 7f52ff7fe700 1 -- 172.21.2.101:0/4283804751 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f52f8002a80 con 0x7f53000cff70 2021-02-10T11:45:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.081 7f53071c1700 1 -- 172.21.2.101:0/4283804751 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53000cff70 msgr2=0x7f5300109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.081 7f53071c1700 1 --2- 172.21.2.101:0/4283804751 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53000cff70 0x7f5300109130 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 rx=0x7f52f8006b80 tx=0x7f52f8009b90).stop 2021-02-10T11:45:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.082 7f53071c1700 1 -- 172.21.2.101:0/4283804751 shutdown_connections 2021-02-10T11:45:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.082 7f53071c1700 1 --2- 172.21.2.101:0/4283804751 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f53000cff70 0x7f5300109130 unknown :-1 s=CLOSED pgs=2831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.082 7f53071c1700 1 --2- 172.21.2.101:0/4283804751 >> v2:172.21.2.108:3300/0 conn(0x7f53000d1870 0x7f53001b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.082 7f53071c1700 1 -- 172.21.2.101:0/4283804751 >> 172.21.2.101:0/4283804751 conn(0x7f53000ce420 msgr2=0x7f53000d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:29.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.082 7f53071c1700 1 -- 172.21.2.101:0/4283804751 shutdown_connections 2021-02-10T11:45:29.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.082 7f53071c1700 1 -- 172.21.2.101:0/4283804751 wait complete. 2021-02-10T11:45:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.097 7f53071c1700 1 Processor -- start 2021-02-10T11:45:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.097 7f53071c1700 1 -- start start 2021-02-10T11:45:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.097 7f53071c1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53000c2370 0x7f530018f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.097 7f53071c1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5300084cb0 0x7f53001a7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:29.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.097 7f53071c1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53001a7a90 0x7f5300085700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.097 7f53071c1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53000be840 con 0x7f5300084cb0 2021-02-10T11:45:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.097 7f53071c1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53000c29a0 con 0x7f53001a7a90 2021-02-10T11:45:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.097 7f53071c1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53001a7e90 con 0x7f53000c2370 2021-02-10T11:45:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.097 7f52fffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5300084cb0 0x7f53001a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.098 7f52fffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5300084cb0 0x7f53001a7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60638/0 (socket says 172.21.2.101:60638) 2021-02-10T11:45:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.098 7f52fffff700 1 -- 172.21.2.101:0/70233941 learned_addr learned my addr 172.21.2.101:0/70233941 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.098 7f52fffff700 1 -- 172.21.2.101:0/70233941 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53000c2370 msgr2=0x7f530018f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.098 7f52fffff700 1 --2- 172.21.2.101:0/70233941 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53000c2370 0x7f530018f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.098 7f52fffff700 1 -- 172.21.2.101:0/70233941 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53001a7a90 msgr2=0x7f5300085700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.098 7f52fffff700 1 --2- 172.21.2.101:0/70233941 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53001a7a90 0x7f5300085700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.098 7f52fffff700 1 -- 172.21.2.101:0/70233941 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53001a80b0 con 0x7f5300084cb0 2021-02-10T11:45:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.098 7f52fffff700 1 --2- 172.21.2.101:0/70233941 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5300084cb0 0x7f53001a7470 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 rx=0x7f52f0009940 tx=0x7f52f000bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:29.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.098 7f52fd7fa700 1 -- 172.21.2.101:0/70233941 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f52f0000fe0 con 0x7f5300084cb0 2021-02-10T11:45:29.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.098 7f52fd7fa700 1 -- 172.21.2.101:0/70233941 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f0000d30 con 0x7f5300084cb0 2021-02-10T11:45:29.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.098 7f53071c1700 1 -- 172.21.2.101:0/70233941 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53001a80b0 con 0x7f5300084cb0 2021-02-10T11:45:29.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.099 7f52fd7fa700 1 -- 172.21.2.101:0/70233941 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f52f000c570 con 0x7f5300084cb0 2021-02-10T11:45:29.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.099 7f53071c1700 1 -- 172.21.2.101:0/70233941 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53001a80b0 con 0x7f5300084cb0 2021-02-10T11:45:29.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.099 7f52fd7fa700 1 -- 172.21.2.101:0/70233941 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f52f0001230 con 0x7f5300084cb0 2021-02-10T11:45:29.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.099 7f52fd7fa700 1 --2- 172.21.2.101:0/70233941 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52e4018730 0x7f52e401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:29.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.099 7f52fd7fa700 1 -- 172.21.2.101:0/70233941 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f52f00329f0 con 0x7f5300084cb0 2021-02-10T11:45:29.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.099 7f5304fdd700 1 --2- 172.21.2.101:0/70233941 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52e4018730 0x7f52e401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:29.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.099 7f5304fdd700 1 --2- 172.21.2.101:0/70233941 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52e4018730 0x7f52e401ac10 crc :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:29.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.130 7f53071c1700 1 -- 172.21.2.101:0/70233941 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53000be840 con 0x7f5300084cb0 2021-02-10T11:45:29.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.132 7f52fd7fa700 1 -- 172.21.2.101:0/70233941 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f52f000be30 con 0x7f5300084cb0 2021-02-10T11:45:29.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.282 7f53071c1700 1 -- 172.21.2.101:0/70233941 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f53000be840 con 0x7f5300084cb0 2021-02-10T11:45:29.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.285 7f52fd7fa700 1 -- 172.21.2.101:0/70233941 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f52f0001230 con 0x7f5300084cb0 2021-02-10T11:45:29.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.290 7f53071c1700 1 -- 172.21.2.101:0/70233941 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52e4018730 msgr2=0x7f52e401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:29.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.290 7f53071c1700 1 --2- 172.21.2.101:0/70233941 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52e4018730 0x7f52e401ac10 crc :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:29.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.290 7f53071c1700 1 -- 172.21.2.101:0/70233941 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5300084cb0 msgr2=0x7f53001a7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:29.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.290 7f53071c1700 1 --2- 172.21.2.101:0/70233941 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5300084cb0 0x7f53001a7470 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 rx=0x7f52f0009940 tx=0x7f52f000bcf0).stop 2021-02-10T11:45:29.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.291 7f53071c1700 1 -- 172.21.2.101:0/70233941 shutdown_connections 2021-02-10T11:45:29.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.291 7f53071c1700 1 --2- 172.21.2.101:0/70233941 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f53001a7a90 0x7f5300085700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:29.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.291 7f53071c1700 1 --2- 172.21.2.101:0/70233941 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5300084cb0 0x7f53001a7470 unknown :-1 s=CLOSED pgs=2832 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:29.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.291 7f53071c1700 1 --2- 172.21.2.101:0/70233941 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f52e4018730 0x7f52e401ac10 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:29.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.291 7f53071c1700 1 --2- 172.21.2.101:0/70233941 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f53000c2370 0x7f530018f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:29.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.291 7f53071c1700 1 -- 172.21.2.101:0/70233941 >> 172.21.2.101:0/70233941 conn(0x7f53000b5dc0 msgr2=0x7f53000c2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:29.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.291 7f53071c1700 1 -- 172.21.2.101:0/70233941 shutdown_connections 2021-02-10T11:45:29.289 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:29.291 7f53071c1700 1 -- 172.21.2.101:0/70233941 wait complete. 2021-02-10T11:45:29.297 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:45:29.297 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:45:29.644 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:45:29.991 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:45:29.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:29.992 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:45:29.992 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:45:30.326 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2021-02-10T11:45:30.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:45:30.327 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-02-10T11:45:30.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:30.327 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:45:30.327 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:45:30.662 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:45:30.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:45:30.663 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:45:30.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:30.663 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:45:30.663 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:45:30.997 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:45:30.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:45:30.998 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:45:30.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:30.998 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:45:30.998 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:45:31.333 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:45:31.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:45:31.334 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:45:31.334 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json-pretty 2021-02-10T11:45:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.748 7f98bd34c700 1 Processor -- start 2021-02-10T11:45:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.748 7f98bd34c700 1 -- start start 2021-02-10T11:45:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.748 7f98bd34c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98b80d0340 0x7f98b80ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.748 7f98bd34c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b80f0b10 con 0x7f98b80d1c70 2021-02-10T11:45:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.748 7f98bd34c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b81b0be0 con 0x7f98b80d1870 2021-02-10T11:45:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.748 7f98bd34c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b80e5ed0 con 0x7f98b80d0340 2021-02-10T11:45:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.749 7f98b787e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f98b80d1c70 0x7f98b80efc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:32910/0 (socket says 172.21.2.101:32910) 2021-02-10T11:45:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.749 7f98b787e700 1 -- 172.21.2.101:0/1761082283 learned_addr learned my addr 172.21.2.101:0/1761082283 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.749 7f98b687c700 1 --2- 172.21.2.101:0/1761082283 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98b80d0340 0x7f98b80ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:31.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.749 7f98b607b700 1 -- 172.21.2.101:0/1761082283 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1481547569 0 0) 0x7f98b0001500 con 0x7f98b80d1c70 2021-02-10T11:45:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.749 7f98b607b700 1 -- 172.21.2.101:0/1761082283 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98a4002fe0 con 0x7f98b80d1c70 2021-02-10T11:45:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.749 7f98b607b700 1 -- 172.21.2.101:0/1761082283 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576519865 0 0) 0x7f98a0000c60 con 0x7f98b80d1870 2021-02-10T11:45:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.749 7f98b687c700 1 -- 172.21.2.101:0/1761082283 >> v1:172.21.2.108:6789/0 conn(0x7f98b80d1870 legacy=0x7f98b81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.749 7f98b687c700 1 -- 172.21.2.101:0/1761082283 >> v1:172.21.2.101:6789/0 conn(0x7f98b80d1c70 legacy=0x7f98b80efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.749 7f98b687c700 1 -- 172.21.2.101:0/1761082283 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b80f0450 con 0x7f98b80d0340 2021-02-10T11:45:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.750 7f98b687c700 1 --2- 172.21.2.101:0/1761082283 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98b80d0340 0x7f98b80ef640 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f98a8006650 tx=0x7f98a8009a80).ready entity=mon.2 client_cookie=91845658c872e189 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.750 7f98b607b700 1 -- 172.21.2.101:0/1761082283 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f98a800c0c0 con 0x7f98b80d0340 2021-02-10T11:45:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.750 7f98b607b700 1 -- 172.21.2.101:0/1761082283 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a800c340 con 0x7f98b80d0340 2021-02-10T11:45:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.750 7f98b607b700 1 -- 172.21.2.101:0/1761082283 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f98a800c860 con 0x7f98b80d0340 2021-02-10T11:45:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.750 7f98bd34c700 1 -- 172.21.2.101:0/1761082283 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98b80d0340 msgr2=0x7f98b80ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.750 7f98bd34c700 1 --2- 172.21.2.101:0/1761082283 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98b80d0340 0x7f98b80ef640 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f98a8006650 tx=0x7f98a8009a80).stop 2021-02-10T11:45:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.751 7f98bd34c700 1 -- 172.21.2.101:0/1761082283 shutdown_connections 2021-02-10T11:45:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.751 7f98bd34c700 1 --2- 172.21.2.101:0/1761082283 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98b80d0340 0x7f98b80ef640 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:31.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.751 7f98bd34c700 1 -- 172.21.2.101:0/1761082283 >> 172.21.2.101:0/1761082283 conn(0x7f98b80ce450 msgr2=0x7f98b80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:31.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.751 7f98bd34c700 1 -- 172.21.2.101:0/1761082283 shutdown_connections 2021-02-10T11:45:31.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.751 7f98bd34c700 1 -- 172.21.2.101:0/1761082283 wait complete. 2021-02-10T11:45:31.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.765 7f98bd34c700 1 Processor -- start 2021-02-10T11:45:31.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.765 7f98bd34c700 1 -- start start 2021-02-10T11:45:31.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.766 7f98bd34c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98b81b7cf0 0x7f98b819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:31.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.766 7f98bd34c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b81b80f0 0x7f98b81b84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:31.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.766 7f98bd34c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98b819da20 0x7f98b819de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:31.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.766 7f98bd34c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b80a77c0 con 0x7f98b81b80f0 2021-02-10T11:45:31.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.766 7f98bd34c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b819d4a0 con 0x7f98b819da20 2021-02-10T11:45:31.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.766 7f98bd34c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b80954f0 con 0x7f98b81b7cf0 2021-02-10T11:45:31.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.766 7f98b687c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b81b80f0 0x7f98b81b84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:31.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.766 7f98b687c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b81b80f0 0x7f98b81b84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60654/0 (socket says 172.21.2.101:60654) 2021-02-10T11:45:31.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.766 7f98b687c700 1 -- 172.21.2.101:0/3377610738 learned_addr learned my addr 172.21.2.101:0/3377610738 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:31.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.766 7f98b707d700 1 --2- 172.21.2.101:0/3377610738 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98b81b7cf0 0x7f98b819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:31.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.766 7f98b687c700 1 -- 172.21.2.101:0/3377610738 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98b81b7cf0 msgr2=0x7f98b819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:31.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.766 7f98b787e700 1 --2- 172.21.2.101:0/3377610738 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98b819da20 0x7f98b819de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:31.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.766 7f98b687c700 1 --2- 172.21.2.101:0/3377610738 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98b81b7cf0 0x7f98b819cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:31.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.766 7f98b687c700 1 -- 172.21.2.101:0/3377610738 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98b819da20 msgr2=0x7f98b819de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:31.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.766 7f98b687c700 1 --2- 172.21.2.101:0/3377610738 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98b819da20 0x7f98b819de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:31.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.766 7f98b687c700 1 -- 172.21.2.101:0/3377610738 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b81b8d60 con 0x7f98b81b80f0 2021-02-10T11:45:31.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.766 7f98b707d700 1 --2- 172.21.2.101:0/3377610738 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98b81b7cf0 0x7f98b819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:31.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.766 7f98b787e700 1 --2- 172.21.2.101:0/3377610738 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98b819da20 0x7f98b819de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:31.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.766 7f98b687c700 1 --2- 172.21.2.101:0/3377610738 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b81b80f0 0x7f98b81b84f0 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 rx=0x7f98a800b5e0 tx=0x7f98a800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:31.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.767 7f98affff700 1 -- 172.21.2.101:0/3377610738 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f98a800bd70 con 0x7f98b81b80f0 2021-02-10T11:45:31.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.767 7f98affff700 1 -- 172.21.2.101:0/3377610738 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a8009290 con 0x7f98b81b80f0 2021-02-10T11:45:31.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.767 7f98bd34c700 1 -- 172.21.2.101:0/3377610738 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b81b8d60 con 0x7f98b81b80f0 2021-02-10T11:45:31.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.767 7f98bd34c700 1 -- 172.21.2.101:0/3377610738 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b81b8d60 con 0x7f98b81b80f0 2021-02-10T11:45:31.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.767 7f98affff700 1 -- 172.21.2.101:0/3377610738 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f98a8002270 con 0x7f98b81b80f0 2021-02-10T11:45:31.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.767 7f98affff700 1 -- 172.21.2.101:0/3377610738 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f98a8002020 con 0x7f98b81b80f0 2021-02-10T11:45:31.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.767 7f98affff700 1 --2- 172.21.2.101:0/3377610738 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f989c0186c0 0x7f989c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:31.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.768 7f98affff700 1 -- 172.21.2.101:0/3377610738 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f98a80327e0 con 0x7f98b81b80f0 2021-02-10T11:45:31.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.768 7f98b707d700 1 --2- 172.21.2.101:0/3377610738 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f989c0186c0 0x7f989c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:31.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.768 7f98b707d700 1 --2- 172.21.2.101:0/3377610738 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f989c0186c0 0x7f989c01abc0 crc :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:31.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.799 7f98bd34c700 1 -- 172.21.2.101:0/3377610738 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b80ebc00 con 0x7f98b81b80f0 2021-02-10T11:45:31.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.801 7f98affff700 1 -- 172.21.2.101:0/3377610738 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f98a8000c90 con 0x7f98b81b80f0 2021-02-10T11:45:31.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.925 7f98bd34c700 1 -- 172.21.2.101:0/3377610738 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f98b80f0b10 con 0x7f98b81b80f0 2021-02-10T11:45:31.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.926 7f98affff700 1 -- 172.21.2.101:0/3377610738 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v156) v1 ==== 81+0+30374 (secure 0 0 0) 0x7f98a8002000 con 0x7f98b81b80f0 2021-02-10T11:45:31.925 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:45:31.925 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:45:31.925 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 156, 2021-02-10T11:45:31.926 INFO:teuthology.orchestra.run.gibba001.stdout: "fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:45:31.926 INFO:teuthology.orchestra.run.gibba001.stdout: "created": "2021-02-10 11:12:36.788366", 2021-02-10T11:45:31.926 INFO:teuthology.orchestra.run.gibba001.stdout: "modified": "2021-02-10 11:44:57.770382", 2021-02-10T11:45:31.926 INFO:teuthology.orchestra.run.gibba001.stdout: "last_up_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:45:31.926 INFO:teuthology.orchestra.run.gibba001.stdout: "last_in_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:45:31.926 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-02-10T11:45:31.926 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_num": 5799936, 2021-02-10T11:45:31.926 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_set": [ 2021-02-10T11:45:31.926 INFO:teuthology.orchestra.run.gibba001.stdout: "pglog_hardlimit", 2021-02-10T11:45:31.927 INFO:teuthology.orchestra.run.gibba001.stdout: "purged_snapdirs", 2021-02-10T11:45:31.927 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_deletes", 2021-02-10T11:45:31.927 INFO:teuthology.orchestra.run.gibba001.stdout: "sortbitwise" 2021-02-10T11:45:31.927 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:45:31.927 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_version": 4, 2021-02-10T11:45:31.927 INFO:teuthology.orchestra.run.gibba001.stdout: "full_ratio": 0.94999998807907104, 2021-02-10T11:45:31.927 INFO:teuthology.orchestra.run.gibba001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-02-10T11:45:31.927 INFO:teuthology.orchestra.run.gibba001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-02-10T11:45:31.927 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_snapshot": "", 2021-02-10T11:45:31.928 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_max": 33, 2021-02-10T11:45:31.928 INFO:teuthology.orchestra.run.gibba001.stdout: "max_osd": 8, 2021-02-10T11:45:31.928 INFO:teuthology.orchestra.run.gibba001.stdout: "require_min_compat_client": "jewel", 2021-02-10T11:45:31.928 INFO:teuthology.orchestra.run.gibba001.stdout: "min_compat_client": "jewel", 2021-02-10T11:45:31.928 INFO:teuthology.orchestra.run.gibba001.stdout: "require_osd_release": "nautilus", 2021-02-10T11:45:31.928 INFO:teuthology.orchestra.run.gibba001.stdout: "pools": [ 2021-02-10T11:45:31.928 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.928 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 1, 2021-02-10T11:45:31.928 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "rbd", 2021-02-10T11:45:31.929 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:13:08.037472", 2021-02-10T11:45:31.929 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:45:31.929 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:45:31.929 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:45:31.929 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:45:31.929 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:45:31.929 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:45:31.929 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:45:31.929 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:45:31.930 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:45:31.930 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:45:31.930 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:45:31.930 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:45:31.930 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:45:31.930 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:45:31.930 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:45:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:45:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:45:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:45:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:45:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:45:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "16", 2021-02-10T11:45:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:45:31.931 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:45:31.932 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:45:31.932 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:45:31.932 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:45:31.932 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:45:31.932 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:45:31.932 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:45:31.932 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:45:31.932 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:45:31.932 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:45:31.932 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:45:31.933 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:45:31.933 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:45:31.933 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:45:31.933 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:45:31.933 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:45:31.933 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:45:31.933 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:45:31.933 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:45:31.933 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:45:31.934 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:45:31.934 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:45:31.934 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:45:31.934 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:45:31.934 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:45:31.934 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.934 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:45:31.934 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:45:31.934 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:45:31.935 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:45:31.935 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:45:31.935 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:45:31.935 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:45:31.935 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:45:31.935 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:45:31.935 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:45:31.935 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:45:31.935 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:45:31.936 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:45:31.936 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd": {} 2021-02-10T11:45:31.936 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.936 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.936 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.936 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 32, 2021-02-10T11:45:31.936 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_metadata", 2021-02-10T11:45:31.936 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:44:54.736504", 2021-02-10T11:45:31.936 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:45:31.937 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:45:31.937 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:45:31.937 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:45:31.937 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:45:31.937 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:45:31.937 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:45:31.937 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:45:31.937 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:45:31.937 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:45:31.938 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:45:31.938 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:45:31.938 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:45:31.938 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:45:31.938 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:45:31.938 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:45:31.938 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:45:31.938 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:45:31.938 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:45:31.939 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:45:31.939 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.939 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "155", 2021-02-10T11:45:31.939 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:45:31.939 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:45:31.939 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:45:31.939 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:45:31.939 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:45:31.939 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:45:31.939 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:45:31.940 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:45:31.940 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:45:31.940 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:45:31.940 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:45:31.940 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:45:31.940 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:45:31.940 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:45:31.940 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:45:31.941 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:45:31.941 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:45:31.941 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:45:31.941 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:45:31.941 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:45:31.941 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:45:31.941 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:45:31.941 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:45:31.941 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:45:31.942 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:45:31.942 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:45:31.942 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.942 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:45:31.942 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:45:31.942 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:45:31.942 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:45:31.942 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:45:31.943 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:45:31.943 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:45:31.943 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:45:31.943 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:45:31.943 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:45:31.943 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:45:31.943 INFO:teuthology.orchestra.run.gibba001.stdout: "options": { 2021-02-10T11:45:31.943 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_bias": 4, 2021-02-10T11:45:31.943 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_min": 16, 2021-02-10T11:45:31.944 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_priority": 5 2021-02-10T11:45:31.944 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.944 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:45:31.944 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:45:31.944 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": "cephfs" 2021-02-10T11:45:31.944 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.944 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.944 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.944 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.945 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 33, 2021-02-10T11:45:31.945 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_data", 2021-02-10T11:45:31.945 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:44:55.811077", 2021-02-10T11:45:31.945 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:45:31.945 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:45:31.945 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:45:31.945 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:45:31.945 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:45:31.945 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:45:31.945 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:45:31.946 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:45:31.946 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:45:31.946 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:45:31.946 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:45:31.946 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:45:31.946 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:45:31.946 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:45:31.946 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:45:31.946 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:45:31.947 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:45:31.947 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:45:31.947 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:45:31.947 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:45:31.947 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.947 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "156", 2021-02-10T11:45:31.947 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:45:31.947 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:45:31.947 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:45:31.948 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:45:31.948 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:45:31.948 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:45:31.948 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:45:31.948 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:45:31.948 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:45:31.948 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:45:31.948 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:45:31.948 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:45:31.949 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:45:31.949 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:45:31.949 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:45:31.949 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:45:31.949 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:45:31.949 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:45:31.949 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:45:31.949 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:45:31.949 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:45:31.949 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:45:31.950 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:45:31.950 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:45:31.950 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:45:31.950 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:45:31.950 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.950 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:45:31.950 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:45:31.950 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:45:31.951 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:45:31.951 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:45:31.951 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:45:31.951 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:45:31.951 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:45:31.951 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:45:31.951 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:45:31.951 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:45:31.951 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:45:31.952 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:45:31.952 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:45:31.952 INFO:teuthology.orchestra.run.gibba001.stdout: "data": "cephfs" 2021-02-10T11:45:31.952 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.952 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.952 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.952 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:45:31.952 INFO:teuthology.orchestra.run.gibba001.stdout: "osds": [ 2021-02-10T11:45:31.952 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.953 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:45:31.953 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5", 2021-02-10T11:45:31.953 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:45:31.953 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:45:31.953 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:45:31.953 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:45:31.953 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:45:31.953 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:45:31.953 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:45:31.954 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 152, 2021-02-10T11:45:31.954 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:45:31.954 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:45:31.954 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:45:31.954 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.954 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.954 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.954 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6802", 2021-02-10T11:45:31.955 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:45:31.955 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.955 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.955 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.955 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6805", 2021-02-10T11:45:31.955 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:45:31.955 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.955 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.955 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.956 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:45:31.956 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.956 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.956 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.956 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6807", 2021-02-10T11:45:31.956 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:45:31.956 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.956 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.956 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.956 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6809", 2021-02-10T11:45:31.957 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:45:31.957 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.957 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.957 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.957 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:45:31.957 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.957 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.957 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.957 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6817", 2021-02-10T11:45:31.958 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:45:31.958 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.958 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.958 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.958 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6820", 2021-02-10T11:45:31.958 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:45:31.958 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.958 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.958 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.959 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:45:31.959 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.959 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.959 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.959 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6812", 2021-02-10T11:45:31.959 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:45:31.959 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.960 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.960 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.960 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6814", 2021-02-10T11:45:31.960 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:45:31.960 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.960 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.960 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.961 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6805/25093", 2021-02-10T11:45:31.961 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6809/25093", 2021-02-10T11:45:31.961 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6820/25093", 2021-02-10T11:45:31.961 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6814/25093", 2021-02-10T11:45:31.961 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:45:31.961 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:45:31.962 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:45:31.962 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.962 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.962 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.962 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:45:31.962 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "ddc04ce3-b313-4707-8270-585391e38ecc", 2021-02-10T11:45:31.963 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:45:31.963 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:45:31.963 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:45:31.963 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:45:31.963 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:45:31.963 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:45:31.963 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:45:31.964 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 145, 2021-02-10T11:45:31.964 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:45:31.964 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:45:31.964 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:45:31.964 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.964 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.964 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.964 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6803", 2021-02-10T11:45:31.965 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:45:31.965 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.965 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.965 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.965 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6804", 2021-02-10T11:45:31.965 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:45:31.965 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.965 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.965 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.965 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:45:31.965 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.965 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.965 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.966 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6806", 2021-02-10T11:45:31.966 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:45:31.966 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.966 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.966 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.966 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6808", 2021-02-10T11:45:31.966 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:45:31.966 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.966 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.966 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.966 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:45:31.966 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.967 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.967 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.967 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6818", 2021-02-10T11:45:31.967 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:45:31.967 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.967 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.967 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.967 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6821", 2021-02-10T11:45:31.967 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:45:31.967 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.967 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.967 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.967 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:45:31.968 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.968 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.968 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.968 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6810", 2021-02-10T11:45:31.968 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:45:31.968 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.968 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.968 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.968 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6815", 2021-02-10T11:45:31.969 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:45:31.969 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.969 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.969 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.969 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6804/25092", 2021-02-10T11:45:31.969 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6808/25092", 2021-02-10T11:45:31.969 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6821/25092", 2021-02-10T11:45:31.969 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6815/25092", 2021-02-10T11:45:31.970 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:45:31.970 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:45:31.970 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:45:31.970 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.970 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.970 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.970 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:45:31.970 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3", 2021-02-10T11:45:31.970 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:45:31.970 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:45:31.970 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:45:31.970 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:45:31.970 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:45:31.971 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:45:31.971 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:45:31.971 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 154, 2021-02-10T11:45:31.971 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:45:31.971 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:45:31.971 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:45:31.971 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.971 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.971 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.971 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6811", 2021-02-10T11:45:31.971 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:45:31.971 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.971 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.971 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.972 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6813", 2021-02-10T11:45:31.972 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:45:31.972 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.972 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.972 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.972 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:45:31.972 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.972 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.972 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.972 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6816", 2021-02-10T11:45:31.972 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:45:31.972 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.972 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.972 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.973 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6819", 2021-02-10T11:45:31.973 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:45:31.973 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.973 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.973 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.973 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:45:31.973 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.973 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.973 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.973 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6825", 2021-02-10T11:45:31.973 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:45:31.973 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.973 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.973 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.974 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6827", 2021-02-10T11:45:31.974 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:45:31.974 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.974 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.974 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.974 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:45:31.974 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.974 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.974 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.974 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6822", 2021-02-10T11:45:31.974 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:45:31.974 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.975 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.975 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.975 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6823", 2021-02-10T11:45:31.975 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:45:31.975 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.975 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.975 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.975 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6813/25094", 2021-02-10T11:45:31.975 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6819/25094", 2021-02-10T11:45:31.975 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6827/25094", 2021-02-10T11:45:31.975 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6823/25094", 2021-02-10T11:45:31.975 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:45:31.975 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:45:31.976 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:45:31.976 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.976 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.976 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.976 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:45:31.976 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "0d9ef9b1-98aa-4d37-aa16-6305c7d46d52", 2021-02-10T11:45:31.976 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:45:31.976 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:45:31.976 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:45:31.976 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:45:31.976 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:45:31.976 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:45:31.976 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:45:31.977 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 154, 2021-02-10T11:45:31.977 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:45:31.977 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:45:31.977 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:45:31.977 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.977 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.977 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.977 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6824", 2021-02-10T11:45:31.977 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:45:31.977 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.977 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.977 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.977 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6826", 2021-02-10T11:45:31.977 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:45:31.978 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.978 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.978 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.978 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:45:31.978 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.978 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.978 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.978 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6828", 2021-02-10T11:45:31.978 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:45:31.978 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.978 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.978 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.979 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6829", 2021-02-10T11:45:31.979 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:45:31.979 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.979 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.979 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.979 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:45:31.980 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.980 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.980 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.980 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6832", 2021-02-10T11:45:31.980 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:45:31.980 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.980 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.980 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.980 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6833", 2021-02-10T11:45:31.980 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:45:31.980 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.980 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.980 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.981 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:45:31.981 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.981 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.981 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.981 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6830", 2021-02-10T11:45:31.981 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:45:31.981 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.981 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.981 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.981 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6831", 2021-02-10T11:45:31.981 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:45:31.981 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.981 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.982 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.982 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6826/25095", 2021-02-10T11:45:31.982 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6829/25095", 2021-02-10T11:45:31.982 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6833/25095", 2021-02-10T11:45:31.982 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6831/25095", 2021-02-10T11:45:31.982 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:45:31.982 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:45:31.982 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:45:31.982 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.982 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.982 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.982 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:45:31.983 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "36742c21-bfeb-4e11-bce9-8e745378b605", 2021-02-10T11:45:31.983 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:45:31.983 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:45:31.983 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:45:31.983 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:45:31.983 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:45:31.983 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:45:31.983 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:45:31.983 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 145, 2021-02-10T11:45:31.983 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:45:31.983 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:45:31.983 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:45:31.983 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.983 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.984 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.984 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6816", 2021-02-10T11:45:31.984 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:45:31.984 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.984 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.984 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.984 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6818", 2021-02-10T11:45:31.984 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:45:31.984 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.984 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.984 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.984 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:45:31.984 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.985 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.985 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.985 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6820", 2021-02-10T11:45:31.985 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:45:31.985 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.985 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.985 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.985 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6822", 2021-02-10T11:45:31.985 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:45:31.985 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.985 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.985 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.985 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:45:31.985 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.986 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.986 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.986 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6828", 2021-02-10T11:45:31.986 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:45:31.986 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.986 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.986 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.986 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6830", 2021-02-10T11:45:31.986 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:45:31.986 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.986 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.986 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.986 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:45:31.987 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.987 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.987 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.987 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6824", 2021-02-10T11:45:31.987 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:45:31.987 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.987 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.987 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.987 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6826", 2021-02-10T11:45:31.987 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:45:31.987 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.987 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.987 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.988 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6818/36380", 2021-02-10T11:45:31.988 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6822/36380", 2021-02-10T11:45:31.988 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6830/36380", 2021-02-10T11:45:31.988 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6826/36380", 2021-02-10T11:45:31.988 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:45:31.988 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:45:31.988 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:45:31.988 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.988 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.988 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.988 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:45:31.988 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "88e694e8-fe98-4d1b-88c8-8c9907ece60b", 2021-02-10T11:45:31.988 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:45:31.988 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:45:31.989 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:45:31.989 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:45:31.989 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:45:31.989 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:45:31.989 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:45:31.989 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 154, 2021-02-10T11:45:31.989 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:45:31.989 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:45:31.989 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:45:31.989 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.989 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.989 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.989 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6808", 2021-02-10T11:45:31.990 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:45:31.990 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.990 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.990 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.990 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6809", 2021-02-10T11:45:31.990 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:45:31.990 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.990 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.990 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.990 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:45:31.990 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.990 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.990 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.991 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6810", 2021-02-10T11:45:31.991 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:45:31.991 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.991 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.991 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.991 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6811", 2021-02-10T11:45:31.991 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:45:31.991 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.991 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.991 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.991 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:45:31.991 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.991 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.992 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.992 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6814", 2021-02-10T11:45:31.992 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:45:31.992 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.992 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.992 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.992 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6815", 2021-02-10T11:45:31.992 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:45:31.992 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.992 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.992 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.992 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:45:31.992 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.993 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.993 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.993 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6812", 2021-02-10T11:45:31.993 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:45:31.993 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.993 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.993 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.993 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6813", 2021-02-10T11:45:31.993 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:45:31.993 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.993 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.993 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.993 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6809/36383", 2021-02-10T11:45:31.993 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6811/36383", 2021-02-10T11:45:31.994 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6815/36383", 2021-02-10T11:45:31.994 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6813/36383", 2021-02-10T11:45:31.994 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:45:31.994 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:45:31.994 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:45:31.994 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.994 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.994 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.994 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:45:31.994 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "87a13c6c-cf85-4aaf-916b-e73913484a49", 2021-02-10T11:45:31.994 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:45:31.994 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:45:31.995 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:45:31.995 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:45:31.995 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:45:31.995 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:45:31.995 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:45:31.995 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 152, 2021-02-10T11:45:31.995 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:45:31.995 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:45:31.995 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:45:31.995 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.995 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.995 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.995 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6800", 2021-02-10T11:45:31.995 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:45:31.996 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.996 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.996 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.996 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6801", 2021-02-10T11:45:31.996 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:45:31.996 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.996 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.996 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.996 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:45:31.996 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.996 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.996 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.996 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6802", 2021-02-10T11:45:31.996 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:45:31.997 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.997 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.997 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.997 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6803", 2021-02-10T11:45:31.997 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:45:31.997 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.997 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.997 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.997 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:45:31.997 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.998 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.998 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:31.998 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6806", 2021-02-10T11:45:31.998 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:45:31.998 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.998 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:31.998 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:31.998 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6807", 2021-02-10T11:45:31.999 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:45:31.999 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:31.999 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:31.999 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:31.999 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:45:31.999 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:31.999 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:32.000 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:32.000 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6804", 2021-02-10T11:45:32.000 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:45:32.000 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:32.000 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:32.000 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:32.000 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6805", 2021-02-10T11:45:32.000 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:45:32.000 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:32.000 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:32.000 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:32.000 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6801/36381", 2021-02-10T11:45:32.000 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6803/36381", 2021-02-10T11:45:32.001 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6807/36381", 2021-02-10T11:45:32.001 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6805/36381", 2021-02-10T11:45:32.001 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:45:32.001 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:45:32.001 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:45:32.001 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:32.001 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:32.001 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:32.001 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:45:32.001 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "7b90823f-854c-4592-a95b-a7fa98f93254", 2021-02-10T11:45:32.001 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:45:32.001 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:45:32.001 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:45:32.002 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:45:32.002 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:45:32.002 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:45:32.002 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:45:32.002 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 144, 2021-02-10T11:45:32.002 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:45:32.002 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:45:32.002 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:45:32.002 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:32.002 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:32.002 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:32.002 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6817", 2021-02-10T11:45:32.002 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:45:32.003 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:32.003 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:32.003 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:32.003 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6819", 2021-02-10T11:45:32.003 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:45:32.003 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:32.003 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:32.003 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:32.003 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:45:32.003 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:32.003 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:32.003 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:32.003 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6821", 2021-02-10T11:45:32.004 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:45:32.004 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:32.004 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:32.004 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:32.004 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6823", 2021-02-10T11:45:32.004 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:45:32.004 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:32.004 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:32.004 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:32.004 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:45:32.004 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:32.004 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:32.004 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:32.005 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6829", 2021-02-10T11:45:32.005 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:45:32.005 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:32.005 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:32.005 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:32.005 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6831", 2021-02-10T11:45:32.005 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:45:32.005 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:32.005 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:32.005 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:32.005 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:45:32.005 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:45:32.005 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:32.006 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:45:32.006 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6825", 2021-02-10T11:45:32.006 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:45:32.006 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:32.006 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:32.006 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:32.006 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6827", 2021-02-10T11:45:32.006 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:45:32.006 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:32.006 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:32.006 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:32.006 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6819/36382", 2021-02-10T11:45:32.007 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6823/36382", 2021-02-10T11:45:32.007 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6831/36382", 2021-02-10T11:45:32.007 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6827/36382", 2021-02-10T11:45:32.007 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:45:32.007 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:45:32.007 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:45:32.007 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:32.007 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:32.007 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:45:32.007 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_xinfo": [ 2021-02-10T11:45:32.007 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:32.007 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:45:32.007 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:45:32.008 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:45:32.008 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:45:32.008 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:45:32.008 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:45:32.008 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:32.008 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:32.008 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:45:32.008 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:45:32.008 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:45:32.008 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:45:32.008 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:45:32.008 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:45:32.008 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:32.009 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:32.009 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:45:32.009 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:45:32.009 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:45:32.009 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:45:32.009 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:45:32.009 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:45:32.009 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:32.009 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:32.009 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:45:32.009 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:45:32.009 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:45:32.009 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:45:32.009 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:45:32.010 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:45:32.010 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:32.010 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:32.010 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:45:32.010 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:45:32.010 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:45:32.010 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:45:32.010 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:45:32.010 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:45:32.010 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:32.010 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:32.010 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:45:32.011 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:45:32.011 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:45:32.011 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:45:32.011 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:45:32.011 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:45:32.011 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:32.011 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:32.011 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:45:32.011 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:45:32.011 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:45:32.011 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:45:32.011 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:45:32.011 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:45:32.011 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:32.012 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:32.012 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:45:32.012 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:45:32.012 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:45:32.012 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:45:32.012 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:45:32.012 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:45:32.012 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:32.012 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:45:32.012 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap": [], 2021-02-10T11:45:32.012 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap_items": [], 2021-02-10T11:45:32.012 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_temp": [], 2021-02-10T11:45:32.012 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_temp": [], 2021-02-10T11:45:32.013 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklist": {}, 2021-02-10T11:45:32.013 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profiles": { 2021-02-10T11:45:32.013 INFO:teuthology.orchestra.run.gibba001.stdout: "default": { 2021-02-10T11:45:32.013 INFO:teuthology.orchestra.run.gibba001.stdout: "crush-failure-domain": "osd", 2021-02-10T11:45:32.013 INFO:teuthology.orchestra.run.gibba001.stdout: "k": "2", 2021-02-10T11:45:32.013 INFO:teuthology.orchestra.run.gibba001.stdout: "m": "1", 2021-02-10T11:45:32.013 INFO:teuthology.orchestra.run.gibba001.stdout: "plugin": "jerasure", 2021-02-10T11:45:32.013 INFO:teuthology.orchestra.run.gibba001.stdout: "ruleset-failure-domain": "osd", 2021-02-10T11:45:32.013 INFO:teuthology.orchestra.run.gibba001.stdout: "technique": "reed_sol_van" 2021-02-10T11:45:32.013 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:32.013 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:32.013 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps_queue": [], 2021-02-10T11:45:32.013 INFO:teuthology.orchestra.run.gibba001.stdout: "new_removed_snaps": [], 2021-02-10T11:45:32.013 INFO:teuthology.orchestra.run.gibba001.stdout: "new_purged_snaps": [], 2021-02-10T11:45:32.014 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_node_flags": {}, 2021-02-10T11:45:32.014 INFO:teuthology.orchestra.run.gibba001.stdout: "device_class_flags": {} 2021-02-10T11:45:32.014 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:45:32.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.927 7f98bd34c700 1 -- 172.21.2.101:0/3377610738 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f989c0186c0 msgr2=0x7f989c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:32.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.927 7f98bd34c700 1 --2- 172.21.2.101:0/3377610738 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f989c0186c0 0x7f989c01abc0 crc :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:32.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.927 7f98bd34c700 1 -- 172.21.2.101:0/3377610738 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b81b80f0 msgr2=0x7f98b81b84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:32.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.927 7f98bd34c700 1 --2- 172.21.2.101:0/3377610738 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b81b80f0 0x7f98b81b84f0 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 rx=0x7f98a800b5e0 tx=0x7f98a800b980).stop 2021-02-10T11:45:32.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.928 7f98bd34c700 1 -- 172.21.2.101:0/3377610738 shutdown_connections 2021-02-10T11:45:32.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.928 7f98bd34c700 1 --2- 172.21.2.101:0/3377610738 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f98b819da20 0x7f98b819de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:32.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.928 7f98bd34c700 1 --2- 172.21.2.101:0/3377610738 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98b81b80f0 0x7f98b81b84f0 unknown :-1 s=CLOSED pgs=2834 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:32.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.928 7f98bd34c700 1 --2- 172.21.2.101:0/3377610738 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f989c0186c0 0x7f989c01abc0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:32.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.928 7f98bd34c700 1 --2- 172.21.2.101:0/3377610738 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98b81b7cf0 0x7f98b819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:32.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.928 7f98bd34c700 1 -- 172.21.2.101:0/3377610738 >> 172.21.2.101:0/3377610738 conn(0x7f98b80ed810 msgr2=0x7f98b8094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:32.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.928 7f98bd34c700 1 -- 172.21.2.101:0/3377610738 shutdown_connections 2021-02-10T11:45:32.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:31.928 7f98bd34c700 1 -- 172.21.2.101:0/3377610738 wait complete. 2021-02-10T11:45:32.017 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:45:32.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.430 7f8d10929700 1 Processor -- start 2021-02-10T11:45:32.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.430 7f8d10929700 1 -- start start 2021-02-10T11:45:32.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.430 7f8d10929700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8d0c1b0870 0x7f8d0c0f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:32.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.430 7f8d10929700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d0c0efe00 0x7f8d0c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:32.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.430 7f8d10929700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d0c1b0c70 con 0x7f8d0c1b0470 2021-02-10T11:45:32.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.430 7f8d10929700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d0c0f0b50 con 0x7f8d0c1b0870 2021-02-10T11:45:32.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.430 7f8d10929700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c0f0820 con 0x7f8d0c0efe00 2021-02-10T11:45:32.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.430 7f8d0a61c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8d0c1b0470 0x7f8d0c0ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:32920/0 (socket says 172.21.2.101:32920) 2021-02-10T11:45:32.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.430 7f8d0a61c700 1 -- 172.21.2.101:0/1957890899 learned_addr learned my addr 172.21.2.101:0/1957890899 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:32.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.431 7f8d0961a700 1 -- 172.21.2.101:0/1957890899 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1987318786 0 0) 0x7f8cfc001180 con 0x7f8d0c1b0470 2021-02-10T11:45:32.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.431 7f8d0961a700 1 -- 172.21.2.101:0/1957890899 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cf8002fe0 con 0x7f8d0c1b0470 2021-02-10T11:45:32.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.431 7f8d0ae1d700 1 --2- 172.21.2.101:0/1957890899 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d0c0efe00 0x7f8d0c0f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:32.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.431 7f8d0961a700 1 -- 172.21.2.101:0/1957890899 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1460541966 0 0) 0x7f8cfc0010e0 con 0x7f8d0c1b0470 2021-02-10T11:45:32.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.431 7f8d0961a700 1 -- 172.21.2.101:0/1957890899 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d0c0efe00 msgr2=0x7f8d0c0f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:32.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.431 7f8d09e1b700 1 --2- 172.21.2.101:0/1957890899 >> v2:172.21.2.108:3300/0 conn(0x7f8d0c1b0870 0x7f8d0c0f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:32.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.431 7f8d0961a700 1 --2- 172.21.2.101:0/1957890899 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d0c0efe00 0x7f8d0c0f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:32.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.431 7f8d0961a700 1 -- 172.21.2.101:0/1957890899 >> v2:172.21.2.108:3300/0 conn(0x7f8d0c1b0870 msgr2=0x7f8d0c0f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:32.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.431 7f8d0961a700 1 --2- 172.21.2.101:0/1957890899 >> v2:172.21.2.108:3300/0 conn(0x7f8d0c1b0870 0x7f8d0c0f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:32.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.431 7f8d0961a700 1 -- 172.21.2.101:0/1957890899 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c0e52e0 con 0x7f8d0c1b0470 2021-02-10T11:45:32.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.431 7f8d0961a700 1 -- 172.21.2.101:0/1957890899 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f8cfc001600 con 0x7f8d0c1b0470 2021-02-10T11:45:32.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.431 7f8d0961a700 1 -- 172.21.2.101:0/1957890899 >> v1:172.21.2.101:6789/0 conn(0x7f8d0c1b0470 legacy=0x7f8d0c0ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:32.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.431 7f8d0961a700 1 --2- 172.21.2.101:0/1957890899 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8cf8003ae0 0x7f8cf8005f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:32.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.431 7f8d0961a700 1 --2- 172.21.2.101:0/1957890899 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8cf8006810 0x7f8cf8008d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:32.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.431 7f8d0961a700 1 --2- 172.21.2.101:0/1957890899 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8cf8009630 0x7f8cf800bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:32.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.431 7f8d0961a700 1 -- 172.21.2.101:0/1957890899 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf800c370 con 0x7f8cf8006810 2021-02-10T11:45:32.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.431 7f8d0961a700 1 -- 172.21.2.101:0/1957890899 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf800c590 con 0x7f8cf8003ae0 2021-02-10T11:45:32.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.431 7f8d0961a700 1 -- 172.21.2.101:0/1957890899 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf800c7b0 con 0x7f8cf8009630 2021-02-10T11:45:32.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.431 7f8d09e1b700 1 --2- 172.21.2.101:0/1957890899 >> v2:172.21.2.108:3300/0 conn(0x7f8d0c1b0870 0x7f8d0c0f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:32.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.431 7f8d09e1b700 1 --2- 172.21.2.101:0/1957890899 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8cf8006810 0x7f8cf8008d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:32.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.431 7f8d0a61c700 1 --2- 172.21.2.101:0/1957890899 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8cf8003ae0 0x7f8cf8005f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:32.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.431 7f8d0ae1d700 1 --2- 172.21.2.101:0/1957890899 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8cf8009630 0x7f8cf800bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:32.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.432 7f8d09e1b700 1 -- 172.21.2.101:0/1957890899 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8cf8009630 msgr2=0x7f8cf800bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:32.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.432 7f8d09e1b700 1 --2- 172.21.2.101:0/1957890899 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8cf8009630 0x7f8cf800bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:32.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.432 7f8d09e1b700 1 -- 172.21.2.101:0/1957890899 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8cf8003ae0 msgr2=0x7f8cf8005f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:32.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.432 7f8d09e1b700 1 --2- 172.21.2.101:0/1957890899 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8cf8003ae0 0x7f8cf8005f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:32.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.432 7f8d09e1b700 1 -- 172.21.2.101:0/1957890899 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf800ca50 con 0x7f8cf8006810 2021-02-10T11:45:32.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.432 7f8d09e1b700 1 --2- 172.21.2.101:0/1957890899 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8cf8006810 0x7f8cf8008d30 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 rx=0x7f8cf404ecb0 tx=0x7f8cf4050300).ready entity=mon.0 client_cookie=a811bd77a917a4a1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:32.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.432 7f8d0961a700 1 -- 172.21.2.101:0/1957890899 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8cf404eda0 con 0x7f8cf8006810 2021-02-10T11:45:32.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.432 7f8d0961a700 1 -- 172.21.2.101:0/1957890899 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf40506f0 con 0x7f8cf8006810 2021-02-10T11:45:32.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.432 7f8d0961a700 1 -- 172.21.2.101:0/1957890899 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8cf4050c10 con 0x7f8cf8006810 2021-02-10T11:45:32.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.432 7f8d10929700 1 -- 172.21.2.101:0/1957890899 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8cf8006810 msgr2=0x7f8cf8008d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:32.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.432 7f8d10929700 1 --2- 172.21.2.101:0/1957890899 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8cf8006810 0x7f8cf8008d30 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 rx=0x7f8cf404ecb0 tx=0x7f8cf4050300).stop 2021-02-10T11:45:32.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.432 7f8d0ae1d700 1 -- 172.21.2.101:0/1957890899 reap_dead start 2021-02-10T11:45:32.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.433 7f8d10929700 1 -- 172.21.2.101:0/1957890899 shutdown_connections 2021-02-10T11:45:32.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.433 7f8d10929700 1 -- 172.21.2.101:0/1957890899 >> 172.21.2.101:0/1957890899 conn(0x7f8d0c0ce3e0 msgr2=0x7f8d0c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:32.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.433 7f8d10929700 1 -- 172.21.2.101:0/1957890899 shutdown_connections 2021-02-10T11:45:32.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.433 7f8d10929700 1 -- 172.21.2.101:0/1957890899 wait complete. 2021-02-10T11:45:32.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.448 7f8d10929700 1 Processor -- start 2021-02-10T11:45:32.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.448 7f8d10929700 1 -- start start 2021-02-10T11:45:32.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.448 7f8d10929700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d0c108990 0x7f8d0c108d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:32.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.448 7f8d10929700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d0c0af260 0x7f8d0c0af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:32.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.448 7f8d10929700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d0c187e20 0x7f8d0c188220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:32.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.448 7f8d10929700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c0afc80 con 0x7f8d0c0af260 2021-02-10T11:45:32.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.448 7f8d10929700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c0b04d0 con 0x7f8d0c108990 2021-02-10T11:45:32.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.448 7f8d10929700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c1095c0 con 0x7f8d0c187e20 2021-02-10T11:45:32.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.448 7f8d09e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d0c0af260 0x7f8d0c0af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:32.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.448 7f8d09e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d0c0af260 0x7f8d0c0af660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60670/0 (socket says 172.21.2.101:60670) 2021-02-10T11:45:32.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.448 7f8d09e1b700 1 -- 172.21.2.101:0/11950831 learned_addr learned my addr 172.21.2.101:0/11950831 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:32.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.449 7f8d09e1b700 1 -- 172.21.2.101:0/11950831 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d0c187e20 msgr2=0x7f8d0c188220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:32.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.449 7f8d09e1b700 1 --2- 172.21.2.101:0/11950831 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d0c187e20 0x7f8d0c188220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:32.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.449 7f8d09e1b700 1 -- 172.21.2.101:0/11950831 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d0c108990 msgr2=0x7f8d0c108d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:32.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.449 7f8d09e1b700 1 --2- 172.21.2.101:0/11950831 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d0c108990 0x7f8d0c108d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:32.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.449 7f8d09e1b700 1 -- 172.21.2.101:0/11950831 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c188840 con 0x7f8d0c0af260 2021-02-10T11:45:32.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.449 7f8d09e1b700 1 --2- 172.21.2.101:0/11950831 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d0c0af260 0x7f8d0c0af660 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 rx=0x7f8cf404e650 tx=0x7f8cf4050d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:32.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.449 7f8d02ffd700 1 -- 172.21.2.101:0/11950831 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8cf404b510 con 0x7f8d0c0af260 2021-02-10T11:45:32.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.449 7f8d02ffd700 1 -- 172.21.2.101:0/11950831 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf404b7d0 con 0x7f8d0c0af260 2021-02-10T11:45:32.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.449 7f8d10929700 1 -- 172.21.2.101:0/11950831 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d0c188840 con 0x7f8d0c0af260 2021-02-10T11:45:32.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.449 7f8d10929700 1 -- 172.21.2.101:0/11950831 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0c188dd0 con 0x7f8d0c0af260 2021-02-10T11:45:32.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.449 7f8d02ffd700 1 -- 172.21.2.101:0/11950831 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8cf404bcb0 con 0x7f8d0c0af260 2021-02-10T11:45:32.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.449 7f8d02ffd700 1 -- 172.21.2.101:0/11950831 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8cf404b300 con 0x7f8d0c0af260 2021-02-10T11:45:32.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.450 7f8d02ffd700 1 --2- 172.21.2.101:0/11950831 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8cf0018730 0x7f8cf001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:32.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.450 7f8d02ffd700 1 -- 172.21.2.101:0/11950831 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8cf4076b50 con 0x7f8d0c0af260 2021-02-10T11:45:32.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.450 7f8d0a61c700 1 --2- 172.21.2.101:0/11950831 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8cf0018730 0x7f8cf001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:32.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.450 7f8d0a61c700 1 --2- 172.21.2.101:0/11950831 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8cf0018730 0x7f8cf001ac10 crc :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:32.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.480 7f8d10929700 1 -- 172.21.2.101:0/11950831 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d0c0affa0 con 0x7f8d0c0af260 2021-02-10T11:45:32.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.482 7f8d02ffd700 1 -- 172.21.2.101:0/11950831 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8cf4000ce0 con 0x7f8d0c0af260 2021-02-10T11:45:32.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.611 7f8d10929700 1 -- 172.21.2.101:0/11950831 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d0c0affa0 con 0x7f8d0c0af260 2021-02-10T11:45:32.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.611 7f8d02ffd700 1 -- 172.21.2.101:0/11950831 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+2105 (secure 0 0 0) 0x7f8cf404b300 con 0x7f8d0c0af260 2021-02-10T11:45:32.610 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 132 2021-02-10T11:45:32.610 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:45:32.610 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":132,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":12135,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/3467749863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3467749863},{"type":"v1","addr":"172.21.2.101:6835","nonce":3467749863}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":128}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:44:56.768458","modified":"2021-02-10 11:44:58.773048","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8242":{"gid":8242,"name":"b","rank":0,"incarnation":129,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/485291097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":485291097},{"type":"v1","addr":"172.21.2.108:6833","nonce":485291097}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2021-02-10T11:45:32.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.613 7f8d10929700 1 -- 172.21.2.101:0/11950831 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8cf0018730 msgr2=0x7f8cf001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:32.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.613 7f8d10929700 1 --2- 172.21.2.101:0/11950831 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8cf0018730 0x7f8cf001ac10 crc :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:32.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.613 7f8d10929700 1 -- 172.21.2.101:0/11950831 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d0c0af260 msgr2=0x7f8d0c0af660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:32.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.613 7f8d10929700 1 --2- 172.21.2.101:0/11950831 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d0c0af260 0x7f8d0c0af660 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 rx=0x7f8cf404e650 tx=0x7f8cf4050d10).stop 2021-02-10T11:45:32.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.613 7f8d10929700 1 -- 172.21.2.101:0/11950831 shutdown_connections 2021-02-10T11:45:32.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.613 7f8d10929700 1 --2- 172.21.2.101:0/11950831 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8cf0018730 0x7f8cf001ac10 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:32.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.613 7f8d10929700 1 --2- 172.21.2.101:0/11950831 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8d0c187e20 0x7f8d0c188220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:32.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.613 7f8d10929700 1 --2- 172.21.2.101:0/11950831 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8d0c0af260 0x7f8d0c0af660 unknown :-1 s=CLOSED pgs=2837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:32.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.613 7f8d10929700 1 --2- 172.21.2.101:0/11950831 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8d0c108990 0x7f8d0c108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:32.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.613 7f8d10929700 1 -- 172.21.2.101:0/11950831 >> 172.21.2.101:0/11950831 conn(0x7f8d0c0bc6c0 msgr2=0x7f8d0c0b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:32.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.613 7f8d10929700 1 -- 172.21.2.101:0/11950831 shutdown_connections 2021-02-10T11:45:32.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:32.614 7f8d10929700 1 -- 172.21.2.101:0/11950831 wait complete. 2021-02-10T11:45:32.620 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs fail cephfs 2021-02-10T11:45:33.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.036 7efd85cc8700 1 Processor -- start 2021-02-10T11:45:33.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.036 7efd85cc8700 1 -- start start 2021-02-10T11:45:33.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.036 7efd85cc8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7efd800d0340 0x7efd800ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:33.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.036 7efd85cc8700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd800f0b10 con 0x7efd800d1c70 2021-02-10T11:45:33.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.036 7efd85cc8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd801b0be0 con 0x7efd800d1870 2021-02-10T11:45:33.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.036 7efd85cc8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd800e5ed0 con 0x7efd800d0340 2021-02-10T11:45:33.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.036 7efd7effd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7efd800d0340 0x7efd800ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:33.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.036 7efd7effd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7efd800d0340 0x7efd800ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60678/0 (socket says 172.21.2.101:60678) 2021-02-10T11:45:33.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.036 7efd7effd700 1 -- 172.21.2.101:0/71840892 learned_addr learned my addr 172.21.2.101:0/71840892 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:33.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.036 7efd7effd700 1 -- 172.21.2.101:0/71840892 >> v1:172.21.2.108:6789/0 conn(0x7efd800d1870 legacy=0x7efd801b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:45:33.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.036 7efd7effd700 1 -- 172.21.2.101:0/71840892 >> v1:172.21.2.101:6789/0 conn(0x7efd800d1c70 legacy=0x7efd800efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:33.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.036 7efd7effd700 1 -- 172.21.2.101:0/71840892 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd800f0450 con 0x7efd800d0340 2021-02-10T11:45:33.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.037 7efd7effd700 1 --2- 172.21.2.101:0/71840892 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd800d0340 0x7efd800ef640 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 rx=0x7efd68006700 tx=0x7efd68009b00).ready entity=mon.0 client_cookie=64a666acbea5a2d9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:33.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.037 7efd7e7fc700 1 -- 172.21.2.101:0/71840892 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd6800c0c0 con 0x7efd800d0340 2021-02-10T11:45:33.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.037 7efd7e7fc700 1 -- 172.21.2.101:0/71840892 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6800c340 con 0x7efd800d0340 2021-02-10T11:45:33.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.037 7efd7e7fc700 1 -- 172.21.2.101:0/71840892 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd6800c820 con 0x7efd800d0340 2021-02-10T11:45:33.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.037 7efd85cc8700 1 -- 172.21.2.101:0/71840892 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd800d0340 msgr2=0x7efd800ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:33.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.037 7efd85cc8700 1 --2- 172.21.2.101:0/71840892 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd800d0340 0x7efd800ef640 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 rx=0x7efd68006700 tx=0x7efd68009b00).stop 2021-02-10T11:45:33.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.037 7efd85cc8700 1 -- 172.21.2.101:0/71840892 shutdown_connections 2021-02-10T11:45:33.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.037 7efd85cc8700 1 --2- 172.21.2.101:0/71840892 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd800d0340 0x7efd800ef640 unknown :-1 s=CLOSED pgs=2839 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:33.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.037 7efd85cc8700 1 -- 172.21.2.101:0/71840892 >> 172.21.2.101:0/71840892 conn(0x7efd800ce450 msgr2=0x7efd800d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:33.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.037 7efd85cc8700 1 -- 172.21.2.101:0/71840892 shutdown_connections 2021-02-10T11:45:33.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.037 7efd85cc8700 1 -- 172.21.2.101:0/71840892 wait complete. 2021-02-10T11:45:33.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.052 7efd85cc8700 1 Processor -- start 2021-02-10T11:45:33.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.052 7efd85cc8700 1 -- start start 2021-02-10T11:45:33.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.052 7efd85cc8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd8008a420 0x7efd8008f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:33.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.052 7efd85cc8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd8008a820 0x7efd800b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:33.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.052 7efd85cc8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd8008fe80 0x7efd800b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:33.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.052 7efd85cc8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8008f2e0 con 0x7efd8008a420 2021-02-10T11:45:33.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.052 7efd85cc8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd800b1d60 con 0x7efd8008fe80 2021-02-10T11:45:33.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.052 7efd85cc8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd801c6910 con 0x7efd8008a820 2021-02-10T11:45:33.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.053 7efd7f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd8008a420 0x7efd8008f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:33.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.053 7efd7f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd8008a420 0x7efd8008f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60684/0 (socket says 172.21.2.101:60684) 2021-02-10T11:45:33.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.053 7efd7f7fe700 1 -- 172.21.2.101:0/3892972643 learned_addr learned my addr 172.21.2.101:0/3892972643 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:33.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.053 7efd7f7fe700 1 -- 172.21.2.101:0/3892972643 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd8008a820 msgr2=0x7efd800b0e40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:33.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.053 7efd7f7fe700 1 --2- 172.21.2.101:0/3892972643 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd8008a820 0x7efd800b0e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:33.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.053 7efd7f7fe700 1 -- 172.21.2.101:0/3892972643 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd8008fe80 msgr2=0x7efd800b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:33.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.053 7efd7f7fe700 1 --2- 172.21.2.101:0/3892972643 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd8008fe80 0x7efd800b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:33.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.053 7efd7f7fe700 1 -- 172.21.2.101:0/3892972643 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8008e0c0 con 0x7efd8008a420 2021-02-10T11:45:33.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.053 7efd7f7fe700 1 --2- 172.21.2.101:0/3892972643 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd8008a420 0x7efd8008f860 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 rx=0x7efd7000b530 tx=0x7efd7000b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:33.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.053 7efd77fff700 1 -- 172.21.2.101:0/3892972643 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd7000bcc0 con 0x7efd8008a420 2021-02-10T11:45:33.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.053 7efd77fff700 1 -- 172.21.2.101:0/3892972643 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd70006020 con 0x7efd8008a420 2021-02-10T11:45:33.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.053 7efd77fff700 1 -- 172.21.2.101:0/3892972643 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd700054a0 con 0x7efd8008a420 2021-02-10T11:45:33.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.053 7efd85cc8700 1 -- 172.21.2.101:0/3892972643 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd8008e0c0 con 0x7efd8008a420 2021-02-10T11:45:33.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.053 7efd85cc8700 1 -- 172.21.2.101:0/3892972643 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8008e650 con 0x7efd8008a420 2021-02-10T11:45:33.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.054 7efd77fff700 1 -- 172.21.2.101:0/3892972643 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7efd70006200 con 0x7efd8008a420 2021-02-10T11:45:33.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.054 7efd77fff700 1 --2- 172.21.2.101:0/3892972643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd640186c0 0x7efd6401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:33.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.054 7efd77fff700 1 -- 172.21.2.101:0/3892972643 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5573+0+0 (secure 0 0 0) 0x7efd70032560 con 0x7efd8008a420 2021-02-10T11:45:33.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.054 7efd7effd700 1 --2- 172.21.2.101:0/3892972643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd640186c0 0x7efd6401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:33.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.054 7efd7effd700 1 --2- 172.21.2.101:0/3892972643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd640186c0 0x7efd6401abc0 crc :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:33.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.086 7efd85cc8700 1 -- 172.21.2.101:0/3892972643 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd800ffa00 con 0x7efd8008a420 2021-02-10T11:45:33.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.089 7efd77fff700 1 -- 172.21.2.101:0/3892972643 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7efd70000c10 con 0x7efd8008a420 2021-02-10T11:45:33.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.215 7efd85cc8700 1 -- 172.21.2.101:0/3892972643 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7efd800f06f0 con 0x7efd8008a420 2021-02-10T11:45:33.983 INFO:tasks.ceph.mon.a.gibba001.stderr:2021-02-10 11:45:33.985 7f974d04d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-02-10T11:45:33.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:33.991 7efd77fff700 1 -- 172.21.2.101:0/3892972643 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v133) v1 ==== 161+0+0 (secure 0 0 0) 0x7efd7001f520 con 0x7efd8008a420 2021-02-10T11:45:34.032 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:45:34.033 INFO:teuthology.orchestra.run.gibba001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-02-10T11:45:34.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.036 7efd85cc8700 1 -- 172.21.2.101:0/3892972643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd640186c0 msgr2=0x7efd6401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:34.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.036 7efd85cc8700 1 --2- 172.21.2.101:0/3892972643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd640186c0 0x7efd6401abc0 crc :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:34.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.036 7efd85cc8700 1 -- 172.21.2.101:0/3892972643 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd8008a420 msgr2=0x7efd8008f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:34.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.036 7efd85cc8700 1 --2- 172.21.2.101:0/3892972643 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd8008a420 0x7efd8008f860 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 rx=0x7efd7000b530 tx=0x7efd7000b8d0).stop 2021-02-10T11:45:34.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.037 7efd85cc8700 1 -- 172.21.2.101:0/3892972643 shutdown_connections 2021-02-10T11:45:34.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.037 7efd85cc8700 1 --2- 172.21.2.101:0/3892972643 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efd640186c0 0x7efd6401abc0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:34.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.037 7efd85cc8700 1 --2- 172.21.2.101:0/3892972643 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efd8008a820 0x7efd800b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:34.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.037 7efd85cc8700 1 --2- 172.21.2.101:0/3892972643 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efd8008fe80 0x7efd800b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:34.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.037 7efd85cc8700 1 --2- 172.21.2.101:0/3892972643 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efd8008a420 0x7efd8008f860 unknown :-1 s=CLOSED pgs=2840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:34.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.037 7efd85cc8700 1 -- 172.21.2.101:0/3892972643 >> 172.21.2.101:0/3892972643 conn(0x7efd800a74c0 msgr2=0x7efd800bc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:34.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.037 7efd85cc8700 1 -- 172.21.2.101:0/3892972643 shutdown_connections 2021-02-10T11:45:34.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.037 7efd85cc8700 1 -- 172.21.2.101:0/3892972643 wait complete. 2021-02-10T11:45:34.043 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:45:34.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.458 7f8a54d82700 1 Processor -- start 2021-02-10T11:45:34.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.458 7f8a54d82700 1 -- start start 2021-02-10T11:45:34.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.459 7f8a54d82700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8a500d1c70 0x7f8a500f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:34.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.459 7f8a54d82700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a500f0b10 con 0x7f8a500d0340 2021-02-10T11:45:34.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.459 7f8a54d82700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a501b0be0 con 0x7f8a500d1870 2021-02-10T11:45:34.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.459 7f8a54d82700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a500e5ed0 con 0x7f8a500d1c70 2021-02-10T11:45:34.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.459 7f8a4e312700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8a500d0340 0x7f8a500ef640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:32956/0 (socket says 172.21.2.101:32956) 2021-02-10T11:45:34.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.459 7f8a4e312700 1 -- 172.21.2.101:0/449139564 learned_addr learned my addr 172.21.2.101:0/449139564 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:34.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.459 7f8a4db11700 1 -- 172.21.2.101:0/449139564 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2707844620 0 0) 0x7f8a400010b0 con 0x7f8a500d0340 2021-02-10T11:45:34.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.459 7f8a4db11700 1 -- 172.21.2.101:0/449139564 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a3c002fe0 con 0x7f8a500d0340 2021-02-10T11:45:34.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4f314700 1 --2- 172.21.2.101:0/449139564 >> v2:172.21.2.108:3300/0 conn(0x7f8a500d1c70 0x7f8a500f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:34.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4db11700 1 -- 172.21.2.101:0/449139564 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2917616700 0 0) 0x7f8a400010b0 con 0x7f8a500d0340 2021-02-10T11:45:34.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4db11700 1 -- 172.21.2.101:0/449139564 >> v2:172.21.2.108:3300/0 conn(0x7f8a500d1c70 msgr2=0x7f8a500f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:34.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4db11700 1 --2- 172.21.2.101:0/449139564 >> v2:172.21.2.108:3300/0 conn(0x7f8a500d1c70 0x7f8a500f20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:34.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4db11700 1 -- 172.21.2.101:0/449139564 >> v1:172.21.2.108:6789/0 conn(0x7f8a500d1870 legacy=0x7f8a501b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:34.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4db11700 1 -- 172.21.2.101:0/449139564 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a500efe30 con 0x7f8a500d0340 2021-02-10T11:45:34.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4db11700 1 -- 172.21.2.101:0/449139564 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f8a400015b0 con 0x7f8a500d0340 2021-02-10T11:45:34.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4db11700 1 -- 172.21.2.101:0/449139564 >> v1:172.21.2.101:6789/0 conn(0x7f8a500d0340 legacy=0x7f8a500ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:34.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4db11700 1 --2- 172.21.2.101:0/449139564 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a3c003a60 0x7f8a3c005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:34.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4db11700 1 --2- 172.21.2.101:0/449139564 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a3c006790 0x7f8a3c008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:34.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4db11700 1 --2- 172.21.2.101:0/449139564 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a3c009590 0x7f8a3c00bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:34.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4db11700 1 -- 172.21.2.101:0/449139564 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c00c300 con 0x7f8a3c003a60 2021-02-10T11:45:34.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4db11700 1 -- 172.21.2.101:0/449139564 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c00c520 con 0x7f8a3c006790 2021-02-10T11:45:34.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4db11700 1 -- 172.21.2.101:0/449139564 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c00c740 con 0x7f8a3c009590 2021-02-10T11:45:34.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4eb13700 1 --2- 172.21.2.101:0/449139564 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a3c003a60 0x7f8a3c005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:34.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4e312700 1 --2- 172.21.2.101:0/449139564 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a3c006790 0x7f8a3c008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:34.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4f314700 1 --2- 172.21.2.101:0/449139564 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a3c009590 0x7f8a3c00bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:34.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4eb13700 1 -- 172.21.2.101:0/449139564 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a3c009590 msgr2=0x7f8a3c00bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:34.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4eb13700 1 --2- 172.21.2.101:0/449139564 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a3c009590 0x7f8a3c00bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:34.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4eb13700 1 -- 172.21.2.101:0/449139564 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a3c006790 msgr2=0x7f8a3c008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:34.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4eb13700 1 --2- 172.21.2.101:0/449139564 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a3c006790 0x7f8a3c008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:34.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4eb13700 1 -- 172.21.2.101:0/449139564 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3c00c9e0 con 0x7f8a3c003a60 2021-02-10T11:45:34.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4f314700 1 -- 172.21.2.101:0/449139564 reap_dead start 2021-02-10T11:45:34.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.460 7f8a4eb13700 1 --2- 172.21.2.101:0/449139564 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a3c003a60 0x7f8a3c005ef0 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 rx=0x7f8a3804a750 tx=0x7f8a3804da60).ready entity=mon.0 client_cookie=ae6ae99ea076636a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:34.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.461 7f8a4db11700 1 -- 172.21.2.101:0/449139564 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8a380500c0 con 0x7f8a3c003a60 2021-02-10T11:45:34.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.461 7f8a4db11700 1 -- 172.21.2.101:0/449139564 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a380500c0 con 0x7f8a3c003a60 2021-02-10T11:45:34.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.461 7f8a4db11700 1 -- 172.21.2.101:0/449139564 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8a380503c0 con 0x7f8a3c003a60 2021-02-10T11:45:34.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.461 7f8a54d82700 1 -- 172.21.2.101:0/449139564 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a3c003a60 msgr2=0x7f8a3c005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:34.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.461 7f8a54d82700 1 --2- 172.21.2.101:0/449139564 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a3c003a60 0x7f8a3c005ef0 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 rx=0x7f8a3804a750 tx=0x7f8a3804da60).stop 2021-02-10T11:45:34.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.461 7f8a54d82700 1 -- 172.21.2.101:0/449139564 shutdown_connections 2021-02-10T11:45:34.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.461 7f8a54d82700 1 --2- 172.21.2.101:0/449139564 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a3c003a60 0x7f8a3c005ef0 unknown :-1 s=CLOSED pgs=2842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:34.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.461 7f8a54d82700 1 -- 172.21.2.101:0/449139564 >> 172.21.2.101:0/449139564 conn(0x7f8a500ce450 msgr2=0x7f8a500d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:34.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.461 7f8a54d82700 1 -- 172.21.2.101:0/449139564 shutdown_connections 2021-02-10T11:45:34.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.461 7f8a54d82700 1 -- 172.21.2.101:0/449139564 wait complete. 2021-02-10T11:45:34.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.476 7f8a54d82700 1 Processor -- start 2021-02-10T11:45:34.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.476 7f8a54d82700 1 -- start start 2021-02-10T11:45:34.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.476 7f8a54d82700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a501a4a60 0x7f8a500c1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:34.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.476 7f8a54d82700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a500c2290 0x7f8a500c2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:34.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.476 7f8a54d82700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a500c2cb0 0x7f8a500bfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:34.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.476 7f8a54d82700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a500c31c0 con 0x7f8a500c2290 2021-02-10T11:45:34.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.476 7f8a54d82700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a500f05f0 con 0x7f8a500c2cb0 2021-02-10T11:45:34.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.476 7f8a54d82700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a500c0340 con 0x7f8a501a4a60 2021-02-10T11:45:34.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.477 7f8a4e312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a500c2290 0x7f8a500c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:34.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.477 7f8a4e312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a500c2290 0x7f8a500c2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60704/0 (socket says 172.21.2.101:60704) 2021-02-10T11:45:34.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.477 7f8a4e312700 1 -- 172.21.2.101:0/405294894 learned_addr learned my addr 172.21.2.101:0/405294894 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:34.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.477 7f8a4e312700 1 -- 172.21.2.101:0/405294894 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a501a4a60 msgr2=0x7f8a500c1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:34.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.477 7f8a4e312700 1 --2- 172.21.2.101:0/405294894 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a501a4a60 0x7f8a500c1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:34.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.477 7f8a4e312700 1 -- 172.21.2.101:0/405294894 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a500c2cb0 msgr2=0x7f8a500bfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:34.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.477 7f8a4e312700 1 --2- 172.21.2.101:0/405294894 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a500c2cb0 0x7f8a500bfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:34.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.477 7f8a4e312700 1 -- 172.21.2.101:0/405294894 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a500c0560 con 0x7f8a500c2290 2021-02-10T11:45:34.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.477 7f8a4e312700 1 --2- 172.21.2.101:0/405294894 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a500c2290 0x7f8a500c2690 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 rx=0x7f8a40001660 tx=0x7f8a4000a930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:34.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.477 7f8a477fe700 1 -- 172.21.2.101:0/405294894 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8a4000ad20 con 0x7f8a500c2290 2021-02-10T11:45:34.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.477 7f8a477fe700 1 -- 172.21.2.101:0/405294894 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a400016a0 con 0x7f8a500c2290 2021-02-10T11:45:34.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.477 7f8a54d82700 1 -- 172.21.2.101:0/405294894 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a500c0560 con 0x7f8a500c2290 2021-02-10T11:45:34.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.477 7f8a477fe700 1 -- 172.21.2.101:0/405294894 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8a40004540 con 0x7f8a500c2290 2021-02-10T11:45:34.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.477 7f8a54d82700 1 -- 172.21.2.101:0/405294894 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a500c0af0 con 0x7f8a500c2290 2021-02-10T11:45:34.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.478 7f8a477fe700 1 -- 172.21.2.101:0/405294894 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8a40008000 con 0x7f8a500c2290 2021-02-10T11:45:34.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.478 7f8a477fe700 1 --2- 172.21.2.101:0/405294894 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a340186c0 0x7f8a3401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:34.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.478 7f8a477fe700 1 -- 172.21.2.101:0/405294894 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f8a400325a0 con 0x7f8a500c2290 2021-02-10T11:45:34.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.478 7f8a4eb13700 1 --2- 172.21.2.101:0/405294894 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a340186c0 0x7f8a3401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:34.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.479 7f8a4eb13700 1 --2- 172.21.2.101:0/405294894 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a340186c0 0x7f8a3401abc0 crc :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:34.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.509 7f8a54d82700 1 -- 172.21.2.101:0/405294894 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a500c3140 con 0x7f8a500c2290 2021-02-10T11:45:34.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.511 7f8a477fe700 1 -- 172.21.2.101:0/405294894 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8a40000c90 con 0x7f8a500c2290 2021-02-10T11:45:34.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.640 7f8a54d82700 1 -- 172.21.2.101:0/405294894 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a500c3140 con 0x7f8a500c2290 2021-02-10T11:45:34.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.640 7f8a477fe700 1 -- 172.21.2.101:0/405294894 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+1763 (secure 0 0 0) 0x7f8a4000ad20 con 0x7f8a500c2290 2021-02-10T11:45:34.639 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:45:34.639 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":133,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":12135,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/3467749863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":3467749863},{"type":"v1","addr":"172.21.2.101:6835","nonce":3467749863}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":128}],"filesystems":[{"mdsmap":{"epoch":133,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:44:56.768458","modified":"2021-02-10 11:45:33.985982","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":157,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2021-02-10T11:45:34.639 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 133 2021-02-10T11:45:34.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.642 7f8a54d82700 1 -- 172.21.2.101:0/405294894 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a340186c0 msgr2=0x7f8a3401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:34.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.642 7f8a54d82700 1 --2- 172.21.2.101:0/405294894 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a340186c0 0x7f8a3401abc0 crc :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:34.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.642 7f8a54d82700 1 -- 172.21.2.101:0/405294894 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a500c2290 msgr2=0x7f8a500c2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:34.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.642 7f8a54d82700 1 --2- 172.21.2.101:0/405294894 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a500c2290 0x7f8a500c2690 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 rx=0x7f8a40001660 tx=0x7f8a4000a930).stop 2021-02-10T11:45:34.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.642 7f8a54d82700 1 -- 172.21.2.101:0/405294894 shutdown_connections 2021-02-10T11:45:34.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.642 7f8a54d82700 1 --2- 172.21.2.101:0/405294894 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a500c2cb0 0x7f8a500bfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:34.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.642 7f8a54d82700 1 --2- 172.21.2.101:0/405294894 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a500c2290 0x7f8a500c2690 unknown :-1 s=CLOSED pgs=2843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:34.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.642 7f8a54d82700 1 --2- 172.21.2.101:0/405294894 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a340186c0 0x7f8a3401abc0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:34.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.642 7f8a54d82700 1 --2- 172.21.2.101:0/405294894 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a501a4a60 0x7f8a500c1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:34.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.642 7f8a54d82700 1 -- 172.21.2.101:0/405294894 >> 172.21.2.101:0/405294894 conn(0x7f8a500ebe30 msgr2=0x7f8a500bf200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:34.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.643 7f8a54d82700 1 -- 172.21.2.101:0/405294894 shutdown_connections 2021-02-10T11:45:34.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:34.643 7f8a54d82700 1 -- 172.21.2.101:0/405294894 wait complete. 2021-02-10T11:45:34.649 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-02-10T11:45:35.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.070 7fa6586db700 1 Processor -- start 2021-02-10T11:45:35.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.070 7fa6586db700 1 -- start start 2021-02-10T11:45:35.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.070 7fa6586db700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa6500d0340 0x7fa6500ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:35.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.070 7fa6586db700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa6500d1c70 0x7fa6500efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:35.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.070 7fa6586db700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6500f0b10 con 0x7fa6500d1870 2021-02-10T11:45:35.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.070 7fa6586db700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6500e5ed0 con 0x7fa6500d0340 2021-02-10T11:45:35.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.070 7fa6586db700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6501b0be0 con 0x7fa6500d1c70 2021-02-10T11:45:35.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.070 7fa6564f7700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa6500d1870 0x7fa6501b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:32974/0 (socket says 172.21.2.101:32974) 2021-02-10T11:45:35.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.070 7fa6564f7700 1 -- 172.21.2.101:0/1407402445 learned_addr learned my addr 172.21.2.101:0/1407402445 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:35.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.071 7fa656cf8700 1 --2- 172.21.2.101:0/1407402445 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa6500d1c70 0x7fa6500efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:35.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.071 7fa6554f5700 1 -- 172.21.2.101:0/1407402445 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996864164 0 0) 0x7fa64c0010b0 con 0x7fa6500d1870 2021-02-10T11:45:35.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.071 7fa6554f5700 1 -- 172.21.2.101:0/1407402445 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa63c002fe0 con 0x7fa6500d1870 2021-02-10T11:45:35.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.071 7fa655cf6700 1 --2- 172.21.2.101:0/1407402445 >> v2:172.21.2.108:3300/0 conn(0x7fa6500d0340 0x7fa6500ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:35.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.071 7fa6554f5700 1 -- 172.21.2.101:0/1407402445 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1355266989 0 0) 0x7fa64c0010b0 con 0x7fa6500d1870 2021-02-10T11:45:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.071 7fa6554f5700 1 -- 172.21.2.101:0/1407402445 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa6500d1c70 msgr2=0x7fa6500efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.071 7fa6554f5700 1 --2- 172.21.2.101:0/1407402445 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa6500d1c70 0x7fa6500efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.071 7fa6554f5700 1 -- 172.21.2.101:0/1407402445 >> v2:172.21.2.108:3300/0 conn(0x7fa6500d0340 msgr2=0x7fa6500ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.071 7fa6554f5700 1 --2- 172.21.2.101:0/1407402445 >> v2:172.21.2.108:3300/0 conn(0x7fa6500d0340 0x7fa6500ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.071 7fa6554f5700 1 -- 172.21.2.101:0/1407402445 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6500f20a0 con 0x7fa6500d1870 2021-02-10T11:45:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.071 7fa6554f5700 1 -- 172.21.2.101:0/1407402445 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fa64c0015b0 con 0x7fa6500d1870 2021-02-10T11:45:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.071 7fa6554f5700 1 -- 172.21.2.101:0/1407402445 >> v1:172.21.2.101:6789/0 conn(0x7fa6500d1870 legacy=0x7fa6501b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.071 7fa6554f5700 1 --2- 172.21.2.101:0/1407402445 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa63c003ae0 0x7fa63c006000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.071 7fa6554f5700 1 --2- 172.21.2.101:0/1407402445 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa63c006820 0x7fa63c008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.071 7fa6554f5700 1 --2- 172.21.2.101:0/1407402445 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa63c009640 0x7fa63c00bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.071 7fa6554f5700 1 -- 172.21.2.101:0/1407402445 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c050370 con 0x7fa63c006820 2021-02-10T11:45:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.071 7fa6554f5700 1 -- 172.21.2.101:0/1407402445 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c050590 con 0x7fa63c003ae0 2021-02-10T11:45:35.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.071 7fa6554f5700 1 -- 172.21.2.101:0/1407402445 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c0507b0 con 0x7fa63c009640 2021-02-10T11:45:35.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.071 7fa656cf8700 1 --2- 172.21.2.101:0/1407402445 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa6500d1c70 0x7fa6500efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:45:35.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.071 7fa655cf6700 1 --2- 172.21.2.101:0/1407402445 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa63c006820 0x7fa63c008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:35.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.071 7fa6564f7700 1 --2- 172.21.2.101:0/1407402445 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa63c003ae0 0x7fa63c006000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:35.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.071 7fa656cf8700 1 --2- 172.21.2.101:0/1407402445 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa63c009640 0x7fa63c00bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:35.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.072 7fa655cf6700 1 -- 172.21.2.101:0/1407402445 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa63c009640 msgr2=0x7fa63c00bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:35.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.072 7fa655cf6700 1 --2- 172.21.2.101:0/1407402445 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa63c009640 0x7fa63c00bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:35.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.072 7fa655cf6700 1 -- 172.21.2.101:0/1407402445 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa63c003ae0 msgr2=0x7fa63c006000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:35.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.072 7fa655cf6700 1 --2- 172.21.2.101:0/1407402445 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa63c003ae0 0x7fa63c006000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:35.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.072 7fa655cf6700 1 -- 172.21.2.101:0/1407402445 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c050a50 con 0x7fa63c006820 2021-02-10T11:45:35.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.072 7fa656cf8700 1 --2- 172.21.2.101:0/1407402445 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa63c009640 0x7fa63c00bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:35.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.072 7fa6564f7700 1 --2- 172.21.2.101:0/1407402445 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa63c003ae0 0x7fa63c006000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:45:35.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.072 7fa655cf6700 1 --2- 172.21.2.101:0/1407402445 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa63c006820 0x7fa63c008d40 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 rx=0x7fa64000c8f0 tx=0x7fa64000cc90).ready entity=mon.0 client_cookie=91fc020027e0f549 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:35.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.072 7fa6554f5700 1 -- 172.21.2.101:0/1407402445 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa64000cdd0 con 0x7fa63c006820 2021-02-10T11:45:35.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.072 7fa6554f5700 1 -- 172.21.2.101:0/1407402445 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa640009310 con 0x7fa63c006820 2021-02-10T11:45:35.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.072 7fa6554f5700 1 -- 172.21.2.101:0/1407402445 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa640009830 con 0x7fa63c006820 2021-02-10T11:45:35.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.072 7fa6586db700 1 -- 172.21.2.101:0/1407402445 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa63c006820 msgr2=0x7fa63c008d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:35.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.072 7fa6586db700 1 --2- 172.21.2.101:0/1407402445 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa63c006820 0x7fa63c008d40 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 rx=0x7fa64000c8f0 tx=0x7fa64000cc90).stop 2021-02-10T11:45:35.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.072 7fa656cf8700 1 -- 172.21.2.101:0/1407402445 reap_dead start 2021-02-10T11:45:35.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.072 7fa6586db700 1 -- 172.21.2.101:0/1407402445 shutdown_connections 2021-02-10T11:45:35.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.072 7fa6586db700 1 -- 172.21.2.101:0/1407402445 >> 172.21.2.101:0/1407402445 conn(0x7fa6500ce450 msgr2=0x7fa6500d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:35.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.073 7fa6586db700 1 -- 172.21.2.101:0/1407402445 shutdown_connections 2021-02-10T11:45:35.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.073 7fa6586db700 1 -- 172.21.2.101:0/1407402445 wait complete. 2021-02-10T11:45:35.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.087 7fa6586db700 1 Processor -- start 2021-02-10T11:45:35.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.087 7fa6586db700 1 -- start start 2021-02-10T11:45:35.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.088 7fa6586db700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa6500a6940 0x7fa6500a6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:35.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.088 7fa6586db700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa650092f20 0x7fa650093320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:35.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.088 7fa6586db700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa650094140 0x7fa650094540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:35.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.088 7fa6586db700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa650093940 con 0x7fa650092f20 2021-02-10T11:45:35.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.088 7fa6586db700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa650093bc0 con 0x7fa650094140 2021-02-10T11:45:35.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.088 7fa6586db700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500b11f0 con 0x7fa6500a6940 2021-02-10T11:45:35.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.088 7fa655cf6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa650092f20 0x7fa650093320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:35.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.088 7fa655cf6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa650092f20 0x7fa650093320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60728/0 (socket says 172.21.2.101:60728) 2021-02-10T11:45:35.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.088 7fa655cf6700 1 -- 172.21.2.101:0/3533655496 learned_addr learned my addr 172.21.2.101:0/3533655496 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:35.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.088 7fa655cf6700 1 -- 172.21.2.101:0/3533655496 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa6500a6940 msgr2=0x7fa6500a6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:35.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.088 7fa655cf6700 1 --2- 172.21.2.101:0/3533655496 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa6500a6940 0x7fa6500a6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:35.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.088 7fa655cf6700 1 -- 172.21.2.101:0/3533655496 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa650094140 msgr2=0x7fa650094540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:35.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.088 7fa655cf6700 1 --2- 172.21.2.101:0/3533655496 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa650094140 0x7fa650094540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:35.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.088 7fa655cf6700 1 -- 172.21.2.101:0/3533655496 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa650094b60 con 0x7fa650092f20 2021-02-10T11:45:35.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.088 7fa655cf6700 1 --2- 172.21.2.101:0/3533655496 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa650092f20 0x7fa650093320 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 rx=0x7fa640009e20 tx=0x7fa64000b300).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:35.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.088 7fa646ffd700 1 -- 172.21.2.101:0/3533655496 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa64000c7c0 con 0x7fa650092f20 2021-02-10T11:45:35.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.088 7fa646ffd700 1 -- 172.21.2.101:0/3533655496 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64000ca80 con 0x7fa650092f20 2021-02-10T11:45:35.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.088 7fa646ffd700 1 -- 172.21.2.101:0/3533655496 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa64000b6f0 con 0x7fa650092f20 2021-02-10T11:45:35.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.088 7fa6586db700 1 -- 172.21.2.101:0/3533655496 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa650094b60 con 0x7fa650092f20 2021-02-10T11:45:35.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.088 7fa6586db700 1 -- 172.21.2.101:0/3533655496 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa650094b60 con 0x7fa650092f20 2021-02-10T11:45:35.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.089 7fa646ffd700 1 -- 172.21.2.101:0/3533655496 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa640000c90 con 0x7fa650092f20 2021-02-10T11:45:35.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.089 7fa646ffd700 1 --2- 172.21.2.101:0/3533655496 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa6380186c0 0x7fa63801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:35.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.089 7fa646ffd700 1 -- 172.21.2.101:0/3533655496 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fa640032890 con 0x7fa650092f20 2021-02-10T11:45:35.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.089 7fa6564f7700 1 --2- 172.21.2.101:0/3533655496 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa6380186c0 0x7fa63801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:35.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.090 7fa6564f7700 1 --2- 172.21.2.101:0/3533655496 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa6380186c0 0x7fa63801abc0 crc :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:35.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.120 7fa6586db700 1 -- 172.21.2.101:0/3533655496 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa650093940 con 0x7fa650092f20 2021-02-10T11:45:35.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.123 7fa646ffd700 1 -- 172.21.2.101:0/3533655496 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa64000c7c0 con 0x7fa650092f20 2021-02-10T11:45:35.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.249 7fa6586db700 1 -- 172.21.2.101:0/3533655496 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa650093940 con 0x7fa650092f20 2021-02-10T11:45:35.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:35.999 7fa646ffd700 1 -- 172.21.2.101:0/3533655496 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v134) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa64000ca60 con 0x7fa650092f20 2021-02-10T11:45:36.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.020 7fa6586db700 1 -- 172.21.2.101:0/3533655496 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa6380186c0 msgr2=0x7fa63801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:36.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.020 7fa6586db700 1 --2- 172.21.2.101:0/3533655496 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa6380186c0 0x7fa63801abc0 crc :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:36.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.020 7fa6586db700 1 -- 172.21.2.101:0/3533655496 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa650092f20 msgr2=0x7fa650093320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:36.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.020 7fa6586db700 1 --2- 172.21.2.101:0/3533655496 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa650092f20 0x7fa650093320 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 rx=0x7fa640009e20 tx=0x7fa64000b300).stop 2021-02-10T11:45:36.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.020 7fa6586db700 1 -- 172.21.2.101:0/3533655496 shutdown_connections 2021-02-10T11:45:36.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.020 7fa6586db700 1 --2- 172.21.2.101:0/3533655496 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa650094140 0x7fa650094540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:36.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.020 7fa6586db700 1 --2- 172.21.2.101:0/3533655496 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa650092f20 0x7fa650093320 unknown :-1 s=CLOSED pgs=2846 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:36.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.020 7fa6586db700 1 --2- 172.21.2.101:0/3533655496 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa6380186c0 0x7fa63801abc0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:36.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.020 7fa6586db700 1 --2- 172.21.2.101:0/3533655496 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa6500a6940 0x7fa6500a6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:36.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.020 7fa6586db700 1 -- 172.21.2.101:0/3533655496 >> 172.21.2.101:0/3533655496 conn(0x7fa6500bc0a0 msgr2=0x7fa6500ebe70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:36.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.021 7fa6586db700 1 -- 172.21.2.101:0/3533655496 shutdown_connections 2021-02-10T11:45:36.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.021 7fa6586db700 1 -- 172.21.2.101:0/3533655496 wait complete. 2021-02-10T11:45:36.027 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-02-10T11:45:36.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.446 7fbe5f498700 1 Processor -- start 2021-02-10T11:45:36.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.446 7fbe5f498700 1 -- start start 2021-02-10T11:45:36.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.446 7fbe5f498700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbe580d1880 0x7fbe581b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:36.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.446 7fbe5f498700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe581b0af0 0x7fbe58108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:36.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.446 7fbe5f498700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe580fe1d0 con 0x7fbe580d1c80 2021-02-10T11:45:36.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.446 7fbe5f498700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe58109a70 con 0x7fbe580d1880 2021-02-10T11:45:36.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.446 7fbe5f498700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe58109760 con 0x7fbe581b0af0 2021-02-10T11:45:36.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.446 7fbe5d2b4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbe580d1880 0x7fbe581b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:36.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.446 7fbe5d2b4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbe580d1880 0x7fbe581b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60732/0 (socket says 172.21.2.101:60732) 2021-02-10T11:45:36.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.446 7fbe5d2b4700 1 -- 172.21.2.101:0/2362809835 learned_addr learned my addr 172.21.2.101:0/2362809835 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:36.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.447 7fbe5d2b4700 1 -- 172.21.2.101:0/2362809835 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe581b0af0 msgr2=0x7fbe58108950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:45:36.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.447 7fbe5d2b4700 1 --2- 172.21.2.101:0/2362809835 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe581b0af0 0x7fbe58108950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:36.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.447 7fbe5d2b4700 1 -- 172.21.2.101:0/2362809835 >> v1:172.21.2.108:6789/0 conn(0x7fbe580d1c80 legacy=0x7fbe58108f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:45:36.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.447 7fbe5d2b4700 1 -- 172.21.2.101:0/2362809835 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe58088070 con 0x7fbe580d1880 2021-02-10T11:45:36.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.447 7fbe5d2b4700 1 --2- 172.21.2.101:0/2362809835 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe580d1880 0x7fbe581b04d0 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 rx=0x7fbe54006700 tx=0x7fbe54009b00).ready entity=mon.0 client_cookie=faea5aea74481acc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:36.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.447 7fbe4ffff700 1 -- 172.21.2.101:0/2362809835 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbe5400c0c0 con 0x7fbe580d1880 2021-02-10T11:45:36.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.447 7fbe4ffff700 1 -- 172.21.2.101:0/2362809835 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5400c340 con 0x7fbe580d1880 2021-02-10T11:45:36.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.447 7fbe4ffff700 1 -- 172.21.2.101:0/2362809835 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbe5400c820 con 0x7fbe580d1880 2021-02-10T11:45:36.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.447 7fbe5f498700 1 -- 172.21.2.101:0/2362809835 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe580d1880 msgr2=0x7fbe581b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:36.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.447 7fbe5f498700 1 --2- 172.21.2.101:0/2362809835 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe580d1880 0x7fbe581b04d0 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 rx=0x7fbe54006700 tx=0x7fbe54009b00).stop 2021-02-10T11:45:36.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.448 7fbe5f498700 1 -- 172.21.2.101:0/2362809835 shutdown_connections 2021-02-10T11:45:36.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.448 7fbe5f498700 1 --2- 172.21.2.101:0/2362809835 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe581b0af0 0x7fbe58108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:36.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.448 7fbe5f498700 1 --2- 172.21.2.101:0/2362809835 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe580d1880 0x7fbe581b04d0 unknown :-1 s=CLOSED pgs=2847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:36.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.448 7fbe5f498700 1 -- 172.21.2.101:0/2362809835 >> 172.21.2.101:0/2362809835 conn(0x7fbe580ce430 msgr2=0x7fbe580d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:36.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.448 7fbe5f498700 1 -- 172.21.2.101:0/2362809835 shutdown_connections 2021-02-10T11:45:36.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.448 7fbe5f498700 1 -- 172.21.2.101:0/2362809835 wait complete. 2021-02-10T11:45:36.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.463 7fbe5f498700 1 Processor -- start 2021-02-10T11:45:36.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.463 7fbe5f498700 1 -- start start 2021-02-10T11:45:36.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.463 7fbe5f498700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe58187e80 0x7fbe58188280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:36.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.463 7fbe5f498700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbe581888a0 0x7fbe58189490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:36.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.463 7fbe5f498700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe58189ab0 0x7fbe58189eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:36.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.463 7fbe5f498700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe58188ca0 con 0x7fbe58187e80 2021-02-10T11:45:36.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.463 7fbe5f498700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe58188f10 con 0x7fbe581888a0 2021-02-10T11:45:36.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.463 7fbe5f498700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe58099520 con 0x7fbe58189ab0 2021-02-10T11:45:36.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.463 7fbe5d2b4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe58187e80 0x7fbe58188280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:36.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.463 7fbe5d2b4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe58187e80 0x7fbe58188280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60734/0 (socket says 172.21.2.101:60734) 2021-02-10T11:45:36.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.463 7fbe5d2b4700 1 -- 172.21.2.101:0/3421200422 learned_addr learned my addr 172.21.2.101:0/3421200422 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:36.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.463 7fbe5d2b4700 1 -- 172.21.2.101:0/3421200422 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe58189ab0 msgr2=0x7fbe58189eb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:36.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.463 7fbe5d2b4700 1 --2- 172.21.2.101:0/3421200422 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe58189ab0 0x7fbe58189eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:36.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.463 7fbe5d2b4700 1 -- 172.21.2.101:0/3421200422 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbe581888a0 msgr2=0x7fbe58189490 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:36.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.463 7fbe5d2b4700 1 --2- 172.21.2.101:0/3421200422 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbe581888a0 0x7fbe58189490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:36.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.463 7fbe5d2b4700 1 -- 172.21.2.101:0/3421200422 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe580999c0 con 0x7fbe58187e80 2021-02-10T11:45:36.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.464 7fbe5d2b4700 1 --2- 172.21.2.101:0/3421200422 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe58187e80 0x7fbe58188280 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 rx=0x7fbe54006f20 tx=0x7fbe5400b9b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:36.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.464 7fbe4dffb700 1 -- 172.21.2.101:0/3421200422 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbe54009100 con 0x7fbe58187e80 2021-02-10T11:45:36.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.464 7fbe4dffb700 1 -- 172.21.2.101:0/3421200422 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5400bda0 con 0x7fbe58187e80 2021-02-10T11:45:36.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.464 7fbe4dffb700 1 -- 172.21.2.101:0/3421200422 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbe540052e0 con 0x7fbe58187e80 2021-02-10T11:45:36.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.464 7fbe5f498700 1 -- 172.21.2.101:0/3421200422 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe580999c0 con 0x7fbe58187e80 2021-02-10T11:45:36.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.464 7fbe5f498700 1 -- 172.21.2.101:0/3421200422 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe580999c0 con 0x7fbe58187e80 2021-02-10T11:45:36.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.464 7fbe4dffb700 1 -- 172.21.2.101:0/3421200422 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbe54009410 con 0x7fbe58187e80 2021-02-10T11:45:36.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.464 7fbe4dffb700 1 --2- 172.21.2.101:0/3421200422 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe380186c0 0x7fbe3801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:36.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.464 7fbe4dffb700 1 -- 172.21.2.101:0/3421200422 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fbe54032810 con 0x7fbe58187e80 2021-02-10T11:45:36.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.465 7fbe5cab3700 1 --2- 172.21.2.101:0/3421200422 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe380186c0 0x7fbe3801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:36.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.465 7fbe5cab3700 1 --2- 172.21.2.101:0/3421200422 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe380186c0 0x7fbe3801abc0 crc :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:36.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.496 7fbe5f498700 1 -- 172.21.2.101:0/3421200422 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe58195bd0 con 0x7fbe58187e80 2021-02-10T11:45:36.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.499 7fbe4dffb700 1 -- 172.21.2.101:0/3421200422 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbe54006470 con 0x7fbe58187e80 2021-02-10T11:45:36.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:36.625 7fbe5f498700 1 -- 172.21.2.101:0/3421200422 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbe58195bd0 con 0x7fbe58187e80 2021-02-10T11:45:37.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.002 7fbe4dffb700 1 -- 172.21.2.101:0/3421200422 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v158) v1 ==== 185+0+0 (secure 0 0 0) 0x7fbe5401f500 con 0x7fbe58187e80 2021-02-10T11:45:37.041 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' removed 2021-02-10T11:45:37.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.044 7fbe5f498700 1 -- 172.21.2.101:0/3421200422 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe380186c0 msgr2=0x7fbe3801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:37.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.044 7fbe5f498700 1 --2- 172.21.2.101:0/3421200422 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe380186c0 0x7fbe3801abc0 crc :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:37.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.044 7fbe5f498700 1 -- 172.21.2.101:0/3421200422 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe58187e80 msgr2=0x7fbe58188280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:37.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.044 7fbe5f498700 1 --2- 172.21.2.101:0/3421200422 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe58187e80 0x7fbe58188280 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 rx=0x7fbe54006f20 tx=0x7fbe5400b9b0).stop 2021-02-10T11:45:37.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.044 7fbe5f498700 1 -- 172.21.2.101:0/3421200422 shutdown_connections 2021-02-10T11:45:37.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.044 7fbe5f498700 1 --2- 172.21.2.101:0/3421200422 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbe380186c0 0x7fbe3801abc0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:37.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.044 7fbe5f498700 1 --2- 172.21.2.101:0/3421200422 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbe58189ab0 0x7fbe58189eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:37.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.044 7fbe5f498700 1 --2- 172.21.2.101:0/3421200422 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbe581888a0 0x7fbe58189490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:37.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.044 7fbe5f498700 1 --2- 172.21.2.101:0/3421200422 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbe58187e80 0x7fbe58188280 unknown :-1 s=CLOSED pgs=2848 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:37.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.044 7fbe5f498700 1 -- 172.21.2.101:0/3421200422 >> 172.21.2.101:0/3421200422 conn(0x7fbe5819f940 msgr2=0x7fbe5818c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:37.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.045 7fbe5f498700 1 -- 172.21.2.101:0/3421200422 shutdown_connections 2021-02-10T11:45:37.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.045 7fbe5f498700 1 -- 172.21.2.101:0/3421200422 wait complete. 2021-02-10T11:45:37.051 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-02-10T11:45:37.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.468 7f2c1d0c6700 1 Processor -- start 2021-02-10T11:45:37.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.468 7f2c1d0c6700 1 -- start start 2021-02-10T11:45:37.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.469 7f2c1d0c6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c180d1880 0x7f2c181b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:37.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.469 7f2c1d0c6700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c18109a70 con 0x7f2c181b0af0 2021-02-10T11:45:37.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.469 7f2c1d0c6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c18088070 con 0x7f2c180cff80 2021-02-10T11:45:37.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.469 7f2c1d0c6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c180e5f00 con 0x7f2c180d1880 2021-02-10T11:45:37.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.469 7f2c1659c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f2c181b0af0 0x7f2c18108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33004/0 (socket says 172.21.2.101:33004) 2021-02-10T11:45:37.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.469 7f2c16d9d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c180d1880 0x7f2c181b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:37.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.469 7f2c1659c700 1 -- 172.21.2.101:0/1889911128 learned_addr learned my addr 172.21.2.101:0/1889911128 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:37.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.469 7f2c15d9b700 1 -- 172.21.2.101:0/1889911128 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3243488390 0 0) 0x7f2c08000ce0 con 0x7f2c181b0af0 2021-02-10T11:45:37.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.469 7f2c16d9d700 1 -- 172.21.2.101:0/1889911128 >> v1:172.21.2.108:6789/0 conn(0x7f2c180cff80 legacy=0x7f2c18109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:37.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.469 7f2c16d9d700 1 -- 172.21.2.101:0/1889911128 >> v1:172.21.2.101:6789/0 conn(0x7f2c181b0af0 legacy=0x7f2c18108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:37.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.469 7f2c16d9d700 1 -- 172.21.2.101:0/1889911128 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c180d1c80 con 0x7f2c180d1880 2021-02-10T11:45:37.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.470 7f2c16d9d700 1 --2- 172.21.2.101:0/1889911128 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c180d1880 0x7f2c181b04d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f2c00006650 tx=0x7f2c00009a80).ready entity=mon.2 client_cookie=24f2b66ae181e37b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:37.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.470 7f2c15d9b700 1 -- 172.21.2.101:0/1889911128 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2c0000c0c0 con 0x7f2c180d1880 2021-02-10T11:45:37.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.470 7f2c15d9b700 1 -- 172.21.2.101:0/1889911128 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c0000c340 con 0x7f2c180d1880 2021-02-10T11:45:37.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.470 7f2c15d9b700 1 -- 172.21.2.101:0/1889911128 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2c0000c860 con 0x7f2c180d1880 2021-02-10T11:45:37.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.470 7f2c1d0c6700 1 -- 172.21.2.101:0/1889911128 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c180d1880 msgr2=0x7f2c181b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:37.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.470 7f2c1d0c6700 1 --2- 172.21.2.101:0/1889911128 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c180d1880 0x7f2c181b04d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f2c00006650 tx=0x7f2c00009a80).stop 2021-02-10T11:45:37.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.471 7f2c1d0c6700 1 -- 172.21.2.101:0/1889911128 shutdown_connections 2021-02-10T11:45:37.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.471 7f2c1d0c6700 1 --2- 172.21.2.101:0/1889911128 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c180d1880 0x7f2c181b04d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:37.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.471 7f2c1d0c6700 1 -- 172.21.2.101:0/1889911128 >> 172.21.2.101:0/1889911128 conn(0x7f2c180ce430 msgr2=0x7f2c180d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:37.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.471 7f2c1d0c6700 1 -- 172.21.2.101:0/1889911128 shutdown_connections 2021-02-10T11:45:37.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.471 7f2c1d0c6700 1 -- 172.21.2.101:0/1889911128 wait complete. 2021-02-10T11:45:37.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.485 7f2c1d0c6700 1 Processor -- start 2021-02-10T11:45:37.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.485 7f2c1d0c6700 1 -- start start 2021-02-10T11:45:37.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.486 7f2c1d0c6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c1809ef20 0x7f2c18187e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:37.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.486 7f2c1d0c6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c181884a0 0x7f2c181888a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:37.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.486 7f2c1d0c6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2c18189840 0x7f2c18189c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.486 7f2c1d0c6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c181b3c10 con 0x7f2c1809ef20 2021-02-10T11:45:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.486 7f2c1d0c6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c181964f0 con 0x7f2c18189840 2021-02-10T11:45:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.486 7f2c1d0c6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c18188ec0 con 0x7f2c181884a0 2021-02-10T11:45:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.486 7f2c16d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c1809ef20 0x7f2c18187e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.486 7f2c16d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c1809ef20 0x7f2c18187e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60746/0 (socket says 172.21.2.101:60746) 2021-02-10T11:45:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.486 7f2c16d9d700 1 -- 172.21.2.101:0/2703223970 learned_addr learned my addr 172.21.2.101:0/2703223970 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.487 7f2c1659c700 1 --2- 172.21.2.101:0/2703223970 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c181884a0 0x7f2c181888a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.487 7f2c16d9d700 1 -- 172.21.2.101:0/2703223970 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c181884a0 msgr2=0x7f2c181888a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.487 7f2c16d9d700 1 --2- 172.21.2.101:0/2703223970 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c181884a0 0x7f2c181888a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.487 7f2c16d9d700 1 -- 172.21.2.101:0/2703223970 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2c18189840 msgr2=0x7f2c18189c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.487 7f2c16d9d700 1 --2- 172.21.2.101:0/2703223970 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2c18189840 0x7f2c18189c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.487 7f2c16d9d700 1 -- 172.21.2.101:0/2703223970 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1809f320 con 0x7f2c1809ef20 2021-02-10T11:45:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.487 7f2c1659c700 1 --2- 172.21.2.101:0/2703223970 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c181884a0 0x7f2c181888a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:37.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.487 7f2c16d9d700 1 --2- 172.21.2.101:0/2703223970 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c1809ef20 0x7f2c18187e80 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 rx=0x7f2c00009650 tx=0x7f2c0000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.487 7f2c0f7fe700 1 -- 172.21.2.101:0/2703223970 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2c000093a0 con 0x7f2c1809ef20 2021-02-10T11:45:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.487 7f2c0f7fe700 1 -- 172.21.2.101:0/2703223970 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c0000bde0 con 0x7f2c1809ef20 2021-02-10T11:45:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.487 7f2c0f7fe700 1 -- 172.21.2.101:0/2703223970 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2c000052e0 con 0x7f2c1809ef20 2021-02-10T11:45:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.487 7f2c1d0c6700 1 -- 172.21.2.101:0/2703223970 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c1809f320 con 0x7f2c1809ef20 2021-02-10T11:45:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.487 7f2c1d0c6700 1 -- 172.21.2.101:0/2703223970 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c1809f320 con 0x7f2c1809ef20 2021-02-10T11:45:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.488 7f2c0f7fe700 1 -- 172.21.2.101:0/2703223970 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2c00006030 con 0x7f2c1809ef20 2021-02-10T11:45:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.488 7f2c0f7fe700 1 --2- 172.21.2.101:0/2703223970 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2bfc018700 0x7f2bfc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.488 7f2c0f7fe700 1 -- 172.21.2.101:0/2703223970 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f2c00032680 con 0x7f2c1809ef20 2021-02-10T11:45:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.488 7f2c1659c700 1 --2- 172.21.2.101:0/2703223970 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2bfc018700 0x7f2bfc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:37.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.488 7f2c1659c700 1 --2- 172.21.2.101:0/2703223970 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2bfc018700 0x7f2bfc01abc0 crc :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:37.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.519 7f2c1d0c6700 1 -- 172.21.2.101:0/2703223970 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c1818abb0 con 0x7f2c1809ef20 2021-02-10T11:45:37.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.522 7f2c0f7fe700 1 -- 172.21.2.101:0/2703223970 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2c00000940 con 0x7f2c1809ef20 2021-02-10T11:45:37.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:37.647 7f2c1d0c6700 1 -- 172.21.2.101:0/2703223970 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2c1818abb0 con 0x7f2c1809ef20 2021-02-10T11:45:37.720 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:45:37.723 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:45:38.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.008 7f2c0f7fe700 1 -- 172.21.2.101:0/2703223970 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v159) v1 ==== 173+0+0 (secure 0 0 0) 0x7f2c0001f520 con 0x7f2c1809ef20 2021-02-10T11:45:38.012 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' removed 2021-02-10T11:45:38.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.016 7f2c1d0c6700 1 -- 172.21.2.101:0/2703223970 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2bfc018700 msgr2=0x7f2bfc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:38.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.016 7f2c1d0c6700 1 --2- 172.21.2.101:0/2703223970 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2bfc018700 0x7f2bfc01abc0 crc :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:38.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.016 7f2c1d0c6700 1 -- 172.21.2.101:0/2703223970 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c1809ef20 msgr2=0x7f2c18187e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:38.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.016 7f2c1d0c6700 1 --2- 172.21.2.101:0/2703223970 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c1809ef20 0x7f2c18187e80 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 rx=0x7f2c00009650 tx=0x7f2c0000b960).stop 2021-02-10T11:45:38.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.016 7f2c1d0c6700 1 -- 172.21.2.101:0/2703223970 shutdown_connections 2021-02-10T11:45:38.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.016 7f2c1d0c6700 1 --2- 172.21.2.101:0/2703223970 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2bfc018700 0x7f2bfc01abc0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:38.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.016 7f2c1d0c6700 1 --2- 172.21.2.101:0/2703223970 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2c181884a0 0x7f2c181888a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:38.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.016 7f2c1d0c6700 1 --2- 172.21.2.101:0/2703223970 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2c18189840 0x7f2c18189c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:38.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.016 7f2c1d0c6700 1 --2- 172.21.2.101:0/2703223970 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2c1809ef20 0x7f2c18187e80 unknown :-1 s=CLOSED pgs=2850 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:38.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.016 7f2c1d0c6700 1 -- 172.21.2.101:0/2703223970 >> 172.21.2.101:0/2703223970 conn(0x7f2c1819f970 msgr2=0x7f2c181a61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:38.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.016 7f2c1d0c6700 1 -- 172.21.2.101:0/2703223970 shutdown_connections 2021-02-10T11:45:38.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.016 7f2c1d0c6700 1 -- 172.21.2.101:0/2703223970 wait complete. 2021-02-10T11:45:38.023 INFO:tasks.ceph.mds.a:Restarting daemon 2021-02-10T11:45:38.023 INFO:tasks.ceph.mds.a:Stopping old one... 2021-02-10T11:45:38.023 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-02-10T11:45:38.023 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:45:38.023 INFO:tasks.ceph.mds.b:Restarting daemon 2021-02-10T11:45:38.023 INFO:tasks.ceph.mds.b:Stopping old one... 2021-02-10T11:45:38.024 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-02-10T11:45:38.024 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:45:38.029 INFO:tasks.ceph.mds.a:Stopped 2021-02-10T11:45:38.030 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-02-10T11:45:38.030 INFO:tasks.ceph.mds.b:Stopped 2021-02-10T11:45:38.031 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-02-10T11:45:38.072 INFO:tasks.ceph.mds.a:Started 2021-02-10T11:45:38.072 DEBUG:teuthology.parallel:result is None 2021-02-10T11:45:38.073 INFO:tasks.ceph.mds.b:Started 2021-02-10T11:45:38.073 DEBUG:teuthology.parallel:result is None 2021-02-10T11:45:38.073 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd blacklist clear 2021-02-10T11:45:38.478 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:45:38.484 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:45:38.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.492 7fc3ba0e4700 1 Processor -- start 2021-02-10T11:45:38.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.492 7fc3ba0e4700 1 -- start start 2021-02-10T11:45:38.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.493 7fc3ba0e4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3b40d0340 0x7fc3b40ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:38.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.493 7fc3ba0e4700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc3b40d1c70 0x7fc3b40efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:38.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.493 7fc3ba0e4700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b40f0b10 con 0x7fc3b40d1870 2021-02-10T11:45:38.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.493 7fc3ba0e4700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b40e5ed0 con 0x7fc3b40d1c70 2021-02-10T11:45:38.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.493 7fc3ba0e4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40d2fe0 con 0x7fc3b40d0340 2021-02-10T11:45:38.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.493 7fc3b2ffd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3b40d0340 0x7fc3b40ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:38.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.493 7fc3b37fe700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fc3b40d1870 0x7fc3b41b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:54668/0 (socket says 172.21.2.101:54668) 2021-02-10T11:45:38.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.493 7fc3b37fe700 1 -- 172.21.2.101:0/2261756576 learned_addr learned my addr 172.21.2.101:0/2261756576 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:38.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.493 7fc3b3fff700 1 --2- 172.21.2.101:0/2261756576 >> v2:172.21.2.108:3300/0 conn(0x7fc3b40d1c70 0x7fc3b40efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:38.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.494 7fc3b3fff700 1 -- 172.21.2.101:0/2261756576 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3b40d0340 msgr2=0x7fc3b40ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:38.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.494 7fc3b3fff700 1 --2- 172.21.2.101:0/2261756576 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3b40d0340 0x7fc3b40ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:38.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.494 7fc3b3fff700 1 -- 172.21.2.101:0/2261756576 >> v1:172.21.2.108:6789/0 conn(0x7fc3b40d1870 legacy=0x7fc3b41b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:38.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.494 7fc3b3fff700 1 -- 172.21.2.101:0/2261756576 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b41b0be0 con 0x7fc3b40d1c70 2021-02-10T11:45:38.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.494 7fc3b3fff700 1 --2- 172.21.2.101:0/2261756576 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3b40d1c70 0x7fc3b40efc60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7fc3ac006e50 tx=0x7fc3ac009e70).ready entity=mon.1 client_cookie=64a2e18bf08185f1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:38.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.494 7fc3b27fc700 1 -- 172.21.2.101:0/2261756576 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc3ac006a20 con 0x7fc3b40d1c70 2021-02-10T11:45:38.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.494 7fc3b27fc700 1 -- 172.21.2.101:0/2261756576 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ac006790 con 0x7fc3b40d1c70 2021-02-10T11:45:38.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.494 7fc3b27fc700 1 -- 172.21.2.101:0/2261756576 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc3ac00b700 con 0x7fc3b40d1c70 2021-02-10T11:45:38.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.494 7fc3ba0e4700 1 -- 172.21.2.101:0/2261756576 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3b40d1c70 msgr2=0x7fc3b40efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:38.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.494 7fc3ba0e4700 1 --2- 172.21.2.101:0/2261756576 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3b40d1c70 0x7fc3b40efc60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7fc3ac006e50 tx=0x7fc3ac009e70).stop 2021-02-10T11:45:38.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.495 7fc3ba0e4700 1 -- 172.21.2.101:0/2261756576 shutdown_connections 2021-02-10T11:45:38.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.495 7fc3ba0e4700 1 --2- 172.21.2.101:0/2261756576 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3b40d0340 0x7fc3b40ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:38.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.495 7fc3ba0e4700 1 --2- 172.21.2.101:0/2261756576 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3b40d1c70 0x7fc3b40efc60 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:38.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.495 7fc3ba0e4700 1 -- 172.21.2.101:0/2261756576 >> 172.21.2.101:0/2261756576 conn(0x7fc3b40ce450 msgr2=0x7fc3b40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:38.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.495 7fc3ba0e4700 1 -- 172.21.2.101:0/2261756576 shutdown_connections 2021-02-10T11:45:38.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.495 7fc3ba0e4700 1 -- 172.21.2.101:0/2261756576 wait complete. 2021-02-10T11:45:38.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.510 7fc3ba0e4700 1 Processor -- start 2021-02-10T11:45:38.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.510 7fc3ba0e4700 1 -- start start 2021-02-10T11:45:38.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc3ba0e4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3b419ffa0 0x7fc3b40eda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:38.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc3ba0e4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3b41a03a0 0x7fc3b40ee030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:38.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc3ba0e4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3b40ee650 0x7fc3b40ebe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:38.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc3ba0e4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40ed490 con 0x7fc3b40ee650 2021-02-10T11:45:38.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc3ba0e4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40eea50 con 0x7fc3b419ffa0 2021-02-10T11:45:38.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc3ba0e4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40eec30 con 0x7fc3b41a03a0 2021-02-10T11:45:38.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc3b3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3b40ee650 0x7fc3b40ebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:38.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc3b3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3b40ee650 0x7fc3b40ebe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60774/0 (socket says 172.21.2.101:60774) 2021-02-10T11:45:38.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc3b3fff700 1 -- 172.21.2.101:0/3917019877 learned_addr learned my addr 172.21.2.101:0/3917019877 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:38.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc3b2ffd700 1 --2- 172.21.2.101:0/3917019877 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3b41a03a0 0x7fc3b40ee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:38.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc3b3fff700 1 -- 172.21.2.101:0/3917019877 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3b41a03a0 msgr2=0x7fc3b40ee030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:38.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc3b3fff700 1 --2- 172.21.2.101:0/3917019877 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3b41a03a0 0x7fc3b40ee030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:38.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc3b37fe700 1 --2- 172.21.2.101:0/3917019877 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3b419ffa0 0x7fc3b40eda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:38.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc3b3fff700 1 -- 172.21.2.101:0/3917019877 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3b419ffa0 msgr2=0x7fc3b40eda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:38.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc3b3fff700 1 --2- 172.21.2.101:0/3917019877 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3b419ffa0 0x7fc3b40eda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:38.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc3b3fff700 1 -- 172.21.2.101:0/3917019877 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b40ec4a0 con 0x7fc3b40ee650 2021-02-10T11:45:38.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc3b37fe700 1 --2- 172.21.2.101:0/3917019877 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3b419ffa0 0x7fc3b40eda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:38.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc3b3fff700 1 --2- 172.21.2.101:0/3917019877 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3b40ee650 0x7fc3b40ebe80 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 rx=0x7fc3ac006e60 tx=0x7fc3ac00bc70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:38.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc39bfff700 1 -- 172.21.2.101:0/3917019877 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc3ac009130 con 0x7fc3b40ee650 2021-02-10T11:45:38.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc39bfff700 1 -- 172.21.2.101:0/3917019877 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ac00be40 con 0x7fc3b40ee650 2021-02-10T11:45:38.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc39bfff700 1 -- 172.21.2.101:0/3917019877 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc3ac001390 con 0x7fc3b40ee650 2021-02-10T11:45:38.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc3ba0e4700 1 -- 172.21.2.101:0/3917019877 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b40ec4a0 con 0x7fc3b40ee650 2021-02-10T11:45:38.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.511 7fc3ba0e4700 1 -- 172.21.2.101:0/3917019877 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b40ec4a0 con 0x7fc3b40ee650 2021-02-10T11:45:38.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.512 7fc39bfff700 1 -- 172.21.2.101:0/3917019877 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc3ac0056b0 con 0x7fc3b40ee650 2021-02-10T11:45:38.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.512 7fc39bfff700 1 --2- 172.21.2.101:0/3917019877 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc394018730 0x7fc39401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:38.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.512 7fc39bfff700 1 -- 172.21.2.101:0/3917019877 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4839+0+0 (secure 0 0 0) 0x7fc3ac032860 con 0x7fc3b40ee650 2021-02-10T11:45:38.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.512 7fc3b37fe700 1 --2- 172.21.2.101:0/3917019877 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc394018730 0x7fc39401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:38.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.512 7fc3b37fe700 1 --2- 172.21.2.101:0/3917019877 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc394018730 0x7fc39401abc0 crc :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:38.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.541 7fc3ba0e4700 1 -- 172.21.2.101:0/3917019877 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b40ebc50 con 0x7fc3b40ee650 2021-02-10T11:45:38.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.543 7fc39bfff700 1 -- 172.21.2.101:0/3917019877 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc3ac000df0 con 0x7fc3b40ee650 2021-02-10T11:45:38.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:38.682 7fc3ba0e4700 1 -- 172.21.2.101:0/3917019877 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fc3b40ebc50 con 0x7fc3b40ee650 2021-02-10T11:45:39.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.013 7fc39bfff700 1 -- 172.21.2.101:0/3917019877 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v160) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc3ac00bdb0 con 0x7fc3b40ee650 2021-02-10T11:45:39.048 INFO:teuthology.orchestra.run.gibba001.stderr: removed all blacklist entries 2021-02-10T11:45:39.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.051 7fc3ba0e4700 1 -- 172.21.2.101:0/3917019877 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc394018730 msgr2=0x7fc39401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:39.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.051 7fc3ba0e4700 1 --2- 172.21.2.101:0/3917019877 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc394018730 0x7fc39401abc0 crc :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:39.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.051 7fc3ba0e4700 1 -- 172.21.2.101:0/3917019877 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3b40ee650 msgr2=0x7fc3b40ebe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:39.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.051 7fc3ba0e4700 1 --2- 172.21.2.101:0/3917019877 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3b40ee650 0x7fc3b40ebe80 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 rx=0x7fc3ac006e60 tx=0x7fc3ac00bc70).stop 2021-02-10T11:45:39.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.051 7fc3ba0e4700 1 -- 172.21.2.101:0/3917019877 shutdown_connections 2021-02-10T11:45:39.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.051 7fc3ba0e4700 1 --2- 172.21.2.101:0/3917019877 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc394018730 0x7fc39401abc0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:39.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.051 7fc3ba0e4700 1 --2- 172.21.2.101:0/3917019877 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc3b41a03a0 0x7fc3b40ee030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:39.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.051 7fc3ba0e4700 1 --2- 172.21.2.101:0/3917019877 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc3b40ee650 0x7fc3b40ebe80 unknown :-1 s=CLOSED pgs=2853 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:39.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.051 7fc3ba0e4700 1 --2- 172.21.2.101:0/3917019877 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc3b419ffa0 0x7fc3b40eda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:39.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.052 7fc3ba0e4700 1 -- 172.21.2.101:0/3917019877 >> 172.21.2.101:0/3917019877 conn(0x7fc3b4094d90 msgr2=0x7fc3b408a420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:39.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.052 7fc3ba0e4700 1 -- 172.21.2.101:0/3917019877 shutdown_connections 2021-02-10T11:45:39.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.052 7fc3ba0e4700 1 -- 172.21.2.101:0/3917019877 wait complete. 2021-02-10T11:45:39.058 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2021-02-10T11:45:39.058 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:45:39.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.474 7f38ba1ba700 1 Processor -- start 2021-02-10T11:45:39.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.474 7f38ba1ba700 1 -- start start 2021-02-10T11:45:39.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.475 7f38ba1ba700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38b41b0870 0x7f38b40f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:39.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.475 7f38ba1ba700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b41b0c70 con 0x7f38b40efe00 2021-02-10T11:45:39.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.475 7f38ba1ba700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b40f0b50 con 0x7f38b41b0470 2021-02-10T11:45:39.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.475 7f38ba1ba700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b40e5ea0 con 0x7f38b41b0870 2021-02-10T11:45:39.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.475 7f38b3fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f38b40efe00 0x7f38b40f0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33044/0 (socket says 172.21.2.101:33044) 2021-02-10T11:45:39.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.475 7f38b3fff700 1 -- 172.21.2.101:0/434597231 learned_addr learned my addr 172.21.2.101:0/434597231 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:39.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.475 7f38b2ffd700 1 --2- 172.21.2.101:0/434597231 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38b41b0870 0x7f38b40f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:39.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.475 7f38b27fc700 1 -- 172.21.2.101:0/434597231 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2906720551 0 0) 0x7f38ac001500 con 0x7f38b40efe00 2021-02-10T11:45:39.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.475 7f38b27fc700 1 -- 172.21.2.101:0/434597231 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f389c002fe0 con 0x7f38b40efe00 2021-02-10T11:45:39.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b27fc700 1 -- 172.21.2.101:0/434597231 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82661036 0 0) 0x7f38a4000c60 con 0x7f38b41b0470 2021-02-10T11:45:39.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b27fc700 1 -- 172.21.2.101:0/434597231 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f389c003210 con 0x7f38b41b0470 2021-02-10T11:45:39.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b27fc700 1 -- 172.21.2.101:0/434597231 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2297637350 0 0) 0x7f38ac001270 con 0x7f38b40efe00 2021-02-10T11:45:39.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b27fc700 1 -- 172.21.2.101:0/434597231 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38b41b0870 msgr2=0x7f38b40f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:39.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b27fc700 1 --2- 172.21.2.101:0/434597231 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38b41b0870 0x7f38b40f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:39.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b27fc700 1 -- 172.21.2.101:0/434597231 >> v1:172.21.2.108:6789/0 conn(0x7f38b41b0470 legacy=0x7f38b40ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:39.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b27fc700 1 -- 172.21.2.101:0/434597231 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b40e52e0 con 0x7f38b40efe00 2021-02-10T11:45:39.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b27fc700 1 -- 172.21.2.101:0/434597231 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f38ac001b50 con 0x7f38b40efe00 2021-02-10T11:45:39.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b2ffd700 1 --2- 172.21.2.101:0/434597231 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38b41b0870 0x7f38b40f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:45:39.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b27fc700 1 -- 172.21.2.101:0/434597231 >> v1:172.21.2.101:6789/0 conn(0x7f38b40efe00 legacy=0x7f38b40f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:39.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b27fc700 1 --2- 172.21.2.101:0/434597231 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f389c003b20 0x7f389c0065b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:39.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b27fc700 1 --2- 172.21.2.101:0/434597231 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f389c0040d0 0x7f389c008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:39.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b27fc700 1 --2- 172.21.2.101:0/434597231 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f389c009520 0x7f389c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:39.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b27fc700 1 -- 172.21.2.101:0/434597231 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c050300 con 0x7f389c009520 2021-02-10T11:45:39.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b27fc700 1 -- 172.21.2.101:0/434597231 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c0504e0 con 0x7f389c0040d0 2021-02-10T11:45:39.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b27fc700 1 -- 172.21.2.101:0/434597231 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389c050700 con 0x7f389c003b20 2021-02-10T11:45:39.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b37fe700 1 --2- 172.21.2.101:0/434597231 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f389c003b20 0x7f389c0065b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:39.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b3fff700 1 --2- 172.21.2.101:0/434597231 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f389c009520 0x7f389c00ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:39.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b2ffd700 1 --2- 172.21.2.101:0/434597231 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f389c0040d0 0x7f389c008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:39.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b3fff700 1 -- 172.21.2.101:0/434597231 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f389c003b20 msgr2=0x7f389c0065b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:39.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b3fff700 1 --2- 172.21.2.101:0/434597231 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f389c003b20 0x7f389c0065b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:39.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b3fff700 1 -- 172.21.2.101:0/434597231 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f389c0040d0 msgr2=0x7f389c008c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:39.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b3fff700 1 --2- 172.21.2.101:0/434597231 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f389c0040d0 0x7f389c008c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:39.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b3fff700 1 -- 172.21.2.101:0/434597231 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389c0509a0 con 0x7f389c009520 2021-02-10T11:45:39.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b37fe700 1 --2- 172.21.2.101:0/434597231 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f389c003b20 0x7f389c0065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:45:39.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b2ffd700 1 --2- 172.21.2.101:0/434597231 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f389c0040d0 0x7f389c008c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:45:39.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.476 7f38b3fff700 1 --2- 172.21.2.101:0/434597231 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f389c009520 0x7f389c00ba40 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 rx=0x7f38ac007e30 tx=0x7f38ac00a310).ready entity=mon.0 client_cookie=c95cd5cf364219df server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:39.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.477 7f38b27fc700 1 -- 172.21.2.101:0/434597231 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f38ac007a00 con 0x7f389c009520 2021-02-10T11:45:39.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.477 7f38b27fc700 1 -- 172.21.2.101:0/434597231 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ac0070c0 con 0x7f389c009520 2021-02-10T11:45:39.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.477 7f38b27fc700 1 -- 172.21.2.101:0/434597231 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f38ac004720 con 0x7f389c009520 2021-02-10T11:45:39.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.477 7f38ba1ba700 1 -- 172.21.2.101:0/434597231 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f389c009520 msgr2=0x7f389c00ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:39.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.477 7f38ba1ba700 1 --2- 172.21.2.101:0/434597231 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f389c009520 0x7f389c00ba40 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 rx=0x7f38ac007e30 tx=0x7f38ac00a310).stop 2021-02-10T11:45:39.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.477 7f38b3fff700 1 -- 172.21.2.101:0/434597231 reap_dead start 2021-02-10T11:45:39.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.477 7f38ba1ba700 1 -- 172.21.2.101:0/434597231 shutdown_connections 2021-02-10T11:45:39.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.477 7f38ba1ba700 1 -- 172.21.2.101:0/434597231 >> 172.21.2.101:0/434597231 conn(0x7f38b40ce3e0 msgr2=0x7f38b40e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:39.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.477 7f38ba1ba700 1 -- 172.21.2.101:0/434597231 shutdown_connections 2021-02-10T11:45:39.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.477 7f38ba1ba700 1 -- 172.21.2.101:0/434597231 wait complete. 2021-02-10T11:45:39.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.492 7f38ba1ba700 1 Processor -- start 2021-02-10T11:45:39.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.492 7f38ba1ba700 1 -- start start 2021-02-10T11:45:39.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.493 7f38ba1ba700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38b40bba80 0x7f38b4108e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:39.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.493 7f38ba1ba700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38b40bbe80 0x7f38b40bd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:39.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.493 7f38ba1ba700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38b41094b0 0x7f38b40bdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:39.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.493 7f38ba1ba700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b41089f0 con 0x7f38b40bba80 2021-02-10T11:45:39.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.493 7f38ba1ba700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b40c2ec0 con 0x7f38b41094b0 2021-02-10T11:45:39.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.493 7f38ba1ba700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b40c30e0 con 0x7f38b40bbe80 2021-02-10T11:45:39.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.493 7f38b37fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38b40bba80 0x7f38b4108e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:39.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.493 7f38b37fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38b40bba80 0x7f38b4108e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60790/0 (socket says 172.21.2.101:60790) 2021-02-10T11:45:39.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.493 7f38b37fe700 1 -- 172.21.2.101:0/2853347276 learned_addr learned my addr 172.21.2.101:0/2853347276 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:39.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.493 7f38b2ffd700 1 --2- 172.21.2.101:0/2853347276 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38b40bbe80 0x7f38b40bd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:39.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.493 7f38b3fff700 1 --2- 172.21.2.101:0/2853347276 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38b41094b0 0x7f38b40bdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:39.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.493 7f38b37fe700 1 -- 172.21.2.101:0/2853347276 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38b40bbe80 msgr2=0x7f38b40bd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:39.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.493 7f38b37fe700 1 --2- 172.21.2.101:0/2853347276 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38b40bbe80 0x7f38b40bd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:39.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.493 7f38b37fe700 1 -- 172.21.2.101:0/2853347276 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38b41094b0 msgr2=0x7f38b40bdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:39.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.493 7f38b37fe700 1 --2- 172.21.2.101:0/2853347276 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38b41094b0 0x7f38b40bdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:39.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.493 7f38b37fe700 1 -- 172.21.2.101:0/2853347276 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b40be200 con 0x7f38b40bba80 2021-02-10T11:45:39.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.493 7f38b2ffd700 1 --2- 172.21.2.101:0/2853347276 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38b40bbe80 0x7f38b40bd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:45:39.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.493 7f38b3fff700 1 --2- 172.21.2.101:0/2853347276 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38b41094b0 0x7f38b40bdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:45:39.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.494 7f38b37fe700 1 --2- 172.21.2.101:0/2853347276 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38b40bba80 0x7f38b4108e90 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 rx=0x7f38a4006b40 tx=0x7f38a400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:39.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.494 7f389bfff700 1 -- 172.21.2.101:0/2853347276 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f38a400bd80 con 0x7f38b40bba80 2021-02-10T11:45:39.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.494 7f389bfff700 1 -- 172.21.2.101:0/2853347276 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a4000c10 con 0x7f38b40bba80 2021-02-10T11:45:39.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.494 7f389bfff700 1 -- 172.21.2.101:0/2853347276 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f38a40022b0 con 0x7f38b40bba80 2021-02-10T11:45:39.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.494 7f38ba1ba700 1 -- 172.21.2.101:0/2853347276 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b40be200 con 0x7f38b40bba80 2021-02-10T11:45:39.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.494 7f38ba1ba700 1 -- 172.21.2.101:0/2853347276 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b40be200 con 0x7f38b40bba80 2021-02-10T11:45:39.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.495 7f389bfff700 1 -- 172.21.2.101:0/2853347276 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f38a400a000 con 0x7f38b40bba80 2021-02-10T11:45:39.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.495 7f389bfff700 1 --2- 172.21.2.101:0/2853347276 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38940186c0 0x7f389401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:39.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.495 7f389bfff700 1 -- 172.21.2.101:0/2853347276 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f38a4032200 con 0x7f38b40bba80 2021-02-10T11:45:39.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.495 7f38b2ffd700 1 --2- 172.21.2.101:0/2853347276 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38940186c0 0x7f389401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:39.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.495 7f38b2ffd700 1 --2- 172.21.2.101:0/2853347276 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38940186c0 0x7f389401abc0 crc :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:39.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.525 7f38ba1ba700 1 -- 172.21.2.101:0/2853347276 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b40f0b50 con 0x7f38b40bba80 2021-02-10T11:45:39.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.527 7f389bfff700 1 -- 172.21.2.101:0/2853347276 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f38a402e330 con 0x7f38b40bba80 2021-02-10T11:45:39.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.659 7f38ba1ba700 1 -- 172.21.2.101:0/2853347276 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f38b40f0b50 con 0x7f38b40bba80 2021-02-10T11:45:39.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.659 7f389bfff700 1 -- 172.21.2.101:0/2853347276 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v90) v1 ==== 80+0+4513 (secure 0 0 0) 0x7f38a402e330 con 0x7f38b40bba80 2021-02-10T11:45:39.658 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:45:39.658 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:45:39.658 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:45:39.658 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:39.658 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:45:39.658 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:45:39.658 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:45:39.658 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:45:39.659 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:45:39.659 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:45:39.659 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:45:39.659 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:39.659 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:39.659 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:39.659 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:45:39.659 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:45:39.659 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:45:39.659 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:45:39.659 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:45:39.659 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:45:39.659 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:45:39.659 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:39.660 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:39.660 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:39.660 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:45:39.660 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:45:39.660 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:45:39.660 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:45:39.660 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:45:39.660 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:45:39.660 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:39.660 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:39.660 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:39.660 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:45:39.660 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:45:39.661 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:45:39.661 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:45:39.661 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:45:39.661 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:45:39.661 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:39.661 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:39.661 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:39.661 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:45:39.661 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:45:39.661 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:45:39.661 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:45:39.661 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:45:39.661 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:45:39.662 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:39.662 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:39.662 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:39.662 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:45:39.662 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:45:39.662 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:45:39.662 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:45:39.662 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:45:39.662 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:45:39.662 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:39.662 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:39.662 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:39.662 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:45:39.662 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:45:39.663 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:45:39.663 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:45:39.663 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:45:39.663 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:45:39.663 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:39.663 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:39.663 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:39.663 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:45:39.663 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:45:39.663 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:45:39.663 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:45:39.663 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:45:39.663 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:45:39.664 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:39.664 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:39.664 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:39.664 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:45:39.664 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:45:39.664 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:45:39.664 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:45:39.664 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:45:39.664 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:45:39.664 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:39.664 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:39.664 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:39.664 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:45:39.665 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:45:39.665 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:45:39.665 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:45:39.665 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:45:39.665 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:45:39.665 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:39.665 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:39.665 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:39.665 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:45:39.665 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:45:39.665 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:45:39.665 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:45:39.665 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:45:39.666 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:45:39.666 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:39.666 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:39.666 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:39.666 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:45:39.666 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:45:39.666 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:45:39.666 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:45:39.666 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:45:39.666 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:45:39.666 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:39.666 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:39.666 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:39.667 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:45:39.667 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:45:39.667 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:45:39.667 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:45:39.667 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:45:39.667 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:45:39.667 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:39.667 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:39.667 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:39.667 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:45:39.667 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:45:39.667 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:45:39.667 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:45:39.668 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:45:39.668 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:45:39.668 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:39.668 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:39.668 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:39.668 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:45:39.668 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:45:39.668 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:45:39.668 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:45:39.668 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:45:39.668 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:45:39.668 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:45:39.668 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:39.669 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:39.669 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:39.669 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:45:39.669 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:45:39.669 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:45:39.669 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:45:39.669 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:45:39.669 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:45:39.669 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:39.669 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:39.669 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:39.669 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:45:39.669 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:45:39.670 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:45:39.670 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:45:39.670 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:45:39.670 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:45:39.670 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:39.670 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:39.670 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:45:39.670 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:45:39.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.661 7f38ba1ba700 1 -- 172.21.2.101:0/2853347276 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38940186c0 msgr2=0x7f389401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:39.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.661 7f38ba1ba700 1 --2- 172.21.2.101:0/2853347276 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38940186c0 0x7f389401abc0 crc :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:39.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.661 7f38ba1ba700 1 -- 172.21.2.101:0/2853347276 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38b40bba80 msgr2=0x7f38b4108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:39.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.661 7f38ba1ba700 1 --2- 172.21.2.101:0/2853347276 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38b40bba80 0x7f38b4108e90 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 rx=0x7f38a4006b40 tx=0x7f38a400b990).stop 2021-02-10T11:45:39.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.661 7f38ba1ba700 1 -- 172.21.2.101:0/2853347276 shutdown_connections 2021-02-10T11:45:39.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.661 7f38ba1ba700 1 --2- 172.21.2.101:0/2853347276 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38940186c0 0x7f389401abc0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:39.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.661 7f38ba1ba700 1 --2- 172.21.2.101:0/2853347276 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38b40bbe80 0x7f38b40bd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:39.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.661 7f38ba1ba700 1 --2- 172.21.2.101:0/2853347276 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38b41094b0 0x7f38b40bdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:39.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.661 7f38ba1ba700 1 --2- 172.21.2.101:0/2853347276 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38b40bba80 0x7f38b4108e90 unknown :-1 s=CLOSED pgs=2856 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:39.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.661 7f38ba1ba700 1 -- 172.21.2.101:0/2853347276 >> 172.21.2.101:0/2853347276 conn(0x7f38b40bc6c0 msgr2=0x7f38b40b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:39.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.661 7f38ba1ba700 1 -- 172.21.2.101:0/2853347276 shutdown_connections 2021-02-10T11:45:39.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:39.662 7f38ba1ba700 1 -- 172.21.2.101:0/2853347276 wait complete. 2021-02-10T11:45:39.673 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:45:40.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.089 7fcfffee0700 1 Processor -- start 2021-02-10T11:45:40.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.089 7fcfffee0700 1 -- start start 2021-02-10T11:45:40.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.089 7fcfffee0700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fcff80d1870 0x7fcff81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:40.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.089 7fcfffee0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcff80f0650 0x7fcff80cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:40.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.089 7fcfffee0700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff80f0b10 con 0x7fcff81b0a10 2021-02-10T11:45:40.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.089 7fcfffee0700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcff80e5ed0 con 0x7fcff80d1870 2021-02-10T11:45:40.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.089 7fcfffee0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff80d1c70 con 0x7fcff80f0650 2021-02-10T11:45:40.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.090 7fcffd4fb700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fcff81b0a10 0x7fcff80cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33062/0 (socket says 172.21.2.101:33062) 2021-02-10T11:45:40.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.090 7fcffd4fb700 1 -- 172.21.2.101:0/3582878926 learned_addr learned my addr 172.21.2.101:0/3582878926 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:40.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.090 7fcffe4fd700 1 --2- 172.21.2.101:0/3582878926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcff80f0650 0x7fcff80cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:40.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.090 7fcffdcfc700 1 --2- 172.21.2.101:0/3582878926 >> v2:172.21.2.108:3300/0 conn(0x7fcff80d1870 0x7fcff81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:40.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.090 7fcffccfa700 1 -- 172.21.2.101:0/3582878926 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3723305561 0 0) 0x7fcfe80009b0 con 0x7fcff81b0a10 2021-02-10T11:45:40.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.090 7fcffccfa700 1 -- 172.21.2.101:0/3582878926 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfe4002fe0 con 0x7fcff81b0a10 2021-02-10T11:45:40.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.090 7fcffe4fd700 1 -- 172.21.2.101:0/3582878926 >> v2:172.21.2.108:3300/0 conn(0x7fcff80d1870 msgr2=0x7fcff81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:40.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.090 7fcffe4fd700 1 --2- 172.21.2.101:0/3582878926 >> v2:172.21.2.108:3300/0 conn(0x7fcff80d1870 0x7fcff81b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:40.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.090 7fcffe4fd700 1 -- 172.21.2.101:0/3582878926 >> v1:172.21.2.101:6789/0 conn(0x7fcff81b0a10 legacy=0x7fcff80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:40.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.090 7fcffe4fd700 1 -- 172.21.2.101:0/3582878926 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff80d1e90 con 0x7fcff80f0650 2021-02-10T11:45:40.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.090 7fcffdcfc700 1 --2- 172.21.2.101:0/3582878926 >> v2:172.21.2.108:3300/0 conn(0x7fcff80d1870 0x7fcff81b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:45:40.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.090 7fcffe4fd700 1 --2- 172.21.2.101:0/3582878926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcff80f0650 0x7fcff80cfb20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7fcff0006e50 tx=0x7fcff0009e70).ready entity=mon.2 client_cookie=7ceb0ea56472b4b8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:40.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.091 7fcffccfa700 1 -- 172.21.2.101:0/3582878926 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcff0006a20 con 0x7fcff80f0650 2021-02-10T11:45:40.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.091 7fcffccfa700 1 -- 172.21.2.101:0/3582878926 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff0006790 con 0x7fcff80f0650 2021-02-10T11:45:40.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.091 7fcffccfa700 1 -- 172.21.2.101:0/3582878926 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcff000b700 con 0x7fcff80f0650 2021-02-10T11:45:40.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.091 7fcfffee0700 1 -- 172.21.2.101:0/3582878926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcff80f0650 msgr2=0x7fcff80cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:40.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.091 7fcfffee0700 1 --2- 172.21.2.101:0/3582878926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcff80f0650 0x7fcff80cfb20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7fcff0006e50 tx=0x7fcff0009e70).stop 2021-02-10T11:45:40.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.092 7fcfffee0700 1 -- 172.21.2.101:0/3582878926 shutdown_connections 2021-02-10T11:45:40.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.092 7fcfffee0700 1 --2- 172.21.2.101:0/3582878926 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcff80f0650 0x7fcff80cfb20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:40.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.092 7fcfffee0700 1 --2- 172.21.2.101:0/3582878926 >> v2:172.21.2.108:3300/0 conn(0x7fcff80d1870 0x7fcff81b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:40.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.092 7fcfffee0700 1 -- 172.21.2.101:0/3582878926 >> 172.21.2.101:0/3582878926 conn(0x7fcff80ce450 msgr2=0x7fcff80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:40.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.092 7fcfffee0700 1 -- 172.21.2.101:0/3582878926 shutdown_connections 2021-02-10T11:45:40.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.092 7fcfffee0700 1 -- 172.21.2.101:0/3582878926 wait complete. 2021-02-10T11:45:40.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.106 7fcfffee0700 1 Processor -- start 2021-02-10T11:45:40.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.106 7fcfffee0700 1 -- start start 2021-02-10T11:45:40.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.107 7fcfffee0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcff8195650 0x7fcff80b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:40.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.107 7fcfffee0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcff80b5b80 0x7fcff81a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:40.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.107 7fcfffee0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcff80acfa0 0x7fcff81a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:40.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.107 7fcfffee0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff80b6010 con 0x7fcff80acfa0 2021-02-10T11:45:40.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.107 7fcfffee0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff81a3a60 con 0x7fcff8195650 2021-02-10T11:45:40.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.107 7fcfffee0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff81a3c40 con 0x7fcff80b5b80 2021-02-10T11:45:40.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.107 7fcffd4fb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcff80b5b80 0x7fcff81a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.107 7fcffe4fd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcff80acfa0 0x7fcff81a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.107 7fcffe4fd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcff80acfa0 0x7fcff81a3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60804/0 (socket says 172.21.2.101:60804) 2021-02-10T11:45:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.107 7fcffe4fd700 1 -- 172.21.2.101:0/2815288780 learned_addr learned my addr 172.21.2.101:0/2815288780 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.107 7fcffdcfc700 1 --2- 172.21.2.101:0/2815288780 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcff8195650 0x7fcff80b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.107 7fcffe4fd700 1 -- 172.21.2.101:0/2815288780 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcff80b5b80 msgr2=0x7fcff81a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.107 7fcffe4fd700 1 --2- 172.21.2.101:0/2815288780 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcff80b5b80 0x7fcff81a2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.107 7fcffe4fd700 1 -- 172.21.2.101:0/2815288780 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcff8195650 msgr2=0x7fcff80b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.107 7fcffe4fd700 1 --2- 172.21.2.101:0/2815288780 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcff8195650 0x7fcff80b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.107 7fcffe4fd700 1 -- 172.21.2.101:0/2815288780 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff80bcae0 con 0x7fcff80acfa0 2021-02-10T11:45:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.107 7fcffe4fd700 1 --2- 172.21.2.101:0/2815288780 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcff80acfa0 0x7fcff81a3440 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 rx=0x7fcff00099f0 tx=0x7fcff000bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.108 7fcfee7fc700 1 -- 172.21.2.101:0/2815288780 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcff00093d0 con 0x7fcff80acfa0 2021-02-10T11:45:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.108 7fcfee7fc700 1 -- 172.21.2.101:0/2815288780 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff0006880 con 0x7fcff80acfa0 2021-02-10T11:45:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.108 7fcfffee0700 1 -- 172.21.2.101:0/2815288780 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff80bcae0 con 0x7fcff80acfa0 2021-02-10T11:45:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.108 7fcfee7fc700 1 -- 172.21.2.101:0/2815288780 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcff000c570 con 0x7fcff80acfa0 2021-02-10T11:45:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.108 7fcfffee0700 1 -- 172.21.2.101:0/2815288780 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff80bcae0 con 0x7fcff80acfa0 2021-02-10T11:45:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.108 7fcfee7fc700 1 -- 172.21.2.101:0/2815288780 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fcff0009350 con 0x7fcff80acfa0 2021-02-10T11:45:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.108 7fcfee7fc700 1 --2- 172.21.2.101:0/2815288780 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcfe0018730 0x7fcfe001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.108 7fcfee7fc700 1 -- 172.21.2.101:0/2815288780 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fcff0009350 con 0x7fcff80acfa0 2021-02-10T11:45:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.108 7fcffdcfc700 1 --2- 172.21.2.101:0/2815288780 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcfe0018730 0x7fcfe001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.109 7fcffdcfc700 1 --2- 172.21.2.101:0/2815288780 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcfe0018730 0x7fcfe001ac10 crc :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:40.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.138 7fcfffee0700 1 -- 172.21.2.101:0/2815288780 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff80b6010 con 0x7fcff80acfa0 2021-02-10T11:45:40.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.140 7fcfee7fc700 1 -- 172.21.2.101:0/2815288780 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fcff0009350 con 0x7fcff80acfa0 2021-02-10T11:45:40.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.266 7fcfffee0700 1 -- 172.21.2.101:0/2815288780 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcff80b6010 con 0x7fcff80acfa0 2021-02-10T11:45:40.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.267 7fcfee7fc700 1 -- 172.21.2.101:0/2815288780 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+11154 (secure 0 0 0) 0x7fcff001f860 con 0x7fcff80acfa0 2021-02-10T11:45:40.265 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:45:40.265 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":160,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:45:39.008153","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:45:40.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.268 7fcfffee0700 1 -- 172.21.2.101:0/2815288780 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcfe0018730 msgr2=0x7fcfe001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:40.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.268 7fcfffee0700 1 --2- 172.21.2.101:0/2815288780 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcfe0018730 0x7fcfe001ac10 crc :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:40.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.268 7fcfffee0700 1 -- 172.21.2.101:0/2815288780 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcff80acfa0 msgr2=0x7fcff81a3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:40.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.268 7fcfffee0700 1 --2- 172.21.2.101:0/2815288780 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcff80acfa0 0x7fcff81a3440 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 rx=0x7fcff00099f0 tx=0x7fcff000bd10).stop 2021-02-10T11:45:40.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.269 7fcfffee0700 1 -- 172.21.2.101:0/2815288780 shutdown_connections 2021-02-10T11:45:40.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.269 7fcfffee0700 1 --2- 172.21.2.101:0/2815288780 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcfe0018730 0x7fcfe001ac10 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:40.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.269 7fcfffee0700 1 --2- 172.21.2.101:0/2815288780 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcff80b5b80 0x7fcff81a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:40.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.269 7fcfffee0700 1 --2- 172.21.2.101:0/2815288780 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcff80acfa0 0x7fcff81a3440 unknown :-1 s=CLOSED pgs=2858 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:40.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.269 7fcfffee0700 1 --2- 172.21.2.101:0/2815288780 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcff8195650 0x7fcff80b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:40.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.269 7fcfffee0700 1 -- 172.21.2.101:0/2815288780 >> 172.21.2.101:0/2815288780 conn(0x7fcff80b3040 msgr2=0x7fcff80c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:40.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.269 7fcfffee0700 1 -- 172.21.2.101:0/2815288780 shutdown_connections 2021-02-10T11:45:40.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.269 7fcfffee0700 1 -- 172.21.2.101:0/2815288780 wait complete. 2021-02-10T11:45:40.275 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-10T11:45:40.276 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:45:40.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.692 7f2926c18700 1 Processor -- start 2021-02-10T11:45:40.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.692 7f2926c18700 1 -- start start 2021-02-10T11:45:40.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.693 7f2926c18700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29200d1870 0x7f29201b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:40.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.693 7f2926c18700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f29201b0ae0 0x7f2920108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:40.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.693 7f2926c18700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f29200d1c70 0x7f2920108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:40.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.693 7f2926c18700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29200e5ef0 con 0x7f29201b0ae0 2021-02-10T11:45:40.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.693 7f2926c18700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2920109a60 con 0x7f29200d1c70 2021-02-10T11:45:40.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.693 7f2926c18700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2920088070 con 0x7f29200d1870 2021-02-10T11:45:40.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.693 7f291ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f29201b0ae0 0x7f2920108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:40.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.693 7f291ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f29201b0ae0 0x7f2920108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60810/0 (socket says 172.21.2.101:60810) 2021-02-10T11:45:40.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.693 7f291ffff700 1 -- 172.21.2.101:0/618243974 learned_addr learned my addr 172.21.2.101:0/618243974 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:40.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.693 7f291ffff700 1 -- 172.21.2.101:0/618243974 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29200d1870 msgr2=0x7f29201b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:45:40.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.693 7f291ffff700 1 --2- 172.21.2.101:0/618243974 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29200d1870 0x7f29201b04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:40.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.693 7f291ffff700 1 -- 172.21.2.101:0/618243974 >> v2:172.21.2.108:3300/0 conn(0x7f29200d1c70 msgr2=0x7f2920108f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:45:40.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.693 7f291ffff700 1 --2- 172.21.2.101:0/618243974 >> v2:172.21.2.108:3300/0 conn(0x7f29200d1c70 0x7f2920108f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:40.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.694 7f291ffff700 1 -- 172.21.2.101:0/618243974 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2920109580 con 0x7f29201b0ae0 2021-02-10T11:45:40.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.694 7f291ffff700 1 --2- 172.21.2.101:0/618243974 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29201b0ae0 0x7f2920108940 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 rx=0x7f2910006a80 tx=0x7f2910009f00).ready entity=mon.0 client_cookie=3bb933d1090dadf9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:40.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.694 7f291f7fe700 1 -- 172.21.2.101:0/618243974 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f291000b5c0 con 0x7f29201b0ae0 2021-02-10T11:45:40.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.694 7f291f7fe700 1 -- 172.21.2.101:0/618243974 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291000b800 con 0x7f29201b0ae0 2021-02-10T11:45:40.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.694 7f291f7fe700 1 -- 172.21.2.101:0/618243974 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f291000bd20 con 0x7f29201b0ae0 2021-02-10T11:45:40.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.694 7f2926c18700 1 -- 172.21.2.101:0/618243974 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29201b0ae0 msgr2=0x7f2920108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:40.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.694 7f2926c18700 1 --2- 172.21.2.101:0/618243974 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29201b0ae0 0x7f2920108940 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 rx=0x7f2910006a80 tx=0x7f2910009f00).stop 2021-02-10T11:45:40.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.695 7f2926c18700 1 -- 172.21.2.101:0/618243974 shutdown_connections 2021-02-10T11:45:40.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.695 7f2926c18700 1 --2- 172.21.2.101:0/618243974 >> v2:172.21.2.108:3300/0 conn(0x7f29200d1c70 0x7f2920108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:40.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.695 7f2926c18700 1 --2- 172.21.2.101:0/618243974 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29201b0ae0 0x7f2920108940 unknown :-1 s=CLOSED pgs=2859 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:40.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.695 7f2926c18700 1 --2- 172.21.2.101:0/618243974 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29200d1870 0x7f29201b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:40.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.695 7f2926c18700 1 -- 172.21.2.101:0/618243974 >> 172.21.2.101:0/618243974 conn(0x7f29200ce420 msgr2=0x7f29200d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:40.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.695 7f2926c18700 1 -- 172.21.2.101:0/618243974 shutdown_connections 2021-02-10T11:45:40.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.695 7f2926c18700 1 -- 172.21.2.101:0/618243974 wait complete. 2021-02-10T11:45:40.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.710 7f2926c18700 1 Processor -- start 2021-02-10T11:45:40.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.710 7f2926c18700 1 -- start start 2021-02-10T11:45:40.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.710 7f2926c18700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29200872c0 0x7f29201a7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:40.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.710 7f2926c18700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29201a7ba0 0x7f29200a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:40.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.710 7f2926c18700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29201a7fa0 0x7f29200a6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:40.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.710 7f2926c18700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2920086e20 con 0x7f29201a7ba0 2021-02-10T11:45:40.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.710 7f2926c18700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2920190c60 con 0x7f29201a7fa0 2021-02-10T11:45:40.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.710 7f2926c18700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29200a74a0 con 0x7f29200872c0 2021-02-10T11:45:40.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.710 7f291ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29201a7ba0 0x7f29200a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:40.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.710 7f291ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29201a7ba0 0x7f29200a6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60818/0 (socket says 172.21.2.101:60818) 2021-02-10T11:45:40.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.710 7f291ffff700 1 -- 172.21.2.101:0/1413467516 learned_addr learned my addr 172.21.2.101:0/1413467516 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:40.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.711 7f291ffff700 1 -- 172.21.2.101:0/1413467516 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29200872c0 msgr2=0x7f29201a7580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:40.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.711 7f291ffff700 1 --2- 172.21.2.101:0/1413467516 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29200872c0 0x7f29201a7580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:40.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.711 7f291ffff700 1 -- 172.21.2.101:0/1413467516 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29201a7fa0 msgr2=0x7f29200a6e80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:40.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.711 7f291ffff700 1 --2- 172.21.2.101:0/1413467516 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29201a7fa0 0x7f29200a6e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:40.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.711 7f291ffff700 1 -- 172.21.2.101:0/1413467516 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29200a76c0 con 0x7f29201a7ba0 2021-02-10T11:45:40.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.711 7f291ffff700 1 --2- 172.21.2.101:0/1413467516 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29201a7ba0 0x7f29200a6860 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 rx=0x7f2910006a90 tx=0x7f291000bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:40.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.711 7f291d7fa700 1 -- 172.21.2.101:0/1413467516 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2910000d30 con 0x7f29201a7ba0 2021-02-10T11:45:40.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.711 7f291d7fa700 1 -- 172.21.2.101:0/1413467516 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2910006710 con 0x7f29201a7ba0 2021-02-10T11:45:40.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.711 7f291d7fa700 1 -- 172.21.2.101:0/1413467516 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f291000c4e0 con 0x7f29201a7ba0 2021-02-10T11:45:40.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.711 7f2926c18700 1 -- 172.21.2.101:0/1413467516 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29200a76c0 con 0x7f29201a7ba0 2021-02-10T11:45:40.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.711 7f2926c18700 1 -- 172.21.2.101:0/1413467516 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29200a76c0 con 0x7f29201a7ba0 2021-02-10T11:45:40.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.712 7f291d7fa700 1 -- 172.21.2.101:0/1413467516 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2910007000 con 0x7f29201a7ba0 2021-02-10T11:45:40.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.712 7f291d7fa700 1 --2- 172.21.2.101:0/1413467516 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29040186c0 0x7f290401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:40.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.712 7f291d7fa700 1 -- 172.21.2.101:0/1413467516 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f29100325e0 con 0x7f29201a7ba0 2021-02-10T11:45:40.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.712 7f2924a34700 1 --2- 172.21.2.101:0/1413467516 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29040186c0 0x7f290401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:40.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.712 7f2924a34700 1 --2- 172.21.2.101:0/1413467516 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29040186c0 0x7f290401abc0 crc :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:40.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.743 7f2926c18700 1 -- 172.21.2.101:0/1413467516 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2920086e20 con 0x7f29201a7ba0 2021-02-10T11:45:40.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.746 7f291d7fa700 1 -- 172.21.2.101:0/1413467516 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2910028270 con 0x7f29201a7ba0 2021-02-10T11:45:40.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.873 7f2926c18700 1 -- 172.21.2.101:0/1413467516 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f2920086e20 con 0x7f29201a7ba0 2021-02-10T11:45:40.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.873 7f291d7fa700 1 -- 172.21.2.101:0/1413467516 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v160) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2910028270 con 0x7f29201a7ba0 2021-02-10T11:45:40.872 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:45:40.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.875 7f2926c18700 1 -- 172.21.2.101:0/1413467516 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29040186c0 msgr2=0x7f290401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:40.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.875 7f2926c18700 1 --2- 172.21.2.101:0/1413467516 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29040186c0 0x7f290401abc0 crc :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:40.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.875 7f2926c18700 1 -- 172.21.2.101:0/1413467516 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29201a7ba0 msgr2=0x7f29200a6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:40.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.875 7f2926c18700 1 --2- 172.21.2.101:0/1413467516 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29201a7ba0 0x7f29200a6860 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 rx=0x7f2910006a90 tx=0x7f291000bd10).stop 2021-02-10T11:45:40.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.875 7f2926c18700 1 -- 172.21.2.101:0/1413467516 shutdown_connections 2021-02-10T11:45:40.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.875 7f2926c18700 1 --2- 172.21.2.101:0/1413467516 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29201a7fa0 0x7f29200a6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:40.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.875 7f2926c18700 1 --2- 172.21.2.101:0/1413467516 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29201a7ba0 0x7f29200a6860 unknown :-1 s=CLOSED pgs=2860 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:40.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.875 7f2926c18700 1 --2- 172.21.2.101:0/1413467516 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29040186c0 0x7f290401abc0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:40.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.875 7f2926c18700 1 --2- 172.21.2.101:0/1413467516 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29200872c0 0x7f29201a7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:40.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.875 7f2926c18700 1 -- 172.21.2.101:0/1413467516 >> 172.21.2.101:0/1413467516 conn(0x7f29200be980 msgr2=0x7f2920092cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:40.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.875 7f2926c18700 1 -- 172.21.2.101:0/1413467516 shutdown_connections 2021-02-10T11:45:40.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:40.875 7f2926c18700 1 -- 172.21.2.101:0/1413467516 wait complete. 2021-02-10T11:45:40.882 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --format=json-pretty osd lspools 2021-02-10T11:45:41.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.296 7f9640f0b700 1 Processor -- start 2021-02-10T11:45:41.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.296 7f9640f0b700 1 -- start start 2021-02-10T11:45:41.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.297 7f9640f0b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f963c0d1870 0x7f963c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:41.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.297 7f9640f0b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f963c0f0650 0x7f963c0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:41.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.297 7f9640f0b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f963c0f0b10 con 0x7f963c1b0a10 2021-02-10T11:45:41.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.297 7f9640f0b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f963c0fe230 con 0x7f963c0f0650 2021-02-10T11:45:41.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.297 7f9640f0b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f963c0e5ed0 con 0x7f963c0d1870 2021-02-10T11:45:41.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.298 7f963b314700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f963c0f0650 0x7f963c0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:41.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.298 7f963ab13700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f963c0d1870 0x7f963c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:41.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.298 7f963a312700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f963c1b0a10 0x7f963c0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:54730/0 (socket says 172.21.2.101:54730) 2021-02-10T11:45:41.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.298 7f963a312700 1 -- 172.21.2.101:0/374344266 learned_addr learned my addr 172.21.2.101:0/374344266 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:41.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.298 7f963b314700 1 -- 172.21.2.101:0/374344266 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f963c0d1870 msgr2=0x7f963c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:41.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.298 7f963b314700 1 --2- 172.21.2.101:0/374344266 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f963c0d1870 0x7f963c1b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:41.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.298 7f963b314700 1 -- 172.21.2.101:0/374344266 >> v1:172.21.2.108:6789/0 conn(0x7f963c1b0a10 legacy=0x7f963c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:41.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.298 7f963b314700 1 -- 172.21.2.101:0/374344266 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963c0f20a0 con 0x7f963c0f0650 2021-02-10T11:45:41.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.298 7f963b314700 1 --2- 172.21.2.101:0/374344266 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f963c0f0650 0x7f963c0cfb20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7f9634006e50 tx=0x7f9634009df0).ready entity=mon.1 client_cookie=3fefd8da374f927 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:41.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.298 7f9639b11700 1 -- 172.21.2.101:0/374344266 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f963400b510 con 0x7f963c0f0650 2021-02-10T11:45:41.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.298 7f9639b11700 1 -- 172.21.2.101:0/374344266 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9634003030 con 0x7f963c0f0650 2021-02-10T11:45:41.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.298 7f9639b11700 1 -- 172.21.2.101:0/374344266 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f963400b7d0 con 0x7f963c0f0650 2021-02-10T11:45:41.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.299 7f9640f0b700 1 -- 172.21.2.101:0/374344266 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f963c0f0650 msgr2=0x7f963c0cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:41.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.299 7f9640f0b700 1 --2- 172.21.2.101:0/374344266 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f963c0f0650 0x7f963c0cfb20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7f9634006e50 tx=0x7f9634009df0).stop 2021-02-10T11:45:41.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.299 7f9640f0b700 1 -- 172.21.2.101:0/374344266 shutdown_connections 2021-02-10T11:45:41.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.299 7f9640f0b700 1 --2- 172.21.2.101:0/374344266 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f963c0f0650 0x7f963c0cfb20 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:41.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.299 7f9640f0b700 1 --2- 172.21.2.101:0/374344266 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f963c0d1870 0x7f963c1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:41.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.299 7f9640f0b700 1 -- 172.21.2.101:0/374344266 >> 172.21.2.101:0/374344266 conn(0x7f963c0ce450 msgr2=0x7f963c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:41.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.299 7f9640f0b700 1 -- 172.21.2.101:0/374344266 shutdown_connections 2021-02-10T11:45:41.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.299 7f9640f0b700 1 -- 172.21.2.101:0/374344266 wait complete. 2021-02-10T11:45:41.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.314 7f9640f0b700 1 Processor -- start 2021-02-10T11:45:41.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.314 7f9640f0b700 1 -- start start 2021-02-10T11:45:41.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.314 7f9640f0b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f963c1942b0 0x7f963c0b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:41.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.314 7f9640f0b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f963c195070 0x7f963c195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:41.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.314 7f9640f0b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f963c0bb8e0 0x7f963c0bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:41.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.314 7f9640f0b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c1946b0 con 0x7f963c1942b0 2021-02-10T11:45:41.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.314 7f9640f0b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c0b3b70 con 0x7f963c0bb8e0 2021-02-10T11:45:41.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.314 7f9640f0b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f963c0b3d90 con 0x7f963c195070 2021-02-10T11:45:41.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.314 7f963ab13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f963c1942b0 0x7f963c0b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:41.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.314 7f963ab13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f963c1942b0 0x7f963c0b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60830/0 (socket says 172.21.2.101:60830) 2021-02-10T11:45:41.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.314 7f963ab13700 1 -- 172.21.2.101:0/3479770191 learned_addr learned my addr 172.21.2.101:0/3479770191 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:41.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.315 7f963ab13700 1 -- 172.21.2.101:0/3479770191 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f963c195070 msgr2=0x7f963c195470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:41.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.315 7f963ab13700 1 --2- 172.21.2.101:0/3479770191 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f963c195070 0x7f963c195470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:41.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.315 7f963ab13700 1 -- 172.21.2.101:0/3479770191 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f963c0bb8e0 msgr2=0x7f963c0bbce0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:41.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.315 7f963ab13700 1 --2- 172.21.2.101:0/3479770191 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f963c0bb8e0 0x7f963c0bbce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:41.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.315 7f963ab13700 1 -- 172.21.2.101:0/3479770191 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963c1a49e0 con 0x7f963c1942b0 2021-02-10T11:45:41.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.315 7f963ab13700 1 --2- 172.21.2.101:0/3479770191 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f963c1942b0 0x7f963c0b3550 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 rx=0x7f962400b560 tx=0x7f962400b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:41.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.315 7f96337fe700 1 -- 172.21.2.101:0/3479770191 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f962400bcf0 con 0x7f963c1942b0 2021-02-10T11:45:41.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.315 7f96337fe700 1 -- 172.21.2.101:0/3479770191 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9624000c90 con 0x7f963c1942b0 2021-02-10T11:45:41.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.315 7f96337fe700 1 -- 172.21.2.101:0/3479770191 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96240022b0 con 0x7f963c1942b0 2021-02-10T11:45:41.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.315 7f9640f0b700 1 -- 172.21.2.101:0/3479770191 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f963c1a49e0 con 0x7f963c1942b0 2021-02-10T11:45:41.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.315 7f9640f0b700 1 -- 172.21.2.101:0/3479770191 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f963c1a49e0 con 0x7f963c1942b0 2021-02-10T11:45:41.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.316 7f96337fe700 1 -- 172.21.2.101:0/3479770191 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9624006b00 con 0x7f963c1942b0 2021-02-10T11:45:41.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.316 7f96337fe700 1 --2- 172.21.2.101:0/3479770191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96200186c0 0x7f962001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:41.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.316 7f96337fe700 1 -- 172.21.2.101:0/3479770191 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f9624032250 con 0x7f963c1942b0 2021-02-10T11:45:41.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.316 7f963a312700 1 --2- 172.21.2.101:0/3479770191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96200186c0 0x7f962001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:41.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.317 7f963a312700 1 --2- 172.21.2.101:0/3479770191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96200186c0 0x7f962001abc0 crc :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:41.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.346 7f9640f0b700 1 -- 172.21.2.101:0/3479770191 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f963c0bc5f0 con 0x7f963c1942b0 2021-02-10T11:45:41.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.349 7f96337fe700 1 -- 172.21.2.101:0/3479770191 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f962400bcf0 con 0x7f963c1942b0 2021-02-10T11:45:41.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.472 7f9640f0b700 1 -- 172.21.2.101:0/3479770191 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f963c0bc5f0 con 0x7f963c1942b0 2021-02-10T11:45:41.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.473 7f96337fe700 1 -- 172.21.2.101:0/3479770191 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v160) v1 ==== 84+0+64 (secure 0 0 0) 0x7f9624006b00 con 0x7f963c1942b0 2021-02-10T11:45:41.472 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:45:41.472 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:45:41.472 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:45:41.472 INFO:teuthology.orchestra.run.gibba001.stdout: "poolnum": 1, 2021-02-10T11:45:41.472 INFO:teuthology.orchestra.run.gibba001.stdout: "poolname": "rbd" 2021-02-10T11:45:41.472 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:41.472 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:45:41.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.475 7f9640f0b700 1 -- 172.21.2.101:0/3479770191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96200186c0 msgr2=0x7f962001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:41.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.475 7f9640f0b700 1 --2- 172.21.2.101:0/3479770191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96200186c0 0x7f962001abc0 crc :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:41.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.475 7f9640f0b700 1 -- 172.21.2.101:0/3479770191 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f963c1942b0 msgr2=0x7f963c0b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:41.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.475 7f9640f0b700 1 --2- 172.21.2.101:0/3479770191 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f963c1942b0 0x7f963c0b3550 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 rx=0x7f962400b560 tx=0x7f962400b900).stop 2021-02-10T11:45:41.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.475 7f9640f0b700 1 -- 172.21.2.101:0/3479770191 shutdown_connections 2021-02-10T11:45:41.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.475 7f9640f0b700 1 --2- 172.21.2.101:0/3479770191 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96200186c0 0x7f962001abc0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:41.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.475 7f9640f0b700 1 --2- 172.21.2.101:0/3479770191 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f963c195070 0x7f963c195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:41.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.475 7f9640f0b700 1 --2- 172.21.2.101:0/3479770191 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f963c0bb8e0 0x7f963c0bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:41.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.475 7f9640f0b700 1 --2- 172.21.2.101:0/3479770191 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f963c1942b0 0x7f963c0b3550 unknown :-1 s=CLOSED pgs=2861 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:41.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.475 7f9640f0b700 1 -- 172.21.2.101:0/3479770191 >> 172.21.2.101:0/3479770191 conn(0x7f963c0b3040 msgr2=0x7f963c0c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:41.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.475 7f9640f0b700 1 -- 172.21.2.101:0/3479770191 shutdown_connections 2021-02-10T11:45:41.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.475 7f9640f0b700 1 -- 172.21.2.101:0/3479770191 wait complete. 2021-02-10T11:45:41.481 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-02-10T11:45:41.481 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-02-10T11:45:41.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.899 7f56d3807700 1 Processor -- start 2021-02-10T11:45:41.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.899 7f56d3807700 1 -- start start 2021-02-10T11:45:41.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.899 7f56d3807700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56cc10afe0 0x7f56cc0cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:41.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.899 7f56d3807700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56cc0fe1b0 con 0x7f56cc0d1860 2021-02-10T11:45:41.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.899 7f56d3807700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56cc1b0ca0 con 0x7f56cc0d1c60 2021-02-10T11:45:41.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.899 7f56d3807700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc109a50 con 0x7f56cc10afe0 2021-02-10T11:45:41.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.900 7f56d1623700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f56cc0d1860 0x7f56cc1b04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33100/0 (socket says 172.21.2.101:33100) 2021-02-10T11:45:41.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.900 7f56d1623700 1 -- 172.21.2.101:0/3495148244 learned_addr learned my addr 172.21.2.101:0/3495148244 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:41.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.900 7f56d1e24700 1 --2- 172.21.2.101:0/3495148244 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56cc10afe0 0x7f56cc0cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:41.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.900 7f56c3fff700 1 -- 172.21.2.101:0/3495148244 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1782855404 0 0) 0x7f56c80010b0 con 0x7f56cc0d1860 2021-02-10T11:45:41.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.900 7f56c3fff700 1 -- 172.21.2.101:0/3495148244 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56b8002fe0 con 0x7f56cc0d1860 2021-02-10T11:45:41.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.901 7f56d1e24700 1 -- 172.21.2.101:0/3495148244 >> v1:172.21.2.108:6789/0 conn(0x7f56cc0d1c60 legacy=0x7f56cc0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:41.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.901 7f56d1e24700 1 -- 172.21.2.101:0/3495148244 >> v1:172.21.2.101:6789/0 conn(0x7f56cc0d1860 legacy=0x7f56cc1b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:41.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.901 7f56d1e24700 1 -- 172.21.2.101:0/3495148244 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc10b3e0 con 0x7f56cc10afe0 2021-02-10T11:45:41.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.901 7f56d1e24700 1 --2- 172.21.2.101:0/3495148244 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56cc10afe0 0x7f56cc0cf740 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f56c4006e50 tx=0x7f56c4009e70).ready entity=mon.2 client_cookie=87dd2ea174ce7617 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:41.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.901 7f56c3fff700 1 -- 172.21.2.101:0/3495148244 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f56c4006a20 con 0x7f56cc10afe0 2021-02-10T11:45:41.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.901 7f56c3fff700 1 -- 172.21.2.101:0/3495148244 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c4006790 con 0x7f56cc10afe0 2021-02-10T11:45:41.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.901 7f56c3fff700 1 -- 172.21.2.101:0/3495148244 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f56c400b700 con 0x7f56cc10afe0 2021-02-10T11:45:41.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.901 7f56d3807700 1 -- 172.21.2.101:0/3495148244 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56cc10afe0 msgr2=0x7f56cc0cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:41.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.901 7f56d3807700 1 --2- 172.21.2.101:0/3495148244 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56cc10afe0 0x7f56cc0cf740 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f56c4006e50 tx=0x7f56c4009e70).stop 2021-02-10T11:45:41.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.902 7f56d3807700 1 -- 172.21.2.101:0/3495148244 shutdown_connections 2021-02-10T11:45:41.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.902 7f56d3807700 1 --2- 172.21.2.101:0/3495148244 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56cc10afe0 0x7f56cc0cf740 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:41.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.902 7f56d3807700 1 -- 172.21.2.101:0/3495148244 >> 172.21.2.101:0/3495148244 conn(0x7f56cc0ce410 msgr2=0x7f56cc0d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:41.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.902 7f56d3807700 1 -- 172.21.2.101:0/3495148244 shutdown_connections 2021-02-10T11:45:41.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.902 7f56d3807700 1 -- 172.21.2.101:0/3495148244 wait complete. 2021-02-10T11:45:41.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.917 7f56d3807700 1 Processor -- start 2021-02-10T11:45:41.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.917 7f56d3807700 1 -- start start 2021-02-10T11:45:41.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.917 7f56d3807700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56cc0bfa10 0x7f56cc0b82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:41.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.917 7f56d3807700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56cc1a4a70 0x7f56cc094c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:41.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.917 7f56d3807700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56cc0c0410 0x7f56cc0c0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:41.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.917 7f56d3807700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0c3a50 con 0x7f56cc1a4a70 2021-02-10T11:45:41.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.917 7f56d3807700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0b88c0 con 0x7f56cc0bfa10 2021-02-10T11:45:41.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.917 7f56d3807700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc095260 con 0x7f56cc0c0410 2021-02-10T11:45:41.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.918 7f56d0e22700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56cc1a4a70 0x7f56cc094c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:41.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.918 7f56d0e22700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56cc1a4a70 0x7f56cc094c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60842/0 (socket says 172.21.2.101:60842) 2021-02-10T11:45:41.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.918 7f56d0e22700 1 -- 172.21.2.101:0/1059552719 learned_addr learned my addr 172.21.2.101:0/1059552719 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:41.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.918 7f56d0e22700 1 -- 172.21.2.101:0/1059552719 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56cc0c0410 msgr2=0x7f56cc0c0810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:41.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.918 7f56d0e22700 1 --2- 172.21.2.101:0/1059552719 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56cc0c0410 0x7f56cc0c0810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:41.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.918 7f56d0e22700 1 -- 172.21.2.101:0/1059552719 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56cc0bfa10 msgr2=0x7f56cc0b82a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:41.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.918 7f56d0e22700 1 --2- 172.21.2.101:0/1059552719 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56cc0bfa10 0x7f56cc0b82a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:41.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.918 7f56d0e22700 1 -- 172.21.2.101:0/1059552719 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc0c3540 con 0x7f56cc1a4a70 2021-02-10T11:45:41.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.918 7f56d0e22700 1 --2- 172.21.2.101:0/1059552719 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56cc1a4a70 0x7f56cc094c40 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 rx=0x7f56bc006ed0 tx=0x7f56bc00b8e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:41.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.918 7f56c1ffb700 1 -- 172.21.2.101:0/1059552719 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f56bc006240 con 0x7f56cc1a4a70 2021-02-10T11:45:41.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.918 7f56c1ffb700 1 -- 172.21.2.101:0/1059552719 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc0054b0 con 0x7f56cc1a4a70 2021-02-10T11:45:41.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.918 7f56c1ffb700 1 -- 172.21.2.101:0/1059552719 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f56bc0059d0 con 0x7f56cc1a4a70 2021-02-10T11:45:41.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.918 7f56d3807700 1 -- 172.21.2.101:0/1059552719 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56cc0c3540 con 0x7f56cc1a4a70 2021-02-10T11:45:41.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.918 7f56d3807700 1 -- 172.21.2.101:0/1059552719 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56cc0c3540 con 0x7f56cc1a4a70 2021-02-10T11:45:41.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.919 7f56c1ffb700 1 -- 172.21.2.101:0/1059552719 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f56bc006030 con 0x7f56cc1a4a70 2021-02-10T11:45:41.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.919 7f56c1ffb700 1 --2- 172.21.2.101:0/1059552719 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56ac0186d0 0x7f56ac01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:41.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.919 7f56c1ffb700 1 -- 172.21.2.101:0/1059552719 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f56bc0324e0 con 0x7f56cc1a4a70 2021-02-10T11:45:41.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.919 7f56d1623700 1 --2- 172.21.2.101:0/1059552719 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56ac0186d0 0x7f56ac01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:41.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.919 7f56d1623700 1 --2- 172.21.2.101:0/1059552719 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56ac0186d0 0x7f56ac01abd0 crc :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:41.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.949 7f56d3807700 1 -- 172.21.2.101:0/1059552719 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56cc0bfee0 con 0x7f56cc1a4a70 2021-02-10T11:45:41.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:41.952 7f56c1ffb700 1 -- 172.21.2.101:0/1059552719 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f56bc000de0 con 0x7f56cc1a4a70 2021-02-10T11:45:42.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.081 7f56d3807700 1 -- 172.21.2.101:0/1059552719 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"} v 0) v1 -- 0x7f56cc0bfee0 con 0x7f56cc1a4a70 2021-02-10T11:45:42.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.493 7f56c1ffb700 1 -- 172.21.2.101:0/1059552719 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"}]=0 pool 'cephfs_metadata' created v161) v1 ==== 133+0+0 (secure 0 0 0) 0x7f56bc006030 con 0x7f56cc1a4a70 2021-02-10T11:45:42.495 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' created 2021-02-10T11:45:42.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.498 7f56d3807700 1 -- 172.21.2.101:0/1059552719 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56ac0186d0 msgr2=0x7f56ac01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:42.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.498 7f56d3807700 1 --2- 172.21.2.101:0/1059552719 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56ac0186d0 0x7f56ac01abd0 crc :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:42.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.498 7f56d3807700 1 -- 172.21.2.101:0/1059552719 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56cc1a4a70 msgr2=0x7f56cc094c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:42.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.498 7f56d3807700 1 --2- 172.21.2.101:0/1059552719 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56cc1a4a70 0x7f56cc094c40 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 rx=0x7f56bc006ed0 tx=0x7f56bc00b8e0).stop 2021-02-10T11:45:42.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.499 7f56d3807700 1 -- 172.21.2.101:0/1059552719 shutdown_connections 2021-02-10T11:45:42.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.499 7f56d3807700 1 --2- 172.21.2.101:0/1059552719 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56ac0186d0 0x7f56ac01abd0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:42.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.499 7f56d3807700 1 --2- 172.21.2.101:0/1059552719 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56cc0c0410 0x7f56cc0c0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:42.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.499 7f56d3807700 1 --2- 172.21.2.101:0/1059552719 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56cc1a4a70 0x7f56cc094c40 unknown :-1 s=CLOSED pgs=2863 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:42.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.499 7f56d3807700 1 --2- 172.21.2.101:0/1059552719 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56cc0bfa10 0x7f56cc0b82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:42.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.499 7f56d3807700 1 -- 172.21.2.101:0/1059552719 >> 172.21.2.101:0/1059552719 conn(0x7f56cc0a6a80 msgr2=0x7f56cc0ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:42.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.499 7f56d3807700 1 -- 172.21.2.101:0/1059552719 shutdown_connections 2021-02-10T11:45:42.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.499 7f56d3807700 1 -- 172.21.2.101:0/1059552719 wait complete. 2021-02-10T11:45:42.506 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_data 8 2021-02-10T11:45:42.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.922 7f0d23d01700 1 Processor -- start 2021-02-10T11:45:42.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.922 7f0d23d01700 1 -- start start 2021-02-10T11:45:42.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.923 7f0d23d01700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0d1c0d1870 0x7f0d1c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:42.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.923 7f0d23d01700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0d1c0cff70 0x7f0d1c109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:42.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.923 7f0d23d01700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d1c109a60 con 0x7f0d1c1b0ae0 2021-02-10T11:45:42.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.923 7f0d23d01700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1c109750 con 0x7f0d1c0d1870 2021-02-10T11:45:42.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.923 7f0d23d01700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c088070 con 0x7f0d1c0cff70 2021-02-10T11:45:42.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.923 7f0d2131c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f0d1c1b0ae0 0x7f0d1c108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33110/0 (socket says 172.21.2.101:33110) 2021-02-10T11:45:42.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.923 7f0d2131c700 1 -- 172.21.2.101:0/1326286210 learned_addr learned my addr 172.21.2.101:0/1326286210 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:42.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.923 7f0d20b1b700 1 -- 172.21.2.101:0/1326286210 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941740356 0 0) 0x7f0d180010b0 con 0x7f0d1c1b0ae0 2021-02-10T11:45:42.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.923 7f0d20b1b700 1 -- 172.21.2.101:0/1326286210 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d08002fe0 con 0x7f0d1c1b0ae0 2021-02-10T11:45:42.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.923 7f0d2231e700 1 --2- 172.21.2.101:0/1326286210 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0d1c0cff70 0x7f0d1c109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:42.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.923 7f0d21b1d700 1 --2- 172.21.2.101:0/1326286210 >> v2:172.21.2.108:3300/0 conn(0x7f0d1c0d1870 0x7f0d1c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:42.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d20b1b700 1 -- 172.21.2.101:0/1326286210 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1517797253 0 0) 0x7f0d180010b0 con 0x7f0d1c1b0ae0 2021-02-10T11:45:42.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d20b1b700 1 -- 172.21.2.101:0/1326286210 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0d1c0cff70 msgr2=0x7f0d1c109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:42.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d20b1b700 1 --2- 172.21.2.101:0/1326286210 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0d1c0cff70 0x7f0d1c109130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:42.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d20b1b700 1 -- 172.21.2.101:0/1326286210 >> v2:172.21.2.108:3300/0 conn(0x7f0d1c0d1870 msgr2=0x7f0d1c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:42.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d20b1b700 1 --2- 172.21.2.101:0/1326286210 >> v2:172.21.2.108:3300/0 conn(0x7f0d1c0d1870 0x7f0d1c1b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:42.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d20b1b700 1 -- 172.21.2.101:0/1326286210 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c0d1c70 con 0x7f0d1c1b0ae0 2021-02-10T11:45:42.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d20b1b700 1 -- 172.21.2.101:0/1326286210 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f0d180015b0 con 0x7f0d1c1b0ae0 2021-02-10T11:45:42.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d20b1b700 1 -- 172.21.2.101:0/1326286210 >> v1:172.21.2.101:6789/0 conn(0x7f0d1c1b0ae0 legacy=0x7f0d1c108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:42.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d20b1b700 1 --2- 172.21.2.101:0/1326286210 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d08003af0 0x7f0d08006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:42.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d20b1b700 1 --2- 172.21.2.101:0/1326286210 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0d08006830 0x7f0d08008d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:42.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d20b1b700 1 --2- 172.21.2.101:0/1326286210 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0d080095c0 0x7f0d0800bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:42.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d20b1b700 1 -- 172.21.2.101:0/1326286210 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d08050380 con 0x7f0d08003af0 2021-02-10T11:45:42.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d20b1b700 1 -- 172.21.2.101:0/1326286210 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d080505a0 con 0x7f0d080095c0 2021-02-10T11:45:42.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d20b1b700 1 -- 172.21.2.101:0/1326286210 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d080507c0 con 0x7f0d08006830 2021-02-10T11:45:42.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d21b1d700 1 --2- 172.21.2.101:0/1326286210 >> v2:172.21.2.108:3300/0 conn(0x7f0d1c0d1870 0x7f0d1c1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:45:42.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d2131c700 1 --2- 172.21.2.101:0/1326286210 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0d08006830 0x7f0d08008d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:42.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d21b1d700 1 --2- 172.21.2.101:0/1326286210 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d08003af0 0x7f0d08006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:42.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d2231e700 1 --2- 172.21.2.101:0/1326286210 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0d080095c0 0x7f0d0800bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:42.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d2131c700 1 -- 172.21.2.101:0/1326286210 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0d080095c0 msgr2=0x7f0d0800bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:42.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d2131c700 1 --2- 172.21.2.101:0/1326286210 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0d080095c0 0x7f0d0800bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:42.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d2131c700 1 -- 172.21.2.101:0/1326286210 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d08003af0 msgr2=0x7f0d08006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:42.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d2131c700 1 --2- 172.21.2.101:0/1326286210 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d08003af0 0x7f0d08006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:42.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d2131c700 1 -- 172.21.2.101:0/1326286210 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d08050a60 con 0x7f0d08006830 2021-02-10T11:45:42.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d21b1d700 1 --2- 172.21.2.101:0/1326286210 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d08003af0 0x7f0d08006010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:45:42.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.924 7f0d2131c700 1 --2- 172.21.2.101:0/1326286210 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0d08006830 0x7f0d08008d20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f0d18001360 tx=0x7f0d18007c70).ready entity=mon.2 client_cookie=d32f5e6449021faa server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:42.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.925 7f0d20b1b700 1 -- 172.21.2.101:0/1326286210 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0d180009b0 con 0x7f0d08006830 2021-02-10T11:45:42.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.925 7f0d20b1b700 1 -- 172.21.2.101:0/1326286210 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d18001450 con 0x7f0d08006830 2021-02-10T11:45:42.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.925 7f0d20b1b700 1 -- 172.21.2.101:0/1326286210 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0d1800a520 con 0x7f0d08006830 2021-02-10T11:45:42.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.925 7f0d23d01700 1 -- 172.21.2.101:0/1326286210 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0d08006830 msgr2=0x7f0d08008d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:42.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.925 7f0d23d01700 1 --2- 172.21.2.101:0/1326286210 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0d08006830 0x7f0d08008d20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f0d18001360 tx=0x7f0d18007c70).stop 2021-02-10T11:45:42.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.925 7f0d2231e700 1 -- 172.21.2.101:0/1326286210 reap_dead start 2021-02-10T11:45:42.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.925 7f0d23d01700 1 -- 172.21.2.101:0/1326286210 shutdown_connections 2021-02-10T11:45:42.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.925 7f0d23d01700 1 -- 172.21.2.101:0/1326286210 >> 172.21.2.101:0/1326286210 conn(0x7f0d1c0ce420 msgr2=0x7f0d1c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:42.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.925 7f0d23d01700 1 -- 172.21.2.101:0/1326286210 shutdown_connections 2021-02-10T11:45:42.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.925 7f0d23d01700 1 -- 172.21.2.101:0/1326286210 wait complete. 2021-02-10T11:45:42.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.940 7f0d23d01700 1 Processor -- start 2021-02-10T11:45:42.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.940 7f0d23d01700 1 -- start start 2021-02-10T11:45:42.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.940 7f0d23d01700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d1c1a79a0 0x7f0d1c1a7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:42.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.940 7f0d23d01700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0d1c0bdb40 0x7f0d1c0bdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:42.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.940 7f0d23d01700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0d1c082ca0 0x7f0d1c0830a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:42.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.940 7f0d23d01700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c19d990 con 0x7f0d1c1a79a0 2021-02-10T11:45:42.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.940 7f0d23d01700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c19dbb0 con 0x7f0d1c0bdb40 2021-02-10T11:45:42.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.940 7f0d23d01700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c0836c0 con 0x7f0d1c082ca0 2021-02-10T11:45:42.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.940 7f0d21b1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d1c1a79a0 0x7f0d1c1a7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:42.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.940 7f0d21b1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d1c1a79a0 0x7f0d1c1a7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60858/0 (socket says 172.21.2.101:60858) 2021-02-10T11:45:42.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.940 7f0d21b1d700 1 -- 172.21.2.101:0/1542367763 learned_addr learned my addr 172.21.2.101:0/1542367763 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:42.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.941 7f0d21b1d700 1 -- 172.21.2.101:0/1542367763 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0d1c082ca0 msgr2=0x7f0d1c0830a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:42.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.941 7f0d21b1d700 1 --2- 172.21.2.101:0/1542367763 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0d1c082ca0 0x7f0d1c0830a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:42.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.941 7f0d21b1d700 1 -- 172.21.2.101:0/1542367763 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0d1c0bdb40 msgr2=0x7f0d1c0bdf40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:42.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.941 7f0d21b1d700 1 --2- 172.21.2.101:0/1542367763 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0d1c0bdb40 0x7f0d1c0bdf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:42.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.941 7f0d21b1d700 1 -- 172.21.2.101:0/1542367763 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c0838e0 con 0x7f0d1c1a79a0 2021-02-10T11:45:42.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.941 7f0d21b1d700 1 --2- 172.21.2.101:0/1542367763 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d1c1a79a0 0x7f0d1c1a7da0 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 rx=0x7f0d1000b650 tx=0x7f0d1000b9f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:42.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.941 7f0d0e7fc700 1 -- 172.21.2.101:0/1542367763 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0d100067d0 con 0x7f0d1c1a79a0 2021-02-10T11:45:42.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.941 7f0d0e7fc700 1 -- 172.21.2.101:0/1542367763 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d10002030 con 0x7f0d1c1a79a0 2021-02-10T11:45:42.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.941 7f0d0e7fc700 1 -- 172.21.2.101:0/1542367763 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0d10002550 con 0x7f0d1c1a79a0 2021-02-10T11:45:42.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.941 7f0d23d01700 1 -- 172.21.2.101:0/1542367763 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1c0838e0 con 0x7f0d1c1a79a0 2021-02-10T11:45:42.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.941 7f0d23d01700 1 -- 172.21.2.101:0/1542367763 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d1c0838e0 con 0x7f0d1c1a79a0 2021-02-10T11:45:42.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.942 7f0d0e7fc700 1 -- 172.21.2.101:0/1542367763 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0d10000c90 con 0x7f0d1c1a79a0 2021-02-10T11:45:42.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.942 7f0d0e7fc700 1 --2- 172.21.2.101:0/1542367763 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0d040186c0 0x7f0d0401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:42.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.942 7f0d0e7fc700 1 -- 172.21.2.101:0/1542367763 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5107+0+0 (secure 0 0 0) 0x7f0d10032390 con 0x7f0d1c1a79a0 2021-02-10T11:45:42.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.942 7f0d2131c700 1 --2- 172.21.2.101:0/1542367763 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0d040186c0 0x7f0d0401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:42.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.942 7f0d2131c700 1 --2- 172.21.2.101:0/1542367763 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0d040186c0 0x7f0d0401abc0 crc :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:42.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.973 7f0d23d01700 1 -- 172.21.2.101:0/1542367763 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d1c082a70 con 0x7f0d1c1a79a0 2021-02-10T11:45:42.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:42.975 7f0d0e7fc700 1 -- 172.21.2.101:0/1542367763 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0d100067d0 con 0x7f0d1c1a79a0 2021-02-10T11:45:43.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.105 7f0d23d01700 1 -- 172.21.2.101:0/1542367763 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"} v 0) v1 -- 0x7f0d1c082a70 con 0x7f0d1c1a79a0 2021-02-10T11:45:43.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.497 7f0d0e7fc700 1 -- 172.21.2.101:0/1542367763 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"}]=0 pool 'cephfs_data' created v162) v1 ==== 125+0+0 (secure 0 0 0) 0x7f0d10000c90 con 0x7f0d1c1a79a0 2021-02-10T11:45:43.518 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' created 2021-02-10T11:45:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.522 7f0d23d01700 1 -- 172.21.2.101:0/1542367763 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0d040186c0 msgr2=0x7f0d0401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.522 7f0d23d01700 1 --2- 172.21.2.101:0/1542367763 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0d040186c0 0x7f0d0401abc0 crc :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.522 7f0d23d01700 1 -- 172.21.2.101:0/1542367763 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d1c1a79a0 msgr2=0x7f0d1c1a7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.522 7f0d23d01700 1 --2- 172.21.2.101:0/1542367763 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d1c1a79a0 0x7f0d1c1a7da0 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 rx=0x7f0d1000b650 tx=0x7f0d1000b9f0).stop 2021-02-10T11:45:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.522 7f0d23d01700 1 -- 172.21.2.101:0/1542367763 shutdown_connections 2021-02-10T11:45:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.522 7f0d23d01700 1 --2- 172.21.2.101:0/1542367763 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0d040186c0 0x7f0d0401abc0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.522 7f0d23d01700 1 --2- 172.21.2.101:0/1542367763 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0d1c082ca0 0x7f0d1c0830a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.522 7f0d23d01700 1 --2- 172.21.2.101:0/1542367763 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0d1c0bdb40 0x7f0d1c0bdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.522 7f0d23d01700 1 --2- 172.21.2.101:0/1542367763 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0d1c1a79a0 0x7f0d1c1a7da0 unknown :-1 s=CLOSED pgs=2865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.522 7f0d23d01700 1 -- 172.21.2.101:0/1542367763 >> 172.21.2.101:0/1542367763 conn(0x7f0d1c0bef00 msgr2=0x7f0d1c09f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:43.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.522 7f0d23d01700 1 -- 172.21.2.101:0/1542367763 shutdown_connections 2021-02-10T11:45:43.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.522 7f0d23d01700 1 -- 172.21.2.101:0/1542367763 wait complete. 2021-02-10T11:45:43.529 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-02-10T11:45:43.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.946 7f38eef47700 1 Processor -- start 2021-02-10T11:45:43.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.946 7f38eef47700 1 -- start start 2021-02-10T11:45:43.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.947 7f38eef47700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f38e80d1c80 0x7f38e80cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:43.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.947 7f38eef47700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38e80cf590 0x7f38e80cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:43.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.947 7f38eef47700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e80fe1d0 con 0x7f38e80d1880 2021-02-10T11:45:43.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.947 7f38eef47700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e81b0cc0 con 0x7f38e80d1c80 2021-02-10T11:45:43.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.947 7f38eef47700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e8109a70 con 0x7f38e80cf590 2021-02-10T11:45:43.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.947 7f38e7fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f38e80d1c80 0x7f38e80cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:43.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.947 7f38e7fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f38e80d1c80 0x7f38e80cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60868/0 (socket says 172.21.2.101:60868) 2021-02-10T11:45:43.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.947 7f38e7fff700 1 -- 172.21.2.101:0/894106640 learned_addr learned my addr 172.21.2.101:0/894106640 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:43.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.947 7f38ecd63700 1 --1- 172.21.2.101:0/894106640 >> v1:172.21.2.101:6789/0 conn(0x7f38e80d1880 0x7f38e81b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33130/0 (socket says 172.21.2.101:33130) 2021-02-10T11:45:43.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.947 7f38ed564700 1 --2- 172.21.2.101:0/894106640 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38e80cf590 0x7f38e80cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:43.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.947 7f38e7fff700 1 -- 172.21.2.101:0/894106640 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38e80cf590 msgr2=0x7f38e80cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:43.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.947 7f38e7fff700 1 --2- 172.21.2.101:0/894106640 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38e80cf590 0x7f38e80cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:43.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.947 7f38e7fff700 1 -- 172.21.2.101:0/894106640 >> v1:172.21.2.101:6789/0 conn(0x7f38e80d1880 legacy=0x7f38e81b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:43.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.947 7f38e7fff700 1 -- 172.21.2.101:0/894106640 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e8088070 con 0x7f38e80d1c80 2021-02-10T11:45:43.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.947 7f38e77fe700 1 -- 172.21.2.101:0/894106640 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3172533097 0 0) 0x7f38d0000ce0 con 0x7f38e80d1880 2021-02-10T11:45:43.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.947 7f38ed564700 1 --2- 172.21.2.101:0/894106640 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38e80cf590 0x7f38e80cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:43.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.947 7f38e7fff700 1 --2- 172.21.2.101:0/894106640 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38e80d1c80 0x7f38e80cef70 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 rx=0x7f38d8006700 tx=0x7f38d8009b80).ready entity=mon.0 client_cookie=519fae034e3571d6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:43.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.947 7f38e77fe700 1 -- 172.21.2.101:0/894106640 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f38d800c0c0 con 0x7f38e80d1c80 2021-02-10T11:45:43.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.947 7f38e77fe700 1 -- 172.21.2.101:0/894106640 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d800c340 con 0x7f38e80d1c80 2021-02-10T11:45:43.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.947 7f38e77fe700 1 -- 172.21.2.101:0/894106640 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f38d800c820 con 0x7f38e80d1c80 2021-02-10T11:45:43.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.947 7f38eef47700 1 -- 172.21.2.101:0/894106640 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38e80d1c80 msgr2=0x7f38e80cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:43.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.947 7f38eef47700 1 --2- 172.21.2.101:0/894106640 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38e80d1c80 0x7f38e80cef70 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 rx=0x7f38d8006700 tx=0x7f38d8009b80).stop 2021-02-10T11:45:43.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.948 7f38eef47700 1 -- 172.21.2.101:0/894106640 shutdown_connections 2021-02-10T11:45:43.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.948 7f38eef47700 1 --2- 172.21.2.101:0/894106640 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38e80cf590 0x7f38e80cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:43.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.948 7f38eef47700 1 --2- 172.21.2.101:0/894106640 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38e80d1c80 0x7f38e80cef70 unknown :-1 s=CLOSED pgs=2867 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:43.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.948 7f38eef47700 1 -- 172.21.2.101:0/894106640 >> 172.21.2.101:0/894106640 conn(0x7f38e80ce430 msgr2=0x7f38e80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:43.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.948 7f38eef47700 1 -- 172.21.2.101:0/894106640 shutdown_connections 2021-02-10T11:45:43.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.948 7f38eef47700 1 -- 172.21.2.101:0/894106640 wait complete. 2021-02-10T11:45:43.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.963 7f38eef47700 1 Processor -- start 2021-02-10T11:45:43.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.963 7f38eef47700 1 -- start start 2021-02-10T11:45:43.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.963 7f38eef47700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38e80b4be0 0x7f38e80b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:43.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.963 7f38eef47700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38e80aa520 0x7f38e80aa920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:43.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.963 7f38eef47700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38e80aaf40 0x7f38e80bf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:43.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.963 7f38eef47700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e80ab3c0 con 0x7f38e80aaf40 2021-02-10T11:45:43.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.963 7f38eef47700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e80b5640 con 0x7f38e80aa520 2021-02-10T11:45:43.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.963 7f38eef47700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e81a4a50 con 0x7f38e80b4be0 2021-02-10T11:45:43.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.963 7f38ed564700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38e80aaf40 0x7f38e80bf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:43.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.963 7f38ed564700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38e80aaf40 0x7f38e80bf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60878/0 (socket says 172.21.2.101:60878) 2021-02-10T11:45:43.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.963 7f38ed564700 1 -- 172.21.2.101:0/2706444982 learned_addr learned my addr 172.21.2.101:0/2706444982 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:43.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.964 7f38ed564700 1 -- 172.21.2.101:0/2706444982 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38e80b4be0 msgr2=0x7f38e80b4fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:43.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.964 7f38ed564700 1 --2- 172.21.2.101:0/2706444982 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38e80b4be0 0x7f38e80b4fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:43.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.964 7f38ed564700 1 -- 172.21.2.101:0/2706444982 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38e80aa520 msgr2=0x7f38e80aa920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:43.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.964 7f38ed564700 1 --2- 172.21.2.101:0/2706444982 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38e80aa520 0x7f38e80aa920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:43.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.964 7f38ed564700 1 -- 172.21.2.101:0/2706444982 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e80bfb50 con 0x7f38e80aaf40 2021-02-10T11:45:43.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.964 7f38ed564700 1 --2- 172.21.2.101:0/2706444982 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38e80aaf40 0x7f38e80bf530 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 rx=0x7f38e0009980 tx=0x7f38e000bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:43.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.964 7f38e57fa700 1 -- 172.21.2.101:0/2706444982 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f38e0006030 con 0x7f38e80aaf40 2021-02-10T11:45:43.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.964 7f38e57fa700 1 -- 172.21.2.101:0/2706444982 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e0006790 con 0x7f38e80aaf40 2021-02-10T11:45:43.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.964 7f38eef47700 1 -- 172.21.2.101:0/2706444982 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e80bfb50 con 0x7f38e80aaf40 2021-02-10T11:45:43.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.964 7f38eef47700 1 -- 172.21.2.101:0/2706444982 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e80c00e0 con 0x7f38e80aaf40 2021-02-10T11:45:43.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.964 7f38e57fa700 1 -- 172.21.2.101:0/2706444982 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f38e0006520 con 0x7f38e80aaf40 2021-02-10T11:45:43.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.965 7f38e57fa700 1 -- 172.21.2.101:0/2706444982 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f38e0006030 con 0x7f38e80aaf40 2021-02-10T11:45:43.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.965 7f38e57fa700 1 --2- 172.21.2.101:0/2706444982 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38cc0186c0 0x7f38cc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:43.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.965 7f38e57fa700 1 -- 172.21.2.101:0/2706444982 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5457+0+0 (secure 0 0 0) 0x7f38e0032850 con 0x7f38e80aaf40 2021-02-10T11:45:43.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.965 7f38ecd63700 1 --2- 172.21.2.101:0/2706444982 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38cc0186c0 0x7f38cc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:43.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.965 7f38ecd63700 1 --2- 172.21.2.101:0/2706444982 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38cc0186c0 0x7f38cc01abc0 crc :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:43.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.996 7f38eef47700 1 -- 172.21.2.101:0/2706444982 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e80ab340 con 0x7f38e80aaf40 2021-02-10T11:45:43.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:43.999 7f38e57fa700 1 -- 172.21.2.101:0/2706444982 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f38e0006690 con 0x7f38e80aaf40 2021-02-10T11:45:44.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.127 7f38eef47700 1 -- 172.21.2.101:0/2706444982 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"} v 0) v1 -- 0x7f38e80ab340 con 0x7f38e80aaf40 2021-02-10T11:45:44.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.131 7f38e57fa700 1 -- 172.21.2.101:0/2706444982 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"}]=0 new fs with metadata pool 34 and data pool 35 v136) v1 ==== 189+0+0 (secure 0 0 0) 0x7f38e0006030 con 0x7f38e80aaf40 2021-02-10T11:45:44.132 INFO:teuthology.orchestra.run.gibba001.stderr:new fs with metadata pool 34 and data pool 35 2021-02-10T11:45:44.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.136 7f38eef47700 1 -- 172.21.2.101:0/2706444982 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38cc0186c0 msgr2=0x7f38cc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:44.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.136 7f38eef47700 1 --2- 172.21.2.101:0/2706444982 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38cc0186c0 0x7f38cc01abc0 crc :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:44.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.136 7f38eef47700 1 -- 172.21.2.101:0/2706444982 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38e80aaf40 msgr2=0x7f38e80bf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:44.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.136 7f38eef47700 1 --2- 172.21.2.101:0/2706444982 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38e80aaf40 0x7f38e80bf530 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 rx=0x7f38e0009980 tx=0x7f38e000bcd0).stop 2021-02-10T11:45:44.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.136 7f38eef47700 1 -- 172.21.2.101:0/2706444982 shutdown_connections 2021-02-10T11:45:44.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.136 7f38eef47700 1 --2- 172.21.2.101:0/2706444982 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38e80aaf40 0x7f38e80bf530 unknown :-1 s=CLOSED pgs=2868 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:44.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.136 7f38eef47700 1 --2- 172.21.2.101:0/2706444982 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38e80aa520 0x7f38e80aa920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:44.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.136 7f38eef47700 1 --2- 172.21.2.101:0/2706444982 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38cc0186c0 0x7f38cc01abc0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:44.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.136 7f38eef47700 1 --2- 172.21.2.101:0/2706444982 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38e80b4be0 0x7f38e80b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:44.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.136 7f38eef47700 1 -- 172.21.2.101:0/2706444982 >> 172.21.2.101:0/2706444982 conn(0x7f38e809de50 msgr2=0x7f38e80b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:44.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.136 7f38eef47700 1 -- 172.21.2.101:0/2706444982 shutdown_connections 2021-02-10T11:45:44.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.136 7f38eef47700 1 -- 172.21.2.101:0/2706444982 wait complete. 2021-02-10T11:45:44.143 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:45:44.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.556 7f5ebe10a700 1 Processor -- start 2021-02-10T11:45:44.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.556 7f5ebe10a700 1 -- start start 2021-02-10T11:45:44.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.557 7f5ebe10a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eb80d0340 0x7f5eb80ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:44.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.557 7f5ebe10a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb80f0b10 con 0x7f5eb80d1870 2021-02-10T11:45:44.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.557 7f5ebe10a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb81b0be0 con 0x7f5eb80d1c70 2021-02-10T11:45:44.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.557 7f5ebe10a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb80e5ed0 con 0x7f5eb80d0340 2021-02-10T11:45:44.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.557 7f5eb77fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f5eb80d1870 0x7f5eb81b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33144/0 (socket says 172.21.2.101:33144) 2021-02-10T11:45:44.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.557 7f5eb77fe700 1 -- 172.21.2.101:0/4181463275 learned_addr learned my addr 172.21.2.101:0/4181463275 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:44.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.557 7f5eb6ffd700 1 --2- 172.21.2.101:0/4181463275 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eb80d0340 0x7f5eb80ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:44.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.557 7f5eb67fc700 1 -- 172.21.2.101:0/4181463275 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 84345437 0 0) 0x7f5ea00010b0 con 0x7f5eb80d1870 2021-02-10T11:45:44.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.558 7f5eb67fc700 1 -- 172.21.2.101:0/4181463275 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ea4002fe0 con 0x7f5eb80d1870 2021-02-10T11:45:44.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.558 7f5eb6ffd700 1 -- 172.21.2.101:0/4181463275 >> v1:172.21.2.108:6789/0 conn(0x7f5eb80d1c70 legacy=0x7f5eb80efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:44.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.558 7f5eb6ffd700 1 -- 172.21.2.101:0/4181463275 >> v1:172.21.2.101:6789/0 conn(0x7f5eb80d1870 legacy=0x7f5eb81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:44.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.558 7f5eb6ffd700 1 -- 172.21.2.101:0/4181463275 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb80f0450 con 0x7f5eb80d0340 2021-02-10T11:45:44.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.558 7f5eb6ffd700 1 --2- 172.21.2.101:0/4181463275 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eb80d0340 0x7f5eb80ef640 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f5ea8006650 tx=0x7f5ea8009a80).ready entity=mon.2 client_cookie=33a656cc0c80333f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:44.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.558 7f5eb67fc700 1 -- 172.21.2.101:0/4181463275 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5ea800c0c0 con 0x7f5eb80d0340 2021-02-10T11:45:44.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.558 7f5eb67fc700 1 -- 172.21.2.101:0/4181463275 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea800c340 con 0x7f5eb80d0340 2021-02-10T11:45:44.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.558 7f5ebe10a700 1 -- 172.21.2.101:0/4181463275 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eb80d0340 msgr2=0x7f5eb80ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:44.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.558 7f5ebe10a700 1 --2- 172.21.2.101:0/4181463275 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eb80d0340 0x7f5eb80ef640 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f5ea8006650 tx=0x7f5ea8009a80).stop 2021-02-10T11:45:44.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.558 7f5eb67fc700 1 -- 172.21.2.101:0/4181463275 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5ea80009b0 con 0x7f5eb80d0340 2021-02-10T11:45:44.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.559 7f5ebe10a700 1 -- 172.21.2.101:0/4181463275 shutdown_connections 2021-02-10T11:45:44.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.559 7f5ebe10a700 1 --2- 172.21.2.101:0/4181463275 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eb80d0340 0x7f5eb80ef640 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:44.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.559 7f5ebe10a700 1 -- 172.21.2.101:0/4181463275 >> 172.21.2.101:0/4181463275 conn(0x7f5eb80ce450 msgr2=0x7f5eb80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:44.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.559 7f5ebe10a700 1 -- 172.21.2.101:0/4181463275 shutdown_connections 2021-02-10T11:45:44.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.559 7f5ebe10a700 1 -- 172.21.2.101:0/4181463275 wait complete. 2021-02-10T11:45:44.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.574 7f5ebe10a700 1 Processor -- start 2021-02-10T11:45:44.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.574 7f5ebe10a700 1 -- start start 2021-02-10T11:45:44.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.574 7f5ebe10a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eb818f0b0 0x7f5eb818e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:44.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.574 7f5ebe10a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5eb818eba0 0x7f5eb81a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:44.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.574 7f5ebe10a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eb81a6c00 0x7f5eb81a7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:44.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.574 7f5ebe10a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb81a71d0 con 0x7f5eb81a6c00 2021-02-10T11:45:44.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.574 7f5ebe10a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb81a7d70 con 0x7f5eb818eba0 2021-02-10T11:45:44.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.574 7f5ebe10a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb81a7f50 con 0x7f5eb818f0b0 2021-02-10T11:45:44.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.574 7f5eb7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eb81a6c00 0x7f5eb81a7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:44.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.574 7f5eb7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eb81a6c00 0x7f5eb81a7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60890/0 (socket says 172.21.2.101:60890) 2021-02-10T11:45:44.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.574 7f5eb7fff700 1 -- 172.21.2.101:0/2283609786 learned_addr learned my addr 172.21.2.101:0/2283609786 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:44.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.575 7f5eb7fff700 1 -- 172.21.2.101:0/2283609786 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eb818f0b0 msgr2=0x7f5eb818e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:44.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.575 7f5eb7fff700 1 --2- 172.21.2.101:0/2283609786 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eb818f0b0 0x7f5eb818e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:44.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.575 7f5eb7fff700 1 -- 172.21.2.101:0/2283609786 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5eb818eba0 msgr2=0x7f5eb81a65e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:44.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.575 7f5eb7fff700 1 --2- 172.21.2.101:0/2283609786 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5eb818eba0 0x7f5eb81a65e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:44.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.575 7f5eb7fff700 1 -- 172.21.2.101:0/2283609786 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb81a8170 con 0x7f5eb81a6c00 2021-02-10T11:45:44.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.575 7f5eb7fff700 1 --2- 172.21.2.101:0/2283609786 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eb81a6c00 0x7f5eb81a7750 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 rx=0x7f5eb0009940 tx=0x7f5eb000bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:44.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.575 7f5eaffff700 1 -- 172.21.2.101:0/2283609786 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5eb0000ce0 con 0x7f5eb81a6c00 2021-02-10T11:45:44.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.575 7f5eaffff700 1 -- 172.21.2.101:0/2283609786 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb000c2f0 con 0x7f5eb81a6c00 2021-02-10T11:45:44.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.575 7f5eaffff700 1 -- 172.21.2.101:0/2283609786 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5eb000c810 con 0x7f5eb81a6c00 2021-02-10T11:45:44.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.575 7f5ebe10a700 1 -- 172.21.2.101:0/2283609786 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb81a8170 con 0x7f5eb81a6c00 2021-02-10T11:45:44.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.575 7f5ebe10a700 1 -- 172.21.2.101:0/2283609786 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb81a8170 con 0x7f5eb81a6c00 2021-02-10T11:45:44.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.576 7f5eaffff700 1 -- 172.21.2.101:0/2283609786 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5eb0001220 con 0x7f5eb81a6c00 2021-02-10T11:45:44.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.576 7f5eaffff700 1 --2- 172.21.2.101:0/2283609786 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5e9c0186d0 0x7f5e9c01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:44.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.576 7f5eaffff700 1 -- 172.21.2.101:0/2283609786 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5eb0032b20 con 0x7f5eb81a6c00 2021-02-10T11:45:44.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.576 7f5eb77fe700 1 --2- 172.21.2.101:0/2283609786 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5e9c0186d0 0x7f5e9c01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:44.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.576 7f5eb77fe700 1 --2- 172.21.2.101:0/2283609786 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5e9c0186d0 0x7f5e9c01abd0 crc :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:44.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.607 7f5ebe10a700 1 -- 172.21.2.101:0/2283609786 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb80f0b10 con 0x7f5eb81a6c00 2021-02-10T11:45:44.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.609 7f5eaffff700 1 -- 172.21.2.101:0/2283609786 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5eb000be30 con 0x7f5eb81a6c00 2021-02-10T11:45:44.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.732 7f5ebe10a700 1 -- 172.21.2.101:0/2283609786 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5eb80f0b10 con 0x7f5eb81a6c00 2021-02-10T11:45:44.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.733 7f5eaffff700 1 -- 172.21.2.101:0/2283609786 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+14020 (secure 0 0 0) 0x7f5eb001f860 con 0x7f5eb81a6c00 2021-02-10T11:45:44.732 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:45:44.732 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":163,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:45:44.128910","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:45:42.100593","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":35,"pool_name":"cephfs_data","create_time":"2021-02-10 11:45:43.124384","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:45:44.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.734 7f5ebe10a700 1 -- 172.21.2.101:0/2283609786 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5e9c0186d0 msgr2=0x7f5e9c01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:44.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.734 7f5ebe10a700 1 --2- 172.21.2.101:0/2283609786 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5e9c0186d0 0x7f5e9c01abd0 crc :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:44.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.734 7f5ebe10a700 1 -- 172.21.2.101:0/2283609786 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eb81a6c00 msgr2=0x7f5eb81a7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:44.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.734 7f5ebe10a700 1 --2- 172.21.2.101:0/2283609786 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eb81a6c00 0x7f5eb81a7750 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 rx=0x7f5eb0009940 tx=0x7f5eb000bcf0).stop 2021-02-10T11:45:44.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.735 7f5ebe10a700 1 -- 172.21.2.101:0/2283609786 shutdown_connections 2021-02-10T11:45:44.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.735 7f5ebe10a700 1 --2- 172.21.2.101:0/2283609786 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5eb81a6c00 0x7f5eb81a7750 unknown :-1 s=CLOSED pgs=2870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:44.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.735 7f5ebe10a700 1 --2- 172.21.2.101:0/2283609786 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5eb818eba0 0x7f5eb81a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:44.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.735 7f5ebe10a700 1 --2- 172.21.2.101:0/2283609786 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5e9c0186d0 0x7f5e9c01abd0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:44.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.735 7f5ebe10a700 1 --2- 172.21.2.101:0/2283609786 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5eb818f0b0 0x7f5eb818e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:44.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.735 7f5ebe10a700 1 -- 172.21.2.101:0/2283609786 >> 172.21.2.101:0/2283609786 conn(0x7f5eb81a38b0 msgr2=0x7f5eb80a3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:44.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.735 7f5ebe10a700 1 -- 172.21.2.101:0/2283609786 shutdown_connections 2021-02-10T11:45:44.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:44.735 7f5ebe10a700 1 -- 172.21.2.101:0/2283609786 wait complete. 2021-02-10T11:45:44.742 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:45:45.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.156 7fcb9223d700 1 Processor -- start 2021-02-10T11:45:45.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.156 7fcb9223d700 1 -- start start 2021-02-10T11:45:45.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.156 7fcb9223d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcb8c0d1870 0x7fcb8c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:45.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.156 7fcb9223d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb8c1b0a10 0x7fcb8c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:45.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.156 7fcb9223d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8c0f0b10 con 0x7fcb8c0d1c70 2021-02-10T11:45:45.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.156 7fcb9223d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb8c0e5ed0 con 0x7fcb8c0d1870 2021-02-10T11:45:45.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.156 7fcb9223d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c0f0650 con 0x7fcb8c1b0a10 2021-02-10T11:45:45.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.157 7fcb8bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcb8c0d1870 0x7fcb8c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:45.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.157 7fcb8bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fcb8c0d1870 0x7fcb8c1b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60896/0 (socket says 172.21.2.101:60896) 2021-02-10T11:45:45.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.157 7fcb8bfff700 1 -- 172.21.2.101:0/2421473875 learned_addr learned my addr 172.21.2.101:0/2421473875 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:45.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.157 7fcb8b7fe700 1 --2- 172.21.2.101:0/2421473875 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb8c1b0a10 0x7fcb8c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:45.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.157 7fcb8bfff700 1 -- 172.21.2.101:0/2421473875 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb8c1b0a10 msgr2=0x7fcb8c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:45.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.157 7fcb8bfff700 1 --2- 172.21.2.101:0/2421473875 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb8c1b0a10 0x7fcb8c0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:45.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.157 7fcb8bfff700 1 -- 172.21.2.101:0/2421473875 >> v1:172.21.2.108:6789/0 conn(0x7fcb8c0d1c70 legacy=0x7fcb8c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:45.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.157 7fcb8bfff700 1 -- 172.21.2.101:0/2421473875 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c0f0870 con 0x7fcb8c0d1870 2021-02-10T11:45:45.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.157 7fcb8bfff700 1 --2- 172.21.2.101:0/2421473875 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb8c0d1870 0x7fcb8c1b03f0 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 rx=0x7fcb7c006700 tx=0x7fcb7c009b00).ready entity=mon.0 client_cookie=8eda74e414f8d3fb server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:45.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.157 7fcb8affd700 1 -- 172.21.2.101:0/2421473875 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcb7c00c0c0 con 0x7fcb8c0d1870 2021-02-10T11:45:45.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.157 7fcb8affd700 1 -- 172.21.2.101:0/2421473875 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c00c340 con 0x7fcb8c0d1870 2021-02-10T11:45:45.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.157 7fcb8affd700 1 -- 172.21.2.101:0/2421473875 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcb7c00c820 con 0x7fcb8c0d1870 2021-02-10T11:45:45.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.157 7fcb9223d700 1 -- 172.21.2.101:0/2421473875 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb8c0d1870 msgr2=0x7fcb8c1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:45.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.157 7fcb9223d700 1 --2- 172.21.2.101:0/2421473875 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb8c0d1870 0x7fcb8c1b03f0 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 rx=0x7fcb7c006700 tx=0x7fcb7c009b00).stop 2021-02-10T11:45:45.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.158 7fcb9223d700 1 -- 172.21.2.101:0/2421473875 shutdown_connections 2021-02-10T11:45:45.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.158 7fcb9223d700 1 --2- 172.21.2.101:0/2421473875 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb8c1b0a10 0x7fcb8c0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:45.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.158 7fcb9223d700 1 --2- 172.21.2.101:0/2421473875 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb8c0d1870 0x7fcb8c1b03f0 unknown :-1 s=CLOSED pgs=2871 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:45.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.158 7fcb9223d700 1 -- 172.21.2.101:0/2421473875 >> 172.21.2.101:0/2421473875 conn(0x7fcb8c0ce450 msgr2=0x7fcb8c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:45.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.158 7fcb9223d700 1 -- 172.21.2.101:0/2421473875 shutdown_connections 2021-02-10T11:45:45.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.158 7fcb9223d700 1 -- 172.21.2.101:0/2421473875 wait complete. 2021-02-10T11:45:45.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.173 7fcb9223d700 1 Processor -- start 2021-02-10T11:45:45.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.173 7fcb9223d700 1 -- start start 2021-02-10T11:45:45.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.173 7fcb9223d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcb8c195650 0x7fcb8c0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:45.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.173 7fcb9223d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb8c0b5b80 0x7fcb8c1a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:45.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.173 7fcb9223d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb8c0acfa0 0x7fcb8c1a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:45.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.173 7fcb9223d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c0b6010 con 0x7fcb8c0b5b80 2021-02-10T11:45:45.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.173 7fcb9223d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c1a3a60 con 0x7fcb8c195650 2021-02-10T11:45:45.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.173 7fcb9223d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c1a3c40 con 0x7fcb8c0acfa0 2021-02-10T11:45:45.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.173 7fcb8b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb8c0b5b80 0x7fcb8c1a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:45.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.173 7fcb8b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb8c0b5b80 0x7fcb8c1a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60904/0 (socket says 172.21.2.101:60904) 2021-02-10T11:45:45.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.173 7fcb8b7fe700 1 -- 172.21.2.101:0/3042008220 learned_addr learned my addr 172.21.2.101:0/3042008220 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:45.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.174 7fcb8b7fe700 1 -- 172.21.2.101:0/3042008220 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb8c0acfa0 msgr2=0x7fcb8c1a3440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:45.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.174 7fcb8b7fe700 1 --2- 172.21.2.101:0/3042008220 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb8c0acfa0 0x7fcb8c1a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:45.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.174 7fcb8b7fe700 1 -- 172.21.2.101:0/3042008220 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcb8c195650 msgr2=0x7fcb8c0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:45.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.174 7fcb8b7fe700 1 --2- 172.21.2.101:0/3042008220 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcb8c195650 0x7fcb8c0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:45.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.174 7fcb8b7fe700 1 -- 172.21.2.101:0/3042008220 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c0bcae0 con 0x7fcb8c0b5b80 2021-02-10T11:45:45.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.174 7fcb8b7fe700 1 --2- 172.21.2.101:0/3042008220 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb8c0b5b80 0x7fcb8c1a2e20 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 rx=0x7fcb80006f00 tx=0x7fcb8000b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:45.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.174 7fcb88ff9700 1 -- 172.21.2.101:0/3042008220 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcb8000bd30 con 0x7fcb8c0b5b80 2021-02-10T11:45:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.174 7fcb88ff9700 1 -- 172.21.2.101:0/3042008220 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb80002030 con 0x7fcb8c0b5b80 2021-02-10T11:45:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.174 7fcb88ff9700 1 -- 172.21.2.101:0/3042008220 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcb80002550 con 0x7fcb8c0b5b80 2021-02-10T11:45:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.174 7fcb9223d700 1 -- 172.21.2.101:0/3042008220 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8c0bcae0 con 0x7fcb8c0b5b80 2021-02-10T11:45:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.174 7fcb9223d700 1 -- 172.21.2.101:0/3042008220 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8c0bd070 con 0x7fcb8c0b5b80 2021-02-10T11:45:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.175 7fcb88ff9700 1 -- 172.21.2.101:0/3042008220 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fcb80000c90 con 0x7fcb8c0b5b80 2021-02-10T11:45:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.175 7fcb88ff9700 1 --2- 172.21.2.101:0/3042008220 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcb6c0186c0 0x7fcb6c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.175 7fcb88ff9700 1 -- 172.21.2.101:0/3042008220 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fcb80032580 con 0x7fcb8c0b5b80 2021-02-10T11:45:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.175 7fcb8bfff700 1 --2- 172.21.2.101:0/3042008220 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcb6c0186c0 0x7fcb6c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:45.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.175 7fcb8bfff700 1 --2- 172.21.2.101:0/3042008220 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcb6c0186c0 0x7fcb6c01abc0 crc :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:45.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.206 7fcb9223d700 1 -- 172.21.2.101:0/3042008220 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8c0f0ab0 con 0x7fcb8c0b5b80 2021-02-10T11:45:45.207 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.209 7fcb88ff9700 1 -- 172.21.2.101:0/3042008220 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fcb800009b0 con 0x7fcb8c0b5b80 2021-02-10T11:45:45.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.335 7fcb9223d700 1 -- 172.21.2.101:0/3042008220 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb8c0f0ab0 con 0x7fcb8c0b5b80 2021-02-10T11:45:45.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.335 7fcb88ff9700 1 -- 172.21.2.101:0/3042008220 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+14007 (secure 0 0 0) 0x7fcb80000c90 con 0x7fcb8c0b5b80 2021-02-10T11:45:45.334 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:45:45.334 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":164,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:45:45.131083","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:45:42.100593","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":35,"pool_name":"cephfs_data","create_time":"2021-02-10 11:45:43.124384","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:45:45.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.337 7fcb9223d700 1 -- 172.21.2.101:0/3042008220 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcb6c0186c0 msgr2=0x7fcb6c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:45.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.337 7fcb9223d700 1 --2- 172.21.2.101:0/3042008220 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcb6c0186c0 0x7fcb6c01abc0 crc :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:45.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.337 7fcb9223d700 1 -- 172.21.2.101:0/3042008220 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb8c0b5b80 msgr2=0x7fcb8c1a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:45.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.337 7fcb9223d700 1 --2- 172.21.2.101:0/3042008220 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb8c0b5b80 0x7fcb8c1a2e20 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 rx=0x7fcb80006f00 tx=0x7fcb8000b940).stop 2021-02-10T11:45:45.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.337 7fcb9223d700 1 -- 172.21.2.101:0/3042008220 shutdown_connections 2021-02-10T11:45:45.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.337 7fcb9223d700 1 --2- 172.21.2.101:0/3042008220 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcb6c0186c0 0x7fcb6c01abc0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:45.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.337 7fcb9223d700 1 --2- 172.21.2.101:0/3042008220 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcb8c0acfa0 0x7fcb8c1a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:45.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.337 7fcb9223d700 1 --2- 172.21.2.101:0/3042008220 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcb8c0b5b80 0x7fcb8c1a2e20 unknown :-1 s=CLOSED pgs=2872 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:45.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.337 7fcb9223d700 1 --2- 172.21.2.101:0/3042008220 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcb8c195650 0x7fcb8c0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:45.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.337 7fcb9223d700 1 -- 172.21.2.101:0/3042008220 >> 172.21.2.101:0/3042008220 conn(0x7fcb8c0b3040 msgr2=0x7fcb8c0c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:45.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.337 7fcb9223d700 1 -- 172.21.2.101:0/3042008220 shutdown_connections 2021-02-10T11:45:45.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.337 7fcb9223d700 1 -- 172.21.2.101:0/3042008220 wait complete. 2021-02-10T11:45:45.344 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-02-10T11:45:45.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.762 7f1dfcb54700 1 Processor -- start 2021-02-10T11:45:45.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.762 7f1dfcb54700 1 -- start start 2021-02-10T11:45:45.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.762 7f1dfcb54700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1df80d1870 0x7f1df81b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:45.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.762 7f1dfcb54700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1df81b0ae0 0x7f1df8108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.762 7f1dfcb54700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1df80d1c70 0x7f1df8108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.762 7f1dfcb54700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df80fe1c0 con 0x7f1df80d1c70 2021-02-10T11:45:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.762 7f1dfcb54700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df8109a60 con 0x7f1df80d1870 2021-02-10T11:45:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.762 7f1dfcb54700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df8088070 con 0x7f1df81b0ae0 2021-02-10T11:45:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.762 7f1df705f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1df80d1c70 0x7f1df8108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.762 7f1df705f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1df80d1c70 0x7f1df8108f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60908/0 (socket says 172.21.2.101:60908) 2021-02-10T11:45:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.762 7f1df705f700 1 -- 172.21.2.101:0/2279026741 learned_addr learned my addr 172.21.2.101:0/2279026741 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.763 7f1df705f700 1 -- 172.21.2.101:0/2279026741 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1df81b0ae0 msgr2=0x7f1df8108940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:45:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.763 7f1df705f700 1 --2- 172.21.2.101:0/2279026741 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1df81b0ae0 0x7f1df8108940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.763 7f1df705f700 1 -- 172.21.2.101:0/2279026741 >> v2:172.21.2.108:3300/0 conn(0x7f1df80d1870 msgr2=0x7f1df81b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:45:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.763 7f1df705f700 1 --2- 172.21.2.101:0/2279026741 >> v2:172.21.2.108:3300/0 conn(0x7f1df80d1870 0x7f1df81b04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.763 7f1df705f700 1 -- 172.21.2.101:0/2279026741 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df8109580 con 0x7f1df80d1c70 2021-02-10T11:45:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.763 7f1df705f700 1 --2- 172.21.2.101:0/2279026741 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1df80d1c70 0x7f1df8108f60 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 rx=0x7f1de8006b80 tx=0x7f1de8009b90).ready entity=mon.0 client_cookie=237cbd9997150292 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:45.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.763 7f1df585c700 1 -- 172.21.2.101:0/2279026741 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1de80022e0 con 0x7f1df80d1c70 2021-02-10T11:45:45.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.763 7f1df585c700 1 -- 172.21.2.101:0/2279026741 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de8002560 con 0x7f1df80d1c70 2021-02-10T11:45:45.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.763 7f1df585c700 1 -- 172.21.2.101:0/2279026741 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1de8002a80 con 0x7f1df80d1c70 2021-02-10T11:45:45.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.763 7f1dfcb54700 1 -- 172.21.2.101:0/2279026741 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1df80d1c70 msgr2=0x7f1df8108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:45.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.763 7f1dfcb54700 1 --2- 172.21.2.101:0/2279026741 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1df80d1c70 0x7f1df8108f60 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 rx=0x7f1de8006b80 tx=0x7f1de8009b90).stop 2021-02-10T11:45:45.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.764 7f1dfcb54700 1 -- 172.21.2.101:0/2279026741 shutdown_connections 2021-02-10T11:45:45.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.764 7f1dfcb54700 1 --2- 172.21.2.101:0/2279026741 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1df81b0ae0 0x7f1df8108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:45.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.764 7f1dfcb54700 1 --2- 172.21.2.101:0/2279026741 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1df80d1c70 0x7f1df8108f60 unknown :-1 s=CLOSED pgs=2873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:45.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.764 7f1dfcb54700 1 --2- 172.21.2.101:0/2279026741 >> v2:172.21.2.108:3300/0 conn(0x7f1df80d1870 0x7f1df81b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:45.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.764 7f1dfcb54700 1 -- 172.21.2.101:0/2279026741 >> 172.21.2.101:0/2279026741 conn(0x7f1df80ce420 msgr2=0x7f1df80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:45.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.764 7f1dfcb54700 1 -- 172.21.2.101:0/2279026741 shutdown_connections 2021-02-10T11:45:45.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.764 7f1dfcb54700 1 -- 172.21.2.101:0/2279026741 wait complete. 2021-02-10T11:45:45.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.778 7f1dfcb54700 1 Processor -- start 2021-02-10T11:45:45.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.779 7f1dfcb54700 1 -- start start 2021-02-10T11:45:45.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.779 7f1dfcb54700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1df80c3820 0x7f1df80a7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:45.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.779 7f1dfcb54700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1df81a44c0 0x7f1df81a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:45.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.779 7f1dfcb54700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1df80863d0 0x7f1df80867d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:45.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.779 7f1dfcb54700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df8085ed0 con 0x7f1df80863d0 2021-02-10T11:45:45.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.779 7f1dfcb54700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df80c0740 con 0x7f1df80c3820 2021-02-10T11:45:45.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.779 7f1dfcb54700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df8086df0 con 0x7f1df81a44c0 2021-02-10T11:45:45.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.779 7f1df705f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1df80863d0 0x7f1df80867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:45.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.779 7f1df705f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1df80863d0 0x7f1df80867d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60916/0 (socket says 172.21.2.101:60916) 2021-02-10T11:45:45.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.779 7f1df705f700 1 -- 172.21.2.101:0/3415189868 learned_addr learned my addr 172.21.2.101:0/3415189868 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:45.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.780 7f1df705f700 1 -- 172.21.2.101:0/3415189868 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1df81a44c0 msgr2=0x7f1df81a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:45.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.780 7f1df705f700 1 --2- 172.21.2.101:0/3415189868 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1df81a44c0 0x7f1df81a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:45.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.780 7f1df705f700 1 -- 172.21.2.101:0/3415189868 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1df80c3820 msgr2=0x7f1df80a7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:45.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.780 7f1df705f700 1 --2- 172.21.2.101:0/3415189868 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1df80c3820 0x7f1df80a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:45.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.780 7f1df705f700 1 -- 172.21.2.101:0/3415189868 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df8087010 con 0x7f1df80863d0 2021-02-10T11:45:45.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.780 7f1df705f700 1 --2- 172.21.2.101:0/3415189868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1df80863d0 0x7f1df80867d0 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 rx=0x7f1de8006f10 tx=0x7f1de800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:45.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.780 7f1de77fe700 1 -- 172.21.2.101:0/3415189868 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1de8000b90 con 0x7f1df80863d0 2021-02-10T11:45:45.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.780 7f1de77fe700 1 -- 172.21.2.101:0/3415189868 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de800bd50 con 0x7f1df80863d0 2021-02-10T11:45:45.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.780 7f1de77fe700 1 -- 172.21.2.101:0/3415189868 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1de8002220 con 0x7f1df80863d0 2021-02-10T11:45:45.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.780 7f1dfcb54700 1 -- 172.21.2.101:0/3415189868 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df8087010 con 0x7f1df80863d0 2021-02-10T11:45:45.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.780 7f1dfcb54700 1 -- 172.21.2.101:0/3415189868 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df8083000 con 0x7f1df80863d0 2021-02-10T11:45:45.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.781 7f1de77fe700 1 -- 172.21.2.101:0/3415189868 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1de80054a0 con 0x7f1df80863d0 2021-02-10T11:45:45.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.781 7f1de77fe700 1 --2- 172.21.2.101:0/3415189868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1ddc0186c0 0x7f1ddc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:45.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.781 7f1de77fe700 1 -- 172.21.2.101:0/3415189868 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1de8032430 con 0x7f1df80863d0 2021-02-10T11:45:45.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.781 7f1df685e700 1 --2- 172.21.2.101:0/3415189868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1ddc0186c0 0x7f1ddc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:45.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.781 7f1df685e700 1 --2- 172.21.2.101:0/3415189868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1ddc0186c0 0x7f1ddc01abc0 crc :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:45.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.812 7f1dfcb54700 1 -- 172.21.2.101:0/3415189868 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df8085e50 con 0x7f1df80863d0 2021-02-10T11:45:45.813 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.815 7f1de77fe700 1 -- 172.21.2.101:0/3415189868 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1de80090e0 con 0x7f1df80863d0 2021-02-10T11:45:45.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:45.952 7f1dfcb54700 1 -- 172.21.2.101:0/3415189868 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"} v 0) v1 -- 0x7f1df80fe1c0 con 0x7f1df80863d0 2021-02-10T11:45:46.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.141 7f1de77fe700 1 -- 172.21.2.101:0/3415189868 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"}]=0 v139) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1de80054a0 con 0x7f1df80863d0 2021-02-10T11:45:46.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.170 7f1dfcb54700 1 -- 172.21.2.101:0/3415189868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1ddc0186c0 msgr2=0x7f1ddc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:46.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.170 7f1dfcb54700 1 --2- 172.21.2.101:0/3415189868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1ddc0186c0 0x7f1ddc01abc0 crc :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:46.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.170 7f1dfcb54700 1 -- 172.21.2.101:0/3415189868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1df80863d0 msgr2=0x7f1df80867d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:46.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.170 7f1dfcb54700 1 --2- 172.21.2.101:0/3415189868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1df80863d0 0x7f1df80867d0 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 rx=0x7f1de8006f10 tx=0x7f1de800b960).stop 2021-02-10T11:45:46.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.170 7f1dfcb54700 1 -- 172.21.2.101:0/3415189868 shutdown_connections 2021-02-10T11:45:46.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.170 7f1dfcb54700 1 --2- 172.21.2.101:0/3415189868 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1ddc0186c0 0x7f1ddc01abc0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:46.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.170 7f1dfcb54700 1 --2- 172.21.2.101:0/3415189868 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1df81a44c0 0x7f1df81a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:46.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.170 7f1dfcb54700 1 --2- 172.21.2.101:0/3415189868 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1df80863d0 0x7f1df80867d0 unknown :-1 s=CLOSED pgs=2874 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:46.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.170 7f1dfcb54700 1 --2- 172.21.2.101:0/3415189868 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1df80c3820 0x7f1df80a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:46.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.170 7f1dfcb54700 1 -- 172.21.2.101:0/3415189868 >> 172.21.2.101:0/3415189868 conn(0x7f1df80872c0 msgr2=0x7f1df8095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:46.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.170 7f1dfcb54700 1 -- 172.21.2.101:0/3415189868 shutdown_connections 2021-02-10T11:45:46.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.170 7f1dfcb54700 1 -- 172.21.2.101:0/3415189868 wait complete. 2021-02-10T11:45:46.176 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:45:46.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.591 7f451ec74700 1 Processor -- start 2021-02-10T11:45:46.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.591 7f451ec74700 1 -- start start 2021-02-10T11:45:46.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.591 7f451ec74700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f45180d1870 0x7f45181b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:46.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.591 7f451ec74700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45180f0650 0x7f45180cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:46.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.591 7f451ec74700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45180f0b10 con 0x7f45181b0a10 2021-02-10T11:45:46.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.591 7f451ec74700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45180e5ed0 con 0x7f45180d1870 2021-02-10T11:45:46.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.591 7f451ec74700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45180d1c70 con 0x7f45180f0650 2021-02-10T11:45:46.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.592 7f4517fff700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f45181b0a10 0x7f45180cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:54826/0 (socket says 172.21.2.101:54826) 2021-02-10T11:45:46.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.592 7f451d291700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45180f0650 0x7f45180cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:46.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.592 7f4517fff700 1 -- 172.21.2.101:0/3963359581 learned_addr learned my addr 172.21.2.101:0/3963359581 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:46.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.592 7f451ca90700 1 --2- 172.21.2.101:0/3963359581 >> v2:172.21.2.108:3300/0 conn(0x7f45180d1870 0x7f45181b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:46.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.593 7f451ca90700 1 -- 172.21.2.101:0/3963359581 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45180f0650 msgr2=0x7f45180cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:46.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.593 7f451ca90700 1 --2- 172.21.2.101:0/3963359581 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45180f0650 0x7f45180cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:46.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.593 7f451ca90700 1 -- 172.21.2.101:0/3963359581 >> v1:172.21.2.108:6789/0 conn(0x7f45181b0a10 legacy=0x7f45180cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:46.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.593 7f45177fe700 1 -- 172.21.2.101:0/3963359581 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597485954 0 0) 0x7f4508001190 con 0x7f45181b0a10 2021-02-10T11:45:46.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.593 7f451ca90700 1 -- 172.21.2.101:0/3963359581 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45180d1e90 con 0x7f45180d1870 2021-02-10T11:45:46.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.593 7f451d291700 1 --2- 172.21.2.101:0/3963359581 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45180f0650 0x7f45180cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:45:46.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.593 7f451ca90700 1 --2- 172.21.2.101:0/3963359581 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f45180d1870 0x7f45181b03f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f4500006650 tx=0x7f4500009a80).ready entity=mon.1 client_cookie=9c41eab7662c2b5e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:46.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.594 7f45177fe700 1 -- 172.21.2.101:0/3963359581 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f45000009b0 con 0x7f45180d1870 2021-02-10T11:45:46.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.594 7f45177fe700 1 -- 172.21.2.101:0/3963359581 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450000c0c0 con 0x7f45180d1870 2021-02-10T11:45:46.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.594 7f45177fe700 1 -- 172.21.2.101:0/3963359581 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f450000c5e0 con 0x7f45180d1870 2021-02-10T11:45:46.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.594 7f451ec74700 1 -- 172.21.2.101:0/3963359581 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f45180d1870 msgr2=0x7f45181b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:46.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.594 7f451ec74700 1 --2- 172.21.2.101:0/3963359581 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f45180d1870 0x7f45181b03f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f4500006650 tx=0x7f4500009a80).stop 2021-02-10T11:45:46.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.594 7f451ec74700 1 -- 172.21.2.101:0/3963359581 shutdown_connections 2021-02-10T11:45:46.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.594 7f451ec74700 1 --2- 172.21.2.101:0/3963359581 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45180f0650 0x7f45180cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:46.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.594 7f451ec74700 1 --2- 172.21.2.101:0/3963359581 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f45180d1870 0x7f45181b03f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:46.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.594 7f451ec74700 1 -- 172.21.2.101:0/3963359581 >> 172.21.2.101:0/3963359581 conn(0x7f45180ce450 msgr2=0x7f45180d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:46.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.594 7f451ec74700 1 -- 172.21.2.101:0/3963359581 shutdown_connections 2021-02-10T11:45:46.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.594 7f451ec74700 1 -- 172.21.2.101:0/3963359581 wait complete. 2021-02-10T11:45:46.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.609 7f451ec74700 1 Processor -- start 2021-02-10T11:45:46.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.609 7f451ec74700 1 -- start start 2021-02-10T11:45:46.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.609 7f451ec74700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45181a4a60 0x7f45180c1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:46.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.609 7f451ec74700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f45180c2290 0x7f45180c2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:46.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.609 7f451ec74700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45180c2cb0 0x7f45180bfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:46.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.609 7f451ec74700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45180f05b0 con 0x7f45181a4a60 2021-02-10T11:45:46.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.609 7f451ec74700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45180f07d0 con 0x7f45180c2290 2021-02-10T11:45:46.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.609 7f451ec74700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45180c0340 con 0x7f45180c2cb0 2021-02-10T11:45:46.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.610 7f451d291700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45180c2cb0 0x7f45180bfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:46.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.610 7f451d291700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45180c2cb0 0x7f45180bfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:53658/0 (socket says 172.21.2.101:53658) 2021-02-10T11:45:46.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.610 7f451d291700 1 -- 172.21.2.101:0/2385965713 learned_addr learned my addr 172.21.2.101:0/2385965713 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:46.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.610 7f451ca90700 1 --2- 172.21.2.101:0/2385965713 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45181a4a60 0x7f45180c1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:46.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.610 7f4517fff700 1 --2- 172.21.2.101:0/2385965713 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f45180c2290 0x7f45180c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:46.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.610 7f451d291700 1 -- 172.21.2.101:0/2385965713 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f45180c2290 msgr2=0x7f45180c2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:46.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.610 7f451d291700 1 --2- 172.21.2.101:0/2385965713 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f45180c2290 0x7f45180c2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:46.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.610 7f451d291700 1 -- 172.21.2.101:0/2385965713 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45181a4a60 msgr2=0x7f45180c1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:46.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.610 7f451d291700 1 --2- 172.21.2.101:0/2385965713 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45181a4a60 0x7f45180c1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:46.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.610 7f451d291700 1 -- 172.21.2.101:0/2385965713 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45180c0560 con 0x7f45180c2cb0 2021-02-10T11:45:46.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.610 7f451d291700 1 --2- 172.21.2.101:0/2385965713 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45180c2cb0 0x7f45180bfd20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f451000b970 tx=0x7f451000bd10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:46.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.610 7f45157fa700 1 -- 172.21.2.101:0/2385965713 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4510000f90 con 0x7f45180c2cb0 2021-02-10T11:45:46.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.610 7f45157fa700 1 -- 172.21.2.101:0/2385965713 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4510006790 con 0x7f45180c2cb0 2021-02-10T11:45:46.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.610 7f451ec74700 1 -- 172.21.2.101:0/2385965713 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45180c0560 con 0x7f45180c2cb0 2021-02-10T11:45:46.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.610 7f45157fa700 1 -- 172.21.2.101:0/2385965713 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f45100015f0 con 0x7f45180c2cb0 2021-02-10T11:45:46.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.610 7f451ec74700 1 -- 172.21.2.101:0/2385965713 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45180c0560 con 0x7f45180c2cb0 2021-02-10T11:45:46.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.611 7f45157fa700 1 -- 172.21.2.101:0/2385965713 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f45100011f0 con 0x7f45180c2cb0 2021-02-10T11:45:46.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.611 7f45157fa700 1 --2- 172.21.2.101:0/2385965713 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44fc0186d0 0x7f44fc01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:46.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.611 7f45157fa700 1 -- 172.21.2.101:0/2385965713 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4510032b20 con 0x7f45180c2cb0 2021-02-10T11:45:46.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.611 7f451ca90700 1 --2- 172.21.2.101:0/2385965713 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44fc0186d0 0x7f44fc01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:46.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.611 7f451ca90700 1 --2- 172.21.2.101:0/2385965713 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44fc0186d0 0x7f44fc01abd0 crc :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:46.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.641 7f451ec74700 1 -- 172.21.2.101:0/2385965713 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45180f0b10 con 0x7f45180c2cb0 2021-02-10T11:45:46.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.645 7f45157fa700 1 -- 172.21.2.101:0/2385965713 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4510000df0 con 0x7f45180c2cb0 2021-02-10T11:45:46.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.774 7f451ec74700 1 -- 172.21.2.101:0/2385965713 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45180f0b10 con 0x7f45180c2cb0 2021-02-10T11:45:46.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.775 7f45157fa700 1 -- 172.21.2.101:0/2385965713 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+2104 (secure 0 0 0) 0x7f45100011f0 con 0x7f45180c2cb0 2021-02-10T11:45:46.773 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:45:46.773 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":139,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8371,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/453344112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":453344112},{"type":"v1","addr":"172.21.2.101:6835","nonce":453344112}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":135}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:45:44.128996","modified":"2021-02-10 11:45:46.139945","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7097":{"gid":7097,"name":"b","rank":0,"incarnation":136,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2456711519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2456711519},{"type":"v1","addr":"172.21.2.108:6833","nonce":2456711519}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":17}]} 2021-02-10T11:45:46.773 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 139 2021-02-10T11:45:46.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.776 7f451ec74700 1 -- 172.21.2.101:0/2385965713 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44fc0186d0 msgr2=0x7f44fc01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:46.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.776 7f451ec74700 1 --2- 172.21.2.101:0/2385965713 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44fc0186d0 0x7f44fc01abd0 crc :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:46.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.776 7f451ec74700 1 -- 172.21.2.101:0/2385965713 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45180c2cb0 msgr2=0x7f45180bfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:46.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.776 7f451ec74700 1 --2- 172.21.2.101:0/2385965713 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45180c2cb0 0x7f45180bfd20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f451000b970 tx=0x7f451000bd10).stop 2021-02-10T11:45:46.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.777 7f451ec74700 1 -- 172.21.2.101:0/2385965713 shutdown_connections 2021-02-10T11:45:46.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.777 7f451ec74700 1 --2- 172.21.2.101:0/2385965713 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44fc0186d0 0x7f44fc01abd0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:46.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.777 7f451ec74700 1 --2- 172.21.2.101:0/2385965713 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45180c2cb0 0x7f45180bfd20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:46.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.777 7f451ec74700 1 --2- 172.21.2.101:0/2385965713 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f45180c2290 0x7f45180c2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:46.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.777 7f451ec74700 1 --2- 172.21.2.101:0/2385965713 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45181a4a60 0x7f45180c1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:46.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.777 7f451ec74700 1 -- 172.21.2.101:0/2385965713 >> 172.21.2.101:0/2385965713 conn(0x7f45180ebe30 msgr2=0x7f45180bf200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:46.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.777 7f451ec74700 1 -- 172.21.2.101:0/2385965713 shutdown_connections 2021-02-10T11:45:46.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:46.777 7f451ec74700 1 -- 172.21.2.101:0/2385965713 wait complete. 2021-02-10T11:45:46.783 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:45:47.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.199 7fc372aea700 1 Processor -- start 2021-02-10T11:45:47.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.199 7fc372aea700 1 -- start start 2021-02-10T11:45:47.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.200 7fc372aea700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc36c0d0340 0x7fc36c0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:47.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.200 7fc372aea700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc36c0f0b10 con 0x7fc36c0d1870 2021-02-10T11:45:47.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.200 7fc372aea700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc36c1b0be0 con 0x7fc36c0d1c70 2021-02-10T11:45:47.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.200 7fc372aea700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c0e5ed0 con 0x7fc36c0d0340 2021-02-10T11:45:47.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.200 7fc36bfff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc36c0d0340 0x7fc36c0ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:47.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.200 7fc371107700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fc36c0d1c70 0x7fc36c0efc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:54838/0 (socket says 172.21.2.101:54838) 2021-02-10T11:45:47.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.200 7fc371107700 1 -- 172.21.2.101:0/1779162527 learned_addr learned my addr 172.21.2.101:0/1779162527 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:47.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.200 7fc370906700 1 --1- 172.21.2.101:0/1779162527 >> v1:172.21.2.101:6789/0 conn(0x7fc36c0d1870 0x7fc36c1b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33196/0 (socket says 172.21.2.101:33196) 2021-02-10T11:45:47.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.201 7fc36b7fe700 1 -- 172.21.2.101:0/1779162527 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3117542154 0 0) 0x7fc3640010c0 con 0x7fc36c0d1c70 2021-02-10T11:45:47.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.201 7fc36bfff700 1 -- 172.21.2.101:0/1779162527 >> v1:172.21.2.108:6789/0 conn(0x7fc36c0d1c70 legacy=0x7fc36c0efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:47.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.201 7fc36bfff700 1 -- 172.21.2.101:0/1779162527 >> v1:172.21.2.101:6789/0 conn(0x7fc36c0d1870 legacy=0x7fc36c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:47.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.201 7fc36bfff700 1 -- 172.21.2.101:0/1779162527 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36c0f0450 con 0x7fc36c0d0340 2021-02-10T11:45:47.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.201 7fc36bfff700 1 --2- 172.21.2.101:0/1779162527 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc36c0d0340 0x7fc36c0ef640 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7fc35c0069d0 tx=0x7fc35c009e40).ready entity=mon.2 client_cookie=e546f8b617ccc3ed server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:47.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.201 7fc36b7fe700 1 -- 172.21.2.101:0/1779162527 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc35c00b510 con 0x7fc36c0d0340 2021-02-10T11:45:47.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.201 7fc36b7fe700 1 -- 172.21.2.101:0/1779162527 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35c00b790 con 0x7fc36c0d0340 2021-02-10T11:45:47.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.201 7fc36b7fe700 1 -- 172.21.2.101:0/1779162527 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc35c00bcb0 con 0x7fc36c0d0340 2021-02-10T11:45:47.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.201 7fc372aea700 1 -- 172.21.2.101:0/1779162527 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc36c0d0340 msgr2=0x7fc36c0ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:47.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.201 7fc372aea700 1 --2- 172.21.2.101:0/1779162527 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc36c0d0340 0x7fc36c0ef640 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7fc35c0069d0 tx=0x7fc35c009e40).stop 2021-02-10T11:45:47.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.202 7fc372aea700 1 -- 172.21.2.101:0/1779162527 shutdown_connections 2021-02-10T11:45:47.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.202 7fc372aea700 1 --2- 172.21.2.101:0/1779162527 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc36c0d0340 0x7fc36c0ef640 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:47.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.202 7fc372aea700 1 -- 172.21.2.101:0/1779162527 >> 172.21.2.101:0/1779162527 conn(0x7fc36c0ce450 msgr2=0x7fc36c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:47.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.202 7fc372aea700 1 -- 172.21.2.101:0/1779162527 shutdown_connections 2021-02-10T11:45:47.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.202 7fc372aea700 1 -- 172.21.2.101:0/1779162527 wait complete. 2021-02-10T11:45:47.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.217 7fc372aea700 1 Processor -- start 2021-02-10T11:45:47.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.217 7fc372aea700 1 -- start start 2021-02-10T11:45:47.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.217 7fc372aea700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc36c1a8cd0 0x7fc36c093740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:47.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.217 7fc372aea700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc36c18ecc0 0x7fc36c1949e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:47.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.217 7fc372aea700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc36c18f0c0 0x7fc36c195000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:47.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.217 7fc372aea700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c093f60 con 0x7fc36c1a8cd0 2021-02-10T11:45:47.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.217 7fc372aea700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c1a8830 con 0x7fc36c18f0c0 2021-02-10T11:45:47.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.217 7fc372aea700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c1a8a10 con 0x7fc36c18ecc0 2021-02-10T11:45:47.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.217 7fc370906700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc36c1a8cd0 0x7fc36c093740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:47.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.217 7fc370906700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc36c1a8cd0 0x7fc36c093740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60938/0 (socket says 172.21.2.101:60938) 2021-02-10T11:45:47.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.217 7fc370906700 1 -- 172.21.2.101:0/395885469 learned_addr learned my addr 172.21.2.101:0/395885469 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:47.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.218 7fc36bfff700 1 --2- 172.21.2.101:0/395885469 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc36c18ecc0 0x7fc36c1949e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:47.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.218 7fc371107700 1 --2- 172.21.2.101:0/395885469 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc36c18f0c0 0x7fc36c195000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:47.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.218 7fc370906700 1 -- 172.21.2.101:0/395885469 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc36c18ecc0 msgr2=0x7fc36c1949e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:47.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.218 7fc370906700 1 --2- 172.21.2.101:0/395885469 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc36c18ecc0 0x7fc36c1949e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:47.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.218 7fc370906700 1 -- 172.21.2.101:0/395885469 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc36c18f0c0 msgr2=0x7fc36c195000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:47.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.218 7fc370906700 1 --2- 172.21.2.101:0/395885469 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc36c18f0c0 0x7fc36c195000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:47.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.218 7fc370906700 1 -- 172.21.2.101:0/395885469 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36c1a90d0 con 0x7fc36c1a8cd0 2021-02-10T11:45:47.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.218 7fc36bfff700 1 --2- 172.21.2.101:0/395885469 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc36c18ecc0 0x7fc36c1949e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:47.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.218 7fc371107700 1 --2- 172.21.2.101:0/395885469 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc36c18f0c0 0x7fc36c195000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:47.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.218 7fc370906700 1 --2- 172.21.2.101:0/395885469 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc36c1a8cd0 0x7fc36c093740 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 rx=0x7fc35400a590 tx=0x7fc35400a930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:47.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.218 7fc3697fa700 1 -- 172.21.2.101:0/395885469 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc354000ef0 con 0x7fc36c1a8cd0 2021-02-10T11:45:47.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.218 7fc3697fa700 1 -- 172.21.2.101:0/395885469 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35400ad20 con 0x7fc36c1a8cd0 2021-02-10T11:45:47.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.218 7fc3697fa700 1 -- 172.21.2.101:0/395885469 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc354004620 con 0x7fc36c1a8cd0 2021-02-10T11:45:47.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.218 7fc372aea700 1 -- 172.21.2.101:0/395885469 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36c1a90d0 con 0x7fc36c1a8cd0 2021-02-10T11:45:47.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.218 7fc372aea700 1 -- 172.21.2.101:0/395885469 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36c195910 con 0x7fc36c1a8cd0 2021-02-10T11:45:47.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.219 7fc3697fa700 1 -- 172.21.2.101:0/395885469 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc3540043a0 con 0x7fc36c1a8cd0 2021-02-10T11:45:47.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.219 7fc3697fa700 1 --2- 172.21.2.101:0/395885469 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3500186c0 0x7fc35001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:47.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.219 7fc3697fa700 1 -- 172.21.2.101:0/395885469 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc3540323c0 con 0x7fc36c1a8cd0 2021-02-10T11:45:47.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.219 7fc36bfff700 1 --2- 172.21.2.101:0/395885469 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3500186c0 0x7fc35001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:47.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.219 7fc36bfff700 1 --2- 172.21.2.101:0/395885469 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3500186c0 0x7fc35001abc0 crc :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:47.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.250 7fc372aea700 1 -- 172.21.2.101:0/395885469 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc36c195910 con 0x7fc36c1a8cd0 2021-02-10T11:45:47.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.252 7fc3697fa700 1 -- 172.21.2.101:0/395885469 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc354000ef0 con 0x7fc36c1a8cd0 2021-02-10T11:45:47.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.377 7fc372aea700 1 -- 172.21.2.101:0/395885469 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc36c195910 con 0x7fc36c1a8cd0 2021-02-10T11:45:47.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.378 7fc3697fa700 1 -- 172.21.2.101:0/395885469 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+14007 (secure 0 0 0) 0x7fc3540043a0 con 0x7fc36c1a8cd0 2021-02-10T11:45:47.377 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:45:47.377 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":164,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:45:45.131083","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:45:42.100593","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":35,"pool_name":"cephfs_data","create_time":"2021-02-10 11:45:43.124384","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:45:47.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.379 7fc372aea700 1 -- 172.21.2.101:0/395885469 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3500186c0 msgr2=0x7fc35001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:47.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.379 7fc372aea700 1 --2- 172.21.2.101:0/395885469 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3500186c0 0x7fc35001abc0 crc :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:47.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.379 7fc372aea700 1 -- 172.21.2.101:0/395885469 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc36c1a8cd0 msgr2=0x7fc36c093740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:47.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.379 7fc372aea700 1 --2- 172.21.2.101:0/395885469 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc36c1a8cd0 0x7fc36c093740 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 rx=0x7fc35400a590 tx=0x7fc35400a930).stop 2021-02-10T11:45:47.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.379 7fc372aea700 1 -- 172.21.2.101:0/395885469 shutdown_connections 2021-02-10T11:45:47.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.379 7fc372aea700 1 --2- 172.21.2.101:0/395885469 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc3500186c0 0x7fc35001abc0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:47.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.379 7fc372aea700 1 --2- 172.21.2.101:0/395885469 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc36c18ecc0 0x7fc36c1949e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:47.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.379 7fc372aea700 1 --2- 172.21.2.101:0/395885469 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc36c18f0c0 0x7fc36c195000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:47.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.379 7fc372aea700 1 --2- 172.21.2.101:0/395885469 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc36c1a8cd0 0x7fc36c093740 unknown :-1 s=CLOSED pgs=2876 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:47.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.379 7fc372aea700 1 -- 172.21.2.101:0/395885469 >> 172.21.2.101:0/395885469 conn(0x7fc36c1a38b0 msgr2=0x7fc36c1a2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:47.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.380 7fc372aea700 1 -- 172.21.2.101:0/395885469 shutdown_connections 2021-02-10T11:45:47.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.380 7fc372aea700 1 -- 172.21.2.101:0/395885469 wait complete. 2021-02-10T11:45:47.386 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:45:47.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.813 7fae10a6a700 1 Processor -- start 2021-02-10T11:45:47.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.813 7fae10a6a700 1 -- start start 2021-02-10T11:45:47.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.813 7fae10a6a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fae0c0d1880 0x7fae0c1b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:47.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.813 7fae10a6a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fae0c1b0af0 0x7fae0c108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:47.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.813 7fae10a6a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0c0fe1d0 con 0x7fae0c0d1c80 2021-02-10T11:45:47.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.813 7fae10a6a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0c109a70 con 0x7fae0c0d1880 2021-02-10T11:45:47.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.813 7fae10a6a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0c109760 con 0x7fae0c1b0af0 2021-02-10T11:45:47.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.814 7fae0a85e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fae0c0d1880 0x7fae0c1b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:47.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.814 7fae0a85e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fae0c0d1880 0x7fae0c1b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60946/0 (socket says 172.21.2.101:60946) 2021-02-10T11:45:47.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.814 7fae0a85e700 1 -- 172.21.2.101:0/360629735 learned_addr learned my addr 172.21.2.101:0/360629735 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:47.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.814 7fae0a85e700 1 -- 172.21.2.101:0/360629735 >> v2:172.21.2.108:3300/0 conn(0x7fae0c1b0af0 msgr2=0x7fae0c108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:47.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.814 7fae0a85e700 1 --2- 172.21.2.101:0/360629735 >> v2:172.21.2.108:3300/0 conn(0x7fae0c1b0af0 0x7fae0c108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:47.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.814 7fae0a85e700 1 -- 172.21.2.101:0/360629735 >> v1:172.21.2.108:6789/0 conn(0x7fae0c0d1c80 legacy=0x7fae0c108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:47.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.814 7fae0a85e700 1 -- 172.21.2.101:0/360629735 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0c088070 con 0x7fae0c0d1880 2021-02-10T11:45:47.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.814 7fae0a85e700 1 --2- 172.21.2.101:0/360629735 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae0c0d1880 0x7fae0c1b04d0 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 rx=0x7fadf4006700 tx=0x7fadf4009b80).ready entity=mon.0 client_cookie=caefb030d075202b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:47.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.814 7fae0985c700 1 -- 172.21.2.101:0/360629735 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fadf4000c90 con 0x7fae0c0d1880 2021-02-10T11:45:47.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.814 7fae0985c700 1 -- 172.21.2.101:0/360629735 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf400c0c0 con 0x7fae0c0d1880 2021-02-10T11:45:47.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.814 7fae0985c700 1 -- 172.21.2.101:0/360629735 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fadf400c5a0 con 0x7fae0c0d1880 2021-02-10T11:45:47.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.815 7fae10a6a700 1 -- 172.21.2.101:0/360629735 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae0c0d1880 msgr2=0x7fae0c1b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:47.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.815 7fae10a6a700 1 --2- 172.21.2.101:0/360629735 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae0c0d1880 0x7fae0c1b04d0 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 rx=0x7fadf4006700 tx=0x7fadf4009b80).stop 2021-02-10T11:45:47.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.815 7fae10a6a700 1 -- 172.21.2.101:0/360629735 shutdown_connections 2021-02-10T11:45:47.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.815 7fae10a6a700 1 --2- 172.21.2.101:0/360629735 >> v2:172.21.2.108:3300/0 conn(0x7fae0c1b0af0 0x7fae0c108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:47.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.815 7fae10a6a700 1 --2- 172.21.2.101:0/360629735 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae0c0d1880 0x7fae0c1b04d0 unknown :-1 s=CLOSED pgs=2877 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:47.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.815 7fae10a6a700 1 -- 172.21.2.101:0/360629735 >> 172.21.2.101:0/360629735 conn(0x7fae0c0ce430 msgr2=0x7fae0c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:47.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.815 7fae10a6a700 1 -- 172.21.2.101:0/360629735 shutdown_connections 2021-02-10T11:45:47.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.815 7fae10a6a700 1 -- 172.21.2.101:0/360629735 wait complete. 2021-02-10T11:45:47.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.830 7fae10a6a700 1 Processor -- start 2021-02-10T11:45:47.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.830 7fae10a6a700 1 -- start start 2021-02-10T11:45:47.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.830 7fae10a6a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fae0c086810 0x7fae0c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:47.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.830 7fae10a6a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae0c0b04d0 0x7fae0c097450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:47.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.830 7fae10a6a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae0c0b08d0 0x7fae0c097a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:47.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.830 7fae10a6a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c1a02c0 con 0x7fae0c0b08d0 2021-02-10T11:45:47.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.830 7fae10a6a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c098090 con 0x7fae0c086810 2021-02-10T11:45:47.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.830 7fae10a6a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c0982b0 con 0x7fae0c0b04d0 2021-02-10T11:45:47.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.830 7fae0b05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae0c0b08d0 0x7fae0c097a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:47.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.831 7fae0b05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae0c0b08d0 0x7fae0c097a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60956/0 (socket says 172.21.2.101:60956) 2021-02-10T11:45:47.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.831 7fae0b05f700 1 -- 172.21.2.101:0/2807483876 learned_addr learned my addr 172.21.2.101:0/2807483876 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:47.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.831 7fae0b05f700 1 -- 172.21.2.101:0/2807483876 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae0c0b04d0 msgr2=0x7fae0c097450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:47.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.831 7fae0b05f700 1 --2- 172.21.2.101:0/2807483876 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae0c0b04d0 0x7fae0c097450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:47.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.831 7fae0b05f700 1 -- 172.21.2.101:0/2807483876 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fae0c086810 msgr2=0x7fae0c19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:47.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.831 7fae0b05f700 1 --2- 172.21.2.101:0/2807483876 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fae0c086810 0x7fae0c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:47.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.831 7fae0b05f700 1 -- 172.21.2.101:0/2807483876 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0c0984d0 con 0x7fae0c0b08d0 2021-02-10T11:45:47.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.831 7fae0b05f700 1 --2- 172.21.2.101:0/2807483876 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae0c0b08d0 0x7fae0c097a70 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 rx=0x7fae04009940 tx=0x7fae0400bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:47.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.831 7fae037fe700 1 -- 172.21.2.101:0/2807483876 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fae04000f90 con 0x7fae0c0b08d0 2021-02-10T11:45:47.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.831 7fae037fe700 1 -- 172.21.2.101:0/2807483876 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0400c2f0 con 0x7fae0c0b08d0 2021-02-10T11:45:47.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.831 7fae10a6a700 1 -- 172.21.2.101:0/2807483876 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0c0984d0 con 0x7fae0c0b08d0 2021-02-10T11:45:47.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.831 7fae037fe700 1 -- 172.21.2.101:0/2807483876 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fae0400c750 con 0x7fae0c0b08d0 2021-02-10T11:45:47.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.831 7fae10a6a700 1 -- 172.21.2.101:0/2807483876 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0c098b80 con 0x7fae0c0b08d0 2021-02-10T11:45:47.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.832 7fae037fe700 1 -- 172.21.2.101:0/2807483876 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fae04001220 con 0x7fae0c0b08d0 2021-02-10T11:45:47.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.832 7fae037fe700 1 --2- 172.21.2.101:0/2807483876 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fadf0018730 0x7fadf001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:47.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.832 7fae037fe700 1 -- 172.21.2.101:0/2807483876 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fae040329a0 con 0x7fae0c0b08d0 2021-02-10T11:45:47.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.832 7fae0a85e700 1 --2- 172.21.2.101:0/2807483876 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fadf0018730 0x7fadf001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:47.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.832 7fae0a85e700 1 --2- 172.21.2.101:0/2807483876 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fadf0018730 0x7fadf001abc0 crc :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:47.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.863 7fae10a6a700 1 -- 172.21.2.101:0/2807483876 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0c0fe1d0 con 0x7fae0c0b08d0 2021-02-10T11:45:47.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.866 7fae037fe700 1 -- 172.21.2.101:0/2807483876 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fae04000f40 con 0x7fae0c0b08d0 2021-02-10T11:45:47.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:47.997 7fae10a6a700 1 -- 172.21.2.101:0/2807483876 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fae0c0fe1d0 con 0x7fae0c0b08d0 2021-02-10T11:45:47.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.000 7fae037fe700 1 -- 172.21.2.101:0/2807483876 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v91) v1 ==== 184+0+0 (secure 0 0 0) 0x7fae0401fa20 con 0x7fae0c0b08d0 2021-02-10T11:45:47.998 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.0 2021-02-10T11:45:48.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.001 7fae10a6a700 1 -- 172.21.2.101:0/2807483876 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fadf0018730 msgr2=0x7fadf001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:48.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.001 7fae10a6a700 1 --2- 172.21.2.101:0/2807483876 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fadf0018730 0x7fadf001abc0 crc :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:48.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.001 7fae10a6a700 1 -- 172.21.2.101:0/2807483876 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae0c0b08d0 msgr2=0x7fae0c097a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:48.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.001 7fae10a6a700 1 --2- 172.21.2.101:0/2807483876 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae0c0b08d0 0x7fae0c097a70 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 rx=0x7fae04009940 tx=0x7fae0400bcf0).stop 2021-02-10T11:45:48.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.001 7fae10a6a700 1 -- 172.21.2.101:0/2807483876 shutdown_connections 2021-02-10T11:45:48.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.001 7fae10a6a700 1 --2- 172.21.2.101:0/2807483876 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fadf0018730 0x7fadf001abc0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:48.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.001 7fae10a6a700 1 --2- 172.21.2.101:0/2807483876 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fae0c0b04d0 0x7fae0c097450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:48.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.001 7fae10a6a700 1 --2- 172.21.2.101:0/2807483876 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fae0c0b08d0 0x7fae0c097a70 unknown :-1 s=CLOSED pgs=2878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:48.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.001 7fae10a6a700 1 --2- 172.21.2.101:0/2807483876 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fae0c086810 0x7fae0c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:48.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.002 7fae10a6a700 1 -- 172.21.2.101:0/2807483876 >> 172.21.2.101:0/2807483876 conn(0x7fae0c19f880 msgr2=0x7fae0c08f890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:48.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.002 7fae10a6a700 1 -- 172.21.2.101:0/2807483876 shutdown_connections 2021-02-10T11:45:48.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.002 7fae10a6a700 1 -- 172.21.2.101:0/2807483876 wait complete. 2021-02-10T11:45:48.008 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:45:48.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.435 7fab477b2700 1 Processor -- start 2021-02-10T11:45:48.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.435 7fab477b2700 1 -- start start 2021-02-10T11:45:48.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.435 7fab477b2700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fab400d1880 0x7fab401b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:48.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.435 7fab477b2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab400cff80 0x7fab40109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:48.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.435 7fab477b2700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab40109a70 con 0x7fab401b0af0 2021-02-10T11:45:48.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.435 7fab477b2700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab40109760 con 0x7fab400d1880 2021-02-10T11:45:48.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.435 7fab477b2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab40088070 con 0x7fab400cff80 2021-02-10T11:45:48.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.436 7fab44dcd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fab401b0af0 0x7fab40108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33222/0 (socket says 172.21.2.101:33222) 2021-02-10T11:45:48.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.436 7fab44dcd700 1 -- 172.21.2.101:0/2181536965 learned_addr learned my addr 172.21.2.101:0/2181536965 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:48.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.436 7fab45dcf700 1 --2- 172.21.2.101:0/2181536965 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab400cff80 0x7fab40109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:48.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.436 7fab455ce700 1 --2- 172.21.2.101:0/2181536965 >> v2:172.21.2.108:3300/0 conn(0x7fab400d1880 0x7fab401b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:48.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.436 7fab33fff700 1 -- 172.21.2.101:0/2181536965 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3782743641 0 0) 0x7fab3c0010b0 con 0x7fab401b0af0 2021-02-10T11:45:48.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.436 7fab33fff700 1 -- 172.21.2.101:0/2181536965 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab2c002fe0 con 0x7fab401b0af0 2021-02-10T11:45:48.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.436 7fab45dcf700 1 -- 172.21.2.101:0/2181536965 >> v2:172.21.2.108:3300/0 conn(0x7fab400d1880 msgr2=0x7fab401b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:48.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.436 7fab45dcf700 1 --2- 172.21.2.101:0/2181536965 >> v2:172.21.2.108:3300/0 conn(0x7fab400d1880 0x7fab401b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:48.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.436 7fab45dcf700 1 -- 172.21.2.101:0/2181536965 >> v1:172.21.2.101:6789/0 conn(0x7fab401b0af0 legacy=0x7fab40108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:48.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.436 7fab45dcf700 1 -- 172.21.2.101:0/2181536965 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab400d1c80 con 0x7fab400cff80 2021-02-10T11:45:48.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.437 7fab45dcf700 1 --2- 172.21.2.101:0/2181536965 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab400cff80 0x7fab40109140 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fab38006e50 tx=0x7fab38009e70).ready entity=mon.2 client_cookie=dcbc22af942589c9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:48.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.437 7fab33fff700 1 -- 172.21.2.101:0/2181536965 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fab3800b510 con 0x7fab400cff80 2021-02-10T11:45:48.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.437 7fab33fff700 1 -- 172.21.2.101:0/2181536965 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab38006790 con 0x7fab400cff80 2021-02-10T11:45:48.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.437 7fab33fff700 1 -- 172.21.2.101:0/2181536965 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fab3800b940 con 0x7fab400cff80 2021-02-10T11:45:48.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.437 7fab477b2700 1 -- 172.21.2.101:0/2181536965 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab400cff80 msgr2=0x7fab40109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:48.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.437 7fab477b2700 1 --2- 172.21.2.101:0/2181536965 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab400cff80 0x7fab40109140 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fab38006e50 tx=0x7fab38009e70).stop 2021-02-10T11:45:48.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.437 7fab477b2700 1 -- 172.21.2.101:0/2181536965 shutdown_connections 2021-02-10T11:45:48.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.437 7fab477b2700 1 --2- 172.21.2.101:0/2181536965 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab400cff80 0x7fab40109140 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:48.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.437 7fab477b2700 1 --2- 172.21.2.101:0/2181536965 >> v2:172.21.2.108:3300/0 conn(0x7fab400d1880 0x7fab401b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:48.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.437 7fab477b2700 1 -- 172.21.2.101:0/2181536965 >> 172.21.2.101:0/2181536965 conn(0x7fab400ce430 msgr2=0x7fab400d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:48.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.438 7fab477b2700 1 -- 172.21.2.101:0/2181536965 shutdown_connections 2021-02-10T11:45:48.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.438 7fab477b2700 1 -- 172.21.2.101:0/2181536965 wait complete. 2021-02-10T11:45:48.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.452 7fab477b2700 1 Processor -- start 2021-02-10T11:45:48.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.452 7fab477b2700 1 -- start start 2021-02-10T11:45:48.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.453 7fab477b2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab400abd30 0x7fab40082bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:48.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.453 7fab477b2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fab400831d0 0x7fab4008a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:48.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.453 7fab477b2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab400ac130 0x7fab4008aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:48.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.453 7fab477b2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab40083700 con 0x7fab400abd30 2021-02-10T11:45:48.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.453 7fab477b2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4008b0c0 con 0x7fab400831d0 2021-02-10T11:45:48.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.453 7fab477b2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4008b2a0 con 0x7fab400ac130 2021-02-10T11:45:48.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.453 7fab455ce700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab400abd30 0x7fab40082bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:48.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.453 7fab455ce700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab400abd30 0x7fab40082bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60966/0 (socket says 172.21.2.101:60966) 2021-02-10T11:45:48.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.453 7fab455ce700 1 -- 172.21.2.101:0/193367216 learned_addr learned my addr 172.21.2.101:0/193367216 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:48.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.453 7fab45dcf700 1 --2- 172.21.2.101:0/193367216 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab400ac130 0x7fab4008aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:48.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.453 7fab44dcd700 1 --2- 172.21.2.101:0/193367216 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fab400831d0 0x7fab4008a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:48.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.453 7fab45dcf700 1 -- 172.21.2.101:0/193367216 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fab400831d0 msgr2=0x7fab4008a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:48.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.453 7fab45dcf700 1 --2- 172.21.2.101:0/193367216 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fab400831d0 0x7fab4008a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:48.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.453 7fab45dcf700 1 -- 172.21.2.101:0/193367216 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab400abd30 msgr2=0x7fab40082bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:48.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.453 7fab45dcf700 1 --2- 172.21.2.101:0/193367216 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab400abd30 0x7fab40082bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:48.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.453 7fab45dcf700 1 -- 172.21.2.101:0/193367216 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4008b5f0 con 0x7fab400ac130 2021-02-10T11:45:48.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.453 7fab45dcf700 1 --2- 172.21.2.101:0/193367216 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab400ac130 0x7fab4008aaa0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7fab38009890 tx=0x7fab3800bc20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:48.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.453 7fab31ffb700 1 -- 172.21.2.101:0/193367216 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fab3800bd60 con 0x7fab400ac130 2021-02-10T11:45:48.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.453 7fab31ffb700 1 -- 172.21.2.101:0/193367216 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab38006ce0 con 0x7fab400ac130 2021-02-10T11:45:48.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.454 7fab31ffb700 1 -- 172.21.2.101:0/193367216 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fab38002220 con 0x7fab400ac130 2021-02-10T11:45:48.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.454 7fab477b2700 1 -- 172.21.2.101:0/193367216 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4008b5f0 con 0x7fab400ac130 2021-02-10T11:45:48.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.454 7fab477b2700 1 -- 172.21.2.101:0/193367216 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4008bb50 con 0x7fab400ac130 2021-02-10T11:45:48.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.454 7fab31ffb700 1 -- 172.21.2.101:0/193367216 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fab38006ce0 con 0x7fab400ac130 2021-02-10T11:45:48.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.454 7fab31ffb700 1 --2- 172.21.2.101:0/193367216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab200186c0 0x7fab2001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:48.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.454 7fab31ffb700 1 -- 172.21.2.101:0/193367216 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fab380327e0 con 0x7fab400ac130 2021-02-10T11:45:48.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.455 7fab455ce700 1 --2- 172.21.2.101:0/193367216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab200186c0 0x7fab2001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:48.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.455 7fab455ce700 1 --2- 172.21.2.101:0/193367216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab200186c0 0x7fab2001abc0 crc :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:48.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.485 7fab477b2700 1 -- 172.21.2.101:0/193367216 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab40082980 con 0x7fab400ac130 2021-02-10T11:45:48.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.488 7fab31ffb700 1 -- 172.21.2.101:0/193367216 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fab380093d0 con 0x7fab400ac130 2021-02-10T11:45:48.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.618 7fab477b2700 1 -- 172.21.2.101:0/193367216 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fab40082980 con 0x7fab400ac130 2021-02-10T11:45:48.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.623 7fab31ffb700 1 -- 172.21.2.101:0/193367216 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v92) v1 ==== 184+0+0 (secure 0 0 0) 0x7fab3801f520 con 0x7fab400ac130 2021-02-10T11:45:48.623 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.1 2021-02-10T11:45:48.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.626 7fab477b2700 1 -- 172.21.2.101:0/193367216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab200186c0 msgr2=0x7fab2001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:48.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.626 7fab477b2700 1 --2- 172.21.2.101:0/193367216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab200186c0 0x7fab2001abc0 crc :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:48.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.626 7fab477b2700 1 -- 172.21.2.101:0/193367216 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab400ac130 msgr2=0x7fab4008aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:48.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.626 7fab477b2700 1 --2- 172.21.2.101:0/193367216 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab400ac130 0x7fab4008aaa0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7fab38009890 tx=0x7fab3800bc20).stop 2021-02-10T11:45:48.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.627 7fab477b2700 1 -- 172.21.2.101:0/193367216 shutdown_connections 2021-02-10T11:45:48.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.627 7fab477b2700 1 --2- 172.21.2.101:0/193367216 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab200186c0 0x7fab2001abc0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:48.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.627 7fab477b2700 1 --2- 172.21.2.101:0/193367216 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab400ac130 0x7fab4008aaa0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:48.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.627 7fab477b2700 1 --2- 172.21.2.101:0/193367216 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fab400831d0 0x7fab4008a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:48.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.627 7fab477b2700 1 --2- 172.21.2.101:0/193367216 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab400abd30 0x7fab40082bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:48.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.627 7fab477b2700 1 -- 172.21.2.101:0/193367216 >> 172.21.2.101:0/193367216 conn(0x7fab400afc60 msgr2=0x7fab400b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:48.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.627 7fab477b2700 1 -- 172.21.2.101:0/193367216 shutdown_connections 2021-02-10T11:45:48.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:48.627 7fab477b2700 1 -- 172.21.2.101:0/193367216 wait complete. 2021-02-10T11:45:48.633 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:45:49.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.059 7ff5ef4fa700 1 Processor -- start 2021-02-10T11:45:49.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.059 7ff5ef4fa700 1 -- start start 2021-02-10T11:45:49.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.060 7ff5ef4fa700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff5e80d1c60 0x7ff5e80cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:49.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.060 7ff5ef4fa700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5e80cf570 0x7ff5e80cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:49.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.060 7ff5ef4fa700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e80fe1b0 con 0x7ff5e80d1860 2021-02-10T11:45:49.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.060 7ff5ef4fa700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e81b0ca0 con 0x7ff5e80d1c60 2021-02-10T11:45:49.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.060 7ff5ef4fa700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e8109a50 con 0x7ff5e80cf570 2021-02-10T11:45:49.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.060 7ff5ed316700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff5e80d1860 0x7ff5e81b04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33234/0 (socket says 172.21.2.101:33234) 2021-02-10T11:45:49.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.060 7ff5ed316700 1 -- 172.21.2.101:0/4169787488 learned_addr learned my addr 172.21.2.101:0/4169787488 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:45:49.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.060 7ff5dbfff700 1 -- 172.21.2.101:0/4169787488 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028131504 0 0) 0x7ff5dc0010b0 con 0x7ff5e80d1860 2021-02-10T11:45:49.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.060 7ff5edb17700 1 --2- 172.21.2.101:0/4169787488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5e80cf570 0x7ff5e80cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:49.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.060 7ff5dbfff700 1 -- 172.21.2.101:0/4169787488 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5d4002fe0 con 0x7ff5e80d1860 2021-02-10T11:45:49.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.060 7ff5ecb15700 1 --2- 172.21.2.101:0/4169787488 >> v2:172.21.2.108:3300/0 conn(0x7ff5e80d1c60 0x7ff5e80cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:49.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5dbfff700 1 -- 172.21.2.101:0/4169787488 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1807468465 0 0) 0x7ff5dc0010b0 con 0x7ff5e80d1860 2021-02-10T11:45:49.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5dbfff700 1 -- 172.21.2.101:0/4169787488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5e80cf570 msgr2=0x7ff5e80cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:49.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5dbfff700 1 --2- 172.21.2.101:0/4169787488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5e80cf570 0x7ff5e80cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:49.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5dbfff700 1 -- 172.21.2.101:0/4169787488 >> v2:172.21.2.108:3300/0 conn(0x7ff5e80d1c60 msgr2=0x7ff5e80cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:49.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5dbfff700 1 --2- 172.21.2.101:0/4169787488 >> v2:172.21.2.108:3300/0 conn(0x7ff5e80d1c60 0x7ff5e80cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:49.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5dbfff700 1 -- 172.21.2.101:0/4169787488 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e8088070 con 0x7ff5e80d1860 2021-02-10T11:45:49.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5dbfff700 1 -- 172.21.2.101:0/4169787488 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7ff5dc0015b0 con 0x7ff5e80d1860 2021-02-10T11:45:49.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5dbfff700 1 -- 172.21.2.101:0/4169787488 >> v1:172.21.2.101:6789/0 conn(0x7ff5e80d1860 legacy=0x7ff5e81b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:49.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5dbfff700 1 --2- 172.21.2.101:0/4169787488 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5d4003b90 0x7ff5d4005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:49.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5dbfff700 1 --2- 172.21.2.101:0/4169787488 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5d404a7d0 0x7ff5d404cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:49.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5dbfff700 1 --2- 172.21.2.101:0/4169787488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5d404d590 0x7ff5d404fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:49.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5dbfff700 1 -- 172.21.2.101:0/4169787488 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d40502d0 con 0x7ff5d404a7d0 2021-02-10T11:45:49.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5dbfff700 1 -- 172.21.2.101:0/4169787488 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d40504f0 con 0x7ff5d4003b90 2021-02-10T11:45:49.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5dbfff700 1 -- 172.21.2.101:0/4169787488 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d4050710 con 0x7ff5d404d590 2021-02-10T11:45:49.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5ecb15700 1 --2- 172.21.2.101:0/4169787488 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5d404a7d0 0x7ff5d404cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:49.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5ed316700 1 --2- 172.21.2.101:0/4169787488 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5d4003b90 0x7ff5d4005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:49.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5edb17700 1 --2- 172.21.2.101:0/4169787488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5d404d590 0x7ff5d404fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:49.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5ecb15700 1 -- 172.21.2.101:0/4169787488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5d404d590 msgr2=0x7ff5d404fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:49.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5ecb15700 1 --2- 172.21.2.101:0/4169787488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5d404d590 0x7ff5d404fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:49.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5ecb15700 1 -- 172.21.2.101:0/4169787488 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5d4003b90 msgr2=0x7ff5d4005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:49.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5ecb15700 1 --2- 172.21.2.101:0/4169787488 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5d4003b90 0x7ff5d4005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:49.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5ecb15700 1 -- 172.21.2.101:0/4169787488 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d40509b0 con 0x7ff5d404a7d0 2021-02-10T11:45:49.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.061 7ff5ecb15700 1 --2- 172.21.2.101:0/4169787488 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5d404a7d0 0x7ff5d404cc90 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 rx=0x7ff5e400c8f0 tx=0x7ff5e400cc90).ready entity=mon.0 client_cookie=a2a18571a9fc5b51 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:49.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.062 7ff5dbfff700 1 -- 172.21.2.101:0/4169787488 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff5e400cdd0 con 0x7ff5d404a7d0 2021-02-10T11:45:49.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.062 7ff5dbfff700 1 -- 172.21.2.101:0/4169787488 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e40009b0 con 0x7ff5d404a7d0 2021-02-10T11:45:49.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.062 7ff5dbfff700 1 -- 172.21.2.101:0/4169787488 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff5e4009310 con 0x7ff5d404a7d0 2021-02-10T11:45:49.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.062 7ff5ef4fa700 1 -- 172.21.2.101:0/4169787488 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5d404a7d0 msgr2=0x7ff5d404cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:49.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.062 7ff5ef4fa700 1 --2- 172.21.2.101:0/4169787488 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5d404a7d0 0x7ff5d404cc90 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 rx=0x7ff5e400c8f0 tx=0x7ff5e400cc90).stop 2021-02-10T11:45:49.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.062 7ff5edb17700 1 -- 172.21.2.101:0/4169787488 reap_dead start 2021-02-10T11:45:49.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.062 7ff5ef4fa700 1 -- 172.21.2.101:0/4169787488 shutdown_connections 2021-02-10T11:45:49.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.062 7ff5ef4fa700 1 -- 172.21.2.101:0/4169787488 >> 172.21.2.101:0/4169787488 conn(0x7ff5e80ce410 msgr2=0x7ff5e80d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:49.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.063 7ff5ef4fa700 1 -- 172.21.2.101:0/4169787488 shutdown_connections 2021-02-10T11:45:49.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.063 7ff5ef4fa700 1 -- 172.21.2.101:0/4169787488 wait complete. 2021-02-10T11:45:49.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.077 7ff5ef4fa700 1 Processor -- start 2021-02-10T11:45:49.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.077 7ff5ef4fa700 1 -- start start 2021-02-10T11:45:49.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.078 7ff5ef4fa700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5e80b5fb0 0x7ff5e80a81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:49.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.078 7ff5ef4fa700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5e80a8810 0x7ff5e80a2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:49.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.078 7ff5ef4fa700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5e80a8c10 0x7ff5e80a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:49.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.078 7ff5ef4fa700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80b63b0 con 0x7ff5e80a8810 2021-02-10T11:45:49.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.078 7ff5ef4fa700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80a9210 con 0x7ff5e80a8c10 2021-02-10T11:45:49.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.078 7ff5ef4fa700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80b2650 con 0x7ff5e80b5fb0 2021-02-10T11:45:49.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.078 7ff5ecb15700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5e80a8810 0x7ff5e80a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:49.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.078 7ff5ecb15700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5e80a8810 0x7ff5e80a2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60986/0 (socket says 172.21.2.101:60986) 2021-02-10T11:45:49.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.078 7ff5ecb15700 1 -- 172.21.2.101:0/2813666725 learned_addr learned my addr 172.21.2.101:0/2813666725 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:49.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.078 7ff5ed316700 1 --2- 172.21.2.101:0/2813666725 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5e80b5fb0 0x7ff5e80a81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:49.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.078 7ff5edb17700 1 --2- 172.21.2.101:0/2813666725 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5e80a8c10 0x7ff5e80a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:49.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.079 7ff5ecb15700 1 -- 172.21.2.101:0/2813666725 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5e80b5fb0 msgr2=0x7ff5e80a81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:49.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.079 7ff5ecb15700 1 --2- 172.21.2.101:0/2813666725 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5e80b5fb0 0x7ff5e80a81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:49.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.079 7ff5ecb15700 1 -- 172.21.2.101:0/2813666725 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5e80a8c10 msgr2=0x7ff5e80a2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:49.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.079 7ff5ecb15700 1 --2- 172.21.2.101:0/2813666725 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5e80a8c10 0x7ff5e80a2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:49.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.079 7ff5ecb15700 1 -- 172.21.2.101:0/2813666725 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e80a3180 con 0x7ff5e80a8810 2021-02-10T11:45:49.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.079 7ff5ecb15700 1 --2- 172.21.2.101:0/2813666725 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5e80a8810 0x7ff5e80a2540 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 rx=0x7ff5e4006d50 tx=0x7ff5e400b300).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:49.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.079 7ff5d9ffb700 1 -- 172.21.2.101:0/2813666725 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff5e4000c90 con 0x7ff5e80a8810 2021-02-10T11:45:49.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.079 7ff5d9ffb700 1 -- 172.21.2.101:0/2813666725 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e400c500 con 0x7ff5e80a8810 2021-02-10T11:45:49.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.079 7ff5d9ffb700 1 -- 172.21.2.101:0/2813666725 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff5e400cb70 con 0x7ff5e80a8810 2021-02-10T11:45:49.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.079 7ff5ef4fa700 1 -- 172.21.2.101:0/2813666725 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e80a3180 con 0x7ff5e80a8810 2021-02-10T11:45:49.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.079 7ff5ef4fa700 1 -- 172.21.2.101:0/2813666725 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e80a3710 con 0x7ff5e80a8810 2021-02-10T11:45:49.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.080 7ff5d9ffb700 1 -- 172.21.2.101:0/2813666725 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff5e4000c90 con 0x7ff5e80a8810 2021-02-10T11:45:49.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.080 7ff5d9ffb700 1 --2- 172.21.2.101:0/2813666725 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5c80186c0 0x7ff5c801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:49.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.080 7ff5d9ffb700 1 -- 172.21.2.101:0/2813666725 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff5e4032540 con 0x7ff5e80a8810 2021-02-10T11:45:49.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.080 7ff5ed316700 1 --2- 172.21.2.101:0/2813666725 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5c80186c0 0x7ff5c801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:49.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.080 7ff5ed316700 1 --2- 172.21.2.101:0/2813666725 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5c80186c0 0x7ff5c801abc0 crc :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:49.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.109 7ff5ef4fa700 1 -- 172.21.2.101:0/2813666725 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e80fe1b0 con 0x7ff5e80a8810 2021-02-10T11:45:49.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.112 7ff5d9ffb700 1 -- 172.21.2.101:0/2813666725 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff5e4001f00 con 0x7ff5e80a8810 2021-02-10T11:45:49.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.244 7ff5ef4fa700 1 -- 172.21.2.101:0/2813666725 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ff5e80fe1b0 con 0x7ff5e80a8810 2021-02-10T11:45:49.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.247 7ff5d9ffb700 1 -- 172.21.2.101:0/2813666725 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v93) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff5e400c130 con 0x7ff5e80a8810 2021-02-10T11:45:49.249 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.2 2021-02-10T11:45:49.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.252 7ff5ef4fa700 1 -- 172.21.2.101:0/2813666725 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5c80186c0 msgr2=0x7ff5c801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:49.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.252 7ff5ef4fa700 1 --2- 172.21.2.101:0/2813666725 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5c80186c0 0x7ff5c801abc0 crc :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:49.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.252 7ff5ef4fa700 1 -- 172.21.2.101:0/2813666725 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5e80a8810 msgr2=0x7ff5e80a2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:49.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.252 7ff5ef4fa700 1 --2- 172.21.2.101:0/2813666725 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5e80a8810 0x7ff5e80a2540 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 rx=0x7ff5e4006d50 tx=0x7ff5e400b300).stop 2021-02-10T11:45:49.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.253 7ff5ef4fa700 1 -- 172.21.2.101:0/2813666725 shutdown_connections 2021-02-10T11:45:49.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.253 7ff5ef4fa700 1 --2- 172.21.2.101:0/2813666725 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff5e80a8c10 0x7ff5e80a2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:49.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.253 7ff5ef4fa700 1 --2- 172.21.2.101:0/2813666725 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5e80a8810 0x7ff5e80a2540 unknown :-1 s=CLOSED pgs=2882 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:49.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.253 7ff5ef4fa700 1 --2- 172.21.2.101:0/2813666725 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5c80186c0 0x7ff5c801abc0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:49.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.253 7ff5ef4fa700 1 --2- 172.21.2.101:0/2813666725 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5e80b5fb0 0x7ff5e80a81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:49.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.253 7ff5ef4fa700 1 -- 172.21.2.101:0/2813666725 >> 172.21.2.101:0/2813666725 conn(0x7ff5e809e660 msgr2=0x7ff5e80a9550 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:49.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.253 7ff5ef4fa700 1 -- 172.21.2.101:0/2813666725 shutdown_connections 2021-02-10T11:45:49.252 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.253 7ff5ef4fa700 1 -- 172.21.2.101:0/2813666725 wait complete. 2021-02-10T11:45:49.259 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:45:49.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.684 7f3eeceaf700 1 Processor -- start 2021-02-10T11:45:49.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.684 7f3eeceaf700 1 -- start start 2021-02-10T11:45:49.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.684 7f3eeceaf700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3ee80d1c80 0x7f3ee80cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:49.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.684 7f3eeceaf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3ee80cf590 0x7f3ee80cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:49.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.684 7f3eeceaf700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee80fe1d0 con 0x7f3ee80d1880 2021-02-10T11:45:49.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.684 7f3eeceaf700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ee81b0cc0 con 0x7f3ee80cf590 2021-02-10T11:45:49.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.684 7f3eeceaf700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ee8109a70 con 0x7f3ee80d1c80 2021-02-10T11:45:49.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.684 7f3ee7314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3ee80cf590 0x7f3ee80cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:49.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.684 7f3ee7314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3ee80cf590 0x7f3ee80cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60996/0 (socket says 172.21.2.101:60996) 2021-02-10T11:45:49.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.684 7f3ee7314700 1 -- 172.21.2.101:0/1614313634 learned_addr learned my addr 172.21.2.101:0/1614313634 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:49.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.685 7f3ee7314700 1 -- 172.21.2.101:0/1614313634 >> v2:172.21.2.108:3300/0 conn(0x7f3ee80d1c80 msgr2=0x7f3ee80cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:49.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.685 7f3ee7314700 1 --2- 172.21.2.101:0/1614313634 >> v2:172.21.2.108:3300/0 conn(0x7f3ee80d1c80 0x7f3ee80cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:49.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.685 7f3ee7314700 1 -- 172.21.2.101:0/1614313634 >> v1:172.21.2.108:6789/0 conn(0x7f3ee80d1880 legacy=0x7f3ee81b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:49.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.685 7f3ee7314700 1 -- 172.21.2.101:0/1614313634 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee8088070 con 0x7f3ee80cf590 2021-02-10T11:45:49.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.685 7f3ee7314700 1 --2- 172.21.2.101:0/1614313634 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ee80cf590 0x7f3ee80cf990 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 rx=0x7f3ee0006f00 tx=0x7f3ee0009f10).ready entity=mon.0 client_cookie=db5cb67f5a3370db server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:49.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.685 7f3ee5b11700 1 -- 172.21.2.101:0/1614313634 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3ee0003030 con 0x7f3ee80cf590 2021-02-10T11:45:49.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.685 7f3ee5b11700 1 -- 172.21.2.101:0/1614313634 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee0003030 con 0x7f3ee80cf590 2021-02-10T11:45:49.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.685 7f3ee5b11700 1 -- 172.21.2.101:0/1614313634 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3ee000b3b0 con 0x7f3ee80cf590 2021-02-10T11:45:49.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.685 7f3eeceaf700 1 -- 172.21.2.101:0/1614313634 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ee80cf590 msgr2=0x7f3ee80cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:49.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.685 7f3eeceaf700 1 --2- 172.21.2.101:0/1614313634 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ee80cf590 0x7f3ee80cf990 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 rx=0x7f3ee0006f00 tx=0x7f3ee0009f10).stop 2021-02-10T11:45:49.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.686 7f3eeceaf700 1 -- 172.21.2.101:0/1614313634 shutdown_connections 2021-02-10T11:45:49.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.686 7f3eeceaf700 1 --2- 172.21.2.101:0/1614313634 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ee80cf590 0x7f3ee80cf990 unknown :-1 s=CLOSED pgs=2883 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:49.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.686 7f3eeceaf700 1 --2- 172.21.2.101:0/1614313634 >> v2:172.21.2.108:3300/0 conn(0x7f3ee80d1c80 0x7f3ee80cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:49.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.686 7f3eeceaf700 1 -- 172.21.2.101:0/1614313634 >> 172.21.2.101:0/1614313634 conn(0x7f3ee80ce430 msgr2=0x7f3ee80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:49.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.686 7f3eeceaf700 1 -- 172.21.2.101:0/1614313634 shutdown_connections 2021-02-10T11:45:49.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.686 7f3eeceaf700 1 -- 172.21.2.101:0/1614313634 wait complete. 2021-02-10T11:45:49.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.701 7f3eeceaf700 1 Processor -- start 2021-02-10T11:45:49.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.701 7f3eeceaf700 1 -- start start 2021-02-10T11:45:49.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.701 7f3eeceaf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ee8086810 0x7f3ee819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:49.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.701 7f3eeceaf700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ee80b04d0 0x7f3ee8097450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:49.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.701 7f3eeceaf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ee80b08d0 0x7f3ee8097a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:49.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.701 7f3eeceaf700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee81a02c0 con 0x7f3ee8086810 2021-02-10T11:45:49.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.701 7f3eeceaf700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee8098090 con 0x7f3ee80b04d0 2021-02-10T11:45:49.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.701 7f3eeceaf700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee80982b0 con 0x7f3ee80b08d0 2021-02-10T11:45:49.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.701 7f3ee6b13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ee8086810 0x7f3ee819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:49.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.701 7f3ee7314700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ee80b08d0 0x7f3ee8097a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:49.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.701 7f3ee6b13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ee8086810 0x7f3ee819fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:60998/0 (socket says 172.21.2.101:60998) 2021-02-10T11:45:49.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.701 7f3ee7314700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ee80b08d0 0x7f3ee8097a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:53732/0 (socket says 172.21.2.101:53732) 2021-02-10T11:45:49.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.701 7f3ee6b13700 1 -- 172.21.2.101:0/1083787625 learned_addr learned my addr 172.21.2.101:0/1083787625 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:49.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.702 7f3ee6312700 1 --2- 172.21.2.101:0/1083787625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ee80b04d0 0x7f3ee8097450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:49.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.702 7f3ee6b13700 1 -- 172.21.2.101:0/1083787625 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ee80b08d0 msgr2=0x7f3ee8097a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:49.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.702 7f3ee6b13700 1 --2- 172.21.2.101:0/1083787625 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ee80b08d0 0x7f3ee8097a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:49.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.702 7f3ee6b13700 1 -- 172.21.2.101:0/1083787625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ee80b04d0 msgr2=0x7f3ee8097450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:49.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.702 7f3ee6b13700 1 --2- 172.21.2.101:0/1083787625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ee80b04d0 0x7f3ee8097450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:49.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.702 7f3ee6b13700 1 -- 172.21.2.101:0/1083787625 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee80984d0 con 0x7f3ee8086810 2021-02-10T11:45:49.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.702 7f3ee6b13700 1 --2- 172.21.2.101:0/1083787625 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ee8086810 0x7f3ee819fca0 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 rx=0x7f3ed800b5b0 tx=0x7f3ed800b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:49.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.702 7f3edf7fe700 1 -- 172.21.2.101:0/1083787625 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3ed800bd40 con 0x7f3ee8086810 2021-02-10T11:45:49.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.702 7f3edf7fe700 1 -- 172.21.2.101:0/1083787625 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed8006240 con 0x7f3ee8086810 2021-02-10T11:45:49.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.702 7f3eeceaf700 1 -- 172.21.2.101:0/1083787625 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee80984d0 con 0x7f3ee8086810 2021-02-10T11:45:49.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.702 7f3edf7fe700 1 -- 172.21.2.101:0/1083787625 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3ed8005720 con 0x7f3ee8086810 2021-02-10T11:45:49.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.702 7f3eeceaf700 1 -- 172.21.2.101:0/1083787625 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee80984d0 con 0x7f3ee8086810 2021-02-10T11:45:49.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.703 7f3edf7fe700 1 -- 172.21.2.101:0/1083787625 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3ed80054a0 con 0x7f3ee8086810 2021-02-10T11:45:49.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.703 7f3edf7fe700 1 --2- 172.21.2.101:0/1083787625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3ecc018730 0x7f3ecc01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:49.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.703 7f3edf7fe700 1 -- 172.21.2.101:0/1083787625 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3ed8032670 con 0x7f3ee8086810 2021-02-10T11:45:49.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.703 7f3ee6312700 1 --2- 172.21.2.101:0/1083787625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3ecc018730 0x7f3ecc01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:49.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.703 7f3ee6312700 1 --2- 172.21.2.101:0/1083787625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3ecc018730 0x7f3ecc01ac10 crc :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:49.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.734 7f3eeceaf700 1 -- 172.21.2.101:0/1083787625 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee80fe1d0 con 0x7f3ee8086810 2021-02-10T11:45:49.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.737 7f3edf7fe700 1 -- 172.21.2.101:0/1083787625 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3ed8006030 con 0x7f3ee8086810 2021-02-10T11:45:49.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.867 7f3eeceaf700 1 -- 172.21.2.101:0/1083787625 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f3ee80fe1d0 con 0x7f3ee8086810 2021-02-10T11:45:49.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.871 7f3edf7fe700 1 -- 172.21.2.101:0/1083787625 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v94) v1 ==== 184+0+0 (secure 0 0 0) 0x7f3ed80054a0 con 0x7f3ee8086810 2021-02-10T11:45:49.873 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.3 2021-02-10T11:45:49.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.876 7f3eeceaf700 1 -- 172.21.2.101:0/1083787625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3ecc018730 msgr2=0x7f3ecc01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:49.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.876 7f3eeceaf700 1 --2- 172.21.2.101:0/1083787625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3ecc018730 0x7f3ecc01ac10 crc :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:49.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.876 7f3eeceaf700 1 -- 172.21.2.101:0/1083787625 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ee8086810 msgr2=0x7f3ee819fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:49.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.876 7f3eeceaf700 1 --2- 172.21.2.101:0/1083787625 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ee8086810 0x7f3ee819fca0 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 rx=0x7f3ed800b5b0 tx=0x7f3ed800b950).stop 2021-02-10T11:45:49.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.876 7f3eeceaf700 1 -- 172.21.2.101:0/1083787625 shutdown_connections 2021-02-10T11:45:49.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.876 7f3eeceaf700 1 --2- 172.21.2.101:0/1083787625 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3ecc018730 0x7f3ecc01ac10 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:49.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.876 7f3eeceaf700 1 --2- 172.21.2.101:0/1083787625 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ee80b08d0 0x7f3ee8097a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:49.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.876 7f3eeceaf700 1 --2- 172.21.2.101:0/1083787625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ee80b04d0 0x7f3ee8097450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:49.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.876 7f3eeceaf700 1 --2- 172.21.2.101:0/1083787625 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ee8086810 0x7f3ee819fca0 unknown :-1 s=CLOSED pgs=2884 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:49.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.876 7f3eeceaf700 1 -- 172.21.2.101:0/1083787625 >> 172.21.2.101:0/1083787625 conn(0x7f3ee819f880 msgr2=0x7f3ee808f890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:49.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.876 7f3eeceaf700 1 -- 172.21.2.101:0/1083787625 shutdown_connections 2021-02-10T11:45:49.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:49.876 7f3eeceaf700 1 -- 172.21.2.101:0/1083787625 wait complete. 2021-02-10T11:45:49.883 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:45:50.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.297 7fd2efaa3700 1 Processor -- start 2021-02-10T11:45:50.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.297 7fd2efaa3700 1 -- start start 2021-02-10T11:45:50.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.298 7fd2efaa3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd2e80d1870 0x7fd2e81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:50.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.298 7fd2efaa3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd2e81b0a10 0x7fd2e80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:50.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.298 7fd2efaa3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2e80d1c70 0x7fd2e80cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:50.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.298 7fd2efaa3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e80e5ed0 con 0x7fd2e81b0a10 2021-02-10T11:45:50.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.298 7fd2efaa3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e80f0b10 con 0x7fd2e80d1870 2021-02-10T11:45:50.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.298 7fd2efaa3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e80cff70 con 0x7fd2e80d1c70 2021-02-10T11:45:50.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.298 7fd2ed0be700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd2e81b0a10 0x7fd2e80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:50.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.298 7fd2ed0be700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd2e81b0a10 0x7fd2e80cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32774/0 (socket says 172.21.2.101:32774) 2021-02-10T11:45:50.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.298 7fd2ed0be700 1 -- 172.21.2.101:0/1313405957 learned_addr learned my addr 172.21.2.101:0/1313405957 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:50.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.298 7fd2ee0c0700 1 --2- 172.21.2.101:0/1313405957 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2e80d1c70 0x7fd2e80cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:50.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.298 7fd2ed8bf700 1 --2- 172.21.2.101:0/1313405957 >> v2:172.21.2.108:3300/0 conn(0x7fd2e80d1870 0x7fd2e81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:50.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.298 7fd2ed0be700 1 -- 172.21.2.101:0/1313405957 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2e80d1c70 msgr2=0x7fd2e80cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:50.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.298 7fd2ed0be700 1 --2- 172.21.2.101:0/1313405957 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2e80d1c70 0x7fd2e80cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:50.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.298 7fd2ed0be700 1 -- 172.21.2.101:0/1313405957 >> v2:172.21.2.108:3300/0 conn(0x7fd2e80d1870 msgr2=0x7fd2e81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:50.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.298 7fd2ed0be700 1 --2- 172.21.2.101:0/1313405957 >> v2:172.21.2.108:3300/0 conn(0x7fd2e80d1870 0x7fd2e81b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:50.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.298 7fd2ed0be700 1 -- 172.21.2.101:0/1313405957 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e80d0150 con 0x7fd2e81b0a10 2021-02-10T11:45:50.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.298 7fd2ed8bf700 1 --2- 172.21.2.101:0/1313405957 >> v2:172.21.2.108:3300/0 conn(0x7fd2e80d1870 0x7fd2e81b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:50.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.298 7fd2ed0be700 1 --2- 172.21.2.101:0/1313405957 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2e81b0a10 0x7fd2e80cf330 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 rx=0x7fd2e4006700 tx=0x7fd2e4009b00).ready entity=mon.0 client_cookie=cc4b2f3906e93385 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:50.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.299 7fd2ec8bd700 1 -- 172.21.2.101:0/1313405957 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2e400c0c0 con 0x7fd2e81b0a10 2021-02-10T11:45:50.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.299 7fd2ec8bd700 1 -- 172.21.2.101:0/1313405957 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e400c340 con 0x7fd2e81b0a10 2021-02-10T11:45:50.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.299 7fd2ec8bd700 1 -- 172.21.2.101:0/1313405957 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2e400c820 con 0x7fd2e81b0a10 2021-02-10T11:45:50.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.299 7fd2efaa3700 1 -- 172.21.2.101:0/1313405957 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2e81b0a10 msgr2=0x7fd2e80cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:50.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.299 7fd2efaa3700 1 --2- 172.21.2.101:0/1313405957 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2e81b0a10 0x7fd2e80cf330 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 rx=0x7fd2e4006700 tx=0x7fd2e4009b00).stop 2021-02-10T11:45:50.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.299 7fd2efaa3700 1 -- 172.21.2.101:0/1313405957 shutdown_connections 2021-02-10T11:45:50.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.299 7fd2efaa3700 1 --2- 172.21.2.101:0/1313405957 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2e80d1c70 0x7fd2e80cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:50.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.299 7fd2efaa3700 1 --2- 172.21.2.101:0/1313405957 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2e81b0a10 0x7fd2e80cf330 unknown :-1 s=CLOSED pgs=2885 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:50.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.299 7fd2efaa3700 1 --2- 172.21.2.101:0/1313405957 >> v2:172.21.2.108:3300/0 conn(0x7fd2e80d1870 0x7fd2e81b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:50.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.299 7fd2efaa3700 1 -- 172.21.2.101:0/1313405957 >> 172.21.2.101:0/1313405957 conn(0x7fd2e80ce450 msgr2=0x7fd2e80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:50.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.300 7fd2efaa3700 1 -- 172.21.2.101:0/1313405957 shutdown_connections 2021-02-10T11:45:50.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.300 7fd2efaa3700 1 -- 172.21.2.101:0/1313405957 wait complete. 2021-02-10T11:45:50.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.314 7fd2efaa3700 1 Processor -- start 2021-02-10T11:45:50.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.314 7fd2efaa3700 1 -- start start 2021-02-10T11:45:50.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.315 7fd2efaa3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2e8195650 0x7fd2e80b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:50.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.315 7fd2efaa3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2e80b5b80 0x7fd2e81a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:50.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.315 7fd2efaa3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2e80acfa0 0x7fd2e81a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:50.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.315 7fd2efaa3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e80b6010 con 0x7fd2e80b5b80 2021-02-10T11:45:50.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.315 7fd2efaa3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e81a3a60 con 0x7fd2e80acfa0 2021-02-10T11:45:50.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.315 7fd2efaa3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e81a3c40 con 0x7fd2e8195650 2021-02-10T11:45:50.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.315 7fd2ed0be700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2e80b5b80 0x7fd2e81a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:50.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.315 7fd2ed0be700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2e80b5b80 0x7fd2e81a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32780/0 (socket says 172.21.2.101:32780) 2021-02-10T11:45:50.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.315 7fd2ed0be700 1 -- 172.21.2.101:0/1671234115 learned_addr learned my addr 172.21.2.101:0/1671234115 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:50.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.315 7fd2ed0be700 1 -- 172.21.2.101:0/1671234115 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2e8195650 msgr2=0x7fd2e80b5560 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:50.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.315 7fd2ed0be700 1 --2- 172.21.2.101:0/1671234115 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2e8195650 0x7fd2e80b5560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:50.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.315 7fd2ed0be700 1 -- 172.21.2.101:0/1671234115 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2e80acfa0 msgr2=0x7fd2e81a3440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:50.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.315 7fd2ed0be700 1 --2- 172.21.2.101:0/1671234115 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2e80acfa0 0x7fd2e81a3440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:50.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.315 7fd2ed0be700 1 -- 172.21.2.101:0/1671234115 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e80bcae0 con 0x7fd2e80b5b80 2021-02-10T11:45:50.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.316 7fd2ed0be700 1 --2- 172.21.2.101:0/1671234115 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2e80b5b80 0x7fd2e81a2e20 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 rx=0x7fd2e4006f20 tx=0x7fd2e400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:50.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.316 7fd2da7fc700 1 -- 172.21.2.101:0/1671234115 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2e400bd80 con 0x7fd2e80b5b80 2021-02-10T11:45:50.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.316 7fd2da7fc700 1 -- 172.21.2.101:0/1671234115 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e4000c90 con 0x7fd2e80b5b80 2021-02-10T11:45:50.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.316 7fd2efaa3700 1 -- 172.21.2.101:0/1671234115 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e80bcae0 con 0x7fd2e80b5b80 2021-02-10T11:45:50.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.316 7fd2da7fc700 1 -- 172.21.2.101:0/1671234115 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2e40022b0 con 0x7fd2e80b5b80 2021-02-10T11:45:50.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.316 7fd2efaa3700 1 -- 172.21.2.101:0/1671234115 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e80bcae0 con 0x7fd2e80b5b80 2021-02-10T11:45:50.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.316 7fd2da7fc700 1 -- 172.21.2.101:0/1671234115 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd2e4009100 con 0x7fd2e80b5b80 2021-02-10T11:45:50.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.316 7fd2da7fc700 1 --2- 172.21.2.101:0/1671234115 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2d0018730 0x7fd2d001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:50.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.317 7fd2da7fc700 1 -- 172.21.2.101:0/1671234115 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd2e40327d0 con 0x7fd2e80b5b80 2021-02-10T11:45:50.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.317 7fd2ed8bf700 1 --2- 172.21.2.101:0/1671234115 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2d0018730 0x7fd2d001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:50.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.317 7fd2ed8bf700 1 --2- 172.21.2.101:0/1671234115 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2d0018730 0x7fd2d001abc0 crc :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:50.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.347 7fd2efaa3700 1 -- 172.21.2.101:0/1671234115 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e80b6010 con 0x7fd2e80b5b80 2021-02-10T11:45:50.348 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.350 7fd2da7fc700 1 -- 172.21.2.101:0/1671234115 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd2e4006390 con 0x7fd2e80b5b80 2021-02-10T11:45:50.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.478 7fd2efaa3700 1 -- 172.21.2.101:0/1671234115 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2e80b6010 con 0x7fd2e80b5b80 2021-02-10T11:45:50.476 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.478 7fd2da7fc700 1 -- 172.21.2.101:0/1671234115 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+2104 (secure 0 0 0) 0x7fd2e400bd80 con 0x7fd2e80b5b80 2021-02-10T11:45:50.477 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:45:50.477 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":139,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8371,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/453344112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":453344112},{"type":"v1","addr":"172.21.2.101:6835","nonce":453344112}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":135}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:45:44.128996","modified":"2021-02-10 11:45:46.139945","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7097":{"gid":7097,"name":"b","rank":0,"incarnation":136,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2456711519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2456711519},{"type":"v1","addr":"172.21.2.108:6833","nonce":2456711519}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":17}]} 2021-02-10T11:45:50.477 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 139 2021-02-10T11:45:50.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.480 7fd2efaa3700 1 -- 172.21.2.101:0/1671234115 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2d0018730 msgr2=0x7fd2d001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:50.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.480 7fd2efaa3700 1 --2- 172.21.2.101:0/1671234115 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2d0018730 0x7fd2d001abc0 crc :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:50.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.480 7fd2efaa3700 1 -- 172.21.2.101:0/1671234115 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2e80b5b80 msgr2=0x7fd2e81a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:50.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.480 7fd2efaa3700 1 --2- 172.21.2.101:0/1671234115 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2e80b5b80 0x7fd2e81a2e20 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 rx=0x7fd2e4006f20 tx=0x7fd2e400b990).stop 2021-02-10T11:45:50.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.480 7fd2efaa3700 1 -- 172.21.2.101:0/1671234115 shutdown_connections 2021-02-10T11:45:50.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.480 7fd2efaa3700 1 --2- 172.21.2.101:0/1671234115 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2e80acfa0 0x7fd2e81a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:50.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.480 7fd2efaa3700 1 --2- 172.21.2.101:0/1671234115 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2e80b5b80 0x7fd2e81a2e20 unknown :-1 s=CLOSED pgs=2886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:50.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.480 7fd2efaa3700 1 --2- 172.21.2.101:0/1671234115 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2d0018730 0x7fd2d001abc0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:50.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.480 7fd2efaa3700 1 --2- 172.21.2.101:0/1671234115 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2e8195650 0x7fd2e80b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:50.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.480 7fd2efaa3700 1 -- 172.21.2.101:0/1671234115 >> 172.21.2.101:0/1671234115 conn(0x7fd2e80b3040 msgr2=0x7fd2e80c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:50.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.480 7fd2efaa3700 1 -- 172.21.2.101:0/1671234115 shutdown_connections 2021-02-10T11:45:50.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:50.480 7fd2efaa3700 1 -- 172.21.2.101:0/1671234115 wait complete. 2021-02-10T11:45:50.688 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 139, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:45:44.128996', 'modified': '2021-02-10 11:45:46.139945', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7097}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7097': {'gid': 7097, 'name': 'b', 'rank': 0, 'incarnation': 136, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/2456711519', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 2456711519}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 2456711519}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [35], 'metadata_pool': 34, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:45:50.688 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:45:50.689 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:45:51.086 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:45:51.086 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:45:51.086 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:45:51.087 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 7097, 2021-02-10T11:45:51.087 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:45:51.087 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:45:51.087 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 6.9578240100000004, 2021-02-10T11:45:51.087 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 139, 2021-02-10T11:45:51.087 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 164, 2021-02-10T11:45:51.087 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 164, 2021-02-10T11:45:51.087 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 12.607200987000001 2021-02-10T11:45:51.087 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:45:51.090 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 7097, "want_state": "up:active", "state": "up:active", "rank_uptime": 6.9578240100000004, "mdsmap_epoch": 139, "osdmap_epoch": 164, "osdmap_epoch_barrier": 164, "uptime": 12.607200987000001 } 2021-02-10T11:45:51.091 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:45:51.519 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.505 7f2a03998700 1 Processor -- start 2021-02-10T11:45:51.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.505 7f2a03998700 1 -- start start 2021-02-10T11:45:51.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.505 7f2a03998700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f29fc0d1870 0x7f29fc1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:51.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.505 7f2a03998700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f29fc1b0a10 0x7f29fc0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:51.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.505 7f2a03998700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29fc0d1c70 0x7f29fc0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:51.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.505 7f2a03998700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29fc0e5ed0 con 0x7f29fc0d1870 2021-02-10T11:45:51.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.505 7f2a03998700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29fc0f0b10 con 0x7f29fc1b0a10 2021-02-10T11:45:51.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.505 7f2a03998700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc0cff70 con 0x7f29fc0d1c70 2021-02-10T11:45:51.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.506 7f2a017b4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f29fc0d1870 0x7f29fc1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:51.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.506 7f2a017b4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f29fc0d1870 0x7f29fc1b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32784/0 (socket says 172.21.2.101:32784) 2021-02-10T11:45:51.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.506 7f2a017b4700 1 -- 172.21.2.101:0/3960913550 learned_addr learned my addr 172.21.2.101:0/3960913550 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:51.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.506 7f2a017b4700 1 -- 172.21.2.101:0/3960913550 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29fc0d1c70 msgr2=0x7f29fc0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:51.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.506 7f2a017b4700 1 --2- 172.21.2.101:0/3960913550 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29fc0d1c70 0x7f29fc0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:51.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.506 7f2a017b4700 1 -- 172.21.2.101:0/3960913550 >> v2:172.21.2.108:3300/0 conn(0x7f29fc1b0a10 msgr2=0x7f29fc0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:51.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.506 7f2a017b4700 1 --2- 172.21.2.101:0/3960913550 >> v2:172.21.2.108:3300/0 conn(0x7f29fc1b0a10 0x7f29fc0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:51.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.506 7f2a017b4700 1 -- 172.21.2.101:0/3960913550 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29fc0d0150 con 0x7f29fc0d1870 2021-02-10T11:45:51.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.506 7f2a017b4700 1 --2- 172.21.2.101:0/3960913550 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29fc0d1870 0x7f29fc1b03f0 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 rx=0x7f29f0006700 tx=0x7f29f0009b00).ready entity=mon.0 client_cookie=7dcf22abdc9fd303 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:51.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.506 7f29effff700 1 -- 172.21.2.101:0/3960913550 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f29f000c0c0 con 0x7f29fc0d1870 2021-02-10T11:45:51.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.506 7f29effff700 1 -- 172.21.2.101:0/3960913550 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f000c340 con 0x7f29fc0d1870 2021-02-10T11:45:51.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.506 7f29effff700 1 -- 172.21.2.101:0/3960913550 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f29f000c820 con 0x7f29fc0d1870 2021-02-10T11:45:51.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.506 7f2a03998700 1 -- 172.21.2.101:0/3960913550 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29fc0d1870 msgr2=0x7f29fc1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:51.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.506 7f2a03998700 1 --2- 172.21.2.101:0/3960913550 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29fc0d1870 0x7f29fc1b03f0 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 rx=0x7f29f0006700 tx=0x7f29f0009b00).stop 2021-02-10T11:45:51.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.507 7f2a03998700 1 -- 172.21.2.101:0/3960913550 shutdown_connections 2021-02-10T11:45:51.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.507 7f2a03998700 1 --2- 172.21.2.101:0/3960913550 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29fc0d1c70 0x7f29fc0cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:51.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.507 7f2a03998700 1 --2- 172.21.2.101:0/3960913550 >> v2:172.21.2.108:3300/0 conn(0x7f29fc1b0a10 0x7f29fc0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:51.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.507 7f2a03998700 1 --2- 172.21.2.101:0/3960913550 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29fc0d1870 0x7f29fc1b03f0 unknown :-1 s=CLOSED pgs=2887 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:51.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.507 7f2a03998700 1 -- 172.21.2.101:0/3960913550 >> 172.21.2.101:0/3960913550 conn(0x7f29fc0ce450 msgr2=0x7f29fc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:51.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.507 7f2a03998700 1 -- 172.21.2.101:0/3960913550 shutdown_connections 2021-02-10T11:45:51.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.507 7f2a03998700 1 -- 172.21.2.101:0/3960913550 wait complete. 2021-02-10T11:45:51.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.522 7f2a03998700 1 Processor -- start 2021-02-10T11:45:51.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.522 7f2a03998700 1 -- start start 2021-02-10T11:45:51.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.522 7f2a03998700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29fc1b8ad0 0x7f29fc187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:51.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.522 7f2a03998700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29fc1b8ed0 0x7f29fc0b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:51.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.522 7f2a03998700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29fc0b2110 0x7f29fc1b7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:51.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.522 7f2a03998700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc188470 con 0x7f29fc1b8ed0 2021-02-10T11:45:51.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.522 7f2a03998700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc18be10 con 0x7f29fc1b8ad0 2021-02-10T11:45:51.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.522 7f2a03998700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc18c030 con 0x7f29fc0b2110 2021-02-10T11:45:51.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.523 7f2a00fb3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29fc1b8ed0 0x7f29fc0b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:51.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.523 7f2a017b4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29fc1b8ad0 0x7f29fc187e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:51.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.523 7f2a00fb3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29fc1b8ed0 0x7f29fc0b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32792/0 (socket says 172.21.2.101:32792) 2021-02-10T11:45:51.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.523 7f2a01fb5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29fc0b2110 0x7f29fc1b7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:51.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.523 7f2a00fb3700 1 -- 172.21.2.101:0/2243557123 learned_addr learned my addr 172.21.2.101:0/2243557123 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:51.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.523 7f2a00fb3700 1 -- 172.21.2.101:0/2243557123 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29fc0b2110 msgr2=0x7f29fc1b7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:51.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.523 7f2a00fb3700 1 --2- 172.21.2.101:0/2243557123 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29fc0b2110 0x7f29fc1b7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:51.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.523 7f2a00fb3700 1 -- 172.21.2.101:0/2243557123 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29fc1b8ad0 msgr2=0x7f29fc187e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:51.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.523 7f2a00fb3700 1 --2- 172.21.2.101:0/2243557123 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29fc1b8ad0 0x7f29fc187e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:51.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.523 7f2a00fb3700 1 -- 172.21.2.101:0/2243557123 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29fc0b2510 con 0x7f29fc1b8ed0 2021-02-10T11:45:51.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.523 7f2a017b4700 1 --2- 172.21.2.101:0/2243557123 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29fc1b8ad0 0x7f29fc187e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:45:51.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.523 7f2a00fb3700 1 --2- 172.21.2.101:0/2243557123 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29fc1b8ed0 0x7f29fc0b1af0 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 rx=0x7f29f800b530 tx=0x7f29f800b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:51.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.524 7f29edffb700 1 -- 172.21.2.101:0/2243557123 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f29f800bcc0 con 0x7f29fc1b8ed0 2021-02-10T11:45:51.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.524 7f29edffb700 1 -- 172.21.2.101:0/2243557123 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f8000c50 con 0x7f29fc1b8ed0 2021-02-10T11:45:51.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.524 7f2a03998700 1 -- 172.21.2.101:0/2243557123 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29fc0b2510 con 0x7f29fc1b8ed0 2021-02-10T11:45:51.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.524 7f2a03998700 1 -- 172.21.2.101:0/2243557123 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29fc0b2510 con 0x7f29fc1b8ed0 2021-02-10T11:45:51.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.524 7f29edffb700 1 -- 172.21.2.101:0/2243557123 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f29f80022a0 con 0x7f29fc1b8ed0 2021-02-10T11:45:51.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.524 7f29edffb700 1 -- 172.21.2.101:0/2243557123 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f29f8005310 con 0x7f29fc1b8ed0 2021-02-10T11:45:51.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.524 7f29edffb700 1 --2- 172.21.2.101:0/2243557123 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29dc0186c0 0x7f29dc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:51.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.524 7f29edffb700 1 -- 172.21.2.101:0/2243557123 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f29f8032550 con 0x7f29fc1b8ed0 2021-02-10T11:45:51.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.524 7f2a017b4700 1 --2- 172.21.2.101:0/2243557123 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29dc0186c0 0x7f29dc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:51.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.525 7f2a017b4700 1 --2- 172.21.2.101:0/2243557123 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29dc0186c0 0x7f29dc01abc0 crc :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:51.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.555 7f2a03998700 1 -- 172.21.2.101:0/2243557123 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29fc0ebc00 con 0x7f29fc1b8ed0 2021-02-10T11:45:51.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.558 7f29edffb700 1 -- 172.21.2.101:0/2243557123 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f29f8006b90 con 0x7f29fc1b8ed0 2021-02-10T11:45:51.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.684 7f2a03998700 1 -- 172.21.2.101:0/2243557123 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29fc0ebc00 con 0x7f29fc1b8ed0 2021-02-10T11:45:51.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.684 7f29edffb700 1 -- 172.21.2.101:0/2243557123 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f29f8006b90 con 0x7f29fc1b8ed0 2021-02-10T11:45:51.683 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:45:51.683 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":164,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:45:45.131083","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:45:42.100593","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":35,"pool_name":"cephfs_data","create_time":"2021-02-10 11:45:43.124384","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:45:51.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.686 7f2a03998700 1 -- 172.21.2.101:0/2243557123 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29dc0186c0 msgr2=0x7f29dc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:51.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.686 7f2a03998700 1 --2- 172.21.2.101:0/2243557123 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29dc0186c0 0x7f29dc01abc0 crc :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:51.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.686 7f2a03998700 1 -- 172.21.2.101:0/2243557123 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29fc1b8ed0 msgr2=0x7f29fc0b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:51.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.686 7f2a03998700 1 --2- 172.21.2.101:0/2243557123 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29fc1b8ed0 0x7f29fc0b1af0 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 rx=0x7f29f800b530 tx=0x7f29f800b8d0).stop 2021-02-10T11:45:51.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.686 7f2a03998700 1 -- 172.21.2.101:0/2243557123 shutdown_connections 2021-02-10T11:45:51.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.686 7f2a03998700 1 --2- 172.21.2.101:0/2243557123 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f29dc0186c0 0x7f29dc01abc0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:51.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.686 7f2a03998700 1 --2- 172.21.2.101:0/2243557123 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f29fc0b2110 0x7f29fc1b7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:51.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.686 7f2a03998700 1 --2- 172.21.2.101:0/2243557123 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f29fc1b8ed0 0x7f29fc0b1af0 unknown :-1 s=CLOSED pgs=2888 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:51.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.686 7f2a03998700 1 --2- 172.21.2.101:0/2243557123 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f29fc1b8ad0 0x7f29fc187e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:51.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.686 7f2a03998700 1 -- 172.21.2.101:0/2243557123 >> 172.21.2.101:0/2243557123 conn(0x7f29fc0ed810 msgr2=0x7f29fc094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:51.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.686 7f2a03998700 1 -- 172.21.2.101:0/2243557123 shutdown_connections 2021-02-10T11:45:51.685 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:51.686 7f2a03998700 1 -- 172.21.2.101:0/2243557123 wait complete. 2021-02-10T11:45:51.693 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 34, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:45:42.100593', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '163', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 35, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:45:43.124384', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '164', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:45:51.693 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:45:52.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.112 7ff0b0398700 1 Processor -- start 2021-02-10T11:45:52.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.112 7ff0b0398700 1 -- start start 2021-02-10T11:45:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.113 7ff0b0398700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff0a80d1870 0x7ff0a81b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.113 7ff0b0398700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff0a81b0ae0 0x7ff0a8108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.113 7ff0b0398700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a80fe1c0 con 0x7ff0a80d1c70 2021-02-10T11:45:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.113 7ff0b0398700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a8109a60 con 0x7ff0a81b0ae0 2021-02-10T11:45:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.113 7ff0b0398700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a8109750 con 0x7ff0a80d1870 2021-02-10T11:45:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.113 7ff0ad9b3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff0a81b0ae0 0x7ff0a8108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.113 7ff0ad9b3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff0a81b0ae0 0x7ff0a8108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32796/0 (socket says 172.21.2.101:32796) 2021-02-10T11:45:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.113 7ff0ad9b3700 1 -- 172.21.2.101:0/987207345 learned_addr learned my addr 172.21.2.101:0/987207345 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.113 7ff0ad9b3700 1 -- 172.21.2.101:0/987207345 >> v2:172.21.2.108:3300/0 conn(0x7ff0a80d1870 msgr2=0x7ff0a81b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.113 7ff0ad9b3700 1 --2- 172.21.2.101:0/987207345 >> v2:172.21.2.108:3300/0 conn(0x7ff0a80d1870 0x7ff0a81b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.113 7ff0ad9b3700 1 -- 172.21.2.101:0/987207345 >> v1:172.21.2.108:6789/0 conn(0x7ff0a80d1c70 legacy=0x7ff0a8108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.113 7ff0ad9b3700 1 -- 172.21.2.101:0/987207345 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a8088070 con 0x7ff0a81b0ae0 2021-02-10T11:45:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.113 7ff0ad9b3700 1 --2- 172.21.2.101:0/987207345 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0a81b0ae0 0x7ff0a8108940 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 rx=0x7ff0a4006700 tx=0x7ff0a4009b80).ready entity=mon.0 client_cookie=c376c70fc835d1f7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:52.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.113 7ff0ad1b2700 1 -- 172.21.2.101:0/987207345 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0a400c0c0 con 0x7ff0a81b0ae0 2021-02-10T11:45:52.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.113 7ff0ad1b2700 1 -- 172.21.2.101:0/987207345 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a400c440 con 0x7ff0a81b0ae0 2021-02-10T11:45:52.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.113 7ff0ad1b2700 1 -- 172.21.2.101:0/987207345 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0a400c960 con 0x7ff0a81b0ae0 2021-02-10T11:45:52.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.113 7ff0b0398700 1 -- 172.21.2.101:0/987207345 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0a81b0ae0 msgr2=0x7ff0a8108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:52.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.113 7ff0b0398700 1 --2- 172.21.2.101:0/987207345 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0a81b0ae0 0x7ff0a8108940 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 rx=0x7ff0a4006700 tx=0x7ff0a4009b80).stop 2021-02-10T11:45:52.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.114 7ff0b0398700 1 -- 172.21.2.101:0/987207345 shutdown_connections 2021-02-10T11:45:52.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.114 7ff0b0398700 1 --2- 172.21.2.101:0/987207345 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0a81b0ae0 0x7ff0a8108940 unknown :-1 s=CLOSED pgs=2889 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:52.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.114 7ff0b0398700 1 --2- 172.21.2.101:0/987207345 >> v2:172.21.2.108:3300/0 conn(0x7ff0a80d1870 0x7ff0a81b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:52.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.114 7ff0b0398700 1 -- 172.21.2.101:0/987207345 >> 172.21.2.101:0/987207345 conn(0x7ff0a80ce420 msgr2=0x7ff0a80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:52.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.114 7ff0b0398700 1 -- 172.21.2.101:0/987207345 shutdown_connections 2021-02-10T11:45:52.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.114 7ff0b0398700 1 -- 172.21.2.101:0/987207345 wait complete. 2021-02-10T11:45:52.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.129 7ff0b0398700 1 Processor -- start 2021-02-10T11:45:52.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.129 7ff0b0398700 1 -- start start 2021-02-10T11:45:52.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.129 7ff0b0398700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0a80c3820 0x7ff0a8084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:52.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.129 7ff0b0398700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0a80a0290 0x7ff0a80852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:52.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.129 7ff0b0398700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0a8086270 0x7ff0a8086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:52.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.129 7ff0b0398700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a80c0700 con 0x7ff0a80a0290 2021-02-10T11:45:52.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.129 7ff0b0398700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a80858f0 con 0x7ff0a80c3820 2021-02-10T11:45:52.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.129 7ff0b0398700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a8085b10 con 0x7ff0a8086270 2021-02-10T11:45:52.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.129 7ff0ad9b3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0a80a0290 0x7ff0a80852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:52.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.129 7ff0ad9b3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0a80a0290 0x7ff0a80852d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32806/0 (socket says 172.21.2.101:32806) 2021-02-10T11:45:52.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.129 7ff0ad9b3700 1 -- 172.21.2.101:0/1509232331 learned_addr learned my addr 172.21.2.101:0/1509232331 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:52.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.130 7ff0ad9b3700 1 -- 172.21.2.101:0/1509232331 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0a8086270 msgr2=0x7ff0a8086670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:52.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.130 7ff0ad9b3700 1 --2- 172.21.2.101:0/1509232331 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0a8086270 0x7ff0a8086670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:52.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.130 7ff0ad9b3700 1 -- 172.21.2.101:0/1509232331 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0a80c3820 msgr2=0x7ff0a8084cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:52.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.130 7ff0ad9b3700 1 --2- 172.21.2.101:0/1509232331 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0a80c3820 0x7ff0a8084cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:52.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.130 7ff0ad9b3700 1 -- 172.21.2.101:0/1509232331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a8086c90 con 0x7ff0a80a0290 2021-02-10T11:45:52.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.130 7ff0ad9b3700 1 --2- 172.21.2.101:0/1509232331 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0a80a0290 0x7ff0a80852d0 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 rx=0x7ff0a400b5e0 tx=0x7ff0a400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:52.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.130 7ff09affd700 1 -- 172.21.2.101:0/1509232331 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0a400bd70 con 0x7ff0a80a0290 2021-02-10T11:45:52.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.130 7ff09affd700 1 -- 172.21.2.101:0/1509232331 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a4002030 con 0x7ff0a80a0290 2021-02-10T11:45:52.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.130 7ff0b0398700 1 -- 172.21.2.101:0/1509232331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a8086c90 con 0x7ff0a80a0290 2021-02-10T11:45:52.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.130 7ff0b0398700 1 -- 172.21.2.101:0/1509232331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a8086c90 con 0x7ff0a80a0290 2021-02-10T11:45:52.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.130 7ff09affd700 1 -- 172.21.2.101:0/1509232331 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0a4002550 con 0x7ff0a80a0290 2021-02-10T11:45:52.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.131 7ff09affd700 1 -- 172.21.2.101:0/1509232331 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff0a4002210 con 0x7ff0a80a0290 2021-02-10T11:45:52.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.131 7ff09affd700 1 --2- 172.21.2.101:0/1509232331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff090018730 0x7ff09001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:52.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.131 7ff09affd700 1 -- 172.21.2.101:0/1509232331 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff0a40327a0 con 0x7ff0a80a0290 2021-02-10T11:45:52.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.131 7ff0ae1b4700 1 --2- 172.21.2.101:0/1509232331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff090018730 0x7ff09001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:52.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.131 7ff0ae1b4700 1 --2- 172.21.2.101:0/1509232331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff090018730 0x7ff09001ac10 crc :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:52.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.162 7ff0b0398700 1 -- 172.21.2.101:0/1509232331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a8082a70 con 0x7ff0a80a0290 2021-02-10T11:45:52.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.165 7ff09affd700 1 -- 172.21.2.101:0/1509232331 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff0a400bd70 con 0x7ff0a80a0290 2021-02-10T11:45:52.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.292 7ff0b0398700 1 -- 172.21.2.101:0/1509232331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7ff0a8082a70 con 0x7ff0a80a0290 2021-02-10T11:45:52.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.293 7ff09affd700 1 -- 172.21.2.101:0/1509232331 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v164) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff0a4002030 con 0x7ff0a80a0290 2021-02-10T11:45:52.291 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:45:52.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.294 7ff0b0398700 1 -- 172.21.2.101:0/1509232331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff090018730 msgr2=0x7ff09001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.294 7ff0b0398700 1 --2- 172.21.2.101:0/1509232331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff090018730 0x7ff09001ac10 crc :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.294 7ff0b0398700 1 -- 172.21.2.101:0/1509232331 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0a80a0290 msgr2=0x7ff0a80852d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.294 7ff0b0398700 1 --2- 172.21.2.101:0/1509232331 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0a80a0290 0x7ff0a80852d0 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 rx=0x7ff0a400b5e0 tx=0x7ff0a400b980).stop 2021-02-10T11:45:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.294 7ff0b0398700 1 -- 172.21.2.101:0/1509232331 shutdown_connections 2021-02-10T11:45:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.295 7ff0b0398700 1 --2- 172.21.2.101:0/1509232331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff090018730 0x7ff09001ac10 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.295 7ff0b0398700 1 --2- 172.21.2.101:0/1509232331 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0a8086270 0x7ff0a8086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.295 7ff0b0398700 1 --2- 172.21.2.101:0/1509232331 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0a80a0290 0x7ff0a80852d0 unknown :-1 s=CLOSED pgs=2890 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.295 7ff0b0398700 1 --2- 172.21.2.101:0/1509232331 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0a80c3820 0x7ff0a8084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.295 7ff0b0398700 1 -- 172.21.2.101:0/1509232331 >> 172.21.2.101:0/1509232331 conn(0x7ff0a80872c0 msgr2=0x7ff0a8095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.295 7ff0b0398700 1 -- 172.21.2.101:0/1509232331 shutdown_connections 2021-02-10T11:45:52.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.295 7ff0b0398700 1 -- 172.21.2.101:0/1509232331 wait complete. 2021-02-10T11:45:52.301 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:45:52.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.718 7ff9661ce700 1 Processor -- start 2021-02-10T11:45:52.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.719 7ff9661ce700 1 -- start start 2021-02-10T11:45:52.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.719 7ff9661ce700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9600d1870 0x7ff9601b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:52.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.719 7ff9661ce700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff9600cff70 0x7ff960109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:52.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.719 7ff9661ce700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff960109a60 con 0x7ff9601b0ae0 2021-02-10T11:45:52.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.719 7ff9661ce700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff960109750 con 0x7ff9600cff70 2021-02-10T11:45:52.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.719 7ff9661ce700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff960088070 con 0x7ff9600d1870 2021-02-10T11:45:52.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.720 7ff95f7fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9600d1870 0x7ff9601b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:52.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.720 7ff95f7fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9600d1870 0x7ff9601b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:53774/0 (socket says 172.21.2.101:53774) 2021-02-10T11:45:52.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.720 7ff95effd700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7ff9601b0ae0 0x7ff960108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:54946/0 (socket says 172.21.2.101:54946) 2021-02-10T11:45:52.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.720 7ff95f7fe700 1 -- 172.21.2.101:0/3795375290 learned_addr learned my addr 172.21.2.101:0/3795375290 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:52.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.720 7ff95ffff700 1 --2- 172.21.2.101:0/3795375290 >> v2:172.21.2.108:3300/0 conn(0x7ff9600cff70 0x7ff960109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:52.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.720 7ff95f7fe700 1 -- 172.21.2.101:0/3795375290 >> v2:172.21.2.108:3300/0 conn(0x7ff9600cff70 msgr2=0x7ff960109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:52.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.720 7ff95f7fe700 1 --2- 172.21.2.101:0/3795375290 >> v2:172.21.2.108:3300/0 conn(0x7ff9600cff70 0x7ff960109130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:52.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.720 7ff95f7fe700 1 -- 172.21.2.101:0/3795375290 >> v1:172.21.2.108:6789/0 conn(0x7ff9601b0ae0 legacy=0x7ff960108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:52.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.720 7ff95f7fe700 1 -- 172.21.2.101:0/3795375290 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff960088260 con 0x7ff9600d1870 2021-02-10T11:45:52.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.720 7ff95f7fe700 1 --2- 172.21.2.101:0/3795375290 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9600d1870 0x7ff9601b04c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7ff948006700 tx=0x7ff948009b80).ready entity=mon.2 client_cookie=2876001f142c92c3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:52.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.721 7ff95e7fc700 1 -- 172.21.2.101:0/3795375290 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff94800c0c0 con 0x7ff9600d1870 2021-02-10T11:45:52.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.721 7ff95e7fc700 1 -- 172.21.2.101:0/3795375290 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94800c340 con 0x7ff9600d1870 2021-02-10T11:45:52.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.721 7ff95e7fc700 1 -- 172.21.2.101:0/3795375290 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff94800c860 con 0x7ff9600d1870 2021-02-10T11:45:52.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.721 7ff9661ce700 1 -- 172.21.2.101:0/3795375290 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9600d1870 msgr2=0x7ff9601b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:52.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.721 7ff9661ce700 1 --2- 172.21.2.101:0/3795375290 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9600d1870 0x7ff9601b04c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7ff948006700 tx=0x7ff948009b80).stop 2021-02-10T11:45:52.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.721 7ff9661ce700 1 -- 172.21.2.101:0/3795375290 shutdown_connections 2021-02-10T11:45:52.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.721 7ff9661ce700 1 --2- 172.21.2.101:0/3795375290 >> v2:172.21.2.108:3300/0 conn(0x7ff9600cff70 0x7ff960109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:52.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.721 7ff9661ce700 1 --2- 172.21.2.101:0/3795375290 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9600d1870 0x7ff9601b04c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:52.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.721 7ff9661ce700 1 -- 172.21.2.101:0/3795375290 >> 172.21.2.101:0/3795375290 conn(0x7ff9600ce420 msgr2=0x7ff9600d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:52.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.722 7ff9661ce700 1 -- 172.21.2.101:0/3795375290 shutdown_connections 2021-02-10T11:45:52.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.722 7ff9661ce700 1 -- 172.21.2.101:0/3795375290 wait complete. 2021-02-10T11:45:52.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.736 7ff9661ce700 1 Processor -- start 2021-02-10T11:45:52.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.736 7ff9661ce700 1 -- start start 2021-02-10T11:45:52.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.736 7ff9661ce700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9600b1ba0 0x7ff9601a78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:52.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.736 7ff9661ce700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9601a7ef0 0x7ff9600a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:52.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.736 7ff9661ce700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff9600a7060 0x7ff9600b2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:52.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.736 7ff9661ce700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff960190c20 con 0x7ff9601a7ef0 2021-02-10T11:45:52.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.736 7ff9661ce700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff960190710 con 0x7ff9600a7060 2021-02-10T11:45:52.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.736 7ff9661ce700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9600b1fa0 con 0x7ff9600b1ba0 2021-02-10T11:45:52.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.737 7ff95effd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9601a7ef0 0x7ff9600a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:52.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.737 7ff95effd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9601a7ef0 0x7ff9600a6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32814/0 (socket says 172.21.2.101:32814) 2021-02-10T11:45:52.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.737 7ff95effd700 1 -- 172.21.2.101:0/3258672955 learned_addr learned my addr 172.21.2.101:0/3258672955 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:52.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.737 7ff95effd700 1 -- 172.21.2.101:0/3258672955 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9600b1ba0 msgr2=0x7ff9601a78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:52.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.737 7ff95effd700 1 --2- 172.21.2.101:0/3258672955 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9600b1ba0 0x7ff9601a78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:52.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.737 7ff95effd700 1 -- 172.21.2.101:0/3258672955 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff9600a7060 msgr2=0x7ff9600b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:52.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.737 7ff95effd700 1 --2- 172.21.2.101:0/3258672955 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff9600a7060 0x7ff9600b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:52.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.737 7ff95effd700 1 -- 172.21.2.101:0/3258672955 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9600a7460 con 0x7ff9601a7ef0 2021-02-10T11:45:52.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.737 7ff95effd700 1 --2- 172.21.2.101:0/3258672955 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9601a7ef0 0x7ff9600a6a40 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 rx=0x7ff950001c60 tx=0x7ff95000a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:52.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.737 7ff957fff700 1 -- 172.21.2.101:0/3258672955 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff95000acc0 con 0x7ff9601a7ef0 2021-02-10T11:45:52.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.737 7ff957fff700 1 -- 172.21.2.101:0/3258672955 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff950004140 con 0x7ff9601a7ef0 2021-02-10T11:45:52.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.737 7ff9661ce700 1 -- 172.21.2.101:0/3258672955 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9600a7460 con 0x7ff9601a7ef0 2021-02-10T11:45:52.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.737 7ff957fff700 1 -- 172.21.2.101:0/3258672955 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff9500045d0 con 0x7ff9601a7ef0 2021-02-10T11:45:52.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.737 7ff9661ce700 1 -- 172.21.2.101:0/3258672955 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9600a7460 con 0x7ff9601a7ef0 2021-02-10T11:45:52.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.738 7ff957fff700 1 -- 172.21.2.101:0/3258672955 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff950000ce0 con 0x7ff9601a7ef0 2021-02-10T11:45:52.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.738 7ff957fff700 1 --2- 172.21.2.101:0/3258672955 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff944018730 0x7ff94401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:52.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.738 7ff957fff700 1 -- 172.21.2.101:0/3258672955 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff950032640 con 0x7ff9601a7ef0 2021-02-10T11:45:52.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.738 7ff95f7fe700 1 --2- 172.21.2.101:0/3258672955 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff944018730 0x7ff94401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:52.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.738 7ff95f7fe700 1 --2- 172.21.2.101:0/3258672955 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff944018730 0x7ff94401ac10 crc :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:52.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.769 7ff9661ce700 1 -- 172.21.2.101:0/3258672955 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff960190710 con 0x7ff9601a7ef0 2021-02-10T11:45:52.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.771 7ff957fff700 1 -- 172.21.2.101:0/3258672955 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff9500009b0 con 0x7ff9601a7ef0 2021-02-10T11:45:52.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.901 7ff9661ce700 1 -- 172.21.2.101:0/3258672955 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7ff960190710 con 0x7ff9601a7ef0 2021-02-10T11:45:52.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.901 7ff957fff700 1 -- 172.21.2.101:0/3258672955 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v164) v1 ==== 104+0+10 (secure 0 0 0) 0x7ff950000ce0 con 0x7ff9601a7ef0 2021-02-10T11:45:52.900 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:45:52.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.903 7ff9661ce700 1 -- 172.21.2.101:0/3258672955 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff944018730 msgr2=0x7ff94401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:52.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.903 7ff9661ce700 1 --2- 172.21.2.101:0/3258672955 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff944018730 0x7ff94401ac10 crc :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:52.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.903 7ff9661ce700 1 -- 172.21.2.101:0/3258672955 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9601a7ef0 msgr2=0x7ff9600a6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:52.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.903 7ff9661ce700 1 --2- 172.21.2.101:0/3258672955 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9601a7ef0 0x7ff9600a6a40 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 rx=0x7ff950001c60 tx=0x7ff95000a8d0).stop 2021-02-10T11:45:52.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.903 7ff9661ce700 1 -- 172.21.2.101:0/3258672955 shutdown_connections 2021-02-10T11:45:52.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.903 7ff9661ce700 1 --2- 172.21.2.101:0/3258672955 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff9600a7060 0x7ff9600b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:52.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.903 7ff9661ce700 1 --2- 172.21.2.101:0/3258672955 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff9601a7ef0 0x7ff9600a6a40 unknown :-1 s=CLOSED pgs=2891 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:52.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.903 7ff9661ce700 1 --2- 172.21.2.101:0/3258672955 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff944018730 0x7ff94401ac10 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:52.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.903 7ff9661ce700 1 --2- 172.21.2.101:0/3258672955 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff9600b1ba0 0x7ff9601a78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:52.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.903 7ff9661ce700 1 -- 172.21.2.101:0/3258672955 >> 172.21.2.101:0/3258672955 conn(0x7ff9600b5dc0 msgr2=0x7ff960095960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:52.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.904 7ff9661ce700 1 -- 172.21.2.101:0/3258672955 shutdown_connections 2021-02-10T11:45:52.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:52.904 7ff9661ce700 1 -- 172.21.2.101:0/3258672955 wait complete. 2021-02-10T11:45:52.910 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:45:53.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.326 7f086d202700 1 Processor -- start 2021-02-10T11:45:53.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.326 7f086d202700 1 -- start start 2021-02-10T11:45:53.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.327 7f086d202700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f08680d1860 0x7f08681b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:53.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.327 7f086d202700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08681b0ad0 0x7f0868108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:53.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.327 7f086d202700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f08680d1c60 0x7f0868108f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:53.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.327 7f086d202700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08680fe1b0 con 0x7f08680d1860 2021-02-10T11:45:53.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.327 7f086d202700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0868109a50 con 0x7f08680d1c60 2021-02-10T11:45:53.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.327 7f086d202700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0868088070 con 0x7f08681b0ad0 2021-02-10T11:45:53.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.327 7f0866d9d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f08680d1860 0x7f08681b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:53.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.327 7f0866d9d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f08680d1860 0x7f08681b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32822/0 (socket says 172.21.2.101:32822) 2021-02-10T11:45:53.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.327 7f0866d9d700 1 -- 172.21.2.101:0/758839934 learned_addr learned my addr 172.21.2.101:0/758839934 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:53.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.327 7f0866d9d700 1 -- 172.21.2.101:0/758839934 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08681b0ad0 msgr2=0x7f0868108930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:45:53.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.327 7f0866d9d700 1 --2- 172.21.2.101:0/758839934 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08681b0ad0 0x7f0868108930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:53.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.327 7f0866d9d700 1 -- 172.21.2.101:0/758839934 >> v2:172.21.2.108:3300/0 conn(0x7f08680d1c60 msgr2=0x7f0868108f50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:45:53.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.327 7f0866d9d700 1 --2- 172.21.2.101:0/758839934 >> v2:172.21.2.108:3300/0 conn(0x7f08680d1c60 0x7f0868108f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:53.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.327 7f0866d9d700 1 -- 172.21.2.101:0/758839934 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0868109570 con 0x7f08680d1860 2021-02-10T11:45:53.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.328 7f0866d9d700 1 --2- 172.21.2.101:0/758839934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08680d1860 0x7f08681b04b0 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 rx=0x7f0858006700 tx=0x7f0858009b80).ready entity=mon.0 client_cookie=955218a5ac5a76b2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:53.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.328 7f0865d9b700 1 -- 172.21.2.101:0/758839934 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f085800c0c0 con 0x7f08680d1860 2021-02-10T11:45:53.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.328 7f0865d9b700 1 -- 172.21.2.101:0/758839934 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085800c340 con 0x7f08680d1860 2021-02-10T11:45:53.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.328 7f0865d9b700 1 -- 172.21.2.101:0/758839934 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f085800c860 con 0x7f08680d1860 2021-02-10T11:45:53.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.328 7f086d202700 1 -- 172.21.2.101:0/758839934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08680d1860 msgr2=0x7f08681b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:53.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.328 7f086d202700 1 --2- 172.21.2.101:0/758839934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08680d1860 0x7f08681b04b0 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 rx=0x7f0858006700 tx=0x7f0858009b80).stop 2021-02-10T11:45:53.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.329 7f086d202700 1 -- 172.21.2.101:0/758839934 shutdown_connections 2021-02-10T11:45:53.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.329 7f086d202700 1 --2- 172.21.2.101:0/758839934 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08681b0ad0 0x7f0868108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:53.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.329 7f086d202700 1 --2- 172.21.2.101:0/758839934 >> v2:172.21.2.108:3300/0 conn(0x7f08680d1c60 0x7f0868108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:53.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.329 7f086d202700 1 --2- 172.21.2.101:0/758839934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08680d1860 0x7f08681b04b0 unknown :-1 s=CLOSED pgs=2892 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:53.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.329 7f086d202700 1 -- 172.21.2.101:0/758839934 >> 172.21.2.101:0/758839934 conn(0x7f08680ce410 msgr2=0x7f08680d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:53.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.329 7f086d202700 1 -- 172.21.2.101:0/758839934 shutdown_connections 2021-02-10T11:45:53.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.329 7f086d202700 1 -- 172.21.2.101:0/758839934 wait complete. 2021-02-10T11:45:53.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.343 7f086d202700 1 Processor -- start 2021-02-10T11:45:53.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.343 7f086d202700 1 -- start start 2021-02-10T11:45:53.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.344 7f086d202700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f08680bfa10 0x7f08680b82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:53.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.344 7f086d202700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08681a4a70 0x7f0868094c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:53.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.344 7f086d202700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08680c0410 0x7f08680c0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:53.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.344 7f086d202700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08680c3a50 con 0x7f08681a4a70 2021-02-10T11:45:53.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.344 7f086d202700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08680b88c0 con 0x7f08680bfa10 2021-02-10T11:45:53.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.344 7f086d202700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0868095260 con 0x7f08680c0410 2021-02-10T11:45:53.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.344 7f086659c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08681a4a70 0x7f0868094c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:53.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.344 7f086659c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08681a4a70 0x7f0868094c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32832/0 (socket says 172.21.2.101:32832) 2021-02-10T11:45:53.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.344 7f086659c700 1 -- 172.21.2.101:0/624694018 learned_addr learned my addr 172.21.2.101:0/624694018 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:53.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.344 7f086659c700 1 -- 172.21.2.101:0/624694018 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08680c0410 msgr2=0x7f08680c0810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:53.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.344 7f086659c700 1 --2- 172.21.2.101:0/624694018 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08680c0410 0x7f08680c0810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:53.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.344 7f086659c700 1 -- 172.21.2.101:0/624694018 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f08680bfa10 msgr2=0x7f08680b82a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:45:53.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.344 7f086659c700 1 --2- 172.21.2.101:0/624694018 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f08680bfa10 0x7f08680b82a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:53.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.344 7f086659c700 1 -- 172.21.2.101:0/624694018 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08680c3540 con 0x7f08681a4a70 2021-02-10T11:45:53.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.344 7f086659c700 1 --2- 172.21.2.101:0/624694018 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08681a4a70 0x7f0868094c40 secure :-1 s=READY pgs=2893 cs=0 l=1 rev1=1 rx=0x7f085000b5c0 tx=0x7f085000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:53.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.344 7f085f7fe700 1 -- 172.21.2.101:0/624694018 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f085000bd50 con 0x7f08681a4a70 2021-02-10T11:45:53.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.344 7f085f7fe700 1 -- 172.21.2.101:0/624694018 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0850006030 con 0x7f08681a4a70 2021-02-10T11:45:53.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.344 7f085f7fe700 1 -- 172.21.2.101:0/624694018 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0850006360 con 0x7f08681a4a70 2021-02-10T11:45:53.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.344 7f086d202700 1 -- 172.21.2.101:0/624694018 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08680c3540 con 0x7f08681a4a70 2021-02-10T11:45:53.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.344 7f086d202700 1 -- 172.21.2.101:0/624694018 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08680c3540 con 0x7f08681a4a70 2021-02-10T11:45:53.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.345 7f085f7fe700 1 -- 172.21.2.101:0/624694018 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0850006210 con 0x7f08681a4a70 2021-02-10T11:45:53.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.345 7f085f7fe700 1 --2- 172.21.2.101:0/624694018 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f084c0186c0 0x7f084c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:53.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.345 7f085f7fe700 1 -- 172.21.2.101:0/624694018 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0850032550 con 0x7f08681a4a70 2021-02-10T11:45:53.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.345 7f0866d9d700 1 --2- 172.21.2.101:0/624694018 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f084c0186c0 0x7f084c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:53.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.346 7f0866d9d700 1 --2- 172.21.2.101:0/624694018 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f084c0186c0 0x7f084c01abc0 crc :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:53.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.377 7f086d202700 1 -- 172.21.2.101:0/624694018 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08680bfee0 con 0x7f08681a4a70 2021-02-10T11:45:53.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.379 7f085f7fe700 1 -- 172.21.2.101:0/624694018 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0850006b80 con 0x7f08681a4a70 2021-02-10T11:45:53.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.506 7f086d202700 1 -- 172.21.2.101:0/624694018 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f08680bfee0 con 0x7f08681a4a70 2021-02-10T11:45:53.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.506 7f085f7fe700 1 -- 172.21.2.101:0/624694018 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v164) v1 ==== 100+0+10 (secure 0 0 0) 0x7f085000bd50 con 0x7f08681a4a70 2021-02-10T11:45:53.505 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:45:53.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.508 7f086d202700 1 -- 172.21.2.101:0/624694018 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f084c0186c0 msgr2=0x7f084c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:53.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.508 7f086d202700 1 --2- 172.21.2.101:0/624694018 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f084c0186c0 0x7f084c01abc0 crc :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:53.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.508 7f086d202700 1 -- 172.21.2.101:0/624694018 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08681a4a70 msgr2=0x7f0868094c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:53.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.508 7f086d202700 1 --2- 172.21.2.101:0/624694018 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08681a4a70 0x7f0868094c40 secure :-1 s=READY pgs=2893 cs=0 l=1 rev1=1 rx=0x7f085000b5c0 tx=0x7f085000b960).stop 2021-02-10T11:45:53.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.509 7f086d202700 1 -- 172.21.2.101:0/624694018 shutdown_connections 2021-02-10T11:45:53.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.509 7f086d202700 1 --2- 172.21.2.101:0/624694018 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f084c0186c0 0x7f084c01abc0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:53.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.509 7f086d202700 1 --2- 172.21.2.101:0/624694018 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f08680c0410 0x7f08680c0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:53.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.509 7f086d202700 1 --2- 172.21.2.101:0/624694018 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08681a4a70 0x7f0868094c40 unknown :-1 s=CLOSED pgs=2893 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:53.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.509 7f086d202700 1 --2- 172.21.2.101:0/624694018 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f08680bfa10 0x7f08680b82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:53.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.509 7f086d202700 1 -- 172.21.2.101:0/624694018 >> 172.21.2.101:0/624694018 conn(0x7f08680a6a80 msgr2=0x7f08680ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:53.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.509 7f086d202700 1 -- 172.21.2.101:0/624694018 shutdown_connections 2021-02-10T11:45:53.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.509 7f086d202700 1 -- 172.21.2.101:0/624694018 wait complete. 2021-02-10T11:45:53.515 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:45:53.516 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:45:53.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.929 7fe558b23700 1 Processor -- start 2021-02-10T11:45:53.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.929 7fe558b23700 1 -- start start 2021-02-10T11:45:53.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.930 7fe558b23700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe5540d1870 0x7fe5541b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:53.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.930 7fe558b23700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe5541b0a10 0x7fe5540cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:53.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.930 7fe558b23700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5540d1c70 0x7fe5540cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:53.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.930 7fe558b23700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5540e5ed0 con 0x7fe5540d1870 2021-02-10T11:45:53.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.930 7fe558b23700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5540f0b10 con 0x7fe5541b0a10 2021-02-10T11:45:53.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.930 7fe558b23700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5540cff70 con 0x7fe5540d1c70 2021-02-10T11:45:53.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.930 7fe55285e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe5540d1870 0x7fe5541b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:53.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.930 7fe55285e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe5540d1870 0x7fe5541b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32836/0 (socket says 172.21.2.101:32836) 2021-02-10T11:45:53.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.930 7fe55285e700 1 -- 172.21.2.101:0/3115708073 learned_addr learned my addr 172.21.2.101:0/3115708073 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:53.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.930 7fe55285e700 1 -- 172.21.2.101:0/3115708073 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5540d1c70 msgr2=0x7fe5540cf950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:45:53.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.930 7fe55285e700 1 --2- 172.21.2.101:0/3115708073 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5540d1c70 0x7fe5540cf950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:53.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.930 7fe55285e700 1 -- 172.21.2.101:0/3115708073 >> v2:172.21.2.108:3300/0 conn(0x7fe5541b0a10 msgr2=0x7fe5540cf330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:45:53.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.930 7fe55285e700 1 --2- 172.21.2.101:0/3115708073 >> v2:172.21.2.108:3300/0 conn(0x7fe5541b0a10 0x7fe5540cf330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:53.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.930 7fe55285e700 1 -- 172.21.2.101:0/3115708073 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5540d0150 con 0x7fe5540d1870 2021-02-10T11:45:53.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.930 7fe55285e700 1 --2- 172.21.2.101:0/3115708073 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5540d1870 0x7fe5541b03f0 secure :-1 s=READY pgs=2894 cs=0 l=1 rev1=1 rx=0x7fe53c006700 tx=0x7fe53c009b80).ready entity=mon.0 client_cookie=52340980e3e91d35 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:53.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.931 7fe55185c700 1 -- 172.21.2.101:0/3115708073 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe53c00c0c0 con 0x7fe5540d1870 2021-02-10T11:45:53.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.931 7fe55185c700 1 -- 172.21.2.101:0/3115708073 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53c00c340 con 0x7fe5540d1870 2021-02-10T11:45:53.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.931 7fe55185c700 1 -- 172.21.2.101:0/3115708073 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe53c00c860 con 0x7fe5540d1870 2021-02-10T11:45:53.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.931 7fe558b23700 1 -- 172.21.2.101:0/3115708073 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5540d1870 msgr2=0x7fe5541b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:53.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.931 7fe558b23700 1 --2- 172.21.2.101:0/3115708073 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5540d1870 0x7fe5541b03f0 secure :-1 s=READY pgs=2894 cs=0 l=1 rev1=1 rx=0x7fe53c006700 tx=0x7fe53c009b80).stop 2021-02-10T11:45:53.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.931 7fe558b23700 1 -- 172.21.2.101:0/3115708073 shutdown_connections 2021-02-10T11:45:53.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.931 7fe558b23700 1 --2- 172.21.2.101:0/3115708073 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5540d1c70 0x7fe5540cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:53.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.931 7fe558b23700 1 --2- 172.21.2.101:0/3115708073 >> v2:172.21.2.108:3300/0 conn(0x7fe5541b0a10 0x7fe5540cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:53.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.931 7fe558b23700 1 --2- 172.21.2.101:0/3115708073 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5540d1870 0x7fe5541b03f0 unknown :-1 s=CLOSED pgs=2894 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:53.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.931 7fe558b23700 1 -- 172.21.2.101:0/3115708073 >> 172.21.2.101:0/3115708073 conn(0x7fe5540ce450 msgr2=0x7fe5540d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:53.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.931 7fe558b23700 1 -- 172.21.2.101:0/3115708073 shutdown_connections 2021-02-10T11:45:53.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.932 7fe558b23700 1 -- 172.21.2.101:0/3115708073 wait complete. 2021-02-10T11:45:53.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.946 7fe558b23700 1 Processor -- start 2021-02-10T11:45:53.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.946 7fe558b23700 1 -- start start 2021-02-10T11:45:53.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.946 7fe558b23700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5541b8ad0 0x7fe554187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:53.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.946 7fe558b23700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5541b8ed0 0x7fe5540b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:53.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.946 7fe558b23700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5540b2110 0x7fe5541b7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:53.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.947 7fe558b23700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe554188470 con 0x7fe5540b2110 2021-02-10T11:45:53.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.947 7fe558b23700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55418be10 con 0x7fe5541b8ed0 2021-02-10T11:45:53.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.947 7fe558b23700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55418c030 con 0x7fe5541b8ad0 2021-02-10T11:45:53.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.947 7fe55305f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5540b2110 0x7fe5541b7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:53.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.947 7fe55305f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5540b2110 0x7fe5541b7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32846/0 (socket says 172.21.2.101:32846) 2021-02-10T11:45:53.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.947 7fe55305f700 1 -- 172.21.2.101:0/3529378024 learned_addr learned my addr 172.21.2.101:0/3529378024 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:53.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.947 7fe55285e700 1 --2- 172.21.2.101:0/3529378024 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5541b8ad0 0x7fe554187e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:53.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.947 7fe55205d700 1 --2- 172.21.2.101:0/3529378024 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5541b8ed0 0x7fe5540b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:53.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.947 7fe55305f700 1 -- 172.21.2.101:0/3529378024 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5541b8ad0 msgr2=0x7fe554187e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:53.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.947 7fe55305f700 1 --2- 172.21.2.101:0/3529378024 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5541b8ad0 0x7fe554187e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:53.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.947 7fe55305f700 1 -- 172.21.2.101:0/3529378024 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5541b8ed0 msgr2=0x7fe5540b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:53.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.947 7fe55305f700 1 --2- 172.21.2.101:0/3529378024 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5541b8ed0 0x7fe5540b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:53.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.947 7fe55305f700 1 -- 172.21.2.101:0/3529378024 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5540b2510 con 0x7fe5540b2110 2021-02-10T11:45:53.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.947 7fe55305f700 1 --2- 172.21.2.101:0/3529378024 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5540b2110 0x7fe5541b7a00 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 rx=0x7fe54c009960 tx=0x7fe54c00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:53.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.948 7fe54b7fe700 1 -- 172.21.2.101:0/3529378024 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe54c003030 con 0x7fe5540b2110 2021-02-10T11:45:53.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.948 7fe54b7fe700 1 -- 172.21.2.101:0/3529378024 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54c001310 con 0x7fe5540b2110 2021-02-10T11:45:53.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.948 7fe558b23700 1 -- 172.21.2.101:0/3529378024 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5540b2510 con 0x7fe5540b2110 2021-02-10T11:45:53.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.948 7fe558b23700 1 -- 172.21.2.101:0/3529378024 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5541b8020 con 0x7fe5540b2110 2021-02-10T11:45:53.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.948 7fe54b7fe700 1 -- 172.21.2.101:0/3529378024 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe54c001770 con 0x7fe5540b2110 2021-02-10T11:45:53.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.948 7fe54b7fe700 1 -- 172.21.2.101:0/3529378024 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe54c0014f0 con 0x7fe5540b2110 2021-02-10T11:45:53.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.948 7fe54b7fe700 1 --2- 172.21.2.101:0/3529378024 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5380186c0 0x7fe53801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:53.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.949 7fe54b7fe700 1 -- 172.21.2.101:0/3529378024 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe54c032900 con 0x7fe5540b2110 2021-02-10T11:45:53.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.949 7fe55285e700 1 --2- 172.21.2.101:0/3529378024 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5380186c0 0x7fe53801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:53.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.949 7fe55285e700 1 --2- 172.21.2.101:0/3529378024 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5380186c0 0x7fe53801abc0 crc :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:53.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.980 7fe558b23700 1 -- 172.21.2.101:0/3529378024 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5541b88b0 con 0x7fe5540b2110 2021-02-10T11:45:53.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:53.982 7fe54b7fe700 1 -- 172.21.2.101:0/3529378024 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe54c02cf50 con 0x7fe5540b2110 2021-02-10T11:45:54.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:54.110 7fe558b23700 1 -- 172.21.2.101:0/3529378024 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5541b88b0 con 0x7fe5540b2110 2021-02-10T11:45:54.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:54.110 7fe54b7fe700 1 -- 172.21.2.101:0/3529378024 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+2104 (secure 0 0 0) 0x7fe54c02cf50 con 0x7fe5540b2110 2021-02-10T11:45:54.109 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 139 2021-02-10T11:45:54.109 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:45:54.109 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":139,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8371,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/453344112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":453344112},{"type":"v1","addr":"172.21.2.101:6835","nonce":453344112}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":135}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:45:44.128996","modified":"2021-02-10 11:45:46.139945","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7097":{"gid":7097,"name":"b","rank":0,"incarnation":136,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2456711519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2456711519},{"type":"v1","addr":"172.21.2.108:6833","nonce":2456711519}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":17}]} 2021-02-10T11:45:54.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:54.112 7fe558b23700 1 -- 172.21.2.101:0/3529378024 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5380186c0 msgr2=0x7fe53801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:54.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:54.112 7fe558b23700 1 --2- 172.21.2.101:0/3529378024 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5380186c0 0x7fe53801abc0 crc :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:54.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:54.112 7fe558b23700 1 -- 172.21.2.101:0/3529378024 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5540b2110 msgr2=0x7fe5541b7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:54.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:54.112 7fe558b23700 1 --2- 172.21.2.101:0/3529378024 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5540b2110 0x7fe5541b7a00 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 rx=0x7fe54c009960 tx=0x7fe54c00bcf0).stop 2021-02-10T11:45:54.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:54.112 7fe558b23700 1 -- 172.21.2.101:0/3529378024 shutdown_connections 2021-02-10T11:45:54.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:54.112 7fe558b23700 1 --2- 172.21.2.101:0/3529378024 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe5540b2110 0x7fe5541b7a00 unknown :-1 s=CLOSED pgs=2895 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:54.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:54.113 7fe558b23700 1 --2- 172.21.2.101:0/3529378024 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe5541b8ed0 0x7fe5540b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:54.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:54.113 7fe558b23700 1 --2- 172.21.2.101:0/3529378024 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe5380186c0 0x7fe53801abc0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:54.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:54.113 7fe558b23700 1 --2- 172.21.2.101:0/3529378024 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe5541b8ad0 0x7fe554187e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:54.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:54.113 7fe558b23700 1 -- 172.21.2.101:0/3529378024 >> 172.21.2.101:0/3529378024 conn(0x7fe5540ed810 msgr2=0x7fe554094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:54.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:54.113 7fe558b23700 1 -- 172.21.2.101:0/3529378024 shutdown_connections 2021-02-10T11:45:54.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:54.113 7fe558b23700 1 -- 172.21.2.101:0/3529378024 wait complete. 2021-02-10T11:45:54.321 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 139, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:45:44.128996', 'modified': '2021-02-10 11:45:46.139945', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7097}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7097': {'gid': 7097, 'name': 'b', 'rank': 0, 'incarnation': 136, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/2456711519', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 2456711519}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 2456711519}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [35], 'metadata_pool': 34, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:45:54.321 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:45:54.321 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:45:54.720 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:45:54.720 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:45:54.720 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:45:54.720 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 7097, 2021-02-10T11:45:54.720 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:45:54.720 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:45:54.720 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 10.591609350000001, 2021-02-10T11:45:54.721 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 139, 2021-02-10T11:45:54.721 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 164, 2021-02-10T11:45:54.721 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 164, 2021-02-10T11:45:54.721 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 16.240987451999999 2021-02-10T11:45:54.721 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:45:54.724 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 7097, "want_state": "up:active", "state": "up:active", "rank_uptime": 10.591609350000001, "mdsmap_epoch": 139, "osdmap_epoch": 164, "osdmap_epoch_barrier": 164, "uptime": 16.240987451999999 } 2021-02-10T11:45:54.724 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:45:54.724 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-02-10T11:45:54.724 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-02-10T11:45:54.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:54.725 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:45:55.058 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:45:55.404 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:45:55.404 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:45:55.747 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:45:55.747 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:45:55.748 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:45:55.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:55.748 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:45:56.083 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-02-10T11:45:56.083 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:45:56.086 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:45:56.429 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:45:56.430 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:45:56.496 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[169637]: starting ceph client 2021-02-10T11:45:56.496 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:2021-02-10 11:45:56.498 7f79c0a23f80 -1 init, newargv = 0x5635674a01f0 newargc=9 2021-02-10T11:45:56.502 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[169637]: starting fuse 2021-02-10T11:45:56.773 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:45:56.774 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:45:56.775 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:45:56.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:56.775 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:45:57.109 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:45:57.110 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2021-02-10T11:45:57.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:57.110 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:45:57.110 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:45:57.110 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:45:57.110 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:45:57.110 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:45:57.110 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:45:57.111 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:45:57.111 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-02-10T11:45:57.111 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:45:57.111 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:45:57.111 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:45:57.111 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:45:57.111 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:45:57.111 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:45:57.111 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:45:57.111 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:45:57.111 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:45:57.111 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:45:57.111 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:45:57.111 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:45:57.111 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:45:57.112 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.0")) 2021-02-10T11:45:57.112 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:45:57.472 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.0.169637.asok 2021-02-10T11:45:57.475 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.169637.asok 2021-02-10T11:45:57.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:57.476 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.169637.asok status 2021-02-10T11:45:57.867 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:45:57.867 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:45:57.867 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:45:57.868 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:45:57.868 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "0", 2021-02-10T11:45:57.868 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:45:57.868 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-02-10T11:45:57.868 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "169637", 2021-02-10T11:45:57.868 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:45:57.868 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:57.868 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:45:57.868 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:45:57.868 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 12594, 2021-02-10T11:45:57.868 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:45:57.868 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:45:57.868 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:45:57.869 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 12594 2021-02-10T11:45:57.869 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:57.869 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:45:57.869 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:57.869 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:45:57.869 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1544711996 2021-02-10T11:45:57.869 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:45:57.869 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:57.869 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:45:57.869 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:45:57.869 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:45:57.869 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1544711996 2021-02-10T11:45:57.869 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:45:57.869 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.12594 172.21.2.101:0/1544711996", 2021-02-10T11:45:57.870 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/1544711996", 2021-02-10T11:45:57.870 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:45:57.870 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 139, 2021-02-10T11:45:57.870 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 164, 2021-02-10T11:45:57.870 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:45:57.870 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:45:57.870 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:45:57.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:57.872 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:45:57.872 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:45:58.207 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:45:58.207 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:45:58.207 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:45:58.208 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:45:58.553 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:45:58.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.967 7f4ea90cc700 1 Processor -- start 2021-02-10T11:45:58.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.967 7f4ea90cc700 1 -- start start 2021-02-10T11:45:58.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.968 7f4ea90cc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea40d1870 0x7f4ea41b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:58.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.968 7f4ea90cc700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f4ea41b0a10 0x7f4ea40cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:58.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.968 7f4ea90cc700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea40f0b10 con 0x7f4ea40d1c70 2021-02-10T11:45:58.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.968 7f4ea90cc700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea40fe230 con 0x7f4ea41b0a10 2021-02-10T11:45:58.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.968 7f4ea90cc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea40e5ed0 con 0x7f4ea40d1870 2021-02-10T11:45:58.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.968 7f4ea259c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f4ea41b0a10 0x7f4ea40cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:58.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.968 7f4ea259c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f4ea41b0a10 0x7f4ea40cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32866/0 (socket says 172.21.2.101:32866) 2021-02-10T11:45:58.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.968 7f4ea259c700 1 -- 172.21.2.101:0/341551683 learned_addr learned my addr 172.21.2.101:0/341551683 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:58.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.968 7f4ea259c700 1 -- 172.21.2.101:0/341551683 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea40d1870 msgr2=0x7f4ea41b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:45:58.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.968 7f4ea259c700 1 --2- 172.21.2.101:0/341551683 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea40d1870 0x7f4ea41b03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:58.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.968 7f4ea259c700 1 -- 172.21.2.101:0/341551683 >> v1:172.21.2.108:6789/0 conn(0x7f4ea40d1c70 legacy=0x7f4ea40cf950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:45:58.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.968 7f4ea259c700 1 -- 172.21.2.101:0/341551683 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea40f0650 con 0x7f4ea41b0a10 2021-02-10T11:45:58.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.968 7f4ea259c700 1 --2- 172.21.2.101:0/341551683 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ea41b0a10 0x7f4ea40cf330 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 rx=0x7f4e94006700 tx=0x7f4e94009b80).ready entity=mon.0 client_cookie=154c0fb494063f2c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:58.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.968 7f4ea1d9b700 1 -- 172.21.2.101:0/341551683 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e9400c0c0 con 0x7f4ea41b0a10 2021-02-10T11:45:58.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.968 7f4ea1d9b700 1 -- 172.21.2.101:0/341551683 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9400c440 con 0x7f4ea41b0a10 2021-02-10T11:45:58.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.968 7f4ea1d9b700 1 -- 172.21.2.101:0/341551683 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e9400c920 con 0x7f4ea41b0a10 2021-02-10T11:45:58.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.969 7f4ea90cc700 1 -- 172.21.2.101:0/341551683 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ea41b0a10 msgr2=0x7f4ea40cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:58.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.969 7f4ea90cc700 1 --2- 172.21.2.101:0/341551683 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ea41b0a10 0x7f4ea40cf330 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 rx=0x7f4e94006700 tx=0x7f4e94009b80).stop 2021-02-10T11:45:58.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.969 7f4ea90cc700 1 -- 172.21.2.101:0/341551683 shutdown_connections 2021-02-10T11:45:58.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.969 7f4ea90cc700 1 --2- 172.21.2.101:0/341551683 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ea41b0a10 0x7f4ea40cf330 unknown :-1 s=CLOSED pgs=2898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:58.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.969 7f4ea90cc700 1 --2- 172.21.2.101:0/341551683 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea40d1870 0x7f4ea41b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:45:58.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.969 7f4ea90cc700 1 -- 172.21.2.101:0/341551683 >> 172.21.2.101:0/341551683 conn(0x7f4ea40ce450 msgr2=0x7f4ea40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:58.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.969 7f4ea90cc700 1 -- 172.21.2.101:0/341551683 shutdown_connections 2021-02-10T11:45:58.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.969 7f4ea90cc700 1 -- 172.21.2.101:0/341551683 wait complete. 2021-02-10T11:45:58.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.984 7f4ea90cc700 1 Processor -- start 2021-02-10T11:45:58.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.984 7f4ea90cc700 1 -- start start 2021-02-10T11:45:58.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.984 7f4ea90cc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea40b4f30 0x7f4ea418b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:58.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.984 7f4ea90cc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ea4093f40 0x7f4ea4094340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:58.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.984 7f4ea90cc700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ea4094960 0x7f4ea4187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:58.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.984 7f4ea90cc700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea4094e90 con 0x7f4ea4093f40 2021-02-10T11:45:58.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.984 7f4ea90cc700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea418bff0 con 0x7f4ea4094960 2021-02-10T11:45:58.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.984 7f4ea90cc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea408f780 con 0x7f4ea40b4f30 2021-02-10T11:45:58.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.984 7f4ea259c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ea4093f40 0x7f4ea4094340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:58.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.985 7f4ea259c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ea4093f40 0x7f4ea4094340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32870/0 (socket says 172.21.2.101:32870) 2021-02-10T11:45:58.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.985 7f4ea259c700 1 -- 172.21.2.101:0/4226806623 learned_addr learned my addr 172.21.2.101:0/4226806623 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:58.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.985 7f4ea359e700 1 --2- 172.21.2.101:0/4226806623 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ea4094960 0x7f4ea4187e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:58.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.985 7f4ea259c700 1 -- 172.21.2.101:0/4226806623 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea40b4f30 msgr2=0x7f4ea418b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:58.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.985 7f4ea2d9d700 1 --2- 172.21.2.101:0/4226806623 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea40b4f30 0x7f4ea418b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:58.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.985 7f4ea259c700 1 --2- 172.21.2.101:0/4226806623 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea40b4f30 0x7f4ea418b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:58.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.985 7f4ea259c700 1 -- 172.21.2.101:0/4226806623 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ea4094960 msgr2=0x7f4ea4187e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:58.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.985 7f4ea259c700 1 --2- 172.21.2.101:0/4226806623 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ea4094960 0x7f4ea4187e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:58.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.985 7f4ea259c700 1 -- 172.21.2.101:0/4226806623 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea4188470 con 0x7f4ea4093f40 2021-02-10T11:45:58.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.985 7f4ea359e700 1 --2- 172.21.2.101:0/4226806623 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ea4094960 0x7f4ea4187e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:58.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.985 7f4ea2d9d700 1 --2- 172.21.2.101:0/4226806623 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea40b4f30 0x7f4ea418b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:45:58.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.985 7f4ea259c700 1 --2- 172.21.2.101:0/4226806623 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ea4093f40 0x7f4ea4094340 secure :-1 s=READY pgs=2899 cs=0 l=1 rev1=1 rx=0x7f4e9400b5e0 tx=0x7f4e9400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:58.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.985 7f4e9b7fe700 1 -- 172.21.2.101:0/4226806623 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e9400bd70 con 0x7f4ea4093f40 2021-02-10T11:45:58.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.985 7f4e9b7fe700 1 -- 172.21.2.101:0/4226806623 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e94002030 con 0x7f4ea4093f40 2021-02-10T11:45:58.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.985 7f4ea90cc700 1 -- 172.21.2.101:0/4226806623 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea4188470 con 0x7f4ea4093f40 2021-02-10T11:45:58.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.985 7f4ea90cc700 1 -- 172.21.2.101:0/4226806623 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea4188a00 con 0x7f4ea4093f40 2021-02-10T11:45:58.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.986 7f4e9b7fe700 1 -- 172.21.2.101:0/4226806623 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e94002550 con 0x7f4ea4093f40 2021-02-10T11:45:58.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.986 7f4e9b7fe700 1 -- 172.21.2.101:0/4226806623 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4e94002210 con 0x7f4ea4093f40 2021-02-10T11:45:58.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.986 7f4e9b7fe700 1 --2- 172.21.2.101:0/4226806623 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4e88018730 0x7f4e8801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:58.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.986 7f4e9b7fe700 1 -- 172.21.2.101:0/4226806623 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4e94032780 con 0x7f4ea4093f40 2021-02-10T11:45:58.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.986 7f4ea2d9d700 1 --2- 172.21.2.101:0/4226806623 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4e88018730 0x7f4e8801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:58.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:58.987 7f4ea2d9d700 1 --2- 172.21.2.101:0/4226806623 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4e88018730 0x7f4e8801ac10 crc :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:59.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.018 7f4ea90cc700 1 -- 172.21.2.101:0/4226806623 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea408f760 con 0x7f4ea4093f40 2021-02-10T11:45:59.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.020 7f4e9b7fe700 1 -- 172.21.2.101:0/4226806623 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4e940009b0 con 0x7f4ea4093f40 2021-02-10T11:45:59.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.144 7f4ea90cc700 1 -- 172.21.2.101:0/4226806623 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ea408f760 con 0x7f4ea4093f40 2021-02-10T11:45:59.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.145 7f4e9b7fe700 1 -- 172.21.2.101:0/4226806623 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f4e94002030 con 0x7f4ea4093f40 2021-02-10T11:45:59.143 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:45:59.144 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":164,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:45:45.131083","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:45:42.100593","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":35,"pool_name":"cephfs_data","create_time":"2021-02-10 11:45:43.124384","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:45:59.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.147 7f4ea90cc700 1 -- 172.21.2.101:0/4226806623 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4e88018730 msgr2=0x7f4e8801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:59.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.147 7f4ea90cc700 1 --2- 172.21.2.101:0/4226806623 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4e88018730 0x7f4e8801ac10 crc :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:59.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.147 7f4ea90cc700 1 -- 172.21.2.101:0/4226806623 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ea4093f40 msgr2=0x7f4ea4094340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:59.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.147 7f4ea90cc700 1 --2- 172.21.2.101:0/4226806623 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ea4093f40 0x7f4ea4094340 secure :-1 s=READY pgs=2899 cs=0 l=1 rev1=1 rx=0x7f4e9400b5e0 tx=0x7f4e9400b980).stop 2021-02-10T11:45:59.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.147 7f4ea90cc700 1 -- 172.21.2.101:0/4226806623 shutdown_connections 2021-02-10T11:45:59.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.147 7f4ea90cc700 1 --2- 172.21.2.101:0/4226806623 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4ea4094960 0x7f4ea4187e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:59.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.147 7f4ea90cc700 1 --2- 172.21.2.101:0/4226806623 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4ea4093f40 0x7f4ea4094340 unknown :-1 s=CLOSED pgs=2899 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:59.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.147 7f4ea90cc700 1 --2- 172.21.2.101:0/4226806623 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4e88018730 0x7f4e8801ac10 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:59.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.147 7f4ea90cc700 1 --2- 172.21.2.101:0/4226806623 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4ea40b4f30 0x7f4ea418b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:59.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.147 7f4ea90cc700 1 -- 172.21.2.101:0/4226806623 >> 172.21.2.101:0/4226806623 conn(0x7f4ea419f8f0 msgr2=0x7f4ea41a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:59.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.147 7f4ea90cc700 1 -- 172.21.2.101:0/4226806623 shutdown_connections 2021-02-10T11:45:59.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.147 7f4ea90cc700 1 -- 172.21.2.101:0/4226806623 wait complete. 2021-02-10T11:45:59.154 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 34, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:45:42.100593', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '163', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 35, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:45:43.124384', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '164', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:45:59.154 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:45:59.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.571 7ff6b1646700 1 Processor -- start 2021-02-10T11:45:59.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.571 7ff6b1646700 1 -- start start 2021-02-10T11:45:59.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.572 7ff6b1646700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff6ac0d1c70 0x7ff6ac0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:59.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.572 7ff6b1646700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6ac0cf580 0x7ff6ac0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:59.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.572 7ff6b1646700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ac0fe1c0 con 0x7ff6ac0d1870 2021-02-10T11:45:59.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.572 7ff6b1646700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ac1b0cb0 con 0x7ff6ac0d1c70 2021-02-10T11:45:59.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.572 7ff6b1646700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac109a60 con 0x7ff6ac0cf580 2021-02-10T11:45:59.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.572 7ff6aaabe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff6ac0d1c70 0x7ff6ac0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:59.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.572 7ff6aaabe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff6ac0d1c70 0x7ff6ac0cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32878/0 (socket says 172.21.2.101:32878) 2021-02-10T11:45:59.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.572 7ff6aaabe700 1 -- 172.21.2.101:0/1861183021 learned_addr learned my addr 172.21.2.101:0/1861183021 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:59.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.572 7ff6abac0700 1 --2- 172.21.2.101:0/1861183021 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6ac0cf580 0x7ff6ac0cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:59.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.572 7ff6aaabe700 1 -- 172.21.2.101:0/1861183021 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6ac0cf580 msgr2=0x7ff6ac0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:59.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.572 7ff6aaabe700 1 --2- 172.21.2.101:0/1861183021 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6ac0cf580 0x7ff6ac0cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:59.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.572 7ff6aaabe700 1 -- 172.21.2.101:0/1861183021 >> v1:172.21.2.108:6789/0 conn(0x7ff6ac0d1870 legacy=0x7ff6ac1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:45:59.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.572 7ff6aaabe700 1 -- 172.21.2.101:0/1861183021 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac088070 con 0x7ff6ac0d1c70 2021-02-10T11:45:59.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.572 7ff6aaabe700 1 --2- 172.21.2.101:0/1861183021 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6ac0d1c70 0x7ff6ac0cef60 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 rx=0x7ff694006700 tx=0x7ff694009b80).ready entity=mon.0 client_cookie=3f265732dfb511d1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:59.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.573 7ff6aa2bd700 1 -- 172.21.2.101:0/1861183021 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff69400c0c0 con 0x7ff6ac0d1c70 2021-02-10T11:45:59.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.573 7ff6aa2bd700 1 -- 172.21.2.101:0/1861183021 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69400c400 con 0x7ff6ac0d1c70 2021-02-10T11:45:59.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.573 7ff6aa2bd700 1 -- 172.21.2.101:0/1861183021 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff69400c920 con 0x7ff6ac0d1c70 2021-02-10T11:45:59.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.573 7ff6b1646700 1 -- 172.21.2.101:0/1861183021 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6ac0d1c70 msgr2=0x7ff6ac0cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:59.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.573 7ff6b1646700 1 --2- 172.21.2.101:0/1861183021 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6ac0d1c70 0x7ff6ac0cef60 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 rx=0x7ff694006700 tx=0x7ff694009b80).stop 2021-02-10T11:45:59.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.573 7ff6b1646700 1 -- 172.21.2.101:0/1861183021 shutdown_connections 2021-02-10T11:45:59.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.573 7ff6b1646700 1 --2- 172.21.2.101:0/1861183021 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6ac0cf580 0x7ff6ac0cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:59.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.573 7ff6b1646700 1 --2- 172.21.2.101:0/1861183021 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6ac0d1c70 0x7ff6ac0cef60 unknown :-1 s=CLOSED pgs=2900 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:59.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.573 7ff6b1646700 1 -- 172.21.2.101:0/1861183021 >> 172.21.2.101:0/1861183021 conn(0x7ff6ac0ce420 msgr2=0x7ff6ac0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:59.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.573 7ff6b1646700 1 -- 172.21.2.101:0/1861183021 shutdown_connections 2021-02-10T11:45:59.588 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.573 7ff6b1646700 1 -- 172.21.2.101:0/1861183021 wait complete. 2021-02-10T11:45:59.589 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.588 7ff6b1646700 1 Processor -- start 2021-02-10T11:45:59.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.588 7ff6b1646700 1 -- start start 2021-02-10T11:45:59.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.589 7ff6b1646700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6ac085bc0 0x7ff6ac0863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:59.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.589 7ff6b1646700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6ac0954c0 0x7ff6ac096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:59.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.589 7ff6b1646700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6ac0958c0 0x7ff6ac096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:59.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.589 7ff6b1646700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac095d80 con 0x7ff6ac085bc0 2021-02-10T11:45:59.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.589 7ff6b1646700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac0bfa60 con 0x7ff6ac0958c0 2021-02-10T11:45:59.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.589 7ff6b1646700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac085fc0 con 0x7ff6ac0954c0 2021-02-10T11:45:59.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.589 7ff6ab2bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6ac085bc0 0x7ff6ac0863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:59.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.589 7ff6ab2bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6ac085bc0 0x7ff6ac0863e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32884/0 (socket says 172.21.2.101:32884) 2021-02-10T11:45:59.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.589 7ff6ab2bf700 1 -- 172.21.2.101:0/1483002977 learned_addr learned my addr 172.21.2.101:0/1483002977 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:45:59.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.589 7ff6aaabe700 1 --2- 172.21.2.101:0/1483002977 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6ac0954c0 0x7ff6ac096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:59.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.589 7ff6abac0700 1 --2- 172.21.2.101:0/1483002977 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6ac0958c0 0x7ff6ac096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:59.590 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.589 7ff6ab2bf700 1 -- 172.21.2.101:0/1483002977 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6ac0954c0 msgr2=0x7ff6ac096300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:59.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.589 7ff6ab2bf700 1 --2- 172.21.2.101:0/1483002977 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6ac0954c0 0x7ff6ac096300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:59.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.589 7ff6ab2bf700 1 -- 172.21.2.101:0/1483002977 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6ac0958c0 msgr2=0x7ff6ac096920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:59.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.589 7ff6ab2bf700 1 --2- 172.21.2.101:0/1483002977 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6ac0958c0 0x7ff6ac096920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:59.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.589 7ff6ab2bf700 1 -- 172.21.2.101:0/1483002977 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac086c10 con 0x7ff6ac085bc0 2021-02-10T11:45:59.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.590 7ff6ab2bf700 1 --2- 172.21.2.101:0/1483002977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6ac085bc0 0x7ff6ac0863e0 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 rx=0x7ff69c00b5b0 tx=0x7ff69c00b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:59.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.590 7ff6a3fff700 1 -- 172.21.2.101:0/1483002977 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff69c00bd40 con 0x7ff6ac085bc0 2021-02-10T11:45:59.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.590 7ff6a3fff700 1 -- 172.21.2.101:0/1483002977 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c006240 con 0x7ff6ac085bc0 2021-02-10T11:45:59.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.590 7ff6a3fff700 1 -- 172.21.2.101:0/1483002977 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff69c005690 con 0x7ff6ac085bc0 2021-02-10T11:45:59.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.590 7ff6b1646700 1 -- 172.21.2.101:0/1483002977 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ac086c10 con 0x7ff6ac085bc0 2021-02-10T11:45:59.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.590 7ff6b1646700 1 -- 172.21.2.101:0/1483002977 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ac086c10 con 0x7ff6ac085bc0 2021-02-10T11:45:59.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.590 7ff6a3fff700 1 -- 172.21.2.101:0/1483002977 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff69c0054a0 con 0x7ff6ac085bc0 2021-02-10T11:45:59.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.590 7ff6a3fff700 1 --2- 172.21.2.101:0/1483002977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6900186c0 0x7ff69001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:45:59.591 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.590 7ff6a3fff700 1 -- 172.21.2.101:0/1483002977 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff69c032650 con 0x7ff6ac085bc0 2021-02-10T11:45:59.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.590 7ff6aaabe700 1 --2- 172.21.2.101:0/1483002977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6900186c0 0x7ff69001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:45:59.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.591 7ff6aaabe700 1 --2- 172.21.2.101:0/1483002977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6900186c0 0x7ff69001abc0 crc :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:45:59.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.622 7ff6b1646700 1 -- 172.21.2.101:0/1483002977 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ac0fe1c0 con 0x7ff6ac085bc0 2021-02-10T11:45:59.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.624 7ff6a3fff700 1 -- 172.21.2.101:0/1483002977 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff69c0009b0 con 0x7ff6ac085bc0 2021-02-10T11:45:59.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.751 7ff6b1646700 1 -- 172.21.2.101:0/1483002977 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7ff6ac0fe1c0 con 0x7ff6ac085bc0 2021-02-10T11:45:59.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.752 7ff6a3fff700 1 -- 172.21.2.101:0/1483002977 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v164) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff69c0054a0 con 0x7ff6ac085bc0 2021-02-10T11:45:59.750 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:45:59.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.753 7ff6b1646700 1 -- 172.21.2.101:0/1483002977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6900186c0 msgr2=0x7ff69001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:59.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.753 7ff6b1646700 1 --2- 172.21.2.101:0/1483002977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6900186c0 0x7ff69001abc0 crc :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:59.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.753 7ff6b1646700 1 -- 172.21.2.101:0/1483002977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6ac085bc0 msgr2=0x7ff6ac0863e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:45:59.752 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.754 7ff6b1646700 1 --2- 172.21.2.101:0/1483002977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6ac085bc0 0x7ff6ac0863e0 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 rx=0x7ff69c00b5b0 tx=0x7ff69c00b950).stop 2021-02-10T11:45:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.754 7ff6b1646700 1 -- 172.21.2.101:0/1483002977 shutdown_connections 2021-02-10T11:45:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.754 7ff6b1646700 1 --2- 172.21.2.101:0/1483002977 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6900186c0 0x7ff69001abc0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.754 7ff6b1646700 1 --2- 172.21.2.101:0/1483002977 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6ac0954c0 0x7ff6ac096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.754 7ff6b1646700 1 --2- 172.21.2.101:0/1483002977 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6ac0958c0 0x7ff6ac096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.754 7ff6b1646700 1 --2- 172.21.2.101:0/1483002977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6ac085bc0 0x7ff6ac0863e0 unknown :-1 s=CLOSED pgs=2901 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:45:59.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.754 7ff6b1646700 1 -- 172.21.2.101:0/1483002977 >> 172.21.2.101:0/1483002977 conn(0x7ff6ac0bef00 msgr2=0x7ff6ac09d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:45:59.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.754 7ff6b1646700 1 -- 172.21.2.101:0/1483002977 shutdown_connections 2021-02-10T11:45:59.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:45:59.754 7ff6b1646700 1 -- 172.21.2.101:0/1483002977 wait complete. 2021-02-10T11:45:59.760 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:46:00.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.178 7f58541b3700 1 Processor -- start 2021-02-10T11:46:00.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.178 7f58541b3700 1 -- start start 2021-02-10T11:46:00.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.179 7f58541b3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f584c0d1870 0x7f584c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:00.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.179 7f58541b3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f584c1b0ae0 0x7f584c108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:00.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.179 7f58541b3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f584c0d1c70 0x7f584c108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:00.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.179 7f58541b3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f584c0fe1c0 con 0x7f584c0d1870 2021-02-10T11:46:00.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.179 7f58541b3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f584c109a60 con 0x7f584c1b0ae0 2021-02-10T11:46:00.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.179 7f58541b3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584c088070 con 0x7f584c0d1c70 2021-02-10T11:46:00.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.179 7f5851fcf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f584c0d1870 0x7f584c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:00.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.179 7f5851fcf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f584c0d1870 0x7f584c1b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32892/0 (socket says 172.21.2.101:32892) 2021-02-10T11:46:00.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.179 7f5851fcf700 1 -- 172.21.2.101:0/2605038128 learned_addr learned my addr 172.21.2.101:0/2605038128 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:00.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.179 7f5851fcf700 1 -- 172.21.2.101:0/2605038128 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f584c0d1c70 msgr2=0x7f584c108f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:46:00.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.180 7f5851fcf700 1 --2- 172.21.2.101:0/2605038128 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f584c0d1c70 0x7f584c108f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:00.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.180 7f5851fcf700 1 -- 172.21.2.101:0/2605038128 >> v2:172.21.2.108:3300/0 conn(0x7f584c1b0ae0 msgr2=0x7f584c108940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:46:00.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.180 7f5851fcf700 1 --2- 172.21.2.101:0/2605038128 >> v2:172.21.2.108:3300/0 conn(0x7f584c1b0ae0 0x7f584c108940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:00.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.180 7f5851fcf700 1 -- 172.21.2.101:0/2605038128 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584c109580 con 0x7f584c0d1870 2021-02-10T11:46:00.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.180 7f5851fcf700 1 --2- 172.21.2.101:0/2605038128 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f584c0d1870 0x7f584c1b04c0 secure :-1 s=READY pgs=2902 cs=0 l=1 rev1=1 rx=0x7f5848006700 tx=0x7f5848009b80).ready entity=mon.0 client_cookie=664dc0185bc7801e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:00.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.180 7f5850fcd700 1 -- 172.21.2.101:0/2605038128 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f584800c0c0 con 0x7f584c0d1870 2021-02-10T11:46:00.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.180 7f5850fcd700 1 -- 172.21.2.101:0/2605038128 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584800c340 con 0x7f584c0d1870 2021-02-10T11:46:00.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.180 7f5850fcd700 1 -- 172.21.2.101:0/2605038128 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f584800c860 con 0x7f584c0d1870 2021-02-10T11:46:00.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.180 7f58541b3700 1 -- 172.21.2.101:0/2605038128 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f584c0d1870 msgr2=0x7f584c1b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:00.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.180 7f58541b3700 1 --2- 172.21.2.101:0/2605038128 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f584c0d1870 0x7f584c1b04c0 secure :-1 s=READY pgs=2902 cs=0 l=1 rev1=1 rx=0x7f5848006700 tx=0x7f5848009b80).stop 2021-02-10T11:46:00.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.181 7f58541b3700 1 -- 172.21.2.101:0/2605038128 shutdown_connections 2021-02-10T11:46:00.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.181 7f58541b3700 1 --2- 172.21.2.101:0/2605038128 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f584c0d1c70 0x7f584c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:00.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.181 7f58541b3700 1 --2- 172.21.2.101:0/2605038128 >> v2:172.21.2.108:3300/0 conn(0x7f584c1b0ae0 0x7f584c108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:00.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.181 7f58541b3700 1 --2- 172.21.2.101:0/2605038128 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f584c0d1870 0x7f584c1b04c0 unknown :-1 s=CLOSED pgs=2902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:00.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.181 7f58541b3700 1 -- 172.21.2.101:0/2605038128 >> 172.21.2.101:0/2605038128 conn(0x7f584c0ce420 msgr2=0x7f584c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:00.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.181 7f58541b3700 1 -- 172.21.2.101:0/2605038128 shutdown_connections 2021-02-10T11:46:00.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.181 7f58541b3700 1 -- 172.21.2.101:0/2605038128 wait complete. 2021-02-10T11:46:00.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.196 7f58541b3700 1 Processor -- start 2021-02-10T11:46:00.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.196 7f58541b3700 1 -- start start 2021-02-10T11:46:00.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.196 7f58541b3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f584c0c3820 0x7f584c084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:00.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.196 7f58541b3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f584c0a0290 0x7f584c0852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:00.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.196 7f58541b3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f584c086270 0x7f584c086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:00.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.196 7f58541b3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c0c0700 con 0x7f584c0a0290 2021-02-10T11:46:00.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.196 7f58541b3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c0858f0 con 0x7f584c086270 2021-02-10T11:46:00.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.196 7f58541b3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584c085b10 con 0x7f584c0c3820 2021-02-10T11:46:00.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.196 7f58517ce700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f584c0a0290 0x7f584c0852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:00.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.196 7f58517ce700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f584c0a0290 0x7f584c0852d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32898/0 (socket says 172.21.2.101:32898) 2021-02-10T11:46:00.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.197 7f58517ce700 1 -- 172.21.2.101:0/3803129637 learned_addr learned my addr 172.21.2.101:0/3803129637 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:00.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.197 7f58517ce700 1 -- 172.21.2.101:0/3803129637 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f584c0c3820 msgr2=0x7f584c084cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:00.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.197 7f5851fcf700 1 --2- 172.21.2.101:0/3803129637 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f584c0c3820 0x7f584c084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:00.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.197 7f58527d0700 1 --2- 172.21.2.101:0/3803129637 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f584c086270 0x7f584c086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:00.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.197 7f58517ce700 1 --2- 172.21.2.101:0/3803129637 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f584c0c3820 0x7f584c084cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:00.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.197 7f58517ce700 1 -- 172.21.2.101:0/3803129637 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f584c086270 msgr2=0x7f584c086670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:00.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.197 7f58517ce700 1 --2- 172.21.2.101:0/3803129637 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f584c086270 0x7f584c086670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:00.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.197 7f58517ce700 1 -- 172.21.2.101:0/3803129637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584c086c90 con 0x7f584c0a0290 2021-02-10T11:46:00.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.197 7f5851fcf700 1 --2- 172.21.2.101:0/3803129637 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f584c0c3820 0x7f584c084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:46:00.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.197 7f58527d0700 1 --2- 172.21.2.101:0/3803129637 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f584c086270 0x7f584c086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:46:00.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.197 7f58517ce700 1 --2- 172.21.2.101:0/3803129637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f584c0a0290 0x7f584c0852d0 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 rx=0x7f583c00b5c0 tx=0x7f583c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:00.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.197 7f58427fc700 1 -- 172.21.2.101:0/3803129637 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f583c0009b0 con 0x7f584c0a0290 2021-02-10T11:46:00.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.198 7f58427fc700 1 -- 172.21.2.101:0/3803129637 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583c00bd50 con 0x7f584c0a0290 2021-02-10T11:46:00.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.198 7f58427fc700 1 -- 172.21.2.101:0/3803129637 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f583c0062d0 con 0x7f584c0a0290 2021-02-10T11:46:00.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.198 7f58541b3700 1 -- 172.21.2.101:0/3803129637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f584c086c90 con 0x7f584c0a0290 2021-02-10T11:46:00.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.198 7f58541b3700 1 -- 172.21.2.101:0/3803129637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584c086c90 con 0x7f584c0a0290 2021-02-10T11:46:00.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.198 7f58427fc700 1 -- 172.21.2.101:0/3803129637 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f583c006030 con 0x7f584c0a0290 2021-02-10T11:46:00.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.198 7f58427fc700 1 --2- 172.21.2.101:0/3803129637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f58340186c0 0x7f583401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:00.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.198 7f58427fc700 1 -- 172.21.2.101:0/3803129637 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f583c032560 con 0x7f584c0a0290 2021-02-10T11:46:00.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.198 7f5851fcf700 1 --2- 172.21.2.101:0/3803129637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f58340186c0 0x7f583401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:00.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.199 7f5851fcf700 1 --2- 172.21.2.101:0/3803129637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f58340186c0 0x7f583401abc0 crc :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:00.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.229 7f58541b3700 1 -- 172.21.2.101:0/3803129637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584c082a70 con 0x7f584c0a0290 2021-02-10T11:46:00.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.232 7f58427fc700 1 -- 172.21.2.101:0/3803129637 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f583c000c40 con 0x7f584c0a0290 2021-02-10T11:46:00.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.359 7f58541b3700 1 -- 172.21.2.101:0/3803129637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f584c082a70 con 0x7f584c0a0290 2021-02-10T11:46:00.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.359 7f58427fc700 1 -- 172.21.2.101:0/3803129637 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v164) v1 ==== 104+0+10 (secure 0 0 0) 0x7f583c025d30 con 0x7f584c0a0290 2021-02-10T11:46:00.358 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:46:00.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.361 7f58541b3700 1 -- 172.21.2.101:0/3803129637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f58340186c0 msgr2=0x7f583401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:00.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.361 7f58541b3700 1 --2- 172.21.2.101:0/3803129637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f58340186c0 0x7f583401abc0 crc :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:00.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.361 7f58541b3700 1 -- 172.21.2.101:0/3803129637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f584c0a0290 msgr2=0x7f584c0852d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:00.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.361 7f58541b3700 1 --2- 172.21.2.101:0/3803129637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f584c0a0290 0x7f584c0852d0 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 rx=0x7f583c00b5c0 tx=0x7f583c00b960).stop 2021-02-10T11:46:00.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.361 7f58541b3700 1 -- 172.21.2.101:0/3803129637 shutdown_connections 2021-02-10T11:46:00.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.361 7f58541b3700 1 --2- 172.21.2.101:0/3803129637 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f584c086270 0x7f584c086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:00.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.361 7f58541b3700 1 --2- 172.21.2.101:0/3803129637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f584c0a0290 0x7f584c0852d0 unknown :-1 s=CLOSED pgs=2903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:00.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.361 7f58541b3700 1 --2- 172.21.2.101:0/3803129637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f58340186c0 0x7f583401abc0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:00.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.361 7f58541b3700 1 --2- 172.21.2.101:0/3803129637 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f584c0c3820 0x7f584c084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:00.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.361 7f58541b3700 1 -- 172.21.2.101:0/3803129637 >> 172.21.2.101:0/3803129637 conn(0x7f584c0872c0 msgr2=0x7f584c095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:00.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.361 7f58541b3700 1 -- 172.21.2.101:0/3803129637 shutdown_connections 2021-02-10T11:46:00.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.361 7f58541b3700 1 -- 172.21.2.101:0/3803129637 wait complete. 2021-02-10T11:46:00.367 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:46:00.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.783 7f383b922700 1 Processor -- start 2021-02-10T11:46:00.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.784 7f383b922700 1 -- start start 2021-02-10T11:46:00.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.784 7f383b922700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f383410afe0 0x7f38340cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:00.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.784 7f383b922700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38340fe1b0 con 0x7f38340d1c60 2021-02-10T11:46:00.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.784 7f383b922700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38341b0ca0 con 0x7f38340d1860 2021-02-10T11:46:00.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.784 7f383b922700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3834109a50 con 0x7f383410afe0 2021-02-10T11:46:00.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.784 7f3838f3d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f38340d1c60 0x7f38340cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33398/0 (socket says 172.21.2.101:33398) 2021-02-10T11:46:00.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.784 7f3838f3d700 1 -- 172.21.2.101:0/2788377706 learned_addr learned my addr 172.21.2.101:0/2788377706 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:00.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.785 7f3839f3f700 1 --2- 172.21.2.101:0/2788377706 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f383410afe0 0x7f38340cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:00.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.785 7f3827fff700 1 -- 172.21.2.101:0/2788377706 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799410722 0 0) 0x7f38300010b0 con 0x7f38340d1c60 2021-02-10T11:46:00.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.785 7f3827fff700 1 -- 172.21.2.101:0/2788377706 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3820002fe0 con 0x7f38340d1c60 2021-02-10T11:46:00.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.785 7f3839f3f700 1 -- 172.21.2.101:0/2788377706 >> v1:172.21.2.108:6789/0 conn(0x7f38340d1860 legacy=0x7f38341b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:00.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.785 7f3839f3f700 1 -- 172.21.2.101:0/2788377706 >> v1:172.21.2.101:6789/0 conn(0x7f38340d1c60 legacy=0x7f38340cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:00.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.785 7f3839f3f700 1 -- 172.21.2.101:0/2788377706 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383410b3e0 con 0x7f383410afe0 2021-02-10T11:46:00.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.785 7f3839f3f700 1 --2- 172.21.2.101:0/2788377706 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f383410afe0 0x7f38340cf740 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f382c006e50 tx=0x7f382c009e70).ready entity=mon.2 client_cookie=8afb680052e66a1a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:00.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.785 7f3827fff700 1 -- 172.21.2.101:0/2788377706 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f382c006a20 con 0x7f383410afe0 2021-02-10T11:46:00.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.785 7f3827fff700 1 -- 172.21.2.101:0/2788377706 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382c006790 con 0x7f383410afe0 2021-02-10T11:46:00.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.785 7f3827fff700 1 -- 172.21.2.101:0/2788377706 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f382c000f90 con 0x7f383410afe0 2021-02-10T11:46:00.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.785 7f383b922700 1 -- 172.21.2.101:0/2788377706 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f383410afe0 msgr2=0x7f38340cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:00.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.785 7f383b922700 1 --2- 172.21.2.101:0/2788377706 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f383410afe0 0x7f38340cf740 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f382c006e50 tx=0x7f382c009e70).stop 2021-02-10T11:46:00.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.786 7f383b922700 1 -- 172.21.2.101:0/2788377706 shutdown_connections 2021-02-10T11:46:00.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.786 7f383b922700 1 --2- 172.21.2.101:0/2788377706 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f383410afe0 0x7f38340cf740 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:00.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.786 7f383b922700 1 -- 172.21.2.101:0/2788377706 >> 172.21.2.101:0/2788377706 conn(0x7f38340ce410 msgr2=0x7f38340d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:00.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.786 7f383b922700 1 -- 172.21.2.101:0/2788377706 shutdown_connections 2021-02-10T11:46:00.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.786 7f383b922700 1 -- 172.21.2.101:0/2788377706 wait complete. 2021-02-10T11:46:00.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.801 7f383b922700 1 Processor -- start 2021-02-10T11:46:00.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.801 7f383b922700 1 -- start start 2021-02-10T11:46:00.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.801 7f383b922700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38340bfa10 0x7f38340b82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:00.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.801 7f383b922700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38341a4a70 0x7f3834094c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:00.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.801 7f383b922700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38340c0410 0x7f38340c0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:00.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.801 7f383b922700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38340c3a50 con 0x7f38341a4a70 2021-02-10T11:46:00.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.801 7f383b922700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38340b88c0 con 0x7f38340bfa10 2021-02-10T11:46:00.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.801 7f383b922700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3834095260 con 0x7f38340c0410 2021-02-10T11:46:00.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.801 7f3838f3d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38341a4a70 0x7f3834094c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:00.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.801 7f3838f3d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38341a4a70 0x7f3834094c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32912/0 (socket says 172.21.2.101:32912) 2021-02-10T11:46:00.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.801 7f3838f3d700 1 -- 172.21.2.101:0/2931536650 learned_addr learned my addr 172.21.2.101:0/2931536650 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:00.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.802 7f3838f3d700 1 -- 172.21.2.101:0/2931536650 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38340c0410 msgr2=0x7f38340c0810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:00.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.802 7f3838f3d700 1 --2- 172.21.2.101:0/2931536650 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38340c0410 0x7f38340c0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:00.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.802 7f3838f3d700 1 -- 172.21.2.101:0/2931536650 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38340bfa10 msgr2=0x7f38340b82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:00.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.802 7f3838f3d700 1 --2- 172.21.2.101:0/2931536650 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38340bfa10 0x7f38340b82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:00.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.802 7f3838f3d700 1 -- 172.21.2.101:0/2931536650 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38340c3540 con 0x7f38341a4a70 2021-02-10T11:46:00.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.802 7f3838f3d700 1 --2- 172.21.2.101:0/2931536650 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38341a4a70 0x7f3834094c40 secure :-1 s=READY pgs=2905 cs=0 l=1 rev1=1 rx=0x7f3830001c60 tx=0x7f383000a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:00.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.802 7f3825ffb700 1 -- 172.21.2.101:0/2931536650 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3830000e00 con 0x7f38341a4a70 2021-02-10T11:46:00.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.802 7f3825ffb700 1 -- 172.21.2.101:0/2931536650 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383000acc0 con 0x7f38341a4a70 2021-02-10T11:46:00.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.802 7f3825ffb700 1 -- 172.21.2.101:0/2931536650 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f38300045d0 con 0x7f38341a4a70 2021-02-10T11:46:00.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.802 7f383b922700 1 -- 172.21.2.101:0/2931536650 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38340c3540 con 0x7f38341a4a70 2021-02-10T11:46:00.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.802 7f383b922700 1 -- 172.21.2.101:0/2931536650 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38340c3540 con 0x7f38341a4a70 2021-02-10T11:46:00.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.803 7f3825ffb700 1 -- 172.21.2.101:0/2931536650 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3830004140 con 0x7f38341a4a70 2021-02-10T11:46:00.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.803 7f3825ffb700 1 --2- 172.21.2.101:0/2931536650 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38140186d0 0x7f381401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:00.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.803 7f3825ffb700 1 -- 172.21.2.101:0/2931536650 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3830032610 con 0x7f38341a4a70 2021-02-10T11:46:00.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.803 7f383973e700 1 --2- 172.21.2.101:0/2931536650 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38140186d0 0x7f381401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:00.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.803 7f383973e700 1 --2- 172.21.2.101:0/2931536650 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38140186d0 0x7f381401abd0 crc :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:00.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.834 7f383b922700 1 -- 172.21.2.101:0/2931536650 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38340bfee0 con 0x7f38341a4a70 2021-02-10T11:46:00.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.837 7f3825ffb700 1 -- 172.21.2.101:0/2931536650 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3830000e00 con 0x7f38341a4a70 2021-02-10T11:46:00.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.963 7f383b922700 1 -- 172.21.2.101:0/2931536650 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f38340bfee0 con 0x7f38341a4a70 2021-02-10T11:46:00.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.964 7f3825ffb700 1 -- 172.21.2.101:0/2931536650 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v164) v1 ==== 100+0+10 (secure 0 0 0) 0x7f3830004140 con 0x7f38341a4a70 2021-02-10T11:46:00.962 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:46:00.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.966 7f383b922700 1 -- 172.21.2.101:0/2931536650 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38140186d0 msgr2=0x7f381401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:00.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.966 7f383b922700 1 --2- 172.21.2.101:0/2931536650 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38140186d0 0x7f381401abd0 crc :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:00.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.966 7f383b922700 1 -- 172.21.2.101:0/2931536650 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38341a4a70 msgr2=0x7f3834094c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:00.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.966 7f383b922700 1 --2- 172.21.2.101:0/2931536650 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38341a4a70 0x7f3834094c40 secure :-1 s=READY pgs=2905 cs=0 l=1 rev1=1 rx=0x7f3830001c60 tx=0x7f383000a8d0).stop 2021-02-10T11:46:00.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.966 7f383b922700 1 -- 172.21.2.101:0/2931536650 shutdown_connections 2021-02-10T11:46:00.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.966 7f383b922700 1 --2- 172.21.2.101:0/2931536650 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38140186d0 0x7f381401abd0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:00.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.966 7f383b922700 1 --2- 172.21.2.101:0/2931536650 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38340c0410 0x7f38340c0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:00.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.966 7f383b922700 1 --2- 172.21.2.101:0/2931536650 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38341a4a70 0x7f3834094c40 unknown :-1 s=CLOSED pgs=2905 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:00.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.966 7f383b922700 1 --2- 172.21.2.101:0/2931536650 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38340bfa10 0x7f38340b82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:00.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.966 7f383b922700 1 -- 172.21.2.101:0/2931536650 >> 172.21.2.101:0/2931536650 conn(0x7f38340a6a80 msgr2=0x7f38340ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:00.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.966 7f383b922700 1 -- 172.21.2.101:0/2931536650 shutdown_connections 2021-02-10T11:46:00.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:00.966 7f383b922700 1 -- 172.21.2.101:0/2931536650 wait complete. 2021-02-10T11:46:00.972 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:46:00.972 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:46:01.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.387 7f60cc0ad700 1 Processor -- start 2021-02-10T11:46:01.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.387 7f60cc0ad700 1 -- start start 2021-02-10T11:46:01.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.387 7f60cc0ad700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f60c40d1870 0x7f60c41b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:01.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.387 7f60cc0ad700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f60c40f0650 0x7f60c40cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:01.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.387 7f60cc0ad700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c40f0b10 con 0x7f60c41b0a10 2021-02-10T11:46:01.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.387 7f60cc0ad700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c40e5ed0 con 0x7f60c40d1870 2021-02-10T11:46:01.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.387 7f60cc0ad700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c40d1c70 con 0x7f60c40f0650 2021-02-10T11:46:01.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.388 7f60c9ec9700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f60c40d1870 0x7f60c41b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:01.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.388 7f60c9ec9700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f60c40d1870 0x7f60c41b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32918/0 (socket says 172.21.2.101:32918) 2021-02-10T11:46:01.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.388 7f60c9ec9700 1 -- 172.21.2.101:0/2279643601 learned_addr learned my addr 172.21.2.101:0/2279643601 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:01.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.388 7f60ca6ca700 1 --2- 172.21.2.101:0/2279643601 >> v2:172.21.2.108:3300/0 conn(0x7f60c40f0650 0x7f60c40cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:01.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.388 7f60c9ec9700 1 -- 172.21.2.101:0/2279643601 >> v2:172.21.2.108:3300/0 conn(0x7f60c40f0650 msgr2=0x7f60c40cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:01.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.388 7f60c9ec9700 1 --2- 172.21.2.101:0/2279643601 >> v2:172.21.2.108:3300/0 conn(0x7f60c40f0650 0x7f60c40cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:01.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.388 7f60c9ec9700 1 -- 172.21.2.101:0/2279643601 >> v1:172.21.2.108:6789/0 conn(0x7f60c41b0a10 legacy=0x7f60c40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:01.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.388 7f60c9ec9700 1 -- 172.21.2.101:0/2279643601 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c40d1e90 con 0x7f60c40d1870 2021-02-10T11:46:01.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.388 7f60c9ec9700 1 --2- 172.21.2.101:0/2279643601 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60c40d1870 0x7f60c41b03f0 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 rx=0x7f60c0006700 tx=0x7f60c0009b80).ready entity=mon.0 client_cookie=44be1d12a7d62bf5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:01.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.389 7f60c8ec7700 1 -- 172.21.2.101:0/2279643601 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f60c000c0c0 con 0x7f60c40d1870 2021-02-10T11:46:01.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.389 7f60c8ec7700 1 -- 172.21.2.101:0/2279643601 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c000c440 con 0x7f60c40d1870 2021-02-10T11:46:01.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.389 7f60c8ec7700 1 -- 172.21.2.101:0/2279643601 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f60c000c960 con 0x7f60c40d1870 2021-02-10T11:46:01.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.389 7f60cc0ad700 1 -- 172.21.2.101:0/2279643601 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60c40d1870 msgr2=0x7f60c41b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:01.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.389 7f60cc0ad700 1 --2- 172.21.2.101:0/2279643601 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60c40d1870 0x7f60c41b03f0 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 rx=0x7f60c0006700 tx=0x7f60c0009b80).stop 2021-02-10T11:46:01.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.389 7f60cc0ad700 1 -- 172.21.2.101:0/2279643601 shutdown_connections 2021-02-10T11:46:01.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.389 7f60cc0ad700 1 --2- 172.21.2.101:0/2279643601 >> v2:172.21.2.108:3300/0 conn(0x7f60c40f0650 0x7f60c40cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:01.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.389 7f60cc0ad700 1 --2- 172.21.2.101:0/2279643601 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60c40d1870 0x7f60c41b03f0 unknown :-1 s=CLOSED pgs=2906 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:01.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.389 7f60cc0ad700 1 -- 172.21.2.101:0/2279643601 >> 172.21.2.101:0/2279643601 conn(0x7f60c40ce450 msgr2=0x7f60c40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:01.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.390 7f60cc0ad700 1 -- 172.21.2.101:0/2279643601 shutdown_connections 2021-02-10T11:46:01.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.390 7f60cc0ad700 1 -- 172.21.2.101:0/2279643601 wait complete. 2021-02-10T11:46:01.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.404 7f60cc0ad700 1 Processor -- start 2021-02-10T11:46:01.405 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.404 7f60cc0ad700 1 -- start start 2021-02-10T11:46:01.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.404 7f60cc0ad700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f60c41942b0 0x7f60c40b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:01.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.404 7f60cc0ad700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60c4195070 0x7f60c4195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:01.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.404 7f60cc0ad700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f60c40bb8e0 0x7f60c40bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:01.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.404 7f60cc0ad700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c41946b0 con 0x7f60c4195070 2021-02-10T11:46:01.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.404 7f60cc0ad700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c40b3b70 con 0x7f60c40bb8e0 2021-02-10T11:46:01.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.404 7f60cc0ad700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c40b3d90 con 0x7f60c41942b0 2021-02-10T11:46:01.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.404 7f60c96c8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60c4195070 0x7f60c4195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:01.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.404 7f60c96c8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60c4195070 0x7f60c4195470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32924/0 (socket says 172.21.2.101:32924) 2021-02-10T11:46:01.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.404 7f60c96c8700 1 -- 172.21.2.101:0/3430800125 learned_addr learned my addr 172.21.2.101:0/3430800125 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:01.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.405 7f60c96c8700 1 -- 172.21.2.101:0/3430800125 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f60c41942b0 msgr2=0x7f60c40b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:01.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.405 7f60c96c8700 1 --2- 172.21.2.101:0/3430800125 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f60c41942b0 0x7f60c40b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:01.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.405 7f60c96c8700 1 -- 172.21.2.101:0/3430800125 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f60c40bb8e0 msgr2=0x7f60c40bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:01.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.405 7f60c96c8700 1 --2- 172.21.2.101:0/3430800125 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f60c40bb8e0 0x7f60c40bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:01.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.405 7f60c96c8700 1 -- 172.21.2.101:0/3430800125 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c41a49e0 con 0x7f60c4195070 2021-02-10T11:46:01.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.405 7f60c96c8700 1 --2- 172.21.2.101:0/3430800125 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60c4195070 0x7f60c4195470 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 rx=0x7f60b400b5b0 tx=0x7f60b400b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:01.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.405 7f60ba7fc700 1 -- 172.21.2.101:0/3430800125 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f60b400bd40 con 0x7f60c4195070 2021-02-10T11:46:01.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.405 7f60ba7fc700 1 -- 172.21.2.101:0/3430800125 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b4000c60 con 0x7f60c4195070 2021-02-10T11:46:01.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.405 7f60ba7fc700 1 -- 172.21.2.101:0/3430800125 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f60b4006210 con 0x7f60c4195070 2021-02-10T11:46:01.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.405 7f60cc0ad700 1 -- 172.21.2.101:0/3430800125 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c41a49e0 con 0x7f60c4195070 2021-02-10T11:46:01.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.405 7f60cc0ad700 1 -- 172.21.2.101:0/3430800125 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c41a49e0 con 0x7f60c4195070 2021-02-10T11:46:01.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.406 7f60ba7fc700 1 -- 172.21.2.101:0/3430800125 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f60b4006020 con 0x7f60c4195070 2021-02-10T11:46:01.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.406 7f60ba7fc700 1 --2- 172.21.2.101:0/3430800125 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f60ac0186c0 0x7f60ac01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:01.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.406 7f60ba7fc700 1 -- 172.21.2.101:0/3430800125 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f60b40325a0 con 0x7f60c4195070 2021-02-10T11:46:01.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.406 7f60c9ec9700 1 --2- 172.21.2.101:0/3430800125 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f60ac0186c0 0x7f60ac01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:01.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.406 7f60c9ec9700 1 --2- 172.21.2.101:0/3430800125 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f60ac0186c0 0x7f60ac01abc0 crc :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:01.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.438 7f60cc0ad700 1 -- 172.21.2.101:0/3430800125 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c41a49e0 con 0x7f60c4195070 2021-02-10T11:46:01.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.440 7f60ba7fc700 1 -- 172.21.2.101:0/3430800125 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f60b4030d50 con 0x7f60c4195070 2021-02-10T11:46:01.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.567 7f60cc0ad700 1 -- 172.21.2.101:0/3430800125 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60c41a49e0 con 0x7f60c4195070 2021-02-10T11:46:01.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.567 7f60ba7fc700 1 -- 172.21.2.101:0/3430800125 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+2104 (secure 0 0 0) 0x7f60b400bd40 con 0x7f60c4195070 2021-02-10T11:46:01.566 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:46:01.567 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":139,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8371,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/453344112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":453344112},{"type":"v1","addr":"172.21.2.101:6835","nonce":453344112}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":135}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:45:44.128996","modified":"2021-02-10 11:45:46.139945","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7097":{"gid":7097,"name":"b","rank":0,"incarnation":136,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2456711519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2456711519},{"type":"v1","addr":"172.21.2.108:6833","nonce":2456711519}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":17}]} 2021-02-10T11:46:01.567 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 139 2021-02-10T11:46:01.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.570 7f60cc0ad700 1 -- 172.21.2.101:0/3430800125 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f60ac0186c0 msgr2=0x7f60ac01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:01.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.570 7f60cc0ad700 1 --2- 172.21.2.101:0/3430800125 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f60ac0186c0 0x7f60ac01abc0 crc :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:01.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.570 7f60cc0ad700 1 -- 172.21.2.101:0/3430800125 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60c4195070 msgr2=0x7f60c4195470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:01.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.570 7f60cc0ad700 1 --2- 172.21.2.101:0/3430800125 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60c4195070 0x7f60c4195470 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 rx=0x7f60b400b5b0 tx=0x7f60b400b950).stop 2021-02-10T11:46:01.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.570 7f60cc0ad700 1 -- 172.21.2.101:0/3430800125 shutdown_connections 2021-02-10T11:46:01.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.570 7f60cc0ad700 1 --2- 172.21.2.101:0/3430800125 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f60c40bb8e0 0x7f60c40bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:01.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.570 7f60cc0ad700 1 --2- 172.21.2.101:0/3430800125 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60c4195070 0x7f60c4195470 unknown :-1 s=CLOSED pgs=2907 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:01.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.570 7f60cc0ad700 1 --2- 172.21.2.101:0/3430800125 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f60ac0186c0 0x7f60ac01abc0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:01.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.570 7f60cc0ad700 1 --2- 172.21.2.101:0/3430800125 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f60c41942b0 0x7f60c40b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:01.578 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.570 7f60cc0ad700 1 -- 172.21.2.101:0/3430800125 >> 172.21.2.101:0/3430800125 conn(0x7f60c40b3040 msgr2=0x7f60c40c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:01.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.570 7f60cc0ad700 1 -- 172.21.2.101:0/3430800125 shutdown_connections 2021-02-10T11:46:01.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:01.570 7f60cc0ad700 1 -- 172.21.2.101:0/3430800125 wait complete. 2021-02-10T11:46:01.783 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 139, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:45:44.128996', 'modified': '2021-02-10 11:45:46.139945', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7097}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7097': {'gid': 7097, 'name': 'b', 'rank': 0, 'incarnation': 136, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/2456711519', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 2456711519}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 2456711519}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [35], 'metadata_pool': 34, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:46:01.783 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:46:01.783 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:46:02.183 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:46:02.183 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:46:02.183 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:46:02.183 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 7097, 2021-02-10T11:46:02.183 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:46:02.183 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:46:02.183 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 18.054058318999999, 2021-02-10T11:46:02.183 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 139, 2021-02-10T11:46:02.183 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 164, 2021-02-10T11:46:02.183 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 164, 2021-02-10T11:46:02.184 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 23.703435902999999 2021-02-10T11:46:02.184 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:46:02.187 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 7097, "want_state": "up:active", "state": "up:active", "rank_uptime": 18.054058318999999, "mdsmap_epoch": 139, "osdmap_epoch": 164, "osdmap_epoch_barrier": 164, "uptime": 23.703435902999999 } 2021-02-10T11:46:02.187 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:46:02.187 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-02-10T11:46:02.187 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:46:02.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:02.187 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:46:02.525 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:46:02.872 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:46:02.872 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:46:03.216 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:46:03.216 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:46:03.217 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:46:03.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:03.218 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:46:03.553 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:46:03.553 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2021-02-10T11:46:03.554 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:46:03.556 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:46:03.900 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:46:03.900 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:46:03.966 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[170281]: starting ceph client 2021-02-10T11:46:03.966 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:2021-02-10 11:46:03.968 7fa53d34df80 -1 init, newargv = 0x55a4afa9a1a0 newargc=9 2021-02-10T11:46:03.972 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[170281]: starting fuse 2021-02-10T11:46:04.244 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:46:04.244 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:46:04.245 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:46:04.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:04.245 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:46:04.581 INFO:teuthology.orchestra.run.gibba001.stdout:42 2021-02-10T11:46:04.581 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:46:04.581 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2021-02-10T11:46:04.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:04.582 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:46:04.582 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:46:04.582 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:46:04.582 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:46:04.582 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:46:04.582 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:46:04.582 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:46:04.582 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-02-10T11:46:04.582 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:46:04.582 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:46:04.582 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:46:04.583 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:46:04.583 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:46:04.583 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:46:04.583 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:46:04.583 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:46:04.583 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:46:04.583 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:46:04.583 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:46:04.583 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:46:04.583 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:46:04.584 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.1")) 2021-02-10T11:46:04.584 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:46:04.945 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.1.170281.asok 2021-02-10T11:46:04.948 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.170281.asok 2021-02-10T11:46:04.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:04.948 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.170281.asok status 2021-02-10T11:46:05.339 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:46:05.339 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:46:05.339 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:46:05.340 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:46:05.340 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "1", 2021-02-10T11:46:05.340 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:46:05.340 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:46:05.340 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "170281", 2021-02-10T11:46:05.340 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:46:05.340 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:05.340 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:46:05.340 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:46:05.340 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 12630, 2021-02-10T11:46:05.340 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:46:05.340 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:46:05.340 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:46:05.341 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 12630 2021-02-10T11:46:05.341 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:05.341 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:46:05.341 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:05.341 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:46:05.341 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 4099371036 2021-02-10T11:46:05.341 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:05.341 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:05.341 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:46:05.341 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:05.341 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:46:05.341 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 4099371036 2021-02-10T11:46:05.341 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:05.342 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.12630 172.21.2.101:0/4099371036", 2021-02-10T11:46:05.342 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/4099371036", 2021-02-10T11:46:05.342 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:46:05.342 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 139, 2021-02-10T11:46:05.342 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 164, 2021-02-10T11:46:05.342 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:46:05.342 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:46:05.342 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:46:05.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:05.344 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:46:05.344 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:46:05.680 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:46:05.680 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:46:05.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:05.680 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:46:06.027 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2021-02-10T11:46:06.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.447 7f21eb972700 1 Processor -- start 2021-02-10T11:46:06.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.448 7f21eb972700 1 -- start start 2021-02-10T11:46:06.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.448 7f21eb972700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f21e40d1880 0x7f21e41b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:06.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.448 7f21eb972700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e40fe1d0 con 0x7f21e41b0af0 2021-02-10T11:46:06.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.448 7f21eb972700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e4109a70 con 0x7f21e40cff80 2021-02-10T11:46:06.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.448 7f21eb972700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e4088070 con 0x7f21e40d1880 2021-02-10T11:46:06.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.448 7f21e978e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f21e40d1880 0x7f21e41b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:06.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.448 7f21e8f8d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f21e41b0af0 0x7f21e4108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33436/0 (socket says 172.21.2.101:33436) 2021-02-10T11:46:06.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.448 7f21e8f8d700 1 -- 172.21.2.101:0/389293128 learned_addr learned my addr 172.21.2.101:0/389293128 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:06.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.448 7f21e978e700 1 -- 172.21.2.101:0/389293128 >> v1:172.21.2.108:6789/0 conn(0x7f21e40cff80 legacy=0x7f21e4109140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:46:06.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.448 7f21e978e700 1 -- 172.21.2.101:0/389293128 >> v1:172.21.2.101:6789/0 conn(0x7f21e41b0af0 legacy=0x7f21e4108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:06.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.449 7f21e978e700 1 -- 172.21.2.101:0/389293128 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e4088260 con 0x7f21e40d1880 2021-02-10T11:46:06.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.449 7f21e978e700 1 --2- 172.21.2.101:0/389293128 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21e40d1880 0x7f21e41b04d0 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 rx=0x7f21e0006650 tx=0x7f21e0009a80).ready entity=mon.0 client_cookie=f32737fddb00f4f9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:06.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.449 7f21d7fff700 1 -- 172.21.2.101:0/389293128 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21e00009b0 con 0x7f21e40d1880 2021-02-10T11:46:06.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.449 7f21d7fff700 1 -- 172.21.2.101:0/389293128 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e000c0c0 con 0x7f21e40d1880 2021-02-10T11:46:06.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.449 7f21d7fff700 1 -- 172.21.2.101:0/389293128 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21e000c520 con 0x7f21e40d1880 2021-02-10T11:46:06.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.449 7f21eb972700 1 -- 172.21.2.101:0/389293128 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21e40d1880 msgr2=0x7f21e41b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:06.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.449 7f21eb972700 1 --2- 172.21.2.101:0/389293128 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21e40d1880 0x7f21e41b04d0 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 rx=0x7f21e0006650 tx=0x7f21e0009a80).stop 2021-02-10T11:46:06.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.449 7f21eb972700 1 -- 172.21.2.101:0/389293128 shutdown_connections 2021-02-10T11:46:06.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.449 7f21eb972700 1 --2- 172.21.2.101:0/389293128 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21e40d1880 0x7f21e41b04d0 unknown :-1 s=CLOSED pgs=2911 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:06.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.449 7f21eb972700 1 -- 172.21.2.101:0/389293128 >> 172.21.2.101:0/389293128 conn(0x7f21e40ce430 msgr2=0x7f21e40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:06.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.450 7f21eb972700 1 -- 172.21.2.101:0/389293128 shutdown_connections 2021-02-10T11:46:06.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.450 7f21eb972700 1 -- 172.21.2.101:0/389293128 wait complete. 2021-02-10T11:46:06.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.464 7f21eb972700 1 Processor -- start 2021-02-10T11:46:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.464 7f21eb972700 1 -- start start 2021-02-10T11:46:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.464 7f21eb972700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21e41b5bc0 0x7f21e40aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.464 7f21eb972700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21e40c0d90 0x7f21e40c1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.464 7f21eb972700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21e40b1310 0x7f21e40b1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.464 7f21eb972700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e40c17b0 con 0x7f21e40b1310 2021-02-10T11:46:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.464 7f21eb972700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e40af0e0 con 0x7f21e41b5bc0 2021-02-10T11:46:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.464 7f21eb972700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e41b5fc0 con 0x7f21e40c0d90 2021-02-10T11:46:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.464 7f21e9f8f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21e40b1310 0x7f21e40b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.464 7f21e9f8f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21e40b1310 0x7f21e40b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32952/0 (socket says 172.21.2.101:32952) 2021-02-10T11:46:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.465 7f21e9f8f700 1 -- 172.21.2.101:0/1931751033 learned_addr learned my addr 172.21.2.101:0/1931751033 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.465 7f21e9f8f700 1 -- 172.21.2.101:0/1931751033 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21e40c0d90 msgr2=0x7f21e40c1190 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:06.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.465 7f21e9f8f700 1 --2- 172.21.2.101:0/1931751033 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21e40c0d90 0x7f21e40c1190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:06.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.465 7f21e9f8f700 1 -- 172.21.2.101:0/1931751033 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21e41b5bc0 msgr2=0x7f21e40aeac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:06.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.465 7f21e9f8f700 1 --2- 172.21.2.101:0/1931751033 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21e41b5bc0 0x7f21e40aeac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:06.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.465 7f21e9f8f700 1 -- 172.21.2.101:0/1931751033 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e40b1d30 con 0x7f21e40b1310 2021-02-10T11:46:06.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.465 7f21e9f8f700 1 --2- 172.21.2.101:0/1931751033 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21e40b1310 0x7f21e40b1710 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 rx=0x7f21dc009940 tx=0x7f21dc00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:06.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.465 7f21d5ffb700 1 -- 172.21.2.101:0/1931751033 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21dc003030 con 0x7f21e40b1310 2021-02-10T11:46:06.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.465 7f21d5ffb700 1 -- 172.21.2.101:0/1931751033 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21dc001270 con 0x7f21e40b1310 2021-02-10T11:46:06.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.465 7f21d5ffb700 1 -- 172.21.2.101:0/1931751033 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21dc001790 con 0x7f21e40b1310 2021-02-10T11:46:06.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.465 7f21eb972700 1 -- 172.21.2.101:0/1931751033 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e40b1d30 con 0x7f21e40b1310 2021-02-10T11:46:06.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.465 7f21eb972700 1 -- 172.21.2.101:0/1931751033 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e40b03e0 con 0x7f21e40b1310 2021-02-10T11:46:06.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.466 7f21d5ffb700 1 -- 172.21.2.101:0/1931751033 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f21dc001450 con 0x7f21e40b1310 2021-02-10T11:46:06.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.466 7f21d5ffb700 1 --2- 172.21.2.101:0/1931751033 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21c40186c0 0x7f21c401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:06.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.466 7f21d5ffb700 1 -- 172.21.2.101:0/1931751033 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f21dc032900 con 0x7f21e40b1310 2021-02-10T11:46:06.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.466 7f21e978e700 1 --2- 172.21.2.101:0/1931751033 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21c40186c0 0x7f21c401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:06.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.466 7f21e978e700 1 --2- 172.21.2.101:0/1931751033 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21c40186c0 0x7f21c401abc0 crc :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:06.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.498 7f21eb972700 1 -- 172.21.2.101:0/1931751033 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e40c17b0 con 0x7f21e40b1310 2021-02-10T11:46:06.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.500 7f21d5ffb700 1 -- 172.21.2.101:0/1931751033 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f21dc00be30 con 0x7f21e40b1310 2021-02-10T11:46:06.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:06.637 7f21eb972700 1 -- 172.21.2.101:0/1931751033 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f21e40af0e0 con 0x7f21e40b1310 2021-02-10T11:46:07.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:07.631 7f21d5ffb700 1 -- 172.21.2.101:0/1931751033 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v140) v1 ==== 136+0+0 (secure 0 0 0) 0x7f21dc001450 con 0x7f21e40b1310 2021-02-10T11:46:07.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:07.659 7f21eb972700 1 -- 172.21.2.101:0/1931751033 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21c40186c0 msgr2=0x7f21c401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:07.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:07.659 7f21eb972700 1 --2- 172.21.2.101:0/1931751033 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21c40186c0 0x7f21c401abc0 crc :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:07.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:07.659 7f21eb972700 1 -- 172.21.2.101:0/1931751033 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21e40b1310 msgr2=0x7f21e40b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:07.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:07.659 7f21eb972700 1 --2- 172.21.2.101:0/1931751033 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21e40b1310 0x7f21e40b1710 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 rx=0x7f21dc009940 tx=0x7f21dc00bcf0).stop 2021-02-10T11:46:07.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:07.660 7f21eb972700 1 -- 172.21.2.101:0/1931751033 shutdown_connections 2021-02-10T11:46:07.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:07.660 7f21eb972700 1 --2- 172.21.2.101:0/1931751033 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21c40186c0 0x7f21c401abc0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:07.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:07.660 7f21eb972700 1 --2- 172.21.2.101:0/1931751033 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21e40c0d90 0x7f21e40c1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:07.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:07.660 7f21eb972700 1 --2- 172.21.2.101:0/1931751033 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21e40b1310 0x7f21e40b1710 unknown :-1 s=CLOSED pgs=2912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:07.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:07.660 7f21eb972700 1 --2- 172.21.2.101:0/1931751033 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21e41b5bc0 0x7f21e40aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:07.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:07.660 7f21eb972700 1 -- 172.21.2.101:0/1931751033 >> 172.21.2.101:0/1931751033 conn(0x7f21e40bd400 msgr2=0x7f21e40804e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:07.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:07.660 7f21eb972700 1 -- 172.21.2.101:0/1931751033 shutdown_connections 2021-02-10T11:46:07.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:07.660 7f21eb972700 1 -- 172.21.2.101:0/1931751033 wait complete. 2021-02-10T11:46:07.666 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs volume ls 2021-02-10T11:46:08.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.080 7f26adaee700 1 Processor -- start 2021-02-10T11:46:08.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.080 7f26adaee700 1 -- start start 2021-02-10T11:46:08.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.081 7f26adaee700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26a80d1870 0x7f26a81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:08.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.081 7f26adaee700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f26a81b0a10 0x7f26a80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:08.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.081 7f26adaee700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a80f0b10 con 0x7f26a80d1c70 2021-02-10T11:46:08.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.081 7f26adaee700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a80fe230 con 0x7f26a81b0a10 2021-02-10T11:46:08.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.081 7f26adaee700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a80e5ed0 con 0x7f26a80d1870 2021-02-10T11:46:08.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.081 7f26a6ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f26a81b0a10 0x7f26a80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:08.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.081 7f26a6ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f26a81b0a10 0x7f26a80cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32956/0 (socket says 172.21.2.101:32956) 2021-02-10T11:46:08.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.081 7f26a6ffd700 1 -- 172.21.2.101:0/2280190647 learned_addr learned my addr 172.21.2.101:0/2280190647 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:08.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.081 7f26a6ffd700 1 -- 172.21.2.101:0/2280190647 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26a80d1870 msgr2=0x7f26a81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:08.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.081 7f26a77fe700 1 --2- 172.21.2.101:0/2280190647 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26a80d1870 0x7f26a81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:08.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.081 7f26a6ffd700 1 --2- 172.21.2.101:0/2280190647 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26a80d1870 0x7f26a81b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:08.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.081 7f26a6ffd700 1 -- 172.21.2.101:0/2280190647 >> v1:172.21.2.108:6789/0 conn(0x7f26a80d1c70 legacy=0x7f26a80cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:08.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.081 7f26a6ffd700 1 -- 172.21.2.101:0/2280190647 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a80f0650 con 0x7f26a81b0a10 2021-02-10T11:46:08.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.081 7f26a77fe700 1 --2- 172.21.2.101:0/2280190647 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26a80d1870 0x7f26a81b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:46:08.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.082 7f26a6ffd700 1 --2- 172.21.2.101:0/2280190647 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26a81b0a10 0x7f26a80cf330 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 rx=0x7f269c006700 tx=0x7f269c009b80).ready entity=mon.0 client_cookie=248a3cbacd09a630 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:08.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.082 7f26a67fc700 1 -- 172.21.2.101:0/2280190647 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f269c00c0c0 con 0x7f26a81b0a10 2021-02-10T11:46:08.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.082 7f26a67fc700 1 -- 172.21.2.101:0/2280190647 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269c00c340 con 0x7f26a81b0a10 2021-02-10T11:46:08.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.082 7f26a67fc700 1 -- 172.21.2.101:0/2280190647 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f269c00c860 con 0x7f26a81b0a10 2021-02-10T11:46:08.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.082 7f26adaee700 1 -- 172.21.2.101:0/2280190647 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26a81b0a10 msgr2=0x7f26a80cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:08.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.082 7f26adaee700 1 --2- 172.21.2.101:0/2280190647 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26a81b0a10 0x7f26a80cf330 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 rx=0x7f269c006700 tx=0x7f269c009b80).stop 2021-02-10T11:46:08.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.082 7f26adaee700 1 -- 172.21.2.101:0/2280190647 shutdown_connections 2021-02-10T11:46:08.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.083 7f26adaee700 1 --2- 172.21.2.101:0/2280190647 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26a81b0a10 0x7f26a80cf330 unknown :-1 s=CLOSED pgs=2913 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:08.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.083 7f26adaee700 1 --2- 172.21.2.101:0/2280190647 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26a80d1870 0x7f26a81b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:08.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.083 7f26adaee700 1 -- 172.21.2.101:0/2280190647 >> 172.21.2.101:0/2280190647 conn(0x7f26a80ce450 msgr2=0x7f26a80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:08.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.083 7f26adaee700 1 -- 172.21.2.101:0/2280190647 shutdown_connections 2021-02-10T11:46:08.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.083 7f26adaee700 1 -- 172.21.2.101:0/2280190647 wait complete. 2021-02-10T11:46:08.099 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:46:08.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.097 7f26adaee700 1 Processor -- start 2021-02-10T11:46:08.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.097 7f26adaee700 1 -- start start 2021-02-10T11:46:08.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.098 7f26adaee700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26a818ae80 0x7f26a81a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:08.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.098 7f26adaee700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f26a81a18f0 0x7f26a818a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:08.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.098 7f26adaee700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26a818a670 0x7f26a8187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:08.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.098 7f26adaee700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a818b300 con 0x7f26a818a670 2021-02-10T11:46:08.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.098 7f26adaee700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a80eea20 con 0x7f26a81a18f0 2021-02-10T11:46:08.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.098 7f26adaee700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a818aa70 con 0x7f26a818ae80 2021-02-10T11:46:08.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.098 7f26a7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26a818a670 0x7f26a8187e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:08.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.098 7f26a7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26a818a670 0x7f26a8187e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32964/0 (socket says 172.21.2.101:32964) 2021-02-10T11:46:08.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.098 7f26a7fff700 1 -- 172.21.2.101:0/2632214329 learned_addr learned my addr 172.21.2.101:0/2632214329 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:08.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.098 7f26a77fe700 1 --2- 172.21.2.101:0/2632214329 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26a818ae80 0x7f26a81a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:08.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.098 7f26a6ffd700 1 --2- 172.21.2.101:0/2632214329 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f26a81a18f0 0x7f26a818a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:08.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.099 7f26a7fff700 1 -- 172.21.2.101:0/2632214329 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26a818ae80 msgr2=0x7f26a81a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:08.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.099 7f26a7fff700 1 --2- 172.21.2.101:0/2632214329 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26a818ae80 0x7f26a81a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:08.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.099 7f26a7fff700 1 -- 172.21.2.101:0/2632214329 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f26a81a18f0 msgr2=0x7f26a818a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:08.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.099 7f26a7fff700 1 --2- 172.21.2.101:0/2632214329 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f26a81a18f0 0x7f26a818a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:08.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.099 7f26a7fff700 1 -- 172.21.2.101:0/2632214329 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a8188470 con 0x7f26a818a670 2021-02-10T11:46:08.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.099 7f26a7fff700 1 --2- 172.21.2.101:0/2632214329 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26a818a670 0x7f26a8187e50 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 rx=0x7f26a0009940 tx=0x7f26a000bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:08.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.099 7f268ffff700 1 -- 172.21.2.101:0/2632214329 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f26a0000f90 con 0x7f26a818a670 2021-02-10T11:46:08.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.099 7f268ffff700 1 -- 172.21.2.101:0/2632214329 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a000c2f0 con 0x7f26a818a670 2021-02-10T11:46:08.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.099 7f26adaee700 1 -- 172.21.2.101:0/2632214329 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a8188470 con 0x7f26a818a670 2021-02-10T11:46:08.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.099 7f26adaee700 1 -- 172.21.2.101:0/2632214329 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a8188a00 con 0x7f26a818a670 2021-02-10T11:46:08.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.099 7f268ffff700 1 -- 172.21.2.101:0/2632214329 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f26a000c810 con 0x7f26a818a670 2021-02-10T11:46:08.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.100 7f268ffff700 1 -- 172.21.2.101:0/2632214329 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f26a0001220 con 0x7f26a818a670 2021-02-10T11:46:08.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.100 7f268ffff700 1 --2- 172.21.2.101:0/2632214329 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26880186c0 0x7f268801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:08.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.100 7f268ffff700 1 -- 172.21.2.101:0/2632214329 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f26a0032900 con 0x7f26a818a670 2021-02-10T11:46:08.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.100 7f26a77fe700 1 --2- 172.21.2.101:0/2632214329 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26880186c0 0x7f268801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:08.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.100 7f26a77fe700 1 --2- 172.21.2.101:0/2632214329 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26880186c0 0x7f268801abc0 crc :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:08.103 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:46:08.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.131 7f26adaee700 1 -- 172.21.2.101:0/2632214329 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a80ebc90 con 0x7f26a818a670 2021-02-10T11:46:08.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.133 7f268ffff700 1 -- 172.21.2.101:0/2632214329 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f26a0000f40 con 0x7f26a818a670 2021-02-10T11:46:08.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.270 7f26adaee700 1 -- 172.21.2.101:0/2632214329 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs volume ls", "target": ["mgr", ""]}) v1 -- 0x7f26a8188a00 con 0x7f26880186c0 2021-02-10T11:46:08.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.270 7f268ffff700 1 -- 172.21.2.101:0/2632214329 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7f2698006540 con 0x7f26880186c0 2021-02-10T11:46:08.268 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:46:08.268 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:08.268 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "cephfs" 2021-02-10T11:46:08.269 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:08.269 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:46:08.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.272 7f26adaee700 1 -- 172.21.2.101:0/2632214329 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26880186c0 msgr2=0x7f268801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:08.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.272 7f26adaee700 1 --2- 172.21.2.101:0/2632214329 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26880186c0 0x7f268801abc0 crc :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:08.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.272 7f26adaee700 1 -- 172.21.2.101:0/2632214329 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26a818a670 msgr2=0x7f26a8187e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:08.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.272 7f26adaee700 1 --2- 172.21.2.101:0/2632214329 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26a818a670 0x7f26a8187e50 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 rx=0x7f26a0009940 tx=0x7f26a000bcf0).stop 2021-02-10T11:46:08.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.272 7f26adaee700 1 -- 172.21.2.101:0/2632214329 shutdown_connections 2021-02-10T11:46:08.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.272 7f26adaee700 1 --2- 172.21.2.101:0/2632214329 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26a818a670 0x7f26a8187e50 unknown :-1 s=CLOSED pgs=2914 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:08.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.272 7f26adaee700 1 --2- 172.21.2.101:0/2632214329 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f26a81a18f0 0x7f26a818a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:08.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.272 7f26adaee700 1 --2- 172.21.2.101:0/2632214329 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26880186c0 0x7f268801abc0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:08.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.272 7f26adaee700 1 --2- 172.21.2.101:0/2632214329 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26a818ae80 0x7f26a81a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:08.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.272 7f26adaee700 1 -- 172.21.2.101:0/2632214329 >> 172.21.2.101:0/2632214329 conn(0x7f26a80a2fd0 msgr2=0x7f26a80b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:08.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.272 7f26adaee700 1 -- 172.21.2.101:0/2632214329 shutdown_connections 2021-02-10T11:46:08.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.272 7f26adaee700 1 -- 172.21.2.101:0/2632214329 wait complete. 2021-02-10T11:46:08.278 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:46:08.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.735 7f0b43607700 1 Processor -- start 2021-02-10T11:46:08.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.735 7f0b43607700 1 -- start start 2021-02-10T11:46:08.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.736 7f0b43607700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0b3c0d1c70 0x7f0b3c0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:08.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.736 7f0b43607700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0b3c0cf580 0x7f0b3c0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:08.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.736 7f0b43607700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3c0fe1c0 con 0x7f0b3c0d1870 2021-02-10T11:46:08.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.736 7f0b43607700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3c1b0cb0 con 0x7f0b3c0cf580 2021-02-10T11:46:08.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.736 7f0b43607700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3c109a60 con 0x7f0b3c0d1c70 2021-02-10T11:46:08.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.736 7f0b41423700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f0b3c0d1870 0x7f0b3c1b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33464/0 (socket says 172.21.2.101:33464) 2021-02-10T11:46:08.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.736 7f0b41423700 1 -- 172.21.2.101:0/3974056278 learned_addr learned my addr 172.21.2.101:0/3974056278 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:08.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.736 7f0b41c24700 1 --2- 172.21.2.101:0/3974056278 >> v2:172.21.2.101:3300/0 conn(0x7f0b3c0cf580 0x7f0b3c0cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:08.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.736 7f0b41c24700 1 -- 172.21.2.101:0/3974056278 >> v2:172.21.2.108:3300/0 conn(0x7f0b3c0d1c70 msgr2=0x7f0b3c0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:08.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.736 7f0b41c24700 1 --2- 172.21.2.101:0/3974056278 >> v2:172.21.2.108:3300/0 conn(0x7f0b3c0d1c70 0x7f0b3c0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:08.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.736 7f0b41c24700 1 -- 172.21.2.101:0/3974056278 >> v1:172.21.2.101:6789/0 conn(0x7f0b3c0d1870 legacy=0x7f0b3c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:08.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.736 7f0b41c24700 1 -- 172.21.2.101:0/3974056278 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c088070 con 0x7f0b3c0cf580 2021-02-10T11:46:08.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.736 7f0b41c24700 1 --2- 172.21.2.101:0/3974056278 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b3c0cf580 0x7f0b3c0cf980 secure :-1 s=READY pgs=2916 cs=0 l=1 rev1=1 rx=0x7f0b34006e30 tx=0x7f0b34009e60).ready entity=mon.0 client_cookie=7aab5c0aa5a40241 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:08.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.737 7f0b2ffff700 1 -- 172.21.2.101:0/3974056278 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0b34006a50 con 0x7f0b3c0cf580 2021-02-10T11:46:08.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.737 7f0b2ffff700 1 -- 172.21.2.101:0/3974056278 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3400b300 con 0x7f0b3c0cf580 2021-02-10T11:46:08.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.737 7f0b2ffff700 1 -- 172.21.2.101:0/3974056278 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0b3400b820 con 0x7f0b3c0cf580 2021-02-10T11:46:08.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.737 7f0b43607700 1 -- 172.21.2.101:0/3974056278 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b3c0cf580 msgr2=0x7f0b3c0cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:08.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.737 7f0b43607700 1 --2- 172.21.2.101:0/3974056278 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b3c0cf580 0x7f0b3c0cf980 secure :-1 s=READY pgs=2916 cs=0 l=1 rev1=1 rx=0x7f0b34006e30 tx=0x7f0b34009e60).stop 2021-02-10T11:46:08.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.737 7f0b43607700 1 -- 172.21.2.101:0/3974056278 shutdown_connections 2021-02-10T11:46:08.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.737 7f0b43607700 1 --2- 172.21.2.101:0/3974056278 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b3c0cf580 0x7f0b3c0cf980 unknown :-1 s=CLOSED pgs=2916 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:08.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.737 7f0b43607700 1 --2- 172.21.2.101:0/3974056278 >> v2:172.21.2.108:3300/0 conn(0x7f0b3c0d1c70 0x7f0b3c0cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:08.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.737 7f0b43607700 1 -- 172.21.2.101:0/3974056278 >> 172.21.2.101:0/3974056278 conn(0x7f0b3c0ce420 msgr2=0x7f0b3c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:08.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.737 7f0b43607700 1 -- 172.21.2.101:0/3974056278 shutdown_connections 2021-02-10T11:46:08.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.737 7f0b43607700 1 -- 172.21.2.101:0/3974056278 wait complete. 2021-02-10T11:46:08.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.752 7f0b43607700 1 Processor -- start 2021-02-10T11:46:08.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.752 7f0b43607700 1 -- start start 2021-02-10T11:46:08.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.752 7f0b43607700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b3c0c3820 0x7f0b3c084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:08.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.752 7f0b43607700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b3c0a0290 0x7f0b3c0852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:08.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.752 7f0b43607700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b3c086270 0x7f0b3c086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:08.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.752 7f0b43607700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0c0700 con 0x7f0b3c0a0290 2021-02-10T11:46:08.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.752 7f0b43607700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0858f0 con 0x7f0b3c0c3820 2021-02-10T11:46:08.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.752 7f0b43607700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c085b10 con 0x7f0b3c086270 2021-02-10T11:46:08.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.753 7f0b40c22700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b3c0a0290 0x7f0b3c0852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:08.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.753 7f0b40c22700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b3c0a0290 0x7f0b3c0852d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32978/0 (socket says 172.21.2.101:32978) 2021-02-10T11:46:08.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.753 7f0b40c22700 1 -- 172.21.2.101:0/3048307128 learned_addr learned my addr 172.21.2.101:0/3048307128 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:08.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.753 7f0b40c22700 1 -- 172.21.2.101:0/3048307128 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b3c086270 msgr2=0x7f0b3c086670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:08.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.753 7f0b40c22700 1 --2- 172.21.2.101:0/3048307128 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b3c086270 0x7f0b3c086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:08.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.753 7f0b40c22700 1 -- 172.21.2.101:0/3048307128 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b3c0c3820 msgr2=0x7f0b3c084cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:08.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.753 7f0b40c22700 1 --2- 172.21.2.101:0/3048307128 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b3c0c3820 0x7f0b3c084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:08.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.753 7f0b40c22700 1 -- 172.21.2.101:0/3048307128 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c086c90 con 0x7f0b3c0a0290 2021-02-10T11:46:08.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.753 7f0b40c22700 1 --2- 172.21.2.101:0/3048307128 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b3c0a0290 0x7f0b3c0852d0 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 rx=0x7f0b38006f00 tx=0x7f0b3800b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:08.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.753 7f0b2dffb700 1 -- 172.21.2.101:0/3048307128 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0b38006350 con 0x7f0b3c0a0290 2021-02-10T11:46:08.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.753 7f0b2dffb700 1 -- 172.21.2.101:0/3048307128 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3800bd30 con 0x7f0b3c0a0290 2021-02-10T11:46:08.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.753 7f0b2dffb700 1 -- 172.21.2.101:0/3048307128 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0b380020d0 con 0x7f0b3c0a0290 2021-02-10T11:46:08.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.753 7f0b43607700 1 -- 172.21.2.101:0/3048307128 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3c086c90 con 0x7f0b3c0a0290 2021-02-10T11:46:08.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.754 7f0b43607700 1 -- 172.21.2.101:0/3048307128 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3c1a2f00 con 0x7f0b3c0a0290 2021-02-10T11:46:08.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.754 7f0b2dffb700 1 -- 172.21.2.101:0/3048307128 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0b38007020 con 0x7f0b3c0a0290 2021-02-10T11:46:08.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.754 7f0b2dffb700 1 --2- 172.21.2.101:0/3048307128 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b1c0186d0 0x7f0b1c01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:08.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.754 7f0b2dffb700 1 -- 172.21.2.101:0/3048307128 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0b380327c0 con 0x7f0b3c0a0290 2021-02-10T11:46:08.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.754 7f0b41423700 1 --2- 172.21.2.101:0/3048307128 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b1c0186d0 0x7f0b1c01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:08.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.754 7f0b41423700 1 --2- 172.21.2.101:0/3048307128 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b1c0186d0 0x7f0b1c01abd0 crc :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:08.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.786 7f0b43607700 1 -- 172.21.2.101:0/3048307128 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3c082a70 con 0x7f0b3c0a0290 2021-02-10T11:46:08.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.788 7f0b2dffb700 1 -- 172.21.2.101:0/3048307128 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0b38000960 con 0x7f0b3c0a0290 2021-02-10T11:46:08.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.937 7f0b43607700 1 -- 172.21.2.101:0/3048307128 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0b3c082a70 con 0x7f0b3c0a0290 2021-02-10T11:46:08.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.937 7f0b2dffb700 1 -- 172.21.2.101:0/3048307128 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0b38007000 con 0x7f0b3c0a0290 2021-02-10T11:46:08.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.939 7f0b43607700 1 -- 172.21.2.101:0/3048307128 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b1c0186d0 msgr2=0x7f0b1c01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:08.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.939 7f0b43607700 1 --2- 172.21.2.101:0/3048307128 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b1c0186d0 0x7f0b1c01abd0 crc :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:08.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.939 7f0b43607700 1 -- 172.21.2.101:0/3048307128 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b3c0a0290 msgr2=0x7f0b3c0852d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:08.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.939 7f0b43607700 1 --2- 172.21.2.101:0/3048307128 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b3c0a0290 0x7f0b3c0852d0 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 rx=0x7f0b38006f00 tx=0x7f0b3800b940).stop 2021-02-10T11:46:08.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.940 7f0b43607700 1 -- 172.21.2.101:0/3048307128 shutdown_connections 2021-02-10T11:46:08.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.940 7f0b43607700 1 --2- 172.21.2.101:0/3048307128 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b1c0186d0 0x7f0b1c01abd0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:08.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.940 7f0b43607700 1 --2- 172.21.2.101:0/3048307128 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b3c086270 0x7f0b3c086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:08.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.940 7f0b43607700 1 --2- 172.21.2.101:0/3048307128 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b3c0a0290 0x7f0b3c0852d0 unknown :-1 s=CLOSED pgs=2917 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:08.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.940 7f0b43607700 1 --2- 172.21.2.101:0/3048307128 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b3c0c3820 0x7f0b3c084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:08.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.940 7f0b43607700 1 -- 172.21.2.101:0/3048307128 >> 172.21.2.101:0/3048307128 conn(0x7f0b3c0872c0 msgr2=0x7f0b3c095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:08.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.940 7f0b43607700 1 -- 172.21.2.101:0/3048307128 shutdown_connections 2021-02-10T11:46:08.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:08.940 7f0b43607700 1 -- 172.21.2.101:0/3048307128 wait complete. 2021-02-10T11:46:08.946 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000927842 2021-02-10T11:46:09.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.363 7f627c956700 1 Processor -- start 2021-02-10T11:46:09.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.363 7f627c956700 1 -- start start 2021-02-10T11:46:09.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.364 7f627c956700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62780d1870 0x7f62781b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:09.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.364 7f627c956700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f62781b0ae0 0x7f6278108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:09.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.364 7f627c956700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6278109a60 con 0x7f62780d1c70 2021-02-10T11:46:09.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.364 7f627c956700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62780e5ef0 con 0x7f62781b0ae0 2021-02-10T11:46:09.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.364 7f627c956700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6278109750 con 0x7f62780d1870 2021-02-10T11:46:09.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.365 7f627661c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62780d1870 0x7f62781b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:09.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.365 7f627661c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62780d1870 0x7f62781b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:53948/0 (socket says 172.21.2.101:53948) 2021-02-10T11:46:09.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.365 7f627661c700 1 -- 172.21.2.101:0/906067518 learned_addr learned my addr 172.21.2.101:0/906067518 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:09.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.365 7f6276e1d700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f62780d1c70 0x7f6278108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:55120/0 (socket says 172.21.2.101:55120) 2021-02-10T11:46:09.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.365 7f6275e1b700 1 --2- 172.21.2.101:0/906067518 >> v2:172.21.2.108:3300/0 conn(0x7f62781b0ae0 0x7f6278108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:09.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.365 7f627661c700 1 -- 172.21.2.101:0/906067518 >> v2:172.21.2.108:3300/0 conn(0x7f62781b0ae0 msgr2=0x7f6278108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:09.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.365 7f627661c700 1 --2- 172.21.2.101:0/906067518 >> v2:172.21.2.108:3300/0 conn(0x7f62781b0ae0 0x7f6278108940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:09.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.365 7f627661c700 1 -- 172.21.2.101:0/906067518 >> v1:172.21.2.108:6789/0 conn(0x7f62780d1c70 legacy=0x7f6278108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:09.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.365 7f627561a700 1 -- 172.21.2.101:0/906067518 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4084574946 0 0) 0x7f6270001440 con 0x7f62780d1c70 2021-02-10T11:46:09.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.365 7f627661c700 1 -- 172.21.2.101:0/906067518 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6278088070 con 0x7f62780d1870 2021-02-10T11:46:09.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.365 7f627661c700 1 --2- 172.21.2.101:0/906067518 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62780d1870 0x7f62781b04c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f6268006700 tx=0x7f6268009b00).ready entity=mon.2 client_cookie=650f1149ec358d4c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:09.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.366 7f627561a700 1 -- 172.21.2.101:0/906067518 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f626800c0c0 con 0x7f62780d1870 2021-02-10T11:46:09.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.366 7f627561a700 1 -- 172.21.2.101:0/906067518 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626800c400 con 0x7f62780d1870 2021-02-10T11:46:09.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.366 7f627561a700 1 -- 172.21.2.101:0/906067518 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f626800c920 con 0x7f62780d1870 2021-02-10T11:46:09.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.366 7f627c956700 1 -- 172.21.2.101:0/906067518 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62780d1870 msgr2=0x7f62781b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:09.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.366 7f627c956700 1 --2- 172.21.2.101:0/906067518 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62780d1870 0x7f62781b04c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f6268006700 tx=0x7f6268009b00).stop 2021-02-10T11:46:09.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.366 7f627c956700 1 -- 172.21.2.101:0/906067518 shutdown_connections 2021-02-10T11:46:09.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.366 7f627c956700 1 --2- 172.21.2.101:0/906067518 >> v2:172.21.2.108:3300/0 conn(0x7f62781b0ae0 0x7f6278108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:09.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.366 7f627c956700 1 --2- 172.21.2.101:0/906067518 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62780d1870 0x7f62781b04c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:09.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.366 7f627c956700 1 -- 172.21.2.101:0/906067518 >> 172.21.2.101:0/906067518 conn(0x7f62780ce420 msgr2=0x7f62780d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:09.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.367 7f627c956700 1 -- 172.21.2.101:0/906067518 shutdown_connections 2021-02-10T11:46:09.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.367 7f627c956700 1 -- 172.21.2.101:0/906067518 wait complete. 2021-02-10T11:46:09.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.381 7f627c956700 1 Processor -- start 2021-02-10T11:46:09.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.381 7f627c956700 1 -- start start 2021-02-10T11:46:09.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.382 7f627c956700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62780abd60 0x7f62780ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:09.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.382 7f627c956700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62780ac780 0x7f62780acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:09.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.382 7f627c956700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6278083910 0x7f6278083d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:09.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.382 7f627c956700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6278087180 con 0x7f62780ac780 2021-02-10T11:46:09.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.382 7f627c956700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627818fcd0 con 0x7f62780abd60 2021-02-10T11:46:09.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.382 7f627c956700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6278084330 con 0x7f6278083910 2021-02-10T11:46:09.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.382 7f6275e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62780ac780 0x7f62780acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:09.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.382 7f6275e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62780ac780 0x7f62780acb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:32990/0 (socket says 172.21.2.101:32990) 2021-02-10T11:46:09.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.382 7f6275e1b700 1 -- 172.21.2.101:0/195029007 learned_addr learned my addr 172.21.2.101:0/195029007 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:09.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.382 7f6276e1d700 1 --2- 172.21.2.101:0/195029007 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6278083910 0x7f6278083d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:09.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.382 7f627661c700 1 --2- 172.21.2.101:0/195029007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62780abd60 0x7f62780ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:09.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.382 7f6275e1b700 1 -- 172.21.2.101:0/195029007 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6278083910 msgr2=0x7f6278083d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:09.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.382 7f6275e1b700 1 --2- 172.21.2.101:0/195029007 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6278083910 0x7f6278083d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:09.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.382 7f6275e1b700 1 -- 172.21.2.101:0/195029007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62780abd60 msgr2=0x7f62780ac160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:09.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.382 7f6275e1b700 1 --2- 172.21.2.101:0/195029007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62780abd60 0x7f62780ac160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:09.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.382 7f6275e1b700 1 -- 172.21.2.101:0/195029007 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6278084550 con 0x7f62780ac780 2021-02-10T11:46:09.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.383 7f6275e1b700 1 --2- 172.21.2.101:0/195029007 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62780ac780 0x7f62780acb80 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 rx=0x7f6260006f60 tx=0x7f626000b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:09.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.383 7f626effd700 1 -- 172.21.2.101:0/195029007 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f626000bd60 con 0x7f62780ac780 2021-02-10T11:46:09.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.383 7f626effd700 1 -- 172.21.2.101:0/195029007 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6260000c90 con 0x7f62780ac780 2021-02-10T11:46:09.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.383 7f627c956700 1 -- 172.21.2.101:0/195029007 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6278084550 con 0x7f62780ac780 2021-02-10T11:46:09.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.383 7f626effd700 1 -- 172.21.2.101:0/195029007 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6260002220 con 0x7f62780ac780 2021-02-10T11:46:09.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.383 7f627c956700 1 -- 172.21.2.101:0/195029007 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6278084550 con 0x7f62780ac780 2021-02-10T11:46:09.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.383 7f626effd700 1 -- 172.21.2.101:0/195029007 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f62600052e0 con 0x7f62780ac780 2021-02-10T11:46:09.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.383 7f626effd700 1 --2- 172.21.2.101:0/195029007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f625c018700 0x7f625c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:09.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.383 7f626effd700 1 -- 172.21.2.101:0/195029007 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6260032640 con 0x7f62780ac780 2021-02-10T11:46:09.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.383 7f627661c700 1 --2- 172.21.2.101:0/195029007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f625c018700 0x7f625c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:09.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.384 7f627661c700 1 --2- 172.21.2.101:0/195029007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f625c018700 0x7f625c01abc0 crc :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:09.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.413 7f627c956700 1 -- 172.21.2.101:0/195029007 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6278082980 con 0x7f62780ac780 2021-02-10T11:46:09.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.416 7f626effd700 1 -- 172.21.2.101:0/195029007 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6260000c90 con 0x7f62780ac780 2021-02-10T11:46:09.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.543 7f627c956700 1 -- 172.21.2.101:0/195029007 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000927842", "target": ["mgr", ""]}) v1 -- 0x7f6278084dd0 con 0x7f625c018700 2021-02-10T11:46:09.545 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:46:09.547 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:46:09.545 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:46:09.547 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:46:09.545 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:46:09.547 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:46:09.545 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:46:09.547 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:46:09.545 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:46:09.547 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:46:09.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.576 7f626effd700 1 -- 172.21.2.101:0/195029007 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f6268006410 con 0x7f625c018700 2021-02-10T11:46:09.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.608 7f627c956700 1 -- 172.21.2.101:0/195029007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f625c018700 msgr2=0x7f625c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:09.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.608 7f627c956700 1 --2- 172.21.2.101:0/195029007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f625c018700 0x7f625c01abc0 crc :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:09.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.608 7f627c956700 1 -- 172.21.2.101:0/195029007 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62780ac780 msgr2=0x7f62780acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:09.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.608 7f627c956700 1 --2- 172.21.2.101:0/195029007 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62780ac780 0x7f62780acb80 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 rx=0x7f6260006f60 tx=0x7f626000b970).stop 2021-02-10T11:46:09.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.608 7f627c956700 1 -- 172.21.2.101:0/195029007 shutdown_connections 2021-02-10T11:46:09.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.608 7f627c956700 1 --2- 172.21.2.101:0/195029007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f625c018700 0x7f625c01abc0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:09.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.608 7f627c956700 1 --2- 172.21.2.101:0/195029007 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6278083910 0x7f6278083d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:09.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.608 7f627c956700 1 --2- 172.21.2.101:0/195029007 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62780ac780 0x7f62780acb80 unknown :-1 s=CLOSED pgs=2918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:09.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.608 7f627c956700 1 --2- 172.21.2.101:0/195029007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62780abd60 0x7f62780ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:09.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.608 7f627c956700 1 -- 172.21.2.101:0/195029007 >> 172.21.2.101:0/195029007 conn(0x7f62780a2b30 msgr2=0x7f627809ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:09.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.608 7f627c956700 1 -- 172.21.2.101:0/195029007 shutdown_connections 2021-02-10T11:46:09.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:09.609 7f627c956700 1 -- 172.21.2.101:0/195029007 wait complete. 2021-02-10T11:46:09.615 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume snapshot create cephfs subvolume_0000000000927842 snapshot_0000000000667014 2021-02-10T11:46:10.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.037 7f442b802700 1 Processor -- start 2021-02-10T11:46:10.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.037 7f442b802700 1 -- start start 2021-02-10T11:46:10.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.037 7f442b802700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f44240d1880 0x7f44241b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:10.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.037 7f442b802700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44241b0af0 0x7f4424108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:10.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.037 7f442b802700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44240fe1d0 con 0x7f44240d1c80 2021-02-10T11:46:10.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.037 7f442b802700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4424109a70 con 0x7f44240d1880 2021-02-10T11:46:10.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.037 7f442b802700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4424109760 con 0x7f44241b0af0 2021-02-10T11:46:10.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.038 7f4429e1f700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f44240d1c80 0x7f4424108f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:55144/0 (socket says 172.21.2.101:55144) 2021-02-10T11:46:10.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.038 7f4429e1f700 1 -- 172.21.2.101:0/1126785347 learned_addr learned my addr 172.21.2.101:0/1126785347 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:10.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.038 7f442961e700 1 --2- 172.21.2.101:0/1126785347 >> v2:172.21.2.108:3300/0 conn(0x7f44240d1880 0x7f44241b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:10.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.038 7f4428e1d700 1 --2- 172.21.2.101:0/1126785347 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44241b0af0 0x7f4424108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:10.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.038 7f441bfff700 1 -- 172.21.2.101:0/1126785347 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 403205345 0 0) 0x7f4420001190 con 0x7f44240d1c80 2021-02-10T11:46:10.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.038 7f4428e1d700 1 -- 172.21.2.101:0/1126785347 >> v2:172.21.2.108:3300/0 conn(0x7f44240d1880 msgr2=0x7f44241b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:10.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.038 7f4428e1d700 1 --2- 172.21.2.101:0/1126785347 >> v2:172.21.2.108:3300/0 conn(0x7f44240d1880 0x7f44241b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:10.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.038 7f4428e1d700 1 -- 172.21.2.101:0/1126785347 >> v1:172.21.2.108:6789/0 conn(0x7f44240d1c80 legacy=0x7f4424108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:10.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.038 7f4428e1d700 1 -- 172.21.2.101:0/1126785347 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4424088070 con 0x7f44241b0af0 2021-02-10T11:46:10.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.038 7f4428e1d700 1 --2- 172.21.2.101:0/1126785347 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44241b0af0 0x7f4424108950 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f441c0069d0 tx=0x7f441c009e40).ready entity=mon.2 client_cookie=996b2469ac9d2d8e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:10.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.038 7f441bfff700 1 -- 172.21.2.101:0/1126785347 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f441c000d30 con 0x7f44241b0af0 2021-02-10T11:46:10.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.038 7f441bfff700 1 -- 172.21.2.101:0/1126785347 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441c00b300 con 0x7f44241b0af0 2021-02-10T11:46:10.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.038 7f441bfff700 1 -- 172.21.2.101:0/1126785347 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f441c00b820 con 0x7f44241b0af0 2021-02-10T11:46:10.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.039 7f442b802700 1 -- 172.21.2.101:0/1126785347 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44241b0af0 msgr2=0x7f4424108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:10.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.039 7f442b802700 1 --2- 172.21.2.101:0/1126785347 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44241b0af0 0x7f4424108950 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f441c0069d0 tx=0x7f441c009e40).stop 2021-02-10T11:46:10.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.039 7f442b802700 1 -- 172.21.2.101:0/1126785347 shutdown_connections 2021-02-10T11:46:10.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.039 7f442b802700 1 --2- 172.21.2.101:0/1126785347 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f44241b0af0 0x7f4424108950 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:10.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.039 7f442b802700 1 --2- 172.21.2.101:0/1126785347 >> v2:172.21.2.108:3300/0 conn(0x7f44240d1880 0x7f44241b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:10.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.039 7f442b802700 1 -- 172.21.2.101:0/1126785347 >> 172.21.2.101:0/1126785347 conn(0x7f44240ce430 msgr2=0x7f44240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:10.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.039 7f442b802700 1 -- 172.21.2.101:0/1126785347 shutdown_connections 2021-02-10T11:46:10.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.039 7f442b802700 1 -- 172.21.2.101:0/1126785347 wait complete. 2021-02-10T11:46:10.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.054 7f442b802700 1 Processor -- start 2021-02-10T11:46:10.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.054 7f442b802700 1 -- start start 2021-02-10T11:46:10.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.054 7f442b802700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44240aef00 0x7f4424099c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:10.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.054 7f442b802700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f442409a260 0x7f442409a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:10.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.054 7f442b802700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f442409ac80 0x7f442419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:10.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.054 7f442b802700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442419bdd0 con 0x7f442409ac80 2021-02-10T11:46:10.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.054 7f442b802700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442419de70 con 0x7f44240aef00 2021-02-10T11:46:10.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.054 7f442b802700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f442419a2d0 con 0x7f442409a260 2021-02-10T11:46:10.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.055 7f4429e1f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f442409ac80 0x7f442419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:10.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.055 7f4428e1d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f442409a260 0x7f442409a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:10.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.055 7f4429e1f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f442409ac80 0x7f442419a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33016/0 (socket says 172.21.2.101:33016) 2021-02-10T11:46:10.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.055 7f4429e1f700 1 -- 172.21.2.101:0/3502428551 learned_addr learned my addr 172.21.2.101:0/3502428551 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:10.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.055 7f442961e700 1 --2- 172.21.2.101:0/3502428551 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44240aef00 0x7f4424099c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:10.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.055 7f4429e1f700 1 -- 172.21.2.101:0/3502428551 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f442409a260 msgr2=0x7f442409a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:10.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.055 7f4429e1f700 1 --2- 172.21.2.101:0/3502428551 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f442409a260 0x7f442409a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:10.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.055 7f4429e1f700 1 -- 172.21.2.101:0/3502428551 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44240aef00 msgr2=0x7f4424099c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:10.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.055 7f4429e1f700 1 --2- 172.21.2.101:0/3502428551 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44240aef00 0x7f4424099c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:10.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.055 7f4429e1f700 1 -- 172.21.2.101:0/3502428551 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4424199f50 con 0x7f442409ac80 2021-02-10T11:46:10.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.055 7f4429e1f700 1 --2- 172.21.2.101:0/3502428551 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f442409ac80 0x7f442419a850 secure :-1 s=READY pgs=2921 cs=0 l=1 rev1=1 rx=0x7f4420001e40 tx=0x7f442000a880).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:10.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.055 7f4419ffb700 1 -- 172.21.2.101:0/3502428551 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4420000bc0 con 0x7f442409ac80 2021-02-10T11:46:10.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.055 7f4419ffb700 1 -- 172.21.2.101:0/3502428551 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442000ac70 con 0x7f442409ac80 2021-02-10T11:46:10.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.055 7f4419ffb700 1 -- 172.21.2.101:0/3502428551 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4420004510 con 0x7f442409ac80 2021-02-10T11:46:10.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.055 7f442b802700 1 -- 172.21.2.101:0/3502428551 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4424199f50 con 0x7f442409ac80 2021-02-10T11:46:10.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.056 7f442b802700 1 -- 172.21.2.101:0/3502428551 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4424199f50 con 0x7f442409ac80 2021-02-10T11:46:10.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.056 7f4419ffb700 1 -- 172.21.2.101:0/3502428551 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4420008000 con 0x7f442409ac80 2021-02-10T11:46:10.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.056 7f4419ffb700 1 --2- 172.21.2.101:0/3502428551 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44040186c0 0x7f440401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:10.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.056 7f4419ffb700 1 -- 172.21.2.101:0/3502428551 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4420032790 con 0x7f442409ac80 2021-02-10T11:46:10.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.056 7f442961e700 1 --2- 172.21.2.101:0/3502428551 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44040186c0 0x7f440401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:10.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.056 7f442961e700 1 --2- 172.21.2.101:0/3502428551 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44040186c0 0x7f440401abc0 crc :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:10.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.087 7f442b802700 1 -- 172.21.2.101:0/3502428551 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4424082a70 con 0x7f442409ac80 2021-02-10T11:46:10.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.089 7f4419ffb700 1 -- 172.21.2.101:0/3502428551 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4420000b60 con 0x7f442409ac80 2021-02-10T11:46:10.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.217 7f442b802700 1 -- 172.21.2.101:0/3502428551 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000927842", "target": ["mgr", ""], "snap_name": "snapshot_0000000000667014"}) v1 -- 0x7f442419de70 con 0x7f44040186c0 2021-02-10T11:46:10.225 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.228 7f4419ffb700 1 -- 172.21.2.101:0/3502428551 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f4414006a70 con 0x7f44040186c0 2021-02-10T11:46:10.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.234 7f442b802700 1 -- 172.21.2.101:0/3502428551 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44040186c0 msgr2=0x7f440401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:10.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.234 7f442b802700 1 --2- 172.21.2.101:0/3502428551 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44040186c0 0x7f440401abc0 crc :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:10.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.234 7f442b802700 1 -- 172.21.2.101:0/3502428551 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f442409ac80 msgr2=0x7f442419a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:10.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.234 7f442b802700 1 --2- 172.21.2.101:0/3502428551 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f442409ac80 0x7f442419a850 secure :-1 s=READY pgs=2921 cs=0 l=1 rev1=1 rx=0x7f4420001e40 tx=0x7f442000a880).stop 2021-02-10T11:46:10.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.234 7f442b802700 1 -- 172.21.2.101:0/3502428551 shutdown_connections 2021-02-10T11:46:10.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.234 7f442b802700 1 --2- 172.21.2.101:0/3502428551 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44040186c0 0x7f440401abc0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:10.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.234 7f442b802700 1 --2- 172.21.2.101:0/3502428551 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f442409a260 0x7f442409a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:10.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.234 7f442b802700 1 --2- 172.21.2.101:0/3502428551 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f442409ac80 0x7f442419a850 unknown :-1 s=CLOSED pgs=2921 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:10.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.234 7f442b802700 1 --2- 172.21.2.101:0/3502428551 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f44240aef00 0x7f4424099c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:10.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.234 7f442b802700 1 -- 172.21.2.101:0/3502428551 >> 172.21.2.101:0/3502428551 conn(0x7f442418e810 msgr2=0x7f442418c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:10.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.235 7f442b802700 1 -- 172.21.2.101:0/3502428551 shutdown_connections 2021-02-10T11:46:10.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.235 7f442b802700 1 -- 172.21.2.101:0/3502428551 wait complete. 2021-02-10T11:46:10.246 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume snapshot rm cephfs subvolume_0000000000927842 snapshot_0000000000667014 2021-02-10T11:46:10.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.663 7ff267fb2700 1 Processor -- start 2021-02-10T11:46:10.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.663 7ff267fb2700 1 -- start start 2021-02-10T11:46:10.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.664 7ff267fb2700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff2600d1880 0x7ff2601b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:10.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.664 7ff267fb2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2601b0af0 0x7ff260108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:10.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.664 7ff267fb2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff2600d1c80 0x7ff260108f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:10.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.664 7ff267fb2700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2600fe1d0 con 0x7ff2600d1c80 2021-02-10T11:46:10.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.664 7ff267fb2700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff260109a70 con 0x7ff2600d1880 2021-02-10T11:46:10.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.664 7ff267fb2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff260088070 con 0x7ff2601b0af0 2021-02-10T11:46:10.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.664 7ff2665cf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff2600d1c80 0x7ff260108f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:10.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.664 7ff2665cf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff2600d1c80 0x7ff260108f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33024/0 (socket says 172.21.2.101:33024) 2021-02-10T11:46:10.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.664 7ff2665cf700 1 -- 172.21.2.101:0/1564440338 learned_addr learned my addr 172.21.2.101:0/1564440338 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:10.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.664 7ff2665cf700 1 -- 172.21.2.101:0/1564440338 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2601b0af0 msgr2=0x7ff260108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:10.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.664 7ff2655cd700 1 --2- 172.21.2.101:0/1564440338 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2601b0af0 0x7ff260108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:10.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.664 7ff265dce700 1 --2- 172.21.2.101:0/1564440338 >> v2:172.21.2.108:3300/0 conn(0x7ff2600d1880 0x7ff2601b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:10.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.664 7ff2665cf700 1 --2- 172.21.2.101:0/1564440338 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2601b0af0 0x7ff260108950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:10.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.664 7ff2665cf700 1 -- 172.21.2.101:0/1564440338 >> v2:172.21.2.108:3300/0 conn(0x7ff2600d1880 msgr2=0x7ff2601b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:10.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.664 7ff2665cf700 1 --2- 172.21.2.101:0/1564440338 >> v2:172.21.2.108:3300/0 conn(0x7ff2600d1880 0x7ff2601b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:10.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.664 7ff2665cf700 1 -- 172.21.2.101:0/1564440338 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff260109590 con 0x7ff2600d1c80 2021-02-10T11:46:10.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.664 7ff2655cd700 1 --2- 172.21.2.101:0/1564440338 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2601b0af0 0x7ff260108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:46:10.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.664 7ff265dce700 1 --2- 172.21.2.101:0/1564440338 >> v2:172.21.2.108:3300/0 conn(0x7ff2600d1880 0x7ff2601b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:46:10.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.664 7ff2665cf700 1 --2- 172.21.2.101:0/1564440338 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2600d1c80 0x7ff260108f70 secure :-1 s=READY pgs=2922 cs=0 l=1 rev1=1 rx=0x7ff258006f00 tx=0x7ff258009f10).ready entity=mon.0 client_cookie=13d8e360b90cb30d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:10.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.665 7ff264dcc700 1 -- 172.21.2.101:0/1564440338 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff25800b3b0 con 0x7ff2600d1c80 2021-02-10T11:46:10.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.665 7ff264dcc700 1 -- 172.21.2.101:0/1564440338 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff258000d30 con 0x7ff2600d1c80 2021-02-10T11:46:10.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.665 7ff264dcc700 1 -- 172.21.2.101:0/1564440338 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff25800b820 con 0x7ff2600d1c80 2021-02-10T11:46:10.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.665 7ff267fb2700 1 -- 172.21.2.101:0/1564440338 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2600d1c80 msgr2=0x7ff260108f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:10.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.665 7ff267fb2700 1 --2- 172.21.2.101:0/1564440338 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2600d1c80 0x7ff260108f70 secure :-1 s=READY pgs=2922 cs=0 l=1 rev1=1 rx=0x7ff258006f00 tx=0x7ff258009f10).stop 2021-02-10T11:46:10.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.665 7ff267fb2700 1 -- 172.21.2.101:0/1564440338 shutdown_connections 2021-02-10T11:46:10.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.665 7ff267fb2700 1 --2- 172.21.2.101:0/1564440338 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff2601b0af0 0x7ff260108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:10.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.665 7ff267fb2700 1 --2- 172.21.2.101:0/1564440338 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2600d1c80 0x7ff260108f70 unknown :-1 s=CLOSED pgs=2922 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:10.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.665 7ff267fb2700 1 --2- 172.21.2.101:0/1564440338 >> v2:172.21.2.108:3300/0 conn(0x7ff2600d1880 0x7ff2601b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:10.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.665 7ff267fb2700 1 -- 172.21.2.101:0/1564440338 >> 172.21.2.101:0/1564440338 conn(0x7ff2600ce430 msgr2=0x7ff2600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:10.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.666 7ff267fb2700 1 -- 172.21.2.101:0/1564440338 shutdown_connections 2021-02-10T11:46:10.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.666 7ff267fb2700 1 -- 172.21.2.101:0/1564440338 wait complete. 2021-02-10T11:46:10.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.680 7ff267fb2700 1 Processor -- start 2021-02-10T11:46:10.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.680 7ff267fb2700 1 -- start start 2021-02-10T11:46:10.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.680 7ff267fb2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2601a3570 0x7ff2601a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:10.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.680 7ff267fb2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff26009c9f0 0x7ff26009cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:10.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.680 7ff267fb2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff26009d410 0x7ff260098170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:10.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.680 7ff267fb2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26009c470 con 0x7ff2601a3570 2021-02-10T11:46:10.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.680 7ff267fb2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2600b2840 con 0x7ff26009c9f0 2021-02-10T11:46:10.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.680 7ff267fb2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2600b2a20 con 0x7ff26009d410 2021-02-10T11:46:10.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.680 7ff265dce700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2601a3570 0x7ff2601a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:10.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.680 7ff265dce700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2601a3570 0x7ff2601a3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33028/0 (socket says 172.21.2.101:33028) 2021-02-10T11:46:10.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.681 7ff265dce700 1 -- 172.21.2.101:0/773436489 learned_addr learned my addr 172.21.2.101:0/773436489 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:10.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.681 7ff265dce700 1 -- 172.21.2.101:0/773436489 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff26009d410 msgr2=0x7ff260098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:10.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.681 7ff265dce700 1 --2- 172.21.2.101:0/773436489 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff26009d410 0x7ff260098170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:10.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.681 7ff265dce700 1 -- 172.21.2.101:0/773436489 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff26009c9f0 msgr2=0x7ff26009cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:10.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.681 7ff265dce700 1 --2- 172.21.2.101:0/773436489 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff26009c9f0 0x7ff26009cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:10.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.681 7ff265dce700 1 -- 172.21.2.101:0/773436489 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26009d880 con 0x7ff2601a3570 2021-02-10T11:46:10.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.681 7ff265dce700 1 --2- 172.21.2.101:0/773436489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2601a3570 0x7ff2601a3970 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 rx=0x7ff254006f20 tx=0x7ff25400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:10.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.681 7ff2527fc700 1 -- 172.21.2.101:0/773436489 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff25400bd80 con 0x7ff2601a3570 2021-02-10T11:46:10.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.681 7ff2527fc700 1 -- 172.21.2.101:0/773436489 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff254000c90 con 0x7ff2601a3570 2021-02-10T11:46:10.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.681 7ff2527fc700 1 -- 172.21.2.101:0/773436489 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff2540022b0 con 0x7ff2601a3570 2021-02-10T11:46:10.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.681 7ff267fb2700 1 -- 172.21.2.101:0/773436489 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26009d880 con 0x7ff2601a3570 2021-02-10T11:46:10.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.681 7ff267fb2700 1 -- 172.21.2.101:0/773436489 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26009d880 con 0x7ff2601a3570 2021-02-10T11:46:10.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.682 7ff2527fc700 1 -- 172.21.2.101:0/773436489 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff25400a000 con 0x7ff2601a3570 2021-02-10T11:46:10.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.682 7ff2527fc700 1 --2- 172.21.2.101:0/773436489 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2480186c0 0x7ff24801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:10.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.682 7ff2527fc700 1 -- 172.21.2.101:0/773436489 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff254032400 con 0x7ff2601a3570 2021-02-10T11:46:10.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.682 7ff2655cd700 1 --2- 172.21.2.101:0/773436489 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2480186c0 0x7ff24801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:10.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.682 7ff2655cd700 1 --2- 172.21.2.101:0/773436489 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2480186c0 0x7ff24801abc0 crc :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:10.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.713 7ff267fb2700 1 -- 172.21.2.101:0/773436489 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff260082a70 con 0x7ff2601a3570 2021-02-10T11:46:10.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.716 7ff2527fc700 1 -- 172.21.2.101:0/773436489 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff254006bb0 con 0x7ff2601a3570 2021-02-10T11:46:10.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.843 7ff267fb2700 1 -- 172.21.2.101:0/773436489 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000927842", "target": ["mgr", ""], "snap_name": "snapshot_0000000000667014"}) v1 -- 0x7ff260082a70 con 0x7ff2480186c0 2021-02-10T11:46:10.852 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.855 7ff2527fc700 1 -- 172.21.2.101:0/773436489 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7ff25c0009b0 con 0x7ff2480186c0 2021-02-10T11:46:10.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.859 7ff267fb2700 1 -- 172.21.2.101:0/773436489 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2480186c0 msgr2=0x7ff24801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:10.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.859 7ff267fb2700 1 --2- 172.21.2.101:0/773436489 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2480186c0 0x7ff24801abc0 crc :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:10.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.859 7ff267fb2700 1 -- 172.21.2.101:0/773436489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2601a3570 msgr2=0x7ff2601a3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:10.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.859 7ff267fb2700 1 --2- 172.21.2.101:0/773436489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2601a3570 0x7ff2601a3970 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 rx=0x7ff254006f20 tx=0x7ff25400b990).stop 2021-02-10T11:46:10.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.859 7ff267fb2700 1 -- 172.21.2.101:0/773436489 shutdown_connections 2021-02-10T11:46:10.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.859 7ff267fb2700 1 --2- 172.21.2.101:0/773436489 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff2480186c0 0x7ff24801abc0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:10.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.859 7ff267fb2700 1 --2- 172.21.2.101:0/773436489 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff26009d410 0x7ff260098170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:10.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.859 7ff267fb2700 1 --2- 172.21.2.101:0/773436489 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff26009c9f0 0x7ff26009cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:10.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.859 7ff267fb2700 1 --2- 172.21.2.101:0/773436489 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff2601a3570 0x7ff2601a3970 unknown :-1 s=CLOSED pgs=2923 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:10.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.859 7ff267fb2700 1 -- 172.21.2.101:0/773436489 >> 172.21.2.101:0/773436489 conn(0x7ff2600c2cd0 msgr2=0x7ff2600b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:10.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.859 7ff267fb2700 1 -- 172.21.2.101:0/773436489 shutdown_connections 2021-02-10T11:46:10.858 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:10.859 7ff267fb2700 1 -- 172.21.2.101:0/773436489 wait complete. 2021-02-10T11:46:10.866 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume snapshot rm cephfs subvolume_0000000000927842 snapshot_0000000000667014 2021-02-10T11:46:11.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.284 7f2bf3888700 1 Processor -- start 2021-02-10T11:46:11.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.284 7f2bf3888700 1 -- start start 2021-02-10T11:46:11.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.285 7f2bf3888700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bec10b000 0x7f2bec0cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:11.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.285 7f2bf3888700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bec0fe1d0 con 0x7f2bec0d1880 2021-02-10T11:46:11.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.285 7f2bf3888700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bec1b0cc0 con 0x7f2bec0d1c80 2021-02-10T11:46:11.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.285 7f2bf3888700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec109a70 con 0x7f2bec10b000 2021-02-10T11:46:11.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.285 7f2bf16a4700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f2bec0d1880 0x7f2bec1b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33528/0 (socket says 172.21.2.101:33528) 2021-02-10T11:46:11.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.285 7f2bf16a4700 1 -- 172.21.2.101:0/4130493262 learned_addr learned my addr 172.21.2.101:0/4130493262 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:11.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.285 7f2bdffff700 1 -- 172.21.2.101:0/4130493262 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 239060348 0 0) 0x7f2be8001180 con 0x7f2bec0d1880 2021-02-10T11:46:11.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.285 7f2bdffff700 1 -- 172.21.2.101:0/4130493262 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bd8002fe0 con 0x7f2bec0d1880 2021-02-10T11:46:11.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.285 7f2bf1ea5700 1 --2- 172.21.2.101:0/4130493262 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bec10b000 0x7f2bec0cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:11.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.286 7f2bdffff700 1 -- 172.21.2.101:0/4130493262 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2687869725 0 0) 0x7f2be8001180 con 0x7f2bec0d1880 2021-02-10T11:46:11.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.286 7f2bdffff700 1 -- 172.21.2.101:0/4130493262 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bec10b000 msgr2=0x7f2bec0cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:11.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.286 7f2bdffff700 1 --2- 172.21.2.101:0/4130493262 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bec10b000 0x7f2bec0cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:11.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.286 7f2bdffff700 1 -- 172.21.2.101:0/4130493262 >> v1:172.21.2.108:6789/0 conn(0x7f2bec0d1c80 legacy=0x7f2bec0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:11.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.286 7f2bdffff700 1 -- 172.21.2.101:0/4130493262 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bec10b400 con 0x7f2bec0d1880 2021-02-10T11:46:11.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.286 7f2bdffff700 1 -- 172.21.2.101:0/4130493262 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f2be8001680 con 0x7f2bec0d1880 2021-02-10T11:46:11.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.286 7f2bdffff700 1 -- 172.21.2.101:0/4130493262 >> v1:172.21.2.101:6789/0 conn(0x7f2bec0d1880 legacy=0x7f2bec1b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:11.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.286 7f2bdffff700 1 --2- 172.21.2.101:0/4130493262 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2bd8003a60 0x7f2bd8005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:11.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.286 7f2bdffff700 1 --2- 172.21.2.101:0/4130493262 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bd8006790 0x7f2bd8008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:11.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.286 7f2bdffff700 1 --2- 172.21.2.101:0/4130493262 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2bd8009520 0x7f2bd800ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:11.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.286 7f2bdffff700 1 -- 172.21.2.101:0/4130493262 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd800c2d0 con 0x7f2bd8009520 2021-02-10T11:46:11.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.286 7f2bdffff700 1 -- 172.21.2.101:0/4130493262 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd800c4f0 con 0x7f2bd8003a60 2021-02-10T11:46:11.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.286 7f2bdffff700 1 -- 172.21.2.101:0/4130493262 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd8050720 con 0x7f2bd8006790 2021-02-10T11:46:11.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.286 7f2bf16a4700 1 --2- 172.21.2.101:0/4130493262 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2bd8003a60 0x7f2bd8005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:11.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.286 7f2bf1ea5700 1 --2- 172.21.2.101:0/4130493262 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2bd8009520 0x7f2bd800ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:11.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.286 7f2bf0ea3700 1 --2- 172.21.2.101:0/4130493262 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bd8006790 0x7f2bd8008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:11.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.286 7f2bf16a4700 1 -- 172.21.2.101:0/4130493262 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bd8006790 msgr2=0x7f2bd8008c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:11.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.286 7f2bf16a4700 1 --2- 172.21.2.101:0/4130493262 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bd8006790 0x7f2bd8008c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:11.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.286 7f2bf16a4700 1 -- 172.21.2.101:0/4130493262 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2bd8009520 msgr2=0x7f2bd800ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:11.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.286 7f2bf16a4700 1 --2- 172.21.2.101:0/4130493262 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2bd8009520 0x7f2bd800ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:11.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.286 7f2bf16a4700 1 -- 172.21.2.101:0/4130493262 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd8050980 con 0x7f2bd8003a60 2021-02-10T11:46:11.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.286 7f2bf16a4700 1 --2- 172.21.2.101:0/4130493262 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2bd8003a60 0x7f2bd8005ef0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f2be8007970 tx=0x7f2be8007d10).ready entity=mon.1 client_cookie=ae1f93c9fdc3d630 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:11.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.287 7f2bdffff700 1 -- 172.21.2.101:0/4130493262 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2be8000ce0 con 0x7f2bd8003a60 2021-02-10T11:46:11.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.287 7f2bdffff700 1 -- 172.21.2.101:0/4130493262 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be80070c0 con 0x7f2bd8003a60 2021-02-10T11:46:11.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.287 7f2bdffff700 1 -- 172.21.2.101:0/4130493262 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2be8000ce0 con 0x7f2bd8003a60 2021-02-10T11:46:11.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.287 7f2bf3888700 1 -- 172.21.2.101:0/4130493262 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2bd8003a60 msgr2=0x7f2bd8005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:11.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.287 7f2bf3888700 1 --2- 172.21.2.101:0/4130493262 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2bd8003a60 0x7f2bd8005ef0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f2be8007970 tx=0x7f2be8007d10).stop 2021-02-10T11:46:11.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.287 7f2bf1ea5700 1 -- 172.21.2.101:0/4130493262 reap_dead start 2021-02-10T11:46:11.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.287 7f2bf3888700 1 -- 172.21.2.101:0/4130493262 shutdown_connections 2021-02-10T11:46:11.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.287 7f2bf3888700 1 -- 172.21.2.101:0/4130493262 >> 172.21.2.101:0/4130493262 conn(0x7f2bec0ce430 msgr2=0x7f2bec0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:11.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.287 7f2bf3888700 1 -- 172.21.2.101:0/4130493262 shutdown_connections 2021-02-10T11:46:11.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.288 7f2bf3888700 1 -- 172.21.2.101:0/4130493262 wait complete. 2021-02-10T11:46:11.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.302 7f2bf3888700 1 Processor -- start 2021-02-10T11:46:11.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.302 7f2bf3888700 1 -- start start 2021-02-10T11:46:11.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.303 7f2bf3888700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bec1a05e0 0x7f2bec09a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:11.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.303 7f2bf3888700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2bec09a8b0 0x7f2bec09acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:11.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.303 7f2bf3888700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2bec09b2d0 0x7f2bec0beed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:11.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.303 7f2bf3888700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec099d50 con 0x7f2bec09b2d0 2021-02-10T11:46:11.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.303 7f2bf3888700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec0bf4f0 con 0x7f2bec09a8b0 2021-02-10T11:46:11.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.303 7f2bf3888700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec0bf6e0 con 0x7f2bec1a05e0 2021-02-10T11:46:11.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.303 7f2bf1ea5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2bec09b2d0 0x7f2bec0beed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:11.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.303 7f2bf1ea5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2bec09b2d0 0x7f2bec0beed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33050/0 (socket says 172.21.2.101:33050) 2021-02-10T11:46:11.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.303 7f2bf1ea5700 1 -- 172.21.2.101:0/1650741776 learned_addr learned my addr 172.21.2.101:0/1650741776 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:11.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.303 7f2bf16a4700 1 --2- 172.21.2.101:0/1650741776 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bec1a05e0 0x7f2bec09a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:11.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.303 7f2bf1ea5700 1 -- 172.21.2.101:0/1650741776 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bec1a05e0 msgr2=0x7f2bec09a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:11.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.303 7f2bf1ea5700 1 --2- 172.21.2.101:0/1650741776 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bec1a05e0 0x7f2bec09a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:11.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.303 7f2bf1ea5700 1 -- 172.21.2.101:0/1650741776 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2bec09a8b0 msgr2=0x7f2bec09acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:11.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.303 7f2bf1ea5700 1 --2- 172.21.2.101:0/1650741776 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2bec09a8b0 0x7f2bec09acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:11.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.303 7f2bf1ea5700 1 -- 172.21.2.101:0/1650741776 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bec0bf900 con 0x7f2bec09b2d0 2021-02-10T11:46:11.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.303 7f2bf16a4700 1 --2- 172.21.2.101:0/1650741776 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bec1a05e0 0x7f2bec09a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:46:11.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.304 7f2bf1ea5700 1 --2- 172.21.2.101:0/1650741776 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2bec09b2d0 0x7f2bec0beed0 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 rx=0x7f2be4005fd0 tx=0x7f2be400aca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:11.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.304 7f2bddffb700 1 -- 172.21.2.101:0/1650741776 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2be40066f0 con 0x7f2bec09b2d0 2021-02-10T11:46:11.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.304 7f2bddffb700 1 -- 172.21.2.101:0/1650741776 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be40056b0 con 0x7f2bec09b2d0 2021-02-10T11:46:11.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.304 7f2bddffb700 1 -- 172.21.2.101:0/1650741776 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2be400c4f0 con 0x7f2bec09b2d0 2021-02-10T11:46:11.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.304 7f2bf3888700 1 -- 172.21.2.101:0/1650741776 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bec0bf900 con 0x7f2bec09b2d0 2021-02-10T11:46:11.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.304 7f2bf3888700 1 -- 172.21.2.101:0/1650741776 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bec0bf900 con 0x7f2bec09b2d0 2021-02-10T11:46:11.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.305 7f2bddffb700 1 -- 172.21.2.101:0/1650741776 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2be40011f0 con 0x7f2bec09b2d0 2021-02-10T11:46:11.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.305 7f2bddffb700 1 --2- 172.21.2.101:0/1650741776 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2bcc0186c0 0x7f2bcc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:11.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.305 7f2bddffb700 1 -- 172.21.2.101:0/1650741776 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2be4032750 con 0x7f2bec09b2d0 2021-02-10T11:46:11.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.305 7f2bf16a4700 1 --2- 172.21.2.101:0/1650741776 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2bcc0186c0 0x7f2bcc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:11.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.305 7f2bf16a4700 1 --2- 172.21.2.101:0/1650741776 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2bcc0186c0 0x7f2bcc01abc0 crc :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:11.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.334 7f2bf3888700 1 -- 172.21.2.101:0/1650741776 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bec099d50 con 0x7f2bec09b2d0 2021-02-10T11:46:11.335 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.337 7f2bddffb700 1 -- 172.21.2.101:0/1650741776 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2be4006a90 con 0x7f2bec09b2d0 2021-02-10T11:46:11.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.464 7f2bf3888700 1 -- 172.21.2.101:0/1650741776 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000927842", "target": ["mgr", ""], "snap_name": "snapshot_0000000000667014"}) v1 -- 0x7f2bec099d50 con 0x7f2bcc0186c0 2021-02-10T11:46:11.466 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:46:11.468 7f2cc39e1700 -1 mgr.server reply reply (2) No such file or directory snapshot 'snapshot_0000000000667014' does not exist 2021-02-10T11:46:11.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.468 7f2bddffb700 1 -- 172.21.2.101:0/1650741776 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: -2 snapshot 'snapshot_0000000000667014' does not exist) v1 ==== 59+0+0 (crc 0 0 0) 0x7f2be8001020 con 0x7f2bcc0186c0 2021-02-10T11:46:11.469 INFO:teuthology.orchestra.run.gibba001.stderr:Error ENOENT: snapshot 'snapshot_0000000000667014' does not exist 2021-02-10T11:46:11.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.472 7f2bf3888700 1 -- 172.21.2.101:0/1650741776 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2bcc0186c0 msgr2=0x7f2bcc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:11.470 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.472 7f2bf3888700 1 --2- 172.21.2.101:0/1650741776 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2bcc0186c0 0x7f2bcc01abc0 crc :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:11.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.472 7f2bf3888700 1 -- 172.21.2.101:0/1650741776 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2bec09b2d0 msgr2=0x7f2bec0beed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:11.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.472 7f2bf3888700 1 --2- 172.21.2.101:0/1650741776 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2bec09b2d0 0x7f2bec0beed0 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 rx=0x7f2be4005fd0 tx=0x7f2be400aca0).stop 2021-02-10T11:46:11.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.473 7f2bf3888700 1 -- 172.21.2.101:0/1650741776 shutdown_connections 2021-02-10T11:46:11.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.473 7f2bf3888700 1 --2- 172.21.2.101:0/1650741776 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2bec09b2d0 0x7f2bec0beed0 unknown :-1 s=CLOSED pgs=2925 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.473 7f2bf3888700 1 --2- 172.21.2.101:0/1650741776 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2bec09a8b0 0x7f2bec09acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.473 7f2bf3888700 1 --2- 172.21.2.101:0/1650741776 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2bcc0186c0 0x7f2bcc01abc0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.473 7f2bf3888700 1 --2- 172.21.2.101:0/1650741776 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2bec1a05e0 0x7f2bec09a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.473 7f2bf3888700 1 -- 172.21.2.101:0/1650741776 >> 172.21.2.101:0/1650741776 conn(0x7f2bec0a1770 msgr2=0x7f2bec09e210 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.473 7f2bf3888700 1 -- 172.21.2.101:0/1650741776 shutdown_connections 2021-02-10T11:46:11.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.473 7f2bf3888700 1 -- 172.21.2.101:0/1650741776 wait complete. 2021-02-10T11:46:11.479 DEBUG:teuthology.orchestra.run:got remote process result: 2 2021-02-10T11:46:11.479 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000927842 2021-02-10T11:46:11.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.898 7f8c30bca700 1 Processor -- start 2021-02-10T11:46:11.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.898 7f8c30bca700 1 -- start start 2021-02-10T11:46:11.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.898 7f8c30bca700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8c2c0d1870 0x7f8c2c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:11.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.898 7f8c30bca700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8c2c1b0ae0 0x7f8c2c108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:11.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.898 7f8c30bca700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c2c0fe1c0 con 0x7f8c2c0d1c70 2021-02-10T11:46:11.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.898 7f8c30bca700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c2c109a60 con 0x7f8c2c0d1870 2021-02-10T11:46:11.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.898 7f8c30bca700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c2c109750 con 0x7f8c2c1b0ae0 2021-02-10T11:46:11.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.899 7f8c2a85e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8c2c0d1870 0x7f8c2c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:11.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.899 7f8c2b05f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8c2c0d1c70 0x7f8c2c108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33548/0 (socket says 172.21.2.101:33548) 2021-02-10T11:46:11.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.899 7f8c2b05f700 1 -- 172.21.2.101:0/2396255960 learned_addr learned my addr 172.21.2.101:0/2396255960 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:11.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.899 7f8c2a85e700 1 -- 172.21.2.101:0/2396255960 >> v2:172.21.2.108:3300/0 conn(0x7f8c2c1b0ae0 msgr2=0x7f8c2c108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:11.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.899 7f8c2a85e700 1 --2- 172.21.2.101:0/2396255960 >> v2:172.21.2.108:3300/0 conn(0x7f8c2c1b0ae0 0x7f8c2c108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:11.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.899 7f8c2a85e700 1 -- 172.21.2.101:0/2396255960 >> v1:172.21.2.101:6789/0 conn(0x7f8c2c0d1c70 legacy=0x7f8c2c108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:11.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.899 7f8c2985c700 1 -- 172.21.2.101:0/2396255960 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2936702684 0 0) 0x7f8c24001500 con 0x7f8c2c0d1c70 2021-02-10T11:46:11.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.899 7f8c2a85e700 1 -- 172.21.2.101:0/2396255960 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2c088070 con 0x7f8c2c0d1870 2021-02-10T11:46:11.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.899 7f8c2a85e700 1 --2- 172.21.2.101:0/2396255960 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c2c0d1870 0x7f8c2c1b04c0 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 rx=0x7f8c1c006650 tx=0x7f8c1c009a00).ready entity=mon.0 client_cookie=21de50d30d9ffc08 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:11.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.900 7f8c2985c700 1 -- 172.21.2.101:0/2396255960 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c1c009df0 con 0x7f8c2c0d1870 2021-02-10T11:46:11.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.900 7f8c2985c700 1 -- 172.21.2.101:0/2396255960 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1c0009b0 con 0x7f8c2c0d1870 2021-02-10T11:46:11.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.900 7f8c2985c700 1 -- 172.21.2.101:0/2396255960 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c1c00c2b0 con 0x7f8c2c0d1870 2021-02-10T11:46:11.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.900 7f8c30bca700 1 -- 172.21.2.101:0/2396255960 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c2c0d1870 msgr2=0x7f8c2c1b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:11.914 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.900 7f8c30bca700 1 --2- 172.21.2.101:0/2396255960 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c2c0d1870 0x7f8c2c1b04c0 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 rx=0x7f8c1c006650 tx=0x7f8c1c009a00).stop 2021-02-10T11:46:11.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.901 7f8c30bca700 1 -- 172.21.2.101:0/2396255960 shutdown_connections 2021-02-10T11:46:11.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.901 7f8c30bca700 1 --2- 172.21.2.101:0/2396255960 >> v2:172.21.2.108:3300/0 conn(0x7f8c2c1b0ae0 0x7f8c2c108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:11.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.901 7f8c30bca700 1 --2- 172.21.2.101:0/2396255960 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c2c0d1870 0x7f8c2c1b04c0 unknown :-1 s=CLOSED pgs=2927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:11.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.901 7f8c30bca700 1 -- 172.21.2.101:0/2396255960 >> 172.21.2.101:0/2396255960 conn(0x7f8c2c0ce420 msgr2=0x7f8c2c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:11.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.901 7f8c30bca700 1 -- 172.21.2.101:0/2396255960 shutdown_connections 2021-02-10T11:46:11.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.901 7f8c30bca700 1 -- 172.21.2.101:0/2396255960 wait complete. 2021-02-10T11:46:11.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.915 7f8c30bca700 1 Processor -- start 2021-02-10T11:46:11.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.916 7f8c30bca700 1 -- start start 2021-02-10T11:46:11.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.916 7f8c30bca700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c2c0c3820 0x7f8c2c084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:11.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.916 7f8c30bca700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c2c0a0290 0x7f8c2c0852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:11.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.916 7f8c30bca700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c2c086270 0x7f8c2c086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:11.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.916 7f8c30bca700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c0c0700 con 0x7f8c2c0c3820 2021-02-10T11:46:11.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.916 7f8c30bca700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c0858f0 con 0x7f8c2c086270 2021-02-10T11:46:11.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.916 7f8c30bca700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c085b10 con 0x7f8c2c0a0290 2021-02-10T11:46:11.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.916 7f8c2a85e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c2c0c3820 0x7f8c2c084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:11.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.916 7f8c2a85e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c2c0c3820 0x7f8c2c084cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33058/0 (socket says 172.21.2.101:33058) 2021-02-10T11:46:11.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.916 7f8c2a85e700 1 -- 172.21.2.101:0/2542484760 learned_addr learned my addr 172.21.2.101:0/2542484760 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:11.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.916 7f8c2a85e700 1 -- 172.21.2.101:0/2542484760 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c2c0a0290 msgr2=0x7f8c2c0852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:11.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.916 7f8c2a85e700 1 --2- 172.21.2.101:0/2542484760 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c2c0a0290 0x7f8c2c0852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:11.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.916 7f8c2a85e700 1 -- 172.21.2.101:0/2542484760 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c2c086270 msgr2=0x7f8c2c086670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:11.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.916 7f8c2a85e700 1 --2- 172.21.2.101:0/2542484760 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c2c086270 0x7f8c2c086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:11.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.916 7f8c2a85e700 1 -- 172.21.2.101:0/2542484760 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2c086c90 con 0x7f8c2c0c3820 2021-02-10T11:46:11.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.917 7f8c2a85e700 1 --2- 172.21.2.101:0/2542484760 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c2c0c3820 0x7f8c2c084cb0 secure :-1 s=READY pgs=2928 cs=0 l=1 rev1=1 rx=0x7f8c1c0096a0 tx=0x7f8c1c00b9b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:11.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.917 7f8c1b7fe700 1 -- 172.21.2.101:0/2542484760 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c1c0093a0 con 0x7f8c2c0c3820 2021-02-10T11:46:11.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.917 7f8c1b7fe700 1 -- 172.21.2.101:0/2542484760 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1c0067f0 con 0x7f8c2c0c3820 2021-02-10T11:46:11.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.917 7f8c30bca700 1 -- 172.21.2.101:0/2542484760 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c2c086c90 con 0x7f8c2c0c3820 2021-02-10T11:46:11.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.917 7f8c30bca700 1 -- 172.21.2.101:0/2542484760 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2c1a2f00 con 0x7f8c2c0c3820 2021-02-10T11:46:11.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.917 7f8c1b7fe700 1 -- 172.21.2.101:0/2542484760 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c1c0022b0 con 0x7f8c2c0c3820 2021-02-10T11:46:11.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.918 7f8c1b7fe700 1 -- 172.21.2.101:0/2542484760 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8c1c0052e0 con 0x7f8c2c0c3820 2021-02-10T11:46:11.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.918 7f8c1b7fe700 1 --2- 172.21.2.101:0/2542484760 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c10018730 0x7f8c1001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:11.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.918 7f8c1b7fe700 1 -- 172.21.2.101:0/2542484760 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8c1c032780 con 0x7f8c2c0c3820 2021-02-10T11:46:11.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.918 7f8c2a05d700 1 --2- 172.21.2.101:0/2542484760 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c10018730 0x7f8c1001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:11.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.918 7f8c2a05d700 1 --2- 172.21.2.101:0/2542484760 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c10018730 0x7f8c1001ac10 crc :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:11.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.948 7f8c30bca700 1 -- 172.21.2.101:0/2542484760 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c2c082a70 con 0x7f8c2c0c3820 2021-02-10T11:46:11.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:11.951 7f8c1b7fe700 1 -- 172.21.2.101:0/2542484760 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8c1c0093a0 con 0x7f8c2c0c3820 2021-02-10T11:46:12.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:12.076 7f8c30bca700 1 -- 172.21.2.101:0/2542484760 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000927842", "target": ["mgr", ""]}) v1 -- 0x7f8c2c1a2f00 con 0x7f8c10018730 2021-02-10T11:46:12.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:12.085 7f8c1b7fe700 1 -- 172.21.2.101:0/2542484760 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f8c200009b0 con 0x7f8c10018730 2021-02-10T11:46:12.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:12.093 7f8c30bca700 1 -- 172.21.2.101:0/2542484760 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c10018730 msgr2=0x7f8c1001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:12.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:12.093 7f8c30bca700 1 --2- 172.21.2.101:0/2542484760 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c10018730 0x7f8c1001ac10 crc :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:12.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:12.093 7f8c30bca700 1 -- 172.21.2.101:0/2542484760 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c2c0c3820 msgr2=0x7f8c2c084cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:12.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:12.093 7f8c30bca700 1 --2- 172.21.2.101:0/2542484760 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c2c0c3820 0x7f8c2c084cb0 secure :-1 s=READY pgs=2928 cs=0 l=1 rev1=1 rx=0x7f8c1c0096a0 tx=0x7f8c1c00b9b0).stop 2021-02-10T11:46:12.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:12.093 7f8c30bca700 1 -- 172.21.2.101:0/2542484760 shutdown_connections 2021-02-10T11:46:12.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:12.093 7f8c30bca700 1 --2- 172.21.2.101:0/2542484760 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8c10018730 0x7f8c1001ac10 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:12.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:12.093 7f8c30bca700 1 --2- 172.21.2.101:0/2542484760 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8c2c0a0290 0x7f8c2c0852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:12.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:12.093 7f8c30bca700 1 --2- 172.21.2.101:0/2542484760 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8c2c086270 0x7f8c2c086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:12.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:12.093 7f8c30bca700 1 --2- 172.21.2.101:0/2542484760 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8c2c0c3820 0x7f8c2c084cb0 unknown :-1 s=CLOSED pgs=2928 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:12.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:12.093 7f8c30bca700 1 -- 172.21.2.101:0/2542484760 >> 172.21.2.101:0/2542484760 conn(0x7f8c2c0872c0 msgr2=0x7f8c2c095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:12.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:12.093 7f8c30bca700 1 -- 172.21.2.101:0/2542484760 shutdown_connections 2021-02-10T11:46:12.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:12.093 7f8c30bca700 1 -- 172.21.2.101:0/2542484760 wait complete. 2021-02-10T11:46:12.099 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo getfattr --only-values -n ceph.dir.entries /home/ubuntu/cephtest/mnt.0/./volumes/_deleting 2021-02-10T11:46:12.448 INFO:teuthology.orchestra.run.gibba001.stderr:getfattr: Removing leading '/' from absolute path names 2021-02-10T11:46:12.449 INFO:teuthology.orchestra.run.gibba001.stdout:0 2021-02-10T11:46:12.449 DEBUG:tasks.cephfs.mount:Directory ./volumes/_deleting seen empty from 0 after 0s 2021-02-10T11:46:12.449 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:46:12.449 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:46:12.795 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:46:13.139 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:46:13.139 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:13.139 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:46:13.139 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:46:13.476 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:46:13.476 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:46:13.476 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:46:13.477 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:46:13.477 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:46:13.839 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[169637]: fuse finished with error 0 and tester_r 0 2021-02-10T11:46:13.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:13.840 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:46:13.840 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:46:14.175 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:46:14.175 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:46:14.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:46:14.176 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:46:14.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:14.511 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:46:14.512 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:46:14.846 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:46:14.847 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:46:14.847 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:46:14.847 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:46:14.847 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:46:15.198 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[170281]: fuse finished with error 0 and tester_r 0 2021-02-10T11:46:15.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:15.200 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:46:15.200 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:46:15.535 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:46:15.536 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:46:15.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:46:15.536 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:46:15.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:15.872 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:46:15.872 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:46:16.209 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:46:16.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:46:16.210 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:46:16.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:46:16.210 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:46:16.545 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:16.546 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:46:16.546 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:46:16.880 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:46:16.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:46:16.880 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:46:16.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:46:16.881 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:46:17.215 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-02-10T11:46:17.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.631 7eff87b6b700 1 Processor -- start 2021-02-10T11:46:17.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.632 7eff87b6b700 1 -- start start 2021-02-10T11:46:17.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.632 7eff87b6b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7eff800d0340 0x7eff800ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:17.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.632 7eff87b6b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7eff800d1c70 0x7eff800efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:17.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.632 7eff87b6b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff800f0b10 con 0x7eff800d1870 2021-02-10T11:46:17.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.632 7eff87b6b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff800e5ed0 con 0x7eff800d0340 2021-02-10T11:46:17.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.632 7eff87b6b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff801b0be0 con 0x7eff800d1c70 2021-02-10T11:46:17.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.632 7eff85186700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7eff800d0340 0x7eff800ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:17.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.632 7eff85987700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7eff800d1870 0x7eff801b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33558/0 (socket says 172.21.2.101:33558) 2021-02-10T11:46:17.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.632 7eff85987700 1 -- 172.21.2.101:0/3472145955 learned_addr learned my addr 172.21.2.101:0/3472145955 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:17.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.633 7eff85186700 1 -- 172.21.2.101:0/3472145955 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7eff800d1c70 msgr2=0x7eff800efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:17.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.633 7eff85186700 1 --2- 172.21.2.101:0/3472145955 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7eff800d1c70 0x7eff800efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:17.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.633 7eff85186700 1 -- 172.21.2.101:0/3472145955 >> v1:172.21.2.101:6789/0 conn(0x7eff800d1870 legacy=0x7eff801b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:17.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.633 7eff85186700 1 -- 172.21.2.101:0/3472145955 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff800f20a0 con 0x7eff800d0340 2021-02-10T11:46:17.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.633 7eff85186700 1 --2- 172.21.2.101:0/3472145955 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff800d0340 0x7eff800ef640 secure :-1 s=READY pgs=2930 cs=0 l=1 rev1=1 rx=0x7eff7c006650 tx=0x7eff7c009a80).ready entity=mon.0 client_cookie=c52253f00ce36e13 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:17.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.633 7eff84985700 1 -- 172.21.2.101:0/3472145955 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7eff7c00c0c0 con 0x7eff800d0340 2021-02-10T11:46:17.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.633 7eff84985700 1 -- 172.21.2.101:0/3472145955 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7c00c340 con 0x7eff800d0340 2021-02-10T11:46:17.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.633 7eff84985700 1 -- 172.21.2.101:0/3472145955 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7eff7c00c860 con 0x7eff800d0340 2021-02-10T11:46:17.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.633 7eff87b6b700 1 -- 172.21.2.101:0/3472145955 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff800d0340 msgr2=0x7eff800ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:17.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.633 7eff87b6b700 1 --2- 172.21.2.101:0/3472145955 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff800d0340 0x7eff800ef640 secure :-1 s=READY pgs=2930 cs=0 l=1 rev1=1 rx=0x7eff7c006650 tx=0x7eff7c009a80).stop 2021-02-10T11:46:17.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.635 7eff87b6b700 1 -- 172.21.2.101:0/3472145955 shutdown_connections 2021-02-10T11:46:17.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.635 7eff87b6b700 1 --2- 172.21.2.101:0/3472145955 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7eff800d1c70 0x7eff800efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:17.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.635 7eff87b6b700 1 --2- 172.21.2.101:0/3472145955 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff800d0340 0x7eff800ef640 unknown :-1 s=CLOSED pgs=2930 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:17.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.635 7eff87b6b700 1 -- 172.21.2.101:0/3472145955 >> 172.21.2.101:0/3472145955 conn(0x7eff800ce450 msgr2=0x7eff800d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:17.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.635 7eff87b6b700 1 -- 172.21.2.101:0/3472145955 shutdown_connections 2021-02-10T11:46:17.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.635 7eff87b6b700 1 -- 172.21.2.101:0/3472145955 wait complete. 2021-02-10T11:46:17.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.649 7eff87b6b700 1 Processor -- start 2021-02-10T11:46:17.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.649 7eff87b6b700 1 -- start start 2021-02-10T11:46:17.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.649 7eff87b6b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7eff800a2fb0 0x7eff800a33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:17.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.649 7eff87b6b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7eff800a39d0 0x7eff800a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:17.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.649 7eff87b6b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff800a3dd0 0x7eff800a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:17.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.649 7eff87b6b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff800a2620 con 0x7eff800a3dd0 2021-02-10T11:46:17.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.649 7eff87b6b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff800a2840 con 0x7eff800a39d0 2021-02-10T11:46:17.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.649 7eff87b6b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff800a2b80 con 0x7eff800a2fb0 2021-02-10T11:46:17.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.649 7eff86188700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff800a3dd0 0x7eff800a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:17.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.649 7eff86188700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff800a3dd0 0x7eff800a4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33074/0 (socket says 172.21.2.101:33074) 2021-02-10T11:46:17.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.649 7eff86188700 1 -- 172.21.2.101:0/1555417680 learned_addr learned my addr 172.21.2.101:0/1555417680 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:17.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.650 7eff85987700 1 --2- 172.21.2.101:0/1555417680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7eff800a2fb0 0x7eff800a33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:17.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.650 7eff85186700 1 --2- 172.21.2.101:0/1555417680 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7eff800a39d0 0x7eff800a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:17.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.650 7eff86188700 1 -- 172.21.2.101:0/1555417680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7eff800a2fb0 msgr2=0x7eff800a33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:17.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.650 7eff86188700 1 --2- 172.21.2.101:0/1555417680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7eff800a2fb0 0x7eff800a33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:17.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.650 7eff86188700 1 -- 172.21.2.101:0/1555417680 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7eff800a39d0 msgr2=0x7eff800a4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:17.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.650 7eff86188700 1 --2- 172.21.2.101:0/1555417680 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7eff800a39d0 0x7eff800a4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:17.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.650 7eff86188700 1 -- 172.21.2.101:0/1555417680 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff800a54a0 con 0x7eff800a3dd0 2021-02-10T11:46:17.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.650 7eff85186700 1 --2- 172.21.2.101:0/1555417680 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7eff800a39d0 0x7eff800a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:46:17.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.650 7eff86188700 1 --2- 172.21.2.101:0/1555417680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff800a3dd0 0x7eff800a4e80 secure :-1 s=READY pgs=2931 cs=0 l=1 rev1=1 rx=0x7eff7800b8c0 tx=0x7eff7800bc60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:17.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.650 7eff767fc700 1 -- 172.21.2.101:0/1555417680 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7eff78005870 con 0x7eff800a3dd0 2021-02-10T11:46:17.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.650 7eff767fc700 1 -- 172.21.2.101:0/1555417680 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff78005b30 con 0x7eff800a3dd0 2021-02-10T11:46:17.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.650 7eff767fc700 1 -- 172.21.2.101:0/1555417680 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7eff78006030 con 0x7eff800a3dd0 2021-02-10T11:46:17.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.650 7eff87b6b700 1 -- 172.21.2.101:0/1555417680 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff800a54a0 con 0x7eff800a3dd0 2021-02-10T11:46:17.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.650 7eff87b6b700 1 -- 172.21.2.101:0/1555417680 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff800a54a0 con 0x7eff800a3dd0 2021-02-10T11:46:17.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.651 7eff767fc700 1 -- 172.21.2.101:0/1555417680 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7eff78005d10 con 0x7eff800a3dd0 2021-02-10T11:46:17.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.651 7eff767fc700 1 --2- 172.21.2.101:0/1555417680 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7eff680186c0 0x7eff6801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:17.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.651 7eff767fc700 1 -- 172.21.2.101:0/1555417680 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5617+0+0 (secure 0 0 0) 0x7eff78032950 con 0x7eff800a3dd0 2021-02-10T11:46:17.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.651 7eff85987700 1 --2- 172.21.2.101:0/1555417680 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7eff680186c0 0x7eff6801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:17.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.651 7eff85987700 1 --2- 172.21.2.101:0/1555417680 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7eff680186c0 0x7eff6801abc0 crc :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:17.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.682 7eff87b6b700 1 -- 172.21.2.101:0/1555417680 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff800a4330 con 0x7eff800a3dd0 2021-02-10T11:46:17.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.685 7eff767fc700 1 -- 172.21.2.101:0/1555417680 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7eff78005870 con 0x7eff800a3dd0 2021-02-10T11:46:17.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.833 7eff87b6b700 1 -- 172.21.2.101:0/1555417680 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7eff800a4330 con 0x7eff800a3dd0 2021-02-10T11:46:17.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.837 7eff767fc700 1 -- 172.21.2.101:0/1555417680 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v41) v1 ==== 104+0+0 (secure 0 0 0) 0x7eff78006f20 con 0x7eff800a3dd0 2021-02-10T11:46:17.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.842 7eff87b6b700 1 -- 172.21.2.101:0/1555417680 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7eff680186c0 msgr2=0x7eff6801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:17.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.842 7eff87b6b700 1 --2- 172.21.2.101:0/1555417680 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7eff680186c0 0x7eff6801abc0 crc :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:17.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.842 7eff87b6b700 1 -- 172.21.2.101:0/1555417680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff800a3dd0 msgr2=0x7eff800a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:17.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.842 7eff87b6b700 1 --2- 172.21.2.101:0/1555417680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff800a3dd0 0x7eff800a4e80 secure :-1 s=READY pgs=2931 cs=0 l=1 rev1=1 rx=0x7eff7800b8c0 tx=0x7eff7800bc60).stop 2021-02-10T11:46:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.843 7eff87b6b700 1 -- 172.21.2.101:0/1555417680 shutdown_connections 2021-02-10T11:46:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.843 7eff87b6b700 1 --2- 172.21.2.101:0/1555417680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7eff800a3dd0 0x7eff800a4e80 unknown :-1 s=CLOSED pgs=2931 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.843 7eff87b6b700 1 --2- 172.21.2.101:0/1555417680 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7eff800a39d0 0x7eff800a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.843 7eff87b6b700 1 --2- 172.21.2.101:0/1555417680 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7eff680186c0 0x7eff6801abc0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.843 7eff87b6b700 1 --2- 172.21.2.101:0/1555417680 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7eff800a2fb0 0x7eff800a33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:17.841 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.843 7eff87b6b700 1 -- 172.21.2.101:0/1555417680 >> 172.21.2.101:0/1555417680 conn(0x7eff8008f030 msgr2=0x7eff800b5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:17.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.843 7eff87b6b700 1 -- 172.21.2.101:0/1555417680 shutdown_connections 2021-02-10T11:46:17.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:17.843 7eff87b6b700 1 -- 172.21.2.101:0/1555417680 wait complete. 2021-02-10T11:46:17.849 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_snapshot_rm' 2021-02-10T11:46:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.263 7fb69668d700 1 Processor -- start 2021-02-10T11:46:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.263 7fb69668d700 1 -- start start 2021-02-10T11:46:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.264 7fb69668d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb6900d1870 0x7fb6901b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.264 7fb69668d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6901b0a10 0x7fb6900cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.264 7fb69668d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6900f0b10 con 0x7fb6900d1c70 2021-02-10T11:46:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.264 7fb69668d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6900e5ed0 con 0x7fb6900d1870 2021-02-10T11:46:18.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.264 7fb69668d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900f0650 con 0x7fb6901b0a10 2021-02-10T11:46:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.264 7fb694caa700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb6900d1c70 0x7fb6900cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33574/0 (socket says 172.21.2.101:33574) 2021-02-10T11:46:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.264 7fb694caa700 1 -- 172.21.2.101:0/3246585116 learned_addr learned my addr 172.21.2.101:0/3246585116 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.264 7fb68ffff700 1 --2- 172.21.2.101:0/3246585116 >> v2:172.21.2.101:3300/0 conn(0x7fb6900d1870 0x7fb6901b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.264 7fb68effd700 1 -- 172.21.2.101:0/3246585116 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784313619 0 0) 0x7fb688001500 con 0x7fb6900d1c70 2021-02-10T11:46:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.264 7fb68effd700 1 -- 172.21.2.101:0/3246585116 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb678002fe0 con 0x7fb6900d1c70 2021-02-10T11:46:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.264 7fb68ffff700 1 -- 172.21.2.101:0/3246585116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6901b0a10 msgr2=0x7fb6900cf330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:46:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.264 7fb68ffff700 1 --2- 172.21.2.101:0/3246585116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6901b0a10 0x7fb6900cf330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.264 7fb68ffff700 1 -- 172.21.2.101:0/3246585116 >> v1:172.21.2.101:6789/0 conn(0x7fb6900d1c70 legacy=0x7fb6900cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.264 7fb68ffff700 1 -- 172.21.2.101:0/3246585116 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6900f0870 con 0x7fb6900d1870 2021-02-10T11:46:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.265 7fb68ffff700 1 --2- 172.21.2.101:0/3246585116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6900d1870 0x7fb6901b03f0 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 rx=0x7fb680006650 tx=0x7fb680009a80).ready entity=mon.0 client_cookie=e9c1223ae65bd7f7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:18.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.265 7fb68effd700 1 -- 172.21.2.101:0/3246585116 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb68000c0c0 con 0x7fb6900d1870 2021-02-10T11:46:18.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.265 7fb68effd700 1 -- 172.21.2.101:0/3246585116 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68000c340 con 0x7fb6900d1870 2021-02-10T11:46:18.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.265 7fb68effd700 1 -- 172.21.2.101:0/3246585116 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb68000c860 con 0x7fb6900d1870 2021-02-10T11:46:18.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.265 7fb69668d700 1 -- 172.21.2.101:0/3246585116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6900d1870 msgr2=0x7fb6901b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:18.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.265 7fb69668d700 1 --2- 172.21.2.101:0/3246585116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6900d1870 0x7fb6901b03f0 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 rx=0x7fb680006650 tx=0x7fb680009a80).stop 2021-02-10T11:46:18.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.265 7fb69668d700 1 -- 172.21.2.101:0/3246585116 shutdown_connections 2021-02-10T11:46:18.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.265 7fb69668d700 1 --2- 172.21.2.101:0/3246585116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6901b0a10 0x7fb6900cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:18.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.265 7fb69668d700 1 --2- 172.21.2.101:0/3246585116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6900d1870 0x7fb6901b03f0 unknown :-1 s=CLOSED pgs=2933 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:18.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.265 7fb69668d700 1 -- 172.21.2.101:0/3246585116 >> 172.21.2.101:0/3246585116 conn(0x7fb6900ce450 msgr2=0x7fb6900d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:18.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.265 7fb69668d700 1 -- 172.21.2.101:0/3246585116 shutdown_connections 2021-02-10T11:46:18.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.265 7fb69668d700 1 -- 172.21.2.101:0/3246585116 wait complete. 2021-02-10T11:46:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.280 7fb69668d700 1 Processor -- start 2021-02-10T11:46:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.280 7fb69668d700 1 -- start start 2021-02-10T11:46:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.280 7fb69668d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6900aa750 0x7fb6900aab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.280 7fb69668d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6901acfc0 0x7fb6901ad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.280 7fb69668d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb6901ad9e0 0x7fb6901adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.280 7fb69668d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6901ae400 con 0x7fb6900aa750 2021-02-10T11:46:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.280 7fb69668d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6901ac630 con 0x7fb6901ad9e0 2021-02-10T11:46:18.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.280 7fb69668d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6901ac850 con 0x7fb6901acfc0 2021-02-10T11:46:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.280 7fb68ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6900aa750 0x7fb6900aab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.280 7fb68ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6900aa750 0x7fb6900aab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33088/0 (socket says 172.21.2.101:33088) 2021-02-10T11:46:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.280 7fb68ffff700 1 -- 172.21.2.101:0/209527146 learned_addr learned my addr 172.21.2.101:0/209527146 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.281 7fb68ffff700 1 -- 172.21.2.101:0/209527146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6901acfc0 msgr2=0x7fb6901ad3c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.281 7fb68ffff700 1 --2- 172.21.2.101:0/209527146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6901acfc0 0x7fb6901ad3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.281 7fb68ffff700 1 -- 172.21.2.101:0/209527146 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb6901ad9e0 msgr2=0x7fb6901adde0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.281 7fb68ffff700 1 --2- 172.21.2.101:0/209527146 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb6901ad9e0 0x7fb6901adde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.281 7fb68ffff700 1 -- 172.21.2.101:0/209527146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6900ecb10 con 0x7fb6900aa750 2021-02-10T11:46:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.281 7fb68ffff700 1 --2- 172.21.2.101:0/209527146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6900aa750 0x7fb6900aab50 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 rx=0x7fb680009630 tx=0x7fb68000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.281 7fb68cff9700 1 -- 172.21.2.101:0/209527146 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb6800093a0 con 0x7fb6900aa750 2021-02-10T11:46:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.281 7fb69668d700 1 -- 172.21.2.101:0/209527146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6900ecb10 con 0x7fb6900aa750 2021-02-10T11:46:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.281 7fb69668d700 1 -- 172.21.2.101:0/209527146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6900ed0a0 con 0x7fb6900aa750 2021-02-10T11:46:18.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.281 7fb68cff9700 1 -- 172.21.2.101:0/209527146 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb680000940 con 0x7fb6900aa750 2021-02-10T11:46:18.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.281 7fb68cff9700 1 -- 172.21.2.101:0/209527146 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb68000bd70 con 0x7fb6900aa750 2021-02-10T11:46:18.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.282 7fb68cff9700 1 -- 172.21.2.101:0/209527146 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb6800054f0 con 0x7fb6900aa750 2021-02-10T11:46:18.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.282 7fb68cff9700 1 --2- 172.21.2.101:0/209527146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb670018730 0x7fb67001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:18.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.282 7fb68cff9700 1 -- 172.21.2.101:0/209527146 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5617+0+0 (secure 0 0 0) 0x7fb6800327e0 con 0x7fb6900aa750 2021-02-10T11:46:18.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.282 7fb68f7fe700 1 --2- 172.21.2.101:0/209527146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb670018730 0x7fb67001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:18.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.282 7fb68f7fe700 1 --2- 172.21.2.101:0/209527146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb670018730 0x7fb67001ac10 crc :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:18.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.314 7fb69668d700 1 -- 172.21.2.101:0/209527146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6900ebc00 con 0x7fb6900aa750 2021-02-10T11:46:18.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.316 7fb68cff9700 1 -- 172.21.2.101:0/209527146 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb680000940 con 0x7fb6900aa750 2021-02-10T11:46:18.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.446 7fb69668d700 1 -- 172.21.2.101:0/209527146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_snapshot_rm"]} v 0) v1 -- 0x7fb6900f0b10 con 0x7fb6900aa750 2021-02-10T11:46:18.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:18.995 7fb68cff9700 1 -- 172.21.2.101:0/209527146 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_nonexistent_subvolume_snapshot_rm"]}]=0 v788) v1 ==== 155+0+0 (secure 0 0 0) 0x7fb680000940 con 0x7fb6900aa750 2021-02-10T11:46:19.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.016 7fb69668d700 1 -- 172.21.2.101:0/209527146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb670018730 msgr2=0x7fb67001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:19.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.016 7fb69668d700 1 --2- 172.21.2.101:0/209527146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb670018730 0x7fb67001ac10 crc :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:19.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.016 7fb69668d700 1 -- 172.21.2.101:0/209527146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6900aa750 msgr2=0x7fb6900aab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:19.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.016 7fb69668d700 1 --2- 172.21.2.101:0/209527146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6900aa750 0x7fb6900aab50 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 rx=0x7fb680009630 tx=0x7fb68000b980).stop 2021-02-10T11:46:19.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.017 7fb69668d700 1 -- 172.21.2.101:0/209527146 shutdown_connections 2021-02-10T11:46:19.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.017 7fb69668d700 1 --2- 172.21.2.101:0/209527146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb670018730 0x7fb67001ac10 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:19.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.017 7fb69668d700 1 --2- 172.21.2.101:0/209527146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6901acfc0 0x7fb6901ad3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:19.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.017 7fb69668d700 1 --2- 172.21.2.101:0/209527146 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb6901ad9e0 0x7fb6901adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:19.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.017 7fb69668d700 1 --2- 172.21.2.101:0/209527146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6900aa750 0x7fb6900aab50 unknown :-1 s=CLOSED pgs=2934 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:19.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.017 7fb69668d700 1 -- 172.21.2.101:0/209527146 >> 172.21.2.101:0/209527146 conn(0x7fb69019c560 msgr2=0x7fb6900eedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:19.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.017 7fb69668d700 1 -- 172.21.2.101:0/209527146 shutdown_connections 2021-02-10T11:46:19.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.017 7fb69668d700 1 -- 172.21.2.101:0/209527146 wait complete. 2021-02-10T11:46:19.023 INFO:tasks.cephfs_test_runner:test_nonexistent_subvolume_snapshot_rm (tasks.cephfs.test_volumes.TestVolumes) ... ok 2021-02-10T11:46:19.023 INFO:tasks.cephfs_test_runner:Starting test: test_recover_auth_metadata_during_authorize (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:46:19.024 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_recover_auth_metadata_during_authorize' 2021-02-10T11:46:19.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.439 7f3d09747700 1 Processor -- start 2021-02-10T11:46:19.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.439 7f3d09747700 1 -- start start 2021-02-10T11:46:19.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.440 7f3d09747700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d040d1870 0x7f3d041b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:19.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.440 7f3d09747700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3d040f0650 0x7f3d040cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:19.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.440 7f3d09747700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d040f0b10 con 0x7f3d041b0a10 2021-02-10T11:46:19.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.440 7f3d09747700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d040fe230 con 0x7f3d040f0650 2021-02-10T11:46:19.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.440 7f3d09747700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d040e5ed0 con 0x7f3d040d1870 2021-02-10T11:46:19.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.440 7f3d03549700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d040d1870 0x7f3d041b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:19.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.440 7f3d02d48700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f3d041b0a10 0x7f3d040cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:55228/0 (socket says 172.21.2.101:55228) 2021-02-10T11:46:19.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.440 7f3d03d4a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3d040f0650 0x7f3d040cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:19.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.440 7f3d02d48700 1 -- 172.21.2.101:0/919495086 learned_addr learned my addr 172.21.2.101:0/919495086 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:19.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.440 7f3d03549700 1 -- 172.21.2.101:0/919495086 >> v2:172.21.2.108:3300/0 conn(0x7f3d040f0650 msgr2=0x7f3d040cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:19.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.441 7f3d03549700 1 --2- 172.21.2.101:0/919495086 >> v2:172.21.2.108:3300/0 conn(0x7f3d040f0650 0x7f3d040cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:19.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.441 7f3d03549700 1 -- 172.21.2.101:0/919495086 >> v1:172.21.2.108:6789/0 conn(0x7f3d041b0a10 legacy=0x7f3d040cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:19.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.441 7f3d03549700 1 -- 172.21.2.101:0/919495086 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d040f20a0 con 0x7f3d040d1870 2021-02-10T11:46:19.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.441 7f3d03d4a700 1 --2- 172.21.2.101:0/919495086 >> v2:172.21.2.108:3300/0 conn(0x7f3d040f0650 0x7f3d040cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:46:19.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.441 7f3d03549700 1 --2- 172.21.2.101:0/919495086 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d040d1870 0x7f3d041b03f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f3cec006650 tx=0x7f3cec009a80).ready entity=mon.2 client_cookie=d3c5670f3ad022b2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:19.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.441 7f3d02547700 1 -- 172.21.2.101:0/919495086 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3cec0009b0 con 0x7f3d040d1870 2021-02-10T11:46:19.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.441 7f3d02547700 1 -- 172.21.2.101:0/919495086 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cec00c0c0 con 0x7f3d040d1870 2021-02-10T11:46:19.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.441 7f3d02547700 1 -- 172.21.2.101:0/919495086 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3cec00c5e0 con 0x7f3d040d1870 2021-02-10T11:46:19.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.441 7f3d09747700 1 -- 172.21.2.101:0/919495086 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d040d1870 msgr2=0x7f3d041b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:19.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.441 7f3d09747700 1 --2- 172.21.2.101:0/919495086 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d040d1870 0x7f3d041b03f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f3cec006650 tx=0x7f3cec009a80).stop 2021-02-10T11:46:19.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.442 7f3d09747700 1 -- 172.21.2.101:0/919495086 shutdown_connections 2021-02-10T11:46:19.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.442 7f3d09747700 1 --2- 172.21.2.101:0/919495086 >> v2:172.21.2.108:3300/0 conn(0x7f3d040f0650 0x7f3d040cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:19.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.442 7f3d09747700 1 --2- 172.21.2.101:0/919495086 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d040d1870 0x7f3d041b03f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:19.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.442 7f3d09747700 1 -- 172.21.2.101:0/919495086 >> 172.21.2.101:0/919495086 conn(0x7f3d040ce450 msgr2=0x7f3d040d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:19.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.442 7f3d09747700 1 -- 172.21.2.101:0/919495086 shutdown_connections 2021-02-10T11:46:19.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.442 7f3d09747700 1 -- 172.21.2.101:0/919495086 wait complete. 2021-02-10T11:46:19.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.456 7f3d09747700 1 Processor -- start 2021-02-10T11:46:19.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.457 7f3d09747700 1 -- start start 2021-02-10T11:46:19.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.457 7f3d09747700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d040ada90 0x7f3d040ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:19.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.457 7f3d09747700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3d04099d20 0x7f3d0409a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:19.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.457 7f3d09747700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3d0409a740 0x7f3d04097bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:19.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.457 7f3d09747700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d040997a0 con 0x7f3d0409a740 2021-02-10T11:46:19.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.457 7f3d09747700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0418f0c0 con 0x7f3d04099d20 2021-02-10T11:46:19.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.457 7f3d09747700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0409ab40 con 0x7f3d040ada90 2021-02-10T11:46:19.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.457 7f3d03d4a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3d0409a740 0x7f3d04097bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:19.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.457 7f3d03d4a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3d0409a740 0x7f3d04097bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33102/0 (socket says 172.21.2.101:33102) 2021-02-10T11:46:19.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.457 7f3d03d4a700 1 -- 172.21.2.101:0/1913215946 learned_addr learned my addr 172.21.2.101:0/1913215946 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:19.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.457 7f3d03549700 1 --2- 172.21.2.101:0/1913215946 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d040ada90 0x7f3d040ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:19.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.458 7f3d02d48700 1 --2- 172.21.2.101:0/1913215946 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3d04099d20 0x7f3d0409a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:19.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.458 7f3d03d4a700 1 -- 172.21.2.101:0/1913215946 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d040ada90 msgr2=0x7f3d040ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:19.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.458 7f3d03d4a700 1 --2- 172.21.2.101:0/1913215946 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d040ada90 0x7f3d040ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:19.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.458 7f3d03d4a700 1 -- 172.21.2.101:0/1913215946 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3d04099d20 msgr2=0x7f3d0409a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:19.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.458 7f3d03d4a700 1 --2- 172.21.2.101:0/1913215946 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3d04099d20 0x7f3d0409a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:19.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.458 7f3d03d4a700 1 -- 172.21.2.101:0/1913215946 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0409ad60 con 0x7f3d0409a740 2021-02-10T11:46:19.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.458 7f3d02d48700 1 --2- 172.21.2.101:0/1913215946 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3d04099d20 0x7f3d0409a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:46:19.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.458 7f3d03d4a700 1 --2- 172.21.2.101:0/1913215946 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3d0409a740 0x7f3d04097bc0 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 rx=0x7f3cfc00b910 tx=0x7f3cfc00bcb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:19.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.458 7f3cfbfff700 1 -- 172.21.2.101:0/1913215946 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3cfc006aa0 con 0x7f3d0409a740 2021-02-10T11:46:19.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.458 7f3cfbfff700 1 -- 172.21.2.101:0/1913215946 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cfc003030 con 0x7f3d0409a740 2021-02-10T11:46:19.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.458 7f3cfbfff700 1 -- 172.21.2.101:0/1913215946 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3cfc0013e0 con 0x7f3d0409a740 2021-02-10T11:46:19.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.458 7f3d09747700 1 -- 172.21.2.101:0/1913215946 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0409ad60 con 0x7f3d0409a740 2021-02-10T11:46:19.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.458 7f3d09747700 1 -- 172.21.2.101:0/1913215946 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d040984d0 con 0x7f3d0409a740 2021-02-10T11:46:19.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.459 7f3cfbfff700 1 -- 172.21.2.101:0/1913215946 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3cfc0056b0 con 0x7f3d0409a740 2021-02-10T11:46:19.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.459 7f3cfbfff700 1 --2- 172.21.2.101:0/1913215946 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3ce80186c0 0x7f3ce801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:19.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.459 7f3cfbfff700 1 -- 172.21.2.101:0/1913215946 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5617+0+0 (secure 0 0 0) 0x7f3cfc032910 con 0x7f3d0409a740 2021-02-10T11:46:19.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.459 7f3d03549700 1 --2- 172.21.2.101:0/1913215946 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3ce80186c0 0x7f3ce801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:19.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.459 7f3d03549700 1 --2- 172.21.2.101:0/1913215946 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3ce80186c0 0x7f3ce801abc0 crc :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:19.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.490 7f3d09747700 1 -- 172.21.2.101:0/1913215946 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d040ebc00 con 0x7f3d0409a740 2021-02-10T11:46:19.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.492 7f3cfbfff700 1 -- 172.21.2.101:0/1913215946 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3cfc000f40 con 0x7f3d0409a740 2021-02-10T11:46:19.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.623 7f3d09747700 1 -- 172.21.2.101:0/1913215946 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_recover_auth_metadata_during_authorize"]} v 0) v1 -- 0x7f3d040f0b10 con 0x7f3d0409a740 2021-02-10T11:46:19.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:19.997 7f3cfbfff700 1 -- 172.21.2.101:0/1913215946 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_recover_auth_metadata_during_authorize"]}]=0 v789) v1 ==== 163+0+0 (secure 0 0 0) 0x7f3cfc01f6b0 con 0x7f3d0409a740 2021-02-10T11:46:20.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.041 7f3d09747700 1 -- 172.21.2.101:0/1913215946 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3ce80186c0 msgr2=0x7f3ce801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:20.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.041 7f3d09747700 1 --2- 172.21.2.101:0/1913215946 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3ce80186c0 0x7f3ce801abc0 crc :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:20.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.041 7f3d09747700 1 -- 172.21.2.101:0/1913215946 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3d0409a740 msgr2=0x7f3d04097bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:20.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.041 7f3d09747700 1 --2- 172.21.2.101:0/1913215946 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3d0409a740 0x7f3d04097bc0 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 rx=0x7f3cfc00b910 tx=0x7f3cfc00bcb0).stop 2021-02-10T11:46:20.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.042 7f3d09747700 1 -- 172.21.2.101:0/1913215946 shutdown_connections 2021-02-10T11:46:20.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.042 7f3d09747700 1 --2- 172.21.2.101:0/1913215946 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3d0409a740 0x7f3d04097bc0 unknown :-1 s=CLOSED pgs=2935 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:20.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.042 7f3d09747700 1 --2- 172.21.2.101:0/1913215946 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3d04099d20 0x7f3d0409a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:20.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.042 7f3d09747700 1 --2- 172.21.2.101:0/1913215946 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3ce80186c0 0x7f3ce801abc0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:20.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.042 7f3d09747700 1 --2- 172.21.2.101:0/1913215946 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3d040ada90 0x7f3d040ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:20.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.042 7f3d09747700 1 -- 172.21.2.101:0/1913215946 >> 172.21.2.101:0/1913215946 conn(0x7f3d040f0550 msgr2=0x7f3d04191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:20.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.042 7f3d09747700 1 -- 172.21.2.101:0/1913215946 shutdown_connections 2021-02-10T11:46:20.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.042 7f3d09747700 1 -- 172.21.2.101:0/1913215946 wait complete. 2021-02-10T11:46:20.048 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:46:20.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.465 7f1f7be64700 1 Processor -- start 2021-02-10T11:46:20.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.465 7f1f7be64700 1 -- start start 2021-02-10T11:46:20.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.465 7f1f7be64700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1f740d1870 0x7f1f741b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:20.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.465 7f1f7be64700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1f741b0ae0 0x7f1f74108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:20.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.465 7f1f7be64700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f740fe1c0 con 0x7f1f740d1c70 2021-02-10T11:46:20.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.466 7f1f7be64700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f74109a60 con 0x7f1f741b0ae0 2021-02-10T11:46:20.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.466 7f1f7be64700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f74109750 con 0x7f1f740d1870 2021-02-10T11:46:20.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.466 7f1f7947f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1f741b0ae0 0x7f1f74108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:20.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.466 7f1f7947f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1f741b0ae0 0x7f1f74108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33106/0 (socket says 172.21.2.101:33106) 2021-02-10T11:46:20.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.466 7f1f7947f700 1 -- 172.21.2.101:0/1284978201 learned_addr learned my addr 172.21.2.101:0/1284978201 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:20.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.466 7f1f79c80700 1 --2- 172.21.2.101:0/1284978201 >> v2:172.21.2.108:3300/0 conn(0x7f1f740d1870 0x7f1f741b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:20.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.466 7f1f7947f700 1 -- 172.21.2.101:0/1284978201 >> v2:172.21.2.108:3300/0 conn(0x7f1f740d1870 msgr2=0x7f1f741b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:20.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.466 7f1f7947f700 1 --2- 172.21.2.101:0/1284978201 >> v2:172.21.2.108:3300/0 conn(0x7f1f740d1870 0x7f1f741b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:20.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.466 7f1f7947f700 1 -- 172.21.2.101:0/1284978201 >> v1:172.21.2.108:6789/0 conn(0x7f1f740d1c70 legacy=0x7f1f74108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:20.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.466 7f1f7947f700 1 -- 172.21.2.101:0/1284978201 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f74088070 con 0x7f1f741b0ae0 2021-02-10T11:46:20.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.466 7f1f79c80700 1 --2- 172.21.2.101:0/1284978201 >> v2:172.21.2.108:3300/0 conn(0x7f1f740d1870 0x7f1f741b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:46:20.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.466 7f1f7947f700 1 --2- 172.21.2.101:0/1284978201 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f741b0ae0 0x7f1f74108940 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 rx=0x7f1f70006700 tx=0x7f1f70009b80).ready entity=mon.0 client_cookie=a3ead90f0ef820aa server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:20.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.467 7f1f78c7e700 1 -- 172.21.2.101:0/1284978201 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f70000c90 con 0x7f1f741b0ae0 2021-02-10T11:46:20.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.467 7f1f78c7e700 1 -- 172.21.2.101:0/1284978201 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7000c0c0 con 0x7f1f741b0ae0 2021-02-10T11:46:20.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.467 7f1f78c7e700 1 -- 172.21.2.101:0/1284978201 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f7000c5e0 con 0x7f1f741b0ae0 2021-02-10T11:46:20.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.467 7f1f7be64700 1 -- 172.21.2.101:0/1284978201 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f741b0ae0 msgr2=0x7f1f74108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:20.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.467 7f1f7be64700 1 --2- 172.21.2.101:0/1284978201 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f741b0ae0 0x7f1f74108940 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 rx=0x7f1f70006700 tx=0x7f1f70009b80).stop 2021-02-10T11:46:20.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.467 7f1f7be64700 1 -- 172.21.2.101:0/1284978201 shutdown_connections 2021-02-10T11:46:20.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.467 7f1f7be64700 1 --2- 172.21.2.101:0/1284978201 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f741b0ae0 0x7f1f74108940 unknown :-1 s=CLOSED pgs=2936 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:20.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.467 7f1f7be64700 1 --2- 172.21.2.101:0/1284978201 >> v2:172.21.2.108:3300/0 conn(0x7f1f740d1870 0x7f1f741b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:20.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.468 7f1f7be64700 1 -- 172.21.2.101:0/1284978201 >> 172.21.2.101:0/1284978201 conn(0x7f1f740ce420 msgr2=0x7f1f740d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:20.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.468 7f1f7be64700 1 -- 172.21.2.101:0/1284978201 shutdown_connections 2021-02-10T11:46:20.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.468 7f1f7be64700 1 -- 172.21.2.101:0/1284978201 wait complete. 2021-02-10T11:46:20.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.482 7f1f7be64700 1 Processor -- start 2021-02-10T11:46:20.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.482 7f1f7be64700 1 -- start start 2021-02-10T11:46:20.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.483 7f1f7be64700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f740c3820 0x7f1f74084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:20.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.483 7f1f7be64700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f740a0290 0x7f1f740852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:20.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.483 7f1f7be64700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f74086270 0x7f1f74086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:20.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.483 7f1f7be64700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740c0700 con 0x7f1f740c3820 2021-02-10T11:46:20.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.483 7f1f7be64700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740858f0 con 0x7f1f74086270 2021-02-10T11:46:20.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.483 7f1f7be64700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f74085b10 con 0x7f1f740a0290 2021-02-10T11:46:20.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.483 7f1f79c80700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f740c3820 0x7f1f74084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.483 7f1f79c80700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f740c3820 0x7f1f74084cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33110/0 (socket says 172.21.2.101:33110) 2021-02-10T11:46:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.483 7f1f79c80700 1 -- 172.21.2.101:0/51853032 learned_addr learned my addr 172.21.2.101:0/51853032 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.483 7f1f7a481700 1 --2- 172.21.2.101:0/51853032 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f74086270 0x7f1f74086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.483 7f1f7947f700 1 --2- 172.21.2.101:0/51853032 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f740a0290 0x7f1f740852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.483 7f1f7a481700 1 -- 172.21.2.101:0/51853032 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f740a0290 msgr2=0x7f1f740852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.483 7f1f7a481700 1 --2- 172.21.2.101:0/51853032 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f740a0290 0x7f1f740852d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.483 7f1f7a481700 1 -- 172.21.2.101:0/51853032 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f740c3820 msgr2=0x7f1f74084cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.483 7f1f7a481700 1 --2- 172.21.2.101:0/51853032 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f740c3820 0x7f1f74084cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.483 7f1f7a481700 1 -- 172.21.2.101:0/51853032 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f74086c90 con 0x7f1f74086270 2021-02-10T11:46:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.484 7f1f79c80700 1 --2- 172.21.2.101:0/51853032 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f740c3820 0x7f1f74084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:46:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.484 7f1f7a481700 1 --2- 172.21.2.101:0/51853032 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f74086270 0x7f1f74086670 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f1f6c009fc0 tx=0x7f1f6c00bc90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:20.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.484 7f1f667fc700 1 -- 172.21.2.101:0/51853032 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f6c007070 con 0x7f1f74086270 2021-02-10T11:46:20.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.484 7f1f667fc700 1 -- 172.21.2.101:0/51853032 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6c007330 con 0x7f1f74086270 2021-02-10T11:46:20.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.484 7f1f7be64700 1 -- 172.21.2.101:0/51853032 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f74086c90 con 0x7f1f74086270 2021-02-10T11:46:20.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.484 7f1f667fc700 1 -- 172.21.2.101:0/51853032 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f6c007850 con 0x7f1f74086270 2021-02-10T11:46:20.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.484 7f1f7be64700 1 -- 172.21.2.101:0/51853032 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f74086c90 con 0x7f1f74086270 2021-02-10T11:46:20.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.485 7f1f667fc700 1 -- 172.21.2.101:0/51853032 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1f6c007000 con 0x7f1f74086270 2021-02-10T11:46:20.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.485 7f1f667fc700 1 --2- 172.21.2.101:0/51853032 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f5c018730 0x7f1f5c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:20.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.485 7f1f667fc700 1 -- 172.21.2.101:0/51853032 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5617+0+0 (secure 0 0 0) 0x7f1f6c032980 con 0x7f1f74086270 2021-02-10T11:46:20.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.485 7f1f79c80700 1 --2- 172.21.2.101:0/51853032 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f5c018730 0x7f1f5c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:20.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.485 7f1f79c80700 1 --2- 172.21.2.101:0/51853032 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f5c018730 0x7f1f5c01ac10 crc :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:20.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.516 7f1f7be64700 1 -- 172.21.2.101:0/51853032 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f74082a70 con 0x7f1f74086270 2021-02-10T11:46:20.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.520 7f1f667fc700 1 -- 172.21.2.101:0/51853032 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1f6c00bdd0 con 0x7f1f74086270 2021-02-10T11:46:20.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.672 7f1f7be64700 1 -- 172.21.2.101:0/51853032 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1f74082a70 con 0x7f1f74086270 2021-02-10T11:46:20.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.678 7f1f667fc700 1 -- 172.21.2.101:0/51853032 <== mon.1 v2:172.21.2.108:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1f6c007000 con 0x7f1f74086270 2021-02-10T11:46:20.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.681 7f1f7be64700 1 -- 172.21.2.101:0/51853032 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f5c018730 msgr2=0x7f1f5c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:20.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.681 7f1f7be64700 1 --2- 172.21.2.101:0/51853032 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f5c018730 0x7f1f5c01ac10 crc :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:20.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.681 7f1f7be64700 1 -- 172.21.2.101:0/51853032 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f74086270 msgr2=0x7f1f74086670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:20.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.681 7f1f7be64700 1 --2- 172.21.2.101:0/51853032 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f74086270 0x7f1f74086670 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f1f6c009fc0 tx=0x7f1f6c00bc90).stop 2021-02-10T11:46:20.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.681 7f1f7be64700 1 -- 172.21.2.101:0/51853032 shutdown_connections 2021-02-10T11:46:20.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.681 7f1f7be64700 1 --2- 172.21.2.101:0/51853032 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f5c018730 0x7f1f5c01ac10 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:20.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.681 7f1f7be64700 1 --2- 172.21.2.101:0/51853032 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f740a0290 0x7f1f740852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:20.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.681 7f1f7be64700 1 --2- 172.21.2.101:0/51853032 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f74086270 0x7f1f74086670 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:20.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.681 7f1f7be64700 1 --2- 172.21.2.101:0/51853032 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f740c3820 0x7f1f74084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:20.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.681 7f1f7be64700 1 -- 172.21.2.101:0/51853032 >> 172.21.2.101:0/51853032 conn(0x7f1f740872c0 msgr2=0x7f1f74095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:20.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.681 7f1f7be64700 1 -- 172.21.2.101:0/51853032 shutdown_connections 2021-02-10T11:46:20.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:20.681 7f1f7be64700 1 -- 172.21.2.101:0/51853032 wait complete. 2021-02-10T11:46:20.687 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:46:20.687 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:46:21.034 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:46:21.379 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:46:21.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:21.380 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:46:21.380 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:46:21.715 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2021-02-10T11:46:21.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:46:21.716 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-02-10T11:46:21.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:21.716 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:46:21.716 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:46:22.051 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:46:22.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:46:22.052 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:46:22.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:22.052 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:46:22.052 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:46:22.386 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:46:22.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:46:22.386 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:46:22.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:22.386 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:46:22.387 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:46:22.721 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:46:22.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:46:22.721 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:46:22.722 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json-pretty 2021-02-10T11:46:23.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.138 7fd00e8bc700 1 Processor -- start 2021-02-10T11:46:23.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.138 7fd00e8bc700 1 -- start start 2021-02-10T11:46:23.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.139 7fd00e8bc700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd0080d1870 0x7fd0081b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:23.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.139 7fd00e8bc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0080f0650 0x7fd0080cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:23.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.139 7fd00e8bc700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0080f0b10 con 0x7fd0081b0a10 2021-02-10T11:46:23.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.139 7fd00e8bc700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0080e5ed0 con 0x7fd0080d1870 2021-02-10T11:46:23.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.139 7fd00e8bc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0080d1c70 con 0x7fd0080f0650 2021-02-10T11:46:23.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.139 7fd00ced9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0080f0650 0x7fd0080cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:23.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.139 7fd007fff700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd0080d1870 0x7fd0081b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:23.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.139 7fd0077fe700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fd0081b0a10 0x7fd0080cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:55254/0 (socket says 172.21.2.101:55254) 2021-02-10T11:46:23.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.139 7fd0077fe700 1 -- 172.21.2.101:0/18006342 learned_addr learned my addr 172.21.2.101:0/18006342 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:23.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.139 7fd007fff700 1 -- 172.21.2.101:0/18006342 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0080f0650 msgr2=0x7fd0080cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:23.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.139 7fd007fff700 1 --2- 172.21.2.101:0/18006342 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0080f0650 0x7fd0080cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:23.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.139 7fd007fff700 1 -- 172.21.2.101:0/18006342 >> v1:172.21.2.108:6789/0 conn(0x7fd0081b0a10 legacy=0x7fd0080cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:23.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.139 7fd007fff700 1 -- 172.21.2.101:0/18006342 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0080d1e90 con 0x7fd0080d1870 2021-02-10T11:46:23.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.139 7fd00ced9700 1 --2- 172.21.2.101:0/18006342 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0080f0650 0x7fd0080cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:46:23.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.140 7fd007fff700 1 --2- 172.21.2.101:0/18006342 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0080d1870 0x7fd0081b03f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7fcff8006650 tx=0x7fcff8009a80).ready entity=mon.1 client_cookie=ffafe7ddec837894 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:23.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.140 7fd006ffd700 1 -- 172.21.2.101:0/18006342 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcff80009b0 con 0x7fd0080d1870 2021-02-10T11:46:23.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.140 7fd006ffd700 1 -- 172.21.2.101:0/18006342 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff8006740 con 0x7fd0080d1870 2021-02-10T11:46:23.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.140 7fd006ffd700 1 -- 172.21.2.101:0/18006342 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcff800c0c0 con 0x7fd0080d1870 2021-02-10T11:46:23.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.140 7fd00e8bc700 1 -- 172.21.2.101:0/18006342 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0080d1870 msgr2=0x7fd0081b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:23.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.140 7fd00e8bc700 1 --2- 172.21.2.101:0/18006342 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0080d1870 0x7fd0081b03f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7fcff8006650 tx=0x7fcff8009a80).stop 2021-02-10T11:46:23.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.141 7fd00e8bc700 1 -- 172.21.2.101:0/18006342 shutdown_connections 2021-02-10T11:46:23.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.141 7fd00e8bc700 1 --2- 172.21.2.101:0/18006342 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0080f0650 0x7fd0080cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:23.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.141 7fd00e8bc700 1 --2- 172.21.2.101:0/18006342 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0080d1870 0x7fd0081b03f0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:23.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.141 7fd00e8bc700 1 -- 172.21.2.101:0/18006342 >> 172.21.2.101:0/18006342 conn(0x7fd0080ce450 msgr2=0x7fd0080d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:23.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.141 7fd00e8bc700 1 -- 172.21.2.101:0/18006342 shutdown_connections 2021-02-10T11:46:23.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.141 7fd00e8bc700 1 -- 172.21.2.101:0/18006342 wait complete. 2021-02-10T11:46:23.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.155 7fd00e8bc700 1 Processor -- start 2021-02-10T11:46:23.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.155 7fd00e8bc700 1 -- start start 2021-02-10T11:46:23.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.155 7fd00e8bc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0081942b0 0x7fd0080b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:23.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.155 7fd00e8bc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd008195070 0x7fd008195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:23.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.155 7fd00e8bc700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0080bb8e0 0x7fd0080bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:23.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.155 7fd00e8bc700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0081946b0 con 0x7fd008195070 2021-02-10T11:46:23.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.155 7fd00e8bc700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0080b3b70 con 0x7fd0080bb8e0 2021-02-10T11:46:23.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.155 7fd00e8bc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0080b3d90 con 0x7fd0081942b0 2021-02-10T11:46:23.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.156 7fd007fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0081942b0 0x7fd0080b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:23.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.156 7fd0077fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd008195070 0x7fd008195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:23.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.156 7fd007fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0081942b0 0x7fd0080b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:54086/0 (socket says 172.21.2.101:54086) 2021-02-10T11:46:23.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.156 7fd007fff700 1 -- 172.21.2.101:0/2346896054 learned_addr learned my addr 172.21.2.101:0/2346896054 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:23.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.156 7fd00ced9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0080bb8e0 0x7fd0080bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:23.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.156 7fd007fff700 1 -- 172.21.2.101:0/2346896054 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0080bb8e0 msgr2=0x7fd0080bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:23.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.156 7fd007fff700 1 --2- 172.21.2.101:0/2346896054 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0080bb8e0 0x7fd0080bbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:23.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.156 7fd007fff700 1 -- 172.21.2.101:0/2346896054 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd008195070 msgr2=0x7fd008195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:23.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.156 7fd007fff700 1 --2- 172.21.2.101:0/2346896054 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd008195070 0x7fd008195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:23.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.156 7fd007fff700 1 -- 172.21.2.101:0/2346896054 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0081a49e0 con 0x7fd0081942b0 2021-02-10T11:46:23.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.156 7fd007fff700 1 --2- 172.21.2.101:0/2346896054 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0081942b0 0x7fd0080b3550 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7fcff8009630 tx=0x7fcff800b970).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:23.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.156 7fd004ff9700 1 -- 172.21.2.101:0/2346896054 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcff8002030 con 0x7fd0081942b0 2021-02-10T11:46:23.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.156 7fd004ff9700 1 -- 172.21.2.101:0/2346896054 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff8002330 con 0x7fd0081942b0 2021-02-10T11:46:23.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.156 7fd00e8bc700 1 -- 172.21.2.101:0/2346896054 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0081a49e0 con 0x7fd0081942b0 2021-02-10T11:46:23.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.156 7fd004ff9700 1 -- 172.21.2.101:0/2346896054 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcff8002810 con 0x7fd0081942b0 2021-02-10T11:46:23.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.156 7fd00e8bc700 1 -- 172.21.2.101:0/2346896054 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0081a49e0 con 0x7fd0081942b0 2021-02-10T11:46:23.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.157 7fd004ff9700 1 -- 172.21.2.101:0/2346896054 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fcff8000c90 con 0x7fd0081942b0 2021-02-10T11:46:23.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.157 7fd004ff9700 1 --2- 172.21.2.101:0/2346896054 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcfe8018730 0x7fcfe801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:23.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.157 7fd004ff9700 1 -- 172.21.2.101:0/2346896054 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5617+0+0 (secure 0 0 0) 0x7fcff80326f0 con 0x7fd0081942b0 2021-02-10T11:46:23.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.157 7fd0077fe700 1 --2- 172.21.2.101:0/2346896054 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcfe8018730 0x7fcfe801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:23.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.157 7fd0077fe700 1 --2- 172.21.2.101:0/2346896054 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcfe8018730 0x7fcfe801ac10 crc :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:23.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.188 7fd00e8bc700 1 -- 172.21.2.101:0/2346896054 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0080bc5f0 con 0x7fd0081942b0 2021-02-10T11:46:23.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.193 7fd004ff9700 1 -- 172.21.2.101:0/2346896054 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fcff8002020 con 0x7fd0081942b0 2021-02-10T11:46:23.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.319 7fd00e8bc700 1 -- 172.21.2.101:0/2346896054 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd0080bc5f0 con 0x7fd0081942b0 2021-02-10T11:46:23.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.321 7fd004ff9700 1 -- 172.21.2.101:0/2346896054 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v166) v1 ==== 81+0+30586 (secure 0 0 0) 0x7fcff8000c90 con 0x7fd0081942b0 2021-02-10T11:46:23.321 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:46:23.321 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:46:23.321 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 166, 2021-02-10T11:46:23.321 INFO:teuthology.orchestra.run.gibba001.stdout: "fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:46:23.321 INFO:teuthology.orchestra.run.gibba001.stdout: "created": "2021-02-10 11:12:36.788366", 2021-02-10T11:46:23.321 INFO:teuthology.orchestra.run.gibba001.stdout: "modified": "2021-02-10 11:46:16.986219", 2021-02-10T11:46:23.321 INFO:teuthology.orchestra.run.gibba001.stdout: "last_up_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:46:23.321 INFO:teuthology.orchestra.run.gibba001.stdout: "last_in_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:46:23.321 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-02-10T11:46:23.321 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_num": 5799936, 2021-02-10T11:46:23.321 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_set": [ 2021-02-10T11:46:23.322 INFO:teuthology.orchestra.run.gibba001.stdout: "pglog_hardlimit", 2021-02-10T11:46:23.322 INFO:teuthology.orchestra.run.gibba001.stdout: "purged_snapdirs", 2021-02-10T11:46:23.322 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_deletes", 2021-02-10T11:46:23.322 INFO:teuthology.orchestra.run.gibba001.stdout: "sortbitwise" 2021-02-10T11:46:23.322 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:46:23.322 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_version": 4, 2021-02-10T11:46:23.322 INFO:teuthology.orchestra.run.gibba001.stdout: "full_ratio": 0.94999998807907104, 2021-02-10T11:46:23.322 INFO:teuthology.orchestra.run.gibba001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-02-10T11:46:23.322 INFO:teuthology.orchestra.run.gibba001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-02-10T11:46:23.322 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_snapshot": "", 2021-02-10T11:46:23.322 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_max": 35, 2021-02-10T11:46:23.322 INFO:teuthology.orchestra.run.gibba001.stdout: "max_osd": 8, 2021-02-10T11:46:23.323 INFO:teuthology.orchestra.run.gibba001.stdout: "require_min_compat_client": "jewel", 2021-02-10T11:46:23.323 INFO:teuthology.orchestra.run.gibba001.stdout: "min_compat_client": "jewel", 2021-02-10T11:46:23.323 INFO:teuthology.orchestra.run.gibba001.stdout: "require_osd_release": "nautilus", 2021-02-10T11:46:23.323 INFO:teuthology.orchestra.run.gibba001.stdout: "pools": [ 2021-02-10T11:46:23.323 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.323 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 1, 2021-02-10T11:46:23.323 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "rbd", 2021-02-10T11:46:23.323 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:13:08.037472", 2021-02-10T11:46:23.323 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:46:23.323 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:46:23.323 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:46:23.323 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:46:23.324 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:46:23.324 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:46:23.324 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:46:23.324 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:46:23.324 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:46:23.324 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:46:23.324 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:46:23.324 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:46:23.324 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:46:23.324 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:46:23.324 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:46:23.324 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:46:23.324 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:46:23.325 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:46:23.325 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:46:23.325 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:46:23.325 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.325 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "16", 2021-02-10T11:46:23.325 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:46:23.325 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:46:23.325 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:46:23.325 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:46:23.325 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:46:23.325 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:46:23.325 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:46:23.325 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:46:23.325 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:46:23.326 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:46:23.326 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:46:23.326 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:46:23.326 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:46:23.326 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:46:23.326 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:46:23.326 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:46:23.326 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:46:23.326 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:46:23.326 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:46:23.326 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:46:23.326 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:46:23.326 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:46:23.327 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:46:23.327 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:46:23.327 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:46:23.327 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:46:23.327 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.327 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:46:23.327 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:46:23.327 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:46:23.327 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:46:23.327 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:46:23.327 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:46:23.327 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:46:23.327 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:46:23.328 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:46:23.328 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:46:23.328 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:46:23.328 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:46:23.328 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:46:23.328 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd": {} 2021-02-10T11:46:23.328 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.328 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.328 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.328 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 34, 2021-02-10T11:46:23.328 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_metadata", 2021-02-10T11:46:23.328 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:45:42.100593", 2021-02-10T11:46:23.328 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:46:23.329 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:46:23.329 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:46:23.329 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:46:23.329 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:46:23.329 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:46:23.329 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:46:23.329 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:46:23.329 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:46:23.329 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:46:23.329 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:46:23.329 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:46:23.329 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:46:23.329 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:46:23.329 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:46:23.329 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:46:23.330 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:46:23.330 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:46:23.330 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:46:23.330 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:46:23.330 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.330 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "163", 2021-02-10T11:46:23.330 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:46:23.330 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:46:23.330 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:46:23.330 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:46:23.330 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:46:23.330 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:46:23.331 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:46:23.331 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:46:23.331 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:46:23.331 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:46:23.331 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:46:23.331 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:46:23.331 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:46:23.331 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:46:23.331 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:46:23.331 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:46:23.331 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:46:23.331 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:46:23.331 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:46:23.332 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:46:23.332 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:46:23.332 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:46:23.332 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:46:23.332 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:46:23.332 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:46:23.332 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:46:23.332 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.332 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:46:23.332 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:46:23.332 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:46:23.332 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:46:23.332 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:46:23.332 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:46:23.333 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:46:23.333 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:46:23.333 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:46:23.333 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:46:23.333 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:46:23.333 INFO:teuthology.orchestra.run.gibba001.stdout: "options": { 2021-02-10T11:46:23.333 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_bias": 4, 2021-02-10T11:46:23.333 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_min": 16, 2021-02-10T11:46:23.333 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_priority": 5 2021-02-10T11:46:23.333 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.333 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:46:23.333 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:46:23.333 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": "cephfs" 2021-02-10T11:46:23.333 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.334 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.334 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.334 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.334 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 35, 2021-02-10T11:46:23.334 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_data", 2021-02-10T11:46:23.334 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:45:43.124384", 2021-02-10T11:46:23.334 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 8193, 2021-02-10T11:46:23.334 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2021-02-10T11:46:23.334 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:46:23.334 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:46:23.334 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:46:23.334 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:46:23.335 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:46:23.335 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:46:23.335 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:46:23.335 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:46:23.335 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:46:23.335 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:46:23.335 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:46:23.335 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:46:23.335 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:46:23.335 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:46:23.335 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:46:23.335 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:46:23.335 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:46:23.335 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:46:23.336 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.336 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "165", 2021-02-10T11:46:23.336 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:46:23.336 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:46:23.336 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:46:23.336 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:46:23.336 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:46:23.336 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 3, 2021-02-10T11:46:23.336 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 165, 2021-02-10T11:46:23.336 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:46:23.336 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[2~2]", 2021-02-10T11:46:23.336 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:46:23.336 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:46:23.337 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:46:23.337 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:46:23.337 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:46:23.337 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:46:23.337 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:46:23.337 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:46:23.337 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:46:23.337 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:46:23.337 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:46:23.337 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:46:23.337 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:46:23.337 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:46:23.337 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:46:23.338 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:46:23.338 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:46:23.338 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.338 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:46:23.338 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:46:23.338 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:46:23.338 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:46:23.338 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:46:23.338 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:46:23.338 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:46:23.338 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:46:23.338 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:46:23.338 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:46:23.339 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:46:23.339 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:46:23.339 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:46:23.339 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:46:23.339 INFO:teuthology.orchestra.run.gibba001.stdout: "data": "cephfs" 2021-02-10T11:46:23.339 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.339 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.339 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.339 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:46:23.339 INFO:teuthology.orchestra.run.gibba001.stdout: "osds": [ 2021-02-10T11:46:23.339 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.339 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:46:23.339 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5", 2021-02-10T11:46:23.340 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:46:23.340 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:46:23.340 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:46:23.340 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:46:23.340 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:46:23.340 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:46:23.340 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:46:23.340 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 161, 2021-02-10T11:46:23.340 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:46:23.340 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:46:23.340 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:46:23.340 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.340 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.341 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.341 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6802", 2021-02-10T11:46:23.341 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:46:23.341 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.341 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.341 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.341 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6805", 2021-02-10T11:46:23.341 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:46:23.341 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.341 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.341 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.341 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:46:23.341 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.342 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.342 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.342 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6807", 2021-02-10T11:46:23.342 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:46:23.342 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.342 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.342 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.342 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6809", 2021-02-10T11:46:23.342 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:46:23.342 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.342 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.342 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.342 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:46:23.343 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.343 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.343 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.343 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6817", 2021-02-10T11:46:23.343 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:46:23.343 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.343 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.343 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.343 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6820", 2021-02-10T11:46:23.343 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:46:23.343 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.343 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.343 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.343 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:46:23.344 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.344 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.344 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.344 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6812", 2021-02-10T11:46:23.344 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:46:23.344 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.344 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.344 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.344 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6814", 2021-02-10T11:46:23.344 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:46:23.344 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.344 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.344 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.344 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6805/25093", 2021-02-10T11:46:23.345 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6809/25093", 2021-02-10T11:46:23.345 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6820/25093", 2021-02-10T11:46:23.345 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6814/25093", 2021-02-10T11:46:23.345 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:46:23.345 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:46:23.345 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:46:23.345 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.345 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.345 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.345 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:46:23.345 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "ddc04ce3-b313-4707-8270-585391e38ecc", 2021-02-10T11:46:23.345 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:46:23.345 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:46:23.346 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:46:23.346 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:46:23.346 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:46:23.346 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:46:23.346 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:46:23.346 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 145, 2021-02-10T11:46:23.346 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:46:23.346 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:46:23.346 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:46:23.346 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.346 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.346 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.346 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6803", 2021-02-10T11:46:23.346 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:46:23.347 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.347 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.347 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.347 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6804", 2021-02-10T11:46:23.347 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:46:23.347 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.347 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.347 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.347 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:46:23.347 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.347 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.347 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.348 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6806", 2021-02-10T11:46:23.348 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:46:23.348 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.348 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.348 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.348 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6808", 2021-02-10T11:46:23.348 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:46:23.348 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.348 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.348 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.348 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:46:23.348 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.348 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.349 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.349 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6818", 2021-02-10T11:46:23.349 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:46:23.349 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.349 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.349 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.349 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6821", 2021-02-10T11:46:23.349 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:46:23.349 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.349 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.349 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.349 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:46:23.349 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.350 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.350 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.350 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6810", 2021-02-10T11:46:23.350 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:46:23.350 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.350 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.350 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.350 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6815", 2021-02-10T11:46:23.350 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:46:23.350 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.351 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.351 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.351 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6804/25092", 2021-02-10T11:46:23.351 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6808/25092", 2021-02-10T11:46:23.351 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6821/25092", 2021-02-10T11:46:23.352 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6815/25092", 2021-02-10T11:46:23.352 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:46:23.352 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:46:23.352 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:46:23.352 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.352 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.352 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.352 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:46:23.352 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3", 2021-02-10T11:46:23.352 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:46:23.352 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:46:23.352 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:46:23.352 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:46:23.352 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:46:23.353 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:46:23.353 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:46:23.353 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 162, 2021-02-10T11:46:23.353 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:46:23.353 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:46:23.353 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:46:23.353 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.353 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.353 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.353 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6811", 2021-02-10T11:46:23.353 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:46:23.353 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.353 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.353 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.354 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6813", 2021-02-10T11:46:23.354 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:46:23.354 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.354 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.354 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.354 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:46:23.354 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.354 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.354 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.354 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6816", 2021-02-10T11:46:23.354 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:46:23.354 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.354 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.355 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.355 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6819", 2021-02-10T11:46:23.355 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:46:23.355 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.355 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.355 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.355 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:46:23.355 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.355 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.355 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.355 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6825", 2021-02-10T11:46:23.355 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:46:23.355 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.355 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.355 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.356 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6827", 2021-02-10T11:46:23.356 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:46:23.356 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.356 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.356 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.356 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:46:23.356 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.356 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.356 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.356 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6822", 2021-02-10T11:46:23.356 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:46:23.356 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.356 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.357 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.357 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6823", 2021-02-10T11:46:23.357 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:46:23.357 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.357 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.357 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.357 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6813/25094", 2021-02-10T11:46:23.357 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6819/25094", 2021-02-10T11:46:23.357 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6827/25094", 2021-02-10T11:46:23.357 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6823/25094", 2021-02-10T11:46:23.357 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:46:23.357 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:46:23.357 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:46:23.357 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.358 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.358 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.358 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:46:23.358 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "0d9ef9b1-98aa-4d37-aa16-6305c7d46d52", 2021-02-10T11:46:23.358 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:46:23.358 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:46:23.358 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:46:23.358 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:46:23.358 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:46:23.358 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:46:23.358 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:46:23.358 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 162, 2021-02-10T11:46:23.358 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:46:23.359 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:46:23.359 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:46:23.359 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.359 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.359 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.359 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6824", 2021-02-10T11:46:23.359 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:46:23.359 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.359 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.359 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.359 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6826", 2021-02-10T11:46:23.359 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:46:23.359 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.360 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.360 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.360 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:46:23.360 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.360 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.360 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.360 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6828", 2021-02-10T11:46:23.360 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:46:23.360 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.360 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.360 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.360 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6829", 2021-02-10T11:46:23.360 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:46:23.360 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.361 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.361 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.361 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:46:23.361 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.361 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.361 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.361 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6832", 2021-02-10T11:46:23.361 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:46:23.361 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.361 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.361 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.361 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6833", 2021-02-10T11:46:23.361 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:46:23.361 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.362 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.362 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.362 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:46:23.362 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.362 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.362 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.362 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6830", 2021-02-10T11:46:23.362 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:46:23.362 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.362 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.362 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.362 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6831", 2021-02-10T11:46:23.362 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:46:23.362 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.363 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.363 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.363 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6826/25095", 2021-02-10T11:46:23.363 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6829/25095", 2021-02-10T11:46:23.363 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6833/25095", 2021-02-10T11:46:23.363 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6831/25095", 2021-02-10T11:46:23.363 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:46:23.363 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:46:23.363 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:46:23.363 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.363 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.363 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.363 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:46:23.363 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "36742c21-bfeb-4e11-bce9-8e745378b605", 2021-02-10T11:46:23.364 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:46:23.364 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:46:23.364 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:46:23.364 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:46:23.364 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:46:23.364 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:46:23.364 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:46:23.364 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 162, 2021-02-10T11:46:23.364 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:46:23.364 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:46:23.364 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:46:23.364 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.364 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.364 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.365 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6816", 2021-02-10T11:46:23.365 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:46:23.365 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.365 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.365 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.365 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6818", 2021-02-10T11:46:23.365 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:46:23.365 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.365 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.365 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.365 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:46:23.365 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.365 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.365 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.366 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6820", 2021-02-10T11:46:23.366 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:46:23.366 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.366 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.366 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.366 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6822", 2021-02-10T11:46:23.366 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:46:23.366 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.366 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.366 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.366 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:46:23.366 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.366 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.366 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.367 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6828", 2021-02-10T11:46:23.367 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:46:23.367 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.367 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.367 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.367 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6830", 2021-02-10T11:46:23.367 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:46:23.367 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.367 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.367 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.367 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:46:23.367 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.367 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.367 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.368 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6824", 2021-02-10T11:46:23.368 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:46:23.368 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.368 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.368 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.368 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6826", 2021-02-10T11:46:23.368 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:46:23.368 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.368 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.368 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.368 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6818/36380", 2021-02-10T11:46:23.368 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6822/36380", 2021-02-10T11:46:23.368 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6830/36380", 2021-02-10T11:46:23.369 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6826/36380", 2021-02-10T11:46:23.369 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:46:23.369 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:46:23.369 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:46:23.369 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.369 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.369 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.369 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:46:23.369 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "88e694e8-fe98-4d1b-88c8-8c9907ece60b", 2021-02-10T11:46:23.369 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:46:23.369 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:46:23.369 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:46:23.369 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:46:23.369 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:46:23.370 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:46:23.370 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:46:23.370 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 162, 2021-02-10T11:46:23.370 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:46:23.370 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:46:23.370 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:46:23.370 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.370 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.370 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.370 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6808", 2021-02-10T11:46:23.370 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:46:23.370 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.370 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.371 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.371 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6809", 2021-02-10T11:46:23.371 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:46:23.371 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.371 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.371 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.371 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:46:23.371 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.371 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.371 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.371 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6810", 2021-02-10T11:46:23.371 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:46:23.371 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.371 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.372 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.372 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6811", 2021-02-10T11:46:23.372 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:46:23.372 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.372 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.372 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.372 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:46:23.372 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.372 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.372 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.372 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6814", 2021-02-10T11:46:23.372 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:46:23.372 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.372 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.373 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.373 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6815", 2021-02-10T11:46:23.373 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:46:23.373 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.373 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.373 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.373 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:46:23.373 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.373 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.373 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.373 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6812", 2021-02-10T11:46:23.373 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:46:23.373 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.373 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.374 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.374 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6813", 2021-02-10T11:46:23.374 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:46:23.374 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.374 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.374 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.374 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6809/36383", 2021-02-10T11:46:23.374 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6811/36383", 2021-02-10T11:46:23.374 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6815/36383", 2021-02-10T11:46:23.374 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6813/36383", 2021-02-10T11:46:23.374 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:46:23.374 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:46:23.374 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:46:23.375 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.375 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.375 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.375 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:46:23.375 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "87a13c6c-cf85-4aaf-916b-e73913484a49", 2021-02-10T11:46:23.375 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:46:23.375 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:46:23.375 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:46:23.375 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:46:23.375 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:46:23.375 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:46:23.375 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:46:23.375 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 162, 2021-02-10T11:46:23.375 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:46:23.376 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:46:23.376 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:46:23.376 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.376 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.376 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.376 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6800", 2021-02-10T11:46:23.376 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:46:23.376 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.376 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.376 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.376 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6801", 2021-02-10T11:46:23.376 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:46:23.376 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.377 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.377 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.377 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:46:23.377 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.377 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.377 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.377 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6802", 2021-02-10T11:46:23.377 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:46:23.377 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.377 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.377 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.377 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6803", 2021-02-10T11:46:23.378 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:46:23.378 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.378 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.378 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.378 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:46:23.378 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.378 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.378 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.378 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6806", 2021-02-10T11:46:23.378 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:46:23.378 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.379 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.379 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.379 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6807", 2021-02-10T11:46:23.379 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:46:23.379 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.379 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.379 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.379 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:46:23.379 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.379 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.379 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.379 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6804", 2021-02-10T11:46:23.379 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:46:23.380 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.380 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.380 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.380 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6805", 2021-02-10T11:46:23.380 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:46:23.380 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.380 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.380 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.380 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6801/36381", 2021-02-10T11:46:23.380 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6803/36381", 2021-02-10T11:46:23.380 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6807/36381", 2021-02-10T11:46:23.380 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6805/36381", 2021-02-10T11:46:23.381 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:46:23.381 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:46:23.381 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:46:23.381 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.381 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.381 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.381 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:46:23.381 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "7b90823f-854c-4592-a95b-a7fa98f93254", 2021-02-10T11:46:23.381 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:46:23.381 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:46:23.381 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:46:23.381 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:46:23.381 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:46:23.382 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:46:23.382 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:46:23.382 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 161, 2021-02-10T11:46:23.382 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:46:23.382 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:46:23.382 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:46:23.382 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.382 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.382 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.382 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6817", 2021-02-10T11:46:23.382 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:46:23.382 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.382 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.383 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.383 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6819", 2021-02-10T11:46:23.383 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:46:23.383 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.383 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.383 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.383 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:46:23.383 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.383 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.383 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.383 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6821", 2021-02-10T11:46:23.383 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:46:23.383 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.383 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.384 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.384 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6823", 2021-02-10T11:46:23.384 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:46:23.384 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.384 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.384 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.384 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:46:23.384 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.384 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.384 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.384 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6829", 2021-02-10T11:46:23.384 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:46:23.384 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.385 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.385 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.385 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6831", 2021-02-10T11:46:23.385 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:46:23.385 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.385 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.385 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.385 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:46:23.385 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:46:23.385 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.385 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:46:23.385 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6825", 2021-02-10T11:46:23.385 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:46:23.385 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.385 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.386 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:23.386 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6827", 2021-02-10T11:46:23.386 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:46:23.386 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.386 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.386 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.386 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6819/36382", 2021-02-10T11:46:23.386 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6823/36382", 2021-02-10T11:46:23.386 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6831/36382", 2021-02-10T11:46:23.386 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6827/36382", 2021-02-10T11:46:23.386 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:46:23.386 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:46:23.386 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:46:23.387 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.387 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.387 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:46:23.387 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_xinfo": [ 2021-02-10T11:46:23.387 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.387 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:46:23.387 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:46:23.387 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:46:23.387 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:46:23.387 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:46:23.387 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:46:23.387 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.387 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.387 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:46:23.387 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:46:23.388 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:46:23.388 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:46:23.388 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:46:23.388 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:46:23.388 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.388 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.388 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:46:23.388 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:46:23.388 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:46:23.388 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:46:23.388 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:46:23.388 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:46:23.388 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.388 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.389 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:46:23.389 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:46:23.389 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:46:23.389 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:46:23.389 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:46:23.389 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:46:23.389 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.389 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.389 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:46:23.389 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:46:23.389 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:46:23.389 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:46:23.389 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:46:23.390 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:46:23.390 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.390 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.390 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:46:23.390 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:46:23.390 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:46:23.390 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:46:23.390 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:46:23.390 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:46:23.390 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.390 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.390 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:46:23.390 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:46:23.390 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:46:23.391 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:46:23.391 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:46:23.391 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:46:23.391 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.391 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.391 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:46:23.391 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:46:23.391 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:46:23.391 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:46:23.391 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:46:23.391 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:46:23.391 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.391 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:46:23.391 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap": [], 2021-02-10T11:46:23.392 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap_items": [], 2021-02-10T11:46:23.392 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_temp": [], 2021-02-10T11:46:23.392 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_temp": [], 2021-02-10T11:46:23.392 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklist": {}, 2021-02-10T11:46:23.392 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profiles": { 2021-02-10T11:46:23.392 INFO:teuthology.orchestra.run.gibba001.stdout: "default": { 2021-02-10T11:46:23.392 INFO:teuthology.orchestra.run.gibba001.stdout: "crush-failure-domain": "osd", 2021-02-10T11:46:23.392 INFO:teuthology.orchestra.run.gibba001.stdout: "k": "2", 2021-02-10T11:46:23.392 INFO:teuthology.orchestra.run.gibba001.stdout: "m": "1", 2021-02-10T11:46:23.392 INFO:teuthology.orchestra.run.gibba001.stdout: "plugin": "jerasure", 2021-02-10T11:46:23.392 INFO:teuthology.orchestra.run.gibba001.stdout: "ruleset-failure-domain": "osd", 2021-02-10T11:46:23.392 INFO:teuthology.orchestra.run.gibba001.stdout: "technique": "reed_sol_van" 2021-02-10T11:46:23.392 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.393 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:23.393 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps_queue": [], 2021-02-10T11:46:23.393 INFO:teuthology.orchestra.run.gibba001.stdout: "new_removed_snaps": [], 2021-02-10T11:46:23.393 INFO:teuthology.orchestra.run.gibba001.stdout: "new_purged_snaps": [ 2021-02-10T11:46:23.393 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.393 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 35, 2021-02-10T11:46:23.393 INFO:teuthology.orchestra.run.gibba001.stdout: "snaps": [ 2021-02-10T11:46:23.393 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:23.393 INFO:teuthology.orchestra.run.gibba001.stdout: "begin": 2, 2021-02-10T11:46:23.393 INFO:teuthology.orchestra.run.gibba001.stdout: "length": 2 2021-02-10T11:46:23.393 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.393 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:23.393 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:23.393 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:46:23.393 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_node_flags": {}, 2021-02-10T11:46:23.394 INFO:teuthology.orchestra.run.gibba001.stdout: "device_class_flags": {} 2021-02-10T11:46:23.394 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:46:23.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.324 7fd00e8bc700 1 -- 172.21.2.101:0/2346896054 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcfe8018730 msgr2=0x7fcfe801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:23.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.324 7fd00e8bc700 1 --2- 172.21.2.101:0/2346896054 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcfe8018730 0x7fcfe801ac10 crc :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:23.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.324 7fd00e8bc700 1 -- 172.21.2.101:0/2346896054 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0081942b0 msgr2=0x7fd0080b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:23.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.324 7fd00e8bc700 1 --2- 172.21.2.101:0/2346896054 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0081942b0 0x7fd0080b3550 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7fcff8009630 tx=0x7fcff800b970).stop 2021-02-10T11:46:23.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.325 7fd00e8bc700 1 -- 172.21.2.101:0/2346896054 shutdown_connections 2021-02-10T11:46:23.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.325 7fd00e8bc700 1 --2- 172.21.2.101:0/2346896054 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd0080bb8e0 0x7fd0080bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:23.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.325 7fd00e8bc700 1 --2- 172.21.2.101:0/2346896054 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd008195070 0x7fd008195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:23.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.325 7fd00e8bc700 1 --2- 172.21.2.101:0/2346896054 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcfe8018730 0x7fcfe801ac10 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:23.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.325 7fd00e8bc700 1 --2- 172.21.2.101:0/2346896054 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd0081942b0 0x7fd0080b3550 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:23.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.325 7fd00e8bc700 1 -- 172.21.2.101:0/2346896054 >> 172.21.2.101:0/2346896054 conn(0x7fd0080b3040 msgr2=0x7fd0080c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:23.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.325 7fd00e8bc700 1 -- 172.21.2.101:0/2346896054 shutdown_connections 2021-02-10T11:46:23.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.325 7fd00e8bc700 1 -- 172.21.2.101:0/2346896054 wait complete. 2021-02-10T11:46:23.397 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:46:23.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.810 7f5bb292d700 1 Processor -- start 2021-02-10T11:46:23.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.810 7f5bb292d700 1 -- start start 2021-02-10T11:46:23.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.811 7f5bb292d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f5bac0d1870 0x7f5bac1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:23.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.811 7f5bb292d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bac1b0a10 0x7f5bac0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:23.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.811 7f5bb292d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bac0f0b10 con 0x7f5bac0d1c70 2021-02-10T11:46:23.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.811 7f5bb292d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bac0e5ed0 con 0x7f5bac0d1870 2021-02-10T11:46:23.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.811 7f5bb292d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac0f0650 con 0x7f5bac1b0a10 2021-02-10T11:46:23.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.811 7f5bb0f4a700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f5bac0d1c70 0x7f5bac0cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33624/0 (socket says 172.21.2.101:33624) 2021-02-10T11:46:23.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.811 7f5bb0f4a700 1 -- 172.21.2.101:0/1611502510 learned_addr learned my addr 172.21.2.101:0/1611502510 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:23.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.811 7f5baaffd700 1 -- 172.21.2.101:0/1611502510 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 928763219 0 0) 0x7f5ba4001500 con 0x7f5bac0d1c70 2021-02-10T11:46:23.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.811 7f5baaffd700 1 -- 172.21.2.101:0/1611502510 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b94002fe0 con 0x7f5bac0d1c70 2021-02-10T11:46:23.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5baaffd700 1 -- 172.21.2.101:0/1611502510 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2662486097 0 0) 0x7f5ba4001500 con 0x7f5bac0d1c70 2021-02-10T11:46:23.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5baaffd700 1 -- 172.21.2.101:0/1611502510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bac1b0a10 msgr2=0x7f5bac0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:23.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5bab7fe700 1 --2- 172.21.2.101:0/1611502510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bac1b0a10 0x7f5bac0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:23.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5baaffd700 1 --2- 172.21.2.101:0/1611502510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bac1b0a10 0x7f5bac0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:23.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5baaffd700 1 -- 172.21.2.101:0/1611502510 >> v2:172.21.2.108:3300/0 conn(0x7f5bac0d1870 msgr2=0x7f5bac1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:23.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5baaffd700 1 --2- 172.21.2.101:0/1611502510 >> v2:172.21.2.108:3300/0 conn(0x7f5bac0d1870 0x7f5bac1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:23.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5baaffd700 1 -- 172.21.2.101:0/1611502510 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bac0f0870 con 0x7f5bac0d1c70 2021-02-10T11:46:23.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5baaffd700 1 -- 172.21.2.101:0/1611502510 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f5ba4001a00 con 0x7f5bac0d1c70 2021-02-10T11:46:23.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5baaffd700 1 -- 172.21.2.101:0/1611502510 >> v1:172.21.2.101:6789/0 conn(0x7f5bac0d1c70 legacy=0x7f5bac0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:23.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5baaffd700 1 --2- 172.21.2.101:0/1611502510 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5b94003ae0 0x7f5b94006000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:23.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5baaffd700 1 --2- 172.21.2.101:0/1611502510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b94006820 0x7f5b94008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:23.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5baaffd700 1 --2- 172.21.2.101:0/1611502510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b94009640 0x7f5b9400bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:23.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5baaffd700 1 -- 172.21.2.101:0/1611502510 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9400c360 con 0x7f5b94006820 2021-02-10T11:46:23.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5baaffd700 1 -- 172.21.2.101:0/1611502510 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9400c580 con 0x7f5b94003ae0 2021-02-10T11:46:23.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5baaffd700 1 -- 172.21.2.101:0/1611502510 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9400c7a0 con 0x7f5b94009640 2021-02-10T11:46:23.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5bab7fe700 1 --2- 172.21.2.101:0/1611502510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bac1b0a10 0x7f5bac0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:46:23.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5bab7fe700 1 --2- 172.21.2.101:0/1611502510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b94006820 0x7f5b94008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:23.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5babfff700 1 --2- 172.21.2.101:0/1611502510 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5b94003ae0 0x7f5b94006000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:23.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5bb0f4a700 1 --2- 172.21.2.101:0/1611502510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b94009640 0x7f5b9400bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:23.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5bab7fe700 1 -- 172.21.2.101:0/1611502510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b94009640 msgr2=0x7f5b9400bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:23.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5bab7fe700 1 --2- 172.21.2.101:0/1611502510 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b94009640 0x7f5b9400bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:23.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5bab7fe700 1 -- 172.21.2.101:0/1611502510 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5b94003ae0 msgr2=0x7f5b94006000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:23.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5bab7fe700 1 --2- 172.21.2.101:0/1611502510 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5b94003ae0 0x7f5b94006000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:23.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5bab7fe700 1 -- 172.21.2.101:0/1611502510 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9400ca40 con 0x7f5b94006820 2021-02-10T11:46:23.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.812 7f5bab7fe700 1 --2- 172.21.2.101:0/1611502510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b94006820 0x7f5b94008d40 secure :-1 s=READY pgs=2938 cs=0 l=1 rev1=1 rx=0x7f5ba0002e00 tx=0x7f5ba004eab0).ready entity=mon.0 client_cookie=f52a38bcf940eb3f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:23.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.813 7f5baaffd700 1 -- 172.21.2.101:0/1611502510 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5ba0000c90 con 0x7f5b94006820 2021-02-10T11:46:23.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.813 7f5baaffd700 1 -- 172.21.2.101:0/1611502510 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba00510c0 con 0x7f5b94006820 2021-02-10T11:46:23.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.813 7f5baaffd700 1 -- 172.21.2.101:0/1611502510 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5ba0051600 con 0x7f5b94006820 2021-02-10T11:46:23.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.813 7f5bb292d700 1 -- 172.21.2.101:0/1611502510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b94006820 msgr2=0x7f5b94008d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:23.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.813 7f5bb292d700 1 --2- 172.21.2.101:0/1611502510 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b94006820 0x7f5b94008d40 secure :-1 s=READY pgs=2938 cs=0 l=1 rev1=1 rx=0x7f5ba0002e00 tx=0x7f5ba004eab0).stop 2021-02-10T11:46:23.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.813 7f5bb0f4a700 1 -- 172.21.2.101:0/1611502510 reap_dead start 2021-02-10T11:46:23.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.813 7f5bb292d700 1 -- 172.21.2.101:0/1611502510 shutdown_connections 2021-02-10T11:46:23.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.813 7f5bb292d700 1 -- 172.21.2.101:0/1611502510 >> 172.21.2.101:0/1611502510 conn(0x7f5bac0ce450 msgr2=0x7f5bac0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:23.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.814 7f5bb292d700 1 -- 172.21.2.101:0/1611502510 shutdown_connections 2021-02-10T11:46:23.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.814 7f5bb292d700 1 -- 172.21.2.101:0/1611502510 wait complete. 2021-02-10T11:46:23.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.828 7f5bb292d700 1 Processor -- start 2021-02-10T11:46:23.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.828 7f5bb292d700 1 -- start start 2021-02-10T11:46:23.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.829 7f5bb292d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bac1942b0 0x7f5bac0b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:23.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.829 7f5bb292d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bac195070 0x7f5bac195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:23.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.829 7f5bb292d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5bac0bb8e0 0x7f5bac0bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:23.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.829 7f5bb292d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac1946b0 con 0x7f5bac1942b0 2021-02-10T11:46:23.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.829 7f5bb292d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac0b3b70 con 0x7f5bac0bb8e0 2021-02-10T11:46:23.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.829 7f5bb292d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac0b3d90 con 0x7f5bac195070 2021-02-10T11:46:23.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.829 7f5babfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bac1942b0 0x7f5bac0b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:23.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.829 7f5babfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bac1942b0 0x7f5bac0b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33140/0 (socket says 172.21.2.101:33140) 2021-02-10T11:46:23.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.829 7f5babfff700 1 -- 172.21.2.101:0/2045639079 learned_addr learned my addr 172.21.2.101:0/2045639079 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:23.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.829 7f5bab7fe700 1 --2- 172.21.2.101:0/2045639079 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bac195070 0x7f5bac195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:23.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.829 7f5bb0f4a700 1 --2- 172.21.2.101:0/2045639079 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5bac0bb8e0 0x7f5bac0bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:23.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.829 7f5babfff700 1 -- 172.21.2.101:0/2045639079 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bac195070 msgr2=0x7f5bac195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:23.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.829 7f5babfff700 1 --2- 172.21.2.101:0/2045639079 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bac195070 0x7f5bac195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:23.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.829 7f5babfff700 1 -- 172.21.2.101:0/2045639079 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5bac0bb8e0 msgr2=0x7f5bac0bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:23.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.829 7f5babfff700 1 --2- 172.21.2.101:0/2045639079 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5bac0bb8e0 0x7f5bac0bbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:23.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.829 7f5babfff700 1 -- 172.21.2.101:0/2045639079 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bac1a49e0 con 0x7f5bac1942b0 2021-02-10T11:46:23.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.829 7f5bab7fe700 1 --2- 172.21.2.101:0/2045639079 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bac195070 0x7f5bac195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:46:23.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.829 7f5bb0f4a700 1 --2- 172.21.2.101:0/2045639079 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5bac0bb8e0 0x7f5bac0bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:46:23.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.830 7f5babfff700 1 --2- 172.21.2.101:0/2045639079 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bac1942b0 0x7f5bac0b3550 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 rx=0x7f5b9c00b590 tx=0x7f5b9c00b930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:23.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.830 7f5ba8ff9700 1 -- 172.21.2.101:0/2045639079 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5b9c00bd20 con 0x7f5bac1942b0 2021-02-10T11:46:23.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.830 7f5ba8ff9700 1 -- 172.21.2.101:0/2045639079 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9c002030 con 0x7f5bac1942b0 2021-02-10T11:46:23.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.830 7f5bb292d700 1 -- 172.21.2.101:0/2045639079 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bac1a49e0 con 0x7f5bac1942b0 2021-02-10T11:46:23.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.830 7f5bb292d700 1 -- 172.21.2.101:0/2045639079 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bac0bc5f0 con 0x7f5bac1942b0 2021-02-10T11:46:23.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.830 7f5ba8ff9700 1 -- 172.21.2.101:0/2045639079 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5b9c002550 con 0x7f5bac1942b0 2021-02-10T11:46:23.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.831 7f5ba8ff9700 1 -- 172.21.2.101:0/2045639079 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5b9c000c90 con 0x7f5bac1942b0 2021-02-10T11:46:23.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.831 7f5ba8ff9700 1 --2- 172.21.2.101:0/2045639079 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b8c018730 0x7f5b8c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:23.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.831 7f5ba8ff9700 1 -- 172.21.2.101:0/2045639079 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5617+0+0 (secure 0 0 0) 0x7f5b9c0326b0 con 0x7f5bac1942b0 2021-02-10T11:46:23.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.831 7f5bab7fe700 1 --2- 172.21.2.101:0/2045639079 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b8c018730 0x7f5b8c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:23.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.831 7f5bab7fe700 1 --2- 172.21.2.101:0/2045639079 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b8c018730 0x7f5b8c01ac10 crc :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:23.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.861 7f5bb292d700 1 -- 172.21.2.101:0/2045639079 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bac0bc5f0 con 0x7f5bac1942b0 2021-02-10T11:46:23.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.864 7f5ba8ff9700 1 -- 172.21.2.101:0/2045639079 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5b9c000960 con 0x7f5bac1942b0 2021-02-10T11:46:23.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.992 7f5bb292d700 1 -- 172.21.2.101:0/2045639079 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5bac0bc5f0 con 0x7f5bac1942b0 2021-02-10T11:46:23.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.992 7f5ba8ff9700 1 -- 172.21.2.101:0/2045639079 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+2104 (secure 0 0 0) 0x7f5b9c00bd20 con 0x7f5bac1942b0 2021-02-10T11:46:23.990 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:46:23.991 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":140,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8371,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/453344112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":453344112},{"type":"v1","addr":"172.21.2.101:6835","nonce":453344112}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":135}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:45:44.128996","modified":"2021-02-10 11:45:46.139945","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7097":{"gid":7097,"name":"b","rank":0,"incarnation":136,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2456711519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2456711519},{"type":"v1","addr":"172.21.2.108:6833","nonce":2456711519}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":17}]} 2021-02-10T11:46:23.991 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 140 2021-02-10T11:46:23.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.994 7f5bb292d700 1 -- 172.21.2.101:0/2045639079 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b8c018730 msgr2=0x7f5b8c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:23.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.994 7f5bb292d700 1 --2- 172.21.2.101:0/2045639079 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b8c018730 0x7f5b8c01ac10 crc :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:23.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.994 7f5bb292d700 1 -- 172.21.2.101:0/2045639079 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bac1942b0 msgr2=0x7f5bac0b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:23.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.994 7f5bb292d700 1 --2- 172.21.2.101:0/2045639079 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bac1942b0 0x7f5bac0b3550 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 rx=0x7f5b9c00b590 tx=0x7f5b9c00b930).stop 2021-02-10T11:46:23.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.994 7f5bb292d700 1 -- 172.21.2.101:0/2045639079 shutdown_connections 2021-02-10T11:46:23.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.994 7f5bb292d700 1 --2- 172.21.2.101:0/2045639079 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5b8c018730 0x7f5b8c01ac10 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:23.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.994 7f5bb292d700 1 --2- 172.21.2.101:0/2045639079 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bac195070 0x7f5bac195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:23.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.994 7f5bb292d700 1 --2- 172.21.2.101:0/2045639079 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5bac0bb8e0 0x7f5bac0bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:23.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.994 7f5bb292d700 1 --2- 172.21.2.101:0/2045639079 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bac1942b0 0x7f5bac0b3550 unknown :-1 s=CLOSED pgs=2939 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:23.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.994 7f5bb292d700 1 -- 172.21.2.101:0/2045639079 >> 172.21.2.101:0/2045639079 conn(0x7f5bac0b3040 msgr2=0x7f5bac0c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:23.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.995 7f5bb292d700 1 -- 172.21.2.101:0/2045639079 shutdown_connections 2021-02-10T11:46:23.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:23.995 7f5bb292d700 1 -- 172.21.2.101:0/2045639079 wait complete. 2021-02-10T11:46:24.000 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs fail cephfs 2021-02-10T11:46:24.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.414 7fc80c144700 1 Processor -- start 2021-02-10T11:46:24.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.414 7fc80c144700 1 -- start start 2021-02-10T11:46:24.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.414 7fc80c144700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8040d1870 0x7fc8041b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:24.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.414 7fc80c144700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc8041b0a10 0x7fc8040cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:24.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.414 7fc80c144700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8040f0b10 con 0x7fc8040d1c70 2021-02-10T11:46:24.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.414 7fc80c144700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8040fe230 con 0x7fc8041b0a10 2021-02-10T11:46:24.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.414 7fc80c144700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8040e5ed0 con 0x7fc8040d1870 2021-02-10T11:46:24.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.414 7fc80975f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc8041b0a10 0x7fc8040cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:24.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.414 7fc80a761700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc8040d1c70 0x7fc8040cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33642/0 (socket says 172.21.2.101:33642) 2021-02-10T11:46:24.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.414 7fc80a761700 1 -- 172.21.2.101:0/1998199379 learned_addr learned my addr 172.21.2.101:0/1998199379 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:24.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.415 7fc80975f700 1 -- 172.21.2.101:0/1998199379 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8040d1870 msgr2=0x7fc8041b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:46:24.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.415 7fc80975f700 1 --2- 172.21.2.101:0/1998199379 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8040d1870 0x7fc8041b03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:24.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.415 7fc80975f700 1 -- 172.21.2.101:0/1998199379 >> v1:172.21.2.101:6789/0 conn(0x7fc8040d1c70 legacy=0x7fc8040cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:24.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.415 7fc80975f700 1 -- 172.21.2.101:0/1998199379 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8040f0650 con 0x7fc8041b0a10 2021-02-10T11:46:24.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.415 7fc80975f700 1 --2- 172.21.2.101:0/1998199379 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8041b0a10 0x7fc8040cf330 secure :-1 s=READY pgs=2941 cs=0 l=1 rev1=1 rx=0x7fc800006650 tx=0x7fc800009a00).ready entity=mon.0 client_cookie=1e3ddece66e673dd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:24.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.415 7fc808f5e700 1 -- 172.21.2.101:0/1998199379 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc8000009b0 con 0x7fc8041b0a10 2021-02-10T11:46:24.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.415 7fc808f5e700 1 -- 172.21.2.101:0/1998199379 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80000c0c0 con 0x7fc8041b0a10 2021-02-10T11:46:24.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.415 7fc808f5e700 1 -- 172.21.2.101:0/1998199379 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc80000c5a0 con 0x7fc8041b0a10 2021-02-10T11:46:24.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.415 7fc80c144700 1 -- 172.21.2.101:0/1998199379 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8041b0a10 msgr2=0x7fc8040cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:24.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.415 7fc80c144700 1 --2- 172.21.2.101:0/1998199379 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8041b0a10 0x7fc8040cf330 secure :-1 s=READY pgs=2941 cs=0 l=1 rev1=1 rx=0x7fc800006650 tx=0x7fc800009a00).stop 2021-02-10T11:46:24.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.415 7fc80c144700 1 -- 172.21.2.101:0/1998199379 shutdown_connections 2021-02-10T11:46:24.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.415 7fc80c144700 1 --2- 172.21.2.101:0/1998199379 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8041b0a10 0x7fc8040cf330 unknown :-1 s=CLOSED pgs=2941 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:24.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.415 7fc80c144700 1 --2- 172.21.2.101:0/1998199379 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc8040d1870 0x7fc8041b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:24.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.416 7fc80c144700 1 -- 172.21.2.101:0/1998199379 >> 172.21.2.101:0/1998199379 conn(0x7fc8040ce450 msgr2=0x7fc8040d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:24.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.416 7fc80c144700 1 -- 172.21.2.101:0/1998199379 shutdown_connections 2021-02-10T11:46:24.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.416 7fc80c144700 1 -- 172.21.2.101:0/1998199379 wait complete. 2021-02-10T11:46:24.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.430 7fc80c144700 1 Processor -- start 2021-02-10T11:46:24.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.430 7fc80c144700 1 -- start start 2021-02-10T11:46:24.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.431 7fc80c144700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8040a5ef0 0x7fc804188f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:24.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.431 7fc80c144700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8040a62f0 0x7fc804189570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:24.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.431 7fc80c144700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc804189b90 0x7fc80408cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:24.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.431 7fc80c144700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80408c360 con 0x7fc8040a62f0 2021-02-10T11:46:24.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.431 7fc80c144700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc804189f90 con 0x7fc8040a5ef0 2021-02-10T11:46:24.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.431 7fc80c144700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80408d1b0 con 0x7fc804189b90 2021-02-10T11:46:24.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.431 7fc80975f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8040a62f0 0x7fc804189570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:24.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.431 7fc80975f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8040a62f0 0x7fc804189570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33154/0 (socket says 172.21.2.101:33154) 2021-02-10T11:46:24.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.431 7fc80975f700 1 -- 172.21.2.101:0/526413696 learned_addr learned my addr 172.21.2.101:0/526413696 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:24.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.431 7fc80975f700 1 -- 172.21.2.101:0/526413696 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc804189b90 msgr2=0x7fc80408cb90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:24.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.431 7fc80975f700 1 --2- 172.21.2.101:0/526413696 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc804189b90 0x7fc80408cb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:24.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.431 7fc80975f700 1 -- 172.21.2.101:0/526413696 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8040a5ef0 msgr2=0x7fc804188f50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:24.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.431 7fc80975f700 1 --2- 172.21.2.101:0/526413696 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8040a5ef0 0x7fc804188f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:24.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.431 7fc80975f700 1 -- 172.21.2.101:0/526413696 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80408d3d0 con 0x7fc8040a62f0 2021-02-10T11:46:24.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.431 7fc80975f700 1 --2- 172.21.2.101:0/526413696 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8040a62f0 0x7fc804189570 secure :-1 s=READY pgs=2942 cs=0 l=1 rev1=1 rx=0x7fc80000b5e0 tx=0x7fc80000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:24.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.431 7fc7f67fc700 1 -- 172.21.2.101:0/526413696 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc8000090c0 con 0x7fc8040a62f0 2021-02-10T11:46:24.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.431 7fc7f67fc700 1 -- 172.21.2.101:0/526413696 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8000009b0 con 0x7fc8040a62f0 2021-02-10T11:46:24.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.431 7fc7f67fc700 1 -- 172.21.2.101:0/526413696 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc8000022a0 con 0x7fc8040a62f0 2021-02-10T11:46:24.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.432 7fc80c144700 1 -- 172.21.2.101:0/526413696 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc80408d3d0 con 0x7fc8040a62f0 2021-02-10T11:46:24.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.432 7fc80c144700 1 -- 172.21.2.101:0/526413696 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80408d3d0 con 0x7fc8040a62f0 2021-02-10T11:46:24.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.432 7fc7f67fc700 1 -- 172.21.2.101:0/526413696 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc800000c90 con 0x7fc8040a62f0 2021-02-10T11:46:24.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.432 7fc7f67fc700 1 --2- 172.21.2.101:0/526413696 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7ec0186c0 0x7fc7ec01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:24.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.432 7fc7f67fc700 1 -- 172.21.2.101:0/526413696 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5617+0+0 (secure 0 0 0) 0x7fc800032570 con 0x7fc8040a62f0 2021-02-10T11:46:24.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.432 7fc809f60700 1 --2- 172.21.2.101:0/526413696 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7ec0186c0 0x7fc7ec01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:24.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.433 7fc809f60700 1 --2- 172.21.2.101:0/526413696 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7ec0186c0 0x7fc7ec01abc0 crc :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:24.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.464 7fc80c144700 1 -- 172.21.2.101:0/526413696 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8040ee0b0 con 0x7fc8040a62f0 2021-02-10T11:46:24.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.467 7fc7f67fc700 1 -- 172.21.2.101:0/526413696 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc8000244e0 con 0x7fc8040a62f0 2021-02-10T11:46:24.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.596 7fc80c144700 1 -- 172.21.2.101:0/526413696 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc8040ce450 con 0x7fc8040a62f0 2021-02-10T11:46:24.631 INFO:tasks.ceph.mon.a.gibba001.stderr:2021-02-10 11:46:24.634 7f974d04d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-02-10T11:46:24.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.640 7fc7f67fc700 1 -- 172.21.2.101:0/526413696 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v141) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc8000244e0 con 0x7fc8040a62f0 2021-02-10T11:46:24.657 INFO:teuthology.orchestra.run.gibba001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-02-10T11:46:24.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.662 7fc80c144700 1 -- 172.21.2.101:0/526413696 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7ec0186c0 msgr2=0x7fc7ec01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.662 7fc80c144700 1 --2- 172.21.2.101:0/526413696 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7ec0186c0 0x7fc7ec01abc0 crc :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.662 7fc80c144700 1 -- 172.21.2.101:0/526413696 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8040a62f0 msgr2=0x7fc804189570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.662 7fc80c144700 1 --2- 172.21.2.101:0/526413696 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8040a62f0 0x7fc804189570 secure :-1 s=READY pgs=2942 cs=0 l=1 rev1=1 rx=0x7fc80000b5e0 tx=0x7fc80000b980).stop 2021-02-10T11:46:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.662 7fc80c144700 1 -- 172.21.2.101:0/526413696 shutdown_connections 2021-02-10T11:46:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.662 7fc80c144700 1 --2- 172.21.2.101:0/526413696 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc7ec0186c0 0x7fc7ec01abc0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.662 7fc80c144700 1 --2- 172.21.2.101:0/526413696 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc804189b90 0x7fc80408cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.662 7fc80c144700 1 --2- 172.21.2.101:0/526413696 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc8040a62f0 0x7fc804189570 unknown :-1 s=CLOSED pgs=2942 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.662 7fc80c144700 1 --2- 172.21.2.101:0/526413696 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc8040a5ef0 0x7fc804188f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.662 7fc80c144700 1 -- 172.21.2.101:0/526413696 >> 172.21.2.101:0/526413696 conn(0x7fc8040a6eb0 msgr2=0x7fc804098ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.662 7fc80c144700 1 -- 172.21.2.101:0/526413696 shutdown_connections 2021-02-10T11:46:24.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:24.662 7fc80c144700 1 -- 172.21.2.101:0/526413696 wait complete. 2021-02-10T11:46:24.668 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:46:24.688 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:46:25.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.084 7f4a9267a700 1 Processor -- start 2021-02-10T11:46:25.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.084 7f4a9267a700 1 -- start start 2021-02-10T11:46:25.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.084 7f4a9267a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f4a8c0d1870 0x7f4a8c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:25.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.084 7f4a9267a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4a8c1b0a10 0x7f4a8c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:25.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.084 7f4a9267a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a8c0f0b10 con 0x7f4a8c0d1c70 2021-02-10T11:46:25.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.084 7f4a9267a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a8c0e5ed0 con 0x7f4a8c0d1870 2021-02-10T11:46:25.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.084 7f4a9267a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c0f0650 con 0x7f4a8c1b0a10 2021-02-10T11:46:25.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.085 7f4a90c97700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f4a8c0d1c70 0x7f4a8c0cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33654/0 (socket says 172.21.2.101:33654) 2021-02-10T11:46:25.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.085 7f4a90c97700 1 -- 172.21.2.101:0/3093603171 learned_addr learned my addr 172.21.2.101:0/3093603171 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:25.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.085 7f4a8bfff700 1 --2- 172.21.2.101:0/3093603171 >> v2:172.21.2.101:3300/0 conn(0x7f4a8c0d1870 0x7f4a8c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:25.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.085 7f4a8affd700 1 -- 172.21.2.101:0/3093603171 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3114627894 0 0) 0x7f4a84001500 con 0x7f4a8c0d1c70 2021-02-10T11:46:25.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.085 7f4a8affd700 1 -- 172.21.2.101:0/3093603171 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a74002fe0 con 0x7f4a8c0d1c70 2021-02-10T11:46:25.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.085 7f4a8bfff700 1 -- 172.21.2.101:0/3093603171 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4a8c1b0a10 msgr2=0x7f4a8c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:25.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.085 7f4a8bfff700 1 --2- 172.21.2.101:0/3093603171 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4a8c1b0a10 0x7f4a8c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:25.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.085 7f4a8bfff700 1 -- 172.21.2.101:0/3093603171 >> v1:172.21.2.101:6789/0 conn(0x7f4a8c0d1c70 legacy=0x7f4a8c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:25.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.085 7f4a8bfff700 1 -- 172.21.2.101:0/3093603171 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c0f0870 con 0x7f4a8c0d1870 2021-02-10T11:46:25.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.085 7f4a8bfff700 1 --2- 172.21.2.101:0/3093603171 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a8c0d1870 0x7f4a8c1b03f0 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 rx=0x7f4a7c006650 tx=0x7f4a7c009a80).ready entity=mon.0 client_cookie=5ad25fd6e7dc1ce8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:25.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.086 7f4a8affd700 1 -- 172.21.2.101:0/3093603171 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4a7c00c0c0 con 0x7f4a8c0d1870 2021-02-10T11:46:25.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.086 7f4a8affd700 1 -- 172.21.2.101:0/3093603171 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7c00c440 con 0x7f4a8c0d1870 2021-02-10T11:46:25.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.086 7f4a8affd700 1 -- 172.21.2.101:0/3093603171 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4a7c00c960 con 0x7f4a8c0d1870 2021-02-10T11:46:25.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.086 7f4a9267a700 1 -- 172.21.2.101:0/3093603171 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a8c0d1870 msgr2=0x7f4a8c1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:25.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.086 7f4a9267a700 1 --2- 172.21.2.101:0/3093603171 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a8c0d1870 0x7f4a8c1b03f0 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 rx=0x7f4a7c006650 tx=0x7f4a7c009a80).stop 2021-02-10T11:46:25.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.086 7f4a9267a700 1 -- 172.21.2.101:0/3093603171 shutdown_connections 2021-02-10T11:46:25.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.086 7f4a9267a700 1 --2- 172.21.2.101:0/3093603171 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4a8c1b0a10 0x7f4a8c0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:25.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.086 7f4a9267a700 1 --2- 172.21.2.101:0/3093603171 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a8c0d1870 0x7f4a8c1b03f0 unknown :-1 s=CLOSED pgs=2944 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:25.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.086 7f4a9267a700 1 -- 172.21.2.101:0/3093603171 >> 172.21.2.101:0/3093603171 conn(0x7f4a8c0ce450 msgr2=0x7f4a8c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:25.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.086 7f4a9267a700 1 -- 172.21.2.101:0/3093603171 shutdown_connections 2021-02-10T11:46:25.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.086 7f4a9267a700 1 -- 172.21.2.101:0/3093603171 wait complete. 2021-02-10T11:46:25.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.101 7f4a9267a700 1 Processor -- start 2021-02-10T11:46:25.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.101 7f4a9267a700 1 -- start start 2021-02-10T11:46:25.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.101 7f4a9267a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a8c195650 0x7f4a8c0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:25.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.102 7f4a9267a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4a8c0b5b80 0x7f4a8c1a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:25.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.102 7f4a9267a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4a8c0acfa0 0x7f4a8c1a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:25.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.102 7f4a9267a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c0b6010 con 0x7f4a8c195650 2021-02-10T11:46:25.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.102 7f4a9267a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c1a3a60 con 0x7f4a8c0b5b80 2021-02-10T11:46:25.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.102 7f4a9267a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c1a3c40 con 0x7f4a8c0acfa0 2021-02-10T11:46:25.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.102 7f4a8bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a8c195650 0x7f4a8c0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:25.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.102 7f4a8bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a8c195650 0x7f4a8c0b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33164/0 (socket says 172.21.2.101:33164) 2021-02-10T11:46:25.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.102 7f4a8bfff700 1 -- 172.21.2.101:0/501221698 learned_addr learned my addr 172.21.2.101:0/501221698 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:25.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.102 7f4a90c97700 1 --2- 172.21.2.101:0/501221698 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4a8c0acfa0 0x7f4a8c1a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:25.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.102 7f4a8b7fe700 1 --2- 172.21.2.101:0/501221698 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4a8c0b5b80 0x7f4a8c1a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:25.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.102 7f4a8bfff700 1 -- 172.21.2.101:0/501221698 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4a8c0acfa0 msgr2=0x7f4a8c1a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:25.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.102 7f4a8bfff700 1 --2- 172.21.2.101:0/501221698 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4a8c0acfa0 0x7f4a8c1a3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:25.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.102 7f4a8bfff700 1 -- 172.21.2.101:0/501221698 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4a8c0b5b80 msgr2=0x7f4a8c1a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:25.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.102 7f4a8bfff700 1 --2- 172.21.2.101:0/501221698 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4a8c0b5b80 0x7f4a8c1a2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:25.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.102 7f4a8bfff700 1 -- 172.21.2.101:0/501221698 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c0bcae0 con 0x7f4a8c195650 2021-02-10T11:46:25.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.102 7f4a8b7fe700 1 --2- 172.21.2.101:0/501221698 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4a8c0b5b80 0x7f4a8c1a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:46:25.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.102 7f4a8bfff700 1 --2- 172.21.2.101:0/501221698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a8c195650 0x7f4a8c0b5560 secure :-1 s=READY pgs=2945 cs=0 l=1 rev1=1 rx=0x7f4a7c00b580 tx=0x7f4a7c00b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:25.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.102 7f4a88ff9700 1 -- 172.21.2.101:0/501221698 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4a7c000970 con 0x7f4a8c195650 2021-02-10T11:46:25.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.103 7f4a88ff9700 1 -- 172.21.2.101:0/501221698 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7c00bd10 con 0x7f4a8c195650 2021-02-10T11:46:25.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.103 7f4a88ff9700 1 -- 172.21.2.101:0/501221698 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4a7c006220 con 0x7f4a8c195650 2021-02-10T11:46:25.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.103 7f4a9267a700 1 -- 172.21.2.101:0/501221698 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a8c0bcae0 con 0x7f4a8c195650 2021-02-10T11:46:25.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.103 7f4a9267a700 1 -- 172.21.2.101:0/501221698 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a8c0bcae0 con 0x7f4a8c195650 2021-02-10T11:46:25.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.103 7f4a88ff9700 1 -- 172.21.2.101:0/501221698 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4a7c006030 con 0x7f4a8c195650 2021-02-10T11:46:25.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.103 7f4a88ff9700 1 --2- 172.21.2.101:0/501221698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4a6c0186c0 0x7f4a6c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:25.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.103 7f4a88ff9700 1 -- 172.21.2.101:0/501221698 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5675+0+0 (secure 0 0 0) 0x7f4a7c032830 con 0x7f4a8c195650 2021-02-10T11:46:25.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.103 7f4a8b7fe700 1 --2- 172.21.2.101:0/501221698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4a6c0186c0 0x7f4a6c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:25.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.104 7f4a8b7fe700 1 --2- 172.21.2.101:0/501221698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4a6c0186c0 0x7f4a6c01abc0 crc :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:25.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.135 7f4a9267a700 1 -- 172.21.2.101:0/501221698 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a8c0b6010 con 0x7f4a8c195650 2021-02-10T11:46:25.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.137 7f4a88ff9700 1 -- 172.21.2.101:0/501221698 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4a7c000970 con 0x7f4a8c195650 2021-02-10T11:46:25.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.267 7f4a9267a700 1 -- 172.21.2.101:0/501221698 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a8c0b6010 con 0x7f4a8c195650 2021-02-10T11:46:25.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.267 7f4a88ff9700 1 -- 172.21.2.101:0/501221698 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+1759 (secure 0 0 0) 0x7f4a7c006030 con 0x7f4a8c195650 2021-02-10T11:46:25.265 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:46:25.266 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":141,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8371,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/453344112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":453344112},{"type":"v1","addr":"172.21.2.101:6835","nonce":453344112}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":135}],"filesystems":[{"mdsmap":{"epoch":141,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:45:44.128996","modified":"2021-02-10 11:46:24.634952","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":167,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":17}]} 2021-02-10T11:46:25.266 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 141 2021-02-10T11:46:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.269 7f4a9267a700 1 -- 172.21.2.101:0/501221698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4a6c0186c0 msgr2=0x7f4a6c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.269 7f4a9267a700 1 --2- 172.21.2.101:0/501221698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4a6c0186c0 0x7f4a6c01abc0 crc :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.269 7f4a9267a700 1 -- 172.21.2.101:0/501221698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a8c195650 msgr2=0x7f4a8c0b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.269 7f4a9267a700 1 --2- 172.21.2.101:0/501221698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a8c195650 0x7f4a8c0b5560 secure :-1 s=READY pgs=2945 cs=0 l=1 rev1=1 rx=0x7f4a7c00b580 tx=0x7f4a7c00b920).stop 2021-02-10T11:46:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.269 7f4a9267a700 1 -- 172.21.2.101:0/501221698 shutdown_connections 2021-02-10T11:46:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.269 7f4a9267a700 1 --2- 172.21.2.101:0/501221698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4a6c0186c0 0x7f4a6c01abc0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.269 7f4a9267a700 1 --2- 172.21.2.101:0/501221698 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4a8c0acfa0 0x7f4a8c1a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.269 7f4a9267a700 1 --2- 172.21.2.101:0/501221698 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4a8c0b5b80 0x7f4a8c1a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.269 7f4a9267a700 1 --2- 172.21.2.101:0/501221698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a8c195650 0x7f4a8c0b5560 unknown :-1 s=CLOSED pgs=2945 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.269 7f4a9267a700 1 -- 172.21.2.101:0/501221698 >> 172.21.2.101:0/501221698 conn(0x7f4a8c0b3040 msgr2=0x7f4a8c0c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.270 7f4a9267a700 1 -- 172.21.2.101:0/501221698 shutdown_connections 2021-02-10T11:46:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.270 7f4a9267a700 1 -- 172.21.2.101:0/501221698 wait complete. 2021-02-10T11:46:25.276 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-02-10T11:46:25.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.695 7f0bf32d1700 1 Processor -- start 2021-02-10T11:46:25.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.695 7f0bf32d1700 1 -- start start 2021-02-10T11:46:25.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.696 7f0bf32d1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0bec0d1870 0x7f0bec1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:25.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.696 7f0bf32d1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0bec1b0a10 0x7f0bec0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:25.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.696 7f0bf32d1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0bec0d1c70 0x7f0bec0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:25.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.696 7f0bf32d1700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bec0fe200 con 0x7f0bec0d1870 2021-02-10T11:46:25.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.696 7f0bf32d1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bec0e5f10 con 0x7f0bec0d1c70 2021-02-10T11:46:25.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.696 7f0bf32d1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec0f0b50 con 0x7f0bec1b0a10 2021-02-10T11:46:25.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.696 7f0bf10ed700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0bec0d1870 0x7f0bec1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:25.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.696 7f0bf10ed700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0bec0d1870 0x7f0bec1b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33172/0 (socket says 172.21.2.101:33172) 2021-02-10T11:46:25.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.696 7f0bf10ed700 1 -- 172.21.2.101:0/1990641678 learned_addr learned my addr 172.21.2.101:0/1990641678 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:25.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.696 7f0bf10ed700 1 -- 172.21.2.101:0/1990641678 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0bec1b0a10 msgr2=0x7f0bec0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:25.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.696 7f0bf10ed700 1 --2- 172.21.2.101:0/1990641678 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0bec1b0a10 0x7f0bec0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:25.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.696 7f0bf10ed700 1 -- 172.21.2.101:0/1990641678 >> v2:172.21.2.108:3300/0 conn(0x7f0bec0d1c70 msgr2=0x7f0bec0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:25.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.696 7f0bf10ed700 1 --2- 172.21.2.101:0/1990641678 >> v2:172.21.2.108:3300/0 conn(0x7f0bec0d1c70 0x7f0bec0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:25.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.696 7f0bf10ed700 1 -- 172.21.2.101:0/1990641678 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec0cff70 con 0x7f0bec0d1870 2021-02-10T11:46:25.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.697 7f0bf10ed700 1 --2- 172.21.2.101:0/1990641678 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0bec0d1870 0x7f0bec1b03f0 secure :-1 s=READY pgs=2946 cs=0 l=1 rev1=1 rx=0x7f0be8006700 tx=0x7f0be8009b80).ready entity=mon.0 client_cookie=e4b1f2fc2acaed90 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:25.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.697 7f0be3fff700 1 -- 172.21.2.101:0/1990641678 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0be800c0c0 con 0x7f0bec0d1870 2021-02-10T11:46:25.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.697 7f0be3fff700 1 -- 172.21.2.101:0/1990641678 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be800c440 con 0x7f0bec0d1870 2021-02-10T11:46:25.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.697 7f0be3fff700 1 -- 172.21.2.101:0/1990641678 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0be800c960 con 0x7f0bec0d1870 2021-02-10T11:46:25.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.697 7f0bf32d1700 1 -- 172.21.2.101:0/1990641678 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0bec0d1870 msgr2=0x7f0bec1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:25.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.697 7f0bf32d1700 1 --2- 172.21.2.101:0/1990641678 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0bec0d1870 0x7f0bec1b03f0 secure :-1 s=READY pgs=2946 cs=0 l=1 rev1=1 rx=0x7f0be8006700 tx=0x7f0be8009b80).stop 2021-02-10T11:46:25.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.697 7f0bf32d1700 1 -- 172.21.2.101:0/1990641678 shutdown_connections 2021-02-10T11:46:25.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.698 7f0bf32d1700 1 --2- 172.21.2.101:0/1990641678 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0bec1b0a10 0x7f0bec0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:25.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.698 7f0bf32d1700 1 --2- 172.21.2.101:0/1990641678 >> v2:172.21.2.108:3300/0 conn(0x7f0bec0d1c70 0x7f0bec0cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:25.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.698 7f0bf32d1700 1 --2- 172.21.2.101:0/1990641678 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0bec0d1870 0x7f0bec1b03f0 unknown :-1 s=CLOSED pgs=2946 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:25.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.698 7f0bf32d1700 1 -- 172.21.2.101:0/1990641678 >> 172.21.2.101:0/1990641678 conn(0x7f0bec0ce450 msgr2=0x7f0bec0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:25.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.698 7f0bf32d1700 1 -- 172.21.2.101:0/1990641678 shutdown_connections 2021-02-10T11:46:25.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.698 7f0bf32d1700 1 -- 172.21.2.101:0/1990641678 wait complete. 2021-02-10T11:46:25.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.712 7f0bf32d1700 1 Processor -- start 2021-02-10T11:46:25.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.712 7f0bf32d1700 1 -- start start 2021-02-10T11:46:25.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.713 7f0bf32d1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0bec1908b0 0x7f0bec1a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:25.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.713 7f0bf32d1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0bec0bd920 0x7f0bec0bdd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:25.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.713 7f0bf32d1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0bec08d510 0x7f0bec08d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:25.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.713 7f0bf32d1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec08cf90 con 0x7f0bec1908b0 2021-02-10T11:46:25.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.713 7f0bf32d1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec0be340 con 0x7f0bec08d510 2021-02-10T11:46:25.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.713 7f0bf32d1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec0be680 con 0x7f0bec0bd920 2021-02-10T11:46:25.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.713 7f0bf10ed700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0bec1908b0 0x7f0bec1a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:25.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.713 7f0bf10ed700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0bec1908b0 0x7f0bec1a4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33178/0 (socket says 172.21.2.101:33178) 2021-02-10T11:46:25.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.713 7f0bf10ed700 1 -- 172.21.2.101:0/4131755037 learned_addr learned my addr 172.21.2.101:0/4131755037 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:25.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.713 7f0bf10ed700 1 -- 172.21.2.101:0/4131755037 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0bec0bd920 msgr2=0x7f0bec0bdd20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:25.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.713 7f0bf10ed700 1 --2- 172.21.2.101:0/4131755037 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0bec0bd920 0x7f0bec0bdd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:25.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.713 7f0bf10ed700 1 -- 172.21.2.101:0/4131755037 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0bec08d510 msgr2=0x7f0bec08d910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:25.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.713 7f0bf10ed700 1 --2- 172.21.2.101:0/4131755037 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0bec08d510 0x7f0bec08d910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:25.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.713 7f0bf10ed700 1 -- 172.21.2.101:0/4131755037 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec1a4a60 con 0x7f0bec1908b0 2021-02-10T11:46:25.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.713 7f0bf10ed700 1 --2- 172.21.2.101:0/4131755037 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0bec1908b0 0x7f0bec1a4440 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 rx=0x7f0be800b5e0 tx=0x7f0be800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:25.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.714 7f0be1ffb700 1 -- 172.21.2.101:0/4131755037 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0be800bd70 con 0x7f0bec1908b0 2021-02-10T11:46:25.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.714 7f0be1ffb700 1 -- 172.21.2.101:0/4131755037 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be8002030 con 0x7f0bec1908b0 2021-02-10T11:46:25.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.714 7f0be1ffb700 1 -- 172.21.2.101:0/4131755037 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0be8002550 con 0x7f0bec1908b0 2021-02-10T11:46:25.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.714 7f0bf32d1700 1 -- 172.21.2.101:0/4131755037 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bec1a4a60 con 0x7f0bec1908b0 2021-02-10T11:46:25.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.714 7f0bf32d1700 1 -- 172.21.2.101:0/4131755037 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bec08df30 con 0x7f0bec1908b0 2021-02-10T11:46:25.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.714 7f0be1ffb700 1 -- 172.21.2.101:0/4131755037 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0be8000c90 con 0x7f0bec1908b0 2021-02-10T11:46:25.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.714 7f0be1ffb700 1 --2- 172.21.2.101:0/4131755037 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0bcc0186c0 0x7f0bcc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:25.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.714 7f0be1ffb700 1 -- 172.21.2.101:0/4131755037 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5675+0+0 (secure 0 0 0) 0x7f0be8032630 con 0x7f0bec1908b0 2021-02-10T11:46:25.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.715 7f0bf08ec700 1 --2- 172.21.2.101:0/4131755037 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0bcc0186c0 0x7f0bcc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:25.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.715 7f0bf08ec700 1 --2- 172.21.2.101:0/4131755037 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0bcc0186c0 0x7f0bcc01abc0 crc :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:25.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.746 7f0bf32d1700 1 -- 172.21.2.101:0/4131755037 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bec0be680 con 0x7f0bec1908b0 2021-02-10T11:46:25.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.749 7f0be1ffb700 1 -- 172.21.2.101:0/4131755037 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0be8030d80 con 0x7f0bec1908b0 2021-02-10T11:46:25.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:25.876 7f0bf32d1700 1 -- 172.21.2.101:0/4131755037 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0bec0be680 con 0x7f0bec1908b0 2021-02-10T11:46:26.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:26.645 7f0be1ffb700 1 -- 172.21.2.101:0/4131755037 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v142) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0be8000c90 con 0x7f0bec1908b0 2021-02-10T11:46:26.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:26.647 7f0bf32d1700 1 -- 172.21.2.101:0/4131755037 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0bcc0186c0 msgr2=0x7f0bcc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:26.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:26.647 7f0bf32d1700 1 --2- 172.21.2.101:0/4131755037 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0bcc0186c0 0x7f0bcc01abc0 crc :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:26.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:26.648 7f0bf32d1700 1 -- 172.21.2.101:0/4131755037 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0bec1908b0 msgr2=0x7f0bec1a4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:26.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:26.648 7f0bf32d1700 1 --2- 172.21.2.101:0/4131755037 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0bec1908b0 0x7f0bec1a4440 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 rx=0x7f0be800b5e0 tx=0x7f0be800b980).stop 2021-02-10T11:46:26.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:26.648 7f0bf32d1700 1 -- 172.21.2.101:0/4131755037 shutdown_connections 2021-02-10T11:46:26.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:26.648 7f0bf32d1700 1 --2- 172.21.2.101:0/4131755037 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0bcc0186c0 0x7f0bcc01abc0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:26.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:26.648 7f0bf32d1700 1 --2- 172.21.2.101:0/4131755037 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0bec0bd920 0x7f0bec0bdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:26.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:26.648 7f0bf32d1700 1 --2- 172.21.2.101:0/4131755037 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0bec08d510 0x7f0bec08d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:26.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:26.648 7f0bf32d1700 1 --2- 172.21.2.101:0/4131755037 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0bec1908b0 0x7f0bec1a4440 unknown :-1 s=CLOSED pgs=2947 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:26.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:26.648 7f0bf32d1700 1 -- 172.21.2.101:0/4131755037 >> 172.21.2.101:0/4131755037 conn(0x7f0bec08f030 msgr2=0x7f0bec0b5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:26.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:26.648 7f0bf32d1700 1 -- 172.21.2.101:0/4131755037 shutdown_connections 2021-02-10T11:46:26.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:26.648 7f0bf32d1700 1 -- 172.21.2.101:0/4131755037 wait complete. 2021-02-10T11:46:26.653 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-02-10T11:46:27.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.073 7fd3bccdd700 1 Processor -- start 2021-02-10T11:46:27.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.073 7fd3bccdd700 1 -- start start 2021-02-10T11:46:27.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.073 7fd3bccdd700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd3b80d1c80 0x7fd3b80cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:27.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.073 7fd3bccdd700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3b80d0180 con 0x7fd3b80d1880 2021-02-10T11:46:27.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.073 7fd3bccdd700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3b80fe1d0 con 0x7fd3b80cf590 2021-02-10T11:46:27.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.073 7fd3bccdd700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3b81b0cc0 con 0x7fd3b80d1c80 2021-02-10T11:46:27.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.074 7fd3b6ae8700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd3b80d1880 0x7fd3b81b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33676/0 (socket says 172.21.2.101:33676) 2021-02-10T11:46:27.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.074 7fd3b6ae8700 1 -- 172.21.2.101:0/2910555631 learned_addr learned my addr 172.21.2.101:0/2910555631 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:27.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.074 7fd3b5ae6700 1 -- 172.21.2.101:0/2910555631 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 357613227 0 0) 0x7fd3a8001180 con 0x7fd3b80d1880 2021-02-10T11:46:27.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.074 7fd3b5ae6700 1 -- 172.21.2.101:0/2910555631 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3a0002fe0 con 0x7fd3b80d1880 2021-02-10T11:46:27.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.074 7fd3b62e7700 1 --2- 172.21.2.101:0/2910555631 >> v2:172.21.2.108:3300/0 conn(0x7fd3b80d1c80 0x7fd3b80cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:27.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.074 7fd3b5ae6700 1 -- 172.21.2.101:0/2910555631 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 412992266 0 0) 0x7fd3a8001180 con 0x7fd3b80d1880 2021-02-10T11:46:27.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.074 7fd3b5ae6700 1 -- 172.21.2.101:0/2910555631 >> v2:172.21.2.108:3300/0 conn(0x7fd3b80d1c80 msgr2=0x7fd3b80cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:27.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.074 7fd3b5ae6700 1 --2- 172.21.2.101:0/2910555631 >> v2:172.21.2.108:3300/0 conn(0x7fd3b80d1c80 0x7fd3b80cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:27.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.074 7fd3b5ae6700 1 -- 172.21.2.101:0/2910555631 >> v1:172.21.2.108:6789/0 conn(0x7fd3b80cf590 legacy=0x7fd3b80cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:27.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.074 7fd3b5ae6700 1 -- 172.21.2.101:0/2910555631 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b8109a70 con 0x7fd3b80d1880 2021-02-10T11:46:27.089 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.074 7fd3b5ae6700 1 -- 172.21.2.101:0/2910555631 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fd3a8001680 con 0x7fd3b80d1880 2021-02-10T11:46:27.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.074 7fd3b5ae6700 1 -- 172.21.2.101:0/2910555631 >> v1:172.21.2.101:6789/0 conn(0x7fd3b80d1880 legacy=0x7fd3b81b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:27.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.074 7fd3b5ae6700 1 --2- 172.21.2.101:0/2910555631 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3a0003b10 0x7fd3a0006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:27.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.074 7fd3b5ae6700 1 --2- 172.21.2.101:0/2910555631 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3a00067c0 0x7fd3a0008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:27.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.074 7fd3b5ae6700 1 --2- 172.21.2.101:0/2910555631 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd3a0009580 0x7fd3a000baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:27.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.074 7fd3b5ae6700 1 -- 172.21.2.101:0/2910555631 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a000c2f0 con 0x7fd3a0003b10 2021-02-10T11:46:27.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.074 7fd3b5ae6700 1 -- 172.21.2.101:0/2910555631 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a000c510 con 0x7fd3a0009580 2021-02-10T11:46:27.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.074 7fd3b5ae6700 1 -- 172.21.2.101:0/2910555631 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a000c730 con 0x7fd3a00067c0 2021-02-10T11:46:27.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.074 7fd3b6ae8700 1 --2- 172.21.2.101:0/2910555631 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3a0003b10 0x7fd3a0006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:27.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.075 7fd3b62e7700 1 --2- 172.21.2.101:0/2910555631 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3a00067c0 0x7fd3a0008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:27.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.075 7fd3b6ae8700 1 -- 172.21.2.101:0/2910555631 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3a00067c0 msgr2=0x7fd3a0008c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:27.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.075 7fd3b6ae8700 1 --2- 172.21.2.101:0/2910555631 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3a00067c0 0x7fd3a0008c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:27.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.075 7fd3b6ae8700 1 -- 172.21.2.101:0/2910555631 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd3a0009580 msgr2=0x7fd3a000baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:27.090 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.075 7fd3b72e9700 1 --2- 172.21.2.101:0/2910555631 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd3a0009580 0x7fd3a000baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:27.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.075 7fd3b6ae8700 1 --2- 172.21.2.101:0/2910555631 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd3a0009580 0x7fd3a000baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:27.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.075 7fd3b6ae8700 1 -- 172.21.2.101:0/2910555631 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a00509a0 con 0x7fd3a0003b10 2021-02-10T11:46:27.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.075 7fd3b72e9700 1 --2- 172.21.2.101:0/2910555631 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd3a0009580 0x7fd3a000baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:46:27.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.075 7fd3b72e9700 1 -- 172.21.2.101:0/2910555631 reap_dead start 2021-02-10T11:46:27.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.075 7fd3b6ae8700 1 --2- 172.21.2.101:0/2910555631 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3a0003b10 0x7fd3a0006030 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 rx=0x7fd3a80018a0 tx=0x7fd3a8007cc0).ready entity=mon.0 client_cookie=dc6456d70d769ef server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:27.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.075 7fd3b5ae6700 1 -- 172.21.2.101:0/2910555631 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd3a80070c0 con 0x7fd3a0003b10 2021-02-10T11:46:27.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.075 7fd3b5ae6700 1 -- 172.21.2.101:0/2910555631 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a80010e0 con 0x7fd3a0003b10 2021-02-10T11:46:27.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.075 7fd3b5ae6700 1 -- 172.21.2.101:0/2910555631 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd3a800a710 con 0x7fd3a0003b10 2021-02-10T11:46:27.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.075 7fd3bccdd700 1 -- 172.21.2.101:0/2910555631 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3a0003b10 msgr2=0x7fd3a0006030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:27.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.075 7fd3bccdd700 1 --2- 172.21.2.101:0/2910555631 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3a0003b10 0x7fd3a0006030 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 rx=0x7fd3a80018a0 tx=0x7fd3a8007cc0).stop 2021-02-10T11:46:27.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.076 7fd3bccdd700 1 -- 172.21.2.101:0/2910555631 shutdown_connections 2021-02-10T11:46:27.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.076 7fd3bccdd700 1 --2- 172.21.2.101:0/2910555631 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3a0003b10 0x7fd3a0006030 unknown :-1 s=CLOSED pgs=2949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:27.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.076 7fd3bccdd700 1 -- 172.21.2.101:0/2910555631 >> 172.21.2.101:0/2910555631 conn(0x7fd3b80ce430 msgr2=0x7fd3b80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:27.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.076 7fd3bccdd700 1 -- 172.21.2.101:0/2910555631 shutdown_connections 2021-02-10T11:46:27.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.076 7fd3bccdd700 1 -- 172.21.2.101:0/2910555631 wait complete. 2021-02-10T11:46:27.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.091 7fd3bccdd700 1 Processor -- start 2021-02-10T11:46:27.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.091 7fd3bccdd700 1 -- start start 2021-02-10T11:46:27.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.091 7fd3bccdd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3b81a7a70 0x7fd3b81a7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:27.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.091 7fd3bccdd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3b80bf730 0x7fd3b80bfb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.091 7fd3bccdd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd3b80c0150 0x7fd3b80909c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.091 7fd3bccdd700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b8090fe0 con 0x7fd3b81a7a70 2021-02-10T11:46:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.091 7fd3bccdd700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b8091200 con 0x7fd3b80c0150 2021-02-10T11:46:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.091 7fd3bccdd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b8091420 con 0x7fd3b80bf730 2021-02-10T11:46:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.091 7fd3b6ae8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3b81a7a70 0x7fd3b81a7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.091 7fd3b6ae8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3b81a7a70 0x7fd3b81a7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33194/0 (socket says 172.21.2.101:33194) 2021-02-10T11:46:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.091 7fd3b6ae8700 1 -- 172.21.2.101:0/1706586594 learned_addr learned my addr 172.21.2.101:0/1706586594 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.092 7fd3b6ae8700 1 -- 172.21.2.101:0/1706586594 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3b80bf730 msgr2=0x7fd3b80bfb30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.092 7fd3b6ae8700 1 --2- 172.21.2.101:0/1706586594 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3b80bf730 0x7fd3b80bfb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.092 7fd3b6ae8700 1 -- 172.21.2.101:0/1706586594 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd3b80c0150 msgr2=0x7fd3b80909c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.092 7fd3b6ae8700 1 --2- 172.21.2.101:0/1706586594 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd3b80c0150 0x7fd3b80909c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.092 7fd3b6ae8700 1 -- 172.21.2.101:0/1706586594 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b8091640 con 0x7fd3b81a7a70 2021-02-10T11:46:27.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.092 7fd3b6ae8700 1 --2- 172.21.2.101:0/1706586594 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3b81a7a70 0x7fd3b81a7e70 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 rx=0x7fd3a8001380 tx=0x7fd3a800b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:27.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.092 7fd3a77fe700 1 -- 172.21.2.101:0/1706586594 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd3a800bd30 con 0x7fd3b81a7a70 2021-02-10T11:46:27.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.092 7fd3a77fe700 1 -- 172.21.2.101:0/1706586594 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a8001ba0 con 0x7fd3b81a7a70 2021-02-10T11:46:27.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.092 7fd3bccdd700 1 -- 172.21.2.101:0/1706586594 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b8091640 con 0x7fd3b81a7a70 2021-02-10T11:46:27.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.092 7fd3bccdd700 1 -- 172.21.2.101:0/1706586594 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b80c0e70 con 0x7fd3b81a7a70 2021-02-10T11:46:27.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.092 7fd3a77fe700 1 -- 172.21.2.101:0/1706586594 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd3a8004830 con 0x7fd3b81a7a70 2021-02-10T11:46:27.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.093 7fd3a77fe700 1 -- 172.21.2.101:0/1706586594 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd3a8000940 con 0x7fd3b81a7a70 2021-02-10T11:46:27.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.093 7fd3a77fe700 1 --2- 172.21.2.101:0/1706586594 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd39c018730 0x7fd39c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:27.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.093 7fd3a77fe700 1 -- 172.21.2.101:0/1706586594 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5675+0+0 (secure 0 0 0) 0x7fd3a8032810 con 0x7fd3b81a7a70 2021-02-10T11:46:27.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.093 7fd3b62e7700 1 --2- 172.21.2.101:0/1706586594 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd39c018730 0x7fd39c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:27.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.093 7fd3b62e7700 1 --2- 172.21.2.101:0/1706586594 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd39c018730 0x7fd39c01ac10 crc :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:27.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.123 7fd3bccdd700 1 -- 172.21.2.101:0/1706586594 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b80fe1d0 con 0x7fd3b81a7a70 2021-02-10T11:46:27.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.126 7fd3a77fe700 1 -- 172.21.2.101:0/1706586594 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd3a8001610 con 0x7fd3b81a7a70 2021-02-10T11:46:27.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.253 7fd3bccdd700 1 -- 172.21.2.101:0/1706586594 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd3b80fe1d0 con 0x7fd3b81a7a70 2021-02-10T11:46:27.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:27.994 7fd3a77fe700 1 -- 172.21.2.101:0/1706586594 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v168) v1 ==== 185+0+0 (secure 0 0 0) 0x7fd3a8000940 con 0x7fd3b81a7a70 2021-02-10T11:46:28.019 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' removed 2021-02-10T11:46:28.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.023 7fd3bccdd700 1 -- 172.21.2.101:0/1706586594 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd39c018730 msgr2=0x7fd39c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:28.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.023 7fd3bccdd700 1 --2- 172.21.2.101:0/1706586594 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd39c018730 0x7fd39c01ac10 crc :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:28.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.023 7fd3bccdd700 1 -- 172.21.2.101:0/1706586594 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3b81a7a70 msgr2=0x7fd3b81a7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:28.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.023 7fd3bccdd700 1 --2- 172.21.2.101:0/1706586594 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3b81a7a70 0x7fd3b81a7e70 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 rx=0x7fd3a8001380 tx=0x7fd3a800b940).stop 2021-02-10T11:46:28.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.023 7fd3bccdd700 1 -- 172.21.2.101:0/1706586594 shutdown_connections 2021-02-10T11:46:28.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.023 7fd3bccdd700 1 --2- 172.21.2.101:0/1706586594 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd39c018730 0x7fd39c01ac10 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:28.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.023 7fd3bccdd700 1 --2- 172.21.2.101:0/1706586594 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3b80bf730 0x7fd3b80bfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:28.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.023 7fd3bccdd700 1 --2- 172.21.2.101:0/1706586594 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd3b80c0150 0x7fd3b80909c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:28.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.023 7fd3bccdd700 1 --2- 172.21.2.101:0/1706586594 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3b81a7a70 0x7fd3b81a7e70 unknown :-1 s=CLOSED pgs=2950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:28.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.023 7fd3bccdd700 1 -- 172.21.2.101:0/1706586594 >> 172.21.2.101:0/1706586594 conn(0x7fd3b80a69d0 msgr2=0x7fd3b80b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:28.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.023 7fd3bccdd700 1 -- 172.21.2.101:0/1706586594 shutdown_connections 2021-02-10T11:46:28.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.023 7fd3bccdd700 1 -- 172.21.2.101:0/1706586594 wait complete. 2021-02-10T11:46:28.029 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-02-10T11:46:28.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.447 7fd307ec0700 1 Processor -- start 2021-02-10T11:46:28.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.448 7fd307ec0700 1 -- start start 2021-02-10T11:46:28.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.448 7fd307ec0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd3000d1c70 0x7fd3000cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:28.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.448 7fd307ec0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3000cf580 0x7fd3000cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:28.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.448 7fd307ec0700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3000fe1c0 con 0x7fd3000d1870 2021-02-10T11:46:28.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.448 7fd307ec0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3001b0cb0 con 0x7fd3000d1c70 2021-02-10T11:46:28.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.448 7fd307ec0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd300109a60 con 0x7fd3000cf580 2021-02-10T11:46:28.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.448 7fd3054db700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd3000d1c70 0x7fd3000cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:28.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.448 7fd305cdc700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd3000d1870 0x7fd3001b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33694/0 (socket says 172.21.2.101:33694) 2021-02-10T11:46:28.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.448 7fd305cdc700 1 -- 172.21.2.101:0/3519685595 learned_addr learned my addr 172.21.2.101:0/3519685595 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:28.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.449 7fd3054db700 1 -- 172.21.2.101:0/3519685595 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3000cf580 msgr2=0x7fd3000cf980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:46:28.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.449 7fd3054db700 1 --2- 172.21.2.101:0/3519685595 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3000cf580 0x7fd3000cf980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:28.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.449 7fd3054db700 1 -- 172.21.2.101:0/3519685595 >> v1:172.21.2.101:6789/0 conn(0x7fd3000d1870 legacy=0x7fd3001b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:28.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.449 7fd3054db700 1 -- 172.21.2.101:0/3519685595 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd300088070 con 0x7fd3000d1c70 2021-02-10T11:46:28.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.449 7fd3054db700 1 --2- 172.21.2.101:0/3519685595 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3000d1c70 0x7fd3000cef60 secure :-1 s=READY pgs=2952 cs=0 l=1 rev1=1 rx=0x7fd2f4006650 tx=0x7fd2f4009a80).ready entity=mon.0 client_cookie=c373382ff51226d9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:28.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.449 7fd304cda700 1 -- 172.21.2.101:0/3519685595 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2f400c0c0 con 0x7fd3000d1c70 2021-02-10T11:46:28.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.449 7fd304cda700 1 -- 172.21.2.101:0/3519685595 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f400c340 con 0x7fd3000d1c70 2021-02-10T11:46:28.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.449 7fd304cda700 1 -- 172.21.2.101:0/3519685595 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2f400c860 con 0x7fd3000d1c70 2021-02-10T11:46:28.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.449 7fd307ec0700 1 -- 172.21.2.101:0/3519685595 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3000d1c70 msgr2=0x7fd3000cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:28.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.449 7fd307ec0700 1 --2- 172.21.2.101:0/3519685595 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3000d1c70 0x7fd3000cef60 secure :-1 s=READY pgs=2952 cs=0 l=1 rev1=1 rx=0x7fd2f4006650 tx=0x7fd2f4009a80).stop 2021-02-10T11:46:28.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.450 7fd307ec0700 1 -- 172.21.2.101:0/3519685595 shutdown_connections 2021-02-10T11:46:28.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.450 7fd307ec0700 1 --2- 172.21.2.101:0/3519685595 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd3000cf580 0x7fd3000cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:28.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.450 7fd307ec0700 1 --2- 172.21.2.101:0/3519685595 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3000d1c70 0x7fd3000cef60 unknown :-1 s=CLOSED pgs=2952 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:28.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.450 7fd307ec0700 1 -- 172.21.2.101:0/3519685595 >> 172.21.2.101:0/3519685595 conn(0x7fd3000ce420 msgr2=0x7fd3000d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:28.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.450 7fd307ec0700 1 -- 172.21.2.101:0/3519685595 shutdown_connections 2021-02-10T11:46:28.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.450 7fd307ec0700 1 -- 172.21.2.101:0/3519685595 wait complete. 2021-02-10T11:46:28.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.465 7fd307ec0700 1 Processor -- start 2021-02-10T11:46:28.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.465 7fd307ec0700 1 -- start start 2021-02-10T11:46:28.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.465 7fd307ec0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3001b33e0 0x7fd3001a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:28.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.465 7fd307ec0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd3000931c0 0x7fd3001b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:28.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.465 7fd307ec0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd300187e70 0x7fd300188270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:28.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.465 7fd307ec0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3001b2e60 con 0x7fd3001b33e0 2021-02-10T11:46:28.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.465 7fd307ec0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3001b37e0 con 0x7fd3000931c0 2021-02-10T11:46:28.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.465 7fd307ec0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3000935c0 con 0x7fd300187e70 2021-02-10T11:46:28.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.465 7fd305cdc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3001b33e0 0x7fd3001a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:28.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.465 7fd305cdc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3001b33e0 0x7fd3001a2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33208/0 (socket says 172.21.2.101:33208) 2021-02-10T11:46:28.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.465 7fd305cdc700 1 -- 172.21.2.101:0/3653111070 learned_addr learned my addr 172.21.2.101:0/3653111070 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:28.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.466 7fd3054db700 1 --2- 172.21.2.101:0/3653111070 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd3000931c0 0x7fd3001b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:28.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.466 7fd3064dd700 1 --2- 172.21.2.101:0/3653111070 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd300187e70 0x7fd300188270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:28.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.466 7fd305cdc700 1 -- 172.21.2.101:0/3653111070 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd300187e70 msgr2=0x7fd300188270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:28.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.466 7fd305cdc700 1 --2- 172.21.2.101:0/3653111070 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd300187e70 0x7fd300188270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:28.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.466 7fd305cdc700 1 -- 172.21.2.101:0/3653111070 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd3000931c0 msgr2=0x7fd3001b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:28.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.466 7fd305cdc700 1 --2- 172.21.2.101:0/3653111070 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd3000931c0 0x7fd3001b2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:28.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.466 7fd305cdc700 1 -- 172.21.2.101:0/3653111070 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd300188890 con 0x7fd3001b33e0 2021-02-10T11:46:28.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.466 7fd305cdc700 1 --2- 172.21.2.101:0/3653111070 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3001b33e0 0x7fd3001a2b50 secure :-1 s=READY pgs=2953 cs=0 l=1 rev1=1 rx=0x7fd2fc00a570 tx=0x7fd2fc00a910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:28.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.466 7fd2f27fc700 1 -- 172.21.2.101:0/3653111070 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2fc00ad00 con 0x7fd3001b33e0 2021-02-10T11:46:28.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.466 7fd2f27fc700 1 -- 172.21.2.101:0/3653111070 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc001180 con 0x7fd3001b33e0 2021-02-10T11:46:28.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.466 7fd307ec0700 1 -- 172.21.2.101:0/3653111070 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd300188890 con 0x7fd3001b33e0 2021-02-10T11:46:28.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.466 7fd2f27fc700 1 -- 172.21.2.101:0/3653111070 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2fc004620 con 0x7fd3001b33e0 2021-02-10T11:46:28.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.466 7fd307ec0700 1 -- 172.21.2.101:0/3653111070 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd300188890 con 0x7fd3001b33e0 2021-02-10T11:46:28.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.467 7fd2f27fc700 1 -- 172.21.2.101:0/3653111070 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd2fc0043a0 con 0x7fd3001b33e0 2021-02-10T11:46:28.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.467 7fd2f27fc700 1 --2- 172.21.2.101:0/3653111070 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2e8018730 0x7fd2e801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:28.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.467 7fd2f27fc700 1 -- 172.21.2.101:0/3653111070 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5237+0+0 (secure 0 0 0) 0x7fd2fc032630 con 0x7fd3001b33e0 2021-02-10T11:46:28.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.467 7fd3054db700 1 --2- 172.21.2.101:0/3653111070 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2e8018730 0x7fd2e801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:28.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.467 7fd3054db700 1 --2- 172.21.2.101:0/3653111070 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2e8018730 0x7fd2e801ac10 crc :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:28.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.498 7fd307ec0700 1 -- 172.21.2.101:0/3653111070 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3000825d0 con 0x7fd3001b33e0 2021-02-10T11:46:28.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.500 7fd2f27fc700 1 -- 172.21.2.101:0/3653111070 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd2fc000c90 con 0x7fd3001b33e0 2021-02-10T11:46:28.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.626 7fd307ec0700 1 -- 172.21.2.101:0/3653111070 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd3000825d0 con 0x7fd3001b33e0 2021-02-10T11:46:28.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:28.999 7fd2f27fc700 1 -- 172.21.2.101:0/3653111070 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v169) v1 ==== 173+0+0 (secure 0 0 0) 0x7fd2fc001160 con 0x7fd3001b33e0 2021-02-10T11:46:29.038 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' removed 2021-02-10T11:46:29.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.043 7fd307ec0700 1 -- 172.21.2.101:0/3653111070 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2e8018730 msgr2=0x7fd2e801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:29.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.043 7fd307ec0700 1 --2- 172.21.2.101:0/3653111070 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2e8018730 0x7fd2e801ac10 crc :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:29.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.043 7fd307ec0700 1 -- 172.21.2.101:0/3653111070 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3001b33e0 msgr2=0x7fd3001a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:29.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.043 7fd307ec0700 1 --2- 172.21.2.101:0/3653111070 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3001b33e0 0x7fd3001a2b50 secure :-1 s=READY pgs=2953 cs=0 l=1 rev1=1 rx=0x7fd2fc00a570 tx=0x7fd2fc00a910).stop 2021-02-10T11:46:29.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.043 7fd307ec0700 1 -- 172.21.2.101:0/3653111070 shutdown_connections 2021-02-10T11:46:29.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.043 7fd307ec0700 1 --2- 172.21.2.101:0/3653111070 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2e8018730 0x7fd2e801ac10 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:29.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.043 7fd307ec0700 1 --2- 172.21.2.101:0/3653111070 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd300187e70 0x7fd300188270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:29.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.043 7fd307ec0700 1 --2- 172.21.2.101:0/3653111070 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd3000931c0 0x7fd3001b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:29.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.043 7fd307ec0700 1 --2- 172.21.2.101:0/3653111070 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd3001b33e0 0x7fd3001a2b50 unknown :-1 s=CLOSED pgs=2953 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:29.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.043 7fd307ec0700 1 -- 172.21.2.101:0/3653111070 >> 172.21.2.101:0/3653111070 conn(0x7fd3000841e0 msgr2=0x7fd300082800 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:29.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.043 7fd307ec0700 1 -- 172.21.2.101:0/3653111070 shutdown_connections 2021-02-10T11:46:29.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.043 7fd307ec0700 1 -- 172.21.2.101:0/3653111070 wait complete. 2021-02-10T11:46:29.049 INFO:tasks.ceph.mds.a:Restarting daemon 2021-02-10T11:46:29.049 INFO:tasks.ceph.mds.a:Stopping old one... 2021-02-10T11:46:29.049 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-02-10T11:46:29.049 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:46:29.049 INFO:tasks.ceph.mds.b:Restarting daemon 2021-02-10T11:46:29.050 INFO:tasks.ceph.mds.b:Stopping old one... 2021-02-10T11:46:29.050 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-02-10T11:46:29.050 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:46:29.055 INFO:tasks.ceph.mds.a:Stopped 2021-02-10T11:46:29.055 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-02-10T11:46:29.056 INFO:tasks.ceph.mds.b:Stopped 2021-02-10T11:46:29.056 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-02-10T11:46:29.097 INFO:tasks.ceph.mds.a:Started 2021-02-10T11:46:29.097 DEBUG:teuthology.parallel:result is None 2021-02-10T11:46:29.098 INFO:tasks.ceph.mds.b:Started 2021-02-10T11:46:29.098 DEBUG:teuthology.parallel:result is None 2021-02-10T11:46:29.099 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd blacklist clear 2021-02-10T11:46:29.502 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:46:29.504 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:46:29.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.514 7f4cf47a4700 1 Processor -- start 2021-02-10T11:46:29.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.514 7f4cf47a4700 1 -- start start 2021-02-10T11:46:29.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.514 7f4cf47a4700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4cec0d1c70 0x7f4cec0f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:29.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.514 7f4cf47a4700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cec0f0b10 con 0x7f4cec0d0340 2021-02-10T11:46:29.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.514 7f4cf47a4700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cec1b0be0 con 0x7f4cec0d1870 2021-02-10T11:46:29.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.514 7f4cf47a4700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cec0e5ed0 con 0x7f4cec0d1c70 2021-02-10T11:46:29.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.515 7f4cf1dbf700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f4cec0d0340 0x7f4cec0ef640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33720/0 (socket says 172.21.2.101:33720) 2021-02-10T11:46:29.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.515 7f4cf1dbf700 1 -- 172.21.2.101:0/3429720266 learned_addr learned my addr 172.21.2.101:0/3429720266 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:29.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.515 7f4cf2dc1700 1 --2- 172.21.2.101:0/3429720266 >> v2:172.21.2.108:3300/0 conn(0x7f4cec0d1c70 0x7f4cec0f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:29.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.515 7f4cf15be700 1 -- 172.21.2.101:0/3429720266 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 805902539 0 0) 0x7f4ce00010b0 con 0x7f4cec0d0340 2021-02-10T11:46:29.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.515 7f4cf15be700 1 -- 172.21.2.101:0/3429720266 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cd8002fe0 con 0x7f4cec0d0340 2021-02-10T11:46:29.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.515 7f4cf2dc1700 1 -- 172.21.2.101:0/3429720266 >> v1:172.21.2.108:6789/0 conn(0x7f4cec0d1870 legacy=0x7f4cec1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:29.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.515 7f4cf2dc1700 1 -- 172.21.2.101:0/3429720266 >> v1:172.21.2.101:6789/0 conn(0x7f4cec0d0340 legacy=0x7f4cec0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:29.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.515 7f4cf2dc1700 1 -- 172.21.2.101:0/3429720266 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cec0efe30 con 0x7f4cec0d1c70 2021-02-10T11:46:29.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.515 7f4cf2dc1700 1 --2- 172.21.2.101:0/3429720266 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4cec0d1c70 0x7f4cec0f20a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f4ce4006e50 tx=0x7f4ce4009df0).ready entity=mon.1 client_cookie=9bc8255472f7735 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:29.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.516 7f4cf15be700 1 -- 172.21.2.101:0/3429720266 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4ce400b510 con 0x7f4cec0d1c70 2021-02-10T11:46:29.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.516 7f4cf15be700 1 -- 172.21.2.101:0/3429720266 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce4006790 con 0x7f4cec0d1c70 2021-02-10T11:46:29.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.516 7f4cf15be700 1 -- 172.21.2.101:0/3429720266 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4ce400b750 con 0x7f4cec0d1c70 2021-02-10T11:46:29.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.516 7f4cf47a4700 1 -- 172.21.2.101:0/3429720266 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4cec0d1c70 msgr2=0x7f4cec0f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:29.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.516 7f4cf47a4700 1 --2- 172.21.2.101:0/3429720266 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4cec0d1c70 0x7f4cec0f20a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f4ce4006e50 tx=0x7f4ce4009df0).stop 2021-02-10T11:46:29.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.516 7f4cf47a4700 1 -- 172.21.2.101:0/3429720266 shutdown_connections 2021-02-10T11:46:29.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.516 7f4cf47a4700 1 --2- 172.21.2.101:0/3429720266 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4cec0d1c70 0x7f4cec0f20a0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:29.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.516 7f4cf47a4700 1 -- 172.21.2.101:0/3429720266 >> 172.21.2.101:0/3429720266 conn(0x7f4cec0ce450 msgr2=0x7f4cec0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:29.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.516 7f4cf47a4700 1 -- 172.21.2.101:0/3429720266 shutdown_connections 2021-02-10T11:46:29.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.516 7f4cf47a4700 1 -- 172.21.2.101:0/3429720266 wait complete. 2021-02-10T11:46:29.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.531 7f4cf47a4700 1 Processor -- start 2021-02-10T11:46:29.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.531 7f4cf47a4700 1 -- start start 2021-02-10T11:46:29.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.532 7f4cf47a4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4cec1a4a20 0x7f4cec0960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:29.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.532 7f4cf47a4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4cec0966c0 0x7f4cec0c2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:29.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.532 7f4cf47a4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4cec0c2860 0x7f4cec09a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:29.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.532 7f4cf47a4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec0c2c60 con 0x7f4cec0c2860 2021-02-10T11:46:29.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.532 7f4cf47a4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec096cf0 con 0x7f4cec1a4a20 2021-02-10T11:46:29.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.532 7f4cf47a4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec09a6b0 con 0x7f4cec0966c0 2021-02-10T11:46:29.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.532 7f4cf2dc1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4cec0c2860 0x7f4cec09a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:29.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.532 7f4cf2dc1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4cec0c2860 0x7f4cec09a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33234/0 (socket says 172.21.2.101:33234) 2021-02-10T11:46:29.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.532 7f4cf2dc1700 1 -- 172.21.2.101:0/2926910909 learned_addr learned my addr 172.21.2.101:0/2926910909 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:29.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.532 7f4cf1dbf700 1 --2- 172.21.2.101:0/2926910909 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4cec0966c0 0x7f4cec0c2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:29.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.532 7f4cf25c0700 1 --2- 172.21.2.101:0/2926910909 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4cec1a4a20 0x7f4cec0960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:29.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.532 7f4cf2dc1700 1 -- 172.21.2.101:0/2926910909 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4cec0966c0 msgr2=0x7f4cec0c2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:29.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.532 7f4cf2dc1700 1 --2- 172.21.2.101:0/2926910909 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4cec0966c0 0x7f4cec0c2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:29.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.532 7f4cf2dc1700 1 -- 172.21.2.101:0/2926910909 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4cec1a4a20 msgr2=0x7f4cec0960a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:29.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.532 7f4cf2dc1700 1 --2- 172.21.2.101:0/2926910909 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4cec1a4a20 0x7f4cec0960a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:29.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.532 7f4cf2dc1700 1 -- 172.21.2.101:0/2926910909 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cec09a8d0 con 0x7f4cec0c2860 2021-02-10T11:46:29.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.532 7f4cf2dc1700 1 --2- 172.21.2.101:0/2926910909 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4cec0c2860 0x7f4cec09a090 secure :-1 s=READY pgs=2958 cs=0 l=1 rev1=1 rx=0x7f4ce4006fa0 tx=0x7f4ce400bd90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:29.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.532 7f4cdeffd700 1 -- 172.21.2.101:0/2926910909 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4ce40093d0 con 0x7f4cec0c2860 2021-02-10T11:46:29.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.532 7f4cdeffd700 1 -- 172.21.2.101:0/2926910909 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce40082e0 con 0x7f4cec0c2860 2021-02-10T11:46:29.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.532 7f4cdeffd700 1 -- 172.21.2.101:0/2926910909 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4ce4008800 con 0x7f4cec0c2860 2021-02-10T11:46:29.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.532 7f4cf47a4700 1 -- 172.21.2.101:0/2926910909 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cec09a8d0 con 0x7f4cec0c2860 2021-02-10T11:46:29.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.533 7f4cf47a4700 1 -- 172.21.2.101:0/2926910909 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cec09ae60 con 0x7f4cec0c2860 2021-02-10T11:46:29.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.533 7f4cdeffd700 1 -- 172.21.2.101:0/2926910909 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4ce40084c0 con 0x7f4cec0c2860 2021-02-10T11:46:29.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.533 7f4cdeffd700 1 --2- 172.21.2.101:0/2926910909 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cd40186c0 0x7f4cd401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:29.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.533 7f4cdeffd700 1 -- 172.21.2.101:0/2926910909 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 4839+0+0 (secure 0 0 0) 0x7f4ce4032640 con 0x7f4cec0c2860 2021-02-10T11:46:29.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.533 7f4cf25c0700 1 --2- 172.21.2.101:0/2926910909 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cd40186c0 0x7f4cd401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:29.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.533 7f4cf25c0700 1 --2- 172.21.2.101:0/2926910909 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cd40186c0 0x7f4cd401abc0 crc :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:29.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.564 7f4cf47a4700 1 -- 172.21.2.101:0/2926910909 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cec0ee0f0 con 0x7f4cec0c2860 2021-02-10T11:46:29.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.566 7f4cdeffd700 1 -- 172.21.2.101:0/2926910909 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4ce402aef0 con 0x7f4cec0c2860 2021-02-10T11:46:29.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:29.707 7f4cf47a4700 1 -- 172.21.2.101:0/2926910909 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f4cec0ce450 con 0x7f4cec0c2860 2021-02-10T11:46:29.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.001 7f4cdeffd700 1 -- 172.21.2.101:0/2926910909 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v170) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4ce402aef0 con 0x7f4cec0c2860 2021-02-10T11:46:30.019 INFO:teuthology.orchestra.run.gibba001.stderr: removed all blacklist entries 2021-02-10T11:46:30.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.023 7f4cf47a4700 1 -- 172.21.2.101:0/2926910909 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cd40186c0 msgr2=0x7f4cd401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:30.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.023 7f4cf47a4700 1 --2- 172.21.2.101:0/2926910909 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cd40186c0 0x7f4cd401abc0 crc :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:30.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.023 7f4cf47a4700 1 -- 172.21.2.101:0/2926910909 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4cec0c2860 msgr2=0x7f4cec09a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:30.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.023 7f4cf47a4700 1 --2- 172.21.2.101:0/2926910909 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4cec0c2860 0x7f4cec09a090 secure :-1 s=READY pgs=2958 cs=0 l=1 rev1=1 rx=0x7f4ce4006fa0 tx=0x7f4ce400bd90).stop 2021-02-10T11:46:30.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.023 7f4cf47a4700 1 -- 172.21.2.101:0/2926910909 shutdown_connections 2021-02-10T11:46:30.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.023 7f4cf47a4700 1 --2- 172.21.2.101:0/2926910909 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cd40186c0 0x7f4cd401abc0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:30.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.023 7f4cf47a4700 1 --2- 172.21.2.101:0/2926910909 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4cec0966c0 0x7f4cec0c2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:30.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.023 7f4cf47a4700 1 --2- 172.21.2.101:0/2926910909 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4cec0c2860 0x7f4cec09a090 unknown :-1 s=CLOSED pgs=2958 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:30.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.023 7f4cf47a4700 1 --2- 172.21.2.101:0/2926910909 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4cec1a4a20 0x7f4cec0960a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:30.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.023 7f4cf47a4700 1 -- 172.21.2.101:0/2926910909 >> 172.21.2.101:0/2926910909 conn(0x7f4cec09e9b0 msgr2=0x7f4cec0a6900 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:30.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.023 7f4cf47a4700 1 -- 172.21.2.101:0/2926910909 shutdown_connections 2021-02-10T11:46:30.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.023 7f4cf47a4700 1 -- 172.21.2.101:0/2926910909 wait complete. 2021-02-10T11:46:30.030 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2021-02-10T11:46:30.030 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:46:30.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.445 7fe9bd35b700 1 Processor -- start 2021-02-10T11:46:30.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.446 7fe9bd35b700 1 -- start start 2021-02-10T11:46:30.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.446 7fe9bd35b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9b81b0470 0x7fe9b81b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:30.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.446 7fe9bd35b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe9b80ef610 0x7fe9b80efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:30.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.446 7fe9bd35b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b80f0b50 con 0x7fe9b80f0030 2021-02-10T11:46:30.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.446 7fe9bd35b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b80e52e0 con 0x7fe9b80ef610 2021-02-10T11:46:30.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.446 7fe9bd35b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b80e5490 con 0x7fe9b81b0470 2021-02-10T11:46:30.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.446 7fe9b787e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe9b80f0030 0x7fe9b80f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33732/0 (socket says 172.21.2.101:33732) 2021-02-10T11:46:30.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.446 7fe9b787e700 1 -- 172.21.2.101:0/1128998730 learned_addr learned my addr 172.21.2.101:0/1128998730 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:30.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.446 7fe9b687c700 1 --2- 172.21.2.101:0/1128998730 >> v2:172.21.2.101:3300/0 conn(0x7fe9b80ef610 0x7fe9b80efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:30.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.447 7fe9b687c700 1 -- 172.21.2.101:0/1128998730 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9b81b0470 msgr2=0x7fe9b81b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:30.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.447 7fe9b687c700 1 --2- 172.21.2.101:0/1128998730 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9b81b0470 0x7fe9b81b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:30.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.447 7fe9b687c700 1 -- 172.21.2.101:0/1128998730 >> v1:172.21.2.101:6789/0 conn(0x7fe9b80f0030 legacy=0x7fe9b80f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:30.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.447 7fe9b687c700 1 -- 172.21.2.101:0/1128998730 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b80f0430 con 0x7fe9b80ef610 2021-02-10T11:46:30.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.447 7fe9b687c700 1 --2- 172.21.2.101:0/1128998730 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9b80ef610 0x7fe9b80efa10 secure :-1 s=READY pgs=2960 cs=0 l=1 rev1=1 rx=0x7fe9a0006650 tx=0x7fe9a0009a80).ready entity=mon.0 client_cookie=c6b0025e801a671c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:30.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.447 7fe9b607b700 1 -- 172.21.2.101:0/1128998730 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe9a000c0c0 con 0x7fe9b80ef610 2021-02-10T11:46:30.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.447 7fe9b607b700 1 -- 172.21.2.101:0/1128998730 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a000c340 con 0x7fe9b80ef610 2021-02-10T11:46:30.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.447 7fe9b607b700 1 -- 172.21.2.101:0/1128998730 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe9a000c860 con 0x7fe9b80ef610 2021-02-10T11:46:30.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.447 7fe9bd35b700 1 -- 172.21.2.101:0/1128998730 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9b80ef610 msgr2=0x7fe9b80efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:30.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.447 7fe9bd35b700 1 --2- 172.21.2.101:0/1128998730 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9b80ef610 0x7fe9b80efa10 secure :-1 s=READY pgs=2960 cs=0 l=1 rev1=1 rx=0x7fe9a0006650 tx=0x7fe9a0009a80).stop 2021-02-10T11:46:30.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.448 7fe9bd35b700 1 -- 172.21.2.101:0/1128998730 shutdown_connections 2021-02-10T11:46:30.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.448 7fe9bd35b700 1 --2- 172.21.2.101:0/1128998730 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9b80ef610 0x7fe9b80efa10 unknown :-1 s=CLOSED pgs=2960 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:30.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.448 7fe9bd35b700 1 --2- 172.21.2.101:0/1128998730 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9b81b0470 0x7fe9b81b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:30.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.448 7fe9bd35b700 1 -- 172.21.2.101:0/1128998730 >> 172.21.2.101:0/1128998730 conn(0x7fe9b80ce3e0 msgr2=0x7fe9b80e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:30.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.448 7fe9bd35b700 1 -- 172.21.2.101:0/1128998730 shutdown_connections 2021-02-10T11:46:30.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.448 7fe9bd35b700 1 -- 172.21.2.101:0/1128998730 wait complete. 2021-02-10T11:46:30.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.462 7fe9bd35b700 1 Processor -- start 2021-02-10T11:46:30.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.463 7fe9bd35b700 1 -- start start 2021-02-10T11:46:30.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.463 7fe9bd35b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9b80b0970 0x7fe9b8092ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:30.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.463 7fe9bd35b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9b80934f0 0x7fe9b8094dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:30.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.463 7fe9bd35b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe9b80938f0 0x7fe9b80953e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:30.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.463 7fe9bd35b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b8195ad0 con 0x7fe9b80934f0 2021-02-10T11:46:30.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.463 7fe9bd35b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b8095f40 con 0x7fe9b80938f0 2021-02-10T11:46:30.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.463 7fe9bd35b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b8095a30 con 0x7fe9b80b0970 2021-02-10T11:46:30.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.463 7fe9b687c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9b80934f0 0x7fe9b8094dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:30.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.463 7fe9b687c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9b80934f0 0x7fe9b8094dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33244/0 (socket says 172.21.2.101:33244) 2021-02-10T11:46:30.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.463 7fe9b687c700 1 -- 172.21.2.101:0/3352133541 learned_addr learned my addr 172.21.2.101:0/3352133541 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:30.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.464 7fe9b687c700 1 -- 172.21.2.101:0/3352133541 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9b80b0970 msgr2=0x7fe9b8092ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:30.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.464 7fe9b687c700 1 --2- 172.21.2.101:0/3352133541 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9b80b0970 0x7fe9b8092ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:30.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.464 7fe9b687c700 1 -- 172.21.2.101:0/3352133541 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe9b80938f0 msgr2=0x7fe9b80953e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:30.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.464 7fe9b687c700 1 --2- 172.21.2.101:0/3352133541 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe9b80938f0 0x7fe9b80953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:30.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.464 7fe9b687c700 1 -- 172.21.2.101:0/3352133541 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b8095c50 con 0x7fe9b80934f0 2021-02-10T11:46:30.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.464 7fe9b687c700 1 --2- 172.21.2.101:0/3352133541 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9b80934f0 0x7fe9b8094dc0 secure :-1 s=READY pgs=2961 cs=0 l=1 rev1=1 rx=0x7fe9a0009630 tx=0x7fe9a000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:30.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.464 7fe9affff700 1 -- 172.21.2.101:0/3352133541 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe9a00093a0 con 0x7fe9b80934f0 2021-02-10T11:46:30.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.464 7fe9affff700 1 -- 172.21.2.101:0/3352133541 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a000bd70 con 0x7fe9b80934f0 2021-02-10T11:46:30.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.464 7fe9affff700 1 -- 172.21.2.101:0/3352133541 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe9a00054d0 con 0x7fe9b80934f0 2021-02-10T11:46:30.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.464 7fe9bd35b700 1 -- 172.21.2.101:0/3352133541 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b8095c50 con 0x7fe9b80934f0 2021-02-10T11:46:30.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.464 7fe9bd35b700 1 -- 172.21.2.101:0/3352133541 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b80967b0 con 0x7fe9b80934f0 2021-02-10T11:46:30.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.465 7fe9affff700 1 -- 172.21.2.101:0/3352133541 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe9a000a000 con 0x7fe9b80934f0 2021-02-10T11:46:30.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.465 7fe9affff700 1 --2- 172.21.2.101:0/3352133541 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe99c0186c0 0x7fe99c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:30.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.465 7fe9affff700 1 -- 172.21.2.101:0/3352133541 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fe9a0032270 con 0x7fe9b80934f0 2021-02-10T11:46:30.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.465 7fe9b707d700 1 --2- 172.21.2.101:0/3352133541 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe99c0186c0 0x7fe99c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:30.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.465 7fe9b707d700 1 --2- 172.21.2.101:0/3352133541 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe99c0186c0 0x7fe99c01abc0 crc :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:30.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.496 7fe9bd35b700 1 -- 172.21.2.101:0/3352133541 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b8095a00 con 0x7fe9b80934f0 2021-02-10T11:46:30.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.498 7fe9affff700 1 -- 172.21.2.101:0/3352133541 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe9a001afd0 con 0x7fe9b80934f0 2021-02-10T11:46:30.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.630 7fe9bd35b700 1 -- 172.21.2.101:0/3352133541 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe9b8095a00 con 0x7fe9b80934f0 2021-02-10T11:46:30.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.630 7fe9affff700 1 -- 172.21.2.101:0/3352133541 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v94) v1 ==== 80+0+4513 (secure 0 0 0) 0x7fe9a001afd0 con 0x7fe9b80934f0 2021-02-10T11:46:30.629 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:46:30.629 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:46:30.629 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:46:30.629 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:30.629 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:46:30.629 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:46:30.629 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:46:30.629 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:46:30.629 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:46:30.629 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:46:30.629 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:46:30.629 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:30.630 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:30.630 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:30.630 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:46:30.630 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:46:30.630 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:46:30.630 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:46:30.630 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:46:30.630 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:46:30.630 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:46:30.630 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:30.630 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:30.630 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:30.630 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:46:30.631 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:46:30.631 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:46:30.631 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:46:30.631 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:46:30.631 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:46:30.631 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:30.631 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:30.631 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:30.631 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:46:30.631 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:46:30.631 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:46:30.631 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:46:30.631 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:46:30.631 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:46:30.632 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:30.632 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:30.632 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:30.632 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:46:30.632 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:46:30.632 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:46:30.632 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:46:30.632 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:46:30.632 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:46:30.632 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:30.632 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:30.632 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:30.632 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:46:30.632 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:46:30.633 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:46:30.633 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:46:30.633 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:46:30.633 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:46:30.633 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:30.633 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:30.633 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:30.633 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:46:30.633 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:46:30.633 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:46:30.633 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:46:30.633 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:46:30.633 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:46:30.633 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:30.634 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:30.634 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:30.634 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:46:30.634 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:46:30.634 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:46:30.634 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:46:30.634 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:46:30.634 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:46:30.634 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:30.634 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:30.634 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:30.634 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:46:30.635 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:46:30.635 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:46:30.635 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:46:30.635 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:46:30.635 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:46:30.635 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:30.635 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:30.635 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:30.635 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:46:30.635 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:46:30.635 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:46:30.635 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:46:30.635 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:46:30.635 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:46:30.636 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:30.636 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:30.636 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:30.636 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:46:30.636 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:46:30.637 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:46:30.637 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:46:30.637 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:46:30.637 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:46:30.637 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:30.638 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:30.638 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:30.638 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:46:30.638 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:46:30.638 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:46:30.638 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:46:30.639 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:46:30.639 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:46:30.639 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:30.639 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:30.639 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:30.639 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:46:30.639 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:46:30.639 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:46:30.639 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:46:30.639 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:46:30.639 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:46:30.639 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:30.639 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:30.640 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:30.640 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:46:30.640 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:46:30.640 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:46:30.640 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:46:30.640 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:46:30.640 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:46:30.640 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:30.640 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:30.640 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:30.640 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:46:30.640 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:46:30.640 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:46:30.640 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:46:30.641 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:46:30.641 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:46:30.641 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:46:30.641 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:30.641 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:30.641 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:30.641 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:46:30.641 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:46:30.641 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:46:30.641 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:46:30.641 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:46:30.641 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:46:30.641 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:30.642 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:30.642 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:30.642 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:46:30.642 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:46:30.642 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:46:30.642 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:46:30.642 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:46:30.642 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:46:30.642 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:30.642 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:30.642 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:46:30.642 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:46:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.632 7fe9bd35b700 1 -- 172.21.2.101:0/3352133541 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe99c0186c0 msgr2=0x7fe99c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.632 7fe9bd35b700 1 --2- 172.21.2.101:0/3352133541 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe99c0186c0 0x7fe99c01abc0 crc :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.632 7fe9bd35b700 1 -- 172.21.2.101:0/3352133541 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9b80934f0 msgr2=0x7fe9b8094dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.632 7fe9bd35b700 1 --2- 172.21.2.101:0/3352133541 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9b80934f0 0x7fe9b8094dc0 secure :-1 s=READY pgs=2961 cs=0 l=1 rev1=1 rx=0x7fe9a0009630 tx=0x7fe9a000b980).stop 2021-02-10T11:46:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.632 7fe9bd35b700 1 -- 172.21.2.101:0/3352133541 shutdown_connections 2021-02-10T11:46:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.632 7fe9bd35b700 1 --2- 172.21.2.101:0/3352133541 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe9b80938f0 0x7fe9b80953e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.633 7fe9bd35b700 1 --2- 172.21.2.101:0/3352133541 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9b80934f0 0x7fe9b8094dc0 unknown :-1 s=CLOSED pgs=2961 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.633 7fe9bd35b700 1 --2- 172.21.2.101:0/3352133541 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe99c0186c0 0x7fe99c01abc0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.633 7fe9bd35b700 1 --2- 172.21.2.101:0/3352133541 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9b80b0970 0x7fe9b8092ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.633 7fe9bd35b700 1 -- 172.21.2.101:0/3352133541 >> 172.21.2.101:0/3352133541 conn(0x7fe9b819f7a0 msgr2=0x7fe9b80940e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.633 7fe9bd35b700 1 -- 172.21.2.101:0/3352133541 shutdown_connections 2021-02-10T11:46:30.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:30.633 7fe9bd35b700 1 -- 172.21.2.101:0/3352133541 wait complete. 2021-02-10T11:46:30.645 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:46:31.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.061 7fddaaf74700 1 Processor -- start 2021-02-10T11:46:31.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.061 7fddaaf74700 1 -- start start 2021-02-10T11:46:31.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.061 7fddaaf74700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fdda40d0340 0x7fdda40ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:31.075 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.061 7fddaaf74700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdda40d1c70 0x7fdda40efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:31.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.061 7fddaaf74700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdda40f0b10 con 0x7fdda40d1870 2021-02-10T11:46:31.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.061 7fddaaf74700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdda40e5ed0 con 0x7fdda40d0340 2021-02-10T11:46:31.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.061 7fddaaf74700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda41b0be0 con 0x7fdda40d1c70 2021-02-10T11:46:31.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.062 7fdda3fff700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fdda40d0340 0x7fdda40ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:31.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.062 7fdda9591700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdda40d1c70 0x7fdda40efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:31.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.062 7fdda3fff700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fdda40d0340 0x7fdda40ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:43736/0 (socket says 172.21.2.101:43736) 2021-02-10T11:46:31.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.062 7fdda9591700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdda40d1c70 0x7fdda40efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:54218/0 (socket says 172.21.2.101:54218) 2021-02-10T11:46:31.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.062 7fdda3fff700 1 -- 172.21.2.101:0/2105031493 learned_addr learned my addr 172.21.2.101:0/2105031493 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:31.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.062 7fdda3fff700 1 -- 172.21.2.101:0/2105031493 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdda40d1c70 msgr2=0x7fdda40efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:31.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.062 7fdda3fff700 1 --2- 172.21.2.101:0/2105031493 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdda40d1c70 0x7fdda40efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:31.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.062 7fdda3fff700 1 -- 172.21.2.101:0/2105031493 >> v1:172.21.2.108:6789/0 conn(0x7fdda40d1870 legacy=0x7fdda41b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:31.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.062 7fdda3fff700 1 -- 172.21.2.101:0/2105031493 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda40f20a0 con 0x7fdda40d0340 2021-02-10T11:46:31.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.062 7fdda3fff700 1 --2- 172.21.2.101:0/2105031493 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdda40d0340 0x7fdda40ef640 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7fdd94006a80 tx=0x7fdd94009f00).ready entity=mon.1 client_cookie=fe7dd52d7479bed5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:31.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.063 7fdda37fe700 1 -- 172.21.2.101:0/2105031493 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd9400b5c0 con 0x7fdda40d0340 2021-02-10T11:46:31.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.063 7fdda37fe700 1 -- 172.21.2.101:0/2105031493 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9400b800 con 0x7fdda40d0340 2021-02-10T11:46:31.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.063 7fdda37fe700 1 -- 172.21.2.101:0/2105031493 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd9400bd20 con 0x7fdda40d0340 2021-02-10T11:46:31.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.063 7fddaaf74700 1 -- 172.21.2.101:0/2105031493 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdda40d0340 msgr2=0x7fdda40ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:31.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.063 7fddaaf74700 1 --2- 172.21.2.101:0/2105031493 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdda40d0340 0x7fdda40ef640 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7fdd94006a80 tx=0x7fdd94009f00).stop 2021-02-10T11:46:31.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.063 7fddaaf74700 1 -- 172.21.2.101:0/2105031493 shutdown_connections 2021-02-10T11:46:31.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.063 7fddaaf74700 1 --2- 172.21.2.101:0/2105031493 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdda40d1c70 0x7fdda40efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:31.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.063 7fddaaf74700 1 --2- 172.21.2.101:0/2105031493 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdda40d0340 0x7fdda40ef640 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:31.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.063 7fddaaf74700 1 -- 172.21.2.101:0/2105031493 >> 172.21.2.101:0/2105031493 conn(0x7fdda40ce450 msgr2=0x7fdda40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:31.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.064 7fddaaf74700 1 -- 172.21.2.101:0/2105031493 shutdown_connections 2021-02-10T11:46:31.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.064 7fddaaf74700 1 -- 172.21.2.101:0/2105031493 wait complete. 2021-02-10T11:46:31.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.078 7fddaaf74700 1 Processor -- start 2021-02-10T11:46:31.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.078 7fddaaf74700 1 -- start start 2021-02-10T11:46:31.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.078 7fddaaf74700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdda4094040 0x7fdda4093370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:31.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.078 7fddaaf74700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdda4093990 0x7fdda40ad700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:31.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.078 7fddaaf74700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdda40add20 0x7fdda41a2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:31.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.078 7fddaaf74700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda4093d90 con 0x7fdda4094040 2021-02-10T11:46:31.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.078 7fddaaf74700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda41a1e50 con 0x7fdda4093990 2021-02-10T11:46:31.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.078 7fddaaf74700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda4094590 con 0x7fdda40add20 2021-02-10T11:46:31.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.079 7fdda8d90700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdda4094040 0x7fdda4093370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:31.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.079 7fdda8d90700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdda4094040 0x7fdda4093370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33258/0 (socket says 172.21.2.101:33258) 2021-02-10T11:46:31.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.079 7fdda8d90700 1 -- 172.21.2.101:0/1976720734 learned_addr learned my addr 172.21.2.101:0/1976720734 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:31.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.079 7fdda8d90700 1 -- 172.21.2.101:0/1976720734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdda40add20 msgr2=0x7fdda41a2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:31.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.079 7fdda8d90700 1 --2- 172.21.2.101:0/1976720734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdda40add20 0x7fdda41a2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:31.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.079 7fdda8d90700 1 -- 172.21.2.101:0/1976720734 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdda4093990 msgr2=0x7fdda40ad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:31.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.079 7fdda3fff700 1 --2- 172.21.2.101:0/1976720734 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdda4093990 0x7fdda40ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:31.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.079 7fdda8d90700 1 --2- 172.21.2.101:0/1976720734 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdda4093990 0x7fdda40ad700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:31.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.079 7fdda8d90700 1 -- 172.21.2.101:0/1976720734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda40ae120 con 0x7fdda4094040 2021-02-10T11:46:31.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.079 7fdda3fff700 1 --2- 172.21.2.101:0/1976720734 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdda4093990 0x7fdda40ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:46:31.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.079 7fdda8d90700 1 --2- 172.21.2.101:0/1976720734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdda4094040 0x7fdda4093370 secure :-1 s=READY pgs=2962 cs=0 l=1 rev1=1 rx=0x7fdd8c006ed0 tx=0x7fdd8c00b8e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:31.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.080 7fdda17fa700 1 -- 172.21.2.101:0/1976720734 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd8c006020 con 0x7fdda4094040 2021-02-10T11:46:31.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.080 7fdda17fa700 1 -- 172.21.2.101:0/1976720734 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8c0009b0 con 0x7fdda4094040 2021-02-10T11:46:31.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.080 7fddaaf74700 1 -- 172.21.2.101:0/1976720734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda40ae120 con 0x7fdda4094040 2021-02-10T11:46:31.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.080 7fddaaf74700 1 -- 172.21.2.101:0/1976720734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda41a2d90 con 0x7fdda4094040 2021-02-10T11:46:31.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.080 7fdda17fa700 1 -- 172.21.2.101:0/1976720734 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdd8c0054b0 con 0x7fdda4094040 2021-02-10T11:46:31.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.080 7fdda17fa700 1 -- 172.21.2.101:0/1976720734 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdd8c006020 con 0x7fdda4094040 2021-02-10T11:46:31.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.080 7fdda17fa700 1 --2- 172.21.2.101:0/1976720734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd880186c0 0x7fdd8801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:31.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.080 7fdda17fa700 1 -- 172.21.2.101:0/1976720734 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fdd8c032260 con 0x7fdda4094040 2021-02-10T11:46:31.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.081 7fdda3fff700 1 --2- 172.21.2.101:0/1976720734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd880186c0 0x7fdd8801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:31.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.081 7fdda3fff700 1 --2- 172.21.2.101:0/1976720734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd880186c0 0x7fdd8801abc0 crc :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:31.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.111 7fddaaf74700 1 -- 172.21.2.101:0/1976720734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdda40ebc60 con 0x7fdda4094040 2021-02-10T11:46:31.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.114 7fdda17fa700 1 -- 172.21.2.101:0/1976720734 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdd8c030d40 con 0x7fdda4094040 2021-02-10T11:46:31.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.236 7fddaaf74700 1 -- 172.21.2.101:0/1976720734 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdda40f0b10 con 0x7fdda4094040 2021-02-10T11:46:31.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.237 7fdda17fa700 1 -- 172.21.2.101:0/1976720734 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+11154 (secure 0 0 0) 0x7fdd8c005860 con 0x7fdda4094040 2021-02-10T11:46:31.235 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:46:31.235 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":170,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:46:29.998806","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:46:31.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.238 7fddaaf74700 1 -- 172.21.2.101:0/1976720734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd880186c0 msgr2=0x7fdd8801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:31.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.238 7fddaaf74700 1 --2- 172.21.2.101:0/1976720734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd880186c0 0x7fdd8801abc0 crc :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:31.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.238 7fddaaf74700 1 -- 172.21.2.101:0/1976720734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdda4094040 msgr2=0x7fdda4093370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:31.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.238 7fddaaf74700 1 --2- 172.21.2.101:0/1976720734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdda4094040 0x7fdda4093370 secure :-1 s=READY pgs=2962 cs=0 l=1 rev1=1 rx=0x7fdd8c006ed0 tx=0x7fdd8c00b8e0).stop 2021-02-10T11:46:31.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.238 7fddaaf74700 1 -- 172.21.2.101:0/1976720734 shutdown_connections 2021-02-10T11:46:31.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.239 7fddaaf74700 1 --2- 172.21.2.101:0/1976720734 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdd880186c0 0x7fdd8801abc0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:31.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.239 7fddaaf74700 1 --2- 172.21.2.101:0/1976720734 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdda40add20 0x7fdda41a2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:31.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.239 7fddaaf74700 1 --2- 172.21.2.101:0/1976720734 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdda4093990 0x7fdda40ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:31.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.239 7fddaaf74700 1 --2- 172.21.2.101:0/1976720734 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdda4094040 0x7fdda4093370 unknown :-1 s=CLOSED pgs=2962 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:31.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.239 7fddaaf74700 1 -- 172.21.2.101:0/1976720734 >> 172.21.2.101:0/1976720734 conn(0x7fdda419f8a0 msgr2=0x7fdda4094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:31.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.239 7fddaaf74700 1 -- 172.21.2.101:0/1976720734 shutdown_connections 2021-02-10T11:46:31.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.239 7fddaaf74700 1 -- 172.21.2.101:0/1976720734 wait complete. 2021-02-10T11:46:31.245 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-10T11:46:31.245 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:46:31.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.663 7f7db172a700 1 Processor -- start 2021-02-10T11:46:31.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.663 7f7db172a700 1 -- start start 2021-02-10T11:46:31.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.663 7f7db172a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7dac0d1870 0x7f7dac1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:31.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.663 7f7db172a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7dac1b0ae0 0x7f7dac108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:31.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.663 7f7db172a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7dac0d1c70 0x7f7dac108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:31.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.663 7f7db172a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dac0e5ef0 con 0x7f7dac0d1c70 2021-02-10T11:46:31.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.663 7f7db172a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dac109a60 con 0x7f7dac1b0ae0 2021-02-10T11:46:31.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.663 7f7db172a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac088070 con 0x7f7dac0d1870 2021-02-10T11:46:31.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.663 7f7dabac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7dac0d1c70 0x7f7dac108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:31.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.663 7f7dabac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7dac0d1c70 0x7f7dac108f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33264/0 (socket says 172.21.2.101:33264) 2021-02-10T11:46:31.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.663 7f7dabac0700 1 -- 172.21.2.101:0/2321356636 learned_addr learned my addr 172.21.2.101:0/2321356636 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:31.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.664 7f7dabac0700 1 -- 172.21.2.101:0/2321356636 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7dac0d1870 msgr2=0x7f7dac1b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:46:31.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.664 7f7dabac0700 1 --2- 172.21.2.101:0/2321356636 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7dac0d1870 0x7f7dac1b04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:31.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.664 7f7dabac0700 1 -- 172.21.2.101:0/2321356636 >> v2:172.21.2.108:3300/0 conn(0x7f7dac1b0ae0 msgr2=0x7f7dac108940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:46:31.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.664 7f7dabac0700 1 --2- 172.21.2.101:0/2321356636 >> v2:172.21.2.108:3300/0 conn(0x7f7dac1b0ae0 0x7f7dac108940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:31.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.664 7f7dabac0700 1 -- 172.21.2.101:0/2321356636 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dac109580 con 0x7f7dac0d1c70 2021-02-10T11:46:31.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.664 7f7dabac0700 1 --2- 172.21.2.101:0/2321356636 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7dac0d1c70 0x7f7dac108f60 secure :-1 s=READY pgs=2963 cs=0 l=1 rev1=1 rx=0x7f7da4006f00 tx=0x7f7da4009f10).ready entity=mon.0 client_cookie=f33702f7f46f8f4d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:31.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.664 7f7daa2bd700 1 -- 172.21.2.101:0/2321356636 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7da400b3b0 con 0x7f7dac0d1c70 2021-02-10T11:46:31.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.664 7f7daa2bd700 1 -- 172.21.2.101:0/2321356636 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da400b630 con 0x7f7dac0d1c70 2021-02-10T11:46:31.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.664 7f7daa2bd700 1 -- 172.21.2.101:0/2321356636 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7da400bb50 con 0x7f7dac0d1c70 2021-02-10T11:46:31.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.664 7f7db172a700 1 -- 172.21.2.101:0/2321356636 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7dac0d1c70 msgr2=0x7f7dac108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:31.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.664 7f7db172a700 1 --2- 172.21.2.101:0/2321356636 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7dac0d1c70 0x7f7dac108f60 secure :-1 s=READY pgs=2963 cs=0 l=1 rev1=1 rx=0x7f7da4006f00 tx=0x7f7da4009f10).stop 2021-02-10T11:46:31.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.665 7f7db172a700 1 -- 172.21.2.101:0/2321356636 shutdown_connections 2021-02-10T11:46:31.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.665 7f7db172a700 1 --2- 172.21.2.101:0/2321356636 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7dac0d1c70 0x7f7dac108f60 unknown :-1 s=CLOSED pgs=2963 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:31.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.665 7f7db172a700 1 --2- 172.21.2.101:0/2321356636 >> v2:172.21.2.108:3300/0 conn(0x7f7dac1b0ae0 0x7f7dac108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:31.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.665 7f7db172a700 1 --2- 172.21.2.101:0/2321356636 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7dac0d1870 0x7f7dac1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:31.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.665 7f7db172a700 1 -- 172.21.2.101:0/2321356636 >> 172.21.2.101:0/2321356636 conn(0x7f7dac0ce420 msgr2=0x7f7dac0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:31.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.665 7f7db172a700 1 -- 172.21.2.101:0/2321356636 shutdown_connections 2021-02-10T11:46:31.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.665 7f7db172a700 1 -- 172.21.2.101:0/2321356636 wait complete. 2021-02-10T11:46:31.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.679 7f7db172a700 1 Processor -- start 2021-02-10T11:46:31.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.679 7f7db172a700 1 -- start start 2021-02-10T11:46:31.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.680 7f7db172a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7dac1a7bf0 0x7f7dac1a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:31.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.680 7f7db172a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7dac195560 0x7f7dac195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:31.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.680 7f7db172a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7dac195f80 0x7f7dac196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:31.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.680 7f7db172a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac194fe0 con 0x7f7dac195560 2021-02-10T11:46:31.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.680 7f7db172a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac193020 con 0x7f7dac1a7bf0 2021-02-10T11:46:31.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.680 7f7db172a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac193440 con 0x7f7dac195f80 2021-02-10T11:46:31.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.680 7f7daaabe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7dac195560 0x7f7dac195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:31.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.680 7f7daaabe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7dac195560 0x7f7dac195960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33272/0 (socket says 172.21.2.101:33272) 2021-02-10T11:46:31.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.680 7f7daaabe700 1 -- 172.21.2.101:0/2927337667 learned_addr learned my addr 172.21.2.101:0/2927337667 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:31.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.680 7f7daaabe700 1 -- 172.21.2.101:0/2927337667 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7dac195f80 msgr2=0x7f7dac196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:31.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.680 7f7daaabe700 1 --2- 172.21.2.101:0/2927337667 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7dac195f80 0x7f7dac196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:31.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.680 7f7daaabe700 1 -- 172.21.2.101:0/2927337667 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7dac1a7bf0 msgr2=0x7f7dac1a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:31.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.680 7f7daaabe700 1 --2- 172.21.2.101:0/2927337667 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7dac1a7bf0 0x7f7dac1a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:31.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.680 7f7daaabe700 1 -- 172.21.2.101:0/2927337667 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dac193660 con 0x7f7dac195560 2021-02-10T11:46:31.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.680 7f7daaabe700 1 --2- 172.21.2.101:0/2927337667 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7dac195560 0x7f7dac195960 secure :-1 s=READY pgs=2964 cs=0 l=1 rev1=1 rx=0x7f7d9c00b5c0 tx=0x7f7d9c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:31.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.681 7f7da3fff700 1 -- 172.21.2.101:0/2927337667 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7d9c00bd50 con 0x7f7dac195560 2021-02-10T11:46:31.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.681 7f7da3fff700 1 -- 172.21.2.101:0/2927337667 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9c006030 con 0x7f7dac195560 2021-02-10T11:46:31.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.681 7f7db172a700 1 -- 172.21.2.101:0/2927337667 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dac193660 con 0x7f7dac195560 2021-02-10T11:46:31.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.681 7f7da3fff700 1 -- 172.21.2.101:0/2927337667 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7d9c0054d0 con 0x7f7dac195560 2021-02-10T11:46:31.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.681 7f7db172a700 1 -- 172.21.2.101:0/2927337667 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dac193bf0 con 0x7f7dac195560 2021-02-10T11:46:31.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.681 7f7da3fff700 1 -- 172.21.2.101:0/2927337667 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7d9c006210 con 0x7f7dac195560 2021-02-10T11:46:31.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.681 7f7da3fff700 1 --2- 172.21.2.101:0/2927337667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7d900186d0 0x7f7d9001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:31.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.681 7f7da3fff700 1 -- 172.21.2.101:0/2927337667 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f7d9c032300 con 0x7f7dac195560 2021-02-10T11:46:31.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.681 7f7dab2bf700 1 --2- 172.21.2.101:0/2927337667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7d900186d0 0x7f7d9001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:31.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.682 7f7dab2bf700 1 --2- 172.21.2.101:0/2927337667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7d900186d0 0x7f7d9001abd0 crc :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:31.710 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.713 7f7db172a700 1 -- 172.21.2.101:0/2927337667 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dac082980 con 0x7f7dac195560 2021-02-10T11:46:31.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.715 7f7da3fff700 1 -- 172.21.2.101:0/2927337667 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7d9c000c40 con 0x7f7dac195560 2021-02-10T11:46:31.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.843 7f7db172a700 1 -- 172.21.2.101:0/2927337667 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f7dac109a60 con 0x7f7dac195560 2021-02-10T11:46:31.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.843 7f7da3fff700 1 -- 172.21.2.101:0/2927337667 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v170) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7d9c006030 con 0x7f7dac195560 2021-02-10T11:46:31.841 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:46:31.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.845 7f7db172a700 1 -- 172.21.2.101:0/2927337667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7d900186d0 msgr2=0x7f7d9001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:31.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.845 7f7db172a700 1 --2- 172.21.2.101:0/2927337667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7d900186d0 0x7f7d9001abd0 crc :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:31.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.845 7f7db172a700 1 -- 172.21.2.101:0/2927337667 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7dac195560 msgr2=0x7f7dac195960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:31.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.845 7f7db172a700 1 --2- 172.21.2.101:0/2927337667 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7dac195560 0x7f7dac195960 secure :-1 s=READY pgs=2964 cs=0 l=1 rev1=1 rx=0x7f7d9c00b5c0 tx=0x7f7d9c00b960).stop 2021-02-10T11:46:31.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.845 7f7db172a700 1 -- 172.21.2.101:0/2927337667 shutdown_connections 2021-02-10T11:46:31.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.845 7f7db172a700 1 --2- 172.21.2.101:0/2927337667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7d900186d0 0x7f7d9001abd0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:31.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.845 7f7db172a700 1 --2- 172.21.2.101:0/2927337667 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7dac195f80 0x7f7dac196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:31.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.845 7f7db172a700 1 --2- 172.21.2.101:0/2927337667 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7dac195560 0x7f7dac195960 unknown :-1 s=CLOSED pgs=2964 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:31.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.845 7f7db172a700 1 --2- 172.21.2.101:0/2927337667 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7dac1a7bf0 0x7f7dac1a7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:31.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.845 7f7db172a700 1 -- 172.21.2.101:0/2927337667 >> 172.21.2.101:0/2927337667 conn(0x7f7dac0a3270 msgr2=0x7f7dac095970 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:31.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.845 7f7db172a700 1 -- 172.21.2.101:0/2927337667 shutdown_connections 2021-02-10T11:46:31.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:31.845 7f7db172a700 1 -- 172.21.2.101:0/2927337667 wait complete. 2021-02-10T11:46:31.851 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --format=json-pretty osd lspools 2021-02-10T11:46:32.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.266 7f2446297700 1 Processor -- start 2021-02-10T11:46:32.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.266 7f2446297700 1 -- start start 2021-02-10T11:46:32.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.266 7f2446297700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24400d1870 0x7f24401b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:32.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.266 7f2446297700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f24401b0a10 0x7f24400cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:32.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.266 7f2446297700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24400f0b10 con 0x7f24400d1c70 2021-02-10T11:46:32.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.266 7f2446297700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24400fe230 con 0x7f24401b0a10 2021-02-10T11:46:32.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.266 7f2446297700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24400e5ed0 con 0x7f24400d1870 2021-02-10T11:46:32.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.266 7f24448b4700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f24400d1c70 0x7f24400cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33770/0 (socket says 172.21.2.101:33770) 2021-02-10T11:46:32.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.266 7f24448b4700 1 -- 172.21.2.101:0/997662479 learned_addr learned my addr 172.21.2.101:0/997662479 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:32.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.266 7f243f7fe700 1 --2- 172.21.2.101:0/997662479 >> v2:172.21.2.101:3300/0 conn(0x7f24401b0a10 0x7f24400cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:32.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.267 7f243effd700 1 -- 172.21.2.101:0/997662479 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2697071817 0 0) 0x7f2438001180 con 0x7f24400d1c70 2021-02-10T11:46:32.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.267 7f243effd700 1 -- 172.21.2.101:0/997662479 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2428002fe0 con 0x7f24400d1c70 2021-02-10T11:46:32.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.267 7f243f7fe700 1 -- 172.21.2.101:0/997662479 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24400d1870 msgr2=0x7f24401b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:46:32.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.267 7f243f7fe700 1 --2- 172.21.2.101:0/997662479 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24400d1870 0x7f24401b03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:32.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.267 7f243f7fe700 1 -- 172.21.2.101:0/997662479 >> v1:172.21.2.101:6789/0 conn(0x7f24400d1c70 legacy=0x7f24400cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:32.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.267 7f243f7fe700 1 -- 172.21.2.101:0/997662479 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24400f0650 con 0x7f24401b0a10 2021-02-10T11:46:32.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.267 7f243f7fe700 1 --2- 172.21.2.101:0/997662479 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24401b0a10 0x7f24400cf330 secure :-1 s=READY pgs=2966 cs=0 l=1 rev1=1 rx=0x7f2434006650 tx=0x7f2434009a80).ready entity=mon.0 client_cookie=fc1be6da27365f66 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:32.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.267 7f243effd700 1 -- 172.21.2.101:0/997662479 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f243400c0c0 con 0x7f24401b0a10 2021-02-10T11:46:32.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.267 7f243effd700 1 -- 172.21.2.101:0/997662479 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243400c340 con 0x7f24401b0a10 2021-02-10T11:46:32.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.267 7f243effd700 1 -- 172.21.2.101:0/997662479 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f243400c860 con 0x7f24401b0a10 2021-02-10T11:46:32.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.267 7f2446297700 1 -- 172.21.2.101:0/997662479 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24401b0a10 msgr2=0x7f24400cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:32.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.267 7f2446297700 1 --2- 172.21.2.101:0/997662479 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24401b0a10 0x7f24400cf330 secure :-1 s=READY pgs=2966 cs=0 l=1 rev1=1 rx=0x7f2434006650 tx=0x7f2434009a80).stop 2021-02-10T11:46:32.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.268 7f2446297700 1 -- 172.21.2.101:0/997662479 shutdown_connections 2021-02-10T11:46:32.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.268 7f2446297700 1 --2- 172.21.2.101:0/997662479 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24401b0a10 0x7f24400cf330 unknown :-1 s=CLOSED pgs=2966 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:32.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.268 7f2446297700 1 --2- 172.21.2.101:0/997662479 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24400d1870 0x7f24401b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:32.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.268 7f2446297700 1 -- 172.21.2.101:0/997662479 >> 172.21.2.101:0/997662479 conn(0x7f24400ce450 msgr2=0x7f24400d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:32.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.268 7f2446297700 1 -- 172.21.2.101:0/997662479 shutdown_connections 2021-02-10T11:46:32.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.268 7f2446297700 1 -- 172.21.2.101:0/997662479 wait complete. 2021-02-10T11:46:32.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.282 7f2446297700 1 Processor -- start 2021-02-10T11:46:32.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.282 7f2446297700 1 -- start start 2021-02-10T11:46:32.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.282 7f2446297700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24400aa750 0x7f24400aab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:32.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.282 7f2446297700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24401acfc0 0x7f24401ad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:32.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.282 7f2446297700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24401ad9e0 0x7f24401adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:32.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.282 7f2446297700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24401ae400 con 0x7f24401ad9e0 2021-02-10T11:46:32.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.282 7f2446297700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24401ac630 con 0x7f24401acfc0 2021-02-10T11:46:32.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.282 7f2446297700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24401ac850 con 0x7f24400aa750 2021-02-10T11:46:32.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.282 7f24448b4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24401ad9e0 0x7f24401adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:32.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.283 7f24448b4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24401ad9e0 0x7f24401adde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33280/0 (socket says 172.21.2.101:33280) 2021-02-10T11:46:32.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.283 7f24448b4700 1 -- 172.21.2.101:0/3478654226 learned_addr learned my addr 172.21.2.101:0/3478654226 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:32.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.283 7f243ffff700 1 --2- 172.21.2.101:0/3478654226 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24400aa750 0x7f24400aab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:32.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.283 7f243f7fe700 1 --2- 172.21.2.101:0/3478654226 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24401acfc0 0x7f24401ad3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:32.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.283 7f24448b4700 1 -- 172.21.2.101:0/3478654226 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24400aa750 msgr2=0x7f24400aab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:32.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.283 7f24448b4700 1 --2- 172.21.2.101:0/3478654226 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24400aa750 0x7f24400aab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:32.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.283 7f24448b4700 1 -- 172.21.2.101:0/3478654226 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24401acfc0 msgr2=0x7f24401ad3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:32.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.283 7f24448b4700 1 --2- 172.21.2.101:0/3478654226 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24401acfc0 0x7f24401ad3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:32.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.283 7f24448b4700 1 -- 172.21.2.101:0/3478654226 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24400ecb10 con 0x7f24401ad9e0 2021-02-10T11:46:32.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.283 7f243f7fe700 1 --2- 172.21.2.101:0/3478654226 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24401acfc0 0x7f24401ad3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:46:32.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.283 7f24448b4700 1 --2- 172.21.2.101:0/3478654226 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24401ad9e0 0x7f24401adde0 secure :-1 s=READY pgs=2967 cs=0 l=1 rev1=1 rx=0x7f2438006f00 tx=0x7f243800b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:32.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.283 7f243cff9700 1 -- 172.21.2.101:0/3478654226 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f243800bd30 con 0x7f24401ad9e0 2021-02-10T11:46:32.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.283 7f243cff9700 1 -- 172.21.2.101:0/3478654226 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2438006020 con 0x7f24401ad9e0 2021-02-10T11:46:32.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.283 7f243cff9700 1 -- 172.21.2.101:0/3478654226 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f24380043a0 con 0x7f24401ad9e0 2021-02-10T11:46:32.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.284 7f2446297700 1 -- 172.21.2.101:0/3478654226 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24400ecb10 con 0x7f24401ad9e0 2021-02-10T11:46:32.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.284 7f2446297700 1 -- 172.21.2.101:0/3478654226 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24400ecb10 con 0x7f24401ad9e0 2021-02-10T11:46:32.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.284 7f243cff9700 1 -- 172.21.2.101:0/3478654226 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2438006200 con 0x7f24401ad9e0 2021-02-10T11:46:32.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.284 7f243cff9700 1 --2- 172.21.2.101:0/3478654226 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24200186d0 0x7f242001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:32.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.284 7f243cff9700 1 -- 172.21.2.101:0/3478654226 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f2438032440 con 0x7f24401ad9e0 2021-02-10T11:46:32.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.284 7f243ffff700 1 --2- 172.21.2.101:0/3478654226 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24200186d0 0x7f242001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:32.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.285 7f243ffff700 1 --2- 172.21.2.101:0/3478654226 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24200186d0 0x7f242001abd0 crc :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:32.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.316 7f2446297700 1 -- 172.21.2.101:0/3478654226 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24400ebc00 con 0x7f24401ad9e0 2021-02-10T11:46:32.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.318 7f243cff9700 1 -- 172.21.2.101:0/3478654226 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2438000c10 con 0x7f24401ad9e0 2021-02-10T11:46:32.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.442 7f2446297700 1 -- 172.21.2.101:0/3478654226 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f24400ebc00 con 0x7f24401ad9e0 2021-02-10T11:46:32.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.442 7f243cff9700 1 -- 172.21.2.101:0/3478654226 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v170) v1 ==== 84+0+64 (secure 0 0 0) 0x7f243800bd30 con 0x7f24401ad9e0 2021-02-10T11:46:32.440 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:46:32.440 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:46:32.440 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:32.440 INFO:teuthology.orchestra.run.gibba001.stdout: "poolnum": 1, 2021-02-10T11:46:32.440 INFO:teuthology.orchestra.run.gibba001.stdout: "poolname": "rbd" 2021-02-10T11:46:32.441 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:32.441 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:46:32.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.444 7f2446297700 1 -- 172.21.2.101:0/3478654226 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24200186d0 msgr2=0x7f242001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:32.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.444 7f2446297700 1 --2- 172.21.2.101:0/3478654226 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24200186d0 0x7f242001abd0 crc :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:32.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.444 7f2446297700 1 -- 172.21.2.101:0/3478654226 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24401ad9e0 msgr2=0x7f24401adde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:32.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.444 7f2446297700 1 --2- 172.21.2.101:0/3478654226 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24401ad9e0 0x7f24401adde0 secure :-1 s=READY pgs=2967 cs=0 l=1 rev1=1 rx=0x7f2438006f00 tx=0x7f243800b940).stop 2021-02-10T11:46:32.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.444 7f2446297700 1 -- 172.21.2.101:0/3478654226 shutdown_connections 2021-02-10T11:46:32.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.444 7f2446297700 1 --2- 172.21.2.101:0/3478654226 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24401ad9e0 0x7f24401adde0 unknown :-1 s=CLOSED pgs=2967 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:32.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.444 7f2446297700 1 --2- 172.21.2.101:0/3478654226 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24401acfc0 0x7f24401ad3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:32.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.444 7f2446297700 1 --2- 172.21.2.101:0/3478654226 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24200186d0 0x7f242001abd0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:32.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.444 7f2446297700 1 --2- 172.21.2.101:0/3478654226 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24400aa750 0x7f24400aab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:32.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.444 7f2446297700 1 -- 172.21.2.101:0/3478654226 >> 172.21.2.101:0/3478654226 conn(0x7f244019c560 msgr2=0x7f24400eedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:32.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.444 7f2446297700 1 -- 172.21.2.101:0/3478654226 shutdown_connections 2021-02-10T11:46:32.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.444 7f2446297700 1 -- 172.21.2.101:0/3478654226 wait complete. 2021-02-10T11:46:32.450 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-02-10T11:46:32.450 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-02-10T11:46:32.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.868 7f864b346700 1 Processor -- start 2021-02-10T11:46:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.868 7f864b346700 1 -- start start 2021-02-10T11:46:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.868 7f864b346700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f86440d1c70 0x7f86440cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.868 7f864b346700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86440cf580 0x7f86440cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.868 7f864b346700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86440fe1c0 con 0x7f86440d1870 2021-02-10T11:46:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.868 7f864b346700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86441b0cb0 con 0x7f86440d1c70 2021-02-10T11:46:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.868 7f864b346700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8644109a60 con 0x7f86440cf580 2021-02-10T11:46:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.870 7f8648961700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f86440d1c70 0x7f86440cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.870 7f8649963700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86440cf580 0x7f86440cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.870 7f8649963700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86440cf580 0x7f86440cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:54254/0 (socket says 172.21.2.101:54254) 2021-02-10T11:46:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.870 7f8649963700 1 -- 172.21.2.101:0/3696406558 learned_addr learned my addr 172.21.2.101:0/3696406558 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.870 7f8649162700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f86440d1870 0x7f86441b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:55424/0 (socket says 172.21.2.101:55424) 2021-02-10T11:46:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.870 7f8649963700 1 -- 172.21.2.101:0/3696406558 >> v2:172.21.2.108:3300/0 conn(0x7f86440d1c70 msgr2=0x7f86440cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:32.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.870 7f8649963700 1 --2- 172.21.2.101:0/3696406558 >> v2:172.21.2.108:3300/0 conn(0x7f86440d1c70 0x7f86440cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:32.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.870 7f8649963700 1 -- 172.21.2.101:0/3696406558 >> v1:172.21.2.108:6789/0 conn(0x7f86440d1870 legacy=0x7f86441b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:32.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.870 7f8649963700 1 -- 172.21.2.101:0/3696406558 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8644088070 con 0x7f86440cf580 2021-02-10T11:46:32.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.870 7f8649963700 1 --2- 172.21.2.101:0/3696406558 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86440cf580 0x7f86440cf980 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f863c006aa0 tx=0x7f863c009f20).ready entity=mon.2 client_cookie=1c5271ba6007ae1d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:32.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.871 7f8637fff700 1 -- 172.21.2.101:0/3696406558 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f863c00b5c0 con 0x7f86440cf580 2021-02-10T11:46:32.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.871 7f8637fff700 1 -- 172.21.2.101:0/3696406558 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863c003030 con 0x7f86440cf580 2021-02-10T11:46:32.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.871 7f8637fff700 1 -- 172.21.2.101:0/3696406558 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f863c00bb40 con 0x7f86440cf580 2021-02-10T11:46:32.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.871 7f864b346700 1 -- 172.21.2.101:0/3696406558 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86440cf580 msgr2=0x7f86440cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:32.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.871 7f864b346700 1 --2- 172.21.2.101:0/3696406558 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86440cf580 0x7f86440cf980 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f863c006aa0 tx=0x7f863c009f20).stop 2021-02-10T11:46:32.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.872 7f864b346700 1 -- 172.21.2.101:0/3696406558 shutdown_connections 2021-02-10T11:46:32.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.872 7f864b346700 1 --2- 172.21.2.101:0/3696406558 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86440cf580 0x7f86440cf980 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:32.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.872 7f864b346700 1 --2- 172.21.2.101:0/3696406558 >> v2:172.21.2.108:3300/0 conn(0x7f86440d1c70 0x7f86440cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:32.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.872 7f864b346700 1 -- 172.21.2.101:0/3696406558 >> 172.21.2.101:0/3696406558 conn(0x7f86440ce420 msgr2=0x7f86440d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:32.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.872 7f864b346700 1 -- 172.21.2.101:0/3696406558 shutdown_connections 2021-02-10T11:46:32.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.872 7f864b346700 1 -- 172.21.2.101:0/3696406558 wait complete. 2021-02-10T11:46:32.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.886 7f864b346700 1 Processor -- start 2021-02-10T11:46:32.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.886 7f864b346700 1 -- start start 2021-02-10T11:46:32.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.887 7f864b346700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8644191b10 0x7f86440ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:32.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.887 7f864b346700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8644191f10 0x7f864419ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:32.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.887 7f864b346700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86441a05f0 0x7f8644082ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:32.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.887 7f864b346700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8644192440 con 0x7f8644191b10 2021-02-10T11:46:32.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.887 7f864b346700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86441916e0 con 0x7f86441a05f0 2021-02-10T11:46:32.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.887 7f864b346700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8644191900 con 0x7f8644191f10 2021-02-10T11:46:32.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.887 7f8649162700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8644191b10 0x7f86440ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:32.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.887 7f8648961700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8644191f10 0x7f864419ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:32.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.887 7f8648961700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8644191f10 0x7f864419ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:54256/0 (socket says 172.21.2.101:54256) 2021-02-10T11:46:32.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.887 7f8649162700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8644191b10 0x7f86440ae640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33292/0 (socket says 172.21.2.101:33292) 2021-02-10T11:46:32.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.887 7f8648961700 1 -- 172.21.2.101:0/1905556613 learned_addr learned my addr 172.21.2.101:0/1905556613 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:32.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.887 7f8649963700 1 --2- 172.21.2.101:0/1905556613 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86441a05f0 0x7f8644082ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:32.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.888 7f8649162700 1 -- 172.21.2.101:0/1905556613 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8644191f10 msgr2=0x7f864419ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:32.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.888 7f8649162700 1 --2- 172.21.2.101:0/1905556613 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8644191f10 0x7f864419ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:32.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.888 7f8649162700 1 -- 172.21.2.101:0/1905556613 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86441a05f0 msgr2=0x7f8644082ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:32.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.888 7f8649162700 1 --2- 172.21.2.101:0/1905556613 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86441a05f0 0x7f8644082ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:32.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.888 7f8649162700 1 -- 172.21.2.101:0/1905556613 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86440834f0 con 0x7f8644191b10 2021-02-10T11:46:32.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.888 7f8648961700 1 --2- 172.21.2.101:0/1905556613 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8644191f10 0x7f864419ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:46:32.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.888 7f8649162700 1 --2- 172.21.2.101:0/1905556613 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8644191b10 0x7f86440ae640 secure :-1 s=READY pgs=2968 cs=0 l=1 rev1=1 rx=0x7f863800a4d0 tx=0x7f863800a870).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:32.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.888 7f8635ffb700 1 -- 172.21.2.101:0/1905556613 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8638000e20 con 0x7f8644191b10 2021-02-10T11:46:32.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.888 7f8635ffb700 1 -- 172.21.2.101:0/1905556613 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863800ac60 con 0x7f8644191b10 2021-02-10T11:46:32.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.888 7f8635ffb700 1 -- 172.21.2.101:0/1905556613 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f86380043c0 con 0x7f8644191b10 2021-02-10T11:46:32.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.888 7f864b346700 1 -- 172.21.2.101:0/1905556613 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86440834f0 con 0x7f8644191b10 2021-02-10T11:46:32.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.888 7f864b346700 1 -- 172.21.2.101:0/1905556613 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8644083a80 con 0x7f8644191b10 2021-02-10T11:46:32.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.889 7f8635ffb700 1 -- 172.21.2.101:0/1905556613 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8638004140 con 0x7f8644191b10 2021-02-10T11:46:32.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.889 7f8635ffb700 1 --2- 172.21.2.101:0/1905556613 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86240186c0 0x7f862401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:32.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.889 7f8635ffb700 1 -- 172.21.2.101:0/1905556613 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f8638031ff0 con 0x7f8644191b10 2021-02-10T11:46:32.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.889 7f8648961700 1 --2- 172.21.2.101:0/1905556613 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86240186c0 0x7f862401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:32.889 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.889 7f8648961700 1 --2- 172.21.2.101:0/1905556613 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86240186c0 0x7f862401abc0 crc :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:32.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.918 7f864b346700 1 -- 172.21.2.101:0/1905556613 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86440fe1c0 con 0x7f8644191b10 2021-02-10T11:46:32.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:32.921 7f8635ffb700 1 -- 172.21.2.101:0/1905556613 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8638000dd0 con 0x7f8644191b10 2021-02-10T11:46:33.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.048 7f864b346700 1 -- 172.21.2.101:0/1905556613 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"} v 0) v1 -- 0x7f86440fe1c0 con 0x7f8644191b10 2021-02-10T11:46:33.511 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.513 7f8635ffb700 1 -- 172.21.2.101:0/1905556613 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"}]=0 pool 'cephfs_metadata' created v171) v1 ==== 133+0+0 (secure 0 0 0) 0x7f8638004140 con 0x7f8644191b10 2021-02-10T11:46:33.513 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' created 2021-02-10T11:46:33.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.517 7f864b346700 1 -- 172.21.2.101:0/1905556613 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86240186c0 msgr2=0x7f862401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:33.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.517 7f864b346700 1 --2- 172.21.2.101:0/1905556613 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86240186c0 0x7f862401abc0 crc :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:33.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.517 7f864b346700 1 -- 172.21.2.101:0/1905556613 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8644191b10 msgr2=0x7f86440ae640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:33.515 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.517 7f864b346700 1 --2- 172.21.2.101:0/1905556613 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8644191b10 0x7f86440ae640 secure :-1 s=READY pgs=2968 cs=0 l=1 rev1=1 rx=0x7f863800a4d0 tx=0x7f863800a870).stop 2021-02-10T11:46:33.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.518 7f864b346700 1 -- 172.21.2.101:0/1905556613 shutdown_connections 2021-02-10T11:46:33.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.518 7f864b346700 1 --2- 172.21.2.101:0/1905556613 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86240186c0 0x7f862401abc0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:33.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.518 7f864b346700 1 --2- 172.21.2.101:0/1905556613 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8644191f10 0x7f864419ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:33.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.518 7f864b346700 1 --2- 172.21.2.101:0/1905556613 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86441a05f0 0x7f8644082ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:33.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.518 7f864b346700 1 --2- 172.21.2.101:0/1905556613 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8644191b10 0x7f86440ae640 unknown :-1 s=CLOSED pgs=2968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:33.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.518 7f864b346700 1 -- 172.21.2.101:0/1905556613 >> 172.21.2.101:0/1905556613 conn(0x7f86440a02d0 msgr2=0x7f864409f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:33.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.518 7f864b346700 1 -- 172.21.2.101:0/1905556613 shutdown_connections 2021-02-10T11:46:33.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.518 7f864b346700 1 -- 172.21.2.101:0/1905556613 wait complete. 2021-02-10T11:46:33.524 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_data 8 2021-02-10T11:46:33.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.941 7f938f0b9700 1 Processor -- start 2021-02-10T11:46:33.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.941 7f938f0b9700 1 -- start start 2021-02-10T11:46:33.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.941 7f938f0b9700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f93880d1870 0x7f93881b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:33.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.941 7f938f0b9700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f93881b0ae0 0x7f9388108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:33.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.942 7f938f0b9700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93880fe1c0 con 0x7f93880d1c70 2021-02-10T11:46:33.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.942 7f938f0b9700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9388109a60 con 0x7f93880d1870 2021-02-10T11:46:33.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.942 7f938f0b9700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9388109750 con 0x7f93881b0ae0 2021-02-10T11:46:33.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.942 7f938ced5700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f93880d1870 0x7f93881b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:33.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.942 7f938d6d6700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f93880d1c70 0x7f9388108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:55436/0 (socket says 172.21.2.101:55436) 2021-02-10T11:46:33.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.942 7f938ced5700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f93880d1870 0x7f93881b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33302/0 (socket says 172.21.2.101:33302) 2021-02-10T11:46:33.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.942 7f938d6d6700 1 -- 172.21.2.101:0/3732169266 learned_addr learned my addr 172.21.2.101:0/3732169266 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:33.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.942 7f9387fff700 1 --2- 172.21.2.101:0/3732169266 >> v2:172.21.2.108:3300/0 conn(0x7f93881b0ae0 0x7f9388108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:33.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.942 7f938ced5700 1 -- 172.21.2.101:0/3732169266 >> v2:172.21.2.108:3300/0 conn(0x7f93881b0ae0 msgr2=0x7f9388108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:33.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.942 7f938ced5700 1 --2- 172.21.2.101:0/3732169266 >> v2:172.21.2.108:3300/0 conn(0x7f93881b0ae0 0x7f9388108940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:33.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.942 7f938ced5700 1 -- 172.21.2.101:0/3732169266 >> v1:172.21.2.108:6789/0 conn(0x7f93880d1c70 legacy=0x7f9388108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:33.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.942 7f938ced5700 1 -- 172.21.2.101:0/3732169266 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9388088070 con 0x7f93880d1870 2021-02-10T11:46:33.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.942 7f9387fff700 1 --2- 172.21.2.101:0/3732169266 >> v2:172.21.2.108:3300/0 conn(0x7f93881b0ae0 0x7f9388108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:46:33.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.942 7f938ced5700 1 --2- 172.21.2.101:0/3732169266 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93880d1870 0x7f93881b04c0 secure :-1 s=READY pgs=2969 cs=0 l=1 rev1=1 rx=0x7f9370006650 tx=0x7f9370009a80).ready entity=mon.0 client_cookie=93c392214a51c54c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:33.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.943 7f93877fe700 1 -- 172.21.2.101:0/3732169266 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f937000c0c0 con 0x7f93880d1870 2021-02-10T11:46:33.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.943 7f93877fe700 1 -- 172.21.2.101:0/3732169266 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937000c340 con 0x7f93880d1870 2021-02-10T11:46:33.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.943 7f93877fe700 1 -- 172.21.2.101:0/3732169266 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f937000c860 con 0x7f93880d1870 2021-02-10T11:46:33.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.943 7f938f0b9700 1 -- 172.21.2.101:0/3732169266 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93880d1870 msgr2=0x7f93881b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:33.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.943 7f938f0b9700 1 --2- 172.21.2.101:0/3732169266 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93880d1870 0x7f93881b04c0 secure :-1 s=READY pgs=2969 cs=0 l=1 rev1=1 rx=0x7f9370006650 tx=0x7f9370009a80).stop 2021-02-10T11:46:33.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.943 7f938f0b9700 1 -- 172.21.2.101:0/3732169266 shutdown_connections 2021-02-10T11:46:33.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.943 7f938f0b9700 1 --2- 172.21.2.101:0/3732169266 >> v2:172.21.2.108:3300/0 conn(0x7f93881b0ae0 0x7f9388108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:33.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.943 7f938f0b9700 1 --2- 172.21.2.101:0/3732169266 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93880d1870 0x7f93881b04c0 unknown :-1 s=CLOSED pgs=2969 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:33.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.943 7f938f0b9700 1 -- 172.21.2.101:0/3732169266 >> 172.21.2.101:0/3732169266 conn(0x7f93880ce420 msgr2=0x7f93880d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:33.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.943 7f938f0b9700 1 -- 172.21.2.101:0/3732169266 shutdown_connections 2021-02-10T11:46:33.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.943 7f938f0b9700 1 -- 172.21.2.101:0/3732169266 wait complete. 2021-02-10T11:46:33.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.958 7f938f0b9700 1 Processor -- start 2021-02-10T11:46:33.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.958 7f938f0b9700 1 -- start start 2021-02-10T11:46:33.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.958 7f938f0b9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9388191b10 0x7f93880ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:33.959 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.958 7f938f0b9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9388191f10 0x7f938819ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:33.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.958 7f938f0b9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93881a05f0 0x7f9388082ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:33.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.958 7f938f0b9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9388192440 con 0x7f93881a05f0 2021-02-10T11:46:33.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.958 7f938f0b9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93881916e0 con 0x7f9388191b10 2021-02-10T11:46:33.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.958 7f938f0b9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9388191900 con 0x7f9388191f10 2021-02-10T11:46:33.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.959 7f938d6d6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93881a05f0 0x7f9388082ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:33.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.959 7f938ced5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9388191b10 0x7f93880ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:33.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.959 7f938d6d6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93881a05f0 0x7f9388082ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33306/0 (socket says 172.21.2.101:33306) 2021-02-10T11:46:33.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.959 7f938d6d6700 1 -- 172.21.2.101:0/880285293 learned_addr learned my addr 172.21.2.101:0/880285293 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:33.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.959 7f938ced5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9388191b10 0x7f93880ae640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:43790/0 (socket says 172.21.2.101:43790) 2021-02-10T11:46:33.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.959 7f9387fff700 1 --2- 172.21.2.101:0/880285293 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9388191f10 0x7f938819ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:33.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.959 7f938ced5700 1 -- 172.21.2.101:0/880285293 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9388191f10 msgr2=0x7f938819ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:33.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.959 7f938ced5700 1 --2- 172.21.2.101:0/880285293 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9388191f10 0x7f938819ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:33.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.959 7f938ced5700 1 -- 172.21.2.101:0/880285293 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93881a05f0 msgr2=0x7f9388082ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:33.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.959 7f938ced5700 1 --2- 172.21.2.101:0/880285293 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93881a05f0 0x7f9388082ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:33.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.959 7f938ced5700 1 -- 172.21.2.101:0/880285293 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93880834f0 con 0x7f9388191b10 2021-02-10T11:46:33.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.959 7f938d6d6700 1 --2- 172.21.2.101:0/880285293 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93881a05f0 0x7f9388082ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:46:33.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.959 7f938ced5700 1 --2- 172.21.2.101:0/880285293 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9388191b10 0x7f93880ae640 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f937000b4b0 tx=0x7f937000b850).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:33.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.959 7f93857fa700 1 -- 172.21.2.101:0/880285293 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9370009280 con 0x7f9388191b10 2021-02-10T11:46:33.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.960 7f93857fa700 1 -- 172.21.2.101:0/880285293 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937000bc40 con 0x7f9388191b10 2021-02-10T11:46:33.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.960 7f938f0b9700 1 -- 172.21.2.101:0/880285293 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93880834f0 con 0x7f9388191b10 2021-02-10T11:46:33.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.960 7f93857fa700 1 -- 172.21.2.101:0/880285293 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9370009280 con 0x7f9388191b10 2021-02-10T11:46:33.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.960 7f938f0b9700 1 -- 172.21.2.101:0/880285293 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93880834f0 con 0x7f9388191b10 2021-02-10T11:46:33.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.960 7f93857fa700 1 -- 172.21.2.101:0/880285293 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f93700052e0 con 0x7f9388191b10 2021-02-10T11:46:33.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.960 7f93857fa700 1 --2- 172.21.2.101:0/880285293 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f936c0186c0 0x7f936c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:33.961 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.960 7f93857fa700 1 -- 172.21.2.101:0/880285293 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5107+0+0 (secure 0 0 0) 0x7f9370032310 con 0x7f9388191b10 2021-02-10T11:46:33.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.960 7f9387fff700 1 --2- 172.21.2.101:0/880285293 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f936c0186c0 0x7f936c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:33.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.961 7f9387fff700 1 --2- 172.21.2.101:0/880285293 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f936c0186c0 0x7f936c01abc0 crc :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:33.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.991 7f938f0b9700 1 -- 172.21.2.101:0/880285293 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9388192440 con 0x7f9388191b10 2021-02-10T11:46:33.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:33.994 7f93857fa700 1 -- 172.21.2.101:0/880285293 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9370000c90 con 0x7f9388191b10 2021-02-10T11:46:34.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.123 7f938f0b9700 1 -- 172.21.2.101:0/880285293 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"} v 0) v1 -- 0x7f9388192440 con 0x7f9388191b10 2021-02-10T11:46:34.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.521 7f93857fa700 1 -- 172.21.2.101:0/880285293 <== mon.1 v2:172.21.2.108:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"}]=0 pool 'cephfs_data' created v173) v1 ==== 125+0+0 (secure 0 0 0) 0x7f93700052e0 con 0x7f9388191b10 2021-02-10T11:46:34.536 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' created 2021-02-10T11:46:34.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.540 7f938f0b9700 1 -- 172.21.2.101:0/880285293 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f936c0186c0 msgr2=0x7f936c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:34.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.540 7f938f0b9700 1 --2- 172.21.2.101:0/880285293 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f936c0186c0 0x7f936c01abc0 crc :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:34.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.540 7f938f0b9700 1 -- 172.21.2.101:0/880285293 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9388191b10 msgr2=0x7f93880ae640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:34.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.540 7f938f0b9700 1 --2- 172.21.2.101:0/880285293 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9388191b10 0x7f93880ae640 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f937000b4b0 tx=0x7f937000b850).stop 2021-02-10T11:46:34.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.541 7f938f0b9700 1 -- 172.21.2.101:0/880285293 shutdown_connections 2021-02-10T11:46:34.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.541 7f938f0b9700 1 --2- 172.21.2.101:0/880285293 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f936c0186c0 0x7f936c01abc0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:34.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.541 7f938f0b9700 1 --2- 172.21.2.101:0/880285293 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9388191f10 0x7f938819ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:34.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.541 7f938f0b9700 1 --2- 172.21.2.101:0/880285293 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93881a05f0 0x7f9388082ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:34.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.541 7f938f0b9700 1 --2- 172.21.2.101:0/880285293 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9388191b10 0x7f93880ae640 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:34.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.541 7f938f0b9700 1 -- 172.21.2.101:0/880285293 >> 172.21.2.101:0/880285293 conn(0x7f93880a02d0 msgr2=0x7f938809f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:34.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.541 7f938f0b9700 1 -- 172.21.2.101:0/880285293 shutdown_connections 2021-02-10T11:46:34.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.541 7f938f0b9700 1 -- 172.21.2.101:0/880285293 wait complete. 2021-02-10T11:46:34.547 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-02-10T11:46:34.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.965 7fa5cf781700 1 Processor -- start 2021-02-10T11:46:34.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.965 7fa5cf781700 1 -- start start 2021-02-10T11:46:34.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.965 7fa5cf781700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5c80d1880 0x7fa5c81b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:34.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.965 7fa5cf781700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa5c81b0af0 0x7fa5c8108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:34.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.965 7fa5cf781700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa5c80d1c80 0x7fa5c8108f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:34.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.965 7fa5cf781700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c80e5f00 con 0x7fa5c81b0af0 2021-02-10T11:46:34.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.965 7fa5cf781700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c8109a70 con 0x7fa5c80d1c80 2021-02-10T11:46:34.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.965 7fa5cf781700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c8088070 con 0x7fa5c80d1880 2021-02-10T11:46:34.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.966 7fa5ccd9c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa5c81b0af0 0x7fa5c8108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:34.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.966 7fa5ccd9c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa5c81b0af0 0x7fa5c8108950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33314/0 (socket says 172.21.2.101:33314) 2021-02-10T11:46:34.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.966 7fa5ccd9c700 1 -- 172.21.2.101:0/1454370368 learned_addr learned my addr 172.21.2.101:0/1454370368 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:34.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.966 7fa5ccd9c700 1 -- 172.21.2.101:0/1454370368 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5c80d1880 msgr2=0x7fa5c81b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:34.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.966 7fa5ccd9c700 1 --2- 172.21.2.101:0/1454370368 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5c80d1880 0x7fa5c81b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:34.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.966 7fa5ccd9c700 1 -- 172.21.2.101:0/1454370368 >> v2:172.21.2.108:3300/0 conn(0x7fa5c80d1c80 msgr2=0x7fa5c8108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:34.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.966 7fa5ccd9c700 1 --2- 172.21.2.101:0/1454370368 >> v2:172.21.2.108:3300/0 conn(0x7fa5c80d1c80 0x7fa5c8108f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:34.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.966 7fa5ccd9c700 1 -- 172.21.2.101:0/1454370368 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c8109590 con 0x7fa5c81b0af0 2021-02-10T11:46:34.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.966 7fa5ccd9c700 1 --2- 172.21.2.101:0/1454370368 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5c81b0af0 0x7fa5c8108950 secure :-1 s=READY pgs=2970 cs=0 l=1 rev1=1 rx=0x7fa5bc006700 tx=0x7fa5bc009b00).ready entity=mon.0 client_cookie=ea9538e0ab411527 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:34.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.966 7fa5bbfff700 1 -- 172.21.2.101:0/1454370368 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa5bc00c0c0 con 0x7fa5c81b0af0 2021-02-10T11:46:34.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.966 7fa5bbfff700 1 -- 172.21.2.101:0/1454370368 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5bc00c400 con 0x7fa5c81b0af0 2021-02-10T11:46:34.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.966 7fa5bbfff700 1 -- 172.21.2.101:0/1454370368 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa5bc00c920 con 0x7fa5c81b0af0 2021-02-10T11:46:34.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.966 7fa5cf781700 1 -- 172.21.2.101:0/1454370368 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5c81b0af0 msgr2=0x7fa5c8108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:34.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.966 7fa5cf781700 1 --2- 172.21.2.101:0/1454370368 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5c81b0af0 0x7fa5c8108950 secure :-1 s=READY pgs=2970 cs=0 l=1 rev1=1 rx=0x7fa5bc006700 tx=0x7fa5bc009b00).stop 2021-02-10T11:46:34.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.967 7fa5cf781700 1 -- 172.21.2.101:0/1454370368 shutdown_connections 2021-02-10T11:46:34.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.967 7fa5cf781700 1 --2- 172.21.2.101:0/1454370368 >> v2:172.21.2.108:3300/0 conn(0x7fa5c80d1c80 0x7fa5c8108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:34.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.967 7fa5cf781700 1 --2- 172.21.2.101:0/1454370368 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5c81b0af0 0x7fa5c8108950 unknown :-1 s=CLOSED pgs=2970 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:34.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.967 7fa5cf781700 1 --2- 172.21.2.101:0/1454370368 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5c80d1880 0x7fa5c81b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:34.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.967 7fa5cf781700 1 -- 172.21.2.101:0/1454370368 >> 172.21.2.101:0/1454370368 conn(0x7fa5c80ce430 msgr2=0x7fa5c80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:34.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.967 7fa5cf781700 1 -- 172.21.2.101:0/1454370368 shutdown_connections 2021-02-10T11:46:34.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.967 7fa5cf781700 1 -- 172.21.2.101:0/1454370368 wait complete. 2021-02-10T11:46:34.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.982 7fa5cf781700 1 Processor -- start 2021-02-10T11:46:34.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.982 7fa5cf781700 1 -- start start 2021-02-10T11:46:34.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.982 7fa5cf781700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa5c8096710 0x7fa5c8096f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:34.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.982 7fa5cf781700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5c819c360 0x7fa5c80ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:34.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.982 7fa5cf781700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5c80adb40 0x7fa5c819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:34.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.982 7fa5cf781700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c819c970 con 0x7fa5c80adb40 2021-02-10T11:46:34.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.982 7fa5cf781700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c8096b10 con 0x7fa5c8096710 2021-02-10T11:46:34.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.982 7fa5cf781700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c8082c60 con 0x7fa5c819c360 2021-02-10T11:46:34.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.983 7fa5cdd9e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5c80adb40 0x7fa5c819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:34.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.983 7fa5cdd9e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5c80adb40 0x7fa5c819cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33322/0 (socket says 172.21.2.101:33322) 2021-02-10T11:46:34.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.983 7fa5cdd9e700 1 -- 172.21.2.101:0/508380418 learned_addr learned my addr 172.21.2.101:0/508380418 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:34.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.983 7fa5cdd9e700 1 -- 172.21.2.101:0/508380418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5c819c360 msgr2=0x7fa5c80ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:34.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.983 7fa5cdd9e700 1 --2- 172.21.2.101:0/508380418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5c819c360 0x7fa5c80ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:34.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.983 7fa5cdd9e700 1 -- 172.21.2.101:0/508380418 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa5c8096710 msgr2=0x7fa5c8096f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:34.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.983 7fa5cdd9e700 1 --2- 172.21.2.101:0/508380418 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa5c8096710 0x7fa5c8096f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:34.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.983 7fa5cdd9e700 1 -- 172.21.2.101:0/508380418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c8097590 con 0x7fa5c80adb40 2021-02-10T11:46:34.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.983 7fa5cdd9e700 1 --2- 172.21.2.101:0/508380418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5c80adb40 0x7fa5c819cef0 secure :-1 s=READY pgs=2971 cs=0 l=1 rev1=1 rx=0x7fa5c0009980 tx=0x7fa5c000bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:34.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.983 7fa5b9ffb700 1 -- 172.21.2.101:0/508380418 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa5c0003020 con 0x7fa5c80adb40 2021-02-10T11:46:34.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.983 7fa5b9ffb700 1 -- 172.21.2.101:0/508380418 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c0005a80 con 0x7fa5c80adb40 2021-02-10T11:46:34.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.983 7fa5cf781700 1 -- 172.21.2.101:0/508380418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c8097590 con 0x7fa5c80adb40 2021-02-10T11:46:34.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.983 7fa5cf781700 1 -- 172.21.2.101:0/508380418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c80ae230 con 0x7fa5c80adb40 2021-02-10T11:46:34.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.983 7fa5b9ffb700 1 -- 172.21.2.101:0/508380418 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa5c0006030 con 0x7fa5c80adb40 2021-02-10T11:46:34.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.984 7fa5b9ffb700 1 -- 172.21.2.101:0/508380418 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa5c0005c60 con 0x7fa5c80adb40 2021-02-10T11:46:34.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.984 7fa5b9ffb700 1 --2- 172.21.2.101:0/508380418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5a80186c0 0x7fa5a801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:34.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.984 7fa5b9ffb700 1 -- 172.21.2.101:0/508380418 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5457+0+0 (secure 0 0 0) 0x7fa5c00328e0 con 0x7fa5c80adb40 2021-02-10T11:46:34.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.984 7fa5cd59d700 1 --2- 172.21.2.101:0/508380418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5a80186c0 0x7fa5a801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:34.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:34.984 7fa5cd59d700 1 --2- 172.21.2.101:0/508380418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5a80186c0 0x7fa5a801abc0 crc :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:35.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.016 7fa5cf781700 1 -- 172.21.2.101:0/508380418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c8109a70 con 0x7fa5c80adb40 2021-02-10T11:46:35.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.018 7fa5b9ffb700 1 -- 172.21.2.101:0/508380418 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa5c00062f0 con 0x7fa5c80adb40 2021-02-10T11:46:35.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.148 7fa5cf781700 1 -- 172.21.2.101:0/508380418 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"} v 0) v1 -- 0x7fa5c8109a70 con 0x7fa5c80adb40 2021-02-10T11:46:35.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.153 7fa5b9ffb700 1 -- 172.21.2.101:0/508380418 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"}]=0 new fs with metadata pool 36 and data pool 37 v145) v1 ==== 189+0+0 (secure 0 0 0) 0x7fa5c0005c60 con 0x7fa5c80adb40 2021-02-10T11:46:35.155 INFO:teuthology.orchestra.run.gibba001.stderr:new fs with metadata pool 36 and data pool 37 2021-02-10T11:46:35.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.159 7fa5cf781700 1 -- 172.21.2.101:0/508380418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5a80186c0 msgr2=0x7fa5a801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:35.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.159 7fa5cf781700 1 --2- 172.21.2.101:0/508380418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5a80186c0 0x7fa5a801abc0 crc :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:35.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.159 7fa5cf781700 1 -- 172.21.2.101:0/508380418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5c80adb40 msgr2=0x7fa5c819cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:35.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.159 7fa5cf781700 1 --2- 172.21.2.101:0/508380418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5c80adb40 0x7fa5c819cef0 secure :-1 s=READY pgs=2971 cs=0 l=1 rev1=1 rx=0x7fa5c0009980 tx=0x7fa5c000bcd0).stop 2021-02-10T11:46:35.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.160 7fa5cf781700 1 -- 172.21.2.101:0/508380418 shutdown_connections 2021-02-10T11:46:35.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.160 7fa5cf781700 1 --2- 172.21.2.101:0/508380418 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa5a80186c0 0x7fa5a801abc0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:35.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.160 7fa5cf781700 1 --2- 172.21.2.101:0/508380418 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa5c819c360 0x7fa5c80ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:35.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.160 7fa5cf781700 1 --2- 172.21.2.101:0/508380418 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa5c80adb40 0x7fa5c819cef0 unknown :-1 s=CLOSED pgs=2971 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:35.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.160 7fa5cf781700 1 --2- 172.21.2.101:0/508380418 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa5c8096710 0x7fa5c8096f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:35.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.160 7fa5cf781700 1 -- 172.21.2.101:0/508380418 >> 172.21.2.101:0/508380418 conn(0x7fa5c819e990 msgr2=0x7fa5c81918e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:35.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.160 7fa5cf781700 1 -- 172.21.2.101:0/508380418 shutdown_connections 2021-02-10T11:46:35.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.160 7fa5cf781700 1 -- 172.21.2.101:0/508380418 wait complete. 2021-02-10T11:46:35.165 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:46:35.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.580 7fd2ad0b8700 1 Processor -- start 2021-02-10T11:46:35.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.580 7fd2ad0b8700 1 -- start start 2021-02-10T11:46:35.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.580 7fd2ad0b8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd2a80d1870 0x7fd2a81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:35.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.580 7fd2ad0b8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2a81b0a10 0x7fd2a80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:35.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.580 7fd2ad0b8700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a80f0b10 con 0x7fd2a80d1c70 2021-02-10T11:46:35.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.580 7fd2ad0b8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2a80e5ed0 con 0x7fd2a80d1870 2021-02-10T11:46:35.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.580 7fd2ad0b8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a80f0650 con 0x7fd2a81b0a10 2021-02-10T11:46:35.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.580 7fd2a6d9d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd2a80d1870 0x7fd2a81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:35.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.580 7fd2a759e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd2a80d1c70 0x7fd2a80cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33820/0 (socket says 172.21.2.101:33820) 2021-02-10T11:46:35.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.580 7fd2a6d9d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd2a80d1870 0x7fd2a81b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33326/0 (socket says 172.21.2.101:33326) 2021-02-10T11:46:35.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.580 7fd2a759e700 1 -- 172.21.2.101:0/2487737803 learned_addr learned my addr 172.21.2.101:0/2487737803 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:35.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.581 7fd2a6d9d700 1 -- 172.21.2.101:0/2487737803 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2a81b0a10 msgr2=0x7fd2a80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:35.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.581 7fd2a6d9d700 1 --2- 172.21.2.101:0/2487737803 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2a81b0a10 0x7fd2a80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:35.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.581 7fd2a6d9d700 1 -- 172.21.2.101:0/2487737803 >> v1:172.21.2.101:6789/0 conn(0x7fd2a80d1c70 legacy=0x7fd2a80cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:35.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.581 7fd2a6d9d700 1 -- 172.21.2.101:0/2487737803 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a80f0870 con 0x7fd2a80d1870 2021-02-10T11:46:35.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.581 7fd2a6d9d700 1 --2- 172.21.2.101:0/2487737803 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2a80d1870 0x7fd2a81b03f0 secure :-1 s=READY pgs=2973 cs=0 l=1 rev1=1 rx=0x7fd298006650 tx=0x7fd298009a80).ready entity=mon.0 client_cookie=f5a9e5da73bf0ec9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:35.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.581 7fd2a5d9b700 1 -- 172.21.2.101:0/2487737803 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd29800c0c0 con 0x7fd2a80d1870 2021-02-10T11:46:35.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.582 7fd2a5d9b700 1 -- 172.21.2.101:0/2487737803 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29800c340 con 0x7fd2a80d1870 2021-02-10T11:46:35.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.582 7fd2a5d9b700 1 -- 172.21.2.101:0/2487737803 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd29800c860 con 0x7fd2a80d1870 2021-02-10T11:46:35.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.582 7fd2ad0b8700 1 -- 172.21.2.101:0/2487737803 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2a80d1870 msgr2=0x7fd2a81b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:35.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.582 7fd2ad0b8700 1 --2- 172.21.2.101:0/2487737803 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2a80d1870 0x7fd2a81b03f0 secure :-1 s=READY pgs=2973 cs=0 l=1 rev1=1 rx=0x7fd298006650 tx=0x7fd298009a80).stop 2021-02-10T11:46:35.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.582 7fd2ad0b8700 1 -- 172.21.2.101:0/2487737803 shutdown_connections 2021-02-10T11:46:35.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.582 7fd2ad0b8700 1 --2- 172.21.2.101:0/2487737803 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2a81b0a10 0x7fd2a80cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:35.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.582 7fd2ad0b8700 1 --2- 172.21.2.101:0/2487737803 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2a80d1870 0x7fd2a81b03f0 unknown :-1 s=CLOSED pgs=2973 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:35.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.582 7fd2ad0b8700 1 -- 172.21.2.101:0/2487737803 >> 172.21.2.101:0/2487737803 conn(0x7fd2a80ce450 msgr2=0x7fd2a80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:35.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.583 7fd2ad0b8700 1 -- 172.21.2.101:0/2487737803 shutdown_connections 2021-02-10T11:46:35.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.583 7fd2ad0b8700 1 -- 172.21.2.101:0/2487737803 wait complete. 2021-02-10T11:46:35.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.597 7fd2ad0b8700 1 Processor -- start 2021-02-10T11:46:35.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.597 7fd2ad0b8700 1 -- start start 2021-02-10T11:46:35.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.597 7fd2ad0b8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2a808a450 0x7fd2a8095810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:35.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.597 7fd2ad0b8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2a81a2f60 0x7fd2a808afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:35.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.597 7fd2ad0b8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2a81a3360 0x7fd2a808b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:35.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.597 7fd2ad0b8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a808a850 con 0x7fd2a808a450 2021-02-10T11:46:35.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.597 7fd2ad0b8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a80a7750 con 0x7fd2a81a2f60 2021-02-10T11:46:35.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.597 7fd2ad0b8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a808aac0 con 0x7fd2a81a3360 2021-02-10T11:46:35.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.598 7fd2a6d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2a808a450 0x7fd2a8095810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:35.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.598 7fd2a6d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2a808a450 0x7fd2a8095810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33330/0 (socket says 172.21.2.101:33330) 2021-02-10T11:46:35.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.598 7fd2a6d9d700 1 -- 172.21.2.101:0/1275829900 learned_addr learned my addr 172.21.2.101:0/1275829900 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:35.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.598 7fd2a759e700 1 --2- 172.21.2.101:0/1275829900 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2a81a3360 0x7fd2a808b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:35.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.598 7fd2a659c700 1 --2- 172.21.2.101:0/1275829900 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2a81a2f60 0x7fd2a808afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:35.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.598 7fd2a6d9d700 1 -- 172.21.2.101:0/1275829900 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2a81a3360 msgr2=0x7fd2a808b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:35.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.598 7fd2a6d9d700 1 --2- 172.21.2.101:0/1275829900 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2a81a3360 0x7fd2a808b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:35.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.598 7fd2a6d9d700 1 -- 172.21.2.101:0/1275829900 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2a81a2f60 msgr2=0x7fd2a808afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:35.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.598 7fd2a6d9d700 1 --2- 172.21.2.101:0/1275829900 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2a81a2f60 0x7fd2a808afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:35.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.598 7fd2a6d9d700 1 -- 172.21.2.101:0/1275829900 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a8095fa0 con 0x7fd2a808a450 2021-02-10T11:46:35.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.598 7fd2a659c700 1 --2- 172.21.2.101:0/1275829900 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2a81a2f60 0x7fd2a808afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:46:35.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.598 7fd2a6d9d700 1 --2- 172.21.2.101:0/1275829900 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2a808a450 0x7fd2a8095810 secure :-1 s=READY pgs=2974 cs=0 l=1 rev1=1 rx=0x7fd298009630 tx=0x7fd29800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:35.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.599 7fd2977fe700 1 -- 172.21.2.101:0/1275829900 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2980093a0 con 0x7fd2a808a450 2021-02-10T11:46:35.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.599 7fd2977fe700 1 -- 172.21.2.101:0/1275829900 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29800bd70 con 0x7fd2a808a450 2021-02-10T11:46:35.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.599 7fd2977fe700 1 -- 172.21.2.101:0/1275829900 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd298005560 con 0x7fd2a808a450 2021-02-10T11:46:35.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.599 7fd2ad0b8700 1 -- 172.21.2.101:0/1275829900 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a8095fa0 con 0x7fd2a808a450 2021-02-10T11:46:35.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.599 7fd2ad0b8700 1 -- 172.21.2.101:0/1275829900 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a808bf00 con 0x7fd2a808a450 2021-02-10T11:46:35.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.600 7fd2977fe700 1 -- 172.21.2.101:0/1275829900 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd298000c90 con 0x7fd2a808a450 2021-02-10T11:46:35.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.600 7fd2977fe700 1 --2- 172.21.2.101:0/1275829900 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd28c0186c0 0x7fd28c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:35.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.600 7fd2977fe700 1 -- 172.21.2.101:0/1275829900 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd2980325a0 con 0x7fd2a808a450 2021-02-10T11:46:35.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.600 7fd2a659c700 1 --2- 172.21.2.101:0/1275829900 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd28c0186c0 0x7fd28c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:35.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.600 7fd2a659c700 1 --2- 172.21.2.101:0/1275829900 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd28c0186c0 0x7fd28c01abc0 crc :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:35.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.630 7fd2ad0b8700 1 -- 172.21.2.101:0/1275829900 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a80a7710 con 0x7fd2a808a450 2021-02-10T11:46:35.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.633 7fd2977fe700 1 -- 172.21.2.101:0/1275829900 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd298030d50 con 0x7fd2a808a450 2021-02-10T11:46:35.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.758 7fd2ad0b8700 1 -- 172.21.2.101:0/1275829900 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2a80a7710 con 0x7fd2a808a450 2021-02-10T11:46:35.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.758 7fd2977fe700 1 -- 172.21.2.101:0/1275829900 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) v1 ==== 74+0+14020 (secure 0 0 0) 0x7fd298000c90 con 0x7fd2a808a450 2021-02-10T11:46:35.756 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:46:35.757 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":174,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:46:35.149829","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":37,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":36,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:46:33.067171","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":37,"pool_name":"cephfs_data","create_time":"2021-02-10 11:46:34.142991","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:46:35.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.760 7fd2ad0b8700 1 -- 172.21.2.101:0/1275829900 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd28c0186c0 msgr2=0x7fd28c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:35.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.760 7fd2ad0b8700 1 --2- 172.21.2.101:0/1275829900 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd28c0186c0 0x7fd28c01abc0 crc :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:35.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.760 7fd2ad0b8700 1 -- 172.21.2.101:0/1275829900 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2a808a450 msgr2=0x7fd2a8095810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:35.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.760 7fd2ad0b8700 1 --2- 172.21.2.101:0/1275829900 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2a808a450 0x7fd2a8095810 secure :-1 s=READY pgs=2974 cs=0 l=1 rev1=1 rx=0x7fd298009630 tx=0x7fd29800b980).stop 2021-02-10T11:46:35.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.760 7fd2ad0b8700 1 -- 172.21.2.101:0/1275829900 shutdown_connections 2021-02-10T11:46:35.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.760 7fd2ad0b8700 1 --2- 172.21.2.101:0/1275829900 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd28c0186c0 0x7fd28c01abc0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:35.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.760 7fd2ad0b8700 1 --2- 172.21.2.101:0/1275829900 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2a81a3360 0x7fd2a808b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:35.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.760 7fd2ad0b8700 1 --2- 172.21.2.101:0/1275829900 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2a81a2f60 0x7fd2a808afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:35.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.760 7fd2ad0b8700 1 --2- 172.21.2.101:0/1275829900 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2a808a450 0x7fd2a8095810 unknown :-1 s=CLOSED pgs=2974 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:35.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.760 7fd2ad0b8700 1 -- 172.21.2.101:0/1275829900 >> 172.21.2.101:0/1275829900 conn(0x7fd2a81b9b90 msgr2=0x7fd2a818c260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:35.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.760 7fd2ad0b8700 1 -- 172.21.2.101:0/1275829900 shutdown_connections 2021-02-10T11:46:35.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:35.760 7fd2ad0b8700 1 -- 172.21.2.101:0/1275829900 wait complete. 2021-02-10T11:46:35.767 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:46:36.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.184 7fb179c54700 1 Processor -- start 2021-02-10T11:46:36.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.184 7fb179c54700 1 -- start start 2021-02-10T11:46:36.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.184 7fb179c54700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1740d0340 0x7fb1740ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:36.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.184 7fb179c54700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb1740d1c70 0x7fb1740efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:36.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.184 7fb179c54700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1740f0b10 con 0x7fb1740d1870 2021-02-10T11:46:36.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.184 7fb179c54700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1740e5ed0 con 0x7fb1740d1c70 2021-02-10T11:46:36.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.184 7fb179c54700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1741b0be0 con 0x7fb1740d0340 2021-02-10T11:46:36.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.185 7fb1737fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb1740d1870 0x7fb1741b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33832/0 (socket says 172.21.2.101:33832) 2021-02-10T11:46:36.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.185 7fb1737fe700 1 -- 172.21.2.101:0/4216585532 learned_addr learned my addr 172.21.2.101:0/4216585532 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:36.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.185 7fb172ffd700 1 --2- 172.21.2.101:0/4216585532 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1740d0340 0x7fb1740ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:36.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.185 7fb173fff700 1 --2- 172.21.2.101:0/4216585532 >> v2:172.21.2.108:3300/0 conn(0x7fb1740d1c70 0x7fb1740efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:36.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.185 7fb1727fc700 1 -- 172.21.2.101:0/4216585532 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2968603024 0 0) 0x7fb164001180 con 0x7fb1740d1870 2021-02-10T11:46:36.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.185 7fb1727fc700 1 -- 172.21.2.101:0/4216585532 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb160002fe0 con 0x7fb1740d1870 2021-02-10T11:46:36.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.185 7fb173fff700 1 -- 172.21.2.101:0/4216585532 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1740d0340 msgr2=0x7fb1740ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:36.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.185 7fb173fff700 1 --2- 172.21.2.101:0/4216585532 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1740d0340 0x7fb1740ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:36.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.185 7fb173fff700 1 -- 172.21.2.101:0/4216585532 >> v1:172.21.2.101:6789/0 conn(0x7fb1740d1870 legacy=0x7fb1741b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:36.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.185 7fb173fff700 1 -- 172.21.2.101:0/4216585532 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1740f20a0 con 0x7fb1740d1c70 2021-02-10T11:46:36.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.185 7fb172ffd700 1 --2- 172.21.2.101:0/4216585532 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1740d0340 0x7fb1740ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:46:36.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.185 7fb173fff700 1 --2- 172.21.2.101:0/4216585532 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1740d1c70 0x7fb1740efc60 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7fb16c006e50 tx=0x7fb16c009e70).ready entity=mon.1 client_cookie=98cff26c152bbf82 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:36.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.185 7fb1727fc700 1 -- 172.21.2.101:0/4216585532 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb16c006a20 con 0x7fb1740d1c70 2021-02-10T11:46:36.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.186 7fb1727fc700 1 -- 172.21.2.101:0/4216585532 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16c006790 con 0x7fb1740d1c70 2021-02-10T11:46:36.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.186 7fb1727fc700 1 -- 172.21.2.101:0/4216585532 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb16c00b700 con 0x7fb1740d1c70 2021-02-10T11:46:36.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.186 7fb179c54700 1 -- 172.21.2.101:0/4216585532 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1740d1c70 msgr2=0x7fb1740efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:36.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.186 7fb179c54700 1 --2- 172.21.2.101:0/4216585532 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1740d1c70 0x7fb1740efc60 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7fb16c006e50 tx=0x7fb16c009e70).stop 2021-02-10T11:46:36.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.186 7fb179c54700 1 -- 172.21.2.101:0/4216585532 shutdown_connections 2021-02-10T11:46:36.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.186 7fb179c54700 1 --2- 172.21.2.101:0/4216585532 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1740d0340 0x7fb1740ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:36.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.186 7fb179c54700 1 --2- 172.21.2.101:0/4216585532 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1740d1c70 0x7fb1740efc60 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:36.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.186 7fb179c54700 1 -- 172.21.2.101:0/4216585532 >> 172.21.2.101:0/4216585532 conn(0x7fb1740ce450 msgr2=0x7fb1740d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:36.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.186 7fb179c54700 1 -- 172.21.2.101:0/4216585532 shutdown_connections 2021-02-10T11:46:36.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.186 7fb179c54700 1 -- 172.21.2.101:0/4216585532 wait complete. 2021-02-10T11:46:36.202 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.201 7fb179c54700 1 Processor -- start 2021-02-10T11:46:36.202 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.201 7fb179c54700 1 -- start start 2021-02-10T11:46:36.202 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.202 7fb179c54700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1741942b0 0x7fb1740b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:36.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.202 7fb179c54700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb174195070 0x7fb174195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:36.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.202 7fb179c54700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1740bb8e0 0x7fb1740bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:36.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.202 7fb179c54700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1741946b0 con 0x7fb174195070 2021-02-10T11:46:36.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.202 7fb179c54700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1740b3b70 con 0x7fb1741942b0 2021-02-10T11:46:36.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.202 7fb179c54700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1740b3d90 con 0x7fb1740bb8e0 2021-02-10T11:46:36.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.202 7fb172ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb174195070 0x7fb174195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:36.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.202 7fb172ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb174195070 0x7fb174195470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33348/0 (socket says 172.21.2.101:33348) 2021-02-10T11:46:36.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.202 7fb172ffd700 1 -- 172.21.2.101:0/2487037419 learned_addr learned my addr 172.21.2.101:0/2487037419 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:36.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.202 7fb1737fe700 1 --2- 172.21.2.101:0/2487037419 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1741942b0 0x7fb1740b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:36.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.202 7fb173fff700 1 --2- 172.21.2.101:0/2487037419 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1740bb8e0 0x7fb1740bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:36.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.202 7fb172ffd700 1 -- 172.21.2.101:0/2487037419 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1740bb8e0 msgr2=0x7fb1740bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:36.203 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.202 7fb172ffd700 1 --2- 172.21.2.101:0/2487037419 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1740bb8e0 0x7fb1740bbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:36.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.202 7fb172ffd700 1 -- 172.21.2.101:0/2487037419 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1741942b0 msgr2=0x7fb1740b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:36.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.202 7fb172ffd700 1 --2- 172.21.2.101:0/2487037419 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1741942b0 0x7fb1740b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:36.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.202 7fb172ffd700 1 -- 172.21.2.101:0/2487037419 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1741a49e0 con 0x7fb174195070 2021-02-10T11:46:36.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.202 7fb173fff700 1 --2- 172.21.2.101:0/2487037419 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1740bb8e0 0x7fb1740bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:46:36.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.202 7fb1737fe700 1 --2- 172.21.2.101:0/2487037419 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1741942b0 0x7fb1740b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:46:36.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.203 7fb172ffd700 1 --2- 172.21.2.101:0/2487037419 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb174195070 0x7fb174195470 secure :-1 s=READY pgs=2976 cs=0 l=1 rev1=1 rx=0x7fb15c006f40 tx=0x7fb15c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:36.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.203 7fb16bfff700 1 -- 172.21.2.101:0/2487037419 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb15c0009b0 con 0x7fb174195070 2021-02-10T11:46:36.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.203 7fb16bfff700 1 -- 172.21.2.101:0/2487037419 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb15c00bd80 con 0x7fb174195070 2021-02-10T11:46:36.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.203 7fb16bfff700 1 -- 172.21.2.101:0/2487037419 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb15c002220 con 0x7fb174195070 2021-02-10T11:46:36.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.203 7fb179c54700 1 -- 172.21.2.101:0/2487037419 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1741a49e0 con 0x7fb174195070 2021-02-10T11:46:36.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.203 7fb179c54700 1 -- 172.21.2.101:0/2487037419 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1741a49e0 con 0x7fb174195070 2021-02-10T11:46:36.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.204 7fb16bfff700 1 -- 172.21.2.101:0/2487037419 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb15c000c90 con 0x7fb174195070 2021-02-10T11:46:36.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.204 7fb16bfff700 1 --2- 172.21.2.101:0/2487037419 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1580186c0 0x7fb15801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:36.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.204 7fb16bfff700 1 -- 172.21.2.101:0/2487037419 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb15c0325b0 con 0x7fb174195070 2021-02-10T11:46:36.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.204 7fb1737fe700 1 --2- 172.21.2.101:0/2487037419 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1580186c0 0x7fb15801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:36.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.204 7fb1737fe700 1 --2- 172.21.2.101:0/2487037419 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1580186c0 0x7fb15801abc0 crc :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:36.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.234 7fb179c54700 1 -- 172.21.2.101:0/2487037419 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1740bc5f0 con 0x7fb174195070 2021-02-10T11:46:36.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.237 7fb16bfff700 1 -- 172.21.2.101:0/2487037419 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb15c030d60 con 0x7fb174195070 2021-02-10T11:46:36.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.364 7fb179c54700 1 -- 172.21.2.101:0/2487037419 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1740bc5f0 con 0x7fb174195070 2021-02-10T11:46:36.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.365 7fb16bfff700 1 -- 172.21.2.101:0/2487037419 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) v1 ==== 74+0+14007 (secure 0 0 0) 0x7fb15c000c90 con 0x7fb174195070 2021-02-10T11:46:36.362 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:46:36.363 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":175,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:46:36.152442","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":37,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":36,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:46:33.067171","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":37,"pool_name":"cephfs_data","create_time":"2021-02-10 11:46:34.142991","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:46:36.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.366 7fb179c54700 1 -- 172.21.2.101:0/2487037419 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1580186c0 msgr2=0x7fb15801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:36.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.366 7fb179c54700 1 --2- 172.21.2.101:0/2487037419 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1580186c0 0x7fb15801abc0 crc :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:36.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.366 7fb179c54700 1 -- 172.21.2.101:0/2487037419 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb174195070 msgr2=0x7fb174195470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:36.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.366 7fb179c54700 1 --2- 172.21.2.101:0/2487037419 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb174195070 0x7fb174195470 secure :-1 s=READY pgs=2976 cs=0 l=1 rev1=1 rx=0x7fb15c006f40 tx=0x7fb15c00b990).stop 2021-02-10T11:46:36.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.366 7fb179c54700 1 -- 172.21.2.101:0/2487037419 shutdown_connections 2021-02-10T11:46:36.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.366 7fb179c54700 1 --2- 172.21.2.101:0/2487037419 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb1580186c0 0x7fb15801abc0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:36.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.366 7fb179c54700 1 --2- 172.21.2.101:0/2487037419 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb1740bb8e0 0x7fb1740bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:36.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.366 7fb179c54700 1 --2- 172.21.2.101:0/2487037419 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb174195070 0x7fb174195470 unknown :-1 s=CLOSED pgs=2976 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:36.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.366 7fb179c54700 1 --2- 172.21.2.101:0/2487037419 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb1741942b0 0x7fb1740b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:36.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.366 7fb179c54700 1 -- 172.21.2.101:0/2487037419 >> 172.21.2.101:0/2487037419 conn(0x7fb1740b3040 msgr2=0x7fb1740c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:36.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.367 7fb179c54700 1 -- 172.21.2.101:0/2487037419 shutdown_connections 2021-02-10T11:46:36.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.367 7fb179c54700 1 -- 172.21.2.101:0/2487037419 wait complete. 2021-02-10T11:46:36.373 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-02-10T11:46:36.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.792 7fa1fab9c700 1 Processor -- start 2021-02-10T11:46:36.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.792 7fa1fab9c700 1 -- start start 2021-02-10T11:46:36.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.792 7fa1fab9c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa1f40d1870 0x7fa1f41b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:36.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.792 7fa1fab9c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa1f41b0ae0 0x7fa1f4108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:36.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.792 7fa1fab9c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f40fe1c0 con 0x7fa1f40d1c70 2021-02-10T11:46:36.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.792 7fa1fab9c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f4109a60 con 0x7fa1f40d1870 2021-02-10T11:46:36.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.792 7fa1fab9c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f4109750 con 0x7fa1f41b0ae0 2021-02-10T11:46:36.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.792 7fa1f91b9700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa1f40d1c70 0x7fa1f4108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33846/0 (socket says 172.21.2.101:33846) 2021-02-10T11:46:36.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.792 7fa1f91b9700 1 -- 172.21.2.101:0/1021895421 learned_addr learned my addr 172.21.2.101:0/1021895421 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:36.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.792 7fa1f89b8700 1 --2- 172.21.2.101:0/1021895421 >> v2:172.21.2.101:3300/0 conn(0x7fa1f40d1870 0x7fa1f41b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:36.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.793 7fa1f3fff700 1 --2- 172.21.2.101:0/1021895421 >> v2:172.21.2.108:3300/0 conn(0x7fa1f41b0ae0 0x7fa1f4108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:36.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.793 7fa1f89b8700 1 -- 172.21.2.101:0/1021895421 >> v2:172.21.2.108:3300/0 conn(0x7fa1f41b0ae0 msgr2=0x7fa1f4108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:36.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.793 7fa1f89b8700 1 --2- 172.21.2.101:0/1021895421 >> v2:172.21.2.108:3300/0 conn(0x7fa1f41b0ae0 0x7fa1f4108940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:36.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.793 7fa1f89b8700 1 -- 172.21.2.101:0/1021895421 >> v1:172.21.2.101:6789/0 conn(0x7fa1f40d1c70 legacy=0x7fa1f4108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:36.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.793 7fa1f89b8700 1 -- 172.21.2.101:0/1021895421 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f4088070 con 0x7fa1f40d1870 2021-02-10T11:46:36.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.793 7fa1f89b8700 1 --2- 172.21.2.101:0/1021895421 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1f40d1870 0x7fa1f41b04c0 secure :-1 s=READY pgs=2978 cs=0 l=1 rev1=1 rx=0x7fa1dc006650 tx=0x7fa1dc009a80).ready entity=mon.0 client_cookie=53885660699db5ad server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:36.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.793 7fa1f37fe700 1 -- 172.21.2.101:0/1021895421 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa1dc00c0c0 con 0x7fa1f40d1870 2021-02-10T11:46:36.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.793 7fa1f37fe700 1 -- 172.21.2.101:0/1021895421 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1dc00c340 con 0x7fa1f40d1870 2021-02-10T11:46:36.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.793 7fa1f37fe700 1 -- 172.21.2.101:0/1021895421 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa1dc00c790 con 0x7fa1f40d1870 2021-02-10T11:46:36.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.793 7fa1fab9c700 1 -- 172.21.2.101:0/1021895421 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1f40d1870 msgr2=0x7fa1f41b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:36.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.793 7fa1fab9c700 1 --2- 172.21.2.101:0/1021895421 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1f40d1870 0x7fa1f41b04c0 secure :-1 s=READY pgs=2978 cs=0 l=1 rev1=1 rx=0x7fa1dc006650 tx=0x7fa1dc009a80).stop 2021-02-10T11:46:36.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.794 7fa1fab9c700 1 -- 172.21.2.101:0/1021895421 shutdown_connections 2021-02-10T11:46:36.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.794 7fa1fab9c700 1 --2- 172.21.2.101:0/1021895421 >> v2:172.21.2.108:3300/0 conn(0x7fa1f41b0ae0 0x7fa1f4108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:36.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.794 7fa1fab9c700 1 --2- 172.21.2.101:0/1021895421 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1f40d1870 0x7fa1f41b04c0 unknown :-1 s=CLOSED pgs=2978 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:36.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.794 7fa1fab9c700 1 -- 172.21.2.101:0/1021895421 >> 172.21.2.101:0/1021895421 conn(0x7fa1f40ce420 msgr2=0x7fa1f40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:36.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.794 7fa1fab9c700 1 -- 172.21.2.101:0/1021895421 shutdown_connections 2021-02-10T11:46:36.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.794 7fa1fab9c700 1 -- 172.21.2.101:0/1021895421 wait complete. 2021-02-10T11:46:36.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.809 7fa1fab9c700 1 Processor -- start 2021-02-10T11:46:36.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.809 7fa1fab9c700 1 -- start start 2021-02-10T11:46:36.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.809 7fa1fab9c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1f4085bc0 0x7fa1f40863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:36.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.809 7fa1fab9c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1f40954c0 0x7fa1f4096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:36.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.809 7fa1fab9c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1f40958c0 0x7fa1f4096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:36.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.809 7fa1fab9c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f4095d80 con 0x7fa1f4085bc0 2021-02-10T11:46:36.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.809 7fa1fab9c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f40bfa60 con 0x7fa1f40958c0 2021-02-10T11:46:36.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.809 7fa1fab9c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f4085fc0 con 0x7fa1f40954c0 2021-02-10T11:46:36.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.809 7fa1f89b8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1f4085bc0 0x7fa1f40863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:36.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.809 7fa1f89b8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1f4085bc0 0x7fa1f40863e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33360/0 (socket says 172.21.2.101:33360) 2021-02-10T11:46:36.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.809 7fa1f89b8700 1 -- 172.21.2.101:0/693902390 learned_addr learned my addr 172.21.2.101:0/693902390 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:36.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.810 7fa1f89b8700 1 -- 172.21.2.101:0/693902390 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1f40954c0 msgr2=0x7fa1f4096300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:36.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.810 7fa1f89b8700 1 --2- 172.21.2.101:0/693902390 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1f40954c0 0x7fa1f4096300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:36.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.810 7fa1f89b8700 1 -- 172.21.2.101:0/693902390 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1f40958c0 msgr2=0x7fa1f4096920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:36.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.810 7fa1f89b8700 1 --2- 172.21.2.101:0/693902390 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1f40958c0 0x7fa1f4096920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:36.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.810 7fa1f89b8700 1 -- 172.21.2.101:0/693902390 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f4086c10 con 0x7fa1f4085bc0 2021-02-10T11:46:36.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.810 7fa1f89b8700 1 --2- 172.21.2.101:0/693902390 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1f4085bc0 0x7fa1f40863e0 secure :-1 s=READY pgs=2979 cs=0 l=1 rev1=1 rx=0x7fa1dc006e90 tx=0x7fa1dc00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:36.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.810 7fa1f17fa700 1 -- 172.21.2.101:0/693902390 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa1dc00bd80 con 0x7fa1f4085bc0 2021-02-10T11:46:36.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.810 7fa1f17fa700 1 -- 172.21.2.101:0/693902390 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1dc0052e0 con 0x7fa1f4085bc0 2021-02-10T11:46:36.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.810 7fa1f17fa700 1 -- 172.21.2.101:0/693902390 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa1dc005770 con 0x7fa1f4085bc0 2021-02-10T11:46:36.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.810 7fa1fab9c700 1 -- 172.21.2.101:0/693902390 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f4086c10 con 0x7fa1f4085bc0 2021-02-10T11:46:36.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.810 7fa1fab9c700 1 -- 172.21.2.101:0/693902390 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f4086c10 con 0x7fa1f4085bc0 2021-02-10T11:46:36.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.811 7fa1f17fa700 1 -- 172.21.2.101:0/693902390 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa1dc006030 con 0x7fa1f4085bc0 2021-02-10T11:46:36.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.811 7fa1f17fa700 1 --2- 172.21.2.101:0/693902390 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1d80186c0 0x7fa1d801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:36.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.811 7fa1f17fa700 1 -- 172.21.2.101:0/693902390 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa1dc0325c0 con 0x7fa1f4085bc0 2021-02-10T11:46:36.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.811 7fa1f3fff700 1 --2- 172.21.2.101:0/693902390 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1d80186c0 0x7fa1d801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:36.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.811 7fa1f3fff700 1 --2- 172.21.2.101:0/693902390 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1d80186c0 0x7fa1d801abc0 crc :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:36.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.842 7fa1fab9c700 1 -- 172.21.2.101:0/693902390 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f40fe1c0 con 0x7fa1f4085bc0 2021-02-10T11:46:36.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.845 7fa1f17fa700 1 -- 172.21.2.101:0/693902390 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa1dc0063f0 con 0x7fa1f4085bc0 2021-02-10T11:46:36.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:36.980 7fa1fab9c700 1 -- 172.21.2.101:0/693902390 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"} v 0) v1 -- 0x7fa1f40fe1c0 con 0x7fa1f4085bc0 2021-02-10T11:46:37.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.161 7fa1f17fa700 1 -- 172.21.2.101:0/693902390 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"}]=0 v148) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa1dc006030 con 0x7fa1f4085bc0 2021-02-10T11:46:37.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.196 7fa1fab9c700 1 -- 172.21.2.101:0/693902390 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1d80186c0 msgr2=0x7fa1d801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:37.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.196 7fa1fab9c700 1 --2- 172.21.2.101:0/693902390 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1d80186c0 0x7fa1d801abc0 crc :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:37.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.196 7fa1fab9c700 1 -- 172.21.2.101:0/693902390 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1f4085bc0 msgr2=0x7fa1f40863e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:37.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.196 7fa1fab9c700 1 --2- 172.21.2.101:0/693902390 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1f4085bc0 0x7fa1f40863e0 secure :-1 s=READY pgs=2979 cs=0 l=1 rev1=1 rx=0x7fa1dc006e90 tx=0x7fa1dc00b990).stop 2021-02-10T11:46:37.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.197 7fa1fab9c700 1 -- 172.21.2.101:0/693902390 shutdown_connections 2021-02-10T11:46:37.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.197 7fa1fab9c700 1 --2- 172.21.2.101:0/693902390 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1d80186c0 0x7fa1d801abc0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:37.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.197 7fa1fab9c700 1 --2- 172.21.2.101:0/693902390 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1f40954c0 0x7fa1f4096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:37.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.197 7fa1fab9c700 1 --2- 172.21.2.101:0/693902390 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1f40958c0 0x7fa1f4096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:37.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.197 7fa1fab9c700 1 --2- 172.21.2.101:0/693902390 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1f4085bc0 0x7fa1f40863e0 unknown :-1 s=CLOSED pgs=2979 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:37.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.197 7fa1fab9c700 1 -- 172.21.2.101:0/693902390 >> 172.21.2.101:0/693902390 conn(0x7fa1f40bef00 msgr2=0x7fa1f409d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:37.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.197 7fa1fab9c700 1 -- 172.21.2.101:0/693902390 shutdown_connections 2021-02-10T11:46:37.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.197 7fa1fab9c700 1 -- 172.21.2.101:0/693902390 wait complete. 2021-02-10T11:46:37.202 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:46:37.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.617 7fa838722700 1 Processor -- start 2021-02-10T11:46:37.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.617 7fa838722700 1 -- start start 2021-02-10T11:46:37.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.618 7fa838722700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8300d0340 0x7fa8300ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:37.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.618 7fa838722700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa8300d1c70 0x7fa8300efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:37.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.618 7fa838722700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8300f0b10 con 0x7fa8300d1870 2021-02-10T11:46:37.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.618 7fa838722700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8300e5ed0 con 0x7fa8300d1c70 2021-02-10T11:46:37.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.618 7fa838722700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8301b0be0 con 0x7fa8300d0340 2021-02-10T11:46:37.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.618 7fa83653e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa8300d1870 0x7fa8301b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33860/0 (socket says 172.21.2.101:33860) 2021-02-10T11:46:37.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.618 7fa83653e700 1 -- 172.21.2.101:0/1499240101 learned_addr learned my addr 172.21.2.101:0/1499240101 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:37.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.618 7fa83553c700 1 -- 172.21.2.101:0/1499240101 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3978101450 0 0) 0x7fa824001180 con 0x7fa8300d1870 2021-02-10T11:46:37.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.618 7fa83553c700 1 -- 172.21.2.101:0/1499240101 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa81c002fe0 con 0x7fa8300d1870 2021-02-10T11:46:37.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa835d3d700 1 --2- 172.21.2.101:0/1499240101 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8300d0340 0x7fa8300ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:37.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa836d3f700 1 --2- 172.21.2.101:0/1499240101 >> v2:172.21.2.108:3300/0 conn(0x7fa8300d1c70 0x7fa8300efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:37.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa83553c700 1 -- 172.21.2.101:0/1499240101 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1754999722 0 0) 0x7fa824001180 con 0x7fa8300d1870 2021-02-10T11:46:37.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa83553c700 1 -- 172.21.2.101:0/1499240101 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8300d0340 msgr2=0x7fa8300ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:37.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa83553c700 1 --2- 172.21.2.101:0/1499240101 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8300d0340 0x7fa8300ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:37.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa83553c700 1 -- 172.21.2.101:0/1499240101 >> v2:172.21.2.108:3300/0 conn(0x7fa8300d1c70 msgr2=0x7fa8300efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:37.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa83553c700 1 --2- 172.21.2.101:0/1499240101 >> v2:172.21.2.108:3300/0 conn(0x7fa8300d1c70 0x7fa8300efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:37.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa83553c700 1 -- 172.21.2.101:0/1499240101 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8300f20a0 con 0x7fa8300d1870 2021-02-10T11:46:37.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa83553c700 1 -- 172.21.2.101:0/1499240101 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fa824001680 con 0x7fa8300d1870 2021-02-10T11:46:37.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa83553c700 1 -- 172.21.2.101:0/1499240101 >> v1:172.21.2.101:6789/0 conn(0x7fa8300d1870 legacy=0x7fa8301b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:37.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa83553c700 1 --2- 172.21.2.101:0/1499240101 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa81c003ac0 0x7fa81c005ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:37.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa83553c700 1 --2- 172.21.2.101:0/1499240101 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa81c04a780 0x7fa81c04cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:37.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa83553c700 1 --2- 172.21.2.101:0/1499240101 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa81c04d5a0 0x7fa81c04fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:37.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa83553c700 1 -- 172.21.2.101:0/1499240101 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c0502e0 con 0x7fa81c04a780 2021-02-10T11:46:37.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa83553c700 1 -- 172.21.2.101:0/1499240101 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c050500 con 0x7fa81c003ac0 2021-02-10T11:46:37.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa83553c700 1 -- 172.21.2.101:0/1499240101 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c050720 con 0x7fa81c04d5a0 2021-02-10T11:46:37.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa835d3d700 1 --2- 172.21.2.101:0/1499240101 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa81c04a780 0x7fa81c04cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:37.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa83653e700 1 --2- 172.21.2.101:0/1499240101 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa81c003ac0 0x7fa81c005ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:37.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa835d3d700 1 -- 172.21.2.101:0/1499240101 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa81c04d5a0 msgr2=0x7fa81c04fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:37.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa835d3d700 1 --2- 172.21.2.101:0/1499240101 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa81c04d5a0 0x7fa81c04fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:37.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa835d3d700 1 -- 172.21.2.101:0/1499240101 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa81c003ac0 msgr2=0x7fa81c005ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:37.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa835d3d700 1 --2- 172.21.2.101:0/1499240101 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa81c003ac0 0x7fa81c005ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:37.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa835d3d700 1 -- 172.21.2.101:0/1499240101 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c0509c0 con 0x7fa81c04a780 2021-02-10T11:46:37.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa835d3d700 1 --2- 172.21.2.101:0/1499240101 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa81c04a780 0x7fa81c04cca0 secure :-1 s=READY pgs=2981 cs=0 l=1 rev1=1 rx=0x7fa82c006ac0 tx=0x7fa82c009ab0).ready entity=mon.0 client_cookie=911d98a897aeba9a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:37.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa83553c700 1 -- 172.21.2.101:0/1499240101 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa82c00c0c0 con 0x7fa81c04a780 2021-02-10T11:46:37.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa83553c700 1 -- 172.21.2.101:0/1499240101 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82c00c380 con 0x7fa81c04a780 2021-02-10T11:46:37.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa83553c700 1 -- 172.21.2.101:0/1499240101 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa82c0009b0 con 0x7fa81c04a780 2021-02-10T11:46:37.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa838722700 1 -- 172.21.2.101:0/1499240101 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa81c04a780 msgr2=0x7fa81c04cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:37.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa838722700 1 --2- 172.21.2.101:0/1499240101 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa81c04a780 0x7fa81c04cca0 secure :-1 s=READY pgs=2981 cs=0 l=1 rev1=1 rx=0x7fa82c006ac0 tx=0x7fa82c009ab0).stop 2021-02-10T11:46:37.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.619 7fa836d3f700 1 -- 172.21.2.101:0/1499240101 reap_dead start 2021-02-10T11:46:37.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.620 7fa838722700 1 -- 172.21.2.101:0/1499240101 shutdown_connections 2021-02-10T11:46:37.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.620 7fa838722700 1 -- 172.21.2.101:0/1499240101 >> 172.21.2.101:0/1499240101 conn(0x7fa8300ce450 msgr2=0x7fa8300d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:37.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.620 7fa838722700 1 -- 172.21.2.101:0/1499240101 shutdown_connections 2021-02-10T11:46:37.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.620 7fa838722700 1 -- 172.21.2.101:0/1499240101 wait complete. 2021-02-10T11:46:37.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.635 7fa838722700 1 Processor -- start 2021-02-10T11:46:37.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.635 7fa838722700 1 -- start start 2021-02-10T11:46:37.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.635 7fa838722700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa830195650 0x7fa8300b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:37.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.635 7fa838722700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8300b5b80 0x7fa8301a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:37.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.635 7fa838722700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8300acfa0 0x7fa8301a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:37.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.636 7fa838722700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8300b6010 con 0x7fa8300b5b80 2021-02-10T11:46:37.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.636 7fa838722700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8301a3a60 con 0x7fa8300acfa0 2021-02-10T11:46:37.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.636 7fa838722700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8301a3c40 con 0x7fa830195650 2021-02-10T11:46:37.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.636 7fa835d3d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8300b5b80 0x7fa8301a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:37.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.636 7fa835d3d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8300b5b80 0x7fa8301a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33376/0 (socket says 172.21.2.101:33376) 2021-02-10T11:46:37.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.636 7fa835d3d700 1 -- 172.21.2.101:0/16677228 learned_addr learned my addr 172.21.2.101:0/16677228 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:37.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.636 7fa835d3d700 1 -- 172.21.2.101:0/16677228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa830195650 msgr2=0x7fa8300b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:37.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.636 7fa835d3d700 1 --2- 172.21.2.101:0/16677228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa830195650 0x7fa8300b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:37.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.636 7fa835d3d700 1 -- 172.21.2.101:0/16677228 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8300acfa0 msgr2=0x7fa8301a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:37.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.636 7fa835d3d700 1 --2- 172.21.2.101:0/16677228 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8300acfa0 0x7fa8301a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:37.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.636 7fa835d3d700 1 -- 172.21.2.101:0/16677228 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8300bcae0 con 0x7fa8300b5b80 2021-02-10T11:46:37.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.636 7fa835d3d700 1 --2- 172.21.2.101:0/16677228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8300b5b80 0x7fa8301a2e20 secure :-1 s=READY pgs=2982 cs=0 l=1 rev1=1 rx=0x7fa82c009f60 tx=0x7fa82c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:37.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.636 7fa822ffd700 1 -- 172.21.2.101:0/16677228 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa82c009340 con 0x7fa8300b5b80 2021-02-10T11:46:37.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.636 7fa822ffd700 1 -- 172.21.2.101:0/16677228 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82c002030 con 0x7fa8300b5b80 2021-02-10T11:46:37.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.636 7fa822ffd700 1 -- 172.21.2.101:0/16677228 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa82c002490 con 0x7fa8300b5b80 2021-02-10T11:46:37.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.636 7fa838722700 1 -- 172.21.2.101:0/16677228 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8300bcae0 con 0x7fa8300b5b80 2021-02-10T11:46:37.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.636 7fa838722700 1 -- 172.21.2.101:0/16677228 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8300bcae0 con 0x7fa8300b5b80 2021-02-10T11:46:37.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.637 7fa822ffd700 1 -- 172.21.2.101:0/16677228 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa82c009340 con 0x7fa8300b5b80 2021-02-10T11:46:37.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.637 7fa822ffd700 1 --2- 172.21.2.101:0/16677228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8180186c0 0x7fa81801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:37.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.637 7fa822ffd700 1 -- 172.21.2.101:0/16677228 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa82c0323f0 con 0x7fa8300b5b80 2021-02-10T11:46:37.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.637 7fa83653e700 1 --2- 172.21.2.101:0/16677228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8180186c0 0x7fa81801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:37.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.638 7fa83653e700 1 --2- 172.21.2.101:0/16677228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8180186c0 0x7fa81801abc0 crc :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:37.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.668 7fa838722700 1 -- 172.21.2.101:0/16677228 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8301a3c40 con 0x7fa8300b5b80 2021-02-10T11:46:37.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.670 7fa822ffd700 1 -- 172.21.2.101:0/16677228 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa82c02a340 con 0x7fa8300b5b80 2021-02-10T11:46:37.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.802 7fa838722700 1 -- 172.21.2.101:0/16677228 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8301a3c40 con 0x7fa8300b5b80 2021-02-10T11:46:37.800 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.803 7fa822ffd700 1 -- 172.21.2.101:0/16677228 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+2111 (secure 0 0 0) 0x7fa82c02a340 con 0x7fa8300b5b80 2021-02-10T11:46:37.801 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:46:37.801 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":148,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":12780,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2200088074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2200088074},{"type":"v1","addr":"172.21.2.101:6835","nonce":2200088074}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":144}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:46:35.149920","modified":"2021-02-10 11:46:37.158088","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12777},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12777":{"gid":12777,"name":"b","rank":0,"incarnation":145,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/1343655613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":1343655613},{"type":"v1","addr":"172.21.2.108:6833","nonce":1343655613}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[37],"metadata_pool":36,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":18}]} 2021-02-10T11:46:37.801 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 148 2021-02-10T11:46:37.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.805 7fa838722700 1 -- 172.21.2.101:0/16677228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8180186c0 msgr2=0x7fa81801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:37.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.805 7fa838722700 1 --2- 172.21.2.101:0/16677228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8180186c0 0x7fa81801abc0 crc :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:37.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.805 7fa838722700 1 -- 172.21.2.101:0/16677228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8300b5b80 msgr2=0x7fa8301a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:37.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.805 7fa838722700 1 --2- 172.21.2.101:0/16677228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8300b5b80 0x7fa8301a2e20 secure :-1 s=READY pgs=2982 cs=0 l=1 rev1=1 rx=0x7fa82c009f60 tx=0x7fa82c00b990).stop 2021-02-10T11:46:37.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.805 7fa838722700 1 -- 172.21.2.101:0/16677228 shutdown_connections 2021-02-10T11:46:37.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.805 7fa838722700 1 --2- 172.21.2.101:0/16677228 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8300acfa0 0x7fa8301a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:37.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.805 7fa838722700 1 --2- 172.21.2.101:0/16677228 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8300b5b80 0x7fa8301a2e20 unknown :-1 s=CLOSED pgs=2982 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:37.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.805 7fa838722700 1 --2- 172.21.2.101:0/16677228 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8180186c0 0x7fa81801abc0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:37.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.805 7fa838722700 1 --2- 172.21.2.101:0/16677228 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa830195650 0x7fa8300b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:37.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.805 7fa838722700 1 -- 172.21.2.101:0/16677228 >> 172.21.2.101:0/16677228 conn(0x7fa8300b3040 msgr2=0x7fa8300c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:37.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.805 7fa838722700 1 -- 172.21.2.101:0/16677228 shutdown_connections 2021-02-10T11:46:37.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:37.805 7fa838722700 1 -- 172.21.2.101:0/16677228 wait complete. 2021-02-10T11:46:37.811 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:46:38.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.227 7f250bc76700 1 Processor -- start 2021-02-10T11:46:38.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.227 7f250bc76700 1 -- start start 2021-02-10T11:46:38.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.228 7f250bc76700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f25040d1870 0x7f25041b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:38.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.228 7f250bc76700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f25041b0a10 0x7f25040cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:38.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.228 7f250bc76700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25040f0b10 con 0x7f25040d1c70 2021-02-10T11:46:38.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.228 7f250bc76700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25040e5ed0 con 0x7f25041b0a10 2021-02-10T11:46:38.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.228 7f250bc76700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25040f0650 con 0x7f25040d1870 2021-02-10T11:46:38.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.228 7f250a293700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f25040d1c70 0x7f25040cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33882/0 (socket says 172.21.2.101:33882) 2021-02-10T11:46:38.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.228 7f250a293700 1 -- 172.21.2.101:0/3278355982 learned_addr learned my addr 172.21.2.101:0/3278355982 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:38.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.228 7f2509291700 1 --2- 172.21.2.101:0/3278355982 >> v2:172.21.2.101:3300/0 conn(0x7f25041b0a10 0x7f25040cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:38.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.228 7f2508a90700 1 -- 172.21.2.101:0/3278355982 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2731926328 0 0) 0x7f24fc001500 con 0x7f25040d1c70 2021-02-10T11:46:38.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.228 7f2508a90700 1 -- 172.21.2.101:0/3278355982 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24f0002fe0 con 0x7f25040d1c70 2021-02-10T11:46:38.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.228 7f2509291700 1 -- 172.21.2.101:0/3278355982 >> v2:172.21.2.108:3300/0 conn(0x7f25040d1870 msgr2=0x7f25041b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:38.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.228 7f2509291700 1 --2- 172.21.2.101:0/3278355982 >> v2:172.21.2.108:3300/0 conn(0x7f25040d1870 0x7f25041b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:38.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.228 7f2509291700 1 -- 172.21.2.101:0/3278355982 >> v1:172.21.2.101:6789/0 conn(0x7f25040d1c70 legacy=0x7f25040cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:38.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.228 7f2509291700 1 -- 172.21.2.101:0/3278355982 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25040f0840 con 0x7f25041b0a10 2021-02-10T11:46:38.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.229 7f2509291700 1 --2- 172.21.2.101:0/3278355982 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25041b0a10 0x7f25040cf330 secure :-1 s=READY pgs=2984 cs=0 l=1 rev1=1 rx=0x7f2500006650 tx=0x7f2500009a80).ready entity=mon.0 client_cookie=3d62872da9a85f90 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:38.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.229 7f2508a90700 1 -- 172.21.2.101:0/3278355982 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f250000c0c0 con 0x7f25041b0a10 2021-02-10T11:46:38.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.229 7f2508a90700 1 -- 172.21.2.101:0/3278355982 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250000c340 con 0x7f25041b0a10 2021-02-10T11:46:38.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.229 7f2508a90700 1 -- 172.21.2.101:0/3278355982 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f250000c860 con 0x7f25041b0a10 2021-02-10T11:46:38.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.229 7f250bc76700 1 -- 172.21.2.101:0/3278355982 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25041b0a10 msgr2=0x7f25040cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:38.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.229 7f250bc76700 1 --2- 172.21.2.101:0/3278355982 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25041b0a10 0x7f25040cf330 secure :-1 s=READY pgs=2984 cs=0 l=1 rev1=1 rx=0x7f2500006650 tx=0x7f2500009a80).stop 2021-02-10T11:46:38.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.229 7f250bc76700 1 -- 172.21.2.101:0/3278355982 shutdown_connections 2021-02-10T11:46:38.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.229 7f250bc76700 1 --2- 172.21.2.101:0/3278355982 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25041b0a10 0x7f25040cf330 unknown :-1 s=CLOSED pgs=2984 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:38.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.229 7f250bc76700 1 --2- 172.21.2.101:0/3278355982 >> v2:172.21.2.108:3300/0 conn(0x7f25040d1870 0x7f25041b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:38.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.229 7f250bc76700 1 -- 172.21.2.101:0/3278355982 >> 172.21.2.101:0/3278355982 conn(0x7f25040ce450 msgr2=0x7f25040d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:38.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.230 7f250bc76700 1 -- 172.21.2.101:0/3278355982 shutdown_connections 2021-02-10T11:46:38.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.230 7f250bc76700 1 -- 172.21.2.101:0/3278355982 wait complete. 2021-02-10T11:46:38.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.244 7f250bc76700 1 Processor -- start 2021-02-10T11:46:38.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.244 7f250bc76700 1 -- start start 2021-02-10T11:46:38.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.245 7f250bc76700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f250418d870 0x7f25040b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:38.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.245 7f250bc76700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25040b1fd0 0x7f2504099ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:38.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.245 7f250bc76700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25040b23d0 0x7f250409a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:38.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.245 7f250bc76700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250418dc70 con 0x7f25040b1fd0 2021-02-10T11:46:38.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.245 7f250bc76700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2504199e60 con 0x7f25040b23d0 2021-02-10T11:46:38.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.245 7f250bc76700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250409ab30 con 0x7f250418d870 2021-02-10T11:46:38.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.245 7f2509291700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25040b1fd0 0x7f2504099ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:38.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.245 7f2509291700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25040b1fd0 0x7f2504099ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33390/0 (socket says 172.21.2.101:33390) 2021-02-10T11:46:38.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.245 7f2509291700 1 -- 172.21.2.101:0/57344829 learned_addr learned my addr 172.21.2.101:0/57344829 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:38.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.245 7f250a293700 1 --2- 172.21.2.101:0/57344829 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25040b23d0 0x7f250409a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:38.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.245 7f2509a92700 1 --2- 172.21.2.101:0/57344829 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f250418d870 0x7f25040b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:38.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.245 7f2509291700 1 -- 172.21.2.101:0/57344829 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f250418d870 msgr2=0x7f25040b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:38.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.245 7f2509291700 1 --2- 172.21.2.101:0/57344829 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f250418d870 0x7f25040b19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:38.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.245 7f2509291700 1 -- 172.21.2.101:0/57344829 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25040b23d0 msgr2=0x7f250409a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:38.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.245 7f2509291700 1 --2- 172.21.2.101:0/57344829 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25040b23d0 0x7f250409a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:38.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.245 7f2509291700 1 -- 172.21.2.101:0/57344829 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250409ad50 con 0x7f25040b1fd0 2021-02-10T11:46:38.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.245 7f2509291700 1 --2- 172.21.2.101:0/57344829 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25040b1fd0 0x7f2504099ef0 secure :-1 s=READY pgs=2985 cs=0 l=1 rev1=1 rx=0x7f25000096a0 tx=0x7f250000b9b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:38.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.246 7f24f67fc700 1 -- 172.21.2.101:0/57344829 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f25000093a0 con 0x7f25040b1fd0 2021-02-10T11:46:38.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.246 7f24f67fc700 1 -- 172.21.2.101:0/57344829 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25000067f0 con 0x7f25040b1fd0 2021-02-10T11:46:38.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.246 7f24f67fc700 1 -- 172.21.2.101:0/57344829 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f25000022b0 con 0x7f25040b1fd0 2021-02-10T11:46:38.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.246 7f250bc76700 1 -- 172.21.2.101:0/57344829 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f250409ad50 con 0x7f25040b1fd0 2021-02-10T11:46:38.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.246 7f250bc76700 1 -- 172.21.2.101:0/57344829 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250409ad50 con 0x7f25040b1fd0 2021-02-10T11:46:38.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.246 7f24f67fc700 1 -- 172.21.2.101:0/57344829 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f25000052e0 con 0x7f25040b1fd0 2021-02-10T11:46:38.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.247 7f24f67fc700 1 --2- 172.21.2.101:0/57344829 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24ec0186c0 0x7f24ec01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:38.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.247 7f24f67fc700 1 -- 172.21.2.101:0/57344829 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f25000324f0 con 0x7f25040b1fd0 2021-02-10T11:46:38.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.247 7f2509a92700 1 --2- 172.21.2.101:0/57344829 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24ec0186c0 0x7f24ec01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:38.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.247 7f2509a92700 1 --2- 172.21.2.101:0/57344829 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24ec0186c0 0x7f24ec01abc0 crc :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:38.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.278 7f250bc76700 1 -- 172.21.2.101:0/57344829 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25040999c0 con 0x7f25040b1fd0 2021-02-10T11:46:38.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.280 7f24f67fc700 1 -- 172.21.2.101:0/57344829 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2500000c90 con 0x7f25040b1fd0 2021-02-10T11:46:38.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.406 7f250bc76700 1 -- 172.21.2.101:0/57344829 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25040999c0 con 0x7f25040b1fd0 2021-02-10T11:46:38.404 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.407 7f24f67fc700 1 -- 172.21.2.101:0/57344829 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f25000052e0 con 0x7f25040b1fd0 2021-02-10T11:46:38.405 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:46:38.405 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":175,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:46:36.152442","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":37,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":36,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:46:33.067171","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":37,"pool_name":"cephfs_data","create_time":"2021-02-10 11:46:34.142991","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:46:38.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.409 7f250bc76700 1 -- 172.21.2.101:0/57344829 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24ec0186c0 msgr2=0x7f24ec01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:38.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.409 7f250bc76700 1 --2- 172.21.2.101:0/57344829 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24ec0186c0 0x7f24ec01abc0 crc :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:38.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.409 7f250bc76700 1 -- 172.21.2.101:0/57344829 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25040b1fd0 msgr2=0x7f2504099ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:38.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.409 7f250bc76700 1 --2- 172.21.2.101:0/57344829 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25040b1fd0 0x7f2504099ef0 secure :-1 s=READY pgs=2985 cs=0 l=1 rev1=1 rx=0x7f25000096a0 tx=0x7f250000b9b0).stop 2021-02-10T11:46:38.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.409 7f250bc76700 1 -- 172.21.2.101:0/57344829 shutdown_connections 2021-02-10T11:46:38.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.409 7f250bc76700 1 --2- 172.21.2.101:0/57344829 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25040b23d0 0x7f250409a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:38.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.409 7f250bc76700 1 --2- 172.21.2.101:0/57344829 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25040b1fd0 0x7f2504099ef0 unknown :-1 s=CLOSED pgs=2985 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:38.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.409 7f250bc76700 1 --2- 172.21.2.101:0/57344829 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24ec0186c0 0x7f24ec01abc0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:38.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.409 7f250bc76700 1 --2- 172.21.2.101:0/57344829 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f250418d870 0x7f25040b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:38.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.409 7f250bc76700 1 -- 172.21.2.101:0/57344829 >> 172.21.2.101:0/57344829 conn(0x7f25040f0550 msgr2=0x7f2504191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:38.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.409 7f250bc76700 1 -- 172.21.2.101:0/57344829 shutdown_connections 2021-02-10T11:46:38.407 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.409 7f250bc76700 1 -- 172.21.2.101:0/57344829 wait complete. 2021-02-10T11:46:38.415 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:46:38.474 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:46:38.478 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:46:38.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.845 7f939aac8700 1 Processor -- start 2021-02-10T11:46:38.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.845 7f939aac8700 1 -- start start 2021-02-10T11:46:38.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.846 7f939aac8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f93940d1880 0x7f93941b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:38.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.846 7f939aac8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f93940cff80 0x7f9394109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:38.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.846 7f939aac8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93940fe1d0 con 0x7f93941b0af0 2021-02-10T11:46:38.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.846 7f939aac8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9394109ae0 con 0x7f93940cff80 2021-02-10T11:46:38.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.846 7f939aac8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9394109760 con 0x7f93940d1880 2021-02-10T11:46:38.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.846 7f93990e5700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f93940cff80 0x7f9394109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:38.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.846 7f93990e5700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f93940cff80 0x7f9394109140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33402/0 (socket says 172.21.2.101:33402) 2021-02-10T11:46:38.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.846 7f93990e5700 1 -- 172.21.2.101:0/1348853973 learned_addr learned my addr 172.21.2.101:0/1348853973 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:38.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.846 7f93990e5700 1 -- 172.21.2.101:0/1348853973 >> v2:172.21.2.108:3300/0 conn(0x7f93940d1880 msgr2=0x7f93941b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:38.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.846 7f93990e5700 1 --2- 172.21.2.101:0/1348853973 >> v2:172.21.2.108:3300/0 conn(0x7f93940d1880 0x7f93941b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:38.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.846 7f93990e5700 1 -- 172.21.2.101:0/1348853973 >> v1:172.21.2.108:6789/0 conn(0x7f93941b0af0 legacy=0x7f9394108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:38.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.846 7f93990e5700 1 -- 172.21.2.101:0/1348853973 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9394088070 con 0x7f93940cff80 2021-02-10T11:46:38.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.846 7f93990e5700 1 --2- 172.21.2.101:0/1348853973 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93940cff80 0x7f9394109140 secure :-1 s=READY pgs=2986 cs=0 l=1 rev1=1 rx=0x7f938c006b80 tx=0x7f938c009b90).ready entity=mon.0 client_cookie=4acccb2c6b2bd3b0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:38.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.847 7f93937fe700 1 -- 172.21.2.101:0/1348853973 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f938c0022e0 con 0x7f93940cff80 2021-02-10T11:46:38.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.847 7f93937fe700 1 -- 172.21.2.101:0/1348853973 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938c002620 con 0x7f93940cff80 2021-02-10T11:46:38.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.847 7f93937fe700 1 -- 172.21.2.101:0/1348853973 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f938c000c10 con 0x7f93940cff80 2021-02-10T11:46:38.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.847 7f939aac8700 1 -- 172.21.2.101:0/1348853973 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93940cff80 msgr2=0x7f9394109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:38.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.847 7f939aac8700 1 --2- 172.21.2.101:0/1348853973 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93940cff80 0x7f9394109140 secure :-1 s=READY pgs=2986 cs=0 l=1 rev1=1 rx=0x7f938c006b80 tx=0x7f938c009b90).stop 2021-02-10T11:46:38.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.847 7f939aac8700 1 -- 172.21.2.101:0/1348853973 shutdown_connections 2021-02-10T11:46:38.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.847 7f939aac8700 1 --2- 172.21.2.101:0/1348853973 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93940cff80 0x7f9394109140 unknown :-1 s=CLOSED pgs=2986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:38.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.847 7f939aac8700 1 --2- 172.21.2.101:0/1348853973 >> v2:172.21.2.108:3300/0 conn(0x7f93940d1880 0x7f93941b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:38.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.847 7f939aac8700 1 -- 172.21.2.101:0/1348853973 >> 172.21.2.101:0/1348853973 conn(0x7f93940ce430 msgr2=0x7f93940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:38.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.847 7f939aac8700 1 -- 172.21.2.101:0/1348853973 shutdown_connections 2021-02-10T11:46:38.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.847 7f939aac8700 1 -- 172.21.2.101:0/1348853973 wait complete. 2021-02-10T11:46:38.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.862 7f939aac8700 1 Processor -- start 2021-02-10T11:46:38.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.862 7f939aac8700 1 -- start start 2021-02-10T11:46:38.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.863 7f939aac8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9394086810 0x7f939419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:38.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.863 7f939aac8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93940b04d0 0x7f9394097450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:38.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.863 7f939aac8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93940b08d0 0x7f9394097a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:38.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.863 7f939aac8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93941a02c0 con 0x7f9394086810 2021-02-10T11:46:38.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.863 7f939aac8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9394098090 con 0x7f93940b08d0 2021-02-10T11:46:38.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.863 7f939aac8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93940982b0 con 0x7f93940b04d0 2021-02-10T11:46:38.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.863 7f93988e4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9394086810 0x7f939419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:38.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.863 7f93988e4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9394086810 0x7f939419fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33406/0 (socket says 172.21.2.101:33406) 2021-02-10T11:46:38.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.863 7f93988e4700 1 -- 172.21.2.101:0/2757886854 learned_addr learned my addr 172.21.2.101:0/2757886854 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:38.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.863 7f93988e4700 1 -- 172.21.2.101:0/2757886854 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93940b04d0 msgr2=0x7f9394097450 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:38.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.863 7f93988e4700 1 --2- 172.21.2.101:0/2757886854 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93940b04d0 0x7f9394097450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:38.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.863 7f93988e4700 1 -- 172.21.2.101:0/2757886854 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93940b08d0 msgr2=0x7f9394097a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:38.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.863 7f93988e4700 1 --2- 172.21.2.101:0/2757886854 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93940b08d0 0x7f9394097a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:38.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.863 7f93988e4700 1 -- 172.21.2.101:0/2757886854 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93940984d0 con 0x7f9394086810 2021-02-10T11:46:38.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.863 7f93988e4700 1 --2- 172.21.2.101:0/2757886854 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9394086810 0x7f939419fca0 secure :-1 s=READY pgs=2987 cs=0 l=1 rev1=1 rx=0x7f937c006f00 tx=0x7f937c00b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:38.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.863 7f93917fa700 1 -- 172.21.2.101:0/2757886854 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f937c00bd30 con 0x7f9394086810 2021-02-10T11:46:38.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.863 7f93917fa700 1 -- 172.21.2.101:0/2757886854 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937c006c20 con 0x7f9394086810 2021-02-10T11:46:38.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.863 7f93917fa700 1 -- 172.21.2.101:0/2757886854 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f937c002020 con 0x7f9394086810 2021-02-10T11:46:38.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.863 7f939aac8700 1 -- 172.21.2.101:0/2757886854 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93940984d0 con 0x7f9394086810 2021-02-10T11:46:38.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.863 7f939aac8700 1 -- 172.21.2.101:0/2757886854 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9394098b80 con 0x7f9394086810 2021-02-10T11:46:38.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.864 7f93917fa700 1 -- 172.21.2.101:0/2757886854 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f937c005310 con 0x7f9394086810 2021-02-10T11:46:38.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.864 7f93917fa700 1 --2- 172.21.2.101:0/2757886854 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93780186c0 0x7f937801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:38.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.864 7f93917fa700 1 -- 172.21.2.101:0/2757886854 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f937c032570 con 0x7f9394086810 2021-02-10T11:46:38.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.864 7f9393fff700 1 --2- 172.21.2.101:0/2757886854 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93780186c0 0x7f937801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:38.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.864 7f9393fff700 1 --2- 172.21.2.101:0/2757886854 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93780186c0 0x7f937801abc0 crc :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:38.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.895 7f939aac8700 1 -- 172.21.2.101:0/2757886854 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93940fe1d0 con 0x7f9394086810 2021-02-10T11:46:38.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:38.898 7f93917fa700 1 -- 172.21.2.101:0/2757886854 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f937c0009b0 con 0x7f9394086810 2021-02-10T11:46:39.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.029 7f939aac8700 1 -- 172.21.2.101:0/2757886854 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f93940fe1d0 con 0x7f9394086810 2021-02-10T11:46:39.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.033 7f93917fa700 1 -- 172.21.2.101:0/2757886854 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v95) v1 ==== 184+0+0 (secure 0 0 0) 0x7f937c005310 con 0x7f9394086810 2021-02-10T11:46:39.034 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.0 2021-02-10T11:46:39.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.038 7f939aac8700 1 -- 172.21.2.101:0/2757886854 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93780186c0 msgr2=0x7f937801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:39.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.038 7f939aac8700 1 --2- 172.21.2.101:0/2757886854 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93780186c0 0x7f937801abc0 crc :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:39.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.038 7f939aac8700 1 -- 172.21.2.101:0/2757886854 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9394086810 msgr2=0x7f939419fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:39.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.038 7f939aac8700 1 --2- 172.21.2.101:0/2757886854 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9394086810 0x7f939419fca0 secure :-1 s=READY pgs=2987 cs=0 l=1 rev1=1 rx=0x7f937c006f00 tx=0x7f937c00b940).stop 2021-02-10T11:46:39.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.039 7f939aac8700 1 -- 172.21.2.101:0/2757886854 shutdown_connections 2021-02-10T11:46:39.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.039 7f939aac8700 1 --2- 172.21.2.101:0/2757886854 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93780186c0 0x7f937801abc0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:39.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.039 7f939aac8700 1 --2- 172.21.2.101:0/2757886854 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93940b04d0 0x7f9394097450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:39.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.039 7f939aac8700 1 --2- 172.21.2.101:0/2757886854 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93940b08d0 0x7f9394097a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:39.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.039 7f939aac8700 1 --2- 172.21.2.101:0/2757886854 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9394086810 0x7f939419fca0 unknown :-1 s=CLOSED pgs=2987 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:39.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.039 7f939aac8700 1 -- 172.21.2.101:0/2757886854 >> 172.21.2.101:0/2757886854 conn(0x7f939419f880 msgr2=0x7f939408f890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:39.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.039 7f939aac8700 1 -- 172.21.2.101:0/2757886854 shutdown_connections 2021-02-10T11:46:39.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.039 7f939aac8700 1 -- 172.21.2.101:0/2757886854 wait complete. 2021-02-10T11:46:39.045 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:46:39.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.471 7fdb36af6700 1 Processor -- start 2021-02-10T11:46:39.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.471 7fdb36af6700 1 -- start start 2021-02-10T11:46:39.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.471 7fdb36af6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb300d1880 0x7fdb301b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:39.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.471 7fdb36af6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdb301b0af0 0x7fdb30108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:39.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.471 7fdb36af6700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fdb300d1c80 0x7fdb30108f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:39.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.471 7fdb36af6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb300e5f00 con 0x7fdb301b0af0 2021-02-10T11:46:39.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.471 7fdb36af6700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb30109a70 con 0x7fdb300d1c80 2021-02-10T11:46:39.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.471 7fdb36af6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb30088070 con 0x7fdb300d1880 2021-02-10T11:46:39.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.472 7fdb2ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdb301b0af0 0x7fdb30108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:39.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.472 7fdb2ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdb301b0af0 0x7fdb30108950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33412/0 (socket says 172.21.2.101:33412) 2021-02-10T11:46:39.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.472 7fdb2ffff700 1 -- 172.21.2.101:0/1882599981 learned_addr learned my addr 172.21.2.101:0/1882599981 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:39.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.472 7fdb2ffff700 1 -- 172.21.2.101:0/1882599981 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb300d1880 msgr2=0x7fdb301b04d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:46:39.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.472 7fdb2ffff700 1 --2- 172.21.2.101:0/1882599981 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb300d1880 0x7fdb301b04d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:39.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.472 7fdb2ffff700 1 -- 172.21.2.101:0/1882599981 >> v2:172.21.2.108:3300/0 conn(0x7fdb300d1c80 msgr2=0x7fdb30108f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:46:39.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.472 7fdb2ffff700 1 --2- 172.21.2.101:0/1882599981 >> v2:172.21.2.108:3300/0 conn(0x7fdb300d1c80 0x7fdb30108f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:39.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.472 7fdb2ffff700 1 -- 172.21.2.101:0/1882599981 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb30109590 con 0x7fdb301b0af0 2021-02-10T11:46:39.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.472 7fdb2ffff700 1 --2- 172.21.2.101:0/1882599981 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb301b0af0 0x7fdb30108950 secure :-1 s=READY pgs=2988 cs=0 l=1 rev1=1 rx=0x7fdb20006700 tx=0x7fdb20009b80).ready entity=mon.0 client_cookie=ed96dfd7e22c0248 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:39.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.472 7fdb2f7fe700 1 -- 172.21.2.101:0/1882599981 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdb2000c0c0 con 0x7fdb301b0af0 2021-02-10T11:46:39.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.472 7fdb2f7fe700 1 -- 172.21.2.101:0/1882599981 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2000c440 con 0x7fdb301b0af0 2021-02-10T11:46:39.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.472 7fdb2f7fe700 1 -- 172.21.2.101:0/1882599981 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdb2000c960 con 0x7fdb301b0af0 2021-02-10T11:46:39.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.472 7fdb36af6700 1 -- 172.21.2.101:0/1882599981 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb301b0af0 msgr2=0x7fdb30108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:39.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.472 7fdb36af6700 1 --2- 172.21.2.101:0/1882599981 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb301b0af0 0x7fdb30108950 secure :-1 s=READY pgs=2988 cs=0 l=1 rev1=1 rx=0x7fdb20006700 tx=0x7fdb20009b80).stop 2021-02-10T11:46:39.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.473 7fdb36af6700 1 -- 172.21.2.101:0/1882599981 shutdown_connections 2021-02-10T11:46:39.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.473 7fdb36af6700 1 --2- 172.21.2.101:0/1882599981 >> v2:172.21.2.108:3300/0 conn(0x7fdb300d1c80 0x7fdb30108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:39.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.473 7fdb36af6700 1 --2- 172.21.2.101:0/1882599981 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb301b0af0 0x7fdb30108950 unknown :-1 s=CLOSED pgs=2988 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:39.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.473 7fdb36af6700 1 --2- 172.21.2.101:0/1882599981 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb300d1880 0x7fdb301b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:39.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.473 7fdb36af6700 1 -- 172.21.2.101:0/1882599981 >> 172.21.2.101:0/1882599981 conn(0x7fdb300ce430 msgr2=0x7fdb300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:39.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.473 7fdb36af6700 1 -- 172.21.2.101:0/1882599981 shutdown_connections 2021-02-10T11:46:39.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.473 7fdb36af6700 1 -- 172.21.2.101:0/1882599981 wait complete. 2021-02-10T11:46:39.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.488 7fdb36af6700 1 Processor -- start 2021-02-10T11:46:39.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.488 7fdb36af6700 1 -- start start 2021-02-10T11:46:39.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.488 7fdb36af6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb30086dd0 0x7fdb30082b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:39.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.488 7fdb36af6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdb300ae8f0 0x7fdb300aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:39.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.488 7fdb36af6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb301adfb0 0x7fdb301ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:39.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.488 7fdb36af6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb300871d0 con 0x7fdb301adfb0 2021-02-10T11:46:39.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.488 7fdb36af6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb301ae9d0 con 0x7fdb300ae8f0 2021-02-10T11:46:39.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.488 7fdb36af6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb301aebf0 con 0x7fdb30086dd0 2021-02-10T11:46:39.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.489 7fdb35113700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb301adfb0 0x7fdb301ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:39.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.489 7fdb35113700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb301adfb0 0x7fdb301ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33422/0 (socket says 172.21.2.101:33422) 2021-02-10T11:46:39.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.489 7fdb35113700 1 -- 172.21.2.101:0/3114731818 learned_addr learned my addr 172.21.2.101:0/3114731818 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:39.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.489 7fdb35113700 1 -- 172.21.2.101:0/3114731818 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb30086dd0 msgr2=0x7fdb30082b30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:39.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.489 7fdb35113700 1 --2- 172.21.2.101:0/3114731818 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb30086dd0 0x7fdb30082b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:39.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.489 7fdb35113700 1 -- 172.21.2.101:0/3114731818 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdb300ae8f0 msgr2=0x7fdb300aecf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:39.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.489 7fdb35113700 1 --2- 172.21.2.101:0/3114731818 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdb300ae8f0 0x7fdb300aecf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:39.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.489 7fdb35113700 1 -- 172.21.2.101:0/3114731818 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb30083870 con 0x7fdb301adfb0 2021-02-10T11:46:39.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.489 7fdb35113700 1 --2- 172.21.2.101:0/3114731818 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb301adfb0 0x7fdb301ae3b0 secure :-1 s=READY pgs=2989 cs=0 l=1 rev1=1 rx=0x7fdb28009960 tx=0x7fdb2800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:39.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.489 7fdb2d7fa700 1 -- 172.21.2.101:0/3114731818 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdb28003030 con 0x7fdb301adfb0 2021-02-10T11:46:39.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.489 7fdb2d7fa700 1 -- 172.21.2.101:0/3114731818 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb28001310 con 0x7fdb301adfb0 2021-02-10T11:46:39.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.489 7fdb2d7fa700 1 -- 172.21.2.101:0/3114731818 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdb28001830 con 0x7fdb301adfb0 2021-02-10T11:46:39.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.489 7fdb36af6700 1 -- 172.21.2.101:0/3114731818 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb30083870 con 0x7fdb301adfb0 2021-02-10T11:46:39.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.489 7fdb36af6700 1 -- 172.21.2.101:0/3114731818 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb30083e00 con 0x7fdb301adfb0 2021-02-10T11:46:39.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.490 7fdb2d7fa700 1 -- 172.21.2.101:0/3114731818 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdb280014f0 con 0x7fdb301adfb0 2021-02-10T11:46:39.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.490 7fdb2d7fa700 1 --2- 172.21.2.101:0/3114731818 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdb140186c0 0x7fdb1401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:39.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.490 7fdb2d7fa700 1 -- 172.21.2.101:0/3114731818 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fdb28032950 con 0x7fdb301adfb0 2021-02-10T11:46:39.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.490 7fdb34912700 1 --2- 172.21.2.101:0/3114731818 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdb140186c0 0x7fdb1401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:39.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.490 7fdb34912700 1 --2- 172.21.2.101:0/3114731818 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdb140186c0 0x7fdb1401abc0 crc :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:39.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.521 7fdb36af6700 1 -- 172.21.2.101:0/3114731818 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb30082900 con 0x7fdb301adfb0 2021-02-10T11:46:39.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.523 7fdb2d7fa700 1 -- 172.21.2.101:0/3114731818 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdb28001ee0 con 0x7fdb301adfb0 2021-02-10T11:46:39.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.653 7fdb36af6700 1 -- 172.21.2.101:0/3114731818 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fdb30109a70 con 0x7fdb301adfb0 2021-02-10T11:46:39.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.656 7fdb2d7fa700 1 -- 172.21.2.101:0/3114731818 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v96) v1 ==== 184+0+0 (secure 0 0 0) 0x7fdb280013d0 con 0x7fdb301adfb0 2021-02-10T11:46:39.658 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.1 2021-02-10T11:46:39.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.661 7fdb36af6700 1 -- 172.21.2.101:0/3114731818 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdb140186c0 msgr2=0x7fdb1401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:39.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.661 7fdb36af6700 1 --2- 172.21.2.101:0/3114731818 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdb140186c0 0x7fdb1401abc0 crc :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:39.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.661 7fdb36af6700 1 -- 172.21.2.101:0/3114731818 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb301adfb0 msgr2=0x7fdb301ae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:39.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.662 7fdb36af6700 1 --2- 172.21.2.101:0/3114731818 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb301adfb0 0x7fdb301ae3b0 secure :-1 s=READY pgs=2989 cs=0 l=1 rev1=1 rx=0x7fdb28009960 tx=0x7fdb2800bcf0).stop 2021-02-10T11:46:39.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.662 7fdb36af6700 1 -- 172.21.2.101:0/3114731818 shutdown_connections 2021-02-10T11:46:39.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.662 7fdb36af6700 1 --2- 172.21.2.101:0/3114731818 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdb301adfb0 0x7fdb301ae3b0 unknown :-1 s=CLOSED pgs=2989 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:39.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.662 7fdb36af6700 1 --2- 172.21.2.101:0/3114731818 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdb300ae8f0 0x7fdb300aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:39.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.662 7fdb36af6700 1 --2- 172.21.2.101:0/3114731818 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdb140186c0 0x7fdb1401abc0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:39.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.662 7fdb36af6700 1 --2- 172.21.2.101:0/3114731818 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdb30086dd0 0x7fdb30082b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:39.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.662 7fdb36af6700 1 -- 172.21.2.101:0/3114731818 >> 172.21.2.101:0/3114731818 conn(0x7fdb301b9600 msgr2=0x7fdb301b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:39.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.662 7fdb36af6700 1 -- 172.21.2.101:0/3114731818 shutdown_connections 2021-02-10T11:46:39.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:39.662 7fdb36af6700 1 -- 172.21.2.101:0/3114731818 wait complete. 2021-02-10T11:46:39.668 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:46:40.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.094 7f49138c5700 1 Processor -- start 2021-02-10T11:46:40.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.094 7f49138c5700 1 -- start start 2021-02-10T11:46:40.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.095 7f49138c5700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f490c0d1c80 0x7f490c0cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:40.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.095 7f49138c5700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f490c0cf590 0x7f490c0cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:40.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.095 7f49138c5700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490c0fe1d0 con 0x7f490c0d1880 2021-02-10T11:46:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.095 7f49138c5700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f490c1b0cc0 con 0x7f490c0cf590 2021-02-10T11:46:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.095 7f49138c5700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f490c109a70 con 0x7f490c0d1c80 2021-02-10T11:46:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.095 7f49116e1700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f490c0d1880 0x7f490c1b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33920/0 (socket says 172.21.2.101:33920) 2021-02-10T11:46:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.095 7f49116e1700 1 -- 172.21.2.101:0/2787543934 learned_addr learned my addr 172.21.2.101:0/2787543934 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.095 7f4911ee2700 1 --2- 172.21.2.101:0/2787543934 >> v2:172.21.2.101:3300/0 conn(0x7f490c0cf590 0x7f490c0cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.095 7f4911ee2700 1 -- 172.21.2.101:0/2787543934 >> v2:172.21.2.108:3300/0 conn(0x7f490c0d1c80 msgr2=0x7f490c0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.095 7f4911ee2700 1 --2- 172.21.2.101:0/2787543934 >> v2:172.21.2.108:3300/0 conn(0x7f490c0d1c80 0x7f490c0cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.095 7f4911ee2700 1 -- 172.21.2.101:0/2787543934 >> v1:172.21.2.101:6789/0 conn(0x7f490c0d1880 legacy=0x7f490c1b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.095 7f4911ee2700 1 -- 172.21.2.101:0/2787543934 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c088070 con 0x7f490c0cf590 2021-02-10T11:46:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.095 7f4911ee2700 1 --2- 172.21.2.101:0/2787543934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f490c0cf590 0x7f490c0cf990 secure :-1 s=READY pgs=2991 cs=0 l=1 rev1=1 rx=0x7f4904006e30 tx=0x7f4904009e60).ready entity=mon.0 client_cookie=81a219fbec2985b6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.096 7f48fffff700 1 -- 172.21.2.101:0/2787543934 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4904003030 con 0x7f490c0cf590 2021-02-10T11:46:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.096 7f48fffff700 1 -- 172.21.2.101:0/2787543934 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490400b300 con 0x7f490c0cf590 2021-02-10T11:46:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.096 7f48fffff700 1 -- 172.21.2.101:0/2787543934 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f490400b820 con 0x7f490c0cf590 2021-02-10T11:46:40.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.096 7f49138c5700 1 -- 172.21.2.101:0/2787543934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f490c0cf590 msgr2=0x7f490c0cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.096 7f49138c5700 1 --2- 172.21.2.101:0/2787543934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f490c0cf590 0x7f490c0cf990 secure :-1 s=READY pgs=2991 cs=0 l=1 rev1=1 rx=0x7f4904006e30 tx=0x7f4904009e60).stop 2021-02-10T11:46:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.096 7f49138c5700 1 -- 172.21.2.101:0/2787543934 shutdown_connections 2021-02-10T11:46:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.096 7f49138c5700 1 --2- 172.21.2.101:0/2787543934 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f490c0cf590 0x7f490c0cf990 unknown :-1 s=CLOSED pgs=2991 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.096 7f49138c5700 1 --2- 172.21.2.101:0/2787543934 >> v2:172.21.2.108:3300/0 conn(0x7f490c0d1c80 0x7f490c0cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.096 7f49138c5700 1 -- 172.21.2.101:0/2787543934 >> 172.21.2.101:0/2787543934 conn(0x7f490c0ce430 msgr2=0x7f490c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.097 7f49138c5700 1 -- 172.21.2.101:0/2787543934 shutdown_connections 2021-02-10T11:46:40.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.097 7f49138c5700 1 -- 172.21.2.101:0/2787543934 wait complete. 2021-02-10T11:46:40.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.111 7f49138c5700 1 Processor -- start 2021-02-10T11:46:40.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.111 7f49138c5700 1 -- start start 2021-02-10T11:46:40.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.111 7f49138c5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f490c0b0ac0 0x7f490c09b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:40.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.111 7f49138c5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f490c09b710 0x7f490c0982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:40.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.111 7f49138c5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f490c196610 0x7f490c0988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:40.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.111 7f49138c5700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c196c00 con 0x7f490c0b0ac0 2021-02-10T11:46:40.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.111 7f49138c5700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c197180 con 0x7f490c09b710 2021-02-10T11:46:40.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.111 7f49138c5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c197360 con 0x7f490c196610 2021-02-10T11:46:40.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.112 7f49116e1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f490c0b0ac0 0x7f490c09b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:40.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.112 7f49116e1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f490c0b0ac0 0x7f490c09b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33432/0 (socket says 172.21.2.101:33432) 2021-02-10T11:46:40.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.112 7f49116e1700 1 -- 172.21.2.101:0/2252282158 learned_addr learned my addr 172.21.2.101:0/2252282158 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:40.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.112 7f4910ee0700 1 --2- 172.21.2.101:0/2252282158 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f490c09b710 0x7f490c0982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:40.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.112 7f4911ee2700 1 --2- 172.21.2.101:0/2252282158 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f490c196610 0x7f490c0988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:40.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.112 7f49116e1700 1 -- 172.21.2.101:0/2252282158 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f490c196610 msgr2=0x7f490c0988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:40.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.112 7f49116e1700 1 --2- 172.21.2.101:0/2252282158 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f490c196610 0x7f490c0988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:40.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.112 7f49116e1700 1 -- 172.21.2.101:0/2252282158 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f490c09b710 msgr2=0x7f490c0982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:40.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.112 7f49116e1700 1 --2- 172.21.2.101:0/2252282158 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f490c09b710 0x7f490c0982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:40.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.112 7f49116e1700 1 -- 172.21.2.101:0/2252282158 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c197580 con 0x7f490c0b0ac0 2021-02-10T11:46:40.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.112 7f4911ee2700 1 --2- 172.21.2.101:0/2252282158 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f490c196610 0x7f490c0988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:46:40.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.112 7f49116e1700 1 --2- 172.21.2.101:0/2252282158 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f490c0b0ac0 0x7f490c09b0f0 secure :-1 s=READY pgs=2992 cs=0 l=1 rev1=1 rx=0x7f4908001400 tx=0x7f490800a900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:40.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.112 7f48fdffb700 1 -- 172.21.2.101:0/2252282158 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f490800acf0 con 0x7f490c0b0ac0 2021-02-10T11:46:40.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.112 7f48fdffb700 1 -- 172.21.2.101:0/2252282158 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4908000c90 con 0x7f490c0b0ac0 2021-02-10T11:46:40.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.112 7f48fdffb700 1 -- 172.21.2.101:0/2252282158 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4908004590 con 0x7f490c0b0ac0 2021-02-10T11:46:40.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.112 7f49138c5700 1 -- 172.21.2.101:0/2252282158 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490c197580 con 0x7f490c0b0ac0 2021-02-10T11:46:40.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.113 7f49138c5700 1 -- 172.21.2.101:0/2252282158 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490c197580 con 0x7f490c0b0ac0 2021-02-10T11:46:40.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.113 7f48fdffb700 1 -- 172.21.2.101:0/2252282158 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f49080043a0 con 0x7f490c0b0ac0 2021-02-10T11:46:40.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.113 7f48fdffb700 1 --2- 172.21.2.101:0/2252282158 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f48ec0186d0 0x7f48ec01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:40.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.113 7f48fdffb700 1 -- 172.21.2.101:0/2252282158 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4908032790 con 0x7f490c0b0ac0 2021-02-10T11:46:40.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.113 7f4910ee0700 1 --2- 172.21.2.101:0/2252282158 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f48ec0186d0 0x7f48ec01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:40.114 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.114 7f4910ee0700 1 --2- 172.21.2.101:0/2252282158 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f48ec0186d0 0x7f48ec01abd0 crc :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:40.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.145 7f49138c5700 1 -- 172.21.2.101:0/2252282158 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490c0829a0 con 0x7f490c0b0ac0 2021-02-10T11:46:40.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.147 7f48fdffb700 1 -- 172.21.2.101:0/2252282158 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4908007070 con 0x7f490c0b0ac0 2021-02-10T11:46:40.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.277 7f49138c5700 1 -- 172.21.2.101:0/2252282158 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f490c0829a0 con 0x7f490c0b0ac0 2021-02-10T11:46:40.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.281 7f48fdffb700 1 -- 172.21.2.101:0/2252282158 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v97) v1 ==== 184+0+0 (secure 0 0 0) 0x7f49080043a0 con 0x7f490c0b0ac0 2021-02-10T11:46:40.282 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.2 2021-02-10T11:46:40.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.286 7f49138c5700 1 -- 172.21.2.101:0/2252282158 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f48ec0186d0 msgr2=0x7f48ec01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:40.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.286 7f49138c5700 1 --2- 172.21.2.101:0/2252282158 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f48ec0186d0 0x7f48ec01abd0 crc :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:40.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.286 7f49138c5700 1 -- 172.21.2.101:0/2252282158 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f490c0b0ac0 msgr2=0x7f490c09b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:40.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.286 7f49138c5700 1 --2- 172.21.2.101:0/2252282158 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f490c0b0ac0 0x7f490c09b0f0 secure :-1 s=READY pgs=2992 cs=0 l=1 rev1=1 rx=0x7f4908001400 tx=0x7f490800a900).stop 2021-02-10T11:46:40.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.287 7f49138c5700 1 -- 172.21.2.101:0/2252282158 shutdown_connections 2021-02-10T11:46:40.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.287 7f49138c5700 1 --2- 172.21.2.101:0/2252282158 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f48ec0186d0 0x7f48ec01abd0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:40.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.287 7f49138c5700 1 --2- 172.21.2.101:0/2252282158 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f490c196610 0x7f490c0988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:40.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.287 7f49138c5700 1 --2- 172.21.2.101:0/2252282158 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f490c09b710 0x7f490c0982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:40.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.287 7f49138c5700 1 --2- 172.21.2.101:0/2252282158 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f490c0b0ac0 0x7f490c09b0f0 unknown :-1 s=CLOSED pgs=2992 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:40.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.287 7f49138c5700 1 -- 172.21.2.101:0/2252282158 >> 172.21.2.101:0/2252282158 conn(0x7f490c0a2860 msgr2=0x7f490c082bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:40.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.287 7f49138c5700 1 -- 172.21.2.101:0/2252282158 shutdown_connections 2021-02-10T11:46:40.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.287 7f49138c5700 1 -- 172.21.2.101:0/2252282158 wait complete. 2021-02-10T11:46:40.293 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:46:40.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.720 7f935bc71700 1 Processor -- start 2021-02-10T11:46:40.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.720 7f935bc71700 1 -- start start 2021-02-10T11:46:40.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.721 7f935bc71700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f93540d1880 0x7f93541b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:40.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.721 7f935bc71700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93541b0af0 0x7f9354108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:40.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.721 7f935bc71700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93540fe1d0 con 0x7f93540d1c80 2021-02-10T11:46:40.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.721 7f935bc71700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9354109a70 con 0x7f93540d1880 2021-02-10T11:46:40.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.721 7f935bc71700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9354109760 con 0x7f93541b0af0 2021-02-10T11:46:40.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.722 7f935928c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93541b0af0 0x7f9354108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:40.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.722 7f9359a8d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f93540d1880 0x7f93541b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:40.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.722 7f935a28e700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f93540d1c80 0x7f9354108f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:55576/0 (socket says 172.21.2.101:55576) 2021-02-10T11:46:40.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.722 7f935a28e700 1 -- 172.21.2.101:0/2298549066 learned_addr learned my addr 172.21.2.101:0/2298549066 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:40.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.722 7f935928c700 1 -- 172.21.2.101:0/2298549066 >> v2:172.21.2.108:3300/0 conn(0x7f93540d1880 msgr2=0x7f93541b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:40.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.722 7f935928c700 1 --2- 172.21.2.101:0/2298549066 >> v2:172.21.2.108:3300/0 conn(0x7f93540d1880 0x7f93541b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:40.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.722 7f935928c700 1 -- 172.21.2.101:0/2298549066 >> v1:172.21.2.108:6789/0 conn(0x7f93540d1c80 legacy=0x7f9354108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:40.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.722 7f935928c700 1 -- 172.21.2.101:0/2298549066 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9354088070 con 0x7f93541b0af0 2021-02-10T11:46:40.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.722 7f9359a8d700 1 --2- 172.21.2.101:0/2298549066 >> v2:172.21.2.108:3300/0 conn(0x7f93540d1880 0x7f93541b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:46:40.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.722 7f935928c700 1 --2- 172.21.2.101:0/2298549066 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93541b0af0 0x7f9354108950 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f9350006650 tx=0x7f9350009a80).ready entity=mon.2 client_cookie=44ae472188c8013b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:40.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.723 7f9358a8b700 1 -- 172.21.2.101:0/2298549066 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f935000c0c0 con 0x7f93541b0af0 2021-02-10T11:46:40.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.723 7f9358a8b700 1 -- 172.21.2.101:0/2298549066 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935000c340 con 0x7f93541b0af0 2021-02-10T11:46:40.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.723 7f9358a8b700 1 -- 172.21.2.101:0/2298549066 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f935000c860 con 0x7f93541b0af0 2021-02-10T11:46:40.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.723 7f935bc71700 1 -- 172.21.2.101:0/2298549066 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93541b0af0 msgr2=0x7f9354108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:40.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.723 7f935bc71700 1 --2- 172.21.2.101:0/2298549066 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93541b0af0 0x7f9354108950 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f9350006650 tx=0x7f9350009a80).stop 2021-02-10T11:46:40.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.724 7f935bc71700 1 -- 172.21.2.101:0/2298549066 shutdown_connections 2021-02-10T11:46:40.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.724 7f935bc71700 1 --2- 172.21.2.101:0/2298549066 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93541b0af0 0x7f9354108950 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:40.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.724 7f935bc71700 1 --2- 172.21.2.101:0/2298549066 >> v2:172.21.2.108:3300/0 conn(0x7f93540d1880 0x7f93541b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:40.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.724 7f935bc71700 1 -- 172.21.2.101:0/2298549066 >> 172.21.2.101:0/2298549066 conn(0x7f93540ce430 msgr2=0x7f93540d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:40.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.724 7f935bc71700 1 -- 172.21.2.101:0/2298549066 shutdown_connections 2021-02-10T11:46:40.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.724 7f935bc71700 1 -- 172.21.2.101:0/2298549066 wait complete. 2021-02-10T11:46:40.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.738 7f935bc71700 1 Processor -- start 2021-02-10T11:46:40.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.739 7f935bc71700 1 -- start start 2021-02-10T11:46:40.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.739 7f935bc71700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9354095600 0x7f9354095a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:40.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.739 7f935bc71700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9354096020 0x7f9354096420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:40.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.739 7f935bc71700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9354092a10 0x7f9354092e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:40.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.739 7f935bc71700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9354094100 con 0x7f9354092a10 2021-02-10T11:46:40.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.739 7f935bc71700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935419e9e0 con 0x7f9354096020 2021-02-10T11:46:40.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.739 7f935bc71700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9354093430 con 0x7f9354095600 2021-02-10T11:46:40.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.739 7f935a28e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9354092a10 0x7f9354092e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:40.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.739 7f935a28e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9354092a10 0x7f9354092e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33444/0 (socket says 172.21.2.101:33444) 2021-02-10T11:46:40.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.739 7f935a28e700 1 -- 172.21.2.101:0/2857921119 learned_addr learned my addr 172.21.2.101:0/2857921119 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:40.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.740 7f935a28e700 1 -- 172.21.2.101:0/2857921119 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9354095600 msgr2=0x7f9354095a00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:40.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.740 7f935a28e700 1 --2- 172.21.2.101:0/2857921119 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9354095600 0x7f9354095a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:40.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.740 7f935a28e700 1 -- 172.21.2.101:0/2857921119 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9354096020 msgr2=0x7f9354096420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:40.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.740 7f935a28e700 1 --2- 172.21.2.101:0/2857921119 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9354096020 0x7f9354096420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:40.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.740 7f935a28e700 1 -- 172.21.2.101:0/2857921119 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9354093650 con 0x7f9354092a10 2021-02-10T11:46:40.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.740 7f935a28e700 1 --2- 172.21.2.101:0/2857921119 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9354092a10 0x7f9354092e10 secure :-1 s=READY pgs=2993 cs=0 l=1 rev1=1 rx=0x7f934c007f90 tx=0x7f934c00ac70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:40.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.740 7f93467fc700 1 -- 172.21.2.101:0/2857921119 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f934c0012b0 con 0x7f9354092a10 2021-02-10T11:46:40.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.740 7f93467fc700 1 -- 172.21.2.101:0/2857921119 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934c0070d0 con 0x7f9354092a10 2021-02-10T11:46:40.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.740 7f93467fc700 1 -- 172.21.2.101:0/2857921119 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f934c0075b0 con 0x7f9354092a10 2021-02-10T11:46:40.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.740 7f935bc71700 1 -- 172.21.2.101:0/2857921119 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9354093650 con 0x7f9354092a10 2021-02-10T11:46:40.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.740 7f935bc71700 1 -- 172.21.2.101:0/2857921119 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9354093650 con 0x7f9354092a10 2021-02-10T11:46:40.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.741 7f93467fc700 1 -- 172.21.2.101:0/2857921119 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f934c0072b0 con 0x7f9354092a10 2021-02-10T11:46:40.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.741 7f93467fc700 1 --2- 172.21.2.101:0/2857921119 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f933c0186c0 0x7f933c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:40.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.741 7f93467fc700 1 -- 172.21.2.101:0/2857921119 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f934c0328f0 con 0x7f9354092a10 2021-02-10T11:46:40.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.741 7f9359a8d700 1 --2- 172.21.2.101:0/2857921119 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f933c0186c0 0x7f933c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:40.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.741 7f9359a8d700 1 --2- 172.21.2.101:0/2857921119 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f933c0186c0 0x7f933c01abc0 crc :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:40.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.771 7f935bc71700 1 -- 172.21.2.101:0/2857921119 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9354082980 con 0x7f9354092a10 2021-02-10T11:46:40.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.773 7f93467fc700 1 -- 172.21.2.101:0/2857921119 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f934c0012b0 con 0x7f9354092a10 2021-02-10T11:46:40.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.904 7f935bc71700 1 -- 172.21.2.101:0/2857921119 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f9354082980 con 0x7f9354092a10 2021-02-10T11:46:40.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.909 7f93467fc700 1 -- 172.21.2.101:0/2857921119 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v98) v1 ==== 184+0+0 (secure 0 0 0) 0x7f934c0012b0 con 0x7f9354092a10 2021-02-10T11:46:40.909 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.3 2021-02-10T11:46:40.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.913 7f935bc71700 1 -- 172.21.2.101:0/2857921119 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f933c0186c0 msgr2=0x7f933c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:40.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.913 7f935bc71700 1 --2- 172.21.2.101:0/2857921119 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f933c0186c0 0x7f933c01abc0 crc :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:40.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.913 7f935bc71700 1 -- 172.21.2.101:0/2857921119 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9354092a10 msgr2=0x7f9354092e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:40.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.913 7f935bc71700 1 --2- 172.21.2.101:0/2857921119 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9354092a10 0x7f9354092e10 secure :-1 s=READY pgs=2993 cs=0 l=1 rev1=1 rx=0x7f934c007f90 tx=0x7f934c00ac70).stop 2021-02-10T11:46:40.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.913 7f935bc71700 1 -- 172.21.2.101:0/2857921119 shutdown_connections 2021-02-10T11:46:40.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.914 7f935bc71700 1 --2- 172.21.2.101:0/2857921119 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9354092a10 0x7f9354092e10 unknown :-1 s=CLOSED pgs=2993 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:40.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.914 7f935bc71700 1 --2- 172.21.2.101:0/2857921119 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9354096020 0x7f9354096420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:40.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.914 7f935bc71700 1 --2- 172.21.2.101:0/2857921119 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f933c0186c0 0x7f933c01abc0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:40.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.914 7f935bc71700 1 --2- 172.21.2.101:0/2857921119 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9354095600 0x7f9354095a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:40.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.914 7f935bc71700 1 -- 172.21.2.101:0/2857921119 >> 172.21.2.101:0/2857921119 conn(0x7f93541a6190 msgr2=0x7f93540949b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:40.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.914 7f935bc71700 1 -- 172.21.2.101:0/2857921119 shutdown_connections 2021-02-10T11:46:40.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:40.914 7f935bc71700 1 -- 172.21.2.101:0/2857921119 wait complete. 2021-02-10T11:46:40.920 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:46:41.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.336 7fbd6998b700 1 Processor -- start 2021-02-10T11:46:41.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.336 7fbd6998b700 1 -- start start 2021-02-10T11:46:41.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.336 7fbd6998b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbd640d1870 0x7fbd641b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:41.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.336 7fbd6998b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbd641b0a10 0x7fbd640cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:41.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.336 7fbd6998b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd640f0b10 con 0x7fbd640d1c70 2021-02-10T11:46:41.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.336 7fbd6998b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd640e5ed0 con 0x7fbd640d1870 2021-02-10T11:46:41.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.336 7fbd6998b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd640f0650 con 0x7fbd641b0a10 2021-02-10T11:46:41.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.337 7fbd63d75700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fbd640d1c70 0x7fbd640cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33946/0 (socket says 172.21.2.101:33946) 2021-02-10T11:46:41.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.337 7fbd63d75700 1 -- 172.21.2.101:0/238724554 learned_addr learned my addr 172.21.2.101:0/238724554 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:41.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.337 7fbd63574700 1 --2- 172.21.2.101:0/238724554 >> v2:172.21.2.101:3300/0 conn(0x7fbd640d1870 0x7fbd641b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:41.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.337 7fbd62d73700 1 --2- 172.21.2.101:0/238724554 >> v2:172.21.2.108:3300/0 conn(0x7fbd641b0a10 0x7fbd640cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:41.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.337 7fbd62572700 1 -- 172.21.2.101:0/238724554 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74724556 0 0) 0x7fbd5c001500 con 0x7fbd640d1c70 2021-02-10T11:46:41.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.337 7fbd62572700 1 -- 172.21.2.101:0/238724554 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd50002fe0 con 0x7fbd640d1c70 2021-02-10T11:46:41.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.337 7fbd62d73700 1 -- 172.21.2.101:0/238724554 >> v2:172.21.2.101:3300/0 conn(0x7fbd640d1870 msgr2=0x7fbd641b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:41.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.337 7fbd62d73700 1 --2- 172.21.2.101:0/238724554 >> v2:172.21.2.101:3300/0 conn(0x7fbd640d1870 0x7fbd641b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:41.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.337 7fbd62d73700 1 -- 172.21.2.101:0/238724554 >> v1:172.21.2.101:6789/0 conn(0x7fbd640d1c70 legacy=0x7fbd640cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:41.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.337 7fbd62d73700 1 -- 172.21.2.101:0/238724554 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd640f0840 con 0x7fbd641b0a10 2021-02-10T11:46:41.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.337 7fbd63574700 1 --2- 172.21.2.101:0/238724554 >> v2:172.21.2.101:3300/0 conn(0x7fbd640d1870 0x7fbd641b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:46:41.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.337 7fbd62d73700 1 --2- 172.21.2.101:0/238724554 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbd641b0a10 0x7fbd640cf330 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7fbd54006650 tx=0x7fbd54009a80).ready entity=mon.1 client_cookie=f2d83f795b0150db server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:41.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.338 7fbd62572700 1 -- 172.21.2.101:0/238724554 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbd5400c0c0 con 0x7fbd641b0a10 2021-02-10T11:46:41.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.338 7fbd62572700 1 -- 172.21.2.101:0/238724554 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5400c340 con 0x7fbd641b0a10 2021-02-10T11:46:41.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.338 7fbd62572700 1 -- 172.21.2.101:0/238724554 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbd5400c860 con 0x7fbd641b0a10 2021-02-10T11:46:41.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.338 7fbd6998b700 1 -- 172.21.2.101:0/238724554 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbd641b0a10 msgr2=0x7fbd640cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:41.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.338 7fbd6998b700 1 --2- 172.21.2.101:0/238724554 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbd641b0a10 0x7fbd640cf330 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7fbd54006650 tx=0x7fbd54009a80).stop 2021-02-10T11:46:41.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.338 7fbd6998b700 1 -- 172.21.2.101:0/238724554 shutdown_connections 2021-02-10T11:46:41.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.338 7fbd6998b700 1 --2- 172.21.2.101:0/238724554 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbd641b0a10 0x7fbd640cf330 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:41.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.338 7fbd6998b700 1 --2- 172.21.2.101:0/238724554 >> v2:172.21.2.101:3300/0 conn(0x7fbd640d1870 0x7fbd641b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:41.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.338 7fbd6998b700 1 -- 172.21.2.101:0/238724554 >> 172.21.2.101:0/238724554 conn(0x7fbd640ce450 msgr2=0x7fbd640d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:41.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.338 7fbd6998b700 1 -- 172.21.2.101:0/238724554 shutdown_connections 2021-02-10T11:46:41.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.338 7fbd6998b700 1 -- 172.21.2.101:0/238724554 wait complete. 2021-02-10T11:46:41.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.353 7fbd6998b700 1 Processor -- start 2021-02-10T11:46:41.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.353 7fbd6998b700 1 -- start start 2021-02-10T11:46:41.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.353 7fbd6998b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd64195650 0x7fbd640b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:41.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.353 7fbd6998b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd640b5b80 0x7fbd641a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:41.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.353 7fbd6998b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbd640acfa0 0x7fbd641a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:41.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.353 7fbd6998b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd640b6010 con 0x7fbd64195650 2021-02-10T11:46:41.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.353 7fbd6998b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd641a3a60 con 0x7fbd640acfa0 2021-02-10T11:46:41.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.353 7fbd6998b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd641a3c40 con 0x7fbd640b5b80 2021-02-10T11:46:41.354 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.354 7fbd63574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd64195650 0x7fbd640b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:41.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.354 7fbd63574700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd64195650 0x7fbd640b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33456/0 (socket says 172.21.2.101:33456) 2021-02-10T11:46:41.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.354 7fbd63574700 1 -- 172.21.2.101:0/1456618595 learned_addr learned my addr 172.21.2.101:0/1456618595 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:41.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.354 7fbd63574700 1 -- 172.21.2.101:0/1456618595 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd640b5b80 msgr2=0x7fbd641a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:41.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.354 7fbd63574700 1 --2- 172.21.2.101:0/1456618595 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd640b5b80 0x7fbd641a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:41.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.354 7fbd63574700 1 -- 172.21.2.101:0/1456618595 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbd640acfa0 msgr2=0x7fbd641a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:41.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.354 7fbd63574700 1 --2- 172.21.2.101:0/1456618595 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbd640acfa0 0x7fbd641a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:41.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.354 7fbd63574700 1 -- 172.21.2.101:0/1456618595 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd640bcae0 con 0x7fbd64195650 2021-02-10T11:46:41.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.354 7fbd63574700 1 --2- 172.21.2.101:0/1456618595 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd64195650 0x7fbd640b5560 secure :-1 s=READY pgs=2995 cs=0 l=1 rev1=1 rx=0x7fbd4c00b580 tx=0x7fbd4c00b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:41.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.355 7fbd5bfff700 1 -- 172.21.2.101:0/1456618595 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbd4c00bd10 con 0x7fbd64195650 2021-02-10T11:46:41.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.355 7fbd6998b700 1 -- 172.21.2.101:0/1456618595 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd640bcae0 con 0x7fbd64195650 2021-02-10T11:46:41.355 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.355 7fbd6998b700 1 -- 172.21.2.101:0/1456618595 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd640bd070 con 0x7fbd64195650 2021-02-10T11:46:41.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.355 7fbd5bfff700 1 -- 172.21.2.101:0/1456618595 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4c00bd10 con 0x7fbd64195650 2021-02-10T11:46:41.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.355 7fbd5bfff700 1 -- 172.21.2.101:0/1456618595 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbd4c000c90 con 0x7fbd64195650 2021-02-10T11:46:41.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.355 7fbd5bfff700 1 -- 172.21.2.101:0/1456618595 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbd4c006b00 con 0x7fbd64195650 2021-02-10T11:46:41.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.355 7fbd5bfff700 1 --2- 172.21.2.101:0/1456618595 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd480186c0 0x7fbd4801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:41.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.355 7fbd5bfff700 1 -- 172.21.2.101:0/1456618595 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fbd4c0325a0 con 0x7fbd64195650 2021-02-10T11:46:41.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.356 7fbd62d73700 1 --2- 172.21.2.101:0/1456618595 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd480186c0 0x7fbd4801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:41.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.356 7fbd62d73700 1 --2- 172.21.2.101:0/1456618595 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd480186c0 0x7fbd4801abc0 crc :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:41.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.386 7fbd6998b700 1 -- 172.21.2.101:0/1456618595 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd640f0ab0 con 0x7fbd64195650 2021-02-10T11:46:41.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.389 7fbd5bfff700 1 -- 172.21.2.101:0/1456618595 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbd4c025fd0 con 0x7fbd64195650 2021-02-10T11:46:41.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.517 7fbd6998b700 1 -- 172.21.2.101:0/1456618595 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd640f0ab0 con 0x7fbd64195650 2021-02-10T11:46:41.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.517 7fbd5bfff700 1 -- 172.21.2.101:0/1456618595 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+2111 (secure 0 0 0) 0x7fbd4c025fd0 con 0x7fbd64195650 2021-02-10T11:46:41.515 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 148 2021-02-10T11:46:41.515 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:46:41.515 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":148,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":12780,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2200088074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2200088074},{"type":"v1","addr":"172.21.2.101:6835","nonce":2200088074}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":144}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:46:35.149920","modified":"2021-02-10 11:46:37.158088","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12777},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12777":{"gid":12777,"name":"b","rank":0,"incarnation":145,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/1343655613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":1343655613},{"type":"v1","addr":"172.21.2.108:6833","nonce":1343655613}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[37],"metadata_pool":36,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":18}]} 2021-02-10T11:46:41.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.519 7fbd6998b700 1 -- 172.21.2.101:0/1456618595 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd480186c0 msgr2=0x7fbd4801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:41.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.519 7fbd6998b700 1 --2- 172.21.2.101:0/1456618595 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd480186c0 0x7fbd4801abc0 crc :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:41.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.519 7fbd6998b700 1 -- 172.21.2.101:0/1456618595 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd64195650 msgr2=0x7fbd640b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:41.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.519 7fbd6998b700 1 --2- 172.21.2.101:0/1456618595 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd64195650 0x7fbd640b5560 secure :-1 s=READY pgs=2995 cs=0 l=1 rev1=1 rx=0x7fbd4c00b580 tx=0x7fbd4c00b920).stop 2021-02-10T11:46:41.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.519 7fbd6998b700 1 -- 172.21.2.101:0/1456618595 shutdown_connections 2021-02-10T11:46:41.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.519 7fbd6998b700 1 --2- 172.21.2.101:0/1456618595 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbd480186c0 0x7fbd4801abc0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:41.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.519 7fbd6998b700 1 --2- 172.21.2.101:0/1456618595 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbd640b5b80 0x7fbd641a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:41.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.519 7fbd6998b700 1 --2- 172.21.2.101:0/1456618595 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbd640acfa0 0x7fbd641a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:41.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.519 7fbd6998b700 1 --2- 172.21.2.101:0/1456618595 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbd64195650 0x7fbd640b5560 unknown :-1 s=CLOSED pgs=2995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:41.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.519 7fbd6998b700 1 -- 172.21.2.101:0/1456618595 >> 172.21.2.101:0/1456618595 conn(0x7fbd640b3040 msgr2=0x7fbd640c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:41.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.520 7fbd6998b700 1 -- 172.21.2.101:0/1456618595 shutdown_connections 2021-02-10T11:46:41.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:41.520 7fbd6998b700 1 -- 172.21.2.101:0/1456618595 wait complete. 2021-02-10T11:46:41.727 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 148, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:46:35.149920', 'modified': '2021-02-10 11:46:37.158088', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12777}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12777': {'gid': 12777, 'name': 'b', 'rank': 0, 'incarnation': 145, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/1343655613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 1343655613}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 1343655613}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [37], 'metadata_pool': 36, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:46:41.727 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:46:41.728 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:46:42.126 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:46:42.127 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:46:42.127 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:46:42.127 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 12777, 2021-02-10T11:46:42.127 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:46:42.127 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:46:42.127 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 6.9776410909999997, 2021-02-10T11:46:42.127 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 148, 2021-02-10T11:46:42.127 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 175, 2021-02-10T11:46:42.127 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 175, 2021-02-10T11:46:42.127 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 12.622844957 2021-02-10T11:46:42.127 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:46:42.130 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 12777, "want_state": "up:active", "state": "up:active", "rank_uptime": 6.9776410909999997, "mdsmap_epoch": 148, "osdmap_epoch": 175, "osdmap_epoch_barrier": 175, "uptime": 12.622844957 } 2021-02-10T11:46:42.131 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:46:42.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.546 7fad6da8e700 1 Processor -- start 2021-02-10T11:46:42.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.546 7fad6da8e700 1 -- start start 2021-02-10T11:46:42.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.546 7fad6da8e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fad680d1870 0x7fad681b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:42.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.546 7fad6da8e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad680f0650 0x7fad680cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:42.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.546 7fad6da8e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad680f0b10 con 0x7fad681b0a10 2021-02-10T11:46:42.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.546 7fad6da8e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad680e5ed0 con 0x7fad680d1870 2021-02-10T11:46:42.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.546 7fad6da8e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad680d1c70 con 0x7fad680f0650 2021-02-10T11:46:42.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.548 7fad67fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad680f0650 0x7fad680cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:42.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.548 7fad677fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fad680d1870 0x7fad681b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:42.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.548 7fad66ffd700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fad681b0a10 0x7fad680cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:55598/0 (socket says 172.21.2.101:55598) 2021-02-10T11:46:42.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.548 7fad66ffd700 1 -- 172.21.2.101:0/1847199479 learned_addr learned my addr 172.21.2.101:0/1847199479 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:42.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.548 7fad67fff700 1 -- 172.21.2.101:0/1847199479 >> v2:172.21.2.108:3300/0 conn(0x7fad680d1870 msgr2=0x7fad681b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:42.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.548 7fad67fff700 1 --2- 172.21.2.101:0/1847199479 >> v2:172.21.2.108:3300/0 conn(0x7fad680d1870 0x7fad681b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:42.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.548 7fad67fff700 1 -- 172.21.2.101:0/1847199479 >> v1:172.21.2.108:6789/0 conn(0x7fad681b0a10 legacy=0x7fad680cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:42.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.548 7fad67fff700 1 -- 172.21.2.101:0/1847199479 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad680d1e90 con 0x7fad680f0650 2021-02-10T11:46:42.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.548 7fad67fff700 1 --2- 172.21.2.101:0/1847199479 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad680f0650 0x7fad680cfb20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7fad60006e50 tx=0x7fad60009e70).ready entity=mon.2 client_cookie=8439b0ba3e1e96f0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:42.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.548 7fad667fc700 1 -- 172.21.2.101:0/1847199479 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fad60006a20 con 0x7fad680f0650 2021-02-10T11:46:42.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.548 7fad667fc700 1 -- 172.21.2.101:0/1847199479 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad60006790 con 0x7fad680f0650 2021-02-10T11:46:42.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.548 7fad667fc700 1 -- 172.21.2.101:0/1847199479 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fad60000f90 con 0x7fad680f0650 2021-02-10T11:46:42.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.548 7fad6da8e700 1 -- 172.21.2.101:0/1847199479 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad680f0650 msgr2=0x7fad680cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:42.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.548 7fad6da8e700 1 --2- 172.21.2.101:0/1847199479 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad680f0650 0x7fad680cfb20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7fad60006e50 tx=0x7fad60009e70).stop 2021-02-10T11:46:42.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.549 7fad6da8e700 1 -- 172.21.2.101:0/1847199479 shutdown_connections 2021-02-10T11:46:42.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.549 7fad6da8e700 1 --2- 172.21.2.101:0/1847199479 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad680f0650 0x7fad680cfb20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:42.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.549 7fad6da8e700 1 --2- 172.21.2.101:0/1847199479 >> v2:172.21.2.108:3300/0 conn(0x7fad680d1870 0x7fad681b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:42.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.549 7fad6da8e700 1 -- 172.21.2.101:0/1847199479 >> 172.21.2.101:0/1847199479 conn(0x7fad680ce450 msgr2=0x7fad680d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:42.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.549 7fad6da8e700 1 -- 172.21.2.101:0/1847199479 shutdown_connections 2021-02-10T11:46:42.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.549 7fad6da8e700 1 -- 172.21.2.101:0/1847199479 wait complete. 2021-02-10T11:46:42.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.564 7fad6da8e700 1 Processor -- start 2021-02-10T11:46:42.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.564 7fad6da8e700 1 -- start start 2021-02-10T11:46:42.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.564 7fad6da8e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad68195650 0x7fad680b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:42.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.564 7fad6da8e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad680b5b80 0x7fad681a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:42.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.564 7fad6da8e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fad680acfa0 0x7fad681a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:42.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.564 7fad6da8e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad680b6010 con 0x7fad680b5b80 2021-02-10T11:46:42.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.564 7fad6da8e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad681a3a60 con 0x7fad680acfa0 2021-02-10T11:46:42.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.564 7fad6da8e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad681a3c40 con 0x7fad68195650 2021-02-10T11:46:42.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.565 7fad66ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad680b5b80 0x7fad681a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:42.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.565 7fad66ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad680b5b80 0x7fad681a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33470/0 (socket says 172.21.2.101:33470) 2021-02-10T11:46:42.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.565 7fad66ffd700 1 -- 172.21.2.101:0/1953819294 learned_addr learned my addr 172.21.2.101:0/1953819294 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:42.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.565 7fad66ffd700 1 -- 172.21.2.101:0/1953819294 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad68195650 msgr2=0x7fad680b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:42.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.565 7fad66ffd700 1 --2- 172.21.2.101:0/1953819294 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad68195650 0x7fad680b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:42.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.565 7fad66ffd700 1 -- 172.21.2.101:0/1953819294 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fad680acfa0 msgr2=0x7fad681a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:42.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.565 7fad66ffd700 1 --2- 172.21.2.101:0/1953819294 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fad680acfa0 0x7fad681a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:42.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.565 7fad66ffd700 1 -- 172.21.2.101:0/1953819294 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad680bcae0 con 0x7fad680b5b80 2021-02-10T11:46:42.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.565 7fad66ffd700 1 --2- 172.21.2.101:0/1953819294 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad680b5b80 0x7fad681a2e20 secure :-1 s=READY pgs=2996 cs=0 l=1 rev1=1 rx=0x7fad58001c60 tx=0x7fad5800a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:42.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.565 7fad5ffff700 1 -- 172.21.2.101:0/1953819294 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fad5800acc0 con 0x7fad680b5b80 2021-02-10T11:46:42.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.565 7fad5ffff700 1 -- 172.21.2.101:0/1953819294 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad58004140 con 0x7fad680b5b80 2021-02-10T11:46:42.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.565 7fad6da8e700 1 -- 172.21.2.101:0/1953819294 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad680bcae0 con 0x7fad680b5b80 2021-02-10T11:46:42.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.565 7fad5ffff700 1 -- 172.21.2.101:0/1953819294 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fad580045a0 con 0x7fad680b5b80 2021-02-10T11:46:42.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.565 7fad6da8e700 1 -- 172.21.2.101:0/1953819294 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad680bd070 con 0x7fad680b5b80 2021-02-10T11:46:42.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.566 7fad5ffff700 1 -- 172.21.2.101:0/1953819294 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fad58004320 con 0x7fad680b5b80 2021-02-10T11:46:42.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.566 7fad5ffff700 1 --2- 172.21.2.101:0/1953819294 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fad4c0186c0 0x7fad4c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:42.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.566 7fad5ffff700 1 -- 172.21.2.101:0/1953819294 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fad580323a0 con 0x7fad680b5b80 2021-02-10T11:46:42.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.566 7fad677fe700 1 --2- 172.21.2.101:0/1953819294 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fad4c0186c0 0x7fad4c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:42.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.567 7fad677fe700 1 --2- 172.21.2.101:0/1953819294 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fad4c0186c0 0x7fad4c01abc0 crc :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:42.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.596 7fad6da8e700 1 -- 172.21.2.101:0/1953819294 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad680b6010 con 0x7fad680b5b80 2021-02-10T11:46:42.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.599 7fad5ffff700 1 -- 172.21.2.101:0/1953819294 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fad58000dd0 con 0x7fad680b5b80 2021-02-10T11:46:42.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.722 7fad6da8e700 1 -- 172.21.2.101:0/1953819294 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad680b6010 con 0x7fad680b5b80 2021-02-10T11:46:42.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.723 7fad5ffff700 1 -- 172.21.2.101:0/1953819294 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) v1 ==== 74+0+14007 (secure 0 0 0) 0x7fad58004320 con 0x7fad680b5b80 2021-02-10T11:46:42.721 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:46:42.721 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":175,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:46:36.152442","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":37,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":36,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:46:33.067171","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":37,"pool_name":"cephfs_data","create_time":"2021-02-10 11:46:34.142991","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:46:42.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.724 7fad6da8e700 1 -- 172.21.2.101:0/1953819294 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fad4c0186c0 msgr2=0x7fad4c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:42.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.724 7fad6da8e700 1 --2- 172.21.2.101:0/1953819294 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fad4c0186c0 0x7fad4c01abc0 crc :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:42.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.724 7fad6da8e700 1 -- 172.21.2.101:0/1953819294 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad680b5b80 msgr2=0x7fad681a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:42.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.724 7fad6da8e700 1 --2- 172.21.2.101:0/1953819294 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad680b5b80 0x7fad681a2e20 secure :-1 s=READY pgs=2996 cs=0 l=1 rev1=1 rx=0x7fad58001c60 tx=0x7fad5800a8d0).stop 2021-02-10T11:46:42.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.725 7fad6da8e700 1 -- 172.21.2.101:0/1953819294 shutdown_connections 2021-02-10T11:46:42.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.725 7fad6da8e700 1 --2- 172.21.2.101:0/1953819294 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fad680acfa0 0x7fad681a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:42.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.725 7fad6da8e700 1 --2- 172.21.2.101:0/1953819294 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fad680b5b80 0x7fad681a2e20 unknown :-1 s=CLOSED pgs=2996 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:42.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.725 7fad6da8e700 1 --2- 172.21.2.101:0/1953819294 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fad4c0186c0 0x7fad4c01abc0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:42.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.725 7fad6da8e700 1 --2- 172.21.2.101:0/1953819294 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fad68195650 0x7fad680b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:42.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.725 7fad6da8e700 1 -- 172.21.2.101:0/1953819294 >> 172.21.2.101:0/1953819294 conn(0x7fad680b3040 msgr2=0x7fad680c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:42.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.725 7fad6da8e700 1 -- 172.21.2.101:0/1953819294 shutdown_connections 2021-02-10T11:46:42.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:42.725 7fad6da8e700 1 -- 172.21.2.101:0/1953819294 wait complete. 2021-02-10T11:46:42.731 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 36, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:46:33.067171', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '174', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 37, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:46:34.142991', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '175', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:46:42.731 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:46:43.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.149 7f0f6a8df700 1 Processor -- start 2021-02-10T11:46:43.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.150 7f0f6a8df700 1 -- start start 2021-02-10T11:46:43.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.150 7f0f6a8df700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0f640d1870 0x7f0f641b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:43.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.150 7f0f6a8df700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f640fe1c0 con 0x7f0f641b0ae0 2021-02-10T11:46:43.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.150 7f0f6a8df700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f64109a60 con 0x7f0f640cff70 2021-02-10T11:46:43.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.150 7f0f6a8df700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f64088070 con 0x7f0f640d1870 2021-02-10T11:46:43.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.150 7f0f637fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f0f641b0ae0 0x7f0f64108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33972/0 (socket says 172.21.2.101:33972) 2021-02-10T11:46:43.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.150 7f0f637fe700 1 -- 172.21.2.101:0/3503646658 learned_addr learned my addr 172.21.2.101:0/3503646658 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:43.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.151 7f0f62ffd700 1 -- 172.21.2.101:0/3503646658 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1025408559 0 0) 0x7f0f580009b0 con 0x7f0f641b0ae0 2021-02-10T11:46:43.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.151 7f0f62ffd700 1 -- 172.21.2.101:0/3503646658 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f4c002fe0 con 0x7f0f641b0ae0 2021-02-10T11:46:43.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.151 7f0f63fff700 1 --2- 172.21.2.101:0/3503646658 >> v2:172.21.2.108:3300/0 conn(0x7f0f640d1870 0x7f0f641b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:43.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.151 7f0f62ffd700 1 -- 172.21.2.101:0/3503646658 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2843697835 0 0) 0x7f0f58000e20 con 0x7f0f641b0ae0 2021-02-10T11:46:43.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.151 7f0f62ffd700 1 -- 172.21.2.101:0/3503646658 >> v2:172.21.2.108:3300/0 conn(0x7f0f640d1870 msgr2=0x7f0f641b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:43.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.151 7f0f62ffd700 1 --2- 172.21.2.101:0/3503646658 >> v2:172.21.2.108:3300/0 conn(0x7f0f640d1870 0x7f0f641b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:43.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.151 7f0f62ffd700 1 -- 172.21.2.101:0/3503646658 >> v1:172.21.2.108:6789/0 conn(0x7f0f640cff70 legacy=0x7f0f64109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:43.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.151 7f0f62ffd700 1 -- 172.21.2.101:0/3503646658 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f64088260 con 0x7f0f641b0ae0 2021-02-10T11:46:43.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.151 7f0f62ffd700 1 -- 172.21.2.101:0/3503646658 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f0f58001530 con 0x7f0f641b0ae0 2021-02-10T11:46:43.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.151 7f0f62ffd700 1 -- 172.21.2.101:0/3503646658 >> v1:172.21.2.101:6789/0 conn(0x7f0f641b0ae0 legacy=0x7f0f64108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:43.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.151 7f0f62ffd700 1 --2- 172.21.2.101:0/3503646658 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f4c003af0 0x7f0f4c006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:43.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.151 7f0f62ffd700 1 --2- 172.21.2.101:0/3503646658 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f4c006820 0x7f0f4c008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:43.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.151 7f0f62ffd700 1 --2- 172.21.2.101:0/3503646658 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f4c009640 0x7f0f4c00bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:43.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.151 7f0f62ffd700 1 -- 172.21.2.101:0/3503646658 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c00c390 con 0x7f0f4c003af0 2021-02-10T11:46:43.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.151 7f0f62ffd700 1 -- 172.21.2.101:0/3503646658 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c00c5b0 con 0x7f0f4c006820 2021-02-10T11:46:43.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.151 7f0f62ffd700 1 -- 172.21.2.101:0/3503646658 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c00c7d0 con 0x7f0f4c009640 2021-02-10T11:46:43.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.151 7f0f63fff700 1 --2- 172.21.2.101:0/3503646658 >> v2:172.21.2.108:3300/0 conn(0x7f0f640d1870 0x7f0f641b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:46:43.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.151 7f0f63fff700 1 --2- 172.21.2.101:0/3503646658 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f4c003af0 0x7f0f4c006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:43.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.151 7f0f637fe700 1 --2- 172.21.2.101:0/3503646658 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f4c006820 0x7f0f4c008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:43.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.152 7f0f68efc700 1 --2- 172.21.2.101:0/3503646658 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f4c009640 0x7f0f4c00bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:43.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.152 7f0f63fff700 1 -- 172.21.2.101:0/3503646658 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f4c009640 msgr2=0x7f0f4c00bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:43.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.152 7f0f63fff700 1 --2- 172.21.2.101:0/3503646658 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f4c009640 0x7f0f4c00bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:43.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.152 7f0f63fff700 1 -- 172.21.2.101:0/3503646658 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f4c006820 msgr2=0x7f0f4c008d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:43.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.152 7f0f63fff700 1 --2- 172.21.2.101:0/3503646658 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f4c006820 0x7f0f4c008d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:43.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.152 7f0f63fff700 1 -- 172.21.2.101:0/3503646658 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4c00ca70 con 0x7f0f4c003af0 2021-02-10T11:46:43.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.152 7f0f68efc700 1 -- 172.21.2.101:0/3503646658 reap_dead start 2021-02-10T11:46:43.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.152 7f0f63fff700 1 --2- 172.21.2.101:0/3503646658 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f4c003af0 0x7f0f4c006010 secure :-1 s=READY pgs=2998 cs=0 l=1 rev1=1 rx=0x7f0f54006cf0 tx=0x7f0f5404dd20).ready entity=mon.0 client_cookie=d475c6f5aedecf88 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:43.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.152 7f0f62ffd700 1 -- 172.21.2.101:0/3503646658 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0f540065d0 con 0x7f0f4c003af0 2021-02-10T11:46:43.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.152 7f0f62ffd700 1 -- 172.21.2.101:0/3503646658 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f54006910 con 0x7f0f4c003af0 2021-02-10T11:46:43.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.152 7f0f62ffd700 1 -- 172.21.2.101:0/3503646658 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0f540020d0 con 0x7f0f4c003af0 2021-02-10T11:46:43.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.152 7f0f6a8df700 1 -- 172.21.2.101:0/3503646658 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f4c003af0 msgr2=0x7f0f4c006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:43.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.152 7f0f6a8df700 1 --2- 172.21.2.101:0/3503646658 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f4c003af0 0x7f0f4c006010 secure :-1 s=READY pgs=2998 cs=0 l=1 rev1=1 rx=0x7f0f54006cf0 tx=0x7f0f5404dd20).stop 2021-02-10T11:46:43.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.153 7f0f6a8df700 1 -- 172.21.2.101:0/3503646658 shutdown_connections 2021-02-10T11:46:43.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.153 7f0f6a8df700 1 --2- 172.21.2.101:0/3503646658 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f4c003af0 0x7f0f4c006010 unknown :-1 s=CLOSED pgs=2998 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:43.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.153 7f0f6a8df700 1 -- 172.21.2.101:0/3503646658 >> 172.21.2.101:0/3503646658 conn(0x7f0f640ce420 msgr2=0x7f0f640d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:43.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.153 7f0f6a8df700 1 -- 172.21.2.101:0/3503646658 shutdown_connections 2021-02-10T11:46:43.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.153 7f0f6a8df700 1 -- 172.21.2.101:0/3503646658 wait complete. 2021-02-10T11:46:43.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.168 7f0f6a8df700 1 Processor -- start 2021-02-10T11:46:43.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.168 7f0f6a8df700 1 -- start start 2021-02-10T11:46:43.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.168 7f0f6a8df700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f6409bcc0 0x7f0f6409c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:43.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.168 7f0f6a8df700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f640a0820 0x7f0f640a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:43.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.168 7f0f6a8df700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f640a1240 0x7f0f640858d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:43.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.168 7f0f6a8df700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f640a66b0 con 0x7f0f640a0820 2021-02-10T11:46:43.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.168 7f0f6a8df700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f641a49b0 con 0x7f0f6409bcc0 2021-02-10T11:46:43.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.168 7f0f6a8df700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f641a4b90 con 0x7f0f640a1240 2021-02-10T11:46:43.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.168 7f0f637fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f640a0820 0x7f0f640a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:43.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.168 7f0f637fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f640a0820 0x7f0f640a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33488/0 (socket says 172.21.2.101:33488) 2021-02-10T11:46:43.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.168 7f0f637fe700 1 -- 172.21.2.101:0/686310375 learned_addr learned my addr 172.21.2.101:0/686310375 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:43.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.169 7f0f637fe700 1 -- 172.21.2.101:0/686310375 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f640a1240 msgr2=0x7f0f640858d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:43.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.169 7f0f637fe700 1 --2- 172.21.2.101:0/686310375 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f640a1240 0x7f0f640858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:43.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.169 7f0f637fe700 1 -- 172.21.2.101:0/686310375 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f6409bcc0 msgr2=0x7f0f6409c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:43.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.169 7f0f637fe700 1 --2- 172.21.2.101:0/686310375 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f6409bcc0 0x7f0f6409c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:43.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.169 7f0f637fe700 1 -- 172.21.2.101:0/686310375 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f640a1640 con 0x7f0f640a0820 2021-02-10T11:46:43.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.169 7f0f637fe700 1 --2- 172.21.2.101:0/686310375 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f640a0820 0x7f0f640a0c20 secure :-1 s=READY pgs=2999 cs=0 l=1 rev1=1 rx=0x7f0f5800a590 tx=0x7f0f5800a930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:43.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.169 7f0f60ff9700 1 -- 172.21.2.101:0/686310375 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0f58001600 con 0x7f0f640a0820 2021-02-10T11:46:43.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.169 7f0f60ff9700 1 -- 172.21.2.101:0/686310375 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f580018c0 con 0x7f0f640a0820 2021-02-10T11:46:43.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.169 7f0f60ff9700 1 -- 172.21.2.101:0/686310375 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0f58004350 con 0x7f0f640a0820 2021-02-10T11:46:43.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.169 7f0f6a8df700 1 -- 172.21.2.101:0/686310375 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f640a1640 con 0x7f0f640a0820 2021-02-10T11:46:43.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.169 7f0f6a8df700 1 -- 172.21.2.101:0/686310375 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f640861e0 con 0x7f0f640a0820 2021-02-10T11:46:43.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.170 7f0f60ff9700 1 -- 172.21.2.101:0/686310375 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0f58004140 con 0x7f0f640a0820 2021-02-10T11:46:43.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.170 7f0f60ff9700 1 --2- 172.21.2.101:0/686310375 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0f440186c0 0x7f0f4401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:43.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.170 7f0f60ff9700 1 -- 172.21.2.101:0/686310375 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0f58032530 con 0x7f0f640a0820 2021-02-10T11:46:43.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.170 7f0f63fff700 1 --2- 172.21.2.101:0/686310375 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0f440186c0 0x7f0f4401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:43.172 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.170 7f0f63fff700 1 --2- 172.21.2.101:0/686310375 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0f440186c0 0x7f0f4401abc0 crc :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:43.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.200 7f0f6a8df700 1 -- 172.21.2.101:0/686310375 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f64086a70 con 0x7f0f640a0820 2021-02-10T11:46:43.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.202 7f0f60ff9700 1 -- 172.21.2.101:0/686310375 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0f58001600 con 0x7f0f640a0820 2021-02-10T11:46:43.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.330 7f0f6a8df700 1 -- 172.21.2.101:0/686310375 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f0f64086a70 con 0x7f0f640a0820 2021-02-10T11:46:43.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.331 7f0f60ff9700 1 -- 172.21.2.101:0/686310375 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v175) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0f5800ad20 con 0x7f0f640a0820 2021-02-10T11:46:43.329 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:46:43.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.333 7f0f6a8df700 1 -- 172.21.2.101:0/686310375 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0f440186c0 msgr2=0x7f0f4401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:43.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.333 7f0f6a8df700 1 --2- 172.21.2.101:0/686310375 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0f440186c0 0x7f0f4401abc0 crc :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:43.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.333 7f0f6a8df700 1 -- 172.21.2.101:0/686310375 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f640a0820 msgr2=0x7f0f640a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:43.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.333 7f0f6a8df700 1 --2- 172.21.2.101:0/686310375 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f640a0820 0x7f0f640a0c20 secure :-1 s=READY pgs=2999 cs=0 l=1 rev1=1 rx=0x7f0f5800a590 tx=0x7f0f5800a930).stop 2021-02-10T11:46:43.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.333 7f0f6a8df700 1 -- 172.21.2.101:0/686310375 shutdown_connections 2021-02-10T11:46:43.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.333 7f0f6a8df700 1 --2- 172.21.2.101:0/686310375 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0f440186c0 0x7f0f4401abc0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:43.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.333 7f0f6a8df700 1 --2- 172.21.2.101:0/686310375 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0f640a1240 0x7f0f640858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:43.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.333 7f0f6a8df700 1 --2- 172.21.2.101:0/686310375 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0f640a0820 0x7f0f640a0c20 unknown :-1 s=CLOSED pgs=2999 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:43.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.333 7f0f6a8df700 1 --2- 172.21.2.101:0/686310375 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0f6409bcc0 0x7f0f6409c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:43.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.333 7f0f6a8df700 1 -- 172.21.2.101:0/686310375 >> 172.21.2.101:0/686310375 conn(0x7f0f640bef00 msgr2=0x7f0f6409d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:43.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.333 7f0f6a8df700 1 -- 172.21.2.101:0/686310375 shutdown_connections 2021-02-10T11:46:43.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.333 7f0f6a8df700 1 -- 172.21.2.101:0/686310375 wait complete. 2021-02-10T11:46:43.339 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:46:43.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.758 7fd91cd51700 1 Processor -- start 2021-02-10T11:46:43.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.758 7fd91cd51700 1 -- start start 2021-02-10T11:46:43.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.758 7fd91cd51700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd91810aff0 0x7fd9180cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:43.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.758 7fd91cd51700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9180fe1c0 con 0x7fd9180d1c70 2021-02-10T11:46:43.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.758 7fd91cd51700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9181b0cb0 con 0x7fd9180d1870 2021-02-10T11:46:43.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.758 7fd91cd51700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd918109a60 con 0x7fd91810aff0 2021-02-10T11:46:43.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.758 7fd916312700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd9180d1c70 0x7fd9180cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:33990/0 (socket says 172.21.2.101:33990) 2021-02-10T11:46:43.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.758 7fd916312700 1 -- 172.21.2.101:0/2131195454 learned_addr learned my addr 172.21.2.101:0/2131195454 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:43.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.758 7fd917314700 1 --2- 172.21.2.101:0/2131195454 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd91810aff0 0x7fd9180cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:43.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.759 7fd915b11700 1 -- 172.21.2.101:0/2131195454 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767869893 0 0) 0x7fd9000010b0 con 0x7fd9180d1c70 2021-02-10T11:46:43.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.759 7fd915b11700 1 -- 172.21.2.101:0/2131195454 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd904002fe0 con 0x7fd9180d1c70 2021-02-10T11:46:43.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.759 7fd917314700 1 -- 172.21.2.101:0/2131195454 >> v1:172.21.2.108:6789/0 conn(0x7fd9180d1870 legacy=0x7fd9181b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:43.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.759 7fd917314700 1 -- 172.21.2.101:0/2131195454 >> v1:172.21.2.101:6789/0 conn(0x7fd9180d1c70 legacy=0x7fd9180cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:43.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.759 7fd917314700 1 -- 172.21.2.101:0/2131195454 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91810b3f0 con 0x7fd91810aff0 2021-02-10T11:46:43.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.759 7fd917314700 1 --2- 172.21.2.101:0/2131195454 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd91810aff0 0x7fd9180cf750 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fd910006e50 tx=0x7fd910009e70).ready entity=mon.2 client_cookie=b1c6b5c81278a75b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:43.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.759 7fd915b11700 1 -- 172.21.2.101:0/2131195454 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd91000b510 con 0x7fd91810aff0 2021-02-10T11:46:43.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.759 7fd915b11700 1 -- 172.21.2.101:0/2131195454 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd910006790 con 0x7fd91810aff0 2021-02-10T11:46:43.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.759 7fd915b11700 1 -- 172.21.2.101:0/2131195454 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd91000b940 con 0x7fd91810aff0 2021-02-10T11:46:43.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.759 7fd91cd51700 1 -- 172.21.2.101:0/2131195454 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd91810aff0 msgr2=0x7fd9180cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:43.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.759 7fd91cd51700 1 --2- 172.21.2.101:0/2131195454 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd91810aff0 0x7fd9180cf750 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fd910006e50 tx=0x7fd910009e70).stop 2021-02-10T11:46:43.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.760 7fd91cd51700 1 -- 172.21.2.101:0/2131195454 shutdown_connections 2021-02-10T11:46:43.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.760 7fd91cd51700 1 --2- 172.21.2.101:0/2131195454 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd91810aff0 0x7fd9180cf750 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:43.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.760 7fd91cd51700 1 -- 172.21.2.101:0/2131195454 >> 172.21.2.101:0/2131195454 conn(0x7fd9180ce420 msgr2=0x7fd9180d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:43.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.760 7fd91cd51700 1 -- 172.21.2.101:0/2131195454 shutdown_connections 2021-02-10T11:46:43.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.760 7fd91cd51700 1 -- 172.21.2.101:0/2131195454 wait complete. 2021-02-10T11:46:43.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.775 7fd91cd51700 1 Processor -- start 2021-02-10T11:46:43.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.775 7fd91cd51700 1 -- start start 2021-02-10T11:46:43.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.775 7fd91cd51700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9180c3820 0x7fd9180a7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:43.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.775 7fd91cd51700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd9181a44c0 0x7fd9181a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:43.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.775 7fd91cd51700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9180863d0 0x7fd9180867d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:43.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.775 7fd91cd51700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd918085ed0 con 0x7fd9180863d0 2021-02-10T11:46:43.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.775 7fd91cd51700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180c0740 con 0x7fd9181a44c0 2021-02-10T11:46:43.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.775 7fd91cd51700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd918086df0 con 0x7fd9180c3820 2021-02-10T11:46:43.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.775 7fd917314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9180863d0 0x7fd9180867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:43.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.775 7fd917314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9180863d0 0x7fd9180867d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33500/0 (socket says 172.21.2.101:33500) 2021-02-10T11:46:43.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.775 7fd917314700 1 -- 172.21.2.101:0/2939049499 learned_addr learned my addr 172.21.2.101:0/2939049499 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:43.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.775 7fd916312700 1 --2- 172.21.2.101:0/2939049499 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd9181a44c0 0x7fd9181a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:43.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.776 7fd917314700 1 -- 172.21.2.101:0/2939049499 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9180c3820 msgr2=0x7fd9180a7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:43.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.776 7fd916b13700 1 --2- 172.21.2.101:0/2939049499 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9180c3820 0x7fd9180a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:43.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.776 7fd917314700 1 --2- 172.21.2.101:0/2939049499 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9180c3820 0x7fd9180a7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:43.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.776 7fd917314700 1 -- 172.21.2.101:0/2939049499 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd9181a44c0 msgr2=0x7fd9181a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:43.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.776 7fd917314700 1 --2- 172.21.2.101:0/2939049499 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd9181a44c0 0x7fd9181a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:43.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.776 7fd917314700 1 -- 172.21.2.101:0/2939049499 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd918087010 con 0x7fd9180863d0 2021-02-10T11:46:43.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.776 7fd916b13700 1 --2- 172.21.2.101:0/2939049499 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9180c3820 0x7fd9180a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:46:43.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.776 7fd916312700 1 --2- 172.21.2.101:0/2939049499 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd9181a44c0 0x7fd9181a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:46:43.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.776 7fd917314700 1 --2- 172.21.2.101:0/2939049499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9180863d0 0x7fd9180867d0 secure :-1 s=READY pgs=3001 cs=0 l=1 rev1=1 rx=0x7fd91000b970 tx=0x7fd91000bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:43.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.776 7fd90f7fe700 1 -- 172.21.2.101:0/2939049499 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd9100093d0 con 0x7fd9180863d0 2021-02-10T11:46:43.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.776 7fd90f7fe700 1 -- 172.21.2.101:0/2939049499 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd910007240 con 0x7fd9180863d0 2021-02-10T11:46:43.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.776 7fd91cd51700 1 -- 172.21.2.101:0/2939049499 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd918087010 con 0x7fd9180863d0 2021-02-10T11:46:43.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.776 7fd91cd51700 1 -- 172.21.2.101:0/2939049499 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd918087010 con 0x7fd9180863d0 2021-02-10T11:46:43.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.776 7fd90f7fe700 1 -- 172.21.2.101:0/2939049499 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd910007720 con 0x7fd9180863d0 2021-02-10T11:46:43.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.777 7fd90f7fe700 1 -- 172.21.2.101:0/2939049499 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd910007420 con 0x7fd9180863d0 2021-02-10T11:46:43.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.777 7fd90f7fe700 1 --2- 172.21.2.101:0/2939049499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8fc018730 0x7fd8fc01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:43.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.777 7fd90f7fe700 1 -- 172.21.2.101:0/2939049499 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd910032b80 con 0x7fd9180863d0 2021-02-10T11:46:43.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.777 7fd916b13700 1 --2- 172.21.2.101:0/2939049499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8fc018730 0x7fd8fc01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:43.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.777 7fd916b13700 1 --2- 172.21.2.101:0/2939049499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8fc018730 0x7fd8fc01ac10 crc :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:43.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.808 7fd91cd51700 1 -- 172.21.2.101:0/2939049499 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9180fe1c0 con 0x7fd9180863d0 2021-02-10T11:46:43.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.811 7fd90f7fe700 1 -- 172.21.2.101:0/2939049499 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd910032d90 con 0x7fd9180863d0 2021-02-10T11:46:43.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.938 7fd91cd51700 1 -- 172.21.2.101:0/2939049499 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7fd9180fe1c0 con 0x7fd9180863d0 2021-02-10T11:46:43.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.939 7fd90f7fe700 1 -- 172.21.2.101:0/2939049499 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v175) v1 ==== 104+0+10 (secure 0 0 0) 0x7fd91001f880 con 0x7fd9180863d0 2021-02-10T11:46:43.936 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:46:43.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.941 7fd91cd51700 1 -- 172.21.2.101:0/2939049499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8fc018730 msgr2=0x7fd8fc01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:43.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.941 7fd91cd51700 1 --2- 172.21.2.101:0/2939049499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8fc018730 0x7fd8fc01ac10 crc :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:43.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.941 7fd91cd51700 1 -- 172.21.2.101:0/2939049499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9180863d0 msgr2=0x7fd9180867d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:43.938 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.941 7fd91cd51700 1 --2- 172.21.2.101:0/2939049499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9180863d0 0x7fd9180867d0 secure :-1 s=READY pgs=3001 cs=0 l=1 rev1=1 rx=0x7fd91000b970 tx=0x7fd91000bd10).stop 2021-02-10T11:46:43.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.941 7fd91cd51700 1 -- 172.21.2.101:0/2939049499 shutdown_connections 2021-02-10T11:46:43.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.941 7fd91cd51700 1 --2- 172.21.2.101:0/2939049499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd9180863d0 0x7fd9180867d0 unknown :-1 s=CLOSED pgs=3001 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:43.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.941 7fd91cd51700 1 --2- 172.21.2.101:0/2939049499 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd9181a44c0 0x7fd9181a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:43.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.941 7fd91cd51700 1 --2- 172.21.2.101:0/2939049499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd8fc018730 0x7fd8fc01ac10 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:43.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.941 7fd91cd51700 1 --2- 172.21.2.101:0/2939049499 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd9180c3820 0x7fd9180a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:43.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.941 7fd91cd51700 1 -- 172.21.2.101:0/2939049499 >> 172.21.2.101:0/2939049499 conn(0x7fd9180872c0 msgr2=0x7fd918095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:43.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.941 7fd91cd51700 1 -- 172.21.2.101:0/2939049499 shutdown_connections 2021-02-10T11:46:43.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:43.941 7fd91cd51700 1 -- 172.21.2.101:0/2939049499 wait complete. 2021-02-10T11:46:43.947 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:46:44.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.365 7f2b72aa3700 1 Processor -- start 2021-02-10T11:46:44.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.365 7f2b72aa3700 1 -- start start 2021-02-10T11:46:44.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.366 7f2b72aa3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b6c0d1870 0x7f2b6c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:44.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.366 7f2b72aa3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2b6c1b0ae0 0x7f2b6c108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:44.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.366 7f2b72aa3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2b6c0d1c70 0x7f2b6c108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:44.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.366 7f2b72aa3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b6c0e5ef0 con 0x7f2b6c1b0ae0 2021-02-10T11:46:44.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.366 7f2b72aa3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b6c109a60 con 0x7f2b6c0d1c70 2021-02-10T11:46:44.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.366 7f2b72aa3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c088070 con 0x7f2b6c0d1870 2021-02-10T11:46:44.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.366 7f2b6bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2b6c1b0ae0 0x7f2b6c108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:44.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.366 7f2b6bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2b6c1b0ae0 0x7f2b6c108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33506/0 (socket says 172.21.2.101:33506) 2021-02-10T11:46:44.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.366 7f2b6bfff700 1 -- 172.21.2.101:0/2794389848 learned_addr learned my addr 172.21.2.101:0/2794389848 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:44.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.366 7f2b6bfff700 1 -- 172.21.2.101:0/2794389848 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b6c0d1870 msgr2=0x7f2b6c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:44.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.366 7f2b6bfff700 1 --2- 172.21.2.101:0/2794389848 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b6c0d1870 0x7f2b6c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:44.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.366 7f2b6bfff700 1 -- 172.21.2.101:0/2794389848 >> v2:172.21.2.108:3300/0 conn(0x7f2b6c0d1c70 msgr2=0x7f2b6c108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:44.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.366 7f2b6bfff700 1 --2- 172.21.2.101:0/2794389848 >> v2:172.21.2.108:3300/0 conn(0x7f2b6c0d1c70 0x7f2b6c108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:44.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.366 7f2b6bfff700 1 -- 172.21.2.101:0/2794389848 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6c109580 con 0x7f2b6c1b0ae0 2021-02-10T11:46:44.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.366 7f2b6bfff700 1 --2- 172.21.2.101:0/2794389848 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b6c1b0ae0 0x7f2b6c108940 secure :-1 s=READY pgs=3002 cs=0 l=1 rev1=1 rx=0x7f2b5c006700 tx=0x7f2b5c009b80).ready entity=mon.0 client_cookie=418f3f2ac0bc4ba1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:44.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.367 7f2b6b7fe700 1 -- 172.21.2.101:0/2794389848 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b5c00c0c0 con 0x7f2b6c1b0ae0 2021-02-10T11:46:44.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.367 7f2b6b7fe700 1 -- 172.21.2.101:0/2794389848 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5c00c440 con 0x7f2b6c1b0ae0 2021-02-10T11:46:44.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.367 7f2b6b7fe700 1 -- 172.21.2.101:0/2794389848 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b5c00c920 con 0x7f2b6c1b0ae0 2021-02-10T11:46:44.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.367 7f2b72aa3700 1 -- 172.21.2.101:0/2794389848 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b6c1b0ae0 msgr2=0x7f2b6c108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:44.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.367 7f2b72aa3700 1 --2- 172.21.2.101:0/2794389848 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b6c1b0ae0 0x7f2b6c108940 secure :-1 s=READY pgs=3002 cs=0 l=1 rev1=1 rx=0x7f2b5c006700 tx=0x7f2b5c009b80).stop 2021-02-10T11:46:44.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.367 7f2b72aa3700 1 -- 172.21.2.101:0/2794389848 shutdown_connections 2021-02-10T11:46:44.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.367 7f2b72aa3700 1 --2- 172.21.2.101:0/2794389848 >> v2:172.21.2.108:3300/0 conn(0x7f2b6c0d1c70 0x7f2b6c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:44.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.367 7f2b72aa3700 1 --2- 172.21.2.101:0/2794389848 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b6c1b0ae0 0x7f2b6c108940 unknown :-1 s=CLOSED pgs=3002 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:44.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.367 7f2b72aa3700 1 --2- 172.21.2.101:0/2794389848 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b6c0d1870 0x7f2b6c1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:44.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.367 7f2b72aa3700 1 -- 172.21.2.101:0/2794389848 >> 172.21.2.101:0/2794389848 conn(0x7f2b6c0ce420 msgr2=0x7f2b6c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:44.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.367 7f2b72aa3700 1 -- 172.21.2.101:0/2794389848 shutdown_connections 2021-02-10T11:46:44.381 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.367 7f2b72aa3700 1 -- 172.21.2.101:0/2794389848 wait complete. 2021-02-10T11:46:44.382 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.382 7f2b72aa3700 1 Processor -- start 2021-02-10T11:46:44.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.382 7f2b72aa3700 1 -- start start 2021-02-10T11:46:44.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.382 7f2b72aa3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b6c0872c0 0x7f2b6c1a7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:44.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.382 7f2b72aa3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b6c1a7ba0 0x7f2b6c0a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:44.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.382 7f2b72aa3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b6c1a7fa0 0x7f2b6c0a6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:44.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.382 7f2b72aa3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c086e20 con 0x7f2b6c0872c0 2021-02-10T11:46:44.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.382 7f2b72aa3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c190c60 con 0x7f2b6c1a7fa0 2021-02-10T11:46:44.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.382 7f2b72aa3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c0a74a0 con 0x7f2b6c1a7ba0 2021-02-10T11:46:44.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.383 7f2b708bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b6c0872c0 0x7f2b6c1a7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:44.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.383 7f2b708bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b6c0872c0 0x7f2b6c1a7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33514/0 (socket says 172.21.2.101:33514) 2021-02-10T11:46:44.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.383 7f2b708bf700 1 -- 172.21.2.101:0/2988711726 learned_addr learned my addr 172.21.2.101:0/2988711726 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:44.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.383 7f2b708bf700 1 -- 172.21.2.101:0/2988711726 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b6c1a7ba0 msgr2=0x7f2b6c0a6860 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:44.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.383 7f2b708bf700 1 --2- 172.21.2.101:0/2988711726 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b6c1a7ba0 0x7f2b6c0a6860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:44.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.383 7f2b708bf700 1 -- 172.21.2.101:0/2988711726 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b6c1a7fa0 msgr2=0x7f2b6c0a6e80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:44.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.383 7f2b708bf700 1 --2- 172.21.2.101:0/2988711726 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b6c1a7fa0 0x7f2b6c0a6e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:44.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.383 7f2b708bf700 1 -- 172.21.2.101:0/2988711726 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6c0a76c0 con 0x7f2b6c0872c0 2021-02-10T11:46:44.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.383 7f2b708bf700 1 --2- 172.21.2.101:0/2988711726 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b6c0872c0 0x7f2b6c1a7580 secure :-1 s=READY pgs=3003 cs=0 l=1 rev1=1 rx=0x7f2b54006f00 tx=0x7f2b5400b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:44.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.383 7f2b697fa700 1 -- 172.21.2.101:0/2988711726 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b54006350 con 0x7f2b6c0872c0 2021-02-10T11:46:44.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.383 7f2b697fa700 1 -- 172.21.2.101:0/2988711726 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b54006b90 con 0x7f2b6c0872c0 2021-02-10T11:46:44.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.383 7f2b72aa3700 1 -- 172.21.2.101:0/2988711726 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b6c0a76c0 con 0x7f2b6c0872c0 2021-02-10T11:46:44.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.383 7f2b72aa3700 1 -- 172.21.2.101:0/2988711726 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b6c0a76c0 con 0x7f2b6c0872c0 2021-02-10T11:46:44.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.383 7f2b697fa700 1 -- 172.21.2.101:0/2988711726 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b54000c90 con 0x7f2b6c0872c0 2021-02-10T11:46:44.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.384 7f2b697fa700 1 -- 172.21.2.101:0/2988711726 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2b54002240 con 0x7f2b6c0872c0 2021-02-10T11:46:44.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.384 7f2b697fa700 1 --2- 172.21.2.101:0/2988711726 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b500186d0 0x7f2b5001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:44.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.384 7f2b697fa700 1 -- 172.21.2.101:0/2988711726 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2b54032600 con 0x7f2b6c0872c0 2021-02-10T11:46:44.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.384 7f2b6bfff700 1 --2- 172.21.2.101:0/2988711726 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b500186d0 0x7f2b5001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:44.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.384 7f2b6bfff700 1 --2- 172.21.2.101:0/2988711726 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b500186d0 0x7f2b5001abd0 crc :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:44.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.415 7f2b72aa3700 1 -- 172.21.2.101:0/2988711726 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b6c086e20 con 0x7f2b6c0872c0 2021-02-10T11:46:44.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.418 7f2b697fa700 1 -- 172.21.2.101:0/2988711726 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2b54002030 con 0x7f2b6c0872c0 2021-02-10T11:46:44.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.545 7f2b72aa3700 1 -- 172.21.2.101:0/2988711726 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f2b6c086e20 con 0x7f2b6c0872c0 2021-02-10T11:46:44.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.545 7f2b697fa700 1 -- 172.21.2.101:0/2988711726 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v175) v1 ==== 100+0+10 (secure 0 0 0) 0x7f2b54006b90 con 0x7f2b6c0872c0 2021-02-10T11:46:44.544 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:46:44.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.547 7f2b72aa3700 1 -- 172.21.2.101:0/2988711726 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b500186d0 msgr2=0x7f2b5001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:44.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.547 7f2b72aa3700 1 --2- 172.21.2.101:0/2988711726 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b500186d0 0x7f2b5001abd0 crc :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:44.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.547 7f2b72aa3700 1 -- 172.21.2.101:0/2988711726 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b6c0872c0 msgr2=0x7f2b6c1a7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:44.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.547 7f2b72aa3700 1 --2- 172.21.2.101:0/2988711726 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b6c0872c0 0x7f2b6c1a7580 secure :-1 s=READY pgs=3003 cs=0 l=1 rev1=1 rx=0x7f2b54006f00 tx=0x7f2b5400b940).stop 2021-02-10T11:46:44.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.547 7f2b72aa3700 1 -- 172.21.2.101:0/2988711726 shutdown_connections 2021-02-10T11:46:44.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.547 7f2b72aa3700 1 --2- 172.21.2.101:0/2988711726 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2b500186d0 0x7f2b5001abd0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:44.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.547 7f2b72aa3700 1 --2- 172.21.2.101:0/2988711726 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2b6c1a7ba0 0x7f2b6c0a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:44.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.547 7f2b72aa3700 1 --2- 172.21.2.101:0/2988711726 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2b6c1a7fa0 0x7f2b6c0a6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:44.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.547 7f2b72aa3700 1 --2- 172.21.2.101:0/2988711726 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2b6c0872c0 0x7f2b6c1a7580 unknown :-1 s=CLOSED pgs=3003 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:44.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.547 7f2b72aa3700 1 -- 172.21.2.101:0/2988711726 >> 172.21.2.101:0/2988711726 conn(0x7f2b6c0be980 msgr2=0x7f2b6c092cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:44.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.548 7f2b72aa3700 1 -- 172.21.2.101:0/2988711726 shutdown_connections 2021-02-10T11:46:44.547 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.548 7f2b72aa3700 1 -- 172.21.2.101:0/2988711726 wait complete. 2021-02-10T11:46:44.554 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:46:44.554 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:46:44.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.968 7fb4bd2b0700 1 Processor -- start 2021-02-10T11:46:44.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.968 7fb4bd2b0700 1 -- start start 2021-02-10T11:46:44.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.968 7fb4bd2b0700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb4b80d1870 0x7fb4b81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:44.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.968 7fb4bd2b0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb4b81b0a10 0x7fb4b80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:44.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.968 7fb4bd2b0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4b80d1c70 0x7fb4b80cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:44.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.968 7fb4bd2b0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b80e5ed0 con 0x7fb4b81b0a10 2021-02-10T11:46:44.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.968 7fb4bd2b0700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b80f0b10 con 0x7fb4b80d1870 2021-02-10T11:46:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.968 7fb4bd2b0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b80cff70 con 0x7fb4b80d1c70 2021-02-10T11:46:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.969 7fb4b687c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb4b81b0a10 0x7fb4b80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.969 7fb4b687c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb4b81b0a10 0x7fb4b80cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33520/0 (socket says 172.21.2.101:33520) 2021-02-10T11:46:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.969 7fb4b687c700 1 -- 172.21.2.101:0/1770429596 learned_addr learned my addr 172.21.2.101:0/1770429596 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.969 7fb4b707d700 1 --2- 172.21.2.101:0/1770429596 >> v2:172.21.2.108:3300/0 conn(0x7fb4b80d1870 0x7fb4b81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.969 7fb4b787e700 1 --2- 172.21.2.101:0/1770429596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4b80d1c70 0x7fb4b80cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.969 7fb4b687c700 1 -- 172.21.2.101:0/1770429596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4b80d1c70 msgr2=0x7fb4b80cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.969 7fb4b687c700 1 --2- 172.21.2.101:0/1770429596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4b80d1c70 0x7fb4b80cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.969 7fb4b687c700 1 -- 172.21.2.101:0/1770429596 >> v2:172.21.2.108:3300/0 conn(0x7fb4b80d1870 msgr2=0x7fb4b81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.969 7fb4b687c700 1 --2- 172.21.2.101:0/1770429596 >> v2:172.21.2.108:3300/0 conn(0x7fb4b80d1870 0x7fb4b81b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:44.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.969 7fb4b687c700 1 -- 172.21.2.101:0/1770429596 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b80d0150 con 0x7fb4b81b0a10 2021-02-10T11:46:44.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.969 7fb4b787e700 1 --2- 172.21.2.101:0/1770429596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4b80d1c70 0x7fb4b80cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:46:44.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.969 7fb4b687c700 1 --2- 172.21.2.101:0/1770429596 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4b81b0a10 0x7fb4b80cf330 secure :-1 s=READY pgs=3004 cs=0 l=1 rev1=1 rx=0x7fb4a0006700 tx=0x7fb4a0009b80).ready entity=mon.0 client_cookie=88790d9564f3420 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:44.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.970 7fb4b607b700 1 -- 172.21.2.101:0/1770429596 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb4a000c0c0 con 0x7fb4b81b0a10 2021-02-10T11:46:44.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.970 7fb4b607b700 1 -- 172.21.2.101:0/1770429596 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a000c340 con 0x7fb4b81b0a10 2021-02-10T11:46:44.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.970 7fb4b607b700 1 -- 172.21.2.101:0/1770429596 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb4a000c860 con 0x7fb4b81b0a10 2021-02-10T11:46:44.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.970 7fb4bd2b0700 1 -- 172.21.2.101:0/1770429596 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4b81b0a10 msgr2=0x7fb4b80cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:44.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.970 7fb4bd2b0700 1 --2- 172.21.2.101:0/1770429596 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4b81b0a10 0x7fb4b80cf330 secure :-1 s=READY pgs=3004 cs=0 l=1 rev1=1 rx=0x7fb4a0006700 tx=0x7fb4a0009b80).stop 2021-02-10T11:46:44.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.970 7fb4bd2b0700 1 -- 172.21.2.101:0/1770429596 shutdown_connections 2021-02-10T11:46:44.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.970 7fb4bd2b0700 1 --2- 172.21.2.101:0/1770429596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4b80d1c70 0x7fb4b80cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:44.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.970 7fb4bd2b0700 1 --2- 172.21.2.101:0/1770429596 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4b81b0a10 0x7fb4b80cf330 unknown :-1 s=CLOSED pgs=3004 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:44.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.970 7fb4bd2b0700 1 --2- 172.21.2.101:0/1770429596 >> v2:172.21.2.108:3300/0 conn(0x7fb4b80d1870 0x7fb4b81b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:44.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.970 7fb4bd2b0700 1 -- 172.21.2.101:0/1770429596 >> 172.21.2.101:0/1770429596 conn(0x7fb4b80ce450 msgr2=0x7fb4b80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:44.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.970 7fb4bd2b0700 1 -- 172.21.2.101:0/1770429596 shutdown_connections 2021-02-10T11:46:44.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.970 7fb4bd2b0700 1 -- 172.21.2.101:0/1770429596 wait complete. 2021-02-10T11:46:44.985 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.985 7fb4bd2b0700 1 Processor -- start 2021-02-10T11:46:44.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.985 7fb4bd2b0700 1 -- start start 2021-02-10T11:46:44.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.985 7fb4bd2b0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4b8195650 0x7fb4b80b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:44.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.985 7fb4bd2b0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4b80b5b80 0x7fb4b81a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:44.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.985 7fb4bd2b0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4b80acfa0 0x7fb4b81a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:44.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.985 7fb4bd2b0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b80b6010 con 0x7fb4b80acfa0 2021-02-10T11:46:44.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.985 7fb4bd2b0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b81a3a60 con 0x7fb4b8195650 2021-02-10T11:46:44.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.985 7fb4bd2b0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b81a3c40 con 0x7fb4b80b5b80 2021-02-10T11:46:44.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.986 7fb4b787e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4b80acfa0 0x7fb4b81a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:44.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.986 7fb4b787e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4b80acfa0 0x7fb4b81a3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33526/0 (socket says 172.21.2.101:33526) 2021-02-10T11:46:44.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.986 7fb4b787e700 1 -- 172.21.2.101:0/1617375785 learned_addr learned my addr 172.21.2.101:0/1617375785 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:44.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.986 7fb4b787e700 1 -- 172.21.2.101:0/1617375785 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4b80b5b80 msgr2=0x7fb4b81a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:44.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.986 7fb4b787e700 1 --2- 172.21.2.101:0/1617375785 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4b80b5b80 0x7fb4b81a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:44.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.986 7fb4b787e700 1 -- 172.21.2.101:0/1617375785 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4b8195650 msgr2=0x7fb4b80b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:44.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.986 7fb4b787e700 1 --2- 172.21.2.101:0/1617375785 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4b8195650 0x7fb4b80b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:44.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.986 7fb4b787e700 1 -- 172.21.2.101:0/1617375785 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b80bcae0 con 0x7fb4b80acfa0 2021-02-10T11:46:44.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.986 7fb4b787e700 1 --2- 172.21.2.101:0/1617375785 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4b80acfa0 0x7fb4b81a3440 secure :-1 s=READY pgs=3005 cs=0 l=1 rev1=1 rx=0x7fb4b0009980 tx=0x7fb4b000bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:44.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.986 7fb4affff700 1 -- 172.21.2.101:0/1617375785 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb4b000c2f0 con 0x7fb4b80acfa0 2021-02-10T11:46:44.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.986 7fb4affff700 1 -- 172.21.2.101:0/1617375785 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b0006790 con 0x7fb4b80acfa0 2021-02-10T11:46:44.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.986 7fb4affff700 1 -- 172.21.2.101:0/1617375785 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb4b000c7f0 con 0x7fb4b80acfa0 2021-02-10T11:46:44.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.986 7fb4bd2b0700 1 -- 172.21.2.101:0/1617375785 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b80bcae0 con 0x7fb4b80acfa0 2021-02-10T11:46:44.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.987 7fb4bd2b0700 1 -- 172.21.2.101:0/1617375785 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b80bcae0 con 0x7fb4b80acfa0 2021-02-10T11:46:44.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.987 7fb4affff700 1 -- 172.21.2.101:0/1617375785 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb4b00011f0 con 0x7fb4b80acfa0 2021-02-10T11:46:44.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.987 7fb4affff700 1 --2- 172.21.2.101:0/1617375785 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb49c0186c0 0x7fb49c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:44.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.987 7fb4affff700 1 -- 172.21.2.101:0/1617375785 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb4b00328b0 con 0x7fb4b80acfa0 2021-02-10T11:46:44.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.987 7fb4b707d700 1 --2- 172.21.2.101:0/1617375785 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb49c0186c0 0x7fb49c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:44.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:44.988 7fb4b707d700 1 --2- 172.21.2.101:0/1617375785 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb49c0186c0 0x7fb49c01abc0 crc :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:45.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:45.018 7fb4bd2b0700 1 -- 172.21.2.101:0/1617375785 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b80b6010 con 0x7fb4b80acfa0 2021-02-10T11:46:45.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:45.021 7fb4affff700 1 -- 172.21.2.101:0/1617375785 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb4b0020fe0 con 0x7fb4b80acfa0 2021-02-10T11:46:45.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:45.150 7fb4bd2b0700 1 -- 172.21.2.101:0/1617375785 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4b80b6010 con 0x7fb4b80acfa0 2021-02-10T11:46:45.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:45.151 7fb4affff700 1 -- 172.21.2.101:0/1617375785 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+2111 (secure 0 0 0) 0x7fb4b0020fe0 con 0x7fb4b80acfa0 2021-02-10T11:46:45.148 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 148 2021-02-10T11:46:45.149 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:46:45.149 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":148,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":12780,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2200088074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2200088074},{"type":"v1","addr":"172.21.2.101:6835","nonce":2200088074}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":144}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:46:35.149920","modified":"2021-02-10 11:46:37.158088","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12777},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12777":{"gid":12777,"name":"b","rank":0,"incarnation":145,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/1343655613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":1343655613},{"type":"v1","addr":"172.21.2.108:6833","nonce":1343655613}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[37],"metadata_pool":36,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":18}]} 2021-02-10T11:46:45.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:45.152 7fb4bd2b0700 1 -- 172.21.2.101:0/1617375785 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb49c0186c0 msgr2=0x7fb49c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:45.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:45.152 7fb4bd2b0700 1 --2- 172.21.2.101:0/1617375785 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb49c0186c0 0x7fb49c01abc0 crc :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:45.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:45.152 7fb4bd2b0700 1 -- 172.21.2.101:0/1617375785 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4b80acfa0 msgr2=0x7fb4b81a3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:45.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:45.152 7fb4bd2b0700 1 --2- 172.21.2.101:0/1617375785 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4b80acfa0 0x7fb4b81a3440 secure :-1 s=READY pgs=3005 cs=0 l=1 rev1=1 rx=0x7fb4b0009980 tx=0x7fb4b000bcd0).stop 2021-02-10T11:46:45.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:45.153 7fb4bd2b0700 1 -- 172.21.2.101:0/1617375785 shutdown_connections 2021-02-10T11:46:45.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:45.153 7fb4bd2b0700 1 --2- 172.21.2.101:0/1617375785 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb49c0186c0 0x7fb49c01abc0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:45.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:45.153 7fb4bd2b0700 1 --2- 172.21.2.101:0/1617375785 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb4b80b5b80 0x7fb4b81a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:45.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:45.153 7fb4bd2b0700 1 --2- 172.21.2.101:0/1617375785 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb4b80acfa0 0x7fb4b81a3440 unknown :-1 s=CLOSED pgs=3005 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:45.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:45.153 7fb4bd2b0700 1 --2- 172.21.2.101:0/1617375785 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb4b8195650 0x7fb4b80b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:45.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:45.153 7fb4bd2b0700 1 -- 172.21.2.101:0/1617375785 >> 172.21.2.101:0/1617375785 conn(0x7fb4b80b3040 msgr2=0x7fb4b80c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:45.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:45.153 7fb4bd2b0700 1 -- 172.21.2.101:0/1617375785 shutdown_connections 2021-02-10T11:46:45.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:45.153 7fb4bd2b0700 1 -- 172.21.2.101:0/1617375785 wait complete. 2021-02-10T11:46:45.361 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 148, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:46:35.149920', 'modified': '2021-02-10 11:46:37.158088', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12777}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12777': {'gid': 12777, 'name': 'b', 'rank': 0, 'incarnation': 145, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/1343655613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 1343655613}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 1343655613}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [37], 'metadata_pool': 36, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:46:45.361 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:46:45.361 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:46:45.762 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:46:45.762 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:46:45.762 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:46:45.762 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 12777, 2021-02-10T11:46:45.762 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:46:45.762 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:46:45.762 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 10.611036302, 2021-02-10T11:46:45.763 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 148, 2021-02-10T11:46:45.763 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 175, 2021-02-10T11:46:45.763 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 175, 2021-02-10T11:46:45.763 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 16.256240017 2021-02-10T11:46:45.763 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:46:45.764 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 12777, "want_state": "up:active", "state": "up:active", "rank_uptime": 10.611036302, "mdsmap_epoch": 148, "osdmap_epoch": 175, "osdmap_epoch_barrier": 175, "uptime": 16.256240017 } 2021-02-10T11:46:45.764 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:46:45.764 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-02-10T11:46:45.765 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-02-10T11:46:45.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:45.765 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:46:46.106 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:46:46.456 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:46:46.456 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:46:46.800 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:46:46.800 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:46:46.801 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:46:46.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:46.802 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:46:47.137 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-02-10T11:46:47.137 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:46:47.140 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:46:47.484 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:46:47.484 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:46:47.551 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[173502]: starting ceph client 2021-02-10T11:46:47.551 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:2021-02-10 11:46:47.553 7f738048cf80 -1 init, newargv = 0x557efe8ff1a0 newargc=9 2021-02-10T11:46:47.556 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[173502]: starting fuse 2021-02-10T11:46:47.827 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:46:47.827 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:46:47.828 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:46:47.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:47.828 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:46:48.163 INFO:teuthology.orchestra.run.gibba001.stdout:42 2021-02-10T11:46:48.163 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2021-02-10T11:46:48.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:48.164 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:46:48.164 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:46:48.164 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:46:48.164 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:46:48.164 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:46:48.164 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:46:48.164 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:46:48.164 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-02-10T11:46:48.164 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:46:48.164 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:46:48.164 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:46:48.164 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:46:48.164 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:46:48.164 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:46:48.164 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:46:48.165 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:46:48.165 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:46:48.165 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:46:48.165 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:46:48.165 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:46:48.165 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:46:48.165 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.0")) 2021-02-10T11:46:48.165 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:46:48.527 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.0.173502.asok 2021-02-10T11:46:48.529 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.173502.asok 2021-02-10T11:46:48.529 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:48.529 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.173502.asok status 2021-02-10T11:46:48.919 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:46:48.920 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:46:48.920 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:46:48.920 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:46:48.920 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "0", 2021-02-10T11:46:48.920 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:46:48.920 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-02-10T11:46:48.920 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "173502", 2021-02-10T11:46:48.920 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:46:48.920 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:48.920 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:46:48.920 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:46:48.920 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 12933, 2021-02-10T11:46:48.920 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:46:48.921 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:46:48.921 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:46:48.921 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 12933 2021-02-10T11:46:48.921 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:48.921 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:46:48.921 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:48.921 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:46:48.921 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1617866811 2021-02-10T11:46:48.921 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:48.921 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:48.921 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:46:48.921 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:48.921 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:46:48.922 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1617866811 2021-02-10T11:46:48.922 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:48.922 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.12933 172.21.2.101:0/1617866811", 2021-02-10T11:46:48.922 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/1617866811", 2021-02-10T11:46:48.922 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:46:48.922 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 148, 2021-02-10T11:46:48.922 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 175, 2021-02-10T11:46:48.922 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:46:48.922 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:46:48.922 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:46:48.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:48.924 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:46:48.924 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:46:49.260 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:46:49.260 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:46:49.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:49.260 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:46:49.607 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:46:50.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.023 7fd782360700 1 Processor -- start 2021-02-10T11:46:50.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.023 7fd782360700 1 -- start start 2021-02-10T11:46:50.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.024 7fd782360700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd77c0d0340 0x7fd77c0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:50.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.024 7fd782360700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd77c0d1c70 0x7fd77c0efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.024 7fd782360700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd77c0f0b10 con 0x7fd77c0d1870 2021-02-10T11:46:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.024 7fd782360700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd77c0e5ed0 con 0x7fd77c0d1c70 2021-02-10T11:46:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.024 7fd782360700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd77c1b0be0 con 0x7fd77c0d0340 2021-02-10T11:46:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.024 7fd78097d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd77c0d1c70 0x7fd77c0efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.024 7fd78097d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd77c0d1c70 0x7fd77c0efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33556/0 (socket says 172.21.2.101:33556) 2021-02-10T11:46:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.024 7fd78097d700 1 -- 172.21.2.101:0/3470569936 learned_addr learned my addr 172.21.2.101:0/3470569936 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.024 7fd78097d700 1 -- 172.21.2.101:0/3470569936 >> v2:172.21.2.108:3300/0 conn(0x7fd77c0d0340 msgr2=0x7fd77c0ef640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:46:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.024 7fd78097d700 1 --2- 172.21.2.101:0/3470569936 >> v2:172.21.2.108:3300/0 conn(0x7fd77c0d0340 0x7fd77c0ef640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.024 7fd78097d700 1 -- 172.21.2.101:0/3470569936 >> v1:172.21.2.108:6789/0 conn(0x7fd77c0d1870 legacy=0x7fd77c1b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:46:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.024 7fd78097d700 1 -- 172.21.2.101:0/3470569936 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c0f20a0 con 0x7fd77c0d1c70 2021-02-10T11:46:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.024 7fd78097d700 1 --2- 172.21.2.101:0/3470569936 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd77c0d1c70 0x7fd77c0efc60 secure :-1 s=READY pgs=3009 cs=0 l=1 rev1=1 rx=0x7fd774006f00 tx=0x7fd774009f10).ready entity=mon.0 client_cookie=b80788388ce513ca server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.024 7fd77affd700 1 -- 172.21.2.101:0/3470569936 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd77400b3b0 con 0x7fd77c0d1c70 2021-02-10T11:46:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.024 7fd77affd700 1 -- 172.21.2.101:0/3470569936 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77400b630 con 0x7fd77c0d1c70 2021-02-10T11:46:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.024 7fd77affd700 1 -- 172.21.2.101:0/3470569936 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd77400bb50 con 0x7fd77c0d1c70 2021-02-10T11:46:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.024 7fd782360700 1 -- 172.21.2.101:0/3470569936 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd77c0d1c70 msgr2=0x7fd77c0efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.024 7fd782360700 1 --2- 172.21.2.101:0/3470569936 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd77c0d1c70 0x7fd77c0efc60 secure :-1 s=READY pgs=3009 cs=0 l=1 rev1=1 rx=0x7fd774006f00 tx=0x7fd774009f10).stop 2021-02-10T11:46:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.025 7fd782360700 1 -- 172.21.2.101:0/3470569936 shutdown_connections 2021-02-10T11:46:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.025 7fd782360700 1 --2- 172.21.2.101:0/3470569936 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd77c0d1c70 0x7fd77c0efc60 unknown :-1 s=CLOSED pgs=3009 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.025 7fd782360700 1 --2- 172.21.2.101:0/3470569936 >> v2:172.21.2.108:3300/0 conn(0x7fd77c0d0340 0x7fd77c0ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.025 7fd782360700 1 -- 172.21.2.101:0/3470569936 >> 172.21.2.101:0/3470569936 conn(0x7fd77c0ce450 msgr2=0x7fd77c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.025 7fd782360700 1 -- 172.21.2.101:0/3470569936 shutdown_connections 2021-02-10T11:46:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.025 7fd782360700 1 -- 172.21.2.101:0/3470569936 wait complete. 2021-02-10T11:46:50.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.039 7fd782360700 1 Processor -- start 2021-02-10T11:46:50.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.039 7fd782360700 1 -- start start 2021-02-10T11:46:50.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.040 7fd782360700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd77c18f0a0 0x7fd77c196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:50.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.040 7fd782360700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd77c1970d0 0x7fd77c1974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:50.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.040 7fd782360700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd77c094650 0x7fd77c094a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:50.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.040 7fd782360700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c095bd0 con 0x7fd77c1970d0 2021-02-10T11:46:50.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.040 7fd782360700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c0ad7c0 con 0x7fd77c18f0a0 2021-02-10T11:46:50.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.040 7fd782360700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c18ec70 con 0x7fd77c094650 2021-02-10T11:46:50.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.040 7fd77b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd77c1970d0 0x7fd77c1974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:50.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.040 7fd77b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd77c1970d0 0x7fd77c1974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33558/0 (socket says 172.21.2.101:33558) 2021-02-10T11:46:50.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.040 7fd77b7fe700 1 -- 172.21.2.101:0/1389063871 learned_addr learned my addr 172.21.2.101:0/1389063871 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:50.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.040 7fd77b7fe700 1 -- 172.21.2.101:0/1389063871 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd77c094650 msgr2=0x7fd77c094a50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:50.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.040 7fd77b7fe700 1 --2- 172.21.2.101:0/1389063871 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd77c094650 0x7fd77c094a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:50.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.040 7fd77b7fe700 1 -- 172.21.2.101:0/1389063871 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd77c18f0a0 msgr2=0x7fd77c196ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:50.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.040 7fd77b7fe700 1 --2- 172.21.2.101:0/1389063871 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd77c18f0a0 0x7fd77c196ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:50.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.040 7fd77b7fe700 1 -- 172.21.2.101:0/1389063871 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c095070 con 0x7fd77c1970d0 2021-02-10T11:46:50.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.040 7fd77b7fe700 1 --2- 172.21.2.101:0/1389063871 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd77c1970d0 0x7fd77c1974d0 secure :-1 s=READY pgs=3010 cs=0 l=1 rev1=1 rx=0x7fd77000b5c0 tx=0x7fd77000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:50.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.041 7fd778ff9700 1 -- 172.21.2.101:0/1389063871 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd770006b80 con 0x7fd77c1970d0 2021-02-10T11:46:50.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.041 7fd778ff9700 1 -- 172.21.2.101:0/1389063871 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77000bd50 con 0x7fd77c1970d0 2021-02-10T11:46:50.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.041 7fd778ff9700 1 -- 172.21.2.101:0/1389063871 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd7700054d0 con 0x7fd77c1970d0 2021-02-10T11:46:50.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.041 7fd782360700 1 -- 172.21.2.101:0/1389063871 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd77c095070 con 0x7fd77c1970d0 2021-02-10T11:46:50.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.041 7fd782360700 1 -- 172.21.2.101:0/1389063871 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd77c095600 con 0x7fd77c1970d0 2021-02-10T11:46:50.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.041 7fd778ff9700 1 -- 172.21.2.101:0/1389063871 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd770006030 con 0x7fd77c1970d0 2021-02-10T11:46:50.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.042 7fd778ff9700 1 --2- 172.21.2.101:0/1389063871 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd75c0186c0 0x7fd75c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:50.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.042 7fd778ff9700 1 -- 172.21.2.101:0/1389063871 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd770032590 con 0x7fd77c1970d0 2021-02-10T11:46:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.042 7fd77bfff700 1 --2- 172.21.2.101:0/1389063871 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd75c0186c0 0x7fd75c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.042 7fd77bfff700 1 --2- 172.21.2.101:0/1389063871 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd75c0186c0 0x7fd75c01abc0 crc :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:50.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.073 7fd782360700 1 -- 172.21.2.101:0/1389063871 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd77c0ad790 con 0x7fd77c1970d0 2021-02-10T11:46:50.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.076 7fd778ff9700 1 -- 172.21.2.101:0/1389063871 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd770025fc0 con 0x7fd77c1970d0 2021-02-10T11:46:50.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.200 7fd782360700 1 -- 172.21.2.101:0/1389063871 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd77c0ad790 con 0x7fd77c1970d0 2021-02-10T11:46:50.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.201 7fd778ff9700 1 -- 172.21.2.101:0/1389063871 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) v1 ==== 74+0+14007 (secure 0 0 0) 0x7fd770025fc0 con 0x7fd77c1970d0 2021-02-10T11:46:50.198 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:46:50.198 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":175,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:46:36.152442","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":37,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":36,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:46:33.067171","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":37,"pool_name":"cephfs_data","create_time":"2021-02-10 11:46:34.142991","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:46:50.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.202 7fd782360700 1 -- 172.21.2.101:0/1389063871 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd75c0186c0 msgr2=0x7fd75c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:50.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.202 7fd782360700 1 --2- 172.21.2.101:0/1389063871 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd75c0186c0 0x7fd75c01abc0 crc :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:50.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.202 7fd782360700 1 -- 172.21.2.101:0/1389063871 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd77c1970d0 msgr2=0x7fd77c1974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:50.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.202 7fd782360700 1 --2- 172.21.2.101:0/1389063871 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd77c1970d0 0x7fd77c1974d0 secure :-1 s=READY pgs=3010 cs=0 l=1 rev1=1 rx=0x7fd77000b5c0 tx=0x7fd77000b960).stop 2021-02-10T11:46:50.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.202 7fd782360700 1 -- 172.21.2.101:0/1389063871 shutdown_connections 2021-02-10T11:46:50.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.202 7fd782360700 1 --2- 172.21.2.101:0/1389063871 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd75c0186c0 0x7fd75c01abc0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:50.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.202 7fd782360700 1 --2- 172.21.2.101:0/1389063871 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd77c094650 0x7fd77c094a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:50.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.202 7fd782360700 1 --2- 172.21.2.101:0/1389063871 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd77c1970d0 0x7fd77c1974d0 unknown :-1 s=CLOSED pgs=3010 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:50.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.202 7fd782360700 1 --2- 172.21.2.101:0/1389063871 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd77c18f0a0 0x7fd77c196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:50.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.202 7fd782360700 1 -- 172.21.2.101:0/1389063871 >> 172.21.2.101:0/1389063871 conn(0x7fd77c1a38c0 msgr2=0x7fd77c1a2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:50.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.203 7fd782360700 1 -- 172.21.2.101:0/1389063871 shutdown_connections 2021-02-10T11:46:50.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.203 7fd782360700 1 -- 172.21.2.101:0/1389063871 wait complete. 2021-02-10T11:46:50.209 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 36, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:46:33.067171', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '174', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 37, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:46:34.142991', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '175', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:46:50.209 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:46:50.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.628 7f54f10a0700 1 Processor -- start 2021-02-10T11:46:50.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.628 7f54f10a0700 1 -- start start 2021-02-10T11:46:50.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.629 7f54f10a0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54ec0d1870 0x7f54ec1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:50.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.629 7f54f10a0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f54ec1b0ae0 0x7f54ec108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:50.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.629 7f54f10a0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54ec109a60 con 0x7f54ec0d1c70 2021-02-10T11:46:50.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.629 7f54f10a0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54ec0e5ef0 con 0x7f54ec1b0ae0 2021-02-10T11:46:50.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.629 7f54f10a0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec109750 con 0x7f54ec0d1870 2021-02-10T11:46:50.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.629 7f54ea59c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f54ec1b0ae0 0x7f54ec108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:50.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.629 7f54ea59c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f54ec1b0ae0 0x7f54ec108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33568/0 (socket says 172.21.2.101:33568) 2021-02-10T11:46:50.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.629 7f54ea59c700 1 -- 172.21.2.101:0/1578170013 learned_addr learned my addr 172.21.2.101:0/1578170013 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:50.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.629 7f54ea59c700 1 -- 172.21.2.101:0/1578170013 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54ec0d1870 msgr2=0x7f54ec1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:50.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.629 7f54ea59c700 1 --2- 172.21.2.101:0/1578170013 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54ec0d1870 0x7f54ec1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:50.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.629 7f54ea59c700 1 -- 172.21.2.101:0/1578170013 >> v1:172.21.2.108:6789/0 conn(0x7f54ec0d1c70 legacy=0x7f54ec108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:50.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.629 7f54ea59c700 1 -- 172.21.2.101:0/1578170013 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec088070 con 0x7f54ec1b0ae0 2021-02-10T11:46:50.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.629 7f54ea59c700 1 --2- 172.21.2.101:0/1578170013 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54ec1b0ae0 0x7f54ec108940 secure :-1 s=READY pgs=3011 cs=0 l=1 rev1=1 rx=0x7f54dc006700 tx=0x7f54dc009b80).ready entity=mon.0 client_cookie=8f7bf4efe82dfc19 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:50.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.630 7f54e9d9b700 1 -- 172.21.2.101:0/1578170013 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f54dc00c0c0 con 0x7f54ec1b0ae0 2021-02-10T11:46:50.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.630 7f54e9d9b700 1 -- 172.21.2.101:0/1578170013 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc00c340 con 0x7f54ec1b0ae0 2021-02-10T11:46:50.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.630 7f54e9d9b700 1 -- 172.21.2.101:0/1578170013 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f54dc00c880 con 0x7f54ec1b0ae0 2021-02-10T11:46:50.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.630 7f54f10a0700 1 -- 172.21.2.101:0/1578170013 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54ec1b0ae0 msgr2=0x7f54ec108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:50.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.630 7f54f10a0700 1 --2- 172.21.2.101:0/1578170013 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54ec1b0ae0 0x7f54ec108940 secure :-1 s=READY pgs=3011 cs=0 l=1 rev1=1 rx=0x7f54dc006700 tx=0x7f54dc009b80).stop 2021-02-10T11:46:50.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.630 7f54f10a0700 1 -- 172.21.2.101:0/1578170013 shutdown_connections 2021-02-10T11:46:50.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.630 7f54f10a0700 1 --2- 172.21.2.101:0/1578170013 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54ec1b0ae0 0x7f54ec108940 unknown :-1 s=CLOSED pgs=3011 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:50.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.630 7f54f10a0700 1 --2- 172.21.2.101:0/1578170013 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54ec0d1870 0x7f54ec1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:50.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.630 7f54f10a0700 1 -- 172.21.2.101:0/1578170013 >> 172.21.2.101:0/1578170013 conn(0x7f54ec0ce420 msgr2=0x7f54ec0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:50.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.631 7f54f10a0700 1 -- 172.21.2.101:0/1578170013 shutdown_connections 2021-02-10T11:46:50.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.631 7f54f10a0700 1 -- 172.21.2.101:0/1578170013 wait complete. 2021-02-10T11:46:50.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.645 7f54f10a0700 1 Processor -- start 2021-02-10T11:46:50.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.645 7f54f10a0700 1 -- start start 2021-02-10T11:46:50.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.646 7f54f10a0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54ec0c3820 0x7f54ec084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:50.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.646 7f54f10a0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54ec0a0290 0x7f54ec0852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:50.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.646 7f54f10a0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54ec086270 0x7f54ec086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:50.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.646 7f54f10a0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec0c0700 con 0x7f54ec0c3820 2021-02-10T11:46:50.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.646 7f54f10a0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec0858f0 con 0x7f54ec086270 2021-02-10T11:46:50.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.646 7f54f10a0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec085b10 con 0x7f54ec0a0290 2021-02-10T11:46:50.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.646 7f54ead9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54ec0c3820 0x7f54ec084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:50.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.646 7f54ead9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54ec0c3820 0x7f54ec084cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33574/0 (socket says 172.21.2.101:33574) 2021-02-10T11:46:50.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.646 7f54ead9d700 1 -- 172.21.2.101:0/1374806440 learned_addr learned my addr 172.21.2.101:0/1374806440 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:50.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.646 7f54ead9d700 1 -- 172.21.2.101:0/1374806440 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54ec0a0290 msgr2=0x7f54ec0852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:50.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.646 7f54ead9d700 1 --2- 172.21.2.101:0/1374806440 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54ec0a0290 0x7f54ec0852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:50.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.646 7f54ead9d700 1 -- 172.21.2.101:0/1374806440 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54ec086270 msgr2=0x7f54ec086670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.646 7f54ead9d700 1 --2- 172.21.2.101:0/1374806440 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54ec086270 0x7f54ec086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.646 7f54ead9d700 1 -- 172.21.2.101:0/1374806440 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec086c90 con 0x7f54ec0c3820 2021-02-10T11:46:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.647 7f54ead9d700 1 --2- 172.21.2.101:0/1374806440 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54ec0c3820 0x7f54ec084cb0 secure :-1 s=READY pgs=3012 cs=0 l=1 rev1=1 rx=0x7f54d4006f00 tx=0x7f54d400b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.647 7f54e37fe700 1 -- 172.21.2.101:0/1374806440 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f54d400bd30 con 0x7f54ec0c3820 2021-02-10T11:46:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.647 7f54e37fe700 1 -- 172.21.2.101:0/1374806440 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d4000c90 con 0x7f54ec0c3820 2021-02-10T11:46:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.647 7f54e37fe700 1 -- 172.21.2.101:0/1374806440 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f54d40022a0 con 0x7f54ec0c3820 2021-02-10T11:46:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.647 7f54f10a0700 1 -- 172.21.2.101:0/1374806440 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ec086c90 con 0x7f54ec0c3820 2021-02-10T11:46:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.647 7f54f10a0700 1 -- 172.21.2.101:0/1374806440 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ec1a2f00 con 0x7f54ec0c3820 2021-02-10T11:46:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.647 7f54e37fe700 1 -- 172.21.2.101:0/1374806440 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f54d4005310 con 0x7f54ec0c3820 2021-02-10T11:46:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.648 7f54e37fe700 1 --2- 172.21.2.101:0/1374806440 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54d0018730 0x7f54d001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.648 7f54e37fe700 1 -- 172.21.2.101:0/1374806440 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f54d4032620 con 0x7f54ec0c3820 2021-02-10T11:46:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.648 7f54ea59c700 1 --2- 172.21.2.101:0/1374806440 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54d0018730 0x7f54d001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:50.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.648 7f54ea59c700 1 --2- 172.21.2.101:0/1374806440 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54d0018730 0x7f54d001abc0 crc :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:50.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.678 7f54f10a0700 1 -- 172.21.2.101:0/1374806440 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ec082a70 con 0x7f54ec0c3820 2021-02-10T11:46:50.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.681 7f54e37fe700 1 -- 172.21.2.101:0/1374806440 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f54d4000940 con 0x7f54ec0c3820 2021-02-10T11:46:50.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.810 7f54f10a0700 1 -- 172.21.2.101:0/1374806440 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f54ec082a70 con 0x7f54ec0c3820 2021-02-10T11:46:50.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.810 7f54e37fe700 1 -- 172.21.2.101:0/1374806440 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v175) v1 ==== 92+0+10 (secure 0 0 0) 0x7f54d400bd30 con 0x7f54ec0c3820 2021-02-10T11:46:50.808 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:46:50.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.812 7f54f10a0700 1 -- 172.21.2.101:0/1374806440 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54d0018730 msgr2=0x7f54d001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:50.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.812 7f54f10a0700 1 --2- 172.21.2.101:0/1374806440 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54d0018730 0x7f54d001abc0 crc :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:50.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.812 7f54f10a0700 1 -- 172.21.2.101:0/1374806440 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54ec0c3820 msgr2=0x7f54ec084cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:50.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.812 7f54f10a0700 1 --2- 172.21.2.101:0/1374806440 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54ec0c3820 0x7f54ec084cb0 secure :-1 s=READY pgs=3012 cs=0 l=1 rev1=1 rx=0x7f54d4006f00 tx=0x7f54d400b940).stop 2021-02-10T11:46:50.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.813 7f54f10a0700 1 -- 172.21.2.101:0/1374806440 shutdown_connections 2021-02-10T11:46:50.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.813 7f54f10a0700 1 --2- 172.21.2.101:0/1374806440 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f54d0018730 0x7f54d001abc0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:50.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.813 7f54f10a0700 1 --2- 172.21.2.101:0/1374806440 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f54ec0a0290 0x7f54ec0852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:50.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.813 7f54f10a0700 1 --2- 172.21.2.101:0/1374806440 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f54ec086270 0x7f54ec086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:50.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.813 7f54f10a0700 1 --2- 172.21.2.101:0/1374806440 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f54ec0c3820 0x7f54ec084cb0 unknown :-1 s=CLOSED pgs=3012 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:50.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.813 7f54f10a0700 1 -- 172.21.2.101:0/1374806440 >> 172.21.2.101:0/1374806440 conn(0x7f54ec0872c0 msgr2=0x7f54ec095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:50.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.813 7f54f10a0700 1 -- 172.21.2.101:0/1374806440 shutdown_connections 2021-02-10T11:46:50.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:50.813 7f54f10a0700 1 -- 172.21.2.101:0/1374806440 wait complete. 2021-02-10T11:46:50.818 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:46:51.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.237 7f9740b84700 1 Processor -- start 2021-02-10T11:46:51.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.237 7f9740b84700 1 -- start start 2021-02-10T11:46:51.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.237 7f9740b84700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f973c0d1870 0x7f973c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:51.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.237 7f9740b84700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973c109a60 con 0x7f973c1b0ae0 2021-02-10T11:46:51.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.237 7f9740b84700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973c088070 con 0x7f973c0cff70 2021-02-10T11:46:51.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.237 7f9740b84700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0e5ef0 con 0x7f973c0d1870 2021-02-10T11:46:51.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.237 7f973a05d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f973c1b0ae0 0x7f973c108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34076/0 (socket says 172.21.2.101:34076) 2021-02-10T11:46:51.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.238 7f973a05d700 1 -- 172.21.2.101:0/1350853241 learned_addr learned my addr 172.21.2.101:0/1350853241 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:51.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.238 7f973985c700 1 -- 172.21.2.101:0/1350853241 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535410163 0 0) 0x7f972c0010b0 con 0x7f973c1b0ae0 2021-02-10T11:46:51.254 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.238 7f973985c700 1 -- 172.21.2.101:0/1350853241 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9728002fe0 con 0x7f973c1b0ae0 2021-02-10T11:46:51.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.238 7f973a85e700 1 --2- 172.21.2.101:0/1350853241 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f973c0d1870 0x7f973c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:51.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.238 7f973985c700 1 -- 172.21.2.101:0/1350853241 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3671774709 0 0) 0x7f972c0010b0 con 0x7f973c1b0ae0 2021-02-10T11:46:51.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.238 7f973985c700 1 -- 172.21.2.101:0/1350853241 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f973c0d1870 msgr2=0x7f973c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:51.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.238 7f973985c700 1 --2- 172.21.2.101:0/1350853241 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f973c0d1870 0x7f973c1b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:51.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.238 7f973985c700 1 -- 172.21.2.101:0/1350853241 >> v1:172.21.2.108:6789/0 conn(0x7f973c0cff70 legacy=0x7f973c109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:51.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.238 7f973985c700 1 -- 172.21.2.101:0/1350853241 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c0d1c70 con 0x7f973c1b0ae0 2021-02-10T11:46:51.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.238 7f973985c700 1 -- 172.21.2.101:0/1350853241 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f972c0015b0 con 0x7f973c1b0ae0 2021-02-10T11:46:51.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.238 7f973985c700 1 -- 172.21.2.101:0/1350853241 >> v1:172.21.2.101:6789/0 conn(0x7f973c1b0ae0 legacy=0x7f973c108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:51.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.238 7f973985c700 1 --2- 172.21.2.101:0/1350853241 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9728003a60 0x7f9728005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:51.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.238 7f973985c700 1 --2- 172.21.2.101:0/1350853241 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9728006790 0x7f9728008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:51.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.238 7f973985c700 1 --2- 172.21.2.101:0/1350853241 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9728009590 0x7f972800ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:51.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.238 7f973985c700 1 -- 172.21.2.101:0/1350853241 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972800c2d0 con 0x7f9728006790 2021-02-10T11:46:51.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.238 7f973985c700 1 -- 172.21.2.101:0/1350853241 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972800c4f0 con 0x7f9728003a60 2021-02-10T11:46:51.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.238 7f973985c700 1 -- 172.21.2.101:0/1350853241 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f972800c710 con 0x7f9728009590 2021-02-10T11:46:51.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.239 7f973a05d700 1 --2- 172.21.2.101:0/1350853241 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9728006790 0x7f9728008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:51.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.239 7f973a85e700 1 --2- 172.21.2.101:0/1350853241 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9728003a60 0x7f9728005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:51.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.239 7f973b05f700 1 --2- 172.21.2.101:0/1350853241 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9728009590 0x7f972800ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:51.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.239 7f973a05d700 1 -- 172.21.2.101:0/1350853241 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9728009590 msgr2=0x7f972800ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:51.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.239 7f973a05d700 1 --2- 172.21.2.101:0/1350853241 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9728009590 0x7f972800ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:51.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.239 7f973a05d700 1 -- 172.21.2.101:0/1350853241 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9728003a60 msgr2=0x7f9728005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:51.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.239 7f973a05d700 1 --2- 172.21.2.101:0/1350853241 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9728003a60 0x7f9728005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:51.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.239 7f973a05d700 1 -- 172.21.2.101:0/1350853241 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972800c9b0 con 0x7f9728006790 2021-02-10T11:46:51.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.239 7f973a85e700 1 --2- 172.21.2.101:0/1350853241 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9728003a60 0x7f9728005ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:46:51.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.239 7f973a05d700 1 --2- 172.21.2.101:0/1350853241 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9728006790 0x7f9728008c90 secure :-1 s=READY pgs=3014 cs=0 l=1 rev1=1 rx=0x7f972c001c60 tx=0x7f972c007d40).ready entity=mon.0 client_cookie=15bdb8375e1bfe7f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:51.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.239 7f973985c700 1 -- 172.21.2.101:0/1350853241 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f972c0009b0 con 0x7f9728006790 2021-02-10T11:46:51.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.239 7f973985c700 1 -- 172.21.2.101:0/1350853241 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972c0013f0 con 0x7f9728006790 2021-02-10T11:46:51.256 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.239 7f973985c700 1 -- 172.21.2.101:0/1350853241 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f972c04e510 con 0x7f9728006790 2021-02-10T11:46:51.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.239 7f9740b84700 1 -- 172.21.2.101:0/1350853241 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9728006790 msgr2=0x7f9728008c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:51.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.239 7f9740b84700 1 --2- 172.21.2.101:0/1350853241 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9728006790 0x7f9728008c90 secure :-1 s=READY pgs=3014 cs=0 l=1 rev1=1 rx=0x7f972c001c60 tx=0x7f972c007d40).stop 2021-02-10T11:46:51.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.239 7f973b05f700 1 -- 172.21.2.101:0/1350853241 reap_dead start 2021-02-10T11:46:51.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.240 7f9740b84700 1 -- 172.21.2.101:0/1350853241 shutdown_connections 2021-02-10T11:46:51.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.240 7f9740b84700 1 -- 172.21.2.101:0/1350853241 >> 172.21.2.101:0/1350853241 conn(0x7f973c0ce420 msgr2=0x7f973c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:51.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.240 7f9740b84700 1 -- 172.21.2.101:0/1350853241 shutdown_connections 2021-02-10T11:46:51.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.240 7f9740b84700 1 -- 172.21.2.101:0/1350853241 wait complete. 2021-02-10T11:46:51.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.255 7f9740b84700 1 Processor -- start 2021-02-10T11:46:51.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.255 7f9740b84700 1 -- start start 2021-02-10T11:46:51.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.255 7f9740b84700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f973c09bcc0 0x7f973c09c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:51.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.255 7f9740b84700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f973c0a0820 0x7f973c0a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:51.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.255 7f9740b84700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f973c0a1240 0x7f973c0858d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:51.257 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.255 7f9740b84700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0a66b0 con 0x7f973c0a1240 2021-02-10T11:46:51.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.255 7f9740b84700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c1a49b0 con 0x7f973c0a0820 2021-02-10T11:46:51.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.255 7f9740b84700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c1a4b90 con 0x7f973c09bcc0 2021-02-10T11:46:51.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.255 7f973b05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f973c0a1240 0x7f973c0858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:51.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.255 7f973b05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f973c0a1240 0x7f973c0858d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33596/0 (socket says 172.21.2.101:33596) 2021-02-10T11:46:51.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.255 7f973b05f700 1 -- 172.21.2.101:0/1488225629 learned_addr learned my addr 172.21.2.101:0/1488225629 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:51.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.255 7f973a85e700 1 --2- 172.21.2.101:0/1488225629 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f973c09bcc0 0x7f973c09c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:51.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.255 7f973a05d700 1 --2- 172.21.2.101:0/1488225629 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f973c0a0820 0x7f973c0a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:51.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.256 7f973b05f700 1 -- 172.21.2.101:0/1488225629 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f973c09bcc0 msgr2=0x7f973c09c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:51.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.256 7f973b05f700 1 --2- 172.21.2.101:0/1488225629 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f973c09bcc0 0x7f973c09c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:51.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.256 7f973b05f700 1 -- 172.21.2.101:0/1488225629 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f973c0a0820 msgr2=0x7f973c0a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:51.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.256 7f973b05f700 1 --2- 172.21.2.101:0/1488225629 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f973c0a0820 0x7f973c0a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:51.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.256 7f973b05f700 1 -- 172.21.2.101:0/1488225629 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c0a1640 con 0x7f973c0a1240 2021-02-10T11:46:51.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.256 7f973a85e700 1 --2- 172.21.2.101:0/1488225629 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f973c09bcc0 0x7f973c09c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:46:51.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.256 7f973a05d700 1 --2- 172.21.2.101:0/1488225629 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f973c0a0820 0x7f973c0a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:46:51.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.256 7f973b05f700 1 --2- 172.21.2.101:0/1488225629 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f973c0a1240 0x7f973c0858d0 secure :-1 s=READY pgs=3015 cs=0 l=1 rev1=1 rx=0x7f9734009980 tx=0x7f973400bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:51.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.256 7f97337fe700 1 -- 172.21.2.101:0/1488225629 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9734006ba0 con 0x7f973c0a1240 2021-02-10T11:46:51.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.256 7f97337fe700 1 -- 172.21.2.101:0/1488225629 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973400c2f0 con 0x7f973c0a1240 2021-02-10T11:46:51.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.256 7f97337fe700 1 -- 172.21.2.101:0/1488225629 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f973400c740 con 0x7f973c0a1240 2021-02-10T11:46:51.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.256 7f9740b84700 1 -- 172.21.2.101:0/1488225629 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973c0a1640 con 0x7f973c0a1240 2021-02-10T11:46:51.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.256 7f9740b84700 1 -- 172.21.2.101:0/1488225629 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973c0a1640 con 0x7f973c0a1240 2021-02-10T11:46:51.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.257 7f97337fe700 1 -- 172.21.2.101:0/1488225629 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9734006ba0 con 0x7f973c0a1240 2021-02-10T11:46:51.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.257 7f97337fe700 1 --2- 172.21.2.101:0/1488225629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97200186c0 0x7f972001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:51.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.257 7f97337fe700 1 -- 172.21.2.101:0/1488225629 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9734032930 con 0x7f973c0a1240 2021-02-10T11:46:51.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.257 7f973a85e700 1 --2- 172.21.2.101:0/1488225629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97200186c0 0x7f972001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:51.260 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.257 7f973a85e700 1 --2- 172.21.2.101:0/1488225629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97200186c0 0x7f972001abc0 crc :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:51.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.287 7f9740b84700 1 -- 172.21.2.101:0/1488225629 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f973c0fe1c0 con 0x7f973c0a1240 2021-02-10T11:46:51.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.290 7f97337fe700 1 -- 172.21.2.101:0/1488225629 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f973400be10 con 0x7f973c0a1240 2021-02-10T11:46:51.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.419 7f9740b84700 1 -- 172.21.2.101:0/1488225629 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f973c0fe1c0 con 0x7f973c0a1240 2021-02-10T11:46:51.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.419 7f97337fe700 1 -- 172.21.2.101:0/1488225629 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v175) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9734006b80 con 0x7f973c0a1240 2021-02-10T11:46:51.417 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:46:51.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.421 7f9740b84700 1 -- 172.21.2.101:0/1488225629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97200186c0 msgr2=0x7f972001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:51.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.421 7f9740b84700 1 --2- 172.21.2.101:0/1488225629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97200186c0 0x7f972001abc0 crc :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:51.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.421 7f9740b84700 1 -- 172.21.2.101:0/1488225629 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f973c0a1240 msgr2=0x7f973c0858d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:51.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.421 7f9740b84700 1 --2- 172.21.2.101:0/1488225629 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f973c0a1240 0x7f973c0858d0 secure :-1 s=READY pgs=3015 cs=0 l=1 rev1=1 rx=0x7f9734009980 tx=0x7f973400bcd0).stop 2021-02-10T11:46:51.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.421 7f9740b84700 1 -- 172.21.2.101:0/1488225629 shutdown_connections 2021-02-10T11:46:51.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.421 7f9740b84700 1 --2- 172.21.2.101:0/1488225629 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f973c0a1240 0x7f973c0858d0 unknown :-1 s=CLOSED pgs=3015 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:51.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.421 7f9740b84700 1 --2- 172.21.2.101:0/1488225629 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f973c0a0820 0x7f973c0a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:51.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.421 7f9740b84700 1 --2- 172.21.2.101:0/1488225629 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97200186c0 0x7f972001abc0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:51.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.421 7f9740b84700 1 --2- 172.21.2.101:0/1488225629 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f973c09bcc0 0x7f973c09c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:51.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.421 7f9740b84700 1 -- 172.21.2.101:0/1488225629 >> 172.21.2.101:0/1488225629 conn(0x7f973c0bef00 msgr2=0x7f973c09d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:51.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.421 7f9740b84700 1 -- 172.21.2.101:0/1488225629 shutdown_connections 2021-02-10T11:46:51.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.422 7f9740b84700 1 -- 172.21.2.101:0/1488225629 wait complete. 2021-02-10T11:46:51.427 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:46:51.860 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.845 7fb6dcf1d700 1 Processor -- start 2021-02-10T11:46:51.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.846 7fb6dcf1d700 1 -- start start 2021-02-10T11:46:51.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.846 7fb6dcf1d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6d80d1c70 0x7fb6d80cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:51.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.846 7fb6dcf1d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb6d80cf580 0x7fb6d80cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:51.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.846 7fb6dcf1d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d80fe1c0 con 0x7fb6d80d1870 2021-02-10T11:46:51.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.846 7fb6dcf1d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d81b0cb0 con 0x7fb6d80cf580 2021-02-10T11:46:51.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.846 7fb6dcf1d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d8109a60 con 0x7fb6d80d1c70 2021-02-10T11:46:51.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.846 7fb6d7314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb6d80cf580 0x7fb6d80cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:51.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.846 7fb6d7314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb6d80cf580 0x7fb6d80cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33602/0 (socket says 172.21.2.101:33602) 2021-02-10T11:46:51.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.846 7fb6d7314700 1 -- 172.21.2.101:0/147715565 learned_addr learned my addr 172.21.2.101:0/147715565 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:51.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.847 7fb6d7314700 1 -- 172.21.2.101:0/147715565 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6d80d1c70 msgr2=0x7fb6d80cef60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:46:51.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.847 7fb6d7314700 1 --2- 172.21.2.101:0/147715565 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6d80d1c70 0x7fb6d80cef60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:51.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.847 7fb6d7314700 1 -- 172.21.2.101:0/147715565 >> v1:172.21.2.101:6789/0 conn(0x7fb6d80d1870 legacy=0x7fb6d81b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:51.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.847 7fb6d7314700 1 -- 172.21.2.101:0/147715565 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d8088070 con 0x7fb6d80cf580 2021-02-10T11:46:51.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.847 7fb6d7314700 1 --2- 172.21.2.101:0/147715565 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6d80cf580 0x7fb6d80cf980 secure :-1 s=READY pgs=3017 cs=0 l=1 rev1=1 rx=0x7fb6d0006f00 tx=0x7fb6d0009f10).ready entity=mon.0 client_cookie=619a5097adda7ea7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:51.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.847 7fb6d5b11700 1 -- 172.21.2.101:0/147715565 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb6d000b3b0 con 0x7fb6d80cf580 2021-02-10T11:46:51.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.847 7fb6d5b11700 1 -- 172.21.2.101:0/147715565 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d000b730 con 0x7fb6d80cf580 2021-02-10T11:46:51.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.847 7fb6d5b11700 1 -- 172.21.2.101:0/147715565 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb6d000bb90 con 0x7fb6d80cf580 2021-02-10T11:46:51.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.847 7fb6dcf1d700 1 -- 172.21.2.101:0/147715565 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6d80cf580 msgr2=0x7fb6d80cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:51.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.847 7fb6dcf1d700 1 --2- 172.21.2.101:0/147715565 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6d80cf580 0x7fb6d80cf980 secure :-1 s=READY pgs=3017 cs=0 l=1 rev1=1 rx=0x7fb6d0006f00 tx=0x7fb6d0009f10).stop 2021-02-10T11:46:51.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.848 7fb6dcf1d700 1 -- 172.21.2.101:0/147715565 shutdown_connections 2021-02-10T11:46:51.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.848 7fb6dcf1d700 1 --2- 172.21.2.101:0/147715565 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6d80d1c70 0x7fb6d80cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:51.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.848 7fb6dcf1d700 1 --2- 172.21.2.101:0/147715565 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6d80cf580 0x7fb6d80cf980 unknown :-1 s=CLOSED pgs=3017 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:51.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.848 7fb6dcf1d700 1 -- 172.21.2.101:0/147715565 >> 172.21.2.101:0/147715565 conn(0x7fb6d80ce420 msgr2=0x7fb6d80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:51.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.848 7fb6dcf1d700 1 -- 172.21.2.101:0/147715565 shutdown_connections 2021-02-10T11:46:51.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.848 7fb6dcf1d700 1 -- 172.21.2.101:0/147715565 wait complete. 2021-02-10T11:46:51.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.863 7fb6dcf1d700 1 Processor -- start 2021-02-10T11:46:51.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.863 7fb6dcf1d700 1 -- start start 2021-02-10T11:46:51.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.863 7fb6dcf1d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6d80c2370 0x7fb6d818f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:51.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.863 7fb6dcf1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6d8084cb0 0x7fb6d81a7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:51.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.863 7fb6dcf1d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb6d81a7a90 0x7fb6d8085700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:51.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.863 7fb6dcf1d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d80be840 con 0x7fb6d8084cb0 2021-02-10T11:46:51.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.863 7fb6dcf1d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d80c29a0 con 0x7fb6d81a7a90 2021-02-10T11:46:51.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.863 7fb6dcf1d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d81a7e90 con 0x7fb6d80c2370 2021-02-10T11:46:51.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.863 7fb6d6312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6d8084cb0 0x7fb6d81a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:51.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.863 7fb6d6312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6d8084cb0 0x7fb6d81a7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33606/0 (socket says 172.21.2.101:33606) 2021-02-10T11:46:51.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.863 7fb6d6312700 1 -- 172.21.2.101:0/4231061944 learned_addr learned my addr 172.21.2.101:0/4231061944 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:51.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.864 7fb6d6312700 1 -- 172.21.2.101:0/4231061944 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6d80c2370 msgr2=0x7fb6d818f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:51.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.864 7fb6d6312700 1 --2- 172.21.2.101:0/4231061944 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6d80c2370 0x7fb6d818f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:51.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.864 7fb6d6312700 1 -- 172.21.2.101:0/4231061944 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb6d81a7a90 msgr2=0x7fb6d8085700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:51.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.864 7fb6d6312700 1 --2- 172.21.2.101:0/4231061944 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb6d81a7a90 0x7fb6d8085700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:51.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.864 7fb6d6312700 1 -- 172.21.2.101:0/4231061944 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d81a80b0 con 0x7fb6d8084cb0 2021-02-10T11:46:51.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.864 7fb6d6312700 1 --2- 172.21.2.101:0/4231061944 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6d8084cb0 0x7fb6d81a7470 secure :-1 s=READY pgs=3018 cs=0 l=1 rev1=1 rx=0x7fb6c000b5c0 tx=0x7fb6c000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:51.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.864 7fb6cf7fe700 1 -- 172.21.2.101:0/4231061944 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb6c000bd50 con 0x7fb6d8084cb0 2021-02-10T11:46:51.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.864 7fb6cf7fe700 1 -- 172.21.2.101:0/4231061944 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c0006030 con 0x7fb6d8084cb0 2021-02-10T11:46:51.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.864 7fb6dcf1d700 1 -- 172.21.2.101:0/4231061944 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d81a80b0 con 0x7fb6d8084cb0 2021-02-10T11:46:51.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.864 7fb6dcf1d700 1 -- 172.21.2.101:0/4231061944 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d8086010 con 0x7fb6d8084cb0 2021-02-10T11:46:51.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.864 7fb6cf7fe700 1 -- 172.21.2.101:0/4231061944 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb6c00054d0 con 0x7fb6d8084cb0 2021-02-10T11:46:51.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.865 7fb6cf7fe700 1 -- 172.21.2.101:0/4231061944 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb6c0006210 con 0x7fb6d8084cb0 2021-02-10T11:46:51.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.865 7fb6cf7fe700 1 --2- 172.21.2.101:0/4231061944 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb6bc018730 0x7fb6bc01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:51.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.865 7fb6cf7fe700 1 -- 172.21.2.101:0/4231061944 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb6c00327f0 con 0x7fb6d8084cb0 2021-02-10T11:46:51.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.865 7fb6d6b13700 1 --2- 172.21.2.101:0/4231061944 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb6bc018730 0x7fb6bc01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:51.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.865 7fb6d6b13700 1 --2- 172.21.2.101:0/4231061944 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb6bc018730 0x7fb6bc01ac10 crc :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:51.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.896 7fb6dcf1d700 1 -- 172.21.2.101:0/4231061944 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d8082a70 con 0x7fb6d8084cb0 2021-02-10T11:46:51.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:51.898 7fb6cf7fe700 1 -- 172.21.2.101:0/4231061944 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb6c0000960 con 0x7fb6d8084cb0 2021-02-10T11:46:52.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.027 7fb6dcf1d700 1 -- 172.21.2.101:0/4231061944 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7fb6d8082a70 con 0x7fb6d8084cb0 2021-02-10T11:46:52.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.027 7fb6cf7fe700 1 -- 172.21.2.101:0/4231061944 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v175) v1 ==== 100+0+10 (secure 0 0 0) 0x7fb6c000bd50 con 0x7fb6d8084cb0 2021-02-10T11:46:52.025 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:46:52.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.029 7fb6dcf1d700 1 -- 172.21.2.101:0/4231061944 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb6bc018730 msgr2=0x7fb6bc01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:52.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.029 7fb6dcf1d700 1 --2- 172.21.2.101:0/4231061944 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb6bc018730 0x7fb6bc01ac10 crc :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:52.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.029 7fb6dcf1d700 1 -- 172.21.2.101:0/4231061944 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6d8084cb0 msgr2=0x7fb6d81a7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:52.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.029 7fb6dcf1d700 1 --2- 172.21.2.101:0/4231061944 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6d8084cb0 0x7fb6d81a7470 secure :-1 s=READY pgs=3018 cs=0 l=1 rev1=1 rx=0x7fb6c000b5c0 tx=0x7fb6c000b960).stop 2021-02-10T11:46:52.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.030 7fb6dcf1d700 1 -- 172.21.2.101:0/4231061944 shutdown_connections 2021-02-10T11:46:52.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.030 7fb6dcf1d700 1 --2- 172.21.2.101:0/4231061944 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb6d81a7a90 0x7fb6d8085700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:52.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.030 7fb6dcf1d700 1 --2- 172.21.2.101:0/4231061944 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb6d8084cb0 0x7fb6d81a7470 unknown :-1 s=CLOSED pgs=3018 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:52.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.030 7fb6dcf1d700 1 --2- 172.21.2.101:0/4231061944 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb6bc018730 0x7fb6bc01ac10 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:52.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.030 7fb6dcf1d700 1 --2- 172.21.2.101:0/4231061944 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb6d80c2370 0x7fb6d818f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:52.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.030 7fb6dcf1d700 1 -- 172.21.2.101:0/4231061944 >> 172.21.2.101:0/4231061944 conn(0x7fb6d80b5dc0 msgr2=0x7fb6d80c2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:52.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.030 7fb6dcf1d700 1 -- 172.21.2.101:0/4231061944 shutdown_connections 2021-02-10T11:46:52.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.030 7fb6dcf1d700 1 -- 172.21.2.101:0/4231061944 wait complete. 2021-02-10T11:46:52.035 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:46:52.035 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:46:52.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.450 7f37a5949700 1 Processor -- start 2021-02-10T11:46:52.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.450 7f37a5949700 1 -- start start 2021-02-10T11:46:52.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.451 7f37a5949700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37a01b0470 0x7f37a01b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:52.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.451 7f37a5949700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f37a00ef610 0x7f37a00efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:52.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.451 7f37a5949700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f37a00f0030 0x7f37a00f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:52.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.451 7f37a5949700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37a00e5ea0 con 0x7f37a00f0030 2021-02-10T11:46:52.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.451 7f37a5949700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37a00f0b50 con 0x7f37a00ef610 2021-02-10T11:46:52.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.451 7f37a5949700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a00e52e0 con 0x7f37a01b0470 2021-02-10T11:46:52.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.451 7f379fd75700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f37a00f0030 0x7f37a00f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:52.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.451 7f379fd75700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f37a00f0030 0x7f37a00f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33612/0 (socket says 172.21.2.101:33612) 2021-02-10T11:46:52.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.451 7f379fd75700 1 -- 172.21.2.101:0/2398925590 learned_addr learned my addr 172.21.2.101:0/2398925590 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:52.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.451 7f379fd75700 1 -- 172.21.2.101:0/2398925590 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37a01b0470 msgr2=0x7f37a01b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:52.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.451 7f379fd75700 1 --2- 172.21.2.101:0/2398925590 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37a01b0470 0x7f37a01b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:52.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.451 7f379fd75700 1 -- 172.21.2.101:0/2398925590 >> v2:172.21.2.108:3300/0 conn(0x7f37a00ef610 msgr2=0x7f37a00efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:52.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.451 7f379fd75700 1 --2- 172.21.2.101:0/2398925590 >> v2:172.21.2.108:3300/0 conn(0x7f37a00ef610 0x7f37a00efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:52.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.451 7f379fd75700 1 -- 172.21.2.101:0/2398925590 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a00f10d0 con 0x7f37a00f0030 2021-02-10T11:46:52.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.451 7f379fd75700 1 --2- 172.21.2.101:0/2398925590 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37a00f0030 0x7f37a00f0430 secure :-1 s=READY pgs=3019 cs=0 l=1 rev1=1 rx=0x7f3798006f00 tx=0x7f3798009f10).ready entity=mon.0 client_cookie=467bebbec975ef76 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:52.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.451 7f379e572700 1 -- 172.21.2.101:0/2398925590 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f379800b3b0 con 0x7f37a00f0030 2021-02-10T11:46:52.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.451 7f379e572700 1 -- 172.21.2.101:0/2398925590 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379800b730 con 0x7f37a00f0030 2021-02-10T11:46:52.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.451 7f379e572700 1 -- 172.21.2.101:0/2398925590 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f379800bc50 con 0x7f37a00f0030 2021-02-10T11:46:52.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.451 7f37a5949700 1 -- 172.21.2.101:0/2398925590 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37a00f0030 msgr2=0x7f37a00f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:52.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.451 7f37a5949700 1 --2- 172.21.2.101:0/2398925590 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37a00f0030 0x7f37a00f0430 secure :-1 s=READY pgs=3019 cs=0 l=1 rev1=1 rx=0x7f3798006f00 tx=0x7f3798009f10).stop 2021-02-10T11:46:52.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.452 7f37a5949700 1 -- 172.21.2.101:0/2398925590 shutdown_connections 2021-02-10T11:46:52.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.452 7f37a5949700 1 --2- 172.21.2.101:0/2398925590 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37a00f0030 0x7f37a00f0430 unknown :-1 s=CLOSED pgs=3019 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:52.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.452 7f37a5949700 1 --2- 172.21.2.101:0/2398925590 >> v2:172.21.2.108:3300/0 conn(0x7f37a00ef610 0x7f37a00efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:52.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.452 7f37a5949700 1 --2- 172.21.2.101:0/2398925590 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37a01b0470 0x7f37a01b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:52.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.452 7f37a5949700 1 -- 172.21.2.101:0/2398925590 >> 172.21.2.101:0/2398925590 conn(0x7f37a00ce3e0 msgr2=0x7f37a00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:52.465 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.452 7f37a5949700 1 -- 172.21.2.101:0/2398925590 shutdown_connections 2021-02-10T11:46:52.466 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.452 7f37a5949700 1 -- 172.21.2.101:0/2398925590 wait complete. 2021-02-10T11:46:52.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.466 7f37a5949700 1 Processor -- start 2021-02-10T11:46:52.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.467 7f37a5949700 1 -- start start 2021-02-10T11:46:52.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.467 7f37a5949700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37a00b1f20 0x7f37a00b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:52.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.467 7f37a5949700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37a01ad760 0x7f37a01adb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:52.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.467 7f37a5949700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37a01ae180 0x7f37a01ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:52.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.467 7f37a5949700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a01ad1e0 con 0x7f37a01ae180 2021-02-10T11:46:52.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.467 7f37a5949700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a019cb70 con 0x7f37a00b1f20 2021-02-10T11:46:52.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.467 7f37a5949700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a019cea0 con 0x7f37a01ad760 2021-02-10T11:46:52.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.467 7f379fd75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37a01ae180 0x7f37a01ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:52.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.467 7f379fd75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37a01ae180 0x7f37a01ae580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33616/0 (socket says 172.21.2.101:33616) 2021-02-10T11:46:52.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.467 7f379fd75700 1 -- 172.21.2.101:0/2659043323 learned_addr learned my addr 172.21.2.101:0/2659043323 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:52.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.468 7f379fd75700 1 -- 172.21.2.101:0/2659043323 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37a01ad760 msgr2=0x7f37a01adb60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:52.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.468 7f379fd75700 1 --2- 172.21.2.101:0/2659043323 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37a01ad760 0x7f37a01adb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:52.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.468 7f379fd75700 1 -- 172.21.2.101:0/2659043323 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37a00b1f20 msgr2=0x7f37a00b2320 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:52.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.468 7f379fd75700 1 --2- 172.21.2.101:0/2659043323 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37a00b1f20 0x7f37a00b2320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:52.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.468 7f379fd75700 1 -- 172.21.2.101:0/2659043323 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a019d0c0 con 0x7f37a01ae180 2021-02-10T11:46:52.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.468 7f379fd75700 1 --2- 172.21.2.101:0/2659043323 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37a01ae180 0x7f37a01ae580 secure :-1 s=READY pgs=3020 cs=0 l=1 rev1=1 rx=0x7f3798009390 tx=0x7f379800bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:52.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.468 7f3797fff700 1 -- 172.21.2.101:0/2659043323 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3798003030 con 0x7f37a01ae180 2021-02-10T11:46:52.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.468 7f3797fff700 1 -- 172.21.2.101:0/2659043323 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37980068e0 con 0x7f37a01ae180 2021-02-10T11:46:52.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.468 7f3797fff700 1 -- 172.21.2.101:0/2659043323 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f379800c570 con 0x7f37a01ae180 2021-02-10T11:46:52.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.468 7f37a5949700 1 -- 172.21.2.101:0/2659043323 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a019d0c0 con 0x7f37a01ae180 2021-02-10T11:46:52.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.468 7f37a5949700 1 -- 172.21.2.101:0/2659043323 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a019d650 con 0x7f37a01ae180 2021-02-10T11:46:52.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.469 7f3797fff700 1 -- 172.21.2.101:0/2659043323 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3798001270 con 0x7f37a01ae180 2021-02-10T11:46:52.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.469 7f3797fff700 1 --2- 172.21.2.101:0/2659043323 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37840186c0 0x7f378401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:52.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.469 7f3797fff700 1 -- 172.21.2.101:0/2659043323 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f37980328b0 con 0x7f37a01ae180 2021-02-10T11:46:52.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.469 7f379f574700 1 --2- 172.21.2.101:0/2659043323 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37840186c0 0x7f378401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:52.469 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.469 7f379f574700 1 --2- 172.21.2.101:0/2659043323 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37840186c0 0x7f378401abc0 crc :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:52.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.501 7f37a5949700 1 -- 172.21.2.101:0/2659043323 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a00f0b50 con 0x7f37a01ae180 2021-02-10T11:46:52.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.504 7f3797fff700 1 -- 172.21.2.101:0/2659043323 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3798009430 con 0x7f37a01ae180 2021-02-10T11:46:52.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.631 7f37a5949700 1 -- 172.21.2.101:0/2659043323 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37a00f0b50 con 0x7f37a01ae180 2021-02-10T11:46:52.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.632 7f3797fff700 1 -- 172.21.2.101:0/2659043323 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+2111 (secure 0 0 0) 0x7f379801f620 con 0x7f37a01ae180 2021-02-10T11:46:52.630 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 148 2021-02-10T11:46:52.630 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:46:52.630 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":148,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":12780,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2200088074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2200088074},{"type":"v1","addr":"172.21.2.101:6835","nonce":2200088074}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":144}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:46:35.149920","modified":"2021-02-10 11:46:37.158088","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12777},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12777":{"gid":12777,"name":"b","rank":0,"incarnation":145,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/1343655613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":1343655613},{"type":"v1","addr":"172.21.2.108:6833","nonce":1343655613}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[37],"metadata_pool":36,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":18}]} 2021-02-10T11:46:52.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.634 7f37a5949700 1 -- 172.21.2.101:0/2659043323 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37840186c0 msgr2=0x7f378401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:52.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.634 7f37a5949700 1 --2- 172.21.2.101:0/2659043323 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37840186c0 0x7f378401abc0 crc :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:52.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.634 7f37a5949700 1 -- 172.21.2.101:0/2659043323 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37a01ae180 msgr2=0x7f37a01ae580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:52.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.634 7f37a5949700 1 --2- 172.21.2.101:0/2659043323 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37a01ae180 0x7f37a01ae580 secure :-1 s=READY pgs=3020 cs=0 l=1 rev1=1 rx=0x7f3798009390 tx=0x7f379800bd40).stop 2021-02-10T11:46:52.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.634 7f37a5949700 1 -- 172.21.2.101:0/2659043323 shutdown_connections 2021-02-10T11:46:52.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.634 7f37a5949700 1 --2- 172.21.2.101:0/2659043323 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37840186c0 0x7f378401abc0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:52.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.634 7f37a5949700 1 --2- 172.21.2.101:0/2659043323 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37a01ad760 0x7f37a01adb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:52.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.634 7f37a5949700 1 --2- 172.21.2.101:0/2659043323 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37a01ae180 0x7f37a01ae580 unknown :-1 s=CLOSED pgs=3020 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:52.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.634 7f37a5949700 1 --2- 172.21.2.101:0/2659043323 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37a00b1f20 0x7f37a00b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:52.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.634 7f37a5949700 1 -- 172.21.2.101:0/2659043323 >> 172.21.2.101:0/2659043323 conn(0x7f37a0109a10 msgr2=0x7f37a009aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:52.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.635 7f37a5949700 1 -- 172.21.2.101:0/2659043323 shutdown_connections 2021-02-10T11:46:52.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:52.635 7f37a5949700 1 -- 172.21.2.101:0/2659043323 wait complete. 2021-02-10T11:46:52.842 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 148, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:46:35.149920', 'modified': '2021-02-10 11:46:37.158088', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12777}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12777': {'gid': 12777, 'name': 'b', 'rank': 0, 'incarnation': 145, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/1343655613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 1343655613}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 1343655613}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [37], 'metadata_pool': 36, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:46:52.842 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:46:52.842 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:46:53.239 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:46:53.239 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:46:53.240 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:46:53.240 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 12777, 2021-02-10T11:46:53.240 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:46:53.240 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:46:53.240 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 18.090102988999998, 2021-02-10T11:46:53.240 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 148, 2021-02-10T11:46:53.241 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 175, 2021-02-10T11:46:53.241 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 175, 2021-02-10T11:46:53.241 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 23.735307517999999 2021-02-10T11:46:53.241 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:46:53.244 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 12777, "want_state": "up:active", "state": "up:active", "rank_uptime": 18.090102988999998, "mdsmap_epoch": 148, "osdmap_epoch": 175, "osdmap_epoch_barrier": 175, "uptime": 23.735307517999999 } 2021-02-10T11:46:53.244 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:46:53.244 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-02-10T11:46:53.244 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:46:53.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:53.245 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:46:53.579 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:46:53.926 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:46:53.926 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:46:54.271 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:46:54.272 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:46:54.272 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:46:54.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:54.273 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:46:54.607 INFO:teuthology.orchestra.run.gibba001.stdout:42 2021-02-10T11:46:54.608 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2021-02-10T11:46:54.608 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:46:54.610 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:46:54.955 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:46:54.955 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:46:55.021 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[174142]: starting ceph client 2021-02-10T11:46:55.021 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:2021-02-10 11:46:55.023 7f297017df80 -1 init, newargv = 0x5636e3eedd80 newargc=9 2021-02-10T11:46:55.026 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[174142]: starting fuse 2021-02-10T11:46:55.298 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:46:55.298 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:46:55.299 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:46:55.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:55.299 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:46:55.633 INFO:teuthology.orchestra.run.gibba001.stdout:42 2021-02-10T11:46:55.633 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:46:55.634 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2021-02-10T11:46:55.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:55.634 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:46:55.634 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:46:55.634 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:46:55.634 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:46:55.634 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:46:55.634 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:46:55.634 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:46:55.634 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-02-10T11:46:55.635 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:46:55.635 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:46:55.635 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:46:55.635 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:46:55.635 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:46:55.635 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:46:55.635 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:46:55.635 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:46:55.635 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:46:55.635 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:46:55.635 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:46:55.635 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:46:55.635 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:46:55.635 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.1")) 2021-02-10T11:46:55.635 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:46:55.998 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.1.174142.asok 2021-02-10T11:46:56.000 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.174142.asok 2021-02-10T11:46:56.001 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:56.001 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.174142.asok status 2021-02-10T11:46:56.391 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:46:56.391 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:46:56.391 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:46:56.391 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:46:56.392 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "1", 2021-02-10T11:46:56.392 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:46:56.392 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:46:56.392 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "174142", 2021-02-10T11:46:56.392 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:46:56.392 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:56.392 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:46:56.392 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:46:56.392 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 12972, 2021-02-10T11:46:56.392 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:46:56.392 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:46:56.392 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:46:56.392 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 12972 2021-02-10T11:46:56.392 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:56.393 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:46:56.393 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:56.393 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:46:56.393 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 671176016 2021-02-10T11:46:56.393 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:56.393 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:56.393 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:46:56.393 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:46:56.393 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:46:56.393 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 671176016 2021-02-10T11:46:56.393 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:46:56.393 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.12972 172.21.2.101:0/671176016", 2021-02-10T11:46:56.393 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/671176016", 2021-02-10T11:46:56.393 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:46:56.394 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 148, 2021-02-10T11:46:56.394 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 175, 2021-02-10T11:46:56.394 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:46:56.394 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:46:56.394 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:46:56.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:56.396 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:46:56.396 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:46:56.732 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:46:56.732 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:46:56.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:46:56.732 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:46:57.079 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2021-02-10T11:46:57.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.498 7f893e23a700 1 Processor -- start 2021-02-10T11:46:57.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.498 7f893e23a700 1 -- start start 2021-02-10T11:46:57.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.499 7f893e23a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89380d1880 0x7f89381b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:57.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.499 7f893e23a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f89381b0af0 0x7f8938108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:57.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.499 7f893e23a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8938109a70 con 0x7f89380d1c80 2021-02-10T11:46:57.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.499 7f893e23a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89380e5f00 con 0x7f89381b0af0 2021-02-10T11:46:57.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.499 7f893e23a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8938109760 con 0x7f89380d1880 2021-02-10T11:46:57.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.499 7f8937fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89380d1880 0x7f89381b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:57.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.499 7f893c857700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f89380d1c80 0x7f8938108f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34128/0 (socket says 172.21.2.101:34128) 2021-02-10T11:46:57.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.499 7f893c857700 1 -- 172.21.2.101:0/561509511 learned_addr learned my addr 172.21.2.101:0/561509511 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:57.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.499 7f89377fe700 1 --2- 172.21.2.101:0/561509511 >> v2:172.21.2.101:3300/0 conn(0x7f89381b0af0 0x7f8938108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:57.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.499 7f89377fe700 1 -- 172.21.2.101:0/561509511 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89380d1880 msgr2=0x7f89381b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:57.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.500 7f89377fe700 1 --2- 172.21.2.101:0/561509511 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89380d1880 0x7f89381b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:57.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.500 7f89377fe700 1 -- 172.21.2.101:0/561509511 >> v1:172.21.2.101:6789/0 conn(0x7f89380d1c80 legacy=0x7f8938108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:57.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.500 7f89377fe700 1 -- 172.21.2.101:0/561509511 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8938088070 con 0x7f89381b0af0 2021-02-10T11:46:57.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.500 7f8937fff700 1 --2- 172.21.2.101:0/561509511 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89380d1880 0x7f89381b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:46:57.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.500 7f89377fe700 1 --2- 172.21.2.101:0/561509511 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89381b0af0 0x7f8938108950 secure :-1 s=READY pgs=3024 cs=0 l=1 rev1=1 rx=0x7f892c006650 tx=0x7f892c009a80).ready entity=mon.0 client_cookie=fb315234d956b61e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:57.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.500 7f8936ffd700 1 -- 172.21.2.101:0/561509511 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f892c00c0c0 con 0x7f89381b0af0 2021-02-10T11:46:57.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.500 7f8936ffd700 1 -- 172.21.2.101:0/561509511 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892c00c340 con 0x7f89381b0af0 2021-02-10T11:46:57.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.500 7f8936ffd700 1 -- 172.21.2.101:0/561509511 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f892c00c860 con 0x7f89381b0af0 2021-02-10T11:46:57.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.500 7f893e23a700 1 -- 172.21.2.101:0/561509511 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89381b0af0 msgr2=0x7f8938108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:57.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.500 7f893e23a700 1 --2- 172.21.2.101:0/561509511 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89381b0af0 0x7f8938108950 secure :-1 s=READY pgs=3024 cs=0 l=1 rev1=1 rx=0x7f892c006650 tx=0x7f892c009a80).stop 2021-02-10T11:46:57.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.500 7f893e23a700 1 -- 172.21.2.101:0/561509511 shutdown_connections 2021-02-10T11:46:57.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.500 7f893e23a700 1 --2- 172.21.2.101:0/561509511 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f89381b0af0 0x7f8938108950 unknown :-1 s=CLOSED pgs=3024 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:57.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.500 7f893e23a700 1 --2- 172.21.2.101:0/561509511 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89380d1880 0x7f89381b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:57.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.500 7f893e23a700 1 -- 172.21.2.101:0/561509511 >> 172.21.2.101:0/561509511 conn(0x7f89380ce430 msgr2=0x7f89380d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:57.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.500 7f893e23a700 1 -- 172.21.2.101:0/561509511 shutdown_connections 2021-02-10T11:46:57.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.501 7f893e23a700 1 -- 172.21.2.101:0/561509511 wait complete. 2021-02-10T11:46:57.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.515 7f893e23a700 1 Processor -- start 2021-02-10T11:46:57.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.515 7f893e23a700 1 -- start start 2021-02-10T11:46:57.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.516 7f893e23a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f893809a690 0x7f893809aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:57.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.516 7f893e23a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89381b4160 0x7f89381b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:57.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.516 7f893e23a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89381b7dd0 0x7f89381b81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:57.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.516 7f893e23a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89381b4b80 con 0x7f893809a690 2021-02-10T11:46:57.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.516 7f893e23a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89381b87f0 con 0x7f89381b7dd0 2021-02-10T11:46:57.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.516 7f893e23a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89381b8a10 con 0x7f89381b4160 2021-02-10T11:46:57.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.516 7f89377fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89381b4160 0x7f89381b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:57.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.516 7f8937fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f893809a690 0x7f893809aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:57.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.516 7f8937fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f893809a690 0x7f893809aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33642/0 (socket says 172.21.2.101:33642) 2021-02-10T11:46:57.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.516 7f8937fff700 1 -- 172.21.2.101:0/326119809 learned_addr learned my addr 172.21.2.101:0/326119809 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:57.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.516 7f893c857700 1 --2- 172.21.2.101:0/326119809 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89381b7dd0 0x7f89381b81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:57.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.516 7f8937fff700 1 -- 172.21.2.101:0/326119809 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89381b4160 msgr2=0x7f89381b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:57.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.516 7f8937fff700 1 --2- 172.21.2.101:0/326119809 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89381b4160 0x7f89381b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:57.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.516 7f8937fff700 1 -- 172.21.2.101:0/326119809 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89381b7dd0 msgr2=0x7f89381b81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:57.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.516 7f8937fff700 1 --2- 172.21.2.101:0/326119809 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89381b7dd0 0x7f89381b81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:57.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.516 7f8937fff700 1 -- 172.21.2.101:0/326119809 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89381b8c30 con 0x7f893809a690 2021-02-10T11:46:57.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.516 7f8937fff700 1 --2- 172.21.2.101:0/326119809 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f893809a690 0x7f893809aa90 secure :-1 s=READY pgs=3025 cs=0 l=1 rev1=1 rx=0x7f8928006f40 tx=0x7f892800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:57.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.517 7f8934ff9700 1 -- 172.21.2.101:0/326119809 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f89280009b0 con 0x7f893809a690 2021-02-10T11:46:57.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.517 7f8934ff9700 1 -- 172.21.2.101:0/326119809 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892800bd80 con 0x7f893809a690 2021-02-10T11:46:57.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.517 7f8934ff9700 1 -- 172.21.2.101:0/326119809 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f89280022b0 con 0x7f893809a690 2021-02-10T11:46:57.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.517 7f893e23a700 1 -- 172.21.2.101:0/326119809 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89381b8c30 con 0x7f893809a690 2021-02-10T11:46:57.517 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.517 7f893e23a700 1 -- 172.21.2.101:0/326119809 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89381b91c0 con 0x7f893809a690 2021-02-10T11:46:57.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.517 7f8934ff9700 1 -- 172.21.2.101:0/326119809 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8928000c90 con 0x7f893809a690 2021-02-10T11:46:57.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.517 7f8934ff9700 1 --2- 172.21.2.101:0/326119809 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f89180186c0 0x7f891801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:57.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.517 7f8934ff9700 1 -- 172.21.2.101:0/326119809 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f89280325c0 con 0x7f893809a690 2021-02-10T11:46:57.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.517 7f89377fe700 1 --2- 172.21.2.101:0/326119809 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f89180186c0 0x7f891801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:57.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.518 7f89377fe700 1 --2- 172.21.2.101:0/326119809 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f89180186c0 0x7f891801abc0 crc :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:57.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.549 7f893e23a700 1 -- 172.21.2.101:0/326119809 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893809a160 con 0x7f893809a690 2021-02-10T11:46:57.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.552 7f8934ff9700 1 -- 172.21.2.101:0/326119809 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8928006c10 con 0x7f893809a690 2021-02-10T11:46:57.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.686 7f893e23a700 1 -- 172.21.2.101:0/326119809 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f893809a160 con 0x7f893809a690 2021-02-10T11:46:57.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.741 7f8934ff9700 1 -- 172.21.2.101:0/326119809 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v149) v1 ==== 136+0+0 (secure 0 0 0) 0x7f8928000c90 con 0x7f893809a690 2021-02-10T11:46:57.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.752 7f893e23a700 1 -- 172.21.2.101:0/326119809 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f89180186c0 msgr2=0x7f891801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:57.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.752 7f893e23a700 1 --2- 172.21.2.101:0/326119809 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f89180186c0 0x7f891801abc0 crc :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:57.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.752 7f893e23a700 1 -- 172.21.2.101:0/326119809 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f893809a690 msgr2=0x7f893809aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:57.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.752 7f893e23a700 1 --2- 172.21.2.101:0/326119809 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f893809a690 0x7f893809aa90 secure :-1 s=READY pgs=3025 cs=0 l=1 rev1=1 rx=0x7f8928006f40 tx=0x7f892800b990).stop 2021-02-10T11:46:57.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.752 7f893e23a700 1 -- 172.21.2.101:0/326119809 shutdown_connections 2021-02-10T11:46:57.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.752 7f893e23a700 1 --2- 172.21.2.101:0/326119809 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f89180186c0 0x7f891801abc0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:57.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.752 7f893e23a700 1 --2- 172.21.2.101:0/326119809 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f89381b4160 0x7f89381b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:57.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.752 7f893e23a700 1 --2- 172.21.2.101:0/326119809 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f89381b7dd0 0x7f89381b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:57.750 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.752 7f893e23a700 1 --2- 172.21.2.101:0/326119809 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f893809a690 0x7f893809aa90 unknown :-1 s=CLOSED pgs=3025 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:57.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.752 7f893e23a700 1 -- 172.21.2.101:0/326119809 >> 172.21.2.101:0/326119809 conn(0x7f893818e870 msgr2=0x7f893818ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:57.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.753 7f893e23a700 1 -- 172.21.2.101:0/326119809 shutdown_connections 2021-02-10T11:46:57.751 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:57.753 7f893e23a700 1 -- 172.21.2.101:0/326119809 wait complete. 2021-02-10T11:46:57.758 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs volume ls 2021-02-10T11:46:58.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.172 7f1b96519700 1 Processor -- start 2021-02-10T11:46:58.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.172 7f1b96519700 1 -- start start 2021-02-10T11:46:58.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.172 7f1b96519700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1b900d1870 0x7f1b901b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:58.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.172 7f1b96519700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b900f0650 0x7f1b900cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:58.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.172 7f1b96519700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b900f0b10 con 0x7f1b901b0a10 2021-02-10T11:46:58.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.172 7f1b96519700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b900e5ed0 con 0x7f1b900d1870 2021-02-10T11:46:58.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.172 7f1b96519700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900d1c70 con 0x7f1b900f0650 2021-02-10T11:46:58.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.173 7f1b8ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1b900d1870 0x7f1b901b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:58.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.173 7f1b8f7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f1b901b0a10 0x7f1b900cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34142/0 (socket says 172.21.2.101:34142) 2021-02-10T11:46:58.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.173 7f1b8f7fe700 1 -- 172.21.2.101:0/2051662264 learned_addr learned my addr 172.21.2.101:0/2051662264 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:58.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.173 7f1b8ffff700 1 -- 172.21.2.101:0/2051662264 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b900f0650 msgr2=0x7f1b900cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:46:58.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.173 7f1b8ffff700 1 --2- 172.21.2.101:0/2051662264 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b900f0650 0x7f1b900cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:58.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.173 7f1b8ffff700 1 -- 172.21.2.101:0/2051662264 >> v1:172.21.2.101:6789/0 conn(0x7f1b901b0a10 legacy=0x7f1b900cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:58.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.173 7f1b8ffff700 1 -- 172.21.2.101:0/2051662264 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b900d1e90 con 0x7f1b900d1870 2021-02-10T11:46:58.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.173 7f1b8ffff700 1 --2- 172.21.2.101:0/2051662264 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b900d1870 0x7f1b901b03f0 secure :-1 s=READY pgs=3027 cs=0 l=1 rev1=1 rx=0x7f1b80006650 tx=0x7f1b80009a80).ready entity=mon.0 client_cookie=171c77d0bf8aeec6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:58.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.173 7f1b8effd700 1 -- 172.21.2.101:0/2051662264 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1b800009b0 con 0x7f1b900d1870 2021-02-10T11:46:58.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.174 7f1b8effd700 1 -- 172.21.2.101:0/2051662264 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8000c0c0 con 0x7f1b900d1870 2021-02-10T11:46:58.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.174 7f1b8effd700 1 -- 172.21.2.101:0/2051662264 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1b8000c5e0 con 0x7f1b900d1870 2021-02-10T11:46:58.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.174 7f1b96519700 1 -- 172.21.2.101:0/2051662264 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b900d1870 msgr2=0x7f1b901b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:58.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.174 7f1b96519700 1 --2- 172.21.2.101:0/2051662264 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b900d1870 0x7f1b901b03f0 secure :-1 s=READY pgs=3027 cs=0 l=1 rev1=1 rx=0x7f1b80006650 tx=0x7f1b80009a80).stop 2021-02-10T11:46:58.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.174 7f1b96519700 1 -- 172.21.2.101:0/2051662264 shutdown_connections 2021-02-10T11:46:58.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.174 7f1b96519700 1 --2- 172.21.2.101:0/2051662264 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b900f0650 0x7f1b900cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:58.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.174 7f1b96519700 1 --2- 172.21.2.101:0/2051662264 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b900d1870 0x7f1b901b03f0 unknown :-1 s=CLOSED pgs=3027 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:58.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.174 7f1b96519700 1 -- 172.21.2.101:0/2051662264 >> 172.21.2.101:0/2051662264 conn(0x7f1b900ce450 msgr2=0x7f1b900d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:58.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.174 7f1b96519700 1 -- 172.21.2.101:0/2051662264 shutdown_connections 2021-02-10T11:46:58.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.174 7f1b96519700 1 -- 172.21.2.101:0/2051662264 wait complete. 2021-02-10T11:46:58.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.189 7f1b96519700 1 Processor -- start 2021-02-10T11:46:58.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.189 7f1b96519700 1 -- start start 2021-02-10T11:46:58.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.189 7f1b96519700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1b90095040 0x7f1b90193320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:58.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.189 7f1b96519700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b90095520 0x7f1b900bfa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:58.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.189 7f1b96519700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b900c00b0 0x7f1b900eec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:58.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.189 7f1b96519700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900bdd10 con 0x7f1b900c00b0 2021-02-10T11:46:58.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.189 7f1b96519700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900c04b0 con 0x7f1b90095040 2021-02-10T11:46:58.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.189 7f1b96519700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900ef290 con 0x7f1b90095520 2021-02-10T11:46:58.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.190 7f1b94b36700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b900c00b0 0x7f1b900eec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:58.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.190 7f1b94b36700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b900c00b0 0x7f1b900eec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33656/0 (socket says 172.21.2.101:33656) 2021-02-10T11:46:58.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.190 7f1b94b36700 1 -- 172.21.2.101:0/1055443960 learned_addr learned my addr 172.21.2.101:0/1055443960 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:58.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.190 7f1b94b36700 1 -- 172.21.2.101:0/1055443960 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b90095520 msgr2=0x7f1b900bfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:58.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.190 7f1b94b36700 1 --2- 172.21.2.101:0/1055443960 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b90095520 0x7f1b900bfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:58.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.190 7f1b94b36700 1 -- 172.21.2.101:0/1055443960 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1b90095040 msgr2=0x7f1b90193320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:58.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.190 7f1b94b36700 1 --2- 172.21.2.101:0/1055443960 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1b90095040 0x7f1b90193320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:58.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.190 7f1b94b36700 1 -- 172.21.2.101:0/1055443960 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b900ef4b0 con 0x7f1b900c00b0 2021-02-10T11:46:58.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.190 7f1b94b36700 1 --2- 172.21.2.101:0/1055443960 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b900c00b0 0x7f1b900eec70 secure :-1 s=READY pgs=3028 cs=0 l=1 rev1=1 rx=0x7f1b88009980 tx=0x7f1b8800bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:58.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.191 7f1b8cff9700 1 -- 172.21.2.101:0/1055443960 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1b88005870 con 0x7f1b900c00b0 2021-02-10T11:46:58.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.191 7f1b8cff9700 1 -- 172.21.2.101:0/1055443960 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b88005b30 con 0x7f1b900c00b0 2021-02-10T11:46:58.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.191 7f1b96519700 1 -- 172.21.2.101:0/1055443960 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b900ef4b0 con 0x7f1b900c00b0 2021-02-10T11:46:58.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.191 7f1b8cff9700 1 -- 172.21.2.101:0/1055443960 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1b88006030 con 0x7f1b900c00b0 2021-02-10T11:46:58.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.191 7f1b96519700 1 -- 172.21.2.101:0/1055443960 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b900ef4b0 con 0x7f1b900c00b0 2021-02-10T11:46:58.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.191 7f1b8cff9700 1 -- 172.21.2.101:0/1055443960 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1b88006030 con 0x7f1b900c00b0 2021-02-10T11:46:58.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.191 7f1b8cff9700 1 --2- 172.21.2.101:0/1055443960 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b700186d0 0x7f1b7001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:58.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.191 7f1b8cff9700 1 -- 172.21.2.101:0/1055443960 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1b88032b70 con 0x7f1b900c00b0 2021-02-10T11:46:58.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.191 7f1b8ffff700 1 --2- 172.21.2.101:0/1055443960 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b700186d0 0x7f1b7001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:58.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.192 7f1b8ffff700 1 --2- 172.21.2.101:0/1055443960 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b700186d0 0x7f1b7001abd0 crc :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:58.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.222 7f1b96519700 1 -- 172.21.2.101:0/1055443960 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b900efa40 con 0x7f1b900c00b0 2021-02-10T11:46:58.222 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.225 7f1b8cff9700 1 -- 172.21.2.101:0/1055443960 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1b88005b30 con 0x7f1b900c00b0 2021-02-10T11:46:58.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.359 7f1b96519700 1 -- 172.21.2.101:0/1055443960 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs volume ls", "target": ["mgr", ""]}) v1 -- 0x7f1b900efa40 con 0x7f1b700186d0 2021-02-10T11:46:58.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.360 7f1b8cff9700 1 -- 172.21.2.101:0/1055443960 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7f1b80006040 con 0x7f1b700186d0 2021-02-10T11:46:58.358 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:46:58.358 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:46:58.358 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "cephfs" 2021-02-10T11:46:58.358 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:46:58.358 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:46:58.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.362 7f1b96519700 1 -- 172.21.2.101:0/1055443960 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b700186d0 msgr2=0x7f1b7001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:58.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.362 7f1b96519700 1 --2- 172.21.2.101:0/1055443960 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b700186d0 0x7f1b7001abd0 crc :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:58.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.362 7f1b96519700 1 -- 172.21.2.101:0/1055443960 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b900c00b0 msgr2=0x7f1b900eec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:58.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.362 7f1b96519700 1 --2- 172.21.2.101:0/1055443960 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b900c00b0 0x7f1b900eec70 secure :-1 s=READY pgs=3028 cs=0 l=1 rev1=1 rx=0x7f1b88009980 tx=0x7f1b8800bcd0).stop 2021-02-10T11:46:58.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.362 7f1b96519700 1 -- 172.21.2.101:0/1055443960 shutdown_connections 2021-02-10T11:46:58.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.362 7f1b96519700 1 --2- 172.21.2.101:0/1055443960 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1b700186d0 0x7f1b7001abd0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:58.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.362 7f1b96519700 1 --2- 172.21.2.101:0/1055443960 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1b90095520 0x7f1b900bfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:58.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.362 7f1b96519700 1 --2- 172.21.2.101:0/1055443960 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1b900c00b0 0x7f1b900eec70 unknown :-1 s=CLOSED pgs=3028 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:58.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.362 7f1b96519700 1 --2- 172.21.2.101:0/1055443960 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1b90095040 0x7f1b90193320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:58.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.362 7f1b96519700 1 -- 172.21.2.101:0/1055443960 >> 172.21.2.101:0/1055443960 conn(0x7f1b900efc90 msgr2=0x7f1b900bf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:58.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.362 7f1b96519700 1 -- 172.21.2.101:0/1055443960 shutdown_connections 2021-02-10T11:46:58.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.362 7f1b96519700 1 -- 172.21.2.101:0/1055443960 wait complete. 2021-02-10T11:46:58.368 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:46:58.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.790 7fb0cb05e700 1 Processor -- start 2021-02-10T11:46:58.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.790 7fb0cb05e700 1 -- start start 2021-02-10T11:46:58.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.790 7fb0cb05e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb0c410aff0 0x7fb0c40cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:58.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.790 7fb0cb05e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c40fe1c0 con 0x7fb0c40d1870 2021-02-10T11:46:58.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.790 7fb0cb05e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c41b0cb0 con 0x7fb0c40d1c70 2021-02-10T11:46:58.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.790 7fb0cb05e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c4109a60 con 0x7fb0c410aff0 2021-02-10T11:46:58.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.790 7fb0c967b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb0c410aff0 0x7fb0c40cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:58.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.790 7fb0c967b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb0c410aff0 0x7fb0c40cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33660/0 (socket says 172.21.2.101:33660) 2021-02-10T11:46:58.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.790 7fb0c8e7a700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb0c40d1870 0x7fb0c41b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34154/0 (socket says 172.21.2.101:34154) 2021-02-10T11:46:58.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.790 7fb0c8e7a700 1 -- 172.21.2.101:0/737669253 learned_addr learned my addr 172.21.2.101:0/737669253 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:46:58.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.791 7fb0c967b700 1 -- 172.21.2.101:0/737669253 >> v1:172.21.2.108:6789/0 conn(0x7fb0c40d1c70 legacy=0x7fb0c40cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:58.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.791 7fb0c967b700 1 -- 172.21.2.101:0/737669253 >> v1:172.21.2.101:6789/0 conn(0x7fb0c40d1870 legacy=0x7fb0c41b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:58.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.791 7fb0c967b700 1 -- 172.21.2.101:0/737669253 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c410b3f0 con 0x7fb0c410aff0 2021-02-10T11:46:58.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.791 7fb0c967b700 1 --2- 172.21.2.101:0/737669253 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0c410aff0 0x7fb0c40cf750 secure :-1 s=READY pgs=3030 cs=0 l=1 rev1=1 rx=0x7fb0bc006e30 tx=0x7fb0bc009e60).ready entity=mon.0 client_cookie=fc8da60f7cefc42c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:58.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.791 7fb0c37fe700 1 -- 172.21.2.101:0/737669253 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0bc006a50 con 0x7fb0c410aff0 2021-02-10T11:46:58.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.791 7fb0c37fe700 1 -- 172.21.2.101:0/737669253 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0bc00b300 con 0x7fb0c410aff0 2021-02-10T11:46:58.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.791 7fb0c37fe700 1 -- 172.21.2.101:0/737669253 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0bc00b7a0 con 0x7fb0c410aff0 2021-02-10T11:46:58.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.791 7fb0cb05e700 1 -- 172.21.2.101:0/737669253 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0c410aff0 msgr2=0x7fb0c40cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:58.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.791 7fb0cb05e700 1 --2- 172.21.2.101:0/737669253 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0c410aff0 0x7fb0c40cf750 secure :-1 s=READY pgs=3030 cs=0 l=1 rev1=1 rx=0x7fb0bc006e30 tx=0x7fb0bc009e60).stop 2021-02-10T11:46:58.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.792 7fb0cb05e700 1 -- 172.21.2.101:0/737669253 shutdown_connections 2021-02-10T11:46:58.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.792 7fb0cb05e700 1 --2- 172.21.2.101:0/737669253 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0c410aff0 0x7fb0c40cf750 unknown :-1 s=CLOSED pgs=3030 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:58.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.792 7fb0cb05e700 1 -- 172.21.2.101:0/737669253 >> 172.21.2.101:0/737669253 conn(0x7fb0c40ce420 msgr2=0x7fb0c40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:58.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.792 7fb0cb05e700 1 -- 172.21.2.101:0/737669253 shutdown_connections 2021-02-10T11:46:58.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.792 7fb0cb05e700 1 -- 172.21.2.101:0/737669253 wait complete. 2021-02-10T11:46:58.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.807 7fb0cb05e700 1 Processor -- start 2021-02-10T11:46:58.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.807 7fb0cb05e700 1 -- start start 2021-02-10T11:46:58.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.807 7fb0cb05e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0c40a0150 0x7fb0c40b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:58.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.807 7fb0cb05e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0c40a0550 0x7fb0c40b5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:58.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.807 7fb0cb05e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0c40b5f30 0x7fb0c4093270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:58.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.807 7fb0cb05e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c40b6330 con 0x7fb0c40a0550 2021-02-10T11:46:58.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.807 7fb0cb05e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c4093890 con 0x7fb0c40a0150 2021-02-10T11:46:58.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.807 7fb0cb05e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c4093ab0 con 0x7fb0c40b5f30 2021-02-10T11:46:58.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.807 7fb0c3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0c40a0550 0x7fb0c40b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:58.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.807 7fb0c3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0c40a0550 0x7fb0c40b5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33664/0 (socket says 172.21.2.101:33664) 2021-02-10T11:46:58.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.807 7fb0c3fff700 1 -- 172.21.2.101:0/2212726910 learned_addr learned my addr 172.21.2.101:0/2212726910 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:58.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.808 7fb0c967b700 1 --2- 172.21.2.101:0/2212726910 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0c40b5f30 0x7fb0c4093270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:58.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.808 7fb0c8e7a700 1 --2- 172.21.2.101:0/2212726910 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0c40a0150 0x7fb0c40b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:58.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.808 7fb0c3fff700 1 -- 172.21.2.101:0/2212726910 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0c40b5f30 msgr2=0x7fb0c4093270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:58.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.808 7fb0c3fff700 1 --2- 172.21.2.101:0/2212726910 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0c40b5f30 0x7fb0c4093270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:58.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.808 7fb0c3fff700 1 -- 172.21.2.101:0/2212726910 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0c40a0150 msgr2=0x7fb0c40b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:58.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.808 7fb0c3fff700 1 --2- 172.21.2.101:0/2212726910 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0c40a0150 0x7fb0c40b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:58.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.808 7fb0c3fff700 1 -- 172.21.2.101:0/2212726910 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c4093cd0 con 0x7fb0c40a0550 2021-02-10T11:46:58.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.808 7fb0c967b700 1 --2- 172.21.2.101:0/2212726910 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0c40b5f30 0x7fb0c4093270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:46:58.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.808 7fb0c8e7a700 1 --2- 172.21.2.101:0/2212726910 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0c40a0150 0x7fb0c40b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:46:58.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.808 7fb0c3fff700 1 --2- 172.21.2.101:0/2212726910 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0c40a0550 0x7fb0c40b5910 secure :-1 s=READY pgs=3031 cs=0 l=1 rev1=1 rx=0x7fb0b400a570 tx=0x7fb0b400a910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:58.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.808 7fb0c17fa700 1 -- 172.21.2.101:0/2212726910 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0b400ad00 con 0x7fb0c40a0550 2021-02-10T11:46:58.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.808 7fb0c17fa700 1 -- 172.21.2.101:0/2212726910 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b400c0d0 con 0x7fb0c40a0550 2021-02-10T11:46:58.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.808 7fb0c17fa700 1 -- 172.21.2.101:0/2212726910 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0b400c5b0 con 0x7fb0c40a0550 2021-02-10T11:46:58.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.808 7fb0cb05e700 1 -- 172.21.2.101:0/2212726910 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c4093cd0 con 0x7fb0c40a0550 2021-02-10T11:46:58.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.808 7fb0cb05e700 1 -- 172.21.2.101:0/2212726910 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c4094260 con 0x7fb0c40a0550 2021-02-10T11:46:58.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.809 7fb0c17fa700 1 -- 172.21.2.101:0/2212726910 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb0b4008030 con 0x7fb0c40a0550 2021-02-10T11:46:58.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.809 7fb0c17fa700 1 --2- 172.21.2.101:0/2212726910 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0a80186c0 0x7fb0a801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:58.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.809 7fb0c17fa700 1 -- 172.21.2.101:0/2212726910 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb0b4032560 con 0x7fb0c40a0550 2021-02-10T11:46:58.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.809 7fb0c8e7a700 1 --2- 172.21.2.101:0/2212726910 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0a80186c0 0x7fb0a801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:58.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.809 7fb0c8e7a700 1 --2- 172.21.2.101:0/2212726910 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0a80186c0 0x7fb0a801abc0 crc :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:58.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.840 7fb0cb05e700 1 -- 172.21.2.101:0/2212726910 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c4092d40 con 0x7fb0c40a0550 2021-02-10T11:46:58.840 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.843 7fb0c17fa700 1 -- 172.21.2.101:0/2212726910 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb0b4000d90 con 0x7fb0c40a0550 2021-02-10T11:46:58.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.994 7fb0cb05e700 1 -- 172.21.2.101:0/2212726910 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb0c4092d40 con 0x7fb0c40a0550 2021-02-10T11:46:58.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.994 7fb0c17fa700 1 -- 172.21.2.101:0/2212726910 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb0b4008030 con 0x7fb0c40a0550 2021-02-10T11:46:58.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.996 7fb0cb05e700 1 -- 172.21.2.101:0/2212726910 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0a80186c0 msgr2=0x7fb0a801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:58.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.996 7fb0cb05e700 1 --2- 172.21.2.101:0/2212726910 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0a80186c0 0x7fb0a801abc0 crc :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:58.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.996 7fb0cb05e700 1 -- 172.21.2.101:0/2212726910 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0c40a0550 msgr2=0x7fb0c40b5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:58.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.996 7fb0cb05e700 1 --2- 172.21.2.101:0/2212726910 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0c40a0550 0x7fb0c40b5910 secure :-1 s=READY pgs=3031 cs=0 l=1 rev1=1 rx=0x7fb0b400a570 tx=0x7fb0b400a910).stop 2021-02-10T11:46:58.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.996 7fb0cb05e700 1 -- 172.21.2.101:0/2212726910 shutdown_connections 2021-02-10T11:46:58.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.996 7fb0cb05e700 1 --2- 172.21.2.101:0/2212726910 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0a80186c0 0x7fb0a801abc0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:58.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.996 7fb0cb05e700 1 --2- 172.21.2.101:0/2212726910 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0c40b5f30 0x7fb0c4093270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:58.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.996 7fb0cb05e700 1 --2- 172.21.2.101:0/2212726910 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0c40a0550 0x7fb0c40b5910 unknown :-1 s=CLOSED pgs=3031 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:58.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.996 7fb0cb05e700 1 --2- 172.21.2.101:0/2212726910 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0c40a0150 0x7fb0c40b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:58.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.996 7fb0cb05e700 1 -- 172.21.2.101:0/2212726910 >> 172.21.2.101:0/2212726910 conn(0x7fb0c41084d0 msgr2=0x7fb0c40c0940 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:58.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.996 7fb0cb05e700 1 -- 172.21.2.101:0/2212726910 shutdown_connections 2021-02-10T11:46:58.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:58.997 7fb0cb05e700 1 -- 172.21.2.101:0/2212726910 wait complete. 2021-02-10T11:46:59.002 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolumegroup create cephfs group_0000000000357946 2021-02-10T11:46:59.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.421 7fb0bd47c700 1 Processor -- start 2021-02-10T11:46:59.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.421 7fb0bd47c700 1 -- start start 2021-02-10T11:46:59.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.421 7fb0bd47c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb0b80d1c70 0x7fb0b80cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:59.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.421 7fb0bd47c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb0b80cf580 0x7fb0b80cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:59.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.421 7fb0bd47c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0b80fe1c0 con 0x7fb0b80d1870 2021-02-10T11:46:59.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.421 7fb0bd47c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b81b0cb0 con 0x7fb0b80d1c70 2021-02-10T11:46:59.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.421 7fb0bd47c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b8109a60 con 0x7fb0b80cf580 2021-02-10T11:46:59.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.421 7fb0b687c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb0b80d1c70 0x7fb0b80cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:59.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.421 7fb0b687c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb0b80d1c70 0x7fb0b80cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33670/0 (socket says 172.21.2.101:33670) 2021-02-10T11:46:59.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.421 7fb0b687c700 1 -- 172.21.2.101:0/3704081274 learned_addr learned my addr 172.21.2.101:0/3704081274 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:59.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.422 7fb0b687c700 1 -- 172.21.2.101:0/3704081274 >> v2:172.21.2.108:3300/0 conn(0x7fb0b80cf580 msgr2=0x7fb0b80cf980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:46:59.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.422 7fb0b687c700 1 --2- 172.21.2.101:0/3704081274 >> v2:172.21.2.108:3300/0 conn(0x7fb0b80cf580 0x7fb0b80cf980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:59.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.422 7fb0b687c700 1 -- 172.21.2.101:0/3704081274 >> v1:172.21.2.108:6789/0 conn(0x7fb0b80d1870 legacy=0x7fb0b81b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:46:59.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.422 7fb0b687c700 1 -- 172.21.2.101:0/3704081274 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b8088070 con 0x7fb0b80d1c70 2021-02-10T11:46:59.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.422 7fb0b687c700 1 --2- 172.21.2.101:0/3704081274 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0b80d1c70 0x7fb0b80cef60 secure :-1 s=READY pgs=3032 cs=0 l=1 rev1=1 rx=0x7fb0a8006700 tx=0x7fb0a8009b80).ready entity=mon.0 client_cookie=4405c8141f0f5e27 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:59.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.422 7fb0b607b700 1 -- 172.21.2.101:0/3704081274 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0a800c0c0 con 0x7fb0b80d1c70 2021-02-10T11:46:59.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.422 7fb0b607b700 1 -- 172.21.2.101:0/3704081274 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a800c340 con 0x7fb0b80d1c70 2021-02-10T11:46:59.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.422 7fb0b607b700 1 -- 172.21.2.101:0/3704081274 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0a800c860 con 0x7fb0b80d1c70 2021-02-10T11:46:59.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.422 7fb0bd47c700 1 -- 172.21.2.101:0/3704081274 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0b80d1c70 msgr2=0x7fb0b80cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:59.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.422 7fb0bd47c700 1 --2- 172.21.2.101:0/3704081274 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0b80d1c70 0x7fb0b80cef60 secure :-1 s=READY pgs=3032 cs=0 l=1 rev1=1 rx=0x7fb0a8006700 tx=0x7fb0a8009b80).stop 2021-02-10T11:46:59.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.423 7fb0bd47c700 1 -- 172.21.2.101:0/3704081274 shutdown_connections 2021-02-10T11:46:59.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.423 7fb0bd47c700 1 --2- 172.21.2.101:0/3704081274 >> v2:172.21.2.108:3300/0 conn(0x7fb0b80cf580 0x7fb0b80cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:59.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.423 7fb0bd47c700 1 --2- 172.21.2.101:0/3704081274 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0b80d1c70 0x7fb0b80cef60 unknown :-1 s=CLOSED pgs=3032 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:59.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.423 7fb0bd47c700 1 -- 172.21.2.101:0/3704081274 >> 172.21.2.101:0/3704081274 conn(0x7fb0b80ce420 msgr2=0x7fb0b80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:59.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.423 7fb0bd47c700 1 -- 172.21.2.101:0/3704081274 shutdown_connections 2021-02-10T11:46:59.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.423 7fb0bd47c700 1 -- 172.21.2.101:0/3704081274 wait complete. 2021-02-10T11:46:59.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.437 7fb0bd47c700 1 Processor -- start 2021-02-10T11:46:59.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.437 7fb0bd47c700 1 -- start start 2021-02-10T11:46:59.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.438 7fb0bd47c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0b8096060 0x7fb0b808f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:59.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.438 7fb0bd47c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0b808fa20 0x7fb0b80a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:59.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.438 7fb0bd47c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0b8096460 0x7fb0b80a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:59.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.438 7fb0bd47c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b80a4ae0 con 0x7fb0b8096060 2021-02-10T11:46:59.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.438 7fb0bd47c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b808ece0 con 0x7fb0b808fa20 2021-02-10T11:46:59.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.438 7fb0bd47c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b80a44b0 con 0x7fb0b8096460 2021-02-10T11:46:59.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.438 7fb0b707d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0b8096060 0x7fb0b808f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:59.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.438 7fb0b707d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0b8096060 0x7fb0b808f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33676/0 (socket says 172.21.2.101:33676) 2021-02-10T11:46:59.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.438 7fb0b707d700 1 -- 172.21.2.101:0/3028156579 learned_addr learned my addr 172.21.2.101:0/3028156579 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:46:59.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.438 7fb0b707d700 1 -- 172.21.2.101:0/3028156579 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0b8096460 msgr2=0x7fb0b80a5060 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:59.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.438 7fb0b707d700 1 --2- 172.21.2.101:0/3028156579 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0b8096460 0x7fb0b80a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:59.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.438 7fb0b707d700 1 -- 172.21.2.101:0/3028156579 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0b808fa20 msgr2=0x7fb0b80a3e90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:46:59.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.438 7fb0b707d700 1 --2- 172.21.2.101:0/3028156579 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0b808fa20 0x7fb0b80a3e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:59.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.438 7fb0b707d700 1 -- 172.21.2.101:0/3028156579 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b80a46d0 con 0x7fb0b8096060 2021-02-10T11:46:59.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.439 7fb0b707d700 1 --2- 172.21.2.101:0/3028156579 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0b8096060 0x7fb0b808f400 secure :-1 s=READY pgs=3033 cs=0 l=1 rev1=1 rx=0x7fb0a000b5b0 tx=0x7fb0a000b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:59.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.439 7fb0affff700 1 -- 172.21.2.101:0/3028156579 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0a000bd40 con 0x7fb0b8096060 2021-02-10T11:46:59.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.439 7fb0affff700 1 -- 172.21.2.101:0/3028156579 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a0006240 con 0x7fb0b8096060 2021-02-10T11:46:59.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.439 7fb0affff700 1 -- 172.21.2.101:0/3028156579 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0a0005720 con 0x7fb0b8096060 2021-02-10T11:46:59.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.439 7fb0bd47c700 1 -- 172.21.2.101:0/3028156579 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b80a46d0 con 0x7fb0b8096060 2021-02-10T11:46:59.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.439 7fb0bd47c700 1 -- 172.21.2.101:0/3028156579 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b80a5970 con 0x7fb0b8096060 2021-02-10T11:46:59.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.439 7fb0affff700 1 -- 172.21.2.101:0/3028156579 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb0a00054a0 con 0x7fb0b8096060 2021-02-10T11:46:59.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.439 7fb0affff700 1 --2- 172.21.2.101:0/3028156579 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb09c0186c0 0x7fb09c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:46:59.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.439 7fb0affff700 1 -- 172.21.2.101:0/3028156579 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb0a00325a0 con 0x7fb0b8096060 2021-02-10T11:46:59.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.440 7fb0b687c700 1 --2- 172.21.2.101:0/3028156579 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb09c0186c0 0x7fb09c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:46:59.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.440 7fb0b687c700 1 --2- 172.21.2.101:0/3028156579 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb09c0186c0 0x7fb09c01abc0 crc :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:46:59.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.471 7fb0bd47c700 1 -- 172.21.2.101:0/3028156579 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b80fe1c0 con 0x7fb0b8096060 2021-02-10T11:46:59.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.474 7fb0affff700 1 -- 172.21.2.101:0/3028156579 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb0a00009b0 con 0x7fb0b8096060 2021-02-10T11:46:59.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.604 7fb0bd47c700 1 -- 172.21.2.101:0/3028156579 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "target": ["mgr", ""], "group_name": "group_0000000000357946"}) v1 -- 0x7fb0b80a5970 con 0x7fb09c0186c0 2021-02-10T11:46:59.605 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:46:59.608 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:46:59.605 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:46:59.608 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:46:59.605 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:46:59.608 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:46:59.605 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:46:59.608 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:46:59.605 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:46:59.608 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:46:59.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.622 7fb0affff700 1 -- 172.21.2.101:0/3028156579 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fb0a80095d0 con 0x7fb09c0186c0 2021-02-10T11:46:59.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.637 7fb0bd47c700 1 -- 172.21.2.101:0/3028156579 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb09c0186c0 msgr2=0x7fb09c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:59.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.637 7fb0bd47c700 1 --2- 172.21.2.101:0/3028156579 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb09c0186c0 0x7fb09c01abc0 crc :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:59.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.637 7fb0bd47c700 1 -- 172.21.2.101:0/3028156579 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0b8096060 msgr2=0x7fb0b808f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:46:59.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.637 7fb0bd47c700 1 --2- 172.21.2.101:0/3028156579 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0b8096060 0x7fb0b808f400 secure :-1 s=READY pgs=3033 cs=0 l=1 rev1=1 rx=0x7fb0a000b5b0 tx=0x7fb0a000b950).stop 2021-02-10T11:46:59.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.638 7fb0bd47c700 1 -- 172.21.2.101:0/3028156579 shutdown_connections 2021-02-10T11:46:59.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.638 7fb0bd47c700 1 --2- 172.21.2.101:0/3028156579 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb09c0186c0 0x7fb09c01abc0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:59.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.638 7fb0bd47c700 1 --2- 172.21.2.101:0/3028156579 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0b8096460 0x7fb0b80a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:59.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.638 7fb0bd47c700 1 --2- 172.21.2.101:0/3028156579 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0b808fa20 0x7fb0b80a3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:46:59.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.638 7fb0bd47c700 1 --2- 172.21.2.101:0/3028156579 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0b8096060 0x7fb0b808f400 unknown :-1 s=CLOSED pgs=3033 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:46:59.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.638 7fb0bd47c700 1 -- 172.21.2.101:0/3028156579 >> 172.21.2.101:0/3028156579 conn(0x7fb0b80bd1c0 msgr2=0x7fb0b8096b90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:46:59.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.638 7fb0bd47c700 1 -- 172.21.2.101:0/3028156579 shutdown_connections 2021-02-10T11:46:59.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:46:59.638 7fb0bd47c700 1 -- 172.21.2.101:0/3028156579 wait complete. 2021-02-10T11:46:59.643 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000776063 --group_name group_0000000000357946 2021-02-10T11:47:00.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.061 7f172de23700 1 Processor -- start 2021-02-10T11:47:00.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.061 7f172de23700 1 -- start start 2021-02-10T11:47:00.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.062 7f172de23700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17280d1880 0x7f17281b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:00.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.062 7f172de23700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f17280cff80 0x7f1728109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:00.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.062 7f172de23700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1728109a70 con 0x7f17281b0af0 2021-02-10T11:47:00.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.062 7f172de23700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1728109760 con 0x7f17280cff80 2021-02-10T11:47:00.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.062 7f172de23700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1728088070 con 0x7f17280d1880 2021-02-10T11:47:00.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.062 7f17277fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17280d1880 0x7f17281b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:00.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.062 7f1727fff700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f17280cff80 0x7f1728109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:00.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.062 7f17277fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17280d1880 0x7f17281b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:54660/0 (socket says 172.21.2.101:54660) 2021-02-10T11:47:00.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.062 7f1726ffd700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f17281b0af0 0x7f1728108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:55832/0 (socket says 172.21.2.101:55832) 2021-02-10T11:47:00.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.062 7f17277fe700 1 -- 172.21.2.101:0/2790274448 learned_addr learned my addr 172.21.2.101:0/2790274448 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:00.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.063 7f17277fe700 1 -- 172.21.2.101:0/2790274448 >> v2:172.21.2.108:3300/0 conn(0x7f17280cff80 msgr2=0x7f1728109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:00.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.063 7f17277fe700 1 --2- 172.21.2.101:0/2790274448 >> v2:172.21.2.108:3300/0 conn(0x7f17280cff80 0x7f1728109140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:00.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.063 7f17277fe700 1 -- 172.21.2.101:0/2790274448 >> v1:172.21.2.108:6789/0 conn(0x7f17281b0af0 legacy=0x7f1728108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:00.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.063 7f17277fe700 1 -- 172.21.2.101:0/2790274448 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1728088260 con 0x7f17280d1880 2021-02-10T11:47:00.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.063 7f1727fff700 1 --2- 172.21.2.101:0/2790274448 >> v2:172.21.2.108:3300/0 conn(0x7f17280cff80 0x7f1728109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:47:00.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.063 7f17277fe700 1 --2- 172.21.2.101:0/2790274448 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17280d1880 0x7f17281b04d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f1710006700 tx=0x7f1710009b80).ready entity=mon.2 client_cookie=52b9c366f8b18c3a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:00.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.063 7f17267fc700 1 -- 172.21.2.101:0/2790274448 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f171000c0c0 con 0x7f17280d1880 2021-02-10T11:47:00.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.063 7f17267fc700 1 -- 172.21.2.101:0/2790274448 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171000c340 con 0x7f17280d1880 2021-02-10T11:47:00.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.063 7f17267fc700 1 -- 172.21.2.101:0/2790274448 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f17100009b0 con 0x7f17280d1880 2021-02-10T11:47:00.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.063 7f172de23700 1 -- 172.21.2.101:0/2790274448 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17280d1880 msgr2=0x7f17281b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:00.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.063 7f172de23700 1 --2- 172.21.2.101:0/2790274448 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17280d1880 0x7f17281b04d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f1710006700 tx=0x7f1710009b80).stop 2021-02-10T11:47:00.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.064 7f172de23700 1 -- 172.21.2.101:0/2790274448 shutdown_connections 2021-02-10T11:47:00.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.064 7f172de23700 1 --2- 172.21.2.101:0/2790274448 >> v2:172.21.2.108:3300/0 conn(0x7f17280cff80 0x7f1728109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:00.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.064 7f172de23700 1 --2- 172.21.2.101:0/2790274448 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f17280d1880 0x7f17281b04d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:00.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.064 7f172de23700 1 -- 172.21.2.101:0/2790274448 >> 172.21.2.101:0/2790274448 conn(0x7f17280ce430 msgr2=0x7f17280d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:00.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.064 7f172de23700 1 -- 172.21.2.101:0/2790274448 shutdown_connections 2021-02-10T11:47:00.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.064 7f172de23700 1 -- 172.21.2.101:0/2790274448 wait complete. 2021-02-10T11:47:00.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.079 7f172de23700 1 Processor -- start 2021-02-10T11:47:00.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.079 7f172de23700 1 -- start start 2021-02-10T11:47:00.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.079 7f172de23700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f17280b1f10 0x7f17280b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:00.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.079 7f172de23700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1728083160 0x7f1728083560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:00.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.079 7f172de23700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1728083b80 0x7f1728083f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:00.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.079 7f172de23700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172809a820 con 0x7f17280b1f10 2021-02-10T11:47:00.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.079 7f172de23700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172809acc0 con 0x7f1728083b80 2021-02-10T11:47:00.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.079 7f172de23700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17281af2d0 con 0x7f1728083160 2021-02-10T11:47:00.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.080 7f17277fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f17280b1f10 0x7f17280b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:00.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.080 7f17277fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f17280b1f10 0x7f17280b2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33704/0 (socket says 172.21.2.101:33704) 2021-02-10T11:47:00.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.080 7f17277fe700 1 -- 172.21.2.101:0/1535738528 learned_addr learned my addr 172.21.2.101:0/1535738528 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:00.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.080 7f1726ffd700 1 --2- 172.21.2.101:0/1535738528 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1728083160 0x7f1728083560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:00.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.080 7f1727fff700 1 --2- 172.21.2.101:0/1535738528 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1728083b80 0x7f1728083f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:00.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.080 7f17277fe700 1 -- 172.21.2.101:0/1535738528 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1728083160 msgr2=0x7f1728083560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:00.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.080 7f17277fe700 1 --2- 172.21.2.101:0/1535738528 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1728083160 0x7f1728083560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:00.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.080 7f17277fe700 1 -- 172.21.2.101:0/1535738528 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1728083b80 msgr2=0x7f1728083f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:00.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.080 7f17277fe700 1 --2- 172.21.2.101:0/1535738528 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1728083b80 0x7f1728083f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:00.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.080 7f17277fe700 1 -- 172.21.2.101:0/1535738528 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17281af4f0 con 0x7f17280b1f10 2021-02-10T11:47:00.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.080 7f1727fff700 1 --2- 172.21.2.101:0/1535738528 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1728083b80 0x7f1728083f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:47:00.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.080 7f17277fe700 1 --2- 172.21.2.101:0/1535738528 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f17280b1f10 0x7f17280b2310 secure :-1 s=READY pgs=3036 cs=0 l=1 rev1=1 rx=0x7f17100066a0 tx=0x7f171000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:00.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.080 7f171ffff700 1 -- 172.21.2.101:0/1535738528 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f171000bd80 con 0x7f17280b1f10 2021-02-10T11:47:00.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.080 7f171ffff700 1 -- 172.21.2.101:0/1535738528 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17100052e0 con 0x7f17280b1f10 2021-02-10T11:47:00.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.080 7f171ffff700 1 -- 172.21.2.101:0/1535738528 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1710005800 con 0x7f17280b1f10 2021-02-10T11:47:00.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.080 7f172de23700 1 -- 172.21.2.101:0/1535738528 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17281af4f0 con 0x7f17280b1f10 2021-02-10T11:47:00.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.080 7f172de23700 1 -- 172.21.2.101:0/1535738528 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17281af4f0 con 0x7f17280b1f10 2021-02-10T11:47:00.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.081 7f171ffff700 1 -- 172.21.2.101:0/1535738528 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1710006030 con 0x7f17280b1f10 2021-02-10T11:47:00.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.081 7f171ffff700 1 --2- 172.21.2.101:0/1535738528 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f170c018730 0x7f170c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:00.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.081 7f171ffff700 1 -- 172.21.2.101:0/1535738528 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1710032690 con 0x7f17280b1f10 2021-02-10T11:47:00.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.081 7f1726ffd700 1 --2- 172.21.2.101:0/1535738528 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f170c018730 0x7f170c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:00.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.081 7f1726ffd700 1 --2- 172.21.2.101:0/1535738528 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f170c018730 0x7f170c01ac10 crc :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:00.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.111 7f172de23700 1 -- 172.21.2.101:0/1535738528 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f172809a820 con 0x7f17280b1f10 2021-02-10T11:47:00.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.114 7f171ffff700 1 -- 172.21.2.101:0/1535738528 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f171001f550 con 0x7f17280b1f10 2021-02-10T11:47:00.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.240 7f172de23700 1 -- 172.21.2.101:0/1535738528 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000776063", "target": ["mgr", ""], "group_name": "group_0000000000357946"}) v1 -- 0x7f172809a820 con 0x7f170c018730 2021-02-10T11:47:00.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.265 7f171ffff700 1 -- 172.21.2.101:0/1535738528 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f17180010b0 con 0x7f170c018730 2021-02-10T11:47:00.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.273 7f172de23700 1 -- 172.21.2.101:0/1535738528 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f170c018730 msgr2=0x7f170c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:00.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.273 7f172de23700 1 --2- 172.21.2.101:0/1535738528 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f170c018730 0x7f170c01ac10 crc :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:00.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.273 7f172de23700 1 -- 172.21.2.101:0/1535738528 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f17280b1f10 msgr2=0x7f17280b2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:00.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.273 7f172de23700 1 --2- 172.21.2.101:0/1535738528 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f17280b1f10 0x7f17280b2310 secure :-1 s=READY pgs=3036 cs=0 l=1 rev1=1 rx=0x7f17100066a0 tx=0x7f171000b990).stop 2021-02-10T11:47:00.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.273 7f172de23700 1 -- 172.21.2.101:0/1535738528 shutdown_connections 2021-02-10T11:47:00.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.273 7f172de23700 1 --2- 172.21.2.101:0/1535738528 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f170c018730 0x7f170c01ac10 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:00.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.273 7f172de23700 1 --2- 172.21.2.101:0/1535738528 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1728083160 0x7f1728083560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:00.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.273 7f172de23700 1 --2- 172.21.2.101:0/1535738528 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1728083b80 0x7f1728083f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:00.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.273 7f172de23700 1 --2- 172.21.2.101:0/1535738528 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f17280b1f10 0x7f17280b2310 unknown :-1 s=CLOSED pgs=3036 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:00.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.273 7f172de23700 1 -- 172.21.2.101:0/1535738528 >> 172.21.2.101:0/1535738528 conn(0x7f1728087390 msgr2=0x7f1728090c90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:00.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.274 7f172de23700 1 -- 172.21.2.101:0/1535738528 shutdown_connections 2021-02-10T11:47:00.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.274 7f172de23700 1 -- 172.21.2.101:0/1535738528 wait complete. 2021-02-10T11:47:00.279 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume authorize cephfs subvolume_0000000000776063 guest1 --group_name group_0000000000357946 --tenant_id tenant1 2021-02-10T11:47:00.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.709 7f19689f7700 1 Processor -- start 2021-02-10T11:47:00.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.710 7f19689f7700 1 -- start start 2021-02-10T11:47:00.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.710 7f19689f7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f196410a040 0x7f196410a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:00.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.710 7f19689f7700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19641b0d40 con 0x7f19640e53a0 2021-02-10T11:47:00.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.710 7f19689f7700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1964109ac0 con 0x7f196410aa60 2021-02-10T11:47:00.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.710 7f19689f7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19640e5f80 con 0x7f196410a040 2021-02-10T11:47:00.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.710 7f196261c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f19640e53a0 0x7f19641b0550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34204/0 (socket says 172.21.2.101:34204) 2021-02-10T11:47:00.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.710 7f196261c700 1 -- 172.21.2.101:0/33483150 learned_addr learned my addr 172.21.2.101:0/33483150 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:00.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.711 7f196161a700 1 -- 172.21.2.101:0/33483150 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929091496 0 0) 0x7f19540010b0 con 0x7f19640e53a0 2021-02-10T11:47:00.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.711 7f196161a700 1 -- 172.21.2.101:0/33483150 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1950002fe0 con 0x7f19640e53a0 2021-02-10T11:47:00.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.711 7f1961e1b700 1 --2- 172.21.2.101:0/33483150 >> v2:172.21.2.108:3300/0 conn(0x7f196410a040 0x7f196410a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:00.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.711 7f196161a700 1 -- 172.21.2.101:0/33483150 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1491083011 0 0) 0x7f19540010b0 con 0x7f19640e53a0 2021-02-10T11:47:00.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.711 7f196161a700 1 -- 172.21.2.101:0/33483150 >> v2:172.21.2.108:3300/0 conn(0x7f196410a040 msgr2=0x7f196410a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:00.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.711 7f196161a700 1 --2- 172.21.2.101:0/33483150 >> v2:172.21.2.108:3300/0 conn(0x7f196410a040 0x7f196410a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:00.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.711 7f196161a700 1 -- 172.21.2.101:0/33483150 >> v1:172.21.2.108:6789/0 conn(0x7f196410aa60 legacy=0x7f196410ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:00.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.711 7f196161a700 1 -- 172.21.2.101:0/33483150 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19640e57a0 con 0x7f19640e53a0 2021-02-10T11:47:00.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.711 7f196161a700 1 -- 172.21.2.101:0/33483150 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f19540015b0 con 0x7f19640e53a0 2021-02-10T11:47:00.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.711 7f196161a700 1 -- 172.21.2.101:0/33483150 >> v1:172.21.2.101:6789/0 conn(0x7f19640e53a0 legacy=0x7f19641b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:00.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.711 7f196161a700 1 --2- 172.21.2.101:0/33483150 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1950003a70 0x7f1950005f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:00.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.711 7f196161a700 1 --2- 172.21.2.101:0/33483150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19500067a0 0x7f1950008ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:00.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.711 7f196161a700 1 --2- 172.21.2.101:0/33483150 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19500095a0 0x7f195000bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:00.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.711 7f196161a700 1 -- 172.21.2.101:0/33483150 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195000c310 con 0x7f19500067a0 2021-02-10T11:47:00.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.711 7f196161a700 1 -- 172.21.2.101:0/33483150 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195000c530 con 0x7f1950003a70 2021-02-10T11:47:00.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.711 7f196161a700 1 -- 172.21.2.101:0/33483150 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195000c750 con 0x7f19500095a0 2021-02-10T11:47:00.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.711 7f1961e1b700 1 --2- 172.21.2.101:0/33483150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19500067a0 0x7f1950008ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:00.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.712 7f196261c700 1 --2- 172.21.2.101:0/33483150 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1950003a70 0x7f1950005f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:00.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.712 7f1961e1b700 1 -- 172.21.2.101:0/33483150 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19500095a0 msgr2=0x7f195000bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:00.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.712 7f1961e1b700 1 --2- 172.21.2.101:0/33483150 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19500095a0 0x7f195000bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:00.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.712 7f1961e1b700 1 -- 172.21.2.101:0/33483150 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1950003a70 msgr2=0x7f1950005f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:00.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.712 7f1961e1b700 1 --2- 172.21.2.101:0/33483150 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1950003a70 0x7f1950005f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:00.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.712 7f1961e1b700 1 -- 172.21.2.101:0/33483150 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19500509c0 con 0x7f19500067a0 2021-02-10T11:47:00.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.712 7f1962e1d700 1 -- 172.21.2.101:0/33483150 reap_dead start 2021-02-10T11:47:00.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.712 7f196261c700 1 --2- 172.21.2.101:0/33483150 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1950003a70 0x7f1950005f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:47:00.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.712 7f1961e1b700 1 --2- 172.21.2.101:0/33483150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19500067a0 0x7f1950008ca0 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 rx=0x7f194c00c8f0 tx=0x7f194c00cc90).ready entity=mon.0 client_cookie=3ec318e07a20d612 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:00.728 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.712 7f196161a700 1 -- 172.21.2.101:0/33483150 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f194c009310 con 0x7f19500067a0 2021-02-10T11:47:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.712 7f196161a700 1 -- 172.21.2.101:0/33483150 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194c0095d0 con 0x7f19500067a0 2021-02-10T11:47:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.712 7f196161a700 1 -- 172.21.2.101:0/33483150 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f194c009af0 con 0x7f19500067a0 2021-02-10T11:47:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.712 7f19689f7700 1 -- 172.21.2.101:0/33483150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19500067a0 msgr2=0x7f1950008ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.712 7f19689f7700 1 --2- 172.21.2.101:0/33483150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19500067a0 0x7f1950008ca0 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 rx=0x7f194c00c8f0 tx=0x7f194c00cc90).stop 2021-02-10T11:47:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.713 7f19689f7700 1 -- 172.21.2.101:0/33483150 shutdown_connections 2021-02-10T11:47:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.713 7f19689f7700 1 --2- 172.21.2.101:0/33483150 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19500067a0 0x7f1950008ca0 unknown :-1 s=CLOSED pgs=3038 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.713 7f19689f7700 1 -- 172.21.2.101:0/33483150 >> 172.21.2.101:0/33483150 conn(0x7f19640ce4b0 msgr2=0x7f19640e45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.713 7f19689f7700 1 -- 172.21.2.101:0/33483150 shutdown_connections 2021-02-10T11:47:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.713 7f19689f7700 1 -- 172.21.2.101:0/33483150 wait complete. 2021-02-10T11:47:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.728 7f19689f7700 1 Processor -- start 2021-02-10T11:47:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.728 7f19689f7700 1 -- start start 2021-02-10T11:47:00.729 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.728 7f19689f7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19640a8f70 0x7f1964095e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:00.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.728 7f19689f7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19640964a0 0x7f19640968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:00.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.728 7f19689f7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1964096ec0 0x7f1964106520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:00.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.728 7f19689f7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1964095900 con 0x7f19640964a0 2021-02-10T11:47:00.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.728 7f19689f7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1964107be0 con 0x7f1964096ec0 2021-02-10T11:47:00.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.728 7f19689f7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1964106b40 con 0x7f19640a8f70 2021-02-10T11:47:00.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.728 7f1961e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19640964a0 0x7f19640968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:00.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.728 7f1961e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19640964a0 0x7f19640968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33720/0 (socket says 172.21.2.101:33720) 2021-02-10T11:47:00.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.728 7f1961e1b700 1 -- 172.21.2.101:0/1788089346 learned_addr learned my addr 172.21.2.101:0/1788089346 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:00.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.729 7f1961e1b700 1 -- 172.21.2.101:0/1788089346 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19640a8f70 msgr2=0x7f1964095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:00.732 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.729 7f1961e1b700 1 --2- 172.21.2.101:0/1788089346 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19640a8f70 0x7f1964095e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:00.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.729 7f1961e1b700 1 -- 172.21.2.101:0/1788089346 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1964096ec0 msgr2=0x7f1964106520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:00.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.729 7f1961e1b700 1 --2- 172.21.2.101:0/1788089346 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1964096ec0 0x7f1964106520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:00.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.729 7f1961e1b700 1 -- 172.21.2.101:0/1788089346 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1964106d60 con 0x7f19640964a0 2021-02-10T11:47:00.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.729 7f1961e1b700 1 --2- 172.21.2.101:0/1788089346 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19640964a0 0x7f19640968a0 secure :-1 s=READY pgs=3039 cs=0 l=1 rev1=1 rx=0x7f194c006e40 tx=0x7f194c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:00.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.729 7f195affd700 1 -- 172.21.2.101:0/1788089346 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f194c006b20 con 0x7f19640964a0 2021-02-10T11:47:00.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.729 7f195affd700 1 -- 172.21.2.101:0/1788089346 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194c00bde0 con 0x7f19640964a0 2021-02-10T11:47:00.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.729 7f195affd700 1 -- 172.21.2.101:0/1788089346 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f194c002030 con 0x7f19640964a0 2021-02-10T11:47:00.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.729 7f19689f7700 1 -- 172.21.2.101:0/1788089346 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1964106d60 con 0x7f19640964a0 2021-02-10T11:47:00.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.729 7f19689f7700 1 -- 172.21.2.101:0/1788089346 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19641072f0 con 0x7f19640964a0 2021-02-10T11:47:00.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.730 7f195affd700 1 -- 172.21.2.101:0/1788089346 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f194c000c90 con 0x7f19640964a0 2021-02-10T11:47:00.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.730 7f195affd700 1 --2- 172.21.2.101:0/1788089346 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19480186c0 0x7f194801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:00.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.730 7f195affd700 1 -- 172.21.2.101:0/1788089346 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f194c032500 con 0x7f19640964a0 2021-02-10T11:47:00.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.730 7f196261c700 1 --2- 172.21.2.101:0/1788089346 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19480186c0 0x7f194801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:00.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.730 7f196261c700 1 --2- 172.21.2.101:0/1788089346 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19480186c0 0x7f194801abc0 crc :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:00.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.760 7f19689f7700 1 -- 172.21.2.101:0/1788089346 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1964103750 con 0x7f19640964a0 2021-02-10T11:47:00.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.762 7f195affd700 1 -- 172.21.2.101:0/1788089346 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f194c006740 con 0x7f19640964a0 2021-02-10T11:47:00.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.890 7f19689f7700 1 -- 172.21.2.101:0/1788089346 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000776063", "target": ["mgr", ""], "tenant_id": "tenant1", "auth_id": "guest1", "group_name": "group_0000000000357946"}) v1 -- 0x7f1964109ac0 con 0x7f19480186c0 2021-02-10T11:47:00.892 INFO:tasks.ceph.mgr.y.gibba001.stderr:Creating meta for ID guest1 with tenant tenant1 2021-02-10T11:47:00.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.920 7f195affd700 1 -- 172.21.2.101:0/1788089346 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7f1954006e30 con 0x7f19480186c0 2021-02-10T11:47:00.918 INFO:teuthology.orchestra.run.gibba001.stdout:AQC0xyNgxL70NRAAKx6RudmUZp/4RovUimHZOw== 2021-02-10T11:47:00.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.923 7f19689f7700 1 -- 172.21.2.101:0/1788089346 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19480186c0 msgr2=0x7f194801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:00.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.923 7f19689f7700 1 --2- 172.21.2.101:0/1788089346 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19480186c0 0x7f194801abc0 crc :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:00.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.923 7f19689f7700 1 -- 172.21.2.101:0/1788089346 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19640964a0 msgr2=0x7f19640968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:00.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.923 7f19689f7700 1 --2- 172.21.2.101:0/1788089346 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19640964a0 0x7f19640968a0 secure :-1 s=READY pgs=3039 cs=0 l=1 rev1=1 rx=0x7f194c006e40 tx=0x7f194c00b960).stop 2021-02-10T11:47:00.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.923 7f19689f7700 1 -- 172.21.2.101:0/1788089346 shutdown_connections 2021-02-10T11:47:00.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.923 7f19689f7700 1 --2- 172.21.2.101:0/1788089346 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1964096ec0 0x7f1964106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:00.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.923 7f19689f7700 1 --2- 172.21.2.101:0/1788089346 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19640964a0 0x7f19640968a0 unknown :-1 s=CLOSED pgs=3039 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:00.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.923 7f19689f7700 1 --2- 172.21.2.101:0/1788089346 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19480186c0 0x7f194801abc0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:00.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.923 7f19689f7700 1 --2- 172.21.2.101:0/1788089346 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19640a8f70 0x7f1964095e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:00.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.923 7f19689f7700 1 -- 172.21.2.101:0/1788089346 >> 172.21.2.101:0/1788089346 conn(0x7f196409e480 msgr2=0x7f1964107f30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:00.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.923 7f19689f7700 1 -- 172.21.2.101:0/1788089346 shutdown_connections 2021-02-10T11:47:00.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:00.923 7f19689f7700 1 -- 172.21.2.101:0/1788089346 wait complete. 2021-02-10T11:47:00.928 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.1 && sudo ls volumes 2021-02-10T11:47:01.279 INFO:teuthology.orchestra.run.gibba001.stdout:group_0000000000357946 2021-02-10T11:47:01.279 INFO:teuthology.orchestra.run.gibba001.stdout:_group_0000000000357946:subvolume_0000000000776063.meta 2021-02-10T11:47:01.279 INFO:teuthology.orchestra.run.gibba001.stdout:$guest1.meta 2021-02-10T11:47:01.281 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo sudo cat '/home/ubuntu/cephtest/mnt.0/volumes/$guest1.meta' 2021-02-10T11:47:01.647 INFO:teuthology.orchestra.run.gibba001.stdout:{"compat_version": 1, "tenant_id": "tenant1", "version": 5, "dirty": false, "subvolumes": {"group_0000000000357946/subvolume_0000000000776063": {"access_level": "rw", "dirty": false}}} 2021-02-10T11:47:01.648 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.1 && sudo sed -i s/false/true/g 'volumes/$guest1.meta' 2021-02-10T11:47:02.012 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume authorize cephfs subvolume_0000000000776063 guest1 --group_name group_0000000000357946 --tenant_id tenant1 2021-02-10T11:47:02.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.440 7f2d83f89700 1 Processor -- start 2021-02-10T11:47:02.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.440 7f2d83f89700 1 -- start start 2021-02-10T11:47:02.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.441 7f2d83f89700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2d7c0e53a0 0x7f2d7c1b0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:02.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.441 7f2d83f89700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d7c10b050 0x7f2d7c1091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:02.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.441 7f2d83f89700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d7c109ac0 con 0x7f2d7c1b0b70 2021-02-10T11:47:02.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.441 7f2d83f89700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d7c0e57a0 con 0x7f2d7c0e53a0 2021-02-10T11:47:02.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.441 7f2d83f89700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c1097e0 con 0x7f2d7c10b050 2021-02-10T11:47:02.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.442 7f2d825a6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d7c10b050 0x7f2d7c1091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:02.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.442 7f2d81da5700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2d7c0e53a0 0x7f2d7c1b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:02.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.442 7f2d815a4700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f2d7c1b0b70 0x7f2d7c1089d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:55870/0 (socket says 172.21.2.101:55870) 2021-02-10T11:47:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.442 7f2d815a4700 1 -- 172.21.2.101:0/1208025048 learned_addr learned my addr 172.21.2.101:0/1208025048 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.443 7f2d81da5700 1 -- 172.21.2.101:0/1208025048 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d7c10b050 msgr2=0x7f2d7c1091c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.443 7f2d81da5700 1 --2- 172.21.2.101:0/1208025048 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d7c10b050 0x7f2d7c1091c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.443 7f2d81da5700 1 -- 172.21.2.101:0/1208025048 >> v1:172.21.2.108:6789/0 conn(0x7f2d7c1b0b70 legacy=0x7f2d7c1089d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.443 7f2d80da3700 1 -- 172.21.2.101:0/1208025048 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 547866178 0 0) 0x7f2d780010b0 con 0x7f2d7c1b0b70 2021-02-10T11:47:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.443 7f2d81da5700 1 -- 172.21.2.101:0/1208025048 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7c10b450 con 0x7f2d7c0e53a0 2021-02-10T11:47:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.443 7f2d825a6700 1 --2- 172.21.2.101:0/1208025048 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d7c10b050 0x7f2d7c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:47:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.443 7f2d81da5700 1 --2- 172.21.2.101:0/1208025048 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d7c0e53a0 0x7f2d7c1b0550 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7f2d70006650 tx=0x7f2d70009a80).ready entity=mon.1 client_cookie=2690b5bf32d5f12 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.444 7f2d80da3700 1 -- 172.21.2.101:0/1208025048 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d7000c0c0 con 0x7f2d7c0e53a0 2021-02-10T11:47:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.444 7f2d80da3700 1 -- 172.21.2.101:0/1208025048 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7000c340 con 0x7f2d7c0e53a0 2021-02-10T11:47:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.444 7f2d80da3700 1 -- 172.21.2.101:0/1208025048 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d7000c860 con 0x7f2d7c0e53a0 2021-02-10T11:47:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.444 7f2d83f89700 1 -- 172.21.2.101:0/1208025048 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d7c0e53a0 msgr2=0x7f2d7c1b0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:02.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.444 7f2d83f89700 1 --2- 172.21.2.101:0/1208025048 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d7c0e53a0 0x7f2d7c1b0550 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7f2d70006650 tx=0x7f2d70009a80).stop 2021-02-10T11:47:02.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.444 7f2d83f89700 1 -- 172.21.2.101:0/1208025048 shutdown_connections 2021-02-10T11:47:02.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.444 7f2d83f89700 1 --2- 172.21.2.101:0/1208025048 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d7c10b050 0x7f2d7c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:02.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.444 7f2d83f89700 1 --2- 172.21.2.101:0/1208025048 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d7c0e53a0 0x7f2d7c1b0550 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:02.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.444 7f2d83f89700 1 -- 172.21.2.101:0/1208025048 >> 172.21.2.101:0/1208025048 conn(0x7f2d7c0ce4b0 msgr2=0x7f2d7c0e45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:02.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.445 7f2d83f89700 1 -- 172.21.2.101:0/1208025048 shutdown_connections 2021-02-10T11:47:02.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.445 7f2d83f89700 1 -- 172.21.2.101:0/1208025048 wait complete. 2021-02-10T11:47:02.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.459 7f2d83f89700 1 Processor -- start 2021-02-10T11:47:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.459 7f2d83f89700 1 -- start start 2021-02-10T11:47:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.459 7f2d83f89700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d7c1a4a10 0x7f2d7c0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.459 7f2d83f89700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d7c0b5200 0x7f2d7c0a3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.459 7f2d83f89700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d7c0a3b30 0x7f2d7c092a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.459 7f2d83f89700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c0b5710 con 0x7f2d7c1a4a10 2021-02-10T11:47:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.459 7f2d83f89700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c0a3f70 con 0x7f2d7c0b5200 2021-02-10T11:47:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.459 7f2d83f89700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c0a4190 con 0x7f2d7c0a3b30 2021-02-10T11:47:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.460 7f2d81da5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d7c1a4a10 0x7f2d7c0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:02.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.460 7f2d81da5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d7c1a4a10 0x7f2d7c0b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33740/0 (socket says 172.21.2.101:33740) 2021-02-10T11:47:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.460 7f2d81da5700 1 -- 172.21.2.101:0/3990250455 learned_addr learned my addr 172.21.2.101:0/3990250455 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.460 7f2d81da5700 1 -- 172.21.2.101:0/3990250455 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d7c0a3b30 msgr2=0x7f2d7c092a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.460 7f2d81da5700 1 --2- 172.21.2.101:0/3990250455 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d7c0a3b30 0x7f2d7c092a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.460 7f2d81da5700 1 -- 172.21.2.101:0/3990250455 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d7c0b5200 msgr2=0x7f2d7c0a3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.460 7f2d81da5700 1 --2- 172.21.2.101:0/3990250455 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d7c0b5200 0x7f2d7c0a3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.460 7f2d81da5700 1 -- 172.21.2.101:0/3990250455 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7c093030 con 0x7f2d7c1a4a10 2021-02-10T11:47:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.460 7f2d81da5700 1 --2- 172.21.2.101:0/3990250455 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d7c1a4a10 0x7f2d7c0b4be0 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 rx=0x7f2d70006630 tx=0x7f2d7000a9a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.460 7f2d6e7fc700 1 -- 172.21.2.101:0/3990250455 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d7000ad90 con 0x7f2d7c1a4a10 2021-02-10T11:47:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.460 7f2d6e7fc700 1 -- 172.21.2.101:0/3990250455 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d700067f0 con 0x7f2d7c1a4a10 2021-02-10T11:47:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.460 7f2d6e7fc700 1 -- 172.21.2.101:0/3990250455 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d70006ee0 con 0x7f2d7c1a4a10 2021-02-10T11:47:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.460 7f2d83f89700 1 -- 172.21.2.101:0/3990250455 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7c093030 con 0x7f2d7c1a4a10 2021-02-10T11:47:02.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.461 7f2d83f89700 1 -- 172.21.2.101:0/3990250455 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7c093030 con 0x7f2d7c1a4a10 2021-02-10T11:47:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.461 7f2d6e7fc700 1 -- 172.21.2.101:0/3990250455 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2d70001030 con 0x7f2d7c1a4a10 2021-02-10T11:47:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.461 7f2d6e7fc700 1 --2- 172.21.2.101:0/3990250455 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d640186c0 0x7f2d6401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.461 7f2d6e7fc700 1 -- 172.21.2.101:0/3990250455 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2d700323c0 con 0x7f2d7c1a4a10 2021-02-10T11:47:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.461 7f2d815a4700 1 --2- 172.21.2.101:0/3990250455 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d640186c0 0x7f2d6401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:02.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.462 7f2d815a4700 1 --2- 172.21.2.101:0/3990250455 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d640186c0 0x7f2d6401abc0 crc :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:02.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.490 7f2d83f89700 1 -- 172.21.2.101:0/3990250455 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d7c103750 con 0x7f2d7c1a4a10 2021-02-10T11:47:02.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.493 7f2d6e7fc700 1 -- 172.21.2.101:0/3990250455 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2d70027380 con 0x7f2d7c1a4a10 2021-02-10T11:47:02.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.619 7f2d83f89700 1 -- 172.21.2.101:0/3990250455 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000776063", "target": ["mgr", ""], "tenant_id": "tenant1", "auth_id": "guest1", "group_name": "group_0000000000357946"}) v1 -- 0x7f2d7c103750 con 0x7f2d640186c0 2021-02-10T11:47:02.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.680 7f2d6e7fc700 1 -- 172.21.2.101:0/3990250455 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7f2d78000940 con 0x7f2d640186c0 2021-02-10T11:47:02.680 INFO:teuthology.orchestra.run.gibba001.stdout:AQC0xyNgxL70NRAAKx6RudmUZp/4RovUimHZOw== 2021-02-10T11:47:02.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.685 7f2d83f89700 1 -- 172.21.2.101:0/3990250455 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d640186c0 msgr2=0x7f2d6401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:02.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.685 7f2d83f89700 1 --2- 172.21.2.101:0/3990250455 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d640186c0 0x7f2d6401abc0 crc :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:02.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.685 7f2d83f89700 1 -- 172.21.2.101:0/3990250455 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d7c1a4a10 msgr2=0x7f2d7c0b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:02.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.685 7f2d83f89700 1 --2- 172.21.2.101:0/3990250455 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d7c1a4a10 0x7f2d7c0b4be0 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 rx=0x7f2d70006630 tx=0x7f2d7000a9a0).stop 2021-02-10T11:47:02.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.685 7f2d83f89700 1 -- 172.21.2.101:0/3990250455 shutdown_connections 2021-02-10T11:47:02.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.685 7f2d83f89700 1 --2- 172.21.2.101:0/3990250455 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d640186c0 0x7f2d6401abc0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:02.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.685 7f2d83f89700 1 --2- 172.21.2.101:0/3990250455 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d7c0a3b30 0x7f2d7c092a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:02.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.685 7f2d83f89700 1 --2- 172.21.2.101:0/3990250455 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d7c0b5200 0x7f2d7c0a3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:02.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.685 7f2d83f89700 1 --2- 172.21.2.101:0/3990250455 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d7c1a4a10 0x7f2d7c0b4be0 unknown :-1 s=CLOSED pgs=3040 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:02.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.685 7f2d83f89700 1 -- 172.21.2.101:0/3990250455 >> 172.21.2.101:0/3990250455 conn(0x7f2d7c105f80 msgr2=0x7f2d7c0b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:02.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.685 7f2d83f89700 1 -- 172.21.2.101:0/3990250455 shutdown_connections 2021-02-10T11:47:02.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:02.685 7f2d83f89700 1 -- 172.21.2.101:0/3990250455 wait complete. 2021-02-10T11:47:02.690 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo sudo cat '/home/ubuntu/cephtest/mnt.0/volumes/$guest1.meta' 2021-02-10T11:47:03.051 INFO:teuthology.orchestra.run.gibba001.stdout:{"tenant_id": "tenant1", "version": 5, "compat_version": 1, "subvolumes": {"group_0000000000357946/subvolume_0000000000776063": {"access_level": "rw", "dirty": false}}, "dirty": false} 2021-02-10T11:47:03.051 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume deauthorize cephfs subvolume_0000000000776063 guest1 --group_name group_0000000000357946 2021-02-10T11:47:03.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.471 7f9de548e700 1 Processor -- start 2021-02-10T11:47:03.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.471 7f9de548e700 1 -- start start 2021-02-10T11:47:03.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.471 7f9de548e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9de00eb220 0x7f9de00eb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:03.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.471 7f9de548e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f9de010aad0 0x7f9de010aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:03.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.471 7f9de548e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de010b4f0 con 0x7f9de00dce80 2021-02-10T11:47:03.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.471 7f9de548e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de010a550 con 0x7f9de00eb220 2021-02-10T11:47:03.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.471 7f9de548e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de00881d0 con 0x7f9de010aad0 2021-02-10T11:47:03.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.472 7f9dde87c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9de00eb220 0x7f9de00eb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:03.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.472 7f9dde87c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9de00eb220 0x7f9de00eb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33750/0 (socket says 172.21.2.101:33750) 2021-02-10T11:47:03.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.472 7f9dde87c700 1 -- 172.21.2.101:0/3836697349 learned_addr learned my addr 172.21.2.101:0/3836697349 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:03.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.472 7f9ddf87e700 1 --2- 172.21.2.101:0/3836697349 >> v2:172.21.2.108:3300/0 conn(0x7f9de010aad0 0x7f9de010aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:03.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.472 7f9dde87c700 1 -- 172.21.2.101:0/3836697349 >> v2:172.21.2.108:3300/0 conn(0x7f9de010aad0 msgr2=0x7f9de010aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:03.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.472 7f9dde87c700 1 --2- 172.21.2.101:0/3836697349 >> v2:172.21.2.108:3300/0 conn(0x7f9de010aad0 0x7f9de010aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:03.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.472 7f9dde87c700 1 -- 172.21.2.101:0/3836697349 >> v1:172.21.2.108:6789/0 conn(0x7f9de00dce80 legacy=0x7f9de00d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:03.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.472 7f9dde87c700 1 -- 172.21.2.101:0/3836697349 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de00d1f30 con 0x7f9de00eb220 2021-02-10T11:47:03.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.472 7f9dde87c700 1 --2- 172.21.2.101:0/3836697349 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9de00eb220 0x7f9de00eb620 secure :-1 s=READY pgs=3041 cs=0 l=1 rev1=1 rx=0x7f9dd0006700 tx=0x7f9dd0009b80).ready entity=mon.0 client_cookie=ac451767a6f57850 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:03.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.473 7f9dde07b700 1 -- 172.21.2.101:0/3836697349 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9dd000c0c0 con 0x7f9de00eb220 2021-02-10T11:47:03.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.473 7f9dde07b700 1 -- 172.21.2.101:0/3836697349 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd000c440 con 0x7f9de00eb220 2021-02-10T11:47:03.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.473 7f9dde07b700 1 -- 172.21.2.101:0/3836697349 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9dd000c960 con 0x7f9de00eb220 2021-02-10T11:47:03.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.473 7f9de548e700 1 -- 172.21.2.101:0/3836697349 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9de00eb220 msgr2=0x7f9de00eb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:03.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.473 7f9de548e700 1 --2- 172.21.2.101:0/3836697349 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9de00eb220 0x7f9de00eb620 secure :-1 s=READY pgs=3041 cs=0 l=1 rev1=1 rx=0x7f9dd0006700 tx=0x7f9dd0009b80).stop 2021-02-10T11:47:03.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.473 7f9de548e700 1 -- 172.21.2.101:0/3836697349 shutdown_connections 2021-02-10T11:47:03.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.473 7f9de548e700 1 --2- 172.21.2.101:0/3836697349 >> v2:172.21.2.108:3300/0 conn(0x7f9de010aad0 0x7f9de010aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:03.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.473 7f9de548e700 1 --2- 172.21.2.101:0/3836697349 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9de00eb220 0x7f9de00eb620 unknown :-1 s=CLOSED pgs=3041 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:03.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.473 7f9de548e700 1 -- 172.21.2.101:0/3836697349 >> 172.21.2.101:0/3836697349 conn(0x7f9de00ea630 msgr2=0x7f9de00d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:03.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.473 7f9de548e700 1 -- 172.21.2.101:0/3836697349 shutdown_connections 2021-02-10T11:47:03.486 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.473 7f9de548e700 1 -- 172.21.2.101:0/3836697349 wait complete. 2021-02-10T11:47:03.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.488 7f9de548e700 1 Processor -- start 2021-02-10T11:47:03.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.488 7f9de548e700 1 -- start start 2021-02-10T11:47:03.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.488 7f9de548e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9de00b8840 0x7f9de00b7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:03.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.488 7f9de548e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9de00b82e0 0x7f9de00b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:03.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.488 7f9de548e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9de00b9330 0x7f9de00a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:03.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.488 7f9de548e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de00b9730 con 0x7f9de00b82e0 2021-02-10T11:47:03.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.488 7f9de548e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de00b9950 con 0x7f9de00b9330 2021-02-10T11:47:03.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.488 7f9de548e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de00a0ed0 con 0x7f9de00b8840 2021-02-10T11:47:03.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.489 7f9dde87c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9de00b82e0 0x7f9de00b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:03.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.489 7f9dde87c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9de00b82e0 0x7f9de00b8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33756/0 (socket says 172.21.2.101:33756) 2021-02-10T11:47:03.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.489 7f9dde87c700 1 -- 172.21.2.101:0/1145797544 learned_addr learned my addr 172.21.2.101:0/1145797544 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:03.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.489 7f9ddf87e700 1 --2- 172.21.2.101:0/1145797544 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9de00b9330 0x7f9de00a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:03.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.489 7f9ddf07d700 1 --2- 172.21.2.101:0/1145797544 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9de00b8840 0x7f9de00b7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:03.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.489 7f9dde87c700 1 -- 172.21.2.101:0/1145797544 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9de00b8840 msgr2=0x7f9de00b7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:03.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.489 7f9dde87c700 1 --2- 172.21.2.101:0/1145797544 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9de00b8840 0x7f9de00b7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:03.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.489 7f9dde87c700 1 -- 172.21.2.101:0/1145797544 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9de00b9330 msgr2=0x7f9de00a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:03.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.489 7f9dde87c700 1 --2- 172.21.2.101:0/1145797544 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9de00b9330 0x7f9de00a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:03.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.489 7f9dde87c700 1 -- 172.21.2.101:0/1145797544 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de00a10f0 con 0x7f9de00b82e0 2021-02-10T11:47:03.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.489 7f9ddf07d700 1 --2- 172.21.2.101:0/1145797544 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9de00b8840 0x7f9de00b7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:47:03.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.489 7f9dde87c700 1 --2- 172.21.2.101:0/1145797544 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9de00b82e0 0x7f9de00b8d10 secure :-1 s=READY pgs=3042 cs=0 l=1 rev1=1 rx=0x7f9dd000b650 tx=0x7f9dd000b9f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:03.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.490 7f9dd7fff700 1 -- 172.21.2.101:0/1145797544 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9dd000bde0 con 0x7f9de00b82e0 2021-02-10T11:47:03.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.490 7f9dd7fff700 1 -- 172.21.2.101:0/1145797544 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd0006570 con 0x7f9de00b82e0 2021-02-10T11:47:03.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.490 7f9de548e700 1 -- 172.21.2.101:0/1145797544 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de00a10f0 con 0x7f9de00b82e0 2021-02-10T11:47:03.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.490 7f9dd7fff700 1 -- 172.21.2.101:0/1145797544 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9dd00022b0 con 0x7f9de00b82e0 2021-02-10T11:47:03.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.490 7f9de548e700 1 -- 172.21.2.101:0/1145797544 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de00a1680 con 0x7f9de00b82e0 2021-02-10T11:47:03.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.490 7f9dd7fff700 1 -- 172.21.2.101:0/1145797544 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9dd0009f90 con 0x7f9de00b82e0 2021-02-10T11:47:03.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.491 7f9dd7fff700 1 --2- 172.21.2.101:0/1145797544 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9dc40186c0 0x7f9dc401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:03.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.491 7f9dd7fff700 1 -- 172.21.2.101:0/1145797544 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9dd0032560 con 0x7f9de00b82e0 2021-02-10T11:47:03.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.491 7f9ddf07d700 1 --2- 172.21.2.101:0/1145797544 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9dc40186c0 0x7f9dc401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:03.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.491 7f9ddf07d700 1 --2- 172.21.2.101:0/1145797544 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9dc40186c0 0x7f9dc401abc0 crc :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:03.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.522 7f9de548e700 1 -- 172.21.2.101:0/1145797544 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de00f93d0 con 0x7f9de00b82e0 2021-02-10T11:47:03.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.524 7f9dd7fff700 1 -- 172.21.2.101:0/1145797544 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9dd00009b0 con 0x7f9de00b82e0 2021-02-10T11:47:03.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.651 7f9de548e700 1 -- 172.21.2.101:0/1145797544 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume deauthorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000776063", "target": ["mgr", ""], "auth_id": "guest1", "group_name": "group_0000000000357946"}) v1 -- 0x7f9de00a1680 con 0x7f9dc40186c0 2021-02-10T11:47:03.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.687 7f9dd7fff700 1 -- 172.21.2.101:0/1145797544 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f9dc8006860 con 0x7f9dc40186c0 2021-02-10T11:47:03.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.716 7f9de548e700 1 -- 172.21.2.101:0/1145797544 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9dc40186c0 msgr2=0x7f9dc401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:03.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.716 7f9de548e700 1 --2- 172.21.2.101:0/1145797544 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9dc40186c0 0x7f9dc401abc0 crc :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:03.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.716 7f9de548e700 1 -- 172.21.2.101:0/1145797544 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9de00b82e0 msgr2=0x7f9de00b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:03.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.716 7f9de548e700 1 --2- 172.21.2.101:0/1145797544 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9de00b82e0 0x7f9de00b8d10 secure :-1 s=READY pgs=3042 cs=0 l=1 rev1=1 rx=0x7f9dd000b650 tx=0x7f9dd000b9f0).stop 2021-02-10T11:47:03.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.716 7f9de548e700 1 -- 172.21.2.101:0/1145797544 shutdown_connections 2021-02-10T11:47:03.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.716 7f9de548e700 1 --2- 172.21.2.101:0/1145797544 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9de00b9330 0x7f9de00a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:03.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.716 7f9de548e700 1 --2- 172.21.2.101:0/1145797544 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9de00b82e0 0x7f9de00b8d10 unknown :-1 s=CLOSED pgs=3042 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:03.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.716 7f9de548e700 1 --2- 172.21.2.101:0/1145797544 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9dc40186c0 0x7f9dc401abc0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:03.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.716 7f9de548e700 1 --2- 172.21.2.101:0/1145797544 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9de00b8840 0x7f9de00b7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:03.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.716 7f9de548e700 1 -- 172.21.2.101:0/1145797544 >> 172.21.2.101:0/1145797544 conn(0x7f9de00b1660 msgr2=0x7f9de00a8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:03.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.717 7f9de548e700 1 -- 172.21.2.101:0/1145797544 shutdown_connections 2021-02-10T11:47:03.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:03.717 7f9de548e700 1 -- 172.21.2.101:0/1145797544 wait complete. 2021-02-10T11:47:03.720 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:03.721 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:47:03.721 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:47:04.056 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:47:04.056 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:47:04.057 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:47:04.057 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:47:04.057 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:47:04.406 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[174142]: fuse finished with error 0 and tester_r 0 2021-02-10T11:47:04.407 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:04.407 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:47:04.407 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:47:04.742 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:47:04.742 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:47:04.742 INFO:teuthology.orchestra.run:waiting for 900 2021-02-10T11:47:04.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:47:04.743 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:47:05.079 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth rm client.guest1 2021-02-10T11:47:05.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.494 7f138d5f7700 1 Processor -- start 2021-02-10T11:47:05.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.495 7f138d5f7700 1 -- start start 2021-02-10T11:47:05.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.495 7f138d5f7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f13880d0340 0x7f13880ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:05.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.495 7f138d5f7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f13880d1c70 0x7f13880efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:05.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.495 7f138d5f7700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13880f0b10 con 0x7f13880d1870 2021-02-10T11:47:05.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.495 7f138d5f7700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13880e5ed0 con 0x7f13880d0340 2021-02-10T11:47:05.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.495 7f138d5f7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13881b0be0 con 0x7f13880d1c70 2021-02-10T11:47:05.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.495 7f1386abe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f13880d0340 0x7f13880ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:05.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.495 7f1386abe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f13880d0340 0x7f13880ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33762/0 (socket says 172.21.2.101:33762) 2021-02-10T11:47:05.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.495 7f1386abe700 1 -- 172.21.2.101:0/33151319 learned_addr learned my addr 172.21.2.101:0/33151319 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:05.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.497 7f1387ac0700 1 --2- 172.21.2.101:0/33151319 >> v2:172.21.2.108:3300/0 conn(0x7f13880d1c70 0x7f13880efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:05.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.497 7f1386abe700 1 -- 172.21.2.101:0/33151319 >> v2:172.21.2.108:3300/0 conn(0x7f13880d1c70 msgr2=0x7f13880efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:05.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.497 7f1386abe700 1 --2- 172.21.2.101:0/33151319 >> v2:172.21.2.108:3300/0 conn(0x7f13880d1c70 0x7f13880efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:05.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.497 7f1386abe700 1 -- 172.21.2.101:0/33151319 >> v1:172.21.2.108:6789/0 conn(0x7f13880d1870 legacy=0x7f13881b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:05.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.497 7f1386abe700 1 -- 172.21.2.101:0/33151319 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13880f20a0 con 0x7f13880d0340 2021-02-10T11:47:05.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.497 7f1387ac0700 1 --2- 172.21.2.101:0/33151319 >> v2:172.21.2.108:3300/0 conn(0x7f13880d1c70 0x7f13880efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:47:05.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.497 7f1386abe700 1 --2- 172.21.2.101:0/33151319 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13880d0340 0x7f13880ef640 secure :-1 s=READY pgs=3043 cs=0 l=1 rev1=1 rx=0x7f137c006700 tx=0x7f137c009b80).ready entity=mon.0 client_cookie=31e8d0778ffa1f89 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:05.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.497 7f13862bd700 1 -- 172.21.2.101:0/33151319 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f137c00c0c0 con 0x7f13880d0340 2021-02-10T11:47:05.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.497 7f13862bd700 1 -- 172.21.2.101:0/33151319 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137c00c400 con 0x7f13880d0340 2021-02-10T11:47:05.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.497 7f13862bd700 1 -- 172.21.2.101:0/33151319 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f137c00c8e0 con 0x7f13880d0340 2021-02-10T11:47:05.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.497 7f138d5f7700 1 -- 172.21.2.101:0/33151319 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13880d0340 msgr2=0x7f13880ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:05.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.497 7f138d5f7700 1 --2- 172.21.2.101:0/33151319 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13880d0340 0x7f13880ef640 secure :-1 s=READY pgs=3043 cs=0 l=1 rev1=1 rx=0x7f137c006700 tx=0x7f137c009b80).stop 2021-02-10T11:47:05.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.497 7f138d5f7700 1 -- 172.21.2.101:0/33151319 shutdown_connections 2021-02-10T11:47:05.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.497 7f138d5f7700 1 --2- 172.21.2.101:0/33151319 >> v2:172.21.2.108:3300/0 conn(0x7f13880d1c70 0x7f13880efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:05.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.497 7f138d5f7700 1 --2- 172.21.2.101:0/33151319 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13880d0340 0x7f13880ef640 unknown :-1 s=CLOSED pgs=3043 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:05.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.497 7f138d5f7700 1 -- 172.21.2.101:0/33151319 >> 172.21.2.101:0/33151319 conn(0x7f13880ce450 msgr2=0x7f13880d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:05.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.497 7f138d5f7700 1 -- 172.21.2.101:0/33151319 shutdown_connections 2021-02-10T11:47:05.510 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.497 7f138d5f7700 1 -- 172.21.2.101:0/33151319 wait complete. 2021-02-10T11:47:05.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.511 7f138d5f7700 1 Processor -- start 2021-02-10T11:47:05.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.512 7f138d5f7700 1 -- start start 2021-02-10T11:47:05.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.512 7f138d5f7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13880948c0 0x7f1388094d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:05.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.512 7f138d5f7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1388093d40 0x7f1388094140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:05.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.512 7f138d5f7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13880bfc70 0x7f13880c0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:05.512 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.512 7f138d5f7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138808ded0 con 0x7f13880bfc70 2021-02-10T11:47:05.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.512 7f138d5f7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13880953b0 con 0x7f1388093d40 2021-02-10T11:47:05.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.512 7f138d5f7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13880c0690 con 0x7f13880948c0 2021-02-10T11:47:05.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.512 7f1387ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13880bfc70 0x7f13880c0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:05.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.512 7f1387ac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13880bfc70 0x7f13880c0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33770/0 (socket says 172.21.2.101:33770) 2021-02-10T11:47:05.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.512 7f1387ac0700 1 -- 172.21.2.101:0/2735330297 learned_addr learned my addr 172.21.2.101:0/2735330297 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:05.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.512 7f1387ac0700 1 -- 172.21.2.101:0/2735330297 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13880948c0 msgr2=0x7f1388094d90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:47:05.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.512 7f1387ac0700 1 --2- 172.21.2.101:0/2735330297 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13880948c0 0x7f1388094d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:05.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.512 7f1387ac0700 1 -- 172.21.2.101:0/2735330297 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1388093d40 msgr2=0x7f1388094140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:47:05.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.512 7f1387ac0700 1 --2- 172.21.2.101:0/2735330297 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1388093d40 0x7f1388094140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:05.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.512 7f1387ac0700 1 -- 172.21.2.101:0/2735330297 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13880c08b0 con 0x7f13880bfc70 2021-02-10T11:47:05.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.513 7f1387ac0700 1 --2- 172.21.2.101:0/2735330297 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13880bfc70 0x7f13880c0070 secure :-1 s=READY pgs=3044 cs=0 l=1 rev1=1 rx=0x7f1380009980 tx=0x7f138000bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:05.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.513 7f136ffff700 1 -- 172.21.2.101:0/2735330297 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f138000c2f0 con 0x7f13880bfc70 2021-02-10T11:47:05.513 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.513 7f136ffff700 1 -- 172.21.2.101:0/2735330297 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1380006790 con 0x7f13880bfc70 2021-02-10T11:47:05.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.513 7f136ffff700 1 -- 172.21.2.101:0/2735330297 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f138000c7f0 con 0x7f13880bfc70 2021-02-10T11:47:05.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.513 7f138d5f7700 1 -- 172.21.2.101:0/2735330297 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13880c08b0 con 0x7f13880bfc70 2021-02-10T11:47:05.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.513 7f138d5f7700 1 -- 172.21.2.101:0/2735330297 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13880c08b0 con 0x7f13880bfc70 2021-02-10T11:47:05.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.514 7f136ffff700 1 -- 172.21.2.101:0/2735330297 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f13800011f0 con 0x7f13880bfc70 2021-02-10T11:47:05.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.514 7f136ffff700 1 --2- 172.21.2.101:0/2735330297 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13680186c0 0x7f136801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:05.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.514 7f136ffff700 1 -- 172.21.2.101:0/2735330297 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f13800328f0 con 0x7f13880bfc70 2021-02-10T11:47:05.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.514 7f13872bf700 1 --2- 172.21.2.101:0/2735330297 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13680186c0 0x7f136801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:05.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.514 7f13872bf700 1 --2- 172.21.2.101:0/2735330297 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13680186c0 0x7f136801abc0 crc :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:05.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.545 7f138d5f7700 1 -- 172.21.2.101:0/2735330297 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13880ebc90 con 0x7f13880bfc70 2021-02-10T11:47:05.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.548 7f136ffff700 1 -- 172.21.2.101:0/2735330297 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f138000be10 con 0x7f13880bfc70 2021-02-10T11:47:05.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.678 7f138d5f7700 1 -- 172.21.2.101:0/2735330297 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth rm", "entity": "client.guest1"} v 0) v1 -- 0x7f13880f0b10 con 0x7f13880bfc70 2021-02-10T11:47:05.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.679 7f136ffff700 1 -- 172.21.2.101:0/2735330297 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth rm", "entity": "client.guest1"}]=0 entity client.guest1 does not exist v101) v1 ==== 117+0+0 (secure 0 0 0) 0x7f13800011f0 con 0x7f13880bfc70 2021-02-10T11:47:05.677 INFO:teuthology.orchestra.run.gibba001.stderr:entity client.guest1 does not exist 2021-02-10T11:47:05.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.682 7f138d5f7700 1 -- 172.21.2.101:0/2735330297 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13680186c0 msgr2=0x7f136801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:05.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.682 7f138d5f7700 1 --2- 172.21.2.101:0/2735330297 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13680186c0 0x7f136801abc0 crc :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:05.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.682 7f138d5f7700 1 -- 172.21.2.101:0/2735330297 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13880bfc70 msgr2=0x7f13880c0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:05.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.682 7f138d5f7700 1 --2- 172.21.2.101:0/2735330297 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13880bfc70 0x7f13880c0070 secure :-1 s=READY pgs=3044 cs=0 l=1 rev1=1 rx=0x7f1380009980 tx=0x7f138000bcd0).stop 2021-02-10T11:47:05.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.682 7f138d5f7700 1 -- 172.21.2.101:0/2735330297 shutdown_connections 2021-02-10T11:47:05.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.682 7f138d5f7700 1 --2- 172.21.2.101:0/2735330297 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13880bfc70 0x7f13880c0070 unknown :-1 s=CLOSED pgs=3044 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:05.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.682 7f138d5f7700 1 --2- 172.21.2.101:0/2735330297 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1388093d40 0x7f1388094140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:05.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.682 7f138d5f7700 1 --2- 172.21.2.101:0/2735330297 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13680186c0 0x7f136801abc0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:05.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.682 7f138d5f7700 1 --2- 172.21.2.101:0/2735330297 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13880948c0 0x7f1388094d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:05.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.682 7f138d5f7700 1 -- 172.21.2.101:0/2735330297 >> 172.21.2.101:0/2735330297 conn(0x7f13880bf640 msgr2=0x7f138809f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:05.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.682 7f138d5f7700 1 -- 172.21.2.101:0/2735330297 shutdown_connections 2021-02-10T11:47:05.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:05.682 7f138d5f7700 1 -- 172.21.2.101:0/2735330297 wait complete. 2021-02-10T11:47:05.687 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000776063 --group_name group_0000000000357946 2021-02-10T11:47:06.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.104 7f804c72d700 1 Processor -- start 2021-02-10T11:47:06.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.104 7f804c72d700 1 -- start start 2021-02-10T11:47:06.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.104 7f804c72d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f80440d1880 0x7f80441b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:06.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.104 7f804c72d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f80441b0af0 0x7f8044108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:06.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.104 7f804c72d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f80440d1c80 0x7f8044108f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:06.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.104 7f804c72d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80440fe1d0 con 0x7f80441b0af0 2021-02-10T11:47:06.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.104 7f804c72d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8044109a70 con 0x7f80440d1880 2021-02-10T11:47:06.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.104 7f804c72d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8044088070 con 0x7f80440d1c80 2021-02-10T11:47:06.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.105 7f8049d48700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f80441b0af0 0x7f8044108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:06.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.105 7f8049d48700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f80441b0af0 0x7f8044108950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33780/0 (socket says 172.21.2.101:33780) 2021-02-10T11:47:06.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.105 7f8049d48700 1 -- 172.21.2.101:0/35638862 learned_addr learned my addr 172.21.2.101:0/35638862 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:06.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.105 7f8049d48700 1 -- 172.21.2.101:0/35638862 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f80440d1c80 msgr2=0x7f8044108f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:06.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.105 7f8049d48700 1 --2- 172.21.2.101:0/35638862 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f80440d1c80 0x7f8044108f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:06.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.105 7f8049d48700 1 -- 172.21.2.101:0/35638862 >> v2:172.21.2.108:3300/0 conn(0x7f80440d1880 msgr2=0x7f80441b04d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:06.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.105 7f8049d48700 1 --2- 172.21.2.101:0/35638862 >> v2:172.21.2.108:3300/0 conn(0x7f80440d1880 0x7f80441b04d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:06.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.105 7f8049d48700 1 -- 172.21.2.101:0/35638862 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8044109590 con 0x7f80441b0af0 2021-02-10T11:47:06.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.105 7f8049d48700 1 --2- 172.21.2.101:0/35638862 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80441b0af0 0x7f8044108950 secure :-1 s=READY pgs=3045 cs=0 l=1 rev1=1 rx=0x7f8034006700 tx=0x7f8034009b80).ready entity=mon.0 client_cookie=23279c346e626a63 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:06.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.105 7f8049547700 1 -- 172.21.2.101:0/35638862 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f803400c0c0 con 0x7f80441b0af0 2021-02-10T11:47:06.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.105 7f8049547700 1 -- 172.21.2.101:0/35638862 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803400c440 con 0x7f80441b0af0 2021-02-10T11:47:06.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.105 7f8049547700 1 -- 172.21.2.101:0/35638862 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f803400c920 con 0x7f80441b0af0 2021-02-10T11:47:06.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.105 7f804c72d700 1 -- 172.21.2.101:0/35638862 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80441b0af0 msgr2=0x7f8044108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:06.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.105 7f804c72d700 1 --2- 172.21.2.101:0/35638862 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80441b0af0 0x7f8044108950 secure :-1 s=READY pgs=3045 cs=0 l=1 rev1=1 rx=0x7f8034006700 tx=0x7f8034009b80).stop 2021-02-10T11:47:06.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.106 7f804c72d700 1 -- 172.21.2.101:0/35638862 shutdown_connections 2021-02-10T11:47:06.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.106 7f804c72d700 1 --2- 172.21.2.101:0/35638862 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f80440d1c80 0x7f8044108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:06.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.106 7f804c72d700 1 --2- 172.21.2.101:0/35638862 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80441b0af0 0x7f8044108950 unknown :-1 s=CLOSED pgs=3045 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:06.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.106 7f804c72d700 1 --2- 172.21.2.101:0/35638862 >> v2:172.21.2.108:3300/0 conn(0x7f80440d1880 0x7f80441b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:06.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.106 7f804c72d700 1 -- 172.21.2.101:0/35638862 >> 172.21.2.101:0/35638862 conn(0x7f80440ce430 msgr2=0x7f80440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:06.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.106 7f804c72d700 1 -- 172.21.2.101:0/35638862 shutdown_connections 2021-02-10T11:47:06.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.106 7f804c72d700 1 -- 172.21.2.101:0/35638862 wait complete. 2021-02-10T11:47:06.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.121 7f804c72d700 1 Processor -- start 2021-02-10T11:47:06.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.121 7f804c72d700 1 -- start start 2021-02-10T11:47:06.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.121 7f804c72d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80441aca10 0x7f80441ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:06.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.121 7f804c72d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f804418d4f0 0x7f804418d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:06.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.121 7f804c72d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f804418e860 0x7f804418ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:06.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.121 7f804c72d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80441af5a0 con 0x7f80441aca10 2021-02-10T11:47:06.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.121 7f804c72d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804418df10 con 0x7f804418d4f0 2021-02-10T11:47:06.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.121 7f804c72d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804418f280 con 0x7f804418e860 2021-02-10T11:47:06.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.122 7f804a549700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80441aca10 0x7f80441ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:06.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.122 7f804a549700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80441aca10 0x7f80441ace10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33782/0 (socket says 172.21.2.101:33782) 2021-02-10T11:47:06.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.122 7f804a549700 1 -- 172.21.2.101:0/2418666711 learned_addr learned my addr 172.21.2.101:0/2418666711 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:06.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.122 7f804ad4a700 1 --2- 172.21.2.101:0/2418666711 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f804418e860 0x7f804418ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:06.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.122 7f8049d48700 1 --2- 172.21.2.101:0/2418666711 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f804418d4f0 0x7f804418d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:06.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.122 7f804a549700 1 -- 172.21.2.101:0/2418666711 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f804418e860 msgr2=0x7f804418ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:06.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.122 7f804a549700 1 --2- 172.21.2.101:0/2418666711 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f804418e860 0x7f804418ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:06.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.122 7f804a549700 1 -- 172.21.2.101:0/2418666711 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f804418d4f0 msgr2=0x7f804418d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:06.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.122 7f804a549700 1 --2- 172.21.2.101:0/2418666711 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f804418d4f0 0x7f804418d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:06.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.122 7f804a549700 1 -- 172.21.2.101:0/2418666711 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804418f4a0 con 0x7f80441aca10 2021-02-10T11:47:06.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.122 7f8049d48700 1 --2- 172.21.2.101:0/2418666711 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f804418d4f0 0x7f804418d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:47:06.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.122 7f804a549700 1 --2- 172.21.2.101:0/2418666711 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80441aca10 0x7f80441ace10 secure :-1 s=READY pgs=3046 cs=0 l=1 rev1=1 rx=0x7f804000b5c0 tx=0x7f804000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:06.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.122 7f803affd700 1 -- 172.21.2.101:0/2418666711 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f804000bd50 con 0x7f80441aca10 2021-02-10T11:47:06.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.122 7f803affd700 1 -- 172.21.2.101:0/2418666711 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8040006030 con 0x7f80441aca10 2021-02-10T11:47:06.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.122 7f803affd700 1 -- 172.21.2.101:0/2418666711 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f80400054d0 con 0x7f80441aca10 2021-02-10T11:47:06.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.122 7f804c72d700 1 -- 172.21.2.101:0/2418666711 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804418f4a0 con 0x7f80441aca10 2021-02-10T11:47:06.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.123 7f804c72d700 1 -- 172.21.2.101:0/2418666711 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804418f4a0 con 0x7f80441aca10 2021-02-10T11:47:06.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.123 7f803affd700 1 -- 172.21.2.101:0/2418666711 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f804000bd50 con 0x7f80441aca10 2021-02-10T11:47:06.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.123 7f803affd700 1 --2- 172.21.2.101:0/2418666711 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f802c0186c0 0x7f802c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:06.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.123 7f803affd700 1 -- 172.21.2.101:0/2418666711 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f80400347e0 con 0x7f80441aca10 2021-02-10T11:47:06.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.123 7f8049d48700 1 --2- 172.21.2.101:0/2418666711 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f802c0186c0 0x7f802c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:06.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.124 7f8049d48700 1 --2- 172.21.2.101:0/2418666711 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f802c0186c0 0x7f802c01abc0 crc :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:06.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.155 7f804c72d700 1 -- 172.21.2.101:0/2418666711 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8044082ab0 con 0x7f80441aca10 2021-02-10T11:47:06.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.157 7f803affd700 1 -- 172.21.2.101:0/2418666711 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8040006b80 con 0x7f80441aca10 2021-02-10T11:47:06.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.286 7f804c72d700 1 -- 172.21.2.101:0/2418666711 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000776063", "target": ["mgr", ""], "group_name": "group_0000000000357946"}) v1 -- 0x7f8044082ab0 con 0x7f802c0186c0 2021-02-10T11:47:06.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.295 7f803affd700 1 -- 172.21.2.101:0/2418666711 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f80340009b0 con 0x7f802c0186c0 2021-02-10T11:47:06.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.302 7f804c72d700 1 -- 172.21.2.101:0/2418666711 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f802c0186c0 msgr2=0x7f802c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:06.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.302 7f804c72d700 1 --2- 172.21.2.101:0/2418666711 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f802c0186c0 0x7f802c01abc0 crc :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:06.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.302 7f804c72d700 1 -- 172.21.2.101:0/2418666711 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80441aca10 msgr2=0x7f80441ace10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:06.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.302 7f804c72d700 1 --2- 172.21.2.101:0/2418666711 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80441aca10 0x7f80441ace10 secure :-1 s=READY pgs=3046 cs=0 l=1 rev1=1 rx=0x7f804000b5c0 tx=0x7f804000b960).stop 2021-02-10T11:47:06.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.302 7f804c72d700 1 -- 172.21.2.101:0/2418666711 shutdown_connections 2021-02-10T11:47:06.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.302 7f804c72d700 1 --2- 172.21.2.101:0/2418666711 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f802c0186c0 0x7f802c01abc0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:06.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.302 7f804c72d700 1 --2- 172.21.2.101:0/2418666711 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f804418e860 0x7f804418ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:06.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.302 7f804c72d700 1 --2- 172.21.2.101:0/2418666711 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f804418d4f0 0x7f804418d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:06.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.302 7f804c72d700 1 --2- 172.21.2.101:0/2418666711 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80441aca10 0x7f80441ace10 unknown :-1 s=CLOSED pgs=3046 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:06.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.302 7f804c72d700 1 -- 172.21.2.101:0/2418666711 >> 172.21.2.101:0/2418666711 conn(0x7f804419c650 msgr2=0x7f8044082ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:06.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.303 7f804c72d700 1 -- 172.21.2.101:0/2418666711 shutdown_connections 2021-02-10T11:47:06.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.303 7f804c72d700 1 -- 172.21.2.101:0/2418666711 wait complete. 2021-02-10T11:47:06.308 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolumegroup rm cephfs group_0000000000357946 2021-02-10T11:47:06.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.726 7fedbe360700 1 Processor -- start 2021-02-10T11:47:06.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.726 7fedbe360700 1 -- start start 2021-02-10T11:47:06.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedbe360700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fedb80d1870 0x7fedb81b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:06.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedbe360700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fedb81b0ae0 0x7fedb8108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:06.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedbe360700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedb80d1c70 0x7fedb8108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:06.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedbe360700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedb80fe1c0 con 0x7fedb81b0ae0 2021-02-10T11:47:06.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedbe360700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedb8109a60 con 0x7fedb80d1870 2021-02-10T11:47:06.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedbe360700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb8088070 con 0x7fedb80d1c70 2021-02-10T11:47:06.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedb77fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fedb81b0ae0 0x7fedb8108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:06.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedb77fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fedb81b0ae0 0x7fedb8108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33790/0 (socket says 172.21.2.101:33790) 2021-02-10T11:47:06.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedb77fe700 1 -- 172.21.2.101:0/1072810618 learned_addr learned my addr 172.21.2.101:0/1072810618 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:06.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedbc97d700 1 --2- 172.21.2.101:0/1072810618 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedb80d1c70 0x7fedb8108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:06.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedb7fff700 1 --2- 172.21.2.101:0/1072810618 >> v2:172.21.2.108:3300/0 conn(0x7fedb80d1870 0x7fedb81b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:06.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedb77fe700 1 -- 172.21.2.101:0/1072810618 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedb80d1c70 msgr2=0x7fedb8108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:06.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedb77fe700 1 --2- 172.21.2.101:0/1072810618 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedb80d1c70 0x7fedb8108f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:06.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedb77fe700 1 -- 172.21.2.101:0/1072810618 >> v2:172.21.2.108:3300/0 conn(0x7fedb80d1870 msgr2=0x7fedb81b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:06.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedb77fe700 1 --2- 172.21.2.101:0/1072810618 >> v2:172.21.2.108:3300/0 conn(0x7fedb80d1870 0x7fedb81b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:06.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedb77fe700 1 -- 172.21.2.101:0/1072810618 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb8109580 con 0x7fedb81b0ae0 2021-02-10T11:47:06.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedb7fff700 1 --2- 172.21.2.101:0/1072810618 >> v2:172.21.2.108:3300/0 conn(0x7fedb80d1870 0x7fedb81b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:47:06.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedb77fe700 1 --2- 172.21.2.101:0/1072810618 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedb81b0ae0 0x7fedb8108940 secure :-1 s=READY pgs=3047 cs=0 l=1 rev1=1 rx=0x7fedac006700 tx=0x7fedac009b80).ready entity=mon.0 client_cookie=30b9caf628914263 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:06.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedb6ffd700 1 -- 172.21.2.101:0/1072810618 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fedac00c0c0 con 0x7fedb81b0ae0 2021-02-10T11:47:06.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedb6ffd700 1 -- 172.21.2.101:0/1072810618 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedac00c3c0 con 0x7fedb81b0ae0 2021-02-10T11:47:06.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedb6ffd700 1 -- 172.21.2.101:0/1072810618 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fedac00c820 con 0x7fedb81b0ae0 2021-02-10T11:47:06.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedbe360700 1 -- 172.21.2.101:0/1072810618 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedb81b0ae0 msgr2=0x7fedb8108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:06.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.727 7fedbe360700 1 --2- 172.21.2.101:0/1072810618 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedb81b0ae0 0x7fedb8108940 secure :-1 s=READY pgs=3047 cs=0 l=1 rev1=1 rx=0x7fedac006700 tx=0x7fedac009b80).stop 2021-02-10T11:47:06.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.728 7fedbe360700 1 -- 172.21.2.101:0/1072810618 shutdown_connections 2021-02-10T11:47:06.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.728 7fedbe360700 1 --2- 172.21.2.101:0/1072810618 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedb80d1c70 0x7fedb8108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:06.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.728 7fedbe360700 1 --2- 172.21.2.101:0/1072810618 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedb81b0ae0 0x7fedb8108940 unknown :-1 s=CLOSED pgs=3047 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:06.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.728 7fedbe360700 1 --2- 172.21.2.101:0/1072810618 >> v2:172.21.2.108:3300/0 conn(0x7fedb80d1870 0x7fedb81b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:06.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.728 7fedbe360700 1 -- 172.21.2.101:0/1072810618 >> 172.21.2.101:0/1072810618 conn(0x7fedb80ce420 msgr2=0x7fedb80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:06.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.728 7fedbe360700 1 -- 172.21.2.101:0/1072810618 shutdown_connections 2021-02-10T11:47:06.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.728 7fedbe360700 1 -- 172.21.2.101:0/1072810618 wait complete. 2021-02-10T11:47:06.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.743 7fedbe360700 1 Processor -- start 2021-02-10T11:47:06.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.743 7fedbe360700 1 -- start start 2021-02-10T11:47:06.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.743 7fedbe360700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedb8085bc0 0x7fedb80863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:06.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.743 7fedbe360700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedb80954c0 0x7fedb8096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:06.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.743 7fedbe360700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fedb80958c0 0x7fedb8096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:06.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.743 7fedbe360700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb8095d80 con 0x7fedb80954c0 2021-02-10T11:47:06.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.743 7fedbe360700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb80bfa60 con 0x7fedb80958c0 2021-02-10T11:47:06.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.743 7fedbe360700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb8085fc0 con 0x7fedb8085bc0 2021-02-10T11:47:06.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.743 7fedb77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedb80954c0 0x7fedb8096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:06.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.743 7fedb77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedb80954c0 0x7fedb8096300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33800/0 (socket says 172.21.2.101:33800) 2021-02-10T11:47:06.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.743 7fedb77fe700 1 -- 172.21.2.101:0/192561011 learned_addr learned my addr 172.21.2.101:0/192561011 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:06.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.744 7fedbc97d700 1 --2- 172.21.2.101:0/192561011 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fedb80958c0 0x7fedb8096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:06.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.744 7fedb77fe700 1 -- 172.21.2.101:0/192561011 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedb8085bc0 msgr2=0x7fedb80863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:06.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.744 7fedb7fff700 1 --2- 172.21.2.101:0/192561011 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedb8085bc0 0x7fedb80863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:06.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.744 7fedb77fe700 1 --2- 172.21.2.101:0/192561011 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedb8085bc0 0x7fedb80863e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.744 7fedb77fe700 1 -- 172.21.2.101:0/192561011 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fedb80958c0 msgr2=0x7fedb8096920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.744 7fedb77fe700 1 --2- 172.21.2.101:0/192561011 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fedb80958c0 0x7fedb8096920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.744 7fedb77fe700 1 -- 172.21.2.101:0/192561011 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb8086c10 con 0x7fedb80954c0 2021-02-10T11:47:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.744 7fedbc97d700 1 --2- 172.21.2.101:0/192561011 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fedb80958c0 0x7fedb8096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:47:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.744 7fedb7fff700 1 --2- 172.21.2.101:0/192561011 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedb8085bc0 0x7fedb80863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:47:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.744 7fedb77fe700 1 --2- 172.21.2.101:0/192561011 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedb80954c0 0x7fedb8096300 secure :-1 s=READY pgs=3048 cs=0 l=1 rev1=1 rx=0x7fedac006420 tx=0x7fedac00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.744 7fedb4ff9700 1 -- 172.21.2.101:0/192561011 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fedac002080 con 0x7fedb80954c0 2021-02-10T11:47:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.744 7fedb4ff9700 1 -- 172.21.2.101:0/192561011 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedac002400 con 0x7fedb80954c0 2021-02-10T11:47:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.744 7fedb4ff9700 1 -- 172.21.2.101:0/192561011 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fedac0028e0 con 0x7fedb80954c0 2021-02-10T11:47:06.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.744 7fedbe360700 1 -- 172.21.2.101:0/192561011 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb8086c10 con 0x7fedb80954c0 2021-02-10T11:47:06.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.744 7fedbe360700 1 -- 172.21.2.101:0/192561011 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb8097230 con 0x7fedb80954c0 2021-02-10T11:47:06.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.745 7fedb4ff9700 1 -- 172.21.2.101:0/192561011 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fedac002020 con 0x7fedb80954c0 2021-02-10T11:47:06.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.745 7fedb4ff9700 1 --2- 172.21.2.101:0/192561011 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fed980186c0 0x7fed9801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:06.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.745 7fedb4ff9700 1 -- 172.21.2.101:0/192561011 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fedac032590 con 0x7fedb80954c0 2021-02-10T11:47:06.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.745 7fedb7fff700 1 --2- 172.21.2.101:0/192561011 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fed980186c0 0x7fed9801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:06.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.745 7fedb7fff700 1 --2- 172.21.2.101:0/192561011 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fed980186c0 0x7fed9801abc0 crc :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:06.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.777 7fedbe360700 1 -- 172.21.2.101:0/192561011 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb80fe1c0 con 0x7fedb80954c0 2021-02-10T11:47:06.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.779 7fedb4ff9700 1 -- 172.21.2.101:0/192561011 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fedac02cf10 con 0x7fedb80954c0 2021-02-10T11:47:06.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.905 7fedbe360700 1 -- 172.21.2.101:0/192561011 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "target": ["mgr", ""], "group_name": "group_0000000000357946"}) v1 -- 0x7fedb8097230 con 0x7fed980186c0 2021-02-10T11:47:06.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.910 7fedb4ff9700 1 -- 172.21.2.101:0/192561011 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7feda8006ad0 con 0x7fed980186c0 2021-02-10T11:47:06.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.914 7fedbe360700 1 -- 172.21.2.101:0/192561011 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fed980186c0 msgr2=0x7fed9801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:06.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.914 7fedbe360700 1 --2- 172.21.2.101:0/192561011 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fed980186c0 0x7fed9801abc0 crc :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:06.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.914 7fedbe360700 1 -- 172.21.2.101:0/192561011 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedb80954c0 msgr2=0x7fedb8096300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:06.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.914 7fedbe360700 1 --2- 172.21.2.101:0/192561011 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedb80954c0 0x7fedb8096300 secure :-1 s=READY pgs=3048 cs=0 l=1 rev1=1 rx=0x7fedac006420 tx=0x7fedac00b960).stop 2021-02-10T11:47:06.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.914 7fedbe360700 1 -- 172.21.2.101:0/192561011 shutdown_connections 2021-02-10T11:47:06.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.914 7fedbe360700 1 --2- 172.21.2.101:0/192561011 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fedb80958c0 0x7fedb8096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:06.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.914 7fedbe360700 1 --2- 172.21.2.101:0/192561011 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fedb80954c0 0x7fedb8096300 unknown :-1 s=CLOSED pgs=3048 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:06.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.914 7fedbe360700 1 --2- 172.21.2.101:0/192561011 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fed980186c0 0x7fed9801abc0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:06.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.914 7fedbe360700 1 --2- 172.21.2.101:0/192561011 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fedb8085bc0 0x7fedb80863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:06.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.914 7fedbe360700 1 -- 172.21.2.101:0/192561011 >> 172.21.2.101:0/192561011 conn(0x7fedb80bef00 msgr2=0x7fedb809d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:06.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.915 7fedbe360700 1 -- 172.21.2.101:0/192561011 shutdown_connections 2021-02-10T11:47:06.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:06.915 7fedbe360700 1 -- 172.21.2.101:0/192561011 wait complete. 2021-02-10T11:47:06.919 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:47:06.920 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:47:07.266 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:47:07.609 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:47:07.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:07.610 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:47:07.610 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:47:07.947 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:47:07.947 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:47:07.947 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:47:07.947 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:47:07.948 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:47:08.303 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[173502]: fuse finished with error 0 and tester_r 0 2021-02-10T11:47:08.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:08.305 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:47:08.305 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:47:08.639 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:47:08.640 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:47:08.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:47:08.640 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:47:08.852 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:47:08.856 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:47:08.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:08.976 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:47:08.977 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:47:09.312 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:47:09.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:47:09.313 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:47:09.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:47:09.313 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:47:09.648 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:09.648 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:47:09.648 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:47:09.983 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:47:09.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:47:09.983 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:47:09.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:47:09.984 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:47:10.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:10.318 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:47:10.318 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:47:10.652 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:47:10.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:47:10.652 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:47:10.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:47:10.652 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:47:10.987 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-02-10T11:47:11.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.406 7fba6267a700 1 Processor -- start 2021-02-10T11:47:11.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.406 7fba6267a700 1 -- start start 2021-02-10T11:47:11.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.407 7fba6267a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fba5c0d1870 0x7fba5c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:11.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.407 7fba6267a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fba5c0f0650 0x7fba5c0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:11.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.407 7fba6267a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba5c0f0b10 con 0x7fba5c1b0a10 2021-02-10T11:47:11.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.407 7fba6267a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba5c0e5ed0 con 0x7fba5c0d1870 2021-02-10T11:47:11.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.407 7fba6267a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba5c0d1c70 con 0x7fba5c0f0650 2021-02-10T11:47:11.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.407 7fba5bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fba5c0d1870 0x7fba5c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:11.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.407 7fba5bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fba5c0d1870 0x7fba5c1b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33804/0 (socket says 172.21.2.101:33804) 2021-02-10T11:47:11.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.407 7fba5bfff700 1 -- 172.21.2.101:0/3582321450 learned_addr learned my addr 172.21.2.101:0/3582321450 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:11.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.407 7fba5bfff700 1 -- 172.21.2.101:0/3582321450 >> v2:172.21.2.108:3300/0 conn(0x7fba5c0f0650 msgr2=0x7fba5c0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:11.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.407 7fba5bfff700 1 --2- 172.21.2.101:0/3582321450 >> v2:172.21.2.108:3300/0 conn(0x7fba5c0f0650 0x7fba5c0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:11.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.407 7fba5bfff700 1 -- 172.21.2.101:0/3582321450 >> v1:172.21.2.101:6789/0 conn(0x7fba5c1b0a10 legacy=0x7fba5c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:11.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.408 7fba5bfff700 1 -- 172.21.2.101:0/3582321450 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5c0d1e90 con 0x7fba5c0d1870 2021-02-10T11:47:11.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.408 7fba5bfff700 1 --2- 172.21.2.101:0/3582321450 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba5c0d1870 0x7fba5c1b03f0 secure :-1 s=READY pgs=3050 cs=0 l=1 rev1=1 rx=0x7fba4c006700 tx=0x7fba4c009b00).ready entity=mon.0 client_cookie=2071917afe46a322 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:11.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.408 7fba5affd700 1 -- 172.21.2.101:0/3582321450 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fba4c00c0c0 con 0x7fba5c0d1870 2021-02-10T11:47:11.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.408 7fba5affd700 1 -- 172.21.2.101:0/3582321450 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4c00c340 con 0x7fba5c0d1870 2021-02-10T11:47:11.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.408 7fba5affd700 1 -- 172.21.2.101:0/3582321450 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fba4c00c820 con 0x7fba5c0d1870 2021-02-10T11:47:11.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.408 7fba6267a700 1 -- 172.21.2.101:0/3582321450 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba5c0d1870 msgr2=0x7fba5c1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:11.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.408 7fba6267a700 1 --2- 172.21.2.101:0/3582321450 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba5c0d1870 0x7fba5c1b03f0 secure :-1 s=READY pgs=3050 cs=0 l=1 rev1=1 rx=0x7fba4c006700 tx=0x7fba4c009b00).stop 2021-02-10T11:47:11.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.409 7fba6267a700 1 -- 172.21.2.101:0/3582321450 shutdown_connections 2021-02-10T11:47:11.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.409 7fba6267a700 1 --2- 172.21.2.101:0/3582321450 >> v2:172.21.2.108:3300/0 conn(0x7fba5c0f0650 0x7fba5c0cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:11.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.409 7fba6267a700 1 --2- 172.21.2.101:0/3582321450 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba5c0d1870 0x7fba5c1b03f0 unknown :-1 s=CLOSED pgs=3050 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:11.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.409 7fba6267a700 1 -- 172.21.2.101:0/3582321450 >> 172.21.2.101:0/3582321450 conn(0x7fba5c0ce450 msgr2=0x7fba5c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:11.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.409 7fba6267a700 1 -- 172.21.2.101:0/3582321450 shutdown_connections 2021-02-10T11:47:11.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.409 7fba6267a700 1 -- 172.21.2.101:0/3582321450 wait complete. 2021-02-10T11:47:11.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.423 7fba6267a700 1 Processor -- start 2021-02-10T11:47:11.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.423 7fba6267a700 1 -- start start 2021-02-10T11:47:11.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.424 7fba6267a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba5c0a2fb0 0x7fba5c0a33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:11.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.424 7fba6267a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba5c0a39d0 0x7fba5c0a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:11.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.424 7fba6267a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba5c0a3dd0 0x7fba5c0a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:11.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.424 7fba6267a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c0a2620 con 0x7fba5c0a39d0 2021-02-10T11:47:11.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.424 7fba6267a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c0a2840 con 0x7fba5c0a3dd0 2021-02-10T11:47:11.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.424 7fba6267a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c0a2b80 con 0x7fba5c0a2fb0 2021-02-10T11:47:11.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.424 7fba5b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba5c0a39d0 0x7fba5c0a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:11.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.424 7fba5b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba5c0a39d0 0x7fba5c0a4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33810/0 (socket says 172.21.2.101:33810) 2021-02-10T11:47:11.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.424 7fba5b7fe700 1 -- 172.21.2.101:0/2360833542 learned_addr learned my addr 172.21.2.101:0/2360833542 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:11.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.424 7fba5b7fe700 1 -- 172.21.2.101:0/2360833542 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba5c0a2fb0 msgr2=0x7fba5c0a33b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:47:11.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.424 7fba5b7fe700 1 --2- 172.21.2.101:0/2360833542 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba5c0a2fb0 0x7fba5c0a33b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:11.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.424 7fba5b7fe700 1 -- 172.21.2.101:0/2360833542 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba5c0a3dd0 msgr2=0x7fba5c0a4e80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:47:11.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.424 7fba5b7fe700 1 --2- 172.21.2.101:0/2360833542 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba5c0a3dd0 0x7fba5c0a4e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:11.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.424 7fba5b7fe700 1 -- 172.21.2.101:0/2360833542 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5c0a54a0 con 0x7fba5c0a39d0 2021-02-10T11:47:11.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.424 7fba5b7fe700 1 --2- 172.21.2.101:0/2360833542 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba5c0a39d0 0x7fba5c0a4860 secure :-1 s=READY pgs=3051 cs=0 l=1 rev1=1 rx=0x7fba50006ed0 tx=0x7fba5000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:11.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.424 7fba58ff9700 1 -- 172.21.2.101:0/2360833542 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fba5000bd50 con 0x7fba5c0a39d0 2021-02-10T11:47:11.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.424 7fba58ff9700 1 -- 172.21.2.101:0/2360833542 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba50006240 con 0x7fba5c0a39d0 2021-02-10T11:47:11.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.424 7fba6267a700 1 -- 172.21.2.101:0/2360833542 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba5c0a54a0 con 0x7fba5c0a39d0 2021-02-10T11:47:11.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.424 7fba58ff9700 1 -- 172.21.2.101:0/2360833542 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fba50005730 con 0x7fba5c0a39d0 2021-02-10T11:47:11.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.424 7fba6267a700 1 -- 172.21.2.101:0/2360833542 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba5c0a54a0 con 0x7fba5c0a39d0 2021-02-10T11:47:11.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.425 7fba58ff9700 1 -- 172.21.2.101:0/2360833542 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fba500054b0 con 0x7fba5c0a39d0 2021-02-10T11:47:11.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.425 7fba58ff9700 1 --2- 172.21.2.101:0/2360833542 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba3c018730 0x7fba3c01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:11.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.425 7fba58ff9700 1 -- 172.21.2.101:0/2360833542 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fba50032620 con 0x7fba5c0a39d0 2021-02-10T11:47:11.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.425 7fba5bfff700 1 --2- 172.21.2.101:0/2360833542 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba3c018730 0x7fba3c01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:11.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.425 7fba5bfff700 1 --2- 172.21.2.101:0/2360833542 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba3c018730 0x7fba3c01ac10 crc :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:11.454 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.457 7fba6267a700 1 -- 172.21.2.101:0/2360833542 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba5c0ee060 con 0x7fba5c0a39d0 2021-02-10T11:47:11.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.459 7fba58ff9700 1 -- 172.21.2.101:0/2360833542 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fba50006030 con 0x7fba5c0a39d0 2021-02-10T11:47:11.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.607 7fba6267a700 1 -- 172.21.2.101:0/2360833542 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fba5c0a5d20 con 0x7fba5c0a39d0 2021-02-10T11:47:11.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.612 7fba58ff9700 1 -- 172.21.2.101:0/2360833542 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7fba500054b0 con 0x7fba5c0a39d0 2021-02-10T11:47:11.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.616 7fba6267a700 1 -- 172.21.2.101:0/2360833542 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba3c018730 msgr2=0x7fba3c01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:11.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.616 7fba6267a700 1 --2- 172.21.2.101:0/2360833542 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba3c018730 0x7fba3c01ac10 crc :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:11.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.616 7fba6267a700 1 -- 172.21.2.101:0/2360833542 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba5c0a39d0 msgr2=0x7fba5c0a4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:11.613 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.616 7fba6267a700 1 --2- 172.21.2.101:0/2360833542 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba5c0a39d0 0x7fba5c0a4860 secure :-1 s=READY pgs=3051 cs=0 l=1 rev1=1 rx=0x7fba50006ed0 tx=0x7fba5000b960).stop 2021-02-10T11:47:11.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.616 7fba6267a700 1 -- 172.21.2.101:0/2360833542 shutdown_connections 2021-02-10T11:47:11.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.616 7fba6267a700 1 --2- 172.21.2.101:0/2360833542 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fba5c0a3dd0 0x7fba5c0a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:11.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.616 7fba6267a700 1 --2- 172.21.2.101:0/2360833542 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fba5c0a39d0 0x7fba5c0a4860 unknown :-1 s=CLOSED pgs=3051 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:11.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.616 7fba6267a700 1 --2- 172.21.2.101:0/2360833542 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fba3c018730 0x7fba3c01ac10 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:11.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.616 7fba6267a700 1 --2- 172.21.2.101:0/2360833542 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fba5c0a2fb0 0x7fba5c0a33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:11.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.616 7fba6267a700 1 -- 172.21.2.101:0/2360833542 >> 172.21.2.101:0/2360833542 conn(0x7fba5c08f030 msgr2=0x7fba5c0b5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:11.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.617 7fba6267a700 1 -- 172.21.2.101:0/2360833542 shutdown_connections 2021-02-10T11:47:11.614 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:11.617 7fba6267a700 1 -- 172.21.2.101:0/2360833542 wait complete. 2021-02-10T11:47:11.622 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_recover_auth_metadata_during_authorize' 2021-02-10T11:47:12.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.036 7fa1486ab700 1 Processor -- start 2021-02-10T11:47:12.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.036 7fa1486ab700 1 -- start start 2021-02-10T11:47:12.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.037 7fa1486ab700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa1400d0340 0x7fa1400ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:12.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.037 7fa1486ab700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fa1400d1c70 0x7fa1400efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:12.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.037 7fa1486ab700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1400f0b10 con 0x7fa1400d1870 2021-02-10T11:47:12.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.037 7fa1486ab700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1400e5ed0 con 0x7fa1400d0340 2021-02-10T11:47:12.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.037 7fa1486ab700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1401b0be0 con 0x7fa1400d1c70 2021-02-10T11:47:12.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.037 7fa145cc6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa1400d0340 0x7fa1400ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:12.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.037 7fa1464c7700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa1400d1870 0x7fa1401b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34312/0 (socket says 172.21.2.101:34312) 2021-02-10T11:47:12.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.037 7fa1464c7700 1 -- 172.21.2.101:0/2405498977 learned_addr learned my addr 172.21.2.101:0/2405498977 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:12.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.037 7fa145cc6700 1 -- 172.21.2.101:0/2405498977 >> v2:172.21.2.108:3300/0 conn(0x7fa1400d1c70 msgr2=0x7fa1400efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:12.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.037 7fa145cc6700 1 --2- 172.21.2.101:0/2405498977 >> v2:172.21.2.108:3300/0 conn(0x7fa1400d1c70 0x7fa1400efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:12.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.037 7fa145cc6700 1 -- 172.21.2.101:0/2405498977 >> v1:172.21.2.101:6789/0 conn(0x7fa1400d1870 legacy=0x7fa1401b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:12.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.037 7fa145cc6700 1 -- 172.21.2.101:0/2405498977 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1400f20a0 con 0x7fa1400d0340 2021-02-10T11:47:12.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.037 7fa145cc6700 1 --2- 172.21.2.101:0/2405498977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1400d0340 0x7fa1400ef640 secure :-1 s=READY pgs=3053 cs=0 l=1 rev1=1 rx=0x7fa134006650 tx=0x7fa134009a80).ready entity=mon.0 client_cookie=8667584abc0ffa71 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:12.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.037 7fa1454c5700 1 -- 172.21.2.101:0/2405498977 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa13400c0c0 con 0x7fa1400d0340 2021-02-10T11:47:12.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.038 7fa1454c5700 1 -- 172.21.2.101:0/2405498977 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13400c340 con 0x7fa1400d0340 2021-02-10T11:47:12.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.038 7fa1454c5700 1 -- 172.21.2.101:0/2405498977 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa13400c860 con 0x7fa1400d0340 2021-02-10T11:47:12.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.038 7fa1486ab700 1 -- 172.21.2.101:0/2405498977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1400d0340 msgr2=0x7fa1400ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:12.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.038 7fa1486ab700 1 --2- 172.21.2.101:0/2405498977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1400d0340 0x7fa1400ef640 secure :-1 s=READY pgs=3053 cs=0 l=1 rev1=1 rx=0x7fa134006650 tx=0x7fa134009a80).stop 2021-02-10T11:47:12.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.038 7fa1486ab700 1 -- 172.21.2.101:0/2405498977 shutdown_connections 2021-02-10T11:47:12.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.038 7fa1486ab700 1 --2- 172.21.2.101:0/2405498977 >> v2:172.21.2.108:3300/0 conn(0x7fa1400d1c70 0x7fa1400efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:12.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.038 7fa1486ab700 1 --2- 172.21.2.101:0/2405498977 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1400d0340 0x7fa1400ef640 unknown :-1 s=CLOSED pgs=3053 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:12.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.038 7fa1486ab700 1 -- 172.21.2.101:0/2405498977 >> 172.21.2.101:0/2405498977 conn(0x7fa1400ce450 msgr2=0x7fa1400d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:12.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.038 7fa1486ab700 1 -- 172.21.2.101:0/2405498977 shutdown_connections 2021-02-10T11:47:12.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.038 7fa1486ab700 1 -- 172.21.2.101:0/2405498977 wait complete. 2021-02-10T11:47:12.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.053 7fa1486ab700 1 Processor -- start 2021-02-10T11:47:12.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.053 7fa1486ab700 1 -- start start 2021-02-10T11:47:12.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.053 7fa1486ab700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1401947b0 0x7fa1400a69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:12.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.053 7fa1486ab700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1400a6fc0 0x7fa1401a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:12.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.053 7fa1486ab700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1401a1bf0 0x7fa1401a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:12.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.053 7fa1486ab700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1400a73c0 con 0x7fa1400a6fc0 2021-02-10T11:47:12.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.053 7fa1486ab700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14018f170 con 0x7fa1401947b0 2021-02-10T11:47:12.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.053 7fa1486ab700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa140194bb0 con 0x7fa1401a1bf0 2021-02-10T11:47:12.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.053 7fa145cc6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1400a6fc0 0x7fa1401a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:12.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.053 7fa145cc6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1400a6fc0 0x7fa1401a15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33826/0 (socket says 172.21.2.101:33826) 2021-02-10T11:47:12.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.054 7fa145cc6700 1 -- 172.21.2.101:0/2243470109 learned_addr learned my addr 172.21.2.101:0/2243470109 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:12.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.054 7fa145cc6700 1 -- 172.21.2.101:0/2243470109 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1401a1bf0 msgr2=0x7fa1401a1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:12.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.054 7fa146cc8700 1 --2- 172.21.2.101:0/2243470109 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1401a1bf0 0x7fa1401a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:12.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.054 7fa1464c7700 1 --2- 172.21.2.101:0/2243470109 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1401947b0 0x7fa1400a69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:12.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.054 7fa145cc6700 1 --2- 172.21.2.101:0/2243470109 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1401a1bf0 0x7fa1401a1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:12.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.054 7fa145cc6700 1 -- 172.21.2.101:0/2243470109 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1401947b0 msgr2=0x7fa1400a69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:12.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.054 7fa145cc6700 1 --2- 172.21.2.101:0/2243470109 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1401947b0 0x7fa1400a69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:12.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.054 7fa145cc6700 1 -- 172.21.2.101:0/2243470109 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14018ec70 con 0x7fa1400a6fc0 2021-02-10T11:47:12.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.054 7fa146cc8700 1 --2- 172.21.2.101:0/2243470109 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1401a1bf0 0x7fa1401a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:47:12.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.054 7fa1464c7700 1 --2- 172.21.2.101:0/2243470109 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1401947b0 0x7fa1400a69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:47:12.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.054 7fa145cc6700 1 --2- 172.21.2.101:0/2243470109 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1400a6fc0 0x7fa1401a15d0 secure :-1 s=READY pgs=3054 cs=0 l=1 rev1=1 rx=0x7fa134009630 tx=0x7fa13400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:12.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.054 7fa132ffd700 1 -- 172.21.2.101:0/2243470109 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa1340093a0 con 0x7fa1400a6fc0 2021-02-10T11:47:12.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.054 7fa132ffd700 1 -- 172.21.2.101:0/2243470109 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13400bd70 con 0x7fa1400a6fc0 2021-02-10T11:47:12.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.054 7fa132ffd700 1 -- 172.21.2.101:0/2243470109 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa134005560 con 0x7fa1400a6fc0 2021-02-10T11:47:12.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.054 7fa1486ab700 1 -- 172.21.2.101:0/2243470109 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa14018ec70 con 0x7fa1400a6fc0 2021-02-10T11:47:12.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.054 7fa1486ab700 1 -- 172.21.2.101:0/2243470109 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1401a2900 con 0x7fa1400a6fc0 2021-02-10T11:47:12.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.055 7fa132ffd700 1 -- 172.21.2.101:0/2243470109 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa13400a000 con 0x7fa1400a6fc0 2021-02-10T11:47:12.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.055 7fa132ffd700 1 --2- 172.21.2.101:0/2243470109 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1280186c0 0x7fa12801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:12.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.055 7fa132ffd700 1 -- 172.21.2.101:0/2243470109 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa1340325a0 con 0x7fa1400a6fc0 2021-02-10T11:47:12.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.055 7fa1464c7700 1 --2- 172.21.2.101:0/2243470109 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1280186c0 0x7fa12801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:12.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.055 7fa1464c7700 1 --2- 172.21.2.101:0/2243470109 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1280186c0 0x7fa12801abc0 crc :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:12.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.086 7fa1486ab700 1 -- 172.21.2.101:0/2243470109 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1400ee0e0 con 0x7fa1400a6fc0 2021-02-10T11:47:12.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.089 7fa132ffd700 1 -- 172.21.2.101:0/2243470109 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa134009340 con 0x7fa1400a6fc0 2021-02-10T11:47:12.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.223 7fa1486ab700 1 -- 172.21.2.101:0/2243470109 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_recover_auth_metadata_during_authorize"]} v 0) v1 -- 0x7fa1400ce450 con 0x7fa1400a6fc0 2021-02-10T11:47:12.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.950 7fa132ffd700 1 -- 172.21.2.101:0/2243470109 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_recover_auth_metadata_during_authorize"]}]=0 v826) v1 ==== 160+0+0 (secure 0 0 0) 0x7fa13400a000 con 0x7fa1400a6fc0 2021-02-10T11:47:12.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.995 7fa1486ab700 1 -- 172.21.2.101:0/2243470109 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1280186c0 msgr2=0x7fa12801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:12.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.995 7fa1486ab700 1 --2- 172.21.2.101:0/2243470109 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1280186c0 0x7fa12801abc0 crc :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:12.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.995 7fa1486ab700 1 -- 172.21.2.101:0/2243470109 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1400a6fc0 msgr2=0x7fa1401a15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:12.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.995 7fa1486ab700 1 --2- 172.21.2.101:0/2243470109 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1400a6fc0 0x7fa1401a15d0 secure :-1 s=READY pgs=3054 cs=0 l=1 rev1=1 rx=0x7fa134009630 tx=0x7fa13400b980).stop 2021-02-10T11:47:12.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.995 7fa1486ab700 1 -- 172.21.2.101:0/2243470109 shutdown_connections 2021-02-10T11:47:12.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.995 7fa1486ab700 1 --2- 172.21.2.101:0/2243470109 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa1280186c0 0x7fa12801abc0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:12.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.995 7fa1486ab700 1 --2- 172.21.2.101:0/2243470109 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa1401a1bf0 0x7fa1401a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:12.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.995 7fa1486ab700 1 --2- 172.21.2.101:0/2243470109 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa1400a6fc0 0x7fa1401a15d0 unknown :-1 s=CLOSED pgs=3054 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:12.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.995 7fa1486ab700 1 --2- 172.21.2.101:0/2243470109 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa1401947b0 0x7fa1400a69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:12.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.995 7fa1486ab700 1 -- 172.21.2.101:0/2243470109 >> 172.21.2.101:0/2243470109 conn(0x7fa1401a38c0 msgr2=0x7fa1401a2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:12.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.995 7fa1486ab700 1 -- 172.21.2.101:0/2243470109 shutdown_connections 2021-02-10T11:47:12.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:12.995 7fa1486ab700 1 -- 172.21.2.101:0/2243470109 wait complete. 2021-02-10T11:47:13.000 INFO:tasks.cephfs_test_runner:test_recover_auth_metadata_during_authorize (tasks.cephfs.test_volumes.TestVolumes) ... ok 2021-02-10T11:47:13.001 INFO:tasks.cephfs_test_runner:Starting test: test_recover_auth_metadata_during_deauthorize (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:47:13.001 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_recover_auth_metadata_during_deauthorize' 2021-02-10T11:47:13.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.416 7f3174570700 1 Processor -- start 2021-02-10T11:47:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.417 7f3174570700 1 -- start start 2021-02-10T11:47:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.417 7f3174570700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f316c1b0470 0x7f316c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.417 7f3174570700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f316c0f20e0 0x7f316c0f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.417 7f3174570700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316c0f0b50 con 0x7f316c0ef610 2021-02-10T11:47:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.417 7f3174570700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f316c0f0820 con 0x7f316c1b0470 2021-02-10T11:47:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.417 7f3174570700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f316c0f24e0 con 0x7f316c0f20e0 2021-02-10T11:47:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.418 7f317238c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f316c1b0470 0x7f316c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.418 7f317238c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f316c1b0470 0x7f316c1b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33830/0 (socket says 172.21.2.101:33830) 2021-02-10T11:47:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.418 7f317238c700 1 -- 172.21.2.101:0/661892106 learned_addr learned my addr 172.21.2.101:0/661892106 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:13.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.418 7f317238c700 1 -- 172.21.2.101:0/661892106 >> v2:172.21.2.108:3300/0 conn(0x7f316c0f20e0 msgr2=0x7f316c0f0200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.418 7f317238c700 1 --2- 172.21.2.101:0/661892106 >> v2:172.21.2.108:3300/0 conn(0x7f316c0f20e0 0x7f316c0f0200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.418 7f317238c700 1 -- 172.21.2.101:0/661892106 >> v1:172.21.2.108:6789/0 conn(0x7f316c0ef610 legacy=0x7f316c0efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.418 7f317238c700 1 -- 172.21.2.101:0/661892106 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316c0e52e0 con 0x7f316c1b0470 2021-02-10T11:47:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.418 7f317238c700 1 --2- 172.21.2.101:0/661892106 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f316c1b0470 0x7f316c1b0870 secure :-1 s=READY pgs=3055 cs=0 l=1 rev1=1 rx=0x7f3160006700 tx=0x7f3160009b80).ready entity=mon.0 client_cookie=674f4d13715ef45a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.418 7f317138a700 1 -- 172.21.2.101:0/661892106 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f316000c0c0 con 0x7f316c1b0470 2021-02-10T11:47:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.418 7f317138a700 1 -- 172.21.2.101:0/661892106 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316000c440 con 0x7f316c1b0470 2021-02-10T11:47:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.418 7f317138a700 1 -- 172.21.2.101:0/661892106 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f316000c920 con 0x7f316c1b0470 2021-02-10T11:47:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.418 7f3174570700 1 -- 172.21.2.101:0/661892106 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f316c1b0470 msgr2=0x7f316c1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.418 7f3174570700 1 --2- 172.21.2.101:0/661892106 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f316c1b0470 0x7f316c1b0870 secure :-1 s=READY pgs=3055 cs=0 l=1 rev1=1 rx=0x7f3160006700 tx=0x7f3160009b80).stop 2021-02-10T11:47:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.419 7f3174570700 1 -- 172.21.2.101:0/661892106 shutdown_connections 2021-02-10T11:47:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.419 7f3174570700 1 --2- 172.21.2.101:0/661892106 >> v2:172.21.2.108:3300/0 conn(0x7f316c0f20e0 0x7f316c0f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.419 7f3174570700 1 --2- 172.21.2.101:0/661892106 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f316c1b0470 0x7f316c1b0870 unknown :-1 s=CLOSED pgs=3055 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.419 7f3174570700 1 -- 172.21.2.101:0/661892106 >> 172.21.2.101:0/661892106 conn(0x7f316c0ce3e0 msgr2=0x7f316c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:13.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.419 7f3174570700 1 -- 172.21.2.101:0/661892106 shutdown_connections 2021-02-10T11:47:13.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.419 7f3174570700 1 -- 172.21.2.101:0/661892106 wait complete. 2021-02-10T11:47:13.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.434 7f3174570700 1 Processor -- start 2021-02-10T11:47:13.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.434 7f3174570700 1 -- start start 2021-02-10T11:47:13.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.434 7f3174570700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f316c0b0970 0x7f316c092ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:13.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.434 7f3174570700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f316c0934f0 0x7f316c094dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:13.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.434 7f3174570700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f316c0938f0 0x7f316c0953e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:13.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.434 7f3174570700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c195ad0 con 0x7f316c0b0970 2021-02-10T11:47:13.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.434 7f3174570700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c095f40 con 0x7f316c0934f0 2021-02-10T11:47:13.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.434 7f3174570700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f316c095a30 con 0x7f316c0938f0 2021-02-10T11:47:13.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.434 7f317238c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f316c0b0970 0x7f316c092ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:13.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.434 7f317238c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f316c0b0970 0x7f316c092ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33838/0 (socket says 172.21.2.101:33838) 2021-02-10T11:47:13.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.434 7f317238c700 1 -- 172.21.2.101:0/1830877116 learned_addr learned my addr 172.21.2.101:0/1830877116 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:13.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.435 7f317238c700 1 -- 172.21.2.101:0/1830877116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f316c0938f0 msgr2=0x7f316c0953e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:47:13.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.435 7f317238c700 1 --2- 172.21.2.101:0/1830877116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f316c0938f0 0x7f316c0953e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:13.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.435 7f317238c700 1 -- 172.21.2.101:0/1830877116 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f316c0934f0 msgr2=0x7f316c094dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:47:13.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.435 7f317238c700 1 --2- 172.21.2.101:0/1830877116 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f316c0934f0 0x7f316c094dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:13.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.435 7f317238c700 1 -- 172.21.2.101:0/1830877116 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316c095c50 con 0x7f316c0b0970 2021-02-10T11:47:13.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.435 7f317238c700 1 --2- 172.21.2.101:0/1830877116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f316c0b0970 0x7f316c092ed0 secure :-1 s=READY pgs=3056 cs=0 l=1 rev1=1 rx=0x7f316000b5e0 tx=0x7f316000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:13.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.435 7f315effd700 1 -- 172.21.2.101:0/1830877116 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f316000bd70 con 0x7f316c0b0970 2021-02-10T11:47:13.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.435 7f315effd700 1 -- 172.21.2.101:0/1830877116 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3160002030 con 0x7f316c0b0970 2021-02-10T11:47:13.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.435 7f315effd700 1 -- 172.21.2.101:0/1830877116 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3160002550 con 0x7f316c0b0970 2021-02-10T11:47:13.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.435 7f3174570700 1 -- 172.21.2.101:0/1830877116 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316c095c50 con 0x7f316c0b0970 2021-02-10T11:47:13.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.435 7f3174570700 1 -- 172.21.2.101:0/1830877116 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316c0967b0 con 0x7f316c0b0970 2021-02-10T11:47:13.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.436 7f315effd700 1 -- 172.21.2.101:0/1830877116 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3160000c90 con 0x7f316c0b0970 2021-02-10T11:47:13.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.436 7f315effd700 1 --2- 172.21.2.101:0/1830877116 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f31540186c0 0x7f315401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:13.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.436 7f315effd700 1 -- 172.21.2.101:0/1830877116 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f31600325c0 con 0x7f316c0b0970 2021-02-10T11:47:13.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.436 7f3171b8b700 1 --2- 172.21.2.101:0/1830877116 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f31540186c0 0x7f315401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:13.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.436 7f3171b8b700 1 --2- 172.21.2.101:0/1830877116 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f31540186c0 0x7f315401abc0 crc :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:13.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.467 7f3174570700 1 -- 172.21.2.101:0/1830877116 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f316c0e52e0 con 0x7f316c0b0970 2021-02-10T11:47:13.467 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.470 7f315effd700 1 -- 172.21.2.101:0/1830877116 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3160030d70 con 0x7f316c0b0970 2021-02-10T11:47:13.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.602 7f3174570700 1 -- 172.21.2.101:0/1830877116 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_recover_auth_metadata_during_deauthorize"]} v 0) v1 -- 0x7f316c0e52e0 con 0x7f316c0b0970 2021-02-10T11:47:13.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.953 7f315effd700 1 -- 172.21.2.101:0/1830877116 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_recover_auth_metadata_during_deauthorize"]}]=0 v827) v1 ==== 165+0+0 (secure 0 0 0) 0x7f316001f550 con 0x7f316c0b0970 2021-02-10T11:47:13.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.970 7f3174570700 1 -- 172.21.2.101:0/1830877116 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f31540186c0 msgr2=0x7f315401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:13.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.970 7f3174570700 1 --2- 172.21.2.101:0/1830877116 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f31540186c0 0x7f315401abc0 crc :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:13.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.970 7f3174570700 1 -- 172.21.2.101:0/1830877116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f316c0b0970 msgr2=0x7f316c092ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:13.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.970 7f3174570700 1 --2- 172.21.2.101:0/1830877116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f316c0b0970 0x7f316c092ed0 secure :-1 s=READY pgs=3056 cs=0 l=1 rev1=1 rx=0x7f316000b5e0 tx=0x7f316000b980).stop 2021-02-10T11:47:13.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.971 7f3174570700 1 -- 172.21.2.101:0/1830877116 shutdown_connections 2021-02-10T11:47:13.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.971 7f3174570700 1 --2- 172.21.2.101:0/1830877116 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f31540186c0 0x7f315401abc0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:13.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.971 7f3174570700 1 --2- 172.21.2.101:0/1830877116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f316c0938f0 0x7f316c0953e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:13.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.971 7f3174570700 1 --2- 172.21.2.101:0/1830877116 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f316c0934f0 0x7f316c094dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:13.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.971 7f3174570700 1 --2- 172.21.2.101:0/1830877116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f316c0b0970 0x7f316c092ed0 unknown :-1 s=CLOSED pgs=3056 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:13.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.971 7f3174570700 1 -- 172.21.2.101:0/1830877116 >> 172.21.2.101:0/1830877116 conn(0x7f316c19f7a0 msgr2=0x7f316c0940e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:13.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.971 7f3174570700 1 -- 172.21.2.101:0/1830877116 shutdown_connections 2021-02-10T11:47:13.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:13.971 7f3174570700 1 -- 172.21.2.101:0/1830877116 wait complete. 2021-02-10T11:47:13.976 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:47:14.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.394 7f3530185700 1 Processor -- start 2021-02-10T11:47:14.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.395 7f3530185700 1 -- start start 2021-02-10T11:47:14.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.395 7f3530185700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35280d1c70 0x7f35280cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:14.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.395 7f3530185700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f35280cf580 0x7f35280cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:14.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.395 7f3530185700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35280fe1c0 con 0x7f35280d1870 2021-02-10T11:47:14.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.395 7f3530185700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35281b0cb0 con 0x7f35280cf580 2021-02-10T11:47:14.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.395 7f3530185700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3528109a60 con 0x7f35280d1c70 2021-02-10T11:47:14.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.395 7f352e7a2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f35280cf580 0x7f35280cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:14.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.395 7f352e7a2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f35280cf580 0x7f35280cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33846/0 (socket says 172.21.2.101:33846) 2021-02-10T11:47:14.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.395 7f352e7a2700 1 -- 172.21.2.101:0/4047693956 learned_addr learned my addr 172.21.2.101:0/4047693956 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:14.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.396 7f352e7a2700 1 -- 172.21.2.101:0/4047693956 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35280d1c70 msgr2=0x7f35280cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:14.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.396 7f352e7a2700 1 --2- 172.21.2.101:0/4047693956 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35280d1c70 0x7f35280cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:14.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.396 7f352e7a2700 1 -- 172.21.2.101:0/4047693956 >> v1:172.21.2.108:6789/0 conn(0x7f35280d1870 legacy=0x7f35281b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:14.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.396 7f352e7a2700 1 -- 172.21.2.101:0/4047693956 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3528088070 con 0x7f35280cf580 2021-02-10T11:47:14.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.396 7f352e7a2700 1 --2- 172.21.2.101:0/4047693956 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35280cf580 0x7f35280cf980 secure :-1 s=READY pgs=3057 cs=0 l=1 rev1=1 rx=0x7f3520006f00 tx=0x7f3520009f10).ready entity=mon.0 client_cookie=d0c62db9c2b34fc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:14.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.396 7f352cf9f700 1 -- 172.21.2.101:0/4047693956 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f352000b3b0 con 0x7f35280cf580 2021-02-10T11:47:14.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.396 7f352cf9f700 1 -- 172.21.2.101:0/4047693956 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352000b730 con 0x7f35280cf580 2021-02-10T11:47:14.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.396 7f352cf9f700 1 -- 172.21.2.101:0/4047693956 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f352000bb90 con 0x7f35280cf580 2021-02-10T11:47:14.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.396 7f3530185700 1 -- 172.21.2.101:0/4047693956 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35280cf580 msgr2=0x7f35280cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:14.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.396 7f3530185700 1 --2- 172.21.2.101:0/4047693956 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35280cf580 0x7f35280cf980 secure :-1 s=READY pgs=3057 cs=0 l=1 rev1=1 rx=0x7f3520006f00 tx=0x7f3520009f10).stop 2021-02-10T11:47:14.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.397 7f3530185700 1 -- 172.21.2.101:0/4047693956 shutdown_connections 2021-02-10T11:47:14.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.397 7f3530185700 1 --2- 172.21.2.101:0/4047693956 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35280d1c70 0x7f35280cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:14.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.397 7f3530185700 1 --2- 172.21.2.101:0/4047693956 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35280cf580 0x7f35280cf980 unknown :-1 s=CLOSED pgs=3057 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:14.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.397 7f3530185700 1 -- 172.21.2.101:0/4047693956 >> 172.21.2.101:0/4047693956 conn(0x7f35280ce420 msgr2=0x7f35280d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:14.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.397 7f3530185700 1 -- 172.21.2.101:0/4047693956 shutdown_connections 2021-02-10T11:47:14.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.397 7f3530185700 1 -- 172.21.2.101:0/4047693956 wait complete. 2021-02-10T11:47:14.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.412 7f3530185700 1 Processor -- start 2021-02-10T11:47:14.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.412 7f3530185700 1 -- start start 2021-02-10T11:47:14.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.412 7f3530185700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35281a4a30 0x7f352809e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:14.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.412 7f3530185700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f352809ec80 0x7f352809f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:14.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.412 7f3530185700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35280bd6e0 0x7f35280bdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:14.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.412 7f3530185700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352809f720 con 0x7f35281a4a30 2021-02-10T11:47:14.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.412 7f3530185700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35280befe0 con 0x7f352809ec80 2021-02-10T11:47:14.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.412 7f3530185700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35280bf520 con 0x7f35280bd6e0 2021-02-10T11:47:14.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.412 7f352dfa1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35281a4a30 0x7f352809e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:14.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.412 7f352dfa1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35281a4a30 0x7f352809e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33848/0 (socket says 172.21.2.101:33848) 2021-02-10T11:47:14.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.412 7f352dfa1700 1 -- 172.21.2.101:0/1509691285 learned_addr learned my addr 172.21.2.101:0/1509691285 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:14.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.413 7f352dfa1700 1 -- 172.21.2.101:0/1509691285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35280bd6e0 msgr2=0x7f35280bdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:14.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.413 7f352dfa1700 1 --2- 172.21.2.101:0/1509691285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35280bd6e0 0x7f35280bdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:14.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.413 7f352dfa1700 1 -- 172.21.2.101:0/1509691285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f352809ec80 msgr2=0x7f352809f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:14.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.413 7f352dfa1700 1 --2- 172.21.2.101:0/1509691285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f352809ec80 0x7f352809f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:14.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.413 7f352dfa1700 1 -- 172.21.2.101:0/1509691285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35281905b0 con 0x7f35281a4a30 2021-02-10T11:47:14.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.413 7f352dfa1700 1 --2- 172.21.2.101:0/1509691285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35281a4a30 0x7f352809e660 secure :-1 s=READY pgs=3058 cs=0 l=1 rev1=1 rx=0x7f352400b530 tx=0x7f352400b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:14.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.413 7f351a7fc700 1 -- 172.21.2.101:0/1509691285 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f352400bcc0 con 0x7f35281a4a30 2021-02-10T11:47:14.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.413 7f351a7fc700 1 -- 172.21.2.101:0/1509691285 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3524006020 con 0x7f35281a4a30 2021-02-10T11:47:14.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.413 7f351a7fc700 1 -- 172.21.2.101:0/1509691285 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f35240054a0 con 0x7f35281a4a30 2021-02-10T11:47:14.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.413 7f3530185700 1 -- 172.21.2.101:0/1509691285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35281905b0 con 0x7f35281a4a30 2021-02-10T11:47:14.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.413 7f3530185700 1 -- 172.21.2.101:0/1509691285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35281905b0 con 0x7f35281a4a30 2021-02-10T11:47:14.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.414 7f351a7fc700 1 -- 172.21.2.101:0/1509691285 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3524006200 con 0x7f35281a4a30 2021-02-10T11:47:14.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.414 7f351a7fc700 1 --2- 172.21.2.101:0/1509691285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f35100186c0 0x7f351001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:14.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.414 7f351a7fc700 1 -- 172.21.2.101:0/1509691285 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3524032580 con 0x7f35281a4a30 2021-02-10T11:47:14.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.414 7f352d7a0700 1 --2- 172.21.2.101:0/1509691285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f35100186c0 0x7f351001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:14.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.414 7f352d7a0700 1 --2- 172.21.2.101:0/1509691285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f35100186c0 0x7f351001abc0 crc :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:14.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.445 7f3530185700 1 -- 172.21.2.101:0/1509691285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3528082a70 con 0x7f35281a4a30 2021-02-10T11:47:14.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.447 7f351a7fc700 1 -- 172.21.2.101:0/1509691285 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3524030d30 con 0x7f35281a4a30 2021-02-10T11:47:14.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.598 7f3530185700 1 -- 172.21.2.101:0/1509691285 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3528082a70 con 0x7f35281a4a30 2021-02-10T11:47:14.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.602 7f351a7fc700 1 -- 172.21.2.101:0/1509691285 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v44)=0 v44) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3524006200 con 0x7f35281a4a30 2021-02-10T11:47:14.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.607 7f3530185700 1 -- 172.21.2.101:0/1509691285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f35100186c0 msgr2=0x7f351001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:14.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.607 7f3530185700 1 --2- 172.21.2.101:0/1509691285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f35100186c0 0x7f351001abc0 crc :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:14.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.607 7f3530185700 1 -- 172.21.2.101:0/1509691285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35281a4a30 msgr2=0x7f352809e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:14.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.607 7f3530185700 1 --2- 172.21.2.101:0/1509691285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35281a4a30 0x7f352809e660 secure :-1 s=READY pgs=3058 cs=0 l=1 rev1=1 rx=0x7f352400b530 tx=0x7f352400b8d0).stop 2021-02-10T11:47:14.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.607 7f3530185700 1 -- 172.21.2.101:0/1509691285 shutdown_connections 2021-02-10T11:47:14.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.607 7f3530185700 1 --2- 172.21.2.101:0/1509691285 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f35100186c0 0x7f351001abc0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:14.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.607 7f3530185700 1 --2- 172.21.2.101:0/1509691285 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f35280bd6e0 0x7f35280bdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:14.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.607 7f3530185700 1 --2- 172.21.2.101:0/1509691285 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f352809ec80 0x7f352809f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:14.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.607 7f3530185700 1 --2- 172.21.2.101:0/1509691285 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f35281a4a30 0x7f352809e660 unknown :-1 s=CLOSED pgs=3058 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:14.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.607 7f3530185700 1 -- 172.21.2.101:0/1509691285 >> 172.21.2.101:0/1509691285 conn(0x7f3528093c10 msgr2=0x7f35280c3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:14.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.607 7f3530185700 1 -- 172.21.2.101:0/1509691285 shutdown_connections 2021-02-10T11:47:14.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:14.607 7f3530185700 1 -- 172.21.2.101:0/1509691285 wait complete. 2021-02-10T11:47:14.613 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:47:14.613 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:47:14.961 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:47:15.306 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:47:15.306 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:15.306 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:47:15.306 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:47:15.641 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2021-02-10T11:47:15.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:47:15.642 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-02-10T11:47:15.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:15.642 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:47:15.643 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:47:15.978 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:47:15.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:47:15.978 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:47:15.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:15.979 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:47:15.979 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:47:16.314 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:47:16.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:47:16.315 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:47:16.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:16.315 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:47:16.315 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:47:16.650 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:47:16.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:47:16.650 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:47:16.650 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json-pretty 2021-02-10T11:47:17.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.080 7f300cfb2700 1 Processor -- start 2021-02-10T11:47:17.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.080 7f300cfb2700 1 -- start start 2021-02-10T11:47:17.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.081 7f300cfb2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30080d0340 0x7f30080ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:17.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.081 7f300cfb2700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30080f0b10 con 0x7f30080d1870 2021-02-10T11:47:17.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.081 7f300cfb2700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30081b0be0 con 0x7f30080d1c70 2021-02-10T11:47:17.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.081 7f300cfb2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30080e5ed0 con 0x7f30080d0340 2021-02-10T11:47:17.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.081 7f3006d9d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f30080d1870 0x7f30081b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34350/0 (socket says 172.21.2.101:34350) 2021-02-10T11:47:17.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.081 7f3006d9d700 1 -- 172.21.2.101:0/1423628884 learned_addr learned my addr 172.21.2.101:0/1423628884 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:17.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.081 7f3005d9b700 1 -- 172.21.2.101:0/1423628884 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 299561877 0 0) 0x7f2ff0001180 con 0x7f30080d1870 2021-02-10T11:47:17.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.081 7f3005d9b700 1 -- 172.21.2.101:0/1423628884 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ff4002fe0 con 0x7f30080d1870 2021-02-10T11:47:17.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.081 7f3005d9b700 1 -- 172.21.2.101:0/1423628884 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3597636097 0 0) 0x7f2ff0001180 con 0x7f30080d1870 2021-02-10T11:47:17.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.081 7f300659c700 1 --2- 172.21.2.101:0/1423628884 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30080d0340 0x7f30080ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:17.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.081 7f3005d9b700 1 -- 172.21.2.101:0/1423628884 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30080d0340 msgr2=0x7f30080ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:17.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.081 7f3005d9b700 1 --2- 172.21.2.101:0/1423628884 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30080d0340 0x7f30080ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:17.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.082 7f3005d9b700 1 -- 172.21.2.101:0/1423628884 >> v1:172.21.2.108:6789/0 conn(0x7f30080d1c70 legacy=0x7f30080efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:17.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.082 7f3005d9b700 1 -- 172.21.2.101:0/1423628884 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30080f0450 con 0x7f30080d1870 2021-02-10T11:47:17.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.082 7f3005d9b700 1 -- 172.21.2.101:0/1423628884 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f2ff0001680 con 0x7f30080d1870 2021-02-10T11:47:17.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.082 7f300659c700 1 --2- 172.21.2.101:0/1423628884 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30080d0340 0x7f30080ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:47:17.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.082 7f3005d9b700 1 -- 172.21.2.101:0/1423628884 >> v1:172.21.2.101:6789/0 conn(0x7f30080d1870 legacy=0x7f30081b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:17.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.082 7f3005d9b700 1 --2- 172.21.2.101:0/1423628884 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ff4003aa0 0x7f2ff4005eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:17.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.082 7f3005d9b700 1 --2- 172.21.2.101:0/1423628884 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ff4006750 0x7f2ff4008c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:17.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.082 7f3005d9b700 1 --2- 172.21.2.101:0/1423628884 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ff40094f0 0x7f2ff400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:17.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.082 7f3005d9b700 1 -- 172.21.2.101:0/1423628884 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff400c2c0 con 0x7f2ff4006750 2021-02-10T11:47:17.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.082 7f3005d9b700 1 -- 172.21.2.101:0/1423628884 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff400c4e0 con 0x7f2ff40094f0 2021-02-10T11:47:17.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.082 7f3005d9b700 1 -- 172.21.2.101:0/1423628884 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff400c700 con 0x7f2ff4003aa0 2021-02-10T11:47:17.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.082 7f300659c700 1 --2- 172.21.2.101:0/1423628884 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ff4006750 0x7f2ff4008c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:17.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.082 7f3006d9d700 1 --2- 172.21.2.101:0/1423628884 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ff4003aa0 0x7f2ff4005eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:17.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.082 7f300759e700 1 --2- 172.21.2.101:0/1423628884 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ff40094f0 0x7f2ff400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:17.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.082 7f3006d9d700 1 -- 172.21.2.101:0/1423628884 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ff40094f0 msgr2=0x7f2ff400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:17.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.082 7f3006d9d700 1 --2- 172.21.2.101:0/1423628884 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2ff40094f0 0x7f2ff400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:17.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.082 7f3006d9d700 1 -- 172.21.2.101:0/1423628884 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ff4006750 msgr2=0x7f2ff4008c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:17.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.082 7f3006d9d700 1 --2- 172.21.2.101:0/1423628884 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ff4006750 0x7f2ff4008c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:17.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.082 7f3006d9d700 1 -- 172.21.2.101:0/1423628884 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff400c9a0 con 0x7f2ff4003aa0 2021-02-10T11:47:17.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.082 7f300659c700 1 --2- 172.21.2.101:0/1423628884 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2ff4006750 0x7f2ff4008c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:47:17.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.082 7f3006d9d700 1 --2- 172.21.2.101:0/1423628884 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ff4003aa0 0x7f2ff4005eb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f2ff0001880 tx=0x7f2ff0007c70).ready entity=mon.2 client_cookie=5d13c2c60463054e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:17.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.082 7f3005d9b700 1 -- 172.21.2.101:0/1423628884 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2ff00070c0 con 0x7f2ff4003aa0 2021-02-10T11:47:17.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.083 7f3005d9b700 1 -- 172.21.2.101:0/1423628884 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff000a520 con 0x7f2ff4003aa0 2021-02-10T11:47:17.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.083 7f3005d9b700 1 -- 172.21.2.101:0/1423628884 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2ff000aa40 con 0x7f2ff4003aa0 2021-02-10T11:47:17.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.083 7f300cfb2700 1 -- 172.21.2.101:0/1423628884 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ff4003aa0 msgr2=0x7f2ff4005eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:17.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.083 7f300cfb2700 1 --2- 172.21.2.101:0/1423628884 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2ff4003aa0 0x7f2ff4005eb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f2ff0001880 tx=0x7f2ff0007c70).stop 2021-02-10T11:47:17.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.083 7f300759e700 1 -- 172.21.2.101:0/1423628884 reap_dead start 2021-02-10T11:47:17.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.083 7f300cfb2700 1 -- 172.21.2.101:0/1423628884 shutdown_connections 2021-02-10T11:47:17.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.083 7f300cfb2700 1 -- 172.21.2.101:0/1423628884 >> 172.21.2.101:0/1423628884 conn(0x7f30080ce450 msgr2=0x7f30080d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:17.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.083 7f300cfb2700 1 -- 172.21.2.101:0/1423628884 shutdown_connections 2021-02-10T11:47:17.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.083 7f300cfb2700 1 -- 172.21.2.101:0/1423628884 wait complete. 2021-02-10T11:47:17.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.098 7f300cfb2700 1 Processor -- start 2021-02-10T11:47:17.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.098 7f300cfb2700 1 -- start start 2021-02-10T11:47:17.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.098 7f300cfb2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3008195650 0x7f30080b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:17.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.098 7f300cfb2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f30080b5b80 0x7f30081a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:17.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.098 7f300cfb2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30080acfa0 0x7f30081a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:17.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.098 7f300cfb2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30080b6010 con 0x7f3008195650 2021-02-10T11:47:17.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.098 7f300cfb2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30081a3a60 con 0x7f30080b5b80 2021-02-10T11:47:17.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.098 7f300cfb2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30081a3c40 con 0x7f30080acfa0 2021-02-10T11:47:17.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.099 7f3006d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3008195650 0x7f30080b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:17.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.099 7f3006d9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3008195650 0x7f30080b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33870/0 (socket says 172.21.2.101:33870) 2021-02-10T11:47:17.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.099 7f3006d9d700 1 -- 172.21.2.101:0/1772469146 learned_addr learned my addr 172.21.2.101:0/1772469146 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:17.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.099 7f300759e700 1 --2- 172.21.2.101:0/1772469146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30080acfa0 0x7f30081a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:17.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.099 7f300659c700 1 --2- 172.21.2.101:0/1772469146 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f30080b5b80 0x7f30081a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:17.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.099 7f3006d9d700 1 -- 172.21.2.101:0/1772469146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30080acfa0 msgr2=0x7f30081a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:17.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.099 7f3006d9d700 1 --2- 172.21.2.101:0/1772469146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30080acfa0 0x7f30081a3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:17.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.099 7f3006d9d700 1 -- 172.21.2.101:0/1772469146 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f30080b5b80 msgr2=0x7f30081a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:17.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.099 7f3006d9d700 1 --2- 172.21.2.101:0/1772469146 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f30080b5b80 0x7f30081a2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:17.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.099 7f3006d9d700 1 -- 172.21.2.101:0/1772469146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30080bcae0 con 0x7f3008195650 2021-02-10T11:47:17.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.099 7f3006d9d700 1 --2- 172.21.2.101:0/1772469146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3008195650 0x7f30080b5560 secure :-1 s=READY pgs=3060 cs=0 l=1 rev1=1 rx=0x7f2ff000a540 tx=0x7f2ff000a8e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:17.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.100 7f2fff7fe700 1 -- 172.21.2.101:0/1772469146 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2ff00013a0 con 0x7f3008195650 2021-02-10T11:47:17.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.100 7f2fff7fe700 1 -- 172.21.2.101:0/1772469146 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff00043a0 con 0x7f3008195650 2021-02-10T11:47:17.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.100 7f2fff7fe700 1 -- 172.21.2.101:0/1772469146 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2ff0004840 con 0x7f3008195650 2021-02-10T11:47:17.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.100 7f300cfb2700 1 -- 172.21.2.101:0/1772469146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30080bcae0 con 0x7f3008195650 2021-02-10T11:47:17.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.100 7f300cfb2700 1 -- 172.21.2.101:0/1772469146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30080bcae0 con 0x7f3008195650 2021-02-10T11:47:17.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.100 7f2fff7fe700 1 -- 172.21.2.101:0/1772469146 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2ff00013a0 con 0x7f3008195650 2021-02-10T11:47:17.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.100 7f2fff7fe700 1 --2- 172.21.2.101:0/1772469146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2fec0186c0 0x7f2fec01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:17.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.100 7f2fff7fe700 1 -- 172.21.2.101:0/1772469146 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2ff00327c0 con 0x7f3008195650 2021-02-10T11:47:17.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.101 7f300659c700 1 --2- 172.21.2.101:0/1772469146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2fec0186c0 0x7f2fec01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:17.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.101 7f300659c700 1 --2- 172.21.2.101:0/1772469146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2fec0186c0 0x7f2fec01abc0 crc :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:17.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.131 7f300cfb2700 1 -- 172.21.2.101:0/1772469146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30080b6010 con 0x7f3008195650 2021-02-10T11:47:17.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.133 7f2fff7fe700 1 -- 172.21.2.101:0/1772469146 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2ff001f500 con 0x7f3008195650 2021-02-10T11:47:17.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.258 7f300cfb2700 1 -- 172.21.2.101:0/1772469146 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f30080b6010 con 0x7f3008195650 2021-02-10T11:47:17.255 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.259 7f2fff7fe700 1 -- 172.21.2.101:0/1772469146 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v175) v1 ==== 81+0+30374 (secure 0 0 0) 0x7f2ff00013a0 con 0x7f3008195650 2021-02-10T11:47:17.256 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:47:17.256 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:47:17.256 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 175, 2021-02-10T11:47:17.257 INFO:teuthology.orchestra.run.gibba001.stdout: "fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:47:17.257 INFO:teuthology.orchestra.run.gibba001.stdout: "created": "2021-02-10 11:12:36.788366", 2021-02-10T11:47:17.257 INFO:teuthology.orchestra.run.gibba001.stdout: "modified": "2021-02-10 11:46:36.152442", 2021-02-10T11:47:17.257 INFO:teuthology.orchestra.run.gibba001.stdout: "last_up_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:47:17.257 INFO:teuthology.orchestra.run.gibba001.stdout: "last_in_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:47:17.257 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-02-10T11:47:17.257 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_num": 5799936, 2021-02-10T11:47:17.257 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_set": [ 2021-02-10T11:47:17.257 INFO:teuthology.orchestra.run.gibba001.stdout: "pglog_hardlimit", 2021-02-10T11:47:17.257 INFO:teuthology.orchestra.run.gibba001.stdout: "purged_snapdirs", 2021-02-10T11:47:17.257 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_deletes", 2021-02-10T11:47:17.257 INFO:teuthology.orchestra.run.gibba001.stdout: "sortbitwise" 2021-02-10T11:47:17.257 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:47:17.258 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_version": 4, 2021-02-10T11:47:17.258 INFO:teuthology.orchestra.run.gibba001.stdout: "full_ratio": 0.94999998807907104, 2021-02-10T11:47:17.258 INFO:teuthology.orchestra.run.gibba001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-02-10T11:47:17.258 INFO:teuthology.orchestra.run.gibba001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-02-10T11:47:17.258 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_snapshot": "", 2021-02-10T11:47:17.258 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_max": 37, 2021-02-10T11:47:17.258 INFO:teuthology.orchestra.run.gibba001.stdout: "max_osd": 8, 2021-02-10T11:47:17.258 INFO:teuthology.orchestra.run.gibba001.stdout: "require_min_compat_client": "jewel", 2021-02-10T11:47:17.258 INFO:teuthology.orchestra.run.gibba001.stdout: "min_compat_client": "jewel", 2021-02-10T11:47:17.258 INFO:teuthology.orchestra.run.gibba001.stdout: "require_osd_release": "nautilus", 2021-02-10T11:47:17.258 INFO:teuthology.orchestra.run.gibba001.stdout: "pools": [ 2021-02-10T11:47:17.258 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.259 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 1, 2021-02-10T11:47:17.259 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "rbd", 2021-02-10T11:47:17.259 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:13:08.037472", 2021-02-10T11:47:17.259 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:47:17.259 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:47:17.259 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:47:17.259 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:47:17.259 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:47:17.259 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:47:17.259 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:47:17.259 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:47:17.259 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:47:17.259 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:47:17.259 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:47:17.260 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:47:17.260 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:47:17.260 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:47:17.260 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:47:17.260 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:47:17.260 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:47:17.260 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:47:17.260 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:47:17.260 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:47:17.260 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.260 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "16", 2021-02-10T11:47:17.260 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:47:17.260 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:47:17.260 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:47:17.261 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:47:17.261 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:47:17.261 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:47:17.261 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:47:17.261 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:47:17.261 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:47:17.261 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:47:17.261 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:47:17.261 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:47:17.261 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:47:17.261 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:47:17.261 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:47:17.261 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:47:17.262 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:47:17.262 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:47:17.262 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:47:17.262 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:47:17.262 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:47:17.262 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:47:17.262 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:47:17.262 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:47:17.262 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:47:17.262 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:47:17.262 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.262 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:47:17.262 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:47:17.263 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:47:17.263 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:47:17.263 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:47:17.263 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:47:17.263 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:47:17.263 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:47:17.263 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:47:17.263 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:47:17.263 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:47:17.263 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:47:17.263 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:47:17.263 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd": {} 2021-02-10T11:47:17.264 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.264 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.264 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.264 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 36, 2021-02-10T11:47:17.264 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_metadata", 2021-02-10T11:47:17.264 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:46:33.067171", 2021-02-10T11:47:17.264 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:47:17.264 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:47:17.264 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:47:17.264 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:47:17.264 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:47:17.264 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:47:17.264 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:47:17.265 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:47:17.265 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:47:17.265 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:47:17.265 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:47:17.265 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:47:17.265 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:47:17.265 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:47:17.265 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:47:17.265 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:47:17.265 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:47:17.265 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:47:17.265 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:47:17.265 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:47:17.266 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.266 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "174", 2021-02-10T11:47:17.266 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:47:17.266 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:47:17.266 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:47:17.266 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:47:17.266 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:47:17.266 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:47:17.266 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:47:17.266 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:47:17.266 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:47:17.266 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:47:17.267 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:47:17.267 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:47:17.267 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:47:17.267 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:47:17.267 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:47:17.267 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:47:17.267 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:47:17.267 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:47:17.267 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:47:17.267 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:47:17.267 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:47:17.267 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:47:17.267 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:47:17.267 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:47:17.268 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:47:17.268 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:47:17.268 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.268 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:47:17.268 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:47:17.268 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:47:17.268 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:47:17.268 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:47:17.268 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:47:17.268 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:47:17.268 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:47:17.268 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:47:17.268 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:47:17.268 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:47:17.269 INFO:teuthology.orchestra.run.gibba001.stdout: "options": { 2021-02-10T11:47:17.269 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_bias": 4, 2021-02-10T11:47:17.269 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_min": 16, 2021-02-10T11:47:17.269 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_priority": 5 2021-02-10T11:47:17.269 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.269 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:47:17.269 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:47:17.269 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": "cephfs" 2021-02-10T11:47:17.269 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.269 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.269 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.269 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.269 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 37, 2021-02-10T11:47:17.269 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_data", 2021-02-10T11:47:17.270 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:46:34.142991", 2021-02-10T11:47:17.270 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:47:17.270 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:47:17.270 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:47:17.270 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:47:17.270 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:47:17.270 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:47:17.270 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:47:17.270 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:47:17.270 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:47:17.270 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:47:17.270 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:47:17.270 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:47:17.271 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:47:17.271 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:47:17.271 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:47:17.271 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:47:17.271 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:47:17.271 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:47:17.271 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:47:17.271 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:47:17.271 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.271 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "175", 2021-02-10T11:47:17.271 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:47:17.271 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:47:17.271 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:47:17.271 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:47:17.272 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:47:17.272 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:47:17.272 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:47:17.272 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:47:17.272 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:47:17.272 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:47:17.272 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:47:17.272 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:47:17.272 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:47:17.272 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:47:17.272 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:47:17.272 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:47:17.272 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:47:17.272 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:47:17.273 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:47:17.273 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:47:17.273 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:47:17.273 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:47:17.273 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:47:17.273 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:47:17.273 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:47:17.273 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:47:17.273 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.273 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:47:17.273 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:47:17.273 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:47:17.273 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:47:17.274 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:47:17.274 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:47:17.274 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:47:17.274 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:47:17.274 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:47:17.274 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:47:17.274 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:47:17.274 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:47:17.274 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:47:17.274 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:47:17.274 INFO:teuthology.orchestra.run.gibba001.stdout: "data": "cephfs" 2021-02-10T11:47:17.274 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.274 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.275 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.275 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:47:17.275 INFO:teuthology.orchestra.run.gibba001.stdout: "osds": [ 2021-02-10T11:47:17.275 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.275 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:47:17.275 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5", 2021-02-10T11:47:17.275 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:47:17.275 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:47:17.275 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:47:17.275 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:47:17.275 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:47:17.275 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:47:17.275 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:47:17.276 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 173, 2021-02-10T11:47:17.276 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:47:17.276 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:47:17.276 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:47:17.276 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.276 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.276 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.276 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6802", 2021-02-10T11:47:17.276 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:47:17.276 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.276 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.276 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.276 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6805", 2021-02-10T11:47:17.276 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:47:17.277 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.277 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.277 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.277 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:47:17.277 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.277 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.277 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.277 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6807", 2021-02-10T11:47:17.277 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:47:17.277 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.277 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.277 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.277 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6809", 2021-02-10T11:47:17.277 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:47:17.278 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.278 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.278 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.278 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:47:17.278 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.278 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.278 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.278 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6817", 2021-02-10T11:47:17.278 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:47:17.278 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.278 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.278 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.278 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6820", 2021-02-10T11:47:17.278 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:47:17.279 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.279 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.279 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.279 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:47:17.279 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.279 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.279 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.279 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6812", 2021-02-10T11:47:17.279 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:47:17.279 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.279 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.279 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.280 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6814", 2021-02-10T11:47:17.280 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:47:17.280 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.280 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.280 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.280 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6805/25093", 2021-02-10T11:47:17.280 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6809/25093", 2021-02-10T11:47:17.280 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6820/25093", 2021-02-10T11:47:17.280 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6814/25093", 2021-02-10T11:47:17.280 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:47:17.280 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:47:17.280 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:47:17.280 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.281 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.281 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.281 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:47:17.281 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "ddc04ce3-b313-4707-8270-585391e38ecc", 2021-02-10T11:47:17.281 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:47:17.281 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:47:17.281 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:47:17.281 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:47:17.281 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:47:17.281 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:47:17.281 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:47:17.281 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 173, 2021-02-10T11:47:17.281 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:47:17.282 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:47:17.282 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:47:17.282 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.282 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.282 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.282 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6803", 2021-02-10T11:47:17.282 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:47:17.282 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.282 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.282 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.282 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6804", 2021-02-10T11:47:17.282 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:47:17.283 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.283 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.283 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.283 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:47:17.283 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.283 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.283 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.283 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6806", 2021-02-10T11:47:17.283 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:47:17.283 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.283 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.283 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.283 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6808", 2021-02-10T11:47:17.283 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:47:17.284 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.284 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.284 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.284 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:47:17.284 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.284 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.284 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.284 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6818", 2021-02-10T11:47:17.284 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:47:17.284 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.284 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.284 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.284 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6821", 2021-02-10T11:47:17.284 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:47:17.285 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.285 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.285 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.285 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:47:17.285 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.285 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.285 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.285 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6810", 2021-02-10T11:47:17.285 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:47:17.285 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.285 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.285 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.285 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6815", 2021-02-10T11:47:17.285 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:47:17.286 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.286 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.286 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.286 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6804/25092", 2021-02-10T11:47:17.286 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6808/25092", 2021-02-10T11:47:17.286 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6821/25092", 2021-02-10T11:47:17.286 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6815/25092", 2021-02-10T11:47:17.286 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:47:17.286 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:47:17.286 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:47:17.286 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.286 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.286 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.287 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:47:17.287 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3", 2021-02-10T11:47:17.287 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:47:17.287 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:47:17.287 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:47:17.287 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:47:17.287 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:47:17.287 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:47:17.287 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:47:17.287 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 173, 2021-02-10T11:47:17.287 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:47:17.287 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:47:17.287 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:47:17.287 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.288 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.288 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.288 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6811", 2021-02-10T11:47:17.288 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:47:17.288 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.288 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.288 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.288 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6813", 2021-02-10T11:47:17.288 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:47:17.288 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.288 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.288 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.288 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:47:17.288 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.289 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.289 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.289 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6816", 2021-02-10T11:47:17.289 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:47:17.289 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.289 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.289 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.289 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6819", 2021-02-10T11:47:17.289 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:47:17.289 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.289 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.289 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.290 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:47:17.290 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.290 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.290 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.290 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6825", 2021-02-10T11:47:17.290 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:47:17.290 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.290 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.290 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.290 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6827", 2021-02-10T11:47:17.291 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:47:17.291 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.291 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.291 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.291 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:47:17.291 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.291 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.291 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.291 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6822", 2021-02-10T11:47:17.291 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:47:17.291 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.291 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.291 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.292 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6823", 2021-02-10T11:47:17.292 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:47:17.292 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.292 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.292 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.292 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6813/25094", 2021-02-10T11:47:17.292 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6819/25094", 2021-02-10T11:47:17.292 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6827/25094", 2021-02-10T11:47:17.292 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6823/25094", 2021-02-10T11:47:17.292 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:47:17.292 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:47:17.292 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:47:17.292 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.293 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.293 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.293 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:47:17.293 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "0d9ef9b1-98aa-4d37-aa16-6305c7d46d52", 2021-02-10T11:47:17.293 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:47:17.293 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:47:17.293 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:47:17.293 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:47:17.293 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:47:17.293 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:47:17.293 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:47:17.293 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 173, 2021-02-10T11:47:17.294 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:47:17.294 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:47:17.294 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:47:17.294 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.294 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.294 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.294 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6824", 2021-02-10T11:47:17.294 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:47:17.294 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.294 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.295 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.295 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6826", 2021-02-10T11:47:17.295 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:47:17.295 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.295 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.295 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.295 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:47:17.295 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.295 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.295 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.296 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6828", 2021-02-10T11:47:17.296 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:47:17.296 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.296 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.296 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.296 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6829", 2021-02-10T11:47:17.296 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:47:17.296 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.296 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.296 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.296 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:47:17.296 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.296 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.297 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.297 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6832", 2021-02-10T11:47:17.297 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:47:17.297 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.297 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.297 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.297 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6833", 2021-02-10T11:47:17.297 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:47:17.297 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.297 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.297 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.297 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:47:17.297 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.297 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.298 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.298 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6830", 2021-02-10T11:47:17.298 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:47:17.298 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.298 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.298 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.298 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6831", 2021-02-10T11:47:17.298 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:47:17.298 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.298 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.298 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.298 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6826/25095", 2021-02-10T11:47:17.298 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6829/25095", 2021-02-10T11:47:17.299 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6833/25095", 2021-02-10T11:47:17.299 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6831/25095", 2021-02-10T11:47:17.299 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:47:17.299 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:47:17.299 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:47:17.299 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.299 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.299 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.299 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:47:17.299 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "36742c21-bfeb-4e11-bce9-8e745378b605", 2021-02-10T11:47:17.299 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:47:17.299 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:47:17.299 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:47:17.300 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:47:17.300 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:47:17.300 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:47:17.300 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:47:17.300 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 173, 2021-02-10T11:47:17.300 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:47:17.300 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:47:17.300 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:47:17.300 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.300 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.300 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.300 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6816", 2021-02-10T11:47:17.300 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:47:17.300 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.301 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.301 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.301 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6818", 2021-02-10T11:47:17.301 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:47:17.301 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.301 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.301 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.301 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:47:17.301 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.301 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.301 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.301 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6820", 2021-02-10T11:47:17.301 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:47:17.301 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.302 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.302 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.302 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6822", 2021-02-10T11:47:17.302 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:47:17.302 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.302 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.302 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.302 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:47:17.302 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.302 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.302 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.302 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6828", 2021-02-10T11:47:17.302 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:47:17.302 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.303 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.303 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.303 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6830", 2021-02-10T11:47:17.303 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:47:17.303 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.303 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.303 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.303 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:47:17.303 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.303 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.303 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.303 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6824", 2021-02-10T11:47:17.303 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:47:17.304 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.304 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.304 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.304 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6826", 2021-02-10T11:47:17.304 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:47:17.304 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.304 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.304 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.304 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6818/36380", 2021-02-10T11:47:17.304 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6822/36380", 2021-02-10T11:47:17.304 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6830/36380", 2021-02-10T11:47:17.304 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6826/36380", 2021-02-10T11:47:17.304 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:47:17.304 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:47:17.305 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:47:17.305 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.305 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.305 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.305 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:47:17.305 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "88e694e8-fe98-4d1b-88c8-8c9907ece60b", 2021-02-10T11:47:17.305 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:47:17.305 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:47:17.305 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:47:17.305 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:47:17.305 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:47:17.305 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:47:17.305 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:47:17.305 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 173, 2021-02-10T11:47:17.306 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:47:17.306 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:47:17.306 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:47:17.306 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.306 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.306 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.306 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6808", 2021-02-10T11:47:17.306 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:47:17.306 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.306 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.306 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.306 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6809", 2021-02-10T11:47:17.306 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:47:17.307 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.307 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.307 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.307 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:47:17.307 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.307 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.307 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.307 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6810", 2021-02-10T11:47:17.307 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:47:17.307 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.307 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.307 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.307 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6811", 2021-02-10T11:47:17.307 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:47:17.308 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.308 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.308 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.308 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:47:17.308 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.308 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.308 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.308 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6814", 2021-02-10T11:47:17.308 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:47:17.308 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.308 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.308 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.308 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6815", 2021-02-10T11:47:17.308 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:47:17.309 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.309 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.309 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.309 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:47:17.309 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.309 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.309 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.309 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6812", 2021-02-10T11:47:17.309 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:47:17.309 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.309 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.309 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.309 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6813", 2021-02-10T11:47:17.309 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:47:17.310 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.310 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.310 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.310 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6809/36383", 2021-02-10T11:47:17.310 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6811/36383", 2021-02-10T11:47:17.310 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6815/36383", 2021-02-10T11:47:17.310 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6813/36383", 2021-02-10T11:47:17.310 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:47:17.310 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:47:17.310 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:47:17.310 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.310 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.310 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.311 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:47:17.311 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "87a13c6c-cf85-4aaf-916b-e73913484a49", 2021-02-10T11:47:17.311 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:47:17.311 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:47:17.311 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:47:17.311 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:47:17.311 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:47:17.311 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:47:17.311 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:47:17.311 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 173, 2021-02-10T11:47:17.311 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:47:17.311 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:47:17.311 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:47:17.311 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.312 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.312 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.312 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6800", 2021-02-10T11:47:17.312 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:47:17.312 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.312 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.312 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.312 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6801", 2021-02-10T11:47:17.312 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:47:17.312 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.312 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.312 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.312 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:47:17.312 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.313 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.313 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.313 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6802", 2021-02-10T11:47:17.313 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:47:17.313 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.313 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.313 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.313 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6803", 2021-02-10T11:47:17.313 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:47:17.313 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.313 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.313 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.313 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:47:17.313 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.314 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.314 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.314 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6806", 2021-02-10T11:47:17.314 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:47:17.314 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.314 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.314 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.314 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6807", 2021-02-10T11:47:17.314 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:47:17.314 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.314 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.314 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.314 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:47:17.315 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.315 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.315 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.315 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6804", 2021-02-10T11:47:17.315 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:47:17.315 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.315 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.315 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.315 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6805", 2021-02-10T11:47:17.315 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:47:17.316 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.316 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.316 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.316 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6801/36381", 2021-02-10T11:47:17.316 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6803/36381", 2021-02-10T11:47:17.316 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6807/36381", 2021-02-10T11:47:17.316 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6805/36381", 2021-02-10T11:47:17.316 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:47:17.316 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:47:17.316 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:47:17.316 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.316 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.316 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.317 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:47:17.317 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "7b90823f-854c-4592-a95b-a7fa98f93254", 2021-02-10T11:47:17.317 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:47:17.317 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:47:17.317 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:47:17.317 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:47:17.317 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:47:17.317 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:47:17.317 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:47:17.317 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 172, 2021-02-10T11:47:17.317 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:47:17.317 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:47:17.317 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:47:17.318 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.318 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.318 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.318 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6817", 2021-02-10T11:47:17.318 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:47:17.318 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.318 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.318 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.318 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6819", 2021-02-10T11:47:17.318 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:47:17.318 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.318 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.318 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.319 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:47:17.319 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.319 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.319 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.319 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6821", 2021-02-10T11:47:17.319 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:47:17.319 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.319 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.319 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.319 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6823", 2021-02-10T11:47:17.319 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:47:17.319 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.319 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.320 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.320 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:47:17.320 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.320 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.320 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.320 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6829", 2021-02-10T11:47:17.320 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:47:17.320 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.320 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.320 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.320 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6831", 2021-02-10T11:47:17.320 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:47:17.320 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.321 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.321 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.321 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:47:17.321 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:47:17.321 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.321 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:47:17.321 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6825", 2021-02-10T11:47:17.321 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:47:17.321 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.321 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.321 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:17.321 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6827", 2021-02-10T11:47:17.321 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:47:17.322 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.322 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.322 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.322 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6819/36382", 2021-02-10T11:47:17.322 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6823/36382", 2021-02-10T11:47:17.322 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6831/36382", 2021-02-10T11:47:17.322 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6827/36382", 2021-02-10T11:47:17.322 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:47:17.322 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:47:17.322 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:47:17.322 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:17.322 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.323 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:47:17.323 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_xinfo": [ 2021-02-10T11:47:17.323 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.323 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:47:17.323 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:47:17.323 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:47:17.323 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:47:17.323 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:47:17.323 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:47:17.323 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.323 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.323 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:47:17.323 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:47:17.324 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:47:17.324 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:47:17.324 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:47:17.324 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:47:17.324 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.324 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.324 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:47:17.324 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:47:17.324 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:47:17.324 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:47:17.324 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:47:17.324 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:47:17.325 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.325 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.325 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:47:17.325 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:47:17.325 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:47:17.325 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:47:17.325 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:47:17.325 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:47:17.325 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.325 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.325 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:47:17.325 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:47:17.326 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:47:17.326 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:47:17.326 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:47:17.326 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:47:17.326 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.326 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.326 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:47:17.326 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:47:17.326 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:47:17.326 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:47:17.326 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:47:17.326 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:47:17.326 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.327 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.327 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:47:17.327 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:47:17.327 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:47:17.327 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:47:17.327 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:47:17.327 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:47:17.327 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.327 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:17.327 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:47:17.327 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:47:17.327 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:47:17.327 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:47:17.327 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:47:17.328 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:47:17.328 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.328 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:47:17.328 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap": [], 2021-02-10T11:47:17.328 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap_items": [], 2021-02-10T11:47:17.328 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_temp": [], 2021-02-10T11:47:17.328 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_temp": [], 2021-02-10T11:47:17.328 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklist": {}, 2021-02-10T11:47:17.328 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profiles": { 2021-02-10T11:47:17.328 INFO:teuthology.orchestra.run.gibba001.stdout: "default": { 2021-02-10T11:47:17.328 INFO:teuthology.orchestra.run.gibba001.stdout: "crush-failure-domain": "osd", 2021-02-10T11:47:17.328 INFO:teuthology.orchestra.run.gibba001.stdout: "k": "2", 2021-02-10T11:47:17.328 INFO:teuthology.orchestra.run.gibba001.stdout: "m": "1", 2021-02-10T11:47:17.329 INFO:teuthology.orchestra.run.gibba001.stdout: "plugin": "jerasure", 2021-02-10T11:47:17.329 INFO:teuthology.orchestra.run.gibba001.stdout: "ruleset-failure-domain": "osd", 2021-02-10T11:47:17.329 INFO:teuthology.orchestra.run.gibba001.stdout: "technique": "reed_sol_van" 2021-02-10T11:47:17.329 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:17.329 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:17.329 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps_queue": [], 2021-02-10T11:47:17.329 INFO:teuthology.orchestra.run.gibba001.stdout: "new_removed_snaps": [], 2021-02-10T11:47:17.329 INFO:teuthology.orchestra.run.gibba001.stdout: "new_purged_snaps": [], 2021-02-10T11:47:17.329 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_node_flags": {}, 2021-02-10T11:47:17.329 INFO:teuthology.orchestra.run.gibba001.stdout: "device_class_flags": {} 2021-02-10T11:47:17.329 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:47:17.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.260 7f300cfb2700 1 -- 172.21.2.101:0/1772469146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2fec0186c0 msgr2=0x7f2fec01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:17.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.260 7f300cfb2700 1 --2- 172.21.2.101:0/1772469146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2fec0186c0 0x7f2fec01abc0 crc :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:17.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.260 7f300cfb2700 1 -- 172.21.2.101:0/1772469146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3008195650 msgr2=0x7f30080b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:17.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.260 7f300cfb2700 1 --2- 172.21.2.101:0/1772469146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3008195650 0x7f30080b5560 secure :-1 s=READY pgs=3060 cs=0 l=1 rev1=1 rx=0x7f2ff000a540 tx=0x7f2ff000a8e0).stop 2021-02-10T11:47:17.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.260 7f300cfb2700 1 -- 172.21.2.101:0/1772469146 shutdown_connections 2021-02-10T11:47:17.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.260 7f300cfb2700 1 --2- 172.21.2.101:0/1772469146 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2fec0186c0 0x7f2fec01abc0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:17.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.260 7f300cfb2700 1 --2- 172.21.2.101:0/1772469146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f30080acfa0 0x7f30081a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:17.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.260 7f300cfb2700 1 --2- 172.21.2.101:0/1772469146 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f30080b5b80 0x7f30081a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:17.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.260 7f300cfb2700 1 --2- 172.21.2.101:0/1772469146 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3008195650 0x7f30080b5560 unknown :-1 s=CLOSED pgs=3060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:17.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.260 7f300cfb2700 1 -- 172.21.2.101:0/1772469146 >> 172.21.2.101:0/1772469146 conn(0x7f30080b3040 msgr2=0x7f30080c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:17.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.260 7f300cfb2700 1 -- 172.21.2.101:0/1772469146 shutdown_connections 2021-02-10T11:47:17.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.260 7f300cfb2700 1 -- 172.21.2.101:0/1772469146 wait complete. 2021-02-10T11:47:17.332 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:47:17.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.746 7f9ec690d700 1 Processor -- start 2021-02-10T11:47:17.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.746 7f9ec690d700 1 -- start start 2021-02-10T11:47:17.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.746 7f9ec690d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9ec00d1870 0x7f9ec01b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:17.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.746 7f9ec690d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ec01b0a10 0x7f9ec00cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:17.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.746 7f9ec690d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ec00f0b10 con 0x7f9ec00d1c70 2021-02-10T11:47:17.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.746 7f9ec690d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ec00e5ed0 con 0x7f9ec00d1870 2021-02-10T11:47:17.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.746 7f9ec690d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec00f0650 con 0x7f9ec01b0a10 2021-02-10T11:47:17.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.746 7f9ebffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9ec00d1870 0x7f9ec01b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:17.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.746 7f9ebffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9ec00d1870 0x7f9ec01b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33874/0 (socket says 172.21.2.101:33874) 2021-02-10T11:47:17.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.746 7f9ebffff700 1 -- 172.21.2.101:0/4012364159 learned_addr learned my addr 172.21.2.101:0/4012364159 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:17.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.747 7f9ebf7fe700 1 --2- 172.21.2.101:0/4012364159 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ec01b0a10 0x7f9ec00cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:17.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.747 7f9ebffff700 1 -- 172.21.2.101:0/4012364159 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ec01b0a10 msgr2=0x7f9ec00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:17.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.747 7f9ebffff700 1 --2- 172.21.2.101:0/4012364159 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ec01b0a10 0x7f9ec00cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:17.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.747 7f9ebffff700 1 -- 172.21.2.101:0/4012364159 >> v1:172.21.2.108:6789/0 conn(0x7f9ec00d1c70 legacy=0x7f9ec00cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:17.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.747 7f9ebffff700 1 -- 172.21.2.101:0/4012364159 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec00f0870 con 0x7f9ec00d1870 2021-02-10T11:47:17.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.747 7f9ebffff700 1 --2- 172.21.2.101:0/4012364159 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ec00d1870 0x7f9ec01b03f0 secure :-1 s=READY pgs=3061 cs=0 l=1 rev1=1 rx=0x7f9eb0006700 tx=0x7f9eb0009b00).ready entity=mon.0 client_cookie=584b6c681e47ef3a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:17.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.747 7f9ebeffd700 1 -- 172.21.2.101:0/4012364159 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9eb000c0c0 con 0x7f9ec00d1870 2021-02-10T11:47:17.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.747 7f9ebeffd700 1 -- 172.21.2.101:0/4012364159 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb000c340 con 0x7f9ec00d1870 2021-02-10T11:47:17.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.747 7f9ebeffd700 1 -- 172.21.2.101:0/4012364159 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9eb000c820 con 0x7f9ec00d1870 2021-02-10T11:47:17.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.747 7f9ec690d700 1 -- 172.21.2.101:0/4012364159 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ec00d1870 msgr2=0x7f9ec01b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:17.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.747 7f9ec690d700 1 --2- 172.21.2.101:0/4012364159 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ec00d1870 0x7f9ec01b03f0 secure :-1 s=READY pgs=3061 cs=0 l=1 rev1=1 rx=0x7f9eb0006700 tx=0x7f9eb0009b00).stop 2021-02-10T11:47:17.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.748 7f9ec690d700 1 -- 172.21.2.101:0/4012364159 shutdown_connections 2021-02-10T11:47:17.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.748 7f9ec690d700 1 --2- 172.21.2.101:0/4012364159 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ec01b0a10 0x7f9ec00cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:17.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.748 7f9ec690d700 1 --2- 172.21.2.101:0/4012364159 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ec00d1870 0x7f9ec01b03f0 unknown :-1 s=CLOSED pgs=3061 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:17.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.748 7f9ec690d700 1 -- 172.21.2.101:0/4012364159 >> 172.21.2.101:0/4012364159 conn(0x7f9ec00ce450 msgr2=0x7f9ec00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:17.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.748 7f9ec690d700 1 -- 172.21.2.101:0/4012364159 shutdown_connections 2021-02-10T11:47:17.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.748 7f9ec690d700 1 -- 172.21.2.101:0/4012364159 wait complete. 2021-02-10T11:47:17.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.763 7f9ec690d700 1 Processor -- start 2021-02-10T11:47:17.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.763 7f9ec690d700 1 -- start start 2021-02-10T11:47:17.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.763 7f9ec690d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9ec00f0280 0x7f9ec00a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:17.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.763 7f9ec690d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ec00f0680 0x7f9ec00a2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:17.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.763 7f9ec690d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ec00ec180 0x7f9ec00ec580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:17.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.763 7f9ec690d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec00efd00 con 0x7f9ec00ec180 2021-02-10T11:47:17.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.763 7f9ec690d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec00ecba0 con 0x7f9ec00f0280 2021-02-10T11:47:17.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.763 7f9ec690d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec00ecd80 con 0x7f9ec00f0680 2021-02-10T11:47:17.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.763 7f9ec4f2a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ec00ec180 0x7f9ec00ec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:17.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.763 7f9ec4f2a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ec00ec180 0x7f9ec00ec580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33880/0 (socket says 172.21.2.101:33880) 2021-02-10T11:47:17.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.763 7f9ec4f2a700 1 -- 172.21.2.101:0/123984249 learned_addr learned my addr 172.21.2.101:0/123984249 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:17.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.763 7f9ebffff700 1 --2- 172.21.2.101:0/123984249 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9ec00f0280 0x7f9ec00a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:17.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.764 7f9ebf7fe700 1 --2- 172.21.2.101:0/123984249 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ec00f0680 0x7f9ec00a2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:17.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.764 7f9ec4f2a700 1 -- 172.21.2.101:0/123984249 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ec00f0680 msgr2=0x7f9ec00a2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:17.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.764 7f9ec4f2a700 1 --2- 172.21.2.101:0/123984249 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ec00f0680 0x7f9ec00a2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:17.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.764 7f9ec4f2a700 1 -- 172.21.2.101:0/123984249 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9ec00f0280 msgr2=0x7f9ec00a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:17.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.764 7f9ec4f2a700 1 --2- 172.21.2.101:0/123984249 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9ec00f0280 0x7f9ec00a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:17.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.764 7f9ec4f2a700 1 -- 172.21.2.101:0/123984249 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec00ecfa0 con 0x7f9ec00ec180 2021-02-10T11:47:17.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.764 7f9ec4f2a700 1 --2- 172.21.2.101:0/123984249 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ec00ec180 0x7f9ec00ec580 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 rx=0x7f9eb8009940 tx=0x7f9eb800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:17.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.764 7f9ebcff9700 1 -- 172.21.2.101:0/123984249 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9eb8000ce0 con 0x7f9ec00ec180 2021-02-10T11:47:17.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.764 7f9ebcff9700 1 -- 172.21.2.101:0/123984249 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb8000f90 con 0x7f9ec00ec180 2021-02-10T11:47:17.764 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.764 7f9ebcff9700 1 -- 172.21.2.101:0/123984249 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9eb800c570 con 0x7f9ec00ec180 2021-02-10T11:47:17.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.764 7f9ec690d700 1 -- 172.21.2.101:0/123984249 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec00ecfa0 con 0x7f9ec00ec180 2021-02-10T11:47:17.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.764 7f9ec690d700 1 -- 172.21.2.101:0/123984249 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec00ecfa0 con 0x7f9ec00ec180 2021-02-10T11:47:17.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.765 7f9ebcff9700 1 -- 172.21.2.101:0/123984249 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9eb8001220 con 0x7f9ec00ec180 2021-02-10T11:47:17.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.765 7f9ebcff9700 1 --2- 172.21.2.101:0/123984249 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ea00186c0 0x7f9ea001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:17.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.765 7f9ebcff9700 1 -- 172.21.2.101:0/123984249 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9eb80329c0 con 0x7f9ec00ec180 2021-02-10T11:47:17.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.765 7f9ebffff700 1 --2- 172.21.2.101:0/123984249 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ea00186c0 0x7f9ea001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:17.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.765 7f9ebffff700 1 --2- 172.21.2.101:0/123984249 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ea00186c0 0x7f9ea001abc0 crc :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:17.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.796 7f9ec690d700 1 -- 172.21.2.101:0/123984249 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec00ebc60 con 0x7f9ec00ec180 2021-02-10T11:47:17.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.799 7f9ebcff9700 1 -- 172.21.2.101:0/123984249 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9eb800be30 con 0x7f9ec00ec180 2021-02-10T11:47:17.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.925 7f9ec690d700 1 -- 172.21.2.101:0/123984249 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ec00f0b10 con 0x7f9ec00ec180 2021-02-10T11:47:17.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.926 7f9ebcff9700 1 -- 172.21.2.101:0/123984249 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 149 v149) v1 ==== 95+0+2111 (secure 0 0 0) 0x7f9eb801f880 con 0x7f9ec00ec180 2021-02-10T11:47:17.923 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 149 2021-02-10T11:47:17.923 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:47:17.923 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":149,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":12780,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2200088074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2200088074},{"type":"v1","addr":"172.21.2.101:6835","nonce":2200088074}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":144}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:46:35.149920","modified":"2021-02-10 11:46:37.158088","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12777},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12777":{"gid":12777,"name":"b","rank":0,"incarnation":145,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/1343655613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":1343655613},{"type":"v1","addr":"172.21.2.108:6833","nonce":1343655613}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[37],"metadata_pool":36,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":18}]} 2021-02-10T11:47:17.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.927 7f9ec690d700 1 -- 172.21.2.101:0/123984249 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ea00186c0 msgr2=0x7f9ea001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:17.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.927 7f9ec690d700 1 --2- 172.21.2.101:0/123984249 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ea00186c0 0x7f9ea001abc0 crc :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:17.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.927 7f9ec690d700 1 -- 172.21.2.101:0/123984249 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ec00ec180 msgr2=0x7f9ec00ec580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:17.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.927 7f9ec690d700 1 --2- 172.21.2.101:0/123984249 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ec00ec180 0x7f9ec00ec580 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 rx=0x7f9eb8009940 tx=0x7f9eb800bcf0).stop 2021-02-10T11:47:17.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.927 7f9ec690d700 1 -- 172.21.2.101:0/123984249 shutdown_connections 2021-02-10T11:47:17.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.927 7f9ec690d700 1 --2- 172.21.2.101:0/123984249 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ea00186c0 0x7f9ea001abc0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:17.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.927 7f9ec690d700 1 --2- 172.21.2.101:0/123984249 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9ec00f0680 0x7f9ec00a2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:17.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.928 7f9ec690d700 1 --2- 172.21.2.101:0/123984249 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9ec00ec180 0x7f9ec00ec580 unknown :-1 s=CLOSED pgs=3062 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:17.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.928 7f9ec690d700 1 --2- 172.21.2.101:0/123984249 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9ec00f0280 0x7f9ec00a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:17.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.928 7f9ec690d700 1 -- 172.21.2.101:0/123984249 >> 172.21.2.101:0/123984249 conn(0x7f9ec00c2130 msgr2=0x7f9ec009e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:17.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.928 7f9ec690d700 1 -- 172.21.2.101:0/123984249 shutdown_connections 2021-02-10T11:47:17.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:17.928 7f9ec690d700 1 -- 172.21.2.101:0/123984249 wait complete. 2021-02-10T11:47:17.933 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs fail cephfs 2021-02-10T11:47:18.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.350 7f97ce709700 1 Processor -- start 2021-02-10T11:47:18.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.350 7f97ce709700 1 -- start start 2021-02-10T11:47:18.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.350 7f97ce709700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f97c80d1870 0x7f97c81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:18.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.350 7f97ce709700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97c81b0a10 0x7f97c80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:18.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.350 7f97ce709700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c80f0b10 con 0x7f97c80d1c70 2021-02-10T11:47:18.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.350 7f97ce709700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c80e5ed0 con 0x7f97c80d1870 2021-02-10T11:47:18.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.350 7f97ce709700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c80f0650 con 0x7f97c81b0a10 2021-02-10T11:47:18.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.351 7f97ccd26700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f97c80d1c70 0x7f97c80cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:56024/0 (socket says 172.21.2.101:56024) 2021-02-10T11:47:18.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.351 7f97ccd26700 1 -- 172.21.2.101:0/1124388965 learned_addr learned my addr 172.21.2.101:0/1124388965 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:18.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.351 7f97c7fff700 1 --2- 172.21.2.101:0/1124388965 >> v2:172.21.2.108:3300/0 conn(0x7f97c80d1870 0x7f97c81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:18.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.351 7f97c77fe700 1 --2- 172.21.2.101:0/1124388965 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97c81b0a10 0x7f97c80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:18.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.351 7f97c7fff700 1 -- 172.21.2.101:0/1124388965 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97c81b0a10 msgr2=0x7f97c80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:18.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.351 7f97c7fff700 1 --2- 172.21.2.101:0/1124388965 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97c81b0a10 0x7f97c80cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:18.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.351 7f97c7fff700 1 -- 172.21.2.101:0/1124388965 >> v1:172.21.2.108:6789/0 conn(0x7f97c80d1c70 legacy=0x7f97c80cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:18.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.351 7f97c7fff700 1 -- 172.21.2.101:0/1124388965 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c80f0870 con 0x7f97c80d1870 2021-02-10T11:47:18.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.351 7f97c7fff700 1 --2- 172.21.2.101:0/1124388965 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97c80d1870 0x7f97c81b03f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7f97b8006650 tx=0x7f97b8009a80).ready entity=mon.1 client_cookie=ae02a539a752eecd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:18.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.352 7f97c6ffd700 1 -- 172.21.2.101:0/1124388965 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f97b800c0c0 con 0x7f97c80d1870 2021-02-10T11:47:18.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.352 7f97c6ffd700 1 -- 172.21.2.101:0/1124388965 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b800c340 con 0x7f97c80d1870 2021-02-10T11:47:18.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.352 7f97ce709700 1 -- 172.21.2.101:0/1124388965 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97c80d1870 msgr2=0x7f97c81b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:18.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.352 7f97ce709700 1 --2- 172.21.2.101:0/1124388965 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97c80d1870 0x7f97c81b03f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7f97b8006650 tx=0x7f97b8009a80).stop 2021-02-10T11:47:18.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.352 7f97c6ffd700 1 -- 172.21.2.101:0/1124388965 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f97b80009b0 con 0x7f97c80d1870 2021-02-10T11:47:18.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.352 7f97ce709700 1 -- 172.21.2.101:0/1124388965 shutdown_connections 2021-02-10T11:47:18.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.352 7f97ce709700 1 --2- 172.21.2.101:0/1124388965 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97c81b0a10 0x7f97c80cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:18.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.352 7f97ce709700 1 --2- 172.21.2.101:0/1124388965 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97c80d1870 0x7f97c81b03f0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:18.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.352 7f97ce709700 1 -- 172.21.2.101:0/1124388965 >> 172.21.2.101:0/1124388965 conn(0x7f97c80ce450 msgr2=0x7f97c80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:18.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.352 7f97ce709700 1 -- 172.21.2.101:0/1124388965 shutdown_connections 2021-02-10T11:47:18.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.352 7f97ce709700 1 -- 172.21.2.101:0/1124388965 wait complete. 2021-02-10T11:47:18.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.367 7f97ce709700 1 Processor -- start 2021-02-10T11:47:18.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.367 7f97ce709700 1 -- start start 2021-02-10T11:47:18.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.367 7f97ce709700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97c80a60d0 0x7f97c80a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:18.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.367 7f97ce709700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97c80a6560 0x7f97c80a6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:18.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.367 7f97ce709700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f97c80ebec0 0x7f97c80ec2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:18.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.367 7f97ce709700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c808dba0 con 0x7f97c80ebec0 2021-02-10T11:47:18.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.367 7f97ce709700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c808e120 con 0x7f97c80a60d0 2021-02-10T11:47:18.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.367 7f97ce709700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c80a5bc0 con 0x7f97c80a6560 2021-02-10T11:47:18.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.367 7f97ccd26700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f97c80ebec0 0x7f97c80ec2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:18.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.367 7f97ccd26700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f97c80ebec0 0x7f97c80ec2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33892/0 (socket says 172.21.2.101:33892) 2021-02-10T11:47:18.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.367 7f97ccd26700 1 -- 172.21.2.101:0/3993689078 learned_addr learned my addr 172.21.2.101:0/3993689078 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:18.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.367 7f97c7fff700 1 --2- 172.21.2.101:0/3993689078 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97c80a60d0 0x7f97c80a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:18.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.367 7f97c77fe700 1 --2- 172.21.2.101:0/3993689078 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97c80a6560 0x7f97c80a6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:18.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.368 7f97ccd26700 1 -- 172.21.2.101:0/3993689078 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97c80a6560 msgr2=0x7f97c80a6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:18.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.368 7f97ccd26700 1 --2- 172.21.2.101:0/3993689078 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97c80a6560 0x7f97c80a6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:18.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.368 7f97ccd26700 1 -- 172.21.2.101:0/3993689078 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97c80a60d0 msgr2=0x7f97c80a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:18.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.368 7f97ccd26700 1 --2- 172.21.2.101:0/3993689078 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97c80a60d0 0x7f97c80a54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:18.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.368 7f97ccd26700 1 -- 172.21.2.101:0/3993689078 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c80ec8e0 con 0x7f97c80ebec0 2021-02-10T11:47:18.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.368 7f97ccd26700 1 --2- 172.21.2.101:0/3993689078 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f97c80ebec0 0x7f97c80ec2c0 secure :-1 s=READY pgs=3063 cs=0 l=1 rev1=1 rx=0x7f97c0007fc0 tx=0x7f97c000ac80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:18.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.368 7f97c4ff9700 1 -- 172.21.2.101:0/3993689078 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f97c0007020 con 0x7f97c80ebec0 2021-02-10T11:47:18.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.368 7f97c4ff9700 1 -- 172.21.2.101:0/3993689078 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c00072e0 con 0x7f97c80ebec0 2021-02-10T11:47:18.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.368 7f97c4ff9700 1 -- 172.21.2.101:0/3993689078 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f97c00046a0 con 0x7f97c80ebec0 2021-02-10T11:47:18.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.368 7f97ce709700 1 -- 172.21.2.101:0/3993689078 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c80ec8e0 con 0x7f97c80ebec0 2021-02-10T11:47:18.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.368 7f97ce709700 1 -- 172.21.2.101:0/3993689078 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c80ec8e0 con 0x7f97c80ebec0 2021-02-10T11:47:18.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.369 7f97c4ff9700 1 -- 172.21.2.101:0/3993689078 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f97c00046a0 con 0x7f97c80ebec0 2021-02-10T11:47:18.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.369 7f97c4ff9700 1 --2- 172.21.2.101:0/3993689078 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97a80186c0 0x7f97a801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:18.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.369 7f97c4ff9700 1 -- 172.21.2.101:0/3993689078 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f97c0032770 con 0x7f97c80ebec0 2021-02-10T11:47:18.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.369 7f97c7fff700 1 --2- 172.21.2.101:0/3993689078 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97a80186c0 0x7f97a801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:18.369 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.369 7f97c7fff700 1 --2- 172.21.2.101:0/3993689078 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97a80186c0 0x7f97a801abc0 crc :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:18.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.400 7f97ce709700 1 -- 172.21.2.101:0/3993689078 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c80ebc90 con 0x7f97c80ebec0 2021-02-10T11:47:18.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.403 7f97c4ff9700 1 -- 172.21.2.101:0/3993689078 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f97c00226f0 con 0x7f97c80ebec0 2021-02-10T11:47:18.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.530 7f97ce709700 1 -- 172.21.2.101:0/3993689078 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f97c80ebc90 con 0x7f97c80ebec0 2021-02-10T11:47:18.641 INFO:tasks.ceph.mon.a.gibba001.stderr:2021-02-10 11:47:18.644 7f974d04d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-02-10T11:47:18.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.650 7f97c4ff9700 1 -- 172.21.2.101:0/3993689078 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v150) v1 ==== 161+0+0 (secure 0 0 0) 0x7f97c00226f0 con 0x7f97c80ebec0 2021-02-10T11:47:18.691 INFO:teuthology.orchestra.run.gibba001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-02-10T11:47:18.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.696 7f97ce709700 1 -- 172.21.2.101:0/3993689078 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97a80186c0 msgr2=0x7f97a801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:18.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.696 7f97ce709700 1 --2- 172.21.2.101:0/3993689078 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97a80186c0 0x7f97a801abc0 crc :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:18.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.696 7f97ce709700 1 -- 172.21.2.101:0/3993689078 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f97c80ebec0 msgr2=0x7f97c80ec2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:18.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.696 7f97ce709700 1 --2- 172.21.2.101:0/3993689078 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f97c80ebec0 0x7f97c80ec2c0 secure :-1 s=READY pgs=3063 cs=0 l=1 rev1=1 rx=0x7f97c0007fc0 tx=0x7f97c000ac80).stop 2021-02-10T11:47:18.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.696 7f97ce709700 1 -- 172.21.2.101:0/3993689078 shutdown_connections 2021-02-10T11:47:18.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.696 7f97ce709700 1 --2- 172.21.2.101:0/3993689078 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97a80186c0 0x7f97a801abc0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:18.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.696 7f97ce709700 1 --2- 172.21.2.101:0/3993689078 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f97c80a6560 0x7f97c80a6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:18.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.696 7f97ce709700 1 --2- 172.21.2.101:0/3993689078 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f97c80ebec0 0x7f97c80ec2c0 unknown :-1 s=CLOSED pgs=3063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:18.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.696 7f97ce709700 1 --2- 172.21.2.101:0/3993689078 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f97c80a60d0 0x7f97c80a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:18.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.696 7f97ce709700 1 -- 172.21.2.101:0/3993689078 >> 172.21.2.101:0/3993689078 conn(0x7f97c80f0190 msgr2=0x7f97c8094d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:18.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.696 7f97ce709700 1 -- 172.21.2.101:0/3993689078 shutdown_connections 2021-02-10T11:47:18.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:18.697 7f97ce709700 1 -- 172.21.2.101:0/3993689078 wait complete. 2021-02-10T11:47:18.697 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:47:18.703 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:47:19.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.118 7fc6b2bee700 1 Processor -- start 2021-02-10T11:47:19.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.118 7fc6b2bee700 1 -- start start 2021-02-10T11:47:19.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.119 7fc6b2bee700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6ac0d1870 0x7fc6ac1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:19.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.119 7fc6b2bee700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc6ac1b0a10 0x7fc6ac0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:19.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.119 7fc6b2bee700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6ac0f0b10 con 0x7fc6ac0d1c70 2021-02-10T11:47:19.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.119 7fc6b2bee700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6ac0fe230 con 0x7fc6ac1b0a10 2021-02-10T11:47:19.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.119 7fc6b2bee700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac0e5ed0 con 0x7fc6ac0d1870 2021-02-10T11:47:19.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.120 7fc6abfff700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc6ac1b0a10 0x7fc6ac0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:19.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.120 7fc6b0a0a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6ac0d1870 0x7fc6ac1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:19.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.120 7fc6b120b700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fc6ac0d1c70 0x7fc6ac0cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:56040/0 (socket says 172.21.2.101:56040) 2021-02-10T11:47:19.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.120 7fc6b120b700 1 -- 172.21.2.101:0/4147528955 learned_addr learned my addr 172.21.2.101:0/4147528955 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:19.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.120 7fc6abfff700 1 --2- 172.21.2.101:0/4147528955 >> v2:172.21.2.108:3300/0 conn(0x7fc6ac1b0a10 0x7fc6ac0cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:44388/0 (socket says 172.21.2.101:44388) 2021-02-10T11:47:19.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.120 7fc6b0a0a700 1 -- 172.21.2.101:0/4147528955 >> v2:172.21.2.108:3300/0 conn(0x7fc6ac1b0a10 msgr2=0x7fc6ac0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:19.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.120 7fc6b0a0a700 1 --2- 172.21.2.101:0/4147528955 >> v2:172.21.2.108:3300/0 conn(0x7fc6ac1b0a10 0x7fc6ac0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:19.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.120 7fc6b0a0a700 1 -- 172.21.2.101:0/4147528955 >> v1:172.21.2.108:6789/0 conn(0x7fc6ac0d1c70 legacy=0x7fc6ac0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:19.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.120 7fc6b0a0a700 1 -- 172.21.2.101:0/4147528955 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac0f0650 con 0x7fc6ac0d1870 2021-02-10T11:47:19.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.120 7fc6b0a0a700 1 --2- 172.21.2.101:0/4147528955 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6ac0d1870 0x7fc6ac1b03f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fc694006650 tx=0x7fc694009a80).ready entity=mon.2 client_cookie=4871e150b738a3ad server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:19.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.121 7fc6ab7fe700 1 -- 172.21.2.101:0/4147528955 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc6940009b0 con 0x7fc6ac0d1870 2021-02-10T11:47:19.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.121 7fc6ab7fe700 1 -- 172.21.2.101:0/4147528955 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69400c0c0 con 0x7fc6ac0d1870 2021-02-10T11:47:19.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.121 7fc6ab7fe700 1 -- 172.21.2.101:0/4147528955 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc69400c5e0 con 0x7fc6ac0d1870 2021-02-10T11:47:19.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.121 7fc6b2bee700 1 -- 172.21.2.101:0/4147528955 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6ac0d1870 msgr2=0x7fc6ac1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:19.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.121 7fc6b2bee700 1 --2- 172.21.2.101:0/4147528955 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6ac0d1870 0x7fc6ac1b03f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fc694006650 tx=0x7fc694009a80).stop 2021-02-10T11:47:19.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.121 7fc6b2bee700 1 -- 172.21.2.101:0/4147528955 shutdown_connections 2021-02-10T11:47:19.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.121 7fc6b2bee700 1 --2- 172.21.2.101:0/4147528955 >> v2:172.21.2.108:3300/0 conn(0x7fc6ac1b0a10 0x7fc6ac0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:19.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.121 7fc6b2bee700 1 --2- 172.21.2.101:0/4147528955 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6ac0d1870 0x7fc6ac1b03f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:19.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.121 7fc6b2bee700 1 -- 172.21.2.101:0/4147528955 >> 172.21.2.101:0/4147528955 conn(0x7fc6ac0ce450 msgr2=0x7fc6ac0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:19.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.121 7fc6b2bee700 1 -- 172.21.2.101:0/4147528955 shutdown_connections 2021-02-10T11:47:19.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.121 7fc6b2bee700 1 -- 172.21.2.101:0/4147528955 wait complete. 2021-02-10T11:47:19.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.136 7fc6b2bee700 1 Processor -- start 2021-02-10T11:47:19.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.136 7fc6b2bee700 1 -- start start 2021-02-10T11:47:19.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.137 7fc6b2bee700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6ac1b2d50 0x7fc6ac08af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:19.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.137 7fc6b2bee700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6ac0a7180 0x7fc6ac0a7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:19.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.137 7fc6b2bee700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6ac0ecb70 0x7fc6ac0ecf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:19.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.137 7fc6b2bee700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac0ed590 con 0x7fc6ac1b2d50 2021-02-10T11:47:19.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.137 7fc6b2bee700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac0ed740 con 0x7fc6ac0a7180 2021-02-10T11:47:19.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.137 7fc6b2bee700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac0ed960 con 0x7fc6ac0ecb70 2021-02-10T11:47:19.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.137 7fc6b0a0a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6ac1b2d50 0x7fc6ac08af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:19.137 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.137 7fc6b0a0a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6ac1b2d50 0x7fc6ac08af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33908/0 (socket says 172.21.2.101:33908) 2021-02-10T11:47:19.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.137 7fc6b0a0a700 1 -- 172.21.2.101:0/2194566451 learned_addr learned my addr 172.21.2.101:0/2194566451 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:19.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.137 7fc6b120b700 1 --2- 172.21.2.101:0/2194566451 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6ac0ecb70 0x7fc6ac0ecf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:19.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.137 7fc6abfff700 1 --2- 172.21.2.101:0/2194566451 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6ac0a7180 0x7fc6ac0a7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:19.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.137 7fc6b0a0a700 1 -- 172.21.2.101:0/2194566451 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6ac0ecb70 msgr2=0x7fc6ac0ecf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:19.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.137 7fc6b0a0a700 1 --2- 172.21.2.101:0/2194566451 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6ac0ecb70 0x7fc6ac0ecf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:19.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.137 7fc6b0a0a700 1 -- 172.21.2.101:0/2194566451 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6ac0a7180 msgr2=0x7fc6ac0a7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:19.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.137 7fc6b0a0a700 1 --2- 172.21.2.101:0/2194566451 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6ac0a7180 0x7fc6ac0a7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:19.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.137 7fc6b0a0a700 1 -- 172.21.2.101:0/2194566451 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac08b7a0 con 0x7fc6ac1b2d50 2021-02-10T11:47:19.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.137 7fc6abfff700 1 --2- 172.21.2.101:0/2194566451 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6ac0a7180 0x7fc6ac0a7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:47:19.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.138 7fc6b0a0a700 1 --2- 172.21.2.101:0/2194566451 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6ac1b2d50 0x7fc6ac08af30 secure :-1 s=READY pgs=3064 cs=0 l=1 rev1=1 rx=0x7fc694009630 tx=0x7fc69400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:19.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.138 7fc6a97fa700 1 -- 172.21.2.101:0/2194566451 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc6940093a0 con 0x7fc6ac1b2d50 2021-02-10T11:47:19.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.138 7fc6a97fa700 1 -- 172.21.2.101:0/2194566451 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69400bd70 con 0x7fc6ac1b2d50 2021-02-10T11:47:19.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.138 7fc6b2bee700 1 -- 172.21.2.101:0/2194566451 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ac08b7a0 con 0x7fc6ac1b2d50 2021-02-10T11:47:19.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.138 7fc6a97fa700 1 -- 172.21.2.101:0/2194566451 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc6940054d0 con 0x7fc6ac1b2d50 2021-02-10T11:47:19.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.138 7fc6b2bee700 1 -- 172.21.2.101:0/2194566451 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ac08ba40 con 0x7fc6ac1b2d50 2021-02-10T11:47:19.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.138 7fc6a97fa700 1 -- 172.21.2.101:0/2194566451 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc694007020 con 0x7fc6ac1b2d50 2021-02-10T11:47:19.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.138 7fc6a97fa700 1 --2- 172.21.2.101:0/2194566451 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6900186c0 0x7fc69001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:19.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.138 7fc6a97fa700 1 -- 172.21.2.101:0/2194566451 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fc694032440 con 0x7fc6ac1b2d50 2021-02-10T11:47:19.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.139 7fc6abfff700 1 --2- 172.21.2.101:0/2194566451 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6900186c0 0x7fc69001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:19.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.139 7fc6abfff700 1 --2- 172.21.2.101:0/2194566451 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6900186c0 0x7fc69001abc0 crc :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:19.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.168 7fc6b2bee700 1 -- 172.21.2.101:0/2194566451 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6ac199e20 con 0x7fc6ac1b2d50 2021-02-10T11:47:19.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.171 7fc6a97fa700 1 -- 172.21.2.101:0/2194566451 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc694009340 con 0x7fc6ac1b2d50 2021-02-10T11:47:19.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.300 7fc6b2bee700 1 -- 172.21.2.101:0/2194566451 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6ac199e20 con 0x7fc6ac1b2d50 2021-02-10T11:47:19.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.301 7fc6a97fa700 1 -- 172.21.2.101:0/2194566451 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) v1 ==== 95+0+1763 (secure 0 0 0) 0x7fc694007000 con 0x7fc6ac1b2d50 2021-02-10T11:47:19.298 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 150 2021-02-10T11:47:19.299 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:47:19.299 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":150,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":12780,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/2200088074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2200088074},{"type":"v1","addr":"172.21.2.101:6835","nonce":2200088074}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":144}],"filesystems":[{"mdsmap":{"epoch":150,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:46:35.149920","modified":"2021-02-10 11:47:18.645709","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":176,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[37],"metadata_pool":36,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":18}]} 2021-02-10T11:47:19.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.303 7fc6b2bee700 1 -- 172.21.2.101:0/2194566451 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6900186c0 msgr2=0x7fc69001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:19.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.303 7fc6b2bee700 1 --2- 172.21.2.101:0/2194566451 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6900186c0 0x7fc69001abc0 crc :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:19.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.303 7fc6b2bee700 1 -- 172.21.2.101:0/2194566451 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6ac1b2d50 msgr2=0x7fc6ac08af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:19.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.303 7fc6b2bee700 1 --2- 172.21.2.101:0/2194566451 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6ac1b2d50 0x7fc6ac08af30 secure :-1 s=READY pgs=3064 cs=0 l=1 rev1=1 rx=0x7fc694009630 tx=0x7fc69400b980).stop 2021-02-10T11:47:19.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.303 7fc6b2bee700 1 -- 172.21.2.101:0/2194566451 shutdown_connections 2021-02-10T11:47:19.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.303 7fc6b2bee700 1 --2- 172.21.2.101:0/2194566451 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc6900186c0 0x7fc69001abc0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:19.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.303 7fc6b2bee700 1 --2- 172.21.2.101:0/2194566451 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc6ac0ecb70 0x7fc6ac0ecf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:19.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.303 7fc6b2bee700 1 --2- 172.21.2.101:0/2194566451 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc6ac0a7180 0x7fc6ac0a7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:19.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.303 7fc6b2bee700 1 --2- 172.21.2.101:0/2194566451 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc6ac1b2d50 0x7fc6ac08af30 unknown :-1 s=CLOSED pgs=3064 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:19.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.303 7fc6b2bee700 1 -- 172.21.2.101:0/2194566451 >> 172.21.2.101:0/2194566451 conn(0x7fc6ac19c5a0 msgr2=0x7fc6ac197600 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:19.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.303 7fc6b2bee700 1 -- 172.21.2.101:0/2194566451 shutdown_connections 2021-02-10T11:47:19.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.303 7fc6b2bee700 1 -- 172.21.2.101:0/2194566451 wait complete. 2021-02-10T11:47:19.308 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-02-10T11:47:19.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.728 7feff55fd700 1 Processor -- start 2021-02-10T11:47:19.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.728 7feff55fd700 1 -- start start 2021-02-10T11:47:19.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.728 7feff55fd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feff00d1870 0x7feff01b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:19.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.728 7feff55fd700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7feff01b0a10 0x7feff00cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:19.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.728 7feff55fd700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff00f0b10 con 0x7feff00d1c70 2021-02-10T11:47:19.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.728 7feff55fd700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff00fe230 con 0x7feff01b0a10 2021-02-10T11:47:19.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.728 7feff55fd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff00e5ed0 con 0x7feff00d1870 2021-02-10T11:47:19.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.730 7fefeeabe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7feff01b0a10 0x7feff00cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:19.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.730 7fefef2bf700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feff00d1870 0x7feff01b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:19.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.730 7fefefac0700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7feff00d1c70 0x7feff00cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:56050/0 (socket says 172.21.2.101:56050) 2021-02-10T11:47:19.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.730 7fefefac0700 1 -- 172.21.2.101:0/1794162439 learned_addr learned my addr 172.21.2.101:0/1794162439 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:19.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.730 7fefef2bf700 1 -- 172.21.2.101:0/1794162439 >> v2:172.21.2.108:3300/0 conn(0x7feff01b0a10 msgr2=0x7feff00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:19.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.730 7fefef2bf700 1 --2- 172.21.2.101:0/1794162439 >> v2:172.21.2.108:3300/0 conn(0x7feff01b0a10 0x7feff00cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:19.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.730 7fefef2bf700 1 -- 172.21.2.101:0/1794162439 >> v1:172.21.2.108:6789/0 conn(0x7feff00d1c70 legacy=0x7feff00cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:19.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.730 7fefef2bf700 1 -- 172.21.2.101:0/1794162439 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff00f0650 con 0x7feff00d1870 2021-02-10T11:47:19.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.730 7fefeeabe700 1 --2- 172.21.2.101:0/1794162439 >> v2:172.21.2.108:3300/0 conn(0x7feff01b0a10 0x7feff00cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:47:19.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.731 7fefef2bf700 1 --2- 172.21.2.101:0/1794162439 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feff00d1870 0x7feff01b03f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fefd8006650 tx=0x7fefd8009a80).ready entity=mon.2 client_cookie=8f301701ffaf0aff server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:19.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.731 7fefee2bd700 1 -- 172.21.2.101:0/1794162439 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fefd80009b0 con 0x7feff00d1870 2021-02-10T11:47:19.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.731 7fefee2bd700 1 -- 172.21.2.101:0/1794162439 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd800c0c0 con 0x7feff00d1870 2021-02-10T11:47:19.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.731 7fefee2bd700 1 -- 172.21.2.101:0/1794162439 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fefd800c5e0 con 0x7feff00d1870 2021-02-10T11:47:19.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.731 7feff55fd700 1 -- 172.21.2.101:0/1794162439 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feff00d1870 msgr2=0x7feff01b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:19.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.731 7feff55fd700 1 --2- 172.21.2.101:0/1794162439 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feff00d1870 0x7feff01b03f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fefd8006650 tx=0x7fefd8009a80).stop 2021-02-10T11:47:19.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.732 7feff55fd700 1 -- 172.21.2.101:0/1794162439 shutdown_connections 2021-02-10T11:47:19.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.732 7feff55fd700 1 --2- 172.21.2.101:0/1794162439 >> v2:172.21.2.108:3300/0 conn(0x7feff01b0a10 0x7feff00cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:19.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.732 7feff55fd700 1 --2- 172.21.2.101:0/1794162439 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feff00d1870 0x7feff01b03f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:19.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.732 7feff55fd700 1 -- 172.21.2.101:0/1794162439 >> 172.21.2.101:0/1794162439 conn(0x7feff00ce450 msgr2=0x7feff00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:19.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.732 7feff55fd700 1 -- 172.21.2.101:0/1794162439 shutdown_connections 2021-02-10T11:47:19.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.732 7feff55fd700 1 -- 172.21.2.101:0/1794162439 wait complete. 2021-02-10T11:47:19.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.746 7feff55fd700 1 Processor -- start 2021-02-10T11:47:19.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.747 7feff55fd700 1 -- start start 2021-02-10T11:47:19.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.747 7feff55fd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feff01a4a80 0x7feff00906b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:19.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.747 7feff55fd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feff00c30f0 0x7feff00c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:19.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.747 7feff55fd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feff00ef5a0 0x7feff00ef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:19.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.747 7feff55fd700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff00c3b10 con 0x7feff01a4a80 2021-02-10T11:47:19.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.747 7feff55fd700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff00ef020 con 0x7feff00c30f0 2021-02-10T11:47:19.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.747 7feff55fd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff00effc0 con 0x7feff00ef5a0 2021-02-10T11:47:19.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.747 7fefef2bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feff01a4a80 0x7feff00906b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:19.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.747 7fefef2bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feff01a4a80 0x7feff00906b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33918/0 (socket says 172.21.2.101:33918) 2021-02-10T11:47:19.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.747 7fefef2bf700 1 -- 172.21.2.101:0/2603495659 learned_addr learned my addr 172.21.2.101:0/2603495659 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:19.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.747 7fefeeabe700 1 --2- 172.21.2.101:0/2603495659 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feff00c30f0 0x7feff00c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:19.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.747 7fefefac0700 1 --2- 172.21.2.101:0/2603495659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feff00ef5a0 0x7feff00ef9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:19.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.748 7fefef2bf700 1 -- 172.21.2.101:0/2603495659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feff00ef5a0 msgr2=0x7feff00ef9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:19.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.748 7fefef2bf700 1 --2- 172.21.2.101:0/2603495659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feff00ef5a0 0x7feff00ef9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:19.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.748 7fefef2bf700 1 -- 172.21.2.101:0/2603495659 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feff00c30f0 msgr2=0x7feff00c34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:19.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.748 7fefef2bf700 1 --2- 172.21.2.101:0/2603495659 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feff00c30f0 0x7feff00c34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:19.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.748 7fefef2bf700 1 -- 172.21.2.101:0/2603495659 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff00ebe70 con 0x7feff01a4a80 2021-02-10T11:47:19.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.748 7fefef2bf700 1 --2- 172.21.2.101:0/2603495659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feff01a4a80 0x7feff00906b0 secure :-1 s=READY pgs=3065 cs=0 l=1 rev1=1 rx=0x7fefd8009630 tx=0x7fefd800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:19.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.748 7fefe7fff700 1 -- 172.21.2.101:0/2603495659 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fefd80093a0 con 0x7feff01a4a80 2021-02-10T11:47:19.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.748 7fefe7fff700 1 -- 172.21.2.101:0/2603495659 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd800bd70 con 0x7feff01a4a80 2021-02-10T11:47:19.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.748 7fefe7fff700 1 -- 172.21.2.101:0/2603495659 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fefd8005560 con 0x7feff01a4a80 2021-02-10T11:47:19.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.748 7feff55fd700 1 -- 172.21.2.101:0/2603495659 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff00ebe70 con 0x7feff01a4a80 2021-02-10T11:47:19.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.748 7feff55fd700 1 -- 172.21.2.101:0/2603495659 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff00ebe70 con 0x7feff01a4a80 2021-02-10T11:47:19.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.749 7fefe7fff700 1 -- 172.21.2.101:0/2603495659 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fefd800a000 con 0x7feff01a4a80 2021-02-10T11:47:19.748 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.749 7fefe7fff700 1 --2- 172.21.2.101:0/2603495659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fefd40186c0 0x7fefd401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:19.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.749 7fefe7fff700 1 -- 172.21.2.101:0/2603495659 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fefd8032590 con 0x7feff01a4a80 2021-02-10T11:47:19.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.749 7fefeeabe700 1 --2- 172.21.2.101:0/2603495659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fefd40186c0 0x7fefd401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:19.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.749 7fefeeabe700 1 --2- 172.21.2.101:0/2603495659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fefd40186c0 0x7fefd401abc0 crc :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:19.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.778 7feff55fd700 1 -- 172.21.2.101:0/2603495659 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff00f0b10 con 0x7feff01a4a80 2021-02-10T11:47:19.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.781 7fefe7fff700 1 -- 172.21.2.101:0/2603495659 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fefd8030cf0 con 0x7feff01a4a80 2021-02-10T11:47:19.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:19.908 7feff55fd700 1 -- 172.21.2.101:0/2603495659 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7feff00f0b10 con 0x7feff01a4a80 2021-02-10T11:47:20.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:20.653 7fefe7fff700 1 -- 172.21.2.101:0/2603495659 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v151) v1 ==== 104+0+0 (secure 0 0 0) 0x7fefd800a000 con 0x7feff01a4a80 2021-02-10T11:47:20.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:20.679 7feff55fd700 1 -- 172.21.2.101:0/2603495659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fefd40186c0 msgr2=0x7fefd401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:20.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:20.679 7feff55fd700 1 --2- 172.21.2.101:0/2603495659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fefd40186c0 0x7fefd401abc0 crc :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:20.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:20.679 7feff55fd700 1 -- 172.21.2.101:0/2603495659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feff01a4a80 msgr2=0x7feff00906b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:20.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:20.680 7feff55fd700 1 --2- 172.21.2.101:0/2603495659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feff01a4a80 0x7feff00906b0 secure :-1 s=READY pgs=3065 cs=0 l=1 rev1=1 rx=0x7fefd8009630 tx=0x7fefd800b980).stop 2021-02-10T11:47:20.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:20.680 7feff55fd700 1 -- 172.21.2.101:0/2603495659 shutdown_connections 2021-02-10T11:47:20.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:20.680 7feff55fd700 1 --2- 172.21.2.101:0/2603495659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fefd40186c0 0x7fefd401abc0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:20.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:20.680 7feff55fd700 1 --2- 172.21.2.101:0/2603495659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feff00ef5a0 0x7feff00ef9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:20.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:20.680 7feff55fd700 1 --2- 172.21.2.101:0/2603495659 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feff00c30f0 0x7feff00c34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:20.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:20.680 7feff55fd700 1 --2- 172.21.2.101:0/2603495659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feff01a4a80 0x7feff00906b0 unknown :-1 s=CLOSED pgs=3065 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:20.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:20.680 7feff55fd700 1 -- 172.21.2.101:0/2603495659 >> 172.21.2.101:0/2603495659 conn(0x7feff00a6bb0 msgr2=0x7feff0095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:20.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:20.680 7feff55fd700 1 -- 172.21.2.101:0/2603495659 shutdown_connections 2021-02-10T11:47:20.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:20.680 7feff55fd700 1 -- 172.21.2.101:0/2603495659 wait complete. 2021-02-10T11:47:20.685 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-02-10T11:47:21.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.105 7f5b1c9a2700 1 Processor -- start 2021-02-10T11:47:21.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.105 7f5b1c9a2700 1 -- start start 2021-02-10T11:47:21.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.105 7f5b1c9a2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b180d1880 0x7f5b181b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:21.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.105 7f5b1c9a2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5b180cff80 0x7f5b18109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:21.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.105 7f5b1c9a2700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b18109a70 con 0x7f5b181b0af0 2021-02-10T11:47:21.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.105 7f5b1c9a2700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b18109760 con 0x7f5b180cff80 2021-02-10T11:47:21.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.105 7f5b1c9a2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b18088070 con 0x7f5b180d1880 2021-02-10T11:47:21.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.106 7f5b16e1d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5b180cff80 0x7f5b18109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:21.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.106 7f5b16e1d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5b180cff80 0x7f5b18109140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33930/0 (socket says 172.21.2.101:33930) 2021-02-10T11:47:21.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.106 7f5b16e1d700 1 -- 172.21.2.101:0/2544031184 learned_addr learned my addr 172.21.2.101:0/2544031184 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:21.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.106 7f5b16e1d700 1 -- 172.21.2.101:0/2544031184 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b180d1880 msgr2=0x7f5b181b04d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:21.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.106 7f5b16e1d700 1 --2- 172.21.2.101:0/2544031184 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b180d1880 0x7f5b181b04d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:21.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.106 7f5b16e1d700 1 -- 172.21.2.101:0/2544031184 >> v1:172.21.2.108:6789/0 conn(0x7f5b181b0af0 legacy=0x7f5b18108950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:21.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.106 7f5b16e1d700 1 -- 172.21.2.101:0/2544031184 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b18088260 con 0x7f5b180cff80 2021-02-10T11:47:21.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.106 7f5b16e1d700 1 --2- 172.21.2.101:0/2544031184 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b180cff80 0x7f5b18109140 secure :-1 s=READY pgs=3066 cs=0 l=1 rev1=1 rx=0x7f5b10006f00 tx=0x7f5b10009f10).ready entity=mon.0 client_cookie=3a278c000c1336f3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:21.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.106 7f5b1561a700 1 -- 172.21.2.101:0/2544031184 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5b1000b3b0 con 0x7f5b180cff80 2021-02-10T11:47:21.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.106 7f5b1561a700 1 -- 172.21.2.101:0/2544031184 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1000b630 con 0x7f5b180cff80 2021-02-10T11:47:21.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.106 7f5b1561a700 1 -- 172.21.2.101:0/2544031184 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5b1000bb50 con 0x7f5b180cff80 2021-02-10T11:47:21.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.106 7f5b1c9a2700 1 -- 172.21.2.101:0/2544031184 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b180cff80 msgr2=0x7f5b18109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:21.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.106 7f5b1c9a2700 1 --2- 172.21.2.101:0/2544031184 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b180cff80 0x7f5b18109140 secure :-1 s=READY pgs=3066 cs=0 l=1 rev1=1 rx=0x7f5b10006f00 tx=0x7f5b10009f10).stop 2021-02-10T11:47:21.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.107 7f5b1c9a2700 1 -- 172.21.2.101:0/2544031184 shutdown_connections 2021-02-10T11:47:21.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.107 7f5b1c9a2700 1 --2- 172.21.2.101:0/2544031184 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b180cff80 0x7f5b18109140 unknown :-1 s=CLOSED pgs=3066 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:21.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.107 7f5b1c9a2700 1 --2- 172.21.2.101:0/2544031184 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b180d1880 0x7f5b181b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:21.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.107 7f5b1c9a2700 1 -- 172.21.2.101:0/2544031184 >> 172.21.2.101:0/2544031184 conn(0x7f5b180ce430 msgr2=0x7f5b180d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:21.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.107 7f5b1c9a2700 1 -- 172.21.2.101:0/2544031184 shutdown_connections 2021-02-10T11:47:21.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.107 7f5b1c9a2700 1 -- 172.21.2.101:0/2544031184 wait complete. 2021-02-10T11:47:21.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.122 7f5b1c9a2700 1 Processor -- start 2021-02-10T11:47:21.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.122 7f5b1c9a2700 1 -- start start 2021-02-10T11:47:21.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.122 7f5b1c9a2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5b1819e5b0 0x7f5b181afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:21.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.122 7f5b1c9a2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b1819e9b0 0x7f5b181b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:21.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.122 7f5b1c9a2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b181b0250 0x7f5b181b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:21.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.122 7f5b1c9a2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b18195bd0 con 0x7f5b181b0250 2021-02-10T11:47:21.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.122 7f5b1c9a2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b181b0840 con 0x7f5b1819e5b0 2021-02-10T11:47:21.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.122 7f5b1c9a2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1819eed0 con 0x7f5b1819e9b0 2021-02-10T11:47:21.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.122 7f5b16e1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b181b0250 0x7f5b181b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:21.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.122 7f5b16e1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b181b0250 0x7f5b181b13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33936/0 (socket says 172.21.2.101:33936) 2021-02-10T11:47:21.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.122 7f5b16e1d700 1 -- 172.21.2.101:0/4041687396 learned_addr learned my addr 172.21.2.101:0/4041687396 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:21.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.123 7f5b16e1d700 1 -- 172.21.2.101:0/4041687396 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b1819e9b0 msgr2=0x7f5b181b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:21.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.123 7f5b16e1d700 1 --2- 172.21.2.101:0/4041687396 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b1819e9b0 0x7f5b181b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:21.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.123 7f5b16e1d700 1 -- 172.21.2.101:0/4041687396 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5b1819e5b0 msgr2=0x7f5b181afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:21.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.123 7f5b16e1d700 1 --2- 172.21.2.101:0/4041687396 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5b1819e5b0 0x7f5b181afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:21.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.123 7f5b16e1d700 1 -- 172.21.2.101:0/4041687396 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1819f0b0 con 0x7f5b181b0250 2021-02-10T11:47:21.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.123 7f5b16e1d700 1 --2- 172.21.2.101:0/4041687396 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b181b0250 0x7f5b181b13e0 secure :-1 s=READY pgs=3067 cs=0 l=1 rev1=1 rx=0x7f5b1000b9b0 tx=0x7f5b1000bd50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:21.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.123 7f5b0effd700 1 -- 172.21.2.101:0/4041687396 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5b10009450 con 0x7f5b181b0250 2021-02-10T11:47:21.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.123 7f5b0effd700 1 -- 172.21.2.101:0/4041687396 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1000c2f0 con 0x7f5b181b0250 2021-02-10T11:47:21.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.123 7f5b0effd700 1 -- 172.21.2.101:0/4041687396 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5b1000c790 con 0x7f5b181b0250 2021-02-10T11:47:21.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.123 7f5b1c9a2700 1 -- 172.21.2.101:0/4041687396 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1819f0b0 con 0x7f5b181b0250 2021-02-10T11:47:21.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.123 7f5b1c9a2700 1 -- 172.21.2.101:0/4041687396 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1819f0b0 con 0x7f5b181b0250 2021-02-10T11:47:21.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.124 7f5b0effd700 1 -- 172.21.2.101:0/4041687396 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5b10009450 con 0x7f5b181b0250 2021-02-10T11:47:21.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.124 7f5b0effd700 1 --2- 172.21.2.101:0/4041687396 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5afc018730 0x7f5afc01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:21.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.124 7f5b0effd700 1 -- 172.21.2.101:0/4041687396 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f5b10032a80 con 0x7f5b181b0250 2021-02-10T11:47:21.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.124 7f5b1661c700 1 --2- 172.21.2.101:0/4041687396 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5afc018730 0x7f5afc01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:21.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.124 7f5b1661c700 1 --2- 172.21.2.101:0/4041687396 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5afc018730 0x7f5afc01ac10 crc :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:21.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.154 7f5b1c9a2700 1 -- 172.21.2.101:0/4041687396 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b18082aa0 con 0x7f5b181b0250 2021-02-10T11:47:21.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.157 7f5b0effd700 1 -- 172.21.2.101:0/4041687396 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5b10000f90 con 0x7f5b181b0250 2021-02-10T11:47:21.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.283 7f5b1c9a2700 1 -- 172.21.2.101:0/4041687396 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5b18195bd0 con 0x7f5b181b0250 2021-02-10T11:47:21.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.663 7f5b0effd700 1 -- 172.21.2.101:0/4041687396 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v177) v1 ==== 185+0+0 (secure 0 0 0) 0x7f5b10009450 con 0x7f5b181b0250 2021-02-10T11:47:21.698 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' removed 2021-02-10T11:47:21.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.703 7f5b1c9a2700 1 -- 172.21.2.101:0/4041687396 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5afc018730 msgr2=0x7f5afc01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:21.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.703 7f5b1c9a2700 1 --2- 172.21.2.101:0/4041687396 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5afc018730 0x7f5afc01ac10 crc :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:21.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.703 7f5b1c9a2700 1 -- 172.21.2.101:0/4041687396 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b181b0250 msgr2=0x7f5b181b13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:21.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.703 7f5b1c9a2700 1 --2- 172.21.2.101:0/4041687396 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b181b0250 0x7f5b181b13e0 secure :-1 s=READY pgs=3067 cs=0 l=1 rev1=1 rx=0x7f5b1000b9b0 tx=0x7f5b1000bd50).stop 2021-02-10T11:47:21.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.703 7f5b1c9a2700 1 -- 172.21.2.101:0/4041687396 shutdown_connections 2021-02-10T11:47:21.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.703 7f5b1c9a2700 1 --2- 172.21.2.101:0/4041687396 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5afc018730 0x7f5afc01ac10 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:21.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.703 7f5b1c9a2700 1 --2- 172.21.2.101:0/4041687396 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5b1819e9b0 0x7f5b181b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:21.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.703 7f5b1c9a2700 1 --2- 172.21.2.101:0/4041687396 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5b181b0250 0x7f5b181b13e0 unknown :-1 s=CLOSED pgs=3067 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:21.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.703 7f5b1c9a2700 1 --2- 172.21.2.101:0/4041687396 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5b1819e5b0 0x7f5b181afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:21.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.703 7f5b1c9a2700 1 -- 172.21.2.101:0/4041687396 >> 172.21.2.101:0/4041687396 conn(0x7f5b1819f940 msgr2=0x7f5b1818c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:21.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.703 7f5b1c9a2700 1 -- 172.21.2.101:0/4041687396 shutdown_connections 2021-02-10T11:47:21.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:21.703 7f5b1c9a2700 1 -- 172.21.2.101:0/4041687396 wait complete. 2021-02-10T11:47:21.708 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-02-10T11:47:22.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.128 7feb4270f700 1 Processor -- start 2021-02-10T11:47:22.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.128 7feb4270f700 1 -- start start 2021-02-10T11:47:22.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.129 7feb4270f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7feb3c0d1c80 0x7feb3c0cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:22.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.129 7feb4270f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7feb3c0cf590 0x7feb3c0cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:22.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.129 7feb4270f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb3c0fe1d0 con 0x7feb3c0d1880 2021-02-10T11:47:22.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.129 7feb4270f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb3c1b0cc0 con 0x7feb3c0cf590 2021-02-10T11:47:22.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.129 7feb4270f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb3c109a70 con 0x7feb3c0d1c80 2021-02-10T11:47:22.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.129 7feb40d2c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7feb3c0cf590 0x7feb3c0cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:22.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.129 7feb40d2c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7feb3c0cf590 0x7feb3c0cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33942/0 (socket says 172.21.2.101:33942) 2021-02-10T11:47:22.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.129 7feb40d2c700 1 -- 172.21.2.101:0/866434671 learned_addr learned my addr 172.21.2.101:0/866434671 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:22.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.129 7feb40d2c700 1 -- 172.21.2.101:0/866434671 >> v2:172.21.2.108:3300/0 conn(0x7feb3c0d1c80 msgr2=0x7feb3c0cef70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:22.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.129 7feb40d2c700 1 --2- 172.21.2.101:0/866434671 >> v2:172.21.2.108:3300/0 conn(0x7feb3c0d1c80 0x7feb3c0cef70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:22.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.129 7feb40d2c700 1 -- 172.21.2.101:0/866434671 >> v1:172.21.2.108:6789/0 conn(0x7feb3c0d1880 legacy=0x7feb3c1b04d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:22.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.129 7feb40d2c700 1 -- 172.21.2.101:0/866434671 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3c088070 con 0x7feb3c0cf590 2021-02-10T11:47:22.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.129 7feb40d2c700 1 --2- 172.21.2.101:0/866434671 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb3c0cf590 0x7feb3c0cf990 secure :-1 s=READY pgs=3068 cs=0 l=1 rev1=1 rx=0x7feb34006f00 tx=0x7feb34009f10).ready entity=mon.0 client_cookie=6ec23735e6665177 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:22.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.130 7feb3affd700 1 -- 172.21.2.101:0/866434671 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feb3400b3b0 con 0x7feb3c0cf590 2021-02-10T11:47:22.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.130 7feb3affd700 1 -- 172.21.2.101:0/866434671 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3400b730 con 0x7feb3c0cf590 2021-02-10T11:47:22.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.130 7feb3affd700 1 -- 172.21.2.101:0/866434671 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feb3400bc50 con 0x7feb3c0cf590 2021-02-10T11:47:22.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.130 7feb4270f700 1 -- 172.21.2.101:0/866434671 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb3c0cf590 msgr2=0x7feb3c0cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:22.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.130 7feb4270f700 1 --2- 172.21.2.101:0/866434671 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb3c0cf590 0x7feb3c0cf990 secure :-1 s=READY pgs=3068 cs=0 l=1 rev1=1 rx=0x7feb34006f00 tx=0x7feb34009f10).stop 2021-02-10T11:47:22.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.130 7feb4270f700 1 -- 172.21.2.101:0/866434671 shutdown_connections 2021-02-10T11:47:22.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.130 7feb4270f700 1 --2- 172.21.2.101:0/866434671 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb3c0cf590 0x7feb3c0cf990 unknown :-1 s=CLOSED pgs=3068 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:22.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.130 7feb4270f700 1 --2- 172.21.2.101:0/866434671 >> v2:172.21.2.108:3300/0 conn(0x7feb3c0d1c80 0x7feb3c0cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:22.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.130 7feb4270f700 1 -- 172.21.2.101:0/866434671 >> 172.21.2.101:0/866434671 conn(0x7feb3c0ce430 msgr2=0x7feb3c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:22.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.130 7feb4270f700 1 -- 172.21.2.101:0/866434671 shutdown_connections 2021-02-10T11:47:22.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.130 7feb4270f700 1 -- 172.21.2.101:0/866434671 wait complete. 2021-02-10T11:47:22.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.145 7feb4270f700 1 Processor -- start 2021-02-10T11:47:22.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.145 7feb4270f700 1 -- start start 2021-02-10T11:47:22.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.145 7feb4270f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feb3c08fb40 0x7feb3c195720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:22.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.145 7feb4270f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb3c09b4f0 0x7feb3c09b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:22.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.145 7feb4270f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feb3c09bf10 0x7feb3c1940a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:22.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.145 7feb4270f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c1951a0 con 0x7feb3c09b4f0 2021-02-10T11:47:22.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.145 7feb4270f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c094990 con 0x7feb3c08fb40 2021-02-10T11:47:22.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.145 7feb4270f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c1946c0 con 0x7feb3c09bf10 2021-02-10T11:47:22.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.145 7feb3b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb3c09b4f0 0x7feb3c09b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:22.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.145 7feb3b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb3c09b4f0 0x7feb3c09b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33948/0 (socket says 172.21.2.101:33948) 2021-02-10T11:47:22.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.145 7feb3b7fe700 1 -- 172.21.2.101:0/447257882 learned_addr learned my addr 172.21.2.101:0/447257882 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:22.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.146 7feb3b7fe700 1 -- 172.21.2.101:0/447257882 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feb3c09bf10 msgr2=0x7feb3c1940a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:22.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.146 7feb3b7fe700 1 --2- 172.21.2.101:0/447257882 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feb3c09bf10 0x7feb3c1940a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:22.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.146 7feb3b7fe700 1 -- 172.21.2.101:0/447257882 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feb3c08fb40 msgr2=0x7feb3c195720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:22.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.146 7feb3b7fe700 1 --2- 172.21.2.101:0/447257882 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feb3c08fb40 0x7feb3c195720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:22.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.146 7feb3b7fe700 1 -- 172.21.2.101:0/447257882 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3c1948e0 con 0x7feb3c09b4f0 2021-02-10T11:47:22.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.146 7feb3b7fe700 1 --2- 172.21.2.101:0/447257882 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb3c09b4f0 0x7feb3c09b8f0 secure :-1 s=READY pgs=3069 cs=0 l=1 rev1=1 rx=0x7feb3000b5c0 tx=0x7feb3000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:22.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.146 7feb38ff9700 1 -- 172.21.2.101:0/447257882 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feb3000bd50 con 0x7feb3c09b4f0 2021-02-10T11:47:22.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.146 7feb38ff9700 1 -- 172.21.2.101:0/447257882 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb30006030 con 0x7feb3c09b4f0 2021-02-10T11:47:22.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.146 7feb4270f700 1 -- 172.21.2.101:0/447257882 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb3c1948e0 con 0x7feb3c09b4f0 2021-02-10T11:47:22.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.146 7feb4270f700 1 -- 172.21.2.101:0/447257882 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3c194e70 con 0x7feb3c09b4f0 2021-02-10T11:47:22.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.146 7feb38ff9700 1 -- 172.21.2.101:0/447257882 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feb300054d0 con 0x7feb3c09b4f0 2021-02-10T11:47:22.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.147 7feb38ff9700 1 -- 172.21.2.101:0/447257882 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7feb30006210 con 0x7feb3c09b4f0 2021-02-10T11:47:22.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.147 7feb38ff9700 1 --2- 172.21.2.101:0/447257882 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feb1c0186c0 0x7feb1c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:22.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.147 7feb38ff9700 1 -- 172.21.2.101:0/447257882 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5221+0+0 (secure 0 0 0) 0x7feb30032410 con 0x7feb3c09b4f0 2021-02-10T11:47:22.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.147 7feb3bfff700 1 --2- 172.21.2.101:0/447257882 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feb1c0186c0 0x7feb1c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:22.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.147 7feb3bfff700 1 --2- 172.21.2.101:0/447257882 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feb1c0186c0 0x7feb1c01abc0 crc :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:22.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.179 7feb4270f700 1 -- 172.21.2.101:0/447257882 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb3c082ab0 con 0x7feb3c09b4f0 2021-02-10T11:47:22.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.181 7feb38ff9700 1 -- 172.21.2.101:0/447257882 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7feb30000940 con 0x7feb3c09b4f0 2021-02-10T11:47:22.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.309 7feb4270f700 1 -- 172.21.2.101:0/447257882 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feb3c082ab0 con 0x7feb3c09b4f0 2021-02-10T11:47:22.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.668 7feb38ff9700 1 -- 172.21.2.101:0/447257882 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v178) v1 ==== 173+0+0 (secure 0 0 0) 0x7feb3001f540 con 0x7feb3c09b4f0 2021-02-10T11:47:22.671 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' removed 2021-02-10T11:47:22.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.676 7feb4270f700 1 -- 172.21.2.101:0/447257882 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feb1c0186c0 msgr2=0x7feb1c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:22.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.676 7feb4270f700 1 --2- 172.21.2.101:0/447257882 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feb1c0186c0 0x7feb1c01abc0 crc :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:22.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.676 7feb4270f700 1 -- 172.21.2.101:0/447257882 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb3c09b4f0 msgr2=0x7feb3c09b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:22.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.676 7feb4270f700 1 --2- 172.21.2.101:0/447257882 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb3c09b4f0 0x7feb3c09b8f0 secure :-1 s=READY pgs=3069 cs=0 l=1 rev1=1 rx=0x7feb3000b5c0 tx=0x7feb3000b960).stop 2021-02-10T11:47:22.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.676 7feb4270f700 1 -- 172.21.2.101:0/447257882 shutdown_connections 2021-02-10T11:47:22.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.676 7feb4270f700 1 --2- 172.21.2.101:0/447257882 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feb1c0186c0 0x7feb1c01abc0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:22.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.676 7feb4270f700 1 --2- 172.21.2.101:0/447257882 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feb3c09bf10 0x7feb3c1940a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:22.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.676 7feb4270f700 1 --2- 172.21.2.101:0/447257882 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb3c09b4f0 0x7feb3c09b8f0 unknown :-1 s=CLOSED pgs=3069 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:22.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.676 7feb4270f700 1 --2- 172.21.2.101:0/447257882 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feb3c08fb40 0x7feb3c195720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:22.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.676 7feb4270f700 1 -- 172.21.2.101:0/447257882 >> 172.21.2.101:0/447257882 conn(0x7feb3c09afc0 msgr2=0x7feb3c0953e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:22.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.676 7feb4270f700 1 -- 172.21.2.101:0/447257882 shutdown_connections 2021-02-10T11:47:22.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:22.677 7feb4270f700 1 -- 172.21.2.101:0/447257882 wait complete. 2021-02-10T11:47:22.682 INFO:tasks.ceph.mds.a:Restarting daemon 2021-02-10T11:47:22.682 INFO:tasks.ceph.mds.a:Stopping old one... 2021-02-10T11:47:22.682 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-02-10T11:47:22.683 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:47:22.683 INFO:tasks.ceph.mds.b:Restarting daemon 2021-02-10T11:47:22.683 INFO:tasks.ceph.mds.b:Stopping old one... 2021-02-10T11:47:22.683 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-02-10T11:47:22.683 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:47:22.689 INFO:tasks.ceph.mds.a:Stopped 2021-02-10T11:47:22.689 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-02-10T11:47:22.690 INFO:tasks.ceph.mds.b:Stopped 2021-02-10T11:47:22.690 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-02-10T11:47:22.731 INFO:tasks.ceph.mds.b:Started 2021-02-10T11:47:22.731 DEBUG:teuthology.parallel:result is None 2021-02-10T11:47:22.732 INFO:tasks.ceph.mds.a:Started 2021-02-10T11:47:22.732 DEBUG:teuthology.parallel:result is None 2021-02-10T11:47:22.732 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd blacklist clear 2021-02-10T11:47:23.135 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:47:23.140 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:47:23.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.146 7fcbad018700 1 Processor -- start 2021-02-10T11:47:23.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.146 7fcbad018700 1 -- start start 2021-02-10T11:47:23.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.147 7fcbad018700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcba80d1c70 0x7fcba80f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:23.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.147 7fcbad018700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba80f0b10 con 0x7fcba80d0340 2021-02-10T11:47:23.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.147 7fcbad018700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba81b0be0 con 0x7fcba80d1870 2021-02-10T11:47:23.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.147 7fcbad018700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba80e5ed0 con 0x7fcba80d1c70 2021-02-10T11:47:23.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.147 7fcba659c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fcba80d0340 0x7fcba80ef640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34458/0 (socket says 172.21.2.101:34458) 2021-02-10T11:47:23.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.147 7fcba659c700 1 -- 172.21.2.101:0/3738048320 learned_addr learned my addr 172.21.2.101:0/3738048320 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:23.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.147 7fcba759e700 1 --2- 172.21.2.101:0/3738048320 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcba80d1c70 0x7fcba80f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:23.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.147 7fcba5d9b700 1 -- 172.21.2.101:0/3738048320 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1427264698 0 0) 0x7fcb9c0009b0 con 0x7fcba80d0340 2021-02-10T11:47:23.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.147 7fcba5d9b700 1 -- 172.21.2.101:0/3738048320 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb90002fe0 con 0x7fcba80d0340 2021-02-10T11:47:23.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.147 7fcba759e700 1 -- 172.21.2.101:0/3738048320 >> v1:172.21.2.108:6789/0 conn(0x7fcba80d1870 legacy=0x7fcba81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:23.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.147 7fcba759e700 1 -- 172.21.2.101:0/3738048320 >> v1:172.21.2.101:6789/0 conn(0x7fcba80d0340 legacy=0x7fcba80ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:23.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.147 7fcba759e700 1 -- 172.21.2.101:0/3738048320 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba80efe30 con 0x7fcba80d1c70 2021-02-10T11:47:23.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.148 7fcba759e700 1 --2- 172.21.2.101:0/3738048320 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcba80d1c70 0x7fcba80f20a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7fcb98006ad0 tx=0x7fcb98009ad0).ready entity=mon.2 client_cookie=d34ddc88ec5f43d6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:23.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.148 7fcba5d9b700 1 -- 172.21.2.101:0/3738048320 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcb980022e0 con 0x7fcba80d1c70 2021-02-10T11:47:23.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.148 7fcba5d9b700 1 -- 172.21.2.101:0/3738048320 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb98006410 con 0x7fcba80d1c70 2021-02-10T11:47:23.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.148 7fcba5d9b700 1 -- 172.21.2.101:0/3738048320 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcb98002620 con 0x7fcba80d1c70 2021-02-10T11:47:23.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.148 7fcbad018700 1 -- 172.21.2.101:0/3738048320 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcba80d1c70 msgr2=0x7fcba80f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:23.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.148 7fcbad018700 1 --2- 172.21.2.101:0/3738048320 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcba80d1c70 0x7fcba80f20a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7fcb98006ad0 tx=0x7fcb98009ad0).stop 2021-02-10T11:47:23.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.148 7fcbad018700 1 -- 172.21.2.101:0/3738048320 shutdown_connections 2021-02-10T11:47:23.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.148 7fcbad018700 1 --2- 172.21.2.101:0/3738048320 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcba80d1c70 0x7fcba80f20a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:23.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.148 7fcbad018700 1 -- 172.21.2.101:0/3738048320 >> 172.21.2.101:0/3738048320 conn(0x7fcba80ce450 msgr2=0x7fcba80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:23.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.149 7fcbad018700 1 -- 172.21.2.101:0/3738048320 shutdown_connections 2021-02-10T11:47:23.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.149 7fcbad018700 1 -- 172.21.2.101:0/3738048320 wait complete. 2021-02-10T11:47:23.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.163 7fcbad018700 1 Processor -- start 2021-02-10T11:47:23.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.163 7fcbad018700 1 -- start start 2021-02-10T11:47:23.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.164 7fcbad018700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcba8095040 0x7fcba8193320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.164 7fcbad018700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcba8095520 0x7fcba80bfa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.164 7fcbad018700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcba80c00b0 0x7fcba80eec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.164 7fcbad018700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba80bdd10 con 0x7fcba80c00b0 2021-02-10T11:47:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.164 7fcbad018700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba80c04b0 con 0x7fcba8095520 2021-02-10T11:47:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.164 7fcbad018700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba80ef290 con 0x7fcba8095040 2021-02-10T11:47:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.164 7fcba759e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcba80c00b0 0x7fcba80eec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.164 7fcba759e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcba80c00b0 0x7fcba80eec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33970/0 (socket says 172.21.2.101:33970) 2021-02-10T11:47:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.164 7fcba759e700 1 -- 172.21.2.101:0/4133831755 learned_addr learned my addr 172.21.2.101:0/4133831755 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.164 7fcba659c700 1 --2- 172.21.2.101:0/4133831755 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcba8095520 0x7fcba80bfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.164 7fcba6d9d700 1 --2- 172.21.2.101:0/4133831755 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcba8095040 0x7fcba8193320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.164 7fcba759e700 1 -- 172.21.2.101:0/4133831755 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcba8095040 msgr2=0x7fcba8193320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.164 7fcba759e700 1 --2- 172.21.2.101:0/4133831755 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcba8095040 0x7fcba8193320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.164 7fcba759e700 1 -- 172.21.2.101:0/4133831755 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcba8095520 msgr2=0x7fcba80bfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.164 7fcba759e700 1 --2- 172.21.2.101:0/4133831755 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcba8095520 0x7fcba80bfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.164 7fcba759e700 1 -- 172.21.2.101:0/4133831755 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba80ef4b0 con 0x7fcba80c00b0 2021-02-10T11:47:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.164 7fcba659c700 1 --2- 172.21.2.101:0/4133831755 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcba8095520 0x7fcba80bfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:47:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.164 7fcba759e700 1 --2- 172.21.2.101:0/4133831755 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcba80c00b0 0x7fcba80eec70 secure :-1 s=READY pgs=3076 cs=0 l=1 rev1=1 rx=0x7fcb9800b610 tx=0x7fcb9800b9b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.165 7fcb977fe700 1 -- 172.21.2.101:0/4133831755 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcb98000da0 con 0x7fcba80c00b0 2021-02-10T11:47:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.165 7fcb977fe700 1 -- 172.21.2.101:0/4133831755 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb98000b00 con 0x7fcba80c00b0 2021-02-10T11:47:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.165 7fcb977fe700 1 -- 172.21.2.101:0/4133831755 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcb98002220 con 0x7fcba80c00b0 2021-02-10T11:47:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.165 7fcbad018700 1 -- 172.21.2.101:0/4133831755 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba80ef4b0 con 0x7fcba80c00b0 2021-02-10T11:47:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.165 7fcbad018700 1 -- 172.21.2.101:0/4133831755 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba81a72d0 con 0x7fcba80c00b0 2021-02-10T11:47:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.165 7fcb977fe700 1 -- 172.21.2.101:0/4133831755 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fcb98005330 con 0x7fcba80c00b0 2021-02-10T11:47:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.165 7fcb977fe700 1 --2- 172.21.2.101:0/4133831755 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcb8c0186c0 0x7fcb8c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.165 7fcb977fe700 1 -- 172.21.2.101:0/4133831755 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 4839+0+0 (secure 0 0 0) 0x7fcb980322d0 con 0x7fcba80c00b0 2021-02-10T11:47:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.166 7fcba6d9d700 1 --2- 172.21.2.101:0/4133831755 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcb8c0186c0 0x7fcb8c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:23.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.166 7fcba6d9d700 1 --2- 172.21.2.101:0/4133831755 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcb8c0186c0 0x7fcb8c01abc0 crc :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:23.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.196 7fcbad018700 1 -- 172.21.2.101:0/4133831755 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba80efa40 con 0x7fcba80c00b0 2021-02-10T11:47:23.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.199 7fcb977fe700 1 -- 172.21.2.101:0/4133831755 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fcb980176a0 con 0x7fcba80c00b0 2021-02-10T11:47:23.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.339 7fcbad018700 1 -- 172.21.2.101:0/4133831755 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fcba80efa40 con 0x7fcba80c00b0 2021-02-10T11:47:23.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.671 7fcb977fe700 1 -- 172.21.2.101:0/4133831755 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v179) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcb980176a0 con 0x7fcba80c00b0 2021-02-10T11:47:23.702 INFO:teuthology.orchestra.run.gibba001.stderr: removed all blacklist entries 2021-02-10T11:47:23.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.707 7fcbad018700 1 -- 172.21.2.101:0/4133831755 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcb8c0186c0 msgr2=0x7fcb8c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:23.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.707 7fcbad018700 1 --2- 172.21.2.101:0/4133831755 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcb8c0186c0 0x7fcb8c01abc0 crc :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:23.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.707 7fcbad018700 1 -- 172.21.2.101:0/4133831755 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcba80c00b0 msgr2=0x7fcba80eec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:23.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.707 7fcbad018700 1 --2- 172.21.2.101:0/4133831755 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcba80c00b0 0x7fcba80eec70 secure :-1 s=READY pgs=3076 cs=0 l=1 rev1=1 rx=0x7fcb9800b610 tx=0x7fcb9800b9b0).stop 2021-02-10T11:47:23.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.707 7fcbad018700 1 -- 172.21.2.101:0/4133831755 shutdown_connections 2021-02-10T11:47:23.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.707 7fcbad018700 1 --2- 172.21.2.101:0/4133831755 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fcba80c00b0 0x7fcba80eec70 unknown :-1 s=CLOSED pgs=3076 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:23.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.707 7fcbad018700 1 --2- 172.21.2.101:0/4133831755 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fcba8095520 0x7fcba80bfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:23.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.707 7fcbad018700 1 --2- 172.21.2.101:0/4133831755 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fcb8c0186c0 0x7fcb8c01abc0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:23.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.707 7fcbad018700 1 --2- 172.21.2.101:0/4133831755 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fcba8095040 0x7fcba8193320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:23.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.707 7fcbad018700 1 -- 172.21.2.101:0/4133831755 >> 172.21.2.101:0/4133831755 conn(0x7fcba80efc90 msgr2=0x7fcba80bf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:23.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.707 7fcbad018700 1 -- 172.21.2.101:0/4133831755 shutdown_connections 2021-02-10T11:47:23.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:23.707 7fcbad018700 1 -- 172.21.2.101:0/4133831755 wait complete. 2021-02-10T11:47:23.712 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2021-02-10T11:47:23.712 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:47:24.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.126 7f1d0deab700 1 Processor -- start 2021-02-10T11:47:24.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.126 7f1d0deab700 1 -- start start 2021-02-10T11:47:24.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.127 7f1d0deab700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d080d0340 0x7f1d080ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:24.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.127 7f1d0deab700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1d080d1c70 0x7f1d080efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:24.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.127 7f1d0deab700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d080f0b10 con 0x7f1d080d1870 2021-02-10T11:47:24.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.127 7f1d0deab700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d080e5ed0 con 0x7f1d080d1c70 2021-02-10T11:47:24.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.127 7f1d0deab700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d081b0be0 con 0x7f1d080d0340 2021-02-10T11:47:24.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.127 7f1d077fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f1d080d1870 0x7f1d081b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34470/0 (socket says 172.21.2.101:34470) 2021-02-10T11:47:24.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.127 7f1d077fe700 1 -- 172.21.2.101:0/1091925146 learned_addr learned my addr 172.21.2.101:0/1091925146 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:24.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.127 7f1d07fff700 1 --2- 172.21.2.101:0/1091925146 >> v2:172.21.2.108:3300/0 conn(0x7f1d080d1c70 0x7f1d080efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:24.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.127 7f1d06ffd700 1 --2- 172.21.2.101:0/1091925146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d080d0340 0x7f1d080ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:24.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.127 7f1d067fc700 1 -- 172.21.2.101:0/1091925146 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378666277 0 0) 0x7f1cf00010b0 con 0x7f1d080d1870 2021-02-10T11:47:24.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.127 7f1d067fc700 1 -- 172.21.2.101:0/1091925146 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cf4002fe0 con 0x7f1d080d1870 2021-02-10T11:47:24.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.127 7f1d06ffd700 1 -- 172.21.2.101:0/1091925146 >> v2:172.21.2.108:3300/0 conn(0x7f1d080d1c70 msgr2=0x7f1d080efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:24.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.127 7f1d06ffd700 1 --2- 172.21.2.101:0/1091925146 >> v2:172.21.2.108:3300/0 conn(0x7f1d080d1c70 0x7f1d080efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:24.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.127 7f1d06ffd700 1 -- 172.21.2.101:0/1091925146 >> v1:172.21.2.101:6789/0 conn(0x7f1d080d1870 legacy=0x7f1d081b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:24.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.127 7f1d06ffd700 1 -- 172.21.2.101:0/1091925146 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d080f20a0 con 0x7f1d080d0340 2021-02-10T11:47:24.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.127 7f1d06ffd700 1 --2- 172.21.2.101:0/1091925146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d080d0340 0x7f1d080ef640 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f1cf8006650 tx=0x7f1cf8009a80).ready entity=mon.2 client_cookie=a7dc03fc9ba615dc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:24.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.128 7f1d067fc700 1 -- 172.21.2.101:0/1091925146 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1cf800c0c0 con 0x7f1d080d0340 2021-02-10T11:47:24.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.128 7f1d067fc700 1 -- 172.21.2.101:0/1091925146 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf800c340 con 0x7f1d080d0340 2021-02-10T11:47:24.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.128 7f1d067fc700 1 -- 172.21.2.101:0/1091925146 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1cf800c860 con 0x7f1d080d0340 2021-02-10T11:47:24.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.128 7f1d0deab700 1 -- 172.21.2.101:0/1091925146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d080d0340 msgr2=0x7f1d080ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:24.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.128 7f1d0deab700 1 --2- 172.21.2.101:0/1091925146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d080d0340 0x7f1d080ef640 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f1cf8006650 tx=0x7f1cf8009a80).stop 2021-02-10T11:47:24.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.128 7f1d0deab700 1 -- 172.21.2.101:0/1091925146 shutdown_connections 2021-02-10T11:47:24.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.128 7f1d0deab700 1 --2- 172.21.2.101:0/1091925146 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d080d0340 0x7f1d080ef640 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:24.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.128 7f1d0deab700 1 --2- 172.21.2.101:0/1091925146 >> v2:172.21.2.108:3300/0 conn(0x7f1d080d1c70 0x7f1d080efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:24.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.128 7f1d0deab700 1 -- 172.21.2.101:0/1091925146 >> 172.21.2.101:0/1091925146 conn(0x7f1d080ce450 msgr2=0x7f1d080d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:24.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.128 7f1d0deab700 1 -- 172.21.2.101:0/1091925146 shutdown_connections 2021-02-10T11:47:24.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.128 7f1d0deab700 1 -- 172.21.2.101:0/1091925146 wait complete. 2021-02-10T11:47:24.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.143 7f1d0deab700 1 Processor -- start 2021-02-10T11:47:24.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.143 7f1d0deab700 1 -- start start 2021-02-10T11:47:24.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.143 7f1d0deab700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d08093f10 0x7f1d08094310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:24.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.143 7f1d0deab700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1d080ae9e0 0x7f1d080aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:24.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.143 7f1d0deab700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d08092d90 0x7f1d08093190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:24.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.143 7f1d0deab700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0819ebc0 con 0x7f1d08093f10 2021-02-10T11:47:24.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.143 7f1d0deab700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d081b8760 con 0x7f1d080ae9e0 2021-02-10T11:47:24.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.143 7f1d0deab700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d080937b0 con 0x7f1d08092d90 2021-02-10T11:47:24.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.144 7f1d077fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d08093f10 0x7f1d08094310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:24.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.144 7f1d077fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d08093f10 0x7f1d08094310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:33982/0 (socket says 172.21.2.101:33982) 2021-02-10T11:47:24.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.144 7f1d077fe700 1 -- 172.21.2.101:0/381638324 learned_addr learned my addr 172.21.2.101:0/381638324 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:24.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.144 7f1d077fe700 1 -- 172.21.2.101:0/381638324 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d08092d90 msgr2=0x7f1d08093190 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:47:24.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.144 7f1d077fe700 1 --2- 172.21.2.101:0/381638324 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d08092d90 0x7f1d08093190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:24.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.144 7f1d077fe700 1 -- 172.21.2.101:0/381638324 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1d080ae9e0 msgr2=0x7f1d080aede0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:47:24.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.144 7f1d077fe700 1 --2- 172.21.2.101:0/381638324 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1d080ae9e0 0x7f1d080aede0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:24.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.144 7f1d077fe700 1 -- 172.21.2.101:0/381638324 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d080939d0 con 0x7f1d08093f10 2021-02-10T11:47:24.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.144 7f1d077fe700 1 --2- 172.21.2.101:0/381638324 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d08093f10 0x7f1d08094310 secure :-1 s=READY pgs=3078 cs=0 l=1 rev1=1 rx=0x7f1cf0001c60 tx=0x7f1cf000a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:24.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.144 7f1cfffff700 1 -- 172.21.2.101:0/381638324 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1cf000acc0 con 0x7f1d08093f10 2021-02-10T11:47:24.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.144 7f1cfffff700 1 -- 172.21.2.101:0/381638324 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf0004350 con 0x7f1d08093f10 2021-02-10T11:47:24.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.144 7f1d0deab700 1 -- 172.21.2.101:0/381638324 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d080939d0 con 0x7f1d08093f10 2021-02-10T11:47:24.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.144 7f1cfffff700 1 -- 172.21.2.101:0/381638324 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1cf00047b0 con 0x7f1d08093f10 2021-02-10T11:47:24.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.144 7f1d0deab700 1 -- 172.21.2.101:0/381638324 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d08189cd0 con 0x7f1d08093f10 2021-02-10T11:47:24.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.145 7f1cfffff700 1 -- 172.21.2.101:0/381638324 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1cf0004530 con 0x7f1d08093f10 2021-02-10T11:47:24.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.145 7f1cfffff700 1 --2- 172.21.2.101:0/381638324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1cec0186c0 0x7f1cec01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:24.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.145 7f1cfffff700 1 -- 172.21.2.101:0/381638324 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f1cf00320b0 con 0x7f1d08093f10 2021-02-10T11:47:24.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.145 7f1d06ffd700 1 --2- 172.21.2.101:0/381638324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1cec0186c0 0x7f1cec01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:24.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.145 7f1d06ffd700 1 --2- 172.21.2.101:0/381638324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1cec0186c0 0x7f1cec01abc0 crc :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:24.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.176 7f1d0deab700 1 -- 172.21.2.101:0/381638324 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d080ebc60 con 0x7f1d08093f10 2021-02-10T11:47:24.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.179 7f1cfffff700 1 -- 172.21.2.101:0/381638324 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1cf0000940 con 0x7f1d08093f10 2021-02-10T11:47:24.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.313 7f1d0deab700 1 -- 172.21.2.101:0/381638324 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1d080f0b10 con 0x7f1d08093f10 2021-02-10T11:47:24.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.313 7f1cfffff700 1 -- 172.21.2.101:0/381638324 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v101) v1 ==== 80+0+4513 (secure 0 0 0) 0x7f1cf0004530 con 0x7f1d08093f10 2021-02-10T11:47:24.311 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:47:24.311 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:47:24.311 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:47:24.311 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:24.311 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:47:24.311 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:47:24.311 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:47:24.311 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:47:24.311 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:47:24.311 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:47:24.312 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:47:24.312 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:24.312 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:24.312 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:24.312 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:47:24.312 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:47:24.312 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:47:24.312 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:47:24.312 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:47:24.312 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:47:24.312 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:47:24.312 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:24.312 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:24.312 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:24.313 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:47:24.313 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:47:24.313 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:47:24.313 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:47:24.313 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:47:24.313 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:47:24.313 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:24.313 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:24.313 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:24.313 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:47:24.313 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:47:24.313 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:47:24.314 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:47:24.314 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:47:24.314 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:47:24.314 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:24.314 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:24.314 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:24.314 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:47:24.314 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:47:24.314 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:47:24.314 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:47:24.314 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:47:24.314 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:47:24.314 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:24.315 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:24.315 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:24.315 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:47:24.315 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:47:24.315 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:47:24.315 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:47:24.315 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:47:24.315 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:47:24.315 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:24.315 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:24.315 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:24.315 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:47:24.315 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:47:24.315 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:47:24.316 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:47:24.316 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:47:24.316 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:47:24.316 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:24.316 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:24.316 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:24.316 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:47:24.316 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:47:24.316 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:47:24.316 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:47:24.316 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:47:24.316 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:47:24.317 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:24.317 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:24.317 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:24.317 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:47:24.317 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:47:24.317 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:47:24.317 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:47:24.317 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:47:24.317 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:47:24.317 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:24.317 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:24.317 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:24.318 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:47:24.318 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:47:24.318 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:47:24.318 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:47:24.318 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:47:24.318 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:47:24.318 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:24.318 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:24.318 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:24.318 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:47:24.318 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:47:24.318 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:47:24.319 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:47:24.319 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:47:24.319 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:47:24.319 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:24.319 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:24.319 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:24.319 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:47:24.319 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:47:24.319 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:47:24.319 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:47:24.319 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:47:24.319 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:47:24.319 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:24.319 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:24.320 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:24.320 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:47:24.320 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:47:24.320 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:47:24.320 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:47:24.320 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:47:24.320 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:47:24.320 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:24.320 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:24.320 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:24.320 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:47:24.320 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:47:24.320 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:47:24.321 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:47:24.321 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:47:24.321 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:47:24.321 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:24.321 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:24.321 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:24.321 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:47:24.321 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:47:24.321 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:47:24.321 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:47:24.321 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:47:24.321 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:47:24.321 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:47:24.322 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:24.322 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:24.322 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:24.322 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:47:24.322 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:47:24.322 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:47:24.322 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:47:24.322 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:47:24.322 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:47:24.322 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:24.322 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:24.322 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:24.322 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:47:24.323 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:47:24.323 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:47:24.323 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:47:24.323 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:47:24.323 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:47:24.323 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:24.323 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:24.323 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:47:24.323 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:47:24.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.315 7f1d0deab700 1 -- 172.21.2.101:0/381638324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1cec0186c0 msgr2=0x7f1cec01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:24.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.315 7f1d0deab700 1 --2- 172.21.2.101:0/381638324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1cec0186c0 0x7f1cec01abc0 crc :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:24.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.315 7f1d0deab700 1 -- 172.21.2.101:0/381638324 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d08093f10 msgr2=0x7f1d08094310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:24.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.315 7f1d0deab700 1 --2- 172.21.2.101:0/381638324 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d08093f10 0x7f1d08094310 secure :-1 s=READY pgs=3078 cs=0 l=1 rev1=1 rx=0x7f1cf0001c60 tx=0x7f1cf000a8d0).stop 2021-02-10T11:47:24.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.315 7f1d0deab700 1 -- 172.21.2.101:0/381638324 shutdown_connections 2021-02-10T11:47:24.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.315 7f1d0deab700 1 --2- 172.21.2.101:0/381638324 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1cec0186c0 0x7f1cec01abc0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:24.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.315 7f1d0deab700 1 --2- 172.21.2.101:0/381638324 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1d08092d90 0x7f1d08093190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:24.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.315 7f1d0deab700 1 --2- 172.21.2.101:0/381638324 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1d080ae9e0 0x7f1d080aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:24.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.315 7f1d0deab700 1 --2- 172.21.2.101:0/381638324 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1d08093f10 0x7f1d08094310 unknown :-1 s=CLOSED pgs=3078 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:24.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.315 7f1d0deab700 1 -- 172.21.2.101:0/381638324 >> 172.21.2.101:0/381638324 conn(0x7f1d0819f8a0 msgr2=0x7f1d08094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:24.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.316 7f1d0deab700 1 -- 172.21.2.101:0/381638324 shutdown_connections 2021-02-10T11:47:24.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.316 7f1d0deab700 1 -- 172.21.2.101:0/381638324 wait complete. 2021-02-10T11:47:24.326 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:47:24.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.741 7f2055e20700 1 Processor -- start 2021-02-10T11:47:24.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.741 7f2055e20700 1 -- start start 2021-02-10T11:47:24.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.742 7f2055e20700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f20500d1870 0x7f20501b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:24.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.742 7f2055e20700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20500f0b10 con 0x7f20501b0a10 2021-02-10T11:47:24.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.742 7f2055e20700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20500d0310 con 0x7f20500f0650 2021-02-10T11:47:24.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.742 7f2055e20700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20500d0530 con 0x7f20500d1870 2021-02-10T11:47:24.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.742 7f204effd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f20501b0a10 0x7f20500cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34484/0 (socket says 172.21.2.101:34484) 2021-02-10T11:47:24.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.742 7f204effd700 1 -- 172.21.2.101:0/1567371495 learned_addr learned my addr 172.21.2.101:0/1567371495 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:24.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.742 7f204e7fc700 1 -- 172.21.2.101:0/1567371495 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3848558634 0 0) 0x7f20380009b0 con 0x7f20501b0a10 2021-02-10T11:47:24.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.742 7f204e7fc700 1 -- 172.21.2.101:0/1567371495 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f203c002fe0 con 0x7f20501b0a10 2021-02-10T11:47:24.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204e7fc700 1 -- 172.21.2.101:0/1567371495 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3039614313 0 0) 0x7f2038001010 con 0x7f20501b0a10 2021-02-10T11:47:24.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204e7fc700 1 -- 172.21.2.101:0/1567371495 >> v2:172.21.2.108:3300/0 conn(0x7f20500d1870 msgr2=0x7f20501b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:24.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204e7fc700 1 --2- 172.21.2.101:0/1567371495 >> v2:172.21.2.108:3300/0 conn(0x7f20500d1870 0x7f20501b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:24.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204e7fc700 1 -- 172.21.2.101:0/1567371495 >> v1:172.21.2.108:6789/0 conn(0x7f20500f0650 legacy=0x7f20500cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:24.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204e7fc700 1 -- 172.21.2.101:0/1567371495 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20500d1c70 con 0x7f20501b0a10 2021-02-10T11:47:24.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204e7fc700 1 -- 172.21.2.101:0/1567371495 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f2038001590 con 0x7f20501b0a10 2021-02-10T11:47:24.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204e7fc700 1 -- 172.21.2.101:0/1567371495 >> v1:172.21.2.101:6789/0 conn(0x7f20501b0a10 legacy=0x7f20500cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:24.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204e7fc700 1 --2- 172.21.2.101:0/1567371495 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f203c003ae0 0x7f203c005f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:24.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204e7fc700 1 --2- 172.21.2.101:0/1567371495 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f203c006790 0x7f203c008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:24.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204e7fc700 1 --2- 172.21.2.101:0/1567371495 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f203c009590 0x7f203c00bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:24.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204e7fc700 1 -- 172.21.2.101:0/1567371495 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c00c300 con 0x7f203c003ae0 2021-02-10T11:47:24.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204e7fc700 1 -- 172.21.2.101:0/1567371495 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c00c520 con 0x7f203c006790 2021-02-10T11:47:24.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204e7fc700 1 -- 172.21.2.101:0/1567371495 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203c00c740 con 0x7f203c009590 2021-02-10T11:47:24.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204f7fe700 1 --2- 172.21.2.101:0/1567371495 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f203c003ae0 0x7f203c005f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:24.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204effd700 1 --2- 172.21.2.101:0/1567371495 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f203c006790 0x7f203c008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:24.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204f7fe700 1 -- 172.21.2.101:0/1567371495 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f203c009590 msgr2=0x7f203c00bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:24.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204f7fe700 1 --2- 172.21.2.101:0/1567371495 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f203c009590 0x7f203c00bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:24.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204f7fe700 1 -- 172.21.2.101:0/1567371495 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f203c006790 msgr2=0x7f203c008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:24.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204f7fe700 1 --2- 172.21.2.101:0/1567371495 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f203c006790 0x7f203c008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:24.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204f7fe700 1 -- 172.21.2.101:0/1567371495 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c00c9e0 con 0x7f203c003ae0 2021-02-10T11:47:24.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204ffff700 1 -- 172.21.2.101:0/1567371495 reap_dead start 2021-02-10T11:47:24.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204f7fe700 1 --2- 172.21.2.101:0/1567371495 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f203c003ae0 0x7f203c005f10 secure :-1 s=READY pgs=3080 cs=0 l=1 rev1=1 rx=0x7f2040006bf0 tx=0x7f204004fcd0).ready entity=mon.0 client_cookie=2e11b6dc61b263cb server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:24.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204e7fc700 1 -- 172.21.2.101:0/1567371495 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f20400009b0 con 0x7f203c003ae0 2021-02-10T11:47:24.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204e7fc700 1 -- 172.21.2.101:0/1567371495 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2040002030 con 0x7f203c003ae0 2021-02-10T11:47:24.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f204e7fc700 1 -- 172.21.2.101:0/1567371495 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2040002510 con 0x7f203c003ae0 2021-02-10T11:47:24.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f2055e20700 1 -- 172.21.2.101:0/1567371495 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f203c003ae0 msgr2=0x7f203c005f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:24.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.743 7f2055e20700 1 --2- 172.21.2.101:0/1567371495 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f203c003ae0 0x7f203c005f10 secure :-1 s=READY pgs=3080 cs=0 l=1 rev1=1 rx=0x7f2040006bf0 tx=0x7f204004fcd0).stop 2021-02-10T11:47:24.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.744 7f2055e20700 1 -- 172.21.2.101:0/1567371495 shutdown_connections 2021-02-10T11:47:24.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.744 7f2055e20700 1 --2- 172.21.2.101:0/1567371495 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f203c003ae0 0x7f203c005f10 unknown :-1 s=CLOSED pgs=3080 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:24.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.744 7f2055e20700 1 -- 172.21.2.101:0/1567371495 >> 172.21.2.101:0/1567371495 conn(0x7f20500ce450 msgr2=0x7f20500d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:24.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.744 7f2055e20700 1 -- 172.21.2.101:0/1567371495 shutdown_connections 2021-02-10T11:47:24.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.744 7f2055e20700 1 -- 172.21.2.101:0/1567371495 wait complete. 2021-02-10T11:47:24.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.759 7f2055e20700 1 Processor -- start 2021-02-10T11:47:24.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.759 7f2055e20700 1 -- start start 2021-02-10T11:47:24.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.759 7f2055e20700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20501942b0 0x7f20500b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:24.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.759 7f2055e20700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2050195070 0x7f2050195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:24.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.759 7f2055e20700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20500bb8e0 0x7f20500bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:24.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.759 7f2055e20700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20501946b0 con 0x7f20500bb8e0 2021-02-10T11:47:24.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.759 7f2055e20700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20500b3b70 con 0x7f2050195070 2021-02-10T11:47:24.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.759 7f2055e20700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20500b3d90 con 0x7f20501942b0 2021-02-10T11:47:24.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.759 7f204ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20500bb8e0 0x7f20500bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:24.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.759 7f204ffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20500bb8e0 0x7f20500bbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34000/0 (socket says 172.21.2.101:34000) 2021-02-10T11:47:24.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.759 7f204ffff700 1 -- 172.21.2.101:0/2602685642 learned_addr learned my addr 172.21.2.101:0/2602685642 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:24.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.760 7f204f7fe700 1 --2- 172.21.2.101:0/2602685642 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20501942b0 0x7f20500b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:24.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.760 7f204effd700 1 --2- 172.21.2.101:0/2602685642 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2050195070 0x7f2050195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:24.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.760 7f204ffff700 1 -- 172.21.2.101:0/2602685642 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20501942b0 msgr2=0x7f20500b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:24.761 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.760 7f204ffff700 1 --2- 172.21.2.101:0/2602685642 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20501942b0 0x7f20500b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:24.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.760 7f204ffff700 1 -- 172.21.2.101:0/2602685642 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2050195070 msgr2=0x7f2050195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:24.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.760 7f204ffff700 1 --2- 172.21.2.101:0/2602685642 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2050195070 0x7f2050195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:24.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.760 7f204ffff700 1 -- 172.21.2.101:0/2602685642 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20501a49e0 con 0x7f20500bb8e0 2021-02-10T11:47:24.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.760 7f204ffff700 1 --2- 172.21.2.101:0/2602685642 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20500bb8e0 0x7f20500bbce0 secure :-1 s=READY pgs=3081 cs=0 l=1 rev1=1 rx=0x7f204800a8f0 tx=0x7f204800ac90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:24.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.760 7f2047fff700 1 -- 172.21.2.101:0/2602685642 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2048007020 con 0x7f20500bb8e0 2021-02-10T11:47:24.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.760 7f2047fff700 1 -- 172.21.2.101:0/2602685642 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2048000fc0 con 0x7f20500bb8e0 2021-02-10T11:47:24.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.760 7f2055e20700 1 -- 172.21.2.101:0/2602685642 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20501a49e0 con 0x7f20500bb8e0 2021-02-10T11:47:24.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.760 7f2047fff700 1 -- 172.21.2.101:0/2602685642 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f20480045a0 con 0x7f20500bb8e0 2021-02-10T11:47:24.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.760 7f2055e20700 1 -- 172.21.2.101:0/2602685642 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20501a49e0 con 0x7f20500bb8e0 2021-02-10T11:47:24.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.761 7f2047fff700 1 -- 172.21.2.101:0/2602685642 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f204800add0 con 0x7f20500bb8e0 2021-02-10T11:47:24.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.761 7f2047fff700 1 --2- 172.21.2.101:0/2602685642 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f20340186d0 0x7f203401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:24.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.761 7f2047fff700 1 -- 172.21.2.101:0/2602685642 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f2048032640 con 0x7f20500bb8e0 2021-02-10T11:47:24.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.761 7f204f7fe700 1 --2- 172.21.2.101:0/2602685642 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f20340186d0 0x7f203401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:24.763 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.761 7f204f7fe700 1 --2- 172.21.2.101:0/2602685642 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f20340186d0 0x7f203401abd0 crc :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:24.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.792 7f2055e20700 1 -- 172.21.2.101:0/2602685642 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20500bc5f0 con 0x7f20500bb8e0 2021-02-10T11:47:24.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.794 7f2047fff700 1 -- 172.21.2.101:0/2602685642 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2048000fa0 con 0x7f20500bb8e0 2021-02-10T11:47:24.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.918 7f2055e20700 1 -- 172.21.2.101:0/2602685642 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20500bc5f0 con 0x7f20500bb8e0 2021-02-10T11:47:24.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.918 7f2047fff700 1 -- 172.21.2.101:0/2602685642 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v179) v1 ==== 74+0+11154 (secure 0 0 0) 0x7f2048000fa0 con 0x7f20500bb8e0 2021-02-10T11:47:24.915 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:47:24.916 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":179,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:47:23.668298","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":37,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:47:24.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.920 7f2055e20700 1 -- 172.21.2.101:0/2602685642 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f20340186d0 msgr2=0x7f203401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:24.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.920 7f2055e20700 1 --2- 172.21.2.101:0/2602685642 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f20340186d0 0x7f203401abd0 crc :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:24.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.920 7f2055e20700 1 -- 172.21.2.101:0/2602685642 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20500bb8e0 msgr2=0x7f20500bbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:24.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.920 7f2055e20700 1 --2- 172.21.2.101:0/2602685642 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20500bb8e0 0x7f20500bbce0 secure :-1 s=READY pgs=3081 cs=0 l=1 rev1=1 rx=0x7f204800a8f0 tx=0x7f204800ac90).stop 2021-02-10T11:47:24.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.920 7f2055e20700 1 -- 172.21.2.101:0/2602685642 shutdown_connections 2021-02-10T11:47:24.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.920 7f2055e20700 1 --2- 172.21.2.101:0/2602685642 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f20500bb8e0 0x7f20500bbce0 unknown :-1 s=CLOSED pgs=3081 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:24.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.920 7f2055e20700 1 --2- 172.21.2.101:0/2602685642 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2050195070 0x7f2050195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:24.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.920 7f2055e20700 1 --2- 172.21.2.101:0/2602685642 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f20340186d0 0x7f203401abd0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:24.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.920 7f2055e20700 1 --2- 172.21.2.101:0/2602685642 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f20501942b0 0x7f20500b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:24.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.920 7f2055e20700 1 -- 172.21.2.101:0/2602685642 >> 172.21.2.101:0/2602685642 conn(0x7f20500b3040 msgr2=0x7f20500c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:24.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.920 7f2055e20700 1 -- 172.21.2.101:0/2602685642 shutdown_connections 2021-02-10T11:47:24.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:24.920 7f2055e20700 1 -- 172.21.2.101:0/2602685642 wait complete. 2021-02-10T11:47:24.926 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-10T11:47:24.926 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:47:25.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.346 7fdf3a79d700 1 Processor -- start 2021-02-10T11:47:25.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.346 7fdf3a79d700 1 -- start start 2021-02-10T11:47:25.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.346 7fdf3a79d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdf340d1870 0x7fdf341b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:25.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.346 7fdf3a79d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fdf340cff70 0x7fdf34109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:25.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.346 7fdf3a79d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf340fe1c0 con 0x7fdf341b0ae0 2021-02-10T11:47:25.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.346 7fdf3a79d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf34109ad0 con 0x7fdf340d1870 2021-02-10T11:47:25.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.346 7fdf3a79d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf34109750 con 0x7fdf340cff70 2021-02-10T11:47:25.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.347 7fdf33fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdf340d1870 0x7fdf341b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:25.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.347 7fdf337fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fdf341b0ae0 0x7fdf34108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34502/0 (socket says 172.21.2.101:34502) 2021-02-10T11:47:25.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.347 7fdf337fe700 1 -- 172.21.2.101:0/1607863859 learned_addr learned my addr 172.21.2.101:0/1607863859 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:25.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.347 7fdf33fff700 1 -- 172.21.2.101:0/1607863859 >> v2:172.21.2.108:3300/0 conn(0x7fdf340cff70 msgr2=0x7fdf34109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:25.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.347 7fdf33fff700 1 --2- 172.21.2.101:0/1607863859 >> v2:172.21.2.108:3300/0 conn(0x7fdf340cff70 0x7fdf34109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:25.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.347 7fdf33fff700 1 -- 172.21.2.101:0/1607863859 >> v1:172.21.2.101:6789/0 conn(0x7fdf341b0ae0 legacy=0x7fdf34108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:25.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.347 7fdf33fff700 1 -- 172.21.2.101:0/1607863859 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf34088070 con 0x7fdf340d1870 2021-02-10T11:47:25.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.347 7fdf33fff700 1 --2- 172.21.2.101:0/1607863859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf340d1870 0x7fdf341b04c0 secure :-1 s=READY pgs=3083 cs=0 l=1 rev1=1 rx=0x7fdf240069d0 tx=0x7fdf24009e40).ready entity=mon.0 client_cookie=be7d3d08a6721030 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:25.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.347 7fdf32ffd700 1 -- 172.21.2.101:0/1607863859 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdf2400b510 con 0x7fdf340d1870 2021-02-10T11:47:25.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.347 7fdf32ffd700 1 -- 172.21.2.101:0/1607863859 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2400b810 con 0x7fdf340d1870 2021-02-10T11:47:25.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.347 7fdf32ffd700 1 -- 172.21.2.101:0/1607863859 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdf2400bbe0 con 0x7fdf340d1870 2021-02-10T11:47:25.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.347 7fdf3a79d700 1 -- 172.21.2.101:0/1607863859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf340d1870 msgr2=0x7fdf341b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:25.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.347 7fdf3a79d700 1 --2- 172.21.2.101:0/1607863859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf340d1870 0x7fdf341b04c0 secure :-1 s=READY pgs=3083 cs=0 l=1 rev1=1 rx=0x7fdf240069d0 tx=0x7fdf24009e40).stop 2021-02-10T11:47:25.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.348 7fdf3a79d700 1 -- 172.21.2.101:0/1607863859 shutdown_connections 2021-02-10T11:47:25.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.348 7fdf3a79d700 1 --2- 172.21.2.101:0/1607863859 >> v2:172.21.2.108:3300/0 conn(0x7fdf340cff70 0x7fdf34109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:25.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.348 7fdf3a79d700 1 --2- 172.21.2.101:0/1607863859 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf340d1870 0x7fdf341b04c0 unknown :-1 s=CLOSED pgs=3083 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:25.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.348 7fdf3a79d700 1 -- 172.21.2.101:0/1607863859 >> 172.21.2.101:0/1607863859 conn(0x7fdf340ce420 msgr2=0x7fdf340d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:25.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.348 7fdf3a79d700 1 -- 172.21.2.101:0/1607863859 shutdown_connections 2021-02-10T11:47:25.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.348 7fdf3a79d700 1 -- 172.21.2.101:0/1607863859 wait complete. 2021-02-10T11:47:25.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.363 7fdf3a79d700 1 Processor -- start 2021-02-10T11:47:25.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.363 7fdf3a79d700 1 -- start start 2021-02-10T11:47:25.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.363 7fdf3a79d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf340c3820 0x7fdf34084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:25.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.363 7fdf3a79d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf340a0290 0x7fdf340852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:25.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.363 7fdf3a79d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf34086270 0x7fdf34086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:25.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.363 7fdf3a79d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf340c0700 con 0x7fdf34086270 2021-02-10T11:47:25.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.363 7fdf3a79d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf340858f0 con 0x7fdf340c3820 2021-02-10T11:47:25.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.363 7fdf3a79d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf34085b10 con 0x7fdf340a0290 2021-02-10T11:47:25.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.363 7fdf38dba700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf34086270 0x7fdf34086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:25.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.363 7fdf38dba700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf34086270 0x7fdf34086670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34014/0 (socket says 172.21.2.101:34014) 2021-02-10T11:47:25.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.363 7fdf38dba700 1 -- 172.21.2.101:0/1485656127 learned_addr learned my addr 172.21.2.101:0/1485656127 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:25.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.364 7fdf38dba700 1 -- 172.21.2.101:0/1485656127 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf340a0290 msgr2=0x7fdf340852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:25.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.364 7fdf38dba700 1 --2- 172.21.2.101:0/1485656127 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf340a0290 0x7fdf340852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:25.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.364 7fdf38dba700 1 -- 172.21.2.101:0/1485656127 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf340c3820 msgr2=0x7fdf34084cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:25.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.364 7fdf38dba700 1 --2- 172.21.2.101:0/1485656127 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf340c3820 0x7fdf34084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:25.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.364 7fdf38dba700 1 -- 172.21.2.101:0/1485656127 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf34086c90 con 0x7fdf34086270 2021-02-10T11:47:25.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.364 7fdf38dba700 1 --2- 172.21.2.101:0/1485656127 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf34086270 0x7fdf34086670 secure :-1 s=READY pgs=3084 cs=0 l=1 rev1=1 rx=0x7fdf2c006f10 tx=0x7fdf2c00b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:25.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.364 7fdf30ff9700 1 -- 172.21.2.101:0/1485656127 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdf2c006240 con 0x7fdf34086270 2021-02-10T11:47:25.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.364 7fdf30ff9700 1 -- 172.21.2.101:0/1485656127 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2c000c10 con 0x7fdf34086270 2021-02-10T11:47:25.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.364 7fdf3a79d700 1 -- 172.21.2.101:0/1485656127 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf34086c90 con 0x7fdf34086270 2021-02-10T11:47:25.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.364 7fdf3a79d700 1 -- 172.21.2.101:0/1485656127 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf341a2f00 con 0x7fdf34086270 2021-02-10T11:47:25.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.364 7fdf30ff9700 1 -- 172.21.2.101:0/1485656127 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdf2c005770 con 0x7fdf34086270 2021-02-10T11:47:25.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.365 7fdf30ff9700 1 -- 172.21.2.101:0/1485656127 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdf2c0054f0 con 0x7fdf34086270 2021-02-10T11:47:25.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.365 7fdf30ff9700 1 --2- 172.21.2.101:0/1485656127 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf140186c0 0x7fdf1401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:25.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.365 7fdf30ff9700 1 -- 172.21.2.101:0/1485656127 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fdf2c0322a0 con 0x7fdf34086270 2021-02-10T11:47:25.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.365 7fdf33fff700 1 --2- 172.21.2.101:0/1485656127 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf140186c0 0x7fdf1401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:25.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.365 7fdf33fff700 1 --2- 172.21.2.101:0/1485656127 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf140186c0 0x7fdf1401abc0 crc :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:25.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.396 7fdf3a79d700 1 -- 172.21.2.101:0/1485656127 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf34082a70 con 0x7fdf34086270 2021-02-10T11:47:25.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.398 7fdf30ff9700 1 -- 172.21.2.101:0/1485656127 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdf2c000cd0 con 0x7fdf34086270 2021-02-10T11:47:25.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.527 7fdf3a79d700 1 -- 172.21.2.101:0/1485656127 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fdf34082a70 con 0x7fdf34086270 2021-02-10T11:47:25.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.528 7fdf30ff9700 1 -- 172.21.2.101:0/1485656127 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v179) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdf2c0054f0 con 0x7fdf34086270 2021-02-10T11:47:25.525 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:47:25.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.530 7fdf3a79d700 1 -- 172.21.2.101:0/1485656127 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf140186c0 msgr2=0x7fdf1401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:25.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.530 7fdf3a79d700 1 --2- 172.21.2.101:0/1485656127 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf140186c0 0x7fdf1401abc0 crc :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:25.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.530 7fdf3a79d700 1 -- 172.21.2.101:0/1485656127 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf34086270 msgr2=0x7fdf34086670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:25.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.530 7fdf3a79d700 1 --2- 172.21.2.101:0/1485656127 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf34086270 0x7fdf34086670 secure :-1 s=READY pgs=3084 cs=0 l=1 rev1=1 rx=0x7fdf2c006f10 tx=0x7fdf2c00b990).stop 2021-02-10T11:47:25.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.530 7fdf3a79d700 1 -- 172.21.2.101:0/1485656127 shutdown_connections 2021-02-10T11:47:25.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.530 7fdf3a79d700 1 --2- 172.21.2.101:0/1485656127 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdf140186c0 0x7fdf1401abc0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:25.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.530 7fdf3a79d700 1 --2- 172.21.2.101:0/1485656127 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdf340a0290 0x7fdf340852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:25.527 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.530 7fdf3a79d700 1 --2- 172.21.2.101:0/1485656127 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdf34086270 0x7fdf34086670 unknown :-1 s=CLOSED pgs=3084 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:25.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.530 7fdf3a79d700 1 --2- 172.21.2.101:0/1485656127 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdf340c3820 0x7fdf34084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:25.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.530 7fdf3a79d700 1 -- 172.21.2.101:0/1485656127 >> 172.21.2.101:0/1485656127 conn(0x7fdf340872c0 msgr2=0x7fdf34095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:25.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.530 7fdf3a79d700 1 -- 172.21.2.101:0/1485656127 shutdown_connections 2021-02-10T11:47:25.528 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.530 7fdf3a79d700 1 -- 172.21.2.101:0/1485656127 wait complete. 2021-02-10T11:47:25.535 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --format=json-pretty osd lspools 2021-02-10T11:47:25.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.950 7f6a7740e700 1 Processor -- start 2021-02-10T11:47:25.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.950 7f6a7740e700 1 -- start start 2021-02-10T11:47:25.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.951 7f6a7740e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6a700d1870 0x7f6a701b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:25.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.951 7f6a7740e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6a701b0a10 0x7f6a700cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:25.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.951 7f6a7740e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a700f0b10 con 0x7f6a700d1c70 2021-02-10T11:47:25.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.951 7f6a7740e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a700e5ed0 con 0x7f6a700d1870 2021-02-10T11:47:25.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.951 7f6a7740e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a700f0650 con 0x7f6a701b0a10 2021-02-10T11:47:25.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.951 7f6a7522a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6a700d1870 0x7f6a701b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:25.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.951 7f6a7522a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f6a700d1870 0x7f6a701b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34018/0 (socket says 172.21.2.101:34018) 2021-02-10T11:47:25.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.951 7f6a7522a700 1 -- 172.21.2.101:0/4099299275 learned_addr learned my addr 172.21.2.101:0/4099299275 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:25.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.951 7f6a7522a700 1 -- 172.21.2.101:0/4099299275 >> v2:172.21.2.108:3300/0 conn(0x7f6a701b0a10 msgr2=0x7f6a700cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:25.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.951 7f6a7522a700 1 --2- 172.21.2.101:0/4099299275 >> v2:172.21.2.108:3300/0 conn(0x7f6a701b0a10 0x7f6a700cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:25.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.951 7f6a7522a700 1 -- 172.21.2.101:0/4099299275 >> v1:172.21.2.101:6789/0 conn(0x7f6a700d1c70 legacy=0x7f6a700cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:25.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.951 7f6a7522a700 1 -- 172.21.2.101:0/4099299275 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a700f0840 con 0x7f6a700d1870 2021-02-10T11:47:25.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.952 7f6a7522a700 1 --2- 172.21.2.101:0/4099299275 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a700d1870 0x7f6a701b03f0 secure :-1 s=READY pgs=3086 cs=0 l=1 rev1=1 rx=0x7f6a6c006700 tx=0x7f6a6c009b80).ready entity=mon.0 client_cookie=51b9abf422aaeaf5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:25.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.952 7f6a63fff700 1 -- 172.21.2.101:0/4099299275 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a6c000c90 con 0x7f6a700d1870 2021-02-10T11:47:25.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.952 7f6a63fff700 1 -- 172.21.2.101:0/4099299275 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6c00c0c0 con 0x7f6a700d1870 2021-02-10T11:47:25.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.952 7f6a63fff700 1 -- 172.21.2.101:0/4099299275 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a6c00c5e0 con 0x7f6a700d1870 2021-02-10T11:47:25.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.952 7f6a7740e700 1 -- 172.21.2.101:0/4099299275 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a700d1870 msgr2=0x7f6a701b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:25.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.952 7f6a7740e700 1 --2- 172.21.2.101:0/4099299275 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a700d1870 0x7f6a701b03f0 secure :-1 s=READY pgs=3086 cs=0 l=1 rev1=1 rx=0x7f6a6c006700 tx=0x7f6a6c009b80).stop 2021-02-10T11:47:25.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.952 7f6a7740e700 1 -- 172.21.2.101:0/4099299275 shutdown_connections 2021-02-10T11:47:25.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.952 7f6a7740e700 1 --2- 172.21.2.101:0/4099299275 >> v2:172.21.2.108:3300/0 conn(0x7f6a701b0a10 0x7f6a700cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:25.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.952 7f6a7740e700 1 --2- 172.21.2.101:0/4099299275 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a700d1870 0x7f6a701b03f0 unknown :-1 s=CLOSED pgs=3086 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:25.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.952 7f6a7740e700 1 -- 172.21.2.101:0/4099299275 >> 172.21.2.101:0/4099299275 conn(0x7f6a700ce450 msgr2=0x7f6a700d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:25.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.953 7f6a7740e700 1 -- 172.21.2.101:0/4099299275 shutdown_connections 2021-02-10T11:47:25.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.953 7f6a7740e700 1 -- 172.21.2.101:0/4099299275 wait complete. 2021-02-10T11:47:25.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.967 7f6a7740e700 1 Processor -- start 2021-02-10T11:47:25.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.967 7f6a7740e700 1 -- start start 2021-02-10T11:47:25.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.968 7f6a7740e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a700b4f30 0x7f6a7018b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:25.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.968 7f6a7740e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a70093f40 0x7f6a70094340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:25.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.968 7f6a7740e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a70094960 0x7f6a70187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:25.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.968 7f6a7740e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a70094e90 con 0x7f6a70093f40 2021-02-10T11:47:25.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.968 7f6a7740e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7018bff0 con 0x7f6a70094960 2021-02-10T11:47:25.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.968 7f6a7740e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7008f780 con 0x7f6a700b4f30 2021-02-10T11:47:25.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.968 7f6a74a29700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a70093f40 0x7f6a70094340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:25.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.968 7f6a74a29700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a70093f40 0x7f6a70094340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34022/0 (socket says 172.21.2.101:34022) 2021-02-10T11:47:25.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.968 7f6a74a29700 1 -- 172.21.2.101:0/319078651 learned_addr learned my addr 172.21.2.101:0/319078651 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:25.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.968 7f6a74a29700 1 -- 172.21.2.101:0/319078651 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a700b4f30 msgr2=0x7f6a7018b9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:47:25.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.968 7f6a74a29700 1 --2- 172.21.2.101:0/319078651 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a700b4f30 0x7f6a7018b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:25.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.968 7f6a74a29700 1 -- 172.21.2.101:0/319078651 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a70094960 msgr2=0x7f6a70187e50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:47:25.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.968 7f6a74a29700 1 --2- 172.21.2.101:0/319078651 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a70094960 0x7f6a70187e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:25.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.968 7f6a74a29700 1 -- 172.21.2.101:0/319078651 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a70188470 con 0x7f6a70093f40 2021-02-10T11:47:25.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.968 7f6a74a29700 1 --2- 172.21.2.101:0/319078651 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a70093f40 0x7f6a70094340 secure :-1 s=READY pgs=3087 cs=0 l=1 rev1=1 rx=0x7f6a64006f00 tx=0x7f6a6400b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:25.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.968 7f6a61ffb700 1 -- 172.21.2.101:0/319078651 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a6400bd30 con 0x7f6a70093f40 2021-02-10T11:47:25.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.968 7f6a61ffb700 1 -- 172.21.2.101:0/319078651 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a64000c90 con 0x7f6a70093f40 2021-02-10T11:47:25.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.968 7f6a7740e700 1 -- 172.21.2.101:0/319078651 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a70188470 con 0x7f6a70093f40 2021-02-10T11:47:25.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.968 7f6a7740e700 1 -- 172.21.2.101:0/319078651 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a70188a00 con 0x7f6a70093f40 2021-02-10T11:47:25.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.968 7f6a61ffb700 1 -- 172.21.2.101:0/319078651 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a640022a0 con 0x7f6a70093f40 2021-02-10T11:47:25.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.969 7f6a61ffb700 1 -- 172.21.2.101:0/319078651 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6a64005310 con 0x7f6a70093f40 2021-02-10T11:47:25.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.969 7f6a61ffb700 1 --2- 172.21.2.101:0/319078651 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6a500186d0 0x7f6a5001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:25.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.969 7f6a61ffb700 1 -- 172.21.2.101:0/319078651 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f6a64032480 con 0x7f6a70093f40 2021-02-10T11:47:25.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.969 7f6a7522a700 1 --2- 172.21.2.101:0/319078651 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6a500186d0 0x7f6a5001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:25.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:25.970 7f6a7522a700 1 --2- 172.21.2.101:0/319078651 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6a500186d0 0x7f6a5001abd0 crc :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:25.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.001 7f6a7740e700 1 -- 172.21.2.101:0/319078651 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7018bff0 con 0x7f6a70093f40 2021-02-10T11:47:26.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.003 7f6a61ffb700 1 -- 172.21.2.101:0/319078651 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6a64006b90 con 0x7f6a70093f40 2021-02-10T11:47:26.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.128 7f6a7740e700 1 -- 172.21.2.101:0/319078651 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6a7018bff0 con 0x7f6a70093f40 2021-02-10T11:47:26.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.129 7f6a61ffb700 1 -- 172.21.2.101:0/319078651 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v179) v1 ==== 84+0+64 (secure 0 0 0) 0x7f6a6400bd30 con 0x7f6a70093f40 2021-02-10T11:47:26.126 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:47:26.126 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:47:26.127 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:26.127 INFO:teuthology.orchestra.run.gibba001.stdout: "poolnum": 1, 2021-02-10T11:47:26.127 INFO:teuthology.orchestra.run.gibba001.stdout: "poolname": "rbd" 2021-02-10T11:47:26.127 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:26.127 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:47:26.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.131 7f6a7740e700 1 -- 172.21.2.101:0/319078651 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6a500186d0 msgr2=0x7f6a5001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:26.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.131 7f6a7740e700 1 --2- 172.21.2.101:0/319078651 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6a500186d0 0x7f6a5001abd0 crc :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:26.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.131 7f6a7740e700 1 -- 172.21.2.101:0/319078651 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a70093f40 msgr2=0x7f6a70094340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:26.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.131 7f6a7740e700 1 --2- 172.21.2.101:0/319078651 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a70093f40 0x7f6a70094340 secure :-1 s=READY pgs=3087 cs=0 l=1 rev1=1 rx=0x7f6a64006f00 tx=0x7f6a6400b940).stop 2021-02-10T11:47:26.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.131 7f6a7740e700 1 -- 172.21.2.101:0/319078651 shutdown_connections 2021-02-10T11:47:26.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.131 7f6a7740e700 1 --2- 172.21.2.101:0/319078651 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a70094960 0x7f6a70187e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:26.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.131 7f6a7740e700 1 --2- 172.21.2.101:0/319078651 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a70093f40 0x7f6a70094340 unknown :-1 s=CLOSED pgs=3087 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:26.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.131 7f6a7740e700 1 --2- 172.21.2.101:0/319078651 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6a500186d0 0x7f6a5001abd0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:26.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.131 7f6a7740e700 1 --2- 172.21.2.101:0/319078651 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a700b4f30 0x7f6a7018b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:26.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.131 7f6a7740e700 1 -- 172.21.2.101:0/319078651 >> 172.21.2.101:0/319078651 conn(0x7f6a7019f8f0 msgr2=0x7f6a701a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:26.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.131 7f6a7740e700 1 -- 172.21.2.101:0/319078651 shutdown_connections 2021-02-10T11:47:26.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.131 7f6a7740e700 1 -- 172.21.2.101:0/319078651 wait complete. 2021-02-10T11:47:26.136 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-02-10T11:47:26.136 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-02-10T11:47:26.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.555 7fd7e0942700 1 Processor -- start 2021-02-10T11:47:26.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.555 7fd7e0942700 1 -- start start 2021-02-10T11:47:26.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.555 7fd7e0942700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd7dc0d1c70 0x7fd7dc0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:26.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.555 7fd7e0942700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7dc0cf580 0x7fd7dc0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:26.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.555 7fd7e0942700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7dc0fe1c0 con 0x7fd7dc0d1870 2021-02-10T11:47:26.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.555 7fd7e0942700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7dc1b0cb0 con 0x7fd7dc0d1c70 2021-02-10T11:47:26.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.555 7fd7e0942700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc109a60 con 0x7fd7dc0cf580 2021-02-10T11:47:26.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.556 7fd7d9e1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd7dc0d1c70 0x7fd7dc0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:26.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.556 7fd7d9e1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd7dc0d1c70 0x7fd7dc0cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34030/0 (socket says 172.21.2.101:34030) 2021-02-10T11:47:26.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.556 7fd7d9e1b700 1 -- 172.21.2.101:0/3312529042 learned_addr learned my addr 172.21.2.101:0/3312529042 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:26.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.556 7fd7d9e1b700 1 -- 172.21.2.101:0/3312529042 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7dc0cf580 msgr2=0x7fd7dc0cf980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:26.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.556 7fd7d9e1b700 1 --2- 172.21.2.101:0/3312529042 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7dc0cf580 0x7fd7dc0cf980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:26.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.556 7fd7d9e1b700 1 -- 172.21.2.101:0/3312529042 >> v1:172.21.2.101:6789/0 conn(0x7fd7dc0d1870 legacy=0x7fd7dc1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:26.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.556 7fd7d9e1b700 1 -- 172.21.2.101:0/3312529042 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7dc088070 con 0x7fd7dc0d1c70 2021-02-10T11:47:26.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.556 7fd7d9e1b700 1 --2- 172.21.2.101:0/3312529042 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7dc0d1c70 0x7fd7dc0cef60 secure :-1 s=READY pgs=3089 cs=0 l=1 rev1=1 rx=0x7fd7c4006700 tx=0x7fd7c4009b80).ready entity=mon.0 client_cookie=f7e02be75d47f7d0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:26.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.556 7fd7d961a700 1 -- 172.21.2.101:0/3312529042 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd7c400c0c0 con 0x7fd7dc0d1c70 2021-02-10T11:47:26.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.556 7fd7d961a700 1 -- 172.21.2.101:0/3312529042 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c400c340 con 0x7fd7dc0d1c70 2021-02-10T11:47:26.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.556 7fd7d961a700 1 -- 172.21.2.101:0/3312529042 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd7c400c860 con 0x7fd7dc0d1c70 2021-02-10T11:47:26.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.556 7fd7e0942700 1 -- 172.21.2.101:0/3312529042 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7dc0d1c70 msgr2=0x7fd7dc0cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:26.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.556 7fd7e0942700 1 --2- 172.21.2.101:0/3312529042 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7dc0d1c70 0x7fd7dc0cef60 secure :-1 s=READY pgs=3089 cs=0 l=1 rev1=1 rx=0x7fd7c4006700 tx=0x7fd7c4009b80).stop 2021-02-10T11:47:26.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.557 7fd7e0942700 1 -- 172.21.2.101:0/3312529042 shutdown_connections 2021-02-10T11:47:26.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.557 7fd7e0942700 1 --2- 172.21.2.101:0/3312529042 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7dc0cf580 0x7fd7dc0cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:26.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.557 7fd7e0942700 1 --2- 172.21.2.101:0/3312529042 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7dc0d1c70 0x7fd7dc0cef60 unknown :-1 s=CLOSED pgs=3089 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:26.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.557 7fd7e0942700 1 -- 172.21.2.101:0/3312529042 >> 172.21.2.101:0/3312529042 conn(0x7fd7dc0ce420 msgr2=0x7fd7dc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:26.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.557 7fd7e0942700 1 -- 172.21.2.101:0/3312529042 shutdown_connections 2021-02-10T11:47:26.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.557 7fd7e0942700 1 -- 172.21.2.101:0/3312529042 wait complete. 2021-02-10T11:47:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.572 7fd7e0942700 1 Processor -- start 2021-02-10T11:47:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.572 7fd7e0942700 1 -- start start 2021-02-10T11:47:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.572 7fd7e0942700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7dc0c3820 0x7fd7dc0a7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.572 7fd7e0942700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7dc1a44c0 0x7fd7dc1a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.572 7fd7e0942700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7dc0863d0 0x7fd7dc0867d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.572 7fd7e0942700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc085ed0 con 0x7fd7dc0863d0 2021-02-10T11:47:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.572 7fd7e0942700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc0c0740 con 0x7fd7dc1a44c0 2021-02-10T11:47:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.572 7fd7e0942700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc086df0 con 0x7fd7dc0c3820 2021-02-10T11:47:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.572 7fd7dae1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7dc0863d0 0x7fd7dc0867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:26.572 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.572 7fd7dae1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7dc0863d0 0x7fd7dc0867d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34038/0 (socket says 172.21.2.101:34038) 2021-02-10T11:47:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.572 7fd7dae1d700 1 -- 172.21.2.101:0/3804433722 learned_addr learned my addr 172.21.2.101:0/3804433722 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.573 7fd7dae1d700 1 -- 172.21.2.101:0/3804433722 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7dc0c3820 msgr2=0x7fd7dc0a7950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:47:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.573 7fd7dae1d700 1 --2- 172.21.2.101:0/3804433722 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7dc0c3820 0x7fd7dc0a7950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.573 7fd7dae1d700 1 -- 172.21.2.101:0/3804433722 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7dc1a44c0 msgr2=0x7fd7dc1a48c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:47:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.573 7fd7dae1d700 1 --2- 172.21.2.101:0/3804433722 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7dc1a44c0 0x7fd7dc1a48c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.573 7fd7dae1d700 1 -- 172.21.2.101:0/3804433722 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7dc087010 con 0x7fd7dc0863d0 2021-02-10T11:47:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.573 7fd7dae1d700 1 --2- 172.21.2.101:0/3804433722 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7dc0863d0 0x7fd7dc0867d0 secure :-1 s=READY pgs=3090 cs=0 l=1 rev1=1 rx=0x7fd7d4009960 tx=0x7fd7d400bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.573 7fd7d2ffd700 1 -- 172.21.2.101:0/3804433722 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd7d4000d10 con 0x7fd7dc0863d0 2021-02-10T11:47:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.573 7fd7d2ffd700 1 -- 172.21.2.101:0/3804433722 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d4003030 con 0x7fd7dc0863d0 2021-02-10T11:47:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.573 7fd7d2ffd700 1 -- 172.21.2.101:0/3804433722 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd7d4001590 con 0x7fd7dc0863d0 2021-02-10T11:47:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.573 7fd7e0942700 1 -- 172.21.2.101:0/3804433722 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7dc087010 con 0x7fd7dc0863d0 2021-02-10T11:47:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.573 7fd7e0942700 1 -- 172.21.2.101:0/3804433722 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7dc083000 con 0x7fd7dc0863d0 2021-02-10T11:47:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.574 7fd7d2ffd700 1 -- 172.21.2.101:0/3804433722 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd7d4001310 con 0x7fd7dc0863d0 2021-02-10T11:47:26.573 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.574 7fd7d2ffd700 1 --2- 172.21.2.101:0/3804433722 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7c00186c0 0x7fd7c001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:26.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.574 7fd7d2ffd700 1 -- 172.21.2.101:0/3804433722 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fd7d40325f0 con 0x7fd7dc0863d0 2021-02-10T11:47:26.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.574 7fd7da61c700 1 --2- 172.21.2.101:0/3804433722 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7c00186c0 0x7fd7c001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:26.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.574 7fd7da61c700 1 --2- 172.21.2.101:0/3804433722 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7c00186c0 0x7fd7c001abc0 crc :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:26.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.605 7fd7e0942700 1 -- 172.21.2.101:0/3804433722 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7dc0fe1c0 con 0x7fd7dc0863d0 2021-02-10T11:47:26.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.608 7fd7d2ffd700 1 -- 172.21.2.101:0/3804433722 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd7d4000f40 con 0x7fd7dc0863d0 2021-02-10T11:47:26.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:26.737 7fd7e0942700 1 -- 172.21.2.101:0/3804433722 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"} v 0) v1 -- 0x7fd7dc0fe1c0 con 0x7fd7dc0863d0 2021-02-10T11:47:27.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.150 7fd7d2ffd700 1 -- 172.21.2.101:0/3804433722 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"}]=0 pool 'cephfs_metadata' created v180) v1 ==== 133+0+0 (secure 0 0 0) 0x7fd7d4001310 con 0x7fd7dc0863d0 2021-02-10T11:47:27.150 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' created 2021-02-10T11:47:27.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.155 7fd7e0942700 1 -- 172.21.2.101:0/3804433722 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7c00186c0 msgr2=0x7fd7c001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:27.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.155 7fd7e0942700 1 --2- 172.21.2.101:0/3804433722 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7c00186c0 0x7fd7c001abc0 crc :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:27.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.155 7fd7e0942700 1 -- 172.21.2.101:0/3804433722 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7dc0863d0 msgr2=0x7fd7dc0867d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:27.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.155 7fd7e0942700 1 --2- 172.21.2.101:0/3804433722 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7dc0863d0 0x7fd7dc0867d0 secure :-1 s=READY pgs=3090 cs=0 l=1 rev1=1 rx=0x7fd7d4009960 tx=0x7fd7d400bcf0).stop 2021-02-10T11:47:27.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.156 7fd7e0942700 1 -- 172.21.2.101:0/3804433722 shutdown_connections 2021-02-10T11:47:27.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.156 7fd7e0942700 1 --2- 172.21.2.101:0/3804433722 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7dc0863d0 0x7fd7dc0867d0 unknown :-1 s=CLOSED pgs=3090 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:27.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.156 7fd7e0942700 1 --2- 172.21.2.101:0/3804433722 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd7dc1a44c0 0x7fd7dc1a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:27.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.156 7fd7e0942700 1 --2- 172.21.2.101:0/3804433722 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7c00186c0 0x7fd7c001abc0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:27.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.156 7fd7e0942700 1 --2- 172.21.2.101:0/3804433722 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7dc0c3820 0x7fd7dc0a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:27.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.156 7fd7e0942700 1 -- 172.21.2.101:0/3804433722 >> 172.21.2.101:0/3804433722 conn(0x7fd7dc0872c0 msgr2=0x7fd7dc095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:27.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.156 7fd7e0942700 1 -- 172.21.2.101:0/3804433722 shutdown_connections 2021-02-10T11:47:27.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.156 7fd7e0942700 1 -- 172.21.2.101:0/3804433722 wait complete. 2021-02-10T11:47:27.161 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_data 8 2021-02-10T11:47:27.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.579 7f561a1a9700 1 Processor -- start 2021-02-10T11:47:27.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.579 7f561a1a9700 1 -- start start 2021-02-10T11:47:27.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.580 7f561a1a9700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f56140d1c70 0x7f56140cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:27.592 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.580 7f561a1a9700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f56140cf580 0x7f56140cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:27.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.580 7f561a1a9700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56140fe1c0 con 0x7f56140d1870 2021-02-10T11:47:27.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.580 7f561a1a9700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56141b0cb0 con 0x7f56140cf580 2021-02-10T11:47:27.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.580 7f561a1a9700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5614109a60 con 0x7f56140d1c70 2021-02-10T11:47:27.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.580 7f5613fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f56140cf580 0x7f56140cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:27.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.580 7f5613fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f56140cf580 0x7f56140cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34044/0 (socket says 172.21.2.101:34044) 2021-02-10T11:47:27.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.580 7f5613fff700 1 -- 172.21.2.101:0/467025194 learned_addr learned my addr 172.21.2.101:0/467025194 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:27.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.580 7f5612ffd700 1 --2- 172.21.2.101:0/467025194 >> v2:172.21.2.108:3300/0 conn(0x7f56140d1c70 0x7f56140cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:27.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.580 7f5613fff700 1 -- 172.21.2.101:0/467025194 >> v2:172.21.2.108:3300/0 conn(0x7f56140d1c70 msgr2=0x7f56140cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:27.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.580 7f5613fff700 1 --2- 172.21.2.101:0/467025194 >> v2:172.21.2.108:3300/0 conn(0x7f56140d1c70 0x7f56140cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:27.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.580 7f5613fff700 1 -- 172.21.2.101:0/467025194 >> v1:172.21.2.108:6789/0 conn(0x7f56140d1870 legacy=0x7f56141b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:27.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.580 7f5613fff700 1 -- 172.21.2.101:0/467025194 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5614088070 con 0x7f56140cf580 2021-02-10T11:47:27.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.580 7f5613fff700 1 --2- 172.21.2.101:0/467025194 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56140cf580 0x7f56140cf980 secure :-1 s=READY pgs=3091 cs=0 l=1 rev1=1 rx=0x7f560c006f00 tx=0x7f560c009f10).ready entity=mon.0 client_cookie=d73f280309be8725 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:27.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.581 7f56127fc700 1 -- 172.21.2.101:0/467025194 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f560c00b3b0 con 0x7f56140cf580 2021-02-10T11:47:27.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.581 7f56127fc700 1 -- 172.21.2.101:0/467025194 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560c00b730 con 0x7f56140cf580 2021-02-10T11:47:27.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.581 7f56127fc700 1 -- 172.21.2.101:0/467025194 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f560c00bc50 con 0x7f56140cf580 2021-02-10T11:47:27.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.581 7f561a1a9700 1 -- 172.21.2.101:0/467025194 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56140cf580 msgr2=0x7f56140cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:27.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.581 7f561a1a9700 1 --2- 172.21.2.101:0/467025194 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56140cf580 0x7f56140cf980 secure :-1 s=READY pgs=3091 cs=0 l=1 rev1=1 rx=0x7f560c006f00 tx=0x7f560c009f10).stop 2021-02-10T11:47:27.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.581 7f561a1a9700 1 -- 172.21.2.101:0/467025194 shutdown_connections 2021-02-10T11:47:27.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.581 7f561a1a9700 1 --2- 172.21.2.101:0/467025194 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56140cf580 0x7f56140cf980 unknown :-1 s=CLOSED pgs=3091 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:27.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.581 7f561a1a9700 1 --2- 172.21.2.101:0/467025194 >> v2:172.21.2.108:3300/0 conn(0x7f56140d1c70 0x7f56140cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:27.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.581 7f561a1a9700 1 -- 172.21.2.101:0/467025194 >> 172.21.2.101:0/467025194 conn(0x7f56140ce420 msgr2=0x7f56140d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:27.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.581 7f561a1a9700 1 -- 172.21.2.101:0/467025194 shutdown_connections 2021-02-10T11:47:27.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.581 7f561a1a9700 1 -- 172.21.2.101:0/467025194 wait complete. 2021-02-10T11:47:27.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.596 7f561a1a9700 1 Processor -- start 2021-02-10T11:47:27.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.596 7f561a1a9700 1 -- start start 2021-02-10T11:47:27.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.596 7f561a1a9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5614085bc0 0x7f56140863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:27.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.596 7f561a1a9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56140954c0 0x7f5614096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:27.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.596 7f561a1a9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56140958c0 0x7f5614096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:27.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.596 7f561a1a9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5614095d80 con 0x7f5614085bc0 2021-02-10T11:47:27.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.596 7f561a1a9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56140bfa60 con 0x7f56140954c0 2021-02-10T11:47:27.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.596 7f561a1a9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5614085fc0 con 0x7f56140958c0 2021-02-10T11:47:27.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.596 7f56137fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5614085bc0 0x7f56140863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:27.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.596 7f56137fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5614085bc0 0x7f56140863e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34046/0 (socket says 172.21.2.101:34046) 2021-02-10T11:47:27.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.596 7f56137fe700 1 -- 172.21.2.101:0/3499356133 learned_addr learned my addr 172.21.2.101:0/3499356133 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:27.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.597 7f5612ffd700 1 --2- 172.21.2.101:0/3499356133 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56140954c0 0x7f5614096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:27.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.597 7f5613fff700 1 --2- 172.21.2.101:0/3499356133 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56140958c0 0x7f5614096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:27.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.597 7f56137fe700 1 -- 172.21.2.101:0/3499356133 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56140958c0 msgr2=0x7f5614096920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:27.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.597 7f56137fe700 1 --2- 172.21.2.101:0/3499356133 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56140958c0 0x7f5614096920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:27.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.597 7f56137fe700 1 -- 172.21.2.101:0/3499356133 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56140954c0 msgr2=0x7f5614096300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:27.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.597 7f56137fe700 1 --2- 172.21.2.101:0/3499356133 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56140954c0 0x7f5614096300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:27.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.597 7f56137fe700 1 -- 172.21.2.101:0/3499356133 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5614086c10 con 0x7f5614085bc0 2021-02-10T11:47:27.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.597 7f5613fff700 1 --2- 172.21.2.101:0/3499356133 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56140958c0 0x7f5614096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:47:27.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.597 7f56137fe700 1 --2- 172.21.2.101:0/3499356133 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5614085bc0 0x7f56140863e0 secure :-1 s=READY pgs=3092 cs=0 l=1 rev1=1 rx=0x7f560800b5b0 tx=0x7f560800b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:27.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.597 7f55fbfff700 1 -- 172.21.2.101:0/3499356133 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f560800bd40 con 0x7f5614085bc0 2021-02-10T11:47:27.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.597 7f55fbfff700 1 -- 172.21.2.101:0/3499356133 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5608006240 con 0x7f5614085bc0 2021-02-10T11:47:27.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.597 7f55fbfff700 1 -- 172.21.2.101:0/3499356133 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5608005720 con 0x7f5614085bc0 2021-02-10T11:47:27.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.597 7f561a1a9700 1 -- 172.21.2.101:0/3499356133 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5614086c10 con 0x7f5614085bc0 2021-02-10T11:47:27.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.597 7f561a1a9700 1 -- 172.21.2.101:0/3499356133 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5614086c10 con 0x7f5614085bc0 2021-02-10T11:47:27.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.598 7f55fbfff700 1 -- 172.21.2.101:0/3499356133 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f56080054a0 con 0x7f5614085bc0 2021-02-10T11:47:27.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.598 7f55fbfff700 1 --2- 172.21.2.101:0/3499356133 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f55f40186c0 0x7f55f401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:27.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.598 7f55fbfff700 1 -- 172.21.2.101:0/3499356133 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5107+0+0 (secure 0 0 0) 0x7f56080323d0 con 0x7f5614085bc0 2021-02-10T11:47:27.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.598 7f5612ffd700 1 --2- 172.21.2.101:0/3499356133 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f55f40186c0 0x7f55f401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:27.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.598 7f5612ffd700 1 --2- 172.21.2.101:0/3499356133 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f55f40186c0 0x7f55f401abc0 crc :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:27.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.629 7f561a1a9700 1 -- 172.21.2.101:0/3499356133 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56140fe1c0 con 0x7f5614085bc0 2021-02-10T11:47:27.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.632 7f55fbfff700 1 -- 172.21.2.101:0/3499356133 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f560802d3a0 con 0x7f5614085bc0 2021-02-10T11:47:27.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:27.762 7f561a1a9700 1 -- 172.21.2.101:0/3499356133 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"} v 0) v1 -- 0x7f56140fe1c0 con 0x7f5614085bc0 2021-02-10T11:47:28.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.156 7f55fbfff700 1 -- 172.21.2.101:0/3499356133 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"}]=0 pool 'cephfs_data' created v181) v1 ==== 125+0+0 (secure 0 0 0) 0x7f560802d3a0 con 0x7f5614085bc0 2021-02-10T11:47:28.173 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' created 2021-02-10T11:47:28.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.178 7f561a1a9700 1 -- 172.21.2.101:0/3499356133 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f55f40186c0 msgr2=0x7f55f401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:28.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.178 7f561a1a9700 1 --2- 172.21.2.101:0/3499356133 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f55f40186c0 0x7f55f401abc0 crc :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:28.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.178 7f561a1a9700 1 -- 172.21.2.101:0/3499356133 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5614085bc0 msgr2=0x7f56140863e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:28.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.178 7f561a1a9700 1 --2- 172.21.2.101:0/3499356133 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5614085bc0 0x7f56140863e0 secure :-1 s=READY pgs=3092 cs=0 l=1 rev1=1 rx=0x7f560800b5b0 tx=0x7f560800b950).stop 2021-02-10T11:47:28.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.178 7f561a1a9700 1 -- 172.21.2.101:0/3499356133 shutdown_connections 2021-02-10T11:47:28.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.178 7f561a1a9700 1 --2- 172.21.2.101:0/3499356133 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f55f40186c0 0x7f55f401abc0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:28.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.178 7f561a1a9700 1 --2- 172.21.2.101:0/3499356133 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f56140958c0 0x7f5614096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:28.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.178 7f561a1a9700 1 --2- 172.21.2.101:0/3499356133 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56140954c0 0x7f5614096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:28.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.178 7f561a1a9700 1 --2- 172.21.2.101:0/3499356133 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5614085bc0 0x7f56140863e0 unknown :-1 s=CLOSED pgs=3092 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:28.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.178 7f561a1a9700 1 -- 172.21.2.101:0/3499356133 >> 172.21.2.101:0/3499356133 conn(0x7f56140bef00 msgr2=0x7f561409d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:28.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.178 7f561a1a9700 1 -- 172.21.2.101:0/3499356133 shutdown_connections 2021-02-10T11:47:28.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.178 7f561a1a9700 1 -- 172.21.2.101:0/3499356133 wait complete. 2021-02-10T11:47:28.183 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-02-10T11:47:28.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.601 7f3fa07d0700 1 Processor -- start 2021-02-10T11:47:28.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.601 7f3fa07d0700 1 -- start start 2021-02-10T11:47:28.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.602 7f3fa07d0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3f980d1880 0x7f3f981b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:28.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.602 7f3fa07d0700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3f980cff80 0x7f3f98109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:28.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.602 7f3fa07d0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f980fe1d0 con 0x7f3f981b0af0 2021-02-10T11:47:28.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.602 7f3fa07d0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f98109ae0 con 0x7f3f980d1880 2021-02-10T11:47:28.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.602 7f3fa07d0700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f98109760 con 0x7f3f980cff80 2021-02-10T11:47:28.615 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.602 7f3f9e5ec700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3f980d1880 0x7f3f981b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:28.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.602 7f3f9e5ec700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3f980d1880 0x7f3f981b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34056/0 (socket says 172.21.2.101:34056) 2021-02-10T11:47:28.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.602 7f3f9e5ec700 1 -- 172.21.2.101:0/2520875202 learned_addr learned my addr 172.21.2.101:0/2520875202 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:28.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.602 7f3f9e5ec700 1 -- 172.21.2.101:0/2520875202 >> v2:172.21.2.108:3300/0 conn(0x7f3f980cff80 msgr2=0x7f3f98109140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:28.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.602 7f3f9e5ec700 1 --2- 172.21.2.101:0/2520875202 >> v2:172.21.2.108:3300/0 conn(0x7f3f980cff80 0x7f3f98109140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:28.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.602 7f3f9e5ec700 1 -- 172.21.2.101:0/2520875202 >> v1:172.21.2.108:6789/0 conn(0x7f3f981b0af0 legacy=0x7f3f98108950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:28.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.602 7f3f9e5ec700 1 -- 172.21.2.101:0/2520875202 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f98088070 con 0x7f3f980d1880 2021-02-10T11:47:28.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.603 7f3f9e5ec700 1 --2- 172.21.2.101:0/2520875202 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f980d1880 0x7f3f981b04d0 secure :-1 s=READY pgs=3093 cs=0 l=1 rev1=1 rx=0x7f3f94006700 tx=0x7f3f94009b80).ready entity=mon.0 client_cookie=6fb043f77bbd06da server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:28.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.603 7f3f9d5ea700 1 -- 172.21.2.101:0/2520875202 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3f9400c0c0 con 0x7f3f980d1880 2021-02-10T11:47:28.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.603 7f3f9d5ea700 1 -- 172.21.2.101:0/2520875202 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f9400c340 con 0x7f3f980d1880 2021-02-10T11:47:28.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.603 7f3f9d5ea700 1 -- 172.21.2.101:0/2520875202 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3f9400c860 con 0x7f3f980d1880 2021-02-10T11:47:28.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.603 7f3fa07d0700 1 -- 172.21.2.101:0/2520875202 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f980d1880 msgr2=0x7f3f981b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:28.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.603 7f3fa07d0700 1 --2- 172.21.2.101:0/2520875202 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f980d1880 0x7f3f981b04d0 secure :-1 s=READY pgs=3093 cs=0 l=1 rev1=1 rx=0x7f3f94006700 tx=0x7f3f94009b80).stop 2021-02-10T11:47:28.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.604 7f3fa07d0700 1 -- 172.21.2.101:0/2520875202 shutdown_connections 2021-02-10T11:47:28.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.604 7f3fa07d0700 1 --2- 172.21.2.101:0/2520875202 >> v2:172.21.2.108:3300/0 conn(0x7f3f980cff80 0x7f3f98109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:28.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.604 7f3fa07d0700 1 --2- 172.21.2.101:0/2520875202 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f980d1880 0x7f3f981b04d0 unknown :-1 s=CLOSED pgs=3093 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:28.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.604 7f3fa07d0700 1 -- 172.21.2.101:0/2520875202 >> 172.21.2.101:0/2520875202 conn(0x7f3f980ce430 msgr2=0x7f3f980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:28.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.604 7f3fa07d0700 1 -- 172.21.2.101:0/2520875202 shutdown_connections 2021-02-10T11:47:28.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.604 7f3fa07d0700 1 -- 172.21.2.101:0/2520875202 wait complete. 2021-02-10T11:47:28.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.618 7f3fa07d0700 1 Processor -- start 2021-02-10T11:47:28.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.618 7f3fa07d0700 1 -- start start 2021-02-10T11:47:28.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.619 7f3fa07d0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f98187e80 0x7f3f98188280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:28.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.619 7f3fa07d0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3f981888a0 0x7f3f98189490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:28.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.619 7f3fa07d0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f98189ab0 0x7f3f98189eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:28.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.619 7f3fa07d0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f98188ca0 con 0x7f3f98187e80 2021-02-10T11:47:28.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.619 7f3fa07d0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f98188f10 con 0x7f3f981888a0 2021-02-10T11:47:28.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.619 7f3fa07d0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f98099520 con 0x7f3f98189ab0 2021-02-10T11:47:28.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.619 7f3f9e5ec700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f98187e80 0x7f3f98188280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:28.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.619 7f3f9e5ec700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f98187e80 0x7f3f98188280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34060/0 (socket says 172.21.2.101:34060) 2021-02-10T11:47:28.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.619 7f3f9e5ec700 1 -- 172.21.2.101:0/1427274010 learned_addr learned my addr 172.21.2.101:0/1427274010 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:28.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.619 7f3f9e5ec700 1 -- 172.21.2.101:0/1427274010 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f98189ab0 msgr2=0x7f3f98189eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:28.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.619 7f3f9e5ec700 1 --2- 172.21.2.101:0/1427274010 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f98189ab0 0x7f3f98189eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:28.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.619 7f3f9e5ec700 1 -- 172.21.2.101:0/1427274010 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3f981888a0 msgr2=0x7f3f98189490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:28.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.619 7f3f9e5ec700 1 --2- 172.21.2.101:0/1427274010 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3f981888a0 0x7f3f98189490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:28.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.619 7f3f9e5ec700 1 -- 172.21.2.101:0/1427274010 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f980999c0 con 0x7f3f98187e80 2021-02-10T11:47:28.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.619 7f3f9e5ec700 1 --2- 172.21.2.101:0/1427274010 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f98187e80 0x7f3f98188280 secure :-1 s=READY pgs=3094 cs=0 l=1 rev1=1 rx=0x7f3f9400b5e0 tx=0x7f3f9400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:28.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.620 7f3f8effd700 1 -- 172.21.2.101:0/1427274010 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3f9400bd70 con 0x7f3f98187e80 2021-02-10T11:47:28.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.620 7f3f8effd700 1 -- 172.21.2.101:0/1427274010 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f94002030 con 0x7f3f98187e80 2021-02-10T11:47:28.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.620 7f3f8effd700 1 -- 172.21.2.101:0/1427274010 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3f94002550 con 0x7f3f98187e80 2021-02-10T11:47:28.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.620 7f3fa07d0700 1 -- 172.21.2.101:0/1427274010 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f980999c0 con 0x7f3f98187e80 2021-02-10T11:47:28.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.620 7f3fa07d0700 1 -- 172.21.2.101:0/1427274010 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f9818a7c0 con 0x7f3f98187e80 2021-02-10T11:47:28.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.621 7f3f8effd700 1 -- 172.21.2.101:0/1427274010 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3f94000c90 con 0x7f3f98187e80 2021-02-10T11:47:28.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.621 7f3f8effd700 1 --2- 172.21.2.101:0/1427274010 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f800186c0 0x7f3f8001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:28.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.621 7f3f8effd700 1 -- 172.21.2.101:0/1427274010 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5457+0+0 (secure 0 0 0) 0x7f3f94032510 con 0x7f3f98187e80 2021-02-10T11:47:28.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.621 7f3f9ddeb700 1 --2- 172.21.2.101:0/1427274010 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f800186c0 0x7f3f8001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:28.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.621 7f3f9ddeb700 1 --2- 172.21.2.101:0/1427274010 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f800186c0 0x7f3f8001abc0 crc :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:28.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.652 7f3fa07d0700 1 -- 172.21.2.101:0/1427274010 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f98195bd0 con 0x7f3f98187e80 2021-02-10T11:47:28.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.654 7f3f8effd700 1 -- 172.21.2.101:0/1427274010 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3f94000940 con 0x7f3f98187e80 2021-02-10T11:47:28.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.785 7f3fa07d0700 1 -- 172.21.2.101:0/1427274010 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"} v 0) v1 -- 0x7f3f98195bd0 con 0x7f3f98187e80 2021-02-10T11:47:28.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.789 7f3f8effd700 1 -- 172.21.2.101:0/1427274010 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"}]=0 new fs with metadata pool 38 and data pool 39 v153) v1 ==== 189+0+0 (secure 0 0 0) 0x7f3f94000c90 con 0x7f3f98187e80 2021-02-10T11:47:28.788 INFO:teuthology.orchestra.run.gibba001.stderr:new fs with metadata pool 38 and data pool 39 2021-02-10T11:47:28.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.794 7f3fa07d0700 1 -- 172.21.2.101:0/1427274010 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f800186c0 msgr2=0x7f3f8001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:28.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.794 7f3fa07d0700 1 --2- 172.21.2.101:0/1427274010 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f800186c0 0x7f3f8001abc0 crc :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:28.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.794 7f3fa07d0700 1 -- 172.21.2.101:0/1427274010 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f98187e80 msgr2=0x7f3f98188280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:28.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.794 7f3fa07d0700 1 --2- 172.21.2.101:0/1427274010 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f98187e80 0x7f3f98188280 secure :-1 s=READY pgs=3094 cs=0 l=1 rev1=1 rx=0x7f3f9400b5e0 tx=0x7f3f9400b980).stop 2021-02-10T11:47:28.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.794 7f3fa07d0700 1 -- 172.21.2.101:0/1427274010 shutdown_connections 2021-02-10T11:47:28.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.794 7f3fa07d0700 1 --2- 172.21.2.101:0/1427274010 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3f800186c0 0x7f3f8001abc0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:28.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.794 7f3fa07d0700 1 --2- 172.21.2.101:0/1427274010 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3f98189ab0 0x7f3f98189eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:28.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.794 7f3fa07d0700 1 --2- 172.21.2.101:0/1427274010 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3f981888a0 0x7f3f98189490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:28.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.794 7f3fa07d0700 1 --2- 172.21.2.101:0/1427274010 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3f98187e80 0x7f3f98188280 unknown :-1 s=CLOSED pgs=3094 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:28.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.794 7f3fa07d0700 1 -- 172.21.2.101:0/1427274010 >> 172.21.2.101:0/1427274010 conn(0x7f3f9819f940 msgr2=0x7f3f9818c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:28.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.794 7f3fa07d0700 1 -- 172.21.2.101:0/1427274010 shutdown_connections 2021-02-10T11:47:28.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:28.794 7f3fa07d0700 1 -- 172.21.2.101:0/1427274010 wait complete. 2021-02-10T11:47:28.799 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:47:29.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.214 7f0b47ce2700 1 Processor -- start 2021-02-10T11:47:29.228 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.215 7f0b47ce2700 1 -- start start 2021-02-10T11:47:29.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.215 7f0b47ce2700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0b400d1870 0x7f0b401b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:29.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.215 7f0b47ce2700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b400f0b10 con 0x7f0b400f0650 2021-02-10T11:47:29.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.215 7f0b47ce2700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b400d0310 con 0x7f0b401b0a10 2021-02-10T11:47:29.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.215 7f0b47ce2700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b400d0530 con 0x7f0b400d1870 2021-02-10T11:47:29.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.215 7f0b462ff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f0b400f0650 0x7f0b400cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34562/0 (socket says 172.21.2.101:34562) 2021-02-10T11:47:29.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.215 7f0b462ff700 1 -- 172.21.2.101:0/516080057 learned_addr learned my addr 172.21.2.101:0/516080057 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:29.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.215 7f0b45afe700 1 --2- 172.21.2.101:0/516080057 >> v2:172.21.2.108:3300/0 conn(0x7f0b400d1870 0x7f0b401b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:29.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.216 7f0b44afc700 1 -- 172.21.2.101:0/516080057 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507203803 0 0) 0x7f0b38000d30 con 0x7f0b400f0650 2021-02-10T11:47:29.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.216 7f0b45afe700 1 -- 172.21.2.101:0/516080057 >> v1:172.21.2.108:6789/0 conn(0x7f0b401b0a10 legacy=0x7f0b400cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:29.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.216 7f0b45afe700 1 -- 172.21.2.101:0/516080057 >> v1:172.21.2.101:6789/0 conn(0x7f0b400f0650 legacy=0x7f0b400cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:29.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.216 7f0b45afe700 1 -- 172.21.2.101:0/516080057 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b400d1c70 con 0x7f0b400d1870 2021-02-10T11:47:29.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.216 7f0b45afe700 1 --2- 172.21.2.101:0/516080057 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b400d1870 0x7f0b401b03f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7f0b30006650 tx=0x7f0b30009a80).ready entity=mon.1 client_cookie=b4eb6ad6c3bbadeb server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:29.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.216 7f0b44afc700 1 -- 172.21.2.101:0/516080057 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0b3000c0c0 con 0x7f0b400d1870 2021-02-10T11:47:29.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.216 7f0b44afc700 1 -- 172.21.2.101:0/516080057 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3000c3c0 con 0x7f0b400d1870 2021-02-10T11:47:29.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.216 7f0b44afc700 1 -- 172.21.2.101:0/516080057 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0b3000c810 con 0x7f0b400d1870 2021-02-10T11:47:29.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.216 7f0b47ce2700 1 -- 172.21.2.101:0/516080057 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b400d1870 msgr2=0x7f0b401b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:29.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.216 7f0b47ce2700 1 --2- 172.21.2.101:0/516080057 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b400d1870 0x7f0b401b03f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7f0b30006650 tx=0x7f0b30009a80).stop 2021-02-10T11:47:29.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.217 7f0b47ce2700 1 -- 172.21.2.101:0/516080057 shutdown_connections 2021-02-10T11:47:29.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.217 7f0b47ce2700 1 --2- 172.21.2.101:0/516080057 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b400d1870 0x7f0b401b03f0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:29.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.217 7f0b47ce2700 1 -- 172.21.2.101:0/516080057 >> 172.21.2.101:0/516080057 conn(0x7f0b400ce450 msgr2=0x7f0b400d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:29.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.217 7f0b47ce2700 1 -- 172.21.2.101:0/516080057 shutdown_connections 2021-02-10T11:47:29.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.217 7f0b47ce2700 1 -- 172.21.2.101:0/516080057 wait complete. 2021-02-10T11:47:29.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.232 7f0b47ce2700 1 Processor -- start 2021-02-10T11:47:29.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.232 7f0b47ce2700 1 -- start start 2021-02-10T11:47:29.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.232 7f0b47ce2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b400b4f30 0x7f0b4018b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:29.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.232 7f0b47ce2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b40093f40 0x7f0b40094340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:29.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.232 7f0b47ce2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b40094960 0x7f0b40187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:29.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.232 7f0b47ce2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b40094e90 con 0x7f0b400b4f30 2021-02-10T11:47:29.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.232 7f0b47ce2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4018bff0 con 0x7f0b40094960 2021-02-10T11:47:29.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.232 7f0b47ce2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4008f780 con 0x7f0b40093f40 2021-02-10T11:47:29.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.233 7f0b45afe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b400b4f30 0x7f0b4018b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:29.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.233 7f0b45afe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b400b4f30 0x7f0b4018b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34072/0 (socket says 172.21.2.101:34072) 2021-02-10T11:47:29.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.233 7f0b45afe700 1 -- 172.21.2.101:0/3751802029 learned_addr learned my addr 172.21.2.101:0/3751802029 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:29.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.233 7f0b462ff700 1 --2- 172.21.2.101:0/3751802029 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b40094960 0x7f0b40187e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:29.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.233 7f0b452fd700 1 --2- 172.21.2.101:0/3751802029 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b40093f40 0x7f0b40094340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:29.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.233 7f0b45afe700 1 -- 172.21.2.101:0/3751802029 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b40093f40 msgr2=0x7f0b40094340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:29.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.233 7f0b45afe700 1 --2- 172.21.2.101:0/3751802029 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b40093f40 0x7f0b40094340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:29.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.233 7f0b45afe700 1 -- 172.21.2.101:0/3751802029 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b40094960 msgr2=0x7f0b40187e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:29.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.233 7f0b45afe700 1 --2- 172.21.2.101:0/3751802029 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b40094960 0x7f0b40187e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:29.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.233 7f0b45afe700 1 -- 172.21.2.101:0/3751802029 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b40188470 con 0x7f0b400b4f30 2021-02-10T11:47:29.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.233 7f0b462ff700 1 --2- 172.21.2.101:0/3751802029 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b40094960 0x7f0b40187e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:47:29.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.233 7f0b45afe700 1 --2- 172.21.2.101:0/3751802029 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b400b4f30 0x7f0b4018b9d0 secure :-1 s=READY pgs=3096 cs=0 l=1 rev1=1 rx=0x7f0b30006690 tx=0x7f0b3000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:29.235 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.233 7f0b367fc700 1 -- 172.21.2.101:0/3751802029 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0b3000bd50 con 0x7f0b400b4f30 2021-02-10T11:47:29.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.233 7f0b367fc700 1 -- 172.21.2.101:0/3751802029 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b300093a0 con 0x7f0b400b4f30 2021-02-10T11:47:29.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.233 7f0b367fc700 1 -- 172.21.2.101:0/3751802029 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0b30005560 con 0x7f0b400b4f30 2021-02-10T11:47:29.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.233 7f0b47ce2700 1 -- 172.21.2.101:0/3751802029 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b40188470 con 0x7f0b400b4f30 2021-02-10T11:47:29.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.234 7f0b47ce2700 1 -- 172.21.2.101:0/3751802029 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b40188470 con 0x7f0b400b4f30 2021-02-10T11:47:29.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.234 7f0b367fc700 1 -- 172.21.2.101:0/3751802029 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0b30007020 con 0x7f0b400b4f30 2021-02-10T11:47:29.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.234 7f0b367fc700 1 --2- 172.21.2.101:0/3751802029 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b280186c0 0x7f0b2801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:29.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.234 7f0b367fc700 1 -- 172.21.2.101:0/3751802029 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0b30032640 con 0x7f0b400b4f30 2021-02-10T11:47:29.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.234 7f0b452fd700 1 --2- 172.21.2.101:0/3751802029 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b280186c0 0x7f0b2801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:29.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.234 7f0b452fd700 1 --2- 172.21.2.101:0/3751802029 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b280186c0 0x7f0b2801abc0 crc :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:29.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.264 7f0b47ce2700 1 -- 172.21.2.101:0/3751802029 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b400ce450 con 0x7f0b400b4f30 2021-02-10T11:47:29.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.266 7f0b367fc700 1 -- 172.21.2.101:0/3751802029 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0b300067b0 con 0x7f0b400b4f30 2021-02-10T11:47:29.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.392 7f0b47ce2700 1 -- 172.21.2.101:0/3751802029 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b400ce450 con 0x7f0b400b4f30 2021-02-10T11:47:29.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.392 7f0b367fc700 1 -- 172.21.2.101:0/3751802029 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+14020 (secure 0 0 0) 0x7f0b30007020 con 0x7f0b400b4f30 2021-02-10T11:47:29.390 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:47:29.390 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":182,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:47:28.786493","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":39,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":38,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:47:26.756268","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":39,"pool_name":"cephfs_data","create_time":"2021-02-10 11:47:27.781048","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:47:29.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.394 7f0b47ce2700 1 -- 172.21.2.101:0/3751802029 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b280186c0 msgr2=0x7f0b2801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:29.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.394 7f0b47ce2700 1 --2- 172.21.2.101:0/3751802029 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b280186c0 0x7f0b2801abc0 crc :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:29.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.394 7f0b47ce2700 1 -- 172.21.2.101:0/3751802029 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b400b4f30 msgr2=0x7f0b4018b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:29.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.394 7f0b47ce2700 1 --2- 172.21.2.101:0/3751802029 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b400b4f30 0x7f0b4018b9d0 secure :-1 s=READY pgs=3096 cs=0 l=1 rev1=1 rx=0x7f0b30006690 tx=0x7f0b3000b960).stop 2021-02-10T11:47:29.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.394 7f0b47ce2700 1 -- 172.21.2.101:0/3751802029 shutdown_connections 2021-02-10T11:47:29.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.394 7f0b47ce2700 1 --2- 172.21.2.101:0/3751802029 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b280186c0 0x7f0b2801abc0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:29.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.394 7f0b47ce2700 1 --2- 172.21.2.101:0/3751802029 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b40093f40 0x7f0b40094340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:29.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.394 7f0b47ce2700 1 --2- 172.21.2.101:0/3751802029 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b40094960 0x7f0b40187e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:29.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.394 7f0b47ce2700 1 --2- 172.21.2.101:0/3751802029 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b400b4f30 0x7f0b4018b9d0 unknown :-1 s=CLOSED pgs=3096 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:29.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.394 7f0b47ce2700 1 -- 172.21.2.101:0/3751802029 >> 172.21.2.101:0/3751802029 conn(0x7f0b4019f8f0 msgr2=0x7f0b401a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:29.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.394 7f0b47ce2700 1 -- 172.21.2.101:0/3751802029 shutdown_connections 2021-02-10T11:47:29.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.394 7f0b47ce2700 1 -- 172.21.2.101:0/3751802029 wait complete. 2021-02-10T11:47:29.400 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:47:29.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.815 7f3847863700 1 Processor -- start 2021-02-10T11:47:29.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.815 7f3847863700 1 -- start start 2021-02-10T11:47:29.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.816 7f3847863700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f38400d1c70 0x7f38400f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:29.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.816 7f3847863700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38400f0b10 con 0x7f38400d1870 2021-02-10T11:47:29.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.816 7f3847863700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38401b0be0 con 0x7f38400d0340 2021-02-10T11:47:29.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.816 7f3847863700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38400e5ed0 con 0x7f38400d1c70 2021-02-10T11:47:29.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.816 7f3845e80700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f38400d1c70 0x7f38400f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:29.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.816 7f3844e7e700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f38400d0340 0x7f38400ef640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:56216/0 (socket says 172.21.2.101:56216) 2021-02-10T11:47:29.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.816 7f3844e7e700 1 -- 172.21.2.101:0/3582785925 learned_addr learned my addr 172.21.2.101:0/3582785925 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:29.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.816 7f384567f700 1 --1- 172.21.2.101:0/3582785925 >> v1:172.21.2.101:6789/0 conn(0x7f38400d1870 0x7f38401b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34574/0 (socket says 172.21.2.101:34574) 2021-02-10T11:47:29.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.816 7f3845e80700 1 -- 172.21.2.101:0/3582785925 >> v1:172.21.2.108:6789/0 conn(0x7f38400d0340 legacy=0x7f38400ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:29.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.816 7f3845e80700 1 -- 172.21.2.101:0/3582785925 >> v1:172.21.2.101:6789/0 conn(0x7f38400d1870 legacy=0x7f38401b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:29.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.816 7f3845e80700 1 -- 172.21.2.101:0/3582785925 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38400efe30 con 0x7f38400d1c70 2021-02-10T11:47:29.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.816 7f3845e80700 1 --2- 172.21.2.101:0/3582785925 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38400d1c70 0x7f38400f20a0 secure :-1 s=READY pgs=3098 cs=0 l=1 rev1=1 rx=0x7f3838006e30 tx=0x7f3838009e60).ready entity=mon.0 client_cookie=18ae1960b9505402 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:29.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.816 7f3833fff700 1 -- 172.21.2.101:0/3582785925 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3838006a50 con 0x7f38400d1c70 2021-02-10T11:47:29.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.817 7f3833fff700 1 -- 172.21.2.101:0/3582785925 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383800b300 con 0x7f38400d1c70 2021-02-10T11:47:29.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.817 7f3833fff700 1 -- 172.21.2.101:0/3582785925 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f383800b790 con 0x7f38400d1c70 2021-02-10T11:47:29.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.817 7f3847863700 1 -- 172.21.2.101:0/3582785925 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38400d1c70 msgr2=0x7f38400f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:29.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.817 7f3847863700 1 --2- 172.21.2.101:0/3582785925 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38400d1c70 0x7f38400f20a0 secure :-1 s=READY pgs=3098 cs=0 l=1 rev1=1 rx=0x7f3838006e30 tx=0x7f3838009e60).stop 2021-02-10T11:47:29.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.817 7f3847863700 1 -- 172.21.2.101:0/3582785925 shutdown_connections 2021-02-10T11:47:29.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.817 7f3847863700 1 --2- 172.21.2.101:0/3582785925 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38400d1c70 0x7f38400f20a0 unknown :-1 s=CLOSED pgs=3098 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:29.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.817 7f3847863700 1 -- 172.21.2.101:0/3582785925 >> 172.21.2.101:0/3582785925 conn(0x7f38400ce450 msgr2=0x7f38400d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:29.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.817 7f3847863700 1 -- 172.21.2.101:0/3582785925 shutdown_connections 2021-02-10T11:47:29.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.817 7f3847863700 1 -- 172.21.2.101:0/3582785925 wait complete. 2021-02-10T11:47:29.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.832 7f3847863700 1 Processor -- start 2021-02-10T11:47:29.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.832 7f3847863700 1 -- start start 2021-02-10T11:47:29.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.833 7f3847863700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38400af040 0x7f38400af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:29.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.833 7f3847863700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38400afa60 0x7f38400ada20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:29.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.833 7f3847863700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38400afe60 0x7f38400ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:29.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.833 7f3847863700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38400923c0 con 0x7f38400af040 2021-02-10T11:47:29.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.833 7f3847863700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38400b0410 con 0x7f38400afa60 2021-02-10T11:47:29.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.833 7f3847863700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3840091e40 con 0x7f38400afe60 2021-02-10T11:47:29.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.833 7f384567f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38400af040 0x7f38400af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:29.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.833 7f384567f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38400af040 0x7f38400af440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34088/0 (socket says 172.21.2.101:34088) 2021-02-10T11:47:29.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.833 7f384567f700 1 -- 172.21.2.101:0/4088372088 learned_addr learned my addr 172.21.2.101:0/4088372088 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:29.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.833 7f3844e7e700 1 --2- 172.21.2.101:0/4088372088 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38400afa60 0x7f38400ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:29.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.833 7f3845e80700 1 --2- 172.21.2.101:0/4088372088 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38400afe60 0x7f38400ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:29.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.833 7f384567f700 1 -- 172.21.2.101:0/4088372088 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38400afe60 msgr2=0x7f38400ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:29.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.833 7f384567f700 1 --2- 172.21.2.101:0/4088372088 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38400afe60 0x7f38400ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:29.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.833 7f384567f700 1 -- 172.21.2.101:0/4088372088 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38400afa60 msgr2=0x7f38400ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:29.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.833 7f384567f700 1 --2- 172.21.2.101:0/4088372088 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38400afa60 0x7f38400ada20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:29.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.833 7f384567f700 1 -- 172.21.2.101:0/4088372088 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384008f6d0 con 0x7f38400af040 2021-02-10T11:47:29.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.833 7f384567f700 1 --2- 172.21.2.101:0/4088372088 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38400af040 0x7f38400af440 secure :-1 s=READY pgs=3099 cs=0 l=1 rev1=1 rx=0x7f383400a5a0 tx=0x7f383400a940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:29.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.833 7f3831ffb700 1 -- 172.21.2.101:0/4088372088 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3834000c90 con 0x7f38400af040 2021-02-10T11:47:29.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.833 7f3831ffb700 1 -- 172.21.2.101:0/4088372088 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3834000f10 con 0x7f38400af040 2021-02-10T11:47:29.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.833 7f3831ffb700 1 -- 172.21.2.101:0/4088372088 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3834004620 con 0x7f38400af040 2021-02-10T11:47:29.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.833 7f3847863700 1 -- 172.21.2.101:0/4088372088 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f384008f6d0 con 0x7f38400af040 2021-02-10T11:47:29.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.833 7f3847863700 1 -- 172.21.2.101:0/4088372088 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38400ae950 con 0x7f38400af040 2021-02-10T11:47:29.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.834 7f3831ffb700 1 -- 172.21.2.101:0/4088372088 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f38340045b0 con 0x7f38400af040 2021-02-10T11:47:29.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.834 7f3831ffb700 1 --2- 172.21.2.101:0/4088372088 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38200186c0 0x7f382001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:29.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.834 7f3831ffb700 1 -- 172.21.2.101:0/4088372088 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3834032380 con 0x7f38400af040 2021-02-10T11:47:29.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.834 7f3844e7e700 1 --2- 172.21.2.101:0/4088372088 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38200186c0 0x7f382001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:29.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.834 7f3844e7e700 1 --2- 172.21.2.101:0/4088372088 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38200186c0 0x7f382001abc0 crc :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:29.862 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.865 7f3847863700 1 -- 172.21.2.101:0/4088372088 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38400aec00 con 0x7f38400af040 2021-02-10T11:47:29.864 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.868 7f3831ffb700 1 -- 172.21.2.101:0/4088372088 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f38340043a0 con 0x7f38400af040 2021-02-10T11:47:29.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.994 7f3847863700 1 -- 172.21.2.101:0/4088372088 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38400aec00 con 0x7f38400af040 2021-02-10T11:47:29.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.995 7f3831ffb700 1 -- 172.21.2.101:0/4088372088 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f38340045b0 con 0x7f38400af040 2021-02-10T11:47:29.992 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:47:29.992 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":183,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:47:29.788748","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":39,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":38,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:47:26.756268","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":39,"pool_name":"cephfs_data","create_time":"2021-02-10 11:47:27.781048","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:47:29.993 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.996 7f3847863700 1 -- 172.21.2.101:0/4088372088 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38200186c0 msgr2=0x7f382001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:29.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.996 7f3847863700 1 --2- 172.21.2.101:0/4088372088 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38200186c0 0x7f382001abc0 crc :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:29.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.996 7f3847863700 1 -- 172.21.2.101:0/4088372088 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38400af040 msgr2=0x7f38400af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:29.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.996 7f3847863700 1 --2- 172.21.2.101:0/4088372088 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38400af040 0x7f38400af440 secure :-1 s=READY pgs=3099 cs=0 l=1 rev1=1 rx=0x7f383400a5a0 tx=0x7f383400a940).stop 2021-02-10T11:47:29.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.997 7f3847863700 1 -- 172.21.2.101:0/4088372088 shutdown_connections 2021-02-10T11:47:29.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.997 7f3847863700 1 --2- 172.21.2.101:0/4088372088 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f38200186c0 0x7f382001abc0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:29.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.997 7f3847863700 1 --2- 172.21.2.101:0/4088372088 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f38400afe60 0x7f38400ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:29.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.997 7f3847863700 1 --2- 172.21.2.101:0/4088372088 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f38400afa60 0x7f38400ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:29.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.997 7f3847863700 1 --2- 172.21.2.101:0/4088372088 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f38400af040 0x7f38400af440 unknown :-1 s=CLOSED pgs=3099 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:29.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.997 7f3847863700 1 -- 172.21.2.101:0/4088372088 >> 172.21.2.101:0/4088372088 conn(0x7f38400a0030 msgr2=0x7f38400f22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:29.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.997 7f3847863700 1 -- 172.21.2.101:0/4088372088 shutdown_connections 2021-02-10T11:47:29.994 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:29.997 7f3847863700 1 -- 172.21.2.101:0/4088372088 wait complete. 2021-02-10T11:47:30.002 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-02-10T11:47:30.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.421 7f6ad4e5a700 1 Processor -- start 2021-02-10T11:47:30.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.421 7f6ad4e5a700 1 -- start start 2021-02-10T11:47:30.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.422 7f6ad4e5a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ad00d1870 0x7f6ad01b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:30.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.422 7f6ad4e5a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad0109a60 con 0x7f6ad00cff70 2021-02-10T11:47:30.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.422 7f6ad4e5a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad0088070 con 0x7f6ad01b0ae0 2021-02-10T11:47:30.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.422 7f6ad4e5a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad00e5ef0 con 0x7f6ad00d1870 2021-02-10T11:47:30.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.422 7f6acf314700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f6ad00cff70 0x7f6ad0109130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34590/0 (socket says 172.21.2.101:34590) 2021-02-10T11:47:30.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.422 7f6acf314700 1 -- 172.21.2.101:0/1558931852 learned_addr learned my addr 172.21.2.101:0/1558931852 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:30.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.422 7f6aceb13700 1 --2- 172.21.2.101:0/1558931852 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ad00d1870 0x7f6ad01b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:30.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.422 7f6acdb11700 1 -- 172.21.2.101:0/1558931852 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 707389235 0 0) 0x7f6ac0001180 con 0x7f6ad00cff70 2021-02-10T11:47:30.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.422 7f6acdb11700 1 -- 172.21.2.101:0/1558931852 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ab8002fe0 con 0x7f6ad00cff70 2021-02-10T11:47:30.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.423 7f6acdb11700 1 -- 172.21.2.101:0/1558931852 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1953861266 0 0) 0x7f6ac4000ff0 con 0x7f6ad01b0ae0 2021-02-10T11:47:30.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.423 7f6acdb11700 1 -- 172.21.2.101:0/1558931852 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ab8003210 con 0x7f6ad01b0ae0 2021-02-10T11:47:30.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.423 7f6aceb13700 1 -- 172.21.2.101:0/1558931852 >> v1:172.21.2.108:6789/0 conn(0x7f6ad01b0ae0 legacy=0x7f6ad0108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:30.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.423 7f6aceb13700 1 -- 172.21.2.101:0/1558931852 >> v1:172.21.2.101:6789/0 conn(0x7f6ad00cff70 legacy=0x7f6ad0109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:30.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.423 7f6acdb11700 1 -- 172.21.2.101:0/1558931852 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3257087211 0 0) 0x7f6ac0000ef0 con 0x7f6ad00cff70 2021-02-10T11:47:30.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.423 7f6aceb13700 1 -- 172.21.2.101:0/1558931852 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad00d1c70 con 0x7f6ad00d1870 2021-02-10T11:47:30.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.423 7f6aceb13700 1 --2- 172.21.2.101:0/1558931852 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ad00d1870 0x7f6ad01b04c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f6ac80069d0 tx=0x7f6ac8009e40).ready entity=mon.2 client_cookie=1eb1a75347fa79c6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:30.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.423 7f6acdb11700 1 -- 172.21.2.101:0/1558931852 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ac800b510 con 0x7f6ad00d1870 2021-02-10T11:47:30.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.423 7f6acdb11700 1 -- 172.21.2.101:0/1558931852 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac800b810 con 0x7f6ad00d1870 2021-02-10T11:47:30.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.423 7f6acdb11700 1 -- 172.21.2.101:0/1558931852 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ac800bc60 con 0x7f6ad00d1870 2021-02-10T11:47:30.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.423 7f6ad4e5a700 1 -- 172.21.2.101:0/1558931852 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ad00d1870 msgr2=0x7f6ad01b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:30.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.423 7f6ad4e5a700 1 --2- 172.21.2.101:0/1558931852 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ad00d1870 0x7f6ad01b04c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f6ac80069d0 tx=0x7f6ac8009e40).stop 2021-02-10T11:47:30.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.424 7f6ad4e5a700 1 -- 172.21.2.101:0/1558931852 shutdown_connections 2021-02-10T11:47:30.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.424 7f6ad4e5a700 1 --2- 172.21.2.101:0/1558931852 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ad00d1870 0x7f6ad01b04c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:30.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.424 7f6ad4e5a700 1 -- 172.21.2.101:0/1558931852 >> 172.21.2.101:0/1558931852 conn(0x7f6ad00ce420 msgr2=0x7f6ad00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:30.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.424 7f6ad4e5a700 1 -- 172.21.2.101:0/1558931852 shutdown_connections 2021-02-10T11:47:30.437 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.424 7f6ad4e5a700 1 -- 172.21.2.101:0/1558931852 wait complete. 2021-02-10T11:47:30.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.439 7f6ad4e5a700 1 Processor -- start 2021-02-10T11:47:30.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.439 7f6ad4e5a700 1 -- start start 2021-02-10T11:47:30.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.439 7f6ad4e5a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6ad008aa00 0x7f6ad01a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:30.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.439 7f6ad4e5a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ad009f580 0x7f6ad009f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:30.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.439 7f6ad4e5a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ad00a0830 0x7f6ad00a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:30.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.439 7f6ad4e5a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad00a0070 con 0x7f6ad00a0830 2021-02-10T11:47:30.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.439 7f6ad4e5a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad008ae30 con 0x7f6ad008aa00 2021-02-10T11:47:30.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.439 7f6ad4e5a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad008d2a0 con 0x7f6ad009f580 2021-02-10T11:47:30.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.439 7f6acf314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ad00a0830 0x7f6ad00a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:30.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.439 7f6acf314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ad00a0830 0x7f6ad00a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34102/0 (socket says 172.21.2.101:34102) 2021-02-10T11:47:30.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.439 7f6acf314700 1 -- 172.21.2.101:0/3864073006 learned_addr learned my addr 172.21.2.101:0/3864073006 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:30.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.439 7f6aceb13700 1 --2- 172.21.2.101:0/3864073006 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6ad008aa00 0x7f6ad01a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:30.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.439 7f6ace312700 1 --2- 172.21.2.101:0/3864073006 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ad009f580 0x7f6ad009f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:30.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.439 7f6acf314700 1 -- 172.21.2.101:0/3864073006 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ad009f580 msgr2=0x7f6ad009f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:30.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.439 7f6acf314700 1 --2- 172.21.2.101:0/3864073006 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ad009f580 0x7f6ad009f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:30.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.439 7f6acf314700 1 -- 172.21.2.101:0/3864073006 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6ad008aa00 msgr2=0x7f6ad01a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:30.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.439 7f6acf314700 1 --2- 172.21.2.101:0/3864073006 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6ad008aa00 0x7f6ad01a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:30.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.440 7f6acf314700 1 -- 172.21.2.101:0/3864073006 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad01a49d0 con 0x7f6ad00a0830 2021-02-10T11:47:30.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.440 7f6acf314700 1 --2- 172.21.2.101:0/3864073006 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ad00a0830 0x7f6ad00a0c30 secure :-1 s=READY pgs=3101 cs=0 l=1 rev1=1 rx=0x7f6ac000a5a0 tx=0x7f6ac000a940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:30.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.440 7f6abf7fe700 1 -- 172.21.2.101:0/3864073006 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ac000ad30 con 0x7f6ad00a0830 2021-02-10T11:47:30.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.440 7f6abf7fe700 1 -- 172.21.2.101:0/3864073006 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac00043a0 con 0x7f6ad00a0830 2021-02-10T11:47:30.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.440 7f6ad4e5a700 1 -- 172.21.2.101:0/3864073006 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad01a49d0 con 0x7f6ad00a0830 2021-02-10T11:47:30.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.440 7f6ad4e5a700 1 -- 172.21.2.101:0/3864073006 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad00a1540 con 0x7f6ad00a0830 2021-02-10T11:47:30.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.440 7f6abf7fe700 1 -- 172.21.2.101:0/3864073006 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6ac00047f0 con 0x7f6ad00a0830 2021-02-10T11:47:30.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.440 7f6abf7fe700 1 -- 172.21.2.101:0/3864073006 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6ac0004580 con 0x7f6ad00a0830 2021-02-10T11:47:30.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.440 7f6abf7fe700 1 --2- 172.21.2.101:0/3864073006 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ab4018730 0x7f6ab401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:30.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.440 7f6abf7fe700 1 -- 172.21.2.101:0/3864073006 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6ac00327b0 con 0x7f6ad00a0830 2021-02-10T11:47:30.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.441 7f6aceb13700 1 --2- 172.21.2.101:0/3864073006 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ab4018730 0x7f6ab401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:30.441 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.441 7f6aceb13700 1 --2- 172.21.2.101:0/3864073006 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ab4018730 0x7f6ab401ac10 crc :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:30.468 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.471 7f6ad4e5a700 1 -- 172.21.2.101:0/3864073006 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad0082980 con 0x7f6ad00a0830 2021-02-10T11:47:30.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.474 7f6abf7fe700 1 -- 172.21.2.101:0/3864073006 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6ac0000ef0 con 0x7f6ad00a0830 2021-02-10T11:47:30.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.612 7f6ad4e5a700 1 -- 172.21.2.101:0/3864073006 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"} v 0) v1 -- 0x7f6ad0109a60 con 0x7f6ad00a0830 2021-02-10T11:47:30.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.798 7f6abf7fe700 1 -- 172.21.2.101:0/3864073006 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"}]=0 v156) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6ac00043a0 con 0x7f6ad00a0830 2021-02-10T11:47:30.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.830 7f6ad4e5a700 1 -- 172.21.2.101:0/3864073006 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ab4018730 msgr2=0x7f6ab401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:30.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.830 7f6ad4e5a700 1 --2- 172.21.2.101:0/3864073006 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ab4018730 0x7f6ab401ac10 crc :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:30.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.830 7f6ad4e5a700 1 -- 172.21.2.101:0/3864073006 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ad00a0830 msgr2=0x7f6ad00a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:30.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.830 7f6ad4e5a700 1 --2- 172.21.2.101:0/3864073006 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ad00a0830 0x7f6ad00a0c30 secure :-1 s=READY pgs=3101 cs=0 l=1 rev1=1 rx=0x7f6ac000a5a0 tx=0x7f6ac000a940).stop 2021-02-10T11:47:30.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.830 7f6ad4e5a700 1 -- 172.21.2.101:0/3864073006 shutdown_connections 2021-02-10T11:47:30.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.830 7f6ad4e5a700 1 --2- 172.21.2.101:0/3864073006 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ab4018730 0x7f6ab401ac10 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:30.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.830 7f6ad4e5a700 1 --2- 172.21.2.101:0/3864073006 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6ad009f580 0x7f6ad009f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:30.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.830 7f6ad4e5a700 1 --2- 172.21.2.101:0/3864073006 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6ad00a0830 0x7f6ad00a0c30 unknown :-1 s=CLOSED pgs=3101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:30.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.830 7f6ad4e5a700 1 --2- 172.21.2.101:0/3864073006 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6ad008aa00 0x7f6ad01a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:30.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.830 7f6ad4e5a700 1 -- 172.21.2.101:0/3864073006 >> 172.21.2.101:0/3864073006 conn(0x7f6ad00bfb90 msgr2=0x7f6ad00a6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:30.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.830 7f6ad4e5a700 1 -- 172.21.2.101:0/3864073006 shutdown_connections 2021-02-10T11:47:30.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:30.830 7f6ad4e5a700 1 -- 172.21.2.101:0/3864073006 wait complete. 2021-02-10T11:47:30.835 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:47:31.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.250 7f0449f74700 1 Processor -- start 2021-02-10T11:47:31.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.250 7f0449f74700 1 -- start start 2021-02-10T11:47:31.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.251 7f0449f74700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f04440d1870 0x7f04441b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:31.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.251 7f0449f74700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f04441b0a10 0x7f04440cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:31.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.251 7f0449f74700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04440f0b10 con 0x7f04440d1c70 2021-02-10T11:47:31.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.251 7f0449f74700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04440fe230 con 0x7f04441b0a10 2021-02-10T11:47:31.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.251 7f0449f74700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04440e5ed0 con 0x7f04440d1870 2021-02-10T11:47:31.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.251 7f0442ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f04441b0a10 0x7f04440cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:31.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.251 7f0442ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f04441b0a10 0x7f04440cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34106/0 (socket says 172.21.2.101:34106) 2021-02-10T11:47:31.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.251 7f0442ffd700 1 -- 172.21.2.101:0/3788204488 learned_addr learned my addr 172.21.2.101:0/3788204488 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:31.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.251 7f0442ffd700 1 -- 172.21.2.101:0/3788204488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f04440d1870 msgr2=0x7f04441b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:31.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.251 7f0442ffd700 1 --2- 172.21.2.101:0/3788204488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f04440d1870 0x7f04441b03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:31.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.251 7f0442ffd700 1 -- 172.21.2.101:0/3788204488 >> v1:172.21.2.101:6789/0 conn(0x7f04440d1c70 legacy=0x7f04440cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:31.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.251 7f0442ffd700 1 -- 172.21.2.101:0/3788204488 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04440f0650 con 0x7f04441b0a10 2021-02-10T11:47:31.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.252 7f0442ffd700 1 --2- 172.21.2.101:0/3788204488 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04441b0a10 0x7f04440cf330 secure :-1 s=READY pgs=3103 cs=0 l=1 rev1=1 rx=0x7f042c006700 tx=0x7f042c009b00).ready entity=mon.0 client_cookie=857074639e12e3ae server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:31.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.252 7f04427fc700 1 -- 172.21.2.101:0/3788204488 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f042c00c0c0 con 0x7f04441b0a10 2021-02-10T11:47:31.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.252 7f04427fc700 1 -- 172.21.2.101:0/3788204488 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042c00c340 con 0x7f04441b0a10 2021-02-10T11:47:31.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.252 7f04427fc700 1 -- 172.21.2.101:0/3788204488 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f042c00c820 con 0x7f04441b0a10 2021-02-10T11:47:31.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.252 7f0449f74700 1 -- 172.21.2.101:0/3788204488 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04441b0a10 msgr2=0x7f04440cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:31.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.252 7f0449f74700 1 --2- 172.21.2.101:0/3788204488 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04441b0a10 0x7f04440cf330 secure :-1 s=READY pgs=3103 cs=0 l=1 rev1=1 rx=0x7f042c006700 tx=0x7f042c009b00).stop 2021-02-10T11:47:31.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.252 7f0449f74700 1 -- 172.21.2.101:0/3788204488 shutdown_connections 2021-02-10T11:47:31.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.252 7f0449f74700 1 --2- 172.21.2.101:0/3788204488 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f04441b0a10 0x7f04440cf330 unknown :-1 s=CLOSED pgs=3103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:31.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.252 7f0449f74700 1 --2- 172.21.2.101:0/3788204488 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f04440d1870 0x7f04441b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:31.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.252 7f0449f74700 1 -- 172.21.2.101:0/3788204488 >> 172.21.2.101:0/3788204488 conn(0x7f04440ce450 msgr2=0x7f04440d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:31.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.252 7f0449f74700 1 -- 172.21.2.101:0/3788204488 shutdown_connections 2021-02-10T11:47:31.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.252 7f0449f74700 1 -- 172.21.2.101:0/3788204488 wait complete. 2021-02-10T11:47:31.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.267 7f0449f74700 1 Processor -- start 2021-02-10T11:47:31.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.267 7f0449f74700 1 -- start start 2021-02-10T11:47:31.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.267 7f0449f74700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0444197460 0x7f044418a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:31.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.267 7f0449f74700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f044418ab60 0x7f04440b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:31.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.267 7f0449f74700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f04440b7df0 0x7f04440b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:31.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.267 7f0449f74700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04441aa280 con 0x7f0444197460 2021-02-10T11:47:31.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.267 7f0449f74700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04441a7940 con 0x7f04440b7df0 2021-02-10T11:47:31.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.267 7f0449f74700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04441a9e20 con 0x7f044418ab60 2021-02-10T11:47:31.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.268 7f04437fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0444197460 0x7f044418a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:31.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.268 7f04437fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0444197460 0x7f044418a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34110/0 (socket says 172.21.2.101:34110) 2021-02-10T11:47:31.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.268 7f04437fe700 1 -- 172.21.2.101:0/777026267 learned_addr learned my addr 172.21.2.101:0/777026267 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:31.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.268 7f04437fe700 1 -- 172.21.2.101:0/777026267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f044418ab60 msgr2=0x7f04440b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:31.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.268 7f04437fe700 1 --2- 172.21.2.101:0/777026267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f044418ab60 0x7f04440b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:31.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.268 7f04437fe700 1 -- 172.21.2.101:0/777026267 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f04440b7df0 msgr2=0x7f04440b81f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:47:31.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.268 7f04437fe700 1 --2- 172.21.2.101:0/777026267 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f04440b7df0 0x7f04440b81f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:31.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.268 7f04437fe700 1 -- 172.21.2.101:0/777026267 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04441aa780 con 0x7f0444197460 2021-02-10T11:47:31.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.268 7f04437fe700 1 --2- 172.21.2.101:0/777026267 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0444197460 0x7f044418a540 secure :-1 s=READY pgs=3104 cs=0 l=1 rev1=1 rx=0x7f043400b560 tx=0x7f043400b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:31.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.268 7f043bfff700 1 -- 172.21.2.101:0/777026267 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f043400bcf0 con 0x7f0444197460 2021-02-10T11:47:31.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.268 7f043bfff700 1 -- 172.21.2.101:0/777026267 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0434006030 con 0x7f0444197460 2021-02-10T11:47:31.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.268 7f0449f74700 1 -- 172.21.2.101:0/777026267 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04441aa780 con 0x7f0444197460 2021-02-10T11:47:31.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.268 7f043bfff700 1 -- 172.21.2.101:0/777026267 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f04340054d0 con 0x7f0444197460 2021-02-10T11:47:31.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.269 7f0449f74700 1 -- 172.21.2.101:0/777026267 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04441aad10 con 0x7f0444197460 2021-02-10T11:47:31.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.269 7f043bfff700 1 -- 172.21.2.101:0/777026267 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0434006210 con 0x7f0444197460 2021-02-10T11:47:31.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.269 7f043bfff700 1 --2- 172.21.2.101:0/777026267 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04280186c0 0x7f042801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:31.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.269 7f043bfff700 1 -- 172.21.2.101:0/777026267 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f04340325c0 con 0x7f0444197460 2021-02-10T11:47:31.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.269 7f0442ffd700 1 --2- 172.21.2.101:0/777026267 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04280186c0 0x7f042801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:31.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.270 7f0442ffd700 1 --2- 172.21.2.101:0/777026267 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04280186c0 0x7f042801abc0 crc :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:31.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.300 7f0449f74700 1 -- 172.21.2.101:0/777026267 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04440ce450 con 0x7f0444197460 2021-02-10T11:47:31.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.303 7f043bfff700 1 -- 172.21.2.101:0/777026267 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f04340009b0 con 0x7f0444197460 2021-02-10T11:47:31.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.432 7f0449f74700 1 -- 172.21.2.101:0/777026267 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04440ce450 con 0x7f0444197460 2021-02-10T11:47:31.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.433 7f043bfff700 1 -- 172.21.2.101:0/777026267 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+2108 (secure 0 0 0) 0x7f0434000c40 con 0x7f0444197460 2021-02-10T11:47:31.430 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:47:31.430 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":156,"default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":13128,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/122652046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":122652046},{"type":"v1","addr":"172.21.2.101:6835","nonce":122652046}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":152}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:47:28.786578","modified":"2021-02-10 11:47:30.795250","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":13125},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13125":{"gid":13125,"name":"b","rank":0,"incarnation":153,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2314483030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2314483030},{"type":"v1","addr":"172.21.2.108:6833","nonce":2314483030}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[39],"metadata_pool":38,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":19}]} 2021-02-10T11:47:31.430 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 156 2021-02-10T11:47:31.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.434 7f0449f74700 1 -- 172.21.2.101:0/777026267 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04280186c0 msgr2=0x7f042801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:31.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.434 7f0449f74700 1 --2- 172.21.2.101:0/777026267 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04280186c0 0x7f042801abc0 crc :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:31.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.434 7f0449f74700 1 -- 172.21.2.101:0/777026267 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0444197460 msgr2=0x7f044418a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:31.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.434 7f0449f74700 1 --2- 172.21.2.101:0/777026267 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0444197460 0x7f044418a540 secure :-1 s=READY pgs=3104 cs=0 l=1 rev1=1 rx=0x7f043400b560 tx=0x7f043400b900).stop 2021-02-10T11:47:31.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.435 7f0449f74700 1 -- 172.21.2.101:0/777026267 shutdown_connections 2021-02-10T11:47:31.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.435 7f0449f74700 1 --2- 172.21.2.101:0/777026267 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f04280186c0 0x7f042801abc0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:31.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.435 7f0449f74700 1 --2- 172.21.2.101:0/777026267 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f044418ab60 0x7f04440b77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:31.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.435 7f0449f74700 1 --2- 172.21.2.101:0/777026267 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f04440b7df0 0x7f04440b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:31.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.435 7f0449f74700 1 --2- 172.21.2.101:0/777026267 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0444197460 0x7f044418a540 unknown :-1 s=CLOSED pgs=3104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:31.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.435 7f0449f74700 1 -- 172.21.2.101:0/777026267 >> 172.21.2.101:0/777026267 conn(0x7f04440efc70 msgr2=0x7f04440ef410 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:31.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.435 7f0449f74700 1 -- 172.21.2.101:0/777026267 shutdown_connections 2021-02-10T11:47:31.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.435 7f0449f74700 1 -- 172.21.2.101:0/777026267 wait complete. 2021-02-10T11:47:31.440 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:47:31.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.856 7fb22a63a700 1 Processor -- start 2021-02-10T11:47:31.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.857 7fb22a63a700 1 -- start start 2021-02-10T11:47:31.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.857 7fb22a63a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb2240d1870 0x7fb2241b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:31.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.857 7fb22a63a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2241b0a10 0x7fb2240cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:31.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.857 7fb22a63a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb2240d1c70 0x7fb2240cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:31.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.857 7fb22a63a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2240fe200 con 0x7fb2240d1c70 2021-02-10T11:47:31.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.857 7fb22a63a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2240e5f10 con 0x7fb2240d1870 2021-02-10T11:47:31.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.857 7fb22a63a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2240f0b50 con 0x7fb2241b0a10 2021-02-10T11:47:31.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.857 7fb228c57700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb2240d1c70 0x7fb2240cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:31.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.857 7fb228c57700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb2240d1c70 0x7fb2240cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34120/0 (socket says 172.21.2.101:34120) 2021-02-10T11:47:31.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.857 7fb228c57700 1 -- 172.21.2.101:0/3569689084 learned_addr learned my addr 172.21.2.101:0/3569689084 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:31.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.857 7fb223fff700 1 --2- 172.21.2.101:0/3569689084 >> v2:172.21.2.108:3300/0 conn(0x7fb2240d1870 0x7fb2241b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:31.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.857 7fb2237fe700 1 --2- 172.21.2.101:0/3569689084 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2241b0a10 0x7fb2240cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:31.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.857 7fb228c57700 1 -- 172.21.2.101:0/3569689084 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2241b0a10 msgr2=0x7fb2240cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:31.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.857 7fb228c57700 1 --2- 172.21.2.101:0/3569689084 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2241b0a10 0x7fb2240cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:31.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.857 7fb228c57700 1 -- 172.21.2.101:0/3569689084 >> v2:172.21.2.108:3300/0 conn(0x7fb2240d1870 msgr2=0x7fb2241b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:31.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.857 7fb228c57700 1 --2- 172.21.2.101:0/3569689084 >> v2:172.21.2.108:3300/0 conn(0x7fb2240d1870 0x7fb2241b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:31.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.858 7fb228c57700 1 -- 172.21.2.101:0/3569689084 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2240cff70 con 0x7fb2240d1c70 2021-02-10T11:47:31.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.858 7fb223fff700 1 --2- 172.21.2.101:0/3569689084 >> v2:172.21.2.108:3300/0 conn(0x7fb2240d1870 0x7fb2241b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:47:31.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.858 7fb228c57700 1 --2- 172.21.2.101:0/3569689084 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2240d1c70 0x7fb2240cf950 secure :-1 s=READY pgs=3105 cs=0 l=1 rev1=1 rx=0x7fb21c006f00 tx=0x7fb21c009f10).ready entity=mon.0 client_cookie=c3cf1273904f87a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:31.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.858 7fb222ffd700 1 -- 172.21.2.101:0/3569689084 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb21c00b3b0 con 0x7fb2240d1c70 2021-02-10T11:47:31.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.858 7fb222ffd700 1 -- 172.21.2.101:0/3569689084 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21c000d30 con 0x7fb2240d1c70 2021-02-10T11:47:31.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.858 7fb222ffd700 1 -- 172.21.2.101:0/3569689084 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb21c00b8b0 con 0x7fb2240d1c70 2021-02-10T11:47:31.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.858 7fb22a63a700 1 -- 172.21.2.101:0/3569689084 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2240d1c70 msgr2=0x7fb2240cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:31.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.858 7fb22a63a700 1 --2- 172.21.2.101:0/3569689084 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2240d1c70 0x7fb2240cf950 secure :-1 s=READY pgs=3105 cs=0 l=1 rev1=1 rx=0x7fb21c006f00 tx=0x7fb21c009f10).stop 2021-02-10T11:47:31.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.858 7fb22a63a700 1 -- 172.21.2.101:0/3569689084 shutdown_connections 2021-02-10T11:47:31.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.858 7fb22a63a700 1 --2- 172.21.2.101:0/3569689084 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2241b0a10 0x7fb2240cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:31.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.858 7fb22a63a700 1 --2- 172.21.2.101:0/3569689084 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2240d1c70 0x7fb2240cf950 unknown :-1 s=CLOSED pgs=3105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:31.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.858 7fb22a63a700 1 --2- 172.21.2.101:0/3569689084 >> v2:172.21.2.108:3300/0 conn(0x7fb2240d1870 0x7fb2241b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:31.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.858 7fb22a63a700 1 -- 172.21.2.101:0/3569689084 >> 172.21.2.101:0/3569689084 conn(0x7fb2240ce450 msgr2=0x7fb2240d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:31.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.859 7fb22a63a700 1 -- 172.21.2.101:0/3569689084 shutdown_connections 2021-02-10T11:47:31.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.859 7fb22a63a700 1 -- 172.21.2.101:0/3569689084 wait complete. 2021-02-10T11:47:31.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.873 7fb22a63a700 1 Processor -- start 2021-02-10T11:47:31.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.873 7fb22a63a700 1 -- start start 2021-02-10T11:47:31.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.874 7fb22a63a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2241942b0 0x7fb2240b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:31.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.874 7fb22a63a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb224195070 0x7fb224195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:31.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.874 7fb22a63a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2240bb8e0 0x7fb2240bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:31.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.874 7fb22a63a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2241946b0 con 0x7fb2240bb8e0 2021-02-10T11:47:31.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.874 7fb22a63a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2240b3b70 con 0x7fb224195070 2021-02-10T11:47:31.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.874 7fb22a63a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2240b3d90 con 0x7fb2241942b0 2021-02-10T11:47:31.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.874 7fb228c57700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2240bb8e0 0x7fb2240bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:31.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.874 7fb228c57700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2240bb8e0 0x7fb2240bbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34126/0 (socket says 172.21.2.101:34126) 2021-02-10T11:47:31.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.874 7fb228c57700 1 -- 172.21.2.101:0/922326497 learned_addr learned my addr 172.21.2.101:0/922326497 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:31.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.874 7fb223fff700 1 --2- 172.21.2.101:0/922326497 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2241942b0 0x7fb2240b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:31.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.874 7fb2237fe700 1 --2- 172.21.2.101:0/922326497 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb224195070 0x7fb224195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:31.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.874 7fb228c57700 1 -- 172.21.2.101:0/922326497 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2241942b0 msgr2=0x7fb2240b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:31.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.874 7fb228c57700 1 --2- 172.21.2.101:0/922326497 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2241942b0 0x7fb2240b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:31.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.874 7fb228c57700 1 -- 172.21.2.101:0/922326497 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb224195070 msgr2=0x7fb224195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:31.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.874 7fb228c57700 1 --2- 172.21.2.101:0/922326497 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb224195070 0x7fb224195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:31.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.874 7fb228c57700 1 -- 172.21.2.101:0/922326497 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2241a49e0 con 0x7fb2240bb8e0 2021-02-10T11:47:31.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.874 7fb228c57700 1 --2- 172.21.2.101:0/922326497 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2240bb8e0 0x7fb2240bbce0 secure :-1 s=READY pgs=3106 cs=0 l=1 rev1=1 rx=0x7fb21c00b990 tx=0x7fb21c00bd30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:31.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.875 7fb220ff9700 1 -- 172.21.2.101:0/922326497 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb21c000f90 con 0x7fb2240bb8e0 2021-02-10T11:47:31.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.875 7fb220ff9700 1 -- 172.21.2.101:0/922326497 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21c006c30 con 0x7fb2240bb8e0 2021-02-10T11:47:31.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.875 7fb220ff9700 1 -- 172.21.2.101:0/922326497 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb21c005ad0 con 0x7fb2240bb8e0 2021-02-10T11:47:31.875 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.875 7fb22a63a700 1 -- 172.21.2.101:0/922326497 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2241a49e0 con 0x7fb2240bb8e0 2021-02-10T11:47:31.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.875 7fb22a63a700 1 -- 172.21.2.101:0/922326497 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2240bc5f0 con 0x7fb2240bb8e0 2021-02-10T11:47:31.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.876 7fb220ff9700 1 -- 172.21.2.101:0/922326497 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb21c009450 con 0x7fb2240bb8e0 2021-02-10T11:47:31.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.876 7fb220ff9700 1 --2- 172.21.2.101:0/922326497 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2040186c0 0x7fb20401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:31.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.876 7fb220ff9700 1 -- 172.21.2.101:0/922326497 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb21c032780 con 0x7fb2240bb8e0 2021-02-10T11:47:31.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.876 7fb223fff700 1 --2- 172.21.2.101:0/922326497 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2040186c0 0x7fb20401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:31.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.876 7fb223fff700 1 --2- 172.21.2.101:0/922326497 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2040186c0 0x7fb20401abc0 crc :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:31.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.907 7fb22a63a700 1 -- 172.21.2.101:0/922326497 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2240bc5f0 con 0x7fb2240bb8e0 2021-02-10T11:47:31.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:31.910 7fb220ff9700 1 -- 172.21.2.101:0/922326497 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb21c014f50 con 0x7fb2240bb8e0 2021-02-10T11:47:32.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.032 7fb22a63a700 1 -- 172.21.2.101:0/922326497 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2240bc5f0 con 0x7fb2240bb8e0 2021-02-10T11:47:32.030 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.033 7fb220ff9700 1 -- 172.21.2.101:0/922326497 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+14007 (secure 0 0 0) 0x7fb21c014f50 con 0x7fb2240bb8e0 2021-02-10T11:47:32.041 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:47:32.041 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":183,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:47:29.788748","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":39,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":38,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:47:26.756268","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":39,"pool_name":"cephfs_data","create_time":"2021-02-10 11:47:27.781048","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:47:32.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.034 7fb22a63a700 1 -- 172.21.2.101:0/922326497 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2040186c0 msgr2=0x7fb20401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:32.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.034 7fb22a63a700 1 --2- 172.21.2.101:0/922326497 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2040186c0 0x7fb20401abc0 crc :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:32.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.034 7fb22a63a700 1 -- 172.21.2.101:0/922326497 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2240bb8e0 msgr2=0x7fb2240bbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:32.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.034 7fb22a63a700 1 --2- 172.21.2.101:0/922326497 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2240bb8e0 0x7fb2240bbce0 secure :-1 s=READY pgs=3106 cs=0 l=1 rev1=1 rx=0x7fb21c00b990 tx=0x7fb21c00bd30).stop 2021-02-10T11:47:32.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.035 7fb22a63a700 1 -- 172.21.2.101:0/922326497 shutdown_connections 2021-02-10T11:47:32.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.035 7fb22a63a700 1 --2- 172.21.2.101:0/922326497 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2240bb8e0 0x7fb2240bbce0 unknown :-1 s=CLOSED pgs=3106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:32.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.035 7fb22a63a700 1 --2- 172.21.2.101:0/922326497 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb224195070 0x7fb224195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:32.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.035 7fb22a63a700 1 --2- 172.21.2.101:0/922326497 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2040186c0 0x7fb20401abc0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:32.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.035 7fb22a63a700 1 --2- 172.21.2.101:0/922326497 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb2241942b0 0x7fb2240b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:32.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.035 7fb22a63a700 1 -- 172.21.2.101:0/922326497 >> 172.21.2.101:0/922326497 conn(0x7fb2240b3040 msgr2=0x7fb2240c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:32.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.035 7fb22a63a700 1 -- 172.21.2.101:0/922326497 shutdown_connections 2021-02-10T11:47:32.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.035 7fb22a63a700 1 -- 172.21.2.101:0/922326497 wait complete. 2021-02-10T11:47:32.048 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:47:32.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.475 7f8749bb7700 1 Processor -- start 2021-02-10T11:47:32.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.475 7f8749bb7700 1 -- start start 2021-02-10T11:47:32.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.476 7f8749bb7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f87440d1c80 0x7f87440cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:32.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.476 7f8749bb7700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87440d0180 con 0x7f87440cf590 2021-02-10T11:47:32.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.476 7f8749bb7700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87440fe1d0 con 0x7f87440d1880 2021-02-10T11:47:32.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.476 7f8749bb7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87441b0cc0 con 0x7f87440d1c80 2021-02-10T11:47:32.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.476 7f8743fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f87440cf590 0x7f87440cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34624/0 (socket says 172.21.2.101:34624) 2021-02-10T11:47:32.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.476 7f8743fff700 1 -- 172.21.2.101:0/1887798341 learned_addr learned my addr 172.21.2.101:0/1887798341 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:32.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.476 7f87427fc700 1 -- 172.21.2.101:0/1887798341 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 955351343 0 0) 0x7f873c001500 con 0x7f87440cf590 2021-02-10T11:47:32.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.476 7f87427fc700 1 -- 172.21.2.101:0/1887798341 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8730002fe0 con 0x7f87440cf590 2021-02-10T11:47:32.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.476 7f8742ffd700 1 --2- 172.21.2.101:0/1887798341 >> v2:172.21.2.108:3300/0 conn(0x7f87440d1c80 0x7f87440cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:32.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f87427fc700 1 -- 172.21.2.101:0/1887798341 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1033454971 0 0) 0x7f873c001500 con 0x7f87440cf590 2021-02-10T11:47:32.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f87427fc700 1 -- 172.21.2.101:0/1887798341 >> v2:172.21.2.108:3300/0 conn(0x7f87440d1c80 msgr2=0x7f87440cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:32.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f87427fc700 1 --2- 172.21.2.101:0/1887798341 >> v2:172.21.2.108:3300/0 conn(0x7f87440d1c80 0x7f87440cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:32.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f87427fc700 1 -- 172.21.2.101:0/1887798341 >> v1:172.21.2.108:6789/0 conn(0x7f87440d1880 legacy=0x7f87441b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:32.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f87427fc700 1 -- 172.21.2.101:0/1887798341 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8744109a70 con 0x7f87440cf590 2021-02-10T11:47:32.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f87427fc700 1 -- 172.21.2.101:0/1887798341 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f873c001a00 con 0x7f87440cf590 2021-02-10T11:47:32.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f87427fc700 1 -- 172.21.2.101:0/1887798341 >> v1:172.21.2.101:6789/0 conn(0x7f87440cf590 legacy=0x7f87440cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:32.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f87427fc700 1 --2- 172.21.2.101:0/1887798341 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8730003b10 0x7f8730005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:32.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f87427fc700 1 --2- 172.21.2.101:0/1887798341 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87300067c0 0x7f8730008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:32.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f87427fc700 1 --2- 172.21.2.101:0/1887798341 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8730009580 0x7f873000baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:32.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f87427fc700 1 -- 172.21.2.101:0/1887798341 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873000c2f0 con 0x7f8730009580 2021-02-10T11:47:32.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f87427fc700 1 -- 172.21.2.101:0/1887798341 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873000c510 con 0x7f8730003b10 2021-02-10T11:47:32.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f87427fc700 1 -- 172.21.2.101:0/1887798341 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873000c730 con 0x7f87300067c0 2021-02-10T11:47:32.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f87437fe700 1 --2- 172.21.2.101:0/1887798341 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8730003b10 0x7f8730005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:32.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f8743fff700 1 --2- 172.21.2.101:0/1887798341 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8730009580 0x7f873000baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:32.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f8742ffd700 1 --2- 172.21.2.101:0/1887798341 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87300067c0 0x7f8730008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:32.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f87437fe700 1 -- 172.21.2.101:0/1887798341 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87300067c0 msgr2=0x7f8730008c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:32.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f87437fe700 1 --2- 172.21.2.101:0/1887798341 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87300067c0 0x7f8730008c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:32.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f87437fe700 1 -- 172.21.2.101:0/1887798341 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8730009580 msgr2=0x7f873000baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:32.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f87437fe700 1 --2- 172.21.2.101:0/1887798341 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8730009580 0x7f873000baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:32.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f87437fe700 1 -- 172.21.2.101:0/1887798341 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87300509a0 con 0x7f8730003b10 2021-02-10T11:47:32.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f8742ffd700 1 --2- 172.21.2.101:0/1887798341 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87300067c0 0x7f8730008c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:47:32.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f8743fff700 1 --2- 172.21.2.101:0/1887798341 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8730009580 0x7f873000baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:47:32.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f8743fff700 1 -- 172.21.2.101:0/1887798341 reap_dead start 2021-02-10T11:47:32.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f87437fe700 1 --2- 172.21.2.101:0/1887798341 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8730003b10 0x7f8730005fa0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7f872c006af0 tx=0x7f872c009b40).ready entity=mon.1 client_cookie=44f3ac4d9f9cc43a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:32.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f87427fc700 1 -- 172.21.2.101:0/1887798341 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f872c00c0c0 con 0x7f8730003b10 2021-02-10T11:47:32.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.477 7f87427fc700 1 -- 172.21.2.101:0/1887798341 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c00c440 con 0x7f8730003b10 2021-02-10T11:47:32.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.478 7f87427fc700 1 -- 172.21.2.101:0/1887798341 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f872c006710 con 0x7f8730003b10 2021-02-10T11:47:32.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.478 7f8749bb7700 1 -- 172.21.2.101:0/1887798341 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8730003b10 msgr2=0x7f8730005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:32.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.478 7f8749bb7700 1 --2- 172.21.2.101:0/1887798341 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8730003b10 0x7f8730005fa0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7f872c006af0 tx=0x7f872c009b40).stop 2021-02-10T11:47:32.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.478 7f8749bb7700 1 -- 172.21.2.101:0/1887798341 shutdown_connections 2021-02-10T11:47:32.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.478 7f8749bb7700 1 --2- 172.21.2.101:0/1887798341 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8730003b10 0x7f8730005fa0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:32.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.478 7f8749bb7700 1 -- 172.21.2.101:0/1887798341 >> 172.21.2.101:0/1887798341 conn(0x7f87440ce430 msgr2=0x7f87440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:32.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.478 7f8749bb7700 1 -- 172.21.2.101:0/1887798341 shutdown_connections 2021-02-10T11:47:32.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.478 7f8749bb7700 1 -- 172.21.2.101:0/1887798341 wait complete. 2021-02-10T11:47:32.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.492 7f8749bb7700 1 Processor -- start 2021-02-10T11:47:32.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.493 7f8749bb7700 1 -- start start 2021-02-10T11:47:32.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.493 7f8749bb7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87441a0580 0x7f8744086fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:32.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.493 7f8749bb7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87440b66b0 0x7f87440b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:32.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.493 7f8749bb7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8744082bb0 0x7f8744082fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:32.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.493 7f8749bb7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87440875d0 con 0x7f8744082bb0 2021-02-10T11:47:32.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.493 7f8749bb7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87440835d0 con 0x7f87441a0580 2021-02-10T11:47:32.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.493 7f8749bb7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87440837f0 con 0x7f87440b66b0 2021-02-10T11:47:32.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.493 7f8743fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8744082bb0 0x7f8744082fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:32.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.493 7f8743fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8744082bb0 0x7f8744082fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34142/0 (socket says 172.21.2.101:34142) 2021-02-10T11:47:32.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.493 7f8743fff700 1 -- 172.21.2.101:0/2388172964 learned_addr learned my addr 172.21.2.101:0/2388172964 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:32.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.493 7f87437fe700 1 --2- 172.21.2.101:0/2388172964 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87441a0580 0x7f8744086fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:32.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.493 7f8742ffd700 1 --2- 172.21.2.101:0/2388172964 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87440b66b0 0x7f87440b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:32.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.494 7f8743fff700 1 -- 172.21.2.101:0/2388172964 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87440b66b0 msgr2=0x7f87440b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:32.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.494 7f8743fff700 1 --2- 172.21.2.101:0/2388172964 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87440b66b0 0x7f87440b6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:32.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.494 7f8743fff700 1 -- 172.21.2.101:0/2388172964 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87441a0580 msgr2=0x7f8744086fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:32.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.494 7f8743fff700 1 --2- 172.21.2.101:0/2388172964 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87441a0580 0x7f8744086fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:32.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.494 7f8743fff700 1 -- 172.21.2.101:0/2388172964 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8744083a10 con 0x7f8744082bb0 2021-02-10T11:47:32.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.494 7f8743fff700 1 --2- 172.21.2.101:0/2388172964 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8744082bb0 0x7f8744082fb0 secure :-1 s=READY pgs=3108 cs=0 l=1 rev1=1 rx=0x7f873c001c90 tx=0x7f873c00ac80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:32.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.494 7f873bfff700 1 -- 172.21.2.101:0/2388172964 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f873c007020 con 0x7f8744082bb0 2021-02-10T11:47:32.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.494 7f873bfff700 1 -- 172.21.2.101:0/2388172964 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873c0072e0 con 0x7f8744082bb0 2021-02-10T11:47:32.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.494 7f873bfff700 1 -- 172.21.2.101:0/2388172964 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f873c004720 con 0x7f8744082bb0 2021-02-10T11:47:32.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.494 7f8749bb7700 1 -- 172.21.2.101:0/2388172964 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8744083a10 con 0x7f8744082bb0 2021-02-10T11:47:32.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.494 7f8749bb7700 1 -- 172.21.2.101:0/2388172964 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8744083a10 con 0x7f8744082bb0 2021-02-10T11:47:32.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.495 7f873bfff700 1 -- 172.21.2.101:0/2388172964 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f873c000cc0 con 0x7f8744082bb0 2021-02-10T11:47:32.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.495 7f873bfff700 1 --2- 172.21.2.101:0/2388172964 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87280186d0 0x7f872801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:32.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.495 7f873bfff700 1 -- 172.21.2.101:0/2388172964 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f873c032950 con 0x7f8744082bb0 2021-02-10T11:47:32.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.495 7f87437fe700 1 --2- 172.21.2.101:0/2388172964 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87280186d0 0x7f872801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:32.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.495 7f87437fe700 1 --2- 172.21.2.101:0/2388172964 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87280186d0 0x7f872801abd0 crc :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:32.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.525 7f8749bb7700 1 -- 172.21.2.101:0/2388172964 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87440875d0 con 0x7f8744082bb0 2021-02-10T11:47:32.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.528 7f873bfff700 1 -- 172.21.2.101:0/2388172964 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f873c0072c0 con 0x7f8744082bb0 2021-02-10T11:47:32.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.657 7f8749bb7700 1 -- 172.21.2.101:0/2388172964 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f87440875d0 con 0x7f8744082bb0 2021-02-10T11:47:32.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.661 7f873bfff700 1 -- 172.21.2.101:0/2388172964 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v102) v1 ==== 184+0+0 (secure 0 0 0) 0x7f873c0072c0 con 0x7f8744082bb0 2021-02-10T11:47:32.662 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.0 2021-02-10T11:47:32.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.666 7f8749bb7700 1 -- 172.21.2.101:0/2388172964 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87280186d0 msgr2=0x7f872801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:32.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.666 7f8749bb7700 1 --2- 172.21.2.101:0/2388172964 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87280186d0 0x7f872801abd0 crc :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:32.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.666 7f8749bb7700 1 -- 172.21.2.101:0/2388172964 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8744082bb0 msgr2=0x7f8744082fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:32.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.666 7f8749bb7700 1 --2- 172.21.2.101:0/2388172964 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8744082bb0 0x7f8744082fb0 secure :-1 s=READY pgs=3108 cs=0 l=1 rev1=1 rx=0x7f873c001c90 tx=0x7f873c00ac80).stop 2021-02-10T11:47:32.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.666 7f8749bb7700 1 -- 172.21.2.101:0/2388172964 shutdown_connections 2021-02-10T11:47:32.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.666 7f8749bb7700 1 --2- 172.21.2.101:0/2388172964 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f87280186d0 0x7f872801abd0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:32.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.666 7f8749bb7700 1 --2- 172.21.2.101:0/2388172964 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f87440b66b0 0x7f87440b6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:32.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.666 7f8749bb7700 1 --2- 172.21.2.101:0/2388172964 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8744082bb0 0x7f8744082fb0 unknown :-1 s=CLOSED pgs=3108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:32.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.666 7f8749bb7700 1 --2- 172.21.2.101:0/2388172964 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f87441a0580 0x7f8744086fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:32.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.666 7f8749bb7700 1 -- 172.21.2.101:0/2388172964 >> 172.21.2.101:0/2388172964 conn(0x7f87440afc60 msgr2=0x7f87440b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:32.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.667 7f8749bb7700 1 -- 172.21.2.101:0/2388172964 shutdown_connections 2021-02-10T11:47:32.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:32.667 7f8749bb7700 1 -- 172.21.2.101:0/2388172964 wait complete. 2021-02-10T11:47:32.672 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:47:33.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.102 7faee499e700 1 Processor -- start 2021-02-10T11:47:33.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.102 7faee499e700 1 -- start start 2021-02-10T11:47:33.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.103 7faee499e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7faee00d1c80 0x7faee00cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:33.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.103 7faee499e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faee00cf590 0x7faee00cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:33.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.103 7faee499e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee00fe1d0 con 0x7faee00d1880 2021-02-10T11:47:33.116 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.103 7faee499e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faee01b0cc0 con 0x7faee00d1c80 2021-02-10T11:47:33.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.103 7faee499e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee0109a70 con 0x7faee00cf590 2021-02-10T11:47:33.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.103 7faedde1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7faee00d1c80 0x7faee00cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:33.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.103 7faedde1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7faee00d1c80 0x7faee00cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34150/0 (socket says 172.21.2.101:34150) 2021-02-10T11:47:33.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.103 7faedde1b700 1 -- 172.21.2.101:0/152012796 learned_addr learned my addr 172.21.2.101:0/152012796 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:33.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.103 7faede61c700 1 --1- 172.21.2.101:0/152012796 >> v1:172.21.2.101:6789/0 conn(0x7faee00d1880 0x7faee01b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34642/0 (socket says 172.21.2.101:34642) 2021-02-10T11:47:33.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.103 7faedde1b700 1 -- 172.21.2.101:0/152012796 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faee00cf590 msgr2=0x7faee00cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:33.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.103 7faedde1b700 1 --2- 172.21.2.101:0/152012796 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faee00cf590 0x7faee00cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:33.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.103 7faedde1b700 1 -- 172.21.2.101:0/152012796 >> v1:172.21.2.101:6789/0 conn(0x7faee00d1880 legacy=0x7faee01b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:33.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.104 7faedde1b700 1 -- 172.21.2.101:0/152012796 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee0088070 con 0x7faee00d1c80 2021-02-10T11:47:33.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.104 7faedde1b700 1 --2- 172.21.2.101:0/152012796 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faee00d1c80 0x7faee00cef70 secure :-1 s=READY pgs=3110 cs=0 l=1 rev1=1 rx=0x7faed4006700 tx=0x7faed4009b80).ready entity=mon.0 client_cookie=e71deab51dc43686 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:33.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.104 7faedd61a700 1 -- 172.21.2.101:0/152012796 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faed400c0c0 con 0x7faee00d1c80 2021-02-10T11:47:33.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.104 7faedd61a700 1 -- 172.21.2.101:0/152012796 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed400c440 con 0x7faee00d1c80 2021-02-10T11:47:33.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.104 7faedd61a700 1 -- 172.21.2.101:0/152012796 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faed400c920 con 0x7faee00d1c80 2021-02-10T11:47:33.117 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.104 7faee499e700 1 -- 172.21.2.101:0/152012796 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faee00d1c80 msgr2=0x7faee00cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:33.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.104 7faee499e700 1 --2- 172.21.2.101:0/152012796 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faee00d1c80 0x7faee00cef70 secure :-1 s=READY pgs=3110 cs=0 l=1 rev1=1 rx=0x7faed4006700 tx=0x7faed4009b80).stop 2021-02-10T11:47:33.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.105 7faee499e700 1 -- 172.21.2.101:0/152012796 shutdown_connections 2021-02-10T11:47:33.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.105 7faee499e700 1 --2- 172.21.2.101:0/152012796 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faee00cf590 0x7faee00cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:33.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.105 7faee499e700 1 --2- 172.21.2.101:0/152012796 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faee00d1c80 0x7faee00cef70 unknown :-1 s=CLOSED pgs=3110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:33.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.105 7faee499e700 1 -- 172.21.2.101:0/152012796 >> 172.21.2.101:0/152012796 conn(0x7faee00ce430 msgr2=0x7faee00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:33.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.105 7faee499e700 1 -- 172.21.2.101:0/152012796 shutdown_connections 2021-02-10T11:47:33.118 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.105 7faee499e700 1 -- 172.21.2.101:0/152012796 wait complete. 2021-02-10T11:47:33.119 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.119 7faee499e700 1 Processor -- start 2021-02-10T11:47:33.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.120 7faee499e700 1 -- start start 2021-02-10T11:47:33.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.120 7faee499e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faee01b9470 0x7faee01ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:33.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.120 7faee499e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faee01aecf0 0x7faee00957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:33.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.120 7faee499e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faee01af0f0 0x7faee0095e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:33.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.120 7faee499e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee0096d70 con 0x7faee01b9470 2021-02-10T11:47:33.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.120 7faee499e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee00a7780 con 0x7faee01aecf0 2021-02-10T11:47:33.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.120 7faee499e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee01b9870 con 0x7faee01af0f0 2021-02-10T11:47:33.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.120 7faede61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faee01b9470 0x7faee01ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:33.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.120 7faede61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faee01b9470 0x7faee01ae6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34154/0 (socket says 172.21.2.101:34154) 2021-02-10T11:47:33.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.120 7faede61c700 1 -- 172.21.2.101:0/1389097632 learned_addr learned my addr 172.21.2.101:0/1389097632 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:33.120 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.120 7faede61c700 1 -- 172.21.2.101:0/1389097632 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faee01af0f0 msgr2=0x7faee0095e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:33.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.120 7faede61c700 1 --2- 172.21.2.101:0/1389097632 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faee01af0f0 0x7faee0095e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:33.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.120 7faede61c700 1 -- 172.21.2.101:0/1389097632 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faee01aecf0 msgr2=0x7faee00957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:33.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.120 7faede61c700 1 --2- 172.21.2.101:0/1389097632 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faee01aecf0 0x7faee00957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:33.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.120 7faede61c700 1 -- 172.21.2.101:0/1389097632 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee019e6b0 con 0x7faee01b9470 2021-02-10T11:47:33.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.121 7faede61c700 1 --2- 172.21.2.101:0/1389097632 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faee01b9470 0x7faee01ae6d0 secure :-1 s=READY pgs=3111 cs=0 l=1 rev1=1 rx=0x7faed000a590 tx=0x7faed000a930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:33.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.121 7faeceffd700 1 -- 172.21.2.101:0/1389097632 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faed0000ef0 con 0x7faee01b9470 2021-02-10T11:47:33.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.121 7faeceffd700 1 -- 172.21.2.101:0/1389097632 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed000ad20 con 0x7faee01b9470 2021-02-10T11:47:33.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.121 7faeceffd700 1 -- 172.21.2.101:0/1389097632 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faed0004620 con 0x7faee01b9470 2021-02-10T11:47:33.121 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.121 7faee499e700 1 -- 172.21.2.101:0/1389097632 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee019e6b0 con 0x7faee01b9470 2021-02-10T11:47:33.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.121 7faee499e700 1 -- 172.21.2.101:0/1389097632 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee0096720 con 0x7faee01b9470 2021-02-10T11:47:33.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.122 7faeceffd700 1 -- 172.21.2.101:0/1389097632 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7faed00043a0 con 0x7faee01b9470 2021-02-10T11:47:33.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.122 7faeceffd700 1 --2- 172.21.2.101:0/1389097632 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faec40186c0 0x7faec401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:33.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.122 7faeceffd700 1 -- 172.21.2.101:0/1389097632 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7faed0032550 con 0x7faee01b9470 2021-02-10T11:47:33.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.122 7faedde1b700 1 --2- 172.21.2.101:0/1389097632 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faec40186c0 0x7faec401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:33.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.122 7faedde1b700 1 --2- 172.21.2.101:0/1389097632 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faec40186c0 0x7faec401abc0 crc :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:33.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.152 7faee499e700 1 -- 172.21.2.101:0/1389097632 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee00829a0 con 0x7faee01b9470 2021-02-10T11:47:33.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.155 7faeceffd700 1 -- 172.21.2.101:0/1389097632 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7faed0000ef0 con 0x7faee01b9470 2021-02-10T11:47:33.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.287 7faee499e700 1 -- 172.21.2.101:0/1389097632 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7faee00a7780 con 0x7faee01b9470 2021-02-10T11:47:33.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.290 7faeceffd700 1 -- 172.21.2.101:0/1389097632 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v103) v1 ==== 184+0+0 (secure 0 0 0) 0x7faed0004780 con 0x7faee01b9470 2021-02-10T11:47:33.291 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.1 2021-02-10T11:47:33.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.295 7faee499e700 1 -- 172.21.2.101:0/1389097632 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faec40186c0 msgr2=0x7faec401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:33.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.295 7faee499e700 1 --2- 172.21.2.101:0/1389097632 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faec40186c0 0x7faec401abc0 crc :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:33.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.295 7faee499e700 1 -- 172.21.2.101:0/1389097632 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faee01b9470 msgr2=0x7faee01ae6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:33.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.295 7faee499e700 1 --2- 172.21.2.101:0/1389097632 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faee01b9470 0x7faee01ae6d0 secure :-1 s=READY pgs=3111 cs=0 l=1 rev1=1 rx=0x7faed000a590 tx=0x7faed000a930).stop 2021-02-10T11:47:33.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.295 7faee499e700 1 -- 172.21.2.101:0/1389097632 shutdown_connections 2021-02-10T11:47:33.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.295 7faee499e700 1 --2- 172.21.2.101:0/1389097632 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faec40186c0 0x7faec401abc0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:33.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.295 7faee499e700 1 --2- 172.21.2.101:0/1389097632 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faee01af0f0 0x7faee0095e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:33.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.295 7faee499e700 1 --2- 172.21.2.101:0/1389097632 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faee01aecf0 0x7faee00957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:33.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.295 7faee499e700 1 --2- 172.21.2.101:0/1389097632 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faee01b9470 0x7faee01ae6d0 unknown :-1 s=CLOSED pgs=3111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:33.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.296 7faee499e700 1 -- 172.21.2.101:0/1389097632 >> 172.21.2.101:0/1389097632 conn(0x7faee00845b0 msgr2=0x7faee018c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:33.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.296 7faee499e700 1 -- 172.21.2.101:0/1389097632 shutdown_connections 2021-02-10T11:47:33.293 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.296 7faee499e700 1 -- 172.21.2.101:0/1389097632 wait complete. 2021-02-10T11:47:33.301 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:47:33.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.727 7f825dfb4700 1 Processor -- start 2021-02-10T11:47:33.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.727 7f825dfb4700 1 -- start start 2021-02-10T11:47:33.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.728 7f825dfb4700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f82580d1880 0x7f82581b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:33.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.728 7f825dfb4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f82581b0af0 0x7f8258108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:33.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.728 7f825dfb4700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82580fe1d0 con 0x7f82580d1c80 2021-02-10T11:47:33.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.728 7f825dfb4700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8258109a70 con 0x7f82581b0af0 2021-02-10T11:47:33.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.728 7f825dfb4700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8258109760 con 0x7f82580d1880 2021-02-10T11:47:33.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.728 7f8256ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f82581b0af0 0x7f8258108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:33.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.728 7f8256ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f82581b0af0 0x7f8258108950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34162/0 (socket says 172.21.2.101:34162) 2021-02-10T11:47:33.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.728 7f8256ffd700 1 -- 172.21.2.101:0/2745100984 learned_addr learned my addr 172.21.2.101:0/2745100984 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:33.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.728 7f8256ffd700 1 -- 172.21.2.101:0/2745100984 >> v2:172.21.2.108:3300/0 conn(0x7f82580d1880 msgr2=0x7f82581b04d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:33.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.728 7f8256ffd700 1 --2- 172.21.2.101:0/2745100984 >> v2:172.21.2.108:3300/0 conn(0x7f82580d1880 0x7f82581b04d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:33.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.728 7f8256ffd700 1 -- 172.21.2.101:0/2745100984 >> v1:172.21.2.101:6789/0 conn(0x7f82580d1c80 legacy=0x7f8258108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:33.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.728 7f8256ffd700 1 -- 172.21.2.101:0/2745100984 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8258088070 con 0x7f82581b0af0 2021-02-10T11:47:33.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.728 7f8256ffd700 1 --2- 172.21.2.101:0/2745100984 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82581b0af0 0x7f8258108950 secure :-1 s=READY pgs=3113 cs=0 l=1 rev1=1 rx=0x7f8248006700 tx=0x7f8248009b00).ready entity=mon.0 client_cookie=5a7331ef17db611c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:33.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.728 7f82567fc700 1 -- 172.21.2.101:0/2745100984 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f824800c0c0 con 0x7f82581b0af0 2021-02-10T11:47:33.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.729 7f82567fc700 1 -- 172.21.2.101:0/2745100984 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824800c400 con 0x7f82581b0af0 2021-02-10T11:47:33.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.729 7f82567fc700 1 -- 172.21.2.101:0/2745100984 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f824800c8e0 con 0x7f82581b0af0 2021-02-10T11:47:33.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.729 7f825dfb4700 1 -- 172.21.2.101:0/2745100984 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82581b0af0 msgr2=0x7f8258108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:33.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.729 7f825dfb4700 1 --2- 172.21.2.101:0/2745100984 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82581b0af0 0x7f8258108950 secure :-1 s=READY pgs=3113 cs=0 l=1 rev1=1 rx=0x7f8248006700 tx=0x7f8248009b00).stop 2021-02-10T11:47:33.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.729 7f825dfb4700 1 -- 172.21.2.101:0/2745100984 shutdown_connections 2021-02-10T11:47:33.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.729 7f825dfb4700 1 --2- 172.21.2.101:0/2745100984 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f82581b0af0 0x7f8258108950 unknown :-1 s=CLOSED pgs=3113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:33.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.729 7f825dfb4700 1 --2- 172.21.2.101:0/2745100984 >> v2:172.21.2.108:3300/0 conn(0x7f82580d1880 0x7f82581b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:33.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.729 7f825dfb4700 1 -- 172.21.2.101:0/2745100984 >> 172.21.2.101:0/2745100984 conn(0x7f82580ce430 msgr2=0x7f82580d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:33.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.729 7f825dfb4700 1 -- 172.21.2.101:0/2745100984 shutdown_connections 2021-02-10T11:47:33.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.729 7f825dfb4700 1 -- 172.21.2.101:0/2745100984 wait complete. 2021-02-10T11:47:33.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.743 7f825dfb4700 1 Processor -- start 2021-02-10T11:47:33.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.744 7f825dfb4700 1 -- start start 2021-02-10T11:47:33.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.744 7f825dfb4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8258086520 0x7f82581a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:33.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.744 7f825dfb4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8258086920 0x7f8258187e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:33.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.744 7f825dfb4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f82581a22d0 0x7f82581884a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:33.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.744 7f825dfb4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8258099600 con 0x7f8258086920 2021-02-10T11:47:33.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.744 7f825dfb4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82580999f0 con 0x7f8258086520 2021-02-10T11:47:33.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.744 7f825dfb4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8258188ac0 con 0x7f82581a22d0 2021-02-10T11:47:33.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.744 7f8256ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8258086920 0x7f8258187e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:33.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.744 7f8256ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8258086920 0x7f8258187e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34168/0 (socket says 172.21.2.101:34168) 2021-02-10T11:47:33.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.744 7f8256ffd700 1 -- 172.21.2.101:0/443255205 learned_addr learned my addr 172.21.2.101:0/443255205 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:33.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.744 7f8256ffd700 1 -- 172.21.2.101:0/443255205 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f82581a22d0 msgr2=0x7f82581884a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:47:33.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.744 7f8256ffd700 1 --2- 172.21.2.101:0/443255205 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f82581a22d0 0x7f82581884a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:33.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.744 7f8256ffd700 1 -- 172.21.2.101:0/443255205 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8258086520 msgr2=0x7f82581a1cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:47:33.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.744 7f8256ffd700 1 --2- 172.21.2.101:0/443255205 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8258086520 0x7f82581a1cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:33.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.744 7f8256ffd700 1 -- 172.21.2.101:0/443255205 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8258188ce0 con 0x7f8258086920 2021-02-10T11:47:33.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.745 7f8256ffd700 1 --2- 172.21.2.101:0/443255205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8258086920 0x7f8258187e80 secure :-1 s=READY pgs=3114 cs=0 l=1 rev1=1 rx=0x7f824800b660 tx=0x7f824800ba00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:33.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.745 7f824ffff700 1 -- 172.21.2.101:0/443255205 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8248009440 con 0x7f8258086920 2021-02-10T11:47:33.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.745 7f824ffff700 1 -- 172.21.2.101:0/443255205 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8248006390 con 0x7f8258086920 2021-02-10T11:47:33.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.745 7f825dfb4700 1 -- 172.21.2.101:0/443255205 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8258188ce0 con 0x7f8258086920 2021-02-10T11:47:33.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.745 7f824ffff700 1 -- 172.21.2.101:0/443255205 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f82480052e0 con 0x7f8258086920 2021-02-10T11:47:33.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.745 7f825dfb4700 1 -- 172.21.2.101:0/443255205 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8258189270 con 0x7f8258086920 2021-02-10T11:47:33.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.745 7f824ffff700 1 -- 172.21.2.101:0/443255205 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f82480093e0 con 0x7f8258086920 2021-02-10T11:47:33.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.746 7f824ffff700 1 --2- 172.21.2.101:0/443255205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f823c0186d0 0x7f823c01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:33.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.746 7f824ffff700 1 -- 172.21.2.101:0/443255205 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f82480093e0 con 0x7f8258086920 2021-02-10T11:47:33.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.746 7f82577fe700 1 --2- 172.21.2.101:0/443255205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f823c0186d0 0x7f823c01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:33.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.746 7f82577fe700 1 --2- 172.21.2.101:0/443255205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f823c0186d0 0x7f823c01abd0 crc :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:33.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.778 7f825dfb4700 1 -- 172.21.2.101:0/443255205 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82580fe1d0 con 0x7f8258086920 2021-02-10T11:47:33.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.780 7f824ffff700 1 -- 172.21.2.101:0/443255205 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f824801f520 con 0x7f8258086920 2021-02-10T11:47:33.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.909 7f825dfb4700 1 -- 172.21.2.101:0/443255205 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f82580fe1d0 con 0x7f8258086920 2021-02-10T11:47:33.909 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.912 7f824ffff700 1 -- 172.21.2.101:0/443255205 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v104) v1 ==== 184+0+0 (secure 0 0 0) 0x7f82480093e0 con 0x7f8258086920 2021-02-10T11:47:33.909 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.2 2021-02-10T11:47:33.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.914 7f825dfb4700 1 -- 172.21.2.101:0/443255205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f823c0186d0 msgr2=0x7f823c01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:33.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.914 7f825dfb4700 1 --2- 172.21.2.101:0/443255205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f823c0186d0 0x7f823c01abd0 crc :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:33.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.914 7f825dfb4700 1 -- 172.21.2.101:0/443255205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8258086920 msgr2=0x7f8258187e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:33.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.914 7f825dfb4700 1 --2- 172.21.2.101:0/443255205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8258086920 0x7f8258187e80 secure :-1 s=READY pgs=3114 cs=0 l=1 rev1=1 rx=0x7f824800b660 tx=0x7f824800ba00).stop 2021-02-10T11:47:33.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.914 7f825dfb4700 1 -- 172.21.2.101:0/443255205 shutdown_connections 2021-02-10T11:47:33.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.914 7f825dfb4700 1 --2- 172.21.2.101:0/443255205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f823c0186d0 0x7f823c01abd0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:33.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.914 7f825dfb4700 1 --2- 172.21.2.101:0/443255205 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f82581a22d0 0x7f82581884a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:33.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.914 7f825dfb4700 1 --2- 172.21.2.101:0/443255205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8258086920 0x7f8258187e80 unknown :-1 s=CLOSED pgs=3114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:33.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.914 7f825dfb4700 1 --2- 172.21.2.101:0/443255205 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8258086520 0x7f82581a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:33.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.914 7f825dfb4700 1 -- 172.21.2.101:0/443255205 >> 172.21.2.101:0/443255205 conn(0x7f825819f880 msgr2=0x7f825808f890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:33.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.914 7f825dfb4700 1 -- 172.21.2.101:0/443255205 shutdown_connections 2021-02-10T11:47:33.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:33.914 7f825dfb4700 1 -- 172.21.2.101:0/443255205 wait complete. 2021-02-10T11:47:33.919 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:47:34.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.346 7ffb9282e700 1 Processor -- start 2021-02-10T11:47:34.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.346 7ffb9282e700 1 -- start start 2021-02-10T11:47:34.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.347 7ffb9282e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ffb8c0d1c80 0x7ffb8c0cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:34.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.347 7ffb9282e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffb8c0cf590 0x7ffb8c0cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:34.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.347 7ffb9282e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8c0fe1d0 con 0x7ffb8c0d1880 2021-02-10T11:47:34.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.347 7ffb9282e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8c1b0cc0 con 0x7ffb8c0d1c80 2021-02-10T11:47:34.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.347 7ffb9282e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c109a70 con 0x7ffb8c0cf590 2021-02-10T11:47:34.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.347 7ffb8b7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ffb8c0d1c80 0x7ffb8c0cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:34.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.347 7ffb8bfff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ffb8c0d1880 0x7ffb8c1b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34668/0 (socket says 172.21.2.101:34668) 2021-02-10T11:47:34.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.347 7ffb8bfff700 1 -- 172.21.2.101:0/268356776 learned_addr learned my addr 172.21.2.101:0/268356776 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:34.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.347 7ffb90e4b700 1 --2- 172.21.2.101:0/268356776 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffb8c0cf590 0x7ffb8c0cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:34.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.347 7ffb8b7fe700 1 -- 172.21.2.101:0/268356776 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffb8c0cf590 msgr2=0x7ffb8c0cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:34.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.347 7ffb8b7fe700 1 --2- 172.21.2.101:0/268356776 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffb8c0cf590 0x7ffb8c0cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:34.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.347 7ffb8b7fe700 1 -- 172.21.2.101:0/268356776 >> v1:172.21.2.101:6789/0 conn(0x7ffb8c0d1880 legacy=0x7ffb8c1b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:34.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.347 7ffb8b7fe700 1 -- 172.21.2.101:0/268356776 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c088070 con 0x7ffb8c0d1c80 2021-02-10T11:47:34.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.347 7ffb8affd700 1 -- 172.21.2.101:0/268356776 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238522458 0 0) 0x7ffb7c000ce0 con 0x7ffb8c0d1880 2021-02-10T11:47:34.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.347 7ffb8b7fe700 1 --2- 172.21.2.101:0/268356776 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffb8c0d1c80 0x7ffb8c0cef70 secure :-1 s=READY pgs=3116 cs=0 l=1 rev1=1 rx=0x7ffb80006650 tx=0x7ffb80009a00).ready entity=mon.0 client_cookie=39fc22c7c8af2497 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:34.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.348 7ffb8affd700 1 -- 172.21.2.101:0/268356776 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffb80009df0 con 0x7ffb8c0d1c80 2021-02-10T11:47:34.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.348 7ffb8affd700 1 -- 172.21.2.101:0/268356776 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb800009b0 con 0x7ffb8c0d1c80 2021-02-10T11:47:34.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.348 7ffb8affd700 1 -- 172.21.2.101:0/268356776 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffb8000c2b0 con 0x7ffb8c0d1c80 2021-02-10T11:47:34.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.348 7ffb9282e700 1 -- 172.21.2.101:0/268356776 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffb8c0d1c80 msgr2=0x7ffb8c0cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:34.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.348 7ffb9282e700 1 --2- 172.21.2.101:0/268356776 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffb8c0d1c80 0x7ffb8c0cef70 secure :-1 s=READY pgs=3116 cs=0 l=1 rev1=1 rx=0x7ffb80006650 tx=0x7ffb80009a00).stop 2021-02-10T11:47:34.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.348 7ffb9282e700 1 -- 172.21.2.101:0/268356776 shutdown_connections 2021-02-10T11:47:34.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.348 7ffb9282e700 1 --2- 172.21.2.101:0/268356776 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffb8c0cf590 0x7ffb8c0cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:34.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.348 7ffb9282e700 1 --2- 172.21.2.101:0/268356776 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffb8c0d1c80 0x7ffb8c0cef70 unknown :-1 s=CLOSED pgs=3116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:34.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.348 7ffb9282e700 1 -- 172.21.2.101:0/268356776 >> 172.21.2.101:0/268356776 conn(0x7ffb8c0ce430 msgr2=0x7ffb8c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:34.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.348 7ffb9282e700 1 -- 172.21.2.101:0/268356776 shutdown_connections 2021-02-10T11:47:34.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.348 7ffb9282e700 1 -- 172.21.2.101:0/268356776 wait complete. 2021-02-10T11:47:34.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.363 7ffb9282e700 1 Processor -- start 2021-02-10T11:47:34.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.363 7ffb9282e700 1 -- start start 2021-02-10T11:47:34.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.363 7ffb9282e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffb8c095600 0x7ffb8c095a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:34.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.363 7ffb9282e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffb8c096020 0x7ffb8c096420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:34.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.363 7ffb9282e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffb8c092a10 0x7ffb8c092e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:34.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.363 7ffb9282e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c094100 con 0x7ffb8c096020 2021-02-10T11:47:34.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.363 7ffb9282e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c19e9e0 con 0x7ffb8c092a10 2021-02-10T11:47:34.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.363 7ffb9282e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c093430 con 0x7ffb8c095600 2021-02-10T11:47:34.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.363 7ffb8b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffb8c096020 0x7ffb8c096420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:34.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.363 7ffb8b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffb8c096020 0x7ffb8c096420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34180/0 (socket says 172.21.2.101:34180) 2021-02-10T11:47:34.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.363 7ffb8b7fe700 1 -- 172.21.2.101:0/3562822331 learned_addr learned my addr 172.21.2.101:0/3562822331 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:34.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.363 7ffb8bfff700 1 --2- 172.21.2.101:0/3562822331 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffb8c095600 0x7ffb8c095a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:34.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.364 7ffb90e4b700 1 --2- 172.21.2.101:0/3562822331 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffb8c092a10 0x7ffb8c092e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:34.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.364 7ffb8b7fe700 1 -- 172.21.2.101:0/3562822331 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffb8c095600 msgr2=0x7ffb8c095a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:34.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.364 7ffb8b7fe700 1 --2- 172.21.2.101:0/3562822331 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffb8c095600 0x7ffb8c095a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:34.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.364 7ffb8b7fe700 1 -- 172.21.2.101:0/3562822331 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffb8c092a10 msgr2=0x7ffb8c092e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:34.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.364 7ffb8b7fe700 1 --2- 172.21.2.101:0/3562822331 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffb8c092a10 0x7ffb8c092e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:34.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.364 7ffb8b7fe700 1 -- 172.21.2.101:0/3562822331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c093650 con 0x7ffb8c096020 2021-02-10T11:47:34.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.364 7ffb90e4b700 1 --2- 172.21.2.101:0/3562822331 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffb8c092a10 0x7ffb8c092e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:47:34.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.364 7ffb8b7fe700 1 --2- 172.21.2.101:0/3562822331 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffb8c096020 0x7ffb8c096420 secure :-1 s=READY pgs=3117 cs=0 l=1 rev1=1 rx=0x7ffb800096a0 tx=0x7ffb8000b9b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:34.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.364 7ffb88ff9700 1 -- 172.21.2.101:0/3562822331 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffb800093a0 con 0x7ffb8c096020 2021-02-10T11:47:34.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.364 7ffb88ff9700 1 -- 172.21.2.101:0/3562822331 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb800067f0 con 0x7ffb8c096020 2021-02-10T11:47:34.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.364 7ffb9282e700 1 -- 172.21.2.101:0/3562822331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8c093650 con 0x7ffb8c096020 2021-02-10T11:47:34.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.364 7ffb9282e700 1 -- 172.21.2.101:0/3562822331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8c093be0 con 0x7ffb8c096020 2021-02-10T11:47:34.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.364 7ffb88ff9700 1 -- 172.21.2.101:0/3562822331 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffb800022b0 con 0x7ffb8c096020 2021-02-10T11:47:34.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.365 7ffb88ff9700 1 -- 172.21.2.101:0/3562822331 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ffb80000c90 con 0x7ffb8c096020 2021-02-10T11:47:34.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.365 7ffb88ff9700 1 --2- 172.21.2.101:0/3562822331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffb6c018730 0x7ffb6c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:34.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.365 7ffb88ff9700 1 -- 172.21.2.101:0/3562822331 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ffb800327b0 con 0x7ffb8c096020 2021-02-10T11:47:34.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.365 7ffb8bfff700 1 --2- 172.21.2.101:0/3562822331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffb6c018730 0x7ffb6c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:34.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.365 7ffb8bfff700 1 --2- 172.21.2.101:0/3562822331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffb6c018730 0x7ffb6c01abc0 crc :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:34.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.396 7ffb9282e700 1 -- 172.21.2.101:0/3562822331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8c082980 con 0x7ffb8c096020 2021-02-10T11:47:34.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.399 7ffb88ff9700 1 -- 172.21.2.101:0/3562822331 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ffb800093a0 con 0x7ffb8c096020 2021-02-10T11:47:34.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.528 7ffb9282e700 1 -- 172.21.2.101:0/3562822331 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ffb8c094080 con 0x7ffb8c096020 2021-02-10T11:47:34.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.533 7ffb88ff9700 1 -- 172.21.2.101:0/3562822331 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v105) v1 ==== 184+0+0 (secure 0 0 0) 0x7ffb80000c90 con 0x7ffb8c096020 2021-02-10T11:47:34.532 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.3 2021-02-10T11:47:34.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.537 7ffb9282e700 1 -- 172.21.2.101:0/3562822331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffb6c018730 msgr2=0x7ffb6c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:34.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.537 7ffb9282e700 1 --2- 172.21.2.101:0/3562822331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffb6c018730 0x7ffb6c01abc0 crc :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:34.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.537 7ffb9282e700 1 -- 172.21.2.101:0/3562822331 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffb8c096020 msgr2=0x7ffb8c096420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:34.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.537 7ffb9282e700 1 --2- 172.21.2.101:0/3562822331 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffb8c096020 0x7ffb8c096420 secure :-1 s=READY pgs=3117 cs=0 l=1 rev1=1 rx=0x7ffb800096a0 tx=0x7ffb8000b9b0).stop 2021-02-10T11:47:34.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.537 7ffb9282e700 1 -- 172.21.2.101:0/3562822331 shutdown_connections 2021-02-10T11:47:34.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.537 7ffb9282e700 1 --2- 172.21.2.101:0/3562822331 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffb8c092a10 0x7ffb8c092e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:34.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.537 7ffb9282e700 1 --2- 172.21.2.101:0/3562822331 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffb8c096020 0x7ffb8c096420 unknown :-1 s=CLOSED pgs=3117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:34.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.538 7ffb9282e700 1 --2- 172.21.2.101:0/3562822331 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffb6c018730 0x7ffb6c01abc0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:34.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.538 7ffb9282e700 1 --2- 172.21.2.101:0/3562822331 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffb8c095600 0x7ffb8c095a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:34.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.538 7ffb9282e700 1 -- 172.21.2.101:0/3562822331 >> 172.21.2.101:0/3562822331 conn(0x7ffb8c1a6190 msgr2=0x7ffb8c0949b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:34.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.538 7ffb9282e700 1 -- 172.21.2.101:0/3562822331 shutdown_connections 2021-02-10T11:47:34.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.538 7ffb9282e700 1 -- 172.21.2.101:0/3562822331 wait complete. 2021-02-10T11:47:34.543 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:47:34.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.959 7f8b06715700 1 Processor -- start 2021-02-10T11:47:34.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.959 7f8b06715700 1 -- start start 2021-02-10T11:47:34.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.960 7f8b06715700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8b000d1870 0x7f8b001b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:34.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.960 7f8b06715700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8b001b0a10 0x7f8b000cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:34.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.960 7f8b06715700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8b000d1c70 0x7f8b000cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:34.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.960 7f8b06715700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b000e5ed0 con 0x7f8b000d1870 2021-02-10T11:47:34.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.960 7f8b06715700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b000f0b10 con 0x7f8b001b0a10 2021-02-10T11:47:34.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.960 7f8b06715700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b000cff70 con 0x7f8b000d1c70 2021-02-10T11:47:34.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.960 7f8afffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8b000d1870 0x7f8b001b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:34.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.960 7f8afffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8b000d1870 0x7f8b001b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34188/0 (socket says 172.21.2.101:34188) 2021-02-10T11:47:34.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.960 7f8afffff700 1 -- 172.21.2.101:0/2372923600 learned_addr learned my addr 172.21.2.101:0/2372923600 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:34.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.960 7f8afffff700 1 -- 172.21.2.101:0/2372923600 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8b000d1c70 msgr2=0x7f8b000cf950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:34.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.960 7f8afffff700 1 --2- 172.21.2.101:0/2372923600 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8b000d1c70 0x7f8b000cf950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:34.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.960 7f8afffff700 1 -- 172.21.2.101:0/2372923600 >> v2:172.21.2.108:3300/0 conn(0x7f8b001b0a10 msgr2=0x7f8b000cf330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:34.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.960 7f8afffff700 1 --2- 172.21.2.101:0/2372923600 >> v2:172.21.2.108:3300/0 conn(0x7f8b001b0a10 0x7f8b000cf330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:34.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.960 7f8afffff700 1 -- 172.21.2.101:0/2372923600 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b000d0150 con 0x7f8b000d1870 2021-02-10T11:47:34.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.960 7f8afffff700 1 --2- 172.21.2.101:0/2372923600 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8b000d1870 0x7f8b001b03f0 secure :-1 s=READY pgs=3118 cs=0 l=1 rev1=1 rx=0x7f8af0006a80 tx=0x7f8af0009f00).ready entity=mon.0 client_cookie=241f8dc616abf596 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:34.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.960 7f8afeffd700 1 -- 172.21.2.101:0/2372923600 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8af000b5c0 con 0x7f8b000d1870 2021-02-10T11:47:34.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.961 7f8afeffd700 1 -- 172.21.2.101:0/2372923600 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af000b8c0 con 0x7f8b000d1870 2021-02-10T11:47:34.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.961 7f8afeffd700 1 -- 172.21.2.101:0/2372923600 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8af000bd20 con 0x7f8b000d1870 2021-02-10T11:47:34.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.961 7f8b06715700 1 -- 172.21.2.101:0/2372923600 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8b000d1870 msgr2=0x7f8b001b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:34.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.961 7f8b06715700 1 --2- 172.21.2.101:0/2372923600 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8b000d1870 0x7f8b001b03f0 secure :-1 s=READY pgs=3118 cs=0 l=1 rev1=1 rx=0x7f8af0006a80 tx=0x7f8af0009f00).stop 2021-02-10T11:47:34.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.961 7f8b06715700 1 -- 172.21.2.101:0/2372923600 shutdown_connections 2021-02-10T11:47:34.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.961 7f8b06715700 1 --2- 172.21.2.101:0/2372923600 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8b000d1c70 0x7f8b000cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:34.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.961 7f8b06715700 1 --2- 172.21.2.101:0/2372923600 >> v2:172.21.2.108:3300/0 conn(0x7f8b001b0a10 0x7f8b000cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:34.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.961 7f8b06715700 1 --2- 172.21.2.101:0/2372923600 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8b000d1870 0x7f8b001b03f0 unknown :-1 s=CLOSED pgs=3118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:34.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.961 7f8b06715700 1 -- 172.21.2.101:0/2372923600 >> 172.21.2.101:0/2372923600 conn(0x7f8b000ce450 msgr2=0x7f8b000d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:34.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.961 7f8b06715700 1 -- 172.21.2.101:0/2372923600 shutdown_connections 2021-02-10T11:47:34.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.961 7f8b06715700 1 -- 172.21.2.101:0/2372923600 wait complete. 2021-02-10T11:47:34.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.976 7f8b06715700 1 Processor -- start 2021-02-10T11:47:34.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.976 7f8b06715700 1 -- start start 2021-02-10T11:47:34.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.976 7f8b06715700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8b001b7cf0 0x7f8b0019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:34.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.976 7f8b06715700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8b001b80f0 0x7f8b001b84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:34.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.976 7f8b06715700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8b0019da20 0x7f8b0019de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:34.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.976 7f8b06715700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b000a77c0 con 0x7f8b0019da20 2021-02-10T11:47:34.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.976 7f8b06715700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0019d4a0 con 0x7f8b001b7cf0 2021-02-10T11:47:34.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.976 7f8b06715700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b000954f0 con 0x7f8b001b80f0 2021-02-10T11:47:34.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.976 7f8b04d32700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8b0019da20 0x7f8b0019de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:34.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.976 7f8b04d32700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8b0019da20 0x7f8b0019de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34196/0 (socket says 172.21.2.101:34196) 2021-02-10T11:47:34.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.976 7f8b04d32700 1 -- 172.21.2.101:0/1741582408 learned_addr learned my addr 172.21.2.101:0/1741582408 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:34.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.977 7f8b04d32700 1 -- 172.21.2.101:0/1741582408 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8b001b80f0 msgr2=0x7f8b001b84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:34.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.977 7f8b04d32700 1 --2- 172.21.2.101:0/1741582408 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8b001b80f0 0x7f8b001b84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:34.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.977 7f8b04d32700 1 -- 172.21.2.101:0/1741582408 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8b001b7cf0 msgr2=0x7f8b0019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:34.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.977 7f8b04d32700 1 --2- 172.21.2.101:0/1741582408 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8b001b7cf0 0x7f8b0019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:34.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.977 7f8b04d32700 1 -- 172.21.2.101:0/1741582408 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b001b8d60 con 0x7f8b0019da20 2021-02-10T11:47:34.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.977 7f8b04d32700 1 --2- 172.21.2.101:0/1741582408 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8b0019da20 0x7f8b0019de20 secure :-1 s=READY pgs=3119 cs=0 l=1 rev1=1 rx=0x7f8af800b5c0 tx=0x7f8af800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:34.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.977 7f8afcff9700 1 -- 172.21.2.101:0/1741582408 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8af800bd50 con 0x7f8b0019da20 2021-02-10T11:47:34.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.977 7f8afcff9700 1 -- 172.21.2.101:0/1741582408 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af8006030 con 0x7f8b0019da20 2021-02-10T11:47:34.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.977 7f8afcff9700 1 -- 172.21.2.101:0/1741582408 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8af80054d0 con 0x7f8b0019da20 2021-02-10T11:47:34.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.977 7f8b06715700 1 -- 172.21.2.101:0/1741582408 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b001b8d60 con 0x7f8b0019da20 2021-02-10T11:47:34.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.977 7f8b06715700 1 -- 172.21.2.101:0/1741582408 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b001b8d60 con 0x7f8b0019da20 2021-02-10T11:47:34.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.978 7f8afcff9700 1 -- 172.21.2.101:0/1741582408 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8af8006210 con 0x7f8b0019da20 2021-02-10T11:47:34.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.978 7f8afcff9700 1 --2- 172.21.2.101:0/1741582408 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ae4018730 0x7f8ae401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:34.977 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.978 7f8afcff9700 1 -- 172.21.2.101:0/1741582408 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8af8032680 con 0x7f8b0019da20 2021-02-10T11:47:34.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.978 7f8afffff700 1 --2- 172.21.2.101:0/1741582408 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ae4018730 0x7f8ae401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:34.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:34.978 7f8afffff700 1 --2- 172.21.2.101:0/1741582408 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ae4018730 0x7f8ae401abc0 crc :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:35.006 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:35.009 7f8b06715700 1 -- 172.21.2.101:0/1741582408 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b000ebc00 con 0x7f8b0019da20 2021-02-10T11:47:35.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:35.012 7f8afcff9700 1 -- 172.21.2.101:0/1741582408 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8af8006b80 con 0x7f8b0019da20 2021-02-10T11:47:35.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:35.139 7f8b06715700 1 -- 172.21.2.101:0/1741582408 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b000f0b10 con 0x7f8b0019da20 2021-02-10T11:47:35.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:35.139 7f8afcff9700 1 -- 172.21.2.101:0/1741582408 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+2108 (secure 0 0 0) 0x7f8af800bd50 con 0x7f8b0019da20 2021-02-10T11:47:35.137 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 156 2021-02-10T11:47:35.137 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:47:35.137 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":156,"default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":13128,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/122652046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":122652046},{"type":"v1","addr":"172.21.2.101:6835","nonce":122652046}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":152}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:47:28.786578","modified":"2021-02-10 11:47:30.795250","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":13125},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13125":{"gid":13125,"name":"b","rank":0,"incarnation":153,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2314483030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2314483030},{"type":"v1","addr":"172.21.2.108:6833","nonce":2314483030}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[39],"metadata_pool":38,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":19}]} 2021-02-10T11:47:35.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:35.141 7f8b06715700 1 -- 172.21.2.101:0/1741582408 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ae4018730 msgr2=0x7f8ae401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:35.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:35.141 7f8b06715700 1 --2- 172.21.2.101:0/1741582408 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ae4018730 0x7f8ae401abc0 crc :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:35.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:35.141 7f8b06715700 1 -- 172.21.2.101:0/1741582408 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8b0019da20 msgr2=0x7f8b0019de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:35.138 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:35.141 7f8b06715700 1 --2- 172.21.2.101:0/1741582408 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8b0019da20 0x7f8b0019de20 secure :-1 s=READY pgs=3119 cs=0 l=1 rev1=1 rx=0x7f8af800b5c0 tx=0x7f8af800b960).stop 2021-02-10T11:47:35.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:35.142 7f8b06715700 1 -- 172.21.2.101:0/1741582408 shutdown_connections 2021-02-10T11:47:35.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:35.142 7f8b06715700 1 --2- 172.21.2.101:0/1741582408 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8ae4018730 0x7f8ae401abc0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:35.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:35.142 7f8b06715700 1 --2- 172.21.2.101:0/1741582408 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8b001b80f0 0x7f8b001b84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:35.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:35.142 7f8b06715700 1 --2- 172.21.2.101:0/1741582408 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8b0019da20 0x7f8b0019de20 unknown :-1 s=CLOSED pgs=3119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:35.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:35.142 7f8b06715700 1 --2- 172.21.2.101:0/1741582408 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8b001b7cf0 0x7f8b0019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:35.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:35.142 7f8b06715700 1 -- 172.21.2.101:0/1741582408 >> 172.21.2.101:0/1741582408 conn(0x7f8b000ed810 msgr2=0x7f8b00094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:35.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:35.142 7f8b06715700 1 -- 172.21.2.101:0/1741582408 shutdown_connections 2021-02-10T11:47:35.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:35.142 7f8b06715700 1 -- 172.21.2.101:0/1741582408 wait complete. 2021-02-10T11:47:35.349 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 156, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:47:28.786578', 'modified': '2021-02-10 11:47:30.795250', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 13125}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_13125': {'gid': 13125, 'name': 'b', 'rank': 0, 'incarnation': 153, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/2314483030', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 2314483030}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 2314483030}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [39], 'metadata_pool': 38, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:47:35.349 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:47:35.349 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:47:35.748 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:47:35.749 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:47:35.749 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:47:35.749 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 13125, 2021-02-10T11:47:35.749 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:47:35.749 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:47:35.749 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 6.964211401, 2021-02-10T11:47:35.749 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 156, 2021-02-10T11:47:35.749 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 183, 2021-02-10T11:47:35.749 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 183, 2021-02-10T11:47:35.749 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 12.612169321 2021-02-10T11:47:35.749 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:47:35.753 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 13125, "want_state": "up:active", "state": "up:active", "rank_uptime": 6.964211401, "mdsmap_epoch": 156, "osdmap_epoch": 183, "osdmap_epoch_barrier": 183, "uptime": 12.612169321 } 2021-02-10T11:47:35.753 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:47:36.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.170 7ff6da429700 1 Processor -- start 2021-02-10T11:47:36.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.170 7ff6da429700 1 -- start start 2021-02-10T11:47:36.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.171 7ff6da429700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff6d40d0340 0x7ff6d40ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:36.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.171 7ff6da429700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6d40d1c70 0x7ff6d40efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:36.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.171 7ff6da429700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6d40f0b10 con 0x7ff6d40d1870 2021-02-10T11:47:36.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.171 7ff6da429700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6d40e5ed0 con 0x7ff6d40d0340 2021-02-10T11:47:36.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.171 7ff6da429700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d41b0be0 con 0x7ff6d40d1c70 2021-02-10T11:47:36.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.171 7ff6d37fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff6d40d0340 0x7ff6d40ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:36.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.171 7ff6d37fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff6d40d0340 0x7ff6d40ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34204/0 (socket says 172.21.2.101:34204) 2021-02-10T11:47:36.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.171 7ff6d37fe700 1 -- 172.21.2.101:0/556884850 learned_addr learned my addr 172.21.2.101:0/556884850 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:36.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.171 7ff6d8a46700 1 --2- 172.21.2.101:0/556884850 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6d40d1c70 0x7ff6d40efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:36.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.172 7ff6d37fe700 1 -- 172.21.2.101:0/556884850 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6d40d1c70 msgr2=0x7ff6d40efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:36.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.172 7ff6d37fe700 1 --2- 172.21.2.101:0/556884850 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6d40d1c70 0x7ff6d40efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:36.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.172 7ff6d37fe700 1 -- 172.21.2.101:0/556884850 >> v1:172.21.2.101:6789/0 conn(0x7ff6d40d1870 legacy=0x7ff6d41b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:36.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.172 7ff6d37fe700 1 -- 172.21.2.101:0/556884850 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d40f20a0 con 0x7ff6d40d0340 2021-02-10T11:47:36.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.172 7ff6d37fe700 1 --2- 172.21.2.101:0/556884850 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6d40d0340 0x7ff6d40ef640 secure :-1 s=READY pgs=3121 cs=0 l=1 rev1=1 rx=0x7ff6c8006700 tx=0x7ff6c8009b00).ready entity=mon.0 client_cookie=71808a3295e186c4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:36.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.172 7ff6d2ffd700 1 -- 172.21.2.101:0/556884850 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff6c800c0c0 con 0x7ff6d40d0340 2021-02-10T11:47:36.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.172 7ff6d2ffd700 1 -- 172.21.2.101:0/556884850 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c800c400 con 0x7ff6d40d0340 2021-02-10T11:47:36.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.172 7ff6d2ffd700 1 -- 172.21.2.101:0/556884850 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff6c800c8e0 con 0x7ff6d40d0340 2021-02-10T11:47:36.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.172 7ff6da429700 1 -- 172.21.2.101:0/556884850 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6d40d0340 msgr2=0x7ff6d40ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:36.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.172 7ff6da429700 1 --2- 172.21.2.101:0/556884850 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6d40d0340 0x7ff6d40ef640 secure :-1 s=READY pgs=3121 cs=0 l=1 rev1=1 rx=0x7ff6c8006700 tx=0x7ff6c8009b00).stop 2021-02-10T11:47:36.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.173 7ff6da429700 1 -- 172.21.2.101:0/556884850 shutdown_connections 2021-02-10T11:47:36.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.173 7ff6da429700 1 --2- 172.21.2.101:0/556884850 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6d40d1c70 0x7ff6d40efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:36.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.173 7ff6da429700 1 --2- 172.21.2.101:0/556884850 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6d40d0340 0x7ff6d40ef640 unknown :-1 s=CLOSED pgs=3121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:36.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.173 7ff6da429700 1 -- 172.21.2.101:0/556884850 >> 172.21.2.101:0/556884850 conn(0x7ff6d40ce450 msgr2=0x7ff6d40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:36.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.173 7ff6da429700 1 -- 172.21.2.101:0/556884850 shutdown_connections 2021-02-10T11:47:36.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.173 7ff6da429700 1 -- 172.21.2.101:0/556884850 wait complete. 2021-02-10T11:47:36.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.187 7ff6da429700 1 Processor -- start 2021-02-10T11:47:36.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.187 7ff6da429700 1 -- start start 2021-02-10T11:47:36.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.188 7ff6da429700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6d40beeb0 0x7ff6d40bf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:36.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.188 7ff6da429700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6d418ef70 0x7ff6d418f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:36.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.188 7ff6da429700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6d418f990 0x7ff6d40c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:36.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.188 7ff6da429700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d40bfc50 con 0x7ff6d418f990 2021-02-10T11:47:36.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.188 7ff6da429700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d418fd90 con 0x7ff6d418ef70 2021-02-10T11:47:36.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.188 7ff6da429700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d40bf8d0 con 0x7ff6d40beeb0 2021-02-10T11:47:36.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.188 7ff6d8a46700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6d418f990 0x7ff6d40c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:36.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.188 7ff6d8a46700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6d418f990 0x7ff6d40c01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34208/0 (socket says 172.21.2.101:34208) 2021-02-10T11:47:36.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.188 7ff6d8a46700 1 -- 172.21.2.101:0/1358173721 learned_addr learned my addr 172.21.2.101:0/1358173721 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:36.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.188 7ff6d37fe700 1 --2- 172.21.2.101:0/1358173721 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6d418ef70 0x7ff6d418f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:36.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.188 7ff6d3fff700 1 --2- 172.21.2.101:0/1358173721 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6d40beeb0 0x7ff6d40bf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:36.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.189 7ff6d8a46700 1 -- 172.21.2.101:0/1358173721 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6d40beeb0 msgr2=0x7ff6d40bf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:36.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.189 7ff6d8a46700 1 --2- 172.21.2.101:0/1358173721 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6d40beeb0 0x7ff6d40bf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:36.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.189 7ff6d8a46700 1 -- 172.21.2.101:0/1358173721 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6d418ef70 msgr2=0x7ff6d418f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:36.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.189 7ff6d8a46700 1 --2- 172.21.2.101:0/1358173721 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6d418ef70 0x7ff6d418f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:36.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.189 7ff6d8a46700 1 -- 172.21.2.101:0/1358173721 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d40c07f0 con 0x7ff6d418f990 2021-02-10T11:47:36.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.189 7ff6d8a46700 1 --2- 172.21.2.101:0/1358173721 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6d418f990 0x7ff6d40c01d0 secure :-1 s=READY pgs=3122 cs=0 l=1 rev1=1 rx=0x7ff6cc00b970 tx=0x7ff6cc00bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:36.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.189 7ff6d0ff9700 1 -- 172.21.2.101:0/1358173721 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff6cc000f90 con 0x7ff6d418f990 2021-02-10T11:47:36.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.189 7ff6d0ff9700 1 -- 172.21.2.101:0/1358173721 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6cc003030 con 0x7ff6d418f990 2021-02-10T11:47:36.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.189 7ff6da429700 1 -- 172.21.2.101:0/1358173721 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d40c07f0 con 0x7ff6d418f990 2021-02-10T11:47:36.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.189 7ff6d0ff9700 1 -- 172.21.2.101:0/1358173721 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff6cc001470 con 0x7ff6d418f990 2021-02-10T11:47:36.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.189 7ff6da429700 1 -- 172.21.2.101:0/1358173721 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d40c0d80 con 0x7ff6d418f990 2021-02-10T11:47:36.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.190 7ff6d0ff9700 1 -- 172.21.2.101:0/1358173721 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff6cc0011f0 con 0x7ff6d418f990 2021-02-10T11:47:36.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.190 7ff6d0ff9700 1 --2- 172.21.2.101:0/1358173721 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6b4018730 0x7ff6b401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:36.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.190 7ff6d0ff9700 1 -- 172.21.2.101:0/1358173721 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff6cc0329e0 con 0x7ff6d418f990 2021-02-10T11:47:36.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.190 7ff6d3fff700 1 --2- 172.21.2.101:0/1358173721 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6b4018730 0x7ff6b401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:36.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.190 7ff6d3fff700 1 --2- 172.21.2.101:0/1358173721 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6b4018730 0x7ff6b401ac10 crc :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:36.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.221 7ff6da429700 1 -- 172.21.2.101:0/1358173721 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d40ebc60 con 0x7ff6d418f990 2021-02-10T11:47:36.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.223 7ff6d0ff9700 1 -- 172.21.2.101:0/1358173721 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff6cc000f40 con 0x7ff6d418f990 2021-02-10T11:47:36.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.346 7ff6da429700 1 -- 172.21.2.101:0/1358173721 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6d40f0b10 con 0x7ff6d418f990 2021-02-10T11:47:36.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.347 7ff6d0ff9700 1 -- 172.21.2.101:0/1358173721 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+14007 (secure 0 0 0) 0x7ff6cc01f8a0 con 0x7ff6d418f990 2021-02-10T11:47:36.344 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:47:36.344 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":183,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:47:29.788748","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":39,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":38,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:47:26.756268","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":39,"pool_name":"cephfs_data","create_time":"2021-02-10 11:47:27.781048","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:47:36.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.348 7ff6da429700 1 -- 172.21.2.101:0/1358173721 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6b4018730 msgr2=0x7ff6b401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:36.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.348 7ff6da429700 1 --2- 172.21.2.101:0/1358173721 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6b4018730 0x7ff6b401ac10 crc :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:36.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.348 7ff6da429700 1 -- 172.21.2.101:0/1358173721 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6d418f990 msgr2=0x7ff6d40c01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:36.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.348 7ff6da429700 1 --2- 172.21.2.101:0/1358173721 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6d418f990 0x7ff6d40c01d0 secure :-1 s=READY pgs=3122 cs=0 l=1 rev1=1 rx=0x7ff6cc00b970 tx=0x7ff6cc00bd10).stop 2021-02-10T11:47:36.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.348 7ff6da429700 1 -- 172.21.2.101:0/1358173721 shutdown_connections 2021-02-10T11:47:36.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.348 7ff6da429700 1 --2- 172.21.2.101:0/1358173721 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6d418f990 0x7ff6d40c01d0 unknown :-1 s=CLOSED pgs=3122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:36.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.348 7ff6da429700 1 --2- 172.21.2.101:0/1358173721 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6d418ef70 0x7ff6d418f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:36.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.348 7ff6da429700 1 --2- 172.21.2.101:0/1358173721 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff6b4018730 0x7ff6b401ac10 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:36.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.348 7ff6da429700 1 --2- 172.21.2.101:0/1358173721 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6d40beeb0 0x7ff6d40bf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:36.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.348 7ff6da429700 1 -- 172.21.2.101:0/1358173721 >> 172.21.2.101:0/1358173721 conn(0x7ff6d40c2130 msgr2=0x7ff6d409e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:36.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.348 7ff6da429700 1 -- 172.21.2.101:0/1358173721 shutdown_connections 2021-02-10T11:47:36.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.349 7ff6da429700 1 -- 172.21.2.101:0/1358173721 wait complete. 2021-02-10T11:47:36.353 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 38, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:47:26.756268', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '182', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 39, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:47:27.781048', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '183', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:47:36.354 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:47:36.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.773 7f56855f3700 1 Processor -- start 2021-02-10T11:47:36.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.773 7f56855f3700 1 -- start start 2021-02-10T11:47:36.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.774 7f56855f3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f568010aff0 0x7f56800cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:36.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.774 7f56855f3700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56800fe1c0 con 0x7f56800d1870 2021-02-10T11:47:36.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.774 7f56855f3700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56801b0cb0 con 0x7f56800d1c70 2021-02-10T11:47:36.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.774 7f56855f3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5680109a60 con 0x7f568010aff0 2021-02-10T11:47:36.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.774 7f567fac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f568010aff0 0x7f56800cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:36.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.774 7f567fac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f568010aff0 0x7f56800cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34218/0 (socket says 172.21.2.101:34218) 2021-02-10T11:47:36.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.774 7f567fac0700 1 -- 172.21.2.101:0/4288047612 learned_addr learned my addr 172.21.2.101:0/4288047612 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:36.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.774 7f567fac0700 1 -- 172.21.2.101:0/4288047612 >> v1:172.21.2.108:6789/0 conn(0x7f56800d1c70 legacy=0x7f56800cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:36.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.774 7f567fac0700 1 -- 172.21.2.101:0/4288047612 >> v1:172.21.2.101:6789/0 conn(0x7f56800d1870 legacy=0x7f56801b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:36.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.774 7f567fac0700 1 -- 172.21.2.101:0/4288047612 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568010b3f0 con 0x7f568010aff0 2021-02-10T11:47:36.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.774 7f567fac0700 1 --2- 172.21.2.101:0/4288047612 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f568010aff0 0x7f56800cf750 secure :-1 s=READY pgs=3124 cs=0 l=1 rev1=1 rx=0x7f5678006f00 tx=0x7f5678009f10).ready entity=mon.0 client_cookie=d075bf6802fef261 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:36.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.775 7f567e2bd700 1 -- 172.21.2.101:0/4288047612 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5678003030 con 0x7f568010aff0 2021-02-10T11:47:36.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.775 7f567e2bd700 1 -- 172.21.2.101:0/4288047612 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567800b3b0 con 0x7f568010aff0 2021-02-10T11:47:36.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.775 7f567e2bd700 1 -- 172.21.2.101:0/4288047612 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f567800b8d0 con 0x7f568010aff0 2021-02-10T11:47:36.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.775 7f56855f3700 1 -- 172.21.2.101:0/4288047612 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f568010aff0 msgr2=0x7f56800cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:36.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.775 7f56855f3700 1 --2- 172.21.2.101:0/4288047612 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f568010aff0 0x7f56800cf750 secure :-1 s=READY pgs=3124 cs=0 l=1 rev1=1 rx=0x7f5678006f00 tx=0x7f5678009f10).stop 2021-02-10T11:47:36.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.775 7f56855f3700 1 -- 172.21.2.101:0/4288047612 shutdown_connections 2021-02-10T11:47:36.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.775 7f56855f3700 1 --2- 172.21.2.101:0/4288047612 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f568010aff0 0x7f56800cf750 unknown :-1 s=CLOSED pgs=3124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:36.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.775 7f56855f3700 1 -- 172.21.2.101:0/4288047612 >> 172.21.2.101:0/4288047612 conn(0x7f56800ce420 msgr2=0x7f56800d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:36.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.775 7f56855f3700 1 -- 172.21.2.101:0/4288047612 shutdown_connections 2021-02-10T11:47:36.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.775 7f56855f3700 1 -- 172.21.2.101:0/4288047612 wait complete. 2021-02-10T11:47:36.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.790 7f56855f3700 1 Processor -- start 2021-02-10T11:47:36.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.790 7f56855f3700 1 -- start start 2021-02-10T11:47:36.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.790 7f56855f3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56801059f0 0x7f56800be290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:36.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.790 7f56855f3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f568018a490 0x7f568018a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:36.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.790 7f56855f3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56800bf1c0 0x7f56800bf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:36.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.790 7f56855f3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56800be8b0 con 0x7f56801059f0 2021-02-10T11:47:36.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.791 7f56855f3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56800bfbe0 con 0x7f56800bf1c0 2021-02-10T11:47:36.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.791 7f56855f3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56800bfe00 con 0x7f568018a490 2021-02-10T11:47:36.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.791 7f567f2bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56801059f0 0x7f56800be290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:36.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.791 7f567f2bf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56801059f0 0x7f56800be290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34222/0 (socket says 172.21.2.101:34222) 2021-02-10T11:47:36.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.791 7f567f2bf700 1 -- 172.21.2.101:0/2402654093 learned_addr learned my addr 172.21.2.101:0/2402654093 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:36.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.791 7f567f2bf700 1 -- 172.21.2.101:0/2402654093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f568018a490 msgr2=0x7f568018a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:36.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.791 7f567f2bf700 1 --2- 172.21.2.101:0/2402654093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f568018a490 0x7f568018a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:36.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.791 7f567f2bf700 1 -- 172.21.2.101:0/2402654093 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56800bf1c0 msgr2=0x7f56800bf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:36.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.791 7f567f2bf700 1 --2- 172.21.2.101:0/2402654093 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56800bf1c0 0x7f56800bf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:36.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.791 7f567f2bf700 1 -- 172.21.2.101:0/2402654093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56800c0020 con 0x7f56801059f0 2021-02-10T11:47:36.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.791 7f567f2bf700 1 --2- 172.21.2.101:0/2402654093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56801059f0 0x7f56800be290 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 rx=0x7f566800b5b0 tx=0x7f566800b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:36.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.791 7f5677fff700 1 -- 172.21.2.101:0/2402654093 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f566800bd40 con 0x7f56801059f0 2021-02-10T11:47:36.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.791 7f5677fff700 1 -- 172.21.2.101:0/2402654093 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5668000e20 con 0x7f56801059f0 2021-02-10T11:47:36.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.791 7f5677fff700 1 -- 172.21.2.101:0/2402654093 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f56680062a0 con 0x7f56801059f0 2021-02-10T11:47:36.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.791 7f56855f3700 1 -- 172.21.2.101:0/2402654093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56800c0020 con 0x7f56801059f0 2021-02-10T11:47:36.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.792 7f56855f3700 1 -- 172.21.2.101:0/2402654093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56800c05b0 con 0x7f56801059f0 2021-02-10T11:47:36.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.792 7f5677fff700 1 -- 172.21.2.101:0/2402654093 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5668006020 con 0x7f56801059f0 2021-02-10T11:47:36.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.792 7f5677fff700 1 --2- 172.21.2.101:0/2402654093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56640186c0 0x7f566401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:36.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.792 7f5677fff700 1 -- 172.21.2.101:0/2402654093 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5668032450 con 0x7f56801059f0 2021-02-10T11:47:36.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.792 7f567eabe700 1 --2- 172.21.2.101:0/2402654093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56640186c0 0x7f566401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:36.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.793 7f567eabe700 1 --2- 172.21.2.101:0/2402654093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56640186c0 0x7f566401abc0 crc :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:36.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.823 7f56855f3700 1 -- 172.21.2.101:0/2402654093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56800bec90 con 0x7f56801059f0 2021-02-10T11:47:36.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.826 7f5677fff700 1 -- 172.21.2.101:0/2402654093 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5668030d80 con 0x7f56801059f0 2021-02-10T11:47:36.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.957 7f56855f3700 1 -- 172.21.2.101:0/2402654093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f56800bec90 con 0x7f56801059f0 2021-02-10T11:47:36.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.957 7f5677fff700 1 -- 172.21.2.101:0/2402654093 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v183) v1 ==== 92+0+10 (secure 0 0 0) 0x7f566800bd40 con 0x7f56801059f0 2021-02-10T11:47:36.955 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:47:36.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.959 7f56855f3700 1 -- 172.21.2.101:0/2402654093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56640186c0 msgr2=0x7f566401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:36.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.959 7f56855f3700 1 --2- 172.21.2.101:0/2402654093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56640186c0 0x7f566401abc0 crc :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:36.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.959 7f56855f3700 1 -- 172.21.2.101:0/2402654093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56801059f0 msgr2=0x7f56800be290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:36.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.959 7f56855f3700 1 --2- 172.21.2.101:0/2402654093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56801059f0 0x7f56800be290 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 rx=0x7f566800b5b0 tx=0x7f566800b950).stop 2021-02-10T11:47:36.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.959 7f56855f3700 1 -- 172.21.2.101:0/2402654093 shutdown_connections 2021-02-10T11:47:36.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.959 7f56855f3700 1 --2- 172.21.2.101:0/2402654093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f56640186c0 0x7f566401abc0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:36.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.959 7f56855f3700 1 --2- 172.21.2.101:0/2402654093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f568018a490 0x7f568018a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:36.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.959 7f56855f3700 1 --2- 172.21.2.101:0/2402654093 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f56800bf1c0 0x7f56800bf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:36.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.959 7f56855f3700 1 --2- 172.21.2.101:0/2402654093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f56801059f0 0x7f56800be290 unknown :-1 s=CLOSED pgs=3125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:36.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.959 7f56855f3700 1 -- 172.21.2.101:0/2402654093 >> 172.21.2.101:0/2402654093 conn(0x7f56800c41c0 msgr2=0x7f56800b8530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:36.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.960 7f56855f3700 1 -- 172.21.2.101:0/2402654093 shutdown_connections 2021-02-10T11:47:36.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:36.960 7f56855f3700 1 -- 172.21.2.101:0/2402654093 wait complete. 2021-02-10T11:47:36.964 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:47:37.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.382 7f241c6a8700 1 Processor -- start 2021-02-10T11:47:37.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.382 7f241c6a8700 1 -- start start 2021-02-10T11:47:37.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.383 7f241c6a8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f24140d1c70 0x7f24140cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:37.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.383 7f241c6a8700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24140d0170 con 0x7f24140cf580 2021-02-10T11:47:37.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.383 7f241c6a8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24140fe1c0 con 0x7f24140d1870 2021-02-10T11:47:37.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.383 7f241c6a8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24141b0cb0 con 0x7f24140d1c70 2021-02-10T11:47:37.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.383 7f241acc5700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f24140cf580 0x7f24140cf980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34722/0 (socket says 172.21.2.101:34722) 2021-02-10T11:47:37.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.383 7f241acc5700 1 -- 172.21.2.101:0/3507842111 learned_addr learned my addr 172.21.2.101:0/3507842111 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:37.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.383 7f2419cc3700 1 --2- 172.21.2.101:0/3507842111 >> v2:172.21.2.108:3300/0 conn(0x7f24140d1c70 0x7f24140cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:37.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.384 7f24194c2700 1 -- 172.21.2.101:0/3507842111 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2359607017 0 0) 0x7f240c001500 con 0x7f24140cf580 2021-02-10T11:47:37.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.384 7f24194c2700 1 -- 172.21.2.101:0/3507842111 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2400002fe0 con 0x7f24140cf580 2021-02-10T11:47:37.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.384 7f2419cc3700 1 -- 172.21.2.101:0/3507842111 >> v1:172.21.2.108:6789/0 conn(0x7f24140d1870 legacy=0x7f24141b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:37.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.384 7f2419cc3700 1 -- 172.21.2.101:0/3507842111 >> v1:172.21.2.101:6789/0 conn(0x7f24140cf580 legacy=0x7f24140cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:37.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.384 7f2419cc3700 1 -- 172.21.2.101:0/3507842111 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2414109a60 con 0x7f24140d1c70 2021-02-10T11:47:37.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.384 7f2419cc3700 1 --2- 172.21.2.101:0/3507842111 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24140d1c70 0x7f24140cef60 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f2404006650 tx=0x7f2404009a00).ready entity=mon.1 client_cookie=18fd9a607f4e5cae server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:37.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.384 7f24194c2700 1 -- 172.21.2.101:0/3507842111 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f24040009b0 con 0x7f24140d1c70 2021-02-10T11:47:37.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.384 7f24194c2700 1 -- 172.21.2.101:0/3507842111 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240400c0c0 con 0x7f24140d1c70 2021-02-10T11:47:37.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.384 7f24194c2700 1 -- 172.21.2.101:0/3507842111 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f240400c5a0 con 0x7f24140d1c70 2021-02-10T11:47:37.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.384 7f241c6a8700 1 -- 172.21.2.101:0/3507842111 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24140d1c70 msgr2=0x7f24140cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:37.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.384 7f241c6a8700 1 --2- 172.21.2.101:0/3507842111 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24140d1c70 0x7f24140cef60 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f2404006650 tx=0x7f2404009a00).stop 2021-02-10T11:47:37.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.385 7f241c6a8700 1 -- 172.21.2.101:0/3507842111 shutdown_connections 2021-02-10T11:47:37.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.385 7f241c6a8700 1 --2- 172.21.2.101:0/3507842111 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24140d1c70 0x7f24140cef60 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:37.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.385 7f241c6a8700 1 -- 172.21.2.101:0/3507842111 >> 172.21.2.101:0/3507842111 conn(0x7f24140ce420 msgr2=0x7f24140d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:37.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.385 7f241c6a8700 1 -- 172.21.2.101:0/3507842111 shutdown_connections 2021-02-10T11:47:37.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.385 7f241c6a8700 1 -- 172.21.2.101:0/3507842111 wait complete. 2021-02-10T11:47:37.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.399 7f241c6a8700 1 Processor -- start 2021-02-10T11:47:37.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.400 7f241c6a8700 1 -- start start 2021-02-10T11:47:37.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.400 7f241c6a8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24140858f0 0x7f2414085cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:37.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.400 7f241c6a8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2414086310 0x7f2414086710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:37.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.400 7f241c6a8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24140a0580 0x7f24140a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:37.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.400 7f241c6a8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2414085370 con 0x7f2414086310 2021-02-10T11:47:37.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.400 7f241c6a8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24140a0fa0 con 0x7f24140a0580 2021-02-10T11:47:37.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.400 7f241c6a8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24140a1180 con 0x7f24140858f0 2021-02-10T11:47:37.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.400 7f2419cc3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2414086310 0x7f2414086710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:37.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.400 7f2419cc3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2414086310 0x7f2414086710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34234/0 (socket says 172.21.2.101:34234) 2021-02-10T11:47:37.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.400 7f2419cc3700 1 -- 172.21.2.101:0/337972434 learned_addr learned my addr 172.21.2.101:0/337972434 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:37.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.401 7f241a4c4700 1 --2- 172.21.2.101:0/337972434 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24140858f0 0x7f2414085cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:37.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.401 7f2419cc3700 1 -- 172.21.2.101:0/337972434 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24140858f0 msgr2=0x7f2414085cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:37.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.401 7f241acc5700 1 --2- 172.21.2.101:0/337972434 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24140a0580 0x7f24140a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:37.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.401 7f2419cc3700 1 --2- 172.21.2.101:0/337972434 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24140858f0 0x7f2414085cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:37.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.401 7f2419cc3700 1 -- 172.21.2.101:0/337972434 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24140a0580 msgr2=0x7f24140a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:37.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.401 7f2419cc3700 1 --2- 172.21.2.101:0/337972434 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24140a0580 0x7f24140a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:37.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.401 7f2419cc3700 1 -- 172.21.2.101:0/337972434 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24140a13a0 con 0x7f2414086310 2021-02-10T11:47:37.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.401 7f241a4c4700 1 --2- 172.21.2.101:0/337972434 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24140858f0 0x7f2414085cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:47:37.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.401 7f2419cc3700 1 --2- 172.21.2.101:0/337972434 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2414086310 0x7f2414086710 secure :-1 s=READY pgs=3127 cs=0 l=1 rev1=1 rx=0x7f240400b640 tx=0x7f240400b9e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:37.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.401 7f240affd700 1 -- 172.21.2.101:0/337972434 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f240400bdd0 con 0x7f2414086310 2021-02-10T11:47:37.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.401 7f240affd700 1 -- 172.21.2.101:0/337972434 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2404000c90 con 0x7f2414086310 2021-02-10T11:47:37.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.401 7f240affd700 1 -- 172.21.2.101:0/337972434 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f24040022b0 con 0x7f2414086310 2021-02-10T11:47:37.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.401 7f241c6a8700 1 -- 172.21.2.101:0/337972434 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24140a13a0 con 0x7f2414086310 2021-02-10T11:47:37.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.401 7f241c6a8700 1 -- 172.21.2.101:0/337972434 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24140a1930 con 0x7f2414086310 2021-02-10T11:47:37.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.402 7f240affd700 1 -- 172.21.2.101:0/337972434 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2404006a10 con 0x7f2414086310 2021-02-10T11:47:37.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.402 7f240affd700 1 --2- 172.21.2.101:0/337972434 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f23fc0186c0 0x7f23fc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:37.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.402 7f240affd700 1 -- 172.21.2.101:0/337972434 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f24040325c0 con 0x7f2414086310 2021-02-10T11:47:37.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.402 7f241a4c4700 1 --2- 172.21.2.101:0/337972434 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f23fc0186c0 0x7f23fc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:37.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.403 7f241a4c4700 1 --2- 172.21.2.101:0/337972434 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f23fc0186c0 0x7f23fc01abc0 crc :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:37.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.433 7f241c6a8700 1 -- 172.21.2.101:0/337972434 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2414082a70 con 0x7f2414086310 2021-02-10T11:47:37.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.436 7f240affd700 1 -- 172.21.2.101:0/337972434 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f24040009b0 con 0x7f2414086310 2021-02-10T11:47:37.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.562 7f241c6a8700 1 -- 172.21.2.101:0/337972434 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f2414082a70 con 0x7f2414086310 2021-02-10T11:47:37.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.563 7f240affd700 1 -- 172.21.2.101:0/337972434 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v183) v1 ==== 104+0+10 (secure 0 0 0) 0x7f2404000f00 con 0x7f2414086310 2021-02-10T11:47:37.560 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:47:37.561 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.565 7f241c6a8700 1 -- 172.21.2.101:0/337972434 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f23fc0186c0 msgr2=0x7f23fc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:37.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.565 7f241c6a8700 1 --2- 172.21.2.101:0/337972434 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f23fc0186c0 0x7f23fc01abc0 crc :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:37.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.565 7f241c6a8700 1 -- 172.21.2.101:0/337972434 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2414086310 msgr2=0x7f2414086710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:37.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.565 7f241c6a8700 1 --2- 172.21.2.101:0/337972434 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2414086310 0x7f2414086710 secure :-1 s=READY pgs=3127 cs=0 l=1 rev1=1 rx=0x7f240400b640 tx=0x7f240400b9e0).stop 2021-02-10T11:47:37.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.565 7f241c6a8700 1 -- 172.21.2.101:0/337972434 shutdown_connections 2021-02-10T11:47:37.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.565 7f241c6a8700 1 --2- 172.21.2.101:0/337972434 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f24140a0580 0x7f24140a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:37.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.565 7f241c6a8700 1 --2- 172.21.2.101:0/337972434 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2414086310 0x7f2414086710 unknown :-1 s=CLOSED pgs=3127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:37.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.565 7f241c6a8700 1 --2- 172.21.2.101:0/337972434 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f23fc0186c0 0x7f23fc01abc0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:37.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.565 7f241c6a8700 1 --2- 172.21.2.101:0/337972434 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24140858f0 0x7f2414085cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:37.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.565 7f241c6a8700 1 -- 172.21.2.101:0/337972434 >> 172.21.2.101:0/337972434 conn(0x7f24140946f0 msgr2=0x7f2414095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:37.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.565 7f241c6a8700 1 -- 172.21.2.101:0/337972434 shutdown_connections 2021-02-10T11:47:37.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.565 7f241c6a8700 1 -- 172.21.2.101:0/337972434 wait complete. 2021-02-10T11:47:37.570 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:47:38.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.994 7f4a8e8a7700 1 Processor -- start 2021-02-10T11:47:38.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.995 7f4a8e8a7700 1 -- start start 2021-02-10T11:47:38.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.995 7f4a8e8a7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4a880d1c70 0x7f4a880cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:38.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.995 7f4a8e8a7700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a880d0170 con 0x7f4a880cf580 2021-02-10T11:47:38.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.995 7f4a8e8a7700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a880fe1c0 con 0x7f4a880d1870 2021-02-10T11:47:38.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.995 7f4a8e8a7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a881b0cb0 con 0x7f4a880d1c70 2021-02-10T11:47:38.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.995 7f4a8cec4700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f4a880cf580 0x7f4a880cf980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34734/0 (socket says 172.21.2.101:34734) 2021-02-10T11:47:38.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.995 7f4a8cec4700 1 -- 172.21.2.101:0/2460899100 learned_addr learned my addr 172.21.2.101:0/2460899100 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:38.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.995 7f4a86ffd700 1 -- 172.21.2.101:0/2460899100 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193805016 0 0) 0x7f4a80001500 con 0x7f4a880cf580 2021-02-10T11:47:38.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a86ffd700 1 -- 172.21.2.101:0/2460899100 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a6c002fe0 con 0x7f4a880cf580 2021-02-10T11:47:38.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a86ffd700 1 -- 172.21.2.101:0/2460899100 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2188071097 0 0) 0x7f4a80001270 con 0x7f4a880cf580 2021-02-10T11:47:38.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a86ffd700 1 -- 172.21.2.101:0/2460899100 >> v2:172.21.2.108:3300/0 conn(0x7f4a880d1c70 msgr2=0x7f4a880cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:38.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a86ffd700 1 --2- 172.21.2.101:0/2460899100 >> v2:172.21.2.108:3300/0 conn(0x7f4a880d1c70 0x7f4a880cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:38.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a86ffd700 1 -- 172.21.2.101:0/2460899100 >> v1:172.21.2.108:6789/0 conn(0x7f4a880d1870 legacy=0x7f4a881b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:38.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a86ffd700 1 -- 172.21.2.101:0/2460899100 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a88109a60 con 0x7f4a880cf580 2021-02-10T11:47:38.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a86ffd700 1 -- 172.21.2.101:0/2460899100 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f4a80001bd0 con 0x7f4a880cf580 2021-02-10T11:47:38.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a86ffd700 1 -- 172.21.2.101:0/2460899100 >> v1:172.21.2.101:6789/0 conn(0x7f4a880cf580 legacy=0x7f4a880cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:38.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a86ffd700 1 --2- 172.21.2.101:0/2460899100 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4a6c003a60 0x7f4a6c005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:38.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a86ffd700 1 --2- 172.21.2.101:0/2460899100 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a6c006780 0x7f4a6c008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:38.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a86ffd700 1 --2- 172.21.2.101:0/2460899100 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4a6c009580 0x7f4a6c00baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:38.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a86ffd700 1 -- 172.21.2.101:0/2460899100 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c00c2f0 con 0x7f4a6c006780 2021-02-10T11:47:38.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a86ffd700 1 -- 172.21.2.101:0/2460899100 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c00c510 con 0x7f4a6c009580 2021-02-10T11:47:38.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a86ffd700 1 -- 172.21.2.101:0/2460899100 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c00c730 con 0x7f4a6c003a60 2021-02-10T11:47:38.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a877fe700 1 --2- 172.21.2.101:0/2460899100 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a6c006780 0x7f4a6c008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:38.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a87fff700 1 --2- 172.21.2.101:0/2460899100 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4a6c003a60 0x7f4a6c005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:38.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a8cec4700 1 --2- 172.21.2.101:0/2460899100 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4a6c009580 0x7f4a6c00baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:38.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a877fe700 1 -- 172.21.2.101:0/2460899100 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4a6c003a60 msgr2=0x7f4a6c005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:38.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a877fe700 1 --2- 172.21.2.101:0/2460899100 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4a6c003a60 0x7f4a6c005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:38.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a877fe700 1 -- 172.21.2.101:0/2460899100 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4a6c009580 msgr2=0x7f4a6c00baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:38.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a877fe700 1 --2- 172.21.2.101:0/2460899100 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4a6c009580 0x7f4a6c00baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:38.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a8cec4700 1 -- 172.21.2.101:0/2460899100 reap_dead start 2021-02-10T11:47:38.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.996 7f4a877fe700 1 -- 172.21.2.101:0/2460899100 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6c00c9d0 con 0x7f4a6c006780 2021-02-10T11:47:38.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.997 7f4a877fe700 1 --2- 172.21.2.101:0/2460899100 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a6c006780 0x7f4a6c008c80 secure :-1 s=READY pgs=3129 cs=0 l=1 rev1=1 rx=0x7f4a74006bf0 tx=0x7f4a7404fc20).ready entity=mon.0 client_cookie=b7fd8508ea5ed985 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:38.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.997 7f4a86ffd700 1 -- 172.21.2.101:0/2460899100 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4a7404fd60 con 0x7f4a6c006780 2021-02-10T11:47:38.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.997 7f4a86ffd700 1 -- 172.21.2.101:0/2460899100 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a740067e0 con 0x7f4a6c006780 2021-02-10T11:47:38.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.997 7f4a86ffd700 1 -- 172.21.2.101:0/2460899100 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4a74002220 con 0x7f4a6c006780 2021-02-10T11:47:38.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.997 7f4a8e8a7700 1 -- 172.21.2.101:0/2460899100 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a6c006780 msgr2=0x7f4a6c008c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:38.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.997 7f4a8e8a7700 1 --2- 172.21.2.101:0/2460899100 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a6c006780 0x7f4a6c008c80 secure :-1 s=READY pgs=3129 cs=0 l=1 rev1=1 rx=0x7f4a74006bf0 tx=0x7f4a7404fc20).stop 2021-02-10T11:47:38.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.998 7f4a8e8a7700 1 -- 172.21.2.101:0/2460899100 shutdown_connections 2021-02-10T11:47:38.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.998 7f4a8e8a7700 1 --2- 172.21.2.101:0/2460899100 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a6c006780 0x7f4a6c008c80 unknown :-1 s=CLOSED pgs=3129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:38.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.998 7f4a8e8a7700 1 -- 172.21.2.101:0/2460899100 >> 172.21.2.101:0/2460899100 conn(0x7f4a880ce420 msgr2=0x7f4a880d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:38.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.998 7f4a8e8a7700 1 -- 172.21.2.101:0/2460899100 shutdown_connections 2021-02-10T11:47:38.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:37.998 7f4a8e8a7700 1 -- 172.21.2.101:0/2460899100 wait complete. 2021-02-10T11:47:38.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.012 7f4a8e8a7700 1 Processor -- start 2021-02-10T11:47:38.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.012 7f4a8e8a7700 1 -- start start 2021-02-10T11:47:38.014 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.013 7f4a8e8a7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a88085bc0 0x7f4a880863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:38.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.013 7f4a8e8a7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4a880954c0 0x7f4a88096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:38.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.013 7f4a8e8a7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4a880958c0 0x7f4a88096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:38.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.013 7f4a8e8a7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a88095d80 con 0x7f4a88085bc0 2021-02-10T11:47:38.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.013 7f4a8e8a7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a880bfa60 con 0x7f4a880958c0 2021-02-10T11:47:38.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.013 7f4a8e8a7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a88085fc0 con 0x7f4a880954c0 2021-02-10T11:47:38.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.013 7f4a87fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a88085bc0 0x7f4a880863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:38.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.013 7f4a87fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a88085bc0 0x7f4a880863e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34252/0 (socket says 172.21.2.101:34252) 2021-02-10T11:47:38.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.013 7f4a87fff700 1 -- 172.21.2.101:0/2267163007 learned_addr learned my addr 172.21.2.101:0/2267163007 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:38.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.013 7f4a8cec4700 1 --2- 172.21.2.101:0/2267163007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4a880958c0 0x7f4a88096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:38.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.013 7f4a87fff700 1 -- 172.21.2.101:0/2267163007 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4a880954c0 msgr2=0x7f4a88096300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:38.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.013 7f4a877fe700 1 --2- 172.21.2.101:0/2267163007 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4a880954c0 0x7f4a88096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:38.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.013 7f4a87fff700 1 --2- 172.21.2.101:0/2267163007 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4a880954c0 0x7f4a88096300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:38.015 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.013 7f4a87fff700 1 -- 172.21.2.101:0/2267163007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4a880958c0 msgr2=0x7f4a88096920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:38.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.013 7f4a87fff700 1 --2- 172.21.2.101:0/2267163007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4a880958c0 0x7f4a88096920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:38.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.013 7f4a87fff700 1 -- 172.21.2.101:0/2267163007 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a88086c10 con 0x7f4a88085bc0 2021-02-10T11:47:38.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.013 7f4a877fe700 1 --2- 172.21.2.101:0/2267163007 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4a880954c0 0x7f4a88096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:47:38.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.013 7f4a8cec4700 1 --2- 172.21.2.101:0/2267163007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4a880958c0 0x7f4a88096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:47:38.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.013 7f4a87fff700 1 --2- 172.21.2.101:0/2267163007 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a88085bc0 0x7f4a880863e0 secure :-1 s=READY pgs=3130 cs=0 l=1 rev1=1 rx=0x7f4a78005a20 tx=0x7f4a7800a980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:38.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.013 7f4a84ff9700 1 -- 172.21.2.101:0/2267163007 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4a7800ad70 con 0x7f4a88085bc0 2021-02-10T11:47:38.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.013 7f4a84ff9700 1 -- 172.21.2.101:0/2267163007 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7800b070 con 0x7f4a88085bc0 2021-02-10T11:47:38.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.014 7f4a8e8a7700 1 -- 172.21.2.101:0/2267163007 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a88086c10 con 0x7f4a88085bc0 2021-02-10T11:47:38.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.014 7f4a84ff9700 1 -- 172.21.2.101:0/2267163007 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4a7800b500 con 0x7f4a88085bc0 2021-02-10T11:47:38.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.014 7f4a8e8a7700 1 -- 172.21.2.101:0/2267163007 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a88086c10 con 0x7f4a88085bc0 2021-02-10T11:47:38.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.014 7f4a84ff9700 1 -- 172.21.2.101:0/2267163007 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4a78006740 con 0x7f4a88085bc0 2021-02-10T11:47:38.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.014 7f4a84ff9700 1 --2- 172.21.2.101:0/2267163007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4a64018730 0x7f4a6401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:38.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.014 7f4a84ff9700 1 -- 172.21.2.101:0/2267163007 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4a78032630 con 0x7f4a88085bc0 2021-02-10T11:47:38.016 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.014 7f4a877fe700 1 --2- 172.21.2.101:0/2267163007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4a64018730 0x7f4a6401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:38.017 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.014 7f4a877fe700 1 --2- 172.21.2.101:0/2267163007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4a64018730 0x7f4a6401ac10 crc :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:38.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.045 7f4a8e8a7700 1 -- 172.21.2.101:0/2267163007 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a880fe1c0 con 0x7f4a88085bc0 2021-02-10T11:47:38.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.048 7f4a84ff9700 1 -- 172.21.2.101:0/2267163007 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4a7800ad70 con 0x7f4a88085bc0 2021-02-10T11:47:38.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.178 7f4a8e8a7700 1 -- 172.21.2.101:0/2267163007 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f4a880fe1c0 con 0x7f4a88085bc0 2021-02-10T11:47:38.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.178 7f4a84ff9700 1 -- 172.21.2.101:0/2267163007 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v183) v1 ==== 100+0+10 (secure 0 0 0) 0x7f4a78006740 con 0x7f4a88085bc0 2021-02-10T11:47:38.175 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:47:38.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.180 7f4a8e8a7700 1 -- 172.21.2.101:0/2267163007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4a64018730 msgr2=0x7f4a6401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:38.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.180 7f4a8e8a7700 1 --2- 172.21.2.101:0/2267163007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4a64018730 0x7f4a6401ac10 crc :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:38.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.180 7f4a8e8a7700 1 -- 172.21.2.101:0/2267163007 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a88085bc0 msgr2=0x7f4a880863e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:38.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.180 7f4a8e8a7700 1 --2- 172.21.2.101:0/2267163007 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a88085bc0 0x7f4a880863e0 secure :-1 s=READY pgs=3130 cs=0 l=1 rev1=1 rx=0x7f4a78005a20 tx=0x7f4a7800a980).stop 2021-02-10T11:47:38.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.180 7f4a8e8a7700 1 -- 172.21.2.101:0/2267163007 shutdown_connections 2021-02-10T11:47:38.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.180 7f4a8e8a7700 1 --2- 172.21.2.101:0/2267163007 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4a64018730 0x7f4a6401ac10 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:38.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.180 7f4a8e8a7700 1 --2- 172.21.2.101:0/2267163007 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4a880954c0 0x7f4a88096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:38.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.180 7f4a8e8a7700 1 --2- 172.21.2.101:0/2267163007 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4a880958c0 0x7f4a88096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:38.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.180 7f4a8e8a7700 1 --2- 172.21.2.101:0/2267163007 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4a88085bc0 0x7f4a880863e0 unknown :-1 s=CLOSED pgs=3130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:38.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.180 7f4a8e8a7700 1 -- 172.21.2.101:0/2267163007 >> 172.21.2.101:0/2267163007 conn(0x7f4a880bef00 msgr2=0x7f4a8809d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:38.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.181 7f4a8e8a7700 1 -- 172.21.2.101:0/2267163007 shutdown_connections 2021-02-10T11:47:38.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.181 7f4a8e8a7700 1 -- 172.21.2.101:0/2267163007 wait complete. 2021-02-10T11:47:38.186 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:47:38.186 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:47:38.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.601 7f9f9131f700 1 Processor -- start 2021-02-10T11:47:38.616 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.601 7f9f9131f700 1 -- start start 2021-02-10T11:47:38.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.602 7f9f9131f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f8c0d0340 0x7f9f8c0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:38.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.602 7f9f9131f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f8c0f0b10 con 0x7f9f8c0d1c70 2021-02-10T11:47:38.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.602 7f9f9131f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f8c1b0be0 con 0x7f9f8c0d1870 2021-02-10T11:47:38.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.602 7f9f9131f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c0e5ed0 con 0x7f9f8c0d0340 2021-02-10T11:47:38.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.602 7f9f8b87e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f9f8c0d1c70 0x7f9f8c0efc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34756/0 (socket says 172.21.2.101:34756) 2021-02-10T11:47:38.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.602 7f9f8b87e700 1 -- 172.21.2.101:0/3338589973 learned_addr learned my addr 172.21.2.101:0/3338589973 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:38.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.602 7f9f8a07b700 1 -- 172.21.2.101:0/3338589973 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833693796 0 0) 0x7f9f84001500 con 0x7f9f8c0d1c70 2021-02-10T11:47:38.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.603 7f9f8a07b700 1 -- 172.21.2.101:0/3338589973 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f74002fe0 con 0x7f9f8c0d1c70 2021-02-10T11:47:38.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.603 7f9f8a87c700 1 --2- 172.21.2.101:0/3338589973 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f8c0d0340 0x7f9f8c0ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:38.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.603 7f9f8a07b700 1 -- 172.21.2.101:0/3338589973 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1006010749 0 0) 0x7f9f84001460 con 0x7f9f8c0d1c70 2021-02-10T11:47:38.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.603 7f9f8a07b700 1 -- 172.21.2.101:0/3338589973 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f8c0d0340 msgr2=0x7f9f8c0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:38.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.603 7f9f8a07b700 1 --2- 172.21.2.101:0/3338589973 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f8c0d0340 0x7f9f8c0ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:38.617 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.603 7f9f8a07b700 1 -- 172.21.2.101:0/3338589973 >> v1:172.21.2.108:6789/0 conn(0x7f9f8c0d1870 legacy=0x7f9f8c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:38.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.603 7f9f8a07b700 1 -- 172.21.2.101:0/3338589973 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c0f0450 con 0x7f9f8c0d1c70 2021-02-10T11:47:38.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.603 7f9f8a07b700 1 -- 172.21.2.101:0/3338589973 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f9f84001980 con 0x7f9f8c0d1c70 2021-02-10T11:47:38.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.603 7f9f8a07b700 1 -- 172.21.2.101:0/3338589973 >> v1:172.21.2.101:6789/0 conn(0x7f9f8c0d1c70 legacy=0x7f9f8c0efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:38.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.603 7f9f8a07b700 1 --2- 172.21.2.101:0/3338589973 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f74003ac0 0x7f9f74005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:38.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.603 7f9f8a07b700 1 --2- 172.21.2.101:0/3338589973 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f74006790 0x7f9f74008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:38.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.603 7f9f8a07b700 1 --2- 172.21.2.101:0/3338589973 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f74009590 0x7f9f7400ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:38.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.603 7f9f8a07b700 1 -- 172.21.2.101:0/3338589973 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7400c2d0 con 0x7f9f74006790 2021-02-10T11:47:38.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.603 7f9f8a07b700 1 -- 172.21.2.101:0/3338589973 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7400c4f0 con 0x7f9f74003ac0 2021-02-10T11:47:38.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.603 7f9f8a07b700 1 -- 172.21.2.101:0/3338589973 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7400c710 con 0x7f9f74009590 2021-02-10T11:47:38.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.603 7f9f8a87c700 1 --2- 172.21.2.101:0/3338589973 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f74006790 0x7f9f74008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:38.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.603 7f9f8b07d700 1 --2- 172.21.2.101:0/3338589973 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f74003ac0 0x7f9f74005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:38.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.604 7f9f8b87e700 1 --2- 172.21.2.101:0/3338589973 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f74009590 0x7f9f7400ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:38.618 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.604 7f9f8a87c700 1 -- 172.21.2.101:0/3338589973 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f74009590 msgr2=0x7f9f7400ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:38.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.604 7f9f8a87c700 1 --2- 172.21.2.101:0/3338589973 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f74009590 0x7f9f7400ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:38.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.604 7f9f8a87c700 1 -- 172.21.2.101:0/3338589973 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f74003ac0 msgr2=0x7f9f74005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:38.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.604 7f9f8a87c700 1 --2- 172.21.2.101:0/3338589973 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f74003ac0 0x7f9f74005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:38.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.604 7f9f8a87c700 1 -- 172.21.2.101:0/3338589973 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f74050980 con 0x7f9f74006790 2021-02-10T11:47:38.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.604 7f9f8a87c700 1 --2- 172.21.2.101:0/3338589973 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f74006790 0x7f9f74008c90 secure :-1 s=READY pgs=3132 cs=0 l=1 rev1=1 rx=0x7f9f80009310 tx=0x7f9f80009a90).ready entity=mon.0 client_cookie=6051b21925c1f145 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:38.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.604 7f9f8a07b700 1 -- 172.21.2.101:0/3338589973 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9f8000c0c0 con 0x7f9f74006790 2021-02-10T11:47:38.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.604 7f9f8a07b700 1 -- 172.21.2.101:0/3338589973 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f80006400 con 0x7f9f74006790 2021-02-10T11:47:38.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.604 7f9f8a07b700 1 -- 172.21.2.101:0/3338589973 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9f8000c400 con 0x7f9f74006790 2021-02-10T11:47:38.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.604 7f9f9131f700 1 -- 172.21.2.101:0/3338589973 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f74006790 msgr2=0x7f9f74008c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:38.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.604 7f9f9131f700 1 --2- 172.21.2.101:0/3338589973 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f74006790 0x7f9f74008c90 secure :-1 s=READY pgs=3132 cs=0 l=1 rev1=1 rx=0x7f9f80009310 tx=0x7f9f80009a90).stop 2021-02-10T11:47:38.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.604 7f9f8b87e700 1 -- 172.21.2.101:0/3338589973 reap_dead start 2021-02-10T11:47:38.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.605 7f9f9131f700 1 -- 172.21.2.101:0/3338589973 shutdown_connections 2021-02-10T11:47:38.619 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.605 7f9f9131f700 1 -- 172.21.2.101:0/3338589973 >> 172.21.2.101:0/3338589973 conn(0x7f9f8c0ce450 msgr2=0x7f9f8c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:38.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.605 7f9f9131f700 1 -- 172.21.2.101:0/3338589973 shutdown_connections 2021-02-10T11:47:38.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.605 7f9f9131f700 1 -- 172.21.2.101:0/3338589973 wait complete. 2021-02-10T11:47:38.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.620 7f9f9131f700 1 Processor -- start 2021-02-10T11:47:38.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.620 7f9f9131f700 1 -- start start 2021-02-10T11:47:38.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.620 7f9f9131f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f8c0f0280 0x7f9f8c0a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:38.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.620 7f9f9131f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f8c0f0680 0x7f9f8c0a2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:38.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.620 7f9f9131f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f8c0ec180 0x7f9f8c0ec580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:38.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.620 7f9f9131f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c0efd00 con 0x7f9f8c0ec180 2021-02-10T11:47:38.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.620 7f9f9131f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c0ecba0 con 0x7f9f8c0f0680 2021-02-10T11:47:38.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.620 7f9f9131f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c0ecd80 con 0x7f9f8c0f0280 2021-02-10T11:47:38.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.620 7f9f8b87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f8c0ec180 0x7f9f8c0ec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:38.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.620 7f9f8b87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f8c0ec180 0x7f9f8c0ec580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34270/0 (socket says 172.21.2.101:34270) 2021-02-10T11:47:38.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.620 7f9f8b87e700 1 -- 172.21.2.101:0/3585694386 learned_addr learned my addr 172.21.2.101:0/3585694386 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:38.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.620 7f9f8b87e700 1 -- 172.21.2.101:0/3585694386 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f8c0f0280 msgr2=0x7f9f8c0a2310 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:47:38.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.620 7f9f8b87e700 1 --2- 172.21.2.101:0/3585694386 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f8c0f0280 0x7f9f8c0a2310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:38.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.620 7f9f8b87e700 1 -- 172.21.2.101:0/3585694386 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f8c0f0680 msgr2=0x7f9f8c0a2930 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:47:38.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.620 7f9f8b87e700 1 --2- 172.21.2.101:0/3585694386 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f8c0f0680 0x7f9f8c0a2930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:38.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.620 7f9f8b87e700 1 -- 172.21.2.101:0/3585694386 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c0ecfa0 con 0x7f9f8c0ec180 2021-02-10T11:47:38.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.621 7f9f8b87e700 1 --2- 172.21.2.101:0/3585694386 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f8c0ec180 0x7f9f8c0ec580 secure :-1 s=READY pgs=3133 cs=0 l=1 rev1=1 rx=0x7f9f8400a940 tx=0x7f9f8400ace0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:38.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.621 7f9f73fff700 1 -- 172.21.2.101:0/3585694386 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9f840013f0 con 0x7f9f8c0ec180 2021-02-10T11:47:38.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.621 7f9f73fff700 1 -- 172.21.2.101:0/3585694386 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f84007020 con 0x7f9f8c0ec180 2021-02-10T11:47:38.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.621 7f9f73fff700 1 -- 172.21.2.101:0/3585694386 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9f840074c0 con 0x7f9f8c0ec180 2021-02-10T11:47:38.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.621 7f9f9131f700 1 -- 172.21.2.101:0/3585694386 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f8c0ecfa0 con 0x7f9f8c0ec180 2021-02-10T11:47:38.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.621 7f9f9131f700 1 -- 172.21.2.101:0/3585694386 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8c0ed530 con 0x7f9f8c0ec180 2021-02-10T11:47:38.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.622 7f9f73fff700 1 -- 172.21.2.101:0/3585694386 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9f84007200 con 0x7f9f8c0ec180 2021-02-10T11:47:38.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.622 7f9f73fff700 1 --2- 172.21.2.101:0/3585694386 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f6c0186c0 0x7f9f6c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:38.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.622 7f9f73fff700 1 -- 172.21.2.101:0/3585694386 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9f84032880 con 0x7f9f8c0ec180 2021-02-10T11:47:38.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.622 7f9f8b07d700 1 --2- 172.21.2.101:0/3585694386 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f6c0186c0 0x7f9f6c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:38.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.622 7f9f8b07d700 1 --2- 172.21.2.101:0/3585694386 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f6c0186c0 0x7f9f6c01abc0 crc :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:38.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.652 7f9f9131f700 1 -- 172.21.2.101:0/3585694386 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f8c0ebc60 con 0x7f9f8c0ec180 2021-02-10T11:47:38.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.655 7f9f73fff700 1 -- 172.21.2.101:0/3585694386 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9f84000f90 con 0x7f9f8c0ec180 2021-02-10T11:47:38.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.784 7f9f9131f700 1 -- 172.21.2.101:0/3585694386 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f8c0f0b10 con 0x7f9f8c0ec180 2021-02-10T11:47:38.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.784 7f9f73fff700 1 -- 172.21.2.101:0/3585694386 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+2108 (secure 0 0 0) 0x7f9f84007200 con 0x7f9f8c0ec180 2021-02-10T11:47:38.782 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:47:38.782 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":156,"default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":13128,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/122652046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":122652046},{"type":"v1","addr":"172.21.2.101:6835","nonce":122652046}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":152}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:47:28.786578","modified":"2021-02-10 11:47:30.795250","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":13125},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13125":{"gid":13125,"name":"b","rank":0,"incarnation":153,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2314483030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2314483030},{"type":"v1","addr":"172.21.2.108:6833","nonce":2314483030}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[39],"metadata_pool":38,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":19}]} 2021-02-10T11:47:38.782 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 156 2021-02-10T11:47:38.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.786 7f9f9131f700 1 -- 172.21.2.101:0/3585694386 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f6c0186c0 msgr2=0x7f9f6c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:38.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.786 7f9f9131f700 1 --2- 172.21.2.101:0/3585694386 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f6c0186c0 0x7f9f6c01abc0 crc :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:38.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.786 7f9f9131f700 1 -- 172.21.2.101:0/3585694386 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f8c0ec180 msgr2=0x7f9f8c0ec580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:38.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.786 7f9f9131f700 1 --2- 172.21.2.101:0/3585694386 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f8c0ec180 0x7f9f8c0ec580 secure :-1 s=READY pgs=3133 cs=0 l=1 rev1=1 rx=0x7f9f8400a940 tx=0x7f9f8400ace0).stop 2021-02-10T11:47:38.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.786 7f9f9131f700 1 -- 172.21.2.101:0/3585694386 shutdown_connections 2021-02-10T11:47:38.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.786 7f9f9131f700 1 --2- 172.21.2.101:0/3585694386 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9f8c0ec180 0x7f9f8c0ec580 unknown :-1 s=CLOSED pgs=3133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:38.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.786 7f9f9131f700 1 --2- 172.21.2.101:0/3585694386 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9f8c0f0680 0x7f9f8c0a2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:38.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.786 7f9f9131f700 1 --2- 172.21.2.101:0/3585694386 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9f6c0186c0 0x7f9f6c01abc0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:38.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.786 7f9f9131f700 1 --2- 172.21.2.101:0/3585694386 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9f8c0f0280 0x7f9f8c0a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:38.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.786 7f9f9131f700 1 -- 172.21.2.101:0/3585694386 >> 172.21.2.101:0/3585694386 conn(0x7f9f8c0c2130 msgr2=0x7f9f8c09e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:38.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.787 7f9f9131f700 1 -- 172.21.2.101:0/3585694386 shutdown_connections 2021-02-10T11:47:38.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:38.787 7f9f9131f700 1 -- 172.21.2.101:0/3585694386 wait complete. 2021-02-10T11:47:38.994 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 156, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:47:28.786578', 'modified': '2021-02-10 11:47:30.795250', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 13125}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_13125': {'gid': 13125, 'name': 'b', 'rank': 0, 'incarnation': 153, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/2314483030', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 2314483030}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 2314483030}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [39], 'metadata_pool': 38, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:47:38.994 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:47:38.994 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:47:39.221 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:47:39.225 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:47:39.393 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:47:39.394 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:47:39.394 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:47:39.394 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 13125, 2021-02-10T11:47:39.394 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:47:39.394 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:47:39.394 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 10.609108147000001, 2021-02-10T11:47:39.394 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 156, 2021-02-10T11:47:39.394 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 183, 2021-02-10T11:47:39.394 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 183, 2021-02-10T11:47:39.394 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 16.257065527999998 2021-02-10T11:47:39.394 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:47:39.397 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 13125, "want_state": "up:active", "state": "up:active", "rank_uptime": 10.609108147000001, "mdsmap_epoch": 156, "osdmap_epoch": 183, "osdmap_epoch_barrier": 183, "uptime": 16.257065527999998 } 2021-02-10T11:47:39.398 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:47:39.398 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-02-10T11:47:39.398 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-02-10T11:47:39.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:39.398 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:47:39.734 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:47:40.079 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:47:40.079 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:47:40.422 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:47:40.423 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:47:40.423 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:47:40.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:40.424 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:47:40.759 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-02-10T11:47:40.759 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:47:40.762 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:47:41.107 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:47:41.107 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:47:41.176 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[177677]: starting ceph client 2021-02-10T11:47:41.176 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:2021-02-10 11:47:41.178 7f628b320f80 -1 init, newargv = 0x5622ff28d1a0 newargc=9 2021-02-10T11:47:41.181 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[177677]: starting fuse 2021-02-10T11:47:41.450 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:47:41.450 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:47:41.451 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:47:41.451 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:41.451 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:47:41.787 INFO:teuthology.orchestra.run.gibba001.stdout:42 2021-02-10T11:47:41.787 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2021-02-10T11:47:41.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:41.788 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:47:41.788 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:47:41.788 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:47:41.788 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:47:41.788 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:47:41.788 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:47:41.788 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:47:41.788 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-02-10T11:47:41.788 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:47:41.788 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:47:41.788 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:47:41.788 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:47:41.789 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:47:41.789 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:47:41.789 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:47:41.789 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:47:41.789 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:47:41.789 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:47:41.789 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:47:41.789 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:47:41.789 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:47:41.789 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.0")) 2021-02-10T11:47:41.789 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:47:42.152 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.0.177677.asok 2021-02-10T11:47:42.153 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.177677.asok 2021-02-10T11:47:42.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:42.154 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.177677.asok status 2021-02-10T11:47:42.544 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:47:42.545 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:47:42.545 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:47:42.545 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:47:42.545 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "0", 2021-02-10T11:47:42.545 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:47:42.545 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-02-10T11:47:42.545 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "177677", 2021-02-10T11:47:42.545 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:47:42.545 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:42.545 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:47:42.545 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:47:42.545 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 13299, 2021-02-10T11:47:42.545 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:47:42.546 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:47:42.546 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:47:42.546 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 13299 2021-02-10T11:47:42.546 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:42.546 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:47:42.546 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:42.546 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:47:42.546 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 320917936 2021-02-10T11:47:42.546 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:42.546 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:42.546 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:47:42.546 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:42.547 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:47:42.547 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 320917936 2021-02-10T11:47:42.547 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:42.547 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.13299 172.21.2.101:0/320917936", 2021-02-10T11:47:42.547 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/320917936", 2021-02-10T11:47:42.547 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:47:42.547 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 156, 2021-02-10T11:47:42.547 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 183, 2021-02-10T11:47:42.547 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:47:42.547 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:47:42.547 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:47:42.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:42.549 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:47:42.549 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:47:42.886 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:47:42.886 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:47:42.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:42.886 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:47:43.233 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:47:43.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.647 7f63a9a21700 1 Processor -- start 2021-02-10T11:47:43.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.647 7f63a9a21700 1 -- start start 2021-02-10T11:47:43.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.648 7f63a9a21700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63a40d0340 0x7f63a40ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:43.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.648 7f63a9a21700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f63a40d1c70 0x7f63a40efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:43.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.648 7f63a9a21700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63a40f0b10 con 0x7f63a40d1870 2021-02-10T11:47:43.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.648 7f63a9a21700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a40e5ed0 con 0x7f63a40d1c70 2021-02-10T11:47:43.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.648 7f63a9a21700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a41b0be0 con 0x7f63a40d0340 2021-02-10T11:47:43.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.648 7f63a37fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f63a40d1870 0x7f63a41b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34784/0 (socket says 172.21.2.101:34784) 2021-02-10T11:47:43.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.648 7f63a37fe700 1 -- 172.21.2.101:0/1172711054 learned_addr learned my addr 172.21.2.101:0/1172711054 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:43.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.648 7f63a2ffd700 1 --2- 172.21.2.101:0/1172711054 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63a40d0340 0x7f63a40ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:43.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.648 7f63a3fff700 1 --2- 172.21.2.101:0/1172711054 >> v2:172.21.2.108:3300/0 conn(0x7f63a40d1c70 0x7f63a40efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:43.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.649 7f63a2ffd700 1 -- 172.21.2.101:0/1172711054 >> v2:172.21.2.108:3300/0 conn(0x7f63a40d1c70 msgr2=0x7f63a40efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:43.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.649 7f63a2ffd700 1 --2- 172.21.2.101:0/1172711054 >> v2:172.21.2.108:3300/0 conn(0x7f63a40d1c70 0x7f63a40efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:43.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.649 7f63a2ffd700 1 -- 172.21.2.101:0/1172711054 >> v1:172.21.2.101:6789/0 conn(0x7f63a40d1870 legacy=0x7f63a41b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:43.662 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.649 7f63a2ffd700 1 -- 172.21.2.101:0/1172711054 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a40f20a0 con 0x7f63a40d0340 2021-02-10T11:47:43.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.649 7f63a27fc700 1 -- 172.21.2.101:0/1172711054 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3183335336 0 0) 0x7f639c001500 con 0x7f63a40d1870 2021-02-10T11:47:43.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.649 7f63a2ffd700 1 --2- 172.21.2.101:0/1172711054 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63a40d0340 0x7f63a40ef640 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f6398006650 tx=0x7f6398009a80).ready entity=mon.2 client_cookie=463444680a1e4cac server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:43.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.649 7f63a27fc700 1 -- 172.21.2.101:0/1172711054 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f639800c0c0 con 0x7f63a40d0340 2021-02-10T11:47:43.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.649 7f63a27fc700 1 -- 172.21.2.101:0/1172711054 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639800c340 con 0x7f63a40d0340 2021-02-10T11:47:43.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.649 7f63a27fc700 1 -- 172.21.2.101:0/1172711054 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f639800c860 con 0x7f63a40d0340 2021-02-10T11:47:43.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.649 7f63a9a21700 1 -- 172.21.2.101:0/1172711054 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63a40d0340 msgr2=0x7f63a40ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:43.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.649 7f63a9a21700 1 --2- 172.21.2.101:0/1172711054 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63a40d0340 0x7f63a40ef640 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f6398006650 tx=0x7f6398009a80).stop 2021-02-10T11:47:43.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.650 7f63a9a21700 1 -- 172.21.2.101:0/1172711054 shutdown_connections 2021-02-10T11:47:43.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.650 7f63a9a21700 1 --2- 172.21.2.101:0/1172711054 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63a40d0340 0x7f63a40ef640 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:43.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.650 7f63a9a21700 1 --2- 172.21.2.101:0/1172711054 >> v2:172.21.2.108:3300/0 conn(0x7f63a40d1c70 0x7f63a40efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:43.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.650 7f63a9a21700 1 -- 172.21.2.101:0/1172711054 >> 172.21.2.101:0/1172711054 conn(0x7f63a40ce450 msgr2=0x7f63a40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:43.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.650 7f63a9a21700 1 -- 172.21.2.101:0/1172711054 shutdown_connections 2021-02-10T11:47:43.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.650 7f63a9a21700 1 -- 172.21.2.101:0/1172711054 wait complete. 2021-02-10T11:47:43.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.664 7f63a9a21700 1 Processor -- start 2021-02-10T11:47:43.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.665 7f63a9a21700 1 -- start start 2021-02-10T11:47:43.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.665 7f63a9a21700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63a41942b0 0x7f63a40b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:43.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.665 7f63a9a21700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63a4195070 0x7f63a4195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:43.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.665 7f63a9a21700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63a40bb8e0 0x7f63a40bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:43.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.665 7f63a9a21700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a41946b0 con 0x7f63a40bb8e0 2021-02-10T11:47:43.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.665 7f63a9a21700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a40b3b70 con 0x7f63a4195070 2021-02-10T11:47:43.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.665 7f63a9a21700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a40b3d90 con 0x7f63a41942b0 2021-02-10T11:47:43.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.665 7f63a3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63a40bb8e0 0x7f63a40bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:43.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.665 7f63a3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63a40bb8e0 0x7f63a40bbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34300/0 (socket says 172.21.2.101:34300) 2021-02-10T11:47:43.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.665 7f63a37fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63a41942b0 0x7f63a40b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:43.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.665 7f63a3fff700 1 -- 172.21.2.101:0/999153667 learned_addr learned my addr 172.21.2.101:0/999153667 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:43.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.665 7f63a2ffd700 1 --2- 172.21.2.101:0/999153667 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63a4195070 0x7f63a4195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:43.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.666 7f63a3fff700 1 -- 172.21.2.101:0/999153667 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63a41942b0 msgr2=0x7f63a40b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:43.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.666 7f63a3fff700 1 --2- 172.21.2.101:0/999153667 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63a41942b0 0x7f63a40b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:43.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.666 7f63a3fff700 1 -- 172.21.2.101:0/999153667 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63a4195070 msgr2=0x7f63a4195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:43.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.666 7f63a3fff700 1 --2- 172.21.2.101:0/999153667 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63a4195070 0x7f63a4195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:43.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.666 7f63a3fff700 1 -- 172.21.2.101:0/999153667 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a41a49e0 con 0x7f63a40bb8e0 2021-02-10T11:47:43.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.666 7f63a3fff700 1 --2- 172.21.2.101:0/999153667 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63a40bb8e0 0x7f63a40bbce0 secure :-1 s=READY pgs=3136 cs=0 l=1 rev1=1 rx=0x7f639400b5e0 tx=0x7f639400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:43.666 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.666 7f638bfff700 1 -- 172.21.2.101:0/999153667 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6394000c10 con 0x7f63a40bb8e0 2021-02-10T11:47:43.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.666 7f638bfff700 1 -- 172.21.2.101:0/999153667 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639400bd70 con 0x7f63a40bb8e0 2021-02-10T11:47:43.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.666 7f638bfff700 1 -- 172.21.2.101:0/999153667 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6394002560 con 0x7f63a40bb8e0 2021-02-10T11:47:43.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.666 7f63a9a21700 1 -- 172.21.2.101:0/999153667 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a41a49e0 con 0x7f63a40bb8e0 2021-02-10T11:47:43.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.666 7f63a9a21700 1 -- 172.21.2.101:0/999153667 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a41a49e0 con 0x7f63a40bb8e0 2021-02-10T11:47:43.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.667 7f638bfff700 1 -- 172.21.2.101:0/999153667 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6394002020 con 0x7f63a40bb8e0 2021-02-10T11:47:43.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.667 7f638bfff700 1 --2- 172.21.2.101:0/999153667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f63840186c0 0x7f638401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:43.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.667 7f638bfff700 1 -- 172.21.2.101:0/999153667 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f63940327d0 con 0x7f63a40bb8e0 2021-02-10T11:47:43.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.667 7f63a37fe700 1 --2- 172.21.2.101:0/999153667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f63840186c0 0x7f638401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:43.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.667 7f63a37fe700 1 --2- 172.21.2.101:0/999153667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f63840186c0 0x7f638401abc0 crc :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:43.694 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.698 7f63a9a21700 1 -- 172.21.2.101:0/999153667 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a40bc5f0 con 0x7f63a40bb8e0 2021-02-10T11:47:43.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.700 7f638bfff700 1 -- 172.21.2.101:0/999153667 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6394006370 con 0x7f63a40bb8e0 2021-02-10T11:47:43.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.826 7f63a9a21700 1 -- 172.21.2.101:0/999153667 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63a40bc5f0 con 0x7f63a40bb8e0 2021-02-10T11:47:43.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.827 7f638bfff700 1 -- 172.21.2.101:0/999153667 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f6394002000 con 0x7f63a40bb8e0 2021-02-10T11:47:43.824 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:47:43.824 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":183,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:47:29.788748","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":39,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":38,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:47:26.756268","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":39,"pool_name":"cephfs_data","create_time":"2021-02-10 11:47:27.781048","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:47:43.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.828 7f63a9a21700 1 -- 172.21.2.101:0/999153667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f63840186c0 msgr2=0x7f638401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:43.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.828 7f63a9a21700 1 --2- 172.21.2.101:0/999153667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f63840186c0 0x7f638401abc0 crc :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:43.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.828 7f63a9a21700 1 -- 172.21.2.101:0/999153667 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63a40bb8e0 msgr2=0x7f63a40bbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:43.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.828 7f63a9a21700 1 --2- 172.21.2.101:0/999153667 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63a40bb8e0 0x7f63a40bbce0 secure :-1 s=READY pgs=3136 cs=0 l=1 rev1=1 rx=0x7f639400b5e0 tx=0x7f639400b980).stop 2021-02-10T11:47:43.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.828 7f63a9a21700 1 -- 172.21.2.101:0/999153667 shutdown_connections 2021-02-10T11:47:43.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.828 7f63a9a21700 1 --2- 172.21.2.101:0/999153667 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f63a40bb8e0 0x7f63a40bbce0 unknown :-1 s=CLOSED pgs=3136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:43.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.828 7f63a9a21700 1 --2- 172.21.2.101:0/999153667 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f63a4195070 0x7f63a4195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:43.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.828 7f63a9a21700 1 --2- 172.21.2.101:0/999153667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f63840186c0 0x7f638401abc0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:43.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.828 7f63a9a21700 1 --2- 172.21.2.101:0/999153667 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f63a41942b0 0x7f63a40b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:43.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.828 7f63a9a21700 1 -- 172.21.2.101:0/999153667 >> 172.21.2.101:0/999153667 conn(0x7f63a40b3040 msgr2=0x7f63a40c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:43.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.829 7f63a9a21700 1 -- 172.21.2.101:0/999153667 shutdown_connections 2021-02-10T11:47:43.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:43.829 7f63a9a21700 1 -- 172.21.2.101:0/999153667 wait complete. 2021-02-10T11:47:43.834 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 38, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:47:26.756268', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '182', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 39, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:47:27.781048', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '183', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:47:43.834 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:47:44.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.255 7f5a686c8700 1 Processor -- start 2021-02-10T11:47:44.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.255 7f5a686c8700 1 -- start start 2021-02-10T11:47:44.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.256 7f5a686c8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5a600d1c70 0x7f5a600cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:44.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.256 7f5a686c8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a600cf580 0x7f5a600cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:44.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.256 7f5a686c8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a600fe1c0 con 0x7f5a600d1870 2021-02-10T11:47:44.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.256 7f5a686c8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a601b0cb0 con 0x7f5a600d1c70 2021-02-10T11:47:44.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.256 7f5a686c8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a60109a60 con 0x7f5a600cf580 2021-02-10T11:47:44.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.256 7f5a65ce3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5a600d1c70 0x7f5a600cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:44.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.256 7f5a65ce3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5a600d1c70 0x7f5a600cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34304/0 (socket says 172.21.2.101:34304) 2021-02-10T11:47:44.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.256 7f5a65ce3700 1 -- 172.21.2.101:0/3616657347 learned_addr learned my addr 172.21.2.101:0/3616657347 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:44.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.256 7f5a65ce3700 1 -- 172.21.2.101:0/3616657347 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a600cf580 msgr2=0x7f5a600cf980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:44.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.256 7f5a65ce3700 1 --2- 172.21.2.101:0/3616657347 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a600cf580 0x7f5a600cf980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:44.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.256 7f5a65ce3700 1 -- 172.21.2.101:0/3616657347 >> v1:172.21.2.108:6789/0 conn(0x7f5a600d1870 legacy=0x7f5a601b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:44.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.256 7f5a65ce3700 1 -- 172.21.2.101:0/3616657347 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a60088070 con 0x7f5a600d1c70 2021-02-10T11:47:44.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.256 7f5a65ce3700 1 --2- 172.21.2.101:0/3616657347 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a600d1c70 0x7f5a600cef60 secure :-1 s=READY pgs=3137 cs=0 l=1 rev1=1 rx=0x7f5a5c006700 tx=0x7f5a5c009b80).ready entity=mon.0 client_cookie=588b20bb49f8ca1f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:44.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.256 7f5a654e2700 1 -- 172.21.2.101:0/3616657347 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5a5c00c0c0 con 0x7f5a600d1c70 2021-02-10T11:47:44.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.256 7f5a654e2700 1 -- 172.21.2.101:0/3616657347 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5c00c340 con 0x7f5a600d1c70 2021-02-10T11:47:44.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.256 7f5a654e2700 1 -- 172.21.2.101:0/3616657347 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5a5c00c820 con 0x7f5a600d1c70 2021-02-10T11:47:44.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.256 7f5a686c8700 1 -- 172.21.2.101:0/3616657347 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a600d1c70 msgr2=0x7f5a600cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:44.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.256 7f5a686c8700 1 --2- 172.21.2.101:0/3616657347 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a600d1c70 0x7f5a600cef60 secure :-1 s=READY pgs=3137 cs=0 l=1 rev1=1 rx=0x7f5a5c006700 tx=0x7f5a5c009b80).stop 2021-02-10T11:47:44.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.257 7f5a686c8700 1 -- 172.21.2.101:0/3616657347 shutdown_connections 2021-02-10T11:47:44.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.257 7f5a686c8700 1 --2- 172.21.2.101:0/3616657347 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a600cf580 0x7f5a600cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:44.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.257 7f5a686c8700 1 --2- 172.21.2.101:0/3616657347 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a600d1c70 0x7f5a600cef60 unknown :-1 s=CLOSED pgs=3137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:44.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.257 7f5a686c8700 1 -- 172.21.2.101:0/3616657347 >> 172.21.2.101:0/3616657347 conn(0x7f5a600ce420 msgr2=0x7f5a600d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:44.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.257 7f5a686c8700 1 -- 172.21.2.101:0/3616657347 shutdown_connections 2021-02-10T11:47:44.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.257 7f5a686c8700 1 -- 172.21.2.101:0/3616657347 wait complete. 2021-02-10T11:47:44.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.272 7f5a686c8700 1 Processor -- start 2021-02-10T11:47:44.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.272 7f5a686c8700 1 -- start start 2021-02-10T11:47:44.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.272 7f5a686c8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5a600c3820 0x7f5a600a7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:44.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.272 7f5a686c8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a601a44c0 0x7f5a601a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:44.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.272 7f5a686c8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a600863d0 0x7f5a600867d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:44.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.272 7f5a686c8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a60085ed0 con 0x7f5a600863d0 2021-02-10T11:47:44.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.272 7f5a686c8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a600c0740 con 0x7f5a600c3820 2021-02-10T11:47:44.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.272 7f5a686c8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a60086df0 con 0x7f5a601a44c0 2021-02-10T11:47:44.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.272 7f5a66ce5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a600863d0 0x7f5a600867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:44.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.272 7f5a66ce5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a600863d0 0x7f5a600867d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34314/0 (socket says 172.21.2.101:34314) 2021-02-10T11:47:44.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.272 7f5a66ce5700 1 -- 172.21.2.101:0/1368613986 learned_addr learned my addr 172.21.2.101:0/1368613986 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:44.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.273 7f5a664e4700 1 --2- 172.21.2.101:0/1368613986 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5a600c3820 0x7f5a600a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:44.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.273 7f5a65ce3700 1 --2- 172.21.2.101:0/1368613986 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a601a44c0 0x7f5a601a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:44.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.273 7f5a66ce5700 1 -- 172.21.2.101:0/1368613986 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a601a44c0 msgr2=0x7f5a601a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:44.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.273 7f5a66ce5700 1 --2- 172.21.2.101:0/1368613986 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a601a44c0 0x7f5a601a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:44.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.273 7f5a66ce5700 1 -- 172.21.2.101:0/1368613986 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5a600c3820 msgr2=0x7f5a600a7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:44.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.273 7f5a66ce5700 1 --2- 172.21.2.101:0/1368613986 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5a600c3820 0x7f5a600a7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:44.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.273 7f5a66ce5700 1 -- 172.21.2.101:0/1368613986 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a60087010 con 0x7f5a600863d0 2021-02-10T11:47:44.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.273 7f5a66ce5700 1 --2- 172.21.2.101:0/1368613986 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a600863d0 0x7f5a600867d0 secure :-1 s=READY pgs=3138 cs=0 l=1 rev1=1 rx=0x7f5a58009fc0 tx=0x7f5a5800bc40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:44.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.273 7f5a52ffd700 1 -- 172.21.2.101:0/1368613986 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5a58003020 con 0x7f5a600863d0 2021-02-10T11:47:44.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.273 7f5a52ffd700 1 -- 172.21.2.101:0/1368613986 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a58005850 con 0x7f5a600863d0 2021-02-10T11:47:44.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.273 7f5a686c8700 1 -- 172.21.2.101:0/1368613986 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a60087010 con 0x7f5a600863d0 2021-02-10T11:47:44.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.273 7f5a52ffd700 1 -- 172.21.2.101:0/1368613986 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5a58005cb0 con 0x7f5a600863d0 2021-02-10T11:47:44.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.273 7f5a686c8700 1 -- 172.21.2.101:0/1368613986 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a60087010 con 0x7f5a600863d0 2021-02-10T11:47:44.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.274 7f5a52ffd700 1 -- 172.21.2.101:0/1368613986 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5a58003020 con 0x7f5a600863d0 2021-02-10T11:47:44.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.274 7f5a52ffd700 1 --2- 172.21.2.101:0/1368613986 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5a48018730 0x7f5a4801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:44.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.274 7f5a52ffd700 1 -- 172.21.2.101:0/1368613986 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5a58032ba0 con 0x7f5a600863d0 2021-02-10T11:47:44.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.274 7f5a664e4700 1 --2- 172.21.2.101:0/1368613986 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5a48018730 0x7f5a4801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:44.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.274 7f5a664e4700 1 --2- 172.21.2.101:0/1368613986 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5a48018730 0x7f5a4801ac10 crc :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:44.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.306 7f5a686c8700 1 -- 172.21.2.101:0/1368613986 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a600fe1c0 con 0x7f5a600863d0 2021-02-10T11:47:44.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.308 7f5a52ffd700 1 -- 172.21.2.101:0/1368613986 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5a58005850 con 0x7f5a600863d0 2021-02-10T11:47:44.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.435 7f5a686c8700 1 -- 172.21.2.101:0/1368613986 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f5a600fe1c0 con 0x7f5a600863d0 2021-02-10T11:47:44.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.435 7f5a52ffd700 1 -- 172.21.2.101:0/1368613986 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v183) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5a58000d10 con 0x7f5a600863d0 2021-02-10T11:47:44.432 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:47:44.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.437 7f5a686c8700 1 -- 172.21.2.101:0/1368613986 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5a48018730 msgr2=0x7f5a4801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:44.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.437 7f5a686c8700 1 --2- 172.21.2.101:0/1368613986 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5a48018730 0x7f5a4801ac10 crc :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:44.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.437 7f5a686c8700 1 -- 172.21.2.101:0/1368613986 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a600863d0 msgr2=0x7f5a600867d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:44.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.437 7f5a686c8700 1 --2- 172.21.2.101:0/1368613986 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a600863d0 0x7f5a600867d0 secure :-1 s=READY pgs=3138 cs=0 l=1 rev1=1 rx=0x7f5a58009fc0 tx=0x7f5a5800bc40).stop 2021-02-10T11:47:44.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.437 7f5a686c8700 1 -- 172.21.2.101:0/1368613986 shutdown_connections 2021-02-10T11:47:44.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.437 7f5a686c8700 1 --2- 172.21.2.101:0/1368613986 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5a48018730 0x7f5a4801ac10 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:44.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.437 7f5a686c8700 1 --2- 172.21.2.101:0/1368613986 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5a601a44c0 0x7f5a601a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:44.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.437 7f5a686c8700 1 --2- 172.21.2.101:0/1368613986 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5a600863d0 0x7f5a600867d0 unknown :-1 s=CLOSED pgs=3138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:44.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.437 7f5a686c8700 1 --2- 172.21.2.101:0/1368613986 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5a600c3820 0x7f5a600a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:44.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.437 7f5a686c8700 1 -- 172.21.2.101:0/1368613986 >> 172.21.2.101:0/1368613986 conn(0x7f5a600872c0 msgr2=0x7f5a60095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:44.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.438 7f5a686c8700 1 -- 172.21.2.101:0/1368613986 shutdown_connections 2021-02-10T11:47:44.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.438 7f5a686c8700 1 -- 172.21.2.101:0/1368613986 wait complete. 2021-02-10T11:47:44.443 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:47:44.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.863 7f8dc253d700 1 Processor -- start 2021-02-10T11:47:44.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.863 7f8dc253d700 1 -- start start 2021-02-10T11:47:44.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.863 7f8dc253d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8dbc0d1870 0x7f8dbc1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:44.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.863 7f8dc253d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8dbc0cff70 0x7f8dbc109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:44.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.863 7f8dc253d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dbc109a60 con 0x7f8dbc1b0ae0 2021-02-10T11:47:44.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.863 7f8dc253d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dbc109750 con 0x7f8dbc0d1870 2021-02-10T11:47:44.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.863 7f8dc253d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc088070 con 0x7f8dbc0cff70 2021-02-10T11:47:44.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.864 7f8dbbfff700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8dbc0d1870 0x7f8dbc1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:44.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.864 7f8dbb7fe700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f8dbc1b0ae0 0x7f8dbc108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:56454/0 (socket says 172.21.2.101:56454) 2021-02-10T11:47:44.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.864 7f8dbb7fe700 1 -- 172.21.2.101:0/4110835583 learned_addr learned my addr 172.21.2.101:0/4110835583 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:44.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.864 7f8dc0b5a700 1 --2- 172.21.2.101:0/4110835583 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8dbc0cff70 0x7f8dbc109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:44.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.864 7f8dbbfff700 1 -- 172.21.2.101:0/4110835583 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8dbc0cff70 msgr2=0x7f8dbc109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:44.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.864 7f8dbbfff700 1 --2- 172.21.2.101:0/4110835583 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8dbc0cff70 0x7f8dbc109130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:44.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.864 7f8dbbfff700 1 -- 172.21.2.101:0/4110835583 >> v1:172.21.2.108:6789/0 conn(0x7f8dbc1b0ae0 legacy=0x7f8dbc108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:44.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.864 7f8dbbfff700 1 -- 172.21.2.101:0/4110835583 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc0d1c70 con 0x7f8dbc0d1870 2021-02-10T11:47:44.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.864 7f8dbbfff700 1 --2- 172.21.2.101:0/4110835583 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8dbc0d1870 0x7f8dbc1b04c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7f8dac006650 tx=0x7f8dac009a80).ready entity=mon.1 client_cookie=d8175d236b6d84e8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:44.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.865 7f8dbaffd700 1 -- 172.21.2.101:0/4110835583 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8dac00c0c0 con 0x7f8dbc0d1870 2021-02-10T11:47:44.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.865 7f8dbaffd700 1 -- 172.21.2.101:0/4110835583 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dac00c340 con 0x7f8dbc0d1870 2021-02-10T11:47:44.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.865 7f8dbaffd700 1 -- 172.21.2.101:0/4110835583 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8dac00c7a0 con 0x7f8dbc0d1870 2021-02-10T11:47:44.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.865 7f8dc253d700 1 -- 172.21.2.101:0/4110835583 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8dbc0d1870 msgr2=0x7f8dbc1b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:44.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.865 7f8dc253d700 1 --2- 172.21.2.101:0/4110835583 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8dbc0d1870 0x7f8dbc1b04c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7f8dac006650 tx=0x7f8dac009a80).stop 2021-02-10T11:47:44.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.865 7f8dc253d700 1 -- 172.21.2.101:0/4110835583 shutdown_connections 2021-02-10T11:47:44.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.865 7f8dc253d700 1 --2- 172.21.2.101:0/4110835583 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8dbc0cff70 0x7f8dbc109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:44.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.865 7f8dc253d700 1 --2- 172.21.2.101:0/4110835583 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8dbc0d1870 0x7f8dbc1b04c0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:44.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.865 7f8dc253d700 1 -- 172.21.2.101:0/4110835583 >> 172.21.2.101:0/4110835583 conn(0x7f8dbc0ce420 msgr2=0x7f8dbc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:44.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.865 7f8dc253d700 1 -- 172.21.2.101:0/4110835583 shutdown_connections 2021-02-10T11:47:44.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.865 7f8dc253d700 1 -- 172.21.2.101:0/4110835583 wait complete. 2021-02-10T11:47:44.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.880 7f8dc253d700 1 Processor -- start 2021-02-10T11:47:44.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.880 7f8dc253d700 1 -- start start 2021-02-10T11:47:44.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.880 7f8dc253d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8dbc191b10 0x7f8dbc0ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:44.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.880 7f8dc253d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8dbc191f10 0x7f8dbc19ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:44.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.880 7f8dc253d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8dbc1a05f0 0x7f8dbc082ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:44.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.880 7f8dc253d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc192440 con 0x7f8dbc191f10 2021-02-10T11:47:44.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.880 7f8dc253d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc1916e0 con 0x7f8dbc1a05f0 2021-02-10T11:47:44.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.880 7f8dc253d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc191900 con 0x7f8dbc191b10 2021-02-10T11:47:44.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.880 7f8dbb7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8dbc191f10 0x7f8dbc19ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:44.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.880 7f8dbb7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8dbc191f10 0x7f8dbc19ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34320/0 (socket says 172.21.2.101:34320) 2021-02-10T11:47:44.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.881 7f8dbb7fe700 1 -- 172.21.2.101:0/345961695 learned_addr learned my addr 172.21.2.101:0/345961695 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:44.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.881 7f8dbbfff700 1 --2- 172.21.2.101:0/345961695 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8dbc191b10 0x7f8dbc0ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:44.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.881 7f8dc0b5a700 1 --2- 172.21.2.101:0/345961695 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8dbc1a05f0 0x7f8dbc082ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:44.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.881 7f8dbb7fe700 1 -- 172.21.2.101:0/345961695 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8dbc191b10 msgr2=0x7f8dbc0ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:44.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.881 7f8dbb7fe700 1 --2- 172.21.2.101:0/345961695 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8dbc191b10 0x7f8dbc0ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:44.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.881 7f8dbb7fe700 1 -- 172.21.2.101:0/345961695 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8dbc1a05f0 msgr2=0x7f8dbc082ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:44.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.881 7f8dbb7fe700 1 --2- 172.21.2.101:0/345961695 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8dbc1a05f0 0x7f8dbc082ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:44.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.881 7f8dbb7fe700 1 -- 172.21.2.101:0/345961695 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc0834f0 con 0x7f8dbc191f10 2021-02-10T11:47:44.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.881 7f8dbb7fe700 1 --2- 172.21.2.101:0/345961695 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8dbc191f10 0x7f8dbc19ffd0 secure :-1 s=READY pgs=3139 cs=0 l=1 rev1=1 rx=0x7f8db0001c60 tx=0x7f8db000a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:44.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.881 7f8db8ff9700 1 -- 172.21.2.101:0/345961695 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8db000acc0 con 0x7f8dbc191f10 2021-02-10T11:47:44.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.881 7f8db8ff9700 1 -- 172.21.2.101:0/345961695 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db0004140 con 0x7f8dbc191f10 2021-02-10T11:47:44.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.882 7f8db8ff9700 1 -- 172.21.2.101:0/345961695 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8db00045a0 con 0x7f8dbc191f10 2021-02-10T11:47:44.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.882 7f8dc253d700 1 -- 172.21.2.101:0/345961695 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dbc0834f0 con 0x7f8dbc191f10 2021-02-10T11:47:44.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.882 7f8dc253d700 1 -- 172.21.2.101:0/345961695 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dbc0834f0 con 0x7f8dbc191f10 2021-02-10T11:47:44.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.883 7f8db8ff9700 1 -- 172.21.2.101:0/345961695 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8db0000d90 con 0x7f8dbc191f10 2021-02-10T11:47:44.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.883 7f8db8ff9700 1 --2- 172.21.2.101:0/345961695 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d9c0186c0 0x7f8d9c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:44.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.884 7f8db8ff9700 1 -- 172.21.2.101:0/345961695 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8db0032530 con 0x7f8dbc191f10 2021-02-10T11:47:44.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.884 7f8dbbfff700 1 --2- 172.21.2.101:0/345961695 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d9c0186c0 0x7f8d9c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:44.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.884 7f8dbbfff700 1 --2- 172.21.2.101:0/345961695 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d9c0186c0 0x7f8d9c01abc0 crc :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:44.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.913 7f8dc253d700 1 -- 172.21.2.101:0/345961695 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dbc191900 con 0x7f8dbc191f10 2021-02-10T11:47:44.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:44.916 7f8db8ff9700 1 -- 172.21.2.101:0/345961695 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8db00009b0 con 0x7f8dbc191f10 2021-02-10T11:47:45.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.045 7f8dc253d700 1 -- 172.21.2.101:0/345961695 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f8dbc191900 con 0x7f8dbc191f10 2021-02-10T11:47:45.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.045 7f8db8ff9700 1 -- 172.21.2.101:0/345961695 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v183) v1 ==== 104+0+10 (secure 0 0 0) 0x7f8db0000d90 con 0x7f8dbc191f10 2021-02-10T11:47:45.042 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:47:45.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.047 7f8dc253d700 1 -- 172.21.2.101:0/345961695 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d9c0186c0 msgr2=0x7f8d9c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:45.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.047 7f8dc253d700 1 --2- 172.21.2.101:0/345961695 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d9c0186c0 0x7f8d9c01abc0 crc :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:45.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.047 7f8dc253d700 1 -- 172.21.2.101:0/345961695 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8dbc191f10 msgr2=0x7f8dbc19ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:45.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.047 7f8dc253d700 1 --2- 172.21.2.101:0/345961695 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8dbc191f10 0x7f8dbc19ffd0 secure :-1 s=READY pgs=3139 cs=0 l=1 rev1=1 rx=0x7f8db0001c60 tx=0x7f8db000a8d0).stop 2021-02-10T11:47:45.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.047 7f8dc253d700 1 -- 172.21.2.101:0/345961695 shutdown_connections 2021-02-10T11:47:45.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.047 7f8dc253d700 1 --2- 172.21.2.101:0/345961695 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8dbc1a05f0 0x7f8dbc082ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:45.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.047 7f8dc253d700 1 --2- 172.21.2.101:0/345961695 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8dbc191f10 0x7f8dbc19ffd0 unknown :-1 s=CLOSED pgs=3139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:45.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.047 7f8dc253d700 1 --2- 172.21.2.101:0/345961695 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d9c0186c0 0x7f8d9c01abc0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:45.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.047 7f8dc253d700 1 --2- 172.21.2.101:0/345961695 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8dbc191b10 0x7f8dbc0ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:45.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.047 7f8dc253d700 1 -- 172.21.2.101:0/345961695 >> 172.21.2.101:0/345961695 conn(0x7f8dbc0a02d0 msgr2=0x7f8dbc09f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:45.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.048 7f8dc253d700 1 -- 172.21.2.101:0/345961695 shutdown_connections 2021-02-10T11:47:45.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.048 7f8dc253d700 1 -- 172.21.2.101:0/345961695 wait complete. 2021-02-10T11:47:45.052 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:47:45.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.471 7f980b5d7700 1 Processor -- start 2021-02-10T11:47:45.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.471 7f980b5d7700 1 -- start start 2021-02-10T11:47:45.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.471 7f980b5d7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f98040d1c70 0x7f98040cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:45.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.471 7f980b5d7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f98040cf580 0x7f98040cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:45.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.471 7f980b5d7700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98040fe1c0 con 0x7f98040d1870 2021-02-10T11:47:45.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.471 7f980b5d7700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98041b0cb0 con 0x7f98040cf580 2021-02-10T11:47:45.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.471 7f980b5d7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9804109a60 con 0x7f98040d1c70 2021-02-10T11:47:45.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.472 7f9809bf4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f98040cf580 0x7f98040cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:45.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.472 7f9809bf4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f98040cf580 0x7f98040cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34332/0 (socket says 172.21.2.101:34332) 2021-02-10T11:47:45.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.472 7f9809bf4700 1 -- 172.21.2.101:0/633316788 learned_addr learned my addr 172.21.2.101:0/633316788 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:45.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.472 7f9808bf2700 1 --2- 172.21.2.101:0/633316788 >> v2:172.21.2.108:3300/0 conn(0x7f98040d1c70 0x7f98040cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:45.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.472 7f9809bf4700 1 -- 172.21.2.101:0/633316788 >> v2:172.21.2.108:3300/0 conn(0x7f98040d1c70 msgr2=0x7f98040cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:45.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.472 7f9809bf4700 1 --2- 172.21.2.101:0/633316788 >> v2:172.21.2.108:3300/0 conn(0x7f98040d1c70 0x7f98040cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:45.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.472 7f9809bf4700 1 -- 172.21.2.101:0/633316788 >> v1:172.21.2.108:6789/0 conn(0x7f98040d1870 legacy=0x7f98041b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:45.488 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.472 7f9809bf4700 1 -- 172.21.2.101:0/633316788 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9804088070 con 0x7f98040cf580 2021-02-10T11:47:45.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.472 7f9809bf4700 1 --2- 172.21.2.101:0/633316788 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98040cf580 0x7f98040cf980 secure :-1 s=READY pgs=3140 cs=0 l=1 rev1=1 rx=0x7f97fc006f00 tx=0x7f97fc009e90).ready entity=mon.0 client_cookie=192c94f3971c045c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:45.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.473 7f97f7fff700 1 -- 172.21.2.101:0/633316788 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f97fc00b300 con 0x7f98040cf580 2021-02-10T11:47:45.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.473 7f97f7fff700 1 -- 172.21.2.101:0/633316788 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97fc00b580 con 0x7f98040cf580 2021-02-10T11:47:45.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.473 7f97f7fff700 1 -- 172.21.2.101:0/633316788 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f97fc00b9d0 con 0x7f98040cf580 2021-02-10T11:47:45.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.473 7f980b5d7700 1 -- 172.21.2.101:0/633316788 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98040cf580 msgr2=0x7f98040cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:45.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.473 7f980b5d7700 1 --2- 172.21.2.101:0/633316788 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98040cf580 0x7f98040cf980 secure :-1 s=READY pgs=3140 cs=0 l=1 rev1=1 rx=0x7f97fc006f00 tx=0x7f97fc009e90).stop 2021-02-10T11:47:45.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.473 7f980b5d7700 1 -- 172.21.2.101:0/633316788 shutdown_connections 2021-02-10T11:47:45.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.473 7f980b5d7700 1 --2- 172.21.2.101:0/633316788 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98040cf580 0x7f98040cf980 unknown :-1 s=CLOSED pgs=3140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:45.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.473 7f980b5d7700 1 --2- 172.21.2.101:0/633316788 >> v2:172.21.2.108:3300/0 conn(0x7f98040d1c70 0x7f98040cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:45.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.473 7f980b5d7700 1 -- 172.21.2.101:0/633316788 >> 172.21.2.101:0/633316788 conn(0x7f98040ce420 msgr2=0x7f98040d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:45.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.473 7f980b5d7700 1 -- 172.21.2.101:0/633316788 shutdown_connections 2021-02-10T11:47:45.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.473 7f980b5d7700 1 -- 172.21.2.101:0/633316788 wait complete. 2021-02-10T11:47:45.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.488 7f980b5d7700 1 Processor -- start 2021-02-10T11:47:45.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.488 7f980b5d7700 1 -- start start 2021-02-10T11:47:45.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.488 7f980b5d7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98040b04d0 0x7f98041a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:45.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.488 7f980b5d7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98041a8090 0x7f98040c23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:45.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.488 7f980b5d7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f980408d680 0x7f98040c29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:45.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.488 7f980b5d7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98040b0a00 con 0x7f98040b04d0 2021-02-10T11:47:45.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.488 7f980b5d7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98040c2fe0 con 0x7f980408d680 2021-02-10T11:47:45.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.488 7f980b5d7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98040c3200 con 0x7f98041a8090 2021-02-10T11:47:45.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.489 7f98093f3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98040b04d0 0x7f98041a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:45.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.489 7f9809bf4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f980408d680 0x7f98040c29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:45.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.489 7f98093f3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98040b04d0 0x7f98041a7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34334/0 (socket says 172.21.2.101:34334) 2021-02-10T11:47:45.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.489 7f98093f3700 1 -- 172.21.2.101:0/4064452611 learned_addr learned my addr 172.21.2.101:0/4064452611 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:45.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.489 7f9808bf2700 1 --2- 172.21.2.101:0/4064452611 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98041a8090 0x7f98040c23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:45.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.489 7f9809bf4700 1 -- 172.21.2.101:0/4064452611 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98041a8090 msgr2=0x7f98040c23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:45.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.489 7f9809bf4700 1 --2- 172.21.2.101:0/4064452611 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98041a8090 0x7f98040c23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:45.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.489 7f9809bf4700 1 -- 172.21.2.101:0/4064452611 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98040b04d0 msgr2=0x7f98041a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:45.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.489 7f9809bf4700 1 --2- 172.21.2.101:0/4064452611 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98040b04d0 0x7f98041a7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:45.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.489 7f9809bf4700 1 -- 172.21.2.101:0/4064452611 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980408e080 con 0x7f980408d680 2021-02-10T11:47:45.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.489 7f9808bf2700 1 --2- 172.21.2.101:0/4064452611 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98041a8090 0x7f98040c23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:47:45.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.489 7f9809bf4700 1 --2- 172.21.2.101:0/4064452611 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f980408d680 0x7f98040c29c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f97fc009980 tx=0x7f97fc00bc70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:45.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.489 7f97f5ffb700 1 -- 172.21.2.101:0/4064452611 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f97fc006d00 con 0x7f980408d680 2021-02-10T11:47:45.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.490 7f97f5ffb700 1 -- 172.21.2.101:0/4064452611 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97fc00bdb0 con 0x7f980408d680 2021-02-10T11:47:45.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.490 7f97f5ffb700 1 -- 172.21.2.101:0/4064452611 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f97fc00c4e0 con 0x7f980408d680 2021-02-10T11:47:45.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.490 7f980b5d7700 1 -- 172.21.2.101:0/4064452611 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980408e080 con 0x7f980408d680 2021-02-10T11:47:45.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.490 7f980b5d7700 1 -- 172.21.2.101:0/4064452611 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980408e080 con 0x7f980408d680 2021-02-10T11:47:45.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.490 7f97f5ffb700 1 -- 172.21.2.101:0/4064452611 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f97fc001270 con 0x7f980408d680 2021-02-10T11:47:45.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.490 7f97f5ffb700 1 --2- 172.21.2.101:0/4064452611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97e4018730 0x7f97e401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:45.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.490 7f97f5ffb700 1 -- 172.21.2.101:0/4064452611 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f97fc032a70 con 0x7f980408d680 2021-02-10T11:47:45.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.490 7f98093f3700 1 --2- 172.21.2.101:0/4064452611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97e4018730 0x7f97e401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:45.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.491 7f98093f3700 1 --2- 172.21.2.101:0/4064452611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97e4018730 0x7f97e401abc0 crc :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:45.518 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.521 7f980b5d7700 1 -- 172.21.2.101:0/4064452611 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9804082a70 con 0x7f980408d680 2021-02-10T11:47:45.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.525 7f97f5ffb700 1 -- 172.21.2.101:0/4064452611 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f97fc003070 con 0x7f980408d680 2021-02-10T11:47:45.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.654 7f980b5d7700 1 -- 172.21.2.101:0/4064452611 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f9804082a70 con 0x7f980408d680 2021-02-10T11:47:45.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.655 7f97f5ffb700 1 -- 172.21.2.101:0/4064452611 <== mon.1 v2:172.21.2.108:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v183) v1 ==== 100+0+10 (secure 0 0 0) 0x7f97fc003070 con 0x7f980408d680 2021-02-10T11:47:45.652 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:47:45.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.656 7f980b5d7700 1 -- 172.21.2.101:0/4064452611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97e4018730 msgr2=0x7f97e401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:45.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.656 7f980b5d7700 1 --2- 172.21.2.101:0/4064452611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97e4018730 0x7f97e401abc0 crc :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:45.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.656 7f980b5d7700 1 -- 172.21.2.101:0/4064452611 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f980408d680 msgr2=0x7f98040c29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:45.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.656 7f980b5d7700 1 --2- 172.21.2.101:0/4064452611 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f980408d680 0x7f98040c29c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f97fc009980 tx=0x7f97fc00bc70).stop 2021-02-10T11:47:45.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.657 7f980b5d7700 1 -- 172.21.2.101:0/4064452611 shutdown_connections 2021-02-10T11:47:45.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.657 7f980b5d7700 1 --2- 172.21.2.101:0/4064452611 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f97e4018730 0x7f97e401abc0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:45.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.657 7f980b5d7700 1 --2- 172.21.2.101:0/4064452611 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f98041a8090 0x7f98040c23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:45.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.657 7f980b5d7700 1 --2- 172.21.2.101:0/4064452611 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f980408d680 0x7f98040c29c0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:45.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.657 7f980b5d7700 1 --2- 172.21.2.101:0/4064452611 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f98040b04d0 0x7f98041a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:45.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.657 7f980b5d7700 1 -- 172.21.2.101:0/4064452611 >> 172.21.2.101:0/4064452611 conn(0x7f98040be530 msgr2=0x7f980409f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:45.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.657 7f980b5d7700 1 -- 172.21.2.101:0/4064452611 shutdown_connections 2021-02-10T11:47:45.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:45.657 7f980b5d7700 1 -- 172.21.2.101:0/4064452611 wait complete. 2021-02-10T11:47:45.662 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:47:45.662 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:47:46.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.080 7feb32705700 1 Processor -- start 2021-02-10T11:47:46.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.080 7feb32705700 1 -- start start 2021-02-10T11:47:46.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.081 7feb32705700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7feb2c0d1870 0x7feb2c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:46.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.081 7feb32705700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feb2c1b0a10 0x7feb2c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:46.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.081 7feb32705700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2c0f0b10 con 0x7feb2c0d1c70 2021-02-10T11:47:46.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.081 7feb32705700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb2c0e5ed0 con 0x7feb2c0d1870 2021-02-10T11:47:46.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.081 7feb32705700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c0f0650 con 0x7feb2c1b0a10 2021-02-10T11:47:46.093 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.081 7feb2bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7feb2c0d1870 0x7feb2c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:46.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.081 7feb2bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7feb2c0d1870 0x7feb2c1b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34342/0 (socket says 172.21.2.101:34342) 2021-02-10T11:47:46.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.081 7feb2bfff700 1 -- 172.21.2.101:0/3271268402 learned_addr learned my addr 172.21.2.101:0/3271268402 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:46.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.081 7feb2bfff700 1 -- 172.21.2.101:0/3271268402 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feb2c1b0a10 msgr2=0x7feb2c0cf330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:46.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.081 7feb2bfff700 1 --2- 172.21.2.101:0/3271268402 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feb2c1b0a10 0x7feb2c0cf330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:46.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.081 7feb2bfff700 1 -- 172.21.2.101:0/3271268402 >> v1:172.21.2.108:6789/0 conn(0x7feb2c0d1c70 legacy=0x7feb2c0cf950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:46.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.081 7feb2bfff700 1 -- 172.21.2.101:0/3271268402 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c0f0870 con 0x7feb2c0d1870 2021-02-10T11:47:46.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.081 7feb2bfff700 1 --2- 172.21.2.101:0/3271268402 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb2c0d1870 0x7feb2c1b03f0 secure :-1 s=READY pgs=3141 cs=0 l=1 rev1=1 rx=0x7feb1c006700 tx=0x7feb1c009b00).ready entity=mon.0 client_cookie=88db9e77f197dd8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:46.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.082 7feb2affd700 1 -- 172.21.2.101:0/3271268402 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feb1c00c0c0 con 0x7feb2c0d1870 2021-02-10T11:47:46.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.082 7feb2affd700 1 -- 172.21.2.101:0/3271268402 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1c00c0c0 con 0x7feb2c0d1870 2021-02-10T11:47:46.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.082 7feb2affd700 1 -- 172.21.2.101:0/3271268402 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feb1c00c400 con 0x7feb2c0d1870 2021-02-10T11:47:46.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.082 7feb32705700 1 -- 172.21.2.101:0/3271268402 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb2c0d1870 msgr2=0x7feb2c1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:46.094 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.082 7feb32705700 1 --2- 172.21.2.101:0/3271268402 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb2c0d1870 0x7feb2c1b03f0 secure :-1 s=READY pgs=3141 cs=0 l=1 rev1=1 rx=0x7feb1c006700 tx=0x7feb1c009b00).stop 2021-02-10T11:47:46.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.082 7feb32705700 1 -- 172.21.2.101:0/3271268402 shutdown_connections 2021-02-10T11:47:46.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.082 7feb32705700 1 --2- 172.21.2.101:0/3271268402 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feb2c1b0a10 0x7feb2c0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:46.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.082 7feb32705700 1 --2- 172.21.2.101:0/3271268402 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb2c0d1870 0x7feb2c1b03f0 unknown :-1 s=CLOSED pgs=3141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:46.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.082 7feb32705700 1 -- 172.21.2.101:0/3271268402 >> 172.21.2.101:0/3271268402 conn(0x7feb2c0ce450 msgr2=0x7feb2c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:46.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.082 7feb32705700 1 -- 172.21.2.101:0/3271268402 shutdown_connections 2021-02-10T11:47:46.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.082 7feb32705700 1 -- 172.21.2.101:0/3271268402 wait complete. 2021-02-10T11:47:46.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.097 7feb32705700 1 Processor -- start 2021-02-10T11:47:46.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.097 7feb32705700 1 -- start start 2021-02-10T11:47:46.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.097 7feb32705700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feb2c1ae550 0x7feb2c0ef260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:46.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.097 7feb32705700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feb2c0ef880 0x7feb2c08a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:46.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.097 7feb32705700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb2c1a73b0 0x7feb2c08aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:46.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.097 7feb32705700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c1a7940 con 0x7feb2c1a73b0 2021-02-10T11:47:46.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.097 7feb32705700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c1a7e40 con 0x7feb2c0ef880 2021-02-10T11:47:46.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.097 7feb32705700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c1ae950 con 0x7feb2c1ae550 2021-02-10T11:47:46.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.097 7feb30d22700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb2c1a73b0 0x7feb2c08aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:46.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.098 7feb30d22700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb2c1a73b0 0x7feb2c08aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34346/0 (socket says 172.21.2.101:34346) 2021-02-10T11:47:46.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.098 7feb30d22700 1 -- 172.21.2.101:0/2794234798 learned_addr learned my addr 172.21.2.101:0/2794234798 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:46.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.098 7feb2b7fe700 1 --2- 172.21.2.101:0/2794234798 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feb2c0ef880 0x7feb2c08a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:46.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.098 7feb30d22700 1 -- 172.21.2.101:0/2794234798 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feb2c1ae550 msgr2=0x7feb2c0ef260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:46.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.098 7feb2bfff700 1 --2- 172.21.2.101:0/2794234798 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feb2c1ae550 0x7feb2c0ef260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:46.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.098 7feb30d22700 1 --2- 172.21.2.101:0/2794234798 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feb2c1ae550 0x7feb2c0ef260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:46.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.098 7feb30d22700 1 -- 172.21.2.101:0/2794234798 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feb2c0ef880 msgr2=0x7feb2c08a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:46.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.098 7feb30d22700 1 --2- 172.21.2.101:0/2794234798 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feb2c0ef880 0x7feb2c08a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:46.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.098 7feb30d22700 1 -- 172.21.2.101:0/2794234798 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c1aeb70 con 0x7feb2c1a73b0 2021-02-10T11:47:46.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.098 7feb2b7fe700 1 --2- 172.21.2.101:0/2794234798 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feb2c0ef880 0x7feb2c08a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:47:46.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.098 7feb2bfff700 1 --2- 172.21.2.101:0/2794234798 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feb2c1ae550 0x7feb2c0ef260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:47:46.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.098 7feb30d22700 1 --2- 172.21.2.101:0/2794234798 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb2c1a73b0 0x7feb2c08aa70 secure :-1 s=READY pgs=3142 cs=0 l=1 rev1=1 rx=0x7feb24009940 tx=0x7feb2400bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:46.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.098 7feb28ff9700 1 -- 172.21.2.101:0/2794234798 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feb24003030 con 0x7feb2c1a73b0 2021-02-10T11:47:46.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.098 7feb28ff9700 1 -- 172.21.2.101:0/2794234798 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb24001270 con 0x7feb2c1a73b0 2021-02-10T11:47:46.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.098 7feb28ff9700 1 -- 172.21.2.101:0/2794234798 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7feb24001750 con 0x7feb2c1a73b0 2021-02-10T11:47:46.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.098 7feb32705700 1 -- 172.21.2.101:0/2794234798 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2c1aeb70 con 0x7feb2c1a73b0 2021-02-10T11:47:46.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.098 7feb32705700 1 -- 172.21.2.101:0/2794234798 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2c1aeb70 con 0x7feb2c1a73b0 2021-02-10T11:47:46.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.099 7feb28ff9700 1 -- 172.21.2.101:0/2794234798 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7feb24001450 con 0x7feb2c1a73b0 2021-02-10T11:47:46.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.099 7feb28ff9700 1 --2- 172.21.2.101:0/2794234798 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feb0c018700 0x7feb0c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:46.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.099 7feb28ff9700 1 -- 172.21.2.101:0/2794234798 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7feb24032b60 con 0x7feb2c1a73b0 2021-02-10T11:47:46.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.099 7feb2bfff700 1 --2- 172.21.2.101:0/2794234798 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feb0c018700 0x7feb0c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:46.099 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.099 7feb2bfff700 1 --2- 172.21.2.101:0/2794234798 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feb0c018700 0x7feb0c01abc0 crc :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:46.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.130 7feb32705700 1 -- 172.21.2.101:0/2794234798 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb2c0ee060 con 0x7feb2c1a73b0 2021-02-10T11:47:46.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.133 7feb28ff9700 1 -- 172.21.2.101:0/2794234798 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7feb2400be30 con 0x7feb2c1a73b0 2021-02-10T11:47:46.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.262 7feb32705700 1 -- 172.21.2.101:0/2794234798 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb2c0ee060 con 0x7feb2c1a73b0 2021-02-10T11:47:46.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.262 7feb28ff9700 1 -- 172.21.2.101:0/2794234798 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+2108 (secure 0 0 0) 0x7feb2401f8a0 con 0x7feb2c1a73b0 2021-02-10T11:47:46.260 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:47:46.260 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":156,"default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":13128,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/122652046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":122652046},{"type":"v1","addr":"172.21.2.101:6835","nonce":122652046}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":152}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:47:28.786578","modified":"2021-02-10 11:47:30.795250","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":13125},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13125":{"gid":13125,"name":"b","rank":0,"incarnation":153,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2314483030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2314483030},{"type":"v1","addr":"172.21.2.108:6833","nonce":2314483030}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[39],"metadata_pool":38,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":19}]} 2021-02-10T11:47:46.260 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 156 2021-02-10T11:47:46.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.264 7feb32705700 1 -- 172.21.2.101:0/2794234798 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feb0c018700 msgr2=0x7feb0c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:46.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.264 7feb32705700 1 --2- 172.21.2.101:0/2794234798 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feb0c018700 0x7feb0c01abc0 crc :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:46.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.264 7feb32705700 1 -- 172.21.2.101:0/2794234798 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb2c1a73b0 msgr2=0x7feb2c08aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:46.261 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.264 7feb32705700 1 --2- 172.21.2.101:0/2794234798 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb2c1a73b0 0x7feb2c08aa70 secure :-1 s=READY pgs=3142 cs=0 l=1 rev1=1 rx=0x7feb24009940 tx=0x7feb2400bcf0).stop 2021-02-10T11:47:46.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.264 7feb32705700 1 -- 172.21.2.101:0/2794234798 shutdown_connections 2021-02-10T11:47:46.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.264 7feb32705700 1 --2- 172.21.2.101:0/2794234798 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7feb2c1a73b0 0x7feb2c08aa70 unknown :-1 s=CLOSED pgs=3142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:46.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.264 7feb32705700 1 --2- 172.21.2.101:0/2794234798 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7feb2c0ef880 0x7feb2c08a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:46.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.264 7feb32705700 1 --2- 172.21.2.101:0/2794234798 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7feb0c018700 0x7feb0c01abc0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:46.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.264 7feb32705700 1 --2- 172.21.2.101:0/2794234798 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7feb2c1ae550 0x7feb2c0ef260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:46.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.264 7feb32705700 1 -- 172.21.2.101:0/2794234798 >> 172.21.2.101:0/2794234798 conn(0x7feb2c1a8c90 msgr2=0x7feb2c1b38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:46.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.265 7feb32705700 1 -- 172.21.2.101:0/2794234798 shutdown_connections 2021-02-10T11:47:46.262 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:46.265 7feb32705700 1 -- 172.21.2.101:0/2794234798 wait complete. 2021-02-10T11:47:46.471 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 156, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:47:28.786578', 'modified': '2021-02-10 11:47:30.795250', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 13125}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_13125': {'gid': 13125, 'name': 'b', 'rank': 0, 'incarnation': 153, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/2314483030', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 2314483030}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 2314483030}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [39], 'metadata_pool': 38, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:47:46.471 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:47:46.471 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:47:46.870 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:47:46.870 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:47:46.870 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:47:46.870 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 13125, 2021-02-10T11:47:46.870 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:47:46.871 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:47:46.871 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 18.085638012, 2021-02-10T11:47:46.871 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 156, 2021-02-10T11:47:46.871 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 183, 2021-02-10T11:47:46.871 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 183, 2021-02-10T11:47:46.871 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 23.733596502000001 2021-02-10T11:47:46.871 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:47:46.874 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 13125, "want_state": "up:active", "state": "up:active", "rank_uptime": 18.085638012, "mdsmap_epoch": 156, "osdmap_epoch": 183, "osdmap_epoch_barrier": 183, "uptime": 23.733596502000001 } 2021-02-10T11:47:46.874 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:47:46.874 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-02-10T11:47:46.874 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:47:46.875 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:46.875 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:47:47.213 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:47:47.558 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:47:47.558 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:47:47.904 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:47:47.904 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:47:47.905 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:47:47.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:47.906 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:47:48.242 INFO:teuthology.orchestra.run.gibba001.stdout:42 2021-02-10T11:47:48.242 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2021-02-10T11:47:48.243 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:47:48.246 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:47:48.590 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:47:48.591 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:47:48.657 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[178317]: starting ceph client 2021-02-10T11:47:48.657 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:2021-02-10 11:47:48.660 7f8d6cdaff80 -1 init, newargv = 0x556416e9e1a0 newargc=9 2021-02-10T11:47:48.662 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[178317]: starting fuse 2021-02-10T11:47:48.933 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:47:48.933 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:47:48.934 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:47:48.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:48.934 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:47:49.269 INFO:teuthology.orchestra.run.gibba001.stdout:42 2021-02-10T11:47:49.269 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:47:49.269 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2021-02-10T11:47:49.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:49.270 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:47:49.270 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:47:49.270 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:47:49.270 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:47:49.270 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:47:49.270 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:47:49.270 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:47:49.270 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-02-10T11:47:49.270 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:47:49.270 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:47:49.270 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:47:49.270 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:47:49.270 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:47:49.271 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:47:49.271 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:47:49.271 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:47:49.271 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:47:49.271 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:47:49.271 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:47:49.271 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:47:49.271 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:47:49.271 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.1")) 2021-02-10T11:47:49.271 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:47:49.632 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.1.178317.asok 2021-02-10T11:47:49.635 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.178317.asok 2021-02-10T11:47:49.635 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:49.635 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.178317.asok status 2021-02-10T11:47:50.026 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:47:50.026 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:47:50.027 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:47:50.027 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:47:50.027 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "1", 2021-02-10T11:47:50.027 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:47:50.027 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:47:50.027 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "178317", 2021-02-10T11:47:50.027 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:47:50.027 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:50.027 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:47:50.027 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:47:50.027 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 13335, 2021-02-10T11:47:50.027 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:47:50.027 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:47:50.028 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:47:50.028 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 13335 2021-02-10T11:47:50.028 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:50.028 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:47:50.028 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:50.028 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:47:50.028 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 2185477978 2021-02-10T11:47:50.028 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:50.028 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:50.028 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:47:50.028 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:47:50.028 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:47:50.028 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 2185477978 2021-02-10T11:47:50.029 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:47:50.029 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.13335 172.21.2.101:0/2185477978", 2021-02-10T11:47:50.029 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/2185477978", 2021-02-10T11:47:50.029 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:47:50.029 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 156, 2021-02-10T11:47:50.029 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 183, 2021-02-10T11:47:50.029 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:47:50.029 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:47:50.029 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:47:50.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:50.030 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:47:50.031 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:47:50.367 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:47:50.368 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:47:50.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:50.368 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:47:50.714 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2021-02-10T11:47:51.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.134 7f1fe98c0700 1 Processor -- start 2021-02-10T11:47:51.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.134 7f1fe98c0700 1 -- start start 2021-02-10T11:47:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.134 7f1fe98c0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1fe40d1880 0x7f1fe41b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.134 7f1fe98c0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fe41b0af0 0x7f1fe4108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.134 7f1fe98c0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe40fe1d0 con 0x7f1fe40d1c80 2021-02-10T11:47:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.134 7f1fe98c0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe4109a70 con 0x7f1fe40d1880 2021-02-10T11:47:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.134 7f1fe98c0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe4109760 con 0x7f1fe41b0af0 2021-02-10T11:47:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.135 7f1fe3574700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1fe40d1880 0x7f1fe41b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.135 7f1fe3574700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1fe40d1880 0x7f1fe41b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34366/0 (socket says 172.21.2.101:34366) 2021-02-10T11:47:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.135 7f1fe3574700 1 -- 172.21.2.101:0/1272964261 learned_addr learned my addr 172.21.2.101:0/1272964261 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.135 7f1fe2d73700 1 --2- 172.21.2.101:0/1272964261 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fe41b0af0 0x7f1fe4108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.135 7f1fe3574700 1 -- 172.21.2.101:0/1272964261 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fe41b0af0 msgr2=0x7f1fe4108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.135 7f1fe3574700 1 --2- 172.21.2.101:0/1272964261 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fe41b0af0 0x7f1fe4108950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:51.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.135 7f1fe3574700 1 -- 172.21.2.101:0/1272964261 >> v1:172.21.2.108:6789/0 conn(0x7f1fe40d1c80 legacy=0x7f1fe4108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:51.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.135 7f1fe3574700 1 -- 172.21.2.101:0/1272964261 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe4088070 con 0x7f1fe40d1880 2021-02-10T11:47:51.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.135 7f1fe3574700 1 --2- 172.21.2.101:0/1272964261 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fe40d1880 0x7f1fe41b04d0 secure :-1 s=READY pgs=3146 cs=0 l=1 rev1=1 rx=0x7f1fcc006700 tx=0x7f1fcc009b00).ready entity=mon.0 client_cookie=2aa08d64fac43d38 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:51.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.135 7f1fe2572700 1 -- 172.21.2.101:0/1272964261 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1fcc00c0c0 con 0x7f1fe40d1880 2021-02-10T11:47:51.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.135 7f1fe2572700 1 -- 172.21.2.101:0/1272964261 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fcc00c340 con 0x7f1fe40d1880 2021-02-10T11:47:51.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.135 7f1fe2572700 1 -- 172.21.2.101:0/1272964261 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1fcc00c7a0 con 0x7f1fe40d1880 2021-02-10T11:47:51.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.136 7f1fe98c0700 1 -- 172.21.2.101:0/1272964261 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fe40d1880 msgr2=0x7f1fe41b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:51.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.136 7f1fe98c0700 1 --2- 172.21.2.101:0/1272964261 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fe40d1880 0x7f1fe41b04d0 secure :-1 s=READY pgs=3146 cs=0 l=1 rev1=1 rx=0x7f1fcc006700 tx=0x7f1fcc009b00).stop 2021-02-10T11:47:51.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.136 7f1fe98c0700 1 -- 172.21.2.101:0/1272964261 shutdown_connections 2021-02-10T11:47:51.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.136 7f1fe98c0700 1 --2- 172.21.2.101:0/1272964261 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fe41b0af0 0x7f1fe4108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:51.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.136 7f1fe98c0700 1 --2- 172.21.2.101:0/1272964261 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fe40d1880 0x7f1fe41b04d0 unknown :-1 s=CLOSED pgs=3146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:51.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.136 7f1fe98c0700 1 -- 172.21.2.101:0/1272964261 >> 172.21.2.101:0/1272964261 conn(0x7f1fe40ce430 msgr2=0x7f1fe40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:51.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.136 7f1fe98c0700 1 -- 172.21.2.101:0/1272964261 shutdown_connections 2021-02-10T11:47:51.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.136 7f1fe98c0700 1 -- 172.21.2.101:0/1272964261 wait complete. 2021-02-10T11:47:51.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.151 7f1fe98c0700 1 Processor -- start 2021-02-10T11:47:51.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.151 7f1fe98c0700 1 -- start start 2021-02-10T11:47:51.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.151 7f1fe98c0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fe40aef00 0x7f1fe4099c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:51.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.151 7f1fe98c0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fe409a260 0x7f1fe409a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:51.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.151 7f1fe98c0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1fe409ac80 0x7f1fe419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:51.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.151 7f1fe98c0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe419bdd0 con 0x7f1fe409a260 2021-02-10T11:47:51.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.151 7f1fe98c0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe419de70 con 0x7f1fe409ac80 2021-02-10T11:47:51.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.151 7f1fe98c0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe419a2d0 con 0x7f1fe40aef00 2021-02-10T11:47:51.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.151 7f1fe2d73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fe409a260 0x7f1fe409a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:51.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.151 7f1fe2d73700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fe409a260 0x7f1fe409a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34374/0 (socket says 172.21.2.101:34374) 2021-02-10T11:47:51.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.151 7f1fe2d73700 1 -- 172.21.2.101:0/3577460775 learned_addr learned my addr 172.21.2.101:0/3577460775 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:51.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.152 7f1fe3d75700 1 --2- 172.21.2.101:0/3577460775 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1fe409ac80 0x7f1fe419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:51.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.152 7f1fe3574700 1 --2- 172.21.2.101:0/3577460775 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fe40aef00 0x7f1fe4099c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:51.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.152 7f1fe2d73700 1 -- 172.21.2.101:0/3577460775 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fe40aef00 msgr2=0x7f1fe4099c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:51.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.152 7f1fe2d73700 1 --2- 172.21.2.101:0/3577460775 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fe40aef00 0x7f1fe4099c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:51.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.152 7f1fe2d73700 1 -- 172.21.2.101:0/3577460775 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1fe409ac80 msgr2=0x7f1fe419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:51.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.152 7f1fe2d73700 1 --2- 172.21.2.101:0/3577460775 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1fe409ac80 0x7f1fe419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:51.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.152 7f1fe2d73700 1 -- 172.21.2.101:0/3577460775 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe4199f50 con 0x7f1fe409a260 2021-02-10T11:47:51.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.152 7f1fe2d73700 1 --2- 172.21.2.101:0/3577460775 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fe409a260 0x7f1fe409a660 secure :-1 s=READY pgs=3147 cs=0 l=1 rev1=1 rx=0x7f1fd4006be0 tx=0x7f1fd400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:51.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.152 7f1fdbfff700 1 -- 172.21.2.101:0/3577460775 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1fd40064d0 con 0x7f1fe409a260 2021-02-10T11:47:51.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.152 7f1fdbfff700 1 -- 172.21.2.101:0/3577460775 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd400be10 con 0x7f1fe409a260 2021-02-10T11:47:51.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.152 7f1fdbfff700 1 -- 172.21.2.101:0/3577460775 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1fd4002030 con 0x7f1fe409a260 2021-02-10T11:47:51.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.152 7f1fe98c0700 1 -- 172.21.2.101:0/3577460775 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe4199f50 con 0x7f1fe409a260 2021-02-10T11:47:51.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.152 7f1fe98c0700 1 -- 172.21.2.101:0/3577460775 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe419b160 con 0x7f1fe409a260 2021-02-10T11:47:51.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.153 7f1fdbfff700 1 -- 172.21.2.101:0/3577460775 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1fd40052e0 con 0x7f1fe409a260 2021-02-10T11:47:51.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.153 7f1fdbfff700 1 --2- 172.21.2.101:0/3577460775 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1fc8018730 0x7f1fc801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:51.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.153 7f1fdbfff700 1 -- 172.21.2.101:0/3577460775 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1fd40327b0 con 0x7f1fe409a260 2021-02-10T11:47:51.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.153 7f1fe3574700 1 --2- 172.21.2.101:0/3577460775 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1fc8018730 0x7f1fc801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:51.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.153 7f1fe3574700 1 --2- 172.21.2.101:0/3577460775 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1fc8018730 0x7f1fc801abc0 crc :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:51.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.184 7f1fe98c0700 1 -- 172.21.2.101:0/3577460775 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe4082a70 con 0x7f1fe409a260 2021-02-10T11:47:51.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.187 7f1fdbfff700 1 -- 172.21.2.101:0/3577460775 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1fd4006d00 con 0x7f1fe409a260 2021-02-10T11:47:51.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.323 7f1fe98c0700 1 -- 172.21.2.101:0/3577460775 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1fe4199f50 con 0x7f1fe409a260 2021-02-10T11:47:51.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.948 7f1fdbfff700 1 -- 172.21.2.101:0/3577460775 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v157) v1 ==== 136+0+0 (secure 0 0 0) 0x7f1fd40052e0 con 0x7f1fe409a260 2021-02-10T11:47:51.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.993 7f1fe98c0700 1 -- 172.21.2.101:0/3577460775 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1fc8018730 msgr2=0x7f1fc801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:51.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.993 7f1fe98c0700 1 --2- 172.21.2.101:0/3577460775 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1fc8018730 0x7f1fc801abc0 crc :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:51.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.993 7f1fe98c0700 1 -- 172.21.2.101:0/3577460775 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fe409a260 msgr2=0x7f1fe409a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:51.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.993 7f1fe98c0700 1 --2- 172.21.2.101:0/3577460775 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fe409a260 0x7f1fe409a660 secure :-1 s=READY pgs=3147 cs=0 l=1 rev1=1 rx=0x7f1fd4006be0 tx=0x7f1fd400b990).stop 2021-02-10T11:47:51.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.993 7f1fe98c0700 1 -- 172.21.2.101:0/3577460775 shutdown_connections 2021-02-10T11:47:51.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.993 7f1fe98c0700 1 --2- 172.21.2.101:0/3577460775 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1fe409ac80 0x7f1fe419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:51.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.993 7f1fe98c0700 1 --2- 172.21.2.101:0/3577460775 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1fe409a260 0x7f1fe409a660 unknown :-1 s=CLOSED pgs=3147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:51.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.993 7f1fe98c0700 1 --2- 172.21.2.101:0/3577460775 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1fc8018730 0x7f1fc801abc0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:51.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.993 7f1fe98c0700 1 --2- 172.21.2.101:0/3577460775 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1fe40aef00 0x7f1fe4099c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:51.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.993 7f1fe98c0700 1 -- 172.21.2.101:0/3577460775 >> 172.21.2.101:0/3577460775 conn(0x7f1fe418e810 msgr2=0x7f1fe418c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:51.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.994 7f1fe98c0700 1 -- 172.21.2.101:0/3577460775 shutdown_connections 2021-02-10T11:47:51.992 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:51.994 7f1fe98c0700 1 -- 172.21.2.101:0/3577460775 wait complete. 2021-02-10T11:47:51.998 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs volume ls 2021-02-10T11:47:52.426 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.413 7fd2074b3700 1 Processor -- start 2021-02-10T11:47:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.413 7fd2074b3700 1 -- start start 2021-02-10T11:47:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.414 7fd2074b3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd2000d1870 0x7fd2001b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.414 7fd2074b3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd2001b0a10 0x7fd2000cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.414 7fd2074b3700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2000f0b10 con 0x7fd2000d1c70 2021-02-10T11:47:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.414 7fd2074b3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2000e5ed0 con 0x7fd2000d1870 2021-02-10T11:47:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.414 7fd2074b3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2000f0650 con 0x7fd2001b0a10 2021-02-10T11:47:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.414 7fd2052cf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd2000d1870 0x7fd2001b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.414 7fd2052cf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd2000d1870 0x7fd2001b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34378/0 (socket says 172.21.2.101:34378) 2021-02-10T11:47:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.414 7fd2052cf700 1 -- 172.21.2.101:0/3852947376 learned_addr learned my addr 172.21.2.101:0/3852947376 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.414 7fd2052cf700 1 -- 172.21.2.101:0/3852947376 >> v2:172.21.2.108:3300/0 conn(0x7fd2001b0a10 msgr2=0x7fd2000cf330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.414 7fd2052cf700 1 --2- 172.21.2.101:0/3852947376 >> v2:172.21.2.108:3300/0 conn(0x7fd2001b0a10 0x7fd2000cf330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:52.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.414 7fd2052cf700 1 -- 172.21.2.101:0/3852947376 >> v1:172.21.2.108:6789/0 conn(0x7fd2000d1c70 legacy=0x7fd2000cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:52.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.414 7fd2052cf700 1 -- 172.21.2.101:0/3852947376 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2000f0840 con 0x7fd2000d1870 2021-02-10T11:47:52.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.415 7fd2052cf700 1 --2- 172.21.2.101:0/3852947376 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2000d1870 0x7fd2001b03f0 secure :-1 s=READY pgs=3148 cs=0 l=1 rev1=1 rx=0x7fd1fc006700 tx=0x7fd1fc009b80).ready entity=mon.0 client_cookie=b94cb00b4e439116 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:52.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.415 7fd1f3fff700 1 -- 172.21.2.101:0/3852947376 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd1fc00c0c0 con 0x7fd2000d1870 2021-02-10T11:47:52.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.415 7fd1f3fff700 1 -- 172.21.2.101:0/3852947376 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1fc00c440 con 0x7fd2000d1870 2021-02-10T11:47:52.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.415 7fd1f3fff700 1 -- 172.21.2.101:0/3852947376 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd1fc00c960 con 0x7fd2000d1870 2021-02-10T11:47:52.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.415 7fd2074b3700 1 -- 172.21.2.101:0/3852947376 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2000d1870 msgr2=0x7fd2001b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:52.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.415 7fd2074b3700 1 --2- 172.21.2.101:0/3852947376 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2000d1870 0x7fd2001b03f0 secure :-1 s=READY pgs=3148 cs=0 l=1 rev1=1 rx=0x7fd1fc006700 tx=0x7fd1fc009b80).stop 2021-02-10T11:47:52.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.415 7fd2074b3700 1 -- 172.21.2.101:0/3852947376 shutdown_connections 2021-02-10T11:47:52.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.415 7fd2074b3700 1 --2- 172.21.2.101:0/3852947376 >> v2:172.21.2.108:3300/0 conn(0x7fd2001b0a10 0x7fd2000cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:52.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.415 7fd2074b3700 1 --2- 172.21.2.101:0/3852947376 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2000d1870 0x7fd2001b03f0 unknown :-1 s=CLOSED pgs=3148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:52.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.415 7fd2074b3700 1 -- 172.21.2.101:0/3852947376 >> 172.21.2.101:0/3852947376 conn(0x7fd2000ce450 msgr2=0x7fd2000d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:52.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.416 7fd2074b3700 1 -- 172.21.2.101:0/3852947376 shutdown_connections 2021-02-10T11:47:52.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.416 7fd2074b3700 1 -- 172.21.2.101:0/3852947376 wait complete. 2021-02-10T11:47:52.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.430 7fd2074b3700 1 Processor -- start 2021-02-10T11:47:52.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.430 7fd2074b3700 1 -- start start 2021-02-10T11:47:52.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.431 7fd2074b3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2000bb5d0 0x7fd2000bfac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:52.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.431 7fd2074b3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2000a9b50 0x7fd2000a9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:52.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.431 7fd2074b3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2001a7550 0x7fd2001a7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:52.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.431 7fd2074b3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000bb9d0 con 0x7fd2000bb5d0 2021-02-10T11:47:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.431 7fd2074b3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000aa570 con 0x7fd2001a7550 2021-02-10T11:47:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.431 7fd2074b3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000aa790 con 0x7fd2000a9b50 2021-02-10T11:47:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.431 7fd2052cf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2000bb5d0 0x7fd2000bfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.431 7fd2052cf700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2000bb5d0 0x7fd2000bfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34380/0 (socket says 172.21.2.101:34380) 2021-02-10T11:47:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.431 7fd2052cf700 1 -- 172.21.2.101:0/855010430 learned_addr learned my addr 172.21.2.101:0/855010430 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.431 7fd2052cf700 1 -- 172.21.2.101:0/855010430 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2000a9b50 msgr2=0x7fd2000a9f50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:47:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.431 7fd2052cf700 1 --2- 172.21.2.101:0/855010430 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2000a9b50 0x7fd2000a9f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.431 7fd2052cf700 1 -- 172.21.2.101:0/855010430 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2001a7550 msgr2=0x7fd2001a7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.431 7fd2052cf700 1 --2- 172.21.2.101:0/855010430 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2001a7550 0x7fd2001a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.431 7fd2052cf700 1 -- 172.21.2.101:0/855010430 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2001a7f70 con 0x7fd2000bb5d0 2021-02-10T11:47:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.431 7fd2052cf700 1 --2- 172.21.2.101:0/855010430 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2000bb5d0 0x7fd2000bfac0 secure :-1 s=READY pgs=3149 cs=0 l=1 rev1=1 rx=0x7fd1fc00b580 tx=0x7fd1fc00b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.432 7fd1f1ffb700 1 -- 172.21.2.101:0/855010430 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd1fc00bd10 con 0x7fd2000bb5d0 2021-02-10T11:47:52.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.432 7fd1f1ffb700 1 -- 172.21.2.101:0/855010430 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1fc002030 con 0x7fd2000bb5d0 2021-02-10T11:47:52.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.432 7fd1f1ffb700 1 -- 172.21.2.101:0/855010430 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd1fc002400 con 0x7fd2000bb5d0 2021-02-10T11:47:52.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.432 7fd2074b3700 1 -- 172.21.2.101:0/855010430 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2001a7f70 con 0x7fd2000bb5d0 2021-02-10T11:47:52.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.432 7fd2074b3700 1 -- 172.21.2.101:0/855010430 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2001a7f70 con 0x7fd2000bb5d0 2021-02-10T11:47:52.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.432 7fd1f1ffb700 1 -- 172.21.2.101:0/855010430 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd1fc000c90 con 0x7fd2000bb5d0 2021-02-10T11:47:52.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.432 7fd1f1ffb700 1 --2- 172.21.2.101:0/855010430 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1e00186c0 0x7fd1e001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:52.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.432 7fd1f1ffb700 1 -- 172.21.2.101:0/855010430 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd1fc032800 con 0x7fd2000bb5d0 2021-02-10T11:47:52.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.433 7fd204ace700 1 --2- 172.21.2.101:0/855010430 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1e00186c0 0x7fd1e001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:52.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.433 7fd204ace700 1 --2- 172.21.2.101:0/855010430 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1e00186c0 0x7fd1e001abc0 crc :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:52.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.464 7fd2074b3700 1 -- 172.21.2.101:0/855010430 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2000ee0b0 con 0x7fd2000bb5d0 2021-02-10T11:47:52.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.466 7fd1f1ffb700 1 -- 172.21.2.101:0/855010430 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd1fc01f540 con 0x7fd2000bb5d0 2021-02-10T11:47:52.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.602 7fd2074b3700 1 -- 172.21.2.101:0/855010430 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs volume ls", "target": ["mgr", ""]}) v1 -- 0x7fd2000aa570 con 0x7fd1e00186c0 2021-02-10T11:47:52.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.603 7fd1f1ffb700 1 -- 172.21.2.101:0/855010430 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7fd1f4006970 con 0x7fd1e00186c0 2021-02-10T11:47:52.599 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:47:52.599 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:47:52.599 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "cephfs" 2021-02-10T11:47:52.599 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:47:52.599 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:47:52.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.605 7fd2074b3700 1 -- 172.21.2.101:0/855010430 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1e00186c0 msgr2=0x7fd1e001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:52.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.605 7fd2074b3700 1 --2- 172.21.2.101:0/855010430 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1e00186c0 0x7fd1e001abc0 crc :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:52.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.605 7fd2074b3700 1 -- 172.21.2.101:0/855010430 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2000bb5d0 msgr2=0x7fd2000bfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:52.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.605 7fd2074b3700 1 --2- 172.21.2.101:0/855010430 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2000bb5d0 0x7fd2000bfac0 secure :-1 s=READY pgs=3149 cs=0 l=1 rev1=1 rx=0x7fd1fc00b580 tx=0x7fd1fc00b920).stop 2021-02-10T11:47:52.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.605 7fd2074b3700 1 -- 172.21.2.101:0/855010430 shutdown_connections 2021-02-10T11:47:52.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.605 7fd2074b3700 1 --2- 172.21.2.101:0/855010430 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1e00186c0 0x7fd1e001abc0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:52.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.605 7fd2074b3700 1 --2- 172.21.2.101:0/855010430 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2000a9b50 0x7fd2000a9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:52.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.605 7fd2074b3700 1 --2- 172.21.2.101:0/855010430 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2001a7550 0x7fd2001a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:52.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.605 7fd2074b3700 1 --2- 172.21.2.101:0/855010430 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2000bb5d0 0x7fd2000bfac0 unknown :-1 s=CLOSED pgs=3149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:52.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.605 7fd2074b3700 1 -- 172.21.2.101:0/855010430 >> 172.21.2.101:0/855010430 conn(0x7fd2000a6eb0 msgr2=0x7fd2000bf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:52.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.605 7fd2074b3700 1 -- 172.21.2.101:0/855010430 shutdown_connections 2021-02-10T11:47:52.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:52.605 7fd2074b3700 1 -- 172.21.2.101:0/855010430 wait complete. 2021-02-10T11:47:52.609 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:47:53.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.030 7f801ab51700 1 Processor -- start 2021-02-10T11:47:53.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.030 7f801ab51700 1 -- start start 2021-02-10T11:47:53.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.031 7f801ab51700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f80140d1c70 0x7f80140cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:53.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.031 7f801ab51700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f80140cf580 0x7f80140cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:53.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.031 7f801ab51700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80140fe1c0 con 0x7f80140d1870 2021-02-10T11:47:53.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.031 7f801ab51700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80141b0cb0 con 0x7f80140cf580 2021-02-10T11:47:53.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.031 7f801ab51700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8014109a60 con 0x7f80140d1c70 2021-02-10T11:47:53.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.031 7f801896d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f80140d1870 0x7f80141b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34882/0 (socket says 172.21.2.101:34882) 2021-02-10T11:47:53.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.031 7f801896d700 1 -- 172.21.2.101:0/3655759479 learned_addr learned my addr 172.21.2.101:0/3655759479 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:53.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.031 7f801916e700 1 --2- 172.21.2.101:0/3655759479 >> v2:172.21.2.101:3300/0 conn(0x7f80140cf580 0x7f80140cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:53.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.031 7f801916e700 1 -- 172.21.2.101:0/3655759479 >> v2:172.21.2.108:3300/0 conn(0x7f80140d1c70 msgr2=0x7f80140cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:53.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.031 7f80137fe700 1 -- 172.21.2.101:0/3655759479 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2454820608 0 0) 0x7f8004001180 con 0x7f80140d1870 2021-02-10T11:47:53.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.031 7f8013fff700 1 --2- 172.21.2.101:0/3655759479 >> v2:172.21.2.108:3300/0 conn(0x7f80140d1c70 0x7f80140cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:53.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.031 7f801916e700 1 --2- 172.21.2.101:0/3655759479 >> v2:172.21.2.108:3300/0 conn(0x7f80140d1c70 0x7f80140cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:53.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.031 7f801916e700 1 -- 172.21.2.101:0/3655759479 >> v1:172.21.2.101:6789/0 conn(0x7f80140d1870 legacy=0x7f80141b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:53.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.031 7f801916e700 1 -- 172.21.2.101:0/3655759479 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8014088070 con 0x7f80140cf580 2021-02-10T11:47:53.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.031 7f8013fff700 1 --2- 172.21.2.101:0/3655759479 >> v2:172.21.2.108:3300/0 conn(0x7f80140d1c70 0x7f80140cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:47:53.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.031 7f801916e700 1 --2- 172.21.2.101:0/3655759479 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80140cf580 0x7f80140cf980 secure :-1 s=READY pgs=3151 cs=0 l=1 rev1=1 rx=0x7f800c006e30 tx=0x7f800c009e60).ready entity=mon.0 client_cookie=ccd0a3be781d3325 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:53.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.032 7f80137fe700 1 -- 172.21.2.101:0/3655759479 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f800c000f90 con 0x7f80140cf580 2021-02-10T11:47:53.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.032 7f80137fe700 1 -- 172.21.2.101:0/3655759479 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800c00b300 con 0x7f80140cf580 2021-02-10T11:47:53.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.032 7f80137fe700 1 -- 172.21.2.101:0/3655759479 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f800c00b820 con 0x7f80140cf580 2021-02-10T11:47:53.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.032 7f801ab51700 1 -- 172.21.2.101:0/3655759479 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80140cf580 msgr2=0x7f80140cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:53.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.032 7f801ab51700 1 --2- 172.21.2.101:0/3655759479 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80140cf580 0x7f80140cf980 secure :-1 s=READY pgs=3151 cs=0 l=1 rev1=1 rx=0x7f800c006e30 tx=0x7f800c009e60).stop 2021-02-10T11:47:53.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.032 7f801ab51700 1 -- 172.21.2.101:0/3655759479 shutdown_connections 2021-02-10T11:47:53.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.032 7f801ab51700 1 --2- 172.21.2.101:0/3655759479 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80140cf580 0x7f80140cf980 unknown :-1 s=CLOSED pgs=3151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:53.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.032 7f801ab51700 1 --2- 172.21.2.101:0/3655759479 >> v2:172.21.2.108:3300/0 conn(0x7f80140d1c70 0x7f80140cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:53.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.032 7f801ab51700 1 -- 172.21.2.101:0/3655759479 >> 172.21.2.101:0/3655759479 conn(0x7f80140ce420 msgr2=0x7f80140d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:53.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.032 7f801ab51700 1 -- 172.21.2.101:0/3655759479 shutdown_connections 2021-02-10T11:47:53.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.032 7f801ab51700 1 -- 172.21.2.101:0/3655759479 wait complete. 2021-02-10T11:47:53.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.047 7f801ab51700 1 Processor -- start 2021-02-10T11:47:53.046 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.047 7f801ab51700 1 -- start start 2021-02-10T11:47:53.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.047 7f801ab51700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f80140b04d0 0x7f80141a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:53.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.047 7f801ab51700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80141a8090 0x7f80140c23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:53.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.047 7f801ab51700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f801408d680 0x7f80140c29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:53.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.047 7f801ab51700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80140b0a00 con 0x7f80141a8090 2021-02-10T11:47:53.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.047 7f801ab51700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80140c2fe0 con 0x7f80140b04d0 2021-02-10T11:47:53.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.047 7f801ab51700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80140c3200 con 0x7f801408d680 2021-02-10T11:47:53.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.048 7f8013fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80141a8090 0x7f80140c23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:53.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.048 7f8013fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80141a8090 0x7f80140c23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34396/0 (socket says 172.21.2.101:34396) 2021-02-10T11:47:53.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.048 7f8013fff700 1 -- 172.21.2.101:0/1424344245 learned_addr learned my addr 172.21.2.101:0/1424344245 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:53.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.048 7f801896d700 1 --2- 172.21.2.101:0/1424344245 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f80140b04d0 0x7f80141a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:53.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.048 7f8013fff700 1 -- 172.21.2.101:0/1424344245 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f801408d680 msgr2=0x7f80140c29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:53.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.048 7f8013fff700 1 --2- 172.21.2.101:0/1424344245 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f801408d680 0x7f80140c29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:53.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.048 7f8013fff700 1 -- 172.21.2.101:0/1424344245 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f80140b04d0 msgr2=0x7f80141a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:53.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.048 7f8013fff700 1 --2- 172.21.2.101:0/1424344245 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f80140b04d0 0x7f80141a7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:53.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.048 7f8013fff700 1 -- 172.21.2.101:0/1424344245 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801408e080 con 0x7f80141a8090 2021-02-10T11:47:53.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.048 7f8013fff700 1 --2- 172.21.2.101:0/1424344245 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80141a8090 0x7f80140c23a0 secure :-1 s=READY pgs=3152 cs=0 l=1 rev1=1 rx=0x7f8000006f20 tx=0x7f800000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:53.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.048 7f80117fa700 1 -- 172.21.2.101:0/1424344245 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f800000bd80 con 0x7f80141a8090 2021-02-10T11:47:53.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.048 7f80117fa700 1 -- 172.21.2.101:0/1424344245 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8000000c90 con 0x7f80141a8090 2021-02-10T11:47:53.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.048 7f80117fa700 1 -- 172.21.2.101:0/1424344245 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f80000022b0 con 0x7f80141a8090 2021-02-10T11:47:53.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.048 7f801ab51700 1 -- 172.21.2.101:0/1424344245 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f801408e080 con 0x7f80141a8090 2021-02-10T11:47:53.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.049 7f801ab51700 1 -- 172.21.2.101:0/1424344245 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f801408e610 con 0x7f80141a8090 2021-02-10T11:47:53.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.049 7f80117fa700 1 -- 172.21.2.101:0/1424344245 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f800000a000 con 0x7f80141a8090 2021-02-10T11:47:53.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.049 7f80117fa700 1 --2- 172.21.2.101:0/1424344245 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ff00186c0 0x7f7ff001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:53.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.049 7f80117fa700 1 -- 172.21.2.101:0/1424344245 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8000032570 con 0x7f80141a8090 2021-02-10T11:47:53.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.049 7f801896d700 1 --2- 172.21.2.101:0/1424344245 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ff00186c0 0x7f7ff001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:53.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.049 7f801896d700 1 --2- 172.21.2.101:0/1424344245 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ff00186c0 0x7f7ff001abc0 crc :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:53.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.080 7f801ab51700 1 -- 172.21.2.101:0/1424344245 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8014082a70 con 0x7f80141a8090 2021-02-10T11:47:53.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.083 7f80117fa700 1 -- 172.21.2.101:0/1424344245 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f800002ded0 con 0x7f80141a8090 2021-02-10T11:47:53.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.231 7f801ab51700 1 -- 172.21.2.101:0/1424344245 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8014082a70 con 0x7f80141a8090 2021-02-10T11:47:53.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.232 7f80117fa700 1 -- 172.21.2.101:0/1424344245 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v44)=0 v44) v1 ==== 122+0+0 (secure 0 0 0) 0x7f800002ded0 con 0x7f80141a8090 2021-02-10T11:47:53.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.233 7f801ab51700 1 -- 172.21.2.101:0/1424344245 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ff00186c0 msgr2=0x7f7ff001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:53.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.234 7f801ab51700 1 --2- 172.21.2.101:0/1424344245 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ff00186c0 0x7f7ff001abc0 crc :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:53.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.234 7f801ab51700 1 -- 172.21.2.101:0/1424344245 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80141a8090 msgr2=0x7f80140c23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:53.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.234 7f801ab51700 1 --2- 172.21.2.101:0/1424344245 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80141a8090 0x7f80140c23a0 secure :-1 s=READY pgs=3152 cs=0 l=1 rev1=1 rx=0x7f8000006f20 tx=0x7f800000b990).stop 2021-02-10T11:47:53.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.234 7f801ab51700 1 -- 172.21.2.101:0/1424344245 shutdown_connections 2021-02-10T11:47:53.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.234 7f801ab51700 1 --2- 172.21.2.101:0/1424344245 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ff00186c0 0x7f7ff001abc0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:53.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.234 7f801ab51700 1 --2- 172.21.2.101:0/1424344245 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f801408d680 0x7f80140c29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:53.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.234 7f801ab51700 1 --2- 172.21.2.101:0/1424344245 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f80141a8090 0x7f80140c23a0 unknown :-1 s=CLOSED pgs=3152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:53.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.234 7f801ab51700 1 --2- 172.21.2.101:0/1424344245 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f80140b04d0 0x7f80141a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:53.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.234 7f801ab51700 1 -- 172.21.2.101:0/1424344245 >> 172.21.2.101:0/1424344245 conn(0x7f80140be530 msgr2=0x7f801409f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:53.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.234 7f801ab51700 1 -- 172.21.2.101:0/1424344245 shutdown_connections 2021-02-10T11:47:53.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.234 7f801ab51700 1 -- 172.21.2.101:0/1424344245 wait complete. 2021-02-10T11:47:53.238 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolumegroup create cephfs group_0000000000537794 2021-02-10T11:47:53.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.660 7f26b93d4700 1 Processor -- start 2021-02-10T11:47:53.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.660 7f26b93d4700 1 -- start start 2021-02-10T11:47:53.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.660 7f26b93d4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26b410aff0 0x7f26b40cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:53.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.660 7f26b93d4700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b40fe1c0 con 0x7f26b40d1c70 2021-02-10T11:47:53.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.660 7f26b93d4700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b41b0cb0 con 0x7f26b40d1870 2021-02-10T11:47:53.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.660 7f26b93d4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b4109a60 con 0x7f26b410aff0 2021-02-10T11:47:53.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.660 7f26b287c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f26b40d1c70 0x7f26b40cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34896/0 (socket says 172.21.2.101:34896) 2021-02-10T11:47:53.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.660 7f26b287c700 1 -- 172.21.2.101:0/482508643 learned_addr learned my addr 172.21.2.101:0/482508643 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:53.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.661 7f26b387e700 1 --2- 172.21.2.101:0/482508643 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26b410aff0 0x7f26b40cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:53.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.661 7f26b207b700 1 -- 172.21.2.101:0/482508643 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1750273907 0 0) 0x7f26a80010b0 con 0x7f26b40d1c70 2021-02-10T11:47:53.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.661 7f26b207b700 1 -- 172.21.2.101:0/482508643 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f269c002fe0 con 0x7f26b40d1c70 2021-02-10T11:47:53.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.661 7f26b207b700 1 -- 172.21.2.101:0/482508643 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3180036080 0 0) 0x7f26a80010b0 con 0x7f26b40d1c70 2021-02-10T11:47:53.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.661 7f26b207b700 1 -- 172.21.2.101:0/482508643 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26b410aff0 msgr2=0x7f26b40cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:53.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.661 7f26b207b700 1 --2- 172.21.2.101:0/482508643 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26b410aff0 0x7f26b40cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:53.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.661 7f26b207b700 1 -- 172.21.2.101:0/482508643 >> v1:172.21.2.108:6789/0 conn(0x7f26b40d1870 legacy=0x7f26b41b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:53.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.661 7f26b207b700 1 -- 172.21.2.101:0/482508643 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b410b3f0 con 0x7f26b40d1c70 2021-02-10T11:47:53.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.661 7f26b207b700 1 -- 172.21.2.101:0/482508643 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f26a80015b0 con 0x7f26b40d1c70 2021-02-10T11:47:53.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.661 7f26b207b700 1 -- 172.21.2.101:0/482508643 >> v1:172.21.2.101:6789/0 conn(0x7f26b40d1c70 legacy=0x7f26b40cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:53.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.661 7f26b207b700 1 --2- 172.21.2.101:0/482508643 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f269c003aa0 0x7f269c005eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:53.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.661 7f26b207b700 1 --2- 172.21.2.101:0/482508643 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f269c006730 0x7f269c008c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:53.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.661 7f26b207b700 1 --2- 172.21.2.101:0/482508643 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f269c0094f0 0x7f269c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:53.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.661 7f26b207b700 1 -- 172.21.2.101:0/482508643 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c00c2c0 con 0x7f269c0094f0 2021-02-10T11:47:53.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.661 7f26b207b700 1 -- 172.21.2.101:0/482508643 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c00c4e0 con 0x7f269c006730 2021-02-10T11:47:53.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.661 7f26b207b700 1 -- 172.21.2.101:0/482508643 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269c050710 con 0x7f269c003aa0 2021-02-10T11:47:53.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.661 7f26b387e700 1 --2- 172.21.2.101:0/482508643 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f269c0094f0 0x7f269c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:53.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.661 7f26b287c700 1 --2- 172.21.2.101:0/482508643 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f269c006730 0x7f269c008c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:53.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.661 7f26b307d700 1 --2- 172.21.2.101:0/482508643 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f269c003aa0 0x7f269c005eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:53.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.662 7f26b387e700 1 -- 172.21.2.101:0/482508643 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f269c003aa0 msgr2=0x7f269c005eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:53.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.662 7f26b387e700 1 --2- 172.21.2.101:0/482508643 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f269c003aa0 0x7f269c005eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:53.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.662 7f26b387e700 1 -- 172.21.2.101:0/482508643 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f269c006730 msgr2=0x7f269c008c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:53.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.662 7f26b387e700 1 --2- 172.21.2.101:0/482508643 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f269c006730 0x7f269c008c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:53.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.662 7f26b387e700 1 -- 172.21.2.101:0/482508643 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269c050970 con 0x7f269c0094f0 2021-02-10T11:47:53.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.662 7f26b307d700 1 --2- 172.21.2.101:0/482508643 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f269c003aa0 0x7f269c005eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:47:53.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.662 7f26b387e700 1 --2- 172.21.2.101:0/482508643 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f269c0094f0 0x7f269c00ba10 secure :-1 s=READY pgs=3154 cs=0 l=1 rev1=1 rx=0x7f26ac009b10 tx=0x7f26ac009eb0).ready entity=mon.0 client_cookie=e20c65342f629709 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:53.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.662 7f26b207b700 1 -- 172.21.2.101:0/482508643 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f26ac00b510 con 0x7f269c0094f0 2021-02-10T11:47:53.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.662 7f26b207b700 1 -- 172.21.2.101:0/482508643 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ac003030 con 0x7f269c0094f0 2021-02-10T11:47:53.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.662 7f26b207b700 1 -- 172.21.2.101:0/482508643 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f26ac00b790 con 0x7f269c0094f0 2021-02-10T11:47:53.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.662 7f26b93d4700 1 -- 172.21.2.101:0/482508643 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f269c0094f0 msgr2=0x7f269c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:53.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.662 7f26b93d4700 1 --2- 172.21.2.101:0/482508643 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f269c0094f0 0x7f269c00ba10 secure :-1 s=READY pgs=3154 cs=0 l=1 rev1=1 rx=0x7f26ac009b10 tx=0x7f26ac009eb0).stop 2021-02-10T11:47:53.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.662 7f26b387e700 1 -- 172.21.2.101:0/482508643 reap_dead start 2021-02-10T11:47:53.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.663 7f26b93d4700 1 -- 172.21.2.101:0/482508643 shutdown_connections 2021-02-10T11:47:53.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.663 7f26b93d4700 1 -- 172.21.2.101:0/482508643 >> 172.21.2.101:0/482508643 conn(0x7f26b40ce420 msgr2=0x7f26b40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:53.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.663 7f26b93d4700 1 -- 172.21.2.101:0/482508643 shutdown_connections 2021-02-10T11:47:53.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.663 7f26b93d4700 1 -- 172.21.2.101:0/482508643 wait complete. 2021-02-10T11:47:53.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.677 7f26b93d4700 1 Processor -- start 2021-02-10T11:47:53.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.677 7f26b93d4700 1 -- start start 2021-02-10T11:47:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.678 7f26b93d4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f26b409bcc0 0x7f26b409c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.678 7f26b93d4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26b40a0820 0x7f26b40a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.678 7f26b93d4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26b40a1240 0x7f26b40858d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.678 7f26b93d4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b40a66b0 con 0x7f26b40a1240 2021-02-10T11:47:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.678 7f26b93d4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b41a49b0 con 0x7f26b409bcc0 2021-02-10T11:47:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.678 7f26b93d4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b41a4b90 con 0x7f26b40a0820 2021-02-10T11:47:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.678 7f26b387e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26b40a1240 0x7f26b40858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.678 7f26b387e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26b40a1240 0x7f26b40858d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34418/0 (socket says 172.21.2.101:34418) 2021-02-10T11:47:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.678 7f26b387e700 1 -- 172.21.2.101:0/2632465703 learned_addr learned my addr 172.21.2.101:0/2632465703 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:53.680 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.678 7f26b287c700 1 --2- 172.21.2.101:0/2632465703 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26b40a0820 0x7f26b40a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.678 7f26b307d700 1 --2- 172.21.2.101:0/2632465703 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f26b409bcc0 0x7f26b409c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.678 7f26b387e700 1 -- 172.21.2.101:0/2632465703 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26b40a0820 msgr2=0x7f26b40a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.678 7f26b387e700 1 --2- 172.21.2.101:0/2632465703 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26b40a0820 0x7f26b40a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.679 7f26b387e700 1 -- 172.21.2.101:0/2632465703 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f26b409bcc0 msgr2=0x7f26b409c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.679 7f26b387e700 1 --2- 172.21.2.101:0/2632465703 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f26b409bcc0 0x7f26b409c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.679 7f26b387e700 1 -- 172.21.2.101:0/2632465703 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b40a1640 con 0x7f26b40a1240 2021-02-10T11:47:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.679 7f26b387e700 1 --2- 172.21.2.101:0/2632465703 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26b40a1240 0x7f26b40858d0 secure :-1 s=READY pgs=3155 cs=0 l=1 rev1=1 rx=0x7f26ac0099d0 tx=0x7f26ac00bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.679 7f269bfff700 1 -- 172.21.2.101:0/2632465703 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f26ac000df0 con 0x7f26b40a1240 2021-02-10T11:47:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.679 7f269bfff700 1 -- 172.21.2.101:0/2632465703 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ac0090c0 con 0x7f26b40a1240 2021-02-10T11:47:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.679 7f26b93d4700 1 -- 172.21.2.101:0/2632465703 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b40a1640 con 0x7f26b40a1240 2021-02-10T11:47:53.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.679 7f26b93d4700 1 -- 172.21.2.101:0/2632465703 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b40861e0 con 0x7f26b40a1240 2021-02-10T11:47:53.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.679 7f269bfff700 1 -- 172.21.2.101:0/2632465703 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f26ac0013c0 con 0x7f26b40a1240 2021-02-10T11:47:53.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.680 7f269bfff700 1 -- 172.21.2.101:0/2632465703 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f26ac009380 con 0x7f26b40a1240 2021-02-10T11:47:53.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.680 7f269bfff700 1 --2- 172.21.2.101:0/2632465703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26940186c0 0x7f269401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:53.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.680 7f269bfff700 1 -- 172.21.2.101:0/2632465703 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f26ac0328f0 con 0x7f26b40a1240 2021-02-10T11:47:53.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.680 7f26b307d700 1 --2- 172.21.2.101:0/2632465703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26940186c0 0x7f269401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:53.682 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.680 7f26b307d700 1 --2- 172.21.2.101:0/2632465703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26940186c0 0x7f269401abc0 crc :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:53.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.710 7f26b93d4700 1 -- 172.21.2.101:0/2632465703 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b4086a70 con 0x7f26b40a1240 2021-02-10T11:47:53.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.712 7f269bfff700 1 -- 172.21.2.101:0/2632465703 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f26ac02d8e0 con 0x7f26b40a1240 2021-02-10T11:47:53.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.840 7f26b93d4700 1 -- 172.21.2.101:0/2632465703 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "target": ["mgr", ""], "group_name": "group_0000000000537794"}) v1 -- 0x7f26b40a66b0 con 0x7f26940186c0 2021-02-10T11:47:53.840 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:47:53.844 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:47:53.840 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:47:53.844 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:47:53.840 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:47:53.844 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:47:53.840 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:47:53.844 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:47:53.840 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:47:53.844 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:47:53.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.860 7f269bfff700 1 -- 172.21.2.101:0/2632465703 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f26a4006a80 con 0x7f26940186c0 2021-02-10T11:47:53.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.873 7f26b93d4700 1 -- 172.21.2.101:0/2632465703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26940186c0 msgr2=0x7f269401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:53.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.873 7f26b93d4700 1 --2- 172.21.2.101:0/2632465703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26940186c0 0x7f269401abc0 crc :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:53.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.873 7f26b93d4700 1 -- 172.21.2.101:0/2632465703 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26b40a1240 msgr2=0x7f26b40858d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:53.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.873 7f26b93d4700 1 --2- 172.21.2.101:0/2632465703 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26b40a1240 0x7f26b40858d0 secure :-1 s=READY pgs=3155 cs=0 l=1 rev1=1 rx=0x7f26ac0099d0 tx=0x7f26ac00bd10).stop 2021-02-10T11:47:53.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.873 7f26b93d4700 1 -- 172.21.2.101:0/2632465703 shutdown_connections 2021-02-10T11:47:53.869 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.873 7f26b93d4700 1 --2- 172.21.2.101:0/2632465703 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f26940186c0 0x7f269401abc0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:53.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.873 7f26b93d4700 1 --2- 172.21.2.101:0/2632465703 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f26b40a0820 0x7f26b40a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:53.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.873 7f26b93d4700 1 --2- 172.21.2.101:0/2632465703 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f26b40a1240 0x7f26b40858d0 unknown :-1 s=CLOSED pgs=3155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:53.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.873 7f26b93d4700 1 --2- 172.21.2.101:0/2632465703 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f26b409bcc0 0x7f26b409c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:53.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.873 7f26b93d4700 1 -- 172.21.2.101:0/2632465703 >> 172.21.2.101:0/2632465703 conn(0x7f26b40bef00 msgr2=0x7f26b409d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:53.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.874 7f26b93d4700 1 -- 172.21.2.101:0/2632465703 shutdown_connections 2021-02-10T11:47:53.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:53.874 7f26b93d4700 1 -- 172.21.2.101:0/2632465703 wait complete. 2021-02-10T11:47:53.877 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000359483 --group_name group_0000000000537794 2021-02-10T11:47:54.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.298 7f3ed6edc700 1 Processor -- start 2021-02-10T11:47:54.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.298 7f3ed6edc700 1 -- start start 2021-02-10T11:47:54.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.299 7f3ed6edc700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f3ed00d1880 0x7f3ed01b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:54.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.299 7f3ed6edc700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3ed00cff80 0x7f3ed0109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:54.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.299 7f3ed6edc700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ed00fe1d0 con 0x7f3ed01b0af0 2021-02-10T11:47:54.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.299 7f3ed6edc700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed0109ae0 con 0x7f3ed00cff80 2021-02-10T11:47:54.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.299 7f3ed6edc700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed0109760 con 0x7f3ed00d1880 2021-02-10T11:47:54.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.299 7f3ecffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f3ed01b0af0 0x7f3ed0108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34928/0 (socket says 172.21.2.101:34928) 2021-02-10T11:47:54.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.299 7f3ecffff700 1 -- 172.21.2.101:0/1934081984 learned_addr learned my addr 172.21.2.101:0/1934081984 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:54.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.299 7f3ed54f9700 1 --2- 172.21.2.101:0/1934081984 >> v2:172.21.2.101:3300/0 conn(0x7f3ed00cff80 0x7f3ed0109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:54.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.299 7f3ed54f9700 1 -- 172.21.2.101:0/1934081984 >> v2:172.21.2.108:3300/0 conn(0x7f3ed00d1880 msgr2=0x7f3ed01b04d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:54.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.299 7f3ed54f9700 1 --2- 172.21.2.101:0/1934081984 >> v2:172.21.2.108:3300/0 conn(0x7f3ed00d1880 0x7f3ed01b04d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:54.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.299 7f3ed54f9700 1 -- 172.21.2.101:0/1934081984 >> v1:172.21.2.101:6789/0 conn(0x7f3ed01b0af0 legacy=0x7f3ed0108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:54.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.299 7f3ed54f9700 1 -- 172.21.2.101:0/1934081984 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed0088070 con 0x7f3ed00cff80 2021-02-10T11:47:54.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.299 7f3ed54f9700 1 --2- 172.21.2.101:0/1934081984 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ed00cff80 0x7f3ed0109140 secure :-1 s=READY pgs=3159 cs=0 l=1 rev1=1 rx=0x7f3ec8006e30 tx=0x7f3ec8009e60).ready entity=mon.0 client_cookie=e9e3ddbc36590e64 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:54.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.300 7f3ecf7fe700 1 -- 172.21.2.101:0/1934081984 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3ec8006a50 con 0x7f3ed00cff80 2021-02-10T11:47:54.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.300 7f3ecf7fe700 1 -- 172.21.2.101:0/1934081984 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec800b300 con 0x7f3ed00cff80 2021-02-10T11:47:54.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.300 7f3ecf7fe700 1 -- 172.21.2.101:0/1934081984 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3ec800b7e0 con 0x7f3ed00cff80 2021-02-10T11:47:54.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.300 7f3ed6edc700 1 -- 172.21.2.101:0/1934081984 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ed00cff80 msgr2=0x7f3ed0109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:54.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.300 7f3ed6edc700 1 --2- 172.21.2.101:0/1934081984 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ed00cff80 0x7f3ed0109140 secure :-1 s=READY pgs=3159 cs=0 l=1 rev1=1 rx=0x7f3ec8006e30 tx=0x7f3ec8009e60).stop 2021-02-10T11:47:54.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.301 7f3ed6edc700 1 -- 172.21.2.101:0/1934081984 shutdown_connections 2021-02-10T11:47:54.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.301 7f3ed6edc700 1 --2- 172.21.2.101:0/1934081984 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ed00cff80 0x7f3ed0109140 unknown :-1 s=CLOSED pgs=3159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:54.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.301 7f3ed6edc700 1 --2- 172.21.2.101:0/1934081984 >> v2:172.21.2.108:3300/0 conn(0x7f3ed00d1880 0x7f3ed01b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:54.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.301 7f3ed6edc700 1 -- 172.21.2.101:0/1934081984 >> 172.21.2.101:0/1934081984 conn(0x7f3ed00ce430 msgr2=0x7f3ed00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:54.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.301 7f3ed6edc700 1 -- 172.21.2.101:0/1934081984 shutdown_connections 2021-02-10T11:47:54.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.301 7f3ed6edc700 1 -- 172.21.2.101:0/1934081984 wait complete. 2021-02-10T11:47:54.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.315 7f3ed6edc700 1 Processor -- start 2021-02-10T11:47:54.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.315 7f3ed6edc700 1 -- start start 2021-02-10T11:47:54.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.316 7f3ed6edc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ed019e5b0 0x7f3ed01afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:54.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.316 7f3ed6edc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ed019e9b0 0x7f3ed01b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:54.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.316 7f3ed6edc700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ed01b0250 0x7f3ed01b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.316 7f3ed6edc700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed0195bd0 con 0x7f3ed019e5b0 2021-02-10T11:47:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.316 7f3ed6edc700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed01b0840 con 0x7f3ed01b0250 2021-02-10T11:47:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.316 7f3ed6edc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed019eed0 con 0x7f3ed019e9b0 2021-02-10T11:47:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.316 7f3ed4cf8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ed019e5b0 0x7f3ed01afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.316 7f3ed4cf8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ed019e5b0 0x7f3ed01afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34442/0 (socket says 172.21.2.101:34442) 2021-02-10T11:47:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.316 7f3ed4cf8700 1 -- 172.21.2.101:0/2396325205 learned_addr learned my addr 172.21.2.101:0/2396325205 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.316 7f3ed4cf8700 1 -- 172.21.2.101:0/2396325205 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ed019e9b0 msgr2=0x7f3ed01b0dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:47:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.316 7f3ed4cf8700 1 --2- 172.21.2.101:0/2396325205 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ed019e9b0 0x7f3ed01b0dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.316 7f3ed4cf8700 1 -- 172.21.2.101:0/2396325205 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ed01b0250 msgr2=0x7f3ed01b13e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:47:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.316 7f3ed4cf8700 1 --2- 172.21.2.101:0/2396325205 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ed01b0250 0x7f3ed01b13e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.316 7f3ed4cf8700 1 -- 172.21.2.101:0/2396325205 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed019f0b0 con 0x7f3ed019e5b0 2021-02-10T11:47:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.316 7f3ed4cf8700 1 --2- 172.21.2.101:0/2396325205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ed019e5b0 0x7f3ed01afc30 secure :-1 s=READY pgs=3160 cs=0 l=1 rev1=1 rx=0x7f3eb800b5c0 tx=0x7f3eb800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:54.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.317 7f3ecd7fa700 1 -- 172.21.2.101:0/2396325205 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3eb800bd50 con 0x7f3ed019e5b0 2021-02-10T11:47:54.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.317 7f3ecd7fa700 1 -- 172.21.2.101:0/2396325205 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb8006030 con 0x7f3ed019e5b0 2021-02-10T11:47:54.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.317 7f3ed6edc700 1 -- 172.21.2.101:0/2396325205 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed019f0b0 con 0x7f3ed019e5b0 2021-02-10T11:47:54.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.317 7f3ed6edc700 1 -- 172.21.2.101:0/2396325205 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed019f0b0 con 0x7f3ed019e5b0 2021-02-10T11:47:54.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.317 7f3ecd7fa700 1 -- 172.21.2.101:0/2396325205 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3eb80054d0 con 0x7f3ed019e5b0 2021-02-10T11:47:54.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.317 7f3ecd7fa700 1 -- 172.21.2.101:0/2396325205 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3eb8006210 con 0x7f3ed019e5b0 2021-02-10T11:47:54.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.317 7f3ecd7fa700 1 --2- 172.21.2.101:0/2396325205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3eb4018730 0x7f3eb401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:54.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.317 7f3ecd7fa700 1 -- 172.21.2.101:0/2396325205 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3eb8032800 con 0x7f3ed019e5b0 2021-02-10T11:47:54.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.317 7f3ecffff700 1 --2- 172.21.2.101:0/2396325205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3eb4018730 0x7f3eb401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:54.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.318 7f3ecffff700 1 --2- 172.21.2.101:0/2396325205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3eb4018730 0x7f3eb401abc0 crc :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:54.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.348 7f3ed6edc700 1 -- 172.21.2.101:0/2396325205 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed0082aa0 con 0x7f3ed019e5b0 2021-02-10T11:47:54.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.351 7f3ecd7fa700 1 -- 172.21.2.101:0/2396325205 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3eb8006b80 con 0x7f3ed019e5b0 2021-02-10T11:47:54.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.477 7f3ed6edc700 1 -- 172.21.2.101:0/2396325205 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000359483", "target": ["mgr", ""], "group_name": "group_0000000000537794"}) v1 -- 0x7f3ed01b0840 con 0x7f3eb4018730 2021-02-10T11:47:54.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.504 7f3ecd7fa700 1 -- 172.21.2.101:0/2396325205 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f3ec0007690 con 0x7f3eb4018730 2021-02-10T11:47:54.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.511 7f3ed6edc700 1 -- 172.21.2.101:0/2396325205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3eb4018730 msgr2=0x7f3eb401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:54.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.511 7f3ed6edc700 1 --2- 172.21.2.101:0/2396325205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3eb4018730 0x7f3eb401abc0 crc :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:54.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.511 7f3ed6edc700 1 -- 172.21.2.101:0/2396325205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ed019e5b0 msgr2=0x7f3ed01afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:54.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.511 7f3ed6edc700 1 --2- 172.21.2.101:0/2396325205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ed019e5b0 0x7f3ed01afc30 secure :-1 s=READY pgs=3160 cs=0 l=1 rev1=1 rx=0x7f3eb800b5c0 tx=0x7f3eb800b960).stop 2021-02-10T11:47:54.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.511 7f3ed6edc700 1 -- 172.21.2.101:0/2396325205 shutdown_connections 2021-02-10T11:47:54.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.511 7f3ed6edc700 1 --2- 172.21.2.101:0/2396325205 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3eb4018730 0x7f3eb401abc0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:54.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.511 7f3ed6edc700 1 --2- 172.21.2.101:0/2396325205 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3ed019e9b0 0x7f3ed01b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:54.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.511 7f3ed6edc700 1 --2- 172.21.2.101:0/2396325205 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3ed01b0250 0x7f3ed01b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:54.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.511 7f3ed6edc700 1 --2- 172.21.2.101:0/2396325205 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3ed019e5b0 0x7f3ed01afc30 unknown :-1 s=CLOSED pgs=3160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:54.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.511 7f3ed6edc700 1 -- 172.21.2.101:0/2396325205 >> 172.21.2.101:0/2396325205 conn(0x7f3ed019f940 msgr2=0x7f3ed018c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:54.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.511 7f3ed6edc700 1 -- 172.21.2.101:0/2396325205 shutdown_connections 2021-02-10T11:47:54.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.511 7f3ed6edc700 1 -- 172.21.2.101:0/2396325205 wait complete. 2021-02-10T11:47:54.515 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000359484 --group_name group_0000000000537794 2021-02-10T11:47:54.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.935 7f904a3f3700 1 Processor -- start 2021-02-10T11:47:54.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.935 7f904a3f3700 1 -- start start 2021-02-10T11:47:54.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.935 7f904a3f3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f90440d1880 0x7f90441b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:54.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.935 7f904a3f3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f90440cff80 0x7f9044109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:54.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.935 7f904a3f3700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90440fe1d0 con 0x7f90441b0af0 2021-02-10T11:47:54.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.935 7f904a3f3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9044109ae0 con 0x7f90440cff80 2021-02-10T11:47:54.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.935 7f904a3f3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9044109760 con 0x7f90440d1880 2021-02-10T11:47:54.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.936 7f9048a10700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f90440cff80 0x7f9044109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:54.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.936 7f9048a10700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f90440cff80 0x7f9044109140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34450/0 (socket says 172.21.2.101:34450) 2021-02-10T11:47:54.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.936 7f9048a10700 1 -- 172.21.2.101:0/2544286750 learned_addr learned my addr 172.21.2.101:0/2544286750 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:54.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.936 7f9043fff700 1 --2- 172.21.2.101:0/2544286750 >> v2:172.21.2.108:3300/0 conn(0x7f90440d1880 0x7f90441b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:54.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.936 7f9048a10700 1 -- 172.21.2.101:0/2544286750 >> v2:172.21.2.108:3300/0 conn(0x7f90440d1880 msgr2=0x7f90441b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:54.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.936 7f9048a10700 1 --2- 172.21.2.101:0/2544286750 >> v2:172.21.2.108:3300/0 conn(0x7f90440d1880 0x7f90441b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:54.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.936 7f9048a10700 1 -- 172.21.2.101:0/2544286750 >> v1:172.21.2.101:6789/0 conn(0x7f90441b0af0 legacy=0x7f9044108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:54.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.936 7f9042ffd700 1 -- 172.21.2.101:0/2544286750 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2811657158 0 0) 0x7f9038000ff0 con 0x7f90441b0af0 2021-02-10T11:47:54.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.936 7f9048a10700 1 -- 172.21.2.101:0/2544286750 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9044088070 con 0x7f90440cff80 2021-02-10T11:47:54.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.936 7f9043fff700 1 --2- 172.21.2.101:0/2544286750 >> v2:172.21.2.108:3300/0 conn(0x7f90440d1880 0x7f90441b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:47:54.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.936 7f9048a10700 1 --2- 172.21.2.101:0/2544286750 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f90440cff80 0x7f9044109140 secure :-1 s=READY pgs=3162 cs=0 l=1 rev1=1 rx=0x7f903c006f00 tx=0x7f903c009f10).ready entity=mon.0 client_cookie=394a2baa83c99cd9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:54.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.937 7f9042ffd700 1 -- 172.21.2.101:0/2544286750 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f903c00b3b0 con 0x7f90440cff80 2021-02-10T11:47:54.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.937 7f9042ffd700 1 -- 172.21.2.101:0/2544286750 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903c00b730 con 0x7f90440cff80 2021-02-10T11:47:54.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.937 7f9042ffd700 1 -- 172.21.2.101:0/2544286750 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f903c00bc50 con 0x7f90440cff80 2021-02-10T11:47:54.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.937 7f904a3f3700 1 -- 172.21.2.101:0/2544286750 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f90440cff80 msgr2=0x7f9044109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:54.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.937 7f904a3f3700 1 --2- 172.21.2.101:0/2544286750 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f90440cff80 0x7f9044109140 secure :-1 s=READY pgs=3162 cs=0 l=1 rev1=1 rx=0x7f903c006f00 tx=0x7f903c009f10).stop 2021-02-10T11:47:54.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.937 7f904a3f3700 1 -- 172.21.2.101:0/2544286750 shutdown_connections 2021-02-10T11:47:54.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.937 7f904a3f3700 1 --2- 172.21.2.101:0/2544286750 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f90440cff80 0x7f9044109140 unknown :-1 s=CLOSED pgs=3162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:54.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.937 7f904a3f3700 1 --2- 172.21.2.101:0/2544286750 >> v2:172.21.2.108:3300/0 conn(0x7f90440d1880 0x7f90441b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:54.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.937 7f904a3f3700 1 -- 172.21.2.101:0/2544286750 >> 172.21.2.101:0/2544286750 conn(0x7f90440ce430 msgr2=0x7f90440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:54.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.937 7f904a3f3700 1 -- 172.21.2.101:0/2544286750 shutdown_connections 2021-02-10T11:47:54.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.937 7f904a3f3700 1 -- 172.21.2.101:0/2544286750 wait complete. 2021-02-10T11:47:54.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.952 7f904a3f3700 1 Processor -- start 2021-02-10T11:47:54.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.952 7f904a3f3700 1 -- start start 2021-02-10T11:47:54.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.952 7f904a3f3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f90441a05e0 0x7f904409a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:54.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.952 7f904a3f3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f904409a8b0 0x7f904409acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:54.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.952 7f904a3f3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f904409b2d0 0x7f90440beed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:54.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.952 7f904a3f3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9044099d50 con 0x7f904409a8b0 2021-02-10T11:47:54.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.952 7f904a3f3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90440bf4f0 con 0x7f904409b2d0 2021-02-10T11:47:54.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.952 7f904a3f3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90440bf6e0 con 0x7f90441a05e0 2021-02-10T11:47:54.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.953 7f90437fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f904409a8b0 0x7f904409acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:54.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.953 7f90437fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f904409a8b0 0x7f904409acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34456/0 (socket says 172.21.2.101:34456) 2021-02-10T11:47:54.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.953 7f90437fe700 1 -- 172.21.2.101:0/4201921175 learned_addr learned my addr 172.21.2.101:0/4201921175 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:54.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.953 7f90437fe700 1 -- 172.21.2.101:0/4201921175 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f90441a05e0 msgr2=0x7f904409a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:54.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.953 7f90437fe700 1 --2- 172.21.2.101:0/4201921175 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f90441a05e0 0x7f904409a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:54.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.953 7f90437fe700 1 -- 172.21.2.101:0/4201921175 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f904409b2d0 msgr2=0x7f90440beed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:54.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.953 7f90437fe700 1 --2- 172.21.2.101:0/4201921175 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f904409b2d0 0x7f90440beed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:54.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.953 7f90437fe700 1 -- 172.21.2.101:0/4201921175 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90440bf900 con 0x7f904409a8b0 2021-02-10T11:47:54.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.953 7f90437fe700 1 --2- 172.21.2.101:0/4201921175 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f904409a8b0 0x7f904409acb0 secure :-1 s=READY pgs=3163 cs=0 l=1 rev1=1 rx=0x7f9038003010 tx=0x7f903800a8a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:54.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.953 7f9040ff9700 1 -- 172.21.2.101:0/4201921175 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f903800ac90 con 0x7f904409a8b0 2021-02-10T11:47:54.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.953 7f9040ff9700 1 -- 172.21.2.101:0/4201921175 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903800c0d0 con 0x7f904409a8b0 2021-02-10T11:47:54.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.954 7f904a3f3700 1 -- 172.21.2.101:0/4201921175 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90440bf900 con 0x7f904409a8b0 2021-02-10T11:47:54.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.954 7f9040ff9700 1 -- 172.21.2.101:0/4201921175 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f903800c5f0 con 0x7f904409a8b0 2021-02-10T11:47:54.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.954 7f904a3f3700 1 -- 172.21.2.101:0/4201921175 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90440bfe90 con 0x7f904409a8b0 2021-02-10T11:47:54.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.954 7f9040ff9700 1 -- 172.21.2.101:0/4201921175 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9038008030 con 0x7f904409a8b0 2021-02-10T11:47:54.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.954 7f9040ff9700 1 --2- 172.21.2.101:0/4201921175 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90240186c0 0x7f902401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:54.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.954 7f9040ff9700 1 -- 172.21.2.101:0/4201921175 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9038032440 con 0x7f904409a8b0 2021-02-10T11:47:54.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.954 7f9043fff700 1 --2- 172.21.2.101:0/4201921175 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90240186c0 0x7f902401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:54.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.955 7f9043fff700 1 --2- 172.21.2.101:0/4201921175 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90240186c0 0x7f902401abc0 crc :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:54.981 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.985 7f904a3f3700 1 -- 172.21.2.101:0/4201921175 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9044099d50 con 0x7f904409a8b0 2021-02-10T11:47:54.983 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:54.988 7f9040ff9700 1 -- 172.21.2.101:0/4201921175 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9038000990 con 0x7f904409a8b0 2021-02-10T11:47:55.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.115 7f904a3f3700 1 -- 172.21.2.101:0/4201921175 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000359484", "target": ["mgr", ""], "group_name": "group_0000000000537794"}) v1 -- 0x7f90440bfe90 con 0x7f90240186c0 2021-02-10T11:47:55.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.138 7f9040ff9700 1 -- 172.21.2.101:0/4201921175 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f90340009b0 con 0x7f90240186c0 2021-02-10T11:47:55.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.149 7f904a3f3700 1 -- 172.21.2.101:0/4201921175 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90240186c0 msgr2=0x7f902401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:55.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.149 7f904a3f3700 1 --2- 172.21.2.101:0/4201921175 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90240186c0 0x7f902401abc0 crc :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:55.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.149 7f904a3f3700 1 -- 172.21.2.101:0/4201921175 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f904409a8b0 msgr2=0x7f904409acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:55.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.149 7f904a3f3700 1 --2- 172.21.2.101:0/4201921175 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f904409a8b0 0x7f904409acb0 secure :-1 s=READY pgs=3163 cs=0 l=1 rev1=1 rx=0x7f9038003010 tx=0x7f903800a8a0).stop 2021-02-10T11:47:55.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.149 7f904a3f3700 1 -- 172.21.2.101:0/4201921175 shutdown_connections 2021-02-10T11:47:55.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.149 7f904a3f3700 1 --2- 172.21.2.101:0/4201921175 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f904409b2d0 0x7f90440beed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:55.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.149 7f904a3f3700 1 --2- 172.21.2.101:0/4201921175 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f904409a8b0 0x7f904409acb0 unknown :-1 s=CLOSED pgs=3163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:55.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.149 7f904a3f3700 1 --2- 172.21.2.101:0/4201921175 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f90240186c0 0x7f902401abc0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:55.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.149 7f904a3f3700 1 --2- 172.21.2.101:0/4201921175 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f90441a05e0 0x7f904409a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:55.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.149 7f904a3f3700 1 -- 172.21.2.101:0/4201921175 >> 172.21.2.101:0/4201921175 conn(0x7f90440a1770 msgr2=0x7f904409e210 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:55.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.149 7f904a3f3700 1 -- 172.21.2.101:0/4201921175 shutdown_connections 2021-02-10T11:47:55.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.149 7f904a3f3700 1 -- 172.21.2.101:0/4201921175 wait complete. 2021-02-10T11:47:55.153 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume authorize cephfs subvolume_0000000000359483 guest1 --group_name group_0000000000537794 --tenant_id tenant1 2021-02-10T11:47:55.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.581 7f0ec624e700 1 Processor -- start 2021-02-10T11:47:55.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.581 7f0ec624e700 1 -- start start 2021-02-10T11:47:55.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.582 7f0ec624e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0ec00e53a0 0x7f0ec01b0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:55.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.582 7f0ec624e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0ec010b050 0x7f0ec01091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:55.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.582 7f0ec624e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec0109ac0 con 0x7f0ec01b0b70 2021-02-10T11:47:55.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.582 7f0ec624e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec00fe2b0 con 0x7f0ec010b050 2021-02-10T11:47:55.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.582 7f0ec624e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec00e5f80 con 0x7f0ec00e53a0 2021-02-10T11:47:55.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.583 7f0ec486b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0ec010b050 0x7f0ec01091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:55.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.583 7f0ec486b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0ec010b050 0x7f0ec01091c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:44950/0 (socket says 172.21.2.101:44950) 2021-02-10T11:47:55.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.583 7f0ec486b700 1 -- 172.21.2.101:0/3763647039 learned_addr learned my addr 172.21.2.101:0/3763647039 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:55.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.583 7f0ebffff700 1 --2- 172.21.2.101:0/3763647039 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0ec00e53a0 0x7f0ec01b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:55.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.583 7f0ec486b700 1 -- 172.21.2.101:0/3763647039 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0ec00e53a0 msgr2=0x7f0ec01b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:55.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.583 7f0ec486b700 1 --2- 172.21.2.101:0/3763647039 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0ec00e53a0 0x7f0ec01b0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:55.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.583 7f0ec486b700 1 -- 172.21.2.101:0/3763647039 >> v1:172.21.2.108:6789/0 conn(0x7f0ec01b0b70 legacy=0x7f0ec01089d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:55.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.583 7f0ec486b700 1 -- 172.21.2.101:0/3763647039 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec00e57a0 con 0x7f0ec010b050 2021-02-10T11:47:55.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.583 7f0ebffff700 1 --2- 172.21.2.101:0/3763647039 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0ec00e53a0 0x7f0ec01b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:47:55.598 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.583 7f0ec486b700 1 --2- 172.21.2.101:0/3763647039 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0ec010b050 0x7f0ec01091c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f0eb8006aa0 tx=0x7f0eb8009f20).ready entity=mon.1 client_cookie=acbfde559e3ec39a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:55.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.583 7f0ebeffd700 1 -- 172.21.2.101:0/3763647039 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0eb8000f90 con 0x7f0ec010b050 2021-02-10T11:47:55.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.584 7f0ebeffd700 1 -- 172.21.2.101:0/3763647039 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb8003030 con 0x7f0ec010b050 2021-02-10T11:47:55.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.584 7f0ebeffd700 1 -- 172.21.2.101:0/3763647039 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0eb800b7b0 con 0x7f0ec010b050 2021-02-10T11:47:55.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.584 7f0ec624e700 1 -- 172.21.2.101:0/3763647039 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0ec010b050 msgr2=0x7f0ec01091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:55.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.584 7f0ec624e700 1 --2- 172.21.2.101:0/3763647039 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0ec010b050 0x7f0ec01091c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f0eb8006aa0 tx=0x7f0eb8009f20).stop 2021-02-10T11:47:55.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.584 7f0ec624e700 1 -- 172.21.2.101:0/3763647039 shutdown_connections 2021-02-10T11:47:55.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.584 7f0ec624e700 1 --2- 172.21.2.101:0/3763647039 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0ec010b050 0x7f0ec01091c0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:55.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.584 7f0ec624e700 1 --2- 172.21.2.101:0/3763647039 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0ec00e53a0 0x7f0ec01b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:55.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.584 7f0ec624e700 1 -- 172.21.2.101:0/3763647039 >> 172.21.2.101:0/3763647039 conn(0x7f0ec00ce4b0 msgr2=0x7f0ec00e45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:55.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.584 7f0ec624e700 1 -- 172.21.2.101:0/3763647039 shutdown_connections 2021-02-10T11:47:55.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.584 7f0ec624e700 1 -- 172.21.2.101:0/3763647039 wait complete. 2021-02-10T11:47:55.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.599 7f0ec624e700 1 Processor -- start 2021-02-10T11:47:55.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.599 7f0ec624e700 1 -- start start 2021-02-10T11:47:55.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.599 7f0ec624e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0ec01aa750 0x7f0ec00956f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:55.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.599 7f0ec624e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0ec01aab50 0x7f0ec0095d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:55.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.599 7f0ec624e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0ec01a9060 0x7f0ec01a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:55.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.599 7f0ec624e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec01aa1d0 con 0x7f0ec01a9060 2021-02-10T11:47:55.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.599 7f0ec624e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec019a090 con 0x7f0ec01aa750 2021-02-10T11:47:55.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.599 7f0ec624e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec019a530 con 0x7f0ec01aab50 2021-02-10T11:47:55.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.599 7f0ec486b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0ec01a9060 0x7f0ec01a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:55.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.599 7f0ec486b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0ec01a9060 0x7f0ec01a9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34468/0 (socket says 172.21.2.101:34468) 2021-02-10T11:47:55.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.599 7f0ec486b700 1 -- 172.21.2.101:0/1094264259 learned_addr learned my addr 172.21.2.101:0/1094264259 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:55.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.600 7f0ec486b700 1 -- 172.21.2.101:0/1094264259 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0ec01aab50 msgr2=0x7f0ec0095d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:55.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.600 7f0ec486b700 1 --2- 172.21.2.101:0/1094264259 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0ec01aab50 0x7f0ec0095d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:55.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.600 7f0ec486b700 1 -- 172.21.2.101:0/1094264259 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0ec01aa750 msgr2=0x7f0ec00956f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:55.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.600 7f0ec486b700 1 --2- 172.21.2.101:0/1094264259 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0ec01aa750 0x7f0ec00956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:55.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.600 7f0ec486b700 1 -- 172.21.2.101:0/1094264259 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec01a9a80 con 0x7f0ec01a9060 2021-02-10T11:47:55.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.600 7f0ec486b700 1 --2- 172.21.2.101:0/1094264259 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0ec01a9060 0x7f0ec01a9460 secure :-1 s=READY pgs=3164 cs=0 l=1 rev1=1 rx=0x7f0eb8009690 tx=0x7f0eb800bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:55.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.600 7f0ebcff9700 1 -- 172.21.2.101:0/1094264259 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0eb8006240 con 0x7f0ec01a9060 2021-02-10T11:47:55.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.600 7f0ebcff9700 1 -- 172.21.2.101:0/1094264259 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb8003020 con 0x7f0ec01a9060 2021-02-10T11:47:55.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.600 7f0ebcff9700 1 -- 172.21.2.101:0/1094264259 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0eb80056b0 con 0x7f0ec01a9060 2021-02-10T11:47:55.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.600 7f0ec624e700 1 -- 172.21.2.101:0/1094264259 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec01a9a80 con 0x7f0ec01a9060 2021-02-10T11:47:55.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.600 7f0ec624e700 1 -- 172.21.2.101:0/1094264259 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec01aa1d0 con 0x7f0ec01a9060 2021-02-10T11:47:55.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.601 7f0ebcff9700 1 -- 172.21.2.101:0/1094264259 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0eb8006030 con 0x7f0ec01a9060 2021-02-10T11:47:55.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.601 7f0ebcff9700 1 --2- 172.21.2.101:0/1094264259 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0ea00186c0 0x7f0ea001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:55.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.601 7f0ebcff9700 1 -- 172.21.2.101:0/1094264259 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0eb8032b70 con 0x7f0ec01a9060 2021-02-10T11:47:55.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.601 7f0ebffff700 1 --2- 172.21.2.101:0/1094264259 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0ea00186c0 0x7f0ea001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:55.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.601 7f0ebffff700 1 --2- 172.21.2.101:0/1094264259 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0ea00186c0 0x7f0ea001abc0 crc :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:55.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.632 7f0ec624e700 1 -- 172.21.2.101:0/1094264259 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec019a090 con 0x7f0ec01a9060 2021-02-10T11:47:55.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.634 7f0ebcff9700 1 -- 172.21.2.101:0/1094264259 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0eb8003000 con 0x7f0ec01a9060 2021-02-10T11:47:55.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.758 7f0ec624e700 1 -- 172.21.2.101:0/1094264259 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000359483", "target": ["mgr", ""], "tenant_id": "tenant1", "auth_id": "guest1", "group_name": "group_0000000000537794"}) v1 -- 0x7f0ec01aa1d0 con 0x7f0ea00186c0 2021-02-10T11:47:55.760 INFO:tasks.ceph.mgr.y.gibba001.stderr:Creating meta for ID guest1 with tenant tenant1 2021-02-10T11:47:55.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.789 7f0ebcff9700 1 -- 172.21.2.101:0/1094264259 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7f0eb0000960 con 0x7f0ea00186c0 2021-02-10T11:47:55.818 INFO:teuthology.orchestra.run.gibba001.stdout:AQDrxyNgv+AkLhAASDt+lyPe5hWvfDjDDHhBKw== 2021-02-10T11:47:55.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.823 7f0ec624e700 1 -- 172.21.2.101:0/1094264259 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0ea00186c0 msgr2=0x7f0ea001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:55.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.823 7f0ec624e700 1 --2- 172.21.2.101:0/1094264259 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0ea00186c0 0x7f0ea001abc0 crc :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:55.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.823 7f0ec624e700 1 -- 172.21.2.101:0/1094264259 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0ec01a9060 msgr2=0x7f0ec01a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:55.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.823 7f0ec624e700 1 --2- 172.21.2.101:0/1094264259 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0ec01a9060 0x7f0ec01a9460 secure :-1 s=READY pgs=3164 cs=0 l=1 rev1=1 rx=0x7f0eb8009690 tx=0x7f0eb800bd10).stop 2021-02-10T11:47:55.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.823 7f0ec624e700 1 -- 172.21.2.101:0/1094264259 shutdown_connections 2021-02-10T11:47:55.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.823 7f0ec624e700 1 --2- 172.21.2.101:0/1094264259 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0ea00186c0 0x7f0ea001abc0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:55.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.823 7f0ec624e700 1 --2- 172.21.2.101:0/1094264259 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0ec01aab50 0x7f0ec0095d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:55.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.823 7f0ec624e700 1 --2- 172.21.2.101:0/1094264259 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0ec01a9060 0x7f0ec01a9460 unknown :-1 s=CLOSED pgs=3164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:55.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.823 7f0ec624e700 1 --2- 172.21.2.101:0/1094264259 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0ec01aa750 0x7f0ec00956f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:55.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.823 7f0ec624e700 1 -- 172.21.2.101:0/1094264259 >> 172.21.2.101:0/1094264259 conn(0x7f0ec019b000 msgr2=0x7f0ec018c310 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:55.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.823 7f0ec624e700 1 -- 172.21.2.101:0/1094264259 shutdown_connections 2021-02-10T11:47:55.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:55.823 7f0ec624e700 1 -- 172.21.2.101:0/1094264259 wait complete. 2021-02-10T11:47:55.828 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.1 && sudo ls volumes 2021-02-10T11:47:56.177 INFO:teuthology.orchestra.run.gibba001.stdout:group_0000000000537794 2021-02-10T11:47:56.178 INFO:teuthology.orchestra.run.gibba001.stdout:_group_0000000000537794:subvolume_0000000000359483.meta 2021-02-10T11:47:56.178 INFO:teuthology.orchestra.run.gibba001.stdout:_group_0000000000537794:subvolume_0000000000359484.meta 2021-02-10T11:47:56.178 INFO:teuthology.orchestra.run.gibba001.stdout:$guest1.meta 2021-02-10T11:47:56.179 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo sudo cat '/home/ubuntu/cephtest/mnt.0/volumes/$guest1.meta' 2021-02-10T11:47:56.545 INFO:teuthology.orchestra.run.gibba001.stdout:{"compat_version": 1, "tenant_id": "tenant1", "version": 5, "dirty": false, "subvolumes": {"group_0000000000537794/subvolume_0000000000359483": {"access_level": "rw", "dirty": false}}} 2021-02-10T11:47:56.545 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume authorize cephfs subvolume_0000000000359484 guest1 --group_name group_0000000000537794 --tenant_id tenant1 2021-02-10T11:47:56.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.973 7ffbd2411700 1 Processor -- start 2021-02-10T11:47:56.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.973 7ffbd2411700 1 -- start start 2021-02-10T11:47:56.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.973 7ffbd2411700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ffbcc0e53a0 0x7ffbcc1b0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:56.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.973 7ffbd2411700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ffbcc1b0b70 0x7ffbcc1089d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:56.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.973 7ffbd2411700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbcc109ac0 con 0x7ffbcc108ff0 2021-02-10T11:47:56.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.973 7ffbd2411700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbcc0e5f80 con 0x7ffbcc1b0b70 2021-02-10T11:47:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.973 7ffbd2411700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbcc0e57a0 con 0x7ffbcc0e53a0 2021-02-10T11:47:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.974 7ffbcb7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ffbcc1b0b70 0x7ffbcc1089d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.974 7ffbcb7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ffbcc1b0b70 0x7ffbcc1089d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34482/0 (socket says 172.21.2.101:34482) 2021-02-10T11:47:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.974 7ffbcb7fe700 1 -- 172.21.2.101:0/3767490077 learned_addr learned my addr 172.21.2.101:0/3767490077 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.974 7ffbcb7fe700 1 -- 172.21.2.101:0/3767490077 >> v2:172.21.2.108:3300/0 conn(0x7ffbcc0e53a0 msgr2=0x7ffbcc1b0550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.974 7ffbcb7fe700 1 --2- 172.21.2.101:0/3767490077 >> v2:172.21.2.108:3300/0 conn(0x7ffbcc0e53a0 0x7ffbcc1b0550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.974 7ffbcb7fe700 1 -- 172.21.2.101:0/3767490077 >> v1:172.21.2.108:6789/0 conn(0x7ffbcc108ff0 legacy=0x7ffbcc10a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.974 7ffbcb7fe700 1 -- 172.21.2.101:0/3767490077 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbcc088070 con 0x7ffbcc1b0b70 2021-02-10T11:47:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.974 7ffbcb7fe700 1 --2- 172.21.2.101:0/3767490077 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffbcc1b0b70 0x7ffbcc1089d0 secure :-1 s=READY pgs=3165 cs=0 l=1 rev1=1 rx=0x7ffbc0006700 tx=0x7ffbc0009b80).ready entity=mon.0 client_cookie=3aa6a9143ce9bd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.974 7ffbcaffd700 1 -- 172.21.2.101:0/3767490077 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffbc000c0c0 con 0x7ffbcc1b0b70 2021-02-10T11:47:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.974 7ffbcaffd700 1 -- 172.21.2.101:0/3767490077 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc000c400 con 0x7ffbcc1b0b70 2021-02-10T11:47:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.974 7ffbcaffd700 1 -- 172.21.2.101:0/3767490077 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffbc000c8e0 con 0x7ffbcc1b0b70 2021-02-10T11:47:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.974 7ffbd2411700 1 -- 172.21.2.101:0/3767490077 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffbcc1b0b70 msgr2=0x7ffbcc1089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:56.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.974 7ffbd2411700 1 --2- 172.21.2.101:0/3767490077 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffbcc1b0b70 0x7ffbcc1089d0 secure :-1 s=READY pgs=3165 cs=0 l=1 rev1=1 rx=0x7ffbc0006700 tx=0x7ffbc0009b80).stop 2021-02-10T11:47:56.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.975 7ffbd2411700 1 -- 172.21.2.101:0/3767490077 shutdown_connections 2021-02-10T11:47:56.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.975 7ffbd2411700 1 --2- 172.21.2.101:0/3767490077 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffbcc1b0b70 0x7ffbcc1089d0 unknown :-1 s=CLOSED pgs=3165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:56.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.975 7ffbd2411700 1 --2- 172.21.2.101:0/3767490077 >> v2:172.21.2.108:3300/0 conn(0x7ffbcc0e53a0 0x7ffbcc1b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:56.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.975 7ffbd2411700 1 -- 172.21.2.101:0/3767490077 >> 172.21.2.101:0/3767490077 conn(0x7ffbcc0ce4b0 msgr2=0x7ffbcc0e45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:56.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.975 7ffbd2411700 1 -- 172.21.2.101:0/3767490077 shutdown_connections 2021-02-10T11:47:56.988 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.975 7ffbd2411700 1 -- 172.21.2.101:0/3767490077 wait complete. 2021-02-10T11:47:56.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.990 7ffbd2411700 1 Processor -- start 2021-02-10T11:47:56.989 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.990 7ffbd2411700 1 -- start start 2021-02-10T11:47:56.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.990 7ffbd2411700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffbcc1a8820 0x7ffbcc1a8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:56.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.990 7ffbd2411700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffbcc08c6d0 0x7ffbcc08cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:56.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.990 7ffbd2411700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffbcc08d0f0 0x7ffbcc08d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:56.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.990 7ffbd2411700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc1a93a0 con 0x7ffbcc08c6d0 2021-02-10T11:47:56.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.990 7ffbd2411700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc1a9920 con 0x7ffbcc08d0f0 2021-02-10T11:47:56.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.990 7ffbd2411700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc1a9b00 con 0x7ffbcc1a8820 2021-02-10T11:47:56.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.991 7ffbcb7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffbcc08c6d0 0x7ffbcc08cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:56.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.991 7ffbcb7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffbcc08c6d0 0x7ffbcc08cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34490/0 (socket says 172.21.2.101:34490) 2021-02-10T11:47:56.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.991 7ffbcb7fe700 1 -- 172.21.2.101:0/3485829682 learned_addr learned my addr 172.21.2.101:0/3485829682 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:56.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.991 7ffbcb7fe700 1 -- 172.21.2.101:0/3485829682 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffbcc1a8820 msgr2=0x7ffbcc1a8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:56.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.991 7ffbcb7fe700 1 --2- 172.21.2.101:0/3485829682 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffbcc1a8820 0x7ffbcc1a8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:56.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.991 7ffbcb7fe700 1 -- 172.21.2.101:0/3485829682 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffbcc08d0f0 msgr2=0x7ffbcc08d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:56.990 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.991 7ffbcb7fe700 1 --2- 172.21.2.101:0/3485829682 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffbcc08d0f0 0x7ffbcc08d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:56.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.991 7ffbcb7fe700 1 -- 172.21.2.101:0/3485829682 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbcc1a9d20 con 0x7ffbcc08c6d0 2021-02-10T11:47:56.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.991 7ffbcb7fe700 1 --2- 172.21.2.101:0/3485829682 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffbcc08c6d0 0x7ffbcc08cad0 secure :-1 s=READY pgs=3166 cs=0 l=1 rev1=1 rx=0x7ffbc00066a0 tx=0x7ffbc000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:56.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.991 7ffbc8ff9700 1 -- 172.21.2.101:0/3485829682 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffbc000bd80 con 0x7ffbcc08c6d0 2021-02-10T11:47:56.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.991 7ffbc8ff9700 1 -- 172.21.2.101:0/3485829682 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc00052e0 con 0x7ffbcc08c6d0 2021-02-10T11:47:56.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.991 7ffbd2411700 1 -- 172.21.2.101:0/3485829682 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbcc1a9d20 con 0x7ffbcc08c6d0 2021-02-10T11:47:56.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.991 7ffbd2411700 1 -- 172.21.2.101:0/3485829682 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbcc1aa2b0 con 0x7ffbcc08c6d0 2021-02-10T11:47:56.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.991 7ffbc8ff9700 1 -- 172.21.2.101:0/3485829682 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffbc0005800 con 0x7ffbcc08c6d0 2021-02-10T11:47:56.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.992 7ffbc8ff9700 1 -- 172.21.2.101:0/3485829682 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ffbc00057d0 con 0x7ffbcc08c6d0 2021-02-10T11:47:56.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.992 7ffbc8ff9700 1 --2- 172.21.2.101:0/3485829682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffbac0186c0 0x7ffbac01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:56.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.992 7ffbc8ff9700 1 -- 172.21.2.101:0/3485829682 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ffbc0032670 con 0x7ffbcc08c6d0 2021-02-10T11:47:56.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.992 7ffbcbfff700 1 --2- 172.21.2.101:0/3485829682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffbac0186c0 0x7ffbac01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:56.991 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:56.993 7ffbcbfff700 1 --2- 172.21.2.101:0/3485829682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffbac0186c0 0x7ffbac01abc0 crc :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:57.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:57.023 7ffbd2411700 1 -- 172.21.2.101:0/3485829682 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbcc103810 con 0x7ffbcc08c6d0 2021-02-10T11:47:57.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:57.025 7ffbc8ff9700 1 -- 172.21.2.101:0/3485829682 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ffbc002eff0 con 0x7ffbcc08c6d0 2021-02-10T11:47:57.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:57.150 7ffbd2411700 1 -- 172.21.2.101:0/3485829682 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000359484", "target": ["mgr", ""], "tenant_id": "tenant1", "auth_id": "guest1", "group_name": "group_0000000000537794"}) v1 -- 0x7ffbcc1a93a0 con 0x7ffbac0186c0 2021-02-10T11:47:57.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:57.182 7ffbc8ff9700 1 -- 172.21.2.101:0/3485829682 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7ffbbc006e40 con 0x7ffbac0186c0 2021-02-10T11:47:57.210 INFO:teuthology.orchestra.run.gibba001.stdout:AQDrxyNgv+AkLhAASDt+lyPe5hWvfDjDDHhBKw== 2021-02-10T11:47:57.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:57.215 7ffbd2411700 1 -- 172.21.2.101:0/3485829682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffbac0186c0 msgr2=0x7ffbac01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:57.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:57.215 7ffbd2411700 1 --2- 172.21.2.101:0/3485829682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffbac0186c0 0x7ffbac01abc0 crc :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:57.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:57.215 7ffbd2411700 1 -- 172.21.2.101:0/3485829682 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffbcc08c6d0 msgr2=0x7ffbcc08cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:57.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:57.215 7ffbd2411700 1 --2- 172.21.2.101:0/3485829682 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffbcc08c6d0 0x7ffbcc08cad0 secure :-1 s=READY pgs=3166 cs=0 l=1 rev1=1 rx=0x7ffbc00066a0 tx=0x7ffbc000b990).stop 2021-02-10T11:47:57.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:57.215 7ffbd2411700 1 -- 172.21.2.101:0/3485829682 shutdown_connections 2021-02-10T11:47:57.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:57.215 7ffbd2411700 1 --2- 172.21.2.101:0/3485829682 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ffbcc08d0f0 0x7ffbcc08d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:57.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:57.215 7ffbd2411700 1 --2- 172.21.2.101:0/3485829682 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ffbcc08c6d0 0x7ffbcc08cad0 unknown :-1 s=CLOSED pgs=3166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:57.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:57.215 7ffbd2411700 1 --2- 172.21.2.101:0/3485829682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ffbac0186c0 0x7ffbac01abc0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:57.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:57.215 7ffbd2411700 1 --2- 172.21.2.101:0/3485829682 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ffbcc1a8820 0x7ffbcc1a8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:57.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:57.215 7ffbd2411700 1 -- 172.21.2.101:0/3485829682 >> 172.21.2.101:0/3485829682 conn(0x7ffbcc18cb60 msgr2=0x7ffbcc199f40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:57.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:57.215 7ffbd2411700 1 -- 172.21.2.101:0/3485829682 shutdown_connections 2021-02-10T11:47:57.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:57.215 7ffbd2411700 1 -- 172.21.2.101:0/3485829682 wait complete. 2021-02-10T11:47:57.220 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.1 && sudo sed -i s/false/true/g 'volumes/$guest1.meta' 2021-02-10T11:47:57.588 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume deauthorize cephfs subvolume_0000000000359484 guest1 --group_name group_0000000000537794 2021-02-10T11:47:58.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.006 7f60bde03700 1 Processor -- start 2021-02-10T11:47:58.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.006 7f60bde03700 1 -- start start 2021-02-10T11:47:58.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.007 7f60bde03700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f60b80d1890 0x7f60b81b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:58.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.007 7f60bde03700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f60b81b09f0 0x7f60b81089a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:58.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.007 7f60bde03700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60b8109a60 con 0x7f60b80d1c90 2021-02-10T11:47:58.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.007 7f60bde03700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60b80e5f50 con 0x7f60b80d1890 2021-02-10T11:47:58.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.007 7f60bde03700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60b81097b0 con 0x7f60b81b09f0 2021-02-10T11:47:58.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.007 7f60b7fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f60b80d1c90 0x7f60b8108fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:34994/0 (socket says 172.21.2.101:34994) 2021-02-10T11:47:58.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.007 7f60b7fff700 1 -- 172.21.2.101:0/3874450308 learned_addr learned my addr 172.21.2.101:0/3874450308 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:47:58.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.008 7f60b6ffd700 1 --2- 172.21.2.101:0/3874450308 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f60b81b09f0 0x7f60b81089a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:58.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.008 7f60b77fe700 1 --2- 172.21.2.101:0/3874450308 >> v2:172.21.2.108:3300/0 conn(0x7f60b80d1890 0x7f60b81b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:58.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.008 7f60b67fc700 1 -- 172.21.2.101:0/3874450308 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538739685 0 0) 0x7f60b0001500 con 0x7f60b80d1c90 2021-02-10T11:47:58.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.008 7f60b67fc700 1 -- 172.21.2.101:0/3874450308 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60a4002fe0 con 0x7f60b80d1c90 2021-02-10T11:47:58.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.008 7f60b67fc700 1 -- 172.21.2.101:0/3874450308 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 933074579 0 0) 0x7f60b0001500 con 0x7f60b80d1c90 2021-02-10T11:47:58.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.008 7f60b67fc700 1 -- 172.21.2.101:0/3874450308 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f60b81b09f0 msgr2=0x7f60b81089a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:58.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.008 7f60b67fc700 1 --2- 172.21.2.101:0/3874450308 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f60b81b09f0 0x7f60b81089a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:58.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.008 7f60b67fc700 1 -- 172.21.2.101:0/3874450308 >> v2:172.21.2.108:3300/0 conn(0x7f60b80d1890 msgr2=0x7f60b81b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:58.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.008 7f60b67fc700 1 --2- 172.21.2.101:0/3874450308 >> v2:172.21.2.108:3300/0 conn(0x7f60b80d1890 0x7f60b81b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:58.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.008 7f60b67fc700 1 -- 172.21.2.101:0/3874450308 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b8088070 con 0x7f60b80d1c90 2021-02-10T11:47:58.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.008 7f60b67fc700 1 -- 172.21.2.101:0/3874450308 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f60b0001a00 con 0x7f60b80d1c90 2021-02-10T11:47:58.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.008 7f60b67fc700 1 -- 172.21.2.101:0/3874450308 >> v1:172.21.2.101:6789/0 conn(0x7f60b80d1c90 legacy=0x7f60b8108fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:58.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.008 7f60b67fc700 1 --2- 172.21.2.101:0/3874450308 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f60a4003b90 0x7f60a4006000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:58.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.008 7f60b67fc700 1 --2- 172.21.2.101:0/3874450308 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60a404a8b0 0x7f60a404cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:58.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.008 7f60b67fc700 1 --2- 172.21.2.101:0/3874450308 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f60a404d650 0x7f60a404fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:58.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.008 7f60b67fc700 1 -- 172.21.2.101:0/3874450308 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a4050420 con 0x7f60a404a8b0 2021-02-10T11:47:58.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.008 7f60b67fc700 1 -- 172.21.2.101:0/3874450308 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a4050640 con 0x7f60a404d650 2021-02-10T11:47:58.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.008 7f60b67fc700 1 -- 172.21.2.101:0/3874450308 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a4050860 con 0x7f60a4003b90 2021-02-10T11:47:58.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.008 7f60b77fe700 1 --2- 172.21.2.101:0/3874450308 >> v2:172.21.2.108:3300/0 conn(0x7f60b80d1890 0x7f60b81b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:47:58.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.008 7f60b6ffd700 1 --2- 172.21.2.101:0/3874450308 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f60b81b09f0 0x7f60b81089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:47:58.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.008 7f60b6ffd700 1 --2- 172.21.2.101:0/3874450308 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60a404a8b0 0x7f60a404cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:58.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.009 7f60b6ffd700 1 -- 172.21.2.101:0/3874450308 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f60a4003b90 msgr2=0x7f60a4006000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:58.023 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.009 7f60b6ffd700 1 --2- 172.21.2.101:0/3874450308 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f60a4003b90 0x7f60a4006000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:58.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.009 7f60b6ffd700 1 -- 172.21.2.101:0/3874450308 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f60a404d650 msgr2=0x7f60a404fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:47:58.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.009 7f60b6ffd700 1 --2- 172.21.2.101:0/3874450308 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f60a404d650 0x7f60a404fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:58.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.009 7f60b6ffd700 1 -- 172.21.2.101:0/3874450308 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a4050b00 con 0x7f60a404a8b0 2021-02-10T11:47:58.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.009 7f60b6ffd700 1 --2- 172.21.2.101:0/3874450308 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60a404a8b0 0x7f60a404cdb0 secure :-1 s=READY pgs=3168 cs=0 l=1 rev1=1 rx=0x7f60a80063a0 tx=0x7f60a8009aa0).ready entity=mon.0 client_cookie=e5e83e0bcce58fc9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:58.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.009 7f60b67fc700 1 -- 172.21.2.101:0/3874450308 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f60a800c340 con 0x7f60a404a8b0 2021-02-10T11:47:58.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.009 7f60b67fc700 1 -- 172.21.2.101:0/3874450308 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a800c6c0 con 0x7f60a404a8b0 2021-02-10T11:47:58.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.009 7f60b67fc700 1 -- 172.21.2.101:0/3874450308 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f60a800c340 con 0x7f60a404a8b0 2021-02-10T11:47:58.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.009 7f60bde03700 1 -- 172.21.2.101:0/3874450308 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60a404a8b0 msgr2=0x7f60a404cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:58.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.009 7f60bde03700 1 --2- 172.21.2.101:0/3874450308 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60a404a8b0 0x7f60a404cdb0 secure :-1 s=READY pgs=3168 cs=0 l=1 rev1=1 rx=0x7f60a80063a0 tx=0x7f60a8009aa0).stop 2021-02-10T11:47:58.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.009 7f60b7fff700 1 -- 172.21.2.101:0/3874450308 reap_dead start 2021-02-10T11:47:58.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.009 7f60bde03700 1 -- 172.21.2.101:0/3874450308 shutdown_connections 2021-02-10T11:47:58.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.009 7f60bde03700 1 -- 172.21.2.101:0/3874450308 >> 172.21.2.101:0/3874450308 conn(0x7f60b80fdee0 msgr2=0x7f60b80ce450 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:58.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.009 7f60bde03700 1 -- 172.21.2.101:0/3874450308 shutdown_connections 2021-02-10T11:47:58.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.010 7f60bde03700 1 -- 172.21.2.101:0/3874450308 wait complete. 2021-02-10T11:47:58.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.024 7f60bde03700 1 Processor -- start 2021-02-10T11:47:58.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.024 7f60bde03700 1 -- start start 2021-02-10T11:47:58.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.025 7f60bde03700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f60b8092f50 0x7f60b81a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:58.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.025 7f60bde03700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f60b8093350 0x7f60b807e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:58.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.025 7f60bde03700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60b81a1ea0 0x7f60b807ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:58.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.025 7f60bde03700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b81a24e0 con 0x7f60b81a1ea0 2021-02-10T11:47:58.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.025 7f60bde03700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b818a680 con 0x7f60b8092f50 2021-02-10T11:47:58.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.025 7f60bde03700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60b818aae0 con 0x7f60b8093350 2021-02-10T11:47:58.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.025 7f60b7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60b81a1ea0 0x7f60b807ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:58.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.025 7f60b7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60b81a1ea0 0x7f60b807ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34514/0 (socket says 172.21.2.101:34514) 2021-02-10T11:47:58.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.025 7f60b7fff700 1 -- 172.21.2.101:0/462354459 learned_addr learned my addr 172.21.2.101:0/462354459 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:58.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.025 7f60b7fff700 1 -- 172.21.2.101:0/462354459 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f60b8093350 msgr2=0x7f60b807e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:58.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.025 7f60b7fff700 1 --2- 172.21.2.101:0/462354459 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f60b8093350 0x7f60b807e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:58.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.025 7f60b7fff700 1 -- 172.21.2.101:0/462354459 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f60b8092f50 msgr2=0x7f60b81a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:58.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.025 7f60b7fff700 1 --2- 172.21.2.101:0/462354459 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f60b8092f50 0x7f60b81a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:58.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.025 7f60b7fff700 1 -- 172.21.2.101:0/462354459 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b807f330 con 0x7f60b81a1ea0 2021-02-10T11:47:58.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.025 7f60b7fff700 1 --2- 172.21.2.101:0/462354459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60b81a1ea0 0x7f60b807ed10 secure :-1 s=READY pgs=3169 cs=0 l=1 rev1=1 rx=0x7f60b0007f90 tx=0x7f60b000ac50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:58.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.026 7f60affff700 1 -- 172.21.2.101:0/462354459 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f60b0000f20 con 0x7f60b81a1ea0 2021-02-10T11:47:58.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.026 7f60affff700 1 -- 172.21.2.101:0/462354459 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b0007020 con 0x7f60b81a1ea0 2021-02-10T11:47:58.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.026 7f60affff700 1 -- 172.21.2.101:0/462354459 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f60b00074b0 con 0x7f60b81a1ea0 2021-02-10T11:47:58.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.026 7f60bde03700 1 -- 172.21.2.101:0/462354459 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b807f330 con 0x7f60b81a1ea0 2021-02-10T11:47:58.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.026 7f60bde03700 1 -- 172.21.2.101:0/462354459 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b807f330 con 0x7f60b81a1ea0 2021-02-10T11:47:58.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.026 7f60affff700 1 -- 172.21.2.101:0/462354459 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f60b0007200 con 0x7f60b81a1ea0 2021-02-10T11:47:58.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.026 7f60affff700 1 --2- 172.21.2.101:0/462354459 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f609c0186c0 0x7f609c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:58.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.026 7f60affff700 1 -- 172.21.2.101:0/462354459 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f60b0032780 con 0x7f60b81a1ea0 2021-02-10T11:47:58.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.026 7f60b77fe700 1 --2- 172.21.2.101:0/462354459 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f609c0186c0 0x7f609c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:58.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.027 7f60b77fe700 1 --2- 172.21.2.101:0/462354459 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f609c0186c0 0x7f609c01abc0 crc :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:58.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.057 7f60bde03700 1 -- 172.21.2.101:0/462354459 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b807e3b0 con 0x7f60b81a1ea0 2021-02-10T11:47:58.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.059 7f60affff700 1 -- 172.21.2.101:0/462354459 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f60b002f470 con 0x7f60b81a1ea0 2021-02-10T11:47:58.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.184 7f60bde03700 1 -- 172.21.2.101:0/462354459 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume deauthorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000359484", "target": ["mgr", ""], "auth_id": "guest1", "group_name": "group_0000000000537794"}) v1 -- 0x7f60b8109a60 con 0x7f609c0186c0 2021-02-10T11:47:58.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.245 7f60affff700 1 -- 172.21.2.101:0/462354459 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f60a00052e0 con 0x7f609c0186c0 2021-02-10T11:47:58.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.249 7f60bde03700 1 -- 172.21.2.101:0/462354459 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f609c0186c0 msgr2=0x7f609c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:58.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.249 7f60bde03700 1 --2- 172.21.2.101:0/462354459 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f609c0186c0 0x7f609c01abc0 crc :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:58.245 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.249 7f60bde03700 1 -- 172.21.2.101:0/462354459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60b81a1ea0 msgr2=0x7f60b807ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:58.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.249 7f60bde03700 1 --2- 172.21.2.101:0/462354459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60b81a1ea0 0x7f60b807ed10 secure :-1 s=READY pgs=3169 cs=0 l=1 rev1=1 rx=0x7f60b0007f90 tx=0x7f60b000ac50).stop 2021-02-10T11:47:58.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.249 7f60bde03700 1 -- 172.21.2.101:0/462354459 shutdown_connections 2021-02-10T11:47:58.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.249 7f60bde03700 1 --2- 172.21.2.101:0/462354459 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f609c0186c0 0x7f609c01abc0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:58.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.249 7f60bde03700 1 --2- 172.21.2.101:0/462354459 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f60b8093350 0x7f60b807e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:58.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.249 7f60bde03700 1 --2- 172.21.2.101:0/462354459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f60b81a1ea0 0x7f60b807ed10 unknown :-1 s=CLOSED pgs=3169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:58.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.249 7f60bde03700 1 --2- 172.21.2.101:0/462354459 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f60b8092f50 0x7f60b81a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:58.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.249 7f60bde03700 1 -- 172.21.2.101:0/462354459 >> 172.21.2.101:0/462354459 conn(0x7f60b81abcc0 msgr2=0x7f60b81ae430 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:58.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.249 7f60bde03700 1 -- 172.21.2.101:0/462354459 shutdown_connections 2021-02-10T11:47:58.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:58.249 7f60bde03700 1 -- 172.21.2.101:0/462354459 wait complete. 2021-02-10T11:47:58.254 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo sudo cat '/home/ubuntu/cephtest/mnt.0/volumes/$guest1.meta' 2021-02-10T11:47:58.616 INFO:teuthology.orchestra.run.gibba001.stdout:{"tenant_id": "tenant1", "version": 5, "dirty": false, "compat_version": 1, "subvolumes": {"group_0000000000537794/subvolume_0000000000359483": {"access_level": "rw", "dirty": false}}} 2021-02-10T11:47:58.617 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume deauthorize cephfs subvolume_0000000000359483 guest1 --group_name group_0000000000537794 2021-02-10T11:47:59.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.036 7f2a5f653700 1 Processor -- start 2021-02-10T11:47:59.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.036 7f2a5f653700 1 -- start start 2021-02-10T11:47:59.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.037 7f2a5f653700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f2a580dce80 0x7f2a580d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:59.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.037 7f2a5f653700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2a580eb220 0x7f2a580eb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:59.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.037 7f2a5f653700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a5810aad0 0x7f2a5810aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:59.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.037 7f2a5f653700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5810b570 con 0x7f2a580eb220 2021-02-10T11:47:59.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.037 7f2a5f653700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5810a590 con 0x7f2a580dce80 2021-02-10T11:47:59.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.037 7f2a5f653700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580881d0 con 0x7f2a5810aad0 2021-02-10T11:47:59.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.037 7f2a5cc6e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2a580eb220 0x7f2a580eb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:59.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.037 7f2a5cc6e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2a580eb220 0x7f2a580eb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34522/0 (socket says 172.21.2.101:34522) 2021-02-10T11:47:59.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.037 7f2a5cc6e700 1 -- 172.21.2.101:0/3784582172 learned_addr learned my addr 172.21.2.101:0/3784582172 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:59.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.037 7f2a5cc6e700 1 -- 172.21.2.101:0/3784582172 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a5810aad0 msgr2=0x7f2a5810aed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:59.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.037 7f2a5cc6e700 1 --2- 172.21.2.101:0/3784582172 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a5810aad0 0x7f2a5810aed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:59.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.037 7f2a5cc6e700 1 -- 172.21.2.101:0/3784582172 >> v2:172.21.2.108:3300/0 conn(0x7f2a580dce80 msgr2=0x7f2a580d14f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:47:59.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.037 7f2a5cc6e700 1 --2- 172.21.2.101:0/3784582172 >> v2:172.21.2.108:3300/0 conn(0x7f2a580dce80 0x7f2a580d14f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:59.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.037 7f2a5cc6e700 1 -- 172.21.2.101:0/3784582172 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a580d1f30 con 0x7f2a580eb220 2021-02-10T11:47:59.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.037 7f2a5cc6e700 1 --2- 172.21.2.101:0/3784582172 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a580eb220 0x7f2a580eb620 secure :-1 s=READY pgs=3170 cs=0 l=1 rev1=1 rx=0x7f2a4c006700 tx=0x7f2a4c009b80).ready entity=mon.0 client_cookie=cfea414f7925f53d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:59.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.038 7f2a4bfff700 1 -- 172.21.2.101:0/3784582172 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2a4c00c0c0 con 0x7f2a580eb220 2021-02-10T11:47:59.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.038 7f2a4bfff700 1 -- 172.21.2.101:0/3784582172 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4c00c440 con 0x7f2a580eb220 2021-02-10T11:47:59.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.038 7f2a4bfff700 1 -- 172.21.2.101:0/3784582172 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2a4c00c960 con 0x7f2a580eb220 2021-02-10T11:47:59.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.038 7f2a5f653700 1 -- 172.21.2.101:0/3784582172 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a580eb220 msgr2=0x7f2a580eb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:59.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.038 7f2a5f653700 1 --2- 172.21.2.101:0/3784582172 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a580eb220 0x7f2a580eb620 secure :-1 s=READY pgs=3170 cs=0 l=1 rev1=1 rx=0x7f2a4c006700 tx=0x7f2a4c009b80).stop 2021-02-10T11:47:59.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.038 7f2a5f653700 1 -- 172.21.2.101:0/3784582172 shutdown_connections 2021-02-10T11:47:59.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.039 7f2a5f653700 1 --2- 172.21.2.101:0/3784582172 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a5810aad0 0x7f2a5810aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:59.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.039 7f2a5f653700 1 --2- 172.21.2.101:0/3784582172 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a580eb220 0x7f2a580eb620 unknown :-1 s=CLOSED pgs=3170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:59.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.039 7f2a5f653700 1 --2- 172.21.2.101:0/3784582172 >> v2:172.21.2.108:3300/0 conn(0x7f2a580dce80 0x7f2a580d14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:59.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.039 7f2a5f653700 1 -- 172.21.2.101:0/3784582172 >> 172.21.2.101:0/3784582172 conn(0x7f2a580ea630 msgr2=0x7f2a580d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:59.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.039 7f2a5f653700 1 -- 172.21.2.101:0/3784582172 shutdown_connections 2021-02-10T11:47:59.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.039 7f2a5f653700 1 -- 172.21.2.101:0/3784582172 wait complete. 2021-02-10T11:47:59.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.053 7f2a5f653700 1 Processor -- start 2021-02-10T11:47:59.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.053 7f2a5f653700 1 -- start start 2021-02-10T11:47:59.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.054 7f2a5f653700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2a58096340 0x7f2a58095a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:59.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.054 7f2a5f653700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a58193b90 0x7f2a581aa510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:59.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.054 7f2a5f653700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a58193f90 0x7f2a58096760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:59.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.054 7f2a5f653700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a581aa010 con 0x7f2a58193b90 2021-02-10T11:47:59.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.054 7f2a5f653700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a581aab30 con 0x7f2a58096340 2021-02-10T11:47:59.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.054 7f2a5f653700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a581aad20 con 0x7f2a58193f90 2021-02-10T11:47:59.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.054 7f2a5cc6e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a58193b90 0x7f2a581aa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:59.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.054 7f2a5cc6e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a58193b90 0x7f2a581aa510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34526/0 (socket says 172.21.2.101:34526) 2021-02-10T11:47:59.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.054 7f2a5cc6e700 1 -- 172.21.2.101:0/913147355 learned_addr learned my addr 172.21.2.101:0/913147355 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:47:59.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.054 7f2a5cc6e700 1 -- 172.21.2.101:0/913147355 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a58193f90 msgr2=0x7f2a58096760 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:47:59.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.054 7f2a5cc6e700 1 --2- 172.21.2.101:0/913147355 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a58193f90 0x7f2a58096760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:59.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.054 7f2a5cc6e700 1 -- 172.21.2.101:0/913147355 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2a58096340 msgr2=0x7f2a58095a90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:47:59.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.054 7f2a5cc6e700 1 --2- 172.21.2.101:0/913147355 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2a58096340 0x7f2a58095a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:59.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.054 7f2a5cc6e700 1 -- 172.21.2.101:0/913147355 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a58096d80 con 0x7f2a58193b90 2021-02-10T11:47:59.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.054 7f2a5cc6e700 1 --2- 172.21.2.101:0/913147355 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a58193b90 0x7f2a581aa510 secure :-1 s=READY pgs=3171 cs=0 l=1 rev1=1 rx=0x7f2a4c00b5e0 tx=0x7f2a4c00b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:59.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.054 7f2a49ffb700 1 -- 172.21.2.101:0/913147355 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2a4c00bd70 con 0x7f2a58193b90 2021-02-10T11:47:59.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.054 7f2a49ffb700 1 -- 172.21.2.101:0/913147355 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4c002030 con 0x7f2a58193b90 2021-02-10T11:47:59.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.055 7f2a49ffb700 1 -- 172.21.2.101:0/913147355 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2a4c002550 con 0x7f2a58193b90 2021-02-10T11:47:59.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.055 7f2a5f653700 1 -- 172.21.2.101:0/913147355 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a58096d80 con 0x7f2a58193b90 2021-02-10T11:47:59.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.055 7f2a5f653700 1 -- 172.21.2.101:0/913147355 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a58097310 con 0x7f2a58193b90 2021-02-10T11:47:59.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.055 7f2a49ffb700 1 -- 172.21.2.101:0/913147355 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2a4c000c90 con 0x7f2a58193b90 2021-02-10T11:47:59.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.055 7f2a49ffb700 1 --2- 172.21.2.101:0/913147355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a380186c0 0x7f2a3801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:47:59.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.055 7f2a49ffb700 1 -- 172.21.2.101:0/913147355 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2a4c0327e0 con 0x7f2a58193b90 2021-02-10T11:47:59.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.055 7f2a5d46f700 1 --2- 172.21.2.101:0/913147355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a380186c0 0x7f2a3801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:47:59.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.056 7f2a5d46f700 1 --2- 172.21.2.101:0/913147355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a380186c0 0x7f2a3801abc0 crc :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:47:59.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.087 7f2a5f653700 1 -- 172.21.2.101:0/913147355 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a580f9350 con 0x7f2a58193b90 2021-02-10T11:47:59.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.090 7f2a49ffb700 1 -- 172.21.2.101:0/913147355 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2a4c000940 con 0x7f2a58193b90 2021-02-10T11:47:59.212 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.215 7f2a5f653700 1 -- 172.21.2.101:0/913147355 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume deauthorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000359483", "target": ["mgr", ""], "auth_id": "guest1", "group_name": "group_0000000000537794"}) v1 -- 0x7f2a58097310 con 0x7f2a380186c0 2021-02-10T11:47:59.248 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.251 7f2a49ffb700 1 -- 172.21.2.101:0/913147355 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f2a540009b0 con 0x7f2a380186c0 2021-02-10T11:47:59.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.281 7f2a5f653700 1 -- 172.21.2.101:0/913147355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a380186c0 msgr2=0x7f2a3801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:59.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.281 7f2a5f653700 1 --2- 172.21.2.101:0/913147355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a380186c0 0x7f2a3801abc0 crc :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:59.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.281 7f2a5f653700 1 -- 172.21.2.101:0/913147355 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a58193b90 msgr2=0x7f2a581aa510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:47:59.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.281 7f2a5f653700 1 --2- 172.21.2.101:0/913147355 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a58193b90 0x7f2a581aa510 secure :-1 s=READY pgs=3171 cs=0 l=1 rev1=1 rx=0x7f2a4c00b5e0 tx=0x7f2a4c00b980).stop 2021-02-10T11:47:59.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.281 7f2a5f653700 1 -- 172.21.2.101:0/913147355 shutdown_connections 2021-02-10T11:47:59.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.281 7f2a5f653700 1 --2- 172.21.2.101:0/913147355 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2a380186c0 0x7f2a3801abc0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:59.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.281 7f2a5f653700 1 --2- 172.21.2.101:0/913147355 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2a58193f90 0x7f2a58096760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:59.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.281 7f2a5f653700 1 --2- 172.21.2.101:0/913147355 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2a58193b90 0x7f2a581aa510 unknown :-1 s=CLOSED pgs=3171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:47:59.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.281 7f2a5f653700 1 --2- 172.21.2.101:0/913147355 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2a58096340 0x7f2a58095a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:47:59.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.281 7f2a5f653700 1 -- 172.21.2.101:0/913147355 >> 172.21.2.101:0/913147355 conn(0x7f2a5819f300 msgr2=0x7f2a58102360 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:47:59.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.281 7f2a5f653700 1 -- 172.21.2.101:0/913147355 shutdown_connections 2021-02-10T11:47:59.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:47:59.281 7f2a5f653700 1 -- 172.21.2.101:0/913147355 wait complete. 2021-02-10T11:47:59.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:59.285 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:47:59.285 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:47:59.620 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:47:59.621 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:47:59.621 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:47:59.621 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:47:59.621 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:47:59.973 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[178317]: fuse finished with error 0 and tester_r 0 2021-02-10T11:47:59.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:47:59.975 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:47:59.975 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:48:00.311 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:48:00.311 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:48:00.311 INFO:teuthology.orchestra.run:waiting for 900 2021-02-10T11:48:00.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:48:00.312 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:48:00.648 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth rm client.guest1 2021-02-10T11:48:01.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.061 7fa87238a700 1 Processor -- start 2021-02-10T11:48:01.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.061 7fa87238a700 1 -- start start 2021-02-10T11:48:01.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.062 7fa87238a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa86c0d1c70 0x7fa86c0f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:01.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.062 7fa87238a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86c0f0b10 con 0x7fa86c0d1870 2021-02-10T11:48:01.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.062 7fa87238a700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86c1b0be0 con 0x7fa86c0d0340 2021-02-10T11:48:01.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.062 7fa87238a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c0e5ed0 con 0x7fa86c0d1c70 2021-02-10T11:48:01.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.062 7fa86bfff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa86c0d1870 0x7fa86c1b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35028/0 (socket says 172.21.2.101:35028) 2021-02-10T11:48:01.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.062 7fa86bfff700 1 -- 172.21.2.101:0/2612725513 learned_addr learned my addr 172.21.2.101:0/2612725513 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:01.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.062 7fa86affd700 1 -- 172.21.2.101:0/2612725513 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 592588592 0 0) 0x7fa85c000ce0 con 0x7fa86c0d1870 2021-02-10T11:48:01.076 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa86affd700 1 -- 172.21.2.101:0/2612725513 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa854002fe0 con 0x7fa86c0d1870 2021-02-10T11:48:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa8709a7700 1 --2- 172.21.2.101:0/2612725513 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa86c0d1c70 0x7fa86c0f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa86affd700 1 -- 172.21.2.101:0/2612725513 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3995559195 0 0) 0x7fa85c0010e0 con 0x7fa86c0d1870 2021-02-10T11:48:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa86affd700 1 -- 172.21.2.101:0/2612725513 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa86c0d1c70 msgr2=0x7fa86c0f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa86affd700 1 --2- 172.21.2.101:0/2612725513 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa86c0d1c70 0x7fa86c0f20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa86affd700 1 -- 172.21.2.101:0/2612725513 >> v1:172.21.2.108:6789/0 conn(0x7fa86c0d0340 legacy=0x7fa86c0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa86affd700 1 -- 172.21.2.101:0/2612725513 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c0efe30 con 0x7fa86c0d1870 2021-02-10T11:48:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa86affd700 1 -- 172.21.2.101:0/2612725513 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fa85c001660 con 0x7fa86c0d1870 2021-02-10T11:48:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa86affd700 1 -- 172.21.2.101:0/2612725513 >> v1:172.21.2.101:6789/0 conn(0x7fa86c0d1870 legacy=0x7fa86c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa86affd700 1 --2- 172.21.2.101:0/2612725513 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa854003a60 0x7fa854005ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa86affd700 1 --2- 172.21.2.101:0/2612725513 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa854006770 0x7fa854008c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa86affd700 1 --2- 172.21.2.101:0/2612725513 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa854009500 0x7fa85400ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa86affd700 1 -- 172.21.2.101:0/2612725513 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85400c2d0 con 0x7fa854009500 2021-02-10T11:48:01.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa86affd700 1 -- 172.21.2.101:0/2612725513 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85400c4f0 con 0x7fa854003a60 2021-02-10T11:48:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa86affd700 1 -- 172.21.2.101:0/2612725513 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85400c710 con 0x7fa854006770 2021-02-10T11:48:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa8709a7700 1 --2- 172.21.2.101:0/2612725513 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa854009500 0x7fa85400ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa86b7fe700 1 --2- 172.21.2.101:0/2612725513 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa854006770 0x7fa854008c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa86bfff700 1 --2- 172.21.2.101:0/2612725513 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa854003a60 0x7fa854005ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa8709a7700 1 -- 172.21.2.101:0/2612725513 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa854006770 msgr2=0x7fa854008c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa8709a7700 1 --2- 172.21.2.101:0/2612725513 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa854006770 0x7fa854008c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa8709a7700 1 -- 172.21.2.101:0/2612725513 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa854003a60 msgr2=0x7fa854005ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa8709a7700 1 --2- 172.21.2.101:0/2612725513 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa854003a60 0x7fa854005ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa8709a7700 1 -- 172.21.2.101:0/2612725513 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85400c9b0 con 0x7fa854009500 2021-02-10T11:48:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.063 7fa8709a7700 1 --2- 172.21.2.101:0/2612725513 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa854009500 0x7fa85400ba20 secure :-1 s=READY pgs=3173 cs=0 l=1 rev1=1 rx=0x7fa86404ae60 tx=0x7fa86404de00).ready entity=mon.0 client_cookie=f8afea50e7c4ffa4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.064 7fa86affd700 1 -- 172.21.2.101:0/2612725513 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa86404aa30 con 0x7fa854009500 2021-02-10T11:48:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.064 7fa86affd700 1 -- 172.21.2.101:0/2612725513 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86404f300 con 0x7fa854009500 2021-02-10T11:48:01.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.064 7fa86affd700 1 -- 172.21.2.101:0/2612725513 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa86404f820 con 0x7fa854009500 2021-02-10T11:48:01.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.064 7fa87238a700 1 -- 172.21.2.101:0/2612725513 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa854009500 msgr2=0x7fa85400ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:01.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.064 7fa87238a700 1 --2- 172.21.2.101:0/2612725513 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa854009500 0x7fa85400ba20 secure :-1 s=READY pgs=3173 cs=0 l=1 rev1=1 rx=0x7fa86404ae60 tx=0x7fa86404de00).stop 2021-02-10T11:48:01.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.064 7fa8709a7700 1 -- 172.21.2.101:0/2612725513 reap_dead start 2021-02-10T11:48:01.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.064 7fa87238a700 1 -- 172.21.2.101:0/2612725513 shutdown_connections 2021-02-10T11:48:01.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.064 7fa87238a700 1 -- 172.21.2.101:0/2612725513 >> 172.21.2.101:0/2612725513 conn(0x7fa86c0ce450 msgr2=0x7fa86c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:01.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.064 7fa87238a700 1 -- 172.21.2.101:0/2612725513 shutdown_connections 2021-02-10T11:48:01.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.064 7fa87238a700 1 -- 172.21.2.101:0/2612725513 wait complete. 2021-02-10T11:48:01.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.079 7fa87238a700 1 Processor -- start 2021-02-10T11:48:01.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.079 7fa87238a700 1 -- start start 2021-02-10T11:48:01.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.080 7fa87238a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa86c1a7410 0x7fa86c1a7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.080 7fa87238a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa86c0ae040 0x7fa86c0c1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.080 7fa87238a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa86c1a7e30 0x7fa86c0c2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.080 7fa87238a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c1a8230 con 0x7fa86c0ae040 2021-02-10T11:48:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.080 7fa87238a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c0ae5e0 con 0x7fa86c1a7e30 2021-02-10T11:48:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.080 7fa87238a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c0c2880 con 0x7fa86c1a7410 2021-02-10T11:48:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.080 7fa86b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa86c0ae040 0x7fa86c0c1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.080 7fa86b7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa86c0ae040 0x7fa86c0c1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34546/0 (socket says 172.21.2.101:34546) 2021-02-10T11:48:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.080 7fa86b7fe700 1 -- 172.21.2.101:0/1649935781 learned_addr learned my addr 172.21.2.101:0/1649935781 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.080 7fa86b7fe700 1 -- 172.21.2.101:0/1649935781 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa86c1a7410 msgr2=0x7fa86c1a7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.080 7fa86b7fe700 1 --2- 172.21.2.101:0/1649935781 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa86c1a7410 0x7fa86c1a7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.080 7fa86b7fe700 1 -- 172.21.2.101:0/1649935781 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa86c1a7e30 msgr2=0x7fa86c0c2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.080 7fa86b7fe700 1 --2- 172.21.2.101:0/1649935781 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa86c1a7e30 0x7fa86c0c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:01.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.080 7fa86b7fe700 1 -- 172.21.2.101:0/1649935781 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c0c2aa0 con 0x7fa86c0ae040 2021-02-10T11:48:01.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.081 7fa86b7fe700 1 --2- 172.21.2.101:0/1649935781 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa86c0ae040 0x7fa86c0c1c40 secure :-1 s=READY pgs=3174 cs=0 l=1 rev1=1 rx=0x7fa86000b5e0 tx=0x7fa86000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:01.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.081 7fa868ff9700 1 -- 172.21.2.101:0/1649935781 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa86000bd70 con 0x7fa86c0ae040 2021-02-10T11:48:01.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.081 7fa868ff9700 1 -- 172.21.2.101:0/1649935781 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8600063c0 con 0x7fa86c0ae040 2021-02-10T11:48:01.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.081 7fa868ff9700 1 -- 172.21.2.101:0/1649935781 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa860005560 con 0x7fa86c0ae040 2021-02-10T11:48:01.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.081 7fa87238a700 1 -- 172.21.2.101:0/1649935781 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86c0c2aa0 con 0x7fa86c0ae040 2021-02-10T11:48:01.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.081 7fa87238a700 1 -- 172.21.2.101:0/1649935781 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86c0a7c20 con 0x7fa86c0ae040 2021-02-10T11:48:01.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.082 7fa868ff9700 1 -- 172.21.2.101:0/1649935781 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa860006030 con 0x7fa86c0ae040 2021-02-10T11:48:01.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.082 7fa868ff9700 1 --2- 172.21.2.101:0/1649935781 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa84c0186c0 0x7fa84c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:01.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.082 7fa868ff9700 1 -- 172.21.2.101:0/1649935781 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa860032590 con 0x7fa86c0ae040 2021-02-10T11:48:01.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.082 7fa86bfff700 1 --2- 172.21.2.101:0/1649935781 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa84c0186c0 0x7fa84c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:01.082 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.082 7fa86bfff700 1 --2- 172.21.2.101:0/1649935781 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa84c0186c0 0x7fa84c01abc0 crc :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:01.108 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.112 7fa87238a700 1 -- 172.21.2.101:0/1649935781 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86c0a7c20 con 0x7fa86c0ae040 2021-02-10T11:48:01.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.114 7fa868ff9700 1 -- 172.21.2.101:0/1649935781 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa860007250 con 0x7fa86c0ae040 2021-02-10T11:48:01.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.244 7fa87238a700 1 -- 172.21.2.101:0/1649935781 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth rm", "entity": "client.guest1"} v 0) v1 -- 0x7fa86c0a7c20 con 0x7fa86c0ae040 2021-02-10T11:48:01.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.244 7fa868ff9700 1 -- 172.21.2.101:0/1649935781 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth rm", "entity": "client.guest1"}]=0 entity client.guest1 does not exist v109) v1 ==== 117+0+0 (secure 0 0 0) 0x7fa860007250 con 0x7fa86c0ae040 2021-02-10T11:48:01.241 INFO:teuthology.orchestra.run.gibba001.stderr:entity client.guest1 does not exist 2021-02-10T11:48:01.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.246 7fa87238a700 1 -- 172.21.2.101:0/1649935781 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa84c0186c0 msgr2=0x7fa84c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:01.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.246 7fa87238a700 1 --2- 172.21.2.101:0/1649935781 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa84c0186c0 0x7fa84c01abc0 crc :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:01.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.246 7fa87238a700 1 -- 172.21.2.101:0/1649935781 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa86c0ae040 msgr2=0x7fa86c0c1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:01.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.246 7fa87238a700 1 --2- 172.21.2.101:0/1649935781 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa86c0ae040 0x7fa86c0c1c40 secure :-1 s=READY pgs=3174 cs=0 l=1 rev1=1 rx=0x7fa86000b5e0 tx=0x7fa86000b980).stop 2021-02-10T11:48:01.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.246 7fa87238a700 1 -- 172.21.2.101:0/1649935781 shutdown_connections 2021-02-10T11:48:01.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.246 7fa87238a700 1 --2- 172.21.2.101:0/1649935781 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa86c1a7e30 0x7fa86c0c2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:01.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.246 7fa87238a700 1 --2- 172.21.2.101:0/1649935781 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa86c0ae040 0x7fa86c0c1c40 unknown :-1 s=CLOSED pgs=3174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:01.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.246 7fa87238a700 1 --2- 172.21.2.101:0/1649935781 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa84c0186c0 0x7fa84c01abc0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:01.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.246 7fa87238a700 1 --2- 172.21.2.101:0/1649935781 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa86c1a7410 0x7fa86c1a7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:01.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.246 7fa87238a700 1 -- 172.21.2.101:0/1649935781 >> 172.21.2.101:0/1649935781 conn(0x7fa86c0aeb90 msgr2=0x7fa86c0af100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:01.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.246 7fa87238a700 1 -- 172.21.2.101:0/1649935781 shutdown_connections 2021-02-10T11:48:01.244 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.246 7fa87238a700 1 -- 172.21.2.101:0/1649935781 wait complete. 2021-02-10T11:48:01.252 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000359483 --group_name group_0000000000537794 2021-02-10T11:48:01.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.672 7f1de47d8700 1 Processor -- start 2021-02-10T11:48:01.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.672 7f1de47d8700 1 -- start start 2021-02-10T11:48:01.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.673 7f1de47d8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1ddc0d1c80 0x7f1ddc0cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:01.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.673 7f1de47d8700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ddc0d0180 con 0x7f1ddc0cf590 2021-02-10T11:48:01.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.673 7f1de47d8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ddc0fe1d0 con 0x7f1ddc0d1880 2021-02-10T11:48:01.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.673 7f1de47d8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc1b0cc0 con 0x7f1ddc0d1c80 2021-02-10T11:48:01.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.673 7f1de2df5700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f1ddc0cf590 0x7f1ddc0cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35052/0 (socket says 172.21.2.101:35052) 2021-02-10T11:48:01.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.673 7f1de2df5700 1 -- 172.21.2.101:0/2162853595 learned_addr learned my addr 172.21.2.101:0/2162853595 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:01.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.673 7f1de15f2700 1 -- 172.21.2.101:0/2162853595 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900948466 0 0) 0x7f1dd4001500 con 0x7f1ddc0cf590 2021-02-10T11:48:01.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.673 7f1de1df3700 1 --2- 172.21.2.101:0/2162853595 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1ddc0d1c80 0x7f1ddc0cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:01.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.673 7f1de15f2700 1 -- 172.21.2.101:0/2162853595 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dc8002fe0 con 0x7f1ddc0cf590 2021-02-10T11:48:01.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de15f2700 1 -- 172.21.2.101:0/2162853595 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3909721175 0 0) 0x7f1dd4001460 con 0x7f1ddc0cf590 2021-02-10T11:48:01.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de15f2700 1 -- 172.21.2.101:0/2162853595 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1ddc0d1c80 msgr2=0x7f1ddc0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:01.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de15f2700 1 --2- 172.21.2.101:0/2162853595 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1ddc0d1c80 0x7f1ddc0cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:01.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de15f2700 1 -- 172.21.2.101:0/2162853595 >> v1:172.21.2.108:6789/0 conn(0x7f1ddc0d1880 legacy=0x7f1ddc1b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:01.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de15f2700 1 -- 172.21.2.101:0/2162853595 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ddc109a70 con 0x7f1ddc0cf590 2021-02-10T11:48:01.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de15f2700 1 -- 172.21.2.101:0/2162853595 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f1dd4001980 con 0x7f1ddc0cf590 2021-02-10T11:48:01.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de15f2700 1 -- 172.21.2.101:0/2162853595 >> v1:172.21.2.101:6789/0 conn(0x7f1ddc0cf590 legacy=0x7f1ddc0cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:01.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de15f2700 1 --2- 172.21.2.101:0/2162853595 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1dc8003b10 0x7f1dc8005f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:01.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de15f2700 1 --2- 172.21.2.101:0/2162853595 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1dc80067a0 0x7f1dc8008c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:01.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de15f2700 1 --2- 172.21.2.101:0/2162853595 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dc8009500 0x7f1dc800ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:01.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de15f2700 1 -- 172.21.2.101:0/2162853595 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc800c2d0 con 0x7f1dc8009500 2021-02-10T11:48:01.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de15f2700 1 -- 172.21.2.101:0/2162853595 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc800c4f0 con 0x7f1dc8003b10 2021-02-10T11:48:01.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de15f2700 1 -- 172.21.2.101:0/2162853595 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc800c710 con 0x7f1dc80067a0 2021-02-10T11:48:01.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de2df5700 1 --2- 172.21.2.101:0/2162853595 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dc8009500 0x7f1dc800ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:01.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de1df3700 1 --2- 172.21.2.101:0/2162853595 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1dc80067a0 0x7f1dc8008c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:01.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de25f4700 1 --2- 172.21.2.101:0/2162853595 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1dc8003b10 0x7f1dc8005f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:01.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de2df5700 1 -- 172.21.2.101:0/2162853595 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1dc80067a0 msgr2=0x7f1dc8008c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:01.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de2df5700 1 --2- 172.21.2.101:0/2162853595 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1dc80067a0 0x7f1dc8008c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:01.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de2df5700 1 -- 172.21.2.101:0/2162853595 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1dc8003b10 msgr2=0x7f1dc8005f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:01.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de2df5700 1 --2- 172.21.2.101:0/2162853595 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1dc8003b10 0x7f1dc8005f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:01.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de2df5700 1 -- 172.21.2.101:0/2162853595 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc8050980 con 0x7f1dc8009500 2021-02-10T11:48:01.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de25f4700 1 --2- 172.21.2.101:0/2162853595 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1dc8003b10 0x7f1dc8005f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:48:01.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de2df5700 1 --2- 172.21.2.101:0/2162853595 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dc8009500 0x7f1dc800ba20 secure :-1 s=READY pgs=3176 cs=0 l=1 rev1=1 rx=0x7f1dd4001160 tx=0x7f1dd400a310).ready entity=mon.0 client_cookie=e5f97a0c23952980 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:01.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de15f2700 1 -- 172.21.2.101:0/2162853595 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1dd40070c0 con 0x7f1dc8009500 2021-02-10T11:48:01.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de15f2700 1 -- 172.21.2.101:0/2162853595 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd4007340 con 0x7f1dc8009500 2021-02-10T11:48:01.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de15f2700 1 -- 172.21.2.101:0/2162853595 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1dd40049a0 con 0x7f1dc8009500 2021-02-10T11:48:01.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de47d8700 1 -- 172.21.2.101:0/2162853595 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dc8009500 msgr2=0x7f1dc800ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:01.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de47d8700 1 --2- 172.21.2.101:0/2162853595 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1dc8009500 0x7f1dc800ba20 secure :-1 s=READY pgs=3176 cs=0 l=1 rev1=1 rx=0x7f1dd4001160 tx=0x7f1dd400a310).stop 2021-02-10T11:48:01.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.674 7f1de2df5700 1 -- 172.21.2.101:0/2162853595 reap_dead start 2021-02-10T11:48:01.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.675 7f1de47d8700 1 -- 172.21.2.101:0/2162853595 shutdown_connections 2021-02-10T11:48:01.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.675 7f1de47d8700 1 -- 172.21.2.101:0/2162853595 >> 172.21.2.101:0/2162853595 conn(0x7f1ddc0ce430 msgr2=0x7f1ddc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:01.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.675 7f1de47d8700 1 -- 172.21.2.101:0/2162853595 shutdown_connections 2021-02-10T11:48:01.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.675 7f1de47d8700 1 -- 172.21.2.101:0/2162853595 wait complete. 2021-02-10T11:48:01.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.689 7f1de47d8700 1 Processor -- start 2021-02-10T11:48:01.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.689 7f1de47d8700 1 -- start start 2021-02-10T11:48:01.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.690 7f1de47d8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1ddc188400 0x7f1ddc188800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:01.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.690 7f1de47d8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1ddc188e20 0x7f1ddc189220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:01.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.690 7f1de47d8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1ddc09cbc0 0x7f1ddc09cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:01.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.690 7f1de47d8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc187e80 con 0x7f1ddc09cbc0 2021-02-10T11:48:01.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.690 7f1de47d8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc0b27c0 con 0x7f1ddc188400 2021-02-10T11:48:01.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.690 7f1de47d8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc0b29a0 con 0x7f1ddc188e20 2021-02-10T11:48:01.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.690 7f1de2df5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1ddc09cbc0 0x7f1ddc09cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:01.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.690 7f1de2df5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1ddc09cbc0 0x7f1ddc09cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34570/0 (socket says 172.21.2.101:34570) 2021-02-10T11:48:01.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.690 7f1de2df5700 1 -- 172.21.2.101:0/523653031 learned_addr learned my addr 172.21.2.101:0/523653031 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:01.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.690 7f1de25f4700 1 --2- 172.21.2.101:0/523653031 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1ddc188400 0x7f1ddc188800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:01.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.691 7f1de1df3700 1 --2- 172.21.2.101:0/523653031 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1ddc188e20 0x7f1ddc189220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:01.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.691 7f1de2df5700 1 -- 172.21.2.101:0/523653031 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1ddc188e20 msgr2=0x7f1ddc189220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:01.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.691 7f1de2df5700 1 --2- 172.21.2.101:0/523653031 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1ddc188e20 0x7f1ddc189220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:01.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.691 7f1de2df5700 1 -- 172.21.2.101:0/523653031 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1ddc188400 msgr2=0x7f1ddc188800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:01.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.691 7f1de2df5700 1 --2- 172.21.2.101:0/523653031 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1ddc188400 0x7f1ddc188800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:01.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.691 7f1de2df5700 1 -- 172.21.2.101:0/523653031 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ddc09d5e0 con 0x7f1ddc09cbc0 2021-02-10T11:48:01.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.691 7f1de1df3700 1 --2- 172.21.2.101:0/523653031 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1ddc188e20 0x7f1ddc189220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:48:01.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.691 7f1de2df5700 1 --2- 172.21.2.101:0/523653031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1ddc09cbc0 0x7f1ddc09cfc0 secure :-1 s=READY pgs=3177 cs=0 l=1 rev1=1 rx=0x7f1dd400a920 tx=0x7f1dd400acc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:01.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.691 7f1dd2ffd700 1 -- 172.21.2.101:0/523653031 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1dd40070d0 con 0x7f1ddc09cbc0 2021-02-10T11:48:01.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.691 7f1dd2ffd700 1 -- 172.21.2.101:0/523653031 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd4000eb0 con 0x7f1ddc09cbc0 2021-02-10T11:48:01.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.691 7f1dd2ffd700 1 -- 172.21.2.101:0/523653031 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1dd4004930 con 0x7f1ddc09cbc0 2021-02-10T11:48:01.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.691 7f1de47d8700 1 -- 172.21.2.101:0/523653031 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ddc09d5e0 con 0x7f1ddc09cbc0 2021-02-10T11:48:01.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.691 7f1de47d8700 1 -- 172.21.2.101:0/523653031 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ddc09db70 con 0x7f1ddc09cbc0 2021-02-10T11:48:01.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.692 7f1dd2ffd700 1 -- 172.21.2.101:0/523653031 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1dd4000cc0 con 0x7f1ddc09cbc0 2021-02-10T11:48:01.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.692 7f1dd2ffd700 1 --2- 172.21.2.101:0/523653031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1dc40186c0 0x7f1dc401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:01.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.692 7f1dd2ffd700 1 -- 172.21.2.101:0/523653031 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1dd4032740 con 0x7f1ddc09cbc0 2021-02-10T11:48:01.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.692 7f1de25f4700 1 --2- 172.21.2.101:0/523653031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1dc40186c0 0x7f1dc401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:01.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.692 7f1de25f4700 1 --2- 172.21.2.101:0/523653031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1dc40186c0 0x7f1dc401abc0 crc :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:01.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.723 7f1de47d8700 1 -- 172.21.2.101:0/523653031 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ddc082a70 con 0x7f1ddc09cbc0 2021-02-10T11:48:01.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.725 7f1dd2ffd700 1 -- 172.21.2.101:0/523653031 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1dd4007350 con 0x7f1ddc09cbc0 2021-02-10T11:48:01.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.853 7f1de47d8700 1 -- 172.21.2.101:0/523653031 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000359483", "target": ["mgr", ""], "group_name": "group_0000000000537794"}) v1 -- 0x7f1ddc09db70 con 0x7f1dc40186c0 2021-02-10T11:48:01.859 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.863 7f1dd2ffd700 1 -- 172.21.2.101:0/523653031 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f1dd8006e50 con 0x7f1dc40186c0 2021-02-10T11:48:01.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.870 7f1de47d8700 1 -- 172.21.2.101:0/523653031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1dc40186c0 msgr2=0x7f1dc401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:01.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.870 7f1de47d8700 1 --2- 172.21.2.101:0/523653031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1dc40186c0 0x7f1dc401abc0 crc :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:01.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.870 7f1de47d8700 1 -- 172.21.2.101:0/523653031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1ddc09cbc0 msgr2=0x7f1ddc09cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:01.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.870 7f1de47d8700 1 --2- 172.21.2.101:0/523653031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1ddc09cbc0 0x7f1ddc09cfc0 secure :-1 s=READY pgs=3177 cs=0 l=1 rev1=1 rx=0x7f1dd400a920 tx=0x7f1dd400acc0).stop 2021-02-10T11:48:01.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.870 7f1de47d8700 1 -- 172.21.2.101:0/523653031 shutdown_connections 2021-02-10T11:48:01.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.870 7f1de47d8700 1 --2- 172.21.2.101:0/523653031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1dc40186c0 0x7f1dc401abc0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:01.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.870 7f1de47d8700 1 --2- 172.21.2.101:0/523653031 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1ddc188e20 0x7f1ddc189220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:01.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.870 7f1de47d8700 1 --2- 172.21.2.101:0/523653031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1ddc09cbc0 0x7f1ddc09cfc0 unknown :-1 s=CLOSED pgs=3177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:01.867 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.870 7f1de47d8700 1 --2- 172.21.2.101:0/523653031 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1ddc188400 0x7f1ddc188800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:01.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.870 7f1de47d8700 1 -- 172.21.2.101:0/523653031 >> 172.21.2.101:0/523653031 conn(0x7f1ddc0c2cd0 msgr2=0x7f1ddc0b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:01.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.870 7f1de47d8700 1 -- 172.21.2.101:0/523653031 shutdown_connections 2021-02-10T11:48:01.868 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:01.870 7f1de47d8700 1 -- 172.21.2.101:0/523653031 wait complete. 2021-02-10T11:48:01.875 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000359484 --group_name group_0000000000537794 2021-02-10T11:48:02.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.295 7fd2df367700 1 Processor -- start 2021-02-10T11:48:02.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.296 7fd2df367700 1 -- start start 2021-02-10T11:48:02.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.296 7fd2df367700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd2d80d1870 0x7fd2d81b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:02.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.296 7fd2df367700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd2d81b0ae0 0x7fd2d8108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:02.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.296 7fd2df367700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d80fe1c0 con 0x7fd2d80d1c70 2021-02-10T11:48:02.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.296 7fd2df367700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d8109a60 con 0x7fd2d80d1870 2021-02-10T11:48:02.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.296 7fd2df367700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d8109750 con 0x7fd2d81b0ae0 2021-02-10T11:48:02.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.296 7fd2dd183700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd2d80d1870 0x7fd2d81b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:02.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.296 7fd2dd984700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd2d80d1c70 0x7fd2d8108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35068/0 (socket says 172.21.2.101:35068) 2021-02-10T11:48:02.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.296 7fd2dd984700 1 -- 172.21.2.101:0/2181823058 learned_addr learned my addr 172.21.2.101:0/2181823058 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:02.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.296 7fd2dc982700 1 --2- 172.21.2.101:0/2181823058 >> v2:172.21.2.108:3300/0 conn(0x7fd2d81b0ae0 0x7fd2d8108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:02.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.297 7fd2dd183700 1 -- 172.21.2.101:0/2181823058 >> v2:172.21.2.108:3300/0 conn(0x7fd2d81b0ae0 msgr2=0x7fd2d8108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:02.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.297 7fd2dd183700 1 --2- 172.21.2.101:0/2181823058 >> v2:172.21.2.108:3300/0 conn(0x7fd2d81b0ae0 0x7fd2d8108940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:02.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.297 7fd2dd183700 1 -- 172.21.2.101:0/2181823058 >> v1:172.21.2.101:6789/0 conn(0x7fd2d80d1c70 legacy=0x7fd2d8108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:02.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.297 7fd2dd183700 1 -- 172.21.2.101:0/2181823058 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d8088070 con 0x7fd2d80d1870 2021-02-10T11:48:02.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.297 7fd2dc982700 1 --2- 172.21.2.101:0/2181823058 >> v2:172.21.2.108:3300/0 conn(0x7fd2d81b0ae0 0x7fd2d8108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:48:02.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.297 7fd2dd183700 1 --2- 172.21.2.101:0/2181823058 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2d80d1870 0x7fd2d81b04c0 secure :-1 s=READY pgs=3179 cs=0 l=1 rev1=1 rx=0x7fd2d4006650 tx=0x7fd2d4009a80).ready entity=mon.0 client_cookie=dc679d60bb809a4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:02.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.297 7fd2cffff700 1 -- 172.21.2.101:0/2181823058 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2d4006390 con 0x7fd2d80d1870 2021-02-10T11:48:02.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.297 7fd2cffff700 1 -- 172.21.2.101:0/2181823058 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d40009b0 con 0x7fd2d80d1870 2021-02-10T11:48:02.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.297 7fd2cffff700 1 -- 172.21.2.101:0/2181823058 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2d400c2b0 con 0x7fd2d80d1870 2021-02-10T11:48:02.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.297 7fd2df367700 1 -- 172.21.2.101:0/2181823058 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2d80d1870 msgr2=0x7fd2d81b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:02.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.297 7fd2df367700 1 --2- 172.21.2.101:0/2181823058 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2d80d1870 0x7fd2d81b04c0 secure :-1 s=READY pgs=3179 cs=0 l=1 rev1=1 rx=0x7fd2d4006650 tx=0x7fd2d4009a80).stop 2021-02-10T11:48:02.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.298 7fd2df367700 1 -- 172.21.2.101:0/2181823058 shutdown_connections 2021-02-10T11:48:02.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.298 7fd2df367700 1 --2- 172.21.2.101:0/2181823058 >> v2:172.21.2.108:3300/0 conn(0x7fd2d81b0ae0 0x7fd2d8108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:02.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.298 7fd2df367700 1 --2- 172.21.2.101:0/2181823058 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2d80d1870 0x7fd2d81b04c0 unknown :-1 s=CLOSED pgs=3179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:02.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.298 7fd2df367700 1 -- 172.21.2.101:0/2181823058 >> 172.21.2.101:0/2181823058 conn(0x7fd2d80ce420 msgr2=0x7fd2d80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:02.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.298 7fd2df367700 1 -- 172.21.2.101:0/2181823058 shutdown_connections 2021-02-10T11:48:02.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.298 7fd2df367700 1 -- 172.21.2.101:0/2181823058 wait complete. 2021-02-10T11:48:02.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.313 7fd2df367700 1 Processor -- start 2021-02-10T11:48:02.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.313 7fd2df367700 1 -- start start 2021-02-10T11:48:02.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.313 7fd2df367700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2d819e5c0 0x7fd2d8099fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:02.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.313 7fd2df367700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2d809a5d0 0x7fd2d809a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:02.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.313 7fd2df367700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2d80982a0 0x7fd2d80986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:02.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.313 7fd2df367700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d819feb0 con 0x7fd2d809a5d0 2021-02-10T11:48:02.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.313 7fd2df367700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d8099a30 con 0x7fd2d80982a0 2021-02-10T11:48:02.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.313 7fd2df367700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d819fb30 con 0x7fd2d819e5c0 2021-02-10T11:48:02.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.314 7fd2dc982700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2d809a5d0 0x7fd2d809a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:02.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.314 7fd2dc982700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2d809a5d0 0x7fd2d809a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34578/0 (socket says 172.21.2.101:34578) 2021-02-10T11:48:02.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.314 7fd2dc982700 1 -- 172.21.2.101:0/957405557 learned_addr learned my addr 172.21.2.101:0/957405557 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:02.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.314 7fd2dd183700 1 --2- 172.21.2.101:0/957405557 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2d819e5c0 0x7fd2d8099fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:02.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.314 7fd2dd984700 1 --2- 172.21.2.101:0/957405557 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2d80982a0 0x7fd2d80986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:02.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.314 7fd2dc982700 1 -- 172.21.2.101:0/957405557 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2d819e5c0 msgr2=0x7fd2d8099fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:02.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.314 7fd2dc982700 1 --2- 172.21.2.101:0/957405557 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2d819e5c0 0x7fd2d8099fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:02.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.314 7fd2dc982700 1 -- 172.21.2.101:0/957405557 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2d80982a0 msgr2=0x7fd2d80986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:02.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.314 7fd2dc982700 1 --2- 172.21.2.101:0/957405557 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2d80982a0 0x7fd2d80986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:02.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.314 7fd2dc982700 1 -- 172.21.2.101:0/957405557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d819e9c0 con 0x7fd2d809a5d0 2021-02-10T11:48:02.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.314 7fd2dd984700 1 --2- 172.21.2.101:0/957405557 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2d80982a0 0x7fd2d80986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:48:02.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.314 7fd2dc982700 1 --2- 172.21.2.101:0/957405557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2d809a5d0 0x7fd2d809a9d0 secure :-1 s=READY pgs=3180 cs=0 l=1 rev1=1 rx=0x7fd2c800b580 tx=0x7fd2c800b920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:02.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.314 7fd2cdffb700 1 -- 172.21.2.101:0/957405557 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2c800bd10 con 0x7fd2d809a5d0 2021-02-10T11:48:02.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.314 7fd2cdffb700 1 -- 172.21.2.101:0/957405557 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c8002030 con 0x7fd2d809a5d0 2021-02-10T11:48:02.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.314 7fd2cdffb700 1 -- 172.21.2.101:0/957405557 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2c8002490 con 0x7fd2d809a5d0 2021-02-10T11:48:02.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.314 7fd2df367700 1 -- 172.21.2.101:0/957405557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d819e9c0 con 0x7fd2d809a5d0 2021-02-10T11:48:02.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.314 7fd2df367700 1 -- 172.21.2.101:0/957405557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d819e9c0 con 0x7fd2d809a5d0 2021-02-10T11:48:02.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.315 7fd2cdffb700 1 -- 172.21.2.101:0/957405557 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd2c8002210 con 0x7fd2d809a5d0 2021-02-10T11:48:02.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.315 7fd2cdffb700 1 --2- 172.21.2.101:0/957405557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2b8018730 0x7fd2b801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:02.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.315 7fd2cdffb700 1 -- 172.21.2.101:0/957405557 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd2c8032800 con 0x7fd2d809a5d0 2021-02-10T11:48:02.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.315 7fd2dd183700 1 --2- 172.21.2.101:0/957405557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2b8018730 0x7fd2b801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:02.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.315 7fd2dd183700 1 --2- 172.21.2.101:0/957405557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2b8018730 0x7fd2b801abc0 crc :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:02.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.346 7fd2df367700 1 -- 172.21.2.101:0/957405557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d80825d0 con 0x7fd2d809a5d0 2021-02-10T11:48:02.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.348 7fd2cdffb700 1 -- 172.21.2.101:0/957405557 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd2c8000c90 con 0x7fd2d809a5d0 2021-02-10T11:48:02.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.476 7fd2df367700 1 -- 172.21.2.101:0/957405557 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000359484", "target": ["mgr", ""], "group_name": "group_0000000000537794"}) v1 -- 0x7fd2d8099a30 con 0x7fd2b8018730 2021-02-10T11:48:02.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.485 7fd2cdffb700 1 -- 172.21.2.101:0/957405557 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fd2d40009b0 con 0x7fd2b8018730 2021-02-10T11:48:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.493 7fd2df367700 1 -- 172.21.2.101:0/957405557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2b8018730 msgr2=0x7fd2b801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.493 7fd2df367700 1 --2- 172.21.2.101:0/957405557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2b8018730 0x7fd2b801abc0 crc :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.493 7fd2df367700 1 -- 172.21.2.101:0/957405557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2d809a5d0 msgr2=0x7fd2d809a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.493 7fd2df367700 1 --2- 172.21.2.101:0/957405557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2d809a5d0 0x7fd2d809a9d0 secure :-1 s=READY pgs=3180 cs=0 l=1 rev1=1 rx=0x7fd2c800b580 tx=0x7fd2c800b920).stop 2021-02-10T11:48:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.493 7fd2df367700 1 -- 172.21.2.101:0/957405557 shutdown_connections 2021-02-10T11:48:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.493 7fd2df367700 1 --2- 172.21.2.101:0/957405557 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2d80982a0 0x7fd2d80986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.493 7fd2df367700 1 --2- 172.21.2.101:0/957405557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2d809a5d0 0x7fd2d809a9d0 unknown :-1 s=CLOSED pgs=3180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.493 7fd2df367700 1 --2- 172.21.2.101:0/957405557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd2b8018730 0x7fd2b801abc0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:02.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.493 7fd2df367700 1 --2- 172.21.2.101:0/957405557 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2d819e5c0 0x7fd2d8099fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:02.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.493 7fd2df367700 1 -- 172.21.2.101:0/957405557 >> 172.21.2.101:0/957405557 conn(0x7fd2d81b3560 msgr2=0x7fd2d81b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:02.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.493 7fd2df367700 1 -- 172.21.2.101:0/957405557 shutdown_connections 2021-02-10T11:48:02.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.493 7fd2df367700 1 -- 172.21.2.101:0/957405557 wait complete. 2021-02-10T11:48:02.497 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolumegroup rm cephfs group_0000000000537794 2021-02-10T11:48:02.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.915 7f74709db700 1 Processor -- start 2021-02-10T11:48:02.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.915 7f74709db700 1 -- start start 2021-02-10T11:48:02.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.916 7f74709db700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f746c0d1870 0x7f746c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:02.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.916 7f74709db700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f746c0cff70 0x7f746c109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:02.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.916 7f74709db700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f746c0fe1c0 con 0x7f746c1b0ae0 2021-02-10T11:48:02.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.916 7f74709db700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f746c109ad0 con 0x7f746c0d1870 2021-02-10T11:48:02.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.916 7f74709db700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f746c109750 con 0x7f746c0cff70 2021-02-10T11:48:02.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.916 7f7469e1b700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f746c1b0ae0 0x7f746c108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35080/0 (socket says 172.21.2.101:35080) 2021-02-10T11:48:02.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.916 7f746a61c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f746c0d1870 0x7f746c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:02.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.916 7f7469e1b700 1 -- 172.21.2.101:0/3641815392 learned_addr learned my addr 172.21.2.101:0/3641815392 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:02.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.916 7f746ae1d700 1 --2- 172.21.2.101:0/3641815392 >> v2:172.21.2.108:3300/0 conn(0x7f746c0cff70 0x7f746c109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:02.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.916 7f746961a700 1 -- 172.21.2.101:0/3641815392 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3004436860 0 0) 0x7f74540010b0 con 0x7f746c1b0ae0 2021-02-10T11:48:02.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.916 7f746961a700 1 -- 172.21.2.101:0/3641815392 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7458002fe0 con 0x7f746c1b0ae0 2021-02-10T11:48:02.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.916 7f746ae1d700 1 -- 172.21.2.101:0/3641815392 >> v2:172.21.2.101:3300/0 conn(0x7f746c0d1870 msgr2=0x7f746c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:02.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.916 7f746ae1d700 1 --2- 172.21.2.101:0/3641815392 >> v2:172.21.2.101:3300/0 conn(0x7f746c0d1870 0x7f746c1b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:02.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.916 7f746ae1d700 1 -- 172.21.2.101:0/3641815392 >> v1:172.21.2.101:6789/0 conn(0x7f746c1b0ae0 legacy=0x7f746c108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:02.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.917 7f746ae1d700 1 -- 172.21.2.101:0/3641815392 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746c088070 con 0x7f746c0cff70 2021-02-10T11:48:02.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.917 7f746ae1d700 1 --2- 172.21.2.101:0/3641815392 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f746c0cff70 0x7f746c109130 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7f7464006e50 tx=0x7f7464009e70).ready entity=mon.1 client_cookie=6eb423ce2e663bc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:02.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.917 7f746961a700 1 -- 172.21.2.101:0/3641815392 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f746400b510 con 0x7f746c0cff70 2021-02-10T11:48:02.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.917 7f746961a700 1 -- 172.21.2.101:0/3641815392 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7464006790 con 0x7f746c0cff70 2021-02-10T11:48:02.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.917 7f746961a700 1 -- 172.21.2.101:0/3641815392 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f746400b940 con 0x7f746c0cff70 2021-02-10T11:48:02.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.917 7f74709db700 1 -- 172.21.2.101:0/3641815392 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f746c0cff70 msgr2=0x7f746c109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:02.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.917 7f74709db700 1 --2- 172.21.2.101:0/3641815392 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f746c0cff70 0x7f746c109130 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7f7464006e50 tx=0x7f7464009e70).stop 2021-02-10T11:48:02.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.918 7f74709db700 1 -- 172.21.2.101:0/3641815392 shutdown_connections 2021-02-10T11:48:02.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.918 7f74709db700 1 --2- 172.21.2.101:0/3641815392 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f746c0cff70 0x7f746c109130 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:02.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.918 7f74709db700 1 --2- 172.21.2.101:0/3641815392 >> v2:172.21.2.101:3300/0 conn(0x7f746c0d1870 0x7f746c1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:02.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.918 7f74709db700 1 -- 172.21.2.101:0/3641815392 >> 172.21.2.101:0/3641815392 conn(0x7f746c0ce420 msgr2=0x7f746c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:02.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.918 7f74709db700 1 -- 172.21.2.101:0/3641815392 shutdown_connections 2021-02-10T11:48:02.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.918 7f74709db700 1 -- 172.21.2.101:0/3641815392 wait complete. 2021-02-10T11:48:02.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.932 7f74709db700 1 Processor -- start 2021-02-10T11:48:02.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.932 7f74709db700 1 -- start start 2021-02-10T11:48:02.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.933 7f74709db700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f746c0a7950 0x7f746c0b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:02.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.933 7f74709db700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f746c1a7490 0x7f746c1a7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:02.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.933 7f74709db700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f746c1a7eb0 0x7f746c0ac630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:02.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.933 7f74709db700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c0a7fb0 con 0x7f746c0a7950 2021-02-10T11:48:02.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.933 7f74709db700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c0acc50 con 0x7f746c1a7eb0 2021-02-10T11:48:02.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.933 7f74709db700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746c0ace30 con 0x7f746c1a7490 2021-02-10T11:48:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.933 7f746a61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f746c0a7950 0x7f746c0b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.933 7f746a61c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f746c0a7950 0x7f746c0b8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34590/0 (socket says 172.21.2.101:34590) 2021-02-10T11:48:02.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.933 7f746a61c700 1 -- 172.21.2.101:0/3440588048 learned_addr learned my addr 172.21.2.101:0/3440588048 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.933 7f746a61c700 1 -- 172.21.2.101:0/3440588048 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f746c1a7490 msgr2=0x7f746c1a7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.933 7f746a61c700 1 --2- 172.21.2.101:0/3440588048 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f746c1a7490 0x7f746c1a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:02.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.933 7f746a61c700 1 -- 172.21.2.101:0/3440588048 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f746c1a7eb0 msgr2=0x7f746c0ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.933 7f746a61c700 1 --2- 172.21.2.101:0/3440588048 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f746c1a7eb0 0x7f746c0ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.933 7f746a61c700 1 -- 172.21.2.101:0/3440588048 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746c0ad050 con 0x7f746c0a7950 2021-02-10T11:48:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.933 7f746a61c700 1 --2- 172.21.2.101:0/3440588048 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f746c0a7950 0x7f746c0b8df0 secure :-1 s=READY pgs=3182 cs=0 l=1 rev1=1 rx=0x7f745c006f60 tx=0x7f745c00b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.934 7f7462ffd700 1 -- 172.21.2.101:0/3440588048 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f745c0009b0 con 0x7f746c0a7950 2021-02-10T11:48:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.934 7f7462ffd700 1 -- 172.21.2.101:0/3440588048 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745c00bdf0 con 0x7f746c0a7950 2021-02-10T11:48:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.934 7f7462ffd700 1 -- 172.21.2.101:0/3440588048 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f745c002030 con 0x7f746c0a7950 2021-02-10T11:48:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.934 7f74709db700 1 -- 172.21.2.101:0/3440588048 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746c0ad050 con 0x7f746c0a7950 2021-02-10T11:48:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.934 7f74709db700 1 -- 172.21.2.101:0/3440588048 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746c0a8530 con 0x7f746c0a7950 2021-02-10T11:48:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.934 7f7462ffd700 1 -- 172.21.2.101:0/3440588048 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f745c000c90 con 0x7f746c0a7950 2021-02-10T11:48:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.934 7f7462ffd700 1 --2- 172.21.2.101:0/3440588048 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7450018730 0x7f745001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.934 7f7462ffd700 1 -- 172.21.2.101:0/3440588048 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f745c032820 con 0x7f746c0a7950 2021-02-10T11:48:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.934 7f7469e1b700 1 --2- 172.21.2.101:0/3440588048 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7450018730 0x7f745001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:02.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.935 7f7469e1b700 1 --2- 172.21.2.101:0/3440588048 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7450018730 0x7f745001abc0 crc :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:02.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.965 7f74709db700 1 -- 172.21.2.101:0/3440588048 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f746c082a70 con 0x7f746c0a7950 2021-02-10T11:48:02.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:02.968 7f7462ffd700 1 -- 172.21.2.101:0/3440588048 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f745c0009b0 con 0x7f746c0a7950 2021-02-10T11:48:03.092 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:03.095 7f74709db700 1 -- 172.21.2.101:0/3440588048 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "target": ["mgr", ""], "group_name": "group_0000000000537794"}) v1 -- 0x7f746c0a7fb0 con 0x7f7450018730 2021-02-10T11:48:03.096 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:03.100 7f7462ffd700 1 -- 172.21.2.101:0/3440588048 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f7454001590 con 0x7f7450018730 2021-02-10T11:48:03.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:03.104 7f74709db700 1 -- 172.21.2.101:0/3440588048 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7450018730 msgr2=0x7f745001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:03.101 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:03.104 7f74709db700 1 --2- 172.21.2.101:0/3440588048 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7450018730 0x7f745001abc0 crc :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:03.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:03.104 7f74709db700 1 -- 172.21.2.101:0/3440588048 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f746c0a7950 msgr2=0x7f746c0b8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:03.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:03.105 7f74709db700 1 --2- 172.21.2.101:0/3440588048 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f746c0a7950 0x7f746c0b8df0 secure :-1 s=READY pgs=3182 cs=0 l=1 rev1=1 rx=0x7f745c006f60 tx=0x7f745c00b970).stop 2021-02-10T11:48:03.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:03.105 7f74709db700 1 -- 172.21.2.101:0/3440588048 shutdown_connections 2021-02-10T11:48:03.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:03.105 7f74709db700 1 --2- 172.21.2.101:0/3440588048 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7450018730 0x7f745001abc0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:03.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:03.105 7f74709db700 1 --2- 172.21.2.101:0/3440588048 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f746c1a7490 0x7f746c1a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:03.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:03.105 7f74709db700 1 --2- 172.21.2.101:0/3440588048 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f746c1a7eb0 0x7f746c0ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:03.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:03.105 7f74709db700 1 --2- 172.21.2.101:0/3440588048 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f746c0a7950 0x7f746c0b8df0 unknown :-1 s=CLOSED pgs=3182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:03.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:03.105 7f74709db700 1 -- 172.21.2.101:0/3440588048 >> 172.21.2.101:0/3440588048 conn(0x7f746c0a02d0 msgr2=0x7f746c09f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:03.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:03.105 7f74709db700 1 -- 172.21.2.101:0/3440588048 shutdown_connections 2021-02-10T11:48:03.102 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:03.105 7f74709db700 1 -- 172.21.2.101:0/3440588048 wait complete. 2021-02-10T11:48:03.110 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:48:03.110 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:48:03.492 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:48:03.836 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:48:03.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:03.837 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:48:03.837 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:48:04.171 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:48:04.172 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:48:04.172 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:48:04.172 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:48:04.172 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:48:04.527 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[177677]: fuse finished with error 0 and tester_r 0 2021-02-10T11:48:04.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:04.528 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:48:04.528 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:48:04.865 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:48:04.866 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:48:04.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:48:04.866 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:48:05.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:05.202 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:48:05.202 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:48:05.536 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:48:05.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:48:05.537 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:48:05.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:48:05.537 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:48:05.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:05.873 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:48:05.873 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:48:06.209 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:48:06.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:48:06.209 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:48:06.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:48:06.210 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:48:06.544 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:06.544 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:48:06.544 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:48:06.877 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:48:06.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:48:06.878 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:48:06.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:48:06.878 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:48:07.213 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-02-10T11:48:07.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.634 7f4744e2f700 1 Processor -- start 2021-02-10T11:48:07.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.634 7f4744e2f700 1 -- start start 2021-02-10T11:48:07.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.635 7f4744e2f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47400d1870 0x7f47401b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:07.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.635 7f4744e2f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f47401b0a10 0x7f47400cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:07.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.635 7f4744e2f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47400f0b10 con 0x7f47400d1c70 2021-02-10T11:48:07.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.635 7f4744e2f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47400fe230 con 0x7f47401b0a10 2021-02-10T11:48:07.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.635 7f4744e2f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47400e5ed0 con 0x7f47400d1870 2021-02-10T11:48:07.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.635 7f473e312700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f47401b0a10 0x7f47400cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:07.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.635 7f473e312700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f47401b0a10 0x7f47400cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34600/0 (socket says 172.21.2.101:34600) 2021-02-10T11:48:07.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.635 7f473e312700 1 -- 172.21.2.101:0/2525980122 learned_addr learned my addr 172.21.2.101:0/2525980122 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:07.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.635 7f473e312700 1 -- 172.21.2.101:0/2525980122 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47400d1870 msgr2=0x7f47401b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:07.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.635 7f473e312700 1 --2- 172.21.2.101:0/2525980122 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47400d1870 0x7f47401b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:07.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.635 7f473e312700 1 -- 172.21.2.101:0/2525980122 >> v1:172.21.2.108:6789/0 conn(0x7f47400d1c70 legacy=0x7f47400cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:07.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.635 7f473e312700 1 -- 172.21.2.101:0/2525980122 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47400f0650 con 0x7f47401b0a10 2021-02-10T11:48:07.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.635 7f473e312700 1 --2- 172.21.2.101:0/2525980122 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47401b0a10 0x7f47400cf330 secure :-1 s=READY pgs=3183 cs=0 l=1 rev1=1 rx=0x7f4730006700 tx=0x7f4730009b80).ready entity=mon.0 client_cookie=222a1a28eb8eef8c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:07.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.636 7f473db11700 1 -- 172.21.2.101:0/2525980122 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f473000c0c0 con 0x7f47401b0a10 2021-02-10T11:48:07.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.636 7f473db11700 1 -- 172.21.2.101:0/2525980122 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473000c440 con 0x7f47401b0a10 2021-02-10T11:48:07.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.636 7f473db11700 1 -- 172.21.2.101:0/2525980122 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f473000c960 con 0x7f47401b0a10 2021-02-10T11:48:07.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.636 7f4744e2f700 1 -- 172.21.2.101:0/2525980122 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47401b0a10 msgr2=0x7f47400cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:07.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.636 7f4744e2f700 1 --2- 172.21.2.101:0/2525980122 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47401b0a10 0x7f47400cf330 secure :-1 s=READY pgs=3183 cs=0 l=1 rev1=1 rx=0x7f4730006700 tx=0x7f4730009b80).stop 2021-02-10T11:48:07.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.636 7f4744e2f700 1 -- 172.21.2.101:0/2525980122 shutdown_connections 2021-02-10T11:48:07.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.636 7f4744e2f700 1 --2- 172.21.2.101:0/2525980122 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47401b0a10 0x7f47400cf330 unknown :-1 s=CLOSED pgs=3183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:07.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.636 7f4744e2f700 1 --2- 172.21.2.101:0/2525980122 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47400d1870 0x7f47401b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:07.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.636 7f4744e2f700 1 -- 172.21.2.101:0/2525980122 >> 172.21.2.101:0/2525980122 conn(0x7f47400ce450 msgr2=0x7f47400d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:07.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.636 7f4744e2f700 1 -- 172.21.2.101:0/2525980122 shutdown_connections 2021-02-10T11:48:07.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.636 7f4744e2f700 1 -- 172.21.2.101:0/2525980122 wait complete. 2021-02-10T11:48:07.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.651 7f4744e2f700 1 Processor -- start 2021-02-10T11:48:07.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.651 7f4744e2f700 1 -- start start 2021-02-10T11:48:07.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.651 7f4744e2f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47401908b0 0x7f47401a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:07.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.651 7f4744e2f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47400bd920 0x7f47400bdd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:07.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.651 7f4744e2f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f474008d510 0x7f474008d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:07.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.651 7f4744e2f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474008cf90 con 0x7f47401908b0 2021-02-10T11:48:07.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.651 7f4744e2f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47400be340 con 0x7f474008d510 2021-02-10T11:48:07.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.651 7f4744e2f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47400be5f0 con 0x7f47400bd920 2021-02-10T11:48:07.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.652 7f473eb13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47401908b0 0x7f47401a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:07.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.652 7f473eb13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47401908b0 0x7f47401a4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34602/0 (socket says 172.21.2.101:34602) 2021-02-10T11:48:07.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.652 7f473eb13700 1 -- 172.21.2.101:0/1305626057 learned_addr learned my addr 172.21.2.101:0/1305626057 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:07.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.652 7f473eb13700 1 -- 172.21.2.101:0/1305626057 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47400bd920 msgr2=0x7f47400bdd20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:07.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.652 7f473eb13700 1 --2- 172.21.2.101:0/1305626057 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47400bd920 0x7f47400bdd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:07.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.652 7f473eb13700 1 -- 172.21.2.101:0/1305626057 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f474008d510 msgr2=0x7f474008d910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:07.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.652 7f473eb13700 1 --2- 172.21.2.101:0/1305626057 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f474008d510 0x7f474008d910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:07.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.652 7f473eb13700 1 -- 172.21.2.101:0/1305626057 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47401a4a60 con 0x7f47401908b0 2021-02-10T11:48:07.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.652 7f473eb13700 1 --2- 172.21.2.101:0/1305626057 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47401908b0 0x7f47401a4440 secure :-1 s=READY pgs=3184 cs=0 l=1 rev1=1 rx=0x7f4728006f00 tx=0x7f472800b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:07.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.652 7f47377fe700 1 -- 172.21.2.101:0/1305626057 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4728006350 con 0x7f47401908b0 2021-02-10T11:48:07.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.652 7f47377fe700 1 -- 172.21.2.101:0/1305626057 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4728006b90 con 0x7f47401908b0 2021-02-10T11:48:07.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.652 7f4744e2f700 1 -- 172.21.2.101:0/1305626057 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47401a4a60 con 0x7f47401908b0 2021-02-10T11:48:07.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.653 7f4744e2f700 1 -- 172.21.2.101:0/1305626057 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47401a4a60 con 0x7f47401908b0 2021-02-10T11:48:07.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.653 7f47377fe700 1 -- 172.21.2.101:0/1305626057 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4728000c90 con 0x7f47401908b0 2021-02-10T11:48:07.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.653 7f47377fe700 1 -- 172.21.2.101:0/1305626057 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4728002240 con 0x7f47401908b0 2021-02-10T11:48:07.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.653 7f47377fe700 1 --2- 172.21.2.101:0/1305626057 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4724018730 0x7f472401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:07.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.653 7f47377fe700 1 -- 172.21.2.101:0/1305626057 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4728032780 con 0x7f47401908b0 2021-02-10T11:48:07.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.653 7f473e312700 1 --2- 172.21.2.101:0/1305626057 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4724018730 0x7f472401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:07.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.654 7f473e312700 1 --2- 172.21.2.101:0/1305626057 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4724018730 0x7f472401ac10 crc :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:07.681 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.684 7f4744e2f700 1 -- 172.21.2.101:0/1305626057 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47400ee060 con 0x7f47401908b0 2021-02-10T11:48:07.683 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.687 7f47377fe700 1 -- 172.21.2.101:0/1305626057 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4728002010 con 0x7f47401908b0 2021-02-10T11:48:07.833 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.837 7f4744e2f700 1 -- 172.21.2.101:0/1305626057 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f47400ce450 con 0x7f47401908b0 2021-02-10T11:48:07.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.840 7f47377fe700 1 -- 172.21.2.101:0/1305626057 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v45) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4728006b90 con 0x7f47401908b0 2021-02-10T11:48:07.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.845 7f4744e2f700 1 -- 172.21.2.101:0/1305626057 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4724018730 msgr2=0x7f472401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:07.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.845 7f4744e2f700 1 --2- 172.21.2.101:0/1305626057 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4724018730 0x7f472401ac10 crc :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:07.842 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.845 7f4744e2f700 1 -- 172.21.2.101:0/1305626057 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47401908b0 msgr2=0x7f47401a4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:07.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.845 7f4744e2f700 1 --2- 172.21.2.101:0/1305626057 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47401908b0 0x7f47401a4440 secure :-1 s=READY pgs=3184 cs=0 l=1 rev1=1 rx=0x7f4728006f00 tx=0x7f472800b940).stop 2021-02-10T11:48:07.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.846 7f4744e2f700 1 -- 172.21.2.101:0/1305626057 shutdown_connections 2021-02-10T11:48:07.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.846 7f4744e2f700 1 --2- 172.21.2.101:0/1305626057 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4724018730 0x7f472401ac10 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:07.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.846 7f4744e2f700 1 --2- 172.21.2.101:0/1305626057 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47400bd920 0x7f47400bdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:07.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.846 7f4744e2f700 1 --2- 172.21.2.101:0/1305626057 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f474008d510 0x7f474008d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:07.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.846 7f4744e2f700 1 --2- 172.21.2.101:0/1305626057 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47401908b0 0x7f47401a4440 unknown :-1 s=CLOSED pgs=3184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:07.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.846 7f4744e2f700 1 -- 172.21.2.101:0/1305626057 >> 172.21.2.101:0/1305626057 conn(0x7f474008f030 msgr2=0x7f47400b5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:07.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.846 7f4744e2f700 1 -- 172.21.2.101:0/1305626057 shutdown_connections 2021-02-10T11:48:07.843 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:07.846 7f4744e2f700 1 -- 172.21.2.101:0/1305626057 wait complete. 2021-02-10T11:48:07.851 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_recover_auth_metadata_during_deauthorize' 2021-02-10T11:48:08.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.266 7f09fbab2700 1 Processor -- start 2021-02-10T11:48:08.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.266 7f09fbab2700 1 -- start start 2021-02-10T11:48:08.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.266 7f09fbab2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09f40d1c70 0x7f09f40f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:08.280 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.266 7f09fbab2700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f40f0b10 con 0x7f09f40d0340 2021-02-10T11:48:08.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.266 7f09fbab2700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f41b0be0 con 0x7f09f40d1870 2021-02-10T11:48:08.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.266 7f09fbab2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f40e5ed0 con 0x7f09f40d1c70 2021-02-10T11:48:08.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.267 7f09f90cd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f09f40d0340 0x7f09f40ef640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35104/0 (socket says 172.21.2.101:35104) 2021-02-10T11:48:08.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.267 7f09f90cd700 1 -- 172.21.2.101:0/1872101387 learned_addr learned my addr 172.21.2.101:0/1872101387 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:08.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.267 7f09f88cc700 1 -- 172.21.2.101:0/1872101387 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3772847287 0 0) 0x7f09f00010b0 con 0x7f09f40d0340 2021-02-10T11:48:08.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.267 7f09f88cc700 1 -- 172.21.2.101:0/1872101387 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09e0002fe0 con 0x7f09f40d0340 2021-02-10T11:48:08.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.267 7f09fa0cf700 1 --2- 172.21.2.101:0/1872101387 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09f40d1c70 0x7f09f40f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:08.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.267 7f09f88cc700 1 -- 172.21.2.101:0/1872101387 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 365060907 0 0) 0x7f09f00010b0 con 0x7f09f40d0340 2021-02-10T11:48:08.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.267 7f09f88cc700 1 -- 172.21.2.101:0/1872101387 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09f40d1c70 msgr2=0x7f09f40f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:08.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.267 7f09f88cc700 1 --2- 172.21.2.101:0/1872101387 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09f40d1c70 0x7f09f40f20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:08.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.267 7f09f88cc700 1 -- 172.21.2.101:0/1872101387 >> v1:172.21.2.108:6789/0 conn(0x7f09f40d1870 legacy=0x7f09f41b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:08.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.267 7f09f88cc700 1 -- 172.21.2.101:0/1872101387 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f40efe30 con 0x7f09f40d0340 2021-02-10T11:48:08.281 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.267 7f09f88cc700 1 -- 172.21.2.101:0/1872101387 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f09f00015b0 con 0x7f09f40d0340 2021-02-10T11:48:08.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.267 7f09f88cc700 1 -- 172.21.2.101:0/1872101387 >> v1:172.21.2.101:6789/0 conn(0x7f09f40d0340 legacy=0x7f09f40ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:08.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.267 7f09f88cc700 1 --2- 172.21.2.101:0/1872101387 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09e0003aa0 0x7f09e0005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:08.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.267 7f09f88cc700 1 --2- 172.21.2.101:0/1872101387 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09e0006800 0x7f09e0008cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:08.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.267 7f09f88cc700 1 --2- 172.21.2.101:0/1872101387 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09e0009590 0x7f09e000bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:08.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.267 7f09f88cc700 1 -- 172.21.2.101:0/1872101387 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e000c2e0 con 0x7f09e0003aa0 2021-02-10T11:48:08.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.267 7f09f88cc700 1 -- 172.21.2.101:0/1872101387 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e000c500 con 0x7f09e0009590 2021-02-10T11:48:08.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.268 7f09f88cc700 1 -- 172.21.2.101:0/1872101387 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e0050730 con 0x7f09e0006800 2021-02-10T11:48:08.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.268 7f09f98ce700 1 --2- 172.21.2.101:0/1872101387 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09e0003aa0 0x7f09e0005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:08.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.268 7f09fa0cf700 1 --2- 172.21.2.101:0/1872101387 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09e0009590 0x7f09e000bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:08.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.268 7f09f90cd700 1 --2- 172.21.2.101:0/1872101387 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09e0006800 0x7f09e0008cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:08.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.268 7f09fa0cf700 1 -- 172.21.2.101:0/1872101387 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09e0006800 msgr2=0x7f09e0008cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:08.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.268 7f09fa0cf700 1 --2- 172.21.2.101:0/1872101387 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09e0006800 0x7f09e0008cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:08.282 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.268 7f09fa0cf700 1 -- 172.21.2.101:0/1872101387 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09e0003aa0 msgr2=0x7f09e0005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:08.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.268 7f09fa0cf700 1 --2- 172.21.2.101:0/1872101387 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09e0003aa0 0x7f09e0005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:08.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.268 7f09fa0cf700 1 -- 172.21.2.101:0/1872101387 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e0050990 con 0x7f09e0009590 2021-02-10T11:48:08.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.268 7f09f98ce700 1 --2- 172.21.2.101:0/1872101387 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09e0003aa0 0x7f09e0005f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:48:08.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.268 7f09fa0cf700 1 --2- 172.21.2.101:0/1872101387 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09e0009590 0x7f09e000bab0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f09ec006af0 tx=0x7f09ec009e80).ready entity=mon.1 client_cookie=23d1f34a4a2fd96e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:08.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.268 7f09f88cc700 1 -- 172.21.2.101:0/1872101387 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f09ec006de0 con 0x7f09e0009590 2021-02-10T11:48:08.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.268 7f09f88cc700 1 -- 172.21.2.101:0/1872101387 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec00b300 con 0x7f09e0009590 2021-02-10T11:48:08.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.268 7f09f88cc700 1 -- 172.21.2.101:0/1872101387 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f09ec00b820 con 0x7f09e0009590 2021-02-10T11:48:08.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.268 7f09fbab2700 1 -- 172.21.2.101:0/1872101387 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09e0009590 msgr2=0x7f09e000bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:08.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.268 7f09fbab2700 1 --2- 172.21.2.101:0/1872101387 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09e0009590 0x7f09e000bab0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f09ec006af0 tx=0x7f09ec009e80).stop 2021-02-10T11:48:08.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.268 7f09fa0cf700 1 -- 172.21.2.101:0/1872101387 reap_dead start 2021-02-10T11:48:08.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.269 7f09fbab2700 1 -- 172.21.2.101:0/1872101387 shutdown_connections 2021-02-10T11:48:08.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.269 7f09fbab2700 1 -- 172.21.2.101:0/1872101387 >> 172.21.2.101:0/1872101387 conn(0x7f09f40ce450 msgr2=0x7f09f40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:08.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.269 7f09fbab2700 1 -- 172.21.2.101:0/1872101387 shutdown_connections 2021-02-10T11:48:08.283 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.269 7f09fbab2700 1 -- 172.21.2.101:0/1872101387 wait complete. 2021-02-10T11:48:08.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.284 7f09fbab2700 1 Processor -- start 2021-02-10T11:48:08.284 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.284 7f09fbab2700 1 -- start start 2021-02-10T11:48:08.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.284 7f09fbab2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09f41942b0 0x7f09f40b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:08.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.284 7f09fbab2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09f4195070 0x7f09f4195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:08.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.284 7f09fbab2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09f40bb8e0 0x7f09f40bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:08.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.284 7f09fbab2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f41946b0 con 0x7f09f41942b0 2021-02-10T11:48:08.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.284 7f09fbab2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f40b3b70 con 0x7f09f40bb8e0 2021-02-10T11:48:08.285 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.284 7f09fbab2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f40b3d90 con 0x7f09f4195070 2021-02-10T11:48:08.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.284 7f09f98ce700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09f41942b0 0x7f09f40b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:08.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.284 7f09f98ce700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09f41942b0 0x7f09f40b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34622/0 (socket says 172.21.2.101:34622) 2021-02-10T11:48:08.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.284 7f09f98ce700 1 -- 172.21.2.101:0/273651864 learned_addr learned my addr 172.21.2.101:0/273651864 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:08.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.285 7f09f90cd700 1 --2- 172.21.2.101:0/273651864 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09f4195070 0x7f09f4195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:08.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.285 7f09fa0cf700 1 --2- 172.21.2.101:0/273651864 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09f40bb8e0 0x7f09f40bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:08.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.285 7f09f98ce700 1 -- 172.21.2.101:0/273651864 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09f4195070 msgr2=0x7f09f4195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:08.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.285 7f09f98ce700 1 --2- 172.21.2.101:0/273651864 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09f4195070 0x7f09f4195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:08.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.285 7f09f98ce700 1 -- 172.21.2.101:0/273651864 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09f40bb8e0 msgr2=0x7f09f40bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:08.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.285 7f09f98ce700 1 --2- 172.21.2.101:0/273651864 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09f40bb8e0 0x7f09f40bbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:08.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.285 7f09f98ce700 1 -- 172.21.2.101:0/273651864 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f41a49e0 con 0x7f09f41942b0 2021-02-10T11:48:08.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.285 7f09f90cd700 1 --2- 172.21.2.101:0/273651864 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09f4195070 0x7f09f4195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:48:08.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.285 7f09f98ce700 1 --2- 172.21.2.101:0/273651864 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09f41942b0 0x7f09f40b3550 secure :-1 s=READY pgs=3186 cs=0 l=1 rev1=1 rx=0x7f09e8006b50 tx=0x7f09e800b930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:08.286 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.285 7f09e67fc700 1 -- 172.21.2.101:0/273651864 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f09e8006c50 con 0x7f09f41942b0 2021-02-10T11:48:08.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.285 7f09e67fc700 1 -- 172.21.2.101:0/273651864 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e8006030 con 0x7f09f41942b0 2021-02-10T11:48:08.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.285 7f09fbab2700 1 -- 172.21.2.101:0/273651864 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f41a49e0 con 0x7f09f41942b0 2021-02-10T11:48:08.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.285 7f09e67fc700 1 -- 172.21.2.101:0/273651864 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f09e8005540 con 0x7f09f41942b0 2021-02-10T11:48:08.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.285 7f09fbab2700 1 -- 172.21.2.101:0/273651864 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f41a49e0 con 0x7f09f41942b0 2021-02-10T11:48:08.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.286 7f09e67fc700 1 -- 172.21.2.101:0/273651864 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f09e8006c20 con 0x7f09f41942b0 2021-02-10T11:48:08.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.286 7f09e67fc700 1 --2- 172.21.2.101:0/273651864 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09dc018730 0x7f09dc01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:08.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.286 7f09e67fc700 1 -- 172.21.2.101:0/273651864 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f09e8032640 con 0x7f09f41942b0 2021-02-10T11:48:08.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.286 7f09f90cd700 1 --2- 172.21.2.101:0/273651864 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09dc018730 0x7f09dc01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:08.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.286 7f09f90cd700 1 --2- 172.21.2.101:0/273651864 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09dc018730 0x7f09dc01ac10 crc :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:08.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.316 7f09fbab2700 1 -- 172.21.2.101:0/273651864 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f41a49e0 con 0x7f09f41942b0 2021-02-10T11:48:08.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.319 7f09e67fc700 1 -- 172.21.2.101:0/273651864 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f09e80067a0 con 0x7f09f41942b0 2021-02-10T11:48:08.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.452 7f09fbab2700 1 -- 172.21.2.101:0/273651864 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_recover_auth_metadata_during_deauthorize"]} v 0) v1 -- 0x7f09f41a49e0 con 0x7f09f41942b0 2021-02-10T11:48:08.893 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.897 7f09e67fc700 1 -- 172.21.2.101:0/273651864 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_recover_auth_metadata_during_deauthorize"]}]=0 v865) v1 ==== 162+0+0 (secure 0 0 0) 0x7f09e8006c20 con 0x7f09f41942b0 2021-02-10T11:48:08.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.922 7f09fbab2700 1 -- 172.21.2.101:0/273651864 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09dc018730 msgr2=0x7f09dc01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:08.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.922 7f09fbab2700 1 --2- 172.21.2.101:0/273651864 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09dc018730 0x7f09dc01ac10 crc :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:08.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.922 7f09fbab2700 1 -- 172.21.2.101:0/273651864 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09f41942b0 msgr2=0x7f09f40b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:08.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.922 7f09fbab2700 1 --2- 172.21.2.101:0/273651864 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09f41942b0 0x7f09f40b3550 secure :-1 s=READY pgs=3186 cs=0 l=1 rev1=1 rx=0x7f09e8006b50 tx=0x7f09e800b930).stop 2021-02-10T11:48:08.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.922 7f09fbab2700 1 -- 172.21.2.101:0/273651864 shutdown_connections 2021-02-10T11:48:08.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.922 7f09fbab2700 1 --2- 172.21.2.101:0/273651864 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f09dc018730 0x7f09dc01ac10 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:08.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.922 7f09fbab2700 1 --2- 172.21.2.101:0/273651864 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f09f4195070 0x7f09f4195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:08.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.922 7f09fbab2700 1 --2- 172.21.2.101:0/273651864 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f09f40bb8e0 0x7f09f40bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:08.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.922 7f09fbab2700 1 --2- 172.21.2.101:0/273651864 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f09f41942b0 0x7f09f40b3550 unknown :-1 s=CLOSED pgs=3186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:08.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.922 7f09fbab2700 1 -- 172.21.2.101:0/273651864 >> 172.21.2.101:0/273651864 conn(0x7f09f40b3040 msgr2=0x7f09f40c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:08.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.922 7f09fbab2700 1 -- 172.21.2.101:0/273651864 shutdown_connections 2021-02-10T11:48:08.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:08.922 7f09fbab2700 1 -- 172.21.2.101:0/273651864 wait complete. 2021-02-10T11:48:08.927 INFO:tasks.cephfs_test_runner:test_recover_auth_metadata_during_deauthorize (tasks.cephfs.test_volumes.TestVolumes) ... ok 2021-02-10T11:48:08.927 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_authorized_list (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:48:08.927 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_subvolume_authorized_list' 2021-02-10T11:48:09.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.342 7ff0c49c3700 1 Processor -- start 2021-02-10T11:48:09.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.342 7ff0c49c3700 1 -- start start 2021-02-10T11:48:09.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.343 7ff0c49c3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0c01b0870 0x7ff0c00f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:09.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.343 7ff0c49c3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff0c00efe00 0x7ff0c00f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:09.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.343 7ff0c49c3700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c01b0c70 con 0x7ff0c01b0470 2021-02-10T11:48:09.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.343 7ff0c49c3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c00f0b50 con 0x7ff0c00efe00 2021-02-10T11:48:09.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.343 7ff0c49c3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c00f0820 con 0x7ff0c01b0870 2021-02-10T11:48:09.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.344 7ff0bee1d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff0c00efe00 0x7ff0c00f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:09.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.344 7ff0bde1b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0c01b0870 0x7ff0c00f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:09.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.344 7ff0be61c700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7ff0c01b0470 0x7ff0c00ef610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:56764/0 (socket says 172.21.2.101:56764) 2021-02-10T11:48:09.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.344 7ff0be61c700 1 -- 172.21.2.101:0/1596856746 learned_addr learned my addr 172.21.2.101:0/1596856746 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:09.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.344 7ff0bee1d700 1 -- 172.21.2.101:0/1596856746 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0c01b0870 msgr2=0x7ff0c00f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:09.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.344 7ff0bee1d700 1 --2- 172.21.2.101:0/1596856746 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0c01b0870 0x7ff0c00f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:09.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.344 7ff0bee1d700 1 -- 172.21.2.101:0/1596856746 >> v1:172.21.2.108:6789/0 conn(0x7ff0c01b0470 legacy=0x7ff0c00ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:09.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.344 7ff0bee1d700 1 -- 172.21.2.101:0/1596856746 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c00e52e0 con 0x7ff0c00efe00 2021-02-10T11:48:09.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.344 7ff0bee1d700 1 --2- 172.21.2.101:0/1596856746 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0c00efe00 0x7ff0c00f0200 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7ff0b8006e50 tx=0x7ff0b8009e70).ready entity=mon.1 client_cookie=5272492fd07e130b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:09.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.345 7ff0bd61a700 1 -- 172.21.2.101:0/1596856746 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0b800b510 con 0x7ff0c00efe00 2021-02-10T11:48:09.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.345 7ff0bd61a700 1 -- 172.21.2.101:0/1596856746 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b8006790 con 0x7ff0c00efe00 2021-02-10T11:48:09.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.345 7ff0bd61a700 1 -- 172.21.2.101:0/1596856746 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0b800b940 con 0x7ff0c00efe00 2021-02-10T11:48:09.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.345 7ff0c49c3700 1 -- 172.21.2.101:0/1596856746 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0c00efe00 msgr2=0x7ff0c00f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:09.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.345 7ff0c49c3700 1 --2- 172.21.2.101:0/1596856746 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0c00efe00 0x7ff0c00f0200 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7ff0b8006e50 tx=0x7ff0b8009e70).stop 2021-02-10T11:48:09.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.345 7ff0c49c3700 1 -- 172.21.2.101:0/1596856746 shutdown_connections 2021-02-10T11:48:09.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.345 7ff0c49c3700 1 --2- 172.21.2.101:0/1596856746 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0c01b0870 0x7ff0c00f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:09.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.345 7ff0c49c3700 1 --2- 172.21.2.101:0/1596856746 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0c00efe00 0x7ff0c00f0200 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:09.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.345 7ff0c49c3700 1 -- 172.21.2.101:0/1596856746 >> 172.21.2.101:0/1596856746 conn(0x7ff0c00ce3e0 msgr2=0x7ff0c00e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:09.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.346 7ff0c49c3700 1 -- 172.21.2.101:0/1596856746 shutdown_connections 2021-02-10T11:48:09.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.346 7ff0c49c3700 1 -- 172.21.2.101:0/1596856746 wait complete. 2021-02-10T11:48:09.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.360 7ff0c49c3700 1 Processor -- start 2021-02-10T11:48:09.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.360 7ff0c49c3700 1 -- start start 2021-02-10T11:48:09.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.360 7ff0c49c3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0c01af8a0 0x7ff0c0198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:09.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.360 7ff0c49c3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0c01af050 0x7ff0c00aa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:09.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.360 7ff0c49c3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0c0197d00 0x7ff0c0198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:09.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.360 7ff0c49c3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c01988e0 con 0x7ff0c0197d00 2021-02-10T11:48:09.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.360 7ff0c49c3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c01af450 con 0x7ff0c01af050 2021-02-10T11:48:09.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.360 7ff0c49c3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c00aaa10 con 0x7ff0c01af8a0 2021-02-10T11:48:09.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.361 7ff0bee1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0c0197d00 0x7ff0c0198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:09.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.361 7ff0bee1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0c0197d00 0x7ff0c0198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34638/0 (socket says 172.21.2.101:34638) 2021-02-10T11:48:09.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.361 7ff0bee1d700 1 -- 172.21.2.101:0/3804063455 learned_addr learned my addr 172.21.2.101:0/3804063455 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:09.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.361 7ff0be61c700 1 --2- 172.21.2.101:0/3804063455 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0c01af8a0 0x7ff0c0198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:09.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.361 7ff0bde1b700 1 --2- 172.21.2.101:0/3804063455 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0c01af050 0x7ff0c00aa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:09.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.361 7ff0bee1d700 1 -- 172.21.2.101:0/3804063455 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0c01af8a0 msgr2=0x7ff0c0198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:09.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.361 7ff0bee1d700 1 --2- 172.21.2.101:0/3804063455 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0c01af8a0 0x7ff0c0198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:09.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.361 7ff0bee1d700 1 -- 172.21.2.101:0/3804063455 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0c01af050 msgr2=0x7ff0c00aa3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:09.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.361 7ff0bee1d700 1 --2- 172.21.2.101:0/3804063455 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0c01af050 0x7ff0c00aa3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:09.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.361 7ff0bee1d700 1 -- 172.21.2.101:0/3804063455 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c019fec0 con 0x7ff0c0197d00 2021-02-10T11:48:09.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.361 7ff0bde1b700 1 --2- 172.21.2.101:0/3804063455 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0c01af050 0x7ff0c00aa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:48:09.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.361 7ff0bee1d700 1 --2- 172.21.2.101:0/3804063455 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0c0197d00 0x7ff0c0198100 secure :-1 s=READY pgs=3187 cs=0 l=1 rev1=1 rx=0x7ff0b8006c50 tx=0x7ff0b800bd20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:09.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.361 7ff0b6ffd700 1 -- 172.21.2.101:0/3804063455 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0b80093d0 con 0x7ff0c0197d00 2021-02-10T11:48:09.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.361 7ff0b6ffd700 1 -- 172.21.2.101:0/3804063455 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b800be60 con 0x7ff0c0197d00 2021-02-10T11:48:09.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.362 7ff0b6ffd700 1 -- 172.21.2.101:0/3804063455 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0b800c570 con 0x7ff0c0197d00 2021-02-10T11:48:09.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.362 7ff0c49c3700 1 -- 172.21.2.101:0/3804063455 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c019fec0 con 0x7ff0c0197d00 2021-02-10T11:48:09.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.362 7ff0c49c3700 1 -- 172.21.2.101:0/3804063455 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c019fec0 con 0x7ff0c0197d00 2021-02-10T11:48:09.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.362 7ff0b6ffd700 1 -- 172.21.2.101:0/3804063455 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff0b8002ef0 con 0x7ff0c0197d00 2021-02-10T11:48:09.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.362 7ff0b6ffd700 1 --2- 172.21.2.101:0/3804063455 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0a40186c0 0x7ff0a401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:09.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.362 7ff0b6ffd700 1 -- 172.21.2.101:0/3804063455 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff0b8032910 con 0x7ff0c0197d00 2021-02-10T11:48:09.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.363 7ff0be61c700 1 --2- 172.21.2.101:0/3804063455 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0a40186c0 0x7ff0a401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:09.362 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.363 7ff0be61c700 1 --2- 172.21.2.101:0/3804063455 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0a40186c0 0x7ff0a401abc0 crc :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:09.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.392 7ff0c49c3700 1 -- 172.21.2.101:0/3804063455 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c0105a20 con 0x7ff0c0197d00 2021-02-10T11:48:09.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.395 7ff0b6ffd700 1 -- 172.21.2.101:0/3804063455 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff0b8000df0 con 0x7ff0c0197d00 2021-02-10T11:48:09.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.526 7ff0c49c3700 1 -- 172.21.2.101:0/3804063455 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_subvolume_authorized_list"]} v 0) v1 -- 0x7ff0c00f0b50 con 0x7ff0c0197d00 2021-02-10T11:48:09.632 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:48:09.636 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:48:09.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.901 7ff0b6ffd700 1 -- 172.21.2.101:0/3804063455 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_subvolume_authorized_list"]}]=0 v866) v1 ==== 150+0+0 (secure 0 0 0) 0x7ff0b8002ef0 con 0x7ff0c0197d00 2021-02-10T11:48:09.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.944 7ff0c49c3700 1 -- 172.21.2.101:0/3804063455 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0a40186c0 msgr2=0x7ff0a401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:09.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.944 7ff0c49c3700 1 --2- 172.21.2.101:0/3804063455 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0a40186c0 0x7ff0a401abc0 crc :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:09.941 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.945 7ff0c49c3700 1 -- 172.21.2.101:0/3804063455 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0c0197d00 msgr2=0x7ff0c0198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:09.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.945 7ff0c49c3700 1 --2- 172.21.2.101:0/3804063455 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0c0197d00 0x7ff0c0198100 secure :-1 s=READY pgs=3187 cs=0 l=1 rev1=1 rx=0x7ff0b8006c50 tx=0x7ff0b800bd20).stop 2021-02-10T11:48:09.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.945 7ff0c49c3700 1 -- 172.21.2.101:0/3804063455 shutdown_connections 2021-02-10T11:48:09.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.945 7ff0c49c3700 1 --2- 172.21.2.101:0/3804063455 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0c0197d00 0x7ff0c0198100 unknown :-1 s=CLOSED pgs=3187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:09.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.945 7ff0c49c3700 1 --2- 172.21.2.101:0/3804063455 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0c01af050 0x7ff0c00aa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:09.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.945 7ff0c49c3700 1 --2- 172.21.2.101:0/3804063455 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0a40186c0 0x7ff0a401abc0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:09.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.945 7ff0c49c3700 1 --2- 172.21.2.101:0/3804063455 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0c01af8a0 0x7ff0c0198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:09.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.945 7ff0c49c3700 1 -- 172.21.2.101:0/3804063455 >> 172.21.2.101:0/3804063455 conn(0x7ff0c01a37a0 msgr2=0x7ff0c018c230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:09.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.945 7ff0c49c3700 1 -- 172.21.2.101:0/3804063455 shutdown_connections 2021-02-10T11:48:09.942 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:09.945 7ff0c49c3700 1 -- 172.21.2.101:0/3804063455 wait complete. 2021-02-10T11:48:09.950 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:48:10.383 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.370 7fe3646ae700 1 Processor -- start 2021-02-10T11:48:10.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.370 7fe3646ae700 1 -- start start 2021-02-10T11:48:10.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.370 7fe3646ae700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe35c10aff0 0x7fe35c0cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:10.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.370 7fe3646ae700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35c0fe1c0 con 0x7fe35c0d1c70 2021-02-10T11:48:10.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.370 7fe3646ae700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35c1b0cb0 con 0x7fe35c0d1870 2021-02-10T11:48:10.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.370 7fe3646ae700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c109a60 con 0x7fe35c10aff0 2021-02-10T11:48:10.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.371 7fe361cc9700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe35c0d1c70 0x7fe35c0cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35136/0 (socket says 172.21.2.101:35136) 2021-02-10T11:48:10.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.371 7fe361cc9700 1 -- 172.21.2.101:0/1291492729 learned_addr learned my addr 172.21.2.101:0/1291492729 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:10.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.371 7fe362ccb700 1 --2- 172.21.2.101:0/1291492729 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe35c10aff0 0x7fe35c0cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:10.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.371 7fe3614c8700 1 -- 172.21.2.101:0/1291492729 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3417771819 0 0) 0x7fe358001180 con 0x7fe35c0d1c70 2021-02-10T11:48:10.384 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.371 7fe3614c8700 1 -- 172.21.2.101:0/1291492729 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe348002fe0 con 0x7fe35c0d1c70 2021-02-10T11:48:10.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.371 7fe3614c8700 1 -- 172.21.2.101:0/1291492729 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252535156 0 0) 0x7fe350001080 con 0x7fe35c0d1870 2021-02-10T11:48:10.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.371 7fe3614c8700 1 -- 172.21.2.101:0/1291492729 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe348003210 con 0x7fe35c0d1870 2021-02-10T11:48:10.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.371 7fe362ccb700 1 -- 172.21.2.101:0/1291492729 >> v1:172.21.2.108:6789/0 conn(0x7fe35c0d1870 legacy=0x7fe35c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:10.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.371 7fe362ccb700 1 -- 172.21.2.101:0/1291492729 >> v1:172.21.2.101:6789/0 conn(0x7fe35c0d1c70 legacy=0x7fe35c0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:10.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.371 7fe362ccb700 1 -- 172.21.2.101:0/1291492729 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c10b3f0 con 0x7fe35c10aff0 2021-02-10T11:48:10.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.372 7fe362ccb700 1 --2- 172.21.2.101:0/1291492729 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe35c10aff0 0x7fe35c0cf750 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fe354006e50 tx=0x7fe354009e70).ready entity=mon.2 client_cookie=b8b064da269bc14c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:10.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.372 7fe3614c8700 1 -- 172.21.2.101:0/1291492729 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe354006a20 con 0x7fe35c10aff0 2021-02-10T11:48:10.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.372 7fe3614c8700 1 -- 172.21.2.101:0/1291492729 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe354006790 con 0x7fe35c10aff0 2021-02-10T11:48:10.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.372 7fe3614c8700 1 -- 172.21.2.101:0/1291492729 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe35400b700 con 0x7fe35c10aff0 2021-02-10T11:48:10.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.372 7fe3646ae700 1 -- 172.21.2.101:0/1291492729 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe35c10aff0 msgr2=0x7fe35c0cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:10.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.372 7fe3646ae700 1 --2- 172.21.2.101:0/1291492729 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe35c10aff0 0x7fe35c0cf750 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fe354006e50 tx=0x7fe354009e70).stop 2021-02-10T11:48:10.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.373 7fe3646ae700 1 -- 172.21.2.101:0/1291492729 shutdown_connections 2021-02-10T11:48:10.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.373 7fe3646ae700 1 --2- 172.21.2.101:0/1291492729 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe35c10aff0 0x7fe35c0cf750 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:10.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.373 7fe3646ae700 1 -- 172.21.2.101:0/1291492729 >> 172.21.2.101:0/1291492729 conn(0x7fe35c0ce420 msgr2=0x7fe35c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:10.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.373 7fe3646ae700 1 -- 172.21.2.101:0/1291492729 shutdown_connections 2021-02-10T11:48:10.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.373 7fe3646ae700 1 -- 172.21.2.101:0/1291492729 wait complete. 2021-02-10T11:48:10.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.387 7fe3646ae700 1 Processor -- start 2021-02-10T11:48:10.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.387 7fe3646ae700 1 -- start start 2021-02-10T11:48:10.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.388 7fe3646ae700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe35c0a7950 0x7fe35c0b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:10.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.388 7fe3646ae700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe35c1a7490 0x7fe35c1a7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:10.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.388 7fe3646ae700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe35c1a7eb0 0x7fe35c0ac630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:10.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.388 7fe3646ae700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0a7fb0 con 0x7fe35c0a7950 2021-02-10T11:48:10.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.388 7fe3646ae700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0acc50 con 0x7fe35c1a7490 2021-02-10T11:48:10.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.388 7fe3646ae700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0ace30 con 0x7fe35c1a7eb0 2021-02-10T11:48:10.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.388 7fe3624ca700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe35c0a7950 0x7fe35c0b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:10.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.388 7fe3624ca700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe35c0a7950 0x7fe35c0b8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34652/0 (socket says 172.21.2.101:34652) 2021-02-10T11:48:10.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.388 7fe3624ca700 1 -- 172.21.2.101:0/1831765789 learned_addr learned my addr 172.21.2.101:0/1831765789 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:10.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.388 7fe361cc9700 1 --2- 172.21.2.101:0/1831765789 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe35c1a7490 0x7fe35c1a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:10.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.388 7fe362ccb700 1 --2- 172.21.2.101:0/1831765789 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe35c1a7eb0 0x7fe35c0ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:10.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.388 7fe3624ca700 1 -- 172.21.2.101:0/1831765789 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe35c1a7eb0 msgr2=0x7fe35c0ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:10.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.388 7fe3624ca700 1 --2- 172.21.2.101:0/1831765789 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe35c1a7eb0 0x7fe35c0ac630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:10.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.388 7fe3624ca700 1 -- 172.21.2.101:0/1831765789 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe35c1a7490 msgr2=0x7fe35c1a7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:10.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.388 7fe3624ca700 1 --2- 172.21.2.101:0/1831765789 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe35c1a7490 0x7fe35c1a7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:10.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.388 7fe3624ca700 1 -- 172.21.2.101:0/1831765789 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c0ad050 con 0x7fe35c0a7950 2021-02-10T11:48:10.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.388 7fe3624ca700 1 --2- 172.21.2.101:0/1831765789 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe35c0a7950 0x7fe35c0b8df0 secure :-1 s=READY pgs=3189 cs=0 l=1 rev1=1 rx=0x7fe350006f00 tx=0x7fe35000b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:10.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.389 7fe34effd700 1 -- 172.21.2.101:0/1831765789 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe35000bd30 con 0x7fe35c0a7950 2021-02-10T11:48:10.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.389 7fe34effd700 1 -- 172.21.2.101:0/1831765789 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe350000e20 con 0x7fe35c0a7950 2021-02-10T11:48:10.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.389 7fe34effd700 1 -- 172.21.2.101:0/1831765789 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe3500054f0 con 0x7fe35c0a7950 2021-02-10T11:48:10.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.389 7fe3646ae700 1 -- 172.21.2.101:0/1831765789 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35c0ad050 con 0x7fe35c0a7950 2021-02-10T11:48:10.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.389 7fe3646ae700 1 -- 172.21.2.101:0/1831765789 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35c0ad050 con 0x7fe35c0a7950 2021-02-10T11:48:10.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.389 7fe34effd700 1 -- 172.21.2.101:0/1831765789 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe350006030 con 0x7fe35c0a7950 2021-02-10T11:48:10.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.389 7fe34effd700 1 --2- 172.21.2.101:0/1831765789 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3440186c0 0x7fe34401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:10.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.389 7fe34effd700 1 -- 172.21.2.101:0/1831765789 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe350032560 con 0x7fe35c0a7950 2021-02-10T11:48:10.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.390 7fe361cc9700 1 --2- 172.21.2.101:0/1831765789 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3440186c0 0x7fe34401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:10.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.390 7fe361cc9700 1 --2- 172.21.2.101:0/1831765789 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3440186c0 0x7fe34401abc0 crc :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:10.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.420 7fe3646ae700 1 -- 172.21.2.101:0/1831765789 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35c082a70 con 0x7fe35c0a7950 2021-02-10T11:48:10.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.423 7fe34effd700 1 -- 172.21.2.101:0/1831765789 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe35002fbd0 con 0x7fe35c0a7950 2021-02-10T11:48:10.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.569 7fe3646ae700 1 -- 172.21.2.101:0/1831765789 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe35c082a70 con 0x7fe35c0a7950 2021-02-10T11:48:10.570 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.574 7fe34effd700 1 -- 172.21.2.101:0/1831765789 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe35002fbd0 con 0x7fe35c0a7950 2021-02-10T11:48:10.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.579 7fe3646ae700 1 -- 172.21.2.101:0/1831765789 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3440186c0 msgr2=0x7fe34401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:10.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.579 7fe3646ae700 1 --2- 172.21.2.101:0/1831765789 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3440186c0 0x7fe34401abc0 crc :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:10.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.579 7fe3646ae700 1 -- 172.21.2.101:0/1831765789 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe35c0a7950 msgr2=0x7fe35c0b8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:10.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.579 7fe3646ae700 1 --2- 172.21.2.101:0/1831765789 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe35c0a7950 0x7fe35c0b8df0 secure :-1 s=READY pgs=3189 cs=0 l=1 rev1=1 rx=0x7fe350006f00 tx=0x7fe35000b940).stop 2021-02-10T11:48:10.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.579 7fe3646ae700 1 -- 172.21.2.101:0/1831765789 shutdown_connections 2021-02-10T11:48:10.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.579 7fe3646ae700 1 --2- 172.21.2.101:0/1831765789 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe3440186c0 0x7fe34401abc0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:10.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.579 7fe3646ae700 1 --2- 172.21.2.101:0/1831765789 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe35c1a7eb0 0x7fe35c0ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:10.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.579 7fe3646ae700 1 --2- 172.21.2.101:0/1831765789 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe35c1a7490 0x7fe35c1a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:10.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.579 7fe3646ae700 1 --2- 172.21.2.101:0/1831765789 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe35c0a7950 0x7fe35c0b8df0 unknown :-1 s=CLOSED pgs=3189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:10.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.579 7fe3646ae700 1 -- 172.21.2.101:0/1831765789 >> 172.21.2.101:0/1831765789 conn(0x7fe35c0a02d0 msgr2=0x7fe35c09f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:10.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.579 7fe3646ae700 1 -- 172.21.2.101:0/1831765789 shutdown_connections 2021-02-10T11:48:10.576 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:10.579 7fe3646ae700 1 -- 172.21.2.101:0/1831765789 wait complete. 2021-02-10T11:48:10.584 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:48:10.584 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:48:10.930 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:48:11.276 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:48:11.276 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:11.276 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:48:11.277 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:48:11.611 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2021-02-10T11:48:11.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:48:11.612 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-02-10T11:48:11.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:11.612 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:48:11.612 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:48:11.948 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:48:11.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:48:11.948 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:48:11.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:11.949 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:48:11.949 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:48:12.283 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:48:12.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:48:12.283 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:48:12.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:12.284 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:48:12.284 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:48:12.618 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:48:12.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:48:12.619 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:48:12.619 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json-pretty 2021-02-10T11:48:13.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.035 7f1c00ea5700 1 Processor -- start 2021-02-10T11:48:13.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.035 7f1c00ea5700 1 -- start start 2021-02-10T11:48:13.048 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.036 7f1c00ea5700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1bfc0d1870 0x7f1bfc1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:13.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.036 7f1c00ea5700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1bfc1b0a10 0x7f1bfc0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:13.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.036 7f1c00ea5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1bfc0d1c70 0x7f1bfc0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:13.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.036 7f1c00ea5700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bfc0e5ed0 con 0x7f1bfc0d1870 2021-02-10T11:48:13.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.036 7f1c00ea5700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bfc0f0b10 con 0x7f1bfc1b0a10 2021-02-10T11:48:13.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.036 7f1c00ea5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc0cff70 con 0x7f1bfc0d1c70 2021-02-10T11:48:13.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.036 7f1bfab13700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1bfc0d1870 0x7f1bfc1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:13.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.036 7f1bfab13700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1bfc0d1870 0x7f1bfc1b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34656/0 (socket says 172.21.2.101:34656) 2021-02-10T11:48:13.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.036 7f1bfab13700 1 -- 172.21.2.101:0/1573911727 learned_addr learned my addr 172.21.2.101:0/1573911727 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:13.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.036 7f1bfab13700 1 -- 172.21.2.101:0/1573911727 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1bfc0d1c70 msgr2=0x7f1bfc0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:13.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.036 7f1bfab13700 1 --2- 172.21.2.101:0/1573911727 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1bfc0d1c70 0x7f1bfc0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:13.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.036 7f1bfab13700 1 -- 172.21.2.101:0/1573911727 >> v2:172.21.2.108:3300/0 conn(0x7f1bfc1b0a10 msgr2=0x7f1bfc0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:13.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.036 7f1bfab13700 1 --2- 172.21.2.101:0/1573911727 >> v2:172.21.2.108:3300/0 conn(0x7f1bfc1b0a10 0x7f1bfc0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:13.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.036 7f1bfab13700 1 -- 172.21.2.101:0/1573911727 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc0d0150 con 0x7f1bfc0d1870 2021-02-10T11:48:13.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.036 7f1bfab13700 1 --2- 172.21.2.101:0/1573911727 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1bfc0d1870 0x7f1bfc1b03f0 secure :-1 s=READY pgs=3190 cs=0 l=1 rev1=1 rx=0x7f1be4006700 tx=0x7f1be4009b80).ready entity=mon.0 client_cookie=ec8123b5f29e3957 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:13.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.037 7f1bf9b11700 1 -- 172.21.2.101:0/1573911727 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1be400c0c0 con 0x7f1bfc0d1870 2021-02-10T11:48:13.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.037 7f1bf9b11700 1 -- 172.21.2.101:0/1573911727 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be400c400 con 0x7f1bfc0d1870 2021-02-10T11:48:13.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.037 7f1bf9b11700 1 -- 172.21.2.101:0/1573911727 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1be400c920 con 0x7f1bfc0d1870 2021-02-10T11:48:13.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.037 7f1c00ea5700 1 -- 172.21.2.101:0/1573911727 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1bfc0d1870 msgr2=0x7f1bfc1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:13.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.037 7f1c00ea5700 1 --2- 172.21.2.101:0/1573911727 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1bfc0d1870 0x7f1bfc1b03f0 secure :-1 s=READY pgs=3190 cs=0 l=1 rev1=1 rx=0x7f1be4006700 tx=0x7f1be4009b80).stop 2021-02-10T11:48:13.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.037 7f1c00ea5700 1 -- 172.21.2.101:0/1573911727 shutdown_connections 2021-02-10T11:48:13.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.038 7f1c00ea5700 1 --2- 172.21.2.101:0/1573911727 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1bfc0d1c70 0x7f1bfc0cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:13.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.038 7f1c00ea5700 1 --2- 172.21.2.101:0/1573911727 >> v2:172.21.2.108:3300/0 conn(0x7f1bfc1b0a10 0x7f1bfc0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:13.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.038 7f1c00ea5700 1 --2- 172.21.2.101:0/1573911727 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1bfc0d1870 0x7f1bfc1b03f0 unknown :-1 s=CLOSED pgs=3190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:13.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.038 7f1c00ea5700 1 -- 172.21.2.101:0/1573911727 >> 172.21.2.101:0/1573911727 conn(0x7f1bfc0ce450 msgr2=0x7f1bfc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:13.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.038 7f1c00ea5700 1 -- 172.21.2.101:0/1573911727 shutdown_connections 2021-02-10T11:48:13.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.038 7f1c00ea5700 1 -- 172.21.2.101:0/1573911727 wait complete. 2021-02-10T11:48:13.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.052 7f1c00ea5700 1 Processor -- start 2021-02-10T11:48:13.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.052 7f1c00ea5700 1 -- start start 2021-02-10T11:48:13.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.053 7f1c00ea5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1bfc1b7cf0 0x7f1bfc19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:13.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.053 7f1c00ea5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1bfc1b80f0 0x7f1bfc1b84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:13.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.053 7f1c00ea5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1bfc19da20 0x7f1bfc19de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:13.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.053 7f1c00ea5700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc0a77c0 con 0x7f1bfc19da20 2021-02-10T11:48:13.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.053 7f1c00ea5700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc19d4a0 con 0x7f1bfc1b80f0 2021-02-10T11:48:13.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.053 7f1c00ea5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc0954f0 con 0x7f1bfc1b7cf0 2021-02-10T11:48:13.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.053 7f1bfb314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1bfc19da20 0x7f1bfc19de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:13.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.053 7f1bfb314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1bfc19da20 0x7f1bfc19de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34664/0 (socket says 172.21.2.101:34664) 2021-02-10T11:48:13.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.053 7f1bfb314700 1 -- 172.21.2.101:0/2164452362 learned_addr learned my addr 172.21.2.101:0/2164452362 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:13.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.053 7f1bfb314700 1 -- 172.21.2.101:0/2164452362 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1bfc1b7cf0 msgr2=0x7f1bfc19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:13.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.053 7f1bfb314700 1 --2- 172.21.2.101:0/2164452362 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1bfc1b7cf0 0x7f1bfc19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:13.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.053 7f1bfb314700 1 -- 172.21.2.101:0/2164452362 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1bfc1b80f0 msgr2=0x7f1bfc1b84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:13.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.053 7f1bfb314700 1 --2- 172.21.2.101:0/2164452362 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1bfc1b80f0 0x7f1bfc1b84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:13.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.053 7f1bfb314700 1 -- 172.21.2.101:0/2164452362 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc1b8d60 con 0x7f1bfc19da20 2021-02-10T11:48:13.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.053 7f1bfb314700 1 --2- 172.21.2.101:0/2164452362 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1bfc19da20 0x7f1bfc19de20 secure :-1 s=READY pgs=3191 cs=0 l=1 rev1=1 rx=0x7f1bf4009980 tx=0x7f1bf400bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:13.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.053 7f1bf37fe700 1 -- 172.21.2.101:0/2164452362 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1bf4005af0 con 0x7f1bfc19da20 2021-02-10T11:48:13.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.054 7f1bf37fe700 1 -- 172.21.2.101:0/2164452362 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf4005d70 con 0x7f1bfc19da20 2021-02-10T11:48:13.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.054 7f1bf37fe700 1 -- 172.21.2.101:0/2164452362 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1bf40062b0 con 0x7f1bfc19da20 2021-02-10T11:48:13.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.054 7f1c00ea5700 1 -- 172.21.2.101:0/2164452362 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bfc1b8d60 con 0x7f1bfc19da20 2021-02-10T11:48:13.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.054 7f1c00ea5700 1 -- 172.21.2.101:0/2164452362 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bfc19e730 con 0x7f1bfc19da20 2021-02-10T11:48:13.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.054 7f1bf37fe700 1 -- 172.21.2.101:0/2164452362 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1bf4006030 con 0x7f1bfc19da20 2021-02-10T11:48:13.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.054 7f1bf37fe700 1 --2- 172.21.2.101:0/2164452362 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1be00186c0 0x7f1be001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:13.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.054 7f1bf37fe700 1 -- 172.21.2.101:0/2164452362 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1bf40328f0 con 0x7f1bfc19da20 2021-02-10T11:48:13.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.055 7f1bfab13700 1 --2- 172.21.2.101:0/2164452362 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1be00186c0 0x7f1be001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:13.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.055 7f1bfab13700 1 --2- 172.21.2.101:0/2164452362 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1be00186c0 0x7f1be001abc0 crc :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:13.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.086 7f1c00ea5700 1 -- 172.21.2.101:0/2164452362 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bfc0ebc00 con 0x7f1bfc19da20 2021-02-10T11:48:13.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.088 7f1bf37fe700 1 -- 172.21.2.101:0/2164452362 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1bf4025710 con 0x7f1bfc19da20 2021-02-10T11:48:13.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.212 7f1c00ea5700 1 -- 172.21.2.101:0/2164452362 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f1bfc0f0b10 con 0x7f1bfc19da20 2021-02-10T11:48:13.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.213 7f1bf37fe700 1 -- 172.21.2.101:0/2164452362 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v183) v1 ==== 81+0+30374 (secure 0 0 0) 0x7f1bf4025710 con 0x7f1bfc19da20 2021-02-10T11:48:13.210 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:48:13.210 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:48:13.210 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 183, 2021-02-10T11:48:13.210 INFO:teuthology.orchestra.run.gibba001.stdout: "fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:48:13.210 INFO:teuthology.orchestra.run.gibba001.stdout: "created": "2021-02-10 11:12:36.788366", 2021-02-10T11:48:13.210 INFO:teuthology.orchestra.run.gibba001.stdout: "modified": "2021-02-10 11:47:29.788748", 2021-02-10T11:48:13.210 INFO:teuthology.orchestra.run.gibba001.stdout: "last_up_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:48:13.210 INFO:teuthology.orchestra.run.gibba001.stdout: "last_in_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:48:13.210 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-02-10T11:48:13.210 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_num": 5799936, 2021-02-10T11:48:13.210 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_set": [ 2021-02-10T11:48:13.211 INFO:teuthology.orchestra.run.gibba001.stdout: "pglog_hardlimit", 2021-02-10T11:48:13.211 INFO:teuthology.orchestra.run.gibba001.stdout: "purged_snapdirs", 2021-02-10T11:48:13.211 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_deletes", 2021-02-10T11:48:13.211 INFO:teuthology.orchestra.run.gibba001.stdout: "sortbitwise" 2021-02-10T11:48:13.211 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:48:13.211 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_version": 4, 2021-02-10T11:48:13.211 INFO:teuthology.orchestra.run.gibba001.stdout: "full_ratio": 0.94999998807907104, 2021-02-10T11:48:13.211 INFO:teuthology.orchestra.run.gibba001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-02-10T11:48:13.211 INFO:teuthology.orchestra.run.gibba001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-02-10T11:48:13.211 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_snapshot": "", 2021-02-10T11:48:13.211 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_max": 39, 2021-02-10T11:48:13.211 INFO:teuthology.orchestra.run.gibba001.stdout: "max_osd": 8, 2021-02-10T11:48:13.211 INFO:teuthology.orchestra.run.gibba001.stdout: "require_min_compat_client": "jewel", 2021-02-10T11:48:13.212 INFO:teuthology.orchestra.run.gibba001.stdout: "min_compat_client": "jewel", 2021-02-10T11:48:13.212 INFO:teuthology.orchestra.run.gibba001.stdout: "require_osd_release": "nautilus", 2021-02-10T11:48:13.212 INFO:teuthology.orchestra.run.gibba001.stdout: "pools": [ 2021-02-10T11:48:13.212 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.212 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 1, 2021-02-10T11:48:13.212 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "rbd", 2021-02-10T11:48:13.212 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:13:08.037472", 2021-02-10T11:48:13.212 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:48:13.212 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:48:13.212 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:48:13.212 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:48:13.212 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:48:13.212 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:48:13.212 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:48:13.213 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:48:13.213 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:48:13.213 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:48:13.213 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:48:13.213 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:48:13.213 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:48:13.213 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:48:13.213 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:48:13.213 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:48:13.213 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:48:13.213 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:48:13.213 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:48:13.213 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:48:13.213 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.214 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "16", 2021-02-10T11:48:13.214 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:48:13.214 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:48:13.214 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:48:13.214 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:48:13.214 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:48:13.214 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:48:13.214 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:48:13.214 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:48:13.214 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:48:13.214 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:48:13.214 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:48:13.215 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:48:13.215 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:48:13.215 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:48:13.215 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:48:13.215 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:48:13.215 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:48:13.215 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:48:13.215 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:48:13.215 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:48:13.215 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:48:13.215 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:48:13.216 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:48:13.216 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:48:13.216 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:48:13.216 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:48:13.216 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.216 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:48:13.216 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:48:13.216 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:48:13.216 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:48:13.216 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:48:13.216 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:48:13.216 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:48:13.216 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:48:13.217 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:48:13.217 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:48:13.217 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:48:13.217 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:48:13.217 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:48:13.217 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd": {} 2021-02-10T11:48:13.217 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.217 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.217 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.217 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 38, 2021-02-10T11:48:13.217 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_metadata", 2021-02-10T11:48:13.217 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:47:26.756268", 2021-02-10T11:48:13.217 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:48:13.217 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:48:13.218 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:48:13.218 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:48:13.218 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:48:13.218 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:48:13.218 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:48:13.218 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:48:13.218 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:48:13.218 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:48:13.218 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:48:13.218 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:48:13.218 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:48:13.218 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:48:13.218 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:48:13.219 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:48:13.219 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:48:13.219 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:48:13.219 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:48:13.219 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:48:13.219 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.219 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "182", 2021-02-10T11:48:13.219 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:48:13.219 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:48:13.219 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:48:13.219 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:48:13.219 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:48:13.219 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:48:13.219 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:48:13.220 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:48:13.220 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:48:13.220 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:48:13.220 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:48:13.220 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:48:13.220 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:48:13.220 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:48:13.220 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:48:13.220 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:48:13.220 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:48:13.220 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:48:13.220 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:48:13.220 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:48:13.220 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:48:13.221 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:48:13.221 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:48:13.221 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:48:13.221 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:48:13.221 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:48:13.221 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.221 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:48:13.221 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:48:13.221 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:48:13.221 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:48:13.221 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:48:13.221 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:48:13.221 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:48:13.222 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:48:13.222 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:48:13.222 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:48:13.222 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:48:13.222 INFO:teuthology.orchestra.run.gibba001.stdout: "options": { 2021-02-10T11:48:13.222 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_bias": 4, 2021-02-10T11:48:13.222 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_min": 16, 2021-02-10T11:48:13.222 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_priority": 5 2021-02-10T11:48:13.222 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.222 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:48:13.222 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:48:13.222 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": "cephfs" 2021-02-10T11:48:13.222 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.222 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.223 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.223 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.223 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 39, 2021-02-10T11:48:13.223 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_data", 2021-02-10T11:48:13.223 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:47:27.781048", 2021-02-10T11:48:13.223 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:48:13.223 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:48:13.223 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:48:13.223 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:48:13.223 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:48:13.223 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:48:13.223 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:48:13.223 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:48:13.224 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:48:13.224 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:48:13.224 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:48:13.224 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:48:13.224 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:48:13.224 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:48:13.224 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:48:13.224 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:48:13.224 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:48:13.224 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:48:13.224 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:48:13.224 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:48:13.224 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.225 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "183", 2021-02-10T11:48:13.225 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:48:13.225 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:48:13.225 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:48:13.225 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:48:13.225 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:48:13.225 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:48:13.225 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:48:13.225 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:48:13.225 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:48:13.225 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:48:13.225 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:48:13.225 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:48:13.226 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:48:13.226 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:48:13.226 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:48:13.226 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:48:13.226 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:48:13.226 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:48:13.226 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:48:13.226 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:48:13.226 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:48:13.226 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:48:13.226 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:48:13.226 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:48:13.226 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:48:13.227 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:48:13.227 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.227 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:48:13.227 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:48:13.227 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:48:13.227 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:48:13.227 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:48:13.227 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:48:13.227 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:48:13.227 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:48:13.227 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:48:13.227 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:48:13.227 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:48:13.228 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:48:13.228 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:48:13.228 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:48:13.228 INFO:teuthology.orchestra.run.gibba001.stdout: "data": "cephfs" 2021-02-10T11:48:13.228 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.228 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.228 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.228 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:48:13.228 INFO:teuthology.orchestra.run.gibba001.stdout: "osds": [ 2021-02-10T11:48:13.228 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.229 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:48:13.229 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5", 2021-02-10T11:48:13.229 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:48:13.229 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:48:13.229 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:48:13.229 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:48:13.229 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:48:13.229 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:48:13.229 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:48:13.229 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 181, 2021-02-10T11:48:13.229 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:48:13.229 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:48:13.229 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:48:13.230 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.230 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.230 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.230 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6802", 2021-02-10T11:48:13.230 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:48:13.230 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.230 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.230 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.230 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6805", 2021-02-10T11:48:13.230 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:48:13.230 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.230 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.230 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.231 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:48:13.231 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.231 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.231 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.231 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6807", 2021-02-10T11:48:13.231 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:48:13.231 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.231 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.231 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.231 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6809", 2021-02-10T11:48:13.231 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:48:13.231 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.231 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.231 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.232 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:48:13.232 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.232 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.232 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.232 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6817", 2021-02-10T11:48:13.232 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:48:13.232 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.232 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.232 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.232 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6820", 2021-02-10T11:48:13.232 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:48:13.232 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.232 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.232 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.233 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:48:13.233 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.233 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.233 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.233 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6812", 2021-02-10T11:48:13.233 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:48:13.233 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.233 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.233 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.233 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6814", 2021-02-10T11:48:13.233 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:48:13.233 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.233 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.234 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.234 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6805/25093", 2021-02-10T11:48:13.234 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6809/25093", 2021-02-10T11:48:13.234 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6820/25093", 2021-02-10T11:48:13.234 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6814/25093", 2021-02-10T11:48:13.234 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:48:13.234 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:48:13.234 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:48:13.234 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.234 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.234 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.234 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:48:13.234 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "ddc04ce3-b313-4707-8270-585391e38ecc", 2021-02-10T11:48:13.235 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:48:13.235 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:48:13.235 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:48:13.235 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:48:13.235 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:48:13.235 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:48:13.235 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:48:13.235 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 181, 2021-02-10T11:48:13.235 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:48:13.235 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:48:13.235 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:48:13.235 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.235 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.235 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.236 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6803", 2021-02-10T11:48:13.236 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:48:13.236 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.236 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.236 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.236 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6804", 2021-02-10T11:48:13.236 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:48:13.236 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.236 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.236 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.236 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:48:13.236 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.236 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.237 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.237 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6806", 2021-02-10T11:48:13.237 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:48:13.237 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.237 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.237 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.237 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6808", 2021-02-10T11:48:13.237 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:48:13.237 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.237 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.237 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.237 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:48:13.237 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.237 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.238 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.238 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6818", 2021-02-10T11:48:13.238 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:48:13.238 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.238 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.238 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.238 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6821", 2021-02-10T11:48:13.238 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:48:13.238 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.238 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.238 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.238 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:48:13.238 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.239 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.239 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.239 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6810", 2021-02-10T11:48:13.239 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:48:13.239 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.239 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.239 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.239 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6815", 2021-02-10T11:48:13.239 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:48:13.239 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.239 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.239 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.239 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6804/25092", 2021-02-10T11:48:13.239 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6808/25092", 2021-02-10T11:48:13.240 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6821/25092", 2021-02-10T11:48:13.240 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6815/25092", 2021-02-10T11:48:13.240 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:48:13.240 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:48:13.240 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:48:13.240 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.240 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.240 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.240 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:48:13.240 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3", 2021-02-10T11:48:13.240 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:48:13.240 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:48:13.240 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:48:13.241 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:48:13.241 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:48:13.241 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:48:13.241 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:48:13.241 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 181, 2021-02-10T11:48:13.241 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:48:13.241 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:48:13.241 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:48:13.241 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.241 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.241 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.241 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6811", 2021-02-10T11:48:13.242 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:48:13.242 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.242 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.242 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.242 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6813", 2021-02-10T11:48:13.242 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:48:13.242 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.242 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.242 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.242 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:48:13.242 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.242 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.243 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.243 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6816", 2021-02-10T11:48:13.243 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:48:13.243 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.243 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.243 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.243 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6819", 2021-02-10T11:48:13.243 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:48:13.243 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.243 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.243 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.243 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:48:13.243 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.244 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.244 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.244 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6825", 2021-02-10T11:48:13.244 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:48:13.244 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.244 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.244 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.244 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6827", 2021-02-10T11:48:13.244 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:48:13.244 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.244 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.244 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.244 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:48:13.244 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.245 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.245 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.245 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6822", 2021-02-10T11:48:13.245 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:48:13.245 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.245 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.245 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.245 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6823", 2021-02-10T11:48:13.245 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:48:13.245 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.245 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.245 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.245 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6813/25094", 2021-02-10T11:48:13.245 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6819/25094", 2021-02-10T11:48:13.246 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6827/25094", 2021-02-10T11:48:13.246 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6823/25094", 2021-02-10T11:48:13.246 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:48:13.246 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:48:13.246 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:48:13.246 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.246 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.246 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.246 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:48:13.246 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "0d9ef9b1-98aa-4d37-aa16-6305c7d46d52", 2021-02-10T11:48:13.246 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:48:13.246 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:48:13.246 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:48:13.247 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:48:13.247 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:48:13.247 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:48:13.247 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:48:13.247 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 181, 2021-02-10T11:48:13.247 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:48:13.247 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:48:13.247 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:48:13.247 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.247 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.247 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.247 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6824", 2021-02-10T11:48:13.247 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:48:13.247 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.248 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.248 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.248 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6826", 2021-02-10T11:48:13.248 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:48:13.248 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.248 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.248 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.248 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:48:13.248 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.248 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.248 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.248 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6828", 2021-02-10T11:48:13.248 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:48:13.249 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.249 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.249 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.249 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6829", 2021-02-10T11:48:13.249 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:48:13.249 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.249 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.249 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.249 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:48:13.249 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.249 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.249 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.249 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6832", 2021-02-10T11:48:13.249 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:48:13.250 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.250 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.250 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.250 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6833", 2021-02-10T11:48:13.250 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:48:13.250 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.250 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.250 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.250 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:48:13.250 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.250 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.250 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.250 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6830", 2021-02-10T11:48:13.251 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:48:13.251 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.251 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.251 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.251 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6831", 2021-02-10T11:48:13.251 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:48:13.251 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.251 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.251 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.251 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6826/25095", 2021-02-10T11:48:13.251 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6829/25095", 2021-02-10T11:48:13.251 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6833/25095", 2021-02-10T11:48:13.251 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6831/25095", 2021-02-10T11:48:13.251 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:48:13.252 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:48:13.252 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:48:13.252 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.252 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.252 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.252 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:48:13.252 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "36742c21-bfeb-4e11-bce9-8e745378b605", 2021-02-10T11:48:13.252 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:48:13.252 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:48:13.252 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:48:13.252 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:48:13.252 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:48:13.253 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:48:13.253 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:48:13.253 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 181, 2021-02-10T11:48:13.253 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:48:13.253 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:48:13.253 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:48:13.253 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.253 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.253 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.253 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6816", 2021-02-10T11:48:13.254 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:48:13.254 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.254 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.254 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.254 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6818", 2021-02-10T11:48:13.254 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:48:13.254 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.254 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.254 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.254 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:48:13.254 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.254 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.254 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.254 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6820", 2021-02-10T11:48:13.255 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:48:13.255 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.255 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.255 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.255 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6822", 2021-02-10T11:48:13.255 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:48:13.255 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.255 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.255 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.255 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:48:13.255 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.255 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.256 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.256 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6828", 2021-02-10T11:48:13.256 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:48:13.256 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.256 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.256 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.256 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6830", 2021-02-10T11:48:13.256 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:48:13.256 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.256 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.257 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.257 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:48:13.257 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.257 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.257 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.257 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6824", 2021-02-10T11:48:13.257 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:48:13.257 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.257 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.257 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.257 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6826", 2021-02-10T11:48:13.257 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:48:13.257 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.258 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.258 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.258 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6818/36380", 2021-02-10T11:48:13.258 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6822/36380", 2021-02-10T11:48:13.258 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6830/36380", 2021-02-10T11:48:13.258 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6826/36380", 2021-02-10T11:48:13.258 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:48:13.258 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:48:13.258 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:48:13.258 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.258 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.258 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.258 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:48:13.259 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "88e694e8-fe98-4d1b-88c8-8c9907ece60b", 2021-02-10T11:48:13.259 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:48:13.259 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:48:13.259 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:48:13.259 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:48:13.259 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:48:13.259 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:48:13.259 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:48:13.259 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 181, 2021-02-10T11:48:13.259 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:48:13.259 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:48:13.259 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:48:13.259 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.259 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.260 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.260 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6808", 2021-02-10T11:48:13.260 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:48:13.260 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.260 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.260 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.260 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6809", 2021-02-10T11:48:13.260 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:48:13.260 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.260 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.260 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.260 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:48:13.260 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.261 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.261 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.261 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6810", 2021-02-10T11:48:13.261 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:48:13.261 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.261 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.261 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.261 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6811", 2021-02-10T11:48:13.261 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:48:13.261 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.261 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.261 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.261 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:48:13.262 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.262 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.262 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.262 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6814", 2021-02-10T11:48:13.262 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:48:13.262 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.262 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.262 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.262 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6815", 2021-02-10T11:48:13.262 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:48:13.263 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.263 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.263 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.263 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:48:13.263 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.264 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.264 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.264 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6812", 2021-02-10T11:48:13.264 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:48:13.264 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.264 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.264 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.264 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6813", 2021-02-10T11:48:13.264 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:48:13.264 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.264 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.264 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.265 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6809/36383", 2021-02-10T11:48:13.265 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6811/36383", 2021-02-10T11:48:13.265 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6815/36383", 2021-02-10T11:48:13.265 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6813/36383", 2021-02-10T11:48:13.265 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:48:13.265 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:48:13.265 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:48:13.265 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.265 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.265 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.265 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:48:13.265 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "87a13c6c-cf85-4aaf-916b-e73913484a49", 2021-02-10T11:48:13.266 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:48:13.266 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:48:13.266 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:48:13.266 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:48:13.266 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:48:13.266 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:48:13.266 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:48:13.266 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 180, 2021-02-10T11:48:13.266 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:48:13.266 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:48:13.266 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:48:13.266 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.266 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.267 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.267 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6800", 2021-02-10T11:48:13.267 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:48:13.267 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.267 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.267 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.267 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6801", 2021-02-10T11:48:13.267 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:48:13.267 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.267 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.267 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.267 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:48:13.267 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.268 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.268 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.268 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6802", 2021-02-10T11:48:13.268 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:48:13.268 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.268 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.268 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.268 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6803", 2021-02-10T11:48:13.268 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:48:13.268 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.268 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.268 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.268 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:48:13.269 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.269 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.269 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.269 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6806", 2021-02-10T11:48:13.269 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:48:13.269 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.269 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.269 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.269 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6807", 2021-02-10T11:48:13.269 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:48:13.270 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.270 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.270 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.270 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:48:13.270 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.270 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.270 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.270 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6804", 2021-02-10T11:48:13.270 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:48:13.270 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.271 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.271 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.271 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6805", 2021-02-10T11:48:13.271 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:48:13.271 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.271 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.271 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.271 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6801/36381", 2021-02-10T11:48:13.271 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6803/36381", 2021-02-10T11:48:13.271 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6807/36381", 2021-02-10T11:48:13.271 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6805/36381", 2021-02-10T11:48:13.271 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:48:13.272 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:48:13.272 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:48:13.272 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.272 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.272 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.272 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:48:13.272 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "7b90823f-854c-4592-a95b-a7fa98f93254", 2021-02-10T11:48:13.272 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:48:13.272 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:48:13.272 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:48:13.272 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:48:13.272 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:48:13.272 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:48:13.273 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:48:13.273 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 181, 2021-02-10T11:48:13.273 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:48:13.273 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:48:13.273 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:48:13.273 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.273 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.273 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.273 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6817", 2021-02-10T11:48:13.273 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:48:13.273 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.273 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.273 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.274 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6819", 2021-02-10T11:48:13.274 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:48:13.274 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.274 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.274 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.274 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:48:13.274 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.274 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.274 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.274 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6821", 2021-02-10T11:48:13.274 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:48:13.274 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.274 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.275 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.275 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6823", 2021-02-10T11:48:13.275 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:48:13.275 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.275 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.275 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.275 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:48:13.275 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.275 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.275 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.275 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6829", 2021-02-10T11:48:13.275 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:48:13.276 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.276 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.276 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.276 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6831", 2021-02-10T11:48:13.276 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:48:13.276 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.276 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.276 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.276 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:48:13.276 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:48:13.276 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.276 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:48:13.276 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6825", 2021-02-10T11:48:13.277 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:48:13.277 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.277 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.277 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:13.277 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6827", 2021-02-10T11:48:13.277 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:48:13.277 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.277 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.277 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.277 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6819/36382", 2021-02-10T11:48:13.277 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6823/36382", 2021-02-10T11:48:13.277 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6831/36382", 2021-02-10T11:48:13.277 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6827/36382", 2021-02-10T11:48:13.278 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:48:13.278 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:48:13.278 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:48:13.278 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:13.278 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.278 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:48:13.278 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_xinfo": [ 2021-02-10T11:48:13.278 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.278 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:48:13.278 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:48:13.278 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:48:13.278 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:48:13.279 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:48:13.279 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:48:13.279 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.279 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.279 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:48:13.279 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:48:13.279 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:48:13.279 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:48:13.279 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:48:13.279 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:48:13.279 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.279 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.280 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:48:13.280 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:48:13.280 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:48:13.280 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:48:13.280 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:48:13.280 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:48:13.280 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.280 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.280 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:48:13.280 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:48:13.280 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:48:13.280 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:48:13.281 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:48:13.281 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:48:13.281 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.281 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.281 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:48:13.281 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:48:13.281 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:48:13.281 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:48:13.281 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:48:13.282 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:48:13.282 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.282 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.282 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:48:13.282 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:48:13.283 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:48:13.283 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:48:13.283 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:48:13.283 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:48:13.283 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.283 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.283 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:48:13.283 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:48:13.283 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:48:13.283 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:48:13.283 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:48:13.283 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:48:13.284 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.284 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:13.284 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:48:13.284 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:48:13.284 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:48:13.284 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:48:13.284 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:48:13.284 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:48:13.284 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.284 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:48:13.284 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap": [], 2021-02-10T11:48:13.284 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap_items": [], 2021-02-10T11:48:13.284 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_temp": [], 2021-02-10T11:48:13.285 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_temp": [], 2021-02-10T11:48:13.285 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklist": {}, 2021-02-10T11:48:13.285 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profiles": { 2021-02-10T11:48:13.285 INFO:teuthology.orchestra.run.gibba001.stdout: "default": { 2021-02-10T11:48:13.285 INFO:teuthology.orchestra.run.gibba001.stdout: "crush-failure-domain": "osd", 2021-02-10T11:48:13.285 INFO:teuthology.orchestra.run.gibba001.stdout: "k": "2", 2021-02-10T11:48:13.285 INFO:teuthology.orchestra.run.gibba001.stdout: "m": "1", 2021-02-10T11:48:13.285 INFO:teuthology.orchestra.run.gibba001.stdout: "plugin": "jerasure", 2021-02-10T11:48:13.285 INFO:teuthology.orchestra.run.gibba001.stdout: "ruleset-failure-domain": "osd", 2021-02-10T11:48:13.285 INFO:teuthology.orchestra.run.gibba001.stdout: "technique": "reed_sol_van" 2021-02-10T11:48:13.285 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:13.285 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:13.286 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps_queue": [], 2021-02-10T11:48:13.286 INFO:teuthology.orchestra.run.gibba001.stdout: "new_removed_snaps": [], 2021-02-10T11:48:13.286 INFO:teuthology.orchestra.run.gibba001.stdout: "new_purged_snaps": [], 2021-02-10T11:48:13.286 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_node_flags": {}, 2021-02-10T11:48:13.286 INFO:teuthology.orchestra.run.gibba001.stdout: "device_class_flags": {} 2021-02-10T11:48:13.286 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:48:13.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.214 7f1c00ea5700 1 -- 172.21.2.101:0/2164452362 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1be00186c0 msgr2=0x7f1be001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:13.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.214 7f1c00ea5700 1 --2- 172.21.2.101:0/2164452362 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1be00186c0 0x7f1be001abc0 crc :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:13.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.214 7f1c00ea5700 1 -- 172.21.2.101:0/2164452362 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1bfc19da20 msgr2=0x7f1bfc19de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:13.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.214 7f1c00ea5700 1 --2- 172.21.2.101:0/2164452362 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1bfc19da20 0x7f1bfc19de20 secure :-1 s=READY pgs=3191 cs=0 l=1 rev1=1 rx=0x7f1bf4009980 tx=0x7f1bf400bcd0).stop 2021-02-10T11:48:13.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.214 7f1c00ea5700 1 -- 172.21.2.101:0/2164452362 shutdown_connections 2021-02-10T11:48:13.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.214 7f1c00ea5700 1 --2- 172.21.2.101:0/2164452362 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1bfc19da20 0x7f1bfc19de20 unknown :-1 s=CLOSED pgs=3191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:13.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.214 7f1c00ea5700 1 --2- 172.21.2.101:0/2164452362 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1bfc1b80f0 0x7f1bfc1b84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:13.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.214 7f1c00ea5700 1 --2- 172.21.2.101:0/2164452362 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1be00186c0 0x7f1be001abc0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:13.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.214 7f1c00ea5700 1 --2- 172.21.2.101:0/2164452362 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1bfc1b7cf0 0x7f1bfc19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:13.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.214 7f1c00ea5700 1 -- 172.21.2.101:0/2164452362 >> 172.21.2.101:0/2164452362 conn(0x7f1bfc0ed810 msgr2=0x7f1bfc094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:13.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.215 7f1c00ea5700 1 -- 172.21.2.101:0/2164452362 shutdown_connections 2021-02-10T11:48:13.288 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.215 7f1c00ea5700 1 -- 172.21.2.101:0/2164452362 wait complete. 2021-02-10T11:48:13.289 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:48:13.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.707 7fd2be474700 1 Processor -- start 2021-02-10T11:48:13.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.707 7fd2be474700 1 -- start start 2021-02-10T11:48:13.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.707 7fd2be474700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd2b80d1870 0x7fd2b81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:13.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.707 7fd2be474700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2b81b0a10 0x7fd2b80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:13.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.707 7fd2be474700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b80f0b10 con 0x7fd2b80d1c70 2021-02-10T11:48:13.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.707 7fd2be474700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b80e5ed0 con 0x7fd2b80d1870 2021-02-10T11:48:13.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.707 7fd2be474700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b80f0650 con 0x7fd2b81b0a10 2021-02-10T11:48:13.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.708 7fd2bca91700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd2b80d1c70 0x7fd2b80cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35164/0 (socket says 172.21.2.101:35164) 2021-02-10T11:48:13.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.708 7fd2bca91700 1 -- 172.21.2.101:0/1863675665 learned_addr learned my addr 172.21.2.101:0/1863675665 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:13.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.708 7fd2b7fff700 1 --2- 172.21.2.101:0/1863675665 >> v2:172.21.2.101:3300/0 conn(0x7fd2b80d1870 0x7fd2b81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:13.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.708 7fd2b77fe700 1 --2- 172.21.2.101:0/1863675665 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2b81b0a10 0x7fd2b80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:13.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.708 7fd2b7fff700 1 -- 172.21.2.101:0/1863675665 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2b81b0a10 msgr2=0x7fd2b80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:13.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.708 7fd2b7fff700 1 --2- 172.21.2.101:0/1863675665 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2b81b0a10 0x7fd2b80cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:13.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.708 7fd2b7fff700 1 -- 172.21.2.101:0/1863675665 >> v1:172.21.2.101:6789/0 conn(0x7fd2b80d1c70 legacy=0x7fd2b80cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:13.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.708 7fd2b7fff700 1 -- 172.21.2.101:0/1863675665 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b80f0870 con 0x7fd2b80d1870 2021-02-10T11:48:13.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.708 7fd2b7fff700 1 --2- 172.21.2.101:0/1863675665 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2b80d1870 0x7fd2b81b03f0 secure :-1 s=READY pgs=3193 cs=0 l=1 rev1=1 rx=0x7fd2a8006650 tx=0x7fd2a8009a80).ready entity=mon.0 client_cookie=87bde0d966f4a227 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:13.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.708 7fd2b6ffd700 1 -- 172.21.2.101:0/1863675665 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2a800c0c0 con 0x7fd2b80d1870 2021-02-10T11:48:13.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.708 7fd2b6ffd700 1 -- 172.21.2.101:0/1863675665 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a800c340 con 0x7fd2b80d1870 2021-02-10T11:48:13.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.708 7fd2b6ffd700 1 -- 172.21.2.101:0/1863675665 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2a800c860 con 0x7fd2b80d1870 2021-02-10T11:48:13.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.708 7fd2be474700 1 -- 172.21.2.101:0/1863675665 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2b80d1870 msgr2=0x7fd2b81b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:13.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.708 7fd2be474700 1 --2- 172.21.2.101:0/1863675665 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2b80d1870 0x7fd2b81b03f0 secure :-1 s=READY pgs=3193 cs=0 l=1 rev1=1 rx=0x7fd2a8006650 tx=0x7fd2a8009a80).stop 2021-02-10T11:48:13.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.709 7fd2be474700 1 -- 172.21.2.101:0/1863675665 shutdown_connections 2021-02-10T11:48:13.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.709 7fd2be474700 1 --2- 172.21.2.101:0/1863675665 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2b81b0a10 0x7fd2b80cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:13.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.709 7fd2be474700 1 --2- 172.21.2.101:0/1863675665 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2b80d1870 0x7fd2b81b03f0 unknown :-1 s=CLOSED pgs=3193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:13.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.709 7fd2be474700 1 -- 172.21.2.101:0/1863675665 >> 172.21.2.101:0/1863675665 conn(0x7fd2b80ce450 msgr2=0x7fd2b80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:13.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.709 7fd2be474700 1 -- 172.21.2.101:0/1863675665 shutdown_connections 2021-02-10T11:48:13.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.709 7fd2be474700 1 -- 172.21.2.101:0/1863675665 wait complete. 2021-02-10T11:48:13.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.724 7fd2be474700 1 Processor -- start 2021-02-10T11:48:13.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.724 7fd2be474700 1 -- start start 2021-02-10T11:48:13.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.724 7fd2be474700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2b81942b0 0x7fd2b80b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:13.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.724 7fd2be474700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2b8195070 0x7fd2b8195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:13.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.724 7fd2be474700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2b80bb8e0 0x7fd2b80bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:13.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.724 7fd2be474700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b81946b0 con 0x7fd2b81942b0 2021-02-10T11:48:13.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.724 7fd2be474700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b80b3b70 con 0x7fd2b8195070 2021-02-10T11:48:13.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.724 7fd2be474700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b80b3d90 con 0x7fd2b80bb8e0 2021-02-10T11:48:13.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.725 7fd2b7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2b81942b0 0x7fd2b80b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:13.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.725 7fd2b7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2b81942b0 0x7fd2b80b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34674/0 (socket says 172.21.2.101:34674) 2021-02-10T11:48:13.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.725 7fd2b7fff700 1 -- 172.21.2.101:0/3747287441 learned_addr learned my addr 172.21.2.101:0/3747287441 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:13.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.725 7fd2b7fff700 1 -- 172.21.2.101:0/3747287441 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2b80bb8e0 msgr2=0x7fd2b80bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:13.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.725 7fd2b7fff700 1 --2- 172.21.2.101:0/3747287441 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2b80bb8e0 0x7fd2b80bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:13.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.725 7fd2b7fff700 1 -- 172.21.2.101:0/3747287441 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2b8195070 msgr2=0x7fd2b8195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:13.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.725 7fd2b7fff700 1 --2- 172.21.2.101:0/3747287441 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2b8195070 0x7fd2b8195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:13.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.725 7fd2b7fff700 1 -- 172.21.2.101:0/3747287441 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b81a49e0 con 0x7fd2b81942b0 2021-02-10T11:48:13.725 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.725 7fd2b7fff700 1 --2- 172.21.2.101:0/3747287441 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2b81942b0 0x7fd2b80b3550 secure :-1 s=READY pgs=3194 cs=0 l=1 rev1=1 rx=0x7fd2a8009630 tx=0x7fd2a800b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:13.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.725 7fd2b4ff9700 1 -- 172.21.2.101:0/3747287441 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2a800bd70 con 0x7fd2b81942b0 2021-02-10T11:48:13.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.725 7fd2b4ff9700 1 -- 172.21.2.101:0/3747287441 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a8000960 con 0x7fd2b81942b0 2021-02-10T11:48:13.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.725 7fd2b4ff9700 1 -- 172.21.2.101:0/3747287441 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2a8005560 con 0x7fd2b81942b0 2021-02-10T11:48:13.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.725 7fd2be474700 1 -- 172.21.2.101:0/3747287441 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b81a49e0 con 0x7fd2b81942b0 2021-02-10T11:48:13.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.725 7fd2be474700 1 -- 172.21.2.101:0/3747287441 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b80bc5f0 con 0x7fd2b81942b0 2021-02-10T11:48:13.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.726 7fd2b4ff9700 1 -- 172.21.2.101:0/3747287441 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd2a800a000 con 0x7fd2b81942b0 2021-02-10T11:48:13.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.726 7fd2b4ff9700 1 --2- 172.21.2.101:0/3747287441 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd298018730 0x7fd29801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:13.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.726 7fd2b4ff9700 1 -- 172.21.2.101:0/3747287441 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd2a8032690 con 0x7fd2b81942b0 2021-02-10T11:48:13.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.726 7fd2b77fe700 1 --2- 172.21.2.101:0/3747287441 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd298018730 0x7fd29801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:13.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.726 7fd2b77fe700 1 --2- 172.21.2.101:0/3747287441 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd298018730 0x7fd29801abc0 crc :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:13.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.757 7fd2be474700 1 -- 172.21.2.101:0/3747287441 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b80bc5f0 con 0x7fd2b81942b0 2021-02-10T11:48:13.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.760 7fd2b4ff9700 1 -- 172.21.2.101:0/3747287441 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd2a8000940 con 0x7fd2b81942b0 2021-02-10T11:48:13.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.888 7fd2be474700 1 -- 172.21.2.101:0/3747287441 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2b80bc5f0 con 0x7fd2b81942b0 2021-02-10T11:48:13.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.889 7fd2b4ff9700 1 -- 172.21.2.101:0/3747287441 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) v1 ==== 95+0+2108 (secure 0 0 0) 0x7fd2a8000940 con 0x7fd2b81942b0 2021-02-10T11:48:13.886 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:48:13.886 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":157,"default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":13128,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/122652046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":122652046},{"type":"v1","addr":"172.21.2.101:6835","nonce":122652046}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":152}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:47:28.786578","modified":"2021-02-10 11:47:30.795250","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":13125},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13125":{"gid":13125,"name":"b","rank":0,"incarnation":153,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/2314483030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":2314483030},{"type":"v1","addr":"172.21.2.108:6833","nonce":2314483030}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[39],"metadata_pool":38,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":19}]} 2021-02-10T11:48:13.886 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 157 2021-02-10T11:48:13.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.890 7fd2be474700 1 -- 172.21.2.101:0/3747287441 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd298018730 msgr2=0x7fd29801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:13.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.890 7fd2be474700 1 --2- 172.21.2.101:0/3747287441 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd298018730 0x7fd29801abc0 crc :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:13.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.890 7fd2be474700 1 -- 172.21.2.101:0/3747287441 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2b81942b0 msgr2=0x7fd2b80b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:13.887 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.890 7fd2be474700 1 --2- 172.21.2.101:0/3747287441 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2b81942b0 0x7fd2b80b3550 secure :-1 s=READY pgs=3194 cs=0 l=1 rev1=1 rx=0x7fd2a8009630 tx=0x7fd2a800b980).stop 2021-02-10T11:48:13.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.891 7fd2be474700 1 -- 172.21.2.101:0/3747287441 shutdown_connections 2021-02-10T11:48:13.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.891 7fd2be474700 1 --2- 172.21.2.101:0/3747287441 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd298018730 0x7fd29801abc0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:13.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.891 7fd2be474700 1 --2- 172.21.2.101:0/3747287441 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd2b80bb8e0 0x7fd2b80bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:13.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.891 7fd2be474700 1 --2- 172.21.2.101:0/3747287441 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd2b8195070 0x7fd2b8195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:13.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.891 7fd2be474700 1 --2- 172.21.2.101:0/3747287441 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd2b81942b0 0x7fd2b80b3550 unknown :-1 s=CLOSED pgs=3194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:13.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.891 7fd2be474700 1 -- 172.21.2.101:0/3747287441 >> 172.21.2.101:0/3747287441 conn(0x7fd2b80b3040 msgr2=0x7fd2b80c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:13.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.891 7fd2be474700 1 -- 172.21.2.101:0/3747287441 shutdown_connections 2021-02-10T11:48:13.888 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:13.891 7fd2be474700 1 -- 172.21.2.101:0/3747287441 wait complete. 2021-02-10T11:48:13.896 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs fail cephfs 2021-02-10T11:48:14.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.312 7f4c18d3b700 1 Processor -- start 2021-02-10T11:48:14.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.312 7f4c18d3b700 1 -- start start 2021-02-10T11:48:14.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.312 7f4c18d3b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4c140d1870 0x7f4c141b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:14.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.313 7f4c18d3b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c140f0650 0x7f4c140cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:14.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.313 7f4c18d3b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c140f0b10 con 0x7f4c141b0a10 2021-02-10T11:48:14.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.313 7f4c18d3b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c140e5ed0 con 0x7f4c140d1870 2021-02-10T11:48:14.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.313 7f4c18d3b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c140d1c70 con 0x7f4c140f0650 2021-02-10T11:48:14.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.314 7f4c13314700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c140f0650 0x7f4c140cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:14.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.314 7f4c12b13700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4c140d1870 0x7f4c141b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:14.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.314 7f4c13314700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c140f0650 0x7f4c140cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:55650/0 (socket says 172.21.2.101:55650) 2021-02-10T11:48:14.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.314 7f4c12b13700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4c140d1870 0x7f4c141b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:45168/0 (socket says 172.21.2.101:45168) 2021-02-10T11:48:14.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.314 7f4c13314700 1 -- 172.21.2.101:0/3683062702 learned_addr learned my addr 172.21.2.101:0/3683062702 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:14.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.314 7f4c12b13700 1 -- 172.21.2.101:0/3683062702 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c140f0650 msgr2=0x7f4c140cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:14.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.314 7f4c12b13700 1 --2- 172.21.2.101:0/3683062702 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c140f0650 0x7f4c140cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:14.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.314 7f4c12b13700 1 -- 172.21.2.101:0/3683062702 >> v1:172.21.2.108:6789/0 conn(0x7f4c141b0a10 legacy=0x7f4c140cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:14.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.314 7f4c11b11700 1 -- 172.21.2.101:0/3683062702 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915334851 0 0) 0x7f4c04000ff0 con 0x7f4c141b0a10 2021-02-10T11:48:14.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.314 7f4c12b13700 1 -- 172.21.2.101:0/3683062702 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c140d1e90 con 0x7f4c140d1870 2021-02-10T11:48:14.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.314 7f4c13314700 1 --2- 172.21.2.101:0/3683062702 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c140f0650 0x7f4c140cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:48:14.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.314 7f4c12b13700 1 --2- 172.21.2.101:0/3683062702 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c140d1870 0x7f4c141b03f0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7f4bfc006650 tx=0x7f4bfc009a80).ready entity=mon.1 client_cookie=e65ad41b90cf20df server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:14.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.315 7f4c11b11700 1 -- 172.21.2.101:0/3683062702 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4bfc0009b0 con 0x7f4c140d1870 2021-02-10T11:48:14.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.315 7f4c11b11700 1 -- 172.21.2.101:0/3683062702 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bfc00c0c0 con 0x7f4c140d1870 2021-02-10T11:48:14.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.315 7f4c18d3b700 1 -- 172.21.2.101:0/3683062702 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c140d1870 msgr2=0x7f4c141b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:14.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.315 7f4c18d3b700 1 --2- 172.21.2.101:0/3683062702 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c140d1870 0x7f4c141b03f0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7f4bfc006650 tx=0x7f4bfc009a80).stop 2021-02-10T11:48:14.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.315 7f4c11b11700 1 -- 172.21.2.101:0/3683062702 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4bfc0009b0 con 0x7f4c140d1870 2021-02-10T11:48:14.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.315 7f4c18d3b700 1 -- 172.21.2.101:0/3683062702 shutdown_connections 2021-02-10T11:48:14.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.315 7f4c18d3b700 1 --2- 172.21.2.101:0/3683062702 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c140f0650 0x7f4c140cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:14.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.315 7f4c18d3b700 1 --2- 172.21.2.101:0/3683062702 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c140d1870 0x7f4c141b03f0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:14.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.315 7f4c18d3b700 1 -- 172.21.2.101:0/3683062702 >> 172.21.2.101:0/3683062702 conn(0x7f4c140ce450 msgr2=0x7f4c140d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:14.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.315 7f4c18d3b700 1 -- 172.21.2.101:0/3683062702 shutdown_connections 2021-02-10T11:48:14.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.315 7f4c18d3b700 1 -- 172.21.2.101:0/3683062702 wait complete. 2021-02-10T11:48:14.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.330 7f4c18d3b700 1 Processor -- start 2021-02-10T11:48:14.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.330 7f4c18d3b700 1 -- start start 2021-02-10T11:48:14.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.331 7f4c18d3b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c141a4990 0x7f4c140b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:14.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.331 7f4c18d3b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c140b2b50 0x7f4c1409b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:14.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.331 7f4c18d3b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c1409bfb0 0x7f4c14189aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:14.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.331 7f4c18d3b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1409c3b0 con 0x7f4c1409bfb0 2021-02-10T11:48:14.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.331 7f4c18d3b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1418a0c0 con 0x7f4c140b2b50 2021-02-10T11:48:14.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.331 7f4c18d3b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1418a2e0 con 0x7f4c141a4990 2021-02-10T11:48:14.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.331 7f4c13314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c1409bfb0 0x7f4c14189aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:14.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.331 7f4c13314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c1409bfb0 0x7f4c14189aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34690/0 (socket says 172.21.2.101:34690) 2021-02-10T11:48:14.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.331 7f4c13314700 1 -- 172.21.2.101:0/2882789967 learned_addr learned my addr 172.21.2.101:0/2882789967 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:14.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.331 7f4c12b13700 1 --2- 172.21.2.101:0/2882789967 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c141a4990 0x7f4c140b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:14.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.331 7f4c12312700 1 --2- 172.21.2.101:0/2882789967 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c140b2b50 0x7f4c1409b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:14.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.331 7f4c13314700 1 -- 172.21.2.101:0/2882789967 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c141a4990 msgr2=0x7f4c140b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:14.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.331 7f4c13314700 1 --2- 172.21.2.101:0/2882789967 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c141a4990 0x7f4c140b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:14.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.331 7f4c13314700 1 -- 172.21.2.101:0/2882789967 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c140b2b50 msgr2=0x7f4c1409b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:14.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.331 7f4c13314700 1 --2- 172.21.2.101:0/2882789967 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c140b2b50 0x7f4c1409b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:14.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.331 7f4c13314700 1 -- 172.21.2.101:0/2882789967 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1418a500 con 0x7f4c1409bfb0 2021-02-10T11:48:14.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.332 7f4c13314700 1 --2- 172.21.2.101:0/2882789967 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c1409bfb0 0x7f4c14189aa0 secure :-1 s=READY pgs=3195 cs=0 l=1 rev1=1 rx=0x7f4c0c009940 tx=0x7f4c0c00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:14.332 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.332 7f4c0b7fe700 1 -- 172.21.2.101:0/2882789967 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4c0c000f90 con 0x7f4c1409bfb0 2021-02-10T11:48:14.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.332 7f4c0b7fe700 1 -- 172.21.2.101:0/2882789967 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0c006790 con 0x7f4c1409bfb0 2021-02-10T11:48:14.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.332 7f4c0b7fe700 1 -- 172.21.2.101:0/2882789967 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4c0c00c4e0 con 0x7f4c1409bfb0 2021-02-10T11:48:14.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.332 7f4c18d3b700 1 -- 172.21.2.101:0/2882789967 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1418a500 con 0x7f4c1409bfb0 2021-02-10T11:48:14.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.332 7f4c18d3b700 1 -- 172.21.2.101:0/2882789967 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1418a500 con 0x7f4c1409bfb0 2021-02-10T11:48:14.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.333 7f4c0b7fe700 1 -- 172.21.2.101:0/2882789967 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4c0c0011f0 con 0x7f4c1409bfb0 2021-02-10T11:48:14.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.333 7f4c0b7fe700 1 --2- 172.21.2.101:0/2882789967 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bf8018730 0x7f4bf801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:14.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.333 7f4c0b7fe700 1 -- 172.21.2.101:0/2882789967 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4c0c0329c0 con 0x7f4c1409bfb0 2021-02-10T11:48:14.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.333 7f4c12b13700 1 --2- 172.21.2.101:0/2882789967 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bf8018730 0x7f4bf801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:14.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.333 7f4c12b13700 1 --2- 172.21.2.101:0/2882789967 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bf8018730 0x7f4bf801abc0 crc :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:14.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.363 7f4c18d3b700 1 -- 172.21.2.101:0/2882789967 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c140ebc90 con 0x7f4c1409bfb0 2021-02-10T11:48:14.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.365 7f4c0b7fe700 1 -- 172.21.2.101:0/2882789967 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4c0c006770 con 0x7f4c1409bfb0 2021-02-10T11:48:14.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.492 7f4c18d3b700 1 -- 172.21.2.101:0/2882789967 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4c140ebc90 con 0x7f4c1409bfb0 2021-02-10T11:48:14.653 INFO:tasks.ceph.mon.a.gibba001.stderr:2021-02-10 11:48:14.657 7f974d04d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-02-10T11:48:14.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.663 7f4c0b7fe700 1 -- 172.21.2.101:0/2882789967 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v158) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4c0c006770 con 0x7f4c1409bfb0 2021-02-10T11:48:14.703 INFO:teuthology.orchestra.run.gibba001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-02-10T11:48:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.709 7f4c18d3b700 1 -- 172.21.2.101:0/2882789967 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bf8018730 msgr2=0x7f4bf801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.709 7f4c18d3b700 1 --2- 172.21.2.101:0/2882789967 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bf8018730 0x7f4bf801abc0 crc :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.709 7f4c18d3b700 1 -- 172.21.2.101:0/2882789967 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c1409bfb0 msgr2=0x7f4c14189aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.709 7f4c18d3b700 1 --2- 172.21.2.101:0/2882789967 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c1409bfb0 0x7f4c14189aa0 secure :-1 s=READY pgs=3195 cs=0 l=1 rev1=1 rx=0x7f4c0c009940 tx=0x7f4c0c00bcf0).stop 2021-02-10T11:48:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.709 7f4c18d3b700 1 -- 172.21.2.101:0/2882789967 shutdown_connections 2021-02-10T11:48:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.709 7f4c18d3b700 1 --2- 172.21.2.101:0/2882789967 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c1409bfb0 0x7f4c14189aa0 unknown :-1 s=CLOSED pgs=3195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.709 7f4c18d3b700 1 --2- 172.21.2.101:0/2882789967 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c140b2b50 0x7f4c1409b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.709 7f4c18d3b700 1 --2- 172.21.2.101:0/2882789967 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4bf8018730 0x7f4bf801abc0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.709 7f4c18d3b700 1 --2- 172.21.2.101:0/2882789967 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c141a4990 0x7f4c140b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.709 7f4c18d3b700 1 -- 172.21.2.101:0/2882789967 >> 172.21.2.101:0/2882789967 conn(0x7f4c140a6870 msgr2=0x7f4c1408a420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:14.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.709 7f4c18d3b700 1 -- 172.21.2.101:0/2882789967 shutdown_connections 2021-02-10T11:48:14.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:14.709 7f4c18d3b700 1 -- 172.21.2.101:0/2882789967 wait complete. 2021-02-10T11:48:14.711 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:48:14.714 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:48:15.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.130 7f85c498b700 1 Processor -- start 2021-02-10T11:48:15.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.130 7f85c498b700 1 -- start start 2021-02-10T11:48:15.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.130 7f85c498b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f85c00d1870 0x7f85c01b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:15.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.130 7f85c498b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c00f0b10 con 0x7f85c00f0650 2021-02-10T11:48:15.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.130 7f85c498b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c00d0310 con 0x7f85c01b0a10 2021-02-10T11:48:15.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.130 7f85c498b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c00d0530 con 0x7f85c00d1870 2021-02-10T11:48:15.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.130 7f85be61c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f85c00d1870 0x7f85c01b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:15.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.131 7f85be61c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f85c00d1870 0x7f85c01b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34696/0 (socket says 172.21.2.101:34696) 2021-02-10T11:48:15.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.131 7f85be61c700 1 -- 172.21.2.101:0/4078505951 learned_addr learned my addr 172.21.2.101:0/4078505951 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:15.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.131 7f85bd61a700 1 -- 172.21.2.101:0/4078505951 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080228492 0 0) 0x7f85b8000f90 con 0x7f85c00f0650 2021-02-10T11:48:15.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.131 7f85bd61a700 1 -- 172.21.2.101:0/4078505951 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85ac002fe0 con 0x7f85c00f0650 2021-02-10T11:48:15.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.131 7f85be61c700 1 -- 172.21.2.101:0/4078505951 >> v1:172.21.2.108:6789/0 conn(0x7f85c01b0a10 legacy=0x7f85c00cf330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:48:15.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.131 7f85be61c700 1 -- 172.21.2.101:0/4078505951 >> v1:172.21.2.101:6789/0 conn(0x7f85c00f0650 legacy=0x7f85c00cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:15.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.131 7f85be61c700 1 -- 172.21.2.101:0/4078505951 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c00d1c70 con 0x7f85c00d1870 2021-02-10T11:48:15.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.131 7f85be61c700 1 --2- 172.21.2.101:0/4078505951 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85c00d1870 0x7f85c01b03f0 secure :-1 s=READY pgs=3199 cs=0 l=1 rev1=1 rx=0x7f85b0006700 tx=0x7f85b0009b80).ready entity=mon.0 client_cookie=d9957992626e8b86 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:15.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.131 7f85bd61a700 1 -- 172.21.2.101:0/4078505951 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f85b000c0c0 con 0x7f85c00d1870 2021-02-10T11:48:15.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.131 7f85bd61a700 1 -- 172.21.2.101:0/4078505951 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b000c340 con 0x7f85c00d1870 2021-02-10T11:48:15.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.131 7f85bd61a700 1 -- 172.21.2.101:0/4078505951 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f85b000c860 con 0x7f85c00d1870 2021-02-10T11:48:15.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.131 7f85c498b700 1 -- 172.21.2.101:0/4078505951 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85c00d1870 msgr2=0x7f85c01b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:15.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.131 7f85c498b700 1 --2- 172.21.2.101:0/4078505951 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85c00d1870 0x7f85c01b03f0 secure :-1 s=READY pgs=3199 cs=0 l=1 rev1=1 rx=0x7f85b0006700 tx=0x7f85b0009b80).stop 2021-02-10T11:48:15.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.132 7f85c498b700 1 -- 172.21.2.101:0/4078505951 shutdown_connections 2021-02-10T11:48:15.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.132 7f85c498b700 1 --2- 172.21.2.101:0/4078505951 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85c00d1870 0x7f85c01b03f0 unknown :-1 s=CLOSED pgs=3199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:15.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.132 7f85c498b700 1 -- 172.21.2.101:0/4078505951 >> 172.21.2.101:0/4078505951 conn(0x7f85c00ce450 msgr2=0x7f85c00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:15.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.132 7f85c498b700 1 -- 172.21.2.101:0/4078505951 shutdown_connections 2021-02-10T11:48:15.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.132 7f85c498b700 1 -- 172.21.2.101:0/4078505951 wait complete. 2021-02-10T11:48:15.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.147 7f85c498b700 1 Processor -- start 2021-02-10T11:48:15.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.147 7f85c498b700 1 -- start start 2021-02-10T11:48:15.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.147 7f85c498b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f85c00b0ad0 0x7f85c0091f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:15.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.147 7f85c498b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f85c01a8850 0x7f85c01a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:15.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.147 7f85c498b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85c01a6fa0 0x7f85c01a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:15.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.147 7f85c498b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c01abfe0 con 0x7f85c01a6fa0 2021-02-10T11:48:15.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.147 7f85c498b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c01a9c40 con 0x7f85c01a8850 2021-02-10T11:48:15.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.147 7f85c498b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c01a9730 con 0x7f85c00b0ad0 2021-02-10T11:48:15.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.147 7f85bee1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85c01a6fa0 0x7f85c01a73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:15.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.147 7f85bee1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85c01a6fa0 0x7f85c01a73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34704/0 (socket says 172.21.2.101:34704) 2021-02-10T11:48:15.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.147 7f85bee1d700 1 -- 172.21.2.101:0/1864137917 learned_addr learned my addr 172.21.2.101:0/1864137917 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:15.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.148 7f85bee1d700 1 -- 172.21.2.101:0/1864137917 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f85c00b0ad0 msgr2=0x7f85c0091f30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:15.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.148 7f85bee1d700 1 --2- 172.21.2.101:0/1864137917 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f85c00b0ad0 0x7f85c0091f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:15.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.148 7f85bee1d700 1 -- 172.21.2.101:0/1864137917 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f85c01a8850 msgr2=0x7f85c01a8c50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:15.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.148 7f85bee1d700 1 --2- 172.21.2.101:0/1864137917 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f85c01a8850 0x7f85c01a8c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:15.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.148 7f85bee1d700 1 -- 172.21.2.101:0/1864137917 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c01a9270 con 0x7f85c01a6fa0 2021-02-10T11:48:15.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.148 7f85bee1d700 1 --2- 172.21.2.101:0/1864137917 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85c01a6fa0 0x7f85c01a73a0 secure :-1 s=READY pgs=3200 cs=0 l=1 rev1=1 rx=0x7f85b8009980 tx=0x7f85b800bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:15.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.148 7f85b6ffd700 1 -- 172.21.2.101:0/1864137917 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f85b8005870 con 0x7f85c01a6fa0 2021-02-10T11:48:15.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.148 7f85b6ffd700 1 -- 172.21.2.101:0/1864137917 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b8005b30 con 0x7f85c01a6fa0 2021-02-10T11:48:15.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.148 7f85b6ffd700 1 -- 172.21.2.101:0/1864137917 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f85b8006030 con 0x7f85c01a6fa0 2021-02-10T11:48:15.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.148 7f85c498b700 1 -- 172.21.2.101:0/1864137917 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c01a9270 con 0x7f85c01a6fa0 2021-02-10T11:48:15.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.148 7f85c498b700 1 -- 172.21.2.101:0/1864137917 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c01a7cb0 con 0x7f85c01a6fa0 2021-02-10T11:48:15.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.149 7f85b6ffd700 1 -- 172.21.2.101:0/1864137917 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f85b8005d10 con 0x7f85c01a6fa0 2021-02-10T11:48:15.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.149 7f85b6ffd700 1 --2- 172.21.2.101:0/1864137917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f85a40186c0 0x7f85a401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:15.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.149 7f85b6ffd700 1 -- 172.21.2.101:0/1864137917 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f85b8032c00 con 0x7f85c01a6fa0 2021-02-10T11:48:15.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.149 7f85be61c700 1 --2- 172.21.2.101:0/1864137917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f85a40186c0 0x7f85a401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:15.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.149 7f85be61c700 1 --2- 172.21.2.101:0/1864137917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f85a40186c0 0x7f85a401abc0 crc :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:15.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.180 7f85c498b700 1 -- 172.21.2.101:0/1864137917 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c01a9710 con 0x7f85c01a6fa0 2021-02-10T11:48:15.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.183 7f85b6ffd700 1 -- 172.21.2.101:0/1864137917 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f85b8005e00 con 0x7f85c01a6fa0 2021-02-10T11:48:15.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.310 7f85c498b700 1 -- 172.21.2.101:0/1864137917 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85c01a9710 con 0x7f85c01a6fa0 2021-02-10T11:48:15.307 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.311 7f85b6ffd700 1 -- 172.21.2.101:0/1864137917 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 158 v158) v1 ==== 95+0+1760 (secure 0 0 0) 0x7f85b8006f20 con 0x7f85c01a6fa0 2021-02-10T11:48:15.308 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 158 2021-02-10T11:48:15.308 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:48:15.308 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":158,"default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":13128,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/122652046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":122652046},{"type":"v1","addr":"172.21.2.101:6835","nonce":122652046}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":152}],"filesystems":[{"mdsmap":{"epoch":158,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:47:28.786578","modified":"2021-02-10 11:48:14.658279","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":184,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[39],"metadata_pool":38,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":19}]} 2021-02-10T11:48:15.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.313 7f85c498b700 1 -- 172.21.2.101:0/1864137917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f85a40186c0 msgr2=0x7f85a401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:15.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.313 7f85c498b700 1 --2- 172.21.2.101:0/1864137917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f85a40186c0 0x7f85a401abc0 crc :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:15.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.313 7f85c498b700 1 -- 172.21.2.101:0/1864137917 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85c01a6fa0 msgr2=0x7f85c01a73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:15.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.313 7f85c498b700 1 --2- 172.21.2.101:0/1864137917 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85c01a6fa0 0x7f85c01a73a0 secure :-1 s=READY pgs=3200 cs=0 l=1 rev1=1 rx=0x7f85b8009980 tx=0x7f85b800bcd0).stop 2021-02-10T11:48:15.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.313 7f85c498b700 1 -- 172.21.2.101:0/1864137917 shutdown_connections 2021-02-10T11:48:15.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.313 7f85c498b700 1 --2- 172.21.2.101:0/1864137917 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f85c01a6fa0 0x7f85c01a73a0 unknown :-1 s=CLOSED pgs=3200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:15.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.313 7f85c498b700 1 --2- 172.21.2.101:0/1864137917 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f85c01a8850 0x7f85c01a8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:15.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.313 7f85c498b700 1 --2- 172.21.2.101:0/1864137917 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f85a40186c0 0x7f85a401abc0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:15.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.313 7f85c498b700 1 --2- 172.21.2.101:0/1864137917 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f85c00b0ad0 0x7f85c0091f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:15.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.313 7f85c498b700 1 -- 172.21.2.101:0/1864137917 >> 172.21.2.101:0/1864137917 conn(0x7f85c019f8f0 msgr2=0x7f85c01a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:15.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.313 7f85c498b700 1 -- 172.21.2.101:0/1864137917 shutdown_connections 2021-02-10T11:48:15.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.313 7f85c498b700 1 -- 172.21.2.101:0/1864137917 wait complete. 2021-02-10T11:48:15.318 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-02-10T11:48:15.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.740 7f6cc4345700 1 Processor -- start 2021-02-10T11:48:15.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.740 7f6cc4345700 1 -- start start 2021-02-10T11:48:15.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.741 7f6cc4345700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6cbc0d0340 0x7f6cbc0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:15.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.741 7f6cc4345700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6cbc0d1c70 0x7f6cbc0efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:15.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.741 7f6cc4345700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cbc0f0b10 con 0x7f6cbc0d1870 2021-02-10T11:48:15.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.741 7f6cc4345700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cbc0e5ed0 con 0x7f6cbc0d0340 2021-02-10T11:48:15.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.741 7f6cc4345700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc1b0be0 con 0x7f6cbc0d1c70 2021-02-10T11:48:15.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.742 7f6cc2962700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6cbc0d1c70 0x7f6cbc0efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:15.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.742 7f6cc1960700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6cbc0d0340 0x7f6cbc0ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:15.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.742 7f6cc2161700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f6cbc0d1870 0x7f6cbc1b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:56844/0 (socket says 172.21.2.101:56844) 2021-02-10T11:48:15.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.742 7f6cc2161700 1 -- 172.21.2.101:0/827757116 learned_addr learned my addr 172.21.2.101:0/827757116 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:15.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.743 7f6cc2962700 1 -- 172.21.2.101:0/827757116 >> v2:172.21.2.108:3300/0 conn(0x7f6cbc0d0340 msgr2=0x7f6cbc0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:15.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.743 7f6cc2962700 1 --2- 172.21.2.101:0/827757116 >> v2:172.21.2.108:3300/0 conn(0x7f6cbc0d0340 0x7f6cbc0ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:15.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.743 7f6cc2962700 1 -- 172.21.2.101:0/827757116 >> v1:172.21.2.108:6789/0 conn(0x7f6cbc0d1870 legacy=0x7f6cbc1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:15.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.743 7f6cc2962700 1 -- 172.21.2.101:0/827757116 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cbc0f20a0 con 0x7f6cbc0d1c70 2021-02-10T11:48:15.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.743 7f6cc1960700 1 --2- 172.21.2.101:0/827757116 >> v2:172.21.2.108:3300/0 conn(0x7f6cbc0d0340 0x7f6cbc0ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:48:15.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.743 7f6cc2962700 1 --2- 172.21.2.101:0/827757116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6cbc0d1c70 0x7f6cbc0efc60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f6cb4006e50 tx=0x7f6cb4009e70).ready entity=mon.2 client_cookie=7f05f8bc46042687 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:15.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.743 7f6cc115f700 1 -- 172.21.2.101:0/827757116 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6cb4006aa0 con 0x7f6cbc0d1c70 2021-02-10T11:48:15.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.743 7f6cc115f700 1 -- 172.21.2.101:0/827757116 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb4006790 con 0x7f6cbc0d1c70 2021-02-10T11:48:15.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.743 7f6cc115f700 1 -- 172.21.2.101:0/827757116 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6cb400b700 con 0x7f6cbc0d1c70 2021-02-10T11:48:15.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.743 7f6cc4345700 1 -- 172.21.2.101:0/827757116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6cbc0d1c70 msgr2=0x7f6cbc0efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:15.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.743 7f6cc4345700 1 --2- 172.21.2.101:0/827757116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6cbc0d1c70 0x7f6cbc0efc60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f6cb4006e50 tx=0x7f6cb4009e70).stop 2021-02-10T11:48:15.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.744 7f6cc4345700 1 -- 172.21.2.101:0/827757116 shutdown_connections 2021-02-10T11:48:15.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.744 7f6cc4345700 1 --2- 172.21.2.101:0/827757116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6cbc0d1c70 0x7f6cbc0efc60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:15.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.744 7f6cc4345700 1 --2- 172.21.2.101:0/827757116 >> v2:172.21.2.108:3300/0 conn(0x7f6cbc0d0340 0x7f6cbc0ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:15.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.744 7f6cc4345700 1 -- 172.21.2.101:0/827757116 >> 172.21.2.101:0/827757116 conn(0x7f6cbc0ce450 msgr2=0x7f6cbc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:15.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.744 7f6cc4345700 1 -- 172.21.2.101:0/827757116 shutdown_connections 2021-02-10T11:48:15.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.744 7f6cc4345700 1 -- 172.21.2.101:0/827757116 wait complete. 2021-02-10T11:48:15.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.758 7f6cc4345700 1 Processor -- start 2021-02-10T11:48:15.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.759 7f6cc4345700 1 -- start start 2021-02-10T11:48:15.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.759 7f6cc4345700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cbc0ebe70 0x7f6cbc0ec270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:15.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.759 7f6cc4345700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6cbc093830 0x7f6cbc093c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:15.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.759 7f6cc4345700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6cbc094250 0x7f6cbc094650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:15.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.759 7f6cc4345700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc0ec940 con 0x7f6cbc0ebe70 2021-02-10T11:48:15.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.759 7f6cc4345700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc0ece40 con 0x7f6cbc094250 2021-02-10T11:48:15.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.759 7f6cc4345700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc0ed060 con 0x7f6cbc093830 2021-02-10T11:48:15.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.759 7f6cc2161700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cbc0ebe70 0x7f6cbc0ec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:15.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.759 7f6cc2161700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cbc0ebe70 0x7f6cbc0ec270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34714/0 (socket says 172.21.2.101:34714) 2021-02-10T11:48:15.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.759 7f6cc2161700 1 -- 172.21.2.101:0/3390460438 learned_addr learned my addr 172.21.2.101:0/3390460438 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:15.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.760 7f6cc2161700 1 -- 172.21.2.101:0/3390460438 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6cbc093830 msgr2=0x7f6cbc093c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:15.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.760 7f6cc2161700 1 --2- 172.21.2.101:0/3390460438 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6cbc093830 0x7f6cbc093c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:15.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.760 7f6cc2161700 1 -- 172.21.2.101:0/3390460438 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6cbc094250 msgr2=0x7f6cbc094650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:15.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.760 7f6cc2161700 1 --2- 172.21.2.101:0/3390460438 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6cbc094250 0x7f6cbc094650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:15.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.760 7f6cc2161700 1 -- 172.21.2.101:0/3390460438 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cbc0ed280 con 0x7f6cbc0ebe70 2021-02-10T11:48:15.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.760 7f6cc2161700 1 --2- 172.21.2.101:0/3390460438 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cbc0ebe70 0x7f6cbc0ec270 secure :-1 s=READY pgs=3201 cs=0 l=1 rev1=1 rx=0x7f6cb8001c60 tx=0x7f6cb800a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:15.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.760 7f6caeffd700 1 -- 172.21.2.101:0/3390460438 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6cb800acc0 con 0x7f6cbc0ebe70 2021-02-10T11:48:15.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.760 7f6caeffd700 1 -- 172.21.2.101:0/3390460438 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb8004140 con 0x7f6cbc0ebe70 2021-02-10T11:48:15.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.760 7f6caeffd700 1 -- 172.21.2.101:0/3390460438 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6cb8004620 con 0x7f6cbc0ebe70 2021-02-10T11:48:15.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.760 7f6cc4345700 1 -- 172.21.2.101:0/3390460438 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cbc0ed280 con 0x7f6cbc0ebe70 2021-02-10T11:48:15.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.760 7f6cc4345700 1 -- 172.21.2.101:0/3390460438 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cbc0ed810 con 0x7f6cbc0ebe70 2021-02-10T11:48:15.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.761 7f6caeffd700 1 -- 172.21.2.101:0/3390460438 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6cb8004320 con 0x7f6cbc0ebe70 2021-02-10T11:48:15.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.761 7f6caeffd700 1 --2- 172.21.2.101:0/3390460438 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ca40186c0 0x7f6ca401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:15.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.761 7f6caeffd700 1 -- 172.21.2.101:0/3390460438 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f6cb8032800 con 0x7f6cbc0ebe70 2021-02-10T11:48:15.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.761 7f6cc1960700 1 --2- 172.21.2.101:0/3390460438 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ca40186c0 0x7f6ca401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:15.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.761 7f6cc1960700 1 --2- 172.21.2.101:0/3390460438 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ca40186c0 0x7f6ca401abc0 crc :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:15.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.791 7f6cc4345700 1 -- 172.21.2.101:0/3390460438 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cbc0ebc40 con 0x7f6cbc0ebe70 2021-02-10T11:48:15.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.793 7f6caeffd700 1 -- 172.21.2.101:0/3390460438 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6cb8000dd0 con 0x7f6cbc0ebe70 2021-02-10T11:48:15.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:15.922 7f6cc4345700 1 -- 172.21.2.101:0/3390460438 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6cbc0ebc40 con 0x7f6cbc0ebe70 2021-02-10T11:48:16.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:16.667 7f6caeffd700 1 -- 172.21.2.101:0/3390460438 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v159) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6cb8004140 con 0x7f6cbc0ebe70 2021-02-10T11:48:16.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:16.692 7f6cc4345700 1 -- 172.21.2.101:0/3390460438 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ca40186c0 msgr2=0x7f6ca401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:16.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:16.692 7f6cc4345700 1 --2- 172.21.2.101:0/3390460438 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ca40186c0 0x7f6ca401abc0 crc :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:16.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:16.692 7f6cc4345700 1 -- 172.21.2.101:0/3390460438 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cbc0ebe70 msgr2=0x7f6cbc0ec270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:16.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:16.692 7f6cc4345700 1 --2- 172.21.2.101:0/3390460438 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cbc0ebe70 0x7f6cbc0ec270 secure :-1 s=READY pgs=3201 cs=0 l=1 rev1=1 rx=0x7f6cb8001c60 tx=0x7f6cb800a8d0).stop 2021-02-10T11:48:16.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:16.692 7f6cc4345700 1 -- 172.21.2.101:0/3390460438 shutdown_connections 2021-02-10T11:48:16.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:16.692 7f6cc4345700 1 --2- 172.21.2.101:0/3390460438 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6ca40186c0 0x7f6ca401abc0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:16.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:16.692 7f6cc4345700 1 --2- 172.21.2.101:0/3390460438 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6cbc093830 0x7f6cbc093c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:16.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:16.692 7f6cc4345700 1 --2- 172.21.2.101:0/3390460438 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6cbc094250 0x7f6cbc094650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:16.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:16.692 7f6cc4345700 1 --2- 172.21.2.101:0/3390460438 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6cbc0ebe70 0x7f6cbc0ec270 unknown :-1 s=CLOSED pgs=3201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:16.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:16.692 7f6cc4345700 1 -- 172.21.2.101:0/3390460438 >> 172.21.2.101:0/3390460438 conn(0x7f6cbc08fa10 msgr2=0x7f6cbc0bfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:16.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:16.692 7f6cc4345700 1 -- 172.21.2.101:0/3390460438 shutdown_connections 2021-02-10T11:48:16.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:16.692 7f6cc4345700 1 -- 172.21.2.101:0/3390460438 wait complete. 2021-02-10T11:48:16.697 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-02-10T11:48:17.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.116 7ff0a28c6700 1 Processor -- start 2021-02-10T11:48:17.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.116 7ff0a28c6700 1 -- start start 2021-02-10T11:48:17.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.117 7ff0a28c6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff09c0d1880 0x7ff09c1b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:17.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.117 7ff0a28c6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff09c0cff80 0x7ff09c109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:17.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.117 7ff0a28c6700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff09c109a70 con 0x7ff09c1b0af0 2021-02-10T11:48:17.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.117 7ff0a28c6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09c109760 con 0x7ff09c0d1880 2021-02-10T11:48:17.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.117 7ff0a28c6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c088070 con 0x7ff09c0cff80 2021-02-10T11:48:17.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.117 7ff09bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff09c0d1880 0x7ff09c1b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:17.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.117 7ff09b7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff09c1b0af0 0x7ff09c108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35216/0 (socket says 172.21.2.101:35216) 2021-02-10T11:48:17.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.117 7ff09bfff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff09c0d1880 0x7ff09c1b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34722/0 (socket says 172.21.2.101:34722) 2021-02-10T11:48:17.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.117 7ff09b7fe700 1 -- 172.21.2.101:0/690982718 learned_addr learned my addr 172.21.2.101:0/690982718 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:17.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.117 7ff0a0ee3700 1 --2- 172.21.2.101:0/690982718 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff09c0cff80 0x7ff09c109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:17.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.117 7ff09bfff700 1 -- 172.21.2.101:0/690982718 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff09c0cff80 msgr2=0x7ff09c109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:17.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.117 7ff09bfff700 1 --2- 172.21.2.101:0/690982718 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff09c0cff80 0x7ff09c109140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:17.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.117 7ff09bfff700 1 -- 172.21.2.101:0/690982718 >> v1:172.21.2.101:6789/0 conn(0x7ff09c1b0af0 legacy=0x7ff09c108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:17.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.117 7ff09affd700 1 -- 172.21.2.101:0/690982718 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971668069 0 0) 0x7ff090000ce0 con 0x7ff09c1b0af0 2021-02-10T11:48:17.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.117 7ff09bfff700 1 -- 172.21.2.101:0/690982718 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c0d1c80 con 0x7ff09c0d1880 2021-02-10T11:48:17.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.118 7ff09bfff700 1 --2- 172.21.2.101:0/690982718 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff09c0d1880 0x7ff09c1b04d0 secure :-1 s=READY pgs=3203 cs=0 l=1 rev1=1 rx=0x7ff0940069d0 tx=0x7ff094009e40).ready entity=mon.0 client_cookie=763016f845a98ecf server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:17.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.118 7ff09affd700 1 -- 172.21.2.101:0/690982718 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff09400b510 con 0x7ff09c0d1880 2021-02-10T11:48:17.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.118 7ff09affd700 1 -- 172.21.2.101:0/690982718 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09400b890 con 0x7ff09c0d1880 2021-02-10T11:48:17.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.118 7ff09affd700 1 -- 172.21.2.101:0/690982718 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff09400bdb0 con 0x7ff09c0d1880 2021-02-10T11:48:17.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.118 7ff0a28c6700 1 -- 172.21.2.101:0/690982718 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff09c0d1880 msgr2=0x7ff09c1b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:17.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.118 7ff0a28c6700 1 --2- 172.21.2.101:0/690982718 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff09c0d1880 0x7ff09c1b04d0 secure :-1 s=READY pgs=3203 cs=0 l=1 rev1=1 rx=0x7ff0940069d0 tx=0x7ff094009e40).stop 2021-02-10T11:48:17.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.118 7ff0a28c6700 1 -- 172.21.2.101:0/690982718 shutdown_connections 2021-02-10T11:48:17.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.118 7ff0a28c6700 1 --2- 172.21.2.101:0/690982718 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff09c0cff80 0x7ff09c109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:17.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.118 7ff0a28c6700 1 --2- 172.21.2.101:0/690982718 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff09c0d1880 0x7ff09c1b04d0 unknown :-1 s=CLOSED pgs=3203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:17.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.119 7ff0a28c6700 1 -- 172.21.2.101:0/690982718 >> 172.21.2.101:0/690982718 conn(0x7ff09c0ce430 msgr2=0x7ff09c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:17.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.119 7ff0a28c6700 1 -- 172.21.2.101:0/690982718 shutdown_connections 2021-02-10T11:48:17.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.119 7ff0a28c6700 1 -- 172.21.2.101:0/690982718 wait complete. 2021-02-10T11:48:17.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.133 7ff0a28c6700 1 Processor -- start 2021-02-10T11:48:17.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.134 7ff0a28c6700 1 -- start start 2021-02-10T11:48:17.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.134 7ff0a28c6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff09c195000 0x7ff09c096540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:17.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.134 7ff0a28c6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff09c1ab4f0 0x7ff09c0b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:17.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.134 7ff0a28c6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff09c1ab8f0 0x7ff09c0b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:17.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.134 7ff0a28c6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c1abe20 con 0x7ff09c195000 2021-02-10T11:48:17.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.134 7ff0a28c6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c1ac3a0 con 0x7ff09c1ab4f0 2021-02-10T11:48:17.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.134 7ff0a28c6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c085930 con 0x7ff09c1ab8f0 2021-02-10T11:48:17.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.134 7ff09bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff09c195000 0x7ff09c096540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:17.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.134 7ff09bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff09c195000 0x7ff09c096540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34728/0 (socket says 172.21.2.101:34728) 2021-02-10T11:48:17.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.134 7ff09bfff700 1 -- 172.21.2.101:0/3623052474 learned_addr learned my addr 172.21.2.101:0/3623052474 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:17.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.134 7ff09b7fe700 1 --2- 172.21.2.101:0/3623052474 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff09c1ab4f0 0x7ff09c0b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:17.134 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.134 7ff0a0ee3700 1 --2- 172.21.2.101:0/3623052474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff09c1ab8f0 0x7ff09c0b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:17.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.134 7ff09bfff700 1 -- 172.21.2.101:0/3623052474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff09c1ab8f0 msgr2=0x7ff09c0b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:17.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.134 7ff09bfff700 1 --2- 172.21.2.101:0/3623052474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff09c1ab8f0 0x7ff09c0b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:17.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.134 7ff09bfff700 1 -- 172.21.2.101:0/3623052474 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff09c1ab4f0 msgr2=0x7ff09c0b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:17.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.134 7ff09bfff700 1 --2- 172.21.2.101:0/3623052474 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff09c1ab4f0 0x7ff09c0b1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:17.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.134 7ff09bfff700 1 -- 172.21.2.101:0/3623052474 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c0b26d0 con 0x7ff09c195000 2021-02-10T11:48:17.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.135 7ff09bfff700 1 --2- 172.21.2.101:0/3623052474 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff09c195000 0x7ff09c096540 secure :-1 s=READY pgs=3204 cs=0 l=1 rev1=1 rx=0x7ff0940099e0 tx=0x7ff09400bd20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:17.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.135 7ff098ff9700 1 -- 172.21.2.101:0/3623052474 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff094006240 con 0x7ff09c195000 2021-02-10T11:48:17.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.135 7ff098ff9700 1 -- 172.21.2.101:0/3623052474 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09400be60 con 0x7ff09c195000 2021-02-10T11:48:17.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.135 7ff098ff9700 1 -- 172.21.2.101:0/3623052474 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff094005660 con 0x7ff09c195000 2021-02-10T11:48:17.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.135 7ff0a28c6700 1 -- 172.21.2.101:0/3623052474 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09c0b26d0 con 0x7ff09c195000 2021-02-10T11:48:17.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.135 7ff0a28c6700 1 -- 172.21.2.101:0/3623052474 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09c0b26d0 con 0x7ff09c195000 2021-02-10T11:48:17.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.135 7ff098ff9700 1 -- 172.21.2.101:0/3623052474 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff09400a000 con 0x7ff09c195000 2021-02-10T11:48:17.135 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.136 7ff098ff9700 1 --2- 172.21.2.101:0/3623052474 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff07c0186c0 0x7ff07c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:17.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.136 7ff098ff9700 1 -- 172.21.2.101:0/3623052474 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5659+0+0 (secure 0 0 0) 0x7ff094032940 con 0x7ff09c195000 2021-02-10T11:48:17.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.136 7ff09b7fe700 1 --2- 172.21.2.101:0/3623052474 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff07c0186c0 0x7ff07c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:17.136 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.136 7ff09b7fe700 1 --2- 172.21.2.101:0/3623052474 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff07c0186c0 0x7ff07c01abc0 crc :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:17.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.166 7ff0a28c6700 1 -- 172.21.2.101:0/3623052474 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09c082a70 con 0x7ff09c195000 2021-02-10T11:48:17.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.169 7ff098ff9700 1 -- 172.21.2.101:0/3623052474 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff094000d30 con 0x7ff09c195000 2021-02-10T11:48:17.292 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.296 7ff0a28c6700 1 -- 172.21.2.101:0/3623052474 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff09c082a70 con 0x7ff09c195000 2021-02-10T11:48:17.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.672 7ff098ff9700 1 -- 172.21.2.101:0/3623052474 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v185) v1 ==== 185+0+0 (secure 0 0 0) 0x7ff09401f850 con 0x7ff09c195000 2021-02-10T11:48:17.709 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' removed 2021-02-10T11:48:17.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.714 7ff0a28c6700 1 -- 172.21.2.101:0/3623052474 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff07c0186c0 msgr2=0x7ff07c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:17.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.714 7ff0a28c6700 1 --2- 172.21.2.101:0/3623052474 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff07c0186c0 0x7ff07c01abc0 crc :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:17.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.714 7ff0a28c6700 1 -- 172.21.2.101:0/3623052474 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff09c195000 msgr2=0x7ff09c096540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:17.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.714 7ff0a28c6700 1 --2- 172.21.2.101:0/3623052474 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff09c195000 0x7ff09c096540 secure :-1 s=READY pgs=3204 cs=0 l=1 rev1=1 rx=0x7ff0940099e0 tx=0x7ff09400bd20).stop 2021-02-10T11:48:17.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.715 7ff0a28c6700 1 -- 172.21.2.101:0/3623052474 shutdown_connections 2021-02-10T11:48:17.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.715 7ff0a28c6700 1 --2- 172.21.2.101:0/3623052474 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff07c0186c0 0x7ff07c01abc0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:17.711 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.715 7ff0a28c6700 1 --2- 172.21.2.101:0/3623052474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff09c1ab8f0 0x7ff09c0b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:17.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.715 7ff0a28c6700 1 --2- 172.21.2.101:0/3623052474 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff09c1ab4f0 0x7ff09c0b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:17.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.715 7ff0a28c6700 1 --2- 172.21.2.101:0/3623052474 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff09c195000 0x7ff09c096540 unknown :-1 s=CLOSED pgs=3204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:17.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.715 7ff0a28c6700 1 -- 172.21.2.101:0/3623052474 >> 172.21.2.101:0/3623052474 conn(0x7ff09c19c630 msgr2=0x7ff09c082ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:17.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.715 7ff0a28c6700 1 -- 172.21.2.101:0/3623052474 shutdown_connections 2021-02-10T11:48:17.712 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:17.715 7ff0a28c6700 1 -- 172.21.2.101:0/3623052474 wait complete. 2021-02-10T11:48:17.719 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-02-10T11:48:18.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.140 7fd6ba8df700 1 Processor -- start 2021-02-10T11:48:18.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.140 7fd6ba8df700 1 -- start start 2021-02-10T11:48:18.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.141 7fd6ba8df700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6b40d1c80 0x7fd6b40cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:18.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.141 7fd6ba8df700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b40d0180 con 0x7fd6b40cf590 2021-02-10T11:48:18.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.141 7fd6ba8df700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b40fe1d0 con 0x7fd6b40d1880 2021-02-10T11:48:18.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.141 7fd6ba8df700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b41b0cc0 con 0x7fd6b40d1c80 2021-02-10T11:48:18.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.141 7fd6b8efc700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd6b40cf590 0x7fd6b40cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35230/0 (socket says 172.21.2.101:35230) 2021-02-10T11:48:18.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.141 7fd6b8efc700 1 -- 172.21.2.101:0/3314225593 learned_addr learned my addr 172.21.2.101:0/3314225593 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:18.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.141 7fd6b37fe700 1 --2- 172.21.2.101:0/3314225593 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6b40d1c80 0x7fd6b40cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:18.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.141 7fd6b2ffd700 1 -- 172.21.2.101:0/3314225593 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 925068123 0 0) 0x7fd6ac000d30 con 0x7fd6b40cf590 2021-02-10T11:48:18.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.141 7fd6b2ffd700 1 -- 172.21.2.101:0/3314225593 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd69c002fe0 con 0x7fd6b40cf590 2021-02-10T11:48:18.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.142 7fd6b37fe700 1 -- 172.21.2.101:0/3314225593 >> v1:172.21.2.108:6789/0 conn(0x7fd6b40d1880 legacy=0x7fd6b41b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:18.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.142 7fd6b2ffd700 1 -- 172.21.2.101:0/3314225593 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1554752006 0 0) 0x7fd6a4001080 con 0x7fd6b40d1880 2021-02-10T11:48:18.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.142 7fd6b37fe700 1 -- 172.21.2.101:0/3314225593 >> v1:172.21.2.101:6789/0 conn(0x7fd6b40cf590 legacy=0x7fd6b40cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:18.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.142 7fd6b37fe700 1 -- 172.21.2.101:0/3314225593 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b4109a70 con 0x7fd6b40d1c80 2021-02-10T11:48:18.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.142 7fd6b37fe700 1 --2- 172.21.2.101:0/3314225593 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6b40d1c80 0x7fd6b40cef70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7fd6a8006650 tx=0x7fd6a8009a80).ready entity=mon.2 client_cookie=ce0985fd6e54d79c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:18.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.142 7fd6b2ffd700 1 -- 172.21.2.101:0/3314225593 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6a800c0c0 con 0x7fd6b40d1c80 2021-02-10T11:48:18.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.142 7fd6b2ffd700 1 -- 172.21.2.101:0/3314225593 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a800c340 con 0x7fd6b40d1c80 2021-02-10T11:48:18.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.142 7fd6b2ffd700 1 -- 172.21.2.101:0/3314225593 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6a800c820 con 0x7fd6b40d1c80 2021-02-10T11:48:18.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.142 7fd6ba8df700 1 -- 172.21.2.101:0/3314225593 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6b40d1c80 msgr2=0x7fd6b40cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:18.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.142 7fd6ba8df700 1 --2- 172.21.2.101:0/3314225593 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6b40d1c80 0x7fd6b40cef70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7fd6a8006650 tx=0x7fd6a8009a80).stop 2021-02-10T11:48:18.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.143 7fd6ba8df700 1 -- 172.21.2.101:0/3314225593 shutdown_connections 2021-02-10T11:48:18.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.143 7fd6ba8df700 1 --2- 172.21.2.101:0/3314225593 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6b40d1c80 0x7fd6b40cef70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:18.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.143 7fd6ba8df700 1 -- 172.21.2.101:0/3314225593 >> 172.21.2.101:0/3314225593 conn(0x7fd6b40ce430 msgr2=0x7fd6b40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:18.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.143 7fd6ba8df700 1 -- 172.21.2.101:0/3314225593 shutdown_connections 2021-02-10T11:48:18.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.143 7fd6ba8df700 1 -- 172.21.2.101:0/3314225593 wait complete. 2021-02-10T11:48:18.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.157 7fd6ba8df700 1 Processor -- start 2021-02-10T11:48:18.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.157 7fd6ba8df700 1 -- start start 2021-02-10T11:48:18.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.158 7fd6ba8df700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6b41b58d0 0x7fd6b40aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:18.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.158 7fd6ba8df700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6b40aaaf0 0x7fd6b41b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:18.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.158 7fd6ba8df700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6b41b86c0 0x7fd6b419de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:18.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.158 7fd6ba8df700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b41b8ac0 con 0x7fd6b41b58d0 2021-02-10T11:48:18.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.158 7fd6ba8df700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b41b8e20 con 0x7fd6b40aaaf0 2021-02-10T11:48:18.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.158 7fd6ba8df700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b41b5cd0 con 0x7fd6b41b86c0 2021-02-10T11:48:18.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.158 7fd6b3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6b41b58d0 0x7fd6b40aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:18.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.158 7fd6b3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6b41b58d0 0x7fd6b40aa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34744/0 (socket says 172.21.2.101:34744) 2021-02-10T11:48:18.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.158 7fd6b3fff700 1 -- 172.21.2.101:0/4187832503 learned_addr learned my addr 172.21.2.101:0/4187832503 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:18.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.158 7fd6b8efc700 1 --2- 172.21.2.101:0/4187832503 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6b41b86c0 0x7fd6b419de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:18.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.158 7fd6b37fe700 1 --2- 172.21.2.101:0/4187832503 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6b40aaaf0 0x7fd6b41b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:18.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.159 7fd6b3fff700 1 -- 172.21.2.101:0/4187832503 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6b41b86c0 msgr2=0x7fd6b419de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:18.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.159 7fd6b3fff700 1 --2- 172.21.2.101:0/4187832503 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6b41b86c0 0x7fd6b419de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:18.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.159 7fd6b3fff700 1 -- 172.21.2.101:0/4187832503 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6b40aaaf0 msgr2=0x7fd6b41b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:18.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.159 7fd6b3fff700 1 --2- 172.21.2.101:0/4187832503 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6b40aaaf0 0x7fd6b41b80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:18.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.159 7fd6b3fff700 1 -- 172.21.2.101:0/4187832503 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b4095710 con 0x7fd6b41b58d0 2021-02-10T11:48:18.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.159 7fd6b37fe700 1 --2- 172.21.2.101:0/4187832503 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6b40aaaf0 0x7fd6b41b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:48:18.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.159 7fd6b3fff700 1 --2- 172.21.2.101:0/4187832503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6b41b58d0 0x7fd6b40aa4d0 secure :-1 s=READY pgs=3206 cs=0 l=1 rev1=1 rx=0x7fd6a4006f30 tx=0x7fd6a400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:18.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.159 7fd6b0ff9700 1 -- 172.21.2.101:0/4187832503 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6a400bd80 con 0x7fd6b41b58d0 2021-02-10T11:48:18.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.159 7fd6b0ff9700 1 -- 172.21.2.101:0/4187832503 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a4006030 con 0x7fd6b41b58d0 2021-02-10T11:48:18.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.159 7fd6b0ff9700 1 -- 172.21.2.101:0/4187832503 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd6a4005540 con 0x7fd6b41b58d0 2021-02-10T11:48:18.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.159 7fd6ba8df700 1 -- 172.21.2.101:0/4187832503 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b4095710 con 0x7fd6b41b58d0 2021-02-10T11:48:18.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.159 7fd6ba8df700 1 -- 172.21.2.101:0/4187832503 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b419e780 con 0x7fd6b41b58d0 2021-02-10T11:48:18.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.160 7fd6b0ff9700 1 -- 172.21.2.101:0/4187832503 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd6a4006210 con 0x7fd6b41b58d0 2021-02-10T11:48:18.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.160 7fd6b0ff9700 1 --2- 172.21.2.101:0/4187832503 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6940186c0 0x7fd69401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:18.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.160 7fd6b0ff9700 1 -- 172.21.2.101:0/4187832503 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5221+0+0 (secure 0 0 0) 0x7fd6a4032400 con 0x7fd6b41b58d0 2021-02-10T11:48:18.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.160 7fd6b37fe700 1 --2- 172.21.2.101:0/4187832503 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6940186c0 0x7fd69401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:18.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.160 7fd6b37fe700 1 --2- 172.21.2.101:0/4187832503 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6940186c0 0x7fd69401abc0 crc :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:18.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.191 7fd6ba8df700 1 -- 172.21.2.101:0/4187832503 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b4082aa0 con 0x7fd6b41b58d0 2021-02-10T11:48:18.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.193 7fd6b0ff9700 1 -- 172.21.2.101:0/4187832503 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd6a402cee0 con 0x7fd6b41b58d0 2021-02-10T11:48:18.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.321 7fd6ba8df700 1 -- 172.21.2.101:0/4187832503 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd6b4082aa0 con 0x7fd6b41b58d0 2021-02-10T11:48:18.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.677 7fd6b0ff9700 1 -- 172.21.2.101:0/4187832503 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v186) v1 ==== 173+0+0 (secure 0 0 0) 0x7fd6a402cee0 con 0x7fd6b41b58d0 2021-02-10T11:48:18.685 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' removed 2021-02-10T11:48:18.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.690 7fd6ba8df700 1 -- 172.21.2.101:0/4187832503 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6940186c0 msgr2=0x7fd69401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:18.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.690 7fd6ba8df700 1 --2- 172.21.2.101:0/4187832503 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6940186c0 0x7fd69401abc0 crc :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:18.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.690 7fd6ba8df700 1 -- 172.21.2.101:0/4187832503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6b41b58d0 msgr2=0x7fd6b40aa4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:18.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.690 7fd6ba8df700 1 --2- 172.21.2.101:0/4187832503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6b41b58d0 0x7fd6b40aa4d0 secure :-1 s=READY pgs=3206 cs=0 l=1 rev1=1 rx=0x7fd6a4006f30 tx=0x7fd6a400b990).stop 2021-02-10T11:48:18.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.690 7fd6ba8df700 1 -- 172.21.2.101:0/4187832503 shutdown_connections 2021-02-10T11:48:18.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.690 7fd6ba8df700 1 --2- 172.21.2.101:0/4187832503 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd6940186c0 0x7fd69401abc0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:18.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.690 7fd6ba8df700 1 --2- 172.21.2.101:0/4187832503 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd6b41b86c0 0x7fd6b419de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:18.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.690 7fd6ba8df700 1 --2- 172.21.2.101:0/4187832503 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd6b40aaaf0 0x7fd6b41b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:18.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.690 7fd6ba8df700 1 --2- 172.21.2.101:0/4187832503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd6b41b58d0 0x7fd6b40aa4d0 unknown :-1 s=CLOSED pgs=3206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:18.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.690 7fd6ba8df700 1 -- 172.21.2.101:0/4187832503 >> 172.21.2.101:0/4187832503 conn(0x7fd6b419f940 msgr2=0x7fd6b418c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:18.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.690 7fd6ba8df700 1 -- 172.21.2.101:0/4187832503 shutdown_connections 2021-02-10T11:48:18.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:18.690 7fd6ba8df700 1 -- 172.21.2.101:0/4187832503 wait complete. 2021-02-10T11:48:18.696 INFO:tasks.ceph.mds.a:Restarting daemon 2021-02-10T11:48:18.697 INFO:tasks.ceph.mds.a:Stopping old one... 2021-02-10T11:48:18.697 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-02-10T11:48:18.697 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:48:18.697 INFO:tasks.ceph.mds.b:Restarting daemon 2021-02-10T11:48:18.697 INFO:tasks.ceph.mds.b:Stopping old one... 2021-02-10T11:48:18.698 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-02-10T11:48:18.698 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:48:18.706 INFO:tasks.ceph.mds.a:Stopped 2021-02-10T11:48:18.706 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-02-10T11:48:18.719 INFO:tasks.ceph.mds.b:Stopped 2021-02-10T11:48:18.719 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-02-10T11:48:18.753 INFO:tasks.ceph.mds.a:Started 2021-02-10T11:48:18.753 DEBUG:teuthology.parallel:result is None 2021-02-10T11:48:18.814 INFO:tasks.ceph.mds.b:Started 2021-02-10T11:48:18.814 DEBUG:teuthology.parallel:result is None 2021-02-10T11:48:18.815 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd blacklist clear 2021-02-10T11:48:19.162 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:48:19.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.168 7f01b7068700 1 Processor -- start 2021-02-10T11:48:19.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.168 7f01b7068700 1 -- start start 2021-02-10T11:48:19.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.168 7f01b7068700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f01b00d0340 0x7f01b00ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:19.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.168 7f01b7068700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b00f0b10 con 0x7f01b00d1870 2021-02-10T11:48:19.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.168 7f01b7068700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b01b0be0 con 0x7f01b00d1c70 2021-02-10T11:48:19.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.168 7f01b7068700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b00e5ed0 con 0x7f01b00d0340 2021-02-10T11:48:19.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.168 7f01b4e84700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f01b00d1870 0x7f01b01b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35254/0 (socket says 172.21.2.101:35254) 2021-02-10T11:48:19.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.168 7f01b4e84700 1 -- 172.21.2.101:0/1422189356 learned_addr learned my addr 172.21.2.101:0/1422189356 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:19.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01a7fff700 1 --2- 172.21.2.101:0/1422189356 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f01b00d0340 0x7f01b00ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:19.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01a77fe700 1 -- 172.21.2.101:0/1422189356 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3833346138 0 0) 0x7f01ac001060 con 0x7f01b00d1870 2021-02-10T11:48:19.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01a77fe700 1 -- 172.21.2.101:0/1422189356 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0198002fe0 con 0x7f01b00d1870 2021-02-10T11:48:19.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01a77fe700 1 -- 172.21.2.101:0/1422189356 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3865058550 0 0) 0x7f01ac001460 con 0x7f01b00d1870 2021-02-10T11:48:19.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01a77fe700 1 -- 172.21.2.101:0/1422189356 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f01b00d0340 msgr2=0x7f01b00ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:19.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01a77fe700 1 --2- 172.21.2.101:0/1422189356 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f01b00d0340 0x7f01b00ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:19.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01a77fe700 1 -- 172.21.2.101:0/1422189356 >> v1:172.21.2.108:6789/0 conn(0x7f01b00d1c70 legacy=0x7f01b00efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:19.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01a77fe700 1 -- 172.21.2.101:0/1422189356 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b00d3360 con 0x7f01b00d1870 2021-02-10T11:48:19.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01a77fe700 1 -- 172.21.2.101:0/1422189356 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f01ac001980 con 0x7f01b00d1870 2021-02-10T11:48:19.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01a77fe700 1 -- 172.21.2.101:0/1422189356 >> v1:172.21.2.101:6789/0 conn(0x7f01b00d1870 legacy=0x7f01b01b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:19.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01a77fe700 1 --2- 172.21.2.101:0/1422189356 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0198003a70 0x7f0198005f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:19.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01a77fe700 1 --2- 172.21.2.101:0/1422189356 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0198006780 0x7f0198008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:19.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01a77fe700 1 --2- 172.21.2.101:0/1422189356 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0198009510 0x7f019800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:19.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01a77fe700 1 -- 172.21.2.101:0/1422189356 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019800c2e0 con 0x7f0198003a70 2021-02-10T11:48:19.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01a77fe700 1 -- 172.21.2.101:0/1422189356 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019800c500 con 0x7f0198009510 2021-02-10T11:48:19.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01b4e84700 1 --2- 172.21.2.101:0/1422189356 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0198003a70 0x7f0198005f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:19.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01a77fe700 1 -- 172.21.2.101:0/1422189356 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0198050730 con 0x7f0198006780 2021-02-10T11:48:19.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01a7fff700 1 --2- 172.21.2.101:0/1422189356 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0198006780 0x7f0198008c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:19.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01b5685700 1 --2- 172.21.2.101:0/1422189356 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0198009510 0x7f019800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:19.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01b4e84700 1 -- 172.21.2.101:0/1422189356 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0198006780 msgr2=0x7f0198008c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:19.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01b4e84700 1 --2- 172.21.2.101:0/1422189356 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0198006780 0x7f0198008c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:19.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01b4e84700 1 -- 172.21.2.101:0/1422189356 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0198009510 msgr2=0x7f019800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:19.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01b4e84700 1 --2- 172.21.2.101:0/1422189356 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0198009510 0x7f019800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:19.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.169 7f01b4e84700 1 -- 172.21.2.101:0/1422189356 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0198050990 con 0x7f0198003a70 2021-02-10T11:48:19.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.170 7f01b4e84700 1 --2- 172.21.2.101:0/1422189356 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0198003a70 0x7f0198005f00 secure :-1 s=READY pgs=3210 cs=0 l=1 rev1=1 rx=0x7f01ac007cf0 tx=0x7f01ac00a310).ready entity=mon.0 client_cookie=574b7b40dbe543ee server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:19.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.170 7f01a77fe700 1 -- 172.21.2.101:0/1422189356 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f01ac0017c0 con 0x7f0198003a70 2021-02-10T11:48:19.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.170 7f01a77fe700 1 -- 172.21.2.101:0/1422189356 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac0070c0 con 0x7f0198003a70 2021-02-10T11:48:19.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.170 7f01a77fe700 1 -- 172.21.2.101:0/1422189356 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f01ac00a910 con 0x7f0198003a70 2021-02-10T11:48:19.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.170 7f01b7068700 1 -- 172.21.2.101:0/1422189356 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0198003a70 msgr2=0x7f0198005f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:19.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.170 7f01b7068700 1 --2- 172.21.2.101:0/1422189356 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0198003a70 0x7f0198005f00 secure :-1 s=READY pgs=3210 cs=0 l=1 rev1=1 rx=0x7f01ac007cf0 tx=0x7f01ac00a310).stop 2021-02-10T11:48:19.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.170 7f01b5685700 1 -- 172.21.2.101:0/1422189356 reap_dead start 2021-02-10T11:48:19.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.170 7f01b7068700 1 -- 172.21.2.101:0/1422189356 shutdown_connections 2021-02-10T11:48:19.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.170 7f01b7068700 1 -- 172.21.2.101:0/1422189356 >> 172.21.2.101:0/1422189356 conn(0x7f01b00ce450 msgr2=0x7f01b00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:19.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.170 7f01b7068700 1 -- 172.21.2.101:0/1422189356 shutdown_connections 2021-02-10T11:48:19.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.170 7f01b7068700 1 -- 172.21.2.101:0/1422189356 wait complete. 2021-02-10T11:48:19.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.185 7f01b7068700 1 Processor -- start 2021-02-10T11:48:19.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.185 7f01b7068700 1 -- start start 2021-02-10T11:48:19.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.186 7f01b7068700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f01b019ffa0 0x7f01b00eda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:19.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.186 7f01b7068700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f01b01a03a0 0x7f01b00ee030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:19.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.186 7f01b7068700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f01b00ee650 0x7f01b00ebe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:19.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.186 7f01b7068700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b00ed490 con 0x7f01b01a03a0 2021-02-10T11:48:19.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.186 7f01b7068700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b00eea50 con 0x7f01b00ee650 2021-02-10T11:48:19.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.186 7f01b7068700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b00eec30 con 0x7f01b019ffa0 2021-02-10T11:48:19.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.186 7f01a7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f01b01a03a0 0x7f01b00ee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:19.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.186 7f01a7fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f01b01a03a0 0x7f01b00ee030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34774/0 (socket says 172.21.2.101:34774) 2021-02-10T11:48:19.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.186 7f01a7fff700 1 -- 172.21.2.101:0/4261149376 learned_addr learned my addr 172.21.2.101:0/4261149376 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:19.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.186 7f01b4e84700 1 --2- 172.21.2.101:0/4261149376 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f01b019ffa0 0x7f01b00eda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:19.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.186 7f01b5685700 1 --2- 172.21.2.101:0/4261149376 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f01b00ee650 0x7f01b00ebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:19.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.186 7f01a7fff700 1 -- 172.21.2.101:0/4261149376 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f01b019ffa0 msgr2=0x7f01b00eda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:19.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.186 7f01a7fff700 1 --2- 172.21.2.101:0/4261149376 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f01b019ffa0 0x7f01b00eda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:19.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.186 7f01a7fff700 1 -- 172.21.2.101:0/4261149376 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f01b00ee650 msgr2=0x7f01b00ebe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:19.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.186 7f01a7fff700 1 --2- 172.21.2.101:0/4261149376 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f01b00ee650 0x7f01b00ebe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:19.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.186 7f01a7fff700 1 -- 172.21.2.101:0/4261149376 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b00ec4a0 con 0x7f01b01a03a0 2021-02-10T11:48:19.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.186 7f01b5685700 1 --2- 172.21.2.101:0/4261149376 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f01b00ee650 0x7f01b00ebe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:48:19.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.186 7f01a7fff700 1 --2- 172.21.2.101:0/4261149376 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f01b01a03a0 0x7f01b00ee030 secure :-1 s=READY pgs=3211 cs=0 l=1 rev1=1 rx=0x7f01a000b560 tx=0x7f01a000b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:19.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.186 7f01a57fa700 1 -- 172.21.2.101:0/4261149376 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f01a000bcf0 con 0x7f01b01a03a0 2021-02-10T11:48:19.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.187 7f01a57fa700 1 -- 172.21.2.101:0/4261149376 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a0000c90 con 0x7f01b01a03a0 2021-02-10T11:48:19.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.187 7f01a57fa700 1 -- 172.21.2.101:0/4261149376 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f01a0002220 con 0x7f01b01a03a0 2021-02-10T11:48:19.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.187 7f01b7068700 1 -- 172.21.2.101:0/4261149376 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b00ec4a0 con 0x7f01b01a03a0 2021-02-10T11:48:19.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.187 7f01b7068700 1 -- 172.21.2.101:0/4261149376 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b00eca30 con 0x7f01b01a03a0 2021-02-10T11:48:19.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.187 7f01a57fa700 1 -- 172.21.2.101:0/4261149376 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f01a0006b00 con 0x7f01b01a03a0 2021-02-10T11:48:19.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.187 7f01a57fa700 1 --2- 172.21.2.101:0/4261149376 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f01900186c0 0x7f019001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:19.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.187 7f01a57fa700 1 -- 172.21.2.101:0/4261149376 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 4839+0+0 (secure 0 0 0) 0x7f01a0032170 con 0x7f01b01a03a0 2021-02-10T11:48:19.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.187 7f01b4e84700 1 --2- 172.21.2.101:0/4261149376 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f01900186c0 0x7f019001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:19.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.188 7f01b4e84700 1 --2- 172.21.2.101:0/4261149376 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f01900186c0 0x7f019001abc0 crc :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:19.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.217 7f01b7068700 1 -- 172.21.2.101:0/4261149376 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b00eee50 con 0x7f01b01a03a0 2021-02-10T11:48:19.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.220 7f01a57fa700 1 -- 172.21.2.101:0/4261149376 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f01a00214f0 con 0x7f01b01a03a0 2021-02-10T11:48:19.220 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:48:19.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.359 7f01b7068700 1 -- 172.21.2.101:0/4261149376 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f01b00eee50 con 0x7f01b01a03a0 2021-02-10T11:48:19.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.680 7f01a57fa700 1 -- 172.21.2.101:0/4261149376 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v187) v1 ==== 97+0+0 (secure 0 0 0) 0x7f01a00214f0 con 0x7f01b01a03a0 2021-02-10T11:48:19.721 INFO:teuthology.orchestra.run.gibba001.stderr: removed all blacklist entries 2021-02-10T11:48:19.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.726 7f01b7068700 1 -- 172.21.2.101:0/4261149376 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f01900186c0 msgr2=0x7f019001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:19.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.726 7f01b7068700 1 --2- 172.21.2.101:0/4261149376 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f01900186c0 0x7f019001abc0 crc :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:19.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.726 7f01b7068700 1 -- 172.21.2.101:0/4261149376 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f01b01a03a0 msgr2=0x7f01b00ee030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:19.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.726 7f01b7068700 1 --2- 172.21.2.101:0/4261149376 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f01b01a03a0 0x7f01b00ee030 secure :-1 s=READY pgs=3211 cs=0 l=1 rev1=1 rx=0x7f01a000b560 tx=0x7f01a000b900).stop 2021-02-10T11:48:19.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.726 7f01b7068700 1 -- 172.21.2.101:0/4261149376 shutdown_connections 2021-02-10T11:48:19.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.726 7f01b7068700 1 --2- 172.21.2.101:0/4261149376 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f01b00ee650 0x7f01b00ebe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:19.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.726 7f01b7068700 1 --2- 172.21.2.101:0/4261149376 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f01b01a03a0 0x7f01b00ee030 unknown :-1 s=CLOSED pgs=3211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:19.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.726 7f01b7068700 1 --2- 172.21.2.101:0/4261149376 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f01900186c0 0x7f019001abc0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:19.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.726 7f01b7068700 1 --2- 172.21.2.101:0/4261149376 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f01b019ffa0 0x7f01b00eda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:19.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.726 7f01b7068700 1 -- 172.21.2.101:0/4261149376 >> 172.21.2.101:0/4261149376 conn(0x7f01b0094d90 msgr2=0x7f01b008a420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:19.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.726 7f01b7068700 1 -- 172.21.2.101:0/4261149376 shutdown_connections 2021-02-10T11:48:19.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:19.727 7f01b7068700 1 -- 172.21.2.101:0/4261149376 wait complete. 2021-02-10T11:48:19.731 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2021-02-10T11:48:19.732 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:48:20.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.149 7fa7912d0700 1 Processor -- start 2021-02-10T11:48:20.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.149 7fa7912d0700 1 -- start start 2021-02-10T11:48:20.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.149 7fa7912d0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa78c0d1870 0x7fa78c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:20.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.149 7fa7912d0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa78c1b0a10 0x7fa78c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:20.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.149 7fa7912d0700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa78c0f0b10 con 0x7fa78c0d1c70 2021-02-10T11:48:20.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.149 7fa7912d0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa78c0e5ed0 con 0x7fa78c0d1870 2021-02-10T11:48:20.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.149 7fa7912d0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c0f0650 con 0x7fa78c1b0a10 2021-02-10T11:48:20.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.150 7fa78b07d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa78c0d1870 0x7fa78c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:20.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.150 7fa78b07d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa78c0d1870 0x7fa78c1b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34780/0 (socket says 172.21.2.101:34780) 2021-02-10T11:48:20.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.150 7fa78b07d700 1 -- 172.21.2.101:0/2290494966 learned_addr learned my addr 172.21.2.101:0/2290494966 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:20.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.150 7fa78b07d700 1 -- 172.21.2.101:0/2290494966 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa78c1b0a10 msgr2=0x7fa78c0cf330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:48:20.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.150 7fa78b07d700 1 --2- 172.21.2.101:0/2290494966 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa78c1b0a10 0x7fa78c0cf330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:20.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.150 7fa78b07d700 1 -- 172.21.2.101:0/2290494966 >> v1:172.21.2.101:6789/0 conn(0x7fa78c0d1c70 legacy=0x7fa78c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:20.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.150 7fa78b07d700 1 -- 172.21.2.101:0/2290494966 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78c0f0870 con 0x7fa78c0d1870 2021-02-10T11:48:20.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.150 7fa78b07d700 1 --2- 172.21.2.101:0/2290494966 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa78c0d1870 0x7fa78c1b03f0 secure :-1 s=READY pgs=3215 cs=0 l=1 rev1=1 rx=0x7fa774006700 tx=0x7fa774009b00).ready entity=mon.0 client_cookie=d7b3b0e0cd77b1c9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:20.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.150 7fa78a07b700 1 -- 172.21.2.101:0/2290494966 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa77400c0c0 con 0x7fa78c0d1870 2021-02-10T11:48:20.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.150 7fa78a07b700 1 -- 172.21.2.101:0/2290494966 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77400c340 con 0x7fa78c0d1870 2021-02-10T11:48:20.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.150 7fa78a07b700 1 -- 172.21.2.101:0/2290494966 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa77400c820 con 0x7fa78c0d1870 2021-02-10T11:48:20.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.150 7fa7912d0700 1 -- 172.21.2.101:0/2290494966 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa78c0d1870 msgr2=0x7fa78c1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:20.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.150 7fa7912d0700 1 --2- 172.21.2.101:0/2290494966 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa78c0d1870 0x7fa78c1b03f0 secure :-1 s=READY pgs=3215 cs=0 l=1 rev1=1 rx=0x7fa774006700 tx=0x7fa774009b00).stop 2021-02-10T11:48:20.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.151 7fa7912d0700 1 -- 172.21.2.101:0/2290494966 shutdown_connections 2021-02-10T11:48:20.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.151 7fa7912d0700 1 --2- 172.21.2.101:0/2290494966 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa78c1b0a10 0x7fa78c0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:20.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.151 7fa7912d0700 1 --2- 172.21.2.101:0/2290494966 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa78c0d1870 0x7fa78c1b03f0 unknown :-1 s=CLOSED pgs=3215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:20.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.151 7fa7912d0700 1 -- 172.21.2.101:0/2290494966 >> 172.21.2.101:0/2290494966 conn(0x7fa78c0ce450 msgr2=0x7fa78c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:20.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.151 7fa7912d0700 1 -- 172.21.2.101:0/2290494966 shutdown_connections 2021-02-10T11:48:20.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.151 7fa7912d0700 1 -- 172.21.2.101:0/2290494966 wait complete. 2021-02-10T11:48:20.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.165 7fa7912d0700 1 Processor -- start 2021-02-10T11:48:20.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.166 7fa7912d0700 1 -- start start 2021-02-10T11:48:20.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.166 7fa7912d0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa78c0b0ad0 0x7fa78c091f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:20.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.166 7fa7912d0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa78c1a8850 0x7fa78c1a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:20.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.166 7fa7912d0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa78c1a6fa0 0x7fa78c1a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:20.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.166 7fa7912d0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c1abfe0 con 0x7fa78c1a6fa0 2021-02-10T11:48:20.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.166 7fa7912d0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c1a9c40 con 0x7fa78c0b0ad0 2021-02-10T11:48:20.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.166 7fa7912d0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c1a9730 con 0x7fa78c1a8850 2021-02-10T11:48:20.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.166 7fa78b87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa78c1a6fa0 0x7fa78c1a73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:20.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.166 7fa78b87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa78c1a6fa0 0x7fa78c1a73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34784/0 (socket says 172.21.2.101:34784) 2021-02-10T11:48:20.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.166 7fa78b87e700 1 -- 172.21.2.101:0/1081721253 learned_addr learned my addr 172.21.2.101:0/1081721253 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:20.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.166 7fa78b87e700 1 -- 172.21.2.101:0/1081721253 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa78c1a8850 msgr2=0x7fa78c1a8c50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:20.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.166 7fa78b87e700 1 --2- 172.21.2.101:0/1081721253 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa78c1a8850 0x7fa78c1a8c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:20.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.166 7fa78b87e700 1 -- 172.21.2.101:0/1081721253 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa78c0b0ad0 msgr2=0x7fa78c091f30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:20.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.166 7fa78b87e700 1 --2- 172.21.2.101:0/1081721253 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa78c0b0ad0 0x7fa78c091f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:20.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.166 7fa78b87e700 1 -- 172.21.2.101:0/1081721253 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78c1a9270 con 0x7fa78c1a6fa0 2021-02-10T11:48:20.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.167 7fa78b87e700 1 --2- 172.21.2.101:0/1081721253 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa78c1a6fa0 0x7fa78c1a73a0 secure :-1 s=READY pgs=3216 cs=0 l=1 rev1=1 rx=0x7fa784009980 tx=0x7fa78400bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:20.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.167 7fa783fff700 1 -- 172.21.2.101:0/1081721253 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa784005870 con 0x7fa78c1a6fa0 2021-02-10T11:48:20.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.167 7fa783fff700 1 -- 172.21.2.101:0/1081721253 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa784005a70 con 0x7fa78c1a6fa0 2021-02-10T11:48:20.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.167 7fa783fff700 1 -- 172.21.2.101:0/1081721253 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa784006030 con 0x7fa78c1a6fa0 2021-02-10T11:48:20.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.167 7fa7912d0700 1 -- 172.21.2.101:0/1081721253 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa78c1a9270 con 0x7fa78c1a6fa0 2021-02-10T11:48:20.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.167 7fa7912d0700 1 -- 172.21.2.101:0/1081721253 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa78c1a7cb0 con 0x7fa78c1a6fa0 2021-02-10T11:48:20.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.168 7fa783fff700 1 -- 172.21.2.101:0/1081721253 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa784006030 con 0x7fa78c1a6fa0 2021-02-10T11:48:20.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.168 7fa783fff700 1 --2- 172.21.2.101:0/1081721253 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7700186c0 0x7fa77001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:20.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.168 7fa783fff700 1 -- 172.21.2.101:0/1081721253 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fa784032610 con 0x7fa78c1a6fa0 2021-02-10T11:48:20.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.168 7fa78b07d700 1 --2- 172.21.2.101:0/1081721253 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7700186c0 0x7fa77001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:20.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.168 7fa78b07d700 1 --2- 172.21.2.101:0/1081721253 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7700186c0 0x7fa77001abc0 crc :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:20.195 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.199 7fa7912d0700 1 -- 172.21.2.101:0/1081721253 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa78c0ce450 con 0x7fa78c1a6fa0 2021-02-10T11:48:20.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.202 7fa783fff700 1 -- 172.21.2.101:0/1081721253 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa7840210f0 con 0x7fa78c1a6fa0 2021-02-10T11:48:20.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.333 7fa7912d0700 1 -- 172.21.2.101:0/1081721253 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa78c0ce450 con 0x7fa78c1a6fa0 2021-02-10T11:48:20.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.333 7fa783fff700 1 -- 172.21.2.101:0/1081721253 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v109) v1 ==== 80+0+4513 (secure 0 0 0) 0x7fa7840210f0 con 0x7fa78c1a6fa0 2021-02-10T11:48:20.330 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:48:20.330 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:48:20.331 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:48:20.331 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:20.331 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:48:20.331 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:48:20.331 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:48:20.332 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:48:20.332 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:48:20.332 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:48:20.332 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:48:20.332 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:20.332 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:20.332 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:20.332 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:48:20.332 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:48:20.332 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:48:20.332 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:48:20.332 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:48:20.333 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:48:20.333 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:48:20.333 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:20.333 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:20.333 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:20.333 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:48:20.333 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:48:20.333 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:48:20.333 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:48:20.333 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:48:20.333 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:48:20.334 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:20.334 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:20.334 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:20.334 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:48:20.334 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:48:20.334 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:48:20.334 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:48:20.334 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:48:20.334 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:48:20.334 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:20.334 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:20.335 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:20.335 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:48:20.335 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:48:20.335 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:48:20.335 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:48:20.335 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:48:20.335 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:48:20.335 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:20.335 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:20.335 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:20.335 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:48:20.336 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:48:20.336 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:48:20.336 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:48:20.336 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:48:20.336 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:48:20.336 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:20.336 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:20.336 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:20.336 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:48:20.336 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:48:20.336 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:48:20.336 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:48:20.337 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:48:20.337 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:48:20.337 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:20.337 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:20.337 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:20.337 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:48:20.337 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:48:20.337 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:48:20.337 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:48:20.337 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:48:20.337 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:48:20.338 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:20.338 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:20.338 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:20.338 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:48:20.338 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:48:20.338 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:48:20.338 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:48:20.338 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:48:20.338 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:48:20.338 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:20.339 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:20.339 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:20.339 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:48:20.339 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:48:20.339 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:48:20.339 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:48:20.339 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:48:20.339 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:48:20.339 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:20.339 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:20.339 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:20.339 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:48:20.339 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:48:20.340 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:48:20.340 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:48:20.340 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:48:20.340 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:48:20.340 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:20.340 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:20.340 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:20.340 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:48:20.340 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:48:20.340 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:48:20.340 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:48:20.341 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:48:20.341 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:48:20.341 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:20.341 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:20.341 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:20.341 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:48:20.341 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:48:20.341 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:48:20.341 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:48:20.341 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:48:20.341 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:48:20.342 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:20.342 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:20.342 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:20.342 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:48:20.342 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:48:20.342 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:48:20.342 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:48:20.342 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:48:20.342 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:48:20.342 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:20.342 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:20.343 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:20.343 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:48:20.347 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:48:20.347 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:48:20.347 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:48:20.347 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:48:20.348 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:48:20.348 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:48:20.348 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:20.348 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:20.348 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:20.348 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:48:20.348 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:48:20.348 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:48:20.348 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:48:20.348 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:48:20.348 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:48:20.348 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:20.348 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:20.349 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:20.349 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:48:20.349 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:48:20.349 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:48:20.349 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:48:20.349 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:48:20.349 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:48:20.349 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:20.349 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:20.349 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:48:20.349 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:48:20.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.335 7fa7912d0700 1 -- 172.21.2.101:0/1081721253 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7700186c0 msgr2=0x7fa77001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:20.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.335 7fa7912d0700 1 --2- 172.21.2.101:0/1081721253 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7700186c0 0x7fa77001abc0 crc :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:20.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.335 7fa7912d0700 1 -- 172.21.2.101:0/1081721253 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa78c1a6fa0 msgr2=0x7fa78c1a73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:20.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.335 7fa7912d0700 1 --2- 172.21.2.101:0/1081721253 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa78c1a6fa0 0x7fa78c1a73a0 secure :-1 s=READY pgs=3216 cs=0 l=1 rev1=1 rx=0x7fa784009980 tx=0x7fa78400bcd0).stop 2021-02-10T11:48:20.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.335 7fa7912d0700 1 -- 172.21.2.101:0/1081721253 shutdown_connections 2021-02-10T11:48:20.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.335 7fa7912d0700 1 --2- 172.21.2.101:0/1081721253 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7700186c0 0x7fa77001abc0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:20.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.335 7fa7912d0700 1 --2- 172.21.2.101:0/1081721253 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa78c1a8850 0x7fa78c1a8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:20.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.335 7fa7912d0700 1 --2- 172.21.2.101:0/1081721253 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa78c1a6fa0 0x7fa78c1a73a0 unknown :-1 s=CLOSED pgs=3216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:20.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.335 7fa7912d0700 1 --2- 172.21.2.101:0/1081721253 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa78c0b0ad0 0x7fa78c091f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:20.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.335 7fa7912d0700 1 -- 172.21.2.101:0/1081721253 >> 172.21.2.101:0/1081721253 conn(0x7fa78c19f8f0 msgr2=0x7fa78c1a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:20.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.335 7fa7912d0700 1 -- 172.21.2.101:0/1081721253 shutdown_connections 2021-02-10T11:48:20.351 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.336 7fa7912d0700 1 -- 172.21.2.101:0/1081721253 wait complete. 2021-02-10T11:48:20.352 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:48:20.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.769 7f0b70dea700 1 Processor -- start 2021-02-10T11:48:20.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.769 7f0b70dea700 1 -- start start 2021-02-10T11:48:20.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.770 7f0b70dea700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0b6c0d1870 0x7f0b6c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:20.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.770 7f0b70dea700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b6c0f0b10 con 0x7f0b6c1b0a10 2021-02-10T11:48:20.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.770 7f0b70dea700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b6c0d0310 con 0x7f0b6c0f0650 2021-02-10T11:48:20.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.770 7f0b70dea700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b6c0d0530 con 0x7f0b6c0d1870 2021-02-10T11:48:20.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.770 7f0b6a312700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f0b6c1b0a10 0x7f0b6c0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35288/0 (socket says 172.21.2.101:35288) 2021-02-10T11:48:20.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.770 7f0b6a312700 1 -- 172.21.2.101:0/3199049496 learned_addr learned my addr 172.21.2.101:0/3199049496 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:20.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.770 7f0b69b11700 1 -- 172.21.2.101:0/3199049496 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1326163373 0 0) 0x7f0b5c0009b0 con 0x7f0b6c1b0a10 2021-02-10T11:48:20.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.770 7f0b69b11700 1 -- 172.21.2.101:0/3199049496 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b58002fe0 con 0x7f0b6c1b0a10 2021-02-10T11:48:20.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.770 7f0b69b11700 1 -- 172.21.2.101:0/3199049496 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2948157019 0 0) 0x7f0b5c0010b0 con 0x7f0b6c1b0a10 2021-02-10T11:48:20.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.770 7f0b69b11700 1 -- 172.21.2.101:0/3199049496 >> v2:172.21.2.108:3300/0 conn(0x7f0b6c0d1870 msgr2=0x7f0b6c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:20.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.770 7f0b69b11700 1 --2- 172.21.2.101:0/3199049496 >> v2:172.21.2.108:3300/0 conn(0x7f0b6c0d1870 0x7f0b6c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:20.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.770 7f0b69b11700 1 -- 172.21.2.101:0/3199049496 >> v1:172.21.2.108:6789/0 conn(0x7f0b6c0f0650 legacy=0x7f0b6c0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:20.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.770 7f0b69b11700 1 -- 172.21.2.101:0/3199049496 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6c0d1c70 con 0x7f0b6c1b0a10 2021-02-10T11:48:20.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b69b11700 1 -- 172.21.2.101:0/3199049496 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f0b5c0015b0 con 0x7f0b6c1b0a10 2021-02-10T11:48:20.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b69b11700 1 -- 172.21.2.101:0/3199049496 >> v1:172.21.2.101:6789/0 conn(0x7f0b6c1b0a10 legacy=0x7f0b6c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:20.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b69b11700 1 --2- 172.21.2.101:0/3199049496 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b58003ac0 0x7f0b58005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:20.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b69b11700 1 --2- 172.21.2.101:0/3199049496 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b58006790 0x7f0b58008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:20.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b69b11700 1 --2- 172.21.2.101:0/3199049496 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b58009590 0x7f0b5800bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:20.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b69b11700 1 -- 172.21.2.101:0/3199049496 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5800c300 con 0x7f0b58006790 2021-02-10T11:48:20.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b69b11700 1 -- 172.21.2.101:0/3199049496 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5800c520 con 0x7f0b58009590 2021-02-10T11:48:20.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b69b11700 1 -- 172.21.2.101:0/3199049496 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5800c740 con 0x7f0b58003ac0 2021-02-10T11:48:20.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b6a312700 1 --2- 172.21.2.101:0/3199049496 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b58006790 0x7f0b58008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:20.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b6b314700 1 --2- 172.21.2.101:0/3199049496 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b58009590 0x7f0b5800bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:20.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b6ab13700 1 --2- 172.21.2.101:0/3199049496 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b58003ac0 0x7f0b58005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:20.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b6a312700 1 -- 172.21.2.101:0/3199049496 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b58003ac0 msgr2=0x7f0b58005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:20.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b6a312700 1 --2- 172.21.2.101:0/3199049496 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b58003ac0 0x7f0b58005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:20.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b6a312700 1 -- 172.21.2.101:0/3199049496 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b58009590 msgr2=0x7f0b5800bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:20.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b6a312700 1 --2- 172.21.2.101:0/3199049496 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b58009590 0x7f0b5800bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:20.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b6a312700 1 -- 172.21.2.101:0/3199049496 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5800c9e0 con 0x7f0b58006790 2021-02-10T11:48:20.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b6b314700 1 -- 172.21.2.101:0/3199049496 reap_dead start 2021-02-10T11:48:20.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b6a312700 1 --2- 172.21.2.101:0/3199049496 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b58006790 0x7f0b58008c90 secure :-1 s=READY pgs=3218 cs=0 l=1 rev1=1 rx=0x7f0b5c007a40 tx=0x7f0b5c007de0).ready entity=mon.0 client_cookie=89fc92cc22232c0d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:20.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b69b11700 1 -- 172.21.2.101:0/3199049496 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0b5c007030 con 0x7f0b58006790 2021-02-10T11:48:20.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b69b11700 1 -- 172.21.2.101:0/3199049496 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5c00a0c0 con 0x7f0b58006790 2021-02-10T11:48:20.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b69b11700 1 -- 172.21.2.101:0/3199049496 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0b5c00a520 con 0x7f0b58006790 2021-02-10T11:48:20.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b70dea700 1 -- 172.21.2.101:0/3199049496 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b58006790 msgr2=0x7f0b58008c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:20.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.771 7f0b70dea700 1 --2- 172.21.2.101:0/3199049496 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b58006790 0x7f0b58008c90 secure :-1 s=READY pgs=3218 cs=0 l=1 rev1=1 rx=0x7f0b5c007a40 tx=0x7f0b5c007de0).stop 2021-02-10T11:48:20.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.772 7f0b70dea700 1 -- 172.21.2.101:0/3199049496 shutdown_connections 2021-02-10T11:48:20.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.772 7f0b70dea700 1 --2- 172.21.2.101:0/3199049496 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b58006790 0x7f0b58008c90 unknown :-1 s=CLOSED pgs=3218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:20.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.772 7f0b70dea700 1 -- 172.21.2.101:0/3199049496 >> 172.21.2.101:0/3199049496 conn(0x7f0b6c0ce450 msgr2=0x7f0b6c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:20.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.772 7f0b70dea700 1 -- 172.21.2.101:0/3199049496 shutdown_connections 2021-02-10T11:48:20.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.772 7f0b70dea700 1 -- 172.21.2.101:0/3199049496 wait complete. 2021-02-10T11:48:20.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.787 7f0b70dea700 1 Processor -- start 2021-02-10T11:48:20.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.787 7f0b70dea700 1 -- start start 2021-02-10T11:48:20.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.787 7f0b70dea700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b6c195650 0x7f0b6c0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:20.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.787 7f0b70dea700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b6c0b5b80 0x7f0b6c1a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:20.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.787 7f0b70dea700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b6c0acfa0 0x7f0b6c1a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:20.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.787 7f0b70dea700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c0b6010 con 0x7f0b6c0acfa0 2021-02-10T11:48:20.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.787 7f0b70dea700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c1a3a60 con 0x7f0b6c0b5b80 2021-02-10T11:48:20.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.787 7f0b70dea700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c1a3c40 con 0x7f0b6c195650 2021-02-10T11:48:20.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.787 7f0b6b314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b6c0acfa0 0x7f0b6c1a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:20.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.787 7f0b6b314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b6c0acfa0 0x7f0b6c1a3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34804/0 (socket says 172.21.2.101:34804) 2021-02-10T11:48:20.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.787 7f0b6b314700 1 -- 172.21.2.101:0/3153091573 learned_addr learned my addr 172.21.2.101:0/3153091573 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:20.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.788 7f0b6a312700 1 --2- 172.21.2.101:0/3153091573 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b6c0b5b80 0x7f0b6c1a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:20.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.788 7f0b6ab13700 1 --2- 172.21.2.101:0/3153091573 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b6c195650 0x7f0b6c0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:20.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.788 7f0b6b314700 1 -- 172.21.2.101:0/3153091573 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b6c195650 msgr2=0x7f0b6c0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:20.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.788 7f0b6b314700 1 --2- 172.21.2.101:0/3153091573 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b6c195650 0x7f0b6c0b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:20.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.788 7f0b6b314700 1 -- 172.21.2.101:0/3153091573 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b6c0b5b80 msgr2=0x7f0b6c1a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:20.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.788 7f0b6b314700 1 --2- 172.21.2.101:0/3153091573 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b6c0b5b80 0x7f0b6c1a2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:20.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.788 7f0b6b314700 1 -- 172.21.2.101:0/3153091573 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6c0bcae0 con 0x7f0b6c0acfa0 2021-02-10T11:48:20.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.788 7f0b6ab13700 1 --2- 172.21.2.101:0/3153091573 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b6c195650 0x7f0b6c0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:48:20.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.788 7f0b6b314700 1 --2- 172.21.2.101:0/3153091573 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b6c0acfa0 0x7f0b6c1a3440 secure :-1 s=READY pgs=3219 cs=0 l=1 rev1=1 rx=0x7f0b64009980 tx=0x7f0b6400bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:20.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.788 7f0b637fe700 1 -- 172.21.2.101:0/3153091573 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0b6400c2f0 con 0x7f0b6c0acfa0 2021-02-10T11:48:20.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.788 7f0b637fe700 1 -- 172.21.2.101:0/3153091573 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6400c5b0 con 0x7f0b6c0acfa0 2021-02-10T11:48:20.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.788 7f0b637fe700 1 -- 172.21.2.101:0/3153091573 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0b6400ca10 con 0x7f0b6c0acfa0 2021-02-10T11:48:20.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.788 7f0b70dea700 1 -- 172.21.2.101:0/3153091573 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b6c0bcae0 con 0x7f0b6c0acfa0 2021-02-10T11:48:20.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.788 7f0b70dea700 1 -- 172.21.2.101:0/3153091573 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b6c0bd070 con 0x7f0b6c0acfa0 2021-02-10T11:48:20.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.789 7f0b637fe700 1 -- 172.21.2.101:0/3153091573 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0b64000dd0 con 0x7f0b6c0acfa0 2021-02-10T11:48:20.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.789 7f0b637fe700 1 --2- 172.21.2.101:0/3153091573 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b500186c0 0x7f0b5001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:20.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.789 7f0b637fe700 1 -- 172.21.2.101:0/3153091573 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f0b64032630 con 0x7f0b6c0acfa0 2021-02-10T11:48:20.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.789 7f0b6ab13700 1 --2- 172.21.2.101:0/3153091573 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b500186c0 0x7f0b5001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:20.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.789 7f0b6ab13700 1 --2- 172.21.2.101:0/3153091573 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b500186c0 0x7f0b5001abc0 crc :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:20.816 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.819 7f0b70dea700 1 -- 172.21.2.101:0/3153091573 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b6c0b6010 con 0x7f0b6c0acfa0 2021-02-10T11:48:20.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.822 7f0b637fe700 1 -- 172.21.2.101:0/3153091573 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0b6400be10 con 0x7f0b6c0acfa0 2021-02-10T11:48:20.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.947 7f0b70dea700 1 -- 172.21.2.101:0/3153091573 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b6c0b6010 con 0x7f0b6c0acfa0 2021-02-10T11:48:20.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.948 7f0b637fe700 1 -- 172.21.2.101:0/3153091573 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v187) v1 ==== 74+0+11154 (secure 0 0 0) 0x7f0b64000dd0 con 0x7f0b6c0acfa0 2021-02-10T11:48:20.945 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:48:20.945 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":187,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:48:19.677120","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":39,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:48:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.950 7f0b70dea700 1 -- 172.21.2.101:0/3153091573 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b500186c0 msgr2=0x7f0b5001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.950 7f0b70dea700 1 --2- 172.21.2.101:0/3153091573 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b500186c0 0x7f0b5001abc0 crc :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.950 7f0b70dea700 1 -- 172.21.2.101:0/3153091573 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b6c0acfa0 msgr2=0x7f0b6c1a3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.950 7f0b70dea700 1 --2- 172.21.2.101:0/3153091573 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b6c0acfa0 0x7f0b6c1a3440 secure :-1 s=READY pgs=3219 cs=0 l=1 rev1=1 rx=0x7f0b64009980 tx=0x7f0b6400bcd0).stop 2021-02-10T11:48:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.950 7f0b70dea700 1 -- 172.21.2.101:0/3153091573 shutdown_connections 2021-02-10T11:48:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.950 7f0b70dea700 1 --2- 172.21.2.101:0/3153091573 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0b6c0acfa0 0x7f0b6c1a3440 unknown :-1 s=CLOSED pgs=3219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.950 7f0b70dea700 1 --2- 172.21.2.101:0/3153091573 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0b6c0b5b80 0x7f0b6c1a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.950 7f0b70dea700 1 --2- 172.21.2.101:0/3153091573 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0b500186c0 0x7f0b5001abc0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.950 7f0b70dea700 1 --2- 172.21.2.101:0/3153091573 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0b6c195650 0x7f0b6c0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.950 7f0b70dea700 1 -- 172.21.2.101:0/3153091573 >> 172.21.2.101:0/3153091573 conn(0x7f0b6c0b3040 msgr2=0x7f0b6c0c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:20.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.950 7f0b70dea700 1 -- 172.21.2.101:0/3153091573 shutdown_connections 2021-02-10T11:48:20.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:20.950 7f0b70dea700 1 -- 172.21.2.101:0/3153091573 wait complete. 2021-02-10T11:48:20.956 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-10T11:48:20.956 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:48:21.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.374 7f5bd1b8e700 1 Processor -- start 2021-02-10T11:48:21.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.374 7f5bd1b8e700 1 -- start start 2021-02-10T11:48:21.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.375 7f5bd1b8e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5bcc0d1c70 0x7f5bcc0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:21.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.375 7f5bd1b8e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bcc0d0170 con 0x7f5bcc0cf580 2021-02-10T11:48:21.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.375 7f5bd1b8e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bcc0fe1c0 con 0x7f5bcc0d1870 2021-02-10T11:48:21.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.375 7f5bd1b8e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bcc1b0cb0 con 0x7f5bcc0d1c70 2021-02-10T11:48:21.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.375 7f5bcbfff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f5bcc0cf580 0x7f5bcc0cf980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35310/0 (socket says 172.21.2.101:35310) 2021-02-10T11:48:21.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.375 7f5bcaffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f5bcc0d1c70 0x7f5bcc0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:21.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.375 7f5bcbfff700 1 -- 172.21.2.101:0/2573438586 learned_addr learned my addr 172.21.2.101:0/2573438586 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:21.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.375 7f5bca7fc700 1 -- 172.21.2.101:0/2573438586 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592095579 0 0) 0x7f5bc4001500 con 0x7f5bcc0cf580 2021-02-10T11:48:21.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.375 7f5bca7fc700 1 -- 172.21.2.101:0/2573438586 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bb8002fe0 con 0x7f5bcc0cf580 2021-02-10T11:48:21.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.375 7f5bcaffd700 1 -- 172.21.2.101:0/2573438586 >> v1:172.21.2.108:6789/0 conn(0x7f5bcc0d1870 legacy=0x7f5bcc1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:21.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.375 7f5bcaffd700 1 -- 172.21.2.101:0/2573438586 >> v1:172.21.2.101:6789/0 conn(0x7f5bcc0cf580 legacy=0x7f5bcc0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:21.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.375 7f5bcaffd700 1 -- 172.21.2.101:0/2573438586 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bcc109a60 con 0x7f5bcc0d1c70 2021-02-10T11:48:21.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.376 7f5bcaffd700 1 --2- 172.21.2.101:0/2573438586 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bcc0d1c70 0x7f5bcc0cef60 secure :-1 s=READY pgs=3221 cs=0 l=1 rev1=1 rx=0x7f5bbc006650 tx=0x7f5bbc009a80).ready entity=mon.0 client_cookie=94e93236e01d0d30 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:21.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.376 7f5bca7fc700 1 -- 172.21.2.101:0/2573438586 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5bbc00c0c0 con 0x7f5bcc0d1c70 2021-02-10T11:48:21.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.376 7f5bca7fc700 1 -- 172.21.2.101:0/2573438586 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bbc00c340 con 0x7f5bcc0d1c70 2021-02-10T11:48:21.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.376 7f5bca7fc700 1 -- 172.21.2.101:0/2573438586 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5bbc00c860 con 0x7f5bcc0d1c70 2021-02-10T11:48:21.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.376 7f5bd1b8e700 1 -- 172.21.2.101:0/2573438586 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bcc0d1c70 msgr2=0x7f5bcc0cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:21.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.376 7f5bd1b8e700 1 --2- 172.21.2.101:0/2573438586 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bcc0d1c70 0x7f5bcc0cef60 secure :-1 s=READY pgs=3221 cs=0 l=1 rev1=1 rx=0x7f5bbc006650 tx=0x7f5bbc009a80).stop 2021-02-10T11:48:21.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.377 7f5bd1b8e700 1 -- 172.21.2.101:0/2573438586 shutdown_connections 2021-02-10T11:48:21.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.377 7f5bd1b8e700 1 --2- 172.21.2.101:0/2573438586 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bcc0d1c70 0x7f5bcc0cef60 unknown :-1 s=CLOSED pgs=3221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:21.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.377 7f5bd1b8e700 1 -- 172.21.2.101:0/2573438586 >> 172.21.2.101:0/2573438586 conn(0x7f5bcc0ce420 msgr2=0x7f5bcc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:21.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.377 7f5bd1b8e700 1 -- 172.21.2.101:0/2573438586 shutdown_connections 2021-02-10T11:48:21.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.377 7f5bd1b8e700 1 -- 172.21.2.101:0/2573438586 wait complete. 2021-02-10T11:48:21.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.391 7f5bd1b8e700 1 Processor -- start 2021-02-10T11:48:21.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.391 7f5bd1b8e700 1 -- start start 2021-02-10T11:48:21.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.392 7f5bd1b8e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bcc0c3820 0x7f5bcc084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:21.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.392 7f5bd1b8e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bcc0a0290 0x7f5bcc0852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:21.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.392 7f5bd1b8e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5bcc086270 0x7f5bcc086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:21.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.392 7f5bd1b8e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc0c0700 con 0x7f5bcc0c3820 2021-02-10T11:48:21.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.392 7f5bd1b8e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc0858f0 con 0x7f5bcc086270 2021-02-10T11:48:21.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.392 7f5bd1b8e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc085b10 con 0x7f5bcc0a0290 2021-02-10T11:48:21.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.392 7f5bcb7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bcc0c3820 0x7f5bcc084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:21.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.392 7f5bcb7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bcc0c3820 0x7f5bcc084cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34820/0 (socket says 172.21.2.101:34820) 2021-02-10T11:48:21.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.392 7f5bcb7fe700 1 -- 172.21.2.101:0/3938432560 learned_addr learned my addr 172.21.2.101:0/3938432560 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:21.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.392 7f5bcb7fe700 1 -- 172.21.2.101:0/3938432560 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bcc0a0290 msgr2=0x7f5bcc0852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:21.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.392 7f5bcb7fe700 1 --2- 172.21.2.101:0/3938432560 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bcc0a0290 0x7f5bcc0852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:21.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.392 7f5bcb7fe700 1 -- 172.21.2.101:0/3938432560 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5bcc086270 msgr2=0x7f5bcc086670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:21.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.392 7f5bcb7fe700 1 --2- 172.21.2.101:0/3938432560 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5bcc086270 0x7f5bcc086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:21.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.392 7f5bcb7fe700 1 -- 172.21.2.101:0/3938432560 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bcc086c90 con 0x7f5bcc0c3820 2021-02-10T11:48:21.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.393 7f5bcb7fe700 1 --2- 172.21.2.101:0/3938432560 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bcc0c3820 0x7f5bcc084cb0 secure :-1 s=READY pgs=3222 cs=0 l=1 rev1=1 rx=0x7f5bb400b5b0 tx=0x7f5bb400b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:21.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.393 7f5bc3fff700 1 -- 172.21.2.101:0/3938432560 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5bb400bd40 con 0x7f5bcc0c3820 2021-02-10T11:48:21.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.393 7f5bc3fff700 1 -- 172.21.2.101:0/3938432560 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb4006020 con 0x7f5bcc0c3820 2021-02-10T11:48:21.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.393 7f5bd1b8e700 1 -- 172.21.2.101:0/3938432560 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bcc086c90 con 0x7f5bcc0c3820 2021-02-10T11:48:21.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.393 7f5bc3fff700 1 -- 172.21.2.101:0/3938432560 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5bb40054a0 con 0x7f5bcc0c3820 2021-02-10T11:48:21.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.393 7f5bd1b8e700 1 -- 172.21.2.101:0/3938432560 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bcc1a2f00 con 0x7f5bcc0c3820 2021-02-10T11:48:21.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.393 7f5bc3fff700 1 -- 172.21.2.101:0/3938432560 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5bb4006200 con 0x7f5bcc0c3820 2021-02-10T11:48:21.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.393 7f5bc3fff700 1 --2- 172.21.2.101:0/3938432560 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5bb0018730 0x7f5bb001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:21.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.393 7f5bc3fff700 1 -- 172.21.2.101:0/3938432560 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f5bb4032340 con 0x7f5bcc0c3820 2021-02-10T11:48:21.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.393 7f5bcaffd700 1 --2- 172.21.2.101:0/3938432560 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5bb0018730 0x7f5bb001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:21.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.394 7f5bcaffd700 1 --2- 172.21.2.101:0/3938432560 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5bb0018730 0x7f5bb001ac10 crc :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:21.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.425 7f5bd1b8e700 1 -- 172.21.2.101:0/3938432560 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bcc082a70 con 0x7f5bcc0c3820 2021-02-10T11:48:21.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.428 7f5bc3fff700 1 -- 172.21.2.101:0/3938432560 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5bb4000c60 con 0x7f5bcc0c3820 2021-02-10T11:48:21.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.554 7f5bd1b8e700 1 -- 172.21.2.101:0/3938432560 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f5bcc082a70 con 0x7f5bcc0c3820 2021-02-10T11:48:21.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.554 7f5bc3fff700 1 -- 172.21.2.101:0/3938432560 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v187) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5bb4006020 con 0x7f5bcc0c3820 2021-02-10T11:48:21.551 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:48:21.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.556 7f5bd1b8e700 1 -- 172.21.2.101:0/3938432560 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5bb0018730 msgr2=0x7f5bb001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.556 7f5bd1b8e700 1 --2- 172.21.2.101:0/3938432560 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5bb0018730 0x7f5bb001ac10 crc :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.556 7f5bd1b8e700 1 -- 172.21.2.101:0/3938432560 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bcc0c3820 msgr2=0x7f5bcc084cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.556 7f5bd1b8e700 1 --2- 172.21.2.101:0/3938432560 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bcc0c3820 0x7f5bcc084cb0 secure :-1 s=READY pgs=3222 cs=0 l=1 rev1=1 rx=0x7f5bb400b5b0 tx=0x7f5bb400b950).stop 2021-02-10T11:48:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.556 7f5bd1b8e700 1 -- 172.21.2.101:0/3938432560 shutdown_connections 2021-02-10T11:48:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.556 7f5bd1b8e700 1 --2- 172.21.2.101:0/3938432560 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5bb0018730 0x7f5bb001ac10 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.556 7f5bd1b8e700 1 --2- 172.21.2.101:0/3938432560 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5bcc0a0290 0x7f5bcc0852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.556 7f5bd1b8e700 1 --2- 172.21.2.101:0/3938432560 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5bcc086270 0x7f5bcc086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.556 7f5bd1b8e700 1 --2- 172.21.2.101:0/3938432560 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5bcc0c3820 0x7f5bcc084cb0 unknown :-1 s=CLOSED pgs=3222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.556 7f5bd1b8e700 1 -- 172.21.2.101:0/3938432560 >> 172.21.2.101:0/3938432560 conn(0x7f5bcc0872c0 msgr2=0x7f5bcc095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.556 7f5bd1b8e700 1 -- 172.21.2.101:0/3938432560 shutdown_connections 2021-02-10T11:48:21.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.556 7f5bd1b8e700 1 -- 172.21.2.101:0/3938432560 wait complete. 2021-02-10T11:48:21.561 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --format=json-pretty osd lspools 2021-02-10T11:48:21.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.977 7f4c6fa51700 1 Processor -- start 2021-02-10T11:48:21.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.977 7f4c6fa51700 1 -- start start 2021-02-10T11:48:21.995 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.977 7f4c6fa51700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4c680d1870 0x7f4c681b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:21.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.977 7f4c6fa51700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c680f0650 0x7f4c680cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:21.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.977 7f4c6fa51700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c680f0b10 con 0x7f4c681b0a10 2021-02-10T11:48:21.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.977 7f4c6fa51700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c680e5ed0 con 0x7f4c680d1870 2021-02-10T11:48:21.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.977 7f4c6fa51700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c680d1c70 con 0x7f4c680f0650 2021-02-10T11:48:21.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.978 7f4c6d06c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f4c681b0a10 0x7f4c680cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35320/0 (socket says 172.21.2.101:35320) 2021-02-10T11:48:21.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.978 7f4c6d06c700 1 -- 172.21.2.101:0/1098139430 learned_addr learned my addr 172.21.2.101:0/1098139430 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:21.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.978 7f4c6c86b700 1 -- 172.21.2.101:0/1098139430 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3240643885 0 0) 0x7f4c640009b0 con 0x7f4c681b0a10 2021-02-10T11:48:21.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.978 7f4c6c86b700 1 -- 172.21.2.101:0/1098139430 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c54002fe0 con 0x7f4c681b0a10 2021-02-10T11:48:21.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.978 7f4c6d86d700 1 --2- 172.21.2.101:0/1098139430 >> v2:172.21.2.108:3300/0 conn(0x7f4c680d1870 0x7f4c681b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:21.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.978 7f4c6e06e700 1 --2- 172.21.2.101:0/1098139430 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c680f0650 0x7f4c680cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:21.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.978 7f4c6c86b700 1 -- 172.21.2.101:0/1098139430 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 44828423 0 0) 0x7f4c64001010 con 0x7f4c681b0a10 2021-02-10T11:48:21.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6c86b700 1 -- 172.21.2.101:0/1098139430 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c680f0650 msgr2=0x7f4c680cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:21.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6c86b700 1 --2- 172.21.2.101:0/1098139430 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c680f0650 0x7f4c680cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:21.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6c86b700 1 -- 172.21.2.101:0/1098139430 >> v2:172.21.2.108:3300/0 conn(0x7f4c680d1870 msgr2=0x7f4c681b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:21.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6c86b700 1 --2- 172.21.2.101:0/1098139430 >> v2:172.21.2.108:3300/0 conn(0x7f4c680d1870 0x7f4c681b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:21.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6c86b700 1 -- 172.21.2.101:0/1098139430 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c680d1e90 con 0x7f4c681b0a10 2021-02-10T11:48:21.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6c86b700 1 -- 172.21.2.101:0/1098139430 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f4c64001590 con 0x7f4c681b0a10 2021-02-10T11:48:21.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6c86b700 1 -- 172.21.2.101:0/1098139430 >> v1:172.21.2.101:6789/0 conn(0x7f4c681b0a10 legacy=0x7f4c680cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:21.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6c86b700 1 --2- 172.21.2.101:0/1098139430 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c54003ac0 0x7f4c54005f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:21.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6c86b700 1 --2- 172.21.2.101:0/1098139430 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c5404a7c0 0x7f4c5404cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:21.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6c86b700 1 --2- 172.21.2.101:0/1098139430 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c5404d5a0 0x7f4c5404fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:21.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6c86b700 1 -- 172.21.2.101:0/1098139430 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c540502e0 con 0x7f4c5404a7c0 2021-02-10T11:48:21.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6c86b700 1 -- 172.21.2.101:0/1098139430 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c54050500 con 0x7f4c54003ac0 2021-02-10T11:48:21.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6c86b700 1 -- 172.21.2.101:0/1098139430 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c54050720 con 0x7f4c5404d5a0 2021-02-10T11:48:21.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6d06c700 1 --2- 172.21.2.101:0/1098139430 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c5404a7c0 0x7f4c5404cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:21.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6d86d700 1 --2- 172.21.2.101:0/1098139430 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c54003ac0 0x7f4c54005f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:21.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6e06e700 1 --2- 172.21.2.101:0/1098139430 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c5404d5a0 0x7f4c5404fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:21.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6d06c700 1 -- 172.21.2.101:0/1098139430 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c5404d5a0 msgr2=0x7f4c5404fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:21.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6d06c700 1 --2- 172.21.2.101:0/1098139430 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c5404d5a0 0x7f4c5404fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:22.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6d06c700 1 -- 172.21.2.101:0/1098139430 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c54003ac0 msgr2=0x7f4c54005f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:22.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6d06c700 1 --2- 172.21.2.101:0/1098139430 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c54003ac0 0x7f4c54005f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:22.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6d06c700 1 -- 172.21.2.101:0/1098139430 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c540509c0 con 0x7f4c5404a7c0 2021-02-10T11:48:22.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6d86d700 1 --2- 172.21.2.101:0/1098139430 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c54003ac0 0x7f4c54005f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:48:22.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.979 7f4c6d06c700 1 --2- 172.21.2.101:0/1098139430 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c5404a7c0 0x7f4c5404cca0 secure :-1 s=READY pgs=3224 cs=0 l=1 rev1=1 rx=0x7f4c640017d0 tx=0x7f4c64007cb0).ready entity=mon.0 client_cookie=75e7c5792f0e02cf server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:22.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.980 7f4c6c86b700 1 -- 172.21.2.101:0/1098139430 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4c640013a0 con 0x7f4c5404a7c0 2021-02-10T11:48:22.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.980 7f4c6c86b700 1 -- 172.21.2.101:0/1098139430 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6400a310 con 0x7f4c5404a7c0 2021-02-10T11:48:22.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.980 7f4c6c86b700 1 -- 172.21.2.101:0/1098139430 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4c6400a830 con 0x7f4c5404a7c0 2021-02-10T11:48:22.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.980 7f4c6fa51700 1 -- 172.21.2.101:0/1098139430 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c5404a7c0 msgr2=0x7f4c5404cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:22.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.980 7f4c6fa51700 1 --2- 172.21.2.101:0/1098139430 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c5404a7c0 0x7f4c5404cca0 secure :-1 s=READY pgs=3224 cs=0 l=1 rev1=1 rx=0x7f4c640017d0 tx=0x7f4c64007cb0).stop 2021-02-10T11:48:22.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.980 7f4c6e06e700 1 -- 172.21.2.101:0/1098139430 reap_dead start 2021-02-10T11:48:22.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.980 7f4c6fa51700 1 -- 172.21.2.101:0/1098139430 shutdown_connections 2021-02-10T11:48:22.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.980 7f4c6fa51700 1 -- 172.21.2.101:0/1098139430 >> 172.21.2.101:0/1098139430 conn(0x7f4c680ce450 msgr2=0x7f4c680d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:22.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.980 7f4c6fa51700 1 -- 172.21.2.101:0/1098139430 shutdown_connections 2021-02-10T11:48:22.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.980 7f4c6fa51700 1 -- 172.21.2.101:0/1098139430 wait complete. 2021-02-10T11:48:22.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.995 7f4c6fa51700 1 Processor -- start 2021-02-10T11:48:22.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.995 7f4c6fa51700 1 -- start start 2021-02-10T11:48:22.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.996 7f4c6fa51700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c68195650 0x7f4c680b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:22.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.996 7f4c6fa51700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c680b5b80 0x7f4c681a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:22.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.996 7f4c6fa51700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c680acfa0 0x7f4c681a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:22.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.996 7f4c6fa51700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c680b6010 con 0x7f4c68195650 2021-02-10T11:48:22.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.996 7f4c6fa51700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c681a3a60 con 0x7f4c680acfa0 2021-02-10T11:48:22.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.996 7f4c6fa51700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c681a3c40 con 0x7f4c680b5b80 2021-02-10T11:48:22.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.996 7f4c6d86d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c68195650 0x7f4c680b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:22.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.996 7f4c6d86d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c68195650 0x7f4c680b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34840/0 (socket says 172.21.2.101:34840) 2021-02-10T11:48:22.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.996 7f4c6d86d700 1 -- 172.21.2.101:0/2714777792 learned_addr learned my addr 172.21.2.101:0/2714777792 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:22.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.996 7f4c6e06e700 1 --2- 172.21.2.101:0/2714777792 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c680acfa0 0x7f4c681a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:22.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.996 7f4c6d06c700 1 --2- 172.21.2.101:0/2714777792 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c680b5b80 0x7f4c681a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:22.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.996 7f4c6d86d700 1 -- 172.21.2.101:0/2714777792 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c680b5b80 msgr2=0x7f4c681a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:22.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.996 7f4c6d86d700 1 --2- 172.21.2.101:0/2714777792 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c680b5b80 0x7f4c681a2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:22.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.996 7f4c6d86d700 1 -- 172.21.2.101:0/2714777792 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c680acfa0 msgr2=0x7f4c681a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:22.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.996 7f4c6d86d700 1 --2- 172.21.2.101:0/2714777792 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c680acfa0 0x7f4c681a3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:22.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.996 7f4c6d86d700 1 -- 172.21.2.101:0/2714777792 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c680bcae0 con 0x7f4c68195650 2021-02-10T11:48:22.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.996 7f4c6e06e700 1 --2- 172.21.2.101:0/2714777792 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c680acfa0 0x7f4c681a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:48:22.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.997 7f4c6d06c700 1 --2- 172.21.2.101:0/2714777792 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c680b5b80 0x7f4c681a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:48:22.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.997 7f4c6d86d700 1 --2- 172.21.2.101:0/2714777792 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c68195650 0x7f4c680b5560 secure :-1 s=READY pgs=3225 cs=0 l=1 rev1=1 rx=0x7f4c5c006f60 tx=0x7f4c5c00b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:22.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.997 7f4c5a7fc700 1 -- 172.21.2.101:0/2714777792 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4c5c006bb0 con 0x7f4c68195650 2021-02-10T11:48:22.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.997 7f4c5a7fc700 1 -- 172.21.2.101:0/2714777792 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5c006340 con 0x7f4c68195650 2021-02-10T11:48:22.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.997 7f4c5a7fc700 1 -- 172.21.2.101:0/2714777792 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4c5c002030 con 0x7f4c68195650 2021-02-10T11:48:22.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.997 7f4c6fa51700 1 -- 172.21.2.101:0/2714777792 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c680bcae0 con 0x7f4c68195650 2021-02-10T11:48:22.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.997 7f4c6fa51700 1 -- 172.21.2.101:0/2714777792 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c680bcae0 con 0x7f4c68195650 2021-02-10T11:48:22.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.997 7f4c5a7fc700 1 -- 172.21.2.101:0/2714777792 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4c5c006b50 con 0x7f4c68195650 2021-02-10T11:48:22.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.998 7f4c5a7fc700 1 --2- 172.21.2.101:0/2714777792 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4c500186c0 0x7f4c5001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:22.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.998 7f4c5a7fc700 1 -- 172.21.2.101:0/2714777792 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f4c5c032450 con 0x7f4c68195650 2021-02-10T11:48:22.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.998 7f4c6d06c700 1 --2- 172.21.2.101:0/2714777792 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4c500186c0 0x7f4c5001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:22.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:21.998 7f4c6d06c700 1 --2- 172.21.2.101:0/2714777792 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4c500186c0 0x7f4c5001abc0 crc :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:22.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.028 7f4c6fa51700 1 -- 172.21.2.101:0/2714777792 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c680b6010 con 0x7f4c68195650 2021-02-10T11:48:22.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.030 7f4c5a7fc700 1 -- 172.21.2.101:0/2714777792 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4c5c000c90 con 0x7f4c68195650 2021-02-10T11:48:22.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.155 7f4c6fa51700 1 -- 172.21.2.101:0/2714777792 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4c680b6010 con 0x7f4c68195650 2021-02-10T11:48:22.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.156 7f4c5a7fc700 1 -- 172.21.2.101:0/2714777792 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v187) v1 ==== 84+0+64 (secure 0 0 0) 0x7f4c5c006340 con 0x7f4c68195650 2021-02-10T11:48:22.152 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:48:22.154 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:48:22.154 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:22.154 INFO:teuthology.orchestra.run.gibba001.stdout: "poolnum": 1, 2021-02-10T11:48:22.154 INFO:teuthology.orchestra.run.gibba001.stdout: "poolname": "rbd" 2021-02-10T11:48:22.154 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:22.154 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:48:22.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.157 7f4c6fa51700 1 -- 172.21.2.101:0/2714777792 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4c500186c0 msgr2=0x7f4c5001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:22.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.157 7f4c6fa51700 1 --2- 172.21.2.101:0/2714777792 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4c500186c0 0x7f4c5001abc0 crc :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:22.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.157 7f4c6fa51700 1 -- 172.21.2.101:0/2714777792 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c68195650 msgr2=0x7f4c680b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:22.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.157 7f4c6fa51700 1 --2- 172.21.2.101:0/2714777792 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c68195650 0x7f4c680b5560 secure :-1 s=READY pgs=3225 cs=0 l=1 rev1=1 rx=0x7f4c5c006f60 tx=0x7f4c5c00b970).stop 2021-02-10T11:48:22.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.158 7f4c6fa51700 1 -- 172.21.2.101:0/2714777792 shutdown_connections 2021-02-10T11:48:22.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.158 7f4c6fa51700 1 --2- 172.21.2.101:0/2714777792 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4c500186c0 0x7f4c5001abc0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:22.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.158 7f4c6fa51700 1 --2- 172.21.2.101:0/2714777792 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4c680b5b80 0x7f4c681a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:22.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.158 7f4c6fa51700 1 --2- 172.21.2.101:0/2714777792 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4c680acfa0 0x7f4c681a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:22.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.158 7f4c6fa51700 1 --2- 172.21.2.101:0/2714777792 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4c68195650 0x7f4c680b5560 unknown :-1 s=CLOSED pgs=3225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:22.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.158 7f4c6fa51700 1 -- 172.21.2.101:0/2714777792 >> 172.21.2.101:0/2714777792 conn(0x7f4c680b3040 msgr2=0x7f4c680c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:22.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.158 7f4c6fa51700 1 -- 172.21.2.101:0/2714777792 shutdown_connections 2021-02-10T11:48:22.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.158 7f4c6fa51700 1 -- 172.21.2.101:0/2714777792 wait complete. 2021-02-10T11:48:22.163 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-02-10T11:48:22.163 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-02-10T11:48:22.595 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.582 7f9721748700 1 Processor -- start 2021-02-10T11:48:22.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.582 7f9721748700 1 -- start start 2021-02-10T11:48:22.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.583 7f9721748700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f971c0d1870 0x7f971c1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:22.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.583 7f9721748700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f971c0cff70 0x7f971c109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:22.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.583 7f9721748700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f971c0fe1c0 con 0x7f971c1b0ae0 2021-02-10T11:48:22.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.583 7f9721748700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f971c109ad0 con 0x7f971c0cff70 2021-02-10T11:48:22.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.583 7f9721748700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f971c109750 con 0x7f971c0d1870 2021-02-10T11:48:22.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.583 7f971bd4a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f971c0cff70 0x7f971c109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:22.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.583 7f971ad48700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f971c1b0ae0 0x7f971c108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35340/0 (socket says 172.21.2.101:35340) 2021-02-10T11:48:22.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.583 7f971ad48700 1 -- 172.21.2.101:0/535054932 learned_addr learned my addr 172.21.2.101:0/535054932 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:22.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.584 7f971bd4a700 1 -- 172.21.2.101:0/535054932 >> v2:172.21.2.108:3300/0 conn(0x7f971c0d1870 msgr2=0x7f971c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:22.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.584 7f971bd4a700 1 --2- 172.21.2.101:0/535054932 >> v2:172.21.2.108:3300/0 conn(0x7f971c0d1870 0x7f971c1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:22.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.584 7f971bd4a700 1 -- 172.21.2.101:0/535054932 >> v1:172.21.2.101:6789/0 conn(0x7f971c1b0ae0 legacy=0x7f971c108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:22.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.584 7f971bd4a700 1 -- 172.21.2.101:0/535054932 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971c088070 con 0x7f971c0cff70 2021-02-10T11:48:22.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.584 7f971bd4a700 1 --2- 172.21.2.101:0/535054932 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f971c0cff70 0x7f971c109130 secure :-1 s=READY pgs=3227 cs=0 l=1 rev1=1 rx=0x7f9714006e30 tx=0x7f9714009e60).ready entity=mon.0 client_cookie=45bc9b38abf415f6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:22.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.584 7f971a547700 1 -- 172.21.2.101:0/535054932 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9714006a50 con 0x7f971c0cff70 2021-02-10T11:48:22.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.584 7f971a547700 1 -- 172.21.2.101:0/535054932 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971400b300 con 0x7f971c0cff70 2021-02-10T11:48:22.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.584 7f971a547700 1 -- 172.21.2.101:0/535054932 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f971400b820 con 0x7f971c0cff70 2021-02-10T11:48:22.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.584 7f9721748700 1 -- 172.21.2.101:0/535054932 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f971c0cff70 msgr2=0x7f971c109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:22.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.584 7f9721748700 1 --2- 172.21.2.101:0/535054932 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f971c0cff70 0x7f971c109130 secure :-1 s=READY pgs=3227 cs=0 l=1 rev1=1 rx=0x7f9714006e30 tx=0x7f9714009e60).stop 2021-02-10T11:48:22.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.585 7f9721748700 1 -- 172.21.2.101:0/535054932 shutdown_connections 2021-02-10T11:48:22.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.585 7f9721748700 1 --2- 172.21.2.101:0/535054932 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f971c0cff70 0x7f971c109130 unknown :-1 s=CLOSED pgs=3227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:22.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.585 7f9721748700 1 --2- 172.21.2.101:0/535054932 >> v2:172.21.2.108:3300/0 conn(0x7f971c0d1870 0x7f971c1b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:22.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.585 7f9721748700 1 -- 172.21.2.101:0/535054932 >> 172.21.2.101:0/535054932 conn(0x7f971c0ce420 msgr2=0x7f971c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:22.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.585 7f9721748700 1 -- 172.21.2.101:0/535054932 shutdown_connections 2021-02-10T11:48:22.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.585 7f9721748700 1 -- 172.21.2.101:0/535054932 wait complete. 2021-02-10T11:48:22.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.600 7f9721748700 1 Processor -- start 2021-02-10T11:48:22.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.600 7f9721748700 1 -- start start 2021-02-10T11:48:22.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.600 7f9721748700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f971c1906c0 0x7f971c1a78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:22.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.600 7f9721748700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f971c1a7f10 0x7f971c195400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:22.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.600 7f9721748700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f971c190ac0 0x7f971c195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:22.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.600 7f9721748700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c196040 con 0x7f971c190ac0 2021-02-10T11:48:22.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.600 7f9721748700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c0c2730 con 0x7f971c1906c0 2021-02-10T11:48:22.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.600 7f9721748700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f971c0c2950 con 0x7f971c1a7f10 2021-02-10T11:48:22.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.600 7f971bd4a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f971c190ac0 0x7f971c195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:22.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.600 7f971bd4a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f971c190ac0 0x7f971c195a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34850/0 (socket says 172.21.2.101:34850) 2021-02-10T11:48:22.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.600 7f971bd4a700 1 -- 172.21.2.101:0/3714821101 learned_addr learned my addr 172.21.2.101:0/3714821101 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:22.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.601 7f971bd4a700 1 -- 172.21.2.101:0/3714821101 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f971c1a7f10 msgr2=0x7f971c195400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:22.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.601 7f971bd4a700 1 --2- 172.21.2.101:0/3714821101 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f971c1a7f10 0x7f971c195400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:22.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.601 7f971bd4a700 1 -- 172.21.2.101:0/3714821101 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f971c1906c0 msgr2=0x7f971c1a78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:22.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.601 7f971bd4a700 1 --2- 172.21.2.101:0/3714821101 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f971c1906c0 0x7f971c1a78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:22.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.601 7f971bd4a700 1 -- 172.21.2.101:0/3714821101 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971c09ca10 con 0x7f971c190ac0 2021-02-10T11:48:22.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.601 7f971bd4a700 1 --2- 172.21.2.101:0/3714821101 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f971c190ac0 0x7f971c195a20 secure :-1 s=READY pgs=3228 cs=0 l=1 rev1=1 rx=0x7f971400b970 tx=0x7f971400bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:22.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.601 7f9703fff700 1 -- 172.21.2.101:0/3714821101 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9714009380 con 0x7f971c190ac0 2021-02-10T11:48:22.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.601 7f9703fff700 1 -- 172.21.2.101:0/3714821101 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9714005820 con 0x7f971c190ac0 2021-02-10T11:48:22.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.601 7f9703fff700 1 -- 172.21.2.101:0/3714821101 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9714005d40 con 0x7f971c190ac0 2021-02-10T11:48:22.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.601 7f9721748700 1 -- 172.21.2.101:0/3714821101 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f971c09ca10 con 0x7f971c190ac0 2021-02-10T11:48:22.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.601 7f9721748700 1 -- 172.21.2.101:0/3714821101 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f971c09ca10 con 0x7f971c190ac0 2021-02-10T11:48:22.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.602 7f9703fff700 1 -- 172.21.2.101:0/3714821101 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9714005a00 con 0x7f971c190ac0 2021-02-10T11:48:22.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.602 7f9703fff700 1 --2- 172.21.2.101:0/3714821101 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96fc0186c0 0x7f96fc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:22.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.602 7f9703fff700 1 -- 172.21.2.101:0/3714821101 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f9714032610 con 0x7f971c190ac0 2021-02-10T11:48:22.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.602 7f971b549700 1 --2- 172.21.2.101:0/3714821101 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96fc0186c0 0x7f96fc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:22.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.603 7f971b549700 1 --2- 172.21.2.101:0/3714821101 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96fc0186c0 0x7f96fc01abc0 crc :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:22.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.633 7f9721748700 1 -- 172.21.2.101:0/3714821101 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f971c082a70 con 0x7f971c190ac0 2021-02-10T11:48:22.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.635 7f9703fff700 1 -- 172.21.2.101:0/3714821101 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9714009380 con 0x7f971c190ac0 2021-02-10T11:48:22.762 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:22.766 7f9721748700 1 -- 172.21.2.101:0/3714821101 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"} v 0) v1 -- 0x7f971c082a70 con 0x7f971c190ac0 2021-02-10T11:48:23.171 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.174 7f9703fff700 1 -- 172.21.2.101:0/3714821101 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"}]=0 pool 'cephfs_metadata' created v188) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9714005a00 con 0x7f971c190ac0 2021-02-10T11:48:23.178 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' created 2021-02-10T11:48:23.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.184 7f9721748700 1 -- 172.21.2.101:0/3714821101 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96fc0186c0 msgr2=0x7f96fc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:23.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.184 7f9721748700 1 --2- 172.21.2.101:0/3714821101 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96fc0186c0 0x7f96fc01abc0 crc :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:23.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.184 7f9721748700 1 -- 172.21.2.101:0/3714821101 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f971c190ac0 msgr2=0x7f971c195a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:23.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.184 7f9721748700 1 --2- 172.21.2.101:0/3714821101 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f971c190ac0 0x7f971c195a20 secure :-1 s=READY pgs=3228 cs=0 l=1 rev1=1 rx=0x7f971400b970 tx=0x7f971400bd10).stop 2021-02-10T11:48:23.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.184 7f9721748700 1 -- 172.21.2.101:0/3714821101 shutdown_connections 2021-02-10T11:48:23.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.184 7f9721748700 1 --2- 172.21.2.101:0/3714821101 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96fc0186c0 0x7f96fc01abc0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:23.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.184 7f9721748700 1 --2- 172.21.2.101:0/3714821101 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f971c1a7f10 0x7f971c195400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:23.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.184 7f9721748700 1 --2- 172.21.2.101:0/3714821101 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f971c190ac0 0x7f971c195a20 unknown :-1 s=CLOSED pgs=3228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:23.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.184 7f9721748700 1 --2- 172.21.2.101:0/3714821101 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f971c1906c0 0x7f971c1a78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:23.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.184 7f9721748700 1 -- 172.21.2.101:0/3714821101 >> 172.21.2.101:0/3714821101 conn(0x7f971c0b6070 msgr2=0x7f971c095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:23.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.184 7f9721748700 1 -- 172.21.2.101:0/3714821101 shutdown_connections 2021-02-10T11:48:23.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.184 7f9721748700 1 -- 172.21.2.101:0/3714821101 wait complete. 2021-02-10T11:48:23.189 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_data 8 2021-02-10T11:48:23.621 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.608 7fc9dc0f1700 1 Processor -- start 2021-02-10T11:48:23.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.608 7fc9dc0f1700 1 -- start start 2021-02-10T11:48:23.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.609 7fc9dc0f1700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc9d40d1870 0x7fc9d41b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:23.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.609 7fc9dc0f1700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc9d40cff70 0x7fc9d4109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:23.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.609 7fc9dc0f1700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9d40fe1c0 con 0x7fc9d41b0ae0 2021-02-10T11:48:23.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.609 7fc9dc0f1700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d4109ad0 con 0x7fc9d40d1870 2021-02-10T11:48:23.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.609 7fc9dc0f1700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d4109750 con 0x7fc9d40cff70 2021-02-10T11:48:23.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.609 7fc9d9f0d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc9d40d1870 0x7fc9d41b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:23.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.609 7fc9d970c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc9d41b0ae0 0x7fc9d4108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35350/0 (socket says 172.21.2.101:35350) 2021-02-10T11:48:23.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.609 7fc9d970c700 1 -- 172.21.2.101:0/2519436207 learned_addr learned my addr 172.21.2.101:0/2519436207 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:23.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.609 7fc9da70e700 1 --2- 172.21.2.101:0/2519436207 >> v2:172.21.2.108:3300/0 conn(0x7fc9d40cff70 0x7fc9d4109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:23.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.609 7fc9d9f0d700 1 -- 172.21.2.101:0/2519436207 >> v2:172.21.2.108:3300/0 conn(0x7fc9d40cff70 msgr2=0x7fc9d4109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:23.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.609 7fc9d9f0d700 1 --2- 172.21.2.101:0/2519436207 >> v2:172.21.2.108:3300/0 conn(0x7fc9d40cff70 0x7fc9d4109130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:23.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.609 7fc9d9f0d700 1 -- 172.21.2.101:0/2519436207 >> v1:172.21.2.101:6789/0 conn(0x7fc9d41b0ae0 legacy=0x7fc9d4108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:23.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.609 7fc9d9f0d700 1 -- 172.21.2.101:0/2519436207 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d4088070 con 0x7fc9d40d1870 2021-02-10T11:48:23.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.609 7fc9da70e700 1 --2- 172.21.2.101:0/2519436207 >> v2:172.21.2.108:3300/0 conn(0x7fc9d40cff70 0x7fc9d4109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:48:23.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.609 7fc9d9f0d700 1 --2- 172.21.2.101:0/2519436207 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9d40d1870 0x7fc9d41b04c0 secure :-1 s=READY pgs=3230 cs=0 l=1 rev1=1 rx=0x7fc9c8006650 tx=0x7fc9c8009a80).ready entity=mon.0 client_cookie=2da87e9deb8025ce server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:23.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.610 7fc9d8f0b700 1 -- 172.21.2.101:0/2519436207 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc9c80009b0 con 0x7fc9d40d1870 2021-02-10T11:48:23.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.610 7fc9d8f0b700 1 -- 172.21.2.101:0/2519436207 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c800c0c0 con 0x7fc9d40d1870 2021-02-10T11:48:23.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.610 7fc9d8f0b700 1 -- 172.21.2.101:0/2519436207 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc9c800c5e0 con 0x7fc9d40d1870 2021-02-10T11:48:23.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.610 7fc9dc0f1700 1 -- 172.21.2.101:0/2519436207 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9d40d1870 msgr2=0x7fc9d41b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:23.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.610 7fc9dc0f1700 1 --2- 172.21.2.101:0/2519436207 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9d40d1870 0x7fc9d41b04c0 secure :-1 s=READY pgs=3230 cs=0 l=1 rev1=1 rx=0x7fc9c8006650 tx=0x7fc9c8009a80).stop 2021-02-10T11:48:23.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.610 7fc9dc0f1700 1 -- 172.21.2.101:0/2519436207 shutdown_connections 2021-02-10T11:48:23.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.610 7fc9dc0f1700 1 --2- 172.21.2.101:0/2519436207 >> v2:172.21.2.108:3300/0 conn(0x7fc9d40cff70 0x7fc9d4109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:23.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.610 7fc9dc0f1700 1 --2- 172.21.2.101:0/2519436207 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9d40d1870 0x7fc9d41b04c0 unknown :-1 s=CLOSED pgs=3230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:23.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.610 7fc9dc0f1700 1 -- 172.21.2.101:0/2519436207 >> 172.21.2.101:0/2519436207 conn(0x7fc9d40ce420 msgr2=0x7fc9d40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:23.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.611 7fc9dc0f1700 1 -- 172.21.2.101:0/2519436207 shutdown_connections 2021-02-10T11:48:23.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.611 7fc9dc0f1700 1 -- 172.21.2.101:0/2519436207 wait complete. 2021-02-10T11:48:23.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.625 7fc9dc0f1700 1 Processor -- start 2021-02-10T11:48:23.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.625 7fc9dc0f1700 1 -- start start 2021-02-10T11:48:23.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.626 7fc9dc0f1700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9d40c3820 0x7fc9d4084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:23.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.626 7fc9dc0f1700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9d40a0290 0x7fc9d40852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:23.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.626 7fc9dc0f1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9d4086270 0x7fc9d4086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:23.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.626 7fc9dc0f1700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d40c0700 con 0x7fc9d4086270 2021-02-10T11:48:23.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.626 7fc9dc0f1700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d40858f0 con 0x7fc9d40c3820 2021-02-10T11:48:23.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.626 7fc9dc0f1700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d4085b10 con 0x7fc9d40a0290 2021-02-10T11:48:23.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.626 7fc9da70e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9d4086270 0x7fc9d4086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:23.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.626 7fc9da70e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9d4086270 0x7fc9d4086670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34862/0 (socket says 172.21.2.101:34862) 2021-02-10T11:48:23.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.626 7fc9da70e700 1 -- 172.21.2.101:0/1922480729 learned_addr learned my addr 172.21.2.101:0/1922480729 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:23.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.626 7fc9da70e700 1 -- 172.21.2.101:0/1922480729 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9d40a0290 msgr2=0x7fc9d40852d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:23.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.626 7fc9da70e700 1 --2- 172.21.2.101:0/1922480729 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9d40a0290 0x7fc9d40852d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:23.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.626 7fc9da70e700 1 -- 172.21.2.101:0/1922480729 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9d40c3820 msgr2=0x7fc9d4084cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:23.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.626 7fc9da70e700 1 --2- 172.21.2.101:0/1922480729 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9d40c3820 0x7fc9d4084cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:23.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.626 7fc9da70e700 1 -- 172.21.2.101:0/1922480729 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d4086c90 con 0x7fc9d4086270 2021-02-10T11:48:23.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.626 7fc9da70e700 1 --2- 172.21.2.101:0/1922480729 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9d4086270 0x7fc9d4086670 secure :-1 s=READY pgs=3231 cs=0 l=1 rev1=1 rx=0x7fc9cc009980 tx=0x7fc9cc00bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:23.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.626 7fc9c67fc700 1 -- 172.21.2.101:0/1922480729 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc9cc006020 con 0x7fc9d4086270 2021-02-10T11:48:23.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.626 7fc9c67fc700 1 -- 172.21.2.101:0/1922480729 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9cc006320 con 0x7fc9d4086270 2021-02-10T11:48:23.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.626 7fc9c67fc700 1 -- 172.21.2.101:0/1922480729 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc9cc0056b0 con 0x7fc9d4086270 2021-02-10T11:48:23.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.626 7fc9dc0f1700 1 -- 172.21.2.101:0/1922480729 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d4086c90 con 0x7fc9d4086270 2021-02-10T11:48:23.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.626 7fc9dc0f1700 1 -- 172.21.2.101:0/1922480729 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d4086c90 con 0x7fc9d4086270 2021-02-10T11:48:23.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.627 7fc9c67fc700 1 -- 172.21.2.101:0/1922480729 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc9cc006020 con 0x7fc9d4086270 2021-02-10T11:48:23.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.627 7fc9c67fc700 1 --2- 172.21.2.101:0/1922480729 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9bc0186d0 0x7fc9bc01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:23.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.627 7fc9c67fc700 1 -- 172.21.2.101:0/1922480729 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5107+0+0 (secure 0 0 0) 0x7fc9cc0327c0 con 0x7fc9d4086270 2021-02-10T11:48:23.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.627 7fc9d9f0d700 1 --2- 172.21.2.101:0/1922480729 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9bc0186d0 0x7fc9bc01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:23.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.627 7fc9d9f0d700 1 --2- 172.21.2.101:0/1922480729 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9bc0186d0 0x7fc9bc01abd0 crc :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:23.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.659 7fc9dc0f1700 1 -- 172.21.2.101:0/1922480729 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d4082a70 con 0x7fc9d4086270 2021-02-10T11:48:23.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.661 7fc9c67fc700 1 -- 172.21.2.101:0/1922480729 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc9cc00be10 con 0x7fc9d4086270 2021-02-10T11:48:23.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:23.794 7fc9dc0f1700 1 -- 172.21.2.101:0/1922480729 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"} v 0) v1 -- 0x7fc9d4082a70 con 0x7fc9d4086270 2021-02-10T11:48:24.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.177 7fc9c67fc700 1 -- 172.21.2.101:0/1922480729 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"}]=0 pool 'cephfs_data' created v189) v1 ==== 125+0+0 (secure 0 0 0) 0x7fc9cc006020 con 0x7fc9d4086270 2021-02-10T11:48:24.206 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' created 2021-02-10T11:48:24.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.212 7fc9dc0f1700 1 -- 172.21.2.101:0/1922480729 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9bc0186d0 msgr2=0x7fc9bc01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:24.208 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.212 7fc9dc0f1700 1 --2- 172.21.2.101:0/1922480729 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9bc0186d0 0x7fc9bc01abd0 crc :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:24.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.212 7fc9dc0f1700 1 -- 172.21.2.101:0/1922480729 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9d4086270 msgr2=0x7fc9d4086670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:24.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.212 7fc9dc0f1700 1 --2- 172.21.2.101:0/1922480729 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9d4086270 0x7fc9d4086670 secure :-1 s=READY pgs=3231 cs=0 l=1 rev1=1 rx=0x7fc9cc009980 tx=0x7fc9cc00bcd0).stop 2021-02-10T11:48:24.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.212 7fc9dc0f1700 1 -- 172.21.2.101:0/1922480729 shutdown_connections 2021-02-10T11:48:24.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.212 7fc9dc0f1700 1 --2- 172.21.2.101:0/1922480729 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9bc0186d0 0x7fc9bc01abd0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:24.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.212 7fc9dc0f1700 1 --2- 172.21.2.101:0/1922480729 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9d40a0290 0x7fc9d40852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:24.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.212 7fc9dc0f1700 1 --2- 172.21.2.101:0/1922480729 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9d4086270 0x7fc9d4086670 unknown :-1 s=CLOSED pgs=3231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:24.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.212 7fc9dc0f1700 1 --2- 172.21.2.101:0/1922480729 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9d40c3820 0x7fc9d4084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:24.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.212 7fc9dc0f1700 1 -- 172.21.2.101:0/1922480729 >> 172.21.2.101:0/1922480729 conn(0x7fc9d40872c0 msgr2=0x7fc9d4095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:24.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.212 7fc9dc0f1700 1 -- 172.21.2.101:0/1922480729 shutdown_connections 2021-02-10T11:48:24.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.212 7fc9dc0f1700 1 -- 172.21.2.101:0/1922480729 wait complete. 2021-02-10T11:48:24.217 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-02-10T11:48:24.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.637 7f8fc4f18700 1 Processor -- start 2021-02-10T11:48:24.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.637 7f8fc4f18700 1 -- start start 2021-02-10T11:48:24.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.637 7f8fc4f18700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8fc00d1870 0x7f8fc01b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:24.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.637 7f8fc4f18700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8fc01b0ae0 0x7f8fc0108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:24.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.637 7f8fc4f18700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8fc00d1c70 0x7f8fc0108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:24.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.637 7f8fc4f18700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc00fe1c0 con 0x7f8fc00d1c70 2021-02-10T11:48:24.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.637 7f8fc4f18700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc0109a60 con 0x7f8fc00d1870 2021-02-10T11:48:24.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.637 7f8fc4f18700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc0088070 con 0x7f8fc01b0ae0 2021-02-10T11:48:24.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.638 7f8fbf314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8fc00d1c70 0x7f8fc0108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:24.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.638 7f8fbf314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8fc00d1c70 0x7f8fc0108f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34874/0 (socket says 172.21.2.101:34874) 2021-02-10T11:48:24.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.638 7f8fbf314700 1 -- 172.21.2.101:0/3574213699 learned_addr learned my addr 172.21.2.101:0/3574213699 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:24.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.638 7f8fbeb13700 1 --2- 172.21.2.101:0/3574213699 >> v2:172.21.2.108:3300/0 conn(0x7f8fc00d1870 0x7f8fc01b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:24.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.638 7f8fbe312700 1 --2- 172.21.2.101:0/3574213699 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8fc01b0ae0 0x7f8fc0108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:24.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.638 7f8fbf314700 1 -- 172.21.2.101:0/3574213699 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8fc01b0ae0 msgr2=0x7f8fc0108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:24.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.638 7f8fbf314700 1 --2- 172.21.2.101:0/3574213699 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8fc01b0ae0 0x7f8fc0108940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:24.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.638 7f8fbf314700 1 -- 172.21.2.101:0/3574213699 >> v2:172.21.2.108:3300/0 conn(0x7f8fc00d1870 msgr2=0x7f8fc01b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:24.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.638 7f8fbf314700 1 --2- 172.21.2.101:0/3574213699 >> v2:172.21.2.108:3300/0 conn(0x7f8fc00d1870 0x7f8fc01b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:24.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.638 7f8fbf314700 1 -- 172.21.2.101:0/3574213699 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc0109580 con 0x7f8fc00d1c70 2021-02-10T11:48:24.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.638 7f8fbf314700 1 --2- 172.21.2.101:0/3574213699 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8fc00d1c70 0x7f8fc0108f60 secure :-1 s=READY pgs=3232 cs=0 l=1 rev1=1 rx=0x7f8fb8006f00 tx=0x7f8fb8009e90).ready entity=mon.0 client_cookie=2f6d6938b27907c7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:24.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.639 7f8fbdb11700 1 -- 172.21.2.101:0/3574213699 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8fb800b300 con 0x7f8fc00d1c70 2021-02-10T11:48:24.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.639 7f8fbdb11700 1 -- 172.21.2.101:0/3574213699 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb800b640 con 0x7f8fc00d1c70 2021-02-10T11:48:24.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.639 7f8fbdb11700 1 -- 172.21.2.101:0/3574213699 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8fb800ba90 con 0x7f8fc00d1c70 2021-02-10T11:48:24.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.639 7f8fc4f18700 1 -- 172.21.2.101:0/3574213699 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8fc00d1c70 msgr2=0x7f8fc0108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:24.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.639 7f8fc4f18700 1 --2- 172.21.2.101:0/3574213699 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8fc00d1c70 0x7f8fc0108f60 secure :-1 s=READY pgs=3232 cs=0 l=1 rev1=1 rx=0x7f8fb8006f00 tx=0x7f8fb8009e90).stop 2021-02-10T11:48:24.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.639 7f8fc4f18700 1 -- 172.21.2.101:0/3574213699 shutdown_connections 2021-02-10T11:48:24.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.639 7f8fc4f18700 1 --2- 172.21.2.101:0/3574213699 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8fc01b0ae0 0x7f8fc0108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:24.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.639 7f8fc4f18700 1 --2- 172.21.2.101:0/3574213699 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8fc00d1c70 0x7f8fc0108f60 unknown :-1 s=CLOSED pgs=3232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:24.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.639 7f8fc4f18700 1 --2- 172.21.2.101:0/3574213699 >> v2:172.21.2.108:3300/0 conn(0x7f8fc00d1870 0x7f8fc01b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:24.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.639 7f8fc4f18700 1 -- 172.21.2.101:0/3574213699 >> 172.21.2.101:0/3574213699 conn(0x7f8fc00ce420 msgr2=0x7f8fc00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:24.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.640 7f8fc4f18700 1 -- 172.21.2.101:0/3574213699 shutdown_connections 2021-02-10T11:48:24.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.640 7f8fc4f18700 1 -- 172.21.2.101:0/3574213699 wait complete. 2021-02-10T11:48:24.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.654 7f8fc4f18700 1 Processor -- start 2021-02-10T11:48:24.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.654 7f8fc4f18700 1 -- start start 2021-02-10T11:48:24.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.655 7f8fc4f18700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8fc008cde0 0x7f8fc008d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:24.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.655 7f8fc4f18700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8fc008d800 0x7f8fc008dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:24.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.655 7f8fc4f18700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8fc00b4d30 0x7f8fc00b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:24.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.655 7f8fc4f18700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc008c8e0 con 0x7f8fc008d800 2021-02-10T11:48:24.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.655 7f8fc4f18700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc00b6740 con 0x7f8fc008cde0 2021-02-10T11:48:24.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.655 7f8fc4f18700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc00b5750 con 0x7f8fc00b4d30 2021-02-10T11:48:24.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.655 7f8fbe312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8fc008d800 0x7f8fc008dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:24.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.655 7f8fbe312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8fc008d800 0x7f8fc008dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34880/0 (socket says 172.21.2.101:34880) 2021-02-10T11:48:24.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.655 7f8fbe312700 1 -- 172.21.2.101:0/3445369559 learned_addr learned my addr 172.21.2.101:0/3445369559 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:24.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.655 7f8fbe312700 1 -- 172.21.2.101:0/3445369559 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8fc00b4d30 msgr2=0x7f8fc00b5130 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:24.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.655 7f8fbe312700 1 --2- 172.21.2.101:0/3445369559 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8fc00b4d30 0x7f8fc00b5130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:24.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.655 7f8fbe312700 1 -- 172.21.2.101:0/3445369559 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8fc008cde0 msgr2=0x7f8fc008d1e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:24.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.655 7f8fbe312700 1 --2- 172.21.2.101:0/3445369559 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8fc008cde0 0x7f8fc008d1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:24.655 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.655 7f8fbe312700 1 -- 172.21.2.101:0/3445369559 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc00b5970 con 0x7f8fc008d800 2021-02-10T11:48:24.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.655 7f8fbe312700 1 --2- 172.21.2.101:0/3445369559 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8fc008d800 0x7f8fc008dc00 secure :-1 s=READY pgs=3233 cs=0 l=1 rev1=1 rx=0x7f8fa8006f60 tx=0x7f8fa800b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:24.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.655 7f8fb77fe700 1 -- 172.21.2.101:0/3445369559 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8fa800bd60 con 0x7f8fc008d800 2021-02-10T11:48:24.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.656 7f8fb77fe700 1 -- 172.21.2.101:0/3445369559 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa8000c90 con 0x7f8fc008d800 2021-02-10T11:48:24.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.656 7f8fb77fe700 1 -- 172.21.2.101:0/3445369559 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8fa8002220 con 0x7f8fc008d800 2021-02-10T11:48:24.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.656 7f8fc4f18700 1 -- 172.21.2.101:0/3445369559 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc00b5970 con 0x7f8fc008d800 2021-02-10T11:48:24.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.656 7f8fc4f18700 1 -- 172.21.2.101:0/3445369559 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc00b5f00 con 0x7f8fc008d800 2021-02-10T11:48:24.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.656 7f8fb77fe700 1 -- 172.21.2.101:0/3445369559 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8fa80052e0 con 0x7f8fc008d800 2021-02-10T11:48:24.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.656 7f8fb77fe700 1 --2- 172.21.2.101:0/3445369559 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8fa40186c0 0x7f8fa401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:24.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.656 7f8fb77fe700 1 -- 172.21.2.101:0/3445369559 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5457+0+0 (secure 0 0 0) 0x7f8fa8032370 con 0x7f8fc008d800 2021-02-10T11:48:24.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.656 7f8fbeb13700 1 --2- 172.21.2.101:0/3445369559 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8fa40186c0 0x7f8fa401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:24.656 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.657 7f8fbeb13700 1 --2- 172.21.2.101:0/3445369559 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8fa40186c0 0x7f8fa401abc0 crc :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:24.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.687 7f8fc4f18700 1 -- 172.21.2.101:0/3445369559 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc00b6740 con 0x7f8fc008d800 2021-02-10T11:48:24.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.690 7f8fb77fe700 1 -- 172.21.2.101:0/3445369559 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8fa8006b00 con 0x7f8fc008d800 2021-02-10T11:48:24.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.822 7f8fc4f18700 1 -- 172.21.2.101:0/3445369559 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"} v 0) v1 -- 0x7f8fc00b6740 con 0x7f8fc008d800 2021-02-10T11:48:24.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.826 7f8fb77fe700 1 -- 172.21.2.101:0/3445369559 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"}]=0 new fs with metadata pool 40 and data pool 41 v161) v1 ==== 189+0+0 (secure 0 0 0) 0x7f8fa80052e0 con 0x7f8fc008d800 2021-02-10T11:48:24.826 INFO:teuthology.orchestra.run.gibba001.stderr:new fs with metadata pool 40 and data pool 41 2021-02-10T11:48:24.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.831 7f8fc4f18700 1 -- 172.21.2.101:0/3445369559 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8fa40186c0 msgr2=0x7f8fa401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:24.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.831 7f8fc4f18700 1 --2- 172.21.2.101:0/3445369559 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8fa40186c0 0x7f8fa401abc0 crc :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:24.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.831 7f8fc4f18700 1 -- 172.21.2.101:0/3445369559 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8fc008d800 msgr2=0x7f8fc008dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:24.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.831 7f8fc4f18700 1 --2- 172.21.2.101:0/3445369559 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8fc008d800 0x7f8fc008dc00 secure :-1 s=READY pgs=3233 cs=0 l=1 rev1=1 rx=0x7f8fa8006f60 tx=0x7f8fa800b970).stop 2021-02-10T11:48:24.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.832 7f8fc4f18700 1 -- 172.21.2.101:0/3445369559 shutdown_connections 2021-02-10T11:48:24.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.832 7f8fc4f18700 1 --2- 172.21.2.101:0/3445369559 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8fa40186c0 0x7f8fa401abc0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:24.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.832 7f8fc4f18700 1 --2- 172.21.2.101:0/3445369559 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8fc00b4d30 0x7f8fc00b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:24.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.832 7f8fc4f18700 1 --2- 172.21.2.101:0/3445369559 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8fc008d800 0x7f8fc008dc00 secure :-1 s=CLOSED pgs=3233 cs=0 l=1 rev1=1 rx=0x7f8fa8006f60 tx=0x7f8fa800b970).stop 2021-02-10T11:48:24.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.832 7f8fc4f18700 1 --2- 172.21.2.101:0/3445369559 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8fc008cde0 0x7f8fc008d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:24.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.832 7f8fc4f18700 1 -- 172.21.2.101:0/3445369559 >> 172.21.2.101:0/3445369559 conn(0x7f8fc00872d0 msgr2=0x7f8fc00ad590 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:24.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.832 7f8fc4f18700 1 -- 172.21.2.101:0/3445369559 shutdown_connections 2021-02-10T11:48:24.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:24.832 7f8fc4f18700 1 -- 172.21.2.101:0/3445369559 wait complete. 2021-02-10T11:48:24.836 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:48:25.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.254 7fd799337700 1 Processor -- start 2021-02-10T11:48:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.254 7fd799337700 1 -- start start 2021-02-10T11:48:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.254 7fd799337700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd7940d1870 0x7fd7941b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.254 7fd799337700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fd7941b0a10 0x7fd7940cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.254 7fd799337700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7940f0b10 con 0x7fd7940d1c70 2021-02-10T11:48:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.254 7fd799337700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7940e5ed0 con 0x7fd7940d1870 2021-02-10T11:48:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.254 7fd799337700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7940f0650 con 0x7fd7941b0a10 2021-02-10T11:48:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.254 7fd79307d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd7940d1870 0x7fd7941b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.254 7fd79307d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd7940d1870 0x7fd7941b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34894/0 (socket says 172.21.2.101:34894) 2021-02-10T11:48:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.254 7fd79307d700 1 -- 172.21.2.101:0/1419532507 learned_addr learned my addr 172.21.2.101:0/1419532507 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.255 7fd79287c700 1 --2- 172.21.2.101:0/1419532507 >> v2:172.21.2.108:3300/0 conn(0x7fd7941b0a10 0x7fd7940cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.255 7fd79307d700 1 -- 172.21.2.101:0/1419532507 >> v2:172.21.2.108:3300/0 conn(0x7fd7941b0a10 msgr2=0x7fd7940cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.255 7fd79307d700 1 --2- 172.21.2.101:0/1419532507 >> v2:172.21.2.108:3300/0 conn(0x7fd7941b0a10 0x7fd7940cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:25.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.255 7fd79307d700 1 -- 172.21.2.101:0/1419532507 >> v1:172.21.2.108:6789/0 conn(0x7fd7940d1c70 legacy=0x7fd7940cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.255 7fd79307d700 1 -- 172.21.2.101:0/1419532507 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7940f0840 con 0x7fd7940d1870 2021-02-10T11:48:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.255 7fd79307d700 1 --2- 172.21.2.101:0/1419532507 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7940d1870 0x7fd7941b03f0 secure :-1 s=READY pgs=3234 cs=0 l=1 rev1=1 rx=0x7fd78c006a80 tx=0x7fd78c009f00).ready entity=mon.0 client_cookie=56b0dff091ba2c8a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.255 7fd79207b700 1 -- 172.21.2.101:0/1419532507 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd78c00b5c0 con 0x7fd7940d1870 2021-02-10T11:48:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.255 7fd79207b700 1 -- 172.21.2.101:0/1419532507 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78c00b840 con 0x7fd7940d1870 2021-02-10T11:48:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.255 7fd79207b700 1 -- 172.21.2.101:0/1419532507 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd78c00bd60 con 0x7fd7940d1870 2021-02-10T11:48:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.255 7fd799337700 1 -- 172.21.2.101:0/1419532507 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7940d1870 msgr2=0x7fd7941b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.255 7fd799337700 1 --2- 172.21.2.101:0/1419532507 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7940d1870 0x7fd7941b03f0 secure :-1 s=READY pgs=3234 cs=0 l=1 rev1=1 rx=0x7fd78c006a80 tx=0x7fd78c009f00).stop 2021-02-10T11:48:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.256 7fd799337700 1 -- 172.21.2.101:0/1419532507 shutdown_connections 2021-02-10T11:48:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.256 7fd799337700 1 --2- 172.21.2.101:0/1419532507 >> v2:172.21.2.108:3300/0 conn(0x7fd7941b0a10 0x7fd7940cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.256 7fd799337700 1 --2- 172.21.2.101:0/1419532507 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7940d1870 0x7fd7941b03f0 unknown :-1 s=CLOSED pgs=3234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:25.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.256 7fd799337700 1 -- 172.21.2.101:0/1419532507 >> 172.21.2.101:0/1419532507 conn(0x7fd7940ce450 msgr2=0x7fd7940d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:25.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.256 7fd799337700 1 -- 172.21.2.101:0/1419532507 shutdown_connections 2021-02-10T11:48:25.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.256 7fd799337700 1 -- 172.21.2.101:0/1419532507 wait complete. 2021-02-10T11:48:25.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.270 7fd799337700 1 Processor -- start 2021-02-10T11:48:25.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.271 7fd799337700 1 -- start start 2021-02-10T11:48:25.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.271 7fd799337700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd794195650 0x7fd7940b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.271 7fd799337700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7940b5b80 0x7fd7941a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.271 7fd799337700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7940acfa0 0x7fd7941a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.271 7fd799337700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940b6010 con 0x7fd7940b5b80 2021-02-10T11:48:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.271 7fd799337700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7941a3a60 con 0x7fd794195650 2021-02-10T11:48:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.271 7fd799337700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7941a3c40 con 0x7fd7940acfa0 2021-02-10T11:48:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.271 7fd79287c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7940b5b80 0x7fd7941a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.271 7fd79287c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7940b5b80 0x7fd7941a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34902/0 (socket says 172.21.2.101:34902) 2021-02-10T11:48:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.271 7fd79287c700 1 -- 172.21.2.101:0/110042000 learned_addr learned my addr 172.21.2.101:0/110042000 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.272 7fd79287c700 1 -- 172.21.2.101:0/110042000 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7940acfa0 msgr2=0x7fd7941a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.272 7fd79287c700 1 --2- 172.21.2.101:0/110042000 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7940acfa0 0x7fd7941a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.272 7fd79287c700 1 -- 172.21.2.101:0/110042000 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd794195650 msgr2=0x7fd7940b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.272 7fd79287c700 1 --2- 172.21.2.101:0/110042000 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd794195650 0x7fd7940b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:25.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.272 7fd79287c700 1 -- 172.21.2.101:0/110042000 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7940bcae0 con 0x7fd7940b5b80 2021-02-10T11:48:25.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.272 7fd79287c700 1 --2- 172.21.2.101:0/110042000 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7940b5b80 0x7fd7941a2e20 secure :-1 s=READY pgs=3235 cs=0 l=1 rev1=1 rx=0x7fd788006f00 tx=0x7fd78800b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:25.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.272 7fd77bfff700 1 -- 172.21.2.101:0/110042000 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd78800bd30 con 0x7fd7940b5b80 2021-02-10T11:48:25.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.272 7fd77bfff700 1 -- 172.21.2.101:0/110042000 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd788002030 con 0x7fd7940b5b80 2021-02-10T11:48:25.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.272 7fd77bfff700 1 -- 172.21.2.101:0/110042000 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd788002550 con 0x7fd7940b5b80 2021-02-10T11:48:25.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.272 7fd799337700 1 -- 172.21.2.101:0/110042000 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7940bcae0 con 0x7fd7940b5b80 2021-02-10T11:48:25.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.272 7fd799337700 1 -- 172.21.2.101:0/110042000 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7940bd070 con 0x7fd7940b5b80 2021-02-10T11:48:25.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.273 7fd77bfff700 1 -- 172.21.2.101:0/110042000 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd788000c90 con 0x7fd7940b5b80 2021-02-10T11:48:25.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.273 7fd77bfff700 1 --2- 172.21.2.101:0/110042000 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7740186c0 0x7fd77401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:25.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.273 7fd77bfff700 1 -- 172.21.2.101:0/110042000 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd7880325f0 con 0x7fd7940b5b80 2021-02-10T11:48:25.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.273 7fd79307d700 1 --2- 172.21.2.101:0/110042000 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7740186c0 0x7fd77401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:25.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.273 7fd79307d700 1 --2- 172.21.2.101:0/110042000 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7740186c0 0x7fd77401abc0 crc :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:25.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.304 7fd799337700 1 -- 172.21.2.101:0/110042000 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7940f0ab0 con 0x7fd7940b5b80 2021-02-10T11:48:25.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.307 7fd77bfff700 1 -- 172.21.2.101:0/110042000 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd788024a90 con 0x7fd7940b5b80 2021-02-10T11:48:25.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.431 7fd799337700 1 -- 172.21.2.101:0/110042000 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7940f0ab0 con 0x7fd7940b5b80 2021-02-10T11:48:25.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.432 7fd77bfff700 1 -- 172.21.2.101:0/110042000 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) v1 ==== 74+0+14020 (secure 0 0 0) 0x7fd788024a90 con 0x7fd7940b5b80 2021-02-10T11:48:25.429 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:48:25.429 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":190,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:48:24.823917","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":41,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":40,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:48:22.784885","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":41,"pool_name":"cephfs_data","create_time":"2021-02-10 11:48:23.812792","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:48:25.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.433 7fd799337700 1 -- 172.21.2.101:0/110042000 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7740186c0 msgr2=0x7fd77401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:25.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.433 7fd799337700 1 --2- 172.21.2.101:0/110042000 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7740186c0 0x7fd77401abc0 crc :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:25.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.433 7fd799337700 1 -- 172.21.2.101:0/110042000 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7940b5b80 msgr2=0x7fd7941a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:25.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.433 7fd799337700 1 --2- 172.21.2.101:0/110042000 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7940b5b80 0x7fd7941a2e20 secure :-1 s=READY pgs=3235 cs=0 l=1 rev1=1 rx=0x7fd788006f00 tx=0x7fd78800b940).stop 2021-02-10T11:48:25.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.434 7fd799337700 1 -- 172.21.2.101:0/110042000 shutdown_connections 2021-02-10T11:48:25.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.434 7fd799337700 1 --2- 172.21.2.101:0/110042000 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd7740186c0 0x7fd77401abc0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:25.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.434 7fd799337700 1 --2- 172.21.2.101:0/110042000 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd7940acfa0 0x7fd7941a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:25.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.434 7fd799337700 1 --2- 172.21.2.101:0/110042000 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd7940b5b80 0x7fd7941a2e20 unknown :-1 s=CLOSED pgs=3235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:25.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.434 7fd799337700 1 --2- 172.21.2.101:0/110042000 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd794195650 0x7fd7940b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:25.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.434 7fd799337700 1 -- 172.21.2.101:0/110042000 >> 172.21.2.101:0/110042000 conn(0x7fd7940b3040 msgr2=0x7fd7940c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:25.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.434 7fd799337700 1 -- 172.21.2.101:0/110042000 shutdown_connections 2021-02-10T11:48:25.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.434 7fd799337700 1 -- 172.21.2.101:0/110042000 wait complete. 2021-02-10T11:48:25.439 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:48:25.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.857 7f466650e700 1 Processor -- start 2021-02-10T11:48:25.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.857 7f466650e700 1 -- start start 2021-02-10T11:48:25.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.857 7f466650e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f46600d0340 0x7f46600ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:25.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.857 7f466650e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46600f0b10 con 0x7f46600d1870 2021-02-10T11:48:25.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.857 7f466650e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46601b0be0 con 0x7f46600d1c70 2021-02-10T11:48:25.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.857 7f466650e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46600e5ed0 con 0x7f46600d0340 2021-02-10T11:48:25.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.857 7f465ffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f46600d1870 0x7f46601b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35402/0 (socket says 172.21.2.101:35402) 2021-02-10T11:48:25.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.857 7f465ffff700 1 -- 172.21.2.101:0/3339117811 learned_addr learned my addr 172.21.2.101:0/3339117811 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:25.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.858 7f465effd700 1 -- 172.21.2.101:0/3339117811 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1071682416 0 0) 0x7f46500010b0 con 0x7f46600d1870 2021-02-10T11:48:25.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.858 7f465effd700 1 -- 172.21.2.101:0/3339117811 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4648002fe0 con 0x7f46600d1870 2021-02-10T11:48:25.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.858 7f465f7fe700 1 --2- 172.21.2.101:0/3339117811 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f46600d0340 0x7f46600ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:25.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.858 7f465effd700 1 -- 172.21.2.101:0/3339117811 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3715131259 0 0) 0x7f4650001010 con 0x7f46600d1870 2021-02-10T11:48:25.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.858 7f465effd700 1 -- 172.21.2.101:0/3339117811 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f46600d0340 msgr2=0x7f46600ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:25.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.858 7f465effd700 1 --2- 172.21.2.101:0/3339117811 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f46600d0340 0x7f46600ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:25.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.858 7f465effd700 1 -- 172.21.2.101:0/3339117811 >> v1:172.21.2.108:6789/0 conn(0x7f46600d1c70 legacy=0x7f46600efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:25.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.858 7f465effd700 1 -- 172.21.2.101:0/3339117811 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46600f0450 con 0x7f46600d1870 2021-02-10T11:48:25.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.858 7f465effd700 1 -- 172.21.2.101:0/3339117811 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f4650001530 con 0x7f46600d1870 2021-02-10T11:48:25.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.858 7f465effd700 1 -- 172.21.2.101:0/3339117811 >> v1:172.21.2.101:6789/0 conn(0x7f46600d1870 legacy=0x7f46601b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:25.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.858 7f465effd700 1 --2- 172.21.2.101:0/3339117811 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4648003ac0 0x7f4648005ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:25.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.858 7f465effd700 1 --2- 172.21.2.101:0/3339117811 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4648006750 0x7f4648008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:25.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.858 7f465effd700 1 --2- 172.21.2.101:0/3339117811 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4648009510 0x7f464800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:25.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.858 7f465effd700 1 -- 172.21.2.101:0/3339117811 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464800c2e0 con 0x7f4648006750 2021-02-10T11:48:25.872 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.858 7f465effd700 1 -- 172.21.2.101:0/3339117811 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464800c500 con 0x7f4648009510 2021-02-10T11:48:25.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.858 7f465effd700 1 -- 172.21.2.101:0/3339117811 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4648050730 con 0x7f4648003ac0 2021-02-10T11:48:25.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.858 7f465f7fe700 1 --2- 172.21.2.101:0/3339117811 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4648006750 0x7f4648008c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:25.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.858 7f465ffff700 1 --2- 172.21.2.101:0/3339117811 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4648003ac0 0x7f4648005ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:25.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.858 7f465f7fe700 1 -- 172.21.2.101:0/3339117811 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4648003ac0 msgr2=0x7f4648005ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:25.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.858 7f465f7fe700 1 --2- 172.21.2.101:0/3339117811 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4648003ac0 0x7f4648005ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:25.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.858 7f465f7fe700 1 -- 172.21.2.101:0/3339117811 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4648009510 msgr2=0x7f464800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:25.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.858 7f4664b2b700 1 --2- 172.21.2.101:0/3339117811 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4648009510 0x7f464800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:25.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.859 7f465f7fe700 1 --2- 172.21.2.101:0/3339117811 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4648009510 0x7f464800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:25.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.859 7f465f7fe700 1 -- 172.21.2.101:0/3339117811 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4648050990 con 0x7f4648006750 2021-02-10T11:48:25.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.859 7f465ffff700 1 --2- 172.21.2.101:0/3339117811 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4648003ac0 0x7f4648005ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:48:25.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.859 7f4664b2b700 1 --2- 172.21.2.101:0/3339117811 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4648009510 0x7f464800ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:48:25.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.859 7f465f7fe700 1 --2- 172.21.2.101:0/3339117811 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4648006750 0x7f4648008c70 secure :-1 s=READY pgs=3237 cs=0 l=1 rev1=1 rx=0x7f4654009310 tx=0x7f4654009a90).ready entity=mon.0 client_cookie=6359b5841fd2b24f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:25.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.859 7f465effd700 1 -- 172.21.2.101:0/3339117811 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f465400c0c0 con 0x7f4648006750 2021-02-10T11:48:25.873 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.859 7f465effd700 1 -- 172.21.2.101:0/3339117811 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4654006400 con 0x7f4648006750 2021-02-10T11:48:25.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.859 7f465effd700 1 -- 172.21.2.101:0/3339117811 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f465400c400 con 0x7f4648006750 2021-02-10T11:48:25.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.859 7f466650e700 1 -- 172.21.2.101:0/3339117811 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4648006750 msgr2=0x7f4648008c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:25.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.859 7f466650e700 1 --2- 172.21.2.101:0/3339117811 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4648006750 0x7f4648008c70 secure :-1 s=READY pgs=3237 cs=0 l=1 rev1=1 rx=0x7f4654009310 tx=0x7f4654009a90).stop 2021-02-10T11:48:25.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.859 7f4664b2b700 1 -- 172.21.2.101:0/3339117811 reap_dead start 2021-02-10T11:48:25.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.860 7f466650e700 1 -- 172.21.2.101:0/3339117811 shutdown_connections 2021-02-10T11:48:25.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.860 7f466650e700 1 -- 172.21.2.101:0/3339117811 >> 172.21.2.101:0/3339117811 conn(0x7f46600ce450 msgr2=0x7f46600d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:25.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.860 7f466650e700 1 -- 172.21.2.101:0/3339117811 shutdown_connections 2021-02-10T11:48:25.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.860 7f466650e700 1 -- 172.21.2.101:0/3339117811 wait complete. 2021-02-10T11:48:25.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.874 7f466650e700 1 Processor -- start 2021-02-10T11:48:25.874 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.874 7f466650e700 1 -- start start 2021-02-10T11:48:25.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.875 7f466650e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f46601942b0 0x7f46600b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:25.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.875 7f466650e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4660195070 0x7f4660195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:25.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.875 7f466650e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f46600bb8e0 0x7f46600bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:25.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.875 7f466650e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46601946b0 con 0x7f4660195070 2021-02-10T11:48:25.876 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.875 7f466650e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46600b3b70 con 0x7f46601942b0 2021-02-10T11:48:25.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.875 7f466650e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46600b3d90 con 0x7f46600bb8e0 2021-02-10T11:48:25.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.875 7f465f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4660195070 0x7f4660195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:25.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.875 7f465f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4660195070 0x7f4660195470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34922/0 (socket says 172.21.2.101:34922) 2021-02-10T11:48:25.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.875 7f465f7fe700 1 -- 172.21.2.101:0/2873642175 learned_addr learned my addr 172.21.2.101:0/2873642175 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:25.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.875 7f465f7fe700 1 -- 172.21.2.101:0/2873642175 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f46600bb8e0 msgr2=0x7f46600bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:25.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.875 7f465f7fe700 1 --2- 172.21.2.101:0/2873642175 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f46600bb8e0 0x7f46600bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:25.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.875 7f465f7fe700 1 -- 172.21.2.101:0/2873642175 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f46601942b0 msgr2=0x7f46600b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:25.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.875 7f465f7fe700 1 --2- 172.21.2.101:0/2873642175 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f46601942b0 0x7f46600b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:25.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.875 7f465f7fe700 1 -- 172.21.2.101:0/2873642175 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46601a49e0 con 0x7f4660195070 2021-02-10T11:48:25.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.875 7f465f7fe700 1 --2- 172.21.2.101:0/2873642175 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4660195070 0x7f4660195470 secure :-1 s=READY pgs=3238 cs=0 l=1 rev1=1 rx=0x7f4654009670 tx=0x7f465400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:25.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.875 7f465cff9700 1 -- 172.21.2.101:0/2873642175 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4654000940 con 0x7f4660195070 2021-02-10T11:48:25.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.876 7f465cff9700 1 -- 172.21.2.101:0/2873642175 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46540093e0 con 0x7f4660195070 2021-02-10T11:48:25.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.876 7f466650e700 1 -- 172.21.2.101:0/2873642175 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46601a49e0 con 0x7f4660195070 2021-02-10T11:48:25.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.876 7f465cff9700 1 -- 172.21.2.101:0/2873642175 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4654002080 con 0x7f4660195070 2021-02-10T11:48:25.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.876 7f466650e700 1 -- 172.21.2.101:0/2873642175 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46601a49e0 con 0x7f4660195070 2021-02-10T11:48:25.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.876 7f465cff9700 1 -- 172.21.2.101:0/2873642175 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4654002020 con 0x7f4660195070 2021-02-10T11:48:25.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.876 7f465cff9700 1 --2- 172.21.2.101:0/2873642175 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f46400186d0 0x7f464001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:25.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.876 7f465cff9700 1 -- 172.21.2.101:0/2873642175 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4654032640 con 0x7f4660195070 2021-02-10T11:48:25.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.876 7f465ffff700 1 --2- 172.21.2.101:0/2873642175 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f46400186d0 0x7f464001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:25.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.876 7f465ffff700 1 --2- 172.21.2.101:0/2873642175 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f46400186d0 0x7f464001abd0 crc :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:25.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.907 7f466650e700 1 -- 172.21.2.101:0/2873642175 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46600bc5f0 con 0x7f4660195070 2021-02-10T11:48:25.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:25.910 7f465cff9700 1 -- 172.21.2.101:0/2873642175 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f46540093a0 con 0x7f4660195070 2021-02-10T11:48:26.035 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.038 7f466650e700 1 -- 172.21.2.101:0/2873642175 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46600bc5f0 con 0x7f4660195070 2021-02-10T11:48:26.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.039 7f465cff9700 1 -- 172.21.2.101:0/2873642175 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f4654000c90 con 0x7f4660195070 2021-02-10T11:48:26.036 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:48:26.036 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":191,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:48:25.826144","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":41,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":40,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:48:22.784885","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":41,"pool_name":"cephfs_data","create_time":"2021-02-10 11:48:23.812792","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"191","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:48:26.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.040 7f466650e700 1 -- 172.21.2.101:0/2873642175 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f46400186d0 msgr2=0x7f464001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:26.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.040 7f466650e700 1 --2- 172.21.2.101:0/2873642175 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f46400186d0 0x7f464001abd0 crc :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:26.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.040 7f466650e700 1 -- 172.21.2.101:0/2873642175 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4660195070 msgr2=0x7f4660195470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:26.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.040 7f466650e700 1 --2- 172.21.2.101:0/2873642175 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4660195070 0x7f4660195470 secure :-1 s=READY pgs=3238 cs=0 l=1 rev1=1 rx=0x7f4654009670 tx=0x7f465400b960).stop 2021-02-10T11:48:26.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.040 7f466650e700 1 -- 172.21.2.101:0/2873642175 shutdown_connections 2021-02-10T11:48:26.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.040 7f466650e700 1 --2- 172.21.2.101:0/2873642175 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f46400186d0 0x7f464001abd0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:26.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.040 7f466650e700 1 --2- 172.21.2.101:0/2873642175 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f46600bb8e0 0x7f46600bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:26.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.040 7f466650e700 1 --2- 172.21.2.101:0/2873642175 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4660195070 0x7f4660195470 unknown :-1 s=CLOSED pgs=3238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:26.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.040 7f466650e700 1 --2- 172.21.2.101:0/2873642175 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f46601942b0 0x7f46600b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:26.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.040 7f466650e700 1 -- 172.21.2.101:0/2873642175 >> 172.21.2.101:0/2873642175 conn(0x7f46600b3040 msgr2=0x7f46600c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:26.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.041 7f466650e700 1 -- 172.21.2.101:0/2873642175 shutdown_connections 2021-02-10T11:48:26.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.041 7f466650e700 1 -- 172.21.2.101:0/2873642175 wait complete. 2021-02-10T11:48:26.046 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-02-10T11:48:26.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.467 7fd599c6e700 1 Processor -- start 2021-02-10T11:48:26.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.467 7fd599c6e700 1 -- start start 2021-02-10T11:48:26.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.467 7fd599c6e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd5940d1860 0x7fd5941b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:26.479 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.467 7fd599c6e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5940cff60 0x7fd594109120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:26.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.467 7fd599c6e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd594109a50 con 0x7fd5941b0ad0 2021-02-10T11:48:26.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.467 7fd599c6e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd594109740 con 0x7fd5940d1860 2021-02-10T11:48:26.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.467 7fd599c6e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd594088070 con 0x7fd5940cff60 2021-02-10T11:48:26.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.467 7fd5937fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd5940d1860 0x7fd5941b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:26.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.467 7fd592ffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fd5941b0ad0 0x7fd594108930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35422/0 (socket says 172.21.2.101:35422) 2021-02-10T11:48:26.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.467 7fd592ffd700 1 -- 172.21.2.101:0/1188263067 learned_addr learned my addr 172.21.2.101:0/1188263067 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:26.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.468 7fd5937fe700 1 -- 172.21.2.101:0/1188263067 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5940cff60 msgr2=0x7fd594109120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:48:26.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.468 7fd5937fe700 1 --2- 172.21.2.101:0/1188263067 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5940cff60 0x7fd594109120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:26.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.468 7fd5937fe700 1 -- 172.21.2.101:0/1188263067 >> v1:172.21.2.101:6789/0 conn(0x7fd5941b0ad0 legacy=0x7fd594108930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:26.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.468 7fd5937fe700 1 -- 172.21.2.101:0/1188263067 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5940d1c60 con 0x7fd5940d1860 2021-02-10T11:48:26.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.468 7fd5937fe700 1 --2- 172.21.2.101:0/1188263067 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd5940d1860 0x7fd5941b04b0 secure :-1 s=READY pgs=3240 cs=0 l=1 rev1=1 rx=0x7fd57c006650 tx=0x7fd57c009a00).ready entity=mon.0 client_cookie=dbe94303eebbdd38 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:26.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.468 7fd5927fc700 1 -- 172.21.2.101:0/1188263067 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd57c0009b0 con 0x7fd5940d1860 2021-02-10T11:48:26.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.468 7fd5927fc700 1 -- 172.21.2.101:0/1188263067 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57c00c0c0 con 0x7fd5940d1860 2021-02-10T11:48:26.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.468 7fd5927fc700 1 -- 172.21.2.101:0/1188263067 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd57c00c5a0 con 0x7fd5940d1860 2021-02-10T11:48:26.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.468 7fd599c6e700 1 -- 172.21.2.101:0/1188263067 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd5940d1860 msgr2=0x7fd5941b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:26.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.468 7fd599c6e700 1 --2- 172.21.2.101:0/1188263067 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd5940d1860 0x7fd5941b04b0 secure :-1 s=READY pgs=3240 cs=0 l=1 rev1=1 rx=0x7fd57c006650 tx=0x7fd57c009a00).stop 2021-02-10T11:48:26.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.468 7fd599c6e700 1 -- 172.21.2.101:0/1188263067 shutdown_connections 2021-02-10T11:48:26.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.468 7fd599c6e700 1 --2- 172.21.2.101:0/1188263067 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5940cff60 0x7fd594109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:26.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.468 7fd599c6e700 1 --2- 172.21.2.101:0/1188263067 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd5940d1860 0x7fd5941b04b0 unknown :-1 s=CLOSED pgs=3240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:26.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.468 7fd599c6e700 1 -- 172.21.2.101:0/1188263067 >> 172.21.2.101:0/1188263067 conn(0x7fd5940ce410 msgr2=0x7fd5940d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:26.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.469 7fd599c6e700 1 -- 172.21.2.101:0/1188263067 shutdown_connections 2021-02-10T11:48:26.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.469 7fd599c6e700 1 -- 172.21.2.101:0/1188263067 wait complete. 2021-02-10T11:48:26.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.483 7fd599c6e700 1 Processor -- start 2021-02-10T11:48:26.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.483 7fd599c6e700 1 -- start start 2021-02-10T11:48:26.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.483 7fd599c6e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5940a6a90 0x7fd5941a7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:26.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.483 7fd599c6e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd5941a8040 0x7fd5940955d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:26.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.483 7fd599c6e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd594095bf0 0x7fd5940a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:26.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.483 7fd599c6e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5940a6510 con 0x7fd5941a8040 2021-02-10T11:48:26.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.483 7fd599c6e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5940a6e90 con 0x7fd594095bf0 2021-02-10T11:48:26.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.483 7fd599c6e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5940c3670 con 0x7fd5940a6a90 2021-02-10T11:48:26.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.484 7fd592ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd5941a8040 0x7fd5940955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:26.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.484 7fd592ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd5941a8040 0x7fd5940955d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34934/0 (socket says 172.21.2.101:34934) 2021-02-10T11:48:26.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.484 7fd592ffd700 1 -- 172.21.2.101:0/1798277116 learned_addr learned my addr 172.21.2.101:0/1798277116 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:26.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.484 7fd592ffd700 1 -- 172.21.2.101:0/1798277116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5940a6a90 msgr2=0x7fd5941a7a20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:26.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.484 7fd592ffd700 1 --2- 172.21.2.101:0/1798277116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5940a6a90 0x7fd5941a7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:26.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.484 7fd592ffd700 1 -- 172.21.2.101:0/1798277116 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd594095bf0 msgr2=0x7fd5940a4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:26.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.484 7fd592ffd700 1 --2- 172.21.2.101:0/1798277116 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd594095bf0 0x7fd5940a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:26.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.484 7fd592ffd700 1 -- 172.21.2.101:0/1798277116 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd594095ff0 con 0x7fd5941a8040 2021-02-10T11:48:26.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.484 7fd592ffd700 1 --2- 172.21.2.101:0/1798277116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd5941a8040 0x7fd5940955d0 secure :-1 s=READY pgs=3241 cs=0 l=1 rev1=1 rx=0x7fd584001c60 tx=0x7fd58400a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:26.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.484 7fd58bfff700 1 -- 172.21.2.101:0/1798277116 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd58400acc0 con 0x7fd5941a8040 2021-02-10T11:48:26.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.484 7fd58bfff700 1 -- 172.21.2.101:0/1798277116 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd584004140 con 0x7fd5941a8040 2021-02-10T11:48:26.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.484 7fd599c6e700 1 -- 172.21.2.101:0/1798277116 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd594095ff0 con 0x7fd5941a8040 2021-02-10T11:48:26.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.484 7fd58bfff700 1 -- 172.21.2.101:0/1798277116 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd5840045a0 con 0x7fd5941a8040 2021-02-10T11:48:26.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.485 7fd599c6e700 1 -- 172.21.2.101:0/1798277116 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd594095ff0 con 0x7fd5941a8040 2021-02-10T11:48:26.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.485 7fd58bfff700 1 -- 172.21.2.101:0/1798277116 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd584000ce0 con 0x7fd5941a8040 2021-02-10T11:48:26.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.485 7fd58bfff700 1 --2- 172.21.2.101:0/1798277116 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd5780186d0 0x7fd57801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:26.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.485 7fd58bfff700 1 -- 172.21.2.101:0/1798277116 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd584032640 con 0x7fd5941a8040 2021-02-10T11:48:26.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.485 7fd5937fe700 1 --2- 172.21.2.101:0/1798277116 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd5780186d0 0x7fd57801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:26.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.486 7fd5937fe700 1 --2- 172.21.2.101:0/1798277116 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd5780186d0 0x7fd57801abd0 crc :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:26.514 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.517 7fd599c6e700 1 -- 172.21.2.101:0/1798277116 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd594082a20 con 0x7fd5941a8040 2021-02-10T11:48:26.516 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.520 7fd58bfff700 1 -- 172.21.2.101:0/1798277116 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd5840009b0 con 0x7fd5941a8040 2021-02-10T11:48:26.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.656 7fd599c6e700 1 -- 172.21.2.101:0/1798277116 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"} v 0) v1 -- 0x7fd594082a20 con 0x7fd5941a8040 2021-02-10T11:48:26.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.832 7fd58bfff700 1 -- 172.21.2.101:0/1798277116 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"}]=0 v164) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd584000ce0 con 0x7fd5941a8040 2021-02-10T11:48:26.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.874 7fd599c6e700 1 -- 172.21.2.101:0/1798277116 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd5780186d0 msgr2=0x7fd57801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:26.870 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.874 7fd599c6e700 1 --2- 172.21.2.101:0/1798277116 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd5780186d0 0x7fd57801abd0 crc :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:26.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.874 7fd599c6e700 1 -- 172.21.2.101:0/1798277116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd5941a8040 msgr2=0x7fd5940955d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:26.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.874 7fd599c6e700 1 --2- 172.21.2.101:0/1798277116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd5941a8040 0x7fd5940955d0 secure :-1 s=READY pgs=3241 cs=0 l=1 rev1=1 rx=0x7fd584001c60 tx=0x7fd58400a8d0).stop 2021-02-10T11:48:26.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.874 7fd599c6e700 1 -- 172.21.2.101:0/1798277116 shutdown_connections 2021-02-10T11:48:26.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.874 7fd599c6e700 1 --2- 172.21.2.101:0/1798277116 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd594095bf0 0x7fd5940a4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:26.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.874 7fd599c6e700 1 --2- 172.21.2.101:0/1798277116 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd5941a8040 0x7fd5940955d0 unknown :-1 s=CLOSED pgs=3241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:26.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.874 7fd599c6e700 1 --2- 172.21.2.101:0/1798277116 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd5780186d0 0x7fd57801abd0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:26.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.874 7fd599c6e700 1 --2- 172.21.2.101:0/1798277116 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd5940a6a90 0x7fd5941a7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:26.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.874 7fd599c6e700 1 -- 172.21.2.101:0/1798277116 >> 172.21.2.101:0/1798277116 conn(0x7fd594086490 msgr2=0x7fd5940c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:26.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.874 7fd599c6e700 1 -- 172.21.2.101:0/1798277116 shutdown_connections 2021-02-10T11:48:26.871 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:26.874 7fd599c6e700 1 -- 172.21.2.101:0/1798277116 wait complete. 2021-02-10T11:48:26.879 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:48:27.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.294 7f484c4d6700 1 Processor -- start 2021-02-10T11:48:27.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.294 7f484c4d6700 1 -- start start 2021-02-10T11:48:27.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.295 7f484c4d6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48440d0340 0x7f48440ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:27.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.295 7f484c4d6700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48440f0b10 con 0x7f48440d1870 2021-02-10T11:48:27.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.295 7f484c4d6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48441b0be0 con 0x7f48440d1c70 2021-02-10T11:48:27.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.295 7f484c4d6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48440e5ed0 con 0x7f48440d0340 2021-02-10T11:48:27.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.295 7f484a2f2700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f48440d1870 0x7f48441b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35434/0 (socket says 172.21.2.101:35434) 2021-02-10T11:48:27.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.295 7f484a2f2700 1 -- 172.21.2.101:0/757108204 learned_addr learned my addr 172.21.2.101:0/757108204 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:27.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.296 7f48492f0700 1 -- 172.21.2.101:0/757108204 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2658042603 0 0) 0x7f4834000ce0 con 0x7f48440d1870 2021-02-10T11:48:27.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.296 7f48492f0700 1 -- 172.21.2.101:0/757108204 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4830002fe0 con 0x7f48440d1870 2021-02-10T11:48:27.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.296 7f4849af1700 1 --2- 172.21.2.101:0/757108204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48440d0340 0x7f48440ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:27.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.296 7f48492f0700 1 -- 172.21.2.101:0/757108204 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2214514403 0 0) 0x7f4834001140 con 0x7f48440d1870 2021-02-10T11:48:27.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.296 7f48492f0700 1 -- 172.21.2.101:0/757108204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48440d0340 msgr2=0x7f48440ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:27.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.296 7f48492f0700 1 --2- 172.21.2.101:0/757108204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48440d0340 0x7f48440ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:27.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.296 7f48492f0700 1 -- 172.21.2.101:0/757108204 >> v1:172.21.2.108:6789/0 conn(0x7f48440d1c70 legacy=0x7f48440efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:27.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.296 7f48492f0700 1 -- 172.21.2.101:0/757108204 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48440f0450 con 0x7f48440d1870 2021-02-10T11:48:27.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.296 7f48492f0700 1 -- 172.21.2.101:0/757108204 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f4834001640 con 0x7f48440d1870 2021-02-10T11:48:27.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.296 7f48492f0700 1 -- 172.21.2.101:0/757108204 >> v1:172.21.2.101:6789/0 conn(0x7f48440d1870 legacy=0x7f48441b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:27.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.296 7f48492f0700 1 --2- 172.21.2.101:0/757108204 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4830003ac0 0x7f4830005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:27.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.296 7f48492f0700 1 --2- 172.21.2.101:0/757108204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4830006790 0x7f4830008c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:27.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.296 7f48492f0700 1 --2- 172.21.2.101:0/757108204 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4830009500 0x7f483000ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:27.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.296 7f48492f0700 1 -- 172.21.2.101:0/757108204 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483000c2d0 con 0x7f4830009500 2021-02-10T11:48:27.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.296 7f48492f0700 1 -- 172.21.2.101:0/757108204 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483000c4f0 con 0x7f4830003ac0 2021-02-10T11:48:27.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.296 7f48492f0700 1 -- 172.21.2.101:0/757108204 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483000c710 con 0x7f4830006790 2021-02-10T11:48:27.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.296 7f484aaf3700 1 --2- 172.21.2.101:0/757108204 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4830009500 0x7f483000ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:27.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.297 7f4849af1700 1 --2- 172.21.2.101:0/757108204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4830006790 0x7f4830008c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:27.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.297 7f484a2f2700 1 --2- 172.21.2.101:0/757108204 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4830003ac0 0x7f4830005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:27.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.297 7f484aaf3700 1 -- 172.21.2.101:0/757108204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4830006790 msgr2=0x7f4830008c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:27.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.297 7f484aaf3700 1 --2- 172.21.2.101:0/757108204 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4830006790 0x7f4830008c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:27.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.297 7f484aaf3700 1 -- 172.21.2.101:0/757108204 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4830003ac0 msgr2=0x7f4830005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:27.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.297 7f484aaf3700 1 --2- 172.21.2.101:0/757108204 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4830003ac0 0x7f4830005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:27.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.297 7f484aaf3700 1 -- 172.21.2.101:0/757108204 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4830050980 con 0x7f4830009500 2021-02-10T11:48:27.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.297 7f484a2f2700 1 --2- 172.21.2.101:0/757108204 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4830003ac0 0x7f4830005ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:48:27.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.297 7f484aaf3700 1 --2- 172.21.2.101:0/757108204 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4830009500 0x7f483000ba20 secure :-1 s=READY pgs=3243 cs=0 l=1 rev1=1 rx=0x7f483c006e40 tx=0x7f483c009db0).ready entity=mon.0 client_cookie=812d0d58e4dc7520 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:27.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.297 7f48492f0700 1 -- 172.21.2.101:0/757108204 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f483c00b300 con 0x7f4830009500 2021-02-10T11:48:27.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.297 7f48492f0700 1 -- 172.21.2.101:0/757108204 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483c006af0 con 0x7f4830009500 2021-02-10T11:48:27.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.297 7f48492f0700 1 -- 172.21.2.101:0/757108204 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f483c00b7f0 con 0x7f4830009500 2021-02-10T11:48:27.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.297 7f484c4d6700 1 -- 172.21.2.101:0/757108204 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4830009500 msgr2=0x7f483000ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:27.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.297 7f484c4d6700 1 --2- 172.21.2.101:0/757108204 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4830009500 0x7f483000ba20 secure :-1 s=READY pgs=3243 cs=0 l=1 rev1=1 rx=0x7f483c006e40 tx=0x7f483c009db0).stop 2021-02-10T11:48:27.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.297 7f484aaf3700 1 -- 172.21.2.101:0/757108204 reap_dead start 2021-02-10T11:48:27.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.298 7f484c4d6700 1 -- 172.21.2.101:0/757108204 shutdown_connections 2021-02-10T11:48:27.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.298 7f484c4d6700 1 -- 172.21.2.101:0/757108204 >> 172.21.2.101:0/757108204 conn(0x7f48440ce450 msgr2=0x7f48440d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:27.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.298 7f484c4d6700 1 -- 172.21.2.101:0/757108204 shutdown_connections 2021-02-10T11:48:27.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.298 7f484c4d6700 1 -- 172.21.2.101:0/757108204 wait complete. 2021-02-10T11:48:27.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.312 7f484c4d6700 1 Processor -- start 2021-02-10T11:48:27.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.312 7f484c4d6700 1 -- start start 2021-02-10T11:48:27.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.313 7f484c4d6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f484408e360 0x7f48440b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:27.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.313 7f484c4d6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48440a9600 0x7f48440a9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:27.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.313 7f484c4d6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48440aa0a0 0x7f48440ecb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:27.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.313 7f484c4d6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484408dde0 con 0x7f48440a9600 2021-02-10T11:48:27.314 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.313 7f484c4d6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48440ed130 con 0x7f484408e360 2021-02-10T11:48:27.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.313 7f484c4d6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48440ed310 con 0x7f48440aa0a0 2021-02-10T11:48:27.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.313 7f4849af1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48440a9600 0x7f48440a9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:27.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.313 7f4849af1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48440a9600 0x7f48440a9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34950/0 (socket says 172.21.2.101:34950) 2021-02-10T11:48:27.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.313 7f4849af1700 1 -- 172.21.2.101:0/3583865317 learned_addr learned my addr 172.21.2.101:0/3583865317 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:27.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.313 7f4849af1700 1 -- 172.21.2.101:0/3583865317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48440aa0a0 msgr2=0x7f48440ecb10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:27.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.313 7f4849af1700 1 --2- 172.21.2.101:0/3583865317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48440aa0a0 0x7f48440ecb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:27.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.313 7f4849af1700 1 -- 172.21.2.101:0/3583865317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f484408e360 msgr2=0x7f48440b7730 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:27.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.313 7f4849af1700 1 --2- 172.21.2.101:0/3583865317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f484408e360 0x7f48440b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:27.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.313 7f4849af1700 1 -- 172.21.2.101:0/3583865317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48440ed530 con 0x7f48440a9600 2021-02-10T11:48:27.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.313 7f4849af1700 1 --2- 172.21.2.101:0/3583865317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48440a9600 0x7f48440a9a80 secure :-1 s=READY pgs=3244 cs=0 l=1 rev1=1 rx=0x7f4840006f60 tx=0x7f484000b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:27.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.314 7f483affd700 1 -- 172.21.2.101:0/3583865317 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f484000bd60 con 0x7f48440a9600 2021-02-10T11:48:27.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.314 7f483affd700 1 -- 172.21.2.101:0/3583865317 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4840000c90 con 0x7f48440a9600 2021-02-10T11:48:27.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.314 7f483affd700 1 -- 172.21.2.101:0/3583865317 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4840002300 con 0x7f48440a9600 2021-02-10T11:48:27.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.314 7f484c4d6700 1 -- 172.21.2.101:0/3583865317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48440ed530 con 0x7f48440a9600 2021-02-10T11:48:27.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.314 7f484c4d6700 1 -- 172.21.2.101:0/3583865317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48440ed530 con 0x7f48440a9600 2021-02-10T11:48:27.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.314 7f483affd700 1 -- 172.21.2.101:0/3583865317 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4840002020 con 0x7f48440a9600 2021-02-10T11:48:27.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.315 7f483affd700 1 --2- 172.21.2.101:0/3583865317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f482c0186c0 0x7f482c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:27.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.315 7f483affd700 1 -- 172.21.2.101:0/3583865317 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4840032430 con 0x7f48440a9600 2021-02-10T11:48:27.316 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.315 7f484a2f2700 1 --2- 172.21.2.101:0/3583865317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f482c0186c0 0x7f482c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:27.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.315 7f484a2f2700 1 --2- 172.21.2.101:0/3583865317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f482c0186c0 0x7f482c01abc0 crc :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:27.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.345 7f484c4d6700 1 -- 172.21.2.101:0/3583865317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48440ebc00 con 0x7f48440a9600 2021-02-10T11:48:27.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.347 7f483affd700 1 -- 172.21.2.101:0/3583865317 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4840030d60 con 0x7f48440a9600 2021-02-10T11:48:27.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.478 7f484c4d6700 1 -- 172.21.2.101:0/3583865317 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48440f0b10 con 0x7f48440a9600 2021-02-10T11:48:27.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.478 7f483affd700 1 -- 172.21.2.101:0/3583865317 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) v1 ==== 95+0+2108 (secure 0 0 0) 0x7f484000bd60 con 0x7f48440a9600 2021-02-10T11:48:27.475 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:48:27.476 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":164,"default_fscid":20,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":13530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/347510536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":347510536},{"type":"v1","addr":"172.21.2.108:6833","nonce":347510536}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":160}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:48:24.824003","modified":"2021-02-10 11:48:26.829128","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":13518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13518":{"gid":13518,"name":"a","rank":0,"incarnation":161,"state":"up:active","state_seq":3,"addr":"172.21.2.101:6835/2721786668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2721786668},{"type":"v1","addr":"172.21.2.101:6835","nonce":2721786668}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[41],"metadata_pool":40,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":20}]} 2021-02-10T11:48:27.476 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 164 2021-02-10T11:48:27.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.480 7f484c4d6700 1 -- 172.21.2.101:0/3583865317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f482c0186c0 msgr2=0x7f482c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:27.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.480 7f484c4d6700 1 --2- 172.21.2.101:0/3583865317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f482c0186c0 0x7f482c01abc0 crc :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:27.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.480 7f484c4d6700 1 -- 172.21.2.101:0/3583865317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48440a9600 msgr2=0x7f48440a9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:27.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.480 7f484c4d6700 1 --2- 172.21.2.101:0/3583865317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48440a9600 0x7f48440a9a80 secure :-1 s=READY pgs=3244 cs=0 l=1 rev1=1 rx=0x7f4840006f60 tx=0x7f484000b970).stop 2021-02-10T11:48:27.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.481 7f484c4d6700 1 -- 172.21.2.101:0/3583865317 shutdown_connections 2021-02-10T11:48:27.477 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.481 7f484c4d6700 1 --2- 172.21.2.101:0/3583865317 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f482c0186c0 0x7f482c01abc0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:27.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.481 7f484c4d6700 1 --2- 172.21.2.101:0/3583865317 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f48440aa0a0 0x7f48440ecb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:27.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.481 7f484c4d6700 1 --2- 172.21.2.101:0/3583865317 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f48440a9600 0x7f48440a9a80 unknown :-1 s=CLOSED pgs=3244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:27.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.481 7f484c4d6700 1 --2- 172.21.2.101:0/3583865317 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f484408e360 0x7f48440b7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:27.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.481 7f484c4d6700 1 -- 172.21.2.101:0/3583865317 >> 172.21.2.101:0/3583865317 conn(0x7f48440b7e70 msgr2=0x7f48440c3850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:27.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.481 7f484c4d6700 1 -- 172.21.2.101:0/3583865317 shutdown_connections 2021-02-10T11:48:27.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.481 7f484c4d6700 1 -- 172.21.2.101:0/3583865317 wait complete. 2021-02-10T11:48:27.486 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:48:27.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.903 7f1a9b561700 1 Processor -- start 2021-02-10T11:48:27.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.903 7f1a9b561700 1 -- start start 2021-02-10T11:48:27.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.904 7f1a9b561700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1a940d1c70 0x7f1a940f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:27.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.904 7f1a9b561700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a940f0b10 con 0x7f1a940d0340 2021-02-10T11:48:27.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.904 7f1a9b561700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a941b0be0 con 0x7f1a940d1870 2021-02-10T11:48:27.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.904 7f1a9b561700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a940e5ed0 con 0x7f1a940d1c70 2021-02-10T11:48:27.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.904 7f1a99b7e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1a940d1c70 0x7f1a940f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:27.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.904 7f1a99b7e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1a940d1c70 0x7f1a940f20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34962/0 (socket says 172.21.2.101:34962) 2021-02-10T11:48:27.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.904 7f1a99b7e700 1 -- 172.21.2.101:0/3854116467 learned_addr learned my addr 172.21.2.101:0/3854116467 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:27.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.904 7f1a99b7e700 1 -- 172.21.2.101:0/3854116467 >> v1:172.21.2.108:6789/0 conn(0x7f1a940d1870 legacy=0x7f1a941b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:27.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.904 7f1a99b7e700 1 -- 172.21.2.101:0/3854116467 >> v1:172.21.2.101:6789/0 conn(0x7f1a940d0340 legacy=0x7f1a940ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:27.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.904 7f1a99b7e700 1 -- 172.21.2.101:0/3854116467 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a940efe30 con 0x7f1a940d1c70 2021-02-10T11:48:27.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.904 7f1a99b7e700 1 --2- 172.21.2.101:0/3854116467 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a940d1c70 0x7f1a940f20a0 secure :-1 s=READY pgs=3246 cs=0 l=1 rev1=1 rx=0x7f1a8c006f00 tx=0x7f1a8c009f10).ready entity=mon.0 client_cookie=f67ff770bca028a6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:27.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.905 7f1a8bfff700 1 -- 172.21.2.101:0/3854116467 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1a8c00b3b0 con 0x7f1a940d1c70 2021-02-10T11:48:27.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.905 7f1a8bfff700 1 -- 172.21.2.101:0/3854116467 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c00b730 con 0x7f1a940d1c70 2021-02-10T11:48:27.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.905 7f1a8bfff700 1 -- 172.21.2.101:0/3854116467 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1a8c00bc10 con 0x7f1a940d1c70 2021-02-10T11:48:27.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.905 7f1a9b561700 1 -- 172.21.2.101:0/3854116467 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a940d1c70 msgr2=0x7f1a940f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:27.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.905 7f1a9b561700 1 --2- 172.21.2.101:0/3854116467 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a940d1c70 0x7f1a940f20a0 secure :-1 s=READY pgs=3246 cs=0 l=1 rev1=1 rx=0x7f1a8c006f00 tx=0x7f1a8c009f10).stop 2021-02-10T11:48:27.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.905 7f1a9b561700 1 -- 172.21.2.101:0/3854116467 shutdown_connections 2021-02-10T11:48:27.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.905 7f1a9b561700 1 --2- 172.21.2.101:0/3854116467 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a940d1c70 0x7f1a940f20a0 unknown :-1 s=CLOSED pgs=3246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:27.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.905 7f1a9b561700 1 -- 172.21.2.101:0/3854116467 >> 172.21.2.101:0/3854116467 conn(0x7f1a940ce450 msgr2=0x7f1a940d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:27.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.905 7f1a9b561700 1 -- 172.21.2.101:0/3854116467 shutdown_connections 2021-02-10T11:48:27.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.905 7f1a9b561700 1 -- 172.21.2.101:0/3854116467 wait complete. 2021-02-10T11:48:27.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.920 7f1a9b561700 1 Processor -- start 2021-02-10T11:48:27.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.920 7f1a9b561700 1 -- start start 2021-02-10T11:48:27.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.920 7f1a9b561700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a940af040 0x7f1a940af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:27.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.920 7f1a9b561700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a940afa60 0x7f1a940ada20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:27.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.920 7f1a9b561700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a940afe60 0x7f1a940ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:27.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.920 7f1a9b561700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a940923c0 con 0x7f1a940af040 2021-02-10T11:48:27.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.920 7f1a9b561700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a940b0410 con 0x7f1a940afa60 2021-02-10T11:48:27.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.920 7f1a9b561700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a94091e40 con 0x7f1a940afe60 2021-02-10T11:48:27.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.921 7f1a9937d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a940af040 0x7f1a940af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:27.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.921 7f1a9937d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a940af040 0x7f1a940af440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34964/0 (socket says 172.21.2.101:34964) 2021-02-10T11:48:27.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.921 7f1a9937d700 1 -- 172.21.2.101:0/1722919384 learned_addr learned my addr 172.21.2.101:0/1722919384 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:27.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.921 7f1a98b7c700 1 --2- 172.21.2.101:0/1722919384 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a940afa60 0x7f1a940ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:27.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.921 7f1a99b7e700 1 --2- 172.21.2.101:0/1722919384 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a940afe60 0x7f1a940ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:27.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.921 7f1a9937d700 1 -- 172.21.2.101:0/1722919384 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a940afe60 msgr2=0x7f1a940ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:27.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.921 7f1a9937d700 1 --2- 172.21.2.101:0/1722919384 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a940afe60 0x7f1a940ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:27.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.921 7f1a9937d700 1 -- 172.21.2.101:0/1722919384 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a940afa60 msgr2=0x7f1a940ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:27.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.921 7f1a9937d700 1 --2- 172.21.2.101:0/1722919384 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a940afa60 0x7f1a940ada20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:27.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.921 7f1a9937d700 1 -- 172.21.2.101:0/1722919384 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9408f6d0 con 0x7f1a940af040 2021-02-10T11:48:27.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.921 7f1a9937d700 1 --2- 172.21.2.101:0/1722919384 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a940af040 0x7f1a940af440 secure :-1 s=READY pgs=3247 cs=0 l=1 rev1=1 rx=0x7f1a9000b5b0 tx=0x7f1a9000b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:27.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.921 7f1a89ffb700 1 -- 172.21.2.101:0/1722919384 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1a9000bd40 con 0x7f1a940af040 2021-02-10T11:48:27.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.921 7f1a89ffb700 1 -- 172.21.2.101:0/1722919384 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a90000c60 con 0x7f1a940af040 2021-02-10T11:48:27.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.921 7f1a9b561700 1 -- 172.21.2.101:0/1722919384 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a9408f6d0 con 0x7f1a940af040 2021-02-10T11:48:27.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.921 7f1a9b561700 1 -- 172.21.2.101:0/1722919384 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a940ae950 con 0x7f1a940af040 2021-02-10T11:48:27.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.921 7f1a89ffb700 1 -- 172.21.2.101:0/1722919384 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1a900062a0 con 0x7f1a940af040 2021-02-10T11:48:27.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.922 7f1a89ffb700 1 -- 172.21.2.101:0/1722919384 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1a9000a000 con 0x7f1a940af040 2021-02-10T11:48:27.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.922 7f1a89ffb700 1 --2- 172.21.2.101:0/1722919384 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1a74018730 0x7f1a7401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:27.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.922 7f1a89ffb700 1 -- 172.21.2.101:0/1722919384 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1a90032610 con 0x7f1a940af040 2021-02-10T11:48:27.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.922 7f1a98b7c700 1 --2- 172.21.2.101:0/1722919384 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1a74018730 0x7f1a7401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:27.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.922 7f1a98b7c700 1 --2- 172.21.2.101:0/1722919384 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1a74018730 0x7f1a7401abc0 crc :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:27.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.954 7f1a9b561700 1 -- 172.21.2.101:0/1722919384 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a940aec00 con 0x7f1a940af040 2021-02-10T11:48:27.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:27.957 7f1a89ffb700 1 -- 172.21.2.101:0/1722919384 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1a90000940 con 0x7f1a940af040 2021-02-10T11:48:28.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.080 7f1a9b561700 1 -- 172.21.2.101:0/1722919384 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a940aec00 con 0x7f1a940af040 2021-02-10T11:48:28.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.081 7f1a89ffb700 1 -- 172.21.2.101:0/1722919384 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f1a9001f4d0 con 0x7f1a940af040 2021-02-10T11:48:28.078 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:48:28.078 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":191,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:48:25.826144","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":41,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":40,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:48:22.784885","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":41,"pool_name":"cephfs_data","create_time":"2021-02-10 11:48:23.812792","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"191","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:48:28.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.083 7f1a9b561700 1 -- 172.21.2.101:0/1722919384 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1a74018730 msgr2=0x7f1a7401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:28.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.083 7f1a9b561700 1 --2- 172.21.2.101:0/1722919384 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1a74018730 0x7f1a7401abc0 crc :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:28.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.083 7f1a9b561700 1 -- 172.21.2.101:0/1722919384 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a940af040 msgr2=0x7f1a940af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:28.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.083 7f1a9b561700 1 --2- 172.21.2.101:0/1722919384 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a940af040 0x7f1a940af440 secure :-1 s=READY pgs=3247 cs=0 l=1 rev1=1 rx=0x7f1a9000b5b0 tx=0x7f1a9000b950).stop 2021-02-10T11:48:28.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.083 7f1a9b561700 1 -- 172.21.2.101:0/1722919384 shutdown_connections 2021-02-10T11:48:28.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.083 7f1a9b561700 1 --2- 172.21.2.101:0/1722919384 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1a74018730 0x7f1a7401abc0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:28.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.083 7f1a9b561700 1 --2- 172.21.2.101:0/1722919384 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1a940afe60 0x7f1a940ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:28.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.083 7f1a9b561700 1 --2- 172.21.2.101:0/1722919384 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1a940afa60 0x7f1a940ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:28.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.083 7f1a9b561700 1 --2- 172.21.2.101:0/1722919384 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1a940af040 0x7f1a940af440 unknown :-1 s=CLOSED pgs=3247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:28.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.083 7f1a9b561700 1 -- 172.21.2.101:0/1722919384 >> 172.21.2.101:0/1722919384 conn(0x7f1a940a0030 msgr2=0x7f1a940f22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:28.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.083 7f1a9b561700 1 -- 172.21.2.101:0/1722919384 shutdown_connections 2021-02-10T11:48:28.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.083 7f1a9b561700 1 -- 172.21.2.101:0/1722919384 wait complete. 2021-02-10T11:48:28.088 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:48:28.529 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.517 7f81dca40700 1 Processor -- start 2021-02-10T11:48:28.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.517 7f81dca40700 1 -- start start 2021-02-10T11:48:28.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.518 7f81dca40700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f81d80d1c60 0x7f81d80cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:28.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.518 7f81dca40700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f81d80cf570 0x7f81d80cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:28.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.518 7f81dca40700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d80fe1b0 con 0x7f81d80d1860 2021-02-10T11:48:28.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.518 7f81dca40700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d81b0ca0 con 0x7f81d80d1c60 2021-02-10T11:48:28.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.518 7f81dca40700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d8109a50 con 0x7f81d80cf570 2021-02-10T11:48:28.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.518 7f81d5e1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f81d80d1c60 0x7f81d80cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:28.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.518 7f81d5e1b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f81d80d1c60 0x7f81d80cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34972/0 (socket says 172.21.2.101:34972) 2021-02-10T11:48:28.530 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.518 7f81d5e1b700 1 -- 172.21.2.101:0/2589416680 learned_addr learned my addr 172.21.2.101:0/2589416680 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:28.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.518 7f81d5e1b700 1 -- 172.21.2.101:0/2589416680 >> v2:172.21.2.108:3300/0 conn(0x7f81d80cf570 msgr2=0x7f81d80cf970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:48:28.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.518 7f81d5e1b700 1 --2- 172.21.2.101:0/2589416680 >> v2:172.21.2.108:3300/0 conn(0x7f81d80cf570 0x7f81d80cf970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:28.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.518 7f81d5e1b700 1 -- 172.21.2.101:0/2589416680 >> v1:172.21.2.108:6789/0 conn(0x7f81d80d1860 legacy=0x7f81d81b04b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:48:28.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.518 7f81d5e1b700 1 -- 172.21.2.101:0/2589416680 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d8088070 con 0x7f81d80d1c60 2021-02-10T11:48:28.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.518 7f81d5e1b700 1 --2- 172.21.2.101:0/2589416680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81d80d1c60 0x7f81d80cef50 secure :-1 s=READY pgs=3248 cs=0 l=1 rev1=1 rx=0x7f81c8006700 tx=0x7f81c8009b00).ready entity=mon.0 client_cookie=4c51e8aff9c0a7c2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:28.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.518 7f81d561a700 1 -- 172.21.2.101:0/2589416680 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81c800c0c0 con 0x7f81d80d1c60 2021-02-10T11:48:28.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.518 7f81d561a700 1 -- 172.21.2.101:0/2589416680 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c800c400 con 0x7f81d80d1c60 2021-02-10T11:48:28.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.518 7f81d561a700 1 -- 172.21.2.101:0/2589416680 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81c800c860 con 0x7f81d80d1c60 2021-02-10T11:48:28.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.518 7f81dca40700 1 -- 172.21.2.101:0/2589416680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81d80d1c60 msgr2=0x7f81d80cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:28.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.518 7f81dca40700 1 --2- 172.21.2.101:0/2589416680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81d80d1c60 0x7f81d80cef50 secure :-1 s=READY pgs=3248 cs=0 l=1 rev1=1 rx=0x7f81c8006700 tx=0x7f81c8009b00).stop 2021-02-10T11:48:28.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.519 7f81dca40700 1 -- 172.21.2.101:0/2589416680 shutdown_connections 2021-02-10T11:48:28.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.519 7f81dca40700 1 --2- 172.21.2.101:0/2589416680 >> v2:172.21.2.108:3300/0 conn(0x7f81d80cf570 0x7f81d80cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:28.531 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.519 7f81dca40700 1 --2- 172.21.2.101:0/2589416680 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81d80d1c60 0x7f81d80cef50 unknown :-1 s=CLOSED pgs=3248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:28.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.519 7f81dca40700 1 -- 172.21.2.101:0/2589416680 >> 172.21.2.101:0/2589416680 conn(0x7f81d80ce410 msgr2=0x7f81d80d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:28.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.519 7f81dca40700 1 -- 172.21.2.101:0/2589416680 shutdown_connections 2021-02-10T11:48:28.532 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.519 7f81dca40700 1 -- 172.21.2.101:0/2589416680 wait complete. 2021-02-10T11:48:28.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.533 7f81dca40700 1 Processor -- start 2021-02-10T11:48:28.533 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.534 7f81dca40700 1 -- start start 2021-02-10T11:48:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.534 7f81dca40700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81d8099860 0x7f81d8194f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.534 7f81dca40700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81d8099c60 0x7f81d8195540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.534 7f81dca40700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81d818a540 0x7f81d818a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.534 7f81dca40700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d8189fc0 con 0x7f81d818a540 2021-02-10T11:48:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.534 7f81dca40700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d8090820 con 0x7f81d8099860 2021-02-10T11:48:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.534 7f81dca40700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d8090a00 con 0x7f81d8099c60 2021-02-10T11:48:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.534 7f81d6e1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81d818a540 0x7f81d818a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.534 7f81d6e1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81d818a540 0x7f81d818a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34978/0 (socket says 172.21.2.101:34978) 2021-02-10T11:48:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.534 7f81d6e1d700 1 -- 172.21.2.101:0/3219842487 learned_addr learned my addr 172.21.2.101:0/3219842487 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.534 7f81d6e1d700 1 -- 172.21.2.101:0/3219842487 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81d8099c60 msgr2=0x7f81d8195540 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.534 7f81d6e1d700 1 --2- 172.21.2.101:0/3219842487 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81d8099c60 0x7f81d8195540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:28.534 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.534 7f81d6e1d700 1 -- 172.21.2.101:0/3219842487 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81d8099860 msgr2=0x7f81d8194f20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.534 7f81d6e1d700 1 --2- 172.21.2.101:0/3219842487 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81d8099860 0x7f81d8194f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.534 7f81d6e1d700 1 -- 172.21.2.101:0/3219842487 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d818af60 con 0x7f81d818a540 2021-02-10T11:48:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.535 7f81d6e1d700 1 --2- 172.21.2.101:0/3219842487 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81d818a540 0x7f81d818a940 secure :-1 s=READY pgs=3249 cs=0 l=1 rev1=1 rx=0x7f81d0009960 tx=0x7f81d000bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.535 7f81ceffd700 1 -- 172.21.2.101:0/3219842487 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81d0003030 con 0x7f81d818a540 2021-02-10T11:48:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.535 7f81ceffd700 1 -- 172.21.2.101:0/3219842487 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d0001310 con 0x7f81d818a540 2021-02-10T11:48:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.535 7f81ceffd700 1 -- 172.21.2.101:0/3219842487 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81d0001770 con 0x7f81d818a540 2021-02-10T11:48:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.535 7f81dca40700 1 -- 172.21.2.101:0/3219842487 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d818af60 con 0x7f81d818a540 2021-02-10T11:48:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.535 7f81dca40700 1 -- 172.21.2.101:0/3219842487 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d818af60 con 0x7f81d818a540 2021-02-10T11:48:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.535 7f81ceffd700 1 -- 172.21.2.101:0/3219842487 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f81d00014f0 con 0x7f81d818a540 2021-02-10T11:48:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.535 7f81ceffd700 1 --2- 172.21.2.101:0/3219842487 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81bc018730 0x7f81bc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.535 7f81ceffd700 1 -- 172.21.2.101:0/3219842487 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f81d00329f0 con 0x7f81d818a540 2021-02-10T11:48:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.535 7f81d661c700 1 --2- 172.21.2.101:0/3219842487 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81bc018730 0x7f81bc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:28.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.536 7f81d661c700 1 --2- 172.21.2.101:0/3219842487 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81bc018730 0x7f81bc01abc0 crc :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:28.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.567 7f81dca40700 1 -- 172.21.2.101:0/3219842487 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d8082980 con 0x7f81d818a540 2021-02-10T11:48:28.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.570 7f81ceffd700 1 -- 172.21.2.101:0/3219842487 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f81d0001520 con 0x7f81d818a540 2021-02-10T11:48:28.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.699 7f81dca40700 1 -- 172.21.2.101:0/3219842487 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f81d8082980 con 0x7f81d818a540 2021-02-10T11:48:28.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.702 7f81ceffd700 1 -- 172.21.2.101:0/3219842487 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v110) v1 ==== 184+0+0 (secure 0 0 0) 0x7f81d0000d10 con 0x7f81d818a540 2021-02-10T11:48:28.702 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.0 2021-02-10T11:48:28.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.707 7f81dca40700 1 -- 172.21.2.101:0/3219842487 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81bc018730 msgr2=0x7f81bc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:28.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.707 7f81dca40700 1 --2- 172.21.2.101:0/3219842487 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81bc018730 0x7f81bc01abc0 crc :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:28.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.707 7f81dca40700 1 -- 172.21.2.101:0/3219842487 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81d818a540 msgr2=0x7f81d818a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:28.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.707 7f81dca40700 1 --2- 172.21.2.101:0/3219842487 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81d818a540 0x7f81d818a940 secure :-1 s=READY pgs=3249 cs=0 l=1 rev1=1 rx=0x7f81d0009960 tx=0x7f81d000bcf0).stop 2021-02-10T11:48:28.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.708 7f81dca40700 1 -- 172.21.2.101:0/3219842487 shutdown_connections 2021-02-10T11:48:28.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.708 7f81dca40700 1 --2- 172.21.2.101:0/3219842487 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f81bc018730 0x7f81bc01abc0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:28.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.708 7f81dca40700 1 --2- 172.21.2.101:0/3219842487 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f81d8099c60 0x7f81d8195540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:28.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.708 7f81dca40700 1 --2- 172.21.2.101:0/3219842487 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f81d818a540 0x7f81d818a940 unknown :-1 s=CLOSED pgs=3249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:28.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.708 7f81dca40700 1 --2- 172.21.2.101:0/3219842487 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f81d8099860 0x7f81d8194f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:28.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.708 7f81dca40700 1 -- 172.21.2.101:0/3219842487 >> 172.21.2.101:0/3219842487 conn(0x7f81d819f820 msgr2=0x7f81d81a5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:28.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.708 7f81dca40700 1 -- 172.21.2.101:0/3219842487 shutdown_connections 2021-02-10T11:48:28.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:28.708 7f81dca40700 1 -- 172.21.2.101:0/3219842487 wait complete. 2021-02-10T11:48:28.713 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:48:29.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.133 7fc0e2298700 1 Processor -- start 2021-02-10T11:48:29.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.133 7fc0e2298700 1 -- start start 2021-02-10T11:48:29.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.133 7fc0e2298700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc0dc0d1880 0x7fc0dc1b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:29.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.133 7fc0e2298700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0dc0cff80 0x7fc0dc109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:29.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.133 7fc0e2298700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0dc109a70 con 0x7fc0dc1b0af0 2021-02-10T11:48:29.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.133 7fc0e2298700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0dc109760 con 0x7fc0dc0d1880 2021-02-10T11:48:29.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.133 7fc0e2298700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc088070 con 0x7fc0dc0cff80 2021-02-10T11:48:29.147 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.134 7fc0dbfff700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc0dc0d1880 0x7fc0dc1b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:29.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.134 7fc0dbfff700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc0dc0d1880 0x7fc0dc1b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:45470/0 (socket says 172.21.2.101:45470) 2021-02-10T11:48:29.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.134 7fc0dbfff700 1 -- 172.21.2.101:0/132578840 learned_addr learned my addr 172.21.2.101:0/132578840 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:29.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.134 7fc0e08b5700 1 --2- 172.21.2.101:0/132578840 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0dc0cff80 0x7fc0dc109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:29.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.134 7fc0e08b5700 1 -- 172.21.2.101:0/132578840 >> v2:172.21.2.108:3300/0 conn(0x7fc0dc0d1880 msgr2=0x7fc0dc1b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:29.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.134 7fc0e08b5700 1 --2- 172.21.2.101:0/132578840 >> v2:172.21.2.108:3300/0 conn(0x7fc0dc0d1880 0x7fc0dc1b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:29.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.134 7fc0e08b5700 1 -- 172.21.2.101:0/132578840 >> v1:172.21.2.108:6789/0 conn(0x7fc0dc1b0af0 legacy=0x7fc0dc108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:29.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.134 7fc0e08b5700 1 -- 172.21.2.101:0/132578840 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0dc0d1c80 con 0x7fc0dc0cff80 2021-02-10T11:48:29.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.134 7fc0dbfff700 1 --2- 172.21.2.101:0/132578840 >> v2:172.21.2.108:3300/0 conn(0x7fc0dc0d1880 0x7fc0dc1b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:48:29.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.134 7fc0e08b5700 1 --2- 172.21.2.101:0/132578840 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0dc0cff80 0x7fc0dc109140 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fc0d4006e50 tx=0x7fc0d4009e70).ready entity=mon.2 client_cookie=21ab660bf0c407a8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:29.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.135 7fc0daffd700 1 -- 172.21.2.101:0/132578840 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc0d4006a20 con 0x7fc0dc0cff80 2021-02-10T11:48:29.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.135 7fc0daffd700 1 -- 172.21.2.101:0/132578840 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d4006790 con 0x7fc0dc0cff80 2021-02-10T11:48:29.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.135 7fc0daffd700 1 -- 172.21.2.101:0/132578840 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc0d400b700 con 0x7fc0dc0cff80 2021-02-10T11:48:29.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.135 7fc0e2298700 1 -- 172.21.2.101:0/132578840 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0dc0cff80 msgr2=0x7fc0dc109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:29.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.135 7fc0e2298700 1 --2- 172.21.2.101:0/132578840 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0dc0cff80 0x7fc0dc109140 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fc0d4006e50 tx=0x7fc0d4009e70).stop 2021-02-10T11:48:29.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.135 7fc0e2298700 1 -- 172.21.2.101:0/132578840 shutdown_connections 2021-02-10T11:48:29.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.135 7fc0e2298700 1 --2- 172.21.2.101:0/132578840 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0dc0cff80 0x7fc0dc109140 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:29.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.135 7fc0e2298700 1 --2- 172.21.2.101:0/132578840 >> v2:172.21.2.108:3300/0 conn(0x7fc0dc0d1880 0x7fc0dc1b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:29.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.135 7fc0e2298700 1 -- 172.21.2.101:0/132578840 >> 172.21.2.101:0/132578840 conn(0x7fc0dc0ce430 msgr2=0x7fc0dc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:29.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.135 7fc0e2298700 1 -- 172.21.2.101:0/132578840 shutdown_connections 2021-02-10T11:48:29.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.135 7fc0e2298700 1 -- 172.21.2.101:0/132578840 wait complete. 2021-02-10T11:48:29.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.149 7fc0e2298700 1 Processor -- start 2021-02-10T11:48:29.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.150 7fc0e2298700 1 -- start start 2021-02-10T11:48:29.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.150 7fc0e2298700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0dc08d090 0x7fc0dc08c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:29.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.150 7fc0e2298700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0dc08d490 0x7fc0dc0beec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:29.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.150 7fc0e2298700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0dc0bf4e0 0x7fc0dc09fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:29.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.150 7fc0e2298700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc0bf8e0 con 0x7fc0dc08d490 2021-02-10T11:48:29.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.150 7fc0e2298700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc0a0c90 con 0x7fc0dc08d090 2021-02-10T11:48:29.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.150 7fc0e2298700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc0a0eb0 con 0x7fc0dc0bf4e0 2021-02-10T11:48:29.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.150 7fc0db7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0dc08d490 0x7fc0dc0beec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:29.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.150 7fc0db7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0dc08d490 0x7fc0dc0beec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34990/0 (socket says 172.21.2.101:34990) 2021-02-10T11:48:29.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.150 7fc0db7fe700 1 -- 172.21.2.101:0/1199134553 learned_addr learned my addr 172.21.2.101:0/1199134553 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:29.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.151 7fc0dbfff700 1 --2- 172.21.2.101:0/1199134553 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0dc08d090 0x7fc0dc08c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:29.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.151 7fc0e08b5700 1 --2- 172.21.2.101:0/1199134553 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0dc0bf4e0 0x7fc0dc09fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:29.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.151 7fc0db7fe700 1 -- 172.21.2.101:0/1199134553 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0dc0bf4e0 msgr2=0x7fc0dc09fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:29.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.151 7fc0db7fe700 1 --2- 172.21.2.101:0/1199134553 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0dc0bf4e0 0x7fc0dc09fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:29.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.151 7fc0db7fe700 1 -- 172.21.2.101:0/1199134553 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0dc08d090 msgr2=0x7fc0dc08c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:29.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.151 7fc0db7fe700 1 --2- 172.21.2.101:0/1199134553 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0dc08d090 0x7fc0dc08c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:29.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.151 7fc0db7fe700 1 -- 172.21.2.101:0/1199134553 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0dc0a0260 con 0x7fc0dc08d490 2021-02-10T11:48:29.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.151 7fc0e08b5700 1 --2- 172.21.2.101:0/1199134553 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0dc0bf4e0 0x7fc0dc09fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:48:29.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.151 7fc0dbfff700 1 --2- 172.21.2.101:0/1199134553 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0dc08d090 0x7fc0dc08c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:48:29.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.151 7fc0db7fe700 1 --2- 172.21.2.101:0/1199134553 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0dc08d490 0x7fc0dc0beec0 secure :-1 s=READY pgs=3250 cs=0 l=1 rev1=1 rx=0x7fc0d0006ed0 tx=0x7fc0d000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:29.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.151 7fc0d8ff9700 1 -- 172.21.2.101:0/1199134553 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc0d0006240 con 0x7fc0dc08d490 2021-02-10T11:48:29.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.151 7fc0d8ff9700 1 -- 172.21.2.101:0/1199134553 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d00009b0 con 0x7fc0dc08d490 2021-02-10T11:48:29.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.151 7fc0d8ff9700 1 -- 172.21.2.101:0/1199134553 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc0d0005730 con 0x7fc0dc08d490 2021-02-10T11:48:29.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.151 7fc0e2298700 1 -- 172.21.2.101:0/1199134553 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0dc0a0260 con 0x7fc0dc08d490 2021-02-10T11:48:29.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.151 7fc0e2298700 1 -- 172.21.2.101:0/1199134553 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0dc0a0260 con 0x7fc0dc08d490 2021-02-10T11:48:29.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.152 7fc0d8ff9700 1 -- 172.21.2.101:0/1199134553 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc0d0006030 con 0x7fc0dc08d490 2021-02-10T11:48:29.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.152 7fc0d8ff9700 1 --2- 172.21.2.101:0/1199134553 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0bc0186d0 0x7fc0bc01abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:29.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.152 7fc0d8ff9700 1 -- 172.21.2.101:0/1199134553 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc0d00327c0 con 0x7fc0dc08d490 2021-02-10T11:48:29.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.152 7fc0dbfff700 1 --2- 172.21.2.101:0/1199134553 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0bc0186d0 0x7fc0bc01abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:29.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.152 7fc0dbfff700 1 --2- 172.21.2.101:0/1199134553 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0bc0186d0 0x7fc0bc01abd0 crc :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:29.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.183 7fc0e2298700 1 -- 172.21.2.101:0/1199134553 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0dc082980 con 0x7fc0dc08d490 2021-02-10T11:48:29.182 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.186 7fc0d8ff9700 1 -- 172.21.2.101:0/1199134553 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc0d0000de0 con 0x7fc0dc08d490 2021-02-10T11:48:29.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.316 7fc0e2298700 1 -- 172.21.2.101:0/1199134553 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fc0dc082980 con 0x7fc0dc08d490 2021-02-10T11:48:29.315 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.319 7fc0d8ff9700 1 -- 172.21.2.101:0/1199134553 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v111) v1 ==== 184+0+0 (secure 0 0 0) 0x7fc0d0006030 con 0x7fc0dc08d490 2021-02-10T11:48:29.319 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.1 2021-02-10T11:48:29.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.324 7fc0e2298700 1 -- 172.21.2.101:0/1199134553 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0bc0186d0 msgr2=0x7fc0bc01abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:29.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.325 7fc0e2298700 1 --2- 172.21.2.101:0/1199134553 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0bc0186d0 0x7fc0bc01abd0 crc :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:29.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.325 7fc0e2298700 1 -- 172.21.2.101:0/1199134553 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0dc08d490 msgr2=0x7fc0dc0beec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:29.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.325 7fc0e2298700 1 --2- 172.21.2.101:0/1199134553 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0dc08d490 0x7fc0dc0beec0 secure :-1 s=READY pgs=3250 cs=0 l=1 rev1=1 rx=0x7fc0d0006ed0 tx=0x7fc0d000b960).stop 2021-02-10T11:48:29.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.325 7fc0e2298700 1 -- 172.21.2.101:0/1199134553 shutdown_connections 2021-02-10T11:48:29.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.325 7fc0e2298700 1 --2- 172.21.2.101:0/1199134553 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0bc0186d0 0x7fc0bc01abd0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:29.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.325 7fc0e2298700 1 --2- 172.21.2.101:0/1199134553 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc0dc0bf4e0 0x7fc0dc09fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:29.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.325 7fc0e2298700 1 --2- 172.21.2.101:0/1199134553 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0dc08d490 0x7fc0dc0beec0 unknown :-1 s=CLOSED pgs=3250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:29.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.325 7fc0e2298700 1 --2- 172.21.2.101:0/1199134553 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc0dc08d090 0x7fc0dc08c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:29.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.325 7fc0e2298700 1 -- 172.21.2.101:0/1199134553 >> 172.21.2.101:0/1199134553 conn(0x7fc0dc087390 msgr2=0x7fc0dc0a6770 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:29.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.325 7fc0e2298700 1 -- 172.21.2.101:0/1199134553 shutdown_connections 2021-02-10T11:48:29.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.325 7fc0e2298700 1 -- 172.21.2.101:0/1199134553 wait complete. 2021-02-10T11:48:29.330 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:48:29.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.758 7f9468dee700 1 Processor -- start 2021-02-10T11:48:29.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.758 7f9468dee700 1 -- start start 2021-02-10T11:48:29.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.759 7f9468dee700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f94640d1880 0x7f94641b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:29.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.759 7f9468dee700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94640cff80 0x7f9464109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:29.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.759 7f9468dee700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9464109a70 con 0x7f94641b0af0 2021-02-10T11:48:29.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.759 7f9468dee700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9464109760 con 0x7f94640d1880 2021-02-10T11:48:29.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.759 7f9468dee700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9464088070 con 0x7f94640cff80 2021-02-10T11:48:29.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.759 7f9462b13700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f94640d1880 0x7f94641b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:29.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.759 7f9462b13700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f94640d1880 0x7f94641b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:34996/0 (socket says 172.21.2.101:34996) 2021-02-10T11:48:29.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.759 7f9462b13700 1 -- 172.21.2.101:0/3055206479 learned_addr learned my addr 172.21.2.101:0/3055206479 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:29.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.759 7f9462312700 1 --1- 172.21.2.101:0/3055206479 >> v1:172.21.2.108:6789/0 conn(0x7f94641b0af0 0x7f9464108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:57134/0 (socket says 172.21.2.101:57134) 2021-02-10T11:48:29.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.759 7f9463314700 1 --2- 172.21.2.101:0/3055206479 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94640cff80 0x7f9464109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:29.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.759 7f9462b13700 1 -- 172.21.2.101:0/3055206479 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94640cff80 msgr2=0x7f9464109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:29.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.759 7f9462b13700 1 --2- 172.21.2.101:0/3055206479 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94640cff80 0x7f9464109140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:29.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.759 7f9462b13700 1 -- 172.21.2.101:0/3055206479 >> v1:172.21.2.108:6789/0 conn(0x7f94641b0af0 legacy=0x7f9464108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:29.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.759 7f9462b13700 1 -- 172.21.2.101:0/3055206479 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94640d1c80 con 0x7f94640d1880 2021-02-10T11:48:29.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.760 7f9462b13700 1 --2- 172.21.2.101:0/3055206479 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94640d1880 0x7f94641b04d0 secure :-1 s=READY pgs=3251 cs=0 l=1 rev1=1 rx=0x7f9454006700 tx=0x7f9454009b00).ready entity=mon.0 client_cookie=941b2872358eac10 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:29.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.760 7f9461b11700 1 -- 172.21.2.101:0/3055206479 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f945400c0c0 con 0x7f94640d1880 2021-02-10T11:48:29.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.760 7f9461b11700 1 -- 172.21.2.101:0/3055206479 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945400c400 con 0x7f94640d1880 2021-02-10T11:48:29.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.760 7f9461b11700 1 -- 172.21.2.101:0/3055206479 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f945400c920 con 0x7f94640d1880 2021-02-10T11:48:29.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.760 7f9468dee700 1 -- 172.21.2.101:0/3055206479 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94640d1880 msgr2=0x7f94641b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:29.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.760 7f9468dee700 1 --2- 172.21.2.101:0/3055206479 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94640d1880 0x7f94641b04d0 secure :-1 s=READY pgs=3251 cs=0 l=1 rev1=1 rx=0x7f9454006700 tx=0x7f9454009b00).stop 2021-02-10T11:48:29.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.761 7f9468dee700 1 -- 172.21.2.101:0/3055206479 shutdown_connections 2021-02-10T11:48:29.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.761 7f9468dee700 1 --2- 172.21.2.101:0/3055206479 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f94640cff80 0x7f9464109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:29.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.761 7f9468dee700 1 --2- 172.21.2.101:0/3055206479 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94640d1880 0x7f94641b04d0 unknown :-1 s=CLOSED pgs=3251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:29.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.761 7f9468dee700 1 -- 172.21.2.101:0/3055206479 >> 172.21.2.101:0/3055206479 conn(0x7f94640ce430 msgr2=0x7f94640d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:29.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.761 7f9468dee700 1 -- 172.21.2.101:0/3055206479 shutdown_connections 2021-02-10T11:48:29.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.761 7f9468dee700 1 -- 172.21.2.101:0/3055206479 wait complete. 2021-02-10T11:48:29.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.776 7f9468dee700 1 Processor -- start 2021-02-10T11:48:29.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.776 7f9468dee700 1 -- start start 2021-02-10T11:48:29.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.776 7f9468dee700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94641a0580 0x7f9464086fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:29.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.776 7f9468dee700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94640b66b0 0x7f94640b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:29.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.776 7f9468dee700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9464082bb0 0x7f9464082fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:29.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.776 7f9468dee700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94640875d0 con 0x7f94640b66b0 2021-02-10T11:48:29.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.776 7f9468dee700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94640835d0 con 0x7f94641a0580 2021-02-10T11:48:29.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.776 7f9468dee700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94640837f0 con 0x7f9464082bb0 2021-02-10T11:48:29.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.776 7f9462312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94640b66b0 0x7f94640b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:29.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.776 7f9462312700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94640b66b0 0x7f94640b6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35004/0 (socket says 172.21.2.101:35004) 2021-02-10T11:48:29.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.776 7f9462312700 1 -- 172.21.2.101:0/866544193 learned_addr learned my addr 172.21.2.101:0/866544193 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:29.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.777 7f9462312700 1 -- 172.21.2.101:0/866544193 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9464082bb0 msgr2=0x7f9464082fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:29.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.777 7f9462312700 1 --2- 172.21.2.101:0/866544193 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9464082bb0 0x7f9464082fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:29.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.777 7f9462312700 1 -- 172.21.2.101:0/866544193 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94641a0580 msgr2=0x7f9464086fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:29.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.777 7f9462312700 1 --2- 172.21.2.101:0/866544193 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94641a0580 0x7f9464086fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:29.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.777 7f9462312700 1 -- 172.21.2.101:0/866544193 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9464083a10 con 0x7f94640b66b0 2021-02-10T11:48:29.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.777 7f9462312700 1 --2- 172.21.2.101:0/866544193 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94640b66b0 0x7f94640b6ab0 secure :-1 s=READY pgs=3252 cs=0 l=1 rev1=1 rx=0x7f944c001c60 tx=0x7f944c00a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:29.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.777 7f945b7fe700 1 -- 172.21.2.101:0/866544193 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f944c00acc0 con 0x7f94640b66b0 2021-02-10T11:48:29.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.777 7f945b7fe700 1 -- 172.21.2.101:0/866544193 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944c004140 con 0x7f94640b66b0 2021-02-10T11:48:29.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.777 7f945b7fe700 1 -- 172.21.2.101:0/866544193 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f944c0045a0 con 0x7f94640b66b0 2021-02-10T11:48:29.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.777 7f9468dee700 1 -- 172.21.2.101:0/866544193 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9464083a10 con 0x7f94640b66b0 2021-02-10T11:48:29.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.777 7f9468dee700 1 -- 172.21.2.101:0/866544193 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9464083fa0 con 0x7f94640b66b0 2021-02-10T11:48:29.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.778 7f945b7fe700 1 -- 172.21.2.101:0/866544193 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f944c000d90 con 0x7f94640b66b0 2021-02-10T11:48:29.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.778 7f945b7fe700 1 --2- 172.21.2.101:0/866544193 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94480186c0 0x7f944801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:29.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.778 7f945b7fe700 1 -- 172.21.2.101:0/866544193 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f944c032570 con 0x7f94640b66b0 2021-02-10T11:48:29.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.778 7f9462b13700 1 --2- 172.21.2.101:0/866544193 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94480186c0 0x7f944801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:29.778 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.778 7f9462b13700 1 --2- 172.21.2.101:0/866544193 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94480186c0 0x7f944801abc0 crc :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:29.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.808 7f9468dee700 1 -- 172.21.2.101:0/866544193 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94640875d0 con 0x7f94640b66b0 2021-02-10T11:48:29.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.811 7f945b7fe700 1 -- 172.21.2.101:0/866544193 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f944c0009b0 con 0x7f94640b66b0 2021-02-10T11:48:29.940 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.943 7f9468dee700 1 -- 172.21.2.101:0/866544193 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f94640875d0 con 0x7f94640b66b0 2021-02-10T11:48:29.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.947 7f945b7fe700 1 -- 172.21.2.101:0/866544193 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v112) v1 ==== 184+0+0 (secure 0 0 0) 0x7f944c01f500 con 0x7f94640b66b0 2021-02-10T11:48:29.947 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.2 2021-02-10T11:48:29.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.952 7f9468dee700 1 -- 172.21.2.101:0/866544193 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94480186c0 msgr2=0x7f944801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:29.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.952 7f9468dee700 1 --2- 172.21.2.101:0/866544193 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94480186c0 0x7f944801abc0 crc :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:29.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.952 7f9468dee700 1 -- 172.21.2.101:0/866544193 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94640b66b0 msgr2=0x7f94640b6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:29.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.952 7f9468dee700 1 --2- 172.21.2.101:0/866544193 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94640b66b0 0x7f94640b6ab0 secure :-1 s=READY pgs=3252 cs=0 l=1 rev1=1 rx=0x7f944c001c60 tx=0x7f944c00a8d0).stop 2021-02-10T11:48:29.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.952 7f9468dee700 1 -- 172.21.2.101:0/866544193 shutdown_connections 2021-02-10T11:48:29.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.952 7f9468dee700 1 --2- 172.21.2.101:0/866544193 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f94480186c0 0x7f944801abc0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:29.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.952 7f9468dee700 1 --2- 172.21.2.101:0/866544193 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9464082bb0 0x7f9464082fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:29.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.952 7f9468dee700 1 --2- 172.21.2.101:0/866544193 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f94640b66b0 0x7f94640b6ab0 unknown :-1 s=CLOSED pgs=3252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:29.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.952 7f9468dee700 1 --2- 172.21.2.101:0/866544193 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f94641a0580 0x7f9464086fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:29.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.952 7f9468dee700 1 -- 172.21.2.101:0/866544193 >> 172.21.2.101:0/866544193 conn(0x7f94640afc60 msgr2=0x7f94640b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:29.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.952 7f9468dee700 1 -- 172.21.2.101:0/866544193 shutdown_connections 2021-02-10T11:48:29.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:29.952 7f9468dee700 1 -- 172.21.2.101:0/866544193 wait complete. 2021-02-10T11:48:29.957 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:48:30.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.376 7fdc85386700 1 Processor -- start 2021-02-10T11:48:30.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.376 7fdc85386700 1 -- start start 2021-02-10T11:48:30.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.376 7fdc85386700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fdc800d1c80 0x7fdc800cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:30.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.376 7fdc85386700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc800d0180 con 0x7fdc800d1880 2021-02-10T11:48:30.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.376 7fdc85386700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc800fe1d0 con 0x7fdc800cf590 2021-02-10T11:48:30.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.376 7fdc85386700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc801b0cc0 con 0x7fdc800d1c80 2021-02-10T11:48:30.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.376 7fdc7f07d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fdc800d1880 0x7fdc801b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35504/0 (socket says 172.21.2.101:35504) 2021-02-10T11:48:30.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.376 7fdc7f07d700 1 -- 172.21.2.101:0/920225090 learned_addr learned my addr 172.21.2.101:0/920225090 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:30.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.377 7fdc7e07b700 1 -- 172.21.2.101:0/920225090 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815361031 0 0) 0x7fdc68000ce0 con 0x7fdc800d1880 2021-02-10T11:48:30.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.377 7fdc7e07b700 1 -- 172.21.2.101:0/920225090 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc6c002fe0 con 0x7fdc800d1880 2021-02-10T11:48:30.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.377 7fdc7e87c700 1 --2- 172.21.2.101:0/920225090 >> v2:172.21.2.108:3300/0 conn(0x7fdc800d1c80 0x7fdc800cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:30.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.377 7fdc7e07b700 1 -- 172.21.2.101:0/920225090 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1238553194 0 0) 0x7fdc680010e0 con 0x7fdc800d1880 2021-02-10T11:48:30.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.377 7fdc7e07b700 1 -- 172.21.2.101:0/920225090 >> v2:172.21.2.108:3300/0 conn(0x7fdc800d1c80 msgr2=0x7fdc800cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:30.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.377 7fdc7e07b700 1 --2- 172.21.2.101:0/920225090 >> v2:172.21.2.108:3300/0 conn(0x7fdc800d1c80 0x7fdc800cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:30.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.377 7fdc7e07b700 1 -- 172.21.2.101:0/920225090 >> v1:172.21.2.108:6789/0 conn(0x7fdc800cf590 legacy=0x7fdc800cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:30.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.377 7fdc7e07b700 1 -- 172.21.2.101:0/920225090 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc80109a70 con 0x7fdc800d1880 2021-02-10T11:48:30.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.377 7fdc7e07b700 1 -- 172.21.2.101:0/920225090 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fdc68001600 con 0x7fdc800d1880 2021-02-10T11:48:30.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.377 7fdc7e07b700 1 -- 172.21.2.101:0/920225090 >> v1:172.21.2.101:6789/0 conn(0x7fdc800d1880 legacy=0x7fdc801b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:30.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.377 7fdc7e07b700 1 --2- 172.21.2.101:0/920225090 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc6c003af0 0x7fdc6c005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:30.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.377 7fdc7e07b700 1 --2- 172.21.2.101:0/920225090 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc6c0067a0 0x7fdc6c008cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:30.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.377 7fdc7e07b700 1 --2- 172.21.2.101:0/920225090 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6c0095a0 0x7fdc6c00baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:30.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.377 7fdc7e07b700 1 -- 172.21.2.101:0/920225090 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c00c2f0 con 0x7fdc6c0095a0 2021-02-10T11:48:30.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.377 7fdc7e07b700 1 -- 172.21.2.101:0/920225090 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c00c510 con 0x7fdc6c003af0 2021-02-10T11:48:30.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.377 7fdc7e07b700 1 -- 172.21.2.101:0/920225090 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c00c730 con 0x7fdc6c0067a0 2021-02-10T11:48:30.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.377 7fdc7f87e700 1 --2- 172.21.2.101:0/920225090 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6c0095a0 0x7fdc6c00baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:30.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.378 7fdc7e87c700 1 --2- 172.21.2.101:0/920225090 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc6c0067a0 0x7fdc6c008cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:30.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.378 7fdc7f07d700 1 --2- 172.21.2.101:0/920225090 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc6c003af0 0x7fdc6c005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:30.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.378 7fdc7f87e700 1 -- 172.21.2.101:0/920225090 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc6c0067a0 msgr2=0x7fdc6c008cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:30.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.378 7fdc7f87e700 1 --2- 172.21.2.101:0/920225090 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc6c0067a0 0x7fdc6c008cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:30.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.378 7fdc7f87e700 1 -- 172.21.2.101:0/920225090 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc6c003af0 msgr2=0x7fdc6c005f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:30.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.378 7fdc7f87e700 1 --2- 172.21.2.101:0/920225090 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc6c003af0 0x7fdc6c005f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:30.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.378 7fdc7f87e700 1 -- 172.21.2.101:0/920225090 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c00c9d0 con 0x7fdc6c0095a0 2021-02-10T11:48:30.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.378 7fdc7f07d700 1 --2- 172.21.2.101:0/920225090 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc6c003af0 0x7fdc6c005f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:48:30.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.378 7fdc7f87e700 1 -- 172.21.2.101:0/920225090 reap_dead start 2021-02-10T11:48:30.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.378 7fdc7f87e700 1 --2- 172.21.2.101:0/920225090 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6c0095a0 0x7fdc6c00baa0 secure :-1 s=READY pgs=3254 cs=0 l=1 rev1=1 rx=0x7fdc7804d670 tx=0x7fdc7804de50).ready entity=mon.0 client_cookie=b3a41d00e84aaabf server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:30.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.378 7fdc7e07b700 1 -- 172.21.2.101:0/920225090 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc78006a60 con 0x7fdc6c0095a0 2021-02-10T11:48:30.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.378 7fdc7e07b700 1 -- 172.21.2.101:0/920225090 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7804f300 con 0x7fdc6c0095a0 2021-02-10T11:48:30.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.378 7fdc7e07b700 1 -- 172.21.2.101:0/920225090 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc7804f760 con 0x7fdc6c0095a0 2021-02-10T11:48:30.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.378 7fdc85386700 1 -- 172.21.2.101:0/920225090 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6c0095a0 msgr2=0x7fdc6c00baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:30.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.378 7fdc85386700 1 --2- 172.21.2.101:0/920225090 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6c0095a0 0x7fdc6c00baa0 secure :-1 s=READY pgs=3254 cs=0 l=1 rev1=1 rx=0x7fdc7804d670 tx=0x7fdc7804de50).stop 2021-02-10T11:48:30.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.379 7fdc85386700 1 -- 172.21.2.101:0/920225090 shutdown_connections 2021-02-10T11:48:30.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.379 7fdc85386700 1 --2- 172.21.2.101:0/920225090 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc6c0095a0 0x7fdc6c00baa0 unknown :-1 s=CLOSED pgs=3254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:30.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.379 7fdc85386700 1 -- 172.21.2.101:0/920225090 >> 172.21.2.101:0/920225090 conn(0x7fdc800ce430 msgr2=0x7fdc800d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:30.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.379 7fdc85386700 1 -- 172.21.2.101:0/920225090 shutdown_connections 2021-02-10T11:48:30.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.379 7fdc85386700 1 -- 172.21.2.101:0/920225090 wait complete. 2021-02-10T11:48:30.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.394 7fdc85386700 1 Processor -- start 2021-02-10T11:48:30.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.394 7fdc85386700 1 -- start start 2021-02-10T11:48:30.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.394 7fdc85386700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc8008f620 0x7fdc8008fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:30.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.394 7fdc85386700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc80195140 0x7fdc80195540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:30.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.394 7fdc85386700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc80083a40 0x7fdc80083e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:30.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.394 7fdc85386700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc800834c0 con 0x7fdc80083a40 2021-02-10T11:48:30.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.394 7fdc85386700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc80083220 con 0x7fdc8008f620 2021-02-10T11:48:30.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.394 7fdc85386700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc80090040 con 0x7fdc80195140 2021-02-10T11:48:30.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.394 7fdc7f87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc80083a40 0x7fdc80083e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:30.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.394 7fdc7f87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc80083a40 0x7fdc80083e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35024/0 (socket says 172.21.2.101:35024) 2021-02-10T11:48:30.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.394 7fdc7f87e700 1 -- 172.21.2.101:0/140338558 learned_addr learned my addr 172.21.2.101:0/140338558 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:30.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.395 7fdc7f87e700 1 -- 172.21.2.101:0/140338558 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc80195140 msgr2=0x7fdc80195540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:30.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.395 7fdc7f87e700 1 --2- 172.21.2.101:0/140338558 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc80195140 0x7fdc80195540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:30.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.395 7fdc7f87e700 1 -- 172.21.2.101:0/140338558 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc8008f620 msgr2=0x7fdc8008fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:30.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.395 7fdc7f87e700 1 --2- 172.21.2.101:0/140338558 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc8008f620 0x7fdc8008fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:30.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.395 7fdc7f87e700 1 -- 172.21.2.101:0/140338558 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc801a4a00 con 0x7fdc80083a40 2021-02-10T11:48:30.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.395 7fdc7f87e700 1 --2- 172.21.2.101:0/140338558 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc80083a40 0x7fdc80083e40 secure :-1 s=READY pgs=3255 cs=0 l=1 rev1=1 rx=0x7fdc780508f0 tx=0x7fdc78050c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:30.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.395 7fdc77fff700 1 -- 172.21.2.101:0/140338558 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc78000ce0 con 0x7fdc80083a40 2021-02-10T11:48:30.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.395 7fdc77fff700 1 -- 172.21.2.101:0/140338558 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc78006b20 con 0x7fdc80083a40 2021-02-10T11:48:30.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.395 7fdc77fff700 1 -- 172.21.2.101:0/140338558 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc7804b4f0 con 0x7fdc80083a40 2021-02-10T11:48:30.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.395 7fdc85386700 1 -- 172.21.2.101:0/140338558 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc801a4a00 con 0x7fdc80083a40 2021-02-10T11:48:30.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.395 7fdc85386700 1 -- 172.21.2.101:0/140338558 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc80084750 con 0x7fdc80083a40 2021-02-10T11:48:30.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.396 7fdc77fff700 1 -- 172.21.2.101:0/140338558 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdc7804b300 con 0x7fdc80083a40 2021-02-10T11:48:30.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.396 7fdc77fff700 1 --2- 172.21.2.101:0/140338558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc640186c0 0x7fdc6401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:30.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.396 7fdc77fff700 1 -- 172.21.2.101:0/140338558 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fdc780768d0 con 0x7fdc80083a40 2021-02-10T11:48:30.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.396 7fdc7f07d700 1 --2- 172.21.2.101:0/140338558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc640186c0 0x7fdc6401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:30.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.396 7fdc7f07d700 1 --2- 172.21.2.101:0/140338558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc640186c0 0x7fdc6401abc0 crc :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:30.423 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.426 7fdc85386700 1 -- 172.21.2.101:0/140338558 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc800fe1d0 con 0x7fdc80083a40 2021-02-10T11:48:30.425 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.429 7fdc77fff700 1 -- 172.21.2.101:0/140338558 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdc7806b2b0 con 0x7fdc80083a40 2021-02-10T11:48:30.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.557 7fdc85386700 1 -- 172.21.2.101:0/140338558 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fdc800fe1d0 con 0x7fdc80083a40 2021-02-10T11:48:30.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.561 7fdc77fff700 1 -- 172.21.2.101:0/140338558 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v113) v1 ==== 184+0+0 (secure 0 0 0) 0x7fdc7806b2b0 con 0x7fdc80083a40 2021-02-10T11:48:30.561 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.3 2021-02-10T11:48:30.562 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.566 7fdc85386700 1 -- 172.21.2.101:0/140338558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc640186c0 msgr2=0x7fdc6401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:30.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.566 7fdc85386700 1 --2- 172.21.2.101:0/140338558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc640186c0 0x7fdc6401abc0 crc :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:30.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.566 7fdc85386700 1 -- 172.21.2.101:0/140338558 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc80083a40 msgr2=0x7fdc80083e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:30.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.566 7fdc85386700 1 --2- 172.21.2.101:0/140338558 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc80083a40 0x7fdc80083e40 secure :-1 s=READY pgs=3255 cs=0 l=1 rev1=1 rx=0x7fdc780508f0 tx=0x7fdc78050c90).stop 2021-02-10T11:48:30.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.566 7fdc85386700 1 -- 172.21.2.101:0/140338558 shutdown_connections 2021-02-10T11:48:30.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.566 7fdc85386700 1 --2- 172.21.2.101:0/140338558 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdc640186c0 0x7fdc6401abc0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:30.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.566 7fdc85386700 1 --2- 172.21.2.101:0/140338558 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdc80195140 0x7fdc80195540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:30.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.566 7fdc85386700 1 --2- 172.21.2.101:0/140338558 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdc80083a40 0x7fdc80083e40 unknown :-1 s=CLOSED pgs=3255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:30.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.566 7fdc85386700 1 --2- 172.21.2.101:0/140338558 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdc8008f620 0x7fdc8008fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:30.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.566 7fdc85386700 1 -- 172.21.2.101:0/140338558 >> 172.21.2.101:0/140338558 conn(0x7fdc800a1070 msgr2=0x7fdc8008eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:30.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.567 7fdc85386700 1 -- 172.21.2.101:0/140338558 shutdown_connections 2021-02-10T11:48:30.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.567 7fdc85386700 1 -- 172.21.2.101:0/140338558 wait complete. 2021-02-10T11:48:30.571 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:48:30.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.986 7fbb05fe3700 1 Processor -- start 2021-02-10T11:48:30.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.986 7fbb05fe3700 1 -- start start 2021-02-10T11:48:30.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.987 7fbb05fe3700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fbb000d0340 0x7fbb000ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:30.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.987 7fbb05fe3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbb000d1c70 0x7fbb000efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:30.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.987 7fbb05fe3700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb000f0b10 con 0x7fbb000d1870 2021-02-10T11:48:30.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.987 7fbb05fe3700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb000e5ed0 con 0x7fbb000d1c70 2021-02-10T11:48:30.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.987 7fbb05fe3700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb001b0be0 con 0x7fbb000d0340 2021-02-10T11:48:30.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.987 7fbafffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbb000d1c70 0x7fbb000efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:30.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.987 7fbafffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbb000d1c70 0x7fbb000efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35030/0 (socket says 172.21.2.101:35030) 2021-02-10T11:48:30.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.987 7fbafffff700 1 -- 172.21.2.101:0/417103412 learned_addr learned my addr 172.21.2.101:0/417103412 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:30.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.987 7fbaf6ffd700 1 --2- 172.21.2.101:0/417103412 >> v2:172.21.2.108:3300/0 conn(0x7fbb000d0340 0x7fbb000ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:30.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.987 7fbafffff700 1 -- 172.21.2.101:0/417103412 >> v2:172.21.2.108:3300/0 conn(0x7fbb000d0340 msgr2=0x7fbb000ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:30.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.987 7fbafffff700 1 --2- 172.21.2.101:0/417103412 >> v2:172.21.2.108:3300/0 conn(0x7fbb000d0340 0x7fbb000ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:30.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.987 7fbafffff700 1 -- 172.21.2.101:0/417103412 >> v1:172.21.2.108:6789/0 conn(0x7fbb000d1870 legacy=0x7fbb001b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:31.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.987 7fbafffff700 1 -- 172.21.2.101:0/417103412 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb000f20a0 con 0x7fbb000d1c70 2021-02-10T11:48:31.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.987 7fbafffff700 1 --2- 172.21.2.101:0/417103412 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb000d1c70 0x7fbb000efc60 secure :-1 s=READY pgs=3256 cs=0 l=1 rev1=1 rx=0x7fbaf8006f00 tx=0x7fbaf8009f10).ready entity=mon.0 client_cookie=53b7d5aec57dae0f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:31.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.987 7fbafeffd700 1 -- 172.21.2.101:0/417103412 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbaf800b3b0 con 0x7fbb000d1c70 2021-02-10T11:48:31.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.987 7fbafeffd700 1 -- 172.21.2.101:0/417103412 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf800b3b0 con 0x7fbb000d1c70 2021-02-10T11:48:31.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.987 7fbafeffd700 1 -- 172.21.2.101:0/417103412 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbaf800b730 con 0x7fbb000d1c70 2021-02-10T11:48:31.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.987 7fbb05fe3700 1 -- 172.21.2.101:0/417103412 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb000d1c70 msgr2=0x7fbb000efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:31.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.987 7fbb05fe3700 1 --2- 172.21.2.101:0/417103412 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb000d1c70 0x7fbb000efc60 secure :-1 s=READY pgs=3256 cs=0 l=1 rev1=1 rx=0x7fbaf8006f00 tx=0x7fbaf8009f10).stop 2021-02-10T11:48:31.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.988 7fbb05fe3700 1 -- 172.21.2.101:0/417103412 shutdown_connections 2021-02-10T11:48:31.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.988 7fbb05fe3700 1 --2- 172.21.2.101:0/417103412 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb000d1c70 0x7fbb000efc60 unknown :-1 s=CLOSED pgs=3256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:31.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.988 7fbb05fe3700 1 --2- 172.21.2.101:0/417103412 >> v2:172.21.2.108:3300/0 conn(0x7fbb000d0340 0x7fbb000ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:31.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.988 7fbb05fe3700 1 -- 172.21.2.101:0/417103412 >> 172.21.2.101:0/417103412 conn(0x7fbb000ce450 msgr2=0x7fbb000d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:31.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.988 7fbb05fe3700 1 -- 172.21.2.101:0/417103412 shutdown_connections 2021-02-10T11:48:31.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:30.988 7fbb05fe3700 1 -- 172.21.2.101:0/417103412 wait complete. 2021-02-10T11:48:31.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.002 7fbb05fe3700 1 Processor -- start 2021-02-10T11:48:31.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.003 7fbb05fe3700 1 -- start start 2021-02-10T11:48:31.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.003 7fbb05fe3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb00094040 0x7fbb00093370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:31.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.003 7fbb05fe3700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb00093990 0x7fbb000ad700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:31.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.003 7fbb05fe3700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb000add20 0x7fbb001a2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:31.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.003 7fbb05fe3700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb00093d90 con 0x7fbb00094040 2021-02-10T11:48:31.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.003 7fbb05fe3700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb001a1e50 con 0x7fbb00093990 2021-02-10T11:48:31.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.003 7fbb05fe3700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb00094590 con 0x7fbb000add20 2021-02-10T11:48:31.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.003 7fbaff7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb00094040 0x7fbb00093370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:31.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.003 7fbaff7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb00094040 0x7fbb00093370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35032/0 (socket says 172.21.2.101:35032) 2021-02-10T11:48:31.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.003 7fbaff7fe700 1 -- 172.21.2.101:0/989765249 learned_addr learned my addr 172.21.2.101:0/989765249 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:31.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.004 7fbaf6ffd700 1 --2- 172.21.2.101:0/989765249 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb00093990 0x7fbb000ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:31.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.004 7fbafffff700 1 --2- 172.21.2.101:0/989765249 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb000add20 0x7fbb001a2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:31.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.004 7fbaff7fe700 1 -- 172.21.2.101:0/989765249 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb000add20 msgr2=0x7fbb001a2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:31.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.004 7fbaff7fe700 1 --2- 172.21.2.101:0/989765249 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb000add20 0x7fbb001a2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:31.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.004 7fbaff7fe700 1 -- 172.21.2.101:0/989765249 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb00093990 msgr2=0x7fbb000ad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:31.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.004 7fbaff7fe700 1 --2- 172.21.2.101:0/989765249 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb00093990 0x7fbb000ad700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:31.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.004 7fbaff7fe700 1 -- 172.21.2.101:0/989765249 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb000ae120 con 0x7fbb00094040 2021-02-10T11:48:31.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.004 7fbaf6ffd700 1 --2- 172.21.2.101:0/989765249 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb00093990 0x7fbb000ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:48:31.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.004 7fbafffff700 1 --2- 172.21.2.101:0/989765249 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb000add20 0x7fbb001a2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:48:31.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.004 7fbaff7fe700 1 --2- 172.21.2.101:0/989765249 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb00094040 0x7fbb00093370 secure :-1 s=READY pgs=3257 cs=0 l=1 rev1=1 rx=0x7fbaf000b5b0 tx=0x7fbaf000b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:31.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.004 7fbafcff9700 1 -- 172.21.2.101:0/989765249 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbaf000bd40 con 0x7fbb00094040 2021-02-10T11:48:31.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.004 7fbafcff9700 1 -- 172.21.2.101:0/989765249 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf00009b0 con 0x7fbb00094040 2021-02-10T11:48:31.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.004 7fbafcff9700 1 -- 172.21.2.101:0/989765249 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbaf00062a0 con 0x7fbb00094040 2021-02-10T11:48:31.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.004 7fbb05fe3700 1 -- 172.21.2.101:0/989765249 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb000ae120 con 0x7fbb00094040 2021-02-10T11:48:31.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.004 7fbb05fe3700 1 -- 172.21.2.101:0/989765249 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb001a2d90 con 0x7fbb00094040 2021-02-10T11:48:31.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.005 7fbafcff9700 1 -- 172.21.2.101:0/989765249 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbaf0006020 con 0x7fbb00094040 2021-02-10T11:48:31.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.005 7fbafcff9700 1 --2- 172.21.2.101:0/989765249 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbae40186c0 0x7fbae401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:31.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.005 7fbafcff9700 1 -- 172.21.2.101:0/989765249 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fbaf00325e0 con 0x7fbb00094040 2021-02-10T11:48:31.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.005 7fbaf6ffd700 1 --2- 172.21.2.101:0/989765249 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbae40186c0 0x7fbae401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:31.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.005 7fbaf6ffd700 1 --2- 172.21.2.101:0/989765249 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbae40186c0 0x7fbae401abc0 crc :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:31.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.037 7fbb05fe3700 1 -- 172.21.2.101:0/989765249 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb00093d90 con 0x7fbb00094040 2021-02-10T11:48:31.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.040 7fbafcff9700 1 -- 172.21.2.101:0/989765249 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbaf0030d90 con 0x7fbb00094040 2021-02-10T11:48:31.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.167 7fbb05fe3700 1 -- 172.21.2.101:0/989765249 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb00093d90 con 0x7fbb00094040 2021-02-10T11:48:31.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.167 7fbafcff9700 1 -- 172.21.2.101:0/989765249 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) v1 ==== 95+0+2108 (secure 0 0 0) 0x7fbaf000bd40 con 0x7fbb00094040 2021-02-10T11:48:31.164 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:48:31.164 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":164,"default_fscid":20,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":13530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/347510536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":347510536},{"type":"v1","addr":"172.21.2.108:6833","nonce":347510536}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":160}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:48:24.824003","modified":"2021-02-10 11:48:26.829128","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":13518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13518":{"gid":13518,"name":"a","rank":0,"incarnation":161,"state":"up:active","state_seq":3,"addr":"172.21.2.101:6835/2721786668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2721786668},{"type":"v1","addr":"172.21.2.101:6835","nonce":2721786668}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[41],"metadata_pool":40,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":20}]} 2021-02-10T11:48:31.164 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 164 2021-02-10T11:48:31.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.169 7fbb05fe3700 1 -- 172.21.2.101:0/989765249 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbae40186c0 msgr2=0x7fbae401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:31.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.169 7fbb05fe3700 1 --2- 172.21.2.101:0/989765249 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbae40186c0 0x7fbae401abc0 crc :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:31.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.169 7fbb05fe3700 1 -- 172.21.2.101:0/989765249 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb00094040 msgr2=0x7fbb00093370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:31.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.169 7fbb05fe3700 1 --2- 172.21.2.101:0/989765249 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb00094040 0x7fbb00093370 secure :-1 s=READY pgs=3257 cs=0 l=1 rev1=1 rx=0x7fbaf000b5b0 tx=0x7fbaf000b950).stop 2021-02-10T11:48:31.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.169 7fbb05fe3700 1 -- 172.21.2.101:0/989765249 shutdown_connections 2021-02-10T11:48:31.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.169 7fbb05fe3700 1 --2- 172.21.2.101:0/989765249 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbae40186c0 0x7fbae401abc0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:31.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.169 7fbb05fe3700 1 --2- 172.21.2.101:0/989765249 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbb000add20 0x7fbb001a2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:31.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.169 7fbb05fe3700 1 --2- 172.21.2.101:0/989765249 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbb00093990 0x7fbb000ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:31.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.169 7fbb05fe3700 1 --2- 172.21.2.101:0/989765249 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbb00094040 0x7fbb00093370 unknown :-1 s=CLOSED pgs=3257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:31.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.169 7fbb05fe3700 1 -- 172.21.2.101:0/989765249 >> 172.21.2.101:0/989765249 conn(0x7fbb0019f8a0 msgr2=0x7fbb00094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:31.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.169 7fbb05fe3700 1 -- 172.21.2.101:0/989765249 shutdown_connections 2021-02-10T11:48:31.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:31.169 7fbb05fe3700 1 -- 172.21.2.101:0/989765249 wait complete. 2021-02-10T11:48:31.375 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 164, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:48:24.824003', 'modified': '2021-02-10 11:48:26.829128', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 13518}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_13518': {'gid': 13518, 'name': 'a', 'rank': 0, 'incarnation': 161, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.101:6835/2721786668', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.101:6834', 'nonce': 2721786668}, {'type': 'v1', 'addr': '172.21.2.101:6835', 'nonce': 2721786668}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [41], 'metadata_pool': 40, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:48:31.375 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:48:31.376 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok status 2021-02-10T11:48:31.780 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:48:31.780 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:48:31.780 INFO:teuthology.orchestra.run.gibba001.stdout: "whoami": 0, 2021-02-10T11:48:31.780 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 13518, 2021-02-10T11:48:31.780 INFO:teuthology.orchestra.run.gibba001.stdout: "want_state": "up:active", 2021-02-10T11:48:31.780 INFO:teuthology.orchestra.run.gibba001.stdout: "state": "up:active", 2021-02-10T11:48:31.780 INFO:teuthology.orchestra.run.gibba001.stdout: "rank_uptime": 6.9577936659999997, 2021-02-10T11:48:31.781 INFO:teuthology.orchestra.run.gibba001.stdout: "mdsmap_epoch": 164, 2021-02-10T11:48:31.781 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch": 191, 2021-02-10T11:48:31.781 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch_barrier": 191, 2021-02-10T11:48:31.781 INFO:teuthology.orchestra.run.gibba001.stdout: "uptime": 12.616041443 2021-02-10T11:48:31.781 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:48:31.783 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 13518, "want_state": "up:active", "state": "up:active", "rank_uptime": 6.9577936659999997, "mdsmap_epoch": 164, "osdmap_epoch": 191, "osdmap_epoch_barrier": 191, "uptime": 12.616041443 } 2021-02-10T11:48:31.784 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:48:32.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.201 7fa31f376700 1 Processor -- start 2021-02-10T11:48:32.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.201 7fa31f376700 1 -- start start 2021-02-10T11:48:32.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.201 7fa31f376700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3180f05f0 0x7fa3180f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:32.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.201 7fa31f376700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3181b0c70 con 0x7fa3181b0870 2021-02-10T11:48:32.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.201 7fa31f376700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3180f0b50 con 0x7fa3181b0470 2021-02-10T11:48:32.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.201 7fa31f376700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3180e5ea0 con 0x7fa3180f05f0 2021-02-10T11:48:32.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.201 7fa31c991700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa3181b0870 0x7fa3180efe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35532/0 (socket says 172.21.2.101:35532) 2021-02-10T11:48:32.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.201 7fa31c991700 1 -- 172.21.2.101:0/3435421164 learned_addr learned my addr 172.21.2.101:0/3435421164 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:32.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.202 7fa31d993700 1 --2- 172.21.2.101:0/3435421164 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3180f05f0 0x7fa3180f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:32.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.202 7fa30ffff700 1 -- 172.21.2.101:0/3435421164 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3356759201 0 0) 0x7fa308000ce0 con 0x7fa3181b0870 2021-02-10T11:48:32.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.202 7fa30ffff700 1 -- 172.21.2.101:0/3435421164 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa304002fe0 con 0x7fa3181b0870 2021-02-10T11:48:32.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.202 7fa30ffff700 1 -- 172.21.2.101:0/3435421164 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1728894569 0 0) 0x7fa314000c60 con 0x7fa3181b0470 2021-02-10T11:48:32.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.202 7fa30ffff700 1 -- 172.21.2.101:0/3435421164 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa304003210 con 0x7fa3181b0470 2021-02-10T11:48:32.215 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.202 7fa31d993700 1 -- 172.21.2.101:0/3435421164 >> v1:172.21.2.108:6789/0 conn(0x7fa3181b0470 legacy=0x7fa3180ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:32.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.202 7fa31d993700 1 -- 172.21.2.101:0/3435421164 >> v1:172.21.2.101:6789/0 conn(0x7fa3181b0870 legacy=0x7fa3180efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:32.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.202 7fa30ffff700 1 -- 172.21.2.101:0/3435421164 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2694500229 0 0) 0x7fa308001140 con 0x7fa3181b0870 2021-02-10T11:48:32.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.202 7fa31d993700 1 -- 172.21.2.101:0/3435421164 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3180e52e0 con 0x7fa3180f05f0 2021-02-10T11:48:32.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.202 7fa31d993700 1 --2- 172.21.2.101:0/3435421164 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3180f05f0 0x7fa3180f20e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fa310006e50 tx=0x7fa310009e70).ready entity=mon.2 client_cookie=4002ab645e7aff29 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:32.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.203 7fa30ffff700 1 -- 172.21.2.101:0/3435421164 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa310006a20 con 0x7fa3180f05f0 2021-02-10T11:48:32.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.203 7fa30ffff700 1 -- 172.21.2.101:0/3435421164 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa310006790 con 0x7fa3180f05f0 2021-02-10T11:48:32.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.203 7fa30ffff700 1 -- 172.21.2.101:0/3435421164 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa31000b700 con 0x7fa3180f05f0 2021-02-10T11:48:32.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.203 7fa31f376700 1 -- 172.21.2.101:0/3435421164 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3180f05f0 msgr2=0x7fa3180f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:32.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.203 7fa31f376700 1 --2- 172.21.2.101:0/3435421164 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3180f05f0 0x7fa3180f20e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fa310006e50 tx=0x7fa310009e70).stop 2021-02-10T11:48:32.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.203 7fa31f376700 1 -- 172.21.2.101:0/3435421164 shutdown_connections 2021-02-10T11:48:32.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.203 7fa31f376700 1 --2- 172.21.2.101:0/3435421164 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa3180f05f0 0x7fa3180f20e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:32.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.203 7fa31f376700 1 -- 172.21.2.101:0/3435421164 >> 172.21.2.101:0/3435421164 conn(0x7fa3180ce3e0 msgr2=0x7fa3180e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:32.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.204 7fa31f376700 1 -- 172.21.2.101:0/3435421164 shutdown_connections 2021-02-10T11:48:32.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.204 7fa31f376700 1 -- 172.21.2.101:0/3435421164 wait complete. 2021-02-10T11:48:32.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.218 7fa31f376700 1 Processor -- start 2021-02-10T11:48:32.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.218 7fa31f376700 1 -- start start 2021-02-10T11:48:32.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.219 7fa31f376700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa318194960 0x7fa3181b97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:32.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.219 7fa31f376700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa318106c20 0x7fa318107020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:32.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.219 7fa31f376700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3181953c0 0x7fa3181957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:32.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.219 7fa31f376700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa318107860 con 0x7fa318106c20 2021-02-10T11:48:32.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.219 7fa31f376700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa318194e80 con 0x7fa3181953c0 2021-02-10T11:48:32.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.219 7fa31f376700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3181b9dd0 con 0x7fa318194960 2021-02-10T11:48:32.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.219 7fa31c991700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa318106c20 0x7fa318107020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:32.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.219 7fa31c991700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa318106c20 0x7fa318107020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35046/0 (socket says 172.21.2.101:35046) 2021-02-10T11:48:32.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.219 7fa31c991700 1 -- 172.21.2.101:0/2551634798 learned_addr learned my addr 172.21.2.101:0/2551634798 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:32.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.219 7fa31d192700 1 --2- 172.21.2.101:0/2551634798 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa318194960 0x7fa3181b97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:32.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.219 7fa31d993700 1 --2- 172.21.2.101:0/2551634798 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3181953c0 0x7fa3181957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:32.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.220 7fa31c991700 1 -- 172.21.2.101:0/2551634798 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa318194960 msgr2=0x7fa3181b97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:32.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.220 7fa31c991700 1 --2- 172.21.2.101:0/2551634798 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa318194960 0x7fa3181b97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:32.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.220 7fa31c991700 1 -- 172.21.2.101:0/2551634798 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3181953c0 msgr2=0x7fa3181957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:32.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.220 7fa31c991700 1 --2- 172.21.2.101:0/2551634798 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3181953c0 0x7fa3181957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:32.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.220 7fa31c991700 1 -- 172.21.2.101:0/2551634798 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa318195de0 con 0x7fa318106c20 2021-02-10T11:48:32.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.220 7fa31d192700 1 --2- 172.21.2.101:0/2551634798 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa318194960 0x7fa3181b97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:48:32.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.220 7fa31c991700 1 --2- 172.21.2.101:0/2551634798 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa318106c20 0x7fa318107020 secure :-1 s=READY pgs=3259 cs=0 l=1 rev1=1 rx=0x7fa30800a570 tx=0x7fa30800a910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:32.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.220 7fa30dffb700 1 -- 172.21.2.101:0/2551634798 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa30800ad00 con 0x7fa318106c20 2021-02-10T11:48:32.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.220 7fa30dffb700 1 -- 172.21.2.101:0/2551634798 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa308001100 con 0x7fa318106c20 2021-02-10T11:48:32.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.220 7fa31f376700 1 -- 172.21.2.101:0/2551634798 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa318195de0 con 0x7fa318106c20 2021-02-10T11:48:32.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.220 7fa31f376700 1 -- 172.21.2.101:0/2551634798 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa318196370 con 0x7fa318106c20 2021-02-10T11:48:32.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.220 7fa30dffb700 1 -- 172.21.2.101:0/2551634798 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa3080047a0 con 0x7fa318106c20 2021-02-10T11:48:32.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.221 7fa30dffb700 1 -- 172.21.2.101:0/2551634798 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa3080043a0 con 0x7fa318106c20 2021-02-10T11:48:32.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.221 7fa30dffb700 1 --2- 172.21.2.101:0/2551634798 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2f80186c0 0x7fa2f801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:32.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.221 7fa30dffb700 1 -- 172.21.2.101:0/2551634798 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa308032570 con 0x7fa318106c20 2021-02-10T11:48:32.220 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.221 7fa31d192700 1 --2- 172.21.2.101:0/2551634798 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2f80186c0 0x7fa2f801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:32.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.221 7fa31d192700 1 --2- 172.21.2.101:0/2551634798 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2f80186c0 0x7fa2f801abc0 crc :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:32.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.251 7fa31f376700 1 -- 172.21.2.101:0/2551634798 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa318105a20 con 0x7fa318106c20 2021-02-10T11:48:32.250 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.254 7fa30dffb700 1 -- 172.21.2.101:0/2551634798 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa308030d20 con 0x7fa318106c20 2021-02-10T11:48:32.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.378 7fa31f376700 1 -- 172.21.2.101:0/2551634798 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3180f0b50 con 0x7fa318106c20 2021-02-10T11:48:32.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.378 7fa30dffb700 1 -- 172.21.2.101:0/2551634798 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) v1 ==== 74+0+14007 (secure 0 0 0) 0x7fa3080043a0 con 0x7fa318106c20 2021-02-10T11:48:32.375 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:48:32.375 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":191,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:48:25.826144","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":41,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":40,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:48:22.784885","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":41,"pool_name":"cephfs_data","create_time":"2021-02-10 11:48:23.812792","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"191","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:48:32.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.380 7fa31f376700 1 -- 172.21.2.101:0/2551634798 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2f80186c0 msgr2=0x7fa2f801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:32.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.380 7fa31f376700 1 --2- 172.21.2.101:0/2551634798 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2f80186c0 0x7fa2f801abc0 crc :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:32.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.380 7fa31f376700 1 -- 172.21.2.101:0/2551634798 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa318106c20 msgr2=0x7fa318107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:32.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.380 7fa31f376700 1 --2- 172.21.2.101:0/2551634798 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa318106c20 0x7fa318107020 secure :-1 s=READY pgs=3259 cs=0 l=1 rev1=1 rx=0x7fa30800a570 tx=0x7fa30800a910).stop 2021-02-10T11:48:32.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.380 7fa31f376700 1 -- 172.21.2.101:0/2551634798 shutdown_connections 2021-02-10T11:48:32.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.380 7fa31f376700 1 --2- 172.21.2.101:0/2551634798 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa3181953c0 0x7fa3181957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:32.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.380 7fa31f376700 1 --2- 172.21.2.101:0/2551634798 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa318106c20 0x7fa318107020 unknown :-1 s=CLOSED pgs=3259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:32.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.380 7fa31f376700 1 --2- 172.21.2.101:0/2551634798 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa2f80186c0 0x7fa2f801abc0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:32.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.380 7fa31f376700 1 --2- 172.21.2.101:0/2551634798 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa318194960 0x7fa3181b97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:32.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.380 7fa31f376700 1 -- 172.21.2.101:0/2551634798 >> 172.21.2.101:0/2551634798 conn(0x7fa3181a04a0 msgr2=0x7fa3180916f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:32.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.380 7fa31f376700 1 -- 172.21.2.101:0/2551634798 shutdown_connections 2021-02-10T11:48:32.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.380 7fa31f376700 1 -- 172.21.2.101:0/2551634798 wait complete. 2021-02-10T11:48:32.385 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 40, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:48:22.784885', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '190', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 41, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:48:23.812792', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '191', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:48:32.386 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:48:32.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.806 7fe9e7a50700 1 Processor -- start 2021-02-10T11:48:32.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.806 7fe9e7a50700 1 -- start start 2021-02-10T11:48:32.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.807 7fe9e7a50700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9e00d1870 0x7fe9e01b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:32.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.807 7fe9e7a50700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe9e01b0ae0 0x7fe9e0108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:32.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.807 7fe9e7a50700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9e0109a60 con 0x7fe9e00d1c70 2021-02-10T11:48:32.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.807 7fe9e7a50700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e00e5ef0 con 0x7fe9e01b0ae0 2021-02-10T11:48:32.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.807 7fe9e7a50700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e0109750 con 0x7fe9e00d1870 2021-02-10T11:48:32.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.807 7fe9e606d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe9e00d1c70 0x7fe9e0108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35546/0 (socket says 172.21.2.101:35546) 2021-02-10T11:48:32.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.807 7fe9e606d700 1 -- 172.21.2.101:0/2647941135 learned_addr learned my addr 172.21.2.101:0/2647941135 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:32.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.807 7fe9e486a700 1 -- 172.21.2.101:0/2647941135 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 790984364 0 0) 0x7fe9d8001500 con 0x7fe9e00d1c70 2021-02-10T11:48:32.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.807 7fe9e486a700 1 -- 172.21.2.101:0/2647941135 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9cc002fe0 con 0x7fe9e00d1c70 2021-02-10T11:48:32.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e486a700 1 -- 172.21.2.101:0/2647941135 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3471043815 0 0) 0x7fe9d8001500 con 0x7fe9e00d1c70 2021-02-10T11:48:32.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e486a700 1 -- 172.21.2.101:0/2647941135 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9e00d1870 msgr2=0x7fe9e01b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:32.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e486a700 1 --2- 172.21.2.101:0/2647941135 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9e00d1870 0x7fe9e01b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:32.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e486a700 1 -- 172.21.2.101:0/2647941135 >> v2:172.21.2.108:3300/0 conn(0x7fe9e01b0ae0 msgr2=0x7fe9e0108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:32.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e486a700 1 --2- 172.21.2.101:0/2647941135 >> v2:172.21.2.108:3300/0 conn(0x7fe9e01b0ae0 0x7fe9e0108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:32.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e486a700 1 -- 172.21.2.101:0/2647941135 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e0088070 con 0x7fe9e00d1c70 2021-02-10T11:48:32.821 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e486a700 1 -- 172.21.2.101:0/2647941135 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fe9d8001a00 con 0x7fe9e00d1c70 2021-02-10T11:48:32.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e486a700 1 -- 172.21.2.101:0/2647941135 >> v1:172.21.2.101:6789/0 conn(0x7fe9e00d1c70 legacy=0x7fe9e0108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:32.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e486a700 1 --2- 172.21.2.101:0/2647941135 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe9cc003a60 0x7fe9cc005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:32.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e486a700 1 --2- 172.21.2.101:0/2647941135 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9cc006790 0x7fe9cc008c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:32.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e486a700 1 --2- 172.21.2.101:0/2647941135 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9cc009500 0x7fe9cc00ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:32.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e486a700 1 -- 172.21.2.101:0/2647941135 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc00c2d0 con 0x7fe9cc009500 2021-02-10T11:48:32.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e486a700 1 -- 172.21.2.101:0/2647941135 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc00c4f0 con 0x7fe9cc003a60 2021-02-10T11:48:32.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e486a700 1 -- 172.21.2.101:0/2647941135 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc00c710 con 0x7fe9cc006790 2021-02-10T11:48:32.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e606d700 1 --2- 172.21.2.101:0/2647941135 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9cc009500 0x7fe9cc00ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:32.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e586c700 1 --2- 172.21.2.101:0/2647941135 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe9cc003a60 0x7fe9cc005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:32.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e606d700 1 -- 172.21.2.101:0/2647941135 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9cc006790 msgr2=0x7fe9cc008c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:32.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e506b700 1 --2- 172.21.2.101:0/2647941135 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9cc006790 0x7fe9cc008c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:32.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e606d700 1 --2- 172.21.2.101:0/2647941135 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9cc006790 0x7fe9cc008c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:32.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e606d700 1 -- 172.21.2.101:0/2647941135 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe9cc003a60 msgr2=0x7fe9cc005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:32.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e606d700 1 --2- 172.21.2.101:0/2647941135 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe9cc003a60 0x7fe9cc005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:32.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e606d700 1 -- 172.21.2.101:0/2647941135 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc00c9b0 con 0x7fe9cc009500 2021-02-10T11:48:32.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e506b700 1 --2- 172.21.2.101:0/2647941135 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9cc006790 0x7fe9cc008c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:48:32.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e606d700 1 --2- 172.21.2.101:0/2647941135 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9cc009500 0x7fe9cc00ba20 secure :-1 s=READY pgs=3261 cs=0 l=1 rev1=1 rx=0x7fe9d8007f10 tx=0x7fe9d804e320).ready entity=mon.0 client_cookie=4f5059fc0b474ecc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:32.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e486a700 1 -- 172.21.2.101:0/2647941135 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe9d8007890 con 0x7fe9cc009500 2021-02-10T11:48:32.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e486a700 1 -- 172.21.2.101:0/2647941135 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d80070c0 con 0x7fe9cc009500 2021-02-10T11:48:32.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e486a700 1 -- 172.21.2.101:0/2647941135 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe9d8007550 con 0x7fe9cc009500 2021-02-10T11:48:32.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e7a50700 1 -- 172.21.2.101:0/2647941135 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9cc009500 msgr2=0x7fe9cc00ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:32.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e7a50700 1 --2- 172.21.2.101:0/2647941135 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9cc009500 0x7fe9cc00ba20 secure :-1 s=READY pgs=3261 cs=0 l=1 rev1=1 rx=0x7fe9d8007f10 tx=0x7fe9d804e320).stop 2021-02-10T11:48:32.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.808 7fe9e606d700 1 -- 172.21.2.101:0/2647941135 reap_dead start 2021-02-10T11:48:32.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.809 7fe9e7a50700 1 -- 172.21.2.101:0/2647941135 shutdown_connections 2021-02-10T11:48:32.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.809 7fe9e7a50700 1 -- 172.21.2.101:0/2647941135 >> 172.21.2.101:0/2647941135 conn(0x7fe9e00ce420 msgr2=0x7fe9e00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:32.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.809 7fe9e7a50700 1 -- 172.21.2.101:0/2647941135 shutdown_connections 2021-02-10T11:48:32.824 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.809 7fe9e7a50700 1 -- 172.21.2.101:0/2647941135 wait complete. 2021-02-10T11:48:32.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.824 7fe9e7a50700 1 Processor -- start 2021-02-10T11:48:32.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.824 7fe9e7a50700 1 -- start start 2021-02-10T11:48:32.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.825 7fe9e7a50700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9e009bcc0 0x7fe9e009c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:32.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.825 7fe9e7a50700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9e00a0820 0x7fe9e00a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:32.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.825 7fe9e7a50700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe9e00a1240 0x7fe9e00858d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:32.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.825 7fe9e7a50700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e00a66b0 con 0x7fe9e009bcc0 2021-02-10T11:48:32.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.825 7fe9e7a50700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e01a49b0 con 0x7fe9e00a1240 2021-02-10T11:48:32.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.825 7fe9e7a50700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e01a4b90 con 0x7fe9e00a0820 2021-02-10T11:48:32.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.825 7fe9e586c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9e009bcc0 0x7fe9e009c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:32.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.825 7fe9e586c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9e009bcc0 0x7fe9e009c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35062/0 (socket says 172.21.2.101:35062) 2021-02-10T11:48:32.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.825 7fe9e586c700 1 -- 172.21.2.101:0/329635375 learned_addr learned my addr 172.21.2.101:0/329635375 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:32.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.825 7fe9e606d700 1 --2- 172.21.2.101:0/329635375 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe9e00a1240 0x7fe9e00858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:32.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.825 7fe9e506b700 1 --2- 172.21.2.101:0/329635375 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9e00a0820 0x7fe9e00a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:32.826 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.825 7fe9e586c700 1 -- 172.21.2.101:0/329635375 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9e00a0820 msgr2=0x7fe9e00a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:32.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.825 7fe9e586c700 1 --2- 172.21.2.101:0/329635375 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9e00a0820 0x7fe9e00a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:32.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.825 7fe9e586c700 1 -- 172.21.2.101:0/329635375 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe9e00a1240 msgr2=0x7fe9e00858d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:32.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.825 7fe9e586c700 1 --2- 172.21.2.101:0/329635375 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe9e00a1240 0x7fe9e00858d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:32.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.825 7fe9e586c700 1 -- 172.21.2.101:0/329635375 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e00a1640 con 0x7fe9e009bcc0 2021-02-10T11:48:32.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.825 7fe9e586c700 1 --2- 172.21.2.101:0/329635375 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9e009bcc0 0x7fe9e009c0c0 secure :-1 s=READY pgs=3262 cs=0 l=1 rev1=1 rx=0x7fe9dc006f00 tx=0x7fe9dc00b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:32.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.825 7fe9d27fc700 1 -- 172.21.2.101:0/329635375 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe9dc00bd30 con 0x7fe9e009bcc0 2021-02-10T11:48:32.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.825 7fe9d27fc700 1 -- 172.21.2.101:0/329635375 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9dc002030 con 0x7fe9e009bcc0 2021-02-10T11:48:32.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.825 7fe9e7a50700 1 -- 172.21.2.101:0/329635375 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e00a1640 con 0x7fe9e009bcc0 2021-02-10T11:48:32.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.825 7fe9d27fc700 1 -- 172.21.2.101:0/329635375 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe9dc002590 con 0x7fe9e009bcc0 2021-02-10T11:48:32.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.826 7fe9e7a50700 1 -- 172.21.2.101:0/329635375 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e00a1640 con 0x7fe9e009bcc0 2021-02-10T11:48:32.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.826 7fe9d27fc700 1 -- 172.21.2.101:0/329635375 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe9dc000c90 con 0x7fe9e009bcc0 2021-02-10T11:48:32.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.826 7fe9d27fc700 1 --2- 172.21.2.101:0/329635375 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe9c8018730 0x7fe9c801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:32.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.826 7fe9d27fc700 1 -- 172.21.2.101:0/329635375 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe9dc032670 con 0x7fe9e009bcc0 2021-02-10T11:48:32.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.826 7fe9e506b700 1 --2- 172.21.2.101:0/329635375 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe9c8018730 0x7fe9c801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:32.828 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.827 7fe9e506b700 1 --2- 172.21.2.101:0/329635375 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe9c8018730 0x7fe9c801ac10 crc :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:32.853 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.857 7fe9e7a50700 1 -- 172.21.2.101:0/329635375 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e00fe1c0 con 0x7fe9e009bcc0 2021-02-10T11:48:32.855 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.859 7fe9d27fc700 1 -- 172.21.2.101:0/329635375 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe9dc01f530 con 0x7fe9e009bcc0 2021-02-10T11:48:32.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.988 7fe9e7a50700 1 -- 172.21.2.101:0/329635375 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fe9e00fe1c0 con 0x7fe9e009bcc0 2021-02-10T11:48:32.984 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.988 7fe9d27fc700 1 -- 172.21.2.101:0/329635375 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v191) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe9dc000c90 con 0x7fe9e009bcc0 2021-02-10T11:48:32.985 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:48:32.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.990 7fe9e7a50700 1 -- 172.21.2.101:0/329635375 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe9c8018730 msgr2=0x7fe9c801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:32.986 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.990 7fe9e7a50700 1 --2- 172.21.2.101:0/329635375 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe9c8018730 0x7fe9c801ac10 crc :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:32.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.990 7fe9e7a50700 1 -- 172.21.2.101:0/329635375 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9e009bcc0 msgr2=0x7fe9e009c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:32.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.990 7fe9e7a50700 1 --2- 172.21.2.101:0/329635375 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9e009bcc0 0x7fe9e009c0c0 secure :-1 s=READY pgs=3262 cs=0 l=1 rev1=1 rx=0x7fe9dc006f00 tx=0x7fe9dc00b940).stop 2021-02-10T11:48:32.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.990 7fe9e7a50700 1 -- 172.21.2.101:0/329635375 shutdown_connections 2021-02-10T11:48:32.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.990 7fe9e7a50700 1 --2- 172.21.2.101:0/329635375 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe9c8018730 0x7fe9c801ac10 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:32.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.990 7fe9e7a50700 1 --2- 172.21.2.101:0/329635375 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9e00a0820 0x7fe9e00a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:32.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.990 7fe9e7a50700 1 --2- 172.21.2.101:0/329635375 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe9e00a1240 0x7fe9e00858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:32.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.990 7fe9e7a50700 1 --2- 172.21.2.101:0/329635375 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9e009bcc0 0x7fe9e009c0c0 unknown :-1 s=CLOSED pgs=3262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:32.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.990 7fe9e7a50700 1 -- 172.21.2.101:0/329635375 >> 172.21.2.101:0/329635375 conn(0x7fe9e00bef00 msgr2=0x7fe9e009d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:32.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.990 7fe9e7a50700 1 -- 172.21.2.101:0/329635375 shutdown_connections 2021-02-10T11:48:32.987 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:32.990 7fe9e7a50700 1 -- 172.21.2.101:0/329635375 wait complete. 2021-02-10T11:48:32.995 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:48:33.428 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.415 7faab45d2700 1 Processor -- start 2021-02-10T11:48:33.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.415 7faab45d2700 1 -- start start 2021-02-10T11:48:33.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.415 7faab45d2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faaac0d1870 0x7faaac1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:33.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.415 7faab45d2700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaac109a60 con 0x7faaac0cff70 2021-02-10T11:48:33.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.415 7faab45d2700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaac088070 con 0x7faaac1b0ae0 2021-02-10T11:48:33.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.415 7faab45d2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaac0e5ef0 con 0x7faaac0d1870 2021-02-10T11:48:33.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.415 7faab2bef700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7faaac0cff70 0x7faaac109130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35564/0 (socket says 172.21.2.101:35564) 2021-02-10T11:48:33.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.415 7faab2bef700 1 -- 172.21.2.101:0/3623149093 learned_addr learned my addr 172.21.2.101:0/3623149093 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:33.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab23ee700 1 --2- 172.21.2.101:0/3623149093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faaac0d1870 0x7faaac1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:33.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab13ec700 1 -- 172.21.2.101:0/3623149093 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 421420595 0 0) 0x7faaa4001500 con 0x7faaac0cff70 2021-02-10T11:48:33.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab13ec700 1 -- 172.21.2.101:0/3623149093 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa98002fe0 con 0x7faaac0cff70 2021-02-10T11:48:33.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab13ec700 1 -- 172.21.2.101:0/3623149093 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3206914836 0 0) 0x7faaa4001460 con 0x7faaac0cff70 2021-02-10T11:48:33.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab13ec700 1 -- 172.21.2.101:0/3623149093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faaac0d1870 msgr2=0x7faaac1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:33.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab13ec700 1 --2- 172.21.2.101:0/3623149093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faaac0d1870 0x7faaac1b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:33.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab13ec700 1 -- 172.21.2.101:0/3623149093 >> v1:172.21.2.108:6789/0 conn(0x7faaac1b0ae0 legacy=0x7faaac108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:33.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab13ec700 1 -- 172.21.2.101:0/3623149093 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaac0d1c70 con 0x7faaac0cff70 2021-02-10T11:48:33.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab13ec700 1 -- 172.21.2.101:0/3623149093 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7faaa4001980 con 0x7faaac0cff70 2021-02-10T11:48:33.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab13ec700 1 -- 172.21.2.101:0/3623149093 >> v1:172.21.2.101:6789/0 conn(0x7faaac0cff70 legacy=0x7faaac109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:33.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab13ec700 1 --2- 172.21.2.101:0/3623149093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faa98003a60 0x7faa98005f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:33.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab13ec700 1 --2- 172.21.2.101:0/3623149093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faa98006750 0x7faa98008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:33.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab13ec700 1 --2- 172.21.2.101:0/3623149093 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faa980094f0 0x7faa9800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:33.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab13ec700 1 -- 172.21.2.101:0/3623149093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9800c2c0 con 0x7faa98006750 2021-02-10T11:48:33.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab13ec700 1 -- 172.21.2.101:0/3623149093 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9800c4e0 con 0x7faa980094f0 2021-02-10T11:48:33.430 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab13ec700 1 -- 172.21.2.101:0/3623149093 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa9800c700 con 0x7faa98003a60 2021-02-10T11:48:33.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab1bed700 1 --2- 172.21.2.101:0/3623149093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faa98006750 0x7faa98008c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:33.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab1bed700 1 -- 172.21.2.101:0/3623149093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faa98003a60 msgr2=0x7faa98005f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:33.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab1bed700 1 --2- 172.21.2.101:0/3623149093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faa98003a60 0x7faa98005f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:33.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab1bed700 1 -- 172.21.2.101:0/3623149093 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faa980094f0 msgr2=0x7faa9800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:33.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab1bed700 1 --2- 172.21.2.101:0/3623149093 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faa980094f0 0x7faa9800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:33.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab1bed700 1 -- 172.21.2.101:0/3623149093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa98050970 con 0x7faa98006750 2021-02-10T11:48:33.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.416 7faab1bed700 1 --2- 172.21.2.101:0/3623149093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faa98006750 0x7faa98008c70 secure :-1 s=READY pgs=3264 cs=0 l=1 rev1=1 rx=0x7faaa8006ac0 tx=0x7faaa8009af0).ready entity=mon.0 client_cookie=fff8efa8609c58e8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:33.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.417 7faab13ec700 1 -- 172.21.2.101:0/3623149093 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faaa8007280 con 0x7faa98006750 2021-02-10T11:48:33.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.417 7faab13ec700 1 -- 172.21.2.101:0/3623149093 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa8007580 con 0x7faa98006750 2021-02-10T11:48:33.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.417 7faab13ec700 1 -- 172.21.2.101:0/3623149093 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faaa80079d0 con 0x7faa98006750 2021-02-10T11:48:33.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.417 7faab45d2700 1 -- 172.21.2.101:0/3623149093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faa98006750 msgr2=0x7faa98008c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:33.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.417 7faab45d2700 1 --2- 172.21.2.101:0/3623149093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faa98006750 0x7faa98008c70 secure :-1 s=READY pgs=3264 cs=0 l=1 rev1=1 rx=0x7faaa8006ac0 tx=0x7faaa8009af0).stop 2021-02-10T11:48:33.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.417 7faab2bef700 1 -- 172.21.2.101:0/3623149093 reap_dead start 2021-02-10T11:48:33.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.417 7faab45d2700 1 -- 172.21.2.101:0/3623149093 shutdown_connections 2021-02-10T11:48:33.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.417 7faab45d2700 1 -- 172.21.2.101:0/3623149093 >> 172.21.2.101:0/3623149093 conn(0x7faaac0ce420 msgr2=0x7faaac0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:33.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.417 7faab45d2700 1 -- 172.21.2.101:0/3623149093 shutdown_connections 2021-02-10T11:48:33.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.417 7faab45d2700 1 -- 172.21.2.101:0/3623149093 wait complete. 2021-02-10T11:48:33.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.432 7faab45d2700 1 Processor -- start 2021-02-10T11:48:33.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.432 7faab45d2700 1 -- start start 2021-02-10T11:48:33.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.433 7faab45d2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faaac0a7950 0x7faaac091820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:33.433 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.433 7faab45d2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faaac0a7d50 0x7faaac091e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:33.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.433 7faab45d2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faaac18e980 0x7faaac18ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:33.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.433 7faab45d2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac091320 con 0x7faaac0a7950 2021-02-10T11:48:33.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.433 7faab45d2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac0ab8a0 con 0x7faaac18e980 2021-02-10T11:48:33.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.433 7faab45d2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaac18f3a0 con 0x7faaac0a7d50 2021-02-10T11:48:33.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.433 7faab23ee700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faaac0a7950 0x7faaac091820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:33.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.433 7faab23ee700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faaac0a7950 0x7faaac091820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35084/0 (socket says 172.21.2.101:35084) 2021-02-10T11:48:33.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.433 7faab23ee700 1 -- 172.21.2.101:0/1523947907 learned_addr learned my addr 172.21.2.101:0/1523947907 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:33.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.433 7faab23ee700 1 -- 172.21.2.101:0/1523947907 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faaac0a7d50 msgr2=0x7faaac091e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:33.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.433 7faab23ee700 1 --2- 172.21.2.101:0/1523947907 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faaac0a7d50 0x7faaac091e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:33.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.433 7faab23ee700 1 -- 172.21.2.101:0/1523947907 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faaac18e980 msgr2=0x7faaac18ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:33.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.433 7faab23ee700 1 --2- 172.21.2.101:0/1523947907 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faaac18e980 0x7faaac18ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:33.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.433 7faab23ee700 1 -- 172.21.2.101:0/1523947907 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaac18f5c0 con 0x7faaac0a7950 2021-02-10T11:48:33.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.433 7faab23ee700 1 --2- 172.21.2.101:0/1523947907 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faaac0a7950 0x7faaac091820 secure :-1 s=READY pgs=3265 cs=0 l=1 rev1=1 rx=0x7faaa000b5e0 tx=0x7faaa000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:33.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.434 7faa9effd700 1 -- 172.21.2.101:0/1523947907 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faaa000bd70 con 0x7faaac0a7950 2021-02-10T11:48:33.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.434 7faa9effd700 1 -- 172.21.2.101:0/1523947907 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa0006030 con 0x7faaac0a7950 2021-02-10T11:48:33.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.434 7faab45d2700 1 -- 172.21.2.101:0/1523947907 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaac18f5c0 con 0x7faaac0a7950 2021-02-10T11:48:33.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.434 7faab45d2700 1 -- 172.21.2.101:0/1523947907 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaac18fb50 con 0x7faaac0a7950 2021-02-10T11:48:33.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.434 7faa9effd700 1 -- 172.21.2.101:0/1523947907 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faaa00052e0 con 0x7faaac0a7950 2021-02-10T11:48:33.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.434 7faa9effd700 1 -- 172.21.2.101:0/1523947907 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7faaa0006210 con 0x7faaac0a7950 2021-02-10T11:48:33.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.434 7faa9effd700 1 --2- 172.21.2.101:0/1523947907 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faa94018730 0x7faa9401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:33.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.435 7faa9effd700 1 -- 172.21.2.101:0/1523947907 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7faaa00327b0 con 0x7faaac0a7950 2021-02-10T11:48:33.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.435 7faab1bed700 1 --2- 172.21.2.101:0/1523947907 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faa94018730 0x7faa9401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:33.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.435 7faab1bed700 1 --2- 172.21.2.101:0/1523947907 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faa94018730 0x7faa9401ac10 crc :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:33.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.465 7faab45d2700 1 -- 172.21.2.101:0/1523947907 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaac082a70 con 0x7faaac0a7950 2021-02-10T11:48:33.464 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.468 7faa9effd700 1 -- 172.21.2.101:0/1523947907 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7faaa0000960 con 0x7faaac0a7950 2021-02-10T11:48:33.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.597 7faab45d2700 1 -- 172.21.2.101:0/1523947907 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7faaac082a70 con 0x7faaac0a7950 2021-02-10T11:48:33.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.598 7faa9effd700 1 -- 172.21.2.101:0/1523947907 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v191) v1 ==== 104+0+10 (secure 0 0 0) 0x7faaa000bd70 con 0x7faaac0a7950 2021-02-10T11:48:33.595 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:48:33.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.600 7faab45d2700 1 -- 172.21.2.101:0/1523947907 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faa94018730 msgr2=0x7faa9401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:33.596 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.600 7faab45d2700 1 --2- 172.21.2.101:0/1523947907 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faa94018730 0x7faa9401ac10 crc :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:33.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.600 7faab45d2700 1 -- 172.21.2.101:0/1523947907 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faaac0a7950 msgr2=0x7faaac091820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:33.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.600 7faab45d2700 1 --2- 172.21.2.101:0/1523947907 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faaac0a7950 0x7faaac091820 secure :-1 s=READY pgs=3265 cs=0 l=1 rev1=1 rx=0x7faaa000b5e0 tx=0x7faaa000b980).stop 2021-02-10T11:48:33.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.600 7faab45d2700 1 -- 172.21.2.101:0/1523947907 shutdown_connections 2021-02-10T11:48:33.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.600 7faab45d2700 1 --2- 172.21.2.101:0/1523947907 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7faa94018730 0x7faa9401ac10 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:33.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.600 7faab45d2700 1 --2- 172.21.2.101:0/1523947907 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7faaac0a7d50 0x7faaac091e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:33.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.600 7faab45d2700 1 --2- 172.21.2.101:0/1523947907 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7faaac18e980 0x7faaac18ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:33.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.600 7faab45d2700 1 --2- 172.21.2.101:0/1523947907 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7faaac0a7950 0x7faaac091820 unknown :-1 s=CLOSED pgs=3265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:33.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.600 7faab45d2700 1 -- 172.21.2.101:0/1523947907 >> 172.21.2.101:0/1523947907 conn(0x7faaac087480 msgr2=0x7faaac0c0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:33.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.600 7faab45d2700 1 -- 172.21.2.101:0/1523947907 shutdown_connections 2021-02-10T11:48:33.597 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:33.600 7faab45d2700 1 -- 172.21.2.101:0/1523947907 wait complete. 2021-02-10T11:48:33.605 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:48:34.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.024 7f8ab5b48700 1 Processor -- start 2021-02-10T11:48:34.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.025 7f8ab5b48700 1 -- start start 2021-02-10T11:48:34.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.025 7f8ab5b48700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f8ab010aff0 0x7f8ab00cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:34.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.025 7f8ab5b48700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab00fe1c0 con 0x7f8ab00d1c70 2021-02-10T11:48:34.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.025 7f8ab5b48700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab01b0cb0 con 0x7f8ab00d1870 2021-02-10T11:48:34.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.025 7f8ab5b48700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab0109a60 con 0x7f8ab010aff0 2021-02-10T11:48:34.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.025 7f8aaeffd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f8ab00d1c70 0x7f8ab00cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35582/0 (socket says 172.21.2.101:35582) 2021-02-10T11:48:34.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.025 7f8aaeffd700 1 -- 172.21.2.101:0/1139203448 learned_addr learned my addr 172.21.2.101:0/1139203448 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:34.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aae7fc700 1 -- 172.21.2.101:0/1139203448 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141446878 0 0) 0x7f8aa00010b0 con 0x7f8ab00d1c70 2021-02-10T11:48:34.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aae7fc700 1 -- 172.21.2.101:0/1139203448 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a9c002fe0 con 0x7f8ab00d1c70 2021-02-10T11:48:34.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aae7fc700 1 -- 172.21.2.101:0/1139203448 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1314991087 0 0) 0x7f8aa00010b0 con 0x7f8ab00d1c70 2021-02-10T11:48:34.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aae7fc700 1 -- 172.21.2.101:0/1139203448 >> v2:172.21.2.108:3300/0 conn(0x7f8ab010aff0 msgr2=0x7f8ab00cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:34.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aae7fc700 1 --2- 172.21.2.101:0/1139203448 >> v2:172.21.2.108:3300/0 conn(0x7f8ab010aff0 0x7f8ab00cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:34.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aae7fc700 1 -- 172.21.2.101:0/1139203448 >> v1:172.21.2.108:6789/0 conn(0x7f8ab00d1870 legacy=0x7f8ab01b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:34.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aae7fc700 1 -- 172.21.2.101:0/1139203448 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab010b3f0 con 0x7f8ab00d1c70 2021-02-10T11:48:34.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aae7fc700 1 -- 172.21.2.101:0/1139203448 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f8aa00015b0 con 0x7f8ab00d1c70 2021-02-10T11:48:34.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aae7fc700 1 -- 172.21.2.101:0/1139203448 >> v1:172.21.2.101:6789/0 conn(0x7f8ab00d1c70 legacy=0x7f8ab00cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:34.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aae7fc700 1 --2- 172.21.2.101:0/1139203448 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a9c003aa0 0x7f8a9c005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:34.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aae7fc700 1 --2- 172.21.2.101:0/1139203448 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a9c006780 0x7f8a9c008ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:34.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aae7fc700 1 --2- 172.21.2.101:0/1139203448 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a9c0095a0 0x7f8a9c00baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:34.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aae7fc700 1 -- 172.21.2.101:0/1139203448 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c00c2f0 con 0x7f8a9c003aa0 2021-02-10T11:48:34.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aae7fc700 1 -- 172.21.2.101:0/1139203448 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c00c510 con 0x7f8a9c0095a0 2021-02-10T11:48:34.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aae7fc700 1 -- 172.21.2.101:0/1139203448 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c00c730 con 0x7f8a9c006780 2021-02-10T11:48:34.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aaf7fe700 1 --2- 172.21.2.101:0/1139203448 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a9c003aa0 0x7f8a9c005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:34.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aaeffd700 1 --2- 172.21.2.101:0/1139203448 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a9c006780 0x7f8a9c008ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:34.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aaffff700 1 --2- 172.21.2.101:0/1139203448 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a9c0095a0 0x7f8a9c00baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:34.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aaf7fe700 1 -- 172.21.2.101:0/1139203448 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a9c006780 msgr2=0x7f8a9c008ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:34.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aaf7fe700 1 --2- 172.21.2.101:0/1139203448 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8a9c006780 0x7f8a9c008ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:34.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aaf7fe700 1 -- 172.21.2.101:0/1139203448 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a9c0095a0 msgr2=0x7f8a9c00baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:34.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aaf7fe700 1 --2- 172.21.2.101:0/1139203448 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8a9c0095a0 0x7f8a9c00baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:34.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aaffff700 1 -- 172.21.2.101:0/1139203448 reap_dead start 2021-02-10T11:48:34.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aaf7fe700 1 -- 172.21.2.101:0/1139203448 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9c00c9d0 con 0x7f8a9c003aa0 2021-02-10T11:48:34.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aaf7fe700 1 --2- 172.21.2.101:0/1139203448 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a9c003aa0 0x7f8a9c005f60 secure :-1 s=READY pgs=3267 cs=0 l=1 rev1=1 rx=0x7f8a98006ab0 tx=0x7f8a9804fb10).ready entity=mon.0 client_cookie=a8f2b0160644e2d9 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:34.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aae7fc700 1 -- 172.21.2.101:0/1139203448 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8a98006240 con 0x7f8a9c003aa0 2021-02-10T11:48:34.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aae7fc700 1 -- 172.21.2.101:0/1139203448 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a980066a0 con 0x7f8a9c003aa0 2021-02-10T11:48:34.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.026 7f8aae7fc700 1 -- 172.21.2.101:0/1139203448 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8a98005690 con 0x7f8a9c003aa0 2021-02-10T11:48:34.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.027 7f8ab5b48700 1 -- 172.21.2.101:0/1139203448 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a9c003aa0 msgr2=0x7f8a9c005f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:34.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.027 7f8ab5b48700 1 --2- 172.21.2.101:0/1139203448 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a9c003aa0 0x7f8a9c005f60 secure :-1 s=READY pgs=3267 cs=0 l=1 rev1=1 rx=0x7f8a98006ab0 tx=0x7f8a9804fb10).stop 2021-02-10T11:48:34.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.027 7f8ab5b48700 1 -- 172.21.2.101:0/1139203448 shutdown_connections 2021-02-10T11:48:34.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.027 7f8ab5b48700 1 --2- 172.21.2.101:0/1139203448 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8a9c003aa0 0x7f8a9c005f60 unknown :-1 s=CLOSED pgs=3267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:34.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.027 7f8ab5b48700 1 -- 172.21.2.101:0/1139203448 >> 172.21.2.101:0/1139203448 conn(0x7f8ab00ce420 msgr2=0x7f8ab00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:34.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.027 7f8ab5b48700 1 -- 172.21.2.101:0/1139203448 shutdown_connections 2021-02-10T11:48:34.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.027 7f8ab5b48700 1 -- 172.21.2.101:0/1139203448 wait complete. 2021-02-10T11:48:34.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.042 7f8ab5b48700 1 Processor -- start 2021-02-10T11:48:34.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.042 7f8ab5b48700 1 -- start start 2021-02-10T11:48:34.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.042 7f8ab5b48700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8ab009bcc0 0x7f8ab009c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:34.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.042 7f8ab5b48700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ab00a0820 0x7f8ab00a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:34.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.042 7f8ab5b48700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8ab00a1240 0x7f8ab00858d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:34.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.042 7f8ab5b48700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab00a66b0 con 0x7f8ab00a0820 2021-02-10T11:48:34.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.042 7f8ab5b48700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab01a49b0 con 0x7f8ab00a1240 2021-02-10T11:48:34.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.042 7f8ab5b48700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab01a4b90 con 0x7f8ab009bcc0 2021-02-10T11:48:34.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.042 7f8aaeffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ab00a0820 0x7f8ab00a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:34.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.043 7f8aaeffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ab00a0820 0x7f8ab00a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35100/0 (socket says 172.21.2.101:35100) 2021-02-10T11:48:34.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.043 7f8aaeffd700 1 -- 172.21.2.101:0/482339713 learned_addr learned my addr 172.21.2.101:0/482339713 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:34.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.043 7f8aaeffd700 1 -- 172.21.2.101:0/482339713 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8ab009bcc0 msgr2=0x7f8ab009c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:34.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.043 7f8aaeffd700 1 --2- 172.21.2.101:0/482339713 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8ab009bcc0 0x7f8ab009c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:34.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.043 7f8aaeffd700 1 -- 172.21.2.101:0/482339713 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8ab00a1240 msgr2=0x7f8ab00858d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:34.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.043 7f8aaeffd700 1 --2- 172.21.2.101:0/482339713 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8ab00a1240 0x7f8ab00858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:34.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.043 7f8aaeffd700 1 -- 172.21.2.101:0/482339713 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab00a1640 con 0x7f8ab00a0820 2021-02-10T11:48:34.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.043 7f8aaeffd700 1 --2- 172.21.2.101:0/482339713 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ab00a0820 0x7f8ab00a0c20 secure :-1 s=READY pgs=3268 cs=0 l=1 rev1=1 rx=0x7f8aa0001c00 tx=0x7f8aa000a8e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:34.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.043 7f8aa7fff700 1 -- 172.21.2.101:0/482339713 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8aa00009b0 con 0x7f8ab00a0820 2021-02-10T11:48:34.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.043 7f8aa7fff700 1 -- 172.21.2.101:0/482339713 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa000acd0 con 0x7f8ab00a0820 2021-02-10T11:48:34.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.043 7f8ab5b48700 1 -- 172.21.2.101:0/482339713 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab00a1640 con 0x7f8ab00a0820 2021-02-10T11:48:34.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.043 7f8aa7fff700 1 -- 172.21.2.101:0/482339713 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8aa00045d0 con 0x7f8ab00a0820 2021-02-10T11:48:34.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.043 7f8ab5b48700 1 -- 172.21.2.101:0/482339713 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab00861e0 con 0x7f8ab00a0820 2021-02-10T11:48:34.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.044 7f8aa7fff700 1 -- 172.21.2.101:0/482339713 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8aa0004140 con 0x7f8ab00a0820 2021-02-10T11:48:34.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.044 7f8aa7fff700 1 --2- 172.21.2.101:0/482339713 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a940186c0 0x7f8a9401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:34.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.044 7f8aa7fff700 1 -- 172.21.2.101:0/482339713 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8aa0032360 con 0x7f8ab00a0820 2021-02-10T11:48:34.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.044 7f8aaf7fe700 1 --2- 172.21.2.101:0/482339713 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a940186c0 0x7f8a9401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:34.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.044 7f8aaf7fe700 1 --2- 172.21.2.101:0/482339713 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a940186c0 0x7f8a9401abc0 crc :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:34.071 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.075 7f8ab5b48700 1 -- 172.21.2.101:0/482339713 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab0086a70 con 0x7f8ab00a0820 2021-02-10T11:48:34.074 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.078 7f8aa7fff700 1 -- 172.21.2.101:0/482339713 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8aa00009b0 con 0x7f8ab00a0820 2021-02-10T11:48:34.202 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.205 7f8ab5b48700 1 -- 172.21.2.101:0/482339713 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f8ab0086a70 con 0x7f8ab00a0820 2021-02-10T11:48:34.202 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.206 7f8aa7fff700 1 -- 172.21.2.101:0/482339713 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v191) v1 ==== 100+0+10 (secure 0 0 0) 0x7f8aa001f470 con 0x7f8ab00a0820 2021-02-10T11:48:34.203 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:48:34.204 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.208 7f8ab5b48700 1 -- 172.21.2.101:0/482339713 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a940186c0 msgr2=0x7f8a9401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:34.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.208 7f8ab5b48700 1 --2- 172.21.2.101:0/482339713 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a940186c0 0x7f8a9401abc0 crc :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:34.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.208 7f8ab5b48700 1 -- 172.21.2.101:0/482339713 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ab00a0820 msgr2=0x7f8ab00a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:34.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.208 7f8ab5b48700 1 --2- 172.21.2.101:0/482339713 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ab00a0820 0x7f8ab00a0c20 secure :-1 s=READY pgs=3268 cs=0 l=1 rev1=1 rx=0x7f8aa0001c00 tx=0x7f8aa000a8e0).stop 2021-02-10T11:48:34.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.208 7f8ab5b48700 1 -- 172.21.2.101:0/482339713 shutdown_connections 2021-02-10T11:48:34.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.208 7f8ab5b48700 1 --2- 172.21.2.101:0/482339713 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8ab00a1240 0x7f8ab00858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:34.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.208 7f8ab5b48700 1 --2- 172.21.2.101:0/482339713 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8ab00a0820 0x7f8ab00a0c20 unknown :-1 s=CLOSED pgs=3268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:34.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.208 7f8ab5b48700 1 --2- 172.21.2.101:0/482339713 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8a940186c0 0x7f8a9401abc0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:34.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.208 7f8ab5b48700 1 --2- 172.21.2.101:0/482339713 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8ab009bcc0 0x7f8ab009c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:34.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.208 7f8ab5b48700 1 -- 172.21.2.101:0/482339713 >> 172.21.2.101:0/482339713 conn(0x7f8ab00bef00 msgr2=0x7f8ab009d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:34.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.208 7f8ab5b48700 1 -- 172.21.2.101:0/482339713 shutdown_connections 2021-02-10T11:48:34.205 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.208 7f8ab5b48700 1 -- 172.21.2.101:0/482339713 wait complete. 2021-02-10T11:48:34.213 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:48:34.213 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:48:34.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.630 7fd127a07700 1 Processor -- start 2021-02-10T11:48:34.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.630 7fd127a07700 1 -- start start 2021-02-10T11:48:34.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.630 7fd127a07700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1200d0340 0x7fd1200ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:34.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.630 7fd127a07700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd1200d1c70 0x7fd1200efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:34.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.630 7fd127a07700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1200f0b10 con 0x7fd1200d1870 2021-02-10T11:48:34.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.630 7fd127a07700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1200e5ed0 con 0x7fd1200d1c70 2021-02-10T11:48:34.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.630 7fd127a07700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1201b0be0 con 0x7fd1200d0340 2021-02-10T11:48:34.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.630 7fd126024700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd1200d1c70 0x7fd1200efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:34.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.630 7fd126024700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fd1200d1c70 0x7fd1200efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35112/0 (socket says 172.21.2.101:35112) 2021-02-10T11:48:34.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.630 7fd126024700 1 -- 172.21.2.101:0/1952452775 learned_addr learned my addr 172.21.2.101:0/1952452775 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:34.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.631 7fd126024700 1 -- 172.21.2.101:0/1952452775 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1200d0340 msgr2=0x7fd1200ef640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:48:34.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.631 7fd126024700 1 --2- 172.21.2.101:0/1952452775 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1200d0340 0x7fd1200ef640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:34.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.631 7fd126024700 1 -- 172.21.2.101:0/1952452775 >> v1:172.21.2.108:6789/0 conn(0x7fd1200d1870 legacy=0x7fd1201b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:34.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.631 7fd126024700 1 -- 172.21.2.101:0/1952452775 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1200f20a0 con 0x7fd1200d1c70 2021-02-10T11:48:34.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.631 7fd126024700 1 --2- 172.21.2.101:0/1952452775 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1200d1c70 0x7fd1200efc60 secure :-1 s=READY pgs=3269 cs=0 l=1 rev1=1 rx=0x7fd118006f00 tx=0x7fd118009e90).ready entity=mon.0 client_cookie=500b21acd5749bb6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:34.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.631 7fd124821700 1 -- 172.21.2.101:0/1952452775 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd11800b300 con 0x7fd1200d1c70 2021-02-10T11:48:34.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.631 7fd124821700 1 -- 172.21.2.101:0/1952452775 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11800b580 con 0x7fd1200d1c70 2021-02-10T11:48:34.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.631 7fd124821700 1 -- 172.21.2.101:0/1952452775 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd11800b9d0 con 0x7fd1200d1c70 2021-02-10T11:48:34.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.631 7fd127a07700 1 -- 172.21.2.101:0/1952452775 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1200d1c70 msgr2=0x7fd1200efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:34.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.631 7fd127a07700 1 --2- 172.21.2.101:0/1952452775 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1200d1c70 0x7fd1200efc60 secure :-1 s=READY pgs=3269 cs=0 l=1 rev1=1 rx=0x7fd118006f00 tx=0x7fd118009e90).stop 2021-02-10T11:48:34.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.632 7fd127a07700 1 -- 172.21.2.101:0/1952452775 shutdown_connections 2021-02-10T11:48:34.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.632 7fd127a07700 1 --2- 172.21.2.101:0/1952452775 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd1200d0340 0x7fd1200ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:34.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.632 7fd127a07700 1 --2- 172.21.2.101:0/1952452775 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd1200d1c70 0x7fd1200efc60 unknown :-1 s=CLOSED pgs=3269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:34.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.632 7fd127a07700 1 -- 172.21.2.101:0/1952452775 >> 172.21.2.101:0/1952452775 conn(0x7fd1200ce450 msgr2=0x7fd1200d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:34.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.632 7fd127a07700 1 -- 172.21.2.101:0/1952452775 shutdown_connections 2021-02-10T11:48:34.644 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.632 7fd127a07700 1 -- 172.21.2.101:0/1952452775 wait complete. 2021-02-10T11:48:34.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.646 7fd127a07700 1 Processor -- start 2021-02-10T11:48:34.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.646 7fd127a07700 1 -- start start 2021-02-10T11:48:34.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.647 7fd127a07700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd12019ede0 0x7fd12019f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:34.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.647 7fd127a07700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1201aca40 0x7fd1201ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:34.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.647 7fd127a07700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd12019d3c0 0x7fd12019d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:34.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.647 7fd127a07700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1201ad460 con 0x7fd12019ede0 2021-02-10T11:48:34.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.647 7fd127a07700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12019fca0 con 0x7fd1201aca40 2021-02-10T11:48:34.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.647 7fd127a07700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12019dde0 con 0x7fd12019d3c0 2021-02-10T11:48:34.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.647 7fd125823700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd12019ede0 0x7fd12019f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:34.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.647 7fd125823700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd12019ede0 0x7fd12019f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35114/0 (socket says 172.21.2.101:35114) 2021-02-10T11:48:34.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.647 7fd125823700 1 -- 172.21.2.101:0/1172498609 learned_addr learned my addr 172.21.2.101:0/1172498609 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:34.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.647 7fd125823700 1 -- 172.21.2.101:0/1172498609 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd12019d3c0 msgr2=0x7fd12019d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:34.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.647 7fd125823700 1 --2- 172.21.2.101:0/1172498609 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd12019d3c0 0x7fd12019d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:34.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.647 7fd125823700 1 -- 172.21.2.101:0/1172498609 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1201aca40 msgr2=0x7fd1201ace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:34.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.647 7fd125823700 1 --2- 172.21.2.101:0/1172498609 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1201aca40 0x7fd1201ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:34.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.647 7fd125823700 1 -- 172.21.2.101:0/1172498609 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12019e000 con 0x7fd12019ede0 2021-02-10T11:48:34.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.647 7fd125823700 1 --2- 172.21.2.101:0/1172498609 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd12019ede0 0x7fd12019f1e0 secure :-1 s=READY pgs=3270 cs=0 l=1 rev1=1 rx=0x7fd11000b5b0 tx=0x7fd11000b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:34.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.648 7fd1167fc700 1 -- 172.21.2.101:0/1172498609 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd11000bd40 con 0x7fd12019ede0 2021-02-10T11:48:34.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.648 7fd1167fc700 1 -- 172.21.2.101:0/1172498609 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd110000e20 con 0x7fd12019ede0 2021-02-10T11:48:34.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.648 7fd1167fc700 1 -- 172.21.2.101:0/1172498609 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd110006210 con 0x7fd12019ede0 2021-02-10T11:48:34.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.648 7fd127a07700 1 -- 172.21.2.101:0/1172498609 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12019e000 con 0x7fd12019ede0 2021-02-10T11:48:34.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.648 7fd127a07700 1 -- 172.21.2.101:0/1172498609 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12019e000 con 0x7fd12019ede0 2021-02-10T11:48:34.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.648 7fd1167fc700 1 -- 172.21.2.101:0/1172498609 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fd110006020 con 0x7fd12019ede0 2021-02-10T11:48:34.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.648 7fd1167fc700 1 --2- 172.21.2.101:0/1172498609 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1080186c0 0x7fd10801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:34.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.648 7fd1167fc700 1 -- 172.21.2.101:0/1172498609 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fd110032580 con 0x7fd12019ede0 2021-02-10T11:48:34.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.649 7fd125022700 1 --2- 172.21.2.101:0/1172498609 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1080186c0 0x7fd10801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:34.648 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.649 7fd125022700 1 --2- 172.21.2.101:0/1172498609 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1080186c0 0x7fd10801abc0 crc :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:34.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.680 7fd127a07700 1 -- 172.21.2.101:0/1172498609 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd12019e8b0 con 0x7fd12019ede0 2021-02-10T11:48:34.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.683 7fd1167fc700 1 -- 172.21.2.101:0/1172498609 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fd110030d30 con 0x7fd12019ede0 2021-02-10T11:48:34.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.811 7fd127a07700 1 -- 172.21.2.101:0/1172498609 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd12019e8b0 con 0x7fd12019ede0 2021-02-10T11:48:34.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.812 7fd1167fc700 1 -- 172.21.2.101:0/1172498609 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) v1 ==== 95+0+2108 (secure 0 0 0) 0x7fd11000bd40 con 0x7fd12019ede0 2021-02-10T11:48:34.808 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:48:34.809 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":164,"default_fscid":20,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":13530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/347510536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":347510536},{"type":"v1","addr":"172.21.2.108:6833","nonce":347510536}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":160}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:48:24.824003","modified":"2021-02-10 11:48:26.829128","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":13518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13518":{"gid":13518,"name":"a","rank":0,"incarnation":161,"state":"up:active","state_seq":3,"addr":"172.21.2.101:6835/2721786668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2721786668},{"type":"v1","addr":"172.21.2.101:6835","nonce":2721786668}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[41],"metadata_pool":40,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":20}]} 2021-02-10T11:48:34.809 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 164 2021-02-10T11:48:34.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.814 7fd127a07700 1 -- 172.21.2.101:0/1172498609 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1080186c0 msgr2=0x7fd10801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:34.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.814 7fd127a07700 1 --2- 172.21.2.101:0/1172498609 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1080186c0 0x7fd10801abc0 crc :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:34.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.814 7fd127a07700 1 -- 172.21.2.101:0/1172498609 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd12019ede0 msgr2=0x7fd12019f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:34.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.814 7fd127a07700 1 --2- 172.21.2.101:0/1172498609 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd12019ede0 0x7fd12019f1e0 secure :-1 s=READY pgs=3270 cs=0 l=1 rev1=1 rx=0x7fd11000b5b0 tx=0x7fd11000b950).stop 2021-02-10T11:48:34.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.814 7fd127a07700 1 -- 172.21.2.101:0/1172498609 shutdown_connections 2021-02-10T11:48:34.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.814 7fd127a07700 1 --2- 172.21.2.101:0/1172498609 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fd1080186c0 0x7fd10801abc0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:34.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.814 7fd127a07700 1 --2- 172.21.2.101:0/1172498609 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fd12019d3c0 0x7fd12019d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:34.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.814 7fd127a07700 1 --2- 172.21.2.101:0/1172498609 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fd1201aca40 0x7fd1201ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:34.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.814 7fd127a07700 1 --2- 172.21.2.101:0/1172498609 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fd12019ede0 0x7fd12019f1e0 unknown :-1 s=CLOSED pgs=3270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:34.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.814 7fd127a07700 1 -- 172.21.2.101:0/1172498609 >> 172.21.2.101:0/1172498609 conn(0x7fd12019f880 msgr2=0x7fd120094ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:34.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.814 7fd127a07700 1 -- 172.21.2.101:0/1172498609 shutdown_connections 2021-02-10T11:48:34.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:34.814 7fd127a07700 1 -- 172.21.2.101:0/1172498609 wait complete. 2021-02-10T11:48:35.020 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 164, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:48:24.824003', 'modified': '2021-02-10 11:48:26.829128', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 13518}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_13518': {'gid': 13518, 'name': 'a', 'rank': 0, 'incarnation': 161, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.101:6835/2721786668', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.101:6834', 'nonce': 2721786668}, {'type': 'v1', 'addr': '172.21.2.101:6835', 'nonce': 2721786668}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [41], 'metadata_pool': 40, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:48:35.020 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:48:35.021 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok status 2021-02-10T11:48:35.422 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:48:35.422 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:48:35.423 INFO:teuthology.orchestra.run.gibba001.stdout: "whoami": 0, 2021-02-10T11:48:35.423 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 13518, 2021-02-10T11:48:35.423 INFO:teuthology.orchestra.run.gibba001.stdout: "want_state": "up:active", 2021-02-10T11:48:35.423 INFO:teuthology.orchestra.run.gibba001.stdout: "state": "up:active", 2021-02-10T11:48:35.423 INFO:teuthology.orchestra.run.gibba001.stdout: "rank_uptime": 10.601073918999999, 2021-02-10T11:48:35.423 INFO:teuthology.orchestra.run.gibba001.stdout: "mdsmap_epoch": 164, 2021-02-10T11:48:35.423 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch": 191, 2021-02-10T11:48:35.423 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch_barrier": 191, 2021-02-10T11:48:35.423 INFO:teuthology.orchestra.run.gibba001.stdout: "uptime": 16.259322793999999 2021-02-10T11:48:35.423 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:48:35.427 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 13518, "want_state": "up:active", "state": "up:active", "rank_uptime": 10.601073918999999, "mdsmap_epoch": 164, "osdmap_epoch": 191, "osdmap_epoch_barrier": 191, "uptime": 16.259322793999999 } 2021-02-10T11:48:35.427 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:48:35.427 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-02-10T11:48:35.427 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-02-10T11:48:35.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:35.427 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:48:35.764 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:48:36.108 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:48:36.108 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:48:36.453 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:48:36.454 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:48:36.454 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:48:36.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:36.455 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:48:36.789 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-02-10T11:48:36.790 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:48:36.793 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:48:37.136 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:48:37.136 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:48:37.203 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[182070]: starting ceph client 2021-02-10T11:48:37.203 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:2021-02-10 11:48:37.207 7ff1063b3f80 -1 init, newargv = 0x56505e2c91f0 newargc=9 2021-02-10T11:48:37.208 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[182070]: starting fuse 2021-02-10T11:48:37.479 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:48:37.479 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:48:37.480 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:48:37.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:37.480 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:48:37.815 INFO:teuthology.orchestra.run.gibba001.stdout:42 2021-02-10T11:48:37.816 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2021-02-10T11:48:37.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:37.816 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:48:37.816 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:48:37.816 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:48:37.816 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:48:37.816 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:48:37.816 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:48:37.817 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:48:37.817 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-02-10T11:48:37.817 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:48:37.817 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:48:37.817 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:48:37.817 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:48:37.817 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:48:37.817 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:48:37.817 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:48:37.817 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:48:37.817 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:48:37.817 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:48:37.817 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:48:37.817 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:48:37.817 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:48:37.817 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.0")) 2021-02-10T11:48:37.818 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:48:38.180 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.0.182070.asok 2021-02-10T11:48:38.183 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.182070.asok 2021-02-10T11:48:38.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:38.183 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.182070.asok status 2021-02-10T11:48:38.575 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:48:38.575 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:48:38.575 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:48:38.576 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:48:38.576 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "0", 2021-02-10T11:48:38.576 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:48:38.576 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-02-10T11:48:38.576 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "182070", 2021-02-10T11:48:38.577 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:48:38.577 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:38.577 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:48:38.577 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:48:38.577 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 13680, 2021-02-10T11:48:38.578 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:48:38.578 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:48:38.578 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:48:38.578 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 13680 2021-02-10T11:48:38.578 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:38.579 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:48:38.579 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:38.579 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:48:38.579 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 266596040 2021-02-10T11:48:38.579 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:38.579 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:38.580 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:48:38.580 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:38.580 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:48:38.580 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 266596040 2021-02-10T11:48:38.580 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:38.581 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.13680 172.21.2.101:0/266596040", 2021-02-10T11:48:38.581 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/266596040", 2021-02-10T11:48:38.581 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:48:38.581 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 164, 2021-02-10T11:48:38.581 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 191, 2021-02-10T11:48:38.581 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:48:38.582 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:48:38.582 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:48:38.583 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:38.584 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:48:38.584 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:48:38.919 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:48:38.919 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:48:38.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:38.920 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:48:39.266 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:48:39.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.687 7f6a8328d700 1 Processor -- start 2021-02-10T11:48:39.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.687 7f6a8328d700 1 -- start start 2021-02-10T11:48:39.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.688 7f6a8328d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f6a7c0d1870 0x7f6a7c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:39.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.688 7f6a8328d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c0f0b10 con 0x7f6a7c1b0a10 2021-02-10T11:48:39.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.688 7f6a8328d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c0d0310 con 0x7f6a7c0f0650 2021-02-10T11:48:39.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.688 7f6a8328d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a7c0d0530 con 0x7f6a7c0d1870 2021-02-10T11:48:39.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.688 7f6a808a8700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f6a7c1b0a10 0x7f6a7c0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35628/0 (socket says 172.21.2.101:35628) 2021-02-10T11:48:39.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.688 7f6a808a8700 1 -- 172.21.2.101:0/58309172 learned_addr learned my addr 172.21.2.101:0/58309172 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:39.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.689 7f6a73fff700 1 -- 172.21.2.101:0/58309172 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784302144 0 0) 0x7f6a6c0010b0 con 0x7f6a7c1b0a10 2021-02-10T11:48:39.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.689 7f6a73fff700 1 -- 172.21.2.101:0/58309172 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a68002fe0 con 0x7f6a7c1b0a10 2021-02-10T11:48:39.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.689 7f6a810a9700 1 --2- 172.21.2.101:0/58309172 >> v2:172.21.2.108:3300/0 conn(0x7f6a7c0d1870 0x7f6a7c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:39.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.689 7f6a73fff700 1 -- 172.21.2.101:0/58309172 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3663140825 0 0) 0x7f6a6c001010 con 0x7f6a7c1b0a10 2021-02-10T11:48:39.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.689 7f6a73fff700 1 -- 172.21.2.101:0/58309172 >> v2:172.21.2.108:3300/0 conn(0x7f6a7c0d1870 msgr2=0x7f6a7c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:39.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.689 7f6a73fff700 1 --2- 172.21.2.101:0/58309172 >> v2:172.21.2.108:3300/0 conn(0x7f6a7c0d1870 0x7f6a7c1b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:39.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.689 7f6a73fff700 1 -- 172.21.2.101:0/58309172 >> v1:172.21.2.108:6789/0 conn(0x7f6a7c0f0650 legacy=0x7f6a7c0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:39.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.689 7f6a73fff700 1 -- 172.21.2.101:0/58309172 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c0d1c70 con 0x7f6a7c1b0a10 2021-02-10T11:48:39.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.689 7f6a73fff700 1 -- 172.21.2.101:0/58309172 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f6a6c001530 con 0x7f6a7c1b0a10 2021-02-10T11:48:39.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.689 7f6a73fff700 1 -- 172.21.2.101:0/58309172 >> v1:172.21.2.101:6789/0 conn(0x7f6a7c1b0a10 legacy=0x7f6a7c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:39.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.689 7f6a73fff700 1 --2- 172.21.2.101:0/58309172 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a68003a60 0x7f6a68005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:39.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.689 7f6a73fff700 1 --2- 172.21.2.101:0/58309172 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a68006790 0x7f6a68008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:39.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.689 7f6a73fff700 1 --2- 172.21.2.101:0/58309172 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a68009590 0x7f6a6800bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:39.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.689 7f6a73fff700 1 -- 172.21.2.101:0/58309172 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6800c300 con 0x7f6a68009590 2021-02-10T11:48:39.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.689 7f6a73fff700 1 -- 172.21.2.101:0/58309172 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6800c520 con 0x7f6a68003a60 2021-02-10T11:48:39.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.689 7f6a73fff700 1 -- 172.21.2.101:0/58309172 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6800c740 con 0x7f6a68006790 2021-02-10T11:48:39.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.689 7f6a818aa700 1 --2- 172.21.2.101:0/58309172 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a68009590 0x7f6a6800bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:39.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.690 7f6a810a9700 1 --2- 172.21.2.101:0/58309172 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a68003a60 0x7f6a68005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:39.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.690 7f6a808a8700 1 --2- 172.21.2.101:0/58309172 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a68006790 0x7f6a68008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:39.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.690 7f6a818aa700 1 -- 172.21.2.101:0/58309172 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a68006790 msgr2=0x7f6a68008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:39.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.690 7f6a818aa700 1 --2- 172.21.2.101:0/58309172 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a68006790 0x7f6a68008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:39.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.690 7f6a818aa700 1 -- 172.21.2.101:0/58309172 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a68003a60 msgr2=0x7f6a68005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:39.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.690 7f6a818aa700 1 --2- 172.21.2.101:0/58309172 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a68003a60 0x7f6a68005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:39.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.690 7f6a818aa700 1 -- 172.21.2.101:0/58309172 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6800c9e0 con 0x7f6a68009590 2021-02-10T11:48:39.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.690 7f6a818aa700 1 -- 172.21.2.101:0/58309172 reap_dead start 2021-02-10T11:48:39.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.690 7f6a818aa700 1 --2- 172.21.2.101:0/58309172 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a68009590 0x7f6a6800bab0 secure :-1 s=READY pgs=3273 cs=0 l=1 rev1=1 rx=0x7f6a7404d670 tx=0x7f6a7404de50).ready entity=mon.0 client_cookie=4564e32eaf6bc4b8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:39.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.690 7f6a73fff700 1 -- 172.21.2.101:0/58309172 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a74006a60 con 0x7f6a68009590 2021-02-10T11:48:39.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.690 7f6a73fff700 1 -- 172.21.2.101:0/58309172 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7404f300 con 0x7f6a68009590 2021-02-10T11:48:39.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.690 7f6a73fff700 1 -- 172.21.2.101:0/58309172 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a74003020 con 0x7f6a68009590 2021-02-10T11:48:39.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.690 7f6a8328d700 1 -- 172.21.2.101:0/58309172 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a68009590 msgr2=0x7f6a6800bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:39.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.690 7f6a8328d700 1 --2- 172.21.2.101:0/58309172 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a68009590 0x7f6a6800bab0 secure :-1 s=READY pgs=3273 cs=0 l=1 rev1=1 rx=0x7f6a7404d670 tx=0x7f6a7404de50).stop 2021-02-10T11:48:39.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.691 7f6a8328d700 1 -- 172.21.2.101:0/58309172 shutdown_connections 2021-02-10T11:48:39.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.691 7f6a8328d700 1 --2- 172.21.2.101:0/58309172 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a68009590 0x7f6a6800bab0 unknown :-1 s=CLOSED pgs=3273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:39.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.691 7f6a8328d700 1 -- 172.21.2.101:0/58309172 >> 172.21.2.101:0/58309172 conn(0x7f6a7c0ce450 msgr2=0x7f6a7c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:39.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.691 7f6a8328d700 1 -- 172.21.2.101:0/58309172 shutdown_connections 2021-02-10T11:48:39.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.691 7f6a8328d700 1 -- 172.21.2.101:0/58309172 wait complete. 2021-02-10T11:48:39.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.706 7f6a8328d700 1 Processor -- start 2021-02-10T11:48:39.705 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.706 7f6a8328d700 1 -- start start 2021-02-10T11:48:39.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.706 7f6a8328d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a7c0aa700 0x7f6a7c090ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:39.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.706 7f6a8328d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a7c0aab00 0x7f6a7c0910e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:39.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.706 7f6a8328d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a7c0ecb10 0x7f6a7c0ecf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:39.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.706 7f6a8328d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c090540 con 0x7f6a7c0aab00 2021-02-10T11:48:39.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.706 7f6a8328d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0c0170 con 0x7f6a7c0aa700 2021-02-10T11:48:39.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.706 7f6a8328d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0ed530 con 0x7f6a7c0ecb10 2021-02-10T11:48:39.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.706 7f6a808a8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a7c0aab00 0x7f6a7c0910e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:39.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.706 7f6a808a8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a7c0aab00 0x7f6a7c0910e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35144/0 (socket says 172.21.2.101:35144) 2021-02-10T11:48:39.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.706 7f6a808a8700 1 -- 172.21.2.101:0/300502517 learned_addr learned my addr 172.21.2.101:0/300502517 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:39.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.707 7f6a808a8700 1 -- 172.21.2.101:0/300502517 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a7c0ecb10 msgr2=0x7f6a7c0ecf10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:39.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.707 7f6a808a8700 1 --2- 172.21.2.101:0/300502517 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a7c0ecb10 0x7f6a7c0ecf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.707 7f6a808a8700 1 -- 172.21.2.101:0/300502517 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a7c0aa700 msgr2=0x7f6a7c090ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.707 7f6a808a8700 1 --2- 172.21.2.101:0/300502517 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a7c0aa700 0x7f6a7c090ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.707 7f6a808a8700 1 -- 172.21.2.101:0/300502517 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c0ed750 con 0x7f6a7c0aab00 2021-02-10T11:48:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.707 7f6a808a8700 1 --2- 172.21.2.101:0/300502517 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a7c0aab00 0x7f6a7c0910e0 secure :-1 s=READY pgs=3274 cs=0 l=1 rev1=1 rx=0x7f6a6c007630 tx=0x7f6a6c00a8f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.707 7f6a71ffb700 1 -- 172.21.2.101:0/300502517 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a6c00ace0 con 0x7f6a7c0aab00 2021-02-10T11:48:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.707 7f6a71ffb700 1 -- 172.21.2.101:0/300502517 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6c004140 con 0x7f6a7c0aab00 2021-02-10T11:48:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.707 7f6a8328d700 1 -- 172.21.2.101:0/300502517 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7c0ed750 con 0x7f6a7c0aab00 2021-02-10T11:48:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.707 7f6a8328d700 1 -- 172.21.2.101:0/300502517 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c08d110 con 0x7f6a7c0aab00 2021-02-10T11:48:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.707 7f6a71ffb700 1 -- 172.21.2.101:0/300502517 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a6c004660 con 0x7f6a7c0aab00 2021-02-10T11:48:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.708 7f6a71ffb700 1 -- 172.21.2.101:0/300502517 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6a6c004320 con 0x7f6a7c0aab00 2021-02-10T11:48:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.708 7f6a71ffb700 1 --2- 172.21.2.101:0/300502517 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6a5c0186c0 0x7f6a5c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.708 7f6a71ffb700 1 -- 172.21.2.101:0/300502517 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6a6c032580 con 0x7f6a7c0aab00 2021-02-10T11:48:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.708 7f6a810a9700 1 --2- 172.21.2.101:0/300502517 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6a5c0186c0 0x7f6a5c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:39.708 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.708 7f6a810a9700 1 --2- 172.21.2.101:0/300502517 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6a5c0186c0 0x7f6a5c01abc0 crc :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:39.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.738 7f6a8328d700 1 -- 172.21.2.101:0/300502517 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7c0ed9f0 con 0x7f6a7c0aab00 2021-02-10T11:48:39.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.741 7f6a71ffb700 1 -- 172.21.2.101:0/300502517 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6a6c000dd0 con 0x7f6a7c0aab00 2021-02-10T11:48:39.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.866 7f6a8328d700 1 -- 172.21.2.101:0/300502517 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a7c0ebc00 con 0x7f6a7c0aab00 2021-02-10T11:48:39.863 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.867 7f6a71ffb700 1 -- 172.21.2.101:0/300502517 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f6a6c004320 con 0x7f6a7c0aab00 2021-02-10T11:48:39.864 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:48:39.864 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":191,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:48:25.826144","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":41,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":40,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:48:22.784885","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":41,"pool_name":"cephfs_data","create_time":"2021-02-10 11:48:23.812792","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"191","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:48:39.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.869 7f6a8328d700 1 -- 172.21.2.101:0/300502517 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6a5c0186c0 msgr2=0x7f6a5c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:39.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.869 7f6a8328d700 1 --2- 172.21.2.101:0/300502517 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6a5c0186c0 0x7f6a5c01abc0 crc :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:39.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.869 7f6a8328d700 1 -- 172.21.2.101:0/300502517 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a7c0aab00 msgr2=0x7f6a7c0910e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:39.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.869 7f6a8328d700 1 --2- 172.21.2.101:0/300502517 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a7c0aab00 0x7f6a7c0910e0 secure :-1 s=READY pgs=3274 cs=0 l=1 rev1=1 rx=0x7f6a6c007630 tx=0x7f6a6c00a8f0).stop 2021-02-10T11:48:39.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.869 7f6a8328d700 1 -- 172.21.2.101:0/300502517 shutdown_connections 2021-02-10T11:48:39.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.869 7f6a8328d700 1 --2- 172.21.2.101:0/300502517 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6a5c0186c0 0x7f6a5c01abc0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:39.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.869 7f6a8328d700 1 --2- 172.21.2.101:0/300502517 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6a7c0ecb10 0x7f6a7c0ecf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:39.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.869 7f6a8328d700 1 --2- 172.21.2.101:0/300502517 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6a7c0aab00 0x7f6a7c0910e0 unknown :-1 s=CLOSED pgs=3274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:39.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.869 7f6a8328d700 1 --2- 172.21.2.101:0/300502517 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6a7c0aa700 0x7f6a7c090ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:39.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.869 7f6a8328d700 1 -- 172.21.2.101:0/300502517 >> 172.21.2.101:0/300502517 conn(0x7f6a7c0b7e70 msgr2=0x7f6a7c0c3850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:39.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.869 7f6a8328d700 1 -- 172.21.2.101:0/300502517 shutdown_connections 2021-02-10T11:48:39.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:39.869 7f6a8328d700 1 -- 172.21.2.101:0/300502517 wait complete. 2021-02-10T11:48:39.873 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 40, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:48:22.784885', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '190', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 41, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:48:23.812792', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '191', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:48:39.874 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:48:40.000 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:48:40.004 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:48:40.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.294 7ff09e763700 1 Processor -- start 2021-02-10T11:48:40.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.294 7ff09e763700 1 -- start start 2021-02-10T11:48:40.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.295 7ff09e763700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0980d1870 0x7ff0981b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:40.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.295 7ff09e763700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff0980cff70 0x7ff098109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:40.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.295 7ff09e763700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff098109a60 con 0x7ff0981b0ae0 2021-02-10T11:48:40.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.295 7ff09e763700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff098109750 con 0x7ff0980cff70 2021-02-10T11:48:40.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.295 7ff09e763700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff098088070 con 0x7ff0980d1870 2021-02-10T11:48:40.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.295 7ff097fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0980d1870 0x7ff0981b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:40.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.295 7ff09cd80700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff0980cff70 0x7ff098109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:40.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.295 7ff097fff700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0980d1870 0x7ff0981b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:56116/0 (socket says 172.21.2.101:56116) 2021-02-10T11:48:40.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.295 7ff097fff700 1 -- 172.21.2.101:0/873346994 learned_addr learned my addr 172.21.2.101:0/873346994 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:40.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.295 7ff0977fe700 1 --1- 172.21.2.101:0/873346994 >> v1:172.21.2.108:6789/0 conn(0x7ff0981b0ae0 0x7ff098108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:57288/0 (socket says 172.21.2.101:57288) 2021-02-10T11:48:40.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.296 7ff097fff700 1 -- 172.21.2.101:0/873346994 >> v2:172.21.2.108:3300/0 conn(0x7ff0980cff70 msgr2=0x7ff098109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:40.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.296 7ff097fff700 1 --2- 172.21.2.101:0/873346994 >> v2:172.21.2.108:3300/0 conn(0x7ff0980cff70 0x7ff098109130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:40.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.296 7ff097fff700 1 -- 172.21.2.101:0/873346994 >> v1:172.21.2.108:6789/0 conn(0x7ff0981b0ae0 legacy=0x7ff098108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:40.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.296 7ff097fff700 1 -- 172.21.2.101:0/873346994 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff098088260 con 0x7ff0980d1870 2021-02-10T11:48:40.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.296 7ff097fff700 1 --2- 172.21.2.101:0/873346994 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0980d1870 0x7ff0981b04c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7ff088006700 tx=0x7ff088009b80).ready entity=mon.2 client_cookie=41c979f09da49b19 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:40.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.296 7ff096ffd700 1 -- 172.21.2.101:0/873346994 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff08800c0c0 con 0x7ff0980d1870 2021-02-10T11:48:40.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.296 7ff096ffd700 1 -- 172.21.2.101:0/873346994 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08800c340 con 0x7ff0980d1870 2021-02-10T11:48:40.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.296 7ff096ffd700 1 -- 172.21.2.101:0/873346994 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff08800c0c0 con 0x7ff0980d1870 2021-02-10T11:48:40.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.296 7ff09e763700 1 -- 172.21.2.101:0/873346994 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0980d1870 msgr2=0x7ff0981b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:40.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.296 7ff09e763700 1 --2- 172.21.2.101:0/873346994 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0980d1870 0x7ff0981b04c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7ff088006700 tx=0x7ff088009b80).stop 2021-02-10T11:48:40.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.297 7ff09e763700 1 -- 172.21.2.101:0/873346994 shutdown_connections 2021-02-10T11:48:40.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.297 7ff09e763700 1 --2- 172.21.2.101:0/873346994 >> v2:172.21.2.108:3300/0 conn(0x7ff0980cff70 0x7ff098109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:40.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.297 7ff09e763700 1 --2- 172.21.2.101:0/873346994 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0980d1870 0x7ff0981b04c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:40.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.297 7ff09e763700 1 -- 172.21.2.101:0/873346994 >> 172.21.2.101:0/873346994 conn(0x7ff0980ce420 msgr2=0x7ff0980d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:40.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.297 7ff09e763700 1 -- 172.21.2.101:0/873346994 shutdown_connections 2021-02-10T11:48:40.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.297 7ff09e763700 1 -- 172.21.2.101:0/873346994 wait complete. 2021-02-10T11:48:40.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.312 7ff09e763700 1 Processor -- start 2021-02-10T11:48:40.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.312 7ff09e763700 1 -- start start 2021-02-10T11:48:40.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.312 7ff09e763700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0980c1850 0x7ff0980c1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:40.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.312 7ff09e763700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0980c2810 0x7ff0980c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:40.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.312 7ff09e763700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0980c3230 0x7ff0980c3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:40.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.312 7ff09e763700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0980c1c50 con 0x7ff0980c3230 2021-02-10T11:48:40.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.312 7ff09e763700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff098084920 con 0x7ff0980c1850 2021-02-10T11:48:40.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.312 7ff09e763700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff098084b40 con 0x7ff0980c2810 2021-02-10T11:48:40.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.312 7ff09cd80700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0980c3230 0x7ff0980c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:40.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.312 7ff09cd80700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0980c3230 0x7ff0980c3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35160/0 (socket says 172.21.2.101:35160) 2021-02-10T11:48:40.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.312 7ff09cd80700 1 -- 172.21.2.101:0/212598527 learned_addr learned my addr 172.21.2.101:0/212598527 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:40.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.313 7ff09cd80700 1 -- 172.21.2.101:0/212598527 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0980c2810 msgr2=0x7ff0980c2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:40.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.313 7ff09cd80700 1 --2- 172.21.2.101:0/212598527 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0980c2810 0x7ff0980c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:40.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.313 7ff09cd80700 1 -- 172.21.2.101:0/212598527 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0980c1850 msgr2=0x7ff0980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:40.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.313 7ff09cd80700 1 --2- 172.21.2.101:0/212598527 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0980c1850 0x7ff0980c1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:40.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.313 7ff09cd80700 1 -- 172.21.2.101:0/212598527 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff098084d60 con 0x7ff0980c3230 2021-02-10T11:48:40.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.313 7ff09cd80700 1 --2- 172.21.2.101:0/212598527 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0980c3230 0x7ff0980c3630 secure :-1 s=READY pgs=3275 cs=0 l=1 rev1=1 rx=0x7ff09000b970 tx=0x7ff09000bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:40.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.313 7ff094ff9700 1 -- 172.21.2.101:0/212598527 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff090000f90 con 0x7ff0980c3230 2021-02-10T11:48:40.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.313 7ff094ff9700 1 -- 172.21.2.101:0/212598527 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff090003030 con 0x7ff0980c3230 2021-02-10T11:48:40.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.313 7ff09e763700 1 -- 172.21.2.101:0/212598527 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff098084d60 con 0x7ff0980c3230 2021-02-10T11:48:40.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.313 7ff094ff9700 1 -- 172.21.2.101:0/212598527 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff090001400 con 0x7ff0980c3230 2021-02-10T11:48:40.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.313 7ff09e763700 1 -- 172.21.2.101:0/212598527 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff098084d60 con 0x7ff0980c3230 2021-02-10T11:48:40.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.314 7ff094ff9700 1 -- 172.21.2.101:0/212598527 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff090000df0 con 0x7ff0980c3230 2021-02-10T11:48:40.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.314 7ff094ff9700 1 --2- 172.21.2.101:0/212598527 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0780186c0 0x7ff07801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:40.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.314 7ff094ff9700 1 -- 172.21.2.101:0/212598527 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff0900328d0 con 0x7ff0980c3230 2021-02-10T11:48:40.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.314 7ff097fff700 1 --2- 172.21.2.101:0/212598527 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0780186c0 0x7ff07801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:40.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.314 7ff097fff700 1 --2- 172.21.2.101:0/212598527 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0780186c0 0x7ff07801abc0 crc :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:40.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.344 7ff09e763700 1 -- 172.21.2.101:0/212598527 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff098109a60 con 0x7ff0980c3230 2021-02-10T11:48:40.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.347 7ff094ff9700 1 -- 172.21.2.101:0/212598527 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff09001f840 con 0x7ff0980c3230 2021-02-10T11:48:40.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.475 7ff09e763700 1 -- 172.21.2.101:0/212598527 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7ff098109a60 con 0x7ff0980c3230 2021-02-10T11:48:40.471 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.475 7ff094ff9700 1 -- 172.21.2.101:0/212598527 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v191) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff090000df0 con 0x7ff0980c3230 2021-02-10T11:48:40.472 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:48:40.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.477 7ff09e763700 1 -- 172.21.2.101:0/212598527 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0780186c0 msgr2=0x7ff07801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:40.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.477 7ff09e763700 1 --2- 172.21.2.101:0/212598527 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0780186c0 0x7ff07801abc0 crc :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:40.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.477 7ff09e763700 1 -- 172.21.2.101:0/212598527 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0980c3230 msgr2=0x7ff0980c3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:40.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.477 7ff09e763700 1 --2- 172.21.2.101:0/212598527 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0980c3230 0x7ff0980c3630 secure :-1 s=READY pgs=3275 cs=0 l=1 rev1=1 rx=0x7ff09000b970 tx=0x7ff09000bd10).stop 2021-02-10T11:48:40.473 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.477 7ff09e763700 1 -- 172.21.2.101:0/212598527 shutdown_connections 2021-02-10T11:48:40.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.477 7ff09e763700 1 --2- 172.21.2.101:0/212598527 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff0780186c0 0x7ff07801abc0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:40.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.477 7ff09e763700 1 --2- 172.21.2.101:0/212598527 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff0980c2810 0x7ff0980c2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:40.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.477 7ff09e763700 1 --2- 172.21.2.101:0/212598527 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff0980c3230 0x7ff0980c3630 unknown :-1 s=CLOSED pgs=3275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:40.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.477 7ff09e763700 1 --2- 172.21.2.101:0/212598527 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff0980c1850 0x7ff0980c1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:40.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.477 7ff09e763700 1 -- 172.21.2.101:0/212598527 >> 172.21.2.101:0/212598527 conn(0x7ff0980af690 msgr2=0x7ff098094c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:40.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.477 7ff09e763700 1 -- 172.21.2.101:0/212598527 shutdown_connections 2021-02-10T11:48:40.474 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.477 7ff09e763700 1 -- 172.21.2.101:0/212598527 wait complete. 2021-02-10T11:48:40.482 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:48:40.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.900 7f24484b9700 1 Processor -- start 2021-02-10T11:48:40.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.900 7f24484b9700 1 -- start start 2021-02-10T11:48:40.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.901 7f24484b9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f244010aff0 0x7f24400cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:40.915 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.901 7f24484b9700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24400fe1c0 con 0x7f24400d1c70 2021-02-10T11:48:40.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.901 7f24484b9700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24401b0cb0 con 0x7f24400d1870 2021-02-10T11:48:40.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.901 7f24484b9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2440109a60 con 0x7f244010aff0 2021-02-10T11:48:40.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.901 7f2445ad4700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f24400d1c70 0x7f24400cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35658/0 (socket says 172.21.2.101:35658) 2021-02-10T11:48:40.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.901 7f2445ad4700 1 -- 172.21.2.101:0/591839641 learned_addr learned my addr 172.21.2.101:0/591839641 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:40.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.901 7f24452d3700 1 -- 172.21.2.101:0/591839641 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288298804 0 0) 0x7f2430001180 con 0x7f24400d1c70 2021-02-10T11:48:40.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.902 7f24452d3700 1 -- 172.21.2.101:0/591839641 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f242c002fe0 con 0x7f24400d1c70 2021-02-10T11:48:40.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.902 7f2446ad6700 1 --2- 172.21.2.101:0/591839641 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f244010aff0 0x7f24400cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:40.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.902 7f24452d3700 1 -- 172.21.2.101:0/591839641 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4171890100 0 0) 0x7f2430001180 con 0x7f24400d1c70 2021-02-10T11:48:40.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.902 7f24452d3700 1 -- 172.21.2.101:0/591839641 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f244010aff0 msgr2=0x7f24400cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:40.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.902 7f24452d3700 1 --2- 172.21.2.101:0/591839641 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f244010aff0 0x7f24400cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:40.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.902 7f24452d3700 1 -- 172.21.2.101:0/591839641 >> v1:172.21.2.108:6789/0 conn(0x7f24400d1870 legacy=0x7f24401b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:40.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.902 7f24452d3700 1 -- 172.21.2.101:0/591839641 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244010b3f0 con 0x7f24400d1c70 2021-02-10T11:48:40.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.902 7f24452d3700 1 -- 172.21.2.101:0/591839641 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f2430001680 con 0x7f24400d1c70 2021-02-10T11:48:40.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.902 7f24452d3700 1 -- 172.21.2.101:0/591839641 >> v1:172.21.2.101:6789/0 conn(0x7f24400d1c70 legacy=0x7f24400cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:40.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.902 7f24452d3700 1 --2- 172.21.2.101:0/591839641 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f242c003a60 0x7f242c005e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:40.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.902 7f24452d3700 1 --2- 172.21.2.101:0/591839641 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f242c006730 0x7f242c008c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:40.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.902 7f24452d3700 1 --2- 172.21.2.101:0/591839641 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f242c0094f0 0x7f242c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:40.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.902 7f24452d3700 1 -- 172.21.2.101:0/591839641 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c00c2c0 con 0x7f242c006730 2021-02-10T11:48:40.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.902 7f24452d3700 1 -- 172.21.2.101:0/591839641 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c00c4e0 con 0x7f242c0094f0 2021-02-10T11:48:40.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.902 7f24452d3700 1 -- 172.21.2.101:0/591839641 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242c00c700 con 0x7f242c003a60 2021-02-10T11:48:40.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.902 7f2445ad4700 1 --2- 172.21.2.101:0/591839641 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f242c006730 0x7f242c008c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:40.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.902 7f24462d5700 1 --2- 172.21.2.101:0/591839641 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f242c003a60 0x7f242c005e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:40.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.903 7f2446ad6700 1 --2- 172.21.2.101:0/591839641 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f242c0094f0 0x7f242c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:40.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.903 7f2445ad4700 1 -- 172.21.2.101:0/591839641 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f242c003a60 msgr2=0x7f242c005e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:40.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.903 7f2445ad4700 1 --2- 172.21.2.101:0/591839641 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f242c003a60 0x7f242c005e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:40.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.903 7f2445ad4700 1 -- 172.21.2.101:0/591839641 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f242c0094f0 msgr2=0x7f242c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:40.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.903 7f2445ad4700 1 --2- 172.21.2.101:0/591839641 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f242c0094f0 0x7f242c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:40.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.903 7f2445ad4700 1 -- 172.21.2.101:0/591839641 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c050970 con 0x7f242c006730 2021-02-10T11:48:40.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.903 7f2445ad4700 1 --2- 172.21.2.101:0/591839641 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f242c006730 0x7f242c008c50 secure :-1 s=READY pgs=3277 cs=0 l=1 rev1=1 rx=0x7f24300014e0 tx=0x7f2430007d20).ready entity=mon.0 client_cookie=e1bba2e3c49b39f1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:40.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.903 7f24452d3700 1 -- 172.21.2.101:0/591839641 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f24300070c0 con 0x7f242c006730 2021-02-10T11:48:40.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.903 7f24452d3700 1 -- 172.21.2.101:0/591839641 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24300043a0 con 0x7f242c006730 2021-02-10T11:48:40.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.903 7f24452d3700 1 -- 172.21.2.101:0/591839641 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f24300048c0 con 0x7f242c006730 2021-02-10T11:48:40.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.903 7f24484b9700 1 -- 172.21.2.101:0/591839641 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f242c006730 msgr2=0x7f242c008c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:40.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.903 7f24484b9700 1 --2- 172.21.2.101:0/591839641 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f242c006730 0x7f242c008c50 secure :-1 s=READY pgs=3277 cs=0 l=1 rev1=1 rx=0x7f24300014e0 tx=0x7f2430007d20).stop 2021-02-10T11:48:40.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.903 7f2446ad6700 1 -- 172.21.2.101:0/591839641 reap_dead start 2021-02-10T11:48:40.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.904 7f24484b9700 1 -- 172.21.2.101:0/591839641 shutdown_connections 2021-02-10T11:48:40.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.904 7f24484b9700 1 -- 172.21.2.101:0/591839641 >> 172.21.2.101:0/591839641 conn(0x7f24400ce420 msgr2=0x7f24400d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:40.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.904 7f24484b9700 1 -- 172.21.2.101:0/591839641 shutdown_connections 2021-02-10T11:48:40.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.904 7f24484b9700 1 -- 172.21.2.101:0/591839641 wait complete. 2021-02-10T11:48:40.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.919 7f24484b9700 1 Processor -- start 2021-02-10T11:48:40.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.919 7f24484b9700 1 -- start start 2021-02-10T11:48:40.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.919 7f24484b9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2440085bc0 0x7f24400863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:40.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.919 7f24484b9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24400954c0 0x7f2440096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:40.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.919 7f24484b9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24400958c0 0x7f2440096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:40.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.919 7f24484b9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2440095d80 con 0x7f24400954c0 2021-02-10T11:48:40.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.919 7f24484b9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24400bfa60 con 0x7f2440085bc0 2021-02-10T11:48:40.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.919 7f24484b9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2440085fc0 con 0x7f24400958c0 2021-02-10T11:48:40.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.919 7f2445ad4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24400954c0 0x7f2440096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:40.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.919 7f2445ad4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24400954c0 0x7f2440096300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35176/0 (socket says 172.21.2.101:35176) 2021-02-10T11:48:40.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.919 7f2445ad4700 1 -- 172.21.2.101:0/770478125 learned_addr learned my addr 172.21.2.101:0/770478125 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:40.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.920 7f24462d5700 1 --2- 172.21.2.101:0/770478125 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2440085bc0 0x7f24400863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:40.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.920 7f2446ad6700 1 --2- 172.21.2.101:0/770478125 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24400958c0 0x7f2440096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:40.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.920 7f2445ad4700 1 -- 172.21.2.101:0/770478125 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24400958c0 msgr2=0x7f2440096920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:40.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.920 7f2445ad4700 1 --2- 172.21.2.101:0/770478125 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24400958c0 0x7f2440096920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:40.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.920 7f2445ad4700 1 -- 172.21.2.101:0/770478125 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2440085bc0 msgr2=0x7f24400863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:40.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.920 7f2445ad4700 1 --2- 172.21.2.101:0/770478125 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2440085bc0 0x7f24400863e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:40.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.920 7f2445ad4700 1 -- 172.21.2.101:0/770478125 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2440086c10 con 0x7f24400954c0 2021-02-10T11:48:40.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.920 7f24462d5700 1 --2- 172.21.2.101:0/770478125 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2440085bc0 0x7f24400863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:48:40.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.920 7f2445ad4700 1 --2- 172.21.2.101:0/770478125 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24400954c0 0x7f2440096300 secure :-1 s=READY pgs=3278 cs=0 l=1 rev1=1 rx=0x7f2430000f40 tx=0x7f243000a870).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:40.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.921 7f2436ffd700 1 -- 172.21.2.101:0/770478125 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2430007280 con 0x7f24400954c0 2021-02-10T11:48:40.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.921 7f2436ffd700 1 -- 172.21.2.101:0/770478125 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243000ac60 con 0x7f24400954c0 2021-02-10T11:48:40.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.921 7f2436ffd700 1 -- 172.21.2.101:0/770478125 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2430004590 con 0x7f24400954c0 2021-02-10T11:48:40.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.921 7f24484b9700 1 -- 172.21.2.101:0/770478125 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2440086c10 con 0x7f24400954c0 2021-02-10T11:48:40.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.921 7f24484b9700 1 -- 172.21.2.101:0/770478125 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2440097230 con 0x7f24400954c0 2021-02-10T11:48:40.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.921 7f2436ffd700 1 -- 172.21.2.101:0/770478125 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2430007070 con 0x7f24400954c0 2021-02-10T11:48:40.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.921 7f2436ffd700 1 --2- 172.21.2.101:0/770478125 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24280186c0 0x7f242801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:40.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.921 7f2436ffd700 1 -- 172.21.2.101:0/770478125 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2430032550 con 0x7f24400954c0 2021-02-10T11:48:40.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.922 7f24462d5700 1 --2- 172.21.2.101:0/770478125 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24280186c0 0x7f242801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:40.922 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.922 7f24462d5700 1 --2- 172.21.2.101:0/770478125 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24280186c0 0x7f242801abc0 crc :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:40.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.950 7f24484b9700 1 -- 172.21.2.101:0/770478125 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24400fe1c0 con 0x7f24400954c0 2021-02-10T11:48:40.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:40.953 7f2436ffd700 1 -- 172.21.2.101:0/770478125 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2430000cb0 con 0x7f24400954c0 2021-02-10T11:48:41.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.081 7f24484b9700 1 -- 172.21.2.101:0/770478125 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f24400fe1c0 con 0x7f24400954c0 2021-02-10T11:48:41.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.081 7f2436ffd700 1 -- 172.21.2.101:0/770478125 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v191) v1 ==== 104+0+10 (secure 0 0 0) 0x7f2430007050 con 0x7f24400954c0 2021-02-10T11:48:41.078 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:48:41.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.083 7f24484b9700 1 -- 172.21.2.101:0/770478125 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24280186c0 msgr2=0x7f242801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:41.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.083 7f24484b9700 1 --2- 172.21.2.101:0/770478125 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24280186c0 0x7f242801abc0 crc :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:41.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.083 7f24484b9700 1 -- 172.21.2.101:0/770478125 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24400954c0 msgr2=0x7f2440096300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:41.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.083 7f24484b9700 1 --2- 172.21.2.101:0/770478125 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24400954c0 0x7f2440096300 secure :-1 s=READY pgs=3278 cs=0 l=1 rev1=1 rx=0x7f2430000f40 tx=0x7f243000a870).stop 2021-02-10T11:48:41.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.084 7f24484b9700 1 -- 172.21.2.101:0/770478125 shutdown_connections 2021-02-10T11:48:41.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.084 7f24484b9700 1 --2- 172.21.2.101:0/770478125 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f24280186c0 0x7f242801abc0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:41.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.084 7f24484b9700 1 --2- 172.21.2.101:0/770478125 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f24400958c0 0x7f2440096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:41.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.084 7f24484b9700 1 --2- 172.21.2.101:0/770478125 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f24400954c0 0x7f2440096300 unknown :-1 s=CLOSED pgs=3278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:41.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.084 7f24484b9700 1 --2- 172.21.2.101:0/770478125 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2440085bc0 0x7f24400863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:41.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.084 7f24484b9700 1 -- 172.21.2.101:0/770478125 >> 172.21.2.101:0/770478125 conn(0x7f24400bef00 msgr2=0x7f244009d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:41.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.084 7f24484b9700 1 -- 172.21.2.101:0/770478125 shutdown_connections 2021-02-10T11:48:41.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.084 7f24484b9700 1 -- 172.21.2.101:0/770478125 wait complete. 2021-02-10T11:48:41.088 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:48:41.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.508 7fbed13fd700 1 Processor -- start 2021-02-10T11:48:41.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.508 7fbed13fd700 1 -- start start 2021-02-10T11:48:41.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.509 7fbed13fd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbecc0d1870 0x7fbecc1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:41.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.509 7fbed13fd700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbecc0fe1c0 con 0x7fbecc1b0ae0 2021-02-10T11:48:41.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.509 7fbed13fd700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbecc109a60 con 0x7fbecc0cff70 2021-02-10T11:48:41.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.509 7fbed13fd700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbecc088070 con 0x7fbecc0d1870 2021-02-10T11:48:41.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.509 7fbecb07d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbecc0d1870 0x7fbecc1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:41.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.509 7fbeca87c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fbecc1b0ae0 0x7fbecc108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35678/0 (socket says 172.21.2.101:35678) 2021-02-10T11:48:41.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.509 7fbeca87c700 1 -- 172.21.2.101:0/1451957315 learned_addr learned my addr 172.21.2.101:0/1451957315 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:41.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.509 7fbecb07d700 1 -- 172.21.2.101:0/1451957315 >> v1:172.21.2.108:6789/0 conn(0x7fbecc0cff70 legacy=0x7fbecc109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:41.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.509 7fbecb07d700 1 -- 172.21.2.101:0/1451957315 >> v1:172.21.2.101:6789/0 conn(0x7fbecc1b0ae0 legacy=0x7fbecc108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:41.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.509 7fbecb07d700 1 -- 172.21.2.101:0/1451957315 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc088260 con 0x7fbecc0d1870 2021-02-10T11:48:41.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.509 7fbecb07d700 1 --2- 172.21.2.101:0/1451957315 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbecc0d1870 0x7fbecc1b04c0 secure :-1 s=READY pgs=3280 cs=0 l=1 rev1=1 rx=0x7fbeb4006650 tx=0x7fbeb4009a80).ready entity=mon.0 client_cookie=88c52d486f2d884b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:41.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.510 7fbeca07b700 1 -- 172.21.2.101:0/1451957315 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbeb40009b0 con 0x7fbecc0d1870 2021-02-10T11:48:41.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.510 7fbeca07b700 1 -- 172.21.2.101:0/1451957315 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb400c0c0 con 0x7fbecc0d1870 2021-02-10T11:48:41.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.510 7fbeca07b700 1 -- 172.21.2.101:0/1451957315 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbeb400c5e0 con 0x7fbecc0d1870 2021-02-10T11:48:41.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.510 7fbed13fd700 1 -- 172.21.2.101:0/1451957315 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbecc0d1870 msgr2=0x7fbecc1b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:41.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.510 7fbed13fd700 1 --2- 172.21.2.101:0/1451957315 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbecc0d1870 0x7fbecc1b04c0 secure :-1 s=READY pgs=3280 cs=0 l=1 rev1=1 rx=0x7fbeb4006650 tx=0x7fbeb4009a80).stop 2021-02-10T11:48:41.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.510 7fbed13fd700 1 -- 172.21.2.101:0/1451957315 shutdown_connections 2021-02-10T11:48:41.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.510 7fbed13fd700 1 --2- 172.21.2.101:0/1451957315 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbecc0d1870 0x7fbecc1b04c0 unknown :-1 s=CLOSED pgs=3280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:41.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.510 7fbed13fd700 1 -- 172.21.2.101:0/1451957315 >> 172.21.2.101:0/1451957315 conn(0x7fbecc0ce420 msgr2=0x7fbecc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:41.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.510 7fbed13fd700 1 -- 172.21.2.101:0/1451957315 shutdown_connections 2021-02-10T11:48:41.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.510 7fbed13fd700 1 -- 172.21.2.101:0/1451957315 wait complete. 2021-02-10T11:48:41.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.525 7fbed13fd700 1 Processor -- start 2021-02-10T11:48:41.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.525 7fbed13fd700 1 -- start start 2021-02-10T11:48:41.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.525 7fbed13fd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbecc100430 0x7fbecc0b9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:41.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.525 7fbed13fd700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbecc091f30 0x7fbecc092330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:41.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.525 7fbed13fd700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbecc1073b0 0x7fbecc1077b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:41.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.525 7fbed13fd700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc0b99d0 con 0x7fbecc100430 2021-02-10T11:48:41.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.525 7fbed13fd700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc0919b0 con 0x7fbecc1073b0 2021-02-10T11:48:41.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.525 7fbed13fd700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc0b9670 con 0x7fbecc091f30 2021-02-10T11:48:41.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.525 7fbecb07d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbecc100430 0x7fbecc0b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:41.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.525 7fbecb07d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbecc100430 0x7fbecc0b9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35190/0 (socket says 172.21.2.101:35190) 2021-02-10T11:48:41.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.525 7fbecb07d700 1 -- 172.21.2.101:0/2797741076 learned_addr learned my addr 172.21.2.101:0/2797741076 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:41.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.526 7fbecb07d700 1 -- 172.21.2.101:0/2797741076 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbecc091f30 msgr2=0x7fbecc092330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:41.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.526 7fbecb07d700 1 --2- 172.21.2.101:0/2797741076 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbecc091f30 0x7fbecc092330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:41.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.526 7fbecb07d700 1 -- 172.21.2.101:0/2797741076 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbecc1073b0 msgr2=0x7fbecc1077b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:41.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.526 7fbecb07d700 1 --2- 172.21.2.101:0/2797741076 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbecc1073b0 0x7fbecc1077b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:41.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.526 7fbecb07d700 1 -- 172.21.2.101:0/2797741076 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc107dd0 con 0x7fbecc100430 2021-02-10T11:48:41.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.526 7fbecb07d700 1 --2- 172.21.2.101:0/2797741076 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbecc100430 0x7fbecc0b9050 secure :-1 s=READY pgs=3281 cs=0 l=1 rev1=1 rx=0x7fbeb4009630 tx=0x7fbeb400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:41.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.526 7fbec3fff700 1 -- 172.21.2.101:0/2797741076 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbeb40093a0 con 0x7fbecc100430 2021-02-10T11:48:41.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.526 7fbec3fff700 1 -- 172.21.2.101:0/2797741076 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb400bd70 con 0x7fbecc100430 2021-02-10T11:48:41.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.526 7fbed13fd700 1 -- 172.21.2.101:0/2797741076 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbecc107dd0 con 0x7fbecc100430 2021-02-10T11:48:41.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.526 7fbec3fff700 1 -- 172.21.2.101:0/2797741076 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbeb4005560 con 0x7fbecc100430 2021-02-10T11:48:41.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.526 7fbed13fd700 1 -- 172.21.2.101:0/2797741076 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbecc108360 con 0x7fbecc100430 2021-02-10T11:48:41.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.527 7fbec3fff700 1 -- 172.21.2.101:0/2797741076 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbeb4007020 con 0x7fbecc100430 2021-02-10T11:48:41.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.527 7fbec3fff700 1 --2- 172.21.2.101:0/2797741076 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbeb00186c0 0x7fbeb001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:41.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.527 7fbec3fff700 1 -- 172.21.2.101:0/2797741076 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fbeb40323c0 con 0x7fbecc100430 2021-02-10T11:48:41.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.527 7fbeca87c700 1 --2- 172.21.2.101:0/2797741076 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbeb00186c0 0x7fbeb001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:41.526 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.528 7fbeca87c700 1 --2- 172.21.2.101:0/2797741076 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbeb00186c0 0x7fbeb001abc0 crc :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:41.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.559 7fbed13fd700 1 -- 172.21.2.101:0/2797741076 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbecc080780 con 0x7fbecc100430 2021-02-10T11:48:41.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.561 7fbec3fff700 1 -- 172.21.2.101:0/2797741076 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbeb402f050 con 0x7fbecc100430 2021-02-10T11:48:41.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.689 7fbed13fd700 1 -- 172.21.2.101:0/2797741076 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7fbecc0ce420 con 0x7fbecc100430 2021-02-10T11:48:41.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.690 7fbec3fff700 1 -- 172.21.2.101:0/2797741076 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v191) v1 ==== 100+0+10 (secure 0 0 0) 0x7fbeb402f050 con 0x7fbecc100430 2021-02-10T11:48:41.687 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:48:41.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.691 7fbed13fd700 1 -- 172.21.2.101:0/2797741076 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbeb00186c0 msgr2=0x7fbeb001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:41.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.691 7fbed13fd700 1 --2- 172.21.2.101:0/2797741076 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbeb00186c0 0x7fbeb001abc0 crc :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:41.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.692 7fbed13fd700 1 -- 172.21.2.101:0/2797741076 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbecc100430 msgr2=0x7fbecc0b9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:41.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.692 7fbed13fd700 1 --2- 172.21.2.101:0/2797741076 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbecc100430 0x7fbecc0b9050 secure :-1 s=READY pgs=3281 cs=0 l=1 rev1=1 rx=0x7fbeb4009630 tx=0x7fbeb400b980).stop 2021-02-10T11:48:41.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.692 7fbed13fd700 1 -- 172.21.2.101:0/2797741076 shutdown_connections 2021-02-10T11:48:41.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.692 7fbed13fd700 1 --2- 172.21.2.101:0/2797741076 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbeb00186c0 0x7fbeb001abc0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:41.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.692 7fbed13fd700 1 --2- 172.21.2.101:0/2797741076 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbecc091f30 0x7fbecc092330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:41.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.692 7fbed13fd700 1 --2- 172.21.2.101:0/2797741076 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbecc1073b0 0x7fbecc1077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:41.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.692 7fbed13fd700 1 --2- 172.21.2.101:0/2797741076 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbecc100430 0x7fbecc0b9050 unknown :-1 s=CLOSED pgs=3281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:41.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.692 7fbed13fd700 1 -- 172.21.2.101:0/2797741076 >> 172.21.2.101:0/2797741076 conn(0x7fbecc0bcff0 msgr2=0x7fbecc08e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:41.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.692 7fbed13fd700 1 -- 172.21.2.101:0/2797741076 shutdown_connections 2021-02-10T11:48:41.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:41.692 7fbed13fd700 1 -- 172.21.2.101:0/2797741076 wait complete. 2021-02-10T11:48:41.696 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:48:41.697 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:48:42.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.111 7fb590d03700 1 Processor -- start 2021-02-10T11:48:42.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.111 7fb590d03700 1 -- start start 2021-02-10T11:48:42.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.112 7fb590d03700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb58c0d1870 0x7fb58c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:42.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.112 7fb590d03700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb58c0f0650 0x7fb58c0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:42.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.112 7fb590d03700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb58c0f0b10 con 0x7fb58c1b0a10 2021-02-10T11:48:42.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.112 7fb590d03700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb58c0e5ed0 con 0x7fb58c0d1870 2021-02-10T11:48:42.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.112 7fb590d03700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb58c0d1c70 con 0x7fb58c0f0650 2021-02-10T11:48:42.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.112 7fb58ab13700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb58c0d1870 0x7fb58c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:42.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.112 7fb58a312700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb58c1b0a10 0x7fb58c0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35692/0 (socket says 172.21.2.101:35692) 2021-02-10T11:48:42.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.112 7fb58ab13700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb58c0d1870 0x7fb58c1b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35198/0 (socket says 172.21.2.101:35198) 2021-02-10T11:48:42.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.112 7fb58ab13700 1 -- 172.21.2.101:0/2911521375 learned_addr learned my addr 172.21.2.101:0/2911521375 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:42.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.112 7fb58ab13700 1 -- 172.21.2.101:0/2911521375 >> v2:172.21.2.108:3300/0 conn(0x7fb58c0f0650 msgr2=0x7fb58c0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:42.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.112 7fb589b11700 1 -- 172.21.2.101:0/2911521375 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2492051492 0 0) 0x7fb57c001100 con 0x7fb58c1b0a10 2021-02-10T11:48:42.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.112 7fb58ab13700 1 --2- 172.21.2.101:0/2911521375 >> v2:172.21.2.108:3300/0 conn(0x7fb58c0f0650 0x7fb58c0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:42.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.112 7fb58ab13700 1 -- 172.21.2.101:0/2911521375 >> v1:172.21.2.101:6789/0 conn(0x7fb58c1b0a10 legacy=0x7fb58c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:42.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.113 7fb58ab13700 1 -- 172.21.2.101:0/2911521375 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58c0d1e90 con 0x7fb58c0d1870 2021-02-10T11:48:42.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.113 7fb58ab13700 1 --2- 172.21.2.101:0/2911521375 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb58c0d1870 0x7fb58c1b03f0 secure :-1 s=READY pgs=3283 cs=0 l=1 rev1=1 rx=0x7fb574006700 tx=0x7fb574009b80).ready entity=mon.0 client_cookie=da43cbacc8d5d762 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:42.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.113 7fb589b11700 1 -- 172.21.2.101:0/2911521375 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb57400c0c0 con 0x7fb58c0d1870 2021-02-10T11:48:42.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.113 7fb589b11700 1 -- 172.21.2.101:0/2911521375 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57400c440 con 0x7fb58c0d1870 2021-02-10T11:48:42.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.113 7fb589b11700 1 -- 172.21.2.101:0/2911521375 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb57400c960 con 0x7fb58c0d1870 2021-02-10T11:48:42.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.113 7fb590d03700 1 -- 172.21.2.101:0/2911521375 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb58c0d1870 msgr2=0x7fb58c1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:42.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.113 7fb590d03700 1 --2- 172.21.2.101:0/2911521375 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb58c0d1870 0x7fb58c1b03f0 secure :-1 s=READY pgs=3283 cs=0 l=1 rev1=1 rx=0x7fb574006700 tx=0x7fb574009b80).stop 2021-02-10T11:48:42.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.114 7fb590d03700 1 -- 172.21.2.101:0/2911521375 shutdown_connections 2021-02-10T11:48:42.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.114 7fb590d03700 1 --2- 172.21.2.101:0/2911521375 >> v2:172.21.2.108:3300/0 conn(0x7fb58c0f0650 0x7fb58c0cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:42.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.114 7fb590d03700 1 --2- 172.21.2.101:0/2911521375 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb58c0d1870 0x7fb58c1b03f0 unknown :-1 s=CLOSED pgs=3283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:42.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.114 7fb590d03700 1 -- 172.21.2.101:0/2911521375 >> 172.21.2.101:0/2911521375 conn(0x7fb58c0ce450 msgr2=0x7fb58c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:42.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.114 7fb590d03700 1 -- 172.21.2.101:0/2911521375 shutdown_connections 2021-02-10T11:48:42.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.114 7fb590d03700 1 -- 172.21.2.101:0/2911521375 wait complete. 2021-02-10T11:48:42.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.128 7fb590d03700 1 Processor -- start 2021-02-10T11:48:42.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.128 7fb590d03700 1 -- start start 2021-02-10T11:48:42.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.129 7fb590d03700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb58c1b7cf0 0x7fb58c19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:42.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.129 7fb590d03700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb58c1b80f0 0x7fb58c1b84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:42.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.129 7fb590d03700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb58c19da20 0x7fb58c19de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:42.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.129 7fb590d03700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c0a77c0 con 0x7fb58c1b7cf0 2021-02-10T11:48:42.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.129 7fb590d03700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c19d4a0 con 0x7fb58c1b80f0 2021-02-10T11:48:42.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.129 7fb590d03700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c0954f0 con 0x7fb58c19da20 2021-02-10T11:48:42.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.129 7fb58ab13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb58c1b7cf0 0x7fb58c19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:42.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.129 7fb58ab13700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb58c1b7cf0 0x7fb58c19cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35204/0 (socket says 172.21.2.101:35204) 2021-02-10T11:48:42.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.129 7fb58ab13700 1 -- 172.21.2.101:0/3944466789 learned_addr learned my addr 172.21.2.101:0/3944466789 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:42.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.129 7fb58a312700 1 --2- 172.21.2.101:0/3944466789 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb58c1b80f0 0x7fb58c1b84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:42.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.129 7fb58b314700 1 --2- 172.21.2.101:0/3944466789 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb58c19da20 0x7fb58c19de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:42.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.129 7fb58ab13700 1 -- 172.21.2.101:0/3944466789 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb58c19da20 msgr2=0x7fb58c19de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:42.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.129 7fb58ab13700 1 --2- 172.21.2.101:0/3944466789 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb58c19da20 0x7fb58c19de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:42.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.129 7fb58ab13700 1 -- 172.21.2.101:0/3944466789 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb58c1b80f0 msgr2=0x7fb58c1b84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:42.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.129 7fb58ab13700 1 --2- 172.21.2.101:0/3944466789 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb58c1b80f0 0x7fb58c1b84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:42.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.129 7fb58ab13700 1 -- 172.21.2.101:0/3944466789 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58c1b8d60 con 0x7fb58c1b7cf0 2021-02-10T11:48:42.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.129 7fb58a312700 1 --2- 172.21.2.101:0/3944466789 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb58c1b80f0 0x7fb58c1b84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:48:42.129 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.129 7fb58b314700 1 --2- 172.21.2.101:0/3944466789 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb58c19da20 0x7fb58c19de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:48:42.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.130 7fb58ab13700 1 --2- 172.21.2.101:0/3944466789 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb58c1b7cf0 0x7fb58c19cdb0 secure :-1 s=READY pgs=3284 cs=0 l=1 rev1=1 rx=0x7fb57400b5e0 tx=0x7fb57400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:42.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.130 7fb5837fe700 1 -- 172.21.2.101:0/3944466789 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb57400bd70 con 0x7fb58c1b7cf0 2021-02-10T11:48:42.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.130 7fb5837fe700 1 -- 172.21.2.101:0/3944466789 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb574002030 con 0x7fb58c1b7cf0 2021-02-10T11:48:42.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.130 7fb5837fe700 1 -- 172.21.2.101:0/3944466789 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb574002550 con 0x7fb58c1b7cf0 2021-02-10T11:48:42.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.130 7fb590d03700 1 -- 172.21.2.101:0/3944466789 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb58c1b8d60 con 0x7fb58c1b7cf0 2021-02-10T11:48:42.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.130 7fb590d03700 1 -- 172.21.2.101:0/3944466789 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb58c19e730 con 0x7fb58c1b7cf0 2021-02-10T11:48:42.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.130 7fb5837fe700 1 -- 172.21.2.101:0/3944466789 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb574000c90 con 0x7fb58c1b7cf0 2021-02-10T11:48:42.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.131 7fb5837fe700 1 --2- 172.21.2.101:0/3944466789 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5700186c0 0x7fb57001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:42.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.131 7fb5837fe700 1 -- 172.21.2.101:0/3944466789 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb5740325e0 con 0x7fb58c1b7cf0 2021-02-10T11:48:42.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.131 7fb58a312700 1 --2- 172.21.2.101:0/3944466789 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5700186c0 0x7fb57001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:42.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.131 7fb58a312700 1 --2- 172.21.2.101:0/3944466789 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5700186c0 0x7fb57001abc0 crc :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:42.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.161 7fb590d03700 1 -- 172.21.2.101:0/3944466789 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb58c0ebc00 con 0x7fb58c1b7cf0 2021-02-10T11:48:42.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.164 7fb5837fe700 1 -- 172.21.2.101:0/3944466789 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb5740009b0 con 0x7fb58c1b7cf0 2021-02-10T11:48:42.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.291 7fb590d03700 1 -- 172.21.2.101:0/3944466789 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb58c0f0b10 con 0x7fb58c1b7cf0 2021-02-10T11:48:42.287 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.291 7fb5837fe700 1 -- 172.21.2.101:0/3944466789 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) v1 ==== 95+0+2108 (secure 0 0 0) 0x7fb574000c90 con 0x7fb58c1b7cf0 2021-02-10T11:48:42.288 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:48:42.288 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":164,"default_fscid":20,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":13530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/347510536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":347510536},{"type":"v1","addr":"172.21.2.108:6833","nonce":347510536}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":160}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:48:24.824003","modified":"2021-02-10 11:48:26.829128","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":13518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13518":{"gid":13518,"name":"a","rank":0,"incarnation":161,"state":"up:active","state_seq":3,"addr":"172.21.2.101:6835/2721786668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2721786668},{"type":"v1","addr":"172.21.2.101:6835","nonce":2721786668}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[41],"metadata_pool":40,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":20}]} 2021-02-10T11:48:42.289 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 164 2021-02-10T11:48:42.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.293 7fb590d03700 1 -- 172.21.2.101:0/3944466789 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5700186c0 msgr2=0x7fb57001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:42.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.293 7fb590d03700 1 --2- 172.21.2.101:0/3944466789 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5700186c0 0x7fb57001abc0 crc :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:42.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.293 7fb590d03700 1 -- 172.21.2.101:0/3944466789 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb58c1b7cf0 msgr2=0x7fb58c19cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:42.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.293 7fb590d03700 1 --2- 172.21.2.101:0/3944466789 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb58c1b7cf0 0x7fb58c19cdb0 secure :-1 s=READY pgs=3284 cs=0 l=1 rev1=1 rx=0x7fb57400b5e0 tx=0x7fb57400b980).stop 2021-02-10T11:48:42.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.293 7fb590d03700 1 -- 172.21.2.101:0/3944466789 shutdown_connections 2021-02-10T11:48:42.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.293 7fb590d03700 1 --2- 172.21.2.101:0/3944466789 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb5700186c0 0x7fb57001abc0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:42.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.294 7fb590d03700 1 --2- 172.21.2.101:0/3944466789 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb58c19da20 0x7fb58c19de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:42.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.294 7fb590d03700 1 --2- 172.21.2.101:0/3944466789 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb58c1b80f0 0x7fb58c1b84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:42.290 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.294 7fb590d03700 1 --2- 172.21.2.101:0/3944466789 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb58c1b7cf0 0x7fb58c19cdb0 unknown :-1 s=CLOSED pgs=3284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:42.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.294 7fb590d03700 1 -- 172.21.2.101:0/3944466789 >> 172.21.2.101:0/3944466789 conn(0x7fb58c0ed810 msgr2=0x7fb58c094980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:42.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.294 7fb590d03700 1 -- 172.21.2.101:0/3944466789 shutdown_connections 2021-02-10T11:48:42.291 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:42.294 7fb590d03700 1 -- 172.21.2.101:0/3944466789 wait complete. 2021-02-10T11:48:42.499 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 164, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:48:24.824003', 'modified': '2021-02-10 11:48:26.829128', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 13518}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_13518': {'gid': 13518, 'name': 'a', 'rank': 0, 'incarnation': 161, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.101:6835/2721786668', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.101:6834', 'nonce': 2721786668}, {'type': 'v1', 'addr': '172.21.2.101:6835', 'nonce': 2721786668}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [41], 'metadata_pool': 40, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:48:42.499 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:48:42.500 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok status 2021-02-10T11:48:42.902 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:48:42.903 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:48:42.903 INFO:teuthology.orchestra.run.gibba001.stdout: "whoami": 0, 2021-02-10T11:48:42.903 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 13518, 2021-02-10T11:48:42.903 INFO:teuthology.orchestra.run.gibba001.stdout: "want_state": "up:active", 2021-02-10T11:48:42.903 INFO:teuthology.orchestra.run.gibba001.stdout: "state": "up:active", 2021-02-10T11:48:42.903 INFO:teuthology.orchestra.run.gibba001.stdout: "rank_uptime": 18.081358019, 2021-02-10T11:48:42.903 INFO:teuthology.orchestra.run.gibba001.stdout: "mdsmap_epoch": 164, 2021-02-10T11:48:42.903 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch": 191, 2021-02-10T11:48:42.903 INFO:teuthology.orchestra.run.gibba001.stdout: "osdmap_epoch_barrier": 191, 2021-02-10T11:48:42.903 INFO:teuthology.orchestra.run.gibba001.stdout: "uptime": 23.739607488000001 2021-02-10T11:48:42.903 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:48:42.907 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 13518, "want_state": "up:active", "state": "up:active", "rank_uptime": 18.081358019, "mdsmap_epoch": 164, "osdmap_epoch": 191, "osdmap_epoch_barrier": 191, "uptime": 23.739607488000001 } 2021-02-10T11:48:42.907 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:48:42.907 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-02-10T11:48:42.907 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:48:42.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:42.908 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:48:43.244 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:48:43.588 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:48:43.588 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:48:43.932 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:48:43.932 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:48:43.933 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:48:43.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:43.934 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:48:44.268 INFO:teuthology.orchestra.run.gibba001.stdout:42 2021-02-10T11:48:44.269 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2021-02-10T11:48:44.269 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:48:44.272 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:48:44.615 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:48:44.616 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:48:44.683 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[182770]: starting ceph client 2021-02-10T11:48:44.683 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:2021-02-10 11:48:44.686 7f4d2717af80 -1 init, newargv = 0x55ad3219f1c0 newargc=9 2021-02-10T11:48:44.689 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[182770]: starting fuse 2021-02-10T11:48:44.959 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:48:44.959 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:48:44.960 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:48:44.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:44.960 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:48:45.294 INFO:teuthology.orchestra.run.gibba001.stdout:42 2021-02-10T11:48:45.294 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:48:45.295 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2021-02-10T11:48:45.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:45.295 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:48:45.295 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:48:45.295 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:48:45.295 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:48:45.295 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:48:45.295 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:48:45.295 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:48:45.295 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-02-10T11:48:45.296 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:48:45.296 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:48:45.296 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:48:45.296 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:48:45.296 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:48:45.296 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:48:45.296 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:48:45.296 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:48:45.296 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:48:45.296 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:48:45.296 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:48:45.296 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:48:45.296 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:48:45.296 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.1")) 2021-02-10T11:48:45.296 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:48:45.660 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.1.182770.asok 2021-02-10T11:48:45.663 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.182770.asok 2021-02-10T11:48:45.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:45.663 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.182770.asok status 2021-02-10T11:48:46.056 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:48:46.056 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:48:46.056 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:48:46.056 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:48:46.056 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "1", 2021-02-10T11:48:46.056 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:48:46.056 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:48:46.056 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "182770", 2021-02-10T11:48:46.057 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:48:46.057 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:46.057 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:48:46.057 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:48:46.057 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 13719, 2021-02-10T11:48:46.057 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:48:46.057 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:48:46.057 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:48:46.057 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 13719 2021-02-10T11:48:46.057 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:46.057 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:48:46.057 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:46.057 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:48:46.058 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 4240191817 2021-02-10T11:48:46.058 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:46.058 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:46.058 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:48:46.058 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:48:46.058 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:48:46.058 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 4240191817 2021-02-10T11:48:46.058 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:46.058 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.13719 172.21.2.101:0/4240191817", 2021-02-10T11:48:46.058 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/4240191817", 2021-02-10T11:48:46.058 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:48:46.058 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 164, 2021-02-10T11:48:46.058 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 191, 2021-02-10T11:48:46.059 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:48:46.059 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:48:46.059 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:48:46.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:46.060 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:48:46.060 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:48:46.396 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:48:46.396 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:48:46.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:46.397 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:48:46.743 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2021-02-10T11:48:47.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.162 7ff1f42d6700 1 Processor -- start 2021-02-10T11:48:47.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.162 7ff1f42d6700 1 -- start start 2021-02-10T11:48:47.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.163 7ff1f42d6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff1ec0d1880 0x7ff1ec1b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:47.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.163 7ff1f42d6700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff1ec1b0af0 0x7ff1ec108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:47.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.163 7ff1f42d6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ec0fe1d0 con 0x7ff1ec0d1c80 2021-02-10T11:48:47.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.163 7ff1f42d6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ec109a70 con 0x7ff1ec0d1880 2021-02-10T11:48:47.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.163 7ff1f42d6700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ec109760 con 0x7ff1ec1b0af0 2021-02-10T11:48:47.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.163 7ff1f20f2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff1ec0d1880 0x7ff1ec1b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:47.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.163 7ff1f20f2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff1ec0d1880 0x7ff1ec1b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35224/0 (socket says 172.21.2.101:35224) 2021-02-10T11:48:47.175 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.163 7ff1f20f2700 1 -- 172.21.2.101:0/3553809132 learned_addr learned my addr 172.21.2.101:0/3553809132 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:47.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.163 7ff1f20f2700 1 -- 172.21.2.101:0/3553809132 >> v2:172.21.2.108:3300/0 conn(0x7ff1ec1b0af0 msgr2=0x7ff1ec108950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:48:47.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.163 7ff1f20f2700 1 --2- 172.21.2.101:0/3553809132 >> v2:172.21.2.108:3300/0 conn(0x7ff1ec1b0af0 0x7ff1ec108950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:47.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.163 7ff1f20f2700 1 -- 172.21.2.101:0/3553809132 >> v1:172.21.2.108:6789/0 conn(0x7ff1ec0d1c80 legacy=0x7ff1ec108f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:48:47.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.163 7ff1f20f2700 1 -- 172.21.2.101:0/3553809132 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec088070 con 0x7ff1ec0d1880 2021-02-10T11:48:47.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.163 7ff1f20f2700 1 --2- 172.21.2.101:0/3553809132 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec0d1880 0x7ff1ec1b04d0 secure :-1 s=READY pgs=3287 cs=0 l=1 rev1=1 rx=0x7ff1e8006700 tx=0x7ff1e8009b00).ready entity=mon.0 client_cookie=cc2fa66767f23df server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:47.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.164 7ff1f10f0700 1 -- 172.21.2.101:0/3553809132 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1e800c0c0 con 0x7ff1ec0d1880 2021-02-10T11:48:47.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.164 7ff1f10f0700 1 -- 172.21.2.101:0/3553809132 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e800c400 con 0x7ff1ec0d1880 2021-02-10T11:48:47.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.164 7ff1f10f0700 1 -- 172.21.2.101:0/3553809132 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1e800c920 con 0x7ff1ec0d1880 2021-02-10T11:48:47.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.164 7ff1f42d6700 1 -- 172.21.2.101:0/3553809132 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec0d1880 msgr2=0x7ff1ec1b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:47.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.164 7ff1f42d6700 1 --2- 172.21.2.101:0/3553809132 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec0d1880 0x7ff1ec1b04d0 secure :-1 s=READY pgs=3287 cs=0 l=1 rev1=1 rx=0x7ff1e8006700 tx=0x7ff1e8009b00).stop 2021-02-10T11:48:47.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.164 7ff1f42d6700 1 -- 172.21.2.101:0/3553809132 shutdown_connections 2021-02-10T11:48:47.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.164 7ff1f42d6700 1 --2- 172.21.2.101:0/3553809132 >> v2:172.21.2.108:3300/0 conn(0x7ff1ec1b0af0 0x7ff1ec108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:47.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.164 7ff1f42d6700 1 --2- 172.21.2.101:0/3553809132 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec0d1880 0x7ff1ec1b04d0 unknown :-1 s=CLOSED pgs=3287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:47.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.164 7ff1f42d6700 1 -- 172.21.2.101:0/3553809132 >> 172.21.2.101:0/3553809132 conn(0x7ff1ec0ce430 msgr2=0x7ff1ec0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:47.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.164 7ff1f42d6700 1 -- 172.21.2.101:0/3553809132 shutdown_connections 2021-02-10T11:48:47.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.165 7ff1f42d6700 1 -- 172.21.2.101:0/3553809132 wait complete. 2021-02-10T11:48:47.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.179 7ff1f42d6700 1 Processor -- start 2021-02-10T11:48:47.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.179 7ff1f42d6700 1 -- start start 2021-02-10T11:48:47.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.179 7ff1f42d6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1ec19e5b0 0x7ff1ec1afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:47.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.179 7ff1f42d6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec19e9b0 0x7ff1ec1b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:47.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.179 7ff1f42d6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ec1b0250 0x7ff1ec1b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:47.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.179 7ff1f42d6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec195bd0 con 0x7ff1ec19e9b0 2021-02-10T11:48:47.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.179 7ff1f42d6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec1b0840 con 0x7ff1ec19e5b0 2021-02-10T11:48:47.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.179 7ff1f42d6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec19eed0 con 0x7ff1ec1b0250 2021-02-10T11:48:47.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.179 7ff1f18f1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec19e9b0 0x7ff1ec1b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:47.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.179 7ff1f18f1700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec19e9b0 0x7ff1ec1b0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35230/0 (socket says 172.21.2.101:35230) 2021-02-10T11:48:47.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.179 7ff1f18f1700 1 -- 172.21.2.101:0/1190286770 learned_addr learned my addr 172.21.2.101:0/1190286770 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:47.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.180 7ff1f18f1700 1 -- 172.21.2.101:0/1190286770 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ec1b0250 msgr2=0x7ff1ec1b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:47.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.180 7ff1f18f1700 1 --2- 172.21.2.101:0/1190286770 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ec1b0250 0x7ff1ec1b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:47.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.180 7ff1f18f1700 1 -- 172.21.2.101:0/1190286770 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1ec19e5b0 msgr2=0x7ff1ec1afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:47.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.180 7ff1f18f1700 1 --2- 172.21.2.101:0/1190286770 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1ec19e5b0 0x7ff1ec1afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:47.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.180 7ff1f18f1700 1 -- 172.21.2.101:0/1190286770 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec19f0b0 con 0x7ff1ec19e9b0 2021-02-10T11:48:47.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.180 7ff1f18f1700 1 --2- 172.21.2.101:0/1190286770 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec19e9b0 0x7ff1ec1b0dc0 secure :-1 s=READY pgs=3288 cs=0 l=1 rev1=1 rx=0x7ff1e000b5c0 tx=0x7ff1e000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:47.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.180 7ff1deffd700 1 -- 172.21.2.101:0/1190286770 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1e000bd50 con 0x7ff1ec19e9b0 2021-02-10T11:48:47.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.180 7ff1deffd700 1 -- 172.21.2.101:0/1190286770 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e0006030 con 0x7ff1ec19e9b0 2021-02-10T11:48:47.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.180 7ff1deffd700 1 -- 172.21.2.101:0/1190286770 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1e00054d0 con 0x7ff1ec19e9b0 2021-02-10T11:48:47.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.180 7ff1f42d6700 1 -- 172.21.2.101:0/1190286770 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ec19f0b0 con 0x7ff1ec19e9b0 2021-02-10T11:48:47.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.180 7ff1f42d6700 1 -- 172.21.2.101:0/1190286770 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec19f0b0 con 0x7ff1ec19e9b0 2021-02-10T11:48:47.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.181 7ff1deffd700 1 -- 172.21.2.101:0/1190286770 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff1e0006210 con 0x7ff1ec19e9b0 2021-02-10T11:48:47.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.181 7ff1deffd700 1 --2- 172.21.2.101:0/1190286770 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1d40186d0 0x7ff1d401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:47.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.181 7ff1deffd700 1 -- 172.21.2.101:0/1190286770 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff1e0032800 con 0x7ff1ec19e9b0 2021-02-10T11:48:47.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.181 7ff1f20f2700 1 --2- 172.21.2.101:0/1190286770 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1d40186d0 0x7ff1d401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:47.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.181 7ff1f20f2700 1 --2- 172.21.2.101:0/1190286770 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1d40186d0 0x7ff1d401abd0 crc :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:47.209 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.213 7ff1f42d6700 1 -- 172.21.2.101:0/1190286770 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ec19f0b0 con 0x7ff1ec19e9b0 2021-02-10T11:48:47.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.215 7ff1deffd700 1 -- 172.21.2.101:0/1190286770 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff1e0000c40 con 0x7ff1ec19e9b0 2021-02-10T11:48:47.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.351 7ff1f42d6700 1 -- 172.21.2.101:0/1190286770 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff1ec19f0b0 con 0x7ff1ec19e9b0 2021-02-10T11:48:47.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.668 7ff1deffd700 1 -- 172.21.2.101:0/1190286770 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v165) v1 ==== 136+0+0 (secure 0 0 0) 0x7ff1e0006030 con 0x7ff1ec19e9b0 2021-02-10T11:48:47.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.719 7ff1f42d6700 1 -- 172.21.2.101:0/1190286770 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1d40186d0 msgr2=0x7ff1d401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:47.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.719 7ff1f42d6700 1 --2- 172.21.2.101:0/1190286770 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1d40186d0 0x7ff1d401abd0 crc :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:47.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.719 7ff1f42d6700 1 -- 172.21.2.101:0/1190286770 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec19e9b0 msgr2=0x7ff1ec1b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:47.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.719 7ff1f42d6700 1 --2- 172.21.2.101:0/1190286770 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec19e9b0 0x7ff1ec1b0dc0 secure :-1 s=READY pgs=3288 cs=0 l=1 rev1=1 rx=0x7ff1e000b5c0 tx=0x7ff1e000b960).stop 2021-02-10T11:48:47.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.719 7ff1f42d6700 1 -- 172.21.2.101:0/1190286770 shutdown_connections 2021-02-10T11:48:47.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.719 7ff1f42d6700 1 --2- 172.21.2.101:0/1190286770 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff1d40186d0 0x7ff1d401abd0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:47.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.719 7ff1f42d6700 1 --2- 172.21.2.101:0/1190286770 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1ec1b0250 0x7ff1ec1b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:47.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.719 7ff1f42d6700 1 --2- 172.21.2.101:0/1190286770 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1ec19e9b0 0x7ff1ec1b0dc0 unknown :-1 s=CLOSED pgs=3288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:47.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.719 7ff1f42d6700 1 --2- 172.21.2.101:0/1190286770 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1ec19e5b0 0x7ff1ec1afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:47.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.719 7ff1f42d6700 1 -- 172.21.2.101:0/1190286770 >> 172.21.2.101:0/1190286770 conn(0x7ff1ec19f940 msgr2=0x7ff1ec18c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:47.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.719 7ff1f42d6700 1 -- 172.21.2.101:0/1190286770 shutdown_connections 2021-02-10T11:48:47.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:47.719 7ff1f42d6700 1 -- 172.21.2.101:0/1190286770 wait complete. 2021-02-10T11:48:47.723 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs volume ls 2021-02-10T11:48:48.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.140 7f19cfdf6700 1 Processor -- start 2021-02-10T11:48:48.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.140 7f19cfdf6700 1 -- start start 2021-02-10T11:48:48.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.140 7f19cfdf6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f19c80d1860 0x7f19c81b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:48.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.140 7f19cfdf6700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f19c81b0a00 0x7f19c80cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:48.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.140 7f19cfdf6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c80f0b00 con 0x7f19c80d1c60 2021-02-10T11:48:48.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.140 7f19cfdf6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c80e5ec0 con 0x7f19c80d1860 2021-02-10T11:48:48.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.140 7f19cfdf6700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c80f0640 con 0x7f19c81b0a00 2021-02-10T11:48:48.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.140 7f19cdc12700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f19c80d1860 0x7f19c81b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:48.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.140 7f19cdc12700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f19c80d1860 0x7f19c81b03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35236/0 (socket says 172.21.2.101:35236) 2021-02-10T11:48:48.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.140 7f19cdc12700 1 -- 172.21.2.101:0/227456778 learned_addr learned my addr 172.21.2.101:0/227456778 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:48.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.141 7f19cdc12700 1 -- 172.21.2.101:0/227456778 >> v2:172.21.2.108:3300/0 conn(0x7f19c81b0a00 msgr2=0x7f19c80cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:48.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.141 7f19cdc12700 1 --2- 172.21.2.101:0/227456778 >> v2:172.21.2.108:3300/0 conn(0x7f19c81b0a00 0x7f19c80cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:48.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.141 7f19cdc12700 1 -- 172.21.2.101:0/227456778 >> v1:172.21.2.108:6789/0 conn(0x7f19c80d1c60 legacy=0x7f19c80cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:48.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.141 7f19cdc12700 1 -- 172.21.2.101:0/227456778 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c80f0830 con 0x7f19c80d1860 2021-02-10T11:48:48.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.141 7f19cdc12700 1 --2- 172.21.2.101:0/227456778 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19c80d1860 0x7f19c81b03e0 secure :-1 s=READY pgs=3289 cs=0 l=1 rev1=1 rx=0x7f19c4006700 tx=0x7f19c4009b80).ready entity=mon.0 client_cookie=30b85e9cedb665a8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:48.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.141 7f19ccc10700 1 -- 172.21.2.101:0/227456778 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19c40009b0 con 0x7f19c80d1860 2021-02-10T11:48:48.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.141 7f19ccc10700 1 -- 172.21.2.101:0/227456778 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c400c0c0 con 0x7f19c80d1860 2021-02-10T11:48:48.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.141 7f19ccc10700 1 -- 172.21.2.101:0/227456778 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19c400c510 con 0x7f19c80d1860 2021-02-10T11:48:48.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.141 7f19cfdf6700 1 -- 172.21.2.101:0/227456778 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19c80d1860 msgr2=0x7f19c81b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:48.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.141 7f19cfdf6700 1 --2- 172.21.2.101:0/227456778 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19c80d1860 0x7f19c81b03e0 secure :-1 s=READY pgs=3289 cs=0 l=1 rev1=1 rx=0x7f19c4006700 tx=0x7f19c4009b80).stop 2021-02-10T11:48:48.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.141 7f19cfdf6700 1 -- 172.21.2.101:0/227456778 shutdown_connections 2021-02-10T11:48:48.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.141 7f19cfdf6700 1 --2- 172.21.2.101:0/227456778 >> v2:172.21.2.108:3300/0 conn(0x7f19c81b0a00 0x7f19c80cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:48.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.141 7f19cfdf6700 1 --2- 172.21.2.101:0/227456778 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19c80d1860 0x7f19c81b03e0 unknown :-1 s=CLOSED pgs=3289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:48.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.141 7f19cfdf6700 1 -- 172.21.2.101:0/227456778 >> 172.21.2.101:0/227456778 conn(0x7f19c80ce440 msgr2=0x7f19c80d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:48.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.142 7f19cfdf6700 1 -- 172.21.2.101:0/227456778 shutdown_connections 2021-02-10T11:48:48.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.142 7f19cfdf6700 1 -- 172.21.2.101:0/227456778 wait complete. 2021-02-10T11:48:48.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.156 7f19cfdf6700 1 Processor -- start 2021-02-10T11:48:48.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.156 7f19cfdf6700 1 -- start start 2021-02-10T11:48:48.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.157 7f19cfdf6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19c80acbb0 0x7f19c80ac030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:48.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.157 7f19cfdf6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19c80ac650 0x7f19c80ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:48.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.157 7f19cfdf6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19c80ad6a0 0x7f19c80f19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:48.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.157 7f19cfdf6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c80f1430 con 0x7f19c80acbb0 2021-02-10T11:48:48.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.157 7f19cfdf6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c80adaa0 con 0x7f19c80ac650 2021-02-10T11:48:48.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.157 7f19cfdf6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c80f1fd0 con 0x7f19c80ad6a0 2021-02-10T11:48:48.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.157 7f19cdc12700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19c80acbb0 0x7f19c80ac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:48.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.157 7f19cdc12700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19c80acbb0 0x7f19c80ac030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35240/0 (socket says 172.21.2.101:35240) 2021-02-10T11:48:48.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.157 7f19cdc12700 1 -- 172.21.2.101:0/386973539 learned_addr learned my addr 172.21.2.101:0/386973539 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:48.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.157 7f19cdc12700 1 -- 172.21.2.101:0/386973539 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19c80ad6a0 msgr2=0x7f19c80f19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:48.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.157 7f19cdc12700 1 --2- 172.21.2.101:0/386973539 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19c80ad6a0 0x7f19c80f19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:48.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.157 7f19cdc12700 1 -- 172.21.2.101:0/386973539 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19c80ac650 msgr2=0x7f19c80ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:48.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.157 7f19cdc12700 1 --2- 172.21.2.101:0/386973539 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19c80ac650 0x7f19c80ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:48.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.157 7f19cdc12700 1 -- 172.21.2.101:0/386973539 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c80f21f0 con 0x7f19c80acbb0 2021-02-10T11:48:48.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.157 7f19cdc12700 1 --2- 172.21.2.101:0/386973539 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19c80acbb0 0x7f19c80ac030 secure :-1 s=READY pgs=3290 cs=0 l=1 rev1=1 rx=0x7f19c4006420 tx=0x7f19c400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:48.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.157 7f19ba7fc700 1 -- 172.21.2.101:0/386973539 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19c40090d0 con 0x7f19c80acbb0 2021-02-10T11:48:48.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.157 7f19ba7fc700 1 -- 172.21.2.101:0/386973539 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c400bd50 con 0x7f19c80acbb0 2021-02-10T11:48:48.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.157 7f19cfdf6700 1 -- 172.21.2.101:0/386973539 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c80f21f0 con 0x7f19c80acbb0 2021-02-10T11:48:48.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.157 7f19ba7fc700 1 -- 172.21.2.101:0/386973539 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f19c40054d0 con 0x7f19c80acbb0 2021-02-10T11:48:48.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.158 7f19cfdf6700 1 -- 172.21.2.101:0/386973539 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c80f21f0 con 0x7f19c80acbb0 2021-02-10T11:48:48.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.158 7f19ba7fc700 1 -- 172.21.2.101:0/386973539 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f19c4009440 con 0x7f19c80acbb0 2021-02-10T11:48:48.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.158 7f19ba7fc700 1 --2- 172.21.2.101:0/386973539 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19b00186c0 0x7f19b001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:48.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.158 7f19ba7fc700 1 -- 172.21.2.101:0/386973539 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f19c40327e0 con 0x7f19c80acbb0 2021-02-10T11:48:48.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.158 7f19cd411700 1 --2- 172.21.2.101:0/386973539 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19b00186c0 0x7f19b001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:48.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.159 7f19cd411700 1 --2- 172.21.2.101:0/386973539 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19b00186c0 0x7f19b001abc0 crc :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:48.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.190 7f19cfdf6700 1 -- 172.21.2.101:0/386973539 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c80ee060 con 0x7f19c80acbb0 2021-02-10T11:48:48.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.193 7f19ba7fc700 1 -- 172.21.2.101:0/386973539 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f19c4009010 con 0x7f19c80acbb0 2021-02-10T11:48:48.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.326 7f19cfdf6700 1 -- 172.21.2.101:0/386973539 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs volume ls", "target": ["mgr", ""]}) v1 -- 0x7f19c80ce440 con 0x7f19b00186c0 2021-02-10T11:48:48.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.327 7f19ba7fc700 1 -- 172.21.2.101:0/386973539 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7f19bc006e40 con 0x7f19b00186c0 2021-02-10T11:48:48.323 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:48:48.324 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:48.324 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "cephfs" 2021-02-10T11:48:48.324 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:48.324 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:48:48.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.329 7f19cfdf6700 1 -- 172.21.2.101:0/386973539 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19b00186c0 msgr2=0x7f19b001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:48.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.329 7f19cfdf6700 1 --2- 172.21.2.101:0/386973539 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19b00186c0 0x7f19b001abc0 crc :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:48.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.329 7f19cfdf6700 1 -- 172.21.2.101:0/386973539 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19c80acbb0 msgr2=0x7f19c80ac030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:48.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.329 7f19cfdf6700 1 --2- 172.21.2.101:0/386973539 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19c80acbb0 0x7f19c80ac030 secure :-1 s=READY pgs=3290 cs=0 l=1 rev1=1 rx=0x7f19c4006420 tx=0x7f19c400b960).stop 2021-02-10T11:48:48.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.329 7f19cfdf6700 1 -- 172.21.2.101:0/386973539 shutdown_connections 2021-02-10T11:48:48.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.329 7f19cfdf6700 1 --2- 172.21.2.101:0/386973539 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f19b00186c0 0x7f19b001abc0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:48.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.329 7f19cfdf6700 1 --2- 172.21.2.101:0/386973539 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f19c80ad6a0 0x7f19c80f19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:48.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.329 7f19cfdf6700 1 --2- 172.21.2.101:0/386973539 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f19c80ac650 0x7f19c80ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:48.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.329 7f19cfdf6700 1 --2- 172.21.2.101:0/386973539 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f19c80acbb0 0x7f19c80ac030 unknown :-1 s=CLOSED pgs=3290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:48.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.329 7f19cfdf6700 1 -- 172.21.2.101:0/386973539 >> 172.21.2.101:0/386973539 conn(0x7f19c80bea70 msgr2=0x7f19c8095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:48.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.329 7f19cfdf6700 1 -- 172.21.2.101:0/386973539 shutdown_connections 2021-02-10T11:48:48.326 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.329 7f19cfdf6700 1 -- 172.21.2.101:0/386973539 wait complete. 2021-02-10T11:48:48.334 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:48:48.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.756 7fe91e8be700 1 Processor -- start 2021-02-10T11:48:48.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.756 7fe91e8be700 1 -- start start 2021-02-10T11:48:48.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.756 7fe91e8be700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9180d1870 0x7fe9181b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:48.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.756 7fe91e8be700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe9180cff70 0x7fe918109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:48.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.756 7fe91e8be700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe918109a60 con 0x7fe9181b0ae0 2021-02-10T11:48:48.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.756 7fe91e8be700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe918109750 con 0x7fe9180cff70 2021-02-10T11:48:48.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.756 7fe91e8be700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe918088070 con 0x7fe9180d1870 2021-02-10T11:48:48.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.756 7fe91cedb700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe9180cff70 0x7fe918109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:48.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.756 7fe91cedb700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe9180cff70 0x7fe918109130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35250/0 (socket says 172.21.2.101:35250) 2021-02-10T11:48:48.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.756 7fe91cedb700 1 -- 172.21.2.101:0/90522298 learned_addr learned my addr 172.21.2.101:0/90522298 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:48.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.757 7fe91cedb700 1 -- 172.21.2.101:0/90522298 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9180d1870 msgr2=0x7fe9181b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:48:48.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.757 7fe91cedb700 1 --2- 172.21.2.101:0/90522298 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9180d1870 0x7fe9181b04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:48.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.757 7fe91cedb700 1 -- 172.21.2.101:0/90522298 >> v1:172.21.2.101:6789/0 conn(0x7fe9181b0ae0 legacy=0x7fe918108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:48.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.757 7fe91cedb700 1 -- 172.21.2.101:0/90522298 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe918088260 con 0x7fe9180cff70 2021-02-10T11:48:48.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.757 7fe91cedb700 1 --2- 172.21.2.101:0/90522298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9180cff70 0x7fe918109130 secure :-1 s=READY pgs=3292 cs=0 l=1 rev1=1 rx=0x7fe910006f00 tx=0x7fe910009f10).ready entity=mon.0 client_cookie=3bdb43a080fd6e97 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:48.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.757 7fe916ffd700 1 -- 172.21.2.101:0/90522298 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe91000b3b0 con 0x7fe9180cff70 2021-02-10T11:48:48.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.757 7fe916ffd700 1 -- 172.21.2.101:0/90522298 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91000b630 con 0x7fe9180cff70 2021-02-10T11:48:48.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.757 7fe916ffd700 1 -- 172.21.2.101:0/90522298 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe91000bb50 con 0x7fe9180cff70 2021-02-10T11:48:48.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.757 7fe91e8be700 1 -- 172.21.2.101:0/90522298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9180cff70 msgr2=0x7fe918109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:48.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.757 7fe91e8be700 1 --2- 172.21.2.101:0/90522298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9180cff70 0x7fe918109130 secure :-1 s=READY pgs=3292 cs=0 l=1 rev1=1 rx=0x7fe910006f00 tx=0x7fe910009f10).stop 2021-02-10T11:48:48.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.758 7fe91e8be700 1 -- 172.21.2.101:0/90522298 shutdown_connections 2021-02-10T11:48:48.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.758 7fe91e8be700 1 --2- 172.21.2.101:0/90522298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9180cff70 0x7fe918109130 unknown :-1 s=CLOSED pgs=3292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:48.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.758 7fe91e8be700 1 --2- 172.21.2.101:0/90522298 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9180d1870 0x7fe9181b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:48.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.758 7fe91e8be700 1 -- 172.21.2.101:0/90522298 >> 172.21.2.101:0/90522298 conn(0x7fe9180ce420 msgr2=0x7fe9180d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:48.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.758 7fe91e8be700 1 -- 172.21.2.101:0/90522298 shutdown_connections 2021-02-10T11:48:48.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.758 7fe91e8be700 1 -- 172.21.2.101:0/90522298 wait complete. 2021-02-10T11:48:48.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.772 7fe91e8be700 1 Processor -- start 2021-02-10T11:48:48.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.772 7fe91e8be700 1 -- start start 2021-02-10T11:48:48.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.773 7fe91e8be700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9180c3820 0x7fe9180a7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:48.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.773 7fe91e8be700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe9181a44c0 0x7fe9181a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:48.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.773 7fe91e8be700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9180863d0 0x7fe9180867d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:48.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.773 7fe91e8be700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe918085ed0 con 0x7fe9180863d0 2021-02-10T11:48:48.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.773 7fe91e8be700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9180c0740 con 0x7fe9181a44c0 2021-02-10T11:48:48.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.773 7fe91e8be700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe918086df0 con 0x7fe9180c3820 2021-02-10T11:48:48.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.773 7fe91cedb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9180863d0 0x7fe9180867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:48.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.773 7fe91cedb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9180863d0 0x7fe9180867d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35256/0 (socket says 172.21.2.101:35256) 2021-02-10T11:48:48.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.773 7fe91cedb700 1 -- 172.21.2.101:0/1267169093 learned_addr learned my addr 172.21.2.101:0/1267169093 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:48.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.773 7fe91cedb700 1 -- 172.21.2.101:0/1267169093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9180c3820 msgr2=0x7fe9180a7950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:48.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.773 7fe91cedb700 1 --2- 172.21.2.101:0/1267169093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9180c3820 0x7fe9180a7950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:48.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.773 7fe91cedb700 1 -- 172.21.2.101:0/1267169093 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe9181a44c0 msgr2=0x7fe9181a48c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:48.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.773 7fe91cedb700 1 --2- 172.21.2.101:0/1267169093 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe9181a44c0 0x7fe9181a48c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:48.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.773 7fe91cedb700 1 -- 172.21.2.101:0/1267169093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe918087010 con 0x7fe9180863d0 2021-02-10T11:48:48.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.773 7fe91cedb700 1 --2- 172.21.2.101:0/1267169093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9180863d0 0x7fe9180867d0 secure :-1 s=READY pgs=3293 cs=0 l=1 rev1=1 rx=0x7fe910009390 tx=0x7fe91000bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:48.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.773 7fe914ff9700 1 -- 172.21.2.101:0/1267169093 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe910003030 con 0x7fe9180863d0 2021-02-10T11:48:48.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.773 7fe914ff9700 1 -- 172.21.2.101:0/1267169093 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9100068e0 con 0x7fe9180863d0 2021-02-10T11:48:48.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.773 7fe914ff9700 1 -- 172.21.2.101:0/1267169093 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe91000c570 con 0x7fe9180863d0 2021-02-10T11:48:48.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.773 7fe91e8be700 1 -- 172.21.2.101:0/1267169093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe918087010 con 0x7fe9180863d0 2021-02-10T11:48:48.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.773 7fe91e8be700 1 -- 172.21.2.101:0/1267169093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe918083000 con 0x7fe9180863d0 2021-02-10T11:48:48.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.774 7fe914ff9700 1 -- 172.21.2.101:0/1267169093 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe910001270 con 0x7fe9180863d0 2021-02-10T11:48:48.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.774 7fe914ff9700 1 --2- 172.21.2.101:0/1267169093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8f80186d0 0x7fe8f801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:48.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.774 7fe914ff9700 1 -- 172.21.2.101:0/1267169093 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe9100329c0 con 0x7fe9180863d0 2021-02-10T11:48:48.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.774 7fe917fff700 1 --2- 172.21.2.101:0/1267169093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8f80186d0 0x7fe8f801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:48.773 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.774 7fe917fff700 1 --2- 172.21.2.101:0/1267169093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8f80186d0 0x7fe8f801abd0 crc :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:48.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.806 7fe91e8be700 1 -- 172.21.2.101:0/1267169093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe918086fe0 con 0x7fe9180863d0 2021-02-10T11:48:48.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.808 7fe914ff9700 1 -- 172.21.2.101:0/1267169093 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe910009430 con 0x7fe9180863d0 2021-02-10T11:48:48.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.956 7fe91e8be700 1 -- 172.21.2.101:0/1267169093 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe918086fe0 con 0x7fe9180863d0 2021-02-10T11:48:48.953 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.956 7fe914ff9700 1 -- 172.21.2.101:0/1267169093 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe9100068e0 con 0x7fe9180863d0 2021-02-10T11:48:48.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.958 7fe91e8be700 1 -- 172.21.2.101:0/1267169093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8f80186d0 msgr2=0x7fe8f801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:48.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.958 7fe91e8be700 1 --2- 172.21.2.101:0/1267169093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8f80186d0 0x7fe8f801abd0 crc :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:48.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.958 7fe91e8be700 1 -- 172.21.2.101:0/1267169093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9180863d0 msgr2=0x7fe9180867d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:48.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.958 7fe91e8be700 1 --2- 172.21.2.101:0/1267169093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9180863d0 0x7fe9180867d0 secure :-1 s=READY pgs=3293 cs=0 l=1 rev1=1 rx=0x7fe910009390 tx=0x7fe91000bd40).stop 2021-02-10T11:48:48.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.958 7fe91e8be700 1 -- 172.21.2.101:0/1267169093 shutdown_connections 2021-02-10T11:48:48.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.958 7fe91e8be700 1 --2- 172.21.2.101:0/1267169093 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe9180863d0 0x7fe9180867d0 unknown :-1 s=CLOSED pgs=3293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:48.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.958 7fe91e8be700 1 --2- 172.21.2.101:0/1267169093 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe9181a44c0 0x7fe9181a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:48.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.958 7fe91e8be700 1 --2- 172.21.2.101:0/1267169093 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8f80186d0 0x7fe8f801abd0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:48.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.958 7fe91e8be700 1 --2- 172.21.2.101:0/1267169093 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe9180c3820 0x7fe9180a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:48.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.958 7fe91e8be700 1 -- 172.21.2.101:0/1267169093 >> 172.21.2.101:0/1267169093 conn(0x7fe9180872c0 msgr2=0x7fe918095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:48.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.959 7fe91e8be700 1 -- 172.21.2.101:0/1267169093 shutdown_connections 2021-02-10T11:48:48.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:48.959 7fe91e8be700 1 -- 172.21.2.101:0/1267169093 wait complete. 2021-02-10T11:48:48.963 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolumegroup create cephfs group_0000000000375420 2021-02-10T11:48:49.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.382 7fb71667a700 1 Processor -- start 2021-02-10T11:48:49.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.382 7fb71667a700 1 -- start start 2021-02-10T11:48:49.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.383 7fb71667a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb7100d1870 0x7fb7101b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:49.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.383 7fb71667a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7100cff70 0x7fb710109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:49.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.383 7fb71667a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb710109a60 con 0x7fb7101b0ae0 2021-02-10T11:48:49.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.383 7fb71667a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb710109750 con 0x7fb7100d1870 2021-02-10T11:48:49.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.383 7fb71667a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb710088070 con 0x7fb7100cff70 2021-02-10T11:48:49.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.383 7fb70ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb7100d1870 0x7fb7101b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:49.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.383 7fb70f7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb7101b0ae0 0x7fb710108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35756/0 (socket says 172.21.2.101:35756) 2021-02-10T11:48:49.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.383 7fb70ffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb7100d1870 0x7fb7101b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35262/0 (socket says 172.21.2.101:35262) 2021-02-10T11:48:49.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.383 7fb70f7fe700 1 -- 172.21.2.101:0/2703427341 learned_addr learned my addr 172.21.2.101:0/2703427341 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:49.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.383 7fb714c97700 1 --2- 172.21.2.101:0/2703427341 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7100cff70 0x7fb710109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:49.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.384 7fb70ffff700 1 -- 172.21.2.101:0/2703427341 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7100cff70 msgr2=0x7fb710109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:49.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.384 7fb70ffff700 1 --2- 172.21.2.101:0/2703427341 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7100cff70 0x7fb710109130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:49.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.384 7fb70ffff700 1 -- 172.21.2.101:0/2703427341 >> v1:172.21.2.101:6789/0 conn(0x7fb7101b0ae0 legacy=0x7fb710108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:49.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.384 7fb70ffff700 1 -- 172.21.2.101:0/2703427341 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7100d1c70 con 0x7fb7100d1870 2021-02-10T11:48:49.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.384 7fb70ffff700 1 --2- 172.21.2.101:0/2703427341 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7100d1870 0x7fb7101b04c0 secure :-1 s=READY pgs=3295 cs=0 l=1 rev1=1 rx=0x7fb700006650 tx=0x7fb700009a80).ready entity=mon.0 client_cookie=87c218ed90df13f3 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:49.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.384 7fb70effd700 1 -- 172.21.2.101:0/2703427341 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb7000009b0 con 0x7fb7100d1870 2021-02-10T11:48:49.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.384 7fb70effd700 1 -- 172.21.2.101:0/2703427341 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70000c0c0 con 0x7fb7100d1870 2021-02-10T11:48:49.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.384 7fb70effd700 1 -- 172.21.2.101:0/2703427341 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb70000c510 con 0x7fb7100d1870 2021-02-10T11:48:49.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.384 7fb71667a700 1 -- 172.21.2.101:0/2703427341 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7100d1870 msgr2=0x7fb7101b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:49.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.384 7fb71667a700 1 --2- 172.21.2.101:0/2703427341 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7100d1870 0x7fb7101b04c0 secure :-1 s=READY pgs=3295 cs=0 l=1 rev1=1 rx=0x7fb700006650 tx=0x7fb700009a80).stop 2021-02-10T11:48:49.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.385 7fb71667a700 1 -- 172.21.2.101:0/2703427341 shutdown_connections 2021-02-10T11:48:49.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.385 7fb71667a700 1 --2- 172.21.2.101:0/2703427341 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb7100cff70 0x7fb710109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:49.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.385 7fb71667a700 1 --2- 172.21.2.101:0/2703427341 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7100d1870 0x7fb7101b04c0 unknown :-1 s=CLOSED pgs=3295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:49.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.385 7fb71667a700 1 -- 172.21.2.101:0/2703427341 >> 172.21.2.101:0/2703427341 conn(0x7fb7100ce420 msgr2=0x7fb7100d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:49.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.385 7fb71667a700 1 -- 172.21.2.101:0/2703427341 shutdown_connections 2021-02-10T11:48:49.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.385 7fb71667a700 1 -- 172.21.2.101:0/2703427341 wait complete. 2021-02-10T11:48:49.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.400 7fb71667a700 1 Processor -- start 2021-02-10T11:48:49.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.400 7fb71667a700 1 -- start start 2021-02-10T11:48:49.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.400 7fb71667a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb710191b10 0x7fb7100ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:49.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.400 7fb71667a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb710191f10 0x7fb71019ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:49.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.400 7fb71667a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7101a05f0 0x7fb710082ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:49.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.400 7fb71667a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb710192440 con 0x7fb7101a05f0 2021-02-10T11:48:49.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.400 7fb71667a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7101916e0 con 0x7fb710191f10 2021-02-10T11:48:49.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.400 7fb71667a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb710191900 con 0x7fb710191b10 2021-02-10T11:48:49.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.400 7fb714c97700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7101a05f0 0x7fb710082ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:49.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.400 7fb714c97700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7101a05f0 0x7fb710082ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35270/0 (socket says 172.21.2.101:35270) 2021-02-10T11:48:49.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.401 7fb714c97700 1 -- 172.21.2.101:0/3732175637 learned_addr learned my addr 172.21.2.101:0/3732175637 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:49.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.401 7fb714c97700 1 -- 172.21.2.101:0/3732175637 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb710191b10 msgr2=0x7fb7100ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:49.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.401 7fb714c97700 1 --2- 172.21.2.101:0/3732175637 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb710191b10 0x7fb7100ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:49.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.401 7fb714c97700 1 -- 172.21.2.101:0/3732175637 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb710191f10 msgr2=0x7fb71019ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:49.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.401 7fb714c97700 1 --2- 172.21.2.101:0/3732175637 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb710191f10 0x7fb71019ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:49.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.401 7fb714c97700 1 -- 172.21.2.101:0/3732175637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7100834f0 con 0x7fb7101a05f0 2021-02-10T11:48:49.400 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.401 7fb714c97700 1 --2- 172.21.2.101:0/3732175637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7101a05f0 0x7fb710082ed0 secure :-1 s=READY pgs=3296 cs=0 l=1 rev1=1 rx=0x7fb7080099a0 tx=0x7fb70800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:49.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.401 7fb70cff9700 1 -- 172.21.2.101:0/3732175637 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb7080066f0 con 0x7fb7101a05f0 2021-02-10T11:48:49.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.401 7fb70cff9700 1 -- 172.21.2.101:0/3732175637 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb708003030 con 0x7fb7101a05f0 2021-02-10T11:48:49.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.401 7fb71667a700 1 -- 172.21.2.101:0/3732175637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7100834f0 con 0x7fb7101a05f0 2021-02-10T11:48:49.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.401 7fb70cff9700 1 -- 172.21.2.101:0/3732175637 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb708001420 con 0x7fb7101a05f0 2021-02-10T11:48:49.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.402 7fb71667a700 1 -- 172.21.2.101:0/3732175637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb710083a80 con 0x7fb7101a05f0 2021-02-10T11:48:49.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.402 7fb70cff9700 1 -- 172.21.2.101:0/3732175637 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb7080011f0 con 0x7fb7101a05f0 2021-02-10T11:48:49.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.402 7fb70cff9700 1 --2- 172.21.2.101:0/3732175637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb6f0018730 0x7fb6f001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:49.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.402 7fb70cff9700 1 -- 172.21.2.101:0/3732175637 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb7080329b0 con 0x7fb7101a05f0 2021-02-10T11:48:49.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.402 7fb70ffff700 1 --2- 172.21.2.101:0/3732175637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb6f0018730 0x7fb6f001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:49.401 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.403 7fb70ffff700 1 --2- 172.21.2.101:0/3732175637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb6f0018730 0x7fb6f001ac10 crc :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:49.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.433 7fb71667a700 1 -- 172.21.2.101:0/3732175637 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb710191900 con 0x7fb7101a05f0 2021-02-10T11:48:49.432 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.436 7fb70cff9700 1 -- 172.21.2.101:0/3732175637 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb708000f90 con 0x7fb7101a05f0 2021-02-10T11:48:49.560 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.563 7fb71667a700 1 -- 172.21.2.101:0/3732175637 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "target": ["mgr", ""], "group_name": "group_0000000000375420"}) v1 -- 0x7fb710083a80 con 0x7fb6f0018730 2021-02-10T11:48:49.563 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:48:49.567 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:48:49.563 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:48:49.567 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:48:49.563 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:48:49.567 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:48:49.563 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:48:49.567 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:48:49.563 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:48:49.567 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:48:49.575 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.579 7fb70cff9700 1 -- 172.21.2.101:0/3732175637 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fb700000940 con 0x7fb6f0018730 2021-02-10T11:48:49.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.596 7fb71667a700 1 -- 172.21.2.101:0/3732175637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb6f0018730 msgr2=0x7fb6f001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:49.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.596 7fb71667a700 1 --2- 172.21.2.101:0/3732175637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb6f0018730 0x7fb6f001ac10 crc :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:49.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.596 7fb71667a700 1 -- 172.21.2.101:0/3732175637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7101a05f0 msgr2=0x7fb710082ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:49.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.596 7fb71667a700 1 --2- 172.21.2.101:0/3732175637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7101a05f0 0x7fb710082ed0 secure :-1 s=READY pgs=3296 cs=0 l=1 rev1=1 rx=0x7fb7080099a0 tx=0x7fb70800bcf0).stop 2021-02-10T11:48:49.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.597 7fb71667a700 1 -- 172.21.2.101:0/3732175637 shutdown_connections 2021-02-10T11:48:49.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.597 7fb71667a700 1 --2- 172.21.2.101:0/3732175637 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb7101a05f0 0x7fb710082ed0 unknown :-1 s=CLOSED pgs=3296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:49.593 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.597 7fb71667a700 1 --2- 172.21.2.101:0/3732175637 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb710191f10 0x7fb71019ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:49.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.597 7fb71667a700 1 --2- 172.21.2.101:0/3732175637 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb6f0018730 0x7fb6f001ac10 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:49.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.597 7fb71667a700 1 --2- 172.21.2.101:0/3732175637 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb710191b10 0x7fb7100ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:49.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.597 7fb71667a700 1 -- 172.21.2.101:0/3732175637 >> 172.21.2.101:0/3732175637 conn(0x7fb7100a02d0 msgr2=0x7fb71009f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:49.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.597 7fb71667a700 1 -- 172.21.2.101:0/3732175637 shutdown_connections 2021-02-10T11:48:49.594 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:49.597 7fb71667a700 1 -- 172.21.2.101:0/3732175637 wait complete. 2021-02-10T11:48:49.601 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000947247 --group_name group_0000000000375420 2021-02-10T11:48:50.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.023 7f22587d0700 1 Processor -- start 2021-02-10T11:48:50.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.023 7f22587d0700 1 -- start start 2021-02-10T11:48:50.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.023 7f22587d0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f225010aff0 0x7f22500cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:50.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.023 7f22587d0700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22500fe1c0 con 0x7f22500d1870 2021-02-10T11:48:50.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.023 7f22587d0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22501b0cb0 con 0x7f22500d1c70 2021-02-10T11:48:50.036 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.023 7f22587d0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2250109a60 con 0x7f225010aff0 2021-02-10T11:48:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.023 7f22565ec700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f22500d1870 0x7f22501b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35780/0 (socket says 172.21.2.101:35780) 2021-02-10T11:48:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.023 7f22565ec700 1 -- 172.21.2.101:0/2475532253 learned_addr learned my addr 172.21.2.101:0/2475532253 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.024 7f22555ea700 1 -- 172.21.2.101:0/2475532253 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1509900553 0 0) 0x7f224c0010b0 con 0x7f22500d1870 2021-02-10T11:48:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.024 7f22555ea700 1 -- 172.21.2.101:0/2475532253 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f223c002fe0 con 0x7f22500d1870 2021-02-10T11:48:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.024 7f2256ded700 1 --2- 172.21.2.101:0/2475532253 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f225010aff0 0x7f22500cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.024 7f22555ea700 1 -- 172.21.2.101:0/2475532253 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 704594486 0 0) 0x7f224c0010b0 con 0x7f22500d1870 2021-02-10T11:48:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.024 7f22555ea700 1 -- 172.21.2.101:0/2475532253 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f225010aff0 msgr2=0x7f22500cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.024 7f22555ea700 1 --2- 172.21.2.101:0/2475532253 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f225010aff0 0x7f22500cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.024 7f22555ea700 1 -- 172.21.2.101:0/2475532253 >> v1:172.21.2.108:6789/0 conn(0x7f22500d1c70 legacy=0x7f22500cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.024 7f22555ea700 1 -- 172.21.2.101:0/2475532253 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225010b3f0 con 0x7f22500d1870 2021-02-10T11:48:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.024 7f22555ea700 1 -- 172.21.2.101:0/2475532253 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f224c0015b0 con 0x7f22500d1870 2021-02-10T11:48:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.024 7f22555ea700 1 -- 172.21.2.101:0/2475532253 >> v1:172.21.2.101:6789/0 conn(0x7f22500d1870 legacy=0x7f22501b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:50.037 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.024 7f22555ea700 1 --2- 172.21.2.101:0/2475532253 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f223c003aa0 0x7f223c005f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.024 7f22555ea700 1 --2- 172.21.2.101:0/2475532253 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f223c006750 0x7f223c008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.024 7f22555ea700 1 --2- 172.21.2.101:0/2475532253 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f223c009510 0x7f223c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.024 7f22555ea700 1 -- 172.21.2.101:0/2475532253 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c00c2c0 con 0x7f223c006750 2021-02-10T11:48:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.024 7f22555ea700 1 -- 172.21.2.101:0/2475532253 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c00c4e0 con 0x7f223c009510 2021-02-10T11:48:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.024 7f22555ea700 1 -- 172.21.2.101:0/2475532253 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223c00c700 con 0x7f223c003aa0 2021-02-10T11:48:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.024 7f2255deb700 1 --2- 172.21.2.101:0/2475532253 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f223c006750 0x7f223c008c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.024 7f22565ec700 1 --2- 172.21.2.101:0/2475532253 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f223c003aa0 0x7f223c005f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.024 7f2255deb700 1 -- 172.21.2.101:0/2475532253 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f223c003aa0 msgr2=0x7f223c005f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.024 7f2255deb700 1 --2- 172.21.2.101:0/2475532253 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f223c003aa0 0x7f223c005f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.024 7f2255deb700 1 -- 172.21.2.101:0/2475532253 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f223c009510 msgr2=0x7f223c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.024 7f2255deb700 1 --2- 172.21.2.101:0/2475532253 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f223c009510 0x7f223c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.025 7f2255deb700 1 -- 172.21.2.101:0/2475532253 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223c00c9a0 con 0x7f223c006750 2021-02-10T11:48:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.025 7f2255deb700 1 --2- 172.21.2.101:0/2475532253 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f223c006750 0x7f223c008c70 secure :-1 s=READY pgs=3300 cs=0 l=1 rev1=1 rx=0x7f224404aac0 tx=0x7f224404daf0).ready entity=mon.0 client_cookie=c80aa96147c4f6b8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:50.038 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.025 7f22555ea700 1 -- 172.21.2.101:0/2475532253 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f224404a6e0 con 0x7f223c006750 2021-02-10T11:48:50.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.025 7f22555ea700 1 -- 172.21.2.101:0/2475532253 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22440022e0 con 0x7f223c006750 2021-02-10T11:48:50.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.025 7f22555ea700 1 -- 172.21.2.101:0/2475532253 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2244002770 con 0x7f223c006750 2021-02-10T11:48:50.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.025 7f22587d0700 1 -- 172.21.2.101:0/2475532253 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f223c006750 msgr2=0x7f223c008c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:50.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.025 7f22587d0700 1 --2- 172.21.2.101:0/2475532253 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f223c006750 0x7f223c008c70 secure :-1 s=READY pgs=3300 cs=0 l=1 rev1=1 rx=0x7f224404aac0 tx=0x7f224404daf0).stop 2021-02-10T11:48:50.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.025 7f2256ded700 1 -- 172.21.2.101:0/2475532253 reap_dead start 2021-02-10T11:48:50.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.025 7f22587d0700 1 -- 172.21.2.101:0/2475532253 shutdown_connections 2021-02-10T11:48:50.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.025 7f22587d0700 1 -- 172.21.2.101:0/2475532253 >> 172.21.2.101:0/2475532253 conn(0x7f22500ce420 msgr2=0x7f22500d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:50.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.025 7f22587d0700 1 -- 172.21.2.101:0/2475532253 shutdown_connections 2021-02-10T11:48:50.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.025 7f22587d0700 1 -- 172.21.2.101:0/2475532253 wait complete. 2021-02-10T11:48:50.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.040 7f22587d0700 1 Processor -- start 2021-02-10T11:48:50.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.040 7f22587d0700 1 -- start start 2021-02-10T11:48:50.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.041 7f22587d0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22501a05a0 0x7f22500a8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:50.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.041 7f22587d0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22500b5090 0x7f22500b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:50.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.041 7f22587d0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22500b5ab0 0x7f22500b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:50.041 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.041 7f22587d0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22500b6090 con 0x7f22500b5090 2021-02-10T11:48:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.041 7f22587d0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22500a9240 con 0x7f22501a05a0 2021-02-10T11:48:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.041 7f22587d0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22500b2a30 con 0x7f22500b5ab0 2021-02-10T11:48:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.041 7f2255deb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22500b5090 0x7f22500b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.041 7f2255deb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22500b5090 0x7f22500b5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35300/0 (socket says 172.21.2.101:35300) 2021-02-10T11:48:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.041 7f2255deb700 1 -- 172.21.2.101:0/736638984 learned_addr learned my addr 172.21.2.101:0/736638984 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.041 7f2256ded700 1 --2- 172.21.2.101:0/736638984 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22500b5ab0 0x7f22500b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.041 7f22565ec700 1 --2- 172.21.2.101:0/736638984 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22501a05a0 0x7f22500a8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.041 7f2255deb700 1 -- 172.21.2.101:0/736638984 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22500b5ab0 msgr2=0x7f22500b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.041 7f2255deb700 1 --2- 172.21.2.101:0/736638984 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22500b5ab0 0x7f22500b68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.041 7f2255deb700 1 -- 172.21.2.101:0/736638984 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22501a05a0 msgr2=0x7f22500a8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.041 7f2255deb700 1 --2- 172.21.2.101:0/736638984 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22501a05a0 0x7f22500a8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:50.042 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.041 7f2255deb700 1 -- 172.21.2.101:0/736638984 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22500b6ef0 con 0x7f22500b5090 2021-02-10T11:48:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.041 7f2255deb700 1 --2- 172.21.2.101:0/736638984 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22500b5090 0x7f22500b5490 secure :-1 s=READY pgs=3301 cs=0 l=1 rev1=1 rx=0x7f224404a970 tx=0x7f224404f990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.042 7f2242ffd700 1 -- 172.21.2.101:0/736638984 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f224404a5e0 con 0x7f22500b5090 2021-02-10T11:48:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.042 7f2242ffd700 1 -- 172.21.2.101:0/736638984 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f224404fd80 con 0x7f22500b5090 2021-02-10T11:48:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.042 7f2242ffd700 1 -- 172.21.2.101:0/736638984 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2244002300 con 0x7f22500b5090 2021-02-10T11:48:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.042 7f22587d0700 1 -- 172.21.2.101:0/736638984 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22500b6ef0 con 0x7f22500b5090 2021-02-10T11:48:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.042 7f22587d0700 1 -- 172.21.2.101:0/736638984 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22500b6ef0 con 0x7f22500b5090 2021-02-10T11:48:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.042 7f2242ffd700 1 -- 172.21.2.101:0/736638984 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2244000940 con 0x7f22500b5090 2021-02-10T11:48:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.042 7f2242ffd700 1 --2- 172.21.2.101:0/736638984 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22380186c0 0x7f223801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.042 7f2242ffd700 1 -- 172.21.2.101:0/736638984 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f22440765f0 con 0x7f22500b5090 2021-02-10T11:48:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.042 7f22565ec700 1 --2- 172.21.2.101:0/736638984 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22380186c0 0x7f223801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:50.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.043 7f22565ec700 1 --2- 172.21.2.101:0/736638984 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22380186c0 0x7f223801abc0 crc :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:50.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.073 7f22587d0700 1 -- 172.21.2.101:0/736638984 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22500825c0 con 0x7f22500b5090 2021-02-10T11:48:50.073 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.076 7f2242ffd700 1 -- 172.21.2.101:0/736638984 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2244070f70 con 0x7f22500b5090 2021-02-10T11:48:50.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.201 7f22587d0700 1 -- 172.21.2.101:0/736638984 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000947247", "target": ["mgr", ""], "group_name": "group_0000000000375420"}) v1 -- 0x7f22500825c0 con 0x7f22380186c0 2021-02-10T11:48:50.223 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.227 7f2242ffd700 1 -- 172.21.2.101:0/736638984 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f224c000ee0 con 0x7f22380186c0 2021-02-10T11:48:50.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.234 7f22587d0700 1 -- 172.21.2.101:0/736638984 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22380186c0 msgr2=0x7f223801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:50.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.234 7f22587d0700 1 --2- 172.21.2.101:0/736638984 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22380186c0 0x7f223801abc0 crc :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:50.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.234 7f22587d0700 1 -- 172.21.2.101:0/736638984 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22500b5090 msgr2=0x7f22500b5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:50.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.234 7f22587d0700 1 --2- 172.21.2.101:0/736638984 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22500b5090 0x7f22500b5490 secure :-1 s=READY pgs=3301 cs=0 l=1 rev1=1 rx=0x7f224404a970 tx=0x7f224404f990).stop 2021-02-10T11:48:50.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.234 7f22587d0700 1 -- 172.21.2.101:0/736638984 shutdown_connections 2021-02-10T11:48:50.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.234 7f22587d0700 1 --2- 172.21.2.101:0/736638984 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f22380186c0 0x7f223801abc0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:50.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.235 7f22587d0700 1 --2- 172.21.2.101:0/736638984 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f22500b5ab0 0x7f22500b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:50.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.235 7f22587d0700 1 --2- 172.21.2.101:0/736638984 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f22500b5090 0x7f22500b5490 unknown :-1 s=CLOSED pgs=3301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:50.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.235 7f22587d0700 1 --2- 172.21.2.101:0/736638984 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f22501a05a0 0x7f22500a8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:50.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.235 7f22587d0700 1 -- 172.21.2.101:0/736638984 >> 172.21.2.101:0/736638984 conn(0x7f22500bc130 msgr2=0x7f22500a9630 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:50.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.235 7f22587d0700 1 -- 172.21.2.101:0/736638984 shutdown_connections 2021-02-10T11:48:50.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.235 7f22587d0700 1 -- 172.21.2.101:0/736638984 wait complete. 2021-02-10T11:48:50.239 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume authorize cephfs subvolume_0000000000947247 alice --group_name group_0000000000375420 2021-02-10T11:48:50.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.660 7f01b8638700 1 Processor -- start 2021-02-10T11:48:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.660 7f01b8638700 1 -- start start 2021-02-10T11:48:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.660 7f01b8638700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f01b00eb220 0x7f01b00eb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.660 7f01b8638700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f01b010aad0 0x7f01b010aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.660 7f01b8638700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b010b4f0 con 0x7f01b00dce80 2021-02-10T11:48:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.660 7f01b8638700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b010a550 con 0x7f01b00eb220 2021-02-10T11:48:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.660 7f01b8638700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b00881d0 con 0x7f01b010aad0 2021-02-10T11:48:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.661 7f01b5c53700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f01b00eb220 0x7f01b00eb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.661 7f01b6454700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f01b00dce80 0x7f01b00d14f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35802/0 (socket says 172.21.2.101:35802) 2021-02-10T11:48:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.661 7f01b5c53700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f01b00eb220 0x7f01b00eb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35308/0 (socket says 172.21.2.101:35308) 2021-02-10T11:48:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.661 7f01b6454700 1 -- 172.21.2.101:0/2451896497 learned_addr learned my addr 172.21.2.101:0/2451896497 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.661 7f01b5c53700 1 -- 172.21.2.101:0/2451896497 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f01b010aad0 msgr2=0x7f01b010aed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:48:50.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.661 7f01b5c53700 1 --2- 172.21.2.101:0/2451896497 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f01b010aad0 0x7f01b010aed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:50.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.661 7f01b5c53700 1 -- 172.21.2.101:0/2451896497 >> v1:172.21.2.101:6789/0 conn(0x7f01b00dce80 legacy=0x7f01b00d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:50.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.661 7f01b5c53700 1 -- 172.21.2.101:0/2451896497 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b00d1f30 con 0x7f01b00eb220 2021-02-10T11:48:50.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.661 7f01b5c53700 1 --2- 172.21.2.101:0/2451896497 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f01b00eb220 0x7f01b00eb620 secure :-1 s=READY pgs=3303 cs=0 l=1 rev1=1 rx=0x7f01ac006650 tx=0x7f01ac009a00).ready entity=mon.0 client_cookie=6603f801851025d2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:50.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.661 7f01b5452700 1 -- 172.21.2.101:0/2451896497 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f01ac0009b0 con 0x7f01b00eb220 2021-02-10T11:48:50.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.661 7f01b5452700 1 -- 172.21.2.101:0/2451896497 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac00c0c0 con 0x7f01b00eb220 2021-02-10T11:48:50.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.661 7f01b5452700 1 -- 172.21.2.101:0/2451896497 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f01ac00c5a0 con 0x7f01b00eb220 2021-02-10T11:48:50.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.661 7f01b8638700 1 -- 172.21.2.101:0/2451896497 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f01b00eb220 msgr2=0x7f01b00eb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:50.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.661 7f01b8638700 1 --2- 172.21.2.101:0/2451896497 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f01b00eb220 0x7f01b00eb620 secure :-1 s=READY pgs=3303 cs=0 l=1 rev1=1 rx=0x7f01ac006650 tx=0x7f01ac009a00).stop 2021-02-10T11:48:50.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.662 7f01b8638700 1 -- 172.21.2.101:0/2451896497 shutdown_connections 2021-02-10T11:48:50.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.662 7f01b8638700 1 --2- 172.21.2.101:0/2451896497 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f01b010aad0 0x7f01b010aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:50.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.662 7f01b8638700 1 --2- 172.21.2.101:0/2451896497 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f01b00eb220 0x7f01b00eb620 unknown :-1 s=CLOSED pgs=3303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:50.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.662 7f01b8638700 1 -- 172.21.2.101:0/2451896497 >> 172.21.2.101:0/2451896497 conn(0x7f01b00ea630 msgr2=0x7f01b00d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:50.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.662 7f01b8638700 1 -- 172.21.2.101:0/2451896497 shutdown_connections 2021-02-10T11:48:50.675 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.662 7f01b8638700 1 -- 172.21.2.101:0/2451896497 wait complete. 2021-02-10T11:48:50.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.677 7f01b8638700 1 Processor -- start 2021-02-10T11:48:50.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.677 7f01b8638700 1 -- start start 2021-02-10T11:48:50.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.677 7f01b8638700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f01b00f9bd0 0x7f01b0198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:50.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.677 7f01b8638700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f01b00f9fd0 0x7f01b0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:50.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.677 7f01b8638700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f01b0199570 0x7f01b0199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:50.676 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.677 7f01b8638700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b0097510 con 0x7f01b00f9bd0 2021-02-10T11:48:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.677 7f01b8638700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b0198ff0 con 0x7f01b00f9fd0 2021-02-10T11:48:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.677 7f01b8638700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b0097070 con 0x7f01b0199570 2021-02-10T11:48:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.677 7f01b6454700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f01b00f9bd0 0x7f01b0198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.677 7f01b6454700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f01b00f9bd0 0x7f01b0198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35314/0 (socket says 172.21.2.101:35314) 2021-02-10T11:48:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.677 7f01b6454700 1 -- 172.21.2.101:0/2776481138 learned_addr learned my addr 172.21.2.101:0/2776481138 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.678 7f01b6454700 1 -- 172.21.2.101:0/2776481138 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f01b0199570 msgr2=0x7f01b0199970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.678 7f01b6454700 1 --2- 172.21.2.101:0/2776481138 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f01b0199570 0x7f01b0199970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.678 7f01b6454700 1 -- 172.21.2.101:0/2776481138 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f01b00f9fd0 msgr2=0x7f01b0198980 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.678 7f01b6454700 1 --2- 172.21.2.101:0/2776481138 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f01b00f9fd0 0x7f01b0198980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.678 7f01b6454700 1 -- 172.21.2.101:0/2776481138 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b0199f90 con 0x7f01b00f9bd0 2021-02-10T11:48:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.678 7f01b6454700 1 --2- 172.21.2.101:0/2776481138 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f01b00f9bd0 0x7f01b0198360 secure :-1 s=READY pgs=3304 cs=0 l=1 rev1=1 rx=0x7f01a4001410 tx=0x7f01a400a950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.678 7f01a2ffd700 1 -- 172.21.2.101:0/2776481138 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f01a400ad40 con 0x7f01b00f9bd0 2021-02-10T11:48:50.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.678 7f01a2ffd700 1 -- 172.21.2.101:0/2776481138 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a40009b0 con 0x7f01b00f9bd0 2021-02-10T11:48:50.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.678 7f01a2ffd700 1 -- 172.21.2.101:0/2776481138 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f01a4004590 con 0x7f01b00f9bd0 2021-02-10T11:48:50.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.678 7f01b8638700 1 -- 172.21.2.101:0/2776481138 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b0199f90 con 0x7f01b00f9bd0 2021-02-10T11:48:50.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.678 7f01b8638700 1 -- 172.21.2.101:0/2776481138 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b019a520 con 0x7f01b00f9bd0 2021-02-10T11:48:50.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.679 7f01a2ffd700 1 -- 172.21.2.101:0/2776481138 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f01a40043a0 con 0x7f01b00f9bd0 2021-02-10T11:48:50.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.679 7f01a2ffd700 1 --2- 172.21.2.101:0/2776481138 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f01980186c0 0x7f019801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:50.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.679 7f01a2ffd700 1 -- 172.21.2.101:0/2776481138 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f01a40323d0 con 0x7f01b00f9bd0 2021-02-10T11:48:50.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.679 7f01b5c53700 1 --2- 172.21.2.101:0/2776481138 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f01980186c0 0x7f019801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:50.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.679 7f01b5c53700 1 --2- 172.21.2.101:0/2776481138 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f01980186c0 0x7f019801abc0 crc :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:50.707 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.711 7f01b8638700 1 -- 172.21.2.101:0/2776481138 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b00f9390 con 0x7f01b00f9bd0 2021-02-10T11:48:50.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.713 7f01a2ffd700 1 -- 172.21.2.101:0/2776481138 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f01a40288b0 con 0x7f01b00f9bd0 2021-02-10T11:48:50.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.838 7f01b8638700 1 -- 172.21.2.101:0/2776481138 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000947247", "target": ["mgr", ""], "auth_id": "alice", "group_name": "group_0000000000375420"}) v1 -- 0x7f01b0199f90 con 0x7f01980186c0 2021-02-10T11:48:50.839 INFO:tasks.ceph.mgr.y.gibba001.stderr:Creating meta for ID alice with tenant None 2021-02-10T11:48:50.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.870 7f01a2ffd700 1 -- 172.21.2.101:0/2776481138 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f01ac0009b0 con 0x7f01980186c0 2021-02-10T11:48:50.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.904 7f01b8638700 1 -- 172.21.2.101:0/2776481138 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f01980186c0 msgr2=0x7f019801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:50.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.904 7f01b8638700 1 --2- 172.21.2.101:0/2776481138 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f01980186c0 0x7f019801abc0 crc :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:50.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.904 7f01b8638700 1 -- 172.21.2.101:0/2776481138 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f01b00f9bd0 msgr2=0x7f01b0198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:50.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.904 7f01b8638700 1 --2- 172.21.2.101:0/2776481138 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f01b00f9bd0 0x7f01b0198360 secure :-1 s=READY pgs=3304 cs=0 l=1 rev1=1 rx=0x7f01a4001410 tx=0x7f01a400a950).stop 2021-02-10T11:48:50.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.904 7f01b8638700 1 -- 172.21.2.101:0/2776481138 shutdown_connections 2021-02-10T11:48:50.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.904 7f01b8638700 1 --2- 172.21.2.101:0/2776481138 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f01980186c0 0x7f019801abc0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:50.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.904 7f01b8638700 1 --2- 172.21.2.101:0/2776481138 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f01b0199570 0x7f01b0199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:50.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.904 7f01b8638700 1 --2- 172.21.2.101:0/2776481138 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f01b00f9fd0 0x7f01b0198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:50.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.904 7f01b8638700 1 --2- 172.21.2.101:0/2776481138 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f01b00f9bd0 0x7f01b0198360 unknown :-1 s=CLOSED pgs=3304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:50.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.904 7f01b8638700 1 -- 172.21.2.101:0/2776481138 >> 172.21.2.101:0/2776481138 conn(0x7f01b01b1d40 msgr2=0x7f01b01b70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:50.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.904 7f01b8638700 1 -- 172.21.2.101:0/2776481138 shutdown_connections 2021-02-10T11:48:50.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:50.904 7f01b8638700 1 -- 172.21.2.101:0/2776481138 wait complete. 2021-02-10T11:48:50.908 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume authorize cephfs subvolume_0000000000947247 guest1 --group_name group_0000000000375420 2021-02-10T11:48:51.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.327 7f94318f6700 1 Processor -- start 2021-02-10T11:48:51.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.327 7f94318f6700 1 -- start start 2021-02-10T11:48:51.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.328 7f94318f6700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f942c0dce80 0x7f942c0d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:51.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.328 7f94318f6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f942c10b2c0 0x7f942c0eb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:51.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.328 7f94318f6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f942c10a550 con 0x7f942c0eb220 2021-02-10T11:48:51.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.328 7f94318f6700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942c088240 con 0x7f942c0dce80 2021-02-10T11:48:51.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.328 7f94318f6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942c0d1f30 con 0x7f942c10b2c0 2021-02-10T11:48:51.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.329 7f942ad73700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f942c0eb220 0x7f942c10aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:57462/0 (socket says 172.21.2.101:57462) 2021-02-10T11:48:51.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.329 7f942b574700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f942c0dce80 0x7f942c0d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:51.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.329 7f942bd75700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f942c10b2c0 0x7f942c0eb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:51.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.329 7f942ad73700 1 -- 172.21.2.101:0/145397249 learned_addr learned my addr 172.21.2.101:0/145397249 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:51.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.330 7f942b574700 1 -- 172.21.2.101:0/145397249 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f942c10b2c0 msgr2=0x7f942c0eb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:51.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.330 7f942b574700 1 --2- 172.21.2.101:0/145397249 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f942c10b2c0 0x7f942c0eb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:51.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.330 7f942b574700 1 -- 172.21.2.101:0/145397249 >> v1:172.21.2.108:6789/0 conn(0x7f942c0eb220 legacy=0x7f942c10aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:51.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.330 7f942b574700 1 -- 172.21.2.101:0/145397249 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c0dd280 con 0x7f942c0dce80 2021-02-10T11:48:51.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.330 7f942bd75700 1 --2- 172.21.2.101:0/145397249 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f942c10b2c0 0x7f942c0eb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:48:51.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.330 7f942b574700 1 --2- 172.21.2.101:0/145397249 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f942c0dce80 0x7f942c0d14f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f941c006650 tx=0x7f941c009a80).ready entity=mon.1 client_cookie=60b2fbd90a9e8329 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:51.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.331 7f942a572700 1 -- 172.21.2.101:0/145397249 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f941c0009b0 con 0x7f942c0dce80 2021-02-10T11:48:51.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.331 7f942a572700 1 -- 172.21.2.101:0/145397249 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941c00c0c0 con 0x7f942c0dce80 2021-02-10T11:48:51.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.331 7f942a572700 1 -- 172.21.2.101:0/145397249 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f941c00c5e0 con 0x7f942c0dce80 2021-02-10T11:48:51.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.331 7f94318f6700 1 -- 172.21.2.101:0/145397249 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f942c0dce80 msgr2=0x7f942c0d14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:51.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.331 7f94318f6700 1 --2- 172.21.2.101:0/145397249 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f942c0dce80 0x7f942c0d14f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f941c006650 tx=0x7f941c009a80).stop 2021-02-10T11:48:51.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.331 7f94318f6700 1 -- 172.21.2.101:0/145397249 shutdown_connections 2021-02-10T11:48:51.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.331 7f94318f6700 1 --2- 172.21.2.101:0/145397249 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f942c10b2c0 0x7f942c0eb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:51.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.331 7f94318f6700 1 --2- 172.21.2.101:0/145397249 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f942c0dce80 0x7f942c0d14f0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:51.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.331 7f94318f6700 1 -- 172.21.2.101:0/145397249 >> 172.21.2.101:0/145397249 conn(0x7f942c0ea630 msgr2=0x7f942c0d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:51.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.331 7f94318f6700 1 -- 172.21.2.101:0/145397249 shutdown_connections 2021-02-10T11:48:51.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.331 7f94318f6700 1 -- 172.21.2.101:0/145397249 wait complete. 2021-02-10T11:48:51.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.346 7f94318f6700 1 Processor -- start 2021-02-10T11:48:51.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.346 7f94318f6700 1 -- start start 2021-02-10T11:48:51.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.346 7f94318f6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f942c0fc2a0 0x7f942c0aaad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:51.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.346 7f94318f6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f942c0ab0f0 0x7f942c1036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:51.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.346 7f94318f6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f942c103d10 0x7f942c104110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:51.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.346 7f94318f6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c1b7c50 con 0x7f942c103d10 2021-02-10T11:48:51.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.346 7f94318f6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c0fc6a0 con 0x7f942c0ab0f0 2021-02-10T11:48:51.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.346 7f94318f6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942c104730 con 0x7f942c0fc2a0 2021-02-10T11:48:51.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.346 7f942bd75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f942c103d10 0x7f942c104110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:51.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.346 7f942bd75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f942c103d10 0x7f942c104110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35334/0 (socket says 172.21.2.101:35334) 2021-02-10T11:48:51.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.346 7f942bd75700 1 -- 172.21.2.101:0/3480777336 learned_addr learned my addr 172.21.2.101:0/3480777336 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:51.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.347 7f942bd75700 1 -- 172.21.2.101:0/3480777336 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f942c0fc2a0 msgr2=0x7f942c0aaad0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:51.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.347 7f942bd75700 1 --2- 172.21.2.101:0/3480777336 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f942c0fc2a0 0x7f942c0aaad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:51.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.347 7f942bd75700 1 -- 172.21.2.101:0/3480777336 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f942c0ab0f0 msgr2=0x7f942c1036f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:51.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.347 7f942bd75700 1 --2- 172.21.2.101:0/3480777336 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f942c0ab0f0 0x7f942c1036f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:51.346 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.347 7f942bd75700 1 -- 172.21.2.101:0/3480777336 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c104950 con 0x7f942c103d10 2021-02-10T11:48:51.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.347 7f942bd75700 1 --2- 172.21.2.101:0/3480777336 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f942c103d10 0x7f942c104110 secure :-1 s=READY pgs=3305 cs=0 l=1 rev1=1 rx=0x7f942400b970 tx=0x7f942400bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:51.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.347 7f9413fff700 1 -- 172.21.2.101:0/3480777336 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9424000f90 con 0x7f942c103d10 2021-02-10T11:48:51.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.347 7f9413fff700 1 -- 172.21.2.101:0/3480777336 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9424003030 con 0x7f942c103d10 2021-02-10T11:48:51.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.347 7f9413fff700 1 -- 172.21.2.101:0/3480777336 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9424001400 con 0x7f942c103d10 2021-02-10T11:48:51.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.347 7f94318f6700 1 -- 172.21.2.101:0/3480777336 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942c104950 con 0x7f942c103d10 2021-02-10T11:48:51.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.347 7f94318f6700 1 -- 172.21.2.101:0/3480777336 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942c104950 con 0x7f942c103d10 2021-02-10T11:48:51.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.348 7f9413fff700 1 -- 172.21.2.101:0/3480777336 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9424000e70 con 0x7f942c103d10 2021-02-10T11:48:51.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.348 7f9413fff700 1 --2- 172.21.2.101:0/3480777336 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f940c0186c0 0x7f940c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:51.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.348 7f9413fff700 1 -- 172.21.2.101:0/3480777336 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f94240328e0 con 0x7f942c103d10 2021-02-10T11:48:51.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.348 7f942b574700 1 --2- 172.21.2.101:0/3480777336 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f940c0186c0 0x7f940c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:51.347 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.348 7f942b574700 1 --2- 172.21.2.101:0/3480777336 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f940c0186c0 0x7f940c01abc0 crc :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:51.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.378 7f94318f6700 1 -- 172.21.2.101:0/3480777336 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942c0f93d0 con 0x7f942c103d10 2021-02-10T11:48:51.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.380 7f9413fff700 1 -- 172.21.2.101:0/3480777336 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f94240066f0 con 0x7f942c103d10 2021-02-10T11:48:51.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.506 7f94318f6700 1 -- 172.21.2.101:0/3480777336 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000947247", "target": ["mgr", ""], "auth_id": "guest1", "group_name": "group_0000000000375420"}) v1 -- 0x7f942c10a550 con 0x7f940c0186c0 2021-02-10T11:48:51.507 INFO:tasks.ceph.mgr.y.gibba001.stderr:Creating meta for ID guest1 with tenant None 2021-02-10T11:48:51.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.544 7f9413fff700 1 -- 172.21.2.101:0/3480777336 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f941c000d10 con 0x7f940c0186c0 2021-02-10T11:48:51.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.571 7f94318f6700 1 -- 172.21.2.101:0/3480777336 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f940c0186c0 msgr2=0x7f940c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:51.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.571 7f94318f6700 1 --2- 172.21.2.101:0/3480777336 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f940c0186c0 0x7f940c01abc0 crc :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:51.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.571 7f94318f6700 1 -- 172.21.2.101:0/3480777336 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f942c103d10 msgr2=0x7f942c104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:51.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.571 7f94318f6700 1 --2- 172.21.2.101:0/3480777336 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f942c103d10 0x7f942c104110 secure :-1 s=READY pgs=3305 cs=0 l=1 rev1=1 rx=0x7f942400b970 tx=0x7f942400bd10).stop 2021-02-10T11:48:51.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.571 7f94318f6700 1 -- 172.21.2.101:0/3480777336 shutdown_connections 2021-02-10T11:48:51.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.571 7f94318f6700 1 --2- 172.21.2.101:0/3480777336 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f942c103d10 0x7f942c104110 unknown :-1 s=CLOSED pgs=3305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:51.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.571 7f94318f6700 1 --2- 172.21.2.101:0/3480777336 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f942c0ab0f0 0x7f942c1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:51.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.571 7f94318f6700 1 --2- 172.21.2.101:0/3480777336 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f940c0186c0 0x7f940c01abc0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:51.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.571 7f94318f6700 1 --2- 172.21.2.101:0/3480777336 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f942c0fc2a0 0x7f942c0aaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:51.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.571 7f94318f6700 1 -- 172.21.2.101:0/3480777336 >> 172.21.2.101:0/3480777336 conn(0x7f942c19a820 msgr2=0x7f942c092be0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:51.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.571 7f94318f6700 1 -- 172.21.2.101:0/3480777336 shutdown_connections 2021-02-10T11:48:51.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:51.571 7f94318f6700 1 -- 172.21.2.101:0/3480777336 wait complete. 2021-02-10T11:48:51.575 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume authorize cephfs subvolume_0000000000947247 guest2 --group_name group_0000000000375420 --access_level r 2021-02-10T11:48:52.018 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.004 7f21c930f700 1 Processor -- start 2021-02-10T11:48:52.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.004 7f21c930f700 1 -- start start 2021-02-10T11:48:52.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.005 7f21c930f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21c40d1880 0x7f21c410ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:52.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.005 7f21c930f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c41b0d40 con 0x7f21c40e53a0 2021-02-10T11:48:52.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.005 7f21c930f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c4109ac0 con 0x7f21c410a040 2021-02-10T11:48:52.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.005 7f21c930f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c40e57a0 con 0x7f21c40d1880 2021-02-10T11:48:52.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.005 7f21c307d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f21c40e53a0 0x7f21c41b0550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35832/0 (socket says 172.21.2.101:35832) 2021-02-10T11:48:52.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.005 7f21c307d700 1 -- 172.21.2.101:0/3930897474 learned_addr learned my addr 172.21.2.101:0/3930897474 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:52.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.005 7f21c207b700 1 -- 172.21.2.101:0/3930897474 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2823594197 0 0) 0x7f21b4001180 con 0x7f21c40e53a0 2021-02-10T11:48:52.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.005 7f21c207b700 1 -- 172.21.2.101:0/3930897474 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21ac002fe0 con 0x7f21c40e53a0 2021-02-10T11:48:52.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.005 7f21c387e700 1 --2- 172.21.2.101:0/3930897474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21c40d1880 0x7f21c410ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:52.019 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c207b700 1 -- 172.21.2.101:0/3930897474 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 872624055 0 0) 0x7f21b4001180 con 0x7f21c40e53a0 2021-02-10T11:48:52.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c207b700 1 -- 172.21.2.101:0/3930897474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21c40d1880 msgr2=0x7f21c410ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:52.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c207b700 1 --2- 172.21.2.101:0/3930897474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21c40d1880 0x7f21c410ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:52.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c207b700 1 -- 172.21.2.101:0/3930897474 >> v1:172.21.2.108:6789/0 conn(0x7f21c410a040 legacy=0x7f21c410a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:52.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c207b700 1 -- 172.21.2.101:0/3930897474 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c40d1c80 con 0x7f21c40e53a0 2021-02-10T11:48:52.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c207b700 1 -- 172.21.2.101:0/3930897474 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f21b4001680 con 0x7f21c40e53a0 2021-02-10T11:48:52.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c387e700 1 --2- 172.21.2.101:0/3930897474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21c40d1880 0x7f21c410ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:48:52.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c207b700 1 -- 172.21.2.101:0/3930897474 >> v1:172.21.2.101:6789/0 conn(0x7f21c40e53a0 legacy=0x7f21c41b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:52.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c207b700 1 --2- 172.21.2.101:0/3930897474 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21ac003a60 0x7f21ac005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:52.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c207b700 1 --2- 172.21.2.101:0/3930897474 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21ac006790 0x7f21ac008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:52.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c207b700 1 --2- 172.21.2.101:0/3930897474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21ac009590 0x7f21ac00ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:52.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c207b700 1 -- 172.21.2.101:0/3930897474 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac00c2d0 con 0x7f21ac003a60 2021-02-10T11:48:52.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c207b700 1 -- 172.21.2.101:0/3930897474 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac00c4f0 con 0x7f21ac006790 2021-02-10T11:48:52.020 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c207b700 1 -- 172.21.2.101:0/3930897474 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac00c710 con 0x7f21ac009590 2021-02-10T11:48:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c307d700 1 --2- 172.21.2.101:0/3930897474 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21ac003a60 0x7f21ac005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c387e700 1 --2- 172.21.2.101:0/3930897474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21ac009590 0x7f21ac00ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c307d700 1 -- 172.21.2.101:0/3930897474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21ac009590 msgr2=0x7f21ac00ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c307d700 1 --2- 172.21.2.101:0/3930897474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21ac009590 0x7f21ac00ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c307d700 1 -- 172.21.2.101:0/3930897474 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21ac006790 msgr2=0x7f21ac008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c287c700 1 --2- 172.21.2.101:0/3930897474 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21ac006790 0x7f21ac008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c307d700 1 --2- 172.21.2.101:0/3930897474 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21ac006790 0x7f21ac008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c307d700 1 -- 172.21.2.101:0/3930897474 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac00c9b0 con 0x7f21ac003a60 2021-02-10T11:48:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c387e700 1 --2- 172.21.2.101:0/3930897474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21ac009590 0x7f21ac00ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:48:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c287c700 1 --2- 172.21.2.101:0/3930897474 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21ac006790 0x7f21ac008c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:48:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c307d700 1 --2- 172.21.2.101:0/3930897474 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21ac003a60 0x7f21ac005ef0 secure :-1 s=READY pgs=3307 cs=0 l=1 rev1=1 rx=0x7f21b4000e80 tx=0x7f21b404bd10).ready entity=mon.0 client_cookie=8b739edbf9b8da2e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c207b700 1 -- 172.21.2.101:0/3930897474 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21b404e310 con 0x7f21ac003a60 2021-02-10T11:48:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c207b700 1 -- 172.21.2.101:0/3930897474 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b4001370 con 0x7f21ac003a60 2021-02-10T11:48:52.021 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.006 7f21c207b700 1 -- 172.21.2.101:0/3930897474 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21b404e890 con 0x7f21ac003a60 2021-02-10T11:48:52.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.007 7f21c930f700 1 -- 172.21.2.101:0/3930897474 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21ac003a60 msgr2=0x7f21ac005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:52.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.007 7f21c930f700 1 --2- 172.21.2.101:0/3930897474 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21ac003a60 0x7f21ac005ef0 secure :-1 s=READY pgs=3307 cs=0 l=1 rev1=1 rx=0x7f21b4000e80 tx=0x7f21b404bd10).stop 2021-02-10T11:48:52.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.007 7f21c387e700 1 -- 172.21.2.101:0/3930897474 reap_dead start 2021-02-10T11:48:52.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.007 7f21c930f700 1 -- 172.21.2.101:0/3930897474 shutdown_connections 2021-02-10T11:48:52.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.007 7f21c930f700 1 -- 172.21.2.101:0/3930897474 >> 172.21.2.101:0/3930897474 conn(0x7f21c40ce4b0 msgr2=0x7f21c40e45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:52.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.007 7f21c930f700 1 -- 172.21.2.101:0/3930897474 shutdown_connections 2021-02-10T11:48:52.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.007 7f21c930f700 1 -- 172.21.2.101:0/3930897474 wait complete. 2021-02-10T11:48:52.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.022 7f21c930f700 1 Processor -- start 2021-02-10T11:48:52.022 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.022 7f21c930f700 1 -- start start 2021-02-10T11:48:52.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.023 7f21c930f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21c40a8f70 0x7f21c4095e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:52.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.023 7f21c930f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21c40964a0 0x7f21c40968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:52.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.023 7f21c930f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21c4096ec0 0x7f21c4106520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:52.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.023 7f21c930f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c4095900 con 0x7f21c40964a0 2021-02-10T11:48:52.024 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.023 7f21c930f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c4107be0 con 0x7f21c4096ec0 2021-02-10T11:48:52.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.023 7f21c930f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c4106b40 con 0x7f21c40a8f70 2021-02-10T11:48:52.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.023 7f21c287c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21c40964a0 0x7f21c40968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:52.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.023 7f21c287c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21c40964a0 0x7f21c40968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35350/0 (socket says 172.21.2.101:35350) 2021-02-10T11:48:52.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.023 7f21c287c700 1 -- 172.21.2.101:0/574939555 learned_addr learned my addr 172.21.2.101:0/574939555 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:52.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.023 7f21c307d700 1 --2- 172.21.2.101:0/574939555 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21c40a8f70 0x7f21c4095e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:52.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.023 7f21c387e700 1 --2- 172.21.2.101:0/574939555 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21c4096ec0 0x7f21c4106520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:52.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.023 7f21c287c700 1 -- 172.21.2.101:0/574939555 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21c40a8f70 msgr2=0x7f21c4095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:52.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.023 7f21c287c700 1 --2- 172.21.2.101:0/574939555 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21c40a8f70 0x7f21c4095e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:52.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.023 7f21c287c700 1 -- 172.21.2.101:0/574939555 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21c4096ec0 msgr2=0x7f21c4106520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:52.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.023 7f21c287c700 1 --2- 172.21.2.101:0/574939555 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21c4096ec0 0x7f21c4106520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:52.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.023 7f21c287c700 1 -- 172.21.2.101:0/574939555 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c4106d60 con 0x7f21c40964a0 2021-02-10T11:48:52.025 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.023 7f21c387e700 1 --2- 172.21.2.101:0/574939555 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21c4096ec0 0x7f21c4106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:48:52.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.024 7f21c287c700 1 --2- 172.21.2.101:0/574939555 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21c40964a0 0x7f21c40968a0 secure :-1 s=READY pgs=3308 cs=0 l=1 rev1=1 rx=0x7f21b8006f10 tx=0x7f21b800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:52.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.024 7f21abfff700 1 -- 172.21.2.101:0/574939555 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21b800bd80 con 0x7f21c40964a0 2021-02-10T11:48:52.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.024 7f21abfff700 1 -- 172.21.2.101:0/574939555 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b8000ca0 con 0x7f21c40964a0 2021-02-10T11:48:52.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.024 7f21c930f700 1 -- 172.21.2.101:0/574939555 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c4106d60 con 0x7f21c40964a0 2021-02-10T11:48:52.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.024 7f21abfff700 1 -- 172.21.2.101:0/574939555 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21b80022b0 con 0x7f21c40964a0 2021-02-10T11:48:52.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.024 7f21c930f700 1 -- 172.21.2.101:0/574939555 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c4106d60 con 0x7f21c40964a0 2021-02-10T11:48:52.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.024 7f21abfff700 1 -- 172.21.2.101:0/574939555 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f21b80052e0 con 0x7f21c40964a0 2021-02-10T11:48:52.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.024 7f21abfff700 1 --2- 172.21.2.101:0/574939555 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21a4018730 0x7f21a401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:52.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.024 7f21abfff700 1 -- 172.21.2.101:0/574939555 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f21b8032660 con 0x7f21c40964a0 2021-02-10T11:48:52.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.024 7f21c307d700 1 --2- 172.21.2.101:0/574939555 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21a4018730 0x7f21a401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:52.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.025 7f21c307d700 1 --2- 172.21.2.101:0/574939555 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21a4018730 0x7f21a401ac10 crc :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:52.050 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.054 7f21c930f700 1 -- 172.21.2.101:0/574939555 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c4103750 con 0x7f21c40964a0 2021-02-10T11:48:52.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.057 7f21abfff700 1 -- 172.21.2.101:0/574939555 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f21b8006be0 con 0x7f21c40964a0 2021-02-10T11:48:52.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.184 7f21c930f700 1 -- 172.21.2.101:0/574939555 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"access_level": "r", "prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000947247", "target": ["mgr", ""], "auth_id": "guest2", "group_name": "group_0000000000375420"}) v1 -- 0x7f21c4095900 con 0x7f21a4018730 2021-02-10T11:48:52.185 INFO:tasks.ceph.mgr.y.gibba001.stderr:Creating meta for ID guest2 with tenant None 2021-02-10T11:48:52.216 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.220 7f21abfff700 1 -- 172.21.2.101:0/574939555 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f21b4000960 con 0x7f21a4018730 2021-02-10T11:48:52.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.249 7f21c930f700 1 -- 172.21.2.101:0/574939555 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21a4018730 msgr2=0x7f21a401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:52.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.249 7f21c930f700 1 --2- 172.21.2.101:0/574939555 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21a4018730 0x7f21a401ac10 crc :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:52.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.249 7f21c930f700 1 -- 172.21.2.101:0/574939555 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21c40964a0 msgr2=0x7f21c40968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:52.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.249 7f21c930f700 1 --2- 172.21.2.101:0/574939555 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21c40964a0 0x7f21c40968a0 secure :-1 s=READY pgs=3308 cs=0 l=1 rev1=1 rx=0x7f21b8006f10 tx=0x7f21b800b990).stop 2021-02-10T11:48:52.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.249 7f21c930f700 1 -- 172.21.2.101:0/574939555 shutdown_connections 2021-02-10T11:48:52.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.249 7f21c930f700 1 --2- 172.21.2.101:0/574939555 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21c4096ec0 0x7f21c4106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:52.246 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.249 7f21c930f700 1 --2- 172.21.2.101:0/574939555 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21c40964a0 0x7f21c40968a0 unknown :-1 s=CLOSED pgs=3308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:52.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.249 7f21c930f700 1 --2- 172.21.2.101:0/574939555 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21a4018730 0x7f21a401ac10 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:52.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.249 7f21c930f700 1 --2- 172.21.2.101:0/574939555 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21c40a8f70 0x7f21c4095e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:52.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.249 7f21c930f700 1 -- 172.21.2.101:0/574939555 >> 172.21.2.101:0/574939555 conn(0x7f21c409e480 msgr2=0x7f21c4107f30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:52.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.250 7f21c930f700 1 -- 172.21.2.101:0/574939555 shutdown_connections 2021-02-10T11:48:52.247 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.250 7f21c930f700 1 -- 172.21.2.101:0/574939555 wait complete. 2021-02-10T11:48:52.254 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume authorized_list cephfs subvolume_0000000000947247 --group_name group_0000000000375420 2021-02-10T11:48:52.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.674 7f130a62b700 1 Processor -- start 2021-02-10T11:48:52.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.674 7f130a62b700 1 -- start start 2021-02-10T11:48:52.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.675 7f130a62b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13040d1880 0x7f13041b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:52.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.675 7f130a62b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f13041b0af0 0x7f1304108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:52.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.675 7f130a62b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1304109a70 con 0x7f13040d1c80 2021-02-10T11:48:52.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.675 7f130a62b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13040e5f00 con 0x7f13041b0af0 2021-02-10T11:48:52.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.675 7f130a62b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1304109760 con 0x7f13040d1880 2021-02-10T11:48:52.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.676 7f1308c48700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f13040d1c80 0x7f1304108f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:57494/0 (socket says 172.21.2.101:57494) 2021-02-10T11:48:52.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.676 7f1308c48700 1 -- 172.21.2.101:0/2833652572 learned_addr learned my addr 172.21.2.101:0/2833652572 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:52.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.676 7f13037fe700 1 --2- 172.21.2.101:0/2833652572 >> v2:172.21.2.108:3300/0 conn(0x7f13041b0af0 0x7f1304108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:52.688 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.676 7f1303fff700 1 --2- 172.21.2.101:0/2833652572 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13040d1880 0x7f13041b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:52.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.676 7f13037fe700 1 -- 172.21.2.101:0/2833652572 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13040d1880 msgr2=0x7f13041b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:52.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.676 7f13037fe700 1 --2- 172.21.2.101:0/2833652572 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13040d1880 0x7f13041b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:52.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.676 7f13037fe700 1 -- 172.21.2.101:0/2833652572 >> v1:172.21.2.108:6789/0 conn(0x7f13040d1c80 legacy=0x7f1304108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:52.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.676 7f13037fe700 1 -- 172.21.2.101:0/2833652572 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1304088070 con 0x7f13041b0af0 2021-02-10T11:48:52.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.676 7f13037fe700 1 --2- 172.21.2.101:0/2833652572 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f13041b0af0 0x7f1304108950 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7f12f8006650 tx=0x7f12f8009a80).ready entity=mon.1 client_cookie=65993c52c82b57ce server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:52.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.677 7f1302ffd700 1 -- 172.21.2.101:0/2833652572 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f12f80009b0 con 0x7f13041b0af0 2021-02-10T11:48:52.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.677 7f1302ffd700 1 -- 172.21.2.101:0/2833652572 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f800c0c0 con 0x7f13041b0af0 2021-02-10T11:48:52.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.677 7f1302ffd700 1 -- 172.21.2.101:0/2833652572 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f12f800c5e0 con 0x7f13041b0af0 2021-02-10T11:48:52.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.677 7f130a62b700 1 -- 172.21.2.101:0/2833652572 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f13041b0af0 msgr2=0x7f1304108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:52.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.677 7f130a62b700 1 --2- 172.21.2.101:0/2833652572 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f13041b0af0 0x7f1304108950 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7f12f8006650 tx=0x7f12f8009a80).stop 2021-02-10T11:48:52.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.677 7f130a62b700 1 -- 172.21.2.101:0/2833652572 shutdown_connections 2021-02-10T11:48:52.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.677 7f130a62b700 1 --2- 172.21.2.101:0/2833652572 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f13041b0af0 0x7f1304108950 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:52.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.677 7f130a62b700 1 --2- 172.21.2.101:0/2833652572 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f13040d1880 0x7f13041b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:52.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.677 7f130a62b700 1 -- 172.21.2.101:0/2833652572 >> 172.21.2.101:0/2833652572 conn(0x7f13040ce430 msgr2=0x7f13040d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:52.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.677 7f130a62b700 1 -- 172.21.2.101:0/2833652572 shutdown_connections 2021-02-10T11:48:52.690 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.677 7f130a62b700 1 -- 172.21.2.101:0/2833652572 wait complete. 2021-02-10T11:48:52.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.692 7f130a62b700 1 Processor -- start 2021-02-10T11:48:52.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.692 7f130a62b700 1 -- start start 2021-02-10T11:48:52.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.692 7f130a62b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f130418bca0 0x7f1304187e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:52.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.692 7f130a62b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f130418afd0 0x7f130418b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:52.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.692 7f130a62b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13040b1950 0x7f13040b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:52.691 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.692 7f130a62b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130418b9f0 con 0x7f13040b1950 2021-02-10T11:48:52.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.692 7f130a62b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13041884a0 con 0x7f130418bca0 2021-02-10T11:48:52.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.692 7f130a62b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13040b2370 con 0x7f130418afd0 2021-02-10T11:48:52.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.692 7f1308c48700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13040b1950 0x7f13040b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:52.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.692 7f1308c48700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13040b1950 0x7f13040b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35364/0 (socket says 172.21.2.101:35364) 2021-02-10T11:48:52.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.692 7f1308c48700 1 -- 172.21.2.101:0/164003506 learned_addr learned my addr 172.21.2.101:0/164003506 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:52.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.693 7f1308c48700 1 -- 172.21.2.101:0/164003506 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f130418afd0 msgr2=0x7f130418b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:52.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.693 7f1308c48700 1 --2- 172.21.2.101:0/164003506 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f130418afd0 0x7f130418b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:52.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.693 7f1308c48700 1 -- 172.21.2.101:0/164003506 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f130418bca0 msgr2=0x7f1304187e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:52.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.693 7f1308c48700 1 --2- 172.21.2.101:0/164003506 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f130418bca0 0x7f1304187e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:52.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.693 7f1308c48700 1 -- 172.21.2.101:0/164003506 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13040b2590 con 0x7f13040b1950 2021-02-10T11:48:52.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.693 7f1308c48700 1 --2- 172.21.2.101:0/164003506 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13040b1950 0x7f13040b1d50 secure :-1 s=READY pgs=3309 cs=0 l=1 rev1=1 rx=0x7f12fc007fc0 tx=0x7f12fc00ac80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:52.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.693 7f1300ff9700 1 -- 172.21.2.101:0/164003506 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f12fc007020 con 0x7f13040b1950 2021-02-10T11:48:52.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.693 7f1300ff9700 1 -- 172.21.2.101:0/164003506 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12fc007360 con 0x7f13040b1950 2021-02-10T11:48:52.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.693 7f130a62b700 1 -- 172.21.2.101:0/164003506 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13040b2590 con 0x7f13040b1950 2021-02-10T11:48:52.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.693 7f1300ff9700 1 -- 172.21.2.101:0/164003506 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f12fc0046a0 con 0x7f13040b1950 2021-02-10T11:48:52.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.693 7f130a62b700 1 -- 172.21.2.101:0/164003506 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13040b2590 con 0x7f13040b1950 2021-02-10T11:48:52.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.694 7f1300ff9700 1 -- 172.21.2.101:0/164003506 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f12fc00adc0 con 0x7f13040b1950 2021-02-10T11:48:52.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.694 7f1300ff9700 1 --2- 172.21.2.101:0/164003506 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f12e40186d0 0x7f12e401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:52.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.694 7f1300ff9700 1 -- 172.21.2.101:0/164003506 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f12fc032b20 con 0x7f13040b1950 2021-02-10T11:48:52.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.694 7f1303fff700 1 --2- 172.21.2.101:0/164003506 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f12e40186d0 0x7f12e401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:52.693 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.694 7f1303fff700 1 --2- 172.21.2.101:0/164003506 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f12e40186d0 0x7f12e401abd0 crc :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:52.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.725 7f130a62b700 1 -- 172.21.2.101:0/164003506 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f130418b9f0 con 0x7f13040b1950 2021-02-10T11:48:52.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.727 7f1300ff9700 1 -- 172.21.2.101:0/164003506 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f12fc000cc0 con 0x7f13040b1950 2021-02-10T11:48:52.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.851 7f130a62b700 1 -- 172.21.2.101:0/164003506 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume authorized_list", "vol_name": "cephfs", "sub_name": "subvolume_0000000000947247", "target": ["mgr", ""], "group_name": "group_0000000000375420"}) v1 -- 0x7f130418b9f0 con 0x7f12e40186d0 2021-02-10T11:48:52.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.855 7f1300ff9700 1 -- 172.21.2.101:0/164003506 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+111 (crc 0 0 0) 0x7f12f4006e70 con 0x7f12e40186d0 2021-02-10T11:48:52.854 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:48:52.854 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:52.854 INFO:teuthology.orchestra.run.gibba001.stdout: "guest2": "r" 2021-02-10T11:48:52.855 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:52.855 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:52.855 INFO:teuthology.orchestra.run.gibba001.stdout: "alice": "rw" 2021-02-10T11:48:52.855 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:48:52.855 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:48:52.855 INFO:teuthology.orchestra.run.gibba001.stdout: "guest1": "rw" 2021-02-10T11:48:52.855 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:48:52.855 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:48:52.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.859 7f130a62b700 1 -- 172.21.2.101:0/164003506 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f12e40186d0 msgr2=0x7f12e401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:52.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.859 7f130a62b700 1 --2- 172.21.2.101:0/164003506 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f12e40186d0 0x7f12e401abd0 crc :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:52.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.859 7f130a62b700 1 -- 172.21.2.101:0/164003506 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13040b1950 msgr2=0x7f13040b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:52.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.859 7f130a62b700 1 --2- 172.21.2.101:0/164003506 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13040b1950 0x7f13040b1d50 secure :-1 s=READY pgs=3309 cs=0 l=1 rev1=1 rx=0x7f12fc007fc0 tx=0x7f12fc00ac80).stop 2021-02-10T11:48:52.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.860 7f130a62b700 1 -- 172.21.2.101:0/164003506 shutdown_connections 2021-02-10T11:48:52.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.860 7f130a62b700 1 --2- 172.21.2.101:0/164003506 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f12e40186d0 0x7f12e401abd0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:52.856 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.860 7f130a62b700 1 --2- 172.21.2.101:0/164003506 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f130418afd0 0x7f130418b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:52.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.860 7f130a62b700 1 --2- 172.21.2.101:0/164003506 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f13040b1950 0x7f13040b1d50 unknown :-1 s=CLOSED pgs=3309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:52.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.860 7f130a62b700 1 --2- 172.21.2.101:0/164003506 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f130418bca0 0x7f1304187e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:52.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.860 7f130a62b700 1 -- 172.21.2.101:0/164003506 >> 172.21.2.101:0/164003506 conn(0x7f130418e870 msgr2=0x7f130418ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:52.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.860 7f130a62b700 1 -- 172.21.2.101:0/164003506 shutdown_connections 2021-02-10T11:48:52.857 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:52.860 7f130a62b700 1 -- 172.21.2.101:0/164003506 wait complete. 2021-02-10T11:48:52.864 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume deauthorize cephfs subvolume_0000000000947247 alice --group_name group_0000000000375420 2021-02-10T11:48:53.296 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.284 7f65a5c8a700 1 Processor -- start 2021-02-10T11:48:53.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.284 7f65a5c8a700 1 -- start start 2021-02-10T11:48:53.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.284 7f65a5c8a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f65a00dce80 0x7f65a00d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:53.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.284 7f65a5c8a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65a00eb220 0x7f65a00eb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:53.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.284 7f65a5c8a700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a010a550 con 0x7f65a010aad0 2021-02-10T11:48:53.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.284 7f65a5c8a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a0088240 con 0x7f65a00dce80 2021-02-10T11:48:53.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.284 7f65a5c8a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a00d1f30 con 0x7f65a00eb220 2021-02-10T11:48:53.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.284 7f659f7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f65a00dce80 0x7f65a00d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:53.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.284 7f659f7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f65a00dce80 0x7f65a00d14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35370/0 (socket says 172.21.2.101:35370) 2021-02-10T11:48:53.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.284 7f659f7fe700 1 -- 172.21.2.101:0/2521326286 learned_addr learned my addr 172.21.2.101:0/2521326286 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:53.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.285 7f659f7fe700 1 -- 172.21.2.101:0/2521326286 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65a00eb220 msgr2=0x7f65a00eb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:53.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.285 7f659f7fe700 1 --2- 172.21.2.101:0/2521326286 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65a00eb220 0x7f65a00eb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:53.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.285 7f659f7fe700 1 -- 172.21.2.101:0/2521326286 >> v1:172.21.2.101:6789/0 conn(0x7f65a010aad0 legacy=0x7f65a010aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:53.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.285 7f659f7fe700 1 -- 172.21.2.101:0/2521326286 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a00dd280 con 0x7f65a00dce80 2021-02-10T11:48:53.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.285 7f659f7fe700 1 --2- 172.21.2.101:0/2521326286 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65a00dce80 0x7f65a00d14f0 secure :-1 s=READY pgs=3311 cs=0 l=1 rev1=1 rx=0x7f6588006700 tx=0x7f6588009b80).ready entity=mon.0 client_cookie=cafa44dabd97175a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:53.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.285 7f659e7fc700 1 -- 172.21.2.101:0/2521326286 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f658800c0c0 con 0x7f65a00dce80 2021-02-10T11:48:53.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.285 7f659e7fc700 1 -- 172.21.2.101:0/2521326286 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658800c340 con 0x7f65a00dce80 2021-02-10T11:48:53.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.285 7f659e7fc700 1 -- 172.21.2.101:0/2521326286 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f658800c860 con 0x7f65a00dce80 2021-02-10T11:48:53.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.286 7f65a5c8a700 1 -- 172.21.2.101:0/2521326286 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65a00dce80 msgr2=0x7f65a00d14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:53.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.286 7f65a5c8a700 1 --2- 172.21.2.101:0/2521326286 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65a00dce80 0x7f65a00d14f0 secure :-1 s=READY pgs=3311 cs=0 l=1 rev1=1 rx=0x7f6588006700 tx=0x7f6588009b80).stop 2021-02-10T11:48:53.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.286 7f65a5c8a700 1 -- 172.21.2.101:0/2521326286 shutdown_connections 2021-02-10T11:48:53.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.286 7f65a5c8a700 1 --2- 172.21.2.101:0/2521326286 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65a00eb220 0x7f65a00eb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:53.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.286 7f65a5c8a700 1 --2- 172.21.2.101:0/2521326286 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65a00dce80 0x7f65a00d14f0 unknown :-1 s=CLOSED pgs=3311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:53.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.286 7f65a5c8a700 1 -- 172.21.2.101:0/2521326286 >> 172.21.2.101:0/2521326286 conn(0x7f65a00ea630 msgr2=0x7f65a00d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:53.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.286 7f65a5c8a700 1 -- 172.21.2.101:0/2521326286 shutdown_connections 2021-02-10T11:48:53.299 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.286 7f65a5c8a700 1 -- 172.21.2.101:0/2521326286 wait complete. 2021-02-10T11:48:53.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.301 7f65a5c8a700 1 Processor -- start 2021-02-10T11:48:53.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.301 7f65a5c8a700 1 -- start start 2021-02-10T11:48:53.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.301 7f65a5c8a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65a0093f10 0x7f65a0094510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:53.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.301 7f65a5c8a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65a00b0810 0x7f65a00b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:53.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.301 7f65a5c8a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65a01030d0 0x7f65a01034d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:53.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.301 7f65a5c8a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a0103fd0 con 0x7f65a00b0810 2021-02-10T11:48:53.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.301 7f65a5c8a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a0094be0 con 0x7f65a01030d0 2021-02-10T11:48:53.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.301 7f65a5c8a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a0103af0 con 0x7f65a0093f10 2021-02-10T11:48:53.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.301 7f659effd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65a00b0810 0x7f65a00b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:53.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.301 7f659effd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65a00b0810 0x7f65a00b0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35376/0 (socket says 172.21.2.101:35376) 2021-02-10T11:48:53.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.301 7f659effd700 1 -- 172.21.2.101:0/3488743155 learned_addr learned my addr 172.21.2.101:0/3488743155 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:53.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.302 7f659effd700 1 -- 172.21.2.101:0/3488743155 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65a0093f10 msgr2=0x7f65a0094510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:53.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.302 7f659f7fe700 1 --2- 172.21.2.101:0/3488743155 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65a0093f10 0x7f65a0094510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:53.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.302 7f659effd700 1 --2- 172.21.2.101:0/3488743155 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65a0093f10 0x7f65a0094510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:53.302 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.302 7f659effd700 1 -- 172.21.2.101:0/3488743155 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65a01030d0 msgr2=0x7f65a01034d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:53.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.302 7f659effd700 1 --2- 172.21.2.101:0/3488743155 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65a01030d0 0x7f65a01034d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:53.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.302 7f659effd700 1 -- 172.21.2.101:0/3488743155 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a0103d10 con 0x7f65a00b0810 2021-02-10T11:48:53.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.302 7f659effd700 1 --2- 172.21.2.101:0/3488743155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65a00b0810 0x7f65a00b0c10 secure :-1 s=READY pgs=3312 cs=0 l=1 rev1=1 rx=0x7f6590006f00 tx=0x7f659000b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:53.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.302 7f6597fff700 1 -- 172.21.2.101:0/3488743155 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f659000bd30 con 0x7f65a00b0810 2021-02-10T11:48:53.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.302 7f6597fff700 1 -- 172.21.2.101:0/3488743155 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6590000c90 con 0x7f65a00b0810 2021-02-10T11:48:53.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.302 7f65a5c8a700 1 -- 172.21.2.101:0/3488743155 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a0103d10 con 0x7f65a00b0810 2021-02-10T11:48:53.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.302 7f65a5c8a700 1 -- 172.21.2.101:0/3488743155 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a0103d10 con 0x7f65a00b0810 2021-02-10T11:48:53.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.302 7f6597fff700 1 -- 172.21.2.101:0/3488743155 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f65900022a0 con 0x7f65a00b0810 2021-02-10T11:48:53.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.303 7f6597fff700 1 -- 172.21.2.101:0/3488743155 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6590005310 con 0x7f65a00b0810 2021-02-10T11:48:53.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.303 7f6597fff700 1 --2- 172.21.2.101:0/3488743155 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65840186c0 0x7f658401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:53.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.303 7f6597fff700 1 -- 172.21.2.101:0/3488743155 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6590032770 con 0x7f65a00b0810 2021-02-10T11:48:53.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.303 7f659f7fe700 1 --2- 172.21.2.101:0/3488743155 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65840186c0 0x7f658401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:53.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.303 7f659f7fe700 1 --2- 172.21.2.101:0/3488743155 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65840186c0 0x7f658401abc0 crc :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:53.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.334 7f65a5c8a700 1 -- 172.21.2.101:0/3488743155 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a010a550 con 0x7f65a00b0810 2021-02-10T11:48:53.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.337 7f6597fff700 1 -- 172.21.2.101:0/3488743155 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6590006b90 con 0x7f65a00b0810 2021-02-10T11:48:53.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.461 7f65a5c8a700 1 -- 172.21.2.101:0/3488743155 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume deauthorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000947247", "target": ["mgr", ""], "auth_id": "alice", "group_name": "group_0000000000375420"}) v1 -- 0x7f65a010a550 con 0x7f65840186c0 2021-02-10T11:48:53.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.495 7f6597fff700 1 -- 172.21.2.101:0/3488743155 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f65880009b0 con 0x7f65840186c0 2021-02-10T11:48:53.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.527 7f65a5c8a700 1 -- 172.21.2.101:0/3488743155 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65840186c0 msgr2=0x7f658401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:53.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.527 7f65a5c8a700 1 --2- 172.21.2.101:0/3488743155 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65840186c0 0x7f658401abc0 crc :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:53.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.527 7f65a5c8a700 1 -- 172.21.2.101:0/3488743155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65a00b0810 msgr2=0x7f65a00b0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:53.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.527 7f65a5c8a700 1 --2- 172.21.2.101:0/3488743155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65a00b0810 0x7f65a00b0c10 secure :-1 s=READY pgs=3312 cs=0 l=1 rev1=1 rx=0x7f6590006f00 tx=0x7f659000b940).stop 2021-02-10T11:48:53.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.527 7f65a5c8a700 1 -- 172.21.2.101:0/3488743155 shutdown_connections 2021-02-10T11:48:53.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.527 7f65a5c8a700 1 --2- 172.21.2.101:0/3488743155 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65a01030d0 0x7f65a01034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:53.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.527 7f65a5c8a700 1 --2- 172.21.2.101:0/3488743155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65a00b0810 0x7f65a00b0c10 unknown :-1 s=CLOSED pgs=3312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:53.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.527 7f65a5c8a700 1 --2- 172.21.2.101:0/3488743155 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65840186c0 0x7f658401abc0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:53.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.527 7f65a5c8a700 1 --2- 172.21.2.101:0/3488743155 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65a0093f10 0x7f65a0094510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:53.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.527 7f65a5c8a700 1 -- 172.21.2.101:0/3488743155 >> 172.21.2.101:0/3488743155 conn(0x7f65a01a8260 msgr2=0x7f65a00fac60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:53.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.527 7f65a5c8a700 1 -- 172.21.2.101:0/3488743155 shutdown_connections 2021-02-10T11:48:53.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.527 7f65a5c8a700 1 -- 172.21.2.101:0/3488743155 wait complete. 2021-02-10T11:48:53.531 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume deauthorize cephfs subvolume_0000000000947247 guest1 --group_name group_0000000000375420 2021-02-10T11:48:53.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.951 7f62616e7700 1 Processor -- start 2021-02-10T11:48:53.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.951 7f62616e7700 1 -- start start 2021-02-10T11:48:53.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.951 7f62616e7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f625c0eb220 0x7f625c0eb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:53.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.951 7f62616e7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f625c10aad0 0x7f625c10aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:53.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.951 7f62616e7700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f625c10b4f0 con 0x7f625c0dce80 2021-02-10T11:48:53.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.951 7f62616e7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f625c10a550 con 0x7f625c10aad0 2021-02-10T11:48:53.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.951 7f62616e7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625c0881d0 con 0x7f625c0eb220 2021-02-10T11:48:53.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.952 7f625b2bf700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f625c0dce80 0x7f625c0d14f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35876/0 (socket says 172.21.2.101:35876) 2021-02-10T11:48:53.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.952 7f625b2bf700 1 -- 172.21.2.101:0/783346187 learned_addr learned my addr 172.21.2.101:0/783346187 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:53.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.952 7f625bac0700 1 --2- 172.21.2.101:0/783346187 >> v2:172.21.2.108:3300/0 conn(0x7f625c10aad0 0x7f625c10aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:53.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.952 7f6252abe700 1 --2- 172.21.2.101:0/783346187 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f625c0eb220 0x7f625c0eb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:53.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.952 7f625aabe700 1 -- 172.21.2.101:0/783346187 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3162174025 0 0) 0x7f624c000ce0 con 0x7f625c0dce80 2021-02-10T11:48:53.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.952 7f625aabe700 1 -- 172.21.2.101:0/783346187 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6248002fe0 con 0x7f625c0dce80 2021-02-10T11:48:53.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.952 7f6252abe700 1 -- 172.21.2.101:0/783346187 >> v2:172.21.2.108:3300/0 conn(0x7f625c10aad0 msgr2=0x7f625c10aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:53.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.952 7f6252abe700 1 --2- 172.21.2.101:0/783346187 >> v2:172.21.2.108:3300/0 conn(0x7f625c10aad0 0x7f625c10aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:53.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.952 7f6252abe700 1 -- 172.21.2.101:0/783346187 >> v1:172.21.2.101:6789/0 conn(0x7f625c0dce80 legacy=0x7f625c0d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:53.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.952 7f6252abe700 1 -- 172.21.2.101:0/783346187 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625c0d1f30 con 0x7f625c0eb220 2021-02-10T11:48:53.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.952 7f6252abe700 1 --2- 172.21.2.101:0/783346187 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f625c0eb220 0x7f625c0eb620 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f6244006650 tx=0x7f6244009a80).ready entity=mon.2 client_cookie=feabd9268dcf91a4 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:53.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.953 7f625aabe700 1 -- 172.21.2.101:0/783346187 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f62440009b0 con 0x7f625c0eb220 2021-02-10T11:48:53.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.953 7f625aabe700 1 -- 172.21.2.101:0/783346187 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624400c0c0 con 0x7f625c0eb220 2021-02-10T11:48:53.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.953 7f625aabe700 1 -- 172.21.2.101:0/783346187 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f624400c5e0 con 0x7f625c0eb220 2021-02-10T11:48:53.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.953 7f62616e7700 1 -- 172.21.2.101:0/783346187 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f625c0eb220 msgr2=0x7f625c0eb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:53.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.953 7f62616e7700 1 --2- 172.21.2.101:0/783346187 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f625c0eb220 0x7f625c0eb620 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f6244006650 tx=0x7f6244009a80).stop 2021-02-10T11:48:53.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.953 7f62616e7700 1 -- 172.21.2.101:0/783346187 shutdown_connections 2021-02-10T11:48:53.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.953 7f62616e7700 1 --2- 172.21.2.101:0/783346187 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f625c0eb220 0x7f625c0eb620 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:53.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.953 7f62616e7700 1 --2- 172.21.2.101:0/783346187 >> v2:172.21.2.108:3300/0 conn(0x7f625c10aad0 0x7f625c10aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:53.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.953 7f62616e7700 1 -- 172.21.2.101:0/783346187 >> 172.21.2.101:0/783346187 conn(0x7f625c0ea630 msgr2=0x7f625c0d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:53.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.953 7f62616e7700 1 -- 172.21.2.101:0/783346187 shutdown_connections 2021-02-10T11:48:53.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.953 7f62616e7700 1 -- 172.21.2.101:0/783346187 wait complete. 2021-02-10T11:48:53.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.968 7f62616e7700 1 Processor -- start 2021-02-10T11:48:53.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.968 7f62616e7700 1 -- start start 2021-02-10T11:48:53.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.968 7f62616e7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f625c092c60 0x7f625c098bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:53.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.968 7f62616e7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f625c0addc0 0x7f625c0991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:53.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.968 7f62616e7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f625c0970f0 0x7f625c0974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:53.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.968 7f62616e7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625c098650 con 0x7f625c0970f0 2021-02-10T11:48:53.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.968 7f62616e7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625c099810 con 0x7f625c0addc0 2021-02-10T11:48:53.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.968 7f62616e7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625c0999f0 con 0x7f625c092c60 2021-02-10T11:48:53.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.969 7f625bac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f625c0970f0 0x7f625c0974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:53.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.969 7f6252abe700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f625c0addc0 0x7f625c0991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:53.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.969 7f625bac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f625c0970f0 0x7f625c0974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35392/0 (socket says 172.21.2.101:35392) 2021-02-10T11:48:53.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.969 7f625bac0700 1 -- 172.21.2.101:0/2978656027 learned_addr learned my addr 172.21.2.101:0/2978656027 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:53.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.969 7f625b2bf700 1 --2- 172.21.2.101:0/2978656027 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f625c092c60 0x7f625c098bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:53.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.969 7f6252abe700 1 -- 172.21.2.101:0/2978656027 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f625c092c60 msgr2=0x7f625c098bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:53.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.969 7f6252abe700 1 --2- 172.21.2.101:0/2978656027 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f625c092c60 0x7f625c098bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:53.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.969 7f6252abe700 1 -- 172.21.2.101:0/2978656027 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f625c0970f0 msgr2=0x7f625c0974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:53.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.969 7f6252abe700 1 --2- 172.21.2.101:0/2978656027 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f625c0970f0 0x7f625c0974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:53.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.969 7f6252abe700 1 -- 172.21.2.101:0/2978656027 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625c097b40 con 0x7f625c0addc0 2021-02-10T11:48:53.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.969 7f625bac0700 1 --2- 172.21.2.101:0/2978656027 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f625c0970f0 0x7f625c0974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:48:53.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.969 7f6252abe700 1 --2- 172.21.2.101:0/2978656027 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f625c0addc0 0x7f625c0991f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0x7f624400b530 tx=0x7f624400b8d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:53.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.969 7f6258aba700 1 -- 172.21.2.101:0/2978656027 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f62440093a0 con 0x7f625c0addc0 2021-02-10T11:48:53.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.969 7f6258aba700 1 -- 172.21.2.101:0/2978656027 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624400bcc0 con 0x7f625c0addc0 2021-02-10T11:48:53.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.969 7f62616e7700 1 -- 172.21.2.101:0/2978656027 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f625c097b40 con 0x7f625c0addc0 2021-02-10T11:48:53.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.970 7f6258aba700 1 -- 172.21.2.101:0/2978656027 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f62440093a0 con 0x7f625c0addc0 2021-02-10T11:48:53.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.970 7f62616e7700 1 -- 172.21.2.101:0/2978656027 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f625c097b40 con 0x7f625c0addc0 2021-02-10T11:48:53.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.970 7f6258aba700 1 -- 172.21.2.101:0/2978656027 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f62440052e0 con 0x7f625c0addc0 2021-02-10T11:48:53.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.970 7f6258aba700 1 --2- 172.21.2.101:0/2978656027 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6240018730 0x7f624001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:53.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.970 7f6258aba700 1 -- 172.21.2.101:0/2978656027 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f62440326e0 con 0x7f625c0addc0 2021-02-10T11:48:53.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.970 7f625b2bf700 1 --2- 172.21.2.101:0/2978656027 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6240018730 0x7f624001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:53.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:53.971 7f625b2bf700 1 --2- 172.21.2.101:0/2978656027 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6240018730 0x7f624001abc0 crc :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:53.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.002 7f62616e7700 1 -- 172.21.2.101:0/2978656027 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f625c10a550 con 0x7f625c0addc0 2021-02-10T11:48:54.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.005 7f6258aba700 1 -- 172.21.2.101:0/2978656027 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6244009280 con 0x7f625c0addc0 2021-02-10T11:48:54.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.135 7f62616e7700 1 -- 172.21.2.101:0/2978656027 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume deauthorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000947247", "target": ["mgr", ""], "auth_id": "guest1", "group_name": "group_0000000000375420"}) v1 -- 0x7f625c099810 con 0x7f6240018730 2021-02-10T11:48:54.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.170 7f6258aba700 1 -- 172.21.2.101:0/2978656027 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f624c006860 con 0x7f6240018730 2021-02-10T11:48:54.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.200 7f62616e7700 1 -- 172.21.2.101:0/2978656027 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6240018730 msgr2=0x7f624001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:54.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.200 7f62616e7700 1 --2- 172.21.2.101:0/2978656027 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6240018730 0x7f624001abc0 crc :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:54.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.200 7f62616e7700 1 -- 172.21.2.101:0/2978656027 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f625c0addc0 msgr2=0x7f625c0991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:54.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.200 7f62616e7700 1 --2- 172.21.2.101:0/2978656027 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f625c0addc0 0x7f625c0991f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0x7f624400b530 tx=0x7f624400b8d0).stop 2021-02-10T11:48:54.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.201 7f62616e7700 1 -- 172.21.2.101:0/2978656027 shutdown_connections 2021-02-10T11:48:54.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.201 7f62616e7700 1 --2- 172.21.2.101:0/2978656027 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f625c0970f0 0x7f625c0974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:54.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.201 7f62616e7700 1 --2- 172.21.2.101:0/2978656027 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f625c0addc0 0x7f625c0991f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:54.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.201 7f62616e7700 1 --2- 172.21.2.101:0/2978656027 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6240018730 0x7f624001abc0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:54.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.201 7f62616e7700 1 --2- 172.21.2.101:0/2978656027 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f625c092c60 0x7f625c098bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:54.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.201 7f62616e7700 1 -- 172.21.2.101:0/2978656027 >> 172.21.2.101:0/2978656027 conn(0x7f625c0a1390 msgr2=0x7f625c0a8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:54.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.201 7f62616e7700 1 -- 172.21.2.101:0/2978656027 shutdown_connections 2021-02-10T11:48:54.198 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.201 7f62616e7700 1 -- 172.21.2.101:0/2978656027 wait complete. 2021-02-10T11:48:54.205 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume deauthorize cephfs subvolume_0000000000947247 guest2 --group_name group_0000000000375420 2021-02-10T11:48:54.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.623 7f42dd1bc700 1 Processor -- start 2021-02-10T11:48:54.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.623 7f42dd1bc700 1 -- start start 2021-02-10T11:48:54.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.624 7f42dd1bc700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f42d80dce80 0x7f42d80d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:54.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.624 7f42dd1bc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f42d810b2c0 0x7f42d80eb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:54.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.624 7f42dd1bc700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d810a550 con 0x7f42d80eb220 2021-02-10T11:48:54.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.624 7f42dd1bc700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d8088240 con 0x7f42d80dce80 2021-02-10T11:48:54.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.624 7f42dd1bc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80d1f30 con 0x7f42d810b2c0 2021-02-10T11:48:54.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.624 7f42d659c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f42d80eb220 0x7f42d810aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35892/0 (socket says 172.21.2.101:35892) 2021-02-10T11:48:54.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.624 7f42d659c700 1 -- 172.21.2.101:0/747951058 learned_addr learned my addr 172.21.2.101:0/747951058 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:54.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.624 7f42d5d9b700 1 -- 172.21.2.101:0/747951058 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4143186323 0 0) 0x7f42cc0010b0 con 0x7f42d80eb220 2021-02-10T11:48:54.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d5d9b700 1 -- 172.21.2.101:0/747951058 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42c0002fe0 con 0x7f42d80eb220 2021-02-10T11:48:54.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d6d9d700 1 --2- 172.21.2.101:0/747951058 >> v2:172.21.2.108:3300/0 conn(0x7f42d80dce80 0x7f42d80d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:54.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d759e700 1 --2- 172.21.2.101:0/747951058 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f42d810b2c0 0x7f42d80eb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:54.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d5d9b700 1 -- 172.21.2.101:0/747951058 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1487492321 0 0) 0x7f42cc0010b0 con 0x7f42d80eb220 2021-02-10T11:48:54.638 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d5d9b700 1 -- 172.21.2.101:0/747951058 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f42d810b2c0 msgr2=0x7f42d80eb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:54.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d5d9b700 1 --2- 172.21.2.101:0/747951058 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f42d810b2c0 0x7f42d80eb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:54.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d5d9b700 1 -- 172.21.2.101:0/747951058 >> v2:172.21.2.108:3300/0 conn(0x7f42d80dce80 msgr2=0x7f42d80d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:54.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d5d9b700 1 --2- 172.21.2.101:0/747951058 >> v2:172.21.2.108:3300/0 conn(0x7f42d80dce80 0x7f42d80d14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:54.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d5d9b700 1 -- 172.21.2.101:0/747951058 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d80dd280 con 0x7f42d80eb220 2021-02-10T11:48:54.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d5d9b700 1 -- 172.21.2.101:0/747951058 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f42cc0015b0 con 0x7f42d80eb220 2021-02-10T11:48:54.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d5d9b700 1 -- 172.21.2.101:0/747951058 >> v1:172.21.2.101:6789/0 conn(0x7f42d80eb220 legacy=0x7f42d810aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:54.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d5d9b700 1 --2- 172.21.2.101:0/747951058 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f42c0003b10 0x7f42c0006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:54.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d5d9b700 1 --2- 172.21.2.101:0/747951058 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f42c00067a0 0x7f42c0008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:54.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d5d9b700 1 --2- 172.21.2.101:0/747951058 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42c0009530 0x7f42c000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:54.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d5d9b700 1 -- 172.21.2.101:0/747951058 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c000c2c0 con 0x7f42c0009530 2021-02-10T11:48:54.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d5d9b700 1 -- 172.21.2.101:0/747951058 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c00504b0 con 0x7f42c0003b10 2021-02-10T11:48:54.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d5d9b700 1 -- 172.21.2.101:0/747951058 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c00506d0 con 0x7f42c00067a0 2021-02-10T11:48:54.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d759e700 1 --2- 172.21.2.101:0/747951058 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42c0009530 0x7f42c000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:54.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d659c700 1 --2- 172.21.2.101:0/747951058 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f42c00067a0 0x7f42c0008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:54.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d6d9d700 1 --2- 172.21.2.101:0/747951058 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f42c0003b10 0x7f42c0006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:54.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d759e700 1 -- 172.21.2.101:0/747951058 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f42c00067a0 msgr2=0x7f42c0008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:54.640 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d759e700 1 --2- 172.21.2.101:0/747951058 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f42c00067a0 0x7f42c0008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:54.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d759e700 1 -- 172.21.2.101:0/747951058 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f42c0003b10 msgr2=0x7f42c0006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:54.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d759e700 1 --2- 172.21.2.101:0/747951058 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f42c0003b10 0x7f42c0006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:54.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d759e700 1 -- 172.21.2.101:0/747951058 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c0050970 con 0x7f42c0009530 2021-02-10T11:48:54.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.625 7f42d759e700 1 --2- 172.21.2.101:0/747951058 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42c0009530 0x7f42c000ba10 secure :-1 s=READY pgs=3315 cs=0 l=1 rev1=1 rx=0x7f42d0006e50 tx=0x7f42d0009e70).ready entity=mon.0 client_cookie=a82c99a5ef8715ae server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:54.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.626 7f42d5d9b700 1 -- 172.21.2.101:0/747951058 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f42d0006a20 con 0x7f42c0009530 2021-02-10T11:48:54.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.626 7f42d5d9b700 1 -- 172.21.2.101:0/747951058 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d000b300 con 0x7f42c0009530 2021-02-10T11:48:54.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.626 7f42d5d9b700 1 -- 172.21.2.101:0/747951058 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f42d000b820 con 0x7f42c0009530 2021-02-10T11:48:54.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.626 7f42dd1bc700 1 -- 172.21.2.101:0/747951058 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42c0009530 msgr2=0x7f42c000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:54.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.626 7f42dd1bc700 1 --2- 172.21.2.101:0/747951058 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42c0009530 0x7f42c000ba10 secure :-1 s=READY pgs=3315 cs=0 l=1 rev1=1 rx=0x7f42d0006e50 tx=0x7f42d0009e70).stop 2021-02-10T11:48:54.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.626 7f42d759e700 1 -- 172.21.2.101:0/747951058 reap_dead start 2021-02-10T11:48:54.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.626 7f42dd1bc700 1 -- 172.21.2.101:0/747951058 shutdown_connections 2021-02-10T11:48:54.642 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.626 7f42dd1bc700 1 -- 172.21.2.101:0/747951058 >> 172.21.2.101:0/747951058 conn(0x7f42d80ea630 msgr2=0x7f42d80d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:54.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.626 7f42dd1bc700 1 -- 172.21.2.101:0/747951058 shutdown_connections 2021-02-10T11:48:54.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.626 7f42dd1bc700 1 -- 172.21.2.101:0/747951058 wait complete. 2021-02-10T11:48:54.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.641 7f42dd1bc700 1 Processor -- start 2021-02-10T11:48:54.643 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.641 7f42dd1bc700 1 -- start start 2021-02-10T11:48:54.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.641 7f42dd1bc700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f42d80fcf20 0x7f42d8106950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:54.645 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.641 7f42dd1bc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42d809e5f0 0x7f42d809e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:54.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.641 7f42dd1bc700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f42d809f010 0x7f42d81a2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:54.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.641 7f42dd1bc700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80fd320 con 0x7f42d809e5f0 2021-02-10T11:48:54.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.641 7f42dd1bc700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d81a35e0 con 0x7f42d80fcf20 2021-02-10T11:48:54.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.641 7f42dd1bc700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d81a3800 con 0x7f42d809f010 2021-02-10T11:48:54.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.642 7f42d659c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42d809e5f0 0x7f42d809e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:54.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.642 7f42d659c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42d809e5f0 0x7f42d809e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35408/0 (socket says 172.21.2.101:35408) 2021-02-10T11:48:54.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.642 7f42d659c700 1 -- 172.21.2.101:0/2070593136 learned_addr learned my addr 172.21.2.101:0/2070593136 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:54.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.642 7f42d6d9d700 1 --2- 172.21.2.101:0/2070593136 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f42d80fcf20 0x7f42d8106950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:54.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.642 7f42d659c700 1 -- 172.21.2.101:0/2070593136 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f42d809f010 msgr2=0x7f42d81a2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:54.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.642 7f42d659c700 1 --2- 172.21.2.101:0/2070593136 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f42d809f010 0x7f42d81a2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:54.646 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.642 7f42d659c700 1 -- 172.21.2.101:0/2070593136 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f42d80fcf20 msgr2=0x7f42d8106950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:54.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.642 7f42d659c700 1 --2- 172.21.2.101:0/2070593136 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f42d80fcf20 0x7f42d8106950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:54.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.642 7f42d659c700 1 -- 172.21.2.101:0/2070593136 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d81a3a20 con 0x7f42d809e5f0 2021-02-10T11:48:54.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.642 7f42d6d9d700 1 --2- 172.21.2.101:0/2070593136 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f42d80fcf20 0x7f42d8106950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:48:54.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.642 7f42d659c700 1 --2- 172.21.2.101:0/2070593136 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42d809e5f0 0x7f42d809e9f0 secure :-1 s=READY pgs=3316 cs=0 l=1 rev1=1 rx=0x7f42cc001c00 tx=0x7f42cc00a8e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:54.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.642 7f42c77fe700 1 -- 172.21.2.101:0/2070593136 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f42cc0009b0 con 0x7f42d809e5f0 2021-02-10T11:48:54.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.642 7f42c77fe700 1 -- 172.21.2.101:0/2070593136 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42cc00acd0 con 0x7f42d809e5f0 2021-02-10T11:48:54.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.642 7f42dd1bc700 1 -- 172.21.2.101:0/2070593136 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d81a3a20 con 0x7f42d809e5f0 2021-02-10T11:48:54.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.642 7f42dd1bc700 1 -- 172.21.2.101:0/2070593136 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d81071e0 con 0x7f42d809e5f0 2021-02-10T11:48:54.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.642 7f42c77fe700 1 -- 172.21.2.101:0/2070593136 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f42cc004540 con 0x7f42d809e5f0 2021-02-10T11:48:54.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.643 7f42c77fe700 1 -- 172.21.2.101:0/2070593136 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f42cc004140 con 0x7f42d809e5f0 2021-02-10T11:48:54.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.643 7f42c77fe700 1 --2- 172.21.2.101:0/2070593136 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f42bc018730 0x7f42bc01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:54.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.643 7f42c77fe700 1 -- 172.21.2.101:0/2070593136 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f42cc032760 con 0x7f42d809e5f0 2021-02-10T11:48:54.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.643 7f42d6d9d700 1 --2- 172.21.2.101:0/2070593136 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f42bc018730 0x7f42bc01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:54.647 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.644 7f42d6d9d700 1 --2- 172.21.2.101:0/2070593136 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f42bc018730 0x7f42bc01ac10 crc :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:54.670 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.674 7f42dd1bc700 1 -- 172.21.2.101:0/2070593136 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d80f93d0 con 0x7f42d809e5f0 2021-02-10T11:48:54.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.676 7f42c77fe700 1 -- 172.21.2.101:0/2070593136 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f42cc000960 con 0x7f42d809e5f0 2021-02-10T11:48:54.799 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.803 7f42dd1bc700 1 -- 172.21.2.101:0/2070593136 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume deauthorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000947247", "target": ["mgr", ""], "auth_id": "guest2", "group_name": "group_0000000000375420"}) v1 -- 0x7f42d80fd320 con 0x7f42bc018730 2021-02-10T11:48:54.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.836 7f42c77fe700 1 -- 172.21.2.101:0/2070593136 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f42c80009b0 con 0x7f42bc018730 2021-02-10T11:48:54.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.868 7f42dd1bc700 1 -- 172.21.2.101:0/2070593136 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f42bc018730 msgr2=0x7f42bc01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:54.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.868 7f42dd1bc700 1 --2- 172.21.2.101:0/2070593136 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f42bc018730 0x7f42bc01ac10 crc :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:54.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.868 7f42dd1bc700 1 -- 172.21.2.101:0/2070593136 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42d809e5f0 msgr2=0x7f42d809e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:54.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.869 7f42dd1bc700 1 --2- 172.21.2.101:0/2070593136 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42d809e5f0 0x7f42d809e9f0 secure :-1 s=READY pgs=3316 cs=0 l=1 rev1=1 rx=0x7f42cc001c00 tx=0x7f42cc00a8e0).stop 2021-02-10T11:48:54.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.869 7f42dd1bc700 1 -- 172.21.2.101:0/2070593136 shutdown_connections 2021-02-10T11:48:54.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.869 7f42dd1bc700 1 --2- 172.21.2.101:0/2070593136 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f42bc018730 0x7f42bc01ac10 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:54.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.869 7f42dd1bc700 1 --2- 172.21.2.101:0/2070593136 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f42d809f010 0x7f42d81a2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:54.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.869 7f42dd1bc700 1 --2- 172.21.2.101:0/2070593136 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f42d809e5f0 0x7f42d809e9f0 unknown :-1 s=CLOSED pgs=3316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:54.865 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.869 7f42dd1bc700 1 --2- 172.21.2.101:0/2070593136 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f42d80fcf20 0x7f42d8106950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:54.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.869 7f42dd1bc700 1 -- 172.21.2.101:0/2070593136 >> 172.21.2.101:0/2070593136 conn(0x7f42d80b1660 msgr2=0x7f42d80a8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:54.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.869 7f42dd1bc700 1 -- 172.21.2.101:0/2070593136 shutdown_connections 2021-02-10T11:48:54.866 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:54.869 7f42dd1bc700 1 -- 172.21.2.101:0/2070593136 wait complete. 2021-02-10T11:48:54.873 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000947247 --group_name group_0000000000375420 2021-02-10T11:48:55.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.295 7f65067eb700 1 Processor -- start 2021-02-10T11:48:55.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.295 7f65067eb700 1 -- start start 2021-02-10T11:48:55.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.296 7f65067eb700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f650010b000 0x7f65000cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:55.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.296 7f65067eb700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65000fe1d0 con 0x7f65000d1c80 2021-02-10T11:48:55.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.296 7f65067eb700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65001b0cc0 con 0x7f65000d1880 2021-02-10T11:48:55.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.296 7f65067eb700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6500109a70 con 0x7f650010b000 2021-02-10T11:48:55.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.296 7f6504e08700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f650010b000 0x7f65000cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:55.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.296 7f64ff7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f65000d1c80 0x7f65000cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35912/0 (socket says 172.21.2.101:35912) 2021-02-10T11:48:55.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.296 7f64ff7fe700 1 -- 172.21.2.101:0/1193392442 learned_addr learned my addr 172.21.2.101:0/1193392442 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:48:55.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.296 7f6504e08700 1 -- 172.21.2.101:0/1193392442 >> v1:172.21.2.108:6789/0 conn(0x7f65000d1880 legacy=0x7f65001b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:55.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.296 7f6504e08700 1 -- 172.21.2.101:0/1193392442 >> v1:172.21.2.101:6789/0 conn(0x7f65000d1c80 legacy=0x7f65000cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:55.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.296 7f6504e08700 1 -- 172.21.2.101:0/1193392442 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650010b400 con 0x7f650010b000 2021-02-10T11:48:55.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.297 7f6504e08700 1 --2- 172.21.2.101:0/1193392442 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f650010b000 0x7f65000cf760 secure :-1 s=READY pgs=3318 cs=0 l=1 rev1=1 rx=0x7f64f8006e30 tx=0x7f64f8009e60).ready entity=mon.0 client_cookie=156dbe545a1e998f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:55.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.297 7f64feffd700 1 -- 172.21.2.101:0/1193392442 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f64f800b300 con 0x7f650010b000 2021-02-10T11:48:55.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.297 7f64feffd700 1 -- 172.21.2.101:0/1193392442 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f800b680 con 0x7f650010b000 2021-02-10T11:48:55.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.297 7f64feffd700 1 -- 172.21.2.101:0/1193392442 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f64f800bba0 con 0x7f650010b000 2021-02-10T11:48:55.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.297 7f65067eb700 1 -- 172.21.2.101:0/1193392442 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f650010b000 msgr2=0x7f65000cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:55.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.297 7f65067eb700 1 --2- 172.21.2.101:0/1193392442 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f650010b000 0x7f65000cf760 secure :-1 s=READY pgs=3318 cs=0 l=1 rev1=1 rx=0x7f64f8006e30 tx=0x7f64f8009e60).stop 2021-02-10T11:48:55.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.297 7f65067eb700 1 -- 172.21.2.101:0/1193392442 shutdown_connections 2021-02-10T11:48:55.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.297 7f65067eb700 1 --2- 172.21.2.101:0/1193392442 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f650010b000 0x7f65000cf760 unknown :-1 s=CLOSED pgs=3318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:55.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.297 7f65067eb700 1 -- 172.21.2.101:0/1193392442 >> 172.21.2.101:0/1193392442 conn(0x7f65000ce430 msgr2=0x7f65000d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:55.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.297 7f65067eb700 1 -- 172.21.2.101:0/1193392442 shutdown_connections 2021-02-10T11:48:55.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.297 7f65067eb700 1 -- 172.21.2.101:0/1193392442 wait complete. 2021-02-10T11:48:55.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.312 7f65067eb700 1 Processor -- start 2021-02-10T11:48:55.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.312 7f65067eb700 1 -- start start 2021-02-10T11:48:55.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.312 7f65067eb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65001b04f0 0x7f65001b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:55.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.312 7f65067eb700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f650008ac80 0x7f650008b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:55.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.312 7f65067eb700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f650008b6a0 0x7f650008c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:55.311 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.312 7f65067eb700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650008c760 con 0x7f65001b04f0 2021-02-10T11:48:55.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.312 7f65067eb700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650008c980 con 0x7f650008b6a0 2021-02-10T11:48:55.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.312 7f65067eb700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650008cba0 con 0x7f650008ac80 2021-02-10T11:48:55.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.313 7f64fffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65001b04f0 0x7f65001b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:55.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.313 7f64fffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65001b04f0 0x7f65001b08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35424/0 (socket says 172.21.2.101:35424) 2021-02-10T11:48:55.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.313 7f64fffff700 1 -- 172.21.2.101:0/1760202837 learned_addr learned my addr 172.21.2.101:0/1760202837 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:55.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.313 7f6504e08700 1 --2- 172.21.2.101:0/1760202837 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f650008b6a0 0x7f650008c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:55.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.313 7f64ff7fe700 1 --2- 172.21.2.101:0/1760202837 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f650008ac80 0x7f650008b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:55.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.313 7f64fffff700 1 -- 172.21.2.101:0/1760202837 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f650008ac80 msgr2=0x7f650008b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:55.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.313 7f64fffff700 1 --2- 172.21.2.101:0/1760202837 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f650008ac80 0x7f650008b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:55.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.313 7f64fffff700 1 -- 172.21.2.101:0/1760202837 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f650008b6a0 msgr2=0x7f650008c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:55.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.313 7f64fffff700 1 --2- 172.21.2.101:0/1760202837 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f650008b6a0 0x7f650008c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:55.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.313 7f64fffff700 1 -- 172.21.2.101:0/1760202837 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650008cdc0 con 0x7f65001b04f0 2021-02-10T11:48:55.312 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.313 7f64ff7fe700 1 --2- 172.21.2.101:0/1760202837 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f650008ac80 0x7f650008b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:48:55.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.313 7f64fffff700 1 --2- 172.21.2.101:0/1760202837 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65001b04f0 0x7f65001b08f0 secure :-1 s=READY pgs=3319 cs=0 l=1 rev1=1 rx=0x7f64f000b5b0 tx=0x7f64f000b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:55.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.313 7f64fcff9700 1 -- 172.21.2.101:0/1760202837 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f64f000bd40 con 0x7f65001b04f0 2021-02-10T11:48:55.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.314 7f64fcff9700 1 -- 172.21.2.101:0/1760202837 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f0006240 con 0x7f65001b04f0 2021-02-10T11:48:55.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.314 7f64fcff9700 1 -- 172.21.2.101:0/1760202837 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f64f0005720 con 0x7f65001b04f0 2021-02-10T11:48:55.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.314 7f65067eb700 1 -- 172.21.2.101:0/1760202837 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f650008cdc0 con 0x7f65001b04f0 2021-02-10T11:48:55.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.314 7f65067eb700 1 -- 172.21.2.101:0/1760202837 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650008d350 con 0x7f65001b04f0 2021-02-10T11:48:55.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.314 7f64fcff9700 1 -- 172.21.2.101:0/1760202837 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f64f00054a0 con 0x7f65001b04f0 2021-02-10T11:48:55.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.314 7f64fcff9700 1 --2- 172.21.2.101:0/1760202837 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f64e00186c0 0x7f64e001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:55.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.314 7f64fcff9700 1 -- 172.21.2.101:0/1760202837 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f64f0032550 con 0x7f65001b04f0 2021-02-10T11:48:55.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.314 7f64ff7fe700 1 --2- 172.21.2.101:0/1760202837 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f64e00186c0 0x7f64e001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:55.313 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.314 7f64ff7fe700 1 --2- 172.21.2.101:0/1760202837 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f64e00186c0 0x7f64e001abc0 crc :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:55.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.346 7f65067eb700 1 -- 172.21.2.101:0/1760202837 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f650008c760 con 0x7f65001b04f0 2021-02-10T11:48:55.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.349 7f64fcff9700 1 -- 172.21.2.101:0/1760202837 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f64f0000940 con 0x7f65001b04f0 2021-02-10T11:48:55.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.476 7f65067eb700 1 -- 172.21.2.101:0/1760202837 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000947247", "target": ["mgr", ""], "group_name": "group_0000000000375420"}) v1 -- 0x7f650008d350 con 0x7f64e00186c0 2021-02-10T11:48:55.481 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.485 7f64fcff9700 1 -- 172.21.2.101:0/1760202837 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f64f4000e20 con 0x7f64e00186c0 2021-02-10T11:48:55.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.493 7f65067eb700 1 -- 172.21.2.101:0/1760202837 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f64e00186c0 msgr2=0x7f64e001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:55.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.493 7f65067eb700 1 --2- 172.21.2.101:0/1760202837 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f64e00186c0 0x7f64e001abc0 crc :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:55.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.493 7f65067eb700 1 -- 172.21.2.101:0/1760202837 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65001b04f0 msgr2=0x7f65001b08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:55.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.493 7f65067eb700 1 --2- 172.21.2.101:0/1760202837 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65001b04f0 0x7f65001b08f0 secure :-1 s=READY pgs=3319 cs=0 l=1 rev1=1 rx=0x7f64f000b5b0 tx=0x7f64f000b950).stop 2021-02-10T11:48:55.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.493 7f65067eb700 1 -- 172.21.2.101:0/1760202837 shutdown_connections 2021-02-10T11:48:55.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.493 7f65067eb700 1 --2- 172.21.2.101:0/1760202837 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f64e00186c0 0x7f64e001abc0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:55.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.493 7f65067eb700 1 --2- 172.21.2.101:0/1760202837 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f650008ac80 0x7f650008b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:55.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.493 7f65067eb700 1 --2- 172.21.2.101:0/1760202837 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f650008b6a0 0x7f650008c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:55.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.493 7f65067eb700 1 --2- 172.21.2.101:0/1760202837 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65001b04f0 0x7f65001b08f0 unknown :-1 s=CLOSED pgs=3319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:55.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.493 7f65067eb700 1 -- 172.21.2.101:0/1760202837 >> 172.21.2.101:0/1760202837 conn(0x7f650009fb50 msgr2=0x7f65000fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:55.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.493 7f65067eb700 1 -- 172.21.2.101:0/1760202837 shutdown_connections 2021-02-10T11:48:55.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.493 7f65067eb700 1 -- 172.21.2.101:0/1760202837 wait complete. 2021-02-10T11:48:55.499 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolumegroup rm cephfs group_0000000000375420 2021-02-10T11:48:55.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.918 7f21e3262700 1 Processor -- start 2021-02-10T11:48:55.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.919 7f21e3262700 1 -- start start 2021-02-10T11:48:55.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.919 7f21e3262700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21dc0d1870 0x7f21dc1b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:55.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.919 7f21e3262700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f21dc1b0ae0 0x7f21dc108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:55.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.919 7f21e3262700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21dc109a60 con 0x7f21dc0d1c70 2021-02-10T11:48:55.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.919 7f21e3262700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21dc0e5ef0 con 0x7f21dc1b0ae0 2021-02-10T11:48:55.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.919 7f21e3262700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc109750 con 0x7f21dc0d1870 2021-02-10T11:48:55.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.919 7f21e107e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21dc0d1870 0x7f21dc1b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:55.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.919 7f21e107e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21dc0d1870 0x7f21dc1b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:56394/0 (socket says 172.21.2.101:56394) 2021-02-10T11:48:55.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.919 7f21e107e700 1 -- 172.21.2.101:0/3638705995 learned_addr learned my addr 172.21.2.101:0/3638705995 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:55.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.919 7f21e087d700 1 --2- 172.21.2.101:0/3638705995 >> v2:172.21.2.108:3300/0 conn(0x7f21dc1b0ae0 0x7f21dc108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:55.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.920 7f21e107e700 1 -- 172.21.2.101:0/3638705995 >> v2:172.21.2.108:3300/0 conn(0x7f21dc1b0ae0 msgr2=0x7f21dc108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:48:55.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.920 7f21e107e700 1 --2- 172.21.2.101:0/3638705995 >> v2:172.21.2.108:3300/0 conn(0x7f21dc1b0ae0 0x7f21dc108940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:55.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.920 7f21e107e700 1 -- 172.21.2.101:0/3638705995 >> v1:172.21.2.108:6789/0 conn(0x7f21dc0d1c70 legacy=0x7f21dc108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:55.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.920 7f21e107e700 1 -- 172.21.2.101:0/3638705995 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21dc088070 con 0x7f21dc0d1870 2021-02-10T11:48:55.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.920 7f21e107e700 1 --2- 172.21.2.101:0/3638705995 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21dc0d1870 0x7f21dc1b04c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f21d0006700 tx=0x7f21d0009b80).ready entity=mon.2 client_cookie=98091d439c9e6781 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:55.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.920 7f21cffff700 1 -- 172.21.2.101:0/3638705995 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21d000c0c0 con 0x7f21dc0d1870 2021-02-10T11:48:55.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.920 7f21cffff700 1 -- 172.21.2.101:0/3638705995 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d000c340 con 0x7f21dc0d1870 2021-02-10T11:48:55.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.920 7f21cffff700 1 -- 172.21.2.101:0/3638705995 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21d000c860 con 0x7f21dc0d1870 2021-02-10T11:48:55.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.921 7f21e3262700 1 -- 172.21.2.101:0/3638705995 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21dc0d1870 msgr2=0x7f21dc1b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:55.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.921 7f21e3262700 1 --2- 172.21.2.101:0/3638705995 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21dc0d1870 0x7f21dc1b04c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f21d0006700 tx=0x7f21d0009b80).stop 2021-02-10T11:48:55.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.921 7f21e3262700 1 -- 172.21.2.101:0/3638705995 shutdown_connections 2021-02-10T11:48:55.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.921 7f21e3262700 1 --2- 172.21.2.101:0/3638705995 >> v2:172.21.2.108:3300/0 conn(0x7f21dc1b0ae0 0x7f21dc108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:55.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.921 7f21e3262700 1 --2- 172.21.2.101:0/3638705995 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21dc0d1870 0x7f21dc1b04c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:55.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.921 7f21e3262700 1 -- 172.21.2.101:0/3638705995 >> 172.21.2.101:0/3638705995 conn(0x7f21dc0ce420 msgr2=0x7f21dc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:55.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.921 7f21e3262700 1 -- 172.21.2.101:0/3638705995 shutdown_connections 2021-02-10T11:48:55.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.921 7f21e3262700 1 -- 172.21.2.101:0/3638705995 wait complete. 2021-02-10T11:48:55.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.936 7f21e3262700 1 Processor -- start 2021-02-10T11:48:55.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.936 7f21e3262700 1 -- start start 2021-02-10T11:48:55.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.936 7f21e3262700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21dc0b1ba0 0x7f21dc1a78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:55.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.936 7f21e3262700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21dc1a7ef0 0x7f21dc0a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:55.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.936 7f21e3262700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21dc0a7060 0x7f21dc0b2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:55.935 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.936 7f21e3262700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc190c20 con 0x7f21dc0a7060 2021-02-10T11:48:55.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.936 7f21e3262700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc190710 con 0x7f21dc1a7ef0 2021-02-10T11:48:55.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.936 7f21e3262700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc0b1fa0 con 0x7f21dc0b1ba0 2021-02-10T11:48:55.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.937 7f21e187f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21dc0a7060 0x7f21dc0b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:55.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.937 7f21e187f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21dc0a7060 0x7f21dc0b2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35434/0 (socket says 172.21.2.101:35434) 2021-02-10T11:48:55.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.937 7f21e187f700 1 -- 172.21.2.101:0/1460571030 learned_addr learned my addr 172.21.2.101:0/1460571030 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:48:55.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.937 7f21e187f700 1 -- 172.21.2.101:0/1460571030 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21dc0b1ba0 msgr2=0x7f21dc1a78d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:48:55.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.937 7f21e187f700 1 --2- 172.21.2.101:0/1460571030 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21dc0b1ba0 0x7f21dc1a78d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:55.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.937 7f21e187f700 1 -- 172.21.2.101:0/1460571030 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21dc1a7ef0 msgr2=0x7f21dc0a6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:55.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.937 7f21e187f700 1 --2- 172.21.2.101:0/1460571030 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21dc1a7ef0 0x7f21dc0a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:55.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.937 7f21e187f700 1 -- 172.21.2.101:0/1460571030 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21dc0a7460 con 0x7f21dc0a7060 2021-02-10T11:48:55.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.937 7f21e187f700 1 --2- 172.21.2.101:0/1460571030 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21dc0a7060 0x7f21dc0b2920 secure :-1 s=READY pgs=3320 cs=0 l=1 rev1=1 rx=0x7f21d4009940 tx=0x7f21d400bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:55.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.937 7f21cdffb700 1 -- 172.21.2.101:0/1460571030 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21d4000ce0 con 0x7f21dc0a7060 2021-02-10T11:48:55.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.937 7f21cdffb700 1 -- 172.21.2.101:0/1460571030 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d400c2f0 con 0x7f21dc0a7060 2021-02-10T11:48:55.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.937 7f21e3262700 1 -- 172.21.2.101:0/1460571030 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21dc0a7460 con 0x7f21dc0a7060 2021-02-10T11:48:55.936 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.937 7f21e3262700 1 -- 172.21.2.101:0/1460571030 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21dc0b3230 con 0x7f21dc0a7060 2021-02-10T11:48:55.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.937 7f21cdffb700 1 -- 172.21.2.101:0/1460571030 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f21d400c810 con 0x7f21dc0a7060 2021-02-10T11:48:55.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.938 7f21cdffb700 1 -- 172.21.2.101:0/1460571030 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f21d4001220 con 0x7f21dc0a7060 2021-02-10T11:48:55.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.938 7f21cdffb700 1 --2- 172.21.2.101:0/1460571030 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21bc0186c0 0x7f21bc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:48:55.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.938 7f21cdffb700 1 -- 172.21.2.101:0/1460571030 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f21d4032920 con 0x7f21dc0a7060 2021-02-10T11:48:55.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.938 7f21e107e700 1 --2- 172.21.2.101:0/1460571030 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21bc0186c0 0x7f21bc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:48:55.937 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.938 7f21e107e700 1 --2- 172.21.2.101:0/1460571030 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21bc0186c0 0x7f21bc01abc0 crc :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:48:55.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.969 7f21e3262700 1 -- 172.21.2.101:0/1460571030 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21dc082980 con 0x7f21dc0a7060 2021-02-10T11:48:55.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:55.971 7f21cdffb700 1 -- 172.21.2.101:0/1460571030 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f21d4000cc0 con 0x7f21dc0a7060 2021-02-10T11:48:56.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:56.098 7f21e3262700 1 -- 172.21.2.101:0/1460571030 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "target": ["mgr", ""], "group_name": "group_0000000000375420"}) v1 -- 0x7f21dc0b3230 con 0x7f21bc0186c0 2021-02-10T11:48:56.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:56.101 7f21cdffb700 1 -- 172.21.2.101:0/1460571030 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f21d0009100 con 0x7f21bc0186c0 2021-02-10T11:48:56.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:56.107 7f21e3262700 1 -- 172.21.2.101:0/1460571030 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21bc0186c0 msgr2=0x7f21bc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:56.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:56.107 7f21e3262700 1 --2- 172.21.2.101:0/1460571030 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21bc0186c0 0x7f21bc01abc0 crc :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:56.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:56.107 7f21e3262700 1 -- 172.21.2.101:0/1460571030 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21dc0a7060 msgr2=0x7f21dc0b2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:48:56.103 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:56.107 7f21e3262700 1 --2- 172.21.2.101:0/1460571030 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21dc0a7060 0x7f21dc0b2920 secure :-1 s=READY pgs=3320 cs=0 l=1 rev1=1 rx=0x7f21d4009940 tx=0x7f21d400bcf0).stop 2021-02-10T11:48:56.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:56.107 7f21e3262700 1 -- 172.21.2.101:0/1460571030 shutdown_connections 2021-02-10T11:48:56.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:56.107 7f21e3262700 1 --2- 172.21.2.101:0/1460571030 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f21dc0a7060 0x7f21dc0b2920 unknown :-1 s=CLOSED pgs=3320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:56.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:56.107 7f21e3262700 1 --2- 172.21.2.101:0/1460571030 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f21dc1a7ef0 0x7f21dc0a6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:56.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:56.107 7f21e3262700 1 --2- 172.21.2.101:0/1460571030 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f21bc0186c0 0x7f21bc01abc0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:48:56.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:56.107 7f21e3262700 1 --2- 172.21.2.101:0/1460571030 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f21dc0b1ba0 0x7f21dc1a78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:48:56.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:56.107 7f21e3262700 1 -- 172.21.2.101:0/1460571030 >> 172.21.2.101:0/1460571030 conn(0x7f21dc0b5dc0 msgr2=0x7f21dc095960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:48:56.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:56.107 7f21e3262700 1 -- 172.21.2.101:0/1460571030 shutdown_connections 2021-02-10T11:48:56.104 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:48:56.107 7f21e3262700 1 -- 172.21.2.101:0/1460571030 wait complete. 2021-02-10T11:48:56.112 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:48:56.112 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:48:56.459 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:48:56.804 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:48:56.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:56.804 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:48:56.804 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:48:57.141 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:48:57.141 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:48:57.141 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:48:57.142 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:48:57.142 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:48:57.494 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[182070]: fuse finished with error 0 and tester_r 0 2021-02-10T11:48:57.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:57.496 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:48:57.496 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:48:57.831 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:48:57.831 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:48:57.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:48:57.832 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:48:58.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:58.167 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:48:58.167 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:48:58.504 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:48:58.504 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:48:58.504 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:48:58.504 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:48:58.504 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:48:58.855 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[182770]: fuse finished with error 0 and tester_r 0 2021-02-10T11:48:58.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:58.857 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:48:58.857 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:48:59.193 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:48:59.193 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:48:59.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:48:59.194 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:48:59.529 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:48:59.530 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:48:59.530 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:48:59.865 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:48:59.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:48:59.865 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:48:59.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:48:59.866 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:49:00.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:00.200 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:00.200 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:49:00.532 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:49:00.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:49:00.533 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:49:00.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:49:00.533 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:49:00.868 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-02-10T11:49:01.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.289 7fb0ef958700 1 Processor -- start 2021-02-10T11:49:01.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.289 7fb0ef958700 1 -- start start 2021-02-10T11:49:01.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.290 7fb0ef958700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0e80d0340 0x7fb0e80ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:01.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.290 7fb0ef958700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e80f0b10 con 0x7fb0e80d1870 2021-02-10T11:49:01.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.290 7fb0ef958700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e81b0be0 con 0x7fb0e80d1c70 2021-02-10T11:49:01.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.290 7fb0ef958700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e80e5ed0 con 0x7fb0e80d0340 2021-02-10T11:49:01.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.290 7fb0ed774700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb0e80d1870 0x7fb0e81b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35936/0 (socket says 172.21.2.101:35936) 2021-02-10T11:49:01.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.290 7fb0ed774700 1 -- 172.21.2.101:0/2734889560 learned_addr learned my addr 172.21.2.101:0/2734889560 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:01.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.290 7fb0dffff700 1 -- 172.21.2.101:0/2734889560 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3629724107 0 0) 0x7fb0e40010b0 con 0x7fb0e80d1870 2021-02-10T11:49:01.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.290 7fb0dffff700 1 -- 172.21.2.101:0/2734889560 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0d4002fe0 con 0x7fb0e80d1870 2021-02-10T11:49:01.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.290 7fb0ecf73700 1 --2- 172.21.2.101:0/2734889560 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0e80d0340 0x7fb0e80ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:01.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0dffff700 1 -- 172.21.2.101:0/2734889560 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 865381831 0 0) 0x7fb0e40010b0 con 0x7fb0e80d1870 2021-02-10T11:49:01.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0dffff700 1 -- 172.21.2.101:0/2734889560 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0e80d0340 msgr2=0x7fb0e80ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:01.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0dffff700 1 --2- 172.21.2.101:0/2734889560 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0e80d0340 0x7fb0e80ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:01.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0dffff700 1 -- 172.21.2.101:0/2734889560 >> v1:172.21.2.108:6789/0 conn(0x7fb0e80d1c70 legacy=0x7fb0e80efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:01.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0dffff700 1 -- 172.21.2.101:0/2734889560 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e80f0450 con 0x7fb0e80d1870 2021-02-10T11:49:01.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0dffff700 1 -- 172.21.2.101:0/2734889560 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fb0e40015b0 con 0x7fb0e80d1870 2021-02-10T11:49:01.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0dffff700 1 -- 172.21.2.101:0/2734889560 >> v1:172.21.2.101:6789/0 conn(0x7fb0e80d1870 legacy=0x7fb0e81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:01.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0dffff700 1 --2- 172.21.2.101:0/2734889560 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0d4003b10 0x7fb0d4005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:01.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0dffff700 1 --2- 172.21.2.101:0/2734889560 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0d4006770 0x7fb0d4008c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:01.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0dffff700 1 --2- 172.21.2.101:0/2734889560 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0d40094f0 0x7fb0d400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:01.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0dffff700 1 -- 172.21.2.101:0/2734889560 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d400c2c0 con 0x7fb0d40094f0 2021-02-10T11:49:01.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0dffff700 1 -- 172.21.2.101:0/2734889560 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d400c4e0 con 0x7fb0d4006770 2021-02-10T11:49:01.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0dffff700 1 -- 172.21.2.101:0/2734889560 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d400c700 con 0x7fb0d4003b10 2021-02-10T11:49:01.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0edf75700 1 --2- 172.21.2.101:0/2734889560 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0d40094f0 0x7fb0d400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:01.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0ed774700 1 --2- 172.21.2.101:0/2734889560 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0d4003b10 0x7fb0d4005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:01.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0ecf73700 1 --2- 172.21.2.101:0/2734889560 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0d4006770 0x7fb0d4008c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:01.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0edf75700 1 -- 172.21.2.101:0/2734889560 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0d4003b10 msgr2=0x7fb0d4005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:01.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0edf75700 1 --2- 172.21.2.101:0/2734889560 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0d4003b10 0x7fb0d4005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:01.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0edf75700 1 -- 172.21.2.101:0/2734889560 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0d4006770 msgr2=0x7fb0d4008c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:01.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0edf75700 1 --2- 172.21.2.101:0/2734889560 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0d4006770 0x7fb0d4008c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:01.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0edf75700 1 -- 172.21.2.101:0/2734889560 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d4050970 con 0x7fb0d40094f0 2021-02-10T11:49:01.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0ecf73700 1 --2- 172.21.2.101:0/2734889560 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0d4006770 0x7fb0d4008c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:49:01.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.291 7fb0edf75700 1 --2- 172.21.2.101:0/2734889560 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0d40094f0 0x7fb0d400ba10 secure :-1 s=READY pgs=3322 cs=0 l=1 rev1=1 rx=0x7fb0e0006e40 tx=0x7fb0e0009db0).ready entity=mon.0 client_cookie=fd1957fc215d7a3e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:01.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.292 7fb0dffff700 1 -- 172.21.2.101:0/2734889560 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0e0006a60 con 0x7fb0d40094f0 2021-02-10T11:49:01.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.292 7fb0dffff700 1 -- 172.21.2.101:0/2734889560 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e000b390 con 0x7fb0d40094f0 2021-02-10T11:49:01.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.292 7fb0dffff700 1 -- 172.21.2.101:0/2734889560 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0e000b760 con 0x7fb0d40094f0 2021-02-10T11:49:01.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.292 7fb0ef958700 1 -- 172.21.2.101:0/2734889560 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0d40094f0 msgr2=0x7fb0d400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:01.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.292 7fb0ef958700 1 --2- 172.21.2.101:0/2734889560 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0d40094f0 0x7fb0d400ba10 secure :-1 s=READY pgs=3322 cs=0 l=1 rev1=1 rx=0x7fb0e0006e40 tx=0x7fb0e0009db0).stop 2021-02-10T11:49:01.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.292 7fb0edf75700 1 -- 172.21.2.101:0/2734889560 reap_dead start 2021-02-10T11:49:01.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.292 7fb0ef958700 1 -- 172.21.2.101:0/2734889560 shutdown_connections 2021-02-10T11:49:01.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.292 7fb0ef958700 1 -- 172.21.2.101:0/2734889560 >> 172.21.2.101:0/2734889560 conn(0x7fb0e80ce450 msgr2=0x7fb0e80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:01.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.292 7fb0ef958700 1 -- 172.21.2.101:0/2734889560 shutdown_connections 2021-02-10T11:49:01.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.292 7fb0ef958700 1 -- 172.21.2.101:0/2734889560 wait complete. 2021-02-10T11:49:01.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.307 7fb0ef958700 1 Processor -- start 2021-02-10T11:49:01.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.307 7fb0ef958700 1 -- start start 2021-02-10T11:49:01.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.307 7fb0ef958700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0e8094310 0x7fb0e80ebe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:01.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.307 7fb0ef958700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0e8094710 0x7fb0e80bca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:01.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.307 7fb0ef958700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0e80ec490 0x7fb0e80bd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:01.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.307 7fb0ef958700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8093d90 con 0x7fb0e8094310 2021-02-10T11:49:01.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.307 7fb0ef958700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e80bd670 con 0x7fb0e8094710 2021-02-10T11:49:01.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.307 7fb0ef958700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e80939f0 con 0x7fb0e80ec490 2021-02-10T11:49:01.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.307 7fb0ed774700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0e8094310 0x7fb0e80ebe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:01.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.307 7fb0ed774700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0e8094310 0x7fb0e80ebe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35450/0 (socket says 172.21.2.101:35450) 2021-02-10T11:49:01.308 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.307 7fb0ed774700 1 -- 172.21.2.101:0/1755681335 learned_addr learned my addr 172.21.2.101:0/1755681335 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:01.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.308 7fb0ed774700 1 -- 172.21.2.101:0/1755681335 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0e80ec490 msgr2=0x7fb0e80bd050 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:49:01.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.308 7fb0ed774700 1 --2- 172.21.2.101:0/1755681335 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0e80ec490 0x7fb0e80bd050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:01.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.308 7fb0ed774700 1 -- 172.21.2.101:0/1755681335 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0e8094710 msgr2=0x7fb0e80bca30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:49:01.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.308 7fb0ed774700 1 --2- 172.21.2.101:0/1755681335 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0e8094710 0x7fb0e80bca30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:01.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.308 7fb0ed774700 1 -- 172.21.2.101:0/1755681335 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e80bda80 con 0x7fb0e8094310 2021-02-10T11:49:01.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.308 7fb0ed774700 1 --2- 172.21.2.101:0/1755681335 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0e8094310 0x7fb0e80ebe70 secure :-1 s=READY pgs=3323 cs=0 l=1 rev1=1 rx=0x7fb0e4007650 tx=0x7fb0e400a900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:01.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.308 7fb0ddffb700 1 -- 172.21.2.101:0/1755681335 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0e4000940 con 0x7fb0e8094310 2021-02-10T11:49:01.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.308 7fb0ddffb700 1 -- 172.21.2.101:0/1755681335 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e400acf0 con 0x7fb0e8094310 2021-02-10T11:49:01.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.308 7fb0ef958700 1 -- 172.21.2.101:0/1755681335 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e80bda80 con 0x7fb0e8094310 2021-02-10T11:49:01.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.308 7fb0ddffb700 1 -- 172.21.2.101:0/1755681335 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0e40045d0 con 0x7fb0e8094310 2021-02-10T11:49:01.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.308 7fb0ef958700 1 -- 172.21.2.101:0/1755681335 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e81a76d0 con 0x7fb0e8094310 2021-02-10T11:49:01.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.309 7fb0ddffb700 1 -- 172.21.2.101:0/1755681335 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb0e4004140 con 0x7fb0e8094310 2021-02-10T11:49:01.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.309 7fb0ddffb700 1 --2- 172.21.2.101:0/1755681335 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0c80186d0 0x7fb0c801abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:01.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.309 7fb0ddffb700 1 -- 172.21.2.101:0/1755681335 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb0e4032690 con 0x7fb0e8094310 2021-02-10T11:49:01.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.309 7fb0ecf73700 1 --2- 172.21.2.101:0/1755681335 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0c80186d0 0x7fb0c801abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:01.310 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.309 7fb0ecf73700 1 --2- 172.21.2.101:0/1755681335 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0c80186d0 0x7fb0c801abd0 crc :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:01.336 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.340 7fb0ef958700 1 -- 172.21.2.101:0/1755681335 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e80ebc40 con 0x7fb0e8094310 2021-02-10T11:49:01.338 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.342 7fb0ddffb700 1 -- 172.21.2.101:0/1755681335 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb0e4001870 con 0x7fb0e8094310 2021-02-10T11:49:01.487 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.490 7fb0ef958700 1 -- 172.21.2.101:0/1755681335 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb0e80f0b10 con 0x7fb0e8094310 2021-02-10T11:49:01.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.493 7fb0ddffb700 1 -- 172.21.2.101:0/1755681335 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v47) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb0e4004140 con 0x7fb0e8094310 2021-02-10T11:49:01.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.495 7fb0ef958700 1 -- 172.21.2.101:0/1755681335 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0c80186d0 msgr2=0x7fb0c801abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:01.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.495 7fb0ef958700 1 --2- 172.21.2.101:0/1755681335 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0c80186d0 0x7fb0c801abd0 crc :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:01.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.495 7fb0ef958700 1 -- 172.21.2.101:0/1755681335 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0e8094310 msgr2=0x7fb0e80ebe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:01.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.495 7fb0ef958700 1 --2- 172.21.2.101:0/1755681335 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0e8094310 0x7fb0e80ebe70 secure :-1 s=READY pgs=3323 cs=0 l=1 rev1=1 rx=0x7fb0e4007650 tx=0x7fb0e400a900).stop 2021-02-10T11:49:01.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.495 7fb0ef958700 1 -- 172.21.2.101:0/1755681335 shutdown_connections 2021-02-10T11:49:01.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.495 7fb0ef958700 1 --2- 172.21.2.101:0/1755681335 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0c80186d0 0x7fb0c801abd0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:01.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.495 7fb0ef958700 1 --2- 172.21.2.101:0/1755681335 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0e80ec490 0x7fb0e80bd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:01.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.495 7fb0ef958700 1 --2- 172.21.2.101:0/1755681335 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0e8094710 0x7fb0e80bca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:01.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.495 7fb0ef958700 1 --2- 172.21.2.101:0/1755681335 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0e8094310 0x7fb0e80ebe70 unknown :-1 s=CLOSED pgs=3323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:01.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.495 7fb0ef958700 1 -- 172.21.2.101:0/1755681335 >> 172.21.2.101:0/1755681335 conn(0x7fb0e808fa10 msgr2=0x7fb0e80bfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:01.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.495 7fb0ef958700 1 -- 172.21.2.101:0/1755681335 shutdown_connections 2021-02-10T11:49:01.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.495 7fb0ef958700 1 -- 172.21.2.101:0/1755681335 wait complete. 2021-02-10T11:49:01.499 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_subvolume_authorized_list' 2021-02-10T11:49:01.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.915 7fabe6dd0700 1 Processor -- start 2021-02-10T11:49:01.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.915 7fabe6dd0700 1 -- start start 2021-02-10T11:49:01.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.916 7fabe6dd0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fabe00d0340 0x7fabe00ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:01.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.916 7fabe6dd0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabe00d1c70 0x7fabe00efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:01.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.916 7fabe6dd0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe00f0b10 con 0x7fabe00d1870 2021-02-10T11:49:01.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.916 7fabe6dd0700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabe00e5ed0 con 0x7fabe00d0340 2021-02-10T11:49:01.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.916 7fabe6dd0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe01b0be0 con 0x7fabe00d1c70 2021-02-10T11:49:01.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.916 7fabdffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fabe00d0340 0x7fabe00ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:01.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.916 7fabdffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fabe00d0340 0x7fabe00ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35458/0 (socket says 172.21.2.101:35458) 2021-02-10T11:49:01.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.916 7fabdffff700 1 -- 172.21.2.101:0/3662789873 learned_addr learned my addr 172.21.2.101:0/3662789873 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:01.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.916 7fabe53ed700 1 --2- 172.21.2.101:0/3662789873 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabe00d1c70 0x7fabe00efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:01.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.917 7fabdffff700 1 -- 172.21.2.101:0/3662789873 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabe00d1c70 msgr2=0x7fabe00efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:01.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.917 7fabdffff700 1 --2- 172.21.2.101:0/3662789873 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabe00d1c70 0x7fabe00efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:01.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.917 7fabdffff700 1 -- 172.21.2.101:0/3662789873 >> v1:172.21.2.108:6789/0 conn(0x7fabe00d1870 legacy=0x7fabe01b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:01.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.917 7fabdffff700 1 -- 172.21.2.101:0/3662789873 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe00f20a0 con 0x7fabe00d0340 2021-02-10T11:49:01.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.917 7fabdffff700 1 --2- 172.21.2.101:0/3662789873 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabe00d0340 0x7fabe00ef640 secure :-1 s=READY pgs=3324 cs=0 l=1 rev1=1 rx=0x7fabd0006700 tx=0x7fabd0009b80).ready entity=mon.0 client_cookie=bcd7ee62156a1edf server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:01.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.917 7fabdf7fe700 1 -- 172.21.2.101:0/3662789873 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fabd000c0c0 con 0x7fabe00d0340 2021-02-10T11:49:01.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.917 7fabdf7fe700 1 -- 172.21.2.101:0/3662789873 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd000c400 con 0x7fabe00d0340 2021-02-10T11:49:01.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.917 7fabdf7fe700 1 -- 172.21.2.101:0/3662789873 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fabd000c920 con 0x7fabe00d0340 2021-02-10T11:49:01.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.917 7fabe6dd0700 1 -- 172.21.2.101:0/3662789873 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabe00d0340 msgr2=0x7fabe00ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:01.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.917 7fabe6dd0700 1 --2- 172.21.2.101:0/3662789873 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabe00d0340 0x7fabe00ef640 secure :-1 s=READY pgs=3324 cs=0 l=1 rev1=1 rx=0x7fabd0006700 tx=0x7fabd0009b80).stop 2021-02-10T11:49:01.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.918 7fabe6dd0700 1 -- 172.21.2.101:0/3662789873 shutdown_connections 2021-02-10T11:49:01.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.918 7fabe6dd0700 1 --2- 172.21.2.101:0/3662789873 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabe00d1c70 0x7fabe00efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:01.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.918 7fabe6dd0700 1 --2- 172.21.2.101:0/3662789873 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabe00d0340 0x7fabe00ef640 unknown :-1 s=CLOSED pgs=3324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:01.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.918 7fabe6dd0700 1 -- 172.21.2.101:0/3662789873 >> 172.21.2.101:0/3662789873 conn(0x7fabe00ce450 msgr2=0x7fabe00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:01.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.918 7fabe6dd0700 1 -- 172.21.2.101:0/3662789873 shutdown_connections 2021-02-10T11:49:01.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.918 7fabe6dd0700 1 -- 172.21.2.101:0/3662789873 wait complete. 2021-02-10T11:49:01.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.932 7fabe6dd0700 1 Processor -- start 2021-02-10T11:49:01.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.932 7fabe6dd0700 1 -- start start 2021-02-10T11:49:01.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.933 7fabe6dd0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabe0195650 0x7fabe00b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:01.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.933 7fabe6dd0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabe00b5b80 0x7fabe01a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:01.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.933 7fabe6dd0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabe00acfa0 0x7fabe01a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:01.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.933 7fabe6dd0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe00b6010 con 0x7fabe00b5b80 2021-02-10T11:49:01.932 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.933 7fabe6dd0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe01a3a60 con 0x7fabe0195650 2021-02-10T11:49:01.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.933 7fabe6dd0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe01a3c40 con 0x7fabe00acfa0 2021-02-10T11:49:01.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.933 7fabdffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabe00b5b80 0x7fabe01a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:01.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.933 7fabdffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabe00b5b80 0x7fabe01a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35464/0 (socket says 172.21.2.101:35464) 2021-02-10T11:49:01.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.933 7fabdffff700 1 -- 172.21.2.101:0/66968605 learned_addr learned my addr 172.21.2.101:0/66968605 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:01.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.933 7fabe4bec700 1 --2- 172.21.2.101:0/66968605 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabe0195650 0x7fabe00b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:01.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.933 7fabe53ed700 1 --2- 172.21.2.101:0/66968605 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabe00acfa0 0x7fabe01a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:01.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.933 7fabdffff700 1 -- 172.21.2.101:0/66968605 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabe00acfa0 msgr2=0x7fabe01a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:01.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.933 7fabdffff700 1 --2- 172.21.2.101:0/66968605 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabe00acfa0 0x7fabe01a3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:01.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.933 7fabdffff700 1 -- 172.21.2.101:0/66968605 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabe0195650 msgr2=0x7fabe00b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:01.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.933 7fabdffff700 1 --2- 172.21.2.101:0/66968605 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabe0195650 0x7fabe00b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:01.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.933 7fabdffff700 1 -- 172.21.2.101:0/66968605 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe00bcae0 con 0x7fabe00b5b80 2021-02-10T11:49:01.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.933 7fabe53ed700 1 --2- 172.21.2.101:0/66968605 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabe00acfa0 0x7fabe01a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:49:01.933 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.934 7fabdffff700 1 --2- 172.21.2.101:0/66968605 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabe00b5b80 0x7fabe01a2e20 secure :-1 s=READY pgs=3325 cs=0 l=1 rev1=1 rx=0x7fabd00066a0 tx=0x7fabd000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:01.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.934 7fabdd7fa700 1 -- 172.21.2.101:0/66968605 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fabd000bd80 con 0x7fabe00b5b80 2021-02-10T11:49:01.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.934 7fabdd7fa700 1 -- 172.21.2.101:0/66968605 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd00052e0 con 0x7fabe00b5b80 2021-02-10T11:49:01.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.934 7fabe6dd0700 1 -- 172.21.2.101:0/66968605 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe00bcae0 con 0x7fabe00b5b80 2021-02-10T11:49:01.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.934 7fabdd7fa700 1 -- 172.21.2.101:0/66968605 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fabd00057c0 con 0x7fabe00b5b80 2021-02-10T11:49:01.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.934 7fabe6dd0700 1 -- 172.21.2.101:0/66968605 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe00bcae0 con 0x7fabe00b5b80 2021-02-10T11:49:01.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.934 7fabdd7fa700 1 -- 172.21.2.101:0/66968605 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fabd00054c0 con 0x7fabe00b5b80 2021-02-10T11:49:01.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.934 7fabdd7fa700 1 --2- 172.21.2.101:0/66968605 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fabc40186c0 0x7fabc401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:01.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.935 7fabdd7fa700 1 -- 172.21.2.101:0/66968605 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fabd0032790 con 0x7fabe00b5b80 2021-02-10T11:49:01.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.935 7fabe4bec700 1 --2- 172.21.2.101:0/66968605 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fabc40186c0 0x7fabc401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:01.934 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.935 7fabe4bec700 1 --2- 172.21.2.101:0/66968605 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fabc40186c0 0x7fabc401abc0 crc :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:01.962 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.966 7fabe6dd0700 1 -- 172.21.2.101:0/66968605 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe01a3c40 con 0x7fabe00b5b80 2021-02-10T11:49:01.964 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:01.969 7fabdd7fa700 1 -- 172.21.2.101:0/66968605 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fabd00057c0 con 0x7fabe00b5b80 2021-02-10T11:49:02.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:02.103 7fabe6dd0700 1 -- 172.21.2.101:0/66968605 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_subvolume_authorized_list"]} v 0) v1 -- 0x7fabe01a3c40 con 0x7fabe00b5b80 2021-02-10T11:49:02.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:02.952 7fabdd7fa700 1 -- 172.21.2.101:0/66968605 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_subvolume_authorized_list"]}]=0 v904) v1 ==== 147+0+0 (secure 0 0 0) 0x7fabd0005470 con 0x7fabe00b5b80 2021-02-10T11:49:02.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:02.974 7fabe6dd0700 1 -- 172.21.2.101:0/66968605 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fabc40186c0 msgr2=0x7fabc401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:02.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:02.974 7fabe6dd0700 1 --2- 172.21.2.101:0/66968605 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fabc40186c0 0x7fabc401abc0 crc :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:02.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:02.974 7fabe6dd0700 1 -- 172.21.2.101:0/66968605 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabe00b5b80 msgr2=0x7fabe01a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:02.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:02.974 7fabe6dd0700 1 --2- 172.21.2.101:0/66968605 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabe00b5b80 0x7fabe01a2e20 secure :-1 s=READY pgs=3325 cs=0 l=1 rev1=1 rx=0x7fabd00066a0 tx=0x7fabd000b990).stop 2021-02-10T11:49:02.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:02.975 7fabe6dd0700 1 -- 172.21.2.101:0/66968605 shutdown_connections 2021-02-10T11:49:02.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:02.975 7fabe6dd0700 1 --2- 172.21.2.101:0/66968605 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fabc40186c0 0x7fabc401abc0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:02.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:02.975 7fabe6dd0700 1 --2- 172.21.2.101:0/66968605 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fabe00acfa0 0x7fabe01a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:02.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:02.975 7fabe6dd0700 1 --2- 172.21.2.101:0/66968605 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fabe00b5b80 0x7fabe01a2e20 unknown :-1 s=CLOSED pgs=3325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:02.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:02.975 7fabe6dd0700 1 --2- 172.21.2.101:0/66968605 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fabe0195650 0x7fabe00b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:02.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:02.975 7fabe6dd0700 1 -- 172.21.2.101:0/66968605 >> 172.21.2.101:0/66968605 conn(0x7fabe00b3040 msgr2=0x7fabe00c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:02.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:02.975 7fabe6dd0700 1 -- 172.21.2.101:0/66968605 shutdown_connections 2021-02-10T11:49:02.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:02.975 7fabe6dd0700 1 -- 172.21.2.101:0/66968605 wait complete. 2021-02-10T11:49:02.979 INFO:tasks.cephfs_test_runner:test_subvolume_authorized_list (tasks.cephfs.test_volumes.TestVolumes) ... ok 2021-02-10T11:49:02.979 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_clone_in_progress_getpath (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:49:02.980 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_subvolume_clone_in_progress_getpath' 2021-02-10T11:49:03.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.395 7ff559f90700 1 Processor -- start 2021-02-10T11:49:03.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.395 7ff559f90700 1 -- start start 2021-02-10T11:49:03.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.396 7ff559f90700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff5541b0470 0x7ff5541b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:03.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.396 7ff559f90700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff5540ef610 0x7ff5540efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:03.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.396 7ff559f90700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5540f0030 0x7ff5540f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:03.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.396 7ff559f90700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5540e5ea0 con 0x7ff5541b0470 2021-02-10T11:49:03.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.396 7ff559f90700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5540f0b50 con 0x7ff5540ef610 2021-02-10T11:49:03.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.396 7ff559f90700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5540e52e0 con 0x7ff5540f0030 2021-02-10T11:49:03.408 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.396 7ff5537fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff5541b0470 0x7ff5541b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:03.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.396 7ff5537fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff5541b0470 0x7ff5541b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35472/0 (socket says 172.21.2.101:35472) 2021-02-10T11:49:03.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.396 7ff5537fe700 1 -- 172.21.2.101:0/2271712097 learned_addr learned my addr 172.21.2.101:0/2271712097 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:03.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.396 7ff5537fe700 1 -- 172.21.2.101:0/2271712097 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5540f0030 msgr2=0x7ff5540f0430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:49:03.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.396 7ff5537fe700 1 --2- 172.21.2.101:0/2271712097 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5540f0030 0x7ff5540f0430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:03.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.396 7ff5537fe700 1 -- 172.21.2.101:0/2271712097 >> v2:172.21.2.108:3300/0 conn(0x7ff5540ef610 msgr2=0x7ff5540efa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:49:03.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.396 7ff5537fe700 1 --2- 172.21.2.101:0/2271712097 >> v2:172.21.2.108:3300/0 conn(0x7ff5540ef610 0x7ff5540efa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:03.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.396 7ff5537fe700 1 -- 172.21.2.101:0/2271712097 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5540f10d0 con 0x7ff5541b0470 2021-02-10T11:49:03.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.397 7ff5537fe700 1 --2- 172.21.2.101:0/2271712097 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5541b0470 0x7ff5541b0870 secure :-1 s=READY pgs=3326 cs=0 l=1 rev1=1 rx=0x7ff544006700 tx=0x7ff544009b80).ready entity=mon.0 client_cookie=277ca35db1a0bd54 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:03.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.397 7ff5527fc700 1 -- 172.21.2.101:0/2271712097 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff54400c0c0 con 0x7ff5541b0470 2021-02-10T11:49:03.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.397 7ff5527fc700 1 -- 172.21.2.101:0/2271712097 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54400c340 con 0x7ff5541b0470 2021-02-10T11:49:03.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.397 7ff5527fc700 1 -- 172.21.2.101:0/2271712097 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff54400c820 con 0x7ff5541b0470 2021-02-10T11:49:03.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.397 7ff559f90700 1 -- 172.21.2.101:0/2271712097 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5541b0470 msgr2=0x7ff5541b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:03.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.397 7ff559f90700 1 --2- 172.21.2.101:0/2271712097 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5541b0470 0x7ff5541b0870 secure :-1 s=READY pgs=3326 cs=0 l=1 rev1=1 rx=0x7ff544006700 tx=0x7ff544009b80).stop 2021-02-10T11:49:03.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.397 7ff559f90700 1 -- 172.21.2.101:0/2271712097 shutdown_connections 2021-02-10T11:49:03.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.397 7ff559f90700 1 --2- 172.21.2.101:0/2271712097 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff5540f0030 0x7ff5540f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:03.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.397 7ff559f90700 1 --2- 172.21.2.101:0/2271712097 >> v2:172.21.2.108:3300/0 conn(0x7ff5540ef610 0x7ff5540efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:03.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.397 7ff559f90700 1 --2- 172.21.2.101:0/2271712097 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff5541b0470 0x7ff5541b0870 unknown :-1 s=CLOSED pgs=3326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:03.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.397 7ff559f90700 1 -- 172.21.2.101:0/2271712097 >> 172.21.2.101:0/2271712097 conn(0x7ff5540ce3e0 msgr2=0x7ff5540e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:03.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.398 7ff559f90700 1 -- 172.21.2.101:0/2271712097 shutdown_connections 2021-02-10T11:49:03.410 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.398 7ff559f90700 1 -- 172.21.2.101:0/2271712097 wait complete. 2021-02-10T11:49:03.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.412 7ff559f90700 1 Processor -- start 2021-02-10T11:49:03.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.412 7ff559f90700 1 -- start start 2021-02-10T11:49:03.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.412 7ff559f90700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff554188110 0x7ff554188510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:03.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.412 7ff559f90700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff554188b30 0x7ff55418ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:03.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.412 7ff559f90700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff55418f200 0x7ff55418f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:03.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.412 7ff559f90700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5540aba90 con 0x7ff55418f200 2021-02-10T11:49:03.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.412 7ff559f90700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff554189190 con 0x7ff554188110 2021-02-10T11:49:03.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.412 7ff559f90700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff554191620 con 0x7ff554188b30 2021-02-10T11:49:03.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.413 7ff553fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff55418f200 0x7ff55418f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:03.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.413 7ff553fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff55418f200 0x7ff55418f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35478/0 (socket says 172.21.2.101:35478) 2021-02-10T11:49:03.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.413 7ff553fff700 1 -- 172.21.2.101:0/3027017932 learned_addr learned my addr 172.21.2.101:0/3027017932 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:03.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.413 7ff553fff700 1 -- 172.21.2.101:0/3027017932 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff554188b30 msgr2=0x7ff55418ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:03.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.413 7ff553fff700 1 --2- 172.21.2.101:0/3027017932 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff554188b30 0x7ff55418ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:03.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.413 7ff553fff700 1 -- 172.21.2.101:0/3027017932 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff554188110 msgr2=0x7ff554188510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:03.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.413 7ff553fff700 1 --2- 172.21.2.101:0/3027017932 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff554188110 0x7ff554188510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:03.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.413 7ff553fff700 1 -- 172.21.2.101:0/3027017932 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55418fc20 con 0x7ff55418f200 2021-02-10T11:49:03.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.413 7ff553fff700 1 --2- 172.21.2.101:0/3027017932 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff55418f200 0x7ff55418f600 secure :-1 s=READY pgs=3327 cs=0 l=1 rev1=1 rx=0x7ff54c009960 tx=0x7ff54c00bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:03.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.413 7ff53bfff700 1 -- 172.21.2.101:0/3027017932 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff54c000f70 con 0x7ff55418f200 2021-02-10T11:49:03.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.413 7ff53bfff700 1 -- 172.21.2.101:0/3027017932 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54c003030 con 0x7ff55418f200 2021-02-10T11:49:03.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.413 7ff559f90700 1 -- 172.21.2.101:0/3027017932 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55418fc20 con 0x7ff55418f200 2021-02-10T11:49:03.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.413 7ff559f90700 1 -- 172.21.2.101:0/3027017932 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5541904d0 con 0x7ff55418f200 2021-02-10T11:49:03.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.413 7ff53bfff700 1 -- 172.21.2.101:0/3027017932 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff54c001590 con 0x7ff55418f200 2021-02-10T11:49:03.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.414 7ff53bfff700 1 -- 172.21.2.101:0/3027017932 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff54c00a030 con 0x7ff55418f200 2021-02-10T11:49:03.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.414 7ff53bfff700 1 --2- 172.21.2.101:0/3027017932 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5340186c0 0x7ff53401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:03.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.414 7ff53bfff700 1 -- 172.21.2.101:0/3027017932 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff54c032b00 con 0x7ff55418f200 2021-02-10T11:49:03.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.414 7ff5537fe700 1 --2- 172.21.2.101:0/3027017932 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5340186c0 0x7ff53401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:03.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.414 7ff5537fe700 1 --2- 172.21.2.101:0/3027017932 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5340186c0 0x7ff53401abc0 crc :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:03.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.446 7ff559f90700 1 -- 172.21.2.101:0/3027017932 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff554189190 con 0x7ff55418f200 2021-02-10T11:49:03.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.449 7ff53bfff700 1 -- 172.21.2.101:0/3027017932 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff54c000ce0 con 0x7ff55418f200 2021-02-10T11:49:03.579 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.583 7ff559f90700 1 -- 172.21.2.101:0/3027017932 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_subvolume_clone_in_progress_getpath"]} v 0) v1 -- 0x7ff554189190 con 0x7ff55418f200 2021-02-10T11:49:03.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:03.956 7ff53bfff700 1 -- 172.21.2.101:0/3027017932 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_subvolume_clone_in_progress_getpath"]}]=0 v905) v1 ==== 160+0+0 (secure 0 0 0) 0x7ff54c00a030 con 0x7ff55418f200 2021-02-10T11:49:03.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.002 7ff559f90700 1 -- 172.21.2.101:0/3027017932 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5340186c0 msgr2=0x7ff53401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:03.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.002 7ff559f90700 1 --2- 172.21.2.101:0/3027017932 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5340186c0 0x7ff53401abc0 crc :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:03.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.002 7ff559f90700 1 -- 172.21.2.101:0/3027017932 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff55418f200 msgr2=0x7ff55418f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:03.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.002 7ff559f90700 1 --2- 172.21.2.101:0/3027017932 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff55418f200 0x7ff55418f600 secure :-1 s=READY pgs=3327 cs=0 l=1 rev1=1 rx=0x7ff54c009960 tx=0x7ff54c00bcf0).stop 2021-02-10T11:49:03.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.002 7ff559f90700 1 -- 172.21.2.101:0/3027017932 shutdown_connections 2021-02-10T11:49:03.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.002 7ff559f90700 1 --2- 172.21.2.101:0/3027017932 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff5340186c0 0x7ff53401abc0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:03.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.002 7ff559f90700 1 --2- 172.21.2.101:0/3027017932 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff554188b30 0x7ff55418ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:03.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.002 7ff559f90700 1 --2- 172.21.2.101:0/3027017932 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff55418f200 0x7ff55418f600 unknown :-1 s=CLOSED pgs=3327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:03.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.002 7ff559f90700 1 --2- 172.21.2.101:0/3027017932 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff554188110 0x7ff554188510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:03.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.002 7ff559f90700 1 -- 172.21.2.101:0/3027017932 >> 172.21.2.101:0/3027017932 conn(0x7ff55419f7b0 msgr2=0x7ff5541af800 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:03.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.002 7ff559f90700 1 -- 172.21.2.101:0/3027017932 shutdown_connections 2021-02-10T11:49:04.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.002 7ff559f90700 1 -- 172.21.2.101:0/3027017932 wait complete. 2021-02-10T11:49:04.006 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:49:04.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.426 7f8677500700 1 Processor -- start 2021-02-10T11:49:04.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.426 7f8677500700 1 -- start start 2021-02-10T11:49:04.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.427 7f8677500700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f86700d1c70 0x7f86700cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:04.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.427 7f8677500700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f86700cf580 0x7f86700cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:04.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.427 7f8677500700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86700fe1c0 con 0x7f86700d1870 2021-02-10T11:49:04.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.427 7f8677500700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86701b0cb0 con 0x7f86700cf580 2021-02-10T11:49:04.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.427 7f8677500700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8670109a60 con 0x7f86700d1c70 2021-02-10T11:49:04.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.427 7f8675b1d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f86700cf580 0x7f86700cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:04.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.427 7f8675b1d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f86700cf580 0x7f86700cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35482/0 (socket says 172.21.2.101:35482) 2021-02-10T11:49:04.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.427 7f8675b1d700 1 -- 172.21.2.101:0/1145376013 learned_addr learned my addr 172.21.2.101:0/1145376013 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:04.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.427 7f8675b1d700 1 -- 172.21.2.101:0/1145376013 >> v2:172.21.2.108:3300/0 conn(0x7f86700d1c70 msgr2=0x7f86700cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:04.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.427 7f8675b1d700 1 --2- 172.21.2.101:0/1145376013 >> v2:172.21.2.108:3300/0 conn(0x7f86700d1c70 0x7f86700cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:04.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.427 7f8675b1d700 1 -- 172.21.2.101:0/1145376013 >> v1:172.21.2.108:6789/0 conn(0x7f86700d1870 legacy=0x7f86701b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:04.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.427 7f8675b1d700 1 -- 172.21.2.101:0/1145376013 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8670088070 con 0x7f86700cf580 2021-02-10T11:49:04.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.427 7f8675b1d700 1 --2- 172.21.2.101:0/1145376013 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86700cf580 0x7f86700cf980 secure :-1 s=READY pgs=3328 cs=0 l=1 rev1=1 rx=0x7f8668006f00 tx=0x7f8668009f10).ready entity=mon.0 client_cookie=94538bfca0cb102d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:04.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.427 7f8667fff700 1 -- 172.21.2.101:0/1145376013 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f866800b3b0 con 0x7f86700cf580 2021-02-10T11:49:04.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.427 7f8667fff700 1 -- 172.21.2.101:0/1145376013 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866800b630 con 0x7f86700cf580 2021-02-10T11:49:04.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.427 7f8667fff700 1 -- 172.21.2.101:0/1145376013 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f866800bb50 con 0x7f86700cf580 2021-02-10T11:49:04.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.427 7f8677500700 1 -- 172.21.2.101:0/1145376013 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86700cf580 msgr2=0x7f86700cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:04.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.427 7f8677500700 1 --2- 172.21.2.101:0/1145376013 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86700cf580 0x7f86700cf980 secure :-1 s=READY pgs=3328 cs=0 l=1 rev1=1 rx=0x7f8668006f00 tx=0x7f8668009f10).stop 2021-02-10T11:49:04.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.428 7f8677500700 1 -- 172.21.2.101:0/1145376013 shutdown_connections 2021-02-10T11:49:04.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.428 7f8677500700 1 --2- 172.21.2.101:0/1145376013 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86700cf580 0x7f86700cf980 unknown :-1 s=CLOSED pgs=3328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:04.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.428 7f8677500700 1 --2- 172.21.2.101:0/1145376013 >> v2:172.21.2.108:3300/0 conn(0x7f86700d1c70 0x7f86700cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:04.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.428 7f8677500700 1 -- 172.21.2.101:0/1145376013 >> 172.21.2.101:0/1145376013 conn(0x7f86700ce420 msgr2=0x7f86700d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:04.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.428 7f8677500700 1 -- 172.21.2.101:0/1145376013 shutdown_connections 2021-02-10T11:49:04.440 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.428 7f8677500700 1 -- 172.21.2.101:0/1145376013 wait complete. 2021-02-10T11:49:04.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.443 7f8677500700 1 Processor -- start 2021-02-10T11:49:04.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.443 7f8677500700 1 -- start start 2021-02-10T11:49:04.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.443 7f8677500700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86700c3820 0x7f86700a7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:04.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.443 7f8677500700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86701a44c0 0x7f86701a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:04.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.443 7f8677500700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86700863d0 0x7f86700867d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:04.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.443 7f8677500700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8670085ed0 con 0x7f86700863d0 2021-02-10T11:49:04.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.443 7f8677500700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86700c0740 con 0x7f86700c3820 2021-02-10T11:49:04.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.443 7f8677500700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8670086df0 con 0x7f86701a44c0 2021-02-10T11:49:04.442 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.443 7f8675b1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86700863d0 0x7f86700867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:04.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.443 7f8675b1d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86700863d0 0x7f86700867d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35490/0 (socket says 172.21.2.101:35490) 2021-02-10T11:49:04.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.443 7f8675b1d700 1 -- 172.21.2.101:0/3252274616 learned_addr learned my addr 172.21.2.101:0/3252274616 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:04.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.444 7f8675b1d700 1 -- 172.21.2.101:0/3252274616 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86701a44c0 msgr2=0x7f86701a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:04.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.444 7f8675b1d700 1 --2- 172.21.2.101:0/3252274616 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86701a44c0 0x7f86701a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:04.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.444 7f8675b1d700 1 -- 172.21.2.101:0/3252274616 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86700c3820 msgr2=0x7f86700a7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:04.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.444 7f8675b1d700 1 --2- 172.21.2.101:0/3252274616 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86700c3820 0x7f86700a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:04.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.444 7f8675b1d700 1 -- 172.21.2.101:0/3252274616 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8670087010 con 0x7f86700863d0 2021-02-10T11:49:04.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.444 7f8675b1d700 1 --2- 172.21.2.101:0/3252274616 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86700863d0 0x7f86700867d0 secure :-1 s=READY pgs=3329 cs=0 l=1 rev1=1 rx=0x7f8668009970 tx=0x7f866800bcc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:04.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.444 7f8665ffb700 1 -- 172.21.2.101:0/3252274616 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8668009450 con 0x7f86700863d0 2021-02-10T11:49:04.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.444 7f8677500700 1 -- 172.21.2.101:0/3252274616 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8670087010 con 0x7f86700863d0 2021-02-10T11:49:04.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.444 7f8677500700 1 -- 172.21.2.101:0/3252274616 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8670083000 con 0x7f86700863d0 2021-02-10T11:49:04.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.444 7f8665ffb700 1 -- 172.21.2.101:0/3252274616 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8668003080 con 0x7f86700863d0 2021-02-10T11:49:04.443 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.444 7f8665ffb700 1 -- 172.21.2.101:0/3252274616 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8668009140 con 0x7f86700863d0 2021-02-10T11:49:04.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.445 7f8665ffb700 1 -- 172.21.2.101:0/3252274616 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8668001270 con 0x7f86700863d0 2021-02-10T11:49:04.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.445 7f8665ffb700 1 --2- 172.21.2.101:0/3252274616 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8650018730 0x7f865001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:04.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.445 7f8665ffb700 1 -- 172.21.2.101:0/3252274616 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8668032b80 con 0x7f86700863d0 2021-02-10T11:49:04.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.445 7f867531c700 1 --2- 172.21.2.101:0/3252274616 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8650018730 0x7f865001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:04.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.445 7f867531c700 1 --2- 172.21.2.101:0/3252274616 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8650018730 0x7f865001ac10 crc :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:04.472 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.476 7f8677500700 1 -- 172.21.2.101:0/3252274616 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86700fe1c0 con 0x7f86700863d0 2021-02-10T11:49:04.475 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.479 7f8665ffb700 1 -- 172.21.2.101:0/3252274616 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f86680068e0 con 0x7f86700863d0 2021-02-10T11:49:04.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.627 7f8677500700 1 -- 172.21.2.101:0/3252274616 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f86700fe1c0 con 0x7f86700863d0 2021-02-10T11:49:04.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.631 7f8665ffb700 1 -- 172.21.2.101:0/3252274616 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) v1 ==== 122+0+0 (secure 0 0 0) 0x7f866801f8c0 con 0x7f86700863d0 2021-02-10T11:49:04.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.636 7f8677500700 1 -- 172.21.2.101:0/3252274616 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8650018730 msgr2=0x7f865001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:04.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.636 7f8677500700 1 --2- 172.21.2.101:0/3252274616 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8650018730 0x7f865001ac10 crc :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:04.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.636 7f8677500700 1 -- 172.21.2.101:0/3252274616 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86700863d0 msgr2=0x7f86700867d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:04.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.636 7f8677500700 1 --2- 172.21.2.101:0/3252274616 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86700863d0 0x7f86700867d0 secure :-1 s=READY pgs=3329 cs=0 l=1 rev1=1 rx=0x7f8668009970 tx=0x7f866800bcc0).stop 2021-02-10T11:49:04.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.636 7f8677500700 1 -- 172.21.2.101:0/3252274616 shutdown_connections 2021-02-10T11:49:04.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.636 7f8677500700 1 --2- 172.21.2.101:0/3252274616 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8650018730 0x7f865001ac10 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:04.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.636 7f8677500700 1 --2- 172.21.2.101:0/3252274616 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86701a44c0 0x7f86701a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:04.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.636 7f8677500700 1 --2- 172.21.2.101:0/3252274616 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86700863d0 0x7f86700867d0 unknown :-1 s=CLOSED pgs=3329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:04.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.636 7f8677500700 1 --2- 172.21.2.101:0/3252274616 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86700c3820 0x7f86700a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:04.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.636 7f8677500700 1 -- 172.21.2.101:0/3252274616 >> 172.21.2.101:0/3252274616 conn(0x7f86700872c0 msgr2=0x7f8670095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:04.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.637 7f8677500700 1 -- 172.21.2.101:0/3252274616 shutdown_connections 2021-02-10T11:49:04.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:04.637 7f8677500700 1 -- 172.21.2.101:0/3252274616 wait complete. 2021-02-10T11:49:04.641 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:49:04.641 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:49:04.987 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:49:05.332 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:49:05.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:05.332 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:05.332 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:05.668 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2021-02-10T11:49:05.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:49:05.669 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-02-10T11:49:05.669 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:05.669 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:05.669 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:49:06.004 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:49:06.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:49:06.005 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:49:06.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:06.005 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:06.005 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:49:06.340 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:49:06.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:49:06.340 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:49:06.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:06.341 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:06.341 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:49:06.676 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:49:06.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:49:06.676 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:49:06.676 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json-pretty 2021-02-10T11:49:07.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.093 7f1f8e158700 1 Processor -- start 2021-02-10T11:49:07.105 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.093 7f1f8e158700 1 -- start start 2021-02-10T11:49:07.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.094 7f1f8e158700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f1f880d0340 0x7f1f880ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:07.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.094 7f1f8e158700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1f880d1c70 0x7f1f880efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:07.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.094 7f1f8e158700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f880f0b10 con 0x7f1f880d1870 2021-02-10T11:49:07.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.094 7f1f8e158700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f880e5ed0 con 0x7f1f880d1c70 2021-02-10T11:49:07.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.094 7f1f8e158700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f881b0be0 con 0x7f1f880d0340 2021-02-10T11:49:07.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.094 7f1f87fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1f880d1c70 0x7f1f880efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:07.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.094 7f1f877fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f1f880d1870 0x7f1f881b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35988/0 (socket says 172.21.2.101:35988) 2021-02-10T11:49:07.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.094 7f1f877fe700 1 -- 172.21.2.101:0/367005442 learned_addr learned my addr 172.21.2.101:0/367005442 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:07.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.094 7f1f87fff700 1 -- 172.21.2.101:0/367005442 >> v2:172.21.2.108:3300/0 conn(0x7f1f880d0340 msgr2=0x7f1f880ef640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:49:07.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.094 7f1f87fff700 1 --2- 172.21.2.101:0/367005442 >> v2:172.21.2.108:3300/0 conn(0x7f1f880d0340 0x7f1f880ef640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:07.106 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.094 7f1f87fff700 1 -- 172.21.2.101:0/367005442 >> v1:172.21.2.101:6789/0 conn(0x7f1f880d1870 legacy=0x7f1f881b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:07.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.094 7f1f87fff700 1 -- 172.21.2.101:0/367005442 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f880f20a0 con 0x7f1f880d1c70 2021-02-10T11:49:07.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.094 7f1f87fff700 1 --2- 172.21.2.101:0/367005442 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f880d1c70 0x7f1f880efc60 secure :-1 s=READY pgs=3331 cs=0 l=1 rev1=1 rx=0x7f1f80006e30 tx=0x7f1f80009de0).ready entity=mon.0 client_cookie=1938b7ff508d49f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:07.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.095 7f1f867fc700 1 -- 172.21.2.101:0/367005442 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f8000b300 con 0x7f1f880d1c70 2021-02-10T11:49:07.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.095 7f1f867fc700 1 -- 172.21.2.101:0/367005442 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8000b580 con 0x7f1f880d1c70 2021-02-10T11:49:07.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.095 7f1f867fc700 1 -- 172.21.2.101:0/367005442 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f8000baa0 con 0x7f1f880d1c70 2021-02-10T11:49:07.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.095 7f1f8e158700 1 -- 172.21.2.101:0/367005442 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f880d1c70 msgr2=0x7f1f880efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:07.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.095 7f1f8e158700 1 --2- 172.21.2.101:0/367005442 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f880d1c70 0x7f1f880efc60 secure :-1 s=READY pgs=3331 cs=0 l=1 rev1=1 rx=0x7f1f80006e30 tx=0x7f1f80009de0).stop 2021-02-10T11:49:07.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.095 7f1f8e158700 1 -- 172.21.2.101:0/367005442 shutdown_connections 2021-02-10T11:49:07.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.095 7f1f8e158700 1 --2- 172.21.2.101:0/367005442 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f880d1c70 0x7f1f880efc60 unknown :-1 s=CLOSED pgs=3331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:07.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.095 7f1f8e158700 1 --2- 172.21.2.101:0/367005442 >> v2:172.21.2.108:3300/0 conn(0x7f1f880d0340 0x7f1f880ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:07.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.095 7f1f8e158700 1 -- 172.21.2.101:0/367005442 >> 172.21.2.101:0/367005442 conn(0x7f1f880ce450 msgr2=0x7f1f880d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:07.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.095 7f1f8e158700 1 -- 172.21.2.101:0/367005442 shutdown_connections 2021-02-10T11:49:07.107 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.095 7f1f8e158700 1 -- 172.21.2.101:0/367005442 wait complete. 2021-02-10T11:49:07.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.110 7f1f8e158700 1 Processor -- start 2021-02-10T11:49:07.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.110 7f1f8e158700 1 -- start start 2021-02-10T11:49:07.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.110 7f1f8e158700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f8818f0a0 0x7f1f88196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:07.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.110 7f1f8e158700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f881970d0 0x7f1f881974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:07.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.110 7f1f8e158700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f88094650 0x7f1f88094a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:07.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.110 7f1f8e158700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f88095bd0 con 0x7f1f881970d0 2021-02-10T11:49:07.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.110 7f1f8e158700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f880ad7c0 con 0x7f1f8818f0a0 2021-02-10T11:49:07.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.110 7f1f8e158700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8818ec70 con 0x7f1f88094650 2021-02-10T11:49:07.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.110 7f1f86ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f881970d0 0x7f1f881974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:07.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.110 7f1f86ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f881970d0 0x7f1f881974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35500/0 (socket says 172.21.2.101:35500) 2021-02-10T11:49:07.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.110 7f1f86ffd700 1 -- 172.21.2.101:0/1489682268 learned_addr learned my addr 172.21.2.101:0/1489682268 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:07.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.111 7f1f86ffd700 1 -- 172.21.2.101:0/1489682268 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f88094650 msgr2=0x7f1f88094a50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:49:07.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.111 7f1f86ffd700 1 --2- 172.21.2.101:0/1489682268 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f88094650 0x7f1f88094a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:07.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.111 7f1f86ffd700 1 -- 172.21.2.101:0/1489682268 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f8818f0a0 msgr2=0x7f1f88196ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:49:07.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.111 7f1f86ffd700 1 --2- 172.21.2.101:0/1489682268 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f8818f0a0 0x7f1f88196ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:07.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.111 7f1f86ffd700 1 -- 172.21.2.101:0/1489682268 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f88095070 con 0x7f1f881970d0 2021-02-10T11:49:07.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.111 7f1f86ffd700 1 --2- 172.21.2.101:0/1489682268 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f881970d0 0x7f1f881974d0 secure :-1 s=READY pgs=3332 cs=0 l=1 rev1=1 rx=0x7f1f7c00b5c0 tx=0x7f1f7c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:07.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.111 7f1f6ffff700 1 -- 172.21.2.101:0/1489682268 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f7c00bd50 con 0x7f1f881970d0 2021-02-10T11:49:07.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.111 7f1f6ffff700 1 -- 172.21.2.101:0/1489682268 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7c006030 con 0x7f1f881970d0 2021-02-10T11:49:07.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.111 7f1f8e158700 1 -- 172.21.2.101:0/1489682268 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f88095070 con 0x7f1f881970d0 2021-02-10T11:49:07.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.111 7f1f6ffff700 1 -- 172.21.2.101:0/1489682268 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f7c0054d0 con 0x7f1f881970d0 2021-02-10T11:49:07.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.111 7f1f8e158700 1 -- 172.21.2.101:0/1489682268 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f88095600 con 0x7f1f881970d0 2021-02-10T11:49:07.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.111 7f1f6ffff700 1 -- 172.21.2.101:0/1489682268 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1f7c006210 con 0x7f1f881970d0 2021-02-10T11:49:07.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.112 7f1f6ffff700 1 --2- 172.21.2.101:0/1489682268 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f68018730 0x7f1f6801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:07.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.112 7f1f6ffff700 1 -- 172.21.2.101:0/1489682268 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1f7c032640 con 0x7f1f881970d0 2021-02-10T11:49:07.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.112 7f1f877fe700 1 --2- 172.21.2.101:0/1489682268 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f68018730 0x7f1f6801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:07.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.112 7f1f877fe700 1 --2- 172.21.2.101:0/1489682268 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f68018730 0x7f1f6801ac10 crc :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:07.139 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.143 7f1f8e158700 1 -- 172.21.2.101:0/1489682268 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f880ad790 con 0x7f1f881970d0 2021-02-10T11:49:07.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.146 7f1f6ffff700 1 -- 172.21.2.101:0/1489682268 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1f7c000960 con 0x7f1f881970d0 2021-02-10T11:49:07.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.272 7f1f8e158700 1 -- 172.21.2.101:0/1489682268 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f1f880ad790 con 0x7f1f881970d0 2021-02-10T11:49:07.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.272 7f1f6ffff700 1 -- 172.21.2.101:0/1489682268 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v191) v1 ==== 81+0+30374 (secure 0 0 0) 0x7f1f7c006030 con 0x7f1f881970d0 2021-02-10T11:49:07.269 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:49:07.269 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:49:07.270 INFO:teuthology.orchestra.run.gibba001.stdout: "epoch": 191, 2021-02-10T11:49:07.270 INFO:teuthology.orchestra.run.gibba001.stdout: "fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:49:07.270 INFO:teuthology.orchestra.run.gibba001.stdout: "created": "2021-02-10 11:12:36.788366", 2021-02-10T11:49:07.270 INFO:teuthology.orchestra.run.gibba001.stdout: "modified": "2021-02-10 11:48:25.826144", 2021-02-10T11:49:07.270 INFO:teuthology.orchestra.run.gibba001.stdout: "last_up_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:49:07.270 INFO:teuthology.orchestra.run.gibba001.stdout: "last_in_change": "2021-02-10 11:13:03.215004", 2021-02-10T11:49:07.270 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-02-10T11:49:07.270 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_num": 5799936, 2021-02-10T11:49:07.270 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_set": [ 2021-02-10T11:49:07.270 INFO:teuthology.orchestra.run.gibba001.stdout: "pglog_hardlimit", 2021-02-10T11:49:07.270 INFO:teuthology.orchestra.run.gibba001.stdout: "purged_snapdirs", 2021-02-10T11:49:07.270 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_deletes", 2021-02-10T11:49:07.270 INFO:teuthology.orchestra.run.gibba001.stdout: "sortbitwise" 2021-02-10T11:49:07.271 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:49:07.271 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_version": 4, 2021-02-10T11:49:07.271 INFO:teuthology.orchestra.run.gibba001.stdout: "full_ratio": 0.94999998807907104, 2021-02-10T11:49:07.271 INFO:teuthology.orchestra.run.gibba001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-02-10T11:49:07.271 INFO:teuthology.orchestra.run.gibba001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-02-10T11:49:07.271 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_snapshot": "", 2021-02-10T11:49:07.271 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_max": 41, 2021-02-10T11:49:07.271 INFO:teuthology.orchestra.run.gibba001.stdout: "max_osd": 8, 2021-02-10T11:49:07.271 INFO:teuthology.orchestra.run.gibba001.stdout: "require_min_compat_client": "jewel", 2021-02-10T11:49:07.271 INFO:teuthology.orchestra.run.gibba001.stdout: "min_compat_client": "jewel", 2021-02-10T11:49:07.271 INFO:teuthology.orchestra.run.gibba001.stdout: "require_osd_release": "nautilus", 2021-02-10T11:49:07.271 INFO:teuthology.orchestra.run.gibba001.stdout: "pools": [ 2021-02-10T11:49:07.271 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.272 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 1, 2021-02-10T11:49:07.272 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "rbd", 2021-02-10T11:49:07.272 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:13:08.037472", 2021-02-10T11:49:07.272 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:49:07.272 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:49:07.272 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:49:07.272 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:49:07.272 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:49:07.272 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:49:07.272 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:49:07.272 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:49:07.272 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:49:07.272 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:49:07.273 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:49:07.273 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:49:07.273 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:49:07.273 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:49:07.273 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:49:07.273 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:49:07.273 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:49:07.273 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:49:07.273 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:49:07.273 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:49:07.273 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.273 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "16", 2021-02-10T11:49:07.273 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:49:07.274 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:49:07.274 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:49:07.274 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:49:07.274 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:49:07.274 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:49:07.274 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:49:07.274 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:49:07.274 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:49:07.274 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:49:07.274 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:49:07.274 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:49:07.274 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:49:07.275 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:49:07.275 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:49:07.275 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:49:07.275 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:49:07.275 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:49:07.275 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:49:07.275 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:49:07.275 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:49:07.275 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:49:07.275 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:49:07.275 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:49:07.275 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:49:07.275 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:49:07.276 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.276 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:49:07.276 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:49:07.276 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:49:07.276 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:49:07.276 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:49:07.276 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:49:07.276 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:49:07.276 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:49:07.276 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:49:07.276 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:49:07.276 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:49:07.276 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:49:07.277 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:49:07.277 INFO:teuthology.orchestra.run.gibba001.stdout: "rbd": {} 2021-02-10T11:49:07.277 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.277 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.277 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.277 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 40, 2021-02-10T11:49:07.277 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_metadata", 2021-02-10T11:49:07.277 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:48:22.784885", 2021-02-10T11:49:07.277 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:49:07.277 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:49:07.277 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:49:07.277 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:49:07.277 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:49:07.278 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:49:07.278 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:49:07.278 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:49:07.278 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:49:07.278 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:49:07.278 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:49:07.278 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:49:07.278 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:49:07.278 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:49:07.278 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:49:07.278 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:49:07.278 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:49:07.279 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:49:07.279 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:49:07.279 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:49:07.279 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.279 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "190", 2021-02-10T11:49:07.279 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:49:07.279 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:49:07.279 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:49:07.279 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:49:07.279 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:49:07.279 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:49:07.279 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:49:07.279 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:49:07.279 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:49:07.280 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:49:07.280 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:49:07.280 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:49:07.280 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:49:07.280 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:49:07.280 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:49:07.280 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:49:07.280 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:49:07.280 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:49:07.280 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:49:07.280 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:49:07.280 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:49:07.280 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:49:07.281 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:49:07.281 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:49:07.281 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:49:07.281 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:49:07.281 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.281 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:49:07.281 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:49:07.281 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:49:07.281 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:49:07.281 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:49:07.281 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:49:07.281 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:49:07.281 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:49:07.282 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:49:07.282 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:49:07.282 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:49:07.282 INFO:teuthology.orchestra.run.gibba001.stdout: "options": { 2021-02-10T11:49:07.282 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_bias": 4, 2021-02-10T11:49:07.282 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_min": 16, 2021-02-10T11:49:07.282 INFO:teuthology.orchestra.run.gibba001.stdout: "recovery_priority": 5 2021-02-10T11:49:07.282 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.282 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:49:07.282 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:49:07.282 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": "cephfs" 2021-02-10T11:49:07.282 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.282 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.283 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.283 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.283 INFO:teuthology.orchestra.run.gibba001.stdout: "pool": 41, 2021-02-10T11:49:07.283 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_name": "cephfs_data", 2021-02-10T11:49:07.283 INFO:teuthology.orchestra.run.gibba001.stdout: "create_time": "2021-02-10 11:48:23.812792", 2021-02-10T11:49:07.283 INFO:teuthology.orchestra.run.gibba001.stdout: "flags": 1, 2021-02-10T11:49:07.283 INFO:teuthology.orchestra.run.gibba001.stdout: "flags_names": "hashpspool", 2021-02-10T11:49:07.283 INFO:teuthology.orchestra.run.gibba001.stdout: "type": 1, 2021-02-10T11:49:07.283 INFO:teuthology.orchestra.run.gibba001.stdout: "size": 2, 2021-02-10T11:49:07.283 INFO:teuthology.orchestra.run.gibba001.stdout: "min_size": 1, 2021-02-10T11:49:07.283 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_rule": 0, 2021-02-10T11:49:07.283 INFO:teuthology.orchestra.run.gibba001.stdout: "object_hash": 2, 2021-02-10T11:49:07.283 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_autoscale_mode": "warn", 2021-02-10T11:49:07.284 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num": 8, 2021-02-10T11:49:07.284 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num": 8, 2021-02-10T11:49:07.284 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_placement_num_target": 8, 2021-02-10T11:49:07.284 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_target": 8, 2021-02-10T11:49:07.284 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_num_pending": 8, 2021-02-10T11:49:07.284 INFO:teuthology.orchestra.run.gibba001.stdout: "last_pg_merge_meta": { 2021-02-10T11:49:07.284 INFO:teuthology.orchestra.run.gibba001.stdout: "source_pgid": "0.0", 2021-02-10T11:49:07.284 INFO:teuthology.orchestra.run.gibba001.stdout: "ready_epoch": 0, 2021-02-10T11:49:07.284 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_started": 0, 2021-02-10T11:49:07.284 INFO:teuthology.orchestra.run.gibba001.stdout: "last_epoch_clean": 0, 2021-02-10T11:49:07.284 INFO:teuthology.orchestra.run.gibba001.stdout: "source_version": "0'0", 2021-02-10T11:49:07.284 INFO:teuthology.orchestra.run.gibba001.stdout: "target_version": "0'0" 2021-02-10T11:49:07.284 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.284 INFO:teuthology.orchestra.run.gibba001.stdout: "last_change": "191", 2021-02-10T11:49:07.285 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend": "0", 2021-02-10T11:49:07.285 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_prenautilus": "0", 2021-02-10T11:49:07.285 INFO:teuthology.orchestra.run.gibba001.stdout: "last_force_op_resend_preluminous": "0", 2021-02-10T11:49:07.285 INFO:teuthology.orchestra.run.gibba001.stdout: "auid": 0, 2021-02-10T11:49:07.285 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_mode": "selfmanaged", 2021-02-10T11:49:07.285 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_seq": 0, 2021-02-10T11:49:07.285 INFO:teuthology.orchestra.run.gibba001.stdout: "snap_epoch": 0, 2021-02-10T11:49:07.285 INFO:teuthology.orchestra.run.gibba001.stdout: "pool_snaps": [], 2021-02-10T11:49:07.285 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps": "[]", 2021-02-10T11:49:07.285 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_bytes": 0, 2021-02-10T11:49:07.285 INFO:teuthology.orchestra.run.gibba001.stdout: "quota_max_objects": 0, 2021-02-10T11:49:07.285 INFO:teuthology.orchestra.run.gibba001.stdout: "tiers": [], 2021-02-10T11:49:07.285 INFO:teuthology.orchestra.run.gibba001.stdout: "tier_of": -1, 2021-02-10T11:49:07.286 INFO:teuthology.orchestra.run.gibba001.stdout: "read_tier": -1, 2021-02-10T11:49:07.286 INFO:teuthology.orchestra.run.gibba001.stdout: "write_tier": -1, 2021-02-10T11:49:07.286 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_mode": "none", 2021-02-10T11:49:07.286 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_bytes": 0, 2021-02-10T11:49:07.286 INFO:teuthology.orchestra.run.gibba001.stdout: "target_max_objects": 0, 2021-02-10T11:49:07.286 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-02-10T11:49:07.286 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-02-10T11:49:07.286 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_target_full_ratio_micro": 800000, 2021-02-10T11:49:07.286 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_flush_age": 0, 2021-02-10T11:49:07.286 INFO:teuthology.orchestra.run.gibba001.stdout: "cache_min_evict_age": 0, 2021-02-10T11:49:07.286 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profile": "", 2021-02-10T11:49:07.286 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_params": { 2021-02-10T11:49:07.286 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "none" 2021-02-10T11:49:07.287 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.287 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_period": 0, 2021-02-10T11:49:07.287 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_count": 0, 2021-02-10T11:49:07.287 INFO:teuthology.orchestra.run.gibba001.stdout: "use_gmt_hitset": true, 2021-02-10T11:49:07.287 INFO:teuthology.orchestra.run.gibba001.stdout: "min_read_recency_for_promote": 0, 2021-02-10T11:49:07.287 INFO:teuthology.orchestra.run.gibba001.stdout: "min_write_recency_for_promote": 0, 2021-02-10T11:49:07.287 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_grade_decay_rate": 0, 2021-02-10T11:49:07.287 INFO:teuthology.orchestra.run.gibba001.stdout: "hit_set_search_last_n": 0, 2021-02-10T11:49:07.287 INFO:teuthology.orchestra.run.gibba001.stdout: "grade_table": [], 2021-02-10T11:49:07.287 INFO:teuthology.orchestra.run.gibba001.stdout: "stripe_width": 0, 2021-02-10T11:49:07.287 INFO:teuthology.orchestra.run.gibba001.stdout: "expected_num_objects": 0, 2021-02-10T11:49:07.287 INFO:teuthology.orchestra.run.gibba001.stdout: "fast_read": false, 2021-02-10T11:49:07.287 INFO:teuthology.orchestra.run.gibba001.stdout: "options": {}, 2021-02-10T11:49:07.288 INFO:teuthology.orchestra.run.gibba001.stdout: "application_metadata": { 2021-02-10T11:49:07.288 INFO:teuthology.orchestra.run.gibba001.stdout: "cephfs": { 2021-02-10T11:49:07.288 INFO:teuthology.orchestra.run.gibba001.stdout: "data": "cephfs" 2021-02-10T11:49:07.288 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.288 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.288 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.288 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:49:07.288 INFO:teuthology.orchestra.run.gibba001.stdout: "osds": [ 2021-02-10T11:49:07.288 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.288 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:49:07.288 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5", 2021-02-10T11:49:07.288 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:49:07.288 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:49:07.289 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:49:07.289 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:49:07.289 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:49:07.289 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:49:07.289 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:49:07.289 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 188, 2021-02-10T11:49:07.289 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:49:07.289 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:49:07.289 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:49:07.289 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.289 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.289 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.289 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6802", 2021-02-10T11:49:07.290 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:49:07.290 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.290 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.290 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.290 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6805", 2021-02-10T11:49:07.290 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:49:07.290 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.290 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.290 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.290 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:49:07.290 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.290 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.291 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.291 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6807", 2021-02-10T11:49:07.291 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:49:07.291 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.291 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.291 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.291 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6809", 2021-02-10T11:49:07.291 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:49:07.291 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.291 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.291 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.291 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:49:07.291 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.292 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.292 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.292 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6817", 2021-02-10T11:49:07.292 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:49:07.292 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.292 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.292 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.292 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6820", 2021-02-10T11:49:07.292 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:49:07.292 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.292 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.292 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.292 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:49:07.293 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.293 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.293 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.293 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6812", 2021-02-10T11:49:07.293 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:49:07.293 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.293 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.293 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.293 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6814", 2021-02-10T11:49:07.293 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25093 2021-02-10T11:49:07.293 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.293 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.293 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.293 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6805/25093", 2021-02-10T11:49:07.294 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6809/25093", 2021-02-10T11:49:07.294 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6820/25093", 2021-02-10T11:49:07.294 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6814/25093", 2021-02-10T11:49:07.294 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:49:07.294 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:49:07.294 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:49:07.294 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.294 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.294 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.294 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:49:07.294 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "ddc04ce3-b313-4707-8270-585391e38ecc", 2021-02-10T11:49:07.294 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:49:07.295 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:49:07.295 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:49:07.295 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:49:07.295 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:49:07.295 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:49:07.295 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:49:07.295 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 188, 2021-02-10T11:49:07.295 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:49:07.295 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:49:07.295 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:49:07.295 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.295 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.295 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.296 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6803", 2021-02-10T11:49:07.296 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:49:07.296 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.296 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.296 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.296 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6804", 2021-02-10T11:49:07.296 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:49:07.296 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.296 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.296 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.296 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:49:07.296 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.296 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.297 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.297 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6806", 2021-02-10T11:49:07.297 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:49:07.297 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.297 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.297 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.297 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6808", 2021-02-10T11:49:07.297 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:49:07.297 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.297 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.297 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.297 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:49:07.297 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.297 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.298 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.298 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6818", 2021-02-10T11:49:07.298 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:49:07.298 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.298 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.298 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.298 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6821", 2021-02-10T11:49:07.298 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:49:07.298 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.298 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.298 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.298 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:49:07.299 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.299 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.299 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.299 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6810", 2021-02-10T11:49:07.299 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:49:07.299 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.299 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.299 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.299 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6815", 2021-02-10T11:49:07.299 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25092 2021-02-10T11:49:07.299 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.299 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.299 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.300 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6804/25092", 2021-02-10T11:49:07.300 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6808/25092", 2021-02-10T11:49:07.300 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6821/25092", 2021-02-10T11:49:07.300 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6815/25092", 2021-02-10T11:49:07.300 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:49:07.300 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:49:07.300 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:49:07.300 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.300 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.300 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.300 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:49:07.300 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3", 2021-02-10T11:49:07.301 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:49:07.301 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:49:07.301 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:49:07.301 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:49:07.301 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:49:07.301 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:49:07.301 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:49:07.301 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 189, 2021-02-10T11:49:07.301 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:49:07.301 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:49:07.301 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:49:07.301 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.301 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.302 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.302 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6811", 2021-02-10T11:49:07.302 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:49:07.302 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.302 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.302 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.302 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6813", 2021-02-10T11:49:07.302 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:49:07.302 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.302 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.302 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.302 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:49:07.302 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.303 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.303 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.303 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6816", 2021-02-10T11:49:07.303 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:49:07.303 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.303 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.303 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.303 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6819", 2021-02-10T11:49:07.303 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:49:07.303 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.303 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.303 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.303 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:49:07.304 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.304 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.304 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.304 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6825", 2021-02-10T11:49:07.304 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:49:07.304 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.304 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.304 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.304 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6827", 2021-02-10T11:49:07.304 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:49:07.304 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.304 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.304 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.304 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:49:07.305 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.305 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.305 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.305 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6822", 2021-02-10T11:49:07.305 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:49:07.305 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.305 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.305 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.305 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6823", 2021-02-10T11:49:07.305 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25094 2021-02-10T11:49:07.305 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.305 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.305 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.306 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6813/25094", 2021-02-10T11:49:07.306 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6819/25094", 2021-02-10T11:49:07.306 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6827/25094", 2021-02-10T11:49:07.306 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6823/25094", 2021-02-10T11:49:07.306 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:49:07.306 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:49:07.306 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:49:07.306 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.306 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.306 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.306 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:49:07.306 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "0d9ef9b1-98aa-4d37-aa16-6305c7d46d52", 2021-02-10T11:49:07.306 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:49:07.307 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:49:07.307 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:49:07.307 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:49:07.307 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:49:07.307 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:49:07.307 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:49:07.307 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 189, 2021-02-10T11:49:07.307 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:49:07.307 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:49:07.307 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:49:07.307 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.307 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.307 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.308 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6824", 2021-02-10T11:49:07.308 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:49:07.308 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.308 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.308 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.308 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6826", 2021-02-10T11:49:07.308 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:49:07.308 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.308 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.308 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.308 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:49:07.308 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.309 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.309 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.309 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6828", 2021-02-10T11:49:07.309 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:49:07.309 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.309 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.309 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.309 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6829", 2021-02-10T11:49:07.309 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:49:07.309 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.309 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.309 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.309 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:49:07.309 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.310 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.310 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.310 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6832", 2021-02-10T11:49:07.310 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:49:07.310 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.310 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.310 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.310 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6833", 2021-02-10T11:49:07.310 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:49:07.310 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.310 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.310 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.311 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:49:07.311 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.311 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.311 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.311 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6830", 2021-02-10T11:49:07.311 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:49:07.311 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.311 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.311 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.311 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:6831", 2021-02-10T11:49:07.311 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 25095 2021-02-10T11:49:07.311 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.311 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.312 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.312 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.101:6826/25095", 2021-02-10T11:49:07.312 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.101:6829/25095", 2021-02-10T11:49:07.312 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.101:6833/25095", 2021-02-10T11:49:07.312 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.101:6831/25095", 2021-02-10T11:49:07.312 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:49:07.312 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:49:07.312 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:49:07.312 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.312 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.312 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.312 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:49:07.312 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "36742c21-bfeb-4e11-bce9-8e745378b605", 2021-02-10T11:49:07.313 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:49:07.313 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:49:07.313 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:49:07.313 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:49:07.313 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:49:07.313 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:49:07.313 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:49:07.313 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 189, 2021-02-10T11:49:07.313 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:49:07.313 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:49:07.313 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:49:07.313 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.313 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.313 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.314 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6816", 2021-02-10T11:49:07.314 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:49:07.314 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.314 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.314 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.314 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6818", 2021-02-10T11:49:07.314 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:49:07.314 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.314 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.314 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.314 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:49:07.314 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.315 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.315 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.315 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6820", 2021-02-10T11:49:07.315 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:49:07.315 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.315 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.315 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.315 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6822", 2021-02-10T11:49:07.315 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:49:07.315 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.315 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.315 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.315 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:49:07.316 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.316 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.316 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.316 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6828", 2021-02-10T11:49:07.316 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:49:07.316 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.316 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.316 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.316 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6830", 2021-02-10T11:49:07.316 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:49:07.316 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.316 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.316 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.317 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:49:07.317 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.317 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.317 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.317 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6824", 2021-02-10T11:49:07.317 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:49:07.317 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.317 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.317 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.317 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6826", 2021-02-10T11:49:07.317 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36380 2021-02-10T11:49:07.317 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.317 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.317 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.318 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6818/36380", 2021-02-10T11:49:07.318 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6822/36380", 2021-02-10T11:49:07.318 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6830/36380", 2021-02-10T11:49:07.318 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6826/36380", 2021-02-10T11:49:07.318 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:49:07.318 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:49:07.318 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:49:07.318 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.318 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.318 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.318 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:49:07.318 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "88e694e8-fe98-4d1b-88c8-8c9907ece60b", 2021-02-10T11:49:07.318 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:49:07.319 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:49:07.319 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:49:07.319 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:49:07.319 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:49:07.319 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:49:07.319 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:49:07.319 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 181, 2021-02-10T11:49:07.319 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:49:07.319 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:49:07.319 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:49:07.319 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.319 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.319 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.320 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6808", 2021-02-10T11:49:07.320 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:49:07.320 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.320 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.320 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.320 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6809", 2021-02-10T11:49:07.320 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:49:07.320 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.320 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.320 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.320 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:49:07.320 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.320 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.321 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.321 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6810", 2021-02-10T11:49:07.321 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:49:07.321 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.321 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.321 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.321 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6811", 2021-02-10T11:49:07.321 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:49:07.321 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.321 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.321 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.321 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:49:07.321 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.322 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.322 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.322 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6814", 2021-02-10T11:49:07.322 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:49:07.322 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.322 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.322 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.322 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6815", 2021-02-10T11:49:07.322 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:49:07.322 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.322 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.322 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.322 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:49:07.323 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.323 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.323 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.323 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6812", 2021-02-10T11:49:07.323 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:49:07.323 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.323 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.323 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.323 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6813", 2021-02-10T11:49:07.323 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36383 2021-02-10T11:49:07.323 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.323 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.323 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.323 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6809/36383", 2021-02-10T11:49:07.324 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6811/36383", 2021-02-10T11:49:07.324 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6815/36383", 2021-02-10T11:49:07.324 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6813/36383", 2021-02-10T11:49:07.324 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:49:07.324 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:49:07.324 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:49:07.324 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.324 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.324 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.324 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:49:07.324 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "87a13c6c-cf85-4aaf-916b-e73913484a49", 2021-02-10T11:49:07.324 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:49:07.324 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:49:07.325 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:49:07.325 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:49:07.325 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:49:07.325 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:49:07.325 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:49:07.325 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 189, 2021-02-10T11:49:07.325 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:49:07.325 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:49:07.325 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:49:07.325 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.325 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.325 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.325 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6800", 2021-02-10T11:49:07.325 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:49:07.326 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.326 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.326 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.326 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6801", 2021-02-10T11:49:07.326 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:49:07.326 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.326 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.326 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.326 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:49:07.326 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.326 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.326 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.327 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6802", 2021-02-10T11:49:07.327 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:49:07.327 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.327 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.327 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.327 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6803", 2021-02-10T11:49:07.327 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:49:07.327 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.327 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.327 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.327 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:49:07.327 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.327 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.328 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.328 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6806", 2021-02-10T11:49:07.328 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:49:07.328 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.328 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.328 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.328 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6807", 2021-02-10T11:49:07.328 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:49:07.328 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.328 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.328 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.328 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:49:07.328 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.329 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.329 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.329 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6804", 2021-02-10T11:49:07.329 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:49:07.329 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.329 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.329 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.329 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6805", 2021-02-10T11:49:07.329 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36381 2021-02-10T11:49:07.329 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.329 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.329 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.329 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6801/36381", 2021-02-10T11:49:07.330 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6803/36381", 2021-02-10T11:49:07.330 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6807/36381", 2021-02-10T11:49:07.330 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6805/36381", 2021-02-10T11:49:07.330 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:49:07.330 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:49:07.330 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:49:07.330 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.330 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.330 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.330 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:49:07.330 INFO:teuthology.orchestra.run.gibba001.stdout: "uuid": "7b90823f-854c-4592-a95b-a7fa98f93254", 2021-02-10T11:49:07.330 INFO:teuthology.orchestra.run.gibba001.stdout: "up": 1, 2021-02-10T11:49:07.330 INFO:teuthology.orchestra.run.gibba001.stdout: "in": 1, 2021-02-10T11:49:07.331 INFO:teuthology.orchestra.run.gibba001.stdout: "weight": 1, 2021-02-10T11:49:07.331 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_affinity": 1, 2021-02-10T11:49:07.331 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_begin": 0, 2021-02-10T11:49:07.331 INFO:teuthology.orchestra.run.gibba001.stdout: "last_clean_end": 0, 2021-02-10T11:49:07.331 INFO:teuthology.orchestra.run.gibba001.stdout: "up_from": 13, 2021-02-10T11:49:07.331 INFO:teuthology.orchestra.run.gibba001.stdout: "up_thru": 189, 2021-02-10T11:49:07.331 INFO:teuthology.orchestra.run.gibba001.stdout: "down_at": 0, 2021-02-10T11:49:07.331 INFO:teuthology.orchestra.run.gibba001.stdout: "lost_at": 0, 2021-02-10T11:49:07.331 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addrs": { 2021-02-10T11:49:07.331 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.331 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.331 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.331 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6817", 2021-02-10T11:49:07.332 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:49:07.332 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.332 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.332 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.332 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6819", 2021-02-10T11:49:07.332 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:49:07.332 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.332 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.332 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.332 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addrs": { 2021-02-10T11:49:07.332 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.332 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.333 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.333 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6821", 2021-02-10T11:49:07.333 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:49:07.333 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.333 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.333 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.333 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6823", 2021-02-10T11:49:07.333 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:49:07.333 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.333 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.333 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.333 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addrs": { 2021-02-10T11:49:07.333 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.333 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.334 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.334 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6829", 2021-02-10T11:49:07.334 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:49:07.334 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.334 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.334 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.334 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6831", 2021-02-10T11:49:07.334 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:49:07.334 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.334 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.334 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.335 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addrs": { 2021-02-10T11:49:07.335 INFO:teuthology.orchestra.run.gibba001.stdout: "addrvec": [ 2021-02-10T11:49:07.335 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.335 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v2", 2021-02-10T11:49:07.335 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6825", 2021-02-10T11:49:07.335 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:49:07.335 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.335 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.335 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:07.335 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.108:6827", 2021-02-10T11:49:07.335 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 36382 2021-02-10T11:49:07.335 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.335 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.336 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.336 INFO:teuthology.orchestra.run.gibba001.stdout: "public_addr": "172.21.2.108:6819/36382", 2021-02-10T11:49:07.336 INFO:teuthology.orchestra.run.gibba001.stdout: "cluster_addr": "172.21.2.108:6823/36382", 2021-02-10T11:49:07.336 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_back_addr": "172.21.2.108:6831/36382", 2021-02-10T11:49:07.336 INFO:teuthology.orchestra.run.gibba001.stdout: "heartbeat_front_addr": "172.21.2.108:6827/36382", 2021-02-10T11:49:07.336 INFO:teuthology.orchestra.run.gibba001.stdout: "state": [ 2021-02-10T11:49:07.336 INFO:teuthology.orchestra.run.gibba001.stdout: "exists", 2021-02-10T11:49:07.336 INFO:teuthology.orchestra.run.gibba001.stdout: "up" 2021-02-10T11:49:07.336 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:07.336 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.336 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:49:07.336 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_xinfo": [ 2021-02-10T11:49:07.336 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.337 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 0, 2021-02-10T11:49:07.337 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:49:07.337 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:49:07.337 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:49:07.337 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:49:07.337 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:49:07.337 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.337 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.337 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 1, 2021-02-10T11:49:07.337 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:49:07.337 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:49:07.337 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:49:07.337 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:49:07.337 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:49:07.338 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.338 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.338 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 2, 2021-02-10T11:49:07.338 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:49:07.338 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:49:07.338 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:49:07.338 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:49:07.338 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:49:07.338 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.338 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.338 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 3, 2021-02-10T11:49:07.338 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:49:07.339 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:49:07.339 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:49:07.339 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:49:07.339 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:49:07.339 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.339 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.339 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 4, 2021-02-10T11:49:07.339 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:49:07.339 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:49:07.339 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:49:07.339 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:49:07.339 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:49:07.339 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.339 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.340 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 5, 2021-02-10T11:49:07.340 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:49:07.340 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:49:07.340 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:49:07.340 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:49:07.340 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:49:07.340 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.340 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.340 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 6, 2021-02-10T11:49:07.340 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:49:07.340 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:49:07.340 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:49:07.340 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:49:07.340 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:49:07.341 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.341 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:07.341 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": 7, 2021-02-10T11:49:07.341 INFO:teuthology.orchestra.run.gibba001.stdout: "down_stamp": "0.000000", 2021-02-10T11:49:07.341 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_probability": 0, 2021-02-10T11:49:07.341 INFO:teuthology.orchestra.run.gibba001.stdout: "laggy_interval": 0, 2021-02-10T11:49:07.341 INFO:teuthology.orchestra.run.gibba001.stdout: "features": 4611087854035861503, 2021-02-10T11:49:07.341 INFO:teuthology.orchestra.run.gibba001.stdout: "old_weight": 0 2021-02-10T11:49:07.341 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.341 INFO:teuthology.orchestra.run.gibba001.stdout: ], 2021-02-10T11:49:07.341 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap": [], 2021-02-10T11:49:07.341 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_upmap_items": [], 2021-02-10T11:49:07.341 INFO:teuthology.orchestra.run.gibba001.stdout: "pg_temp": [], 2021-02-10T11:49:07.341 INFO:teuthology.orchestra.run.gibba001.stdout: "primary_temp": [], 2021-02-10T11:49:07.342 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklist": {}, 2021-02-10T11:49:07.342 INFO:teuthology.orchestra.run.gibba001.stdout: "erasure_code_profiles": { 2021-02-10T11:49:07.342 INFO:teuthology.orchestra.run.gibba001.stdout: "default": { 2021-02-10T11:49:07.342 INFO:teuthology.orchestra.run.gibba001.stdout: "crush-failure-domain": "osd", 2021-02-10T11:49:07.342 INFO:teuthology.orchestra.run.gibba001.stdout: "k": "2", 2021-02-10T11:49:07.342 INFO:teuthology.orchestra.run.gibba001.stdout: "m": "1", 2021-02-10T11:49:07.342 INFO:teuthology.orchestra.run.gibba001.stdout: "plugin": "jerasure", 2021-02-10T11:49:07.342 INFO:teuthology.orchestra.run.gibba001.stdout: "ruleset-failure-domain": "osd", 2021-02-10T11:49:07.342 INFO:teuthology.orchestra.run.gibba001.stdout: "technique": "reed_sol_van" 2021-02-10T11:49:07.342 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:07.342 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:07.342 INFO:teuthology.orchestra.run.gibba001.stdout: "removed_snaps_queue": [], 2021-02-10T11:49:07.343 INFO:teuthology.orchestra.run.gibba001.stdout: "new_removed_snaps": [], 2021-02-10T11:49:07.343 INFO:teuthology.orchestra.run.gibba001.stdout: "new_purged_snaps": [], 2021-02-10T11:49:07.343 INFO:teuthology.orchestra.run.gibba001.stdout: "crush_node_flags": {}, 2021-02-10T11:49:07.343 INFO:teuthology.orchestra.run.gibba001.stdout: "device_class_flags": {} 2021-02-10T11:49:07.343 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:49:07.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.274 7f1f8e158700 1 -- 172.21.2.101:0/1489682268 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f68018730 msgr2=0x7f1f6801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:07.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.274 7f1f8e158700 1 --2- 172.21.2.101:0/1489682268 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f68018730 0x7f1f6801ac10 crc :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:07.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.274 7f1f8e158700 1 -- 172.21.2.101:0/1489682268 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f881970d0 msgr2=0x7f1f881974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:07.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.274 7f1f8e158700 1 --2- 172.21.2.101:0/1489682268 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f881970d0 0x7f1f881974d0 secure :-1 s=READY pgs=3332 cs=0 l=1 rev1=1 rx=0x7f1f7c00b5c0 tx=0x7f1f7c00b960).stop 2021-02-10T11:49:07.344 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.274 7f1f8e158700 1 -- 172.21.2.101:0/1489682268 shutdown_connections 2021-02-10T11:49:07.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.274 7f1f8e158700 1 --2- 172.21.2.101:0/1489682268 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1f68018730 0x7f1f6801ac10 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:07.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.274 7f1f8e158700 1 --2- 172.21.2.101:0/1489682268 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1f88094650 0x7f1f88094a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:07.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.274 7f1f8e158700 1 --2- 172.21.2.101:0/1489682268 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1f881970d0 0x7f1f881974d0 unknown :-1 s=CLOSED pgs=3332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:07.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.274 7f1f8e158700 1 --2- 172.21.2.101:0/1489682268 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1f8818f0a0 0x7f1f88196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:07.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.274 7f1f8e158700 1 -- 172.21.2.101:0/1489682268 >> 172.21.2.101:0/1489682268 conn(0x7f1f881a38c0 msgr2=0x7f1f881a2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:07.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.275 7f1f8e158700 1 -- 172.21.2.101:0/1489682268 shutdown_connections 2021-02-10T11:49:07.345 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.275 7f1f8e158700 1 -- 172.21.2.101:0/1489682268 wait complete. 2021-02-10T11:49:07.346 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:49:07.775 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.763 7f4612524700 1 Processor -- start 2021-02-10T11:49:07.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.763 7f4612524700 1 -- start start 2021-02-10T11:49:07.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.763 7f4612524700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f460c0d0340 0x7f460c0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:07.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.763 7f4612524700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f460c0f0b10 con 0x7f460c0d1870 2021-02-10T11:49:07.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.763 7f4612524700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f460c1b0be0 con 0x7f460c0d1c70 2021-02-10T11:49:07.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.763 7f4612524700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f460c0e5ed0 con 0x7f460c0d0340 2021-02-10T11:49:07.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.764 7f460bfff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f460c0d1870 0x7f460c1b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:35998/0 (socket says 172.21.2.101:35998) 2021-02-10T11:49:07.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.764 7f460bfff700 1 -- 172.21.2.101:0/2748097891 learned_addr learned my addr 172.21.2.101:0/2748097891 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:07.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.764 7f460b7fe700 1 --2- 172.21.2.101:0/2748097891 >> v2:172.21.2.108:3300/0 conn(0x7f460c0d0340 0x7f460c0ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:07.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.764 7f460affd700 1 -- 172.21.2.101:0/2748097891 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2273549989 0 0) 0x7f45fc0009b0 con 0x7f460c0d1870 2021-02-10T11:49:07.776 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.764 7f460affd700 1 -- 172.21.2.101:0/2748097891 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45f4002fe0 con 0x7f460c0d1870 2021-02-10T11:49:07.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.764 7f460b7fe700 1 -- 172.21.2.101:0/2748097891 >> v1:172.21.2.108:6789/0 conn(0x7f460c0d1c70 legacy=0x7f460c0efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:07.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.764 7f460b7fe700 1 -- 172.21.2.101:0/2748097891 >> v1:172.21.2.101:6789/0 conn(0x7f460c0d1870 legacy=0x7f460c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:07.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.764 7f460b7fe700 1 -- 172.21.2.101:0/2748097891 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c0f0450 con 0x7f460c0d0340 2021-02-10T11:49:07.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.764 7f460b7fe700 1 --2- 172.21.2.101:0/2748097891 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f460c0d0340 0x7f460c0ef640 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7f4600006650 tx=0x7f4600009a80).ready entity=mon.1 client_cookie=a6de5d658614a832 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:07.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.765 7f460affd700 1 -- 172.21.2.101:0/2748097891 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f460000c0c0 con 0x7f460c0d0340 2021-02-10T11:49:07.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.765 7f460affd700 1 -- 172.21.2.101:0/2748097891 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460000c340 con 0x7f460c0d0340 2021-02-10T11:49:07.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.765 7f460affd700 1 -- 172.21.2.101:0/2748097891 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f460000c860 con 0x7f460c0d0340 2021-02-10T11:49:07.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.765 7f4612524700 1 -- 172.21.2.101:0/2748097891 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f460c0d0340 msgr2=0x7f460c0ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:07.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.765 7f4612524700 1 --2- 172.21.2.101:0/2748097891 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f460c0d0340 0x7f460c0ef640 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7f4600006650 tx=0x7f4600009a80).stop 2021-02-10T11:49:07.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.765 7f4612524700 1 -- 172.21.2.101:0/2748097891 shutdown_connections 2021-02-10T11:49:07.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.765 7f4612524700 1 --2- 172.21.2.101:0/2748097891 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f460c0d0340 0x7f460c0ef640 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:07.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.765 7f4612524700 1 -- 172.21.2.101:0/2748097891 >> 172.21.2.101:0/2748097891 conn(0x7f460c0ce450 msgr2=0x7f460c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:07.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.765 7f4612524700 1 -- 172.21.2.101:0/2748097891 shutdown_connections 2021-02-10T11:49:07.777 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.765 7f4612524700 1 -- 172.21.2.101:0/2748097891 wait complete. 2021-02-10T11:49:07.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.780 7f4612524700 1 Processor -- start 2021-02-10T11:49:07.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.780 7f4612524700 1 -- start start 2021-02-10T11:49:07.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.780 7f4612524700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f460c0eef50 0x7f460c0ef350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:07.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.780 7f4612524700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f460c0f1ae0 0x7f460c0f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:07.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.780 7f4612524700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f460c0f2500 0x7f460c18f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:07.779 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.780 7f4612524700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c0f2900 con 0x7f460c0f2500 2021-02-10T11:49:07.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.780 7f4612524700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c18fd50 con 0x7f460c0eef50 2021-02-10T11:49:07.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.780 7f4612524700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460c18ff70 con 0x7f460c0f1ae0 2021-02-10T11:49:07.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.780 7f460bfff700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f460c0eef50 0x7f460c0ef350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:07.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.780 7f4610b41700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f460c0f2500 0x7f460c18f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:07.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.781 7f4610b41700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f460c0f2500 0x7f460c18f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35510/0 (socket says 172.21.2.101:35510) 2021-02-10T11:49:07.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.781 7f4610b41700 1 -- 172.21.2.101:0/145035062 learned_addr learned my addr 172.21.2.101:0/145035062 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:07.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.781 7f460b7fe700 1 --2- 172.21.2.101:0/145035062 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f460c0f1ae0 0x7f460c0f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:07.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.781 7f460bfff700 1 -- 172.21.2.101:0/145035062 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f460c0f1ae0 msgr2=0x7f460c0f1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:07.780 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.781 7f460bfff700 1 --2- 172.21.2.101:0/145035062 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f460c0f1ae0 0x7f460c0f1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:07.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.781 7f460bfff700 1 -- 172.21.2.101:0/145035062 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f460c0f2500 msgr2=0x7f460c18f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:07.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.781 7f460bfff700 1 --2- 172.21.2.101:0/145035062 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f460c0f2500 0x7f460c18f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:07.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.781 7f460bfff700 1 -- 172.21.2.101:0/145035062 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c190190 con 0x7f460c0eef50 2021-02-10T11:49:07.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.781 7f460bfff700 1 --2- 172.21.2.101:0/145035062 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f460c0eef50 0x7f460c0ef350 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f45fc00a4a0 tx=0x7f45fc00a840).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:07.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.781 7f4608ff9700 1 -- 172.21.2.101:0/145035062 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f45fc000d50 con 0x7f460c0eef50 2021-02-10T11:49:07.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.781 7f4608ff9700 1 -- 172.21.2.101:0/145035062 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45fc00ac30 con 0x7f460c0eef50 2021-02-10T11:49:07.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.781 7f4612524700 1 -- 172.21.2.101:0/145035062 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460c190190 con 0x7f460c0eef50 2021-02-10T11:49:07.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.781 7f4608ff9700 1 -- 172.21.2.101:0/145035062 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f45fc0009b0 con 0x7f460c0eef50 2021-02-10T11:49:07.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.781 7f4612524700 1 -- 172.21.2.101:0/145035062 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460c190190 con 0x7f460c0eef50 2021-02-10T11:49:07.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.782 7f4608ff9700 1 -- 172.21.2.101:0/145035062 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f45fc004350 con 0x7f460c0eef50 2021-02-10T11:49:07.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.782 7f4608ff9700 1 --2- 172.21.2.101:0/145035062 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45ec018730 0x7f45ec01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:07.781 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.782 7f4608ff9700 1 -- 172.21.2.101:0/145035062 <== mon.1 v2:172.21.2.108:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f45fc032530 con 0x7f460c0eef50 2021-02-10T11:49:07.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.782 7f460b7fe700 1 --2- 172.21.2.101:0/145035062 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45ec018730 0x7f45ec01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:07.782 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.782 7f460b7fe700 1 --2- 172.21.2.101:0/145035062 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45ec018730 0x7f45ec01ac10 crc :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:07.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.813 7f4612524700 1 -- 172.21.2.101:0/145035062 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f460c0f2900 con 0x7f460c0eef50 2021-02-10T11:49:07.812 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.816 7f4608ff9700 1 -- 172.21.2.101:0/145035062 <== mon.1 v2:172.21.2.108:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f45fc000ec0 con 0x7f460c0eef50 2021-02-10T11:49:07.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.950 7f4612524700 1 -- 172.21.2.101:0/145035062 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f460c0f2900 con 0x7f460c0eef50 2021-02-10T11:49:07.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.950 7f4608ff9700 1 -- 172.21.2.101:0/145035062 <== mon.1 v2:172.21.2.108:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 165 v165) v1 ==== 95+0+2108 (secure 0 0 0) 0x7f45fc00ac30 con 0x7f460c0eef50 2021-02-10T11:49:07.947 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:49:07.947 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":165,"default_fscid":20,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":13530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/347510536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":347510536},{"type":"v1","addr":"172.21.2.108:6833","nonce":347510536}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":160}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:48:24.824003","modified":"2021-02-10 11:48:26.829128","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":13518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13518":{"gid":13518,"name":"a","rank":0,"incarnation":161,"state":"up:active","state_seq":3,"addr":"172.21.2.101:6835/2721786668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":2721786668},{"type":"v1","addr":"172.21.2.101:6835","nonce":2721786668}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[41],"metadata_pool":40,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":20}]} 2021-02-10T11:49:07.947 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 165 2021-02-10T11:49:07.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.952 7f4612524700 1 -- 172.21.2.101:0/145035062 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45ec018730 msgr2=0x7f45ec01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:07.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.952 7f4612524700 1 --2- 172.21.2.101:0/145035062 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45ec018730 0x7f45ec01ac10 crc :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:07.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.952 7f4612524700 1 -- 172.21.2.101:0/145035062 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f460c0eef50 msgr2=0x7f460c0ef350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:07.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.952 7f4612524700 1 --2- 172.21.2.101:0/145035062 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f460c0eef50 0x7f460c0ef350 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f45fc00a4a0 tx=0x7f45fc00a840).stop 2021-02-10T11:49:07.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.952 7f4612524700 1 -- 172.21.2.101:0/145035062 shutdown_connections 2021-02-10T11:49:07.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.952 7f4612524700 1 --2- 172.21.2.101:0/145035062 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45ec018730 0x7f45ec01ac10 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:07.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.952 7f4612524700 1 --2- 172.21.2.101:0/145035062 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f460c0f1ae0 0x7f460c0f1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:07.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.952 7f4612524700 1 --2- 172.21.2.101:0/145035062 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f460c0f2500 0x7f460c18f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:07.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.952 7f4612524700 1 --2- 172.21.2.101:0/145035062 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f460c0eef50 0x7f460c0ef350 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:07.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.952 7f4612524700 1 -- 172.21.2.101:0/145035062 >> 172.21.2.101:0/145035062 conn(0x7f460c19ed80 msgr2=0x7f460c092c60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:07.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.953 7f4612524700 1 -- 172.21.2.101:0/145035062 shutdown_connections 2021-02-10T11:49:07.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:07.953 7f4612524700 1 -- 172.21.2.101:0/145035062 wait complete. 2021-02-10T11:49:07.957 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs fail cephfs 2021-02-10T11:49:08.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.372 7fc460213700 1 Processor -- start 2021-02-10T11:49:08.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.372 7fc460213700 1 -- start start 2021-02-10T11:49:08.385 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.373 7fc460213700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc4580d1870 0x7fc4581b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:08.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.373 7fc460213700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc4580f0650 0x7fc4580cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:08.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.373 7fc460213700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4580f0b10 con 0x7fc4581b0a10 2021-02-10T11:49:08.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.373 7fc460213700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4580fe230 con 0x7fc4580f0650 2021-02-10T11:49:08.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.373 7fc460213700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4580e5ed0 con 0x7fc4580d1870 2021-02-10T11:49:08.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.373 7fc45e830700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc4580f0650 0x7fc4580cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:08.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.373 7fc45e830700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc4580f0650 0x7fc4580cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35514/0 (socket says 172.21.2.101:35514) 2021-02-10T11:49:08.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.373 7fc45e830700 1 -- 172.21.2.101:0/1990054453 learned_addr learned my addr 172.21.2.101:0/1990054453 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:08.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.373 7fc45e830700 1 -- 172.21.2.101:0/1990054453 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc4580d1870 msgr2=0x7fc4581b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:49:08.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.373 7fc45e830700 1 --2- 172.21.2.101:0/1990054453 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc4580d1870 0x7fc4581b03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:08.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.373 7fc45e830700 1 -- 172.21.2.101:0/1990054453 >> v1:172.21.2.108:6789/0 conn(0x7fc4581b0a10 legacy=0x7fc4580cf330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:49:08.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.373 7fc45e830700 1 -- 172.21.2.101:0/1990054453 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4580f20a0 con 0x7fc4580f0650 2021-02-10T11:49:08.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.373 7fc45e830700 1 --2- 172.21.2.101:0/1990054453 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc4580f0650 0x7fc4580cfb20 secure :-1 s=READY pgs=3334 cs=0 l=1 rev1=1 rx=0x7fc450006b80 tx=0x7fc450009b90).ready entity=mon.0 client_cookie=59ab718a26aa141d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:08.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.373 7fc45d02d700 1 -- 172.21.2.101:0/1990054453 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc4500022e0 con 0x7fc4580f0650 2021-02-10T11:49:08.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.374 7fc45d02d700 1 -- 172.21.2.101:0/1990054453 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc450002620 con 0x7fc4580f0650 2021-02-10T11:49:08.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.374 7fc45d02d700 1 -- 172.21.2.101:0/1990054453 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc450000c10 con 0x7fc4580f0650 2021-02-10T11:49:08.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.374 7fc460213700 1 -- 172.21.2.101:0/1990054453 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc4580f0650 msgr2=0x7fc4580cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:08.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.374 7fc460213700 1 --2- 172.21.2.101:0/1990054453 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc4580f0650 0x7fc4580cfb20 secure :-1 s=READY pgs=3334 cs=0 l=1 rev1=1 rx=0x7fc450006b80 tx=0x7fc450009b90).stop 2021-02-10T11:49:08.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.374 7fc460213700 1 -- 172.21.2.101:0/1990054453 shutdown_connections 2021-02-10T11:49:08.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.374 7fc460213700 1 --2- 172.21.2.101:0/1990054453 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc4580f0650 0x7fc4580cfb20 unknown :-1 s=CLOSED pgs=3334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:08.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.374 7fc460213700 1 --2- 172.21.2.101:0/1990054453 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc4580d1870 0x7fc4581b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:08.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.374 7fc460213700 1 -- 172.21.2.101:0/1990054453 >> 172.21.2.101:0/1990054453 conn(0x7fc4580ce450 msgr2=0x7fc4580d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:08.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.374 7fc460213700 1 -- 172.21.2.101:0/1990054453 shutdown_connections 2021-02-10T11:49:08.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.374 7fc460213700 1 -- 172.21.2.101:0/1990054453 wait complete. 2021-02-10T11:49:08.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.389 7fc460213700 1 Processor -- start 2021-02-10T11:49:08.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.389 7fc460213700 1 -- start start 2021-02-10T11:49:08.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.389 7fc460213700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc4580bdda0 0x7fc45818b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:08.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.389 7fc460213700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc45818b870 0x7fc4580c0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:08.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.389 7fc460213700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc45818bc70 0x7fc4580c0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:08.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.389 7fc460213700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45819d390 con 0x7fc45818bc70 2021-02-10T11:49:08.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.389 7fc460213700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4580bd740 con 0x7fc45818b870 2021-02-10T11:49:08.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.389 7fc460213700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4580be1a0 con 0x7fc4580bdda0 2021-02-10T11:49:08.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.389 7fc45e830700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc45818bc70 0x7fc4580c0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:08.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.389 7fc45e830700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc45818bc70 0x7fc4580c0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35522/0 (socket says 172.21.2.101:35522) 2021-02-10T11:49:08.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.389 7fc45e830700 1 -- 172.21.2.101:0/1264164334 learned_addr learned my addr 172.21.2.101:0/1264164334 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:08.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.390 7fc45e830700 1 -- 172.21.2.101:0/1264164334 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc4580bdda0 msgr2=0x7fc45818b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:08.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.390 7fc45e830700 1 --2- 172.21.2.101:0/1264164334 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc4580bdda0 0x7fc45818b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:08.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.390 7fc45e830700 1 -- 172.21.2.101:0/1264164334 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc45818b870 msgr2=0x7fc4580c0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:08.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.390 7fc45e830700 1 --2- 172.21.2.101:0/1264164334 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc45818b870 0x7fc4580c0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:08.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.390 7fc45e830700 1 -- 172.21.2.101:0/1264164334 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4580c0c70 con 0x7fc45818bc70 2021-02-10T11:49:08.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.390 7fc45e830700 1 --2- 172.21.2.101:0/1264164334 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc45818bc70 0x7fc4580c0650 secure :-1 s=READY pgs=3335 cs=0 l=1 rev1=1 rx=0x7fc450006f10 tx=0x7fc45000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:08.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.390 7fc456ffd700 1 -- 172.21.2.101:0/1264164334 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc450000b90 con 0x7fc45818bc70 2021-02-10T11:49:08.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.390 7fc456ffd700 1 -- 172.21.2.101:0/1264164334 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45000bd50 con 0x7fc45818bc70 2021-02-10T11:49:08.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.390 7fc456ffd700 1 -- 172.21.2.101:0/1264164334 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc450002220 con 0x7fc45818bc70 2021-02-10T11:49:08.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.390 7fc460213700 1 -- 172.21.2.101:0/1264164334 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4580c0c70 con 0x7fc45818bc70 2021-02-10T11:49:08.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.390 7fc460213700 1 -- 172.21.2.101:0/1264164334 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4580c1200 con 0x7fc45818bc70 2021-02-10T11:49:08.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.391 7fc456ffd700 1 -- 172.21.2.101:0/1264164334 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc450009310 con 0x7fc45818bc70 2021-02-10T11:49:08.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.391 7fc456ffd700 1 --2- 172.21.2.101:0/1264164334 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4380186c0 0x7fc43801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:08.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.391 7fc456ffd700 1 -- 172.21.2.101:0/1264164334 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc450032590 con 0x7fc45818bc70 2021-02-10T11:49:08.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.391 7fc45e02f700 1 --2- 172.21.2.101:0/1264164334 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4380186c0 0x7fc43801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:08.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.391 7fc45e02f700 1 --2- 172.21.2.101:0/1264164334 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4380186c0 0x7fc43801abc0 crc :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:08.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.422 7fc460213700 1 -- 172.21.2.101:0/1264164334 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4580ee0b0 con 0x7fc45818bc70 2021-02-10T11:49:08.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.425 7fc456ffd700 1 -- 172.21.2.101:0/1264164334 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc4500090e0 con 0x7fc45818bc70 2021-02-10T11:49:08.548 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.552 7fc460213700 1 -- 172.21.2.101:0/1264164334 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc4580ce450 con 0x7fc45818bc70 2021-02-10T11:49:08.664 INFO:tasks.ceph.mon.a.gibba001.stderr:2021-02-10 11:49:08.668 7f974d04d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-02-10T11:49:08.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.673 7fc456ffd700 1 -- 172.21.2.101:0/1264164334 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v166) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc450009310 con 0x7fc45818bc70 2021-02-10T11:49:08.712 INFO:teuthology.orchestra.run.gibba001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-02-10T11:49:08.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.718 7fc460213700 1 -- 172.21.2.101:0/1264164334 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4380186c0 msgr2=0x7fc43801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:08.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.718 7fc460213700 1 --2- 172.21.2.101:0/1264164334 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4380186c0 0x7fc43801abc0 crc :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:08.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.718 7fc460213700 1 -- 172.21.2.101:0/1264164334 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc45818bc70 msgr2=0x7fc4580c0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:08.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.718 7fc460213700 1 --2- 172.21.2.101:0/1264164334 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc45818bc70 0x7fc4580c0650 secure :-1 s=READY pgs=3335 cs=0 l=1 rev1=1 rx=0x7fc450006f10 tx=0x7fc45000b960).stop 2021-02-10T11:49:08.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.718 7fc460213700 1 -- 172.21.2.101:0/1264164334 shutdown_connections 2021-02-10T11:49:08.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.719 7fc460213700 1 --2- 172.21.2.101:0/1264164334 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc45818bc70 0x7fc4580c0650 unknown :-1 s=CLOSED pgs=3335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:08.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.719 7fc460213700 1 --2- 172.21.2.101:0/1264164334 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc45818b870 0x7fc4580c0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:08.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.719 7fc460213700 1 --2- 172.21.2.101:0/1264164334 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc4380186c0 0x7fc43801abc0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:08.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.719 7fc460213700 1 --2- 172.21.2.101:0/1264164334 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc4580bdda0 0x7fc45818b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:08.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.719 7fc460213700 1 -- 172.21.2.101:0/1264164334 >> 172.21.2.101:0/1264164334 conn(0x7fc4580a6eb0 msgr2=0x7fc458098ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:08.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.719 7fc460213700 1 -- 172.21.2.101:0/1264164334 shutdown_connections 2021-02-10T11:49:08.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:08.719 7fc460213700 1 -- 172.21.2.101:0/1264164334 wait complete. 2021-02-10T11:49:08.719 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:49:08.724 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:49:09.152 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.140 7fe834b9a700 1 Processor -- start 2021-02-10T11:49:09.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.140 7fe834b9a700 1 -- start start 2021-02-10T11:49:09.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.141 7fe834b9a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe8300d1870 0x7fe8301b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:09.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.141 7fe834b9a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe8301b0a10 0x7fe8300cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:09.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.141 7fe834b9a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8300d1c70 0x7fe8300cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:09.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.141 7fe834b9a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8300e5ed0 con 0x7fe8301b0a10 2021-02-10T11:49:09.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.141 7fe834b9a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8300f0b10 con 0x7fe8300d1870 2021-02-10T11:49:09.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.141 7fe834b9a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8300cff70 con 0x7fe8300d1c70 2021-02-10T11:49:09.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.141 7fe82e05d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe8301b0a10 0x7fe8300cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:09.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.141 7fe82e05d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe8301b0a10 0x7fe8300cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35538/0 (socket says 172.21.2.101:35538) 2021-02-10T11:49:09.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.141 7fe82e05d700 1 -- 172.21.2.101:0/3217993254 learned_addr learned my addr 172.21.2.101:0/3217993254 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:09.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.141 7fe82e05d700 1 -- 172.21.2.101:0/3217993254 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8300d1c70 msgr2=0x7fe8300cf950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:49:09.153 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.141 7fe82e05d700 1 --2- 172.21.2.101:0/3217993254 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8300d1c70 0x7fe8300cf950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:09.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.141 7fe82e05d700 1 -- 172.21.2.101:0/3217993254 >> v2:172.21.2.108:3300/0 conn(0x7fe8300d1870 msgr2=0x7fe8301b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:49:09.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.141 7fe82e05d700 1 --2- 172.21.2.101:0/3217993254 >> v2:172.21.2.108:3300/0 conn(0x7fe8300d1870 0x7fe8301b03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:09.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.141 7fe82e05d700 1 -- 172.21.2.101:0/3217993254 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8300d0150 con 0x7fe8301b0a10 2021-02-10T11:49:09.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.141 7fe82e05d700 1 --2- 172.21.2.101:0/3217993254 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8301b0a10 0x7fe8300cf330 secure :-1 s=READY pgs=3338 cs=0 l=1 rev1=1 rx=0x7fe818006700 tx=0x7fe818009b80).ready entity=mon.0 client_cookie=f887323d05cd8db2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:09.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.141 7fe82d85c700 1 -- 172.21.2.101:0/3217993254 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe81800c0c0 con 0x7fe8301b0a10 2021-02-10T11:49:09.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.142 7fe82d85c700 1 -- 172.21.2.101:0/3217993254 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81800c340 con 0x7fe8301b0a10 2021-02-10T11:49:09.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.142 7fe82d85c700 1 -- 172.21.2.101:0/3217993254 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe81800c860 con 0x7fe8301b0a10 2021-02-10T11:49:09.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.142 7fe834b9a700 1 -- 172.21.2.101:0/3217993254 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8301b0a10 msgr2=0x7fe8300cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:09.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.142 7fe834b9a700 1 --2- 172.21.2.101:0/3217993254 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8301b0a10 0x7fe8300cf330 secure :-1 s=READY pgs=3338 cs=0 l=1 rev1=1 rx=0x7fe818006700 tx=0x7fe818009b80).stop 2021-02-10T11:49:09.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.142 7fe834b9a700 1 -- 172.21.2.101:0/3217993254 shutdown_connections 2021-02-10T11:49:09.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.142 7fe834b9a700 1 --2- 172.21.2.101:0/3217993254 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8300d1c70 0x7fe8300cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:09.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.142 7fe834b9a700 1 --2- 172.21.2.101:0/3217993254 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8301b0a10 0x7fe8300cf330 unknown :-1 s=CLOSED pgs=3338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:09.154 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.142 7fe834b9a700 1 --2- 172.21.2.101:0/3217993254 >> v2:172.21.2.108:3300/0 conn(0x7fe8300d1870 0x7fe8301b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:09.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.142 7fe834b9a700 1 -- 172.21.2.101:0/3217993254 >> 172.21.2.101:0/3217993254 conn(0x7fe8300ce450 msgr2=0x7fe8300d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:09.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.142 7fe834b9a700 1 -- 172.21.2.101:0/3217993254 shutdown_connections 2021-02-10T11:49:09.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.142 7fe834b9a700 1 -- 172.21.2.101:0/3217993254 wait complete. 2021-02-10T11:49:09.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.157 7fe834b9a700 1 Processor -- start 2021-02-10T11:49:09.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.157 7fe834b9a700 1 -- start start 2021-02-10T11:49:09.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.157 7fe834b9a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8300aeea0 0x7fe8301b81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:09.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.157 7fe834b9a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe830096630 0x7fe8301b87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:09.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.157 7fe834b9a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8300953f0 0x7fe8300957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:09.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.157 7fe834b9a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe830189550 con 0x7fe8300953f0 2021-02-10T11:49:09.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.157 7fe834b9a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe830095e10 con 0x7fe830096630 2021-02-10T11:49:09.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.157 7fe834b9a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe830096030 con 0x7fe8300aeea0 2021-02-10T11:49:09.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.157 7fe82f05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8300953f0 0x7fe8300957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:09.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.157 7fe82f05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8300953f0 0x7fe8300957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35544/0 (socket says 172.21.2.101:35544) 2021-02-10T11:49:09.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.157 7fe82f05f700 1 -- 172.21.2.101:0/426178554 learned_addr learned my addr 172.21.2.101:0/426178554 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:09.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.158 7fe82f05f700 1 -- 172.21.2.101:0/426178554 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8300aeea0 msgr2=0x7fe8301b81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:09.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.158 7fe82f05f700 1 --2- 172.21.2.101:0/426178554 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8300aeea0 0x7fe8301b81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:09.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.158 7fe82f05f700 1 -- 172.21.2.101:0/426178554 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe830096630 msgr2=0x7fe8301b87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:09.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.158 7fe82f05f700 1 --2- 172.21.2.101:0/426178554 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe830096630 0x7fe8301b87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:09.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.158 7fe82f05f700 1 -- 172.21.2.101:0/426178554 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe830096250 con 0x7fe8300953f0 2021-02-10T11:49:09.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.158 7fe82f05f700 1 --2- 172.21.2.101:0/426178554 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8300953f0 0x7fe8300957f0 secure :-1 s=READY pgs=3339 cs=0 l=1 rev1=1 rx=0x7fe828009960 tx=0x7fe82800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:09.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.158 7fe8277fe700 1 -- 172.21.2.101:0/426178554 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe828000d10 con 0x7fe8300953f0 2021-02-10T11:49:09.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.158 7fe8277fe700 1 -- 172.21.2.101:0/426178554 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8280030c0 con 0x7fe8300953f0 2021-02-10T11:49:09.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.158 7fe8277fe700 1 -- 172.21.2.101:0/426178554 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe828001500 con 0x7fe8300953f0 2021-02-10T11:49:09.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.158 7fe834b9a700 1 -- 172.21.2.101:0/426178554 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe830096250 con 0x7fe8300953f0 2021-02-10T11:49:09.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.158 7fe834b9a700 1 -- 172.21.2.101:0/426178554 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe830189dc0 con 0x7fe8300953f0 2021-02-10T11:49:09.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.159 7fe8277fe700 1 -- 172.21.2.101:0/426178554 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe828001310 con 0x7fe8300953f0 2021-02-10T11:49:09.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.159 7fe8277fe700 1 --2- 172.21.2.101:0/426178554 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8140186c0 0x7fe81401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:09.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.159 7fe8277fe700 1 -- 172.21.2.101:0/426178554 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fe828032980 con 0x7fe8300953f0 2021-02-10T11:49:09.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.159 7fe82e85e700 1 --2- 172.21.2.101:0/426178554 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8140186c0 0x7fe81401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:09.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.159 7fe82e85e700 1 --2- 172.21.2.101:0/426178554 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8140186c0 0x7fe81401abc0 crc :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:09.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.190 7fe834b9a700 1 -- 172.21.2.101:0/426178554 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8300ce450 con 0x7fe8300953f0 2021-02-10T11:49:09.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.193 7fe8277fe700 1 -- 172.21.2.101:0/426178554 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe828000f40 con 0x7fe8300953f0 2021-02-10T11:49:09.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.323 7fe834b9a700 1 -- 172.21.2.101:0/426178554 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8300ce450 con 0x7fe8300953f0 2021-02-10T11:49:09.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.323 7fe8277fe700 1 -- 172.21.2.101:0/426178554 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) v1 ==== 95+0+1760 (secure 0 0 0) 0x7fe828001310 con 0x7fe8300953f0 2021-02-10T11:49:09.320 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:49:09.320 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":166,"default_fscid":20,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":13530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.108:6833/347510536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":347510536},{"type":"v1","addr":"172.21.2.108:6833","nonce":347510536}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":160}],"filesystems":[{"mdsmap":{"epoch":166,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:48:24.824003","modified":"2021-02-10 11:49:08.669465","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":192,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[41],"metadata_pool":40,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":20}]} 2021-02-10T11:49:09.320 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 166 2021-02-10T11:49:09.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.325 7fe834b9a700 1 -- 172.21.2.101:0/426178554 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8140186c0 msgr2=0x7fe81401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:09.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.325 7fe834b9a700 1 --2- 172.21.2.101:0/426178554 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8140186c0 0x7fe81401abc0 crc :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:09.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.325 7fe834b9a700 1 -- 172.21.2.101:0/426178554 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8300953f0 msgr2=0x7fe8300957f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:09.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.325 7fe834b9a700 1 --2- 172.21.2.101:0/426178554 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8300953f0 0x7fe8300957f0 secure :-1 s=READY pgs=3339 cs=0 l=1 rev1=1 rx=0x7fe828009960 tx=0x7fe82800bcf0).stop 2021-02-10T11:49:09.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.325 7fe834b9a700 1 -- 172.21.2.101:0/426178554 shutdown_connections 2021-02-10T11:49:09.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.325 7fe834b9a700 1 --2- 172.21.2.101:0/426178554 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe8300953f0 0x7fe8300957f0 unknown :-1 s=CLOSED pgs=3339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:09.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.325 7fe834b9a700 1 --2- 172.21.2.101:0/426178554 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe830096630 0x7fe8301b87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:09.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.325 7fe834b9a700 1 --2- 172.21.2.101:0/426178554 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe8140186c0 0x7fe81401abc0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:09.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.325 7fe834b9a700 1 --2- 172.21.2.101:0/426178554 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe8300aeea0 0x7fe8301b81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:09.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.325 7fe834b9a700 1 -- 172.21.2.101:0/426178554 >> 172.21.2.101:0/426178554 conn(0x7fe8300efc40 msgr2=0x7fe8301b9080 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:09.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.326 7fe834b9a700 1 -- 172.21.2.101:0/426178554 shutdown_connections 2021-02-10T11:49:09.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.326 7fe834b9a700 1 -- 172.21.2.101:0/426178554 wait complete. 2021-02-10T11:49:09.330 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-02-10T11:49:09.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.752 7f9347351700 1 Processor -- start 2021-02-10T11:49:09.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.752 7f9347351700 1 -- start start 2021-02-10T11:49:09.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.753 7f9347351700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93400d0340 0x7f93400ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:09.765 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.753 7f9347351700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f93400d1c70 0x7f93400efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:09.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.753 7f9347351700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93400f0b10 con 0x7f93400d1870 2021-02-10T11:49:09.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.753 7f9347351700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93400e5ed0 con 0x7f93400d1c70 2021-02-10T11:49:09.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.753 7f9347351700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93401b0be0 con 0x7f93400d0340 2021-02-10T11:49:09.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.753 7f934496c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93400d0340 0x7f93400ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:09.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.753 7f934496c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93400d0340 0x7f93400ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:56514/0 (socket says 172.21.2.101:56514) 2021-02-10T11:49:09.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.753 7f934496c700 1 -- 172.21.2.101:0/3110228677 learned_addr learned my addr 172.21.2.101:0/3110228677 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:09.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.753 7f934596e700 1 --2- 172.21.2.101:0/3110228677 >> v2:172.21.2.108:3300/0 conn(0x7f93400d1c70 0x7f93400efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:09.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.753 7f934596e700 1 -- 172.21.2.101:0/3110228677 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93400d0340 msgr2=0x7f93400ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:09.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.753 7f934596e700 1 --2- 172.21.2.101:0/3110228677 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93400d0340 0x7f93400ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:09.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.753 7f934596e700 1 -- 172.21.2.101:0/3110228677 >> v1:172.21.2.108:6789/0 conn(0x7f93400d1870 legacy=0x7f93401b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:09.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.753 7f934596e700 1 -- 172.21.2.101:0/3110228677 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93400f20a0 con 0x7f93400d1c70 2021-02-10T11:49:09.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.753 7f934496c700 1 --2- 172.21.2.101:0/3110228677 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93400d0340 0x7f93400ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:49:09.766 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.754 7f934596e700 1 --2- 172.21.2.101:0/3110228677 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93400d1c70 0x7f93400efc60 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f9338006e50 tx=0x7f9338009e70).ready entity=mon.1 client_cookie=e795e3d224ec1063 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:09.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.754 7f9333fff700 1 -- 172.21.2.101:0/3110228677 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f933800b510 con 0x7f93400d1c70 2021-02-10T11:49:09.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.754 7f9333fff700 1 -- 172.21.2.101:0/3110228677 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9338006790 con 0x7f93400d1c70 2021-02-10T11:49:09.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.754 7f9333fff700 1 -- 172.21.2.101:0/3110228677 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f933800b940 con 0x7f93400d1c70 2021-02-10T11:49:09.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.754 7f9347351700 1 -- 172.21.2.101:0/3110228677 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93400d1c70 msgr2=0x7f93400efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:09.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.754 7f9347351700 1 --2- 172.21.2.101:0/3110228677 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93400d1c70 0x7f93400efc60 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f9338006e50 tx=0x7f9338009e70).stop 2021-02-10T11:49:09.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.755 7f9347351700 1 -- 172.21.2.101:0/3110228677 shutdown_connections 2021-02-10T11:49:09.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.755 7f9347351700 1 --2- 172.21.2.101:0/3110228677 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93400d0340 0x7f93400ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:09.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.755 7f9347351700 1 --2- 172.21.2.101:0/3110228677 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93400d1c70 0x7f93400efc60 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:09.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.755 7f9347351700 1 -- 172.21.2.101:0/3110228677 >> 172.21.2.101:0/3110228677 conn(0x7f93400ce450 msgr2=0x7f93400d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:09.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.755 7f9347351700 1 -- 172.21.2.101:0/3110228677 shutdown_connections 2021-02-10T11:49:09.767 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.755 7f9347351700 1 -- 172.21.2.101:0/3110228677 wait complete. 2021-02-10T11:49:09.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.769 7f9347351700 1 Processor -- start 2021-02-10T11:49:09.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.769 7f9347351700 1 -- start start 2021-02-10T11:49:09.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.770 7f9347351700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93401a06c0 0x7f93400ebe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:09.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.770 7f9347351700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93400ec490 0x7f93400c30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:09.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.770 7f9347351700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93400c3710 0x7f93400bc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:09.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.770 7f9347351700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93400c3b10 con 0x7f93400ec490 2021-02-10T11:49:09.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.770 7f9347351700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93400eca90 con 0x7f93401a06c0 2021-02-10T11:49:09.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.770 7f9347351700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93400bc690 con 0x7f93400c3710 2021-02-10T11:49:09.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.770 7f934496c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93400ec490 0x7f93400c30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:09.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.770 7f934496c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93400ec490 0x7f93400c30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35556/0 (socket says 172.21.2.101:35556) 2021-02-10T11:49:09.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.770 7f934496c700 1 -- 172.21.2.101:0/2685937474 learned_addr learned my addr 172.21.2.101:0/2685937474 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:09.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.770 7f934516d700 1 --2- 172.21.2.101:0/2685937474 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93401a06c0 0x7f93400ebe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:09.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.770 7f934596e700 1 --2- 172.21.2.101:0/2685937474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93400c3710 0x7f93400bc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:09.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.771 7f934496c700 1 -- 172.21.2.101:0/2685937474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93400c3710 msgr2=0x7f93400bc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:09.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.771 7f934496c700 1 --2- 172.21.2.101:0/2685937474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93400c3710 0x7f93400bc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:09.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.771 7f934496c700 1 -- 172.21.2.101:0/2685937474 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93401a06c0 msgr2=0x7f93400ebe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:09.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.771 7f934496c700 1 --2- 172.21.2.101:0/2685937474 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93401a06c0 0x7f93400ebe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:09.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.771 7f934496c700 1 -- 172.21.2.101:0/2685937474 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93400bc870 con 0x7f93400ec490 2021-02-10T11:49:09.770 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.771 7f934496c700 1 --2- 172.21.2.101:0/2685937474 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93400ec490 0x7f93400c30f0 secure :-1 s=READY pgs=3340 cs=0 l=1 rev1=1 rx=0x7f933c006ee0 tx=0x7f933c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:09.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.771 7f9331ffb700 1 -- 172.21.2.101:0/2685937474 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f933c00bd50 con 0x7f93400ec490 2021-02-10T11:49:09.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.771 7f9331ffb700 1 -- 172.21.2.101:0/2685937474 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933c0052e0 con 0x7f93400ec490 2021-02-10T11:49:09.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.771 7f9331ffb700 1 -- 172.21.2.101:0/2685937474 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f933c0057c0 con 0x7f93400ec490 2021-02-10T11:49:09.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.771 7f9347351700 1 -- 172.21.2.101:0/2685937474 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93400bc870 con 0x7f93400ec490 2021-02-10T11:49:09.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.771 7f9347351700 1 -- 172.21.2.101:0/2685937474 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93400bce00 con 0x7f93400ec490 2021-02-10T11:49:09.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.772 7f9331ffb700 1 -- 172.21.2.101:0/2685937474 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f933c0054c0 con 0x7f93400ec490 2021-02-10T11:49:09.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.772 7f9331ffb700 1 --2- 172.21.2.101:0/2685937474 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93200186d0 0x7f932001abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:09.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.772 7f9331ffb700 1 -- 172.21.2.101:0/2685937474 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f933c0326c0 con 0x7f93400ec490 2021-02-10T11:49:09.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.772 7f934516d700 1 --2- 172.21.2.101:0/2685937474 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93200186d0 0x7f932001abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:09.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.772 7f934516d700 1 --2- 172.21.2.101:0/2685937474 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93200186d0 0x7f932001abd0 crc :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:09.798 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.802 7f9347351700 1 -- 172.21.2.101:0/2685937474 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93400ebc40 con 0x7f93400ec490 2021-02-10T11:49:09.801 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.805 7f9331ffb700 1 -- 172.21.2.101:0/2685937474 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f933c0009b0 con 0x7f93400ec490 2021-02-10T11:49:09.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:09.930 7f9347351700 1 -- 172.21.2.101:0/2685937474 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f93400f0b10 con 0x7f93400ec490 2021-02-10T11:49:10.371 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:49:10.374 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:49:10.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:10.675 7f9331ffb700 1 -- 172.21.2.101:0/2685937474 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v167) v1 ==== 104+0+0 (secure 0 0 0) 0x7f933c0052e0 con 0x7f93400ec490 2021-02-10T11:49:10.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:10.700 7f9347351700 1 -- 172.21.2.101:0/2685937474 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93200186d0 msgr2=0x7f932001abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:10.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:10.700 7f9347351700 1 --2- 172.21.2.101:0/2685937474 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93200186d0 0x7f932001abd0 crc :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:10.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:10.700 7f9347351700 1 -- 172.21.2.101:0/2685937474 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93400ec490 msgr2=0x7f93400c30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:10.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:10.700 7f9347351700 1 --2- 172.21.2.101:0/2685937474 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93400ec490 0x7f93400c30f0 secure :-1 s=READY pgs=3340 cs=0 l=1 rev1=1 rx=0x7f933c006ee0 tx=0x7f933c00b960).stop 2021-02-10T11:49:10.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:10.700 7f9347351700 1 -- 172.21.2.101:0/2685937474 shutdown_connections 2021-02-10T11:49:10.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:10.700 7f9347351700 1 --2- 172.21.2.101:0/2685937474 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f93200186d0 0x7f932001abd0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:10.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:10.700 7f9347351700 1 --2- 172.21.2.101:0/2685937474 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f93400c3710 0x7f93400bc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:10.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:10.700 7f9347351700 1 --2- 172.21.2.101:0/2685937474 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f93400ec490 0x7f93400c30f0 unknown :-1 s=CLOSED pgs=3340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:10.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:10.700 7f9347351700 1 --2- 172.21.2.101:0/2685937474 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f93401a06c0 0x7f93400ebe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:10.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:10.700 7f9347351700 1 -- 172.21.2.101:0/2685937474 >> 172.21.2.101:0/2685937474 conn(0x7f93400a6bb0 msgr2=0x7f9340095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:10.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:10.700 7f9347351700 1 -- 172.21.2.101:0/2685937474 shutdown_connections 2021-02-10T11:49:10.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:10.700 7f9347351700 1 -- 172.21.2.101:0/2685937474 wait complete. 2021-02-10T11:49:10.705 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-02-10T11:49:11.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.127 7f8447738700 1 Processor -- start 2021-02-10T11:49:11.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.127 7f8447738700 1 -- start start 2021-02-10T11:49:11.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.128 7f8447738700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f84400d1880 0x7f84401b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:11.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.128 7f8447738700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f84401b0af0 0x7f8440108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:11.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.128 7f8447738700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8440109a70 con 0x7f84400d1c80 2021-02-10T11:49:11.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.128 7f8447738700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84400e5f00 con 0x7f84401b0af0 2021-02-10T11:49:11.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.128 7f8447738700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8440109760 con 0x7f84400d1880 2021-02-10T11:49:11.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.128 7f8444d53700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f84401b0af0 0x7f8440108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:11.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.128 7f8444d53700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f84401b0af0 0x7f8440108950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35564/0 (socket says 172.21.2.101:35564) 2021-02-10T11:49:11.140 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.128 7f8444d53700 1 -- 172.21.2.101:0/1469729038 learned_addr learned my addr 172.21.2.101:0/1469729038 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:11.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.128 7f8444d53700 1 -- 172.21.2.101:0/1469729038 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f84400d1880 msgr2=0x7f84401b04d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:49:11.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.128 7f8444d53700 1 --2- 172.21.2.101:0/1469729038 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f84400d1880 0x7f84401b04d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:11.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.128 7f8444d53700 1 -- 172.21.2.101:0/1469729038 >> v1:172.21.2.108:6789/0 conn(0x7f84400d1c80 legacy=0x7f8440108f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:49:11.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.128 7f8444d53700 1 -- 172.21.2.101:0/1469729038 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8440088070 con 0x7f84401b0af0 2021-02-10T11:49:11.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.128 7f8444d53700 1 --2- 172.21.2.101:0/1469729038 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84401b0af0 0x7f8440108950 secure :-1 s=READY pgs=3341 cs=0 l=1 rev1=1 rx=0x7f8434006700 tx=0x7f8434009b00).ready entity=mon.0 client_cookie=b4849de60b251cd7 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:11.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.128 7f8433fff700 1 -- 172.21.2.101:0/1469729038 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f843400c0c0 con 0x7f84401b0af0 2021-02-10T11:49:11.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.128 7f8433fff700 1 -- 172.21.2.101:0/1469729038 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843400c340 con 0x7f84401b0af0 2021-02-10T11:49:11.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.128 7f8433fff700 1 -- 172.21.2.101:0/1469729038 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f843400c820 con 0x7f84401b0af0 2021-02-10T11:49:11.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.128 7f8447738700 1 -- 172.21.2.101:0/1469729038 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84401b0af0 msgr2=0x7f8440108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:11.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.128 7f8447738700 1 --2- 172.21.2.101:0/1469729038 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84401b0af0 0x7f8440108950 secure :-1 s=READY pgs=3341 cs=0 l=1 rev1=1 rx=0x7f8434006700 tx=0x7f8434009b00).stop 2021-02-10T11:49:11.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.129 7f8447738700 1 -- 172.21.2.101:0/1469729038 shutdown_connections 2021-02-10T11:49:11.141 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.129 7f8447738700 1 --2- 172.21.2.101:0/1469729038 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f84401b0af0 0x7f8440108950 unknown :-1 s=CLOSED pgs=3341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:11.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.129 7f8447738700 1 --2- 172.21.2.101:0/1469729038 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f84400d1880 0x7f84401b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:11.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.129 7f8447738700 1 -- 172.21.2.101:0/1469729038 >> 172.21.2.101:0/1469729038 conn(0x7f84400ce430 msgr2=0x7f84400d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:11.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.129 7f8447738700 1 -- 172.21.2.101:0/1469729038 shutdown_connections 2021-02-10T11:49:11.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.129 7f8447738700 1 -- 172.21.2.101:0/1469729038 wait complete. 2021-02-10T11:49:11.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.144 7f8447738700 1 Processor -- start 2021-02-10T11:49:11.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.144 7f8447738700 1 -- start start 2021-02-10T11:49:11.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.144 7f8447738700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f844019e5b0 0x7f84401afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:11.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.144 7f8447738700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f844019e9b0 0x7f84401b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:11.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.144 7f8447738700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f84401b0250 0x7f84401b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:11.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.144 7f8447738700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8440195bd0 con 0x7f844019e5b0 2021-02-10T11:49:11.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.144 7f8447738700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84401b0840 con 0x7f84401b0250 2021-02-10T11:49:11.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.144 7f8447738700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f844019eed0 con 0x7f844019e9b0 2021-02-10T11:49:11.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.144 7f8445554700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f844019e5b0 0x7f84401afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:11.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.144 7f8445554700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f844019e5b0 0x7f84401afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35570/0 (socket says 172.21.2.101:35570) 2021-02-10T11:49:11.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.144 7f8445554700 1 -- 172.21.2.101:0/1019017596 learned_addr learned my addr 172.21.2.101:0/1019017596 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:11.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.144 7f8444d53700 1 --2- 172.21.2.101:0/1019017596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f844019e9b0 0x7f84401b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:11.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.145 7f8445d55700 1 --2- 172.21.2.101:0/1019017596 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f84401b0250 0x7f84401b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:11.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.145 7f8445554700 1 -- 172.21.2.101:0/1019017596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f844019e9b0 msgr2=0x7f84401b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:11.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.145 7f8445554700 1 --2- 172.21.2.101:0/1019017596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f844019e9b0 0x7f84401b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:11.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.145 7f8445554700 1 -- 172.21.2.101:0/1019017596 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f84401b0250 msgr2=0x7f84401b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:11.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.145 7f8445554700 1 --2- 172.21.2.101:0/1019017596 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f84401b0250 0x7f84401b13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.145 7f8445554700 1 -- 172.21.2.101:0/1019017596 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f844019f0b0 con 0x7f844019e5b0 2021-02-10T11:49:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.145 7f8445554700 1 --2- 172.21.2.101:0/1019017596 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f844019e5b0 0x7f84401afc30 secure :-1 s=READY pgs=3342 cs=0 l=1 rev1=1 rx=0x7f843c00b5c0 tx=0x7f843c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.145 7f8431ffb700 1 -- 172.21.2.101:0/1019017596 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f843c00bd50 con 0x7f844019e5b0 2021-02-10T11:49:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.145 7f8431ffb700 1 -- 172.21.2.101:0/1019017596 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843c006030 con 0x7f844019e5b0 2021-02-10T11:49:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.145 7f8447738700 1 -- 172.21.2.101:0/1019017596 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f844019f0b0 con 0x7f844019e5b0 2021-02-10T11:49:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.145 7f8447738700 1 -- 172.21.2.101:0/1019017596 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84401b1cf0 con 0x7f844019e5b0 2021-02-10T11:49:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.145 7f8431ffb700 1 -- 172.21.2.101:0/1019017596 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f843c0054d0 con 0x7f844019e5b0 2021-02-10T11:49:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.146 7f8431ffb700 1 -- 172.21.2.101:0/1019017596 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f843c006210 con 0x7f844019e5b0 2021-02-10T11:49:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.146 7f8431ffb700 1 --2- 172.21.2.101:0/1019017596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84200186c0 0x7f842001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.146 7f8431ffb700 1 -- 172.21.2.101:0/1019017596 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f843c032820 con 0x7f844019e5b0 2021-02-10T11:49:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.146 7f8444d53700 1 --2- 172.21.2.101:0/1019017596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84200186c0 0x7f842001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:11.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.146 7f8444d53700 1 --2- 172.21.2.101:0/1019017596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84200186c0 0x7f842001abc0 crc :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:11.174 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.178 7f8447738700 1 -- 172.21.2.101:0/1019017596 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84401b0840 con 0x7f844019e5b0 2021-02-10T11:49:11.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.180 7f8431ffb700 1 -- 172.21.2.101:0/1019017596 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f843c000960 con 0x7f844019e5b0 2021-02-10T11:49:11.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.308 7f8447738700 1 -- 172.21.2.101:0/1019017596 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f84401b0840 con 0x7f844019e5b0 2021-02-10T11:49:11.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.682 7f8431ffb700 1 -- 172.21.2.101:0/1019017596 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_metadata", "prefix": "osd pool delete", "pool": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v193) v1 ==== 185+0+0 (secure 0 0 0) 0x7f843c006250 con 0x7f844019e5b0 2021-02-10T11:49:11.721 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' removed 2021-02-10T11:49:11.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.726 7f8447738700 1 -- 172.21.2.101:0/1019017596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84200186c0 msgr2=0x7f842001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:11.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.726 7f8447738700 1 --2- 172.21.2.101:0/1019017596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84200186c0 0x7f842001abc0 crc :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:11.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.726 7f8447738700 1 -- 172.21.2.101:0/1019017596 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f844019e5b0 msgr2=0x7f84401afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:11.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.726 7f8447738700 1 --2- 172.21.2.101:0/1019017596 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f844019e5b0 0x7f84401afc30 secure :-1 s=READY pgs=3342 cs=0 l=1 rev1=1 rx=0x7f843c00b5c0 tx=0x7f843c00b960).stop 2021-02-10T11:49:11.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.727 7f8447738700 1 -- 172.21.2.101:0/1019017596 shutdown_connections 2021-02-10T11:49:11.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.727 7f8447738700 1 --2- 172.21.2.101:0/1019017596 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84200186c0 0x7f842001abc0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:11.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.727 7f8447738700 1 --2- 172.21.2.101:0/1019017596 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f844019e9b0 0x7f84401b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:11.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.727 7f8447738700 1 --2- 172.21.2.101:0/1019017596 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f84401b0250 0x7f84401b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:11.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.727 7f8447738700 1 --2- 172.21.2.101:0/1019017596 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f844019e5b0 0x7f84401afc30 unknown :-1 s=CLOSED pgs=3342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:11.723 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.727 7f8447738700 1 -- 172.21.2.101:0/1019017596 >> 172.21.2.101:0/1019017596 conn(0x7f844019f940 msgr2=0x7f844018c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:11.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.727 7f8447738700 1 -- 172.21.2.101:0/1019017596 shutdown_connections 2021-02-10T11:49:11.724 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:11.727 7f8447738700 1 -- 172.21.2.101:0/1019017596 wait complete. 2021-02-10T11:49:11.731 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-02-10T11:49:12.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.152 7f0a74fc6700 1 Processor -- start 2021-02-10T11:49:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.152 7f0a74fc6700 1 -- start start 2021-02-10T11:49:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.153 7f0a74fc6700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0a700d1880 0x7f0a701b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.153 7f0a74fc6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a701b0af0 0x7f0a70108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.153 7f0a74fc6700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a700fe1d0 con 0x7f0a700d1c80 2021-02-10T11:49:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.153 7f0a74fc6700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a70109a70 con 0x7f0a700d1880 2021-02-10T11:49:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.153 7f0a74fc6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a70109760 con 0x7f0a701b0af0 2021-02-10T11:49:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.153 7f0a6ed9d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0a700d1880 0x7f0a701b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.153 7f0a6ed9d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f0a700d1880 0x7f0a701b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35578/0 (socket says 172.21.2.101:35578) 2021-02-10T11:49:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.153 7f0a6ed9d700 1 -- 172.21.2.101:0/171949078 learned_addr learned my addr 172.21.2.101:0/171949078 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.153 7f0a6ed9d700 1 -- 172.21.2.101:0/171949078 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a701b0af0 msgr2=0x7f0a70108950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:49:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.153 7f0a6ed9d700 1 --2- 172.21.2.101:0/171949078 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a701b0af0 0x7f0a70108950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.153 7f0a6ed9d700 1 -- 172.21.2.101:0/171949078 >> v1:172.21.2.108:6789/0 conn(0x7f0a700d1c80 legacy=0x7f0a70108f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:49:12.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.153 7f0a6ed9d700 1 -- 172.21.2.101:0/171949078 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a70088070 con 0x7f0a700d1880 2021-02-10T11:49:12.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.153 7f0a6ed9d700 1 --2- 172.21.2.101:0/171949078 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a700d1880 0x7f0a701b04d0 secure :-1 s=READY pgs=3343 cs=0 l=1 rev1=1 rx=0x7f0a58006700 tx=0x7f0a58009b80).ready entity=mon.0 client_cookie=b2404bca6d62111 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:12.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.153 7f0a6dd9b700 1 -- 172.21.2.101:0/171949078 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0a5800c0c0 con 0x7f0a700d1880 2021-02-10T11:49:12.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.153 7f0a6dd9b700 1 -- 172.21.2.101:0/171949078 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5800c340 con 0x7f0a700d1880 2021-02-10T11:49:12.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.153 7f0a6dd9b700 1 -- 172.21.2.101:0/171949078 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0a5800c860 con 0x7f0a700d1880 2021-02-10T11:49:12.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.153 7f0a74fc6700 1 -- 172.21.2.101:0/171949078 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a700d1880 msgr2=0x7f0a701b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:12.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.153 7f0a74fc6700 1 --2- 172.21.2.101:0/171949078 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a700d1880 0x7f0a701b04d0 secure :-1 s=READY pgs=3343 cs=0 l=1 rev1=1 rx=0x7f0a58006700 tx=0x7f0a58009b80).stop 2021-02-10T11:49:12.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.154 7f0a74fc6700 1 -- 172.21.2.101:0/171949078 shutdown_connections 2021-02-10T11:49:12.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.154 7f0a74fc6700 1 --2- 172.21.2.101:0/171949078 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a701b0af0 0x7f0a70108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:12.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.154 7f0a74fc6700 1 --2- 172.21.2.101:0/171949078 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a700d1880 0x7f0a701b04d0 unknown :-1 s=CLOSED pgs=3343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:12.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.154 7f0a74fc6700 1 -- 172.21.2.101:0/171949078 >> 172.21.2.101:0/171949078 conn(0x7f0a700ce430 msgr2=0x7f0a700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:12.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.154 7f0a74fc6700 1 -- 172.21.2.101:0/171949078 shutdown_connections 2021-02-10T11:49:12.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.154 7f0a74fc6700 1 -- 172.21.2.101:0/171949078 wait complete. 2021-02-10T11:49:12.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.169 7f0a74fc6700 1 Processor -- start 2021-02-10T11:49:12.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.169 7f0a74fc6700 1 -- start start 2021-02-10T11:49:12.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.169 7f0a74fc6700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0a7008fb40 0x7f0a70195720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:12.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.169 7f0a74fc6700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a7009b4f0 0x7f0a7009b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:12.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.169 7f0a74fc6700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a7009bf10 0x7f0a701940a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:12.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.169 7f0a74fc6700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a701951a0 con 0x7f0a7009b4f0 2021-02-10T11:49:12.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.169 7f0a74fc6700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a70094990 con 0x7f0a7008fb40 2021-02-10T11:49:12.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.169 7f0a74fc6700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a701946c0 con 0x7f0a7009bf10 2021-02-10T11:49:12.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.169 7f0a6e59c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a7009b4f0 0x7f0a7009b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:12.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.169 7f0a6e59c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a7009b4f0 0x7f0a7009b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35586/0 (socket says 172.21.2.101:35586) 2021-02-10T11:49:12.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.169 7f0a6e59c700 1 -- 172.21.2.101:0/1466475787 learned_addr learned my addr 172.21.2.101:0/1466475787 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:12.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.170 7f0a6e59c700 1 -- 172.21.2.101:0/1466475787 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a7009bf10 msgr2=0x7f0a701940a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:12.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.170 7f0a6e59c700 1 --2- 172.21.2.101:0/1466475787 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a7009bf10 0x7f0a701940a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:12.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.170 7f0a6e59c700 1 -- 172.21.2.101:0/1466475787 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0a7008fb40 msgr2=0x7f0a70195720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:12.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.170 7f0a6e59c700 1 --2- 172.21.2.101:0/1466475787 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0a7008fb40 0x7f0a70195720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:12.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.170 7f0a6e59c700 1 -- 172.21.2.101:0/1466475787 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a701948e0 con 0x7f0a7009b4f0 2021-02-10T11:49:12.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.170 7f0a6e59c700 1 --2- 172.21.2.101:0/1466475787 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a7009b4f0 0x7f0a7009b8f0 secure :-1 s=READY pgs=3344 cs=0 l=1 rev1=1 rx=0x7f0a6000b5c0 tx=0x7f0a6000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:12.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.170 7f0a677fe700 1 -- 172.21.2.101:0/1466475787 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0a6000bd50 con 0x7f0a7009b4f0 2021-02-10T11:49:12.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.170 7f0a677fe700 1 -- 172.21.2.101:0/1466475787 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a60006030 con 0x7f0a7009b4f0 2021-02-10T11:49:12.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.170 7f0a677fe700 1 -- 172.21.2.101:0/1466475787 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0a600054d0 con 0x7f0a7009b4f0 2021-02-10T11:49:12.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.170 7f0a74fc6700 1 -- 172.21.2.101:0/1466475787 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a701948e0 con 0x7f0a7009b4f0 2021-02-10T11:49:12.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.170 7f0a74fc6700 1 -- 172.21.2.101:0/1466475787 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a70194e70 con 0x7f0a7009b4f0 2021-02-10T11:49:12.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.171 7f0a677fe700 1 -- 172.21.2.101:0/1466475787 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0a60006210 con 0x7f0a7009b4f0 2021-02-10T11:49:12.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.171 7f0a677fe700 1 --2- 172.21.2.101:0/1466475787 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a540186c0 0x7f0a5401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:12.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.171 7f0a677fe700 1 -- 172.21.2.101:0/1466475787 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f0a60032460 con 0x7f0a7009b4f0 2021-02-10T11:49:12.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.171 7f0a6ed9d700 1 --2- 172.21.2.101:0/1466475787 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a540186c0 0x7f0a5401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:12.170 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.171 7f0a6ed9d700 1 --2- 172.21.2.101:0/1466475787 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a540186c0 0x7f0a5401abc0 crc :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:12.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.202 7f0a74fc6700 1 -- 172.21.2.101:0/1466475787 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a70082ab0 con 0x7f0a7009b4f0 2021-02-10T11:49:12.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.205 7f0a677fe700 1 -- 172.21.2.101:0/1466475787 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0a60025ff0 con 0x7f0a7009b4f0 2021-02-10T11:49:12.331 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.331 7f0a74fc6700 1 -- 172.21.2.101:0/1466475787 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0a70082ab0 con 0x7f0a7009b4f0 2021-02-10T11:49:12.687 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.690 7f0a677fe700 1 -- 172.21.2.101:0/1466475787 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"pool2": "cephfs_data", "prefix": "osd pool delete", "pool": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v194) v1 ==== 173+0+0 (secure 0 0 0) 0x7f0a60025ff0 con 0x7f0a7009b4f0 2021-02-10T11:49:12.694 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' removed 2021-02-10T11:49:12.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.699 7f0a74fc6700 1 -- 172.21.2.101:0/1466475787 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a540186c0 msgr2=0x7f0a5401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:12.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.699 7f0a74fc6700 1 --2- 172.21.2.101:0/1466475787 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a540186c0 0x7f0a5401abc0 crc :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:12.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.699 7f0a74fc6700 1 -- 172.21.2.101:0/1466475787 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a7009b4f0 msgr2=0x7f0a7009b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:12.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.699 7f0a74fc6700 1 --2- 172.21.2.101:0/1466475787 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a7009b4f0 0x7f0a7009b8f0 secure :-1 s=READY pgs=3344 cs=0 l=1 rev1=1 rx=0x7f0a6000b5c0 tx=0x7f0a6000b960).stop 2021-02-10T11:49:12.696 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.699 7f0a74fc6700 1 -- 172.21.2.101:0/1466475787 shutdown_connections 2021-02-10T11:49:12.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.699 7f0a74fc6700 1 --2- 172.21.2.101:0/1466475787 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0a540186c0 0x7f0a5401abc0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:12.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.700 7f0a74fc6700 1 --2- 172.21.2.101:0/1466475787 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0a7009bf10 0x7f0a701940a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:12.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.700 7f0a74fc6700 1 --2- 172.21.2.101:0/1466475787 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0a7009b4f0 0x7f0a7009b8f0 unknown :-1 s=CLOSED pgs=3344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:12.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.700 7f0a74fc6700 1 --2- 172.21.2.101:0/1466475787 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0a7008fb40 0x7f0a70195720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:12.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.700 7f0a74fc6700 1 -- 172.21.2.101:0/1466475787 >> 172.21.2.101:0/1466475787 conn(0x7f0a7009afc0 msgr2=0x7f0a700953e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:12.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.700 7f0a74fc6700 1 -- 172.21.2.101:0/1466475787 shutdown_connections 2021-02-10T11:49:12.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:12.700 7f0a74fc6700 1 -- 172.21.2.101:0/1466475787 wait complete. 2021-02-10T11:49:12.704 INFO:tasks.ceph.mds.a:Restarting daemon 2021-02-10T11:49:12.705 INFO:tasks.ceph.mds.a:Stopping old one... 2021-02-10T11:49:12.705 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-02-10T11:49:12.705 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:49:12.705 INFO:tasks.ceph.mds.b:Restarting daemon 2021-02-10T11:49:12.705 INFO:tasks.ceph.mds.b:Stopping old one... 2021-02-10T11:49:12.705 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-02-10T11:49:12.705 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:49:12.711 INFO:tasks.ceph.mds.a:Stopped 2021-02-10T11:49:12.711 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-02-10T11:49:12.712 INFO:tasks.ceph.mds.b:Stopped 2021-02-10T11:49:12.712 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-02-10T11:49:12.753 INFO:tasks.ceph.mds.a:Started 2021-02-10T11:49:12.753 DEBUG:teuthology.parallel:result is None 2021-02-10T11:49:12.754 INFO:tasks.ceph.mds.b:Started 2021-02-10T11:49:12.754 DEBUG:teuthology.parallel:result is None 2021-02-10T11:49:12.755 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd blacklist clear 2021-02-10T11:49:13.159 INFO:tasks.ceph.mds.b.gibba008.stdout:starting mds.b at 2021-02-10T11:49:13.161 INFO:tasks.ceph.mds.a.gibba001.stdout:starting mds.a at 2021-02-10T11:49:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.173 7f4d1b2df700 1 Processor -- start 2021-02-10T11:49:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.173 7f4d1b2df700 1 -- start start 2021-02-10T11:49:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.173 7f4d1b2df700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f4d140d0340 0x7f4d140ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.173 7f4d1b2df700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f4d140d1c70 0x7f4d140efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.173 7f4d1b2df700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d140f0b10 con 0x7f4d140d1870 2021-02-10T11:49:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.173 7f4d1b2df700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d140e5ed0 con 0x7f4d140d1c70 2021-02-10T11:49:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.173 7f4d1b2df700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d141b0be0 con 0x7f4d140d0340 2021-02-10T11:49:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.173 7f4d190fb700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f4d140d1870 0x7f4d141b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36096/0 (socket says 172.21.2.101:36096) 2021-02-10T11:49:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.173 7f4d190fb700 1 -- 172.21.2.101:0/3804338502 learned_addr learned my addr 172.21.2.101:0/3804338502 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:13.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.173 7f4d198fc700 1 --2- 172.21.2.101:0/3804338502 >> v2:172.21.2.101:3300/0 conn(0x7f4d140d1c70 0x7f4d140efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:13.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.173 7f4d0bfff700 1 -- 172.21.2.101:0/3804338502 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536765631 0 0) 0x7f4d0c001060 con 0x7f4d140d1870 2021-02-10T11:49:13.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.174 7f4d0bfff700 1 -- 172.21.2.101:0/3804338502 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d00002fe0 con 0x7f4d140d1870 2021-02-10T11:49:13.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.174 7f4d188fa700 1 --2- 172.21.2.101:0/3804338502 >> v2:172.21.2.108:3300/0 conn(0x7f4d140d0340 0x7f4d140ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:13.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.174 7f4d198fc700 1 -- 172.21.2.101:0/3804338502 >> v2:172.21.2.108:3300/0 conn(0x7f4d140d0340 msgr2=0x7f4d140ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:13.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.174 7f4d198fc700 1 --2- 172.21.2.101:0/3804338502 >> v2:172.21.2.108:3300/0 conn(0x7f4d140d0340 0x7f4d140ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:13.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.174 7f4d198fc700 1 -- 172.21.2.101:0/3804338502 >> v1:172.21.2.101:6789/0 conn(0x7f4d140d1870 legacy=0x7f4d141b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:13.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.174 7f4d198fc700 1 -- 172.21.2.101:0/3804338502 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d140f20a0 con 0x7f4d140d1c70 2021-02-10T11:49:13.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.174 7f4d198fc700 1 --2- 172.21.2.101:0/3804338502 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d140d1c70 0x7f4d140efc60 secure :-1 s=READY pgs=3348 cs=0 l=1 rev1=1 rx=0x7f4d10006ab0 tx=0x7f4d10009b40).ready entity=mon.0 client_cookie=7c9eefbcdaa0cbb5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:13.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.174 7f4d0bfff700 1 -- 172.21.2.101:0/3804338502 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d100022e0 con 0x7f4d140d1c70 2021-02-10T11:49:13.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.174 7f4d0bfff700 1 -- 172.21.2.101:0/3804338502 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d10002660 con 0x7f4d140d1c70 2021-02-10T11:49:13.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.174 7f4d0bfff700 1 -- 172.21.2.101:0/3804338502 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d10002b40 con 0x7f4d140d1c70 2021-02-10T11:49:13.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.174 7f4d1b2df700 1 -- 172.21.2.101:0/3804338502 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d140d1c70 msgr2=0x7f4d140efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:13.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.174 7f4d1b2df700 1 --2- 172.21.2.101:0/3804338502 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d140d1c70 0x7f4d140efc60 secure :-1 s=READY pgs=3348 cs=0 l=1 rev1=1 rx=0x7f4d10006ab0 tx=0x7f4d10009b40).stop 2021-02-10T11:49:13.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.175 7f4d1b2df700 1 -- 172.21.2.101:0/3804338502 shutdown_connections 2021-02-10T11:49:13.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.175 7f4d1b2df700 1 --2- 172.21.2.101:0/3804338502 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d140d1c70 0x7f4d140efc60 unknown :-1 s=CLOSED pgs=3348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:13.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.175 7f4d1b2df700 1 --2- 172.21.2.101:0/3804338502 >> v2:172.21.2.108:3300/0 conn(0x7f4d140d0340 0x7f4d140ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:13.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.175 7f4d1b2df700 1 -- 172.21.2.101:0/3804338502 >> 172.21.2.101:0/3804338502 conn(0x7f4d140ce450 msgr2=0x7f4d140d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:13.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.175 7f4d1b2df700 1 -- 172.21.2.101:0/3804338502 shutdown_connections 2021-02-10T11:49:13.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.175 7f4d1b2df700 1 -- 172.21.2.101:0/3804338502 wait complete. 2021-02-10T11:49:13.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.190 7f4d1b2df700 1 Processor -- start 2021-02-10T11:49:13.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.190 7f4d1b2df700 1 -- start start 2021-02-10T11:49:13.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.190 7f4d1b2df700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d140b7510 0x7f4d140b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:13.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.190 7f4d1b2df700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d1409d530 0x7f4d1409d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:13.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.190 7f4d1b2df700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d1409df50 0x7f4d1409eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:13.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.190 7f4d1b2df700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d140b8140 con 0x7f4d140b7510 2021-02-10T11:49:13.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.190 7f4d1b2df700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1409e750 con 0x7f4d1409df50 2021-02-10T11:49:13.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.190 7f4d1b2df700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1409e350 con 0x7f4d1409d530 2021-02-10T11:49:13.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.191 7f4d190fb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d140b7510 0x7f4d140b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:13.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.191 7f4d190fb700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d140b7510 0x7f4d140b7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35608/0 (socket says 172.21.2.101:35608) 2021-02-10T11:49:13.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.191 7f4d190fb700 1 -- 172.21.2.101:0/3264730855 learned_addr learned my addr 172.21.2.101:0/3264730855 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:13.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.191 7f4d188fa700 1 --2- 172.21.2.101:0/3264730855 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d1409d530 0x7f4d1409d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:13.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.191 7f4d190fb700 1 -- 172.21.2.101:0/3264730855 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d1409d530 msgr2=0x7f4d1409d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:13.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.191 7f4d190fb700 1 --2- 172.21.2.101:0/3264730855 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d1409d530 0x7f4d1409d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:13.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.191 7f4d190fb700 1 -- 172.21.2.101:0/3264730855 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d1409df50 msgr2=0x7f4d1409eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:13.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.191 7f4d190fb700 1 --2- 172.21.2.101:0/3264730855 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d1409df50 0x7f4d1409eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:13.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.191 7f4d190fb700 1 -- 172.21.2.101:0/3264730855 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1409f2c0 con 0x7f4d140b7510 2021-02-10T11:49:13.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.191 7f4d190fb700 1 --2- 172.21.2.101:0/3264730855 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d140b7510 0x7f4d140b7910 secure :-1 s=READY pgs=3349 cs=0 l=1 rev1=1 rx=0x7f4d0c00a870 tx=0x7f4d0c00ac10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:13.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.191 7f4d09ffb700 1 -- 172.21.2.101:0/3264730855 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d0c007d10 con 0x7f4d140b7510 2021-02-10T11:49:13.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.191 7f4d09ffb700 1 -- 172.21.2.101:0/3264730855 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c0070c0 con 0x7f4d140b7510 2021-02-10T11:49:13.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.191 7f4d09ffb700 1 -- 172.21.2.101:0/3264730855 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d0c007550 con 0x7f4d140b7510 2021-02-10T11:49:13.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.191 7f4d1b2df700 1 -- 172.21.2.101:0/3264730855 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1409f2c0 con 0x7f4d140b7510 2021-02-10T11:49:13.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.191 7f4d1b2df700 1 -- 172.21.2.101:0/3264730855 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1409f2c0 con 0x7f4d140b7510 2021-02-10T11:49:13.191 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.192 7f4d09ffb700 1 -- 172.21.2.101:0/3264730855 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f4d0c0072a0 con 0x7f4d140b7510 2021-02-10T11:49:13.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.192 7f4d09ffb700 1 --2- 172.21.2.101:0/3264730855 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cf40186c0 0x7f4cf401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:13.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.192 7f4d09ffb700 1 -- 172.21.2.101:0/3264730855 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 4839+0+0 (secure 0 0 0) 0x7f4d0c032860 con 0x7f4d140b7510 2021-02-10T11:49:13.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.192 7f4d188fa700 1 --2- 172.21.2.101:0/3264730855 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cf40186c0 0x7f4cf401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:13.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.193 7f4d188fa700 1 --2- 172.21.2.101:0/3264730855 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cf40186c0 0x7f4cf401abc0 crc :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:13.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.222 7f4d1b2df700 1 -- 172.21.2.101:0/3264730855 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d140ebc50 con 0x7f4d140b7510 2021-02-10T11:49:13.221 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.225 7f4d09ffb700 1 -- 172.21.2.101:0/3264730855 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4d0c000d30 con 0x7f4d140b7510 2021-02-10T11:49:13.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.363 7f4d1b2df700 1 -- 172.21.2.101:0/3264730855 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f4d140f0b10 con 0x7f4d140b7510 2021-02-10T11:49:13.689 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.693 7f4d09ffb700 1 -- 172.21.2.101:0/3264730855 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v195) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4d0c007120 con 0x7f4d140b7510 2021-02-10T11:49:13.723 INFO:teuthology.orchestra.run.gibba001.stderr: removed all blacklist entries 2021-02-10T11:49:13.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.729 7f4d1b2df700 1 -- 172.21.2.101:0/3264730855 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cf40186c0 msgr2=0x7f4cf401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:13.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.729 7f4d1b2df700 1 --2- 172.21.2.101:0/3264730855 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cf40186c0 0x7f4cf401abc0 crc :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:13.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.729 7f4d1b2df700 1 -- 172.21.2.101:0/3264730855 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d140b7510 msgr2=0x7f4d140b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:13.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.729 7f4d1b2df700 1 --2- 172.21.2.101:0/3264730855 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d140b7510 0x7f4d140b7910 secure :-1 s=READY pgs=3349 cs=0 l=1 rev1=1 rx=0x7f4d0c00a870 tx=0x7f4d0c00ac10).stop 2021-02-10T11:49:13.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.729 7f4d1b2df700 1 -- 172.21.2.101:0/3264730855 shutdown_connections 2021-02-10T11:49:13.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.729 7f4d1b2df700 1 --2- 172.21.2.101:0/3264730855 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f4cf40186c0 0x7f4cf401abc0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:13.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.729 7f4d1b2df700 1 --2- 172.21.2.101:0/3264730855 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f4d1409d530 0x7f4d1409d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:13.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.729 7f4d1b2df700 1 --2- 172.21.2.101:0/3264730855 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f4d1409df50 0x7f4d1409eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:13.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.729 7f4d1b2df700 1 --2- 172.21.2.101:0/3264730855 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4d140b7510 0x7f4d140b7910 unknown :-1 s=CLOSED pgs=3349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:13.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.729 7f4d1b2df700 1 -- 172.21.2.101:0/3264730855 >> 172.21.2.101:0/3264730855 conn(0x7f4d140b83b0 msgr2=0x7f4d140a6850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:13.726 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.729 7f4d1b2df700 1 -- 172.21.2.101:0/3264730855 shutdown_connections 2021-02-10T11:49:13.727 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:13.729 7f4d1b2df700 1 -- 172.21.2.101:0/3264730855 wait complete. 2021-02-10T11:49:13.734 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2021-02-10T11:49:13.734 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth ls --format=json-pretty 2021-02-10T11:49:14.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.150 7fbbb77e8700 1 Processor -- start 2021-02-10T11:49:14.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.151 7fbbb77e8700 1 -- start start 2021-02-10T11:49:14.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.151 7fbbb77e8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbbb00d1870 0x7fbbb01b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:14.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.151 7fbbb77e8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbbb00f0650 0x7fbbb00cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:14.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.151 7fbbb77e8700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb00f0b10 con 0x7fbbb01b0a10 2021-02-10T11:49:14.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.151 7fbbb77e8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb00fe230 con 0x7fbbb00f0650 2021-02-10T11:49:14.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.151 7fbbb77e8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb00e5ed0 con 0x7fbbb00d1870 2021-02-10T11:49:14.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.151 7fbbb5e05700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbbb00f0650 0x7fbbb00cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:14.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.151 7fbbb5e05700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fbbb00f0650 0x7fbbb00cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35616/0 (socket says 172.21.2.101:35616) 2021-02-10T11:49:14.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.151 7fbbb5e05700 1 -- 172.21.2.101:0/3050328672 learned_addr learned my addr 172.21.2.101:0/3050328672 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:14.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.151 7fbbb5604700 1 --2- 172.21.2.101:0/3050328672 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbbb00d1870 0x7fbbb01b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:14.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.152 7fbba7fff700 1 -- 172.21.2.101:0/3050328672 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3898955652 0 0) 0x7fbba0000ce0 con 0x7fbbb01b0a10 2021-02-10T11:49:14.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.152 7fbba7fff700 1 -- 172.21.2.101:0/3050328672 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb9c002fe0 con 0x7fbbb01b0a10 2021-02-10T11:49:14.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.152 7fbbb5e05700 1 -- 172.21.2.101:0/3050328672 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbbb00d1870 msgr2=0x7fbbb01b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:14.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.152 7fbbb5e05700 1 --2- 172.21.2.101:0/3050328672 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbbb00d1870 0x7fbbb01b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:14.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.152 7fbbb5e05700 1 -- 172.21.2.101:0/3050328672 >> v1:172.21.2.101:6789/0 conn(0x7fbbb01b0a10 legacy=0x7fbbb00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:14.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.152 7fbbb5e05700 1 -- 172.21.2.101:0/3050328672 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb00f20a0 con 0x7fbbb00f0650 2021-02-10T11:49:14.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.152 7fbbb5e05700 1 --2- 172.21.2.101:0/3050328672 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbbb00f0650 0x7fbbb00cfb20 secure :-1 s=READY pgs=3351 cs=0 l=1 rev1=1 rx=0x7fbba8006f00 tx=0x7fbba8009f10).ready entity=mon.0 client_cookie=84f8a3a32b9e0a0e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:14.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.152 7fbba7fff700 1 -- 172.21.2.101:0/3050328672 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbba800b3b0 con 0x7fbbb00f0650 2021-02-10T11:49:14.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.152 7fbba7fff700 1 -- 172.21.2.101:0/3050328672 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba800b630 con 0x7fbbb00f0650 2021-02-10T11:49:14.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.152 7fbba7fff700 1 -- 172.21.2.101:0/3050328672 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbba800bb50 con 0x7fbbb00f0650 2021-02-10T11:49:14.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.152 7fbbb77e8700 1 -- 172.21.2.101:0/3050328672 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbbb00f0650 msgr2=0x7fbbb00cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:14.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.152 7fbbb77e8700 1 --2- 172.21.2.101:0/3050328672 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbbb00f0650 0x7fbbb00cfb20 secure :-1 s=READY pgs=3351 cs=0 l=1 rev1=1 rx=0x7fbba8006f00 tx=0x7fbba8009f10).stop 2021-02-10T11:49:14.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.153 7fbbb77e8700 1 -- 172.21.2.101:0/3050328672 shutdown_connections 2021-02-10T11:49:14.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.153 7fbbb77e8700 1 --2- 172.21.2.101:0/3050328672 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbbb00f0650 0x7fbbb00cfb20 unknown :-1 s=CLOSED pgs=3351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:14.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.153 7fbbb77e8700 1 --2- 172.21.2.101:0/3050328672 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbbb00d1870 0x7fbbb01b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:14.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.153 7fbbb77e8700 1 -- 172.21.2.101:0/3050328672 >> 172.21.2.101:0/3050328672 conn(0x7fbbb00ce450 msgr2=0x7fbbb00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:14.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.153 7fbbb77e8700 1 -- 172.21.2.101:0/3050328672 shutdown_connections 2021-02-10T11:49:14.166 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.153 7fbbb77e8700 1 -- 172.21.2.101:0/3050328672 wait complete. 2021-02-10T11:49:14.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.168 7fbbb77e8700 1 Processor -- start 2021-02-10T11:49:14.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.168 7fbbb77e8700 1 -- start start 2021-02-10T11:49:14.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.168 7fbbb77e8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbbb01942b0 0x7fbbb00b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:14.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.168 7fbbb77e8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbbb0195070 0x7fbbb0195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:14.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.168 7fbbb77e8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbbb00bb8e0 0x7fbbb00bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:14.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.168 7fbbb77e8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb01946b0 con 0x7fbbb01942b0 2021-02-10T11:49:14.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.168 7fbbb77e8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb00b3b70 con 0x7fbbb0195070 2021-02-10T11:49:14.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.168 7fbbb77e8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb00b3d90 con 0x7fbbb00bb8e0 2021-02-10T11:49:14.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.168 7fbbb5604700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbbb01942b0 0x7fbbb00b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:14.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.168 7fbbb5604700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbbb01942b0 0x7fbbb00b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35624/0 (socket says 172.21.2.101:35624) 2021-02-10T11:49:14.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.168 7fbbb5604700 1 -- 172.21.2.101:0/3689528468 learned_addr learned my addr 172.21.2.101:0/3689528468 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:14.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.169 7fbbb5e05700 1 --2- 172.21.2.101:0/3689528468 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbbb00bb8e0 0x7fbbb00bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:14.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.169 7fbbb4e03700 1 --2- 172.21.2.101:0/3689528468 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbbb0195070 0x7fbbb0195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:14.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.169 7fbbb5604700 1 -- 172.21.2.101:0/3689528468 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbbb00bb8e0 msgr2=0x7fbbb00bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:14.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.169 7fbbb5604700 1 --2- 172.21.2.101:0/3689528468 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbbb00bb8e0 0x7fbbb00bbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:14.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.169 7fbbb5604700 1 -- 172.21.2.101:0/3689528468 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbbb0195070 msgr2=0x7fbbb0195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:14.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.169 7fbbb5604700 1 --2- 172.21.2.101:0/3689528468 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbbb0195070 0x7fbbb0195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:14.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.169 7fbbb5604700 1 -- 172.21.2.101:0/3689528468 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb01a49e0 con 0x7fbbb01942b0 2021-02-10T11:49:14.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.169 7fbbb5604700 1 --2- 172.21.2.101:0/3689528468 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbbb01942b0 0x7fbbb00b3550 secure :-1 s=READY pgs=3352 cs=0 l=1 rev1=1 rx=0x7fbbac006f60 tx=0x7fbbac00b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:14.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.169 7fbba5ffb700 1 -- 172.21.2.101:0/3689528468 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbbac00bd60 con 0x7fbbb01942b0 2021-02-10T11:49:14.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.169 7fbba5ffb700 1 -- 172.21.2.101:0/3689528468 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbac000c90 con 0x7fbbb01942b0 2021-02-10T11:49:14.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.169 7fbbb77e8700 1 -- 172.21.2.101:0/3689528468 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb01a49e0 con 0x7fbbb01942b0 2021-02-10T11:49:14.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.169 7fbbb77e8700 1 -- 172.21.2.101:0/3689528468 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb00bc5f0 con 0x7fbbb01942b0 2021-02-10T11:49:14.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.169 7fbba5ffb700 1 -- 172.21.2.101:0/3689528468 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbbac0022b0 con 0x7fbbb01942b0 2021-02-10T11:49:14.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.170 7fbba5ffb700 1 -- 172.21.2.101:0/3689528468 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fbbac006b00 con 0x7fbbb01942b0 2021-02-10T11:49:14.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.170 7fbba5ffb700 1 --2- 172.21.2.101:0/3689528468 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbb900186c0 0x7fbb9001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:14.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.170 7fbba5ffb700 1 -- 172.21.2.101:0/3689528468 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fbbac032440 con 0x7fbbb01942b0 2021-02-10T11:49:14.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.170 7fbbb4e03700 1 --2- 172.21.2.101:0/3689528468 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbb900186c0 0x7fbb9001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:14.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.170 7fbbb4e03700 1 --2- 172.21.2.101:0/3689528468 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbb900186c0 0x7fbb9001abc0 crc :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:14.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.201 7fbbb77e8700 1 -- 172.21.2.101:0/3689528468 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb00bc5f0 con 0x7fbbb01942b0 2021-02-10T11:49:14.199 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.203 7fbba5ffb700 1 -- 172.21.2.101:0/3689528468 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fbbac000960 con 0x7fbbb01942b0 2021-02-10T11:49:14.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.337 7fbbb77e8700 1 -- 172.21.2.101:0/3689528468 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbbb00bc5f0 con 0x7fbbb01942b0 2021-02-10T11:49:14.333 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.337 7fbba5ffb700 1 -- 172.21.2.101:0/3689528468 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v119) v1 ==== 80+0+4513 (secure 0 0 0) 0x7fbbac000c90 con 0x7fbbb01942b0 2021-02-10T11:49:14.334 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:49:14.334 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:49:14.334 INFO:teuthology.orchestra.run.gibba001.stdout: "auth_dump": [ 2021-02-10T11:49:14.334 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:14.335 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.a", 2021-02-10T11:49:14.335 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgSWKpBxAALwB7Nyc2NSvuygYiFUTWVA==", 2021-02-10T11:49:14.335 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:49:14.335 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:49:14.335 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:49:14.335 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:49:14.335 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:49:14.335 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:14.335 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:14.335 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:14.335 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mds.b", 2021-02-10T11:49:14.335 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBYvyNgh8HnMhAAW0BAIqcVgJT/vUKKfHUDwQ==", 2021-02-10T11:49:14.335 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:49:14.336 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:49:14.336 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:49:14.336 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:49:14.336 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:49:14.336 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:14.336 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:14.336 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:14.336 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.0", 2021-02-10T11:49:14.336 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBgvyNgn10nMxAAneZE+uTWuLYHdCy1seyoKw==", 2021-02-10T11:49:14.336 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:49:14.336 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:49:14.336 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:49:14.336 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:49:14.337 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:14.337 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:14.337 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:14.337 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.1", 2021-02-10T11:49:14.337 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBjvyNg7adyNBAA7D8d07Yr3mJP8YI9M4BayQ==", 2021-02-10T11:49:14.337 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:49:14.337 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:49:14.337 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:49:14.337 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:49:14.337 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:14.337 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:14.337 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:14.338 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.2", 2021-02-10T11:49:14.338 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBmvyNg4bfHNBAAT3O4ogRGBCjR4BodVGkQ8w==", 2021-02-10T11:49:14.338 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:49:14.338 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:49:14.338 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:49:14.338 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:49:14.338 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:14.342 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:14.342 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:14.342 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.3", 2021-02-10T11:49:14.342 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBpvyNgikAfNRAA94p9sp6GYSAzQEv61qYyAA==", 2021-02-10T11:49:14.342 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:49:14.343 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:49:14.343 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:49:14.343 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:49:14.343 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:14.343 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:14.343 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:14.343 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.4", 2021-02-10T11:49:14.343 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQByvyNgrEurJxAA30KCD37Z47q6C0R08zTOjA==", 2021-02-10T11:49:14.343 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:49:14.343 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:49:14.343 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:49:14.343 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:49:14.343 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:14.344 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:14.344 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:14.344 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.5", 2021-02-10T11:49:14.344 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB1vyNgRkWdKBAAqTlXTTBl3sN3Gejh2dZYqw==", 2021-02-10T11:49:14.344 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:49:14.344 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:49:14.344 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:49:14.344 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:49:14.344 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:14.344 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:14.344 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:14.344 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.6", 2021-02-10T11:49:14.344 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB4vyNgrunIKBAAqhSksRC3V5WMYwh1gWi/fQ==", 2021-02-10T11:49:14.345 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:49:14.345 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:49:14.345 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:49:14.345 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:49:14.345 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:14.345 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:14.345 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:14.345 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "osd.7", 2021-02-10T11:49:14.345 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQB7vyNgQysZKRAAlUgifbAiV+mVimPswmBZLw==", 2021-02-10T11:49:14.345 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:49:14.345 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:49:14.345 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:49:14.345 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:49:14.346 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:14.346 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:14.346 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:14.346 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.0", 2021-02-10T11:49:14.346 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg4YmIIRAAA3nF1Ssq1gSxIbUVyDe0Mg==", 2021-02-10T11:49:14.346 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:49:14.346 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:49:14.346 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:49:14.346 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:49:14.346 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:14.346 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:14.346 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:14.346 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.1", 2021-02-10T11:49:14.347 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBZvyNg6u0ROBAAy+/rBevzXGqHLi9O4V4Xfg==", 2021-02-10T11:49:14.347 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:49:14.347 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:49:14.347 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:49:14.347 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:49:14.347 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:14.347 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:14.347 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:14.347 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.2", 2021-02-10T11:49:14.347 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgE8ZlExAAIJ75ETgGB//sti9mQ51eUA==", 2021-02-10T11:49:14.347 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:49:14.347 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:49:14.347 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:49:14.348 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:49:14.348 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:14.348 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:14.348 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:14.348 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.3", 2021-02-10T11:49:14.348 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBavyNgzSD1KRAA67j21LNgArozF4f1YtCytg==", 2021-02-10T11:49:14.348 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:49:14.348 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow", 2021-02-10T11:49:14.348 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow r", 2021-02-10T11:49:14.348 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow rw pool=cephfs_data" 2021-02-10T11:49:14.348 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:14.348 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:14.348 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:14.349 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "client.admin", 2021-02-10T11:49:14.349 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBUvyNgCx3sDRAAdsVt91JRqngqEQL69h0JzQ==", 2021-02-10T11:49:14.349 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:49:14.349 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:49:14.349 INFO:teuthology.orchestra.run.gibba001.stdout: "mgr": "allow *", 2021-02-10T11:49:14.349 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow *", 2021-02-10T11:49:14.349 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:49:14.349 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:14.349 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:14.349 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:14.349 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.x", 2021-02-10T11:49:14.349 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNgmrXlLBAA6R2grTfADWkVdY7I5yILmA==", 2021-02-10T11:49:14.349 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:49:14.350 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:49:14.350 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:49:14.350 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:49:14.350 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:14.350 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:14.350 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:14.350 INFO:teuthology.orchestra.run.gibba001.stdout: "entity": "mgr.y", 2021-02-10T11:49:14.350 INFO:teuthology.orchestra.run.gibba001.stdout: "key": "AQBXvyNglglqFBAAIOusEjIwm7ZGfBS2SaWpmg==", 2021-02-10T11:49:14.350 INFO:teuthology.orchestra.run.gibba001.stdout: "caps": { 2021-02-10T11:49:14.350 INFO:teuthology.orchestra.run.gibba001.stdout: "mds": "allow *", 2021-02-10T11:49:14.350 INFO:teuthology.orchestra.run.gibba001.stdout: "mon": "allow profile mgr", 2021-02-10T11:49:14.350 INFO:teuthology.orchestra.run.gibba001.stdout: "osd": "allow *" 2021-02-10T11:49:14.350 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:14.351 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:14.351 INFO:teuthology.orchestra.run.gibba001.stdout: ] 2021-02-10T11:49:14.351 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:49:14.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.339 7fbbb77e8700 1 -- 172.21.2.101:0/3689528468 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbb900186c0 msgr2=0x7fbb9001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:14.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.339 7fbbb77e8700 1 --2- 172.21.2.101:0/3689528468 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbb900186c0 0x7fbb9001abc0 crc :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:14.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.339 7fbbb77e8700 1 -- 172.21.2.101:0/3689528468 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbbb01942b0 msgr2=0x7fbbb00b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:14.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.339 7fbbb77e8700 1 --2- 172.21.2.101:0/3689528468 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbbb01942b0 0x7fbbb00b3550 secure :-1 s=READY pgs=3352 cs=0 l=1 rev1=1 rx=0x7fbbac006f60 tx=0x7fbbac00b970).stop 2021-02-10T11:49:14.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.339 7fbbb77e8700 1 -- 172.21.2.101:0/3689528468 shutdown_connections 2021-02-10T11:49:14.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.339 7fbbb77e8700 1 --2- 172.21.2.101:0/3689528468 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fbb900186c0 0x7fbb9001abc0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:14.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.339 7fbbb77e8700 1 --2- 172.21.2.101:0/3689528468 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fbbb00bb8e0 0x7fbbb00bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:14.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.339 7fbbb77e8700 1 --2- 172.21.2.101:0/3689528468 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fbbb0195070 0x7fbbb0195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:14.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.339 7fbbb77e8700 1 --2- 172.21.2.101:0/3689528468 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fbbb01942b0 0x7fbbb00b3550 unknown :-1 s=CLOSED pgs=3352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:14.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.339 7fbbb77e8700 1 -- 172.21.2.101:0/3689528468 >> 172.21.2.101:0/3689528468 conn(0x7fbbb00b3040 msgr2=0x7fbbb00c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:14.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.340 7fbbb77e8700 1 -- 172.21.2.101:0/3689528468 shutdown_connections 2021-02-10T11:49:14.353 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.340 7fbbb77e8700 1 -- 172.21.2.101:0/3689528468 wait complete. 2021-02-10T11:49:14.353 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:49:14.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.774 7f9d92b8f700 1 Processor -- start 2021-02-10T11:49:14.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.774 7f9d92b8f700 1 -- start start 2021-02-10T11:49:14.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.774 7f9d92b8f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d8c0d1870 0x7f9d8c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:14.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.774 7f9d92b8f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9d8c0f0650 0x7f9d8c0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:14.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.774 7f9d92b8f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d8c0f0b10 con 0x7f9d8c1b0a10 2021-02-10T11:49:14.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.774 7f9d92b8f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d8c0fe230 con 0x7f9d8c0f0650 2021-02-10T11:49:14.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.774 7f9d92b8f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c0e5ed0 con 0x7f9d8c0d1870 2021-02-10T11:49:14.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.774 7f9d911ac700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9d8c0f0650 0x7f9d8c0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:14.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.774 7f9d911ac700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9d8c0f0650 0x7f9d8c0cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35630/0 (socket says 172.21.2.101:35630) 2021-02-10T11:49:14.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.774 7f9d911ac700 1 -- 172.21.2.101:0/4275352455 learned_addr learned my addr 172.21.2.101:0/4275352455 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:14.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.774 7f9d909ab700 1 --2- 172.21.2.101:0/4275352455 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d8c0d1870 0x7f9d8c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:14.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.774 7f9d911ac700 1 -- 172.21.2.101:0/4275352455 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d8c0d1870 msgr2=0x7f9d8c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:14.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.774 7f9d911ac700 1 --2- 172.21.2.101:0/4275352455 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d8c0d1870 0x7f9d8c1b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:14.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.774 7f9d911ac700 1 -- 172.21.2.101:0/4275352455 >> v1:172.21.2.108:6789/0 conn(0x7f9d8c1b0a10 legacy=0x7f9d8c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:14.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.775 7f9d911ac700 1 -- 172.21.2.101:0/4275352455 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8c0f20a0 con 0x7f9d8c0f0650 2021-02-10T11:49:14.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.775 7f9d909ab700 1 --2- 172.21.2.101:0/4275352455 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d8c0d1870 0x7f9d8c1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:49:14.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.775 7f9d911ac700 1 --2- 172.21.2.101:0/4275352455 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d8c0f0650 0x7f9d8c0cfb20 secure :-1 s=READY pgs=3353 cs=0 l=1 rev1=1 rx=0x7f9d84006b80 tx=0x7f9d84009b90).ready entity=mon.0 client_cookie=9b48af8b94d9cd4c server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:14.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.775 7f9d8b7fe700 1 -- 172.21.2.101:0/4275352455 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d840022e0 con 0x7f9d8c0f0650 2021-02-10T11:49:14.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.775 7f9d8b7fe700 1 -- 172.21.2.101:0/4275352455 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d84002620 con 0x7f9d8c0f0650 2021-02-10T11:49:14.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.775 7f9d8b7fe700 1 -- 172.21.2.101:0/4275352455 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d84002b00 con 0x7f9d8c0f0650 2021-02-10T11:49:14.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.775 7f9d92b8f700 1 -- 172.21.2.101:0/4275352455 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d8c0f0650 msgr2=0x7f9d8c0cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:14.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.775 7f9d92b8f700 1 --2- 172.21.2.101:0/4275352455 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d8c0f0650 0x7f9d8c0cfb20 secure :-1 s=READY pgs=3353 cs=0 l=1 rev1=1 rx=0x7f9d84006b80 tx=0x7f9d84009b90).stop 2021-02-10T11:49:14.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.775 7f9d92b8f700 1 -- 172.21.2.101:0/4275352455 shutdown_connections 2021-02-10T11:49:14.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.776 7f9d92b8f700 1 --2- 172.21.2.101:0/4275352455 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d8c0f0650 0x7f9d8c0cfb20 unknown :-1 s=CLOSED pgs=3353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:14.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.776 7f9d92b8f700 1 --2- 172.21.2.101:0/4275352455 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d8c0d1870 0x7f9d8c1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:14.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.776 7f9d92b8f700 1 -- 172.21.2.101:0/4275352455 >> 172.21.2.101:0/4275352455 conn(0x7f9d8c0ce450 msgr2=0x7f9d8c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:14.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.776 7f9d92b8f700 1 -- 172.21.2.101:0/4275352455 shutdown_connections 2021-02-10T11:49:14.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.776 7f9d92b8f700 1 -- 172.21.2.101:0/4275352455 wait complete. 2021-02-10T11:49:14.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.790 7f9d92b8f700 1 Processor -- start 2021-02-10T11:49:14.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.790 7f9d92b8f700 1 -- start start 2021-02-10T11:49:14.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.791 7f9d92b8f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d8c1971e0 0x7f9d8c18eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:14.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.791 7f9d92b8f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d8c1b8180 0x7f9d8c1b8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:14.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.791 7f9d92b8f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d8c1b8ba0 0x7f9d8c0b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:14.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.791 7f9d92b8f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c1b9750 con 0x7f9d8c1971e0 2021-02-10T11:49:14.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.791 7f9d92b8f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c18e930 con 0x7f9d8c1b8180 2021-02-10T11:49:14.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.791 7f9d92b8f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c0b1dd0 con 0x7f9d8c1b8ba0 2021-02-10T11:49:14.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.791 7f9d909ab700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d8c1971e0 0x7f9d8c18eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:14.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.791 7f9d909ab700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d8c1971e0 0x7f9d8c18eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35636/0 (socket says 172.21.2.101:35636) 2021-02-10T11:49:14.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.791 7f9d909ab700 1 -- 172.21.2.101:0/2604379527 learned_addr learned my addr 172.21.2.101:0/2604379527 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:14.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.791 7f9d909ab700 1 -- 172.21.2.101:0/2604379527 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d8c1b8ba0 msgr2=0x7f9d8c0b17b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:49:14.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.791 7f9d909ab700 1 --2- 172.21.2.101:0/2604379527 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d8c1b8ba0 0x7f9d8c0b17b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:14.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.791 7f9d909ab700 1 -- 172.21.2.101:0/2604379527 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d8c1b8180 msgr2=0x7f9d8c1b8580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:49:14.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.791 7f9d909ab700 1 --2- 172.21.2.101:0/2604379527 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d8c1b8180 0x7f9d8c1b8580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:14.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.791 7f9d909ab700 1 -- 172.21.2.101:0/2604379527 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8c0b1ff0 con 0x7f9d8c1971e0 2021-02-10T11:49:14.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.792 7f9d909ab700 1 --2- 172.21.2.101:0/2604379527 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d8c1971e0 0x7f9d8c18eda0 secure :-1 s=READY pgs=3354 cs=0 l=1 rev1=1 rx=0x7f9d74006f20 tx=0x7f9d7400b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:14.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.792 7f9d897fa700 1 -- 172.21.2.101:0/2604379527 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d7400bd80 con 0x7f9d8c1971e0 2021-02-10T11:49:14.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.792 7f9d897fa700 1 -- 172.21.2.101:0/2604379527 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d74000c90 con 0x7f9d8c1971e0 2021-02-10T11:49:14.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.792 7f9d92b8f700 1 -- 172.21.2.101:0/2604379527 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d8c0b1ff0 con 0x7f9d8c1971e0 2021-02-10T11:49:14.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.792 7f9d92b8f700 1 -- 172.21.2.101:0/2604379527 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d8c0b2580 con 0x7f9d8c1971e0 2021-02-10T11:49:14.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.792 7f9d897fa700 1 -- 172.21.2.101:0/2604379527 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d740022b0 con 0x7f9d8c1971e0 2021-02-10T11:49:14.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.793 7f9d897fa700 1 -- 172.21.2.101:0/2604379527 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9d7400a000 con 0x7f9d8c1971e0 2021-02-10T11:49:14.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.793 7f9d897fa700 1 --2- 172.21.2.101:0/2604379527 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d700186c0 0x7f9d7001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:14.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.793 7f9d897fa700 1 -- 172.21.2.101:0/2604379527 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f9d74032210 con 0x7f9d8c1971e0 2021-02-10T11:49:14.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.793 7f9d8bfff700 1 --2- 172.21.2.101:0/2604379527 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d700186c0 0x7f9d7001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:14.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.793 7f9d8bfff700 1 --2- 172.21.2.101:0/2604379527 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d700186c0 0x7f9d7001abc0 crc :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:14.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.823 7f9d92b8f700 1 -- 172.21.2.101:0/2604379527 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d8c0ee060 con 0x7f9d8c1971e0 2021-02-10T11:49:14.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.826 7f9d897fa700 1 -- 172.21.2.101:0/2604379527 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9d74000960 con 0x7f9d8c1971e0 2021-02-10T11:49:14.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.951 7f9d92b8f700 1 -- 172.21.2.101:0/2604379527 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d8c0ce450 con 0x7f9d8c1971e0 2021-02-10T11:49:14.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.952 7f9d897fa700 1 -- 172.21.2.101:0/2604379527 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v195) v1 ==== 74+0+11154 (secure 0 0 0) 0x7f9d7400a000 con 0x7f9d8c1971e0 2021-02-10T11:49:14.948 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:49:14.949 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":195,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:49:13.690545","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":41,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:49:14.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.953 7f9d92b8f700 1 -- 172.21.2.101:0/2604379527 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d700186c0 msgr2=0x7f9d7001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:14.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.953 7f9d92b8f700 1 --2- 172.21.2.101:0/2604379527 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d700186c0 0x7f9d7001abc0 crc :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:14.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.953 7f9d92b8f700 1 -- 172.21.2.101:0/2604379527 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d8c1971e0 msgr2=0x7f9d8c18eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:14.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.953 7f9d92b8f700 1 --2- 172.21.2.101:0/2604379527 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d8c1971e0 0x7f9d8c18eda0 secure :-1 s=READY pgs=3354 cs=0 l=1 rev1=1 rx=0x7f9d74006f20 tx=0x7f9d7400b990).stop 2021-02-10T11:49:14.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.954 7f9d92b8f700 1 -- 172.21.2.101:0/2604379527 shutdown_connections 2021-02-10T11:49:14.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.954 7f9d92b8f700 1 --2- 172.21.2.101:0/2604379527 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9d700186c0 0x7f9d7001abc0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:14.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.954 7f9d92b8f700 1 --2- 172.21.2.101:0/2604379527 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9d8c1b8ba0 0x7f9d8c0b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:14.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.954 7f9d92b8f700 1 --2- 172.21.2.101:0/2604379527 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9d8c1b8180 0x7f9d8c1b8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:14.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.954 7f9d92b8f700 1 --2- 172.21.2.101:0/2604379527 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9d8c1971e0 0x7f9d8c18eda0 unknown :-1 s=CLOSED pgs=3354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:14.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.954 7f9d92b8f700 1 -- 172.21.2.101:0/2604379527 >> 172.21.2.101:0/2604379527 conn(0x7f9d8c19f8f0 msgr2=0x7f9d8c18c260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:14.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.954 7f9d92b8f700 1 -- 172.21.2.101:0/2604379527 shutdown_connections 2021-02-10T11:49:14.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:14.954 7f9d92b8f700 1 -- 172.21.2.101:0/2604379527 wait complete. 2021-02-10T11:49:14.958 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-10T11:49:14.959 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:49:15.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.380 7f860e604700 1 Processor -- start 2021-02-10T11:49:15.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.381 7f860e604700 1 -- start start 2021-02-10T11:49:15.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.381 7f860e604700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f86080d1870 0x7f86081b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:15.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.381 7f860e604700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86081b0ae0 0x7f8608108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:15.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.381 7f860e604700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86080fe1c0 con 0x7f86080d1c70 2021-02-10T11:49:15.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.381 7f860e604700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8608109a60 con 0x7f86080d1870 2021-02-10T11:49:15.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.381 7f860e604700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8608109750 con 0x7f86081b0ae0 2021-02-10T11:49:15.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.381 7f8607fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f86080d1870 0x7f86081b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:15.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.381 7f8607fff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f86080d1870 0x7f86081b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35644/0 (socket says 172.21.2.101:35644) 2021-02-10T11:49:15.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.381 7f8607fff700 1 -- 172.21.2.101:0/1042593737 learned_addr learned my addr 172.21.2.101:0/1042593737 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:15.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.382 7f8607fff700 1 -- 172.21.2.101:0/1042593737 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86081b0ae0 msgr2=0x7f8608108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:15.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.382 7f8607fff700 1 --2- 172.21.2.101:0/1042593737 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86081b0ae0 0x7f8608108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:15.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.382 7f8607fff700 1 -- 172.21.2.101:0/1042593737 >> v1:172.21.2.108:6789/0 conn(0x7f86080d1c70 legacy=0x7f8608108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:15.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.382 7f8607fff700 1 -- 172.21.2.101:0/1042593737 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8608088070 con 0x7f86080d1870 2021-02-10T11:49:15.394 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.382 7f8607fff700 1 --2- 172.21.2.101:0/1042593737 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86080d1870 0x7f86081b04c0 secure :-1 s=READY pgs=3355 cs=0 l=1 rev1=1 rx=0x7f85f0006700 tx=0x7f85f0009b80).ready entity=mon.0 client_cookie=7e9fa8264d3418c2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:15.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.382 7f8606ffd700 1 -- 172.21.2.101:0/1042593737 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f85f000c0c0 con 0x7f86080d1870 2021-02-10T11:49:15.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.382 7f8606ffd700 1 -- 172.21.2.101:0/1042593737 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f000c340 con 0x7f86080d1870 2021-02-10T11:49:15.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.382 7f8606ffd700 1 -- 172.21.2.101:0/1042593737 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f85f000c860 con 0x7f86080d1870 2021-02-10T11:49:15.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.382 7f860e604700 1 -- 172.21.2.101:0/1042593737 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86080d1870 msgr2=0x7f86081b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:15.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.382 7f860e604700 1 --2- 172.21.2.101:0/1042593737 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86080d1870 0x7f86081b04c0 secure :-1 s=READY pgs=3355 cs=0 l=1 rev1=1 rx=0x7f85f0006700 tx=0x7f85f0009b80).stop 2021-02-10T11:49:15.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.383 7f860e604700 1 -- 172.21.2.101:0/1042593737 shutdown_connections 2021-02-10T11:49:15.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.383 7f860e604700 1 --2- 172.21.2.101:0/1042593737 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86081b0ae0 0x7f8608108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:15.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.383 7f860e604700 1 --2- 172.21.2.101:0/1042593737 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86080d1870 0x7f86081b04c0 unknown :-1 s=CLOSED pgs=3355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:15.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.383 7f860e604700 1 -- 172.21.2.101:0/1042593737 >> 172.21.2.101:0/1042593737 conn(0x7f86080ce420 msgr2=0x7f86080d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:15.395 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.383 7f860e604700 1 -- 172.21.2.101:0/1042593737 shutdown_connections 2021-02-10T11:49:15.396 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.383 7f860e604700 1 -- 172.21.2.101:0/1042593737 wait complete. 2021-02-10T11:49:15.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.398 7f860e604700 1 Processor -- start 2021-02-10T11:49:15.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.398 7f860e604700 1 -- start start 2021-02-10T11:49:15.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.398 7f860e604700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86080c3820 0x7f8608084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:15.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.398 7f860e604700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86080a0290 0x7f86080852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:15.397 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.398 7f860e604700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8608086270 0x7f8608086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:15.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.398 7f860e604700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86080c0700 con 0x7f8608086270 2021-02-10T11:49:15.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.398 7f860e604700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86080858f0 con 0x7f86080c3820 2021-02-10T11:49:15.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.398 7f860e604700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8608085b10 con 0x7f86080a0290 2021-02-10T11:49:15.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.399 7f860cc21700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8608086270 0x7f8608086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:15.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.399 7f860cc21700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8608086270 0x7f8608086670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35648/0 (socket says 172.21.2.101:35648) 2021-02-10T11:49:15.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.399 7f860cc21700 1 -- 172.21.2.101:0/2213576337 learned_addr learned my addr 172.21.2.101:0/2213576337 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:15.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.399 7f860cc21700 1 -- 172.21.2.101:0/2213576337 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86080a0290 msgr2=0x7f86080852d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:49:15.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.399 7f860cc21700 1 --2- 172.21.2.101:0/2213576337 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86080a0290 0x7f86080852d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:15.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.399 7f860cc21700 1 -- 172.21.2.101:0/2213576337 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86080c3820 msgr2=0x7f8608084cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:15.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.399 7f860cc21700 1 --2- 172.21.2.101:0/2213576337 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86080c3820 0x7f8608084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:15.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.399 7f860cc21700 1 -- 172.21.2.101:0/2213576337 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8608086c90 con 0x7f8608086270 2021-02-10T11:49:15.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.399 7f860cc21700 1 --2- 172.21.2.101:0/2213576337 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8608086270 0x7f8608086670 secure :-1 s=READY pgs=3356 cs=0 l=1 rev1=1 rx=0x7f860000b8e0 tx=0x7f860000bc80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:15.398 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.399 7f8604ff9700 1 -- 172.21.2.101:0/2213576337 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8600000f90 con 0x7f8608086270 2021-02-10T11:49:15.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.399 7f8604ff9700 1 -- 172.21.2.101:0/2213576337 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860000c2f0 con 0x7f8608086270 2021-02-10T11:49:15.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.399 7f860e604700 1 -- 172.21.2.101:0/2213576337 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8608086c90 con 0x7f8608086270 2021-02-10T11:49:15.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.399 7f8604ff9700 1 -- 172.21.2.101:0/2213576337 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f860000c750 con 0x7f8608086270 2021-02-10T11:49:15.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.399 7f860e604700 1 -- 172.21.2.101:0/2213576337 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8608086c90 con 0x7f8608086270 2021-02-10T11:49:15.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.400 7f8604ff9700 1 -- 172.21.2.101:0/2213576337 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8600001220 con 0x7f8608086270 2021-02-10T11:49:15.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.400 7f8604ff9700 1 --2- 172.21.2.101:0/2213576337 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f85ec018700 0x7f85ec01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:15.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.400 7f8604ff9700 1 -- 172.21.2.101:0/2213576337 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4753+0+0 (secure 0 0 0) 0x7f86000326d0 con 0x7f8608086270 2021-02-10T11:49:15.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.400 7f8607fff700 1 --2- 172.21.2.101:0/2213576337 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f85ec018700 0x7f85ec01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:15.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.400 7f8607fff700 1 --2- 172.21.2.101:0/2213576337 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f85ec018700 0x7f85ec01abc0 crc :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:15.427 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.431 7f860e604700 1 -- 172.21.2.101:0/2213576337 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8608082a70 con 0x7f8608086270 2021-02-10T11:49:15.429 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.433 7f8604ff9700 1 -- 172.21.2.101:0/2213576337 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f860000bdc0 con 0x7f8608086270 2021-02-10T11:49:15.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.560 7f860e604700 1 -- 172.21.2.101:0/2213576337 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f8608082a70 con 0x7f8608086270 2021-02-10T11:49:15.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.560 7f8604ff9700 1 -- 172.21.2.101:0/2213576337 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v195) v1 ==== 92+0+10 (secure 0 0 0) 0x7f860001f8c0 con 0x7f8608086270 2021-02-10T11:49:15.557 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:49:15.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.562 7f860e604700 1 -- 172.21.2.101:0/2213576337 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f85ec018700 msgr2=0x7f85ec01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:15.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.562 7f860e604700 1 --2- 172.21.2.101:0/2213576337 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f85ec018700 0x7f85ec01abc0 crc :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:15.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.562 7f860e604700 1 -- 172.21.2.101:0/2213576337 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8608086270 msgr2=0x7f8608086670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:15.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.562 7f860e604700 1 --2- 172.21.2.101:0/2213576337 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8608086270 0x7f8608086670 secure :-1 s=READY pgs=3356 cs=0 l=1 rev1=1 rx=0x7f860000b8e0 tx=0x7f860000bc80).stop 2021-02-10T11:49:15.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.562 7f860e604700 1 -- 172.21.2.101:0/2213576337 shutdown_connections 2021-02-10T11:49:15.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.562 7f860e604700 1 --2- 172.21.2.101:0/2213576337 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f85ec018700 0x7f85ec01abc0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:15.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.562 7f860e604700 1 --2- 172.21.2.101:0/2213576337 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86080a0290 0x7f86080852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:15.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.562 7f860e604700 1 --2- 172.21.2.101:0/2213576337 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8608086270 0x7f8608086670 unknown :-1 s=CLOSED pgs=3356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:15.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.562 7f860e604700 1 --2- 172.21.2.101:0/2213576337 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86080c3820 0x7f8608084cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:15.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.562 7f860e604700 1 -- 172.21.2.101:0/2213576337 >> 172.21.2.101:0/2213576337 conn(0x7f86080872c0 msgr2=0x7f8608095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:15.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.563 7f860e604700 1 -- 172.21.2.101:0/2213576337 shutdown_connections 2021-02-10T11:49:15.559 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.563 7f860e604700 1 -- 172.21.2.101:0/2213576337 wait complete. 2021-02-10T11:49:15.567 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --format=json-pretty osd lspools 2021-02-10T11:49:15.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.984 7efc8be98700 1 Processor -- start 2021-02-10T11:49:15.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.984 7efc8be98700 1 -- start start 2021-02-10T11:49:15.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.985 7efc8be98700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efc840d0340 0x7efc840ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:15.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.985 7efc8be98700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc840f0b10 con 0x7efc840d1870 2021-02-10T11:49:15.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.985 7efc8be98700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc841b0be0 con 0x7efc840d1c70 2021-02-10T11:49:15.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.985 7efc8be98700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc840e5ed0 con 0x7efc840d0340 2021-02-10T11:49:15.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.985 7efc89cb4700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7efc840d1870 0x7efc841b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36148/0 (socket says 172.21.2.101:36148) 2021-02-10T11:49:15.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.985 7efc89cb4700 1 -- 172.21.2.101:0/3831275533 learned_addr learned my addr 172.21.2.101:0/3831275533 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:15.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.985 7efc894b3700 1 --2- 172.21.2.101:0/3831275533 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efc840d0340 0x7efc840ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:15.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.985 7efc88cb2700 1 -- 172.21.2.101:0/3831275533 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 166803977 0 0) 0x7efc80000ce0 con 0x7efc840d1870 2021-02-10T11:49:15.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.985 7efc88cb2700 1 -- 172.21.2.101:0/3831275533 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc70002fe0 con 0x7efc840d1870 2021-02-10T11:49:15.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.986 7efc88cb2700 1 -- 172.21.2.101:0/3831275533 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 694124997 0 0) 0x7efc800010e0 con 0x7efc840d1870 2021-02-10T11:49:15.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.986 7efc88cb2700 1 -- 172.21.2.101:0/3831275533 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efc840d0340 msgr2=0x7efc840ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:15.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.986 7efc88cb2700 1 --2- 172.21.2.101:0/3831275533 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efc840d0340 0x7efc840ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:15.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.986 7efc88cb2700 1 -- 172.21.2.101:0/3831275533 >> v1:172.21.2.108:6789/0 conn(0x7efc840d1c70 legacy=0x7efc840efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:15.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.986 7efc88cb2700 1 -- 172.21.2.101:0/3831275533 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc840f0450 con 0x7efc840d1870 2021-02-10T11:49:15.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.986 7efc88cb2700 1 -- 172.21.2.101:0/3831275533 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7efc80001660 con 0x7efc840d1870 2021-02-10T11:49:15.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.986 7efc88cb2700 1 -- 172.21.2.101:0/3831275533 >> v1:172.21.2.101:6789/0 conn(0x7efc840d1870 legacy=0x7efc841b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:15.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.986 7efc88cb2700 1 --2- 172.21.2.101:0/3831275533 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc70003b10 0x7efc70005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:15.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.986 7efc88cb2700 1 --2- 172.21.2.101:0/3831275533 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efc70006790 0x7efc70008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:15.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.986 7efc88cb2700 1 --2- 172.21.2.101:0/3831275533 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efc70009500 0x7efc7000ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:15.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.986 7efc88cb2700 1 -- 172.21.2.101:0/3831275533 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7000c2d0 con 0x7efc70003b10 2021-02-10T11:49:16.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.986 7efc88cb2700 1 -- 172.21.2.101:0/3831275533 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7000c4f0 con 0x7efc70009500 2021-02-10T11:49:16.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.986 7efc88cb2700 1 -- 172.21.2.101:0/3831275533 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc7000c710 con 0x7efc70006790 2021-02-10T11:49:16.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.986 7efc89cb4700 1 --2- 172.21.2.101:0/3831275533 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc70003b10 0x7efc70005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:16.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.986 7efc8a4b5700 1 --2- 172.21.2.101:0/3831275533 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efc70009500 0x7efc7000ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:16.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.986 7efc894b3700 1 --2- 172.21.2.101:0/3831275533 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efc70006790 0x7efc70008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:16.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.986 7efc89cb4700 1 -- 172.21.2.101:0/3831275533 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efc70006790 msgr2=0x7efc70008c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:16.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.986 7efc89cb4700 1 --2- 172.21.2.101:0/3831275533 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efc70006790 0x7efc70008c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:16.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.986 7efc89cb4700 1 -- 172.21.2.101:0/3831275533 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efc70009500 msgr2=0x7efc7000ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:16.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.986 7efc89cb4700 1 --2- 172.21.2.101:0/3831275533 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efc70009500 0x7efc7000ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:16.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.986 7efc89cb4700 1 -- 172.21.2.101:0/3831275533 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc70050980 con 0x7efc70003b10 2021-02-10T11:49:16.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.986 7efc89cb4700 1 --2- 172.21.2.101:0/3831275533 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc70003b10 0x7efc70005f80 secure :-1 s=READY pgs=3358 cs=0 l=1 rev1=1 rx=0x7efc800014d0 tx=0x7efc80007c80).ready entity=mon.0 client_cookie=e0df8bdb6532f58 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:16.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.987 7efc88cb2700 1 -- 172.21.2.101:0/3831275533 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efc800070c0 con 0x7efc70003b10 2021-02-10T11:49:16.000 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.987 7efc88cb2700 1 -- 172.21.2.101:0/3831275533 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8000a520 con 0x7efc70003b10 2021-02-10T11:49:16.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.987 7efc88cb2700 1 -- 172.21.2.101:0/3831275533 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efc8000aa40 con 0x7efc70003b10 2021-02-10T11:49:16.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.987 7efc8be98700 1 -- 172.21.2.101:0/3831275533 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc70003b10 msgr2=0x7efc70005f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:16.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.987 7efc8be98700 1 --2- 172.21.2.101:0/3831275533 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc70003b10 0x7efc70005f80 secure :-1 s=READY pgs=3358 cs=0 l=1 rev1=1 rx=0x7efc800014d0 tx=0x7efc80007c80).stop 2021-02-10T11:49:16.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.987 7efc8a4b5700 1 -- 172.21.2.101:0/3831275533 reap_dead start 2021-02-10T11:49:16.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.987 7efc8be98700 1 -- 172.21.2.101:0/3831275533 shutdown_connections 2021-02-10T11:49:16.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.987 7efc8be98700 1 -- 172.21.2.101:0/3831275533 >> 172.21.2.101:0/3831275533 conn(0x7efc840ce450 msgr2=0x7efc840d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:16.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.988 7efc8be98700 1 -- 172.21.2.101:0/3831275533 shutdown_connections 2021-02-10T11:49:16.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:15.988 7efc8be98700 1 -- 172.21.2.101:0/3831275533 wait complete. 2021-02-10T11:49:16.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.002 7efc8be98700 1 Processor -- start 2021-02-10T11:49:16.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.002 7efc8be98700 1 -- start start 2021-02-10T11:49:16.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.003 7efc8be98700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc8419ebb0 0x7efc8419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:16.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.003 7efc8be98700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efc8419f5d0 0x7efc840ebe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:16.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.003 7efc8be98700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efc840ec450 0x7efc840ec850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:16.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.003 7efc8be98700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc840eceb0 con 0x7efc8419ebb0 2021-02-10T11:49:16.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.003 7efc8be98700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8409eb80 con 0x7efc8419f5d0 2021-02-10T11:49:16.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.003 7efc8be98700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc8409e710 con 0x7efc840ec450 2021-02-10T11:49:16.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.003 7efc89cb4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc8419ebb0 0x7efc8419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:16.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.003 7efc89cb4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc8419ebb0 0x7efc8419efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35664/0 (socket says 172.21.2.101:35664) 2021-02-10T11:49:16.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.003 7efc89cb4700 1 -- 172.21.2.101:0/1333171938 learned_addr learned my addr 172.21.2.101:0/1333171938 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:16.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.003 7efc89cb4700 1 -- 172.21.2.101:0/1333171938 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efc840ec450 msgr2=0x7efc840ec850 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:49:16.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.003 7efc89cb4700 1 --2- 172.21.2.101:0/1333171938 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efc840ec450 0x7efc840ec850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:16.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.003 7efc89cb4700 1 -- 172.21.2.101:0/1333171938 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efc8419f5d0 msgr2=0x7efc840ebe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:16.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.003 7efc89cb4700 1 --2- 172.21.2.101:0/1333171938 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efc8419f5d0 0x7efc840ebe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:16.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.003 7efc89cb4700 1 -- 172.21.2.101:0/1333171938 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc840ed430 con 0x7efc8419ebb0 2021-02-10T11:49:16.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.003 7efc89cb4700 1 --2- 172.21.2.101:0/1333171938 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc8419ebb0 0x7efc8419efb0 secure :-1 s=READY pgs=3359 cs=0 l=1 rev1=1 rx=0x7efc80001350 tx=0x7efc800016f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:16.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.004 7efc7a7fc700 1 -- 172.21.2.101:0/1333171938 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efc800078f0 con 0x7efc8419ebb0 2021-02-10T11:49:16.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.004 7efc7a7fc700 1 -- 172.21.2.101:0/1333171938 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc80007bf0 con 0x7efc8419ebb0 2021-02-10T11:49:16.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.004 7efc7a7fc700 1 -- 172.21.2.101:0/1333171938 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efc8000a350 con 0x7efc8419ebb0 2021-02-10T11:49:16.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.004 7efc8be98700 1 -- 172.21.2.101:0/1333171938 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc840ed430 con 0x7efc8419ebb0 2021-02-10T11:49:16.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.004 7efc8be98700 1 -- 172.21.2.101:0/1333171938 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc840ed430 con 0x7efc8419ebb0 2021-02-10T11:49:16.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.005 7efc7a7fc700 1 -- 172.21.2.101:0/1333171938 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7efc80007880 con 0x7efc8419ebb0 2021-02-10T11:49:16.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.005 7efc7a7fc700 1 --2- 172.21.2.101:0/1333171938 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efc6c0186c0 0x7efc6c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:16.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.005 7efc7a7fc700 1 -- 172.21.2.101:0/1333171938 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4753+0+0 (secure 0 0 0) 0x7efc80032240 con 0x7efc8419ebb0 2021-02-10T11:49:16.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.005 7efc894b3700 1 --2- 172.21.2.101:0/1333171938 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efc6c0186c0 0x7efc6c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:16.005 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.005 7efc894b3700 1 --2- 172.21.2.101:0/1333171938 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efc6c0186c0 0x7efc6c01abc0 crc :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:16.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.035 7efc8be98700 1 -- 172.21.2.101:0/1333171938 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc840ebc00 con 0x7efc8419ebb0 2021-02-10T11:49:16.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.037 7efc7a7fc700 1 -- 172.21.2.101:0/1333171938 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7efc80001ae0 con 0x7efc8419ebb0 2021-02-10T11:49:16.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.160 7efc8be98700 1 -- 172.21.2.101:0/1333171938 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7efc840ebc00 con 0x7efc8419ebb0 2021-02-10T11:49:16.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.161 7efc7a7fc700 1 -- 172.21.2.101:0/1333171938 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v195) v1 ==== 84+0+64 (secure 0 0 0) 0x7efc80007880 con 0x7efc8419ebb0 2021-02-10T11:49:16.157 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:49:16.157 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:49:16.158 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:16.158 INFO:teuthology.orchestra.run.gibba001.stdout: "poolnum": 1, 2021-02-10T11:49:16.158 INFO:teuthology.orchestra.run.gibba001.stdout: "poolname": "rbd" 2021-02-10T11:49:16.158 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:16.158 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:49:16.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.163 7efc8be98700 1 -- 172.21.2.101:0/1333171938 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efc6c0186c0 msgr2=0x7efc6c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:16.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.163 7efc8be98700 1 --2- 172.21.2.101:0/1333171938 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efc6c0186c0 0x7efc6c01abc0 crc :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:16.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.163 7efc8be98700 1 -- 172.21.2.101:0/1333171938 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc8419ebb0 msgr2=0x7efc8419efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:16.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.163 7efc8be98700 1 --2- 172.21.2.101:0/1333171938 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc8419ebb0 0x7efc8419efb0 secure :-1 s=READY pgs=3359 cs=0 l=1 rev1=1 rx=0x7efc80001350 tx=0x7efc800016f0).stop 2021-02-10T11:49:16.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.163 7efc8be98700 1 -- 172.21.2.101:0/1333171938 shutdown_connections 2021-02-10T11:49:16.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.163 7efc8be98700 1 --2- 172.21.2.101:0/1333171938 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7efc6c0186c0 0x7efc6c01abc0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:16.159 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.163 7efc8be98700 1 --2- 172.21.2.101:0/1333171938 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7efc840ec450 0x7efc840ec850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:16.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.163 7efc8be98700 1 --2- 172.21.2.101:0/1333171938 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7efc8419f5d0 0x7efc840ebe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:16.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.163 7efc8be98700 1 --2- 172.21.2.101:0/1333171938 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7efc8419ebb0 0x7efc8419efb0 unknown :-1 s=CLOSED pgs=3359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:16.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.163 7efc8be98700 1 -- 172.21.2.101:0/1333171938 >> 172.21.2.101:0/1333171938 conn(0x7efc840c0cc0 msgr2=0x7efc840b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:16.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.163 7efc8be98700 1 -- 172.21.2.101:0/1333171938 shutdown_connections 2021-02-10T11:49:16.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.163 7efc8be98700 1 -- 172.21.2.101:0/1333171938 wait complete. 2021-02-10T11:49:16.168 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-02-10T11:49:16.168 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-02-10T11:49:16.601 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.588 7fc9ec3e5700 1 Processor -- start 2021-02-10T11:49:16.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.588 7fc9ec3e5700 1 -- start start 2021-02-10T11:49:16.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.589 7fc9ec3e5700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc9e40d1c60 0x7fc9e40cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:16.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.589 7fc9ec3e5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e40cf570 0x7fc9e40cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:16.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.589 7fc9ec3e5700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e40fe1b0 con 0x7fc9e40d1860 2021-02-10T11:49:16.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.589 7fc9ec3e5700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e41b0ca0 con 0x7fc9e40d1c60 2021-02-10T11:49:16.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.589 7fc9ec3e5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e4109a50 con 0x7fc9e40cf570 2021-02-10T11:49:16.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.589 7fc9e9a00700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc9e40d1c60 0x7fc9e40cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:16.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.589 7fc9ea201700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7fc9e40d1860 0x7fc9e41b04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:57808/0 (socket says 172.21.2.101:57808) 2021-02-10T11:49:16.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.589 7fc9ea201700 1 -- 172.21.2.101:0/2139980804 learned_addr learned my addr 172.21.2.101:0/2139980804 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:16.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.589 7fc9eaa02700 1 --2- 172.21.2.101:0/2139980804 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e40cf570 0x7fc9e40cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:16.602 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.590 7fc9e91ff700 1 -- 172.21.2.101:0/2139980804 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996675487 0 0) 0x7fc9d4001190 con 0x7fc9e40d1860 2021-02-10T11:49:16.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.590 7fc9e91ff700 1 -- 172.21.2.101:0/2139980804 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9d0002fe0 con 0x7fc9e40d1860 2021-02-10T11:49:16.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.590 7fc9e9a00700 1 -- 172.21.2.101:0/2139980804 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e40cf570 msgr2=0x7fc9e40cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:16.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.590 7fc9e9a00700 1 --2- 172.21.2.101:0/2139980804 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e40cf570 0x7fc9e40cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:16.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.590 7fc9e9a00700 1 -- 172.21.2.101:0/2139980804 >> v1:172.21.2.108:6789/0 conn(0x7fc9e40d1860 legacy=0x7fc9e41b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:16.603 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.590 7fc9e9a00700 1 -- 172.21.2.101:0/2139980804 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e4088070 con 0x7fc9e40d1c60 2021-02-10T11:49:16.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.590 7fc9eaa02700 1 --2- 172.21.2.101:0/2139980804 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e40cf570 0x7fc9e40cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:49:16.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.590 7fc9e9a00700 1 --2- 172.21.2.101:0/2139980804 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9e40d1c60 0x7fc9e40cef50 secure :-1 s=READY pgs=3360 cs=0 l=1 rev1=1 rx=0x7fc9e00069d0 tx=0x7fc9e0009e40).ready entity=mon.0 client_cookie=145079e38da2fcc0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:16.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.590 7fc9e91ff700 1 -- 172.21.2.101:0/2139980804 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc9e000b510 con 0x7fc9e40d1c60 2021-02-10T11:49:16.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.590 7fc9e91ff700 1 -- 172.21.2.101:0/2139980804 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e000b890 con 0x7fc9e40d1c60 2021-02-10T11:49:16.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.590 7fc9e91ff700 1 -- 172.21.2.101:0/2139980804 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc9e000bdb0 con 0x7fc9e40d1c60 2021-02-10T11:49:16.604 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.590 7fc9ec3e5700 1 -- 172.21.2.101:0/2139980804 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9e40d1c60 msgr2=0x7fc9e40cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:16.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.590 7fc9ec3e5700 1 --2- 172.21.2.101:0/2139980804 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9e40d1c60 0x7fc9e40cef50 secure :-1 s=READY pgs=3360 cs=0 l=1 rev1=1 rx=0x7fc9e00069d0 tx=0x7fc9e0009e40).stop 2021-02-10T11:49:16.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.591 7fc9ec3e5700 1 -- 172.21.2.101:0/2139980804 shutdown_connections 2021-02-10T11:49:16.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.591 7fc9ec3e5700 1 --2- 172.21.2.101:0/2139980804 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e40cf570 0x7fc9e40cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:16.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.591 7fc9ec3e5700 1 --2- 172.21.2.101:0/2139980804 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9e40d1c60 0x7fc9e40cef50 unknown :-1 s=CLOSED pgs=3360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:16.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.591 7fc9ec3e5700 1 -- 172.21.2.101:0/2139980804 >> 172.21.2.101:0/2139980804 conn(0x7fc9e40ce410 msgr2=0x7fc9e40d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:16.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.591 7fc9ec3e5700 1 -- 172.21.2.101:0/2139980804 shutdown_connections 2021-02-10T11:49:16.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.591 7fc9ec3e5700 1 -- 172.21.2.101:0/2139980804 wait complete. 2021-02-10T11:49:16.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.606 7fc9ec3e5700 1 Processor -- start 2021-02-10T11:49:16.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.606 7fc9ec3e5700 1 -- start start 2021-02-10T11:49:16.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.606 7fc9ec3e5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9e4085270 0x7fc9e4085670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:16.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.606 7fc9ec3e5700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e4085c90 0x7fc9e4086090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:16.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.606 7fc9ec3e5700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9e40c2740 0x7fc9e40c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:16.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.606 7fc9ec3e5700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e4084cf0 con 0x7fc9e4085270 2021-02-10T11:49:16.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.606 7fc9ec3e5700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e40be1b0 con 0x7fc9e40c2740 2021-02-10T11:49:16.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.606 7fc9ec3e5700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e40be390 con 0x7fc9e4085c90 2021-02-10T11:49:16.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.606 7fc9ea201700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9e4085270 0x7fc9e4085670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:16.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.606 7fc9ea201700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9e4085270 0x7fc9e4085670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35678/0 (socket says 172.21.2.101:35678) 2021-02-10T11:49:16.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.606 7fc9ea201700 1 -- 172.21.2.101:0/829778255 learned_addr learned my addr 172.21.2.101:0/829778255 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:16.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.607 7fc9ea201700 1 -- 172.21.2.101:0/829778255 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e4085c90 msgr2=0x7fc9e4086090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:16.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.607 7fc9ea201700 1 --2- 172.21.2.101:0/829778255 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e4085c90 0x7fc9e4086090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:16.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.607 7fc9ea201700 1 -- 172.21.2.101:0/829778255 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9e40c2740 msgr2=0x7fc9e40c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:16.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.607 7fc9ea201700 1 --2- 172.21.2.101:0/829778255 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9e40c2740 0x7fc9e40c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:16.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.607 7fc9ea201700 1 -- 172.21.2.101:0/829778255 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e40c3160 con 0x7fc9e4085270 2021-02-10T11:49:16.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.607 7fc9ea201700 1 --2- 172.21.2.101:0/829778255 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9e4085270 0x7fc9e4085670 secure :-1 s=READY pgs=3361 cs=0 l=1 rev1=1 rx=0x7fc9d400a4c0 tx=0x7fc9d400a860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:16.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.607 7fc9daffd700 1 -- 172.21.2.101:0/829778255 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc9d4007280 con 0x7fc9e4085270 2021-02-10T11:49:16.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.607 7fc9daffd700 1 -- 172.21.2.101:0/829778255 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d400ac50 con 0x7fc9e4085270 2021-02-10T11:49:16.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.607 7fc9daffd700 1 -- 172.21.2.101:0/829778255 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc9d40045a0 con 0x7fc9e4085270 2021-02-10T11:49:16.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.607 7fc9ec3e5700 1 -- 172.21.2.101:0/829778255 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e40c3160 con 0x7fc9e4085270 2021-02-10T11:49:16.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.607 7fc9ec3e5700 1 -- 172.21.2.101:0/829778255 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e40c36c0 con 0x7fc9e4085270 2021-02-10T11:49:16.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.607 7fc9daffd700 1 -- 172.21.2.101:0/829778255 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc9d4004320 con 0x7fc9e4085270 2021-02-10T11:49:16.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.608 7fc9daffd700 1 --2- 172.21.2.101:0/829778255 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9cc0186c0 0x7fc9cc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:16.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.608 7fc9daffd700 1 -- 172.21.2.101:0/829778255 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4753+0+0 (secure 0 0 0) 0x7fc9d4032200 con 0x7fc9e4085270 2021-02-10T11:49:16.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.608 7fc9e9a00700 1 --2- 172.21.2.101:0/829778255 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9cc0186c0 0x7fc9cc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:16.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.608 7fc9e9a00700 1 --2- 172.21.2.101:0/829778255 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9cc0186c0 0x7fc9cc01abc0 crc :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:16.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.639 7fc9ec3e5700 1 -- 172.21.2.101:0/829778255 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e4082a20 con 0x7fc9e4085270 2021-02-10T11:49:16.637 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.641 7fc9daffd700 1 -- 172.21.2.101:0/829778255 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc9d402dff0 con 0x7fc9e4085270 2021-02-10T11:49:16.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:16.772 7fc9ec3e5700 1 -- 172.21.2.101:0/829778255 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"} v 0) v1 -- 0x7fc9e4082a20 con 0x7fc9e4085270 2021-02-10T11:49:17.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.171 7fc9daffd700 1 -- 172.21.2.101:0/829778255 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_metadata"}]=0 pool 'cephfs_metadata' created v196) v1 ==== 133+0+0 (secure 0 0 0) 0x7fc9d402dff0 con 0x7fc9e4085270 2021-02-10T11:49:17.184 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_metadata' created 2021-02-10T11:49:17.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.190 7fc9ec3e5700 1 -- 172.21.2.101:0/829778255 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9cc0186c0 msgr2=0x7fc9cc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:17.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.190 7fc9ec3e5700 1 --2- 172.21.2.101:0/829778255 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9cc0186c0 0x7fc9cc01abc0 crc :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:17.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.190 7fc9ec3e5700 1 -- 172.21.2.101:0/829778255 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9e4085270 msgr2=0x7fc9e4085670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:17.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.190 7fc9ec3e5700 1 --2- 172.21.2.101:0/829778255 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9e4085270 0x7fc9e4085670 secure :-1 s=READY pgs=3361 cs=0 l=1 rev1=1 rx=0x7fc9d400a4c0 tx=0x7fc9d400a860).stop 2021-02-10T11:49:17.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.190 7fc9ec3e5700 1 -- 172.21.2.101:0/829778255 shutdown_connections 2021-02-10T11:49:17.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.190 7fc9ec3e5700 1 --2- 172.21.2.101:0/829778255 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc9cc0186c0 0x7fc9cc01abc0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:17.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.190 7fc9ec3e5700 1 --2- 172.21.2.101:0/829778255 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc9e4085c90 0x7fc9e4086090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:17.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.190 7fc9ec3e5700 1 --2- 172.21.2.101:0/829778255 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc9e40c2740 0x7fc9e40c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:17.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.190 7fc9ec3e5700 1 --2- 172.21.2.101:0/829778255 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc9e4085270 0x7fc9e4085670 unknown :-1 s=CLOSED pgs=3361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:17.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.190 7fc9ec3e5700 1 -- 172.21.2.101:0/829778255 >> 172.21.2.101:0/829778255 conn(0x7fc9e40b5f10 msgr2=0x7fc9e409f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:17.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.190 7fc9ec3e5700 1 -- 172.21.2.101:0/829778255 shutdown_connections 2021-02-10T11:49:17.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.190 7fc9ec3e5700 1 -- 172.21.2.101:0/829778255 wait complete. 2021-02-10T11:49:17.195 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool create cephfs_data 8 2021-02-10T11:49:17.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.617 7f7aefd2f700 1 Processor -- start 2021-02-10T11:49:17.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.617 7f7aefd2f700 1 -- start start 2021-02-10T11:49:17.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.618 7f7aefd2f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7ae80d1c70 0x7f7ae80cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:17.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.618 7f7aefd2f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f7ae80cf580 0x7f7ae80cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:17.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.618 7f7aefd2f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae80fe1c0 con 0x7f7ae80d1870 2021-02-10T11:49:17.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.618 7f7aefd2f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae81b0cb0 con 0x7f7ae80d1c70 2021-02-10T11:49:17.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.618 7f7aefd2f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae8109a60 con 0x7f7ae80cf580 2021-02-10T11:49:17.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.618 7f7aed34a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7ae80d1c70 0x7f7ae80cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:17.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.618 7f7aed34a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f7ae80d1c70 0x7f7ae80cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35686/0 (socket says 172.21.2.101:35686) 2021-02-10T11:49:17.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.618 7f7aed34a700 1 -- 172.21.2.101:0/1801093821 learned_addr learned my addr 172.21.2.101:0/1801093821 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:17.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.618 7f7aed34a700 1 -- 172.21.2.101:0/1801093821 >> v2:172.21.2.108:3300/0 conn(0x7f7ae80cf580 msgr2=0x7f7ae80cf980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:49:17.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.618 7f7aed34a700 1 --2- 172.21.2.101:0/1801093821 >> v2:172.21.2.108:3300/0 conn(0x7f7ae80cf580 0x7f7ae80cf980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:17.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.618 7f7aed34a700 1 -- 172.21.2.101:0/1801093821 >> v1:172.21.2.108:6789/0 conn(0x7f7ae80d1870 legacy=0x7f7ae81b04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:49:17.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.618 7f7aed34a700 1 -- 172.21.2.101:0/1801093821 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae8088070 con 0x7f7ae80d1c70 2021-02-10T11:49:17.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.618 7f7aed34a700 1 --2- 172.21.2.101:0/1801093821 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ae80d1c70 0x7f7ae80cef60 secure :-1 s=READY pgs=3362 cs=0 l=1 rev1=1 rx=0x7f7ae4006700 tx=0x7f7ae4009b80).ready entity=mon.0 client_cookie=709bdc034ea51f44 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:17.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.618 7f7aecb49700 1 -- 172.21.2.101:0/1801093821 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ae400c0c0 con 0x7f7ae80d1c70 2021-02-10T11:49:17.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.618 7f7aecb49700 1 -- 172.21.2.101:0/1801093821 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae400c340 con 0x7f7ae80d1c70 2021-02-10T11:49:17.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.618 7f7aecb49700 1 -- 172.21.2.101:0/1801093821 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ae400c820 con 0x7f7ae80d1c70 2021-02-10T11:49:17.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.619 7f7aefd2f700 1 -- 172.21.2.101:0/1801093821 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ae80d1c70 msgr2=0x7f7ae80cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:17.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.619 7f7aefd2f700 1 --2- 172.21.2.101:0/1801093821 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ae80d1c70 0x7f7ae80cef60 secure :-1 s=READY pgs=3362 cs=0 l=1 rev1=1 rx=0x7f7ae4006700 tx=0x7f7ae4009b80).stop 2021-02-10T11:49:17.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.619 7f7aefd2f700 1 -- 172.21.2.101:0/1801093821 shutdown_connections 2021-02-10T11:49:17.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.619 7f7aefd2f700 1 --2- 172.21.2.101:0/1801093821 >> v2:172.21.2.108:3300/0 conn(0x7f7ae80cf580 0x7f7ae80cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:17.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.619 7f7aefd2f700 1 --2- 172.21.2.101:0/1801093821 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ae80d1c70 0x7f7ae80cef60 unknown :-1 s=CLOSED pgs=3362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:17.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.619 7f7aefd2f700 1 -- 172.21.2.101:0/1801093821 >> 172.21.2.101:0/1801093821 conn(0x7f7ae80ce420 msgr2=0x7f7ae80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:17.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.619 7f7aefd2f700 1 -- 172.21.2.101:0/1801093821 shutdown_connections 2021-02-10T11:49:17.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.619 7f7aefd2f700 1 -- 172.21.2.101:0/1801093821 wait complete. 2021-02-10T11:49:17.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.633 7f7aefd2f700 1 Processor -- start 2021-02-10T11:49:17.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.633 7f7aefd2f700 1 -- start start 2021-02-10T11:49:17.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.634 7f7aefd2f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ae80c3820 0x7f7ae8084cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:17.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.634 7f7aefd2f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ae80a0290 0x7f7ae80852d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:17.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.634 7f7aefd2f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ae8086270 0x7f7ae8086670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:17.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.634 7f7aefd2f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae80c0700 con 0x7f7ae80c3820 2021-02-10T11:49:17.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.634 7f7aefd2f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae80858f0 con 0x7f7ae80a0290 2021-02-10T11:49:17.633 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.634 7f7aefd2f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae8085b10 con 0x7f7ae8086270 2021-02-10T11:49:17.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.634 7f7aedb4b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ae80c3820 0x7f7ae8084cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:17.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.634 7f7aedb4b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ae80c3820 0x7f7ae8084cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35696/0 (socket says 172.21.2.101:35696) 2021-02-10T11:49:17.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.634 7f7aedb4b700 1 -- 172.21.2.101:0/2929458217 learned_addr learned my addr 172.21.2.101:0/2929458217 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:17.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.634 7f7aedb4b700 1 -- 172.21.2.101:0/2929458217 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ae8086270 msgr2=0x7f7ae8086670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:49:17.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.634 7f7aedb4b700 1 --2- 172.21.2.101:0/2929458217 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ae8086270 0x7f7ae8086670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:17.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.634 7f7aedb4b700 1 -- 172.21.2.101:0/2929458217 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ae80a0290 msgr2=0x7f7ae80852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:17.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.634 7f7aedb4b700 1 --2- 172.21.2.101:0/2929458217 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ae80a0290 0x7f7ae80852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:17.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.634 7f7aedb4b700 1 -- 172.21.2.101:0/2929458217 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae8086c90 con 0x7f7ae80c3820 2021-02-10T11:49:17.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.635 7f7aedb4b700 1 --2- 172.21.2.101:0/2929458217 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ae80c3820 0x7f7ae8084cb0 secure :-1 s=READY pgs=3363 cs=0 l=1 rev1=1 rx=0x7f7adc00b530 tx=0x7f7adc00b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:17.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.635 7f7ada7fc700 1 -- 172.21.2.101:0/2929458217 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7adc00bcc0 con 0x7f7ae80c3820 2021-02-10T11:49:17.634 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.635 7f7ada7fc700 1 -- 172.21.2.101:0/2929458217 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7adc000c60 con 0x7f7ae80c3820 2021-02-10T11:49:17.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.635 7f7ada7fc700 1 -- 172.21.2.101:0/2929458217 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7adc0062a0 con 0x7f7ae80c3820 2021-02-10T11:49:17.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.635 7f7aefd2f700 1 -- 172.21.2.101:0/2929458217 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae8086c90 con 0x7f7ae80c3820 2021-02-10T11:49:17.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.635 7f7aefd2f700 1 -- 172.21.2.101:0/2929458217 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae8086c90 con 0x7f7ae80c3820 2021-02-10T11:49:17.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.636 7f7ada7fc700 1 -- 172.21.2.101:0/2929458217 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7adc006020 con 0x7f7ae80c3820 2021-02-10T11:49:17.635 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.636 7f7ada7fc700 1 --2- 172.21.2.101:0/2929458217 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ad00186c0 0x7f7ad001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:17.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.636 7f7ada7fc700 1 -- 172.21.2.101:0/2929458217 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5107+0+0 (secure 0 0 0) 0x7f7adc0323e0 con 0x7f7ae80c3820 2021-02-10T11:49:17.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.636 7f7aed34a700 1 --2- 172.21.2.101:0/2929458217 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ad00186c0 0x7f7ad001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:17.636 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.636 7f7aed34a700 1 --2- 172.21.2.101:0/2929458217 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ad00186c0 0x7f7ad001abc0 crc :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:17.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.668 7f7aefd2f700 1 -- 172.21.2.101:0/2929458217 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae8082a70 con 0x7f7ae80c3820 2021-02-10T11:49:17.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.671 7f7ada7fc700 1 -- 172.21.2.101:0/2929458217 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7adc030d60 con 0x7f7ae80c3820 2021-02-10T11:49:17.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:17.801 7f7aefd2f700 1 -- 172.21.2.101:0/2929458217 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"} v 0) v1 -- 0x7f7ae8082a70 con 0x7f7ae80c3820 2021-02-10T11:49:18.173 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.177 7f7ada7fc700 1 -- 172.21.2.101:0/2929458217 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pg_num": 8, "pool": "cephfs_data"}]=0 pool 'cephfs_data' created v197) v1 ==== 125+0+0 (secure 0 0 0) 0x7f7adc006020 con 0x7f7ae80c3820 2021-02-10T11:49:18.214 INFO:teuthology.orchestra.run.gibba001.stderr:pool 'cephfs_data' created 2021-02-10T11:49:18.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.219 7f7aefd2f700 1 -- 172.21.2.101:0/2929458217 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ad00186c0 msgr2=0x7f7ad001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:18.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.219 7f7aefd2f700 1 --2- 172.21.2.101:0/2929458217 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ad00186c0 0x7f7ad001abc0 crc :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:18.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.219 7f7aefd2f700 1 -- 172.21.2.101:0/2929458217 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ae80c3820 msgr2=0x7f7ae8084cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:18.217 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.219 7f7aefd2f700 1 --2- 172.21.2.101:0/2929458217 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ae80c3820 0x7f7ae8084cb0 secure :-1 s=READY pgs=3363 cs=0 l=1 rev1=1 rx=0x7f7adc00b530 tx=0x7f7adc00b8d0).stop 2021-02-10T11:49:18.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.219 7f7aefd2f700 1 -- 172.21.2.101:0/2929458217 shutdown_connections 2021-02-10T11:49:18.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.219 7f7aefd2f700 1 --2- 172.21.2.101:0/2929458217 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f7ad00186c0 0x7f7ad001abc0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:18.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.219 7f7aefd2f700 1 --2- 172.21.2.101:0/2929458217 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f7ae8086270 0x7f7ae8086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:18.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.219 7f7aefd2f700 1 --2- 172.21.2.101:0/2929458217 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f7ae80a0290 0x7f7ae80852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:18.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.219 7f7aefd2f700 1 --2- 172.21.2.101:0/2929458217 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f7ae80c3820 0x7f7ae8084cb0 unknown :-1 s=CLOSED pgs=3363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:18.218 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.219 7f7aefd2f700 1 -- 172.21.2.101:0/2929458217 >> 172.21.2.101:0/2929458217 conn(0x7f7ae80872c0 msgr2=0x7f7ae8095700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:18.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.220 7f7aefd2f700 1 -- 172.21.2.101:0/2929458217 shutdown_connections 2021-02-10T11:49:18.219 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.220 7f7aefd2f700 1 -- 172.21.2.101:0/2929458217 wait complete. 2021-02-10T11:49:18.225 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-02-10T11:49:18.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.646 7f3e888ec700 1 Processor -- start 2021-02-10T11:49:18.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.646 7f3e888ec700 1 -- start start 2021-02-10T11:49:18.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.646 7f3e888ec700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3e840d1880 0x7f3e841b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:18.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.646 7f3e888ec700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e840cff80 0x7f3e84109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:18.659 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.646 7f3e888ec700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e84109a70 con 0x7f3e841b0af0 2021-02-10T11:49:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.646 7f3e888ec700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e84109760 con 0x7f3e840d1880 2021-02-10T11:49:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.646 7f3e888ec700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e84088070 con 0x7f3e840cff80 2021-02-10T11:49:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.647 7f3e8261c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3e840d1880 0x7f3e841b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.647 7f3e8261c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f3e840d1880 0x7f3e841b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35702/0 (socket says 172.21.2.101:35702) 2021-02-10T11:49:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.647 7f3e8261c700 1 -- 172.21.2.101:0/2965375513 learned_addr learned my addr 172.21.2.101:0/2965375513 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.647 7f3e8261c700 1 -- 172.21.2.101:0/2965375513 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e840cff80 msgr2=0x7f3e84109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.647 7f3e8261c700 1 --2- 172.21.2.101:0/2965375513 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e840cff80 0x7f3e84109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.647 7f3e8261c700 1 -- 172.21.2.101:0/2965375513 >> v1:172.21.2.101:6789/0 conn(0x7f3e841b0af0 legacy=0x7f3e84108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.647 7f3e8261c700 1 -- 172.21.2.101:0/2965375513 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e840d1c80 con 0x7f3e840d1880 2021-02-10T11:49:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.647 7f3e8261c700 1 --2- 172.21.2.101:0/2965375513 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e840d1880 0x7f3e841b04d0 secure :-1 s=READY pgs=3365 cs=0 l=1 rev1=1 rx=0x7f3e74006700 tx=0x7f3e74009b80).ready entity=mon.0 client_cookie=aff7c6c6a85c9cc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.647 7f3e8161a700 1 -- 172.21.2.101:0/2965375513 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3e7400c0c0 con 0x7f3e840d1880 2021-02-10T11:49:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.647 7f3e8161a700 1 -- 172.21.2.101:0/2965375513 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7400c340 con 0x7f3e840d1880 2021-02-10T11:49:18.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.647 7f3e8161a700 1 -- 172.21.2.101:0/2965375513 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3e7400c860 con 0x7f3e840d1880 2021-02-10T11:49:18.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.648 7f3e888ec700 1 -- 172.21.2.101:0/2965375513 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e840d1880 msgr2=0x7f3e841b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:18.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.648 7f3e888ec700 1 --2- 172.21.2.101:0/2965375513 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e840d1880 0x7f3e841b04d0 secure :-1 s=READY pgs=3365 cs=0 l=1 rev1=1 rx=0x7f3e74006700 tx=0x7f3e74009b80).stop 2021-02-10T11:49:18.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.648 7f3e888ec700 1 -- 172.21.2.101:0/2965375513 shutdown_connections 2021-02-10T11:49:18.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.648 7f3e888ec700 1 --2- 172.21.2.101:0/2965375513 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e840cff80 0x7f3e84109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:18.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.648 7f3e888ec700 1 --2- 172.21.2.101:0/2965375513 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e840d1880 0x7f3e841b04d0 unknown :-1 s=CLOSED pgs=3365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:18.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.648 7f3e888ec700 1 -- 172.21.2.101:0/2965375513 >> 172.21.2.101:0/2965375513 conn(0x7f3e840ce430 msgr2=0x7f3e840d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:18.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.648 7f3e888ec700 1 -- 172.21.2.101:0/2965375513 shutdown_connections 2021-02-10T11:49:18.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.648 7f3e888ec700 1 -- 172.21.2.101:0/2965375513 wait complete. 2021-02-10T11:49:18.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.663 7f3e888ec700 1 Processor -- start 2021-02-10T11:49:18.661 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.663 7f3e888ec700 1 -- start start 2021-02-10T11:49:18.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.663 7f3e888ec700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e8419e5b0 0x7f3e841afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:18.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.663 7f3e888ec700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e8419e9b0 0x7f3e841b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:18.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.663 7f3e888ec700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3e841b0250 0x7f3e841b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:18.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.663 7f3e888ec700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e84195bd0 con 0x7f3e8419e9b0 2021-02-10T11:49:18.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.663 7f3e888ec700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e841b0840 con 0x7f3e841b0250 2021-02-10T11:49:18.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.663 7f3e888ec700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8419eed0 con 0x7f3e8419e5b0 2021-02-10T11:49:18.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.664 7f3e81e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e8419e9b0 0x7f3e841b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:18.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.664 7f3e81e1b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e8419e9b0 0x7f3e841b0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35710/0 (socket says 172.21.2.101:35710) 2021-02-10T11:49:18.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.664 7f3e81e1b700 1 -- 172.21.2.101:0/802243465 learned_addr learned my addr 172.21.2.101:0/802243465 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:18.663 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.664 7f3e8261c700 1 --2- 172.21.2.101:0/802243465 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e8419e5b0 0x7f3e841afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:18.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.664 7f3e82e1d700 1 --2- 172.21.2.101:0/802243465 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3e841b0250 0x7f3e841b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:18.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.664 7f3e81e1b700 1 -- 172.21.2.101:0/802243465 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e8419e5b0 msgr2=0x7f3e841afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:18.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.664 7f3e81e1b700 1 --2- 172.21.2.101:0/802243465 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e8419e5b0 0x7f3e841afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:18.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.664 7f3e81e1b700 1 -- 172.21.2.101:0/802243465 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3e841b0250 msgr2=0x7f3e841b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:18.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.664 7f3e81e1b700 1 --2- 172.21.2.101:0/802243465 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3e841b0250 0x7f3e841b13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:18.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.664 7f3e81e1b700 1 -- 172.21.2.101:0/802243465 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8419f0b0 con 0x7f3e8419e9b0 2021-02-10T11:49:18.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.664 7f3e82e1d700 1 --2- 172.21.2.101:0/802243465 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3e841b0250 0x7f3e841b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:49:18.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.664 7f3e81e1b700 1 --2- 172.21.2.101:0/802243465 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e8419e9b0 0x7f3e841b0dc0 secure :-1 s=READY pgs=3366 cs=0 l=1 rev1=1 rx=0x7f3e78006ed0 tx=0x7f3e7800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:18.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.664 7f3e72ffd700 1 -- 172.21.2.101:0/802243465 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3e78006240 con 0x7f3e8419e9b0 2021-02-10T11:49:18.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.665 7f3e72ffd700 1 -- 172.21.2.101:0/802243465 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e780009b0 con 0x7f3e8419e9b0 2021-02-10T11:49:18.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.665 7f3e72ffd700 1 -- 172.21.2.101:0/802243465 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3e78005730 con 0x7f3e8419e9b0 2021-02-10T11:49:18.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.665 7f3e888ec700 1 -- 172.21.2.101:0/802243465 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8419f0b0 con 0x7f3e8419e9b0 2021-02-10T11:49:18.664 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.665 7f3e888ec700 1 -- 172.21.2.101:0/802243465 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e841b1cf0 con 0x7f3e8419e9b0 2021-02-10T11:49:18.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.665 7f3e72ffd700 1 -- 172.21.2.101:0/802243465 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3e78006030 con 0x7f3e8419e9b0 2021-02-10T11:49:18.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.665 7f3e72ffd700 1 --2- 172.21.2.101:0/802243465 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3e680186c0 0x7f3e6801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:18.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.665 7f3e72ffd700 1 -- 172.21.2.101:0/802243465 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5457+0+0 (secure 0 0 0) 0x7f3e78032360 con 0x7f3e8419e9b0 2021-02-10T11:49:18.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.666 7f3e8261c700 1 --2- 172.21.2.101:0/802243465 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3e680186c0 0x7f3e6801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:18.665 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.666 7f3e8261c700 1 --2- 172.21.2.101:0/802243465 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3e680186c0 0x7f3e6801abc0 crc :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:18.692 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.696 7f3e888ec700 1 -- 172.21.2.101:0/802243465 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e841b1cf0 con 0x7f3e8419e9b0 2021-02-10T11:49:18.695 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.699 7f3e72ffd700 1 -- 172.21.2.101:0/802243465 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3e78030d00 con 0x7f3e8419e9b0 2021-02-10T11:49:18.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.829 7f3e888ec700 1 -- 172.21.2.101:0/802243465 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"} v 0) v1 -- 0x7f3e841b1cf0 con 0x7f3e8419e9b0 2021-02-10T11:49:18.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.833 7f3e72ffd700 1 -- 172.21.2.101:0/802243465 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"data": "cephfs_data", "prefix": "fs new", "force": true, "fs_name": "cephfs", "metadata": "cephfs_metadata"}]=0 new fs with metadata pool 42 and data pool 43 v169) v1 ==== 189+0+0 (secure 0 0 0) 0x7f3e78006030 con 0x7f3e8419e9b0 2021-02-10T11:49:18.832 INFO:teuthology.orchestra.run.gibba001.stderr:new fs with metadata pool 42 and data pool 43 2021-02-10T11:49:18.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.838 7f3e888ec700 1 -- 172.21.2.101:0/802243465 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3e680186c0 msgr2=0x7f3e6801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:18.834 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.838 7f3e888ec700 1 --2- 172.21.2.101:0/802243465 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3e680186c0 0x7f3e6801abc0 crc :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:18.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.838 7f3e888ec700 1 -- 172.21.2.101:0/802243465 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e8419e9b0 msgr2=0x7f3e841b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:18.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.838 7f3e888ec700 1 --2- 172.21.2.101:0/802243465 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e8419e9b0 0x7f3e841b0dc0 secure :-1 s=READY pgs=3366 cs=0 l=1 rev1=1 rx=0x7f3e78006ed0 tx=0x7f3e7800b960).stop 2021-02-10T11:49:18.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.838 7f3e888ec700 1 -- 172.21.2.101:0/802243465 shutdown_connections 2021-02-10T11:49:18.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.838 7f3e888ec700 1 --2- 172.21.2.101:0/802243465 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3e841b0250 0x7f3e841b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:18.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.838 7f3e888ec700 1 --2- 172.21.2.101:0/802243465 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f3e8419e9b0 0x7f3e841b0dc0 unknown :-1 s=CLOSED pgs=3366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:18.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.838 7f3e888ec700 1 --2- 172.21.2.101:0/802243465 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3e680186c0 0x7f3e6801abc0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:18.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.838 7f3e888ec700 1 --2- 172.21.2.101:0/802243465 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3e8419e5b0 0x7f3e841afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:18.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.838 7f3e888ec700 1 -- 172.21.2.101:0/802243465 >> 172.21.2.101:0/802243465 conn(0x7f3e8419f940 msgr2=0x7f3e8418c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:18.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.838 7f3e888ec700 1 -- 172.21.2.101:0/802243465 shutdown_connections 2021-02-10T11:49:18.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:18.838 7f3e888ec700 1 -- 172.21.2.101:0/802243465 wait complete. 2021-02-10T11:49:18.843 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:49:19.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.260 7f5ffc147700 1 Processor -- start 2021-02-10T11:49:19.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.260 7f5ffc147700 1 -- start start 2021-02-10T11:49:19.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.261 7f5ffc147700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f5ff40d1870 0x7f5ff41b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:19.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.261 7f5ffc147700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff40f0b10 con 0x7f5ff40f0650 2021-02-10T11:49:19.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.261 7f5ffc147700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff40d0310 con 0x7f5ff41b0a10 2021-02-10T11:49:19.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.261 7f5ffc147700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff40d0530 con 0x7f5ff40d1870 2021-02-10T11:49:19.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.261 7f5ffa764700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f5ff40f0650 0x7f5ff40cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36214/0 (socket says 172.21.2.101:36214) 2021-02-10T11:49:19.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.261 7f5ffa764700 1 -- 172.21.2.101:0/2777579740 learned_addr learned my addr 172.21.2.101:0/2777579740 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:19.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.261 7f5ff9f63700 1 --2- 172.21.2.101:0/2777579740 >> v2:172.21.2.108:3300/0 conn(0x7f5ff40d1870 0x7f5ff41b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:19.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.261 7f5ff8f61700 1 -- 172.21.2.101:0/2777579740 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1174260026 0 0) 0x7f5fec001500 con 0x7f5ff40f0650 2021-02-10T11:49:19.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.261 7f5ff8f61700 1 -- 172.21.2.101:0/2777579740 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fe0002fe0 con 0x7f5ff40f0650 2021-02-10T11:49:19.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.262 7f5ff9f63700 1 -- 172.21.2.101:0/2777579740 >> v1:172.21.2.108:6789/0 conn(0x7f5ff41b0a10 legacy=0x7f5ff40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:19.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.262 7f5ff8f61700 1 -- 172.21.2.101:0/2777579740 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 841178137 0 0) 0x7f5fec001270 con 0x7f5ff40f0650 2021-02-10T11:49:19.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.262 7f5ff9f63700 1 -- 172.21.2.101:0/2777579740 >> v1:172.21.2.101:6789/0 conn(0x7f5ff40f0650 legacy=0x7f5ff40cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:19.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.262 7f5ff9f63700 1 -- 172.21.2.101:0/2777579740 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff40d1c70 con 0x7f5ff40d1870 2021-02-10T11:49:19.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.262 7f5ff9f63700 1 --2- 172.21.2.101:0/2777579740 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ff40d1870 0x7f5ff41b03f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7f5fe8006650 tx=0x7f5fe8009a80).ready entity=mon.1 client_cookie=e6d069675e33b74 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:19.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.262 7f5ff8f61700 1 -- 172.21.2.101:0/2777579740 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5fe800c0c0 con 0x7f5ff40d1870 2021-02-10T11:49:19.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.262 7f5ff8f61700 1 -- 172.21.2.101:0/2777579740 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe800c340 con 0x7f5ff40d1870 2021-02-10T11:49:19.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.262 7f5ff8f61700 1 -- 172.21.2.101:0/2777579740 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5fe800c860 con 0x7f5ff40d1870 2021-02-10T11:49:19.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.262 7f5ffc147700 1 -- 172.21.2.101:0/2777579740 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ff40d1870 msgr2=0x7f5ff41b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:19.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.262 7f5ffc147700 1 --2- 172.21.2.101:0/2777579740 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ff40d1870 0x7f5ff41b03f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7f5fe8006650 tx=0x7f5fe8009a80).stop 2021-02-10T11:49:19.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.263 7f5ffc147700 1 -- 172.21.2.101:0/2777579740 shutdown_connections 2021-02-10T11:49:19.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.263 7f5ffc147700 1 --2- 172.21.2.101:0/2777579740 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ff40d1870 0x7f5ff41b03f0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:19.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.263 7f5ffc147700 1 -- 172.21.2.101:0/2777579740 >> 172.21.2.101:0/2777579740 conn(0x7f5ff40ce450 msgr2=0x7f5ff40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:19.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.263 7f5ffc147700 1 -- 172.21.2.101:0/2777579740 shutdown_connections 2021-02-10T11:49:19.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.263 7f5ffc147700 1 -- 172.21.2.101:0/2777579740 wait complete. 2021-02-10T11:49:19.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.278 7f5ffc147700 1 Processor -- start 2021-02-10T11:49:19.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.278 7f5ffc147700 1 -- start start 2021-02-10T11:49:19.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.278 7f5ffc147700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ff4093060 0x7f5ff419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:19.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.278 7f5ffc147700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ff419aeb0 0x7f5ff40b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:19.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.278 7f5ffc147700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ff419b2b0 0x7f5ff40b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:19.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.278 7f5ffc147700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff40efd70 con 0x7f5ff419b2b0 2021-02-10T11:49:19.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.278 7f5ffc147700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff419b8a0 con 0x7f5ff4093060 2021-02-10T11:49:19.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.278 7f5ffc147700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff40b25d0 con 0x7f5ff419aeb0 2021-02-10T11:49:19.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.278 7f5ffa764700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ff419b2b0 0x7f5ff40b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:19.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.278 7f5ffa764700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ff419b2b0 0x7f5ff40b1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35726/0 (socket says 172.21.2.101:35726) 2021-02-10T11:49:19.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.278 7f5ffa764700 1 -- 172.21.2.101:0/114757396 learned_addr learned my addr 172.21.2.101:0/114757396 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:19.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.278 7f5ff9762700 1 --2- 172.21.2.101:0/114757396 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ff419aeb0 0x7f5ff40b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:19.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.279 7f5ff9f63700 1 --2- 172.21.2.101:0/114757396 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ff4093060 0x7f5ff419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:19.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.279 7f5ffa764700 1 -- 172.21.2.101:0/114757396 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ff419aeb0 msgr2=0x7f5ff40b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:19.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.279 7f5ffa764700 1 --2- 172.21.2.101:0/114757396 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ff419aeb0 0x7f5ff40b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:19.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.279 7f5ffa764700 1 -- 172.21.2.101:0/114757396 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ff4093060 msgr2=0x7f5ff419a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:19.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.279 7f5ffa764700 1 --2- 172.21.2.101:0/114757396 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ff4093060 0x7f5ff419a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:19.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.279 7f5ffa764700 1 -- 172.21.2.101:0/114757396 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff418e410 con 0x7f5ff419b2b0 2021-02-10T11:49:19.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.279 7f5ff9f63700 1 --2- 172.21.2.101:0/114757396 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ff4093060 0x7f5ff419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:49:19.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.279 7f5ff9762700 1 --2- 172.21.2.101:0/114757396 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ff419aeb0 0x7f5ff40b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:49:19.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.279 7f5ffa764700 1 --2- 172.21.2.101:0/114757396 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ff419b2b0 0x7f5ff40b1fb0 secure :-1 s=READY pgs=3368 cs=0 l=1 rev1=1 rx=0x7f5fec00a8f0 tx=0x7f5fec00ac90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:19.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.279 7f5fe67fc700 1 -- 172.21.2.101:0/114757396 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5fec000ee0 con 0x7f5ff419b2b0 2021-02-10T11:49:19.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.279 7f5fe67fc700 1 -- 172.21.2.101:0/114757396 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec007160 con 0x7f5ff419b2b0 2021-02-10T11:49:19.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.279 7f5ffc147700 1 -- 172.21.2.101:0/114757396 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff418e410 con 0x7f5ff419b2b0 2021-02-10T11:49:19.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.279 7f5fe67fc700 1 -- 172.21.2.101:0/114757396 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5fec007530 con 0x7f5ff419b2b0 2021-02-10T11:49:19.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.279 7f5ffc147700 1 -- 172.21.2.101:0/114757396 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff418e410 con 0x7f5ff419b2b0 2021-02-10T11:49:19.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.280 7f5fe67fc700 1 -- 172.21.2.101:0/114757396 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f5fec007340 con 0x7f5ff419b2b0 2021-02-10T11:49:19.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.280 7f5fe67fc700 1 --2- 172.21.2.101:0/114757396 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5fdc018730 0x7f5fdc01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:19.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.280 7f5fe67fc700 1 -- 172.21.2.101:0/114757396 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5fec0329b0 con 0x7f5ff419b2b0 2021-02-10T11:49:19.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.280 7f5ff9f63700 1 --2- 172.21.2.101:0/114757396 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5fdc018730 0x7f5fdc01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:19.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.280 7f5ff9f63700 1 --2- 172.21.2.101:0/114757396 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5fdc018730 0x7f5fdc01ac10 crc :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:19.306 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.310 7f5ffc147700 1 -- 172.21.2.101:0/114757396 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff40ebc00 con 0x7f5ff419b2b0 2021-02-10T11:49:19.309 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.313 7f5fe67fc700 1 -- 172.21.2.101:0/114757396 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f5fec007d50 con 0x7f5ff419b2b0 2021-02-10T11:49:19.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.439 7f5ffc147700 1 -- 172.21.2.101:0/114757396 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ff40ebc00 con 0x7f5ff419b2b0 2021-02-10T11:49:19.436 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.440 7f5fe67fc700 1 -- 172.21.2.101:0/114757396 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+14020 (secure 0 0 0) 0x7f5fec007160 con 0x7f5ff419b2b0 2021-02-10T11:49:19.436 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:49:19.436 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":198,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:49:18.830697","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":43,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":42,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:49:16.791877","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":43,"pool_name":"cephfs_data","create_time":"2021-02-10 11:49:17.820338","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:49:19.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.441 7f5ffc147700 1 -- 172.21.2.101:0/114757396 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5fdc018730 msgr2=0x7f5fdc01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:19.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.441 7f5ffc147700 1 --2- 172.21.2.101:0/114757396 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5fdc018730 0x7f5fdc01ac10 crc :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:19.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.441 7f5ffc147700 1 -- 172.21.2.101:0/114757396 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ff419b2b0 msgr2=0x7f5ff40b1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:19.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.441 7f5ffc147700 1 --2- 172.21.2.101:0/114757396 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ff419b2b0 0x7f5ff40b1fb0 secure :-1 s=READY pgs=3368 cs=0 l=1 rev1=1 rx=0x7f5fec00a8f0 tx=0x7f5fec00ac90).stop 2021-02-10T11:49:19.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.441 7f5ffc147700 1 -- 172.21.2.101:0/114757396 shutdown_connections 2021-02-10T11:49:19.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.441 7f5ffc147700 1 --2- 172.21.2.101:0/114757396 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f5fdc018730 0x7f5fdc01ac10 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:19.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.441 7f5ffc147700 1 --2- 172.21.2.101:0/114757396 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f5ff419aeb0 0x7f5ff40b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:19.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.441 7f5ffc147700 1 --2- 172.21.2.101:0/114757396 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f5ff419b2b0 0x7f5ff40b1fb0 unknown :-1 s=CLOSED pgs=3368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:19.438 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.441 7f5ffc147700 1 --2- 172.21.2.101:0/114757396 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f5ff4093060 0x7f5ff419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:19.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.441 7f5ffc147700 1 -- 172.21.2.101:0/114757396 >> 172.21.2.101:0/114757396 conn(0x7f5ff419f880 msgr2=0x7f5ff4199a80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:19.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.441 7f5ffc147700 1 -- 172.21.2.101:0/114757396 shutdown_connections 2021-02-10T11:49:19.439 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.441 7f5ffc147700 1 -- 172.21.2.101:0/114757396 wait complete. 2021-02-10T11:49:19.447 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:49:19.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.864 7f59d3b9d700 1 Processor -- start 2021-02-10T11:49:19.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.864 7f59d3b9d700 1 -- start start 2021-02-10T11:49:19.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.864 7f59d3b9d700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f59cc0d0340 0x7f59cc0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:19.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.864 7f59d3b9d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59cc0f0b10 con 0x7f59cc0d1870 2021-02-10T11:49:19.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.864 7f59d3b9d700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59cc1b0be0 con 0x7f59cc0d1c70 2021-02-10T11:49:19.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.864 7f59d3b9d700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59cc0e5ed0 con 0x7f59cc0d0340 2021-02-10T11:49:19.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.864 7f59d19b9700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f59cc0d1870 0x7f59cc1b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36224/0 (socket says 172.21.2.101:36224) 2021-02-10T11:49:19.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.864 7f59d19b9700 1 -- 172.21.2.101:0/3309320679 learned_addr learned my addr 172.21.2.101:0/3309320679 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:19.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.864 7f59d11b8700 1 --2- 172.21.2.101:0/3309320679 >> v2:172.21.2.108:3300/0 conn(0x7f59cc0d0340 0x7f59cc0ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:19.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d09b7700 1 -- 172.21.2.101:0/3309320679 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2450178326 0 0) 0x7f59c80010b0 con 0x7f59cc0d1870 2021-02-10T11:49:19.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d09b7700 1 -- 172.21.2.101:0/3309320679 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59b8002fe0 con 0x7f59cc0d1870 2021-02-10T11:49:19.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d09b7700 1 -- 172.21.2.101:0/3309320679 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3480280982 0 0) 0x7f59c80010b0 con 0x7f59cc0d1870 2021-02-10T11:49:19.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d09b7700 1 -- 172.21.2.101:0/3309320679 >> v2:172.21.2.108:3300/0 conn(0x7f59cc0d0340 msgr2=0x7f59cc0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:19.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d09b7700 1 --2- 172.21.2.101:0/3309320679 >> v2:172.21.2.108:3300/0 conn(0x7f59cc0d0340 0x7f59cc0ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:19.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d09b7700 1 -- 172.21.2.101:0/3309320679 >> v1:172.21.2.108:6789/0 conn(0x7f59cc0d1c70 legacy=0x7f59cc0efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:19.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d09b7700 1 -- 172.21.2.101:0/3309320679 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc0f0450 con 0x7f59cc0d1870 2021-02-10T11:49:19.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d09b7700 1 -- 172.21.2.101:0/3309320679 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f59c80015b0 con 0x7f59cc0d1870 2021-02-10T11:49:19.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d09b7700 1 -- 172.21.2.101:0/3309320679 >> v1:172.21.2.101:6789/0 conn(0x7f59cc0d1870 legacy=0x7f59cc1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:19.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d09b7700 1 --2- 172.21.2.101:0/3309320679 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59b8003aa0 0x7f59b8005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:19.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d09b7700 1 --2- 172.21.2.101:0/3309320679 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59b8006790 0x7f59b8008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:19.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d09b7700 1 --2- 172.21.2.101:0/3309320679 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f59b8009590 0x7f59b800bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:19.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d09b7700 1 -- 172.21.2.101:0/3309320679 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b800c300 con 0x7f59b8006790 2021-02-10T11:49:19.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d09b7700 1 -- 172.21.2.101:0/3309320679 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b800c520 con 0x7f59b8009590 2021-02-10T11:49:19.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d09b7700 1 -- 172.21.2.101:0/3309320679 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b8050750 con 0x7f59b8003aa0 2021-02-10T11:49:19.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d11b8700 1 --2- 172.21.2.101:0/3309320679 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59b8006790 0x7f59b8008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:19.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d19b9700 1 --2- 172.21.2.101:0/3309320679 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59b8003aa0 0x7f59b8005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:19.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d21ba700 1 --2- 172.21.2.101:0/3309320679 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f59b8009590 0x7f59b800bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:19.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d11b8700 1 -- 172.21.2.101:0/3309320679 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59b8003aa0 msgr2=0x7f59b8005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:19.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d11b8700 1 --2- 172.21.2.101:0/3309320679 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59b8003aa0 0x7f59b8005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:19.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d11b8700 1 -- 172.21.2.101:0/3309320679 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f59b8009590 msgr2=0x7f59b800bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:19.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d11b8700 1 --2- 172.21.2.101:0/3309320679 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f59b8009590 0x7f59b800bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:19.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d11b8700 1 -- 172.21.2.101:0/3309320679 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b8050980 con 0x7f59b8006790 2021-02-10T11:49:19.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.865 7f59d21ba700 1 --2- 172.21.2.101:0/3309320679 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f59b8009590 0x7f59b800bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:49:19.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.866 7f59d21ba700 1 -- 172.21.2.101:0/3309320679 reap_dead start 2021-02-10T11:49:19.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.866 7f59d11b8700 1 --2- 172.21.2.101:0/3309320679 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59b8006790 0x7f59b8008c90 secure :-1 s=READY pgs=3370 cs=0 l=1 rev1=1 rx=0x7f59c000c870 tx=0x7f59c000cc10).ready entity=mon.0 client_cookie=219a2693984f128 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:19.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.866 7f59d09b7700 1 -- 172.21.2.101:0/3309320679 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f59c00090d0 con 0x7f59b8006790 2021-02-10T11:49:19.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.866 7f59d09b7700 1 -- 172.21.2.101:0/3309320679 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c0009450 con 0x7f59b8006790 2021-02-10T11:49:19.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.866 7f59d09b7700 1 -- 172.21.2.101:0/3309320679 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f59c0009930 con 0x7f59b8006790 2021-02-10T11:49:19.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.866 7f59d3b9d700 1 -- 172.21.2.101:0/3309320679 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59b8006790 msgr2=0x7f59b8008c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:19.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.866 7f59d3b9d700 1 --2- 172.21.2.101:0/3309320679 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59b8006790 0x7f59b8008c90 secure :-1 s=READY pgs=3370 cs=0 l=1 rev1=1 rx=0x7f59c000c870 tx=0x7f59c000cc10).stop 2021-02-10T11:49:19.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.866 7f59d3b9d700 1 -- 172.21.2.101:0/3309320679 shutdown_connections 2021-02-10T11:49:19.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.866 7f59d3b9d700 1 --2- 172.21.2.101:0/3309320679 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59b8006790 0x7f59b8008c90 unknown :-1 s=CLOSED pgs=3370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:19.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.866 7f59d3b9d700 1 -- 172.21.2.101:0/3309320679 >> 172.21.2.101:0/3309320679 conn(0x7f59cc0ce450 msgr2=0x7f59cc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:19.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.867 7f59d3b9d700 1 -- 172.21.2.101:0/3309320679 shutdown_connections 2021-02-10T11:49:19.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.867 7f59d3b9d700 1 -- 172.21.2.101:0/3309320679 wait complete. 2021-02-10T11:49:19.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.881 7f59d3b9d700 1 Processor -- start 2021-02-10T11:49:19.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.881 7f59d3b9d700 1 -- start start 2021-02-10T11:49:19.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.882 7f59d3b9d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f59cc1942b0 0x7f59cc0b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:19.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.882 7f59d3b9d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59cc195070 0x7f59cc195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:19.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.882 7f59d3b9d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59cc0bb8e0 0x7f59cc0bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:19.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.882 7f59d3b9d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc1946b0 con 0x7f59cc0bb8e0 2021-02-10T11:49:19.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.882 7f59d3b9d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc0b3b70 con 0x7f59cc1942b0 2021-02-10T11:49:19.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.882 7f59d3b9d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc0b3d90 con 0x7f59cc195070 2021-02-10T11:49:19.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.882 7f59d21ba700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59cc0bb8e0 0x7f59cc0bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:19.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.882 7f59d21ba700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59cc0bb8e0 0x7f59cc0bbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35740/0 (socket says 172.21.2.101:35740) 2021-02-10T11:49:19.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.882 7f59d21ba700 1 -- 172.21.2.101:0/1268781842 learned_addr learned my addr 172.21.2.101:0/1268781842 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:19.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.882 7f59d11b8700 1 --2- 172.21.2.101:0/1268781842 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59cc195070 0x7f59cc195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:19.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.882 7f59d19b9700 1 --2- 172.21.2.101:0/1268781842 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f59cc1942b0 0x7f59cc0b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:19.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.882 7f59d11b8700 1 -- 172.21.2.101:0/1268781842 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f59cc1942b0 msgr2=0x7f59cc0b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:19.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.882 7f59d11b8700 1 --2- 172.21.2.101:0/1268781842 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f59cc1942b0 0x7f59cc0b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:19.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.882 7f59d11b8700 1 -- 172.21.2.101:0/1268781842 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59cc0bb8e0 msgr2=0x7f59cc0bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:19.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.882 7f59d11b8700 1 --2- 172.21.2.101:0/1268781842 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59cc0bb8e0 0x7f59cc0bbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:19.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.882 7f59d11b8700 1 -- 172.21.2.101:0/1268781842 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc1a49e0 con 0x7f59cc195070 2021-02-10T11:49:19.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.882 7f59d19b9700 1 --2- 172.21.2.101:0/1268781842 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f59cc1942b0 0x7f59cc0b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:49:19.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.882 7f59d21ba700 1 --2- 172.21.2.101:0/1268781842 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59cc0bb8e0 0x7f59cc0bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:49:19.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.882 7f59d11b8700 1 --2- 172.21.2.101:0/1268781842 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59cc195070 0x7f59cc195470 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f59c0002be0 tx=0x7f59c000b8e0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:19.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.883 7f59be7fc700 1 -- 172.21.2.101:0/1268781842 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f59c0002990 con 0x7f59cc195070 2021-02-10T11:49:19.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.883 7f59be7fc700 1 -- 172.21.2.101:0/1268781842 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c000bd60 con 0x7f59cc195070 2021-02-10T11:49:19.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.883 7f59be7fc700 1 -- 172.21.2.101:0/1268781842 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f59c0000d40 con 0x7f59cc195070 2021-02-10T11:49:19.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.883 7f59d3b9d700 1 -- 172.21.2.101:0/1268781842 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59cc1a49e0 con 0x7f59cc195070 2021-02-10T11:49:19.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.883 7f59d3b9d700 1 -- 172.21.2.101:0/1268781842 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59cc1a49e0 con 0x7f59cc195070 2021-02-10T11:49:19.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.883 7f59be7fc700 1 -- 172.21.2.101:0/1268781842 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f59c0000b50 con 0x7f59cc195070 2021-02-10T11:49:19.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.883 7f59be7fc700 1 --2- 172.21.2.101:0/1268781842 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f59b40186c0 0x7f59b401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:19.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.883 7f59be7fc700 1 -- 172.21.2.101:0/1268781842 <== mon.2 v2:172.21.2.108:3301/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f59c0032430 con 0x7f59cc195070 2021-02-10T11:49:19.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.883 7f59d19b9700 1 --2- 172.21.2.101:0/1268781842 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f59b40186c0 0x7f59b401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:19.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.884 7f59d19b9700 1 --2- 172.21.2.101:0/1268781842 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f59b40186c0 0x7f59b401abc0 crc :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:19.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.914 7f59d3b9d700 1 -- 172.21.2.101:0/1268781842 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59cc1a49e0 con 0x7f59cc195070 2021-02-10T11:49:19.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:19.917 7f59be7fc700 1 -- 172.21.2.101:0/1268781842 <== mon.2 v2:172.21.2.108:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f59c0000940 con 0x7f59cc195070 2021-02-10T11:49:20.039 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.043 7f59d3b9d700 1 -- 172.21.2.101:0/1268781842 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59cc1a49e0 con 0x7f59cc195070 2021-02-10T11:49:20.040 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.045 7f59be7fc700 1 -- 172.21.2.101:0/1268781842 <== mon.2 v2:172.21.2.108:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f59c0000b50 con 0x7f59cc195070 2021-02-10T11:49:20.042 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:49:20.042 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":199,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:49:19.833311","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":43,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":42,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:49:16.791877","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":43,"pool_name":"cephfs_data","create_time":"2021-02-10 11:49:17.820338","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:49:20.043 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.047 7f59d3b9d700 1 -- 172.21.2.101:0/1268781842 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f59b40186c0 msgr2=0x7f59b401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.047 7f59d3b9d700 1 --2- 172.21.2.101:0/1268781842 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f59b40186c0 0x7f59b401abc0 crc :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.047 7f59d3b9d700 1 -- 172.21.2.101:0/1268781842 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59cc195070 msgr2=0x7f59cc195470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.047 7f59d3b9d700 1 --2- 172.21.2.101:0/1268781842 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59cc195070 0x7f59cc195470 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f59c0002be0 tx=0x7f59c000b8e0).stop 2021-02-10T11:49:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.048 7f59d3b9d700 1 -- 172.21.2.101:0/1268781842 shutdown_connections 2021-02-10T11:49:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.048 7f59d3b9d700 1 --2- 172.21.2.101:0/1268781842 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f59b40186c0 0x7f59b401abc0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.048 7f59d3b9d700 1 --2- 172.21.2.101:0/1268781842 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59cc195070 0x7f59cc195470 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.048 7f59d3b9d700 1 --2- 172.21.2.101:0/1268781842 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59cc0bb8e0 0x7f59cc0bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.048 7f59d3b9d700 1 --2- 172.21.2.101:0/1268781842 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f59cc1942b0 0x7f59cc0b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:20.044 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.048 7f59d3b9d700 1 -- 172.21.2.101:0/1268781842 >> 172.21.2.101:0/1268781842 conn(0x7f59cc0b3040 msgr2=0x7f59cc0c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:20.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.048 7f59d3b9d700 1 -- 172.21.2.101:0/1268781842 shutdown_connections 2021-02-10T11:49:20.045 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.048 7f59d3b9d700 1 -- 172.21.2.101:0/1268781842 wait complete. 2021-02-10T11:49:20.053 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-02-10T11:49:20.490 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.473 7ff120129700 1 Processor -- start 2021-02-10T11:49:20.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.473 7ff120129700 1 -- start start 2021-02-10T11:49:20.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.474 7ff120129700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1180d1870 0x7ff1181b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:20.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.474 7ff120129700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7ff1181b0ae0 0x7ff118108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:20.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.474 7ff120129700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff118109a60 con 0x7ff1180d1c70 2021-02-10T11:49:20.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.474 7ff120129700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1180e5ef0 con 0x7ff1181b0ae0 2021-02-10T11:49:20.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.474 7ff120129700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff118109750 con 0x7ff1180d1870 2021-02-10T11:49:20.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.474 7ff11e746700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff1180d1c70 0x7ff118108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36242/0 (socket says 172.21.2.101:36242) 2021-02-10T11:49:20.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.474 7ff11e746700 1 -- 172.21.2.101:0/3295970856 learned_addr learned my addr 172.21.2.101:0/3295970856 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:20.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.474 7ff11cf43700 1 -- 172.21.2.101:0/3295970856 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3910698552 0 0) 0x7ff110001500 con 0x7ff1180d1c70 2021-02-10T11:49:20.492 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.474 7ff11cf43700 1 -- 172.21.2.101:0/3295970856 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff104002fe0 con 0x7ff1180d1c70 2021-02-10T11:49:20.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.474 7ff11d744700 1 --2- 172.21.2.101:0/3295970856 >> v2:172.21.2.108:3300/0 conn(0x7ff1181b0ae0 0x7ff118108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:20.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.474 7ff11df45700 1 --2- 172.21.2.101:0/3295970856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1180d1870 0x7ff1181b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:20.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.475 7ff11cf43700 1 -- 172.21.2.101:0/3295970856 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 476003192 0 0) 0x7ff110001500 con 0x7ff1180d1c70 2021-02-10T11:49:20.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.475 7ff11cf43700 1 -- 172.21.2.101:0/3295970856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1180d1870 msgr2=0x7ff1181b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:20.493 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.475 7ff11cf43700 1 --2- 172.21.2.101:0/3295970856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1180d1870 0x7ff1181b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:20.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.475 7ff11cf43700 1 -- 172.21.2.101:0/3295970856 >> v2:172.21.2.108:3300/0 conn(0x7ff1181b0ae0 msgr2=0x7ff118108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:20.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.475 7ff11cf43700 1 --2- 172.21.2.101:0/3295970856 >> v2:172.21.2.108:3300/0 conn(0x7ff1181b0ae0 0x7ff118108940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:20.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.475 7ff11cf43700 1 -- 172.21.2.101:0/3295970856 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff118088070 con 0x7ff1180d1c70 2021-02-10T11:49:20.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.475 7ff11cf43700 1 -- 172.21.2.101:0/3295970856 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7ff110001a00 con 0x7ff1180d1c70 2021-02-10T11:49:20.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.475 7ff11cf43700 1 -- 172.21.2.101:0/3295970856 >> v1:172.21.2.101:6789/0 conn(0x7ff1180d1c70 legacy=0x7ff118108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:20.494 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.475 7ff11cf43700 1 --2- 172.21.2.101:0/3295970856 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff104003a40 0x7ff104005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:20.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.475 7ff11cf43700 1 --2- 172.21.2.101:0/3295970856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff10404a790 0x7ff10404cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:20.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.475 7ff11cf43700 1 --2- 172.21.2.101:0/3295970856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff10404d590 0x7ff10404fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:20.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.475 7ff11cf43700 1 -- 172.21.2.101:0/3295970856 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1040502d0 con 0x7ff10404a790 2021-02-10T11:49:20.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.475 7ff11cf43700 1 -- 172.21.2.101:0/3295970856 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1040504f0 con 0x7ff104003a40 2021-02-10T11:49:20.495 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.475 7ff11cf43700 1 -- 172.21.2.101:0/3295970856 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff104050710 con 0x7ff10404d590 2021-02-10T11:49:20.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.475 7ff11d744700 1 --2- 172.21.2.101:0/3295970856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff10404a790 0x7ff10404cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:20.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.475 7ff11df45700 1 --2- 172.21.2.101:0/3295970856 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff104003a40 0x7ff104005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:20.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.475 7ff11d744700 1 -- 172.21.2.101:0/3295970856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff10404d590 msgr2=0x7ff10404fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:20.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.475 7ff11d744700 1 --2- 172.21.2.101:0/3295970856 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff10404d590 0x7ff10404fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:20.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.475 7ff11d744700 1 -- 172.21.2.101:0/3295970856 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff104003a40 msgr2=0x7ff104005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:20.496 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.475 7ff11d744700 1 --2- 172.21.2.101:0/3295970856 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff104003a40 0x7ff104005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:20.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.475 7ff11d744700 1 -- 172.21.2.101:0/3295970856 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1040509b0 con 0x7ff10404a790 2021-02-10T11:49:20.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.476 7ff11d744700 1 --2- 172.21.2.101:0/3295970856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff10404a790 0x7ff10404cc90 secure :-1 s=READY pgs=3372 cs=0 l=1 rev1=1 rx=0x7ff10c00c870 tx=0x7ff10c00cc10).ready entity=mon.0 client_cookie=ccb8b44a31a95ddf server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:20.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.476 7ff11cf43700 1 -- 172.21.2.101:0/3295970856 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff10c0090d0 con 0x7ff10404a790 2021-02-10T11:49:20.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.476 7ff11cf43700 1 -- 172.21.2.101:0/3295970856 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10c009350 con 0x7ff10404a790 2021-02-10T11:49:20.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.476 7ff11cf43700 1 -- 172.21.2.101:0/3295970856 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff10c009760 con 0x7ff10404a790 2021-02-10T11:49:20.497 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.476 7ff120129700 1 -- 172.21.2.101:0/3295970856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff10404a790 msgr2=0x7ff10404cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:20.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.476 7ff120129700 1 --2- 172.21.2.101:0/3295970856 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff10404a790 0x7ff10404cc90 secure :-1 s=READY pgs=3372 cs=0 l=1 rev1=1 rx=0x7ff10c00c870 tx=0x7ff10c00cc10).stop 2021-02-10T11:49:20.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.476 7ff11e746700 1 -- 172.21.2.101:0/3295970856 reap_dead start 2021-02-10T11:49:20.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.477 7ff120129700 1 -- 172.21.2.101:0/3295970856 shutdown_connections 2021-02-10T11:49:20.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.477 7ff120129700 1 -- 172.21.2.101:0/3295970856 >> 172.21.2.101:0/3295970856 conn(0x7ff1180ce420 msgr2=0x7ff1180d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:20.498 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.477 7ff120129700 1 -- 172.21.2.101:0/3295970856 shutdown_connections 2021-02-10T11:49:20.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.477 7ff120129700 1 -- 172.21.2.101:0/3295970856 wait complete. 2021-02-10T11:49:20.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.491 7ff120129700 1 Processor -- start 2021-02-10T11:49:20.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.491 7ff120129700 1 -- start start 2021-02-10T11:49:20.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.492 7ff120129700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1180c1850 0x7ff1180c1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:20.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.492 7ff120129700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1180c2810 0x7ff1180c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:20.499 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.492 7ff120129700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1180c3230 0x7ff1180c3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:20.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.492 7ff120129700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1180c1c50 con 0x7ff1180c3230 2021-02-10T11:49:20.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.492 7ff120129700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff118084920 con 0x7ff1180c2810 2021-02-10T11:49:20.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.492 7ff120129700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff118084b40 con 0x7ff1180c1850 2021-02-10T11:49:20.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.492 7ff11e746700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1180c3230 0x7ff1180c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:20.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.492 7ff11e746700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1180c3230 0x7ff1180c3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35760/0 (socket says 172.21.2.101:35760) 2021-02-10T11:49:20.500 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.492 7ff11e746700 1 -- 172.21.2.101:0/653011578 learned_addr learned my addr 172.21.2.101:0/653011578 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:20.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.492 7ff11e746700 1 -- 172.21.2.101:0/653011578 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1180c1850 msgr2=0x7ff1180c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:20.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.492 7ff11e746700 1 --2- 172.21.2.101:0/653011578 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1180c1850 0x7ff1180c1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:20.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.492 7ff11e746700 1 -- 172.21.2.101:0/653011578 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1180c2810 msgr2=0x7ff1180c2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:20.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.492 7ff11e746700 1 --2- 172.21.2.101:0/653011578 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1180c2810 0x7ff1180c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:20.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.492 7ff11e746700 1 -- 172.21.2.101:0/653011578 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff118084d60 con 0x7ff1180c3230 2021-02-10T11:49:20.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.492 7ff11e746700 1 --2- 172.21.2.101:0/653011578 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1180c3230 0x7ff1180c3630 secure :-1 s=READY pgs=3373 cs=0 l=1 rev1=1 rx=0x7ff110007f90 tx=0x7ff11000ac50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:20.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.493 7ff10a7fc700 1 -- 172.21.2.101:0/653011578 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff110000f20 con 0x7ff1180c3230 2021-02-10T11:49:20.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.493 7ff10a7fc700 1 -- 172.21.2.101:0/653011578 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff110007020 con 0x7ff1180c3230 2021-02-10T11:49:20.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.493 7ff10a7fc700 1 -- 172.21.2.101:0/653011578 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff1100074b0 con 0x7ff1180c3230 2021-02-10T11:49:20.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.493 7ff120129700 1 -- 172.21.2.101:0/653011578 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff118084d60 con 0x7ff1180c3230 2021-02-10T11:49:20.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.493 7ff120129700 1 -- 172.21.2.101:0/653011578 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff118084d60 con 0x7ff1180c3230 2021-02-10T11:49:20.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.494 7ff10a7fc700 1 -- 172.21.2.101:0/653011578 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff110007200 con 0x7ff1180c3230 2021-02-10T11:49:20.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.494 7ff10a7fc700 1 --2- 172.21.2.101:0/653011578 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff100018730 0x7ff10001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:20.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.494 7ff10a7fc700 1 -- 172.21.2.101:0/653011578 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff110032b80 con 0x7ff1180c3230 2021-02-10T11:49:20.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.494 7ff11df45700 1 --2- 172.21.2.101:0/653011578 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff100018730 0x7ff10001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:20.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.494 7ff11df45700 1 --2- 172.21.2.101:0/653011578 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff100018730 0x7ff10001ac10 crc :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:20.520 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.524 7ff120129700 1 -- 172.21.2.101:0/653011578 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff118109a60 con 0x7ff1180c3230 2021-02-10T11:49:20.522 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.526 7ff10a7fc700 1 -- 172.21.2.101:0/653011578 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff110007d00 con 0x7ff1180c3230 2021-02-10T11:49:20.658 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.662 7ff120129700 1 -- 172.21.2.101:0/653011578 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"} v 0) v1 -- 0x7ff118109a60 con 0x7ff1180c3230 2021-02-10T11:49:20.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.841 7ff10a7fc700 1 -- 172.21.2.101:0/653011578 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "standby_count_wanted", "prefix": "fs set", "fs_name": "cephfs", "val": "0"}]=0 v172) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff110000d10 con 0x7ff1180c3230 2021-02-10T11:49:20.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.880 7ff120129700 1 -- 172.21.2.101:0/653011578 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff100018730 msgr2=0x7ff10001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:20.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.880 7ff120129700 1 --2- 172.21.2.101:0/653011578 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff100018730 0x7ff10001ac10 crc :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:20.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.880 7ff120129700 1 -- 172.21.2.101:0/653011578 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1180c3230 msgr2=0x7ff1180c3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:20.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.880 7ff120129700 1 --2- 172.21.2.101:0/653011578 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1180c3230 0x7ff1180c3630 secure :-1 s=READY pgs=3373 cs=0 l=1 rev1=1 rx=0x7ff110007f90 tx=0x7ff11000ac50).stop 2021-02-10T11:49:20.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.880 7ff120129700 1 -- 172.21.2.101:0/653011578 shutdown_connections 2021-02-10T11:49:20.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.880 7ff120129700 1 --2- 172.21.2.101:0/653011578 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff1180c3230 0x7ff1180c3630 unknown :-1 s=CLOSED pgs=3373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:20.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.880 7ff120129700 1 --2- 172.21.2.101:0/653011578 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff1180c2810 0x7ff1180c2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:20.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.880 7ff120129700 1 --2- 172.21.2.101:0/653011578 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff100018730 0x7ff10001ac10 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:20.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.880 7ff120129700 1 --2- 172.21.2.101:0/653011578 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff1180c1850 0x7ff1180c1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:20.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.880 7ff120129700 1 -- 172.21.2.101:0/653011578 >> 172.21.2.101:0/653011578 conn(0x7ff1180af690 msgr2=0x7ff118094c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:20.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.880 7ff120129700 1 -- 172.21.2.101:0/653011578 shutdown_connections 2021-02-10T11:49:20.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:20.881 7ff120129700 1 -- 172.21.2.101:0/653011578 wait complete. 2021-02-10T11:49:20.885 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:49:21.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.303 7f59d680b700 1 Processor -- start 2021-02-10T11:49:21.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.303 7f59d680b700 1 -- start start 2021-02-10T11:49:21.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.303 7f59d680b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59d00d1870 0x7f59d01b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:21.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.303 7f59d680b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d00f0b10 con 0x7f59d00f0650 2021-02-10T11:49:21.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.303 7f59d680b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d00d0310 con 0x7f59d01b0a10 2021-02-10T11:49:21.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.304 7f59d680b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d00e5ed0 con 0x7f59d00d1870 2021-02-10T11:49:21.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.304 7f59d4e28700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f59d00f0650 0x7f59d00cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36264/0 (socket says 172.21.2.101:36264) 2021-02-10T11:49:21.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.304 7f59d4e28700 1 -- 172.21.2.101:0/3123580235 learned_addr learned my addr 172.21.2.101:0/3123580235 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:21.317 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.304 7f59ceffd700 1 -- 172.21.2.101:0/3123580235 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198835318 0 0) 0x7f59c8001500 con 0x7f59d00f0650 2021-02-10T11:49:21.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.304 7f59ceffd700 1 -- 172.21.2.101:0/3123580235 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59b8002fe0 con 0x7f59d00f0650 2021-02-10T11:49:21.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.304 7f59cffff700 1 --2- 172.21.2.101:0/3123580235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59d00d1870 0x7f59d01b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:21.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59ceffd700 1 -- 172.21.2.101:0/3123580235 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3717223719 0 0) 0x7f59c8001500 con 0x7f59d00f0650 2021-02-10T11:49:21.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59ceffd700 1 -- 172.21.2.101:0/3123580235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59d00d1870 msgr2=0x7f59d01b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:21.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59ceffd700 1 --2- 172.21.2.101:0/3123580235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59d00d1870 0x7f59d01b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:21.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59ceffd700 1 -- 172.21.2.101:0/3123580235 >> v1:172.21.2.108:6789/0 conn(0x7f59d01b0a10 legacy=0x7f59d00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:21.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59ceffd700 1 -- 172.21.2.101:0/3123580235 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d00d0530 con 0x7f59d00f0650 2021-02-10T11:49:21.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59ceffd700 1 -- 172.21.2.101:0/3123580235 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f59c8001a00 con 0x7f59d00f0650 2021-02-10T11:49:21.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59ceffd700 1 -- 172.21.2.101:0/3123580235 >> v1:172.21.2.101:6789/0 conn(0x7f59d00f0650 legacy=0x7f59d00cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:21.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59ceffd700 1 --2- 172.21.2.101:0/3123580235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59b8003aa0 0x7f59b8005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:21.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59ceffd700 1 --2- 172.21.2.101:0/3123580235 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f59b8006790 0x7f59b8008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:21.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59ceffd700 1 --2- 172.21.2.101:0/3123580235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59b8009590 0x7f59b800ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:21.318 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59ceffd700 1 -- 172.21.2.101:0/3123580235 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b800c2d0 con 0x7f59b8003aa0 2021-02-10T11:49:21.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59ceffd700 1 -- 172.21.2.101:0/3123580235 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b800c4f0 con 0x7f59b8006790 2021-02-10T11:49:21.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59ceffd700 1 -- 172.21.2.101:0/3123580235 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b8050720 con 0x7f59b8009590 2021-02-10T11:49:21.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59cffff700 1 --2- 172.21.2.101:0/3123580235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59b8003aa0 0x7f59b8005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:21.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59d4e28700 1 --2- 172.21.2.101:0/3123580235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59b8009590 0x7f59b800ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:21.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59cffff700 1 -- 172.21.2.101:0/3123580235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59b8009590 msgr2=0x7f59b800ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:21.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59cffff700 1 --2- 172.21.2.101:0/3123580235 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59b8009590 0x7f59b800ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:21.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59cffff700 1 -- 172.21.2.101:0/3123580235 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f59b8006790 msgr2=0x7f59b8008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:21.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59cffff700 1 --2- 172.21.2.101:0/3123580235 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f59b8006790 0x7f59b8008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:21.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59cffff700 1 -- 172.21.2.101:0/3123580235 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b8050980 con 0x7f59b8003aa0 2021-02-10T11:49:21.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59cffff700 1 --2- 172.21.2.101:0/3123580235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59b8003aa0 0x7f59b8005ef0 secure :-1 s=READY pgs=3375 cs=0 l=1 rev1=1 rx=0x7f59c0006ac0 tx=0x7f59c0009ab0).ready entity=mon.0 client_cookie=11e7969caf03f2c1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:21.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59ceffd700 1 -- 172.21.2.101:0/3123580235 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f59c000c0c0 con 0x7f59b8003aa0 2021-02-10T11:49:21.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59ceffd700 1 -- 172.21.2.101:0/3123580235 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c000c3c0 con 0x7f59b8003aa0 2021-02-10T11:49:21.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59ceffd700 1 -- 172.21.2.101:0/3123580235 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f59c000c8e0 con 0x7f59b8003aa0 2021-02-10T11:49:21.319 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59d680b700 1 -- 172.21.2.101:0/3123580235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59b8003aa0 msgr2=0x7f59b8005ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:21.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59d680b700 1 --2- 172.21.2.101:0/3123580235 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59b8003aa0 0x7f59b8005ef0 secure :-1 s=READY pgs=3375 cs=0 l=1 rev1=1 rx=0x7f59c0006ac0 tx=0x7f59c0009ab0).stop 2021-02-10T11:49:21.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.305 7f59d4e28700 1 -- 172.21.2.101:0/3123580235 reap_dead start 2021-02-10T11:49:21.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.306 7f59d680b700 1 -- 172.21.2.101:0/3123580235 shutdown_connections 2021-02-10T11:49:21.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.306 7f59d680b700 1 -- 172.21.2.101:0/3123580235 >> 172.21.2.101:0/3123580235 conn(0x7f59d00ce450 msgr2=0x7f59d00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:21.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.306 7f59d680b700 1 -- 172.21.2.101:0/3123580235 shutdown_connections 2021-02-10T11:49:21.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.306 7f59d680b700 1 -- 172.21.2.101:0/3123580235 wait complete. 2021-02-10T11:49:21.320 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.321 7f59d680b700 1 Processor -- start 2021-02-10T11:49:21.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.321 7f59d680b700 1 -- start start 2021-02-10T11:49:21.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.321 7f59d680b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59d009bb70 0x7f59d00b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:21.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.321 7f59d680b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f59d00b55e0 0x7f59d00c1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:21.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.321 7f59d680b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59d00b59e0 0x7f59d00c1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:21.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.321 7f59d680b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d00b6010 con 0x7f59d00b59e0 2021-02-10T11:49:21.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.321 7f59d680b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d00b6230 con 0x7f59d00b55e0 2021-02-10T11:49:21.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.321 7f59d680b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d009bf70 con 0x7f59d009bb70 2021-02-10T11:49:21.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.321 7f59d4e28700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59d00b59e0 0x7f59d00c1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:21.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.321 7f59d4e28700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59d00b59e0 0x7f59d00c1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35780/0 (socket says 172.21.2.101:35780) 2021-02-10T11:49:21.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.322 7f59d4e28700 1 -- 172.21.2.101:0/3441120581 learned_addr learned my addr 172.21.2.101:0/3441120581 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:21.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.322 7f59cf7fe700 1 --2- 172.21.2.101:0/3441120581 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f59d00b55e0 0x7f59d00c1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:21.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.322 7f59cffff700 1 --2- 172.21.2.101:0/3441120581 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59d009bb70 0x7f59d00b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:21.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.322 7f59d4e28700 1 -- 172.21.2.101:0/3441120581 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59d009bb70 msgr2=0x7f59d00b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:21.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.322 7f59d4e28700 1 --2- 172.21.2.101:0/3441120581 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59d009bb70 0x7f59d00b4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:21.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.322 7f59d4e28700 1 -- 172.21.2.101:0/3441120581 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f59d00b55e0 msgr2=0x7f59d00c1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:21.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.322 7f59d4e28700 1 --2- 172.21.2.101:0/3441120581 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f59d00b55e0 0x7f59d00c1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:21.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.322 7f59d4e28700 1 -- 172.21.2.101:0/3441120581 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d00c1fb0 con 0x7f59d00b59e0 2021-02-10T11:49:21.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.322 7f59cffff700 1 --2- 172.21.2.101:0/3441120581 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59d009bb70 0x7f59d00b4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:49:21.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.322 7f59d4e28700 1 --2- 172.21.2.101:0/3441120581 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59d00b59e0 0x7f59d00c1990 secure :-1 s=READY pgs=3376 cs=0 l=1 rev1=1 rx=0x7f59c8007520 tx=0x7f59c800bcc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:21.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.322 7f59ccff9700 1 -- 172.21.2.101:0/3441120581 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f59c8007bf0 con 0x7f59d00b59e0 2021-02-10T11:49:21.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.323 7f59ccff9700 1 -- 172.21.2.101:0/3441120581 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c8006080 con 0x7f59d00b59e0 2021-02-10T11:49:21.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.323 7f59ccff9700 1 -- 172.21.2.101:0/3441120581 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f59c80064d0 con 0x7f59d00b59e0 2021-02-10T11:49:21.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.323 7f59d680b700 1 -- 172.21.2.101:0/3441120581 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d00c1fb0 con 0x7f59d00b59e0 2021-02-10T11:49:21.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.323 7f59d680b700 1 -- 172.21.2.101:0/3441120581 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d00c2540 con 0x7f59d00b59e0 2021-02-10T11:49:21.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.323 7f59ccff9700 1 -- 172.21.2.101:0/3441120581 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f59c8007bf0 con 0x7f59d00b59e0 2021-02-10T11:49:21.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.323 7f59ccff9700 1 --2- 172.21.2.101:0/3441120581 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f59b00186c0 0x7f59b001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:21.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.323 7f59ccff9700 1 -- 172.21.2.101:0/3441120581 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f59c8032950 con 0x7f59d00b59e0 2021-02-10T11:49:21.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.324 7f59cffff700 1 --2- 172.21.2.101:0/3441120581 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f59b00186c0 0x7f59b001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:21.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.324 7f59cffff700 1 --2- 172.21.2.101:0/3441120581 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f59b00186c0 0x7f59b001abc0 crc :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:21.350 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.354 7f59d680b700 1 -- 172.21.2.101:0/3441120581 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d00c0e40 con 0x7f59d00b59e0 2021-02-10T11:49:21.352 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.356 7f59ccff9700 1 -- 172.21.2.101:0/3441120581 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f59c8007bf0 con 0x7f59d00b59e0 2021-02-10T11:49:21.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.485 7f59d680b700 1 -- 172.21.2.101:0/3441120581 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59d00c0e40 con 0x7f59d00b59e0 2021-02-10T11:49:21.482 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.486 7f59ccff9700 1 -- 172.21.2.101:0/3441120581 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) v1 ==== 95+0+2108 (secure 0 0 0) 0x7f59c8007ef0 con 0x7f59d00b59e0 2021-02-10T11:49:21.483 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:49:21.483 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":172,"default_fscid":21,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":13902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1207472796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1207472796},{"type":"v1","addr":"172.21.2.101:6835","nonce":1207472796}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":168}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:49:18.830789","modified":"2021-02-10 11:49:20.837623","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9100":{"gid":9100,"name":"b","rank":0,"incarnation":169,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/1721728696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":1721728696},{"type":"v1","addr":"172.21.2.108:6833","nonce":1721728696}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[43],"metadata_pool":42,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":21}]} 2021-02-10T11:49:21.483 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 172 2021-02-10T11:49:21.483 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.488 7f59d680b700 1 -- 172.21.2.101:0/3441120581 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f59b00186c0 msgr2=0x7f59b001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:21.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.488 7f59d680b700 1 --2- 172.21.2.101:0/3441120581 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f59b00186c0 0x7f59b001abc0 crc :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:21.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.488 7f59d680b700 1 -- 172.21.2.101:0/3441120581 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59d00b59e0 msgr2=0x7f59d00c1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:21.484 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.488 7f59d680b700 1 --2- 172.21.2.101:0/3441120581 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59d00b59e0 0x7f59d00c1990 secure :-1 s=READY pgs=3376 cs=0 l=1 rev1=1 rx=0x7f59c8007520 tx=0x7f59c800bcc0).stop 2021-02-10T11:49:21.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.488 7f59d680b700 1 -- 172.21.2.101:0/3441120581 shutdown_connections 2021-02-10T11:49:21.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.488 7f59d680b700 1 --2- 172.21.2.101:0/3441120581 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f59d00b59e0 0x7f59d00c1990 unknown :-1 s=CLOSED pgs=3376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:21.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.488 7f59d680b700 1 --2- 172.21.2.101:0/3441120581 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f59d00b55e0 0x7f59d00c1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:21.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.488 7f59d680b700 1 --2- 172.21.2.101:0/3441120581 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f59b00186c0 0x7f59b001abc0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:21.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.488 7f59d680b700 1 --2- 172.21.2.101:0/3441120581 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f59d009bb70 0x7f59d00b4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:21.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.488 7f59d680b700 1 -- 172.21.2.101:0/3441120581 >> 172.21.2.101:0/3441120581 conn(0x7f59d00ac990 msgr2=0x7f59d0091b30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:21.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.488 7f59d680b700 1 -- 172.21.2.101:0/3441120581 shutdown_connections 2021-02-10T11:49:21.485 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.488 7f59d680b700 1 -- 172.21.2.101:0/3441120581 wait complete. 2021-02-10T11:49:21.493 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:49:21.923 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.911 7fc5f3c8d700 1 Processor -- start 2021-02-10T11:49:21.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.911 7fc5f3c8d700 1 -- start start 2021-02-10T11:49:21.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.911 7fc5f3c8d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc5ec0d0340 0x7fc5ec0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:21.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.911 7fc5f3c8d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5ec0d1c70 0x7fc5ec0efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:21.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.912 7fc5f3c8d700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5ec0f0b10 con 0x7fc5ec0d1870 2021-02-10T11:49:21.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.912 7fc5f3c8d700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ec0e5ed0 con 0x7fc5ec0d0340 2021-02-10T11:49:21.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.912 7fc5f3c8d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec1b0be0 con 0x7fc5ec0d1c70 2021-02-10T11:49:21.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.912 7fc5f12a8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc5ec0d0340 0x7fc5ec0ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:21.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.912 7fc5f12a8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc5ec0d0340 0x7fc5ec0ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35786/0 (socket says 172.21.2.101:35786) 2021-02-10T11:49:21.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.912 7fc5f12a8700 1 -- 172.21.2.101:0/592782155 learned_addr learned my addr 172.21.2.101:0/592782155 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:21.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.912 7fc5f12a8700 1 -- 172.21.2.101:0/592782155 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5ec0d1c70 msgr2=0x7fc5ec0efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:21.924 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.912 7fc5f12a8700 1 --2- 172.21.2.101:0/592782155 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5ec0d1c70 0x7fc5ec0efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:21.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.912 7fc5f12a8700 1 -- 172.21.2.101:0/592782155 >> v1:172.21.2.101:6789/0 conn(0x7fc5ec0d1870 legacy=0x7fc5ec1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:21.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.912 7fc5f12a8700 1 -- 172.21.2.101:0/592782155 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec0f20a0 con 0x7fc5ec0d0340 2021-02-10T11:49:21.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.912 7fc5f12a8700 1 --2- 172.21.2.101:0/592782155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5ec0d0340 0x7fc5ec0ef640 secure :-1 s=READY pgs=3378 cs=0 l=1 rev1=1 rx=0x7fc5e8006700 tx=0x7fc5e8009b80).ready entity=mon.0 client_cookie=3bbc439c9b05a2f5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:21.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.912 7fc5f0aa7700 1 -- 172.21.2.101:0/592782155 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc5e8006340 con 0x7fc5ec0d0340 2021-02-10T11:49:21.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.912 7fc5f0aa7700 1 -- 172.21.2.101:0/592782155 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e800c0c0 con 0x7fc5ec0d0340 2021-02-10T11:49:21.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.912 7fc5f0aa7700 1 -- 172.21.2.101:0/592782155 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc5e800c5a0 con 0x7fc5ec0d0340 2021-02-10T11:49:21.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.913 7fc5f3c8d700 1 -- 172.21.2.101:0/592782155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5ec0d0340 msgr2=0x7fc5ec0ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:21.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.913 7fc5f3c8d700 1 --2- 172.21.2.101:0/592782155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5ec0d0340 0x7fc5ec0ef640 secure :-1 s=READY pgs=3378 cs=0 l=1 rev1=1 rx=0x7fc5e8006700 tx=0x7fc5e8009b80).stop 2021-02-10T11:49:21.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.913 7fc5f3c8d700 1 -- 172.21.2.101:0/592782155 shutdown_connections 2021-02-10T11:49:21.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.913 7fc5f3c8d700 1 --2- 172.21.2.101:0/592782155 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5ec0d1c70 0x7fc5ec0efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:21.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.913 7fc5f3c8d700 1 --2- 172.21.2.101:0/592782155 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5ec0d0340 0x7fc5ec0ef640 unknown :-1 s=CLOSED pgs=3378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:21.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.913 7fc5f3c8d700 1 -- 172.21.2.101:0/592782155 >> 172.21.2.101:0/592782155 conn(0x7fc5ec0ce450 msgr2=0x7fc5ec0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:21.925 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.913 7fc5f3c8d700 1 -- 172.21.2.101:0/592782155 shutdown_connections 2021-02-10T11:49:21.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.913 7fc5f3c8d700 1 -- 172.21.2.101:0/592782155 wait complete. 2021-02-10T11:49:21.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.928 7fc5f3c8d700 1 Processor -- start 2021-02-10T11:49:21.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.928 7fc5f3c8d700 1 -- start start 2021-02-10T11:49:21.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.928 7fc5f3c8d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5ec0b4f30 0x7fc5ec18b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:21.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.928 7fc5f3c8d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc5ec093f40 0x7fc5ec094340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:21.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.928 7fc5f3c8d700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5ec094960 0x7fc5ec187e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:21.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.928 7fc5f3c8d700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec094e90 con 0x7fc5ec0b4f30 2021-02-10T11:49:21.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.928 7fc5f3c8d700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec18bff0 con 0x7fc5ec093f40 2021-02-10T11:49:21.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.928 7fc5f3c8d700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec08f780 con 0x7fc5ec094960 2021-02-10T11:49:21.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.929 7fc5f1aa9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5ec0b4f30 0x7fc5ec18b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:21.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.929 7fc5f1aa9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5ec0b4f30 0x7fc5ec18b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35794/0 (socket says 172.21.2.101:35794) 2021-02-10T11:49:21.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.929 7fc5f1aa9700 1 -- 172.21.2.101:0/3621965677 learned_addr learned my addr 172.21.2.101:0/3621965677 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:21.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.929 7fc5f1aa9700 1 -- 172.21.2.101:0/3621965677 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5ec094960 msgr2=0x7fc5ec187e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:21.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.929 7fc5f1aa9700 1 --2- 172.21.2.101:0/3621965677 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5ec094960 0x7fc5ec187e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:21.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.929 7fc5f1aa9700 1 -- 172.21.2.101:0/3621965677 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc5ec093f40 msgr2=0x7fc5ec094340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:21.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.929 7fc5f1aa9700 1 --2- 172.21.2.101:0/3621965677 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc5ec093f40 0x7fc5ec094340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:21.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.929 7fc5f1aa9700 1 -- 172.21.2.101:0/3621965677 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec188470 con 0x7fc5ec0b4f30 2021-02-10T11:49:21.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.929 7fc5f1aa9700 1 --2- 172.21.2.101:0/3621965677 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5ec0b4f30 0x7fc5ec18b9d0 secure :-1 s=READY pgs=3379 cs=0 l=1 rev1=1 rx=0x7fc5e0006ed0 tx=0x7fc5e000b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:21.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.929 7fc5de7fc700 1 -- 172.21.2.101:0/3621965677 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc5e0006020 con 0x7fc5ec0b4f30 2021-02-10T11:49:21.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.929 7fc5de7fc700 1 -- 172.21.2.101:0/3621965677 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e0006360 con 0x7fc5ec0b4f30 2021-02-10T11:49:21.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.929 7fc5f3c8d700 1 -- 172.21.2.101:0/3621965677 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ec188470 con 0x7fc5ec0b4f30 2021-02-10T11:49:21.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.929 7fc5de7fc700 1 -- 172.21.2.101:0/3621965677 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc5e0005730 con 0x7fc5ec0b4f30 2021-02-10T11:49:21.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.930 7fc5f3c8d700 1 -- 172.21.2.101:0/3621965677 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ec188470 con 0x7fc5ec0b4f30 2021-02-10T11:49:21.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.930 7fc5de7fc700 1 -- 172.21.2.101:0/3621965677 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc5e0006020 con 0x7fc5ec0b4f30 2021-02-10T11:49:21.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.930 7fc5de7fc700 1 --2- 172.21.2.101:0/3621965677 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5d4018730 0x7fc5d401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:21.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.930 7fc5de7fc700 1 -- 172.21.2.101:0/3621965677 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc5e0032840 con 0x7fc5ec0b4f30 2021-02-10T11:49:21.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.930 7fc5f12a8700 1 --2- 172.21.2.101:0/3621965677 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5d4018730 0x7fc5d401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:21.929 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.931 7fc5f12a8700 1 --2- 172.21.2.101:0/3621965677 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5d4018730 0x7fc5d401ac10 crc :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:21.957 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.961 7fc5f3c8d700 1 -- 172.21.2.101:0/3621965677 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ec0ce450 con 0x7fc5ec0b4f30 2021-02-10T11:49:21.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:21.964 7fc5de7fc700 1 -- 172.21.2.101:0/3621965677 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc5e0006000 con 0x7fc5ec0b4f30 2021-02-10T11:49:22.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.087 7fc5f3c8d700 1 -- 172.21.2.101:0/3621965677 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5ec0ce450 con 0x7fc5ec0b4f30 2021-02-10T11:49:22.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.088 7fc5de7fc700 1 -- 172.21.2.101:0/3621965677 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) v1 ==== 74+0+14007 (secure 0 0 0) 0x7fc5e0006360 con 0x7fc5ec0b4f30 2021-02-10T11:49:22.084 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:49:22.085 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":199,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:49:19.833311","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":43,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":42,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:49:16.791877","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":43,"pool_name":"cephfs_data","create_time":"2021-02-10 11:49:17.820338","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:49:22.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.090 7fc5f3c8d700 1 -- 172.21.2.101:0/3621965677 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5d4018730 msgr2=0x7fc5d401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:22.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.090 7fc5f3c8d700 1 --2- 172.21.2.101:0/3621965677 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5d4018730 0x7fc5d401ac10 crc :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:22.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.090 7fc5f3c8d700 1 -- 172.21.2.101:0/3621965677 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5ec0b4f30 msgr2=0x7fc5ec18b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:22.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.090 7fc5f3c8d700 1 --2- 172.21.2.101:0/3621965677 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5ec0b4f30 0x7fc5ec18b9d0 secure :-1 s=READY pgs=3379 cs=0 l=1 rev1=1 rx=0x7fc5e0006ed0 tx=0x7fc5e000b960).stop 2021-02-10T11:49:22.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.090 7fc5f3c8d700 1 -- 172.21.2.101:0/3621965677 shutdown_connections 2021-02-10T11:49:22.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.090 7fc5f3c8d700 1 --2- 172.21.2.101:0/3621965677 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc5d4018730 0x7fc5d401ac10 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:22.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.090 7fc5f3c8d700 1 --2- 172.21.2.101:0/3621965677 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc5ec094960 0x7fc5ec187e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:22.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.090 7fc5f3c8d700 1 --2- 172.21.2.101:0/3621965677 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc5ec093f40 0x7fc5ec094340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:22.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.090 7fc5f3c8d700 1 --2- 172.21.2.101:0/3621965677 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc5ec0b4f30 0x7fc5ec18b9d0 unknown :-1 s=CLOSED pgs=3379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:22.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.090 7fc5f3c8d700 1 -- 172.21.2.101:0/3621965677 >> 172.21.2.101:0/3621965677 conn(0x7fc5ec19f8f0 msgr2=0x7fc5ec1a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:22.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.090 7fc5f3c8d700 1 -- 172.21.2.101:0/3621965677 shutdown_connections 2021-02-10T11:49:22.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.090 7fc5f3c8d700 1 -- 172.21.2.101:0/3621965677 wait complete. 2021-02-10T11:49:22.095 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:49:22.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.525 7fb30505c700 1 Processor -- start 2021-02-10T11:49:22.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.525 7fb30505c700 1 -- start start 2021-02-10T11:49:22.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.525 7fb30505c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb3000d1880 0x7fb3001b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:22.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.525 7fb30505c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3000fe1d0 con 0x7fb3000cff80 2021-02-10T11:49:22.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.525 7fb30505c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb300109a70 con 0x7fb3001b0af0 2021-02-10T11:49:22.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.525 7fb30505c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb300088070 con 0x7fb3000d1880 2021-02-10T11:49:22.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.526 7fb2ff59e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb3000cff80 0x7fb300109140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36298/0 (socket says 172.21.2.101:36298) 2021-02-10T11:49:22.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.526 7fb2ff59e700 1 -- 172.21.2.101:0/4277368984 learned_addr learned my addr 172.21.2.101:0/4277368984 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:22.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.526 7fb2fed9d700 1 --2- 172.21.2.101:0/4277368984 >> v2:172.21.2.108:3300/0 conn(0x7fb3000d1880 0x7fb3001b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:22.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.526 7fb2fdd9b700 1 -- 172.21.2.101:0/4277368984 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2282157569 0 0) 0x7fb2f8001500 con 0x7fb3000cff80 2021-02-10T11:49:22.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.526 7fb2fdd9b700 1 -- 172.21.2.101:0/4277368984 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2ec002fe0 con 0x7fb3000cff80 2021-02-10T11:49:22.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.526 7fb2fdd9b700 1 -- 172.21.2.101:0/4277368984 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 783380723 0 0) 0x7fb2f0000ff0 con 0x7fb3001b0af0 2021-02-10T11:49:22.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.526 7fb2fdd9b700 1 -- 172.21.2.101:0/4277368984 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2ec003210 con 0x7fb3001b0af0 2021-02-10T11:49:22.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.526 7fb2fed9d700 1 -- 172.21.2.101:0/4277368984 >> v1:172.21.2.108:6789/0 conn(0x7fb3001b0af0 legacy=0x7fb300108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:22.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.526 7fb2fed9d700 1 -- 172.21.2.101:0/4277368984 >> v1:172.21.2.101:6789/0 conn(0x7fb3000cff80 legacy=0x7fb300109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:22.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.526 7fb2fed9d700 1 -- 172.21.2.101:0/4277368984 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb300088260 con 0x7fb3000d1880 2021-02-10T11:49:22.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.526 7fb2fed9d700 1 --2- 172.21.2.101:0/4277368984 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb3000d1880 0x7fb3001b04d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7fb2e8006650 tx=0x7fb2e8009a80).ready entity=mon.1 client_cookie=f4a00d461d12754b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:22.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.527 7fb2fdd9b700 1 -- 172.21.2.101:0/4277368984 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb2e800c0c0 con 0x7fb3000d1880 2021-02-10T11:49:22.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.527 7fb2fdd9b700 1 -- 172.21.2.101:0/4277368984 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e800c3c0 con 0x7fb3000d1880 2021-02-10T11:49:22.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.527 7fb2fdd9b700 1 -- 172.21.2.101:0/4277368984 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb2e800c820 con 0x7fb3000d1880 2021-02-10T11:49:22.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.527 7fb30505c700 1 -- 172.21.2.101:0/4277368984 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb3000d1880 msgr2=0x7fb3001b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:22.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.527 7fb30505c700 1 --2- 172.21.2.101:0/4277368984 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb3000d1880 0x7fb3001b04d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7fb2e8006650 tx=0x7fb2e8009a80).stop 2021-02-10T11:49:22.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.527 7fb30505c700 1 -- 172.21.2.101:0/4277368984 shutdown_connections 2021-02-10T11:49:22.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.527 7fb30505c700 1 --2- 172.21.2.101:0/4277368984 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb3000d1880 0x7fb3001b04d0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:22.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.527 7fb30505c700 1 -- 172.21.2.101:0/4277368984 >> 172.21.2.101:0/4277368984 conn(0x7fb3000ce430 msgr2=0x7fb3000d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:22.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.528 7fb30505c700 1 -- 172.21.2.101:0/4277368984 shutdown_connections 2021-02-10T11:49:22.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.528 7fb30505c700 1 -- 172.21.2.101:0/4277368984 wait complete. 2021-02-10T11:49:22.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.542 7fb30505c700 1 Processor -- start 2021-02-10T11:49:22.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.542 7fb30505c700 1 -- start start 2021-02-10T11:49:22.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.543 7fb30505c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb300086f40 0x7fb300082bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:22.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.543 7fb30505c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb3001aa960 0x7fb3000a8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:22.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.543 7fb30505c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb300083500 0x7fb300093800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:22.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.543 7fb30505c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb300083ac0 con 0x7fb300083500 2021-02-10T11:49:22.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.543 7fb30505c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3000831f0 con 0x7fb3001aa960 2021-02-10T11:49:22.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.543 7fb30505c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3000a8850 con 0x7fb300086f40 2021-02-10T11:49:22.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.543 7fb2ff59e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb300083500 0x7fb300093800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:22.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.543 7fb2ff59e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb300083500 0x7fb300093800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35810/0 (socket says 172.21.2.101:35810) 2021-02-10T11:49:22.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.543 7fb2ff59e700 1 -- 172.21.2.101:0/4275002959 learned_addr learned my addr 172.21.2.101:0/4275002959 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:22.543 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.543 7fb2fed9d700 1 --2- 172.21.2.101:0/4275002959 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb300086f40 0x7fb300082bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:22.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.543 7fb2fe59c700 1 --2- 172.21.2.101:0/4275002959 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb3001aa960 0x7fb3000a8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:22.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.543 7fb2ff59e700 1 -- 172.21.2.101:0/4275002959 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb300086f40 msgr2=0x7fb300082bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:22.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.543 7fb2ff59e700 1 --2- 172.21.2.101:0/4275002959 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb300086f40 0x7fb300082bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:22.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.543 7fb2ff59e700 1 -- 172.21.2.101:0/4275002959 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb3001aa960 msgr2=0x7fb3000a8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:22.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.543 7fb2ff59e700 1 --2- 172.21.2.101:0/4275002959 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb3001aa960 0x7fb3000a8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:22.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.543 7fb2ff59e700 1 -- 172.21.2.101:0/4275002959 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3000a8a70 con 0x7fb300083500 2021-02-10T11:49:22.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.543 7fb2fed9d700 1 --2- 172.21.2.101:0/4275002959 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb300086f40 0x7fb300082bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:49:22.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.544 7fb2ff59e700 1 --2- 172.21.2.101:0/4275002959 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb300083500 0x7fb300093800 secure :-1 s=READY pgs=3381 cs=0 l=1 rev1=1 rx=0x7fb2f8009980 tx=0x7fb2f800bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:22.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.544 7fb2f77fe700 1 -- 172.21.2.101:0/4275002959 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb2f800be10 con 0x7fb300083500 2021-02-10T11:49:22.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.544 7fb2f77fe700 1 -- 172.21.2.101:0/4275002959 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f8006080 con 0x7fb300083500 2021-02-10T11:49:22.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.544 7fb2f77fe700 1 -- 172.21.2.101:0/4275002959 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb2f8004720 con 0x7fb300083500 2021-02-10T11:49:22.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.544 7fb30505c700 1 -- 172.21.2.101:0/4275002959 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3000a8a70 con 0x7fb300083500 2021-02-10T11:49:22.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.544 7fb30505c700 1 -- 172.21.2.101:0/4275002959 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3000a8a70 con 0x7fb300083500 2021-02-10T11:49:22.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.544 7fb2f77fe700 1 -- 172.21.2.101:0/4275002959 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb2f8006260 con 0x7fb300083500 2021-02-10T11:49:22.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.544 7fb2f77fe700 1 --2- 172.21.2.101:0/4275002959 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2e40186d0 0x7fb2e401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:22.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.544 7fb2f77fe700 1 -- 172.21.2.101:0/4275002959 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb2f80329b0 con 0x7fb300083500 2021-02-10T11:49:22.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.545 7fb2fed9d700 1 --2- 172.21.2.101:0/4275002959 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2e40186d0 0x7fb2e401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:22.545 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.545 7fb2fed9d700 1 --2- 172.21.2.101:0/4275002959 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2e40186d0 0x7fb2e401abd0 crc :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:22.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.575 7fb30505c700 1 -- 172.21.2.101:0/4275002959 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3000826b0 con 0x7fb300083500 2021-02-10T11:49:22.574 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.578 7fb2f77fe700 1 -- 172.21.2.101:0/4275002959 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb2f8006290 con 0x7fb300083500 2021-02-10T11:49:22.706 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.709 7fb30505c700 1 -- 172.21.2.101:0/4275002959 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fb3000826b0 con 0x7fb300083500 2021-02-10T11:49:22.709 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.713 7fb2f77fe700 1 -- 172.21.2.101:0/4275002959 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v120) v1 ==== 184+0+0 (secure 0 0 0) 0x7fb2f8000d30 con 0x7fb300083500 2021-02-10T11:49:22.713 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.0 2021-02-10T11:49:22.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.718 7fb30505c700 1 -- 172.21.2.101:0/4275002959 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2e40186d0 msgr2=0x7fb2e401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:22.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.718 7fb30505c700 1 --2- 172.21.2.101:0/4275002959 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2e40186d0 0x7fb2e401abd0 crc :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:22.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.718 7fb30505c700 1 -- 172.21.2.101:0/4275002959 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb300083500 msgr2=0x7fb300093800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:22.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.718 7fb30505c700 1 --2- 172.21.2.101:0/4275002959 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb300083500 0x7fb300093800 secure :-1 s=READY pgs=3381 cs=0 l=1 rev1=1 rx=0x7fb2f8009980 tx=0x7fb2f800bcd0).stop 2021-02-10T11:49:22.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.718 7fb30505c700 1 -- 172.21.2.101:0/4275002959 shutdown_connections 2021-02-10T11:49:22.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.719 7fb30505c700 1 --2- 172.21.2.101:0/4275002959 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb300083500 0x7fb300093800 unknown :-1 s=CLOSED pgs=3381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:22.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.719 7fb30505c700 1 --2- 172.21.2.101:0/4275002959 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb3001aa960 0x7fb3000a8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:22.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.719 7fb30505c700 1 --2- 172.21.2.101:0/4275002959 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2e40186d0 0x7fb2e401abd0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:22.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.719 7fb30505c700 1 --2- 172.21.2.101:0/4275002959 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb300086f40 0x7fb300082bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:22.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.719 7fb30505c700 1 -- 172.21.2.101:0/4275002959 >> 172.21.2.101:0/4275002959 conn(0x7fb30019d4f0 msgr2=0x7fb3001b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:22.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.719 7fb30505c700 1 -- 172.21.2.101:0/4275002959 shutdown_connections 2021-02-10T11:49:22.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:22.719 7fb30505c700 1 -- 172.21.2.101:0/4275002959 wait complete. 2021-02-10T11:49:22.723 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:49:23.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.151 7ff63827c700 1 Processor -- start 2021-02-10T11:49:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.151 7ff63827c700 1 -- start start 2021-02-10T11:49:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.152 7ff63827c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6300d1880 0x7ff6301b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.152 7ff63827c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff6301b0af0 0x7ff630108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.152 7ff63827c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff630109a70 con 0x7ff6300d1c80 2021-02-10T11:49:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.152 7ff63827c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6300e5f00 con 0x7ff6301b0af0 2021-02-10T11:49:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.152 7ff63827c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff630109760 con 0x7ff6300d1880 2021-02-10T11:49:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.152 7ff635897700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff6301b0af0 0x7ff630108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.152 7ff636899700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff6300d1c80 0x7ff630108f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36308/0 (socket says 172.21.2.101:36308) 2021-02-10T11:49:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.152 7ff636899700 1 -- 172.21.2.101:0/1811279156 learned_addr learned my addr 172.21.2.101:0/1811279156 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.152 7ff635897700 1 -- 172.21.2.101:0/1811279156 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6300d1880 msgr2=0x7ff6301b04d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:49:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.152 7ff635897700 1 --2- 172.21.2.101:0/1811279156 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6300d1880 0x7ff6301b04d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:23.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.152 7ff635897700 1 -- 172.21.2.101:0/1811279156 >> v1:172.21.2.101:6789/0 conn(0x7ff6300d1c80 legacy=0x7ff630108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.152 7ff635897700 1 -- 172.21.2.101:0/1811279156 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff630088070 con 0x7ff6301b0af0 2021-02-10T11:49:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.152 7ff635897700 1 --2- 172.21.2.101:0/1811279156 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6301b0af0 0x7ff630108950 secure :-1 s=READY pgs=3383 cs=0 l=1 rev1=1 rx=0x7ff620006650 tx=0x7ff620009a00).ready entity=mon.0 client_cookie=236a2e9512885db2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.152 7ff635096700 1 -- 172.21.2.101:0/1811279156 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff6200009b0 con 0x7ff6301b0af0 2021-02-10T11:49:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.153 7ff635096700 1 -- 172.21.2.101:0/1811279156 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62000c0c0 con 0x7ff6301b0af0 2021-02-10T11:49:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.153 7ff635096700 1 -- 172.21.2.101:0/1811279156 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff62000c5a0 con 0x7ff6301b0af0 2021-02-10T11:49:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.153 7ff63827c700 1 -- 172.21.2.101:0/1811279156 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6301b0af0 msgr2=0x7ff630108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.153 7ff63827c700 1 --2- 172.21.2.101:0/1811279156 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6301b0af0 0x7ff630108950 secure :-1 s=READY pgs=3383 cs=0 l=1 rev1=1 rx=0x7ff620006650 tx=0x7ff620009a00).stop 2021-02-10T11:49:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.153 7ff63827c700 1 -- 172.21.2.101:0/1811279156 shutdown_connections 2021-02-10T11:49:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.153 7ff63827c700 1 --2- 172.21.2.101:0/1811279156 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6301b0af0 0x7ff630108950 unknown :-1 s=CLOSED pgs=3383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.153 7ff63827c700 1 --2- 172.21.2.101:0/1811279156 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff6300d1880 0x7ff6301b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.153 7ff63827c700 1 -- 172.21.2.101:0/1811279156 >> 172.21.2.101:0/1811279156 conn(0x7ff6300ce430 msgr2=0x7ff6300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.153 7ff63827c700 1 -- 172.21.2.101:0/1811279156 shutdown_connections 2021-02-10T11:49:23.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.153 7ff63827c700 1 -- 172.21.2.101:0/1811279156 wait complete. 2021-02-10T11:49:23.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.168 7ff63827c700 1 Processor -- start 2021-02-10T11:49:23.167 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.168 7ff63827c700 1 -- start start 2021-02-10T11:49:23.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.168 7ff63827c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff630086810 0x7ff63019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:23.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.168 7ff63827c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6300b04d0 0x7ff630097450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:23.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.168 7ff63827c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6300b08d0 0x7ff630097a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:23.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.168 7ff63827c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6301a02c0 con 0x7ff6300b04d0 2021-02-10T11:49:23.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.168 7ff63827c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff630098090 con 0x7ff6300b08d0 2021-02-10T11:49:23.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.168 7ff63827c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6300982b0 con 0x7ff630086810 2021-02-10T11:49:23.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.168 7ff635897700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6300b04d0 0x7ff630097450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:23.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.168 7ff635897700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6300b04d0 0x7ff630097450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35818/0 (socket says 172.21.2.101:35818) 2021-02-10T11:49:23.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.169 7ff635897700 1 -- 172.21.2.101:0/1451813499 learned_addr learned my addr 172.21.2.101:0/1451813499 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:23.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.169 7ff636098700 1 --2- 172.21.2.101:0/1451813499 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff630086810 0x7ff63019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:23.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.169 7ff636899700 1 --2- 172.21.2.101:0/1451813499 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6300b08d0 0x7ff630097a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:23.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.169 7ff635897700 1 -- 172.21.2.101:0/1451813499 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff630086810 msgr2=0x7ff63019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:23.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.169 7ff635897700 1 --2- 172.21.2.101:0/1451813499 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff630086810 0x7ff63019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:23.168 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.169 7ff635897700 1 -- 172.21.2.101:0/1451813499 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6300b08d0 msgr2=0x7ff630097a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:23.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.169 7ff635897700 1 --2- 172.21.2.101:0/1451813499 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6300b08d0 0x7ff630097a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:23.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.169 7ff635897700 1 -- 172.21.2.101:0/1451813499 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6300984d0 con 0x7ff6300b04d0 2021-02-10T11:49:23.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.169 7ff635897700 1 --2- 172.21.2.101:0/1451813499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6300b04d0 0x7ff630097450 secure :-1 s=READY pgs=3384 cs=0 l=1 rev1=1 rx=0x7ff62000b5e0 tx=0x7ff62000b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:23.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.169 7ff626ffd700 1 -- 172.21.2.101:0/1451813499 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff620006390 con 0x7ff6300b04d0 2021-02-10T11:49:23.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.169 7ff626ffd700 1 -- 172.21.2.101:0/1451813499 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff620002030 con 0x7ff6300b04d0 2021-02-10T11:49:23.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.169 7ff63827c700 1 -- 172.21.2.101:0/1451813499 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6300984d0 con 0x7ff6300b04d0 2021-02-10T11:49:23.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.169 7ff63827c700 1 -- 172.21.2.101:0/1451813499 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff630098b80 con 0x7ff6300b04d0 2021-02-10T11:49:23.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.169 7ff626ffd700 1 -- 172.21.2.101:0/1451813499 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff620002360 con 0x7ff6300b04d0 2021-02-10T11:49:23.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.170 7ff626ffd700 1 -- 172.21.2.101:0/1451813499 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff620002210 con 0x7ff6300b04d0 2021-02-10T11:49:23.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.170 7ff626ffd700 1 --2- 172.21.2.101:0/1451813499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff618018730 0x7ff61801ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:23.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.170 7ff626ffd700 1 -- 172.21.2.101:0/1451813499 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff620032620 con 0x7ff6300b04d0 2021-02-10T11:49:23.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.170 7ff636098700 1 --2- 172.21.2.101:0/1451813499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff618018730 0x7ff61801ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:23.169 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.170 7ff636098700 1 --2- 172.21.2.101:0/1451813499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff618018730 0x7ff61801ac10 crc :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:23.197 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.201 7ff63827c700 1 -- 172.21.2.101:0/1451813499 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6300fe1d0 con 0x7ff6300b04d0 2021-02-10T11:49:23.200 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.204 7ff626ffd700 1 -- 172.21.2.101:0/1451813499 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff620002620 con 0x7ff6300b04d0 2021-02-10T11:49:23.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.334 7ff63827c700 1 -- 172.21.2.101:0/1451813499 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ff6300fe1d0 con 0x7ff6300b04d0 2021-02-10T11:49:23.334 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.338 7ff626ffd700 1 -- 172.21.2.101:0/1451813499 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v121) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff620002030 con 0x7ff6300b04d0 2021-02-10T11:49:23.338 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.1 2021-02-10T11:49:23.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.343 7ff63827c700 1 -- 172.21.2.101:0/1451813499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff618018730 msgr2=0x7ff61801ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:23.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.343 7ff63827c700 1 --2- 172.21.2.101:0/1451813499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff618018730 0x7ff61801ac10 crc :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:23.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.343 7ff63827c700 1 -- 172.21.2.101:0/1451813499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6300b04d0 msgr2=0x7ff630097450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:23.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.343 7ff63827c700 1 --2- 172.21.2.101:0/1451813499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6300b04d0 0x7ff630097450 secure :-1 s=READY pgs=3384 cs=0 l=1 rev1=1 rx=0x7ff62000b5e0 tx=0x7ff62000b980).stop 2021-02-10T11:49:23.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.343 7ff63827c700 1 -- 172.21.2.101:0/1451813499 shutdown_connections 2021-02-10T11:49:23.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.343 7ff63827c700 1 --2- 172.21.2.101:0/1451813499 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff6300b08d0 0x7ff630097a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:23.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.343 7ff63827c700 1 --2- 172.21.2.101:0/1451813499 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff6300b04d0 0x7ff630097450 unknown :-1 s=CLOSED pgs=3384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:23.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.343 7ff63827c700 1 --2- 172.21.2.101:0/1451813499 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff618018730 0x7ff61801ac10 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:23.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.343 7ff63827c700 1 --2- 172.21.2.101:0/1451813499 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff630086810 0x7ff63019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:23.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.343 7ff63827c700 1 -- 172.21.2.101:0/1451813499 >> 172.21.2.101:0/1451813499 conn(0x7ff63019f880 msgr2=0x7ff63008f890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:23.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.344 7ff63827c700 1 -- 172.21.2.101:0/1451813499 shutdown_connections 2021-02-10T11:49:23.340 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.344 7ff63827c700 1 -- 172.21.2.101:0/1451813499 wait complete. 2021-02-10T11:49:23.348 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:49:23.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.778 7f1411b6b700 1 Processor -- start 2021-02-10T11:49:23.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.778 7f1411b6b700 1 -- start start 2021-02-10T11:49:23.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.778 7f1411b6b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f140c0d1880 0x7f140c1b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:23.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.778 7f1411b6b700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f140c0cff80 0x7f140c109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:23.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.778 7f1411b6b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f140c109a70 con 0x7f140c1b0af0 2021-02-10T11:49:23.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.778 7f1411b6b700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f140c109760 con 0x7f140c0cff80 2021-02-10T11:49:23.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.778 7f1411b6b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f140c088070 con 0x7f140c0d1880 2021-02-10T11:49:23.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.779 7f140bfff700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f140c0cff80 0x7f140c109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:23.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.779 7f140b7fe700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f140c0d1880 0x7f140c1b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:23.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.779 7f140affd700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f140c1b0af0 0x7f140c108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:57962/0 (socket says 172.21.2.101:57962) 2021-02-10T11:49:23.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.779 7f140affd700 1 -- 172.21.2.101:0/2949201961 learned_addr learned my addr 172.21.2.101:0/2949201961 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:23.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.780 7f140bfff700 1 -- 172.21.2.101:0/2949201961 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f140c0d1880 msgr2=0x7f140c1b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:23.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.780 7f140bfff700 1 --2- 172.21.2.101:0/2949201961 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f140c0d1880 0x7f140c1b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:23.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.780 7f140bfff700 1 -- 172.21.2.101:0/2949201961 >> v1:172.21.2.108:6789/0 conn(0x7f140c1b0af0 legacy=0x7f140c108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:23.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.780 7f140a7fc700 1 -- 172.21.2.101:0/2949201961 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 692846705 0 0) 0x7f13fc0010b0 con 0x7f140c1b0af0 2021-02-10T11:49:23.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.780 7f140bfff700 1 -- 172.21.2.101:0/2949201961 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140c088260 con 0x7f140c0cff80 2021-02-10T11:49:23.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.780 7f140b7fe700 1 --2- 172.21.2.101:0/2949201961 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f140c0d1880 0x7f140c1b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:49:23.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.780 7f140bfff700 1 --2- 172.21.2.101:0/2949201961 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f140c0cff80 0x7f140c109140 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f1404006e50 tx=0x7f1404009e70).ready entity=mon.1 client_cookie=4eff90671cc5ca67 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:23.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.780 7f140a7fc700 1 -- 172.21.2.101:0/2949201961 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f140400b510 con 0x7f140c0cff80 2021-02-10T11:49:23.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.780 7f140a7fc700 1 -- 172.21.2.101:0/2949201961 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1404006790 con 0x7f140c0cff80 2021-02-10T11:49:23.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.780 7f140a7fc700 1 -- 172.21.2.101:0/2949201961 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f140400b940 con 0x7f140c0cff80 2021-02-10T11:49:23.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.781 7f1411b6b700 1 -- 172.21.2.101:0/2949201961 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f140c0cff80 msgr2=0x7f140c109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:23.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.781 7f1411b6b700 1 --2- 172.21.2.101:0/2949201961 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f140c0cff80 0x7f140c109140 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f1404006e50 tx=0x7f1404009e70).stop 2021-02-10T11:49:23.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.781 7f1411b6b700 1 -- 172.21.2.101:0/2949201961 shutdown_connections 2021-02-10T11:49:23.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.781 7f1411b6b700 1 --2- 172.21.2.101:0/2949201961 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f140c0cff80 0x7f140c109140 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:23.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.781 7f1411b6b700 1 --2- 172.21.2.101:0/2949201961 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f140c0d1880 0x7f140c1b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:23.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.781 7f1411b6b700 1 -- 172.21.2.101:0/2949201961 >> 172.21.2.101:0/2949201961 conn(0x7f140c0ce430 msgr2=0x7f140c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:23.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.781 7f1411b6b700 1 -- 172.21.2.101:0/2949201961 shutdown_connections 2021-02-10T11:49:23.794 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.781 7f1411b6b700 1 -- 172.21.2.101:0/2949201961 wait complete. 2021-02-10T11:49:23.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.796 7f1411b6b700 1 Processor -- start 2021-02-10T11:49:23.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.796 7f1411b6b700 1 -- start start 2021-02-10T11:49:23.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.796 7f1411b6b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f140c0b5040 0x7f140c08b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:23.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.796 7f1411b6b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f140c0b5440 0x7f140c0beec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:23.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.796 7f1411b6b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f140c08bf50 0x7f140c0bf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:23.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.796 7f1411b6b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140c0c0110 con 0x7f140c08bf50 2021-02-10T11:49:23.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.796 7f1411b6b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140c0bfb00 con 0x7f140c0b5440 2021-02-10T11:49:23.795 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.796 7f1411b6b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f140c0bfce0 con 0x7f140c0b5040 2021-02-10T11:49:23.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.796 7f140bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f140c08bf50 0x7f140c0bf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:23.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.796 7f140bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f140c08bf50 0x7f140c0bf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35828/0 (socket says 172.21.2.101:35828) 2021-02-10T11:49:23.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.796 7f140bfff700 1 -- 172.21.2.101:0/3442741351 learned_addr learned my addr 172.21.2.101:0/3442741351 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:23.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.797 7f140bfff700 1 -- 172.21.2.101:0/3442741351 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f140c0b5040 msgr2=0x7f140c08b930 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:49:23.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.797 7f140bfff700 1 --2- 172.21.2.101:0/3442741351 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f140c0b5040 0x7f140c08b930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:23.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.797 7f140bfff700 1 -- 172.21.2.101:0/3442741351 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f140c0b5440 msgr2=0x7f140c0beec0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:49:23.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.797 7f140bfff700 1 --2- 172.21.2.101:0/3442741351 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f140c0b5440 0x7f140c0beec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:23.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.797 7f140bfff700 1 -- 172.21.2.101:0/3442741351 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140c08a480 con 0x7f140c08bf50 2021-02-10T11:49:23.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.797 7f140bfff700 1 --2- 172.21.2.101:0/3442741351 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f140c08bf50 0x7f140c0bf4e0 secure :-1 s=READY pgs=3385 cs=0 l=1 rev1=1 rx=0x7f1404006c50 tx=0x7f140400bd20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:23.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.797 7f1403fff700 1 -- 172.21.2.101:0/3442741351 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f14040093d0 con 0x7f140c08bf50 2021-02-10T11:49:23.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.797 7f1403fff700 1 -- 172.21.2.101:0/3442741351 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140400be60 con 0x7f140c08bf50 2021-02-10T11:49:23.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.797 7f1403fff700 1 -- 172.21.2.101:0/3442741351 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f140400c570 con 0x7f140c08bf50 2021-02-10T11:49:23.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.797 7f1411b6b700 1 -- 172.21.2.101:0/3442741351 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f140c08a480 con 0x7f140c08bf50 2021-02-10T11:49:23.796 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.797 7f1411b6b700 1 -- 172.21.2.101:0/3442741351 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f140c08aa10 con 0x7f140c08bf50 2021-02-10T11:49:23.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.798 7f1403fff700 1 -- 172.21.2.101:0/3442741351 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f14040093d0 con 0x7f140c08bf50 2021-02-10T11:49:23.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.798 7f1403fff700 1 --2- 172.21.2.101:0/3442741351 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13f00186c0 0x7f13f001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:23.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.798 7f1403fff700 1 -- 172.21.2.101:0/3442741351 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1404032790 con 0x7f140c08bf50 2021-02-10T11:49:23.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.798 7f140b7fe700 1 --2- 172.21.2.101:0/3442741351 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13f00186c0 0x7f13f001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:23.797 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.798 7f140b7fe700 1 --2- 172.21.2.101:0/3442741351 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13f00186c0 0x7f13f001abc0 crc :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:23.825 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.828 7f1411b6b700 1 -- 172.21.2.101:0/3442741351 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f140c109a70 con 0x7f140c08bf50 2021-02-10T11:49:23.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.831 7f1403fff700 1 -- 172.21.2.101:0/3442741351 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f1404002ef0 con 0x7f140c08bf50 2021-02-10T11:49:23.960 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.964 7f1411b6b700 1 -- 172.21.2.101:0/3442741351 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f140c109a70 con 0x7f140c08bf50 2021-02-10T11:49:23.963 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.967 7f1403fff700 1 -- 172.21.2.101:0/3442741351 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v122) v1 ==== 184+0+0 (secure 0 0 0) 0x7f14040056b0 con 0x7f140c08bf50 2021-02-10T11:49:23.968 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.2 2021-02-10T11:49:23.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.973 7f1411b6b700 1 -- 172.21.2.101:0/3442741351 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13f00186c0 msgr2=0x7f13f001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:23.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.973 7f1411b6b700 1 --2- 172.21.2.101:0/3442741351 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13f00186c0 0x7f13f001abc0 crc :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:23.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.973 7f1411b6b700 1 -- 172.21.2.101:0/3442741351 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f140c08bf50 msgr2=0x7f140c0bf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:23.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.973 7f1411b6b700 1 --2- 172.21.2.101:0/3442741351 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f140c08bf50 0x7f140c0bf4e0 secure :-1 s=READY pgs=3385 cs=0 l=1 rev1=1 rx=0x7f1404006c50 tx=0x7f140400bd20).stop 2021-02-10T11:49:23.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.973 7f1411b6b700 1 -- 172.21.2.101:0/3442741351 shutdown_connections 2021-02-10T11:49:23.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.973 7f1411b6b700 1 --2- 172.21.2.101:0/3442741351 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f140c08bf50 0x7f140c0bf4e0 unknown :-1 s=CLOSED pgs=3385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:23.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.973 7f1411b6b700 1 --2- 172.21.2.101:0/3442741351 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f140c0b5440 0x7f140c0beec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:23.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.973 7f1411b6b700 1 --2- 172.21.2.101:0/3442741351 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f13f00186c0 0x7f13f001abc0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:23.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.973 7f1411b6b700 1 --2- 172.21.2.101:0/3442741351 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f140c0b5040 0x7f140c08b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:23.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.973 7f1411b6b700 1 -- 172.21.2.101:0/3442741351 >> 172.21.2.101:0/3442741351 conn(0x7f140c0a17f0 msgr2=0x7f140c0b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:23.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.973 7f1411b6b700 1 -- 172.21.2.101:0/3442741351 shutdown_connections 2021-02-10T11:49:23.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:23.973 7f1411b6b700 1 -- 172.21.2.101:0/3442741351 wait complete. 2021-02-10T11:49:23.977 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-02-10T11:49:24.411 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.397 7f83844c8700 1 Processor -- start 2021-02-10T11:49:24.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.397 7f83844c8700 1 -- start start 2021-02-10T11:49:24.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.398 7f83844c8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f837c0d1c80 0x7f837c0cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:24.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.398 7f83844c8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f837c0cf590 0x7f837c0cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:24.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.398 7f83844c8700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f837c0fe1d0 con 0x7f837c0d1880 2021-02-10T11:49:24.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.398 7f83844c8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f837c1b0cc0 con 0x7f837c0d1c80 2021-02-10T11:49:24.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.398 7f83844c8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f837c109a70 con 0x7f837c0cf590 2021-02-10T11:49:24.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.398 7f8381ae3700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f837c0d1c80 0x7f837c0cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:24.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.398 7f83822e4700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f837c0d1880 0x7f837c1b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36330/0 (socket says 172.21.2.101:36330) 2021-02-10T11:49:24.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.398 7f83822e4700 1 -- 172.21.2.101:0/717153045 learned_addr learned my addr 172.21.2.101:0/717153045 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:24.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.399 7f83812e2700 1 -- 172.21.2.101:0/717153045 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 173649873 0 0) 0x7f8378000ce0 con 0x7f837c0d1880 2021-02-10T11:49:24.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.399 7f83812e2700 1 -- 172.21.2.101:0/717153045 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8368002fe0 con 0x7f837c0d1880 2021-02-10T11:49:24.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.399 7f8381ae3700 1 -- 172.21.2.101:0/717153045 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f837c0cf590 msgr2=0x7f837c0cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:24.412 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.399 7f8381ae3700 1 --2- 172.21.2.101:0/717153045 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f837c0cf590 0x7f837c0cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:24.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.399 7f8381ae3700 1 -- 172.21.2.101:0/717153045 >> v1:172.21.2.101:6789/0 conn(0x7f837c0d1880 legacy=0x7f837c1b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:24.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.399 7f8381ae3700 1 -- 172.21.2.101:0/717153045 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837c088070 con 0x7f837c0d1c80 2021-02-10T11:49:24.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.399 7f8381ae3700 1 --2- 172.21.2.101:0/717153045 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f837c0d1c80 0x7f837c0cef70 secure :-1 s=READY pgs=3387 cs=0 l=1 rev1=1 rx=0x7f836c006650 tx=0x7f836c009a80).ready entity=mon.0 client_cookie=b508732fa767b00 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:24.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.399 7f83812e2700 1 -- 172.21.2.101:0/717153045 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f836c0009b0 con 0x7f837c0d1c80 2021-02-10T11:49:24.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.399 7f83812e2700 1 -- 172.21.2.101:0/717153045 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f836c00c0c0 con 0x7f837c0d1c80 2021-02-10T11:49:24.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.399 7f83812e2700 1 -- 172.21.2.101:0/717153045 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f836c00c520 con 0x7f837c0d1c80 2021-02-10T11:49:24.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.399 7f83844c8700 1 -- 172.21.2.101:0/717153045 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f837c0d1c80 msgr2=0x7f837c0cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:24.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.399 7f83844c8700 1 --2- 172.21.2.101:0/717153045 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f837c0d1c80 0x7f837c0cef70 secure :-1 s=READY pgs=3387 cs=0 l=1 rev1=1 rx=0x7f836c006650 tx=0x7f836c009a80).stop 2021-02-10T11:49:24.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.400 7f83844c8700 1 -- 172.21.2.101:0/717153045 shutdown_connections 2021-02-10T11:49:24.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.400 7f83844c8700 1 --2- 172.21.2.101:0/717153045 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f837c0cf590 0x7f837c0cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:24.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.400 7f83844c8700 1 --2- 172.21.2.101:0/717153045 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f837c0d1c80 0x7f837c0cef70 unknown :-1 s=CLOSED pgs=3387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:24.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.400 7f83844c8700 1 -- 172.21.2.101:0/717153045 >> 172.21.2.101:0/717153045 conn(0x7f837c0ce430 msgr2=0x7f837c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:24.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.400 7f83844c8700 1 -- 172.21.2.101:0/717153045 shutdown_connections 2021-02-10T11:49:24.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.400 7f83844c8700 1 -- 172.21.2.101:0/717153045 wait complete. 2021-02-10T11:49:24.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.415 7f83844c8700 1 Processor -- start 2021-02-10T11:49:24.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.415 7f83844c8700 1 -- start start 2021-02-10T11:49:24.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.415 7f83844c8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f837c1a0580 0x7f837c086fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:24.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.415 7f83844c8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f837c0b66b0 0x7f837c0b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:24.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.415 7f83844c8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f837c082bb0 0x7f837c082fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:24.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.415 7f83844c8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837c0875d0 con 0x7f837c0b66b0 2021-02-10T11:49:24.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.415 7f83844c8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837c0835d0 con 0x7f837c1a0580 2021-02-10T11:49:24.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.415 7f83844c8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f837c0837f0 con 0x7f837c082bb0 2021-02-10T11:49:24.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.415 7f8381ae3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f837c0b66b0 0x7f837c0b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:24.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.415 7f8381ae3700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f837c0b66b0 0x7f837c0b6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35844/0 (socket says 172.21.2.101:35844) 2021-02-10T11:49:24.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.415 7f8381ae3700 1 -- 172.21.2.101:0/255901717 learned_addr learned my addr 172.21.2.101:0/255901717 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:24.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.416 7f8381ae3700 1 -- 172.21.2.101:0/255901717 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f837c082bb0 msgr2=0x7f837c082fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:49:24.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.416 7f8381ae3700 1 --2- 172.21.2.101:0/255901717 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f837c082bb0 0x7f837c082fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:24.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.416 7f8381ae3700 1 -- 172.21.2.101:0/255901717 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f837c1a0580 msgr2=0x7f837c086fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:49:24.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.416 7f8381ae3700 1 --2- 172.21.2.101:0/255901717 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f837c1a0580 0x7f837c086fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:24.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.416 7f8381ae3700 1 -- 172.21.2.101:0/255901717 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837c083a10 con 0x7f837c0b66b0 2021-02-10T11:49:24.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.416 7f8381ae3700 1 --2- 172.21.2.101:0/255901717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f837c0b66b0 0x7f837c0b6ab0 secure :-1 s=READY pgs=3388 cs=0 l=1 rev1=1 rx=0x7f836c009630 tx=0x7f836c00b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:24.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.416 7f8372ffd700 1 -- 172.21.2.101:0/255901717 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f836c0093a0 con 0x7f837c0b66b0 2021-02-10T11:49:24.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.416 7f8372ffd700 1 -- 172.21.2.101:0/255901717 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f836c00bd70 con 0x7f837c0b66b0 2021-02-10T11:49:24.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.416 7f83844c8700 1 -- 172.21.2.101:0/255901717 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f837c083a10 con 0x7f837c0b66b0 2021-02-10T11:49:24.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.416 7f8372ffd700 1 -- 172.21.2.101:0/255901717 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f836c005560 con 0x7f837c0b66b0 2021-02-10T11:49:24.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.416 7f83844c8700 1 -- 172.21.2.101:0/255901717 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f837c083fa0 con 0x7f837c0b66b0 2021-02-10T11:49:24.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.417 7f8372ffd700 1 -- 172.21.2.101:0/255901717 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f836c00a000 con 0x7f837c0b66b0 2021-02-10T11:49:24.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.417 7f8372ffd700 1 --2- 172.21.2.101:0/255901717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8364018730 0x7f836401ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:24.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.417 7f8372ffd700 1 -- 172.21.2.101:0/255901717 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f836c0327d0 con 0x7f837c0b66b0 2021-02-10T11:49:24.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.417 7f83822e4700 1 --2- 172.21.2.101:0/255901717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8364018730 0x7f836401ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:24.417 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.417 7f83822e4700 1 --2- 172.21.2.101:0/255901717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8364018730 0x7f836401ac10 crc :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:24.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.448 7f83844c8700 1 -- 172.21.2.101:0/255901717 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f837c082980 con 0x7f837c0b66b0 2021-02-10T11:49:24.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.451 7f8372ffd700 1 -- 172.21.2.101:0/255901717 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f836c009340 con 0x7f837c0b66b0 2021-02-10T11:49:24.577 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.580 7f83844c8700 1 -- 172.21.2.101:0/255901717 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f837c082980 con 0x7f837c0b66b0 2021-02-10T11:49:24.580 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.584 7f8372ffd700 1 -- 172.21.2.101:0/255901717 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v123) v1 ==== 184+0+0 (secure 0 0 0) 0x7f836c00a330 con 0x7f837c0b66b0 2021-02-10T11:49:24.584 INFO:teuthology.orchestra.run.gibba001.stderr:updated caps for client.3 2021-02-10T11:49:24.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.589 7f83844c8700 1 -- 172.21.2.101:0/255901717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8364018730 msgr2=0x7f836401ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:24.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.589 7f83844c8700 1 --2- 172.21.2.101:0/255901717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8364018730 0x7f836401ac10 crc :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:24.585 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.589 7f83844c8700 1 -- 172.21.2.101:0/255901717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f837c0b66b0 msgr2=0x7f837c0b6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:24.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.589 7f83844c8700 1 --2- 172.21.2.101:0/255901717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f837c0b66b0 0x7f837c0b6ab0 secure :-1 s=READY pgs=3388 cs=0 l=1 rev1=1 rx=0x7f836c009630 tx=0x7f836c00b980).stop 2021-02-10T11:49:24.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.590 7f83844c8700 1 -- 172.21.2.101:0/255901717 shutdown_connections 2021-02-10T11:49:24.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.590 7f83844c8700 1 --2- 172.21.2.101:0/255901717 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8364018730 0x7f836401ac10 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:24.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.590 7f83844c8700 1 --2- 172.21.2.101:0/255901717 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f837c082bb0 0x7f837c082fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:24.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.590 7f83844c8700 1 --2- 172.21.2.101:0/255901717 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f837c0b66b0 0x7f837c0b6ab0 unknown :-1 s=CLOSED pgs=3388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:24.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.590 7f83844c8700 1 --2- 172.21.2.101:0/255901717 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f837c1a0580 0x7f837c086fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:24.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.590 7f83844c8700 1 -- 172.21.2.101:0/255901717 >> 172.21.2.101:0/255901717 conn(0x7f837c0afc60 msgr2=0x7f837c0b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:24.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.590 7f83844c8700 1 -- 172.21.2.101:0/255901717 shutdown_connections 2021-02-10T11:49:24.587 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:24.590 7f83844c8700 1 -- 172.21.2.101:0/255901717 wait complete. 2021-02-10T11:49:24.594 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:49:25.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.012 7fe7fccb4700 1 Processor -- start 2021-02-10T11:49:25.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.012 7fe7fccb4700 1 -- start start 2021-02-10T11:49:25.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.012 7fe7fccb4700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fe7f80d1c70 0x7fe7f80f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:25.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.012 7fe7fccb4700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7f80f0b10 con 0x7fe7f80d1870 2021-02-10T11:49:25.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.012 7fe7fccb4700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7f81b0be0 con 0x7fe7f80d0340 2021-02-10T11:49:25.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.012 7fe7fccb4700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f80e5ed0 con 0x7fe7f80d1c70 2021-02-10T11:49:25.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.012 7fe7f685e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe7f80d1870 0x7fe7f81b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36344/0 (socket says 172.21.2.101:36344) 2021-02-10T11:49:25.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.012 7fe7f685e700 1 -- 172.21.2.101:0/2372403125 learned_addr learned my addr 172.21.2.101:0/2372403125 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:25.026 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.013 7fe7f585c700 1 -- 172.21.2.101:0/2372403125 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22585654 0 0) 0x7fe7e8000ce0 con 0x7fe7f80d1870 2021-02-10T11:49:25.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.013 7fe7f585c700 1 -- 172.21.2.101:0/2372403125 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7e4002fe0 con 0x7fe7f80d1870 2021-02-10T11:49:25.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.013 7fe7f585c700 1 -- 172.21.2.101:0/2372403125 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3434413537 0 0) 0x7fe7e80010e0 con 0x7fe7f80d1870 2021-02-10T11:49:25.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.013 7fe7f585c700 1 -- 172.21.2.101:0/2372403125 >> v2:172.21.2.108:3300/0 conn(0x7fe7f80d1c70 msgr2=0x7fe7f80f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:25.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.013 7fe7f585c700 1 --2- 172.21.2.101:0/2372403125 >> v2:172.21.2.108:3300/0 conn(0x7fe7f80d1c70 0x7fe7f80f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:25.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.013 7fe7f585c700 1 -- 172.21.2.101:0/2372403125 >> v1:172.21.2.108:6789/0 conn(0x7fe7f80d0340 legacy=0x7fe7f80ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:25.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.013 7fe7f585c700 1 -- 172.21.2.101:0/2372403125 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f80efe30 con 0x7fe7f80d1870 2021-02-10T11:49:25.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.013 7fe7f585c700 1 -- 172.21.2.101:0/2372403125 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fe7e8001660 con 0x7fe7f80d1870 2021-02-10T11:49:25.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.013 7fe7f585c700 1 -- 172.21.2.101:0/2372403125 >> v1:172.21.2.101:6789/0 conn(0x7fe7f80d1870 legacy=0x7fe7f81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:25.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.013 7fe7f585c700 1 --2- 172.21.2.101:0/2372403125 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe7e4003ac0 0x7fe7e4005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:25.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.013 7fe7f585c700 1 --2- 172.21.2.101:0/2372403125 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7e4006780 0x7fe7e4008ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:25.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.013 7fe7f585c700 1 --2- 172.21.2.101:0/2372403125 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7e40095a0 0x7fe7e400baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:25.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.013 7fe7f585c700 1 -- 172.21.2.101:0/2372403125 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e400c2f0 con 0x7fe7e40095a0 2021-02-10T11:49:25.027 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.013 7fe7f585c700 1 -- 172.21.2.101:0/2372403125 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e400c510 con 0x7fe7e4003ac0 2021-02-10T11:49:25.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.013 7fe7f585c700 1 -- 172.21.2.101:0/2372403125 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e400c730 con 0x7fe7e4006780 2021-02-10T11:49:25.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.013 7fe7f705f700 1 --2- 172.21.2.101:0/2372403125 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7e40095a0 0x7fe7e400baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:25.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.013 7fe7f685e700 1 --2- 172.21.2.101:0/2372403125 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe7e4003ac0 0x7fe7e4005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:25.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.013 7fe7f705f700 1 -- 172.21.2.101:0/2372403125 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7e4006780 msgr2=0x7fe7e4008ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:25.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.013 7fe7f705f700 1 --2- 172.21.2.101:0/2372403125 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7e4006780 0x7fe7e4008ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:25.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.013 7fe7f705f700 1 -- 172.21.2.101:0/2372403125 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe7e4003ac0 msgr2=0x7fe7e4005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:25.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.013 7fe7f705f700 1 --2- 172.21.2.101:0/2372403125 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe7e4003ac0 0x7fe7e4005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:25.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.013 7fe7f705f700 1 -- 172.21.2.101:0/2372403125 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e400c9d0 con 0x7fe7e40095a0 2021-02-10T11:49:25.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.014 7fe7f705f700 1 -- 172.21.2.101:0/2372403125 reap_dead start 2021-02-10T11:49:25.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.014 7fe7f705f700 1 --2- 172.21.2.101:0/2372403125 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7e40095a0 0x7fe7e400baa0 secure :-1 s=READY pgs=3390 cs=0 l=1 rev1=1 rx=0x7fe7f0006f80 tx=0x7fe7f004f310).ready entity=mon.0 client_cookie=5f90670384763bc8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:25.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.014 7fe7f585c700 1 -- 172.21.2.101:0/2372403125 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe7f0009d60 con 0x7fe7e40095a0 2021-02-10T11:49:25.028 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.014 7fe7f585c700 1 -- 172.21.2.101:0/2372403125 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f0006ba0 con 0x7fe7e40095a0 2021-02-10T11:49:25.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.014 7fe7f585c700 1 -- 172.21.2.101:0/2372403125 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe7f0001490 con 0x7fe7e40095a0 2021-02-10T11:49:25.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.014 7fe7fccb4700 1 -- 172.21.2.101:0/2372403125 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7e40095a0 msgr2=0x7fe7e400baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:25.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.014 7fe7fccb4700 1 --2- 172.21.2.101:0/2372403125 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7e40095a0 0x7fe7e400baa0 secure :-1 s=READY pgs=3390 cs=0 l=1 rev1=1 rx=0x7fe7f0006f80 tx=0x7fe7f004f310).stop 2021-02-10T11:49:25.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.015 7fe7fccb4700 1 -- 172.21.2.101:0/2372403125 shutdown_connections 2021-02-10T11:49:25.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.015 7fe7fccb4700 1 --2- 172.21.2.101:0/2372403125 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7e40095a0 0x7fe7e400baa0 unknown :-1 s=CLOSED pgs=3390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:25.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.015 7fe7fccb4700 1 -- 172.21.2.101:0/2372403125 >> 172.21.2.101:0/2372403125 conn(0x7fe7f80ce450 msgr2=0x7fe7f80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:25.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.015 7fe7fccb4700 1 -- 172.21.2.101:0/2372403125 shutdown_connections 2021-02-10T11:49:25.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.015 7fe7fccb4700 1 -- 172.21.2.101:0/2372403125 wait complete. 2021-02-10T11:49:25.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.030 7fe7fccb4700 1 Processor -- start 2021-02-10T11:49:25.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.030 7fe7fccb4700 1 -- start start 2021-02-10T11:49:25.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.030 7fe7fccb4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7f8195650 0x7fe7f80b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:25.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.030 7fe7fccb4700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe7f80b5b80 0x7fe7f81a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:25.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.030 7fe7fccb4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7f80acfa0 0x7fe7f81a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:25.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.030 7fe7fccb4700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f80b6010 con 0x7fe7f8195650 2021-02-10T11:49:25.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.030 7fe7fccb4700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f81a3a60 con 0x7fe7f80b5b80 2021-02-10T11:49:25.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.030 7fe7fccb4700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f81a3c40 con 0x7fe7f80acfa0 2021-02-10T11:49:25.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.030 7fe7f685e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7f8195650 0x7fe7f80b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:25.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.030 7fe7f705f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7f80acfa0 0x7fe7f81a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:25.031 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.030 7fe7f705f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7f80acfa0 0x7fe7f81a3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:56824/0 (socket says 172.21.2.101:56824) 2021-02-10T11:49:25.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.030 7fe7f685e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7f8195650 0x7fe7f80b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35862/0 (socket says 172.21.2.101:35862) 2021-02-10T11:49:25.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.030 7fe7f705f700 1 -- 172.21.2.101:0/2380478659 learned_addr learned my addr 172.21.2.101:0/2380478659 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:25.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.030 7fe7f605d700 1 --2- 172.21.2.101:0/2380478659 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe7f80b5b80 0x7fe7f81a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:25.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.031 7fe7f685e700 1 -- 172.21.2.101:0/2380478659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7f80acfa0 msgr2=0x7fe7f81a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:25.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.031 7fe7f685e700 1 --2- 172.21.2.101:0/2380478659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7f80acfa0 0x7fe7f81a3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:25.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.031 7fe7f685e700 1 -- 172.21.2.101:0/2380478659 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe7f80b5b80 msgr2=0x7fe7f81a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:25.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.031 7fe7f685e700 1 --2- 172.21.2.101:0/2380478659 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe7f80b5b80 0x7fe7f81a2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:25.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.031 7fe7f685e700 1 -- 172.21.2.101:0/2380478659 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f80bcae0 con 0x7fe7f8195650 2021-02-10T11:49:25.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.031 7fe7f605d700 1 --2- 172.21.2.101:0/2380478659 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe7f80b5b80 0x7fe7f81a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:49:25.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.031 7fe7f685e700 1 --2- 172.21.2.101:0/2380478659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7f8195650 0x7fe7f80b5560 secure :-1 s=READY pgs=3391 cs=0 l=1 rev1=1 rx=0x7fe7e8009fc0 tx=0x7fe7e800b810).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:25.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.031 7fe7ef7fe700 1 -- 172.21.2.101:0/2380478659 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe7e8006030 con 0x7fe7f8195650 2021-02-10T11:49:25.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.031 7fe7ef7fe700 1 -- 172.21.2.101:0/2380478659 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e80063b0 con 0x7fe7f8195650 2021-02-10T11:49:25.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.031 7fe7ef7fe700 1 -- 172.21.2.101:0/2380478659 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe7e80045b0 con 0x7fe7f8195650 2021-02-10T11:49:25.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.031 7fe7fccb4700 1 -- 172.21.2.101:0/2380478659 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f80bcae0 con 0x7fe7f8195650 2021-02-10T11:49:25.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.031 7fe7fccb4700 1 -- 172.21.2.101:0/2380478659 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f80bd070 con 0x7fe7f8195650 2021-02-10T11:49:25.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.032 7fe7ef7fe700 1 -- 172.21.2.101:0/2380478659 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe7e80043a0 con 0x7fe7f8195650 2021-02-10T11:49:25.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.032 7fe7ef7fe700 1 --2- 172.21.2.101:0/2380478659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7dc0186c0 0x7fe7dc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:25.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.032 7fe7ef7fe700 1 -- 172.21.2.101:0/2380478659 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe7e80326b0 con 0x7fe7f8195650 2021-02-10T11:49:25.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.032 7fe7f605d700 1 --2- 172.21.2.101:0/2380478659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7dc0186c0 0x7fe7dc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:25.033 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.032 7fe7f605d700 1 --2- 172.21.2.101:0/2380478659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7dc0186c0 0x7fe7dc01abc0 crc :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:25.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.062 7fe7fccb4700 1 -- 172.21.2.101:0/2380478659 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f80b6010 con 0x7fe7f8195650 2021-02-10T11:49:25.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.064 7fe7ef7fe700 1 -- 172.21.2.101:0/2380478659 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe7e8004870 con 0x7fe7f8195650 2021-02-10T11:49:25.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.193 7fe7fccb4700 1 -- 172.21.2.101:0/2380478659 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7f80b6010 con 0x7fe7f8195650 2021-02-10T11:49:25.190 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.194 7fe7ef7fe700 1 -- 172.21.2.101:0/2380478659 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) v1 ==== 95+0+2108 (secure 0 0 0) 0x7fe7e80043a0 con 0x7fe7f8195650 2021-02-10T11:49:25.191 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:49:25.191 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":172,"default_fscid":21,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":13902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1207472796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1207472796},{"type":"v1","addr":"172.21.2.101:6835","nonce":1207472796}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":168}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:49:18.830789","modified":"2021-02-10 11:49:20.837623","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9100":{"gid":9100,"name":"b","rank":0,"incarnation":169,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/1721728696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":1721728696},{"type":"v1","addr":"172.21.2.108:6833","nonce":1721728696}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[43],"metadata_pool":42,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":21}]} 2021-02-10T11:49:25.191 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 172 2021-02-10T11:49:25.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.196 7fe7fccb4700 1 -- 172.21.2.101:0/2380478659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7dc0186c0 msgr2=0x7fe7dc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:25.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.196 7fe7fccb4700 1 --2- 172.21.2.101:0/2380478659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7dc0186c0 0x7fe7dc01abc0 crc :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:25.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.196 7fe7fccb4700 1 -- 172.21.2.101:0/2380478659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7f8195650 msgr2=0x7fe7f80b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:25.192 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.196 7fe7fccb4700 1 --2- 172.21.2.101:0/2380478659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7f8195650 0x7fe7f80b5560 secure :-1 s=READY pgs=3391 cs=0 l=1 rev1=1 rx=0x7fe7e8009fc0 tx=0x7fe7e800b810).stop 2021-02-10T11:49:25.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.196 7fe7fccb4700 1 -- 172.21.2.101:0/2380478659 shutdown_connections 2021-02-10T11:49:25.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.196 7fe7fccb4700 1 --2- 172.21.2.101:0/2380478659 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe7dc0186c0 0x7fe7dc01abc0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:25.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.196 7fe7fccb4700 1 --2- 172.21.2.101:0/2380478659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe7f80acfa0 0x7fe7f81a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:25.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.196 7fe7fccb4700 1 --2- 172.21.2.101:0/2380478659 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe7f80b5b80 0x7fe7f81a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:25.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.196 7fe7fccb4700 1 --2- 172.21.2.101:0/2380478659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe7f8195650 0x7fe7f80b5560 unknown :-1 s=CLOSED pgs=3391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:25.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.196 7fe7fccb4700 1 -- 172.21.2.101:0/2380478659 >> 172.21.2.101:0/2380478659 conn(0x7fe7f80b3040 msgr2=0x7fe7f80c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:25.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.196 7fe7fccb4700 1 -- 172.21.2.101:0/2380478659 shutdown_connections 2021-02-10T11:49:25.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:25.196 7fe7fccb4700 1 -- 172.21.2.101:0/2380478659 wait complete. 2021-02-10T11:49:25.402 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 172, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:49:18.830789', 'modified': '2021-02-10 11:49:20.837623', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9100}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9100': {'gid': 9100, 'name': 'b', 'rank': 0, 'incarnation': 169, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/1721728696', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 1721728696}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 1721728696}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [43], 'metadata_pool': 42, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:49:25.403 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:49:25.403 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:49:25.801 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:49:25.801 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:49:25.801 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:49:25.801 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 9100, 2021-02-10T11:49:25.801 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:49:25.801 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:49:25.801 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 6.9719303930000001, 2021-02-10T11:49:25.802 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 172, 2021-02-10T11:49:25.802 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 199, 2021-02-10T11:49:25.802 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 199, 2021-02-10T11:49:25.802 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 12.641048244 2021-02-10T11:49:25.802 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:49:25.805 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 9100, "want_state": "up:active", "state": "up:active", "rank_uptime": 6.9719303930000001, "mdsmap_epoch": 172, "osdmap_epoch": 199, "osdmap_epoch_barrier": 199, "uptime": 12.641048244 } 2021-02-10T11:49:25.806 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:49:26.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.223 7f451bd07700 1 Processor -- start 2021-02-10T11:49:26.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.224 7f451bd07700 1 -- start start 2021-02-10T11:49:26.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.224 7f451bd07700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f45140d1870 0x7f45141b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:26.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.224 7f451bd07700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45140f0650 0x7f45140cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:26.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.224 7f451bd07700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45140f0b10 con 0x7f45141b0a10 2021-02-10T11:49:26.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.224 7f451bd07700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45140e5ed0 con 0x7f45140d1870 2021-02-10T11:49:26.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.224 7f451bd07700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45140d1c70 con 0x7f45140f0650 2021-02-10T11:49:26.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.224 7f4519b23700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f45140d1870 0x7f45141b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:26.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.224 7f4519322700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f45141b0a10 0x7f45140cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36362/0 (socket says 172.21.2.101:36362) 2021-02-10T11:49:26.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.224 7f4519322700 1 -- 172.21.2.101:0/1660126831 learned_addr learned my addr 172.21.2.101:0/1660126831 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:26.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.225 7f451a324700 1 --2- 172.21.2.101:0/1660126831 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45140f0650 0x7f45140cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:26.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.225 7f4519b23700 1 -- 172.21.2.101:0/1660126831 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45140f0650 msgr2=0x7f45140cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:26.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.225 7f4519b23700 1 --2- 172.21.2.101:0/1660126831 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45140f0650 0x7f45140cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:26.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.225 7f4519b23700 1 -- 172.21.2.101:0/1660126831 >> v1:172.21.2.101:6789/0 conn(0x7f45141b0a10 legacy=0x7f45140cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:26.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.225 7f4518b21700 1 -- 172.21.2.101:0/1660126831 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1311022727 0 0) 0x7f4508000ce0 con 0x7f45141b0a10 2021-02-10T11:49:26.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.225 7f4519b23700 1 -- 172.21.2.101:0/1660126831 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45140d1e90 con 0x7f45140d1870 2021-02-10T11:49:26.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.225 7f4519b23700 1 --2- 172.21.2.101:0/1660126831 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45140d1870 0x7f45141b03f0 secure :-1 s=READY pgs=3393 cs=0 l=1 rev1=1 rx=0x7f4510006650 tx=0x7f4510009a00).ready entity=mon.0 client_cookie=a63817d1c069cebf server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:26.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.225 7f4518b21700 1 -- 172.21.2.101:0/1660126831 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4510009df0 con 0x7f45140d1870 2021-02-10T11:49:26.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.225 7f4518b21700 1 -- 172.21.2.101:0/1660126831 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45100009b0 con 0x7f45140d1870 2021-02-10T11:49:26.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.225 7f4518b21700 1 -- 172.21.2.101:0/1660126831 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f451000c2b0 con 0x7f45140d1870 2021-02-10T11:49:26.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.226 7f451bd07700 1 -- 172.21.2.101:0/1660126831 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45140d1870 msgr2=0x7f45141b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:26.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.226 7f451bd07700 1 --2- 172.21.2.101:0/1660126831 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45140d1870 0x7f45141b03f0 secure :-1 s=READY pgs=3393 cs=0 l=1 rev1=1 rx=0x7f4510006650 tx=0x7f4510009a00).stop 2021-02-10T11:49:26.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.226 7f451bd07700 1 -- 172.21.2.101:0/1660126831 shutdown_connections 2021-02-10T11:49:26.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.226 7f451bd07700 1 --2- 172.21.2.101:0/1660126831 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45140f0650 0x7f45140cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:26.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.226 7f451bd07700 1 --2- 172.21.2.101:0/1660126831 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f45140d1870 0x7f45141b03f0 unknown :-1 s=CLOSED pgs=3393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:26.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.226 7f451bd07700 1 -- 172.21.2.101:0/1660126831 >> 172.21.2.101:0/1660126831 conn(0x7f45140ce450 msgr2=0x7f45140d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:26.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.226 7f451bd07700 1 -- 172.21.2.101:0/1660126831 shutdown_connections 2021-02-10T11:49:26.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.226 7f451bd07700 1 -- 172.21.2.101:0/1660126831 wait complete. 2021-02-10T11:49:26.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.241 7f451bd07700 1 Processor -- start 2021-02-10T11:49:26.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.241 7f451bd07700 1 -- start start 2021-02-10T11:49:26.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.241 7f451bd07700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45140ada90 0x7f45140ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:26.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.241 7f451bd07700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4514099d20 0x7f451409a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:26.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.242 7f451bd07700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f451409a740 0x7f4514097bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:26.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.242 7f451bd07700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45140997a0 con 0x7f4514099d20 2021-02-10T11:49:26.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.242 7f451bd07700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451418f0c0 con 0x7f451409a740 2021-02-10T11:49:26.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.242 7f451bd07700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451409ab40 con 0x7f45140ada90 2021-02-10T11:49:26.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.242 7f4519322700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4514099d20 0x7f451409a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:26.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.242 7f4519322700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4514099d20 0x7f451409a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35876/0 (socket says 172.21.2.101:35876) 2021-02-10T11:49:26.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.242 7f4519322700 1 -- 172.21.2.101:0/3721883846 learned_addr learned my addr 172.21.2.101:0/3721883846 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:26.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.242 7f4519b23700 1 --2- 172.21.2.101:0/3721883846 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45140ada90 0x7f45140ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:26.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.242 7f451a324700 1 --2- 172.21.2.101:0/3721883846 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f451409a740 0x7f4514097bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:26.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.242 7f4519322700 1 -- 172.21.2.101:0/3721883846 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45140ada90 msgr2=0x7f45140ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:26.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.242 7f4519322700 1 --2- 172.21.2.101:0/3721883846 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45140ada90 0x7f45140ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:26.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.242 7f4519322700 1 -- 172.21.2.101:0/3721883846 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f451409a740 msgr2=0x7f4514097bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:26.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.242 7f4519322700 1 --2- 172.21.2.101:0/3721883846 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f451409a740 0x7f4514097bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:26.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.242 7f4519322700 1 -- 172.21.2.101:0/3721883846 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451409ad60 con 0x7f4514099d20 2021-02-10T11:49:26.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.242 7f4519322700 1 --2- 172.21.2.101:0/3721883846 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4514099d20 0x7f451409a120 secure :-1 s=READY pgs=3394 cs=0 l=1 rev1=1 rx=0x7f4508001430 tx=0x7f450800a930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:26.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.242 7f45067fc700 1 -- 172.21.2.101:0/3721883846 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f450800ad20 con 0x7f4514099d20 2021-02-10T11:49:26.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.242 7f45067fc700 1 -- 172.21.2.101:0/3721883846 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4508000ce0 con 0x7f4514099d20 2021-02-10T11:49:26.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.242 7f451bd07700 1 -- 172.21.2.101:0/3721883846 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f451409ad60 con 0x7f4514099d20 2021-02-10T11:49:26.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.242 7f45067fc700 1 -- 172.21.2.101:0/3721883846 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4508004620 con 0x7f4514099d20 2021-02-10T11:49:26.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.243 7f451bd07700 1 -- 172.21.2.101:0/3721883846 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f451409ad60 con 0x7f4514099d20 2021-02-10T11:49:26.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.243 7f45067fc700 1 -- 172.21.2.101:0/3721883846 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f45080043a0 con 0x7f4514099d20 2021-02-10T11:49:26.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.243 7f45067fc700 1 --2- 172.21.2.101:0/3721883846 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44fc018730 0x7f44fc01ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:26.242 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.243 7f45067fc700 1 -- 172.21.2.101:0/3721883846 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f4508032640 con 0x7f4514099d20 2021-02-10T11:49:26.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.243 7f4519b23700 1 --2- 172.21.2.101:0/3721883846 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44fc018730 0x7f44fc01ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:26.243 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.244 7f4519b23700 1 --2- 172.21.2.101:0/3721883846 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44fc018730 0x7f44fc01ac10 crc :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:26.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.273 7f451bd07700 1 -- 172.21.2.101:0/3721883846 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45140ebc00 con 0x7f4514099d20 2021-02-10T11:49:26.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.276 7f45067fc700 1 -- 172.21.2.101:0/3721883846 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f4508000990 con 0x7f4514099d20 2021-02-10T11:49:26.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.403 7f451bd07700 1 -- 172.21.2.101:0/3721883846 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45140f0b10 con 0x7f4514099d20 2021-02-10T11:49:26.399 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.404 7f45067fc700 1 -- 172.21.2.101:0/3721883846 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f4508000ce0 con 0x7f4514099d20 2021-02-10T11:49:26.400 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:49:26.400 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":199,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:49:19.833311","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":43,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":42,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:49:16.791877","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":43,"pool_name":"cephfs_data","create_time":"2021-02-10 11:49:17.820338","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:49:26.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.405 7f451bd07700 1 -- 172.21.2.101:0/3721883846 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44fc018730 msgr2=0x7f44fc01ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:26.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.405 7f451bd07700 1 --2- 172.21.2.101:0/3721883846 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44fc018730 0x7f44fc01ac10 crc :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:26.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.405 7f451bd07700 1 -- 172.21.2.101:0/3721883846 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4514099d20 msgr2=0x7f451409a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:26.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.405 7f451bd07700 1 --2- 172.21.2.101:0/3721883846 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4514099d20 0x7f451409a120 secure :-1 s=READY pgs=3394 cs=0 l=1 rev1=1 rx=0x7f4508001430 tx=0x7f450800a930).stop 2021-02-10T11:49:26.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.405 7f451bd07700 1 -- 172.21.2.101:0/3721883846 shutdown_connections 2021-02-10T11:49:26.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.405 7f451bd07700 1 --2- 172.21.2.101:0/3721883846 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f451409a740 0x7f4514097bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:26.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.405 7f451bd07700 1 --2- 172.21.2.101:0/3721883846 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4514099d20 0x7f451409a120 unknown :-1 s=CLOSED pgs=3394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:26.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.405 7f451bd07700 1 --2- 172.21.2.101:0/3721883846 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f44fc018730 0x7f44fc01ac10 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:26.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.405 7f451bd07700 1 --2- 172.21.2.101:0/3721883846 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45140ada90 0x7f45140ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:26.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.405 7f451bd07700 1 -- 172.21.2.101:0/3721883846 >> 172.21.2.101:0/3721883846 conn(0x7f45140f0550 msgr2=0x7f4514191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:26.402 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.406 7f451bd07700 1 -- 172.21.2.101:0/3721883846 shutdown_connections 2021-02-10T11:49:26.403 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.406 7f451bd07700 1 -- 172.21.2.101:0/3721883846 wait complete. 2021-02-10T11:49:26.410 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 42, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:49:16.791877', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '198', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 43, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:49:17.820338', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '199', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:49:26.410 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:49:26.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.830 7f45775e0700 1 Processor -- start 2021-02-10T11:49:26.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.830 7f45775e0700 1 -- start start 2021-02-10T11:49:26.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.831 7f45775e0700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f45700d1c70 0x7f45700cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:26.844 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.831 7f45775e0700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45700d0170 con 0x7f45700cf580 2021-02-10T11:49:26.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.831 7f45775e0700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45700fe1c0 con 0x7f45700d1870 2021-02-10T11:49:26.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.831 7f45775e0700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45701b0cb0 con 0x7f45700d1c70 2021-02-10T11:49:26.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.831 7f4575bfd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f45700cf580 0x7f45700cf980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36374/0 (socket says 172.21.2.101:36374) 2021-02-10T11:49:26.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.831 7f4575bfd700 1 -- 172.21.2.101:0/365837110 learned_addr learned my addr 172.21.2.101:0/365837110 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:26.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.831 7f4563fff700 1 -- 172.21.2.101:0/365837110 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2691068250 0 0) 0x7f4568001500 con 0x7f45700cf580 2021-02-10T11:49:26.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.831 7f4563fff700 1 -- 172.21.2.101:0/365837110 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f455c002fe0 con 0x7f45700cf580 2021-02-10T11:49:26.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4574bfb700 1 --2- 172.21.2.101:0/365837110 >> v2:172.21.2.108:3300/0 conn(0x7f45700d1c70 0x7f45700cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:26.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4563fff700 1 -- 172.21.2.101:0/365837110 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4290961580 0 0) 0x7f4568001460 con 0x7f45700cf580 2021-02-10T11:49:26.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4563fff700 1 -- 172.21.2.101:0/365837110 >> v2:172.21.2.108:3300/0 conn(0x7f45700d1c70 msgr2=0x7f45700cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:26.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4563fff700 1 --2- 172.21.2.101:0/365837110 >> v2:172.21.2.108:3300/0 conn(0x7f45700d1c70 0x7f45700cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:26.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4563fff700 1 -- 172.21.2.101:0/365837110 >> v1:172.21.2.108:6789/0 conn(0x7f45700d1870 legacy=0x7f45701b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:26.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4563fff700 1 -- 172.21.2.101:0/365837110 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4570109a60 con 0x7f45700cf580 2021-02-10T11:49:26.845 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4563fff700 1 -- 172.21.2.101:0/365837110 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f4568001980 con 0x7f45700cf580 2021-02-10T11:49:26.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4563fff700 1 -- 172.21.2.101:0/365837110 >> v1:172.21.2.101:6789/0 conn(0x7f45700cf580 legacy=0x7f45700cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:26.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4563fff700 1 --2- 172.21.2.101:0/365837110 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f455c003b10 0x7f455c006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:26.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4563fff700 1 --2- 172.21.2.101:0/365837110 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f455c0067e0 0x7f455c008cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:26.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4563fff700 1 --2- 172.21.2.101:0/365837110 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f455c0095c0 0x7f455c00baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:26.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4563fff700 1 -- 172.21.2.101:0/365837110 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c00c2f0 con 0x7f455c0095c0 2021-02-10T11:49:26.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4563fff700 1 -- 172.21.2.101:0/365837110 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c00c510 con 0x7f455c0067e0 2021-02-10T11:49:26.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4563fff700 1 -- 172.21.2.101:0/365837110 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455c00c730 con 0x7f455c003b10 2021-02-10T11:49:26.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4575bfd700 1 --2- 172.21.2.101:0/365837110 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f455c0095c0 0x7f455c00baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:26.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f45753fc700 1 --2- 172.21.2.101:0/365837110 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f455c003b10 0x7f455c006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:26.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4574bfb700 1 --2- 172.21.2.101:0/365837110 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f455c0067e0 0x7f455c008cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:26.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4575bfd700 1 -- 172.21.2.101:0/365837110 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f455c003b10 msgr2=0x7f455c006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:26.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4575bfd700 1 --2- 172.21.2.101:0/365837110 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f455c003b10 0x7f455c006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:26.846 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4575bfd700 1 -- 172.21.2.101:0/365837110 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f455c0067e0 msgr2=0x7f455c008cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:26.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4575bfd700 1 --2- 172.21.2.101:0/365837110 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f455c0067e0 0x7f455c008cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:26.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4575bfd700 1 -- 172.21.2.101:0/365837110 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c00c9d0 con 0x7f455c0095c0 2021-02-10T11:49:26.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4575bfd700 1 -- 172.21.2.101:0/365837110 reap_dead start 2021-02-10T11:49:26.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.832 7f4575bfd700 1 --2- 172.21.2.101:0/365837110 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f455c0095c0 0x7f455c00baa0 secure :-1 s=READY pgs=3396 cs=0 l=1 rev1=1 rx=0x7f4568001f50 tx=0x7f456804e320).ready entity=mon.0 client_cookie=a64655ca8e3e23c8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:26.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.833 7f4563fff700 1 -- 172.21.2.101:0/365837110 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4568001460 con 0x7f455c0095c0 2021-02-10T11:49:26.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.833 7f4563fff700 1 -- 172.21.2.101:0/365837110 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45680070c0 con 0x7f455c0095c0 2021-02-10T11:49:26.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.833 7f4563fff700 1 -- 172.21.2.101:0/365837110 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f45680075a0 con 0x7f455c0095c0 2021-02-10T11:49:26.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.833 7f45775e0700 1 -- 172.21.2.101:0/365837110 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f455c0095c0 msgr2=0x7f455c00baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:26.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.833 7f45775e0700 1 --2- 172.21.2.101:0/365837110 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f455c0095c0 0x7f455c00baa0 secure :-1 s=READY pgs=3396 cs=0 l=1 rev1=1 rx=0x7f4568001f50 tx=0x7f456804e320).stop 2021-02-10T11:49:26.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.833 7f45775e0700 1 -- 172.21.2.101:0/365837110 shutdown_connections 2021-02-10T11:49:26.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.833 7f45775e0700 1 --2- 172.21.2.101:0/365837110 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f455c0095c0 0x7f455c00baa0 unknown :-1 s=CLOSED pgs=3396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:26.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.833 7f45775e0700 1 -- 172.21.2.101:0/365837110 >> 172.21.2.101:0/365837110 conn(0x7f45700ce420 msgr2=0x7f45700d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:26.847 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.834 7f45775e0700 1 -- 172.21.2.101:0/365837110 shutdown_connections 2021-02-10T11:49:26.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.834 7f45775e0700 1 -- 172.21.2.101:0/365837110 wait complete. 2021-02-10T11:49:26.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.848 7f45775e0700 1 Processor -- start 2021-02-10T11:49:26.848 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.848 7f45775e0700 1 -- start start 2021-02-10T11:49:26.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.849 7f45775e0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4570085bc0 0x7f45700863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:26.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.849 7f45775e0700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45700954c0 0x7f4570096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:26.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.849 7f45775e0700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f45700958c0 0x7f4570096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:26.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.849 7f45775e0700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4570095d80 con 0x7f4570085bc0 2021-02-10T11:49:26.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.849 7f45775e0700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45700bfa60 con 0x7f45700958c0 2021-02-10T11:49:26.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.849 7f45775e0700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4570085fc0 con 0x7f45700954c0 2021-02-10T11:49:26.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.849 7f45753fc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4570085bc0 0x7f45700863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:26.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.849 7f45753fc700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4570085bc0 0x7f45700863e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35892/0 (socket says 172.21.2.101:35892) 2021-02-10T11:49:26.849 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.849 7f45753fc700 1 -- 172.21.2.101:0/3948831341 learned_addr learned my addr 172.21.2.101:0/3948831341 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:26.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.849 7f45753fc700 1 -- 172.21.2.101:0/3948831341 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45700954c0 msgr2=0x7f4570096300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:26.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.849 7f45753fc700 1 --2- 172.21.2.101:0/3948831341 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45700954c0 0x7f4570096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:26.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.849 7f45753fc700 1 -- 172.21.2.101:0/3948831341 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f45700958c0 msgr2=0x7f4570096920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:26.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.849 7f45753fc700 1 --2- 172.21.2.101:0/3948831341 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f45700958c0 0x7f4570096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:26.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.849 7f45753fc700 1 -- 172.21.2.101:0/3948831341 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4570086c10 con 0x7f4570085bc0 2021-02-10T11:49:26.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.849 7f45753fc700 1 --2- 172.21.2.101:0/3948831341 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4570085bc0 0x7f45700863e0 secure :-1 s=READY pgs=3397 cs=0 l=1 rev1=1 rx=0x7f456c006f60 tx=0x7f456c00b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:26.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.850 7f4561ffb700 1 -- 172.21.2.101:0/3948831341 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f456c006340 con 0x7f4570085bc0 2021-02-10T11:49:26.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.850 7f4561ffb700 1 -- 172.21.2.101:0/3948831341 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456c00bd60 con 0x7f4570085bc0 2021-02-10T11:49:26.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.850 7f4561ffb700 1 -- 172.21.2.101:0/3948831341 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f456c002030 con 0x7f4570085bc0 2021-02-10T11:49:26.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.850 7f45775e0700 1 -- 172.21.2.101:0/3948831341 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4570086c10 con 0x7f4570085bc0 2021-02-10T11:49:26.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.850 7f45775e0700 1 -- 172.21.2.101:0/3948831341 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4570097230 con 0x7f4570085bc0 2021-02-10T11:49:26.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.850 7f4561ffb700 1 -- 172.21.2.101:0/3948831341 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f456c000c90 con 0x7f4570085bc0 2021-02-10T11:49:26.850 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.850 7f4561ffb700 1 --2- 172.21.2.101:0/3948831341 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45500186c0 0x7f455001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:26.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.850 7f4561ffb700 1 -- 172.21.2.101:0/3948831341 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f456c032560 con 0x7f4570085bc0 2021-02-10T11:49:26.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.851 7f4574bfb700 1 --2- 172.21.2.101:0/3948831341 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45500186c0 0x7f455001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:26.851 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.851 7f4574bfb700 1 --2- 172.21.2.101:0/3948831341 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45500186c0 0x7f455001abc0 crc :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:26.877 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.881 7f45775e0700 1 -- 172.21.2.101:0/3948831341 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45700fe1c0 con 0x7f4570085bc0 2021-02-10T11:49:26.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:26.883 7f4561ffb700 1 -- 172.21.2.101:0/3948831341 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f456c000940 con 0x7f4570085bc0 2021-02-10T11:49:27.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.013 7f45775e0700 1 -- 172.21.2.101:0/3948831341 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7f45700fe1c0 con 0x7f4570085bc0 2021-02-10T11:49:27.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.013 7f4561ffb700 1 -- 172.21.2.101:0/3948831341 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v199) v1 ==== 92+0+10 (secure 0 0 0) 0x7f456c000c90 con 0x7f4570085bc0 2021-02-10T11:49:27.010 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:49:27.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.015 7f45775e0700 1 -- 172.21.2.101:0/3948831341 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45500186c0 msgr2=0x7f455001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:27.011 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.015 7f45775e0700 1 --2- 172.21.2.101:0/3948831341 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45500186c0 0x7f455001abc0 crc :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:27.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.015 7f45775e0700 1 -- 172.21.2.101:0/3948831341 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4570085bc0 msgr2=0x7f45700863e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:27.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.015 7f45775e0700 1 --2- 172.21.2.101:0/3948831341 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4570085bc0 0x7f45700863e0 secure :-1 s=READY pgs=3397 cs=0 l=1 rev1=1 rx=0x7f456c006f60 tx=0x7f456c00b970).stop 2021-02-10T11:49:27.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.016 7f45775e0700 1 -- 172.21.2.101:0/3948831341 shutdown_connections 2021-02-10T11:49:27.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.016 7f45775e0700 1 --2- 172.21.2.101:0/3948831341 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f45500186c0 0x7f455001abc0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:27.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.016 7f45775e0700 1 --2- 172.21.2.101:0/3948831341 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f45700954c0 0x7f4570096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:27.012 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.016 7f45775e0700 1 --2- 172.21.2.101:0/3948831341 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f45700958c0 0x7f4570096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:27.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.016 7f45775e0700 1 --2- 172.21.2.101:0/3948831341 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f4570085bc0 0x7f45700863e0 unknown :-1 s=CLOSED pgs=3397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:27.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.016 7f45775e0700 1 -- 172.21.2.101:0/3948831341 >> 172.21.2.101:0/3948831341 conn(0x7f45700bef00 msgr2=0x7f457009d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:27.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.016 7f45775e0700 1 -- 172.21.2.101:0/3948831341 shutdown_connections 2021-02-10T11:49:27.013 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.016 7f45775e0700 1 -- 172.21.2.101:0/3948831341 wait complete. 2021-02-10T11:49:27.020 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:49:27.455 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.442 7f0fc6687700 1 Processor -- start 2021-02-10T11:49:27.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.442 7f0fc6687700 1 -- start start 2021-02-10T11:49:27.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.443 7f0fc6687700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0fc00d1c60 0x7f0fc00cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:27.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.443 7f0fc6687700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0fc00cf570 0x7f0fc00cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:27.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.443 7f0fc6687700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc00fe1b0 con 0x7f0fc00d1860 2021-02-10T11:49:27.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.443 7f0fc6687700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc01b0ca0 con 0x7f0fc00d1c60 2021-02-10T11:49:27.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.443 7f0fc6687700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc0109a50 con 0x7f0fc00cf570 2021-02-10T11:49:27.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.444 7f0fc4ca4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0fc00cf570 0x7f0fc00cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:27.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.444 7f0fbf7fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0fc00d1c60 0x7f0fc00cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:27.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.444 7f0fbffff700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f0fc00d1860 0x7f0fc01b04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:58034/0 (socket says 172.21.2.101:58034) 2021-02-10T11:49:27.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.444 7f0fc4ca4700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0fc00cf570 0x7f0fc00cf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:56866/0 (socket says 172.21.2.101:56866) 2021-02-10T11:49:27.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.444 7f0fbffff700 1 -- 172.21.2.101:0/2549704625 learned_addr learned my addr 172.21.2.101:0/2549704625 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:27.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.444 7f0fbf7fe700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f0fc00d1c60 0x7f0fc00cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:46386/0 (socket says 172.21.2.101:46386) 2021-02-10T11:49:27.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.444 7f0fbf7fe700 1 -- 172.21.2.101:0/2549704625 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0fc00cf570 msgr2=0x7f0fc00cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:27.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.444 7f0fbf7fe700 1 --2- 172.21.2.101:0/2549704625 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0fc00cf570 0x7f0fc00cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:27.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.444 7f0fbf7fe700 1 -- 172.21.2.101:0/2549704625 >> v1:172.21.2.108:6789/0 conn(0x7f0fc00d1860 legacy=0x7f0fc01b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:27.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.444 7f0fbf7fe700 1 -- 172.21.2.101:0/2549704625 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc0088070 con 0x7f0fc00d1c60 2021-02-10T11:49:27.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.444 7f0fc4ca4700 1 --2- 172.21.2.101:0/2549704625 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0fc00cf570 0x7f0fc00cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:49:27.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.444 7f0fbf7fe700 1 --2- 172.21.2.101:0/2549704625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0fc00d1c60 0x7f0fc00cef50 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f0fa8006650 tx=0x7f0fa8009a80).ready entity=mon.1 client_cookie=1943192e23486e82 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:27.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.445 7f0fbeffd700 1 -- 172.21.2.101:0/2549704625 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0fa800c0c0 con 0x7f0fc00d1c60 2021-02-10T11:49:27.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.445 7f0fbeffd700 1 -- 172.21.2.101:0/2549704625 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa800c340 con 0x7f0fc00d1c60 2021-02-10T11:49:27.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.445 7f0fbeffd700 1 -- 172.21.2.101:0/2549704625 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0fa800c860 con 0x7f0fc00d1c60 2021-02-10T11:49:27.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.445 7f0fc6687700 1 -- 172.21.2.101:0/2549704625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0fc00d1c60 msgr2=0x7f0fc00cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:27.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.445 7f0fc6687700 1 --2- 172.21.2.101:0/2549704625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0fc00d1c60 0x7f0fc00cef50 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f0fa8006650 tx=0x7f0fa8009a80).stop 2021-02-10T11:49:27.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.445 7f0fc6687700 1 -- 172.21.2.101:0/2549704625 shutdown_connections 2021-02-10T11:49:27.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.445 7f0fc6687700 1 --2- 172.21.2.101:0/2549704625 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0fc00cf570 0x7f0fc00cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:27.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.445 7f0fc6687700 1 --2- 172.21.2.101:0/2549704625 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0fc00d1c60 0x7f0fc00cef50 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:27.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.445 7f0fc6687700 1 -- 172.21.2.101:0/2549704625 >> 172.21.2.101:0/2549704625 conn(0x7f0fc00ce410 msgr2=0x7f0fc00d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:27.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.445 7f0fc6687700 1 -- 172.21.2.101:0/2549704625 shutdown_connections 2021-02-10T11:49:27.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.445 7f0fc6687700 1 -- 172.21.2.101:0/2549704625 wait complete. 2021-02-10T11:49:27.460 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.460 7f0fc6687700 1 Processor -- start 2021-02-10T11:49:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.460 7f0fc6687700 1 -- start start 2021-02-10T11:49:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.460 7f0fc6687700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0fc00adf50 0x7f0fc00ae350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.460 7f0fc6687700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0fc00ae970 0x7f0fc00aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.460 7f0fc6687700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0fc00982c0 0x7f0fc00986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.460 7f0fc6687700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc009f450 con 0x7f0fc00adf50 2021-02-10T11:49:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.460 7f0fc6687700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc0098ce0 con 0x7f0fc00982c0 2021-02-10T11:49:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.460 7f0fc6687700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc0098ed0 con 0x7f0fc00ae970 2021-02-10T11:49:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.461 7f0fbffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0fc00adf50 0x7f0fc00ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.461 7f0fbffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0fc00adf50 0x7f0fc00ae350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35906/0 (socket says 172.21.2.101:35906) 2021-02-10T11:49:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.461 7f0fbffff700 1 -- 172.21.2.101:0/2355396287 learned_addr learned my addr 172.21.2.101:0/2355396287 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.461 7f0fbf7fe700 1 --2- 172.21.2.101:0/2355396287 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0fc00ae970 0x7f0fc00aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:27.461 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.461 7f0fc4ca4700 1 --2- 172.21.2.101:0/2355396287 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0fc00982c0 0x7f0fc00986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:27.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.461 7f0fbffff700 1 -- 172.21.2.101:0/2355396287 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0fc00ae970 msgr2=0x7f0fc00aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:27.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.461 7f0fbffff700 1 --2- 172.21.2.101:0/2355396287 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0fc00ae970 0x7f0fc00aed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:27.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.461 7f0fbffff700 1 -- 172.21.2.101:0/2355396287 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0fc00982c0 msgr2=0x7f0fc00986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:27.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.461 7f0fbffff700 1 --2- 172.21.2.101:0/2355396287 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0fc00982c0 0x7f0fc00986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:27.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.461 7f0fbffff700 1 -- 172.21.2.101:0/2355396287 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc00990f0 con 0x7f0fc00adf50 2021-02-10T11:49:27.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.461 7f0fbf7fe700 1 --2- 172.21.2.101:0/2355396287 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0fc00ae970 0x7f0fc00aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:49:27.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.461 7f0fc4ca4700 1 --2- 172.21.2.101:0/2355396287 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0fc00982c0 0x7f0fc00986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:49:27.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.461 7f0fbffff700 1 --2- 172.21.2.101:0/2355396287 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0fc00adf50 0x7f0fc00ae350 secure :-1 s=READY pgs=3398 cs=0 l=1 rev1=1 rx=0x7f0fb0007fa0 tx=0x7f0fb000a950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:27.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.461 7f0fbcff9700 1 -- 172.21.2.101:0/2355396287 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0fb00070d0 con 0x7f0fc00adf50 2021-02-10T11:49:27.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.461 7f0fbcff9700 1 -- 172.21.2.101:0/2355396287 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb000ad40 con 0x7f0fc00adf50 2021-02-10T11:49:27.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.461 7f0fc6687700 1 -- 172.21.2.101:0/2355396287 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc00990f0 con 0x7f0fc00adf50 2021-02-10T11:49:27.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.461 7f0fbcff9700 1 -- 172.21.2.101:0/2355396287 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0fb00045a0 con 0x7f0fc00adf50 2021-02-10T11:49:27.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.462 7f0fc6687700 1 -- 172.21.2.101:0/2355396287 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc00990f0 con 0x7f0fc00adf50 2021-02-10T11:49:27.462 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.462 7f0fbcff9700 1 -- 172.21.2.101:0/2355396287 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f0fb0000e20 con 0x7f0fc00adf50 2021-02-10T11:49:27.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.462 7f0fbcff9700 1 --2- 172.21.2.101:0/2355396287 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0fa4018700 0x7f0fa401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:27.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.462 7f0fbcff9700 1 -- 172.21.2.101:0/2355396287 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0fb0032650 con 0x7f0fc00adf50 2021-02-10T11:49:27.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.462 7f0fbf7fe700 1 --2- 172.21.2.101:0/2355396287 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0fa4018700 0x7f0fa401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:27.463 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.462 7f0fbf7fe700 1 --2- 172.21.2.101:0/2355396287 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0fa4018700 0x7f0fa401abc0 crc :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:27.489 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.492 7f0fc6687700 1 -- 172.21.2.101:0/2355396287 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc0082a20 con 0x7f0fc00adf50 2021-02-10T11:49:27.491 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.495 7f0fbcff9700 1 -- 172.21.2.101:0/2355396287 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f0fb0000c10 con 0x7f0fc00adf50 2021-02-10T11:49:27.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.623 7f0fc6687700 1 -- 172.21.2.101:0/2355396287 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f0fc0082a20 con 0x7f0fc00adf50 2021-02-10T11:49:27.620 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.624 7f0fbcff9700 1 -- 172.21.2.101:0/2355396287 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v199) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0fb000ad40 con 0x7f0fc00adf50 2021-02-10T11:49:27.621 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:49:27.622 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.626 7f0fc6687700 1 -- 172.21.2.101:0/2355396287 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0fa4018700 msgr2=0x7f0fa401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:27.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.626 7f0fc6687700 1 --2- 172.21.2.101:0/2355396287 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0fa4018700 0x7f0fa401abc0 crc :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:27.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.626 7f0fc6687700 1 -- 172.21.2.101:0/2355396287 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0fc00adf50 msgr2=0x7f0fc00ae350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:27.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.626 7f0fc6687700 1 --2- 172.21.2.101:0/2355396287 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0fc00adf50 0x7f0fc00ae350 secure :-1 s=READY pgs=3398 cs=0 l=1 rev1=1 rx=0x7f0fb0007fa0 tx=0x7f0fb000a950).stop 2021-02-10T11:49:27.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.626 7f0fc6687700 1 -- 172.21.2.101:0/2355396287 shutdown_connections 2021-02-10T11:49:27.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.626 7f0fc6687700 1 --2- 172.21.2.101:0/2355396287 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0fa4018700 0x7f0fa401abc0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:27.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.626 7f0fc6687700 1 --2- 172.21.2.101:0/2355396287 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0fc00ae970 0x7f0fc00aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:27.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.626 7f0fc6687700 1 --2- 172.21.2.101:0/2355396287 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f0fc00982c0 0x7f0fc00986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:27.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.626 7f0fc6687700 1 --2- 172.21.2.101:0/2355396287 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0fc00adf50 0x7f0fc00ae350 unknown :-1 s=CLOSED pgs=3398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:27.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.626 7f0fc6687700 1 -- 172.21.2.101:0/2355396287 >> 172.21.2.101:0/2355396287 conn(0x7f0fc00bee90 msgr2=0x7f0fc00bb880 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:27.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.627 7f0fc6687700 1 -- 172.21.2.101:0/2355396287 shutdown_connections 2021-02-10T11:49:27.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:27.627 7f0fc6687700 1 -- 172.21.2.101:0/2355396287 wait complete. 2021-02-10T11:49:27.631 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:49:28.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.051 7f84446a9700 1 Processor -- start 2021-02-10T11:49:28.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.051 7f84446a9700 1 -- start start 2021-02-10T11:49:28.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.052 7f84446a9700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f843c0d1c70 0x7f843c0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:28.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.052 7f84446a9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f843c0cf580 0x7f843c0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:28.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.052 7f84446a9700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f843c0fe1c0 con 0x7f843c0d1870 2021-02-10T11:49:28.064 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.052 7f84446a9700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843c1b0cb0 con 0x7f843c0d1c70 2021-02-10T11:49:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.052 7f84446a9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843c109a60 con 0x7f843c0cf580 2021-02-10T11:49:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.052 7f8441cc4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f843c0d1c70 0x7f843c0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.052 7f8441cc4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f843c0d1c70 0x7f843c0cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35912/0 (socket says 172.21.2.101:35912) 2021-02-10T11:49:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.052 7f8441cc4700 1 -- 172.21.2.101:0/3432831675 learned_addr learned my addr 172.21.2.101:0/3432831675 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.052 7f8442cc6700 1 --2- 172.21.2.101:0/3432831675 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f843c0cf580 0x7f843c0cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.052 7f8442cc6700 1 -- 172.21.2.101:0/3432831675 >> v2:172.21.2.101:3300/0 conn(0x7f843c0d1c70 msgr2=0x7f843c0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.052 7f8442cc6700 1 --2- 172.21.2.101:0/3432831675 >> v2:172.21.2.101:3300/0 conn(0x7f843c0d1c70 0x7f843c0cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.052 7f8442cc6700 1 -- 172.21.2.101:0/3432831675 >> v1:172.21.2.108:6789/0 conn(0x7f843c0d1870 legacy=0x7f843c1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.052 7f8442cc6700 1 -- 172.21.2.101:0/3432831675 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c088070 con 0x7f843c0cf580 2021-02-10T11:49:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.052 7f8441cc4700 1 --2- 172.21.2.101:0/3432831675 >> v2:172.21.2.101:3300/0 conn(0x7f843c0d1c70 0x7f843c0cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:49:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.052 7f8442cc6700 1 --2- 172.21.2.101:0/3432831675 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f843c0cf580 0x7f843c0cf980 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f8434006e50 tx=0x7f8434009e70).ready entity=mon.2 client_cookie=5df29495ca291c53 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.053 7f84414c3700 1 -- 172.21.2.101:0/3432831675 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8434000f90 con 0x7f843c0cf580 2021-02-10T11:49:28.065 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.053 7f84414c3700 1 -- 172.21.2.101:0/3432831675 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84340066f0 con 0x7f843c0cf580 2021-02-10T11:49:28.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.053 7f84414c3700 1 -- 172.21.2.101:0/3432831675 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f843400b700 con 0x7f843c0cf580 2021-02-10T11:49:28.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.053 7f84446a9700 1 -- 172.21.2.101:0/3432831675 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f843c0cf580 msgr2=0x7f843c0cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:28.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.053 7f84446a9700 1 --2- 172.21.2.101:0/3432831675 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f843c0cf580 0x7f843c0cf980 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f8434006e50 tx=0x7f8434009e70).stop 2021-02-10T11:49:28.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.054 7f84446a9700 1 -- 172.21.2.101:0/3432831675 shutdown_connections 2021-02-10T11:49:28.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.054 7f84446a9700 1 --2- 172.21.2.101:0/3432831675 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f843c0cf580 0x7f843c0cf980 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:28.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.054 7f84446a9700 1 --2- 172.21.2.101:0/3432831675 >> v2:172.21.2.101:3300/0 conn(0x7f843c0d1c70 0x7f843c0cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:28.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.054 7f84446a9700 1 -- 172.21.2.101:0/3432831675 >> 172.21.2.101:0/3432831675 conn(0x7f843c0ce420 msgr2=0x7f843c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:28.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.054 7f84446a9700 1 -- 172.21.2.101:0/3432831675 shutdown_connections 2021-02-10T11:49:28.066 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.054 7f84446a9700 1 -- 172.21.2.101:0/3432831675 wait complete. 2021-02-10T11:49:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.068 7f84446a9700 1 Processor -- start 2021-02-10T11:49:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.068 7f84446a9700 1 -- start start 2021-02-10T11:49:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.069 7f84446a9700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f843c085bc0 0x7f843c0863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.069 7f84446a9700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f843c0954c0 0x7f843c096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:28.068 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.069 7f84446a9700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f843c0958c0 0x7f843c096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.069 7f84446a9700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c095d80 con 0x7f843c085bc0 2021-02-10T11:49:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.069 7f84446a9700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c0bfa60 con 0x7f843c0958c0 2021-02-10T11:49:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.069 7f84446a9700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843c085fc0 con 0x7f843c0954c0 2021-02-10T11:49:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.069 7f84424c5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f843c085bc0 0x7f843c0863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.069 7f84424c5700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f843c085bc0 0x7f843c0863e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35918/0 (socket says 172.21.2.101:35918) 2021-02-10T11:49:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.069 7f84424c5700 1 -- 172.21.2.101:0/3259949009 learned_addr learned my addr 172.21.2.101:0/3259949009 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.069 7f84424c5700 1 -- 172.21.2.101:0/3259949009 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f843c0954c0 msgr2=0x7f843c096300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.069 7f84424c5700 1 --2- 172.21.2.101:0/3259949009 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f843c0954c0 0x7f843c096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.069 7f84424c5700 1 -- 172.21.2.101:0/3259949009 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f843c0958c0 msgr2=0x7f843c096920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.069 7f84424c5700 1 --2- 172.21.2.101:0/3259949009 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f843c0958c0 0x7f843c096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.069 7f84424c5700 1 -- 172.21.2.101:0/3259949009 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c086c10 con 0x7f843c085bc0 2021-02-10T11:49:28.069 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.070 7f84424c5700 1 --2- 172.21.2.101:0/3259949009 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f843c085bc0 0x7f843c0863e0 secure :-1 s=READY pgs=3399 cs=0 l=1 rev1=1 rx=0x7f8438006ed0 tx=0x7f843800b8e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:28.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.070 7f8432ffd700 1 -- 172.21.2.101:0/3259949009 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8438006240 con 0x7f843c085bc0 2021-02-10T11:49:28.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.070 7f8432ffd700 1 -- 172.21.2.101:0/3259949009 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84380054b0 con 0x7f843c085bc0 2021-02-10T11:49:28.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.070 7f8432ffd700 1 -- 172.21.2.101:0/3259949009 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8438005990 con 0x7f843c085bc0 2021-02-10T11:49:28.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.070 7f84446a9700 1 -- 172.21.2.101:0/3259949009 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f843c086c10 con 0x7f843c085bc0 2021-02-10T11:49:28.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.070 7f84446a9700 1 -- 172.21.2.101:0/3259949009 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843c097230 con 0x7f843c085bc0 2021-02-10T11:49:28.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.071 7f8432ffd700 1 -- 172.21.2.101:0/3259949009 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8438006030 con 0x7f843c085bc0 2021-02-10T11:49:28.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.071 7f8432ffd700 1 --2- 172.21.2.101:0/3259949009 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84240186c0 0x7f842401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:28.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.071 7f8432ffd700 1 -- 172.21.2.101:0/3259949009 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8438032790 con 0x7f843c085bc0 2021-02-10T11:49:28.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.071 7f8441cc4700 1 --2- 172.21.2.101:0/3259949009 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84240186c0 0x7f842401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:28.070 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.071 7f8441cc4700 1 --2- 172.21.2.101:0/3259949009 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84240186c0 0x7f842401abc0 crc :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:28.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.101 7f84446a9700 1 -- 172.21.2.101:0/3259949009 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f843c0fe1c0 con 0x7f843c085bc0 2021-02-10T11:49:28.100 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.104 7f8432ffd700 1 -- 172.21.2.101:0/3259949009 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f843801f4d0 con 0x7f843c085bc0 2021-02-10T11:49:28.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.230 7f84446a9700 1 -- 172.21.2.101:0/3259949009 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f843c0fe1c0 con 0x7f843c085bc0 2021-02-10T11:49:28.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.231 7f8432ffd700 1 -- 172.21.2.101:0/3259949009 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v199) v1 ==== 100+0+10 (secure 0 0 0) 0x7f8438006030 con 0x7f843c085bc0 2021-02-10T11:49:28.228 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:49:28.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.233 7f84446a9700 1 -- 172.21.2.101:0/3259949009 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84240186c0 msgr2=0x7f842401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:28.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.233 7f84446a9700 1 --2- 172.21.2.101:0/3259949009 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84240186c0 0x7f842401abc0 crc :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:28.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.233 7f84446a9700 1 -- 172.21.2.101:0/3259949009 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f843c085bc0 msgr2=0x7f843c0863e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:28.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.233 7f84446a9700 1 --2- 172.21.2.101:0/3259949009 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f843c085bc0 0x7f843c0863e0 secure :-1 s=READY pgs=3399 cs=0 l=1 rev1=1 rx=0x7f8438006ed0 tx=0x7f843800b8e0).stop 2021-02-10T11:49:28.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.233 7f84446a9700 1 -- 172.21.2.101:0/3259949009 shutdown_connections 2021-02-10T11:49:28.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.233 7f84446a9700 1 --2- 172.21.2.101:0/3259949009 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f84240186c0 0x7f842401abc0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:28.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.233 7f84446a9700 1 --2- 172.21.2.101:0/3259949009 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f843c0954c0 0x7f843c096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:28.229 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.233 7f84446a9700 1 --2- 172.21.2.101:0/3259949009 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f843c0958c0 0x7f843c096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:28.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.233 7f84446a9700 1 --2- 172.21.2.101:0/3259949009 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f843c085bc0 0x7f843c0863e0 unknown :-1 s=CLOSED pgs=3399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:28.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.233 7f84446a9700 1 -- 172.21.2.101:0/3259949009 >> 172.21.2.101:0/3259949009 conn(0x7f843c0bef00 msgr2=0x7f843c09d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:28.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.233 7f84446a9700 1 -- 172.21.2.101:0/3259949009 shutdown_connections 2021-02-10T11:49:28.230 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.233 7f84446a9700 1 -- 172.21.2.101:0/3259949009 wait complete. 2021-02-10T11:49:28.238 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:49:28.238 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:49:28.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.655 7f773c564700 1 Processor -- start 2021-02-10T11:49:28.667 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.655 7f773c564700 1 -- start start 2021-02-10T11:49:28.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.656 7f773c564700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f77341b0470 0x7f77341b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:28.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.656 7f773c564700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f77340ef610 0x7f77340efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:28.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.656 7f773c564700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77340f0b50 con 0x7f77340f0030 2021-02-10T11:49:28.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.656 7f773c564700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77340e52e0 con 0x7f77340ef610 2021-02-10T11:49:28.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.656 7f773c564700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77340e5490 con 0x7f77341b0470 2021-02-10T11:49:28.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.656 7f773ab81700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f77340f0030 0x7f77340f10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36422/0 (socket says 172.21.2.101:36422) 2021-02-10T11:49:28.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.656 7f773ab81700 1 -- 172.21.2.101:0/1371647349 learned_addr learned my addr 172.21.2.101:0/1371647349 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:28.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.656 7f7739b7f700 1 --2- 172.21.2.101:0/1371647349 >> v2:172.21.2.101:3300/0 conn(0x7f77340ef610 0x7f77340efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:28.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.656 7f773a380700 1 --2- 172.21.2.101:0/1371647349 >> v2:172.21.2.108:3300/0 conn(0x7f77341b0470 0x7f77341b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:28.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.656 7f7739b7f700 1 -- 172.21.2.101:0/1371647349 >> v2:172.21.2.108:3300/0 conn(0x7f77341b0470 msgr2=0x7f77341b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:28.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.656 7f7739b7f700 1 --2- 172.21.2.101:0/1371647349 >> v2:172.21.2.108:3300/0 conn(0x7f77341b0470 0x7f77341b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:28.668 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.656 7f7739b7f700 1 -- 172.21.2.101:0/1371647349 >> v1:172.21.2.101:6789/0 conn(0x7f77340f0030 legacy=0x7f77340f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:28.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.656 7f7739b7f700 1 -- 172.21.2.101:0/1371647349 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77340f0430 con 0x7f77340ef610 2021-02-10T11:49:28.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.656 7f773937e700 1 -- 172.21.2.101:0/1371647349 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 915458493 0 0) 0x7f772c001500 con 0x7f77340f0030 2021-02-10T11:49:28.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.656 7f7739b7f700 1 --2- 172.21.2.101:0/1371647349 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77340ef610 0x7f77340efa10 secure :-1 s=READY pgs=3401 cs=0 l=1 rev1=1 rx=0x7f7730006650 tx=0x7f7730009a80).ready entity=mon.0 client_cookie=9f232e5f00e8733d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:28.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.657 7f773937e700 1 -- 172.21.2.101:0/1371647349 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f773000c0c0 con 0x7f77340ef610 2021-02-10T11:49:28.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.657 7f773937e700 1 -- 172.21.2.101:0/1371647349 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773000c340 con 0x7f77340ef610 2021-02-10T11:49:28.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.657 7f773937e700 1 -- 172.21.2.101:0/1371647349 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f773000c860 con 0x7f77340ef610 2021-02-10T11:49:28.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.657 7f773c564700 1 -- 172.21.2.101:0/1371647349 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77340ef610 msgr2=0x7f77340efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:28.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.657 7f773c564700 1 --2- 172.21.2.101:0/1371647349 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77340ef610 0x7f77340efa10 secure :-1 s=READY pgs=3401 cs=0 l=1 rev1=1 rx=0x7f7730006650 tx=0x7f7730009a80).stop 2021-02-10T11:49:28.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.657 7f773c564700 1 -- 172.21.2.101:0/1371647349 shutdown_connections 2021-02-10T11:49:28.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.657 7f773c564700 1 --2- 172.21.2.101:0/1371647349 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77340ef610 0x7f77340efa10 unknown :-1 s=CLOSED pgs=3401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:28.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.657 7f773c564700 1 --2- 172.21.2.101:0/1371647349 >> v2:172.21.2.108:3300/0 conn(0x7f77341b0470 0x7f77341b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:28.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.657 7f773c564700 1 -- 172.21.2.101:0/1371647349 >> 172.21.2.101:0/1371647349 conn(0x7f77340ce3e0 msgr2=0x7f77340e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:28.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.657 7f773c564700 1 -- 172.21.2.101:0/1371647349 shutdown_connections 2021-02-10T11:49:28.669 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.657 7f773c564700 1 -- 172.21.2.101:0/1371647349 wait complete. 2021-02-10T11:49:28.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.672 7f773c564700 1 Processor -- start 2021-02-10T11:49:28.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.672 7f773c564700 1 -- start start 2021-02-10T11:49:28.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.672 7f773c564700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f773408ebb0 0x7f773408efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:28.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.672 7f773c564700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77340b65c0 0x7f77340b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:28.671 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.672 7f773c564700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77340b6fe0 0x7f77340b7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:28.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.672 7f773c564700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77340b76d0 con 0x7f77340b65c0 2021-02-10T11:49:28.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.672 7f773c564700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773408f5d0 con 0x7f773408ebb0 2021-02-10T11:49:28.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.672 7f773c564700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77341a04d0 con 0x7f77340b6fe0 2021-02-10T11:49:28.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.672 7f7739b7f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77340b65c0 0x7f77340b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:28.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.672 7f7739b7f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77340b65c0 0x7f77340b69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35932/0 (socket says 172.21.2.101:35932) 2021-02-10T11:49:28.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.672 7f7739b7f700 1 -- 172.21.2.101:0/2364561767 learned_addr learned my addr 172.21.2.101:0/2364561767 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:28.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.672 7f773a380700 1 --2- 172.21.2.101:0/2364561767 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f773408ebb0 0x7f773408efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:28.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.672 7f773ab81700 1 --2- 172.21.2.101:0/2364561767 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77340b6fe0 0x7f77340b7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:28.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.673 7f7739b7f700 1 -- 172.21.2.101:0/2364561767 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77340b6fe0 msgr2=0x7f77340b7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:28.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.673 7f7739b7f700 1 --2- 172.21.2.101:0/2364561767 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77340b6fe0 0x7f77340b7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:28.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.673 7f7739b7f700 1 -- 172.21.2.101:0/2364561767 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f773408ebb0 msgr2=0x7f773408efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:28.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.673 7f7739b7f700 1 --2- 172.21.2.101:0/2364561767 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f773408ebb0 0x7f773408efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:28.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.673 7f7739b7f700 1 -- 172.21.2.101:0/2364561767 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77340b81f0 con 0x7f77340b65c0 2021-02-10T11:49:28.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.673 7f7739b7f700 1 --2- 172.21.2.101:0/2364561767 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77340b65c0 0x7f77340b69c0 secure :-1 s=READY pgs=3402 cs=0 l=1 rev1=1 rx=0x7f7730009630 tx=0x7f773000b900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:28.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.673 7f7726ffd700 1 -- 172.21.2.101:0/2364561767 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f77300093a0 con 0x7f77340b65c0 2021-02-10T11:49:28.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.673 7f7726ffd700 1 -- 172.21.2.101:0/2364561767 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773000bcf0 con 0x7f77340b65c0 2021-02-10T11:49:28.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.673 7f773c564700 1 -- 172.21.2.101:0/2364561767 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77340b81f0 con 0x7f77340b65c0 2021-02-10T11:49:28.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.673 7f7726ffd700 1 -- 172.21.2.101:0/2364561767 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f77300054d0 con 0x7f77340b65c0 2021-02-10T11:49:28.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.673 7f773c564700 1 -- 172.21.2.101:0/2364561767 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77340b81f0 con 0x7f77340b65c0 2021-02-10T11:49:28.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.674 7f7726ffd700 1 -- 172.21.2.101:0/2364561767 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7730008240 con 0x7f77340b65c0 2021-02-10T11:49:28.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.674 7f7726ffd700 1 --2- 172.21.2.101:0/2364561767 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f771c018730 0x7f771c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:28.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.674 7f7726ffd700 1 -- 172.21.2.101:0/2364561767 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7730032600 con 0x7f77340b65c0 2021-02-10T11:49:28.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.674 7f773a380700 1 --2- 172.21.2.101:0/2364561767 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f771c018730 0x7f771c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:28.673 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.674 7f773a380700 1 --2- 172.21.2.101:0/2364561767 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f771c018730 0x7f771c01abc0 crc :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:28.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.705 7f773c564700 1 -- 172.21.2.101:0/2364561767 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7734105a20 con 0x7f77340b65c0 2021-02-10T11:49:28.704 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.708 7f7726ffd700 1 -- 172.21.2.101:0/2364561767 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7730007f10 con 0x7f77340b65c0 2021-02-10T11:49:28.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.835 7f773c564700 1 -- 172.21.2.101:0/2364561767 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77340f0b50 con 0x7f77340b65c0 2021-02-10T11:49:28.832 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.835 7f7726ffd700 1 -- 172.21.2.101:0/2364561767 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) v1 ==== 95+0+2108 (secure 0 0 0) 0x7f773000bcf0 con 0x7f77340b65c0 2021-02-10T11:49:28.832 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:49:28.832 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":172,"default_fscid":21,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":13902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1207472796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1207472796},{"type":"v1","addr":"172.21.2.101:6835","nonce":1207472796}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":168}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:49:18.830789","modified":"2021-02-10 11:49:20.837623","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9100":{"gid":9100,"name":"b","rank":0,"incarnation":169,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/1721728696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":1721728696},{"type":"v1","addr":"172.21.2.108:6833","nonce":1721728696}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[43],"metadata_pool":42,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":21}]} 2021-02-10T11:49:28.833 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 172 2021-02-10T11:49:28.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.837 7f773c564700 1 -- 172.21.2.101:0/2364561767 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f771c018730 msgr2=0x7f771c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:28.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.837 7f773c564700 1 --2- 172.21.2.101:0/2364561767 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f771c018730 0x7f771c01abc0 crc :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:28.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.837 7f773c564700 1 -- 172.21.2.101:0/2364561767 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77340b65c0 msgr2=0x7f77340b69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:28.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.837 7f773c564700 1 --2- 172.21.2.101:0/2364561767 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77340b65c0 0x7f77340b69c0 secure :-1 s=READY pgs=3402 cs=0 l=1 rev1=1 rx=0x7f7730009630 tx=0x7f773000b900).stop 2021-02-10T11:49:28.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.838 7f773c564700 1 -- 172.21.2.101:0/2364561767 shutdown_connections 2021-02-10T11:49:28.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.838 7f773c564700 1 --2- 172.21.2.101:0/2364561767 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f771c018730 0x7f771c01abc0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:28.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.838 7f773c564700 1 --2- 172.21.2.101:0/2364561767 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f77340b6fe0 0x7f77340b7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:28.835 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.838 7f773c564700 1 --2- 172.21.2.101:0/2364561767 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f77340b65c0 0x7f77340b69c0 unknown :-1 s=CLOSED pgs=3402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:28.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.838 7f773c564700 1 --2- 172.21.2.101:0/2364561767 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f773408ebb0 0x7f773408efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:28.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.838 7f773c564700 1 -- 172.21.2.101:0/2364561767 >> 172.21.2.101:0/2364561767 conn(0x7f773409cb20 msgr2=0x7f77340c3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:28.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.838 7f773c564700 1 -- 172.21.2.101:0/2364561767 shutdown_connections 2021-02-10T11:49:28.836 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:28.838 7f773c564700 1 -- 172.21.2.101:0/2364561767 wait complete. 2021-02-10T11:49:29.044 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 172, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:49:18.830789', 'modified': '2021-02-10 11:49:20.837623', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9100}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9100': {'gid': 9100, 'name': 'b', 'rank': 0, 'incarnation': 169, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/1721728696', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 1721728696}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 1721728696}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [43], 'metadata_pool': 42, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:49:29.045 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:49:29.045 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:49:29.441 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:49:29.441 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:49:29.441 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:49:29.441 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 9100, 2021-02-10T11:49:29.441 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:49:29.442 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:49:29.442 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 10.612131805000001, 2021-02-10T11:49:29.442 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 172, 2021-02-10T11:49:29.442 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 199, 2021-02-10T11:49:29.442 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 199, 2021-02-10T11:49:29.442 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 16.281250351000001 2021-02-10T11:49:29.442 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:49:29.445 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 9100, "want_state": "up:active", "state": "up:active", "rank_uptime": 10.612131805000001, "mdsmap_epoch": 172, "osdmap_epoch": 199, "osdmap_epoch_barrier": 199, "uptime": 16.281250351000001 } 2021-02-10T11:49:29.445 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:49:29.445 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-02-10T11:49:29.446 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-02-10T11:49:29.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:29.446 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:29.783 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:49:30.127 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:49:30.127 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:49:30.471 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:49:30.542 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:49:30.542 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:49:30.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:30.543 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:49:30.878 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-02-10T11:49:30.878 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:30.881 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:49:31.226 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:49:31.227 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:49:31.294 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[186294]: starting ceph client 2021-02-10T11:49:31.294 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:2021-02-10 11:49:31.297 7f5b43c16f80 -1 init, newargv = 0x5628682721f0 newargc=9 2021-02-10T11:49:31.300 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[186294]: starting fuse 2021-02-10T11:49:31.568 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:49:31.569 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:49:31.569 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:49:31.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:31.570 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:49:31.904 INFO:teuthology.orchestra.run.gibba001.stdout:42 2021-02-10T11:49:31.905 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2021-02-10T11:49:31.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:31.905 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:49:31.905 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:49:31.905 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:49:31.905 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:49:31.905 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:49:31.905 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:49:31.905 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:49:31.906 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-02-10T11:49:31.906 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:49:31.906 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:49:31.906 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:49:31.906 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:49:31.906 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:49:31.906 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:49:31.906 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:49:31.906 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:49:31.906 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:49:31.906 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:49:31.906 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:49:31.906 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:49:31.906 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:49:31.906 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.0")) 2021-02-10T11:49:31.907 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:49:32.270 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.0.186294.asok 2021-02-10T11:49:32.273 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.186294.asok 2021-02-10T11:49:32.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:32.273 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.186294.asok status 2021-02-10T11:49:32.667 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:49:32.667 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:49:32.667 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:49:32.667 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:49:32.667 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "0", 2021-02-10T11:49:32.668 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:49:32.668 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-02-10T11:49:32.668 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "186294", 2021-02-10T11:49:32.668 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:49:32.668 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:32.668 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:49:32.668 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:49:32.668 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 14064, 2021-02-10T11:49:32.668 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:49:32.668 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:49:32.668 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:49:32.668 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 14064 2021-02-10T11:49:32.668 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:32.669 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:49:32.669 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:32.669 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:49:32.669 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1792741295 2021-02-10T11:49:32.669 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:32.669 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:32.669 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:49:32.669 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:32.669 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:49:32.669 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 1792741295 2021-02-10T11:49:32.669 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:32.669 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.14064 172.21.2.101:0/1792741295", 2021-02-10T11:49:32.670 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/1792741295", 2021-02-10T11:49:32.670 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:49:32.670 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 172, 2021-02-10T11:49:32.670 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 199, 2021-02-10T11:49:32.670 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:49:32.670 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:49:32.670 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:49:32.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:32.672 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:32.672 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:33.008 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:49:33.008 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:49:33.008 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:33.008 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:33.355 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:49:33.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.771 7f082a0c8700 1 Processor -- start 2021-02-10T11:49:33.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.771 7f082a0c8700 1 -- start start 2021-02-10T11:49:33.783 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.771 7f082a0c8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f08240d0340 0x7f08240ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:33.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.771 7f082a0c8700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08240f0b10 con 0x7f08240d1c70 2021-02-10T11:49:33.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.771 7f082a0c8700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08241b0be0 con 0x7f08240d1870 2021-02-10T11:49:33.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.771 7f082a0c8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08240e5ed0 con 0x7f08240d0340 2021-02-10T11:49:33.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.772 7f0822ffd700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f08240d0340 0x7f08240ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:33.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.772 7f0823fff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f08240d1c70 0x7f08240efc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36444/0 (socket says 172.21.2.101:36444) 2021-02-10T11:49:33.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.772 7f0823fff700 1 -- 172.21.2.101:0/3890003383 learned_addr learned my addr 172.21.2.101:0/3890003383 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:33.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.772 7f0822ffd700 1 -- 172.21.2.101:0/3890003383 >> v1:172.21.2.108:6789/0 conn(0x7f08240d1870 legacy=0x7f08241b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:49:33.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.772 7f0822ffd700 1 -- 172.21.2.101:0/3890003383 >> v1:172.21.2.101:6789/0 conn(0x7f08240d1c70 legacy=0x7f08240efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:33.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.772 7f0822ffd700 1 -- 172.21.2.101:0/3890003383 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08240f0450 con 0x7f08240d0340 2021-02-10T11:49:33.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.772 7f0822ffd700 1 --2- 172.21.2.101:0/3890003383 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08240d0340 0x7f08240ef640 secure :-1 s=READY pgs=3406 cs=0 l=1 rev1=1 rx=0x7f080c006650 tx=0x7f080c009a00).ready entity=mon.0 client_cookie=1cb182aa212d63df server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:33.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.772 7f08227fc700 1 -- 172.21.2.101:0/3890003383 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f080c0009b0 con 0x7f08240d0340 2021-02-10T11:49:33.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.772 7f08227fc700 1 -- 172.21.2.101:0/3890003383 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080c00c0c0 con 0x7f08240d0340 2021-02-10T11:49:33.784 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.773 7f08227fc700 1 -- 172.21.2.101:0/3890003383 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f080c00c5a0 con 0x7f08240d0340 2021-02-10T11:49:33.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.773 7f082a0c8700 1 -- 172.21.2.101:0/3890003383 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08240d0340 msgr2=0x7f08240ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:33.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.773 7f082a0c8700 1 --2- 172.21.2.101:0/3890003383 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08240d0340 0x7f08240ef640 secure :-1 s=READY pgs=3406 cs=0 l=1 rev1=1 rx=0x7f080c006650 tx=0x7f080c009a00).stop 2021-02-10T11:49:33.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.773 7f082a0c8700 1 -- 172.21.2.101:0/3890003383 shutdown_connections 2021-02-10T11:49:33.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.773 7f082a0c8700 1 --2- 172.21.2.101:0/3890003383 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f08240d0340 0x7f08240ef640 unknown :-1 s=CLOSED pgs=3406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:33.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.773 7f082a0c8700 1 -- 172.21.2.101:0/3890003383 >> 172.21.2.101:0/3890003383 conn(0x7f08240ce450 msgr2=0x7f08240d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:33.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.773 7f082a0c8700 1 -- 172.21.2.101:0/3890003383 shutdown_connections 2021-02-10T11:49:33.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.773 7f082a0c8700 1 -- 172.21.2.101:0/3890003383 wait complete. 2021-02-10T11:49:33.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.788 7f082a0c8700 1 Processor -- start 2021-02-10T11:49:33.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.788 7f082a0c8700 1 -- start start 2021-02-10T11:49:33.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.788 7f082a0c8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0824103520 0x7f08240c3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:33.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.788 7f082a0c8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0824103920 0x7f08240b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:33.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.788 7f082a0c8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f082408a450 0x7f082408a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:33.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.788 7f082a0c8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08240c3c60 con 0x7f0824103920 2021-02-10T11:49:33.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.788 7f082a0c8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08240b38f0 con 0x7f082408a450 2021-02-10T11:49:33.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.788 7f082a0c8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08240c0be0 con 0x7f0824103520 2021-02-10T11:49:33.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.788 7f0822ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0824103920 0x7f08240b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:33.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.788 7f0822ffd700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0824103920 0x7f08240b32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35954/0 (socket says 172.21.2.101:35954) 2021-02-10T11:49:33.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.788 7f0822ffd700 1 -- 172.21.2.101:0/1088004698 learned_addr learned my addr 172.21.2.101:0/1088004698 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:33.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.789 7f0822ffd700 1 -- 172.21.2.101:0/1088004698 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0824103520 msgr2=0x7f08240c3640 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:49:33.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.789 7f0822ffd700 1 --2- 172.21.2.101:0/1088004698 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0824103520 0x7f08240c3640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:33.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.789 7f0822ffd700 1 -- 172.21.2.101:0/1088004698 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f082408a450 msgr2=0x7f082408a850 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:49:33.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.789 7f0822ffd700 1 --2- 172.21.2.101:0/1088004698 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f082408a450 0x7f082408a850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:33.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.789 7f0822ffd700 1 -- 172.21.2.101:0/1088004698 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082408ae70 con 0x7f0824103920 2021-02-10T11:49:33.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.789 7f0822ffd700 1 --2- 172.21.2.101:0/1088004698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0824103920 0x7f08240b32d0 secure :-1 s=READY pgs=3407 cs=0 l=1 rev1=1 rx=0x7f080c00b590 tx=0x7f080c00b930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:33.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.789 7f081bfff700 1 -- 172.21.2.101:0/1088004698 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f080c00bd20 con 0x7f0824103920 2021-02-10T11:49:33.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.789 7f081bfff700 1 -- 172.21.2.101:0/1088004698 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080c000c90 con 0x7f0824103920 2021-02-10T11:49:33.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.789 7f081bfff700 1 -- 172.21.2.101:0/1088004698 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f080c0022b0 con 0x7f0824103920 2021-02-10T11:49:33.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.789 7f082a0c8700 1 -- 172.21.2.101:0/1088004698 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f082408ae70 con 0x7f0824103920 2021-02-10T11:49:33.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.789 7f082a0c8700 1 -- 172.21.2.101:0/1088004698 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082408ae70 con 0x7f0824103920 2021-02-10T11:49:33.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.790 7f081bfff700 1 -- 172.21.2.101:0/1088004698 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f080c0052e0 con 0x7f0824103920 2021-02-10T11:49:33.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.790 7f081bfff700 1 --2- 172.21.2.101:0/1088004698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0808018730 0x7f080801abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:33.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.790 7f081bfff700 1 -- 172.21.2.101:0/1088004698 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f080c0327c0 con 0x7f0824103920 2021-02-10T11:49:33.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.790 7f08237fe700 1 --2- 172.21.2.101:0/1088004698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0808018730 0x7f080801abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:33.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.791 7f08237fe700 1 --2- 172.21.2.101:0/1088004698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0808018730 0x7f080801abf0 crc :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:33.817 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.821 7f082a0c8700 1 -- 172.21.2.101:0/1088004698 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08240f06f0 con 0x7f0824103920 2021-02-10T11:49:33.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.824 7f081bfff700 1 -- 172.21.2.101:0/1088004698 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f080c006ac0 con 0x7f0824103920 2021-02-10T11:49:33.943 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.947 7f082a0c8700 1 -- 172.21.2.101:0/1088004698 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08240f06f0 con 0x7f0824103920 2021-02-10T11:49:33.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.948 7f081bfff700 1 -- 172.21.2.101:0/1088004698 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f080c01f500 con 0x7f0824103920 2021-02-10T11:49:33.945 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:49:33.945 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":199,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:49:19.833311","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":43,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":42,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:49:16.791877","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":43,"pool_name":"cephfs_data","create_time":"2021-02-10 11:49:17.820338","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:49:33.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.949 7f082a0c8700 1 -- 172.21.2.101:0/1088004698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0808018730 msgr2=0x7f080801abf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:33.946 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.949 7f082a0c8700 1 --2- 172.21.2.101:0/1088004698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0808018730 0x7f080801abf0 crc :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:33.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.949 7f082a0c8700 1 -- 172.21.2.101:0/1088004698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0824103920 msgr2=0x7f08240b32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:33.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.949 7f082a0c8700 1 --2- 172.21.2.101:0/1088004698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0824103920 0x7f08240b32d0 secure :-1 s=READY pgs=3407 cs=0 l=1 rev1=1 rx=0x7f080c00b590 tx=0x7f080c00b930).stop 2021-02-10T11:49:33.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.950 7f082a0c8700 1 -- 172.21.2.101:0/1088004698 shutdown_connections 2021-02-10T11:49:33.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.950 7f082a0c8700 1 --2- 172.21.2.101:0/1088004698 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f082408a450 0x7f082408a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:33.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.950 7f082a0c8700 1 --2- 172.21.2.101:0/1088004698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f0824103920 0x7f08240b32d0 unknown :-1 s=CLOSED pgs=3407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:33.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.950 7f082a0c8700 1 --2- 172.21.2.101:0/1088004698 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f0808018730 0x7f080801abf0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:33.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.950 7f082a0c8700 1 --2- 172.21.2.101:0/1088004698 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f0824103520 0x7f08240c3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:33.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.950 7f082a0c8700 1 -- 172.21.2.101:0/1088004698 >> 172.21.2.101:0/1088004698 conn(0x7f082410a820 msgr2=0x7f0824108150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:33.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.950 7f082a0c8700 1 -- 172.21.2.101:0/1088004698 shutdown_connections 2021-02-10T11:49:33.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:33.950 7f082a0c8700 1 -- 172.21.2.101:0/1088004698 wait complete. 2021-02-10T11:49:33.954 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2021-02-10 11:13:08.037472', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 42, 'pool_name': 'cephfs_metadata', 'create_time': '2021-02-10 11:49:16.791877', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '198', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 43, 'pool_name': 'cephfs_data', 'create_time': '2021-02-10 11:49:17.820338', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '199', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-02-10T11:49:33.955 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get rbd pg_num 2021-02-10T11:49:34.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.375 7fe6f3d9b700 1 Processor -- start 2021-02-10T11:49:34.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.375 7fe6f3d9b700 1 -- start start 2021-02-10T11:49:34.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.376 7fe6f3d9b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe6ec10aff0 0x7fe6ec0cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:34.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.376 7fe6f3d9b700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6ec0fe1c0 con 0x7fe6ec0d1c70 2021-02-10T11:49:34.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.376 7fe6f3d9b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6ec1b0cb0 con 0x7fe6ec0d1870 2021-02-10T11:49:34.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.376 7fe6f3d9b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6ec109a60 con 0x7fe6ec10aff0 2021-02-10T11:49:34.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.376 7fe6f23b8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fe6ec10aff0 0x7fe6ec0cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:34.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.376 7fe6f13b6700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fe6ec0d1c70 0x7fe6ec0cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36456/0 (socket says 172.21.2.101:36456) 2021-02-10T11:49:34.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.376 7fe6f13b6700 1 -- 172.21.2.101:0/3026896458 learned_addr learned my addr 172.21.2.101:0/3026896458 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:34.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.377 7fe6f23b8700 1 -- 172.21.2.101:0/3026896458 >> v1:172.21.2.108:6789/0 conn(0x7fe6ec0d1870 legacy=0x7fe6ec1b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:34.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.377 7fe6f23b8700 1 -- 172.21.2.101:0/3026896458 >> v1:172.21.2.101:6789/0 conn(0x7fe6ec0d1c70 legacy=0x7fe6ec0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:34.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.377 7fe6f23b8700 1 -- 172.21.2.101:0/3026896458 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ec10b3f0 con 0x7fe6ec10aff0 2021-02-10T11:49:34.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.377 7fe6f23b8700 1 --2- 172.21.2.101:0/3026896458 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6ec10aff0 0x7fe6ec0cf750 secure :-1 s=READY pgs=3409 cs=0 l=1 rev1=1 rx=0x7fe6e4006e30 tx=0x7fe6e4009e60).ready entity=mon.0 client_cookie=d48f1fef2eba3441 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:34.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.377 7fe6f0bb5700 1 -- 172.21.2.101:0/3026896458 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe6e4006a50 con 0x7fe6ec10aff0 2021-02-10T11:49:34.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.377 7fe6f0bb5700 1 -- 172.21.2.101:0/3026896458 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e400b300 con 0x7fe6ec10aff0 2021-02-10T11:49:34.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.377 7fe6f0bb5700 1 -- 172.21.2.101:0/3026896458 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe6e400b820 con 0x7fe6ec10aff0 2021-02-10T11:49:34.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.377 7fe6f3d9b700 1 -- 172.21.2.101:0/3026896458 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6ec10aff0 msgr2=0x7fe6ec0cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:34.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.377 7fe6f3d9b700 1 --2- 172.21.2.101:0/3026896458 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6ec10aff0 0x7fe6ec0cf750 secure :-1 s=READY pgs=3409 cs=0 l=1 rev1=1 rx=0x7fe6e4006e30 tx=0x7fe6e4009e60).stop 2021-02-10T11:49:34.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.377 7fe6f3d9b700 1 -- 172.21.2.101:0/3026896458 shutdown_connections 2021-02-10T11:49:34.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.377 7fe6f3d9b700 1 --2- 172.21.2.101:0/3026896458 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6ec10aff0 0x7fe6ec0cf750 unknown :-1 s=CLOSED pgs=3409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:34.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.377 7fe6f3d9b700 1 -- 172.21.2.101:0/3026896458 >> 172.21.2.101:0/3026896458 conn(0x7fe6ec0ce420 msgr2=0x7fe6ec0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:34.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.378 7fe6f3d9b700 1 -- 172.21.2.101:0/3026896458 shutdown_connections 2021-02-10T11:49:34.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.378 7fe6f3d9b700 1 -- 172.21.2.101:0/3026896458 wait complete. 2021-02-10T11:49:34.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.392 7fe6f3d9b700 1 Processor -- start 2021-02-10T11:49:34.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.392 7fe6f3d9b700 1 -- start start 2021-02-10T11:49:34.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.393 7fe6f3d9b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6ec0c0740 0x7fe6ec1c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:34.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.393 7fe6f3d9b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6ec1c5ba0 0x7fe6ec0a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:34.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.393 7fe6f3d9b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6ec0b6e60 0x7fe6ec0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:34.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.393 7fe6f3d9b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec0a6880 con 0x7fe6ec0b6e60 2021-02-10T11:49:34.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.393 7fe6f3d9b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec0a6aa0 con 0x7fe6ec1c5ba0 2021-02-10T11:49:34.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.393 7fe6f3d9b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec0a6cc0 con 0x7fe6ec0c0740 2021-02-10T11:49:34.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.393 7fe6f23b8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6ec0b6e60 0x7fe6ec0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:34.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.393 7fe6f23b8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6ec0b6e60 0x7fe6ec0a6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35972/0 (socket says 172.21.2.101:35972) 2021-02-10T11:49:34.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.393 7fe6f23b8700 1 -- 172.21.2.101:0/564870638 learned_addr learned my addr 172.21.2.101:0/564870638 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:34.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.393 7fe6f23b8700 1 -- 172.21.2.101:0/564870638 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6ec0c0740 msgr2=0x7fe6ec1c5580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:49:34.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.393 7fe6f23b8700 1 --2- 172.21.2.101:0/564870638 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6ec0c0740 0x7fe6ec1c5580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:34.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.393 7fe6f23b8700 1 -- 172.21.2.101:0/564870638 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6ec1c5ba0 msgr2=0x7fe6ec0a5c40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:49:34.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.393 7fe6f23b8700 1 --2- 172.21.2.101:0/564870638 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6ec1c5ba0 0x7fe6ec0a5c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:34.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.393 7fe6f23b8700 1 -- 172.21.2.101:0/564870638 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ec0a6ee0 con 0x7fe6ec0b6e60 2021-02-10T11:49:34.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.393 7fe6f23b8700 1 --2- 172.21.2.101:0/564870638 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6ec0b6e60 0x7fe6ec0a6260 secure :-1 s=READY pgs=3410 cs=0 l=1 rev1=1 rx=0x7fe6e400b970 tx=0x7fe6e400bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:34.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.394 7fe6de7fc700 1 -- 172.21.2.101:0/564870638 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe6e40093d0 con 0x7fe6ec0b6e60 2021-02-10T11:49:34.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.394 7fe6de7fc700 1 -- 172.21.2.101:0/564870638 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e4005820 con 0x7fe6ec0b6e60 2021-02-10T11:49:34.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.394 7fe6de7fc700 1 -- 172.21.2.101:0/564870638 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe6e4005d00 con 0x7fe6ec0b6e60 2021-02-10T11:49:34.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.394 7fe6f3d9b700 1 -- 172.21.2.101:0/564870638 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6ec0a6ee0 con 0x7fe6ec0b6e60 2021-02-10T11:49:34.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.394 7fe6f3d9b700 1 -- 172.21.2.101:0/564870638 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6ec0a6ee0 con 0x7fe6ec0b6e60 2021-02-10T11:49:34.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.394 7fe6de7fc700 1 -- 172.21.2.101:0/564870638 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fe6e4005a00 con 0x7fe6ec0b6e60 2021-02-10T11:49:34.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.394 7fe6de7fc700 1 --2- 172.21.2.101:0/564870638 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6d40186c0 0x7fe6d401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:34.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.394 7fe6de7fc700 1 -- 172.21.2.101:0/564870638 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe6e4032950 con 0x7fe6ec0b6e60 2021-02-10T11:49:34.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.394 7fe6f1bb7700 1 --2- 172.21.2.101:0/564870638 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6d40186c0 0x7fe6d401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:34.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.395 7fe6f1bb7700 1 --2- 172.21.2.101:0/564870638 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6d40186c0 0x7fe6d401abc0 crc :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:34.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.426 7fe6f3d9b700 1 -- 172.21.2.101:0/564870638 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6ec0807d0 con 0x7fe6ec0b6e60 2021-02-10T11:49:34.424 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.428 7fe6de7fc700 1 -- 172.21.2.101:0/564870638 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fe6e4000f60 con 0x7fe6ec0b6e60 2021-02-10T11:49:34.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.554 7fe6f3d9b700 1 -- 172.21.2.101:0/564870638 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"} v 0) v1 -- 0x7fe6ec088070 con 0x7fe6ec0b6e60 2021-02-10T11:49:34.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.555 7fe6de7fc700 1 -- 172.21.2.101:0/564870638 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "rbd"}]=0 v199) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe6e40059e0 con 0x7fe6ec0b6e60 2021-02-10T11:49:34.552 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:49:34.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.557 7fe6f3d9b700 1 -- 172.21.2.101:0/564870638 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6d40186c0 msgr2=0x7fe6d401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:34.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.557 7fe6f3d9b700 1 --2- 172.21.2.101:0/564870638 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6d40186c0 0x7fe6d401abc0 crc :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:34.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.557 7fe6f3d9b700 1 -- 172.21.2.101:0/564870638 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6ec0b6e60 msgr2=0x7fe6ec0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:34.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.557 7fe6f3d9b700 1 --2- 172.21.2.101:0/564870638 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6ec0b6e60 0x7fe6ec0a6260 secure :-1 s=READY pgs=3410 cs=0 l=1 rev1=1 rx=0x7fe6e400b970 tx=0x7fe6e400bd10).stop 2021-02-10T11:49:34.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.557 7fe6f3d9b700 1 -- 172.21.2.101:0/564870638 shutdown_connections 2021-02-10T11:49:34.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.557 7fe6f3d9b700 1 --2- 172.21.2.101:0/564870638 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fe6ec0b6e60 0x7fe6ec0a6260 unknown :-1 s=CLOSED pgs=3410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:34.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.557 7fe6f3d9b700 1 --2- 172.21.2.101:0/564870638 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fe6ec1c5ba0 0x7fe6ec0a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:34.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.557 7fe6f3d9b700 1 --2- 172.21.2.101:0/564870638 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fe6d40186c0 0x7fe6d401abc0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:34.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.557 7fe6f3d9b700 1 --2- 172.21.2.101:0/564870638 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fe6ec0c0740 0x7fe6ec1c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:34.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.557 7fe6f3d9b700 1 -- 172.21.2.101:0/564870638 >> 172.21.2.101:0/564870638 conn(0x7fe6ec0c4120 msgr2=0x7fe6ec0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:34.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.557 7fe6f3d9b700 1 -- 172.21.2.101:0/564870638 shutdown_connections 2021-02-10T11:49:34.554 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.557 7fe6f3d9b700 1 -- 172.21.2.101:0/564870638 wait complete. 2021-02-10T11:49:34.561 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-02-10T11:49:34.996 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.983 7f73f6a3c700 1 Processor -- start 2021-02-10T11:49:34.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.983 7f73f6a3c700 1 -- start start 2021-02-10T11:49:34.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.983 7f73f6a3c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73f00d1c70 0x7f73f00cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:34.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.983 7f73f6a3c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f73f00cf580 0x7f73f00cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:34.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.983 7f73f6a3c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73f00fe1c0 con 0x7f73f00d1870 2021-02-10T11:49:34.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.983 7f73f6a3c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73f01b0cb0 con 0x7f73f00cf580 2021-02-10T11:49:34.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.983 7f73f6a3c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f0109a60 con 0x7f73f00d1c70 2021-02-10T11:49:34.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.984 7f73f4858700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f73f00d1870 0x7f73f01b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36470/0 (socket says 172.21.2.101:36470) 2021-02-10T11:49:34.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.984 7f73f4858700 1 -- 172.21.2.101:0/4288276727 learned_addr learned my addr 172.21.2.101:0/4288276727 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:34.997 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.984 7f73f5059700 1 --2- 172.21.2.101:0/4288276727 >> v2:172.21.2.108:3300/0 conn(0x7f73f00cf580 0x7f73f00cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:34.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.984 7f73effff700 1 --2- 172.21.2.101:0/4288276727 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73f00d1c70 0x7f73f00cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:34.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.984 7f73ef7fe700 1 -- 172.21.2.101:0/4288276727 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 699120593 0 0) 0x7f73d8000ce0 con 0x7f73f00d1870 2021-02-10T11:49:34.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.984 7f73ef7fe700 1 -- 172.21.2.101:0/4288276727 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73dc002fe0 con 0x7f73f00d1870 2021-02-10T11:49:34.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.984 7f73f5059700 1 -- 172.21.2.101:0/4288276727 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73f00d1c70 msgr2=0x7f73f00cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:34.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.984 7f73f5059700 1 --2- 172.21.2.101:0/4288276727 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73f00d1c70 0x7f73f00cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:34.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.984 7f73f5059700 1 -- 172.21.2.101:0/4288276727 >> v1:172.21.2.101:6789/0 conn(0x7f73f00d1870 legacy=0x7f73f01b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:34.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.984 7f73f5059700 1 -- 172.21.2.101:0/4288276727 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f0088070 con 0x7f73f00cf580 2021-02-10T11:49:34.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.984 7f73effff700 1 --2- 172.21.2.101:0/4288276727 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73f00d1c70 0x7f73f00cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:49:34.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.985 7f73f5059700 1 --2- 172.21.2.101:0/4288276727 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73f00cf580 0x7f73f00cf980 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f73e8006ad0 tx=0x7f73e8009ad0).ready entity=mon.1 client_cookie=c5538904c94643b2 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:34.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.985 7f73ef7fe700 1 -- 172.21.2.101:0/4288276727 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73e80066a0 con 0x7f73f00cf580 2021-02-10T11:49:34.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.985 7f73ef7fe700 1 -- 172.21.2.101:0/4288276727 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e8006410 con 0x7f73f00cf580 2021-02-10T11:49:34.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.985 7f73ef7fe700 1 -- 172.21.2.101:0/4288276727 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73e80024d0 con 0x7f73f00cf580 2021-02-10T11:49:34.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.986 7f73f6a3c700 1 -- 172.21.2.101:0/4288276727 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73f00cf580 msgr2=0x7f73f00cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:34.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.986 7f73f6a3c700 1 --2- 172.21.2.101:0/4288276727 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73f00cf580 0x7f73f00cf980 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f73e8006ad0 tx=0x7f73e8009ad0).stop 2021-02-10T11:49:34.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.986 7f73f6a3c700 1 -- 172.21.2.101:0/4288276727 shutdown_connections 2021-02-10T11:49:34.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.986 7f73f6a3c700 1 --2- 172.21.2.101:0/4288276727 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73f00d1c70 0x7f73f00cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:34.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.986 7f73f6a3c700 1 --2- 172.21.2.101:0/4288276727 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73f00cf580 0x7f73f00cf980 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:34.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.986 7f73f6a3c700 1 -- 172.21.2.101:0/4288276727 >> 172.21.2.101:0/4288276727 conn(0x7f73f00ce420 msgr2=0x7f73f00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:34.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.986 7f73f6a3c700 1 -- 172.21.2.101:0/4288276727 shutdown_connections 2021-02-10T11:49:34.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:34.986 7f73f6a3c700 1 -- 172.21.2.101:0/4288276727 wait complete. 2021-02-10T11:49:35.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.001 7f73f6a3c700 1 Processor -- start 2021-02-10T11:49:35.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.001 7f73f6a3c700 1 -- start start 2021-02-10T11:49:35.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.002 7f73f6a3c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73f0085bc0 0x7f73f00863e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:35.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.002 7f73f6a3c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73f00954c0 0x7f73f0096300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:35.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.002 7f73f6a3c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73f00958c0 0x7f73f0096920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:35.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.002 7f73f6a3c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f0095d80 con 0x7f73f00954c0 2021-02-10T11:49:35.001 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.002 7f73f6a3c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f00bfa60 con 0x7f73f0085bc0 2021-02-10T11:49:35.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.002 7f73f6a3c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f0085fc0 con 0x7f73f00958c0 2021-02-10T11:49:35.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.002 7f73effff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73f00954c0 0x7f73f0096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:35.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.002 7f73effff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73f00954c0 0x7f73f0096300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35980/0 (socket says 172.21.2.101:35980) 2021-02-10T11:49:35.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.002 7f73effff700 1 -- 172.21.2.101:0/3649608897 learned_addr learned my addr 172.21.2.101:0/3649608897 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:35.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.002 7f73f4858700 1 --2- 172.21.2.101:0/3649608897 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73f0085bc0 0x7f73f00863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:35.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.002 7f73f5059700 1 --2- 172.21.2.101:0/3649608897 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73f00958c0 0x7f73f0096920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:35.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.002 7f73effff700 1 -- 172.21.2.101:0/3649608897 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73f00958c0 msgr2=0x7f73f0096920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:35.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.002 7f73effff700 1 --2- 172.21.2.101:0/3649608897 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73f00958c0 0x7f73f0096920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:35.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.002 7f73effff700 1 -- 172.21.2.101:0/3649608897 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73f0085bc0 msgr2=0x7f73f00863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:35.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.002 7f73effff700 1 --2- 172.21.2.101:0/3649608897 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73f0085bc0 0x7f73f00863e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:35.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.002 7f73effff700 1 -- 172.21.2.101:0/3649608897 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f0086c10 con 0x7f73f00954c0 2021-02-10T11:49:35.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.003 7f73effff700 1 --2- 172.21.2.101:0/3649608897 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73f00954c0 0x7f73f0096300 secure :-1 s=READY pgs=3412 cs=0 l=1 rev1=1 rx=0x7f73e000b970 tx=0x7f73e000bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:35.002 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.004 7f73ed7fa700 1 -- 172.21.2.101:0/3649608897 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73e0000d30 con 0x7f73f00954c0 2021-02-10T11:49:35.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.004 7f73ed7fa700 1 -- 172.21.2.101:0/3649608897 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e0006240 con 0x7f73f00954c0 2021-02-10T11:49:35.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.004 7f73ed7fa700 1 -- 172.21.2.101:0/3649608897 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73e0005660 con 0x7f73f00954c0 2021-02-10T11:49:35.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.004 7f73f6a3c700 1 -- 172.21.2.101:0/3649608897 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f0086c10 con 0x7f73f00954c0 2021-02-10T11:49:35.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.004 7f73f6a3c700 1 -- 172.21.2.101:0/3649608897 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f0086c10 con 0x7f73f00954c0 2021-02-10T11:49:35.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.005 7f73ed7fa700 1 -- 172.21.2.101:0/3649608897 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f73e0005660 con 0x7f73f00954c0 2021-02-10T11:49:35.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.005 7f73ed7fa700 1 --2- 172.21.2.101:0/3649608897 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73d40186c0 0x7f73d401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:35.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.005 7f73ed7fa700 1 -- 172.21.2.101:0/3649608897 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f73e0032ba0 con 0x7f73f00954c0 2021-02-10T11:49:35.003 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.005 7f73f4858700 1 --2- 172.21.2.101:0/3649608897 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73d40186c0 0x7f73d401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:35.004 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.005 7f73f4858700 1 --2- 172.21.2.101:0/3649608897 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73d40186c0 0x7f73d401abc0 crc :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:35.029 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.033 7f73f6a3c700 1 -- 172.21.2.101:0/3649608897 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73f00fe1c0 con 0x7f73f00954c0 2021-02-10T11:49:35.032 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.036 7f73ed7fa700 1 -- 172.21.2.101:0/3649608897 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f73e0006240 con 0x7f73f00954c0 2021-02-10T11:49:35.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.165 7f73f6a3c700 1 -- 172.21.2.101:0/3649608897 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"} v 0) v1 -- 0x7f73f00fe1c0 con 0x7f73f00954c0 2021-02-10T11:49:35.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.165 7f73ed7fa700 1 -- 172.21.2.101:0/3649608897 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_metadata"}]=0 v199) v1 ==== 104+0+10 (secure 0 0 0) 0x7f73e0006450 con 0x7f73f00954c0 2021-02-10T11:49:35.162 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:49:35.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.167 7f73f6a3c700 1 -- 172.21.2.101:0/3649608897 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73d40186c0 msgr2=0x7f73d401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:35.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.167 7f73f6a3c700 1 --2- 172.21.2.101:0/3649608897 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73d40186c0 0x7f73d401abc0 crc :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:35.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.167 7f73f6a3c700 1 -- 172.21.2.101:0/3649608897 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73f00954c0 msgr2=0x7f73f0096300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:35.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.167 7f73f6a3c700 1 --2- 172.21.2.101:0/3649608897 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73f00954c0 0x7f73f0096300 secure :-1 s=READY pgs=3412 cs=0 l=1 rev1=1 rx=0x7f73e000b970 tx=0x7f73e000bd10).stop 2021-02-10T11:49:35.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.168 7f73f6a3c700 1 -- 172.21.2.101:0/3649608897 shutdown_connections 2021-02-10T11:49:35.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.168 7f73f6a3c700 1 --2- 172.21.2.101:0/3649608897 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f73d40186c0 0x7f73d401abc0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:35.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.168 7f73f6a3c700 1 --2- 172.21.2.101:0/3649608897 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f73f00958c0 0x7f73f0096920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:35.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.168 7f73f6a3c700 1 --2- 172.21.2.101:0/3649608897 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f73f00954c0 0x7f73f0096300 unknown :-1 s=CLOSED pgs=3412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:35.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.168 7f73f6a3c700 1 --2- 172.21.2.101:0/3649608897 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f73f0085bc0 0x7f73f00863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:35.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.168 7f73f6a3c700 1 -- 172.21.2.101:0/3649608897 >> 172.21.2.101:0/3649608897 conn(0x7f73f00bef00 msgr2=0x7f73f009d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:35.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.168 7f73f6a3c700 1 -- 172.21.2.101:0/3649608897 shutdown_connections 2021-02-10T11:49:35.165 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.168 7f73f6a3c700 1 -- 172.21.2.101:0/3649608897 wait complete. 2021-02-10T11:49:35.172 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-02-10T11:49:35.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.592 7f3085c98700 1 Processor -- start 2021-02-10T11:49:35.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.592 7f3085c98700 1 -- start start 2021-02-10T11:49:35.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.592 7f3085c98700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f308010aff0 0x7f30800cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:35.605 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.592 7f3085c98700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30800fe1c0 con 0x7f30800d1c70 2021-02-10T11:49:35.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.592 7f3085c98700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30801b0cb0 con 0x7f30800d1870 2021-02-10T11:49:35.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.592 7f3085c98700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3080109a60 con 0x7f308010aff0 2021-02-10T11:49:35.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.593 7f307effd700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f30800d1c70 0x7f30800cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36480/0 (socket says 172.21.2.101:36480) 2021-02-10T11:49:35.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.593 7f307effd700 1 -- 172.21.2.101:0/3510819090 learned_addr learned my addr 172.21.2.101:0/3510819090 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:35.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.593 7f307ffff700 1 --2- 172.21.2.101:0/3510819090 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f308010aff0 0x7f30800cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:35.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.593 7f307e7fc700 1 -- 172.21.2.101:0/3510819090 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1668390782 0 0) 0x7f3068001180 con 0x7f30800d1c70 2021-02-10T11:49:35.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.593 7f307e7fc700 1 -- 172.21.2.101:0/3510819090 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f306c002fe0 con 0x7f30800d1c70 2021-02-10T11:49:35.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.593 7f307e7fc700 1 -- 172.21.2.101:0/3510819090 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2067904429 0 0) 0x7f3070000c60 con 0x7f30800d1870 2021-02-10T11:49:35.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.593 7f307e7fc700 1 -- 172.21.2.101:0/3510819090 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f306c004370 con 0x7f30800d1870 2021-02-10T11:49:35.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.593 7f307ffff700 1 -- 172.21.2.101:0/3510819090 >> v1:172.21.2.108:6789/0 conn(0x7f30800d1870 legacy=0x7f30801b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:35.606 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.593 7f307ffff700 1 -- 172.21.2.101:0/3510819090 >> v1:172.21.2.101:6789/0 conn(0x7f30800d1c70 legacy=0x7f30800cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:35.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.593 7f307ffff700 1 -- 172.21.2.101:0/3510819090 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308010b3f0 con 0x7f308010aff0 2021-02-10T11:49:35.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.594 7f307ffff700 1 --2- 172.21.2.101:0/3510819090 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f308010aff0 0x7f30800cf750 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f3078006e50 tx=0x7f3078009e70).ready entity=mon.2 client_cookie=b50e2a6015ca26a8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:35.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.594 7f307e7fc700 1 -- 172.21.2.101:0/3510819090 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3078006a20 con 0x7f308010aff0 2021-02-10T11:49:35.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.594 7f307e7fc700 1 -- 172.21.2.101:0/3510819090 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3078006790 con 0x7f308010aff0 2021-02-10T11:49:35.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.594 7f307e7fc700 1 -- 172.21.2.101:0/3510819090 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f307800b700 con 0x7f308010aff0 2021-02-10T11:49:35.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.594 7f3085c98700 1 -- 172.21.2.101:0/3510819090 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f308010aff0 msgr2=0x7f30800cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:35.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.594 7f3085c98700 1 --2- 172.21.2.101:0/3510819090 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f308010aff0 0x7f30800cf750 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f3078006e50 tx=0x7f3078009e70).stop 2021-02-10T11:49:35.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.595 7f3085c98700 1 -- 172.21.2.101:0/3510819090 shutdown_connections 2021-02-10T11:49:35.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.595 7f3085c98700 1 --2- 172.21.2.101:0/3510819090 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f308010aff0 0x7f30800cf750 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:35.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.595 7f3085c98700 1 -- 172.21.2.101:0/3510819090 >> 172.21.2.101:0/3510819090 conn(0x7f30800ce420 msgr2=0x7f30800d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:35.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.595 7f3085c98700 1 -- 172.21.2.101:0/3510819090 shutdown_connections 2021-02-10T11:49:35.607 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.595 7f3085c98700 1 -- 172.21.2.101:0/3510819090 wait complete. 2021-02-10T11:49:35.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.609 7f3085c98700 1 Processor -- start 2021-02-10T11:49:35.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.609 7f3085c98700 1 -- start start 2021-02-10T11:49:35.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.610 7f3085c98700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30800c2370 0x7f308018f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:35.609 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.610 7f3085c98700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3080084cb0 0x7f30801a7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:35.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.610 7f3085c98700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f30801a7a90 0x7f3080085700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:35.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.610 7f3085c98700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30800be840 con 0x7f30800c2370 2021-02-10T11:49:35.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.610 7f3085c98700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30800c29a0 con 0x7f30801a7a90 2021-02-10T11:49:35.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.610 7f3085c98700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30801a7e90 con 0x7f3080084cb0 2021-02-10T11:49:35.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.610 7f307f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30800c2370 0x7f308018f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:35.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.610 7f307f7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30800c2370 0x7f308018f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:35992/0 (socket says 172.21.2.101:35992) 2021-02-10T11:49:35.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.610 7f307f7fe700 1 -- 172.21.2.101:0/1378034543 learned_addr learned my addr 172.21.2.101:0/1378034543 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:35.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.610 7f307effd700 1 --2- 172.21.2.101:0/1378034543 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3080084cb0 0x7f30801a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:35.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.610 7f307ffff700 1 --2- 172.21.2.101:0/1378034543 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f30801a7a90 0x7f3080085700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:35.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.610 7f307f7fe700 1 -- 172.21.2.101:0/1378034543 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3080084cb0 msgr2=0x7f30801a7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:35.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.610 7f307f7fe700 1 --2- 172.21.2.101:0/1378034543 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3080084cb0 0x7f30801a7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:35.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.610 7f307f7fe700 1 -- 172.21.2.101:0/1378034543 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f30801a7a90 msgr2=0x7f3080085700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:35.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.610 7f307f7fe700 1 --2- 172.21.2.101:0/1378034543 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f30801a7a90 0x7f3080085700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:35.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.610 7f307f7fe700 1 -- 172.21.2.101:0/1378034543 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30801a80b0 con 0x7f30800c2370 2021-02-10T11:49:35.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.611 7f307f7fe700 1 --2- 172.21.2.101:0/1378034543 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30800c2370 0x7f308018f870 secure :-1 s=READY pgs=3414 cs=0 l=1 rev1=1 rx=0x7f3070006f10 tx=0x7f307000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:35.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.611 7f3077fff700 1 -- 172.21.2.101:0/1378034543 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f307000bd80 con 0x7f30800c2370 2021-02-10T11:49:35.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.611 7f3085c98700 1 -- 172.21.2.101:0/1378034543 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30801a80b0 con 0x7f30800c2370 2021-02-10T11:49:35.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.611 7f3077fff700 1 -- 172.21.2.101:0/1378034543 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3070000e20 con 0x7f30800c2370 2021-02-10T11:49:35.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.611 7f3077fff700 1 -- 172.21.2.101:0/1378034543 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3070000bc0 con 0x7f30800c2370 2021-02-10T11:49:35.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.611 7f3085c98700 1 -- 172.21.2.101:0/1378034543 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30801a80b0 con 0x7f30800c2370 2021-02-10T11:49:35.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.611 7f3077fff700 1 -- 172.21.2.101:0/1378034543 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3070009fe0 con 0x7f30800c2370 2021-02-10T11:49:35.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.612 7f3077fff700 1 --2- 172.21.2.101:0/1378034543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30640186c0 0x7f306401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:35.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.612 7f3077fff700 1 -- 172.21.2.101:0/1378034543 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3070032780 con 0x7f30800c2370 2021-02-10T11:49:35.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.612 7f307effd700 1 --2- 172.21.2.101:0/1378034543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30640186c0 0x7f306401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:35.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.612 7f307effd700 1 --2- 172.21.2.101:0/1378034543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30640186c0 0x7f306401abc0 crc :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:35.639 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.642 7f3085c98700 1 -- 172.21.2.101:0/1378034543 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30800be840 con 0x7f30800c2370 2021-02-10T11:49:35.641 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.645 7f3077fff700 1 -- 172.21.2.101:0/1378034543 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3070000bc0 con 0x7f30800c2370 2021-02-10T11:49:35.768 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.772 7f3085c98700 1 -- 172.21.2.101:0/1378034543 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"} v 0) v1 -- 0x7f30800be840 con 0x7f30800c2370 2021-02-10T11:49:35.769 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.772 7f3077fff700 1 -- 172.21.2.101:0/1378034543 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"var": "pg_num", "prefix": "osd pool get", "pool": "cephfs_data"}]=0 v199) v1 ==== 100+0+10 (secure 0 0 0) 0x7f3070000e20 con 0x7f30800c2370 2021-02-10T11:49:35.769 INFO:teuthology.orchestra.run.gibba001.stdout:pg_num: 8 2021-02-10T11:49:35.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.774 7f3085c98700 1 -- 172.21.2.101:0/1378034543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30640186c0 msgr2=0x7f306401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:35.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.774 7f3085c98700 1 --2- 172.21.2.101:0/1378034543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30640186c0 0x7f306401abc0 crc :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:35.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.774 7f3085c98700 1 -- 172.21.2.101:0/1378034543 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30800c2370 msgr2=0x7f308018f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:35.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.774 7f3085c98700 1 --2- 172.21.2.101:0/1378034543 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30800c2370 0x7f308018f870 secure :-1 s=READY pgs=3414 cs=0 l=1 rev1=1 rx=0x7f3070006f10 tx=0x7f307000b990).stop 2021-02-10T11:49:35.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.774 7f3085c98700 1 -- 172.21.2.101:0/1378034543 shutdown_connections 2021-02-10T11:49:35.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.774 7f3085c98700 1 --2- 172.21.2.101:0/1378034543 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f30640186c0 0x7f306401abc0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:35.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.774 7f3085c98700 1 --2- 172.21.2.101:0/1378034543 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3080084cb0 0x7f30801a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:35.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.774 7f3085c98700 1 --2- 172.21.2.101:0/1378034543 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f30801a7a90 0x7f3080085700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:35.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.774 7f3085c98700 1 --2- 172.21.2.101:0/1378034543 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f30800c2370 0x7f308018f870 unknown :-1 s=CLOSED pgs=3414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:35.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.774 7f3085c98700 1 -- 172.21.2.101:0/1378034543 >> 172.21.2.101:0/1378034543 conn(0x7f30800b5dc0 msgr2=0x7f30800c2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:35.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.775 7f3085c98700 1 -- 172.21.2.101:0/1378034543 shutdown_connections 2021-02-10T11:49:35.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:35.775 7f3085c98700 1 -- 172.21.2.101:0/1378034543 wait complete. 2021-02-10T11:49:35.780 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-02-10T11:49:35.781 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs dump --format=json 2021-02-10T11:49:36.227 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.212 7f78fe96c700 1 Processor -- start 2021-02-10T11:49:39.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.212 7f78fe96c700 1 -- start start 2021-02-10T11:49:39.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.213 7f78fe96c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f78f80d1870 0x7f78f81b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:39.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.213 7f78fe96c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f78f80f0650 0x7f78f80cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:39.155 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.213 7f78fe96c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f80f0b10 con 0x7f78f81b0a10 2021-02-10T11:49:39.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.213 7f78fe96c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f80fe230 con 0x7f78f80f0650 2021-02-10T11:49:39.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.213 7f78fe96c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f80e5ed0 con 0x7f78f80d1870 2021-02-10T11:49:39.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.214 7f78f77fe700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f78f81b0a10 0x7f78f80cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:58134/0 (socket says 172.21.2.101:58134) 2021-02-10T11:49:39.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.214 7f78f77fe700 1 -- 172.21.2.101:0/4233934987 learned_addr learned my addr 172.21.2.101:0/4233934987 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:39.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.214 7f78fcf89700 1 --2- 172.21.2.101:0/4233934987 >> v2:172.21.2.108:3300/0 conn(0x7f78f80f0650 0x7f78f80cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:39.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.214 7f78f7fff700 1 --2- 172.21.2.101:0/4233934987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f78f80d1870 0x7f78f81b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:39.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.214 7f78f7fff700 1 -- 172.21.2.101:0/4233934987 >> v2:172.21.2.108:3300/0 conn(0x7f78f80f0650 msgr2=0x7f78f80cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:39.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.214 7f78f7fff700 1 --2- 172.21.2.101:0/4233934987 >> v2:172.21.2.108:3300/0 conn(0x7f78f80f0650 0x7f78f80cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:39.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.214 7f78f7fff700 1 -- 172.21.2.101:0/4233934987 >> v1:172.21.2.108:6789/0 conn(0x7f78f81b0a10 legacy=0x7f78f80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:39.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.214 7f78f7fff700 1 -- 172.21.2.101:0/4233934987 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f80f20a0 con 0x7f78f80d1870 2021-02-10T11:49:39.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.214 7f78f6ffd700 1 -- 172.21.2.101:0/4233934987 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3389707245 0 0) 0x7f78ec0009b0 con 0x7f78f81b0a10 2021-02-10T11:49:39.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.214 7f78fcf89700 1 --2- 172.21.2.101:0/4233934987 >> v2:172.21.2.108:3300/0 conn(0x7f78f80f0650 0x7f78f80cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:49:39.156 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.214 7f78f7fff700 1 --2- 172.21.2.101:0/4233934987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f78f80d1870 0x7f78f81b03f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f78e8006650 tx=0x7f78e8009a80).ready entity=mon.2 client_cookie=f65c95f99785d576 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:39.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.215 7f78f6ffd700 1 -- 172.21.2.101:0/4233934987 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f78e800c0c0 con 0x7f78f80d1870 2021-02-10T11:49:39.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.215 7f78f6ffd700 1 -- 172.21.2.101:0/4233934987 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e800c340 con 0x7f78f80d1870 2021-02-10T11:49:39.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.215 7f78f6ffd700 1 -- 172.21.2.101:0/4233934987 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f78e800c860 con 0x7f78f80d1870 2021-02-10T11:49:39.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.215 7f78fe96c700 1 -- 172.21.2.101:0/4233934987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f78f80d1870 msgr2=0x7f78f81b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:39.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.215 7f78fe96c700 1 --2- 172.21.2.101:0/4233934987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f78f80d1870 0x7f78f81b03f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f78e8006650 tx=0x7f78e8009a80).stop 2021-02-10T11:49:39.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.216 7f78fe96c700 1 -- 172.21.2.101:0/4233934987 shutdown_connections 2021-02-10T11:49:39.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.216 7f78fe96c700 1 --2- 172.21.2.101:0/4233934987 >> v2:172.21.2.108:3300/0 conn(0x7f78f80f0650 0x7f78f80cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:39.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.216 7f78fe96c700 1 --2- 172.21.2.101:0/4233934987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f78f80d1870 0x7f78f81b03f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:39.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.216 7f78fe96c700 1 -- 172.21.2.101:0/4233934987 >> 172.21.2.101:0/4233934987 conn(0x7f78f80ce450 msgr2=0x7f78f80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:39.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.216 7f78fe96c700 1 -- 172.21.2.101:0/4233934987 shutdown_connections 2021-02-10T11:49:39.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.216 7f78fe96c700 1 -- 172.21.2.101:0/4233934987 wait complete. 2021-02-10T11:49:39.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.230 7f78fe96c700 1 Processor -- start 2021-02-10T11:49:39.157 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.230 7f78fe96c700 1 -- start start 2021-02-10T11:49:39.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.231 7f78fe96c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f78f81a4a60 0x7f78f80c1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:39.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.231 7f78fe96c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f78f80c2290 0x7f78f80c2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:39.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.231 7f78fe96c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f78f80c2cb0 0x7f78f80bfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:39.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.231 7f78fe96c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f80f05b0 con 0x7f78f80c2290 2021-02-10T11:49:39.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.231 7f78fe96c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f80f07d0 con 0x7f78f80c2cb0 2021-02-10T11:49:39.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.231 7f78fe96c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f80c0340 con 0x7f78f81a4a60 2021-02-10T11:49:39.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.231 7f78f77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f78f80c2290 0x7f78f80c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:39.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.231 7f78f77fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f78f80c2290 0x7f78f80c2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36006/0 (socket says 172.21.2.101:36006) 2021-02-10T11:49:39.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.231 7f78f77fe700 1 -- 172.21.2.101:0/2109983283 learned_addr learned my addr 172.21.2.101:0/2109983283 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:39.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.231 7f78fcf89700 1 --2- 172.21.2.101:0/2109983283 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f78f80c2cb0 0x7f78f80bfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:39.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.231 7f78f7fff700 1 --2- 172.21.2.101:0/2109983283 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f78f81a4a60 0x7f78f80c1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:39.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.231 7f78f77fe700 1 -- 172.21.2.101:0/2109983283 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f78f81a4a60 msgr2=0x7f78f80c1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:39.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.231 7f78f77fe700 1 --2- 172.21.2.101:0/2109983283 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f78f81a4a60 0x7f78f80c1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:39.161 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.231 7f78f77fe700 1 -- 172.21.2.101:0/2109983283 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f78f80c2cb0 msgr2=0x7f78f80bfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:39.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.231 7f78f77fe700 1 --2- 172.21.2.101:0/2109983283 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f78f80c2cb0 0x7f78f80bfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:39.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.231 7f78f77fe700 1 -- 172.21.2.101:0/2109983283 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f80c0560 con 0x7f78f80c2290 2021-02-10T11:49:39.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.232 7f78f77fe700 1 --2- 172.21.2.101:0/2109983283 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f78f80c2290 0x7f78f80c2690 secure :-1 s=READY pgs=3415 cs=0 l=1 rev1=1 rx=0x7f78ec001c60 tx=0x7f78ec00a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:39.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.232 7f78f4ff9700 1 -- 172.21.2.101:0/2109983283 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f78ec00acc0 con 0x7f78f80c2290 2021-02-10T11:49:39.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.232 7f78f4ff9700 1 -- 172.21.2.101:0/2109983283 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec004140 con 0x7f78f80c2290 2021-02-10T11:49:39.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.232 7f78fe96c700 1 -- 172.21.2.101:0/2109983283 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f80c0560 con 0x7f78f80c2290 2021-02-10T11:49:39.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.232 7f78f4ff9700 1 -- 172.21.2.101:0/2109983283 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f78ec0045a0 con 0x7f78f80c2290 2021-02-10T11:49:39.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.232 7f78fe96c700 1 -- 172.21.2.101:0/2109983283 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f80c0560 con 0x7f78f80c2290 2021-02-10T11:49:39.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.232 7f78f4ff9700 1 -- 172.21.2.101:0/2109983283 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f78ec000dd0 con 0x7f78f80c2290 2021-02-10T11:49:39.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.233 7f78f4ff9700 1 --2- 172.21.2.101:0/2109983283 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f78d80186c0 0x7f78d801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:39.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.233 7f78f4ff9700 1 -- 172.21.2.101:0/2109983283 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f78ec032360 con 0x7f78f80c2290 2021-02-10T11:49:39.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.233 7f78f7fff700 1 --2- 172.21.2.101:0/2109983283 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f78d80186c0 0x7f78d801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:39.162 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.233 7f78f7fff700 1 --2- 172.21.2.101:0/2109983283 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f78d80186c0 0x7f78d801abc0 crc :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:39.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.263 7f78fe96c700 1 -- 172.21.2.101:0/2109983283 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f80f0af0 con 0x7f78f80c2290 2021-02-10T11:49:39.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.266 7f78f4ff9700 1 -- 172.21.2.101:0/2109983283 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f78ec025f10 con 0x7f78f80c2290 2021-02-10T11:49:39.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.395 7f78fe96c700 1 -- 172.21.2.101:0/2109983283 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78f80f0af0 con 0x7f78f80c2290 2021-02-10T11:49:39.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.395 7f78f4ff9700 1 -- 172.21.2.101:0/2109983283 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) v1 ==== 95+0+2108 (secure 0 0 0) 0x7f78ec025f10 con 0x7f78f80c2290 2021-02-10T11:49:39.163 INFO:teuthology.orchestra.run.gibba001.stderr:dumped fsmap epoch 172 2021-02-10T11:49:39.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.397 7f78fe96c700 1 -- 172.21.2.101:0/2109983283 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f78d80186c0 msgr2=0x7f78d801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:39.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.397 7f78fe96c700 1 --2- 172.21.2.101:0/2109983283 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f78d80186c0 0x7f78d801abc0 crc :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:39.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.397 7f78fe96c700 1 -- 172.21.2.101:0/2109983283 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f78f80c2290 msgr2=0x7f78f80c2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:39.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.397 7f78fe96c700 1 --2- 172.21.2.101:0/2109983283 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f78f80c2290 0x7f78f80c2690 secure :-1 s=READY pgs=3415 cs=0 l=1 rev1=1 rx=0x7f78ec001c60 tx=0x7f78ec00a8d0).stop 2021-02-10T11:49:39.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.397 7f78fe96c700 1 -- 172.21.2.101:0/2109983283 shutdown_connections 2021-02-10T11:49:39.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.397 7f78fe96c700 1 --2- 172.21.2.101:0/2109983283 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f78f80c2cb0 0x7f78f80bfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:39.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.397 7f78fe96c700 1 --2- 172.21.2.101:0/2109983283 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f78f80c2290 0x7f78f80c2690 unknown :-1 s=CLOSED pgs=3415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:39.163 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.397 7f78fe96c700 1 --2- 172.21.2.101:0/2109983283 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f78d80186c0 0x7f78d801abc0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:39.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.397 7f78fe96c700 1 --2- 172.21.2.101:0/2109983283 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f78f81a4a60 0x7f78f80c1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:39.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.397 7f78fe96c700 1 -- 172.21.2.101:0/2109983283 >> 172.21.2.101:0/2109983283 conn(0x7f78f80ebe30 msgr2=0x7f78f80bf200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:39.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.397 7f78fe96c700 1 -- 172.21.2.101:0/2109983283 shutdown_connections 2021-02-10T11:49:39.164 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:36.397 7f78fe96c700 1 -- 172.21.2.101:0/2109983283 wait complete. 2021-02-10T11:49:39.164 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:49:39.164 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":172,"default_fscid":21,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":13902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.101:6835/1207472796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6834","nonce":1207472796},{"type":"v1","addr":"172.21.2.101:6835","nonce":1207472796}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":168}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-02-10 11:49:18.830789","modified":"2021-02-10 11:49:20.837623","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9100":{"gid":9100,"name":"b","rank":0,"incarnation":169,"state":"up:active","state_seq":3,"addr":"172.21.2.108:6833/1721728696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6832","nonce":1721728696},{"type":"v1","addr":"172.21.2.108:6833","nonce":1721728696}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[43],"metadata_pool":42,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":21}]} 2021-02-10T11:49:39.366 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 172, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-02-10 11:49:18.830789', 'modified': '2021-02-10 11:49:20.837623', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9100}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9100': {'gid': 9100, 'name': 'b', 'rank': 0, 'incarnation': 169, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.108:6833/1721728696', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.108:6832', 'nonce': 1721728696}, {'type': 'v1', 'addr': '172.21.2.108:6833', 'nonce': 1721728696}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [43], 'metadata_pool': 42, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-02-10T11:49:39.366 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-02-10T11:49:39.366 DEBUG:teuthology.orchestra.run.gibba008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2021-02-10T11:49:39.765 INFO:teuthology.orchestra.run.gibba008.stdout:{ 2021-02-10T11:49:39.765 INFO:teuthology.orchestra.run.gibba008.stdout: "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", 2021-02-10T11:49:39.765 INFO:teuthology.orchestra.run.gibba008.stdout: "whoami": 0, 2021-02-10T11:49:39.765 INFO:teuthology.orchestra.run.gibba008.stdout: "id": 9100, 2021-02-10T11:49:39.765 INFO:teuthology.orchestra.run.gibba008.stdout: "want_state": "up:active", 2021-02-10T11:49:39.765 INFO:teuthology.orchestra.run.gibba008.stdout: "state": "up:active", 2021-02-10T11:49:39.765 INFO:teuthology.orchestra.run.gibba008.stdout: "rank_uptime": 20.935597497, 2021-02-10T11:49:39.765 INFO:teuthology.orchestra.run.gibba008.stdout: "mdsmap_epoch": 172, 2021-02-10T11:49:39.765 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch": 199, 2021-02-10T11:49:39.766 INFO:teuthology.orchestra.run.gibba008.stdout: "osdmap_epoch_barrier": 199, 2021-02-10T11:49:39.766 INFO:teuthology.orchestra.run.gibba008.stdout: "uptime": 26.604717315999999 2021-02-10T11:49:39.766 INFO:teuthology.orchestra.run.gibba008.stdout:} 2021-02-10T11:49:39.769 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a6663ba9-2653-4542-8b6a-9323b4e7b545", "whoami": 0, "id": 9100, "want_state": "up:active", "state": "up:active", "rank_uptime": 20.935597497, "mdsmap_epoch": 172, "osdmap_epoch": 199, "osdmap_epoch_barrier": 199, "uptime": 26.604717315999999 } 2021-02-10T11:49:39.769 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-02-10T11:49:39.769 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-02-10T11:49:39.769 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-02-10T11:49:39.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:39.769 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:49:40.105 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:49:40.451 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:49:40.451 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:49:40.731 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:49:40.734 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:49:40.795 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:49:40.795 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:49:40.796 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:49:40.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:40.796 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:49:41.066 INFO:teuthology.orchestra.run.gibba001.stdout:42 2021-02-10T11:49:41.067 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2021-02-10T11:49:41.067 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:49:41.069 DEBUG:teuthology.orchestra.run.gibba001:> sudo modprobe fuse 2021-02-10T11:49:41.414 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-02-10T11:49:41.414 DEBUG:teuthology.orchestra.run.gibba001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-02-10T11:49:41.480 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[186961]: starting ceph client 2021-02-10T11:49:41.480 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:2021-02-10 11:49:41.484 7f5d9e5d7f80 -1 init, newargv = 0x55aba41ea1a0 newargc=9 2021-02-10T11:49:41.485 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[186961]: starting fuse 2021-02-10T11:49:41.758 INFO:teuthology.orchestra.run.gibba001.stderr:mount: /sys/fs/fuse/connections is already mounted or /sys/fs/fuse/connections busy 2021-02-10T11:49:41.758 INFO:teuthology.orchestra.run.gibba001.stderr: /sys/fs/fuse/connections is already mounted on /sys/fs/fuse/connections 2021-02-10T11:49:41.759 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-02-10T11:49:41.759 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:41.759 DEBUG:teuthology.orchestra.run.gibba001:> ls /sys/fs/fuse/connections 2021-02-10T11:49:42.095 INFO:teuthology.orchestra.run.gibba001.stdout:42 2021-02-10T11:49:42.095 INFO:teuthology.orchestra.run.gibba001.stdout:43 2021-02-10T11:49:42.095 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2021-02-10T11:49:42.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:42.096 DEBUG:teuthology.orchestra.run.gibba001:> sudo python3 -c ' 2021-02-10T11:49:42.096 DEBUG:teuthology.orchestra.run.gibba001:> import glob 2021-02-10T11:49:42.096 DEBUG:teuthology.orchestra.run.gibba001:> import re 2021-02-10T11:49:42.096 DEBUG:teuthology.orchestra.run.gibba001:> import os 2021-02-10T11:49:42.096 DEBUG:teuthology.orchestra.run.gibba001:> import subprocess 2021-02-10T11:49:42.096 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:49:42.096 DEBUG:teuthology.orchestra.run.gibba001:> def find_socket(client_name): 2021-02-10T11:49:42.096 DEBUG:teuthology.orchestra.run.gibba001:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-02-10T11:49:42.096 DEBUG:teuthology.orchestra.run.gibba001:> files = glob.glob(asok_path) 2021-02-10T11:49:42.096 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:49:42.096 DEBUG:teuthology.orchestra.run.gibba001:> # Given a non-glob path, it better be there 2021-02-10T11:49:42.096 DEBUG:teuthology.orchestra.run.gibba001:> if "*" not in asok_path: 2021-02-10T11:49:42.097 DEBUG:teuthology.orchestra.run.gibba001:> assert(len(files) == 1) 2021-02-10T11:49:42.097 DEBUG:teuthology.orchestra.run.gibba001:> return files[0] 2021-02-10T11:49:42.097 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:49:42.097 DEBUG:teuthology.orchestra.run.gibba001:> for f in files: 2021-02-10T11:49:42.097 DEBUG:teuthology.orchestra.run.gibba001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-02-10T11:49:42.097 DEBUG:teuthology.orchestra.run.gibba001:> if os.path.exists("/proc/{0}".format(pid)): 2021-02-10T11:49:42.097 DEBUG:teuthology.orchestra.run.gibba001:> return f 2021-02-10T11:49:42.097 DEBUG:teuthology.orchestra.run.gibba001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-02-10T11:49:42.097 DEBUG:teuthology.orchestra.run.gibba001:> 2021-02-10T11:49:42.097 DEBUG:teuthology.orchestra.run.gibba001:> print(find_socket("client.1")) 2021-02-10T11:49:42.097 DEBUG:teuthology.orchestra.run.gibba001:> ' 2021-02-10T11:49:42.459 INFO:teuthology.orchestra.run.gibba001.stdout:/var/run/ceph/ceph-client.1.186961.asok 2021-02-10T11:49:42.462 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.186961.asok 2021-02-10T11:49:42.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:42.462 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.186961.asok status 2021-02-10T11:49:42.854 INFO:teuthology.orchestra.run.gibba001.stdout:{ 2021-02-10T11:49:42.854 INFO:teuthology.orchestra.run.gibba001.stdout: "metadata": { 2021-02-10T11:49:42.854 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_sha1": "962fac13b1a781b8c7d25c1eed50d93813b8f5f0", 2021-02-10T11:49:42.854 INFO:teuthology.orchestra.run.gibba001.stdout: "ceph_version": "ceph version 14.2.16-280-g962fac1 (962fac13b1a781b8c7d25c1eed50d93813b8f5f0) nautilus (stable)", 2021-02-10T11:49:42.854 INFO:teuthology.orchestra.run.gibba001.stdout: "entity_id": "1", 2021-02-10T11:49:42.854 INFO:teuthology.orchestra.run.gibba001.stdout: "hostname": "gibba001", 2021-02-10T11:49:42.855 INFO:teuthology.orchestra.run.gibba001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-02-10T11:49:42.855 INFO:teuthology.orchestra.run.gibba001.stdout: "pid": "186961", 2021-02-10T11:49:42.855 INFO:teuthology.orchestra.run.gibba001.stdout: "root": "/" 2021-02-10T11:49:42.855 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:42.855 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_count": 0, 2021-02-10T11:49:42.855 INFO:teuthology.orchestra.run.gibba001.stdout: "dentry_pinned_count": 0, 2021-02-10T11:49:42.855 INFO:teuthology.orchestra.run.gibba001.stdout: "id": 14103, 2021-02-10T11:49:42.855 INFO:teuthology.orchestra.run.gibba001.stdout: "inst": { 2021-02-10T11:49:42.855 INFO:teuthology.orchestra.run.gibba001.stdout: "name": { 2021-02-10T11:49:42.855 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "client", 2021-02-10T11:49:42.855 INFO:teuthology.orchestra.run.gibba001.stdout: "num": 14103 2021-02-10T11:49:42.855 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:42.855 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:49:42.855 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:42.856 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:49:42.856 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 996854160 2021-02-10T11:49:42.856 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:42.856 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:42.856 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": { 2021-02-10T11:49:42.856 INFO:teuthology.orchestra.run.gibba001.stdout: "type": "v1", 2021-02-10T11:49:42.856 INFO:teuthology.orchestra.run.gibba001.stdout: "addr": "172.21.2.101:0", 2021-02-10T11:49:42.856 INFO:teuthology.orchestra.run.gibba001.stdout: "nonce": 996854160 2021-02-10T11:49:42.856 INFO:teuthology.orchestra.run.gibba001.stdout: }, 2021-02-10T11:49:42.856 INFO:teuthology.orchestra.run.gibba001.stdout: "inst_str": "client.14103 172.21.2.101:0/996854160", 2021-02-10T11:49:42.856 INFO:teuthology.orchestra.run.gibba001.stdout: "addr_str": "172.21.2.101:0/996854160", 2021-02-10T11:49:42.856 INFO:teuthology.orchestra.run.gibba001.stdout: "inode_count": 1, 2021-02-10T11:49:42.857 INFO:teuthology.orchestra.run.gibba001.stdout: "mds_epoch": 172, 2021-02-10T11:49:42.857 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch": 199, 2021-02-10T11:49:42.857 INFO:teuthology.orchestra.run.gibba001.stdout: "osd_epoch_barrier": 0, 2021-02-10T11:49:42.857 INFO:teuthology.orchestra.run.gibba001.stdout: "blacklisted": false 2021-02-10T11:49:42.857 INFO:teuthology.orchestra.run.gibba001.stdout:} 2021-02-10T11:49:42.858 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:42.858 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:42.858 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:49:43.193 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:49:43.194 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:49:43.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:43.194 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:49:43.541 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2021-02-10T11:49:43.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.961 7f37e5e82700 1 Processor -- start 2021-02-10T11:49:43.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.961 7f37e5e82700 1 -- start start 2021-02-10T11:49:43.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.962 7f37e5e82700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f37e00d1c80 0x7f37e00cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:43.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.962 7f37e5e82700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e00d0180 con 0x7f37e00d1880 2021-02-10T11:49:43.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.962 7f37e5e82700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e00fe1d0 con 0x7f37e00cf590 2021-02-10T11:49:43.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.962 7f37e5e82700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37e01b0cc0 con 0x7f37e00d1c80 2021-02-10T11:49:43.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.962 7f37df7fe700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f37e00d1880 0x7f37e01b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36524/0 (socket says 172.21.2.101:36524) 2021-02-10T11:49:43.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.962 7f37df7fe700 1 -- 172.21.2.101:0/1308652120 learned_addr learned my addr 172.21.2.101:0/1308652120 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:43.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.962 7f37d6ffd700 1 --2- 172.21.2.101:0/1308652120 >> v2:172.21.2.108:3300/0 conn(0x7f37e00d1c80 0x7f37e00cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:43.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.962 7f37deffd700 1 -- 172.21.2.101:0/1308652120 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2152058204 0 0) 0x7f37d0000ce0 con 0x7f37e00d1880 2021-02-10T11:49:43.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.962 7f37deffd700 1 -- 172.21.2.101:0/1308652120 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37cc002fe0 con 0x7f37e00d1880 2021-02-10T11:49:43.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.963 7f37d6ffd700 1 -- 172.21.2.101:0/1308652120 >> v1:172.21.2.108:6789/0 conn(0x7f37e00cf590 legacy=0x7f37e00cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:43.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.963 7f37deffd700 1 -- 172.21.2.101:0/1308652120 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3266213947 0 0) 0x7f37d00010e0 con 0x7f37e00d1880 2021-02-10T11:49:43.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.963 7f37d6ffd700 1 -- 172.21.2.101:0/1308652120 >> v1:172.21.2.101:6789/0 conn(0x7f37e00d1880 legacy=0x7f37e01b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:43.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.963 7f37d6ffd700 1 -- 172.21.2.101:0/1308652120 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e0109a70 con 0x7f37e00d1c80 2021-02-10T11:49:43.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.963 7f37d6ffd700 1 --2- 172.21.2.101:0/1308652120 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37e00d1c80 0x7f37e00cef70 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7f37c8006650 tx=0x7f37c8009a80).ready entity=mon.1 client_cookie=2b9449107a414045 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:43.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.963 7f37deffd700 1 -- 172.21.2.101:0/1308652120 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f37c80009b0 con 0x7f37e00d1c80 2021-02-10T11:49:43.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.963 7f37deffd700 1 -- 172.21.2.101:0/1308652120 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c800c0c0 con 0x7f37e00d1c80 2021-02-10T11:49:43.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.963 7f37deffd700 1 -- 172.21.2.101:0/1308652120 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f37c800c520 con 0x7f37e00d1c80 2021-02-10T11:49:43.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.963 7f37e5e82700 1 -- 172.21.2.101:0/1308652120 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37e00d1c80 msgr2=0x7f37e00cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:43.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.963 7f37e5e82700 1 --2- 172.21.2.101:0/1308652120 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37e00d1c80 0x7f37e00cef70 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7f37c8006650 tx=0x7f37c8009a80).stop 2021-02-10T11:49:43.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.964 7f37e5e82700 1 -- 172.21.2.101:0/1308652120 shutdown_connections 2021-02-10T11:49:43.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.964 7f37e5e82700 1 --2- 172.21.2.101:0/1308652120 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37e00d1c80 0x7f37e00cef70 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:43.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.964 7f37e5e82700 1 -- 172.21.2.101:0/1308652120 >> 172.21.2.101:0/1308652120 conn(0x7f37e00ce430 msgr2=0x7f37e00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:43.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.964 7f37e5e82700 1 -- 172.21.2.101:0/1308652120 shutdown_connections 2021-02-10T11:49:43.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.964 7f37e5e82700 1 -- 172.21.2.101:0/1308652120 wait complete. 2021-02-10T11:49:43.976 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.979 7f37e5e82700 1 Processor -- start 2021-02-10T11:49:43.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.979 7f37e5e82700 1 -- start start 2021-02-10T11:49:43.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.979 7f37e5e82700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37e0083f90 0x7f37e00a8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:43.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.979 7f37e5e82700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37e018f770 0x7f37e018fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:43.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.979 7f37e5e82700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37e0190190 0x7f37e0190cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:43.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.979 7f37e5e82700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e0084820 con 0x7f37e0083f90 2021-02-10T11:49:43.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.979 7f37e5e82700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e0084390 con 0x7f37e018f770 2021-02-10T11:49:43.978 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.979 7f37e5e82700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e00a8850 con 0x7f37e0190190 2021-02-10T11:49:43.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.979 7f37df7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37e0083f90 0x7f37e00a8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:43.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.979 7f37df7fe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37e0083f90 0x7f37e00a8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36036/0 (socket says 172.21.2.101:36036) 2021-02-10T11:49:43.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.979 7f37df7fe700 1 -- 172.21.2.101:0/2330125021 learned_addr learned my addr 172.21.2.101:0/2330125021 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:43.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.980 7f37df7fe700 1 -- 172.21.2.101:0/2330125021 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37e0190190 msgr2=0x7f37e0190cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:43.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.980 7f37df7fe700 1 --2- 172.21.2.101:0/2330125021 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37e0190190 0x7f37e0190cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:43.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.980 7f37df7fe700 1 -- 172.21.2.101:0/2330125021 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37e018f770 msgr2=0x7f37e018fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:43.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.980 7f37df7fe700 1 --2- 172.21.2.101:0/2330125021 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37e018f770 0x7f37e018fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:43.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.980 7f37df7fe700 1 -- 172.21.2.101:0/2330125021 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e01912f0 con 0x7f37e0083f90 2021-02-10T11:49:43.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.980 7f37df7fe700 1 --2- 172.21.2.101:0/2330125021 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37e0083f90 0x7f37e00a8230 secure :-1 s=READY pgs=3420 cs=0 l=1 rev1=1 rx=0x7f37d0007ec0 tx=0x7f37d000c990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:43.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.980 7f37dcff9700 1 -- 172.21.2.101:0/2330125021 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f37d0000ce0 con 0x7f37e0083f90 2021-02-10T11:49:43.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.980 7f37dcff9700 1 -- 172.21.2.101:0/2330125021 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d000cd80 con 0x7f37e0083f90 2021-02-10T11:49:43.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.980 7f37dcff9700 1 -- 172.21.2.101:0/2330125021 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f37d00072a0 con 0x7f37e0083f90 2021-02-10T11:49:43.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.980 7f37e5e82700 1 -- 172.21.2.101:0/2330125021 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e01912f0 con 0x7f37e0083f90 2021-02-10T11:49:43.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.980 7f37e5e82700 1 -- 172.21.2.101:0/2330125021 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e01912f0 con 0x7f37e0083f90 2021-02-10T11:49:43.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.981 7f37dcff9700 1 -- 172.21.2.101:0/2330125021 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f37d0000f40 con 0x7f37e0083f90 2021-02-10T11:49:43.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.981 7f37dcff9700 1 --2- 172.21.2.101:0/2330125021 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37c40186c0 0x7f37c401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:43.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.981 7f37dcff9700 1 -- 172.21.2.101:0/2330125021 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f37d0032760 con 0x7f37e0083f90 2021-02-10T11:49:43.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.981 7f37d6ffd700 1 --2- 172.21.2.101:0/2330125021 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37c40186c0 0x7f37c401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:43.980 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:43.981 7f37d6ffd700 1 --2- 172.21.2.101:0/2330125021 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37c40186c0 0x7f37c401abc0 crc :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:44.008 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.012 7f37e5e82700 1 -- 172.21.2.101:0/2330125021 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37e0082aa0 con 0x7f37e0083f90 2021-02-10T11:49:44.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.014 7f37dcff9700 1 -- 172.21.2.101:0/2330125021 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f37d0007b50 con 0x7f37e0083f90 2021-02-10T11:49:44.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.150 7f37e5e82700 1 -- 172.21.2.101:0/2330125021 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f37e0082aa0 con 0x7f37e0083f90 2021-02-10T11:49:44.201 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.204 7f37dcff9700 1 -- 172.21.2.101:0/2330125021 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v173) v1 ==== 136+0+0 (secure 0 0 0) 0x7f37d0000f40 con 0x7f37e0083f90 2021-02-10T11:49:44.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.216 7f37e5e82700 1 -- 172.21.2.101:0/2330125021 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37c40186c0 msgr2=0x7f37c401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:44.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.216 7f37e5e82700 1 --2- 172.21.2.101:0/2330125021 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37c40186c0 0x7f37c401abc0 crc :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:44.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.216 7f37e5e82700 1 -- 172.21.2.101:0/2330125021 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37e0083f90 msgr2=0x7f37e00a8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:44.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.216 7f37e5e82700 1 --2- 172.21.2.101:0/2330125021 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37e0083f90 0x7f37e00a8230 secure :-1 s=READY pgs=3420 cs=0 l=1 rev1=1 rx=0x7f37d0007ec0 tx=0x7f37d000c990).stop 2021-02-10T11:49:44.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.216 7f37e5e82700 1 -- 172.21.2.101:0/2330125021 shutdown_connections 2021-02-10T11:49:44.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.216 7f37e5e82700 1 --2- 172.21.2.101:0/2330125021 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f37c40186c0 0x7f37c401abc0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:44.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.216 7f37e5e82700 1 --2- 172.21.2.101:0/2330125021 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f37e0190190 0x7f37e0190cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:44.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.216 7f37e5e82700 1 --2- 172.21.2.101:0/2330125021 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f37e018f770 0x7f37e018fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:44.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.216 7f37e5e82700 1 --2- 172.21.2.101:0/2330125021 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f37e0083f90 0x7f37e00a8230 unknown :-1 s=CLOSED pgs=3420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:44.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.216 7f37e5e82700 1 -- 172.21.2.101:0/2330125021 >> 172.21.2.101:0/2330125021 conn(0x7f37e019c650 msgr2=0x7f37e0082cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:44.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.217 7f37e5e82700 1 -- 172.21.2.101:0/2330125021 shutdown_connections 2021-02-10T11:49:44.213 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.217 7f37e5e82700 1 -- 172.21.2.101:0/2330125021 wait complete. 2021-02-10T11:49:44.221 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs volume ls 2021-02-10T11:49:44.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.637 7f863964b700 1 Processor -- start 2021-02-10T11:49:44.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.637 7f863964b700 1 -- start start 2021-02-10T11:49:44.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.638 7f863964b700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f86340d1870 0x7f86341b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:44.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.638 7f863964b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86340f0650 0x7f86340cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:44.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.638 7f863964b700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86340f0b10 con 0x7f86341b0a10 2021-02-10T11:49:44.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.638 7f863964b700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86340e5ed0 con 0x7f86340d1870 2021-02-10T11:49:44.649 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.638 7f863964b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86340d1c70 con 0x7f86340f0650 2021-02-10T11:49:44.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.638 7f86332bf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f86340d1870 0x7f86341b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:44.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.638 7f86332bf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f86340d1870 0x7f86341b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36044/0 (socket says 172.21.2.101:36044) 2021-02-10T11:49:44.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.638 7f86332bf700 1 -- 172.21.2.101:0/697184673 learned_addr learned my addr 172.21.2.101:0/697184673 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:44.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.638 7f86332bf700 1 -- 172.21.2.101:0/697184673 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86340f0650 msgr2=0x7f86340cfb20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:49:44.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.638 7f86332bf700 1 --2- 172.21.2.101:0/697184673 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86340f0650 0x7f86340cfb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:44.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.638 7f86332bf700 1 -- 172.21.2.101:0/697184673 >> v1:172.21.2.108:6789/0 conn(0x7f86341b0a10 legacy=0x7f86340cf330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:49:44.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.638 7f86332bf700 1 -- 172.21.2.101:0/697184673 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86340d1e90 con 0x7f86340d1870 2021-02-10T11:49:44.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.638 7f86332bf700 1 --2- 172.21.2.101:0/697184673 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86340d1870 0x7f86341b03f0 secure :-1 s=READY pgs=3421 cs=0 l=1 rev1=1 rx=0x7f8624006700 tx=0x7f8624009b80).ready entity=mon.0 client_cookie=ab8e371edd538920 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:44.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.638 7f86322bd700 1 -- 172.21.2.101:0/697184673 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f862400c0c0 con 0x7f86340d1870 2021-02-10T11:49:44.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.639 7f86322bd700 1 -- 172.21.2.101:0/697184673 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862400c340 con 0x7f86340d1870 2021-02-10T11:49:44.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.639 7f86322bd700 1 -- 172.21.2.101:0/697184673 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f862400c820 con 0x7f86340d1870 2021-02-10T11:49:44.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.639 7f863964b700 1 -- 172.21.2.101:0/697184673 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86340d1870 msgr2=0x7f86341b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:44.650 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.639 7f863964b700 1 --2- 172.21.2.101:0/697184673 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86340d1870 0x7f86341b03f0 secure :-1 s=READY pgs=3421 cs=0 l=1 rev1=1 rx=0x7f8624006700 tx=0x7f8624009b80).stop 2021-02-10T11:49:44.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.639 7f863964b700 1 -- 172.21.2.101:0/697184673 shutdown_connections 2021-02-10T11:49:44.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.639 7f863964b700 1 --2- 172.21.2.101:0/697184673 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86340f0650 0x7f86340cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:44.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.639 7f863964b700 1 --2- 172.21.2.101:0/697184673 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86340d1870 0x7f86341b03f0 unknown :-1 s=CLOSED pgs=3421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:44.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.639 7f863964b700 1 -- 172.21.2.101:0/697184673 >> 172.21.2.101:0/697184673 conn(0x7f86340ce450 msgr2=0x7f86340d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:44.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.639 7f863964b700 1 -- 172.21.2.101:0/697184673 shutdown_connections 2021-02-10T11:49:44.651 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.639 7f863964b700 1 -- 172.21.2.101:0/697184673 wait complete. 2021-02-10T11:49:44.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.653 7f863964b700 1 Processor -- start 2021-02-10T11:49:44.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.654 7f863964b700 1 -- start start 2021-02-10T11:49:44.652 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.654 7f863964b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86340bb5d0 0x7f86340bfac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:44.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.654 7f863964b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86340a9b50 0x7f86340a9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:44.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.654 7f863964b700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86341a7550 0x7f86341a7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:44.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.654 7f863964b700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86340bb9d0 con 0x7f86340a9b50 2021-02-10T11:49:44.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.654 7f863964b700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86340aa570 con 0x7f86341a7550 2021-02-10T11:49:44.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.654 7f863964b700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86340aa790 con 0x7f86340bb5d0 2021-02-10T11:49:44.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.654 7f8632abe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86340a9b50 0x7f86340a9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:44.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.654 7f8632abe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86340a9b50 0x7f86340a9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36052/0 (socket says 172.21.2.101:36052) 2021-02-10T11:49:44.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.654 7f8632abe700 1 -- 172.21.2.101:0/4021472791 learned_addr learned my addr 172.21.2.101:0/4021472791 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:44.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.654 7f8632abe700 1 -- 172.21.2.101:0/4021472791 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86340bb5d0 msgr2=0x7f86340bfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:44.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.654 7f8632abe700 1 --2- 172.21.2.101:0/4021472791 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86340bb5d0 0x7f86340bfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:44.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.654 7f8632abe700 1 -- 172.21.2.101:0/4021472791 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86341a7550 msgr2=0x7f86341a7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:44.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.654 7f8632abe700 1 --2- 172.21.2.101:0/4021472791 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86341a7550 0x7f86341a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:44.653 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.654 7f8632abe700 1 -- 172.21.2.101:0/4021472791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86341a7f70 con 0x7f86340a9b50 2021-02-10T11:49:44.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.655 7f8632abe700 1 --2- 172.21.2.101:0/4021472791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86340a9b50 0x7f86340a9f50 secure :-1 s=READY pgs=3422 cs=0 l=1 rev1=1 rx=0x7f861c00b530 tx=0x7f861c00b8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:44.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.655 7f862bfff700 1 -- 172.21.2.101:0/4021472791 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f861c00bcc0 con 0x7f86340a9b50 2021-02-10T11:49:44.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.655 7f862bfff700 1 -- 172.21.2.101:0/4021472791 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861c006020 con 0x7f86340a9b50 2021-02-10T11:49:44.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.655 7f862bfff700 1 -- 172.21.2.101:0/4021472791 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f861c0054a0 con 0x7f86340a9b50 2021-02-10T11:49:44.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.655 7f863964b700 1 -- 172.21.2.101:0/4021472791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86341a7f70 con 0x7f86340a9b50 2021-02-10T11:49:44.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.655 7f863964b700 1 -- 172.21.2.101:0/4021472791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86341a7f70 con 0x7f86340a9b50 2021-02-10T11:49:44.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.655 7f862bfff700 1 -- 172.21.2.101:0/4021472791 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f861c006200 con 0x7f86340a9b50 2021-02-10T11:49:44.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.656 7f862bfff700 1 --2- 172.21.2.101:0/4021472791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86180186c0 0x7f861801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:44.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.656 7f862bfff700 1 -- 172.21.2.101:0/4021472791 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f861c032410 con 0x7f86340a9b50 2021-02-10T11:49:44.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.656 7f86332bf700 1 --2- 172.21.2.101:0/4021472791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86180186c0 0x7f861801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:44.654 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.656 7f86332bf700 1 --2- 172.21.2.101:0/4021472791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86180186c0 0x7f861801abc0 crc :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:44.684 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.687 7f863964b700 1 -- 172.21.2.101:0/4021472791 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86340ee0b0 con 0x7f86340a9b50 2021-02-10T11:49:44.686 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.690 7f862bfff700 1 -- 172.21.2.101:0/4021472791 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f861c0214e0 con 0x7f86340a9b50 2021-02-10T11:49:44.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.824 7f863964b700 1 -- 172.21.2.101:0/4021472791 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs volume ls", "target": ["mgr", ""]}) v1 -- 0x7f86340aa570 con 0x7f86180186c0 2021-02-10T11:49:44.820 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.824 7f862bfff700 1 -- 172.21.2.101:0/4021472791 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+41 (crc 0 0 0) 0x7f8624000970 con 0x7f86180186c0 2021-02-10T11:49:44.821 INFO:teuthology.orchestra.run.gibba001.stdout:[ 2021-02-10T11:49:44.821 INFO:teuthology.orchestra.run.gibba001.stdout: { 2021-02-10T11:49:44.821 INFO:teuthology.orchestra.run.gibba001.stdout: "name": "cephfs" 2021-02-10T11:49:44.821 INFO:teuthology.orchestra.run.gibba001.stdout: } 2021-02-10T11:49:44.821 INFO:teuthology.orchestra.run.gibba001.stdout:] 2021-02-10T11:49:44.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.826 7f863964b700 1 -- 172.21.2.101:0/4021472791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86180186c0 msgr2=0x7f861801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:44.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.826 7f863964b700 1 --2- 172.21.2.101:0/4021472791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86180186c0 0x7f861801abc0 crc :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:44.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.826 7f863964b700 1 -- 172.21.2.101:0/4021472791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86340a9b50 msgr2=0x7f86340a9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:44.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.826 7f863964b700 1 --2- 172.21.2.101:0/4021472791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86340a9b50 0x7f86340a9f50 secure :-1 s=READY pgs=3422 cs=0 l=1 rev1=1 rx=0x7f861c00b530 tx=0x7f861c00b8d0).stop 2021-02-10T11:49:44.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.826 7f863964b700 1 -- 172.21.2.101:0/4021472791 shutdown_connections 2021-02-10T11:49:44.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.826 7f863964b700 1 --2- 172.21.2.101:0/4021472791 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f86341a7550 0x7f86341a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:44.822 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.826 7f863964b700 1 --2- 172.21.2.101:0/4021472791 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f86340a9b50 0x7f86340a9f50 unknown :-1 s=CLOSED pgs=3422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:44.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.826 7f863964b700 1 --2- 172.21.2.101:0/4021472791 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f86180186c0 0x7f861801abc0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:44.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.826 7f863964b700 1 --2- 172.21.2.101:0/4021472791 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f86340bb5d0 0x7f86340bfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:44.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.826 7f863964b700 1 -- 172.21.2.101:0/4021472791 >> 172.21.2.101:0/4021472791 conn(0x7f86340a6eb0 msgr2=0x7f86340bf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:44.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.826 7f863964b700 1 -- 172.21.2.101:0/4021472791 shutdown_connections 2021-02-10T11:49:44.823 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:44.826 7f863964b700 1 -- 172.21.2.101:0/4021472791 wait complete. 2021-02-10T11:49:44.830 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-02-10T11:49:45.263 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.251 7fb0f4bd7700 1 Processor -- start 2021-02-10T11:49:45.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.251 7fb0f4bd7700 1 -- start start 2021-02-10T11:49:45.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.252 7fb0f4bd7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb0f010aff0 0x7fb0f00cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:45.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.252 7fb0f4bd7700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f00fe1c0 con 0x7fb0f00d1870 2021-02-10T11:49:45.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.252 7fb0f4bd7700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f01b0cb0 con 0x7fb0f00d1c70 2021-02-10T11:49:45.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.252 7fb0f4bd7700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f0109a60 con 0x7fb0f010aff0 2021-02-10T11:49:45.264 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.252 7fb0ef05f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb0f010aff0 0x7fb0f00cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:45.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.252 7fb0ee85e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb0f00d1870 0x7fb0f01b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36550/0 (socket says 172.21.2.101:36550) 2021-02-10T11:49:45.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.252 7fb0ef05f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb0f010aff0 0x7fb0f00cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36058/0 (socket says 172.21.2.101:36058) 2021-02-10T11:49:45.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.252 7fb0ee85e700 1 -- 172.21.2.101:0/815373858 learned_addr learned my addr 172.21.2.101:0/815373858 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:49:45.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.252 7fb0ef05f700 1 -- 172.21.2.101:0/815373858 >> v1:172.21.2.108:6789/0 conn(0x7fb0f00d1c70 legacy=0x7fb0f00cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:45.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.252 7fb0ef05f700 1 -- 172.21.2.101:0/815373858 >> v1:172.21.2.101:6789/0 conn(0x7fb0f00d1870 legacy=0x7fb0f01b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:45.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.252 7fb0ef05f700 1 -- 172.21.2.101:0/815373858 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f010b3f0 con 0x7fb0f010aff0 2021-02-10T11:49:45.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.252 7fb0ef05f700 1 --2- 172.21.2.101:0/815373858 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0f010aff0 0x7fb0f00cf750 secure :-1 s=READY pgs=3424 cs=0 l=1 rev1=1 rx=0x7fb0e8006e30 tx=0x7fb0e8009e60).ready entity=mon.0 client_cookie=e06194564b209b27 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:45.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.252 7fb0ed85c700 1 -- 172.21.2.101:0/815373858 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0e8003030 con 0x7fb0f010aff0 2021-02-10T11:49:45.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.252 7fb0ed85c700 1 -- 172.21.2.101:0/815373858 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e800b300 con 0x7fb0f010aff0 2021-02-10T11:49:45.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.252 7fb0ed85c700 1 -- 172.21.2.101:0/815373858 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0e800b820 con 0x7fb0f010aff0 2021-02-10T11:49:45.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.252 7fb0f4bd7700 1 -- 172.21.2.101:0/815373858 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0f010aff0 msgr2=0x7fb0f00cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:45.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.252 7fb0f4bd7700 1 --2- 172.21.2.101:0/815373858 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0f010aff0 0x7fb0f00cf750 secure :-1 s=READY pgs=3424 cs=0 l=1 rev1=1 rx=0x7fb0e8006e30 tx=0x7fb0e8009e60).stop 2021-02-10T11:49:45.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.253 7fb0f4bd7700 1 -- 172.21.2.101:0/815373858 shutdown_connections 2021-02-10T11:49:45.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.253 7fb0f4bd7700 1 --2- 172.21.2.101:0/815373858 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0f010aff0 0x7fb0f00cf750 unknown :-1 s=CLOSED pgs=3424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:45.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.253 7fb0f4bd7700 1 -- 172.21.2.101:0/815373858 >> 172.21.2.101:0/815373858 conn(0x7fb0f00ce420 msgr2=0x7fb0f00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:45.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.253 7fb0f4bd7700 1 -- 172.21.2.101:0/815373858 shutdown_connections 2021-02-10T11:49:45.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.253 7fb0f4bd7700 1 -- 172.21.2.101:0/815373858 wait complete. 2021-02-10T11:49:45.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.267 7fb0f4bd7700 1 Processor -- start 2021-02-10T11:49:45.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.267 7fb0f4bd7700 1 -- start start 2021-02-10T11:49:45.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.268 7fb0f4bd7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0f00c0740 0x7fb0f01c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:45.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.268 7fb0f4bd7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0f01c5ba0 0x7fb0f00a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:45.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.268 7fb0f4bd7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0f00b6e60 0x7fb0f00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:45.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.268 7fb0f4bd7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00a6880 con 0x7fb0f00b6e60 2021-02-10T11:49:45.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.268 7fb0f4bd7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00a6aa0 con 0x7fb0f01c5ba0 2021-02-10T11:49:45.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.268 7fb0f4bd7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00a6cc0 con 0x7fb0f00c0740 2021-02-10T11:49:45.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.268 7fb0ef05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0f00b6e60 0x7fb0f00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:45.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.268 7fb0ef05f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0f00b6e60 0x7fb0f00a6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36060/0 (socket says 172.21.2.101:36060) 2021-02-10T11:49:45.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.268 7fb0ef05f700 1 -- 172.21.2.101:0/2517466366 learned_addr learned my addr 172.21.2.101:0/2517466366 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:45.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.268 7fb0ee05d700 1 --2- 172.21.2.101:0/2517466366 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0f01c5ba0 0x7fb0f00a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:45.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.268 7fb0ee85e700 1 --2- 172.21.2.101:0/2517466366 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0f00c0740 0x7fb0f01c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:45.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.269 7fb0ef05f700 1 -- 172.21.2.101:0/2517466366 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0f00c0740 msgr2=0x7fb0f01c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:45.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.269 7fb0ef05f700 1 --2- 172.21.2.101:0/2517466366 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0f00c0740 0x7fb0f01c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:45.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.269 7fb0ef05f700 1 -- 172.21.2.101:0/2517466366 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0f01c5ba0 msgr2=0x7fb0f00a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:45.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.269 7fb0ef05f700 1 --2- 172.21.2.101:0/2517466366 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0f01c5ba0 0x7fb0f00a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:45.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.269 7fb0ef05f700 1 -- 172.21.2.101:0/2517466366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f00a6ee0 con 0x7fb0f00b6e60 2021-02-10T11:49:45.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.269 7fb0ef05f700 1 --2- 172.21.2.101:0/2517466366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0f00b6e60 0x7fb0f00a6260 secure :-1 s=READY pgs=3425 cs=0 l=1 rev1=1 rx=0x7fb0e80068b0 tx=0x7fb0e800bd20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:45.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.269 7fb0e77fe700 1 -- 172.21.2.101:0/2517466366 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0e80096c0 con 0x7fb0f00b6e60 2021-02-10T11:49:45.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.269 7fb0e77fe700 1 -- 172.21.2.101:0/2517466366 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e800be60 con 0x7fb0f00b6e60 2021-02-10T11:49:45.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.269 7fb0e77fe700 1 -- 172.21.2.101:0/2517466366 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0e8001670 con 0x7fb0f00b6e60 2021-02-10T11:49:45.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.269 7fb0f4bd7700 1 -- 172.21.2.101:0/2517466366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f00a6ee0 con 0x7fb0f00b6e60 2021-02-10T11:49:45.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.269 7fb0f4bd7700 1 -- 172.21.2.101:0/2517466366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f00a6ee0 con 0x7fb0f00b6e60 2021-02-10T11:49:45.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.270 7fb0e77fe700 1 -- 172.21.2.101:0/2517466366 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb0e8001270 con 0x7fb0f00b6e60 2021-02-10T11:49:45.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.270 7fb0e77fe700 1 --2- 172.21.2.101:0/2517466366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0d4018730 0x7fb0d401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:45.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.270 7fb0e77fe700 1 -- 172.21.2.101:0/2517466366 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb0e80329c0 con 0x7fb0f00b6e60 2021-02-10T11:49:45.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.270 7fb0ee85e700 1 --2- 172.21.2.101:0/2517466366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0d4018730 0x7fb0d401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:45.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.270 7fb0ee85e700 1 --2- 172.21.2.101:0/2517466366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0d4018730 0x7fb0d401abc0 crc :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:45.298 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.302 7fb0f4bd7700 1 -- 172.21.2.101:0/2517466366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f00807d0 con 0x7fb0f00b6e60 2021-02-10T11:49:45.300 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.304 7fb0e77fe700 1 -- 172.21.2.101:0/2517466366 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb0e80096c0 con 0x7fb0f00b6e60 2021-02-10T11:49:45.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.453 7fb0f4bd7700 1 -- 172.21.2.101:0/2517466366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb0f0088070 con 0x7fb0f00b6e60 2021-02-10T11:49:45.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.453 7fb0e77fe700 1 -- 172.21.2.101:0/2517466366 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb0e8001270 con 0x7fb0f00b6e60 2021-02-10T11:49:45.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.455 7fb0f4bd7700 1 -- 172.21.2.101:0/2517466366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0d4018730 msgr2=0x7fb0d401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:45.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.455 7fb0f4bd7700 1 --2- 172.21.2.101:0/2517466366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0d4018730 0x7fb0d401abc0 crc :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:45.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.455 7fb0f4bd7700 1 -- 172.21.2.101:0/2517466366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0f00b6e60 msgr2=0x7fb0f00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:45.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.455 7fb0f4bd7700 1 --2- 172.21.2.101:0/2517466366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0f00b6e60 0x7fb0f00a6260 secure :-1 s=READY pgs=3425 cs=0 l=1 rev1=1 rx=0x7fb0e80068b0 tx=0x7fb0e800bd20).stop 2021-02-10T11:49:45.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.455 7fb0f4bd7700 1 -- 172.21.2.101:0/2517466366 shutdown_connections 2021-02-10T11:49:45.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.455 7fb0f4bd7700 1 --2- 172.21.2.101:0/2517466366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb0f00b6e60 0x7fb0f00a6260 unknown :-1 s=CLOSED pgs=3425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:45.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.455 7fb0f4bd7700 1 --2- 172.21.2.101:0/2517466366 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb0f01c5ba0 0x7fb0f00a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:45.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.455 7fb0f4bd7700 1 --2- 172.21.2.101:0/2517466366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb0d4018730 0x7fb0d401abc0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:45.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.455 7fb0f4bd7700 1 --2- 172.21.2.101:0/2517466366 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb0f00c0740 0x7fb0f01c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:45.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.455 7fb0f4bd7700 1 -- 172.21.2.101:0/2517466366 >> 172.21.2.101:0/2517466366 conn(0x7fb0f00c4120 msgr2=0x7fb0f00ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:45.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.455 7fb0f4bd7700 1 -- 172.21.2.101:0/2517466366 shutdown_connections 2021-02-10T11:49:45.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.455 7fb0f4bd7700 1 -- 172.21.2.101:0/2517466366 wait complete. 2021-02-10T11:49:45.460 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000951555 2021-02-10T11:49:45.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.881 7fded6d8a700 1 Processor -- start 2021-02-10T11:49:45.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.881 7fded6d8a700 1 -- start start 2021-02-10T11:49:45.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.882 7fded6d8a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fded00d1870 0x7fded01b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:45.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.882 7fded6d8a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fded01b0ae0 0x7fded0108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:45.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.882 7fded6d8a700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fded00d1c70 0x7fded0108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:45.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.882 7fded6d8a700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded00e5ef0 con 0x7fded01b0ae0 2021-02-10T11:49:45.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.882 7fded6d8a700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded0109a60 con 0x7fded00d1c70 2021-02-10T11:49:45.894 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.882 7fded6d8a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded0088070 con 0x7fded00d1870 2021-02-10T11:49:45.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.882 7fdecffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fded01b0ae0 0x7fded0108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:45.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.882 7fdecffff700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fded01b0ae0 0x7fded0108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36068/0 (socket says 172.21.2.101:36068) 2021-02-10T11:49:45.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.882 7fdecffff700 1 -- 172.21.2.101:0/2424060615 learned_addr learned my addr 172.21.2.101:0/2424060615 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:45.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.882 7fdecffff700 1 -- 172.21.2.101:0/2424060615 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fded00d1870 msgr2=0x7fded01b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:45.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.882 7fdecffff700 1 --2- 172.21.2.101:0/2424060615 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fded00d1870 0x7fded01b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:45.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.882 7fdecffff700 1 -- 172.21.2.101:0/2424060615 >> v2:172.21.2.108:3300/0 conn(0x7fded00d1c70 msgr2=0x7fded0108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:49:45.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.882 7fdecffff700 1 --2- 172.21.2.101:0/2424060615 >> v2:172.21.2.108:3300/0 conn(0x7fded00d1c70 0x7fded0108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:45.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.882 7fdecffff700 1 -- 172.21.2.101:0/2424060615 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded0109580 con 0x7fded01b0ae0 2021-02-10T11:49:45.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.883 7fdecffff700 1 --2- 172.21.2.101:0/2424060615 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fded01b0ae0 0x7fded0108940 secure :-1 s=READY pgs=3426 cs=0 l=1 rev1=1 rx=0x7fdec0006700 tx=0x7fdec0009b80).ready entity=mon.0 client_cookie=995cb790c6beabc5 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:45.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.883 7fdecf7fe700 1 -- 172.21.2.101:0/2424060615 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdec000c0c0 con 0x7fded01b0ae0 2021-02-10T11:49:45.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.883 7fdecf7fe700 1 -- 172.21.2.101:0/2424060615 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec000c400 con 0x7fded01b0ae0 2021-02-10T11:49:45.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.883 7fdecf7fe700 1 -- 172.21.2.101:0/2424060615 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdec000c860 con 0x7fded01b0ae0 2021-02-10T11:49:45.895 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.883 7fded6d8a700 1 -- 172.21.2.101:0/2424060615 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fded01b0ae0 msgr2=0x7fded0108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:45.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.883 7fded6d8a700 1 --2- 172.21.2.101:0/2424060615 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fded01b0ae0 0x7fded0108940 secure :-1 s=READY pgs=3426 cs=0 l=1 rev1=1 rx=0x7fdec0006700 tx=0x7fdec0009b80).stop 2021-02-10T11:49:45.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.884 7fded6d8a700 1 -- 172.21.2.101:0/2424060615 shutdown_connections 2021-02-10T11:49:45.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.884 7fded6d8a700 1 --2- 172.21.2.101:0/2424060615 >> v2:172.21.2.108:3300/0 conn(0x7fded00d1c70 0x7fded0108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:45.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.884 7fded6d8a700 1 --2- 172.21.2.101:0/2424060615 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fded01b0ae0 0x7fded0108940 unknown :-1 s=CLOSED pgs=3426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:45.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.884 7fded6d8a700 1 --2- 172.21.2.101:0/2424060615 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fded00d1870 0x7fded01b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:45.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.884 7fded6d8a700 1 -- 172.21.2.101:0/2424060615 >> 172.21.2.101:0/2424060615 conn(0x7fded00ce420 msgr2=0x7fded00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:45.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.884 7fded6d8a700 1 -- 172.21.2.101:0/2424060615 shutdown_connections 2021-02-10T11:49:45.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.884 7fded6d8a700 1 -- 172.21.2.101:0/2424060615 wait complete. 2021-02-10T11:49:45.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.898 7fded6d8a700 1 Processor -- start 2021-02-10T11:49:45.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.898 7fded6d8a700 1 -- start start 2021-02-10T11:49:45.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.899 7fded6d8a700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fded00a73c0 0x7fded00a6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:45.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.899 7fded6d8a700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fded00add40 0x7fded00ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:45.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.899 7fded6d8a700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fded00ae760 0x7fded00af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:45.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.899 7fded6d8a700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded00aec00 con 0x7fded00add40 2021-02-10T11:49:45.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.899 7fded6d8a700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded00875d0 con 0x7fded00ae760 2021-02-10T11:49:45.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.899 7fded6d8a700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded00af7a0 con 0x7fded00a73c0 2021-02-10T11:49:45.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.899 7fdecffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fded00add40 0x7fded00ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:45.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.899 7fdecffff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fded00add40 0x7fded00ae140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36072/0 (socket says 172.21.2.101:36072) 2021-02-10T11:49:45.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.899 7fdecffff700 1 -- 172.21.2.101:0/3138144001 learned_addr learned my addr 172.21.2.101:0/3138144001 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:45.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.900 7fded53a7700 1 --2- 172.21.2.101:0/3138144001 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fded00ae760 0x7fded00af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:45.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.900 7fded4ba6700 1 --2- 172.21.2.101:0/3138144001 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fded00a73c0 0x7fded00a6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:45.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.900 7fdecffff700 1 -- 172.21.2.101:0/3138144001 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fded00a73c0 msgr2=0x7fded00a6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:45.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.900 7fdecffff700 1 --2- 172.21.2.101:0/3138144001 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fded00a73c0 0x7fded00a6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:45.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.900 7fdecffff700 1 -- 172.21.2.101:0/3138144001 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fded00ae760 msgr2=0x7fded00af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:45.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.900 7fdecffff700 1 --2- 172.21.2.101:0/3138144001 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fded00ae760 0x7fded00af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:45.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.900 7fdecffff700 1 -- 172.21.2.101:0/3138144001 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded00af9c0 con 0x7fded00add40 2021-02-10T11:49:45.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.900 7fded4ba6700 1 --2- 172.21.2.101:0/3138144001 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fded00a73c0 0x7fded00a6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:49:45.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.900 7fded53a7700 1 --2- 172.21.2.101:0/3138144001 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fded00ae760 0x7fded00af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:49:45.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.900 7fdecffff700 1 --2- 172.21.2.101:0/3138144001 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fded00add40 0x7fded00ae140 secure :-1 s=READY pgs=3427 cs=0 l=1 rev1=1 rx=0x7fdec00066a0 tx=0x7fdec000b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:45.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.900 7fdecd7fa700 1 -- 172.21.2.101:0/3138144001 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdec000bd80 con 0x7fded00add40 2021-02-10T11:49:45.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.900 7fdecd7fa700 1 -- 172.21.2.101:0/3138144001 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec00052e0 con 0x7fded00add40 2021-02-10T11:49:45.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.900 7fded6d8a700 1 -- 172.21.2.101:0/3138144001 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded00af9c0 con 0x7fded00add40 2021-02-10T11:49:45.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.900 7fdecd7fa700 1 -- 172.21.2.101:0/3138144001 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdec0005740 con 0x7fded00add40 2021-02-10T11:49:45.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.901 7fded6d8a700 1 -- 172.21.2.101:0/3138144001 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded00af9c0 con 0x7fded00add40 2021-02-10T11:49:45.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.901 7fdecd7fa700 1 -- 172.21.2.101:0/3138144001 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdec0006030 con 0x7fded00add40 2021-02-10T11:49:45.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.901 7fdecd7fa700 1 --2- 172.21.2.101:0/3138144001 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdeb4018730 0x7fdeb401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:45.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.901 7fdecd7fa700 1 -- 172.21.2.101:0/3138144001 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fdec0032650 con 0x7fded00add40 2021-02-10T11:49:45.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.901 7fded4ba6700 1 --2- 172.21.2.101:0/3138144001 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdeb4018730 0x7fdeb401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:45.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.902 7fded4ba6700 1 --2- 172.21.2.101:0/3138144001 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdeb4018730 0x7fdeb401abc0 crc :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:45.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.932 7fded6d8a700 1 -- 172.21.2.101:0/3138144001 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded0082980 con 0x7fded00add40 2021-02-10T11:49:45.930 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:45.934 7fdecd7fa700 1 -- 172.21.2.101:0/3138144001 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdec001f510 con 0x7fded00add40 2021-02-10T11:49:46.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.061 7fded6d8a700 1 -- 172.21.2.101:0/3138144001 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000951555", "target": ["mgr", ""]}) v1 -- 0x7fded00875d0 con 0x7fdeb4018730 2021-02-10T11:49:46.060 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:49:46.064 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:49:46.060 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:49:46.064 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:49:46.060 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:49:46.064 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:49:46.060 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:49:46.064 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:49:46.061 INFO:tasks.ceph.mgr.y.gibba001.stderr:2021-02-10 11:49:46.064 7f2cc39e1700 -1 client.0 error registering admin socket command: (17) File exists 2021-02-10T11:49:46.091 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.095 7fdecd7fa700 1 -- 172.21.2.101:0/3138144001 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fdeb8006930 con 0x7fdeb4018730 2021-02-10T11:49:46.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.126 7fded6d8a700 1 -- 172.21.2.101:0/3138144001 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdeb4018730 msgr2=0x7fdeb401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:46.122 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.126 7fded6d8a700 1 --2- 172.21.2.101:0/3138144001 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdeb4018730 0x7fdeb401abc0 crc :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:46.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.126 7fded6d8a700 1 -- 172.21.2.101:0/3138144001 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fded00add40 msgr2=0x7fded00ae140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:46.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.126 7fded6d8a700 1 --2- 172.21.2.101:0/3138144001 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fded00add40 0x7fded00ae140 secure :-1 s=READY pgs=3427 cs=0 l=1 rev1=1 rx=0x7fdec00066a0 tx=0x7fdec000b990).stop 2021-02-10T11:49:46.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.126 7fded6d8a700 1 -- 172.21.2.101:0/3138144001 shutdown_connections 2021-02-10T11:49:46.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.126 7fded6d8a700 1 --2- 172.21.2.101:0/3138144001 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fded00ae760 0x7fded00af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:46.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.126 7fded6d8a700 1 --2- 172.21.2.101:0/3138144001 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fded00add40 0x7fded00ae140 unknown :-1 s=CLOSED pgs=3427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:46.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.126 7fded6d8a700 1 --2- 172.21.2.101:0/3138144001 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdeb4018730 0x7fdeb401abc0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:46.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.126 7fded6d8a700 1 --2- 172.21.2.101:0/3138144001 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fded00a73c0 0x7fded00a6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:46.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.126 7fded6d8a700 1 -- 172.21.2.101:0/3138144001 >> 172.21.2.101:0/3138144001 conn(0x7fded00be980 msgr2=0x7fded0092cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:46.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.126 7fded6d8a700 1 -- 172.21.2.101:0/3138144001 shutdown_connections 2021-02-10T11:49:46.123 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.126 7fded6d8a700 1 -- 172.21.2.101:0/3138144001 wait complete. 2021-02-10T11:49:46.130 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000951555 2021-02-10T11:49:46.563 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.551 7f2df4a5f700 1 Processor -- start 2021-02-10T11:49:46.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.551 7f2df4a5f700 1 -- start start 2021-02-10T11:49:46.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.552 7f2df4a5f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2df00d1860 0x7f2df01b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:46.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.552 7f2df4a5f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2df00cff60 0x7f2df0109120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:46.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.552 7f2df4a5f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df0109a50 con 0x7f2df01b0ad0 2021-02-10T11:49:46.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.552 7f2df4a5f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df0109740 con 0x7f2df00d1860 2021-02-10T11:49:46.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.552 7f2df4a5f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df0088070 con 0x7f2df00cff60 2021-02-10T11:49:46.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.552 7f2dee85e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2df00d1860 0x7f2df01b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:46.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.552 7f2dee85e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2df00d1860 0x7f2df01b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36094/0 (socket says 172.21.2.101:36094) 2021-02-10T11:49:46.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.552 7f2dee85e700 1 -- 172.21.2.101:0/480322636 learned_addr learned my addr 172.21.2.101:0/480322636 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:46.564 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.552 7f2dee85e700 1 -- 172.21.2.101:0/480322636 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2df00cff60 msgr2=0x7f2df0109120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:49:46.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.552 7f2dee85e700 1 --2- 172.21.2.101:0/480322636 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2df00cff60 0x7f2df0109120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:46.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.552 7f2dee85e700 1 -- 172.21.2.101:0/480322636 >> v1:172.21.2.108:6789/0 conn(0x7f2df01b0ad0 legacy=0x7f2df0108930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:49:46.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.552 7f2dee85e700 1 -- 172.21.2.101:0/480322636 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df00d1c60 con 0x7f2df00d1860 2021-02-10T11:49:46.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.552 7f2dee85e700 1 --2- 172.21.2.101:0/480322636 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2df00d1860 0x7f2df01b04b0 secure :-1 s=READY pgs=3430 cs=0 l=1 rev1=1 rx=0x7f2dd8006700 tx=0x7f2dd8009b80).ready entity=mon.0 client_cookie=ac15c1b7606da340 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:46.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.552 7f2ded85c700 1 -- 172.21.2.101:0/480322636 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2dd800c0c0 con 0x7f2df00d1860 2021-02-10T11:49:46.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.552 7f2ded85c700 1 -- 172.21.2.101:0/480322636 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd800c340 con 0x7f2df00d1860 2021-02-10T11:49:46.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.552 7f2ded85c700 1 -- 172.21.2.101:0/480322636 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2dd800c820 con 0x7f2df00d1860 2021-02-10T11:49:46.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.552 7f2df4a5f700 1 -- 172.21.2.101:0/480322636 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2df00d1860 msgr2=0x7f2df01b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:46.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.552 7f2df4a5f700 1 --2- 172.21.2.101:0/480322636 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2df00d1860 0x7f2df01b04b0 secure :-1 s=READY pgs=3430 cs=0 l=1 rev1=1 rx=0x7f2dd8006700 tx=0x7f2dd8009b80).stop 2021-02-10T11:49:46.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.553 7f2df4a5f700 1 -- 172.21.2.101:0/480322636 shutdown_connections 2021-02-10T11:49:46.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.553 7f2df4a5f700 1 --2- 172.21.2.101:0/480322636 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2df00cff60 0x7f2df0109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:46.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.553 7f2df4a5f700 1 --2- 172.21.2.101:0/480322636 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2df00d1860 0x7f2df01b04b0 unknown :-1 s=CLOSED pgs=3430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:46.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.553 7f2df4a5f700 1 -- 172.21.2.101:0/480322636 >> 172.21.2.101:0/480322636 conn(0x7f2df00ce410 msgr2=0x7f2df00d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:46.565 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.553 7f2df4a5f700 1 -- 172.21.2.101:0/480322636 shutdown_connections 2021-02-10T11:49:46.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.553 7f2df4a5f700 1 -- 172.21.2.101:0/480322636 wait complete. 2021-02-10T11:49:46.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.567 7f2df4a5f700 1 Processor -- start 2021-02-10T11:49:46.566 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.568 7f2df4a5f700 1 -- start start 2021-02-10T11:49:46.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.568 7f2df4a5f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2df01a05e0 0x7f2df018b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:46.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.568 7f2df4a5f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2df00b8780 0x7f2df00bfa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:46.567 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.568 7f2df4a5f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2df00c0030 0x7f2df0189e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:46.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.568 7f2df4a5f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df00b82a0 con 0x7f2df00b8780 2021-02-10T11:49:46.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.568 7f2df4a5f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df00c0430 con 0x7f2df00c0030 2021-02-10T11:49:46.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.568 7f2df4a5f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df00c0650 con 0x7f2df01a05e0 2021-02-10T11:49:46.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.568 7f2dee05d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2df00b8780 0x7f2df00bfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:46.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.568 7f2dee05d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2df00b8780 0x7f2df00bfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36102/0 (socket says 172.21.2.101:36102) 2021-02-10T11:49:46.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.568 7f2dee05d700 1 -- 172.21.2.101:0/1189433557 learned_addr learned my addr 172.21.2.101:0/1189433557 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:49:46.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.569 7f2dee05d700 1 -- 172.21.2.101:0/1189433557 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2df01a05e0 msgr2=0x7f2df018b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:46.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.569 7f2dee05d700 1 --2- 172.21.2.101:0/1189433557 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2df01a05e0 0x7f2df018b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:46.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.569 7f2dee05d700 1 -- 172.21.2.101:0/1189433557 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2df00c0030 msgr2=0x7f2df0189e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:46.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.569 7f2dee05d700 1 --2- 172.21.2.101:0/1189433557 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2df00c0030 0x7f2df0189e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:46.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.569 7f2dee05d700 1 -- 172.21.2.101:0/1189433557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df018a490 con 0x7f2df00b8780 2021-02-10T11:49:46.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.569 7f2dee05d700 1 --2- 172.21.2.101:0/1189433557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2df00b8780 0x7f2df00bfa10 secure :-1 s=READY pgs=3431 cs=0 l=1 rev1=1 rx=0x7f2de000b5b0 tx=0x7f2de000b950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:46.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.569 7f2de77fe700 1 -- 172.21.2.101:0/1189433557 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2de000bd40 con 0x7f2df00b8780 2021-02-10T11:49:46.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.569 7f2de77fe700 1 -- 172.21.2.101:0/1189433557 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de0006020 con 0x7f2df00b8780 2021-02-10T11:49:46.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.569 7f2de77fe700 1 -- 172.21.2.101:0/1189433557 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2de00054a0 con 0x7f2df00b8780 2021-02-10T11:49:46.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.569 7f2df4a5f700 1 -- 172.21.2.101:0/1189433557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df018a490 con 0x7f2df00b8780 2021-02-10T11:49:46.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.569 7f2df4a5f700 1 -- 172.21.2.101:0/1189433557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df018a490 con 0x7f2df00b8780 2021-02-10T11:49:46.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.570 7f2de77fe700 1 -- 172.21.2.101:0/1189433557 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2de0006200 con 0x7f2df00b8780 2021-02-10T11:49:46.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.570 7f2de77fe700 1 --2- 172.21.2.101:0/1189433557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2dd40186c0 0x7f2dd401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:49:46.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.570 7f2de77fe700 1 -- 172.21.2.101:0/1189433557 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2de00325f0 con 0x7f2df00b8780 2021-02-10T11:49:46.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.570 7f2dee85e700 1 --2- 172.21.2.101:0/1189433557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2dd40186c0 0x7f2dd401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:49:46.569 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.570 7f2dee85e700 1 --2- 172.21.2.101:0/1189433557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2dd40186c0 0x7f2dd401abc0 crc :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:49:46.599 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.602 7f2df4a5f700 1 -- 172.21.2.101:0/1189433557 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df0082a20 con 0x7f2df00b8780 2021-02-10T11:49:46.600 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.604 7f2de77fe700 1 -- 172.21.2.101:0/1189433557 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2de0000c60 con 0x7f2df00b8780 2021-02-10T11:49:46.730 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.734 7f2df4a5f700 1 -- 172.21.2.101:0/1189433557 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000951555", "target": ["mgr", ""]}) v1 -- 0x7f2df0082a20 con 0x7f2dd40186c0 2021-02-10T11:49:46.733 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.737 7f2de77fe700 1 -- 172.21.2.101:0/1189433557 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+82 (crc 0 0 0) 0x7f2dd8000940 con 0x7f2dd40186c0 2021-02-10T11:49:46.733 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe 2021-02-10T11:49:46.735 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.739 7f2df4a5f700 1 -- 172.21.2.101:0/1189433557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2dd40186c0 msgr2=0x7f2dd401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:46.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.739 7f2df4a5f700 1 --2- 172.21.2.101:0/1189433557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2dd40186c0 0x7f2dd401abc0 crc :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:46.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.739 7f2df4a5f700 1 -- 172.21.2.101:0/1189433557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2df00b8780 msgr2=0x7f2df00bfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:49:46.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.739 7f2df4a5f700 1 --2- 172.21.2.101:0/1189433557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2df00b8780 0x7f2df00bfa10 secure :-1 s=READY pgs=3431 cs=0 l=1 rev1=1 rx=0x7f2de000b5b0 tx=0x7f2de000b950).stop 2021-02-10T11:49:46.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.739 7f2df4a5f700 1 -- 172.21.2.101:0/1189433557 shutdown_connections 2021-02-10T11:49:46.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.739 7f2df4a5f700 1 --2- 172.21.2.101:0/1189433557 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2df00c0030 0x7f2df0189e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:46.736 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.739 7f2df4a5f700 1 --2- 172.21.2.101:0/1189433557 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2df00b8780 0x7f2df00bfa10 unknown :-1 s=CLOSED pgs=3431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:46.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.739 7f2df4a5f700 1 --2- 172.21.2.101:0/1189433557 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2dd40186c0 0x7f2dd401abc0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:49:46.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.739 7f2df4a5f700 1 --2- 172.21.2.101:0/1189433557 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2df01a05e0 0x7f2df018b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:49:46.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.739 7f2df4a5f700 1 -- 172.21.2.101:0/1189433557 >> 172.21.2.101:0/1189433557 conn(0x7f2df00a6a80 msgr2=0x7f2df00ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:49:46.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.739 7f2df4a5f700 1 -- 172.21.2.101:0/1189433557 shutdown_connections 2021-02-10T11:49:46.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:49:46.739 7f2df4a5f700 1 -- 172.21.2.101:0/1189433557 wait complete. 2021-02-10T11:49:46.743 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000951555 with 64 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe 2021-02-10T11:49:46.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:46.744 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:46.744 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:47.081 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:49:47.081 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:49:47.081 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:49:47.469 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:49:47.470 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:49:47.470 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0285677 s, 36.7 MB/s 2021-02-10T11:49:47.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:47.471 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:47.471 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:47.806 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:49:47.806 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:49:47.807 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:49:48.180 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:49:48.180 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:49:48.180 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0251558 s, 41.7 MB/s 2021-02-10T11:49:48.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:48.181 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:48.181 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:48.517 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:49:48.517 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:49:48.517 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:49:48.892 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:49:48.892 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:49:48.892 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0257766 s, 40.7 MB/s 2021-02-10T11:49:48.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:48.893 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:48.893 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:49.228 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:49:49.229 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:49:49.229 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:49:49.604 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:49:49.604 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:49:49.604 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0247195 s, 42.4 MB/s 2021-02-10T11:49:49.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:49.605 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:49.605 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:49.942 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:49:49.942 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:49:49.943 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:49:50.318 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:49:50.318 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:49:50.318 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0254014 s, 41.3 MB/s 2021-02-10T11:49:50.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:50.319 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:50.319 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:50.655 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:49:50.656 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:49:50.656 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:49:51.031 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:49:51.031 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:49:51.031 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246359 s, 42.6 MB/s 2021-02-10T11:49:51.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:51.033 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:51.033 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:51.368 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:49:51.368 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:49:51.368 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:49:51.743 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:49:51.743 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:49:51.743 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236807 s, 44.3 MB/s 2021-02-10T11:49:51.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:51.744 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:51.744 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:52.080 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:49:52.080 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:49:52.080 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:49:52.460 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:49:52.460 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:49:52.460 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0282962 s, 37.1 MB/s 2021-02-10T11:49:52.461 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:52.461 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:52.461 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:52.796 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:49:52.797 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:49:52.797 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.8 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:49:53.174 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:49:53.174 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:49:53.174 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.026177 s, 40.1 MB/s 2021-02-10T11:49:53.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:53.175 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:53.175 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:53.516 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:49:53.516 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:49:53.516 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.9 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:49:53.889 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:49:53.890 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:49:53.890 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231037 s, 45.4 MB/s 2021-02-10T11:49:53.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:53.891 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:53.891 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:54.228 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:49:54.228 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:49:54.228 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.10 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:49:54.603 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:49:54.603 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:49:54.603 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246429 s, 42.6 MB/s 2021-02-10T11:49:54.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:54.604 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:54.604 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:54.940 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:49:54.940 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:49:54.940 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.11 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:49:55.313 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:49:55.313 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:49:55.313 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236769 s, 44.3 MB/s 2021-02-10T11:49:55.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:55.314 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:55.314 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:55.651 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:49:55.652 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:49:55.652 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.12 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:49:56.028 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:49:56.028 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:49:56.028 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234982 s, 44.6 MB/s 2021-02-10T11:49:56.029 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:56.030 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:56.030 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:56.367 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:49:56.367 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:49:56.368 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.13 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:49:56.743 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:49:56.743 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:49:56.743 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0246691 s, 42.5 MB/s 2021-02-10T11:49:56.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:56.744 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:56.744 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:57.080 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:49:57.080 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:49:57.081 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.14 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:49:57.474 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:49:57.475 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:49:57.475 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239803 s, 43.7 MB/s 2021-02-10T11:49:57.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:57.476 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:57.476 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:57.836 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:49:57.836 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:49:57.836 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.15 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:49:58.231 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:49:58.231 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:49:58.231 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234659 s, 44.7 MB/s 2021-02-10T11:49:58.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:58.232 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:58.233 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:58.572 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:49:58.572 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:49:58.572 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.16 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:49:58.956 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:49:58.956 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:49:58.957 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238784 s, 43.9 MB/s 2021-02-10T11:49:58.958 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:58.958 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:58.958 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:49:59.295 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:49:59.295 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:49:59.296 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.17 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:49:59.672 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:49:59.672 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:49:59.672 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253855 s, 41.3 MB/s 2021-02-10T11:49:59.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:49:59.673 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:49:59.674 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:00.009 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:00.010 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:00.010 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.18 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:00.383 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:00.384 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:00.384 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231192 s, 45.4 MB/s 2021-02-10T11:50:00.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:00.385 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:00.385 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:00.722 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:00.722 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:00.722 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.19 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:01.095 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:01.095 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:01.095 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0223371 s, 46.9 MB/s 2021-02-10T11:50:01.096 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:01.096 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:01.096 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:01.433 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:01.433 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:01.433 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.20 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:01.810 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:01.810 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:01.810 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0232315 s, 45.1 MB/s 2021-02-10T11:50:01.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:01.811 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:01.811 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:02.152 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:02.152 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:02.152 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.21 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:02.527 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:02.527 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:02.527 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024479 s, 42.8 MB/s 2021-02-10T11:50:02.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:02.528 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:02.528 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:02.864 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:02.864 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:02.865 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.22 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:03.237 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:03.237 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:03.238 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0236607 s, 44.3 MB/s 2021-02-10T11:50:03.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:03.239 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:03.239 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:03.574 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:03.574 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:03.574 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.23 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:03.947 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:03.947 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:03.947 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0220867 s, 47.5 MB/s 2021-02-10T11:50:03.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:03.948 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:03.949 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:04.283 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:04.284 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:04.284 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.24 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:04.662 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:04.663 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:04.663 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0256128 s, 40.9 MB/s 2021-02-10T11:50:04.664 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:04.664 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:04.664 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:05.000 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:05.000 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:05.000 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.25 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:05.374 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:05.374 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:05.374 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0227876 s, 46.0 MB/s 2021-02-10T11:50:05.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:05.375 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:05.375 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:05.711 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:05.712 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:05.712 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.26 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:06.087 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:06.087 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:06.087 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243688 s, 43.0 MB/s 2021-02-10T11:50:06.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:06.088 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:06.088 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:06.423 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:06.424 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:06.424 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.27 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:06.800 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:06.800 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:06.801 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0244315 s, 42.9 MB/s 2021-02-10T11:50:06.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:06.802 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:06.802 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:07.138 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:07.138 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:07.138 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.28 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:07.512 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:07.512 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:07.512 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0238689 s, 43.9 MB/s 2021-02-10T11:50:07.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:07.513 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:07.513 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:07.849 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:07.849 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:07.850 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.29 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:08.220 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:08.220 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:08.220 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0230492 s, 45.5 MB/s 2021-02-10T11:50:08.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:08.221 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:08.222 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:08.556 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:08.557 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:08.557 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.30 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:08.930 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:08.931 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:08.931 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0234635 s, 44.7 MB/s 2021-02-10T11:50:08.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:08.932 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:08.932 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:09.267 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:09.267 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:09.267 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.31 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:09.643 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:09.644 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:09.644 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0230487 s, 45.5 MB/s 2021-02-10T11:50:09.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:09.645 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:09.645 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:09.982 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:09.982 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:09.982 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.32 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:10.360 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:10.360 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:10.360 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.026112 s, 40.2 MB/s 2021-02-10T11:50:10.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:10.363 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:10.363 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:10.700 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:10.700 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:10.701 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.33 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:11.077 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:11.077 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:11.077 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0245858 s, 42.6 MB/s 2021-02-10T11:50:11.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:11.078 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:11.078 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:11.099 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:50:11.123 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:50:11.415 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:11.415 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:11.416 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.34 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:11.789 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:11.789 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:11.790 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0225928 s, 46.4 MB/s 2021-02-10T11:50:11.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:11.791 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:11.791 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:12.127 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:12.127 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:12.127 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.35 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:12.504 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:12.504 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:12.504 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.026126 s, 40.1 MB/s 2021-02-10T11:50:12.505 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:12.505 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:12.505 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:12.841 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:12.842 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:12.842 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.36 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:13.217 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:13.217 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:13.217 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024797 s, 42.3 MB/s 2021-02-10T11:50:13.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:13.218 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:13.218 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:13.553 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:13.554 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:13.554 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.37 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:13.930 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:13.930 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:13.930 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024551 s, 42.7 MB/s 2021-02-10T11:50:13.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:13.931 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:13.931 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:14.267 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:14.267 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:14.267 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.38 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:14.644 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:14.644 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:14.644 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0241074 s, 43.5 MB/s 2021-02-10T11:50:14.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:14.645 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:14.645 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:14.980 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:14.981 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:14.981 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.39 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:15.355 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:15.356 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:15.356 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023182 s, 45.2 MB/s 2021-02-10T11:50:15.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:15.357 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:15.357 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:15.694 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:15.694 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:15.694 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.40 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:16.069 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:16.069 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:16.069 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237705 s, 44.1 MB/s 2021-02-10T11:50:16.070 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:16.071 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:16.071 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:16.408 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:16.408 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:16.408 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.41 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:16.781 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:16.781 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:16.782 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0230963 s, 45.4 MB/s 2021-02-10T11:50:16.782 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:16.783 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:16.783 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:17.117 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:17.118 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:17.118 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.42 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:17.492 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:17.493 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:17.493 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0233714 s, 44.9 MB/s 2021-02-10T11:50:17.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:17.494 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:17.494 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:17.830 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:17.831 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:17.831 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.43 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:18.203 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:18.203 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:18.203 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239607 s, 43.8 MB/s 2021-02-10T11:50:18.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:18.204 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:18.204 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:18.540 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:18.541 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:18.541 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.44 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:18.915 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:18.915 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:18.915 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.024406 s, 43.0 MB/s 2021-02-10T11:50:18.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:18.916 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:18.917 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:19.253 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:19.253 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:19.253 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.45 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:19.632 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:19.632 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:19.632 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0258787 s, 40.5 MB/s 2021-02-10T11:50:19.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:19.633 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:19.633 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:19.970 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:19.970 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:19.970 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.46 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:20.344 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:20.345 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:20.345 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0231434 s, 45.3 MB/s 2021-02-10T11:50:20.346 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:20.346 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:20.346 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:20.682 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:20.682 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:20.682 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.47 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:21.059 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:21.059 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:21.059 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0248978 s, 42.1 MB/s 2021-02-10T11:50:21.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:21.060 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:21.061 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:21.396 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:21.397 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:21.397 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.48 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:21.770 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:21.770 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:21.770 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0223189 s, 47.0 MB/s 2021-02-10T11:50:21.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:21.771 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:21.771 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:22.108 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:22.108 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:22.108 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.49 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:22.486 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:22.486 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:22.486 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0243107 s, 43.1 MB/s 2021-02-10T11:50:22.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:22.487 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:22.487 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:22.822 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:22.823 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:22.823 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.50 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:23.198 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:23.198 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:23.198 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252108 s, 41.6 MB/s 2021-02-10T11:50:23.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:23.200 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:23.200 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:23.535 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:23.535 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:23.535 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.51 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:23.909 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:23.909 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:23.909 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.023471 s, 44.7 MB/s 2021-02-10T11:50:23.910 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:23.910 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:23.910 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:24.246 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:24.247 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:24.247 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.52 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:24.622 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:24.622 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:24.622 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0239994 s, 43.7 MB/s 2021-02-10T11:50:24.623 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:24.623 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:24.623 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:24.961 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:24.961 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:24.962 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.53 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:25.336 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:25.337 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:25.337 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237358 s, 44.2 MB/s 2021-02-10T11:50:25.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:25.338 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:25.338 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:25.674 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:25.674 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:25.674 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.54 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:26.048 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:26.048 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:26.048 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0225076 s, 46.6 MB/s 2021-02-10T11:50:26.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:26.050 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:26.050 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:26.385 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:26.386 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:26.386 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.55 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:26.761 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:26.761 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:26.761 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0235165 s, 44.6 MB/s 2021-02-10T11:50:26.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:26.762 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:26.762 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:27.098 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:27.098 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:27.098 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.56 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:27.473 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:27.473 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:27.473 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0237315 s, 44.2 MB/s 2021-02-10T11:50:27.474 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:27.474 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:27.474 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:27.811 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:27.811 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:27.811 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.57 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:28.188 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:28.318 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:28.318 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0257594 s, 40.7 MB/s 2021-02-10T11:50:28.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:28.320 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:28.320 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:28.682 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:28.682 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:28.683 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.58 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:29.061 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:29.062 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:29.062 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0250703 s, 41.8 MB/s 2021-02-10T11:50:29.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:29.063 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:29.063 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:29.405 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:29.405 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:29.406 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.59 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:29.783 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:29.783 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:29.783 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0252217 s, 41.6 MB/s 2021-02-10T11:50:29.784 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:29.784 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:29.784 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:30.120 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:30.121 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:30.121 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.60 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:30.500 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:30.500 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:30.500 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0253606 s, 41.3 MB/s 2021-02-10T11:50:30.501 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:30.501 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:30.501 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:30.837 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:30.838 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:30.838 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.61 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:31.213 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:31.213 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:31.213 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0223187 s, 47.0 MB/s 2021-02-10T11:50:31.214 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:31.214 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:31.214 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:31.550 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:31.550 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:31.551 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.62 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:31.928 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:31.928 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:31.928 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0255839 s, 41.0 MB/s 2021-02-10T11:50:31.929 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:31.929 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:31.929 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:32.265 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:32.266 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:32.266 DEBUG:teuthology.orchestra.run.gibba001:> cd /home/ubuntu/cephtest/mnt.0 && sudo dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000951555/a7a77761-2053-402b-a0a8-31cae336c8fe/subvolume_file.63 bs=1M conv=fdatasync count=1 seek=0 2021-02-10T11:50:32.643 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records in 2021-02-10T11:50:32.643 INFO:teuthology.orchestra.run.gibba001.stderr:1+0 records out 2021-02-10T11:50:32.643 INFO:teuthology.orchestra.run.gibba001.stderr:1048576 bytes (1.0 MB) copied, 0.0261491 s, 40.1 MB/s 2021-02-10T11:50:32.644 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume snapshot create cephfs subvolume_0000000000951555 snapshot_0000000000014068 2021-02-10T11:50:33.077 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.064 7fdfd9288700 1 Processor -- start 2021-02-10T11:50:33.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.064 7fdfd9288700 1 -- start start 2021-02-10T11:50:33.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.065 7fdfd9288700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdfd40d1c80 0x7fdfd40cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:33.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.065 7fdfd9288700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd40d0180 con 0x7fdfd40cf590 2021-02-10T11:50:33.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.065 7fdfd9288700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd40fe1d0 con 0x7fdfd40d1880 2021-02-10T11:50:33.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.065 7fdfd9288700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd41b0cc0 con 0x7fdfd40d1c80 2021-02-10T11:50:33.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.065 7fdfd387e700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fdfd40cf590 0x7fdfd40cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36614/0 (socket says 172.21.2.101:36614) 2021-02-10T11:50:33.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.065 7fdfd387e700 1 -- 172.21.2.101:0/2510264232 learned_addr learned my addr 172.21.2.101:0/2510264232 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:50:33.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.065 7fdfd207b700 1 -- 172.21.2.101:0/2510264232 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029558808 0 0) 0x7fdfcc001500 con 0x7fdfd40cf590 2021-02-10T11:50:33.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.065 7fdfd207b700 1 -- 172.21.2.101:0/2510264232 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfc0002fe0 con 0x7fdfd40cf590 2021-02-10T11:50:33.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.065 7fdfd287c700 1 --2- 172.21.2.101:0/2510264232 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdfd40d1c80 0x7fdfd40cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:33.078 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.066 7fdfd207b700 1 -- 172.21.2.101:0/2510264232 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3298014774 0 0) 0x7fdfcc001460 con 0x7fdfd40cf590 2021-02-10T11:50:33.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.066 7fdfd207b700 1 -- 172.21.2.101:0/2510264232 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdfd40d1c80 msgr2=0x7fdfd40cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:33.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.066 7fdfd207b700 1 --2- 172.21.2.101:0/2510264232 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdfd40d1c80 0x7fdfd40cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:33.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.066 7fdfd207b700 1 -- 172.21.2.101:0/2510264232 >> v1:172.21.2.108:6789/0 conn(0x7fdfd40d1880 legacy=0x7fdfd41b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:33.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.066 7fdfd207b700 1 -- 172.21.2.101:0/2510264232 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd4109a70 con 0x7fdfd40cf590 2021-02-10T11:50:33.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.066 7fdfd207b700 1 -- 172.21.2.101:0/2510264232 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fdfcc001980 con 0x7fdfd40cf590 2021-02-10T11:50:33.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.066 7fdfd207b700 1 -- 172.21.2.101:0/2510264232 >> v1:172.21.2.101:6789/0 conn(0x7fdfd40cf590 legacy=0x7fdfd40cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:33.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.066 7fdfd207b700 1 --2- 172.21.2.101:0/2510264232 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfc0003aa0 0x7fdfc0005ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:33.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.066 7fdfd207b700 1 --2- 172.21.2.101:0/2510264232 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdfc0006770 0x7fdfc0008c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:33.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.066 7fdfd207b700 1 --2- 172.21.2.101:0/2510264232 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdfc0009500 0x7fdfc000ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:33.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.066 7fdfd207b700 1 -- 172.21.2.101:0/2510264232 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc000c2d0 con 0x7fdfc0003aa0 2021-02-10T11:50:33.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.066 7fdfd207b700 1 -- 172.21.2.101:0/2510264232 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc000c4f0 con 0x7fdfc0009500 2021-02-10T11:50:33.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.066 7fdfd207b700 1 -- 172.21.2.101:0/2510264232 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc000c710 con 0x7fdfc0006770 2021-02-10T11:50:33.079 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.066 7fdfd307d700 1 --2- 172.21.2.101:0/2510264232 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfc0003aa0 0x7fdfc0005ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:33.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.066 7fdfd287c700 1 --2- 172.21.2.101:0/2510264232 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdfc0006770 0x7fdfc0008c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:33.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.066 7fdfd387e700 1 --2- 172.21.2.101:0/2510264232 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdfc0009500 0x7fdfc000ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:33.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.066 7fdfd307d700 1 -- 172.21.2.101:0/2510264232 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdfc0006770 msgr2=0x7fdfc0008c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:33.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.066 7fdfd307d700 1 --2- 172.21.2.101:0/2510264232 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdfc0006770 0x7fdfc0008c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:33.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.066 7fdfd307d700 1 -- 172.21.2.101:0/2510264232 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdfc0009500 msgr2=0x7fdfc000ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:33.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.066 7fdfd307d700 1 --2- 172.21.2.101:0/2510264232 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdfc0009500 0x7fdfc000ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:33.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.066 7fdfd307d700 1 -- 172.21.2.101:0/2510264232 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc0050980 con 0x7fdfc0003aa0 2021-02-10T11:50:33.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.066 7fdfd307d700 1 --2- 172.21.2.101:0/2510264232 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfc0003aa0 0x7fdfc0005ed0 secure :-1 s=READY pgs=3433 cs=0 l=1 rev1=1 rx=0x7fdfbc006ac0 tx=0x7fdfbc009a70).ready entity=mon.0 client_cookie=da3d883d128408fb server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:33.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.067 7fdfd207b700 1 -- 172.21.2.101:0/2510264232 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdfbc007280 con 0x7fdfc0003aa0 2021-02-10T11:50:33.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.067 7fdfd207b700 1 -- 172.21.2.101:0/2510264232 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfbc007580 con 0x7fdfc0003aa0 2021-02-10T11:50:33.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.067 7fdfd207b700 1 -- 172.21.2.101:0/2510264232 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdfbc007aa0 con 0x7fdfc0003aa0 2021-02-10T11:50:33.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.067 7fdfd9288700 1 -- 172.21.2.101:0/2510264232 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfc0003aa0 msgr2=0x7fdfc0005ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:33.080 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.067 7fdfd9288700 1 --2- 172.21.2.101:0/2510264232 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfc0003aa0 0x7fdfc0005ed0 secure :-1 s=READY pgs=3433 cs=0 l=1 rev1=1 rx=0x7fdfbc006ac0 tx=0x7fdfbc009a70).stop 2021-02-10T11:50:33.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.067 7fdfd387e700 1 -- 172.21.2.101:0/2510264232 reap_dead start 2021-02-10T11:50:33.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.067 7fdfd9288700 1 -- 172.21.2.101:0/2510264232 shutdown_connections 2021-02-10T11:50:33.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.067 7fdfd9288700 1 -- 172.21.2.101:0/2510264232 >> 172.21.2.101:0/2510264232 conn(0x7fdfd40ce430 msgr2=0x7fdfd40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:33.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.067 7fdfd9288700 1 -- 172.21.2.101:0/2510264232 shutdown_connections 2021-02-10T11:50:33.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.067 7fdfd9288700 1 -- 172.21.2.101:0/2510264232 wait complete. 2021-02-10T11:50:33.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.082 7fdfd9288700 1 Processor -- start 2021-02-10T11:50:33.081 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.082 7fdfd9288700 1 -- start start 2021-02-10T11:50:33.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.083 7fdfd9288700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdfd40b4bb0 0x7fdfd409ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:33.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.083 7fdfd9288700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfd409b400 0x7fdfd409b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:33.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.083 7fdfd9288700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdfd4086480 0x7fdfd4086880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:33.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.083 7fdfd9288700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd40b4100 con 0x7fdfd409b400 2021-02-10T11:50:33.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.083 7fdfd9288700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd409be60 con 0x7fdfd4086480 2021-02-10T11:50:33.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.083 7fdfd9288700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd40b4600 con 0x7fdfd40b4bb0 2021-02-10T11:50:33.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.083 7fdfd287c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfd409b400 0x7fdfd409b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:33.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.083 7fdfd287c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfd409b400 0x7fdfd409b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36130/0 (socket says 172.21.2.101:36130) 2021-02-10T11:50:33.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.083 7fdfd287c700 1 -- 172.21.2.101:0/2703626987 learned_addr learned my addr 172.21.2.101:0/2703626987 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:33.083 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.083 7fdfd287c700 1 -- 172.21.2.101:0/2703626987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdfd40b4bb0 msgr2=0x7fdfd409ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:33.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.083 7fdfd287c700 1 --2- 172.21.2.101:0/2703626987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdfd40b4bb0 0x7fdfd409ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:33.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.083 7fdfd287c700 1 -- 172.21.2.101:0/2703626987 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdfd4086480 msgr2=0x7fdfd4086880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:33.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.083 7fdfd287c700 1 --2- 172.21.2.101:0/2703626987 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdfd4086480 0x7fdfd4086880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:33.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.083 7fdfd287c700 1 -- 172.21.2.101:0/2703626987 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd40b4fb0 con 0x7fdfd409b400 2021-02-10T11:50:33.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.083 7fdfd287c700 1 --2- 172.21.2.101:0/2703626987 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfd409b400 0x7fdfd409b800 secure :-1 s=READY pgs=3434 cs=0 l=1 rev1=1 rx=0x7fdfc4006f60 tx=0x7fdfc400b970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:33.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.084 7fdfcbfff700 1 -- 172.21.2.101:0/2703626987 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdfc400bd60 con 0x7fdfd409b400 2021-02-10T11:50:33.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.084 7fdfcbfff700 1 -- 172.21.2.101:0/2703626987 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc4000c90 con 0x7fdfd409b400 2021-02-10T11:50:33.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.084 7fdfcbfff700 1 -- 172.21.2.101:0/2703626987 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdfc4002300 con 0x7fdfd409b400 2021-02-10T11:50:33.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.084 7fdfd9288700 1 -- 172.21.2.101:0/2703626987 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd40b4fb0 con 0x7fdfd409b400 2021-02-10T11:50:33.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.084 7fdfd9288700 1 -- 172.21.2.101:0/2703626987 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd4087190 con 0x7fdfd409b400 2021-02-10T11:50:33.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.084 7fdfcbfff700 1 -- 172.21.2.101:0/2703626987 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fdfc4006b00 con 0x7fdfd409b400 2021-02-10T11:50:33.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.084 7fdfcbfff700 1 --2- 172.21.2.101:0/2703626987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdfb80186c0 0x7fdfb801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:33.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.084 7fdfcbfff700 1 -- 172.21.2.101:0/2703626987 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fdfc4032550 con 0x7fdfd409b400 2021-02-10T11:50:33.084 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.084 7fdfd307d700 1 --2- 172.21.2.101:0/2703626987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdfb80186c0 0x7fdfb801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:33.085 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.085 7fdfd307d700 1 --2- 172.21.2.101:0/2703626987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdfb80186c0 0x7fdfb801abc0 crc :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:33.110 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.115 7fdfd9288700 1 -- 172.21.2.101:0/2703626987 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd40b45a0 con 0x7fdfd409b400 2021-02-10T11:50:33.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.117 7fdfcbfff700 1 -- 172.21.2.101:0/2703626987 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fdfc4000940 con 0x7fdfd409b400 2021-02-10T11:50:33.241 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.245 7fdfd9288700 1 -- 172.21.2.101:0/2703626987 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000951555", "target": ["mgr", ""], "snap_name": "snapshot_0000000000014068"}) v1 -- 0x7fdfd4087190 con 0x7fdfb80186c0 2021-02-10T11:50:33.251 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.256 7fdfcbfff700 1 -- 172.21.2.101:0/2703626987 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fdfbc001fa0 con 0x7fdfb80186c0 2021-02-10T11:50:33.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.262 7fdfd9288700 1 -- 172.21.2.101:0/2703626987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdfb80186c0 msgr2=0x7fdfb801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:33.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.262 7fdfd9288700 1 --2- 172.21.2.101:0/2703626987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdfb80186c0 0x7fdfb801abc0 crc :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:33.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.262 7fdfd9288700 1 -- 172.21.2.101:0/2703626987 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfd409b400 msgr2=0x7fdfd409b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:33.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.262 7fdfd9288700 1 --2- 172.21.2.101:0/2703626987 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfd409b400 0x7fdfd409b800 secure :-1 s=READY pgs=3434 cs=0 l=1 rev1=1 rx=0x7fdfc4006f60 tx=0x7fdfc400b970).stop 2021-02-10T11:50:33.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.262 7fdfd9288700 1 -- 172.21.2.101:0/2703626987 shutdown_connections 2021-02-10T11:50:33.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.262 7fdfd9288700 1 --2- 172.21.2.101:0/2703626987 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdfd4086480 0x7fdfd4086880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:33.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.262 7fdfd9288700 1 --2- 172.21.2.101:0/2703626987 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdfd409b400 0x7fdfd409b800 unknown :-1 s=CLOSED pgs=3434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:33.258 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.262 7fdfd9288700 1 --2- 172.21.2.101:0/2703626987 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fdfb80186c0 0x7fdfb801abc0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:33.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.262 7fdfd9288700 1 --2- 172.21.2.101:0/2703626987 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdfd40b4bb0 0x7fdfd409ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:33.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.262 7fdfd9288700 1 -- 172.21.2.101:0/2703626987 >> 172.21.2.101:0/2703626987 conn(0x7fdfd409ebf0 msgr2=0x7fdfd40b53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:33.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.262 7fdfd9288700 1 -- 172.21.2.101:0/2703626987 shutdown_connections 2021-02-10T11:50:33.259 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.263 7fdfd9288700 1 -- 172.21.2.101:0/2703626987 wait complete. 2021-02-10T11:50:33.266 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume snapshot clone cephfs subvolume_0000000000951555 snapshot_0000000000014068 clone_0000000000698700 2021-02-10T11:50:33.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.685 7fa8e15de700 1 Processor -- start 2021-02-10T11:50:33.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.685 7fa8e15de700 1 -- start start 2021-02-10T11:50:33.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.686 7fa8e15de700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8dc0eb220 0x7fa8dc0eb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:33.697 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.686 7fa8e15de700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa8dc10aad0 0x7fa8dc10aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:33.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.686 7fa8e15de700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8dc10b4f0 con 0x7fa8dc0dce80 2021-02-10T11:50:33.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.686 7fa8e15de700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8dc10a550 con 0x7fa8dc10aad0 2021-02-10T11:50:33.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.686 7fa8e15de700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc0881d0 con 0x7fa8dc0eb220 2021-02-10T11:50:33.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.686 7fa8dbac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa8dc10aad0 0x7fa8dc10aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:33.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.686 7fa8dbac0700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fa8dc10aad0 0x7fa8dc10aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36138/0 (socket says 172.21.2.101:36138) 2021-02-10T11:50:33.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.686 7fa8dbac0700 1 -- 172.21.2.101:0/2042614899 learned_addr learned my addr 172.21.2.101:0/2042614899 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:33.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.686 7fa8dbac0700 1 -- 172.21.2.101:0/2042614899 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8dc0eb220 msgr2=0x7fa8dc0eb620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:50:33.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.686 7fa8dbac0700 1 --2- 172.21.2.101:0/2042614899 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8dc0eb220 0x7fa8dc0eb620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:33.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.686 7fa8dbac0700 1 -- 172.21.2.101:0/2042614899 >> v1:172.21.2.108:6789/0 conn(0x7fa8dc0dce80 legacy=0x7fa8dc0d14f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:50:33.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.686 7fa8dbac0700 1 -- 172.21.2.101:0/2042614899 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc0d1f30 con 0x7fa8dc10aad0 2021-02-10T11:50:33.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.686 7fa8dbac0700 1 --2- 172.21.2.101:0/2042614899 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8dc10aad0 0x7fa8dc10aed0 secure :-1 s=READY pgs=3435 cs=0 l=1 rev1=1 rx=0x7fa8d4006f00 tx=0x7fa8d4009f10).ready entity=mon.0 client_cookie=6f185a3c1338838b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:33.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.687 7fa8da2bd700 1 -- 172.21.2.101:0/2042614899 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa8d400b3b0 con 0x7fa8dc10aad0 2021-02-10T11:50:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.687 7fa8da2bd700 1 -- 172.21.2.101:0/2042614899 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d400b730 con 0x7fa8dc10aad0 2021-02-10T11:50:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.687 7fa8da2bd700 1 -- 172.21.2.101:0/2042614899 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa8d400bc50 con 0x7fa8dc10aad0 2021-02-10T11:50:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.687 7fa8e15de700 1 -- 172.21.2.101:0/2042614899 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8dc10aad0 msgr2=0x7fa8dc10aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.687 7fa8e15de700 1 --2- 172.21.2.101:0/2042614899 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8dc10aad0 0x7fa8dc10aed0 secure :-1 s=READY pgs=3435 cs=0 l=1 rev1=1 rx=0x7fa8d4006f00 tx=0x7fa8d4009f10).stop 2021-02-10T11:50:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.687 7fa8e15de700 1 -- 172.21.2.101:0/2042614899 shutdown_connections 2021-02-10T11:50:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.687 7fa8e15de700 1 --2- 172.21.2.101:0/2042614899 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8dc0eb220 0x7fa8dc0eb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.687 7fa8e15de700 1 --2- 172.21.2.101:0/2042614899 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8dc10aad0 0x7fa8dc10aed0 unknown :-1 s=CLOSED pgs=3435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.687 7fa8e15de700 1 -- 172.21.2.101:0/2042614899 >> 172.21.2.101:0/2042614899 conn(0x7fa8dc0ea630 msgr2=0x7fa8dc0d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.687 7fa8e15de700 1 -- 172.21.2.101:0/2042614899 shutdown_connections 2021-02-10T11:50:33.699 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.687 7fa8e15de700 1 -- 172.21.2.101:0/2042614899 wait complete. 2021-02-10T11:50:33.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.702 7fa8e15de700 1 Processor -- start 2021-02-10T11:50:33.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.702 7fa8e15de700 1 -- start start 2021-02-10T11:50:33.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.702 7fa8e15de700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8dc107e70 0x7fa8dc090900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:33.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.702 7fa8e15de700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8dc0b0fa0 0x7fa8dc08c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:33.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.702 7fa8e15de700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8dc08ca80 0x7fa8dc08ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:33.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.702 7fa8e15de700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc0910e0 con 0x7fa8dc0b0fa0 2021-02-10T11:50:33.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.702 7fa8e15de700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc091620 con 0x7fa8dc107e70 2021-02-10T11:50:33.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.702 7fa8e15de700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc091840 con 0x7fa8dc08ca80 2021-02-10T11:50:33.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.702 7fa8daabe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8dc0b0fa0 0x7fa8dc08c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:33.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.702 7fa8daabe700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8dc0b0fa0 0x7fa8dc08c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36144/0 (socket says 172.21.2.101:36144) 2021-02-10T11:50:33.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.702 7fa8daabe700 1 -- 172.21.2.101:0/3366420522 learned_addr learned my addr 172.21.2.101:0/3366420522 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:33.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.703 7fa8db2bf700 1 --2- 172.21.2.101:0/3366420522 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8dc107e70 0x7fa8dc090900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:33.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.703 7fa8dbac0700 1 --2- 172.21.2.101:0/3366420522 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8dc08ca80 0x7fa8dc08ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:33.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.703 7fa8daabe700 1 -- 172.21.2.101:0/3366420522 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8dc08ca80 msgr2=0x7fa8dc08ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:33.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.703 7fa8daabe700 1 --2- 172.21.2.101:0/3366420522 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8dc08ca80 0x7fa8dc08ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:33.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.703 7fa8daabe700 1 -- 172.21.2.101:0/3366420522 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8dc107e70 msgr2=0x7fa8dc090900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:33.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.703 7fa8daabe700 1 --2- 172.21.2.101:0/3366420522 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8dc107e70 0x7fa8dc090900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:33.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.703 7fa8daabe700 1 -- 172.21.2.101:0/3366420522 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc091a60 con 0x7fa8dc0b0fa0 2021-02-10T11:50:33.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.703 7fa8dbac0700 1 --2- 172.21.2.101:0/3366420522 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8dc08ca80 0x7fa8dc08ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:50:33.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.703 7fa8daabe700 1 --2- 172.21.2.101:0/3366420522 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8dc0b0fa0 0x7fa8dc08c460 secure :-1 s=READY pgs=3436 cs=0 l=1 rev1=1 rx=0x7fa8c400b5c0 tx=0x7fa8c400b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:33.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.703 7fa8d3fff700 1 -- 172.21.2.101:0/3366420522 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa8c40009b0 con 0x7fa8dc0b0fa0 2021-02-10T11:50:33.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.703 7fa8d3fff700 1 -- 172.21.2.101:0/3366420522 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c400bd50 con 0x7fa8dc0b0fa0 2021-02-10T11:50:33.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.703 7fa8d3fff700 1 -- 172.21.2.101:0/3366420522 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa8c4006240 con 0x7fa8dc0b0fa0 2021-02-10T11:50:33.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.703 7fa8e15de700 1 -- 172.21.2.101:0/3366420522 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8dc091a60 con 0x7fa8dc0b0fa0 2021-02-10T11:50:33.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.704 7fa8e15de700 1 -- 172.21.2.101:0/3366420522 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8dc091ff0 con 0x7fa8dc0b0fa0 2021-02-10T11:50:33.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.704 7fa8d3fff700 1 -- 172.21.2.101:0/3366420522 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa8c4006030 con 0x7fa8dc0b0fa0 2021-02-10T11:50:33.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.704 7fa8d3fff700 1 --2- 172.21.2.101:0/3366420522 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8c00186c0 0x7fa8c001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:33.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.704 7fa8d3fff700 1 -- 172.21.2.101:0/3366420522 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa8c4032420 con 0x7fa8dc0b0fa0 2021-02-10T11:50:33.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.704 7fa8db2bf700 1 --2- 172.21.2.101:0/3366420522 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8c00186c0 0x7fa8c001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:33.703 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.704 7fa8db2bf700 1 --2- 172.21.2.101:0/3366420522 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8c00186c0 0x7fa8c001abc0 crc :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:33.731 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.736 7fa8e15de700 1 -- 172.21.2.101:0/3366420522 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8dc0f9360 con 0x7fa8dc0b0fa0 2021-02-10T11:50:33.734 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.738 7fa8d3fff700 1 -- 172.21.2.101:0/3366420522 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa8c4000940 con 0x7fa8dc0b0fa0 2021-02-10T11:50:33.861 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.866 7fa8e15de700 1 -- 172.21.2.101:0/3366420522 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvolume_0000000000951555", "target": ["mgr", ""], "snap_name": "snapshot_0000000000014068", "target_sub_name": "clone_0000000000698700"}) v1 -- 0x7fa8dc10a550 con 0x7fa8c00186c0 2021-02-10T11:50:33.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.912 7fa8d3fff700 1 -- 172.21.2.101:0/3366420522 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fa8cc006d20 con 0x7fa8c00186c0 2021-02-10T11:50:33.926 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.931 7fa8e15de700 1 -- 172.21.2.101:0/3366420522 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8c00186c0 msgr2=0x7fa8c001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:33.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.931 7fa8e15de700 1 --2- 172.21.2.101:0/3366420522 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8c00186c0 0x7fa8c001abc0 crc :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:33.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.931 7fa8e15de700 1 -- 172.21.2.101:0/3366420522 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8dc0b0fa0 msgr2=0x7fa8dc08c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:33.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.931 7fa8e15de700 1 --2- 172.21.2.101:0/3366420522 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8dc0b0fa0 0x7fa8dc08c460 secure :-1 s=READY pgs=3436 cs=0 l=1 rev1=1 rx=0x7fa8c400b5c0 tx=0x7fa8c400b960).stop 2021-02-10T11:50:33.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.931 7fa8e15de700 1 -- 172.21.2.101:0/3366420522 shutdown_connections 2021-02-10T11:50:33.927 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.931 7fa8e15de700 1 --2- 172.21.2.101:0/3366420522 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa8c00186c0 0x7fa8c001abc0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:33.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.931 7fa8e15de700 1 --2- 172.21.2.101:0/3366420522 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa8dc08ca80 0x7fa8dc08ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:33.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.931 7fa8e15de700 1 --2- 172.21.2.101:0/3366420522 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa8dc0b0fa0 0x7fa8dc08c460 unknown :-1 s=CLOSED pgs=3436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:33.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.931 7fa8e15de700 1 --2- 172.21.2.101:0/3366420522 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa8dc107e70 0x7fa8dc090900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:33.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.931 7fa8e15de700 1 -- 172.21.2.101:0/3366420522 >> 172.21.2.101:0/3366420522 conn(0x7fa8dc1a8260 msgr2=0x7fa8dc1a7230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:33.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.931 7fa8e15de700 1 -- 172.21.2.101:0/3366420522 shutdown_connections 2021-02-10T11:50:33.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:33.931 7fa8e15de700 1 -- 172.21.2.101:0/3366420522 wait complete. 2021-02-10T11:50:33.935 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph fs subvolume getpath cephfs clone_0000000000698700 2021-02-10T11:50:34.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.361 7f626ff3c700 1 Processor -- start 2021-02-10T11:50:34.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.361 7f626ff3c700 1 -- start start 2021-02-10T11:50:34.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.362 7f626ff3c700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f62680d1860 0x7f62681b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:34.374 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.362 7f626ff3c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f62680cff60 0x7f6268109120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:34.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.362 7f626ff3c700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62680fe1b0 con 0x7f62681b0ad0 2021-02-10T11:50:34.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.362 7f626ff3c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6268109ac0 con 0x7f62680cff60 2021-02-10T11:50:34.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.362 7f626ff3c700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6268109740 con 0x7f62680d1860 2021-02-10T11:50:34.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.362 7f626dd58700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f62680d1860 0x7f62681b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:34.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.362 7f626e559700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f62680cff60 0x7f6268109120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:34.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.362 7f626e559700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f62680cff60 0x7f6268109120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36162/0 (socket says 172.21.2.101:36162) 2021-02-10T11:50:34.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.362 7f626e559700 1 -- 172.21.2.101:0/312906127 learned_addr learned my addr 172.21.2.101:0/312906127 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:34.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.362 7f626dd58700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f62680d1860 0x7f62681b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:46644/0 (socket says 172.21.2.101:46644) 2021-02-10T11:50:34.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.362 7f626dd58700 1 -- 172.21.2.101:0/312906127 >> v2:172.21.2.101:3300/0 conn(0x7f62680cff60 msgr2=0x7f6268109120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:34.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.362 7f626dd58700 1 --2- 172.21.2.101:0/312906127 >> v2:172.21.2.101:3300/0 conn(0x7f62680cff60 0x7f6268109120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:34.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.362 7f626dd58700 1 -- 172.21.2.101:0/312906127 >> v1:172.21.2.108:6789/0 conn(0x7f62681b0ad0 legacy=0x7f6268108930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:34.375 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.362 7f626dd58700 1 -- 172.21.2.101:0/312906127 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6268088070 con 0x7f62680d1860 2021-02-10T11:50:34.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.362 7f626dd58700 1 --2- 172.21.2.101:0/312906127 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62680d1860 0x7f62681b04b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f62600069d0 tx=0x7f6260009dc0).ready entity=mon.1 client_cookie=bc0b4b99a2f67698 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:34.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.362 7f626cd56700 1 -- 172.21.2.101:0/312906127 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f626000b510 con 0x7f62680d1860 2021-02-10T11:50:34.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.362 7f626cd56700 1 -- 172.21.2.101:0/312906127 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6260006ac0 con 0x7f62680d1860 2021-02-10T11:50:34.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.362 7f626cd56700 1 -- 172.21.2.101:0/312906127 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f626000ba80 con 0x7f62680d1860 2021-02-10T11:50:34.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.362 7f626ff3c700 1 -- 172.21.2.101:0/312906127 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62680d1860 msgr2=0x7f62681b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:34.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.362 7f626ff3c700 1 --2- 172.21.2.101:0/312906127 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62680d1860 0x7f62681b04b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f62600069d0 tx=0x7f6260009dc0).stop 2021-02-10T11:50:34.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.363 7f626ff3c700 1 -- 172.21.2.101:0/312906127 shutdown_connections 2021-02-10T11:50:34.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.363 7f626ff3c700 1 --2- 172.21.2.101:0/312906127 >> v2:172.21.2.101:3300/0 conn(0x7f62680cff60 0x7f6268109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:34.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.363 7f626ff3c700 1 --2- 172.21.2.101:0/312906127 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62680d1860 0x7f62681b04b0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:34.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.363 7f626ff3c700 1 -- 172.21.2.101:0/312906127 >> 172.21.2.101:0/312906127 conn(0x7f62680ce410 msgr2=0x7f62680d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:34.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.363 7f626ff3c700 1 -- 172.21.2.101:0/312906127 shutdown_connections 2021-02-10T11:50:34.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.363 7f626ff3c700 1 -- 172.21.2.101:0/312906127 wait complete. 2021-02-10T11:50:34.376 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.378 7f626ff3c700 1 Processor -- start 2021-02-10T11:50:34.377 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.379 7f626ff3c700 1 -- start start 2021-02-10T11:50:34.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.379 7f626ff3c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62680adf50 0x7f62680ae350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:34.378 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.379 7f626ff3c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62680ae970 0x7f62680aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:34.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.379 7f626ff3c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62680982c0 0x7f62680986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:34.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.379 7f626ff3c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626809f450 con 0x7f62680982c0 2021-02-10T11:50:34.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.379 7f626ff3c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6268098ce0 con 0x7f62680adf50 2021-02-10T11:50:34.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.379 7f626ff3c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6268098ed0 con 0x7f62680ae970 2021-02-10T11:50:34.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.379 7f626e559700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62680982c0 0x7f62680986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:34.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.379 7f626e559700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62680982c0 0x7f62680986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36166/0 (socket says 172.21.2.101:36166) 2021-02-10T11:50:34.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.379 7f626e559700 1 -- 172.21.2.101:0/757163585 learned_addr learned my addr 172.21.2.101:0/757163585 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:34.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.379 7f626e559700 1 -- 172.21.2.101:0/757163585 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62680ae970 msgr2=0x7f62680aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:34.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.379 7f626e559700 1 --2- 172.21.2.101:0/757163585 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62680ae970 0x7f62680aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:34.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.379 7f626e559700 1 -- 172.21.2.101:0/757163585 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62680adf50 msgr2=0x7f62680ae350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:34.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.379 7f626e559700 1 --2- 172.21.2.101:0/757163585 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62680adf50 0x7f62680ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:34.379 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.379 7f626e559700 1 -- 172.21.2.101:0/757163585 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62680990f0 con 0x7f62680982c0 2021-02-10T11:50:34.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.379 7f626e559700 1 --2- 172.21.2.101:0/757163585 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62680982c0 0x7f62680986c0 secure :-1 s=READY pgs=3437 cs=0 l=1 rev1=1 rx=0x7f626400b590 tx=0x7f626400b930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:34.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.379 7f625e7fc700 1 -- 172.21.2.101:0/757163585 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6264000c10 con 0x7f62680982c0 2021-02-10T11:50:34.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.380 7f625e7fc700 1 -- 172.21.2.101:0/757163585 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626400bd20 con 0x7f62680982c0 2021-02-10T11:50:34.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.380 7f626ff3c700 1 -- 172.21.2.101:0/757163585 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62680990f0 con 0x7f62680982c0 2021-02-10T11:50:34.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.380 7f625e7fc700 1 -- 172.21.2.101:0/757163585 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f62640054a0 con 0x7f62680982c0 2021-02-10T11:50:34.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.380 7f626ff3c700 1 -- 172.21.2.101:0/757163585 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62680990f0 con 0x7f62680982c0 2021-02-10T11:50:34.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.380 7f625e7fc700 1 -- 172.21.2.101:0/757163585 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6264000ab0 con 0x7f62680982c0 2021-02-10T11:50:34.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.380 7f625e7fc700 1 --2- 172.21.2.101:0/757163585 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6250018730 0x7f625001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:34.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.380 7f625e7fc700 1 -- 172.21.2.101:0/757163585 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6264032600 con 0x7f62680982c0 2021-02-10T11:50:34.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.380 7f626dd58700 1 --2- 172.21.2.101:0/757163585 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6250018730 0x7f625001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:34.380 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.380 7f626dd58700 1 --2- 172.21.2.101:0/757163585 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6250018730 0x7f625001ac10 crc :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:34.406 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.410 7f626ff3c700 1 -- 172.21.2.101:0/757163585 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6268082a20 con 0x7f62680982c0 2021-02-10T11:50:34.409 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.413 7f625e7fc700 1 -- 172.21.2.101:0/757163585 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6264006030 con 0x7f62680982c0 2021-02-10T11:50:34.544 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.548 7f626ff3c700 1 -- 172.21.2.101:0/757163585 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "clone_0000000000698700", "target": ["mgr", ""]}) v1 -- 0x7f6268098ce0 con 0x7f6250018730 2021-02-10T11:50:34.546 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.551 7f625e7fc700 1 -- 172.21.2.101:0/757163585 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+78 (crc 0 0 0) 0x7f6260006850 con 0x7f6250018730 2021-02-10T11:50:34.547 INFO:teuthology.orchestra.run.gibba001.stdout:/volumes/_nogroup/clone_0000000000698700/007402a5-fc92-4b21-b7d2-775c80084b01 2021-02-10T11:50:34.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.553 7f626ff3c700 1 -- 172.21.2.101:0/757163585 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6250018730 msgr2=0x7f625001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:34.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.553 7f626ff3c700 1 --2- 172.21.2.101:0/757163585 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6250018730 0x7f625001ac10 crc :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:34.549 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.553 7f626ff3c700 1 -- 172.21.2.101:0/757163585 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62680982c0 msgr2=0x7f62680986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:34.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.553 7f626ff3c700 1 --2- 172.21.2.101:0/757163585 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62680982c0 0x7f62680986c0 secure :-1 s=READY pgs=3437 cs=0 l=1 rev1=1 rx=0x7f626400b590 tx=0x7f626400b930).stop 2021-02-10T11:50:34.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.553 7f626ff3c700 1 -- 172.21.2.101:0/757163585 shutdown_connections 2021-02-10T11:50:34.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.553 7f626ff3c700 1 --2- 172.21.2.101:0/757163585 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6250018730 0x7f625001ac10 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:34.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.553 7f626ff3c700 1 --2- 172.21.2.101:0/757163585 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f62680ae970 0x7f62680aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:34.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.553 7f626ff3c700 1 --2- 172.21.2.101:0/757163585 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f62680982c0 0x7f62680986c0 unknown :-1 s=CLOSED pgs=3437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:34.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.553 7f626ff3c700 1 --2- 172.21.2.101:0/757163585 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f62680adf50 0x7f62680ae350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:34.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.553 7f626ff3c700 1 -- 172.21.2.101:0/757163585 >> 172.21.2.101:0/757163585 conn(0x7f62680bee90 msgr2=0x7f62680bb880 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:34.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.553 7f626ff3c700 1 -- 172.21.2.101:0/757163585 shutdown_connections 2021-02-10T11:50:34.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:34.553 7f626ff3c700 1 -- 172.21.2.101:0/757163585 wait complete. 2021-02-10T11:50:34.557 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-02-10T11:50:34.557 DEBUG:teuthology.orchestra.run.gibba001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:50:34.902 DEBUG:teuthology.orchestra.run.gibba008:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-02-10T11:50:35.248 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-02-10T11:50:35.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:35.248 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:35.248 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:35.584 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:35.584 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:35.585 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:50:35.585 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:50:35.585 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:35.941 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba001.stderr:ceph-fuse[186294]: fuse finished with error 0 and tester_r 0 2021-02-10T11:50:35.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:35.943 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:35.943 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:36.278 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:50:36.278 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:50:36.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:50:36.279 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:36.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:36.616 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:36.616 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:50:36.953 INFO:teuthology.orchestra.run.gibba001.stdout:fuseblk 2021-02-10T11:50:36.953 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-02-10T11:50:36.953 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba001.front.sepia.ceph.com... 2021-02-10T11:50:36.953 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-02-10T11:50:36.953 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:50:37.308 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba001.stderr:ceph-fuse[186961]: fuse finished with error 0 and tester_r 0 2021-02-10T11:50:37.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:37.309 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:37.309 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:50:37.644 INFO:teuthology.orchestra.run.gibba001.stdout:ext2/ext3 2021-02-10T11:50:37.644 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-02-10T11:50:37.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:50:37.645 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:50:37.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:37.981 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:37.981 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:50:38.315 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:50:38.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:50:38.316 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:50:38.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:50:38.316 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:50:38.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:38.650 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:38.650 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:50:38.985 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:50:38.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:50:38.986 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:50:38.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-02-10T11:50:38.986 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:50:39.320 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-02-10T11:50:39.753 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.741 7f96bef74700 1 Processor -- start 2021-02-10T11:50:39.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.741 7f96bef74700 1 -- start start 2021-02-10T11:50:39.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.741 7f96bef74700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f96b80d1860 0x7f96b81b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:39.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.741 7f96bef74700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f96b81b0a00 0x7f96b80cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:39.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.741 7f96bef74700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b80f0b00 con 0x7f96b80d1c60 2021-02-10T11:50:39.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.741 7f96bef74700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b80e5ec0 con 0x7f96b80d1860 2021-02-10T11:50:39.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.741 7f96bef74700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b80f0640 con 0x7f96b81b0a00 2021-02-10T11:50:39.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.742 7f96bcd90700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f96b80d1860 0x7f96b81b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:39.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.742 7f96bd591700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f96b80d1c60 0x7f96b80cf940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36670/0 (socket says 172.21.2.101:36670) 2021-02-10T11:50:39.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.742 7f96bd591700 1 -- 172.21.2.101:0/732479327 learned_addr learned my addr 172.21.2.101:0/732479327 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:50:39.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.742 7f96bcd90700 1 -- 172.21.2.101:0/732479327 >> v2:172.21.2.108:3300/0 conn(0x7f96b81b0a00 msgr2=0x7f96b80cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:39.754 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.742 7f96b7fff700 1 --2- 172.21.2.101:0/732479327 >> v2:172.21.2.108:3300/0 conn(0x7f96b81b0a00 0x7f96b80cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:39.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.742 7f96b77fe700 1 -- 172.21.2.101:0/732479327 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 548645952 0 0) 0x7f96b0001500 con 0x7f96b80d1c60 2021-02-10T11:50:39.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.742 7f96bcd90700 1 --2- 172.21.2.101:0/732479327 >> v2:172.21.2.108:3300/0 conn(0x7f96b81b0a00 0x7f96b80cf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:39.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.742 7f96bcd90700 1 -- 172.21.2.101:0/732479327 >> v1:172.21.2.101:6789/0 conn(0x7f96b80d1c60 legacy=0x7f96b80cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:39.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.742 7f96bcd90700 1 -- 172.21.2.101:0/732479327 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b80f0830 con 0x7f96b80d1860 2021-02-10T11:50:39.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.742 7f96b7fff700 1 --2- 172.21.2.101:0/732479327 >> v2:172.21.2.108:3300/0 conn(0x7f96b81b0a00 0x7f96b80cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:50:39.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.742 7f96bcd90700 1 --2- 172.21.2.101:0/732479327 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96b80d1860 0x7f96b81b03e0 secure :-1 s=READY pgs=3439 cs=0 l=1 rev1=1 rx=0x7f96a0006650 tx=0x7f96a0009a80).ready entity=mon.0 client_cookie=c019d505bf7cb9a0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:39.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.743 7f96b77fe700 1 -- 172.21.2.101:0/732479327 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96a0006390 con 0x7f96b80d1860 2021-02-10T11:50:39.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.743 7f96b77fe700 1 -- 172.21.2.101:0/732479327 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a00009b0 con 0x7f96b80d1860 2021-02-10T11:50:39.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.743 7f96b77fe700 1 -- 172.21.2.101:0/732479327 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96a000c2b0 con 0x7f96b80d1860 2021-02-10T11:50:39.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.743 7f96bef74700 1 -- 172.21.2.101:0/732479327 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96b80d1860 msgr2=0x7f96b81b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:39.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.743 7f96bef74700 1 --2- 172.21.2.101:0/732479327 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96b80d1860 0x7f96b81b03e0 secure :-1 s=READY pgs=3439 cs=0 l=1 rev1=1 rx=0x7f96a0006650 tx=0x7f96a0009a80).stop 2021-02-10T11:50:39.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.743 7f96bef74700 1 -- 172.21.2.101:0/732479327 shutdown_connections 2021-02-10T11:50:39.755 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.743 7f96bef74700 1 --2- 172.21.2.101:0/732479327 >> v2:172.21.2.108:3300/0 conn(0x7f96b81b0a00 0x7f96b80cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:39.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.743 7f96bef74700 1 --2- 172.21.2.101:0/732479327 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96b80d1860 0x7f96b81b03e0 unknown :-1 s=CLOSED pgs=3439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:39.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.743 7f96bef74700 1 -- 172.21.2.101:0/732479327 >> 172.21.2.101:0/732479327 conn(0x7f96b80ce440 msgr2=0x7f96b80d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:39.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.743 7f96bef74700 1 -- 172.21.2.101:0/732479327 shutdown_connections 2021-02-10T11:50:39.756 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.743 7f96bef74700 1 -- 172.21.2.101:0/732479327 wait complete. 2021-02-10T11:50:39.757 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.758 7f96bef74700 1 Processor -- start 2021-02-10T11:50:39.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.758 7f96bef74700 1 -- start start 2021-02-10T11:50:39.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.758 7f96bef74700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96b80bfa10 0x7f96b8098b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:39.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.759 7f96bef74700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96b80b7470 0x7f96b80b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:39.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.759 7f96bef74700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96b80c0710 0x7f96b80c0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:39.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.759 7f96bef74700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b80c0190 con 0x7f96b80c0710 2021-02-10T11:50:39.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.759 7f96bef74700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b80bfe10 con 0x7f96b80bfa10 2021-02-10T11:50:39.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.759 7f96bef74700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b80c1130 con 0x7f96b80b7470 2021-02-10T11:50:39.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.759 7f96bd591700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96b80c0710 0x7f96b80c0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:39.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.759 7f96bd591700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96b80c0710 0x7f96b80c0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36182/0 (socket says 172.21.2.101:36182) 2021-02-10T11:50:39.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.759 7f96bd591700 1 -- 172.21.2.101:0/2316250534 learned_addr learned my addr 172.21.2.101:0/2316250534 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:39.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.759 7f96bcd90700 1 --2- 172.21.2.101:0/2316250534 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96b80bfa10 0x7f96b8098b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:39.758 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.759 7f96bd591700 1 -- 172.21.2.101:0/2316250534 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96b80b7470 msgr2=0x7f96b80b7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:39.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.759 7f96b7fff700 1 --2- 172.21.2.101:0/2316250534 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96b80b7470 0x7f96b80b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:39.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.759 7f96bd591700 1 --2- 172.21.2.101:0/2316250534 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96b80b7470 0x7f96b80b7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:39.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.759 7f96bd591700 1 -- 172.21.2.101:0/2316250534 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96b80bfa10 msgr2=0x7f96b8098b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:39.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.759 7f96bd591700 1 --2- 172.21.2.101:0/2316250534 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96b80bfa10 0x7f96b8098b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:39.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.759 7f96bd591700 1 -- 172.21.2.101:0/2316250534 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b80c1350 con 0x7f96b80c0710 2021-02-10T11:50:39.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.759 7f96bcd90700 1 --2- 172.21.2.101:0/2316250534 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96b80bfa10 0x7f96b8098b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:50:39.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.759 7f96b7fff700 1 --2- 172.21.2.101:0/2316250534 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96b80b7470 0x7f96b80b7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:50:39.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.759 7f96bd591700 1 --2- 172.21.2.101:0/2316250534 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96b80c0710 0x7f96b80c0b10 secure :-1 s=READY pgs=3440 cs=0 l=1 rev1=1 rx=0x7f96b0001c10 tx=0x7f96b000ac60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:39.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.760 7f96b57fa700 1 -- 172.21.2.101:0/2316250534 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96b0000f90 con 0x7f96b80c0710 2021-02-10T11:50:39.759 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.760 7f96b57fa700 1 -- 172.21.2.101:0/2316250534 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b00070d0 con 0x7f96b80c0710 2021-02-10T11:50:39.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.760 7f96b57fa700 1 -- 172.21.2.101:0/2316250534 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96b0004720 con 0x7f96b80c0710 2021-02-10T11:50:39.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.760 7f96bef74700 1 -- 172.21.2.101:0/2316250534 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b80c1350 con 0x7f96b80c0710 2021-02-10T11:50:39.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.760 7f96bef74700 1 -- 172.21.2.101:0/2316250534 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b80c1350 con 0x7f96b80c0710 2021-02-10T11:50:39.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.760 7f96b57fa700 1 -- 172.21.2.101:0/2316250534 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f96b00072b0 con 0x7f96b80c0710 2021-02-10T11:50:39.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.760 7f96b57fa700 1 --2- 172.21.2.101:0/2316250534 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f969c0186c0 0x7f969c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:39.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.760 7f96b57fa700 1 -- 172.21.2.101:0/2316250534 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f96b00328e0 con 0x7f96b80c0710 2021-02-10T11:50:39.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.761 7f96bcd90700 1 --2- 172.21.2.101:0/2316250534 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f969c0186c0 0x7f969c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:39.760 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.761 7f96bcd90700 1 --2- 172.21.2.101:0/2316250534 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f969c0186c0 0x7f969c01abc0 crc :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:39.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.791 7f96bef74700 1 -- 172.21.2.101:0/2316250534 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b80ebbe0 con 0x7f96b80c0710 2021-02-10T11:50:39.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.794 7f96b57fa700 1 -- 172.21.2.101:0/2316250534 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f96b0000cc0 con 0x7f96b80c0710 2021-02-10T11:50:39.939 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.943 7f96bef74700 1 -- 172.21.2.101:0/2316250534 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f96b80f0b00 con 0x7f96b80c0710 2021-02-10T11:50:39.944 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.948 7f96b57fa700 1 -- 172.21.2.101:0/2316250534 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7f96b00072b0 con 0x7f96b80c0710 2021-02-10T11:50:39.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.952 7f96bef74700 1 -- 172.21.2.101:0/2316250534 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f969c0186c0 msgr2=0x7f969c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:39.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.952 7f96bef74700 1 --2- 172.21.2.101:0/2316250534 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f969c0186c0 0x7f969c01abc0 crc :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:39.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.952 7f96bef74700 1 -- 172.21.2.101:0/2316250534 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96b80c0710 msgr2=0x7f96b80c0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:39.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.952 7f96bef74700 1 --2- 172.21.2.101:0/2316250534 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96b80c0710 0x7f96b80c0b10 secure :-1 s=READY pgs=3440 cs=0 l=1 rev1=1 rx=0x7f96b0001c10 tx=0x7f96b000ac60).stop 2021-02-10T11:50:39.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.952 7f96bef74700 1 -- 172.21.2.101:0/2316250534 shutdown_connections 2021-02-10T11:50:39.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.952 7f96bef74700 1 --2- 172.21.2.101:0/2316250534 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f969c0186c0 0x7f969c01abc0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:39.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.952 7f96bef74700 1 --2- 172.21.2.101:0/2316250534 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96b80b7470 0x7f96b80b7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:39.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.952 7f96bef74700 1 --2- 172.21.2.101:0/2316250534 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96b80c0710 0x7f96b80c0b10 unknown :-1 s=CLOSED pgs=3440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:39.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.952 7f96bef74700 1 --2- 172.21.2.101:0/2316250534 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96b80bfa10 0x7f96b8098b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:39.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.952 7f96bef74700 1 -- 172.21.2.101:0/2316250534 >> 172.21.2.101:0/2316250534 conn(0x7f96b809fd30 msgr2=0x7f96b809a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:39.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.952 7f96bef74700 1 -- 172.21.2.101:0/2316250534 shutdown_connections 2021-02-10T11:50:39.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:39.952 7f96bef74700 1 -- 172.21.2.101:0/2316250534 wait complete. 2021-02-10T11:50:39.957 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_subvolume_clone_in_progress_getpath' 2021-02-10T11:50:40.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.373 7fc190330700 1 Processor -- start 2021-02-10T11:50:40.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.373 7fc190330700 1 -- start start 2021-02-10T11:50:40.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.374 7fc190330700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fc1880d1870 0x7fc1881b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:40.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.374 7fc190330700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1880f0b10 con 0x7fc1880f0650 2021-02-10T11:50:40.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.374 7fc190330700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1880d0310 con 0x7fc1881b0a10 2021-02-10T11:50:40.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.374 7fc190330700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1880d0530 con 0x7fc1880d1870 2021-02-10T11:50:40.386 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.374 7fc18e94d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc1880f0650 0x7fc1880cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36684/0 (socket says 172.21.2.101:36684) 2021-02-10T11:50:40.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.374 7fc18e94d700 1 -- 172.21.2.101:0/1808686896 learned_addr learned my addr 172.21.2.101:0/1808686896 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:50:40.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.374 7fc18e14c700 1 --2- 172.21.2.101:0/1808686896 >> v2:172.21.2.108:3300/0 conn(0x7fc1880d1870 0x7fc1881b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:40.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.374 7fc18d14a700 1 -- 172.21.2.101:0/1808686896 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3483630815 0 0) 0x7fc180000c10 con 0x7fc1880f0650 2021-02-10T11:50:40.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.374 7fc18d14a700 1 -- 172.21.2.101:0/1808686896 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc170002fe0 con 0x7fc1880f0650 2021-02-10T11:50:40.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.374 7fc18d14a700 1 -- 172.21.2.101:0/1808686896 <== mon.1 v1:172.21.2.108:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1924822996 0 0) 0x7fc17c001400 con 0x7fc1881b0a10 2021-02-10T11:50:40.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.374 7fc18d14a700 1 -- 172.21.2.101:0/1808686896 --> v1:172.21.2.108:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc170004370 con 0x7fc1881b0a10 2021-02-10T11:50:40.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.375 7fc18e14c700 1 -- 172.21.2.101:0/1808686896 >> v1:172.21.2.108:6789/0 conn(0x7fc1881b0a10 legacy=0x7fc1880cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:40.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.375 7fc18e14c700 1 -- 172.21.2.101:0/1808686896 >> v1:172.21.2.101:6789/0 conn(0x7fc1880f0650 legacy=0x7fc1880cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:40.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.375 7fc18e14c700 1 -- 172.21.2.101:0/1808686896 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1880d1c70 con 0x7fc1880d1870 2021-02-10T11:50:40.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.375 7fc18e14c700 1 --2- 172.21.2.101:0/1808686896 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc1880d1870 0x7fc1881b03f0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7fc178006650 tx=0x7fc178009a80).ready entity=mon.1 client_cookie=2776e6274025280d server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:40.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.375 7fc18d14a700 1 -- 172.21.2.101:0/1808686896 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc17800c0c0 con 0x7fc1880d1870 2021-02-10T11:50:40.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.375 7fc18d14a700 1 -- 172.21.2.101:0/1808686896 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17800c340 con 0x7fc1880d1870 2021-02-10T11:50:40.387 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.375 7fc18d14a700 1 -- 172.21.2.101:0/1808686896 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc17800c860 con 0x7fc1880d1870 2021-02-10T11:50:40.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.375 7fc190330700 1 -- 172.21.2.101:0/1808686896 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc1880d1870 msgr2=0x7fc1881b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:40.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.375 7fc190330700 1 --2- 172.21.2.101:0/1808686896 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc1880d1870 0x7fc1881b03f0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7fc178006650 tx=0x7fc178009a80).stop 2021-02-10T11:50:40.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.376 7fc190330700 1 -- 172.21.2.101:0/1808686896 shutdown_connections 2021-02-10T11:50:40.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.376 7fc190330700 1 --2- 172.21.2.101:0/1808686896 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc1880d1870 0x7fc1881b03f0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:40.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.376 7fc190330700 1 -- 172.21.2.101:0/1808686896 >> 172.21.2.101:0/1808686896 conn(0x7fc1880ce450 msgr2=0x7fc1880d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:40.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.376 7fc190330700 1 -- 172.21.2.101:0/1808686896 shutdown_connections 2021-02-10T11:50:40.388 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.376 7fc190330700 1 -- 172.21.2.101:0/1808686896 wait complete. 2021-02-10T11:50:40.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.391 7fc190330700 1 Processor -- start 2021-02-10T11:50:40.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.391 7fc190330700 1 -- start start 2021-02-10T11:50:40.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.391 7fc190330700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1880970b0 0x7fc1880ebe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:40.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.391 7fc190330700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc188098810 0x7fc188098c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:40.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.391 7fc190330700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc188099230 0x7fc1880b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:40.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.391 7fc190330700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1880ec6d0 con 0x7fc1880970b0 2021-02-10T11:50:40.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.391 7fc190330700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1880974b0 con 0x7fc188099230 2021-02-10T11:50:40.389 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.391 7fc190330700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1880b1ee0 con 0x7fc188098810 2021-02-10T11:50:40.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.391 7fc18e14c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1880970b0 0x7fc1880ebe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:40.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.391 7fc18d94b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc188098810 0x7fc188098c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:40.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.391 7fc18e94d700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc188099230 0x7fc1880b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:40.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.391 7fc18e14c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1880970b0 0x7fc1880ebe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36192/0 (socket says 172.21.2.101:36192) 2021-02-10T11:50:40.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.391 7fc18d94b700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc188098810 0x7fc188098c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3301/0 says I am v2:172.21.2.101:57156/0 (socket says 172.21.2.101:57156) 2021-02-10T11:50:40.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.391 7fc18e14c700 1 -- 172.21.2.101:0/2675681600 learned_addr learned my addr 172.21.2.101:0/2675681600 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:40.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.391 7fc18e94d700 1 --2- 172.21.2.101:0/2675681600 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc188099230 0x7fc1880b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.108:3300/0 says I am v2:172.21.2.101:46678/0 (socket says 172.21.2.101:46678) 2021-02-10T11:50:40.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.392 7fc18e14c700 1 -- 172.21.2.101:0/2675681600 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc188098810 msgr2=0x7fc188098c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:40.391 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.392 7fc18e14c700 1 --2- 172.21.2.101:0/2675681600 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc188098810 0x7fc188098c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:40.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.392 7fc18e14c700 1 -- 172.21.2.101:0/2675681600 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc188099230 msgr2=0x7fc1880b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:40.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.392 7fc18e14c700 1 --2- 172.21.2.101:0/2675681600 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc188099230 0x7fc1880b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:40.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.392 7fc18e14c700 1 -- 172.21.2.101:0/2675681600 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1880b2100 con 0x7fc1880970b0 2021-02-10T11:50:40.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.392 7fc18e14c700 1 --2- 172.21.2.101:0/2675681600 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1880970b0 0x7fc1880ebe30 secure :-1 s=READY pgs=3442 cs=0 l=1 rev1=1 rx=0x7fc178009650 tx=0x7fc17800b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:40.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.392 7fc186ffd700 1 -- 172.21.2.101:0/2675681600 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc1780093a0 con 0x7fc1880970b0 2021-02-10T11:50:40.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.392 7fc186ffd700 1 -- 172.21.2.101:0/2675681600 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17800bd50 con 0x7fc1880970b0 2021-02-10T11:50:40.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.392 7fc186ffd700 1 -- 172.21.2.101:0/2675681600 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc1780052e0 con 0x7fc1880970b0 2021-02-10T11:50:40.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.392 7fc190330700 1 -- 172.21.2.101:0/2675681600 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1880b2100 con 0x7fc1880970b0 2021-02-10T11:50:40.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.392 7fc190330700 1 -- 172.21.2.101:0/2675681600 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1880b2100 con 0x7fc1880970b0 2021-02-10T11:50:40.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.393 7fc186ffd700 1 -- 172.21.2.101:0/2675681600 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc178006030 con 0x7fc1880970b0 2021-02-10T11:50:40.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.393 7fc186ffd700 1 --2- 172.21.2.101:0/2675681600 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1680186c0 0x7fc16801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:40.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.393 7fc186ffd700 1 -- 172.21.2.101:0/2675681600 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc178032550 con 0x7fc1880970b0 2021-02-10T11:50:40.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.393 7fc18d94b700 1 --2- 172.21.2.101:0/2675681600 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1680186c0 0x7fc16801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:40.393 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.393 7fc18d94b700 1 --2- 172.21.2.101:0/2675681600 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1680186c0 0x7fc16801abc0 crc :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:40.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.424 7fc190330700 1 -- 172.21.2.101:0/2675681600 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1880ebc00 con 0x7fc1880970b0 2021-02-10T11:50:40.422 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.426 7fc186ffd700 1 -- 172.21.2.101:0/2675681600 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fc178000940 con 0x7fc1880970b0 2021-02-10T11:50:40.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.559 7fc190330700 1 -- 172.21.2.101:0/2675681600 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_subvolume_clone_in_progress_getpath"]} v 0) v1 -- 0x7fc1880ebc00 con 0x7fc1880970b0 2021-02-10T11:50:40.945 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.949 7fc186ffd700 1 -- 172.21.2.101:0/2675681600 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_subvolume_clone_in_progress_getpath"]}]=0 v944) v1 ==== 157+0+0 (secure 0 0 0) 0x7fc178006030 con 0x7fc1880970b0 2021-02-10T11:50:40.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.977 7fc190330700 1 -- 172.21.2.101:0/2675681600 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1680186c0 msgr2=0x7fc16801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:40.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.977 7fc190330700 1 --2- 172.21.2.101:0/2675681600 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1680186c0 0x7fc16801abc0 crc :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:40.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.977 7fc190330700 1 -- 172.21.2.101:0/2675681600 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1880970b0 msgr2=0x7fc1880ebe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:40.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.977 7fc190330700 1 --2- 172.21.2.101:0/2675681600 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1880970b0 0x7fc1880ebe30 secure :-1 s=READY pgs=3442 cs=0 l=1 rev1=1 rx=0x7fc178009650 tx=0x7fc17800b960).stop 2021-02-10T11:50:40.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.978 7fc190330700 1 -- 172.21.2.101:0/2675681600 shutdown_connections 2021-02-10T11:50:40.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.978 7fc190330700 1 --2- 172.21.2.101:0/2675681600 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc1680186c0 0x7fc16801abc0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:40.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.978 7fc190330700 1 --2- 172.21.2.101:0/2675681600 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc188098810 0x7fc188098c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:40.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.978 7fc190330700 1 --2- 172.21.2.101:0/2675681600 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc188099230 0x7fc1880b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:40.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.978 7fc190330700 1 --2- 172.21.2.101:0/2675681600 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc1880970b0 0x7fc1880ebe30 unknown :-1 s=CLOSED pgs=3442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:40.974 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.978 7fc190330700 1 -- 172.21.2.101:0/2675681600 >> 172.21.2.101:0/2675681600 conn(0x7fc18819a730 msgr2=0x7fc18819e480 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:40.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.978 7fc190330700 1 -- 172.21.2.101:0/2675681600 shutdown_connections 2021-02-10T11:50:40.975 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:40.978 7fc190330700 1 -- 172.21.2.101:0/2675681600 wait complete. 2021-02-10T11:50:40.983 INFO:tasks.cephfs_test_runner:test_subvolume_clone_in_progress_getpath (tasks.cephfs.test_volumes.TestVolumes) ... ERROR 2021-02-10T11:50:40.983 INFO:tasks.cephfs_test_runner: 2021-02-10T11:50:40.984 INFO:tasks.cephfs_test_runner:====================================================================== 2021-02-10T11:50:40.984 INFO:tasks.cephfs_test_runner:ERROR: test_subvolume_clone_in_progress_getpath (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:50:40.984 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2021-02-10T11:50:40.984 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-02-10T11:50:40.984 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_962fac13b1a781b8c7d25c1eed50d93813b8f5f0/qa/tasks/cephfs/test_volumes.py", line 4052, in test_subvolume_clone_in_progress_getpath 2021-02-10T11:50:40.984 INFO:tasks.cephfs_test_runner: raise RuntimeError("expected fetching path of an pending clone to fail") 2021-02-10T11:50:40.984 INFO:tasks.cephfs_test_runner:RuntimeError: expected fetching path of an pending clone to fail 2021-02-10T11:50:40.984 INFO:tasks.cephfs_test_runner: 2021-02-10T11:50:40.984 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2021-02-10T11:50:40.984 INFO:tasks.cephfs_test_runner:Ran 21 tests in 2183.134s 2021-02-10T11:50:40.984 INFO:tasks.cephfs_test_runner: 2021-02-10T11:50:40.984 INFO:tasks.cephfs_test_runner:FAILED (errors=1, skipped=1) 2021-02-10T11:50:40.984 INFO:tasks.cephfs_test_runner: 2021-02-10T11:50:40.985 INFO:tasks.cephfs_test_runner:====================================================================== 2021-02-10T11:50:40.985 INFO:tasks.cephfs_test_runner:ERROR: test_subvolume_clone_in_progress_getpath (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:50:40.985 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2021-02-10T11:50:40.985 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-02-10T11:50:40.985 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_962fac13b1a781b8c7d25c1eed50d93813b8f5f0/qa/tasks/cephfs/test_volumes.py", line 4052, in test_subvolume_clone_in_progress_getpath 2021-02-10T11:50:40.985 INFO:tasks.cephfs_test_runner: raise RuntimeError("expected fetching path of an pending clone to fail") 2021-02-10T11:50:40.985 INFO:tasks.cephfs_test_runner:RuntimeError: expected fetching path of an pending clone to fail 2021-02-10T11:50:40.985 INFO:tasks.cephfs_test_runner: 2021-02-10T11:50:40.985 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/teuthology/run_tasks.py", line 94, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_962fac13b1a781b8c7d25c1eed50d93813b8f5f0/qa/tasks/cephfs_test_runner.py", line 207, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_subvolume_clone_in_progress_getpath (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:50:41.132 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=721a0ebf395f448d9b43c52b61c0d7a9 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/teuthology/run_tasks.py", line 94, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_962fac13b1a781b8c7d25c1eed50d93813b8f5f0/qa/tasks/cephfs_test_runner.py", line 207, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_subvolume_clone_in_progress_getpath (tasks.cephfs.test_volumes.TestVolumes) 2021-02-10T11:50:41.133 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2021-02-10T11:50:41.142 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2021-02-10T11:50:41.197 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2021-02-10T11:50:41.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:41.198 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:41.198 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-02-10T11:50:41.484 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:50:41.488 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:50:41.535 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2021-02-10T11:50:41.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:50:41.536 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-02-10T11:50:41.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:41.536 DEBUG:teuthology.orchestra.run.gibba001:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:41.536 DEBUG:teuthology.orchestra.run.gibba001:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-02-10T11:50:41.822 INFO:teuthology.orchestra.run.gibba001.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.1’: No such file or directory 2021-02-10T11:50:41.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:50:41.823 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-02-10T11:50:41.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:41.823 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:41.823 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-02-10T11:50:42.157 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.2’: No such file or directory 2021-02-10T11:50:42.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:50:42.157 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-02-10T11:50:42.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-02-10T11:50:42.158 DEBUG:teuthology.orchestra.run.gibba008:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-02-10T11:50:42.158 DEBUG:teuthology.orchestra.run.gibba008:> ' -- /home/ubuntu/cephtest/mnt.3) 2021-02-10T11:50:42.493 INFO:teuthology.orchestra.run.gibba008.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2021-02-10T11:50:42.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:50:42.494 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2021-02-10T11:50:42.494 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2021-02-10T11:50:42.514 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2021-02-10T11:50:42.514 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:50:42.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.933 7f9ae0c59700 1 Processor -- start 2021-02-10T11:50:42.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.933 7f9ae0c59700 1 -- start start 2021-02-10T11:50:42.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.934 7f9ae0c59700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9adc0d1870 0x7f9adc1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:42.947 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.934 7f9ae0c59700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9adc0f0650 0x7f9adc0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:42.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.934 7f9ae0c59700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9adc0f0b10 con 0x7f9adc1b0a10 2021-02-10T11:50:42.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.934 7f9ae0c59700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9adc0fe230 con 0x7f9adc0f0650 2021-02-10T11:50:42.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.934 7f9ae0c59700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc0e5ed0 con 0x7f9adc0d1870 2021-02-10T11:50:42.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.934 7f9ada05d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f9adc1b0a10 0x7f9adc0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36696/0 (socket says 172.21.2.101:36696) 2021-02-10T11:50:42.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.934 7f9ada05d700 1 -- 172.21.2.101:0/690595950 learned_addr learned my addr 172.21.2.101:0/690595950 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:50:42.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.934 7f9adb05f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9adc0f0650 0x7f9adc0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:42.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.934 7f9ada85e700 1 --2- 172.21.2.101:0/690595950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9adc0d1870 0x7f9adc1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:42.948 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.934 7f9adb05f700 1 -- 172.21.2.101:0/690595950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9adc0d1870 msgr2=0x7f9adc1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:42.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.934 7f9adb05f700 1 --2- 172.21.2.101:0/690595950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9adc0d1870 0x7f9adc1b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:42.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.934 7f9adb05f700 1 -- 172.21.2.101:0/690595950 >> v1:172.21.2.101:6789/0 conn(0x7f9adc1b0a10 legacy=0x7f9adc0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:42.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.934 7f9adb05f700 1 -- 172.21.2.101:0/690595950 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9adc0f20a0 con 0x7f9adc0f0650 2021-02-10T11:50:42.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.934 7f9ada85e700 1 --2- 172.21.2.101:0/690595950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9adc0d1870 0x7f9adc1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:50:42.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.934 7f9adb05f700 1 --2- 172.21.2.101:0/690595950 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9adc0f0650 0x7f9adc0cfb20 secure :-1 s=READY pgs=3444 cs=0 l=1 rev1=1 rx=0x7f9ad4006e30 tx=0x7f9ad4009e60).ready entity=mon.0 client_cookie=50357b568d944360 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:42.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.935 7f9ad985c700 1 -- 172.21.2.101:0/690595950 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9ad400b300 con 0x7f9adc0f0650 2021-02-10T11:50:42.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.935 7f9ad985c700 1 -- 172.21.2.101:0/690595950 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad4000d30 con 0x7f9adc0f0650 2021-02-10T11:50:42.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.935 7f9ad985c700 1 -- 172.21.2.101:0/690595950 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9ad400b830 con 0x7f9adc0f0650 2021-02-10T11:50:42.949 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.935 7f9ae0c59700 1 -- 172.21.2.101:0/690595950 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9adc0f0650 msgr2=0x7f9adc0cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:42.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.935 7f9ae0c59700 1 --2- 172.21.2.101:0/690595950 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9adc0f0650 0x7f9adc0cfb20 secure :-1 s=READY pgs=3444 cs=0 l=1 rev1=1 rx=0x7f9ad4006e30 tx=0x7f9ad4009e60).stop 2021-02-10T11:50:42.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.936 7f9ae0c59700 1 -- 172.21.2.101:0/690595950 shutdown_connections 2021-02-10T11:50:42.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.936 7f9ae0c59700 1 --2- 172.21.2.101:0/690595950 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9adc0f0650 0x7f9adc0cfb20 unknown :-1 s=CLOSED pgs=3444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:42.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.936 7f9ae0c59700 1 --2- 172.21.2.101:0/690595950 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9adc0d1870 0x7f9adc1b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:42.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.936 7f9ae0c59700 1 -- 172.21.2.101:0/690595950 >> 172.21.2.101:0/690595950 conn(0x7f9adc0ce450 msgr2=0x7f9adc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:42.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.936 7f9ae0c59700 1 -- 172.21.2.101:0/690595950 shutdown_connections 2021-02-10T11:50:42.950 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.936 7f9ae0c59700 1 -- 172.21.2.101:0/690595950 wait complete. 2021-02-10T11:50:42.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.952 7f9ae0c59700 1 Processor -- start 2021-02-10T11:50:42.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.952 7f9ae0c59700 1 -- start start 2021-02-10T11:50:42.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.952 7f9ae0c59700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9adc0b1ea0 0x7f9adc0b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:42.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.952 7f9ae0c59700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9adc0ebe30 0x7f9adc0ec230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:42.951 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.952 7f9ae0c59700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9adc0ec850 0x7f9adc0ecc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:42.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.952 7f9ae0c59700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc0ed300 con 0x7f9adc0ebe30 2021-02-10T11:50:42.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.952 7f9ae0c59700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc1afe10 con 0x7f9adc0ec850 2021-02-10T11:50:42.952 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.952 7f9ae0c59700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc1af800 con 0x7f9adc0b1ea0 2021-02-10T11:50:42.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.952 7f9ada05d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9adc0ebe30 0x7f9adc0ec230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:42.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.952 7f9ada05d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9adc0ebe30 0x7f9adc0ec230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36208/0 (socket says 172.21.2.101:36208) 2021-02-10T11:50:42.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.952 7f9ada05d700 1 -- 172.21.2.101:0/84866609 learned_addr learned my addr 172.21.2.101:0/84866609 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:42.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.952 7f9ada85e700 1 --2- 172.21.2.101:0/84866609 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9adc0b1ea0 0x7f9adc0b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:42.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.953 7f9adb05f700 1 --2- 172.21.2.101:0/84866609 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9adc0ec850 0x7f9adc0ecc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:42.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.953 7f9ada05d700 1 -- 172.21.2.101:0/84866609 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9adc0b1ea0 msgr2=0x7f9adc0b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:42.954 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.953 7f9ada05d700 1 --2- 172.21.2.101:0/84866609 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9adc0b1ea0 0x7f9adc0b22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:42.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.953 7f9ada05d700 1 -- 172.21.2.101:0/84866609 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9adc0ec850 msgr2=0x7f9adc0ecc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:42.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.953 7f9ada05d700 1 --2- 172.21.2.101:0/84866609 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9adc0ec850 0x7f9adc0ecc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:42.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.953 7f9ada05d700 1 -- 172.21.2.101:0/84866609 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9adc0ed880 con 0x7f9adc0ebe30 2021-02-10T11:50:42.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.953 7f9ada05d700 1 --2- 172.21.2.101:0/84866609 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9adc0ebe30 0x7f9adc0ec230 secure :-1 s=READY pgs=3445 cs=0 l=1 rev1=1 rx=0x7f9acc001c60 tx=0x7f9acc00a8d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:42.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.953 7f9ad37fe700 1 -- 172.21.2.101:0/84866609 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9acc00acc0 con 0x7f9adc0ebe30 2021-02-10T11:50:42.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.953 7f9ad37fe700 1 -- 172.21.2.101:0/84866609 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9acc004140 con 0x7f9adc0ebe30 2021-02-10T11:50:42.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.953 7f9ae0c59700 1 -- 172.21.2.101:0/84866609 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9adc0ed880 con 0x7f9adc0ebe30 2021-02-10T11:50:42.955 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.953 7f9ae0c59700 1 -- 172.21.2.101:0/84866609 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9adc0ed880 con 0x7f9adc0ebe30 2021-02-10T11:50:42.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.953 7f9ad37fe700 1 -- 172.21.2.101:0/84866609 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9acc004660 con 0x7f9adc0ebe30 2021-02-10T11:50:42.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.954 7f9ad37fe700 1 -- 172.21.2.101:0/84866609 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9acc000ce0 con 0x7f9adc0ebe30 2021-02-10T11:50:42.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.954 7f9ad37fe700 1 --2- 172.21.2.101:0/84866609 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ac0018730 0x7f9ac001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:42.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.954 7f9ad37fe700 1 -- 172.21.2.101:0/84866609 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9acc0327c0 con 0x7f9adc0ebe30 2021-02-10T11:50:42.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.954 7f9ada85e700 1 --2- 172.21.2.101:0/84866609 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ac0018730 0x7f9ac001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:42.956 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.954 7f9ada85e700 1 --2- 172.21.2.101:0/84866609 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ac0018730 0x7f9ac001ac10 crc :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:42.979 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.984 7f9ae0c59700 1 -- 172.21.2.101:0/84866609 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9adc0ebc00 con 0x7f9adc0ebe30 2021-02-10T11:50:42.982 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:42.986 7f9ad37fe700 1 -- 172.21.2.101:0/84866609 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f9acc0009b0 con 0x7f9adc0ebe30 2021-02-10T11:50:43.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.113 7f9ae0c59700 1 -- 172.21.2.101:0/84866609 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9adc0f0b10 con 0x7f9adc0ebe30 2021-02-10T11:50:43.109 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.114 7f9ad37fe700 1 -- 172.21.2.101:0/84866609 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f9acc000ce0 con 0x7f9adc0ebe30 2021-02-10T11:50:43.110 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:50:43.110 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":199,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:49:19.833311","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":43,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":42,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:49:16.791877","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":43,"pool_name":"cephfs_data","create_time":"2021-02-10 11:49:17.820338","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:50:43.111 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.116 7f9ae0c59700 1 -- 172.21.2.101:0/84866609 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ac0018730 msgr2=0x7f9ac001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:43.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.116 7f9ae0c59700 1 --2- 172.21.2.101:0/84866609 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ac0018730 0x7f9ac001ac10 crc :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:43.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.116 7f9ae0c59700 1 -- 172.21.2.101:0/84866609 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9adc0ebe30 msgr2=0x7f9adc0ec230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:43.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.116 7f9ae0c59700 1 --2- 172.21.2.101:0/84866609 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9adc0ebe30 0x7f9adc0ec230 secure :-1 s=READY pgs=3445 cs=0 l=1 rev1=1 rx=0x7f9acc001c60 tx=0x7f9acc00a8d0).stop 2021-02-10T11:50:43.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.116 7f9ae0c59700 1 -- 172.21.2.101:0/84866609 shutdown_connections 2021-02-10T11:50:43.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.116 7f9ae0c59700 1 --2- 172.21.2.101:0/84866609 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9adc0ec850 0x7f9adc0ecc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:43.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.116 7f9ae0c59700 1 --2- 172.21.2.101:0/84866609 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9adc0ebe30 0x7f9adc0ec230 unknown :-1 s=CLOSED pgs=3445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:43.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.116 7f9ae0c59700 1 --2- 172.21.2.101:0/84866609 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9ac0018730 0x7f9ac001ac10 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:43.112 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.116 7f9ae0c59700 1 --2- 172.21.2.101:0/84866609 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9adc0b1ea0 0x7f9adc0b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:43.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.116 7f9ae0c59700 1 -- 172.21.2.101:0/84866609 >> 172.21.2.101:0/84866609 conn(0x7f9adc19f120 msgr2=0x7f9adc19e480 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:43.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.116 7f9ae0c59700 1 -- 172.21.2.101:0/84866609 shutdown_connections 2021-02-10T11:50:43.113 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.116 7f9ae0c59700 1 -- 172.21.2.101:0/84866609 wait complete. 2021-02-10T11:50:43.121 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph pg dump --format=json 2021-02-10T11:50:43.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.537 7ff76463e700 1 Processor -- start 2021-02-10T11:50:43.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.537 7ff76463e700 1 -- start start 2021-02-10T11:50:43.550 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.538 7ff76463e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff75c0d1870 0x7ff75c1b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:43.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.538 7ff76463e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75c0f0b10 con 0x7ff75c1b0a10 2021-02-10T11:50:43.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.538 7ff76463e700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75c0d0310 con 0x7ff75c0f0650 2021-02-10T11:50:43.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.538 7ff76463e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75c0d0530 con 0x7ff75c0d1870 2021-02-10T11:50:43.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.538 7ff76245a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7ff75c0d1870 0x7ff75c1b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:43.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.538 7ff761c59700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7ff75c1b0a10 0x7ff75c0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36708/0 (socket says 172.21.2.101:36708) 2021-02-10T11:50:43.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.538 7ff761c59700 1 -- 172.21.2.101:0/2513987698 learned_addr learned my addr 172.21.2.101:0/2513987698 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:50:43.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.538 7ff761458700 1 -- 172.21.2.101:0/2513987698 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910563406 0 0) 0x7ff7500009b0 con 0x7ff75c1b0a10 2021-02-10T11:50:43.551 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.538 7ff761458700 1 -- 172.21.2.101:0/2513987698 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff748002fe0 con 0x7ff75c1b0a10 2021-02-10T11:50:43.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.539 7ff76245a700 1 -- 172.21.2.101:0/2513987698 >> v1:172.21.2.108:6789/0 conn(0x7ff75c0f0650 legacy=0x7ff75c0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:43.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.539 7ff76245a700 1 -- 172.21.2.101:0/2513987698 >> v1:172.21.2.101:6789/0 conn(0x7ff75c1b0a10 legacy=0x7ff75c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:43.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.539 7ff76245a700 1 -- 172.21.2.101:0/2513987698 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c0d1c70 con 0x7ff75c0d1870 2021-02-10T11:50:43.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.539 7ff76245a700 1 --2- 172.21.2.101:0/2513987698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff75c0d1870 0x7ff75c1b03f0 secure :-1 s=READY pgs=3447 cs=0 l=1 rev1=1 rx=0x7ff758006650 tx=0x7ff758009a80).ready entity=mon.0 client_cookie=e0c5bd51a90d26c1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:43.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.539 7ff761458700 1 -- 172.21.2.101:0/2513987698 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff75800c0c0 con 0x7ff75c0d1870 2021-02-10T11:50:43.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.539 7ff761458700 1 -- 172.21.2.101:0/2513987698 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75800c340 con 0x7ff75c0d1870 2021-02-10T11:50:43.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.539 7ff761458700 1 -- 172.21.2.101:0/2513987698 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff75800c860 con 0x7ff75c0d1870 2021-02-10T11:50:43.552 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.539 7ff76463e700 1 -- 172.21.2.101:0/2513987698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff75c0d1870 msgr2=0x7ff75c1b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:43.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.539 7ff76463e700 1 --2- 172.21.2.101:0/2513987698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff75c0d1870 0x7ff75c1b03f0 secure :-1 s=READY pgs=3447 cs=0 l=1 rev1=1 rx=0x7ff758006650 tx=0x7ff758009a80).stop 2021-02-10T11:50:43.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.540 7ff76463e700 1 -- 172.21.2.101:0/2513987698 shutdown_connections 2021-02-10T11:50:43.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.540 7ff76463e700 1 --2- 172.21.2.101:0/2513987698 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff75c0d1870 0x7ff75c1b03f0 unknown :-1 s=CLOSED pgs=3447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:43.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.540 7ff76463e700 1 -- 172.21.2.101:0/2513987698 >> 172.21.2.101:0/2513987698 conn(0x7ff75c0ce450 msgr2=0x7ff75c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:43.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.540 7ff76463e700 1 -- 172.21.2.101:0/2513987698 shutdown_connections 2021-02-10T11:50:43.553 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.540 7ff76463e700 1 -- 172.21.2.101:0/2513987698 wait complete. 2021-02-10T11:50:43.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.554 7ff76463e700 1 Processor -- start 2021-02-10T11:50:43.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.555 7ff76463e700 1 -- start start 2021-02-10T11:50:43.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.555 7ff76463e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff75c0b0ad0 0x7ff75c091f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:43.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.555 7ff76463e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff75c1a8850 0x7ff75c1a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:43.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.555 7ff76463e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff75c1a6fa0 0x7ff75c1a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:43.555 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.555 7ff76463e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c1abfe0 con 0x7ff75c1a6fa0 2021-02-10T11:50:43.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.555 7ff76463e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c1a9c40 con 0x7ff75c1a8850 2021-02-10T11:50:43.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.555 7ff76463e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c1a9730 con 0x7ff75c0b0ad0 2021-02-10T11:50:43.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.555 7ff762c5b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff75c1a6fa0 0x7ff75c1a73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:43.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.555 7ff762c5b700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff75c1a6fa0 0x7ff75c1a73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36218/0 (socket says 172.21.2.101:36218) 2021-02-10T11:50:43.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.555 7ff762c5b700 1 -- 172.21.2.101:0/493881180 learned_addr learned my addr 172.21.2.101:0/493881180 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:43.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.555 7ff762c5b700 1 -- 172.21.2.101:0/493881180 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff75c0b0ad0 msgr2=0x7ff75c091f30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:50:43.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.555 7ff762c5b700 1 --2- 172.21.2.101:0/493881180 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff75c0b0ad0 0x7ff75c091f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:43.556 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.555 7ff762c5b700 1 -- 172.21.2.101:0/493881180 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff75c1a8850 msgr2=0x7ff75c1a8c50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:50:43.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.555 7ff762c5b700 1 --2- 172.21.2.101:0/493881180 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff75c1a8850 0x7ff75c1a8c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:43.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.555 7ff762c5b700 1 -- 172.21.2.101:0/493881180 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c1a9270 con 0x7ff75c1a6fa0 2021-02-10T11:50:43.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.556 7ff762c5b700 1 --2- 172.21.2.101:0/493881180 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff75c1a6fa0 0x7ff75c1a73a0 secure :-1 s=READY pgs=3448 cs=0 l=1 rev1=1 rx=0x7ff754009940 tx=0x7ff75400bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:43.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.556 7ff74effd700 1 -- 172.21.2.101:0/493881180 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff754000ce0 con 0x7ff75c1a6fa0 2021-02-10T11:50:43.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.556 7ff74effd700 1 -- 172.21.2.101:0/493881180 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75400c2f0 con 0x7ff75c1a6fa0 2021-02-10T11:50:43.557 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.556 7ff76463e700 1 -- 172.21.2.101:0/493881180 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75c1a9270 con 0x7ff75c1a6fa0 2021-02-10T11:50:43.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.556 7ff74effd700 1 -- 172.21.2.101:0/493881180 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff75400c810 con 0x7ff75c1a6fa0 2021-02-10T11:50:43.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.556 7ff76463e700 1 -- 172.21.2.101:0/493881180 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75c1a7cb0 con 0x7ff75c1a6fa0 2021-02-10T11:50:43.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.557 7ff74effd700 1 -- 172.21.2.101:0/493881180 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7ff754001220 con 0x7ff75c1a6fa0 2021-02-10T11:50:43.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.557 7ff74effd700 1 --2- 172.21.2.101:0/493881180 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7440186d0 0x7ff74401abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:43.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.557 7ff74effd700 1 -- 172.21.2.101:0/493881180 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff754032b20 con 0x7ff75c1a6fa0 2021-02-10T11:50:43.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.557 7ff76245a700 1 --2- 172.21.2.101:0/493881180 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7440186d0 0x7ff74401abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:43.558 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.557 7ff76245a700 1 --2- 172.21.2.101:0/493881180 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7440186d0 0x7ff74401abd0 crc :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:43.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.588 7ff76463e700 1 -- 172.21.2.101:0/493881180 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff75c1a9710 con 0x7ff75c1a6fa0 2021-02-10T11:50:43.586 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.591 7ff74effd700 1 -- 172.21.2.101:0/493881180 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7ff754000cc0 con 0x7ff75c1a6fa0 2021-02-10T11:50:43.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.724 7ff76463e700 1 -- 172.21.2.101:0/493881180 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "pg dump", "target": ["mgr", ""], "format": "json"}) v1 -- 0x7ff75c1a7cb0 con 0x7ff7440186d0 2021-02-10T11:50:43.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.726 7ff74effd700 1 -- 172.21.2.101:0/493881180 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 dumped all) v1 ==== 18+0+101163 (crc 0 0 0) 0x7ff758006390 con 0x7ff7440186d0 2021-02-10T11:50:43.728 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:50:43.729 INFO:teuthology.orchestra.run.gibba001.stderr:dumped all 2021-02-10T11:50:43.744 INFO:teuthology.orchestra.run.gibba001.stdout:{"pg_ready":true,"pg_map":{"version":1332,"stamp":"2021-02-10 11:50:42.635928","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":134994218,"num_objects":154,"num_object_clones":1,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":65,"num_read_kb":65537,"num_write":421,"num_write_kb":131925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":405,"ondisk_log_size":405,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"kb":754974720,"kb_used":8672688,"kb_used_data":283816,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746302032,"statfs":{"total":773094113280,"available":764213280768,"internally_reserved":8589934592,"allocated":290627584,"data_stored":278830052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":72753708,"num_objects":72,"num_object_clones":1,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":65,"num_read_kb":65537,"num_write":196,"num_write_kb":71091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002375"},"pg_stats":[{"pgid":"43.6","version":"199'22","reported_seq":"40","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:34.454332","last_change":"2021-02-10 11:49:18.829257","last_active":"2021-02-10 11:50:34.454332","last_peered":"2021-02-10 11:50:34.454332","last_clean":"2021-02-10 11:50:34.454332","last_became_active":"2021-02-10 11:49:18.828991","last_became_peered":"2021-02-10 11:49:18.828991","last_unstale":"2021-02-10 11:50:34.454332","last_undegraded":"2021-02-10 11:50:34.454332","last_fullsized":"2021-02-10 11:50:34.454332","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:18.173888","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:18.173888","last_clean_scrub_stamp":"2021-02-10 11:49:18.173888","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23068672,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":9,"num_read_kb":9216,"num_write":22,"num_write_kb":22528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.7","version":"199'8","reported_seq":"18","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:49:20.506753","last_change":"2021-02-10 11:49:18.171840","last_active":"2021-02-10 11:49:20.506753","last_peered":"2021-02-10 11:49:20.506753","last_clean":"2021-02-10 11:49:20.506753","last_became_active":"2021-02-10 11:49:18.171627","last_became_peered":"2021-02-10 11:49:18.171627","last_unstale":"2021-02-10 11:49:20.506753","last_undegraded":"2021-02-10 11:49:20.506753","last_fullsized":"2021-02-10 11:49:20.506753","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:17.168696","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:17.168696","last_clean_scrub_stamp":"2021-02-10 11:49:17.168696","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":566,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"43.7","version":"199'14","reported_seq":"30","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:34.469585","last_change":"2021-02-10 11:49:18.837246","last_active":"2021-02-10 11:50:34.469585","last_peered":"2021-02-10 11:50:34.469585","last_clean":"2021-02-10 11:50:34.469585","last_became_active":"2021-02-10 11:49:18.836985","last_became_peered":"2021-02-10 11:49:18.836985","last_unstale":"2021-02-10 11:50:34.469585","last_undegraded":"2021-02-10 11:50:34.469585","last_fullsized":"2021-02-10 11:50:34.469585","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:18.173888","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:18.173888","last_clean_scrub_stamp":"2021-02-10 11:49:18.173888","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":14680064,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":7,"num_read_kb":7168,"num_write":14,"num_write_kb":14336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.6","version":"199'16","reported_seq":"26","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:35.839518","last_change":"2021-02-10 11:49:18.172149","last_active":"2021-02-10 11:50:35.839518","last_peered":"2021-02-10 11:50:35.839518","last_clean":"2021-02-10 11:50:35.839518","last_became_active":"2021-02-10 11:49:18.171957","last_became_peered":"2021-02-10 11:49:18.171957","last_unstale":"2021-02-10 11:50:35.839518","last_undegraded":"2021-02-10 11:50:35.839518","last_fullsized":"2021-02-10 11:50:35.839518","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:17.168696","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:17.168696","last_clean_scrub_stamp":"2021-02-10 11:49:17.168696","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"43.5","version":"199'23","reported_seq":"41","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:34.507597","last_change":"2021-02-10 11:49:18.840653","last_active":"2021-02-10 11:50:34.507597","last_peered":"2021-02-10 11:50:34.507597","last_clean":"2021-02-10 11:50:34.507597","last_became_active":"2021-02-10 11:49:18.840373","last_became_peered":"2021-02-10 11:49:18.840373","last_unstale":"2021-02-10 11:50:34.507597","last_undegraded":"2021-02-10 11:50:34.507597","last_fullsized":"2021-02-10 11:50:34.507597","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:18.173888","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:18.173888","last_clean_scrub_stamp":"2021-02-10 11:49:18.173888","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777506,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":10,"num_read_kb":10240,"num_write":20,"num_write_kb":16387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.4","version":"199'211","reported_seq":"221","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:35.944286","last_change":"2021-02-10 11:49:18.171244","last_active":"2021-02-10 11:50:35.944286","last_peered":"2021-02-10 11:50:35.944286","last_clean":"2021-02-10 11:50:35.944286","last_became_active":"2021-02-10 11:49:18.171030","last_became_peered":"2021-02-10 11:49:18.171030","last_unstale":"2021-02-10 11:50:35.944286","last_undegraded":"2021-02-10 11:50:35.944286","last_fullsized":"2021-02-10 11:50:35.944286","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:17.168696","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:17.168696","last_clean_scrub_stamp":"2021-02-10 11:49:17.168696","log_size":211,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":775202,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":211,"num_write_kb":818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"43.2","version":"199'20","reported_seq":"37","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:34.448896","last_change":"2021-02-10 11:49:18.841296","last_active":"2021-02-10 11:50:34.448896","last_peered":"2021-02-10 11:50:34.448896","last_clean":"2021-02-10 11:50:34.448896","last_became_active":"2021-02-10 11:49:18.841126","last_became_peered":"2021-02-10 11:49:18.841126","last_unstale":"2021-02-10 11:50:34.448896","last_undegraded":"2021-02-10 11:50:34.448896","last_fullsized":"2021-02-10 11:50:34.448896","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:18.173888","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:18.173888","last_clean_scrub_stamp":"2021-02-10 11:49:18.173888","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":8,"num_read_kb":8192,"num_write":20,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.3","version":"199'6","reported_seq":"16","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:49:20.506531","last_change":"2021-02-10 11:49:18.172382","last_active":"2021-02-10 11:49:20.506531","last_peered":"2021-02-10 11:49:20.506531","last_clean":"2021-02-10 11:49:20.506531","last_became_active":"2021-02-10 11:49:18.172169","last_became_peered":"2021-02-10 11:49:18.172169","last_unstale":"2021-02-10 11:49:20.506531","last_undegraded":"2021-02-10 11:49:20.506531","last_fullsized":"2021-02-10 11:49:20.506531","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:17.168696","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:17.168696","last_clean_scrub_stamp":"2021-02-10 11:49:17.168696","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"43.3","version":"199'25","reported_seq":"42","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:34.512593","last_change":"2021-02-10 11:49:18.837187","last_active":"2021-02-10 11:50:34.512593","last_peered":"2021-02-10 11:50:34.512593","last_clean":"2021-02-10 11:50:34.512593","last_became_active":"2021-02-10 11:49:18.836982","last_became_peered":"2021-02-10 11:49:18.836982","last_unstale":"2021-02-10 11:50:34.512593","last_undegraded":"2021-02-10 11:50:34.512593","last_fullsized":"2021-02-10 11:50:34.512593","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:18.173888","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:18.173888","last_clean_scrub_stamp":"2021-02-10 11:49:18.173888","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18874505,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":8192,"num_write":25,"num_write_kb":18436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.2","version":"198'1","reported_seq":"11","reported_epoch":"198","state":"active+clean","last_fresh":"2021-02-10 11:49:18.838787","last_change":"2021-02-10 11:49:18.181800","last_active":"2021-02-10 11:49:18.838787","last_peered":"2021-02-10 11:49:18.838787","last_clean":"2021-02-10 11:49:18.838787","last_became_active":"2021-02-10 11:49:18.181627","last_became_peered":"2021-02-10 11:49:18.181627","last_unstale":"2021-02-10 11:49:18.838787","last_undegraded":"2021-02-10 11:49:18.838787","last_fullsized":"2021-02-10 11:49:18.838787","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:17.168696","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:17.168696","last_clean_scrub_stamp":"2021-02-10 11:49:17.168696","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"43.0","version":"199'19","reported_seq":"38","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:34.414507","last_change":"2021-02-10 11:49:18.831187","last_active":"2021-02-10 11:50:34.414507","last_peered":"2021-02-10 11:50:34.414507","last_clean":"2021-02-10 11:50:34.414507","last_became_active":"2021-02-10 11:49:18.830931","last_became_peered":"2021-02-10 11:49:18.830931","last_unstale":"2021-02-10 11:50:34.414507","last_undegraded":"2021-02-10 11:50:34.414507","last_fullsized":"2021-02-10 11:50:34.414507","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:18.173888","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:18.173888","last_clean_scrub_stamp":"2021-02-10 11:49:18.173888","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18874368,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":10,"num_read_kb":10240,"num_write":18,"num_write_kb":18432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.1","version":"199'2","reported_seq":"13","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:35.851918","last_change":"2021-02-10 11:49:18.181797","last_active":"2021-02-10 11:50:35.851918","last_peered":"2021-02-10 11:50:35.851918","last_clean":"2021-02-10 11:50:35.851918","last_became_active":"2021-02-10 11:49:18.181580","last_became_peered":"2021-02-10 11:49:18.181580","last_unstale":"2021-02-10 11:50:35.851918","last_undegraded":"2021-02-10 11:50:35.851918","last_fullsized":"2021-02-10 11:50:35.851918","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:17.168696","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:17.168696","last_clean_scrub_stamp":"2021-02-10 11:49:17.168696","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":101,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"43.1","version":"199'11","reported_seq":"26","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:34.479243","last_change":"2021-02-10 11:49:18.829002","last_active":"2021-02-10 11:50:34.479243","last_peered":"2021-02-10 11:50:34.479243","last_clean":"2021-02-10 11:50:34.479243","last_became_active":"2021-02-10 11:49:18.828750","last_became_peered":"2021-02-10 11:49:18.828750","last_unstale":"2021-02-10 11:50:34.479243","last_undegraded":"2021-02-10 11:50:34.479243","last_fullsized":"2021-02-10 11:50:34.479243","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:18.173888","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:18.173888","last_clean_scrub_stamp":"2021-02-10 11:49:18.173888","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10485760,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":6,"num_read_kb":6144,"num_write":10,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.0","version":"198'8","reported_seq":"18","reported_epoch":"198","state":"active+clean","last_fresh":"2021-02-10 11:49:18.832529","last_change":"2021-02-10 11:49:18.171195","last_active":"2021-02-10 11:49:18.832529","last_peered":"2021-02-10 11:49:18.832529","last_clean":"2021-02-10 11:49:18.832529","last_became_active":"2021-02-10 11:49:18.170941","last_became_peered":"2021-02-10 11:49:18.170941","last_unstale":"2021-02-10 11:49:18.832529","last_undegraded":"2021-02-10 11:49:18.832529","last_fullsized":"2021-02-10 11:49:18.832529","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:17.168696","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:17.168696","last_clean_scrub_stamp":"2021-02-10 11:49:17.168696","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"178","reported_epoch":"198","state":"active+clean","last_fresh":"2021-02-10 11:49:18.827035","last_change":"2021-02-10 11:13:09.495109","last_active":"2021-02-10 11:49:18.827035","last_peered":"2021-02-10 11:49:18.827035","last_clean":"2021-02-10 11:49:18.827035","last_became_active":"2021-02-10 11:13:09.494906","last_became_peered":"2021-02-10 11:13:09.494906","last_unstale":"2021-02-10 11:49:18.827035","last_undegraded":"2021-02-10 11:49:18.827035","last_fullsized":"2021-02-10 11:49:18.827035","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"178","reported_epoch":"198","state":"active+clean","last_fresh":"2021-02-10 11:49:18.827010","last_change":"2021-02-10 11:13:09.495461","last_active":"2021-02-10 11:49:18.827010","last_peered":"2021-02-10 11:49:18.827010","last_clean":"2021-02-10 11:49:18.827010","last_became_active":"2021-02-10 11:13:09.495226","last_became_peered":"2021-02-10 11:13:09.495226","last_unstale":"2021-02-10 11:49:18.827010","last_undegraded":"2021-02-10 11:49:18.827010","last_fullsized":"2021-02-10 11:49:18.827010","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"189","reported_epoch":"198","state":"active+clean","last_fresh":"2021-02-10 11:49:18.826014","last_change":"2021-02-10 11:13:09.495287","last_active":"2021-02-10 11:49:18.826014","last_peered":"2021-02-10 11:49:18.826014","last_clean":"2021-02-10 11:49:18.826014","last_became_active":"2021-02-10 11:13:09.495041","last_became_peered":"2021-02-10 11:13:09.495041","last_unstale":"2021-02-10 11:49:18.826014","last_undegraded":"2021-02-10 11:49:18.826014","last_fullsized":"2021-02-10 11:49:18.826014","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"183","reported_epoch":"198","state":"active+clean","last_fresh":"2021-02-10 11:49:18.836129","last_change":"2021-02-10 11:13:09.487892","last_active":"2021-02-10 11:49:18.836129","last_peered":"2021-02-10 11:49:18.836129","last_clean":"2021-02-10 11:49:18.836129","last_became_active":"2021-02-10 11:13:09.487627","last_became_peered":"2021-02-10 11:13:09.487627","last_unstale":"2021-02-10 11:49:18.836129","last_undegraded":"2021-02-10 11:49:18.836129","last_fullsized":"2021-02-10 11:49:18.836129","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"177","reported_epoch":"198","state":"active+clean","last_fresh":"2021-02-10 11:49:18.828364","last_change":"2021-02-10 11:13:09.495589","last_active":"2021-02-10 11:49:18.828364","last_peered":"2021-02-10 11:49:18.828364","last_clean":"2021-02-10 11:49:18.828364","last_became_active":"2021-02-10 11:13:09.495399","last_became_peered":"2021-02-10 11:13:09.495399","last_unstale":"2021-02-10 11:49:18.828364","last_undegraded":"2021-02-10 11:49:18.828364","last_fullsized":"2021-02-10 11:49:18.828364","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"43.4","version":"199'10","reported_seq":"25","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:34.433388","last_change":"2021-02-10 11:49:18.837659","last_active":"2021-02-10 11:50:34.433388","last_peered":"2021-02-10 11:50:34.433388","last_clean":"2021-02-10 11:50:34.433388","last_became_active":"2021-02-10 11:49:18.837387","last_became_peered":"2021-02-10 11:49:18.837387","last_unstale":"2021-02-10 11:50:34.433388","last_undegraded":"2021-02-10 11:50:34.433388","last_fullsized":"2021-02-10 11:50:34.433388","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:18.173888","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:18.173888","last_clean_scrub_stamp":"2021-02-10 11:49:18.173888","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10485760,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":6,"num_read_kb":6144,"num_write":10,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.5","version":"199'9","reported_seq":"19","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:35.840310","last_change":"2021-02-10 11:49:18.171763","last_active":"2021-02-10 11:50:35.840310","last_peered":"2021-02-10 11:50:35.840310","last_clean":"2021-02-10 11:50:35.840310","last_became_active":"2021-02-10 11:49:18.171537","last_became_peered":"2021-02-10 11:49:18.171537","last_unstale":"2021-02-10 11:50:35.840310","last_undegraded":"2021-02-10 11:50:35.840310","last_fullsized":"2021-02-10 11:50:35.840310","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:17.168696","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:17.168696","last_clean_scrub_stamp":"2021-02-10 11:49:17.168696","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"192","reported_epoch":"198","state":"active+clean","last_fresh":"2021-02-10 11:49:18.826272","last_change":"2021-02-10 11:13:09.495570","last_active":"2021-02-10 11:49:18.826272","last_peered":"2021-02-10 11:49:18.826272","last_clean":"2021-02-10 11:49:18.826272","last_became_active":"2021-02-10 11:13:09.495383","last_became_peered":"2021-02-10 11:13:09.495383","last_unstale":"2021-02-10 11:49:18.826272","last_undegraded":"2021-02-10 11:49:18.826272","last_fullsized":"2021-02-10 11:49:18.826272","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"177","reported_epoch":"198","state":"active+clean","last_fresh":"2021-02-10 11:49:18.828366","last_change":"2021-02-10 11:13:09.495605","last_active":"2021-02-10 11:49:18.828366","last_peered":"2021-02-10 11:49:18.828366","last_clean":"2021-02-10 11:49:18.828366","last_became_active":"2021-02-10 11:13:09.495261","last_became_peered":"2021-02-10 11:13:09.495261","last_unstale":"2021-02-10 11:49:18.828366","last_undegraded":"2021-02-10 11:49:18.828366","last_fullsized":"2021-02-10 11:49:18.828366","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"175","reported_epoch":"198","state":"active+clean","last_fresh":"2021-02-10 11:49:18.836119","last_change":"2021-02-10 11:13:09.488300","last_active":"2021-02-10 11:49:18.836119","last_peered":"2021-02-10 11:49:18.836119","last_clean":"2021-02-10 11:49:18.836119","last_became_active":"2021-02-10 11:13:09.488122","last_became_peered":"2021-02-10 11:13:09.488122","last_unstale":"2021-02-10 11:49:18.836119","last_undegraded":"2021-02-10 11:49:18.836119","last_fullsized":"2021-02-10 11:49:18.836119","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":43,"num_pg":8,"stat_sum":{"num_bytes":134218155,"num_objects":131,"num_object_clones":1,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":0,"num_read":64,"num_read_kb":65536,"num_write":139,"num_write_kb":131079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":268460032,"data_stored":268436310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":16,"acting":16,"num_store_stats":8},{"poolid":42,"num_pg":8,"stat_sum":{"num_bytes":776063,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282,"num_write_kb":846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1622016,"data_stored":1552126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":261,"ondisk_log_size":261,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575303,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1087976,"kb_used_data":39392,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93283864,"statfs":{"total":96636764160,"available":95522676736,"internally_reserved":1073741824,"allocated":40337408,"data_stored":38854895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 10 11:50:13 2021","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.73099999999999998,"15min":0.76500000000000001},"min":{"1min":0.373,"5min":0.36899999999999999,"15min":0.36199999999999999},"max":{"1min":1.0720000000000001,"5min":1.2290000000000001,"15min":1.232},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77900000000000003,"15min":0.79500000000000004},"min":{"1min":0.442,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.1499999999999999,"5min":1.228,"15min":1.24},"last":0.91200000000000003}]},{"osd":1,"last update":"Wed Feb 10 11:50:13 2021","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.78900000000000003,"15min":0.81999999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.98099999999999998,"5min":1.179,"15min":1.2789999999999999},"last":1},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.79200000000000004,"15min":0.82499999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0389999999999999,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.153}]},{"osd":2,"last update":"Wed Feb 10 11:50:13 2021","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.81299999999999994,"15min":0.84999999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.99099999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.41799999999999998},"max":{"1min":1.01,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.028}]},{"osd":3,"last update":"Wed Feb 10 11:50:13 2021","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.90100000000000002,"15min":0.91600000000000004},"min":{"1min":0.49399999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1000000000000001,"5min":1.427,"15min":1.427},"last":1.248},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.94699999999999995,"15min":0.95799999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.442},"max":{"1min":1.101,"5min":1.333,"15min":1.3520000000000001},"last":1.2310000000000001}]},{"osd":4,"last update":"Wed Feb 10 11:50:13 2021","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70199999999999996,"15min":0.69699999999999995},"min":{"1min":0.47499999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90200000000000002,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.52000000000000002,"5min":0.46899999999999997,"15min":0.41899999999999998},"max":{"1min":1.008,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.57499999999999996}]},{"osd":5,"last update":"Wed Feb 10 11:50:13 2021","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79100000000000004,"15min":0.81200000000000006},"min":{"1min":0.44,"5min":0.434,"15min":0.42799999999999999},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.86799999999999999,"15min":0.879},"min":{"1min":0.54300000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.089,"5min":1.5,"15min":1.5},"last":0.874}]},{"osd":6,"last update":"Wed Feb 10 11:50:13 2021","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.81899999999999995,"15min":0.80300000000000005},"min":{"1min":0.52000000000000002,"5min":0.5,"15min":0.48499999999999999},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.84699999999999998,"15min":0.83999999999999997},"min":{"1min":0.53100000000000003,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":1.077,"5min":1.363,"15min":1.363},"last":0.75800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575303,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1070296,"kb_used_data":21712,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93301544,"statfs":{"total":96636764160,"available":95540781056,"internally_reserved":1073741824,"allocated":22233088,"data_stored":20754806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 10 11:50:20 2021","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.91100000000000003,"15min":0.92000000000000004},"min":{"1min":0.45600000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.403,"5min":1.589,"15min":1.589},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.93899999999999995,"15min":0.95699999999999996},"min":{"1min":0.58599999999999997,"5min":0.44400000000000001,"15min":0.40200000000000002},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.5229999999999999},"last":1.0720000000000001}]},{"osd":1,"last update":"Wed Feb 10 11:50:20 2021","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91100000000000003,"15min":0.90300000000000002},"min":{"1min":0.59799999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.226,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95899999999999996,"15min":0.94999999999999996},"min":{"1min":0.501,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2849999999999999,"5min":1.591,"15min":1.591},"last":1.042}]},{"osd":2,"last update":"Wed Feb 10 11:50:20 2021","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.82799999999999996,"15min":0.82599999999999996},"min":{"1min":0.55200000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.151,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.86099999999999999,"15min":0.84599999999999997},"min":{"1min":0.55300000000000005,"5min":0.44500000000000001,"15min":0.40200000000000002},"max":{"1min":1.333,"5min":1.333,"15min":1.4570000000000001},"last":0.90900000000000003}]},{"osd":3,"last update":"Wed Feb 10 11:50:20 2021","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.84199999999999997,"15min":0.85199999999999998},"min":{"1min":0.52500000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1819999999999999,"5min":1.2430000000000001,"15min":1.383},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.83999999999999997,"15min":0.83099999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.41799999999999998},"max":{"1min":1.167,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.80400000000000005}]},{"osd":4,"last update":"Wed Feb 10 11:50:20 2021","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69999999999999996,"15min":0.72399999999999998},"min":{"1min":0.498,"5min":0.40799999999999997,"15min":0.38900000000000001},"max":{"1min":0.98999999999999999,"5min":1.2669999999999999,"15min":1.2949999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.51200000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94299999999999995,"5min":1.145,"15min":1.145},"last":0.66400000000000003}]},{"osd":5,"last update":"Wed Feb 10 11:50:20 2021","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.47899999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92900000000000005,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.80700000000000005,"15min":0.79600000000000004},"min":{"1min":0.64900000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.0640000000000001,"5min":1.506,"15min":1.506},"last":0.76000000000000001}]},{"osd":7,"last update":"Wed Feb 10 11:50:20 2021","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.86399999999999999,"15min":0.871},"min":{"1min":0.47999999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.022,"5min":1.351,"15min":1.351},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.89800000000000002,"15min":0.89400000000000002},"min":{"1min":0.59799999999999998,"5min":0.47599999999999998,"15min":0.47299999999999998},"max":{"1min":1.127,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.95499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575303,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1083868,"kb_used_data":35284,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93287972,"statfs":{"total":96636764160,"available":95526883328,"internally_reserved":1073741824,"allocated":36130816,"data_stored":34659794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 10 11:50:19 2021","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.373},"max":{"1min":1.2390000000000001,"5min":1.393,"15min":1.393},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.81899999999999995,"15min":0.82799999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.1120000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":1.0780000000000001}]},{"osd":1,"last update":"Wed Feb 10 11:50:19 2021","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.81499999999999995,"15min":0.83799999999999997},"min":{"1min":0.38200000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.135,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.82899999999999996,"15min":0.84299999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.41699999999999998},"max":{"1min":1.22,"5min":1.222,"15min":1.2350000000000001},"last":0.97899999999999998}]},{"osd":2,"last update":"Wed Feb 10 11:50:19 2021","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.79500000000000004,"15min":0.80900000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.151,"5min":1.2789999999999999,"15min":1.331},"last":1.01},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.87,"15min":0.86699999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.349,"5min":1.349,"15min":1.3740000000000001},"last":0.95499999999999996}]},{"osd":3,"last update":"Wed Feb 10 11:50:19 2021","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.86499999999999999,"15min":0.875},"min":{"1min":0.64400000000000002,"5min":0.47999999999999998,"15min":0.41899999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.427},"last":1.177},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.83299999999999996,"15min":0.84899999999999998},"min":{"1min":0.44800000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.077,"5min":1.335,"15min":1.335},"last":0.92200000000000004}]},{"osd":4,"last update":"Wed Feb 10 11:50:19 2021","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.73199999999999998,"15min":0.71399999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.36799999999999999},"max":{"1min":0.95899999999999996,"5min":1.161,"15min":1.216},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.45400000000000001,"5min":0.42099999999999999,"15min":0.35499999999999998},"max":{"1min":1.0580000000000001,"5min":1.196,"15min":1.3560000000000001},"last":0.877}]},{"osd":6,"last update":"Wed Feb 10 11:50:19 2021","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.71999999999999997,"15min":0.71499999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.39100000000000001},"max":{"1min":0.86699999999999999,"5min":1.276,"15min":1.276},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.79900000000000004,"15min":0.79000000000000004},"min":{"1min":0.54800000000000004,"5min":0.47999999999999998,"15min":0.41799999999999998},"max":{"1min":1.075,"5min":1.2649999999999999,"15min":1.3999999999999999},"last":0.90700000000000003}]},{"osd":7,"last update":"Wed Feb 10 11:50:19 2021","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.82499999999999996,"15min":0.81100000000000005},"min":{"1min":0.53800000000000003,"5min":0.48599999999999999,"15min":0.46899999999999997},"max":{"1min":1.006,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.2290000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.79000000000000004,"15min":0.79300000000000004},"min":{"1min":0.55500000000000005,"5min":0.46700000000000003,"15min":0.45900000000000002},"max":{"1min":0.97799999999999998,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.1220000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575303,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1074388,"kb_used_data":25804,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93297452,"statfs":{"total":96636764160,"available":95536590848,"internally_reserved":1073741824,"allocated":26423296,"data_stored":24949076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 10 11:50:03 2021","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.85199999999999998,"15min":0.89900000000000002},"min":{"1min":0.38700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.3380000000000001,"5min":1.411,"15min":1.6850000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.93799999999999994,"15min":0.94399999999999995},"min":{"1min":0.503,"5min":0.42299999999999999,"15min":0.40100000000000002},"max":{"1min":1.2989999999999999,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.84799999999999998}]},{"osd":1,"last update":"Wed Feb 10 11:50:03 2021","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.83699999999999997,"15min":0.83199999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40000000000000002},"max":{"1min":1.244,"5min":1.2509999999999999,"15min":1.468},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.81499999999999995,"15min":0.82699999999999996},"min":{"1min":0.47599999999999998,"5min":0.371,"15min":0.36399999999999999},"max":{"1min":1.167,"5min":1.2629999999999999,"15min":1.49},"last":0.90400000000000003}]},{"osd":2,"last update":"Wed Feb 10 11:50:03 2021","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.80300000000000005,"15min":0.81799999999999995},"min":{"1min":0.48999999999999999,"5min":0.438,"15min":0.36599999999999999},"max":{"1min":1.284,"5min":1.2929999999999999,"15min":1.5660000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.82399999999999995,"15min":0.83199999999999996},"min":{"1min":0.501,"5min":0.42199999999999999,"15min":0.41999999999999998},"max":{"1min":1.1679999999999999,"5min":1.254,"15min":1.607},"last":0.747}]},{"osd":3,"last update":"Wed Feb 10 11:50:03 2021","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.81200000000000006,"15min":0.82399999999999995},"min":{"1min":0.436,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.194,"5min":1.2709999999999999,"15min":1.6319999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.84199999999999997,"15min":0.85999999999999999},"min":{"1min":0.52000000000000002,"5min":0.46000000000000002,"15min":0.45800000000000002},"max":{"1min":1.204,"5min":1.3029999999999999,"15min":1.482},"last":0.97099999999999997}]},{"osd":5,"last update":"Wed Feb 10 11:50:03 2021","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69699999999999995,"15min":0.71299999999999997},"min":{"1min":0.499,"5min":0.36099999999999999,"15min":0.35699999999999998},"max":{"1min":1.0589999999999999,"5min":1.1699999999999999,"15min":1.236},"last":0.625},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71099999999999997,"15min":0.70899999999999996},"min":{"1min":0.52200000000000002,"5min":0.45800000000000002,"15min":0.40100000000000002},"max":{"1min":1.111,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.51500000000000001}]},{"osd":6,"last update":"Wed Feb 10 11:50:03 2021","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.70599999999999996,"15min":0.73099999999999998},"min":{"1min":0.53600000000000003,"5min":0.45000000000000001,"15min":0.42299999999999999},"max":{"1min":0.92000000000000004,"5min":1.24,"15min":1.9590000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.81599999999999995,"15min":0.80100000000000005},"min":{"1min":0.59599999999999997,"5min":0.46899999999999997,"15min":0.45300000000000001},"max":{"1min":1.1439999999999999,"5min":1.262,"15min":3.351},"last":0.79800000000000004}]},{"osd":7,"last update":"Wed Feb 10 11:50:03 2021","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.78200000000000003,"15min":0.79200000000000004},"min":{"1min":0.51900000000000002,"5min":0.502,"15min":0.48299999999999998},"max":{"1min":1.306,"5min":1.306,"15min":1.504},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80400000000000005,"15min":0.82799999999999996},"min":{"1min":0.59799999999999998,"5min":0.52000000000000002,"15min":0.46899999999999997},"max":{"1min":1.103,"5min":1.181,"15min":1.3149999999999999},"last":0.72699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575302,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1061532,"kb_used_data":12752,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93310308,"statfs":{"total":96636764160,"available":95549755392,"internally_reserved":1073741824,"allocated":13058048,"data_stored":11590996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 10 11:50:24 2021","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56100000000000005,"15min":0.57999999999999996},"min":{"1min":0.29699999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.96799999999999997,"5min":0.96899999999999997,"15min":1.0129999999999999},"last":0.745},{"interface":"front","average":{"1min":0.629,"5min":0.58499999999999996,"15min":0.61099999999999999},"min":{"1min":0.39500000000000002,"5min":0.34100000000000003,"15min":0.32700000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":1.196},"last":0.59199999999999997}]},{"osd":2,"last update":"Wed Feb 10 11:50:24 2021","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.35099999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.92300000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.308},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.98699999999999999},"last":0.42799999999999999}]},{"osd":3,"last update":"Wed Feb 10 11:50:24 2021","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.505,"15min":0.51200000000000001},"min":{"1min":0.36799999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77500000000000002,"5min":0.878,"15min":0.998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.437,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84799999999999998,"5min":0.86599999999999999,"15min":1.036},"last":0.63500000000000001}]},{"osd":4,"last update":"Wed Feb 10 11:50:24 2021","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.95599999999999996,"15min":0.95099999999999996},"min":{"1min":0.55700000000000005,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.298,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.92200000000000004,"15min":0.93200000000000005},"min":{"1min":0.56499999999999995,"5min":0.49099999999999999,"15min":0.47399999999999998},"max":{"1min":1.21,"5min":1.4179999999999999,"15min":1.49},"last":0.79300000000000004}]},{"osd":5,"last update":"Wed Feb 10 11:50:24 2021","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.94299999999999995,"15min":0.94999999999999996},"min":{"1min":0.627,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1839999999999999,"5min":1.347,"15min":1.347},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.95099999999999996,"15min":0.97099999999999997},"min":{"1min":0.61599999999999999,"5min":0.55600000000000005,"15min":0.54300000000000004},"max":{"1min":1.248,"5min":1.502,"15min":1.6180000000000001},"last":0.86099999999999999}]},{"osd":6,"last update":"Wed Feb 10 11:50:24 2021","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.97299999999999998,"15min":0.97999999999999998},"min":{"1min":0.63700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.272,"5min":1.4550000000000001,"15min":1.5469999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.96099999999999997,"15min":0.97399999999999998},"min":{"1min":0.67700000000000005,"5min":0.57699999999999996,"15min":0.53900000000000003},"max":{"1min":1.2969999999999999,"5min":1.3400000000000001,"15min":1.444},"last":0.80300000000000005}]},{"osd":7,"last update":"Wed Feb 10 11:50:24 2021","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.98899999999999999,"15min":0.996},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.495},"max":{"1min":1.2549999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.97699999999999998,"15min":0.97999999999999998},"min":{"1min":0.64700000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.3560000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.87}]}]},{"osd":1,"up_from":13,"seq":55834575302,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1092068,"kb_used_data":43480,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93279772,"statfs":{"total":96636764160,"available":95518486528,"internally_reserved":1073741824,"allocated":44523520,"data_stored":43048633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 10 11:50:18 2021","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.52600000000000002,"15min":0.505},"min":{"1min":0.371,"5min":0.32600000000000001,"15min":0.30099999999999999},"max":{"1min":0.66100000000000003,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.497},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51300000000000001,"15min":0.50900000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.309},"max":{"1min":0.874,"5min":0.874,"15min":0.90400000000000003},"last":0.48499999999999999}]},{"osd":2,"last update":"Wed Feb 10 11:50:18 2021","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56999999999999995,"15min":0.55700000000000005},"min":{"1min":0.39700000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57199999999999995,"15min":0.56399999999999995},"min":{"1min":0.41299999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86099999999999999,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.39600000000000002}]},{"osd":3,"last update":"Wed Feb 10 11:50:18 2021","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61799999999999999,"15min":0.61199999999999999},"min":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.38100000000000001},"max":{"1min":0.99299999999999999,"5min":1.056,"15min":1.056},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65000000000000002,"15min":0.63300000000000001},"min":{"1min":0.45500000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.075,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.54600000000000004}]},{"osd":4,"last update":"Wed Feb 10 11:50:18 2021","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.0369999999999999,"15min":1.0009999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.61299999999999999},"max":{"1min":1.284,"5min":1.609,"15min":1.629},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0369999999999999,"15min":1.0049999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.42299999999999999},"max":{"1min":1.2210000000000001,"5min":1.4790000000000001,"15min":1.6299999999999999},"last":0.97499999999999998}]},{"osd":5,"last update":"Wed Feb 10 11:50:18 2021","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1359999999999999,"15min":1.115},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3240000000000001,"5min":1.518,"15min":1.665},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.1240000000000001,"15min":1.107},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.431,"5min":1.694,"15min":1.74},"last":1.1000000000000001}]},{"osd":6,"last update":"Wed Feb 10 11:50:18 2021","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.03,"15min":1.0069999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.53000000000000003},"max":{"1min":1.2529999999999999,"5min":1.46,"15min":1.5700000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.0489999999999999,"15min":1.0269999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.268,"5min":1.613,"15min":1.613},"last":0.93200000000000005}]},{"osd":7,"last update":"Wed Feb 10 11:50:18 2021","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.0409999999999999,"15min":1.0309999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.59299999999999997},"max":{"1min":1.331,"5min":1.456,"15min":1.482},"last":1.048},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.056,"15min":1.0329999999999999},"min":{"1min":0.73499999999999999,"5min":0.68200000000000005,"15min":0.55300000000000005},"max":{"1min":1.1830000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.0840000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575303,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1122792,"kb_used_data":74208,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93249048,"statfs":{"total":96636764160,"available":95487025152,"internally_reserved":1073741824,"allocated":75988992,"data_stored":74506385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 10 11:49:56 2021","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51200000000000001,"15min":0.497},"min":{"1min":0.33300000000000002,"5min":0.29999999999999999,"15min":0.27100000000000002},"max":{"1min":0.91100000000000003,"5min":1.069,"15min":1.069},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.50700000000000001,"15min":0.503},"min":{"1min":0.38800000000000001,"5min":0.32700000000000001,"15min":0.29999999999999999},"max":{"1min":0.73699999999999999,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.42599999999999999}]},{"osd":1,"last update":"Wed Feb 10 11:49:56 2021","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":1.0209999999999999,"5min":1.131,"15min":1.131},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.443,"5min":0.33000000000000002,"15min":0.32700000000000001},"max":{"1min":0.70699999999999996,"5min":0.89600000000000002,"15min":1.016},"last":0.47899999999999998}]},{"osd":3,"last update":"Wed Feb 10 11:49:56 2021","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.32000000000000001},"max":{"1min":0.94499999999999995,"5min":1.097,"15min":1.097},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.38100000000000001,"5min":0.371,"15min":0.34499999999999997},"max":{"1min":0.70499999999999996,"5min":1.252,"15min":1.252},"last":0.60499999999999998}]},{"osd":4,"last update":"Wed Feb 10 11:49:56 2021","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.98099999999999998,"15min":0.97099999999999997},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.495},"max":{"1min":1.272,"5min":1.393,"15min":1.4470000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.99199999999999999,"15min":0.98799999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3340000000000001,"5min":1.343,"15min":1.417},"last":0.93899999999999995}]},{"osd":5,"last update":"Wed Feb 10 11:49:56 2021","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.113,"15min":1.095},"min":{"1min":0.66700000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.4139999999999999,"5min":1.4550000000000001,"15min":1.5249999999999999},"last":1.056},{"interface":"front","average":{"1min":1.173,"5min":1.137,"15min":1.123},"min":{"1min":0.77000000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.476,"5min":1.476,"15min":1.5149999999999999},"last":1.0960000000000001}]},{"osd":6,"last update":"Wed Feb 10 11:49:56 2021","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1,"15min":0.99299999999999999},"min":{"1min":0.64600000000000002,"5min":0.61199999999999999,"15min":0.504},"max":{"1min":1.3340000000000001,"5min":1.3500000000000001,"15min":1.381},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.028,"5min":1.0049999999999999,"15min":1.0089999999999999},"min":{"1min":0.76300000000000001,"5min":0.60399999999999998,"15min":0.53700000000000003},"max":{"1min":1.2989999999999999,"5min":1.4470000000000001,"15min":1.472},"last":0.91800000000000004}]},{"osd":7,"last update":"Wed Feb 10 11:49:56 2021","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.038,"15min":1.02},"min":{"1min":0.752,"5min":0.60699999999999998,"15min":0.55500000000000005},"max":{"1min":1.3580000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.875},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0189999999999999,"15min":0.996},"min":{"1min":0.68600000000000005,"5min":0.64800000000000002,"15min":0.58899999999999997},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.081}]}]},{"osd":3,"up_from":13,"seq":55834575303,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1079768,"kb_used_data":31184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93292072,"statfs":{"total":96636764160,"available":95531081728,"internally_reserved":1073741824,"allocated":31932416,"data_stored":30465467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 10 11:50:26 2021","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46600000000000003,"15min":0.46700000000000003},"min":{"1min":0.34599999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71399999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.499,"15min":0.495},"min":{"1min":0.38,"5min":0.317,"15min":0.317},"max":{"1min":0.628,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.73399999999999999}]},{"osd":1,"last update":"Wed Feb 10 11:50:26 2021","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47999999999999998,"15min":0.48599999999999999},"min":{"1min":0.374,"5min":0.316,"15min":0.316},"max":{"1min":0.57099999999999995,"5min":0.77000000000000002,"15min":0.89700000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.439,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.95899999999999996,"5min":1.0289999999999999,"15min":1.0409999999999999},"last":0.91700000000000004}]},{"osd":2,"last update":"Wed Feb 10 11:50:26 2021","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.46800000000000003,"5min":0.35399999999999998,"15min":0.34100000000000003},"max":{"1min":0.92900000000000005,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68000000000000005,"15min":0.68500000000000005},"min":{"1min":0.50700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89600000000000002,"5min":1.397,"15min":1.397},"last":0.97099999999999997}]},{"osd":4,"last update":"Wed Feb 10 11:50:26 2021","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.95199999999999996,"15min":0.95699999999999996},"min":{"1min":0.60799999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.2350000000000001,"5min":1.472,"15min":1.472},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.93799999999999994,"15min":0.94099999999999995},"min":{"1min":0.67100000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2470000000000001,"5min":1.351,"15min":1.575},"last":1.0369999999999999}]},{"osd":5,"last update":"Wed Feb 10 11:50:26 2021","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.0549999999999999,"15min":1.071},"min":{"1min":0.59599999999999997,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.431,"5min":1.621,"15min":1.621},"last":1.101},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0660000000000001,"15min":1.0600000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.48299999999999998},"max":{"1min":1.3380000000000001,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.2909999999999999}]},{"osd":6,"last update":"Wed Feb 10 11:50:26 2021","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.98399999999999999,"15min":0.95899999999999996},"min":{"1min":0.76600000000000001,"5min":0.63300000000000001,"15min":0.54500000000000004},"max":{"1min":1.222,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.024,"5min":1.01,"15min":0.98999999999999999},"min":{"1min":0.68200000000000005,"5min":0.61199999999999999,"15min":0.57999999999999996},"max":{"1min":1.288,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.196}]},{"osd":7,"last update":"Wed Feb 10 11:50:26 2021","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1080000000000001,"15min":1.1020000000000001},"min":{"1min":0.89000000000000001,"5min":0.67200000000000004,"15min":0.63200000000000001},"max":{"1min":1.413,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.083,"15min":1.0880000000000001},"min":{"1min":0.88100000000000001,"5min":0.67700000000000005,"15min":0.67300000000000004},"max":{"1min":1.405,"5min":1.508,"15min":1.5900000000000001},"last":1.3009999999999999}]}]}]}} 2021-02-10T11:50:43.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.728 7ff76463e700 1 -- 172.21.2.101:0/493881180 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7440186d0 msgr2=0x7ff74401abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:43.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.728 7ff76463e700 1 --2- 172.21.2.101:0/493881180 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7440186d0 0x7ff74401abd0 crc :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:43.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.728 7ff76463e700 1 -- 172.21.2.101:0/493881180 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff75c1a6fa0 msgr2=0x7ff75c1a73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:43.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.728 7ff76463e700 1 --2- 172.21.2.101:0/493881180 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff75c1a6fa0 0x7ff75c1a73a0 secure :-1 s=READY pgs=3448 cs=0 l=1 rev1=1 rx=0x7ff754009940 tx=0x7ff75400bcf0).stop 2021-02-10T11:50:43.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.729 7ff76463e700 1 -- 172.21.2.101:0/493881180 shutdown_connections 2021-02-10T11:50:43.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.729 7ff76463e700 1 --2- 172.21.2.101:0/493881180 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7ff75c1a6fa0 0x7ff75c1a73a0 unknown :-1 s=CLOSED pgs=3448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:43.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.729 7ff76463e700 1 --2- 172.21.2.101:0/493881180 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7ff75c1a8850 0x7ff75c1a8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:43.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.729 7ff76463e700 1 --2- 172.21.2.101:0/493881180 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7ff7440186d0 0x7ff74401abd0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:43.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.729 7ff76463e700 1 --2- 172.21.2.101:0/493881180 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7ff75c0b0ad0 0x7ff75c091f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:43.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.729 7ff76463e700 1 -- 172.21.2.101:0/493881180 >> 172.21.2.101:0/493881180 conn(0x7ff75c19f8f0 msgr2=0x7ff75c1a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:43.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.729 7ff76463e700 1 -- 172.21.2.101:0/493881180 shutdown_connections 2021-02-10T11:50:43.746 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:43.729 7ff76463e700 1 -- 172.21.2.101:0/493881180 wait complete. 2021-02-10T11:50:43.748 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd dump --format=json 2021-02-10T11:50:44.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.165 7f6561cb2700 1 Processor -- start 2021-02-10T11:50:44.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.166 7f6561cb2700 1 -- start start 2021-02-10T11:50:44.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.166 7f6561cb2700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f655c1b0470 0x7f655c1b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:44.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.166 7f6561cb2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f655c0ef610 0x7f655c0efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:44.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.166 7f6561cb2700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f655c0f0030 0x7f655c0f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:44.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.166 7f6561cb2700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f655c0e5ea0 con 0x7f655c1b0470 2021-02-10T11:50:44.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.166 7f6561cb2700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f655c0f0b50 con 0x7f655c0f0030 2021-02-10T11:50:44.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.166 7f6561cb2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f655c0e52e0 con 0x7f655c0ef610 2021-02-10T11:50:44.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.166 7f655b7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f655c1b0470 0x7f655c1b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:44.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.166 7f655b7fe700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f655c1b0470 0x7f655c1b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36226/0 (socket says 172.21.2.101:36226) 2021-02-10T11:50:44.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.166 7f655b7fe700 1 -- 172.21.2.101:0/3474892659 learned_addr learned my addr 172.21.2.101:0/3474892659 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:44.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.166 7f655b7fe700 1 -- 172.21.2.101:0/3474892659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f655c0ef610 msgr2=0x7f655c0efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:44.179 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.166 7f655b7fe700 1 --2- 172.21.2.101:0/3474892659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f655c0ef610 0x7f655c0efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:44.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.167 7f655b7fe700 1 -- 172.21.2.101:0/3474892659 >> v2:172.21.2.108:3300/0 conn(0x7f655c0f0030 msgr2=0x7f655c0f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:44.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.167 7f655b7fe700 1 --2- 172.21.2.101:0/3474892659 >> v2:172.21.2.108:3300/0 conn(0x7f655c0f0030 0x7f655c0f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:44.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.167 7f655b7fe700 1 -- 172.21.2.101:0/3474892659 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655c0f10d0 con 0x7f655c1b0470 2021-02-10T11:50:44.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.167 7f655b7fe700 1 --2- 172.21.2.101:0/3474892659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f655c1b0470 0x7f655c1b0870 secure :-1 s=READY pgs=3449 cs=0 l=1 rev1=1 rx=0x7f6544006700 tx=0x7f6544009b80).ready entity=mon.0 client_cookie=45b2c5e120f0a421 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:44.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.167 7f655a7fc700 1 -- 172.21.2.101:0/3474892659 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f654400c0c0 con 0x7f655c1b0470 2021-02-10T11:50:44.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.167 7f655a7fc700 1 -- 172.21.2.101:0/3474892659 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654400c340 con 0x7f655c1b0470 2021-02-10T11:50:44.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.167 7f655a7fc700 1 -- 172.21.2.101:0/3474892659 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f654400c860 con 0x7f655c1b0470 2021-02-10T11:50:44.180 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.167 7f6561cb2700 1 -- 172.21.2.101:0/3474892659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f655c1b0470 msgr2=0x7f655c1b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:44.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.167 7f6561cb2700 1 --2- 172.21.2.101:0/3474892659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f655c1b0470 0x7f655c1b0870 secure :-1 s=READY pgs=3449 cs=0 l=1 rev1=1 rx=0x7f6544006700 tx=0x7f6544009b80).stop 2021-02-10T11:50:44.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.168 7f6561cb2700 1 -- 172.21.2.101:0/3474892659 shutdown_connections 2021-02-10T11:50:44.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.168 7f6561cb2700 1 --2- 172.21.2.101:0/3474892659 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f655c0ef610 0x7f655c0efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:44.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.168 7f6561cb2700 1 --2- 172.21.2.101:0/3474892659 >> v2:172.21.2.108:3300/0 conn(0x7f655c0f0030 0x7f655c0f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:44.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.168 7f6561cb2700 1 --2- 172.21.2.101:0/3474892659 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f655c1b0470 0x7f655c1b0870 unknown :-1 s=CLOSED pgs=3449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:44.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.168 7f6561cb2700 1 -- 172.21.2.101:0/3474892659 >> 172.21.2.101:0/3474892659 conn(0x7f655c0ce3e0 msgr2=0x7f655c0e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:44.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.168 7f6561cb2700 1 -- 172.21.2.101:0/3474892659 shutdown_connections 2021-02-10T11:50:44.181 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.168 7f6561cb2700 1 -- 172.21.2.101:0/3474892659 wait complete. 2021-02-10T11:50:44.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.182 7f6561cb2700 1 Processor -- start 2021-02-10T11:50:44.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.182 7f6561cb2700 1 -- start start 2021-02-10T11:50:44.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.183 7f6561cb2700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f655c1095b0 0x7f655c19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:44.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.183 7f6561cb2700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f655c1099b0 0x7f655c0933c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:44.183 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.183 7f6561cb2700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f655c0939e0 0x7f655c1ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:44.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.183 7f6561cb2700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655c093de0 con 0x7f655c0939e0 2021-02-10T11:50:44.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.183 7f6561cb2700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655c0aed40 con 0x7f655c1099b0 2021-02-10T11:50:44.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.183 7f6561cb2700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f655c1ad420 con 0x7f655c1095b0 2021-02-10T11:50:44.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.183 7f655bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f655c0939e0 0x7f655c1ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:44.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.183 7f655bfff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f655c0939e0 0x7f655c1ace00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36234/0 (socket says 172.21.2.101:36234) 2021-02-10T11:50:44.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.183 7f655bfff700 1 -- 172.21.2.101:0/326435031 learned_addr learned my addr 172.21.2.101:0/326435031 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:44.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.183 7f655bfff700 1 -- 172.21.2.101:0/326435031 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f655c1095b0 msgr2=0x7f655c19e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:44.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.183 7f655bfff700 1 --2- 172.21.2.101:0/326435031 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f655c1095b0 0x7f655c19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:44.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.183 7f655bfff700 1 -- 172.21.2.101:0/326435031 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f655c1099b0 msgr2=0x7f655c0933c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:44.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.183 7f655bfff700 1 --2- 172.21.2.101:0/326435031 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f655c1099b0 0x7f655c0933c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:44.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.183 7f655bfff700 1 -- 172.21.2.101:0/326435031 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655c1ad640 con 0x7f655c0939e0 2021-02-10T11:50:44.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.183 7f655bfff700 1 --2- 172.21.2.101:0/326435031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f655c0939e0 0x7f655c1ace00 secure :-1 s=READY pgs=3450 cs=0 l=1 rev1=1 rx=0x7f6554009980 tx=0x7f655400bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:44.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.184 7f6553fff700 1 -- 172.21.2.101:0/326435031 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6554005af0 con 0x7f655c0939e0 2021-02-10T11:50:44.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.184 7f6553fff700 1 -- 172.21.2.101:0/326435031 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6554005df0 con 0x7f655c0939e0 2021-02-10T11:50:44.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.184 7f6561cb2700 1 -- 172.21.2.101:0/326435031 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f655c1ad640 con 0x7f655c0939e0 2021-02-10T11:50:44.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.184 7f6561cb2700 1 -- 172.21.2.101:0/326435031 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f655c1adbd0 con 0x7f655c0939e0 2021-02-10T11:50:44.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.184 7f6553fff700 1 -- 172.21.2.101:0/326435031 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f65540062b0 con 0x7f655c0939e0 2021-02-10T11:50:44.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.184 7f6553fff700 1 -- 172.21.2.101:0/326435031 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6554005af0 con 0x7f655c0939e0 2021-02-10T11:50:44.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.184 7f6553fff700 1 --2- 172.21.2.101:0/326435031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65400186c0 0x7f654001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:44.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.185 7f6553fff700 1 -- 172.21.2.101:0/326435031 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6554032910 con 0x7f655c0939e0 2021-02-10T11:50:44.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.185 7f655b7fe700 1 --2- 172.21.2.101:0/326435031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65400186c0 0x7f654001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:44.186 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.185 7f655b7fe700 1 --2- 172.21.2.101:0/326435031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65400186c0 0x7f654001abc0 crc :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:44.211 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.216 7f6561cb2700 1 -- 172.21.2.101:0/326435031 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f655c0f0b50 con 0x7f655c0939e0 2021-02-10T11:50:44.214 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.218 7f6553fff700 1 -- 172.21.2.101:0/326435031 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6554006570 con 0x7f655c0939e0 2021-02-10T11:50:44.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.343 7f6561cb2700 1 -- 172.21.2.101:0/326435031 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f655c0f0b50 con 0x7f655c0939e0 2021-02-10T11:50:44.339 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.344 7f6553fff700 1 -- 172.21.2.101:0/326435031 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) v1 ==== 74+0+14007 (secure 0 0 0) 0x7f6554005af0 con 0x7f655c0939e0 2021-02-10T11:50:44.340 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:50:44.340 INFO:teuthology.orchestra.run.gibba001.stdout:{"epoch":199,"fsid":"a6663ba9-2653-4542-8b6a-9323b4e7b545","created":"2021-02-10 11:12:36.788366","modified":"2021-02-10 11:49:19.833311","last_up_change":"2021-02-10 11:13:03.215004","last_in_change":"2021-02-10 11:13:03.215004","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":43,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2021-02-10 11:13:08.037472","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":42,"pool_name":"cephfs_metadata","create_time":"2021-02-10 11:49:16.791877","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":43,"pool_name":"cephfs_data","create_time":"2021-02-10 11:49:17.820338","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d9fa1c93-e44a-4c6c-b7c7-9d63606e73b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6802","nonce":25093},{"type":"v1","addr":"172.21.2.101:6805","nonce":25093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6807","nonce":25093},{"type":"v1","addr":"172.21.2.101:6809","nonce":25093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6817","nonce":25093},{"type":"v1","addr":"172.21.2.101:6820","nonce":25093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6812","nonce":25093},{"type":"v1","addr":"172.21.2.101:6814","nonce":25093}]},"public_addr":"172.21.2.101:6805/25093","cluster_addr":"172.21.2.101:6809/25093","heartbeat_back_addr":"172.21.2.101:6820/25093","heartbeat_front_addr":"172.21.2.101:6814/25093","state":["exists","up"]},{"osd":1,"uuid":"ddc04ce3-b313-4707-8270-585391e38ecc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6803","nonce":25092},{"type":"v1","addr":"172.21.2.101:6804","nonce":25092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6806","nonce":25092},{"type":"v1","addr":"172.21.2.101:6808","nonce":25092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6818","nonce":25092},{"type":"v1","addr":"172.21.2.101:6821","nonce":25092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6810","nonce":25092},{"type":"v1","addr":"172.21.2.101:6815","nonce":25092}]},"public_addr":"172.21.2.101:6804/25092","cluster_addr":"172.21.2.101:6808/25092","heartbeat_back_addr":"172.21.2.101:6821/25092","heartbeat_front_addr":"172.21.2.101:6815/25092","state":["exists","up"]},{"osd":2,"uuid":"e5c54f85-6e1f-4df0-85e4-65e9a8e3dab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6811","nonce":25094},{"type":"v1","addr":"172.21.2.101:6813","nonce":25094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6816","nonce":25094},{"type":"v1","addr":"172.21.2.101:6819","nonce":25094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6825","nonce":25094},{"type":"v1","addr":"172.21.2.101:6827","nonce":25094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6822","nonce":25094},{"type":"v1","addr":"172.21.2.101:6823","nonce":25094}]},"public_addr":"172.21.2.101:6813/25094","cluster_addr":"172.21.2.101:6819/25094","heartbeat_back_addr":"172.21.2.101:6827/25094","heartbeat_front_addr":"172.21.2.101:6823/25094","state":["exists","up"]},{"osd":3,"uuid":"0d9ef9b1-98aa-4d37-aa16-6305c7d46d52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6824","nonce":25095},{"type":"v1","addr":"172.21.2.101:6826","nonce":25095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6828","nonce":25095},{"type":"v1","addr":"172.21.2.101:6829","nonce":25095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6832","nonce":25095},{"type":"v1","addr":"172.21.2.101:6833","nonce":25095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.101:6830","nonce":25095},{"type":"v1","addr":"172.21.2.101:6831","nonce":25095}]},"public_addr":"172.21.2.101:6826/25095","cluster_addr":"172.21.2.101:6829/25095","heartbeat_back_addr":"172.21.2.101:6833/25095","heartbeat_front_addr":"172.21.2.101:6831/25095","state":["exists","up"]},{"osd":4,"uuid":"36742c21-bfeb-4e11-bce9-8e745378b605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6816","nonce":36380},{"type":"v1","addr":"172.21.2.108:6818","nonce":36380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6820","nonce":36380},{"type":"v1","addr":"172.21.2.108:6822","nonce":36380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6828","nonce":36380},{"type":"v1","addr":"172.21.2.108:6830","nonce":36380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6824","nonce":36380},{"type":"v1","addr":"172.21.2.108:6826","nonce":36380}]},"public_addr":"172.21.2.108:6818/36380","cluster_addr":"172.21.2.108:6822/36380","heartbeat_back_addr":"172.21.2.108:6830/36380","heartbeat_front_addr":"172.21.2.108:6826/36380","state":["exists","up"]},{"osd":5,"uuid":"88e694e8-fe98-4d1b-88c8-8c9907ece60b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6808","nonce":36383},{"type":"v1","addr":"172.21.2.108:6809","nonce":36383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6810","nonce":36383},{"type":"v1","addr":"172.21.2.108:6811","nonce":36383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6814","nonce":36383},{"type":"v1","addr":"172.21.2.108:6815","nonce":36383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6812","nonce":36383},{"type":"v1","addr":"172.21.2.108:6813","nonce":36383}]},"public_addr":"172.21.2.108:6809/36383","cluster_addr":"172.21.2.108:6811/36383","heartbeat_back_addr":"172.21.2.108:6815/36383","heartbeat_front_addr":"172.21.2.108:6813/36383","state":["exists","up"]},{"osd":6,"uuid":"87a13c6c-cf85-4aaf-916b-e73913484a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6800","nonce":36381},{"type":"v1","addr":"172.21.2.108:6801","nonce":36381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6802","nonce":36381},{"type":"v1","addr":"172.21.2.108:6803","nonce":36381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6806","nonce":36381},{"type":"v1","addr":"172.21.2.108:6807","nonce":36381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6804","nonce":36381},{"type":"v1","addr":"172.21.2.108:6805","nonce":36381}]},"public_addr":"172.21.2.108:6801/36381","cluster_addr":"172.21.2.108:6803/36381","heartbeat_back_addr":"172.21.2.108:6807/36381","heartbeat_front_addr":"172.21.2.108:6805/36381","state":["exists","up"]},{"osd":7,"uuid":"7b90823f-854c-4592-a95b-a7fa98f93254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6817","nonce":36382},{"type":"v1","addr":"172.21.2.108:6819","nonce":36382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6821","nonce":36382},{"type":"v1","addr":"172.21.2.108:6823","nonce":36382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6829","nonce":36382},{"type":"v1","addr":"172.21.2.108:6831","nonce":36382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.108:6825","nonce":36382},{"type":"v1","addr":"172.21.2.108:6827","nonce":36382}]},"public_addr":"172.21.2.108:6819/36382","cluster_addr":"172.21.2.108:6823/36382","heartbeat_back_addr":"172.21.2.108:6831/36382","heartbeat_front_addr":"172.21.2.108:6827/36382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-02-10T11:50:44.341 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.345 7f6561cb2700 1 -- 172.21.2.101:0/326435031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65400186c0 msgr2=0x7f654001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:44.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.345 7f6561cb2700 1 --2- 172.21.2.101:0/326435031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65400186c0 0x7f654001abc0 crc :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:44.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.345 7f6561cb2700 1 -- 172.21.2.101:0/326435031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f655c0939e0 msgr2=0x7f655c1ace00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:44.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.345 7f6561cb2700 1 --2- 172.21.2.101:0/326435031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f655c0939e0 0x7f655c1ace00 secure :-1 s=READY pgs=3450 cs=0 l=1 rev1=1 rx=0x7f6554009980 tx=0x7f655400bcd0).stop 2021-02-10T11:50:44.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.346 7f6561cb2700 1 -- 172.21.2.101:0/326435031 shutdown_connections 2021-02-10T11:50:44.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.346 7f6561cb2700 1 --2- 172.21.2.101:0/326435031 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f655c0939e0 0x7f655c1ace00 unknown :-1 s=CLOSED pgs=3450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:44.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.346 7f6561cb2700 1 --2- 172.21.2.101:0/326435031 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f655c1099b0 0x7f655c0933c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:44.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.346 7f6561cb2700 1 --2- 172.21.2.101:0/326435031 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65400186c0 0x7f654001abc0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:44.342 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.346 7f6561cb2700 1 --2- 172.21.2.101:0/326435031 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f655c1095b0 0x7f655c19e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:44.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.346 7f6561cb2700 1 -- 172.21.2.101:0/326435031 >> 172.21.2.101:0/326435031 conn(0x7f655c19f7b0 msgr2=0x7f655c18f380 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:44.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.346 7f6561cb2700 1 -- 172.21.2.101:0/326435031 shutdown_connections 2021-02-10T11:50:44.343 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:44.346 7f6561cb2700 1 -- 172.21.2.101:0/326435031 wait complete. 2021-02-10T11:50:45.351 INFO:tasks.ceph:Scrubbing osd.0 2021-02-10T11:50:45.351 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2021-02-10T11:50:45.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.772 7f47acc24700 1 Processor -- start 2021-02-10T11:50:45.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.772 7f47acc24700 1 -- start start 2021-02-10T11:50:45.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.773 7f47acc24700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f47a80d1880 0x7f47a81b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:45.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.773 7f47acc24700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a80fe1d0 con 0x7f47a81b0af0 2021-02-10T11:50:45.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.773 7f47acc24700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a8109a70 con 0x7f47a80cff80 2021-02-10T11:50:45.785 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.773 7f47acc24700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a8088070 con 0x7f47a80d1880 2021-02-10T11:50:45.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.773 7f47a685e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f47a80d1880 0x7f47a81b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:45.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.773 7f47a605d700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f47a81b0af0 0x7f47a8108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36732/0 (socket says 172.21.2.101:36732) 2021-02-10T11:50:45.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.773 7f47a605d700 1 -- 172.21.2.101:0/1726409459 learned_addr learned my addr 172.21.2.101:0/1726409459 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:50:45.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.773 7f47a705f700 1 --1- >> v1:172.21.2.108:6789/0 conn(0x7f47a80cff80 0x7f47a8109140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.108:6789/0 says I am v1:172.21.2.101:58378/0 (socket says 172.21.2.101:58378) 2021-02-10T11:50:45.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.774 7f47a685e700 1 -- 172.21.2.101:0/1726409459 >> v1:172.21.2.108:6789/0 conn(0x7f47a80cff80 legacy=0x7f47a8109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:45.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.774 7f47a685e700 1 -- 172.21.2.101:0/1726409459 >> v1:172.21.2.101:6789/0 conn(0x7f47a81b0af0 legacy=0x7f47a8108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:45.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.774 7f47a685e700 1 -- 172.21.2.101:0/1726409459 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a8088260 con 0x7f47a80d1880 2021-02-10T11:50:45.786 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.774 7f47a685e700 1 --2- 172.21.2.101:0/1726409459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47a80d1880 0x7f47a81b04d0 secure :-1 s=READY pgs=3452 cs=0 l=1 rev1=1 rx=0x7f4790006650 tx=0x7f4790009a80).ready entity=mon.0 client_cookie=770400fb73aabd86 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:45.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.774 7f47a585c700 1 -- 172.21.2.101:0/1726409459 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f47900009b0 con 0x7f47a80d1880 2021-02-10T11:50:45.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.774 7f47a585c700 1 -- 172.21.2.101:0/1726409459 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479000c0c0 con 0x7f47a80d1880 2021-02-10T11:50:45.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.774 7f47a585c700 1 -- 172.21.2.101:0/1726409459 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f479000c5e0 con 0x7f47a80d1880 2021-02-10T11:50:45.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.774 7f47acc24700 1 -- 172.21.2.101:0/1726409459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47a80d1880 msgr2=0x7f47a81b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:45.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.774 7f47acc24700 1 --2- 172.21.2.101:0/1726409459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47a80d1880 0x7f47a81b04d0 secure :-1 s=READY pgs=3452 cs=0 l=1 rev1=1 rx=0x7f4790006650 tx=0x7f4790009a80).stop 2021-02-10T11:50:45.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.775 7f47acc24700 1 -- 172.21.2.101:0/1726409459 shutdown_connections 2021-02-10T11:50:45.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.775 7f47acc24700 1 --2- 172.21.2.101:0/1726409459 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47a80d1880 0x7f47a81b04d0 unknown :-1 s=CLOSED pgs=3452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:45.787 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.775 7f47acc24700 1 -- 172.21.2.101:0/1726409459 >> 172.21.2.101:0/1726409459 conn(0x7f47a80ce430 msgr2=0x7f47a80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:45.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.775 7f47acc24700 1 -- 172.21.2.101:0/1726409459 shutdown_connections 2021-02-10T11:50:45.788 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.775 7f47acc24700 1 -- 172.21.2.101:0/1726409459 wait complete. 2021-02-10T11:50:45.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.789 7f47acc24700 1 Processor -- start 2021-02-10T11:50:45.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.789 7f47acc24700 1 -- start start 2021-02-10T11:50:45.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.790 7f47acc24700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f47a809fbb0 0x7f47a81b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:45.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.790 7f47acc24700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47a809ffb0 0x7f47a808a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:45.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.790 7f47acc24700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47a808af50 0x7f47a809c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:45.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.790 7f47acc24700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a809f6b0 con 0x7f47a808af50 2021-02-10T11:50:45.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.790 7f47acc24700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a808b390 con 0x7f47a809fbb0 2021-02-10T11:50:45.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.790 7f47acc24700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a81b6060 con 0x7f47a809ffb0 2021-02-10T11:50:45.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.790 7f47a705f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47a808af50 0x7f47a809c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:45.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.790 7f47a705f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47a808af50 0x7f47a809c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36244/0 (socket says 172.21.2.101:36244) 2021-02-10T11:50:45.790 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.790 7f47a705f700 1 -- 172.21.2.101:0/4288605088 learned_addr learned my addr 172.21.2.101:0/4288605088 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:45.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.790 7f47a685e700 1 --2- 172.21.2.101:0/4288605088 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f47a809fbb0 0x7f47a81b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:45.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.790 7f47a605d700 1 --2- 172.21.2.101:0/4288605088 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47a809ffb0 0x7f47a808a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:45.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.791 7f47a705f700 1 -- 172.21.2.101:0/4288605088 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47a809ffb0 msgr2=0x7f47a808a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:45.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.791 7f47a705f700 1 --2- 172.21.2.101:0/4288605088 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f47a809ffb0 0x7f47a808a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:45.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.791 7f47a705f700 1 -- 172.21.2.101:0/4288605088 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f47a809fbb0 msgr2=0x7f47a81b5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:45.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.791 7f47a705f700 1 --2- 172.21.2.101:0/4288605088 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f47a809fbb0 0x7f47a81b5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:45.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.791 7f47a705f700 1 -- 172.21.2.101:0/4288605088 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a809ca30 con 0x7f47a808af50 2021-02-10T11:50:45.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.791 7f47a705f700 1 --2- 172.21.2.101:0/4288605088 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47a808af50 0x7f47a809c410 secure :-1 s=READY pgs=3453 cs=0 l=1 rev1=1 rx=0x7f47a000a940 tx=0x7f47a000ace0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:45.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.791 7f479f7fe700 1 -- 172.21.2.101:0/4288605088 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f47a00011f0 con 0x7f47a808af50 2021-02-10T11:50:45.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.791 7f479f7fe700 1 -- 172.21.2.101:0/4288605088 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a00070d0 con 0x7f47a808af50 2021-02-10T11:50:45.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.791 7f479f7fe700 1 -- 172.21.2.101:0/4288605088 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f47a0007560 con 0x7f47a808af50 2021-02-10T11:50:45.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.791 7f47acc24700 1 -- 172.21.2.101:0/4288605088 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a809ca30 con 0x7f47a808af50 2021-02-10T11:50:45.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.791 7f47acc24700 1 -- 172.21.2.101:0/4288605088 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a809cfc0 con 0x7f47a808af50 2021-02-10T11:50:45.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.792 7f479f7fe700 1 -- 172.21.2.101:0/4288605088 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f47a00072b0 con 0x7f47a808af50 2021-02-10T11:50:45.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.792 7f479f7fe700 1 --2- 172.21.2.101:0/4288605088 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f478c0186c0 0x7f478c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:45.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.792 7f479f7fe700 1 -- 172.21.2.101:0/4288605088 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f47a0032800 con 0x7f47a808af50 2021-02-10T11:50:45.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.792 7f47a685e700 1 --2- 172.21.2.101:0/4288605088 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f478c0186c0 0x7f478c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:45.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.793 7f47a685e700 1 --2- 172.21.2.101:0/4288605088 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f478c0186c0 0x7f478c01abc0 crc :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:45.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.823 7f47acc24700 1 --2- 172.21.2.101:0/4288605088 >> [v2:172.21.2.101:6802/25093,v1:172.21.2.101:6805/25093] conn(0x7f47a80c4620 0x7f47a80627c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:45.818 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.823 7f47acc24700 1 -- 172.21.2.101:0/4288605088 --> [v2:172.21.2.101:6802/25093,v1:172.21.2.101:6805/25093] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47a809cfc0 con 0x7f47a80c4620 2021-02-10T11:50:45.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.823 7f47a605d700 1 --2- 172.21.2.101:0/4288605088 >> [v2:172.21.2.101:6802/25093,v1:172.21.2.101:6805/25093] conn(0x7f47a80c4620 0x7f47a80627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:45.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.823 7f47a605d700 1 --2- 172.21.2.101:0/4288605088 >> [v2:172.21.2.101:6802/25093,v1:172.21.2.101:6805/25093] conn(0x7f47a80c4620 0x7f47a80627c0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:45.819 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.824 7f479f7fe700 1 -- 172.21.2.101:0/4288605088 <== osd.0 v2:172.21.2.101:6802/25093 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (crc 0 0 0) 0x7f4798001a00 con 0x7f47a80c4620 2021-02-10T11:50:45.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.832 7f47acc24700 1 -- 172.21.2.101:0/4288605088 --> [v2:172.21.2.101:6802/25093,v1:172.21.2.101:6805/25093] -- command(tid 2: {"prefix": "config set", "key": "osd_debug_deep_scrub_sleep", "value": "0"}) v1 -- 0x7f47a809ca30 con 0x7f47a80c4620 2021-02-10T11:50:45.827 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.832 7f479f7fe700 1 -- 172.21.2.101:0/4288605088 <== osd.0 v2:172.21.2.101:6802/25093 2 ==== command_reply(tid 2: 0 Set osd_debug_deep_scrub_sleep to 0) v1 ==== 43+0+0 (crc 0 0 0) 0x7f4798001a00 con 0x7f47a80c4620 2021-02-10T11:50:45.828 INFO:teuthology.orchestra.run.gibba001.stderr:Set osd_debug_deep_scrub_sleep to 0 2021-02-10T11:50:45.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.833 7f47acc24700 1 -- 172.21.2.101:0/4288605088 >> [v2:172.21.2.101:6802/25093,v1:172.21.2.101:6805/25093] conn(0x7f47a80c4620 msgr2=0x7f47a80627c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:45.829 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.833 7f47acc24700 1 --2- 172.21.2.101:0/4288605088 >> [v2:172.21.2.101:6802/25093,v1:172.21.2.101:6805/25093] conn(0x7f47a80c4620 0x7f47a80627c0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:45.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.833 7f47acc24700 1 -- 172.21.2.101:0/4288605088 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f478c0186c0 msgr2=0x7f478c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:45.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.833 7f47acc24700 1 --2- 172.21.2.101:0/4288605088 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f478c0186c0 0x7f478c01abc0 crc :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:45.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.833 7f47acc24700 1 -- 172.21.2.101:0/4288605088 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47a808af50 msgr2=0x7f47a809c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:45.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.833 7f47acc24700 1 --2- 172.21.2.101:0/4288605088 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f47a808af50 0x7f47a809c410 secure :-1 s=READY pgs=3453 cs=0 l=1 rev1=1 rx=0x7f47a000a940 tx=0x7f47a000ace0).stop 2021-02-10T11:50:45.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.833 7f47a705f700 1 -- 172.21.2.101:0/4288605088 reap_dead start 2021-02-10T11:50:45.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.833 7f47acc24700 1 -- 172.21.2.101:0/4288605088 shutdown_connections 2021-02-10T11:50:45.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.833 7f47acc24700 1 -- 172.21.2.101:0/4288605088 >> 172.21.2.101:0/4288605088 conn(0x7f47a809bad0 msgr2=0x7f47a80804e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:45.830 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.834 7f47acc24700 1 -- 172.21.2.101:0/4288605088 shutdown_connections 2021-02-10T11:50:45.831 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:45.834 7f47acc24700 1 -- 172.21.2.101:0/4288605088 wait complete. 2021-02-10T11:50:45.837 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd deep-scrub 0 2021-02-10T11:50:46.269 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.255 7fa7e8418700 1 Processor -- start 2021-02-10T11:50:46.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.255 7fa7e8418700 1 -- start start 2021-02-10T11:50:46.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.256 7fa7e8418700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7e00d1870 0x7fa7e01b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:46.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.256 7fa7e8418700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e00f0b10 con 0x7fa7e01b0a10 2021-02-10T11:50:46.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.256 7fa7e8418700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e00d0310 con 0x7fa7e00f0650 2021-02-10T11:50:46.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.256 7fa7e8418700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e00e5ed0 con 0x7fa7e00d1870 2021-02-10T11:50:46.270 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.256 7fa7e5a33700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fa7e01b0a10 0x7fa7e00cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36748/0 (socket says 172.21.2.101:36748) 2021-02-10T11:50:46.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.256 7fa7e5a33700 1 -- 172.21.2.101:0/4198295110 learned_addr learned my addr 172.21.2.101:0/4198295110 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:50:46.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.256 7fa7e5232700 1 -- 172.21.2.101:0/4198295110 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091997966 0 0) 0x7fa7dc0010b0 con 0x7fa7e01b0a10 2021-02-10T11:50:46.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.256 7fa7e5232700 1 -- 172.21.2.101:0/4198295110 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7cc002fe0 con 0x7fa7e01b0a10 2021-02-10T11:50:46.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.257 7fa7e6234700 1 --2- 172.21.2.101:0/4198295110 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7e00d1870 0x7fa7e01b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:46.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.257 7fa7e5232700 1 -- 172.21.2.101:0/4198295110 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3909877993 0 0) 0x7fa7dc001010 con 0x7fa7e01b0a10 2021-02-10T11:50:46.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.257 7fa7e5232700 1 -- 172.21.2.101:0/4198295110 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7e00d1870 msgr2=0x7fa7e01b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:46.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.257 7fa7e5232700 1 --2- 172.21.2.101:0/4198295110 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7e00d1870 0x7fa7e01b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:46.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.257 7fa7e5232700 1 -- 172.21.2.101:0/4198295110 >> v1:172.21.2.108:6789/0 conn(0x7fa7e00f0650 legacy=0x7fa7e00cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:46.271 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.257 7fa7e5232700 1 -- 172.21.2.101:0/4198295110 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e00d0530 con 0x7fa7e01b0a10 2021-02-10T11:50:46.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.257 7fa7e5232700 1 -- 172.21.2.101:0/4198295110 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7fa7dc001530 con 0x7fa7e01b0a10 2021-02-10T11:50:46.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.257 7fa7e5232700 1 -- 172.21.2.101:0/4198295110 >> v1:172.21.2.101:6789/0 conn(0x7fa7e01b0a10 legacy=0x7fa7e00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:46.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.257 7fa7e5232700 1 --2- 172.21.2.101:0/4198295110 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7cc003b10 0x7fa7cc005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:46.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.257 7fa7e5232700 1 --2- 172.21.2.101:0/4198295110 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7cc006820 0x7fa7cc008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:46.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.257 7fa7e5232700 1 --2- 172.21.2.101:0/4198295110 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7cc009620 0x7fa7cc00bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:46.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.257 7fa7e5232700 1 -- 172.21.2.101:0/4198295110 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc00c360 con 0x7fa7cc003b10 2021-02-10T11:50:46.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.257 7fa7e5232700 1 -- 172.21.2.101:0/4198295110 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc00c580 con 0x7fa7cc006820 2021-02-10T11:50:46.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.257 7fa7e5232700 1 -- 172.21.2.101:0/4198295110 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc00c7a0 con 0x7fa7cc009620 2021-02-10T11:50:46.272 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.257 7fa7e6234700 1 --2- 172.21.2.101:0/4198295110 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7e00d1870 0x7fa7e01b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:50:46.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.257 7fa7e6234700 1 --2- 172.21.2.101:0/4198295110 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7cc003b10 0x7fa7cc005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:46.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.257 7fa7e5a33700 1 --2- 172.21.2.101:0/4198295110 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7cc006820 0x7fa7cc008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:46.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.257 7fa7e6a35700 1 --2- 172.21.2.101:0/4198295110 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7cc009620 0x7fa7cc00bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:46.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.258 7fa7e6234700 1 -- 172.21.2.101:0/4198295110 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7cc009620 msgr2=0x7fa7cc00bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:46.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.258 7fa7e6234700 1 --2- 172.21.2.101:0/4198295110 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7cc009620 0x7fa7cc00bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:46.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.258 7fa7e6234700 1 -- 172.21.2.101:0/4198295110 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7cc006820 msgr2=0x7fa7cc008d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:46.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.258 7fa7e6234700 1 --2- 172.21.2.101:0/4198295110 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7cc006820 0x7fa7cc008d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:46.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.258 7fa7e6234700 1 -- 172.21.2.101:0/4198295110 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7cc00ca40 con 0x7fa7cc003b10 2021-02-10T11:50:46.273 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.258 7fa7e5a33700 1 --2- 172.21.2.101:0/4198295110 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7cc006820 0x7fa7cc008d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:50:46.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.258 7fa7e6234700 1 --2- 172.21.2.101:0/4198295110 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7cc003b10 0x7fa7cc005f80 secure :-1 s=READY pgs=3455 cs=0 l=1 rev1=1 rx=0x7fa7d40064c0 tx=0x7fa7d404dac0).ready entity=mon.0 client_cookie=dbd75510bcc9bbe8 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:46.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.258 7fa7e5232700 1 -- 172.21.2.101:0/4198295110 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa7d40500c0 con 0x7fa7cc003b10 2021-02-10T11:50:46.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.258 7fa7e5232700 1 -- 172.21.2.101:0/4198295110 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d4050340 con 0x7fa7cc003b10 2021-02-10T11:50:46.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.258 7fa7e5232700 1 -- 172.21.2.101:0/4198295110 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa7d4050820 con 0x7fa7cc003b10 2021-02-10T11:50:46.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.258 7fa7e8418700 1 -- 172.21.2.101:0/4198295110 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7cc003b10 msgr2=0x7fa7cc005f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:46.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.258 7fa7e8418700 1 --2- 172.21.2.101:0/4198295110 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7cc003b10 0x7fa7cc005f80 secure :-1 s=READY pgs=3455 cs=0 l=1 rev1=1 rx=0x7fa7d40064c0 tx=0x7fa7d404dac0).stop 2021-02-10T11:50:46.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.258 7fa7e6a35700 1 -- 172.21.2.101:0/4198295110 reap_dead start 2021-02-10T11:50:46.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.259 7fa7e8418700 1 -- 172.21.2.101:0/4198295110 shutdown_connections 2021-02-10T11:50:46.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.259 7fa7e8418700 1 -- 172.21.2.101:0/4198295110 >> 172.21.2.101:0/4198295110 conn(0x7fa7e00ce450 msgr2=0x7fa7e00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:46.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.259 7fa7e8418700 1 -- 172.21.2.101:0/4198295110 shutdown_connections 2021-02-10T11:50:46.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.259 7fa7e8418700 1 -- 172.21.2.101:0/4198295110 wait complete. 2021-02-10T11:50:46.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.274 7fa7e8418700 1 Processor -- start 2021-02-10T11:50:46.275 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.274 7fa7e8418700 1 -- start start 2021-02-10T11:50:46.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.274 7fa7e8418700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7e01a43d0 0x7fa7e00a3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:46.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.274 7fa7e8418700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7e01a47d0 0x7fa7e00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:46.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.274 7fa7e8418700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7e00a40d0 0x7fa7e00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:46.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.274 7fa7e8418700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e00a4840 con 0x7fa7e01a47d0 2021-02-10T11:50:46.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.274 7fa7e8418700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e00a4510 con 0x7fa7e01a43d0 2021-02-10T11:50:46.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.274 7fa7e8418700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e00a3060 con 0x7fa7e00a40d0 2021-02-10T11:50:46.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.274 7fa7e5a33700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7e01a47d0 0x7fa7e00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:46.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.275 7fa7e5a33700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7e01a47d0 0x7fa7e00a3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36266/0 (socket says 172.21.2.101:36266) 2021-02-10T11:50:46.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.275 7fa7e5a33700 1 -- 172.21.2.101:0/1860469100 learned_addr learned my addr 172.21.2.101:0/1860469100 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:46.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.275 7fa7e5a33700 1 -- 172.21.2.101:0/1860469100 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7e00a40d0 msgr2=0x7fa7e00a4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:46.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.275 7fa7e5a33700 1 --2- 172.21.2.101:0/1860469100 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7e00a40d0 0x7fa7e00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:46.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.275 7fa7e5a33700 1 -- 172.21.2.101:0/1860469100 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7e01a43d0 msgr2=0x7fa7e00a3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:46.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.275 7fa7e5a33700 1 --2- 172.21.2.101:0/1860469100 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7e01a43d0 0x7fa7e00a3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:46.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.275 7fa7e5a33700 1 -- 172.21.2.101:0/1860469100 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e00a5360 con 0x7fa7e01a47d0 2021-02-10T11:50:46.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.275 7fa7e5a33700 1 --2- 172.21.2.101:0/1860469100 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7e01a47d0 0x7fa7e00a3ab0 secure :-1 s=READY pgs=3456 cs=0 l=1 rev1=1 rx=0x7fa7dc001c10 tx=0x7fa7dc00a8f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:46.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.275 7fa7d2ffd700 1 -- 172.21.2.101:0/1860469100 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa7dc000c90 con 0x7fa7e01a47d0 2021-02-10T11:50:46.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.275 7fa7d2ffd700 1 -- 172.21.2.101:0/1860469100 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7dc0012c0 con 0x7fa7e01a47d0 2021-02-10T11:50:46.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.275 7fa7d2ffd700 1 -- 172.21.2.101:0/1860469100 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa7dc004350 con 0x7fa7e01a47d0 2021-02-10T11:50:46.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.275 7fa7e8418700 1 -- 172.21.2.101:0/1860469100 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e00a5360 con 0x7fa7e01a47d0 2021-02-10T11:50:46.278 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.275 7fa7e8418700 1 -- 172.21.2.101:0/1860469100 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e00a58f0 con 0x7fa7e01a47d0 2021-02-10T11:50:46.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.276 7fa7d2ffd700 1 -- 172.21.2.101:0/1860469100 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fa7dc004140 con 0x7fa7e01a47d0 2021-02-10T11:50:46.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.276 7fa7d2ffd700 1 --2- 172.21.2.101:0/1860469100 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7c80186c0 0x7fa7c801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:46.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.276 7fa7d2ffd700 1 -- 172.21.2.101:0/1860469100 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fa7dc0325a0 con 0x7fa7e01a47d0 2021-02-10T11:50:46.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.276 7fa7e6234700 1 --2- 172.21.2.101:0/1860469100 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7c80186c0 0x7fa7c801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:46.279 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.276 7fa7e6234700 1 --2- 172.21.2.101:0/1860469100 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7c80186c0 0x7fa7c801abc0 crc :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:46.301 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.306 7fa7e8418700 1 -- 172.21.2.101:0/1860469100 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e00ebd00 con 0x7fa7e01a47d0 2021-02-10T11:50:46.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.308 7fa7d2ffd700 1 -- 172.21.2.101:0/1860469100 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fa7dc000960 con 0x7fa7e01a47d0 2021-02-10T11:50:46.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.435 7fa7e8418700 1 -- 172.21.2.101:0/1860469100 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mgr", ""]}) v1 -- 0x7fa7e00a58f0 con 0x7fa7c80186c0 2021-02-10T11:50:46.431 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.436 7fa7d2ffd700 1 -- 172.21.2.101:0/1860469100 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (crc 0 0 0) 0x7fa7d4006550 con 0x7fa7c80186c0 2021-02-10T11:50:46.432 INFO:teuthology.orchestra.run.gibba001.stderr:instructed osd(s) 0 to deep-scrub 2021-02-10T11:50:46.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.438 7fa7e8418700 1 -- 172.21.2.101:0/1860469100 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7c80186c0 msgr2=0x7fa7c801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:46.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.438 7fa7e8418700 1 --2- 172.21.2.101:0/1860469100 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7c80186c0 0x7fa7c801abc0 crc :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:46.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.438 7fa7e8418700 1 -- 172.21.2.101:0/1860469100 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7e01a47d0 msgr2=0x7fa7e00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:46.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.438 7fa7e8418700 1 --2- 172.21.2.101:0/1860469100 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7e01a47d0 0x7fa7e00a3ab0 secure :-1 s=READY pgs=3456 cs=0 l=1 rev1=1 rx=0x7fa7dc001c10 tx=0x7fa7dc00a8f0).stop 2021-02-10T11:50:46.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.438 7fa7e8418700 1 -- 172.21.2.101:0/1860469100 shutdown_connections 2021-02-10T11:50:46.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.438 7fa7e8418700 1 --2- 172.21.2.101:0/1860469100 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fa7c80186c0 0x7fa7c801abc0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:46.434 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.438 7fa7e8418700 1 --2- 172.21.2.101:0/1860469100 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fa7e00a40d0 0x7fa7e00a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:46.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.438 7fa7e8418700 1 --2- 172.21.2.101:0/1860469100 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fa7e01a47d0 0x7fa7e00a3ab0 unknown :-1 s=CLOSED pgs=3456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:46.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.438 7fa7e8418700 1 --2- 172.21.2.101:0/1860469100 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fa7e01a43d0 0x7fa7e00a3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:46.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.438 7fa7e8418700 1 -- 172.21.2.101:0/1860469100 >> 172.21.2.101:0/1860469100 conn(0x7fa7e00beee0 msgr2=0x7fa7e009f200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:46.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.438 7fa7e8418700 1 -- 172.21.2.101:0/1860469100 shutdown_connections 2021-02-10T11:50:46.435 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.438 7fa7e8418700 1 -- 172.21.2.101:0/1860469100 wait complete. 2021-02-10T11:50:46.442 INFO:tasks.ceph:Scrubbing osd.1 2021-02-10T11:50:46.442 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2021-02-10T11:50:46.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.865 7fb287dc7700 1 Processor -- start 2021-02-10T11:50:46.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.865 7fb287dc7700 1 -- start start 2021-02-10T11:50:46.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.865 7fb287dc7700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb2800d1870 0x7fb2801b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:46.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.865 7fb287dc7700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb2800cff70 0x7fb280109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:46.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.865 7fb287dc7700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2800fe1c0 con 0x7fb2801b0ae0 2021-02-10T11:50:46.878 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.865 7fb287dc7700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb280109ad0 con 0x7fb2800cff70 2021-02-10T11:50:46.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.865 7fb287dc7700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb280109750 con 0x7fb2800d1870 2021-02-10T11:50:46.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.866 7fb2863e4700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb2800cff70 0x7fb280109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:46.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.866 7fb2853e2700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb2801b0ae0 0x7fb280108940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36766/0 (socket says 172.21.2.101:36766) 2021-02-10T11:50:46.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.866 7fb2853e2700 1 -- 172.21.2.101:0/2790157124 learned_addr learned my addr 172.21.2.101:0/2790157124 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:50:46.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.866 7fb284be1700 1 -- 172.21.2.101:0/2790157124 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 714483716 0 0) 0x7fb27c0010b0 con 0x7fb2801b0ae0 2021-02-10T11:50:46.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.866 7fb2863e4700 1 -- 172.21.2.101:0/2790157124 >> v2:172.21.2.108:3300/0 conn(0x7fb2800d1870 msgr2=0x7fb2801b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:46.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.866 7fb2863e4700 1 --2- 172.21.2.101:0/2790157124 >> v2:172.21.2.108:3300/0 conn(0x7fb2800d1870 0x7fb2801b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:46.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.866 7fb2863e4700 1 -- 172.21.2.101:0/2790157124 >> v1:172.21.2.101:6789/0 conn(0x7fb2801b0ae0 legacy=0x7fb280108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:46.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.866 7fb2863e4700 1 -- 172.21.2.101:0/2790157124 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb280088070 con 0x7fb2800cff70 2021-02-10T11:50:46.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.866 7fb2863e4700 1 --2- 172.21.2.101:0/2790157124 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2800cff70 0x7fb280109130 secure :-1 s=READY pgs=3458 cs=0 l=1 rev1=1 rx=0x7fb278006e30 tx=0x7fb278009e60).ready entity=mon.0 client_cookie=cec5ecd28d6990ef server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:46.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.867 7fb284be1700 1 -- 172.21.2.101:0/2790157124 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb278006a50 con 0x7fb2800cff70 2021-02-10T11:50:46.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.867 7fb284be1700 1 -- 172.21.2.101:0/2790157124 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27800b300 con 0x7fb2800cff70 2021-02-10T11:50:46.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.867 7fb284be1700 1 -- 172.21.2.101:0/2790157124 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb27800b820 con 0x7fb2800cff70 2021-02-10T11:50:46.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.867 7fb287dc7700 1 -- 172.21.2.101:0/2790157124 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2800cff70 msgr2=0x7fb280109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:46.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.867 7fb287dc7700 1 --2- 172.21.2.101:0/2790157124 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2800cff70 0x7fb280109130 secure :-1 s=READY pgs=3458 cs=0 l=1 rev1=1 rx=0x7fb278006e30 tx=0x7fb278009e60).stop 2021-02-10T11:50:46.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.867 7fb287dc7700 1 -- 172.21.2.101:0/2790157124 shutdown_connections 2021-02-10T11:50:46.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.867 7fb287dc7700 1 --2- 172.21.2.101:0/2790157124 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2800cff70 0x7fb280109130 unknown :-1 s=CLOSED pgs=3458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:46.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.867 7fb287dc7700 1 --2- 172.21.2.101:0/2790157124 >> v2:172.21.2.108:3300/0 conn(0x7fb2800d1870 0x7fb2801b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:46.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.867 7fb287dc7700 1 -- 172.21.2.101:0/2790157124 >> 172.21.2.101:0/2790157124 conn(0x7fb2800ce420 msgr2=0x7fb2800d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:46.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.868 7fb287dc7700 1 -- 172.21.2.101:0/2790157124 shutdown_connections 2021-02-10T11:50:46.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.868 7fb287dc7700 1 -- 172.21.2.101:0/2790157124 wait complete. 2021-02-10T11:50:46.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.882 7fb287dc7700 1 Processor -- start 2021-02-10T11:50:46.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.882 7fb287dc7700 1 -- start start 2021-02-10T11:50:46.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.883 7fb287dc7700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb28019f040 0x7fb2801a7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:46.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.883 7fb287dc7700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2800b1fb0 0x7fb2800b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:46.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.883 7fb287dc7700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb280082800 0x7fb280082c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:46.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.883 7fb287dc7700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2800833a0 con 0x7fb280082800 2021-02-10T11:50:46.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.883 7fb287dc7700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2801a6f90 con 0x7fb2800b1fb0 2021-02-10T11:50:46.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.883 7fb287dc7700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28019f650 con 0x7fb28019f040 2021-02-10T11:50:46.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.883 7fb2863e4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb280082800 0x7fb280082c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:46.883 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.883 7fb2863e4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb280082800 0x7fb280082c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36276/0 (socket says 172.21.2.101:36276) 2021-02-10T11:50:46.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.883 7fb2863e4700 1 -- 172.21.2.101:0/3234446298 learned_addr learned my addr 172.21.2.101:0/3234446298 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:46.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.883 7fb2863e4700 1 -- 172.21.2.101:0/3234446298 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb28019f040 msgr2=0x7fb2801a7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:46.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.883 7fb2863e4700 1 --2- 172.21.2.101:0/3234446298 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb28019f040 0x7fb2801a7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:46.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.883 7fb2863e4700 1 -- 172.21.2.101:0/3234446298 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2800b1fb0 msgr2=0x7fb2800b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:46.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.883 7fb2863e4700 1 --2- 172.21.2.101:0/3234446298 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2800b1fb0 0x7fb2800b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:46.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.883 7fb2863e4700 1 -- 172.21.2.101:0/3234446298 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2801a7b30 con 0x7fb280082800 2021-02-10T11:50:46.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.883 7fb2863e4700 1 --2- 172.21.2.101:0/3234446298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb280082800 0x7fb280082c00 secure :-1 s=READY pgs=3459 cs=0 l=1 rev1=1 rx=0x7fb27800b970 tx=0x7fb27800bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:46.884 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.884 7fb2727fc700 1 -- 172.21.2.101:0/3234446298 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb278009380 con 0x7fb280082800 2021-02-10T11:50:46.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.884 7fb2727fc700 1 -- 172.21.2.101:0/3234446298 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb278005820 con 0x7fb280082800 2021-02-10T11:50:46.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.884 7fb2727fc700 1 -- 172.21.2.101:0/3234446298 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb278005d40 con 0x7fb280082800 2021-02-10T11:50:46.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.884 7fb287dc7700 1 -- 172.21.2.101:0/3234446298 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2801a7b30 con 0x7fb280082800 2021-02-10T11:50:46.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.884 7fb287dc7700 1 -- 172.21.2.101:0/3234446298 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2801a7b30 con 0x7fb280082800 2021-02-10T11:50:46.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.885 7fb2727fc700 1 -- 172.21.2.101:0/3234446298 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb278005a00 con 0x7fb280082800 2021-02-10T11:50:46.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.885 7fb2727fc700 1 --2- 172.21.2.101:0/3234446298 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2680186c0 0x7fb26801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:46.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.885 7fb2727fc700 1 -- 172.21.2.101:0/3234446298 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb2780328e0 con 0x7fb280082800 2021-02-10T11:50:46.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.885 7fb285be3700 1 --2- 172.21.2.101:0/3234446298 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2680186c0 0x7fb26801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:46.885 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.885 7fb285be3700 1 --2- 172.21.2.101:0/3234446298 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2680186c0 0x7fb26801abc0 crc :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:46.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.915 7fb287dc7700 1 --2- 172.21.2.101:0/3234446298 >> [v2:172.21.2.101:6803/25092,v1:172.21.2.101:6804/25092] conn(0x7fb2801a65c0 0x7fb2800627c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:46.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.915 7fb287dc7700 1 -- 172.21.2.101:0/3234446298 --> [v2:172.21.2.101:6803/25092,v1:172.21.2.101:6804/25092] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2800825d0 con 0x7fb2801a65c0 2021-02-10T11:50:46.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.915 7fb2853e2700 1 --2- 172.21.2.101:0/3234446298 >> [v2:172.21.2.101:6803/25092,v1:172.21.2.101:6804/25092] conn(0x7fb2801a65c0 0x7fb2800627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:46.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.916 7fb2853e2700 1 --2- 172.21.2.101:0/3234446298 >> [v2:172.21.2.101:6803/25092,v1:172.21.2.101:6804/25092] conn(0x7fb2801a65c0 0x7fb2800627c0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:46.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.916 7fb2727fc700 1 -- 172.21.2.101:0/3234446298 <== osd.1 v2:172.21.2.101:6803/25092 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (crc 0 0 0) 0x7fb27c000940 con 0x7fb2801a65c0 2021-02-10T11:50:46.917 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.922 7fb287dc7700 1 -- 172.21.2.101:0/3234446298 --> [v2:172.21.2.101:6803/25092,v1:172.21.2.101:6804/25092] -- command(tid 2: {"prefix": "config set", "key": "osd_debug_deep_scrub_sleep", "value": "0"}) v1 -- 0x7fb2800825d0 con 0x7fb2801a65c0 2021-02-10T11:50:46.918 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.922 7fb2727fc700 1 -- 172.21.2.101:0/3234446298 <== osd.1 v2:172.21.2.101:6803/25092 2 ==== command_reply(tid 2: 0 Set osd_debug_deep_scrub_sleep to 0) v1 ==== 43+0+0 (crc 0 0 0) 0x7fb27c001760 con 0x7fb2801a65c0 2021-02-10T11:50:46.918 INFO:teuthology.orchestra.run.gibba001.stderr:Set osd_debug_deep_scrub_sleep to 0 2021-02-10T11:50:46.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.923 7fb287dc7700 1 -- 172.21.2.101:0/3234446298 >> [v2:172.21.2.101:6803/25092,v1:172.21.2.101:6804/25092] conn(0x7fb2801a65c0 msgr2=0x7fb2800627c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:46.919 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.923 7fb287dc7700 1 --2- 172.21.2.101:0/3234446298 >> [v2:172.21.2.101:6803/25092,v1:172.21.2.101:6804/25092] conn(0x7fb2801a65c0 0x7fb2800627c0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:46.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.923 7fb287dc7700 1 -- 172.21.2.101:0/3234446298 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2680186c0 msgr2=0x7fb26801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:46.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.923 7fb287dc7700 1 --2- 172.21.2.101:0/3234446298 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2680186c0 0x7fb26801abc0 crc :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:46.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.923 7fb287dc7700 1 -- 172.21.2.101:0/3234446298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb280082800 msgr2=0x7fb280082c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:46.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.923 7fb287dc7700 1 --2- 172.21.2.101:0/3234446298 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb280082800 0x7fb280082c00 secure :-1 s=READY pgs=3459 cs=0 l=1 rev1=1 rx=0x7fb27800b970 tx=0x7fb27800bd10).stop 2021-02-10T11:50:46.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.924 7fb2863e4700 1 -- 172.21.2.101:0/3234446298 reap_dead start 2021-02-10T11:50:46.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.924 7fb287dc7700 1 -- 172.21.2.101:0/3234446298 shutdown_connections 2021-02-10T11:50:46.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.924 7fb287dc7700 1 -- 172.21.2.101:0/3234446298 >> 172.21.2.101:0/3234446298 conn(0x7fb2801a2f90 msgr2=0x7fb28009aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:46.920 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.924 7fb287dc7700 1 -- 172.21.2.101:0/3234446298 shutdown_connections 2021-02-10T11:50:46.921 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:46.924 7fb287dc7700 1 -- 172.21.2.101:0/3234446298 wait complete. 2021-02-10T11:50:46.927 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd deep-scrub 1 2021-02-10T11:50:47.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.344 7f337c152700 1 Processor -- start 2021-02-10T11:50:47.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.344 7f337c152700 1 -- start start 2021-02-10T11:50:47.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.344 7f337c152700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f33740d1870 0x7f33741b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:47.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.344 7f337c152700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33741b0a10 0x7f33740cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:47.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.344 7f337c152700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33740f0b10 con 0x7f33740d1c70 2021-02-10T11:50:47.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.344 7f337c152700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33740e5ed0 con 0x7f33740d1870 2021-02-10T11:50:47.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.344 7f337c152700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33740f0650 con 0x7f33741b0a10 2021-02-10T11:50:47.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.345 7f337a76f700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f33740d1c70 0x7f33740cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36780/0 (socket says 172.21.2.101:36780) 2021-02-10T11:50:47.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.345 7f337a76f700 1 -- 172.21.2.101:0/2763401877 learned_addr learned my addr 172.21.2.101:0/2763401877 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:50:47.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.345 7f3378f6c700 1 -- 172.21.2.101:0/2763401877 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3844422871 0 0) 0x7f336c001500 con 0x7f33740d1c70 2021-02-10T11:50:47.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.345 7f337976d700 1 --2- 172.21.2.101:0/2763401877 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33741b0a10 0x7f33740cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:47.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.345 7f3379f6e700 1 --2- 172.21.2.101:0/2763401877 >> v2:172.21.2.108:3300/0 conn(0x7f33740d1870 0x7f33741b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:47.358 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.345 7f3378f6c700 1 -- 172.21.2.101:0/2763401877 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3360002fe0 con 0x7f33740d1c70 2021-02-10T11:50:47.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.345 7f3379f6e700 1 -- 172.21.2.101:0/2763401877 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33741b0a10 msgr2=0x7f33740cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:47.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.345 7f3379f6e700 1 --2- 172.21.2.101:0/2763401877 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33741b0a10 0x7f33740cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:47.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.345 7f3378f6c700 1 -- 172.21.2.101:0/2763401877 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2715890373 0 0) 0x7f336c001500 con 0x7f33740d1c70 2021-02-10T11:50:47.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.345 7f3379f6e700 1 -- 172.21.2.101:0/2763401877 >> v1:172.21.2.101:6789/0 conn(0x7f33740d1c70 legacy=0x7f33740cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:47.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.345 7f3379f6e700 1 -- 172.21.2.101:0/2763401877 --> v2:172.21.2.108:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33740f0870 con 0x7f33740d1870 2021-02-10T11:50:47.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.345 7f337976d700 1 --2- 172.21.2.101:0/2763401877 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33741b0a10 0x7f33740cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-02-10T11:50:47.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.345 7f3379f6e700 1 --2- 172.21.2.101:0/2763401877 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f33740d1870 0x7f33741b03f0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7f3370006650 tx=0x7f3370009a80).ready entity=mon.1 client_cookie=18d8ef4231248bdd server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:47.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.346 7f3378f6c700 1 -- 172.21.2.101:0/2763401877 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f33700009b0 con 0x7f33740d1870 2021-02-10T11:50:47.359 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.346 7f3378f6c700 1 -- 172.21.2.101:0/2763401877 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337000c0c0 con 0x7f33740d1870 2021-02-10T11:50:47.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.346 7f3378f6c700 1 -- 172.21.2.101:0/2763401877 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f337000c510 con 0x7f33740d1870 2021-02-10T11:50:47.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.346 7f337c152700 1 -- 172.21.2.101:0/2763401877 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f33740d1870 msgr2=0x7f33741b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:47.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.346 7f337c152700 1 --2- 172.21.2.101:0/2763401877 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f33740d1870 0x7f33741b03f0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7f3370006650 tx=0x7f3370009a80).stop 2021-02-10T11:50:47.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.346 7f337c152700 1 -- 172.21.2.101:0/2763401877 shutdown_connections 2021-02-10T11:50:47.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.346 7f337c152700 1 --2- 172.21.2.101:0/2763401877 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f33741b0a10 0x7f33740cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:47.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.346 7f337c152700 1 --2- 172.21.2.101:0/2763401877 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f33740d1870 0x7f33741b03f0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:47.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.346 7f337c152700 1 -- 172.21.2.101:0/2763401877 >> 172.21.2.101:0/2763401877 conn(0x7f33740ce450 msgr2=0x7f33740d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:47.360 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.346 7f337c152700 1 -- 172.21.2.101:0/2763401877 shutdown_connections 2021-02-10T11:50:47.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.346 7f337c152700 1 -- 172.21.2.101:0/2763401877 wait complete. 2021-02-10T11:50:47.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.361 7f337c152700 1 Processor -- start 2021-02-10T11:50:47.361 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.361 7f337c152700 1 -- start start 2021-02-10T11:50:47.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.361 7f337c152700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3374190660 0x7f33740ed140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:47.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.361 7f337c152700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33740ed760 0x7f33741a7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:47.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.361 7f337c152700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3374190a60 0x7f33741a78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:47.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.361 7f337c152700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33740ef770 con 0x7f33740ed760 2021-02-10T11:50:47.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.361 7f337c152700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33741a7ec0 con 0x7f3374190660 2021-02-10T11:50:47.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.361 7f337c152700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33741a80e0 con 0x7f3374190a60 2021-02-10T11:50:47.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.362 7f337976d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33740ed760 0x7f33741a7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:47.363 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.362 7f337976d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33740ed760 0x7f33741a7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36294/0 (socket says 172.21.2.101:36294) 2021-02-10T11:50:47.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.362 7f337976d700 1 -- 172.21.2.101:0/2391725424 learned_addr learned my addr 172.21.2.101:0/2391725424 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:47.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.362 7f3379f6e700 1 --2- 172.21.2.101:0/2391725424 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3374190660 0x7f33740ed140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:47.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.362 7f337a76f700 1 --2- 172.21.2.101:0/2391725424 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3374190a60 0x7f33741a78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:47.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.362 7f337976d700 1 -- 172.21.2.101:0/2391725424 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3374190a60 msgr2=0x7f33741a78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:47.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.362 7f337976d700 1 --2- 172.21.2.101:0/2391725424 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3374190a60 0x7f33741a78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:47.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.362 7f337976d700 1 -- 172.21.2.101:0/2391725424 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3374190660 msgr2=0x7f33740ed140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:47.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.362 7f337976d700 1 --2- 172.21.2.101:0/2391725424 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3374190660 0x7f33740ed140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:47.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.362 7f337976d700 1 -- 172.21.2.101:0/2391725424 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f337409a1b0 con 0x7f33740ed760 2021-02-10T11:50:47.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.362 7f337976d700 1 --2- 172.21.2.101:0/2391725424 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33740ed760 0x7f33741a7280 secure :-1 s=READY pgs=3461 cs=0 l=1 rev1=1 rx=0x7f3368006f40 tx=0x7f336800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:47.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.362 7f33667fc700 1 -- 172.21.2.101:0/2391725424 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f33680009b0 con 0x7f33740ed760 2021-02-10T11:50:47.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.362 7f33667fc700 1 -- 172.21.2.101:0/2391725424 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336800bd80 con 0x7f33740ed760 2021-02-10T11:50:47.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.362 7f33667fc700 1 -- 172.21.2.101:0/2391725424 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f33680022b0 con 0x7f33740ed760 2021-02-10T11:50:47.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.362 7f337c152700 1 -- 172.21.2.101:0/2391725424 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f337409a1b0 con 0x7f33740ed760 2021-02-10T11:50:47.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.363 7f337c152700 1 -- 172.21.2.101:0/2391725424 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f337409a1b0 con 0x7f33740ed760 2021-02-10T11:50:47.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.363 7f33667fc700 1 -- 172.21.2.101:0/2391725424 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3368000c90 con 0x7f33740ed760 2021-02-10T11:50:47.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.363 7f33667fc700 1 --2- 172.21.2.101:0/2391725424 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f335c0186c0 0x7f335c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:47.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.363 7f33667fc700 1 -- 172.21.2.101:0/2391725424 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3368032430 con 0x7f33740ed760 2021-02-10T11:50:47.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.363 7f3379f6e700 1 --2- 172.21.2.101:0/2391725424 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f335c0186c0 0x7f335c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:47.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.364 7f3379f6e700 1 --2- 172.21.2.101:0/2391725424 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f335c0186c0 0x7f335c01abc0 crc :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:47.390 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.394 7f337c152700 1 -- 172.21.2.101:0/2391725424 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33740ebc50 con 0x7f33740ed760 2021-02-10T11:50:47.392 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.397 7f33667fc700 1 -- 172.21.2.101:0/2391725424 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f3368006bd0 con 0x7f33740ed760 2021-02-10T11:50:47.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.525 7f337c152700 1 -- 172.21.2.101:0/2391725424 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mgr", ""]}) v1 -- 0x7f33740ebc50 con 0x7f335c0186c0 2021-02-10T11:50:47.521 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.525 7f33667fc700 1 -- 172.21.2.101:0/2391725424 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (crc 0 0 0) 0x7f33700009b0 con 0x7f335c0186c0 2021-02-10T11:50:47.521 INFO:teuthology.orchestra.run.gibba001.stderr:instructed osd(s) 1 to deep-scrub 2021-02-10T11:50:47.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.527 7f337c152700 1 -- 172.21.2.101:0/2391725424 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f335c0186c0 msgr2=0x7f335c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:47.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.527 7f337c152700 1 --2- 172.21.2.101:0/2391725424 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f335c0186c0 0x7f335c01abc0 crc :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:47.523 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.527 7f337c152700 1 -- 172.21.2.101:0/2391725424 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33740ed760 msgr2=0x7f33741a7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:47.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.527 7f337c152700 1 --2- 172.21.2.101:0/2391725424 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33740ed760 0x7f33741a7280 secure :-1 s=READY pgs=3461 cs=0 l=1 rev1=1 rx=0x7f3368006f40 tx=0x7f336800b990).stop 2021-02-10T11:50:47.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.527 7f337c152700 1 -- 172.21.2.101:0/2391725424 shutdown_connections 2021-02-10T11:50:47.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.527 7f337c152700 1 --2- 172.21.2.101:0/2391725424 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f335c0186c0 0x7f335c01abc0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:47.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.527 7f337c152700 1 --2- 172.21.2.101:0/2391725424 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f3374190a60 0x7f33741a78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:47.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.527 7f337c152700 1 --2- 172.21.2.101:0/2391725424 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f33740ed760 0x7f33741a7280 unknown :-1 s=CLOSED pgs=3461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:47.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.527 7f337c152700 1 --2- 172.21.2.101:0/2391725424 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f3374190660 0x7f33740ed140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:47.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.527 7f337c152700 1 -- 172.21.2.101:0/2391725424 >> 172.21.2.101:0/2391725424 conn(0x7f33740ae980 msgr2=0x7f33740baef0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:47.524 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.527 7f337c152700 1 -- 172.21.2.101:0/2391725424 shutdown_connections 2021-02-10T11:50:47.525 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.527 7f337c152700 1 -- 172.21.2.101:0/2391725424 wait complete. 2021-02-10T11:50:47.532 INFO:tasks.ceph:Scrubbing osd.2 2021-02-10T11:50:47.532 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2021-02-10T11:50:47.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.953 7f65da720700 1 Processor -- start 2021-02-10T11:50:47.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.953 7f65da720700 1 -- start start 2021-02-10T11:50:47.965 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.953 7f65da720700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f65d40d1880 0x7f65d41b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:47.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.953 7f65da720700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65d41b0af0 0x7f65d4108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:47.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.953 7f65da720700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f65d40d1c80 0x7f65d4108f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:47.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.953 7f65da720700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d40fe1d0 con 0x7f65d40d1c80 2021-02-10T11:50:47.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.953 7f65da720700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d4109a70 con 0x7f65d40d1880 2021-02-10T11:50:47.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.953 7f65da720700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d4088070 con 0x7f65d41b0af0 2021-02-10T11:50:47.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.954 7f65d8d3d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f65d40d1c80 0x7f65d4108f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:47.966 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.954 7f65d8d3d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f65d40d1c80 0x7f65d4108f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36304/0 (socket says 172.21.2.101:36304) 2021-02-10T11:50:47.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.954 7f65d8d3d700 1 -- 172.21.2.101:0/1153381923 learned_addr learned my addr 172.21.2.101:0/1153381923 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:47.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.954 7f65d8d3d700 1 -- 172.21.2.101:0/1153381923 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65d41b0af0 msgr2=0x7f65d4108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:47.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.954 7f65d8d3d700 1 --2- 172.21.2.101:0/1153381923 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65d41b0af0 0x7f65d4108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:47.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.954 7f65d8d3d700 1 -- 172.21.2.101:0/1153381923 >> v2:172.21.2.108:3300/0 conn(0x7f65d40d1880 msgr2=0x7f65d41b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:47.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.954 7f65d8d3d700 1 --2- 172.21.2.101:0/1153381923 >> v2:172.21.2.108:3300/0 conn(0x7f65d40d1880 0x7f65d41b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:47.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.954 7f65d8d3d700 1 -- 172.21.2.101:0/1153381923 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d4109590 con 0x7f65d40d1c80 2021-02-10T11:50:47.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.954 7f65d8d3d700 1 --2- 172.21.2.101:0/1153381923 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65d40d1c80 0x7f65d4108f70 secure :-1 s=READY pgs=3462 cs=0 l=1 rev1=1 rx=0x7f65cc006f00 tx=0x7f65cc009f10).ready entity=mon.0 client_cookie=fa1a0ad79f177d1e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:47.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.954 7f65d2ffd700 1 -- 172.21.2.101:0/1153381923 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f65cc00b3b0 con 0x7f65d40d1c80 2021-02-10T11:50:47.967 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.954 7f65d2ffd700 1 -- 172.21.2.101:0/1153381923 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65cc000d30 con 0x7f65d40d1c80 2021-02-10T11:50:47.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.954 7f65d2ffd700 1 -- 172.21.2.101:0/1153381923 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f65cc00b820 con 0x7f65d40d1c80 2021-02-10T11:50:47.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.954 7f65da720700 1 -- 172.21.2.101:0/1153381923 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65d40d1c80 msgr2=0x7f65d4108f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:47.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.954 7f65da720700 1 --2- 172.21.2.101:0/1153381923 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65d40d1c80 0x7f65d4108f70 secure :-1 s=READY pgs=3462 cs=0 l=1 rev1=1 rx=0x7f65cc006f00 tx=0x7f65cc009f10).stop 2021-02-10T11:50:47.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.955 7f65da720700 1 -- 172.21.2.101:0/1153381923 shutdown_connections 2021-02-10T11:50:47.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.955 7f65da720700 1 --2- 172.21.2.101:0/1153381923 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65d41b0af0 0x7f65d4108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:47.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.955 7f65da720700 1 --2- 172.21.2.101:0/1153381923 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65d40d1c80 0x7f65d4108f70 unknown :-1 s=CLOSED pgs=3462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:47.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.955 7f65da720700 1 --2- 172.21.2.101:0/1153381923 >> v2:172.21.2.108:3300/0 conn(0x7f65d40d1880 0x7f65d41b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:47.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.955 7f65da720700 1 -- 172.21.2.101:0/1153381923 >> 172.21.2.101:0/1153381923 conn(0x7f65d40ce430 msgr2=0x7f65d40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:47.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.955 7f65da720700 1 -- 172.21.2.101:0/1153381923 shutdown_connections 2021-02-10T11:50:47.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.955 7f65da720700 1 -- 172.21.2.101:0/1153381923 wait complete. 2021-02-10T11:50:47.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.970 7f65da720700 1 Processor -- start 2021-02-10T11:50:47.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.970 7f65da720700 1 -- start start 2021-02-10T11:50:47.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.970 7f65da720700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65d41b8e50 0x7f65d4091bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:47.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.970 7f65da720700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65d41b9250 0x7f65d40aa460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:47.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.970 7f65da720700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65d40aaa80 0x7f65d41acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:47.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.970 7f65da720700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d41b9770 con 0x7f65d41b8e50 2021-02-10T11:50:47.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.970 7f65da720700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d41ac570 con 0x7f65d40aaa80 2021-02-10T11:50:47.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.970 7f65da720700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d41ac160 con 0x7f65d41b9250 2021-02-10T11:50:47.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.971 7f65d3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65d41b8e50 0x7f65d4091bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:47.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.971 7f65d3fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65d41b8e50 0x7f65d4091bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36306/0 (socket says 172.21.2.101:36306) 2021-02-10T11:50:47.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.971 7f65d3fff700 1 -- 172.21.2.101:0/147048124 learned_addr learned my addr 172.21.2.101:0/147048124 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:47.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.971 7f65d3fff700 1 -- 172.21.2.101:0/147048124 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65d41b9250 msgr2=0x7f65d40aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:47.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.971 7f65d3fff700 1 --2- 172.21.2.101:0/147048124 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f65d41b9250 0x7f65d40aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:47.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.971 7f65d3fff700 1 -- 172.21.2.101:0/147048124 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65d40aaa80 msgr2=0x7f65d41acaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:47.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.971 7f65d3fff700 1 --2- 172.21.2.101:0/147048124 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f65d40aaa80 0x7f65d41acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:47.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.971 7f65d3fff700 1 -- 172.21.2.101:0/147048124 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d41ad110 con 0x7f65d41b8e50 2021-02-10T11:50:47.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.971 7f65d3fff700 1 --2- 172.21.2.101:0/147048124 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65d41b8e50 0x7f65d4091bf0 secure :-1 s=READY pgs=3463 cs=0 l=1 rev1=1 rx=0x7f65c4006f00 tx=0x7f65c400b940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:47.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.971 7f65d0ff9700 1 -- 172.21.2.101:0/147048124 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f65c400bd30 con 0x7f65d41b8e50 2021-02-10T11:50:47.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.971 7f65d0ff9700 1 -- 172.21.2.101:0/147048124 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c4000c90 con 0x7f65d41b8e50 2021-02-10T11:50:47.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.971 7f65d0ff9700 1 -- 172.21.2.101:0/147048124 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f65c40022a0 con 0x7f65d41b8e50 2021-02-10T11:50:47.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.971 7f65da720700 1 -- 172.21.2.101:0/147048124 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d41ad110 con 0x7f65d41b8e50 2021-02-10T11:50:47.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.971 7f65da720700 1 -- 172.21.2.101:0/147048124 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d41ad6a0 con 0x7f65d41b8e50 2021-02-10T11:50:47.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.972 7f65d0ff9700 1 -- 172.21.2.101:0/147048124 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f65c4005310 con 0x7f65d41b8e50 2021-02-10T11:50:47.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.972 7f65d0ff9700 1 --2- 172.21.2.101:0/147048124 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65b40186c0 0x7f65b401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:47.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.972 7f65d0ff9700 1 -- 172.21.2.101:0/147048124 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f65c4032510 con 0x7f65d41b8e50 2021-02-10T11:50:47.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.972 7f65d37fe700 1 --2- 172.21.2.101:0/147048124 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65b40186c0 0x7f65b401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:47.973 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:47.973 7f65d37fe700 1 --2- 172.21.2.101:0/147048124 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65b40186c0 0x7f65b401abc0 crc :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:47.998 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.003 7f65da720700 1 --2- 172.21.2.101:0/147048124 >> [v2:172.21.2.101:6811/25094,v1:172.21.2.101:6813/25094] conn(0x7f65d40627c0 0x7f65d4062bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:47.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.003 7f65da720700 1 -- 172.21.2.101:0/147048124 --> [v2:172.21.2.101:6811/25094,v1:172.21.2.101:6813/25094] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f65d41ac160 con 0x7f65d40627c0 2021-02-10T11:50:47.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.003 7f65d8d3d700 1 --2- 172.21.2.101:0/147048124 >> [v2:172.21.2.101:6811/25094,v1:172.21.2.101:6813/25094] conn(0x7f65d40627c0 0x7f65d4062bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:47.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.003 7f65d8d3d700 1 --2- 172.21.2.101:0/147048124 >> [v2:172.21.2.101:6811/25094,v1:172.21.2.101:6813/25094] conn(0x7f65d40627c0 0x7f65d4062bc0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:47.999 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.004 7f65d0ff9700 1 -- 172.21.2.101:0/147048124 <== osd.2 v2:172.21.2.101:6811/25094 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (crc 0 0 0) 0x7f65cc002e60 con 0x7f65d40627c0 2021-02-10T11:50:48.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.011 7f65da720700 1 -- 172.21.2.101:0/147048124 --> [v2:172.21.2.101:6811/25094,v1:172.21.2.101:6813/25094] -- command(tid 2: {"prefix": "config set", "key": "osd_debug_deep_scrub_sleep", "value": "0"}) v1 -- 0x7f65d41ad6a0 con 0x7f65d40627c0 2021-02-10T11:50:48.007 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.012 7f65d0ff9700 1 -- 172.21.2.101:0/147048124 <== osd.2 v2:172.21.2.101:6811/25094 2 ==== command_reply(tid 2: 0 Set osd_debug_deep_scrub_sleep to 0) v1 ==== 43+0+0 (crc 0 0 0) 0x7f65cc0090c0 con 0x7f65d40627c0 2021-02-10T11:50:48.008 INFO:teuthology.orchestra.run.gibba001.stderr:Set osd_debug_deep_scrub_sleep to 0 2021-02-10T11:50:48.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.013 7f65da720700 1 -- 172.21.2.101:0/147048124 >> [v2:172.21.2.101:6811/25094,v1:172.21.2.101:6813/25094] conn(0x7f65d40627c0 msgr2=0x7f65d4062bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:48.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.013 7f65da720700 1 --2- 172.21.2.101:0/147048124 >> [v2:172.21.2.101:6811/25094,v1:172.21.2.101:6813/25094] conn(0x7f65d40627c0 0x7f65d4062bc0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:48.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.013 7f65da720700 1 -- 172.21.2.101:0/147048124 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65b40186c0 msgr2=0x7f65b401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:48.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.013 7f65da720700 1 --2- 172.21.2.101:0/147048124 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f65b40186c0 0x7f65b401abc0 crc :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:48.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.013 7f65da720700 1 -- 172.21.2.101:0/147048124 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65d41b8e50 msgr2=0x7f65d4091bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:48.009 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.013 7f65da720700 1 --2- 172.21.2.101:0/147048124 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f65d41b8e50 0x7f65d4091bf0 secure :-1 s=READY pgs=3463 cs=0 l=1 rev1=1 rx=0x7f65c4006f00 tx=0x7f65c400b940).stop 2021-02-10T11:50:48.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.013 7f65d8d3d700 1 -- 172.21.2.101:0/147048124 reap_dead start 2021-02-10T11:50:48.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.013 7f65da720700 1 -- 172.21.2.101:0/147048124 shutdown_connections 2021-02-10T11:50:48.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.013 7f65da720700 1 -- 172.21.2.101:0/147048124 >> 172.21.2.101:0/147048124 conn(0x7f65d40846e0 msgr2=0x7f65d41ab3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:48.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.013 7f65da720700 1 -- 172.21.2.101:0/147048124 shutdown_connections 2021-02-10T11:50:48.010 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.013 7f65da720700 1 -- 172.21.2.101:0/147048124 wait complete. 2021-02-10T11:50:48.017 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd deep-scrub 2 2021-02-10T11:50:48.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.432 7fb28aa0e700 1 Processor -- start 2021-02-10T11:50:48.444 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.432 7fb28aa0e700 1 -- start start 2021-02-10T11:50:48.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.432 7fb28aa0e700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb2840d1870 0x7fb2841b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:48.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.432 7fb28aa0e700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb2840f0650 0x7fb2840cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:48.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.432 7fb28aa0e700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2840f0b10 con 0x7fb2841b0a10 2021-02-10T11:50:48.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.432 7fb28aa0e700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2840e5ed0 con 0x7fb2840d1870 2021-02-10T11:50:48.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.432 7fb28aa0e700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2840d1c70 con 0x7fb2840f0650 2021-02-10T11:50:48.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.433 7fb28882a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb2840d1870 0x7fb2841b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:48.445 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.433 7fb28882a700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb2840d1870 0x7fb2841b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36318/0 (socket says 172.21.2.101:36318) 2021-02-10T11:50:48.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.433 7fb28882a700 1 -- 172.21.2.101:0/424485562 learned_addr learned my addr 172.21.2.101:0/424485562 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:48.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.433 7fb28882a700 1 -- 172.21.2.101:0/424485562 >> v2:172.21.2.108:3300/0 conn(0x7fb2840f0650 msgr2=0x7fb2840cfb20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:50:48.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.433 7fb28882a700 1 --2- 172.21.2.101:0/424485562 >> v2:172.21.2.108:3300/0 conn(0x7fb2840f0650 0x7fb2840cfb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:48.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.433 7fb28882a700 1 -- 172.21.2.101:0/424485562 >> v1:172.21.2.101:6789/0 conn(0x7fb2841b0a10 legacy=0x7fb2840cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:48.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.433 7fb28882a700 1 -- 172.21.2.101:0/424485562 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2840d1e90 con 0x7fb2840d1870 2021-02-10T11:50:48.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.433 7fb28882a700 1 --2- 172.21.2.101:0/424485562 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2840d1870 0x7fb2841b03f0 secure :-1 s=READY pgs=3465 cs=0 l=1 rev1=1 rx=0x7fb26c006700 tx=0x7fb26c009b80).ready entity=mon.0 client_cookie=54c133eedd6dad9f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:48.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.433 7fb2837fe700 1 -- 172.21.2.101:0/424485562 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb26c00c0c0 con 0x7fb2840d1870 2021-02-10T11:50:48.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.433 7fb2837fe700 1 -- 172.21.2.101:0/424485562 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c00c400 con 0x7fb2840d1870 2021-02-10T11:50:48.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.433 7fb2837fe700 1 -- 172.21.2.101:0/424485562 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb26c00c8e0 con 0x7fb2840d1870 2021-02-10T11:50:48.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.433 7fb28aa0e700 1 -- 172.21.2.101:0/424485562 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2840d1870 msgr2=0x7fb2841b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:48.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.433 7fb28aa0e700 1 --2- 172.21.2.101:0/424485562 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2840d1870 0x7fb2841b03f0 secure :-1 s=READY pgs=3465 cs=0 l=1 rev1=1 rx=0x7fb26c006700 tx=0x7fb26c009b80).stop 2021-02-10T11:50:48.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.434 7fb28aa0e700 1 -- 172.21.2.101:0/424485562 shutdown_connections 2021-02-10T11:50:48.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.434 7fb28aa0e700 1 --2- 172.21.2.101:0/424485562 >> v2:172.21.2.108:3300/0 conn(0x7fb2840f0650 0x7fb2840cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:48.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.434 7fb28aa0e700 1 --2- 172.21.2.101:0/424485562 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb2840d1870 0x7fb2841b03f0 unknown :-1 s=CLOSED pgs=3465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:48.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.434 7fb28aa0e700 1 -- 172.21.2.101:0/424485562 >> 172.21.2.101:0/424485562 conn(0x7fb2840ce450 msgr2=0x7fb2840d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:48.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.434 7fb28aa0e700 1 -- 172.21.2.101:0/424485562 shutdown_connections 2021-02-10T11:50:48.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.434 7fb28aa0e700 1 -- 172.21.2.101:0/424485562 wait complete. 2021-02-10T11:50:48.449 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.449 7fb28aa0e700 1 Processor -- start 2021-02-10T11:50:48.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.449 7fb28aa0e700 1 -- start start 2021-02-10T11:50:48.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.449 7fb28aa0e700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2840bdda0 0x7fb28418b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:48.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.449 7fb28aa0e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb28418b870 0x7fb2840c0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:48.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.449 7fb28aa0e700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb28418bc70 0x7fb2840c0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:48.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.449 7fb28aa0e700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28419d390 con 0x7fb28418b870 2021-02-10T11:50:48.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.449 7fb28aa0e700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840bd740 con 0x7fb2840bdda0 2021-02-10T11:50:48.450 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.449 7fb28aa0e700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840be1a0 con 0x7fb28418bc70 2021-02-10T11:50:48.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.449 7fb283fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb28418b870 0x7fb2840c0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:48.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.450 7fb283fff700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb28418b870 0x7fb2840c0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36322/0 (socket says 172.21.2.101:36322) 2021-02-10T11:50:48.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.450 7fb283fff700 1 -- 172.21.2.101:0/566238277 learned_addr learned my addr 172.21.2.101:0/566238277 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:48.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.450 7fb283fff700 1 -- 172.21.2.101:0/566238277 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb28418bc70 msgr2=0x7fb2840c0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:48.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.450 7fb283fff700 1 --2- 172.21.2.101:0/566238277 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb28418bc70 0x7fb2840c0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:48.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.450 7fb283fff700 1 -- 172.21.2.101:0/566238277 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2840bdda0 msgr2=0x7fb28418b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:48.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.450 7fb283fff700 1 --2- 172.21.2.101:0/566238277 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2840bdda0 0x7fb28418b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:48.451 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.450 7fb283fff700 1 -- 172.21.2.101:0/566238277 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2840c0c70 con 0x7fb28418b870 2021-02-10T11:50:48.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.450 7fb283fff700 1 --2- 172.21.2.101:0/566238277 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb28418b870 0x7fb2840c0030 secure :-1 s=READY pgs=3466 cs=0 l=1 rev1=1 rx=0x7fb274009980 tx=0x7fb27400bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:48.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.450 7fb2817fa700 1 -- 172.21.2.101:0/566238277 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb27400c2f0 con 0x7fb28418b870 2021-02-10T11:50:48.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.450 7fb2817fa700 1 -- 172.21.2.101:0/566238277 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb274003270 con 0x7fb28418b870 2021-02-10T11:50:48.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.450 7fb28aa0e700 1 -- 172.21.2.101:0/566238277 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2840c0c70 con 0x7fb28418b870 2021-02-10T11:50:48.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.450 7fb2817fa700 1 -- 172.21.2.101:0/566238277 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb27400c7b0 con 0x7fb28418b870 2021-02-10T11:50:48.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.450 7fb28aa0e700 1 -- 172.21.2.101:0/566238277 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2840c0c70 con 0x7fb28418b870 2021-02-10T11:50:48.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.451 7fb2817fa700 1 -- 172.21.2.101:0/566238277 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb274001270 con 0x7fb28418b870 2021-02-10T11:50:48.452 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.451 7fb2817fa700 1 --2- 172.21.2.101:0/566238277 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2680186c0 0x7fb26801abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:48.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.451 7fb2817fa700 1 -- 172.21.2.101:0/566238277 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb274032930 con 0x7fb28418b870 2021-02-10T11:50:48.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.451 7fb28882a700 1 --2- 172.21.2.101:0/566238277 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2680186c0 0x7fb26801abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:48.453 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.452 7fb28882a700 1 --2- 172.21.2.101:0/566238277 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2680186c0 0x7fb26801abc0 crc :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:48.478 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.482 7fb28aa0e700 1 -- 172.21.2.101:0/566238277 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb28419d390 con 0x7fb28418b870 2021-02-10T11:50:48.480 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.485 7fb2817fa700 1 -- 172.21.2.101:0/566238277 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb27400be10 con 0x7fb28418b870 2021-02-10T11:50:48.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.612 7fb28aa0e700 1 -- 172.21.2.101:0/566238277 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mgr", ""]}) v1 -- 0x7fb28419d390 con 0x7fb2680186c0 2021-02-10T11:50:48.608 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.612 7fb2817fa700 1 -- 172.21.2.101:0/566238277 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (crc 0 0 0) 0x7fb26c000940 con 0x7fb2680186c0 2021-02-10T11:50:48.609 INFO:teuthology.orchestra.run.gibba001.stderr:instructed osd(s) 2 to deep-scrub 2021-02-10T11:50:48.610 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.614 7fb28aa0e700 1 -- 172.21.2.101:0/566238277 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2680186c0 msgr2=0x7fb26801abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:48.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.614 7fb28aa0e700 1 --2- 172.21.2.101:0/566238277 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2680186c0 0x7fb26801abc0 crc :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:48.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.614 7fb28aa0e700 1 -- 172.21.2.101:0/566238277 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb28418b870 msgr2=0x7fb2840c0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:48.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.614 7fb28aa0e700 1 --2- 172.21.2.101:0/566238277 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb28418b870 0x7fb2840c0030 secure :-1 s=READY pgs=3466 cs=0 l=1 rev1=1 rx=0x7fb274009980 tx=0x7fb27400bcd0).stop 2021-02-10T11:50:48.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.615 7fb28aa0e700 1 -- 172.21.2.101:0/566238277 shutdown_connections 2021-02-10T11:50:48.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.615 7fb28aa0e700 1 --2- 172.21.2.101:0/566238277 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb2680186c0 0x7fb26801abc0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:48.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.615 7fb28aa0e700 1 --2- 172.21.2.101:0/566238277 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb28418bc70 0x7fb2840c0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:48.611 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.615 7fb28aa0e700 1 --2- 172.21.2.101:0/566238277 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb28418b870 0x7fb2840c0030 unknown :-1 s=CLOSED pgs=3466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:48.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.615 7fb28aa0e700 1 --2- 172.21.2.101:0/566238277 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb2840bdda0 0x7fb28418b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:48.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.615 7fb28aa0e700 1 -- 172.21.2.101:0/566238277 >> 172.21.2.101:0/566238277 conn(0x7fb2840a6eb0 msgr2=0x7fb284098ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:48.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.615 7fb28aa0e700 1 -- 172.21.2.101:0/566238277 shutdown_connections 2021-02-10T11:50:48.612 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:48.615 7fb28aa0e700 1 -- 172.21.2.101:0/566238277 wait complete. 2021-02-10T11:50:48.619 INFO:tasks.ceph:Scrubbing osd.3 2021-02-10T11:50:48.619 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2021-02-10T11:50:49.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.041 7f9c37822700 1 Processor -- start 2021-02-10T11:50:49.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.041 7f9c37822700 1 -- start start 2021-02-10T11:50:49.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.041 7f9c37822700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f9c300d1c80 0x7f9c300cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:49.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.041 7f9c37822700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9c300cf590 0x7f9c300cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:49.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.041 7f9c37822700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c300fe1d0 con 0x7f9c300d1880 2021-02-10T11:50:49.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.041 7f9c37822700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c301b0cc0 con 0x7f9c300cf590 2021-02-10T11:50:49.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.041 7f9c37822700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c30109a70 con 0x7f9c300d1c80 2021-02-10T11:50:49.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.041 7f9c35e3f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9c300cf590 0x7f9c300cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:49.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.041 7f9c35e3f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f9c300cf590 0x7f9c300cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36330/0 (socket says 172.21.2.101:36330) 2021-02-10T11:50:49.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.041 7f9c35e3f700 1 -- 172.21.2.101:0/886950558 learned_addr learned my addr 172.21.2.101:0/886950558 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:49.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.041 7f9c35e3f700 1 -- 172.21.2.101:0/886950558 >> v2:172.21.2.108:3300/0 conn(0x7f9c300d1c80 msgr2=0x7f9c300cef70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:50:49.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.041 7f9c35e3f700 1 --2- 172.21.2.101:0/886950558 >> v2:172.21.2.108:3300/0 conn(0x7f9c300d1c80 0x7f9c300cef70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:49.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.041 7f9c35e3f700 1 -- 172.21.2.101:0/886950558 >> v1:172.21.2.108:6789/0 conn(0x7f9c300d1880 legacy=0x7f9c301b04d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:50:49.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.042 7f9c35e3f700 1 -- 172.21.2.101:0/886950558 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c30088070 con 0x7f9c300cf590 2021-02-10T11:50:49.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.042 7f9c35e3f700 1 --2- 172.21.2.101:0/886950558 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c300cf590 0x7f9c300cf990 secure :-1 s=READY pgs=3467 cs=0 l=1 rev1=1 rx=0x7f9c28006f00 tx=0x7f9c28009f10).ready entity=mon.0 client_cookie=55cb668fe248b43a server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:49.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.042 7f9c23fff700 1 -- 172.21.2.101:0/886950558 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9c2800b3b0 con 0x7f9c300cf590 2021-02-10T11:50:49.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.042 7f9c23fff700 1 -- 172.21.2.101:0/886950558 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2800b630 con 0x7f9c300cf590 2021-02-10T11:50:49.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.042 7f9c23fff700 1 -- 172.21.2.101:0/886950558 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9c2800bb50 con 0x7f9c300cf590 2021-02-10T11:50:49.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.042 7f9c37822700 1 -- 172.21.2.101:0/886950558 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c300cf590 msgr2=0x7f9c300cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:49.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.042 7f9c37822700 1 --2- 172.21.2.101:0/886950558 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c300cf590 0x7f9c300cf990 secure :-1 s=READY pgs=3467 cs=0 l=1 rev1=1 rx=0x7f9c28006f00 tx=0x7f9c28009f10).stop 2021-02-10T11:50:49.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.042 7f9c37822700 1 -- 172.21.2.101:0/886950558 shutdown_connections 2021-02-10T11:50:49.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.042 7f9c37822700 1 --2- 172.21.2.101:0/886950558 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c300cf590 0x7f9c300cf990 unknown :-1 s=CLOSED pgs=3467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:49.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.042 7f9c37822700 1 --2- 172.21.2.101:0/886950558 >> v2:172.21.2.108:3300/0 conn(0x7f9c300d1c80 0x7f9c300cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:49.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.042 7f9c37822700 1 -- 172.21.2.101:0/886950558 >> 172.21.2.101:0/886950558 conn(0x7f9c300ce430 msgr2=0x7f9c300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:49.055 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.043 7f9c37822700 1 -- 172.21.2.101:0/886950558 shutdown_connections 2021-02-10T11:50:49.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.043 7f9c37822700 1 -- 172.21.2.101:0/886950558 wait complete. 2021-02-10T11:50:49.057 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.057 7f9c37822700 1 Processor -- start 2021-02-10T11:50:49.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.057 7f9c37822700 1 -- start start 2021-02-10T11:50:49.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.057 7f9c37822700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9c300a6cc0 0x7f9c3008a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:49.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.057 7f9c37822700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c30085580 0x7f9c3008aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:49.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.057 7f9c37822700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9c300b7810 0x7f9c300b7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:49.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.058 7f9c37822700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3008b1f0 con 0x7f9c30085580 2021-02-10T11:50:49.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.058 7f9c37822700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c300b8230 con 0x7f9c300a6cc0 2021-02-10T11:50:49.058 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.058 7f9c37822700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c300b8450 con 0x7f9c300b7810 2021-02-10T11:50:49.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.058 7f9c34e3d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c30085580 0x7f9c3008aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:49.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.058 7f9c34e3d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c30085580 0x7f9c3008aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36336/0 (socket says 172.21.2.101:36336) 2021-02-10T11:50:49.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.058 7f9c34e3d700 1 -- 172.21.2.101:0/844066701 learned_addr learned my addr 172.21.2.101:0/844066701 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:49.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.058 7f9c34e3d700 1 -- 172.21.2.101:0/844066701 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9c300b7810 msgr2=0x7f9c300b7c10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:50:49.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.058 7f9c34e3d700 1 --2- 172.21.2.101:0/844066701 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f9c300b7810 0x7f9c300b7c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:49.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.058 7f9c34e3d700 1 -- 172.21.2.101:0/844066701 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9c300a6cc0 msgr2=0x7f9c3008a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:49.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.058 7f9c34e3d700 1 --2- 172.21.2.101:0/844066701 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f9c300a6cc0 0x7f9c3008a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:49.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.058 7f9c34e3d700 1 -- 172.21.2.101:0/844066701 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c300b8670 con 0x7f9c30085580 2021-02-10T11:50:49.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.058 7f9c34e3d700 1 --2- 172.21.2.101:0/844066701 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c30085580 0x7f9c3008aa40 secure :-1 s=READY pgs=3468 cs=0 l=1 rev1=1 rx=0x7f9c2c00b5c0 tx=0x7f9c2c00b960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:49.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.058 7f9c21ffb700 1 -- 172.21.2.101:0/844066701 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9c2c00bd50 con 0x7f9c30085580 2021-02-10T11:50:49.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.059 7f9c21ffb700 1 -- 172.21.2.101:0/844066701 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2c006030 con 0x7f9c30085580 2021-02-10T11:50:49.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.059 7f9c21ffb700 1 -- 172.21.2.101:0/844066701 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9c2c0054d0 con 0x7f9c30085580 2021-02-10T11:50:49.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.059 7f9c37822700 1 -- 172.21.2.101:0/844066701 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c300b8670 con 0x7f9c30085580 2021-02-10T11:50:49.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.059 7f9c37822700 1 -- 172.21.2.101:0/844066701 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c300b8670 con 0x7f9c30085580 2021-02-10T11:50:49.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.059 7f9c21ffb700 1 -- 172.21.2.101:0/844066701 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f9c2c006210 con 0x7f9c30085580 2021-02-10T11:50:49.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.059 7f9c21ffb700 1 --2- 172.21.2.101:0/844066701 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9c10018730 0x7f9c1001ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:49.060 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.059 7f9c21ffb700 1 -- 172.21.2.101:0/844066701 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f9c2c032620 con 0x7f9c30085580 2021-02-10T11:50:49.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.059 7f9c3563e700 1 --2- 172.21.2.101:0/844066701 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9c10018730 0x7f9c1001ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:49.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.060 7f9c3563e700 1 --2- 172.21.2.101:0/844066701 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9c10018730 0x7f9c1001ac10 crc :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:49.086 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.091 7f9c37822700 1 --2- 172.21.2.101:0/844066701 >> [v2:172.21.2.101:6824/25095,v1:172.21.2.101:6826/25095] conn(0x7f9c300b8230 0x7f9c3008c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:49.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.091 7f9c37822700 1 -- 172.21.2.101:0/844066701 --> [v2:172.21.2.101:6824/25095,v1:172.21.2.101:6826/25095] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9c30082ab0 con 0x7f9c300b8230 2021-02-10T11:50:49.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.091 7f9c35e3f700 1 --2- 172.21.2.101:0/844066701 >> [v2:172.21.2.101:6824/25095,v1:172.21.2.101:6826/25095] conn(0x7f9c300b8230 0x7f9c3008c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:49.087 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.091 7f9c35e3f700 1 --2- 172.21.2.101:0/844066701 >> [v2:172.21.2.101:6824/25095,v1:172.21.2.101:6826/25095] conn(0x7f9c300b8230 0x7f9c3008c0c0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:49.088 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.092 7f9c21ffb700 1 -- 172.21.2.101:0/844066701 <== osd.3 v2:172.21.2.101:6824/25095 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (crc 0 0 0) 0x7f9c28009420 con 0x7f9c300b8230 2021-02-10T11:50:49.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.099 7f9c37822700 1 -- 172.21.2.101:0/844066701 --> [v2:172.21.2.101:6824/25095,v1:172.21.2.101:6826/25095] -- command(tid 2: {"prefix": "config set", "key": "osd_debug_deep_scrub_sleep", "value": "0"}) v1 -- 0x7f9c300b8670 con 0x7f9c300b8230 2021-02-10T11:50:49.095 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.100 7f9c21ffb700 1 -- 172.21.2.101:0/844066701 <== osd.3 v2:172.21.2.101:6824/25095 2 ==== command_reply(tid 2: 0 Set osd_debug_deep_scrub_sleep to 0) v1 ==== 43+0+0 (crc 0 0 0) 0x7f9c280092a0 con 0x7f9c300b8230 2021-02-10T11:50:49.096 INFO:teuthology.orchestra.run.gibba001.stderr:Set osd_debug_deep_scrub_sleep to 0 2021-02-10T11:50:49.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.101 7f9c37822700 1 -- 172.21.2.101:0/844066701 >> [v2:172.21.2.101:6824/25095,v1:172.21.2.101:6826/25095] conn(0x7f9c300b8230 msgr2=0x7f9c3008c0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:49.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.101 7f9c37822700 1 --2- 172.21.2.101:0/844066701 >> [v2:172.21.2.101:6824/25095,v1:172.21.2.101:6826/25095] conn(0x7f9c300b8230 0x7f9c3008c0c0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:49.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.101 7f9c37822700 1 -- 172.21.2.101:0/844066701 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9c10018730 msgr2=0x7f9c1001ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:49.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.101 7f9c37822700 1 --2- 172.21.2.101:0/844066701 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f9c10018730 0x7f9c1001ac10 crc :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:49.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.101 7f9c37822700 1 -- 172.21.2.101:0/844066701 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c30085580 msgr2=0x7f9c3008aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:49.097 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.101 7f9c37822700 1 --2- 172.21.2.101:0/844066701 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f9c30085580 0x7f9c3008aa40 secure :-1 s=READY pgs=3468 cs=0 l=1 rev1=1 rx=0x7f9c2c00b5c0 tx=0x7f9c2c00b960).stop 2021-02-10T11:50:49.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.101 7f9c35e3f700 1 -- 172.21.2.101:0/844066701 reap_dead start 2021-02-10T11:50:49.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.101 7f9c37822700 1 -- 172.21.2.101:0/844066701 shutdown_connections 2021-02-10T11:50:49.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.101 7f9c37822700 1 -- 172.21.2.101:0/844066701 >> 172.21.2.101:0/844066701 conn(0x7f9c3019b540 msgr2=0x7f9c30082ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:49.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.101 7f9c37822700 1 -- 172.21.2.101:0/844066701 shutdown_connections 2021-02-10T11:50:49.098 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.101 7f9c37822700 1 -- 172.21.2.101:0/844066701 wait complete. 2021-02-10T11:50:49.105 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd deep-scrub 3 2021-02-10T11:50:49.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.522 7fb8eb534700 1 Processor -- start 2021-02-10T11:50:49.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.522 7fb8eb534700 1 -- start start 2021-02-10T11:50:49.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.523 7fb8eb534700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fb8e40d1870 0x7fb8e41b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:49.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.523 7fb8eb534700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fb8e41b0a10 0x7fb8e40cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:49.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.523 7fb8eb534700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8e40f0b10 con 0x7fb8e40d1c70 2021-02-10T11:50:49.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.523 7fb8eb534700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e40e5ed0 con 0x7fb8e41b0a10 2021-02-10T11:50:49.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.523 7fb8eb534700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e40f0650 con 0x7fb8e40d1870 2021-02-10T11:50:49.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.523 7fb8e9b51700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fb8e40d1c70 0x7fb8e40cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36842/0 (socket says 172.21.2.101:36842) 2021-02-10T11:50:49.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.523 7fb8e9b51700 1 -- 172.21.2.101:0/2797113616 learned_addr learned my addr 172.21.2.101:0/2797113616 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:50:49.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.523 7fb8e8b4f700 1 --2- 172.21.2.101:0/2797113616 >> v2:172.21.2.101:3300/0 conn(0x7fb8e41b0a10 0x7fb8e40cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:49.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.523 7fb8e8b4f700 1 -- 172.21.2.101:0/2797113616 >> v2:172.21.2.108:3300/0 conn(0x7fb8e40d1870 msgr2=0x7fb8e41b03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:50:49.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.523 7fb8e8b4f700 1 --2- 172.21.2.101:0/2797113616 >> v2:172.21.2.108:3300/0 conn(0x7fb8e40d1870 0x7fb8e41b03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:49.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.523 7fb8e8b4f700 1 -- 172.21.2.101:0/2797113616 >> v1:172.21.2.101:6789/0 conn(0x7fb8e40d1c70 legacy=0x7fb8e40cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:49.536 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.523 7fb8e8b4f700 1 -- 172.21.2.101:0/2797113616 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e40f0840 con 0x7fb8e41b0a10 2021-02-10T11:50:49.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.523 7fb8e8b4f700 1 --2- 172.21.2.101:0/2797113616 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8e41b0a10 0x7fb8e40cf330 secure :-1 s=READY pgs=3470 cs=0 l=1 rev1=1 rx=0x7fb8d8006650 tx=0x7fb8d8009a00).ready entity=mon.0 client_cookie=2027dcc83180ce18 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:49.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.524 7fb8d7fff700 1 -- 172.21.2.101:0/2797113616 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb8d80009b0 con 0x7fb8e41b0a10 2021-02-10T11:50:49.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.524 7fb8d7fff700 1 -- 172.21.2.101:0/2797113616 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d800c0c0 con 0x7fb8e41b0a10 2021-02-10T11:50:49.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.524 7fb8d7fff700 1 -- 172.21.2.101:0/2797113616 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb8d800c5a0 con 0x7fb8e41b0a10 2021-02-10T11:50:49.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.524 7fb8eb534700 1 -- 172.21.2.101:0/2797113616 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8e41b0a10 msgr2=0x7fb8e40cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:49.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.524 7fb8eb534700 1 --2- 172.21.2.101:0/2797113616 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8e41b0a10 0x7fb8e40cf330 secure :-1 s=READY pgs=3470 cs=0 l=1 rev1=1 rx=0x7fb8d8006650 tx=0x7fb8d8009a00).stop 2021-02-10T11:50:49.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.524 7fb8eb534700 1 -- 172.21.2.101:0/2797113616 shutdown_connections 2021-02-10T11:50:49.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.524 7fb8eb534700 1 --2- 172.21.2.101:0/2797113616 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8e41b0a10 0x7fb8e40cf330 unknown :-1 s=CLOSED pgs=3470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:49.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.524 7fb8eb534700 1 --2- 172.21.2.101:0/2797113616 >> v2:172.21.2.108:3300/0 conn(0x7fb8e40d1870 0x7fb8e41b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:49.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.524 7fb8eb534700 1 -- 172.21.2.101:0/2797113616 >> 172.21.2.101:0/2797113616 conn(0x7fb8e40ce450 msgr2=0x7fb8e40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:49.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.524 7fb8eb534700 1 -- 172.21.2.101:0/2797113616 shutdown_connections 2021-02-10T11:50:49.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.524 7fb8eb534700 1 -- 172.21.2.101:0/2797113616 wait complete. 2021-02-10T11:50:49.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.539 7fb8eb534700 1 Processor -- start 2021-02-10T11:50:49.538 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.539 7fb8eb534700 1 -- start start 2021-02-10T11:50:49.539 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.539 7fb8eb534700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb8e40bb5d0 0x7fb8e40bfac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:49.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.539 7fb8eb534700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb8e40a9b50 0x7fb8e40a9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:49.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.539 7fb8eb534700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8e41a7550 0x7fb8e41a7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:49.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.539 7fb8eb534700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e40bb9d0 con 0x7fb8e41a7550 2021-02-10T11:50:49.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.539 7fb8eb534700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e40aa570 con 0x7fb8e40bb5d0 2021-02-10T11:50:49.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.539 7fb8eb534700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e40aa790 con 0x7fb8e40a9b50 2021-02-10T11:50:49.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.540 7fb8e9b51700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8e41a7550 0x7fb8e41a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:49.540 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.540 7fb8e9b51700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8e41a7550 0x7fb8e41a7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36352/0 (socket says 172.21.2.101:36352) 2021-02-10T11:50:49.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.540 7fb8e9b51700 1 -- 172.21.2.101:0/24423682 learned_addr learned my addr 172.21.2.101:0/24423682 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:49.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.540 7fb8e9b51700 1 -- 172.21.2.101:0/24423682 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb8e40a9b50 msgr2=0x7fb8e40a9f50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:50:49.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.540 7fb8e9b51700 1 --2- 172.21.2.101:0/24423682 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb8e40a9b50 0x7fb8e40a9f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:49.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.540 7fb8e9b51700 1 -- 172.21.2.101:0/24423682 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb8e40bb5d0 msgr2=0x7fb8e40bfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:49.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.540 7fb8e9b51700 1 --2- 172.21.2.101:0/24423682 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb8e40bb5d0 0x7fb8e40bfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:49.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.540 7fb8e9b51700 1 -- 172.21.2.101:0/24423682 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e41a7f70 con 0x7fb8e41a7550 2021-02-10T11:50:49.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.540 7fb8e9b51700 1 --2- 172.21.2.101:0/24423682 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8e41a7550 0x7fb8e41a7950 secure :-1 s=READY pgs=3471 cs=0 l=1 rev1=1 rx=0x7fb8dc00a900 tx=0x7fb8dc00aca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:49.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.540 7fb8d5ffb700 1 -- 172.21.2.101:0/24423682 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb8dc001270 con 0x7fb8e41a7550 2021-02-10T11:50:49.541 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.540 7fb8d5ffb700 1 -- 172.21.2.101:0/24423682 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8dc007020 con 0x7fb8e41a7550 2021-02-10T11:50:49.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.540 7fb8d5ffb700 1 -- 172.21.2.101:0/24423682 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb8dc007540 con 0x7fb8e41a7550 2021-02-10T11:50:49.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.540 7fb8eb534700 1 -- 172.21.2.101:0/24423682 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e41a7f70 con 0x7fb8e41a7550 2021-02-10T11:50:49.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.541 7fb8eb534700 1 -- 172.21.2.101:0/24423682 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e41a7f70 con 0x7fb8e41a7550 2021-02-10T11:50:49.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.541 7fb8d5ffb700 1 -- 172.21.2.101:0/24423682 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fb8dc007200 con 0x7fb8e41a7550 2021-02-10T11:50:49.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.541 7fb8d5ffb700 1 --2- 172.21.2.101:0/24423682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8c40186c0 0x7fb8c401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:49.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.541 7fb8d5ffb700 1 -- 172.21.2.101:0/24423682 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb8dc0328d0 con 0x7fb8e41a7550 2021-02-10T11:50:49.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.541 7fb8e9350700 1 --2- 172.21.2.101:0/24423682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8c40186c0 0x7fb8c401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:49.542 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.541 7fb8e9350700 1 --2- 172.21.2.101:0/24423682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8c40186c0 0x7fb8c401abc0 crc :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:49.568 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.573 7fb8eb534700 1 -- 172.21.2.101:0/24423682 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e40ee0b0 con 0x7fb8e41a7550 2021-02-10T11:50:49.571 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.575 7fb8d5ffb700 1 -- 172.21.2.101:0/24423682 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fb8dc02ff40 con 0x7fb8e41a7550 2021-02-10T11:50:49.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.702 7fb8eb534700 1 -- 172.21.2.101:0/24423682 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mgr", ""]}) v1 -- 0x7fb8e40ee0b0 con 0x7fb8c40186c0 2021-02-10T11:50:49.698 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.703 7fb8d5ffb700 1 -- 172.21.2.101:0/24423682 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (crc 0 0 0) 0x7fb8e00009b0 con 0x7fb8c40186c0 2021-02-10T11:50:49.699 INFO:teuthology.orchestra.run.gibba001.stderr:instructed osd(s) 3 to deep-scrub 2021-02-10T11:50:49.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.705 7fb8eb534700 1 -- 172.21.2.101:0/24423682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8c40186c0 msgr2=0x7fb8c401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:49.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.705 7fb8eb534700 1 --2- 172.21.2.101:0/24423682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8c40186c0 0x7fb8c401abc0 crc :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:49.700 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.705 7fb8eb534700 1 -- 172.21.2.101:0/24423682 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8e41a7550 msgr2=0x7fb8e41a7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:49.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.705 7fb8eb534700 1 --2- 172.21.2.101:0/24423682 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8e41a7550 0x7fb8e41a7950 secure :-1 s=READY pgs=3471 cs=0 l=1 rev1=1 rx=0x7fb8dc00a900 tx=0x7fb8dc00aca0).stop 2021-02-10T11:50:49.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.705 7fb8eb534700 1 -- 172.21.2.101:0/24423682 shutdown_connections 2021-02-10T11:50:49.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.705 7fb8eb534700 1 --2- 172.21.2.101:0/24423682 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fb8c40186c0 0x7fb8c401abc0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:49.701 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.705 7fb8eb534700 1 --2- 172.21.2.101:0/24423682 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fb8e40a9b50 0x7fb8e40a9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:49.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.705 7fb8eb534700 1 --2- 172.21.2.101:0/24423682 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fb8e41a7550 0x7fb8e41a7950 unknown :-1 s=CLOSED pgs=3471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:49.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.705 7fb8eb534700 1 --2- 172.21.2.101:0/24423682 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fb8e40bb5d0 0x7fb8e40bfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:49.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.705 7fb8eb534700 1 -- 172.21.2.101:0/24423682 >> 172.21.2.101:0/24423682 conn(0x7fb8e40a6eb0 msgr2=0x7fb8e40bf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:49.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.705 7fb8eb534700 1 -- 172.21.2.101:0/24423682 shutdown_connections 2021-02-10T11:50:49.702 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:49.705 7fb8eb534700 1 -- 172.21.2.101:0/24423682 wait complete. 2021-02-10T11:50:49.709 INFO:tasks.ceph:Scrubbing osd.4 2021-02-10T11:50:49.710 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2021-02-10T11:50:50.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.130 7fab75648700 1 Processor -- start 2021-02-10T11:50:50.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.130 7fab75648700 1 -- start start 2021-02-10T11:50:50.142 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.130 7fab75648700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fab700d1880 0x7fab701b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:50.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.130 7fab75648700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fab701b0af0 0x7fab70108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:50.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.130 7fab75648700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab700d1c80 0x7fab70108f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:50.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.130 7fab75648700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab700fe1d0 con 0x7fab700d1880 2021-02-10T11:50:50.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.130 7fab75648700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab70109a70 con 0x7fab701b0af0 2021-02-10T11:50:50.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.130 7fab75648700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab70088070 con 0x7fab700d1c80 2021-02-10T11:50:50.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.130 7fab6f2bf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fab700d1880 0x7fab701b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:50.143 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.130 7fab6f2bf700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fab700d1880 0x7fab701b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36358/0 (socket says 172.21.2.101:36358) 2021-02-10T11:50:50.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.131 7fab6f2bf700 1 -- 172.21.2.101:0/3683335657 learned_addr learned my addr 172.21.2.101:0/3683335657 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:50.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.131 7fab6fac0700 1 --2- 172.21.2.101:0/3683335657 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab700d1c80 0x7fab70108f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:50.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.131 7fab66abe700 1 --2- 172.21.2.101:0/3683335657 >> v2:172.21.2.108:3300/0 conn(0x7fab701b0af0 0x7fab70108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:50.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.131 7fab6f2bf700 1 -- 172.21.2.101:0/3683335657 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab700d1c80 msgr2=0x7fab70108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:50.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.131 7fab6f2bf700 1 --2- 172.21.2.101:0/3683335657 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab700d1c80 0x7fab70108f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:50.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.131 7fab6f2bf700 1 -- 172.21.2.101:0/3683335657 >> v2:172.21.2.108:3300/0 conn(0x7fab701b0af0 msgr2=0x7fab70108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:50.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.131 7fab6f2bf700 1 --2- 172.21.2.101:0/3683335657 >> v2:172.21.2.108:3300/0 conn(0x7fab701b0af0 0x7fab70108950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:50.144 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.131 7fab6f2bf700 1 -- 172.21.2.101:0/3683335657 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab70109590 con 0x7fab700d1880 2021-02-10T11:50:50.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.131 7fab6f2bf700 1 --2- 172.21.2.101:0/3683335657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab700d1880 0x7fab701b04d0 secure :-1 s=READY pgs=3472 cs=0 l=1 rev1=1 rx=0x7fab60006700 tx=0x7fab60009b80).ready entity=mon.0 client_cookie=8830695a23e9d941 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:50.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.131 7fab6eabe700 1 -- 172.21.2.101:0/3683335657 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fab6000c0c0 con 0x7fab700d1880 2021-02-10T11:50:50.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.131 7fab6eabe700 1 -- 172.21.2.101:0/3683335657 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6000c440 con 0x7fab700d1880 2021-02-10T11:50:50.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.131 7fab6eabe700 1 -- 172.21.2.101:0/3683335657 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fab6000c960 con 0x7fab700d1880 2021-02-10T11:50:50.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.131 7fab75648700 1 -- 172.21.2.101:0/3683335657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab700d1880 msgr2=0x7fab701b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:50.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.131 7fab75648700 1 --2- 172.21.2.101:0/3683335657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab700d1880 0x7fab701b04d0 secure :-1 s=READY pgs=3472 cs=0 l=1 rev1=1 rx=0x7fab60006700 tx=0x7fab60009b80).stop 2021-02-10T11:50:50.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.132 7fab75648700 1 -- 172.21.2.101:0/3683335657 shutdown_connections 2021-02-10T11:50:50.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.132 7fab75648700 1 --2- 172.21.2.101:0/3683335657 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab700d1c80 0x7fab70108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:50.145 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.132 7fab75648700 1 --2- 172.21.2.101:0/3683335657 >> v2:172.21.2.108:3300/0 conn(0x7fab701b0af0 0x7fab70108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:50.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.132 7fab75648700 1 --2- 172.21.2.101:0/3683335657 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab700d1880 0x7fab701b04d0 unknown :-1 s=CLOSED pgs=3472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:50.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.132 7fab75648700 1 -- 172.21.2.101:0/3683335657 >> 172.21.2.101:0/3683335657 conn(0x7fab700ce430 msgr2=0x7fab700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:50.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.132 7fab75648700 1 -- 172.21.2.101:0/3683335657 shutdown_connections 2021-02-10T11:50:50.146 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.132 7fab75648700 1 -- 172.21.2.101:0/3683335657 wait complete. 2021-02-10T11:50:50.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.147 7fab75648700 1 Processor -- start 2021-02-10T11:50:50.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.147 7fab75648700 1 -- start start 2021-02-10T11:50:50.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.147 7fab75648700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fab7008a460 0x7fab7008a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:50.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.147 7fab75648700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab7008ae80 0x7fab7008b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:50.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.147 7fab75648700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab700c0ee0 0x7fab700c12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:50.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.147 7fab75648700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab700c2650 con 0x7fab700c0ee0 2021-02-10T11:50:50.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.147 7fab75648700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab70099590 con 0x7fab7008a460 2021-02-10T11:50:50.148 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.147 7fab75648700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab70099770 con 0x7fab7008ae80 2021-02-10T11:50:50.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.148 7fab6fac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab700c0ee0 0x7fab700c12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:50.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.148 7fab6fac0700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab700c0ee0 0x7fab700c12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36364/0 (socket says 172.21.2.101:36364) 2021-02-10T11:50:50.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.148 7fab6fac0700 1 -- 172.21.2.101:0/1798556438 learned_addr learned my addr 172.21.2.101:0/1798556438 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:50.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.148 7fab66abe700 1 --2- 172.21.2.101:0/1798556438 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab7008ae80 0x7fab7008b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:50.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.148 7fab6f2bf700 1 --2- 172.21.2.101:0/1798556438 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fab7008a460 0x7fab7008a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:50.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.148 7fab6fac0700 1 -- 172.21.2.101:0/1798556438 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab7008ae80 msgr2=0x7fab7008b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:50.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.148 7fab6fac0700 1 --2- 172.21.2.101:0/1798556438 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fab7008ae80 0x7fab7008b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:50.149 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.148 7fab6fac0700 1 -- 172.21.2.101:0/1798556438 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fab7008a460 msgr2=0x7fab7008a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:50.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.148 7fab6fac0700 1 --2- 172.21.2.101:0/1798556438 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fab7008a460 0x7fab7008a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:50.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.148 7fab6fac0700 1 -- 172.21.2.101:0/1798556438 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab701a4950 con 0x7fab700c0ee0 2021-02-10T11:50:50.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.148 7fab6f2bf700 1 --2- 172.21.2.101:0/1798556438 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fab7008a460 0x7fab7008a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:50:50.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.148 7fab6fac0700 1 --2- 172.21.2.101:0/1798556438 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab700c0ee0 0x7fab700c12e0 secure :-1 s=READY pgs=3473 cs=0 l=1 rev1=1 rx=0x7fab6800b970 tx=0x7fab6800bd10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:50.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.148 7fab6caba700 1 -- 172.21.2.101:0/1798556438 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fab68000f90 con 0x7fab700c0ee0 2021-02-10T11:50:50.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.148 7fab6caba700 1 -- 172.21.2.101:0/1798556438 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab68001400 con 0x7fab700c0ee0 2021-02-10T11:50:50.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.148 7fab75648700 1 -- 172.21.2.101:0/1798556438 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab701a4950 con 0x7fab700c0ee0 2021-02-10T11:50:50.150 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.148 7fab6caba700 1 -- 172.21.2.101:0/1798556438 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fab680018e0 con 0x7fab700c0ee0 2021-02-10T11:50:50.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.148 7fab75648700 1 -- 172.21.2.101:0/1798556438 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab701a4950 con 0x7fab700c0ee0 2021-02-10T11:50:50.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.149 7fab6caba700 1 -- 172.21.2.101:0/1798556438 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fab68000df0 con 0x7fab700c0ee0 2021-02-10T11:50:50.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.149 7fab6caba700 1 --2- 172.21.2.101:0/1798556438 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab540186c0 0x7fab5401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:50.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.149 7fab6caba700 1 -- 172.21.2.101:0/1798556438 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fab680329a0 con 0x7fab700c0ee0 2021-02-10T11:50:50.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.149 7fab6f2bf700 1 --2- 172.21.2.101:0/1798556438 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab540186c0 0x7fab5401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:50.151 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.149 7fab6f2bf700 1 --2- 172.21.2.101:0/1798556438 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab540186c0 0x7fab5401abc0 crc :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:50.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.180 7fab75648700 1 --2- 172.21.2.101:0/1798556438 >> [v2:172.21.2.108:6816/36380,v1:172.21.2.108:6818/36380] conn(0x7fab700c4620 0x7fab700627c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:50.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.180 7fab75648700 1 -- 172.21.2.101:0/1798556438 --> [v2:172.21.2.108:6816/36380,v1:172.21.2.108:6818/36380] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab70099590 con 0x7fab700c4620 2021-02-10T11:50:50.176 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.181 7fab66abe700 1 --2- 172.21.2.101:0/1798556438 >> [v2:172.21.2.108:6816/36380,v1:172.21.2.108:6818/36380] conn(0x7fab700c4620 0x7fab700627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:50.177 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.182 7fab66abe700 1 --2- 172.21.2.101:0/1798556438 >> [v2:172.21.2.108:6816/36380,v1:172.21.2.108:6818/36380] conn(0x7fab700c4620 0x7fab700627c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:50.178 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.182 7fab6caba700 1 -- 172.21.2.101:0/1798556438 <== osd.4 v2:172.21.2.108:6816/36380 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (crc 0 0 0) 0x7fab58000940 con 0x7fab700c4620 2021-02-10T11:50:50.184 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.189 7fab75648700 1 -- 172.21.2.101:0/1798556438 --> [v2:172.21.2.108:6816/36380,v1:172.21.2.108:6818/36380] -- command(tid 2: {"prefix": "config set", "key": "osd_debug_deep_scrub_sleep", "value": "0"}) v1 -- 0x7fab70082aa0 con 0x7fab700c4620 2021-02-10T11:50:50.185 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.189 7fab6caba700 1 -- 172.21.2.101:0/1798556438 <== osd.4 v2:172.21.2.108:6816/36380 2 ==== command_reply(tid 2: 0 Set osd_debug_deep_scrub_sleep to 0) v1 ==== 43+0+0 (crc 0 0 0) 0x7fab58000940 con 0x7fab700c4620 2021-02-10T11:50:50.186 INFO:teuthology.orchestra.run.gibba001.stderr:Set osd_debug_deep_scrub_sleep to 0 2021-02-10T11:50:50.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.190 7fab75648700 1 -- 172.21.2.101:0/1798556438 >> [v2:172.21.2.108:6816/36380,v1:172.21.2.108:6818/36380] conn(0x7fab700c4620 msgr2=0x7fab700627c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:50.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.190 7fab75648700 1 --2- 172.21.2.101:0/1798556438 >> [v2:172.21.2.108:6816/36380,v1:172.21.2.108:6818/36380] conn(0x7fab700c4620 0x7fab700627c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:50.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.191 7fab75648700 1 -- 172.21.2.101:0/1798556438 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab540186c0 msgr2=0x7fab5401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:50.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.191 7fab75648700 1 --2- 172.21.2.101:0/1798556438 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fab540186c0 0x7fab5401abc0 crc :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:50.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.191 7fab75648700 1 -- 172.21.2.101:0/1798556438 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab700c0ee0 msgr2=0x7fab700c12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:50.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.191 7fab75648700 1 --2- 172.21.2.101:0/1798556438 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fab700c0ee0 0x7fab700c12e0 secure :-1 s=READY pgs=3473 cs=0 l=1 rev1=1 rx=0x7fab6800b970 tx=0x7fab6800bd10).stop 2021-02-10T11:50:50.187 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.191 7fab6fac0700 1 -- 172.21.2.101:0/1798556438 reap_dead start 2021-02-10T11:50:50.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.191 7fab75648700 1 -- 172.21.2.101:0/1798556438 shutdown_connections 2021-02-10T11:50:50.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.191 7fab75648700 1 -- 172.21.2.101:0/1798556438 >> 172.21.2.101:0/1798556438 conn(0x7fab700a1770 msgr2=0x7fab7009e210 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:50.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.191 7fab75648700 1 -- 172.21.2.101:0/1798556438 shutdown_connections 2021-02-10T11:50:50.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.191 7fab75648700 1 -- 172.21.2.101:0/1798556438 wait complete. 2021-02-10T11:50:50.195 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd deep-scrub 4 2021-02-10T11:50:50.623 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.611 7f96c6ec8700 1 Processor -- start 2021-02-10T11:50:50.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.611 7f96c6ec8700 1 -- start start 2021-02-10T11:50:50.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.612 7f96c6ec8700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f96c00d1870 0x7f96c01b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:50.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.612 7f96c6ec8700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f96c00f0650 0x7f96c00cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:50.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.612 7f96c6ec8700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c00f0b10 con 0x7f96c01b0a10 2021-02-10T11:50:50.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.612 7f96c6ec8700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c00e5ed0 con 0x7f96c00f0650 2021-02-10T11:50:50.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.612 7f96c6ec8700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c00d1c70 con 0x7f96c00d1870 2021-02-10T11:50:50.624 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.612 7f96bffff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f96c01b0a10 0x7f96c00cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36864/0 (socket says 172.21.2.101:36864) 2021-02-10T11:50:50.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.612 7f96bffff700 1 -- 172.21.2.101:0/985847426 learned_addr learned my addr 172.21.2.101:0/985847426 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:50:50.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.612 7f96c54e5700 1 --2- 172.21.2.101:0/985847426 >> v2:172.21.2.101:3300/0 conn(0x7f96c00f0650 0x7f96c00cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:50.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.612 7f96bf7fe700 1 -- 172.21.2.101:0/985847426 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 413368679 0 0) 0x7f96b0001180 con 0x7f96c01b0a10 2021-02-10T11:50:50.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.612 7f96bf7fe700 1 -- 172.21.2.101:0/985847426 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96ac002fe0 con 0x7f96c01b0a10 2021-02-10T11:50:50.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.612 7f96c4ce4700 1 --2- 172.21.2.101:0/985847426 >> v2:172.21.2.108:3300/0 conn(0x7f96c00d1870 0x7f96c01b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:50.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.612 7f96c54e5700 1 -- 172.21.2.101:0/985847426 >> v2:172.21.2.108:3300/0 conn(0x7f96c00d1870 msgr2=0x7f96c01b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:50.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.612 7f96c54e5700 1 --2- 172.21.2.101:0/985847426 >> v2:172.21.2.108:3300/0 conn(0x7f96c00d1870 0x7f96c01b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:50.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.612 7f96c54e5700 1 -- 172.21.2.101:0/985847426 >> v1:172.21.2.101:6789/0 conn(0x7f96c01b0a10 legacy=0x7f96c00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:50.625 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.612 7f96c54e5700 1 -- 172.21.2.101:0/985847426 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c00d1e90 con 0x7f96c00f0650 2021-02-10T11:50:50.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.612 7f96c4ce4700 1 --2- 172.21.2.101:0/985847426 >> v2:172.21.2.108:3300/0 conn(0x7f96c00d1870 0x7f96c01b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-02-10T11:50:50.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.613 7f96c54e5700 1 --2- 172.21.2.101:0/985847426 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96c00f0650 0x7f96c00cfb20 secure :-1 s=READY pgs=3475 cs=0 l=1 rev1=1 rx=0x7f96b8006e30 tx=0x7f96b8009e60).ready entity=mon.0 client_cookie=c01ab7d5b3ceb9c1 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:50.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.613 7f96bf7fe700 1 -- 172.21.2.101:0/985847426 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96b8006a50 con 0x7f96c00f0650 2021-02-10T11:50:50.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.613 7f96bf7fe700 1 -- 172.21.2.101:0/985847426 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b800b300 con 0x7f96c00f0650 2021-02-10T11:50:50.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.613 7f96bf7fe700 1 -- 172.21.2.101:0/985847426 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96b800b820 con 0x7f96c00f0650 2021-02-10T11:50:50.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.613 7f96c6ec8700 1 -- 172.21.2.101:0/985847426 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96c00f0650 msgr2=0x7f96c00cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:50.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.613 7f96c6ec8700 1 --2- 172.21.2.101:0/985847426 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96c00f0650 0x7f96c00cfb20 secure :-1 s=READY pgs=3475 cs=0 l=1 rev1=1 rx=0x7f96b8006e30 tx=0x7f96b8009e60).stop 2021-02-10T11:50:50.626 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.613 7f96c6ec8700 1 -- 172.21.2.101:0/985847426 shutdown_connections 2021-02-10T11:50:50.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.613 7f96c6ec8700 1 --2- 172.21.2.101:0/985847426 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96c00f0650 0x7f96c00cfb20 unknown :-1 s=CLOSED pgs=3475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:50.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.613 7f96c6ec8700 1 --2- 172.21.2.101:0/985847426 >> v2:172.21.2.108:3300/0 conn(0x7f96c00d1870 0x7f96c01b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:50.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.613 7f96c6ec8700 1 -- 172.21.2.101:0/985847426 >> 172.21.2.101:0/985847426 conn(0x7f96c00ce450 msgr2=0x7f96c00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:50.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.613 7f96c6ec8700 1 -- 172.21.2.101:0/985847426 shutdown_connections 2021-02-10T11:50:50.627 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.613 7f96c6ec8700 1 -- 172.21.2.101:0/985847426 wait complete. 2021-02-10T11:50:50.628 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.628 7f96c6ec8700 1 Processor -- start 2021-02-10T11:50:50.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.628 7f96c6ec8700 1 -- start start 2021-02-10T11:50:50.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.629 7f96c6ec8700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96c01a06a0 0x7f96c009f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:50.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.629 7f96c6ec8700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96c009f9a0 0x7f96c008b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:50.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.629 7f96c6ec8700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96c008bb80 0x7f96c008bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:50.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.629 7f96c6ec8700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c008c780 con 0x7f96c01a06a0 2021-02-10T11:50:50.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.629 7f96c6ec8700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c009fda0 con 0x7f96c009f9a0 2021-02-10T11:50:50.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.629 7f96c6ec8700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c008cd00 con 0x7f96c008bb80 2021-02-10T11:50:50.629 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.629 7f96c4ce4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96c01a06a0 0x7f96c009f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:50.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.629 7f96c4ce4700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96c01a06a0 0x7f96c009f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36376/0 (socket says 172.21.2.101:36376) 2021-02-10T11:50:50.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.629 7f96c4ce4700 1 -- 172.21.2.101:0/338070366 learned_addr learned my addr 172.21.2.101:0/338070366 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:50.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.629 7f96c4ce4700 1 -- 172.21.2.101:0/338070366 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96c008bb80 msgr2=0x7f96c008bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:50.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.629 7f96c4ce4700 1 --2- 172.21.2.101:0/338070366 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96c008bb80 0x7f96c008bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:50.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.629 7f96c4ce4700 1 -- 172.21.2.101:0/338070366 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96c009f9a0 msgr2=0x7f96c008b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:50.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.629 7f96c4ce4700 1 --2- 172.21.2.101:0/338070366 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96c009f9a0 0x7f96c008b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:50.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.629 7f96c4ce4700 1 -- 172.21.2.101:0/338070366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c008cf20 con 0x7f96c01a06a0 2021-02-10T11:50:50.630 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.629 7f96c4ce4700 1 --2- 172.21.2.101:0/338070366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96c01a06a0 0x7f96c009f380 secure :-1 s=READY pgs=3476 cs=0 l=1 rev1=1 rx=0x7f96a8006f20 tx=0x7f96a800b990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:50.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.629 7f96bd7fa700 1 -- 172.21.2.101:0/338070366 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96a800bd80 con 0x7f96c01a06a0 2021-02-10T11:50:50.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.629 7f96bd7fa700 1 -- 172.21.2.101:0/338070366 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a8000c90 con 0x7f96c01a06a0 2021-02-10T11:50:50.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.629 7f96c6ec8700 1 -- 172.21.2.101:0/338070366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c008cf20 con 0x7f96c01a06a0 2021-02-10T11:50:50.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.629 7f96bd7fa700 1 -- 172.21.2.101:0/338070366 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96a80022b0 con 0x7f96c01a06a0 2021-02-10T11:50:50.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.629 7f96c6ec8700 1 -- 172.21.2.101:0/338070366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c008d4b0 con 0x7f96c01a06a0 2021-02-10T11:50:50.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.630 7f96bd7fa700 1 -- 172.21.2.101:0/338070366 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f96a800a000 con 0x7f96c01a06a0 2021-02-10T11:50:50.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.630 7f96bd7fa700 1 --2- 172.21.2.101:0/338070366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96a40186c0 0x7f96a401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:50.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.630 7f96bd7fa700 1 -- 172.21.2.101:0/338070366 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f96a8032410 con 0x7f96c01a06a0 2021-02-10T11:50:50.631 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.630 7f96bffff700 1 --2- 172.21.2.101:0/338070366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96a40186c0 0x7f96a401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:50.632 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.630 7f96bffff700 1 --2- 172.21.2.101:0/338070366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96a40186c0 0x7f96a401abc0 crc :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:50.657 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.662 7f96c6ec8700 1 -- 172.21.2.101:0/338070366 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c00ebc50 con 0x7f96c01a06a0 2021-02-10T11:50:50.660 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.664 7f96bd7fa700 1 -- 172.21.2.101:0/338070366 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f96a80009b0 con 0x7f96c01a06a0 2021-02-10T11:50:50.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.793 7f96c6ec8700 1 -- 172.21.2.101:0/338070366 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mgr", ""]}) v1 -- 0x7f96c008d4b0 con 0x7f96a40186c0 2021-02-10T11:50:50.789 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.793 7f96bd7fa700 1 -- 172.21.2.101:0/338070366 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (crc 0 0 0) 0x7f96b0000ce0 con 0x7f96a40186c0 2021-02-10T11:50:50.790 INFO:teuthology.orchestra.run.gibba001.stderr:instructed osd(s) 4 to deep-scrub 2021-02-10T11:50:50.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.795 7f96c6ec8700 1 -- 172.21.2.101:0/338070366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96a40186c0 msgr2=0x7f96a401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:50.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.795 7f96c6ec8700 1 --2- 172.21.2.101:0/338070366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96a40186c0 0x7f96a401abc0 crc :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:50.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.795 7f96c6ec8700 1 -- 172.21.2.101:0/338070366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96c01a06a0 msgr2=0x7f96c009f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:50.791 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.795 7f96c6ec8700 1 --2- 172.21.2.101:0/338070366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96c01a06a0 0x7f96c009f380 secure :-1 s=READY pgs=3476 cs=0 l=1 rev1=1 rx=0x7f96a8006f20 tx=0x7f96a800b990).stop 2021-02-10T11:50:50.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.796 7f96c6ec8700 1 -- 172.21.2.101:0/338070366 shutdown_connections 2021-02-10T11:50:50.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.796 7f96c6ec8700 1 --2- 172.21.2.101:0/338070366 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f96a40186c0 0x7f96a401abc0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:50.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.796 7f96c6ec8700 1 --2- 172.21.2.101:0/338070366 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f96c008bb80 0x7f96c008bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:50.792 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.796 7f96c6ec8700 1 --2- 172.21.2.101:0/338070366 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f96c009f9a0 0x7f96c008b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:50.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.796 7f96c6ec8700 1 --2- 172.21.2.101:0/338070366 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f96c01a06a0 0x7f96c009f380 unknown :-1 s=CLOSED pgs=3476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:50.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.796 7f96c6ec8700 1 -- 172.21.2.101:0/338070366 >> 172.21.2.101:0/338070366 conn(0x7f96c00b83b0 msgr2=0x7f96c00a6850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:50.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.796 7f96c6ec8700 1 -- 172.21.2.101:0/338070366 shutdown_connections 2021-02-10T11:50:50.793 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:50.796 7f96c6ec8700 1 -- 172.21.2.101:0/338070366 wait complete. 2021-02-10T11:50:50.800 INFO:tasks.ceph:Scrubbing osd.5 2021-02-10T11:50:50.800 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2021-02-10T11:50:51.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.219 7f1bdcd2f700 1 Processor -- start 2021-02-10T11:50:51.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.219 7f1bdcd2f700 1 -- start start 2021-02-10T11:50:51.231 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.220 7f1bdcd2f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1bd80d1880 0x7f1bd81b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:51.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.220 7f1bdcd2f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1bd80cff80 0x7f1bd8109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:51.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.220 7f1bdcd2f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bd8109a70 con 0x7f1bd81b0af0 2021-02-10T11:50:51.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.220 7f1bdcd2f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bd8109760 con 0x7f1bd80cff80 2021-02-10T11:50:51.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.220 7f1bdcd2f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd8088070 con 0x7f1bd80d1880 2021-02-10T11:50:51.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.220 7f1bd7314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1bd80cff80 0x7f1bd8109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:51.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.220 7f1bd7314700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f1bd80cff80 0x7f1bd8109140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36384/0 (socket says 172.21.2.101:36384) 2021-02-10T11:50:51.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.220 7f1bd7314700 1 -- 172.21.2.101:0/799709103 learned_addr learned my addr 172.21.2.101:0/799709103 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:51.232 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.220 7f1bd7314700 1 -- 172.21.2.101:0/799709103 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1bd80d1880 msgr2=0x7f1bd81b04d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:50:51.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.220 7f1bd7314700 1 --2- 172.21.2.101:0/799709103 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1bd80d1880 0x7f1bd81b04d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:51.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.220 7f1bd7314700 1 -- 172.21.2.101:0/799709103 >> v1:172.21.2.108:6789/0 conn(0x7f1bd81b0af0 legacy=0x7f1bd8108950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:50:51.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.220 7f1bd7314700 1 -- 172.21.2.101:0/799709103 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd8088260 con 0x7f1bd80cff80 2021-02-10T11:50:51.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.220 7f1bd7314700 1 --2- 172.21.2.101:0/799709103 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1bd80cff80 0x7f1bd8109140 secure :-1 s=READY pgs=3477 cs=0 l=1 rev1=1 rx=0x7f1bd0006f00 tx=0x7f1bd0009f10).ready entity=mon.0 client_cookie=83b30b5630909b7e server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:51.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.220 7f1bd5b11700 1 -- 172.21.2.101:0/799709103 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1bd0003030 con 0x7f1bd80cff80 2021-02-10T11:50:51.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.221 7f1bd5b11700 1 -- 172.21.2.101:0/799709103 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd000b3b0 con 0x7f1bd80cff80 2021-02-10T11:50:51.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.221 7f1bd5b11700 1 -- 172.21.2.101:0/799709103 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1bd000b850 con 0x7f1bd80cff80 2021-02-10T11:50:51.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.221 7f1bdcd2f700 1 -- 172.21.2.101:0/799709103 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1bd80cff80 msgr2=0x7f1bd8109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:51.233 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.221 7f1bdcd2f700 1 --2- 172.21.2.101:0/799709103 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1bd80cff80 0x7f1bd8109140 secure :-1 s=READY pgs=3477 cs=0 l=1 rev1=1 rx=0x7f1bd0006f00 tx=0x7f1bd0009f10).stop 2021-02-10T11:50:51.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.221 7f1bdcd2f700 1 -- 172.21.2.101:0/799709103 shutdown_connections 2021-02-10T11:50:51.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.221 7f1bdcd2f700 1 --2- 172.21.2.101:0/799709103 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1bd80cff80 0x7f1bd8109140 unknown :-1 s=CLOSED pgs=3477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:51.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.221 7f1bdcd2f700 1 --2- 172.21.2.101:0/799709103 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1bd80d1880 0x7f1bd81b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:51.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.221 7f1bdcd2f700 1 -- 172.21.2.101:0/799709103 >> 172.21.2.101:0/799709103 conn(0x7f1bd80ce430 msgr2=0x7f1bd80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:51.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.221 7f1bdcd2f700 1 -- 172.21.2.101:0/799709103 shutdown_connections 2021-02-10T11:50:51.234 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.221 7f1bdcd2f700 1 -- 172.21.2.101:0/799709103 wait complete. 2021-02-10T11:50:51.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.235 7f1bdcd2f700 1 Processor -- start 2021-02-10T11:50:51.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.235 7f1bdcd2f700 1 -- start start 2021-02-10T11:50:51.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.236 7f1bdcd2f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1bd818cb30 0x7f1bd81a7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:51.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.236 7f1bdcd2f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1bd80b4eb0 0x7f1bd81a8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:51.236 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.236 7f1bdcd2f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1bd81a8980 0x7f1bd81a9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:51.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.236 7f1bdcd2f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd809a5f0 con 0x7f1bd81a8980 2021-02-10T11:50:51.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.236 7f1bdcd2f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd81a8d80 con 0x7f1bd818cb30 2021-02-10T11:50:51.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.236 7f1bdcd2f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd81a8fa0 con 0x7f1bd80b4eb0 2021-02-10T11:50:51.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.236 7f1bd7314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1bd81a8980 0x7f1bd81a9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:51.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.236 7f1bd7314700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1bd81a8980 0x7f1bd81a9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36390/0 (socket says 172.21.2.101:36390) 2021-02-10T11:50:51.237 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.236 7f1bd7314700 1 -- 172.21.2.101:0/4034182882 learned_addr learned my addr 172.21.2.101:0/4034182882 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:51.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.236 7f1bd7314700 1 -- 172.21.2.101:0/4034182882 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1bd80b4eb0 msgr2=0x7f1bd81a8360 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:50:51.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.236 7f1bd7314700 1 --2- 172.21.2.101:0/4034182882 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f1bd80b4eb0 0x7f1bd81a8360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:51.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.236 7f1bd7314700 1 -- 172.21.2.101:0/4034182882 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1bd818cb30 msgr2=0x7f1bd81a7d40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:50:51.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.236 7f1bd7314700 1 --2- 172.21.2.101:0/4034182882 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f1bd818cb30 0x7f1bd81a7d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:51.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.236 7f1bd7314700 1 -- 172.21.2.101:0/4034182882 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd81a9e40 con 0x7f1bd81a8980 2021-02-10T11:50:51.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.236 7f1bd7314700 1 --2- 172.21.2.101:0/4034182882 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1bd81a8980 0x7f1bd81a9820 secure :-1 s=READY pgs=3478 cs=0 l=1 rev1=1 rx=0x7f1bd0009310 tx=0x7f1bd000bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:51.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.237 7f1bcf7fe700 1 -- 172.21.2.101:0/4034182882 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1bd0009820 con 0x7f1bd81a8980 2021-02-10T11:50:51.238 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.237 7f1bcf7fe700 1 -- 172.21.2.101:0/4034182882 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd000c2f0 con 0x7f1bd81a8980 2021-02-10T11:50:51.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.237 7f1bdcd2f700 1 -- 172.21.2.101:0/4034182882 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bd81a9e40 con 0x7f1bd81a8980 2021-02-10T11:50:51.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.237 7f1bcf7fe700 1 -- 172.21.2.101:0/4034182882 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1bd000c7d0 con 0x7f1bd81a8980 2021-02-10T11:50:51.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.237 7f1bdcd2f700 1 -- 172.21.2.101:0/4034182882 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bd81aa3d0 con 0x7f1bd81a8980 2021-02-10T11:50:51.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.237 7f1bcf7fe700 1 -- 172.21.2.101:0/4034182882 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f1bd0001270 con 0x7f1bd81a8980 2021-02-10T11:50:51.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.237 7f1bcf7fe700 1 --2- 172.21.2.101:0/4034182882 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1bbc0186c0 0x7f1bbc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:51.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.237 7f1bcf7fe700 1 -- 172.21.2.101:0/4034182882 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1bd0032790 con 0x7f1bd81a8980 2021-02-10T11:50:51.239 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.238 7f1bd6b13700 1 --2- 172.21.2.101:0/4034182882 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1bbc0186c0 0x7f1bbc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:51.240 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.238 7f1bd6b13700 1 --2- 172.21.2.101:0/4034182882 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1bbc0186c0 0x7f1bbc01abc0 crc :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:51.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.269 7f1bdcd2f700 1 --2- 172.21.2.101:0/4034182882 >> [v2:172.21.2.108:6808/36383,v1:172.21.2.108:6809/36383] conn(0x7f1bd80627c0 0x7f1bd8062bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:51.265 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.269 7f1bdcd2f700 1 -- 172.21.2.101:0/4034182882 --> [v2:172.21.2.108:6808/36383,v1:172.21.2.108:6809/36383] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1bd81aa3d0 con 0x7f1bd80627c0 2021-02-10T11:50:51.266 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.271 7f1bd6312700 1 --2- 172.21.2.101:0/4034182882 >> [v2:172.21.2.108:6808/36383,v1:172.21.2.108:6809/36383] conn(0x7f1bd80627c0 0x7f1bd8062bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:51.267 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.271 7f1bd6312700 1 --2- 172.21.2.101:0/4034182882 >> [v2:172.21.2.108:6808/36383,v1:172.21.2.108:6809/36383] conn(0x7f1bd80627c0 0x7f1bd8062bc0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:51.268 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.272 7f1bcf7fe700 1 -- 172.21.2.101:0/4034182882 <== osd.5 v2:172.21.2.108:6808/36383 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (crc 0 0 0) 0x7f1bc0006970 con 0x7f1bd80627c0 2021-02-10T11:50:51.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.278 7f1bdcd2f700 1 -- 172.21.2.101:0/4034182882 --> [v2:172.21.2.108:6808/36383,v1:172.21.2.108:6809/36383] -- command(tid 2: {"prefix": "config set", "key": "osd_debug_deep_scrub_sleep", "value": "0"}) v1 -- 0x7f1bd81a9e40 con 0x7f1bd80627c0 2021-02-10T11:50:51.274 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.278 7f1bcf7fe700 1 -- 172.21.2.101:0/4034182882 <== osd.5 v2:172.21.2.108:6808/36383 2 ==== command_reply(tid 2: 0 Set osd_debug_deep_scrub_sleep to 0) v1 ==== 43+0+0 (crc 0 0 0) 0x7f1bc00069f0 con 0x7f1bd80627c0 2021-02-10T11:50:51.275 INFO:teuthology.orchestra.run.gibba001.stderr:Set osd_debug_deep_scrub_sleep to 0 2021-02-10T11:50:51.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.280 7f1bdcd2f700 1 -- 172.21.2.101:0/4034182882 >> [v2:172.21.2.108:6808/36383,v1:172.21.2.108:6809/36383] conn(0x7f1bd80627c0 msgr2=0x7f1bd8062bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:51.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.280 7f1bdcd2f700 1 --2- 172.21.2.101:0/4034182882 >> [v2:172.21.2.108:6808/36383,v1:172.21.2.108:6809/36383] conn(0x7f1bd80627c0 0x7f1bd8062bc0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:51.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.280 7f1bdcd2f700 1 -- 172.21.2.101:0/4034182882 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1bbc0186c0 msgr2=0x7f1bbc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:51.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.280 7f1bdcd2f700 1 --2- 172.21.2.101:0/4034182882 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f1bbc0186c0 0x7f1bbc01abc0 crc :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:51.276 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.280 7f1bdcd2f700 1 -- 172.21.2.101:0/4034182882 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1bd81a8980 msgr2=0x7f1bd81a9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:51.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.280 7f1bdcd2f700 1 --2- 172.21.2.101:0/4034182882 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f1bd81a8980 0x7f1bd81a9820 secure :-1 s=READY pgs=3478 cs=0 l=1 rev1=1 rx=0x7f1bd0009310 tx=0x7f1bd000bd40).stop 2021-02-10T11:50:51.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.280 7f1bd7314700 1 -- 172.21.2.101:0/4034182882 reap_dead start 2021-02-10T11:50:51.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.280 7f1bdcd2f700 1 -- 172.21.2.101:0/4034182882 shutdown_connections 2021-02-10T11:50:51.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.280 7f1bdcd2f700 1 -- 172.21.2.101:0/4034182882 >> 172.21.2.101:0/4034182882 conn(0x7f1bd819f940 msgr2=0x7f1bd8086040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:51.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.280 7f1bdcd2f700 1 -- 172.21.2.101:0/4034182882 shutdown_connections 2021-02-10T11:50:51.277 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.280 7f1bdcd2f700 1 -- 172.21.2.101:0/4034182882 wait complete. 2021-02-10T11:50:51.284 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd deep-scrub 5 2021-02-10T11:50:51.713 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.701 7f2d42c4c700 1 Processor -- start 2021-02-10T11:50:51.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.701 7f2d42c4c700 1 -- start start 2021-02-10T11:50:51.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.701 7f2d42c4c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2d3c0d1860 0x7f2d3c1b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:51.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.701 7f2d42c4c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d3c0f0640 0x7f2d3c0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:51.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.701 7f2d42c4c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d3c0f0b00 con 0x7f2d3c1b0a00 2021-02-10T11:50:51.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.701 7f2d42c4c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d3c0e5ec0 con 0x7f2d3c0d1860 2021-02-10T11:50:51.714 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.701 7f2d42c4c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c0d1c60 con 0x7f2d3c0f0640 2021-02-10T11:50:51.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.701 7f2d40a68700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f2d3c0d1860 0x7f2d3c1b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:51.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.701 7f2d3bfff700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f2d3c1b0a00 0x7f2d3c0cf320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36892/0 (socket says 172.21.2.101:36892) 2021-02-10T11:50:51.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.702 7f2d3bfff700 1 -- 172.21.2.101:0/2259546745 learned_addr learned my addr 172.21.2.101:0/2259546745 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:50:51.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.702 7f2d41269700 1 --2- 172.21.2.101:0/2259546745 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d3c0f0640 0x7f2d3c0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:51.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.702 7f2d40a68700 1 -- 172.21.2.101:0/2259546745 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d3c0f0640 msgr2=0x7f2d3c0cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:51.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.702 7f2d40a68700 1 --2- 172.21.2.101:0/2259546745 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d3c0f0640 0x7f2d3c0cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:51.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.702 7f2d40a68700 1 -- 172.21.2.101:0/2259546745 >> v1:172.21.2.101:6789/0 conn(0x7f2d3c1b0a00 legacy=0x7f2d3c0cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:51.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.702 7f2d3b7fe700 1 -- 172.21.2.101:0/2259546745 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610545332 0 0) 0x7f2d2c001180 con 0x7f2d3c1b0a00 2021-02-10T11:50:51.715 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.702 7f2d40a68700 1 -- 172.21.2.101:0/2259546745 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3c0d1e80 con 0x7f2d3c0d1860 2021-02-10T11:50:51.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.702 7f2d40a68700 1 --2- 172.21.2.101:0/2259546745 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d3c0d1860 0x7f2d3c1b03e0 secure :-1 s=READY pgs=3480 cs=0 l=1 rev1=1 rx=0x7f2d24006650 tx=0x7f2d24009a80).ready entity=mon.0 client_cookie=c05fff1b3b4de49f server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:51.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.702 7f2d3b7fe700 1 -- 172.21.2.101:0/2259546745 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d2400c0c0 con 0x7f2d3c0d1860 2021-02-10T11:50:51.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.702 7f2d3b7fe700 1 -- 172.21.2.101:0/2259546745 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2400c340 con 0x7f2d3c0d1860 2021-02-10T11:50:51.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.702 7f2d3b7fe700 1 -- 172.21.2.101:0/2259546745 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d2400c860 con 0x7f2d3c0d1860 2021-02-10T11:50:51.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.703 7f2d42c4c700 1 -- 172.21.2.101:0/2259546745 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d3c0d1860 msgr2=0x7f2d3c1b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:51.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.703 7f2d42c4c700 1 --2- 172.21.2.101:0/2259546745 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d3c0d1860 0x7f2d3c1b03e0 secure :-1 s=READY pgs=3480 cs=0 l=1 rev1=1 rx=0x7f2d24006650 tx=0x7f2d24009a80).stop 2021-02-10T11:50:51.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.703 7f2d42c4c700 1 -- 172.21.2.101:0/2259546745 shutdown_connections 2021-02-10T11:50:51.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.703 7f2d42c4c700 1 --2- 172.21.2.101:0/2259546745 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d3c0f0640 0x7f2d3c0cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:51.716 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.703 7f2d42c4c700 1 --2- 172.21.2.101:0/2259546745 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d3c0d1860 0x7f2d3c1b03e0 unknown :-1 s=CLOSED pgs=3480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:51.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.703 7f2d42c4c700 1 -- 172.21.2.101:0/2259546745 >> 172.21.2.101:0/2259546745 conn(0x7f2d3c0ce440 msgr2=0x7f2d3c0d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:51.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.703 7f2d42c4c700 1 -- 172.21.2.101:0/2259546745 shutdown_connections 2021-02-10T11:50:51.717 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.703 7f2d42c4c700 1 -- 172.21.2.101:0/2259546745 wait complete. 2021-02-10T11:50:51.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.718 7f2d42c4c700 1 Processor -- start 2021-02-10T11:50:51.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.718 7f2d42c4c700 1 -- start start 2021-02-10T11:50:51.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.719 7f2d42c4c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d3c0b4550 0x7f2d3c0c3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:51.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.719 7f2d42c4c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d3c0b4950 0x7f2d3c0c08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:51.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.719 7f2d42c4c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d3c0c0ee0 0x7f2d3c0b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:51.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.719 7f2d42c4c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c0b5990 con 0x7f2d3c0b4550 2021-02-10T11:50:51.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.719 7f2d42c4c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c0b5b80 con 0x7f2d3c0b4950 2021-02-10T11:50:51.719 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.719 7f2d42c4c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c0b5da0 con 0x7f2d3c0c0ee0 2021-02-10T11:50:51.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.719 7f2d40a68700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d3c0b4550 0x7f2d3c0c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:51.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.719 7f2d40a68700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d3c0b4550 0x7f2d3c0c3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36406/0 (socket says 172.21.2.101:36406) 2021-02-10T11:50:51.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.719 7f2d40a68700 1 -- 172.21.2.101:0/2301618667 learned_addr learned my addr 172.21.2.101:0/2301618667 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:51.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.719 7f2d41269700 1 --2- 172.21.2.101:0/2301618667 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d3c0c0ee0 0x7f2d3c0b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:51.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.719 7f2d3bfff700 1 --2- 172.21.2.101:0/2301618667 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d3c0b4950 0x7f2d3c0c08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:51.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.719 7f2d40a68700 1 -- 172.21.2.101:0/2301618667 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d3c0c0ee0 msgr2=0x7f2d3c0b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:51.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.719 7f2d40a68700 1 --2- 172.21.2.101:0/2301618667 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d3c0c0ee0 0x7f2d3c0b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:51.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.719 7f2d40a68700 1 -- 172.21.2.101:0/2301618667 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d3c0b4950 msgr2=0x7f2d3c0c08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:51.720 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.719 7f2d40a68700 1 --2- 172.21.2.101:0/2301618667 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d3c0b4950 0x7f2d3c0c08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:51.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.719 7f2d40a68700 1 -- 172.21.2.101:0/2301618667 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3c0b5fc0 con 0x7f2d3c0b4550 2021-02-10T11:50:51.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.720 7f2d40a68700 1 --2- 172.21.2.101:0/2301618667 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d3c0b4550 0x7f2d3c0c3450 secure :-1 s=READY pgs=3481 cs=0 l=1 rev1=1 rx=0x7f2d24009630 tx=0x7f2d2400b980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:51.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.720 7f2d397fa700 1 -- 172.21.2.101:0/2301618667 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d240093a0 con 0x7f2d3c0b4550 2021-02-10T11:50:51.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.720 7f2d397fa700 1 -- 172.21.2.101:0/2301618667 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2400bd70 con 0x7f2d3c0b4550 2021-02-10T11:50:51.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.720 7f2d42c4c700 1 -- 172.21.2.101:0/2301618667 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d3c0b5fc0 con 0x7f2d3c0b4550 2021-02-10T11:50:51.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.720 7f2d42c4c700 1 -- 172.21.2.101:0/2301618667 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d3c0b6550 con 0x7f2d3c0b4550 2021-02-10T11:50:51.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.720 7f2d397fa700 1 -- 172.21.2.101:0/2301618667 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2d24005560 con 0x7f2d3c0b4550 2021-02-10T11:50:51.721 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.721 7f2d397fa700 1 -- 172.21.2.101:0/2301618667 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f2d2400a000 con 0x7f2d3c0b4550 2021-02-10T11:50:51.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.721 7f2d397fa700 1 --2- 172.21.2.101:0/2301618667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d200186c0 0x7f2d2001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:51.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.721 7f2d397fa700 1 -- 172.21.2.101:0/2301618667 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2d240325b0 con 0x7f2d3c0b4550 2021-02-10T11:50:51.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.721 7f2d3bfff700 1 --2- 172.21.2.101:0/2301618667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d200186c0 0x7f2d2001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:51.722 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.721 7f2d3bfff700 1 --2- 172.21.2.101:0/2301618667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d200186c0 0x7f2d2001abc0 crc :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:51.747 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.751 7f2d42c4c700 1 -- 172.21.2.101:0/2301618667 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d3c0f0b00 con 0x7f2d3c0b4550 2021-02-10T11:50:51.749 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.753 7f2d397fa700 1 -- 172.21.2.101:0/2301618667 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f2d24009340 con 0x7f2d3c0b4550 2021-02-10T11:50:51.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.883 7f2d42c4c700 1 -- 172.21.2.101:0/2301618667 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mgr", ""]}) v1 -- 0x7f2d3c0c3af0 con 0x7f2d200186c0 2021-02-10T11:50:51.879 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.883 7f2d397fa700 1 -- 172.21.2.101:0/2301618667 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (crc 0 0 0) 0x7f2d2c001460 con 0x7f2d200186c0 2021-02-10T11:50:51.879 INFO:teuthology.orchestra.run.gibba001.stderr:instructed osd(s) 5 to deep-scrub 2021-02-10T11:50:51.880 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.885 7f2d42c4c700 1 -- 172.21.2.101:0/2301618667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d200186c0 msgr2=0x7f2d2001abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:51.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.885 7f2d42c4c700 1 --2- 172.21.2.101:0/2301618667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d200186c0 0x7f2d2001abc0 crc :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:51.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.885 7f2d42c4c700 1 -- 172.21.2.101:0/2301618667 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d3c0b4550 msgr2=0x7f2d3c0c3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:51.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.885 7f2d42c4c700 1 --2- 172.21.2.101:0/2301618667 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d3c0b4550 0x7f2d3c0c3450 secure :-1 s=READY pgs=3481 cs=0 l=1 rev1=1 rx=0x7f2d24009630 tx=0x7f2d2400b980).stop 2021-02-10T11:50:51.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.885 7f2d42c4c700 1 -- 172.21.2.101:0/2301618667 shutdown_connections 2021-02-10T11:50:51.881 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.885 7f2d42c4c700 1 --2- 172.21.2.101:0/2301618667 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f2d200186c0 0x7f2d2001abc0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:51.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.885 7f2d42c4c700 1 --2- 172.21.2.101:0/2301618667 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f2d3c0c0ee0 0x7f2d3c0b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:51.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.885 7f2d42c4c700 1 --2- 172.21.2.101:0/2301618667 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f2d3c0b4950 0x7f2d3c0c08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:51.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.885 7f2d42c4c700 1 --2- 172.21.2.101:0/2301618667 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f2d3c0b4550 0x7f2d3c0c3450 unknown :-1 s=CLOSED pgs=3481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:51.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.885 7f2d42c4c700 1 -- 172.21.2.101:0/2301618667 >> 172.21.2.101:0/2301618667 conn(0x7f2d3c0f00e0 msgr2=0x7f2d3c08fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:51.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.885 7f2d42c4c700 1 -- 172.21.2.101:0/2301618667 shutdown_connections 2021-02-10T11:50:51.882 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:51.885 7f2d42c4c700 1 -- 172.21.2.101:0/2301618667 wait complete. 2021-02-10T11:50:51.889 INFO:tasks.ceph:Scrubbing osd.6 2021-02-10T11:50:51.890 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2021-02-10T11:50:52.321 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.310 7f2602616700 1 Processor -- start 2021-02-10T11:50:52.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.310 7f2602616700 1 -- start start 2021-02-10T11:50:52.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.310 7f2602616700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25fc0d1880 0x7f25fc1b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:52.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.310 7f2602616700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f25fc1b0af0 0x7f25fc108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:52.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.310 7f2602616700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f25fc0d1c80 0x7f25fc108f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:52.322 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.310 7f2602616700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25fc0e5f00 con 0x7f25fc0d1c80 2021-02-10T11:50:52.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.310 7f2602616700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25fc109a70 con 0x7f25fc1b0af0 2021-02-10T11:50:52.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.310 7f2602616700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc088070 con 0x7f25fc0d1880 2021-02-10T11:50:52.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.310 7f2600c33700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f25fc0d1c80 0x7f25fc108f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:52.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.310 7f2600c33700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f25fc0d1c80 0x7f25fc108f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36414/0 (socket says 172.21.2.101:36414) 2021-02-10T11:50:52.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.310 7f2600c33700 1 -- 172.21.2.101:0/2271095151 learned_addr learned my addr 172.21.2.101:0/2271095151 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:52.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.311 7f2600c33700 1 -- 172.21.2.101:0/2271095151 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25fc0d1880 msgr2=0x7f25fc1b04d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:50:52.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.311 7f2600c33700 1 --2- 172.21.2.101:0/2271095151 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25fc0d1880 0x7f25fc1b04d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:52.323 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.311 7f2600c33700 1 -- 172.21.2.101:0/2271095151 >> v2:172.21.2.108:3300/0 conn(0x7f25fc1b0af0 msgr2=0x7f25fc108950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-02-10T11:50:52.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.311 7f2600c33700 1 --2- 172.21.2.101:0/2271095151 >> v2:172.21.2.108:3300/0 conn(0x7f25fc1b0af0 0x7f25fc108950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:52.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.311 7f2600c33700 1 -- 172.21.2.101:0/2271095151 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25fc109590 con 0x7f25fc0d1c80 2021-02-10T11:50:52.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.311 7f2600c33700 1 --2- 172.21.2.101:0/2271095151 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25fc0d1c80 0x7f25fc108f70 secure :-1 s=READY pgs=3482 cs=0 l=1 rev1=1 rx=0x7f25f4006f00 tx=0x7f25f4009f10).ready entity=mon.0 client_cookie=e215ea6f91508cc6 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:52.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.311 7f25faffd700 1 -- 172.21.2.101:0/2271095151 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f25f400b3b0 con 0x7f25fc0d1c80 2021-02-10T11:50:52.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.311 7f25faffd700 1 -- 172.21.2.101:0/2271095151 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f400b730 con 0x7f25fc0d1c80 2021-02-10T11:50:52.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.311 7f25faffd700 1 -- 172.21.2.101:0/2271095151 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f25f400bc50 con 0x7f25fc0d1c80 2021-02-10T11:50:52.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.311 7f2602616700 1 -- 172.21.2.101:0/2271095151 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25fc0d1c80 msgr2=0x7f25fc108f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:52.324 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.311 7f2602616700 1 --2- 172.21.2.101:0/2271095151 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25fc0d1c80 0x7f25fc108f70 secure :-1 s=READY pgs=3482 cs=0 l=1 rev1=1 rx=0x7f25f4006f00 tx=0x7f25f4009f10).stop 2021-02-10T11:50:52.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.312 7f2602616700 1 -- 172.21.2.101:0/2271095151 shutdown_connections 2021-02-10T11:50:52.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.312 7f2602616700 1 --2- 172.21.2.101:0/2271095151 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25fc0d1c80 0x7f25fc108f70 unknown :-1 s=CLOSED pgs=3482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:52.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.312 7f2602616700 1 --2- 172.21.2.101:0/2271095151 >> v2:172.21.2.108:3300/0 conn(0x7f25fc1b0af0 0x7f25fc108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:52.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.312 7f2602616700 1 --2- 172.21.2.101:0/2271095151 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25fc0d1880 0x7f25fc1b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:52.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.312 7f2602616700 1 -- 172.21.2.101:0/2271095151 >> 172.21.2.101:0/2271095151 conn(0x7f25fc0ce430 msgr2=0x7f25fc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:52.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.312 7f2602616700 1 -- 172.21.2.101:0/2271095151 shutdown_connections 2021-02-10T11:50:52.325 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.312 7f2602616700 1 -- 172.21.2.101:0/2271095151 wait complete. 2021-02-10T11:50:52.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.326 7f2602616700 1 Processor -- start 2021-02-10T11:50:52.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.326 7f2602616700 1 -- start start 2021-02-10T11:50:52.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.327 7f2602616700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25fc1ad6e0 0x7f25fc0b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:52.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.327 7f2602616700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25fc0b2530 0x7f25fc1af630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:52.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.327 7f2602616700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25fc1afc50 0x7f25fc19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:52.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.327 7f2602616700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc1b00d0 con 0x7f25fc1afc50 2021-02-10T11:50:52.327 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.327 7f2602616700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc1b02f0 con 0x7f25fc1ad6e0 2021-02-10T11:50:52.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.327 7f2602616700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc19b140 con 0x7f25fc0b2530 2021-02-10T11:50:52.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.327 7f2600c33700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25fc1afc50 0x7f25fc19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:52.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.327 7f2600c33700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25fc1afc50 0x7f25fc19ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36420/0 (socket says 172.21.2.101:36420) 2021-02-10T11:50:52.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.327 7f2600c33700 1 -- 172.21.2.101:0/1708799976 learned_addr learned my addr 172.21.2.101:0/1708799976 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:52.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.327 7f2600c33700 1 -- 172.21.2.101:0/1708799976 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25fc0b2530 msgr2=0x7f25fc1af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:52.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.327 7f2600c33700 1 --2- 172.21.2.101:0/1708799976 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f25fc0b2530 0x7f25fc1af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:52.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.327 7f2600c33700 1 -- 172.21.2.101:0/1708799976 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25fc1ad6e0 msgr2=0x7f25fc0b1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:52.328 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.327 7f2600c33700 1 --2- 172.21.2.101:0/1708799976 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f25fc1ad6e0 0x7f25fc0b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:52.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.327 7f2600c33700 1 -- 172.21.2.101:0/1708799976 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25fc19b360 con 0x7f25fc1afc50 2021-02-10T11:50:52.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.327 7f2600c33700 1 --2- 172.21.2.101:0/1708799976 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25fc1afc50 0x7f25fc19ab20 secure :-1 s=READY pgs=3483 cs=0 l=1 rev1=1 rx=0x7f25f4009970 tx=0x7f25f400bc40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:52.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.328 7f25f8ff9700 1 -- 172.21.2.101:0/1708799976 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f25f4009450 con 0x7f25fc1afc50 2021-02-10T11:50:52.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.328 7f25f8ff9700 1 -- 172.21.2.101:0/1708799976 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f400bd80 con 0x7f25fc1afc50 2021-02-10T11:50:52.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.328 7f25f8ff9700 1 -- 172.21.2.101:0/1708799976 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f25f400c4e0 con 0x7f25fc1afc50 2021-02-10T11:50:52.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.328 7f2602616700 1 -- 172.21.2.101:0/1708799976 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25fc19b360 con 0x7f25fc1afc50 2021-02-10T11:50:52.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.328 7f2602616700 1 -- 172.21.2.101:0/1708799976 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25fc19b8f0 con 0x7f25fc1afc50 2021-02-10T11:50:52.329 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.328 7f25f8ff9700 1 -- 172.21.2.101:0/1708799976 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f25f4001270 con 0x7f25fc1afc50 2021-02-10T11:50:52.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.328 7f25f8ff9700 1 --2- 172.21.2.101:0/1708799976 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25dc0186c0 0x7f25dc01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:52.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.328 7f25f8ff9700 1 -- 172.21.2.101:0/1708799976 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f25f40327a0 con 0x7f25fc1afc50 2021-02-10T11:50:52.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.329 7f25fbfff700 1 --2- 172.21.2.101:0/1708799976 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25dc0186c0 0x7f25dc01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:52.330 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.329 7f25fbfff700 1 --2- 172.21.2.101:0/1708799976 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25dc0186c0 0x7f25dc01abc0 crc :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:52.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.360 7f2602616700 1 --2- 172.21.2.101:0/1708799976 >> [v2:172.21.2.108:6800/36381,v1:172.21.2.108:6801/36381] conn(0x7f25fc19cc40 0x7f25fc19d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:52.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.360 7f2602616700 1 -- 172.21.2.101:0/1708799976 --> [v2:172.21.2.108:6800/36381,v1:172.21.2.108:6801/36381] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f25fc19b8f0 con 0x7f25fc19cc40 2021-02-10T11:50:52.356 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.361 7f25fb7fe700 1 --2- 172.21.2.101:0/1708799976 >> [v2:172.21.2.108:6800/36381,v1:172.21.2.108:6801/36381] conn(0x7f25fc19cc40 0x7f25fc19d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:52.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.361 7f25fb7fe700 1 --2- 172.21.2.101:0/1708799976 >> [v2:172.21.2.108:6800/36381,v1:172.21.2.108:6801/36381] conn(0x7f25fc19cc40 0x7f25fc19d040 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:52.357 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.362 7f25f8ff9700 1 -- 172.21.2.101:0/1708799976 <== osd.6 v2:172.21.2.108:6800/36381 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (crc 0 0 0) 0x7f25f0006530 con 0x7f25fc19cc40 2021-02-10T11:50:52.364 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.369 7f2602616700 1 -- 172.21.2.101:0/1708799976 --> [v2:172.21.2.108:6800/36381,v1:172.21.2.108:6801/36381] -- command(tid 2: {"prefix": "config set", "key": "osd_debug_deep_scrub_sleep", "value": "0"}) v1 -- 0x7f25fc19b8f0 con 0x7f25fc19cc40 2021-02-10T11:50:52.365 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.369 7f25f8ff9700 1 -- 172.21.2.101:0/1708799976 <== osd.6 v2:172.21.2.108:6800/36381 2 ==== command_reply(tid 2: 0 Set osd_debug_deep_scrub_sleep to 0) v1 ==== 43+0+0 (crc 0 0 0) 0x7f25f0006b70 con 0x7f25fc19cc40 2021-02-10T11:50:52.365 INFO:teuthology.orchestra.run.gibba001.stderr:Set osd_debug_deep_scrub_sleep to 0 2021-02-10T11:50:52.366 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.370 7f2602616700 1 -- 172.21.2.101:0/1708799976 >> [v2:172.21.2.108:6800/36381,v1:172.21.2.108:6801/36381] conn(0x7f25fc19cc40 msgr2=0x7f25fc19d040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:52.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.370 7f2602616700 1 --2- 172.21.2.101:0/1708799976 >> [v2:172.21.2.108:6800/36381,v1:172.21.2.108:6801/36381] conn(0x7f25fc19cc40 0x7f25fc19d040 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:52.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.370 7f2602616700 1 -- 172.21.2.101:0/1708799976 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25dc0186c0 msgr2=0x7f25dc01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:52.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.370 7f2602616700 1 --2- 172.21.2.101:0/1708799976 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f25dc0186c0 0x7f25dc01abc0 crc :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:52.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.370 7f2602616700 1 -- 172.21.2.101:0/1708799976 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25fc1afc50 msgr2=0x7f25fc19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:52.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.370 7f2602616700 1 --2- 172.21.2.101:0/1708799976 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f25fc1afc50 0x7f25fc19ab20 secure :-1 s=READY pgs=3483 cs=0 l=1 rev1=1 rx=0x7f25f4009970 tx=0x7f25f400bc40).stop 2021-02-10T11:50:52.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.370 7f2600c33700 1 -- 172.21.2.101:0/1708799976 reap_dead start 2021-02-10T11:50:52.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.371 7f2602616700 1 -- 172.21.2.101:0/1708799976 shutdown_connections 2021-02-10T11:50:52.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.371 7f2602616700 1 -- 172.21.2.101:0/1708799976 >> 172.21.2.101:0/1708799976 conn(0x7f25fc1a9200 msgr2=0x7f25fc1b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:52.367 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.371 7f2602616700 1 -- 172.21.2.101:0/1708799976 shutdown_connections 2021-02-10T11:50:52.368 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.371 7f2602616700 1 -- 172.21.2.101:0/1708799976 wait complete. 2021-02-10T11:50:52.374 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd deep-scrub 6 2021-02-10T11:50:52.802 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.790 7f8dbcc07700 1 Processor -- start 2021-02-10T11:50:52.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.790 7f8dbcc07700 1 -- start start 2021-02-10T11:50:52.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.791 7f8dbcc07700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8db80d0340 0x7f8db80ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:52.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.791 7f8dbcc07700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db80f0b10 con 0x7f8db80d1870 2021-02-10T11:50:52.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.791 7f8dbcc07700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db81b0be0 con 0x7f8db80d1c70 2021-02-10T11:50:52.803 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.791 7f8dbcc07700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db80e5ed0 con 0x7f8db80d0340 2021-02-10T11:50:52.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.791 7f8db605d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8db80d0340 0x7f8db80ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:52.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.791 7f8db605d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f8db80d0340 0x7f8db80ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36426/0 (socket says 172.21.2.101:36426) 2021-02-10T11:50:52.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.791 7f8db605d700 1 -- 172.21.2.101:0/4051145503 learned_addr learned my addr 172.21.2.101:0/4051145503 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:52.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.791 7f8db685e700 1 --1- 172.21.2.101:0/4051145503 >> v1:172.21.2.101:6789/0 conn(0x7f8db80d1870 0x7f8db81b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36920/0 (socket says 172.21.2.101:36920) 2021-02-10T11:50:52.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.791 7f8db605d700 1 -- 172.21.2.101:0/4051145503 >> v1:172.21.2.108:6789/0 conn(0x7f8db80d1c70 legacy=0x7f8db80efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:52.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.791 7f8db605d700 1 -- 172.21.2.101:0/4051145503 >> v1:172.21.2.101:6789/0 conn(0x7f8db80d1870 legacy=0x7f8db81b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:52.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.791 7f8db605d700 1 -- 172.21.2.101:0/4051145503 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db80f0450 con 0x7f8db80d0340 2021-02-10T11:50:52.804 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.791 7f8db605d700 1 --2- 172.21.2.101:0/4051145503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8db80d0340 0x7f8db80ef640 secure :-1 s=READY pgs=3485 cs=0 l=1 rev1=1 rx=0x7f8da8006700 tx=0x7f8da8009b80).ready entity=mon.0 client_cookie=4fa48e6dba748c71 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:52.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.792 7f8db585c700 1 -- 172.21.2.101:0/4051145503 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8da800c0c0 con 0x7f8db80d0340 2021-02-10T11:50:52.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.792 7f8db585c700 1 -- 172.21.2.101:0/4051145503 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da800c440 con 0x7f8db80d0340 2021-02-10T11:50:52.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.792 7f8db585c700 1 -- 172.21.2.101:0/4051145503 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8da800c960 con 0x7f8db80d0340 2021-02-10T11:50:52.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.792 7f8dbcc07700 1 -- 172.21.2.101:0/4051145503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8db80d0340 msgr2=0x7f8db80ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:52.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.792 7f8dbcc07700 1 --2- 172.21.2.101:0/4051145503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8db80d0340 0x7f8db80ef640 secure :-1 s=READY pgs=3485 cs=0 l=1 rev1=1 rx=0x7f8da8006700 tx=0x7f8da8009b80).stop 2021-02-10T11:50:52.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.792 7f8dbcc07700 1 -- 172.21.2.101:0/4051145503 shutdown_connections 2021-02-10T11:50:52.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.792 7f8dbcc07700 1 --2- 172.21.2.101:0/4051145503 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8db80d0340 0x7f8db80ef640 unknown :-1 s=CLOSED pgs=3485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:52.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.792 7f8dbcc07700 1 -- 172.21.2.101:0/4051145503 >> 172.21.2.101:0/4051145503 conn(0x7f8db80ce450 msgr2=0x7f8db80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:52.805 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.792 7f8dbcc07700 1 -- 172.21.2.101:0/4051145503 shutdown_connections 2021-02-10T11:50:52.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.792 7f8dbcc07700 1 -- 172.21.2.101:0/4051145503 wait complete. 2021-02-10T11:50:52.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.807 7f8dbcc07700 1 Processor -- start 2021-02-10T11:50:52.806 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.807 7f8dbcc07700 1 -- start start 2021-02-10T11:50:52.807 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.807 7f8dbcc07700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8db80b1130 0x7f8db80b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:52.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.807 7f8dbcc07700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8db80b1b50 0x7f8db80bb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:52.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.807 7f8dbcc07700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8db80bb650 0x7f8db80bba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:52.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.807 7f8dbcc07700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db808baa0 con 0x7f8db80b1130 2021-02-10T11:50:52.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.807 7f8dbcc07700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db80bc070 con 0x7f8db80b1b50 2021-02-10T11:50:52.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.807 7f8dbcc07700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db808c020 con 0x7f8db80bb650 2021-02-10T11:50:52.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.807 7f8db685e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8db80b1130 0x7f8db80b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:52.808 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.808 7f8db685e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8db80b1130 0x7f8db80b1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36434/0 (socket says 172.21.2.101:36434) 2021-02-10T11:50:52.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.808 7f8db685e700 1 -- 172.21.2.101:0/2213741377 learned_addr learned my addr 172.21.2.101:0/2213741377 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:52.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.808 7f8db605d700 1 --2- 172.21.2.101:0/2213741377 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8db80b1b50 0x7f8db80bb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:52.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.808 7f8db685e700 1 -- 172.21.2.101:0/2213741377 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8db80bb650 msgr2=0x7f8db80bba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:52.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.808 7f8db685e700 1 --2- 172.21.2.101:0/2213741377 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8db80bb650 0x7f8db80bba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:52.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.808 7f8db685e700 1 -- 172.21.2.101:0/2213741377 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8db80b1b50 msgr2=0x7f8db80bb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:52.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.808 7f8db685e700 1 --2- 172.21.2.101:0/2213741377 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8db80b1b50 0x7f8db80bb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:52.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.808 7f8db685e700 1 -- 172.21.2.101:0/2213741377 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db808c240 con 0x7f8db80b1130 2021-02-10T11:50:52.809 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.808 7f8db685e700 1 --2- 172.21.2.101:0/2213741377 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8db80b1130 0x7f8db80b1530 secure :-1 s=READY pgs=3486 cs=0 l=1 rev1=1 rx=0x7f8da00014d0 tx=0x7f8da000a8a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:52.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.808 7f8daf7fe700 1 -- 172.21.2.101:0/2213741377 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8da0000ef0 con 0x7f8db80b1130 2021-02-10T11:50:52.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.808 7f8daf7fe700 1 -- 172.21.2.101:0/2213741377 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da000ac90 con 0x7f8db80b1130 2021-02-10T11:50:52.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.808 7f8daf7fe700 1 -- 172.21.2.101:0/2213741377 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8da0004590 con 0x7f8db80b1130 2021-02-10T11:50:52.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.808 7f8dbcc07700 1 -- 172.21.2.101:0/2213741377 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db808c240 con 0x7f8db80b1130 2021-02-10T11:50:52.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.808 7f8dbcc07700 1 -- 172.21.2.101:0/2213741377 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db808c240 con 0x7f8db80b1130 2021-02-10T11:50:52.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.809 7f8daf7fe700 1 -- 172.21.2.101:0/2213741377 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f8da00045b0 con 0x7f8db80b1130 2021-02-10T11:50:52.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.809 7f8daf7fe700 1 --2- 172.21.2.101:0/2213741377 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d9c0186c0 0x7f8d9c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:52.810 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.809 7f8daf7fe700 1 -- 172.21.2.101:0/2213741377 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f8da0032430 con 0x7f8db80b1130 2021-02-10T11:50:52.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.809 7f8db605d700 1 --2- 172.21.2.101:0/2213741377 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d9c0186c0 0x7f8d9c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:52.811 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.809 7f8db605d700 1 --2- 172.21.2.101:0/2213741377 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d9c0186c0 0x7f8d9c01abc0 crc :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:52.837 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.840 7f8dbcc07700 1 -- 172.21.2.101:0/2213741377 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db80bc070 con 0x7f8db80b1130 2021-02-10T11:50:52.839 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.843 7f8daf7fe700 1 -- 172.21.2.101:0/2213741377 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f8da0027f20 con 0x7f8db80b1130 2021-02-10T11:50:52.968 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.972 7f8dbcc07700 1 -- 172.21.2.101:0/2213741377 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mgr", ""]}) v1 -- 0x7f8db808baa0 con 0x7f8d9c0186c0 2021-02-10T11:50:52.969 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.972 7f8daf7fe700 1 -- 172.21.2.101:0/2213741377 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (crc 0 0 0) 0x7f8da80009b0 con 0x7f8d9c0186c0 2021-02-10T11:50:52.969 INFO:teuthology.orchestra.run.gibba001.stderr:instructed osd(s) 6 to deep-scrub 2021-02-10T11:50:52.970 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.975 7f8dbcc07700 1 -- 172.21.2.101:0/2213741377 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d9c0186c0 msgr2=0x7f8d9c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:52.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.975 7f8dbcc07700 1 --2- 172.21.2.101:0/2213741377 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d9c0186c0 0x7f8d9c01abc0 crc :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:52.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.975 7f8dbcc07700 1 -- 172.21.2.101:0/2213741377 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8db80b1130 msgr2=0x7f8db80b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:52.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.975 7f8dbcc07700 1 --2- 172.21.2.101:0/2213741377 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8db80b1130 0x7f8db80b1530 secure :-1 s=READY pgs=3486 cs=0 l=1 rev1=1 rx=0x7f8da00014d0 tx=0x7f8da000a8a0).stop 2021-02-10T11:50:52.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.975 7f8dbcc07700 1 -- 172.21.2.101:0/2213741377 shutdown_connections 2021-02-10T11:50:52.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.975 7f8dbcc07700 1 --2- 172.21.2.101:0/2213741377 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f8d9c0186c0 0x7f8d9c01abc0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:52.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.975 7f8dbcc07700 1 --2- 172.21.2.101:0/2213741377 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f8db80bb650 0x7f8db80bba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:52.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.975 7f8dbcc07700 1 --2- 172.21.2.101:0/2213741377 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f8db80b1b50 0x7f8db80bb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:52.971 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.975 7f8dbcc07700 1 --2- 172.21.2.101:0/2213741377 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f8db80b1130 0x7f8db80b1530 unknown :-1 s=CLOSED pgs=3486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:52.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.975 7f8dbcc07700 1 -- 172.21.2.101:0/2213741377 >> 172.21.2.101:0/2213741377 conn(0x7f8db80a74c0 msgr2=0x7f8db80bc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:52.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.975 7f8dbcc07700 1 -- 172.21.2.101:0/2213741377 shutdown_connections 2021-02-10T11:50:52.972 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:52.975 7f8dbcc07700 1 -- 172.21.2.101:0/2213741377 wait complete. 2021-02-10T11:50:52.979 INFO:tasks.ceph:Scrubbing osd.7 2021-02-10T11:50:52.979 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2021-02-10T11:50:53.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.400 7fc0852ec700 1 Processor -- start 2021-02-10T11:50:53.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.400 7fc0852ec700 1 -- start start 2021-02-10T11:50:53.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.401 7fc0852ec700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc0800d1880 0x7fc0801b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:53.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.401 7fc0852ec700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0800fe1d0 con 0x7fc0801b0af0 2021-02-10T11:50:53.413 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.401 7fc0852ec700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc080109a70 con 0x7fc0800cff80 2021-02-10T11:50:53.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.401 7fc0852ec700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc080088070 con 0x7fc0800d1880 2021-02-10T11:50:53.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.401 7fc07f07d700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fc0800d1880 0x7fc0801b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:53.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.401 7fc07e87c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7fc0801b0af0 0x7fc080108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36934/0 (socket says 172.21.2.101:36934) 2021-02-10T11:50:53.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.401 7fc07e87c700 1 -- 172.21.2.101:0/2904433648 learned_addr learned my addr 172.21.2.101:0/2904433648 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:50:53.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.401 7fc07f07d700 1 -- 172.21.2.101:0/2904433648 >> v1:172.21.2.108:6789/0 conn(0x7fc0800cff80 legacy=0x7fc080109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:53.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.401 7fc07f07d700 1 -- 172.21.2.101:0/2904433648 >> v1:172.21.2.101:6789/0 conn(0x7fc0801b0af0 legacy=0x7fc080108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:53.414 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.401 7fc07e07b700 1 -- 172.21.2.101:0/2904433648 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1338839478 0 0) 0x7fc0700009b0 con 0x7fc0801b0af0 2021-02-10T11:50:53.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.401 7fc07f07d700 1 -- 172.21.2.101:0/2904433648 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc080088260 con 0x7fc0800d1880 2021-02-10T11:50:53.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.402 7fc07f07d700 1 --2- 172.21.2.101:0/2904433648 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0800d1880 0x7fc0801b04d0 secure :-1 s=READY pgs=3488 cs=0 l=1 rev1=1 rx=0x7fc068006650 tx=0x7fc068009a00).ready entity=mon.0 client_cookie=66c52c1c9a930766 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:53.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.402 7fc07e07b700 1 -- 172.21.2.101:0/2904433648 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc068009df0 con 0x7fc0800d1880 2021-02-10T11:50:53.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.402 7fc07e07b700 1 -- 172.21.2.101:0/2904433648 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0680009b0 con 0x7fc0800d1880 2021-02-10T11:50:53.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.402 7fc07e07b700 1 -- 172.21.2.101:0/2904433648 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc06800c2b0 con 0x7fc0800d1880 2021-02-10T11:50:53.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.402 7fc0852ec700 1 -- 172.21.2.101:0/2904433648 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0800d1880 msgr2=0x7fc0801b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:53.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.402 7fc0852ec700 1 --2- 172.21.2.101:0/2904433648 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0800d1880 0x7fc0801b04d0 secure :-1 s=READY pgs=3488 cs=0 l=1 rev1=1 rx=0x7fc068006650 tx=0x7fc068009a00).stop 2021-02-10T11:50:53.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.403 7fc0852ec700 1 -- 172.21.2.101:0/2904433648 shutdown_connections 2021-02-10T11:50:53.415 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.403 7fc0852ec700 1 --2- 172.21.2.101:0/2904433648 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0800d1880 0x7fc0801b04d0 unknown :-1 s=CLOSED pgs=3488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:53.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.403 7fc0852ec700 1 -- 172.21.2.101:0/2904433648 >> 172.21.2.101:0/2904433648 conn(0x7fc0800ce430 msgr2=0x7fc0800d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:53.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.403 7fc0852ec700 1 -- 172.21.2.101:0/2904433648 shutdown_connections 2021-02-10T11:50:53.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.403 7fc0852ec700 1 -- 172.21.2.101:0/2904433648 wait complete. 2021-02-10T11:50:53.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.417 7fc0852ec700 1 Processor -- start 2021-02-10T11:50:53.416 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.417 7fc0852ec700 1 -- start start 2021-02-10T11:50:53.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.418 7fc0852ec700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc08019e5b0 0x7fc0801afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:53.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.418 7fc0852ec700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc08019e9b0 0x7fc0801b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:53.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.418 7fc0852ec700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0801b0250 0x7fc0801b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:53.418 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.418 7fc0852ec700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc080195bd0 con 0x7fc0801b0250 2021-02-10T11:50:53.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.418 7fc0852ec700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0801b0840 con 0x7fc08019e5b0 2021-02-10T11:50:53.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.418 7fc0852ec700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08019eed0 con 0x7fc08019e9b0 2021-02-10T11:50:53.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.418 7fc07f87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0801b0250 0x7fc0801b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:53.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.418 7fc07f87e700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0801b0250 0x7fc0801b13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36442/0 (socket says 172.21.2.101:36442) 2021-02-10T11:50:53.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.418 7fc07f87e700 1 -- 172.21.2.101:0/218825297 learned_addr learned my addr 172.21.2.101:0/218825297 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:53.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.418 7fc07f87e700 1 -- 172.21.2.101:0/218825297 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc08019e9b0 msgr2=0x7fc0801b0dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:50:53.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.418 7fc07f87e700 1 --2- 172.21.2.101:0/218825297 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fc08019e9b0 0x7fc0801b0dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:53.419 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.418 7fc07f87e700 1 -- 172.21.2.101:0/218825297 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc08019e5b0 msgr2=0x7fc0801afc30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:50:53.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.418 7fc07f87e700 1 --2- 172.21.2.101:0/218825297 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fc08019e5b0 0x7fc0801afc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:53.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.418 7fc07f87e700 1 -- 172.21.2.101:0/218825297 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08019f0b0 con 0x7fc0801b0250 2021-02-10T11:50:53.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.418 7fc07f87e700 1 --2- 172.21.2.101:0/218825297 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0801b0250 0x7fc0801b13e0 secure :-1 s=READY pgs=3489 cs=0 l=1 rev1=1 rx=0x7fc0780099a0 tx=0x7fc07800bcf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:53.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.418 7fc077fff700 1 -- 172.21.2.101:0/218825297 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc078003020 con 0x7fc0801b0250 2021-02-10T11:50:53.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.419 7fc077fff700 1 -- 172.21.2.101:0/218825297 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc078003220 con 0x7fc0801b0250 2021-02-10T11:50:53.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.419 7fc077fff700 1 -- 172.21.2.101:0/218825297 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc078005a90 con 0x7fc0801b0250 2021-02-10T11:50:53.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.419 7fc0852ec700 1 -- 172.21.2.101:0/218825297 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08019f0b0 con 0x7fc0801b0250 2021-02-10T11:50:53.420 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.419 7fc0852ec700 1 -- 172.21.2.101:0/218825297 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08019f0b0 con 0x7fc0801b0250 2021-02-10T11:50:53.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.419 7fc077fff700 1 -- 172.21.2.101:0/218825297 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fc0780058a0 con 0x7fc0801b0250 2021-02-10T11:50:53.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.419 7fc077fff700 1 --2- 172.21.2.101:0/218825297 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0640186c0 0x7fc06401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:53.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.419 7fc077fff700 1 -- 172.21.2.101:0/218825297 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc0780328e0 con 0x7fc0801b0250 2021-02-10T11:50:53.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.419 7fc07f07d700 1 --2- 172.21.2.101:0/218825297 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0640186c0 0x7fc06401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:53.421 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.420 7fc07f07d700 1 --2- 172.21.2.101:0/218825297 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0640186c0 0x7fc06401abc0 crc :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:53.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.451 7fc0852ec700 1 --2- 172.21.2.101:0/218825297 >> [v2:172.21.2.108:6817/36382,v1:172.21.2.108:6819/36382] conn(0x7fc0800627c0 0x7fc080062bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:53.446 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.451 7fc0852ec700 1 -- 172.21.2.101:0/218825297 --> [v2:172.21.2.108:6817/36382,v1:172.21.2.108:6819/36382] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc0801b0840 con 0x7fc0800627c0 2021-02-10T11:50:53.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.451 7fc07e87c700 1 --2- 172.21.2.101:0/218825297 >> [v2:172.21.2.108:6817/36382,v1:172.21.2.108:6819/36382] conn(0x7fc0800627c0 0x7fc080062bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:53.447 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.452 7fc07e87c700 1 --2- 172.21.2.101:0/218825297 >> [v2:172.21.2.108:6817/36382,v1:172.21.2.108:6819/36382] conn(0x7fc0800627c0 0x7fc080062bc0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:53.448 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.452 7fc077fff700 1 -- 172.21.2.101:0/218825297 <== osd.7 v2:172.21.2.108:6817/36382 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (crc 0 0 0) 0x7fc0700014a0 con 0x7fc0800627c0 2021-02-10T11:50:53.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.460 7fc0852ec700 1 -- 172.21.2.101:0/218825297 --> [v2:172.21.2.108:6817/36382,v1:172.21.2.108:6819/36382] -- command(tid 2: {"prefix": "config set", "key": "osd_debug_deep_scrub_sleep", "value": "0"}) v1 -- 0x7fc080082aa0 con 0x7fc0800627c0 2021-02-10T11:50:53.456 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.460 7fc077fff700 1 -- 172.21.2.101:0/218825297 <== osd.7 v2:172.21.2.108:6817/36382 2 ==== command_reply(tid 2: 0 Set osd_debug_deep_scrub_sleep to 0) v1 ==== 43+0+0 (crc 0 0 0) 0x7fc070000e00 con 0x7fc0800627c0 2021-02-10T11:50:53.457 INFO:teuthology.orchestra.run.gibba001.stderr:Set osd_debug_deep_scrub_sleep to 0 2021-02-10T11:50:53.457 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.461 7fc0852ec700 1 -- 172.21.2.101:0/218825297 >> [v2:172.21.2.108:6817/36382,v1:172.21.2.108:6819/36382] conn(0x7fc0800627c0 msgr2=0x7fc080062bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:53.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.461 7fc0852ec700 1 --2- 172.21.2.101:0/218825297 >> [v2:172.21.2.108:6817/36382,v1:172.21.2.108:6819/36382] conn(0x7fc0800627c0 0x7fc080062bc0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:53.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.461 7fc0852ec700 1 -- 172.21.2.101:0/218825297 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0640186c0 msgr2=0x7fc06401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:53.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.461 7fc0852ec700 1 --2- 172.21.2.101:0/218825297 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fc0640186c0 0x7fc06401abc0 crc :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:53.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.461 7fc0852ec700 1 -- 172.21.2.101:0/218825297 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0801b0250 msgr2=0x7fc0801b13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:53.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.461 7fc0852ec700 1 --2- 172.21.2.101:0/218825297 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fc0801b0250 0x7fc0801b13e0 secure :-1 s=READY pgs=3489 cs=0 l=1 rev1=1 rx=0x7fc0780099a0 tx=0x7fc07800bcf0).stop 2021-02-10T11:50:53.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.461 7fc07f87e700 1 -- 172.21.2.101:0/218825297 reap_dead start 2021-02-10T11:50:53.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.461 7fc0852ec700 1 -- 172.21.2.101:0/218825297 shutdown_connections 2021-02-10T11:50:53.458 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.461 7fc0852ec700 1 -- 172.21.2.101:0/218825297 >> 172.21.2.101:0/218825297 conn(0x7fc08019f940 msgr2=0x7fc08018c290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:53.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.462 7fc0852ec700 1 -- 172.21.2.101:0/218825297 shutdown_connections 2021-02-10T11:50:53.459 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.462 7fc0852ec700 1 -- 172.21.2.101:0/218825297 wait complete. 2021-02-10T11:50:53.466 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph osd deep-scrub 7 2021-02-10T11:50:53.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.882 7f6784461700 1 Processor -- start 2021-02-10T11:50:53.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.882 7f6784461700 1 -- start start 2021-02-10T11:50:53.896 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.883 7f6784461700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7f677c0d1c70 0x7f677c0f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:53.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.883 7f6784461700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f677c0f0b10 con 0x7f677c0d0340 2021-02-10T11:50:53.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.883 7f6784461700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f677c1b0be0 con 0x7f677c0d1870 2021-02-10T11:50:53.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.883 7f6784461700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f677c0e5ed0 con 0x7f677c0d1c70 2021-02-10T11:50:53.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.883 7f6781a7c700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f677c0d0340 0x7f677c0ef640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36948/0 (socket says 172.21.2.101:36948) 2021-02-10T11:50:53.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.883 7f6781a7c700 1 -- 172.21.2.101:0/2773068876 learned_addr learned my addr 172.21.2.101:0/2773068876 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:50:53.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.883 7f678127b700 1 -- 172.21.2.101:0/2773068876 <== mon.0 v1:172.21.2.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 398147414 0 0) 0x7f676c0010b0 con 0x7f677c0d0340 2021-02-10T11:50:53.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.883 7f678127b700 1 -- 172.21.2.101:0/2773068876 --> v1:172.21.2.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6768002fe0 con 0x7f677c0d0340 2021-02-10T11:50:53.897 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.884 7f6782a7e700 1 --2- 172.21.2.101:0/2773068876 >> v2:172.21.2.108:3300/0 conn(0x7f677c0d1c70 0x7f677c0f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:53.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.884 7f678127b700 1 -- 172.21.2.101:0/2773068876 <== mon.0 v1:172.21.2.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2949266913 0 0) 0x7f676c0010b0 con 0x7f677c0d0340 2021-02-10T11:50:53.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.884 7f678127b700 1 -- 172.21.2.101:0/2773068876 >> v2:172.21.2.108:3300/0 conn(0x7f677c0d1c70 msgr2=0x7f677c0f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:53.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.884 7f678127b700 1 --2- 172.21.2.101:0/2773068876 >> v2:172.21.2.108:3300/0 conn(0x7f677c0d1c70 0x7f677c0f20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:53.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.884 7f678127b700 1 -- 172.21.2.101:0/2773068876 >> v1:172.21.2.108:6789/0 conn(0x7f677c0d1870 legacy=0x7f677c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:53.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.884 7f678127b700 1 -- 172.21.2.101:0/2773068876 --> v1:172.21.2.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677c0efe30 con 0x7f677c0d0340 2021-02-10T11:50:53.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.884 7f678127b700 1 -- 172.21.2.101:0/2773068876 <== mon.0 v1:172.21.2.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1263900581 0 0) 0x7f676c0015b0 con 0x7f677c0d0340 2021-02-10T11:50:53.898 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.884 7f6782a7e700 1 --2- 172.21.2.101:0/2773068876 >> v2:172.21.2.108:3300/0 conn(0x7f677c0d1c70 0x7f677c0f20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:50:53.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.884 7f678127b700 1 -- 172.21.2.101:0/2773068876 >> v1:172.21.2.101:6789/0 conn(0x7f677c0d0340 legacy=0x7f677c0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:53.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.884 7f678127b700 1 --2- 172.21.2.101:0/2773068876 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6768003a60 0x7f6768005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:53.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.884 7f678127b700 1 --2- 172.21.2.101:0/2773068876 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6768006790 0x7f6768008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:53.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.884 7f678127b700 1 --2- 172.21.2.101:0/2773068876 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6768009590 0x7f676800bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:53.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.884 7f678127b700 1 -- 172.21.2.101:0/2773068876 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676800c300 con 0x7f6768009590 2021-02-10T11:50:53.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.884 7f678127b700 1 -- 172.21.2.101:0/2773068876 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676800c520 con 0x7f6768006790 2021-02-10T11:50:53.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.884 7f678127b700 1 -- 172.21.2.101:0/2773068876 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676800c740 con 0x7f6768003a60 2021-02-10T11:50:53.899 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.884 7f6782a7e700 1 --2- 172.21.2.101:0/2773068876 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6768009590 0x7f676800bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:53.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.884 7f6781a7c700 1 --2- 172.21.2.101:0/2773068876 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6768006790 0x7f6768008c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:53.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.884 7f678227d700 1 --2- 172.21.2.101:0/2773068876 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6768003a60 0x7f6768005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:53.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.884 7f6782a7e700 1 -- 172.21.2.101:0/2773068876 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6768003a60 msgr2=0x7f6768005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:53.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.884 7f6782a7e700 1 --2- 172.21.2.101:0/2773068876 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f6768003a60 0x7f6768005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:53.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.884 7f6782a7e700 1 -- 172.21.2.101:0/2773068876 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6768006790 msgr2=0x7f6768008c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:53.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.884 7f6782a7e700 1 --2- 172.21.2.101:0/2773068876 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f6768006790 0x7f6768008c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:53.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.885 7f6782a7e700 1 -- 172.21.2.101:0/2773068876 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67680509f0 con 0x7f6768009590 2021-02-10T11:50:53.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.885 7f6782a7e700 1 -- 172.21.2.101:0/2773068876 reap_dead start 2021-02-10T11:50:53.900 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.885 7f6782a7e700 1 --2- 172.21.2.101:0/2773068876 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6768009590 0x7f676800bab0 secure :-1 s=READY pgs=3491 cs=0 l=1 rev1=1 rx=0x7f67740090f0 tx=0x7f677400b300).ready entity=mon.0 client_cookie=75a842551c966c80 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:53.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.885 7f678127b700 1 -- 172.21.2.101:0/2773068876 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6774006790 con 0x7f6768009590 2021-02-10T11:50:53.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.885 7f678127b700 1 -- 172.21.2.101:0/2773068876 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6774003030 con 0x7f6768009590 2021-02-10T11:50:53.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.885 7f678127b700 1 -- 172.21.2.101:0/2773068876 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f677400b970 con 0x7f6768009590 2021-02-10T11:50:53.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.885 7f6784461700 1 -- 172.21.2.101:0/2773068876 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6768009590 msgr2=0x7f676800bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:53.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.885 7f6784461700 1 --2- 172.21.2.101:0/2773068876 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6768009590 0x7f676800bab0 secure :-1 s=READY pgs=3491 cs=0 l=1 rev1=1 rx=0x7f67740090f0 tx=0x7f677400b300).stop 2021-02-10T11:50:53.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.886 7f6784461700 1 -- 172.21.2.101:0/2773068876 shutdown_connections 2021-02-10T11:50:53.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.886 7f6784461700 1 --2- 172.21.2.101:0/2773068876 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f6768009590 0x7f676800bab0 unknown :-1 s=CLOSED pgs=3491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:53.901 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.886 7f6784461700 1 -- 172.21.2.101:0/2773068876 >> 172.21.2.101:0/2773068876 conn(0x7f677c0ce450 msgr2=0x7f677c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:53.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.886 7f6784461700 1 -- 172.21.2.101:0/2773068876 shutdown_connections 2021-02-10T11:50:53.902 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.886 7f6784461700 1 -- 172.21.2.101:0/2773068876 wait complete. 2021-02-10T11:50:53.903 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.901 7f6784461700 1 Processor -- start 2021-02-10T11:50:53.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.901 7f6784461700 1 -- start start 2021-02-10T11:50:53.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.901 7f6784461700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f677c19ffa0 0x7f677c0eda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:53.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.901 7f6784461700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f677c1a03a0 0x7f677c0ee030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:53.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.901 7f6784461700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f677c0ee650 0x7f677c0ebe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:53.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.901 7f6784461700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c0ed490 con 0x7f677c19ffa0 2021-02-10T11:50:53.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.901 7f6784461700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c0eea50 con 0x7f677c0ee650 2021-02-10T11:50:53.904 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.901 7f6784461700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677c0eec30 con 0x7f677c1a03a0 2021-02-10T11:50:53.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.902 7f678227d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f677c19ffa0 0x7f677c0eda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:53.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.902 7f678227d700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f677c19ffa0 0x7f677c0eda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36466/0 (socket says 172.21.2.101:36466) 2021-02-10T11:50:53.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.902 7f678227d700 1 -- 172.21.2.101:0/3117576702 learned_addr learned my addr 172.21.2.101:0/3117576702 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:53.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.902 7f6781a7c700 1 --2- 172.21.2.101:0/3117576702 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f677c1a03a0 0x7f677c0ee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:53.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.902 7f6782a7e700 1 --2- 172.21.2.101:0/3117576702 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f677c0ee650 0x7f677c0ebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:53.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.902 7f678227d700 1 -- 172.21.2.101:0/3117576702 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f677c1a03a0 msgr2=0x7f677c0ee030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:53.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.902 7f678227d700 1 --2- 172.21.2.101:0/3117576702 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f677c1a03a0 0x7f677c0ee030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:53.905 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.902 7f678227d700 1 -- 172.21.2.101:0/3117576702 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f677c0ee650 msgr2=0x7f677c0ebe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:53.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.902 7f678227d700 1 --2- 172.21.2.101:0/3117576702 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f677c0ee650 0x7f677c0ebe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:53.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.902 7f678227d700 1 -- 172.21.2.101:0/3117576702 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677c0ec4a0 con 0x7f677c19ffa0 2021-02-10T11:50:53.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.902 7f6781a7c700 1 --2- 172.21.2.101:0/3117576702 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f677c1a03a0 0x7f677c0ee030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:50:53.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.902 7f6782a7e700 1 --2- 172.21.2.101:0/3117576702 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f677c0ee650 0x7f677c0ebe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-02-10T11:50:53.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.902 7f678227d700 1 --2- 172.21.2.101:0/3117576702 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f677c19ffa0 0x7f677c0eda10 secure :-1 s=READY pgs=3492 cs=0 l=1 rev1=1 rx=0x7f677800b610 tx=0x7f677800b9b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:53.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.902 7f6772ffd700 1 -- 172.21.2.101:0/3117576702 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6778000960 con 0x7f677c19ffa0 2021-02-10T11:50:53.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.902 7f6772ffd700 1 -- 172.21.2.101:0/3117576702 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677800bda0 con 0x7f677c19ffa0 2021-02-10T11:50:53.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.902 7f6772ffd700 1 -- 172.21.2.101:0/3117576702 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6778002030 con 0x7f677c19ffa0 2021-02-10T11:50:53.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.902 7f6784461700 1 -- 172.21.2.101:0/3117576702 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f677c0ec4a0 con 0x7f677c19ffa0 2021-02-10T11:50:53.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.902 7f6784461700 1 -- 172.21.2.101:0/3117576702 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f677c0eca30 con 0x7f677c19ffa0 2021-02-10T11:50:53.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.903 7f6772ffd700 1 -- 172.21.2.101:0/3117576702 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f6778000ca0 con 0x7f677c19ffa0 2021-02-10T11:50:53.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.903 7f6772ffd700 1 --2- 172.21.2.101:0/3117576702 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6764018730 0x7f676401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:53.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.903 7f6772ffd700 1 -- 172.21.2.101:0/3117576702 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6778032660 con 0x7f677c19ffa0 2021-02-10T11:50:53.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.903 7f6781a7c700 1 --2- 172.21.2.101:0/3117576702 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6764018730 0x7f676401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:53.908 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.904 7f6781a7c700 1 --2- 172.21.2.101:0/3117576702 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6764018730 0x7f676401abc0 crc :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:53.928 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.933 7f6784461700 1 -- 172.21.2.101:0/3117576702 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f677c0ebc50 con 0x7f677c19ffa0 2021-02-10T11:50:53.931 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:53.935 7f6772ffd700 1 -- 172.21.2.101:0/3117576702 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f6778000940 con 0x7f677c19ffa0 2021-02-10T11:50:54.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.063 7f6784461700 1 -- 172.21.2.101:0/3117576702 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mgr", ""]}) v1 -- 0x7f677c0eca30 con 0x7f6764018730 2021-02-10T11:50:54.059 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.063 7f6772ffd700 1 -- 172.21.2.101:0/3117576702 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (crc 0 0 0) 0x7f676c001730 con 0x7f6764018730 2021-02-10T11:50:54.060 INFO:teuthology.orchestra.run.gibba001.stderr:instructed osd(s) 7 to deep-scrub 2021-02-10T11:50:54.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.065 7f6784461700 1 -- 172.21.2.101:0/3117576702 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6764018730 msgr2=0x7f676401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:54.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.065 7f6784461700 1 --2- 172.21.2.101:0/3117576702 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6764018730 0x7f676401abc0 crc :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:54.061 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.065 7f6784461700 1 -- 172.21.2.101:0/3117576702 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f677c19ffa0 msgr2=0x7f677c0eda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:54.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.065 7f6784461700 1 --2- 172.21.2.101:0/3117576702 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f677c19ffa0 0x7f677c0eda10 secure :-1 s=READY pgs=3492 cs=0 l=1 rev1=1 rx=0x7f677800b610 tx=0x7f677800b9b0).stop 2021-02-10T11:50:54.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.065 7f6784461700 1 -- 172.21.2.101:0/3117576702 shutdown_connections 2021-02-10T11:50:54.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.065 7f6784461700 1 --2- 172.21.2.101:0/3117576702 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f6764018730 0x7f676401abc0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:54.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.065 7f6784461700 1 --2- 172.21.2.101:0/3117576702 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f677c1a03a0 0x7f677c0ee030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:54.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.065 7f6784461700 1 --2- 172.21.2.101:0/3117576702 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f677c0ee650 0x7f677c0ebe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:54.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.065 7f6784461700 1 --2- 172.21.2.101:0/3117576702 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f677c19ffa0 0x7f677c0eda10 unknown :-1 s=CLOSED pgs=3492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:54.062 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.065 7f6784461700 1 -- 172.21.2.101:0/3117576702 >> 172.21.2.101:0/3117576702 conn(0x7f677c094d90 msgr2=0x7f677c08a420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:54.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.065 7f6784461700 1 -- 172.21.2.101:0/3117576702 shutdown_connections 2021-02-10T11:50:54.063 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.065 7f6784461700 1 -- 172.21.2.101:0/3117576702 wait complete. 2021-02-10T11:50:54.070 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph pg dump --format=json 2021-02-10T11:50:54.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.488 7fdea997f700 1 Processor -- start 2021-02-10T11:50:54.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.488 7fdea997f700 1 -- start start 2021-02-10T11:50:54.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.489 7fdea997f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdea40d0340 0x7fdea40ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:54.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.489 7fdea997f700 1 --2- >> v2:172.21.2.108:3300/0 conn(0x7fdea40d1c70 0x7fdea40efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:54.501 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.489 7fdea997f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea40f0b10 con 0x7fdea40d1870 2021-02-10T11:50:54.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.489 7fdea997f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea40e5ed0 con 0x7fdea40d0340 2021-02-10T11:50:54.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.489 7fdea997f700 1 -- --> v2:172.21.2.108:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea41b0be0 con 0x7fdea40d1c70 2021-02-10T11:50:54.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.489 7fdea2d73700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdea40d0340 0x7fdea40ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:54.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.489 7fdea2d73700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7fdea40d0340 0x7fdea40ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36472/0 (socket says 172.21.2.101:36472) 2021-02-10T11:50:54.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.489 7fdea2d73700 1 -- 172.21.2.101:0/3636526207 learned_addr learned my addr 172.21.2.101:0/3636526207 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:54.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.489 7fdea2d73700 1 -- 172.21.2.101:0/3636526207 >> v2:172.21.2.108:3300/0 conn(0x7fdea40d1c70 msgr2=0x7fdea40efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:54.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.489 7fdea2d73700 1 --2- 172.21.2.101:0/3636526207 >> v2:172.21.2.108:3300/0 conn(0x7fdea40d1c70 0x7fdea40efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:54.502 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.490 7fdea2d73700 1 -- 172.21.2.101:0/3636526207 >> v1:172.21.2.108:6789/0 conn(0x7fdea40d1870 legacy=0x7fdea41b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:50:54.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.490 7fdea2d73700 1 -- 172.21.2.101:0/3636526207 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea40f20a0 con 0x7fdea40d0340 2021-02-10T11:50:54.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.490 7fdea2d73700 1 --2- 172.21.2.101:0/3636526207 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdea40d0340 0x7fdea40ef640 secure :-1 s=READY pgs=3493 cs=0 l=1 rev1=1 rx=0x7fde98006700 tx=0x7fde98009b80).ready entity=mon.0 client_cookie=18edf4e0f91b9cc server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:54.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.490 7fdea2572700 1 -- 172.21.2.101:0/3636526207 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fde9800c0c0 con 0x7fdea40d0340 2021-02-10T11:50:54.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.490 7fdea2572700 1 -- 172.21.2.101:0/3636526207 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9800c440 con 0x7fdea40d0340 2021-02-10T11:50:54.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.490 7fdea2572700 1 -- 172.21.2.101:0/3636526207 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fde9800c960 con 0x7fdea40d0340 2021-02-10T11:50:54.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.490 7fdea997f700 1 -- 172.21.2.101:0/3636526207 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdea40d0340 msgr2=0x7fdea40ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:54.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.490 7fdea997f700 1 --2- 172.21.2.101:0/3636526207 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdea40d0340 0x7fdea40ef640 secure :-1 s=READY pgs=3493 cs=0 l=1 rev1=1 rx=0x7fde98006700 tx=0x7fde98009b80).stop 2021-02-10T11:50:54.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.491 7fdea997f700 1 -- 172.21.2.101:0/3636526207 shutdown_connections 2021-02-10T11:50:54.503 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.491 7fdea997f700 1 --2- 172.21.2.101:0/3636526207 >> v2:172.21.2.108:3300/0 conn(0x7fdea40d1c70 0x7fdea40efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:54.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.491 7fdea997f700 1 --2- 172.21.2.101:0/3636526207 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdea40d0340 0x7fdea40ef640 unknown :-1 s=CLOSED pgs=3493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:54.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.491 7fdea997f700 1 -- 172.21.2.101:0/3636526207 >> 172.21.2.101:0/3636526207 conn(0x7fdea40ce450 msgr2=0x7fdea40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:54.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.491 7fdea997f700 1 -- 172.21.2.101:0/3636526207 shutdown_connections 2021-02-10T11:50:54.504 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.491 7fdea997f700 1 -- 172.21.2.101:0/3636526207 wait complete. 2021-02-10T11:50:54.505 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.506 7fdea997f700 1 Processor -- start 2021-02-10T11:50:54.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.506 7fdea997f700 1 -- start start 2021-02-10T11:50:54.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.506 7fdea997f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdea419a3d0 0x7fdea41ab160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:54.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.506 7fdea997f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdea41ab780 0x7fdea40b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:54.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.506 7fdea997f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdea41abb80 0x7fdea40b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:54.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.506 7fdea997f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea41ae200 con 0x7fdea41abb80 2021-02-10T11:50:54.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.506 7fdea997f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea4092930 con 0x7fdea419a3d0 2021-02-10T11:50:54.506 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.506 7fdea997f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea40b4290 con 0x7fdea41ab780 2021-02-10T11:50:54.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.506 7fdea3d75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdea41abb80 0x7fdea40b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:54.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.506 7fdea3d75700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdea41abb80 0x7fdea40b3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36482/0 (socket says 172.21.2.101:36482) 2021-02-10T11:50:54.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.506 7fdea3d75700 1 -- 172.21.2.101:0/1035178434 learned_addr learned my addr 172.21.2.101:0/1035178434 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:50:54.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.507 7fdea3d75700 1 -- 172.21.2.101:0/1035178434 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdea41ab780 msgr2=0x7fdea40b3650 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:50:54.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.507 7fdea3d75700 1 --2- 172.21.2.101:0/1035178434 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdea41ab780 0x7fdea40b3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:54.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.507 7fdea3d75700 1 -- 172.21.2.101:0/1035178434 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdea419a3d0 msgr2=0x7fdea41ab160 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:50:54.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.507 7fdea3d75700 1 --2- 172.21.2.101:0/1035178434 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdea419a3d0 0x7fdea41ab160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:54.507 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.507 7fdea3d75700 1 -- 172.21.2.101:0/1035178434 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea40ee360 con 0x7fdea41abb80 2021-02-10T11:50:54.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.507 7fdea3d75700 1 --2- 172.21.2.101:0/1035178434 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdea41abb80 0x7fdea40b3c70 secure :-1 s=READY pgs=3494 cs=0 l=1 rev1=1 rx=0x7fde9c009980 tx=0x7fde9c00bcd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:54.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.507 7fde8bfff700 1 -- 172.21.2.101:0/1035178434 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fde9c005870 con 0x7fdea41abb80 2021-02-10T11:50:54.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.507 7fde8bfff700 1 -- 172.21.2.101:0/1035178434 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9c005b30 con 0x7fdea41abb80 2021-02-10T11:50:54.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.507 7fdea997f700 1 -- 172.21.2.101:0/1035178434 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea40ee360 con 0x7fdea41abb80 2021-02-10T11:50:54.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.507 7fdea997f700 1 -- 172.21.2.101:0/1035178434 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea40ee8f0 con 0x7fdea41abb80 2021-02-10T11:50:54.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.507 7fde8bfff700 1 -- 172.21.2.101:0/1035178434 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fde9c006030 con 0x7fdea41abb80 2021-02-10T11:50:54.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.508 7fde8bfff700 1 -- 172.21.2.101:0/1035178434 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7fde9c005d10 con 0x7fdea41abb80 2021-02-10T11:50:54.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.508 7fde8bfff700 1 --2- 172.21.2.101:0/1035178434 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fde840186c0 0x7fde8401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:50:54.508 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.508 7fde8bfff700 1 -- 172.21.2.101:0/1035178434 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fde9c0329e0 con 0x7fdea41abb80 2021-02-10T11:50:54.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.508 7fdea3574700 1 --2- 172.21.2.101:0/1035178434 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fde840186c0 0x7fde8401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:50:54.509 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.508 7fdea3574700 1 --2- 172.21.2.101:0/1035178434 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fde840186c0 0x7fde8401abc0 crc :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:50:54.535 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.539 7fdea997f700 1 -- 172.21.2.101:0/1035178434 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea40ce450 con 0x7fdea41abb80 2021-02-10T11:50:54.537 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.542 7fde8bfff700 1 -- 172.21.2.101:0/1035178434 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7fde9c02def0 con 0x7fdea41abb80 2021-02-10T11:50:54.672 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.676 7fdea997f700 1 -- 172.21.2.101:0/1035178434 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "pg dump", "target": ["mgr", ""], "format": "json"}) v1 -- 0x7fdea40ee8f0 con 0x7fde840186c0 2021-02-10T11:50:54.674 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.679 7fde8bfff700 1 -- 172.21.2.101:0/1035178434 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 dumped all) v1 ==== 18+0+101060 (crc 0 0 0) 0x7fde94006970 con 0x7fde840186c0 2021-02-10T11:50:54.677 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:50:54.677 INFO:teuthology.orchestra.run.gibba001.stderr:dumped all 2021-02-10T11:50:54.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.681 7fdea997f700 1 -- 172.21.2.101:0/1035178434 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fde840186c0 msgr2=0x7fde8401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:54.677 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.681 7fdea997f700 1 --2- 172.21.2.101:0/1035178434 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fde840186c0 0x7fde8401abc0 crc :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:54.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.681 7fdea997f700 1 -- 172.21.2.101:0/1035178434 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdea41abb80 msgr2=0x7fdea40b3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:50:54.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.681 7fdea997f700 1 --2- 172.21.2.101:0/1035178434 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdea41abb80 0x7fdea40b3c70 secure :-1 s=READY pgs=3494 cs=0 l=1 rev1=1 rx=0x7fde9c009980 tx=0x7fde9c00bcd0).stop 2021-02-10T11:50:54.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.681 7fdea997f700 1 -- 172.21.2.101:0/1035178434 shutdown_connections 2021-02-10T11:50:54.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.681 7fdea997f700 1 --2- 172.21.2.101:0/1035178434 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7fde840186c0 0x7fde8401abc0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:54.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.681 7fdea997f700 1 --2- 172.21.2.101:0/1035178434 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7fdea41ab780 0x7fdea40b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:54.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.681 7fdea997f700 1 --2- 172.21.2.101:0/1035178434 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7fdea41abb80 0x7fdea40b3c70 unknown :-1 s=CLOSED pgs=3494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:50:54.678 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.681 7fdea997f700 1 --2- 172.21.2.101:0/1035178434 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7fdea419a3d0 0x7fdea41ab160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:50:54.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.681 7fdea997f700 1 -- 172.21.2.101:0/1035178434 >> 172.21.2.101:0/1035178434 conn(0x7fdea4093640 msgr2=0x7fdea418eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:50:54.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.681 7fdea997f700 1 -- 172.21.2.101:0/1035178434 shutdown_connections 2021-02-10T11:50:54.679 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:50:54.682 7fdea997f700 1 -- 172.21.2.101:0/1035178434 wait complete. 2021-02-10T11:50:54.685 INFO:teuthology.orchestra.run.gibba001.stdout:{"pg_ready":true,"pg_map":{"version":1338,"stamp":"2021-02-10 11:50:54.638644","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":135159657,"num_objects":154,"num_object_clones":1,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":65,"num_read_kb":65537,"num_write":438,"num_write_kb":132094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":424,"ondisk_log_size":424,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"kb":754974720,"kb_used":8673008,"kb_used_data":284136,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746301712,"statfs":{"total":773094113280,"available":764212953088,"internally_reserved":8589934592,"allocated":290955264,"data_stored":279160930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":165439,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002715"},"pg_stats":[{"pgid":"43.6","version":"199'22","reported_seq":"40","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:34.454332","last_change":"2021-02-10 11:49:18.829257","last_active":"2021-02-10 11:50:34.454332","last_peered":"2021-02-10 11:50:34.454332","last_clean":"2021-02-10 11:50:34.454332","last_became_active":"2021-02-10 11:49:18.828991","last_became_peered":"2021-02-10 11:49:18.828991","last_unstale":"2021-02-10 11:50:34.454332","last_undegraded":"2021-02-10 11:50:34.454332","last_fullsized":"2021-02-10 11:50:34.454332","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:18.173888","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:18.173888","last_clean_scrub_stamp":"2021-02-10 11:49:18.173888","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23068672,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":9,"num_read_kb":9216,"num_write":22,"num_write_kb":22528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.7","version":"199'8","reported_seq":"18","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:49:20.506753","last_change":"2021-02-10 11:49:18.171840","last_active":"2021-02-10 11:49:20.506753","last_peered":"2021-02-10 11:49:20.506753","last_clean":"2021-02-10 11:49:20.506753","last_became_active":"2021-02-10 11:49:18.171627","last_became_peered":"2021-02-10 11:49:18.171627","last_unstale":"2021-02-10 11:49:20.506753","last_undegraded":"2021-02-10 11:49:20.506753","last_fullsized":"2021-02-10 11:49:20.506753","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:17.168696","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:17.168696","last_clean_scrub_stamp":"2021-02-10 11:49:17.168696","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":566,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"43.7","version":"199'14","reported_seq":"30","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:34.469585","last_change":"2021-02-10 11:49:18.837246","last_active":"2021-02-10 11:50:34.469585","last_peered":"2021-02-10 11:50:34.469585","last_clean":"2021-02-10 11:50:34.469585","last_became_active":"2021-02-10 11:49:18.836985","last_became_peered":"2021-02-10 11:49:18.836985","last_unstale":"2021-02-10 11:50:34.469585","last_undegraded":"2021-02-10 11:50:34.469585","last_fullsized":"2021-02-10 11:50:34.469585","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:18.173888","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:18.173888","last_clean_scrub_stamp":"2021-02-10 11:49:18.173888","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":14680064,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":7,"num_read_kb":7168,"num_write":14,"num_write_kb":14336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.6","version":"199'17","reported_seq":"27","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:38.163378","last_change":"2021-02-10 11:49:18.172149","last_active":"2021-02-10 11:50:38.163378","last_peered":"2021-02-10 11:50:38.163378","last_clean":"2021-02-10 11:50:38.163378","last_became_active":"2021-02-10 11:49:18.171957","last_became_peered":"2021-02-10 11:49:18.171957","last_unstale":"2021-02-10 11:50:38.163378","last_undegraded":"2021-02-10 11:50:38.163378","last_fullsized":"2021-02-10 11:50:38.163378","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:17.168696","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:17.168696","last_clean_scrub_stamp":"2021-02-10 11:49:17.168696","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"43.5","version":"199'23","reported_seq":"41","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:34.507597","last_change":"2021-02-10 11:49:18.840653","last_active":"2021-02-10 11:50:34.507597","last_peered":"2021-02-10 11:50:34.507597","last_clean":"2021-02-10 11:50:34.507597","last_became_active":"2021-02-10 11:49:18.840373","last_became_peered":"2021-02-10 11:49:18.840373","last_unstale":"2021-02-10 11:50:34.507597","last_undegraded":"2021-02-10 11:50:34.507597","last_fullsized":"2021-02-10 11:50:34.507597","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:18.173888","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:18.173888","last_clean_scrub_stamp":"2021-02-10 11:49:18.173888","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777506,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":10,"num_read_kb":10240,"num_write":20,"num_write_kb":16387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.4","version":"199'229","reported_seq":"242","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:53.165380","last_change":"2021-02-10 11:50:52.689551","last_active":"2021-02-10 11:50:53.165380","last_peered":"2021-02-10 11:50:53.165380","last_clean":"2021-02-10 11:50:53.165380","last_became_active":"2021-02-10 11:49:18.171030","last_became_peered":"2021-02-10 11:49:18.171030","last_unstale":"2021-02-10 11:50:53.165380","last_undegraded":"2021-02-10 11:50:53.165380","last_fullsized":"2021-02-10 11:50:53.165380","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'226","last_scrub_stamp":"2021-02-10 11:50:52.689518","last_deep_scrub":"199'226","last_deep_scrub_stamp":"2021-02-10 11:50:52.689518","last_clean_scrub_stamp":"2021-02-10 11:50:52.689518","log_size":229,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":940641,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":226,"num_write_kb":987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"43.2","version":"199'20","reported_seq":"37","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:34.448896","last_change":"2021-02-10 11:49:18.841296","last_active":"2021-02-10 11:50:34.448896","last_peered":"2021-02-10 11:50:34.448896","last_clean":"2021-02-10 11:50:34.448896","last_became_active":"2021-02-10 11:49:18.841126","last_became_peered":"2021-02-10 11:49:18.841126","last_unstale":"2021-02-10 11:50:34.448896","last_undegraded":"2021-02-10 11:50:34.448896","last_fullsized":"2021-02-10 11:50:34.448896","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:18.173888","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:18.173888","last_clean_scrub_stamp":"2021-02-10 11:49:18.173888","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":8,"num_read_kb":8192,"num_write":20,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.3","version":"199'6","reported_seq":"16","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:49:20.506531","last_change":"2021-02-10 11:49:18.172382","last_active":"2021-02-10 11:49:20.506531","last_peered":"2021-02-10 11:49:20.506531","last_clean":"2021-02-10 11:49:20.506531","last_became_active":"2021-02-10 11:49:18.172169","last_became_peered":"2021-02-10 11:49:18.172169","last_unstale":"2021-02-10 11:49:20.506531","last_undegraded":"2021-02-10 11:49:20.506531","last_fullsized":"2021-02-10 11:49:20.506531","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:17.168696","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:17.168696","last_clean_scrub_stamp":"2021-02-10 11:49:17.168696","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"43.3","version":"199'25","reported_seq":"42","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:34.512593","last_change":"2021-02-10 11:49:18.837187","last_active":"2021-02-10 11:50:34.512593","last_peered":"2021-02-10 11:50:34.512593","last_clean":"2021-02-10 11:50:34.512593","last_became_active":"2021-02-10 11:49:18.836982","last_became_peered":"2021-02-10 11:49:18.836982","last_unstale":"2021-02-10 11:50:34.512593","last_undegraded":"2021-02-10 11:50:34.512593","last_fullsized":"2021-02-10 11:50:34.512593","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:18.173888","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:18.173888","last_clean_scrub_stamp":"2021-02-10 11:49:18.173888","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18874505,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":8192,"num_write":25,"num_write_kb":18436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.2","version":"198'1","reported_seq":"14","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:53.457772","last_change":"2021-02-10 11:50:53.457803","last_active":"2021-02-10 11:50:53.457772","last_peered":"2021-02-10 11:50:53.457772","last_clean":"2021-02-10 11:50:53.457772","last_became_active":"2021-02-10 11:49:18.181627","last_became_peered":"2021-02-10 11:49:18.181627","last_unstale":"2021-02-10 11:50:53.457772","last_undegraded":"2021-02-10 11:50:53.457772","last_fullsized":"2021-02-10 11:50:53.457772","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'1","last_scrub_stamp":"2021-02-10 11:50:53.457769","last_deep_scrub":"198'1","last_deep_scrub_stamp":"2021-02-10 11:50:53.457769","last_clean_scrub_stamp":"2021-02-10 11:50:53.457769","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"43.0","version":"199'19","reported_seq":"38","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:34.414507","last_change":"2021-02-10 11:49:18.831187","last_active":"2021-02-10 11:50:34.414507","last_peered":"2021-02-10 11:50:34.414507","last_clean":"2021-02-10 11:50:34.414507","last_became_active":"2021-02-10 11:49:18.830931","last_became_peered":"2021-02-10 11:49:18.830931","last_unstale":"2021-02-10 11:50:34.414507","last_undegraded":"2021-02-10 11:50:34.414507","last_fullsized":"2021-02-10 11:50:34.414507","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:18.173888","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:18.173888","last_clean_scrub_stamp":"2021-02-10 11:49:18.173888","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18874368,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":10,"num_read_kb":10240,"num_write":18,"num_write_kb":18432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.1","version":"199'2","reported_seq":"13","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:35.851918","last_change":"2021-02-10 11:49:18.181797","last_active":"2021-02-10 11:50:35.851918","last_peered":"2021-02-10 11:50:35.851918","last_clean":"2021-02-10 11:50:35.851918","last_became_active":"2021-02-10 11:49:18.181580","last_became_peered":"2021-02-10 11:49:18.181580","last_unstale":"2021-02-10 11:50:35.851918","last_undegraded":"2021-02-10 11:50:35.851918","last_fullsized":"2021-02-10 11:50:35.851918","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:17.168696","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:17.168696","last_clean_scrub_stamp":"2021-02-10 11:49:17.168696","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":101,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"43.1","version":"199'11","reported_seq":"26","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:34.479243","last_change":"2021-02-10 11:49:18.829002","last_active":"2021-02-10 11:50:34.479243","last_peered":"2021-02-10 11:50:34.479243","last_clean":"2021-02-10 11:50:34.479243","last_became_active":"2021-02-10 11:49:18.828750","last_became_peered":"2021-02-10 11:49:18.828750","last_unstale":"2021-02-10 11:50:34.479243","last_undegraded":"2021-02-10 11:50:34.479243","last_fullsized":"2021-02-10 11:50:34.479243","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:18.173888","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:18.173888","last_clean_scrub_stamp":"2021-02-10 11:49:18.173888","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10485760,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":6,"num_read_kb":6144,"num_write":10,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.0","version":"198'8","reported_seq":"18","reported_epoch":"198","state":"active+clean","last_fresh":"2021-02-10 11:49:18.832529","last_change":"2021-02-10 11:49:18.171195","last_active":"2021-02-10 11:49:18.832529","last_peered":"2021-02-10 11:49:18.832529","last_clean":"2021-02-10 11:49:18.832529","last_became_active":"2021-02-10 11:49:18.170941","last_became_peered":"2021-02-10 11:49:18.170941","last_unstale":"2021-02-10 11:49:18.832529","last_undegraded":"2021-02-10 11:49:18.832529","last_fullsized":"2021-02-10 11:49:18.832529","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:17.168696","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:17.168696","last_clean_scrub_stamp":"2021-02-10 11:49:17.168696","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"178","reported_epoch":"198","state":"active+clean","last_fresh":"2021-02-10 11:49:18.827035","last_change":"2021-02-10 11:13:09.495109","last_active":"2021-02-10 11:49:18.827035","last_peered":"2021-02-10 11:49:18.827035","last_clean":"2021-02-10 11:49:18.827035","last_became_active":"2021-02-10 11:13:09.494906","last_became_peered":"2021-02-10 11:13:09.494906","last_unstale":"2021-02-10 11:49:18.827035","last_undegraded":"2021-02-10 11:49:18.827035","last_fullsized":"2021-02-10 11:49:18.827035","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"181","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:53.363802","last_change":"2021-02-10 11:50:53.363832","last_active":"2021-02-10 11:50:53.363802","last_peered":"2021-02-10 11:50:53.363802","last_clean":"2021-02-10 11:50:53.363802","last_became_active":"2021-02-10 11:13:09.495226","last_became_peered":"2021-02-10 11:13:09.495226","last_unstale":"2021-02-10 11:50:53.363802","last_undegraded":"2021-02-10 11:50:53.363802","last_fullsized":"2021-02-10 11:50:53.363802","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:50:53.363799","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:50:53.363799","last_clean_scrub_stamp":"2021-02-10 11:50:53.363799","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"192","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:51.671246","last_change":"2021-02-10 11:50:51.671279","last_active":"2021-02-10 11:50:51.671246","last_peered":"2021-02-10 11:50:51.671246","last_clean":"2021-02-10 11:50:51.671246","last_became_active":"2021-02-10 11:13:09.495041","last_became_peered":"2021-02-10 11:13:09.495041","last_unstale":"2021-02-10 11:50:51.671246","last_undegraded":"2021-02-10 11:50:51.671246","last_fullsized":"2021-02-10 11:50:51.671246","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:50:51.671243","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:50:51.671243","last_clean_scrub_stamp":"2021-02-10 11:50:51.671243","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"186","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:53.890084","last_change":"2021-02-10 11:50:53.890143","last_active":"2021-02-10 11:50:53.890084","last_peered":"2021-02-10 11:50:53.890084","last_clean":"2021-02-10 11:50:53.890084","last_became_active":"2021-02-10 11:13:09.487627","last_became_peered":"2021-02-10 11:13:09.487627","last_unstale":"2021-02-10 11:50:53.890084","last_undegraded":"2021-02-10 11:50:53.890084","last_fullsized":"2021-02-10 11:50:53.890084","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:50:53.890064","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:50:53.890064","last_clean_scrub_stamp":"2021-02-10 11:50:53.890064","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"177","reported_epoch":"198","state":"active+clean","last_fresh":"2021-02-10 11:49:18.828364","last_change":"2021-02-10 11:13:09.495589","last_active":"2021-02-10 11:49:18.828364","last_peered":"2021-02-10 11:49:18.828364","last_clean":"2021-02-10 11:49:18.828364","last_became_active":"2021-02-10 11:13:09.495399","last_became_peered":"2021-02-10 11:13:09.495399","last_unstale":"2021-02-10 11:49:18.828364","last_undegraded":"2021-02-10 11:49:18.828364","last_fullsized":"2021-02-10 11:49:18.828364","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"43.4","version":"199'10","reported_seq":"25","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:34.433388","last_change":"2021-02-10 11:49:18.837659","last_active":"2021-02-10 11:50:34.433388","last_peered":"2021-02-10 11:50:34.433388","last_clean":"2021-02-10 11:50:34.433388","last_became_active":"2021-02-10 11:49:18.837387","last_became_peered":"2021-02-10 11:49:18.837387","last_unstale":"2021-02-10 11:50:34.433388","last_undegraded":"2021-02-10 11:50:34.433388","last_fullsized":"2021-02-10 11:50:34.433388","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:18.173888","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:18.173888","last_clean_scrub_stamp":"2021-02-10 11:49:18.173888","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10485760,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":6,"num_read_kb":6144,"num_write":10,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.5","version":"199'9","reported_seq":"19","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:35.840310","last_change":"2021-02-10 11:49:18.171763","last_active":"2021-02-10 11:50:35.840310","last_peered":"2021-02-10 11:50:35.840310","last_clean":"2021-02-10 11:50:35.840310","last_became_active":"2021-02-10 11:49:18.171537","last_became_peered":"2021-02-10 11:49:18.171537","last_unstale":"2021-02-10 11:50:35.840310","last_undegraded":"2021-02-10 11:50:35.840310","last_fullsized":"2021-02-10 11:50:35.840310","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:49:17.168696","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:49:17.168696","last_clean_scrub_stamp":"2021-02-10 11:49:17.168696","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"192","reported_epoch":"198","state":"active+clean","last_fresh":"2021-02-10 11:49:18.826272","last_change":"2021-02-10 11:13:09.495570","last_active":"2021-02-10 11:49:18.826272","last_peered":"2021-02-10 11:49:18.826272","last_clean":"2021-02-10 11:49:18.826272","last_became_active":"2021-02-10 11:13:09.495383","last_became_peered":"2021-02-10 11:13:09.495383","last_unstale":"2021-02-10 11:49:18.826272","last_undegraded":"2021-02-10 11:49:18.826272","last_fullsized":"2021-02-10 11:49:18.826272","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"177","reported_epoch":"198","state":"active+clean","last_fresh":"2021-02-10 11:49:18.828366","last_change":"2021-02-10 11:13:09.495605","last_active":"2021-02-10 11:49:18.828366","last_peered":"2021-02-10 11:49:18.828366","last_clean":"2021-02-10 11:49:18.828366","last_became_active":"2021-02-10 11:13:09.495261","last_became_peered":"2021-02-10 11:13:09.495261","last_unstale":"2021-02-10 11:49:18.828366","last_undegraded":"2021-02-10 11:49:18.828366","last_fullsized":"2021-02-10 11:49:18.828366","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:13:08.474168","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:13:08.474168","last_clean_scrub_stamp":"2021-02-10 11:13:08.474168","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"178","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:50.109695","last_change":"2021-02-10 11:50:50.109727","last_active":"2021-02-10 11:50:50.109695","last_peered":"2021-02-10 11:50:50.109695","last_clean":"2021-02-10 11:50:50.109695","last_became_active":"2021-02-10 11:13:09.488122","last_became_peered":"2021-02-10 11:13:09.488122","last_unstale":"2021-02-10 11:50:50.109695","last_undegraded":"2021-02-10 11:50:50.109695","last_fullsized":"2021-02-10 11:50:50.109695","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:50:50.109692","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:50:50.109692","last_clean_scrub_stamp":"2021-02-10 11:50:50.109692","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":43,"num_pg":8,"stat_sum":{"num_bytes":134218155,"num_objects":131,"num_object_clones":1,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":0,"num_read":64,"num_read_kb":65536,"num_write":139,"num_write_kb":131079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":268460032,"data_stored":268436310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":16,"acting":16,"num_store_stats":8},{"poolid":42,"num_pg":8,"stat_sum":{"num_bytes":941502,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":299,"num_write_kb":1015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1949696,"data_stored":1883004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575306,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1087976,"kb_used_data":39392,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93283864,"statfs":{"total":96636764160,"available":95522676736,"internally_reserved":1073741824,"allocated":40337408,"data_stored":38854895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 10 11:50:13 2021","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.73099999999999998,"15min":0.76500000000000001},"min":{"1min":0.373,"5min":0.36899999999999999,"15min":0.36199999999999999},"max":{"1min":1.0720000000000001,"5min":1.2290000000000001,"15min":1.232},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77900000000000003,"15min":0.79500000000000004},"min":{"1min":0.442,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.1499999999999999,"5min":1.228,"15min":1.24},"last":0.995}]},{"osd":1,"last update":"Wed Feb 10 11:50:13 2021","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.78900000000000003,"15min":0.81999999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.98099999999999998,"5min":1.179,"15min":1.2789999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.79200000000000004,"15min":0.82499999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0389999999999999,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.224}]},{"osd":2,"last update":"Wed Feb 10 11:50:13 2021","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.81299999999999994,"15min":0.84999999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.99099999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.087},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.41799999999999998},"max":{"1min":1.01,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.97899999999999998}]},{"osd":3,"last update":"Wed Feb 10 11:50:13 2021","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.90100000000000002,"15min":0.91600000000000004},"min":{"1min":0.49399999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1000000000000001,"5min":1.427,"15min":1.427},"last":1.153},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.94699999999999995,"15min":0.95799999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.442},"max":{"1min":1.101,"5min":1.333,"15min":1.3520000000000001},"last":1.165}]},{"osd":4,"last update":"Wed Feb 10 11:50:13 2021","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70199999999999996,"15min":0.69699999999999995},"min":{"1min":0.47499999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90200000000000002,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.52000000000000002,"5min":0.46899999999999997,"15min":0.41899999999999998},"max":{"1min":1.008,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.62}]},{"osd":5,"last update":"Wed Feb 10 11:50:13 2021","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79100000000000004,"15min":0.81200000000000006},"min":{"1min":0.44,"5min":0.434,"15min":0.42799999999999999},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.86799999999999999,"15min":0.879},"min":{"1min":0.54300000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.089,"5min":1.5,"15min":1.5},"last":0.96399999999999997}]},{"osd":6,"last update":"Wed Feb 10 11:50:13 2021","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.81899999999999995,"15min":0.80300000000000005},"min":{"1min":0.52000000000000002,"5min":0.5,"15min":0.48499999999999999},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.84699999999999998,"15min":0.83999999999999997},"min":{"1min":0.53100000000000003,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":1.077,"5min":1.363,"15min":1.363},"last":0.84099999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575306,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1070456,"kb_used_data":21872,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93301384,"statfs":{"total":96636764160,"available":95540617216,"internally_reserved":1073741824,"allocated":22396928,"data_stored":20920245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 10 11:50:20 2021","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.91100000000000003,"15min":0.92000000000000004},"min":{"1min":0.45600000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.403,"5min":1.589,"15min":1.589},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.93899999999999995,"15min":0.95699999999999996},"min":{"1min":0.58599999999999997,"5min":0.44400000000000001,"15min":0.40200000000000002},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.5229999999999999},"last":0.46899999999999997}]},{"osd":1,"last update":"Wed Feb 10 11:50:20 2021","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91100000000000003,"15min":0.90300000000000002},"min":{"1min":0.59799999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.226,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95899999999999996,"15min":0.94999999999999996},"min":{"1min":0.501,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2849999999999999,"5min":1.591,"15min":1.591},"last":0.78500000000000003}]},{"osd":2,"last update":"Wed Feb 10 11:50:20 2021","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.82799999999999996,"15min":0.82599999999999996},"min":{"1min":0.55200000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.151,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.872,"5min":0.86099999999999999,"15min":0.84599999999999997},"min":{"1min":0.55300000000000005,"5min":0.44500000000000001,"15min":0.40200000000000002},"max":{"1min":1.333,"5min":1.333,"15min":1.4570000000000001},"last":0.59799999999999998}]},{"osd":3,"last update":"Wed Feb 10 11:50:20 2021","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.84199999999999997,"15min":0.85199999999999998},"min":{"1min":0.52500000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1819999999999999,"5min":1.2430000000000001,"15min":1.383},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.83999999999999997,"15min":0.83099999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.41799999999999998},"max":{"1min":1.167,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.63500000000000001}]},{"osd":4,"last update":"Wed Feb 10 11:50:20 2021","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69999999999999996,"15min":0.72399999999999998},"min":{"1min":0.498,"5min":0.40799999999999997,"15min":0.38900000000000001},"max":{"1min":0.98999999999999999,"5min":1.2669999999999999,"15min":1.2949999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.51200000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94299999999999995,"5min":1.145,"15min":1.145},"last":0.81699999999999995}]},{"osd":5,"last update":"Wed Feb 10 11:50:20 2021","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.47899999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92900000000000005,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.80700000000000005,"15min":0.79600000000000004},"min":{"1min":0.64900000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.0640000000000001,"5min":1.506,"15min":1.506},"last":0.89200000000000002}]},{"osd":7,"last update":"Wed Feb 10 11:50:20 2021","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.86399999999999999,"15min":0.871},"min":{"1min":0.47999999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.022,"5min":1.351,"15min":1.351},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.89800000000000002,"15min":0.89400000000000002},"min":{"1min":0.59799999999999998,"5min":0.47599999999999998,"15min":0.47299999999999998},"max":{"1min":1.127,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.96399999999999997}]}]},{"osd":5,"up_from":13,"seq":55834575306,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1083868,"kb_used_data":35284,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93287972,"statfs":{"total":96636764160,"available":95526883328,"internally_reserved":1073741824,"allocated":36130816,"data_stored":34659794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 10 11:50:19 2021","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.373},"max":{"1min":1.2390000000000001,"5min":1.393,"15min":1.393},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.81899999999999995,"15min":0.82799999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.1120000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.496}]},{"osd":1,"last update":"Wed Feb 10 11:50:19 2021","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.81499999999999995,"15min":0.83799999999999997},"min":{"1min":0.38200000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.135,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.874,"5min":0.82899999999999996,"15min":0.84299999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.41699999999999998},"max":{"1min":1.22,"5min":1.222,"15min":1.2350000000000001},"last":0.54700000000000004}]},{"osd":2,"last update":"Wed Feb 10 11:50:19 2021","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.79500000000000004,"15min":0.80900000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.151,"5min":1.2789999999999999,"15min":1.331},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.87,"15min":0.86699999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.349,"5min":1.349,"15min":1.3740000000000001},"last":0.57799999999999996}]},{"osd":3,"last update":"Wed Feb 10 11:50:19 2021","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.86499999999999999,"15min":0.875},"min":{"1min":0.64400000000000002,"5min":0.47999999999999998,"15min":0.41899999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.427},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.83299999999999996,"15min":0.84899999999999998},"min":{"1min":0.44800000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.077,"5min":1.335,"15min":1.335},"last":0.624}]},{"osd":4,"last update":"Wed Feb 10 11:50:19 2021","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.73199999999999998,"15min":0.71399999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.36799999999999999},"max":{"1min":0.95899999999999996,"5min":1.161,"15min":1.216},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.45400000000000001,"5min":0.42099999999999999,"15min":0.35499999999999998},"max":{"1min":1.0580000000000001,"5min":1.196,"15min":1.3560000000000001},"last":0.67200000000000004}]},{"osd":6,"last update":"Wed Feb 10 11:50:19 2021","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.71999999999999997,"15min":0.71499999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.39100000000000001},"max":{"1min":0.86699999999999999,"5min":1.276,"15min":1.276},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.79900000000000004,"15min":0.79000000000000004},"min":{"1min":0.54800000000000004,"5min":0.47999999999999998,"15min":0.41799999999999998},"max":{"1min":1.075,"5min":1.2649999999999999,"15min":1.3999999999999999},"last":0.86599999999999999}]},{"osd":7,"last update":"Wed Feb 10 11:50:19 2021","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.82499999999999996,"15min":0.81100000000000005},"min":{"1min":0.53800000000000003,"5min":0.48599999999999999,"15min":0.46899999999999997},"max":{"1min":1.006,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.79000000000000004,"15min":0.79300000000000004},"min":{"1min":0.55500000000000005,"5min":0.46700000000000003,"15min":0.45900000000000002},"max":{"1min":0.97799999999999998,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.78100000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575306,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1074548,"kb_used_data":25964,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93297292,"statfs":{"total":96636764160,"available":95536427008,"internally_reserved":1073741824,"allocated":26587136,"data_stored":25114515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 10 11:50:03 2021","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.85199999999999998,"15min":0.89900000000000002},"min":{"1min":0.38700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.3380000000000001,"5min":1.411,"15min":1.6850000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.93799999999999994,"15min":0.94399999999999995},"min":{"1min":0.503,"5min":0.42299999999999999,"15min":0.40100000000000002},"max":{"1min":1.2989999999999999,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.60099999999999998}]},{"osd":1,"last update":"Wed Feb 10 11:50:03 2021","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.83699999999999997,"15min":0.83199999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40000000000000002},"max":{"1min":1.244,"5min":1.2509999999999999,"15min":1.468},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.81499999999999995,"15min":0.82699999999999996},"min":{"1min":0.47599999999999998,"5min":0.371,"15min":0.36399999999999999},"max":{"1min":1.167,"5min":1.2629999999999999,"15min":1.49},"last":0.438}]},{"osd":2,"last update":"Wed Feb 10 11:50:03 2021","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.80300000000000005,"15min":0.81799999999999995},"min":{"1min":0.48999999999999999,"5min":0.438,"15min":0.36599999999999999},"max":{"1min":1.284,"5min":1.2929999999999999,"15min":1.5660000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.82399999999999995,"15min":0.83199999999999996},"min":{"1min":0.501,"5min":0.42199999999999999,"15min":0.41999999999999998},"max":{"1min":1.1679999999999999,"5min":1.254,"15min":1.607},"last":0.40999999999999998}]},{"osd":3,"last update":"Wed Feb 10 11:50:03 2021","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.81200000000000006,"15min":0.82399999999999995},"min":{"1min":0.436,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.194,"5min":1.2709999999999999,"15min":1.6319999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.84199999999999997,"15min":0.85999999999999999},"min":{"1min":0.52000000000000002,"5min":0.46000000000000002,"15min":0.45800000000000002},"max":{"1min":1.204,"5min":1.3029999999999999,"15min":1.482},"last":0.67500000000000004}]},{"osd":5,"last update":"Wed Feb 10 11:50:03 2021","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69699999999999995,"15min":0.71299999999999997},"min":{"1min":0.499,"5min":0.36099999999999999,"15min":0.35699999999999998},"max":{"1min":1.0589999999999999,"5min":1.1699999999999999,"15min":1.236},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71099999999999997,"15min":0.70899999999999996},"min":{"1min":0.52200000000000002,"5min":0.45800000000000002,"15min":0.40100000000000002},"max":{"1min":1.111,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.66300000000000003}]},{"osd":6,"last update":"Wed Feb 10 11:50:03 2021","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.70599999999999996,"15min":0.73099999999999998},"min":{"1min":0.53600000000000003,"5min":0.45000000000000001,"15min":0.42299999999999999},"max":{"1min":0.92000000000000004,"5min":1.24,"15min":1.9590000000000001},"last":0.621},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.81599999999999995,"15min":0.80100000000000005},"min":{"1min":0.59599999999999997,"5min":0.46899999999999997,"15min":0.45300000000000001},"max":{"1min":1.1439999999999999,"5min":1.262,"15min":3.351},"last":0.65400000000000003}]},{"osd":7,"last update":"Wed Feb 10 11:50:03 2021","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.78200000000000003,"15min":0.79200000000000004},"min":{"1min":0.51900000000000002,"5min":0.502,"15min":0.48299999999999998},"max":{"1min":1.306,"5min":1.306,"15min":1.504},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80400000000000005,"15min":0.82799999999999996},"min":{"1min":0.59799999999999998,"5min":0.52000000000000002,"15min":0.46899999999999997},"max":{"1min":1.103,"5min":1.181,"15min":1.3149999999999999},"last":0.75800000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575305,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1061532,"kb_used_data":12752,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93310308,"statfs":{"total":96636764160,"available":95549755392,"internally_reserved":1073741824,"allocated":13058048,"data_stored":11590996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 10 11:50:24 2021","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56100000000000005,"15min":0.57999999999999996},"min":{"1min":0.29699999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.96799999999999997,"5min":0.96899999999999997,"15min":1.0129999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.58499999999999996,"15min":0.61099999999999999},"min":{"1min":0.39500000000000002,"5min":0.34100000000000003,"15min":0.32700000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":1.196},"last":0.52300000000000002}]},{"osd":2,"last update":"Wed Feb 10 11:50:24 2021","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.35099999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.92300000000000004},"last":0.627},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.308},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.98699999999999999},"last":0.43099999999999999}]},{"osd":3,"last update":"Wed Feb 10 11:50:24 2021","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.505,"15min":0.51200000000000001},"min":{"1min":0.36799999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77500000000000002,"5min":0.878,"15min":0.998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.437,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84799999999999998,"5min":0.86599999999999999,"15min":1.036},"last":0.74199999999999999}]},{"osd":4,"last update":"Wed Feb 10 11:50:24 2021","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.95599999999999996,"15min":0.95099999999999996},"min":{"1min":0.55700000000000005,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.298,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.075},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.92200000000000004,"15min":0.93200000000000005},"min":{"1min":0.56499999999999995,"5min":0.49099999999999999,"15min":0.47399999999999998},"max":{"1min":1.21,"5min":1.4179999999999999,"15min":1.49},"last":1.048}]},{"osd":5,"last update":"Wed Feb 10 11:50:24 2021","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.94299999999999995,"15min":0.94999999999999996},"min":{"1min":0.627,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1839999999999999,"5min":1.347,"15min":1.347},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.95099999999999996,"15min":0.97099999999999997},"min":{"1min":0.61599999999999999,"5min":0.55600000000000005,"15min":0.54300000000000004},"max":{"1min":1.248,"5min":1.502,"15min":1.6180000000000001},"last":0.88500000000000001}]},{"osd":6,"last update":"Wed Feb 10 11:50:24 2021","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.97299999999999998,"15min":0.97999999999999998},"min":{"1min":0.63700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.272,"5min":1.4550000000000001,"15min":1.5469999999999999},"last":0.998},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.96099999999999997,"15min":0.97399999999999998},"min":{"1min":0.67700000000000005,"5min":0.57699999999999996,"15min":0.53900000000000003},"max":{"1min":1.2969999999999999,"5min":1.3400000000000001,"15min":1.444},"last":0.877}]},{"osd":7,"last update":"Wed Feb 10 11:50:24 2021","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.98899999999999999,"15min":0.996},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.495},"max":{"1min":1.2549999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.97699999999999998,"15min":0.97999999999999998},"min":{"1min":0.64700000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.3560000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.94499999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575305,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1092068,"kb_used_data":43480,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93279772,"statfs":{"total":96636764160,"available":95518486528,"internally_reserved":1073741824,"allocated":44523520,"data_stored":43048633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 10 11:50:18 2021","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.52600000000000002,"15min":0.505},"min":{"1min":0.371,"5min":0.32600000000000001,"15min":0.30099999999999999},"max":{"1min":0.66100000000000003,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51300000000000001,"15min":0.50900000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.309},"max":{"1min":0.874,"5min":0.874,"15min":0.90400000000000003},"last":0.55800000000000005}]},{"osd":2,"last update":"Wed Feb 10 11:50:18 2021","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56999999999999995,"15min":0.55700000000000005},"min":{"1min":0.39700000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57199999999999995,"15min":0.56399999999999995},"min":{"1min":0.41299999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86099999999999999,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.434}]},{"osd":3,"last update":"Wed Feb 10 11:50:18 2021","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61799999999999999,"15min":0.61199999999999999},"min":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.38100000000000001},"max":{"1min":0.99299999999999999,"5min":1.056,"15min":1.056},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65000000000000002,"15min":0.63300000000000001},"min":{"1min":0.45500000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.075,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.72299999999999998}]},{"osd":4,"last update":"Wed Feb 10 11:50:18 2021","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.0369999999999999,"15min":1.0009999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.61299999999999999},"max":{"1min":1.284,"5min":1.609,"15min":1.629},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0369999999999999,"15min":1.0049999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.42299999999999999},"max":{"1min":1.2210000000000001,"5min":1.4790000000000001,"15min":1.6299999999999999},"last":0.76300000000000001}]},{"osd":5,"last update":"Wed Feb 10 11:50:18 2021","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1359999999999999,"15min":1.115},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3240000000000001,"5min":1.518,"15min":1.665},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.083,"5min":1.1240000000000001,"15min":1.107},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.431,"5min":1.694,"15min":1.74},"last":0.77500000000000002}]},{"osd":6,"last update":"Wed Feb 10 11:50:18 2021","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.03,"15min":1.0069999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.53000000000000003},"max":{"1min":1.2529999999999999,"5min":1.46,"15min":1.5700000000000001},"last":0.624},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.0489999999999999,"15min":1.0269999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.268,"5min":1.613,"15min":1.613},"last":0.79200000000000004}]},{"osd":7,"last update":"Wed Feb 10 11:50:18 2021","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.0409999999999999,"15min":1.0309999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.59299999999999997},"max":{"1min":1.331,"5min":1.456,"15min":1.482},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.056,"15min":1.0329999999999999},"min":{"1min":0.73499999999999999,"5min":0.68200000000000005,"15min":0.55300000000000005},"max":{"1min":1.1830000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.85699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575306,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1122792,"kb_used_data":74208,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93249048,"statfs":{"total":96636764160,"available":95487025152,"internally_reserved":1073741824,"allocated":75988992,"data_stored":74506385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 10 11:49:56 2021","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51200000000000001,"15min":0.497},"min":{"1min":0.33300000000000002,"5min":0.29999999999999999,"15min":0.27100000000000002},"max":{"1min":0.91100000000000003,"5min":1.069,"15min":1.069},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.50700000000000001,"15min":0.503},"min":{"1min":0.38800000000000001,"5min":0.32700000000000001,"15min":0.29999999999999999},"max":{"1min":0.73699999999999999,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48699999999999999}]},{"osd":1,"last update":"Wed Feb 10 11:49:56 2021","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":1.0209999999999999,"5min":1.131,"15min":1.131},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.443,"5min":0.33000000000000002,"15min":0.32700000000000001},"max":{"1min":0.70699999999999996,"5min":0.89600000000000002,"15min":1.016},"last":0.371}]},{"osd":3,"last update":"Wed Feb 10 11:49:56 2021","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.32000000000000001},"max":{"1min":0.94499999999999995,"5min":1.097,"15min":1.097},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.38100000000000001,"5min":0.371,"15min":0.34499999999999997},"max":{"1min":0.70499999999999996,"5min":1.252,"15min":1.252},"last":0.499}]},{"osd":4,"last update":"Wed Feb 10 11:49:56 2021","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.98099999999999998,"15min":0.97099999999999997},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.495},"max":{"1min":1.272,"5min":1.393,"15min":1.4470000000000001},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.99199999999999999,"15min":0.98799999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3340000000000001,"5min":1.343,"15min":1.417},"last":1.1299999999999999}]},{"osd":5,"last update":"Wed Feb 10 11:49:56 2021","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.113,"15min":1.095},"min":{"1min":0.66700000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.4139999999999999,"5min":1.4550000000000001,"15min":1.5249999999999999},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.173,"5min":1.137,"15min":1.123},"min":{"1min":0.77000000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.476,"5min":1.476,"15min":1.5149999999999999},"last":1.4530000000000001}]},{"osd":6,"last update":"Wed Feb 10 11:49:56 2021","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1,"15min":0.99299999999999999},"min":{"1min":0.64600000000000002,"5min":0.61199999999999999,"15min":0.504},"max":{"1min":1.3340000000000001,"5min":1.3500000000000001,"15min":1.381},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.0049999999999999,"15min":1.0089999999999999},"min":{"1min":0.76300000000000001,"5min":0.60399999999999998,"15min":0.53700000000000003},"max":{"1min":1.2989999999999999,"5min":1.4470000000000001,"15min":1.472},"last":1.141}]},{"osd":7,"last update":"Wed Feb 10 11:49:56 2021","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.038,"15min":1.02},"min":{"1min":0.752,"5min":0.60699999999999998,"15min":0.55500000000000005},"max":{"1min":1.3580000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.327},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0189999999999999,"15min":0.996},"min":{"1min":0.68600000000000005,"5min":0.64800000000000002,"15min":0.58899999999999997},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.4339999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575306,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1079768,"kb_used_data":31184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93292072,"statfs":{"total":96636764160,"available":95531081728,"internally_reserved":1073741824,"allocated":31932416,"data_stored":30465467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 10 11:50:26 2021","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46600000000000003,"15min":0.46700000000000003},"min":{"1min":0.34599999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71399999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.499,"15min":0.495},"min":{"1min":0.38,"5min":0.317,"15min":0.317},"max":{"1min":0.628,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.437}]},{"osd":1,"last update":"Wed Feb 10 11:50:26 2021","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47999999999999998,"15min":0.48599999999999999},"min":{"1min":0.374,"5min":0.316,"15min":0.316},"max":{"1min":0.57099999999999995,"5min":0.77000000000000002,"15min":0.89700000000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.439,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.95899999999999996,"5min":1.0289999999999999,"15min":1.0409999999999999},"last":0.46500000000000002}]},{"osd":2,"last update":"Wed Feb 10 11:50:26 2021","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.46800000000000003,"5min":0.35399999999999998,"15min":0.34100000000000003},"max":{"1min":0.92900000000000005,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68000000000000005,"15min":0.68500000000000005},"min":{"1min":0.50700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89600000000000002,"5min":1.397,"15min":1.397},"last":0.60699999999999998}]},{"osd":4,"last update":"Wed Feb 10 11:50:26 2021","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.95199999999999996,"15min":0.95699999999999996},"min":{"1min":0.60799999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.2350000000000001,"5min":1.472,"15min":1.472},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.93799999999999994,"15min":0.94099999999999995},"min":{"1min":0.67100000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2470000000000001,"5min":1.351,"15min":1.575},"last":0.94199999999999995}]},{"osd":5,"last update":"Wed Feb 10 11:50:26 2021","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.0549999999999999,"15min":1.071},"min":{"1min":0.59599999999999997,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.431,"5min":1.621,"15min":1.621},"last":0.999},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0660000000000001,"15min":1.0600000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.48299999999999998},"max":{"1min":1.3380000000000001,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.98199999999999998}]},{"osd":6,"last update":"Wed Feb 10 11:50:26 2021","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.98399999999999999,"15min":0.95899999999999996},"min":{"1min":0.76600000000000001,"5min":0.63300000000000001,"15min":0.54500000000000004},"max":{"1min":1.222,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.024,"5min":1.01,"15min":0.98999999999999999},"min":{"1min":0.68200000000000005,"5min":0.61199999999999999,"15min":0.57999999999999996},"max":{"1min":1.288,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.95499999999999996}]},{"osd":7,"last update":"Wed Feb 10 11:50:26 2021","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1080000000000001,"15min":1.1020000000000001},"min":{"1min":0.89000000000000001,"5min":0.67200000000000004,"15min":0.63200000000000001},"max":{"1min":1.413,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.131,"5min":1.083,"15min":1.0880000000000001},"min":{"1min":0.88100000000000001,"5min":0.67700000000000005,"15min":0.67300000000000004},"max":{"1min":1.405,"5min":1.508,"15min":1.5900000000000001},"last":1.071}]}]}]}} 2021-02-10T11:50:54.690 INFO:tasks.ceph:pgid 43.6 last_scrub_stamp 2021-02-10 11:49:18.173888 time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=49, tm_sec=18, tm_wday=2, tm_yday=41, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=50, tm_sec=44, tm_wday=2, tm_yday=41, tm_isdst=0) 2021-02-10T11:50:54.690 INFO:tasks.ceph:pgid 42.7 last_scrub_stamp 2021-02-10 11:49:17.168696 time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=49, tm_sec=17, tm_wday=2, tm_yday=41, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=50, tm_sec=44, tm_wday=2, tm_yday=41, tm_isdst=0) 2021-02-10T11:50:54.690 INFO:tasks.ceph:pgid 43.7 last_scrub_stamp 2021-02-10 11:49:18.173888 time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=49, tm_sec=18, tm_wday=2, tm_yday=41, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=50, tm_sec=44, tm_wday=2, tm_yday=41, tm_isdst=0) 2021-02-10T11:50:54.690 INFO:tasks.ceph:pgid 42.6 last_scrub_stamp 2021-02-10 11:49:17.168696 time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=49, tm_sec=17, tm_wday=2, tm_yday=41, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=50, tm_sec=44, tm_wday=2, tm_yday=41, tm_isdst=0) 2021-02-10T11:50:54.691 INFO:tasks.ceph:pgid 43.5 last_scrub_stamp 2021-02-10 11:49:18.173888 time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=49, tm_sec=18, tm_wday=2, tm_yday=41, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=50, tm_sec=44, tm_wday=2, tm_yday=41, tm_isdst=0) 2021-02-10T11:50:54.691 INFO:tasks.ceph:pgid 43.2 last_scrub_stamp 2021-02-10 11:49:18.173888 time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=49, tm_sec=18, tm_wday=2, tm_yday=41, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=50, tm_sec=44, tm_wday=2, tm_yday=41, tm_isdst=0) 2021-02-10T11:50:54.691 INFO:tasks.ceph:pgid 42.3 last_scrub_stamp 2021-02-10 11:49:17.168696 time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=49, tm_sec=17, tm_wday=2, tm_yday=41, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=50, tm_sec=44, tm_wday=2, tm_yday=41, tm_isdst=0) 2021-02-10T11:50:54.691 INFO:tasks.ceph:pgid 43.3 last_scrub_stamp 2021-02-10 11:49:18.173888 time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=49, tm_sec=18, tm_wday=2, tm_yday=41, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=50, tm_sec=44, tm_wday=2, tm_yday=41, tm_isdst=0) 2021-02-10T11:50:54.691 INFO:tasks.ceph:pgid 43.0 last_scrub_stamp 2021-02-10 11:49:18.173888 time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=49, tm_sec=18, tm_wday=2, tm_yday=41, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=50, tm_sec=44, tm_wday=2, tm_yday=41, tm_isdst=0) 2021-02-10T11:50:54.691 INFO:tasks.ceph:pgid 42.1 last_scrub_stamp 2021-02-10 11:49:17.168696 time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=49, tm_sec=17, tm_wday=2, tm_yday=41, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=50, tm_sec=44, tm_wday=2, tm_yday=41, tm_isdst=0) 2021-02-10T11:50:54.692 INFO:tasks.ceph:pgid 43.1 last_scrub_stamp 2021-02-10 11:49:18.173888 time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=49, tm_sec=18, tm_wday=2, tm_yday=41, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=50, tm_sec=44, tm_wday=2, tm_yday=41, tm_isdst=0) 2021-02-10T11:50:54.692 INFO:tasks.ceph:pgid 42.0 last_scrub_stamp 2021-02-10 11:49:17.168696 time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=49, tm_sec=17, tm_wday=2, tm_yday=41, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=50, tm_sec=44, tm_wday=2, tm_yday=41, tm_isdst=0) 2021-02-10T11:50:54.692 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2021-02-10 11:13:08.474168 time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=13, tm_sec=8, tm_wday=2, tm_yday=41, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=50, tm_sec=44, tm_wday=2, tm_yday=41, tm_isdst=0) 2021-02-10T11:50:54.692 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2021-02-10 11:13:08.474168 time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=13, tm_sec=8, tm_wday=2, tm_yday=41, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=50, tm_sec=44, tm_wday=2, tm_yday=41, tm_isdst=0) 2021-02-10T11:50:54.692 INFO:tasks.ceph:pgid 43.4 last_scrub_stamp 2021-02-10 11:49:18.173888 time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=49, tm_sec=18, tm_wday=2, tm_yday=41, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=50, tm_sec=44, tm_wday=2, tm_yday=41, tm_isdst=0) 2021-02-10T11:50:54.692 INFO:tasks.ceph:pgid 42.5 last_scrub_stamp 2021-02-10 11:49:17.168696 time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=49, tm_sec=17, tm_wday=2, tm_yday=41, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=50, tm_sec=44, tm_wday=2, tm_yday=41, tm_isdst=0) 2021-02-10T11:50:54.693 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2021-02-10 11:13:08.474168 time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=13, tm_sec=8, tm_wday=2, tm_yday=41, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=50, tm_sec=44, tm_wday=2, tm_yday=41, tm_isdst=0) 2021-02-10T11:50:54.693 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2021-02-10 11:13:08.474168 time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=13, tm_sec=8, tm_wday=2, tm_yday=41, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=2, tm_mday=10, tm_hour=11, tm_min=50, tm_sec=44, tm_wday=2, tm_yday=41, tm_isdst=0) 2021-02-10T11:50:54.693 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2021-02-10T11:51:11.862 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:51:11.865 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:51:14.694 DEBUG:teuthology.orchestra.run.gibba001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph pg dump --format=json 2021-02-10T11:51:15.124 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.112 7f323426c700 1 Processor -- start 2021-02-10T11:51:15.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.112 7f323426c700 1 -- start start 2021-02-10T11:51:15.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.112 7f323426c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f322c0d0340 0x7f322c0ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:51:15.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.112 7f323426c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f322c0d1c70 0x7f322c0efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:51:15.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.113 7f323426c700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322c0f0b10 con 0x7f322c0d1870 2021-02-10T11:51:15.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.113 7f323426c700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f322c0e5ed0 con 0x7f322c0d1c70 2021-02-10T11:51:15.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.113 7f323426c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c1b0be0 con 0x7f322c0d0340 2021-02-10T11:51:15.125 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.113 7f3232889700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f322c0d1c70 0x7f322c0efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:51:15.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.113 7f3232088700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f322c0d1870 0x7f322c1b03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36980/0 (socket says 172.21.2.101:36980) 2021-02-10T11:51:15.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.113 7f3232088700 1 -- 172.21.2.101:0/178344710 learned_addr learned my addr 172.21.2.101:0/178344710 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:51:15.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.113 7f3232889700 1 --2- 172.21.2.101:0/178344710 >> v2:172.21.2.101:3300/0 conn(0x7f322c0d1c70 0x7f322c0efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36490/0 (socket says 172.21.2.101:36490) 2021-02-10T11:51:15.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.113 7f3231887700 1 --2- 172.21.2.101:0/178344710 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f322c0d0340 0x7f322c0ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:51:15.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.113 7f3232889700 1 -- 172.21.2.101:0/178344710 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f322c0d0340 msgr2=0x7f322c0ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-02-10T11:51:15.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.113 7f3232889700 1 --2- 172.21.2.101:0/178344710 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f322c0d0340 0x7f322c0ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:51:15.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.113 7f3232889700 1 -- 172.21.2.101:0/178344710 >> v1:172.21.2.101:6789/0 conn(0x7f322c0d1870 legacy=0x7f322c1b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:51:15.126 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.113 7f3232889700 1 -- 172.21.2.101:0/178344710 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c0f20a0 con 0x7f322c0d1c70 2021-02-10T11:51:15.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.113 7f3232889700 1 --2- 172.21.2.101:0/178344710 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f322c0d1c70 0x7f322c0efc60 secure :-1 s=READY pgs=3496 cs=0 l=1 rev1=1 rx=0x7f3224006e30 tx=0x7f3224009de0).ready entity=mon.0 client_cookie=63b85c73aa9b826b server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:51:15.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.113 7f3231086700 1 -- 172.21.2.101:0/178344710 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f322400b300 con 0x7f322c0d1c70 2021-02-10T11:51:15.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.113 7f3231086700 1 -- 172.21.2.101:0/178344710 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3224006a50 con 0x7f322c0d1c70 2021-02-10T11:51:15.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.113 7f3231086700 1 -- 172.21.2.101:0/178344710 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f322400b900 con 0x7f322c0d1c70 2021-02-10T11:51:15.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.113 7f323426c700 1 -- 172.21.2.101:0/178344710 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f322c0d1c70 msgr2=0x7f322c0efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:51:15.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.113 7f323426c700 1 --2- 172.21.2.101:0/178344710 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f322c0d1c70 0x7f322c0efc60 secure :-1 s=READY pgs=3496 cs=0 l=1 rev1=1 rx=0x7f3224006e30 tx=0x7f3224009de0).stop 2021-02-10T11:51:15.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.114 7f323426c700 1 -- 172.21.2.101:0/178344710 shutdown_connections 2021-02-10T11:51:15.127 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.114 7f323426c700 1 --2- 172.21.2.101:0/178344710 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f322c0d0340 0x7f322c0ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-02-10T11:51:15.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.114 7f323426c700 1 --2- 172.21.2.101:0/178344710 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f322c0d1c70 0x7f322c0efc60 unknown :-1 s=CLOSED pgs=3496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:51:15.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.114 7f323426c700 1 -- 172.21.2.101:0/178344710 >> 172.21.2.101:0/178344710 conn(0x7f322c0ce450 msgr2=0x7f322c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:51:15.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.114 7f323426c700 1 -- 172.21.2.101:0/178344710 shutdown_connections 2021-02-10T11:51:15.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.114 7f323426c700 1 -- 172.21.2.101:0/178344710 wait complete. 2021-02-10T11:51:15.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.129 7f323426c700 1 Processor -- start 2021-02-10T11:51:15.128 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.129 7f323426c700 1 -- start start 2021-02-10T11:51:15.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.129 7f323426c700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f322c0ada90 0x7f322c0ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:51:15.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.129 7f323426c700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f322c099d20 0x7f322c09a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:51:15.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.129 7f323426c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f322c09a740 0x7f322c097bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:51:15.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.129 7f323426c700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0997a0 con 0x7f322c09a740 2021-02-10T11:51:15.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.129 7f323426c700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c18f0c0 con 0x7f322c099d20 2021-02-10T11:51:15.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.129 7f323426c700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c09ab40 con 0x7f322c0ada90 2021-02-10T11:51:15.130 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.129 7f3232889700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f322c09a740 0x7f322c097bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:51:15.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.129 7f3232889700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f322c09a740 0x7f322c097bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36492/0 (socket says 172.21.2.101:36492) 2021-02-10T11:51:15.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.129 7f3232889700 1 -- 172.21.2.101:0/656538250 learned_addr learned my addr 172.21.2.101:0/656538250 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:51:15.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.129 7f3232088700 1 --2- 172.21.2.101:0/656538250 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f322c0ada90 0x7f322c0ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:51:15.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.129 7f3231887700 1 --2- 172.21.2.101:0/656538250 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f322c099d20 0x7f322c09a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:51:15.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.130 7f3232889700 1 -- 172.21.2.101:0/656538250 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f322c0ada90 msgr2=0x7f322c0ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:51:15.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.130 7f3232889700 1 --2- 172.21.2.101:0/656538250 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f322c0ada90 0x7f322c0ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:51:15.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.130 7f3232889700 1 -- 172.21.2.101:0/656538250 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f322c099d20 msgr2=0x7f322c09a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:51:15.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.130 7f3232889700 1 --2- 172.21.2.101:0/656538250 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f322c099d20 0x7f322c09a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:51:15.131 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.130 7f3232889700 1 -- 172.21.2.101:0/656538250 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c09ad60 con 0x7f322c09a740 2021-02-10T11:51:15.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.130 7f3232889700 1 --2- 172.21.2.101:0/656538250 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f322c09a740 0x7f322c097bc0 secure :-1 s=READY pgs=3497 cs=0 l=1 rev1=1 rx=0x7f3224009910 tx=0x7f322400bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:51:15.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.130 7f321effd700 1 -- 172.21.2.101:0/656538250 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3224009070 con 0x7f322c09a740 2021-02-10T11:51:15.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.130 7f321effd700 1 -- 172.21.2.101:0/656538250 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3224001270 con 0x7f322c09a740 2021-02-10T11:51:15.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.130 7f323426c700 1 -- 172.21.2.101:0/656538250 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322c09ad60 con 0x7f322c09a740 2021-02-10T11:51:15.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.130 7f321effd700 1 -- 172.21.2.101:0/656538250 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f32240017d0 con 0x7f322c09a740 2021-02-10T11:51:15.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.130 7f323426c700 1 -- 172.21.2.101:0/656538250 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322c0984d0 con 0x7f322c09a740 2021-02-10T11:51:15.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.131 7f321effd700 1 -- 172.21.2.101:0/656538250 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f3224000d30 con 0x7f322c09a740 2021-02-10T11:51:15.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.131 7f321effd700 1 --2- 172.21.2.101:0/656538250 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3214018730 0x7f321401abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:51:15.132 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.131 7f321effd700 1 -- 172.21.2.101:0/656538250 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f3224032a50 con 0x7f322c09a740 2021-02-10T11:51:15.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.131 7f3232088700 1 --2- 172.21.2.101:0/656538250 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3214018730 0x7f321401abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:51:15.133 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.131 7f3232088700 1 --2- 172.21.2.101:0/656538250 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3214018730 0x7f321401abc0 crc :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:51:15.158 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.162 7f323426c700 1 -- 172.21.2.101:0/656538250 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322c0ebc00 con 0x7f322c09a740 2021-02-10T11:51:15.160 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.165 7f321effd700 1 -- 172.21.2.101:0/656538250 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f32240093b0 con 0x7f322c09a740 2021-02-10T11:51:15.294 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.299 7f323426c700 1 -- 172.21.2.101:0/656538250 --> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] -- command(tid 0: {"prefix": "pg dump", "target": ["mgr", ""], "format": "json"}) v1 -- 0x7f322c0984d0 con 0x7f3214018730 2021-02-10T11:51:15.297 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.301 7f321effd700 1 -- 172.21.2.101:0/656538250 <== mgr.4100 v2:172.21.2.101:6800/24165 1 ==== command_reply(tid 0: 0 dumped all) v1 ==== 18+0+101274 (crc 0 0 0) 0x7f3228001a00 con 0x7f3214018730 2021-02-10T11:51:15.299 INFO:teuthology.orchestra.run.gibba001.stdout: 2021-02-10T11:51:15.303 INFO:teuthology.orchestra.run.gibba001.stderr:dumped all 2021-02-10T11:51:15.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.303 7f323426c700 1 -- 172.21.2.101:0/656538250 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3214018730 msgr2=0x7f321401abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:51:15.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.303 7f323426c700 1 --2- 172.21.2.101:0/656538250 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3214018730 0x7f321401abc0 crc :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:51:15.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.303 7f323426c700 1 -- 172.21.2.101:0/656538250 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f322c09a740 msgr2=0x7f322c097bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:51:15.303 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.303 7f323426c700 1 --2- 172.21.2.101:0/656538250 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f322c09a740 0x7f322c097bc0 secure :-1 s=READY pgs=3497 cs=0 l=1 rev1=1 rx=0x7f3224009910 tx=0x7f322400bd40).stop 2021-02-10T11:51:15.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.304 7f323426c700 1 -- 172.21.2.101:0/656538250 shutdown_connections 2021-02-10T11:51:15.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.304 7f323426c700 1 --2- 172.21.2.101:0/656538250 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f322c09a740 0x7f322c097bc0 unknown :-1 s=CLOSED pgs=3497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:51:15.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.304 7f323426c700 1 --2- 172.21.2.101:0/656538250 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f322c099d20 0x7f322c09a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:51:15.304 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.304 7f323426c700 1 --2- 172.21.2.101:0/656538250 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f3214018730 0x7f321401abc0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:51:15.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.304 7f323426c700 1 --2- 172.21.2.101:0/656538250 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f322c0ada90 0x7f322c0ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:51:15.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.304 7f323426c700 1 -- 172.21.2.101:0/656538250 >> 172.21.2.101:0/656538250 conn(0x7f322c0f0550 msgr2=0x7f322c191730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:51:15.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.304 7f323426c700 1 -- 172.21.2.101:0/656538250 shutdown_connections 2021-02-10T11:51:15.305 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.304 7f323426c700 1 -- 172.21.2.101:0/656538250 wait complete. 2021-02-10T11:51:15.307 INFO:teuthology.orchestra.run.gibba001.stdout:{"pg_ready":true,"pg_map":{"version":1348,"stamp":"2021-02-10 11:51:14.644563","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":135159657,"num_objects":154,"num_object_clones":1,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":65,"num_read_kb":65537,"num_write":438,"num_write_kb":132094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5006,"num_omap_keys":18,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":534,"ondisk_log_size":534,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"kb":754974720,"kb_used":8673008,"kb_used_data":284136,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746301712,"statfs":{"total":773094113280,"available":764212953088,"internally_reserved":8589934592,"allocated":290955264,"data_stored":279160930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5006,"num_omap_keys":18,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003426"},"pg_stats":[{"pgid":"43.6","version":"199'44","reported_seq":"65","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:51:09.914849","last_change":"2021-02-10 11:51:09.914873","last_active":"2021-02-10 11:51:09.914849","last_peered":"2021-02-10 11:51:09.914849","last_clean":"2021-02-10 11:51:09.914849","last_became_active":"2021-02-10 11:49:18.828991","last_became_peered":"2021-02-10 11:49:18.828991","last_unstale":"2021-02-10 11:51:09.914849","last_undegraded":"2021-02-10 11:51:09.914849","last_fullsized":"2021-02-10 11:51:09.914849","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'44","last_scrub_stamp":"2021-02-10 11:51:09.914846","last_deep_scrub":"199'44","last_deep_scrub_stamp":"2021-02-10 11:51:09.914846","last_clean_scrub_stamp":"2021-02-10 11:51:09.914846","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23068672,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":9,"num_read_kb":9216,"num_write":22,"num_write_kb":22528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.7","version":"199'12","reported_seq":"25","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:51:07.631445","last_change":"2021-02-10 11:51:07.631469","last_active":"2021-02-10 11:51:07.631445","last_peered":"2021-02-10 11:51:07.631445","last_clean":"2021-02-10 11:51:07.631445","last_became_active":"2021-02-10 11:49:18.171627","last_became_peered":"2021-02-10 11:49:18.171627","last_unstale":"2021-02-10 11:51:07.631445","last_undegraded":"2021-02-10 11:51:07.631445","last_fullsized":"2021-02-10 11:51:07.631445","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'12","last_scrub_stamp":"2021-02-10 11:51:07.631442","last_deep_scrub":"199'12","last_deep_scrub_stamp":"2021-02-10 11:51:07.631442","last_clean_scrub_stamp":"2021-02-10 11:51:07.631442","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":566,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"43.7","version":"199'28","reported_seq":"47","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:51:07.459649","last_change":"2021-02-10 11:51:07.459681","last_active":"2021-02-10 11:51:07.459649","last_peered":"2021-02-10 11:51:07.459649","last_clean":"2021-02-10 11:51:07.459649","last_became_active":"2021-02-10 11:49:18.836985","last_became_peered":"2021-02-10 11:49:18.836985","last_unstale":"2021-02-10 11:51:07.459649","last_undegraded":"2021-02-10 11:51:07.459649","last_fullsized":"2021-02-10 11:51:07.459649","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'28","last_scrub_stamp":"2021-02-10 11:51:07.459644","last_deep_scrub":"199'28","last_deep_scrub_stamp":"2021-02-10 11:51:07.459644","last_clean_scrub_stamp":"2021-02-10 11:51:07.459644","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":14680064,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":7,"num_read_kb":7168,"num_write":14,"num_write_kb":14336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.6","version":"199'18","reported_seq":"31","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:51:02.886542","last_change":"2021-02-10 11:51:02.886567","last_active":"2021-02-10 11:51:02.886542","last_peered":"2021-02-10 11:51:02.886542","last_clean":"2021-02-10 11:51:02.886542","last_became_active":"2021-02-10 11:49:18.171957","last_became_peered":"2021-02-10 11:49:18.171957","last_unstale":"2021-02-10 11:51:02.886542","last_undegraded":"2021-02-10 11:51:02.886542","last_fullsized":"2021-02-10 11:51:02.886542","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'18","last_scrub_stamp":"2021-02-10 11:51:02.886536","last_deep_scrub":"199'18","last_deep_scrub_stamp":"2021-02-10 11:51:02.886536","last_clean_scrub_stamp":"2021-02-10 11:51:02.886536","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":386,"num_omap_keys":8,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"43.5","version":"199'34","reported_seq":"55","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:51:02.192516","last_change":"2021-02-10 11:51:02.192539","last_active":"2021-02-10 11:51:02.192516","last_peered":"2021-02-10 11:51:02.192516","last_clean":"2021-02-10 11:51:02.192516","last_became_active":"2021-02-10 11:49:18.840373","last_became_peered":"2021-02-10 11:49:18.840373","last_unstale":"2021-02-10 11:51:02.192516","last_undegraded":"2021-02-10 11:51:02.192516","last_fullsized":"2021-02-10 11:51:02.192516","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'34","last_scrub_stamp":"2021-02-10 11:51:02.192499","last_deep_scrub":"199'34","last_deep_scrub_stamp":"2021-02-10 11:51:02.192499","last_clean_scrub_stamp":"2021-02-10 11:51:02.192499","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777506,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":10,"num_read_kb":10240,"num_write":20,"num_write_kb":16387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.4","version":"199'229","reported_seq":"242","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:53.165380","last_change":"2021-02-10 11:50:52.689551","last_active":"2021-02-10 11:50:53.165380","last_peered":"2021-02-10 11:50:53.165380","last_clean":"2021-02-10 11:50:53.165380","last_became_active":"2021-02-10 11:49:18.171030","last_became_peered":"2021-02-10 11:49:18.171030","last_unstale":"2021-02-10 11:50:53.165380","last_undegraded":"2021-02-10 11:50:53.165380","last_fullsized":"2021-02-10 11:50:53.165380","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'226","last_scrub_stamp":"2021-02-10 11:50:52.689518","last_deep_scrub":"199'226","last_deep_scrub_stamp":"2021-02-10 11:50:52.689518","last_clean_scrub_stamp":"2021-02-10 11:50:52.689518","log_size":229,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":940641,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":226,"num_write_kb":987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"43.2","version":"199'27","reported_seq":"47","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:56.517089","last_change":"2021-02-10 11:50:56.517114","last_active":"2021-02-10 11:50:56.517089","last_peered":"2021-02-10 11:50:56.517089","last_clean":"2021-02-10 11:50:56.517089","last_became_active":"2021-02-10 11:49:18.841126","last_became_peered":"2021-02-10 11:49:18.841126","last_unstale":"2021-02-10 11:50:56.517089","last_undegraded":"2021-02-10 11:50:56.517089","last_fullsized":"2021-02-10 11:50:56.517089","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'27","last_scrub_stamp":"2021-02-10 11:50:56.517086","last_deep_scrub":"199'27","last_deep_scrub_stamp":"2021-02-10 11:50:56.517086","last_clean_scrub_stamp":"2021-02-10 11:50:56.517086","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":8,"num_read_kb":8192,"num_write":20,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.3","version":"199'9","reported_seq":"22","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:51:00.390706","last_change":"2021-02-10 11:51:00.390728","last_active":"2021-02-10 11:51:00.390706","last_peered":"2021-02-10 11:51:00.390706","last_clean":"2021-02-10 11:51:00.390706","last_became_active":"2021-02-10 11:49:18.172169","last_became_peered":"2021-02-10 11:49:18.172169","last_unstale":"2021-02-10 11:51:00.390706","last_undegraded":"2021-02-10 11:51:00.390706","last_fullsized":"2021-02-10 11:51:00.390706","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'9","last_scrub_stamp":"2021-02-10 11:51:00.390701","last_deep_scrub":"199'9","last_deep_scrub_stamp":"2021-02-10 11:51:00.390701","last_clean_scrub_stamp":"2021-02-10 11:51:00.390701","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4620,"num_omap_keys":10,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"43.3","version":"199'33","reported_seq":"53","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:51:02.458485","last_change":"2021-02-10 11:51:02.458530","last_active":"2021-02-10 11:51:02.458485","last_peered":"2021-02-10 11:51:02.458485","last_clean":"2021-02-10 11:51:02.458485","last_became_active":"2021-02-10 11:49:18.836982","last_became_peered":"2021-02-10 11:49:18.836982","last_unstale":"2021-02-10 11:51:02.458485","last_undegraded":"2021-02-10 11:51:02.458485","last_fullsized":"2021-02-10 11:51:02.458485","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'33","last_scrub_stamp":"2021-02-10 11:51:02.458481","last_deep_scrub":"199'33","last_deep_scrub_stamp":"2021-02-10 11:51:02.458481","last_clean_scrub_stamp":"2021-02-10 11:51:02.458481","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18874505,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":8192,"num_write":25,"num_write_kb":18436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.2","version":"198'1","reported_seq":"14","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:53.457772","last_change":"2021-02-10 11:50:53.457803","last_active":"2021-02-10 11:50:53.457772","last_peered":"2021-02-10 11:50:53.457772","last_clean":"2021-02-10 11:50:53.457772","last_became_active":"2021-02-10 11:49:18.181627","last_became_peered":"2021-02-10 11:49:18.181627","last_unstale":"2021-02-10 11:50:53.457772","last_undegraded":"2021-02-10 11:50:53.457772","last_fullsized":"2021-02-10 11:50:53.457772","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'1","last_scrub_stamp":"2021-02-10 11:50:53.457769","last_deep_scrub":"198'1","last_deep_scrub_stamp":"2021-02-10 11:50:53.457769","last_clean_scrub_stamp":"2021-02-10 11:50:53.457769","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"43.0","version":"199'36","reported_seq":"58","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:51:04.425559","last_change":"2021-02-10 11:51:04.425582","last_active":"2021-02-10 11:51:04.425559","last_peered":"2021-02-10 11:51:04.425559","last_clean":"2021-02-10 11:51:04.425559","last_became_active":"2021-02-10 11:49:18.830931","last_became_peered":"2021-02-10 11:49:18.830931","last_unstale":"2021-02-10 11:51:04.425559","last_undegraded":"2021-02-10 11:51:04.425559","last_fullsized":"2021-02-10 11:51:04.425559","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'36","last_scrub_stamp":"2021-02-10 11:51:04.425555","last_deep_scrub":"199'36","last_deep_scrub_stamp":"2021-02-10 11:51:04.425555","last_clean_scrub_stamp":"2021-02-10 11:51:04.425555","log_size":36,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18874368,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":10,"num_read_kb":10240,"num_write":18,"num_write_kb":18432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.1","version":"199'2","reported_seq":"16","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:51:00.215039","last_change":"2021-02-10 11:51:00.215077","last_active":"2021-02-10 11:51:00.215039","last_peered":"2021-02-10 11:51:00.215039","last_clean":"2021-02-10 11:51:00.215039","last_became_active":"2021-02-10 11:49:18.181580","last_became_peered":"2021-02-10 11:49:18.181580","last_unstale":"2021-02-10 11:51:00.215039","last_undegraded":"2021-02-10 11:51:00.215039","last_fullsized":"2021-02-10 11:51:00.215039","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'2","last_scrub_stamp":"2021-02-10 11:51:00.215035","last_deep_scrub":"199'2","last_deep_scrub_stamp":"2021-02-10 11:51:00.215035","last_clean_scrub_stamp":"2021-02-10 11:51:00.215035","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":101,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"43.1","version":"199'21","reported_seq":"39","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:51:07.919919","last_change":"2021-02-10 11:51:07.919944","last_active":"2021-02-10 11:51:07.919919","last_peered":"2021-02-10 11:51:07.919919","last_clean":"2021-02-10 11:51:07.919919","last_became_active":"2021-02-10 11:49:18.828750","last_became_peered":"2021-02-10 11:49:18.828750","last_unstale":"2021-02-10 11:51:07.919919","last_undegraded":"2021-02-10 11:51:07.919919","last_fullsized":"2021-02-10 11:51:07.919919","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'21","last_scrub_stamp":"2021-02-10 11:51:07.919916","last_deep_scrub":"199'21","last_deep_scrub_stamp":"2021-02-10 11:51:07.919916","last_clean_scrub_stamp":"2021-02-10 11:51:07.919916","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10485760,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":6,"num_read_kb":6144,"num_write":10,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.0","version":"199'12","reported_seq":"25","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:56.856641","last_change":"2021-02-10 11:50:56.856681","last_active":"2021-02-10 11:50:56.856641","last_peered":"2021-02-10 11:50:56.856641","last_clean":"2021-02-10 11:50:56.856641","last_became_active":"2021-02-10 11:49:18.170941","last_became_peered":"2021-02-10 11:49:18.170941","last_unstale":"2021-02-10 11:50:56.856641","last_undegraded":"2021-02-10 11:50:56.856641","last_fullsized":"2021-02-10 11:50:56.856641","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'12","last_scrub_stamp":"2021-02-10 11:50:56.856637","last_deep_scrub":"199'12","last_deep_scrub_stamp":"2021-02-10 11:50:56.856637","last_clean_scrub_stamp":"2021-02-10 11:50:56.856637","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"181","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:56.338044","last_change":"2021-02-10 11:50:56.338076","last_active":"2021-02-10 11:50:56.338044","last_peered":"2021-02-10 11:50:56.338044","last_clean":"2021-02-10 11:50:56.338044","last_became_active":"2021-02-10 11:13:09.494906","last_became_peered":"2021-02-10 11:13:09.494906","last_unstale":"2021-02-10 11:50:56.338044","last_undegraded":"2021-02-10 11:50:56.338044","last_fullsized":"2021-02-10 11:50:56.338044","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:50:56.338041","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:50:56.338041","last_clean_scrub_stamp":"2021-02-10 11:50:56.338041","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"181","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:53.363802","last_change":"2021-02-10 11:50:53.363832","last_active":"2021-02-10 11:50:53.363802","last_peered":"2021-02-10 11:50:53.363802","last_clean":"2021-02-10 11:50:53.363802","last_became_active":"2021-02-10 11:13:09.495226","last_became_peered":"2021-02-10 11:13:09.495226","last_unstale":"2021-02-10 11:50:53.363802","last_undegraded":"2021-02-10 11:50:53.363802","last_fullsized":"2021-02-10 11:50:53.363802","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:50:53.363799","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:50:53.363799","last_clean_scrub_stamp":"2021-02-10 11:50:53.363799","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"192","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:51.671246","last_change":"2021-02-10 11:50:51.671279","last_active":"2021-02-10 11:50:51.671246","last_peered":"2021-02-10 11:50:51.671246","last_clean":"2021-02-10 11:50:51.671246","last_became_active":"2021-02-10 11:13:09.495041","last_became_peered":"2021-02-10 11:13:09.495041","last_unstale":"2021-02-10 11:50:51.671246","last_undegraded":"2021-02-10 11:50:51.671246","last_fullsized":"2021-02-10 11:50:51.671246","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:50:51.671243","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:50:51.671243","last_clean_scrub_stamp":"2021-02-10 11:50:51.671243","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"186","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:53.890084","last_change":"2021-02-10 11:50:53.890143","last_active":"2021-02-10 11:50:53.890084","last_peered":"2021-02-10 11:50:53.890084","last_clean":"2021-02-10 11:50:53.890084","last_became_active":"2021-02-10 11:13:09.487627","last_became_peered":"2021-02-10 11:13:09.487627","last_unstale":"2021-02-10 11:50:53.890084","last_undegraded":"2021-02-10 11:50:53.890084","last_fullsized":"2021-02-10 11:50:53.890084","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:50:53.890064","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:50:53.890064","last_clean_scrub_stamp":"2021-02-10 11:50:53.890064","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"180","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:56.752028","last_change":"2021-02-10 11:50:56.752075","last_active":"2021-02-10 11:50:56.752028","last_peered":"2021-02-10 11:50:56.752028","last_clean":"2021-02-10 11:50:56.752028","last_became_active":"2021-02-10 11:13:09.495399","last_became_peered":"2021-02-10 11:13:09.495399","last_unstale":"2021-02-10 11:50:56.752028","last_undegraded":"2021-02-10 11:50:56.752028","last_fullsized":"2021-02-10 11:50:56.752028","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:50:56.752025","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:50:56.752025","last_clean_scrub_stamp":"2021-02-10 11:50:56.752025","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"43.4","version":"199'16","reported_seq":"34","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:55.838663","last_change":"2021-02-10 11:50:55.838689","last_active":"2021-02-10 11:50:55.838663","last_peered":"2021-02-10 11:50:55.838663","last_clean":"2021-02-10 11:50:55.838663","last_became_active":"2021-02-10 11:49:18.837387","last_became_peered":"2021-02-10 11:49:18.837387","last_unstale":"2021-02-10 11:50:55.838663","last_undegraded":"2021-02-10 11:50:55.838663","last_fullsized":"2021-02-10 11:50:55.838663","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'16","last_scrub_stamp":"2021-02-10 11:50:55.838660","last_deep_scrub":"199'16","last_deep_scrub_stamp":"2021-02-10 11:50:55.838660","last_clean_scrub_stamp":"2021-02-10 11:50:55.838660","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10485760,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":6,"num_read_kb":6144,"num_write":10,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.5","version":"199'12","reported_seq":"25","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:58.877793","last_change":"2021-02-10 11:50:58.877819","last_active":"2021-02-10 11:50:58.877793","last_peered":"2021-02-10 11:50:58.877793","last_clean":"2021-02-10 11:50:58.877793","last_became_active":"2021-02-10 11:49:18.171537","last_became_peered":"2021-02-10 11:49:18.171537","last_unstale":"2021-02-10 11:50:58.877793","last_undegraded":"2021-02-10 11:50:58.877793","last_fullsized":"2021-02-10 11:50:58.877793","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'12","last_scrub_stamp":"2021-02-10 11:50:58.877786","last_deep_scrub":"199'12","last_deep_scrub_stamp":"2021-02-10 11:50:58.877786","last_clean_scrub_stamp":"2021-02-10 11:50:58.877786","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"195","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:54.817517","last_change":"2021-02-10 11:50:54.817551","last_active":"2021-02-10 11:50:54.817517","last_peered":"2021-02-10 11:50:54.817517","last_clean":"2021-02-10 11:50:54.817517","last_became_active":"2021-02-10 11:13:09.495383","last_became_peered":"2021-02-10 11:13:09.495383","last_unstale":"2021-02-10 11:50:54.817517","last_undegraded":"2021-02-10 11:50:54.817517","last_fullsized":"2021-02-10 11:50:54.817517","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:50:54.817514","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:50:54.817514","last_clean_scrub_stamp":"2021-02-10 11:50:54.817514","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"180","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:51:04.707076","last_change":"2021-02-10 11:51:04.707106","last_active":"2021-02-10 11:51:04.707076","last_peered":"2021-02-10 11:51:04.707076","last_clean":"2021-02-10 11:51:04.707076","last_became_active":"2021-02-10 11:13:09.495261","last_became_peered":"2021-02-10 11:13:09.495261","last_unstale":"2021-02-10 11:51:04.707076","last_undegraded":"2021-02-10 11:51:04.707076","last_fullsized":"2021-02-10 11:51:04.707076","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:51:04.707073","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:51:04.707073","last_clean_scrub_stamp":"2021-02-10 11:51:04.707073","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"178","reported_epoch":"199","state":"active+clean","last_fresh":"2021-02-10 11:50:50.109695","last_change":"2021-02-10 11:50:50.109727","last_active":"2021-02-10 11:50:50.109695","last_peered":"2021-02-10 11:50:50.109695","last_clean":"2021-02-10 11:50:50.109695","last_became_active":"2021-02-10 11:13:09.488122","last_became_peered":"2021-02-10 11:13:09.488122","last_unstale":"2021-02-10 11:50:50.109695","last_undegraded":"2021-02-10 11:50:50.109695","last_fullsized":"2021-02-10 11:50:50.109695","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-10 11:50:50.109692","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-10 11:50:50.109692","last_clean_scrub_stamp":"2021-02-10 11:50:50.109692","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":43,"num_pg":8,"stat_sum":{"num_bytes":134218155,"num_objects":131,"num_object_clones":1,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":0,"num_read":64,"num_read_kb":65536,"num_write":139,"num_write_kb":131079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":268460032,"data_stored":268436310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":239,"ondisk_log_size":239,"up":16,"acting":16,"num_store_stats":8},{"poolid":42,"num_pg":8,"stat_sum":{"num_bytes":941502,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":299,"num_write_kb":1015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5006,"num_omap_keys":18,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1949696,"data_stored":1883004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":295,"ondisk_log_size":295,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575310,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1087976,"kb_used_data":39392,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93283864,"statfs":{"total":96636764160,"available":95522676736,"internally_reserved":1073741824,"allocated":40337408,"data_stored":38854895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 10 11:50:13 2021","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.73099999999999998,"15min":0.76500000000000001},"min":{"1min":0.373,"5min":0.36899999999999999,"15min":0.36199999999999999},"max":{"1min":1.0720000000000001,"5min":1.2290000000000001,"15min":1.232},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77900000000000003,"15min":0.79500000000000004},"min":{"1min":0.442,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.1499999999999999,"5min":1.228,"15min":1.24},"last":0.84099999999999997}]},{"osd":1,"last update":"Wed Feb 10 11:50:13 2021","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.78900000000000003,"15min":0.81999999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.98099999999999998,"5min":1.179,"15min":1.2789999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.79200000000000004,"15min":0.82499999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0389999999999999,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.86299999999999999}]},{"osd":2,"last update":"Wed Feb 10 11:50:13 2021","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.81299999999999994,"15min":0.84999999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.99099999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.41799999999999998},"max":{"1min":1.01,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed Feb 10 11:50:13 2021","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.90100000000000002,"15min":0.91600000000000004},"min":{"1min":0.49399999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1000000000000001,"5min":1.427,"15min":1.427},"last":1.018},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.94699999999999995,"15min":0.95799999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.442},"max":{"1min":1.101,"5min":1.333,"15min":1.3520000000000001},"last":1.008}]},{"osd":4,"last update":"Wed Feb 10 11:50:13 2021","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70199999999999996,"15min":0.69699999999999995},"min":{"1min":0.47499999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90200000000000002,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.52000000000000002,"5min":0.46899999999999997,"15min":0.41899999999999998},"max":{"1min":1.008,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.57999999999999996}]},{"osd":5,"last update":"Wed Feb 10 11:50:13 2021","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79100000000000004,"15min":0.81200000000000006},"min":{"1min":0.44,"5min":0.434,"15min":0.42799999999999999},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.86799999999999999,"15min":0.879},"min":{"1min":0.54300000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.089,"5min":1.5,"15min":1.5},"last":0.64900000000000002}]},{"osd":6,"last update":"Wed Feb 10 11:50:13 2021","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.81899999999999995,"15min":0.80300000000000005},"min":{"1min":0.52000000000000002,"5min":0.5,"15min":0.48499999999999999},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.84699999999999998,"15min":0.83999999999999997},"min":{"1min":0.53100000000000003,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":1.077,"5min":1.363,"15min":1.363},"last":0.59599999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575310,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1070456,"kb_used_data":21872,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93301384,"statfs":{"total":96636764160,"available":95540617216,"internally_reserved":1073741824,"allocated":22396928,"data_stored":20920245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 10 11:50:20 2021","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.91100000000000003,"15min":0.92000000000000004},"min":{"1min":0.45600000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.403,"5min":1.589,"15min":1.589},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.93899999999999995,"15min":0.95699999999999996},"min":{"1min":0.58599999999999997,"5min":0.44400000000000001,"15min":0.40200000000000002},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.5229999999999999},"last":0.93500000000000005}]},{"osd":1,"last update":"Wed Feb 10 11:50:20 2021","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91100000000000003,"15min":0.90300000000000002},"min":{"1min":0.59799999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.226,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95899999999999996,"15min":0.94999999999999996},"min":{"1min":0.501,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2849999999999999,"5min":1.591,"15min":1.591},"last":0.90300000000000002}]},{"osd":2,"last update":"Wed Feb 10 11:50:20 2021","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.82799999999999996,"15min":0.82599999999999996},"min":{"1min":0.55200000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.151,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.872,"5min":0.86099999999999999,"15min":0.84599999999999997},"min":{"1min":0.55300000000000005,"5min":0.44500000000000001,"15min":0.40200000000000002},"max":{"1min":1.333,"5min":1.333,"15min":1.4570000000000001},"last":0.81200000000000006}]},{"osd":3,"last update":"Wed Feb 10 11:50:20 2021","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.84199999999999997,"15min":0.85199999999999998},"min":{"1min":0.52500000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1819999999999999,"5min":1.2430000000000001,"15min":1.383},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.83999999999999997,"15min":0.83099999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.41799999999999998},"max":{"1min":1.167,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.82399999999999995}]},{"osd":4,"last update":"Wed Feb 10 11:50:20 2021","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69999999999999996,"15min":0.72399999999999998},"min":{"1min":0.498,"5min":0.40799999999999997,"15min":0.38900000000000001},"max":{"1min":0.98999999999999999,"5min":1.2669999999999999,"15min":1.2949999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.51200000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94299999999999995,"5min":1.145,"15min":1.145},"last":0.53800000000000003}]},{"osd":5,"last update":"Wed Feb 10 11:50:20 2021","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.47899999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92900000000000005,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.80700000000000005,"15min":0.79600000000000004},"min":{"1min":0.64900000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.0640000000000001,"5min":1.506,"15min":1.506},"last":0.52600000000000002}]},{"osd":7,"last update":"Wed Feb 10 11:50:20 2021","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.86399999999999999,"15min":0.871},"min":{"1min":0.47999999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.022,"5min":1.351,"15min":1.351},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.89800000000000002,"15min":0.89400000000000002},"min":{"1min":0.59799999999999998,"5min":0.47599999999999998,"15min":0.47299999999999998},"max":{"1min":1.127,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.75}]}]},{"osd":5,"up_from":13,"seq":55834575310,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1083868,"kb_used_data":35284,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93287972,"statfs":{"total":96636764160,"available":95526883328,"internally_reserved":1073741824,"allocated":36130816,"data_stored":34659794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 10 11:50:19 2021","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.373},"max":{"1min":1.2390000000000001,"5min":1.393,"15min":1.393},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.81899999999999995,"15min":0.82799999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.1120000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.98699999999999999}]},{"osd":1,"last update":"Wed Feb 10 11:50:19 2021","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.81499999999999995,"15min":0.83799999999999997},"min":{"1min":0.38200000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.135,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.87},{"interface":"front","average":{"1min":0.874,"5min":0.82899999999999996,"15min":0.84299999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.41699999999999998},"max":{"1min":1.22,"5min":1.222,"15min":1.2350000000000001},"last":0.91500000000000004}]},{"osd":2,"last update":"Wed Feb 10 11:50:19 2021","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.79500000000000004,"15min":0.80900000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.151,"5min":1.2789999999999999,"15min":1.331},"last":0.879},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.87,"15min":0.86699999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.349,"5min":1.349,"15min":1.3740000000000001},"last":0.78600000000000003}]},{"osd":3,"last update":"Wed Feb 10 11:50:19 2021","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.86499999999999999,"15min":0.875},"min":{"1min":0.64400000000000002,"5min":0.47999999999999998,"15min":0.41899999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.427},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.83299999999999996,"15min":0.84899999999999998},"min":{"1min":0.44800000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.077,"5min":1.335,"15min":1.335},"last":0.95799999999999996}]},{"osd":4,"last update":"Wed Feb 10 11:50:19 2021","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.73199999999999998,"15min":0.71399999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.36799999999999999},"max":{"1min":0.95899999999999996,"5min":1.161,"15min":1.216},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.45400000000000001,"5min":0.42099999999999999,"15min":0.35499999999999998},"max":{"1min":1.0580000000000001,"5min":1.196,"15min":1.3560000000000001},"last":0.629}]},{"osd":6,"last update":"Wed Feb 10 11:50:19 2021","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.71999999999999997,"15min":0.71499999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.39100000000000001},"max":{"1min":0.86699999999999999,"5min":1.276,"15min":1.276},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.79900000000000004,"15min":0.79000000000000004},"min":{"1min":0.54800000000000004,"5min":0.47999999999999998,"15min":0.41799999999999998},"max":{"1min":1.075,"5min":1.2649999999999999,"15min":1.3999999999999999},"last":0.64600000000000002}]},{"osd":7,"last update":"Wed Feb 10 11:50:19 2021","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.82499999999999996,"15min":0.81100000000000005},"min":{"1min":0.53800000000000003,"5min":0.48599999999999999,"15min":0.46899999999999997},"max":{"1min":1.006,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.79000000000000004,"15min":0.79300000000000004},"min":{"1min":0.55500000000000005,"5min":0.46700000000000003,"15min":0.45900000000000002},"max":{"1min":0.97799999999999998,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.67100000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575310,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1074548,"kb_used_data":25964,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93297292,"statfs":{"total":96636764160,"available":95536427008,"internally_reserved":1073741824,"allocated":26587136,"data_stored":25114515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 10 11:51:05 2021","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.871,"15min":0.90500000000000003},"min":{"1min":0.39900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.3129999999999999,"5min":1.411,"15min":1.6850000000000001},"last":1.276},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.60099999999999998,"5min":0.42299999999999999,"15min":0.40100000000000002},"max":{"1min":1.2829999999999999,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.083}]},{"osd":1,"last update":"Wed Feb 10 11:51:05 2021","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.85699999999999998,"15min":0.84199999999999997},"min":{"1min":0.41699999999999998,"5min":0.40200000000000002,"15min":0.40000000000000002},"max":{"1min":1.194,"5min":1.2509999999999999,"15min":1.468},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.83099999999999996,"15min":0.83399999999999996},"min":{"1min":0.438,"5min":0.371,"15min":0.36399999999999999},"max":{"1min":1.194,"5min":1.234,"15min":1.49},"last":1.024}]},{"osd":2,"last update":"Wed Feb 10 11:51:05 2021","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.50700000000000001,"5min":0.438,"15min":0.36599999999999999},"max":{"1min":1.1950000000000001,"5min":1.284,"15min":1.5660000000000001},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.82699999999999996,"15min":0.83699999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.607},"last":0.96399999999999997}]},{"osd":3,"last update":"Wed Feb 10 11:51:05 2021","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.82999999999999996,"15min":0.83299999999999996},"min":{"1min":0.47399999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2470000000000001,"5min":1.2709999999999999,"15min":1.6319999999999999},"last":1.0760000000000001},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.86099999999999999,"15min":0.871},"min":{"1min":0.55700000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.268,"5min":1.3029999999999999,"15min":1.482},"last":1.1850000000000001}]},{"osd":5,"last update":"Wed Feb 10 11:51:05 2021","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69399999999999995,"15min":0.71399999999999997},"min":{"1min":0.46400000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.92200000000000004,"5min":1.1699999999999999,"15min":1.236},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70099999999999996,"15min":0.70899999999999996},"min":{"1min":0.51500000000000001,"5min":0.45800000000000002,"15min":0.42799999999999999},"max":{"1min":0.85199999999999998,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.79700000000000004}]},{"osd":6,"last update":"Wed Feb 10 11:51:05 2021","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68700000000000006,"15min":0.72899999999999998},"min":{"1min":0.52200000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1060000000000001,"5min":1.24,"15min":1.9590000000000001},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.78200000000000003,"15min":0.79600000000000004},"min":{"1min":0.54300000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85699999999999998,"5min":1.262,"15min":3.351},"last":0.70699999999999996}]},{"osd":7,"last update":"Wed Feb 10 11:51:05 2021","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76900000000000002,"15min":0.79100000000000004},"min":{"1min":0.50700000000000001,"5min":0.502,"15min":0.48299999999999998},"max":{"1min":1.03,"5min":1.306,"15min":1.504},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.79900000000000004,"15min":0.82699999999999996},"min":{"1min":0.59699999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95399999999999996,"5min":1.181,"15min":1.3149999999999999},"last":0.85599999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575309,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1061532,"kb_used_data":12752,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93310308,"statfs":{"total":96636764160,"available":95549755392,"internally_reserved":1073741824,"allocated":13058048,"data_stored":11590996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 10 11:50:24 2021","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56100000000000005,"15min":0.57999999999999996},"min":{"1min":0.29699999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.96799999999999997,"5min":0.96899999999999997,"15min":1.0129999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.58499999999999996,"15min":0.61099999999999999},"min":{"1min":0.39500000000000002,"5min":0.34100000000000003,"15min":0.32700000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":1.196},"last":0.44900000000000001}]},{"osd":2,"last update":"Wed Feb 10 11:50:24 2021","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.35099999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.92300000000000004},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.308},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.98699999999999999},"last":0.67100000000000004}]},{"osd":3,"last update":"Wed Feb 10 11:50:24 2021","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.505,"15min":0.51200000000000001},"min":{"1min":0.36799999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77500000000000002,"5min":0.878,"15min":0.998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.437,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84799999999999998,"5min":0.86599999999999999,"15min":1.036},"last":0.49399999999999999}]},{"osd":4,"last update":"Wed Feb 10 11:50:24 2021","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.95599999999999996,"15min":0.95099999999999996},"min":{"1min":0.55700000000000005,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.298,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.92200000000000004,"15min":0.93200000000000005},"min":{"1min":0.56499999999999995,"5min":0.49099999999999999,"15min":0.47399999999999998},"max":{"1min":1.21,"5min":1.4179999999999999,"15min":1.49},"last":0.64900000000000002}]},{"osd":5,"last update":"Wed Feb 10 11:50:24 2021","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.94299999999999995,"15min":0.94999999999999996},"min":{"1min":0.627,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1839999999999999,"5min":1.347,"15min":1.347},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.95099999999999996,"15min":0.97099999999999997},"min":{"1min":0.61599999999999999,"5min":0.55600000000000005,"15min":0.54300000000000004},"max":{"1min":1.248,"5min":1.502,"15min":1.6180000000000001},"last":0.77400000000000002}]},{"osd":6,"last update":"Wed Feb 10 11:50:24 2021","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.97299999999999998,"15min":0.97999999999999998},"min":{"1min":0.63700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.272,"5min":1.4550000000000001,"15min":1.5469999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.96099999999999997,"15min":0.97399999999999998},"min":{"1min":0.67700000000000005,"5min":0.57699999999999996,"15min":0.53900000000000003},"max":{"1min":1.2969999999999999,"5min":1.3400000000000001,"15min":1.444},"last":0.747}]},{"osd":7,"last update":"Wed Feb 10 11:50:24 2021","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.98899999999999999,"15min":0.996},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.495},"max":{"1min":1.2549999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.97699999999999998,"15min":0.97999999999999998},"min":{"1min":0.64700000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.3560000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.75700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575309,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1092068,"kb_used_data":43480,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93279772,"statfs":{"total":96636764160,"available":95518486528,"internally_reserved":1073741824,"allocated":44523520,"data_stored":43048633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 10 11:50:18 2021","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.52600000000000002,"15min":0.505},"min":{"1min":0.371,"5min":0.32600000000000001,"15min":0.30099999999999999},"max":{"1min":0.66100000000000003,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51300000000000001,"15min":0.50900000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.309},"max":{"1min":0.874,"5min":0.874,"15min":0.90400000000000003},"last":0.36899999999999999}]},{"osd":2,"last update":"Wed Feb 10 11:50:18 2021","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56999999999999995,"15min":0.55700000000000005},"min":{"1min":0.39700000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57199999999999995,"15min":0.56399999999999995},"min":{"1min":0.41299999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86099999999999999,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.54900000000000004}]},{"osd":3,"last update":"Wed Feb 10 11:50:18 2021","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61799999999999999,"15min":0.61199999999999999},"min":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.38100000000000001},"max":{"1min":0.99299999999999999,"5min":1.056,"15min":1.056},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65000000000000002,"15min":0.63300000000000001},"min":{"1min":0.45500000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.075,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.58999999999999997}]},{"osd":4,"last update":"Wed Feb 10 11:50:18 2021","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.0369999999999999,"15min":1.0009999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.61299999999999999},"max":{"1min":1.284,"5min":1.609,"15min":1.629},"last":1.137},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0369999999999999,"15min":1.0049999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.42299999999999999},"max":{"1min":1.2210000000000001,"5min":1.4790000000000001,"15min":1.6299999999999999},"last":1.2430000000000001}]},{"osd":5,"last update":"Wed Feb 10 11:50:18 2021","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1359999999999999,"15min":1.115},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3240000000000001,"5min":1.518,"15min":1.665},"last":1.272},{"interface":"front","average":{"1min":1.083,"5min":1.1240000000000001,"15min":1.107},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.431,"5min":1.694,"15min":1.74},"last":1.202}]},{"osd":6,"last update":"Wed Feb 10 11:50:18 2021","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.03,"15min":1.0069999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.53000000000000003},"max":{"1min":1.2529999999999999,"5min":1.46,"15min":1.5700000000000001},"last":1.179},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.0489999999999999,"15min":1.0269999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.268,"5min":1.613,"15min":1.613},"last":1.071}]},{"osd":7,"last update":"Wed Feb 10 11:50:18 2021","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.0409999999999999,"15min":1.0309999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.59299999999999997},"max":{"1min":1.331,"5min":1.456,"15min":1.482},"last":1.3120000000000001},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.056,"15min":1.0329999999999999},"min":{"1min":0.73499999999999999,"5min":0.68200000000000005,"15min":0.55300000000000005},"max":{"1min":1.1830000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.1579999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575310,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1122792,"kb_used_data":74208,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93249048,"statfs":{"total":96636764160,"available":95487025152,"internally_reserved":1073741824,"allocated":75988992,"data_stored":74506385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 10 11:50:58 2021","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51700000000000002,"15min":0.498},"min":{"1min":0.379,"5min":0.29999999999999999,"15min":0.27100000000000002},"max":{"1min":0.74299999999999999,"5min":1.069,"15min":1.069},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.499,"15min":0.5},"min":{"1min":0.39600000000000002,"5min":0.32700000000000001,"15min":0.29999999999999999},"max":{"1min":0.68300000000000005,"5min":0.95499999999999996,"15min":0.95799999999999996},"last":0.45400000000000001}]},{"osd":1,"last update":"Wed Feb 10 11:50:58 2021","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.36199999999999999,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":0.66600000000000004,"5min":1.131,"15min":1.131},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55800000000000005,"15min":0.54900000000000004},"min":{"1min":0.33900000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67800000000000005,"5min":0.89600000000000002,"15min":1.016},"last":0.53600000000000003}]},{"osd":3,"last update":"Wed Feb 10 11:50:58 2021","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57599999999999996,"15min":0.55800000000000005},"min":{"1min":0.40899999999999997,"5min":0.35599999999999998,"15min":0.32000000000000001},"max":{"1min":0.72799999999999998,"5min":1.097,"15min":1.097},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59999999999999998,"15min":0.58499999999999996},"min":{"1min":0.442,"5min":0.371,"15min":0.34499999999999997},"max":{"1min":0.78900000000000003,"5min":1.252,"15min":1.252},"last":0.58199999999999996}]},{"osd":4,"last update":"Wed Feb 10 11:50:58 2021","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.97499999999999998,"15min":0.96399999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.495},"max":{"1min":1.2330000000000001,"5min":1.393,"15min":1.4470000000000001},"last":1.173},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1.0009999999999999,"15min":0.97999999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3260000000000001,"5min":1.343,"15min":1.417},"last":1.0960000000000001}]},{"osd":5,"last update":"Wed Feb 10 11:50:58 2021","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.115,"15min":1.089},"min":{"1min":0.72499999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.379,"5min":1.4139999999999999,"15min":1.5249999999999999},"last":1.254},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1419999999999999,"15min":1.1160000000000001},"min":{"1min":0.70999999999999996,"5min":0.69599999999999995,"15min":0.54500000000000004},"max":{"1min":1.4530000000000001,"5min":1.476,"15min":1.5149999999999999},"last":1.1879999999999999}]},{"osd":6,"last update":"Wed Feb 10 11:50:58 2021","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.0029999999999999,"15min":0.98799999999999999},"min":{"1min":0.65400000000000003,"5min":0.61199999999999999,"15min":0.504},"max":{"1min":1.2789999999999999,"5min":1.3500000000000001,"15min":1.3779999999999999},"last":1.2350000000000001},{"interface":"front","average":{"1min":0.98899999999999999,"5min":1.0189999999999999,"15min":1.006},"min":{"1min":0.66200000000000003,"5min":0.60399999999999998,"15min":0.53700000000000003},"max":{"1min":1.3120000000000001,"5min":1.4470000000000001,"15min":1.472},"last":1.0629999999999999}]},{"osd":7,"last update":"Wed Feb 10 11:50:58 2021","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0369999999999999,"15min":1.0149999999999999},"min":{"1min":0.66300000000000003,"5min":0.60699999999999998,"15min":0.55500000000000005},"max":{"1min":1.327,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.268},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.03,"15min":0.99399999999999999},"min":{"1min":0.68300000000000005,"5min":0.64800000000000002,"15min":0.58899999999999997},"max":{"1min":1.4339999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.022}]}]},{"osd":3,"up_from":13,"seq":55834575310,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1079768,"kb_used_data":31184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93292072,"statfs":{"total":96636764160,"available":95531081728,"internally_reserved":1073741824,"allocated":31932416,"data_stored":30465467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 10 11:50:26 2021","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46600000000000003,"15min":0.46700000000000003},"min":{"1min":0.34599999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71399999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.499,"15min":0.495},"min":{"1min":0.38,"5min":0.317,"15min":0.317},"max":{"1min":0.628,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.48099999999999998}]},{"osd":1,"last update":"Wed Feb 10 11:50:26 2021","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47999999999999998,"15min":0.48599999999999999},"min":{"1min":0.374,"5min":0.316,"15min":0.316},"max":{"1min":0.57099999999999995,"5min":0.77000000000000002,"15min":0.89700000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.439,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.95899999999999996,"5min":1.0289999999999999,"15min":1.0409999999999999},"last":0.46000000000000002}]},{"osd":2,"last update":"Wed Feb 10 11:50:26 2021","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.46800000000000003,"5min":0.35399999999999998,"15min":0.34100000000000003},"max":{"1min":0.92900000000000005,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68000000000000005,"15min":0.68500000000000005},"min":{"1min":0.50700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89600000000000002,"5min":1.397,"15min":1.397},"last":0.67000000000000004}]},{"osd":4,"last update":"Wed Feb 10 11:50:26 2021","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.95199999999999996,"15min":0.95699999999999996},"min":{"1min":0.60799999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.2350000000000001,"5min":1.472,"15min":1.472},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.93799999999999994,"15min":0.94099999999999995},"min":{"1min":0.67100000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2470000000000001,"5min":1.351,"15min":1.575},"last":0.872}]},{"osd":5,"last update":"Wed Feb 10 11:50:26 2021","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.0549999999999999,"15min":1.071},"min":{"1min":0.59599999999999997,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.431,"5min":1.621,"15min":1.621},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0660000000000001,"15min":1.0600000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.48299999999999998},"max":{"1min":1.3380000000000001,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.0940000000000001}]},{"osd":6,"last update":"Wed Feb 10 11:50:26 2021","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.98399999999999999,"15min":0.95899999999999996},"min":{"1min":0.76600000000000001,"5min":0.63300000000000001,"15min":0.54500000000000004},"max":{"1min":1.222,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.024,"5min":1.01,"15min":0.98999999999999999},"min":{"1min":0.68200000000000005,"5min":0.61199999999999999,"15min":0.57999999999999996},"max":{"1min":1.288,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.98799999999999999}]},{"osd":7,"last update":"Wed Feb 10 11:50:26 2021","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1080000000000001,"15min":1.1020000000000001},"min":{"1min":0.89000000000000001,"5min":0.67200000000000004,"15min":0.63200000000000001},"max":{"1min":1.413,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.131,"5min":1.083,"15min":1.0880000000000001},"min":{"1min":0.88100000000000001,"5min":0.67700000000000005,"15min":0.67300000000000004},"max":{"1min":1.405,"5min":1.508,"15min":1.5900000000000001},"last":1.161}]}]}]}} 2021-02-10T11:51:15.310 DEBUG:teuthology.orchestra.run.gibba001:> sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell mon.* injectargs -- --no-mon-health-to-clog 2021-02-10T11:51:15.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.725 7f742c47f700 1 Processor -- start 2021-02-10T11:51:15.737 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.725 7f742c47f700 1 -- start start 2021-02-10T11:51:15.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.725 7f742c47f700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f74240d1880 0x7f742410ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-02-10T11:51:15.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.725 7f742c47f700 1 -- --> v1:172.21.2.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74241b0d40 con 0x7f742410a040 2021-02-10T11:51:15.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.725 7f742c47f700 1 -- --> v1:172.21.2.108:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7424109ac0 con 0x7f74240e53a0 2021-02-10T11:51:15.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.725 7f742c47f700 1 -- --> v2:172.21.2.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74240e5f80 con 0x7f74240d1880 2021-02-10T11:51:15.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.726 7f742aa9c700 1 --2- >> v2:172.21.2.101:3300/0 conn(0x7f74240d1880 0x7f742410ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:51:15.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.726 7f7429a9a700 1 --1- >> v1:172.21.2.101:6789/0 conn(0x7f742410a040 0x7f742410a440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.101:6789/0 says I am v1:172.21.2.101:36994/0 (socket says 172.21.2.101:36994) 2021-02-10T11:51:15.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.726 7f7429a9a700 1 -- 172.21.2.101:0/320862338 learned_addr learned my addr 172.21.2.101:0/320862338 (peer_addr_for_me v1:172.21.2.101:0/0) 2021-02-10T11:51:15.738 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.726 7f742aa9c700 1 -- 172.21.2.101:0/320862338 >> v1:172.21.2.108:6789/0 conn(0x7f74240e53a0 legacy=0x7f74241b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:51:15.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.726 7f742aa9c700 1 -- 172.21.2.101:0/320862338 >> v1:172.21.2.101:6789/0 conn(0x7f742410a040 legacy=0x7f742410a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:51:15.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.726 7f742aa9c700 1 -- 172.21.2.101:0/320862338 --> v2:172.21.2.101:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74240e57a0 con 0x7f74240d1880 2021-02-10T11:51:15.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.726 7f742aa9c700 1 --2- 172.21.2.101:0/320862338 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f74240d1880 0x7f742410ac30 secure :-1 s=READY pgs=3499 cs=0 l=1 rev1=1 rx=0x7f741c006e30 tx=0x7f741c009e60).ready entity=mon.0 client_cookie=27ce78f8a996cd76 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:51:15.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.726 7f7429299700 1 -- 172.21.2.101:0/320862338 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f741c00b300 con 0x7f74240d1880 2021-02-10T11:51:15.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.726 7f7429299700 1 -- 172.21.2.101:0/320862338 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741c000d30 con 0x7f74240d1880 2021-02-10T11:51:15.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.726 7f7429299700 1 -- 172.21.2.101:0/320862338 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f741c00b800 con 0x7f74240d1880 2021-02-10T11:51:15.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.726 7f742c47f700 1 -- 172.21.2.101:0/320862338 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f74240d1880 msgr2=0x7f742410ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:51:15.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.726 7f742c47f700 1 --2- 172.21.2.101:0/320862338 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f74240d1880 0x7f742410ac30 secure :-1 s=READY pgs=3499 cs=0 l=1 rev1=1 rx=0x7f741c006e30 tx=0x7f741c009e60).stop 2021-02-10T11:51:15.739 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.727 7f742c47f700 1 -- 172.21.2.101:0/320862338 shutdown_connections 2021-02-10T11:51:15.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.727 7f742c47f700 1 --2- 172.21.2.101:0/320862338 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f74240d1880 0x7f742410ac30 unknown :-1 s=CLOSED pgs=3499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:51:15.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.727 7f742c47f700 1 -- 172.21.2.101:0/320862338 >> 172.21.2.101:0/320862338 conn(0x7f74240ce4b0 msgr2=0x7f74240e45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:51:15.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.727 7f742c47f700 1 -- 172.21.2.101:0/320862338 shutdown_connections 2021-02-10T11:51:15.740 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.727 7f742c47f700 1 -- 172.21.2.101:0/320862338 wait complete. 2021-02-10T11:51:15.741 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.742 7f742c47f700 1 Processor -- start 2021-02-10T11:51:15.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.742 7f742c47f700 1 -- start start 2021-02-10T11:51:15.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.742 7f742c47f700 1 --2- >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f742407fc50 0x7f742409b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:51:15.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.742 7f742c47f700 1 --2- >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f74240bfbf0 0x7f74240bfff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:51:15.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.742 7f742c47f700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f74240993d0 0x7f74240997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:51:15.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.742 7f742c47f700 1 -- --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742409ad40 con 0x7f74240993d0 2021-02-10T11:51:15.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.742 7f742c47f700 1 -- --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74240c0650 con 0x7f742407fc50 2021-02-10T11:51:15.742 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.742 7f742c47f700 1 -- --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74241c24e0 con 0x7f74240bfbf0 2021-02-10T11:51:15.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.742 7f742aa9c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f74240993d0 0x7f74240997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:51:15.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.743 7f742aa9c700 1 --2- >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f74240993d0 0x7f74240997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.101:3300/0 says I am v2:172.21.2.101:36508/0 (socket says 172.21.2.101:36508) 2021-02-10T11:51:15.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.743 7f742aa9c700 1 -- 172.21.2.101:0/1612995854 learned_addr learned my addr 172.21.2.101:0/1612995854 (peer_addr_for_me v2:172.21.2.101:0/0) 2021-02-10T11:51:15.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.743 7f742aa9c700 1 -- 172.21.2.101:0/1612995854 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f74240bfbf0 msgr2=0x7f74240bfff0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:51:15.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.743 7f742aa9c700 1 --2- 172.21.2.101:0/1612995854 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f74240bfbf0 0x7f74240bfff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:51:15.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.743 7f742aa9c700 1 -- 172.21.2.101:0/1612995854 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f742407fc50 msgr2=0x7f742409b240 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-02-10T11:51:15.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.743 7f742aa9c700 1 --2- 172.21.2.101:0/1612995854 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f742407fc50 0x7f742409b240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-02-10T11:51:15.743 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.743 7f742aa9c700 1 -- 172.21.2.101:0/1612995854 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7424099df0 con 0x7f74240993d0 2021-02-10T11:51:15.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.743 7f742aa9c700 1 --2- 172.21.2.101:0/1612995854 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f74240993d0 0x7f74240997d0 secure :-1 s=READY pgs=3500 cs=0 l=1 rev1=1 rx=0x7f741c0099e0 tx=0x7f741c00bd40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:51:15.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.743 7f7416ffd700 1 -- 172.21.2.101:0/1612995854 <== mon.0 v2:172.21.2.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f741c009700 con 0x7f74240993d0 2021-02-10T11:51:15.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.743 7f7416ffd700 1 -- 172.21.2.101:0/1612995854 <== mon.0 v2:172.21.2.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741c000d90 con 0x7f74240993d0 2021-02-10T11:51:15.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.743 7f7416ffd700 1 -- 172.21.2.101:0/1612995854 <== mon.0 v2:172.21.2.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f741c001670 con 0x7f74240993d0 2021-02-10T11:51:15.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.743 7f742c47f700 1 -- 172.21.2.101:0/1612995854 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7424099df0 con 0x7f74240993d0 2021-02-10T11:51:15.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.743 7f742c47f700 1 -- 172.21.2.101:0/1612995854 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f742409a380 con 0x7f74240993d0 2021-02-10T11:51:15.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.744 7f7416ffd700 1 -- 172.21.2.101:0/1612995854 <== mon.0 v2:172.21.2.101:3300/0 4 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f741c001270 con 0x7f74240993d0 2021-02-10T11:51:15.744 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.744 7f7416ffd700 1 --2- 172.21.2.101:0/1612995854 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f740c0186c0 0x7f740c01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:51:15.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.744 7f7416ffd700 1 -- 172.21.2.101:0/1612995854 <== mon.0 v2:172.21.2.101:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f741c0329b0 con 0x7f74240993d0 2021-02-10T11:51:15.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.744 7f742a29b700 1 --2- 172.21.2.101:0/1612995854 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f740c0186c0 0x7f740c01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:51:15.745 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.744 7f742a29b700 1 --2- 172.21.2.101:0/1612995854 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f740c0186c0 0x7f740c01abc0 crc :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:51:15.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.775 7f742c47f700 1 -- 172.21.2.101:0/1612995854 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f742407c040 con 0x7f74240993d0 2021-02-10T11:51:15.771 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.776 7f7416ffd700 1 -- 172.21.2.101:0/1612995854 <== mon.0 v2:172.21.2.101:3300/0 6 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+968 (secure 0 0 0) 0x7f741c003080 con 0x7f74240993d0 2021-02-10T11:51:15.772 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.777 7f742c47f700 1 -- 172.21.2.101:0/1612995854 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74240e6170 con 0x7f74240993d0 2021-02-10T11:51:15.774 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.779 7f7416ffd700 1 -- 172.21.2.101:0/1612995854 <== mon.0 v2:172.21.2.101:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f741c001270 con 0x7f74240993d0 2021-02-10T11:51:15.906 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.911 7f742c47f700 1 -- 172.21.2.101:0/1612995854 --> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] -- mon_command({"injected_args": ["--no-mon-health-to-clog"], "prefix": "injectargs"} v 0) v1 -- 0x7f742409acc0 con 0x7f74240993d0 2021-02-10T11:51:15.907 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.912 7f7416ffd700 1 -- 172.21.2.101:0/1612995854 <== mon.0 v2:172.21.2.101:3300/0 8 ==== mon_command_ack([{"injected_args": ["--no-mon-health-to-clog"], "prefix": "injectargs"}]=0 injectargs:mon_health_to_clog = 'false' v0) v1 ==== 144+0+0 (secure 0 0 0) 0x7f741c003080 con 0x7f74240993d0 2021-02-10T11:51:15.910 INFO:teuthology.orchestra.run.gibba001.stderr:mon.a: injectargs:mon_health_to_clog = 'false' 2021-02-10T11:51:15.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.914 7f742c47f700 1 -- 172.21.2.101:0/1612995854 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f74240993d0 msgr2=0x7f74240997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:51:15.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.914 7f742c47f700 1 --2- 172.21.2.101:0/1612995854 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f74240993d0 0x7f74240997d0 secure :-1 s=READY pgs=3500 cs=0 l=1 rev1=1 rx=0x7f741c0099e0 tx=0x7f741c00bd40).stop 2021-02-10T11:51:15.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.914 7f742c47f700 1 --2- 172.21.2.101:0/1612995854 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f74240c4660 0x7f742409b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:51:15.910 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.914 7f742c47f700 1 -- 172.21.2.101:0/1612995854 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74241c2700 con 0x7f74240c4660 2021-02-10T11:51:15.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.915 7f7429a9a700 1 --2- 172.21.2.101:0/1612995854 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f74240c4660 0x7f742409b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:51:15.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.916 7f7429a9a700 1 -- 172.21.2.101:0/1612995854 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+}) v3 -- 0x7f742409a380 con 0x7f74240c4660 2021-02-10T11:51:15.911 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.916 7f7429a9a700 1 -- 172.21.2.101:0/1612995854 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f742000a4c0 con 0x7f74240c4660 2021-02-10T11:51:15.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.916 7f7429a9a700 1 --2- 172.21.2.101:0/1612995854 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f74240c4660 0x7f742409b240 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7f742000ac80 tx=0x7f742000ab90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:51:15.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.917 7f7416ffd700 1 -- 172.21.2.101:0/1612995854 <== mon.1 v2:172.21.2.108:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7420000e20 con 0x7f74240c4660 2021-02-10T11:51:15.912 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.917 7f7416ffd700 1 -- 172.21.2.101:0/1612995854 <== mon.1 v2:172.21.2.108:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742000a800 con 0x7f74240c4660 2021-02-10T11:51:15.913 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.917 7f7416ffd700 1 -- 172.21.2.101:0/1612995854 <== mon.1 v2:172.21.2.108:3300/0 3 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f7420000dd0 con 0x7f74240c4660 2021-02-10T11:51:15.916 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:15.920 7f7416ffd700 1 -- 172.21.2.101:0/1612995854 <== mon.1 v2:172.21.2.108:3300/0 4 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7420000dd0 con 0x7f74240c4660 2021-02-10T11:51:16.047 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.052 7f742c47f700 1 -- 172.21.2.101:0/1612995854 --> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] -- mon_command({"injected_args": ["--no-mon-health-to-clog"], "prefix": "injectargs"} v 0) v1 -- 0x7f742407c040 con 0x7f74240c4660 2021-02-10T11:51:16.049 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.054 7f7416ffd700 1 -- 172.21.2.101:0/1612995854 <== mon.1 v2:172.21.2.108:3300/0 5 ==== mon_command_ack([{"injected_args": ["--no-mon-health-to-clog"], "prefix": "injectargs"}]=0 injectargs:mon_health_to_clog = 'false' v0) v1 ==== 144+0+0 (secure 0 0 0) 0x7f74200016b0 con 0x7f74240c4660 2021-02-10T11:51:16.050 INFO:teuthology.orchestra.run.gibba001.stderr:mon.b: injectargs:mon_health_to_clog = 'false' 2021-02-10T11:51:16.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.056 7f742c47f700 1 -- 172.21.2.101:0/1612995854 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f74240c4660 msgr2=0x7f742409b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:51:16.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.056 7f742c47f700 1 --2- 172.21.2.101:0/1612995854 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f74240c4660 0x7f742409b240 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7f742000ac80 tx=0x7f742000ab90).stop 2021-02-10T11:51:16.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.056 7f742c47f700 1 --2- 172.21.2.101:0/1612995854 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f742407fc50 0x7f74240846d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-02-10T11:51:16.051 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.056 7f742c47f700 1 -- 172.21.2.101:0/1612995854 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74241c2700 con 0x7f742407fc50 2021-02-10T11:51:16.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.057 7f7429a9a700 1 --2- 172.21.2.101:0/1612995854 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f742407fc50 0x7f74240846d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-02-10T11:51:16.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.058 7f7429a9a700 1 -- 172.21.2.101:0/1612995854 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+}) v3 -- 0x7f74241c24e0 con 0x7f742407fc50 2021-02-10T11:51:16.052 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.058 7f7429a9a700 1 -- 172.21.2.101:0/1612995854 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f742001f800 con 0x7f742407fc50 2021-02-10T11:51:16.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.058 7f7429a9a700 1 --2- 172.21.2.101:0/1612995854 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f742407fc50 0x7f74240846d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f742001fd60 tx=0x7f742001fc70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-02-10T11:51:16.053 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.059 7f7416ffd700 1 -- 172.21.2.101:0/1612995854 <== mon.2 v2:172.21.2.108:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f74200016b0 con 0x7f742407fc50 2021-02-10T11:51:16.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.059 7f7416ffd700 1 -- 172.21.2.101:0/1612995854 <== mon.2 v2:172.21.2.108:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742000c660 con 0x7f742407fc50 2021-02-10T11:51:16.054 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.059 7f7416ffd700 1 -- 172.21.2.101:0/1612995854 <== mon.2 v2:172.21.2.108:3301/0 3 ==== mgrmap(e 3) v1 ==== 33230+0+0 (secure 0 0 0) 0x7f742000c900 con 0x7f742407fc50 2021-02-10T11:51:16.056 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.061 7f7416ffd700 1 -- 172.21.2.101:0/1612995854 <== mon.2 v2:172.21.2.108:3301/0 4 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+111044 (secure 0 0 0) 0x7f7420027a30 con 0x7f742407fc50 2021-02-10T11:51:16.188 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.193 7f742c47f700 1 -- 172.21.2.101:0/1612995854 --> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] -- mon_command({"injected_args": ["--no-mon-health-to-clog"], "prefix": "injectargs"} v 0) v1 -- 0x7f74240e6170 con 0x7f742407fc50 2021-02-10T11:51:16.189 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.194 7f7416ffd700 1 -- 172.21.2.101:0/1612995854 <== mon.2 v2:172.21.2.108:3301/0 5 ==== mon_command_ack([{"injected_args": ["--no-mon-health-to-clog"], "prefix": "injectargs"}]=0 injectargs:mon_health_to_clog = 'false' v0) v1 ==== 144+0+0 (secure 0 0 0) 0x7f7420006130 con 0x7f742407fc50 2021-02-10T11:51:16.191 INFO:teuthology.orchestra.run.gibba001.stderr:mon.c: injectargs:mon_health_to_clog = 'false' 2021-02-10T11:51:16.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.198 7f742c47f700 1 -- 172.21.2.101:0/1612995854 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f740c0186c0 msgr2=0x7f740c01abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:51:16.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.198 7f742c47f700 1 --2- 172.21.2.101:0/1612995854 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f740c0186c0 0x7f740c01abc0 crc :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:51:16.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.198 7f742c47f700 1 -- 172.21.2.101:0/1612995854 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f742407fc50 msgr2=0x7f74240846d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-02-10T11:51:16.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.198 7f742c47f700 1 --2- 172.21.2.101:0/1612995854 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f742407fc50 0x7f74240846d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f742001fd60 tx=0x7f742001fc70).stop 2021-02-10T11:51:16.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.198 7f742c47f700 1 -- 172.21.2.101:0/1612995854 shutdown_connections 2021-02-10T11:51:16.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.198 7f742c47f700 1 --2- 172.21.2.101:0/1612995854 >> [v2:172.21.2.108:3301/0,v1:172.21.2.108:6790/0] conn(0x7f742407fc50 0x7f74240846d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:51:16.193 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.198 7f742c47f700 1 --2- 172.21.2.101:0/1612995854 >> [v2:172.21.2.101:6800/24165,v1:172.21.2.101:6801/24165] conn(0x7f740c0186c0 0x7f740c01abc0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:51:16.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.198 7f742c47f700 1 --2- 172.21.2.101:0/1612995854 >> [v2:172.21.2.108:3300/0,v1:172.21.2.108:6789/0] conn(0x7f74240c4660 0x7f742409b240 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:51:16.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.198 7f742c47f700 1 --2- 172.21.2.101:0/1612995854 >> [v2:172.21.2.101:3300/0,v1:172.21.2.101:6789/0] conn(0x7f74240993d0 0x7f74240997d0 unknown :-1 s=CLOSED pgs=3500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-02-10T11:51:16.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.198 7f742c47f700 1 -- 172.21.2.101:0/1612995854 >> 172.21.2.101:0/1612995854 conn(0x7f742407c270 msgr2=0x7f74240b5500 unknown :-1 s=STATE_NONE l=0).mark_down 2021-02-10T11:51:16.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.198 7f742c47f700 1 -- 172.21.2.101:0/1612995854 shutdown_connections 2021-02-10T11:51:16.194 INFO:teuthology.orchestra.run.gibba001.stderr:2021-02-10 11:51:16.198 7f742c47f700 1 -- 172.21.2.101:0/1612995854 wait complete. 2021-02-10T11:51:16.201 INFO:teuthology.misc:Shutting down mds daemons... 2021-02-10T11:51:16.201 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-02-10T11:51:16.201 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:51:16.208 INFO:tasks.ceph.mds.a:Stopped 2021-02-10T11:51:16.208 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-02-10T11:51:16.208 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:51:16.215 INFO:tasks.ceph.mds.b:Stopped 2021-02-10T11:51:16.215 INFO:teuthology.misc:Shutting down osd daemons... 2021-02-10T11:51:16.215 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2021-02-10T11:51:16.215 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:51:16.277 INFO:tasks.ceph.osd.0:Stopped 2021-02-10T11:51:16.277 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2021-02-10T11:51:16.277 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:51:16.347 INFO:tasks.ceph.osd.1:Stopped 2021-02-10T11:51:16.347 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2021-02-10T11:51:16.347 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:51:16.410 INFO:tasks.ceph.osd.2:Stopped 2021-02-10T11:51:16.411 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2021-02-10T11:51:16.411 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:51:16.475 INFO:tasks.ceph.osd.3:Stopped 2021-02-10T11:51:16.475 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2021-02-10T11:51:16.476 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:51:16.504 INFO:tasks.ceph.osd.4:Stopped 2021-02-10T11:51:16.504 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2021-02-10T11:51:16.505 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:51:16.574 INFO:tasks.ceph.osd.5:Stopped 2021-02-10T11:51:16.574 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2021-02-10T11:51:16.574 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:51:16.646 INFO:tasks.ceph.osd.6:Stopped 2021-02-10T11:51:16.646 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2021-02-10T11:51:16.646 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:51:16.711 INFO:tasks.ceph.osd.7:Stopped 2021-02-10T11:51:16.711 INFO:teuthology.misc:Shutting down mgr daemons... 2021-02-10T11:51:16.711 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2021-02-10T11:51:16.712 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:51:16.732 INFO:tasks.ceph.mgr.y:Stopped 2021-02-10T11:51:16.732 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2021-02-10T11:51:16.732 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:51:16.764 INFO:tasks.ceph.mgr.x:Stopped 2021-02-10T11:51:16.764 INFO:teuthology.misc:Shutting down mon daemons... 2021-02-10T11:51:16.764 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2021-02-10T11:51:16.764 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:51:16.782 INFO:tasks.ceph.mon.a:Stopped 2021-02-10T11:51:16.782 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2021-02-10T11:51:16.783 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:51:16.812 INFO:tasks.ceph.mon.b:Stopped 2021-02-10T11:51:16.812 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2021-02-10T11:51:16.812 INFO:teuthology.orchestra.run:waiting for 300 2021-02-10T11:51:16.859 INFO:tasks.ceph.mon.c:Stopped 2021-02-10T11:51:16.859 INFO:tasks.ceph:Checking cluster log for badness... 2021-02-10T11:51:16.860 DEBUG:teuthology.orchestra.run.gibba001:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2021-02-10T11:51:17.208 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@gibba001.front.sepia.ceph.com 2021-02-10T11:51:17.209 DEBUG:teuthology.orchestra.run.gibba001:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2021-02-10T11:51:17.774 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@gibba001.front.sepia.ceph.com 2021-02-10T11:51:17.774 DEBUG:teuthology.orchestra.run.gibba001:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2021-02-10T11:51:18.179 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@gibba001.front.sepia.ceph.com 2021-02-10T11:51:18.179 DEBUG:teuthology.orchestra.run.gibba001:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2021-02-10T11:51:18.571 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@gibba001.front.sepia.ceph.com 2021-02-10T11:51:18.572 DEBUG:teuthology.orchestra.run.gibba001:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2021-02-10T11:51:18.981 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@gibba008.front.sepia.ceph.com 2021-02-10T11:51:18.981 DEBUG:teuthology.orchestra.run.gibba008:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2021-02-10T11:51:19.377 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@gibba008.front.sepia.ceph.com 2021-02-10T11:51:19.377 DEBUG:teuthology.orchestra.run.gibba008:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2021-02-10T11:51:19.781 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@gibba008.front.sepia.ceph.com 2021-02-10T11:51:19.781 DEBUG:teuthology.orchestra.run.gibba008:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2021-02-10T11:51:20.190 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@gibba008.front.sepia.ceph.com 2021-02-10T11:51:20.190 DEBUG:teuthology.orchestra.run.gibba008:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2021-02-10T11:51:20.594 INFO:tasks.ceph:Archiving mon data... 2021-02-10T11:51:20.596 DEBUG:teuthology.misc:Transferring archived files from gibba001:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/khiremat-2021-02-10_10:13:45-fs-wip-khiremat-mgr-authid-bp-7-nautilus-distro-basic-gibba/5872719/data/mon.a.tgz 2021-02-10T11:51:20.596 DEBUG:teuthology.orchestra.run.gibba001:> mktemp 2021-02-10T11:51:20.932 INFO:teuthology.orchestra.run.gibba001.stdout:/tmp/tmp.TdhPiciJnY 2021-02-10T11:51:20.933 DEBUG:teuthology.orchestra.run.gibba001:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.TdhPiciJnY 2021-02-10T11:51:21.493 DEBUG:teuthology.orchestra.run.gibba001:> sudo chmod 0666 /tmp/tmp.TdhPiciJnY 2021-02-10T11:51:22.176 DEBUG:teuthology.orchestra.remote:gibba001:/tmp/tmp.TdhPiciJnY is 1014KB 2021-02-10T11:51:22.573 DEBUG:teuthology.orchestra.run.gibba001:> rm -fr /tmp/tmp.TdhPiciJnY 2021-02-10T11:51:22.909 DEBUG:teuthology.misc:Transferring archived files from gibba008:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/khiremat-2021-02-10_10:13:45-fs-wip-khiremat-mgr-authid-bp-7-nautilus-distro-basic-gibba/5872719/data/mon.b.tgz 2021-02-10T11:51:22.909 DEBUG:teuthology.orchestra.run.gibba008:> mktemp 2021-02-10T11:51:23.244 INFO:teuthology.orchestra.run.gibba008.stdout:/tmp/tmp.FhI9v9G4LU 2021-02-10T11:51:23.245 DEBUG:teuthology.orchestra.run.gibba008:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.FhI9v9G4LU 2021-02-10T11:51:23.813 DEBUG:teuthology.orchestra.run.gibba008:> sudo chmod 0666 /tmp/tmp.FhI9v9G4LU 2021-02-10T11:51:24.491 DEBUG:teuthology.orchestra.remote:gibba008:/tmp/tmp.FhI9v9G4LU is 1MB 2021-02-10T11:51:24.890 DEBUG:teuthology.orchestra.run.gibba008:> rm -fr /tmp/tmp.FhI9v9G4LU 2021-02-10T11:51:25.223 DEBUG:teuthology.misc:Transferring archived files from gibba008:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/khiremat-2021-02-10_10:13:45-fs-wip-khiremat-mgr-authid-bp-7-nautilus-distro-basic-gibba/5872719/data/mon.c.tgz 2021-02-10T11:51:25.224 DEBUG:teuthology.orchestra.run.gibba008:> mktemp 2021-02-10T11:51:25.557 INFO:teuthology.orchestra.run.gibba008.stdout:/tmp/tmp.u5qmgtnons 2021-02-10T11:51:25.558 DEBUG:teuthology.orchestra.run.gibba008:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.u5qmgtnons 2021-02-10T11:51:26.114 DEBUG:teuthology.orchestra.run.gibba008:> sudo chmod 0666 /tmp/tmp.u5qmgtnons 2021-02-10T11:51:26.792 DEBUG:teuthology.orchestra.remote:gibba008:/tmp/tmp.u5qmgtnons is 1MB 2021-02-10T11:51:27.189 DEBUG:teuthology.orchestra.run.gibba008:> rm -fr /tmp/tmp.u5qmgtnons 2021-02-10T11:51:27.523 INFO:tasks.ceph:Cleaning ceph cluster... 2021-02-10T11:51:27.524 DEBUG:teuthology.orchestra.run.gibba001:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2021-02-10T11:51:27.528 DEBUG:teuthology.orchestra.run.gibba008:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2021-02-10T11:51:27.871 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2021-02-10T11:51:27.871 DEBUG:teuthology.orchestra.run.gibba001:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2021-02-10T11:51:27.874 DEBUG:teuthology.orchestra.run.gibba008:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2021-02-10T11:51:28.216 INFO:teuthology.orchestra.run.gibba001.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2021-02-10T11:51:28.218 INFO:teuthology.orchestra.run.gibba008.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2021-02-10T11:51:28.223 INFO:tasks.ceph:Archiving crash dumps... 2021-02-10T11:51:28.225 DEBUG:teuthology.misc:Transferring archived files from gibba001:/var/lib/ceph/crash to /home/teuthworker/archive/khiremat-2021-02-10_10:13:45-fs-wip-khiremat-mgr-authid-bp-7-nautilus-distro-basic-gibba/5872719/remote/gibba001/crash 2021-02-10T11:51:28.226 DEBUG:teuthology.orchestra.run.gibba001:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2021-02-10T11:51:28.570 DEBUG:teuthology.misc:Transferring archived files from gibba008:/var/lib/ceph/crash to /home/teuthworker/archive/khiremat-2021-02-10_10:13:45-fs-wip-khiremat-mgr-authid-bp-7-nautilus-distro-basic-gibba/5872719/remote/gibba008/crash 2021-02-10T11:51:28.571 DEBUG:teuthology.orchestra.run.gibba008:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2021-02-10T11:51:28.915 INFO:tasks.ceph:Shutting down logrotate 2021-02-10T11:51:28.916 DEBUG:teuthology.orchestra.run.gibba001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:51:28.919 DEBUG:teuthology.orchestra.run.gibba008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-02-10T11:51:29.316 DEBUG:teuthology.orchestra.run.gibba001:> sudo rm /etc/logrotate.d/ceph-test.conf 2021-02-10T11:51:29.661 DEBUG:teuthology.orchestra.run.gibba008:> sudo rm /etc/logrotate.d/ceph-test.conf 2021-02-10T11:51:30.005 INFO:tasks.ceph:Compressing logs... 2021-02-10T11:51:30.005 DEBUG:teuthology.orchestra.run.gibba001:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-02-10T11:51:30.010 DEBUG:teuthology.orchestra.run.gibba008:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-02-10T11:51:49.694 INFO:tasks.ceph:Archiving logs... 2021-02-10T11:51:49.694 DEBUG:teuthology.misc:Transferring archived files from gibba001:/var/log/ceph to /home/teuthworker/archive/khiremat-2021-02-10_10:13:45-fs-wip-khiremat-mgr-authid-bp-7-nautilus-distro-basic-gibba/5872719/remote/gibba001/log 2021-02-10T11:51:49.695 DEBUG:teuthology.orchestra.run.gibba001:> sudo tar cz -f - -C /var/log/ceph -- . 2021-02-10T11:51:54.008 DEBUG:teuthology.misc:Transferring archived files from gibba008:/var/log/ceph to /home/teuthworker/archive/khiremat-2021-02-10_10:13:45-fs-wip-khiremat-mgr-authid-bp-7-nautilus-distro-basic-gibba/5872719/remote/gibba008/log 2021-02-10T11:51:54.008 DEBUG:teuthology.orchestra.run.gibba008:> sudo tar cz -f - -C /var/log/ceph -- . 2021-02-10T11:51:57.667 DEBUG:teuthology.run_tasks:Unwinding manager install 2021-02-10T11:51:57.678 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits... 2021-02-10T11:51:57.678 DEBUG:teuthology.orchestra.run.gibba001:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2021-02-10T11:51:57.682 DEBUG:teuthology.orchestra.run.gibba008:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2021-02-10T11:51:58.082 DEBUG:teuthology.run_tasks:Unwinding manager clock 2021-02-10T11:51:58.092 INFO:teuthology.task.clock:Checking final clock skew... 2021-02-10T11:51:58.093 DEBUG:teuthology.orchestra.run.gibba001:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-02-10T11:51:58.438 INFO:teuthology.orchestra.run.gibba001.stdout: remote refid st t when poll reach delay offset jitter 2021-02-10T11:51:58.439 INFO:teuthology.orchestra.run.gibba001.stdout:============================================================================== 2021-02-10T11:51:58.439 INFO:teuthology.orchestra.run.gibba001.stdout:+hv01.front.sepi 216.240.36.24 3 u 115 128 377 0.099 -1.135 2.997 2021-02-10T11:51:58.439 INFO:teuthology.orchestra.run.gibba001.stdout:*hv02.front.sepi 163.237.218.18 2 u 32 128 377 0.102 0.786 4.552 2021-02-10T11:51:58.439 INFO:teuthology.orchestra.run.gibba001.stdout:+hv03.front.sepi 74.6.168.72 3 u 102 128 377 0.044 -1.683 2.027 2021-02-10T11:51:58.439 INFO:teuthology.orchestra.run.gibba001.stdout:+hv04.front.sepi 144.172.126.201 3 u 23 128 377 0.063 -1.080 2.787 2021-02-10T11:51:58.440 DEBUG:teuthology.orchestra.run.gibba008:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-02-10T11:51:58.785 INFO:teuthology.orchestra.run.gibba008.stdout: remote refid st t when poll reach delay offset jitter 2021-02-10T11:51:58.785 INFO:teuthology.orchestra.run.gibba008.stdout:============================================================================== 2021-02-10T11:51:58.785 INFO:teuthology.orchestra.run.gibba008.stdout:-hv01.front.sepi 216.240.36.24 3 u 30 128 377 0.098 3.479 4.565 2021-02-10T11:51:58.785 INFO:teuthology.orchestra.run.gibba008.stdout:*hv02.front.sepi 163.237.218.18 2 u 89 128 377 0.079 4.416 6.559 2021-02-10T11:51:58.785 INFO:teuthology.orchestra.run.gibba008.stdout:+hv03.front.sepi 74.6.168.72 3 u 32 128 377 0.051 4.536 4.507 2021-02-10T11:51:58.786 INFO:teuthology.orchestra.run.gibba008.stdout:+hv04.front.sepi 144.172.126.201 3 u 85 128 377 0.068 -0.640 5.606 2021-02-10T11:51:58.786 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2021-02-10T11:51:58.797 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2021-02-10T11:51:58.797 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2021-02-10T11:51:58.842 DEBUG:teuthology.orchestra.run.gibba001:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2021-02-10T11:51:59.210 DEBUG:teuthology.orchestra.run.gibba008:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2021-02-10T11:51:59.582 DEBUG:teuthology.orchestra.run.gibba001:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\)' 2021-02-10T11:51:59.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:51:59.928 DEBUG:teuthology.orchestra.run.gibba008:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\)' 2021-02-10T11:52:00.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:52:00.272 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2021-02-10T11:52:00.283 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2021-02-10T11:52:00.324 INFO:teuthology.task.internal:Duration was 3024.668109 seconds 2021-02-10T11:52:00.325 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2021-02-10T11:52:00.339 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2021-02-10T11:52:00.339 DEBUG:teuthology.orchestra.run.gibba001:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-02-10T11:52:00.343 DEBUG:teuthology.orchestra.run.gibba008:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-02-10T11:52:00.701 INFO:teuthology.orchestra.run.gibba001.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2021-02-10T11:52:00.702 INFO:teuthology.orchestra.run.gibba008.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2021-02-10T11:52:00.719 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2021-02-10T11:52:00.719 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@gibba001.front.sepia.ceph.com 2021-02-10T11:52:00.719 DEBUG:teuthology.orchestra.run.gibba001:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/*.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2021-02-10T11:52:01.067 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@gibba008.front.sepia.ceph.com 2021-02-10T11:52:01.067 DEBUG:teuthology.orchestra.run.gibba008:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/*.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2021-02-10T11:52:01.411 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2021-02-10T11:52:01.411 DEBUG:teuthology.orchestra.run.gibba001:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-02-10T11:52:01.415 DEBUG:teuthology.orchestra.run.gibba008:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-02-10T11:52:01.777 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2021-02-10T11:52:01.787 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2021-02-10T11:52:01.788 DEBUG:teuthology.orchestra.run.gibba001:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2021-02-10T11:52:02.133 DEBUG:teuthology.orchestra.run.gibba008:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2021-02-10T11:52:02.478 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2021-02-10T11:52:02.489 DEBUG:teuthology.orchestra.run.gibba001:> sudo sysctl -w kernel.core_pattern=core && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2021-02-10T11:52:02.493 DEBUG:teuthology.orchestra.run.gibba008:> sudo sysctl -w kernel.core_pattern=core && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2021-02-10T11:52:02.833 INFO:teuthology.orchestra.run.gibba001.stdout:kernel.core_pattern = core 2021-02-10T11:52:02.835 INFO:teuthology.orchestra.run.gibba008.stdout:kernel.core_pattern = core 2021-02-10T11:52:02.837 DEBUG:teuthology.orchestra.run.gibba001:> test -e /home/ubuntu/cephtest/archive/coredump 2021-02-10T11:52:03.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:52:03.172 DEBUG:teuthology.orchestra.run.gibba008:> test -e /home/ubuntu/cephtest/archive/coredump 2021-02-10T11:52:03.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-10T11:52:03.506 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2021-02-10T11:52:03.517 INFO:teuthology.task.internal:Transferring archived files... 2021-02-10T11:52:03.518 DEBUG:teuthology.misc:Transferring archived files from gibba001:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/khiremat-2021-02-10_10:13:45-fs-wip-khiremat-mgr-authid-bp-7-nautilus-distro-basic-gibba/5872719/remote/gibba001 2021-02-10T11:52:03.518 DEBUG:teuthology.orchestra.run.gibba001:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2021-02-10T11:52:03.878 DEBUG:teuthology.misc:Transferring archived files from gibba008:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/khiremat-2021-02-10_10:13:45-fs-wip-khiremat-mgr-authid-bp-7-nautilus-distro-basic-gibba/5872719/remote/gibba008 2021-02-10T11:52:03.878 DEBUG:teuthology.orchestra.run.gibba008:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2021-02-10T11:52:04.231 INFO:teuthology.task.internal:Removing archive directory... 2021-02-10T11:52:04.231 DEBUG:teuthology.orchestra.run.gibba001:> rm -rf -- /home/ubuntu/cephtest/archive 2021-02-10T11:52:04.235 DEBUG:teuthology.orchestra.run.gibba008:> rm -rf -- /home/ubuntu/cephtest/archive 2021-02-10T11:52:04.569 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2021-02-10T11:52:04.581 INFO:teuthology.task.internal:Not uploading archives. 2021-02-10T11:52:04.581 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2021-02-10T11:52:04.598 INFO:teuthology.task.internal:Tidying up after the test... 2021-02-10T11:52:04.598 DEBUG:teuthology.orchestra.run.gibba001:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2021-02-10T11:52:04.602 DEBUG:teuthology.orchestra.run.gibba008:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2021-02-10T11:52:04.933 INFO:teuthology.orchestra.run.gibba001.stdout:393221 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 10 11:52 /home/ubuntu/cephtest 2021-02-10T11:52:04.936 INFO:teuthology.orchestra.run.gibba008.stdout:393217 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 10 11:52 /home/ubuntu/cephtest 2021-02-10T11:52:04.937 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2021-02-10T11:52:05.139 INFO:teuthology.nuke:Checking targets against current locks 2021-02-10T11:52:05.221 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/virtualenv/bin/teuthology", line 33, in sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology')()) File "/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/scripts/run.py", line 38, in main teuthology.run.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/teuthology/run.py", line 412, in main report_outcome(config, archive, fake_ctx.summary, fake_ctx) File "/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/teuthology/run.py", line 269, in report_outcome nuke(fake_ctx, fake_ctx.lock) File "/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/teuthology/nuke/__init__.py", line 266, in nuke for unnuked in p: File "/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/teuthology/nuke/__init__.py", line 280, in nuke_one keep_logs = ctx.keep_logs File "/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/teuthology/config.py", line 255, in __getattr__ raise AttributeError(name) AttributeError: keep_logs